[2023-12-27 05:39:56,413][19170] Saving configuration to ./train_mujoco/mujoco_pendulum_APPO/config.json... [2023-12-27 05:39:57,416][19170] Rollout worker 0 uses device cpu [2023-12-27 05:39:57,417][19170] Rollout worker 1 uses device cpu [2023-12-27 05:39:57,418][19170] Rollout worker 2 uses device cpu [2023-12-27 05:39:57,418][19170] Rollout worker 3 uses device cpu [2023-12-27 05:39:57,419][19170] Rollout worker 4 uses device cpu [2023-12-27 05:39:57,419][19170] Rollout worker 5 uses device cpu [2023-12-27 05:39:57,419][19170] Rollout worker 6 uses device cpu [2023-12-27 05:39:57,420][19170] Rollout worker 7 uses device cpu [2023-12-27 05:39:57,420][19170] Rollout worker 8 uses device cpu [2023-12-27 05:39:57,421][19170] Rollout worker 9 uses device cpu [2023-12-27 05:39:57,421][19170] Rollout worker 10 uses device cpu [2023-12-27 05:39:57,422][19170] Rollout worker 11 uses device cpu [2023-12-27 05:39:57,422][19170] Rollout worker 12 uses device cpu [2023-12-27 05:39:57,423][19170] Rollout worker 13 uses device cpu [2023-12-27 05:39:57,423][19170] Rollout worker 14 uses device cpu [2023-12-27 05:39:57,423][19170] Rollout worker 15 uses device cpu [2023-12-27 05:39:57,450][19170] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-27 05:39:57,451][19170] InferenceWorker_p0-w0: min num requests: 2 [2023-12-27 05:39:57,454][19170] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-27 05:39:57,455][19170] InferenceWorker_p1-w0: min num requests: 2 [2023-12-27 05:39:57,504][19170] Starting all processes... [2023-12-27 05:39:57,504][19170] Starting process learner_proc0 [2023-12-27 05:39:57,507][19170] Starting process learner_proc1 [2023-12-27 05:39:57,554][19170] Starting all processes... [2023-12-27 05:39:57,562][19170] Starting process inference_proc0-0 [2023-12-27 05:39:57,562][19170] Starting process inference_proc1-0 [2023-12-27 05:39:57,563][19170] Starting process rollout_proc0 [2023-12-27 05:39:57,563][19170] Starting process rollout_proc1 [2023-12-27 05:39:57,563][19170] Starting process rollout_proc2 [2023-12-27 05:39:57,564][19170] Starting process rollout_proc3 [2023-12-27 05:39:57,569][19170] Starting process rollout_proc4 [2023-12-27 05:39:57,569][19170] Starting process rollout_proc5 [2023-12-27 05:39:57,570][19170] Starting process rollout_proc6 [2023-12-27 05:39:57,574][19170] Starting process rollout_proc7 [2023-12-27 05:39:57,579][19170] Starting process rollout_proc8 [2023-12-27 05:39:57,581][19170] Starting process rollout_proc9 [2023-12-27 05:39:57,581][19170] Starting process rollout_proc10 [2023-12-27 05:39:57,581][19170] Starting process rollout_proc11 [2023-12-27 05:39:57,582][19170] Starting process rollout_proc12 [2023-12-27 05:39:57,583][19170] Starting process rollout_proc13 [2023-12-27 05:39:57,584][19170] Starting process rollout_proc14 [2023-12-27 05:39:57,611][19170] Starting process rollout_proc15 [2023-12-27 05:40:00,442][20381] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-27 05:40:00,443][20381] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-12-27 05:40:00,482][20381] Num visible devices: 1 [2023-12-27 05:40:00,490][20384] Worker 3 uses CPU cores [6, 7] [2023-12-27 05:40:00,514][20395] Worker 7 uses CPU cores [14, 15] [2023-12-27 05:40:00,522][20393] Worker 10 uses CPU cores [20, 21] [2023-12-27 05:40:00,525][20383] Worker 1 uses CPU cores [2, 3] [2023-12-27 05:40:00,577][20433] Worker 13 uses CPU cores [26, 27] [2023-12-27 05:40:00,642][20291] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-27 05:40:00,642][20291] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-12-27 05:40:00,683][20291] Num visible devices: 1 [2023-12-27 05:40:00,707][20291] Setting fixed seed 1234 [2023-12-27 05:40:00,709][20291] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-27 05:40:00,709][20291] Initializing actor-critic model on device cuda:0 [2023-12-27 05:40:00,710][20291] RunningMeanStd input shape: (4,) [2023-12-27 05:40:00,710][20291] RunningMeanStd input shape: (1,) [2023-12-27 05:40:00,724][20431] Worker 9 uses CPU cores [18, 19] [2023-12-27 05:40:00,751][20291] Created Actor Critic model with architecture: [2023-12-27 05:40:00,752][20291] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=1, bias=True) ) ) [2023-12-27 05:40:00,758][20436] Worker 14 uses CPU cores [28, 29] [2023-12-27 05:40:00,821][20292] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-12-27 05:40:00,822][20292] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-12-27 05:40:00,866][20386] Worker 2 uses CPU cores [4, 5] [2023-12-27 05:40:00,870][20385] Worker 4 uses CPU cores [8, 9] [2023-12-27 05:40:00,870][20292] Num visible devices: 1 [2023-12-27 05:40:00,879][20382] Worker 0 uses CPU cores [0, 1] [2023-12-27 05:40:00,899][20292] Setting fixed seed 1234 [2023-12-27 05:40:00,900][20292] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-12-27 05:40:00,901][20292] Initializing actor-critic model on device cuda:0 [2023-12-27 05:40:00,901][20292] RunningMeanStd input shape: (4,) [2023-12-27 05:40:00,902][20292] RunningMeanStd input shape: (1,) [2023-12-27 05:40:00,915][20435] Worker 11 uses CPU cores [22, 23] [2023-12-27 05:40:00,928][20389] Worker 5 uses CPU cores [10, 11] [2023-12-27 05:40:00,936][20292] Created Actor Critic model with architecture: [2023-12-27 05:40:00,936][20292] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): MlpEncoder( (mlp_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=Tanh) (2): RecursiveScriptModule(original_name=Linear) (3): RecursiveScriptModule(original_name=Tanh) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=64, out_features=1, bias=True) (action_parameterization): ActionParameterizationContinuousNonAdaptiveStddev( (distribution_linear): Linear(in_features=64, out_features=1, bias=True) ) ) [2023-12-27 05:40:00,973][20392] Worker 8 uses CPU cores [16, 17] [2023-12-27 05:40:00,974][20391] Worker 6 uses CPU cores [12, 13] [2023-12-27 05:40:00,986][20437] Worker 15 uses CPU cores [30, 31] [2023-12-27 05:40:01,006][20347] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-27 05:40:01,006][20347] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-12-27 05:40:01,049][20434] Worker 12 uses CPU cores [24, 25] [2023-12-27 05:40:01,051][20347] Num visible devices: 1 [2023-12-27 05:40:01,389][20291] Using optimizer [2023-12-27 05:40:01,390][20291] No checkpoints found [2023-12-27 05:40:01,390][20291] Did not load from checkpoint, starting from scratch! [2023-12-27 05:40:01,390][20291] Initialized policy 0 weights for model version 0 [2023-12-27 05:40:01,392][20291] LearnerWorker_p0 finished initialization! [2023-12-27 05:40:01,392][20291] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-12-27 05:40:01,588][20292] Using optimizer [2023-12-27 05:40:01,589][20292] No checkpoints found [2023-12-27 05:40:01,589][20292] Did not load from checkpoint, starting from scratch! [2023-12-27 05:40:01,590][20292] Initialized policy 1 weights for model version 0 [2023-12-27 05:40:01,591][20292] LearnerWorker_p1 finished initialization! [2023-12-27 05:40:01,592][20292] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-12-27 05:40:01,987][20347] RunningMeanStd input shape: (4,) [2023-12-27 05:40:01,988][20347] RunningMeanStd input shape: (1,) [2023-12-27 05:40:02,023][19170] Inference worker 0-0 is ready! [2023-12-27 05:40:02,170][20381] RunningMeanStd input shape: (4,) [2023-12-27 05:40:02,171][20381] RunningMeanStd input shape: (1,) [2023-12-27 05:40:02,204][19170] Inference worker 1-0 is ready! [2023-12-27 05:40:02,205][19170] All inference workers are ready! Signal rollout workers to start! [2023-12-27 05:40:02,206][20389] EnvRunner 5-0 uses policy 1 [2023-12-27 05:40:02,206][20385] EnvRunner 4-0 uses policy 0 [2023-12-27 05:40:02,206][20391] EnvRunner 6-0 uses policy 0 [2023-12-27 05:40:02,206][20433] EnvRunner 13-0 uses policy 1 [2023-12-27 05:40:02,206][20383] EnvRunner 1-0 uses policy 1 [2023-12-27 05:40:02,207][20436] EnvRunner 14-0 uses policy 0 [2023-12-27 05:40:02,207][20382] EnvRunner 0-0 uses policy 0 [2023-12-27 05:40:02,207][20435] EnvRunner 11-0 uses policy 1 [2023-12-27 05:40:02,207][20434] EnvRunner 12-0 uses policy 0 [2023-12-27 05:40:02,207][20392] EnvRunner 8-0 uses policy 0 [2023-12-27 05:40:02,207][20395] EnvRunner 7-0 uses policy 1 [2023-12-27 05:40:02,207][20431] EnvRunner 9-0 uses policy 1 [2023-12-27 05:40:02,207][20393] EnvRunner 10-0 uses policy 0 [2023-12-27 05:40:02,207][20384] EnvRunner 3-0 uses policy 1 [2023-12-27 05:40:02,207][20437] EnvRunner 15-0 uses policy 1 [2023-12-27 05:40:02,207][20386] EnvRunner 2-0 uses policy 0 [2023-12-27 05:40:02,303][20392] EnvRunner 8-1 uses policy 0 [2023-12-27 05:40:02,303][20389] EnvRunner 5-1 uses policy 1 [2023-12-27 05:40:02,305][20382] EnvRunner 0-1 uses policy 0 [2023-12-27 05:40:02,311][20433] EnvRunner 13-1 uses policy 1 [2023-12-27 05:40:02,313][20393] EnvRunner 10-1 uses policy 0 [2023-12-27 05:40:02,317][20436] EnvRunner 14-1 uses policy 0 [2023-12-27 05:40:02,323][20391] EnvRunner 6-1 uses policy 0 [2023-12-27 05:40:02,324][20386] EnvRunner 2-1 uses policy 0 [2023-12-27 05:40:02,331][20384] EnvRunner 3-1 uses policy 1 [2023-12-27 05:40:02,334][20385] EnvRunner 4-1 uses policy 0 [2023-12-27 05:40:02,334][20395] EnvRunner 7-1 uses policy 1 [2023-12-27 05:40:02,335][20434] EnvRunner 12-1 uses policy 0 [2023-12-27 05:40:02,346][20437] EnvRunner 15-1 uses policy 1 [2023-12-27 05:40:02,347][20435] EnvRunner 11-1 uses policy 1 [2023-12-27 05:40:02,347][20431] EnvRunner 9-1 uses policy 1 [2023-12-27 05:40:02,347][20383] EnvRunner 1-1 uses policy 1 [2023-12-27 05:40:02,362][19170] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-12-27 05:40:04,603][20291] Signal inference workers to stop experience collection... [2023-12-27 05:40:04,607][20381] InferenceWorker_p1-w0: stopping experience collection [2023-12-27 05:40:04,610][20347] InferenceWorker_p0-w0: stopping experience collection [2023-12-27 05:40:05,123][20291] Signal inference workers to resume experience collection... [2023-12-27 05:40:05,124][20347] InferenceWorker_p0-w0: resuming experience collection [2023-12-27 05:40:05,124][20381] InferenceWorker_p1-w0: resuming experience collection [2023-12-27 05:40:05,995][20292] Signal inference workers to stop experience collection... [2023-12-27 05:40:06,339][20292] Signal inference workers to resume experience collection... [2023-12-27 05:40:06,883][20381] Updated weights for policy 1, policy_version 67 (0.0009) [2023-12-27 05:40:06,936][20381] Updated weights for policy 1, policy_version 77 (0.0009) [2023-12-27 05:40:06,990][20381] Updated weights for policy 1, policy_version 87 (0.0008) [2023-12-27 05:40:07,087][20347] Updated weights for policy 0, policy_version 72 (0.0325) [2023-12-27 05:40:07,143][20347] Updated weights for policy 0, policy_version 82 (0.0005) [2023-12-27 05:40:07,195][20347] Updated weights for policy 0, policy_version 92 (0.0009) [2023-12-27 05:40:07,284][20381] Updated weights for policy 1, policy_version 97 (0.0007) [2023-12-27 05:40:07,289][20347] Updated weights for policy 0, policy_version 102 (0.0008) [2023-12-27 05:40:07,340][20381] Updated weights for policy 1, policy_version 107 (0.0005) [2023-12-27 05:40:07,345][20347] Updated weights for policy 0, policy_version 112 (0.0010) [2023-12-27 05:40:07,363][19170] Fps is (10 sec: 9829.9, 60 sec: 9829.9, 300 sec: 9829.9). Total num frames: 49152. Throughput: 0: 3348.6, 1: 3264.6. Samples: 33068. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 05:40:07,364][19170] Avg episode reward: [(0, '14.100'), (1, '12.310')] [2023-12-27 05:40:07,396][20347] Updated weights for policy 0, policy_version 122 (0.0010) [2023-12-27 05:40:07,401][20381] Updated weights for policy 1, policy_version 117 (0.0006) [2023-12-27 05:40:07,425][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000000128_32768.pth... [2023-12-27 05:40:07,437][19170] Heartbeat connected on Batcher_0 [2023-12-27 05:40:07,440][19170] Heartbeat connected on LearnerWorker_p0 [2023-12-27 05:40:07,444][19170] Heartbeat connected on Batcher_1 [2023-12-27 05:40:07,455][19170] Heartbeat connected on InferenceWorker_p0-w0 [2023-12-27 05:40:07,459][19170] Heartbeat connected on RolloutWorker_w0 [2023-12-27 05:40:07,460][19170] Heartbeat connected on InferenceWorker_p1-w0 [2023-12-27 05:40:07,462][19170] Heartbeat connected on RolloutWorker_w1 [2023-12-27 05:40:07,464][19170] Heartbeat connected on RolloutWorker_w2 [2023-12-27 05:40:07,467][19170] Heartbeat connected on RolloutWorker_w3 [2023-12-27 05:40:07,470][20381] Updated weights for policy 1, policy_version 127 (0.0008) [2023-12-27 05:40:07,471][19170] Heartbeat connected on RolloutWorker_w4 [2023-12-27 05:40:07,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000000128_32768.pth... [2023-12-27 05:40:07,477][19170] Heartbeat connected on RolloutWorker_w5 [2023-12-27 05:40:07,478][19170] Heartbeat connected on RolloutWorker_w6 [2023-12-27 05:40:07,479][19170] Heartbeat connected on LearnerWorker_p1 [2023-12-27 05:40:07,480][19170] Heartbeat connected on RolloutWorker_w7 [2023-12-27 05:40:07,483][19170] Heartbeat connected on RolloutWorker_w8 [2023-12-27 05:40:07,484][19170] Heartbeat connected on RolloutWorker_w9 [2023-12-27 05:40:07,488][19170] Heartbeat connected on RolloutWorker_w10 [2023-12-27 05:40:07,490][19170] Heartbeat connected on RolloutWorker_w11 [2023-12-27 05:40:07,493][19170] Heartbeat connected on RolloutWorker_w12 [2023-12-27 05:40:07,496][19170] Heartbeat connected on RolloutWorker_w13 [2023-12-27 05:40:07,499][19170] Heartbeat connected on RolloutWorker_w14 [2023-12-27 05:40:07,502][19170] Heartbeat connected on RolloutWorker_w15 [2023-12-27 05:40:07,973][20347] Updated weights for policy 0, policy_version 132 (0.0010) [2023-12-27 05:40:08,034][20347] Updated weights for policy 0, policy_version 142 (0.0010) [2023-12-27 05:40:08,089][20347] Updated weights for policy 0, policy_version 152 (0.0010) [2023-12-27 05:40:08,221][20381] Updated weights for policy 1, policy_version 137 (0.0005) [2023-12-27 05:40:08,280][20381] Updated weights for policy 1, policy_version 147 (0.0005) [2023-12-27 05:40:08,341][20381] Updated weights for policy 1, policy_version 157 (0.0007) [2023-12-27 05:40:08,751][20347] Updated weights for policy 0, policy_version 162 (0.0009) [2023-12-27 05:40:08,810][20347] Updated weights for policy 0, policy_version 172 (0.0005) [2023-12-27 05:40:08,873][20347] Updated weights for policy 0, policy_version 182 (0.0007) [2023-12-27 05:40:08,939][20347] Updated weights for policy 0, policy_version 192 (0.0008) [2023-12-27 05:40:09,036][20381] Updated weights for policy 1, policy_version 167 (0.0008) [2023-12-27 05:40:09,084][20381] Updated weights for policy 1, policy_version 177 (0.0007) [2023-12-27 05:40:09,131][20381] Updated weights for policy 1, policy_version 187 (0.0007) [2023-12-27 05:40:09,619][20347] Updated weights for policy 0, policy_version 202 (0.0008) [2023-12-27 05:40:09,671][20347] Updated weights for policy 0, policy_version 212 (0.0008) [2023-12-27 05:40:09,724][20347] Updated weights for policy 0, policy_version 222 (0.0009) [2023-12-27 05:40:09,851][20381] Updated weights for policy 1, policy_version 197 (0.0008) [2023-12-27 05:40:09,915][20381] Updated weights for policy 1, policy_version 207 (0.0007) [2023-12-27 05:40:09,975][20381] Updated weights for policy 1, policy_version 217 (0.0009) [2023-12-27 05:40:10,497][20347] Updated weights for policy 0, policy_version 232 (0.0008) [2023-12-27 05:40:10,551][20347] Updated weights for policy 0, policy_version 242 (0.0007) [2023-12-27 05:40:10,608][20347] Updated weights for policy 0, policy_version 252 (0.0007) [2023-12-27 05:40:10,666][20381] Updated weights for policy 1, policy_version 227 (0.0007) [2023-12-27 05:40:10,725][20381] Updated weights for policy 1, policy_version 237 (0.0007) [2023-12-27 05:40:10,779][20381] Updated weights for policy 1, policy_version 247 (0.0008) [2023-12-27 05:40:11,308][20347] Updated weights for policy 0, policy_version 262 (0.0007) [2023-12-27 05:40:11,371][20347] Updated weights for policy 0, policy_version 272 (0.0008) [2023-12-27 05:40:11,440][20347] Updated weights for policy 0, policy_version 282 (0.0007) [2023-12-27 05:40:11,531][20381] Updated weights for policy 1, policy_version 257 (0.0008) [2023-12-27 05:40:11,586][20381] Updated weights for policy 1, policy_version 267 (0.0007) [2023-12-27 05:40:11,652][20381] Updated weights for policy 1, policy_version 277 (0.0008) [2023-12-27 05:40:11,707][20381] Updated weights for policy 1, policy_version 287 (0.0007) [2023-12-27 05:40:12,171][20347] Updated weights for policy 0, policy_version 292 (0.0007) [2023-12-27 05:40:12,228][20347] Updated weights for policy 0, policy_version 302 (0.0007) [2023-12-27 05:40:12,287][20347] Updated weights for policy 0, policy_version 312 (0.0008) [2023-12-27 05:40:12,362][19170] Fps is (10 sec: 15564.8, 60 sec: 15564.8, 300 sec: 15564.8). Total num frames: 155648. Throughput: 0: 7216.0, 1: 7130.8. Samples: 143468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:40:12,363][19170] Avg episode reward: [(0, '62.150'), (1, '58.620')] [2023-12-27 05:40:12,465][20381] Updated weights for policy 1, policy_version 297 (0.0011) [2023-12-27 05:40:12,524][20381] Updated weights for policy 1, policy_version 307 (0.0008) [2023-12-27 05:40:12,584][20381] Updated weights for policy 1, policy_version 317 (0.0007) [2023-12-27 05:40:13,010][20347] Updated weights for policy 0, policy_version 322 (0.0007) [2023-12-27 05:40:13,063][20347] Updated weights for policy 0, policy_version 332 (0.0007) [2023-12-27 05:40:13,109][20347] Updated weights for policy 0, policy_version 342 (0.0007) [2023-12-27 05:40:13,158][20347] Updated weights for policy 0, policy_version 352 (0.0007) [2023-12-27 05:40:13,247][20381] Updated weights for policy 1, policy_version 327 (0.0010) [2023-12-27 05:40:13,300][20381] Updated weights for policy 1, policy_version 337 (0.0006) [2023-12-27 05:40:13,353][20381] Updated weights for policy 1, policy_version 347 (0.0006) [2023-12-27 05:40:13,908][20347] Updated weights for policy 0, policy_version 362 (0.0007) [2023-12-27 05:40:13,960][20347] Updated weights for policy 0, policy_version 372 (0.0007) [2023-12-27 05:40:14,007][20381] Updated weights for policy 1, policy_version 357 (0.0011) [2023-12-27 05:40:14,013][20347] Updated weights for policy 0, policy_version 382 (0.0006) [2023-12-27 05:40:14,062][20381] Updated weights for policy 1, policy_version 367 (0.0011) [2023-12-27 05:40:14,117][20381] Updated weights for policy 1, policy_version 377 (0.0010) [2023-12-27 05:40:14,734][20347] Updated weights for policy 0, policy_version 392 (0.0007) [2023-12-27 05:40:14,775][20381] Updated weights for policy 1, policy_version 387 (0.0010) [2023-12-27 05:40:14,795][20347] Updated weights for policy 0, policy_version 402 (0.0009) [2023-12-27 05:40:14,836][20381] Updated weights for policy 1, policy_version 397 (0.0011) [2023-12-27 05:40:14,854][20347] Updated weights for policy 0, policy_version 412 (0.0006) [2023-12-27 05:40:14,900][20381] Updated weights for policy 1, policy_version 407 (0.0009) [2023-12-27 05:40:15,566][20381] Updated weights for policy 1, policy_version 417 (0.0008) [2023-12-27 05:40:15,601][20347] Updated weights for policy 0, policy_version 422 (0.0008) [2023-12-27 05:40:15,622][20381] Updated weights for policy 1, policy_version 427 (0.0011) [2023-12-27 05:40:15,654][20347] Updated weights for policy 0, policy_version 432 (0.0010) [2023-12-27 05:40:15,670][20381] Updated weights for policy 1, policy_version 437 (0.0011) [2023-12-27 05:40:15,703][20347] Updated weights for policy 0, policy_version 442 (0.0006) [2023-12-27 05:40:15,714][20381] Updated weights for policy 1, policy_version 447 (0.0006) [2023-12-27 05:40:16,340][20347] Updated weights for policy 0, policy_version 452 (0.0008) [2023-12-27 05:40:16,391][20347] Updated weights for policy 0, policy_version 462 (0.0010) [2023-12-27 05:40:16,406][20381] Updated weights for policy 1, policy_version 457 (0.0010) [2023-12-27 05:40:16,437][20347] Updated weights for policy 0, policy_version 472 (0.0010) [2023-12-27 05:40:16,462][20381] Updated weights for policy 1, policy_version 467 (0.0007) [2023-12-27 05:40:16,517][20381] Updated weights for policy 1, policy_version 477 (0.0006) [2023-12-27 05:40:17,121][20381] Updated weights for policy 1, policy_version 487 (0.0005) [2023-12-27 05:40:17,158][20347] Updated weights for policy 0, policy_version 482 (0.0010) [2023-12-27 05:40:17,175][20381] Updated weights for policy 1, policy_version 497 (0.0010) [2023-12-27 05:40:17,213][20347] Updated weights for policy 0, policy_version 492 (0.0010) [2023-12-27 05:40:17,229][20381] Updated weights for policy 1, policy_version 507 (0.0007) [2023-12-27 05:40:17,272][20347] Updated weights for policy 0, policy_version 502 (0.0010) [2023-12-27 05:40:17,335][20347] Updated weights for policy 0, policy_version 512 (0.0010) [2023-12-27 05:40:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 17476.3, 300 sec: 17476.3). Total num frames: 262144. Throughput: 0: 8745.9, 1: 8893.6. Samples: 264592. Policy #0 lag: (min: 26.0, avg: 36.4, max: 58.0) [2023-12-27 05:40:17,363][19170] Avg episode reward: [(0, '146.390'), (1, '155.050')] [2023-12-27 05:40:17,364][20291] Saving new best policy, reward=146.390! [2023-12-27 05:40:17,364][20292] Saving new best policy, reward=155.050! [2023-12-27 05:40:17,838][20381] Updated weights for policy 1, policy_version 517 (0.0011) [2023-12-27 05:40:17,891][20381] Updated weights for policy 1, policy_version 527 (0.0010) [2023-12-27 05:40:17,942][20381] Updated weights for policy 1, policy_version 537 (0.0010) [2023-12-27 05:40:17,950][20347] Updated weights for policy 0, policy_version 522 (0.0008) [2023-12-27 05:40:18,006][20347] Updated weights for policy 0, policy_version 532 (0.0006) [2023-12-27 05:40:18,057][20347] Updated weights for policy 0, policy_version 542 (0.0007) [2023-12-27 05:40:18,627][20381] Updated weights for policy 1, policy_version 547 (0.0010) [2023-12-27 05:40:18,682][20381] Updated weights for policy 1, policy_version 557 (0.0010) [2023-12-27 05:40:18,739][20381] Updated weights for policy 1, policy_version 567 (0.0010) [2023-12-27 05:40:18,766][20347] Updated weights for policy 0, policy_version 552 (0.0006) [2023-12-27 05:40:18,827][20347] Updated weights for policy 0, policy_version 562 (0.0007) [2023-12-27 05:40:18,885][20347] Updated weights for policy 0, policy_version 572 (0.0007) [2023-12-27 05:40:19,437][20381] Updated weights for policy 1, policy_version 577 (0.0010) [2023-12-27 05:40:19,500][20381] Updated weights for policy 1, policy_version 587 (0.0008) [2023-12-27 05:40:19,559][20381] Updated weights for policy 1, policy_version 597 (0.0011) [2023-12-27 05:40:19,612][20347] Updated weights for policy 0, policy_version 582 (0.0009) [2023-12-27 05:40:19,619][20381] Updated weights for policy 1, policy_version 607 (0.0011) [2023-12-27 05:40:19,673][20347] Updated weights for policy 0, policy_version 592 (0.0007) [2023-12-27 05:40:19,733][20347] Updated weights for policy 0, policy_version 602 (0.0007) [2023-12-27 05:40:20,350][20381] Updated weights for policy 1, policy_version 617 (0.0011) [2023-12-27 05:40:20,407][20381] Updated weights for policy 1, policy_version 627 (0.0011) [2023-12-27 05:40:20,407][20347] Updated weights for policy 0, policy_version 612 (0.0007) [2023-12-27 05:40:20,467][20381] Updated weights for policy 1, policy_version 637 (0.0010) [2023-12-27 05:40:20,473][20347] Updated weights for policy 0, policy_version 622 (0.0008) [2023-12-27 05:40:20,538][20347] Updated weights for policy 0, policy_version 632 (0.0007) [2023-12-27 05:40:21,176][20381] Updated weights for policy 1, policy_version 647 (0.0008) [2023-12-27 05:40:21,177][20347] Updated weights for policy 0, policy_version 642 (0.0007) [2023-12-27 05:40:21,237][20347] Updated weights for policy 0, policy_version 652 (0.0006) [2023-12-27 05:40:21,243][20381] Updated weights for policy 1, policy_version 657 (0.0006) [2023-12-27 05:40:21,302][20347] Updated weights for policy 0, policy_version 662 (0.0007) [2023-12-27 05:40:21,306][20381] Updated weights for policy 1, policy_version 667 (0.0008) [2023-12-27 05:40:21,364][20347] Updated weights for policy 0, policy_version 672 (0.0008) [2023-12-27 05:40:22,054][20381] Updated weights for policy 1, policy_version 677 (0.0006) [2023-12-27 05:40:22,080][20347] Updated weights for policy 0, policy_version 682 (0.0007) [2023-12-27 05:40:22,101][20381] Updated weights for policy 1, policy_version 687 (0.0008) [2023-12-27 05:40:22,137][20347] Updated weights for policy 0, policy_version 692 (0.0006) [2023-12-27 05:40:22,147][20381] Updated weights for policy 1, policy_version 697 (0.0008) [2023-12-27 05:40:22,193][20347] Updated weights for policy 0, policy_version 702 (0.0007) [2023-12-27 05:40:22,363][19170] Fps is (10 sec: 20479.5, 60 sec: 18022.2, 300 sec: 18022.2). Total num frames: 360448. Throughput: 0: 8059.7, 1: 8201.1. Samples: 325220. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 05:40:22,363][19170] Avg episode reward: [(0, '263.820'), (1, '276.410')] [2023-12-27 05:40:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000000704_180224.pth... [2023-12-27 05:40:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000000704_180224.pth... [2023-12-27 05:40:22,374][20291] Saving new best policy, reward=263.820! [2023-12-27 05:40:22,374][20292] Saving new best policy, reward=276.410! [2023-12-27 05:40:22,878][20381] Updated weights for policy 1, policy_version 707 (0.0009) [2023-12-27 05:40:22,917][20347] Updated weights for policy 0, policy_version 712 (0.0006) [2023-12-27 05:40:22,934][20381] Updated weights for policy 1, policy_version 717 (0.0011) [2023-12-27 05:40:22,967][20347] Updated weights for policy 0, policy_version 722 (0.0006) [2023-12-27 05:40:22,992][20381] Updated weights for policy 1, policy_version 727 (0.0010) [2023-12-27 05:40:23,024][20347] Updated weights for policy 0, policy_version 732 (0.0007) [2023-12-27 05:40:23,588][20381] Updated weights for policy 1, policy_version 737 (0.0006) [2023-12-27 05:40:23,642][20381] Updated weights for policy 1, policy_version 747 (0.0005) [2023-12-27 05:40:23,664][20347] Updated weights for policy 0, policy_version 742 (0.0009) [2023-12-27 05:40:23,698][20381] Updated weights for policy 1, policy_version 757 (0.0005) [2023-12-27 05:40:23,715][20347] Updated weights for policy 0, policy_version 752 (0.0010) [2023-12-27 05:40:23,754][20381] Updated weights for policy 1, policy_version 767 (0.0005) [2023-12-27 05:40:23,769][20347] Updated weights for policy 0, policy_version 762 (0.0010) [2023-12-27 05:40:24,344][20381] Updated weights for policy 1, policy_version 777 (0.0010) [2023-12-27 05:40:24,400][20347] Updated weights for policy 0, policy_version 772 (0.0010) [2023-12-27 05:40:24,400][20381] Updated weights for policy 1, policy_version 787 (0.0011) [2023-12-27 05:40:24,455][20347] Updated weights for policy 0, policy_version 782 (0.0009) [2023-12-27 05:40:24,459][20381] Updated weights for policy 1, policy_version 797 (0.0011) [2023-12-27 05:40:24,509][20347] Updated weights for policy 0, policy_version 792 (0.0007) [2023-12-27 05:40:25,101][20347] Updated weights for policy 0, policy_version 802 (0.0006) [2023-12-27 05:40:25,108][20381] Updated weights for policy 1, policy_version 807 (0.0009) [2023-12-27 05:40:25,162][20347] Updated weights for policy 0, policy_version 812 (0.0010) [2023-12-27 05:40:25,166][20381] Updated weights for policy 1, policy_version 817 (0.0010) [2023-12-27 05:40:25,217][20347] Updated weights for policy 0, policy_version 822 (0.0008) [2023-12-27 05:40:25,227][20381] Updated weights for policy 1, policy_version 827 (0.0010) [2023-12-27 05:40:25,270][20347] Updated weights for policy 0, policy_version 832 (0.0008) [2023-12-27 05:40:25,884][20381] Updated weights for policy 1, policy_version 837 (0.0010) [2023-12-27 05:40:25,909][20347] Updated weights for policy 0, policy_version 842 (0.0008) [2023-12-27 05:40:25,931][20381] Updated weights for policy 1, policy_version 847 (0.0009) [2023-12-27 05:40:25,959][20347] Updated weights for policy 0, policy_version 852 (0.0007) [2023-12-27 05:40:25,983][20381] Updated weights for policy 1, policy_version 857 (0.0005) [2023-12-27 05:40:26,006][20347] Updated weights for policy 0, policy_version 862 (0.0007) [2023-12-27 05:40:26,602][20381] Updated weights for policy 1, policy_version 867 (0.0006) [2023-12-27 05:40:26,616][20347] Updated weights for policy 0, policy_version 872 (0.0007) [2023-12-27 05:40:26,659][20381] Updated weights for policy 1, policy_version 877 (0.0006) [2023-12-27 05:40:26,665][20347] Updated weights for policy 0, policy_version 882 (0.0005) [2023-12-27 05:40:26,709][20381] Updated weights for policy 1, policy_version 887 (0.0005) [2023-12-27 05:40:26,711][20347] Updated weights for policy 0, policy_version 892 (0.0005) [2023-12-27 05:40:27,326][20381] Updated weights for policy 1, policy_version 897 (0.0007) [2023-12-27 05:40:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 18350.1, 300 sec: 18350.1). Total num frames: 458752. Throughput: 0: 8967.2, 1: 9053.0. Samples: 450504. Policy #0 lag: (min: 16.0, avg: 39.0, max: 48.0) [2023-12-27 05:40:27,363][19170] Avg episode reward: [(0, '732.440'), (1, '738.130')] [2023-12-27 05:40:27,364][20291] Saving new best policy, reward=732.440! [2023-12-27 05:40:27,377][20381] Updated weights for policy 1, policy_version 907 (0.0005) [2023-12-27 05:40:27,429][20381] Updated weights for policy 1, policy_version 917 (0.0005) [2023-12-27 05:40:27,436][20347] Updated weights for policy 0, policy_version 902 (0.0008) [2023-12-27 05:40:27,485][20381] Updated weights for policy 1, policy_version 927 (0.0005) [2023-12-27 05:40:27,489][20292] Saving new best policy, reward=738.130! [2023-12-27 05:40:27,498][20347] Updated weights for policy 0, policy_version 912 (0.0010) [2023-12-27 05:40:27,549][20347] Updated weights for policy 0, policy_version 922 (0.0010) [2023-12-27 05:40:28,083][20381] Updated weights for policy 1, policy_version 937 (0.0010) [2023-12-27 05:40:28,136][20381] Updated weights for policy 1, policy_version 947 (0.0010) [2023-12-27 05:40:28,141][20347] Updated weights for policy 0, policy_version 932 (0.0007) [2023-12-27 05:40:28,191][20381] Updated weights for policy 1, policy_version 957 (0.0010) [2023-12-27 05:40:28,192][20347] Updated weights for policy 0, policy_version 942 (0.0005) [2023-12-27 05:40:28,249][20347] Updated weights for policy 0, policy_version 952 (0.0007) [2023-12-27 05:40:28,785][20347] Updated weights for policy 0, policy_version 962 (0.0010) [2023-12-27 05:40:28,842][20347] Updated weights for policy 0, policy_version 972 (0.0010) [2023-12-27 05:40:28,848][20381] Updated weights for policy 1, policy_version 967 (0.0008) [2023-12-27 05:40:28,900][20347] Updated weights for policy 0, policy_version 982 (0.0010) [2023-12-27 05:40:28,902][20381] Updated weights for policy 1, policy_version 977 (0.0006) [2023-12-27 05:40:28,957][20381] Updated weights for policy 1, policy_version 987 (0.0006) [2023-12-27 05:40:28,963][20347] Updated weights for policy 0, policy_version 992 (0.0010) [2023-12-27 05:40:29,571][20381] Updated weights for policy 1, policy_version 997 (0.0006) [2023-12-27 05:40:29,611][20347] Updated weights for policy 0, policy_version 1002 (0.0010) [2023-12-27 05:40:29,631][20381] Updated weights for policy 1, policy_version 1007 (0.0007) [2023-12-27 05:40:29,675][20347] Updated weights for policy 0, policy_version 1012 (0.0010) [2023-12-27 05:40:29,687][20381] Updated weights for policy 1, policy_version 1017 (0.0007) [2023-12-27 05:40:29,736][20347] Updated weights for policy 0, policy_version 1022 (0.0010) [2023-12-27 05:40:30,286][20381] Updated weights for policy 1, policy_version 1027 (0.0006) [2023-12-27 05:40:30,338][20381] Updated weights for policy 1, policy_version 1037 (0.0007) [2023-12-27 05:40:30,355][20347] Updated weights for policy 0, policy_version 1032 (0.0010) [2023-12-27 05:40:30,399][20381] Updated weights for policy 1, policy_version 1047 (0.0005) [2023-12-27 05:40:30,408][20347] Updated weights for policy 0, policy_version 1042 (0.0010) [2023-12-27 05:40:30,453][20347] Updated weights for policy 0, policy_version 1052 (0.0010) [2023-12-27 05:40:31,071][20381] Updated weights for policy 1, policy_version 1057 (0.0006) [2023-12-27 05:40:31,127][20347] Updated weights for policy 0, policy_version 1062 (0.0010) [2023-12-27 05:40:31,132][20381] Updated weights for policy 1, policy_version 1067 (0.0008) [2023-12-27 05:40:31,192][20381] Updated weights for policy 1, policy_version 1077 (0.0007) [2023-12-27 05:40:31,193][20347] Updated weights for policy 0, policy_version 1072 (0.0011) [2023-12-27 05:40:31,251][20347] Updated weights for policy 0, policy_version 1082 (0.0010) [2023-12-27 05:40:31,253][20381] Updated weights for policy 1, policy_version 1087 (0.0008) [2023-12-27 05:40:31,955][20347] Updated weights for policy 0, policy_version 1092 (0.0009) [2023-12-27 05:40:31,961][20381] Updated weights for policy 1, policy_version 1097 (0.0007) [2023-12-27 05:40:32,011][20347] Updated weights for policy 0, policy_version 1102 (0.0010) [2023-12-27 05:40:32,017][20381] Updated weights for policy 1, policy_version 1107 (0.0006) [2023-12-27 05:40:32,063][20347] Updated weights for policy 0, policy_version 1112 (0.0010) [2023-12-27 05:40:32,074][20381] Updated weights for policy 1, policy_version 1117 (0.0006) [2023-12-27 05:40:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 19114.6, 300 sec: 19114.6). Total num frames: 573440. Throughput: 0: 9644.0, 1: 9703.8. Samples: 580436. Policy #0 lag: (min: 13.0, avg: 15.1, max: 45.0) [2023-12-27 05:40:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:32,364][20291] Saving new best policy, reward=1000.000! [2023-12-27 05:40:32,364][20292] Saving new best policy, reward=1000.000! [2023-12-27 05:40:32,757][20347] Updated weights for policy 0, policy_version 1122 (0.0009) [2023-12-27 05:40:32,764][20381] Updated weights for policy 1, policy_version 1127 (0.0008) [2023-12-27 05:40:32,809][20347] Updated weights for policy 0, policy_version 1132 (0.0005) [2023-12-27 05:40:32,829][20381] Updated weights for policy 1, policy_version 1137 (0.0008) [2023-12-27 05:40:32,857][20347] Updated weights for policy 0, policy_version 1142 (0.0005) [2023-12-27 05:40:32,882][20381] Updated weights for policy 1, policy_version 1147 (0.0006) [2023-12-27 05:40:32,921][20347] Updated weights for policy 0, policy_version 1152 (0.0005) [2023-12-27 05:40:33,465][20347] Updated weights for policy 0, policy_version 1162 (0.0005) [2023-12-27 05:40:33,518][20347] Updated weights for policy 0, policy_version 1172 (0.0007) [2023-12-27 05:40:33,522][20381] Updated weights for policy 1, policy_version 1157 (0.0008) [2023-12-27 05:40:33,571][20381] Updated weights for policy 1, policy_version 1167 (0.0008) [2023-12-27 05:40:33,573][20347] Updated weights for policy 0, policy_version 1182 (0.0008) [2023-12-27 05:40:33,625][20381] Updated weights for policy 1, policy_version 1177 (0.0010) [2023-12-27 05:40:34,218][20347] Updated weights for policy 0, policy_version 1192 (0.0009) [2023-12-27 05:40:34,221][20381] Updated weights for policy 1, policy_version 1187 (0.0010) [2023-12-27 05:40:34,272][20381] Updated weights for policy 1, policy_version 1197 (0.0010) [2023-12-27 05:40:34,279][20347] Updated weights for policy 0, policy_version 1202 (0.0006) [2023-12-27 05:40:34,333][20381] Updated weights for policy 1, policy_version 1207 (0.0009) [2023-12-27 05:40:34,340][20347] Updated weights for policy 0, policy_version 1212 (0.0005) [2023-12-27 05:40:34,956][20347] Updated weights for policy 0, policy_version 1222 (0.0006) [2023-12-27 05:40:35,004][20347] Updated weights for policy 0, policy_version 1232 (0.0006) [2023-12-27 05:40:35,006][20381] Updated weights for policy 1, policy_version 1217 (0.0010) [2023-12-27 05:40:35,055][20347] Updated weights for policy 0, policy_version 1242 (0.0005) [2023-12-27 05:40:35,067][20381] Updated weights for policy 1, policy_version 1227 (0.0006) [2023-12-27 05:40:35,122][20381] Updated weights for policy 1, policy_version 1237 (0.0007) [2023-12-27 05:40:35,183][20381] Updated weights for policy 1, policy_version 1247 (0.0007) [2023-12-27 05:40:35,729][20347] Updated weights for policy 0, policy_version 1252 (0.0006) [2023-12-27 05:40:35,773][20347] Updated weights for policy 0, policy_version 1262 (0.0008) [2023-12-27 05:40:35,819][20347] Updated weights for policy 0, policy_version 1272 (0.0005) [2023-12-27 05:40:35,857][20381] Updated weights for policy 1, policy_version 1257 (0.0010) [2023-12-27 05:40:35,908][20381] Updated weights for policy 1, policy_version 1267 (0.0010) [2023-12-27 05:40:35,963][20381] Updated weights for policy 1, policy_version 1277 (0.0010) [2023-12-27 05:40:36,484][20347] Updated weights for policy 0, policy_version 1282 (0.0006) [2023-12-27 05:40:36,549][20347] Updated weights for policy 0, policy_version 1292 (0.0008) [2023-12-27 05:40:36,594][20381] Updated weights for policy 1, policy_version 1287 (0.0007) [2023-12-27 05:40:36,613][20347] Updated weights for policy 0, policy_version 1302 (0.0008) [2023-12-27 05:40:36,652][20381] Updated weights for policy 1, policy_version 1297 (0.0009) [2023-12-27 05:40:36,673][20347] Updated weights for policy 0, policy_version 1312 (0.0006) [2023-12-27 05:40:36,713][20381] Updated weights for policy 1, policy_version 1307 (0.0007) [2023-12-27 05:40:37,337][20381] Updated weights for policy 1, policy_version 1317 (0.0010) [2023-12-27 05:40:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 19192.7, 300 sec: 19192.7). Total num frames: 671744. Throughput: 0: 9209.3, 1: 9229.4. Samples: 645352. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) [2023-12-27 05:40:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:37,387][20347] Updated weights for policy 0, policy_version 1322 (0.0007) [2023-12-27 05:40:37,392][20381] Updated weights for policy 1, policy_version 1327 (0.0010) [2023-12-27 05:40:37,437][20347] Updated weights for policy 0, policy_version 1332 (0.0005) [2023-12-27 05:40:37,454][20381] Updated weights for policy 1, policy_version 1337 (0.0010) [2023-12-27 05:40:37,488][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000001344_344064.pth... [2023-12-27 05:40:37,491][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000000128_32768.pth [2023-12-27 05:40:37,493][20347] Updated weights for policy 0, policy_version 1342 (0.0005) [2023-12-27 05:40:37,500][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000001344_344064.pth... [2023-12-27 05:40:37,504][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000000128_32768.pth [2023-12-27 05:40:38,082][20381] Updated weights for policy 1, policy_version 1347 (0.0011) [2023-12-27 05:40:38,127][20381] Updated weights for policy 1, policy_version 1357 (0.0010) [2023-12-27 05:40:38,175][20381] Updated weights for policy 1, policy_version 1367 (0.0010) [2023-12-27 05:40:38,207][20347] Updated weights for policy 0, policy_version 1352 (0.0006) [2023-12-27 05:40:38,267][20347] Updated weights for policy 0, policy_version 1362 (0.0006) [2023-12-27 05:40:38,318][20347] Updated weights for policy 0, policy_version 1372 (0.0006) [2023-12-27 05:40:38,850][20381] Updated weights for policy 1, policy_version 1377 (0.0009) [2023-12-27 05:40:38,899][20381] Updated weights for policy 1, policy_version 1387 (0.0010) [2023-12-27 05:40:38,950][20381] Updated weights for policy 1, policy_version 1397 (0.0009) [2023-12-27 05:40:39,003][20347] Updated weights for policy 0, policy_version 1382 (0.0006) [2023-12-27 05:40:39,009][20381] Updated weights for policy 1, policy_version 1407 (0.0007) [2023-12-27 05:40:39,056][20347] Updated weights for policy 0, policy_version 1392 (0.0007) [2023-12-27 05:40:39,102][20347] Updated weights for policy 0, policy_version 1402 (0.0007) [2023-12-27 05:40:39,684][20381] Updated weights for policy 1, policy_version 1417 (0.0008) [2023-12-27 05:40:39,742][20381] Updated weights for policy 1, policy_version 1427 (0.0006) [2023-12-27 05:40:39,750][20347] Updated weights for policy 0, policy_version 1412 (0.0008) [2023-12-27 05:40:39,803][20381] Updated weights for policy 1, policy_version 1437 (0.0006) [2023-12-27 05:40:39,812][20347] Updated weights for policy 0, policy_version 1422 (0.0007) [2023-12-27 05:40:39,882][20347] Updated weights for policy 0, policy_version 1432 (0.0013) [2023-12-27 05:40:40,460][20381] Updated weights for policy 1, policy_version 1447 (0.0008) [2023-12-27 05:40:40,504][20381] Updated weights for policy 1, policy_version 1457 (0.0007) [2023-12-27 05:40:40,544][20347] Updated weights for policy 0, policy_version 1442 (0.0008) [2023-12-27 05:40:40,550][20381] Updated weights for policy 1, policy_version 1467 (0.0007) [2023-12-27 05:40:40,598][20347] Updated weights for policy 0, policy_version 1452 (0.0010) [2023-12-27 05:40:40,660][20347] Updated weights for policy 0, policy_version 1462 (0.0010) [2023-12-27 05:40:40,721][20347] Updated weights for policy 0, policy_version 1472 (0.0010) [2023-12-27 05:40:41,219][20381] Updated weights for policy 1, policy_version 1477 (0.0006) [2023-12-27 05:40:41,278][20381] Updated weights for policy 1, policy_version 1487 (0.0008) [2023-12-27 05:40:41,341][20381] Updated weights for policy 1, policy_version 1497 (0.0008) [2023-12-27 05:40:41,403][20347] Updated weights for policy 0, policy_version 1482 (0.0009) [2023-12-27 05:40:41,463][20347] Updated weights for policy 0, policy_version 1492 (0.0008) [2023-12-27 05:40:41,518][20347] Updated weights for policy 0, policy_version 1502 (0.0008) [2023-12-27 05:40:42,053][20381] Updated weights for policy 1, policy_version 1507 (0.0007) [2023-12-27 05:40:42,103][20381] Updated weights for policy 1, policy_version 1517 (0.0008) [2023-12-27 05:40:42,154][20381] Updated weights for policy 1, policy_version 1527 (0.0008) [2023-12-27 05:40:42,243][20347] Updated weights for policy 0, policy_version 1512 (0.0008) [2023-12-27 05:40:42,311][20347] Updated weights for policy 0, policy_version 1522 (0.0011) [2023-12-27 05:40:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19456.0, 300 sec: 19456.0). Total num frames: 778240. Throughput: 0: 9588.9, 1: 9663.0. Samples: 770076. Policy #0 lag: (min: 10.0, avg: 18.6, max: 42.0) [2023-12-27 05:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:42,377][20347] Updated weights for policy 0, policy_version 1532 (0.0008) [2023-12-27 05:40:42,896][20381] Updated weights for policy 1, policy_version 1537 (0.0007) [2023-12-27 05:40:42,955][20381] Updated weights for policy 1, policy_version 1547 (0.0008) [2023-12-27 05:40:43,000][20381] Updated weights for policy 1, policy_version 1557 (0.0008) [2023-12-27 05:40:43,055][20381] Updated weights for policy 1, policy_version 1567 (0.0009) [2023-12-27 05:40:43,061][20347] Updated weights for policy 0, policy_version 1542 (0.0009) [2023-12-27 05:40:43,119][20347] Updated weights for policy 0, policy_version 1552 (0.0010) [2023-12-27 05:40:43,180][20347] Updated weights for policy 0, policy_version 1562 (0.0010) [2023-12-27 05:40:43,773][20381] Updated weights for policy 1, policy_version 1577 (0.0008) [2023-12-27 05:40:43,803][20347] Updated weights for policy 0, policy_version 1572 (0.0010) [2023-12-27 05:40:43,835][20381] Updated weights for policy 1, policy_version 1587 (0.0005) [2023-12-27 05:40:43,860][20347] Updated weights for policy 0, policy_version 1582 (0.0010) [2023-12-27 05:40:43,889][20381] Updated weights for policy 1, policy_version 1597 (0.0006) [2023-12-27 05:40:43,920][20347] Updated weights for policy 0, policy_version 1592 (0.0010) [2023-12-27 05:40:44,526][20381] Updated weights for policy 1, policy_version 1607 (0.0005) [2023-12-27 05:40:44,555][20347] Updated weights for policy 0, policy_version 1602 (0.0010) [2023-12-27 05:40:44,581][20381] Updated weights for policy 1, policy_version 1617 (0.0006) [2023-12-27 05:40:44,611][20347] Updated weights for policy 0, policy_version 1612 (0.0009) [2023-12-27 05:40:44,636][20381] Updated weights for policy 1, policy_version 1627 (0.0007) [2023-12-27 05:40:44,670][20347] Updated weights for policy 0, policy_version 1622 (0.0010) [2023-12-27 05:40:44,728][20347] Updated weights for policy 0, policy_version 1632 (0.0010) [2023-12-27 05:40:45,331][20381] Updated weights for policy 1, policy_version 1637 (0.0006) [2023-12-27 05:40:45,383][20381] Updated weights for policy 1, policy_version 1647 (0.0007) [2023-12-27 05:40:45,423][20347] Updated weights for policy 0, policy_version 1642 (0.0010) [2023-12-27 05:40:45,438][20381] Updated weights for policy 1, policy_version 1657 (0.0010) [2023-12-27 05:40:45,476][20347] Updated weights for policy 0, policy_version 1652 (0.0010) [2023-12-27 05:40:45,528][20347] Updated weights for policy 0, policy_version 1662 (0.0009) [2023-12-27 05:40:46,063][20381] Updated weights for policy 1, policy_version 1667 (0.0009) [2023-12-27 05:40:46,116][20381] Updated weights for policy 1, policy_version 1677 (0.0005) [2023-12-27 05:40:46,171][20381] Updated weights for policy 1, policy_version 1687 (0.0005) [2023-12-27 05:40:46,183][20347] Updated weights for policy 0, policy_version 1672 (0.0009) [2023-12-27 05:40:46,232][20347] Updated weights for policy 0, policy_version 1682 (0.0010) [2023-12-27 05:40:46,293][20347] Updated weights for policy 0, policy_version 1692 (0.0010) [2023-12-27 05:40:46,710][20381] Updated weights for policy 1, policy_version 1697 (0.0005) [2023-12-27 05:40:46,774][20381] Updated weights for policy 1, policy_version 1707 (0.0005) [2023-12-27 05:40:46,832][20381] Updated weights for policy 1, policy_version 1717 (0.0005) [2023-12-27 05:40:46,894][20347] Updated weights for policy 0, policy_version 1702 (0.0010) [2023-12-27 05:40:46,896][20381] Updated weights for policy 1, policy_version 1727 (0.0005) [2023-12-27 05:40:46,956][20347] Updated weights for policy 0, policy_version 1712 (0.0010) [2023-12-27 05:40:47,018][20347] Updated weights for policy 0, policy_version 1722 (0.0010) [2023-12-27 05:40:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 884736. Throughput: 0: 9902.8, 1: 9982.8. Samples: 894856. Policy #0 lag: (min: 4.0, avg: 14.5, max: 36.0) [2023-12-27 05:40:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:47,466][20381] Updated weights for policy 1, policy_version 1737 (0.0008) [2023-12-27 05:40:47,518][20381] Updated weights for policy 1, policy_version 1747 (0.0009) [2023-12-27 05:40:47,573][20381] Updated weights for policy 1, policy_version 1757 (0.0010) [2023-12-27 05:40:47,693][20347] Updated weights for policy 0, policy_version 1732 (0.0010) [2023-12-27 05:40:47,752][20347] Updated weights for policy 0, policy_version 1742 (0.0011) [2023-12-27 05:40:47,805][20347] Updated weights for policy 0, policy_version 1752 (0.0011) [2023-12-27 05:40:48,270][20381] Updated weights for policy 1, policy_version 1767 (0.0008) [2023-12-27 05:40:48,322][20381] Updated weights for policy 1, policy_version 1777 (0.0008) [2023-12-27 05:40:48,384][20381] Updated weights for policy 1, policy_version 1787 (0.0008) [2023-12-27 05:40:48,486][20347] Updated weights for policy 0, policy_version 1762 (0.0010) [2023-12-27 05:40:48,534][20347] Updated weights for policy 0, policy_version 1772 (0.0010) [2023-12-27 05:40:48,589][20347] Updated weights for policy 0, policy_version 1782 (0.0010) [2023-12-27 05:40:48,649][20347] Updated weights for policy 0, policy_version 1792 (0.0009) [2023-12-27 05:40:49,120][20381] Updated weights for policy 1, policy_version 1797 (0.0007) [2023-12-27 05:40:49,165][20381] Updated weights for policy 1, policy_version 1807 (0.0007) [2023-12-27 05:40:49,217][20381] Updated weights for policy 1, policy_version 1817 (0.0007) [2023-12-27 05:40:49,276][20347] Updated weights for policy 0, policy_version 1802 (0.0007) [2023-12-27 05:40:49,342][20347] Updated weights for policy 0, policy_version 1812 (0.0008) [2023-12-27 05:40:49,398][20347] Updated weights for policy 0, policy_version 1822 (0.0008) [2023-12-27 05:40:49,933][20381] Updated weights for policy 1, policy_version 1827 (0.0008) [2023-12-27 05:40:49,990][20381] Updated weights for policy 1, policy_version 1837 (0.0007) [2023-12-27 05:40:50,055][20381] Updated weights for policy 1, policy_version 1847 (0.0008) [2023-12-27 05:40:50,082][20347] Updated weights for policy 0, policy_version 1832 (0.0007) [2023-12-27 05:40:50,140][20347] Updated weights for policy 0, policy_version 1842 (0.0006) [2023-12-27 05:40:50,201][20347] Updated weights for policy 0, policy_version 1852 (0.0007) [2023-12-27 05:40:50,746][20381] Updated weights for policy 1, policy_version 1857 (0.0010) [2023-12-27 05:40:50,797][20381] Updated weights for policy 1, policy_version 1867 (0.0007) [2023-12-27 05:40:50,850][20381] Updated weights for policy 1, policy_version 1877 (0.0008) [2023-12-27 05:40:50,883][20347] Updated weights for policy 0, policy_version 1862 (0.0007) [2023-12-27 05:40:50,908][20381] Updated weights for policy 1, policy_version 1887 (0.0010) [2023-12-27 05:40:50,934][20347] Updated weights for policy 0, policy_version 1872 (0.0005) [2023-12-27 05:40:50,990][20347] Updated weights for policy 0, policy_version 1882 (0.0006) [2023-12-27 05:40:51,600][20381] Updated weights for policy 1, policy_version 1897 (0.0011) [2023-12-27 05:40:51,662][20381] Updated weights for policy 1, policy_version 1907 (0.0008) [2023-12-27 05:40:51,705][20347] Updated weights for policy 0, policy_version 1892 (0.0007) [2023-12-27 05:40:51,726][20381] Updated weights for policy 1, policy_version 1917 (0.0008) [2023-12-27 05:40:51,773][20347] Updated weights for policy 0, policy_version 1902 (0.0006) [2023-12-27 05:40:51,829][20347] Updated weights for policy 0, policy_version 1912 (0.0007) [2023-12-27 05:40:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19660.8). Total num frames: 983040. Throughput: 0: 10218.1, 1: 10278.5. Samples: 955412. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 05:40:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000001920_491520.pth... [2023-12-27 05:40:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000000704_180224.pth [2023-12-27 05:40:52,393][20381] Updated weights for policy 1, policy_version 1927 (0.0010) [2023-12-27 05:40:52,448][20381] Updated weights for policy 1, policy_version 1937 (0.0010) [2023-12-27 05:40:52,477][20347] Updated weights for policy 0, policy_version 1922 (0.0007) [2023-12-27 05:40:52,501][20381] Updated weights for policy 1, policy_version 1947 (0.0010) [2023-12-27 05:40:52,524][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000001952_499712.pth... [2023-12-27 05:40:52,525][20347] Updated weights for policy 0, policy_version 1932 (0.0008) [2023-12-27 05:40:52,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000000704_180224.pth [2023-12-27 05:40:52,579][20347] Updated weights for policy 0, policy_version 1942 (0.0008) [2023-12-27 05:40:52,638][20347] Updated weights for policy 0, policy_version 1952 (0.0008) [2023-12-27 05:40:53,141][20381] Updated weights for policy 1, policy_version 1957 (0.0009) [2023-12-27 05:40:53,187][20381] Updated weights for policy 1, policy_version 1967 (0.0010) [2023-12-27 05:40:53,236][20381] Updated weights for policy 1, policy_version 1977 (0.0010) [2023-12-27 05:40:53,336][20347] Updated weights for policy 0, policy_version 1962 (0.0010) [2023-12-27 05:40:53,391][20347] Updated weights for policy 0, policy_version 1972 (0.0010) [2023-12-27 05:40:53,445][20347] Updated weights for policy 0, policy_version 1982 (0.0009) [2023-12-27 05:40:53,920][20381] Updated weights for policy 1, policy_version 1987 (0.0010) [2023-12-27 05:40:53,975][20381] Updated weights for policy 1, policy_version 1997 (0.0010) [2023-12-27 05:40:54,022][20347] Updated weights for policy 0, policy_version 1992 (0.0006) [2023-12-27 05:40:54,024][20381] Updated weights for policy 1, policy_version 2007 (0.0010) [2023-12-27 05:40:54,077][20347] Updated weights for policy 0, policy_version 2002 (0.0005) [2023-12-27 05:40:54,127][20347] Updated weights for policy 0, policy_version 2012 (0.0007) [2023-12-27 05:40:54,722][20381] Updated weights for policy 1, policy_version 2017 (0.0010) [2023-12-27 05:40:54,771][20381] Updated weights for policy 1, policy_version 2027 (0.0010) [2023-12-27 05:40:54,789][20347] Updated weights for policy 0, policy_version 2022 (0.0006) [2023-12-27 05:40:54,816][20381] Updated weights for policy 1, policy_version 2037 (0.0009) [2023-12-27 05:40:54,843][20347] Updated weights for policy 0, policy_version 2032 (0.0005) [2023-12-27 05:40:54,879][20381] Updated weights for policy 1, policy_version 2047 (0.0008) [2023-12-27 05:40:54,900][20347] Updated weights for policy 0, policy_version 2042 (0.0005) [2023-12-27 05:40:55,496][20347] Updated weights for policy 0, policy_version 2052 (0.0006) [2023-12-27 05:40:55,551][20347] Updated weights for policy 0, policy_version 2062 (0.0005) [2023-12-27 05:40:55,562][20381] Updated weights for policy 1, policy_version 2057 (0.0007) [2023-12-27 05:40:55,613][20347] Updated weights for policy 0, policy_version 2072 (0.0007) [2023-12-27 05:40:55,617][20381] Updated weights for policy 1, policy_version 2067 (0.0011) [2023-12-27 05:40:55,673][20381] Updated weights for policy 1, policy_version 2077 (0.0010) [2023-12-27 05:40:56,175][20347] Updated weights for policy 0, policy_version 2082 (0.0006) [2023-12-27 05:40:56,232][20347] Updated weights for policy 0, policy_version 2092 (0.0007) [2023-12-27 05:40:56,286][20347] Updated weights for policy 0, policy_version 2102 (0.0006) [2023-12-27 05:40:56,321][20381] Updated weights for policy 1, policy_version 2087 (0.0009) [2023-12-27 05:40:56,343][20347] Updated weights for policy 0, policy_version 2112 (0.0010) [2023-12-27 05:40:56,378][20381] Updated weights for policy 1, policy_version 2097 (0.0008) [2023-12-27 05:40:56,422][20381] Updated weights for policy 1, policy_version 2107 (0.0008) [2023-12-27 05:40:56,885][20347] Updated weights for policy 0, policy_version 2122 (0.0010) [2023-12-27 05:40:56,945][20347] Updated weights for policy 0, policy_version 2132 (0.0006) [2023-12-27 05:40:57,005][20381] Updated weights for policy 1, policy_version 2117 (0.0006) [2023-12-27 05:40:57,007][20347] Updated weights for policy 0, policy_version 2142 (0.0005) [2023-12-27 05:40:57,061][20381] Updated weights for policy 1, policy_version 2127 (0.0006) [2023-12-27 05:40:57,117][20381] Updated weights for policy 1, policy_version 2137 (0.0006) [2023-12-27 05:40:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 19958.7, 300 sec: 19958.7). Total num frames: 1097728. Throughput: 0: 10404.9, 1: 10438.1. Samples: 1081404. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 05:40:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:40:57,612][20347] Updated weights for policy 0, policy_version 2152 (0.0006) [2023-12-27 05:40:57,665][20347] Updated weights for policy 0, policy_version 2162 (0.0006) [2023-12-27 05:40:57,722][20347] Updated weights for policy 0, policy_version 2172 (0.0005) [2023-12-27 05:40:57,727][20381] Updated weights for policy 1, policy_version 2147 (0.0008) [2023-12-27 05:40:57,785][20381] Updated weights for policy 1, policy_version 2157 (0.0010) [2023-12-27 05:40:57,832][20381] Updated weights for policy 1, policy_version 2167 (0.0005) [2023-12-27 05:40:58,359][20347] Updated weights for policy 0, policy_version 2182 (0.0007) [2023-12-27 05:40:58,417][20347] Updated weights for policy 0, policy_version 2192 (0.0006) [2023-12-27 05:40:58,479][20347] Updated weights for policy 0, policy_version 2202 (0.0008) [2023-12-27 05:40:58,544][20381] Updated weights for policy 1, policy_version 2177 (0.0006) [2023-12-27 05:40:58,606][20381] Updated weights for policy 1, policy_version 2187 (0.0008) [2023-12-27 05:40:58,669][20381] Updated weights for policy 1, policy_version 2197 (0.0008) [2023-12-27 05:40:58,726][20381] Updated weights for policy 1, policy_version 2207 (0.0008) [2023-12-27 05:40:59,338][20347] Updated weights for policy 0, policy_version 2212 (0.0007) [2023-12-27 05:40:59,408][20347] Updated weights for policy 0, policy_version 2222 (0.0007) [2023-12-27 05:40:59,436][20381] Updated weights for policy 1, policy_version 2217 (0.0010) [2023-12-27 05:40:59,466][20347] Updated weights for policy 0, policy_version 2232 (0.0006) [2023-12-27 05:40:59,484][20381] Updated weights for policy 1, policy_version 2227 (0.0010) [2023-12-27 05:40:59,536][20381] Updated weights for policy 1, policy_version 2237 (0.0010) [2023-12-27 05:41:00,128][20347] Updated weights for policy 0, policy_version 2242 (0.0006) [2023-12-27 05:41:00,179][20347] Updated weights for policy 0, policy_version 2252 (0.0005) [2023-12-27 05:41:00,228][20347] Updated weights for policy 0, policy_version 2262 (0.0006) [2023-12-27 05:41:00,269][20381] Updated weights for policy 1, policy_version 2247 (0.0009) [2023-12-27 05:41:00,282][20347] Updated weights for policy 0, policy_version 2272 (0.0007) [2023-12-27 05:41:00,326][20381] Updated weights for policy 1, policy_version 2257 (0.0008) [2023-12-27 05:41:00,378][20381] Updated weights for policy 1, policy_version 2267 (0.0008) [2023-12-27 05:41:00,973][20347] Updated weights for policy 0, policy_version 2282 (0.0008) [2023-12-27 05:41:01,024][20381] Updated weights for policy 1, policy_version 2277 (0.0008) [2023-12-27 05:41:01,026][20347] Updated weights for policy 0, policy_version 2292 (0.0007) [2023-12-27 05:41:01,087][20347] Updated weights for policy 0, policy_version 2302 (0.0006) [2023-12-27 05:41:01,088][20381] Updated weights for policy 1, policy_version 2287 (0.0007) [2023-12-27 05:41:01,153][20381] Updated weights for policy 1, policy_version 2297 (0.0006) [2023-12-27 05:41:01,716][20381] Updated weights for policy 1, policy_version 2307 (0.0006) [2023-12-27 05:41:01,781][20381] Updated weights for policy 1, policy_version 2317 (0.0006) [2023-12-27 05:41:01,835][20381] Updated weights for policy 1, policy_version 2327 (0.0006) [2023-12-27 05:41:01,837][20347] Updated weights for policy 0, policy_version 2312 (0.0007) [2023-12-27 05:41:01,888][20347] Updated weights for policy 0, policy_version 2322 (0.0005) [2023-12-27 05:41:01,936][20347] Updated weights for policy 0, policy_version 2332 (0.0007) [2023-12-27 05:41:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19933.9). Total num frames: 1196032. Throughput: 0: 10461.5, 1: 10470.8. Samples: 1206548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 05:41:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:02,483][20381] Updated weights for policy 1, policy_version 2337 (0.0007) [2023-12-27 05:41:02,532][20381] Updated weights for policy 1, policy_version 2347 (0.0008) [2023-12-27 05:41:02,597][20381] Updated weights for policy 1, policy_version 2357 (0.0007) [2023-12-27 05:41:02,647][20381] Updated weights for policy 1, policy_version 2367 (0.0006) [2023-12-27 05:41:02,656][20347] Updated weights for policy 0, policy_version 2342 (0.0007) [2023-12-27 05:41:02,716][20347] Updated weights for policy 0, policy_version 2352 (0.0008) [2023-12-27 05:41:02,773][20347] Updated weights for policy 0, policy_version 2362 (0.0008) [2023-12-27 05:41:03,343][20381] Updated weights for policy 1, policy_version 2377 (0.0007) [2023-12-27 05:41:03,374][20347] Updated weights for policy 0, policy_version 2372 (0.0008) [2023-12-27 05:41:03,394][20381] Updated weights for policy 1, policy_version 2387 (0.0005) [2023-12-27 05:41:03,429][20347] Updated weights for policy 0, policy_version 2382 (0.0006) [2023-12-27 05:41:03,443][20381] Updated weights for policy 1, policy_version 2397 (0.0005) [2023-12-27 05:41:03,483][20347] Updated weights for policy 0, policy_version 2392 (0.0006) [2023-12-27 05:41:04,083][20347] Updated weights for policy 0, policy_version 2402 (0.0006) [2023-12-27 05:41:04,129][20381] Updated weights for policy 1, policy_version 2407 (0.0009) [2023-12-27 05:41:04,145][20347] Updated weights for policy 0, policy_version 2412 (0.0008) [2023-12-27 05:41:04,189][20381] Updated weights for policy 1, policy_version 2417 (0.0011) [2023-12-27 05:41:04,197][20347] Updated weights for policy 0, policy_version 2422 (0.0008) [2023-12-27 05:41:04,248][20381] Updated weights for policy 1, policy_version 2427 (0.0010) [2023-12-27 05:41:04,255][20347] Updated weights for policy 0, policy_version 2432 (0.0008) [2023-12-27 05:41:04,908][20381] Updated weights for policy 1, policy_version 2437 (0.0009) [2023-12-27 05:41:04,943][20347] Updated weights for policy 0, policy_version 2442 (0.0005) [2023-12-27 05:41:04,964][20381] Updated weights for policy 1, policy_version 2447 (0.0006) [2023-12-27 05:41:05,000][20347] Updated weights for policy 0, policy_version 2452 (0.0006) [2023-12-27 05:41:05,018][20381] Updated weights for policy 1, policy_version 2457 (0.0007) [2023-12-27 05:41:05,055][20347] Updated weights for policy 0, policy_version 2462 (0.0006) [2023-12-27 05:41:05,693][20347] Updated weights for policy 0, policy_version 2472 (0.0008) [2023-12-27 05:41:05,723][20381] Updated weights for policy 1, policy_version 2467 (0.0007) [2023-12-27 05:41:05,749][20347] Updated weights for policy 0, policy_version 2482 (0.0009) [2023-12-27 05:41:05,773][20381] Updated weights for policy 1, policy_version 2477 (0.0007) [2023-12-27 05:41:05,801][20347] Updated weights for policy 0, policy_version 2492 (0.0006) [2023-12-27 05:41:05,821][20381] Updated weights for policy 1, policy_version 2487 (0.0005) [2023-12-27 05:41:06,451][20347] Updated weights for policy 0, policy_version 2502 (0.0006) [2023-12-27 05:41:06,504][20347] Updated weights for policy 0, policy_version 2512 (0.0008) [2023-12-27 05:41:06,531][20381] Updated weights for policy 1, policy_version 2497 (0.0007) [2023-12-27 05:41:06,560][20347] Updated weights for policy 0, policy_version 2522 (0.0009) [2023-12-27 05:41:06,590][20381] Updated weights for policy 1, policy_version 2507 (0.0005) [2023-12-27 05:41:06,644][20381] Updated weights for policy 1, policy_version 2517 (0.0005) [2023-12-27 05:41:06,702][20381] Updated weights for policy 1, policy_version 2527 (0.0006) [2023-12-27 05:41:07,143][20347] Updated weights for policy 0, policy_version 2532 (0.0008) [2023-12-27 05:41:07,193][20347] Updated weights for policy 0, policy_version 2542 (0.0006) [2023-12-27 05:41:07,242][20347] Updated weights for policy 0, policy_version 2552 (0.0007) [2023-12-27 05:41:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20038.9). Total num frames: 1302528. Throughput: 0: 10513.3, 1: 10466.4. Samples: 1269304. Policy #0 lag: (min: 31.0, avg: 42.3, max: 63.0) [2023-12-27 05:41:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000002560_655360.pth... [2023-12-27 05:41:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000001344_344064.pth [2023-12-27 05:41:07,375][20381] Updated weights for policy 1, policy_version 2537 (0.0008) [2023-12-27 05:41:07,426][20381] Updated weights for policy 1, policy_version 2547 (0.0005) [2023-12-27 05:41:07,477][20381] Updated weights for policy 1, policy_version 2557 (0.0005) [2023-12-27 05:41:07,490][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000002560_655360.pth... [2023-12-27 05:41:07,493][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000001344_344064.pth [2023-12-27 05:41:07,801][20347] Updated weights for policy 0, policy_version 2562 (0.0005) [2023-12-27 05:41:07,852][20347] Updated weights for policy 0, policy_version 2572 (0.0005) [2023-12-27 05:41:07,910][20347] Updated weights for policy 0, policy_version 2582 (0.0005) [2023-12-27 05:41:07,961][20347] Updated weights for policy 0, policy_version 2592 (0.0005) [2023-12-27 05:41:08,089][20381] Updated weights for policy 1, policy_version 2567 (0.0005) [2023-12-27 05:41:08,137][20381] Updated weights for policy 1, policy_version 2577 (0.0009) [2023-12-27 05:41:08,190][20381] Updated weights for policy 1, policy_version 2587 (0.0009) [2023-12-27 05:41:08,496][20347] Updated weights for policy 0, policy_version 2602 (0.0008) [2023-12-27 05:41:08,547][20347] Updated weights for policy 0, policy_version 2612 (0.0005) [2023-12-27 05:41:08,608][20347] Updated weights for policy 0, policy_version 2622 (0.0006) [2023-12-27 05:41:08,874][20381] Updated weights for policy 1, policy_version 2597 (0.0008) [2023-12-27 05:41:08,943][20381] Updated weights for policy 1, policy_version 2607 (0.0008) [2023-12-27 05:41:09,013][20381] Updated weights for policy 1, policy_version 2617 (0.0009) [2023-12-27 05:41:09,195][20347] Updated weights for policy 0, policy_version 2632 (0.0009) [2023-12-27 05:41:09,263][20347] Updated weights for policy 0, policy_version 2642 (0.0007) [2023-12-27 05:41:09,314][20347] Updated weights for policy 0, policy_version 2652 (0.0007) [2023-12-27 05:41:09,707][20381] Updated weights for policy 1, policy_version 2627 (0.0007) [2023-12-27 05:41:09,769][20381] Updated weights for policy 1, policy_version 2637 (0.0005) [2023-12-27 05:41:09,832][20381] Updated weights for policy 1, policy_version 2647 (0.0006) [2023-12-27 05:41:10,026][20347] Updated weights for policy 0, policy_version 2662 (0.0007) [2023-12-27 05:41:10,087][20347] Updated weights for policy 0, policy_version 2672 (0.0009) [2023-12-27 05:41:10,144][20347] Updated weights for policy 0, policy_version 2682 (0.0008) [2023-12-27 05:41:10,406][20381] Updated weights for policy 1, policy_version 2657 (0.0008) [2023-12-27 05:41:10,465][20381] Updated weights for policy 1, policy_version 2667 (0.0008) [2023-12-27 05:41:10,520][20381] Updated weights for policy 1, policy_version 2677 (0.0007) [2023-12-27 05:41:10,576][20381] Updated weights for policy 1, policy_version 2687 (0.0006) [2023-12-27 05:41:10,835][20347] Updated weights for policy 0, policy_version 2692 (0.0009) [2023-12-27 05:41:10,893][20347] Updated weights for policy 0, policy_version 2702 (0.0009) [2023-12-27 05:41:10,945][20347] Updated weights for policy 0, policy_version 2712 (0.0007) [2023-12-27 05:41:11,242][20381] Updated weights for policy 1, policy_version 2697 (0.0008) [2023-12-27 05:41:11,305][20381] Updated weights for policy 1, policy_version 2707 (0.0009) [2023-12-27 05:41:11,372][20381] Updated weights for policy 1, policy_version 2717 (0.0007) [2023-12-27 05:41:11,686][20347] Updated weights for policy 0, policy_version 2722 (0.0007) [2023-12-27 05:41:11,745][20347] Updated weights for policy 0, policy_version 2732 (0.0007) [2023-12-27 05:41:11,812][20347] Updated weights for policy 0, policy_version 2742 (0.0008) [2023-12-27 05:41:11,869][20347] Updated weights for policy 0, policy_version 2752 (0.0007) [2023-12-27 05:41:12,080][20381] Updated weights for policy 1, policy_version 2727 (0.0007) [2023-12-27 05:41:12,141][20381] Updated weights for policy 1, policy_version 2737 (0.0008) [2023-12-27 05:41:12,194][20381] Updated weights for policy 1, policy_version 2747 (0.0008) [2023-12-27 05:41:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20128.9). Total num frames: 1409024. Throughput: 0: 10574.8, 1: 10470.6. Samples: 1397548. Policy #0 lag: (min: 15.0, avg: 16.8, max: 47.0) [2023-12-27 05:41:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:12,522][20347] Updated weights for policy 0, policy_version 2762 (0.0009) [2023-12-27 05:41:12,573][20347] Updated weights for policy 0, policy_version 2772 (0.0008) [2023-12-27 05:41:12,624][20347] Updated weights for policy 0, policy_version 2782 (0.0007) [2023-12-27 05:41:12,879][20381] Updated weights for policy 1, policy_version 2757 (0.0007) [2023-12-27 05:41:12,941][20381] Updated weights for policy 1, policy_version 2767 (0.0008) [2023-12-27 05:41:12,999][20381] Updated weights for policy 1, policy_version 2777 (0.0007) [2023-12-27 05:41:13,311][20347] Updated weights for policy 0, policy_version 2792 (0.0006) [2023-12-27 05:41:13,362][20347] Updated weights for policy 0, policy_version 2802 (0.0007) [2023-12-27 05:41:13,410][20347] Updated weights for policy 0, policy_version 2812 (0.0008) [2023-12-27 05:41:13,654][20381] Updated weights for policy 1, policy_version 2787 (0.0007) [2023-12-27 05:41:13,701][20381] Updated weights for policy 1, policy_version 2797 (0.0005) [2023-12-27 05:41:13,755][20381] Updated weights for policy 1, policy_version 2807 (0.0005) [2023-12-27 05:41:14,081][20347] Updated weights for policy 0, policy_version 2822 (0.0006) [2023-12-27 05:41:14,147][20347] Updated weights for policy 0, policy_version 2832 (0.0008) [2023-12-27 05:41:14,204][20347] Updated weights for policy 0, policy_version 2842 (0.0008) [2023-12-27 05:41:14,356][20381] Updated weights for policy 1, policy_version 2817 (0.0006) [2023-12-27 05:41:14,408][20381] Updated weights for policy 1, policy_version 2827 (0.0010) [2023-12-27 05:41:14,471][20381] Updated weights for policy 1, policy_version 2837 (0.0010) [2023-12-27 05:41:14,531][20381] Updated weights for policy 1, policy_version 2847 (0.0008) [2023-12-27 05:41:14,898][20347] Updated weights for policy 0, policy_version 2852 (0.0007) [2023-12-27 05:41:14,961][20347] Updated weights for policy 0, policy_version 2862 (0.0007) [2023-12-27 05:41:15,025][20347] Updated weights for policy 0, policy_version 2872 (0.0006) [2023-12-27 05:41:15,216][20381] Updated weights for policy 1, policy_version 2857 (0.0011) [2023-12-27 05:41:15,276][20381] Updated weights for policy 1, policy_version 2867 (0.0011) [2023-12-27 05:41:15,339][20381] Updated weights for policy 1, policy_version 2877 (0.0007) [2023-12-27 05:41:15,622][20347] Updated weights for policy 0, policy_version 2882 (0.0007) [2023-12-27 05:41:15,681][20347] Updated weights for policy 0, policy_version 2892 (0.0005) [2023-12-27 05:41:15,740][20347] Updated weights for policy 0, policy_version 2902 (0.0006) [2023-12-27 05:41:15,797][20347] Updated weights for policy 0, policy_version 2912 (0.0006) [2023-12-27 05:41:15,969][20381] Updated weights for policy 1, policy_version 2887 (0.0009) [2023-12-27 05:41:16,029][20381] Updated weights for policy 1, policy_version 2897 (0.0008) [2023-12-27 05:41:16,082][20381] Updated weights for policy 1, policy_version 2907 (0.0006) [2023-12-27 05:41:16,431][20347] Updated weights for policy 0, policy_version 2922 (0.0007) [2023-12-27 05:41:16,473][20347] Updated weights for policy 0, policy_version 2932 (0.0008) [2023-12-27 05:41:16,525][20347] Updated weights for policy 0, policy_version 2942 (0.0008) [2023-12-27 05:41:16,703][20381] Updated weights for policy 1, policy_version 2917 (0.0010) [2023-12-27 05:41:16,748][20381] Updated weights for policy 1, policy_version 2927 (0.0010) [2023-12-27 05:41:16,802][20381] Updated weights for policy 1, policy_version 2937 (0.0009) [2023-12-27 05:41:17,247][20347] Updated weights for policy 0, policy_version 2952 (0.0007) [2023-12-27 05:41:17,308][20347] Updated weights for policy 0, policy_version 2962 (0.0007) [2023-12-27 05:41:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20097.7). Total num frames: 1507328. Throughput: 0: 10495.5, 1: 10425.5. Samples: 1521876. Policy #0 lag: (min: 15.0, avg: 19.2, max: 47.0) [2023-12-27 05:41:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:17,368][20347] Updated weights for policy 0, policy_version 2972 (0.0007) [2023-12-27 05:41:17,482][20381] Updated weights for policy 1, policy_version 2947 (0.0010) [2023-12-27 05:41:17,538][20381] Updated weights for policy 1, policy_version 2957 (0.0010) [2023-12-27 05:41:17,593][20381] Updated weights for policy 1, policy_version 2967 (0.0008) [2023-12-27 05:41:17,999][20347] Updated weights for policy 0, policy_version 2982 (0.0008) [2023-12-27 05:41:18,055][20347] Updated weights for policy 0, policy_version 2992 (0.0006) [2023-12-27 05:41:18,112][20347] Updated weights for policy 0, policy_version 3002 (0.0007) [2023-12-27 05:41:18,282][20381] Updated weights for policy 1, policy_version 2977 (0.0008) [2023-12-27 05:41:18,340][20381] Updated weights for policy 1, policy_version 2987 (0.0008) [2023-12-27 05:41:18,389][20381] Updated weights for policy 1, policy_version 2997 (0.0007) [2023-12-27 05:41:18,448][20381] Updated weights for policy 1, policy_version 3007 (0.0007) [2023-12-27 05:41:18,841][20347] Updated weights for policy 0, policy_version 3012 (0.0008) [2023-12-27 05:41:18,912][20347] Updated weights for policy 0, policy_version 3022 (0.0007) [2023-12-27 05:41:18,975][20347] Updated weights for policy 0, policy_version 3032 (0.0007) [2023-12-27 05:41:19,079][20381] Updated weights for policy 1, policy_version 3017 (0.0006) [2023-12-27 05:41:19,134][20381] Updated weights for policy 1, policy_version 3027 (0.0005) [2023-12-27 05:41:19,179][20381] Updated weights for policy 1, policy_version 3037 (0.0005) [2023-12-27 05:41:19,698][20347] Updated weights for policy 0, policy_version 3042 (0.0006) [2023-12-27 05:41:19,754][20347] Updated weights for policy 0, policy_version 3052 (0.0007) [2023-12-27 05:41:19,812][20347] Updated weights for policy 0, policy_version 3062 (0.0006) [2023-12-27 05:41:19,879][20347] Updated weights for policy 0, policy_version 3072 (0.0007) [2023-12-27 05:41:19,884][20381] Updated weights for policy 1, policy_version 3047 (0.0007) [2023-12-27 05:41:19,945][20381] Updated weights for policy 1, policy_version 3057 (0.0008) [2023-12-27 05:41:19,999][20381] Updated weights for policy 1, policy_version 3067 (0.0008) [2023-12-27 05:41:20,621][20381] Updated weights for policy 1, policy_version 3077 (0.0007) [2023-12-27 05:41:20,638][20347] Updated weights for policy 0, policy_version 3082 (0.0007) [2023-12-27 05:41:20,681][20381] Updated weights for policy 1, policy_version 3087 (0.0006) [2023-12-27 05:41:20,695][20347] Updated weights for policy 0, policy_version 3092 (0.0006) [2023-12-27 05:41:20,744][20381] Updated weights for policy 1, policy_version 3097 (0.0008) [2023-12-27 05:41:20,753][20347] Updated weights for policy 0, policy_version 3102 (0.0009) [2023-12-27 05:41:21,402][20381] Updated weights for policy 1, policy_version 3107 (0.0006) [2023-12-27 05:41:21,423][20347] Updated weights for policy 0, policy_version 3112 (0.0007) [2023-12-27 05:41:21,461][20381] Updated weights for policy 1, policy_version 3117 (0.0007) [2023-12-27 05:41:21,482][20347] Updated weights for policy 0, policy_version 3122 (0.0008) [2023-12-27 05:41:21,520][20381] Updated weights for policy 1, policy_version 3127 (0.0006) [2023-12-27 05:41:21,534][20347] Updated weights for policy 0, policy_version 3132 (0.0008) [2023-12-27 05:41:22,231][20381] Updated weights for policy 1, policy_version 3137 (0.0010) [2023-12-27 05:41:22,245][20347] Updated weights for policy 0, policy_version 3142 (0.0007) [2023-12-27 05:41:22,291][20381] Updated weights for policy 1, policy_version 3147 (0.0011) [2023-12-27 05:41:22,308][20347] Updated weights for policy 0, policy_version 3152 (0.0007) [2023-12-27 05:41:22,354][20381] Updated weights for policy 1, policy_version 3157 (0.0010) [2023-12-27 05:41:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.1, 300 sec: 20070.4). Total num frames: 1605632. Throughput: 0: 10418.6, 1: 10411.2. Samples: 1582700. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 05:41:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:22,372][20347] Updated weights for policy 0, policy_version 3162 (0.0009) [2023-12-27 05:41:22,404][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000003168_811008.pth... [2023-12-27 05:41:22,408][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000001920_491520.pth [2023-12-27 05:41:22,409][20381] Updated weights for policy 1, policy_version 3167 (0.0011) [2023-12-27 05:41:22,413][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000003168_811008.pth... [2023-12-27 05:41:22,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000001952_499712.pth [2023-12-27 05:41:23,080][20347] Updated weights for policy 0, policy_version 3172 (0.0006) [2023-12-27 05:41:23,108][20381] Updated weights for policy 1, policy_version 3177 (0.0007) [2023-12-27 05:41:23,130][20347] Updated weights for policy 0, policy_version 3182 (0.0007) [2023-12-27 05:41:23,163][20381] Updated weights for policy 1, policy_version 3187 (0.0006) [2023-12-27 05:41:23,178][20347] Updated weights for policy 0, policy_version 3192 (0.0007) [2023-12-27 05:41:23,226][20381] Updated weights for policy 1, policy_version 3197 (0.0006) [2023-12-27 05:41:23,779][20347] Updated weights for policy 0, policy_version 3202 (0.0007) [2023-12-27 05:41:23,832][20347] Updated weights for policy 0, policy_version 3212 (0.0007) [2023-12-27 05:41:23,873][20381] Updated weights for policy 1, policy_version 3207 (0.0010) [2023-12-27 05:41:23,884][20347] Updated weights for policy 0, policy_version 3222 (0.0007) [2023-12-27 05:41:23,933][20381] Updated weights for policy 1, policy_version 3217 (0.0010) [2023-12-27 05:41:23,938][20347] Updated weights for policy 0, policy_version 3232 (0.0007) [2023-12-27 05:41:23,987][20381] Updated weights for policy 1, policy_version 3227 (0.0005) [2023-12-27 05:41:24,545][20347] Updated weights for policy 0, policy_version 3242 (0.0010) [2023-12-27 05:41:24,562][20381] Updated weights for policy 1, policy_version 3237 (0.0008) [2023-12-27 05:41:24,598][20347] Updated weights for policy 0, policy_version 3252 (0.0009) [2023-12-27 05:41:24,618][20381] Updated weights for policy 1, policy_version 3247 (0.0010) [2023-12-27 05:41:24,656][20347] Updated weights for policy 0, policy_version 3262 (0.0010) [2023-12-27 05:41:24,670][20381] Updated weights for policy 1, policy_version 3257 (0.0010) [2023-12-27 05:41:25,298][20347] Updated weights for policy 0, policy_version 3272 (0.0006) [2023-12-27 05:41:25,323][20381] Updated weights for policy 1, policy_version 3267 (0.0010) [2023-12-27 05:41:25,354][20347] Updated weights for policy 0, policy_version 3282 (0.0005) [2023-12-27 05:41:25,379][20381] Updated weights for policy 1, policy_version 3277 (0.0005) [2023-12-27 05:41:25,410][20347] Updated weights for policy 0, policy_version 3292 (0.0005) [2023-12-27 05:41:25,439][20381] Updated weights for policy 1, policy_version 3287 (0.0005) [2023-12-27 05:41:26,023][20347] Updated weights for policy 0, policy_version 3302 (0.0008) [2023-12-27 05:41:26,079][20381] Updated weights for policy 1, policy_version 3297 (0.0007) [2023-12-27 05:41:26,081][20347] Updated weights for policy 0, policy_version 3312 (0.0010) [2023-12-27 05:41:26,133][20347] Updated weights for policy 0, policy_version 3322 (0.0010) [2023-12-27 05:41:26,137][20381] Updated weights for policy 1, policy_version 3307 (0.0011) [2023-12-27 05:41:26,194][20381] Updated weights for policy 1, policy_version 3317 (0.0010) [2023-12-27 05:41:26,251][20381] Updated weights for policy 1, policy_version 3327 (0.0010) [2023-12-27 05:41:26,807][20347] Updated weights for policy 0, policy_version 3332 (0.0009) [2023-12-27 05:41:26,853][20347] Updated weights for policy 0, policy_version 3342 (0.0007) [2023-12-27 05:41:26,898][20347] Updated weights for policy 0, policy_version 3352 (0.0006) [2023-12-27 05:41:26,929][20381] Updated weights for policy 1, policy_version 3337 (0.0010) [2023-12-27 05:41:26,982][20381] Updated weights for policy 1, policy_version 3347 (0.0010) [2023-12-27 05:41:27,036][20381] Updated weights for policy 1, policy_version 3357 (0.0010) [2023-12-27 05:41:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20239.0). Total num frames: 1720320. Throughput: 0: 10453.4, 1: 10399.5. Samples: 1708456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:41:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:27,545][20347] Updated weights for policy 0, policy_version 3362 (0.0005) [2023-12-27 05:41:27,582][20381] Updated weights for policy 1, policy_version 3367 (0.0008) [2023-12-27 05:41:27,602][20347] Updated weights for policy 0, policy_version 3372 (0.0007) [2023-12-27 05:41:27,630][20381] Updated weights for policy 1, policy_version 3377 (0.0005) [2023-12-27 05:41:27,660][20347] Updated weights for policy 0, policy_version 3382 (0.0008) [2023-12-27 05:41:27,683][20381] Updated weights for policy 1, policy_version 3387 (0.0006) [2023-12-27 05:41:27,714][20347] Updated weights for policy 0, policy_version 3392 (0.0006) [2023-12-27 05:41:28,248][20381] Updated weights for policy 1, policy_version 3397 (0.0009) [2023-12-27 05:41:28,296][20381] Updated weights for policy 1, policy_version 3407 (0.0006) [2023-12-27 05:41:28,351][20381] Updated weights for policy 1, policy_version 3417 (0.0007) [2023-12-27 05:41:28,437][20347] Updated weights for policy 0, policy_version 3402 (0.0007) [2023-12-27 05:41:28,495][20347] Updated weights for policy 0, policy_version 3412 (0.0006) [2023-12-27 05:41:28,545][20347] Updated weights for policy 0, policy_version 3422 (0.0008) [2023-12-27 05:41:28,962][20381] Updated weights for policy 1, policy_version 3427 (0.0009) [2023-12-27 05:41:29,017][20381] Updated weights for policy 1, policy_version 3437 (0.0010) [2023-12-27 05:41:29,068][20381] Updated weights for policy 1, policy_version 3447 (0.0010) [2023-12-27 05:41:29,164][20347] Updated weights for policy 0, policy_version 3432 (0.0010) [2023-12-27 05:41:29,215][20347] Updated weights for policy 0, policy_version 3442 (0.0010) [2023-12-27 05:41:29,273][20347] Updated weights for policy 0, policy_version 3452 (0.0010) [2023-12-27 05:41:29,733][20381] Updated weights for policy 1, policy_version 3457 (0.0010) [2023-12-27 05:41:29,783][20381] Updated weights for policy 1, policy_version 3467 (0.0005) [2023-12-27 05:41:29,842][20381] Updated weights for policy 1, policy_version 3477 (0.0008) [2023-12-27 05:41:29,901][20381] Updated weights for policy 1, policy_version 3487 (0.0010) [2023-12-27 05:41:29,940][20347] Updated weights for policy 0, policy_version 3462 (0.0010) [2023-12-27 05:41:29,997][20347] Updated weights for policy 0, policy_version 3472 (0.0009) [2023-12-27 05:41:30,053][20347] Updated weights for policy 0, policy_version 3482 (0.0005) [2023-12-27 05:41:30,532][20381] Updated weights for policy 1, policy_version 3497 (0.0006) [2023-12-27 05:41:30,582][20381] Updated weights for policy 1, policy_version 3507 (0.0006) [2023-12-27 05:41:30,610][20347] Updated weights for policy 0, policy_version 3492 (0.0005) [2023-12-27 05:41:30,628][20381] Updated weights for policy 1, policy_version 3517 (0.0009) [2023-12-27 05:41:30,661][20347] Updated weights for policy 0, policy_version 3502 (0.0005) [2023-12-27 05:41:30,712][20347] Updated weights for policy 0, policy_version 3512 (0.0006) [2023-12-27 05:41:31,230][20381] Updated weights for policy 1, policy_version 3527 (0.0010) [2023-12-27 05:41:31,289][20381] Updated weights for policy 1, policy_version 3537 (0.0011) [2023-12-27 05:41:31,342][20381] Updated weights for policy 1, policy_version 3547 (0.0010) [2023-12-27 05:41:31,375][20347] Updated weights for policy 0, policy_version 3522 (0.0010) [2023-12-27 05:41:31,426][20347] Updated weights for policy 0, policy_version 3532 (0.0005) [2023-12-27 05:41:31,485][20347] Updated weights for policy 0, policy_version 3542 (0.0006) [2023-12-27 05:41:31,546][20347] Updated weights for policy 0, policy_version 3552 (0.0008) [2023-12-27 05:41:32,028][20381] Updated weights for policy 1, policy_version 3557 (0.0006) [2023-12-27 05:41:32,080][20381] Updated weights for policy 1, policy_version 3567 (0.0007) [2023-12-27 05:41:32,129][20381] Updated weights for policy 1, policy_version 3577 (0.0006) [2023-12-27 05:41:32,246][20347] Updated weights for policy 0, policy_version 3562 (0.0008) [2023-12-27 05:41:32,312][20347] Updated weights for policy 0, policy_version 3572 (0.0008) [2023-12-27 05:41:32,362][19170] Fps is (10 sec: 22118.8, 60 sec: 20889.6, 300 sec: 20298.0). Total num frames: 1826816. Throughput: 0: 10483.7, 1: 10488.6. Samples: 1838612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:41:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:32,375][20347] Updated weights for policy 0, policy_version 3582 (0.0007) [2023-12-27 05:41:32,753][20381] Updated weights for policy 1, policy_version 3587 (0.0006) [2023-12-27 05:41:32,807][20381] Updated weights for policy 1, policy_version 3597 (0.0005) [2023-12-27 05:41:32,865][20381] Updated weights for policy 1, policy_version 3607 (0.0005) [2023-12-27 05:41:33,071][20347] Updated weights for policy 0, policy_version 3592 (0.0007) [2023-12-27 05:41:33,127][20347] Updated weights for policy 0, policy_version 3602 (0.0007) [2023-12-27 05:41:33,175][20347] Updated weights for policy 0, policy_version 3612 (0.0008) [2023-12-27 05:41:33,464][20381] Updated weights for policy 1, policy_version 3617 (0.0006) [2023-12-27 05:41:33,518][20381] Updated weights for policy 1, policy_version 3627 (0.0006) [2023-12-27 05:41:33,579][20381] Updated weights for policy 1, policy_version 3637 (0.0005) [2023-12-27 05:41:33,637][20381] Updated weights for policy 1, policy_version 3647 (0.0006) [2023-12-27 05:41:33,857][20347] Updated weights for policy 0, policy_version 3622 (0.0008) [2023-12-27 05:41:33,902][20347] Updated weights for policy 0, policy_version 3632 (0.0008) [2023-12-27 05:41:33,950][20347] Updated weights for policy 0, policy_version 3642 (0.0007) [2023-12-27 05:41:34,221][20381] Updated weights for policy 1, policy_version 3657 (0.0008) [2023-12-27 05:41:34,282][20381] Updated weights for policy 1, policy_version 3667 (0.0007) [2023-12-27 05:41:34,340][20381] Updated weights for policy 1, policy_version 3677 (0.0005) [2023-12-27 05:41:34,663][20347] Updated weights for policy 0, policy_version 3652 (0.0007) [2023-12-27 05:41:34,706][20347] Updated weights for policy 0, policy_version 3662 (0.0007) [2023-12-27 05:41:34,764][20347] Updated weights for policy 0, policy_version 3672 (0.0005) [2023-12-27 05:41:35,002][20381] Updated weights for policy 1, policy_version 3687 (0.0007) [2023-12-27 05:41:35,053][20381] Updated weights for policy 1, policy_version 3697 (0.0007) [2023-12-27 05:41:35,105][20381] Updated weights for policy 1, policy_version 3707 (0.0007) [2023-12-27 05:41:35,391][20347] Updated weights for policy 0, policy_version 3682 (0.0005) [2023-12-27 05:41:35,449][20347] Updated weights for policy 0, policy_version 3692 (0.0007) [2023-12-27 05:41:35,501][20347] Updated weights for policy 0, policy_version 3702 (0.0009) [2023-12-27 05:41:35,549][20347] Updated weights for policy 0, policy_version 3712 (0.0007) [2023-12-27 05:41:35,764][20381] Updated weights for policy 1, policy_version 3717 (0.0007) [2023-12-27 05:41:35,814][20381] Updated weights for policy 1, policy_version 3727 (0.0007) [2023-12-27 05:41:35,866][20381] Updated weights for policy 1, policy_version 3737 (0.0005) [2023-12-27 05:41:36,257][20347] Updated weights for policy 0, policy_version 3722 (0.0007) [2023-12-27 05:41:36,323][20347] Updated weights for policy 0, policy_version 3732 (0.0008) [2023-12-27 05:41:36,388][20347] Updated weights for policy 0, policy_version 3742 (0.0008) [2023-12-27 05:41:36,489][20381] Updated weights for policy 1, policy_version 3747 (0.0005) [2023-12-27 05:41:36,553][20381] Updated weights for policy 1, policy_version 3757 (0.0006) [2023-12-27 05:41:36,628][20381] Updated weights for policy 1, policy_version 3767 (0.0007) [2023-12-27 05:41:37,079][20347] Updated weights for policy 0, policy_version 3752 (0.0009) [2023-12-27 05:41:37,131][20347] Updated weights for policy 0, policy_version 3762 (0.0008) [2023-12-27 05:41:37,186][20347] Updated weights for policy 0, policy_version 3772 (0.0008) [2023-12-27 05:41:37,192][20381] Updated weights for policy 1, policy_version 3777 (0.0006) [2023-12-27 05:41:37,247][20381] Updated weights for policy 1, policy_version 3787 (0.0006) [2023-12-27 05:41:37,303][20381] Updated weights for policy 1, policy_version 3797 (0.0006) [2023-12-27 05:41:37,359][20381] Updated weights for policy 1, policy_version 3807 (0.0007) [2023-12-27 05:41:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20350.6). Total num frames: 1933312. Throughput: 0: 10479.9, 1: 10550.7. Samples: 1901788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:41:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000003808_974848.pth... [2023-12-27 05:41:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000003776_966656.pth... [2023-12-27 05:41:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000002560_655360.pth [2023-12-27 05:41:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000002560_655360.pth [2023-12-27 05:41:37,923][20347] Updated weights for policy 0, policy_version 3782 (0.0007) [2023-12-27 05:41:37,978][20347] Updated weights for policy 0, policy_version 3792 (0.0008) [2023-12-27 05:41:37,989][20381] Updated weights for policy 1, policy_version 3817 (0.0008) [2023-12-27 05:41:38,036][20347] Updated weights for policy 0, policy_version 3802 (0.0007) [2023-12-27 05:41:38,049][20381] Updated weights for policy 1, policy_version 3827 (0.0008) [2023-12-27 05:41:38,096][20381] Updated weights for policy 1, policy_version 3837 (0.0007) [2023-12-27 05:41:38,663][20381] Updated weights for policy 1, policy_version 3847 (0.0006) [2023-12-27 05:41:38,715][20381] Updated weights for policy 1, policy_version 3857 (0.0005) [2023-12-27 05:41:38,734][20347] Updated weights for policy 0, policy_version 3812 (0.0006) [2023-12-27 05:41:38,761][20381] Updated weights for policy 1, policy_version 3867 (0.0005) [2023-12-27 05:41:38,791][20347] Updated weights for policy 0, policy_version 3822 (0.0007) [2023-12-27 05:41:38,856][20347] Updated weights for policy 0, policy_version 3832 (0.0007) [2023-12-27 05:41:39,370][20381] Updated weights for policy 1, policy_version 3877 (0.0006) [2023-12-27 05:41:39,431][20381] Updated weights for policy 1, policy_version 3887 (0.0007) [2023-12-27 05:41:39,486][20381] Updated weights for policy 1, policy_version 3897 (0.0005) [2023-12-27 05:41:39,589][20347] Updated weights for policy 0, policy_version 3842 (0.0008) [2023-12-27 05:41:39,640][20347] Updated weights for policy 0, policy_version 3852 (0.0009) [2023-12-27 05:41:39,696][20347] Updated weights for policy 0, policy_version 3862 (0.0008) [2023-12-27 05:41:39,752][20347] Updated weights for policy 0, policy_version 3872 (0.0008) [2023-12-27 05:41:40,122][20381] Updated weights for policy 1, policy_version 3907 (0.0006) [2023-12-27 05:41:40,182][20381] Updated weights for policy 1, policy_version 3917 (0.0008) [2023-12-27 05:41:40,234][20381] Updated weights for policy 1, policy_version 3927 (0.0008) [2023-12-27 05:41:40,506][20347] Updated weights for policy 0, policy_version 3882 (0.0006) [2023-12-27 05:41:40,564][20347] Updated weights for policy 0, policy_version 3892 (0.0006) [2023-12-27 05:41:40,619][20347] Updated weights for policy 0, policy_version 3902 (0.0006) [2023-12-27 05:41:40,942][20381] Updated weights for policy 1, policy_version 3937 (0.0008) [2023-12-27 05:41:40,997][20381] Updated weights for policy 1, policy_version 3947 (0.0005) [2023-12-27 05:41:41,055][20381] Updated weights for policy 1, policy_version 3957 (0.0006) [2023-12-27 05:41:41,111][20381] Updated weights for policy 1, policy_version 3967 (0.0008) [2023-12-27 05:41:41,343][20347] Updated weights for policy 0, policy_version 3913 (0.0008) [2023-12-27 05:41:41,404][20347] Updated weights for policy 0, policy_version 3923 (0.0008) [2023-12-27 05:41:41,465][20347] Updated weights for policy 0, policy_version 3933 (0.0007) [2023-12-27 05:41:41,868][20381] Updated weights for policy 1, policy_version 3977 (0.0009) [2023-12-27 05:41:41,922][20381] Updated weights for policy 1, policy_version 3987 (0.0009) [2023-12-27 05:41:41,977][20381] Updated weights for policy 1, policy_version 3997 (0.0008) [2023-12-27 05:41:42,110][20347] Updated weights for policy 0, policy_version 3943 (0.0007) [2023-12-27 05:41:42,172][20347] Updated weights for policy 0, policy_version 3953 (0.0007) [2023-12-27 05:41:42,217][20347] Updated weights for policy 0, policy_version 3963 (0.0008) [2023-12-27 05:41:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20398.1). Total num frames: 2039808. Throughput: 0: 10367.6, 1: 10636.0. Samples: 2026564. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 05:41:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:42,736][20381] Updated weights for policy 1, policy_version 4007 (0.0008) [2023-12-27 05:41:42,798][20381] Updated weights for policy 1, policy_version 4017 (0.0008) [2023-12-27 05:41:42,855][20381] Updated weights for policy 1, policy_version 4027 (0.0008) [2023-12-27 05:41:42,920][20347] Updated weights for policy 0, policy_version 3973 (0.0008) [2023-12-27 05:41:42,978][20347] Updated weights for policy 0, policy_version 3983 (0.0007) [2023-12-27 05:41:43,033][20347] Updated weights for policy 0, policy_version 3993 (0.0008) [2023-12-27 05:41:43,568][20381] Updated weights for policy 1, policy_version 4037 (0.0008) [2023-12-27 05:41:43,620][20381] Updated weights for policy 1, policy_version 4047 (0.0008) [2023-12-27 05:41:43,663][20347] Updated weights for policy 0, policy_version 4003 (0.0007) [2023-12-27 05:41:43,673][20381] Updated weights for policy 1, policy_version 4057 (0.0007) [2023-12-27 05:41:43,721][20347] Updated weights for policy 0, policy_version 4013 (0.0006) [2023-12-27 05:41:43,776][20347] Updated weights for policy 0, policy_version 4023 (0.0006) [2023-12-27 05:41:44,379][20381] Updated weights for policy 1, policy_version 4067 (0.0008) [2023-12-27 05:41:44,384][20347] Updated weights for policy 0, policy_version 4033 (0.0007) [2023-12-27 05:41:44,434][20347] Updated weights for policy 0, policy_version 4043 (0.0008) [2023-12-27 05:41:44,434][20381] Updated weights for policy 1, policy_version 4077 (0.0007) [2023-12-27 05:41:44,491][20347] Updated weights for policy 0, policy_version 4053 (0.0007) [2023-12-27 05:41:44,493][20381] Updated weights for policy 1, policy_version 4087 (0.0007) [2023-12-27 05:41:44,542][20347] Updated weights for policy 0, policy_version 4063 (0.0010) [2023-12-27 05:41:45,157][20381] Updated weights for policy 1, policy_version 4097 (0.0005) [2023-12-27 05:41:45,220][20381] Updated weights for policy 1, policy_version 4107 (0.0006) [2023-12-27 05:41:45,253][20347] Updated weights for policy 0, policy_version 4073 (0.0010) [2023-12-27 05:41:45,281][20381] Updated weights for policy 1, policy_version 4117 (0.0007) [2023-12-27 05:41:45,322][20347] Updated weights for policy 0, policy_version 4083 (0.0011) [2023-12-27 05:41:45,338][20381] Updated weights for policy 1, policy_version 4127 (0.0007) [2023-12-27 05:41:45,395][20347] Updated weights for policy 0, policy_version 4093 (0.0009) [2023-12-27 05:41:45,917][20381] Updated weights for policy 1, policy_version 4137 (0.0009) [2023-12-27 05:41:45,968][20381] Updated weights for policy 1, policy_version 4147 (0.0008) [2023-12-27 05:41:46,016][20347] Updated weights for policy 0, policy_version 4103 (0.0010) [2023-12-27 05:41:46,016][20381] Updated weights for policy 1, policy_version 4157 (0.0007) [2023-12-27 05:41:46,069][20347] Updated weights for policy 0, policy_version 4113 (0.0010) [2023-12-27 05:41:46,121][20347] Updated weights for policy 0, policy_version 4123 (0.0005) [2023-12-27 05:41:46,684][20381] Updated weights for policy 1, policy_version 4167 (0.0006) [2023-12-27 05:41:46,747][20381] Updated weights for policy 1, policy_version 4177 (0.0006) [2023-12-27 05:41:46,809][20381] Updated weights for policy 1, policy_version 4187 (0.0007) [2023-12-27 05:41:46,828][20347] Updated weights for policy 0, policy_version 4133 (0.0007) [2023-12-27 05:41:46,884][20347] Updated weights for policy 0, policy_version 4143 (0.0007) [2023-12-27 05:41:46,946][20347] Updated weights for policy 0, policy_version 4153 (0.0008) [2023-12-27 05:41:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20363.0). Total num frames: 2138112. Throughput: 0: 10376.3, 1: 10592.8. Samples: 2150156. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 05:41:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:47,410][20381] Updated weights for policy 1, policy_version 4197 (0.0007) [2023-12-27 05:41:47,470][20381] Updated weights for policy 1, policy_version 4207 (0.0007) [2023-12-27 05:41:47,528][20381] Updated weights for policy 1, policy_version 4217 (0.0007) [2023-12-27 05:41:47,663][20347] Updated weights for policy 0, policy_version 4163 (0.0008) [2023-12-27 05:41:47,723][20347] Updated weights for policy 0, policy_version 4173 (0.0007) [2023-12-27 05:41:47,787][20347] Updated weights for policy 0, policy_version 4183 (0.0008) [2023-12-27 05:41:48,156][20381] Updated weights for policy 1, policy_version 4227 (0.0008) [2023-12-27 05:41:48,209][20381] Updated weights for policy 1, policy_version 4237 (0.0010) [2023-12-27 05:41:48,263][20381] Updated weights for policy 1, policy_version 4247 (0.0010) [2023-12-27 05:41:48,378][20347] Updated weights for policy 0, policy_version 4193 (0.0006) [2023-12-27 05:41:48,437][20347] Updated weights for policy 0, policy_version 4203 (0.0007) [2023-12-27 05:41:48,489][20347] Updated weights for policy 0, policy_version 4213 (0.0008) [2023-12-27 05:41:48,538][20347] Updated weights for policy 0, policy_version 4223 (0.0007) [2023-12-27 05:41:48,952][20381] Updated weights for policy 1, policy_version 4257 (0.0010) [2023-12-27 05:41:49,019][20381] Updated weights for policy 1, policy_version 4267 (0.0006) [2023-12-27 05:41:49,080][20381] Updated weights for policy 1, policy_version 4277 (0.0011) [2023-12-27 05:41:49,135][20381] Updated weights for policy 1, policy_version 4287 (0.0008) [2023-12-27 05:41:49,275][20347] Updated weights for policy 0, policy_version 4233 (0.0010) [2023-12-27 05:41:49,328][20347] Updated weights for policy 0, policy_version 4243 (0.0009) [2023-12-27 05:41:49,386][20347] Updated weights for policy 0, policy_version 4253 (0.0007) [2023-12-27 05:41:49,686][20381] Updated weights for policy 1, policy_version 4297 (0.0009) [2023-12-27 05:41:49,748][20381] Updated weights for policy 1, policy_version 4307 (0.0011) [2023-12-27 05:41:49,800][20381] Updated weights for policy 1, policy_version 4317 (0.0010) [2023-12-27 05:41:50,081][20347] Updated weights for policy 0, policy_version 4263 (0.0008) [2023-12-27 05:41:50,140][20347] Updated weights for policy 0, policy_version 4273 (0.0008) [2023-12-27 05:41:50,194][20347] Updated weights for policy 0, policy_version 4283 (0.0008) [2023-12-27 05:41:50,494][20381] Updated weights for policy 1, policy_version 4327 (0.0011) [2023-12-27 05:41:50,556][20381] Updated weights for policy 1, policy_version 4337 (0.0009) [2023-12-27 05:41:50,617][20381] Updated weights for policy 1, policy_version 4347 (0.0008) [2023-12-27 05:41:50,838][20347] Updated weights for policy 0, policy_version 4293 (0.0008) [2023-12-27 05:41:50,890][20347] Updated weights for policy 0, policy_version 4303 (0.0008) [2023-12-27 05:41:50,941][20347] Updated weights for policy 0, policy_version 4313 (0.0007) [2023-12-27 05:41:51,298][20381] Updated weights for policy 1, policy_version 4357 (0.0009) [2023-12-27 05:41:51,356][20381] Updated weights for policy 1, policy_version 4367 (0.0011) [2023-12-27 05:41:51,416][20381] Updated weights for policy 1, policy_version 4377 (0.0011) [2023-12-27 05:41:51,701][20347] Updated weights for policy 0, policy_version 4323 (0.0007) [2023-12-27 05:41:51,766][20347] Updated weights for policy 0, policy_version 4333 (0.0007) [2023-12-27 05:41:51,811][20347] Updated weights for policy 0, policy_version 4343 (0.0007) [2023-12-27 05:41:52,176][20381] Updated weights for policy 1, policy_version 4387 (0.0011) [2023-12-27 05:41:52,235][20381] Updated weights for policy 1, policy_version 4397 (0.0011) [2023-12-27 05:41:52,295][20381] Updated weights for policy 1, policy_version 4407 (0.0009) [2023-12-27 05:41:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 21026.1, 300 sec: 20405.5). Total num frames: 2244608. Throughput: 0: 10338.9, 1: 10633.4. Samples: 2213060. Policy #0 lag: (min: 21.0, avg: 22.3, max: 51.0) [2023-12-27 05:41:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000004352_1114112.pth... [2023-12-27 05:41:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000004416_1130496.pth... [2023-12-27 05:41:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000003168_811008.pth [2023-12-27 05:41:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000003168_811008.pth [2023-12-27 05:41:52,482][20347] Updated weights for policy 0, policy_version 4353 (0.0007) [2023-12-27 05:41:52,538][20347] Updated weights for policy 0, policy_version 4363 (0.0008) [2023-12-27 05:41:52,597][20347] Updated weights for policy 0, policy_version 4373 (0.0009) [2023-12-27 05:41:52,651][20347] Updated weights for policy 0, policy_version 4383 (0.0007) [2023-12-27 05:41:52,945][20381] Updated weights for policy 1, policy_version 4417 (0.0008) [2023-12-27 05:41:52,988][20381] Updated weights for policy 1, policy_version 4427 (0.0007) [2023-12-27 05:41:53,039][20381] Updated weights for policy 1, policy_version 4437 (0.0007) [2023-12-27 05:41:53,090][20381] Updated weights for policy 1, policy_version 4447 (0.0007) [2023-12-27 05:41:53,399][20347] Updated weights for policy 0, policy_version 4393 (0.0010) [2023-12-27 05:41:53,457][20347] Updated weights for policy 0, policy_version 4403 (0.0009) [2023-12-27 05:41:53,512][20347] Updated weights for policy 0, policy_version 4413 (0.0010) [2023-12-27 05:41:53,804][20381] Updated weights for policy 1, policy_version 4457 (0.0007) [2023-12-27 05:41:53,851][20381] Updated weights for policy 1, policy_version 4467 (0.0007) [2023-12-27 05:41:53,904][20381] Updated weights for policy 1, policy_version 4477 (0.0008) [2023-12-27 05:41:54,144][20347] Updated weights for policy 0, policy_version 4423 (0.0009) [2023-12-27 05:41:54,210][20347] Updated weights for policy 0, policy_version 4433 (0.0008) [2023-12-27 05:41:54,271][20347] Updated weights for policy 0, policy_version 4443 (0.0008) [2023-12-27 05:41:54,554][20381] Updated weights for policy 1, policy_version 4487 (0.0007) [2023-12-27 05:41:54,618][20381] Updated weights for policy 1, policy_version 4497 (0.0005) [2023-12-27 05:41:54,683][20381] Updated weights for policy 1, policy_version 4507 (0.0007) [2023-12-27 05:41:54,985][20347] Updated weights for policy 0, policy_version 4453 (0.0008) [2023-12-27 05:41:55,046][20347] Updated weights for policy 0, policy_version 4463 (0.0008) [2023-12-27 05:41:55,109][20347] Updated weights for policy 0, policy_version 4473 (0.0007) [2023-12-27 05:41:55,322][20381] Updated weights for policy 1, policy_version 4517 (0.0008) [2023-12-27 05:41:55,375][20381] Updated weights for policy 1, policy_version 4527 (0.0006) [2023-12-27 05:41:55,435][20381] Updated weights for policy 1, policy_version 4537 (0.0006) [2023-12-27 05:41:55,789][20347] Updated weights for policy 0, policy_version 4483 (0.0007) [2023-12-27 05:41:55,857][20347] Updated weights for policy 0, policy_version 4493 (0.0005) [2023-12-27 05:41:55,909][20347] Updated weights for policy 0, policy_version 4503 (0.0007) [2023-12-27 05:41:56,050][20381] Updated weights for policy 1, policy_version 4547 (0.0008) [2023-12-27 05:41:56,107][20381] Updated weights for policy 1, policy_version 4557 (0.0007) [2023-12-27 05:41:56,163][20381] Updated weights for policy 1, policy_version 4567 (0.0008) [2023-12-27 05:41:56,477][20347] Updated weights for policy 0, policy_version 4513 (0.0008) [2023-12-27 05:41:56,525][20347] Updated weights for policy 0, policy_version 4523 (0.0005) [2023-12-27 05:41:56,575][20347] Updated weights for policy 0, policy_version 4533 (0.0005) [2023-12-27 05:41:56,629][20347] Updated weights for policy 0, policy_version 4543 (0.0005) [2023-12-27 05:41:56,894][20381] Updated weights for policy 1, policy_version 4577 (0.0009) [2023-12-27 05:41:56,946][20381] Updated weights for policy 1, policy_version 4587 (0.0009) [2023-12-27 05:41:57,003][20381] Updated weights for policy 1, policy_version 4597 (0.0007) [2023-12-27 05:41:57,061][20381] Updated weights for policy 1, policy_version 4607 (0.0008) [2023-12-27 05:41:57,136][20347] Updated weights for policy 0, policy_version 4553 (0.0005) [2023-12-27 05:41:57,186][20347] Updated weights for policy 0, policy_version 4563 (0.0005) [2023-12-27 05:41:57,241][20347] Updated weights for policy 0, policy_version 4573 (0.0006) [2023-12-27 05:41:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20444.4). Total num frames: 2351104. Throughput: 0: 10212.9, 1: 10614.9. Samples: 2334800. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 05:41:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:41:57,752][20381] Updated weights for policy 1, policy_version 4617 (0.0007) [2023-12-27 05:41:57,809][20381] Updated weights for policy 1, policy_version 4627 (0.0007) [2023-12-27 05:41:57,829][20347] Updated weights for policy 0, policy_version 4583 (0.0006) [2023-12-27 05:41:57,863][20381] Updated weights for policy 1, policy_version 4637 (0.0006) [2023-12-27 05:41:57,884][20347] Updated weights for policy 0, policy_version 4593 (0.0005) [2023-12-27 05:41:57,931][20347] Updated weights for policy 0, policy_version 4603 (0.0007) [2023-12-27 05:41:58,611][20381] Updated weights for policy 1, policy_version 4647 (0.0007) [2023-12-27 05:41:58,622][20347] Updated weights for policy 0, policy_version 4613 (0.0007) [2023-12-27 05:41:58,666][20381] Updated weights for policy 1, policy_version 4657 (0.0007) [2023-12-27 05:41:58,684][20347] Updated weights for policy 0, policy_version 4623 (0.0008) [2023-12-27 05:41:58,735][20381] Updated weights for policy 1, policy_version 4667 (0.0008) [2023-12-27 05:41:58,749][20347] Updated weights for policy 0, policy_version 4633 (0.0010) [2023-12-27 05:41:59,468][20381] Updated weights for policy 1, policy_version 4677 (0.0007) [2023-12-27 05:41:59,505][20347] Updated weights for policy 0, policy_version 4643 (0.0007) [2023-12-27 05:41:59,527][20381] Updated weights for policy 1, policy_version 4687 (0.0006) [2023-12-27 05:41:59,565][20347] Updated weights for policy 0, policy_version 4653 (0.0008) [2023-12-27 05:41:59,588][20381] Updated weights for policy 1, policy_version 4697 (0.0006) [2023-12-27 05:41:59,622][20347] Updated weights for policy 0, policy_version 4663 (0.0007) [2023-12-27 05:42:00,216][20381] Updated weights for policy 1, policy_version 4707 (0.0007) [2023-12-27 05:42:00,245][20347] Updated weights for policy 0, policy_version 4673 (0.0008) [2023-12-27 05:42:00,266][20381] Updated weights for policy 1, policy_version 4717 (0.0006) [2023-12-27 05:42:00,299][20347] Updated weights for policy 0, policy_version 4683 (0.0007) [2023-12-27 05:42:00,330][20381] Updated weights for policy 1, policy_version 4727 (0.0007) [2023-12-27 05:42:00,348][20347] Updated weights for policy 0, policy_version 4693 (0.0006) [2023-12-27 05:42:00,398][20347] Updated weights for policy 0, policy_version 4703 (0.0006) [2023-12-27 05:42:01,001][20381] Updated weights for policy 1, policy_version 4737 (0.0006) [2023-12-27 05:42:01,058][20381] Updated weights for policy 1, policy_version 4747 (0.0007) [2023-12-27 05:42:01,080][20347] Updated weights for policy 0, policy_version 4713 (0.0007) [2023-12-27 05:42:01,115][20381] Updated weights for policy 1, policy_version 4757 (0.0007) [2023-12-27 05:42:01,137][20347] Updated weights for policy 0, policy_version 4723 (0.0008) [2023-12-27 05:42:01,175][20381] Updated weights for policy 1, policy_version 4767 (0.0007) [2023-12-27 05:42:01,196][20347] Updated weights for policy 0, policy_version 4733 (0.0007) [2023-12-27 05:42:01,855][20381] Updated weights for policy 1, policy_version 4777 (0.0007) [2023-12-27 05:42:01,862][20347] Updated weights for policy 0, policy_version 4743 (0.0008) [2023-12-27 05:42:01,916][20347] Updated weights for policy 0, policy_version 4753 (0.0008) [2023-12-27 05:42:01,917][20381] Updated weights for policy 1, policy_version 4787 (0.0007) [2023-12-27 05:42:01,972][20347] Updated weights for policy 0, policy_version 4763 (0.0006) [2023-12-27 05:42:01,977][20381] Updated weights for policy 1, policy_version 4797 (0.0008) [2023-12-27 05:42:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20411.7). Total num frames: 2449408. Throughput: 0: 10284.7, 1: 10536.0. Samples: 2458808. Policy #0 lag: (min: 10.0, avg: 22.6, max: 42.0) [2023-12-27 05:42:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:02,631][20347] Updated weights for policy 0, policy_version 4773 (0.0006) [2023-12-27 05:42:02,687][20347] Updated weights for policy 0, policy_version 4783 (0.0007) [2023-12-27 05:42:02,688][20381] Updated weights for policy 1, policy_version 4807 (0.0006) [2023-12-27 05:42:02,744][20347] Updated weights for policy 0, policy_version 4793 (0.0007) [2023-12-27 05:42:02,753][20381] Updated weights for policy 1, policy_version 4817 (0.0007) [2023-12-27 05:42:02,807][20381] Updated weights for policy 1, policy_version 4827 (0.0007) [2023-12-27 05:42:03,435][20347] Updated weights for policy 0, policy_version 4803 (0.0007) [2023-12-27 05:42:03,484][20347] Updated weights for policy 0, policy_version 4813 (0.0006) [2023-12-27 05:42:03,490][20381] Updated weights for policy 1, policy_version 4837 (0.0007) [2023-12-27 05:42:03,539][20347] Updated weights for policy 0, policy_version 4823 (0.0007) [2023-12-27 05:42:03,546][20381] Updated weights for policy 1, policy_version 4847 (0.0007) [2023-12-27 05:42:03,595][20381] Updated weights for policy 1, policy_version 4857 (0.0007) [2023-12-27 05:42:04,218][20347] Updated weights for policy 0, policy_version 4833 (0.0006) [2023-12-27 05:42:04,280][20347] Updated weights for policy 0, policy_version 4843 (0.0005) [2023-12-27 05:42:04,300][20381] Updated weights for policy 1, policy_version 4867 (0.0007) [2023-12-27 05:42:04,343][20347] Updated weights for policy 0, policy_version 4853 (0.0006) [2023-12-27 05:42:04,363][20381] Updated weights for policy 1, policy_version 4877 (0.0007) [2023-12-27 05:42:04,405][20347] Updated weights for policy 0, policy_version 4863 (0.0007) [2023-12-27 05:42:04,422][20381] Updated weights for policy 1, policy_version 4887 (0.0009) [2023-12-27 05:42:05,025][20347] Updated weights for policy 0, policy_version 4873 (0.0007) [2023-12-27 05:42:05,037][20381] Updated weights for policy 1, policy_version 4897 (0.0010) [2023-12-27 05:42:05,076][20347] Updated weights for policy 0, policy_version 4883 (0.0007) [2023-12-27 05:42:05,094][20381] Updated weights for policy 1, policy_version 4907 (0.0007) [2023-12-27 05:42:05,134][20347] Updated weights for policy 0, policy_version 4893 (0.0010) [2023-12-27 05:42:05,147][20381] Updated weights for policy 1, policy_version 4917 (0.0007) [2023-12-27 05:42:05,201][20381] Updated weights for policy 1, policy_version 4927 (0.0008) [2023-12-27 05:42:05,768][20347] Updated weights for policy 0, policy_version 4903 (0.0010) [2023-12-27 05:42:05,823][20381] Updated weights for policy 1, policy_version 4937 (0.0007) [2023-12-27 05:42:05,825][20347] Updated weights for policy 0, policy_version 4913 (0.0010) [2023-12-27 05:42:05,869][20381] Updated weights for policy 1, policy_version 4947 (0.0007) [2023-12-27 05:42:05,880][20347] Updated weights for policy 0, policy_version 4923 (0.0006) [2023-12-27 05:42:05,912][20381] Updated weights for policy 1, policy_version 4957 (0.0007) [2023-12-27 05:42:06,548][20347] Updated weights for policy 0, policy_version 4933 (0.0008) [2023-12-27 05:42:06,594][20381] Updated weights for policy 1, policy_version 4967 (0.0009) [2023-12-27 05:42:06,607][20347] Updated weights for policy 0, policy_version 4943 (0.0010) [2023-12-27 05:42:06,654][20381] Updated weights for policy 1, policy_version 4977 (0.0011) [2023-12-27 05:42:06,665][20347] Updated weights for policy 0, policy_version 4953 (0.0010) [2023-12-27 05:42:06,711][20381] Updated weights for policy 1, policy_version 4987 (0.0010) [2023-12-27 05:42:07,304][20347] Updated weights for policy 0, policy_version 4963 (0.0010) [2023-12-27 05:42:07,352][20347] Updated weights for policy 0, policy_version 4973 (0.0010) [2023-12-27 05:42:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20381.7). Total num frames: 2547712. Throughput: 0: 10331.4, 1: 10514.2. Samples: 2520748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:42:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000004992_1277952.pth... [2023-12-27 05:42:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000003808_974848.pth [2023-12-27 05:42:07,403][20347] Updated weights for policy 0, policy_version 4983 (0.0010) [2023-12-27 05:42:07,424][20381] Updated weights for policy 1, policy_version 4997 (0.0008) [2023-12-27 05:42:07,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000004992_1277952.pth... [2023-12-27 05:42:07,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000003776_966656.pth [2023-12-27 05:42:07,480][20381] Updated weights for policy 1, policy_version 5007 (0.0006) [2023-12-27 05:42:07,538][20381] Updated weights for policy 1, policy_version 5017 (0.0005) [2023-12-27 05:42:08,106][20347] Updated weights for policy 0, policy_version 4993 (0.0010) [2023-12-27 05:42:08,166][20347] Updated weights for policy 0, policy_version 5003 (0.0007) [2023-12-27 05:42:08,182][20381] Updated weights for policy 1, policy_version 5027 (0.0006) [2023-12-27 05:42:08,216][20347] Updated weights for policy 0, policy_version 5013 (0.0007) [2023-12-27 05:42:08,234][20381] Updated weights for policy 1, policy_version 5037 (0.0008) [2023-12-27 05:42:08,264][20347] Updated weights for policy 0, policy_version 5023 (0.0005) [2023-12-27 05:42:08,292][20381] Updated weights for policy 1, policy_version 5047 (0.0008) [2023-12-27 05:42:08,908][20347] Updated weights for policy 0, policy_version 5033 (0.0007) [2023-12-27 05:42:08,956][20347] Updated weights for policy 0, policy_version 5043 (0.0009) [2023-12-27 05:42:08,971][20381] Updated weights for policy 1, policy_version 5057 (0.0007) [2023-12-27 05:42:09,016][20347] Updated weights for policy 0, policy_version 5053 (0.0007) [2023-12-27 05:42:09,017][20381] Updated weights for policy 1, policy_version 5067 (0.0005) [2023-12-27 05:42:09,062][20381] Updated weights for policy 1, policy_version 5077 (0.0005) [2023-12-27 05:42:09,109][20381] Updated weights for policy 1, policy_version 5087 (0.0006) [2023-12-27 05:42:09,728][20347] Updated weights for policy 0, policy_version 5063 (0.0007) [2023-12-27 05:42:09,789][20347] Updated weights for policy 0, policy_version 5073 (0.0005) [2023-12-27 05:42:09,851][20347] Updated weights for policy 0, policy_version 5083 (0.0009) [2023-12-27 05:42:09,881][20381] Updated weights for policy 1, policy_version 5097 (0.0010) [2023-12-27 05:42:09,938][20381] Updated weights for policy 1, policy_version 5107 (0.0008) [2023-12-27 05:42:09,992][20381] Updated weights for policy 1, policy_version 5117 (0.0007) [2023-12-27 05:42:10,495][20347] Updated weights for policy 0, policy_version 5093 (0.0009) [2023-12-27 05:42:10,546][20347] Updated weights for policy 0, policy_version 5103 (0.0005) [2023-12-27 05:42:10,597][20347] Updated weights for policy 0, policy_version 5113 (0.0005) [2023-12-27 05:42:10,723][20381] Updated weights for policy 1, policy_version 5127 (0.0008) [2023-12-27 05:42:10,781][20381] Updated weights for policy 1, policy_version 5137 (0.0007) [2023-12-27 05:42:10,836][20381] Updated weights for policy 1, policy_version 5147 (0.0005) [2023-12-27 05:42:11,218][20347] Updated weights for policy 0, policy_version 5123 (0.0007) [2023-12-27 05:42:11,283][20347] Updated weights for policy 0, policy_version 5133 (0.0009) [2023-12-27 05:42:11,347][20347] Updated weights for policy 0, policy_version 5143 (0.0010) [2023-12-27 05:42:11,550][20381] Updated weights for policy 1, policy_version 5157 (0.0008) [2023-12-27 05:42:11,610][20381] Updated weights for policy 1, policy_version 5167 (0.0008) [2023-12-27 05:42:11,668][20381] Updated weights for policy 1, policy_version 5177 (0.0007) [2023-12-27 05:42:11,992][20347] Updated weights for policy 0, policy_version 5153 (0.0010) [2023-12-27 05:42:12,051][20347] Updated weights for policy 0, policy_version 5163 (0.0010) [2023-12-27 05:42:12,108][20347] Updated weights for policy 0, policy_version 5173 (0.0010) [2023-12-27 05:42:12,161][20347] Updated weights for policy 0, policy_version 5183 (0.0008) [2023-12-27 05:42:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20417.0). Total num frames: 2654208. Throughput: 0: 10338.0, 1: 10473.7. Samples: 2644984. Policy #0 lag: (min: 27.0, avg: 31.3, max: 59.0) [2023-12-27 05:42:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:12,398][20381] Updated weights for policy 1, policy_version 5187 (0.0007) [2023-12-27 05:42:12,452][20381] Updated weights for policy 1, policy_version 5197 (0.0008) [2023-12-27 05:42:12,501][20381] Updated weights for policy 1, policy_version 5207 (0.0008) [2023-12-27 05:42:12,822][20347] Updated weights for policy 0, policy_version 5193 (0.0011) [2023-12-27 05:42:12,869][20347] Updated weights for policy 0, policy_version 5203 (0.0010) [2023-12-27 05:42:12,922][20347] Updated weights for policy 0, policy_version 5213 (0.0010) [2023-12-27 05:42:13,196][20381] Updated weights for policy 1, policy_version 5217 (0.0009) [2023-12-27 05:42:13,244][20381] Updated weights for policy 1, policy_version 5227 (0.0007) [2023-12-27 05:42:13,291][20381] Updated weights for policy 1, policy_version 5237 (0.0007) [2023-12-27 05:42:13,338][20381] Updated weights for policy 1, policy_version 5247 (0.0007) [2023-12-27 05:42:13,620][20347] Updated weights for policy 0, policy_version 5223 (0.0010) [2023-12-27 05:42:13,665][20347] Updated weights for policy 0, policy_version 5233 (0.0010) [2023-12-27 05:42:13,713][20347] Updated weights for policy 0, policy_version 5243 (0.0010) [2023-12-27 05:42:14,022][20381] Updated weights for policy 1, policy_version 5257 (0.0007) [2023-12-27 05:42:14,069][20381] Updated weights for policy 1, policy_version 5267 (0.0007) [2023-12-27 05:42:14,118][20381] Updated weights for policy 1, policy_version 5277 (0.0006) [2023-12-27 05:42:14,419][20347] Updated weights for policy 0, policy_version 5253 (0.0010) [2023-12-27 05:42:14,476][20347] Updated weights for policy 0, policy_version 5263 (0.0010) [2023-12-27 05:42:14,531][20347] Updated weights for policy 0, policy_version 5273 (0.0010) [2023-12-27 05:42:14,736][20381] Updated weights for policy 1, policy_version 5287 (0.0008) [2023-12-27 05:42:14,802][20381] Updated weights for policy 1, policy_version 5297 (0.0008) [2023-12-27 05:42:14,860][20381] Updated weights for policy 1, policy_version 5307 (0.0006) [2023-12-27 05:42:15,196][20347] Updated weights for policy 0, policy_version 5283 (0.0009) [2023-12-27 05:42:15,261][20347] Updated weights for policy 0, policy_version 5293 (0.0007) [2023-12-27 05:42:15,315][20347] Updated weights for policy 0, policy_version 5303 (0.0005) [2023-12-27 05:42:15,592][20381] Updated weights for policy 1, policy_version 5317 (0.0008) [2023-12-27 05:42:15,658][20381] Updated weights for policy 1, policy_version 5327 (0.0008) [2023-12-27 05:42:15,720][20381] Updated weights for policy 1, policy_version 5337 (0.0008) [2023-12-27 05:42:15,913][20347] Updated weights for policy 0, policy_version 5313 (0.0006) [2023-12-27 05:42:15,972][20347] Updated weights for policy 0, policy_version 5323 (0.0007) [2023-12-27 05:42:16,037][20347] Updated weights for policy 0, policy_version 5333 (0.0008) [2023-12-27 05:42:16,097][20347] Updated weights for policy 0, policy_version 5343 (0.0007) [2023-12-27 05:42:16,423][20381] Updated weights for policy 1, policy_version 5347 (0.0007) [2023-12-27 05:42:16,477][20381] Updated weights for policy 1, policy_version 5357 (0.0005) [2023-12-27 05:42:16,542][20381] Updated weights for policy 1, policy_version 5367 (0.0006) [2023-12-27 05:42:16,720][20347] Updated weights for policy 0, policy_version 5353 (0.0007) [2023-12-27 05:42:16,777][20347] Updated weights for policy 0, policy_version 5363 (0.0009) [2023-12-27 05:42:16,833][20347] Updated weights for policy 0, policy_version 5373 (0.0007) [2023-12-27 05:42:17,132][20381] Updated weights for policy 1, policy_version 5377 (0.0007) [2023-12-27 05:42:17,193][20381] Updated weights for policy 1, policy_version 5387 (0.0010) [2023-12-27 05:42:17,252][20381] Updated weights for policy 1, policy_version 5397 (0.0010) [2023-12-27 05:42:17,313][20381] Updated weights for policy 1, policy_version 5407 (0.0010) [2023-12-27 05:42:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20449.7). Total num frames: 2760704. Throughput: 0: 10311.2, 1: 10350.5. Samples: 2768388. Policy #0 lag: (min: 5.0, avg: 10.1, max: 37.0) [2023-12-27 05:42:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:17,540][20347] Updated weights for policy 0, policy_version 5383 (0.0008) [2023-12-27 05:42:17,600][20347] Updated weights for policy 0, policy_version 5393 (0.0007) [2023-12-27 05:42:17,654][20347] Updated weights for policy 0, policy_version 5403 (0.0006) [2023-12-27 05:42:18,003][20381] Updated weights for policy 1, policy_version 5417 (0.0007) [2023-12-27 05:42:18,069][20381] Updated weights for policy 1, policy_version 5427 (0.0007) [2023-12-27 05:42:18,126][20381] Updated weights for policy 1, policy_version 5437 (0.0007) [2023-12-27 05:42:18,365][20347] Updated weights for policy 0, policy_version 5413 (0.0008) [2023-12-27 05:42:18,422][20347] Updated weights for policy 0, policy_version 5423 (0.0008) [2023-12-27 05:42:18,480][20347] Updated weights for policy 0, policy_version 5433 (0.0009) [2023-12-27 05:42:18,791][20381] Updated weights for policy 1, policy_version 5447 (0.0007) [2023-12-27 05:42:18,850][20381] Updated weights for policy 1, policy_version 5457 (0.0007) [2023-12-27 05:42:18,905][20381] Updated weights for policy 1, policy_version 5467 (0.0008) [2023-12-27 05:42:19,160][20347] Updated weights for policy 0, policy_version 5443 (0.0007) [2023-12-27 05:42:19,220][20347] Updated weights for policy 0, policy_version 5453 (0.0006) [2023-12-27 05:42:19,281][20347] Updated weights for policy 0, policy_version 5463 (0.0007) [2023-12-27 05:42:19,539][20381] Updated weights for policy 1, policy_version 5477 (0.0006) [2023-12-27 05:42:19,600][20381] Updated weights for policy 1, policy_version 5487 (0.0007) [2023-12-27 05:42:19,658][20381] Updated weights for policy 1, policy_version 5497 (0.0007) [2023-12-27 05:42:19,937][20347] Updated weights for policy 0, policy_version 5473 (0.0006) [2023-12-27 05:42:19,992][20347] Updated weights for policy 0, policy_version 5483 (0.0009) [2023-12-27 05:42:20,081][20347] Updated weights for policy 0, policy_version 5493 (0.0008) [2023-12-27 05:42:20,139][20347] Updated weights for policy 0, policy_version 5503 (0.0008) [2023-12-27 05:42:20,271][20381] Updated weights for policy 1, policy_version 5507 (0.0007) [2023-12-27 05:42:20,329][20381] Updated weights for policy 1, policy_version 5517 (0.0006) [2023-12-27 05:42:20,384][20381] Updated weights for policy 1, policy_version 5527 (0.0006) [2023-12-27 05:42:20,860][20347] Updated weights for policy 0, policy_version 5513 (0.0009) [2023-12-27 05:42:20,915][20347] Updated weights for policy 0, policy_version 5523 (0.0010) [2023-12-27 05:42:20,977][20347] Updated weights for policy 0, policy_version 5533 (0.0010) [2023-12-27 05:42:21,022][20381] Updated weights for policy 1, policy_version 5537 (0.0006) [2023-12-27 05:42:21,087][20381] Updated weights for policy 1, policy_version 5547 (0.0007) [2023-12-27 05:42:21,147][20381] Updated weights for policy 1, policy_version 5557 (0.0008) [2023-12-27 05:42:21,196][20381] Updated weights for policy 1, policy_version 5567 (0.0008) [2023-12-27 05:42:21,688][20347] Updated weights for policy 0, policy_version 5543 (0.0009) [2023-12-27 05:42:21,753][20347] Updated weights for policy 0, policy_version 5553 (0.0008) [2023-12-27 05:42:21,808][20347] Updated weights for policy 0, policy_version 5563 (0.0008) [2023-12-27 05:42:21,907][20381] Updated weights for policy 1, policy_version 5577 (0.0008) [2023-12-27 05:42:21,958][20381] Updated weights for policy 1, policy_version 5587 (0.0008) [2023-12-27 05:42:22,010][20381] Updated weights for policy 1, policy_version 5597 (0.0008) [2023-12-27 05:42:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.7, 300 sec: 20421.5). Total num frames: 2859008. Throughput: 0: 10306.0, 1: 10323.3. Samples: 2830104. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 05:42:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000005600_1433600.pth... [2023-12-27 05:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000005568_1425408.pth... [2023-12-27 05:42:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000004352_1114112.pth [2023-12-27 05:42:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000004416_1130496.pth [2023-12-27 05:42:22,462][20347] Updated weights for policy 0, policy_version 5573 (0.0007) [2023-12-27 05:42:22,521][20347] Updated weights for policy 0, policy_version 5583 (0.0008) [2023-12-27 05:42:22,579][20347] Updated weights for policy 0, policy_version 5593 (0.0008) [2023-12-27 05:42:22,740][20381] Updated weights for policy 1, policy_version 5607 (0.0007) [2023-12-27 05:42:22,801][20381] Updated weights for policy 1, policy_version 5617 (0.0008) [2023-12-27 05:42:22,862][20381] Updated weights for policy 1, policy_version 5627 (0.0008) [2023-12-27 05:42:23,207][20347] Updated weights for policy 0, policy_version 5603 (0.0009) [2023-12-27 05:42:23,262][20347] Updated weights for policy 0, policy_version 5613 (0.0007) [2023-12-27 05:42:23,317][20347] Updated weights for policy 0, policy_version 5623 (0.0008) [2023-12-27 05:42:23,590][20381] Updated weights for policy 1, policy_version 5637 (0.0008) [2023-12-27 05:42:23,634][20381] Updated weights for policy 1, policy_version 5647 (0.0009) [2023-12-27 05:42:23,683][20381] Updated weights for policy 1, policy_version 5657 (0.0006) [2023-12-27 05:42:23,909][20347] Updated weights for policy 0, policy_version 5633 (0.0008) [2023-12-27 05:42:23,956][20347] Updated weights for policy 0, policy_version 5643 (0.0007) [2023-12-27 05:42:24,008][20347] Updated weights for policy 0, policy_version 5653 (0.0009) [2023-12-27 05:42:24,062][20347] Updated weights for policy 0, policy_version 5663 (0.0010) [2023-12-27 05:42:24,328][20381] Updated weights for policy 1, policy_version 5667 (0.0006) [2023-12-27 05:42:24,390][20381] Updated weights for policy 1, policy_version 5677 (0.0009) [2023-12-27 05:42:24,444][20381] Updated weights for policy 1, policy_version 5687 (0.0010) [2023-12-27 05:42:24,727][20347] Updated weights for policy 0, policy_version 5673 (0.0006) [2023-12-27 05:42:24,787][20347] Updated weights for policy 0, policy_version 5683 (0.0010) [2023-12-27 05:42:24,844][20347] Updated weights for policy 0, policy_version 5693 (0.0010) [2023-12-27 05:42:25,054][20381] Updated weights for policy 1, policy_version 5697 (0.0010) [2023-12-27 05:42:25,103][20381] Updated weights for policy 1, policy_version 5707 (0.0007) [2023-12-27 05:42:25,154][20381] Updated weights for policy 1, policy_version 5717 (0.0008) [2023-12-27 05:42:25,211][20381] Updated weights for policy 1, policy_version 5727 (0.0010) [2023-12-27 05:42:25,483][20347] Updated weights for policy 0, policy_version 5703 (0.0010) [2023-12-27 05:42:25,542][20347] Updated weights for policy 0, policy_version 5713 (0.0008) [2023-12-27 05:42:25,585][20347] Updated weights for policy 0, policy_version 5723 (0.0008) [2023-12-27 05:42:25,818][20381] Updated weights for policy 1, policy_version 5737 (0.0008) [2023-12-27 05:42:25,879][20381] Updated weights for policy 1, policy_version 5747 (0.0007) [2023-12-27 05:42:25,937][20381] Updated weights for policy 1, policy_version 5757 (0.0008) [2023-12-27 05:42:26,230][20347] Updated weights for policy 0, policy_version 5733 (0.0006) [2023-12-27 05:42:26,289][20347] Updated weights for policy 0, policy_version 5743 (0.0005) [2023-12-27 05:42:26,340][20347] Updated weights for policy 0, policy_version 5753 (0.0005) [2023-12-27 05:42:26,651][20381] Updated weights for policy 1, policy_version 5767 (0.0008) [2023-12-27 05:42:26,704][20381] Updated weights for policy 1, policy_version 5777 (0.0008) [2023-12-27 05:42:26,764][20381] Updated weights for policy 1, policy_version 5787 (0.0007) [2023-12-27 05:42:26,930][20347] Updated weights for policy 0, policy_version 5763 (0.0006) [2023-12-27 05:42:26,991][20347] Updated weights for policy 0, policy_version 5773 (0.0007) [2023-12-27 05:42:27,045][20347] Updated weights for policy 0, policy_version 5783 (0.0008) [2023-12-27 05:42:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20451.7). Total num frames: 2965504. Throughput: 0: 10383.4, 1: 10228.7. Samples: 2954116. Policy #0 lag: (min: 1.0, avg: 25.3, max: 33.0) [2023-12-27 05:42:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:27,499][20381] Updated weights for policy 1, policy_version 5797 (0.0008) [2023-12-27 05:42:27,554][20381] Updated weights for policy 1, policy_version 5807 (0.0008) [2023-12-27 05:42:27,597][20347] Updated weights for policy 0, policy_version 5793 (0.0007) [2023-12-27 05:42:27,612][20381] Updated weights for policy 1, policy_version 5817 (0.0007) [2023-12-27 05:42:27,654][20347] Updated weights for policy 0, policy_version 5803 (0.0005) [2023-12-27 05:42:27,704][20347] Updated weights for policy 0, policy_version 5813 (0.0005) [2023-12-27 05:42:27,755][20347] Updated weights for policy 0, policy_version 5823 (0.0005) [2023-12-27 05:42:28,320][20381] Updated weights for policy 1, policy_version 5827 (0.0007) [2023-12-27 05:42:28,380][20381] Updated weights for policy 1, policy_version 5837 (0.0007) [2023-12-27 05:42:28,386][20347] Updated weights for policy 0, policy_version 5833 (0.0008) [2023-12-27 05:42:28,437][20381] Updated weights for policy 1, policy_version 5847 (0.0006) [2023-12-27 05:42:28,446][20347] Updated weights for policy 0, policy_version 5843 (0.0006) [2023-12-27 05:42:28,505][20347] Updated weights for policy 0, policy_version 5853 (0.0006) [2023-12-27 05:42:29,084][20381] Updated weights for policy 1, policy_version 5857 (0.0007) [2023-12-27 05:42:29,136][20381] Updated weights for policy 1, policy_version 5867 (0.0007) [2023-12-27 05:42:29,194][20381] Updated weights for policy 1, policy_version 5877 (0.0008) [2023-12-27 05:42:29,201][20347] Updated weights for policy 0, policy_version 5863 (0.0008) [2023-12-27 05:42:29,250][20381] Updated weights for policy 1, policy_version 5887 (0.0009) [2023-12-27 05:42:29,262][20347] Updated weights for policy 0, policy_version 5873 (0.0007) [2023-12-27 05:42:29,307][20347] Updated weights for policy 0, policy_version 5883 (0.0007) [2023-12-27 05:42:29,893][20381] Updated weights for policy 1, policy_version 5897 (0.0008) [2023-12-27 05:42:29,949][20381] Updated weights for policy 1, policy_version 5907 (0.0008) [2023-12-27 05:42:30,005][20381] Updated weights for policy 1, policy_version 5917 (0.0007) [2023-12-27 05:42:30,027][20347] Updated weights for policy 0, policy_version 5893 (0.0008) [2023-12-27 05:42:30,089][20347] Updated weights for policy 0, policy_version 5903 (0.0007) [2023-12-27 05:42:30,148][20347] Updated weights for policy 0, policy_version 5913 (0.0008) [2023-12-27 05:42:30,652][20381] Updated weights for policy 1, policy_version 5927 (0.0006) [2023-12-27 05:42:30,703][20381] Updated weights for policy 1, policy_version 5937 (0.0007) [2023-12-27 05:42:30,755][20381] Updated weights for policy 1, policy_version 5947 (0.0006) [2023-12-27 05:42:30,841][20347] Updated weights for policy 0, policy_version 5923 (0.0007) [2023-12-27 05:42:30,892][20347] Updated weights for policy 0, policy_version 5933 (0.0005) [2023-12-27 05:42:30,941][20347] Updated weights for policy 0, policy_version 5943 (0.0008) [2023-12-27 05:42:31,335][20381] Updated weights for policy 1, policy_version 5957 (0.0007) [2023-12-27 05:42:31,402][20381] Updated weights for policy 1, policy_version 5967 (0.0008) [2023-12-27 05:42:31,462][20381] Updated weights for policy 1, policy_version 5977 (0.0008) [2023-12-27 05:42:31,575][20347] Updated weights for policy 0, policy_version 5953 (0.0007) [2023-12-27 05:42:31,631][20347] Updated weights for policy 0, policy_version 5963 (0.0007) [2023-12-27 05:42:31,689][20347] Updated weights for policy 0, policy_version 5973 (0.0008) [2023-12-27 05:42:31,751][20347] Updated weights for policy 0, policy_version 5983 (0.0008) [2023-12-27 05:42:32,094][20381] Updated weights for policy 1, policy_version 5987 (0.0007) [2023-12-27 05:42:32,152][20381] Updated weights for policy 1, policy_version 5997 (0.0008) [2023-12-27 05:42:32,208][20381] Updated weights for policy 1, policy_version 6007 (0.0007) [2023-12-27 05:42:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20480.0). Total num frames: 3072000. Throughput: 0: 10424.6, 1: 10262.8. Samples: 3081088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:42:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:32,419][20347] Updated weights for policy 0, policy_version 5993 (0.0007) [2023-12-27 05:42:32,462][20347] Updated weights for policy 0, policy_version 6003 (0.0007) [2023-12-27 05:42:32,512][20347] Updated weights for policy 0, policy_version 6013 (0.0007) [2023-12-27 05:42:32,900][20381] Updated weights for policy 1, policy_version 6017 (0.0009) [2023-12-27 05:42:32,955][20381] Updated weights for policy 1, policy_version 6027 (0.0007) [2023-12-27 05:42:33,003][20381] Updated weights for policy 1, policy_version 6037 (0.0007) [2023-12-27 05:42:33,052][20381] Updated weights for policy 1, policy_version 6047 (0.0009) [2023-12-27 05:42:33,192][20347] Updated weights for policy 0, policy_version 6023 (0.0007) [2023-12-27 05:42:33,247][20347] Updated weights for policy 0, policy_version 6033 (0.0007) [2023-12-27 05:42:33,303][20347] Updated weights for policy 0, policy_version 6043 (0.0007) [2023-12-27 05:42:33,745][20381] Updated weights for policy 1, policy_version 6057 (0.0010) [2023-12-27 05:42:33,789][20381] Updated weights for policy 1, policy_version 6067 (0.0010) [2023-12-27 05:42:33,840][20381] Updated weights for policy 1, policy_version 6077 (0.0010) [2023-12-27 05:42:33,993][20347] Updated weights for policy 0, policy_version 6053 (0.0007) [2023-12-27 05:42:34,044][20347] Updated weights for policy 0, policy_version 6063 (0.0007) [2023-12-27 05:42:34,091][20347] Updated weights for policy 0, policy_version 6073 (0.0007) [2023-12-27 05:42:34,468][20381] Updated weights for policy 1, policy_version 6087 (0.0007) [2023-12-27 05:42:34,523][20381] Updated weights for policy 1, policy_version 6097 (0.0005) [2023-12-27 05:42:34,572][20381] Updated weights for policy 1, policy_version 6107 (0.0006) [2023-12-27 05:42:34,827][20347] Updated weights for policy 0, policy_version 6083 (0.0007) [2023-12-27 05:42:34,883][20347] Updated weights for policy 0, policy_version 6093 (0.0006) [2023-12-27 05:42:34,944][20347] Updated weights for policy 0, policy_version 6103 (0.0008) [2023-12-27 05:42:35,198][20381] Updated weights for policy 1, policy_version 6117 (0.0008) [2023-12-27 05:42:35,250][20381] Updated weights for policy 1, policy_version 6127 (0.0008) [2023-12-27 05:42:35,305][20381] Updated weights for policy 1, policy_version 6137 (0.0008) [2023-12-27 05:42:35,614][20347] Updated weights for policy 0, policy_version 6113 (0.0007) [2023-12-27 05:42:35,680][20347] Updated weights for policy 0, policy_version 6123 (0.0005) [2023-12-27 05:42:35,737][20347] Updated weights for policy 0, policy_version 6133 (0.0007) [2023-12-27 05:42:35,797][20347] Updated weights for policy 0, policy_version 6143 (0.0006) [2023-12-27 05:42:35,890][20381] Updated weights for policy 1, policy_version 6147 (0.0006) [2023-12-27 05:42:35,954][20381] Updated weights for policy 1, policy_version 6157 (0.0005) [2023-12-27 05:42:36,013][20381] Updated weights for policy 1, policy_version 6167 (0.0007) [2023-12-27 05:42:36,401][20347] Updated weights for policy 0, policy_version 6153 (0.0005) [2023-12-27 05:42:36,454][20347] Updated weights for policy 0, policy_version 6163 (0.0005) [2023-12-27 05:42:36,506][20347] Updated weights for policy 0, policy_version 6173 (0.0005) [2023-12-27 05:42:36,679][20381] Updated weights for policy 1, policy_version 6177 (0.0008) [2023-12-27 05:42:36,738][20381] Updated weights for policy 1, policy_version 6187 (0.0011) [2023-12-27 05:42:36,799][20381] Updated weights for policy 1, policy_version 6197 (0.0011) [2023-12-27 05:42:36,863][20381] Updated weights for policy 1, policy_version 6207 (0.0009) [2023-12-27 05:42:37,057][20347] Updated weights for policy 0, policy_version 6183 (0.0008) [2023-12-27 05:42:37,115][20347] Updated weights for policy 0, policy_version 6193 (0.0007) [2023-12-27 05:42:37,165][20347] Updated weights for policy 0, policy_version 6203 (0.0006) [2023-12-27 05:42:37,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20506.4). Total num frames: 3178496. Throughput: 0: 10408.9, 1: 10254.3. Samples: 3142904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:42:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000006208_1589248.pth... [2023-12-27 05:42:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000006208_1589248.pth... [2023-12-27 05:42:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000004992_1277952.pth [2023-12-27 05:42:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000004992_1277952.pth [2023-12-27 05:42:37,476][20381] Updated weights for policy 1, policy_version 6217 (0.0010) [2023-12-27 05:42:37,526][20381] Updated weights for policy 1, policy_version 6227 (0.0010) [2023-12-27 05:42:37,575][20381] Updated weights for policy 1, policy_version 6237 (0.0010) [2023-12-27 05:42:37,817][20347] Updated weights for policy 0, policy_version 6213 (0.0007) [2023-12-27 05:42:37,864][20347] Updated weights for policy 0, policy_version 6223 (0.0005) [2023-12-27 05:42:37,915][20347] Updated weights for policy 0, policy_version 6233 (0.0010) [2023-12-27 05:42:38,220][20381] Updated weights for policy 1, policy_version 6247 (0.0007) [2023-12-27 05:42:38,264][20381] Updated weights for policy 1, policy_version 6257 (0.0007) [2023-12-27 05:42:38,312][20381] Updated weights for policy 1, policy_version 6267 (0.0010) [2023-12-27 05:42:38,594][20347] Updated weights for policy 0, policy_version 6243 (0.0009) [2023-12-27 05:42:38,651][20347] Updated weights for policy 0, policy_version 6253 (0.0008) [2023-12-27 05:42:38,710][20347] Updated weights for policy 0, policy_version 6263 (0.0007) [2023-12-27 05:42:38,954][20381] Updated weights for policy 1, policy_version 6277 (0.0011) [2023-12-27 05:42:38,999][20381] Updated weights for policy 1, policy_version 6287 (0.0010) [2023-12-27 05:42:39,048][20381] Updated weights for policy 1, policy_version 6297 (0.0010) [2023-12-27 05:42:39,433][20347] Updated weights for policy 0, policy_version 6273 (0.0008) [2023-12-27 05:42:39,494][20347] Updated weights for policy 0, policy_version 6283 (0.0007) [2023-12-27 05:42:39,545][20347] Updated weights for policy 0, policy_version 6293 (0.0007) [2023-12-27 05:42:39,604][20347] Updated weights for policy 0, policy_version 6303 (0.0009) [2023-12-27 05:42:39,770][20381] Updated weights for policy 1, policy_version 6307 (0.0010) [2023-12-27 05:42:39,831][20381] Updated weights for policy 1, policy_version 6317 (0.0011) [2023-12-27 05:42:39,888][20381] Updated weights for policy 1, policy_version 6327 (0.0011) [2023-12-27 05:42:40,318][20347] Updated weights for policy 0, policy_version 6313 (0.0008) [2023-12-27 05:42:40,373][20347] Updated weights for policy 0, policy_version 6323 (0.0007) [2023-12-27 05:42:40,425][20347] Updated weights for policy 0, policy_version 6333 (0.0007) [2023-12-27 05:42:40,555][20381] Updated weights for policy 1, policy_version 6337 (0.0010) [2023-12-27 05:42:40,618][20381] Updated weights for policy 1, policy_version 6347 (0.0008) [2023-12-27 05:42:40,677][20381] Updated weights for policy 1, policy_version 6357 (0.0008) [2023-12-27 05:42:40,733][20381] Updated weights for policy 1, policy_version 6367 (0.0008) [2023-12-27 05:42:41,168][20347] Updated weights for policy 0, policy_version 6343 (0.0009) [2023-12-27 05:42:41,229][20347] Updated weights for policy 0, policy_version 6353 (0.0006) [2023-12-27 05:42:41,289][20347] Updated weights for policy 0, policy_version 6363 (0.0008) [2023-12-27 05:42:41,405][20381] Updated weights for policy 1, policy_version 6377 (0.0008) [2023-12-27 05:42:41,467][20381] Updated weights for policy 1, policy_version 6387 (0.0007) [2023-12-27 05:42:41,527][20381] Updated weights for policy 1, policy_version 6397 (0.0008) [2023-12-27 05:42:41,994][20347] Updated weights for policy 0, policy_version 6373 (0.0006) [2023-12-27 05:42:42,063][20347] Updated weights for policy 0, policy_version 6383 (0.0005) [2023-12-27 05:42:42,128][20347] Updated weights for policy 0, policy_version 6393 (0.0005) [2023-12-27 05:42:42,195][20381] Updated weights for policy 1, policy_version 6407 (0.0008) [2023-12-27 05:42:42,255][20381] Updated weights for policy 1, policy_version 6417 (0.0008) [2023-12-27 05:42:42,313][20381] Updated weights for policy 1, policy_version 6427 (0.0008) [2023-12-27 05:42:42,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20531.2). Total num frames: 3284992. Throughput: 0: 10469.6, 1: 10300.1. Samples: 3269436. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 05:42:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:42,756][20347] Updated weights for policy 0, policy_version 6403 (0.0006) [2023-12-27 05:42:42,805][20347] Updated weights for policy 0, policy_version 6413 (0.0008) [2023-12-27 05:42:42,861][20347] Updated weights for policy 0, policy_version 6423 (0.0008) [2023-12-27 05:42:42,978][20381] Updated weights for policy 1, policy_version 6437 (0.0007) [2023-12-27 05:42:43,036][20381] Updated weights for policy 1, policy_version 6447 (0.0008) [2023-12-27 05:42:43,091][20381] Updated weights for policy 1, policy_version 6457 (0.0006) [2023-12-27 05:42:43,501][20347] Updated weights for policy 0, policy_version 6433 (0.0007) [2023-12-27 05:42:43,549][20347] Updated weights for policy 0, policy_version 6443 (0.0008) [2023-12-27 05:42:43,600][20347] Updated weights for policy 0, policy_version 6453 (0.0010) [2023-12-27 05:42:43,649][20347] Updated weights for policy 0, policy_version 6463 (0.0007) [2023-12-27 05:42:43,757][20381] Updated weights for policy 1, policy_version 6467 (0.0007) [2023-12-27 05:42:43,811][20381] Updated weights for policy 1, policy_version 6477 (0.0006) [2023-12-27 05:42:43,876][20381] Updated weights for policy 1, policy_version 6487 (0.0006) [2023-12-27 05:42:44,295][20347] Updated weights for policy 0, policy_version 6473 (0.0008) [2023-12-27 05:42:44,344][20347] Updated weights for policy 0, policy_version 6483 (0.0008) [2023-12-27 05:42:44,402][20347] Updated weights for policy 0, policy_version 6493 (0.0008) [2023-12-27 05:42:44,446][20381] Updated weights for policy 1, policy_version 6497 (0.0006) [2023-12-27 05:42:44,492][20381] Updated weights for policy 1, policy_version 6507 (0.0005) [2023-12-27 05:42:44,539][20381] Updated weights for policy 1, policy_version 6517 (0.0006) [2023-12-27 05:42:44,594][20381] Updated weights for policy 1, policy_version 6527 (0.0007) [2023-12-27 05:42:45,130][20347] Updated weights for policy 0, policy_version 6503 (0.0007) [2023-12-27 05:42:45,189][20347] Updated weights for policy 0, policy_version 6513 (0.0008) [2023-12-27 05:42:45,240][20381] Updated weights for policy 1, policy_version 6537 (0.0006) [2023-12-27 05:42:45,246][20347] Updated weights for policy 0, policy_version 6523 (0.0007) [2023-12-27 05:42:45,297][20381] Updated weights for policy 1, policy_version 6547 (0.0006) [2023-12-27 05:42:45,353][20381] Updated weights for policy 1, policy_version 6557 (0.0008) [2023-12-27 05:42:45,970][20347] Updated weights for policy 0, policy_version 6533 (0.0007) [2023-12-27 05:42:45,976][20381] Updated weights for policy 1, policy_version 6567 (0.0007) [2023-12-27 05:42:46,026][20347] Updated weights for policy 0, policy_version 6543 (0.0005) [2023-12-27 05:42:46,033][20381] Updated weights for policy 1, policy_version 6577 (0.0007) [2023-12-27 05:42:46,077][20347] Updated weights for policy 0, policy_version 6553 (0.0005) [2023-12-27 05:42:46,090][20381] Updated weights for policy 1, policy_version 6587 (0.0008) [2023-12-27 05:42:46,736][20347] Updated weights for policy 0, policy_version 6563 (0.0007) [2023-12-27 05:42:46,796][20381] Updated weights for policy 1, policy_version 6597 (0.0006) [2023-12-27 05:42:46,798][20347] Updated weights for policy 0, policy_version 6573 (0.0007) [2023-12-27 05:42:46,847][20381] Updated weights for policy 1, policy_version 6607 (0.0007) [2023-12-27 05:42:46,852][20347] Updated weights for policy 0, policy_version 6583 (0.0007) [2023-12-27 05:42:46,895][20381] Updated weights for policy 1, policy_version 6617 (0.0008) [2023-12-27 05:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20504.8). Total num frames: 3383296. Throughput: 0: 10386.6, 1: 10381.8. Samples: 3393384. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 05:42:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:47,527][20381] Updated weights for policy 1, policy_version 6627 (0.0007) [2023-12-27 05:42:47,541][20347] Updated weights for policy 0, policy_version 6593 (0.0006) [2023-12-27 05:42:47,576][20381] Updated weights for policy 1, policy_version 6637 (0.0005) [2023-12-27 05:42:47,598][20347] Updated weights for policy 0, policy_version 6603 (0.0006) [2023-12-27 05:42:47,622][20381] Updated weights for policy 1, policy_version 6647 (0.0005) [2023-12-27 05:42:47,666][20347] Updated weights for policy 0, policy_version 6613 (0.0007) [2023-12-27 05:42:47,725][20347] Updated weights for policy 0, policy_version 6623 (0.0010) [2023-12-27 05:42:48,294][20381] Updated weights for policy 1, policy_version 6657 (0.0005) [2023-12-27 05:42:48,352][20381] Updated weights for policy 1, policy_version 6667 (0.0007) [2023-12-27 05:42:48,411][20381] Updated weights for policy 1, policy_version 6677 (0.0007) [2023-12-27 05:42:48,448][20347] Updated weights for policy 0, policy_version 6633 (0.0010) [2023-12-27 05:42:48,461][20381] Updated weights for policy 1, policy_version 6687 (0.0006) [2023-12-27 05:42:48,504][20347] Updated weights for policy 0, policy_version 6643 (0.0010) [2023-12-27 05:42:48,564][20347] Updated weights for policy 0, policy_version 6653 (0.0010) [2023-12-27 05:42:49,113][20381] Updated weights for policy 1, policy_version 6697 (0.0008) [2023-12-27 05:42:49,172][20381] Updated weights for policy 1, policy_version 6707 (0.0007) [2023-12-27 05:42:49,208][20347] Updated weights for policy 0, policy_version 6663 (0.0010) [2023-12-27 05:42:49,236][20381] Updated weights for policy 1, policy_version 6717 (0.0007) [2023-12-27 05:42:49,266][20347] Updated weights for policy 0, policy_version 6673 (0.0007) [2023-12-27 05:42:49,330][20347] Updated weights for policy 0, policy_version 6683 (0.0009) [2023-12-27 05:42:49,841][20381] Updated weights for policy 1, policy_version 6727 (0.0007) [2023-12-27 05:42:49,904][20381] Updated weights for policy 1, policy_version 6737 (0.0008) [2023-12-27 05:42:49,965][20381] Updated weights for policy 1, policy_version 6747 (0.0007) [2023-12-27 05:42:49,971][20347] Updated weights for policy 0, policy_version 6694 (0.0010) [2023-12-27 05:42:50,029][20347] Updated weights for policy 0, policy_version 6704 (0.0008) [2023-12-27 05:42:50,089][20347] Updated weights for policy 0, policy_version 6714 (0.0007) [2023-12-27 05:42:50,624][20381] Updated weights for policy 1, policy_version 6757 (0.0006) [2023-12-27 05:42:50,673][20381] Updated weights for policy 1, policy_version 6767 (0.0007) [2023-12-27 05:42:50,723][20381] Updated weights for policy 1, policy_version 6777 (0.0007) [2023-12-27 05:42:50,748][20347] Updated weights for policy 0, policy_version 6724 (0.0009) [2023-12-27 05:42:50,805][20347] Updated weights for policy 0, policy_version 6734 (0.0007) [2023-12-27 05:42:50,854][20347] Updated weights for policy 0, policy_version 6744 (0.0007) [2023-12-27 05:42:51,480][20381] Updated weights for policy 1, policy_version 6787 (0.0006) [2023-12-27 05:42:51,537][20381] Updated weights for policy 1, policy_version 6797 (0.0008) [2023-12-27 05:42:51,567][20347] Updated weights for policy 0, policy_version 6754 (0.0007) [2023-12-27 05:42:51,595][20381] Updated weights for policy 1, policy_version 6807 (0.0007) [2023-12-27 05:42:51,627][20347] Updated weights for policy 0, policy_version 6764 (0.0007) [2023-12-27 05:42:51,686][20347] Updated weights for policy 0, policy_version 6774 (0.0008) [2023-12-27 05:42:51,744][20347] Updated weights for policy 0, policy_version 6784 (0.0008) [2023-12-27 05:42:52,313][20381] Updated weights for policy 1, policy_version 6817 (0.0007) [2023-12-27 05:42:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20480.0). Total num frames: 3481600. Throughput: 0: 10382.6, 1: 10425.9. Samples: 3457128. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) [2023-12-27 05:42:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:52,376][20381] Updated weights for policy 1, policy_version 6827 (0.0008) [2023-12-27 05:42:52,416][20347] Updated weights for policy 0, policy_version 6794 (0.0007) [2023-12-27 05:42:52,431][20381] Updated weights for policy 1, policy_version 6837 (0.0007) [2023-12-27 05:42:52,475][20347] Updated weights for policy 0, policy_version 6804 (0.0006) [2023-12-27 05:42:52,485][20381] Updated weights for policy 1, policy_version 6847 (0.0006) [2023-12-27 05:42:52,489][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000006848_1753088.pth... [2023-12-27 05:42:52,492][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000005600_1433600.pth [2023-12-27 05:42:52,527][20347] Updated weights for policy 0, policy_version 6814 (0.0008) [2023-12-27 05:42:52,536][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000006816_1744896.pth... [2023-12-27 05:42:52,540][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000005568_1425408.pth [2023-12-27 05:42:53,159][20381] Updated weights for policy 1, policy_version 6857 (0.0007) [2023-12-27 05:42:53,197][20347] Updated weights for policy 0, policy_version 6824 (0.0007) [2023-12-27 05:42:53,206][20381] Updated weights for policy 1, policy_version 6867 (0.0008) [2023-12-27 05:42:53,241][20347] Updated weights for policy 0, policy_version 6834 (0.0006) [2023-12-27 05:42:53,254][20381] Updated weights for policy 1, policy_version 6877 (0.0008) [2023-12-27 05:42:53,288][20347] Updated weights for policy 0, policy_version 6844 (0.0006) [2023-12-27 05:42:53,935][20381] Updated weights for policy 1, policy_version 6887 (0.0005) [2023-12-27 05:42:53,990][20381] Updated weights for policy 1, policy_version 6897 (0.0005) [2023-12-27 05:42:54,022][20347] Updated weights for policy 0, policy_version 6854 (0.0008) [2023-12-27 05:42:54,041][20381] Updated weights for policy 1, policy_version 6907 (0.0005) [2023-12-27 05:42:54,081][20347] Updated weights for policy 0, policy_version 6864 (0.0005) [2023-12-27 05:42:54,144][20347] Updated weights for policy 0, policy_version 6874 (0.0008) [2023-12-27 05:42:54,658][20381] Updated weights for policy 1, policy_version 6917 (0.0006) [2023-12-27 05:42:54,703][20381] Updated weights for policy 1, policy_version 6927 (0.0007) [2023-12-27 05:42:54,749][20381] Updated weights for policy 1, policy_version 6937 (0.0005) [2023-12-27 05:42:54,773][20347] Updated weights for policy 0, policy_version 6884 (0.0007) [2023-12-27 05:42:54,834][20347] Updated weights for policy 0, policy_version 6894 (0.0007) [2023-12-27 05:42:54,882][20347] Updated weights for policy 0, policy_version 6904 (0.0007) [2023-12-27 05:42:55,381][20381] Updated weights for policy 1, policy_version 6947 (0.0006) [2023-12-27 05:42:55,445][20381] Updated weights for policy 1, policy_version 6957 (0.0005) [2023-12-27 05:42:55,493][20381] Updated weights for policy 1, policy_version 6967 (0.0006) [2023-12-27 05:42:55,526][20347] Updated weights for policy 0, policy_version 6914 (0.0009) [2023-12-27 05:42:55,575][20347] Updated weights for policy 0, policy_version 6924 (0.0006) [2023-12-27 05:42:55,634][20347] Updated weights for policy 0, policy_version 6934 (0.0007) [2023-12-27 05:42:55,681][20347] Updated weights for policy 0, policy_version 6944 (0.0006) [2023-12-27 05:42:56,025][20381] Updated weights for policy 1, policy_version 6977 (0.0005) [2023-12-27 05:42:56,076][20381] Updated weights for policy 1, policy_version 6987 (0.0007) [2023-12-27 05:42:56,139][20381] Updated weights for policy 1, policy_version 6997 (0.0008) [2023-12-27 05:42:56,191][20381] Updated weights for policy 1, policy_version 7007 (0.0007) [2023-12-27 05:42:56,316][20347] Updated weights for policy 0, policy_version 6954 (0.0008) [2023-12-27 05:42:56,373][20347] Updated weights for policy 0, policy_version 6964 (0.0006) [2023-12-27 05:42:56,437][20347] Updated weights for policy 0, policy_version 6974 (0.0006) [2023-12-27 05:42:56,782][20381] Updated weights for policy 1, policy_version 7017 (0.0007) [2023-12-27 05:42:56,840][20381] Updated weights for policy 1, policy_version 7027 (0.0008) [2023-12-27 05:42:56,891][20381] Updated weights for policy 1, policy_version 7037 (0.0010) [2023-12-27 05:42:56,997][20347] Updated weights for policy 0, policy_version 6984 (0.0006) [2023-12-27 05:42:57,046][20347] Updated weights for policy 0, policy_version 6994 (0.0007) [2023-12-27 05:42:57,092][20347] Updated weights for policy 0, policy_version 7004 (0.0007) [2023-12-27 05:42:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20550.2). Total num frames: 3596288. Throughput: 0: 10357.4, 1: 10476.6. Samples: 3582508. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 05:42:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:42:57,531][20381] Updated weights for policy 1, policy_version 7047 (0.0007) [2023-12-27 05:42:57,579][20381] Updated weights for policy 1, policy_version 7057 (0.0010) [2023-12-27 05:42:57,634][20381] Updated weights for policy 1, policy_version 7067 (0.0010) [2023-12-27 05:42:57,732][20347] Updated weights for policy 0, policy_version 7014 (0.0007) [2023-12-27 05:42:57,775][20347] Updated weights for policy 0, policy_version 7024 (0.0007) [2023-12-27 05:42:57,835][20347] Updated weights for policy 0, policy_version 7034 (0.0007) [2023-12-27 05:42:58,321][20381] Updated weights for policy 1, policy_version 7077 (0.0009) [2023-12-27 05:42:58,389][20381] Updated weights for policy 1, policy_version 7087 (0.0008) [2023-12-27 05:42:58,455][20381] Updated weights for policy 1, policy_version 7097 (0.0008) [2023-12-27 05:42:58,580][20347] Updated weights for policy 0, policy_version 7044 (0.0008) [2023-12-27 05:42:58,640][20347] Updated weights for policy 0, policy_version 7054 (0.0007) [2023-12-27 05:42:58,698][20347] Updated weights for policy 0, policy_version 7064 (0.0007) [2023-12-27 05:42:59,197][20381] Updated weights for policy 1, policy_version 7107 (0.0008) [2023-12-27 05:42:59,270][20381] Updated weights for policy 1, policy_version 7117 (0.0008) [2023-12-27 05:42:59,334][20381] Updated weights for policy 1, policy_version 7127 (0.0008) [2023-12-27 05:42:59,493][20347] Updated weights for policy 0, policy_version 7074 (0.0007) [2023-12-27 05:42:59,549][20347] Updated weights for policy 0, policy_version 7084 (0.0008) [2023-12-27 05:42:59,609][20347] Updated weights for policy 0, policy_version 7094 (0.0007) [2023-12-27 05:42:59,666][20347] Updated weights for policy 0, policy_version 7104 (0.0009) [2023-12-27 05:43:00,043][20381] Updated weights for policy 1, policy_version 7137 (0.0008) [2023-12-27 05:43:00,093][20381] Updated weights for policy 1, policy_version 7147 (0.0007) [2023-12-27 05:43:00,147][20381] Updated weights for policy 1, policy_version 7157 (0.0008) [2023-12-27 05:43:00,205][20381] Updated weights for policy 1, policy_version 7167 (0.0008) [2023-12-27 05:43:00,349][20347] Updated weights for policy 0, policy_version 7114 (0.0008) [2023-12-27 05:43:00,398][20347] Updated weights for policy 0, policy_version 7124 (0.0008) [2023-12-27 05:43:00,453][20347] Updated weights for policy 0, policy_version 7134 (0.0007) [2023-12-27 05:43:00,908][20381] Updated weights for policy 1, policy_version 7177 (0.0007) [2023-12-27 05:43:00,955][20381] Updated weights for policy 1, policy_version 7187 (0.0006) [2023-12-27 05:43:01,003][20381] Updated weights for policy 1, policy_version 7197 (0.0005) [2023-12-27 05:43:01,129][20347] Updated weights for policy 0, policy_version 7144 (0.0007) [2023-12-27 05:43:01,196][20347] Updated weights for policy 0, policy_version 7154 (0.0007) [2023-12-27 05:43:01,261][20347] Updated weights for policy 0, policy_version 7164 (0.0007) [2023-12-27 05:43:01,665][20381] Updated weights for policy 1, policy_version 7207 (0.0007) [2023-12-27 05:43:01,722][20381] Updated weights for policy 1, policy_version 7217 (0.0008) [2023-12-27 05:43:01,781][20381] Updated weights for policy 1, policy_version 7227 (0.0006) [2023-12-27 05:43:01,992][20347] Updated weights for policy 0, policy_version 7174 (0.0007) [2023-12-27 05:43:02,048][20347] Updated weights for policy 0, policy_version 7184 (0.0007) [2023-12-27 05:43:02,093][20347] Updated weights for policy 0, policy_version 7194 (0.0008) [2023-12-27 05:43:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20525.5). Total num frames: 3694592. Throughput: 0: 10320.5, 1: 10485.3. Samples: 3704644. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 05:43:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:02,485][20381] Updated weights for policy 1, policy_version 7237 (0.0007) [2023-12-27 05:43:02,544][20381] Updated weights for policy 1, policy_version 7247 (0.0008) [2023-12-27 05:43:02,603][20381] Updated weights for policy 1, policy_version 7257 (0.0008) [2023-12-27 05:43:02,707][20347] Updated weights for policy 0, policy_version 7204 (0.0007) [2023-12-27 05:43:02,768][20347] Updated weights for policy 0, policy_version 7214 (0.0005) [2023-12-27 05:43:02,828][20347] Updated weights for policy 0, policy_version 7224 (0.0005) [2023-12-27 05:43:03,293][20381] Updated weights for policy 1, policy_version 7267 (0.0008) [2023-12-27 05:43:03,344][20381] Updated weights for policy 1, policy_version 7277 (0.0007) [2023-12-27 05:43:03,397][20347] Updated weights for policy 0, policy_version 7234 (0.0006) [2023-12-27 05:43:03,403][20381] Updated weights for policy 1, policy_version 7287 (0.0005) [2023-12-27 05:43:03,453][20347] Updated weights for policy 0, policy_version 7244 (0.0007) [2023-12-27 05:43:03,502][20347] Updated weights for policy 0, policy_version 7254 (0.0008) [2023-12-27 05:43:03,550][20347] Updated weights for policy 0, policy_version 7264 (0.0008) [2023-12-27 05:43:03,973][20381] Updated weights for policy 1, policy_version 7297 (0.0005) [2023-12-27 05:43:04,030][20381] Updated weights for policy 1, policy_version 7307 (0.0007) [2023-12-27 05:43:04,083][20381] Updated weights for policy 1, policy_version 7317 (0.0007) [2023-12-27 05:43:04,143][20381] Updated weights for policy 1, policy_version 7327 (0.0008) [2023-12-27 05:43:04,296][20347] Updated weights for policy 0, policy_version 7274 (0.0008) [2023-12-27 05:43:04,348][20347] Updated weights for policy 0, policy_version 7284 (0.0007) [2023-12-27 05:43:04,400][20347] Updated weights for policy 0, policy_version 7294 (0.0007) [2023-12-27 05:43:04,774][20381] Updated weights for policy 1, policy_version 7337 (0.0010) [2023-12-27 05:43:04,823][20381] Updated weights for policy 1, policy_version 7347 (0.0010) [2023-12-27 05:43:04,875][20381] Updated weights for policy 1, policy_version 7357 (0.0010) [2023-12-27 05:43:05,119][20347] Updated weights for policy 0, policy_version 7304 (0.0007) [2023-12-27 05:43:05,165][20347] Updated weights for policy 0, policy_version 7314 (0.0008) [2023-12-27 05:43:05,216][20347] Updated weights for policy 0, policy_version 7324 (0.0007) [2023-12-27 05:43:05,544][20381] Updated weights for policy 1, policy_version 7367 (0.0007) [2023-12-27 05:43:05,587][20381] Updated weights for policy 1, policy_version 7377 (0.0005) [2023-12-27 05:43:05,631][20381] Updated weights for policy 1, policy_version 7387 (0.0006) [2023-12-27 05:43:05,936][20347] Updated weights for policy 0, policy_version 7334 (0.0006) [2023-12-27 05:43:05,995][20347] Updated weights for policy 0, policy_version 7344 (0.0005) [2023-12-27 05:43:06,057][20347] Updated weights for policy 0, policy_version 7354 (0.0005) [2023-12-27 05:43:06,295][20381] Updated weights for policy 1, policy_version 7397 (0.0007) [2023-12-27 05:43:06,357][20381] Updated weights for policy 1, policy_version 7407 (0.0009) [2023-12-27 05:43:06,413][20381] Updated weights for policy 1, policy_version 7417 (0.0011) [2023-12-27 05:43:06,696][20347] Updated weights for policy 0, policy_version 7364 (0.0008) [2023-12-27 05:43:06,760][20347] Updated weights for policy 0, policy_version 7374 (0.0010) [2023-12-27 05:43:06,821][20347] Updated weights for policy 0, policy_version 7384 (0.0010) [2023-12-27 05:43:07,051][20381] Updated weights for policy 1, policy_version 7427 (0.0011) [2023-12-27 05:43:07,104][20381] Updated weights for policy 1, policy_version 7437 (0.0011) [2023-12-27 05:43:07,163][20381] Updated weights for policy 1, policy_version 7447 (0.0010) [2023-12-27 05:43:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.5, 300 sec: 20546.4). Total num frames: 3801088. Throughput: 0: 10342.8, 1: 10500.7. Samples: 3768064. Policy #0 lag: (min: 9.0, avg: 25.5, max: 41.0) [2023-12-27 05:43:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000007392_1892352.pth... [2023-12-27 05:43:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000007456_1908736.pth... [2023-12-27 05:43:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000006208_1589248.pth [2023-12-27 05:43:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000006208_1589248.pth [2023-12-27 05:43:07,511][20347] Updated weights for policy 0, policy_version 7394 (0.0008) [2023-12-27 05:43:07,559][20347] Updated weights for policy 0, policy_version 7404 (0.0008) [2023-12-27 05:43:07,614][20347] Updated weights for policy 0, policy_version 7414 (0.0008) [2023-12-27 05:43:07,667][20347] Updated weights for policy 0, policy_version 7424 (0.0006) [2023-12-27 05:43:07,786][20381] Updated weights for policy 1, policy_version 7457 (0.0009) [2023-12-27 05:43:07,837][20381] Updated weights for policy 1, policy_version 7467 (0.0005) [2023-12-27 05:43:07,893][20381] Updated weights for policy 1, policy_version 7477 (0.0005) [2023-12-27 05:43:07,949][20381] Updated weights for policy 1, policy_version 7487 (0.0005) [2023-12-27 05:43:08,332][20347] Updated weights for policy 0, policy_version 7434 (0.0007) [2023-12-27 05:43:08,392][20347] Updated weights for policy 0, policy_version 7444 (0.0008) [2023-12-27 05:43:08,444][20347] Updated weights for policy 0, policy_version 7454 (0.0007) [2023-12-27 05:43:08,603][20381] Updated weights for policy 1, policy_version 7497 (0.0008) [2023-12-27 05:43:08,650][20381] Updated weights for policy 1, policy_version 7507 (0.0008) [2023-12-27 05:43:08,698][20381] Updated weights for policy 1, policy_version 7517 (0.0006) [2023-12-27 05:43:09,068][20347] Updated weights for policy 0, policy_version 7464 (0.0008) [2023-12-27 05:43:09,127][20347] Updated weights for policy 0, policy_version 7474 (0.0005) [2023-12-27 05:43:09,185][20347] Updated weights for policy 0, policy_version 7484 (0.0005) [2023-12-27 05:43:09,469][20381] Updated weights for policy 1, policy_version 7527 (0.0007) [2023-12-27 05:43:09,532][20381] Updated weights for policy 1, policy_version 7537 (0.0008) [2023-12-27 05:43:09,596][20381] Updated weights for policy 1, policy_version 7547 (0.0008) [2023-12-27 05:43:09,829][20347] Updated weights for policy 0, policy_version 7494 (0.0007) [2023-12-27 05:43:09,891][20347] Updated weights for policy 0, policy_version 7504 (0.0008) [2023-12-27 05:43:09,966][20347] Updated weights for policy 0, policy_version 7514 (0.0007) [2023-12-27 05:43:10,327][20381] Updated weights for policy 1, policy_version 7557 (0.0008) [2023-12-27 05:43:10,392][20381] Updated weights for policy 1, policy_version 7567 (0.0008) [2023-12-27 05:43:10,460][20381] Updated weights for policy 1, policy_version 7577 (0.0008) [2023-12-27 05:43:10,557][20347] Updated weights for policy 0, policy_version 7524 (0.0007) [2023-12-27 05:43:10,614][20347] Updated weights for policy 0, policy_version 7534 (0.0005) [2023-12-27 05:43:10,668][20347] Updated weights for policy 0, policy_version 7544 (0.0006) [2023-12-27 05:43:11,162][20381] Updated weights for policy 1, policy_version 7587 (0.0007) [2023-12-27 05:43:11,213][20381] Updated weights for policy 1, policy_version 7597 (0.0008) [2023-12-27 05:43:11,269][20381] Updated weights for policy 1, policy_version 7607 (0.0008) [2023-12-27 05:43:11,287][20347] Updated weights for policy 0, policy_version 7554 (0.0006) [2023-12-27 05:43:11,346][20347] Updated weights for policy 0, policy_version 7564 (0.0007) [2023-12-27 05:43:11,409][20347] Updated weights for policy 0, policy_version 7574 (0.0008) [2023-12-27 05:43:11,462][20347] Updated weights for policy 0, policy_version 7584 (0.0007) [2023-12-27 05:43:12,030][20381] Updated weights for policy 1, policy_version 7617 (0.0007) [2023-12-27 05:43:12,087][20381] Updated weights for policy 1, policy_version 7627 (0.0005) [2023-12-27 05:43:12,144][20381] Updated weights for policy 1, policy_version 7637 (0.0007) [2023-12-27 05:43:12,182][20347] Updated weights for policy 0, policy_version 7594 (0.0009) [2023-12-27 05:43:12,189][20381] Updated weights for policy 1, policy_version 7647 (0.0009) [2023-12-27 05:43:12,233][20347] Updated weights for policy 0, policy_version 7604 (0.0005) [2023-12-27 05:43:12,288][20347] Updated weights for policy 0, policy_version 7614 (0.0009) [2023-12-27 05:43:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20566.2). Total num frames: 3907584. Throughput: 0: 10365.1, 1: 10486.4. Samples: 3892432. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 05:43:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:12,894][20381] Updated weights for policy 1, policy_version 7657 (0.0006) [2023-12-27 05:43:12,951][20381] Updated weights for policy 1, policy_version 7667 (0.0008) [2023-12-27 05:43:12,954][20347] Updated weights for policy 0, policy_version 7624 (0.0006) [2023-12-27 05:43:13,010][20381] Updated weights for policy 1, policy_version 7677 (0.0008) [2023-12-27 05:43:13,013][20347] Updated weights for policy 0, policy_version 7634 (0.0005) [2023-12-27 05:43:13,064][20347] Updated weights for policy 0, policy_version 7644 (0.0005) [2023-12-27 05:43:13,682][20347] Updated weights for policy 0, policy_version 7654 (0.0007) [2023-12-27 05:43:13,714][20381] Updated weights for policy 1, policy_version 7687 (0.0006) [2023-12-27 05:43:13,734][20347] Updated weights for policy 0, policy_version 7664 (0.0007) [2023-12-27 05:43:13,759][20381] Updated weights for policy 1, policy_version 7697 (0.0005) [2023-12-27 05:43:13,783][20347] Updated weights for policy 0, policy_version 7674 (0.0006) [2023-12-27 05:43:13,807][20381] Updated weights for policy 1, policy_version 7707 (0.0005) [2023-12-27 05:43:14,441][20347] Updated weights for policy 0, policy_version 7684 (0.0006) [2023-12-27 05:43:14,493][20347] Updated weights for policy 0, policy_version 7694 (0.0006) [2023-12-27 05:43:14,526][20381] Updated weights for policy 1, policy_version 7717 (0.0006) [2023-12-27 05:43:14,550][20347] Updated weights for policy 0, policy_version 7704 (0.0006) [2023-12-27 05:43:14,580][20381] Updated weights for policy 1, policy_version 7727 (0.0008) [2023-12-27 05:43:14,637][20381] Updated weights for policy 1, policy_version 7737 (0.0007) [2023-12-27 05:43:15,173][20347] Updated weights for policy 0, policy_version 7714 (0.0006) [2023-12-27 05:43:15,234][20347] Updated weights for policy 0, policy_version 7724 (0.0010) [2023-12-27 05:43:15,296][20347] Updated weights for policy 0, policy_version 7734 (0.0010) [2023-12-27 05:43:15,324][20381] Updated weights for policy 1, policy_version 7747 (0.0008) [2023-12-27 05:43:15,350][20347] Updated weights for policy 0, policy_version 7744 (0.0006) [2023-12-27 05:43:15,375][20381] Updated weights for policy 1, policy_version 7757 (0.0007) [2023-12-27 05:43:15,427][20381] Updated weights for policy 1, policy_version 7767 (0.0008) [2023-12-27 05:43:15,934][20347] Updated weights for policy 0, policy_version 7754 (0.0005) [2023-12-27 05:43:15,986][20347] Updated weights for policy 0, policy_version 7764 (0.0006) [2023-12-27 05:43:16,044][20347] Updated weights for policy 0, policy_version 7774 (0.0007) [2023-12-27 05:43:16,128][20381] Updated weights for policy 1, policy_version 7777 (0.0007) [2023-12-27 05:43:16,176][20381] Updated weights for policy 1, policy_version 7787 (0.0008) [2023-12-27 05:43:16,221][20381] Updated weights for policy 1, policy_version 7797 (0.0008) [2023-12-27 05:43:16,265][20381] Updated weights for policy 1, policy_version 7807 (0.0008) [2023-12-27 05:43:16,625][20347] Updated weights for policy 0, policy_version 7784 (0.0006) [2023-12-27 05:43:16,681][20347] Updated weights for policy 0, policy_version 7794 (0.0005) [2023-12-27 05:43:16,736][20347] Updated weights for policy 0, policy_version 7804 (0.0009) [2023-12-27 05:43:17,010][20381] Updated weights for policy 1, policy_version 7817 (0.0006) [2023-12-27 05:43:17,066][20381] Updated weights for policy 1, policy_version 7827 (0.0008) [2023-12-27 05:43:17,118][20381] Updated weights for policy 1, policy_version 7837 (0.0010) [2023-12-27 05:43:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20543.0). Total num frames: 4005888. Throughput: 0: 10383.9, 1: 10399.2. Samples: 4016328. Policy #0 lag: (min: 31.0, avg: 45.3, max: 63.0) [2023-12-27 05:43:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:17,378][20347] Updated weights for policy 0, policy_version 7814 (0.0010) [2023-12-27 05:43:17,429][20347] Updated weights for policy 0, policy_version 7824 (0.0007) [2023-12-27 05:43:17,489][20347] Updated weights for policy 0, policy_version 7834 (0.0008) [2023-12-27 05:43:17,722][20381] Updated weights for policy 1, policy_version 7847 (0.0007) [2023-12-27 05:43:17,790][20381] Updated weights for policy 1, policy_version 7857 (0.0005) [2023-12-27 05:43:17,856][20381] Updated weights for policy 1, policy_version 7867 (0.0005) [2023-12-27 05:43:18,112][20347] Updated weights for policy 0, policy_version 7844 (0.0008) [2023-12-27 05:43:18,171][20347] Updated weights for policy 0, policy_version 7854 (0.0009) [2023-12-27 05:43:18,227][20347] Updated weights for policy 0, policy_version 7864 (0.0009) [2023-12-27 05:43:18,426][20381] Updated weights for policy 1, policy_version 7877 (0.0006) [2023-12-27 05:43:18,478][20381] Updated weights for policy 1, policy_version 7887 (0.0008) [2023-12-27 05:43:18,525][20381] Updated weights for policy 1, policy_version 7897 (0.0008) [2023-12-27 05:43:18,980][20347] Updated weights for policy 0, policy_version 7874 (0.0008) [2023-12-27 05:43:19,041][20347] Updated weights for policy 0, policy_version 7884 (0.0008) [2023-12-27 05:43:19,112][20347] Updated weights for policy 0, policy_version 7894 (0.0008) [2023-12-27 05:43:19,172][20347] Updated weights for policy 0, policy_version 7904 (0.0007) [2023-12-27 05:43:19,248][20381] Updated weights for policy 1, policy_version 7907 (0.0007) [2023-12-27 05:43:19,310][20381] Updated weights for policy 1, policy_version 7917 (0.0008) [2023-12-27 05:43:19,379][20381] Updated weights for policy 1, policy_version 7927 (0.0007) [2023-12-27 05:43:19,890][20347] Updated weights for policy 0, policy_version 7914 (0.0011) [2023-12-27 05:43:19,958][20347] Updated weights for policy 0, policy_version 7924 (0.0008) [2023-12-27 05:43:20,022][20347] Updated weights for policy 0, policy_version 7934 (0.0006) [2023-12-27 05:43:20,051][20381] Updated weights for policy 1, policy_version 7937 (0.0008) [2023-12-27 05:43:20,114][20381] Updated weights for policy 1, policy_version 7947 (0.0008) [2023-12-27 05:43:20,166][20381] Updated weights for policy 1, policy_version 7957 (0.0007) [2023-12-27 05:43:20,221][20381] Updated weights for policy 1, policy_version 7967 (0.0007) [2023-12-27 05:43:20,693][20347] Updated weights for policy 0, policy_version 7944 (0.0007) [2023-12-27 05:43:20,752][20347] Updated weights for policy 0, policy_version 7954 (0.0008) [2023-12-27 05:43:20,812][20347] Updated weights for policy 0, policy_version 7964 (0.0007) [2023-12-27 05:43:20,971][20381] Updated weights for policy 1, policy_version 7977 (0.0008) [2023-12-27 05:43:21,033][20381] Updated weights for policy 1, policy_version 7987 (0.0008) [2023-12-27 05:43:21,096][20381] Updated weights for policy 1, policy_version 7997 (0.0008) [2023-12-27 05:43:21,580][20347] Updated weights for policy 0, policy_version 7974 (0.0007) [2023-12-27 05:43:21,641][20347] Updated weights for policy 0, policy_version 7984 (0.0008) [2023-12-27 05:43:21,706][20347] Updated weights for policy 0, policy_version 7994 (0.0006) [2023-12-27 05:43:21,818][20381] Updated weights for policy 1, policy_version 8007 (0.0009) [2023-12-27 05:43:21,880][20381] Updated weights for policy 1, policy_version 8017 (0.0008) [2023-12-27 05:43:21,940][20381] Updated weights for policy 1, policy_version 8027 (0.0008) [2023-12-27 05:43:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20520.9). Total num frames: 4104192. Throughput: 0: 10380.9, 1: 10390.8. Samples: 4077632. Policy #0 lag: (min: 31.0, avg: 45.3, max: 63.0) [2023-12-27 05:43:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000008032_2056192.pth... [2023-12-27 05:43:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000006848_1753088.pth [2023-12-27 05:43:22,382][20347] Updated weights for policy 0, policy_version 8004 (0.0008) [2023-12-27 05:43:22,438][20347] Updated weights for policy 0, policy_version 8014 (0.0008) [2023-12-27 05:43:22,499][20347] Updated weights for policy 0, policy_version 8024 (0.0007) [2023-12-27 05:43:22,551][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000008032_2056192.pth... [2023-12-27 05:43:22,555][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000006816_1744896.pth [2023-12-27 05:43:22,604][20381] Updated weights for policy 1, policy_version 8037 (0.0007) [2023-12-27 05:43:22,659][20381] Updated weights for policy 1, policy_version 8047 (0.0006) [2023-12-27 05:43:22,710][20381] Updated weights for policy 1, policy_version 8057 (0.0007) [2023-12-27 05:43:23,247][20347] Updated weights for policy 0, policy_version 8034 (0.0008) [2023-12-27 05:43:23,298][20347] Updated weights for policy 0, policy_version 8044 (0.0007) [2023-12-27 05:43:23,344][20381] Updated weights for policy 1, policy_version 8067 (0.0007) [2023-12-27 05:43:23,351][20347] Updated weights for policy 0, policy_version 8054 (0.0007) [2023-12-27 05:43:23,397][20347] Updated weights for policy 0, policy_version 8064 (0.0007) [2023-12-27 05:43:23,399][20381] Updated weights for policy 1, policy_version 8077 (0.0006) [2023-12-27 05:43:23,450][20381] Updated weights for policy 1, policy_version 8087 (0.0007) [2023-12-27 05:43:24,098][20347] Updated weights for policy 0, policy_version 8074 (0.0006) [2023-12-27 05:43:24,138][20381] Updated weights for policy 1, policy_version 8097 (0.0008) [2023-12-27 05:43:24,153][20347] Updated weights for policy 0, policy_version 8084 (0.0008) [2023-12-27 05:43:24,194][20381] Updated weights for policy 1, policy_version 8107 (0.0010) [2023-12-27 05:43:24,205][20347] Updated weights for policy 0, policy_version 8094 (0.0008) [2023-12-27 05:43:24,254][20381] Updated weights for policy 1, policy_version 8117 (0.0009) [2023-12-27 05:43:24,303][20381] Updated weights for policy 1, policy_version 8127 (0.0008) [2023-12-27 05:43:24,807][20347] Updated weights for policy 0, policy_version 8104 (0.0006) [2023-12-27 05:43:24,870][20347] Updated weights for policy 0, policy_version 8114 (0.0006) [2023-12-27 05:43:24,926][20347] Updated weights for policy 0, policy_version 8124 (0.0005) [2023-12-27 05:43:24,991][20381] Updated weights for policy 1, policy_version 8137 (0.0006) [2023-12-27 05:43:25,041][20381] Updated weights for policy 1, policy_version 8147 (0.0008) [2023-12-27 05:43:25,091][20381] Updated weights for policy 1, policy_version 8157 (0.0007) [2023-12-27 05:43:25,534][20347] Updated weights for policy 0, policy_version 8134 (0.0010) [2023-12-27 05:43:25,583][20347] Updated weights for policy 0, policy_version 8144 (0.0010) [2023-12-27 05:43:25,634][20347] Updated weights for policy 0, policy_version 8154 (0.0009) [2023-12-27 05:43:25,795][20381] Updated weights for policy 1, policy_version 8167 (0.0008) [2023-12-27 05:43:25,858][20381] Updated weights for policy 1, policy_version 8177 (0.0008) [2023-12-27 05:43:25,913][20381] Updated weights for policy 1, policy_version 8187 (0.0007) [2023-12-27 05:43:26,322][20347] Updated weights for policy 0, policy_version 8164 (0.0009) [2023-12-27 05:43:26,373][20347] Updated weights for policy 0, policy_version 8174 (0.0010) [2023-12-27 05:43:26,423][20347] Updated weights for policy 0, policy_version 8184 (0.0009) [2023-12-27 05:43:26,529][20381] Updated weights for policy 1, policy_version 8197 (0.0006) [2023-12-27 05:43:26,574][20381] Updated weights for policy 1, policy_version 8207 (0.0005) [2023-12-27 05:43:26,617][20381] Updated weights for policy 1, policy_version 8217 (0.0005) [2023-12-27 05:43:27,069][20347] Updated weights for policy 0, policy_version 8194 (0.0008) [2023-12-27 05:43:27,117][20347] Updated weights for policy 0, policy_version 8204 (0.0010) [2023-12-27 05:43:27,161][20347] Updated weights for policy 0, policy_version 8214 (0.0010) [2023-12-27 05:43:27,203][20381] Updated weights for policy 1, policy_version 8227 (0.0007) [2023-12-27 05:43:27,205][20347] Updated weights for policy 0, policy_version 8224 (0.0010) [2023-12-27 05:43:27,254][20381] Updated weights for policy 1, policy_version 8237 (0.0010) [2023-12-27 05:43:27,315][20381] Updated weights for policy 1, policy_version 8247 (0.0010) [2023-12-27 05:43:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20539.9). Total num frames: 4210688. Throughput: 0: 10345.2, 1: 10324.9. Samples: 4199588. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 05:43:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:27,795][20347] Updated weights for policy 0, policy_version 8234 (0.0010) [2023-12-27 05:43:27,838][20347] Updated weights for policy 0, policy_version 8244 (0.0010) [2023-12-27 05:43:27,883][20347] Updated weights for policy 0, policy_version 8254 (0.0010) [2023-12-27 05:43:27,886][20381] Updated weights for policy 1, policy_version 8257 (0.0006) [2023-12-27 05:43:27,943][20381] Updated weights for policy 1, policy_version 8267 (0.0005) [2023-12-27 05:43:27,998][20381] Updated weights for policy 1, policy_version 8277 (0.0005) [2023-12-27 05:43:28,053][20381] Updated weights for policy 1, policy_version 8287 (0.0005) [2023-12-27 05:43:28,554][20347] Updated weights for policy 0, policy_version 8264 (0.0007) [2023-12-27 05:43:28,619][20347] Updated weights for policy 0, policy_version 8274 (0.0005) [2023-12-27 05:43:28,682][20381] Updated weights for policy 1, policy_version 8297 (0.0006) [2023-12-27 05:43:28,684][20347] Updated weights for policy 0, policy_version 8284 (0.0006) [2023-12-27 05:43:28,736][20381] Updated weights for policy 1, policy_version 8307 (0.0008) [2023-12-27 05:43:28,791][20381] Updated weights for policy 1, policy_version 8317 (0.0008) [2023-12-27 05:43:29,323][20347] Updated weights for policy 0, policy_version 8294 (0.0008) [2023-12-27 05:43:29,389][20347] Updated weights for policy 0, policy_version 8304 (0.0008) [2023-12-27 05:43:29,447][20347] Updated weights for policy 0, policy_version 8314 (0.0007) [2023-12-27 05:43:29,476][20381] Updated weights for policy 1, policy_version 8327 (0.0008) [2023-12-27 05:43:29,533][20381] Updated weights for policy 1, policy_version 8337 (0.0008) [2023-12-27 05:43:29,587][20381] Updated weights for policy 1, policy_version 8347 (0.0007) [2023-12-27 05:43:30,137][20347] Updated weights for policy 0, policy_version 8324 (0.0007) [2023-12-27 05:43:30,191][20347] Updated weights for policy 0, policy_version 8334 (0.0008) [2023-12-27 05:43:30,249][20347] Updated weights for policy 0, policy_version 8344 (0.0008) [2023-12-27 05:43:30,273][20381] Updated weights for policy 1, policy_version 8357 (0.0008) [2023-12-27 05:43:30,329][20381] Updated weights for policy 1, policy_version 8367 (0.0007) [2023-12-27 05:43:30,387][20381] Updated weights for policy 1, policy_version 8377 (0.0008) [2023-12-27 05:43:30,882][20347] Updated weights for policy 0, policy_version 8354 (0.0008) [2023-12-27 05:43:30,927][20347] Updated weights for policy 0, policy_version 8364 (0.0005) [2023-12-27 05:43:30,971][20347] Updated weights for policy 0, policy_version 8374 (0.0005) [2023-12-27 05:43:31,028][20347] Updated weights for policy 0, policy_version 8384 (0.0007) [2023-12-27 05:43:31,054][20381] Updated weights for policy 1, policy_version 8387 (0.0007) [2023-12-27 05:43:31,113][20381] Updated weights for policy 1, policy_version 8397 (0.0008) [2023-12-27 05:43:31,179][20381] Updated weights for policy 1, policy_version 8407 (0.0007) [2023-12-27 05:43:31,786][20347] Updated weights for policy 0, policy_version 8394 (0.0008) [2023-12-27 05:43:31,828][20381] Updated weights for policy 1, policy_version 8417 (0.0007) [2023-12-27 05:43:31,838][20347] Updated weights for policy 0, policy_version 8404 (0.0007) [2023-12-27 05:43:31,884][20381] Updated weights for policy 1, policy_version 8427 (0.0007) [2023-12-27 05:43:31,893][20347] Updated weights for policy 0, policy_version 8414 (0.0006) [2023-12-27 05:43:31,941][20381] Updated weights for policy 1, policy_version 8437 (0.0007) [2023-12-27 05:43:31,993][20381] Updated weights for policy 1, policy_version 8447 (0.0007) [2023-12-27 05:43:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20558.0). Total num frames: 4317184. Throughput: 0: 10391.8, 1: 10363.0. Samples: 4327348. Policy #0 lag: (min: 6.0, avg: 30.6, max: 32.0) [2023-12-27 05:43:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:32,593][20347] Updated weights for policy 0, policy_version 8424 (0.0005) [2023-12-27 05:43:32,654][20347] Updated weights for policy 0, policy_version 8434 (0.0005) [2023-12-27 05:43:32,706][20347] Updated weights for policy 0, policy_version 8444 (0.0006) [2023-12-27 05:43:32,708][20381] Updated weights for policy 1, policy_version 8457 (0.0006) [2023-12-27 05:43:32,762][20381] Updated weights for policy 1, policy_version 8467 (0.0007) [2023-12-27 05:43:32,818][20381] Updated weights for policy 1, policy_version 8477 (0.0007) [2023-12-27 05:43:33,301][20347] Updated weights for policy 0, policy_version 8454 (0.0007) [2023-12-27 05:43:33,349][20347] Updated weights for policy 0, policy_version 8464 (0.0008) [2023-12-27 05:43:33,410][20347] Updated weights for policy 0, policy_version 8474 (0.0008) [2023-12-27 05:43:33,525][20381] Updated weights for policy 1, policy_version 8487 (0.0006) [2023-12-27 05:43:33,573][20381] Updated weights for policy 1, policy_version 8497 (0.0007) [2023-12-27 05:43:33,623][20381] Updated weights for policy 1, policy_version 8507 (0.0008) [2023-12-27 05:43:34,059][20347] Updated weights for policy 0, policy_version 8484 (0.0007) [2023-12-27 05:43:34,113][20347] Updated weights for policy 0, policy_version 8494 (0.0005) [2023-12-27 05:43:34,172][20347] Updated weights for policy 0, policy_version 8504 (0.0006) [2023-12-27 05:43:34,393][20381] Updated weights for policy 1, policy_version 8517 (0.0008) [2023-12-27 05:43:34,443][20381] Updated weights for policy 1, policy_version 8527 (0.0008) [2023-12-27 05:43:34,497][20381] Updated weights for policy 1, policy_version 8537 (0.0005) [2023-12-27 05:43:34,837][20347] Updated weights for policy 0, policy_version 8514 (0.0007) [2023-12-27 05:43:34,883][20347] Updated weights for policy 0, policy_version 8524 (0.0005) [2023-12-27 05:43:34,931][20347] Updated weights for policy 0, policy_version 8534 (0.0005) [2023-12-27 05:43:34,984][20347] Updated weights for policy 0, policy_version 8544 (0.0009) [2023-12-27 05:43:35,151][20381] Updated weights for policy 1, policy_version 8547 (0.0006) [2023-12-27 05:43:35,205][20381] Updated weights for policy 1, policy_version 8557 (0.0007) [2023-12-27 05:43:35,252][20381] Updated weights for policy 1, policy_version 8567 (0.0005) [2023-12-27 05:43:35,622][20347] Updated weights for policy 0, policy_version 8554 (0.0009) [2023-12-27 05:43:35,670][20347] Updated weights for policy 0, policy_version 8564 (0.0008) [2023-12-27 05:43:35,714][20347] Updated weights for policy 0, policy_version 8574 (0.0007) [2023-12-27 05:43:35,938][20381] Updated weights for policy 1, policy_version 8577 (0.0007) [2023-12-27 05:43:36,002][20381] Updated weights for policy 1, policy_version 8587 (0.0010) [2023-12-27 05:43:36,065][20381] Updated weights for policy 1, policy_version 8597 (0.0010) [2023-12-27 05:43:36,131][20381] Updated weights for policy 1, policy_version 8607 (0.0010) [2023-12-27 05:43:36,376][20347] Updated weights for policy 0, policy_version 8584 (0.0010) [2023-12-27 05:43:36,436][20347] Updated weights for policy 0, policy_version 8594 (0.0011) [2023-12-27 05:43:36,496][20347] Updated weights for policy 0, policy_version 8604 (0.0010) [2023-12-27 05:43:36,818][20381] Updated weights for policy 1, policy_version 8617 (0.0008) [2023-12-27 05:43:36,873][20381] Updated weights for policy 1, policy_version 8627 (0.0008) [2023-12-27 05:43:36,935][20381] Updated weights for policy 1, policy_version 8637 (0.0008) [2023-12-27 05:43:37,178][20347] Updated weights for policy 0, policy_version 8614 (0.0010) [2023-12-27 05:43:37,227][20347] Updated weights for policy 0, policy_version 8624 (0.0010) [2023-12-27 05:43:37,279][20347] Updated weights for policy 0, policy_version 8634 (0.0010) [2023-12-27 05:43:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20575.2). Total num frames: 4423680. Throughput: 0: 10408.4, 1: 10304.8. Samples: 4389224. Policy #0 lag: (min: 6.0, avg: 30.6, max: 32.0) [2023-12-27 05:43:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000008640_2211840.pth... [2023-12-27 05:43:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000008640_2211840.pth... [2023-12-27 05:43:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000007392_1892352.pth [2023-12-27 05:43:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000007456_1908736.pth [2023-12-27 05:43:37,645][20381] Updated weights for policy 1, policy_version 8647 (0.0006) [2023-12-27 05:43:37,693][20381] Updated weights for policy 1, policy_version 8657 (0.0005) [2023-12-27 05:43:37,741][20381] Updated weights for policy 1, policy_version 8667 (0.0008) [2023-12-27 05:43:37,930][20347] Updated weights for policy 0, policy_version 8644 (0.0009) [2023-12-27 05:43:37,981][20347] Updated weights for policy 0, policy_version 8654 (0.0008) [2023-12-27 05:43:38,029][20347] Updated weights for policy 0, policy_version 8664 (0.0006) [2023-12-27 05:43:38,395][20381] Updated weights for policy 1, policy_version 8677 (0.0010) [2023-12-27 05:43:38,458][20381] Updated weights for policy 1, policy_version 8687 (0.0010) [2023-12-27 05:43:38,541][20381] Updated weights for policy 1, policy_version 8697 (0.0006) [2023-12-27 05:43:38,625][20347] Updated weights for policy 0, policy_version 8674 (0.0005) [2023-12-27 05:43:38,684][20347] Updated weights for policy 0, policy_version 8684 (0.0005) [2023-12-27 05:43:38,740][20347] Updated weights for policy 0, policy_version 8694 (0.0005) [2023-12-27 05:43:38,791][20347] Updated weights for policy 0, policy_version 8704 (0.0005) [2023-12-27 05:43:39,141][20381] Updated weights for policy 1, policy_version 8707 (0.0007) [2023-12-27 05:43:39,195][20381] Updated weights for policy 1, policy_version 8717 (0.0005) [2023-12-27 05:43:39,253][20381] Updated weights for policy 1, policy_version 8727 (0.0008) [2023-12-27 05:43:39,453][20347] Updated weights for policy 0, policy_version 8714 (0.0007) [2023-12-27 05:43:39,518][20347] Updated weights for policy 0, policy_version 8724 (0.0008) [2023-12-27 05:43:39,580][20347] Updated weights for policy 0, policy_version 8734 (0.0008) [2023-12-27 05:43:39,883][20381] Updated weights for policy 1, policy_version 8737 (0.0007) [2023-12-27 05:43:39,935][20381] Updated weights for policy 1, policy_version 8747 (0.0011) [2023-12-27 05:43:39,994][20381] Updated weights for policy 1, policy_version 8757 (0.0011) [2023-12-27 05:43:40,055][20381] Updated weights for policy 1, policy_version 8767 (0.0011) [2023-12-27 05:43:40,269][20347] Updated weights for policy 0, policy_version 8744 (0.0007) [2023-12-27 05:43:40,324][20347] Updated weights for policy 0, policy_version 8754 (0.0007) [2023-12-27 05:43:40,380][20347] Updated weights for policy 0, policy_version 8764 (0.0007) [2023-12-27 05:43:40,778][20381] Updated weights for policy 1, policy_version 8777 (0.0011) [2023-12-27 05:43:40,826][20381] Updated weights for policy 1, policy_version 8787 (0.0010) [2023-12-27 05:43:40,877][20381] Updated weights for policy 1, policy_version 8797 (0.0010) [2023-12-27 05:43:41,107][20347] Updated weights for policy 0, policy_version 8774 (0.0007) [2023-12-27 05:43:41,169][20347] Updated weights for policy 0, policy_version 8784 (0.0008) [2023-12-27 05:43:41,217][20347] Updated weights for policy 0, policy_version 8794 (0.0007) [2023-12-27 05:43:41,650][20381] Updated weights for policy 1, policy_version 8807 (0.0008) [2023-12-27 05:43:41,716][20381] Updated weights for policy 1, policy_version 8817 (0.0008) [2023-12-27 05:43:41,786][20381] Updated weights for policy 1, policy_version 8827 (0.0009) [2023-12-27 05:43:41,864][20347] Updated weights for policy 0, policy_version 8804 (0.0007) [2023-12-27 05:43:41,920][20347] Updated weights for policy 0, policy_version 8814 (0.0008) [2023-12-27 05:43:41,984][20347] Updated weights for policy 0, policy_version 8824 (0.0006) [2023-12-27 05:43:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20554.5). Total num frames: 4521984. Throughput: 0: 10431.5, 1: 10274.1. Samples: 4514260. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 05:43:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:42,488][20381] Updated weights for policy 1, policy_version 8837 (0.0008) [2023-12-27 05:43:42,556][20381] Updated weights for policy 1, policy_version 8847 (0.0006) [2023-12-27 05:43:42,612][20381] Updated weights for policy 1, policy_version 8857 (0.0008) [2023-12-27 05:43:42,631][20347] Updated weights for policy 0, policy_version 8834 (0.0006) [2023-12-27 05:43:42,684][20347] Updated weights for policy 0, policy_version 8844 (0.0005) [2023-12-27 05:43:42,741][20347] Updated weights for policy 0, policy_version 8854 (0.0005) [2023-12-27 05:43:42,798][20347] Updated weights for policy 0, policy_version 8864 (0.0005) [2023-12-27 05:43:43,277][20381] Updated weights for policy 1, policy_version 8867 (0.0008) [2023-12-27 05:43:43,326][20381] Updated weights for policy 1, policy_version 8877 (0.0008) [2023-12-27 05:43:43,373][20381] Updated weights for policy 1, policy_version 8887 (0.0009) [2023-12-27 05:43:43,444][20347] Updated weights for policy 0, policy_version 8874 (0.0009) [2023-12-27 05:43:43,500][20347] Updated weights for policy 0, policy_version 8884 (0.0010) [2023-12-27 05:43:43,556][20347] Updated weights for policy 0, policy_version 8894 (0.0005) [2023-12-27 05:43:44,041][20381] Updated weights for policy 1, policy_version 8897 (0.0010) [2023-12-27 05:43:44,098][20381] Updated weights for policy 1, policy_version 8907 (0.0007) [2023-12-27 05:43:44,156][20381] Updated weights for policy 1, policy_version 8917 (0.0005) [2023-12-27 05:43:44,168][20347] Updated weights for policy 0, policy_version 8904 (0.0009) [2023-12-27 05:43:44,207][20381] Updated weights for policy 1, policy_version 8927 (0.0005) [2023-12-27 05:43:44,216][20347] Updated weights for policy 0, policy_version 8914 (0.0009) [2023-12-27 05:43:44,265][20347] Updated weights for policy 0, policy_version 8924 (0.0010) [2023-12-27 05:43:44,863][20381] Updated weights for policy 1, policy_version 8937 (0.0008) [2023-12-27 05:43:44,894][20347] Updated weights for policy 0, policy_version 8934 (0.0010) [2023-12-27 05:43:44,921][20381] Updated weights for policy 1, policy_version 8947 (0.0006) [2023-12-27 05:43:44,958][20347] Updated weights for policy 0, policy_version 8944 (0.0009) [2023-12-27 05:43:44,977][20381] Updated weights for policy 1, policy_version 8957 (0.0007) [2023-12-27 05:43:45,015][20347] Updated weights for policy 0, policy_version 8954 (0.0008) [2023-12-27 05:43:45,675][20381] Updated weights for policy 1, policy_version 8967 (0.0007) [2023-12-27 05:43:45,703][20347] Updated weights for policy 0, policy_version 8964 (0.0009) [2023-12-27 05:43:45,725][20381] Updated weights for policy 1, policy_version 8977 (0.0006) [2023-12-27 05:43:45,749][20347] Updated weights for policy 0, policy_version 8974 (0.0010) [2023-12-27 05:43:45,777][20381] Updated weights for policy 1, policy_version 8987 (0.0005) [2023-12-27 05:43:45,795][20347] Updated weights for policy 0, policy_version 8984 (0.0010) [2023-12-27 05:43:46,478][20347] Updated weights for policy 0, policy_version 8994 (0.0009) [2023-12-27 05:43:46,479][20381] Updated weights for policy 1, policy_version 8997 (0.0006) [2023-12-27 05:43:46,539][20381] Updated weights for policy 1, policy_version 9007 (0.0008) [2023-12-27 05:43:46,543][20347] Updated weights for policy 0, policy_version 9004 (0.0005) [2023-12-27 05:43:46,596][20381] Updated weights for policy 1, policy_version 9017 (0.0007) [2023-12-27 05:43:46,600][20347] Updated weights for policy 0, policy_version 9014 (0.0006) [2023-12-27 05:43:46,663][20347] Updated weights for policy 0, policy_version 9024 (0.0006) [2023-12-27 05:43:47,264][20381] Updated weights for policy 1, policy_version 9027 (0.0007) [2023-12-27 05:43:47,287][20347] Updated weights for policy 0, policy_version 9034 (0.0007) [2023-12-27 05:43:47,325][20381] Updated weights for policy 1, policy_version 9037 (0.0006) [2023-12-27 05:43:47,342][20347] Updated weights for policy 0, policy_version 9044 (0.0005) [2023-12-27 05:43:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.6, 300 sec: 20534.6). Total num frames: 4620288. Throughput: 0: 10504.1, 1: 10253.3. Samples: 4638724. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 05:43:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:47,383][20381] Updated weights for policy 1, policy_version 9047 (0.0006) [2023-12-27 05:43:47,396][20347] Updated weights for policy 0, policy_version 9054 (0.0007) [2023-12-27 05:43:47,978][20347] Updated weights for policy 0, policy_version 9064 (0.0007) [2023-12-27 05:43:48,039][20347] Updated weights for policy 0, policy_version 9074 (0.0007) [2023-12-27 05:43:48,076][20381] Updated weights for policy 1, policy_version 9057 (0.0007) [2023-12-27 05:43:48,093][20347] Updated weights for policy 0, policy_version 9084 (0.0006) [2023-12-27 05:43:48,132][20381] Updated weights for policy 1, policy_version 9067 (0.0006) [2023-12-27 05:43:48,186][20381] Updated weights for policy 1, policy_version 9077 (0.0008) [2023-12-27 05:43:48,242][20381] Updated weights for policy 1, policy_version 9087 (0.0008) [2023-12-27 05:43:48,782][20347] Updated weights for policy 0, policy_version 9094 (0.0008) [2023-12-27 05:43:48,840][20347] Updated weights for policy 0, policy_version 9104 (0.0011) [2023-12-27 05:43:48,899][20347] Updated weights for policy 0, policy_version 9114 (0.0011) [2023-12-27 05:43:48,948][20381] Updated weights for policy 1, policy_version 9097 (0.0006) [2023-12-27 05:43:48,999][20381] Updated weights for policy 1, policy_version 9107 (0.0007) [2023-12-27 05:43:49,061][20381] Updated weights for policy 1, policy_version 9117 (0.0007) [2023-12-27 05:43:49,569][20347] Updated weights for policy 0, policy_version 9124 (0.0009) [2023-12-27 05:43:49,625][20347] Updated weights for policy 0, policy_version 9134 (0.0008) [2023-12-27 05:43:49,687][20347] Updated weights for policy 0, policy_version 9144 (0.0006) [2023-12-27 05:43:49,738][20381] Updated weights for policy 1, policy_version 9127 (0.0010) [2023-12-27 05:43:49,794][20381] Updated weights for policy 1, policy_version 9137 (0.0011) [2023-12-27 05:43:49,862][20381] Updated weights for policy 1, policy_version 9147 (0.0010) [2023-12-27 05:43:50,384][20347] Updated weights for policy 0, policy_version 9154 (0.0006) [2023-12-27 05:43:50,441][20347] Updated weights for policy 0, policy_version 9164 (0.0007) [2023-12-27 05:43:50,502][20347] Updated weights for policy 0, policy_version 9174 (0.0007) [2023-12-27 05:43:50,562][20381] Updated weights for policy 1, policy_version 9157 (0.0006) [2023-12-27 05:43:50,565][20347] Updated weights for policy 0, policy_version 9184 (0.0009) [2023-12-27 05:43:50,624][20381] Updated weights for policy 1, policy_version 9167 (0.0008) [2023-12-27 05:43:50,684][20381] Updated weights for policy 1, policy_version 9177 (0.0007) [2023-12-27 05:43:51,170][20347] Updated weights for policy 0, policy_version 9194 (0.0009) [2023-12-27 05:43:51,217][20347] Updated weights for policy 0, policy_version 9204 (0.0007) [2023-12-27 05:43:51,274][20347] Updated weights for policy 0, policy_version 9214 (0.0006) [2023-12-27 05:43:51,433][20381] Updated weights for policy 1, policy_version 9187 (0.0008) [2023-12-27 05:43:51,491][20381] Updated weights for policy 1, policy_version 9197 (0.0011) [2023-12-27 05:43:51,553][20381] Updated weights for policy 1, policy_version 9207 (0.0011) [2023-12-27 05:43:51,920][20347] Updated weights for policy 0, policy_version 9224 (0.0008) [2023-12-27 05:43:51,979][20347] Updated weights for policy 0, policy_version 9234 (0.0007) [2023-12-27 05:43:52,038][20347] Updated weights for policy 0, policy_version 9244 (0.0005) [2023-12-27 05:43:52,186][20381] Updated weights for policy 1, policy_version 9217 (0.0010) [2023-12-27 05:43:52,245][20381] Updated weights for policy 1, policy_version 9227 (0.0010) [2023-12-27 05:43:52,305][20381] Updated weights for policy 1, policy_version 9237 (0.0011) [2023-12-27 05:43:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20551.2). Total num frames: 4726784. Throughput: 0: 10505.6, 1: 10204.0. Samples: 4699996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:43:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000009248_2367488.pth... [2023-12-27 05:43:52,369][20381] Updated weights for policy 1, policy_version 9247 (0.0009) [2023-12-27 05:43:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000008032_2056192.pth [2023-12-27 05:43:52,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000009248_2367488.pth... [2023-12-27 05:43:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000008032_2056192.pth [2023-12-27 05:43:52,686][20347] Updated weights for policy 0, policy_version 9254 (0.0009) [2023-12-27 05:43:52,741][20347] Updated weights for policy 0, policy_version 9264 (0.0011) [2023-12-27 05:43:52,799][20347] Updated weights for policy 0, policy_version 9274 (0.0010) [2023-12-27 05:43:52,992][20381] Updated weights for policy 1, policy_version 9257 (0.0006) [2023-12-27 05:43:53,037][20381] Updated weights for policy 1, policy_version 9267 (0.0005) [2023-12-27 05:43:53,093][20381] Updated weights for policy 1, policy_version 9277 (0.0005) [2023-12-27 05:43:53,462][20347] Updated weights for policy 0, policy_version 9284 (0.0008) [2023-12-27 05:43:53,526][20347] Updated weights for policy 0, policy_version 9294 (0.0007) [2023-12-27 05:43:53,589][20347] Updated weights for policy 0, policy_version 9304 (0.0010) [2023-12-27 05:43:53,702][20381] Updated weights for policy 1, policy_version 9287 (0.0005) [2023-12-27 05:43:53,748][20381] Updated weights for policy 1, policy_version 9297 (0.0005) [2023-12-27 05:43:53,796][20381] Updated weights for policy 1, policy_version 9307 (0.0008) [2023-12-27 05:43:54,108][20347] Updated weights for policy 0, policy_version 9314 (0.0009) [2023-12-27 05:43:54,164][20347] Updated weights for policy 0, policy_version 9324 (0.0006) [2023-12-27 05:43:54,209][20347] Updated weights for policy 0, policy_version 9334 (0.0010) [2023-12-27 05:43:54,254][20347] Updated weights for policy 0, policy_version 9344 (0.0006) [2023-12-27 05:43:54,444][20381] Updated weights for policy 1, policy_version 9317 (0.0009) [2023-12-27 05:43:54,502][20381] Updated weights for policy 1, policy_version 9327 (0.0010) [2023-12-27 05:43:54,551][20381] Updated weights for policy 1, policy_version 9337 (0.0005) [2023-12-27 05:43:54,873][20347] Updated weights for policy 0, policy_version 9354 (0.0008) [2023-12-27 05:43:54,922][20347] Updated weights for policy 0, policy_version 9364 (0.0010) [2023-12-27 05:43:54,980][20347] Updated weights for policy 0, policy_version 9374 (0.0009) [2023-12-27 05:43:55,166][20381] Updated weights for policy 1, policy_version 9347 (0.0007) [2023-12-27 05:43:55,223][20381] Updated weights for policy 1, policy_version 9357 (0.0007) [2023-12-27 05:43:55,281][20381] Updated weights for policy 1, policy_version 9367 (0.0007) [2023-12-27 05:43:55,624][20347] Updated weights for policy 0, policy_version 9384 (0.0008) [2023-12-27 05:43:55,677][20347] Updated weights for policy 0, policy_version 9394 (0.0008) [2023-12-27 05:43:55,732][20347] Updated weights for policy 0, policy_version 9404 (0.0008) [2023-12-27 05:43:55,861][20381] Updated weights for policy 1, policy_version 9377 (0.0008) [2023-12-27 05:43:55,914][20381] Updated weights for policy 1, policy_version 9387 (0.0007) [2023-12-27 05:43:55,980][20381] Updated weights for policy 1, policy_version 9397 (0.0008) [2023-12-27 05:43:56,035][20381] Updated weights for policy 1, policy_version 9407 (0.0007) [2023-12-27 05:43:56,433][20347] Updated weights for policy 0, policy_version 9414 (0.0008) [2023-12-27 05:43:56,484][20347] Updated weights for policy 0, policy_version 9424 (0.0007) [2023-12-27 05:43:56,538][20347] Updated weights for policy 0, policy_version 9434 (0.0005) [2023-12-27 05:43:56,709][20381] Updated weights for policy 1, policy_version 9417 (0.0008) [2023-12-27 05:43:56,765][20381] Updated weights for policy 1, policy_version 9427 (0.0007) [2023-12-27 05:43:56,812][20381] Updated weights for policy 1, policy_version 9437 (0.0007) [2023-12-27 05:43:57,178][20347] Updated weights for policy 0, policy_version 9444 (0.0006) [2023-12-27 05:43:57,228][20347] Updated weights for policy 0, policy_version 9454 (0.0007) [2023-12-27 05:43:57,277][20347] Updated weights for policy 0, policy_version 9464 (0.0007) [2023-12-27 05:43:57,363][19170] Fps is (10 sec: 22117.8, 60 sec: 20753.0, 300 sec: 20602.0). Total num frames: 4841472. Throughput: 0: 10546.7, 1: 10264.8. Samples: 4828952. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:43:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:43:57,494][20381] Updated weights for policy 1, policy_version 9447 (0.0010) [2023-12-27 05:43:57,550][20381] Updated weights for policy 1, policy_version 9457 (0.0011) [2023-12-27 05:43:57,602][20381] Updated weights for policy 1, policy_version 9467 (0.0010) [2023-12-27 05:43:57,911][20347] Updated weights for policy 0, policy_version 9474 (0.0006) [2023-12-27 05:43:57,968][20347] Updated weights for policy 0, policy_version 9484 (0.0005) [2023-12-27 05:43:58,025][20347] Updated weights for policy 0, policy_version 9494 (0.0005) [2023-12-27 05:43:58,069][20347] Updated weights for policy 0, policy_version 9504 (0.0005) [2023-12-27 05:43:58,237][20381] Updated weights for policy 1, policy_version 9477 (0.0010) [2023-12-27 05:43:58,296][20381] Updated weights for policy 1, policy_version 9487 (0.0011) [2023-12-27 05:43:58,359][20381] Updated weights for policy 1, policy_version 9497 (0.0009) [2023-12-27 05:43:58,727][20347] Updated weights for policy 0, policy_version 9514 (0.0008) [2023-12-27 05:43:58,791][20347] Updated weights for policy 0, policy_version 9524 (0.0008) [2023-12-27 05:43:58,856][20347] Updated weights for policy 0, policy_version 9534 (0.0008) [2023-12-27 05:43:59,078][20381] Updated weights for policy 1, policy_version 9507 (0.0009) [2023-12-27 05:43:59,136][20381] Updated weights for policy 1, policy_version 9517 (0.0008) [2023-12-27 05:43:59,189][20381] Updated weights for policy 1, policy_version 9527 (0.0007) [2023-12-27 05:43:59,607][20347] Updated weights for policy 0, policy_version 9544 (0.0007) [2023-12-27 05:43:59,677][20347] Updated weights for policy 0, policy_version 9554 (0.0006) [2023-12-27 05:43:59,727][20347] Updated weights for policy 0, policy_version 9564 (0.0007) [2023-12-27 05:43:59,863][20381] Updated weights for policy 1, policy_version 9537 (0.0008) [2023-12-27 05:43:59,921][20381] Updated weights for policy 1, policy_version 9547 (0.0007) [2023-12-27 05:43:59,978][20381] Updated weights for policy 1, policy_version 9557 (0.0008) [2023-12-27 05:44:00,038][20381] Updated weights for policy 1, policy_version 9567 (0.0008) [2023-12-27 05:44:00,417][20347] Updated weights for policy 0, policy_version 9574 (0.0006) [2023-12-27 05:44:00,465][20347] Updated weights for policy 0, policy_version 9584 (0.0006) [2023-12-27 05:44:00,511][20347] Updated weights for policy 0, policy_version 9594 (0.0006) [2023-12-27 05:44:00,717][20381] Updated weights for policy 1, policy_version 9577 (0.0008) [2023-12-27 05:44:00,773][20381] Updated weights for policy 1, policy_version 9587 (0.0008) [2023-12-27 05:44:00,823][20381] Updated weights for policy 1, policy_version 9597 (0.0008) [2023-12-27 05:44:01,177][20347] Updated weights for policy 0, policy_version 9604 (0.0007) [2023-12-27 05:44:01,241][20347] Updated weights for policy 0, policy_version 9614 (0.0007) [2023-12-27 05:44:01,305][20347] Updated weights for policy 0, policy_version 9624 (0.0007) [2023-12-27 05:44:01,471][20381] Updated weights for policy 1, policy_version 9607 (0.0006) [2023-12-27 05:44:01,518][20381] Updated weights for policy 1, policy_version 9617 (0.0005) [2023-12-27 05:44:01,565][20381] Updated weights for policy 1, policy_version 9627 (0.0005) [2023-12-27 05:44:02,038][20347] Updated weights for policy 0, policy_version 9634 (0.0007) [2023-12-27 05:44:02,099][20347] Updated weights for policy 0, policy_version 9644 (0.0006) [2023-12-27 05:44:02,148][20347] Updated weights for policy 0, policy_version 9654 (0.0007) [2023-12-27 05:44:02,189][20381] Updated weights for policy 1, policy_version 9637 (0.0008) [2023-12-27 05:44:02,195][20347] Updated weights for policy 0, policy_version 9664 (0.0006) [2023-12-27 05:44:02,244][20381] Updated weights for policy 1, policy_version 9647 (0.0010) [2023-12-27 05:44:02,303][20381] Updated weights for policy 1, policy_version 9657 (0.0011) [2023-12-27 05:44:02,362][19170] Fps is (10 sec: 22118.8, 60 sec: 20889.6, 300 sec: 20616.5). Total num frames: 4947968. Throughput: 0: 10452.5, 1: 10337.4. Samples: 4951876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:44:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:02,868][20347] Updated weights for policy 0, policy_version 9674 (0.0007) [2023-12-27 05:44:02,924][20347] Updated weights for policy 0, policy_version 9684 (0.0005) [2023-12-27 05:44:02,951][20381] Updated weights for policy 1, policy_version 9667 (0.0011) [2023-12-27 05:44:02,983][20347] Updated weights for policy 0, policy_version 9694 (0.0005) [2023-12-27 05:44:03,003][20381] Updated weights for policy 1, policy_version 9677 (0.0010) [2023-12-27 05:44:03,061][20381] Updated weights for policy 1, policy_version 9687 (0.0011) [2023-12-27 05:44:03,501][20347] Updated weights for policy 0, policy_version 9704 (0.0006) [2023-12-27 05:44:03,550][20347] Updated weights for policy 0, policy_version 9714 (0.0005) [2023-12-27 05:44:03,603][20347] Updated weights for policy 0, policy_version 9724 (0.0007) [2023-12-27 05:44:03,696][20381] Updated weights for policy 1, policy_version 9697 (0.0011) [2023-12-27 05:44:03,752][20381] Updated weights for policy 1, policy_version 9707 (0.0011) [2023-12-27 05:44:03,808][20381] Updated weights for policy 1, policy_version 9717 (0.0011) [2023-12-27 05:44:03,862][20381] Updated weights for policy 1, policy_version 9727 (0.0011) [2023-12-27 05:44:04,324][20347] Updated weights for policy 0, policy_version 9734 (0.0011) [2023-12-27 05:44:04,382][20347] Updated weights for policy 0, policy_version 9744 (0.0010) [2023-12-27 05:44:04,441][20347] Updated weights for policy 0, policy_version 9754 (0.0010) [2023-12-27 05:44:04,564][20381] Updated weights for policy 1, policy_version 9737 (0.0010) [2023-12-27 05:44:04,619][20381] Updated weights for policy 1, policy_version 9747 (0.0010) [2023-12-27 05:44:04,678][20381] Updated weights for policy 1, policy_version 9757 (0.0011) [2023-12-27 05:44:05,034][20347] Updated weights for policy 0, policy_version 9764 (0.0008) [2023-12-27 05:44:05,086][20347] Updated weights for policy 0, policy_version 9774 (0.0008) [2023-12-27 05:44:05,138][20347] Updated weights for policy 0, policy_version 9784 (0.0006) [2023-12-27 05:44:05,365][20381] Updated weights for policy 1, policy_version 9767 (0.0008) [2023-12-27 05:44:05,419][20381] Updated weights for policy 1, policy_version 9777 (0.0007) [2023-12-27 05:44:05,470][20381] Updated weights for policy 1, policy_version 9787 (0.0007) [2023-12-27 05:44:05,717][20347] Updated weights for policy 0, policy_version 9794 (0.0005) [2023-12-27 05:44:05,769][20347] Updated weights for policy 0, policy_version 9804 (0.0008) [2023-12-27 05:44:05,823][20347] Updated weights for policy 0, policy_version 9814 (0.0010) [2023-12-27 05:44:05,883][20347] Updated weights for policy 0, policy_version 9824 (0.0010) [2023-12-27 05:44:06,148][20381] Updated weights for policy 1, policy_version 9797 (0.0007) [2023-12-27 05:44:06,202][20381] Updated weights for policy 1, policy_version 9807 (0.0006) [2023-12-27 05:44:06,255][20381] Updated weights for policy 1, policy_version 9817 (0.0006) [2023-12-27 05:44:06,505][20347] Updated weights for policy 0, policy_version 9834 (0.0007) [2023-12-27 05:44:06,569][20347] Updated weights for policy 0, policy_version 9844 (0.0007) [2023-12-27 05:44:06,632][20347] Updated weights for policy 0, policy_version 9854 (0.0008) [2023-12-27 05:44:06,952][20381] Updated weights for policy 1, policy_version 9827 (0.0006) [2023-12-27 05:44:07,000][20381] Updated weights for policy 1, policy_version 9837 (0.0008) [2023-12-27 05:44:07,049][20381] Updated weights for policy 1, policy_version 9847 (0.0008) [2023-12-27 05:44:07,244][20347] Updated weights for policy 0, policy_version 9864 (0.0006) [2023-12-27 05:44:07,300][20347] Updated weights for policy 0, policy_version 9874 (0.0005) [2023-12-27 05:44:07,352][20347] Updated weights for policy 0, policy_version 9884 (0.0007) [2023-12-27 05:44:07,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20597.0). Total num frames: 5046272. Throughput: 0: 10526.5, 1: 10329.9. Samples: 5016168. Policy #0 lag: (min: 31.0, avg: 31.1, max: 35.0) [2023-12-27 05:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000009856_2523136.pth... [2023-12-27 05:44:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000008640_2211840.pth [2023-12-27 05:44:07,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000009888_2531328.pth... [2023-12-27 05:44:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000008640_2211840.pth [2023-12-27 05:44:07,725][20381] Updated weights for policy 1, policy_version 9857 (0.0009) [2023-12-27 05:44:07,774][20381] Updated weights for policy 1, policy_version 9867 (0.0005) [2023-12-27 05:44:07,827][20381] Updated weights for policy 1, policy_version 9877 (0.0006) [2023-12-27 05:44:07,883][20347] Updated weights for policy 0, policy_version 9894 (0.0006) [2023-12-27 05:44:07,886][20381] Updated weights for policy 1, policy_version 9887 (0.0007) [2023-12-27 05:44:07,946][20347] Updated weights for policy 0, policy_version 9904 (0.0005) [2023-12-27 05:44:08,002][20347] Updated weights for policy 0, policy_version 9914 (0.0005) [2023-12-27 05:44:08,523][20381] Updated weights for policy 1, policy_version 9897 (0.0010) [2023-12-27 05:44:08,575][20381] Updated weights for policy 1, policy_version 9907 (0.0010) [2023-12-27 05:44:08,623][20381] Updated weights for policy 1, policy_version 9917 (0.0010) [2023-12-27 05:44:08,667][20347] Updated weights for policy 0, policy_version 9924 (0.0008) [2023-12-27 05:44:08,721][20347] Updated weights for policy 0, policy_version 9934 (0.0010) [2023-12-27 05:44:08,772][20347] Updated weights for policy 0, policy_version 9944 (0.0010) [2023-12-27 05:44:09,247][20381] Updated weights for policy 1, policy_version 9927 (0.0010) [2023-12-27 05:44:09,299][20381] Updated weights for policy 1, policy_version 9937 (0.0008) [2023-12-27 05:44:09,354][20381] Updated weights for policy 1, policy_version 9947 (0.0008) [2023-12-27 05:44:09,436][20347] Updated weights for policy 0, policy_version 9954 (0.0010) [2023-12-27 05:44:09,500][20347] Updated weights for policy 0, policy_version 9964 (0.0008) [2023-12-27 05:44:09,564][20347] Updated weights for policy 0, policy_version 9974 (0.0007) [2023-12-27 05:44:09,613][20347] Updated weights for policy 0, policy_version 9984 (0.0008) [2023-12-27 05:44:09,984][20381] Updated weights for policy 1, policy_version 9957 (0.0009) [2023-12-27 05:44:10,044][20381] Updated weights for policy 1, policy_version 9967 (0.0008) [2023-12-27 05:44:10,106][20381] Updated weights for policy 1, policy_version 9977 (0.0007) [2023-12-27 05:44:10,338][20347] Updated weights for policy 0, policy_version 9994 (0.0007) [2023-12-27 05:44:10,394][20347] Updated weights for policy 0, policy_version 10004 (0.0007) [2023-12-27 05:44:10,443][20347] Updated weights for policy 0, policy_version 10014 (0.0007) [2023-12-27 05:44:10,794][20381] Updated weights for policy 1, policy_version 9987 (0.0007) [2023-12-27 05:44:10,854][20381] Updated weights for policy 1, policy_version 9997 (0.0006) [2023-12-27 05:44:10,914][20381] Updated weights for policy 1, policy_version 10007 (0.0006) [2023-12-27 05:44:11,152][20347] Updated weights for policy 0, policy_version 10024 (0.0008) [2023-12-27 05:44:11,223][20347] Updated weights for policy 0, policy_version 10034 (0.0007) [2023-12-27 05:44:11,287][20347] Updated weights for policy 0, policy_version 10044 (0.0008) [2023-12-27 05:44:11,576][20381] Updated weights for policy 1, policy_version 10017 (0.0007) [2023-12-27 05:44:11,645][20381] Updated weights for policy 1, policy_version 10027 (0.0007) [2023-12-27 05:44:11,702][20381] Updated weights for policy 1, policy_version 10037 (0.0008) [2023-12-27 05:44:11,767][20381] Updated weights for policy 1, policy_version 10047 (0.0008) [2023-12-27 05:44:12,065][20347] Updated weights for policy 0, policy_version 10054 (0.0008) [2023-12-27 05:44:12,119][20347] Updated weights for policy 0, policy_version 10064 (0.0007) [2023-12-27 05:44:12,178][20347] Updated weights for policy 0, policy_version 10074 (0.0005) [2023-12-27 05:44:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20611.1). Total num frames: 5152768. Throughput: 0: 10606.8, 1: 10382.8. Samples: 5144120. Policy #0 lag: (min: 4.0, avg: 5.0, max: 21.0) [2023-12-27 05:44:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:12,392][20381] Updated weights for policy 1, policy_version 10057 (0.0008) [2023-12-27 05:44:12,440][20381] Updated weights for policy 1, policy_version 10067 (0.0008) [2023-12-27 05:44:12,487][20381] Updated weights for policy 1, policy_version 10077 (0.0008) [2023-12-27 05:44:12,825][20347] Updated weights for policy 0, policy_version 10084 (0.0007) [2023-12-27 05:44:12,886][20347] Updated weights for policy 0, policy_version 10094 (0.0008) [2023-12-27 05:44:12,940][20347] Updated weights for policy 0, policy_version 10104 (0.0006) [2023-12-27 05:44:13,160][20381] Updated weights for policy 1, policy_version 10087 (0.0008) [2023-12-27 05:44:13,215][20381] Updated weights for policy 1, policy_version 10097 (0.0010) [2023-12-27 05:44:13,261][20381] Updated weights for policy 1, policy_version 10107 (0.0010) [2023-12-27 05:44:13,617][20347] Updated weights for policy 0, policy_version 10114 (0.0006) [2023-12-27 05:44:13,668][20347] Updated weights for policy 0, policy_version 10124 (0.0010) [2023-12-27 05:44:13,716][20347] Updated weights for policy 0, policy_version 10134 (0.0010) [2023-12-27 05:44:13,770][20347] Updated weights for policy 0, policy_version 10144 (0.0010) [2023-12-27 05:44:13,904][20381] Updated weights for policy 1, policy_version 10117 (0.0008) [2023-12-27 05:44:13,958][20381] Updated weights for policy 1, policy_version 10127 (0.0010) [2023-12-27 05:44:14,018][20381] Updated weights for policy 1, policy_version 10137 (0.0010) [2023-12-27 05:44:14,435][20347] Updated weights for policy 0, policy_version 10154 (0.0010) [2023-12-27 05:44:14,491][20347] Updated weights for policy 0, policy_version 10164 (0.0009) [2023-12-27 05:44:14,549][20347] Updated weights for policy 0, policy_version 10174 (0.0010) [2023-12-27 05:44:14,644][20381] Updated weights for policy 1, policy_version 10147 (0.0010) [2023-12-27 05:44:14,692][20381] Updated weights for policy 1, policy_version 10157 (0.0010) [2023-12-27 05:44:14,747][20381] Updated weights for policy 1, policy_version 10167 (0.0010) [2023-12-27 05:44:15,191][20347] Updated weights for policy 0, policy_version 10184 (0.0006) [2023-12-27 05:44:15,248][20347] Updated weights for policy 0, policy_version 10194 (0.0007) [2023-12-27 05:44:15,297][20347] Updated weights for policy 0, policy_version 10204 (0.0008) [2023-12-27 05:44:15,484][20381] Updated weights for policy 1, policy_version 10177 (0.0010) [2023-12-27 05:44:15,538][20381] Updated weights for policy 1, policy_version 10187 (0.0010) [2023-12-27 05:44:15,587][20381] Updated weights for policy 1, policy_version 10197 (0.0010) [2023-12-27 05:44:15,642][20381] Updated weights for policy 1, policy_version 10207 (0.0010) [2023-12-27 05:44:15,846][20347] Updated weights for policy 0, policy_version 10214 (0.0005) [2023-12-27 05:44:15,890][20347] Updated weights for policy 0, policy_version 10224 (0.0007) [2023-12-27 05:44:15,937][20347] Updated weights for policy 0, policy_version 10234 (0.0006) [2023-12-27 05:44:16,345][20381] Updated weights for policy 1, policy_version 10217 (0.0010) [2023-12-27 05:44:16,388][20381] Updated weights for policy 1, policy_version 10227 (0.0006) [2023-12-27 05:44:16,433][20381] Updated weights for policy 1, policy_version 10237 (0.0005) [2023-12-27 05:44:16,588][20347] Updated weights for policy 0, policy_version 10244 (0.0005) [2023-12-27 05:44:16,633][20347] Updated weights for policy 0, policy_version 10254 (0.0005) [2023-12-27 05:44:16,684][20347] Updated weights for policy 0, policy_version 10264 (0.0007) [2023-12-27 05:44:17,035][20381] Updated weights for policy 1, policy_version 10247 (0.0005) [2023-12-27 05:44:17,089][20381] Updated weights for policy 1, policy_version 10257 (0.0005) [2023-12-27 05:44:17,157][20381] Updated weights for policy 1, policy_version 10267 (0.0008) [2023-12-27 05:44:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20624.6). Total num frames: 5259264. Throughput: 0: 10597.1, 1: 10357.0. Samples: 5270280. Policy #0 lag: (min: 4.0, avg: 5.0, max: 21.0) [2023-12-27 05:44:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:17,379][20347] Updated weights for policy 0, policy_version 10274 (0.0008) [2023-12-27 05:44:17,438][20347] Updated weights for policy 0, policy_version 10284 (0.0006) [2023-12-27 05:44:17,493][20347] Updated weights for policy 0, policy_version 10294 (0.0008) [2023-12-27 05:44:17,550][20347] Updated weights for policy 0, policy_version 10304 (0.0007) [2023-12-27 05:44:17,709][20381] Updated weights for policy 1, policy_version 10277 (0.0006) [2023-12-27 05:44:17,766][20381] Updated weights for policy 1, policy_version 10287 (0.0007) [2023-12-27 05:44:17,820][20381] Updated weights for policy 1, policy_version 10297 (0.0007) [2023-12-27 05:44:18,234][20347] Updated weights for policy 0, policy_version 10314 (0.0006) [2023-12-27 05:44:18,296][20347] Updated weights for policy 0, policy_version 10324 (0.0006) [2023-12-27 05:44:18,357][20347] Updated weights for policy 0, policy_version 10334 (0.0007) [2023-12-27 05:44:18,490][20381] Updated weights for policy 1, policy_version 10307 (0.0005) [2023-12-27 05:44:18,556][20381] Updated weights for policy 1, policy_version 10317 (0.0007) [2023-12-27 05:44:18,617][20381] Updated weights for policy 1, policy_version 10327 (0.0007) [2023-12-27 05:44:18,996][20347] Updated weights for policy 0, policy_version 10344 (0.0006) [2023-12-27 05:44:19,052][20347] Updated weights for policy 0, policy_version 10354 (0.0010) [2023-12-27 05:44:19,109][20347] Updated weights for policy 0, policy_version 10364 (0.0008) [2023-12-27 05:44:19,354][20381] Updated weights for policy 1, policy_version 10337 (0.0008) [2023-12-27 05:44:19,415][20381] Updated weights for policy 1, policy_version 10347 (0.0006) [2023-12-27 05:44:19,475][20381] Updated weights for policy 1, policy_version 10357 (0.0006) [2023-12-27 05:44:19,537][20381] Updated weights for policy 1, policy_version 10367 (0.0006) [2023-12-27 05:44:19,790][20347] Updated weights for policy 0, policy_version 10374 (0.0009) [2023-12-27 05:44:19,859][20347] Updated weights for policy 0, policy_version 10385 (0.0009) [2023-12-27 05:44:19,931][20347] Updated weights for policy 0, policy_version 10395 (0.0008) [2023-12-27 05:44:20,161][20381] Updated weights for policy 1, policy_version 10377 (0.0006) [2023-12-27 05:44:20,224][20381] Updated weights for policy 1, policy_version 10387 (0.0007) [2023-12-27 05:44:20,287][20381] Updated weights for policy 1, policy_version 10397 (0.0008) [2023-12-27 05:44:20,675][20347] Updated weights for policy 0, policy_version 10405 (0.0008) [2023-12-27 05:44:20,736][20347] Updated weights for policy 0, policy_version 10415 (0.0008) [2023-12-27 05:44:20,799][20347] Updated weights for policy 0, policy_version 10425 (0.0007) [2023-12-27 05:44:21,012][20381] Updated weights for policy 1, policy_version 10407 (0.0006) [2023-12-27 05:44:21,080][20381] Updated weights for policy 1, policy_version 10417 (0.0008) [2023-12-27 05:44:21,149][20381] Updated weights for policy 1, policy_version 10427 (0.0007) [2023-12-27 05:44:21,539][20347] Updated weights for policy 0, policy_version 10435 (0.0008) [2023-12-27 05:44:21,595][20347] Updated weights for policy 0, policy_version 10445 (0.0007) [2023-12-27 05:44:21,661][20347] Updated weights for policy 0, policy_version 10455 (0.0008) [2023-12-27 05:44:21,835][20381] Updated weights for policy 1, policy_version 10437 (0.0007) [2023-12-27 05:44:21,898][20381] Updated weights for policy 1, policy_version 10447 (0.0008) [2023-12-27 05:44:21,964][20381] Updated weights for policy 1, policy_version 10457 (0.0007) [2023-12-27 05:44:22,310][20347] Updated weights for policy 0, policy_version 10465 (0.0008) [2023-12-27 05:44:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20606.0). Total num frames: 5357568. Throughput: 0: 10551.1, 1: 10391.0. Samples: 5331612. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 05:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000010464_2678784.pth... [2023-12-27 05:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000009248_2367488.pth [2023-12-27 05:44:22,373][20347] Updated weights for policy 0, policy_version 10475 (0.0008) [2023-12-27 05:44:22,433][20347] Updated weights for policy 0, policy_version 10485 (0.0007) [2023-12-27 05:44:22,485][20347] Updated weights for policy 0, policy_version 10495 (0.0006) [2023-12-27 05:44:22,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000010496_2686976.pth... [2023-12-27 05:44:22,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000009248_2367488.pth [2023-12-27 05:44:22,701][20381] Updated weights for policy 1, policy_version 10467 (0.0007) [2023-12-27 05:44:22,763][20381] Updated weights for policy 1, policy_version 10477 (0.0007) [2023-12-27 05:44:22,819][20381] Updated weights for policy 1, policy_version 10487 (0.0008) [2023-12-27 05:44:23,136][20347] Updated weights for policy 0, policy_version 10505 (0.0005) [2023-12-27 05:44:23,191][20347] Updated weights for policy 0, policy_version 10515 (0.0009) [2023-12-27 05:44:23,249][20347] Updated weights for policy 0, policy_version 10525 (0.0010) [2023-12-27 05:44:23,539][20381] Updated weights for policy 1, policy_version 10497 (0.0008) [2023-12-27 05:44:23,584][20381] Updated weights for policy 1, policy_version 10507 (0.0010) [2023-12-27 05:44:23,629][20381] Updated weights for policy 1, policy_version 10517 (0.0010) [2023-12-27 05:44:23,679][20381] Updated weights for policy 1, policy_version 10527 (0.0009) [2023-12-27 05:44:23,888][20347] Updated weights for policy 0, policy_version 10535 (0.0010) [2023-12-27 05:44:23,946][20347] Updated weights for policy 0, policy_version 10545 (0.0006) [2023-12-27 05:44:24,005][20347] Updated weights for policy 0, policy_version 10555 (0.0005) [2023-12-27 05:44:24,369][20381] Updated weights for policy 1, policy_version 10537 (0.0006) [2023-12-27 05:44:24,431][20381] Updated weights for policy 1, policy_version 10547 (0.0008) [2023-12-27 05:44:24,487][20381] Updated weights for policy 1, policy_version 10557 (0.0005) [2023-12-27 05:44:24,643][20347] Updated weights for policy 0, policy_version 10565 (0.0006) [2023-12-27 05:44:24,700][20347] Updated weights for policy 0, policy_version 10575 (0.0008) [2023-12-27 05:44:24,752][20347] Updated weights for policy 0, policy_version 10585 (0.0008) [2023-12-27 05:44:25,107][20381] Updated weights for policy 1, policy_version 10567 (0.0008) [2023-12-27 05:44:25,159][20381] Updated weights for policy 1, policy_version 10577 (0.0008) [2023-12-27 05:44:25,214][20381] Updated weights for policy 1, policy_version 10587 (0.0008) [2023-12-27 05:44:25,403][20347] Updated weights for policy 0, policy_version 10595 (0.0007) [2023-12-27 05:44:25,456][20347] Updated weights for policy 0, policy_version 10605 (0.0008) [2023-12-27 05:44:25,499][20347] Updated weights for policy 0, policy_version 10615 (0.0008) [2023-12-27 05:44:25,921][20381] Updated weights for policy 1, policy_version 10597 (0.0007) [2023-12-27 05:44:25,973][20381] Updated weights for policy 1, policy_version 10607 (0.0007) [2023-12-27 05:44:26,019][20381] Updated weights for policy 1, policy_version 10617 (0.0008) [2023-12-27 05:44:26,192][20347] Updated weights for policy 0, policy_version 10625 (0.0007) [2023-12-27 05:44:26,236][20347] Updated weights for policy 0, policy_version 10635 (0.0007) [2023-12-27 05:44:26,283][20347] Updated weights for policy 0, policy_version 10645 (0.0008) [2023-12-27 05:44:26,325][20347] Updated weights for policy 0, policy_version 10655 (0.0007) [2023-12-27 05:44:26,663][20381] Updated weights for policy 1, policy_version 10627 (0.0007) [2023-12-27 05:44:26,722][20381] Updated weights for policy 1, policy_version 10637 (0.0008) [2023-12-27 05:44:26,789][20381] Updated weights for policy 1, policy_version 10647 (0.0007) [2023-12-27 05:44:27,035][20347] Updated weights for policy 0, policy_version 10665 (0.0005) [2023-12-27 05:44:27,083][20347] Updated weights for policy 0, policy_version 10675 (0.0006) [2023-12-27 05:44:27,125][20347] Updated weights for policy 0, policy_version 10685 (0.0007) [2023-12-27 05:44:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20619.1). Total num frames: 5464064. Throughput: 0: 10512.0, 1: 10338.8. Samples: 5452548. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 05:44:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:27,476][20381] Updated weights for policy 1, policy_version 10657 (0.0008) [2023-12-27 05:44:27,523][20381] Updated weights for policy 1, policy_version 10667 (0.0007) [2023-12-27 05:44:27,570][20381] Updated weights for policy 1, policy_version 10677 (0.0007) [2023-12-27 05:44:27,615][20381] Updated weights for policy 1, policy_version 10687 (0.0008) [2023-12-27 05:44:27,741][20347] Updated weights for policy 0, policy_version 10695 (0.0007) [2023-12-27 05:44:27,795][20347] Updated weights for policy 0, policy_version 10705 (0.0007) [2023-12-27 05:44:27,843][20347] Updated weights for policy 0, policy_version 10715 (0.0006) [2023-12-27 05:44:28,310][20381] Updated weights for policy 1, policy_version 10697 (0.0007) [2023-12-27 05:44:28,370][20381] Updated weights for policy 1, policy_version 10707 (0.0007) [2023-12-27 05:44:28,415][20381] Updated weights for policy 1, policy_version 10717 (0.0007) [2023-12-27 05:44:28,522][20347] Updated weights for policy 0, policy_version 10725 (0.0007) [2023-12-27 05:44:28,572][20347] Updated weights for policy 0, policy_version 10735 (0.0007) [2023-12-27 05:44:28,622][20347] Updated weights for policy 0, policy_version 10745 (0.0007) [2023-12-27 05:44:29,059][20381] Updated weights for policy 1, policy_version 10727 (0.0006) [2023-12-27 05:44:29,108][20381] Updated weights for policy 1, policy_version 10737 (0.0008) [2023-12-27 05:44:29,159][20381] Updated weights for policy 1, policy_version 10747 (0.0008) [2023-12-27 05:44:29,358][20347] Updated weights for policy 0, policy_version 10755 (0.0008) [2023-12-27 05:44:29,419][20347] Updated weights for policy 0, policy_version 10765 (0.0008) [2023-12-27 05:44:29,478][20347] Updated weights for policy 0, policy_version 10775 (0.0007) [2023-12-27 05:44:29,922][20381] Updated weights for policy 1, policy_version 10757 (0.0008) [2023-12-27 05:44:29,984][20381] Updated weights for policy 1, policy_version 10767 (0.0007) [2023-12-27 05:44:30,045][20381] Updated weights for policy 1, policy_version 10777 (0.0007) [2023-12-27 05:44:30,155][20347] Updated weights for policy 0, policy_version 10785 (0.0009) [2023-12-27 05:44:30,210][20347] Updated weights for policy 0, policy_version 10795 (0.0007) [2023-12-27 05:44:30,259][20347] Updated weights for policy 0, policy_version 10805 (0.0006) [2023-12-27 05:44:30,315][20347] Updated weights for policy 0, policy_version 10815 (0.0005) [2023-12-27 05:44:30,678][20381] Updated weights for policy 1, policy_version 10787 (0.0007) [2023-12-27 05:44:30,737][20381] Updated weights for policy 1, policy_version 10797 (0.0006) [2023-12-27 05:44:30,791][20381] Updated weights for policy 1, policy_version 10807 (0.0005) [2023-12-27 05:44:30,847][20347] Updated weights for policy 0, policy_version 10825 (0.0005) [2023-12-27 05:44:30,902][20347] Updated weights for policy 0, policy_version 10835 (0.0005) [2023-12-27 05:44:30,958][20347] Updated weights for policy 0, policy_version 10845 (0.0006) [2023-12-27 05:44:31,399][20381] Updated weights for policy 1, policy_version 10817 (0.0007) [2023-12-27 05:44:31,446][20381] Updated weights for policy 1, policy_version 10827 (0.0008) [2023-12-27 05:44:31,498][20381] Updated weights for policy 1, policy_version 10837 (0.0008) [2023-12-27 05:44:31,550][20381] Updated weights for policy 1, policy_version 10847 (0.0007) [2023-12-27 05:44:31,629][20347] Updated weights for policy 0, policy_version 10855 (0.0006) [2023-12-27 05:44:31,689][20347] Updated weights for policy 0, policy_version 10865 (0.0006) [2023-12-27 05:44:31,747][20347] Updated weights for policy 0, policy_version 10875 (0.0008) [2023-12-27 05:44:32,314][20381] Updated weights for policy 1, policy_version 10857 (0.0006) [2023-12-27 05:44:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20601.4). Total num frames: 5562368. Throughput: 0: 10499.2, 1: 10377.3. Samples: 5578168. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-12-27 05:44:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:32,377][20381] Updated weights for policy 1, policy_version 10867 (0.0008) [2023-12-27 05:44:32,434][20347] Updated weights for policy 0, policy_version 10885 (0.0006) [2023-12-27 05:44:32,438][20381] Updated weights for policy 1, policy_version 10877 (0.0006) [2023-12-27 05:44:32,497][20347] Updated weights for policy 0, policy_version 10895 (0.0006) [2023-12-27 05:44:32,559][20347] Updated weights for policy 0, policy_version 10905 (0.0006) [2023-12-27 05:44:33,089][20381] Updated weights for policy 1, policy_version 10887 (0.0009) [2023-12-27 05:44:33,146][20381] Updated weights for policy 1, policy_version 10897 (0.0006) [2023-12-27 05:44:33,182][20347] Updated weights for policy 0, policy_version 10915 (0.0007) [2023-12-27 05:44:33,201][20381] Updated weights for policy 1, policy_version 10907 (0.0009) [2023-12-27 05:44:33,230][20347] Updated weights for policy 0, policy_version 10925 (0.0007) [2023-12-27 05:44:33,284][20347] Updated weights for policy 0, policy_version 10935 (0.0007) [2023-12-27 05:44:33,743][20381] Updated weights for policy 1, policy_version 10917 (0.0008) [2023-12-27 05:44:33,798][20381] Updated weights for policy 1, policy_version 10928 (0.0007) [2023-12-27 05:44:33,843][20381] Updated weights for policy 1, policy_version 10938 (0.0005) [2023-12-27 05:44:34,038][20347] Updated weights for policy 0, policy_version 10945 (0.0007) [2023-12-27 05:44:34,097][20347] Updated weights for policy 0, policy_version 10955 (0.0008) [2023-12-27 05:44:34,149][20347] Updated weights for policy 0, policy_version 10965 (0.0007) [2023-12-27 05:44:34,214][20347] Updated weights for policy 0, policy_version 10975 (0.0008) [2023-12-27 05:44:34,472][20381] Updated weights for policy 1, policy_version 10948 (0.0007) [2023-12-27 05:44:34,537][20381] Updated weights for policy 1, policy_version 10958 (0.0008) [2023-12-27 05:44:34,593][20381] Updated weights for policy 1, policy_version 10968 (0.0007) [2023-12-27 05:44:34,959][20347] Updated weights for policy 0, policy_version 10985 (0.0006) [2023-12-27 05:44:35,010][20347] Updated weights for policy 0, policy_version 10995 (0.0007) [2023-12-27 05:44:35,060][20347] Updated weights for policy 0, policy_version 11005 (0.0008) [2023-12-27 05:44:35,185][20381] Updated weights for policy 1, policy_version 10978 (0.0007) [2023-12-27 05:44:35,232][20381] Updated weights for policy 1, policy_version 10988 (0.0005) [2023-12-27 05:44:35,284][20381] Updated weights for policy 1, policy_version 10998 (0.0006) [2023-12-27 05:44:35,332][20381] Updated weights for policy 1, policy_version 11008 (0.0005) [2023-12-27 05:44:35,685][20347] Updated weights for policy 0, policy_version 11015 (0.0008) [2023-12-27 05:44:35,740][20347] Updated weights for policy 0, policy_version 11025 (0.0007) [2023-12-27 05:44:35,795][20347] Updated weights for policy 0, policy_version 11035 (0.0005) [2023-12-27 05:44:36,039][20381] Updated weights for policy 1, policy_version 11018 (0.0007) [2023-12-27 05:44:36,094][20381] Updated weights for policy 1, policy_version 11028 (0.0008) [2023-12-27 05:44:36,151][20381] Updated weights for policy 1, policy_version 11038 (0.0006) [2023-12-27 05:44:36,425][20347] Updated weights for policy 0, policy_version 11045 (0.0007) [2023-12-27 05:44:36,483][20347] Updated weights for policy 0, policy_version 11055 (0.0009) [2023-12-27 05:44:36,537][20347] Updated weights for policy 0, policy_version 11065 (0.0008) [2023-12-27 05:44:36,796][20381] Updated weights for policy 1, policy_version 11048 (0.0007) [2023-12-27 05:44:36,847][20381] Updated weights for policy 1, policy_version 11058 (0.0009) [2023-12-27 05:44:36,907][20381] Updated weights for policy 1, policy_version 11068 (0.0009) [2023-12-27 05:44:37,236][20347] Updated weights for policy 0, policy_version 11075 (0.0008) [2023-12-27 05:44:37,292][20347] Updated weights for policy 0, policy_version 11085 (0.0006) [2023-12-27 05:44:37,341][20347] Updated weights for policy 0, policy_version 11095 (0.0005) [2023-12-27 05:44:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20614.0). Total num frames: 5668864. Throughput: 0: 10466.3, 1: 10458.4. Samples: 5641608. Policy #0 lag: (min: 5.0, avg: 10.8, max: 37.0) [2023-12-27 05:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000011072_2834432.pth... [2023-12-27 05:44:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000009856_2523136.pth [2023-12-27 05:44:37,389][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000011104_2842624.pth... [2023-12-27 05:44:37,392][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000009888_2531328.pth [2023-12-27 05:44:37,624][20381] Updated weights for policy 1, policy_version 11078 (0.0007) [2023-12-27 05:44:37,671][20381] Updated weights for policy 1, policy_version 11088 (0.0006) [2023-12-27 05:44:37,732][20381] Updated weights for policy 1, policy_version 11098 (0.0008) [2023-12-27 05:44:38,011][20347] Updated weights for policy 0, policy_version 11105 (0.0007) [2023-12-27 05:44:38,070][20347] Updated weights for policy 0, policy_version 11115 (0.0010) [2023-12-27 05:44:38,134][20347] Updated weights for policy 0, policy_version 11125 (0.0010) [2023-12-27 05:44:38,195][20347] Updated weights for policy 0, policy_version 11135 (0.0010) [2023-12-27 05:44:38,397][20381] Updated weights for policy 1, policy_version 11108 (0.0008) [2023-12-27 05:44:38,458][20381] Updated weights for policy 1, policy_version 11118 (0.0010) [2023-12-27 05:44:38,513][20381] Updated weights for policy 1, policy_version 11128 (0.0009) [2023-12-27 05:44:38,821][20347] Updated weights for policy 0, policy_version 11145 (0.0008) [2023-12-27 05:44:38,880][20347] Updated weights for policy 0, policy_version 11155 (0.0007) [2023-12-27 05:44:38,939][20347] Updated weights for policy 0, policy_version 11165 (0.0008) [2023-12-27 05:44:39,168][20381] Updated weights for policy 1, policy_version 11138 (0.0006) [2023-12-27 05:44:39,232][20381] Updated weights for policy 1, policy_version 11148 (0.0008) [2023-12-27 05:44:39,296][20381] Updated weights for policy 1, policy_version 11158 (0.0007) [2023-12-27 05:44:39,355][20381] Updated weights for policy 1, policy_version 11168 (0.0007) [2023-12-27 05:44:39,596][20347] Updated weights for policy 0, policy_version 11175 (0.0007) [2023-12-27 05:44:39,664][20347] Updated weights for policy 0, policy_version 11185 (0.0007) [2023-12-27 05:44:39,726][20347] Updated weights for policy 0, policy_version 11195 (0.0006) [2023-12-27 05:44:40,071][20381] Updated weights for policy 1, policy_version 11178 (0.0011) [2023-12-27 05:44:40,130][20381] Updated weights for policy 1, policy_version 11188 (0.0011) [2023-12-27 05:44:40,187][20381] Updated weights for policy 1, policy_version 11198 (0.0011) [2023-12-27 05:44:40,347][20347] Updated weights for policy 0, policy_version 11205 (0.0007) [2023-12-27 05:44:40,407][20347] Updated weights for policy 0, policy_version 11215 (0.0008) [2023-12-27 05:44:40,464][20347] Updated weights for policy 0, policy_version 11225 (0.0008) [2023-12-27 05:44:40,805][20381] Updated weights for policy 1, policy_version 11208 (0.0008) [2023-12-27 05:44:40,855][20381] Updated weights for policy 1, policy_version 11218 (0.0005) [2023-12-27 05:44:40,909][20381] Updated weights for policy 1, policy_version 11228 (0.0006) [2023-12-27 05:44:41,123][20347] Updated weights for policy 0, policy_version 11235 (0.0008) [2023-12-27 05:44:41,187][20347] Updated weights for policy 0, policy_version 11245 (0.0008) [2023-12-27 05:44:41,246][20347] Updated weights for policy 0, policy_version 11255 (0.0008) [2023-12-27 05:44:41,624][20381] Updated weights for policy 1, policy_version 11238 (0.0008) [2023-12-27 05:44:41,690][20381] Updated weights for policy 1, policy_version 11248 (0.0007) [2023-12-27 05:44:41,758][20381] Updated weights for policy 1, policy_version 11258 (0.0008) [2023-12-27 05:44:41,909][20347] Updated weights for policy 0, policy_version 11265 (0.0007) [2023-12-27 05:44:41,971][20347] Updated weights for policy 0, policy_version 11275 (0.0008) [2023-12-27 05:44:42,029][20347] Updated weights for policy 0, policy_version 11285 (0.0007) [2023-12-27 05:44:42,091][20347] Updated weights for policy 0, policy_version 11295 (0.0008) [2023-12-27 05:44:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20626.3). Total num frames: 5775360. Throughput: 0: 10426.7, 1: 10419.2. Samples: 5767012. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 05:44:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:42,438][20381] Updated weights for policy 1, policy_version 11268 (0.0008) [2023-12-27 05:44:42,491][20381] Updated weights for policy 1, policy_version 11278 (0.0009) [2023-12-27 05:44:42,540][20381] Updated weights for policy 1, policy_version 11288 (0.0008) [2023-12-27 05:44:42,731][20347] Updated weights for policy 0, policy_version 11305 (0.0006) [2023-12-27 05:44:42,788][20347] Updated weights for policy 0, policy_version 11315 (0.0005) [2023-12-27 05:44:42,851][20347] Updated weights for policy 0, policy_version 11325 (0.0005) [2023-12-27 05:44:43,247][20381] Updated weights for policy 1, policy_version 11298 (0.0006) [2023-12-27 05:44:43,305][20381] Updated weights for policy 1, policy_version 11308 (0.0010) [2023-12-27 05:44:43,363][20381] Updated weights for policy 1, policy_version 11318 (0.0010) [2023-12-27 05:44:43,386][20347] Updated weights for policy 0, policy_version 11335 (0.0008) [2023-12-27 05:44:43,416][20381] Updated weights for policy 1, policy_version 11328 (0.0010) [2023-12-27 05:44:43,441][20347] Updated weights for policy 0, policy_version 11345 (0.0008) [2023-12-27 05:44:43,494][20347] Updated weights for policy 0, policy_version 11355 (0.0007) [2023-12-27 05:44:44,104][20381] Updated weights for policy 1, policy_version 11338 (0.0010) [2023-12-27 05:44:44,158][20381] Updated weights for policy 1, policy_version 11348 (0.0010) [2023-12-27 05:44:44,164][20347] Updated weights for policy 0, policy_version 11365 (0.0006) [2023-12-27 05:44:44,214][20381] Updated weights for policy 1, policy_version 11358 (0.0010) [2023-12-27 05:44:44,219][20347] Updated weights for policy 0, policy_version 11375 (0.0006) [2023-12-27 05:44:44,278][20347] Updated weights for policy 0, policy_version 11385 (0.0007) [2023-12-27 05:44:44,903][20347] Updated weights for policy 0, policy_version 11395 (0.0008) [2023-12-27 05:44:44,913][20381] Updated weights for policy 1, policy_version 11368 (0.0007) [2023-12-27 05:44:44,961][20347] Updated weights for policy 0, policy_version 11405 (0.0011) [2023-12-27 05:44:44,974][20381] Updated weights for policy 1, policy_version 11378 (0.0010) [2023-12-27 05:44:45,022][20347] Updated weights for policy 0, policy_version 11415 (0.0011) [2023-12-27 05:44:45,033][20381] Updated weights for policy 1, policy_version 11388 (0.0010) [2023-12-27 05:44:45,650][20347] Updated weights for policy 0, policy_version 11425 (0.0009) [2023-12-27 05:44:45,700][20347] Updated weights for policy 0, policy_version 11435 (0.0010) [2023-12-27 05:44:45,703][20381] Updated weights for policy 1, policy_version 11398 (0.0009) [2023-12-27 05:44:45,752][20347] Updated weights for policy 0, policy_version 11445 (0.0010) [2023-12-27 05:44:45,755][20381] Updated weights for policy 1, policy_version 11408 (0.0010) [2023-12-27 05:44:45,811][20381] Updated weights for policy 1, policy_version 11418 (0.0011) [2023-12-27 05:44:45,839][20347] Updated weights for policy 0, policy_version 11455 (0.0009) [2023-12-27 05:44:46,483][20347] Updated weights for policy 0, policy_version 11465 (0.0007) [2023-12-27 05:44:46,500][20381] Updated weights for policy 1, policy_version 11428 (0.0010) [2023-12-27 05:44:46,531][20347] Updated weights for policy 0, policy_version 11475 (0.0005) [2023-12-27 05:44:46,555][20381] Updated weights for policy 1, policy_version 11438 (0.0010) [2023-12-27 05:44:46,590][20347] Updated weights for policy 0, policy_version 11485 (0.0006) [2023-12-27 05:44:46,608][20381] Updated weights for policy 1, policy_version 11448 (0.0010) [2023-12-27 05:44:47,231][20347] Updated weights for policy 0, policy_version 11495 (0.0006) [2023-12-27 05:44:47,276][20381] Updated weights for policy 1, policy_version 11458 (0.0010) [2023-12-27 05:44:47,288][20347] Updated weights for policy 0, policy_version 11505 (0.0008) [2023-12-27 05:44:47,335][20381] Updated weights for policy 1, policy_version 11468 (0.0010) [2023-12-27 05:44:47,342][20347] Updated weights for policy 0, policy_version 11515 (0.0010) [2023-12-27 05:44:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20609.3). Total num frames: 5873664. Throughput: 0: 10520.3, 1: 10370.2. Samples: 5891952. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 05:44:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:47,390][20381] Updated weights for policy 1, policy_version 11478 (0.0011) [2023-12-27 05:44:47,439][20381] Updated weights for policy 1, policy_version 11488 (0.0011) [2023-12-27 05:44:47,965][20347] Updated weights for policy 0, policy_version 11525 (0.0010) [2023-12-27 05:44:48,018][20347] Updated weights for policy 0, policy_version 11535 (0.0010) [2023-12-27 05:44:48,070][20347] Updated weights for policy 0, policy_version 11545 (0.0010) [2023-12-27 05:44:48,102][20381] Updated weights for policy 1, policy_version 11498 (0.0011) [2023-12-27 05:44:48,150][20381] Updated weights for policy 1, policy_version 11508 (0.0010) [2023-12-27 05:44:48,202][20381] Updated weights for policy 1, policy_version 11518 (0.0011) [2023-12-27 05:44:48,762][20347] Updated weights for policy 0, policy_version 11555 (0.0010) [2023-12-27 05:44:48,822][20347] Updated weights for policy 0, policy_version 11565 (0.0010) [2023-12-27 05:44:48,881][20347] Updated weights for policy 0, policy_version 11575 (0.0011) [2023-12-27 05:44:48,889][20381] Updated weights for policy 1, policy_version 11528 (0.0011) [2023-12-27 05:44:48,948][20381] Updated weights for policy 1, policy_version 11538 (0.0011) [2023-12-27 05:44:49,002][20381] Updated weights for policy 1, policy_version 11548 (0.0006) [2023-12-27 05:44:49,595][20347] Updated weights for policy 0, policy_version 11585 (0.0011) [2023-12-27 05:44:49,661][20347] Updated weights for policy 0, policy_version 11596 (0.0008) [2023-12-27 05:44:49,699][20381] Updated weights for policy 1, policy_version 11558 (0.0009) [2023-12-27 05:44:49,716][20347] Updated weights for policy 0, policy_version 11606 (0.0007) [2023-12-27 05:44:49,763][20381] Updated weights for policy 1, policy_version 11568 (0.0010) [2023-12-27 05:44:49,766][20347] Updated weights for policy 0, policy_version 11616 (0.0008) [2023-12-27 05:44:49,819][20381] Updated weights for policy 1, policy_version 11578 (0.0008) [2023-12-27 05:44:50,462][20347] Updated weights for policy 0, policy_version 11626 (0.0007) [2023-12-27 05:44:50,523][20347] Updated weights for policy 0, policy_version 11636 (0.0009) [2023-12-27 05:44:50,529][20381] Updated weights for policy 1, policy_version 11588 (0.0009) [2023-12-27 05:44:50,577][20347] Updated weights for policy 0, policy_version 11646 (0.0006) [2023-12-27 05:44:50,587][20381] Updated weights for policy 1, policy_version 11598 (0.0006) [2023-12-27 05:44:50,644][20381] Updated weights for policy 1, policy_version 11608 (0.0007) [2023-12-27 05:44:51,288][20347] Updated weights for policy 0, policy_version 11656 (0.0010) [2023-12-27 05:44:51,333][20381] Updated weights for policy 1, policy_version 11618 (0.0006) [2023-12-27 05:44:51,356][20347] Updated weights for policy 0, policy_version 11667 (0.0011) [2023-12-27 05:44:51,397][20381] Updated weights for policy 1, policy_version 11628 (0.0008) [2023-12-27 05:44:51,417][20347] Updated weights for policy 0, policy_version 11677 (0.0007) [2023-12-27 05:44:51,453][20381] Updated weights for policy 1, policy_version 11638 (0.0008) [2023-12-27 05:44:51,507][20381] Updated weights for policy 1, policy_version 11648 (0.0008) [2023-12-27 05:44:52,123][20347] Updated weights for policy 0, policy_version 11687 (0.0009) [2023-12-27 05:44:52,187][20347] Updated weights for policy 0, policy_version 11697 (0.0010) [2023-12-27 05:44:52,242][20347] Updated weights for policy 0, policy_version 11707 (0.0007) [2023-12-27 05:44:52,244][20381] Updated weights for policy 1, policy_version 11658 (0.0008) [2023-12-27 05:44:52,306][20381] Updated weights for policy 1, policy_version 11668 (0.0008) [2023-12-27 05:44:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20621.2). Total num frames: 5980160. Throughput: 0: 10461.4, 1: 10345.4. Samples: 5952476. Policy #0 lag: (min: 4.0, avg: 6.3, max: 36.0) [2023-12-27 05:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000011712_2998272.pth... [2023-12-27 05:44:52,368][20381] Updated weights for policy 1, policy_version 11678 (0.0008) [2023-12-27 05:44:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000010496_2686976.pth [2023-12-27 05:44:52,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000011680_2990080.pth... [2023-12-27 05:44:52,384][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000010464_2678784.pth [2023-12-27 05:44:52,886][20347] Updated weights for policy 0, policy_version 11717 (0.0010) [2023-12-27 05:44:52,931][20347] Updated weights for policy 0, policy_version 11727 (0.0008) [2023-12-27 05:44:52,979][20347] Updated weights for policy 0, policy_version 11737 (0.0007) [2023-12-27 05:44:53,014][20381] Updated weights for policy 1, policy_version 11688 (0.0006) [2023-12-27 05:44:53,074][20381] Updated weights for policy 1, policy_version 11698 (0.0005) [2023-12-27 05:44:53,137][20381] Updated weights for policy 1, policy_version 11708 (0.0005) [2023-12-27 05:44:53,681][20381] Updated weights for policy 1, policy_version 11718 (0.0005) [2023-12-27 05:44:53,725][20347] Updated weights for policy 0, policy_version 11747 (0.0008) [2023-12-27 05:44:53,742][20381] Updated weights for policy 1, policy_version 11728 (0.0005) [2023-12-27 05:44:53,774][20347] Updated weights for policy 0, policy_version 11757 (0.0007) [2023-12-27 05:44:53,790][20381] Updated weights for policy 1, policy_version 11738 (0.0005) [2023-12-27 05:44:53,830][20347] Updated weights for policy 0, policy_version 11767 (0.0008) [2023-12-27 05:44:54,347][20381] Updated weights for policy 1, policy_version 11748 (0.0006) [2023-12-27 05:44:54,412][20381] Updated weights for policy 1, policy_version 11758 (0.0006) [2023-12-27 05:44:54,471][20381] Updated weights for policy 1, policy_version 11768 (0.0008) [2023-12-27 05:44:54,504][20347] Updated weights for policy 0, policy_version 11777 (0.0007) [2023-12-27 05:44:54,561][20347] Updated weights for policy 0, policy_version 11787 (0.0007) [2023-12-27 05:44:54,619][20347] Updated weights for policy 0, policy_version 11797 (0.0007) [2023-12-27 05:44:54,675][20347] Updated weights for policy 0, policy_version 11807 (0.0006) [2023-12-27 05:44:55,103][20381] Updated weights for policy 1, policy_version 11778 (0.0007) [2023-12-27 05:44:55,153][20381] Updated weights for policy 1, policy_version 11788 (0.0005) [2023-12-27 05:44:55,204][20381] Updated weights for policy 1, policy_version 11798 (0.0006) [2023-12-27 05:44:55,256][20381] Updated weights for policy 1, policy_version 11808 (0.0007) [2023-12-27 05:44:55,385][20347] Updated weights for policy 0, policy_version 11817 (0.0008) [2023-12-27 05:44:55,444][20347] Updated weights for policy 0, policy_version 11827 (0.0007) [2023-12-27 05:44:55,496][20347] Updated weights for policy 0, policy_version 11837 (0.0009) [2023-12-27 05:44:55,913][20381] Updated weights for policy 1, policy_version 11818 (0.0007) [2023-12-27 05:44:55,959][20381] Updated weights for policy 1, policy_version 11828 (0.0007) [2023-12-27 05:44:56,005][20381] Updated weights for policy 1, policy_version 11838 (0.0010) [2023-12-27 05:44:56,139][20347] Updated weights for policy 0, policy_version 11847 (0.0010) [2023-12-27 05:44:56,184][20347] Updated weights for policy 0, policy_version 11857 (0.0010) [2023-12-27 05:44:56,237][20347] Updated weights for policy 0, policy_version 11867 (0.0009) [2023-12-27 05:44:56,739][20381] Updated weights for policy 1, policy_version 11848 (0.0008) [2023-12-27 05:44:56,792][20381] Updated weights for policy 1, policy_version 11858 (0.0007) [2023-12-27 05:44:56,844][20381] Updated weights for policy 1, policy_version 11868 (0.0007) [2023-12-27 05:44:56,895][20347] Updated weights for policy 0, policy_version 11877 (0.0008) [2023-12-27 05:44:56,951][20347] Updated weights for policy 0, policy_version 11887 (0.0010) [2023-12-27 05:44:57,005][20347] Updated weights for policy 0, policy_version 11897 (0.0010) [2023-12-27 05:44:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.2, 300 sec: 20632.7). Total num frames: 6086656. Throughput: 0: 10367.1, 1: 10347.3. Samples: 6076268. Policy #0 lag: (min: 26.0, avg: 34.8, max: 58.0) [2023-12-27 05:44:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:44:57,535][20381] Updated weights for policy 1, policy_version 11878 (0.0006) [2023-12-27 05:44:57,596][20381] Updated weights for policy 1, policy_version 11888 (0.0005) [2023-12-27 05:44:57,638][20347] Updated weights for policy 0, policy_version 11907 (0.0010) [2023-12-27 05:44:57,651][20381] Updated weights for policy 1, policy_version 11898 (0.0005) [2023-12-27 05:44:57,685][20347] Updated weights for policy 0, policy_version 11917 (0.0010) [2023-12-27 05:44:57,733][20347] Updated weights for policy 0, policy_version 11927 (0.0010) [2023-12-27 05:44:58,288][20381] Updated weights for policy 1, policy_version 11908 (0.0006) [2023-12-27 05:44:58,348][20381] Updated weights for policy 1, policy_version 11918 (0.0007) [2023-12-27 05:44:58,399][20347] Updated weights for policy 0, policy_version 11937 (0.0009) [2023-12-27 05:44:58,411][20381] Updated weights for policy 1, policy_version 11928 (0.0008) [2023-12-27 05:44:58,462][20347] Updated weights for policy 0, policy_version 11947 (0.0010) [2023-12-27 05:44:58,521][20347] Updated weights for policy 0, policy_version 11957 (0.0011) [2023-12-27 05:44:58,579][20347] Updated weights for policy 0, policy_version 11967 (0.0009) [2023-12-27 05:44:59,204][20381] Updated weights for policy 1, policy_version 11938 (0.0008) [2023-12-27 05:44:59,272][20381] Updated weights for policy 1, policy_version 11948 (0.0008) [2023-12-27 05:44:59,282][20347] Updated weights for policy 0, policy_version 11977 (0.0008) [2023-12-27 05:44:59,331][20381] Updated weights for policy 1, policy_version 11958 (0.0008) [2023-12-27 05:44:59,342][20347] Updated weights for policy 0, policy_version 11987 (0.0011) [2023-12-27 05:44:59,395][20381] Updated weights for policy 1, policy_version 11968 (0.0007) [2023-12-27 05:44:59,404][20347] Updated weights for policy 0, policy_version 11997 (0.0010) [2023-12-27 05:45:00,082][20347] Updated weights for policy 0, policy_version 12007 (0.0007) [2023-12-27 05:45:00,121][20381] Updated weights for policy 1, policy_version 11978 (0.0007) [2023-12-27 05:45:00,128][20347] Updated weights for policy 0, policy_version 12017 (0.0008) [2023-12-27 05:45:00,175][20347] Updated weights for policy 0, policy_version 12027 (0.0006) [2023-12-27 05:45:00,180][20381] Updated weights for policy 1, policy_version 11988 (0.0007) [2023-12-27 05:45:00,238][20381] Updated weights for policy 1, policy_version 11998 (0.0007) [2023-12-27 05:45:00,869][20347] Updated weights for policy 0, policy_version 12037 (0.0006) [2023-12-27 05:45:00,904][20381] Updated weights for policy 1, policy_version 12008 (0.0007) [2023-12-27 05:45:00,925][20347] Updated weights for policy 0, policy_version 12047 (0.0006) [2023-12-27 05:45:00,949][20381] Updated weights for policy 1, policy_version 12018 (0.0005) [2023-12-27 05:45:00,980][20347] Updated weights for policy 0, policy_version 12057 (0.0006) [2023-12-27 05:45:01,004][20381] Updated weights for policy 1, policy_version 12028 (0.0006) [2023-12-27 05:45:01,616][20347] Updated weights for policy 0, policy_version 12067 (0.0006) [2023-12-27 05:45:01,677][20347] Updated weights for policy 0, policy_version 12077 (0.0007) [2023-12-27 05:45:01,691][20381] Updated weights for policy 1, policy_version 12038 (0.0007) [2023-12-27 05:45:01,737][20347] Updated weights for policy 0, policy_version 12087 (0.0008) [2023-12-27 05:45:01,754][20381] Updated weights for policy 1, policy_version 12048 (0.0007) [2023-12-27 05:45:01,810][20381] Updated weights for policy 1, policy_version 12058 (0.0008) [2023-12-27 05:45:02,275][20347] Updated weights for policy 0, policy_version 12097 (0.0006) [2023-12-27 05:45:02,328][20347] Updated weights for policy 0, policy_version 12107 (0.0006) [2023-12-27 05:45:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 6184960. Throughput: 0: 10371.6, 1: 10260.9. Samples: 6198744. Policy #0 lag: (min: 26.0, avg: 34.8, max: 58.0) [2023-12-27 05:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:02,392][20347] Updated weights for policy 0, policy_version 12117 (0.0008) [2023-12-27 05:45:02,450][20347] Updated weights for policy 0, policy_version 12127 (0.0006) [2023-12-27 05:45:02,579][20381] Updated weights for policy 1, policy_version 12068 (0.0007) [2023-12-27 05:45:02,624][20381] Updated weights for policy 1, policy_version 12078 (0.0008) [2023-12-27 05:45:02,668][20381] Updated weights for policy 1, policy_version 12088 (0.0008) [2023-12-27 05:45:03,050][20347] Updated weights for policy 0, policy_version 12137 (0.0006) [2023-12-27 05:45:03,116][20347] Updated weights for policy 0, policy_version 12147 (0.0008) [2023-12-27 05:45:03,166][20347] Updated weights for policy 0, policy_version 12157 (0.0008) [2023-12-27 05:45:03,367][20381] Updated weights for policy 1, policy_version 12098 (0.0008) [2023-12-27 05:45:03,418][20381] Updated weights for policy 1, policy_version 12108 (0.0010) [2023-12-27 05:45:03,464][20381] Updated weights for policy 1, policy_version 12118 (0.0010) [2023-12-27 05:45:03,516][20381] Updated weights for policy 1, policy_version 12128 (0.0010) [2023-12-27 05:45:03,798][20347] Updated weights for policy 0, policy_version 12167 (0.0008) [2023-12-27 05:45:03,859][20347] Updated weights for policy 0, policy_version 12177 (0.0008) [2023-12-27 05:45:03,925][20347] Updated weights for policy 0, policy_version 12187 (0.0010) [2023-12-27 05:45:04,150][20381] Updated weights for policy 1, policy_version 12138 (0.0007) [2023-12-27 05:45:04,215][20381] Updated weights for policy 1, policy_version 12148 (0.0007) [2023-12-27 05:45:04,277][20381] Updated weights for policy 1, policy_version 12158 (0.0008) [2023-12-27 05:45:04,636][20347] Updated weights for policy 0, policy_version 12197 (0.0010) [2023-12-27 05:45:04,694][20347] Updated weights for policy 0, policy_version 12207 (0.0010) [2023-12-27 05:45:04,754][20347] Updated weights for policy 0, policy_version 12217 (0.0010) [2023-12-27 05:45:04,907][20381] Updated weights for policy 1, policy_version 12168 (0.0008) [2023-12-27 05:45:04,964][20381] Updated weights for policy 1, policy_version 12178 (0.0008) [2023-12-27 05:45:05,016][20381] Updated weights for policy 1, policy_version 12188 (0.0008) [2023-12-27 05:45:05,435][20347] Updated weights for policy 0, policy_version 12227 (0.0010) [2023-12-27 05:45:05,499][20347] Updated weights for policy 0, policy_version 12237 (0.0011) [2023-12-27 05:45:05,559][20347] Updated weights for policy 0, policy_version 12247 (0.0006) [2023-12-27 05:45:05,752][20381] Updated weights for policy 1, policy_version 12198 (0.0008) [2023-12-27 05:45:05,808][20381] Updated weights for policy 1, policy_version 12208 (0.0007) [2023-12-27 05:45:05,865][20381] Updated weights for policy 1, policy_version 12218 (0.0007) [2023-12-27 05:45:06,212][20347] Updated weights for policy 0, policy_version 12257 (0.0009) [2023-12-27 05:45:06,280][20347] Updated weights for policy 0, policy_version 12267 (0.0010) [2023-12-27 05:45:06,348][20347] Updated weights for policy 0, policy_version 12277 (0.0011) [2023-12-27 05:45:06,417][20347] Updated weights for policy 0, policy_version 12287 (0.0011) [2023-12-27 05:45:06,500][20381] Updated weights for policy 1, policy_version 12228 (0.0006) [2023-12-27 05:45:06,556][20381] Updated weights for policy 1, policy_version 12238 (0.0005) [2023-12-27 05:45:06,615][20381] Updated weights for policy 1, policy_version 12248 (0.0006) [2023-12-27 05:45:07,065][20347] Updated weights for policy 0, policy_version 12297 (0.0011) [2023-12-27 05:45:07,124][20347] Updated weights for policy 0, policy_version 12307 (0.0008) [2023-12-27 05:45:07,178][20347] Updated weights for policy 0, policy_version 12317 (0.0006) [2023-12-27 05:45:07,258][20381] Updated weights for policy 1, policy_version 12258 (0.0007) [2023-12-27 05:45:07,307][20381] Updated weights for policy 1, policy_version 12268 (0.0006) [2023-12-27 05:45:07,360][20381] Updated weights for policy 1, policy_version 12278 (0.0007) [2023-12-27 05:45:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 6291456. Throughput: 0: 10411.7, 1: 10256.0. Samples: 6261656. Policy #0 lag: (min: 8.0, avg: 32.8, max: 40.0) [2023-12-27 05:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000012320_3153920.pth... [2023-12-27 05:45:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000011104_2842624.pth [2023-12-27 05:45:07,418][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000012288_3145728.pth... [2023-12-27 05:45:07,420][20381] Updated weights for policy 1, policy_version 12288 (0.0007) [2023-12-27 05:45:07,423][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000011072_2834432.pth [2023-12-27 05:45:07,853][20347] Updated weights for policy 0, policy_version 12327 (0.0010) [2023-12-27 05:45:07,914][20347] Updated weights for policy 0, policy_version 12337 (0.0010) [2023-12-27 05:45:07,968][20347] Updated weights for policy 0, policy_version 12347 (0.0010) [2023-12-27 05:45:08,066][20381] Updated weights for policy 1, policy_version 12298 (0.0007) [2023-12-27 05:45:08,116][20381] Updated weights for policy 1, policy_version 12308 (0.0008) [2023-12-27 05:45:08,173][20381] Updated weights for policy 1, policy_version 12318 (0.0008) [2023-12-27 05:45:08,654][20347] Updated weights for policy 0, policy_version 12357 (0.0010) [2023-12-27 05:45:08,715][20347] Updated weights for policy 0, policy_version 12367 (0.0010) [2023-12-27 05:45:08,780][20347] Updated weights for policy 0, policy_version 12377 (0.0009) [2023-12-27 05:45:08,924][20381] Updated weights for policy 1, policy_version 12328 (0.0007) [2023-12-27 05:45:08,986][20381] Updated weights for policy 1, policy_version 12338 (0.0007) [2023-12-27 05:45:09,044][20381] Updated weights for policy 1, policy_version 12348 (0.0007) [2023-12-27 05:45:09,445][20347] Updated weights for policy 0, policy_version 12387 (0.0007) [2023-12-27 05:45:09,504][20347] Updated weights for policy 0, policy_version 12397 (0.0009) [2023-12-27 05:45:09,556][20347] Updated weights for policy 0, policy_version 12407 (0.0009) [2023-12-27 05:45:09,837][20381] Updated weights for policy 1, policy_version 12358 (0.0008) [2023-12-27 05:45:09,902][20381] Updated weights for policy 1, policy_version 12368 (0.0009) [2023-12-27 05:45:09,963][20381] Updated weights for policy 1, policy_version 12378 (0.0009) [2023-12-27 05:45:10,430][20347] Updated weights for policy 0, policy_version 12417 (0.0008) [2023-12-27 05:45:10,487][20347] Updated weights for policy 0, policy_version 12427 (0.0005) [2023-12-27 05:45:10,540][20347] Updated weights for policy 0, policy_version 12437 (0.0005) [2023-12-27 05:45:10,587][20347] Updated weights for policy 0, policy_version 12447 (0.0005) [2023-12-27 05:45:10,794][20381] Updated weights for policy 1, policy_version 12388 (0.0009) [2023-12-27 05:45:10,856][20381] Updated weights for policy 1, policy_version 12398 (0.0008) [2023-12-27 05:45:10,917][20381] Updated weights for policy 1, policy_version 12408 (0.0008) [2023-12-27 05:45:11,180][20347] Updated weights for policy 0, policy_version 12457 (0.0008) [2023-12-27 05:45:11,237][20347] Updated weights for policy 0, policy_version 12467 (0.0007) [2023-12-27 05:45:11,304][20347] Updated weights for policy 0, policy_version 12477 (0.0008) [2023-12-27 05:45:11,672][20381] Updated weights for policy 1, policy_version 12418 (0.0006) [2023-12-27 05:45:11,734][20381] Updated weights for policy 1, policy_version 12428 (0.0008) [2023-12-27 05:45:11,794][20381] Updated weights for policy 1, policy_version 12438 (0.0008) [2023-12-27 05:45:11,855][20381] Updated weights for policy 1, policy_version 12448 (0.0008) [2023-12-27 05:45:12,088][20347] Updated weights for policy 0, policy_version 12487 (0.0009) [2023-12-27 05:45:12,158][20347] Updated weights for policy 0, policy_version 12497 (0.0008) [2023-12-27 05:45:12,224][20347] Updated weights for policy 0, policy_version 12507 (0.0008) [2023-12-27 05:45:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 6389760. Throughput: 0: 10382.7, 1: 10217.1. Samples: 6379536. Policy #0 lag: (min: 8.0, avg: 32.8, max: 40.0) [2023-12-27 05:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:12,573][20381] Updated weights for policy 1, policy_version 12458 (0.0005) [2023-12-27 05:45:12,633][20381] Updated weights for policy 1, policy_version 12468 (0.0006) [2023-12-27 05:45:12,687][20381] Updated weights for policy 1, policy_version 12478 (0.0008) [2023-12-27 05:45:12,918][20347] Updated weights for policy 0, policy_version 12517 (0.0008) [2023-12-27 05:45:12,977][20347] Updated weights for policy 0, policy_version 12527 (0.0007) [2023-12-27 05:45:13,044][20347] Updated weights for policy 0, policy_version 12537 (0.0007) [2023-12-27 05:45:13,349][20381] Updated weights for policy 1, policy_version 12488 (0.0007) [2023-12-27 05:45:13,407][20381] Updated weights for policy 1, policy_version 12498 (0.0008) [2023-12-27 05:45:13,458][20381] Updated weights for policy 1, policy_version 12508 (0.0006) [2023-12-27 05:45:13,745][20347] Updated weights for policy 0, policy_version 12547 (0.0008) [2023-12-27 05:45:13,806][20347] Updated weights for policy 0, policy_version 12557 (0.0008) [2023-12-27 05:45:13,858][20347] Updated weights for policy 0, policy_version 12567 (0.0007) [2023-12-27 05:45:14,087][20381] Updated weights for policy 1, policy_version 12518 (0.0007) [2023-12-27 05:45:14,147][20381] Updated weights for policy 1, policy_version 12528 (0.0008) [2023-12-27 05:45:14,207][20381] Updated weights for policy 1, policy_version 12538 (0.0009) [2023-12-27 05:45:14,508][20347] Updated weights for policy 0, policy_version 12577 (0.0008) [2023-12-27 05:45:14,570][20347] Updated weights for policy 0, policy_version 12587 (0.0008) [2023-12-27 05:45:14,627][20347] Updated weights for policy 0, policy_version 12597 (0.0009) [2023-12-27 05:45:14,687][20347] Updated weights for policy 0, policy_version 12607 (0.0006) [2023-12-27 05:45:14,948][20381] Updated weights for policy 1, policy_version 12548 (0.0008) [2023-12-27 05:45:15,006][20381] Updated weights for policy 1, policy_version 12558 (0.0009) [2023-12-27 05:45:15,057][20381] Updated weights for policy 1, policy_version 12568 (0.0010) [2023-12-27 05:45:15,293][20347] Updated weights for policy 0, policy_version 12617 (0.0006) [2023-12-27 05:45:15,354][20347] Updated weights for policy 0, policy_version 12627 (0.0005) [2023-12-27 05:45:15,410][20347] Updated weights for policy 0, policy_version 12637 (0.0005) [2023-12-27 05:45:15,780][20381] Updated weights for policy 1, policy_version 12578 (0.0009) [2023-12-27 05:45:15,834][20381] Updated weights for policy 1, policy_version 12588 (0.0008) [2023-12-27 05:45:15,885][20381] Updated weights for policy 1, policy_version 12598 (0.0007) [2023-12-27 05:45:15,929][20381] Updated weights for policy 1, policy_version 12608 (0.0007) [2023-12-27 05:45:15,997][20347] Updated weights for policy 0, policy_version 12647 (0.0008) [2023-12-27 05:45:16,042][20347] Updated weights for policy 0, policy_version 12657 (0.0009) [2023-12-27 05:45:16,091][20347] Updated weights for policy 0, policy_version 12667 (0.0008) [2023-12-27 05:45:16,515][20381] Updated weights for policy 1, policy_version 12618 (0.0005) [2023-12-27 05:45:16,563][20381] Updated weights for policy 1, policy_version 12628 (0.0007) [2023-12-27 05:45:16,624][20381] Updated weights for policy 1, policy_version 12638 (0.0008) [2023-12-27 05:45:16,762][20347] Updated weights for policy 0, policy_version 12677 (0.0010) [2023-12-27 05:45:16,813][20347] Updated weights for policy 0, policy_version 12687 (0.0010) [2023-12-27 05:45:16,859][20347] Updated weights for policy 0, policy_version 12697 (0.0010) [2023-12-27 05:45:17,266][20381] Updated weights for policy 1, policy_version 12648 (0.0006) [2023-12-27 05:45:17,323][20381] Updated weights for policy 1, policy_version 12658 (0.0006) [2023-12-27 05:45:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 6488064. Throughput: 0: 10347.3, 1: 10205.4. Samples: 6503040. Policy #0 lag: (min: 19.0, avg: 27.1, max: 51.0) [2023-12-27 05:45:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:17,375][20381] Updated weights for policy 1, policy_version 12668 (0.0009) [2023-12-27 05:45:17,480][20347] Updated weights for policy 0, policy_version 12707 (0.0009) [2023-12-27 05:45:17,537][20347] Updated weights for policy 0, policy_version 12717 (0.0007) [2023-12-27 05:45:17,594][20347] Updated weights for policy 0, policy_version 12727 (0.0008) [2023-12-27 05:45:18,083][20381] Updated weights for policy 1, policy_version 12678 (0.0007) [2023-12-27 05:45:18,141][20381] Updated weights for policy 1, policy_version 12688 (0.0007) [2023-12-27 05:45:18,175][20347] Updated weights for policy 0, policy_version 12737 (0.0006) [2023-12-27 05:45:18,193][20381] Updated weights for policy 1, policy_version 12698 (0.0009) [2023-12-27 05:45:18,231][20347] Updated weights for policy 0, policy_version 12747 (0.0005) [2023-12-27 05:45:18,295][20347] Updated weights for policy 0, policy_version 12757 (0.0007) [2023-12-27 05:45:18,356][20347] Updated weights for policy 0, policy_version 12767 (0.0010) [2023-12-27 05:45:18,891][20381] Updated weights for policy 1, policy_version 12708 (0.0009) [2023-12-27 05:45:18,949][20381] Updated weights for policy 1, policy_version 12718 (0.0007) [2023-12-27 05:45:18,996][20347] Updated weights for policy 0, policy_version 12777 (0.0009) [2023-12-27 05:45:19,005][20381] Updated weights for policy 1, policy_version 12728 (0.0006) [2023-12-27 05:45:19,057][20347] Updated weights for policy 0, policy_version 12787 (0.0008) [2023-12-27 05:45:19,121][20347] Updated weights for policy 0, policy_version 12797 (0.0008) [2023-12-27 05:45:19,657][20381] Updated weights for policy 1, policy_version 12738 (0.0005) [2023-12-27 05:45:19,718][20381] Updated weights for policy 1, policy_version 12748 (0.0005) [2023-12-27 05:45:19,773][20381] Updated weights for policy 1, policy_version 12758 (0.0005) [2023-12-27 05:45:19,839][20381] Updated weights for policy 1, policy_version 12768 (0.0007) [2023-12-27 05:45:19,858][20347] Updated weights for policy 0, policy_version 12807 (0.0008) [2023-12-27 05:45:19,916][20347] Updated weights for policy 0, policy_version 12818 (0.0008) [2023-12-27 05:45:19,986][20347] Updated weights for policy 0, policy_version 12828 (0.0008) [2023-12-27 05:45:20,472][20381] Updated weights for policy 1, policy_version 12778 (0.0007) [2023-12-27 05:45:20,530][20381] Updated weights for policy 1, policy_version 12788 (0.0006) [2023-12-27 05:45:20,597][20381] Updated weights for policy 1, policy_version 12798 (0.0008) [2023-12-27 05:45:20,713][20347] Updated weights for policy 0, policy_version 12838 (0.0008) [2023-12-27 05:45:20,769][20347] Updated weights for policy 0, policy_version 12848 (0.0007) [2023-12-27 05:45:20,834][20347] Updated weights for policy 0, policy_version 12858 (0.0008) [2023-12-27 05:45:21,314][20381] Updated weights for policy 1, policy_version 12808 (0.0009) [2023-12-27 05:45:21,383][20381] Updated weights for policy 1, policy_version 12818 (0.0008) [2023-12-27 05:45:21,446][20381] Updated weights for policy 1, policy_version 12828 (0.0008) [2023-12-27 05:45:21,506][20347] Updated weights for policy 0, policy_version 12868 (0.0007) [2023-12-27 05:45:21,566][20347] Updated weights for policy 0, policy_version 12878 (0.0008) [2023-12-27 05:45:21,631][20347] Updated weights for policy 0, policy_version 12888 (0.0007) [2023-12-27 05:45:22,173][20381] Updated weights for policy 1, policy_version 12838 (0.0007) [2023-12-27 05:45:22,232][20381] Updated weights for policy 1, policy_version 12848 (0.0006) [2023-12-27 05:45:22,286][20347] Updated weights for policy 0, policy_version 12898 (0.0006) [2023-12-27 05:45:22,290][20381] Updated weights for policy 1, policy_version 12858 (0.0009) [2023-12-27 05:45:22,349][20347] Updated weights for policy 0, policy_version 12908 (0.0007) [2023-12-27 05:45:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 6594560. Throughput: 0: 10374.6, 1: 10146.4. Samples: 6565052. Policy #0 lag: (min: 19.0, avg: 27.1, max: 51.0) [2023-12-27 05:45:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000012864_3293184.pth... [2023-12-27 05:45:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000011680_2990080.pth [2023-12-27 05:45:22,411][20347] Updated weights for policy 0, policy_version 12918 (0.0008) [2023-12-27 05:45:22,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000012928_3309568.pth... [2023-12-27 05:45:22,464][20347] Updated weights for policy 0, policy_version 12928 (0.0008) [2023-12-27 05:45:22,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000011712_2998272.pth [2023-12-27 05:45:22,969][20381] Updated weights for policy 1, policy_version 12868 (0.0011) [2023-12-27 05:45:23,029][20381] Updated weights for policy 1, policy_version 12878 (0.0011) [2023-12-27 05:45:23,089][20381] Updated weights for policy 1, policy_version 12888 (0.0010) [2023-12-27 05:45:23,102][20347] Updated weights for policy 0, policy_version 12938 (0.0010) [2023-12-27 05:45:23,158][20347] Updated weights for policy 0, policy_version 12948 (0.0010) [2023-12-27 05:45:23,216][20347] Updated weights for policy 0, policy_version 12958 (0.0010) [2023-12-27 05:45:23,788][20381] Updated weights for policy 1, policy_version 12898 (0.0011) [2023-12-27 05:45:23,836][20381] Updated weights for policy 1, policy_version 12908 (0.0010) [2023-12-27 05:45:23,890][20347] Updated weights for policy 0, policy_version 12968 (0.0010) [2023-12-27 05:45:23,891][20381] Updated weights for policy 1, policy_version 12918 (0.0011) [2023-12-27 05:45:23,939][20347] Updated weights for policy 0, policy_version 12978 (0.0010) [2023-12-27 05:45:23,943][20381] Updated weights for policy 1, policy_version 12928 (0.0011) [2023-12-27 05:45:23,990][20347] Updated weights for policy 0, policy_version 12988 (0.0010) [2023-12-27 05:45:24,630][20381] Updated weights for policy 1, policy_version 12938 (0.0008) [2023-12-27 05:45:24,657][20347] Updated weights for policy 0, policy_version 12998 (0.0008) [2023-12-27 05:45:24,680][20381] Updated weights for policy 1, policy_version 12948 (0.0008) [2023-12-27 05:45:24,709][20347] Updated weights for policy 0, policy_version 13008 (0.0007) [2023-12-27 05:45:24,733][20381] Updated weights for policy 1, policy_version 12958 (0.0006) [2023-12-27 05:45:24,759][20347] Updated weights for policy 0, policy_version 13018 (0.0006) [2023-12-27 05:45:25,290][20381] Updated weights for policy 1, policy_version 12968 (0.0005) [2023-12-27 05:45:25,349][20381] Updated weights for policy 1, policy_version 12978 (0.0005) [2023-12-27 05:45:25,356][20347] Updated weights for policy 0, policy_version 13028 (0.0005) [2023-12-27 05:45:25,402][20381] Updated weights for policy 1, policy_version 12988 (0.0006) [2023-12-27 05:45:25,408][20347] Updated weights for policy 0, policy_version 13038 (0.0006) [2023-12-27 05:45:25,461][20347] Updated weights for policy 0, policy_version 13048 (0.0006) [2023-12-27 05:45:25,934][20381] Updated weights for policy 1, policy_version 12998 (0.0010) [2023-12-27 05:45:25,986][20381] Updated weights for policy 1, policy_version 13008 (0.0010) [2023-12-27 05:45:26,044][20381] Updated weights for policy 1, policy_version 13018 (0.0010) [2023-12-27 05:45:26,170][20347] Updated weights for policy 0, policy_version 13058 (0.0007) [2023-12-27 05:45:26,222][20347] Updated weights for policy 0, policy_version 13068 (0.0008) [2023-12-27 05:45:26,273][20347] Updated weights for policy 0, policy_version 13078 (0.0007) [2023-12-27 05:45:26,320][20347] Updated weights for policy 0, policy_version 13088 (0.0007) [2023-12-27 05:45:26,680][20381] Updated weights for policy 1, policy_version 13028 (0.0009) [2023-12-27 05:45:26,722][20381] Updated weights for policy 1, policy_version 13038 (0.0008) [2023-12-27 05:45:26,770][20381] Updated weights for policy 1, policy_version 13048 (0.0007) [2023-12-27 05:45:27,012][20347] Updated weights for policy 0, policy_version 13098 (0.0007) [2023-12-27 05:45:27,068][20347] Updated weights for policy 0, policy_version 13108 (0.0007) [2023-12-27 05:45:27,127][20347] Updated weights for policy 0, policy_version 13118 (0.0007) [2023-12-27 05:45:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 6701056. Throughput: 0: 10356.5, 1: 10157.9. Samples: 6690160. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 05:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:27,457][20381] Updated weights for policy 1, policy_version 13058 (0.0007) [2023-12-27 05:45:27,510][20381] Updated weights for policy 1, policy_version 13068 (0.0005) [2023-12-27 05:45:27,575][20381] Updated weights for policy 1, policy_version 13078 (0.0005) [2023-12-27 05:45:27,624][20381] Updated weights for policy 1, policy_version 13088 (0.0007) [2023-12-27 05:45:27,674][20347] Updated weights for policy 0, policy_version 13128 (0.0005) [2023-12-27 05:45:27,724][20347] Updated weights for policy 0, policy_version 13138 (0.0005) [2023-12-27 05:45:27,783][20347] Updated weights for policy 0, policy_version 13148 (0.0007) [2023-12-27 05:45:28,210][20381] Updated weights for policy 1, policy_version 13098 (0.0008) [2023-12-27 05:45:28,262][20381] Updated weights for policy 1, policy_version 13108 (0.0006) [2023-12-27 05:45:28,317][20381] Updated weights for policy 1, policy_version 13118 (0.0006) [2023-12-27 05:45:28,470][20347] Updated weights for policy 0, policy_version 13158 (0.0008) [2023-12-27 05:45:28,518][20347] Updated weights for policy 0, policy_version 13168 (0.0008) [2023-12-27 05:45:28,579][20347] Updated weights for policy 0, policy_version 13178 (0.0005) [2023-12-27 05:45:28,906][20381] Updated weights for policy 1, policy_version 13128 (0.0007) [2023-12-27 05:45:28,956][20381] Updated weights for policy 1, policy_version 13138 (0.0008) [2023-12-27 05:45:29,015][20381] Updated weights for policy 1, policy_version 13148 (0.0008) [2023-12-27 05:45:29,230][20347] Updated weights for policy 0, policy_version 13188 (0.0006) [2023-12-27 05:45:29,287][20347] Updated weights for policy 0, policy_version 13198 (0.0007) [2023-12-27 05:45:29,345][20347] Updated weights for policy 0, policy_version 13208 (0.0008) [2023-12-27 05:45:29,769][20381] Updated weights for policy 1, policy_version 13158 (0.0008) [2023-12-27 05:45:29,830][20381] Updated weights for policy 1, policy_version 13168 (0.0008) [2023-12-27 05:45:29,885][20381] Updated weights for policy 1, policy_version 13178 (0.0005) [2023-12-27 05:45:29,930][20347] Updated weights for policy 0, policy_version 13218 (0.0008) [2023-12-27 05:45:29,995][20347] Updated weights for policy 0, policy_version 13228 (0.0008) [2023-12-27 05:45:30,054][20347] Updated weights for policy 0, policy_version 13238 (0.0007) [2023-12-27 05:45:30,117][20347] Updated weights for policy 0, policy_version 13248 (0.0008) [2023-12-27 05:45:30,502][20381] Updated weights for policy 1, policy_version 13188 (0.0007) [2023-12-27 05:45:30,549][20381] Updated weights for policy 1, policy_version 13198 (0.0009) [2023-12-27 05:45:30,609][20381] Updated weights for policy 1, policy_version 13208 (0.0007) [2023-12-27 05:45:30,768][20347] Updated weights for policy 0, policy_version 13258 (0.0005) [2023-12-27 05:45:30,814][20347] Updated weights for policy 0, policy_version 13268 (0.0008) [2023-12-27 05:45:30,857][20347] Updated weights for policy 0, policy_version 13278 (0.0010) [2023-12-27 05:45:31,343][20381] Updated weights for policy 1, policy_version 13218 (0.0008) [2023-12-27 05:45:31,410][20381] Updated weights for policy 1, policy_version 13228 (0.0009) [2023-12-27 05:45:31,463][20381] Updated weights for policy 1, policy_version 13238 (0.0007) [2023-12-27 05:45:31,482][20347] Updated weights for policy 0, policy_version 13288 (0.0009) [2023-12-27 05:45:31,518][20381] Updated weights for policy 1, policy_version 13248 (0.0005) [2023-12-27 05:45:31,539][20347] Updated weights for policy 0, policy_version 13298 (0.0007) [2023-12-27 05:45:31,596][20347] Updated weights for policy 0, policy_version 13308 (0.0006) [2023-12-27 05:45:32,191][20347] Updated weights for policy 0, policy_version 13318 (0.0009) [2023-12-27 05:45:32,193][20381] Updated weights for policy 1, policy_version 13258 (0.0007) [2023-12-27 05:45:32,246][20347] Updated weights for policy 0, policy_version 13328 (0.0007) [2023-12-27 05:45:32,250][20381] Updated weights for policy 1, policy_version 13268 (0.0005) [2023-12-27 05:45:32,305][20347] Updated weights for policy 0, policy_version 13338 (0.0007) [2023-12-27 05:45:32,311][20381] Updated weights for policy 1, policy_version 13278 (0.0006) [2023-12-27 05:45:32,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 6815744. Throughput: 0: 10375.9, 1: 10214.7. Samples: 6818528. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 05:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:32,910][20347] Updated weights for policy 0, policy_version 13348 (0.0007) [2023-12-27 05:45:32,961][20347] Updated weights for policy 0, policy_version 13358 (0.0007) [2023-12-27 05:45:33,013][20347] Updated weights for policy 0, policy_version 13368 (0.0007) [2023-12-27 05:45:33,022][20381] Updated weights for policy 1, policy_version 13288 (0.0005) [2023-12-27 05:45:33,072][20381] Updated weights for policy 1, policy_version 13298 (0.0006) [2023-12-27 05:45:33,126][20381] Updated weights for policy 1, policy_version 13308 (0.0007) [2023-12-27 05:45:33,640][20347] Updated weights for policy 0, policy_version 13378 (0.0008) [2023-12-27 05:45:33,697][20347] Updated weights for policy 0, policy_version 13388 (0.0010) [2023-12-27 05:45:33,756][20347] Updated weights for policy 0, policy_version 13398 (0.0010) [2023-12-27 05:45:33,806][20347] Updated weights for policy 0, policy_version 13408 (0.0006) [2023-12-27 05:45:33,846][20381] Updated weights for policy 1, policy_version 13318 (0.0008) [2023-12-27 05:45:33,891][20381] Updated weights for policy 1, policy_version 13328 (0.0007) [2023-12-27 05:45:33,937][20381] Updated weights for policy 1, policy_version 13338 (0.0007) [2023-12-27 05:45:34,517][20347] Updated weights for policy 0, policy_version 13418 (0.0007) [2023-12-27 05:45:34,577][20347] Updated weights for policy 0, policy_version 13428 (0.0008) [2023-12-27 05:45:34,634][20347] Updated weights for policy 0, policy_version 13438 (0.0008) [2023-12-27 05:45:34,675][20381] Updated weights for policy 1, policy_version 13348 (0.0008) [2023-12-27 05:45:34,740][20381] Updated weights for policy 1, policy_version 13358 (0.0008) [2023-12-27 05:45:34,794][20381] Updated weights for policy 1, policy_version 13368 (0.0008) [2023-12-27 05:45:35,241][20347] Updated weights for policy 0, policy_version 13448 (0.0006) [2023-12-27 05:45:35,301][20347] Updated weights for policy 0, policy_version 13458 (0.0005) [2023-12-27 05:45:35,364][20347] Updated weights for policy 0, policy_version 13468 (0.0007) [2023-12-27 05:45:35,558][20381] Updated weights for policy 1, policy_version 13378 (0.0007) [2023-12-27 05:45:35,611][20381] Updated weights for policy 1, policy_version 13388 (0.0005) [2023-12-27 05:45:35,662][20381] Updated weights for policy 1, policy_version 13398 (0.0006) [2023-12-27 05:45:35,713][20381] Updated weights for policy 1, policy_version 13408 (0.0005) [2023-12-27 05:45:35,958][20347] Updated weights for policy 0, policy_version 13478 (0.0009) [2023-12-27 05:45:36,017][20347] Updated weights for policy 0, policy_version 13488 (0.0007) [2023-12-27 05:45:36,075][20347] Updated weights for policy 0, policy_version 13498 (0.0005) [2023-12-27 05:45:36,368][20381] Updated weights for policy 1, policy_version 13418 (0.0006) [2023-12-27 05:45:36,427][20381] Updated weights for policy 1, policy_version 13428 (0.0007) [2023-12-27 05:45:36,491][20381] Updated weights for policy 1, policy_version 13438 (0.0006) [2023-12-27 05:45:36,751][20347] Updated weights for policy 0, policy_version 13508 (0.0008) [2023-12-27 05:45:36,810][20347] Updated weights for policy 0, policy_version 13518 (0.0007) [2023-12-27 05:45:36,867][20347] Updated weights for policy 0, policy_version 13528 (0.0010) [2023-12-27 05:45:37,080][20381] Updated weights for policy 1, policy_version 13448 (0.0007) [2023-12-27 05:45:37,140][20381] Updated weights for policy 1, policy_version 13458 (0.0008) [2023-12-27 05:45:37,200][20381] Updated weights for policy 1, policy_version 13468 (0.0008) [2023-12-27 05:45:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 6914048. Throughput: 0: 10421.4, 1: 10185.7. Samples: 6879792. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 05:45:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000013536_3465216.pth... [2023-12-27 05:45:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000013472_3448832.pth... [2023-12-27 05:45:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000012320_3153920.pth [2023-12-27 05:45:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000012288_3145728.pth [2023-12-27 05:45:37,569][20347] Updated weights for policy 0, policy_version 13538 (0.0010) [2023-12-27 05:45:37,617][20347] Updated weights for policy 0, policy_version 13548 (0.0010) [2023-12-27 05:45:37,669][20347] Updated weights for policy 0, policy_version 13558 (0.0010) [2023-12-27 05:45:37,727][20347] Updated weights for policy 0, policy_version 13568 (0.0011) [2023-12-27 05:45:37,902][20381] Updated weights for policy 1, policy_version 13478 (0.0008) [2023-12-27 05:45:37,957][20381] Updated weights for policy 1, policy_version 13488 (0.0007) [2023-12-27 05:45:38,005][20381] Updated weights for policy 1, policy_version 13498 (0.0007) [2023-12-27 05:45:38,427][20347] Updated weights for policy 0, policy_version 13578 (0.0009) [2023-12-27 05:45:38,484][20347] Updated weights for policy 0, policy_version 13588 (0.0007) [2023-12-27 05:45:38,545][20347] Updated weights for policy 0, policy_version 13598 (0.0010) [2023-12-27 05:45:38,746][20381] Updated weights for policy 1, policy_version 13508 (0.0007) [2023-12-27 05:45:38,798][20381] Updated weights for policy 1, policy_version 13518 (0.0008) [2023-12-27 05:45:38,849][20381] Updated weights for policy 1, policy_version 13528 (0.0007) [2023-12-27 05:45:39,227][20347] Updated weights for policy 0, policy_version 13608 (0.0010) [2023-12-27 05:45:39,280][20347] Updated weights for policy 0, policy_version 13618 (0.0011) [2023-12-27 05:45:39,333][20347] Updated weights for policy 0, policy_version 13628 (0.0010) [2023-12-27 05:45:39,591][20381] Updated weights for policy 1, policy_version 13538 (0.0008) [2023-12-27 05:45:39,645][20381] Updated weights for policy 1, policy_version 13548 (0.0007) [2023-12-27 05:45:39,697][20381] Updated weights for policy 1, policy_version 13558 (0.0007) [2023-12-27 05:45:39,750][20381] Updated weights for policy 1, policy_version 13568 (0.0007) [2023-12-27 05:45:40,051][20347] Updated weights for policy 0, policy_version 13638 (0.0011) [2023-12-27 05:45:40,114][20347] Updated weights for policy 0, policy_version 13648 (0.0011) [2023-12-27 05:45:40,167][20347] Updated weights for policy 0, policy_version 13658 (0.0011) [2023-12-27 05:45:40,478][20381] Updated weights for policy 1, policy_version 13578 (0.0008) [2023-12-27 05:45:40,527][20381] Updated weights for policy 1, policy_version 13588 (0.0007) [2023-12-27 05:45:40,572][20381] Updated weights for policy 1, policy_version 13598 (0.0007) [2023-12-27 05:45:40,873][20347] Updated weights for policy 0, policy_version 13668 (0.0010) [2023-12-27 05:45:40,919][20347] Updated weights for policy 0, policy_version 13678 (0.0010) [2023-12-27 05:45:40,969][20347] Updated weights for policy 0, policy_version 13688 (0.0010) [2023-12-27 05:45:41,335][20381] Updated weights for policy 1, policy_version 13608 (0.0007) [2023-12-27 05:45:41,399][20381] Updated weights for policy 1, policy_version 13618 (0.0008) [2023-12-27 05:45:41,449][20381] Updated weights for policy 1, policy_version 13628 (0.0008) [2023-12-27 05:45:41,752][20347] Updated weights for policy 0, policy_version 13698 (0.0010) [2023-12-27 05:45:41,814][20347] Updated weights for policy 0, policy_version 13708 (0.0010) [2023-12-27 05:45:41,882][20347] Updated weights for policy 0, policy_version 13718 (0.0010) [2023-12-27 05:45:41,943][20347] Updated weights for policy 0, policy_version 13728 (0.0010) [2023-12-27 05:45:42,181][20381] Updated weights for policy 1, policy_version 13638 (0.0007) [2023-12-27 05:45:42,233][20381] Updated weights for policy 1, policy_version 13648 (0.0008) [2023-12-27 05:45:42,294][20381] Updated weights for policy 1, policy_version 13658 (0.0008) [2023-12-27 05:45:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 7012352. Throughput: 0: 10430.6, 1: 10118.2. Samples: 7000968. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 05:45:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:42,660][20347] Updated weights for policy 0, policy_version 13738 (0.0010) [2023-12-27 05:45:42,715][20347] Updated weights for policy 0, policy_version 13748 (0.0010) [2023-12-27 05:45:42,764][20347] Updated weights for policy 0, policy_version 13758 (0.0010) [2023-12-27 05:45:43,033][20381] Updated weights for policy 1, policy_version 13668 (0.0007) [2023-12-27 05:45:43,082][20381] Updated weights for policy 1, policy_version 13678 (0.0007) [2023-12-27 05:45:43,128][20381] Updated weights for policy 1, policy_version 13688 (0.0007) [2023-12-27 05:45:43,459][20347] Updated weights for policy 0, policy_version 13768 (0.0010) [2023-12-27 05:45:43,513][20347] Updated weights for policy 0, policy_version 13778 (0.0010) [2023-12-27 05:45:43,565][20347] Updated weights for policy 0, policy_version 13788 (0.0010) [2023-12-27 05:45:43,857][20381] Updated weights for policy 1, policy_version 13698 (0.0007) [2023-12-27 05:45:43,905][20381] Updated weights for policy 1, policy_version 13708 (0.0007) [2023-12-27 05:45:43,957][20381] Updated weights for policy 1, policy_version 13718 (0.0007) [2023-12-27 05:45:44,012][20381] Updated weights for policy 1, policy_version 13728 (0.0007) [2023-12-27 05:45:44,263][20347] Updated weights for policy 0, policy_version 13798 (0.0010) [2023-12-27 05:45:44,322][20347] Updated weights for policy 0, policy_version 13808 (0.0010) [2023-12-27 05:45:44,372][20347] Updated weights for policy 0, policy_version 13818 (0.0010) [2023-12-27 05:45:44,726][20381] Updated weights for policy 1, policy_version 13738 (0.0007) [2023-12-27 05:45:44,780][20381] Updated weights for policy 1, policy_version 13748 (0.0008) [2023-12-27 05:45:44,833][20381] Updated weights for policy 1, policy_version 13758 (0.0007) [2023-12-27 05:45:45,046][20347] Updated weights for policy 0, policy_version 13828 (0.0010) [2023-12-27 05:45:45,098][20347] Updated weights for policy 0, policy_version 13838 (0.0010) [2023-12-27 05:45:45,160][20347] Updated weights for policy 0, policy_version 13848 (0.0010) [2023-12-27 05:45:45,559][20381] Updated weights for policy 1, policy_version 13768 (0.0007) [2023-12-27 05:45:45,614][20381] Updated weights for policy 1, policy_version 13778 (0.0007) [2023-12-27 05:45:45,663][20381] Updated weights for policy 1, policy_version 13788 (0.0007) [2023-12-27 05:45:45,843][20347] Updated weights for policy 0, policy_version 13858 (0.0010) [2023-12-27 05:45:45,897][20347] Updated weights for policy 0, policy_version 13868 (0.0009) [2023-12-27 05:45:45,958][20347] Updated weights for policy 0, policy_version 13878 (0.0007) [2023-12-27 05:45:46,011][20347] Updated weights for policy 0, policy_version 13888 (0.0006) [2023-12-27 05:45:46,363][20381] Updated weights for policy 1, policy_version 13798 (0.0007) [2023-12-27 05:45:46,421][20381] Updated weights for policy 1, policy_version 13808 (0.0007) [2023-12-27 05:45:46,489][20381] Updated weights for policy 1, policy_version 13818 (0.0007) [2023-12-27 05:45:46,692][20347] Updated weights for policy 0, policy_version 13898 (0.0010) [2023-12-27 05:45:46,749][20347] Updated weights for policy 0, policy_version 13908 (0.0010) [2023-12-27 05:45:46,803][20347] Updated weights for policy 0, policy_version 13918 (0.0010) [2023-12-27 05:45:47,183][20381] Updated weights for policy 1, policy_version 13828 (0.0008) [2023-12-27 05:45:47,232][20381] Updated weights for policy 1, policy_version 13838 (0.0007) [2023-12-27 05:45:47,275][20381] Updated weights for policy 1, policy_version 13848 (0.0007) [2023-12-27 05:45:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 7110656. Throughput: 0: 10372.3, 1: 10096.9. Samples: 7119856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:45:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:47,479][20347] Updated weights for policy 0, policy_version 13928 (0.0010) [2023-12-27 05:45:47,537][20347] Updated weights for policy 0, policy_version 13938 (0.0010) [2023-12-27 05:45:47,594][20347] Updated weights for policy 0, policy_version 13948 (0.0010) [2023-12-27 05:45:47,987][20381] Updated weights for policy 1, policy_version 13858 (0.0007) [2023-12-27 05:45:48,032][20381] Updated weights for policy 1, policy_version 13868 (0.0007) [2023-12-27 05:45:48,086][20381] Updated weights for policy 1, policy_version 13878 (0.0008) [2023-12-27 05:45:48,134][20381] Updated weights for policy 1, policy_version 13888 (0.0008) [2023-12-27 05:45:48,264][20347] Updated weights for policy 0, policy_version 13958 (0.0010) [2023-12-27 05:45:48,322][20347] Updated weights for policy 0, policy_version 13968 (0.0010) [2023-12-27 05:45:48,380][20347] Updated weights for policy 0, policy_version 13978 (0.0010) [2023-12-27 05:45:48,876][20381] Updated weights for policy 1, policy_version 13898 (0.0008) [2023-12-27 05:45:48,934][20381] Updated weights for policy 1, policy_version 13908 (0.0008) [2023-12-27 05:45:48,998][20381] Updated weights for policy 1, policy_version 13918 (0.0006) [2023-12-27 05:45:49,044][20347] Updated weights for policy 0, policy_version 13988 (0.0008) [2023-12-27 05:45:49,092][20347] Updated weights for policy 0, policy_version 13998 (0.0005) [2023-12-27 05:45:49,152][20347] Updated weights for policy 0, policy_version 14008 (0.0006) [2023-12-27 05:45:49,667][20381] Updated weights for policy 1, policy_version 13928 (0.0008) [2023-12-27 05:45:49,722][20381] Updated weights for policy 1, policy_version 13938 (0.0008) [2023-12-27 05:45:49,781][20381] Updated weights for policy 1, policy_version 13948 (0.0008) [2023-12-27 05:45:49,799][20347] Updated weights for policy 0, policy_version 14018 (0.0007) [2023-12-27 05:45:49,861][20347] Updated weights for policy 0, policy_version 14028 (0.0007) [2023-12-27 05:45:49,915][20347] Updated weights for policy 0, policy_version 14038 (0.0007) [2023-12-27 05:45:49,982][20347] Updated weights for policy 0, policy_version 14048 (0.0007) [2023-12-27 05:45:50,488][20381] Updated weights for policy 1, policy_version 13958 (0.0007) [2023-12-27 05:45:50,548][20381] Updated weights for policy 1, policy_version 13968 (0.0005) [2023-12-27 05:45:50,602][20381] Updated weights for policy 1, policy_version 13978 (0.0007) [2023-12-27 05:45:50,701][20347] Updated weights for policy 0, policy_version 14058 (0.0007) [2023-12-27 05:45:50,763][20347] Updated weights for policy 0, policy_version 14068 (0.0008) [2023-12-27 05:45:50,814][20347] Updated weights for policy 0, policy_version 14078 (0.0009) [2023-12-27 05:45:51,218][20381] Updated weights for policy 1, policy_version 13988 (0.0007) [2023-12-27 05:45:51,281][20381] Updated weights for policy 1, policy_version 13998 (0.0007) [2023-12-27 05:45:51,341][20381] Updated weights for policy 1, policy_version 14008 (0.0007) [2023-12-27 05:45:51,497][20347] Updated weights for policy 0, policy_version 14088 (0.0009) [2023-12-27 05:45:51,548][20347] Updated weights for policy 0, policy_version 14098 (0.0007) [2023-12-27 05:45:51,610][20347] Updated weights for policy 0, policy_version 14108 (0.0008) [2023-12-27 05:45:52,051][20381] Updated weights for policy 1, policy_version 14018 (0.0007) [2023-12-27 05:45:52,109][20381] Updated weights for policy 1, policy_version 14028 (0.0008) [2023-12-27 05:45:52,155][20381] Updated weights for policy 1, policy_version 14038 (0.0008) [2023-12-27 05:45:52,212][20381] Updated weights for policy 1, policy_version 14048 (0.0008) [2023-12-27 05:45:52,237][20347] Updated weights for policy 0, policy_version 14118 (0.0008) [2023-12-27 05:45:52,286][20347] Updated weights for policy 0, policy_version 14128 (0.0009) [2023-12-27 05:45:52,336][20347] Updated weights for policy 0, policy_version 14138 (0.0008) [2023-12-27 05:45:52,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 7208960. Throughput: 0: 10340.8, 1: 10061.4. Samples: 7179760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:45:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000014048_3596288.pth... [2023-12-27 05:45:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000014144_3620864.pth... [2023-12-27 05:45:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000012864_3293184.pth [2023-12-27 05:45:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000012928_3309568.pth [2023-12-27 05:45:52,949][20381] Updated weights for policy 1, policy_version 14058 (0.0008) [2023-12-27 05:45:53,002][20381] Updated weights for policy 1, policy_version 14068 (0.0008) [2023-12-27 05:45:53,054][20381] Updated weights for policy 1, policy_version 14078 (0.0008) [2023-12-27 05:45:53,078][20347] Updated weights for policy 0, policy_version 14148 (0.0009) [2023-12-27 05:45:53,141][20347] Updated weights for policy 0, policy_version 14158 (0.0008) [2023-12-27 05:45:53,193][20347] Updated weights for policy 0, policy_version 14168 (0.0007) [2023-12-27 05:45:53,754][20381] Updated weights for policy 1, policy_version 14088 (0.0008) [2023-12-27 05:45:53,821][20381] Updated weights for policy 1, policy_version 14098 (0.0008) [2023-12-27 05:45:53,881][20381] Updated weights for policy 1, policy_version 14108 (0.0006) [2023-12-27 05:45:53,887][20347] Updated weights for policy 0, policy_version 14178 (0.0008) [2023-12-27 05:45:53,950][20347] Updated weights for policy 0, policy_version 14188 (0.0008) [2023-12-27 05:45:54,004][20347] Updated weights for policy 0, policy_version 14198 (0.0007) [2023-12-27 05:45:54,063][20347] Updated weights for policy 0, policy_version 14208 (0.0008) [2023-12-27 05:45:54,550][20381] Updated weights for policy 1, policy_version 14118 (0.0007) [2023-12-27 05:45:54,606][20381] Updated weights for policy 1, policy_version 14128 (0.0007) [2023-12-27 05:45:54,659][20381] Updated weights for policy 1, policy_version 14138 (0.0007) [2023-12-27 05:45:54,737][20347] Updated weights for policy 0, policy_version 14218 (0.0007) [2023-12-27 05:45:54,782][20347] Updated weights for policy 0, policy_version 14228 (0.0008) [2023-12-27 05:45:54,832][20347] Updated weights for policy 0, policy_version 14238 (0.0007) [2023-12-27 05:45:55,351][20381] Updated weights for policy 1, policy_version 14148 (0.0007) [2023-12-27 05:45:55,401][20381] Updated weights for policy 1, policy_version 14158 (0.0008) [2023-12-27 05:45:55,447][20381] Updated weights for policy 1, policy_version 14168 (0.0007) [2023-12-27 05:45:55,555][20347] Updated weights for policy 0, policy_version 14248 (0.0006) [2023-12-27 05:45:55,616][20347] Updated weights for policy 0, policy_version 14258 (0.0008) [2023-12-27 05:45:55,669][20347] Updated weights for policy 0, policy_version 14268 (0.0007) [2023-12-27 05:45:56,153][20381] Updated weights for policy 1, policy_version 14178 (0.0008) [2023-12-27 05:45:56,202][20381] Updated weights for policy 1, policy_version 14188 (0.0007) [2023-12-27 05:45:56,250][20381] Updated weights for policy 1, policy_version 14198 (0.0008) [2023-12-27 05:45:56,295][20381] Updated weights for policy 1, policy_version 14208 (0.0008) [2023-12-27 05:45:56,339][20347] Updated weights for policy 0, policy_version 14278 (0.0008) [2023-12-27 05:45:56,386][20347] Updated weights for policy 0, policy_version 14288 (0.0007) [2023-12-27 05:45:56,451][20347] Updated weights for policy 0, policy_version 14298 (0.0005) [2023-12-27 05:45:57,022][20381] Updated weights for policy 1, policy_version 14218 (0.0008) [2023-12-27 05:45:57,073][20381] Updated weights for policy 1, policy_version 14228 (0.0007) [2023-12-27 05:45:57,114][20347] Updated weights for policy 0, policy_version 14308 (0.0007) [2023-12-27 05:45:57,130][20381] Updated weights for policy 1, policy_version 14238 (0.0006) [2023-12-27 05:45:57,159][20347] Updated weights for policy 0, policy_version 14318 (0.0006) [2023-12-27 05:45:57,211][20347] Updated weights for policy 0, policy_version 14328 (0.0008) [2023-12-27 05:45:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 7315456. Throughput: 0: 10354.3, 1: 10118.9. Samples: 7300832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:45:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:45:57,833][20381] Updated weights for policy 1, policy_version 14248 (0.0007) [2023-12-27 05:45:57,885][20381] Updated weights for policy 1, policy_version 14258 (0.0007) [2023-12-27 05:45:57,894][20347] Updated weights for policy 0, policy_version 14338 (0.0007) [2023-12-27 05:45:57,936][20381] Updated weights for policy 1, policy_version 14268 (0.0006) [2023-12-27 05:45:57,947][20347] Updated weights for policy 0, policy_version 14348 (0.0006) [2023-12-27 05:45:57,994][20347] Updated weights for policy 0, policy_version 14358 (0.0005) [2023-12-27 05:45:58,042][20347] Updated weights for policy 0, policy_version 14368 (0.0005) [2023-12-27 05:45:58,541][20381] Updated weights for policy 1, policy_version 14278 (0.0007) [2023-12-27 05:45:58,602][20381] Updated weights for policy 1, policy_version 14288 (0.0006) [2023-12-27 05:45:58,662][20381] Updated weights for policy 1, policy_version 14298 (0.0007) [2023-12-27 05:45:58,791][20347] Updated weights for policy 0, policy_version 14378 (0.0008) [2023-12-27 05:45:58,856][20347] Updated weights for policy 0, policy_version 14388 (0.0009) [2023-12-27 05:45:58,920][20347] Updated weights for policy 0, policy_version 14398 (0.0008) [2023-12-27 05:45:59,375][20381] Updated weights for policy 1, policy_version 14308 (0.0008) [2023-12-27 05:45:59,431][20381] Updated weights for policy 1, policy_version 14318 (0.0008) [2023-12-27 05:45:59,486][20381] Updated weights for policy 1, policy_version 14328 (0.0008) [2023-12-27 05:45:59,695][20347] Updated weights for policy 0, policy_version 14408 (0.0010) [2023-12-27 05:45:59,762][20347] Updated weights for policy 0, policy_version 14418 (0.0010) [2023-12-27 05:45:59,812][20347] Updated weights for policy 0, policy_version 14428 (0.0008) [2023-12-27 05:46:00,137][20381] Updated weights for policy 1, policy_version 14338 (0.0007) [2023-12-27 05:46:00,188][20381] Updated weights for policy 1, policy_version 14348 (0.0007) [2023-12-27 05:46:00,240][20381] Updated weights for policy 1, policy_version 14358 (0.0007) [2023-12-27 05:46:00,298][20381] Updated weights for policy 1, policy_version 14368 (0.0007) [2023-12-27 05:46:00,459][20347] Updated weights for policy 0, policy_version 14438 (0.0009) [2023-12-27 05:46:00,516][20347] Updated weights for policy 0, policy_version 14448 (0.0010) [2023-12-27 05:46:00,569][20347] Updated weights for policy 0, policy_version 14458 (0.0010) [2023-12-27 05:46:00,911][20381] Updated weights for policy 1, policy_version 14378 (0.0005) [2023-12-27 05:46:00,962][20381] Updated weights for policy 1, policy_version 14388 (0.0005) [2023-12-27 05:46:01,005][20381] Updated weights for policy 1, policy_version 14398 (0.0005) [2023-12-27 05:46:01,189][20347] Updated weights for policy 0, policy_version 14468 (0.0009) [2023-12-27 05:46:01,243][20347] Updated weights for policy 0, policy_version 14478 (0.0010) [2023-12-27 05:46:01,311][20347] Updated weights for policy 0, policy_version 14488 (0.0010) [2023-12-27 05:46:01,714][20381] Updated weights for policy 1, policy_version 14408 (0.0006) [2023-12-27 05:46:01,771][20381] Updated weights for policy 1, policy_version 14418 (0.0006) [2023-12-27 05:46:01,823][20381] Updated weights for policy 1, policy_version 14428 (0.0009) [2023-12-27 05:46:01,987][20347] Updated weights for policy 0, policy_version 14498 (0.0010) [2023-12-27 05:46:02,043][20347] Updated weights for policy 0, policy_version 14508 (0.0006) [2023-12-27 05:46:02,094][20347] Updated weights for policy 0, policy_version 14518 (0.0008) [2023-12-27 05:46:02,154][20347] Updated weights for policy 0, policy_version 14528 (0.0010) [2023-12-27 05:46:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20716.1). Total num frames: 7413760. Throughput: 0: 10353.4, 1: 10134.5. Samples: 7424992. Policy #0 lag: (min: 27.0, avg: 35.3, max: 59.0) [2023-12-27 05:46:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:02,480][20381] Updated weights for policy 1, policy_version 14438 (0.0008) [2023-12-27 05:46:02,536][20381] Updated weights for policy 1, policy_version 14448 (0.0007) [2023-12-27 05:46:02,587][20381] Updated weights for policy 1, policy_version 14458 (0.0007) [2023-12-27 05:46:02,816][20347] Updated weights for policy 0, policy_version 14538 (0.0010) [2023-12-27 05:46:02,876][20347] Updated weights for policy 0, policy_version 14548 (0.0010) [2023-12-27 05:46:02,933][20347] Updated weights for policy 0, policy_version 14558 (0.0010) [2023-12-27 05:46:03,195][20381] Updated weights for policy 1, policy_version 14468 (0.0008) [2023-12-27 05:46:03,243][20381] Updated weights for policy 1, policy_version 14478 (0.0007) [2023-12-27 05:46:03,295][20381] Updated weights for policy 1, policy_version 14488 (0.0007) [2023-12-27 05:46:03,548][20347] Updated weights for policy 0, policy_version 14568 (0.0006) [2023-12-27 05:46:03,603][20347] Updated weights for policy 0, policy_version 14578 (0.0005) [2023-12-27 05:46:03,661][20347] Updated weights for policy 0, policy_version 14588 (0.0005) [2023-12-27 05:46:04,035][20381] Updated weights for policy 1, policy_version 14498 (0.0007) [2023-12-27 05:46:04,099][20381] Updated weights for policy 1, policy_version 14508 (0.0006) [2023-12-27 05:46:04,162][20381] Updated weights for policy 1, policy_version 14518 (0.0008) [2023-12-27 05:46:04,224][20381] Updated weights for policy 1, policy_version 14528 (0.0009) [2023-12-27 05:46:04,252][20347] Updated weights for policy 0, policy_version 14598 (0.0006) [2023-12-27 05:46:04,314][20347] Updated weights for policy 0, policy_version 14608 (0.0007) [2023-12-27 05:46:04,375][20347] Updated weights for policy 0, policy_version 14618 (0.0008) [2023-12-27 05:46:04,841][20381] Updated weights for policy 1, policy_version 14538 (0.0006) [2023-12-27 05:46:04,890][20381] Updated weights for policy 1, policy_version 14548 (0.0006) [2023-12-27 05:46:04,937][20381] Updated weights for policy 1, policy_version 14558 (0.0007) [2023-12-27 05:46:05,116][20347] Updated weights for policy 0, policy_version 14628 (0.0007) [2023-12-27 05:46:05,178][20347] Updated weights for policy 0, policy_version 14638 (0.0005) [2023-12-27 05:46:05,240][20347] Updated weights for policy 0, policy_version 14648 (0.0005) [2023-12-27 05:46:05,633][20381] Updated weights for policy 1, policy_version 14568 (0.0010) [2023-12-27 05:46:05,680][20381] Updated weights for policy 1, policy_version 14578 (0.0010) [2023-12-27 05:46:05,731][20381] Updated weights for policy 1, policy_version 14588 (0.0010) [2023-12-27 05:46:05,882][20347] Updated weights for policy 0, policy_version 14658 (0.0008) [2023-12-27 05:46:05,943][20347] Updated weights for policy 0, policy_version 14668 (0.0008) [2023-12-27 05:46:06,000][20347] Updated weights for policy 0, policy_version 14678 (0.0007) [2023-12-27 05:46:06,051][20347] Updated weights for policy 0, policy_version 14688 (0.0007) [2023-12-27 05:46:06,439][20381] Updated weights for policy 1, policy_version 14598 (0.0010) [2023-12-27 05:46:06,500][20381] Updated weights for policy 1, policy_version 14608 (0.0011) [2023-12-27 05:46:06,562][20381] Updated weights for policy 1, policy_version 14618 (0.0011) [2023-12-27 05:46:06,793][20347] Updated weights for policy 0, policy_version 14698 (0.0008) [2023-12-27 05:46:06,845][20347] Updated weights for policy 0, policy_version 14708 (0.0007) [2023-12-27 05:46:06,905][20347] Updated weights for policy 0, policy_version 14718 (0.0007) [2023-12-27 05:46:07,249][20381] Updated weights for policy 1, policy_version 14628 (0.0010) [2023-12-27 05:46:07,305][20381] Updated weights for policy 1, policy_version 14638 (0.0005) [2023-12-27 05:46:07,363][20381] Updated weights for policy 1, policy_version 14648 (0.0005) [2023-12-27 05:46:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20688.3). Total num frames: 7512064. Throughput: 0: 10356.0, 1: 10144.2. Samples: 7487560. Policy #0 lag: (min: 27.0, avg: 35.3, max: 59.0) [2023-12-27 05:46:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000014720_3768320.pth... [2023-12-27 05:46:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000013536_3465216.pth [2023-12-27 05:46:07,411][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000014656_3751936.pth... [2023-12-27 05:46:07,415][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000013472_3448832.pth [2023-12-27 05:46:07,626][20347] Updated weights for policy 0, policy_version 14728 (0.0008) [2023-12-27 05:46:07,682][20347] Updated weights for policy 0, policy_version 14738 (0.0007) [2023-12-27 05:46:07,738][20347] Updated weights for policy 0, policy_version 14748 (0.0008) [2023-12-27 05:46:08,036][20381] Updated weights for policy 1, policy_version 14658 (0.0006) [2023-12-27 05:46:08,094][20381] Updated weights for policy 1, policy_version 14668 (0.0010) [2023-12-27 05:46:08,145][20381] Updated weights for policy 1, policy_version 14678 (0.0010) [2023-12-27 05:46:08,206][20381] Updated weights for policy 1, policy_version 14688 (0.0010) [2023-12-27 05:46:08,454][20347] Updated weights for policy 0, policy_version 14758 (0.0006) [2023-12-27 05:46:08,512][20347] Updated weights for policy 0, policy_version 14768 (0.0008) [2023-12-27 05:46:08,568][20347] Updated weights for policy 0, policy_version 14778 (0.0008) [2023-12-27 05:46:08,836][20381] Updated weights for policy 1, policy_version 14698 (0.0008) [2023-12-27 05:46:08,892][20381] Updated weights for policy 1, policy_version 14708 (0.0008) [2023-12-27 05:46:08,952][20381] Updated weights for policy 1, policy_version 14718 (0.0008) [2023-12-27 05:46:09,243][20347] Updated weights for policy 0, policy_version 14788 (0.0007) [2023-12-27 05:46:09,305][20347] Updated weights for policy 0, policy_version 14798 (0.0008) [2023-12-27 05:46:09,376][20347] Updated weights for policy 0, policy_version 14808 (0.0009) [2023-12-27 05:46:09,673][20381] Updated weights for policy 1, policy_version 14728 (0.0006) [2023-12-27 05:46:09,732][20381] Updated weights for policy 1, policy_version 14738 (0.0006) [2023-12-27 05:46:09,790][20381] Updated weights for policy 1, policy_version 14748 (0.0005) [2023-12-27 05:46:10,079][20347] Updated weights for policy 0, policy_version 14818 (0.0008) [2023-12-27 05:46:10,134][20347] Updated weights for policy 0, policy_version 14828 (0.0008) [2023-12-27 05:46:10,192][20347] Updated weights for policy 0, policy_version 14838 (0.0006) [2023-12-27 05:46:10,255][20347] Updated weights for policy 0, policy_version 14848 (0.0007) [2023-12-27 05:46:10,394][20381] Updated weights for policy 1, policy_version 14758 (0.0007) [2023-12-27 05:46:10,458][20381] Updated weights for policy 1, policy_version 14768 (0.0005) [2023-12-27 05:46:10,515][20381] Updated weights for policy 1, policy_version 14778 (0.0007) [2023-12-27 05:46:10,954][20347] Updated weights for policy 0, policy_version 14858 (0.0007) [2023-12-27 05:46:11,011][20347] Updated weights for policy 0, policy_version 14868 (0.0007) [2023-12-27 05:46:11,077][20347] Updated weights for policy 0, policy_version 14878 (0.0008) [2023-12-27 05:46:11,106][20381] Updated weights for policy 1, policy_version 14788 (0.0007) [2023-12-27 05:46:11,166][20381] Updated weights for policy 1, policy_version 14798 (0.0008) [2023-12-27 05:46:11,219][20381] Updated weights for policy 1, policy_version 14808 (0.0008) [2023-12-27 05:46:11,758][20347] Updated weights for policy 0, policy_version 14888 (0.0008) [2023-12-27 05:46:11,809][20347] Updated weights for policy 0, policy_version 14898 (0.0007) [2023-12-27 05:46:11,868][20347] Updated weights for policy 0, policy_version 14908 (0.0007) [2023-12-27 05:46:11,954][20381] Updated weights for policy 1, policy_version 14818 (0.0009) [2023-12-27 05:46:12,017][20381] Updated weights for policy 1, policy_version 14828 (0.0006) [2023-12-27 05:46:12,080][20381] Updated weights for policy 1, policy_version 14838 (0.0006) [2023-12-27 05:46:12,132][20381] Updated weights for policy 1, policy_version 14848 (0.0008) [2023-12-27 05:46:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 7618560. Throughput: 0: 10273.6, 1: 10141.6. Samples: 7608844. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 05:46:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:12,561][20347] Updated weights for policy 0, policy_version 14918 (0.0007) [2023-12-27 05:46:12,620][20347] Updated weights for policy 0, policy_version 14928 (0.0007) [2023-12-27 05:46:12,675][20347] Updated weights for policy 0, policy_version 14938 (0.0007) [2023-12-27 05:46:12,786][20381] Updated weights for policy 1, policy_version 14858 (0.0008) [2023-12-27 05:46:12,836][20381] Updated weights for policy 1, policy_version 14868 (0.0008) [2023-12-27 05:46:12,886][20381] Updated weights for policy 1, policy_version 14878 (0.0008) [2023-12-27 05:46:13,345][20347] Updated weights for policy 0, policy_version 14948 (0.0007) [2023-12-27 05:46:13,400][20347] Updated weights for policy 0, policy_version 14958 (0.0007) [2023-12-27 05:46:13,454][20347] Updated weights for policy 0, policy_version 14968 (0.0007) [2023-12-27 05:46:13,596][20381] Updated weights for policy 1, policy_version 14888 (0.0007) [2023-12-27 05:46:13,641][20381] Updated weights for policy 1, policy_version 14898 (0.0007) [2023-12-27 05:46:13,699][20381] Updated weights for policy 1, policy_version 14908 (0.0007) [2023-12-27 05:46:14,147][20347] Updated weights for policy 0, policy_version 14978 (0.0008) [2023-12-27 05:46:14,203][20347] Updated weights for policy 0, policy_version 14988 (0.0008) [2023-12-27 05:46:14,264][20347] Updated weights for policy 0, policy_version 14998 (0.0008) [2023-12-27 05:46:14,315][20347] Updated weights for policy 0, policy_version 15008 (0.0010) [2023-12-27 05:46:14,386][20381] Updated weights for policy 1, policy_version 14918 (0.0008) [2023-12-27 05:46:14,443][20381] Updated weights for policy 1, policy_version 14928 (0.0007) [2023-12-27 05:46:14,500][20381] Updated weights for policy 1, policy_version 14938 (0.0007) [2023-12-27 05:46:15,022][20347] Updated weights for policy 0, policy_version 15018 (0.0011) [2023-12-27 05:46:15,082][20347] Updated weights for policy 0, policy_version 15028 (0.0010) [2023-12-27 05:46:15,145][20347] Updated weights for policy 0, policy_version 15038 (0.0010) [2023-12-27 05:46:15,221][20381] Updated weights for policy 1, policy_version 14948 (0.0007) [2023-12-27 05:46:15,280][20381] Updated weights for policy 1, policy_version 14958 (0.0008) [2023-12-27 05:46:15,346][20381] Updated weights for policy 1, policy_version 14968 (0.0008) [2023-12-27 05:46:15,829][20347] Updated weights for policy 0, policy_version 15048 (0.0010) [2023-12-27 05:46:15,890][20347] Updated weights for policy 0, policy_version 15058 (0.0010) [2023-12-27 05:46:15,943][20347] Updated weights for policy 0, policy_version 15068 (0.0010) [2023-12-27 05:46:16,030][20381] Updated weights for policy 1, policy_version 14978 (0.0007) [2023-12-27 05:46:16,080][20381] Updated weights for policy 1, policy_version 14988 (0.0008) [2023-12-27 05:46:16,129][20381] Updated weights for policy 1, policy_version 14998 (0.0007) [2023-12-27 05:46:16,176][20381] Updated weights for policy 1, policy_version 15008 (0.0007) [2023-12-27 05:46:16,620][20347] Updated weights for policy 0, policy_version 15078 (0.0010) [2023-12-27 05:46:16,665][20347] Updated weights for policy 0, policy_version 15088 (0.0010) [2023-12-27 05:46:16,716][20347] Updated weights for policy 0, policy_version 15098 (0.0010) [2023-12-27 05:46:16,863][20381] Updated weights for policy 1, policy_version 15018 (0.0011) [2023-12-27 05:46:16,918][20381] Updated weights for policy 1, policy_version 15028 (0.0010) [2023-12-27 05:46:16,974][20381] Updated weights for policy 1, policy_version 15038 (0.0009) [2023-12-27 05:46:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 7716864. Throughput: 0: 10167.3, 1: 10083.9. Samples: 7729832. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 05:46:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:17,406][20347] Updated weights for policy 0, policy_version 15108 (0.0010) [2023-12-27 05:46:17,466][20347] Updated weights for policy 0, policy_version 15118 (0.0007) [2023-12-27 05:46:17,527][20347] Updated weights for policy 0, policy_version 15128 (0.0005) [2023-12-27 05:46:17,574][20381] Updated weights for policy 1, policy_version 15048 (0.0005) [2023-12-27 05:46:17,622][20381] Updated weights for policy 1, policy_version 15058 (0.0007) [2023-12-27 05:46:17,667][20381] Updated weights for policy 1, policy_version 15068 (0.0010) [2023-12-27 05:46:18,075][20347] Updated weights for policy 0, policy_version 15138 (0.0006) [2023-12-27 05:46:18,126][20347] Updated weights for policy 0, policy_version 15148 (0.0008) [2023-12-27 05:46:18,181][20347] Updated weights for policy 0, policy_version 15158 (0.0007) [2023-12-27 05:46:18,235][20347] Updated weights for policy 0, policy_version 15168 (0.0010) [2023-12-27 05:46:18,359][20381] Updated weights for policy 1, policy_version 15078 (0.0010) [2023-12-27 05:46:18,421][20381] Updated weights for policy 1, policy_version 15088 (0.0011) [2023-12-27 05:46:18,483][20381] Updated weights for policy 1, policy_version 15098 (0.0011) [2023-12-27 05:46:18,909][20347] Updated weights for policy 0, policy_version 15178 (0.0007) [2023-12-27 05:46:18,959][20347] Updated weights for policy 0, policy_version 15188 (0.0007) [2023-12-27 05:46:19,006][20347] Updated weights for policy 0, policy_version 15198 (0.0007) [2023-12-27 05:46:19,146][20381] Updated weights for policy 1, policy_version 15108 (0.0010) [2023-12-27 05:46:19,201][20381] Updated weights for policy 1, policy_version 15118 (0.0010) [2023-12-27 05:46:19,263][20381] Updated weights for policy 1, policy_version 15128 (0.0009) [2023-12-27 05:46:19,687][20347] Updated weights for policy 0, policy_version 15208 (0.0007) [2023-12-27 05:46:19,740][20347] Updated weights for policy 0, policy_version 15218 (0.0007) [2023-12-27 05:46:19,798][20347] Updated weights for policy 0, policy_version 15228 (0.0008) [2023-12-27 05:46:20,026][20381] Updated weights for policy 1, policy_version 15138 (0.0008) [2023-12-27 05:46:20,085][20381] Updated weights for policy 1, policy_version 15148 (0.0008) [2023-12-27 05:46:20,143][20381] Updated weights for policy 1, policy_version 15158 (0.0008) [2023-12-27 05:46:20,204][20381] Updated weights for policy 1, policy_version 15168 (0.0009) [2023-12-27 05:46:20,579][20347] Updated weights for policy 0, policy_version 15238 (0.0008) [2023-12-27 05:46:20,641][20347] Updated weights for policy 0, policy_version 15248 (0.0007) [2023-12-27 05:46:20,704][20347] Updated weights for policy 0, policy_version 15258 (0.0007) [2023-12-27 05:46:20,940][20381] Updated weights for policy 1, policy_version 15178 (0.0007) [2023-12-27 05:46:21,011][20381] Updated weights for policy 1, policy_version 15188 (0.0005) [2023-12-27 05:46:21,075][20381] Updated weights for policy 1, policy_version 15198 (0.0008) [2023-12-27 05:46:21,389][20347] Updated weights for policy 0, policy_version 15268 (0.0008) [2023-12-27 05:46:21,446][20347] Updated weights for policy 0, policy_version 15278 (0.0007) [2023-12-27 05:46:21,510][20347] Updated weights for policy 0, policy_version 15288 (0.0006) [2023-12-27 05:46:21,737][20381] Updated weights for policy 1, policy_version 15208 (0.0007) [2023-12-27 05:46:21,792][20381] Updated weights for policy 1, policy_version 15218 (0.0007) [2023-12-27 05:46:21,848][20381] Updated weights for policy 1, policy_version 15228 (0.0009) [2023-12-27 05:46:22,190][20347] Updated weights for policy 0, policy_version 15298 (0.0007) [2023-12-27 05:46:22,247][20347] Updated weights for policy 0, policy_version 15308 (0.0008) [2023-12-27 05:46:22,298][20347] Updated weights for policy 0, policy_version 15318 (0.0008) [2023-12-27 05:46:22,355][20347] Updated weights for policy 0, policy_version 15328 (0.0008) [2023-12-27 05:46:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 7823360. Throughput: 0: 10150.6, 1: 10125.6. Samples: 7792220. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 05:46:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000015328_3923968.pth... [2023-12-27 05:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000015232_3899392.pth... [2023-12-27 05:46:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000014144_3620864.pth [2023-12-27 05:46:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000014048_3596288.pth [2023-12-27 05:46:22,586][20381] Updated weights for policy 1, policy_version 15238 (0.0008) [2023-12-27 05:46:22,643][20381] Updated weights for policy 1, policy_version 15248 (0.0008) [2023-12-27 05:46:22,701][20381] Updated weights for policy 1, policy_version 15258 (0.0007) [2023-12-27 05:46:23,084][20347] Updated weights for policy 0, policy_version 15338 (0.0008) [2023-12-27 05:46:23,134][20347] Updated weights for policy 0, policy_version 15348 (0.0008) [2023-12-27 05:46:23,182][20347] Updated weights for policy 0, policy_version 15358 (0.0007) [2023-12-27 05:46:23,366][20381] Updated weights for policy 1, policy_version 15268 (0.0009) [2023-12-27 05:46:23,414][20381] Updated weights for policy 1, policy_version 15278 (0.0008) [2023-12-27 05:46:23,463][20381] Updated weights for policy 1, policy_version 15288 (0.0007) [2023-12-27 05:46:23,855][20347] Updated weights for policy 0, policy_version 15368 (0.0007) [2023-12-27 05:46:23,918][20347] Updated weights for policy 0, policy_version 15378 (0.0007) [2023-12-27 05:46:23,977][20347] Updated weights for policy 0, policy_version 15388 (0.0008) [2023-12-27 05:46:24,150][20381] Updated weights for policy 1, policy_version 15298 (0.0007) [2023-12-27 05:46:24,208][20381] Updated weights for policy 1, policy_version 15308 (0.0007) [2023-12-27 05:46:24,256][20381] Updated weights for policy 1, policy_version 15318 (0.0007) [2023-12-27 05:46:24,307][20381] Updated weights for policy 1, policy_version 15328 (0.0008) [2023-12-27 05:46:24,679][20347] Updated weights for policy 0, policy_version 15398 (0.0006) [2023-12-27 05:46:24,729][20347] Updated weights for policy 0, policy_version 15408 (0.0005) [2023-12-27 05:46:24,786][20347] Updated weights for policy 0, policy_version 15418 (0.0007) [2023-12-27 05:46:25,031][20381] Updated weights for policy 1, policy_version 15338 (0.0008) [2023-12-27 05:46:25,091][20381] Updated weights for policy 1, policy_version 15348 (0.0009) [2023-12-27 05:46:25,146][20381] Updated weights for policy 1, policy_version 15358 (0.0010) [2023-12-27 05:46:25,440][20347] Updated weights for policy 0, policy_version 15428 (0.0008) [2023-12-27 05:46:25,492][20347] Updated weights for policy 0, policy_version 15438 (0.0010) [2023-12-27 05:46:25,539][20347] Updated weights for policy 0, policy_version 15448 (0.0010) [2023-12-27 05:46:25,820][20381] Updated weights for policy 1, policy_version 15368 (0.0010) [2023-12-27 05:46:25,877][20381] Updated weights for policy 1, policy_version 15378 (0.0006) [2023-12-27 05:46:25,927][20381] Updated weights for policy 1, policy_version 15388 (0.0005) [2023-12-27 05:46:26,228][20347] Updated weights for policy 0, policy_version 15458 (0.0010) [2023-12-27 05:46:26,284][20347] Updated weights for policy 0, policy_version 15468 (0.0010) [2023-12-27 05:46:26,341][20347] Updated weights for policy 0, policy_version 15478 (0.0010) [2023-12-27 05:46:26,395][20347] Updated weights for policy 0, policy_version 15488 (0.0010) [2023-12-27 05:46:26,574][20381] Updated weights for policy 1, policy_version 15398 (0.0008) [2023-12-27 05:46:26,627][20381] Updated weights for policy 1, policy_version 15408 (0.0010) [2023-12-27 05:46:26,678][20381] Updated weights for policy 1, policy_version 15418 (0.0009) [2023-12-27 05:46:27,060][20347] Updated weights for policy 0, policy_version 15498 (0.0010) [2023-12-27 05:46:27,114][20347] Updated weights for policy 0, policy_version 15508 (0.0010) [2023-12-27 05:46:27,168][20347] Updated weights for policy 0, policy_version 15518 (0.0010) [2023-12-27 05:46:27,312][20381] Updated weights for policy 1, policy_version 15428 (0.0007) [2023-12-27 05:46:27,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20660.5). Total num frames: 7921664. Throughput: 0: 10130.2, 1: 10129.2. Samples: 7912644. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 05:46:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:27,374][20381] Updated weights for policy 1, policy_version 15438 (0.0007) [2023-12-27 05:46:27,418][20381] Updated weights for policy 1, policy_version 15448 (0.0008) [2023-12-27 05:46:27,820][20347] Updated weights for policy 0, policy_version 15528 (0.0010) [2023-12-27 05:46:27,868][20347] Updated weights for policy 0, policy_version 15538 (0.0010) [2023-12-27 05:46:27,915][20347] Updated weights for policy 0, policy_version 15548 (0.0010) [2023-12-27 05:46:28,066][20381] Updated weights for policy 1, policy_version 15458 (0.0007) [2023-12-27 05:46:28,116][20381] Updated weights for policy 1, policy_version 15468 (0.0005) [2023-12-27 05:46:28,166][20381] Updated weights for policy 1, policy_version 15478 (0.0006) [2023-12-27 05:46:28,222][20381] Updated weights for policy 1, policy_version 15488 (0.0007) [2023-12-27 05:46:28,561][20347] Updated weights for policy 0, policy_version 15558 (0.0007) [2023-12-27 05:46:28,620][20347] Updated weights for policy 0, policy_version 15568 (0.0005) [2023-12-27 05:46:28,677][20347] Updated weights for policy 0, policy_version 15578 (0.0005) [2023-12-27 05:46:28,870][20381] Updated weights for policy 1, policy_version 15498 (0.0005) [2023-12-27 05:46:28,934][20381] Updated weights for policy 1, policy_version 15508 (0.0007) [2023-12-27 05:46:28,995][20381] Updated weights for policy 1, policy_version 15518 (0.0008) [2023-12-27 05:46:29,300][20347] Updated weights for policy 0, policy_version 15588 (0.0008) [2023-12-27 05:46:29,367][20347] Updated weights for policy 0, policy_version 15598 (0.0008) [2023-12-27 05:46:29,422][20347] Updated weights for policy 0, policy_version 15608 (0.0008) [2023-12-27 05:46:29,568][20381] Updated weights for policy 1, policy_version 15528 (0.0006) [2023-12-27 05:46:29,619][20381] Updated weights for policy 1, policy_version 15538 (0.0007) [2023-12-27 05:46:29,674][20381] Updated weights for policy 1, policy_version 15548 (0.0007) [2023-12-27 05:46:30,119][20347] Updated weights for policy 0, policy_version 15618 (0.0008) [2023-12-27 05:46:30,182][20347] Updated weights for policy 0, policy_version 15628 (0.0008) [2023-12-27 05:46:30,232][20347] Updated weights for policy 0, policy_version 15638 (0.0008) [2023-12-27 05:46:30,292][20347] Updated weights for policy 0, policy_version 15648 (0.0007) [2023-12-27 05:46:30,306][20381] Updated weights for policy 1, policy_version 15558 (0.0007) [2023-12-27 05:46:30,359][20381] Updated weights for policy 1, policy_version 15568 (0.0007) [2023-12-27 05:46:30,418][20381] Updated weights for policy 1, policy_version 15578 (0.0005) [2023-12-27 05:46:31,003][20347] Updated weights for policy 0, policy_version 15658 (0.0006) [2023-12-27 05:46:31,040][20381] Updated weights for policy 1, policy_version 15588 (0.0007) [2023-12-27 05:46:31,064][20347] Updated weights for policy 0, policy_version 15668 (0.0006) [2023-12-27 05:46:31,104][20381] Updated weights for policy 1, policy_version 15598 (0.0006) [2023-12-27 05:46:31,115][20347] Updated weights for policy 0, policy_version 15678 (0.0006) [2023-12-27 05:46:31,167][20381] Updated weights for policy 1, policy_version 15608 (0.0007) [2023-12-27 05:46:31,759][20347] Updated weights for policy 0, policy_version 15688 (0.0006) [2023-12-27 05:46:31,810][20347] Updated weights for policy 0, policy_version 15698 (0.0005) [2023-12-27 05:46:31,857][20381] Updated weights for policy 1, policy_version 15618 (0.0008) [2023-12-27 05:46:31,869][20347] Updated weights for policy 0, policy_version 15708 (0.0006) [2023-12-27 05:46:31,919][20381] Updated weights for policy 1, policy_version 15628 (0.0007) [2023-12-27 05:46:31,968][20381] Updated weights for policy 1, policy_version 15638 (0.0007) [2023-12-27 05:46:32,015][20381] Updated weights for policy 1, policy_version 15648 (0.0008) [2023-12-27 05:46:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20660.5). Total num frames: 8028160. Throughput: 0: 10175.6, 1: 10253.4. Samples: 8039160. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 05:46:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:32,513][20347] Updated weights for policy 0, policy_version 15718 (0.0008) [2023-12-27 05:46:32,567][20347] Updated weights for policy 0, policy_version 15728 (0.0010) [2023-12-27 05:46:32,628][20347] Updated weights for policy 0, policy_version 15738 (0.0010) [2023-12-27 05:46:32,712][20381] Updated weights for policy 1, policy_version 15658 (0.0010) [2023-12-27 05:46:32,767][20381] Updated weights for policy 1, policy_version 15668 (0.0010) [2023-12-27 05:46:32,821][20381] Updated weights for policy 1, policy_version 15678 (0.0010) [2023-12-27 05:46:33,249][20347] Updated weights for policy 0, policy_version 15748 (0.0008) [2023-12-27 05:46:33,294][20347] Updated weights for policy 0, policy_version 15758 (0.0005) [2023-12-27 05:46:33,339][20347] Updated weights for policy 0, policy_version 15768 (0.0005) [2023-12-27 05:46:33,422][20381] Updated weights for policy 1, policy_version 15688 (0.0006) [2023-12-27 05:46:33,480][20381] Updated weights for policy 1, policy_version 15698 (0.0005) [2023-12-27 05:46:33,537][20381] Updated weights for policy 1, policy_version 15708 (0.0005) [2023-12-27 05:46:33,988][20347] Updated weights for policy 0, policy_version 15778 (0.0010) [2023-12-27 05:46:34,034][20347] Updated weights for policy 0, policy_version 15788 (0.0007) [2023-12-27 05:46:34,085][20347] Updated weights for policy 0, policy_version 15798 (0.0006) [2023-12-27 05:46:34,094][20381] Updated weights for policy 1, policy_version 15718 (0.0008) [2023-12-27 05:46:34,136][20347] Updated weights for policy 0, policy_version 15808 (0.0006) [2023-12-27 05:46:34,143][20381] Updated weights for policy 1, policy_version 15728 (0.0010) [2023-12-27 05:46:34,207][20381] Updated weights for policy 1, policy_version 15738 (0.0007) [2023-12-27 05:46:34,876][20381] Updated weights for policy 1, policy_version 15748 (0.0009) [2023-12-27 05:46:34,888][20347] Updated weights for policy 0, policy_version 15818 (0.0006) [2023-12-27 05:46:34,936][20381] Updated weights for policy 1, policy_version 15758 (0.0010) [2023-12-27 05:46:34,947][20347] Updated weights for policy 0, policy_version 15828 (0.0006) [2023-12-27 05:46:35,002][20381] Updated weights for policy 1, policy_version 15768 (0.0008) [2023-12-27 05:46:35,009][20347] Updated weights for policy 0, policy_version 15838 (0.0005) [2023-12-27 05:46:35,631][20347] Updated weights for policy 0, policy_version 15848 (0.0006) [2023-12-27 05:46:35,677][20381] Updated weights for policy 1, policy_version 15778 (0.0009) [2023-12-27 05:46:35,686][20347] Updated weights for policy 0, policy_version 15858 (0.0007) [2023-12-27 05:46:35,734][20381] Updated weights for policy 1, policy_version 15788 (0.0011) [2023-12-27 05:46:35,743][20347] Updated weights for policy 0, policy_version 15868 (0.0006) [2023-12-27 05:46:35,783][20381] Updated weights for policy 1, policy_version 15798 (0.0011) [2023-12-27 05:46:35,838][20381] Updated weights for policy 1, policy_version 15808 (0.0011) [2023-12-27 05:46:36,383][20347] Updated weights for policy 0, policy_version 15878 (0.0008) [2023-12-27 05:46:36,446][20347] Updated weights for policy 0, policy_version 15888 (0.0010) [2023-12-27 05:46:36,508][20347] Updated weights for policy 0, policy_version 15898 (0.0009) [2023-12-27 05:46:36,539][20381] Updated weights for policy 1, policy_version 15818 (0.0011) [2023-12-27 05:46:36,589][20381] Updated weights for policy 1, policy_version 15828 (0.0009) [2023-12-27 05:46:36,645][20381] Updated weights for policy 1, policy_version 15838 (0.0008) [2023-12-27 05:46:37,160][20347] Updated weights for policy 0, policy_version 15908 (0.0010) [2023-12-27 05:46:37,216][20347] Updated weights for policy 0, policy_version 15918 (0.0010) [2023-12-27 05:46:37,217][20381] Updated weights for policy 1, policy_version 15848 (0.0010) [2023-12-27 05:46:37,262][20347] Updated weights for policy 0, policy_version 15928 (0.0010) [2023-12-27 05:46:37,273][20381] Updated weights for policy 1, policy_version 15858 (0.0011) [2023-12-27 05:46:37,333][20381] Updated weights for policy 1, policy_version 15868 (0.0011) [2023-12-27 05:46:37,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 8142848. Throughput: 0: 10195.6, 1: 10319.8. Samples: 8102948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:46:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000015872_4063232.pth... [2023-12-27 05:46:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000015936_4079616.pth... [2023-12-27 05:46:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000014656_3751936.pth [2023-12-27 05:46:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000014720_3768320.pth [2023-12-27 05:46:37,949][20381] Updated weights for policy 1, policy_version 15878 (0.0008) [2023-12-27 05:46:37,962][20347] Updated weights for policy 0, policy_version 15938 (0.0010) [2023-12-27 05:46:38,003][20381] Updated weights for policy 1, policy_version 15888 (0.0011) [2023-12-27 05:46:38,013][20347] Updated weights for policy 0, policy_version 15948 (0.0010) [2023-12-27 05:46:38,056][20381] Updated weights for policy 1, policy_version 15898 (0.0011) [2023-12-27 05:46:38,067][20347] Updated weights for policy 0, policy_version 15958 (0.0010) [2023-12-27 05:46:38,116][20347] Updated weights for policy 0, policy_version 15968 (0.0010) [2023-12-27 05:46:38,755][20381] Updated weights for policy 1, policy_version 15908 (0.0011) [2023-12-27 05:46:38,817][20381] Updated weights for policy 1, policy_version 15918 (0.0011) [2023-12-27 05:46:38,835][20347] Updated weights for policy 0, policy_version 15978 (0.0008) [2023-12-27 05:46:38,874][20381] Updated weights for policy 1, policy_version 15928 (0.0011) [2023-12-27 05:46:38,891][20347] Updated weights for policy 0, policy_version 15988 (0.0007) [2023-12-27 05:46:38,943][20347] Updated weights for policy 0, policy_version 15998 (0.0007) [2023-12-27 05:46:39,617][20381] Updated weights for policy 1, policy_version 15938 (0.0010) [2023-12-27 05:46:39,657][20347] Updated weights for policy 0, policy_version 16008 (0.0010) [2023-12-27 05:46:39,679][20381] Updated weights for policy 1, policy_version 15948 (0.0006) [2023-12-27 05:46:39,717][20347] Updated weights for policy 0, policy_version 16018 (0.0010) [2023-12-27 05:46:39,734][20381] Updated weights for policy 1, policy_version 15958 (0.0006) [2023-12-27 05:46:39,783][20347] Updated weights for policy 0, policy_version 16028 (0.0010) [2023-12-27 05:46:39,793][20381] Updated weights for policy 1, policy_version 15968 (0.0006) [2023-12-27 05:46:40,443][20347] Updated weights for policy 0, policy_version 16038 (0.0010) [2023-12-27 05:46:40,473][20381] Updated weights for policy 1, policy_version 15978 (0.0006) [2023-12-27 05:46:40,503][20347] Updated weights for policy 0, policy_version 16048 (0.0010) [2023-12-27 05:46:40,524][20381] Updated weights for policy 1, policy_version 15988 (0.0005) [2023-12-27 05:46:40,559][20347] Updated weights for policy 0, policy_version 16058 (0.0010) [2023-12-27 05:46:40,580][20381] Updated weights for policy 1, policy_version 15998 (0.0006) [2023-12-27 05:46:41,202][20347] Updated weights for policy 0, policy_version 16068 (0.0010) [2023-12-27 05:46:41,263][20347] Updated weights for policy 0, policy_version 16078 (0.0008) [2023-12-27 05:46:41,316][20347] Updated weights for policy 0, policy_version 16088 (0.0008) [2023-12-27 05:46:41,318][20381] Updated weights for policy 1, policy_version 16008 (0.0006) [2023-12-27 05:46:41,382][20381] Updated weights for policy 1, policy_version 16018 (0.0007) [2023-12-27 05:46:41,441][20381] Updated weights for policy 1, policy_version 16028 (0.0008) [2023-12-27 05:46:42,055][20347] Updated weights for policy 0, policy_version 16098 (0.0010) [2023-12-27 05:46:42,074][20381] Updated weights for policy 1, policy_version 16038 (0.0007) [2023-12-27 05:46:42,113][20347] Updated weights for policy 0, policy_version 16108 (0.0008) [2023-12-27 05:46:42,131][20381] Updated weights for policy 1, policy_version 16048 (0.0006) [2023-12-27 05:46:42,171][20347] Updated weights for policy 0, policy_version 16118 (0.0005) [2023-12-27 05:46:42,188][20381] Updated weights for policy 1, policy_version 16058 (0.0006) [2023-12-27 05:46:42,228][20347] Updated weights for policy 0, policy_version 16128 (0.0006) [2023-12-27 05:46:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 8241152. Throughput: 0: 10227.5, 1: 10353.2. Samples: 8226964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:46:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:42,876][20381] Updated weights for policy 1, policy_version 16068 (0.0006) [2023-12-27 05:46:42,878][20347] Updated weights for policy 0, policy_version 16138 (0.0008) [2023-12-27 05:46:42,928][20381] Updated weights for policy 1, policy_version 16078 (0.0005) [2023-12-27 05:46:42,933][20347] Updated weights for policy 0, policy_version 16148 (0.0007) [2023-12-27 05:46:42,976][20381] Updated weights for policy 1, policy_version 16088 (0.0005) [2023-12-27 05:46:42,989][20347] Updated weights for policy 0, policy_version 16158 (0.0006) [2023-12-27 05:46:43,593][20347] Updated weights for policy 0, policy_version 16168 (0.0006) [2023-12-27 05:46:43,652][20347] Updated weights for policy 0, policy_version 16178 (0.0007) [2023-12-27 05:46:43,706][20347] Updated weights for policy 0, policy_version 16188 (0.0007) [2023-12-27 05:46:43,711][20381] Updated weights for policy 1, policy_version 16098 (0.0007) [2023-12-27 05:46:43,772][20381] Updated weights for policy 1, policy_version 16108 (0.0008) [2023-12-27 05:46:43,826][20381] Updated weights for policy 1, policy_version 16118 (0.0008) [2023-12-27 05:46:43,878][20381] Updated weights for policy 1, policy_version 16128 (0.0008) [2023-12-27 05:46:44,338][20347] Updated weights for policy 0, policy_version 16198 (0.0007) [2023-12-27 05:46:44,401][20347] Updated weights for policy 0, policy_version 16208 (0.0005) [2023-12-27 05:46:44,457][20347] Updated weights for policy 0, policy_version 16218 (0.0005) [2023-12-27 05:46:44,474][20381] Updated weights for policy 1, policy_version 16138 (0.0008) [2023-12-27 05:46:44,530][20381] Updated weights for policy 1, policy_version 16148 (0.0008) [2023-12-27 05:46:44,582][20381] Updated weights for policy 1, policy_version 16158 (0.0006) [2023-12-27 05:46:44,999][20347] Updated weights for policy 0, policy_version 16228 (0.0005) [2023-12-27 05:46:45,050][20347] Updated weights for policy 0, policy_version 16238 (0.0005) [2023-12-27 05:46:45,100][20347] Updated weights for policy 0, policy_version 16248 (0.0007) [2023-12-27 05:46:45,340][20381] Updated weights for policy 1, policy_version 16168 (0.0007) [2023-12-27 05:46:45,397][20381] Updated weights for policy 1, policy_version 16178 (0.0005) [2023-12-27 05:46:45,447][20381] Updated weights for policy 1, policy_version 16188 (0.0007) [2023-12-27 05:46:45,691][20347] Updated weights for policy 0, policy_version 16258 (0.0006) [2023-12-27 05:46:45,738][20347] Updated weights for policy 0, policy_version 16268 (0.0007) [2023-12-27 05:46:45,802][20347] Updated weights for policy 0, policy_version 16278 (0.0006) [2023-12-27 05:46:45,850][20347] Updated weights for policy 0, policy_version 16288 (0.0005) [2023-12-27 05:46:46,069][20381] Updated weights for policy 1, policy_version 16198 (0.0008) [2023-12-27 05:46:46,128][20381] Updated weights for policy 1, policy_version 16208 (0.0008) [2023-12-27 05:46:46,184][20381] Updated weights for policy 1, policy_version 16218 (0.0008) [2023-12-27 05:46:46,500][20347] Updated weights for policy 0, policy_version 16298 (0.0008) [2023-12-27 05:46:46,558][20347] Updated weights for policy 0, policy_version 16308 (0.0007) [2023-12-27 05:46:46,625][20347] Updated weights for policy 0, policy_version 16318 (0.0007) [2023-12-27 05:46:46,806][20381] Updated weights for policy 1, policy_version 16228 (0.0007) [2023-12-27 05:46:46,865][20381] Updated weights for policy 1, policy_version 16238 (0.0005) [2023-12-27 05:46:46,922][20381] Updated weights for policy 1, policy_version 16248 (0.0007) [2023-12-27 05:46:47,238][20347] Updated weights for policy 0, policy_version 16328 (0.0009) [2023-12-27 05:46:47,282][20347] Updated weights for policy 0, policy_version 16338 (0.0010) [2023-12-27 05:46:47,339][20347] Updated weights for policy 0, policy_version 16348 (0.0010) [2023-12-27 05:46:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 8347648. Throughput: 0: 10312.0, 1: 10346.9. Samples: 8354644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:46:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:47,568][20381] Updated weights for policy 1, policy_version 16258 (0.0010) [2023-12-27 05:46:47,624][20381] Updated weights for policy 1, policy_version 16268 (0.0011) [2023-12-27 05:46:47,682][20381] Updated weights for policy 1, policy_version 16278 (0.0010) [2023-12-27 05:46:47,740][20381] Updated weights for policy 1, policy_version 16288 (0.0010) [2023-12-27 05:46:47,950][20347] Updated weights for policy 0, policy_version 16358 (0.0010) [2023-12-27 05:46:47,996][20347] Updated weights for policy 0, policy_version 16368 (0.0010) [2023-12-27 05:46:48,045][20347] Updated weights for policy 0, policy_version 16378 (0.0010) [2023-12-27 05:46:48,426][20381] Updated weights for policy 1, policy_version 16298 (0.0011) [2023-12-27 05:46:48,477][20381] Updated weights for policy 1, policy_version 16308 (0.0011) [2023-12-27 05:46:48,522][20381] Updated weights for policy 1, policy_version 16318 (0.0010) [2023-12-27 05:46:48,698][20347] Updated weights for policy 0, policy_version 16388 (0.0010) [2023-12-27 05:46:48,750][20347] Updated weights for policy 0, policy_version 16398 (0.0010) [2023-12-27 05:46:48,796][20347] Updated weights for policy 0, policy_version 16408 (0.0010) [2023-12-27 05:46:49,241][20381] Updated weights for policy 1, policy_version 16328 (0.0009) [2023-12-27 05:46:49,300][20381] Updated weights for policy 1, policy_version 16338 (0.0007) [2023-12-27 05:46:49,364][20381] Updated weights for policy 1, policy_version 16348 (0.0009) [2023-12-27 05:46:49,481][20347] Updated weights for policy 0, policy_version 16418 (0.0010) [2023-12-27 05:46:49,540][20347] Updated weights for policy 0, policy_version 16428 (0.0008) [2023-12-27 05:46:49,598][20347] Updated weights for policy 0, policy_version 16438 (0.0008) [2023-12-27 05:46:49,661][20347] Updated weights for policy 0, policy_version 16448 (0.0007) [2023-12-27 05:46:49,991][20381] Updated weights for policy 1, policy_version 16358 (0.0009) [2023-12-27 05:46:50,052][20381] Updated weights for policy 1, policy_version 16368 (0.0011) [2023-12-27 05:46:50,103][20381] Updated weights for policy 1, policy_version 16378 (0.0010) [2023-12-27 05:46:50,331][20347] Updated weights for policy 0, policy_version 16458 (0.0007) [2023-12-27 05:46:50,395][20347] Updated weights for policy 0, policy_version 16468 (0.0006) [2023-12-27 05:46:50,448][20347] Updated weights for policy 0, policy_version 16478 (0.0005) [2023-12-27 05:46:50,804][20381] Updated weights for policy 1, policy_version 16388 (0.0010) [2023-12-27 05:46:50,861][20381] Updated weights for policy 1, policy_version 16398 (0.0010) [2023-12-27 05:46:50,920][20381] Updated weights for policy 1, policy_version 16408 (0.0010) [2023-12-27 05:46:51,066][20347] Updated weights for policy 0, policy_version 16488 (0.0008) [2023-12-27 05:46:51,121][20347] Updated weights for policy 0, policy_version 16498 (0.0008) [2023-12-27 05:46:51,173][20347] Updated weights for policy 0, policy_version 16508 (0.0008) [2023-12-27 05:46:51,620][20381] Updated weights for policy 1, policy_version 16418 (0.0010) [2023-12-27 05:46:51,682][20381] Updated weights for policy 1, policy_version 16428 (0.0007) [2023-12-27 05:46:51,750][20381] Updated weights for policy 1, policy_version 16438 (0.0008) [2023-12-27 05:46:51,807][20381] Updated weights for policy 1, policy_version 16448 (0.0007) [2023-12-27 05:46:51,942][20347] Updated weights for policy 0, policy_version 16518 (0.0007) [2023-12-27 05:46:51,999][20347] Updated weights for policy 0, policy_version 16528 (0.0007) [2023-12-27 05:46:52,058][20347] Updated weights for policy 0, policy_version 16538 (0.0007) [2023-12-27 05:46:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 8445952. Throughput: 0: 10316.0, 1: 10334.2. Samples: 8416816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:46:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000016448_4210688.pth... [2023-12-27 05:46:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000016544_4235264.pth... [2023-12-27 05:46:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000015232_3899392.pth [2023-12-27 05:46:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000015328_3923968.pth [2023-12-27 05:46:52,505][20381] Updated weights for policy 1, policy_version 16458 (0.0006) [2023-12-27 05:46:52,564][20381] Updated weights for policy 1, policy_version 16468 (0.0006) [2023-12-27 05:46:52,622][20381] Updated weights for policy 1, policy_version 16478 (0.0006) [2023-12-27 05:46:52,770][20347] Updated weights for policy 0, policy_version 16548 (0.0008) [2023-12-27 05:46:52,832][20347] Updated weights for policy 0, policy_version 16558 (0.0008) [2023-12-27 05:46:52,890][20347] Updated weights for policy 0, policy_version 16568 (0.0008) [2023-12-27 05:46:53,236][20381] Updated weights for policy 1, policy_version 16488 (0.0007) [2023-12-27 05:46:53,293][20381] Updated weights for policy 1, policy_version 16498 (0.0008) [2023-12-27 05:46:53,347][20381] Updated weights for policy 1, policy_version 16508 (0.0008) [2023-12-27 05:46:53,530][20347] Updated weights for policy 0, policy_version 16578 (0.0007) [2023-12-27 05:46:53,581][20347] Updated weights for policy 0, policy_version 16588 (0.0005) [2023-12-27 05:46:53,631][20347] Updated weights for policy 0, policy_version 16598 (0.0006) [2023-12-27 05:46:53,683][20347] Updated weights for policy 0, policy_version 16608 (0.0007) [2023-12-27 05:46:54,072][20381] Updated weights for policy 1, policy_version 16518 (0.0008) [2023-12-27 05:46:54,132][20381] Updated weights for policy 1, policy_version 16528 (0.0008) [2023-12-27 05:46:54,181][20381] Updated weights for policy 1, policy_version 16538 (0.0007) [2023-12-27 05:46:54,345][20347] Updated weights for policy 0, policy_version 16618 (0.0005) [2023-12-27 05:46:54,399][20347] Updated weights for policy 0, policy_version 16628 (0.0005) [2023-12-27 05:46:54,459][20347] Updated weights for policy 0, policy_version 16638 (0.0006) [2023-12-27 05:46:54,823][20381] Updated weights for policy 1, policy_version 16548 (0.0007) [2023-12-27 05:46:54,879][20381] Updated weights for policy 1, policy_version 16558 (0.0006) [2023-12-27 05:46:54,944][20381] Updated weights for policy 1, policy_version 16568 (0.0008) [2023-12-27 05:46:55,071][20347] Updated weights for policy 0, policy_version 16648 (0.0008) [2023-12-27 05:46:55,132][20347] Updated weights for policy 0, policy_version 16658 (0.0008) [2023-12-27 05:46:55,192][20347] Updated weights for policy 0, policy_version 16668 (0.0008) [2023-12-27 05:46:55,526][20381] Updated weights for policy 1, policy_version 16578 (0.0008) [2023-12-27 05:46:55,592][20381] Updated weights for policy 1, policy_version 16588 (0.0006) [2023-12-27 05:46:55,655][20381] Updated weights for policy 1, policy_version 16598 (0.0006) [2023-12-27 05:46:55,708][20381] Updated weights for policy 1, policy_version 16608 (0.0005) [2023-12-27 05:46:55,903][20347] Updated weights for policy 0, policy_version 16678 (0.0007) [2023-12-27 05:46:55,962][20347] Updated weights for policy 0, policy_version 16688 (0.0005) [2023-12-27 05:46:56,014][20347] Updated weights for policy 0, policy_version 16698 (0.0008) [2023-12-27 05:46:56,276][20381] Updated weights for policy 1, policy_version 16618 (0.0007) [2023-12-27 05:46:56,333][20381] Updated weights for policy 1, policy_version 16628 (0.0008) [2023-12-27 05:46:56,393][20381] Updated weights for policy 1, policy_version 16638 (0.0005) [2023-12-27 05:46:56,651][20347] Updated weights for policy 0, policy_version 16708 (0.0008) [2023-12-27 05:46:56,698][20347] Updated weights for policy 0, policy_version 16718 (0.0010) [2023-12-27 05:46:56,746][20347] Updated weights for policy 0, policy_version 16728 (0.0010) [2023-12-27 05:46:56,944][20381] Updated weights for policy 1, policy_version 16648 (0.0007) [2023-12-27 05:46:56,988][20381] Updated weights for policy 1, policy_version 16658 (0.0009) [2023-12-27 05:46:57,037][20381] Updated weights for policy 1, policy_version 16668 (0.0008) [2023-12-27 05:46:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 8552448. Throughput: 0: 10377.0, 1: 10354.6. Samples: 8541764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:46:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:46:57,409][20347] Updated weights for policy 0, policy_version 16738 (0.0010) [2023-12-27 05:46:57,457][20347] Updated weights for policy 0, policy_version 16748 (0.0010) [2023-12-27 05:46:57,501][20347] Updated weights for policy 0, policy_version 16758 (0.0010) [2023-12-27 05:46:57,545][20347] Updated weights for policy 0, policy_version 16768 (0.0010) [2023-12-27 05:46:57,747][20381] Updated weights for policy 1, policy_version 16678 (0.0008) [2023-12-27 05:46:57,802][20381] Updated weights for policy 1, policy_version 16688 (0.0008) [2023-12-27 05:46:57,855][20381] Updated weights for policy 1, policy_version 16698 (0.0007) [2023-12-27 05:46:58,248][20347] Updated weights for policy 0, policy_version 16778 (0.0011) [2023-12-27 05:46:58,306][20347] Updated weights for policy 0, policy_version 16788 (0.0010) [2023-12-27 05:46:58,367][20347] Updated weights for policy 0, policy_version 16798 (0.0008) [2023-12-27 05:46:58,581][20381] Updated weights for policy 1, policy_version 16708 (0.0007) [2023-12-27 05:46:58,644][20381] Updated weights for policy 1, policy_version 16718 (0.0008) [2023-12-27 05:46:58,712][20381] Updated weights for policy 1, policy_version 16728 (0.0007) [2023-12-27 05:46:59,116][20347] Updated weights for policy 0, policy_version 16808 (0.0008) [2023-12-27 05:46:59,172][20347] Updated weights for policy 0, policy_version 16818 (0.0007) [2023-12-27 05:46:59,236][20347] Updated weights for policy 0, policy_version 16828 (0.0007) [2023-12-27 05:46:59,481][20381] Updated weights for policy 1, policy_version 16738 (0.0007) [2023-12-27 05:46:59,537][20381] Updated weights for policy 1, policy_version 16748 (0.0008) [2023-12-27 05:46:59,596][20381] Updated weights for policy 1, policy_version 16758 (0.0008) [2023-12-27 05:46:59,641][20381] Updated weights for policy 1, policy_version 16768 (0.0007) [2023-12-27 05:46:59,893][20347] Updated weights for policy 0, policy_version 16838 (0.0008) [2023-12-27 05:46:59,955][20347] Updated weights for policy 0, policy_version 16848 (0.0008) [2023-12-27 05:47:00,015][20347] Updated weights for policy 0, policy_version 16858 (0.0007) [2023-12-27 05:47:00,291][20381] Updated weights for policy 1, policy_version 16778 (0.0008) [2023-12-27 05:47:00,348][20381] Updated weights for policy 1, policy_version 16788 (0.0008) [2023-12-27 05:47:00,401][20381] Updated weights for policy 1, policy_version 16798 (0.0007) [2023-12-27 05:47:00,633][20347] Updated weights for policy 0, policy_version 16868 (0.0006) [2023-12-27 05:47:00,697][20347] Updated weights for policy 0, policy_version 16878 (0.0008) [2023-12-27 05:47:00,750][20347] Updated weights for policy 0, policy_version 16889 (0.0009) [2023-12-27 05:47:01,080][20381] Updated weights for policy 1, policy_version 16808 (0.0007) [2023-12-27 05:47:01,144][20381] Updated weights for policy 1, policy_version 16818 (0.0009) [2023-12-27 05:47:01,198][20381] Updated weights for policy 1, policy_version 16828 (0.0008) [2023-12-27 05:47:01,440][20347] Updated weights for policy 0, policy_version 16899 (0.0007) [2023-12-27 05:47:01,486][20347] Updated weights for policy 0, policy_version 16909 (0.0009) [2023-12-27 05:47:01,542][20347] Updated weights for policy 0, policy_version 16919 (0.0008) [2023-12-27 05:47:01,860][20381] Updated weights for policy 1, policy_version 16838 (0.0007) [2023-12-27 05:47:01,916][20381] Updated weights for policy 1, policy_version 16848 (0.0007) [2023-12-27 05:47:01,971][20381] Updated weights for policy 1, policy_version 16858 (0.0007) [2023-12-27 05:47:02,222][20347] Updated weights for policy 0, policy_version 16929 (0.0008) [2023-12-27 05:47:02,283][20347] Updated weights for policy 0, policy_version 16939 (0.0007) [2023-12-27 05:47:02,338][20347] Updated weights for policy 0, policy_version 16949 (0.0007) [2023-12-27 05:47:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 8650752. Throughput: 0: 10397.7, 1: 10368.1. Samples: 8664292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:47:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:02,398][20347] Updated weights for policy 0, policy_version 16959 (0.0009) [2023-12-27 05:47:02,652][20381] Updated weights for policy 1, policy_version 16868 (0.0008) [2023-12-27 05:47:02,707][20381] Updated weights for policy 1, policy_version 16878 (0.0008) [2023-12-27 05:47:02,755][20381] Updated weights for policy 1, policy_version 16888 (0.0007) [2023-12-27 05:47:03,031][20347] Updated weights for policy 0, policy_version 16969 (0.0008) [2023-12-27 05:47:03,079][20347] Updated weights for policy 0, policy_version 16979 (0.0007) [2023-12-27 05:47:03,131][20347] Updated weights for policy 0, policy_version 16989 (0.0007) [2023-12-27 05:47:03,455][20381] Updated weights for policy 1, policy_version 16898 (0.0008) [2023-12-27 05:47:03,503][20381] Updated weights for policy 1, policy_version 16908 (0.0008) [2023-12-27 05:47:03,547][20381] Updated weights for policy 1, policy_version 16918 (0.0008) [2023-12-27 05:47:03,595][20381] Updated weights for policy 1, policy_version 16928 (0.0009) [2023-12-27 05:47:03,763][20347] Updated weights for policy 0, policy_version 16999 (0.0006) [2023-12-27 05:47:03,813][20347] Updated weights for policy 0, policy_version 17009 (0.0005) [2023-12-27 05:47:03,871][20347] Updated weights for policy 0, policy_version 17019 (0.0007) [2023-12-27 05:47:04,302][20381] Updated weights for policy 1, policy_version 16938 (0.0008) [2023-12-27 05:47:04,361][20381] Updated weights for policy 1, policy_version 16948 (0.0008) [2023-12-27 05:47:04,419][20381] Updated weights for policy 1, policy_version 16958 (0.0007) [2023-12-27 05:47:04,459][20347] Updated weights for policy 0, policy_version 17029 (0.0007) [2023-12-27 05:47:04,511][20347] Updated weights for policy 0, policy_version 17039 (0.0007) [2023-12-27 05:47:04,566][20347] Updated weights for policy 0, policy_version 17049 (0.0008) [2023-12-27 05:47:05,099][20381] Updated weights for policy 1, policy_version 16968 (0.0005) [2023-12-27 05:47:05,154][20381] Updated weights for policy 1, policy_version 16978 (0.0005) [2023-12-27 05:47:05,217][20381] Updated weights for policy 1, policy_version 16988 (0.0010) [2023-12-27 05:47:05,306][20347] Updated weights for policy 0, policy_version 17059 (0.0008) [2023-12-27 05:47:05,358][20347] Updated weights for policy 0, policy_version 17069 (0.0008) [2023-12-27 05:47:05,417][20347] Updated weights for policy 0, policy_version 17079 (0.0007) [2023-12-27 05:47:05,861][20381] Updated weights for policy 1, policy_version 16998 (0.0010) [2023-12-27 05:47:05,909][20381] Updated weights for policy 1, policy_version 17008 (0.0010) [2023-12-27 05:47:05,964][20381] Updated weights for policy 1, policy_version 17018 (0.0010) [2023-12-27 05:47:06,119][20347] Updated weights for policy 0, policy_version 17089 (0.0007) [2023-12-27 05:47:06,181][20347] Updated weights for policy 0, policy_version 17099 (0.0008) [2023-12-27 05:47:06,234][20347] Updated weights for policy 0, policy_version 17109 (0.0007) [2023-12-27 05:47:06,287][20347] Updated weights for policy 0, policy_version 17119 (0.0006) [2023-12-27 05:47:06,620][20381] Updated weights for policy 1, policy_version 17028 (0.0010) [2023-12-27 05:47:06,672][20381] Updated weights for policy 1, policy_version 17038 (0.0007) [2023-12-27 05:47:06,727][20381] Updated weights for policy 1, policy_version 17048 (0.0009) [2023-12-27 05:47:06,923][20347] Updated weights for policy 0, policy_version 17129 (0.0007) [2023-12-27 05:47:06,975][20347] Updated weights for policy 0, policy_version 17139 (0.0008) [2023-12-27 05:47:07,030][20347] Updated weights for policy 0, policy_version 17149 (0.0007) [2023-12-27 05:47:07,333][20381] Updated weights for policy 1, policy_version 17058 (0.0007) [2023-12-27 05:47:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 8757248. Throughput: 0: 10411.0, 1: 10380.0. Samples: 8727816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:47:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000017152_4390912.pth... [2023-12-27 05:47:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000015936_4079616.pth [2023-12-27 05:47:07,386][20381] Updated weights for policy 1, policy_version 17068 (0.0011) [2023-12-27 05:47:07,442][20381] Updated weights for policy 1, policy_version 17078 (0.0008) [2023-12-27 05:47:07,498][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000017088_4374528.pth... [2023-12-27 05:47:07,499][20381] Updated weights for policy 1, policy_version 17088 (0.0007) [2023-12-27 05:47:07,503][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000015872_4063232.pth [2023-12-27 05:47:07,766][20347] Updated weights for policy 0, policy_version 17159 (0.0008) [2023-12-27 05:47:07,820][20347] Updated weights for policy 0, policy_version 17169 (0.0008) [2023-12-27 05:47:07,880][20347] Updated weights for policy 0, policy_version 17179 (0.0007) [2023-12-27 05:47:08,152][20381] Updated weights for policy 1, policy_version 17098 (0.0008) [2023-12-27 05:47:08,207][20381] Updated weights for policy 1, policy_version 17108 (0.0008) [2023-12-27 05:47:08,259][20381] Updated weights for policy 1, policy_version 17118 (0.0006) [2023-12-27 05:47:08,577][20347] Updated weights for policy 0, policy_version 17189 (0.0008) [2023-12-27 05:47:08,628][20347] Updated weights for policy 0, policy_version 17199 (0.0007) [2023-12-27 05:47:08,684][20347] Updated weights for policy 0, policy_version 17209 (0.0008) [2023-12-27 05:47:08,973][20381] Updated weights for policy 1, policy_version 17128 (0.0008) [2023-12-27 05:47:09,032][20381] Updated weights for policy 1, policy_version 17138 (0.0007) [2023-12-27 05:47:09,088][20381] Updated weights for policy 1, policy_version 17148 (0.0007) [2023-12-27 05:47:09,428][20347] Updated weights for policy 0, policy_version 17219 (0.0008) [2023-12-27 05:47:09,492][20347] Updated weights for policy 0, policy_version 17229 (0.0008) [2023-12-27 05:47:09,561][20347] Updated weights for policy 0, policy_version 17239 (0.0008) [2023-12-27 05:47:09,780][20381] Updated weights for policy 1, policy_version 17158 (0.0006) [2023-12-27 05:47:09,848][20381] Updated weights for policy 1, policy_version 17168 (0.0008) [2023-12-27 05:47:09,910][20381] Updated weights for policy 1, policy_version 17178 (0.0008) [2023-12-27 05:47:10,311][20347] Updated weights for policy 0, policy_version 17249 (0.0007) [2023-12-27 05:47:10,371][20347] Updated weights for policy 0, policy_version 17259 (0.0008) [2023-12-27 05:47:10,421][20347] Updated weights for policy 0, policy_version 17269 (0.0008) [2023-12-27 05:47:10,481][20347] Updated weights for policy 0, policy_version 17279 (0.0008) [2023-12-27 05:47:10,571][20381] Updated weights for policy 1, policy_version 17188 (0.0008) [2023-12-27 05:47:10,624][20381] Updated weights for policy 1, policy_version 17198 (0.0008) [2023-12-27 05:47:10,679][20381] Updated weights for policy 1, policy_version 17208 (0.0006) [2023-12-27 05:47:11,132][20347] Updated weights for policy 0, policy_version 17289 (0.0008) [2023-12-27 05:47:11,192][20347] Updated weights for policy 0, policy_version 17299 (0.0008) [2023-12-27 05:47:11,254][20347] Updated weights for policy 0, policy_version 17309 (0.0008) [2023-12-27 05:47:11,406][20381] Updated weights for policy 1, policy_version 17218 (0.0009) [2023-12-27 05:47:11,461][20381] Updated weights for policy 1, policy_version 17228 (0.0008) [2023-12-27 05:47:11,516][20381] Updated weights for policy 1, policy_version 17238 (0.0008) [2023-12-27 05:47:11,571][20381] Updated weights for policy 1, policy_version 17248 (0.0008) [2023-12-27 05:47:12,048][20347] Updated weights for policy 0, policy_version 17319 (0.0008) [2023-12-27 05:47:12,113][20347] Updated weights for policy 0, policy_version 17329 (0.0006) [2023-12-27 05:47:12,171][20347] Updated weights for policy 0, policy_version 17339 (0.0005) [2023-12-27 05:47:12,292][20381] Updated weights for policy 1, policy_version 17258 (0.0008) [2023-12-27 05:47:12,359][20381] Updated weights for policy 1, policy_version 17268 (0.0008) [2023-12-27 05:47:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 8855552. Throughput: 0: 10407.6, 1: 10407.7. Samples: 8849328. Policy #0 lag: (min: 26.0, avg: 35.3, max: 58.0) [2023-12-27 05:47:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:12,413][20381] Updated weights for policy 1, policy_version 17278 (0.0008) [2023-12-27 05:47:12,800][20347] Updated weights for policy 0, policy_version 17349 (0.0008) [2023-12-27 05:47:12,853][20347] Updated weights for policy 0, policy_version 17359 (0.0007) [2023-12-27 05:47:12,912][20347] Updated weights for policy 0, policy_version 17369 (0.0007) [2023-12-27 05:47:13,096][20381] Updated weights for policy 1, policy_version 17288 (0.0008) [2023-12-27 05:47:13,147][20381] Updated weights for policy 1, policy_version 17298 (0.0008) [2023-12-27 05:47:13,202][20381] Updated weights for policy 1, policy_version 17308 (0.0007) [2023-12-27 05:47:13,622][20347] Updated weights for policy 0, policy_version 17379 (0.0007) [2023-12-27 05:47:13,680][20347] Updated weights for policy 0, policy_version 17389 (0.0007) [2023-12-27 05:47:13,735][20347] Updated weights for policy 0, policy_version 17399 (0.0007) [2023-12-27 05:47:13,876][20381] Updated weights for policy 1, policy_version 17318 (0.0010) [2023-12-27 05:47:13,926][20381] Updated weights for policy 1, policy_version 17328 (0.0010) [2023-12-27 05:47:13,974][20381] Updated weights for policy 1, policy_version 17338 (0.0010) [2023-12-27 05:47:14,445][20347] Updated weights for policy 0, policy_version 17409 (0.0007) [2023-12-27 05:47:14,519][20347] Updated weights for policy 0, policy_version 17419 (0.0008) [2023-12-27 05:47:14,575][20347] Updated weights for policy 0, policy_version 17429 (0.0008) [2023-12-27 05:47:14,630][20347] Updated weights for policy 0, policy_version 17439 (0.0007) [2023-12-27 05:47:14,656][20381] Updated weights for policy 1, policy_version 17348 (0.0009) [2023-12-27 05:47:14,704][20381] Updated weights for policy 1, policy_version 17358 (0.0007) [2023-12-27 05:47:14,759][20381] Updated weights for policy 1, policy_version 17368 (0.0007) [2023-12-27 05:47:15,259][20347] Updated weights for policy 0, policy_version 17449 (0.0008) [2023-12-27 05:47:15,315][20347] Updated weights for policy 0, policy_version 17459 (0.0006) [2023-12-27 05:47:15,371][20347] Updated weights for policy 0, policy_version 17469 (0.0005) [2023-12-27 05:47:15,509][20381] Updated weights for policy 1, policy_version 17378 (0.0007) [2023-12-27 05:47:15,562][20381] Updated weights for policy 1, policy_version 17388 (0.0006) [2023-12-27 05:47:15,614][20381] Updated weights for policy 1, policy_version 17398 (0.0005) [2023-12-27 05:47:15,667][20381] Updated weights for policy 1, policy_version 17408 (0.0005) [2023-12-27 05:47:16,051][20347] Updated weights for policy 0, policy_version 17479 (0.0005) [2023-12-27 05:47:16,105][20347] Updated weights for policy 0, policy_version 17489 (0.0005) [2023-12-27 05:47:16,164][20347] Updated weights for policy 0, policy_version 17499 (0.0005) [2023-12-27 05:47:16,235][20381] Updated weights for policy 1, policy_version 17418 (0.0007) [2023-12-27 05:47:16,293][20381] Updated weights for policy 1, policy_version 17428 (0.0007) [2023-12-27 05:47:16,351][20381] Updated weights for policy 1, policy_version 17438 (0.0008) [2023-12-27 05:47:16,803][20347] Updated weights for policy 0, policy_version 17509 (0.0007) [2023-12-27 05:47:16,867][20347] Updated weights for policy 0, policy_version 17519 (0.0006) [2023-12-27 05:47:16,922][20347] Updated weights for policy 0, policy_version 17529 (0.0010) [2023-12-27 05:47:17,017][20381] Updated weights for policy 1, policy_version 17448 (0.0007) [2023-12-27 05:47:17,074][20381] Updated weights for policy 1, policy_version 17458 (0.0007) [2023-12-27 05:47:17,136][20381] Updated weights for policy 1, policy_version 17468 (0.0008) [2023-12-27 05:47:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 8962048. Throughput: 0: 10372.9, 1: 10346.0. Samples: 8971508. Policy #0 lag: (min: 26.0, avg: 35.3, max: 58.0) [2023-12-27 05:47:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:17,527][20347] Updated weights for policy 0, policy_version 17539 (0.0007) [2023-12-27 05:47:17,584][20347] Updated weights for policy 0, policy_version 17549 (0.0010) [2023-12-27 05:47:17,638][20347] Updated weights for policy 0, policy_version 17559 (0.0010) [2023-12-27 05:47:17,732][20381] Updated weights for policy 1, policy_version 17478 (0.0007) [2023-12-27 05:47:17,778][20381] Updated weights for policy 1, policy_version 17488 (0.0005) [2023-12-27 05:47:17,829][20381] Updated weights for policy 1, policy_version 17498 (0.0005) [2023-12-27 05:47:18,295][20347] Updated weights for policy 0, policy_version 17569 (0.0010) [2023-12-27 05:47:18,358][20347] Updated weights for policy 0, policy_version 17579 (0.0010) [2023-12-27 05:47:18,398][20381] Updated weights for policy 1, policy_version 17508 (0.0006) [2023-12-27 05:47:18,418][20347] Updated weights for policy 0, policy_version 17589 (0.0010) [2023-12-27 05:47:18,447][20381] Updated weights for policy 1, policy_version 17518 (0.0007) [2023-12-27 05:47:18,474][20347] Updated weights for policy 0, policy_version 17599 (0.0007) [2023-12-27 05:47:18,505][20381] Updated weights for policy 1, policy_version 17528 (0.0007) [2023-12-27 05:47:19,140][20347] Updated weights for policy 0, policy_version 17609 (0.0010) [2023-12-27 05:47:19,199][20347] Updated weights for policy 0, policy_version 17619 (0.0010) [2023-12-27 05:47:19,204][20381] Updated weights for policy 1, policy_version 17538 (0.0006) [2023-12-27 05:47:19,262][20347] Updated weights for policy 0, policy_version 17629 (0.0011) [2023-12-27 05:47:19,268][20381] Updated weights for policy 1, policy_version 17548 (0.0008) [2023-12-27 05:47:19,325][20381] Updated weights for policy 1, policy_version 17558 (0.0008) [2023-12-27 05:47:19,387][20381] Updated weights for policy 1, policy_version 17568 (0.0007) [2023-12-27 05:47:19,847][20347] Updated weights for policy 0, policy_version 17639 (0.0007) [2023-12-27 05:47:19,901][20347] Updated weights for policy 0, policy_version 17649 (0.0008) [2023-12-27 05:47:19,958][20347] Updated weights for policy 0, policy_version 17659 (0.0008) [2023-12-27 05:47:20,117][20381] Updated weights for policy 1, policy_version 17578 (0.0008) [2023-12-27 05:47:20,181][20381] Updated weights for policy 1, policy_version 17588 (0.0008) [2023-12-27 05:47:20,241][20381] Updated weights for policy 1, policy_version 17598 (0.0008) [2023-12-27 05:47:20,643][20347] Updated weights for policy 0, policy_version 17669 (0.0007) [2023-12-27 05:47:20,708][20347] Updated weights for policy 0, policy_version 17679 (0.0006) [2023-12-27 05:47:20,768][20347] Updated weights for policy 0, policy_version 17689 (0.0005) [2023-12-27 05:47:20,945][20381] Updated weights for policy 1, policy_version 17608 (0.0007) [2023-12-27 05:47:21,002][20381] Updated weights for policy 1, policy_version 17618 (0.0008) [2023-12-27 05:47:21,066][20381] Updated weights for policy 1, policy_version 17628 (0.0008) [2023-12-27 05:47:21,409][20347] Updated weights for policy 0, policy_version 17699 (0.0006) [2023-12-27 05:47:21,471][20347] Updated weights for policy 0, policy_version 17709 (0.0005) [2023-12-27 05:47:21,537][20347] Updated weights for policy 0, policy_version 17719 (0.0005) [2023-12-27 05:47:21,784][20381] Updated weights for policy 1, policy_version 17638 (0.0008) [2023-12-27 05:47:21,848][20381] Updated weights for policy 1, policy_version 17648 (0.0007) [2023-12-27 05:47:21,904][20381] Updated weights for policy 1, policy_version 17658 (0.0007) [2023-12-27 05:47:22,154][20347] Updated weights for policy 0, policy_version 17729 (0.0006) [2023-12-27 05:47:22,209][20347] Updated weights for policy 0, policy_version 17739 (0.0006) [2023-12-27 05:47:22,273][20347] Updated weights for policy 0, policy_version 17749 (0.0007) [2023-12-27 05:47:22,333][20347] Updated weights for policy 0, policy_version 17759 (0.0008) [2023-12-27 05:47:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 9068544. Throughput: 0: 10382.9, 1: 10331.4. Samples: 9035092. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-27 05:47:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000017664_4521984.pth... [2023-12-27 05:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000017760_4546560.pth... [2023-12-27 05:47:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000016448_4210688.pth [2023-12-27 05:47:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000016544_4235264.pth [2023-12-27 05:47:22,662][20381] Updated weights for policy 1, policy_version 17668 (0.0008) [2023-12-27 05:47:22,715][20381] Updated weights for policy 1, policy_version 17678 (0.0007) [2023-12-27 05:47:22,775][20381] Updated weights for policy 1, policy_version 17688 (0.0007) [2023-12-27 05:47:22,931][20347] Updated weights for policy 0, policy_version 17769 (0.0008) [2023-12-27 05:47:22,994][20347] Updated weights for policy 0, policy_version 17779 (0.0008) [2023-12-27 05:47:23,058][20347] Updated weights for policy 0, policy_version 17789 (0.0008) [2023-12-27 05:47:23,499][20381] Updated weights for policy 1, policy_version 17698 (0.0007) [2023-12-27 05:47:23,554][20381] Updated weights for policy 1, policy_version 17708 (0.0008) [2023-12-27 05:47:23,605][20381] Updated weights for policy 1, policy_version 17718 (0.0007) [2023-12-27 05:47:23,650][20381] Updated weights for policy 1, policy_version 17728 (0.0010) [2023-12-27 05:47:23,729][20347] Updated weights for policy 0, policy_version 17799 (0.0007) [2023-12-27 05:47:23,785][20347] Updated weights for policy 0, policy_version 17809 (0.0007) [2023-12-27 05:47:23,849][20347] Updated weights for policy 0, policy_version 17819 (0.0007) [2023-12-27 05:47:24,356][20381] Updated weights for policy 1, policy_version 17738 (0.0008) [2023-12-27 05:47:24,409][20381] Updated weights for policy 1, policy_version 17748 (0.0006) [2023-12-27 05:47:24,461][20381] Updated weights for policy 1, policy_version 17758 (0.0007) [2023-12-27 05:47:24,544][20347] Updated weights for policy 0, policy_version 17829 (0.0008) [2023-12-27 05:47:24,600][20347] Updated weights for policy 0, policy_version 17839 (0.0008) [2023-12-27 05:47:24,649][20347] Updated weights for policy 0, policy_version 17849 (0.0007) [2023-12-27 05:47:25,160][20381] Updated weights for policy 1, policy_version 17768 (0.0007) [2023-12-27 05:47:25,222][20381] Updated weights for policy 1, policy_version 17778 (0.0008) [2023-12-27 05:47:25,276][20381] Updated weights for policy 1, policy_version 17788 (0.0007) [2023-12-27 05:47:25,283][20347] Updated weights for policy 0, policy_version 17859 (0.0007) [2023-12-27 05:47:25,338][20347] Updated weights for policy 0, policy_version 17869 (0.0007) [2023-12-27 05:47:25,387][20347] Updated weights for policy 0, policy_version 17879 (0.0008) [2023-12-27 05:47:25,915][20381] Updated weights for policy 1, policy_version 17798 (0.0007) [2023-12-27 05:47:25,977][20381] Updated weights for policy 1, policy_version 17808 (0.0008) [2023-12-27 05:47:26,032][20381] Updated weights for policy 1, policy_version 17818 (0.0007) [2023-12-27 05:47:26,041][20347] Updated weights for policy 0, policy_version 17889 (0.0007) [2023-12-27 05:47:26,093][20347] Updated weights for policy 0, policy_version 17899 (0.0010) [2023-12-27 05:47:26,151][20347] Updated weights for policy 0, policy_version 17909 (0.0010) [2023-12-27 05:47:26,205][20347] Updated weights for policy 0, policy_version 17919 (0.0010) [2023-12-27 05:47:26,615][20381] Updated weights for policy 1, policy_version 17828 (0.0006) [2023-12-27 05:47:26,664][20381] Updated weights for policy 1, policy_version 17838 (0.0005) [2023-12-27 05:47:26,716][20381] Updated weights for policy 1, policy_version 17848 (0.0005) [2023-12-27 05:47:26,782][20347] Updated weights for policy 0, policy_version 17929 (0.0010) [2023-12-27 05:47:26,838][20347] Updated weights for policy 0, policy_version 17939 (0.0010) [2023-12-27 05:47:26,905][20347] Updated weights for policy 0, policy_version 17949 (0.0010) [2023-12-27 05:47:27,324][20381] Updated weights for policy 1, policy_version 17858 (0.0006) [2023-12-27 05:47:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 9166848. Throughput: 0: 10417.7, 1: 10276.7. Samples: 9158212. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-27 05:47:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:27,378][20381] Updated weights for policy 1, policy_version 17868 (0.0010) [2023-12-27 05:47:27,428][20381] Updated weights for policy 1, policy_version 17878 (0.0010) [2023-12-27 05:47:27,476][20347] Updated weights for policy 0, policy_version 17959 (0.0010) [2023-12-27 05:47:27,487][20381] Updated weights for policy 1, policy_version 17888 (0.0010) [2023-12-27 05:47:27,530][20347] Updated weights for policy 0, policy_version 17969 (0.0010) [2023-12-27 05:47:27,591][20347] Updated weights for policy 0, policy_version 17979 (0.0010) [2023-12-27 05:47:28,093][20381] Updated weights for policy 1, policy_version 17898 (0.0007) [2023-12-27 05:47:28,146][20381] Updated weights for policy 1, policy_version 17908 (0.0007) [2023-12-27 05:47:28,202][20381] Updated weights for policy 1, policy_version 17918 (0.0007) [2023-12-27 05:47:28,214][20347] Updated weights for policy 0, policy_version 17989 (0.0010) [2023-12-27 05:47:28,257][20347] Updated weights for policy 0, policy_version 17999 (0.0009) [2023-12-27 05:47:28,303][20347] Updated weights for policy 0, policy_version 18009 (0.0007) [2023-12-27 05:47:28,932][20381] Updated weights for policy 1, policy_version 17928 (0.0009) [2023-12-27 05:47:28,978][20347] Updated weights for policy 0, policy_version 18019 (0.0007) [2023-12-27 05:47:28,980][20381] Updated weights for policy 1, policy_version 17938 (0.0010) [2023-12-27 05:47:29,025][20381] Updated weights for policy 1, policy_version 17948 (0.0008) [2023-12-27 05:47:29,033][20347] Updated weights for policy 0, policy_version 18029 (0.0010) [2023-12-27 05:47:29,089][20347] Updated weights for policy 0, policy_version 18039 (0.0010) [2023-12-27 05:47:29,690][20381] Updated weights for policy 1, policy_version 17958 (0.0008) [2023-12-27 05:47:29,751][20381] Updated weights for policy 1, policy_version 17968 (0.0006) [2023-12-27 05:47:29,759][20347] Updated weights for policy 0, policy_version 18049 (0.0009) [2023-12-27 05:47:29,811][20381] Updated weights for policy 1, policy_version 17978 (0.0006) [2023-12-27 05:47:29,815][20347] Updated weights for policy 0, policy_version 18059 (0.0006) [2023-12-27 05:47:29,877][20347] Updated weights for policy 0, policy_version 18069 (0.0008) [2023-12-27 05:47:29,938][20347] Updated weights for policy 0, policy_version 18079 (0.0008) [2023-12-27 05:47:30,436][20381] Updated weights for policy 1, policy_version 17988 (0.0007) [2023-12-27 05:47:30,488][20381] Updated weights for policy 1, policy_version 17998 (0.0005) [2023-12-27 05:47:30,538][20381] Updated weights for policy 1, policy_version 18008 (0.0005) [2023-12-27 05:47:30,644][20347] Updated weights for policy 0, policy_version 18089 (0.0007) [2023-12-27 05:47:30,689][20347] Updated weights for policy 0, policy_version 18099 (0.0005) [2023-12-27 05:47:30,743][20347] Updated weights for policy 0, policy_version 18109 (0.0005) [2023-12-27 05:47:31,079][20381] Updated weights for policy 1, policy_version 18018 (0.0006) [2023-12-27 05:47:31,144][20381] Updated weights for policy 1, policy_version 18028 (0.0008) [2023-12-27 05:47:31,198][20381] Updated weights for policy 1, policy_version 18038 (0.0007) [2023-12-27 05:47:31,259][20381] Updated weights for policy 1, policy_version 18048 (0.0006) [2023-12-27 05:47:31,369][20347] Updated weights for policy 0, policy_version 18119 (0.0009) [2023-12-27 05:47:31,427][20347] Updated weights for policy 0, policy_version 18129 (0.0007) [2023-12-27 05:47:31,479][20347] Updated weights for policy 0, policy_version 18139 (0.0008) [2023-12-27 05:47:31,870][20381] Updated weights for policy 1, policy_version 18058 (0.0006) [2023-12-27 05:47:31,928][20381] Updated weights for policy 1, policy_version 18068 (0.0007) [2023-12-27 05:47:31,983][20381] Updated weights for policy 1, policy_version 18078 (0.0007) [2023-12-27 05:47:32,208][20347] Updated weights for policy 0, policy_version 18149 (0.0008) [2023-12-27 05:47:32,272][20347] Updated weights for policy 0, policy_version 18159 (0.0006) [2023-12-27 05:47:32,322][20347] Updated weights for policy 0, policy_version 18169 (0.0007) [2023-12-27 05:47:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 9281536. Throughput: 0: 10390.2, 1: 10359.5. Samples: 9288376. Policy #0 lag: (min: 31.0, avg: 44.6, max: 63.0) [2023-12-27 05:47:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:32,645][20381] Updated weights for policy 1, policy_version 18088 (0.0006) [2023-12-27 05:47:32,703][20381] Updated weights for policy 1, policy_version 18098 (0.0005) [2023-12-27 05:47:32,759][20381] Updated weights for policy 1, policy_version 18108 (0.0005) [2023-12-27 05:47:32,946][20347] Updated weights for policy 0, policy_version 18179 (0.0008) [2023-12-27 05:47:33,002][20347] Updated weights for policy 0, policy_version 18189 (0.0005) [2023-12-27 05:47:33,060][20347] Updated weights for policy 0, policy_version 18199 (0.0005) [2023-12-27 05:47:33,308][20381] Updated weights for policy 1, policy_version 18118 (0.0007) [2023-12-27 05:47:33,369][20381] Updated weights for policy 1, policy_version 18128 (0.0008) [2023-12-27 05:47:33,420][20381] Updated weights for policy 1, policy_version 18138 (0.0007) [2023-12-27 05:47:33,571][20347] Updated weights for policy 0, policy_version 18209 (0.0005) [2023-12-27 05:47:33,620][20347] Updated weights for policy 0, policy_version 18219 (0.0005) [2023-12-27 05:47:33,682][20347] Updated weights for policy 0, policy_version 18229 (0.0005) [2023-12-27 05:47:33,740][20347] Updated weights for policy 0, policy_version 18239 (0.0010) [2023-12-27 05:47:34,046][20381] Updated weights for policy 1, policy_version 18148 (0.0008) [2023-12-27 05:47:34,097][20381] Updated weights for policy 1, policy_version 18158 (0.0010) [2023-12-27 05:47:34,149][20381] Updated weights for policy 1, policy_version 18168 (0.0010) [2023-12-27 05:47:34,405][20347] Updated weights for policy 0, policy_version 18249 (0.0009) [2023-12-27 05:47:34,470][20347] Updated weights for policy 0, policy_version 18259 (0.0010) [2023-12-27 05:47:34,527][20347] Updated weights for policy 0, policy_version 18269 (0.0009) [2023-12-27 05:47:34,854][20381] Updated weights for policy 1, policy_version 18178 (0.0010) [2023-12-27 05:47:34,915][20381] Updated weights for policy 1, policy_version 18188 (0.0007) [2023-12-27 05:47:34,975][20381] Updated weights for policy 1, policy_version 18198 (0.0005) [2023-12-27 05:47:35,032][20381] Updated weights for policy 1, policy_version 18208 (0.0005) [2023-12-27 05:47:35,179][20347] Updated weights for policy 0, policy_version 18279 (0.0008) [2023-12-27 05:47:35,252][20347] Updated weights for policy 0, policy_version 18289 (0.0007) [2023-12-27 05:47:35,308][20347] Updated weights for policy 0, policy_version 18299 (0.0005) [2023-12-27 05:47:35,577][20381] Updated weights for policy 1, policy_version 18218 (0.0008) [2023-12-27 05:47:35,632][20381] Updated weights for policy 1, policy_version 18228 (0.0007) [2023-12-27 05:47:35,679][20381] Updated weights for policy 1, policy_version 18238 (0.0007) [2023-12-27 05:47:35,960][20347] Updated weights for policy 0, policy_version 18309 (0.0005) [2023-12-27 05:47:36,021][20347] Updated weights for policy 0, policy_version 18319 (0.0010) [2023-12-27 05:47:36,080][20347] Updated weights for policy 0, policy_version 18329 (0.0006) [2023-12-27 05:47:36,338][20381] Updated weights for policy 1, policy_version 18248 (0.0007) [2023-12-27 05:47:36,398][20381] Updated weights for policy 1, policy_version 18258 (0.0008) [2023-12-27 05:47:36,456][20381] Updated weights for policy 1, policy_version 18268 (0.0007) [2023-12-27 05:47:36,680][20347] Updated weights for policy 0, policy_version 18339 (0.0010) [2023-12-27 05:47:36,737][20347] Updated weights for policy 0, policy_version 18349 (0.0010) [2023-12-27 05:47:36,794][20347] Updated weights for policy 0, policy_version 18359 (0.0010) [2023-12-27 05:47:37,122][20381] Updated weights for policy 1, policy_version 18278 (0.0007) [2023-12-27 05:47:37,173][20381] Updated weights for policy 1, policy_version 18288 (0.0005) [2023-12-27 05:47:37,226][20381] Updated weights for policy 1, policy_version 18298 (0.0007) [2023-12-27 05:47:37,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 9388032. Throughput: 0: 10420.7, 1: 10420.8. Samples: 9354684. Policy #0 lag: (min: 31.0, avg: 31.5, max: 48.0) [2023-12-27 05:47:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000018304_4685824.pth... [2023-12-27 05:47:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000018368_4702208.pth... [2023-12-27 05:47:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000017088_4374528.pth [2023-12-27 05:47:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000017152_4390912.pth [2023-12-27 05:47:37,468][20347] Updated weights for policy 0, policy_version 18369 (0.0010) [2023-12-27 05:47:37,515][20347] Updated weights for policy 0, policy_version 18379 (0.0010) [2023-12-27 05:47:37,567][20347] Updated weights for policy 0, policy_version 18389 (0.0010) [2023-12-27 05:47:37,619][20347] Updated weights for policy 0, policy_version 18399 (0.0010) [2023-12-27 05:47:37,864][20381] Updated weights for policy 1, policy_version 18308 (0.0007) [2023-12-27 05:47:37,910][20381] Updated weights for policy 1, policy_version 18318 (0.0007) [2023-12-27 05:47:37,964][20381] Updated weights for policy 1, policy_version 18328 (0.0007) [2023-12-27 05:47:38,281][20347] Updated weights for policy 0, policy_version 18409 (0.0010) [2023-12-27 05:47:38,347][20347] Updated weights for policy 0, policy_version 18419 (0.0009) [2023-12-27 05:47:38,403][20347] Updated weights for policy 0, policy_version 18429 (0.0008) [2023-12-27 05:47:38,622][20381] Updated weights for policy 1, policy_version 18338 (0.0007) [2023-12-27 05:47:38,687][20381] Updated weights for policy 1, policy_version 18348 (0.0006) [2023-12-27 05:47:38,744][20381] Updated weights for policy 1, policy_version 18358 (0.0007) [2023-12-27 05:47:38,802][20381] Updated weights for policy 1, policy_version 18368 (0.0008) [2023-12-27 05:47:39,079][20347] Updated weights for policy 0, policy_version 18439 (0.0006) [2023-12-27 05:47:39,142][20347] Updated weights for policy 0, policy_version 18449 (0.0008) [2023-12-27 05:47:39,195][20347] Updated weights for policy 0, policy_version 18459 (0.0010) [2023-12-27 05:47:39,514][20381] Updated weights for policy 1, policy_version 18378 (0.0009) [2023-12-27 05:47:39,577][20381] Updated weights for policy 1, policy_version 18388 (0.0007) [2023-12-27 05:47:39,643][20381] Updated weights for policy 1, policy_version 18398 (0.0008) [2023-12-27 05:47:39,793][20347] Updated weights for policy 0, policy_version 18469 (0.0009) [2023-12-27 05:47:39,854][20347] Updated weights for policy 0, policy_version 18479 (0.0011) [2023-12-27 05:47:39,918][20347] Updated weights for policy 0, policy_version 18489 (0.0008) [2023-12-27 05:47:40,364][20381] Updated weights for policy 1, policy_version 18408 (0.0008) [2023-12-27 05:47:40,432][20381] Updated weights for policy 1, policy_version 18418 (0.0007) [2023-12-27 05:47:40,491][20381] Updated weights for policy 1, policy_version 18428 (0.0007) [2023-12-27 05:47:40,550][20347] Updated weights for policy 0, policy_version 18499 (0.0010) [2023-12-27 05:47:40,612][20347] Updated weights for policy 0, policy_version 18509 (0.0008) [2023-12-27 05:47:40,671][20347] Updated weights for policy 0, policy_version 18519 (0.0006) [2023-12-27 05:47:41,166][20381] Updated weights for policy 1, policy_version 18438 (0.0008) [2023-12-27 05:47:41,220][20381] Updated weights for policy 1, policy_version 18448 (0.0009) [2023-12-27 05:47:41,280][20381] Updated weights for policy 1, policy_version 18458 (0.0009) [2023-12-27 05:47:41,329][20347] Updated weights for policy 0, policy_version 18529 (0.0010) [2023-12-27 05:47:41,389][20347] Updated weights for policy 0, policy_version 18539 (0.0009) [2023-12-27 05:47:41,449][20347] Updated weights for policy 0, policy_version 18549 (0.0008) [2023-12-27 05:47:41,510][20347] Updated weights for policy 0, policy_version 18559 (0.0008) [2023-12-27 05:47:42,027][20381] Updated weights for policy 1, policy_version 18468 (0.0008) [2023-12-27 05:47:42,085][20381] Updated weights for policy 1, policy_version 18478 (0.0008) [2023-12-27 05:47:42,141][20381] Updated weights for policy 1, policy_version 18488 (0.0008) [2023-12-27 05:47:42,232][20347] Updated weights for policy 0, policy_version 18569 (0.0007) [2023-12-27 05:47:42,293][20347] Updated weights for policy 0, policy_version 18579 (0.0008) [2023-12-27 05:47:42,341][20347] Updated weights for policy 0, policy_version 18589 (0.0008) [2023-12-27 05:47:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 9494528. Throughput: 0: 10475.7, 1: 10380.9. Samples: 9480312. Policy #0 lag: (min: 31.0, avg: 31.5, max: 48.0) [2023-12-27 05:47:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:42,790][20381] Updated weights for policy 1, policy_version 18498 (0.0006) [2023-12-27 05:47:42,849][20381] Updated weights for policy 1, policy_version 18508 (0.0007) [2023-12-27 05:47:42,898][20381] Updated weights for policy 1, policy_version 18518 (0.0007) [2023-12-27 05:47:42,943][20381] Updated weights for policy 1, policy_version 18528 (0.0008) [2023-12-27 05:47:43,056][20347] Updated weights for policy 0, policy_version 18599 (0.0008) [2023-12-27 05:47:43,112][20347] Updated weights for policy 0, policy_version 18609 (0.0007) [2023-12-27 05:47:43,167][20347] Updated weights for policy 0, policy_version 18619 (0.0008) [2023-12-27 05:47:43,646][20381] Updated weights for policy 1, policy_version 18538 (0.0007) [2023-12-27 05:47:43,698][20381] Updated weights for policy 1, policy_version 18548 (0.0008) [2023-12-27 05:47:43,756][20381] Updated weights for policy 1, policy_version 18558 (0.0009) [2023-12-27 05:47:43,822][20347] Updated weights for policy 0, policy_version 18629 (0.0007) [2023-12-27 05:47:43,873][20347] Updated weights for policy 0, policy_version 18639 (0.0005) [2023-12-27 05:47:43,924][20347] Updated weights for policy 0, policy_version 18649 (0.0007) [2023-12-27 05:47:44,444][20381] Updated weights for policy 1, policy_version 18568 (0.0006) [2023-12-27 05:47:44,503][20381] Updated weights for policy 1, policy_version 18578 (0.0006) [2023-12-27 05:47:44,561][20381] Updated weights for policy 1, policy_version 18588 (0.0006) [2023-12-27 05:47:44,626][20347] Updated weights for policy 0, policy_version 18659 (0.0007) [2023-12-27 05:47:44,678][20347] Updated weights for policy 0, policy_version 18669 (0.0007) [2023-12-27 05:47:44,727][20347] Updated weights for policy 0, policy_version 18679 (0.0007) [2023-12-27 05:47:45,192][20381] Updated weights for policy 1, policy_version 18598 (0.0006) [2023-12-27 05:47:45,248][20381] Updated weights for policy 1, policy_version 18608 (0.0005) [2023-12-27 05:47:45,301][20381] Updated weights for policy 1, policy_version 18618 (0.0006) [2023-12-27 05:47:45,488][20347] Updated weights for policy 0, policy_version 18689 (0.0008) [2023-12-27 05:47:45,549][20347] Updated weights for policy 0, policy_version 18699 (0.0007) [2023-12-27 05:47:45,594][20347] Updated weights for policy 0, policy_version 18709 (0.0008) [2023-12-27 05:47:45,641][20347] Updated weights for policy 0, policy_version 18719 (0.0007) [2023-12-27 05:47:45,919][20381] Updated weights for policy 1, policy_version 18628 (0.0006) [2023-12-27 05:47:45,971][20381] Updated weights for policy 1, policy_version 18638 (0.0008) [2023-12-27 05:47:46,023][20381] Updated weights for policy 1, policy_version 18648 (0.0008) [2023-12-27 05:47:46,351][20347] Updated weights for policy 0, policy_version 18729 (0.0008) [2023-12-27 05:47:46,402][20347] Updated weights for policy 0, policy_version 18739 (0.0007) [2023-12-27 05:47:46,451][20347] Updated weights for policy 0, policy_version 18749 (0.0008) [2023-12-27 05:47:46,721][20381] Updated weights for policy 1, policy_version 18658 (0.0007) [2023-12-27 05:47:46,776][20381] Updated weights for policy 1, policy_version 18668 (0.0008) [2023-12-27 05:47:46,822][20381] Updated weights for policy 1, policy_version 18678 (0.0007) [2023-12-27 05:47:46,874][20381] Updated weights for policy 1, policy_version 18688 (0.0008) [2023-12-27 05:47:47,150][20347] Updated weights for policy 0, policy_version 18759 (0.0006) [2023-12-27 05:47:47,205][20347] Updated weights for policy 0, policy_version 18769 (0.0005) [2023-12-27 05:47:47,260][20347] Updated weights for policy 0, policy_version 18779 (0.0005) [2023-12-27 05:47:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 9592832. Throughput: 0: 10414.6, 1: 10403.1. Samples: 9601088. Policy #0 lag: (min: 31.0, avg: 31.5, max: 48.0) [2023-12-27 05:47:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:47,536][20381] Updated weights for policy 1, policy_version 18698 (0.0005) [2023-12-27 05:47:47,590][20381] Updated weights for policy 1, policy_version 18708 (0.0008) [2023-12-27 05:47:47,643][20381] Updated weights for policy 1, policy_version 18718 (0.0008) [2023-12-27 05:47:47,836][20347] Updated weights for policy 0, policy_version 18789 (0.0007) [2023-12-27 05:47:47,889][20347] Updated weights for policy 0, policy_version 18799 (0.0007) [2023-12-27 05:47:47,941][20347] Updated weights for policy 0, policy_version 18809 (0.0007) [2023-12-27 05:47:48,255][20381] Updated weights for policy 1, policy_version 18728 (0.0006) [2023-12-27 05:47:48,316][20381] Updated weights for policy 1, policy_version 18738 (0.0005) [2023-12-27 05:47:48,379][20381] Updated weights for policy 1, policy_version 18748 (0.0008) [2023-12-27 05:47:48,672][20347] Updated weights for policy 0, policy_version 18819 (0.0008) [2023-12-27 05:47:48,721][20347] Updated weights for policy 0, policy_version 18829 (0.0009) [2023-12-27 05:47:48,771][20347] Updated weights for policy 0, policy_version 18839 (0.0008) [2023-12-27 05:47:48,958][20381] Updated weights for policy 1, policy_version 18758 (0.0007) [2023-12-27 05:47:49,006][20381] Updated weights for policy 1, policy_version 18768 (0.0006) [2023-12-27 05:47:49,055][20381] Updated weights for policy 1, policy_version 18778 (0.0006) [2023-12-27 05:47:49,545][20347] Updated weights for policy 0, policy_version 18849 (0.0007) [2023-12-27 05:47:49,608][20347] Updated weights for policy 0, policy_version 18859 (0.0007) [2023-12-27 05:47:49,671][20347] Updated weights for policy 0, policy_version 18869 (0.0005) [2023-12-27 05:47:49,731][20381] Updated weights for policy 1, policy_version 18788 (0.0006) [2023-12-27 05:47:49,733][20347] Updated weights for policy 0, policy_version 18879 (0.0006) [2023-12-27 05:47:49,780][20381] Updated weights for policy 1, policy_version 18798 (0.0007) [2023-12-27 05:47:49,836][20381] Updated weights for policy 1, policy_version 18808 (0.0008) [2023-12-27 05:47:50,375][20347] Updated weights for policy 0, policy_version 18889 (0.0010) [2023-12-27 05:47:50,437][20347] Updated weights for policy 0, policy_version 18899 (0.0010) [2023-12-27 05:47:50,491][20347] Updated weights for policy 0, policy_version 18909 (0.0010) [2023-12-27 05:47:50,541][20381] Updated weights for policy 1, policy_version 18818 (0.0006) [2023-12-27 05:47:50,598][20381] Updated weights for policy 1, policy_version 18828 (0.0008) [2023-12-27 05:47:50,660][20381] Updated weights for policy 1, policy_version 18838 (0.0008) [2023-12-27 05:47:50,722][20381] Updated weights for policy 1, policy_version 18848 (0.0007) [2023-12-27 05:47:51,166][20347] Updated weights for policy 0, policy_version 18919 (0.0010) [2023-12-27 05:47:51,217][20347] Updated weights for policy 0, policy_version 18929 (0.0008) [2023-12-27 05:47:51,270][20347] Updated weights for policy 0, policy_version 18939 (0.0007) [2023-12-27 05:47:51,405][20381] Updated weights for policy 1, policy_version 18858 (0.0007) [2023-12-27 05:47:51,453][20381] Updated weights for policy 1, policy_version 18868 (0.0007) [2023-12-27 05:47:51,500][20381] Updated weights for policy 1, policy_version 18878 (0.0007) [2023-12-27 05:47:52,044][20347] Updated weights for policy 0, policy_version 18949 (0.0008) [2023-12-27 05:47:52,104][20347] Updated weights for policy 0, policy_version 18959 (0.0008) [2023-12-27 05:47:52,163][20347] Updated weights for policy 0, policy_version 18969 (0.0006) [2023-12-27 05:47:52,188][20381] Updated weights for policy 1, policy_version 18888 (0.0007) [2023-12-27 05:47:52,236][20381] Updated weights for policy 1, policy_version 18898 (0.0007) [2023-12-27 05:47:52,290][20381] Updated weights for policy 1, policy_version 18908 (0.0007) [2023-12-27 05:47:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 9699328. Throughput: 0: 10375.6, 1: 10439.2. Samples: 9664484. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 05:47:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000018912_4841472.pth... [2023-12-27 05:47:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000018976_4857856.pth... [2023-12-27 05:47:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000017664_4521984.pth [2023-12-27 05:47:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000017760_4546560.pth [2023-12-27 05:47:52,876][20381] Updated weights for policy 1, policy_version 18918 (0.0005) [2023-12-27 05:47:52,883][20347] Updated weights for policy 0, policy_version 18979 (0.0006) [2023-12-27 05:47:52,931][20381] Updated weights for policy 1, policy_version 18928 (0.0006) [2023-12-27 05:47:52,940][20347] Updated weights for policy 0, policy_version 18989 (0.0005) [2023-12-27 05:47:52,984][20381] Updated weights for policy 1, policy_version 18938 (0.0008) [2023-12-27 05:47:53,000][20347] Updated weights for policy 0, policy_version 18999 (0.0005) [2023-12-27 05:47:53,582][20381] Updated weights for policy 1, policy_version 18948 (0.0006) [2023-12-27 05:47:53,596][20347] Updated weights for policy 0, policy_version 19009 (0.0006) [2023-12-27 05:47:53,644][20381] Updated weights for policy 1, policy_version 18958 (0.0005) [2023-12-27 05:47:53,656][20347] Updated weights for policy 0, policy_version 19019 (0.0005) [2023-12-27 05:47:53,704][20381] Updated weights for policy 1, policy_version 18968 (0.0007) [2023-12-27 05:47:53,711][20347] Updated weights for policy 0, policy_version 19029 (0.0005) [2023-12-27 05:47:53,767][20347] Updated weights for policy 0, policy_version 19039 (0.0006) [2023-12-27 05:47:54,320][20381] Updated weights for policy 1, policy_version 18978 (0.0007) [2023-12-27 05:47:54,369][20347] Updated weights for policy 0, policy_version 19049 (0.0006) [2023-12-27 05:47:54,374][20381] Updated weights for policy 1, policy_version 18988 (0.0005) [2023-12-27 05:47:54,417][20347] Updated weights for policy 0, policy_version 19059 (0.0006) [2023-12-27 05:47:54,431][20381] Updated weights for policy 1, policy_version 18998 (0.0006) [2023-12-27 05:47:54,467][20347] Updated weights for policy 0, policy_version 19069 (0.0007) [2023-12-27 05:47:54,499][20381] Updated weights for policy 1, policy_version 19008 (0.0005) [2023-12-27 05:47:55,036][20381] Updated weights for policy 1, policy_version 19018 (0.0007) [2023-12-27 05:47:55,100][20381] Updated weights for policy 1, policy_version 19028 (0.0007) [2023-12-27 05:47:55,128][20347] Updated weights for policy 0, policy_version 19079 (0.0008) [2023-12-27 05:47:55,163][20381] Updated weights for policy 1, policy_version 19038 (0.0007) [2023-12-27 05:47:55,186][20347] Updated weights for policy 0, policy_version 19089 (0.0009) [2023-12-27 05:47:55,245][20347] Updated weights for policy 0, policy_version 19099 (0.0008) [2023-12-27 05:47:55,816][20381] Updated weights for policy 1, policy_version 19048 (0.0007) [2023-12-27 05:47:55,869][20381] Updated weights for policy 1, policy_version 19058 (0.0008) [2023-12-27 05:47:55,918][20381] Updated weights for policy 1, policy_version 19068 (0.0007) [2023-12-27 05:47:55,946][20347] Updated weights for policy 0, policy_version 19109 (0.0006) [2023-12-27 05:47:56,005][20347] Updated weights for policy 0, policy_version 19119 (0.0007) [2023-12-27 05:47:56,062][20347] Updated weights for policy 0, policy_version 19129 (0.0007) [2023-12-27 05:47:56,644][20381] Updated weights for policy 1, policy_version 19078 (0.0007) [2023-12-27 05:47:56,699][20381] Updated weights for policy 1, policy_version 19088 (0.0007) [2023-12-27 05:47:56,715][20347] Updated weights for policy 0, policy_version 19139 (0.0008) [2023-12-27 05:47:56,751][20381] Updated weights for policy 1, policy_version 19098 (0.0006) [2023-12-27 05:47:56,771][20347] Updated weights for policy 0, policy_version 19149 (0.0010) [2023-12-27 05:47:56,825][20347] Updated weights for policy 0, policy_version 19159 (0.0010) [2023-12-27 05:47:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 9797632. Throughput: 0: 10421.9, 1: 10511.2. Samples: 9791316. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 05:47:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:47:57,461][20347] Updated weights for policy 0, policy_version 19169 (0.0007) [2023-12-27 05:47:57,468][20381] Updated weights for policy 1, policy_version 19108 (0.0006) [2023-12-27 05:47:57,515][20347] Updated weights for policy 0, policy_version 19179 (0.0005) [2023-12-27 05:47:57,539][20381] Updated weights for policy 1, policy_version 19118 (0.0007) [2023-12-27 05:47:57,560][20347] Updated weights for policy 0, policy_version 19189 (0.0009) [2023-12-27 05:47:57,584][20381] Updated weights for policy 1, policy_version 19128 (0.0005) [2023-12-27 05:47:57,609][20347] Updated weights for policy 0, policy_version 19199 (0.0010) [2023-12-27 05:47:58,181][20347] Updated weights for policy 0, policy_version 19209 (0.0008) [2023-12-27 05:47:58,244][20347] Updated weights for policy 0, policy_version 19219 (0.0008) [2023-12-27 05:47:58,288][20381] Updated weights for policy 1, policy_version 19138 (0.0005) [2023-12-27 05:47:58,308][20347] Updated weights for policy 0, policy_version 19229 (0.0008) [2023-12-27 05:47:58,351][20381] Updated weights for policy 1, policy_version 19148 (0.0008) [2023-12-27 05:47:58,418][20381] Updated weights for policy 1, policy_version 19158 (0.0008) [2023-12-27 05:47:58,475][20381] Updated weights for policy 1, policy_version 19168 (0.0008) [2023-12-27 05:47:59,076][20347] Updated weights for policy 0, policy_version 19239 (0.0010) [2023-12-27 05:47:59,123][20347] Updated weights for policy 0, policy_version 19249 (0.0007) [2023-12-27 05:47:59,136][20381] Updated weights for policy 1, policy_version 19178 (0.0010) [2023-12-27 05:47:59,180][20347] Updated weights for policy 0, policy_version 19259 (0.0005) [2023-12-27 05:47:59,191][20381] Updated weights for policy 1, policy_version 19188 (0.0010) [2023-12-27 05:47:59,252][20381] Updated weights for policy 1, policy_version 19198 (0.0009) [2023-12-27 05:47:59,796][20347] Updated weights for policy 0, policy_version 19269 (0.0006) [2023-12-27 05:47:59,833][20381] Updated weights for policy 1, policy_version 19208 (0.0006) [2023-12-27 05:47:59,853][20347] Updated weights for policy 0, policy_version 19279 (0.0008) [2023-12-27 05:47:59,893][20381] Updated weights for policy 1, policy_version 19218 (0.0008) [2023-12-27 05:47:59,914][20347] Updated weights for policy 0, policy_version 19289 (0.0007) [2023-12-27 05:47:59,951][20381] Updated weights for policy 1, policy_version 19228 (0.0006) [2023-12-27 05:48:00,567][20347] Updated weights for policy 0, policy_version 19299 (0.0007) [2023-12-27 05:48:00,617][20347] Updated weights for policy 0, policy_version 19309 (0.0007) [2023-12-27 05:48:00,640][20381] Updated weights for policy 1, policy_version 19238 (0.0009) [2023-12-27 05:48:00,674][20347] Updated weights for policy 0, policy_version 19319 (0.0005) [2023-12-27 05:48:00,694][20381] Updated weights for policy 1, policy_version 19248 (0.0010) [2023-12-27 05:48:00,752][20381] Updated weights for policy 1, policy_version 19258 (0.0010) [2023-12-27 05:48:01,318][20347] Updated weights for policy 0, policy_version 19329 (0.0005) [2023-12-27 05:48:01,378][20347] Updated weights for policy 0, policy_version 19339 (0.0007) [2023-12-27 05:48:01,437][20381] Updated weights for policy 1, policy_version 19268 (0.0008) [2023-12-27 05:48:01,439][20347] Updated weights for policy 0, policy_version 19349 (0.0008) [2023-12-27 05:48:01,493][20347] Updated weights for policy 0, policy_version 19359 (0.0008) [2023-12-27 05:48:01,498][20381] Updated weights for policy 1, policy_version 19278 (0.0005) [2023-12-27 05:48:01,559][20381] Updated weights for policy 1, policy_version 19288 (0.0006) [2023-12-27 05:48:02,172][20347] Updated weights for policy 0, policy_version 19369 (0.0010) [2023-12-27 05:48:02,178][20381] Updated weights for policy 1, policy_version 19298 (0.0006) [2023-12-27 05:48:02,229][20347] Updated weights for policy 0, policy_version 19379 (0.0010) [2023-12-27 05:48:02,230][20381] Updated weights for policy 1, policy_version 19308 (0.0007) [2023-12-27 05:48:02,293][20347] Updated weights for policy 0, policy_version 19389 (0.0010) [2023-12-27 05:48:02,294][20381] Updated weights for policy 1, policy_version 19318 (0.0006) [2023-12-27 05:48:02,348][20381] Updated weights for policy 1, policy_version 19328 (0.0007) [2023-12-27 05:48:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20716.1). Total num frames: 9912320. Throughput: 0: 10485.7, 1: 10525.1. Samples: 9916992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:02,891][20347] Updated weights for policy 0, policy_version 19399 (0.0008) [2023-12-27 05:48:02,947][20347] Updated weights for policy 0, policy_version 19409 (0.0005) [2023-12-27 05:48:03,004][20347] Updated weights for policy 0, policy_version 19419 (0.0005) [2023-12-27 05:48:03,035][20381] Updated weights for policy 1, policy_version 19338 (0.0005) [2023-12-27 05:48:03,088][20381] Updated weights for policy 1, policy_version 19348 (0.0008) [2023-12-27 05:48:03,138][20381] Updated weights for policy 1, policy_version 19358 (0.0007) [2023-12-27 05:48:03,558][20347] Updated weights for policy 0, policy_version 19429 (0.0005) [2023-12-27 05:48:03,620][20347] Updated weights for policy 0, policy_version 19439 (0.0005) [2023-12-27 05:48:03,680][20347] Updated weights for policy 0, policy_version 19449 (0.0005) [2023-12-27 05:48:03,801][20381] Updated weights for policy 1, policy_version 19368 (0.0007) [2023-12-27 05:48:03,854][20381] Updated weights for policy 1, policy_version 19378 (0.0007) [2023-12-27 05:48:03,912][20381] Updated weights for policy 1, policy_version 19388 (0.0007) [2023-12-27 05:48:04,346][20347] Updated weights for policy 0, policy_version 19459 (0.0007) [2023-12-27 05:48:04,406][20347] Updated weights for policy 0, policy_version 19469 (0.0008) [2023-12-27 05:48:04,455][20347] Updated weights for policy 0, policy_version 19479 (0.0007) [2023-12-27 05:48:04,529][20381] Updated weights for policy 1, policy_version 19398 (0.0008) [2023-12-27 05:48:04,581][20381] Updated weights for policy 1, policy_version 19408 (0.0008) [2023-12-27 05:48:04,636][20381] Updated weights for policy 1, policy_version 19418 (0.0006) [2023-12-27 05:48:05,188][20347] Updated weights for policy 0, policy_version 19489 (0.0007) [2023-12-27 05:48:05,238][20347] Updated weights for policy 0, policy_version 19499 (0.0007) [2023-12-27 05:48:05,245][20381] Updated weights for policy 1, policy_version 19428 (0.0009) [2023-12-27 05:48:05,295][20381] Updated weights for policy 1, policy_version 19438 (0.0011) [2023-12-27 05:48:05,297][20347] Updated weights for policy 0, policy_version 19509 (0.0007) [2023-12-27 05:48:05,352][20381] Updated weights for policy 1, policy_version 19448 (0.0011) [2023-12-27 05:48:05,354][20347] Updated weights for policy 0, policy_version 19519 (0.0005) [2023-12-27 05:48:06,057][20381] Updated weights for policy 1, policy_version 19458 (0.0011) [2023-12-27 05:48:06,059][20347] Updated weights for policy 0, policy_version 19529 (0.0007) [2023-12-27 05:48:06,110][20381] Updated weights for policy 1, policy_version 19468 (0.0011) [2023-12-27 05:48:06,115][20347] Updated weights for policy 0, policy_version 19539 (0.0006) [2023-12-27 05:48:06,162][20381] Updated weights for policy 1, policy_version 19478 (0.0011) [2023-12-27 05:48:06,175][20347] Updated weights for policy 0, policy_version 19549 (0.0006) [2023-12-27 05:48:06,219][20381] Updated weights for policy 1, policy_version 19488 (0.0009) [2023-12-27 05:48:06,875][20347] Updated weights for policy 0, policy_version 19559 (0.0008) [2023-12-27 05:48:06,933][20347] Updated weights for policy 0, policy_version 19569 (0.0008) [2023-12-27 05:48:06,939][20381] Updated weights for policy 1, policy_version 19498 (0.0006) [2023-12-27 05:48:06,991][20347] Updated weights for policy 0, policy_version 19579 (0.0006) [2023-12-27 05:48:07,000][20381] Updated weights for policy 1, policy_version 19508 (0.0007) [2023-12-27 05:48:07,058][20381] Updated weights for policy 1, policy_version 19518 (0.0007) [2023-12-27 05:48:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 10010624. Throughput: 0: 10491.7, 1: 10534.2. Samples: 9981256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000019520_4997120.pth... [2023-12-27 05:48:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000019584_5013504.pth... [2023-12-27 05:48:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000018304_4685824.pth [2023-12-27 05:48:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000018368_4702208.pth [2023-12-27 05:48:07,664][20347] Updated weights for policy 0, policy_version 19589 (0.0007) [2023-12-27 05:48:07,719][20381] Updated weights for policy 1, policy_version 19528 (0.0006) [2023-12-27 05:48:07,722][20347] Updated weights for policy 0, policy_version 19599 (0.0010) [2023-12-27 05:48:07,772][20381] Updated weights for policy 1, policy_version 19538 (0.0006) [2023-12-27 05:48:07,777][20347] Updated weights for policy 0, policy_version 19609 (0.0010) [2023-12-27 05:48:07,828][20381] Updated weights for policy 1, policy_version 19548 (0.0005) [2023-12-27 05:48:08,439][20347] Updated weights for policy 0, policy_version 19619 (0.0009) [2023-12-27 05:48:08,473][20381] Updated weights for policy 1, policy_version 19558 (0.0008) [2023-12-27 05:48:08,492][20347] Updated weights for policy 0, policy_version 19629 (0.0006) [2023-12-27 05:48:08,521][20381] Updated weights for policy 1, policy_version 19568 (0.0008) [2023-12-27 05:48:08,543][20347] Updated weights for policy 0, policy_version 19639 (0.0006) [2023-12-27 05:48:08,575][20381] Updated weights for policy 1, policy_version 19578 (0.0006) [2023-12-27 05:48:09,198][20347] Updated weights for policy 0, policy_version 19649 (0.0006) [2023-12-27 05:48:09,259][20347] Updated weights for policy 0, policy_version 19659 (0.0007) [2023-12-27 05:48:09,311][20347] Updated weights for policy 0, policy_version 19669 (0.0007) [2023-12-27 05:48:09,313][20381] Updated weights for policy 1, policy_version 19588 (0.0007) [2023-12-27 05:48:09,381][20381] Updated weights for policy 1, policy_version 19598 (0.0008) [2023-12-27 05:48:09,386][20347] Updated weights for policy 0, policy_version 19679 (0.0007) [2023-12-27 05:48:09,441][20381] Updated weights for policy 1, policy_version 19608 (0.0007) [2023-12-27 05:48:10,082][20347] Updated weights for policy 0, policy_version 19689 (0.0010) [2023-12-27 05:48:10,140][20347] Updated weights for policy 0, policy_version 19699 (0.0010) [2023-12-27 05:48:10,193][20381] Updated weights for policy 1, policy_version 19618 (0.0006) [2023-12-27 05:48:10,199][20347] Updated weights for policy 0, policy_version 19709 (0.0010) [2023-12-27 05:48:10,251][20381] Updated weights for policy 1, policy_version 19628 (0.0007) [2023-12-27 05:48:10,307][20381] Updated weights for policy 1, policy_version 19638 (0.0008) [2023-12-27 05:48:10,360][20381] Updated weights for policy 1, policy_version 19648 (0.0009) [2023-12-27 05:48:10,808][20347] Updated weights for policy 0, policy_version 19719 (0.0009) [2023-12-27 05:48:10,855][20347] Updated weights for policy 0, policy_version 19729 (0.0006) [2023-12-27 05:48:10,913][20347] Updated weights for policy 0, policy_version 19739 (0.0007) [2023-12-27 05:48:11,136][20381] Updated weights for policy 1, policy_version 19658 (0.0008) [2023-12-27 05:48:11,195][20381] Updated weights for policy 1, policy_version 19668 (0.0007) [2023-12-27 05:48:11,253][20381] Updated weights for policy 1, policy_version 19678 (0.0008) [2023-12-27 05:48:11,629][20347] Updated weights for policy 0, policy_version 19749 (0.0007) [2023-12-27 05:48:11,692][20347] Updated weights for policy 0, policy_version 19759 (0.0008) [2023-12-27 05:48:11,762][20347] Updated weights for policy 0, policy_version 19769 (0.0009) [2023-12-27 05:48:12,023][20381] Updated weights for policy 1, policy_version 19688 (0.0007) [2023-12-27 05:48:12,082][20381] Updated weights for policy 1, policy_version 19698 (0.0008) [2023-12-27 05:48:12,142][20381] Updated weights for policy 1, policy_version 19708 (0.0008) [2023-12-27 05:48:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 10108928. Throughput: 0: 10444.3, 1: 10510.0. Samples: 10101160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:12,468][20347] Updated weights for policy 0, policy_version 19779 (0.0010) [2023-12-27 05:48:12,530][20347] Updated weights for policy 0, policy_version 19789 (0.0011) [2023-12-27 05:48:12,592][20347] Updated weights for policy 0, policy_version 19799 (0.0010) [2023-12-27 05:48:12,851][20381] Updated weights for policy 1, policy_version 19718 (0.0008) [2023-12-27 05:48:12,917][20381] Updated weights for policy 1, policy_version 19728 (0.0008) [2023-12-27 05:48:12,977][20381] Updated weights for policy 1, policy_version 19738 (0.0006) [2023-12-27 05:48:13,251][20347] Updated weights for policy 0, policy_version 19809 (0.0010) [2023-12-27 05:48:13,308][20347] Updated weights for policy 0, policy_version 19819 (0.0010) [2023-12-27 05:48:13,365][20347] Updated weights for policy 0, policy_version 19829 (0.0010) [2023-12-27 05:48:13,433][20347] Updated weights for policy 0, policy_version 19839 (0.0010) [2023-12-27 05:48:13,597][20381] Updated weights for policy 1, policy_version 19748 (0.0006) [2023-12-27 05:48:13,646][20381] Updated weights for policy 1, policy_version 19758 (0.0008) [2023-12-27 05:48:13,696][20381] Updated weights for policy 1, policy_version 19768 (0.0008) [2023-12-27 05:48:14,078][20347] Updated weights for policy 0, policy_version 19849 (0.0010) [2023-12-27 05:48:14,142][20347] Updated weights for policy 0, policy_version 19859 (0.0010) [2023-12-27 05:48:14,193][20347] Updated weights for policy 0, policy_version 19869 (0.0010) [2023-12-27 05:48:14,346][20381] Updated weights for policy 1, policy_version 19778 (0.0010) [2023-12-27 05:48:14,397][20381] Updated weights for policy 1, policy_version 19788 (0.0007) [2023-12-27 05:48:14,455][20381] Updated weights for policy 1, policy_version 19798 (0.0008) [2023-12-27 05:48:14,516][20381] Updated weights for policy 1, policy_version 19808 (0.0009) [2023-12-27 05:48:14,754][20347] Updated weights for policy 0, policy_version 19879 (0.0007) [2023-12-27 05:48:14,812][20347] Updated weights for policy 0, policy_version 19889 (0.0008) [2023-12-27 05:48:14,867][20347] Updated weights for policy 0, policy_version 19899 (0.0008) [2023-12-27 05:48:15,263][20381] Updated weights for policy 1, policy_version 19818 (0.0007) [2023-12-27 05:48:15,318][20381] Updated weights for policy 1, policy_version 19828 (0.0007) [2023-12-27 05:48:15,372][20381] Updated weights for policy 1, policy_version 19838 (0.0005) [2023-12-27 05:48:15,607][20347] Updated weights for policy 0, policy_version 19909 (0.0008) [2023-12-27 05:48:15,664][20347] Updated weights for policy 0, policy_version 19919 (0.0008) [2023-12-27 05:48:15,719][20347] Updated weights for policy 0, policy_version 19929 (0.0008) [2023-12-27 05:48:16,014][20381] Updated weights for policy 1, policy_version 19848 (0.0007) [2023-12-27 05:48:16,059][20381] Updated weights for policy 1, policy_version 19858 (0.0007) [2023-12-27 05:48:16,112][20381] Updated weights for policy 1, policy_version 19868 (0.0008) [2023-12-27 05:48:16,357][20347] Updated weights for policy 0, policy_version 19939 (0.0008) [2023-12-27 05:48:16,412][20347] Updated weights for policy 0, policy_version 19949 (0.0006) [2023-12-27 05:48:16,463][20347] Updated weights for policy 0, policy_version 19959 (0.0007) [2023-12-27 05:48:16,793][20381] Updated weights for policy 1, policy_version 19878 (0.0006) [2023-12-27 05:48:16,845][20381] Updated weights for policy 1, policy_version 19888 (0.0008) [2023-12-27 05:48:16,897][20381] Updated weights for policy 1, policy_version 19898 (0.0009) [2023-12-27 05:48:17,031][20347] Updated weights for policy 0, policy_version 19969 (0.0005) [2023-12-27 05:48:17,090][20347] Updated weights for policy 0, policy_version 19979 (0.0006) [2023-12-27 05:48:17,148][20347] Updated weights for policy 0, policy_version 19989 (0.0005) [2023-12-27 05:48:17,195][20347] Updated weights for policy 0, policy_version 19999 (0.0005) [2023-12-27 05:48:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 10215424. Throughput: 0: 10417.6, 1: 10394.4. Samples: 10224916. Policy #0 lag: (min: 31.0, avg: 46.5, max: 63.0) [2023-12-27 05:48:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:17,543][20381] Updated weights for policy 1, policy_version 19908 (0.0007) [2023-12-27 05:48:17,604][20381] Updated weights for policy 1, policy_version 19918 (0.0005) [2023-12-27 05:48:17,650][20381] Updated weights for policy 1, policy_version 19928 (0.0005) [2023-12-27 05:48:17,793][20347] Updated weights for policy 0, policy_version 20009 (0.0008) [2023-12-27 05:48:17,847][20347] Updated weights for policy 0, policy_version 20019 (0.0010) [2023-12-27 05:48:17,899][20347] Updated weights for policy 0, policy_version 20029 (0.0010) [2023-12-27 05:48:18,296][20381] Updated weights for policy 1, policy_version 19938 (0.0005) [2023-12-27 05:48:18,358][20381] Updated weights for policy 1, policy_version 19948 (0.0007) [2023-12-27 05:48:18,418][20381] Updated weights for policy 1, policy_version 19958 (0.0008) [2023-12-27 05:48:18,482][20381] Updated weights for policy 1, policy_version 19968 (0.0008) [2023-12-27 05:48:18,595][20347] Updated weights for policy 0, policy_version 20039 (0.0010) [2023-12-27 05:48:18,660][20347] Updated weights for policy 0, policy_version 20049 (0.0010) [2023-12-27 05:48:18,719][20347] Updated weights for policy 0, policy_version 20059 (0.0010) [2023-12-27 05:48:19,187][20381] Updated weights for policy 1, policy_version 19978 (0.0007) [2023-12-27 05:48:19,248][20381] Updated weights for policy 1, policy_version 19988 (0.0008) [2023-12-27 05:48:19,310][20381] Updated weights for policy 1, policy_version 19998 (0.0007) [2023-12-27 05:48:19,375][20347] Updated weights for policy 0, policy_version 20069 (0.0009) [2023-12-27 05:48:19,435][20347] Updated weights for policy 0, policy_version 20079 (0.0007) [2023-12-27 05:48:19,503][20347] Updated weights for policy 0, policy_version 20089 (0.0010) [2023-12-27 05:48:19,983][20381] Updated weights for policy 1, policy_version 20008 (0.0008) [2023-12-27 05:48:20,039][20381] Updated weights for policy 1, policy_version 20018 (0.0007) [2023-12-27 05:48:20,101][20381] Updated weights for policy 1, policy_version 20028 (0.0008) [2023-12-27 05:48:20,188][20347] Updated weights for policy 0, policy_version 20099 (0.0008) [2023-12-27 05:48:20,247][20347] Updated weights for policy 0, policy_version 20109 (0.0010) [2023-12-27 05:48:20,309][20347] Updated weights for policy 0, policy_version 20119 (0.0010) [2023-12-27 05:48:20,857][20381] Updated weights for policy 1, policy_version 20038 (0.0009) [2023-12-27 05:48:20,916][20381] Updated weights for policy 1, policy_version 20048 (0.0008) [2023-12-27 05:48:20,971][20381] Updated weights for policy 1, policy_version 20058 (0.0007) [2023-12-27 05:48:20,990][20347] Updated weights for policy 0, policy_version 20129 (0.0010) [2023-12-27 05:48:21,053][20347] Updated weights for policy 0, policy_version 20139 (0.0010) [2023-12-27 05:48:21,119][20347] Updated weights for policy 0, policy_version 20149 (0.0008) [2023-12-27 05:48:21,187][20347] Updated weights for policy 0, policy_version 20159 (0.0011) [2023-12-27 05:48:21,711][20381] Updated weights for policy 1, policy_version 20068 (0.0009) [2023-12-27 05:48:21,775][20381] Updated weights for policy 1, policy_version 20078 (0.0008) [2023-12-27 05:48:21,834][20381] Updated weights for policy 1, policy_version 20088 (0.0006) [2023-12-27 05:48:21,872][20347] Updated weights for policy 0, policy_version 20169 (0.0008) [2023-12-27 05:48:21,930][20347] Updated weights for policy 0, policy_version 20179 (0.0007) [2023-12-27 05:48:21,998][20347] Updated weights for policy 0, policy_version 20189 (0.0007) [2023-12-27 05:48:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 10313728. Throughput: 0: 10384.2, 1: 10333.1. Samples: 10286960. Policy #0 lag: (min: 31.0, avg: 46.5, max: 63.0) [2023-12-27 05:48:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000020096_5144576.pth... [2023-12-27 05:48:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000020192_5169152.pth... [2023-12-27 05:48:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000018976_4857856.pth [2023-12-27 05:48:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000018912_4841472.pth [2023-12-27 05:48:22,463][20381] Updated weights for policy 1, policy_version 20098 (0.0006) [2023-12-27 05:48:22,524][20381] Updated weights for policy 1, policy_version 20108 (0.0007) [2023-12-27 05:48:22,576][20381] Updated weights for policy 1, policy_version 20118 (0.0008) [2023-12-27 05:48:22,634][20381] Updated weights for policy 1, policy_version 20128 (0.0007) [2023-12-27 05:48:22,670][20347] Updated weights for policy 0, policy_version 20199 (0.0010) [2023-12-27 05:48:22,727][20347] Updated weights for policy 0, policy_version 20209 (0.0010) [2023-12-27 05:48:22,781][20347] Updated weights for policy 0, policy_version 20219 (0.0010) [2023-12-27 05:48:23,317][20381] Updated weights for policy 1, policy_version 20138 (0.0009) [2023-12-27 05:48:23,372][20381] Updated weights for policy 1, policy_version 20148 (0.0007) [2023-12-27 05:48:23,396][20347] Updated weights for policy 0, policy_version 20229 (0.0008) [2023-12-27 05:48:23,429][20381] Updated weights for policy 1, policy_version 20158 (0.0007) [2023-12-27 05:48:23,449][20347] Updated weights for policy 0, policy_version 20239 (0.0007) [2023-12-27 05:48:23,493][20347] Updated weights for policy 0, policy_version 20249 (0.0010) [2023-12-27 05:48:24,027][20381] Updated weights for policy 1, policy_version 20168 (0.0009) [2023-12-27 05:48:24,084][20381] Updated weights for policy 1, policy_version 20178 (0.0010) [2023-12-27 05:48:24,091][20347] Updated weights for policy 0, policy_version 20259 (0.0009) [2023-12-27 05:48:24,143][20381] Updated weights for policy 1, policy_version 20188 (0.0010) [2023-12-27 05:48:24,152][20347] Updated weights for policy 0, policy_version 20269 (0.0007) [2023-12-27 05:48:24,202][20347] Updated weights for policy 0, policy_version 20279 (0.0007) [2023-12-27 05:48:24,806][20347] Updated weights for policy 0, policy_version 20289 (0.0009) [2023-12-27 05:48:24,811][20381] Updated weights for policy 1, policy_version 20198 (0.0011) [2023-12-27 05:48:24,857][20347] Updated weights for policy 0, policy_version 20299 (0.0010) [2023-12-27 05:48:24,876][20381] Updated weights for policy 1, policy_version 20208 (0.0010) [2023-12-27 05:48:24,914][20347] Updated weights for policy 0, policy_version 20309 (0.0006) [2023-12-27 05:48:24,938][20381] Updated weights for policy 1, policy_version 20218 (0.0010) [2023-12-27 05:48:24,964][20347] Updated weights for policy 0, policy_version 20319 (0.0005) [2023-12-27 05:48:25,556][20381] Updated weights for policy 1, policy_version 20228 (0.0010) [2023-12-27 05:48:25,561][20347] Updated weights for policy 0, policy_version 20329 (0.0006) [2023-12-27 05:48:25,611][20381] Updated weights for policy 1, policy_version 20238 (0.0010) [2023-12-27 05:48:25,618][20347] Updated weights for policy 0, policy_version 20339 (0.0005) [2023-12-27 05:48:25,669][20381] Updated weights for policy 1, policy_version 20248 (0.0010) [2023-12-27 05:48:25,677][20347] Updated weights for policy 0, policy_version 20349 (0.0005) [2023-12-27 05:48:26,262][20347] Updated weights for policy 0, policy_version 20359 (0.0009) [2023-12-27 05:48:26,283][20381] Updated weights for policy 1, policy_version 20258 (0.0009) [2023-12-27 05:48:26,311][20347] Updated weights for policy 0, policy_version 20369 (0.0010) [2023-12-27 05:48:26,335][20381] Updated weights for policy 1, policy_version 20268 (0.0006) [2023-12-27 05:48:26,363][20347] Updated weights for policy 0, policy_version 20379 (0.0010) [2023-12-27 05:48:26,377][20381] Updated weights for policy 1, policy_version 20278 (0.0006) [2023-12-27 05:48:26,429][20381] Updated weights for policy 1, policy_version 20288 (0.0008) [2023-12-27 05:48:26,994][20347] Updated weights for policy 0, policy_version 20389 (0.0010) [2023-12-27 05:48:27,059][20347] Updated weights for policy 0, policy_version 20399 (0.0010) [2023-12-27 05:48:27,063][20381] Updated weights for policy 1, policy_version 20298 (0.0005) [2023-12-27 05:48:27,113][20381] Updated weights for policy 1, policy_version 20308 (0.0008) [2023-12-27 05:48:27,119][20347] Updated weights for policy 0, policy_version 20409 (0.0010) [2023-12-27 05:48:27,167][20381] Updated weights for policy 1, policy_version 20318 (0.0010) [2023-12-27 05:48:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 10428416. Throughput: 0: 10386.7, 1: 10360.3. Samples: 10413928. Policy #0 lag: (min: 31.0, avg: 46.5, max: 63.0) [2023-12-27 05:48:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:27,749][20347] Updated weights for policy 0, policy_version 20419 (0.0010) [2023-12-27 05:48:27,797][20347] Updated weights for policy 0, policy_version 20429 (0.0010) [2023-12-27 05:48:27,837][20381] Updated weights for policy 1, policy_version 20328 (0.0011) [2023-12-27 05:48:27,851][20347] Updated weights for policy 0, policy_version 20439 (0.0010) [2023-12-27 05:48:27,901][20381] Updated weights for policy 1, policy_version 20338 (0.0011) [2023-12-27 05:48:27,953][20381] Updated weights for policy 1, policy_version 20348 (0.0010) [2023-12-27 05:48:28,500][20347] Updated weights for policy 0, policy_version 20449 (0.0010) [2023-12-27 05:48:28,558][20347] Updated weights for policy 0, policy_version 20459 (0.0008) [2023-12-27 05:48:28,618][20347] Updated weights for policy 0, policy_version 20469 (0.0007) [2023-12-27 05:48:28,648][20381] Updated weights for policy 1, policy_version 20358 (0.0011) [2023-12-27 05:48:28,675][20347] Updated weights for policy 0, policy_version 20479 (0.0008) [2023-12-27 05:48:28,703][20381] Updated weights for policy 1, policy_version 20368 (0.0008) [2023-12-27 05:48:28,748][20381] Updated weights for policy 1, policy_version 20378 (0.0005) [2023-12-27 05:48:29,354][20347] Updated weights for policy 0, policy_version 20489 (0.0008) [2023-12-27 05:48:29,397][20381] Updated weights for policy 1, policy_version 20388 (0.0006) [2023-12-27 05:48:29,410][20347] Updated weights for policy 0, policy_version 20499 (0.0006) [2023-12-27 05:48:29,455][20381] Updated weights for policy 1, policy_version 20398 (0.0006) [2023-12-27 05:48:29,469][20347] Updated weights for policy 0, policy_version 20509 (0.0007) [2023-12-27 05:48:29,505][20381] Updated weights for policy 1, policy_version 20408 (0.0006) [2023-12-27 05:48:30,159][20381] Updated weights for policy 1, policy_version 20418 (0.0005) [2023-12-27 05:48:30,196][20347] Updated weights for policy 0, policy_version 20519 (0.0008) [2023-12-27 05:48:30,214][20381] Updated weights for policy 1, policy_version 20428 (0.0006) [2023-12-27 05:48:30,243][20347] Updated weights for policy 0, policy_version 20529 (0.0005) [2023-12-27 05:48:30,274][20381] Updated weights for policy 1, policy_version 20438 (0.0006) [2023-12-27 05:48:30,299][20347] Updated weights for policy 0, policy_version 20539 (0.0007) [2023-12-27 05:48:30,326][20381] Updated weights for policy 1, policy_version 20448 (0.0005) [2023-12-27 05:48:30,892][20381] Updated weights for policy 1, policy_version 20458 (0.0010) [2023-12-27 05:48:30,939][20381] Updated weights for policy 1, policy_version 20468 (0.0009) [2023-12-27 05:48:30,987][20381] Updated weights for policy 1, policy_version 20478 (0.0006) [2023-12-27 05:48:30,997][20347] Updated weights for policy 0, policy_version 20549 (0.0007) [2023-12-27 05:48:31,059][20347] Updated weights for policy 0, policy_version 20559 (0.0008) [2023-12-27 05:48:31,112][20347] Updated weights for policy 0, policy_version 20569 (0.0007) [2023-12-27 05:48:31,642][20381] Updated weights for policy 1, policy_version 20488 (0.0009) [2023-12-27 05:48:31,700][20381] Updated weights for policy 1, policy_version 20498 (0.0008) [2023-12-27 05:48:31,766][20381] Updated weights for policy 1, policy_version 20508 (0.0009) [2023-12-27 05:48:31,798][20347] Updated weights for policy 0, policy_version 20579 (0.0007) [2023-12-27 05:48:31,856][20347] Updated weights for policy 0, policy_version 20589 (0.0005) [2023-12-27 05:48:31,923][20347] Updated weights for policy 0, policy_version 20599 (0.0006) [2023-12-27 05:48:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 10526720. Throughput: 0: 10465.8, 1: 10415.5. Samples: 10540744. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:48:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:32,376][20381] Updated weights for policy 1, policy_version 20518 (0.0007) [2023-12-27 05:48:32,435][20381] Updated weights for policy 1, policy_version 20528 (0.0006) [2023-12-27 05:48:32,491][20381] Updated weights for policy 1, policy_version 20538 (0.0007) [2023-12-27 05:48:32,586][20347] Updated weights for policy 0, policy_version 20609 (0.0008) [2023-12-27 05:48:32,656][20347] Updated weights for policy 0, policy_version 20619 (0.0008) [2023-12-27 05:48:32,719][20347] Updated weights for policy 0, policy_version 20629 (0.0008) [2023-12-27 05:48:32,780][20347] Updated weights for policy 0, policy_version 20639 (0.0007) [2023-12-27 05:48:33,061][20381] Updated weights for policy 1, policy_version 20548 (0.0006) [2023-12-27 05:48:33,106][20381] Updated weights for policy 1, policy_version 20558 (0.0005) [2023-12-27 05:48:33,159][20381] Updated weights for policy 1, policy_version 20568 (0.0005) [2023-12-27 05:48:33,484][20347] Updated weights for policy 0, policy_version 20649 (0.0008) [2023-12-27 05:48:33,539][20347] Updated weights for policy 0, policy_version 20659 (0.0007) [2023-12-27 05:48:33,595][20347] Updated weights for policy 0, policy_version 20669 (0.0007) [2023-12-27 05:48:33,723][20381] Updated weights for policy 1, policy_version 20578 (0.0005) [2023-12-27 05:48:33,775][20381] Updated weights for policy 1, policy_version 20588 (0.0010) [2023-12-27 05:48:33,824][20381] Updated weights for policy 1, policy_version 20598 (0.0008) [2023-12-27 05:48:33,869][20381] Updated weights for policy 1, policy_version 20608 (0.0005) [2023-12-27 05:48:34,356][20347] Updated weights for policy 0, policy_version 20679 (0.0008) [2023-12-27 05:48:34,412][20347] Updated weights for policy 0, policy_version 20689 (0.0007) [2023-12-27 05:48:34,440][20381] Updated weights for policy 1, policy_version 20618 (0.0011) [2023-12-27 05:48:34,466][20347] Updated weights for policy 0, policy_version 20699 (0.0007) [2023-12-27 05:48:34,496][20381] Updated weights for policy 1, policy_version 20628 (0.0005) [2023-12-27 05:48:34,550][20381] Updated weights for policy 1, policy_version 20638 (0.0006) [2023-12-27 05:48:35,166][20347] Updated weights for policy 0, policy_version 20709 (0.0007) [2023-12-27 05:48:35,214][20381] Updated weights for policy 1, policy_version 20648 (0.0009) [2023-12-27 05:48:35,223][20347] Updated weights for policy 0, policy_version 20719 (0.0007) [2023-12-27 05:48:35,270][20381] Updated weights for policy 1, policy_version 20658 (0.0010) [2023-12-27 05:48:35,276][20347] Updated weights for policy 0, policy_version 20729 (0.0008) [2023-12-27 05:48:35,322][20381] Updated weights for policy 1, policy_version 20668 (0.0010) [2023-12-27 05:48:35,970][20347] Updated weights for policy 0, policy_version 20739 (0.0006) [2023-12-27 05:48:35,988][20381] Updated weights for policy 1, policy_version 20678 (0.0011) [2023-12-27 05:48:36,037][20347] Updated weights for policy 0, policy_version 20749 (0.0007) [2023-12-27 05:48:36,041][20381] Updated weights for policy 1, policy_version 20688 (0.0011) [2023-12-27 05:48:36,088][20347] Updated weights for policy 0, policy_version 20759 (0.0007) [2023-12-27 05:48:36,099][20381] Updated weights for policy 1, policy_version 20698 (0.0011) [2023-12-27 05:48:36,772][20347] Updated weights for policy 0, policy_version 20769 (0.0006) [2023-12-27 05:48:36,791][20381] Updated weights for policy 1, policy_version 20708 (0.0011) [2023-12-27 05:48:36,828][20347] Updated weights for policy 0, policy_version 20779 (0.0005) [2023-12-27 05:48:36,853][20381] Updated weights for policy 1, policy_version 20718 (0.0011) [2023-12-27 05:48:36,885][20347] Updated weights for policy 0, policy_version 20789 (0.0005) [2023-12-27 05:48:36,912][20381] Updated weights for policy 1, policy_version 20728 (0.0011) [2023-12-27 05:48:36,933][20347] Updated weights for policy 0, policy_version 20799 (0.0005) [2023-12-27 05:48:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 10633216. Throughput: 0: 10428.1, 1: 10457.3. Samples: 10604328. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:48:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000020736_5308416.pth... [2023-12-27 05:48:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000020800_5324800.pth... [2023-12-27 05:48:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000019520_4997120.pth [2023-12-27 05:48:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000019584_5013504.pth [2023-12-27 05:48:37,587][20381] Updated weights for policy 1, policy_version 20738 (0.0011) [2023-12-27 05:48:37,626][20347] Updated weights for policy 0, policy_version 20809 (0.0006) [2023-12-27 05:48:37,641][20381] Updated weights for policy 1, policy_version 20748 (0.0007) [2023-12-27 05:48:37,685][20347] Updated weights for policy 0, policy_version 20819 (0.0008) [2023-12-27 05:48:37,693][20381] Updated weights for policy 1, policy_version 20758 (0.0005) [2023-12-27 05:48:37,743][20347] Updated weights for policy 0, policy_version 20829 (0.0008) [2023-12-27 05:48:37,758][20381] Updated weights for policy 1, policy_version 20768 (0.0008) [2023-12-27 05:48:38,327][20381] Updated weights for policy 1, policy_version 20778 (0.0006) [2023-12-27 05:48:38,363][20347] Updated weights for policy 0, policy_version 20839 (0.0007) [2023-12-27 05:48:38,399][20381] Updated weights for policy 1, policy_version 20788 (0.0008) [2023-12-27 05:48:38,419][20347] Updated weights for policy 0, policy_version 20849 (0.0006) [2023-12-27 05:48:38,464][20381] Updated weights for policy 1, policy_version 20798 (0.0008) [2023-12-27 05:48:38,471][20347] Updated weights for policy 0, policy_version 20859 (0.0006) [2023-12-27 05:48:39,047][20381] Updated weights for policy 1, policy_version 20808 (0.0010) [2023-12-27 05:48:39,101][20381] Updated weights for policy 1, policy_version 20818 (0.0010) [2023-12-27 05:48:39,151][20381] Updated weights for policy 1, policy_version 20828 (0.0006) [2023-12-27 05:48:39,193][20347] Updated weights for policy 0, policy_version 20869 (0.0006) [2023-12-27 05:48:39,255][20347] Updated weights for policy 0, policy_version 20879 (0.0007) [2023-12-27 05:48:39,321][20347] Updated weights for policy 0, policy_version 20889 (0.0007) [2023-12-27 05:48:39,841][20381] Updated weights for policy 1, policy_version 20838 (0.0007) [2023-12-27 05:48:39,896][20381] Updated weights for policy 1, policy_version 20848 (0.0008) [2023-12-27 05:48:39,961][20381] Updated weights for policy 1, policy_version 20858 (0.0010) [2023-12-27 05:48:39,996][20347] Updated weights for policy 0, policy_version 20899 (0.0006) [2023-12-27 05:48:40,052][20347] Updated weights for policy 0, policy_version 20909 (0.0009) [2023-12-27 05:48:40,105][20347] Updated weights for policy 0, policy_version 20919 (0.0008) [2023-12-27 05:48:40,653][20381] Updated weights for policy 1, policy_version 20868 (0.0009) [2023-12-27 05:48:40,706][20381] Updated weights for policy 1, policy_version 20878 (0.0008) [2023-12-27 05:48:40,753][20381] Updated weights for policy 1, policy_version 20888 (0.0007) [2023-12-27 05:48:40,815][20347] Updated weights for policy 0, policy_version 20929 (0.0008) [2023-12-27 05:48:40,869][20347] Updated weights for policy 0, policy_version 20939 (0.0008) [2023-12-27 05:48:40,918][20347] Updated weights for policy 0, policy_version 20949 (0.0007) [2023-12-27 05:48:40,966][20347] Updated weights for policy 0, policy_version 20959 (0.0007) [2023-12-27 05:48:41,522][20381] Updated weights for policy 1, policy_version 20898 (0.0008) [2023-12-27 05:48:41,582][20381] Updated weights for policy 1, policy_version 20908 (0.0008) [2023-12-27 05:48:41,642][20381] Updated weights for policy 1, policy_version 20918 (0.0008) [2023-12-27 05:48:41,691][20347] Updated weights for policy 0, policy_version 20969 (0.0007) [2023-12-27 05:48:41,692][20381] Updated weights for policy 1, policy_version 20928 (0.0007) [2023-12-27 05:48:41,757][20347] Updated weights for policy 0, policy_version 20979 (0.0008) [2023-12-27 05:48:41,820][20347] Updated weights for policy 0, policy_version 20989 (0.0007) [2023-12-27 05:48:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 10731520. Throughput: 0: 10389.9, 1: 10402.1. Samples: 10726960. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:48:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:42,479][20381] Updated weights for policy 1, policy_version 20938 (0.0009) [2023-12-27 05:48:42,481][20347] Updated weights for policy 0, policy_version 20999 (0.0005) [2023-12-27 05:48:42,540][20381] Updated weights for policy 1, policy_version 20948 (0.0007) [2023-12-27 05:48:42,545][20347] Updated weights for policy 0, policy_version 21009 (0.0009) [2023-12-27 05:48:42,606][20381] Updated weights for policy 1, policy_version 20958 (0.0008) [2023-12-27 05:48:42,611][20347] Updated weights for policy 0, policy_version 21019 (0.0010) [2023-12-27 05:48:43,215][20347] Updated weights for policy 0, policy_version 21029 (0.0010) [2023-12-27 05:48:43,219][20381] Updated weights for policy 1, policy_version 20968 (0.0007) [2023-12-27 05:48:43,274][20381] Updated weights for policy 1, policy_version 20978 (0.0007) [2023-12-27 05:48:43,276][20347] Updated weights for policy 0, policy_version 21039 (0.0010) [2023-12-27 05:48:43,334][20347] Updated weights for policy 0, policy_version 21049 (0.0010) [2023-12-27 05:48:43,336][20381] Updated weights for policy 1, policy_version 20988 (0.0007) [2023-12-27 05:48:43,907][20347] Updated weights for policy 0, policy_version 21059 (0.0008) [2023-12-27 05:48:43,958][20347] Updated weights for policy 0, policy_version 21069 (0.0010) [2023-12-27 05:48:43,980][20381] Updated weights for policy 1, policy_version 20998 (0.0010) [2023-12-27 05:48:44,011][20347] Updated weights for policy 0, policy_version 21079 (0.0010) [2023-12-27 05:48:44,038][20381] Updated weights for policy 1, policy_version 21008 (0.0011) [2023-12-27 05:48:44,091][20381] Updated weights for policy 1, policy_version 21018 (0.0011) [2023-12-27 05:48:44,694][20347] Updated weights for policy 0, policy_version 21089 (0.0010) [2023-12-27 05:48:44,751][20347] Updated weights for policy 0, policy_version 21099 (0.0007) [2023-12-27 05:48:44,782][20381] Updated weights for policy 1, policy_version 21028 (0.0010) [2023-12-27 05:48:44,815][20347] Updated weights for policy 0, policy_version 21109 (0.0007) [2023-12-27 05:48:44,845][20381] Updated weights for policy 1, policy_version 21038 (0.0008) [2023-12-27 05:48:44,873][20347] Updated weights for policy 0, policy_version 21119 (0.0005) [2023-12-27 05:48:44,905][20381] Updated weights for policy 1, policy_version 21048 (0.0011) [2023-12-27 05:48:45,469][20347] Updated weights for policy 0, policy_version 21129 (0.0009) [2023-12-27 05:48:45,533][20347] Updated weights for policy 0, policy_version 21139 (0.0010) [2023-12-27 05:48:45,582][20347] Updated weights for policy 0, policy_version 21149 (0.0010) [2023-12-27 05:48:45,605][20381] Updated weights for policy 1, policy_version 21058 (0.0011) [2023-12-27 05:48:45,666][20381] Updated weights for policy 1, policy_version 21068 (0.0008) [2023-12-27 05:48:45,719][20381] Updated weights for policy 1, policy_version 21078 (0.0005) [2023-12-27 05:48:45,772][20381] Updated weights for policy 1, policy_version 21088 (0.0005) [2023-12-27 05:48:46,274][20347] Updated weights for policy 0, policy_version 21159 (0.0006) [2023-12-27 05:48:46,338][20347] Updated weights for policy 0, policy_version 21169 (0.0005) [2023-12-27 05:48:46,393][20347] Updated weights for policy 0, policy_version 21179 (0.0005) [2023-12-27 05:48:46,399][20381] Updated weights for policy 1, policy_version 21098 (0.0006) [2023-12-27 05:48:46,452][20381] Updated weights for policy 1, policy_version 21108 (0.0007) [2023-12-27 05:48:46,512][20381] Updated weights for policy 1, policy_version 21118 (0.0007) [2023-12-27 05:48:47,026][20347] Updated weights for policy 0, policy_version 21189 (0.0008) [2023-12-27 05:48:47,078][20347] Updated weights for policy 0, policy_version 21199 (0.0009) [2023-12-27 05:48:47,135][20347] Updated weights for policy 0, policy_version 21209 (0.0007) [2023-12-27 05:48:47,143][20381] Updated weights for policy 1, policy_version 21128 (0.0006) [2023-12-27 05:48:47,200][20381] Updated weights for policy 1, policy_version 21138 (0.0007) [2023-12-27 05:48:47,251][20381] Updated weights for policy 1, policy_version 21148 (0.0007) [2023-12-27 05:48:47,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 10846208. Throughput: 0: 10394.1, 1: 10383.4. Samples: 10851984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:47,863][20347] Updated weights for policy 0, policy_version 21219 (0.0006) [2023-12-27 05:48:47,872][20381] Updated weights for policy 1, policy_version 21158 (0.0008) [2023-12-27 05:48:47,913][20347] Updated weights for policy 0, policy_version 21229 (0.0007) [2023-12-27 05:48:47,930][20381] Updated weights for policy 1, policy_version 21168 (0.0005) [2023-12-27 05:48:47,965][20347] Updated weights for policy 0, policy_version 21239 (0.0007) [2023-12-27 05:48:47,991][20381] Updated weights for policy 1, policy_version 21178 (0.0007) [2023-12-27 05:48:48,569][20381] Updated weights for policy 1, policy_version 21188 (0.0006) [2023-12-27 05:48:48,621][20381] Updated weights for policy 1, policy_version 21198 (0.0005) [2023-12-27 05:48:48,677][20381] Updated weights for policy 1, policy_version 21208 (0.0006) [2023-12-27 05:48:48,721][20347] Updated weights for policy 0, policy_version 21249 (0.0007) [2023-12-27 05:48:48,777][20347] Updated weights for policy 0, policy_version 21259 (0.0008) [2023-12-27 05:48:48,836][20347] Updated weights for policy 0, policy_version 21269 (0.0008) [2023-12-27 05:48:48,892][20347] Updated weights for policy 0, policy_version 21279 (0.0008) [2023-12-27 05:48:49,276][20381] Updated weights for policy 1, policy_version 21218 (0.0007) [2023-12-27 05:48:49,338][20381] Updated weights for policy 1, policy_version 21228 (0.0008) [2023-12-27 05:48:49,397][20381] Updated weights for policy 1, policy_version 21238 (0.0008) [2023-12-27 05:48:49,453][20381] Updated weights for policy 1, policy_version 21248 (0.0008) [2023-12-27 05:48:49,642][20347] Updated weights for policy 0, policy_version 21289 (0.0008) [2023-12-27 05:48:49,695][20347] Updated weights for policy 0, policy_version 21299 (0.0008) [2023-12-27 05:48:49,749][20347] Updated weights for policy 0, policy_version 21309 (0.0007) [2023-12-27 05:48:50,188][20381] Updated weights for policy 1, policy_version 21258 (0.0008) [2023-12-27 05:48:50,249][20381] Updated weights for policy 1, policy_version 21268 (0.0008) [2023-12-27 05:48:50,308][20381] Updated weights for policy 1, policy_version 21278 (0.0008) [2023-12-27 05:48:50,453][20347] Updated weights for policy 0, policy_version 21319 (0.0009) [2023-12-27 05:48:50,507][20347] Updated weights for policy 0, policy_version 21329 (0.0010) [2023-12-27 05:48:50,562][20347] Updated weights for policy 0, policy_version 21339 (0.0009) [2023-12-27 05:48:51,029][20381] Updated weights for policy 1, policy_version 21288 (0.0008) [2023-12-27 05:48:51,086][20381] Updated weights for policy 1, policy_version 21298 (0.0006) [2023-12-27 05:48:51,150][20381] Updated weights for policy 1, policy_version 21308 (0.0006) [2023-12-27 05:48:51,255][20347] Updated weights for policy 0, policy_version 21349 (0.0007) [2023-12-27 05:48:51,313][20347] Updated weights for policy 0, policy_version 21359 (0.0007) [2023-12-27 05:48:51,371][20347] Updated weights for policy 0, policy_version 21369 (0.0007) [2023-12-27 05:48:51,830][20381] Updated weights for policy 1, policy_version 21318 (0.0005) [2023-12-27 05:48:51,885][20381] Updated weights for policy 1, policy_version 21328 (0.0005) [2023-12-27 05:48:51,946][20381] Updated weights for policy 1, policy_version 21338 (0.0006) [2023-12-27 05:48:52,099][20347] Updated weights for policy 0, policy_version 21379 (0.0007) [2023-12-27 05:48:52,159][20347] Updated weights for policy 0, policy_version 21389 (0.0005) [2023-12-27 05:48:52,218][20347] Updated weights for policy 0, policy_version 21399 (0.0007) [2023-12-27 05:48:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 10944512. Throughput: 0: 10309.7, 1: 10391.9. Samples: 10912828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000021344_5464064.pth... [2023-12-27 05:48:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000021408_5480448.pth... [2023-12-27 05:48:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000020096_5144576.pth [2023-12-27 05:48:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000020192_5169152.pth [2023-12-27 05:48:52,589][20381] Updated weights for policy 1, policy_version 21348 (0.0008) [2023-12-27 05:48:52,645][20381] Updated weights for policy 1, policy_version 21358 (0.0008) [2023-12-27 05:48:52,697][20381] Updated weights for policy 1, policy_version 21368 (0.0008) [2023-12-27 05:48:52,852][20347] Updated weights for policy 0, policy_version 21409 (0.0008) [2023-12-27 05:48:52,916][20347] Updated weights for policy 0, policy_version 21419 (0.0006) [2023-12-27 05:48:52,976][20347] Updated weights for policy 0, policy_version 21429 (0.0007) [2023-12-27 05:48:53,038][20347] Updated weights for policy 0, policy_version 21439 (0.0008) [2023-12-27 05:48:53,345][20381] Updated weights for policy 1, policy_version 21378 (0.0008) [2023-12-27 05:48:53,402][20381] Updated weights for policy 1, policy_version 21388 (0.0005) [2023-12-27 05:48:53,456][20381] Updated weights for policy 1, policy_version 21398 (0.0008) [2023-12-27 05:48:53,505][20381] Updated weights for policy 1, policy_version 21408 (0.0009) [2023-12-27 05:48:53,568][20347] Updated weights for policy 0, policy_version 21449 (0.0008) [2023-12-27 05:48:53,620][20347] Updated weights for policy 0, policy_version 21459 (0.0007) [2023-12-27 05:48:53,670][20347] Updated weights for policy 0, policy_version 21469 (0.0007) [2023-12-27 05:48:54,192][20381] Updated weights for policy 1, policy_version 21418 (0.0007) [2023-12-27 05:48:54,244][20381] Updated weights for policy 1, policy_version 21428 (0.0009) [2023-12-27 05:48:54,294][20347] Updated weights for policy 0, policy_version 21479 (0.0007) [2023-12-27 05:48:54,296][20381] Updated weights for policy 1, policy_version 21438 (0.0006) [2023-12-27 05:48:54,355][20347] Updated weights for policy 0, policy_version 21489 (0.0007) [2023-12-27 05:48:54,418][20347] Updated weights for policy 0, policy_version 21499 (0.0007) [2023-12-27 05:48:54,906][20381] Updated weights for policy 1, policy_version 21448 (0.0007) [2023-12-27 05:48:54,949][20381] Updated weights for policy 1, policy_version 21458 (0.0007) [2023-12-27 05:48:55,005][20381] Updated weights for policy 1, policy_version 21468 (0.0008) [2023-12-27 05:48:55,093][20347] Updated weights for policy 0, policy_version 21509 (0.0009) [2023-12-27 05:48:55,151][20347] Updated weights for policy 0, policy_version 21519 (0.0010) [2023-12-27 05:48:55,220][20347] Updated weights for policy 0, policy_version 21529 (0.0006) [2023-12-27 05:48:55,638][20381] Updated weights for policy 1, policy_version 21478 (0.0008) [2023-12-27 05:48:55,694][20381] Updated weights for policy 1, policy_version 21488 (0.0007) [2023-12-27 05:48:55,754][20381] Updated weights for policy 1, policy_version 21498 (0.0007) [2023-12-27 05:48:55,866][20347] Updated weights for policy 0, policy_version 21539 (0.0006) [2023-12-27 05:48:55,924][20347] Updated weights for policy 0, policy_version 21549 (0.0009) [2023-12-27 05:48:55,990][20347] Updated weights for policy 0, policy_version 21559 (0.0006) [2023-12-27 05:48:56,456][20381] Updated weights for policy 1, policy_version 21508 (0.0008) [2023-12-27 05:48:56,518][20381] Updated weights for policy 1, policy_version 21518 (0.0006) [2023-12-27 05:48:56,573][20381] Updated weights for policy 1, policy_version 21528 (0.0006) [2023-12-27 05:48:56,614][20347] Updated weights for policy 0, policy_version 21569 (0.0005) [2023-12-27 05:48:56,673][20347] Updated weights for policy 0, policy_version 21579 (0.0005) [2023-12-27 05:48:56,736][20347] Updated weights for policy 0, policy_version 21589 (0.0006) [2023-12-27 05:48:56,783][20347] Updated weights for policy 0, policy_version 21599 (0.0007) [2023-12-27 05:48:57,220][20381] Updated weights for policy 1, policy_version 21538 (0.0008) [2023-12-27 05:48:57,277][20381] Updated weights for policy 1, policy_version 21548 (0.0008) [2023-12-27 05:48:57,332][20381] Updated weights for policy 1, policy_version 21558 (0.0010) [2023-12-27 05:48:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 11042816. Throughput: 0: 10353.9, 1: 10481.4. Samples: 11038744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:48:57,391][20381] Updated weights for policy 1, policy_version 21568 (0.0010) [2023-12-27 05:48:57,408][20347] Updated weights for policy 0, policy_version 21609 (0.0010) [2023-12-27 05:48:57,465][20347] Updated weights for policy 0, policy_version 21619 (0.0010) [2023-12-27 05:48:57,515][20347] Updated weights for policy 0, policy_version 21629 (0.0010) [2023-12-27 05:48:58,042][20381] Updated weights for policy 1, policy_version 21578 (0.0010) [2023-12-27 05:48:58,100][20381] Updated weights for policy 1, policy_version 21588 (0.0009) [2023-12-27 05:48:58,153][20381] Updated weights for policy 1, policy_version 21598 (0.0009) [2023-12-27 05:48:58,162][20347] Updated weights for policy 0, policy_version 21639 (0.0010) [2023-12-27 05:48:58,221][20347] Updated weights for policy 0, policy_version 21649 (0.0008) [2023-12-27 05:48:58,283][20347] Updated weights for policy 0, policy_version 21659 (0.0009) [2023-12-27 05:48:58,932][20381] Updated weights for policy 1, policy_version 21608 (0.0008) [2023-12-27 05:48:59,003][20381] Updated weights for policy 1, policy_version 21618 (0.0010) [2023-12-27 05:48:59,056][20381] Updated weights for policy 1, policy_version 21628 (0.0008) [2023-12-27 05:48:59,124][20347] Updated weights for policy 0, policy_version 21669 (0.0008) [2023-12-27 05:48:59,188][20347] Updated weights for policy 0, policy_version 21679 (0.0008) [2023-12-27 05:48:59,261][20347] Updated weights for policy 0, policy_version 21689 (0.0008) [2023-12-27 05:48:59,796][20381] Updated weights for policy 1, policy_version 21638 (0.0007) [2023-12-27 05:48:59,852][20381] Updated weights for policy 1, policy_version 21648 (0.0008) [2023-12-27 05:48:59,903][20381] Updated weights for policy 1, policy_version 21658 (0.0008) [2023-12-27 05:48:59,987][20347] Updated weights for policy 0, policy_version 21699 (0.0006) [2023-12-27 05:49:00,042][20347] Updated weights for policy 0, policy_version 21709 (0.0007) [2023-12-27 05:49:00,094][20347] Updated weights for policy 0, policy_version 21719 (0.0006) [2023-12-27 05:49:00,629][20381] Updated weights for policy 1, policy_version 21668 (0.0008) [2023-12-27 05:49:00,675][20381] Updated weights for policy 1, policy_version 21678 (0.0008) [2023-12-27 05:49:00,720][20347] Updated weights for policy 0, policy_version 21729 (0.0006) [2023-12-27 05:49:00,740][20381] Updated weights for policy 1, policy_version 21688 (0.0008) [2023-12-27 05:49:00,775][20347] Updated weights for policy 0, policy_version 21739 (0.0006) [2023-12-27 05:49:00,831][20347] Updated weights for policy 0, policy_version 21749 (0.0007) [2023-12-27 05:49:00,880][20347] Updated weights for policy 0, policy_version 21759 (0.0008) [2023-12-27 05:49:01,430][20381] Updated weights for policy 1, policy_version 21698 (0.0008) [2023-12-27 05:49:01,487][20381] Updated weights for policy 1, policy_version 21708 (0.0007) [2023-12-27 05:49:01,548][20381] Updated weights for policy 1, policy_version 21718 (0.0008) [2023-12-27 05:49:01,583][20347] Updated weights for policy 0, policy_version 21769 (0.0006) [2023-12-27 05:49:01,604][20381] Updated weights for policy 1, policy_version 21728 (0.0007) [2023-12-27 05:49:01,642][20347] Updated weights for policy 0, policy_version 21779 (0.0009) [2023-12-27 05:49:01,700][20347] Updated weights for policy 0, policy_version 21789 (0.0006) [2023-12-27 05:49:02,212][20381] Updated weights for policy 1, policy_version 21738 (0.0008) [2023-12-27 05:49:02,275][20381] Updated weights for policy 1, policy_version 21748 (0.0007) [2023-12-27 05:49:02,330][20347] Updated weights for policy 0, policy_version 21799 (0.0007) [2023-12-27 05:49:02,332][20381] Updated weights for policy 1, policy_version 21758 (0.0010) [2023-12-27 05:49:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 11149312. Throughput: 0: 10294.7, 1: 10465.3. Samples: 11159112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:49:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:02,392][20347] Updated weights for policy 0, policy_version 21809 (0.0007) [2023-12-27 05:49:02,436][20347] Updated weights for policy 0, policy_version 21819 (0.0005) [2023-12-27 05:49:02,966][20381] Updated weights for policy 1, policy_version 21768 (0.0007) [2023-12-27 05:49:03,023][20381] Updated weights for policy 1, policy_version 21778 (0.0010) [2023-12-27 05:49:03,058][20347] Updated weights for policy 0, policy_version 21829 (0.0005) [2023-12-27 05:49:03,077][20381] Updated weights for policy 1, policy_version 21788 (0.0010) [2023-12-27 05:49:03,107][20347] Updated weights for policy 0, policy_version 21839 (0.0006) [2023-12-27 05:49:03,157][20347] Updated weights for policy 0, policy_version 21849 (0.0007) [2023-12-27 05:49:03,744][20381] Updated weights for policy 1, policy_version 21798 (0.0007) [2023-12-27 05:49:03,801][20381] Updated weights for policy 1, policy_version 21808 (0.0007) [2023-12-27 05:49:03,852][20347] Updated weights for policy 0, policy_version 21859 (0.0008) [2023-12-27 05:49:03,856][20381] Updated weights for policy 1, policy_version 21818 (0.0006) [2023-12-27 05:49:03,911][20347] Updated weights for policy 0, policy_version 21869 (0.0008) [2023-12-27 05:49:03,961][20347] Updated weights for policy 0, policy_version 21879 (0.0007) [2023-12-27 05:49:04,538][20381] Updated weights for policy 1, policy_version 21828 (0.0007) [2023-12-27 05:49:04,585][20381] Updated weights for policy 1, policy_version 21838 (0.0008) [2023-12-27 05:49:04,641][20381] Updated weights for policy 1, policy_version 21848 (0.0005) [2023-12-27 05:49:04,684][20347] Updated weights for policy 0, policy_version 21889 (0.0007) [2023-12-27 05:49:04,736][20347] Updated weights for policy 0, policy_version 21899 (0.0007) [2023-12-27 05:49:04,784][20347] Updated weights for policy 0, policy_version 21909 (0.0008) [2023-12-27 05:49:04,836][20347] Updated weights for policy 0, policy_version 21919 (0.0007) [2023-12-27 05:49:05,281][20381] Updated weights for policy 1, policy_version 21858 (0.0006) [2023-12-27 05:49:05,327][20381] Updated weights for policy 1, policy_version 21868 (0.0009) [2023-12-27 05:49:05,379][20381] Updated weights for policy 1, policy_version 21878 (0.0009) [2023-12-27 05:49:05,426][20381] Updated weights for policy 1, policy_version 21888 (0.0008) [2023-12-27 05:49:05,517][20347] Updated weights for policy 0, policy_version 21929 (0.0008) [2023-12-27 05:49:05,572][20347] Updated weights for policy 0, policy_version 21939 (0.0007) [2023-12-27 05:49:05,620][20347] Updated weights for policy 0, policy_version 21949 (0.0007) [2023-12-27 05:49:06,039][20381] Updated weights for policy 1, policy_version 21898 (0.0008) [2023-12-27 05:49:06,099][20381] Updated weights for policy 1, policy_version 21908 (0.0007) [2023-12-27 05:49:06,159][20381] Updated weights for policy 1, policy_version 21918 (0.0008) [2023-12-27 05:49:06,375][20347] Updated weights for policy 0, policy_version 21959 (0.0008) [2023-12-27 05:49:06,438][20347] Updated weights for policy 0, policy_version 21969 (0.0007) [2023-12-27 05:49:06,498][20347] Updated weights for policy 0, policy_version 21979 (0.0007) [2023-12-27 05:49:06,830][20381] Updated weights for policy 1, policy_version 21928 (0.0008) [2023-12-27 05:49:06,891][20381] Updated weights for policy 1, policy_version 21938 (0.0008) [2023-12-27 05:49:06,950][20381] Updated weights for policy 1, policy_version 21948 (0.0006) [2023-12-27 05:49:07,078][20347] Updated weights for policy 0, policy_version 21989 (0.0005) [2023-12-27 05:49:07,128][20347] Updated weights for policy 0, policy_version 21999 (0.0005) [2023-12-27 05:49:07,183][20347] Updated weights for policy 0, policy_version 22009 (0.0005) [2023-12-27 05:49:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 11255808. Throughput: 0: 10280.6, 1: 10491.7. Samples: 11221716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:49:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000021952_5619712.pth... [2023-12-27 05:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000022016_5636096.pth... [2023-12-27 05:49:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000020736_5308416.pth [2023-12-27 05:49:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000020800_5324800.pth [2023-12-27 05:49:07,576][20381] Updated weights for policy 1, policy_version 21958 (0.0006) [2023-12-27 05:49:07,628][20381] Updated weights for policy 1, policy_version 21968 (0.0008) [2023-12-27 05:49:07,680][20381] Updated weights for policy 1, policy_version 21978 (0.0008) [2023-12-27 05:49:07,765][20347] Updated weights for policy 0, policy_version 22019 (0.0005) [2023-12-27 05:49:07,822][20347] Updated weights for policy 0, policy_version 22029 (0.0006) [2023-12-27 05:49:07,867][20347] Updated weights for policy 0, policy_version 22039 (0.0007) [2023-12-27 05:49:08,395][20381] Updated weights for policy 1, policy_version 21988 (0.0009) [2023-12-27 05:49:08,433][20347] Updated weights for policy 0, policy_version 22049 (0.0007) [2023-12-27 05:49:08,461][20381] Updated weights for policy 1, policy_version 21998 (0.0007) [2023-12-27 05:49:08,486][20347] Updated weights for policy 0, policy_version 22059 (0.0006) [2023-12-27 05:49:08,521][20381] Updated weights for policy 1, policy_version 22008 (0.0007) [2023-12-27 05:49:08,542][20347] Updated weights for policy 0, policy_version 22069 (0.0006) [2023-12-27 05:49:08,593][20347] Updated weights for policy 0, policy_version 22079 (0.0006) [2023-12-27 05:49:09,186][20381] Updated weights for policy 1, policy_version 22018 (0.0007) [2023-12-27 05:49:09,241][20381] Updated weights for policy 1, policy_version 22028 (0.0007) [2023-12-27 05:49:09,301][20347] Updated weights for policy 0, policy_version 22089 (0.0008) [2023-12-27 05:49:09,302][20381] Updated weights for policy 1, policy_version 22038 (0.0008) [2023-12-27 05:49:09,359][20347] Updated weights for policy 0, policy_version 22099 (0.0007) [2023-12-27 05:49:09,363][20381] Updated weights for policy 1, policy_version 22048 (0.0007) [2023-12-27 05:49:09,433][20347] Updated weights for policy 0, policy_version 22109 (0.0007) [2023-12-27 05:49:10,037][20381] Updated weights for policy 1, policy_version 22058 (0.0007) [2023-12-27 05:49:10,097][20381] Updated weights for policy 1, policy_version 22068 (0.0008) [2023-12-27 05:49:10,148][20347] Updated weights for policy 0, policy_version 22119 (0.0006) [2023-12-27 05:49:10,158][20381] Updated weights for policy 1, policy_version 22078 (0.0006) [2023-12-27 05:49:10,197][20347] Updated weights for policy 0, policy_version 22129 (0.0006) [2023-12-27 05:49:10,265][20347] Updated weights for policy 0, policy_version 22139 (0.0007) [2023-12-27 05:49:10,785][20381] Updated weights for policy 1, policy_version 22088 (0.0006) [2023-12-27 05:49:10,844][20381] Updated weights for policy 1, policy_version 22098 (0.0008) [2023-12-27 05:49:10,902][20381] Updated weights for policy 1, policy_version 22108 (0.0007) [2023-12-27 05:49:11,000][20347] Updated weights for policy 0, policy_version 22149 (0.0008) [2023-12-27 05:49:11,064][20347] Updated weights for policy 0, policy_version 22159 (0.0008) [2023-12-27 05:49:11,131][20347] Updated weights for policy 0, policy_version 22169 (0.0008) [2023-12-27 05:49:11,591][20381] Updated weights for policy 1, policy_version 22118 (0.0007) [2023-12-27 05:49:11,653][20381] Updated weights for policy 1, policy_version 22128 (0.0006) [2023-12-27 05:49:11,711][20381] Updated weights for policy 1, policy_version 22138 (0.0006) [2023-12-27 05:49:11,865][20347] Updated weights for policy 0, policy_version 22179 (0.0007) [2023-12-27 05:49:11,920][20347] Updated weights for policy 0, policy_version 22189 (0.0008) [2023-12-27 05:49:11,976][20347] Updated weights for policy 0, policy_version 22199 (0.0008) [2023-12-27 05:49:12,300][20381] Updated weights for policy 1, policy_version 22148 (0.0008) [2023-12-27 05:49:12,357][20381] Updated weights for policy 1, policy_version 22158 (0.0011) [2023-12-27 05:49:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 11354112. Throughput: 0: 10253.7, 1: 10488.3. Samples: 11347316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:49:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:12,411][20381] Updated weights for policy 1, policy_version 22168 (0.0010) [2023-12-27 05:49:12,694][20347] Updated weights for policy 0, policy_version 22209 (0.0007) [2023-12-27 05:49:12,751][20347] Updated weights for policy 0, policy_version 22219 (0.0005) [2023-12-27 05:49:12,808][20347] Updated weights for policy 0, policy_version 22229 (0.0010) [2023-12-27 05:49:12,867][20347] Updated weights for policy 0, policy_version 22239 (0.0010) [2023-12-27 05:49:13,086][20381] Updated weights for policy 1, policy_version 22178 (0.0009) [2023-12-27 05:49:13,147][20381] Updated weights for policy 1, policy_version 22188 (0.0005) [2023-12-27 05:49:13,201][20381] Updated weights for policy 1, policy_version 22198 (0.0006) [2023-12-27 05:49:13,251][20381] Updated weights for policy 1, policy_version 22208 (0.0005) [2023-12-27 05:49:13,546][20347] Updated weights for policy 0, policy_version 22249 (0.0011) [2023-12-27 05:49:13,601][20347] Updated weights for policy 0, policy_version 22259 (0.0011) [2023-12-27 05:49:13,655][20347] Updated weights for policy 0, policy_version 22269 (0.0010) [2023-12-27 05:49:13,851][20381] Updated weights for policy 1, policy_version 22218 (0.0008) [2023-12-27 05:49:13,905][20381] Updated weights for policy 1, policy_version 22228 (0.0009) [2023-12-27 05:49:13,959][20381] Updated weights for policy 1, policy_version 22238 (0.0008) [2023-12-27 05:49:14,243][20347] Updated weights for policy 0, policy_version 22279 (0.0010) [2023-12-27 05:49:14,299][20347] Updated weights for policy 0, policy_version 22289 (0.0009) [2023-12-27 05:49:14,353][20347] Updated weights for policy 0, policy_version 22299 (0.0008) [2023-12-27 05:49:14,641][20381] Updated weights for policy 1, policy_version 22248 (0.0006) [2023-12-27 05:49:14,694][20381] Updated weights for policy 1, policy_version 22258 (0.0005) [2023-12-27 05:49:14,745][20381] Updated weights for policy 1, policy_version 22268 (0.0008) [2023-12-27 05:49:15,032][20347] Updated weights for policy 0, policy_version 22309 (0.0007) [2023-12-27 05:49:15,084][20347] Updated weights for policy 0, policy_version 22319 (0.0006) [2023-12-27 05:49:15,146][20347] Updated weights for policy 0, policy_version 22329 (0.0006) [2023-12-27 05:49:15,492][20381] Updated weights for policy 1, policy_version 22278 (0.0009) [2023-12-27 05:49:15,541][20381] Updated weights for policy 1, policy_version 22288 (0.0006) [2023-12-27 05:49:15,591][20381] Updated weights for policy 1, policy_version 22298 (0.0005) [2023-12-27 05:49:15,708][20347] Updated weights for policy 0, policy_version 22339 (0.0006) [2023-12-27 05:49:15,768][20347] Updated weights for policy 0, policy_version 22349 (0.0006) [2023-12-27 05:49:15,831][20347] Updated weights for policy 0, policy_version 22359 (0.0005) [2023-12-27 05:49:16,171][20381] Updated weights for policy 1, policy_version 22308 (0.0005) [2023-12-27 05:49:16,226][20381] Updated weights for policy 1, policy_version 22318 (0.0008) [2023-12-27 05:49:16,271][20381] Updated weights for policy 1, policy_version 22328 (0.0009) [2023-12-27 05:49:16,462][20347] Updated weights for policy 0, policy_version 22369 (0.0008) [2023-12-27 05:49:16,518][20347] Updated weights for policy 0, policy_version 22379 (0.0008) [2023-12-27 05:49:16,573][20347] Updated weights for policy 0, policy_version 22389 (0.0008) [2023-12-27 05:49:16,638][20347] Updated weights for policy 0, policy_version 22399 (0.0008) [2023-12-27 05:49:16,892][20381] Updated weights for policy 1, policy_version 22338 (0.0009) [2023-12-27 05:49:16,947][20381] Updated weights for policy 1, policy_version 22348 (0.0008) [2023-12-27 05:49:16,991][20381] Updated weights for policy 1, policy_version 22358 (0.0008) [2023-12-27 05:49:17,040][20381] Updated weights for policy 1, policy_version 22368 (0.0005) [2023-12-27 05:49:17,253][20347] Updated weights for policy 0, policy_version 22409 (0.0010) [2023-12-27 05:49:17,298][20347] Updated weights for policy 0, policy_version 22419 (0.0010) [2023-12-27 05:49:17,358][20347] Updated weights for policy 0, policy_version 22429 (0.0008) [2023-12-27 05:49:17,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 11460608. Throughput: 0: 10259.9, 1: 10486.6. Samples: 11474336. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-12-27 05:49:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:17,677][20381] Updated weights for policy 1, policy_version 22378 (0.0008) [2023-12-27 05:49:17,740][20381] Updated weights for policy 1, policy_version 22388 (0.0008) [2023-12-27 05:49:17,800][20381] Updated weights for policy 1, policy_version 22398 (0.0008) [2023-12-27 05:49:18,050][20347] Updated weights for policy 0, policy_version 22439 (0.0008) [2023-12-27 05:49:18,109][20347] Updated weights for policy 0, policy_version 22449 (0.0006) [2023-12-27 05:49:18,167][20347] Updated weights for policy 0, policy_version 22459 (0.0007) [2023-12-27 05:49:18,465][20381] Updated weights for policy 1, policy_version 22408 (0.0008) [2023-12-27 05:49:18,511][20381] Updated weights for policy 1, policy_version 22418 (0.0007) [2023-12-27 05:49:18,575][20381] Updated weights for policy 1, policy_version 22428 (0.0006) [2023-12-27 05:49:18,854][20347] Updated weights for policy 0, policy_version 22469 (0.0008) [2023-12-27 05:49:18,903][20347] Updated weights for policy 0, policy_version 22479 (0.0006) [2023-12-27 05:49:18,955][20347] Updated weights for policy 0, policy_version 22489 (0.0006) [2023-12-27 05:49:19,287][20381] Updated weights for policy 1, policy_version 22438 (0.0008) [2023-12-27 05:49:19,353][20381] Updated weights for policy 1, policy_version 22448 (0.0007) [2023-12-27 05:49:19,410][20381] Updated weights for policy 1, policy_version 22458 (0.0008) [2023-12-27 05:49:19,559][20347] Updated weights for policy 0, policy_version 22499 (0.0005) [2023-12-27 05:49:19,621][20347] Updated weights for policy 0, policy_version 22509 (0.0005) [2023-12-27 05:49:19,685][20347] Updated weights for policy 0, policy_version 22519 (0.0007) [2023-12-27 05:49:20,105][20381] Updated weights for policy 1, policy_version 22468 (0.0008) [2023-12-27 05:49:20,162][20381] Updated weights for policy 1, policy_version 22478 (0.0009) [2023-12-27 05:49:20,233][20381] Updated weights for policy 1, policy_version 22488 (0.0008) [2023-12-27 05:49:20,382][20347] Updated weights for policy 0, policy_version 22529 (0.0008) [2023-12-27 05:49:20,430][20347] Updated weights for policy 0, policy_version 22539 (0.0007) [2023-12-27 05:49:20,488][20347] Updated weights for policy 0, policy_version 22549 (0.0008) [2023-12-27 05:49:20,540][20347] Updated weights for policy 0, policy_version 22559 (0.0007) [2023-12-27 05:49:20,911][20381] Updated weights for policy 1, policy_version 22498 (0.0008) [2023-12-27 05:49:20,976][20381] Updated weights for policy 1, policy_version 22508 (0.0007) [2023-12-27 05:49:21,041][20381] Updated weights for policy 1, policy_version 22518 (0.0008) [2023-12-27 05:49:21,101][20381] Updated weights for policy 1, policy_version 22528 (0.0008) [2023-12-27 05:49:21,257][20347] Updated weights for policy 0, policy_version 22569 (0.0008) [2023-12-27 05:49:21,324][20347] Updated weights for policy 0, policy_version 22579 (0.0008) [2023-12-27 05:49:21,394][20347] Updated weights for policy 0, policy_version 22589 (0.0008) [2023-12-27 05:49:21,777][20381] Updated weights for policy 1, policy_version 22538 (0.0008) [2023-12-27 05:49:21,841][20381] Updated weights for policy 1, policy_version 22548 (0.0006) [2023-12-27 05:49:21,906][20381] Updated weights for policy 1, policy_version 22558 (0.0007) [2023-12-27 05:49:22,079][20347] Updated weights for policy 0, policy_version 22599 (0.0008) [2023-12-27 05:49:22,134][20347] Updated weights for policy 0, policy_version 22609 (0.0008) [2023-12-27 05:49:22,196][20347] Updated weights for policy 0, policy_version 22619 (0.0008) [2023-12-27 05:49:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 11567104. Throughput: 0: 10315.5, 1: 10387.1. Samples: 11535940. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-12-27 05:49:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000022560_5775360.pth... [2023-12-27 05:49:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000022624_5791744.pth... [2023-12-27 05:49:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000021344_5464064.pth [2023-12-27 05:49:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000021408_5480448.pth [2023-12-27 05:49:22,590][20381] Updated weights for policy 1, policy_version 22568 (0.0007) [2023-12-27 05:49:22,652][20381] Updated weights for policy 1, policy_version 22578 (0.0007) [2023-12-27 05:49:22,709][20381] Updated weights for policy 1, policy_version 22588 (0.0009) [2023-12-27 05:49:22,819][20347] Updated weights for policy 0, policy_version 22629 (0.0007) [2023-12-27 05:49:22,884][20347] Updated weights for policy 0, policy_version 22639 (0.0009) [2023-12-27 05:49:22,945][20347] Updated weights for policy 0, policy_version 22649 (0.0011) [2023-12-27 05:49:23,341][20381] Updated weights for policy 1, policy_version 22598 (0.0007) [2023-12-27 05:49:23,405][20381] Updated weights for policy 1, policy_version 22608 (0.0008) [2023-12-27 05:49:23,470][20381] Updated weights for policy 1, policy_version 22618 (0.0008) [2023-12-27 05:49:23,547][20347] Updated weights for policy 0, policy_version 22659 (0.0009) [2023-12-27 05:49:23,611][20347] Updated weights for policy 0, policy_version 22669 (0.0005) [2023-12-27 05:49:23,673][20347] Updated weights for policy 0, policy_version 22679 (0.0005) [2023-12-27 05:49:24,148][20381] Updated weights for policy 1, policy_version 22628 (0.0007) [2023-12-27 05:49:24,203][20381] Updated weights for policy 1, policy_version 22638 (0.0005) [2023-12-27 05:49:24,248][20381] Updated weights for policy 1, policy_version 22648 (0.0005) [2023-12-27 05:49:24,304][20347] Updated weights for policy 0, policy_version 22689 (0.0009) [2023-12-27 05:49:24,367][20347] Updated weights for policy 0, policy_version 22699 (0.0007) [2023-12-27 05:49:24,428][20347] Updated weights for policy 0, policy_version 22709 (0.0010) [2023-12-27 05:49:24,484][20347] Updated weights for policy 0, policy_version 22719 (0.0010) [2023-12-27 05:49:24,782][20381] Updated weights for policy 1, policy_version 22658 (0.0006) [2023-12-27 05:49:24,833][20381] Updated weights for policy 1, policy_version 22668 (0.0007) [2023-12-27 05:49:24,880][20381] Updated weights for policy 1, policy_version 22678 (0.0006) [2023-12-27 05:49:24,923][20381] Updated weights for policy 1, policy_version 22688 (0.0007) [2023-12-27 05:49:25,141][20347] Updated weights for policy 0, policy_version 22729 (0.0010) [2023-12-27 05:49:25,192][20347] Updated weights for policy 0, policy_version 22739 (0.0010) [2023-12-27 05:49:25,252][20347] Updated weights for policy 0, policy_version 22749 (0.0010) [2023-12-27 05:49:25,622][20381] Updated weights for policy 1, policy_version 22698 (0.0007) [2023-12-27 05:49:25,669][20381] Updated weights for policy 1, policy_version 22708 (0.0007) [2023-12-27 05:49:25,718][20381] Updated weights for policy 1, policy_version 22718 (0.0007) [2023-12-27 05:49:25,920][20347] Updated weights for policy 0, policy_version 22759 (0.0010) [2023-12-27 05:49:25,972][20347] Updated weights for policy 0, policy_version 22769 (0.0010) [2023-12-27 05:49:26,030][20347] Updated weights for policy 0, policy_version 22779 (0.0010) [2023-12-27 05:49:26,408][20381] Updated weights for policy 1, policy_version 22728 (0.0007) [2023-12-27 05:49:26,455][20381] Updated weights for policy 1, policy_version 22738 (0.0007) [2023-12-27 05:49:26,501][20381] Updated weights for policy 1, policy_version 22748 (0.0007) [2023-12-27 05:49:26,693][20347] Updated weights for policy 0, policy_version 22789 (0.0008) [2023-12-27 05:49:26,741][20347] Updated weights for policy 0, policy_version 22799 (0.0005) [2023-12-27 05:49:26,791][20347] Updated weights for policy 0, policy_version 22809 (0.0005) [2023-12-27 05:49:27,136][20381] Updated weights for policy 1, policy_version 22758 (0.0006) [2023-12-27 05:49:27,182][20381] Updated weights for policy 1, policy_version 22768 (0.0005) [2023-12-27 05:49:27,228][20381] Updated weights for policy 1, policy_version 22778 (0.0005) [2023-12-27 05:49:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 11673600. Throughput: 0: 10360.7, 1: 10393.0. Samples: 11660876. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-12-27 05:49:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:27,413][20347] Updated weights for policy 0, policy_version 22819 (0.0005) [2023-12-27 05:49:27,459][20347] Updated weights for policy 0, policy_version 22829 (0.0005) [2023-12-27 05:49:27,511][20347] Updated weights for policy 0, policy_version 22839 (0.0005) [2023-12-27 05:49:27,838][20381] Updated weights for policy 1, policy_version 22788 (0.0005) [2023-12-27 05:49:27,888][20381] Updated weights for policy 1, policy_version 22798 (0.0005) [2023-12-27 05:49:27,935][20381] Updated weights for policy 1, policy_version 22808 (0.0005) [2023-12-27 05:49:28,148][20347] Updated weights for policy 0, policy_version 22849 (0.0006) [2023-12-27 05:49:28,192][20347] Updated weights for policy 0, policy_version 22859 (0.0010) [2023-12-27 05:49:28,247][20347] Updated weights for policy 0, policy_version 22869 (0.0009) [2023-12-27 05:49:28,301][20347] Updated weights for policy 0, policy_version 22879 (0.0010) [2023-12-27 05:49:28,550][20381] Updated weights for policy 1, policy_version 22818 (0.0007) [2023-12-27 05:49:28,604][20381] Updated weights for policy 1, policy_version 22828 (0.0007) [2023-12-27 05:49:28,651][20381] Updated weights for policy 1, policy_version 22838 (0.0007) [2023-12-27 05:49:28,702][20381] Updated weights for policy 1, policy_version 22848 (0.0008) [2023-12-27 05:49:28,988][20347] Updated weights for policy 0, policy_version 22889 (0.0010) [2023-12-27 05:49:29,043][20347] Updated weights for policy 0, policy_version 22899 (0.0010) [2023-12-27 05:49:29,100][20347] Updated weights for policy 0, policy_version 22909 (0.0010) [2023-12-27 05:49:29,335][20381] Updated weights for policy 1, policy_version 22858 (0.0008) [2023-12-27 05:49:29,400][20381] Updated weights for policy 1, policy_version 22868 (0.0007) [2023-12-27 05:49:29,452][20381] Updated weights for policy 1, policy_version 22878 (0.0007) [2023-12-27 05:49:29,798][20347] Updated weights for policy 0, policy_version 22919 (0.0010) [2023-12-27 05:49:29,868][20347] Updated weights for policy 0, policy_version 22929 (0.0011) [2023-12-27 05:49:29,929][20347] Updated weights for policy 0, policy_version 22939 (0.0011) [2023-12-27 05:49:30,200][20381] Updated weights for policy 1, policy_version 22888 (0.0007) [2023-12-27 05:49:30,255][20381] Updated weights for policy 1, policy_version 22898 (0.0007) [2023-12-27 05:49:30,309][20381] Updated weights for policy 1, policy_version 22908 (0.0007) [2023-12-27 05:49:30,634][20347] Updated weights for policy 0, policy_version 22949 (0.0011) [2023-12-27 05:49:30,683][20347] Updated weights for policy 0, policy_version 22959 (0.0010) [2023-12-27 05:49:30,746][20347] Updated weights for policy 0, policy_version 22969 (0.0009) [2023-12-27 05:49:30,965][20381] Updated weights for policy 1, policy_version 22918 (0.0006) [2023-12-27 05:49:31,025][20381] Updated weights for policy 1, policy_version 22928 (0.0008) [2023-12-27 05:49:31,088][20381] Updated weights for policy 1, policy_version 22938 (0.0007) [2023-12-27 05:49:31,367][20347] Updated weights for policy 0, policy_version 22979 (0.0010) [2023-12-27 05:49:31,427][20347] Updated weights for policy 0, policy_version 22989 (0.0009) [2023-12-27 05:49:31,489][20347] Updated weights for policy 0, policy_version 22999 (0.0008) [2023-12-27 05:49:31,755][20381] Updated weights for policy 1, policy_version 22948 (0.0008) [2023-12-27 05:49:31,806][20381] Updated weights for policy 1, policy_version 22958 (0.0007) [2023-12-27 05:49:31,857][20381] Updated weights for policy 1, policy_version 22968 (0.0008) [2023-12-27 05:49:32,123][20347] Updated weights for policy 0, policy_version 23009 (0.0007) [2023-12-27 05:49:32,185][20347] Updated weights for policy 0, policy_version 23019 (0.0008) [2023-12-27 05:49:32,242][20347] Updated weights for policy 0, policy_version 23029 (0.0008) [2023-12-27 05:49:32,303][20347] Updated weights for policy 0, policy_version 23039 (0.0006) [2023-12-27 05:49:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 11780096. Throughput: 0: 10343.0, 1: 10452.0. Samples: 11787752. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 05:49:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:32,563][20381] Updated weights for policy 1, policy_version 22978 (0.0008) [2023-12-27 05:49:32,620][20381] Updated weights for policy 1, policy_version 22988 (0.0007) [2023-12-27 05:49:32,675][20381] Updated weights for policy 1, policy_version 22998 (0.0009) [2023-12-27 05:49:32,736][20381] Updated weights for policy 1, policy_version 23008 (0.0007) [2023-12-27 05:49:32,896][20347] Updated weights for policy 0, policy_version 23049 (0.0005) [2023-12-27 05:49:32,955][20347] Updated weights for policy 0, policy_version 23059 (0.0005) [2023-12-27 05:49:33,001][20347] Updated weights for policy 0, policy_version 23069 (0.0008) [2023-12-27 05:49:33,308][20381] Updated weights for policy 1, policy_version 23018 (0.0005) [2023-12-27 05:49:33,356][20381] Updated weights for policy 1, policy_version 23028 (0.0007) [2023-12-27 05:49:33,405][20381] Updated weights for policy 1, policy_version 23038 (0.0008) [2023-12-27 05:49:33,622][20347] Updated weights for policy 0, policy_version 23079 (0.0006) [2023-12-27 05:49:33,668][20347] Updated weights for policy 0, policy_version 23089 (0.0005) [2023-12-27 05:49:33,724][20347] Updated weights for policy 0, policy_version 23099 (0.0005) [2023-12-27 05:49:33,965][20381] Updated weights for policy 1, policy_version 23048 (0.0006) [2023-12-27 05:49:34,020][20381] Updated weights for policy 1, policy_version 23058 (0.0005) [2023-12-27 05:49:34,073][20381] Updated weights for policy 1, policy_version 23068 (0.0007) [2023-12-27 05:49:34,369][20347] Updated weights for policy 0, policy_version 23109 (0.0007) [2023-12-27 05:49:34,415][20347] Updated weights for policy 0, policy_version 23119 (0.0008) [2023-12-27 05:49:34,470][20347] Updated weights for policy 0, policy_version 23129 (0.0008) [2023-12-27 05:49:34,638][20381] Updated weights for policy 1, policy_version 23078 (0.0006) [2023-12-27 05:49:34,692][20381] Updated weights for policy 1, policy_version 23088 (0.0008) [2023-12-27 05:49:34,751][20381] Updated weights for policy 1, policy_version 23098 (0.0008) [2023-12-27 05:49:35,240][20347] Updated weights for policy 0, policy_version 23139 (0.0008) [2023-12-27 05:49:35,292][20347] Updated weights for policy 0, policy_version 23149 (0.0008) [2023-12-27 05:49:35,336][20381] Updated weights for policy 1, policy_version 23108 (0.0006) [2023-12-27 05:49:35,348][20347] Updated weights for policy 0, policy_version 23159 (0.0009) [2023-12-27 05:49:35,395][20381] Updated weights for policy 1, policy_version 23118 (0.0007) [2023-12-27 05:49:35,444][20381] Updated weights for policy 1, policy_version 23128 (0.0009) [2023-12-27 05:49:36,043][20381] Updated weights for policy 1, policy_version 23138 (0.0006) [2023-12-27 05:49:36,081][20347] Updated weights for policy 0, policy_version 23169 (0.0008) [2023-12-27 05:49:36,091][20381] Updated weights for policy 1, policy_version 23148 (0.0011) [2023-12-27 05:49:36,142][20347] Updated weights for policy 0, policy_version 23179 (0.0008) [2023-12-27 05:49:36,152][20381] Updated weights for policy 1, policy_version 23158 (0.0008) [2023-12-27 05:49:36,201][20347] Updated weights for policy 0, policy_version 23189 (0.0011) [2023-12-27 05:49:36,207][20381] Updated weights for policy 1, policy_version 23168 (0.0007) [2023-12-27 05:49:36,260][20347] Updated weights for policy 0, policy_version 23199 (0.0010) [2023-12-27 05:49:36,864][20347] Updated weights for policy 0, policy_version 23209 (0.0006) [2023-12-27 05:49:36,904][20381] Updated weights for policy 1, policy_version 23178 (0.0011) [2023-12-27 05:49:36,914][20347] Updated weights for policy 0, policy_version 23219 (0.0005) [2023-12-27 05:49:36,950][20381] Updated weights for policy 1, policy_version 23188 (0.0010) [2023-12-27 05:49:36,965][20347] Updated weights for policy 0, policy_version 23229 (0.0007) [2023-12-27 05:49:37,009][20381] Updated weights for policy 1, policy_version 23198 (0.0011) [2023-12-27 05:49:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 11886592. Throughput: 0: 10421.0, 1: 10503.6. Samples: 11854432. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 05:49:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000023232_5947392.pth... [2023-12-27 05:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000023200_5939200.pth... [2023-12-27 05:49:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000022016_5636096.pth [2023-12-27 05:49:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000021952_5619712.pth [2023-12-27 05:49:37,603][20347] Updated weights for policy 0, policy_version 23239 (0.0010) [2023-12-27 05:49:37,652][20347] Updated weights for policy 0, policy_version 23249 (0.0010) [2023-12-27 05:49:37,708][20347] Updated weights for policy 0, policy_version 23259 (0.0010) [2023-12-27 05:49:37,714][20381] Updated weights for policy 1, policy_version 23208 (0.0006) [2023-12-27 05:49:37,771][20381] Updated weights for policy 1, policy_version 23218 (0.0007) [2023-12-27 05:49:37,822][20381] Updated weights for policy 1, policy_version 23228 (0.0007) [2023-12-27 05:49:38,408][20347] Updated weights for policy 0, policy_version 23269 (0.0008) [2023-12-27 05:49:38,460][20347] Updated weights for policy 0, policy_version 23279 (0.0005) [2023-12-27 05:49:38,463][20381] Updated weights for policy 1, policy_version 23238 (0.0007) [2023-12-27 05:49:38,519][20347] Updated weights for policy 0, policy_version 23289 (0.0005) [2023-12-27 05:49:38,521][20381] Updated weights for policy 1, policy_version 23248 (0.0011) [2023-12-27 05:49:38,575][20381] Updated weights for policy 1, policy_version 23258 (0.0009) [2023-12-27 05:49:39,172][20381] Updated weights for policy 1, policy_version 23268 (0.0009) [2023-12-27 05:49:39,191][20347] Updated weights for policy 0, policy_version 23299 (0.0007) [2023-12-27 05:49:39,234][20381] Updated weights for policy 1, policy_version 23278 (0.0011) [2023-12-27 05:49:39,247][20347] Updated weights for policy 0, policy_version 23309 (0.0006) [2023-12-27 05:49:39,294][20381] Updated weights for policy 1, policy_version 23288 (0.0011) [2023-12-27 05:49:39,307][20347] Updated weights for policy 0, policy_version 23319 (0.0006) [2023-12-27 05:49:39,955][20347] Updated weights for policy 0, policy_version 23329 (0.0007) [2023-12-27 05:49:40,017][20347] Updated weights for policy 0, policy_version 23339 (0.0007) [2023-12-27 05:49:40,037][20381] Updated weights for policy 1, policy_version 23298 (0.0008) [2023-12-27 05:49:40,082][20347] Updated weights for policy 0, policy_version 23349 (0.0007) [2023-12-27 05:49:40,092][20381] Updated weights for policy 1, policy_version 23308 (0.0007) [2023-12-27 05:49:40,139][20347] Updated weights for policy 0, policy_version 23359 (0.0006) [2023-12-27 05:49:40,149][20381] Updated weights for policy 1, policy_version 23318 (0.0005) [2023-12-27 05:49:40,206][20381] Updated weights for policy 1, policy_version 23328 (0.0008) [2023-12-27 05:49:40,766][20347] Updated weights for policy 0, policy_version 23369 (0.0006) [2023-12-27 05:49:40,806][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000006 [2023-12-27 05:49:40,857][20381] Updated weights for policy 1, policy_version 23338 (0.0007) [2023-12-27 05:49:40,910][20381] Updated weights for policy 1, policy_version 23348 (0.0009) [2023-12-27 05:49:40,970][20381] Updated weights for policy 1, policy_version 23358 (0.0008) [2023-12-27 05:49:41,506][20347] Updated weights for policy 0, policy_version 23379 (0.0005) [2023-12-27 05:49:41,594][20347] Updated weights for policy 0, policy_version 23389 (0.0005) [2023-12-27 05:49:41,658][20347] Updated weights for policy 0, policy_version 23399 (0.0007) [2023-12-27 05:49:41,691][20381] Updated weights for policy 1, policy_version 23368 (0.0010) [2023-12-27 05:49:41,762][20381] Updated weights for policy 1, policy_version 23378 (0.0010) [2023-12-27 05:49:41,821][20381] Updated weights for policy 1, policy_version 23388 (0.0010) [2023-12-27 05:49:42,293][20347] Updated weights for policy 0, policy_version 23409 (0.0006) [2023-12-27 05:49:42,355][20347] Updated weights for policy 0, policy_version 23419 (0.0008) [2023-12-27 05:49:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 11984896. Throughput: 0: 10407.2, 1: 10508.3. Samples: 11979944. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 05:49:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:42,418][20347] Updated weights for policy 0, policy_version 23429 (0.0009) [2023-12-27 05:49:42,482][20347] Updated weights for policy 0, policy_version 23439 (0.0008) [2023-12-27 05:49:42,524][20381] Updated weights for policy 1, policy_version 23398 (0.0007) [2023-12-27 05:49:42,587][20381] Updated weights for policy 1, policy_version 23408 (0.0006) [2023-12-27 05:49:42,645][20381] Updated weights for policy 1, policy_version 23418 (0.0008) [2023-12-27 05:49:43,166][20347] Updated weights for policy 0, policy_version 23449 (0.0010) [2023-12-27 05:49:43,218][20347] Updated weights for policy 0, policy_version 23459 (0.0010) [2023-12-27 05:49:43,272][20347] Updated weights for policy 0, policy_version 23469 (0.0010) [2023-12-27 05:49:43,305][20381] Updated weights for policy 1, policy_version 23428 (0.0009) [2023-12-27 05:49:43,353][20381] Updated weights for policy 1, policy_version 23438 (0.0007) [2023-12-27 05:49:43,404][20381] Updated weights for policy 1, policy_version 23448 (0.0008) [2023-12-27 05:49:43,901][20347] Updated weights for policy 0, policy_version 23479 (0.0007) [2023-12-27 05:49:43,947][20347] Updated weights for policy 0, policy_version 23489 (0.0007) [2023-12-27 05:49:43,996][20347] Updated weights for policy 0, policy_version 23499 (0.0008) [2023-12-27 05:49:44,137][20381] Updated weights for policy 1, policy_version 23458 (0.0008) [2023-12-27 05:49:44,190][20381] Updated weights for policy 1, policy_version 23468 (0.0008) [2023-12-27 05:49:44,250][20381] Updated weights for policy 1, policy_version 23478 (0.0008) [2023-12-27 05:49:44,309][20381] Updated weights for policy 1, policy_version 23488 (0.0008) [2023-12-27 05:49:44,686][20347] Updated weights for policy 0, policy_version 23509 (0.0007) [2023-12-27 05:49:44,750][20347] Updated weights for policy 0, policy_version 23519 (0.0007) [2023-12-27 05:49:44,805][20347] Updated weights for policy 0, policy_version 23529 (0.0007) [2023-12-27 05:49:44,997][20381] Updated weights for policy 1, policy_version 23498 (0.0009) [2023-12-27 05:49:45,060][20381] Updated weights for policy 1, policy_version 23508 (0.0011) [2023-12-27 05:49:45,123][20381] Updated weights for policy 1, policy_version 23518 (0.0011) [2023-12-27 05:49:45,413][20347] Updated weights for policy 0, policy_version 23539 (0.0006) [2023-12-27 05:49:45,476][20347] Updated weights for policy 0, policy_version 23549 (0.0005) [2023-12-27 05:49:45,534][20347] Updated weights for policy 0, policy_version 23559 (0.0005) [2023-12-27 05:49:45,791][20381] Updated weights for policy 1, policy_version 23528 (0.0009) [2023-12-27 05:49:45,854][20381] Updated weights for policy 1, policy_version 23538 (0.0008) [2023-12-27 05:49:45,916][20381] Updated weights for policy 1, policy_version 23548 (0.0006) [2023-12-27 05:49:46,084][20347] Updated weights for policy 0, policy_version 23569 (0.0007) [2023-12-27 05:49:46,151][20347] Updated weights for policy 0, policy_version 23579 (0.0008) [2023-12-27 05:49:46,211][20347] Updated weights for policy 0, policy_version 23589 (0.0010) [2023-12-27 05:49:46,265][20347] Updated weights for policy 0, policy_version 23599 (0.0010) [2023-12-27 05:49:46,544][20381] Updated weights for policy 1, policy_version 23558 (0.0006) [2023-12-27 05:49:46,600][20381] Updated weights for policy 1, policy_version 23568 (0.0007) [2023-12-27 05:49:46,658][20381] Updated weights for policy 1, policy_version 23578 (0.0007) [2023-12-27 05:49:46,817][20347] Updated weights for policy 0, policy_version 23609 (0.0007) [2023-12-27 05:49:46,872][20347] Updated weights for policy 0, policy_version 23619 (0.0008) [2023-12-27 05:49:46,922][20347] Updated weights for policy 0, policy_version 23629 (0.0007) [2023-12-27 05:49:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.2, 300 sec: 20716.1). Total num frames: 12091392. Throughput: 0: 10500.7, 1: 10513.6. Samples: 12104756. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 05:49:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:47,407][20381] Updated weights for policy 1, policy_version 23588 (0.0007) [2023-12-27 05:49:47,464][20381] Updated weights for policy 1, policy_version 23598 (0.0005) [2023-12-27 05:49:47,521][20381] Updated weights for policy 1, policy_version 23608 (0.0006) [2023-12-27 05:49:47,534][20347] Updated weights for policy 0, policy_version 23639 (0.0007) [2023-12-27 05:49:47,588][20347] Updated weights for policy 0, policy_version 23649 (0.0006) [2023-12-27 05:49:47,643][20347] Updated weights for policy 0, policy_version 23659 (0.0008) [2023-12-27 05:49:48,075][20381] Updated weights for policy 1, policy_version 23618 (0.0006) [2023-12-27 05:49:48,132][20381] Updated weights for policy 1, policy_version 23628 (0.0006) [2023-12-27 05:49:48,189][20381] Updated weights for policy 1, policy_version 23638 (0.0007) [2023-12-27 05:49:48,241][20381] Updated weights for policy 1, policy_version 23648 (0.0008) [2023-12-27 05:49:48,347][20347] Updated weights for policy 0, policy_version 23669 (0.0009) [2023-12-27 05:49:48,406][20347] Updated weights for policy 0, policy_version 23679 (0.0010) [2023-12-27 05:49:48,466][20347] Updated weights for policy 0, policy_version 23689 (0.0008) [2023-12-27 05:49:48,862][20381] Updated weights for policy 1, policy_version 23658 (0.0009) [2023-12-27 05:49:48,909][20381] Updated weights for policy 1, policy_version 23668 (0.0009) [2023-12-27 05:49:48,955][20381] Updated weights for policy 1, policy_version 23678 (0.0008) [2023-12-27 05:49:49,153][20347] Updated weights for policy 0, policy_version 23699 (0.0008) [2023-12-27 05:49:49,213][20347] Updated weights for policy 0, policy_version 23709 (0.0010) [2023-12-27 05:49:49,271][20347] Updated weights for policy 0, policy_version 23719 (0.0010) [2023-12-27 05:49:49,676][20381] Updated weights for policy 1, policy_version 23688 (0.0007) [2023-12-27 05:49:49,729][20381] Updated weights for policy 1, policy_version 23698 (0.0007) [2023-12-27 05:49:49,783][20381] Updated weights for policy 1, policy_version 23708 (0.0007) [2023-12-27 05:49:49,958][20347] Updated weights for policy 0, policy_version 23729 (0.0011) [2023-12-27 05:49:50,009][20347] Updated weights for policy 0, policy_version 23739 (0.0008) [2023-12-27 05:49:50,070][20347] Updated weights for policy 0, policy_version 23749 (0.0008) [2023-12-27 05:49:50,128][20347] Updated weights for policy 0, policy_version 23759 (0.0008) [2023-12-27 05:49:50,457][20381] Updated weights for policy 1, policy_version 23718 (0.0006) [2023-12-27 05:49:50,502][20381] Updated weights for policy 1, policy_version 23728 (0.0005) [2023-12-27 05:49:50,550][20381] Updated weights for policy 1, policy_version 23738 (0.0005) [2023-12-27 05:49:50,821][20347] Updated weights for policy 0, policy_version 23769 (0.0010) [2023-12-27 05:49:50,885][20347] Updated weights for policy 0, policy_version 23779 (0.0011) [2023-12-27 05:49:50,944][20347] Updated weights for policy 0, policy_version 23789 (0.0010) [2023-12-27 05:49:51,192][20381] Updated weights for policy 1, policy_version 23748 (0.0007) [2023-12-27 05:49:51,247][20381] Updated weights for policy 1, policy_version 23758 (0.0006) [2023-12-27 05:49:51,300][20381] Updated weights for policy 1, policy_version 23768 (0.0007) [2023-12-27 05:49:51,620][20347] Updated weights for policy 0, policy_version 23799 (0.0008) [2023-12-27 05:49:51,689][20347] Updated weights for policy 0, policy_version 23809 (0.0010) [2023-12-27 05:49:51,752][20347] Updated weights for policy 0, policy_version 23819 (0.0010) [2023-12-27 05:49:52,041][20381] Updated weights for policy 1, policy_version 23778 (0.0007) [2023-12-27 05:49:52,108][20381] Updated weights for policy 1, policy_version 23788 (0.0006) [2023-12-27 05:49:52,170][20381] Updated weights for policy 1, policy_version 23798 (0.0007) [2023-12-27 05:49:52,223][20381] Updated weights for policy 1, policy_version 23808 (0.0009) [2023-12-27 05:49:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 12197888. Throughput: 0: 10495.6, 1: 10511.8. Samples: 12167052. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-27 05:49:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000023808_6094848.pth... [2023-12-27 05:49:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000023824_6103040.pth... [2023-12-27 05:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000022560_5775360.pth [2023-12-27 05:49:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000022624_5791744.pth [2023-12-27 05:49:52,416][20347] Updated weights for policy 0, policy_version 23829 (0.0008) [2023-12-27 05:49:52,470][20347] Updated weights for policy 0, policy_version 23839 (0.0008) [2023-12-27 05:49:52,535][20347] Updated weights for policy 0, policy_version 23849 (0.0008) [2023-12-27 05:49:52,872][20381] Updated weights for policy 1, policy_version 23818 (0.0007) [2023-12-27 05:49:52,933][20381] Updated weights for policy 1, policy_version 23828 (0.0009) [2023-12-27 05:49:52,996][20381] Updated weights for policy 1, policy_version 23838 (0.0008) [2023-12-27 05:49:53,168][20347] Updated weights for policy 0, policy_version 23859 (0.0008) [2023-12-27 05:49:53,216][20347] Updated weights for policy 0, policy_version 23869 (0.0010) [2023-12-27 05:49:53,260][20347] Updated weights for policy 0, policy_version 23879 (0.0010) [2023-12-27 05:49:53,621][20381] Updated weights for policy 1, policy_version 23848 (0.0008) [2023-12-27 05:49:53,673][20381] Updated weights for policy 1, policy_version 23858 (0.0007) [2023-12-27 05:49:53,724][20381] Updated weights for policy 1, policy_version 23868 (0.0008) [2023-12-27 05:49:53,939][20347] Updated weights for policy 0, policy_version 23889 (0.0007) [2023-12-27 05:49:53,994][20347] Updated weights for policy 0, policy_version 23899 (0.0008) [2023-12-27 05:49:54,050][20347] Updated weights for policy 0, policy_version 23909 (0.0007) [2023-12-27 05:49:54,108][20347] Updated weights for policy 0, policy_version 23919 (0.0008) [2023-12-27 05:49:54,424][20381] Updated weights for policy 1, policy_version 23878 (0.0008) [2023-12-27 05:49:54,478][20381] Updated weights for policy 1, policy_version 23888 (0.0008) [2023-12-27 05:49:54,535][20381] Updated weights for policy 1, policy_version 23898 (0.0007) [2023-12-27 05:49:54,785][20347] Updated weights for policy 0, policy_version 23929 (0.0007) [2023-12-27 05:49:54,842][20347] Updated weights for policy 0, policy_version 23939 (0.0007) [2023-12-27 05:49:54,895][20347] Updated weights for policy 0, policy_version 23949 (0.0007) [2023-12-27 05:49:55,191][20381] Updated weights for policy 1, policy_version 23908 (0.0006) [2023-12-27 05:49:55,254][20381] Updated weights for policy 1, policy_version 23918 (0.0005) [2023-12-27 05:49:55,313][20381] Updated weights for policy 1, policy_version 23928 (0.0005) [2023-12-27 05:49:55,568][20347] Updated weights for policy 0, policy_version 23959 (0.0009) [2023-12-27 05:49:55,628][20347] Updated weights for policy 0, policy_version 23969 (0.0010) [2023-12-27 05:49:55,677][20347] Updated weights for policy 0, policy_version 23979 (0.0010) [2023-12-27 05:49:55,858][20381] Updated weights for policy 1, policy_version 23938 (0.0010) [2023-12-27 05:49:55,913][20381] Updated weights for policy 1, policy_version 23948 (0.0007) [2023-12-27 05:49:55,971][20381] Updated weights for policy 1, policy_version 23958 (0.0007) [2023-12-27 05:49:56,022][20381] Updated weights for policy 1, policy_version 23968 (0.0007) [2023-12-27 05:49:56,357][20347] Updated weights for policy 0, policy_version 23989 (0.0010) [2023-12-27 05:49:56,401][20347] Updated weights for policy 0, policy_version 23999 (0.0010) [2023-12-27 05:49:56,458][20347] Updated weights for policy 0, policy_version 24009 (0.0010) [2023-12-27 05:49:56,672][20381] Updated weights for policy 1, policy_version 23978 (0.0007) [2023-12-27 05:49:56,731][20381] Updated weights for policy 1, policy_version 23988 (0.0006) [2023-12-27 05:49:56,791][20381] Updated weights for policy 1, policy_version 23998 (0.0005) [2023-12-27 05:49:57,128][20347] Updated weights for policy 0, policy_version 24019 (0.0010) [2023-12-27 05:49:57,181][20347] Updated weights for policy 0, policy_version 24029 (0.0010) [2023-12-27 05:49:57,231][20347] Updated weights for policy 0, policy_version 24039 (0.0010) [2023-12-27 05:49:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 12304384. Throughput: 0: 10469.0, 1: 10524.8. Samples: 12292036. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-27 05:49:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:49:57,411][20381] Updated weights for policy 1, policy_version 24008 (0.0006) [2023-12-27 05:49:57,470][20381] Updated weights for policy 1, policy_version 24018 (0.0008) [2023-12-27 05:49:57,529][20381] Updated weights for policy 1, policy_version 24028 (0.0007) [2023-12-27 05:49:57,864][20347] Updated weights for policy 0, policy_version 24049 (0.0010) [2023-12-27 05:49:57,920][20347] Updated weights for policy 0, policy_version 24059 (0.0007) [2023-12-27 05:49:57,971][20347] Updated weights for policy 0, policy_version 24069 (0.0005) [2023-12-27 05:49:58,034][20347] Updated weights for policy 0, policy_version 24079 (0.0006) [2023-12-27 05:49:58,133][20381] Updated weights for policy 1, policy_version 24038 (0.0007) [2023-12-27 05:49:58,197][20381] Updated weights for policy 1, policy_version 24048 (0.0007) [2023-12-27 05:49:58,257][20381] Updated weights for policy 1, policy_version 24058 (0.0008) [2023-12-27 05:49:58,660][20347] Updated weights for policy 0, policy_version 24089 (0.0009) [2023-12-27 05:49:58,728][20347] Updated weights for policy 0, policy_version 24099 (0.0009) [2023-12-27 05:49:58,791][20347] Updated weights for policy 0, policy_version 24109 (0.0007) [2023-12-27 05:49:58,968][20381] Updated weights for policy 1, policy_version 24068 (0.0007) [2023-12-27 05:49:59,028][20381] Updated weights for policy 1, policy_version 24078 (0.0007) [2023-12-27 05:49:59,086][20381] Updated weights for policy 1, policy_version 24088 (0.0008) [2023-12-27 05:49:59,528][20347] Updated weights for policy 0, policy_version 24119 (0.0010) [2023-12-27 05:49:59,585][20347] Updated weights for policy 0, policy_version 24129 (0.0010) [2023-12-27 05:49:59,644][20347] Updated weights for policy 0, policy_version 24139 (0.0010) [2023-12-27 05:49:59,749][20381] Updated weights for policy 1, policy_version 24098 (0.0007) [2023-12-27 05:49:59,803][20381] Updated weights for policy 1, policy_version 24108 (0.0007) [2023-12-27 05:49:59,867][20381] Updated weights for policy 1, policy_version 24118 (0.0008) [2023-12-27 05:49:59,931][20381] Updated weights for policy 1, policy_version 24128 (0.0007) [2023-12-27 05:50:00,287][20347] Updated weights for policy 0, policy_version 24149 (0.0010) [2023-12-27 05:50:00,346][20347] Updated weights for policy 0, policy_version 24159 (0.0010) [2023-12-27 05:50:00,407][20347] Updated weights for policy 0, policy_version 24169 (0.0005) [2023-12-27 05:50:00,575][20381] Updated weights for policy 1, policy_version 24138 (0.0008) [2023-12-27 05:50:00,634][20381] Updated weights for policy 1, policy_version 24148 (0.0008) [2023-12-27 05:50:00,690][20381] Updated weights for policy 1, policy_version 24158 (0.0009) [2023-12-27 05:50:00,999][20347] Updated weights for policy 0, policy_version 24179 (0.0006) [2023-12-27 05:50:01,061][20347] Updated weights for policy 0, policy_version 24189 (0.0008) [2023-12-27 05:50:01,123][20347] Updated weights for policy 0, policy_version 24199 (0.0007) [2023-12-27 05:50:01,430][20381] Updated weights for policy 1, policy_version 24168 (0.0009) [2023-12-27 05:50:01,491][20381] Updated weights for policy 1, policy_version 24178 (0.0011) [2023-12-27 05:50:01,548][20381] Updated weights for policy 1, policy_version 24188 (0.0010) [2023-12-27 05:50:01,877][20347] Updated weights for policy 0, policy_version 24209 (0.0007) [2023-12-27 05:50:01,929][20347] Updated weights for policy 0, policy_version 24219 (0.0007) [2023-12-27 05:50:01,988][20347] Updated weights for policy 0, policy_version 24229 (0.0007) [2023-12-27 05:50:02,053][20347] Updated weights for policy 0, policy_version 24239 (0.0009) [2023-12-27 05:50:02,148][20381] Updated weights for policy 1, policy_version 24198 (0.0006) [2023-12-27 05:50:02,200][20381] Updated weights for policy 1, policy_version 24208 (0.0005) [2023-12-27 05:50:02,250][20381] Updated weights for policy 1, policy_version 24218 (0.0006) [2023-12-27 05:50:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 12410880. Throughput: 0: 10442.0, 1: 10491.5. Samples: 12416340. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-12-27 05:50:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:02,710][20347] Updated weights for policy 0, policy_version 24249 (0.0006) [2023-12-27 05:50:02,760][20347] Updated weights for policy 0, policy_version 24259 (0.0007) [2023-12-27 05:50:02,815][20347] Updated weights for policy 0, policy_version 24269 (0.0007) [2023-12-27 05:50:02,867][20381] Updated weights for policy 1, policy_version 24228 (0.0008) [2023-12-27 05:50:02,919][20381] Updated weights for policy 1, policy_version 24238 (0.0008) [2023-12-27 05:50:02,964][20381] Updated weights for policy 1, policy_version 24248 (0.0007) [2023-12-27 05:50:03,465][20347] Updated weights for policy 0, policy_version 24279 (0.0006) [2023-12-27 05:50:03,525][20347] Updated weights for policy 0, policy_version 24289 (0.0007) [2023-12-27 05:50:03,577][20347] Updated weights for policy 0, policy_version 24299 (0.0007) [2023-12-27 05:50:03,679][20381] Updated weights for policy 1, policy_version 24258 (0.0006) [2023-12-27 05:50:03,723][20381] Updated weights for policy 1, policy_version 24268 (0.0005) [2023-12-27 05:50:03,768][20381] Updated weights for policy 1, policy_version 24278 (0.0007) [2023-12-27 05:50:03,821][20381] Updated weights for policy 1, policy_version 24288 (0.0005) [2023-12-27 05:50:04,236][20347] Updated weights for policy 0, policy_version 24309 (0.0007) [2023-12-27 05:50:04,291][20347] Updated weights for policy 0, policy_version 24319 (0.0008) [2023-12-27 05:50:04,346][20347] Updated weights for policy 0, policy_version 24329 (0.0008) [2023-12-27 05:50:04,442][20381] Updated weights for policy 1, policy_version 24298 (0.0005) [2023-12-27 05:50:04,504][20381] Updated weights for policy 1, policy_version 24308 (0.0007) [2023-12-27 05:50:04,565][20381] Updated weights for policy 1, policy_version 24318 (0.0006) [2023-12-27 05:50:05,078][20347] Updated weights for policy 0, policy_version 24339 (0.0009) [2023-12-27 05:50:05,135][20347] Updated weights for policy 0, policy_version 24349 (0.0010) [2023-12-27 05:50:05,177][20381] Updated weights for policy 1, policy_version 24328 (0.0005) [2023-12-27 05:50:05,190][20347] Updated weights for policy 0, policy_version 24359 (0.0010) [2023-12-27 05:50:05,227][20381] Updated weights for policy 1, policy_version 24338 (0.0008) [2023-12-27 05:50:05,278][20381] Updated weights for policy 1, policy_version 24348 (0.0007) [2023-12-27 05:50:05,764][20347] Updated weights for policy 0, policy_version 24369 (0.0010) [2023-12-27 05:50:05,815][20347] Updated weights for policy 0, policy_version 24379 (0.0005) [2023-12-27 05:50:05,876][20347] Updated weights for policy 0, policy_version 24389 (0.0005) [2023-12-27 05:50:05,932][20381] Updated weights for policy 1, policy_version 24358 (0.0007) [2023-12-27 05:50:05,934][20347] Updated weights for policy 0, policy_version 24399 (0.0006) [2023-12-27 05:50:05,977][20381] Updated weights for policy 1, policy_version 24368 (0.0007) [2023-12-27 05:50:06,023][20381] Updated weights for policy 1, policy_version 24378 (0.0007) [2023-12-27 05:50:06,518][20347] Updated weights for policy 0, policy_version 24409 (0.0006) [2023-12-27 05:50:06,579][20347] Updated weights for policy 0, policy_version 24419 (0.0006) [2023-12-27 05:50:06,637][20347] Updated weights for policy 0, policy_version 24429 (0.0007) [2023-12-27 05:50:06,721][20381] Updated weights for policy 1, policy_version 24388 (0.0007) [2023-12-27 05:50:06,786][20381] Updated weights for policy 1, policy_version 24398 (0.0009) [2023-12-27 05:50:06,856][20381] Updated weights for policy 1, policy_version 24408 (0.0009) [2023-12-27 05:50:07,276][20347] Updated weights for policy 0, policy_version 24439 (0.0006) [2023-12-27 05:50:07,330][20347] Updated weights for policy 0, policy_version 24449 (0.0005) [2023-12-27 05:50:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.7, 300 sec: 20743.8). Total num frames: 12509184. Throughput: 0: 10436.5, 1: 10541.7. Samples: 12479960. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 05:50:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000024416_6250496.pth... [2023-12-27 05:50:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000023200_5939200.pth [2023-12-27 05:50:07,382][20347] Updated weights for policy 0, policy_version 24459 (0.0007) [2023-12-27 05:50:07,404][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000024464_6266880.pth... [2023-12-27 05:50:07,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000023232_5947392.pth [2023-12-27 05:50:07,517][20381] Updated weights for policy 1, policy_version 24418 (0.0008) [2023-12-27 05:50:07,569][20381] Updated weights for policy 1, policy_version 24428 (0.0007) [2023-12-27 05:50:07,618][20381] Updated weights for policy 1, policy_version 24438 (0.0008) [2023-12-27 05:50:07,678][20381] Updated weights for policy 1, policy_version 24448 (0.0007) [2023-12-27 05:50:08,072][20347] Updated weights for policy 0, policy_version 24469 (0.0008) [2023-12-27 05:50:08,119][20347] Updated weights for policy 0, policy_version 24479 (0.0007) [2023-12-27 05:50:08,170][20347] Updated weights for policy 0, policy_version 24489 (0.0007) [2023-12-27 05:50:08,271][20381] Updated weights for policy 1, policy_version 24458 (0.0008) [2023-12-27 05:50:08,331][20381] Updated weights for policy 1, policy_version 24468 (0.0007) [2023-12-27 05:50:08,391][20381] Updated weights for policy 1, policy_version 24478 (0.0008) [2023-12-27 05:50:08,893][20347] Updated weights for policy 0, policy_version 24499 (0.0007) [2023-12-27 05:50:08,938][20347] Updated weights for policy 0, policy_version 24509 (0.0007) [2023-12-27 05:50:08,996][20347] Updated weights for policy 0, policy_version 24519 (0.0008) [2023-12-27 05:50:09,037][20381] Updated weights for policy 1, policy_version 24488 (0.0006) [2023-12-27 05:50:09,091][20381] Updated weights for policy 1, policy_version 24498 (0.0008) [2023-12-27 05:50:09,141][20381] Updated weights for policy 1, policy_version 24508 (0.0008) [2023-12-27 05:50:09,754][20347] Updated weights for policy 0, policy_version 24529 (0.0007) [2023-12-27 05:50:09,807][20347] Updated weights for policy 0, policy_version 24539 (0.0007) [2023-12-27 05:50:09,821][20381] Updated weights for policy 1, policy_version 24518 (0.0007) [2023-12-27 05:50:09,874][20381] Updated weights for policy 1, policy_version 24528 (0.0006) [2023-12-27 05:50:09,875][20347] Updated weights for policy 0, policy_version 24549 (0.0007) [2023-12-27 05:50:09,937][20381] Updated weights for policy 1, policy_version 24538 (0.0007) [2023-12-27 05:50:09,937][20347] Updated weights for policy 0, policy_version 24559 (0.0008) [2023-12-27 05:50:10,640][20381] Updated weights for policy 1, policy_version 24548 (0.0009) [2023-12-27 05:50:10,641][20347] Updated weights for policy 0, policy_version 24569 (0.0006) [2023-12-27 05:50:10,689][20347] Updated weights for policy 0, policy_version 24579 (0.0005) [2023-12-27 05:50:10,691][20381] Updated weights for policy 1, policy_version 24558 (0.0010) [2023-12-27 05:50:10,738][20347] Updated weights for policy 0, policy_version 24589 (0.0005) [2023-12-27 05:50:10,744][20381] Updated weights for policy 1, policy_version 24568 (0.0011) [2023-12-27 05:50:11,490][20381] Updated weights for policy 1, policy_version 24578 (0.0009) [2023-12-27 05:50:11,531][20347] Updated weights for policy 0, policy_version 24599 (0.0006) [2023-12-27 05:50:11,556][20381] Updated weights for policy 1, policy_version 24588 (0.0006) [2023-12-27 05:50:11,590][20347] Updated weights for policy 0, policy_version 24609 (0.0007) [2023-12-27 05:50:11,624][20381] Updated weights for policy 1, policy_version 24598 (0.0008) [2023-12-27 05:50:11,656][20347] Updated weights for policy 0, policy_version 24619 (0.0008) [2023-12-27 05:50:11,679][20381] Updated weights for policy 1, policy_version 24608 (0.0008) [2023-12-27 05:50:12,355][20347] Updated weights for policy 0, policy_version 24629 (0.0008) [2023-12-27 05:50:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 12607488. Throughput: 0: 10430.2, 1: 10533.0. Samples: 12604224. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 05:50:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:12,377][20381] Updated weights for policy 1, policy_version 24618 (0.0007) [2023-12-27 05:50:12,403][20347] Updated weights for policy 0, policy_version 24639 (0.0008) [2023-12-27 05:50:12,441][20381] Updated weights for policy 1, policy_version 24628 (0.0006) [2023-12-27 05:50:12,458][20347] Updated weights for policy 0, policy_version 24649 (0.0007) [2023-12-27 05:50:12,495][20381] Updated weights for policy 1, policy_version 24638 (0.0007) [2023-12-27 05:50:13,118][20381] Updated weights for policy 1, policy_version 24648 (0.0009) [2023-12-27 05:50:13,162][20347] Updated weights for policy 0, policy_version 24659 (0.0005) [2023-12-27 05:50:13,169][20381] Updated weights for policy 1, policy_version 24658 (0.0007) [2023-12-27 05:50:13,214][20347] Updated weights for policy 0, policy_version 24669 (0.0006) [2023-12-27 05:50:13,224][20381] Updated weights for policy 1, policy_version 24668 (0.0007) [2023-12-27 05:50:13,260][20347] Updated weights for policy 0, policy_version 24679 (0.0005) [2023-12-27 05:50:13,842][20381] Updated weights for policy 1, policy_version 24678 (0.0006) [2023-12-27 05:50:13,885][20381] Updated weights for policy 1, policy_version 24688 (0.0005) [2023-12-27 05:50:13,932][20381] Updated weights for policy 1, policy_version 24698 (0.0006) [2023-12-27 05:50:13,949][20347] Updated weights for policy 0, policy_version 24689 (0.0006) [2023-12-27 05:50:14,004][20347] Updated weights for policy 0, policy_version 24699 (0.0008) [2023-12-27 05:50:14,059][20347] Updated weights for policy 0, policy_version 24709 (0.0008) [2023-12-27 05:50:14,117][20347] Updated weights for policy 0, policy_version 24719 (0.0007) [2023-12-27 05:50:14,565][20381] Updated weights for policy 1, policy_version 24708 (0.0007) [2023-12-27 05:50:14,613][20381] Updated weights for policy 1, policy_version 24718 (0.0010) [2023-12-27 05:50:14,664][20381] Updated weights for policy 1, policy_version 24728 (0.0010) [2023-12-27 05:50:14,753][20347] Updated weights for policy 0, policy_version 24729 (0.0006) [2023-12-27 05:50:14,817][20347] Updated weights for policy 0, policy_version 24739 (0.0008) [2023-12-27 05:50:14,876][20347] Updated weights for policy 0, policy_version 24749 (0.0008) [2023-12-27 05:50:15,339][20381] Updated weights for policy 1, policy_version 24738 (0.0007) [2023-12-27 05:50:15,391][20381] Updated weights for policy 1, policy_version 24748 (0.0008) [2023-12-27 05:50:15,444][20381] Updated weights for policy 1, policy_version 24758 (0.0009) [2023-12-27 05:50:15,504][20381] Updated weights for policy 1, policy_version 24768 (0.0010) [2023-12-27 05:50:15,593][20347] Updated weights for policy 0, policy_version 24759 (0.0008) [2023-12-27 05:50:15,651][20347] Updated weights for policy 0, policy_version 24769 (0.0008) [2023-12-27 05:50:15,715][20347] Updated weights for policy 0, policy_version 24779 (0.0007) [2023-12-27 05:50:16,178][20381] Updated weights for policy 1, policy_version 24778 (0.0011) [2023-12-27 05:50:16,235][20381] Updated weights for policy 1, policy_version 24788 (0.0010) [2023-12-27 05:50:16,280][20381] Updated weights for policy 1, policy_version 24798 (0.0010) [2023-12-27 05:50:16,303][20347] Updated weights for policy 0, policy_version 24789 (0.0005) [2023-12-27 05:50:16,351][20347] Updated weights for policy 0, policy_version 24799 (0.0006) [2023-12-27 05:50:16,404][20347] Updated weights for policy 0, policy_version 24809 (0.0008) [2023-12-27 05:50:16,918][20381] Updated weights for policy 1, policy_version 24808 (0.0010) [2023-12-27 05:50:16,973][20381] Updated weights for policy 1, policy_version 24818 (0.0010) [2023-12-27 05:50:17,034][20381] Updated weights for policy 1, policy_version 24828 (0.0011) [2023-12-27 05:50:17,087][20347] Updated weights for policy 0, policy_version 24819 (0.0008) [2023-12-27 05:50:17,141][20347] Updated weights for policy 0, policy_version 24829 (0.0010) [2023-12-27 05:50:17,194][20347] Updated weights for policy 0, policy_version 24839 (0.0010) [2023-12-27 05:50:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 12722176. Throughput: 0: 10399.5, 1: 10516.1. Samples: 12728952. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 05:50:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:17,673][20381] Updated weights for policy 1, policy_version 24838 (0.0010) [2023-12-27 05:50:17,723][20381] Updated weights for policy 1, policy_version 24848 (0.0008) [2023-12-27 05:50:17,758][20347] Updated weights for policy 0, policy_version 24849 (0.0010) [2023-12-27 05:50:17,778][20381] Updated weights for policy 1, policy_version 24858 (0.0007) [2023-12-27 05:50:17,810][20347] Updated weights for policy 0, policy_version 24859 (0.0010) [2023-12-27 05:50:17,861][20347] Updated weights for policy 0, policy_version 24869 (0.0010) [2023-12-27 05:50:17,915][20347] Updated weights for policy 0, policy_version 24879 (0.0010) [2023-12-27 05:50:18,469][20381] Updated weights for policy 1, policy_version 24868 (0.0007) [2023-12-27 05:50:18,525][20381] Updated weights for policy 1, policy_version 24878 (0.0011) [2023-12-27 05:50:18,584][20381] Updated weights for policy 1, policy_version 24888 (0.0011) [2023-12-27 05:50:18,611][20347] Updated weights for policy 0, policy_version 24889 (0.0010) [2023-12-27 05:50:18,667][20347] Updated weights for policy 0, policy_version 24899 (0.0010) [2023-12-27 05:50:18,725][20347] Updated weights for policy 0, policy_version 24909 (0.0011) [2023-12-27 05:50:19,243][20381] Updated weights for policy 1, policy_version 24898 (0.0011) [2023-12-27 05:50:19,300][20381] Updated weights for policy 1, policy_version 24908 (0.0010) [2023-12-27 05:50:19,368][20381] Updated weights for policy 1, policy_version 24918 (0.0009) [2023-12-27 05:50:19,412][20347] Updated weights for policy 0, policy_version 24919 (0.0010) [2023-12-27 05:50:19,421][20381] Updated weights for policy 1, policy_version 24928 (0.0006) [2023-12-27 05:50:19,474][20347] Updated weights for policy 0, policy_version 24929 (0.0010) [2023-12-27 05:50:19,536][20347] Updated weights for policy 0, policy_version 24939 (0.0010) [2023-12-27 05:50:20,128][20381] Updated weights for policy 1, policy_version 24938 (0.0007) [2023-12-27 05:50:20,192][20381] Updated weights for policy 1, policy_version 24948 (0.0007) [2023-12-27 05:50:20,245][20347] Updated weights for policy 0, policy_version 24949 (0.0011) [2023-12-27 05:50:20,255][20381] Updated weights for policy 1, policy_version 24958 (0.0006) [2023-12-27 05:50:20,308][20347] Updated weights for policy 0, policy_version 24959 (0.0010) [2023-12-27 05:50:20,362][20347] Updated weights for policy 0, policy_version 24969 (0.0008) [2023-12-27 05:50:20,961][20381] Updated weights for policy 1, policy_version 24968 (0.0008) [2023-12-27 05:50:21,020][20381] Updated weights for policy 1, policy_version 24978 (0.0008) [2023-12-27 05:50:21,052][20347] Updated weights for policy 0, policy_version 24979 (0.0008) [2023-12-27 05:50:21,084][20381] Updated weights for policy 1, policy_version 24988 (0.0006) [2023-12-27 05:50:21,116][20347] Updated weights for policy 0, policy_version 24989 (0.0008) [2023-12-27 05:50:21,175][20347] Updated weights for policy 0, policy_version 24999 (0.0009) [2023-12-27 05:50:21,838][20381] Updated weights for policy 1, policy_version 24998 (0.0007) [2023-12-27 05:50:21,869][20347] Updated weights for policy 0, policy_version 25009 (0.0008) [2023-12-27 05:50:21,893][20381] Updated weights for policy 1, policy_version 25008 (0.0007) [2023-12-27 05:50:21,918][20347] Updated weights for policy 0, policy_version 25019 (0.0006) [2023-12-27 05:50:21,951][20381] Updated weights for policy 1, policy_version 25018 (0.0007) [2023-12-27 05:50:21,978][20347] Updated weights for policy 0, policy_version 25029 (0.0006) [2023-12-27 05:50:22,040][20347] Updated weights for policy 0, policy_version 25039 (0.0006) [2023-12-27 05:50:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 12820480. Throughput: 0: 10387.8, 1: 10424.0. Samples: 12790964. Policy #0 lag: (min: 1.0, avg: 23.3, max: 33.0) [2023-12-27 05:50:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000025024_6406144.pth... [2023-12-27 05:50:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000025040_6414336.pth... [2023-12-27 05:50:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000023808_6094848.pth [2023-12-27 05:50:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000023824_6103040.pth [2023-12-27 05:50:22,663][20347] Updated weights for policy 0, policy_version 25049 (0.0007) [2023-12-27 05:50:22,710][20381] Updated weights for policy 1, policy_version 25028 (0.0008) [2023-12-27 05:50:22,717][20347] Updated weights for policy 0, policy_version 25059 (0.0007) [2023-12-27 05:50:22,768][20381] Updated weights for policy 1, policy_version 25038 (0.0008) [2023-12-27 05:50:22,776][20347] Updated weights for policy 0, policy_version 25069 (0.0005) [2023-12-27 05:50:22,834][20381] Updated weights for policy 1, policy_version 25048 (0.0009) [2023-12-27 05:50:23,408][20347] Updated weights for policy 0, policy_version 25079 (0.0007) [2023-12-27 05:50:23,452][20347] Updated weights for policy 0, policy_version 25089 (0.0007) [2023-12-27 05:50:23,504][20347] Updated weights for policy 0, policy_version 25099 (0.0008) [2023-12-27 05:50:23,545][20381] Updated weights for policy 1, policy_version 25058 (0.0007) [2023-12-27 05:50:23,604][20381] Updated weights for policy 1, policy_version 25068 (0.0008) [2023-12-27 05:50:23,677][20381] Updated weights for policy 1, policy_version 25078 (0.0007) [2023-12-27 05:50:23,727][20381] Updated weights for policy 1, policy_version 25088 (0.0007) [2023-12-27 05:50:24,139][20347] Updated weights for policy 0, policy_version 25109 (0.0006) [2023-12-27 05:50:24,191][20347] Updated weights for policy 0, policy_version 25119 (0.0005) [2023-12-27 05:50:24,252][20347] Updated weights for policy 0, policy_version 25129 (0.0008) [2023-12-27 05:50:24,394][20381] Updated weights for policy 1, policy_version 25098 (0.0006) [2023-12-27 05:50:24,450][20381] Updated weights for policy 1, policy_version 25108 (0.0007) [2023-12-27 05:50:24,514][20381] Updated weights for policy 1, policy_version 25118 (0.0007) [2023-12-27 05:50:24,830][20347] Updated weights for policy 0, policy_version 25139 (0.0009) [2023-12-27 05:50:24,891][20347] Updated weights for policy 0, policy_version 25149 (0.0009) [2023-12-27 05:50:24,946][20347] Updated weights for policy 0, policy_version 25159 (0.0010) [2023-12-27 05:50:25,224][20381] Updated weights for policy 1, policy_version 25128 (0.0007) [2023-12-27 05:50:25,278][20381] Updated weights for policy 1, policy_version 25138 (0.0007) [2023-12-27 05:50:25,341][20381] Updated weights for policy 1, policy_version 25148 (0.0007) [2023-12-27 05:50:25,589][20347] Updated weights for policy 0, policy_version 25169 (0.0010) [2023-12-27 05:50:25,645][20347] Updated weights for policy 0, policy_version 25179 (0.0005) [2023-12-27 05:50:25,694][20347] Updated weights for policy 0, policy_version 25189 (0.0005) [2023-12-27 05:50:25,745][20347] Updated weights for policy 0, policy_version 25199 (0.0010) [2023-12-27 05:50:25,970][20381] Updated weights for policy 1, policy_version 25158 (0.0008) [2023-12-27 05:50:26,025][20381] Updated weights for policy 1, policy_version 25168 (0.0007) [2023-12-27 05:50:26,073][20381] Updated weights for policy 1, policy_version 25178 (0.0006) [2023-12-27 05:50:26,414][20347] Updated weights for policy 0, policy_version 25209 (0.0008) [2023-12-27 05:50:26,461][20347] Updated weights for policy 0, policy_version 25219 (0.0007) [2023-12-27 05:50:26,505][20347] Updated weights for policy 0, policy_version 25229 (0.0010) [2023-12-27 05:50:26,744][20381] Updated weights for policy 1, policy_version 25188 (0.0007) [2023-12-27 05:50:26,803][20381] Updated weights for policy 1, policy_version 25198 (0.0005) [2023-12-27 05:50:26,853][20381] Updated weights for policy 1, policy_version 25208 (0.0005) [2023-12-27 05:50:27,188][20347] Updated weights for policy 0, policy_version 25239 (0.0010) [2023-12-27 05:50:27,235][20347] Updated weights for policy 0, policy_version 25249 (0.0010) [2023-12-27 05:50:27,286][20347] Updated weights for policy 0, policy_version 25259 (0.0010) [2023-12-27 05:50:27,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 12926976. Throughput: 0: 10407.3, 1: 10343.9. Samples: 12913756. Policy #0 lag: (min: 1.0, avg: 23.3, max: 33.0) [2023-12-27 05:50:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:27,437][20381] Updated weights for policy 1, policy_version 25218 (0.0010) [2023-12-27 05:50:27,487][20381] Updated weights for policy 1, policy_version 25228 (0.0010) [2023-12-27 05:50:27,544][20381] Updated weights for policy 1, policy_version 25238 (0.0010) [2023-12-27 05:50:27,588][20381] Updated weights for policy 1, policy_version 25248 (0.0010) [2023-12-27 05:50:27,919][20347] Updated weights for policy 0, policy_version 25269 (0.0010) [2023-12-27 05:50:27,963][20347] Updated weights for policy 0, policy_version 25279 (0.0010) [2023-12-27 05:50:28,012][20347] Updated weights for policy 0, policy_version 25289 (0.0010) [2023-12-27 05:50:28,191][20381] Updated weights for policy 1, policy_version 25258 (0.0007) [2023-12-27 05:50:28,239][20381] Updated weights for policy 1, policy_version 25268 (0.0007) [2023-12-27 05:50:28,304][20381] Updated weights for policy 1, policy_version 25278 (0.0008) [2023-12-27 05:50:28,712][20347] Updated weights for policy 0, policy_version 25299 (0.0010) [2023-12-27 05:50:28,765][20347] Updated weights for policy 0, policy_version 25309 (0.0010) [2023-12-27 05:50:28,823][20347] Updated weights for policy 0, policy_version 25319 (0.0008) [2023-12-27 05:50:28,892][20381] Updated weights for policy 1, policy_version 25288 (0.0008) [2023-12-27 05:50:28,949][20381] Updated weights for policy 1, policy_version 25298 (0.0007) [2023-12-27 05:50:29,000][20381] Updated weights for policy 1, policy_version 25308 (0.0007) [2023-12-27 05:50:29,467][20347] Updated weights for policy 0, policy_version 25329 (0.0008) [2023-12-27 05:50:29,515][20347] Updated weights for policy 0, policy_version 25339 (0.0010) [2023-12-27 05:50:29,570][20347] Updated weights for policy 0, policy_version 25349 (0.0010) [2023-12-27 05:50:29,603][20381] Updated weights for policy 1, policy_version 25318 (0.0007) [2023-12-27 05:50:29,624][20347] Updated weights for policy 0, policy_version 25359 (0.0010) [2023-12-27 05:50:29,653][20381] Updated weights for policy 1, policy_version 25328 (0.0006) [2023-12-27 05:50:29,702][20381] Updated weights for policy 1, policy_version 25338 (0.0007) [2023-12-27 05:50:30,269][20347] Updated weights for policy 0, policy_version 25369 (0.0006) [2023-12-27 05:50:30,328][20347] Updated weights for policy 0, policy_version 25379 (0.0007) [2023-12-27 05:50:30,339][20381] Updated weights for policy 1, policy_version 25348 (0.0007) [2023-12-27 05:50:30,389][20347] Updated weights for policy 0, policy_version 25389 (0.0007) [2023-12-27 05:50:30,396][20381] Updated weights for policy 1, policy_version 25358 (0.0008) [2023-12-27 05:50:30,455][20381] Updated weights for policy 1, policy_version 25368 (0.0008) [2023-12-27 05:50:30,991][20347] Updated weights for policy 0, policy_version 25399 (0.0009) [2023-12-27 05:50:31,051][20347] Updated weights for policy 0, policy_version 25409 (0.0010) [2023-12-27 05:50:31,100][20381] Updated weights for policy 1, policy_version 25378 (0.0005) [2023-12-27 05:50:31,105][20347] Updated weights for policy 0, policy_version 25419 (0.0009) [2023-12-27 05:50:31,164][20381] Updated weights for policy 1, policy_version 25388 (0.0007) [2023-12-27 05:50:31,218][20381] Updated weights for policy 1, policy_version 25398 (0.0005) [2023-12-27 05:50:31,281][20381] Updated weights for policy 1, policy_version 25408 (0.0007) [2023-12-27 05:50:31,780][20347] Updated weights for policy 0, policy_version 25429 (0.0007) [2023-12-27 05:50:31,838][20347] Updated weights for policy 0, policy_version 25439 (0.0007) [2023-12-27 05:50:31,896][20347] Updated weights for policy 0, policy_version 25449 (0.0007) [2023-12-27 05:50:31,968][20381] Updated weights for policy 1, policy_version 25418 (0.0007) [2023-12-27 05:50:32,031][20381] Updated weights for policy 1, policy_version 25428 (0.0008) [2023-12-27 05:50:32,092][20381] Updated weights for policy 1, policy_version 25438 (0.0007) [2023-12-27 05:50:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 13033472. Throughput: 0: 10375.0, 1: 10463.0. Samples: 13042468. Policy #0 lag: (min: 1.0, avg: 23.3, max: 33.0) [2023-12-27 05:50:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:32,589][20347] Updated weights for policy 0, policy_version 25459 (0.0008) [2023-12-27 05:50:32,641][20347] Updated weights for policy 0, policy_version 25469 (0.0009) [2023-12-27 05:50:32,705][20347] Updated weights for policy 0, policy_version 25479 (0.0010) [2023-12-27 05:50:32,833][20381] Updated weights for policy 1, policy_version 25448 (0.0007) [2023-12-27 05:50:32,888][20381] Updated weights for policy 1, policy_version 25458 (0.0008) [2023-12-27 05:50:32,948][20381] Updated weights for policy 1, policy_version 25468 (0.0008) [2023-12-27 05:50:33,359][20347] Updated weights for policy 0, policy_version 25489 (0.0010) [2023-12-27 05:50:33,416][20347] Updated weights for policy 0, policy_version 25499 (0.0010) [2023-12-27 05:50:33,476][20347] Updated weights for policy 0, policy_version 25509 (0.0010) [2023-12-27 05:50:33,532][20347] Updated weights for policy 0, policy_version 25519 (0.0010) [2023-12-27 05:50:33,635][20381] Updated weights for policy 1, policy_version 25478 (0.0007) [2023-12-27 05:50:33,690][20381] Updated weights for policy 1, policy_version 25488 (0.0006) [2023-12-27 05:50:33,748][20381] Updated weights for policy 1, policy_version 25498 (0.0007) [2023-12-27 05:50:34,199][20347] Updated weights for policy 0, policy_version 25529 (0.0010) [2023-12-27 05:50:34,258][20347] Updated weights for policy 0, policy_version 25539 (0.0010) [2023-12-27 05:50:34,319][20347] Updated weights for policy 0, policy_version 25549 (0.0010) [2023-12-27 05:50:34,444][20381] Updated weights for policy 1, policy_version 25508 (0.0007) [2023-12-27 05:50:34,489][20381] Updated weights for policy 1, policy_version 25518 (0.0007) [2023-12-27 05:50:34,541][20381] Updated weights for policy 1, policy_version 25528 (0.0007) [2023-12-27 05:50:34,999][20347] Updated weights for policy 0, policy_version 25559 (0.0010) [2023-12-27 05:50:35,050][20347] Updated weights for policy 0, policy_version 25569 (0.0009) [2023-12-27 05:50:35,104][20347] Updated weights for policy 0, policy_version 25579 (0.0010) [2023-12-27 05:50:35,191][20381] Updated weights for policy 1, policy_version 25538 (0.0007) [2023-12-27 05:50:35,249][20381] Updated weights for policy 1, policy_version 25548 (0.0007) [2023-12-27 05:50:35,301][20381] Updated weights for policy 1, policy_version 25558 (0.0007) [2023-12-27 05:50:35,349][20381] Updated weights for policy 1, policy_version 25568 (0.0007) [2023-12-27 05:50:35,776][20347] Updated weights for policy 0, policy_version 25589 (0.0010) [2023-12-27 05:50:35,841][20347] Updated weights for policy 0, policy_version 25599 (0.0010) [2023-12-27 05:50:35,901][20347] Updated weights for policy 0, policy_version 25609 (0.0010) [2023-12-27 05:50:35,969][20381] Updated weights for policy 1, policy_version 25578 (0.0005) [2023-12-27 05:50:36,022][20381] Updated weights for policy 1, policy_version 25588 (0.0006) [2023-12-27 05:50:36,077][20381] Updated weights for policy 1, policy_version 25598 (0.0006) [2023-12-27 05:50:36,510][20347] Updated weights for policy 0, policy_version 25619 (0.0009) [2023-12-27 05:50:36,570][20347] Updated weights for policy 0, policy_version 25629 (0.0008) [2023-12-27 05:50:36,621][20347] Updated weights for policy 0, policy_version 25639 (0.0010) [2023-12-27 05:50:36,662][20381] Updated weights for policy 1, policy_version 25608 (0.0006) [2023-12-27 05:50:36,722][20381] Updated weights for policy 1, policy_version 25618 (0.0007) [2023-12-27 05:50:36,770][20381] Updated weights for policy 1, policy_version 25628 (0.0008) [2023-12-27 05:50:37,265][20347] Updated weights for policy 0, policy_version 25649 (0.0010) [2023-12-27 05:50:37,317][20347] Updated weights for policy 0, policy_version 25659 (0.0010) [2023-12-27 05:50:37,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 13131776. Throughput: 0: 10378.6, 1: 10444.0. Samples: 13104068. Policy #0 lag: (min: 1.0, avg: 23.3, max: 33.0) [2023-12-27 05:50:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000025632_6561792.pth... [2023-12-27 05:50:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000024416_6250496.pth [2023-12-27 05:50:37,373][20347] Updated weights for policy 0, policy_version 25669 (0.0010) [2023-12-27 05:50:37,436][20347] Updated weights for policy 0, policy_version 25679 (0.0011) [2023-12-27 05:50:37,441][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000025680_6578176.pth... [2023-12-27 05:50:37,446][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000024464_6266880.pth [2023-12-27 05:50:37,509][20381] Updated weights for policy 1, policy_version 25638 (0.0009) [2023-12-27 05:50:37,570][20381] Updated weights for policy 1, policy_version 25648 (0.0011) [2023-12-27 05:50:37,626][20381] Updated weights for policy 1, policy_version 25658 (0.0010) [2023-12-27 05:50:38,106][20347] Updated weights for policy 0, policy_version 25689 (0.0008) [2023-12-27 05:50:38,159][20347] Updated weights for policy 0, policy_version 25699 (0.0005) [2023-12-27 05:50:38,216][20347] Updated weights for policy 0, policy_version 25709 (0.0005) [2023-12-27 05:50:38,290][20381] Updated weights for policy 1, policy_version 25668 (0.0010) [2023-12-27 05:50:38,355][20381] Updated weights for policy 1, policy_version 25678 (0.0009) [2023-12-27 05:50:38,415][20381] Updated weights for policy 1, policy_version 25688 (0.0008) [2023-12-27 05:50:38,868][20347] Updated weights for policy 0, policy_version 25719 (0.0008) [2023-12-27 05:50:38,934][20347] Updated weights for policy 0, policy_version 25729 (0.0008) [2023-12-27 05:50:38,973][20381] Updated weights for policy 1, policy_version 25698 (0.0005) [2023-12-27 05:50:39,000][20347] Updated weights for policy 0, policy_version 25739 (0.0008) [2023-12-27 05:50:39,033][20381] Updated weights for policy 1, policy_version 25708 (0.0009) [2023-12-27 05:50:39,085][20381] Updated weights for policy 1, policy_version 25718 (0.0010) [2023-12-27 05:50:39,147][20381] Updated weights for policy 1, policy_version 25728 (0.0011) [2023-12-27 05:50:39,676][20347] Updated weights for policy 0, policy_version 25749 (0.0008) [2023-12-27 05:50:39,736][20347] Updated weights for policy 0, policy_version 25759 (0.0008) [2023-12-27 05:50:39,795][20347] Updated weights for policy 0, policy_version 25769 (0.0008) [2023-12-27 05:50:39,859][20381] Updated weights for policy 1, policy_version 25738 (0.0007) [2023-12-27 05:50:39,915][20381] Updated weights for policy 1, policy_version 25748 (0.0008) [2023-12-27 05:50:39,975][20381] Updated weights for policy 1, policy_version 25758 (0.0007) [2023-12-27 05:50:40,508][20347] Updated weights for policy 0, policy_version 25779 (0.0010) [2023-12-27 05:50:40,566][20347] Updated weights for policy 0, policy_version 25789 (0.0008) [2023-12-27 05:50:40,627][20347] Updated weights for policy 0, policy_version 25799 (0.0008) [2023-12-27 05:50:40,680][20381] Updated weights for policy 1, policy_version 25768 (0.0006) [2023-12-27 05:50:40,739][20381] Updated weights for policy 1, policy_version 25778 (0.0007) [2023-12-27 05:50:40,803][20381] Updated weights for policy 1, policy_version 25788 (0.0008) [2023-12-27 05:50:41,355][20347] Updated weights for policy 0, policy_version 25809 (0.0007) [2023-12-27 05:50:41,419][20347] Updated weights for policy 0, policy_version 25819 (0.0008) [2023-12-27 05:50:41,481][20347] Updated weights for policy 0, policy_version 25829 (0.0008) [2023-12-27 05:50:41,535][20381] Updated weights for policy 1, policy_version 25798 (0.0007) [2023-12-27 05:50:41,536][20347] Updated weights for policy 0, policy_version 25839 (0.0008) [2023-12-27 05:50:41,594][20381] Updated weights for policy 1, policy_version 25808 (0.0008) [2023-12-27 05:50:41,661][20381] Updated weights for policy 1, policy_version 25818 (0.0008) [2023-12-27 05:50:42,224][20347] Updated weights for policy 0, policy_version 25849 (0.0010) [2023-12-27 05:50:42,284][20347] Updated weights for policy 0, policy_version 25859 (0.0011) [2023-12-27 05:50:42,344][20381] Updated weights for policy 1, policy_version 25828 (0.0007) [2023-12-27 05:50:42,347][20347] Updated weights for policy 0, policy_version 25869 (0.0010) [2023-12-27 05:50:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 13230080. Throughput: 0: 10390.5, 1: 10422.8. Samples: 13228632. Policy #0 lag: (min: 17.0, avg: 40.6, max: 49.0) [2023-12-27 05:50:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:42,404][20381] Updated weights for policy 1, policy_version 25838 (0.0007) [2023-12-27 05:50:42,449][20381] Updated weights for policy 1, policy_version 25848 (0.0007) [2023-12-27 05:50:43,047][20347] Updated weights for policy 0, policy_version 25879 (0.0007) [2023-12-27 05:50:43,090][20381] Updated weights for policy 1, policy_version 25858 (0.0007) [2023-12-27 05:50:43,098][20347] Updated weights for policy 0, policy_version 25889 (0.0006) [2023-12-27 05:50:43,144][20381] Updated weights for policy 1, policy_version 25868 (0.0010) [2023-12-27 05:50:43,148][20347] Updated weights for policy 0, policy_version 25899 (0.0005) [2023-12-27 05:50:43,197][20381] Updated weights for policy 1, policy_version 25878 (0.0010) [2023-12-27 05:50:43,246][20381] Updated weights for policy 1, policy_version 25888 (0.0008) [2023-12-27 05:50:43,728][20347] Updated weights for policy 0, policy_version 25909 (0.0006) [2023-12-27 05:50:43,774][20347] Updated weights for policy 0, policy_version 25919 (0.0007) [2023-12-27 05:50:43,820][20347] Updated weights for policy 0, policy_version 25929 (0.0008) [2023-12-27 05:50:43,877][20381] Updated weights for policy 1, policy_version 25898 (0.0008) [2023-12-27 05:50:43,930][20381] Updated weights for policy 1, policy_version 25908 (0.0007) [2023-12-27 05:50:43,981][20381] Updated weights for policy 1, policy_version 25918 (0.0006) [2023-12-27 05:50:44,469][20347] Updated weights for policy 0, policy_version 25939 (0.0006) [2023-12-27 05:50:44,522][20347] Updated weights for policy 0, policy_version 25949 (0.0006) [2023-12-27 05:50:44,555][20381] Updated weights for policy 1, policy_version 25928 (0.0009) [2023-12-27 05:50:44,574][20347] Updated weights for policy 0, policy_version 25959 (0.0007) [2023-12-27 05:50:44,607][20381] Updated weights for policy 1, policy_version 25938 (0.0010) [2023-12-27 05:50:44,657][20381] Updated weights for policy 1, policy_version 25948 (0.0010) [2023-12-27 05:50:45,199][20347] Updated weights for policy 0, policy_version 25969 (0.0009) [2023-12-27 05:50:45,262][20347] Updated weights for policy 0, policy_version 25979 (0.0006) [2023-12-27 05:50:45,327][20347] Updated weights for policy 0, policy_version 25989 (0.0009) [2023-12-27 05:50:45,335][20381] Updated weights for policy 1, policy_version 25958 (0.0009) [2023-12-27 05:50:45,383][20347] Updated weights for policy 0, policy_version 25999 (0.0010) [2023-12-27 05:50:45,391][20381] Updated weights for policy 1, policy_version 25968 (0.0010) [2023-12-27 05:50:45,451][20381] Updated weights for policy 1, policy_version 25978 (0.0008) [2023-12-27 05:50:45,953][20347] Updated weights for policy 0, policy_version 26009 (0.0006) [2023-12-27 05:50:46,013][20347] Updated weights for policy 0, policy_version 26019 (0.0009) [2023-12-27 05:50:46,056][20381] Updated weights for policy 1, policy_version 25988 (0.0010) [2023-12-27 05:50:46,072][20347] Updated weights for policy 0, policy_version 26029 (0.0010) [2023-12-27 05:50:46,104][20381] Updated weights for policy 1, policy_version 25998 (0.0009) [2023-12-27 05:50:46,155][20381] Updated weights for policy 1, policy_version 26008 (0.0007) [2023-12-27 05:50:46,757][20347] Updated weights for policy 0, policy_version 26039 (0.0009) [2023-12-27 05:50:46,818][20347] Updated weights for policy 0, policy_version 26049 (0.0007) [2023-12-27 05:50:46,857][20381] Updated weights for policy 1, policy_version 26018 (0.0007) [2023-12-27 05:50:46,880][20347] Updated weights for policy 0, policy_version 26059 (0.0006) [2023-12-27 05:50:46,914][20381] Updated weights for policy 1, policy_version 26028 (0.0007) [2023-12-27 05:50:46,969][20381] Updated weights for policy 1, policy_version 26038 (0.0007) [2023-12-27 05:50:47,016][20381] Updated weights for policy 1, policy_version 26048 (0.0007) [2023-12-27 05:50:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 13344768. Throughput: 0: 10438.3, 1: 10442.8. Samples: 13355992. Policy #0 lag: (min: 17.0, avg: 40.6, max: 49.0) [2023-12-27 05:50:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:47,504][20347] Updated weights for policy 0, policy_version 26069 (0.0008) [2023-12-27 05:50:47,548][20347] Updated weights for policy 0, policy_version 26079 (0.0008) [2023-12-27 05:50:47,593][20347] Updated weights for policy 0, policy_version 26089 (0.0005) [2023-12-27 05:50:47,717][20381] Updated weights for policy 1, policy_version 26058 (0.0005) [2023-12-27 05:50:47,768][20381] Updated weights for policy 1, policy_version 26068 (0.0005) [2023-12-27 05:50:47,823][20381] Updated weights for policy 1, policy_version 26078 (0.0005) [2023-12-27 05:50:48,244][20347] Updated weights for policy 0, policy_version 26099 (0.0007) [2023-12-27 05:50:48,304][20347] Updated weights for policy 0, policy_version 26109 (0.0010) [2023-12-27 05:50:48,360][20381] Updated weights for policy 1, policy_version 26088 (0.0007) [2023-12-27 05:50:48,369][20347] Updated weights for policy 0, policy_version 26119 (0.0010) [2023-12-27 05:50:48,416][20381] Updated weights for policy 1, policy_version 26098 (0.0009) [2023-12-27 05:50:48,481][20381] Updated weights for policy 1, policy_version 26108 (0.0009) [2023-12-27 05:50:49,038][20347] Updated weights for policy 0, policy_version 26129 (0.0008) [2023-12-27 05:50:49,093][20347] Updated weights for policy 0, policy_version 26139 (0.0007) [2023-12-27 05:50:49,143][20347] Updated weights for policy 0, policy_version 26149 (0.0008) [2023-12-27 05:50:49,174][20381] Updated weights for policy 1, policy_version 26118 (0.0008) [2023-12-27 05:50:49,206][20347] Updated weights for policy 0, policy_version 26159 (0.0007) [2023-12-27 05:50:49,232][20381] Updated weights for policy 1, policy_version 26128 (0.0008) [2023-12-27 05:50:49,292][20381] Updated weights for policy 1, policy_version 26138 (0.0007) [2023-12-27 05:50:49,813][20347] Updated weights for policy 0, policy_version 26169 (0.0007) [2023-12-27 05:50:49,882][20347] Updated weights for policy 0, policy_version 26179 (0.0007) [2023-12-27 05:50:49,946][20347] Updated weights for policy 0, policy_version 26189 (0.0008) [2023-12-27 05:50:50,059][20381] Updated weights for policy 1, policy_version 26148 (0.0008) [2023-12-27 05:50:50,119][20381] Updated weights for policy 1, policy_version 26158 (0.0007) [2023-12-27 05:50:50,175][20381] Updated weights for policy 1, policy_version 26168 (0.0008) [2023-12-27 05:50:50,667][20347] Updated weights for policy 0, policy_version 26199 (0.0008) [2023-12-27 05:50:50,721][20347] Updated weights for policy 0, policy_version 26209 (0.0006) [2023-12-27 05:50:50,778][20347] Updated weights for policy 0, policy_version 26219 (0.0008) [2023-12-27 05:50:50,805][20381] Updated weights for policy 1, policy_version 26178 (0.0008) [2023-12-27 05:50:50,853][20381] Updated weights for policy 1, policy_version 26188 (0.0008) [2023-12-27 05:50:50,907][20381] Updated weights for policy 1, policy_version 26198 (0.0007) [2023-12-27 05:50:50,963][20381] Updated weights for policy 1, policy_version 26208 (0.0007) [2023-12-27 05:50:51,494][20347] Updated weights for policy 0, policy_version 26229 (0.0008) [2023-12-27 05:50:51,549][20347] Updated weights for policy 0, policy_version 26239 (0.0007) [2023-12-27 05:50:51,609][20347] Updated weights for policy 0, policy_version 26249 (0.0007) [2023-12-27 05:50:51,649][20381] Updated weights for policy 1, policy_version 26218 (0.0007) [2023-12-27 05:50:51,698][20381] Updated weights for policy 1, policy_version 26228 (0.0008) [2023-12-27 05:50:51,761][20381] Updated weights for policy 1, policy_version 26238 (0.0009) [2023-12-27 05:50:52,304][20347] Updated weights for policy 0, policy_version 26259 (0.0008) [2023-12-27 05:50:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 13443072. Throughput: 0: 10443.6, 1: 10407.3. Samples: 13418248. Policy #0 lag: (min: 17.0, avg: 40.6, max: 49.0) [2023-12-27 05:50:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000026240_6717440.pth... [2023-12-27 05:50:52,371][20347] Updated weights for policy 0, policy_version 26269 (0.0008) [2023-12-27 05:50:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000025024_6406144.pth [2023-12-27 05:50:52,422][20347] Updated weights for policy 0, policy_version 26279 (0.0007) [2023-12-27 05:50:52,457][20381] Updated weights for policy 1, policy_version 26248 (0.0010) [2023-12-27 05:50:52,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000026288_6733824.pth... [2023-12-27 05:50:52,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000025040_6414336.pth [2023-12-27 05:50:52,522][20381] Updated weights for policy 1, policy_version 26258 (0.0010) [2023-12-27 05:50:52,584][20381] Updated weights for policy 1, policy_version 26268 (0.0010) [2023-12-27 05:50:53,156][20347] Updated weights for policy 0, policy_version 26289 (0.0006) [2023-12-27 05:50:53,189][20381] Updated weights for policy 1, policy_version 26278 (0.0008) [2023-12-27 05:50:53,214][20347] Updated weights for policy 0, policy_version 26299 (0.0011) [2023-12-27 05:50:53,241][20381] Updated weights for policy 1, policy_version 26288 (0.0005) [2023-12-27 05:50:53,272][20347] Updated weights for policy 0, policy_version 26309 (0.0010) [2023-12-27 05:50:53,298][20381] Updated weights for policy 1, policy_version 26298 (0.0005) [2023-12-27 05:50:53,327][20347] Updated weights for policy 0, policy_version 26319 (0.0010) [2023-12-27 05:50:53,860][20381] Updated weights for policy 1, policy_version 26308 (0.0006) [2023-12-27 05:50:53,924][20381] Updated weights for policy 1, policy_version 26318 (0.0005) [2023-12-27 05:50:53,927][20347] Updated weights for policy 0, policy_version 26329 (0.0006) [2023-12-27 05:50:53,983][20381] Updated weights for policy 1, policy_version 26328 (0.0005) [2023-12-27 05:50:53,984][20347] Updated weights for policy 0, policy_version 26339 (0.0005) [2023-12-27 05:50:54,047][20347] Updated weights for policy 0, policy_version 26349 (0.0010) [2023-12-27 05:50:54,622][20381] Updated weights for policy 1, policy_version 26338 (0.0008) [2023-12-27 05:50:54,671][20381] Updated weights for policy 1, policy_version 26348 (0.0010) [2023-12-27 05:50:54,699][20347] Updated weights for policy 0, policy_version 26359 (0.0008) [2023-12-27 05:50:54,720][20381] Updated weights for policy 1, policy_version 26358 (0.0010) [2023-12-27 05:50:54,744][20347] Updated weights for policy 0, policy_version 26369 (0.0005) [2023-12-27 05:50:54,778][20381] Updated weights for policy 1, policy_version 26368 (0.0010) [2023-12-27 05:50:54,795][20347] Updated weights for policy 0, policy_version 26379 (0.0006) [2023-12-27 05:50:55,392][20381] Updated weights for policy 1, policy_version 26378 (0.0005) [2023-12-27 05:50:55,424][20347] Updated weights for policy 0, policy_version 26389 (0.0006) [2023-12-27 05:50:55,442][20381] Updated weights for policy 1, policy_version 26388 (0.0005) [2023-12-27 05:50:55,485][20347] Updated weights for policy 0, policy_version 26399 (0.0007) [2023-12-27 05:50:55,492][20381] Updated weights for policy 1, policy_version 26398 (0.0005) [2023-12-27 05:50:55,538][20347] Updated weights for policy 0, policy_version 26409 (0.0008) [2023-12-27 05:50:56,075][20381] Updated weights for policy 1, policy_version 26408 (0.0005) [2023-12-27 05:50:56,133][20381] Updated weights for policy 1, policy_version 26418 (0.0005) [2023-12-27 05:50:56,147][20347] Updated weights for policy 0, policy_version 26419 (0.0007) [2023-12-27 05:50:56,191][20381] Updated weights for policy 1, policy_version 26428 (0.0005) [2023-12-27 05:50:56,194][20347] Updated weights for policy 0, policy_version 26429 (0.0008) [2023-12-27 05:50:56,246][20347] Updated weights for policy 0, policy_version 26439 (0.0005) [2023-12-27 05:50:56,785][20381] Updated weights for policy 1, policy_version 26438 (0.0006) [2023-12-27 05:50:56,837][20381] Updated weights for policy 1, policy_version 26448 (0.0005) [2023-12-27 05:50:56,885][20347] Updated weights for policy 0, policy_version 26449 (0.0009) [2023-12-27 05:50:56,890][20381] Updated weights for policy 1, policy_version 26458 (0.0005) [2023-12-27 05:50:56,942][20347] Updated weights for policy 0, policy_version 26459 (0.0006) [2023-12-27 05:50:57,001][20347] Updated weights for policy 0, policy_version 26469 (0.0005) [2023-12-27 05:50:57,045][20347] Updated weights for policy 0, policy_version 26479 (0.0005) [2023-12-27 05:50:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 13557760. Throughput: 0: 10450.5, 1: 10496.8. Samples: 13546852. Policy #0 lag: (min: 7.0, avg: 16.5, max: 39.0) [2023-12-27 05:50:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:50:57,451][20381] Updated weights for policy 1, policy_version 26468 (0.0006) [2023-12-27 05:50:57,496][20381] Updated weights for policy 1, policy_version 26478 (0.0005) [2023-12-27 05:50:57,541][20381] Updated weights for policy 1, policy_version 26488 (0.0005) [2023-12-27 05:50:57,663][20347] Updated weights for policy 0, policy_version 26489 (0.0007) [2023-12-27 05:50:57,704][20347] Updated weights for policy 0, policy_version 26499 (0.0007) [2023-12-27 05:50:57,755][20347] Updated weights for policy 0, policy_version 26509 (0.0008) [2023-12-27 05:50:58,110][20381] Updated weights for policy 1, policy_version 26498 (0.0005) [2023-12-27 05:50:58,169][20381] Updated weights for policy 1, policy_version 26508 (0.0006) [2023-12-27 05:50:58,225][20381] Updated weights for policy 1, policy_version 26518 (0.0007) [2023-12-27 05:50:58,283][20381] Updated weights for policy 1, policy_version 26528 (0.0008) [2023-12-27 05:50:58,403][20347] Updated weights for policy 0, policy_version 26519 (0.0010) [2023-12-27 05:50:58,458][20347] Updated weights for policy 0, policy_version 26529 (0.0010) [2023-12-27 05:50:58,507][20347] Updated weights for policy 0, policy_version 26539 (0.0010) [2023-12-27 05:50:58,996][20381] Updated weights for policy 1, policy_version 26538 (0.0007) [2023-12-27 05:50:59,053][20381] Updated weights for policy 1, policy_version 26548 (0.0007) [2023-12-27 05:50:59,107][20381] Updated weights for policy 1, policy_version 26558 (0.0007) [2023-12-27 05:50:59,268][20347] Updated weights for policy 0, policy_version 26549 (0.0010) [2023-12-27 05:50:59,325][20347] Updated weights for policy 0, policy_version 26559 (0.0011) [2023-12-27 05:50:59,389][20347] Updated weights for policy 0, policy_version 26569 (0.0011) [2023-12-27 05:50:59,814][20381] Updated weights for policy 1, policy_version 26568 (0.0007) [2023-12-27 05:50:59,875][20381] Updated weights for policy 1, policy_version 26578 (0.0006) [2023-12-27 05:50:59,939][20381] Updated weights for policy 1, policy_version 26588 (0.0007) [2023-12-27 05:51:00,085][20347] Updated weights for policy 0, policy_version 26579 (0.0011) [2023-12-27 05:51:00,149][20347] Updated weights for policy 0, policy_version 26589 (0.0010) [2023-12-27 05:51:00,201][20347] Updated weights for policy 0, policy_version 26599 (0.0007) [2023-12-27 05:51:00,544][20381] Updated weights for policy 1, policy_version 26598 (0.0006) [2023-12-27 05:51:00,587][20381] Updated weights for policy 1, policy_version 26608 (0.0005) [2023-12-27 05:51:00,643][20381] Updated weights for policy 1, policy_version 26618 (0.0005) [2023-12-27 05:51:00,804][20347] Updated weights for policy 0, policy_version 26609 (0.0006) [2023-12-27 05:51:00,865][20347] Updated weights for policy 0, policy_version 26619 (0.0006) [2023-12-27 05:51:00,924][20347] Updated weights for policy 0, policy_version 26629 (0.0006) [2023-12-27 05:51:00,976][20347] Updated weights for policy 0, policy_version 26639 (0.0006) [2023-12-27 05:51:01,225][20381] Updated weights for policy 1, policy_version 26628 (0.0007) [2023-12-27 05:51:01,281][20381] Updated weights for policy 1, policy_version 26638 (0.0011) [2023-12-27 05:51:01,349][20381] Updated weights for policy 1, policy_version 26648 (0.0010) [2023-12-27 05:51:01,642][20347] Updated weights for policy 0, policy_version 26649 (0.0008) [2023-12-27 05:51:01,688][20347] Updated weights for policy 0, policy_version 26659 (0.0007) [2023-12-27 05:51:01,749][20347] Updated weights for policy 0, policy_version 26669 (0.0011) [2023-12-27 05:51:01,939][20381] Updated weights for policy 1, policy_version 26658 (0.0008) [2023-12-27 05:51:01,988][20381] Updated weights for policy 1, policy_version 26668 (0.0007) [2023-12-27 05:51:02,037][20381] Updated weights for policy 1, policy_version 26678 (0.0007) [2023-12-27 05:51:02,087][20381] Updated weights for policy 1, policy_version 26688 (0.0007) [2023-12-27 05:51:02,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 13664256. Throughput: 0: 10469.8, 1: 10559.9. Samples: 13675288. Policy #0 lag: (min: 7.0, avg: 16.5, max: 39.0) [2023-12-27 05:51:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:02,455][20347] Updated weights for policy 0, policy_version 26679 (0.0010) [2023-12-27 05:51:02,506][20347] Updated weights for policy 0, policy_version 26689 (0.0010) [2023-12-27 05:51:02,568][20347] Updated weights for policy 0, policy_version 26699 (0.0010) [2023-12-27 05:51:02,797][20381] Updated weights for policy 1, policy_version 26698 (0.0010) [2023-12-27 05:51:02,847][20381] Updated weights for policy 1, policy_version 26708 (0.0011) [2023-12-27 05:51:02,896][20381] Updated weights for policy 1, policy_version 26718 (0.0011) [2023-12-27 05:51:03,253][20347] Updated weights for policy 0, policy_version 26709 (0.0008) [2023-12-27 05:51:03,301][20347] Updated weights for policy 0, policy_version 26719 (0.0010) [2023-12-27 05:51:03,357][20347] Updated weights for policy 0, policy_version 26729 (0.0011) [2023-12-27 05:51:03,516][20381] Updated weights for policy 1, policy_version 26728 (0.0006) [2023-12-27 05:51:03,574][20381] Updated weights for policy 1, policy_version 26738 (0.0005) [2023-12-27 05:51:03,633][20381] Updated weights for policy 1, policy_version 26748 (0.0007) [2023-12-27 05:51:04,033][20347] Updated weights for policy 0, policy_version 26739 (0.0008) [2023-12-27 05:51:04,088][20347] Updated weights for policy 0, policy_version 26749 (0.0008) [2023-12-27 05:51:04,153][20347] Updated weights for policy 0, policy_version 26759 (0.0006) [2023-12-27 05:51:04,248][20381] Updated weights for policy 1, policy_version 26758 (0.0009) [2023-12-27 05:51:04,306][20381] Updated weights for policy 1, policy_version 26768 (0.0007) [2023-12-27 05:51:04,359][20381] Updated weights for policy 1, policy_version 26778 (0.0008) [2023-12-27 05:51:04,798][20347] Updated weights for policy 0, policy_version 26769 (0.0007) [2023-12-27 05:51:04,848][20347] Updated weights for policy 0, policy_version 26779 (0.0006) [2023-12-27 05:51:04,903][20347] Updated weights for policy 0, policy_version 26789 (0.0005) [2023-12-27 05:51:04,954][20347] Updated weights for policy 0, policy_version 26799 (0.0005) [2023-12-27 05:51:05,063][20381] Updated weights for policy 1, policy_version 26788 (0.0007) [2023-12-27 05:51:05,112][20381] Updated weights for policy 1, policy_version 26798 (0.0010) [2023-12-27 05:51:05,166][20381] Updated weights for policy 1, policy_version 26808 (0.0010) [2023-12-27 05:51:05,569][20347] Updated weights for policy 0, policy_version 26809 (0.0007) [2023-12-27 05:51:05,623][20347] Updated weights for policy 0, policy_version 26819 (0.0007) [2023-12-27 05:51:05,668][20347] Updated weights for policy 0, policy_version 26829 (0.0007) [2023-12-27 05:51:05,795][20381] Updated weights for policy 1, policy_version 26818 (0.0011) [2023-12-27 05:51:05,850][20381] Updated weights for policy 1, policy_version 26828 (0.0010) [2023-12-27 05:51:05,904][20381] Updated weights for policy 1, policy_version 26838 (0.0010) [2023-12-27 05:51:05,955][20381] Updated weights for policy 1, policy_version 26848 (0.0010) [2023-12-27 05:51:06,308][20347] Updated weights for policy 0, policy_version 26839 (0.0007) [2023-12-27 05:51:06,370][20347] Updated weights for policy 0, policy_version 26849 (0.0007) [2023-12-27 05:51:06,431][20347] Updated weights for policy 0, policy_version 26859 (0.0007) [2023-12-27 05:51:06,620][20381] Updated weights for policy 1, policy_version 26858 (0.0005) [2023-12-27 05:51:06,671][20381] Updated weights for policy 1, policy_version 26868 (0.0005) [2023-12-27 05:51:06,725][20381] Updated weights for policy 1, policy_version 26878 (0.0005) [2023-12-27 05:51:07,171][20347] Updated weights for policy 0, policy_version 26869 (0.0007) [2023-12-27 05:51:07,228][20347] Updated weights for policy 0, policy_version 26879 (0.0007) [2023-12-27 05:51:07,293][20347] Updated weights for policy 0, policy_version 26889 (0.0008) [2023-12-27 05:51:07,330][20381] Updated weights for policy 1, policy_version 26888 (0.0009) [2023-12-27 05:51:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 13770752. Throughput: 0: 10466.8, 1: 10586.1. Samples: 13738340. Policy #0 lag: (min: 7.0, avg: 16.5, max: 39.0) [2023-12-27 05:51:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000026896_6889472.pth... [2023-12-27 05:51:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000025680_6578176.pth [2023-12-27 05:51:07,390][20381] Updated weights for policy 1, policy_version 26898 (0.0011) [2023-12-27 05:51:07,443][20381] Updated weights for policy 1, policy_version 26908 (0.0010) [2023-12-27 05:51:07,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000026912_6889472.pth... [2023-12-27 05:51:07,470][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000025632_6561792.pth [2023-12-27 05:51:07,975][20347] Updated weights for policy 0, policy_version 26899 (0.0007) [2023-12-27 05:51:08,033][20347] Updated weights for policy 0, policy_version 26909 (0.0007) [2023-12-27 05:51:08,083][20347] Updated weights for policy 0, policy_version 26919 (0.0007) [2023-12-27 05:51:08,133][20381] Updated weights for policy 1, policy_version 26918 (0.0008) [2023-12-27 05:51:08,191][20381] Updated weights for policy 1, policy_version 26928 (0.0006) [2023-12-27 05:51:08,248][20381] Updated weights for policy 1, policy_version 26938 (0.0007) [2023-12-27 05:51:08,663][20347] Updated weights for policy 0, policy_version 26929 (0.0005) [2023-12-27 05:51:08,722][20347] Updated weights for policy 0, policy_version 26939 (0.0005) [2023-12-27 05:51:08,785][20347] Updated weights for policy 0, policy_version 26949 (0.0005) [2023-12-27 05:51:08,839][20347] Updated weights for policy 0, policy_version 26959 (0.0005) [2023-12-27 05:51:08,953][20381] Updated weights for policy 1, policy_version 26948 (0.0008) [2023-12-27 05:51:09,001][20381] Updated weights for policy 1, policy_version 26958 (0.0007) [2023-12-27 05:51:09,056][20381] Updated weights for policy 1, policy_version 26968 (0.0008) [2023-12-27 05:51:09,440][20347] Updated weights for policy 0, policy_version 26969 (0.0008) [2023-12-27 05:51:09,503][20347] Updated weights for policy 0, policy_version 26979 (0.0008) [2023-12-27 05:51:09,562][20347] Updated weights for policy 0, policy_version 26989 (0.0009) [2023-12-27 05:51:09,764][20381] Updated weights for policy 1, policy_version 26978 (0.0008) [2023-12-27 05:51:09,829][20381] Updated weights for policy 1, policy_version 26988 (0.0008) [2023-12-27 05:51:09,894][20381] Updated weights for policy 1, policy_version 26998 (0.0007) [2023-12-27 05:51:09,964][20381] Updated weights for policy 1, policy_version 27008 (0.0008) [2023-12-27 05:51:10,228][20347] Updated weights for policy 0, policy_version 26999 (0.0008) [2023-12-27 05:51:10,290][20347] Updated weights for policy 0, policy_version 27009 (0.0007) [2023-12-27 05:51:10,347][20347] Updated weights for policy 0, policy_version 27019 (0.0009) [2023-12-27 05:51:10,680][20381] Updated weights for policy 1, policy_version 27018 (0.0008) [2023-12-27 05:51:10,737][20381] Updated weights for policy 1, policy_version 27028 (0.0007) [2023-12-27 05:51:10,801][20381] Updated weights for policy 1, policy_version 27038 (0.0008) [2023-12-27 05:51:10,983][20347] Updated weights for policy 0, policy_version 27029 (0.0007) [2023-12-27 05:51:11,050][20347] Updated weights for policy 0, policy_version 27039 (0.0007) [2023-12-27 05:51:11,108][20347] Updated weights for policy 0, policy_version 27049 (0.0007) [2023-12-27 05:51:11,518][20381] Updated weights for policy 1, policy_version 27048 (0.0009) [2023-12-27 05:51:11,570][20381] Updated weights for policy 1, policy_version 27058 (0.0011) [2023-12-27 05:51:11,629][20381] Updated weights for policy 1, policy_version 27068 (0.0010) [2023-12-27 05:51:11,810][20347] Updated weights for policy 0, policy_version 27059 (0.0007) [2023-12-27 05:51:11,869][20347] Updated weights for policy 0, policy_version 27069 (0.0006) [2023-12-27 05:51:11,929][20347] Updated weights for policy 0, policy_version 27079 (0.0006) [2023-12-27 05:51:12,276][20381] Updated weights for policy 1, policy_version 27078 (0.0010) [2023-12-27 05:51:12,335][20381] Updated weights for policy 1, policy_version 27088 (0.0010) [2023-12-27 05:51:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 13869056. Throughput: 0: 10461.4, 1: 10631.3. Samples: 13862920. Policy #0 lag: (min: 7.0, avg: 16.5, max: 39.0) [2023-12-27 05:51:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:12,400][20381] Updated weights for policy 1, policy_version 27098 (0.0008) [2023-12-27 05:51:12,609][20347] Updated weights for policy 0, policy_version 27089 (0.0007) [2023-12-27 05:51:12,663][20347] Updated weights for policy 0, policy_version 27099 (0.0009) [2023-12-27 05:51:12,724][20347] Updated weights for policy 0, policy_version 27109 (0.0007) [2023-12-27 05:51:12,779][20347] Updated weights for policy 0, policy_version 27119 (0.0007) [2023-12-27 05:51:12,996][20381] Updated weights for policy 1, policy_version 27108 (0.0006) [2023-12-27 05:51:13,047][20381] Updated weights for policy 1, policy_version 27118 (0.0006) [2023-12-27 05:51:13,104][20381] Updated weights for policy 1, policy_version 27128 (0.0006) [2023-12-27 05:51:13,396][20347] Updated weights for policy 0, policy_version 27129 (0.0008) [2023-12-27 05:51:13,441][20347] Updated weights for policy 0, policy_version 27139 (0.0009) [2023-12-27 05:51:13,488][20347] Updated weights for policy 0, policy_version 27149 (0.0007) [2023-12-27 05:51:13,665][20381] Updated weights for policy 1, policy_version 27138 (0.0005) [2023-12-27 05:51:13,721][20381] Updated weights for policy 1, policy_version 27148 (0.0006) [2023-12-27 05:51:13,772][20381] Updated weights for policy 1, policy_version 27158 (0.0005) [2023-12-27 05:51:13,816][20381] Updated weights for policy 1, policy_version 27168 (0.0005) [2023-12-27 05:51:14,200][20347] Updated weights for policy 0, policy_version 27159 (0.0007) [2023-12-27 05:51:14,259][20347] Updated weights for policy 0, policy_version 27169 (0.0007) [2023-12-27 05:51:14,314][20347] Updated weights for policy 0, policy_version 27179 (0.0007) [2023-12-27 05:51:14,407][20381] Updated weights for policy 1, policy_version 27178 (0.0005) [2023-12-27 05:51:14,456][20381] Updated weights for policy 1, policy_version 27188 (0.0006) [2023-12-27 05:51:14,503][20381] Updated weights for policy 1, policy_version 27198 (0.0007) [2023-12-27 05:51:15,050][20347] Updated weights for policy 0, policy_version 27189 (0.0007) [2023-12-27 05:51:15,112][20347] Updated weights for policy 0, policy_version 27199 (0.0007) [2023-12-27 05:51:15,176][20347] Updated weights for policy 0, policy_version 27209 (0.0008) [2023-12-27 05:51:15,197][20381] Updated weights for policy 1, policy_version 27208 (0.0006) [2023-12-27 05:51:15,259][20381] Updated weights for policy 1, policy_version 27218 (0.0007) [2023-12-27 05:51:15,314][20381] Updated weights for policy 1, policy_version 27228 (0.0009) [2023-12-27 05:51:15,847][20347] Updated weights for policy 0, policy_version 27219 (0.0007) [2023-12-27 05:51:15,906][20347] Updated weights for policy 0, policy_version 27229 (0.0007) [2023-12-27 05:51:15,961][20347] Updated weights for policy 0, policy_version 27239 (0.0008) [2023-12-27 05:51:16,003][20381] Updated weights for policy 1, policy_version 27238 (0.0008) [2023-12-27 05:51:16,061][20381] Updated weights for policy 1, policy_version 27248 (0.0007) [2023-12-27 05:51:16,121][20381] Updated weights for policy 1, policy_version 27258 (0.0008) [2023-12-27 05:51:16,646][20347] Updated weights for policy 0, policy_version 27249 (0.0007) [2023-12-27 05:51:16,701][20347] Updated weights for policy 0, policy_version 27259 (0.0007) [2023-12-27 05:51:16,744][20347] Updated weights for policy 0, policy_version 27269 (0.0006) [2023-12-27 05:51:16,793][20347] Updated weights for policy 0, policy_version 27279 (0.0006) [2023-12-27 05:51:16,795][20381] Updated weights for policy 1, policy_version 27268 (0.0008) [2023-12-27 05:51:16,843][20381] Updated weights for policy 1, policy_version 27278 (0.0007) [2023-12-27 05:51:16,891][20381] Updated weights for policy 1, policy_version 27288 (0.0007) [2023-12-27 05:51:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 13975552. Throughput: 0: 10399.6, 1: 10616.7. Samples: 13988204. Policy #0 lag: (min: 31.0, avg: 31.4, max: 42.0) [2023-12-27 05:51:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:17,517][20381] Updated weights for policy 1, policy_version 27298 (0.0009) [2023-12-27 05:51:17,547][20347] Updated weights for policy 0, policy_version 27289 (0.0006) [2023-12-27 05:51:17,586][20381] Updated weights for policy 1, policy_version 27308 (0.0008) [2023-12-27 05:51:17,611][20347] Updated weights for policy 0, policy_version 27299 (0.0005) [2023-12-27 05:51:17,652][20381] Updated weights for policy 1, policy_version 27318 (0.0008) [2023-12-27 05:51:17,668][20347] Updated weights for policy 0, policy_version 27309 (0.0007) [2023-12-27 05:51:17,717][20381] Updated weights for policy 1, policy_version 27328 (0.0008) [2023-12-27 05:51:18,287][20347] Updated weights for policy 0, policy_version 27319 (0.0008) [2023-12-27 05:51:18,295][20381] Updated weights for policy 1, policy_version 27338 (0.0006) [2023-12-27 05:51:18,342][20347] Updated weights for policy 0, policy_version 27329 (0.0008) [2023-12-27 05:51:18,367][20381] Updated weights for policy 1, policy_version 27348 (0.0007) [2023-12-27 05:51:18,400][20347] Updated weights for policy 0, policy_version 27339 (0.0007) [2023-12-27 05:51:18,427][20381] Updated weights for policy 1, policy_version 27358 (0.0006) [2023-12-27 05:51:19,037][20381] Updated weights for policy 1, policy_version 27368 (0.0009) [2023-12-27 05:51:19,090][20347] Updated weights for policy 0, policy_version 27349 (0.0006) [2023-12-27 05:51:19,092][20381] Updated weights for policy 1, policy_version 27378 (0.0010) [2023-12-27 05:51:19,150][20381] Updated weights for policy 1, policy_version 27388 (0.0009) [2023-12-27 05:51:19,152][20347] Updated weights for policy 0, policy_version 27359 (0.0005) [2023-12-27 05:51:19,206][20347] Updated weights for policy 0, policy_version 27369 (0.0006) [2023-12-27 05:51:19,864][20381] Updated weights for policy 1, policy_version 27398 (0.0008) [2023-12-27 05:51:19,905][20347] Updated weights for policy 0, policy_version 27379 (0.0009) [2023-12-27 05:51:19,932][20381] Updated weights for policy 1, policy_version 27408 (0.0007) [2023-12-27 05:51:19,965][20347] Updated weights for policy 0, policy_version 27389 (0.0008) [2023-12-27 05:51:19,993][20381] Updated weights for policy 1, policy_version 27418 (0.0007) [2023-12-27 05:51:20,020][20347] Updated weights for policy 0, policy_version 27399 (0.0007) [2023-12-27 05:51:20,645][20381] Updated weights for policy 1, policy_version 27428 (0.0008) [2023-12-27 05:51:20,706][20381] Updated weights for policy 1, policy_version 27438 (0.0007) [2023-12-27 05:51:20,741][20347] Updated weights for policy 0, policy_version 27409 (0.0008) [2023-12-27 05:51:20,771][20381] Updated weights for policy 1, policy_version 27448 (0.0008) [2023-12-27 05:51:20,800][20347] Updated weights for policy 0, policy_version 27419 (0.0011) [2023-12-27 05:51:20,857][20347] Updated weights for policy 0, policy_version 27429 (0.0007) [2023-12-27 05:51:20,923][20347] Updated weights for policy 0, policy_version 27439 (0.0007) [2023-12-27 05:51:21,437][20381] Updated weights for policy 1, policy_version 27458 (0.0008) [2023-12-27 05:51:21,501][20381] Updated weights for policy 1, policy_version 27468 (0.0006) [2023-12-27 05:51:21,564][20381] Updated weights for policy 1, policy_version 27478 (0.0006) [2023-12-27 05:51:21,628][20381] Updated weights for policy 1, policy_version 27488 (0.0006) [2023-12-27 05:51:21,730][20347] Updated weights for policy 0, policy_version 27449 (0.0007) [2023-12-27 05:51:21,786][20347] Updated weights for policy 0, policy_version 27459 (0.0008) [2023-12-27 05:51:21,840][20347] Updated weights for policy 0, policy_version 27469 (0.0010) [2023-12-27 05:51:22,241][20381] Updated weights for policy 1, policy_version 27498 (0.0007) [2023-12-27 05:51:22,296][20381] Updated weights for policy 1, policy_version 27508 (0.0008) [2023-12-27 05:51:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 14073856. Throughput: 0: 10381.9, 1: 10647.2. Samples: 14050380. Policy #0 lag: (min: 31.0, avg: 31.4, max: 42.0) [2023-12-27 05:51:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:22,364][20381] Updated weights for policy 1, policy_version 27518 (0.0007) [2023-12-27 05:51:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000027472_7036928.pth... [2023-12-27 05:51:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000026288_6733824.pth [2023-12-27 05:51:22,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000027520_7045120.pth... [2023-12-27 05:51:22,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000026240_6717440.pth [2023-12-27 05:51:22,577][20347] Updated weights for policy 0, policy_version 27479 (0.0006) [2023-12-27 05:51:22,642][20347] Updated weights for policy 0, policy_version 27489 (0.0005) [2023-12-27 05:51:22,702][20347] Updated weights for policy 0, policy_version 27499 (0.0007) [2023-12-27 05:51:23,022][20381] Updated weights for policy 1, policy_version 27528 (0.0010) [2023-12-27 05:51:23,078][20381] Updated weights for policy 1, policy_version 27538 (0.0006) [2023-12-27 05:51:23,137][20381] Updated weights for policy 1, policy_version 27548 (0.0005) [2023-12-27 05:51:23,389][20347] Updated weights for policy 0, policy_version 27509 (0.0008) [2023-12-27 05:51:23,441][20347] Updated weights for policy 0, policy_version 27519 (0.0008) [2023-12-27 05:51:23,496][20347] Updated weights for policy 0, policy_version 27529 (0.0008) [2023-12-27 05:51:23,670][20381] Updated weights for policy 1, policy_version 27558 (0.0005) [2023-12-27 05:51:23,722][20381] Updated weights for policy 1, policy_version 27568 (0.0005) [2023-12-27 05:51:23,778][20381] Updated weights for policy 1, policy_version 27578 (0.0005) [2023-12-27 05:51:24,181][20347] Updated weights for policy 0, policy_version 27539 (0.0007) [2023-12-27 05:51:24,243][20347] Updated weights for policy 0, policy_version 27549 (0.0006) [2023-12-27 05:51:24,308][20347] Updated weights for policy 0, policy_version 27559 (0.0005) [2023-12-27 05:51:24,323][20381] Updated weights for policy 1, policy_version 27588 (0.0007) [2023-12-27 05:51:24,378][20381] Updated weights for policy 1, policy_version 27598 (0.0010) [2023-12-27 05:51:24,433][20381] Updated weights for policy 1, policy_version 27608 (0.0010) [2023-12-27 05:51:24,925][20347] Updated weights for policy 0, policy_version 27569 (0.0007) [2023-12-27 05:51:24,987][20347] Updated weights for policy 0, policy_version 27579 (0.0006) [2023-12-27 05:51:25,047][20347] Updated weights for policy 0, policy_version 27589 (0.0007) [2023-12-27 05:51:25,053][20381] Updated weights for policy 1, policy_version 27618 (0.0010) [2023-12-27 05:51:25,099][20347] Updated weights for policy 0, policy_version 27599 (0.0007) [2023-12-27 05:51:25,108][20381] Updated weights for policy 1, policy_version 27628 (0.0011) [2023-12-27 05:51:25,160][20381] Updated weights for policy 1, policy_version 27638 (0.0010) [2023-12-27 05:51:25,209][20381] Updated weights for policy 1, policy_version 27648 (0.0010) [2023-12-27 05:51:25,693][20347] Updated weights for policy 0, policy_version 27609 (0.0010) [2023-12-27 05:51:25,750][20347] Updated weights for policy 0, policy_version 27619 (0.0007) [2023-12-27 05:51:25,800][20347] Updated weights for policy 0, policy_version 27629 (0.0007) [2023-12-27 05:51:25,913][20381] Updated weights for policy 1, policy_version 27658 (0.0010) [2023-12-27 05:51:25,961][20381] Updated weights for policy 1, policy_version 27668 (0.0010) [2023-12-27 05:51:26,005][20381] Updated weights for policy 1, policy_version 27678 (0.0008) [2023-12-27 05:51:26,386][20347] Updated weights for policy 0, policy_version 27639 (0.0006) [2023-12-27 05:51:26,436][20347] Updated weights for policy 0, policy_version 27649 (0.0007) [2023-12-27 05:51:26,483][20347] Updated weights for policy 0, policy_version 27659 (0.0007) [2023-12-27 05:51:26,659][20381] Updated weights for policy 1, policy_version 27688 (0.0009) [2023-12-27 05:51:26,713][20381] Updated weights for policy 1, policy_version 27698 (0.0010) [2023-12-27 05:51:26,767][20381] Updated weights for policy 1, policy_version 27708 (0.0010) [2023-12-27 05:51:27,179][20347] Updated weights for policy 0, policy_version 27669 (0.0007) [2023-12-27 05:51:27,232][20347] Updated weights for policy 0, policy_version 27679 (0.0007) [2023-12-27 05:51:27,283][20347] Updated weights for policy 0, policy_version 27689 (0.0007) [2023-12-27 05:51:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20882.7). Total num frames: 14188544. Throughput: 0: 10343.4, 1: 10716.2. Samples: 14176316. Policy #0 lag: (min: 31.0, avg: 31.4, max: 42.0) [2023-12-27 05:51:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:27,457][20381] Updated weights for policy 1, policy_version 27718 (0.0010) [2023-12-27 05:51:27,507][20381] Updated weights for policy 1, policy_version 27728 (0.0010) [2023-12-27 05:51:27,552][20381] Updated weights for policy 1, policy_version 27738 (0.0010) [2023-12-27 05:51:27,958][20347] Updated weights for policy 0, policy_version 27699 (0.0006) [2023-12-27 05:51:28,010][20347] Updated weights for policy 0, policy_version 27709 (0.0007) [2023-12-27 05:51:28,060][20347] Updated weights for policy 0, policy_version 27719 (0.0007) [2023-12-27 05:51:28,244][20381] Updated weights for policy 1, policy_version 27748 (0.0010) [2023-12-27 05:51:28,301][20381] Updated weights for policy 1, policy_version 27758 (0.0010) [2023-12-27 05:51:28,363][20381] Updated weights for policy 1, policy_version 27768 (0.0009) [2023-12-27 05:51:28,726][20347] Updated weights for policy 0, policy_version 27729 (0.0007) [2023-12-27 05:51:28,786][20347] Updated weights for policy 0, policy_version 27739 (0.0005) [2023-12-27 05:51:28,842][20347] Updated weights for policy 0, policy_version 27749 (0.0006) [2023-12-27 05:51:28,895][20347] Updated weights for policy 0, policy_version 27759 (0.0005) [2023-12-27 05:51:29,020][20381] Updated weights for policy 1, policy_version 27778 (0.0008) [2023-12-27 05:51:29,068][20381] Updated weights for policy 1, policy_version 27788 (0.0009) [2023-12-27 05:51:29,120][20381] Updated weights for policy 1, policy_version 27798 (0.0006) [2023-12-27 05:51:29,179][20381] Updated weights for policy 1, policy_version 27808 (0.0010) [2023-12-27 05:51:29,496][20347] Updated weights for policy 0, policy_version 27769 (0.0006) [2023-12-27 05:51:29,552][20347] Updated weights for policy 0, policy_version 27779 (0.0008) [2023-12-27 05:51:29,615][20347] Updated weights for policy 0, policy_version 27789 (0.0008) [2023-12-27 05:51:29,810][20381] Updated weights for policy 1, policy_version 27818 (0.0006) [2023-12-27 05:51:29,872][20381] Updated weights for policy 1, policy_version 27828 (0.0008) [2023-12-27 05:51:29,935][20381] Updated weights for policy 1, policy_version 27838 (0.0007) [2023-12-27 05:51:30,320][20347] Updated weights for policy 0, policy_version 27799 (0.0009) [2023-12-27 05:51:30,377][20347] Updated weights for policy 0, policy_version 27809 (0.0010) [2023-12-27 05:51:30,431][20347] Updated weights for policy 0, policy_version 27819 (0.0010) [2023-12-27 05:51:30,583][20381] Updated weights for policy 1, policy_version 27848 (0.0006) [2023-12-27 05:51:30,633][20381] Updated weights for policy 1, policy_version 27858 (0.0005) [2023-12-27 05:51:30,691][20381] Updated weights for policy 1, policy_version 27868 (0.0005) [2023-12-27 05:51:31,094][20347] Updated weights for policy 0, policy_version 27829 (0.0010) [2023-12-27 05:51:31,154][20347] Updated weights for policy 0, policy_version 27839 (0.0010) [2023-12-27 05:51:31,217][20347] Updated weights for policy 0, policy_version 27849 (0.0007) [2023-12-27 05:51:31,258][20381] Updated weights for policy 1, policy_version 27878 (0.0008) [2023-12-27 05:51:31,328][20381] Updated weights for policy 1, policy_version 27888 (0.0009) [2023-12-27 05:51:31,393][20381] Updated weights for policy 1, policy_version 27898 (0.0008) [2023-12-27 05:51:31,936][20347] Updated weights for policy 0, policy_version 27859 (0.0007) [2023-12-27 05:51:31,994][20347] Updated weights for policy 0, policy_version 27869 (0.0007) [2023-12-27 05:51:32,007][20381] Updated weights for policy 1, policy_version 27908 (0.0008) [2023-12-27 05:51:32,056][20381] Updated weights for policy 1, policy_version 27918 (0.0009) [2023-12-27 05:51:32,056][20347] Updated weights for policy 0, policy_version 27879 (0.0009) [2023-12-27 05:51:32,111][20381] Updated weights for policy 1, policy_version 27928 (0.0010) [2023-12-27 05:51:32,362][19170] Fps is (10 sec: 22118.7, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 14295040. Throughput: 0: 10315.5, 1: 10713.9. Samples: 14302320. Policy #0 lag: (min: 31.0, avg: 31.4, max: 42.0) [2023-12-27 05:51:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:32,738][20347] Updated weights for policy 0, policy_version 27889 (0.0007) [2023-12-27 05:51:32,766][20381] Updated weights for policy 1, policy_version 27938 (0.0009) [2023-12-27 05:51:32,798][20347] Updated weights for policy 0, policy_version 27899 (0.0005) [2023-12-27 05:51:32,817][20381] Updated weights for policy 1, policy_version 27948 (0.0005) [2023-12-27 05:51:32,856][20347] Updated weights for policy 0, policy_version 27909 (0.0005) [2023-12-27 05:51:32,871][20381] Updated weights for policy 1, policy_version 27958 (0.0006) [2023-12-27 05:51:32,908][20347] Updated weights for policy 0, policy_version 27919 (0.0007) [2023-12-27 05:51:32,932][20381] Updated weights for policy 1, policy_version 27968 (0.0008) [2023-12-27 05:51:33,469][20347] Updated weights for policy 0, policy_version 27929 (0.0006) [2023-12-27 05:51:33,471][20381] Updated weights for policy 1, policy_version 27978 (0.0010) [2023-12-27 05:51:33,525][20347] Updated weights for policy 0, policy_version 27939 (0.0006) [2023-12-27 05:51:33,527][20381] Updated weights for policy 1, policy_version 27988 (0.0010) [2023-12-27 05:51:33,579][20381] Updated weights for policy 1, policy_version 27998 (0.0010) [2023-12-27 05:51:33,582][20347] Updated weights for policy 0, policy_version 27949 (0.0005) [2023-12-27 05:51:34,216][20381] Updated weights for policy 1, policy_version 28008 (0.0010) [2023-12-27 05:51:34,265][20381] Updated weights for policy 1, policy_version 28018 (0.0010) [2023-12-27 05:51:34,295][20347] Updated weights for policy 0, policy_version 27959 (0.0006) [2023-12-27 05:51:34,323][20381] Updated weights for policy 1, policy_version 28028 (0.0010) [2023-12-27 05:51:34,360][20347] Updated weights for policy 0, policy_version 27969 (0.0008) [2023-12-27 05:51:34,419][20347] Updated weights for policy 0, policy_version 27979 (0.0007) [2023-12-27 05:51:35,003][20381] Updated weights for policy 1, policy_version 28038 (0.0011) [2023-12-27 05:51:35,063][20381] Updated weights for policy 1, policy_version 28048 (0.0011) [2023-12-27 05:51:35,099][20347] Updated weights for policy 0, policy_version 27989 (0.0007) [2023-12-27 05:51:35,122][20381] Updated weights for policy 1, policy_version 28058 (0.0010) [2023-12-27 05:51:35,156][20347] Updated weights for policy 0, policy_version 27999 (0.0005) [2023-12-27 05:51:35,207][20347] Updated weights for policy 0, policy_version 28009 (0.0005) [2023-12-27 05:51:35,795][20381] Updated weights for policy 1, policy_version 28068 (0.0009) [2023-12-27 05:51:35,847][20381] Updated weights for policy 1, policy_version 28078 (0.0010) [2023-12-27 05:51:35,865][20347] Updated weights for policy 0, policy_version 28019 (0.0007) [2023-12-27 05:51:35,896][20381] Updated weights for policy 1, policy_version 28088 (0.0010) [2023-12-27 05:51:35,920][20347] Updated weights for policy 0, policy_version 28029 (0.0007) [2023-12-27 05:51:35,983][20347] Updated weights for policy 0, policy_version 28039 (0.0010) [2023-12-27 05:51:36,574][20381] Updated weights for policy 1, policy_version 28098 (0.0008) [2023-12-27 05:51:36,623][20381] Updated weights for policy 1, policy_version 28108 (0.0010) [2023-12-27 05:51:36,671][20381] Updated weights for policy 1, policy_version 28118 (0.0010) [2023-12-27 05:51:36,681][20347] Updated weights for policy 0, policy_version 28049 (0.0010) [2023-12-27 05:51:36,726][20381] Updated weights for policy 1, policy_version 28128 (0.0010) [2023-12-27 05:51:36,746][20347] Updated weights for policy 0, policy_version 28059 (0.0010) [2023-12-27 05:51:36,807][20347] Updated weights for policy 0, policy_version 28069 (0.0010) [2023-12-27 05:51:36,865][20347] Updated weights for policy 0, policy_version 28079 (0.0010) [2023-12-27 05:51:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 14393344. Throughput: 0: 10310.7, 1: 10755.2. Samples: 14366216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:51:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000028080_7192576.pth... [2023-12-27 05:51:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000026896_6889472.pth [2023-12-27 05:51:37,411][20381] Updated weights for policy 1, policy_version 28138 (0.0007) [2023-12-27 05:51:37,470][20381] Updated weights for policy 1, policy_version 28148 (0.0008) [2023-12-27 05:51:37,528][20381] Updated weights for policy 1, policy_version 28158 (0.0006) [2023-12-27 05:51:37,537][20347] Updated weights for policy 0, policy_version 28089 (0.0010) [2023-12-27 05:51:37,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000028160_7208960.pth... [2023-12-27 05:51:37,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000026912_6889472.pth [2023-12-27 05:51:37,596][20347] Updated weights for policy 0, policy_version 28099 (0.0010) [2023-12-27 05:51:37,652][20347] Updated weights for policy 0, policy_version 28109 (0.0010) [2023-12-27 05:51:38,201][20381] Updated weights for policy 1, policy_version 28168 (0.0007) [2023-12-27 05:51:38,255][20381] Updated weights for policy 1, policy_version 28178 (0.0007) [2023-12-27 05:51:38,297][20347] Updated weights for policy 0, policy_version 28119 (0.0010) [2023-12-27 05:51:38,310][20381] Updated weights for policy 1, policy_version 28188 (0.0005) [2023-12-27 05:51:38,359][20347] Updated weights for policy 0, policy_version 28129 (0.0012) [2023-12-27 05:51:38,422][20347] Updated weights for policy 0, policy_version 28139 (0.0006) [2023-12-27 05:51:38,905][20381] Updated weights for policy 1, policy_version 28198 (0.0007) [2023-12-27 05:51:38,963][20381] Updated weights for policy 1, policy_version 28208 (0.0006) [2023-12-27 05:51:39,019][20381] Updated weights for policy 1, policy_version 28218 (0.0005) [2023-12-27 05:51:39,071][20347] Updated weights for policy 0, policy_version 28149 (0.0008) [2023-12-27 05:51:39,139][20347] Updated weights for policy 0, policy_version 28159 (0.0010) [2023-12-27 05:51:39,193][20347] Updated weights for policy 0, policy_version 28169 (0.0010) [2023-12-27 05:51:39,643][20381] Updated weights for policy 1, policy_version 28228 (0.0006) [2023-12-27 05:51:39,698][20381] Updated weights for policy 1, policy_version 28238 (0.0008) [2023-12-27 05:51:39,756][20381] Updated weights for policy 1, policy_version 28248 (0.0007) [2023-12-27 05:51:39,877][20347] Updated weights for policy 0, policy_version 28179 (0.0010) [2023-12-27 05:51:39,935][20347] Updated weights for policy 0, policy_version 28189 (0.0008) [2023-12-27 05:51:39,989][20347] Updated weights for policy 0, policy_version 28199 (0.0008) [2023-12-27 05:51:40,504][20381] Updated weights for policy 1, policy_version 28258 (0.0008) [2023-12-27 05:51:40,564][20381] Updated weights for policy 1, policy_version 28268 (0.0009) [2023-12-27 05:51:40,579][20347] Updated weights for policy 0, policy_version 28209 (0.0008) [2023-12-27 05:51:40,619][20381] Updated weights for policy 1, policy_version 28278 (0.0006) [2023-12-27 05:51:40,632][20347] Updated weights for policy 0, policy_version 28219 (0.0005) [2023-12-27 05:51:40,680][20381] Updated weights for policy 1, policy_version 28288 (0.0005) [2023-12-27 05:51:40,682][20347] Updated weights for policy 0, policy_version 28229 (0.0006) [2023-12-27 05:51:40,730][20347] Updated weights for policy 0, policy_version 28239 (0.0006) [2023-12-27 05:51:41,374][20381] Updated weights for policy 1, policy_version 28298 (0.0008) [2023-12-27 05:51:41,412][20347] Updated weights for policy 0, policy_version 28249 (0.0008) [2023-12-27 05:51:41,432][20381] Updated weights for policy 1, policy_version 28308 (0.0007) [2023-12-27 05:51:41,471][20347] Updated weights for policy 0, policy_version 28259 (0.0008) [2023-12-27 05:51:41,492][20381] Updated weights for policy 1, policy_version 28318 (0.0007) [2023-12-27 05:51:41,529][20347] Updated weights for policy 0, policy_version 28269 (0.0008) [2023-12-27 05:51:42,197][20381] Updated weights for policy 1, policy_version 28328 (0.0008) [2023-12-27 05:51:42,203][20347] Updated weights for policy 0, policy_version 28279 (0.0006) [2023-12-27 05:51:42,253][20381] Updated weights for policy 1, policy_version 28338 (0.0008) [2023-12-27 05:51:42,260][20347] Updated weights for policy 0, policy_version 28289 (0.0005) [2023-12-27 05:51:42,316][20381] Updated weights for policy 1, policy_version 28348 (0.0008) [2023-12-27 05:51:42,318][20347] Updated weights for policy 0, policy_version 28299 (0.0007) [2023-12-27 05:51:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21299.2, 300 sec: 20882.7). Total num frames: 14508032. Throughput: 0: 10317.6, 1: 10668.4. Samples: 14491220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:51:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:42,941][20381] Updated weights for policy 1, policy_version 28358 (0.0006) [2023-12-27 05:51:43,002][20381] Updated weights for policy 1, policy_version 28368 (0.0007) [2023-12-27 05:51:43,039][20347] Updated weights for policy 0, policy_version 28309 (0.0008) [2023-12-27 05:51:43,056][20381] Updated weights for policy 1, policy_version 28378 (0.0005) [2023-12-27 05:51:43,086][20347] Updated weights for policy 0, policy_version 28319 (0.0007) [2023-12-27 05:51:43,133][20347] Updated weights for policy 0, policy_version 28329 (0.0008) [2023-12-27 05:51:43,635][20381] Updated weights for policy 1, policy_version 28388 (0.0005) [2023-12-27 05:51:43,696][20381] Updated weights for policy 1, policy_version 28398 (0.0005) [2023-12-27 05:51:43,760][20347] Updated weights for policy 0, policy_version 28339 (0.0007) [2023-12-27 05:51:43,761][20381] Updated weights for policy 1, policy_version 28408 (0.0009) [2023-12-27 05:51:43,823][20347] Updated weights for policy 0, policy_version 28349 (0.0009) [2023-12-27 05:51:43,876][20347] Updated weights for policy 0, policy_version 28359 (0.0005) [2023-12-27 05:51:44,370][20381] Updated weights for policy 1, policy_version 28418 (0.0011) [2023-12-27 05:51:44,430][20381] Updated weights for policy 1, policy_version 28428 (0.0007) [2023-12-27 05:51:44,479][20381] Updated weights for policy 1, policy_version 28438 (0.0005) [2023-12-27 05:51:44,527][20381] Updated weights for policy 1, policy_version 28448 (0.0005) [2023-12-27 05:51:44,532][20347] Updated weights for policy 0, policy_version 28369 (0.0007) [2023-12-27 05:51:44,592][20347] Updated weights for policy 0, policy_version 28379 (0.0010) [2023-12-27 05:51:44,646][20347] Updated weights for policy 0, policy_version 28389 (0.0010) [2023-12-27 05:51:44,697][20347] Updated weights for policy 0, policy_version 28399 (0.0010) [2023-12-27 05:51:45,123][20381] Updated weights for policy 1, policy_version 28458 (0.0011) [2023-12-27 05:51:45,172][20381] Updated weights for policy 1, policy_version 28468 (0.0011) [2023-12-27 05:51:45,216][20381] Updated weights for policy 1, policy_version 28478 (0.0008) [2023-12-27 05:51:45,305][20347] Updated weights for policy 0, policy_version 28409 (0.0009) [2023-12-27 05:51:45,363][20347] Updated weights for policy 0, policy_version 28419 (0.0010) [2023-12-27 05:51:45,418][20347] Updated weights for policy 0, policy_version 28429 (0.0010) [2023-12-27 05:51:45,884][20381] Updated weights for policy 1, policy_version 28488 (0.0007) [2023-12-27 05:51:45,939][20381] Updated weights for policy 1, policy_version 28498 (0.0007) [2023-12-27 05:51:46,003][20381] Updated weights for policy 1, policy_version 28508 (0.0007) [2023-12-27 05:51:46,094][20347] Updated weights for policy 0, policy_version 28439 (0.0010) [2023-12-27 05:51:46,150][20347] Updated weights for policy 0, policy_version 28449 (0.0010) [2023-12-27 05:51:46,198][20347] Updated weights for policy 0, policy_version 28459 (0.0010) [2023-12-27 05:51:46,720][20381] Updated weights for policy 1, policy_version 28518 (0.0006) [2023-12-27 05:51:46,778][20381] Updated weights for policy 1, policy_version 28528 (0.0007) [2023-12-27 05:51:46,826][20347] Updated weights for policy 0, policy_version 28469 (0.0010) [2023-12-27 05:51:46,841][20381] Updated weights for policy 1, policy_version 28538 (0.0007) [2023-12-27 05:51:46,884][20347] Updated weights for policy 0, policy_version 28479 (0.0010) [2023-12-27 05:51:46,936][20347] Updated weights for policy 0, policy_version 28489 (0.0010) [2023-12-27 05:51:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 14606336. Throughput: 0: 10320.9, 1: 10617.2. Samples: 14617504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:51:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:47,517][20381] Updated weights for policy 1, policy_version 28548 (0.0009) [2023-12-27 05:51:47,573][20381] Updated weights for policy 1, policy_version 28558 (0.0007) [2023-12-27 05:51:47,608][20347] Updated weights for policy 0, policy_version 28499 (0.0010) [2023-12-27 05:51:47,620][20381] Updated weights for policy 1, policy_version 28568 (0.0008) [2023-12-27 05:51:47,656][20347] Updated weights for policy 0, policy_version 28509 (0.0010) [2023-12-27 05:51:47,704][20347] Updated weights for policy 0, policy_version 28519 (0.0010) [2023-12-27 05:51:48,224][20381] Updated weights for policy 1, policy_version 28578 (0.0005) [2023-12-27 05:51:48,298][20381] Updated weights for policy 1, policy_version 28588 (0.0005) [2023-12-27 05:51:48,372][20381] Updated weights for policy 1, policy_version 28598 (0.0008) [2023-12-27 05:51:48,403][20347] Updated weights for policy 0, policy_version 28529 (0.0010) [2023-12-27 05:51:48,433][20381] Updated weights for policy 1, policy_version 28608 (0.0009) [2023-12-27 05:51:48,460][20347] Updated weights for policy 0, policy_version 28539 (0.0009) [2023-12-27 05:51:48,521][20347] Updated weights for policy 0, policy_version 28549 (0.0008) [2023-12-27 05:51:48,586][20347] Updated weights for policy 0, policy_version 28559 (0.0009) [2023-12-27 05:51:49,083][20381] Updated weights for policy 1, policy_version 28618 (0.0008) [2023-12-27 05:51:49,146][20381] Updated weights for policy 1, policy_version 28628 (0.0008) [2023-12-27 05:51:49,207][20381] Updated weights for policy 1, policy_version 28638 (0.0007) [2023-12-27 05:51:49,244][20347] Updated weights for policy 0, policy_version 28569 (0.0008) [2023-12-27 05:51:49,298][20347] Updated weights for policy 0, policy_version 28579 (0.0007) [2023-12-27 05:51:49,356][20347] Updated weights for policy 0, policy_version 28589 (0.0007) [2023-12-27 05:51:49,937][20347] Updated weights for policy 0, policy_version 28599 (0.0007) [2023-12-27 05:51:50,001][20347] Updated weights for policy 0, policy_version 28609 (0.0006) [2023-12-27 05:51:50,026][20381] Updated weights for policy 1, policy_version 28648 (0.0007) [2023-12-27 05:51:50,064][20347] Updated weights for policy 0, policy_version 28619 (0.0006) [2023-12-27 05:51:50,088][20381] Updated weights for policy 1, policy_version 28658 (0.0007) [2023-12-27 05:51:50,154][20381] Updated weights for policy 1, policy_version 28668 (0.0008) [2023-12-27 05:51:50,635][20347] Updated weights for policy 0, policy_version 28629 (0.0006) [2023-12-27 05:51:50,690][20347] Updated weights for policy 0, policy_version 28639 (0.0005) [2023-12-27 05:51:50,751][20347] Updated weights for policy 0, policy_version 28649 (0.0005) [2023-12-27 05:51:50,876][20381] Updated weights for policy 1, policy_version 28678 (0.0007) [2023-12-27 05:51:50,933][20381] Updated weights for policy 1, policy_version 28688 (0.0007) [2023-12-27 05:51:50,991][20381] Updated weights for policy 1, policy_version 28698 (0.0008) [2023-12-27 05:51:51,385][20347] Updated weights for policy 0, policy_version 28659 (0.0007) [2023-12-27 05:51:51,438][20347] Updated weights for policy 0, policy_version 28669 (0.0008) [2023-12-27 05:51:51,493][20347] Updated weights for policy 0, policy_version 28679 (0.0008) [2023-12-27 05:51:51,667][20381] Updated weights for policy 1, policy_version 28708 (0.0007) [2023-12-27 05:51:51,724][20381] Updated weights for policy 1, policy_version 28718 (0.0006) [2023-12-27 05:51:51,789][20381] Updated weights for policy 1, policy_version 28728 (0.0006) [2023-12-27 05:51:52,238][20347] Updated weights for policy 0, policy_version 28689 (0.0007) [2023-12-27 05:51:52,298][20347] Updated weights for policy 0, policy_version 28699 (0.0008) [2023-12-27 05:51:52,357][20347] Updated weights for policy 0, policy_version 28709 (0.0007) [2023-12-27 05:51:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 14704640. Throughput: 0: 10339.3, 1: 10571.9. Samples: 14679344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:51:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000028736_7356416.pth... [2023-12-27 05:51:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000027520_7045120.pth [2023-12-27 05:51:52,398][20381] Updated weights for policy 1, policy_version 28738 (0.0005) [2023-12-27 05:51:52,412][20347] Updated weights for policy 0, policy_version 28719 (0.0007) [2023-12-27 05:51:52,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000028720_7356416.pth... [2023-12-27 05:51:52,420][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000027472_7036928.pth [2023-12-27 05:51:52,455][20381] Updated weights for policy 1, policy_version 28748 (0.0006) [2023-12-27 05:51:52,504][20381] Updated weights for policy 1, policy_version 28758 (0.0008) [2023-12-27 05:51:52,567][20381] Updated weights for policy 1, policy_version 28768 (0.0007) [2023-12-27 05:51:53,162][20381] Updated weights for policy 1, policy_version 28778 (0.0006) [2023-12-27 05:51:53,167][20347] Updated weights for policy 0, policy_version 28729 (0.0006) [2023-12-27 05:51:53,216][20381] Updated weights for policy 1, policy_version 28788 (0.0007) [2023-12-27 05:51:53,229][20347] Updated weights for policy 0, policy_version 28739 (0.0008) [2023-12-27 05:51:53,277][20381] Updated weights for policy 1, policy_version 28798 (0.0009) [2023-12-27 05:51:53,283][20347] Updated weights for policy 0, policy_version 28749 (0.0005) [2023-12-27 05:51:53,951][20381] Updated weights for policy 1, policy_version 28808 (0.0006) [2023-12-27 05:51:53,956][20347] Updated weights for policy 0, policy_version 28759 (0.0008) [2023-12-27 05:51:54,001][20381] Updated weights for policy 1, policy_version 28818 (0.0007) [2023-12-27 05:51:54,017][20347] Updated weights for policy 0, policy_version 28769 (0.0007) [2023-12-27 05:51:54,059][20381] Updated weights for policy 1, policy_version 28828 (0.0006) [2023-12-27 05:51:54,082][20347] Updated weights for policy 0, policy_version 28779 (0.0006) [2023-12-27 05:51:54,706][20381] Updated weights for policy 1, policy_version 28838 (0.0010) [2023-12-27 05:51:54,738][20347] Updated weights for policy 0, policy_version 28789 (0.0009) [2023-12-27 05:51:54,758][20381] Updated weights for policy 1, policy_version 28848 (0.0010) [2023-12-27 05:51:54,790][20347] Updated weights for policy 0, policy_version 28799 (0.0010) [2023-12-27 05:51:54,804][20381] Updated weights for policy 1, policy_version 28858 (0.0010) [2023-12-27 05:51:54,838][20347] Updated weights for policy 0, policy_version 28809 (0.0010) [2023-12-27 05:51:55,480][20347] Updated weights for policy 0, policy_version 28819 (0.0010) [2023-12-27 05:51:55,488][20381] Updated weights for policy 1, policy_version 28868 (0.0010) [2023-12-27 05:51:55,536][20347] Updated weights for policy 0, policy_version 28829 (0.0005) [2023-12-27 05:51:55,543][20381] Updated weights for policy 1, policy_version 28878 (0.0010) [2023-12-27 05:51:55,587][20347] Updated weights for policy 0, policy_version 28839 (0.0005) [2023-12-27 05:51:55,598][20381] Updated weights for policy 1, policy_version 28888 (0.0010) [2023-12-27 05:51:56,157][20347] Updated weights for policy 0, policy_version 28849 (0.0006) [2023-12-27 05:51:56,209][20347] Updated weights for policy 0, policy_version 28859 (0.0010) [2023-12-27 05:51:56,263][20347] Updated weights for policy 0, policy_version 28869 (0.0010) [2023-12-27 05:51:56,304][20381] Updated weights for policy 1, policy_version 28898 (0.0010) [2023-12-27 05:51:56,320][20347] Updated weights for policy 0, policy_version 28879 (0.0010) [2023-12-27 05:51:56,355][20381] Updated weights for policy 1, policy_version 28908 (0.0010) [2023-12-27 05:51:56,405][20381] Updated weights for policy 1, policy_version 28918 (0.0010) [2023-12-27 05:51:56,455][20381] Updated weights for policy 1, policy_version 28928 (0.0005) [2023-12-27 05:51:57,002][20347] Updated weights for policy 0, policy_version 28889 (0.0008) [2023-12-27 05:51:57,054][20347] Updated weights for policy 0, policy_version 28899 (0.0007) [2023-12-27 05:51:57,099][20347] Updated weights for policy 0, policy_version 28909 (0.0006) [2023-12-27 05:51:57,119][20381] Updated weights for policy 1, policy_version 28938 (0.0010) [2023-12-27 05:51:57,167][20381] Updated weights for policy 1, policy_version 28948 (0.0010) [2023-12-27 05:51:57,224][20381] Updated weights for policy 1, policy_version 28958 (0.0010) [2023-12-27 05:51:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20910.4). Total num frames: 14819328. Throughput: 0: 10327.0, 1: 10606.9. Samples: 14804944. Policy #0 lag: (min: 3.0, avg: 9.1, max: 35.0) [2023-12-27 05:51:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:51:57,785][20347] Updated weights for policy 0, policy_version 28919 (0.0007) [2023-12-27 05:51:57,828][20347] Updated weights for policy 0, policy_version 28929 (0.0007) [2023-12-27 05:51:57,874][20347] Updated weights for policy 0, policy_version 28939 (0.0007) [2023-12-27 05:51:57,918][20381] Updated weights for policy 1, policy_version 28968 (0.0010) [2023-12-27 05:51:57,969][20381] Updated weights for policy 1, policy_version 28978 (0.0010) [2023-12-27 05:51:58,024][20381] Updated weights for policy 1, policy_version 28988 (0.0010) [2023-12-27 05:51:58,599][20347] Updated weights for policy 0, policy_version 28949 (0.0007) [2023-12-27 05:51:58,660][20347] Updated weights for policy 0, policy_version 28959 (0.0007) [2023-12-27 05:51:58,725][20347] Updated weights for policy 0, policy_version 28969 (0.0008) [2023-12-27 05:51:58,728][20381] Updated weights for policy 1, policy_version 28998 (0.0010) [2023-12-27 05:51:58,796][20381] Updated weights for policy 1, policy_version 29008 (0.0008) [2023-12-27 05:51:58,864][20381] Updated weights for policy 1, policy_version 29018 (0.0009) [2023-12-27 05:51:59,467][20347] Updated weights for policy 0, policy_version 28979 (0.0009) [2023-12-27 05:51:59,535][20347] Updated weights for policy 0, policy_version 28989 (0.0008) [2023-12-27 05:51:59,561][20381] Updated weights for policy 1, policy_version 29028 (0.0011) [2023-12-27 05:51:59,590][20347] Updated weights for policy 0, policy_version 28999 (0.0007) [2023-12-27 05:51:59,621][20381] Updated weights for policy 1, policy_version 29038 (0.0011) [2023-12-27 05:51:59,678][20381] Updated weights for policy 1, policy_version 29048 (0.0010) [2023-12-27 05:52:00,235][20347] Updated weights for policy 0, policy_version 29009 (0.0006) [2023-12-27 05:52:00,293][20347] Updated weights for policy 0, policy_version 29019 (0.0005) [2023-12-27 05:52:00,346][20381] Updated weights for policy 1, policy_version 29058 (0.0010) [2023-12-27 05:52:00,351][20347] Updated weights for policy 0, policy_version 29029 (0.0005) [2023-12-27 05:52:00,397][20381] Updated weights for policy 1, policy_version 29068 (0.0007) [2023-12-27 05:52:00,412][20347] Updated weights for policy 0, policy_version 29039 (0.0006) [2023-12-27 05:52:00,453][20381] Updated weights for policy 1, policy_version 29078 (0.0005) [2023-12-27 05:52:00,513][20381] Updated weights for policy 1, policy_version 29088 (0.0005) [2023-12-27 05:52:00,960][20347] Updated weights for policy 0, policy_version 29049 (0.0007) [2023-12-27 05:52:01,026][20347] Updated weights for policy 0, policy_version 29059 (0.0007) [2023-12-27 05:52:01,095][20347] Updated weights for policy 0, policy_version 29069 (0.0006) [2023-12-27 05:52:01,154][20381] Updated weights for policy 1, policy_version 29098 (0.0008) [2023-12-27 05:52:01,208][20381] Updated weights for policy 1, policy_version 29108 (0.0006) [2023-12-27 05:52:01,267][20381] Updated weights for policy 1, policy_version 29118 (0.0008) [2023-12-27 05:52:01,729][20347] Updated weights for policy 0, policy_version 29079 (0.0007) [2023-12-27 05:52:01,793][20347] Updated weights for policy 0, policy_version 29089 (0.0005) [2023-12-27 05:52:01,855][20347] Updated weights for policy 0, policy_version 29099 (0.0006) [2023-12-27 05:52:01,959][20381] Updated weights for policy 1, policy_version 29128 (0.0006) [2023-12-27 05:52:02,025][20381] Updated weights for policy 1, policy_version 29138 (0.0006) [2023-12-27 05:52:02,083][20381] Updated weights for policy 1, policy_version 29148 (0.0006) [2023-12-27 05:52:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 14917632. Throughput: 0: 10366.6, 1: 10514.1. Samples: 14927836. Policy #0 lag: (min: 3.0, avg: 9.1, max: 35.0) [2023-12-27 05:52:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:02,459][20347] Updated weights for policy 0, policy_version 29109 (0.0006) [2023-12-27 05:52:02,524][20347] Updated weights for policy 0, policy_version 29119 (0.0010) [2023-12-27 05:52:02,582][20347] Updated weights for policy 0, policy_version 29129 (0.0010) [2023-12-27 05:52:02,744][20381] Updated weights for policy 1, policy_version 29158 (0.0007) [2023-12-27 05:52:02,801][20381] Updated weights for policy 1, policy_version 29168 (0.0005) [2023-12-27 05:52:02,858][20381] Updated weights for policy 1, policy_version 29178 (0.0005) [2023-12-27 05:52:03,242][20347] Updated weights for policy 0, policy_version 29139 (0.0010) [2023-12-27 05:52:03,293][20347] Updated weights for policy 0, policy_version 29149 (0.0010) [2023-12-27 05:52:03,340][20347] Updated weights for policy 0, policy_version 29159 (0.0010) [2023-12-27 05:52:03,416][20381] Updated weights for policy 1, policy_version 29188 (0.0005) [2023-12-27 05:52:03,471][20381] Updated weights for policy 1, policy_version 29198 (0.0005) [2023-12-27 05:52:03,532][20381] Updated weights for policy 1, policy_version 29208 (0.0008) [2023-12-27 05:52:04,018][20347] Updated weights for policy 0, policy_version 29169 (0.0010) [2023-12-27 05:52:04,083][20347] Updated weights for policy 0, policy_version 29179 (0.0006) [2023-12-27 05:52:04,125][20381] Updated weights for policy 1, policy_version 29218 (0.0009) [2023-12-27 05:52:04,143][20347] Updated weights for policy 0, policy_version 29189 (0.0006) [2023-12-27 05:52:04,184][20381] Updated weights for policy 1, policy_version 29229 (0.0010) [2023-12-27 05:52:04,198][20347] Updated weights for policy 0, policy_version 29199 (0.0008) [2023-12-27 05:52:04,238][20381] Updated weights for policy 1, policy_version 29239 (0.0009) [2023-12-27 05:52:04,870][20347] Updated weights for policy 0, policy_version 29209 (0.0008) [2023-12-27 05:52:04,922][20347] Updated weights for policy 0, policy_version 29219 (0.0010) [2023-12-27 05:52:04,980][20347] Updated weights for policy 0, policy_version 29229 (0.0010) [2023-12-27 05:52:04,990][20381] Updated weights for policy 1, policy_version 29249 (0.0008) [2023-12-27 05:52:05,050][20381] Updated weights for policy 1, policy_version 29259 (0.0006) [2023-12-27 05:52:05,107][20381] Updated weights for policy 1, policy_version 29269 (0.0005) [2023-12-27 05:52:05,169][20381] Updated weights for policy 1, policy_version 29279 (0.0007) [2023-12-27 05:52:05,599][20347] Updated weights for policy 0, policy_version 29239 (0.0008) [2023-12-27 05:52:05,648][20347] Updated weights for policy 0, policy_version 29249 (0.0010) [2023-12-27 05:52:05,702][20347] Updated weights for policy 0, policy_version 29259 (0.0009) [2023-12-27 05:52:05,800][20381] Updated weights for policy 1, policy_version 29289 (0.0007) [2023-12-27 05:52:05,857][20381] Updated weights for policy 1, policy_version 29299 (0.0008) [2023-12-27 05:52:05,913][20381] Updated weights for policy 1, policy_version 29309 (0.0007) [2023-12-27 05:52:06,418][20347] Updated weights for policy 0, policy_version 29269 (0.0009) [2023-12-27 05:52:06,477][20347] Updated weights for policy 0, policy_version 29279 (0.0010) [2023-12-27 05:52:06,539][20347] Updated weights for policy 0, policy_version 29289 (0.0010) [2023-12-27 05:52:06,563][20381] Updated weights for policy 1, policy_version 29319 (0.0007) [2023-12-27 05:52:06,615][20381] Updated weights for policy 1, policy_version 29329 (0.0008) [2023-12-27 05:52:06,664][20381] Updated weights for policy 1, policy_version 29339 (0.0007) [2023-12-27 05:52:07,135][20347] Updated weights for policy 0, policy_version 29299 (0.0009) [2023-12-27 05:52:07,191][20347] Updated weights for policy 0, policy_version 29309 (0.0007) [2023-12-27 05:52:07,255][20347] Updated weights for policy 0, policy_version 29319 (0.0008) [2023-12-27 05:52:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 15024128. Throughput: 0: 10392.7, 1: 10516.6. Samples: 14991296. Policy #0 lag: (min: 3.0, avg: 9.1, max: 35.0) [2023-12-27 05:52:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000029344_7512064.pth... [2023-12-27 05:52:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000029328_7512064.pth... [2023-12-27 05:52:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000028160_7208960.pth [2023-12-27 05:52:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000028080_7192576.pth [2023-12-27 05:52:07,421][20381] Updated weights for policy 1, policy_version 29349 (0.0008) [2023-12-27 05:52:07,466][20381] Updated weights for policy 1, policy_version 29359 (0.0008) [2023-12-27 05:52:07,515][20381] Updated weights for policy 1, policy_version 29369 (0.0008) [2023-12-27 05:52:07,945][20347] Updated weights for policy 0, policy_version 29329 (0.0007) [2023-12-27 05:52:08,007][20347] Updated weights for policy 0, policy_version 29339 (0.0007) [2023-12-27 05:52:08,090][20347] Updated weights for policy 0, policy_version 29349 (0.0005) [2023-12-27 05:52:08,148][20347] Updated weights for policy 0, policy_version 29359 (0.0005) [2023-12-27 05:52:08,165][20381] Updated weights for policy 1, policy_version 29379 (0.0008) [2023-12-27 05:52:08,223][20381] Updated weights for policy 1, policy_version 29390 (0.0009) [2023-12-27 05:52:08,280][20381] Updated weights for policy 1, policy_version 29400 (0.0008) [2023-12-27 05:52:08,723][20347] Updated weights for policy 0, policy_version 29369 (0.0007) [2023-12-27 05:52:08,779][20347] Updated weights for policy 0, policy_version 29379 (0.0008) [2023-12-27 05:52:08,835][20347] Updated weights for policy 0, policy_version 29389 (0.0008) [2023-12-27 05:52:08,995][20381] Updated weights for policy 1, policy_version 29410 (0.0008) [2023-12-27 05:52:09,047][20381] Updated weights for policy 1, policy_version 29420 (0.0009) [2023-12-27 05:52:09,104][20381] Updated weights for policy 1, policy_version 29430 (0.0009) [2023-12-27 05:52:09,153][20381] Updated weights for policy 1, policy_version 29440 (0.0008) [2023-12-27 05:52:09,479][20347] Updated weights for policy 0, policy_version 29399 (0.0008) [2023-12-27 05:52:09,540][20347] Updated weights for policy 0, policy_version 29409 (0.0008) [2023-12-27 05:52:09,599][20347] Updated weights for policy 0, policy_version 29419 (0.0007) [2023-12-27 05:52:09,899][20381] Updated weights for policy 1, policy_version 29450 (0.0008) [2023-12-27 05:52:09,963][20381] Updated weights for policy 1, policy_version 29460 (0.0007) [2023-12-27 05:52:10,028][20381] Updated weights for policy 1, policy_version 29470 (0.0007) [2023-12-27 05:52:10,259][20347] Updated weights for policy 0, policy_version 29429 (0.0007) [2023-12-27 05:52:10,314][20347] Updated weights for policy 0, policy_version 29439 (0.0005) [2023-12-27 05:52:10,376][20347] Updated weights for policy 0, policy_version 29449 (0.0007) [2023-12-27 05:52:10,729][20381] Updated weights for policy 1, policy_version 29480 (0.0008) [2023-12-27 05:52:10,790][20381] Updated weights for policy 1, policy_version 29490 (0.0007) [2023-12-27 05:52:10,845][20381] Updated weights for policy 1, policy_version 29500 (0.0007) [2023-12-27 05:52:11,040][20347] Updated weights for policy 0, policy_version 29459 (0.0010) [2023-12-27 05:52:11,090][20347] Updated weights for policy 0, policy_version 29469 (0.0010) [2023-12-27 05:52:11,154][20347] Updated weights for policy 0, policy_version 29479 (0.0010) [2023-12-27 05:52:11,594][20381] Updated weights for policy 1, policy_version 29510 (0.0007) [2023-12-27 05:52:11,662][20381] Updated weights for policy 1, policy_version 29520 (0.0008) [2023-12-27 05:52:11,718][20381] Updated weights for policy 1, policy_version 29530 (0.0008) [2023-12-27 05:52:11,866][20347] Updated weights for policy 0, policy_version 29489 (0.0010) [2023-12-27 05:52:11,915][20347] Updated weights for policy 0, policy_version 29499 (0.0010) [2023-12-27 05:52:11,966][20347] Updated weights for policy 0, policy_version 29509 (0.0010) [2023-12-27 05:52:12,021][20347] Updated weights for policy 0, policy_version 29519 (0.0010) [2023-12-27 05:52:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 15122432. Throughput: 0: 10467.9, 1: 10398.3. Samples: 15115296. Policy #0 lag: (min: 3.0, avg: 9.1, max: 35.0) [2023-12-27 05:52:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:12,433][20381] Updated weights for policy 1, policy_version 29540 (0.0006) [2023-12-27 05:52:12,490][20381] Updated weights for policy 1, policy_version 29550 (0.0007) [2023-12-27 05:52:12,541][20381] Updated weights for policy 1, policy_version 29560 (0.0008) [2023-12-27 05:52:12,682][20347] Updated weights for policy 0, policy_version 29529 (0.0007) [2023-12-27 05:52:12,746][20347] Updated weights for policy 0, policy_version 29539 (0.0008) [2023-12-27 05:52:12,809][20347] Updated weights for policy 0, policy_version 29549 (0.0008) [2023-12-27 05:52:13,177][20381] Updated weights for policy 1, policy_version 29570 (0.0006) [2023-12-27 05:52:13,235][20381] Updated weights for policy 1, policy_version 29580 (0.0005) [2023-12-27 05:52:13,296][20381] Updated weights for policy 1, policy_version 29590 (0.0005) [2023-12-27 05:52:13,346][20381] Updated weights for policy 1, policy_version 29600 (0.0008) [2023-12-27 05:52:13,411][20347] Updated weights for policy 0, policy_version 29559 (0.0007) [2023-12-27 05:52:13,470][20347] Updated weights for policy 0, policy_version 29569 (0.0007) [2023-12-27 05:52:13,528][20347] Updated weights for policy 0, policy_version 29579 (0.0010) [2023-12-27 05:52:13,951][20381] Updated weights for policy 1, policy_version 29610 (0.0005) [2023-12-27 05:52:14,010][20381] Updated weights for policy 1, policy_version 29620 (0.0007) [2023-12-27 05:52:14,055][20381] Updated weights for policy 1, policy_version 29630 (0.0008) [2023-12-27 05:52:14,176][20347] Updated weights for policy 0, policy_version 29589 (0.0009) [2023-12-27 05:52:14,222][20347] Updated weights for policy 0, policy_version 29599 (0.0007) [2023-12-27 05:52:14,266][20347] Updated weights for policy 0, policy_version 29609 (0.0007) [2023-12-27 05:52:14,724][20381] Updated weights for policy 1, policy_version 29640 (0.0006) [2023-12-27 05:52:14,785][20381] Updated weights for policy 1, policy_version 29650 (0.0006) [2023-12-27 05:52:14,845][20381] Updated weights for policy 1, policy_version 29660 (0.0007) [2023-12-27 05:52:14,997][20347] Updated weights for policy 0, policy_version 29619 (0.0007) [2023-12-27 05:52:15,062][20347] Updated weights for policy 0, policy_version 29629 (0.0005) [2023-12-27 05:52:15,120][20347] Updated weights for policy 0, policy_version 29639 (0.0006) [2023-12-27 05:52:15,501][20381] Updated weights for policy 1, policy_version 29670 (0.0007) [2023-12-27 05:52:15,554][20381] Updated weights for policy 1, policy_version 29680 (0.0008) [2023-12-27 05:52:15,610][20381] Updated weights for policy 1, policy_version 29690 (0.0008) [2023-12-27 05:52:15,734][20347] Updated weights for policy 0, policy_version 29649 (0.0008) [2023-12-27 05:52:15,798][20347] Updated weights for policy 0, policy_version 29659 (0.0009) [2023-12-27 05:52:15,851][20347] Updated weights for policy 0, policy_version 29669 (0.0008) [2023-12-27 05:52:15,915][20347] Updated weights for policy 0, policy_version 29679 (0.0005) [2023-12-27 05:52:16,350][20381] Updated weights for policy 1, policy_version 29700 (0.0009) [2023-12-27 05:52:16,404][20381] Updated weights for policy 1, policy_version 29710 (0.0007) [2023-12-27 05:52:16,455][20381] Updated weights for policy 1, policy_version 29720 (0.0007) [2023-12-27 05:52:16,456][20347] Updated weights for policy 0, policy_version 29689 (0.0006) [2023-12-27 05:52:16,517][20347] Updated weights for policy 0, policy_version 29699 (0.0005) [2023-12-27 05:52:16,568][20347] Updated weights for policy 0, policy_version 29709 (0.0006) [2023-12-27 05:52:17,099][20347] Updated weights for policy 0, policy_version 29719 (0.0007) [2023-12-27 05:52:17,167][20347] Updated weights for policy 0, policy_version 29729 (0.0008) [2023-12-27 05:52:17,203][20381] Updated weights for policy 1, policy_version 29730 (0.0007) [2023-12-27 05:52:17,217][20347] Updated weights for policy 0, policy_version 29739 (0.0007) [2023-12-27 05:52:17,256][20381] Updated weights for policy 1, policy_version 29740 (0.0006) [2023-12-27 05:52:17,315][20381] Updated weights for policy 1, policy_version 29750 (0.0008) [2023-12-27 05:52:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 15228928. Throughput: 0: 10529.2, 1: 10343.7. Samples: 15241596. Policy #0 lag: (min: 31.0, avg: 31.1, max: 40.0) [2023-12-27 05:52:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:17,367][20381] Updated weights for policy 1, policy_version 29760 (0.0008) [2023-12-27 05:52:17,897][20347] Updated weights for policy 0, policy_version 29749 (0.0005) [2023-12-27 05:52:17,948][20347] Updated weights for policy 0, policy_version 29759 (0.0005) [2023-12-27 05:52:17,983][20381] Updated weights for policy 1, policy_version 29770 (0.0006) [2023-12-27 05:52:18,003][20347] Updated weights for policy 0, policy_version 29769 (0.0006) [2023-12-27 05:52:18,039][20381] Updated weights for policy 1, policy_version 29780 (0.0008) [2023-12-27 05:52:18,085][20381] Updated weights for policy 1, policy_version 29790 (0.0007) [2023-12-27 05:52:18,638][20347] Updated weights for policy 0, policy_version 29779 (0.0006) [2023-12-27 05:52:18,688][20347] Updated weights for policy 0, policy_version 29789 (0.0007) [2023-12-27 05:52:18,744][20347] Updated weights for policy 0, policy_version 29799 (0.0006) [2023-12-27 05:52:18,760][20381] Updated weights for policy 1, policy_version 29800 (0.0007) [2023-12-27 05:52:18,820][20381] Updated weights for policy 1, policy_version 29810 (0.0006) [2023-12-27 05:52:18,881][20381] Updated weights for policy 1, policy_version 29820 (0.0008) [2023-12-27 05:52:19,468][20347] Updated weights for policy 0, policy_version 29809 (0.0006) [2023-12-27 05:52:19,523][20381] Updated weights for policy 1, policy_version 29830 (0.0006) [2023-12-27 05:52:19,528][20347] Updated weights for policy 0, policy_version 29819 (0.0007) [2023-12-27 05:52:19,581][20381] Updated weights for policy 1, policy_version 29840 (0.0006) [2023-12-27 05:52:19,582][20347] Updated weights for policy 0, policy_version 29829 (0.0006) [2023-12-27 05:52:19,635][20381] Updated weights for policy 1, policy_version 29850 (0.0005) [2023-12-27 05:52:19,640][20347] Updated weights for policy 0, policy_version 29839 (0.0007) [2023-12-27 05:52:20,358][20347] Updated weights for policy 0, policy_version 29849 (0.0007) [2023-12-27 05:52:20,371][20381] Updated weights for policy 1, policy_version 29860 (0.0007) [2023-12-27 05:52:20,413][20347] Updated weights for policy 0, policy_version 29859 (0.0006) [2023-12-27 05:52:20,427][20381] Updated weights for policy 1, policy_version 29870 (0.0005) [2023-12-27 05:52:20,465][20347] Updated weights for policy 0, policy_version 29869 (0.0007) [2023-12-27 05:52:20,478][20381] Updated weights for policy 1, policy_version 29880 (0.0006) [2023-12-27 05:52:21,126][20381] Updated weights for policy 1, policy_version 29890 (0.0008) [2023-12-27 05:52:21,147][20347] Updated weights for policy 0, policy_version 29879 (0.0006) [2023-12-27 05:52:21,191][20381] Updated weights for policy 1, policy_version 29900 (0.0008) [2023-12-27 05:52:21,211][20347] Updated weights for policy 0, policy_version 29889 (0.0007) [2023-12-27 05:52:21,257][20381] Updated weights for policy 1, policy_version 29910 (0.0006) [2023-12-27 05:52:21,274][20347] Updated weights for policy 0, policy_version 29899 (0.0007) [2023-12-27 05:52:21,316][20381] Updated weights for policy 1, policy_version 29920 (0.0006) [2023-12-27 05:52:21,965][20347] Updated weights for policy 0, policy_version 29909 (0.0007) [2023-12-27 05:52:21,979][20381] Updated weights for policy 1, policy_version 29930 (0.0010) [2023-12-27 05:52:22,023][20347] Updated weights for policy 0, policy_version 29919 (0.0007) [2023-12-27 05:52:22,035][20381] Updated weights for policy 1, policy_version 29940 (0.0007) [2023-12-27 05:52:22,083][20347] Updated weights for policy 0, policy_version 29929 (0.0006) [2023-12-27 05:52:22,091][20381] Updated weights for policy 1, policy_version 29950 (0.0008) [2023-12-27 05:52:22,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 20910.4). Total num frames: 15335424. Throughput: 0: 10515.3, 1: 10303.0. Samples: 15303036. Policy #0 lag: (min: 31.0, avg: 31.1, max: 40.0) [2023-12-27 05:52:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000029952_7667712.pth... [2023-12-27 05:52:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000029936_7667712.pth... [2023-12-27 05:52:22,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000028720_7356416.pth [2023-12-27 05:52:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000028736_7356416.pth [2023-12-27 05:52:22,796][20347] Updated weights for policy 0, policy_version 29939 (0.0008) [2023-12-27 05:52:22,807][20381] Updated weights for policy 1, policy_version 29960 (0.0010) [2023-12-27 05:52:22,855][20347] Updated weights for policy 0, policy_version 29949 (0.0007) [2023-12-27 05:52:22,864][20381] Updated weights for policy 1, policy_version 29970 (0.0011) [2023-12-27 05:52:22,917][20347] Updated weights for policy 0, policy_version 29959 (0.0006) [2023-12-27 05:52:22,919][20381] Updated weights for policy 1, policy_version 29980 (0.0006) [2023-12-27 05:52:23,508][20381] Updated weights for policy 1, policy_version 29990 (0.0007) [2023-12-27 05:52:23,565][20381] Updated weights for policy 1, policy_version 30000 (0.0008) [2023-12-27 05:52:23,620][20381] Updated weights for policy 1, policy_version 30010 (0.0007) [2023-12-27 05:52:23,654][20347] Updated weights for policy 0, policy_version 29969 (0.0007) [2023-12-27 05:52:23,706][20347] Updated weights for policy 0, policy_version 29979 (0.0007) [2023-12-27 05:52:23,769][20347] Updated weights for policy 0, policy_version 29989 (0.0008) [2023-12-27 05:52:23,825][20347] Updated weights for policy 0, policy_version 29999 (0.0008) [2023-12-27 05:52:24,333][20381] Updated weights for policy 1, policy_version 30020 (0.0007) [2023-12-27 05:52:24,393][20381] Updated weights for policy 1, policy_version 30030 (0.0009) [2023-12-27 05:52:24,437][20347] Updated weights for policy 0, policy_version 30009 (0.0008) [2023-12-27 05:52:24,448][20381] Updated weights for policy 1, policy_version 30040 (0.0009) [2023-12-27 05:52:24,493][20347] Updated weights for policy 0, policy_version 30019 (0.0010) [2023-12-27 05:52:24,551][20347] Updated weights for policy 0, policy_version 30029 (0.0010) [2023-12-27 05:52:25,113][20381] Updated weights for policy 1, policy_version 30050 (0.0008) [2023-12-27 05:52:25,154][20347] Updated weights for policy 0, policy_version 30039 (0.0010) [2023-12-27 05:52:25,167][20381] Updated weights for policy 1, policy_version 30060 (0.0006) [2023-12-27 05:52:25,205][20347] Updated weights for policy 0, policy_version 30049 (0.0008) [2023-12-27 05:52:25,222][20381] Updated weights for policy 1, policy_version 30070 (0.0006) [2023-12-27 05:52:25,254][20347] Updated weights for policy 0, policy_version 30059 (0.0010) [2023-12-27 05:52:25,274][20381] Updated weights for policy 1, policy_version 30080 (0.0005) [2023-12-27 05:52:25,921][20347] Updated weights for policy 0, policy_version 30069 (0.0009) [2023-12-27 05:52:25,947][20381] Updated weights for policy 1, policy_version 30090 (0.0007) [2023-12-27 05:52:25,973][20347] Updated weights for policy 0, policy_version 30079 (0.0005) [2023-12-27 05:52:26,011][20381] Updated weights for policy 1, policy_version 30100 (0.0007) [2023-12-27 05:52:26,025][20347] Updated weights for policy 0, policy_version 30089 (0.0006) [2023-12-27 05:52:26,064][20381] Updated weights for policy 1, policy_version 30110 (0.0006) [2023-12-27 05:52:26,586][20347] Updated weights for policy 0, policy_version 30099 (0.0005) [2023-12-27 05:52:26,636][20347] Updated weights for policy 0, policy_version 30109 (0.0007) [2023-12-27 05:52:26,687][20347] Updated weights for policy 0, policy_version 30119 (0.0008) [2023-12-27 05:52:26,737][20381] Updated weights for policy 1, policy_version 30120 (0.0008) [2023-12-27 05:52:26,792][20381] Updated weights for policy 1, policy_version 30130 (0.0008) [2023-12-27 05:52:26,846][20381] Updated weights for policy 1, policy_version 30140 (0.0005) [2023-12-27 05:52:27,317][20347] Updated weights for policy 0, policy_version 30129 (0.0008) [2023-12-27 05:52:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 15433728. Throughput: 0: 10488.3, 1: 10293.0. Samples: 15426376. Policy #0 lag: (min: 31.0, avg: 31.1, max: 40.0) [2023-12-27 05:52:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:27,368][20347] Updated weights for policy 0, policy_version 30139 (0.0005) [2023-12-27 05:52:27,422][20347] Updated weights for policy 0, policy_version 30149 (0.0005) [2023-12-27 05:52:27,458][20381] Updated weights for policy 1, policy_version 30150 (0.0008) [2023-12-27 05:52:27,482][20347] Updated weights for policy 0, policy_version 30159 (0.0006) [2023-12-27 05:52:27,507][20381] Updated weights for policy 1, policy_version 30160 (0.0007) [2023-12-27 05:52:27,570][20381] Updated weights for policy 1, policy_version 30170 (0.0005) [2023-12-27 05:52:28,033][20347] Updated weights for policy 0, policy_version 30169 (0.0007) [2023-12-27 05:52:28,084][20347] Updated weights for policy 0, policy_version 30179 (0.0007) [2023-12-27 05:52:28,139][20347] Updated weights for policy 0, policy_version 30189 (0.0006) [2023-12-27 05:52:28,220][20381] Updated weights for policy 1, policy_version 30180 (0.0008) [2023-12-27 05:52:28,274][20381] Updated weights for policy 1, policy_version 30190 (0.0005) [2023-12-27 05:52:28,334][20381] Updated weights for policy 1, policy_version 30200 (0.0010) [2023-12-27 05:52:28,836][20347] Updated weights for policy 0, policy_version 30199 (0.0007) [2023-12-27 05:52:28,893][20347] Updated weights for policy 0, policy_version 30209 (0.0008) [2023-12-27 05:52:28,948][20347] Updated weights for policy 0, policy_version 30219 (0.0007) [2023-12-27 05:52:28,968][20381] Updated weights for policy 1, policy_version 30210 (0.0010) [2023-12-27 05:52:29,021][20381] Updated weights for policy 1, policy_version 30220 (0.0009) [2023-12-27 05:52:29,077][20381] Updated weights for policy 1, policy_version 30230 (0.0005) [2023-12-27 05:52:29,130][20381] Updated weights for policy 1, policy_version 30240 (0.0005) [2023-12-27 05:52:29,596][20347] Updated weights for policy 0, policy_version 30229 (0.0007) [2023-12-27 05:52:29,657][20347] Updated weights for policy 0, policy_version 30239 (0.0005) [2023-12-27 05:52:29,714][20347] Updated weights for policy 0, policy_version 30249 (0.0005) [2023-12-27 05:52:29,740][20381] Updated weights for policy 1, policy_version 30250 (0.0008) [2023-12-27 05:52:29,788][20381] Updated weights for policy 1, policy_version 30260 (0.0010) [2023-12-27 05:52:29,847][20381] Updated weights for policy 1, policy_version 30270 (0.0009) [2023-12-27 05:52:30,347][20347] Updated weights for policy 0, policy_version 30259 (0.0006) [2023-12-27 05:52:30,395][20347] Updated weights for policy 0, policy_version 30269 (0.0008) [2023-12-27 05:52:30,439][20347] Updated weights for policy 0, policy_version 30279 (0.0007) [2023-12-27 05:52:30,496][20381] Updated weights for policy 1, policy_version 30280 (0.0007) [2023-12-27 05:52:30,555][20381] Updated weights for policy 1, policy_version 30290 (0.0009) [2023-12-27 05:52:30,610][20381] Updated weights for policy 1, policy_version 30300 (0.0010) [2023-12-27 05:52:31,100][20347] Updated weights for policy 0, policy_version 30289 (0.0010) [2023-12-27 05:52:31,162][20347] Updated weights for policy 0, policy_version 30299 (0.0008) [2023-12-27 05:52:31,199][20381] Updated weights for policy 1, policy_version 30310 (0.0009) [2023-12-27 05:52:31,221][20347] Updated weights for policy 0, policy_version 30309 (0.0005) [2023-12-27 05:52:31,252][20381] Updated weights for policy 1, policy_version 30320 (0.0010) [2023-12-27 05:52:31,285][20347] Updated weights for policy 0, policy_version 30319 (0.0010) [2023-12-27 05:52:31,308][20381] Updated weights for policy 1, policy_version 30330 (0.0011) [2023-12-27 05:52:31,898][20347] Updated weights for policy 0, policy_version 30329 (0.0006) [2023-12-27 05:52:31,944][20347] Updated weights for policy 0, policy_version 30339 (0.0005) [2023-12-27 05:52:32,000][20347] Updated weights for policy 0, policy_version 30349 (0.0006) [2023-12-27 05:52:32,062][20381] Updated weights for policy 1, policy_version 30340 (0.0010) [2023-12-27 05:52:32,125][20381] Updated weights for policy 1, policy_version 30350 (0.0007) [2023-12-27 05:52:32,188][20381] Updated weights for policy 1, policy_version 30360 (0.0008) [2023-12-27 05:52:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 15548416. Throughput: 0: 10569.2, 1: 10290.9. Samples: 15556204. Policy #0 lag: (min: 31.0, avg: 31.1, max: 40.0) [2023-12-27 05:52:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:32,600][20347] Updated weights for policy 0, policy_version 30359 (0.0010) [2023-12-27 05:52:32,660][20347] Updated weights for policy 0, policy_version 30369 (0.0010) [2023-12-27 05:52:32,725][20347] Updated weights for policy 0, policy_version 30379 (0.0010) [2023-12-27 05:52:32,798][20381] Updated weights for policy 1, policy_version 30370 (0.0007) [2023-12-27 05:52:32,849][20381] Updated weights for policy 1, policy_version 30380 (0.0009) [2023-12-27 05:52:32,906][20381] Updated weights for policy 1, policy_version 30390 (0.0006) [2023-12-27 05:52:32,961][20381] Updated weights for policy 1, policy_version 30400 (0.0008) [2023-12-27 05:52:33,297][20347] Updated weights for policy 0, policy_version 30389 (0.0010) [2023-12-27 05:52:33,348][20347] Updated weights for policy 0, policy_version 30399 (0.0010) [2023-12-27 05:52:33,414][20347] Updated weights for policy 0, policy_version 30409 (0.0010) [2023-12-27 05:52:33,616][20381] Updated weights for policy 1, policy_version 30410 (0.0005) [2023-12-27 05:52:33,672][20381] Updated weights for policy 1, policy_version 30420 (0.0007) [2023-12-27 05:52:33,720][20381] Updated weights for policy 1, policy_version 30430 (0.0007) [2023-12-27 05:52:33,967][20347] Updated weights for policy 0, policy_version 30419 (0.0007) [2023-12-27 05:52:34,015][20347] Updated weights for policy 0, policy_version 30429 (0.0005) [2023-12-27 05:52:34,057][20347] Updated weights for policy 0, policy_version 30439 (0.0005) [2023-12-27 05:52:34,415][20381] Updated weights for policy 1, policy_version 30440 (0.0006) [2023-12-27 05:52:34,480][20381] Updated weights for policy 1, policy_version 30450 (0.0006) [2023-12-27 05:52:34,542][20381] Updated weights for policy 1, policy_version 30460 (0.0007) [2023-12-27 05:52:34,743][20347] Updated weights for policy 0, policy_version 30449 (0.0007) [2023-12-27 05:52:34,800][20347] Updated weights for policy 0, policy_version 30459 (0.0010) [2023-12-27 05:52:34,857][20347] Updated weights for policy 0, policy_version 30469 (0.0010) [2023-12-27 05:52:34,917][20347] Updated weights for policy 0, policy_version 30479 (0.0010) [2023-12-27 05:52:35,212][20381] Updated weights for policy 1, policy_version 30470 (0.0008) [2023-12-27 05:52:35,264][20381] Updated weights for policy 1, policy_version 30480 (0.0008) [2023-12-27 05:52:35,315][20381] Updated weights for policy 1, policy_version 30490 (0.0009) [2023-12-27 05:52:35,528][20347] Updated weights for policy 0, policy_version 30489 (0.0011) [2023-12-27 05:52:35,582][20347] Updated weights for policy 0, policy_version 30499 (0.0007) [2023-12-27 05:52:35,638][20347] Updated weights for policy 0, policy_version 30509 (0.0006) [2023-12-27 05:52:36,008][20381] Updated weights for policy 1, policy_version 30500 (0.0007) [2023-12-27 05:52:36,063][20381] Updated weights for policy 1, policy_version 30510 (0.0005) [2023-12-27 05:52:36,118][20381] Updated weights for policy 1, policy_version 30520 (0.0007) [2023-12-27 05:52:36,267][20347] Updated weights for policy 0, policy_version 30519 (0.0007) [2023-12-27 05:52:36,318][20347] Updated weights for policy 0, policy_version 30529 (0.0006) [2023-12-27 05:52:36,366][20347] Updated weights for policy 0, policy_version 30539 (0.0006) [2023-12-27 05:52:36,711][20381] Updated weights for policy 1, policy_version 30530 (0.0006) [2023-12-27 05:52:36,768][20381] Updated weights for policy 1, policy_version 30540 (0.0008) [2023-12-27 05:52:36,824][20381] Updated weights for policy 1, policy_version 30550 (0.0008) [2023-12-27 05:52:36,875][20381] Updated weights for policy 1, policy_version 30560 (0.0007) [2023-12-27 05:52:37,012][20347] Updated weights for policy 0, policy_version 30549 (0.0007) [2023-12-27 05:52:37,063][20347] Updated weights for policy 0, policy_version 30559 (0.0007) [2023-12-27 05:52:37,115][20347] Updated weights for policy 0, policy_version 30569 (0.0006) [2023-12-27 05:52:37,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.2, 300 sec: 20882.7). Total num frames: 15654912. Throughput: 0: 10605.9, 1: 10332.4. Samples: 15621564. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-12-27 05:52:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000030560_7823360.pth... [2023-12-27 05:52:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000030576_7831552.pth... [2023-12-27 05:52:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000029344_7512064.pth [2023-12-27 05:52:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000029328_7512064.pth [2023-12-27 05:52:37,626][20381] Updated weights for policy 1, policy_version 30570 (0.0007) [2023-12-27 05:52:37,691][20381] Updated weights for policy 1, policy_version 30580 (0.0008) [2023-12-27 05:52:37,718][20347] Updated weights for policy 0, policy_version 30579 (0.0005) [2023-12-27 05:52:37,759][20381] Updated weights for policy 1, policy_version 30590 (0.0009) [2023-12-27 05:52:37,769][20347] Updated weights for policy 0, policy_version 30589 (0.0005) [2023-12-27 05:52:37,823][20347] Updated weights for policy 0, policy_version 30599 (0.0005) [2023-12-27 05:52:38,437][20381] Updated weights for policy 1, policy_version 30600 (0.0006) [2023-12-27 05:52:38,466][20347] Updated weights for policy 0, policy_version 30609 (0.0005) [2023-12-27 05:52:38,498][20381] Updated weights for policy 1, policy_version 30610 (0.0006) [2023-12-27 05:52:38,521][20347] Updated weights for policy 0, policy_version 30619 (0.0007) [2023-12-27 05:52:38,552][20381] Updated weights for policy 1, policy_version 30620 (0.0005) [2023-12-27 05:52:38,573][20347] Updated weights for policy 0, policy_version 30629 (0.0007) [2023-12-27 05:52:38,634][20347] Updated weights for policy 0, policy_version 30639 (0.0007) [2023-12-27 05:52:39,129][20381] Updated weights for policy 1, policy_version 30630 (0.0006) [2023-12-27 05:52:39,179][20381] Updated weights for policy 1, policy_version 30640 (0.0007) [2023-12-27 05:52:39,241][20381] Updated weights for policy 1, policy_version 30650 (0.0008) [2023-12-27 05:52:39,262][20347] Updated weights for policy 0, policy_version 30649 (0.0006) [2023-12-27 05:52:39,325][20347] Updated weights for policy 0, policy_version 30659 (0.0006) [2023-12-27 05:52:39,391][20347] Updated weights for policy 0, policy_version 30669 (0.0008) [2023-12-27 05:52:39,955][20381] Updated weights for policy 1, policy_version 30660 (0.0008) [2023-12-27 05:52:40,019][20381] Updated weights for policy 1, policy_version 30670 (0.0008) [2023-12-27 05:52:40,023][20347] Updated weights for policy 0, policy_version 30679 (0.0007) [2023-12-27 05:52:40,080][20381] Updated weights for policy 1, policy_version 30680 (0.0008) [2023-12-27 05:52:40,080][20347] Updated weights for policy 0, policy_version 30689 (0.0006) [2023-12-27 05:52:40,139][20347] Updated weights for policy 0, policy_version 30699 (0.0006) [2023-12-27 05:52:40,731][20347] Updated weights for policy 0, policy_version 30709 (0.0005) [2023-12-27 05:52:40,780][20347] Updated weights for policy 0, policy_version 30719 (0.0005) [2023-12-27 05:52:40,822][20381] Updated weights for policy 1, policy_version 30690 (0.0007) [2023-12-27 05:52:40,838][20347] Updated weights for policy 0, policy_version 30729 (0.0008) [2023-12-27 05:52:40,871][20381] Updated weights for policy 1, policy_version 30700 (0.0006) [2023-12-27 05:52:40,922][20381] Updated weights for policy 1, policy_version 30710 (0.0007) [2023-12-27 05:52:40,968][20381] Updated weights for policy 1, policy_version 30720 (0.0008) [2023-12-27 05:52:41,532][20347] Updated weights for policy 0, policy_version 30739 (0.0007) [2023-12-27 05:52:41,590][20347] Updated weights for policy 0, policy_version 30749 (0.0007) [2023-12-27 05:52:41,650][20347] Updated weights for policy 0, policy_version 30759 (0.0009) [2023-12-27 05:52:41,678][20381] Updated weights for policy 1, policy_version 30730 (0.0007) [2023-12-27 05:52:41,734][20381] Updated weights for policy 1, policy_version 30740 (0.0008) [2023-12-27 05:52:41,798][20381] Updated weights for policy 1, policy_version 30750 (0.0011) [2023-12-27 05:52:42,348][20347] Updated weights for policy 0, policy_version 30769 (0.0006) [2023-12-27 05:52:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 15753216. Throughput: 0: 10677.3, 1: 10284.6. Samples: 15748232. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-12-27 05:52:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:42,414][20347] Updated weights for policy 0, policy_version 30779 (0.0007) [2023-12-27 05:52:42,440][20381] Updated weights for policy 1, policy_version 30760 (0.0008) [2023-12-27 05:52:42,472][20347] Updated weights for policy 0, policy_version 30789 (0.0008) [2023-12-27 05:52:42,503][20381] Updated weights for policy 1, policy_version 30770 (0.0005) [2023-12-27 05:52:42,521][20347] Updated weights for policy 0, policy_version 30799 (0.0007) [2023-12-27 05:52:42,562][20381] Updated weights for policy 1, policy_version 30780 (0.0007) [2023-12-27 05:52:43,135][20347] Updated weights for policy 0, policy_version 30809 (0.0007) [2023-12-27 05:52:43,186][20381] Updated weights for policy 1, policy_version 30790 (0.0006) [2023-12-27 05:52:43,191][20347] Updated weights for policy 0, policy_version 30819 (0.0007) [2023-12-27 05:52:43,243][20381] Updated weights for policy 1, policy_version 30800 (0.0006) [2023-12-27 05:52:43,245][20347] Updated weights for policy 0, policy_version 30829 (0.0008) [2023-12-27 05:52:43,311][20381] Updated weights for policy 1, policy_version 30810 (0.0005) [2023-12-27 05:52:43,878][20381] Updated weights for policy 1, policy_version 30820 (0.0007) [2023-12-27 05:52:43,901][20347] Updated weights for policy 0, policy_version 30839 (0.0006) [2023-12-27 05:52:43,933][20381] Updated weights for policy 1, policy_version 30830 (0.0006) [2023-12-27 05:52:43,964][20347] Updated weights for policy 0, policy_version 30849 (0.0005) [2023-12-27 05:52:43,989][20381] Updated weights for policy 1, policy_version 30840 (0.0006) [2023-12-27 05:52:44,030][20347] Updated weights for policy 0, policy_version 30859 (0.0005) [2023-12-27 05:52:44,610][20347] Updated weights for policy 0, policy_version 30869 (0.0006) [2023-12-27 05:52:44,622][20381] Updated weights for policy 1, policy_version 30850 (0.0006) [2023-12-27 05:52:44,662][20347] Updated weights for policy 0, policy_version 30879 (0.0009) [2023-12-27 05:52:44,676][20381] Updated weights for policy 1, policy_version 30860 (0.0006) [2023-12-27 05:52:44,716][20347] Updated weights for policy 0, policy_version 30889 (0.0009) [2023-12-27 05:52:44,733][20381] Updated weights for policy 1, policy_version 30870 (0.0006) [2023-12-27 05:52:44,794][20381] Updated weights for policy 1, policy_version 30880 (0.0007) [2023-12-27 05:52:45,433][20347] Updated weights for policy 0, policy_version 30899 (0.0009) [2023-12-27 05:52:45,490][20347] Updated weights for policy 0, policy_version 30909 (0.0006) [2023-12-27 05:52:45,506][20381] Updated weights for policy 1, policy_version 30890 (0.0008) [2023-12-27 05:52:45,550][20347] Updated weights for policy 0, policy_version 30919 (0.0005) [2023-12-27 05:52:45,559][20381] Updated weights for policy 1, policy_version 30900 (0.0007) [2023-12-27 05:52:45,607][20381] Updated weights for policy 1, policy_version 30910 (0.0007) [2023-12-27 05:52:46,135][20347] Updated weights for policy 0, policy_version 30929 (0.0006) [2023-12-27 05:52:46,200][20347] Updated weights for policy 0, policy_version 30939 (0.0008) [2023-12-27 05:52:46,269][20347] Updated weights for policy 0, policy_version 30949 (0.0008) [2023-12-27 05:52:46,283][20381] Updated weights for policy 1, policy_version 30920 (0.0009) [2023-12-27 05:52:46,326][20347] Updated weights for policy 0, policy_version 30959 (0.0010) [2023-12-27 05:52:46,343][20381] Updated weights for policy 1, policy_version 30930 (0.0011) [2023-12-27 05:52:46,401][20381] Updated weights for policy 1, policy_version 30940 (0.0006) [2023-12-27 05:52:46,933][20347] Updated weights for policy 0, policy_version 30969 (0.0006) [2023-12-27 05:52:46,990][20347] Updated weights for policy 0, policy_version 30979 (0.0007) [2023-12-27 05:52:47,046][20347] Updated weights for policy 0, policy_version 30989 (0.0006) [2023-12-27 05:52:47,066][20381] Updated weights for policy 1, policy_version 30950 (0.0008) [2023-12-27 05:52:47,123][20381] Updated weights for policy 1, policy_version 30960 (0.0010) [2023-12-27 05:52:47,177][20381] Updated weights for policy 1, policy_version 30970 (0.0010) [2023-12-27 05:52:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20910.4). Total num frames: 15867904. Throughput: 0: 10707.9, 1: 10343.4. Samples: 15875144. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-12-27 05:52:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:47,635][20347] Updated weights for policy 0, policy_version 30999 (0.0010) [2023-12-27 05:52:47,683][20347] Updated weights for policy 0, policy_version 31009 (0.0010) [2023-12-27 05:52:47,739][20347] Updated weights for policy 0, policy_version 31019 (0.0010) [2023-12-27 05:52:47,827][20381] Updated weights for policy 1, policy_version 30980 (0.0008) [2023-12-27 05:52:47,884][20381] Updated weights for policy 1, policy_version 30990 (0.0011) [2023-12-27 05:52:47,932][20381] Updated weights for policy 1, policy_version 31000 (0.0010) [2023-12-27 05:52:48,335][20347] Updated weights for policy 0, policy_version 31029 (0.0010) [2023-12-27 05:52:48,391][20347] Updated weights for policy 0, policy_version 31039 (0.0010) [2023-12-27 05:52:48,442][20347] Updated weights for policy 0, policy_version 31049 (0.0011) [2023-12-27 05:52:48,608][20381] Updated weights for policy 1, policy_version 31010 (0.0010) [2023-12-27 05:52:48,659][20381] Updated weights for policy 1, policy_version 31020 (0.0008) [2023-12-27 05:52:48,717][20381] Updated weights for policy 1, policy_version 31030 (0.0007) [2023-12-27 05:52:48,769][20381] Updated weights for policy 1, policy_version 31040 (0.0007) [2023-12-27 05:52:49,143][20347] Updated weights for policy 0, policy_version 31059 (0.0010) [2023-12-27 05:52:49,191][20347] Updated weights for policy 0, policy_version 31069 (0.0010) [2023-12-27 05:52:49,252][20347] Updated weights for policy 0, policy_version 31079 (0.0011) [2023-12-27 05:52:49,505][20381] Updated weights for policy 1, policy_version 31050 (0.0007) [2023-12-27 05:52:49,560][20381] Updated weights for policy 1, policy_version 31060 (0.0007) [2023-12-27 05:52:49,615][20381] Updated weights for policy 1, policy_version 31070 (0.0008) [2023-12-27 05:52:49,922][20347] Updated weights for policy 0, policy_version 31089 (0.0010) [2023-12-27 05:52:49,986][20347] Updated weights for policy 0, policy_version 31099 (0.0006) [2023-12-27 05:52:50,046][20347] Updated weights for policy 0, policy_version 31109 (0.0007) [2023-12-27 05:52:50,112][20347] Updated weights for policy 0, policy_version 31119 (0.0007) [2023-12-27 05:52:50,298][20381] Updated weights for policy 1, policy_version 31080 (0.0008) [2023-12-27 05:52:50,353][20381] Updated weights for policy 1, policy_version 31090 (0.0008) [2023-12-27 05:52:50,407][20381] Updated weights for policy 1, policy_version 31100 (0.0010) [2023-12-27 05:52:50,710][20347] Updated weights for policy 0, policy_version 31129 (0.0007) [2023-12-27 05:52:50,776][20347] Updated weights for policy 0, policy_version 31139 (0.0007) [2023-12-27 05:52:50,834][20347] Updated weights for policy 0, policy_version 31149 (0.0007) [2023-12-27 05:52:51,174][20381] Updated weights for policy 1, policy_version 31110 (0.0007) [2023-12-27 05:52:51,235][20381] Updated weights for policy 1, policy_version 31120 (0.0008) [2023-12-27 05:52:51,298][20381] Updated weights for policy 1, policy_version 31130 (0.0008) [2023-12-27 05:52:51,501][20347] Updated weights for policy 0, policy_version 31159 (0.0006) [2023-12-27 05:52:51,572][20347] Updated weights for policy 0, policy_version 31169 (0.0006) [2023-12-27 05:52:51,638][20347] Updated weights for policy 0, policy_version 31179 (0.0007) [2023-12-27 05:52:51,958][20381] Updated weights for policy 1, policy_version 31140 (0.0006) [2023-12-27 05:52:52,014][20381] Updated weights for policy 1, policy_version 31150 (0.0008) [2023-12-27 05:52:52,069][20381] Updated weights for policy 1, policy_version 31160 (0.0008) [2023-12-27 05:52:52,287][20347] Updated weights for policy 0, policy_version 31189 (0.0008) [2023-12-27 05:52:52,343][20347] Updated weights for policy 0, policy_version 31199 (0.0007) [2023-12-27 05:52:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 15966208. Throughput: 0: 10730.7, 1: 10308.7. Samples: 15938072. Policy #0 lag: (min: 13.0, avg: 20.1, max: 45.0) [2023-12-27 05:52:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000031168_7979008.pth... [2023-12-27 05:52:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000029952_7667712.pth [2023-12-27 05:52:52,401][20347] Updated weights for policy 0, policy_version 31209 (0.0008) [2023-12-27 05:52:52,441][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000031216_7995392.pth... [2023-12-27 05:52:52,444][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000029936_7667712.pth [2023-12-27 05:52:52,780][20381] Updated weights for policy 1, policy_version 31170 (0.0007) [2023-12-27 05:52:52,830][20381] Updated weights for policy 1, policy_version 31180 (0.0005) [2023-12-27 05:52:52,895][20381] Updated weights for policy 1, policy_version 31190 (0.0006) [2023-12-27 05:52:52,946][20381] Updated weights for policy 1, policy_version 31200 (0.0007) [2023-12-27 05:52:53,117][20347] Updated weights for policy 0, policy_version 31219 (0.0007) [2023-12-27 05:52:53,182][20347] Updated weights for policy 0, policy_version 31229 (0.0007) [2023-12-27 05:52:53,239][20347] Updated weights for policy 0, policy_version 31239 (0.0006) [2023-12-27 05:52:53,652][20381] Updated weights for policy 1, policy_version 31211 (0.0009) [2023-12-27 05:52:53,698][20381] Updated weights for policy 1, policy_version 31221 (0.0008) [2023-12-27 05:52:53,746][20381] Updated weights for policy 1, policy_version 31231 (0.0009) [2023-12-27 05:52:53,789][20347] Updated weights for policy 0, policy_version 31249 (0.0005) [2023-12-27 05:52:53,838][20347] Updated weights for policy 0, policy_version 31259 (0.0007) [2023-12-27 05:52:53,890][20347] Updated weights for policy 0, policy_version 31269 (0.0007) [2023-12-27 05:52:53,942][20347] Updated weights for policy 0, policy_version 31279 (0.0008) [2023-12-27 05:52:54,441][20381] Updated weights for policy 1, policy_version 31241 (0.0008) [2023-12-27 05:52:54,491][20381] Updated weights for policy 1, policy_version 31251 (0.0008) [2023-12-27 05:52:54,547][20381] Updated weights for policy 1, policy_version 31261 (0.0008) [2023-12-27 05:52:54,652][20347] Updated weights for policy 0, policy_version 31289 (0.0008) [2023-12-27 05:52:54,704][20347] Updated weights for policy 0, policy_version 31299 (0.0009) [2023-12-27 05:52:54,757][20347] Updated weights for policy 0, policy_version 31309 (0.0009) [2023-12-27 05:52:55,151][20381] Updated weights for policy 1, policy_version 31271 (0.0007) [2023-12-27 05:52:55,212][20381] Updated weights for policy 1, policy_version 31281 (0.0008) [2023-12-27 05:52:55,270][20381] Updated weights for policy 1, policy_version 31291 (0.0008) [2023-12-27 05:52:55,504][20347] Updated weights for policy 0, policy_version 31319 (0.0008) [2023-12-27 05:52:55,563][20347] Updated weights for policy 0, policy_version 31329 (0.0008) [2023-12-27 05:52:55,621][20347] Updated weights for policy 0, policy_version 31339 (0.0008) [2023-12-27 05:52:55,894][20381] Updated weights for policy 1, policy_version 31301 (0.0008) [2023-12-27 05:52:55,950][20381] Updated weights for policy 1, policy_version 31311 (0.0006) [2023-12-27 05:52:56,003][20381] Updated weights for policy 1, policy_version 31321 (0.0009) [2023-12-27 05:52:56,289][20347] Updated weights for policy 0, policy_version 31349 (0.0006) [2023-12-27 05:52:56,348][20347] Updated weights for policy 0, policy_version 31359 (0.0008) [2023-12-27 05:52:56,403][20347] Updated weights for policy 0, policy_version 31369 (0.0009) [2023-12-27 05:52:56,599][20381] Updated weights for policy 1, policy_version 31331 (0.0006) [2023-12-27 05:52:56,652][20381] Updated weights for policy 1, policy_version 31341 (0.0005) [2023-12-27 05:52:56,697][20381] Updated weights for policy 1, policy_version 31351 (0.0009) [2023-12-27 05:52:57,008][20347] Updated weights for policy 0, policy_version 31379 (0.0005) [2023-12-27 05:52:57,057][20347] Updated weights for policy 0, policy_version 31389 (0.0005) [2023-12-27 05:52:57,112][20347] Updated weights for policy 0, policy_version 31399 (0.0005) [2023-12-27 05:52:57,326][20381] Updated weights for policy 1, policy_version 31361 (0.0010) [2023-12-27 05:52:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 16072704. Throughput: 0: 10685.0, 1: 10327.6. Samples: 16060860. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 05:52:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:52:57,383][20381] Updated weights for policy 1, policy_version 31371 (0.0009) [2023-12-27 05:52:57,441][20381] Updated weights for policy 1, policy_version 31381 (0.0010) [2023-12-27 05:52:57,506][20381] Updated weights for policy 1, policy_version 31391 (0.0010) [2023-12-27 05:52:57,647][20347] Updated weights for policy 0, policy_version 31409 (0.0005) [2023-12-27 05:52:57,698][20347] Updated weights for policy 0, policy_version 31419 (0.0005) [2023-12-27 05:52:57,756][20347] Updated weights for policy 0, policy_version 31429 (0.0010) [2023-12-27 05:52:57,807][20347] Updated weights for policy 0, policy_version 31439 (0.0010) [2023-12-27 05:52:58,127][20381] Updated weights for policy 1, policy_version 31401 (0.0006) [2023-12-27 05:52:58,188][20381] Updated weights for policy 1, policy_version 31411 (0.0007) [2023-12-27 05:52:58,247][20381] Updated weights for policy 1, policy_version 31421 (0.0008) [2023-12-27 05:52:58,484][20347] Updated weights for policy 0, policy_version 31449 (0.0010) [2023-12-27 05:52:58,546][20347] Updated weights for policy 0, policy_version 31459 (0.0010) [2023-12-27 05:52:58,605][20347] Updated weights for policy 0, policy_version 31469 (0.0010) [2023-12-27 05:52:59,060][20381] Updated weights for policy 1, policy_version 31431 (0.0007) [2023-12-27 05:52:59,117][20381] Updated weights for policy 1, policy_version 31441 (0.0007) [2023-12-27 05:52:59,176][20381] Updated weights for policy 1, policy_version 31451 (0.0007) [2023-12-27 05:52:59,329][20347] Updated weights for policy 0, policy_version 31479 (0.0010) [2023-12-27 05:52:59,395][20347] Updated weights for policy 0, policy_version 31489 (0.0010) [2023-12-27 05:52:59,452][20347] Updated weights for policy 0, policy_version 31499 (0.0010) [2023-12-27 05:52:59,857][20381] Updated weights for policy 1, policy_version 31461 (0.0007) [2023-12-27 05:52:59,924][20381] Updated weights for policy 1, policy_version 31471 (0.0007) [2023-12-27 05:52:59,986][20381] Updated weights for policy 1, policy_version 31481 (0.0007) [2023-12-27 05:53:00,062][20347] Updated weights for policy 0, policy_version 31509 (0.0008) [2023-12-27 05:53:00,123][20347] Updated weights for policy 0, policy_version 31519 (0.0005) [2023-12-27 05:53:00,181][20347] Updated weights for policy 0, policy_version 31529 (0.0006) [2023-12-27 05:53:00,643][20381] Updated weights for policy 1, policy_version 31491 (0.0007) [2023-12-27 05:53:00,697][20381] Updated weights for policy 1, policy_version 31501 (0.0007) [2023-12-27 05:53:00,755][20381] Updated weights for policy 1, policy_version 31511 (0.0007) [2023-12-27 05:53:00,811][20347] Updated weights for policy 0, policy_version 31539 (0.0010) [2023-12-27 05:53:00,858][20347] Updated weights for policy 0, policy_version 31549 (0.0010) [2023-12-27 05:53:00,903][20347] Updated weights for policy 0, policy_version 31559 (0.0010) [2023-12-27 05:53:01,417][20381] Updated weights for policy 1, policy_version 31521 (0.0007) [2023-12-27 05:53:01,463][20381] Updated weights for policy 1, policy_version 31531 (0.0007) [2023-12-27 05:53:01,521][20381] Updated weights for policy 1, policy_version 31541 (0.0007) [2023-12-27 05:53:01,564][20381] Updated weights for policy 1, policy_version 31551 (0.0007) [2023-12-27 05:53:01,633][20347] Updated weights for policy 0, policy_version 31569 (0.0010) [2023-12-27 05:53:01,687][20347] Updated weights for policy 0, policy_version 31579 (0.0005) [2023-12-27 05:53:01,749][20347] Updated weights for policy 0, policy_version 31589 (0.0007) [2023-12-27 05:53:01,800][20347] Updated weights for policy 0, policy_version 31599 (0.0008) [2023-12-27 05:53:02,308][20381] Updated weights for policy 1, policy_version 31561 (0.0010) [2023-12-27 05:53:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 16171008. Throughput: 0: 10669.1, 1: 10346.5. Samples: 16187296. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 05:53:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:02,375][20381] Updated weights for policy 1, policy_version 31571 (0.0011) [2023-12-27 05:53:02,407][20347] Updated weights for policy 0, policy_version 31609 (0.0007) [2023-12-27 05:53:02,435][20381] Updated weights for policy 1, policy_version 31581 (0.0011) [2023-12-27 05:53:02,464][20347] Updated weights for policy 0, policy_version 31619 (0.0010) [2023-12-27 05:53:02,522][20347] Updated weights for policy 0, policy_version 31629 (0.0010) [2023-12-27 05:53:03,116][20381] Updated weights for policy 1, policy_version 31591 (0.0011) [2023-12-27 05:53:03,162][20381] Updated weights for policy 1, policy_version 31601 (0.0011) [2023-12-27 05:53:03,198][20347] Updated weights for policy 0, policy_version 31639 (0.0007) [2023-12-27 05:53:03,218][20381] Updated weights for policy 1, policy_version 31611 (0.0011) [2023-12-27 05:53:03,255][20347] Updated weights for policy 0, policy_version 31649 (0.0007) [2023-12-27 05:53:03,305][20347] Updated weights for policy 0, policy_version 31659 (0.0010) [2023-12-27 05:53:03,845][20347] Updated weights for policy 0, policy_version 31669 (0.0010) [2023-12-27 05:53:03,908][20347] Updated weights for policy 0, policy_version 31679 (0.0009) [2023-12-27 05:53:03,914][20381] Updated weights for policy 1, policy_version 31621 (0.0008) [2023-12-27 05:53:03,971][20347] Updated weights for policy 0, policy_version 31689 (0.0007) [2023-12-27 05:53:03,971][20381] Updated weights for policy 1, policy_version 31631 (0.0007) [2023-12-27 05:53:04,030][20381] Updated weights for policy 1, policy_version 31641 (0.0005) [2023-12-27 05:53:04,612][20381] Updated weights for policy 1, policy_version 31651 (0.0006) [2023-12-27 05:53:04,656][20347] Updated weights for policy 0, policy_version 31699 (0.0010) [2023-12-27 05:53:04,663][20381] Updated weights for policy 1, policy_version 31661 (0.0007) [2023-12-27 05:53:04,676][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-27 05:53:04,711][20347] Updated weights for policy 0, policy_version 31709 (0.0010) [2023-12-27 05:53:04,755][20347] Updated weights for policy 0, policy_version 31719 (0.0006) [2023-12-27 05:53:05,307][20381] Updated weights for policy 1, policy_version 31671 (0.0005) [2023-12-27 05:53:05,326][20347] Updated weights for policy 0, policy_version 31729 (0.0005) [2023-12-27 05:53:05,368][20381] Updated weights for policy 1, policy_version 31681 (0.0007) [2023-12-27 05:53:05,386][20347] Updated weights for policy 0, policy_version 31739 (0.0005) [2023-12-27 05:53:05,420][20381] Updated weights for policy 1, policy_version 31691 (0.0007) [2023-12-27 05:53:05,434][20347] Updated weights for policy 0, policy_version 31749 (0.0005) [2023-12-27 05:53:05,483][20347] Updated weights for policy 0, policy_version 31759 (0.0005) [2023-12-27 05:53:06,077][20347] Updated weights for policy 0, policy_version 31769 (0.0009) [2023-12-27 05:53:06,106][20381] Updated weights for policy 1, policy_version 31701 (0.0009) [2023-12-27 05:53:06,150][20347] Updated weights for policy 0, policy_version 31779 (0.0010) [2023-12-27 05:53:06,168][20381] Updated weights for policy 1, policy_version 31711 (0.0009) [2023-12-27 05:53:06,213][20347] Updated weights for policy 0, policy_version 31789 (0.0010) [2023-12-27 05:53:06,227][20381] Updated weights for policy 1, policy_version 31721 (0.0008) [2023-12-27 05:53:06,875][20347] Updated weights for policy 0, policy_version 31799 (0.0008) [2023-12-27 05:53:06,932][20347] Updated weights for policy 0, policy_version 31809 (0.0006) [2023-12-27 05:53:06,988][20381] Updated weights for policy 1, policy_version 31731 (0.0008) [2023-12-27 05:53:06,988][20347] Updated weights for policy 0, policy_version 31819 (0.0005) [2023-12-27 05:53:07,056][20381] Updated weights for policy 1, policy_version 31741 (0.0008) [2023-12-27 05:53:07,125][20381] Updated weights for policy 1, policy_version 31751 (0.0008) [2023-12-27 05:53:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 16285696. Throughput: 0: 10716.2, 1: 10385.8. Samples: 16252628. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 05:53:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000031824_8151040.pth... [2023-12-27 05:53:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000031760_8134656.pth... [2023-12-27 05:53:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000030560_7823360.pth [2023-12-27 05:53:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000030576_7831552.pth [2023-12-27 05:53:07,600][20347] Updated weights for policy 0, policy_version 31829 (0.0006) [2023-12-27 05:53:07,655][20347] Updated weights for policy 0, policy_version 31839 (0.0007) [2023-12-27 05:53:07,702][20347] Updated weights for policy 0, policy_version 31849 (0.0007) [2023-12-27 05:53:07,825][20381] Updated weights for policy 1, policy_version 31761 (0.0009) [2023-12-27 05:53:07,873][20381] Updated weights for policy 1, policy_version 31771 (0.0010) [2023-12-27 05:53:07,925][20381] Updated weights for policy 1, policy_version 31781 (0.0005) [2023-12-27 05:53:07,975][20381] Updated weights for policy 1, policy_version 31791 (0.0005) [2023-12-27 05:53:08,469][20347] Updated weights for policy 0, policy_version 31859 (0.0007) [2023-12-27 05:53:08,520][20347] Updated weights for policy 0, policy_version 31869 (0.0007) [2023-12-27 05:53:08,572][20347] Updated weights for policy 0, policy_version 31879 (0.0007) [2023-12-27 05:53:08,596][20381] Updated weights for policy 1, policy_version 31801 (0.0009) [2023-12-27 05:53:08,642][20381] Updated weights for policy 1, policy_version 31811 (0.0010) [2023-12-27 05:53:08,686][20381] Updated weights for policy 1, policy_version 31821 (0.0010) [2023-12-27 05:53:09,286][20347] Updated weights for policy 0, policy_version 31889 (0.0006) [2023-12-27 05:53:09,343][20381] Updated weights for policy 1, policy_version 31831 (0.0007) [2023-12-27 05:53:09,351][20347] Updated weights for policy 0, policy_version 31899 (0.0007) [2023-12-27 05:53:09,410][20381] Updated weights for policy 1, policy_version 31841 (0.0008) [2023-12-27 05:53:09,417][20347] Updated weights for policy 0, policy_version 31909 (0.0010) [2023-12-27 05:53:09,469][20381] Updated weights for policy 1, policy_version 31851 (0.0008) [2023-12-27 05:53:09,473][20347] Updated weights for policy 0, policy_version 31919 (0.0010) [2023-12-27 05:53:10,172][20347] Updated weights for policy 0, policy_version 31929 (0.0006) [2023-12-27 05:53:10,229][20381] Updated weights for policy 1, policy_version 31861 (0.0007) [2023-12-27 05:53:10,240][20347] Updated weights for policy 0, policy_version 31939 (0.0006) [2023-12-27 05:53:10,287][20381] Updated weights for policy 1, policy_version 31871 (0.0006) [2023-12-27 05:53:10,305][20347] Updated weights for policy 0, policy_version 31949 (0.0006) [2023-12-27 05:53:10,353][20381] Updated weights for policy 1, policy_version 31881 (0.0007) [2023-12-27 05:53:10,827][20347] Updated weights for policy 0, policy_version 31959 (0.0007) [2023-12-27 05:53:10,887][20347] Updated weights for policy 0, policy_version 31969 (0.0005) [2023-12-27 05:53:10,942][20347] Updated weights for policy 0, policy_version 31979 (0.0005) [2023-12-27 05:53:11,065][20381] Updated weights for policy 1, policy_version 31891 (0.0008) [2023-12-27 05:53:11,113][20381] Updated weights for policy 1, policy_version 31901 (0.0007) [2023-12-27 05:53:11,180][20381] Updated weights for policy 1, policy_version 31911 (0.0008) [2023-12-27 05:53:11,610][20347] Updated weights for policy 0, policy_version 31989 (0.0006) [2023-12-27 05:53:11,673][20347] Updated weights for policy 0, policy_version 31999 (0.0008) [2023-12-27 05:53:11,738][20347] Updated weights for policy 0, policy_version 32009 (0.0007) [2023-12-27 05:53:11,885][20381] Updated weights for policy 1, policy_version 31921 (0.0008) [2023-12-27 05:53:11,948][20381] Updated weights for policy 1, policy_version 31931 (0.0008) [2023-12-27 05:53:12,004][20381] Updated weights for policy 1, policy_version 31941 (0.0006) [2023-12-27 05:53:12,061][20381] Updated weights for policy 1, policy_version 31951 (0.0008) [2023-12-27 05:53:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 16384000. Throughput: 0: 10764.0, 1: 10337.3. Samples: 16375936. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 05:53:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:12,390][20347] Updated weights for policy 0, policy_version 32019 (0.0007) [2023-12-27 05:53:12,442][20347] Updated weights for policy 0, policy_version 32029 (0.0006) [2023-12-27 05:53:12,498][20347] Updated weights for policy 0, policy_version 32039 (0.0008) [2023-12-27 05:53:12,773][20381] Updated weights for policy 1, policy_version 31961 (0.0009) [2023-12-27 05:53:12,828][20381] Updated weights for policy 1, policy_version 31971 (0.0006) [2023-12-27 05:53:12,885][20381] Updated weights for policy 1, policy_version 31981 (0.0006) [2023-12-27 05:53:13,083][20347] Updated weights for policy 0, policy_version 32049 (0.0010) [2023-12-27 05:53:13,135][20347] Updated weights for policy 0, policy_version 32059 (0.0005) [2023-12-27 05:53:13,187][20347] Updated weights for policy 0, policy_version 32069 (0.0005) [2023-12-27 05:53:13,238][20347] Updated weights for policy 0, policy_version 32079 (0.0005) [2023-12-27 05:53:13,585][20381] Updated weights for policy 1, policy_version 31991 (0.0008) [2023-12-27 05:53:13,643][20381] Updated weights for policy 1, policy_version 32001 (0.0007) [2023-12-27 05:53:13,693][20381] Updated weights for policy 1, policy_version 32011 (0.0007) [2023-12-27 05:53:13,813][20347] Updated weights for policy 0, policy_version 32089 (0.0005) [2023-12-27 05:53:13,875][20347] Updated weights for policy 0, policy_version 32099 (0.0005) [2023-12-27 05:53:13,940][20347] Updated weights for policy 0, policy_version 32109 (0.0005) [2023-12-27 05:53:14,425][20381] Updated weights for policy 1, policy_version 32021 (0.0008) [2023-12-27 05:53:14,463][20347] Updated weights for policy 0, policy_version 32119 (0.0009) [2023-12-27 05:53:14,477][20381] Updated weights for policy 1, policy_version 32031 (0.0005) [2023-12-27 05:53:14,515][20347] Updated weights for policy 0, policy_version 32129 (0.0010) [2023-12-27 05:53:14,531][20381] Updated weights for policy 1, policy_version 32041 (0.0005) [2023-12-27 05:53:14,565][20347] Updated weights for policy 0, policy_version 32139 (0.0010) [2023-12-27 05:53:15,209][20381] Updated weights for policy 1, policy_version 32051 (0.0006) [2023-12-27 05:53:15,272][20381] Updated weights for policy 1, policy_version 32061 (0.0008) [2023-12-27 05:53:15,282][20347] Updated weights for policy 0, policy_version 32149 (0.0010) [2023-12-27 05:53:15,329][20381] Updated weights for policy 1, policy_version 32071 (0.0006) [2023-12-27 05:53:15,331][20347] Updated weights for policy 0, policy_version 32159 (0.0010) [2023-12-27 05:53:15,386][20347] Updated weights for policy 0, policy_version 32169 (0.0010) [2023-12-27 05:53:15,997][20381] Updated weights for policy 1, policy_version 32081 (0.0005) [2023-12-27 05:53:16,050][20381] Updated weights for policy 1, policy_version 32091 (0.0005) [2023-12-27 05:53:16,077][20347] Updated weights for policy 0, policy_version 32179 (0.0010) [2023-12-27 05:53:16,100][20381] Updated weights for policy 1, policy_version 32101 (0.0005) [2023-12-27 05:53:16,134][20347] Updated weights for policy 0, policy_version 32189 (0.0010) [2023-12-27 05:53:16,148][20381] Updated weights for policy 1, policy_version 32111 (0.0007) [2023-12-27 05:53:16,193][20347] Updated weights for policy 0, policy_version 32199 (0.0011) [2023-12-27 05:53:16,801][20381] Updated weights for policy 1, policy_version 32121 (0.0006) [2023-12-27 05:53:16,845][20347] Updated weights for policy 0, policy_version 32209 (0.0010) [2023-12-27 05:53:16,868][20381] Updated weights for policy 1, policy_version 32131 (0.0005) [2023-12-27 05:53:16,902][20347] Updated weights for policy 0, policy_version 32219 (0.0005) [2023-12-27 05:53:16,924][20381] Updated weights for policy 1, policy_version 32141 (0.0005) [2023-12-27 05:53:16,951][20347] Updated weights for policy 0, policy_version 32229 (0.0005) [2023-12-27 05:53:17,000][20347] Updated weights for policy 0, policy_version 32239 (0.0005) [2023-12-27 05:53:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 16490496. Throughput: 0: 10741.2, 1: 10281.7. Samples: 16502236. Policy #0 lag: (min: 30.0, avg: 38.3, max: 62.0) [2023-12-27 05:53:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:17,534][20381] Updated weights for policy 1, policy_version 32151 (0.0006) [2023-12-27 05:53:17,590][20381] Updated weights for policy 1, policy_version 32161 (0.0007) [2023-12-27 05:53:17,602][20347] Updated weights for policy 0, policy_version 32249 (0.0010) [2023-12-27 05:53:17,653][20381] Updated weights for policy 1, policy_version 32171 (0.0006) [2023-12-27 05:53:17,662][20347] Updated weights for policy 0, policy_version 32259 (0.0009) [2023-12-27 05:53:17,707][20347] Updated weights for policy 0, policy_version 32269 (0.0010) [2023-12-27 05:53:18,310][20381] Updated weights for policy 1, policy_version 32181 (0.0006) [2023-12-27 05:53:18,315][20347] Updated weights for policy 0, policy_version 32279 (0.0007) [2023-12-27 05:53:18,371][20381] Updated weights for policy 1, policy_version 32191 (0.0008) [2023-12-27 05:53:18,380][20347] Updated weights for policy 0, policy_version 32289 (0.0008) [2023-12-27 05:53:18,426][20381] Updated weights for policy 1, policy_version 32201 (0.0008) [2023-12-27 05:53:18,435][20347] Updated weights for policy 0, policy_version 32299 (0.0008) [2023-12-27 05:53:19,073][20381] Updated weights for policy 1, policy_version 32211 (0.0007) [2023-12-27 05:53:19,102][20347] Updated weights for policy 0, policy_version 32309 (0.0005) [2023-12-27 05:53:19,116][20381] Updated weights for policy 1, policy_version 32221 (0.0007) [2023-12-27 05:53:19,168][20347] Updated weights for policy 0, policy_version 32319 (0.0005) [2023-12-27 05:53:19,171][20381] Updated weights for policy 1, policy_version 32231 (0.0007) [2023-12-27 05:53:19,229][20347] Updated weights for policy 0, policy_version 32329 (0.0007) [2023-12-27 05:53:19,871][20347] Updated weights for policy 0, policy_version 32339 (0.0006) [2023-12-27 05:53:19,886][20381] Updated weights for policy 1, policy_version 32241 (0.0008) [2023-12-27 05:53:19,935][20347] Updated weights for policy 0, policy_version 32349 (0.0007) [2023-12-27 05:53:19,949][20381] Updated weights for policy 1, policy_version 32251 (0.0010) [2023-12-27 05:53:19,996][20347] Updated weights for policy 0, policy_version 32359 (0.0006) [2023-12-27 05:53:20,002][20381] Updated weights for policy 1, policy_version 32261 (0.0010) [2023-12-27 05:53:20,062][20381] Updated weights for policy 1, policy_version 32271 (0.0010) [2023-12-27 05:53:20,691][20347] Updated weights for policy 0, policy_version 32369 (0.0006) [2023-12-27 05:53:20,747][20347] Updated weights for policy 0, policy_version 32379 (0.0005) [2023-12-27 05:53:20,756][20381] Updated weights for policy 1, policy_version 32281 (0.0011) [2023-12-27 05:53:20,804][20347] Updated weights for policy 0, policy_version 32389 (0.0006) [2023-12-27 05:53:20,814][20381] Updated weights for policy 1, policy_version 32291 (0.0011) [2023-12-27 05:53:20,864][20347] Updated weights for policy 0, policy_version 32399 (0.0005) [2023-12-27 05:53:20,877][20381] Updated weights for policy 1, policy_version 32301 (0.0011) [2023-12-27 05:53:21,554][20347] Updated weights for policy 0, policy_version 32409 (0.0007) [2023-12-27 05:53:21,600][20347] Updated weights for policy 0, policy_version 32419 (0.0007) [2023-12-27 05:53:21,630][20381] Updated weights for policy 1, policy_version 32311 (0.0010) [2023-12-27 05:53:21,667][20347] Updated weights for policy 0, policy_version 32429 (0.0008) [2023-12-27 05:53:21,693][20381] Updated weights for policy 1, policy_version 32321 (0.0009) [2023-12-27 05:53:21,760][20381] Updated weights for policy 1, policy_version 32331 (0.0008) [2023-12-27 05:53:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20882.7). Total num frames: 16588800. Throughput: 0: 10700.2, 1: 10262.8. Samples: 16564904. Policy #0 lag: (min: 30.0, avg: 38.3, max: 62.0) [2023-12-27 05:53:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000032336_8282112.pth... [2023-12-27 05:53:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000031168_7979008.pth [2023-12-27 05:53:22,403][20347] Updated weights for policy 0, policy_version 32439 (0.0007) [2023-12-27 05:53:22,452][20381] Updated weights for policy 1, policy_version 32341 (0.0007) [2023-12-27 05:53:22,457][20347] Updated weights for policy 0, policy_version 32449 (0.0006) [2023-12-27 05:53:22,502][20381] Updated weights for policy 1, policy_version 32351 (0.0006) [2023-12-27 05:53:22,515][20347] Updated weights for policy 0, policy_version 32459 (0.0006) [2023-12-27 05:53:22,543][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000032464_8314880.pth... [2023-12-27 05:53:22,547][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000031216_7995392.pth [2023-12-27 05:53:22,555][20381] Updated weights for policy 1, policy_version 32361 (0.0006) [2023-12-27 05:53:23,187][20347] Updated weights for policy 0, policy_version 32469 (0.0006) [2023-12-27 05:53:23,203][20381] Updated weights for policy 1, policy_version 32371 (0.0008) [2023-12-27 05:53:23,240][20347] Updated weights for policy 0, policy_version 32479 (0.0007) [2023-12-27 05:53:23,252][20381] Updated weights for policy 1, policy_version 32381 (0.0009) [2023-12-27 05:53:23,310][20381] Updated weights for policy 1, policy_version 32391 (0.0006) [2023-12-27 05:53:23,311][20347] Updated weights for policy 0, policy_version 32489 (0.0007) [2023-12-27 05:53:24,011][20347] Updated weights for policy 0, policy_version 32499 (0.0007) [2023-12-27 05:53:24,014][20381] Updated weights for policy 1, policy_version 32401 (0.0006) [2023-12-27 05:53:24,064][20347] Updated weights for policy 0, policy_version 32509 (0.0006) [2023-12-27 05:53:24,077][20381] Updated weights for policy 1, policy_version 32411 (0.0006) [2023-12-27 05:53:24,122][20347] Updated weights for policy 0, policy_version 32519 (0.0007) [2023-12-27 05:53:24,136][20381] Updated weights for policy 1, policy_version 32421 (0.0006) [2023-12-27 05:53:24,193][20381] Updated weights for policy 1, policy_version 32431 (0.0007) [2023-12-27 05:53:24,783][20381] Updated weights for policy 1, policy_version 32441 (0.0006) [2023-12-27 05:53:24,806][20347] Updated weights for policy 0, policy_version 32529 (0.0006) [2023-12-27 05:53:24,843][20381] Updated weights for policy 1, policy_version 32451 (0.0005) [2023-12-27 05:53:24,858][20347] Updated weights for policy 0, policy_version 32539 (0.0007) [2023-12-27 05:53:24,900][20381] Updated weights for policy 1, policy_version 32461 (0.0008) [2023-12-27 05:53:24,905][20347] Updated weights for policy 0, policy_version 32549 (0.0007) [2023-12-27 05:53:24,947][20347] Updated weights for policy 0, policy_version 32559 (0.0006) [2023-12-27 05:53:25,535][20381] Updated weights for policy 1, policy_version 32471 (0.0007) [2023-12-27 05:53:25,596][20381] Updated weights for policy 1, policy_version 32481 (0.0007) [2023-12-27 05:53:25,658][20381] Updated weights for policy 1, policy_version 32491 (0.0010) [2023-12-27 05:53:25,666][20347] Updated weights for policy 0, policy_version 32569 (0.0006) [2023-12-27 05:53:25,715][20347] Updated weights for policy 0, policy_version 32579 (0.0005) [2023-12-27 05:53:25,762][20347] Updated weights for policy 0, policy_version 32589 (0.0005) [2023-12-27 05:53:26,272][20381] Updated weights for policy 1, policy_version 32501 (0.0008) [2023-12-27 05:53:26,332][20381] Updated weights for policy 1, policy_version 32511 (0.0007) [2023-12-27 05:53:26,391][20381] Updated weights for policy 1, policy_version 32521 (0.0008) [2023-12-27 05:53:26,422][20347] Updated weights for policy 0, policy_version 32599 (0.0007) [2023-12-27 05:53:26,476][20347] Updated weights for policy 0, policy_version 32609 (0.0007) [2023-12-27 05:53:26,532][20347] Updated weights for policy 0, policy_version 32619 (0.0008) [2023-12-27 05:53:26,979][20381] Updated weights for policy 1, policy_version 32531 (0.0006) [2023-12-27 05:53:27,039][20381] Updated weights for policy 1, policy_version 32541 (0.0007) [2023-12-27 05:53:27,095][20381] Updated weights for policy 1, policy_version 32551 (0.0008) [2023-12-27 05:53:27,247][20347] Updated weights for policy 0, policy_version 32629 (0.0008) [2023-12-27 05:53:27,305][20347] Updated weights for policy 0, policy_version 32639 (0.0008) [2023-12-27 05:53:27,360][20347] Updated weights for policy 0, policy_version 32649 (0.0006) [2023-12-27 05:53:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 16695296. Throughput: 0: 10578.5, 1: 10299.7. Samples: 16687752. Policy #0 lag: (min: 30.0, avg: 38.3, max: 62.0) [2023-12-27 05:53:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:27,689][20381] Updated weights for policy 1, policy_version 32561 (0.0007) [2023-12-27 05:53:27,747][20381] Updated weights for policy 1, policy_version 32571 (0.0005) [2023-12-27 05:53:27,799][20381] Updated weights for policy 1, policy_version 32581 (0.0006) [2023-12-27 05:53:27,853][20381] Updated weights for policy 1, policy_version 32591 (0.0007) [2023-12-27 05:53:28,065][20347] Updated weights for policy 0, policy_version 32659 (0.0007) [2023-12-27 05:53:28,106][20347] Updated weights for policy 0, policy_version 32669 (0.0007) [2023-12-27 05:53:28,151][20347] Updated weights for policy 0, policy_version 32679 (0.0007) [2023-12-27 05:53:28,501][20381] Updated weights for policy 1, policy_version 32601 (0.0006) [2023-12-27 05:53:28,555][20381] Updated weights for policy 1, policy_version 32611 (0.0006) [2023-12-27 05:53:28,603][20381] Updated weights for policy 1, policy_version 32621 (0.0005) [2023-12-27 05:53:28,840][20347] Updated weights for policy 0, policy_version 32689 (0.0007) [2023-12-27 05:53:28,894][20347] Updated weights for policy 0, policy_version 32699 (0.0008) [2023-12-27 05:53:28,942][20347] Updated weights for policy 0, policy_version 32709 (0.0008) [2023-12-27 05:53:28,991][20347] Updated weights for policy 0, policy_version 32719 (0.0008) [2023-12-27 05:53:29,169][20381] Updated weights for policy 1, policy_version 32631 (0.0006) [2023-12-27 05:53:29,217][20381] Updated weights for policy 1, policy_version 32641 (0.0006) [2023-12-27 05:53:29,273][20381] Updated weights for policy 1, policy_version 32651 (0.0008) [2023-12-27 05:53:29,707][20347] Updated weights for policy 0, policy_version 32729 (0.0007) [2023-12-27 05:53:29,751][20347] Updated weights for policy 0, policy_version 32739 (0.0007) [2023-12-27 05:53:29,800][20347] Updated weights for policy 0, policy_version 32749 (0.0007) [2023-12-27 05:53:29,947][20381] Updated weights for policy 1, policy_version 32661 (0.0008) [2023-12-27 05:53:30,009][20381] Updated weights for policy 1, policy_version 32671 (0.0008) [2023-12-27 05:53:30,066][20381] Updated weights for policy 1, policy_version 32681 (0.0008) [2023-12-27 05:53:30,501][20347] Updated weights for policy 0, policy_version 32759 (0.0006) [2023-12-27 05:53:30,546][20347] Updated weights for policy 0, policy_version 32769 (0.0005) [2023-12-27 05:53:30,593][20347] Updated weights for policy 0, policy_version 32779 (0.0007) [2023-12-27 05:53:30,740][20381] Updated weights for policy 1, policy_version 32691 (0.0007) [2023-12-27 05:53:30,791][20381] Updated weights for policy 1, policy_version 32701 (0.0009) [2023-12-27 05:53:30,846][20381] Updated weights for policy 1, policy_version 32711 (0.0008) [2023-12-27 05:53:31,209][20347] Updated weights for policy 0, policy_version 32789 (0.0007) [2023-12-27 05:53:31,275][20347] Updated weights for policy 0, policy_version 32799 (0.0008) [2023-12-27 05:53:31,325][20347] Updated weights for policy 0, policy_version 32809 (0.0008) [2023-12-27 05:53:31,512][20381] Updated weights for policy 1, policy_version 32721 (0.0009) [2023-12-27 05:53:31,575][20381] Updated weights for policy 1, policy_version 32731 (0.0005) [2023-12-27 05:53:31,636][20381] Updated weights for policy 1, policy_version 32741 (0.0006) [2023-12-27 05:53:31,692][20381] Updated weights for policy 1, policy_version 32751 (0.0006) [2023-12-27 05:53:32,043][20347] Updated weights for policy 0, policy_version 32819 (0.0008) [2023-12-27 05:53:32,098][20347] Updated weights for policy 0, policy_version 32829 (0.0008) [2023-12-27 05:53:32,148][20347] Updated weights for policy 0, policy_version 32839 (0.0008) [2023-12-27 05:53:32,308][20381] Updated weights for policy 1, policy_version 32761 (0.0007) [2023-12-27 05:53:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 16801792. Throughput: 0: 10521.6, 1: 10335.1. Samples: 16813696. Policy #0 lag: (min: 30.0, avg: 38.3, max: 62.0) [2023-12-27 05:53:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:32,372][20381] Updated weights for policy 1, policy_version 32771 (0.0008) [2023-12-27 05:53:32,425][20381] Updated weights for policy 1, policy_version 32781 (0.0007) [2023-12-27 05:53:32,887][20347] Updated weights for policy 0, policy_version 32849 (0.0007) [2023-12-27 05:53:32,940][20347] Updated weights for policy 0, policy_version 32859 (0.0006) [2023-12-27 05:53:32,994][20347] Updated weights for policy 0, policy_version 32869 (0.0007) [2023-12-27 05:53:33,053][20347] Updated weights for policy 0, policy_version 32879 (0.0006) [2023-12-27 05:53:33,073][20381] Updated weights for policy 1, policy_version 32791 (0.0007) [2023-12-27 05:53:33,127][20381] Updated weights for policy 1, policy_version 32801 (0.0008) [2023-12-27 05:53:33,181][20381] Updated weights for policy 1, policy_version 32811 (0.0007) [2023-12-27 05:53:33,690][20347] Updated weights for policy 0, policy_version 32889 (0.0009) [2023-12-27 05:53:33,751][20347] Updated weights for policy 0, policy_version 32899 (0.0010) [2023-12-27 05:53:33,796][20347] Updated weights for policy 0, policy_version 32909 (0.0010) [2023-12-27 05:53:33,841][20381] Updated weights for policy 1, policy_version 32821 (0.0007) [2023-12-27 05:53:33,897][20381] Updated weights for policy 1, policy_version 32831 (0.0007) [2023-12-27 05:53:33,953][20381] Updated weights for policy 1, policy_version 32841 (0.0006) [2023-12-27 05:53:34,479][20347] Updated weights for policy 0, policy_version 32919 (0.0010) [2023-12-27 05:53:34,502][20381] Updated weights for policy 1, policy_version 32851 (0.0006) [2023-12-27 05:53:34,535][20347] Updated weights for policy 0, policy_version 32929 (0.0011) [2023-12-27 05:53:34,565][20381] Updated weights for policy 1, policy_version 32861 (0.0009) [2023-12-27 05:53:34,583][20347] Updated weights for policy 0, policy_version 32939 (0.0006) [2023-12-27 05:53:34,630][20381] Updated weights for policy 1, policy_version 32871 (0.0008) [2023-12-27 05:53:35,215][20347] Updated weights for policy 0, policy_version 32949 (0.0005) [2023-12-27 05:53:35,268][20347] Updated weights for policy 0, policy_version 32959 (0.0008) [2023-12-27 05:53:35,310][20381] Updated weights for policy 1, policy_version 32881 (0.0008) [2023-12-27 05:53:35,317][20347] Updated weights for policy 0, policy_version 32969 (0.0007) [2023-12-27 05:53:35,370][20381] Updated weights for policy 1, policy_version 32891 (0.0005) [2023-12-27 05:53:35,430][20381] Updated weights for policy 1, policy_version 32901 (0.0005) [2023-12-27 05:53:35,490][20381] Updated weights for policy 1, policy_version 32911 (0.0005) [2023-12-27 05:53:36,009][20381] Updated weights for policy 1, policy_version 32921 (0.0005) [2023-12-27 05:53:36,033][20347] Updated weights for policy 0, policy_version 32979 (0.0007) [2023-12-27 05:53:36,058][20381] Updated weights for policy 1, policy_version 32931 (0.0005) [2023-12-27 05:53:36,082][20347] Updated weights for policy 0, policy_version 32989 (0.0005) [2023-12-27 05:53:36,111][20381] Updated weights for policy 1, policy_version 32941 (0.0006) [2023-12-27 05:53:36,144][20347] Updated weights for policy 0, policy_version 32999 (0.0008) [2023-12-27 05:53:36,706][20381] Updated weights for policy 1, policy_version 32951 (0.0005) [2023-12-27 05:53:36,750][20381] Updated weights for policy 1, policy_version 32961 (0.0005) [2023-12-27 05:53:36,800][20381] Updated weights for policy 1, policy_version 32971 (0.0005) [2023-12-27 05:53:36,812][20347] Updated weights for policy 0, policy_version 33009 (0.0006) [2023-12-27 05:53:36,867][20347] Updated weights for policy 0, policy_version 33019 (0.0008) [2023-12-27 05:53:36,914][20347] Updated weights for policy 0, policy_version 33029 (0.0005) [2023-12-27 05:53:36,967][20347] Updated weights for policy 0, policy_version 33039 (0.0006) [2023-12-27 05:53:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20938.2). Total num frames: 16908288. Throughput: 0: 10501.5, 1: 10369.5. Samples: 16877268. Policy #0 lag: (min: 13.0, avg: 25.4, max: 45.0) [2023-12-27 05:53:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000033040_8462336.pth... [2023-12-27 05:53:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000032976_8445952.pth... [2023-12-27 05:53:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000031760_8134656.pth [2023-12-27 05:53:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000031824_8151040.pth [2023-12-27 05:53:37,425][20381] Updated weights for policy 1, policy_version 32981 (0.0006) [2023-12-27 05:53:37,489][20381] Updated weights for policy 1, policy_version 32991 (0.0008) [2023-12-27 05:53:37,556][20381] Updated weights for policy 1, policy_version 33001 (0.0008) [2023-12-27 05:53:37,645][20347] Updated weights for policy 0, policy_version 33049 (0.0007) [2023-12-27 05:53:37,702][20347] Updated weights for policy 0, policy_version 33059 (0.0007) [2023-12-27 05:53:37,769][20347] Updated weights for policy 0, policy_version 33069 (0.0007) [2023-12-27 05:53:38,281][20381] Updated weights for policy 1, policy_version 33011 (0.0008) [2023-12-27 05:53:38,341][20381] Updated weights for policy 1, policy_version 33021 (0.0008) [2023-12-27 05:53:38,400][20381] Updated weights for policy 1, policy_version 33031 (0.0007) [2023-12-27 05:53:38,421][20347] Updated weights for policy 0, policy_version 33079 (0.0006) [2023-12-27 05:53:38,479][20347] Updated weights for policy 0, policy_version 33089 (0.0006) [2023-12-27 05:53:38,539][20347] Updated weights for policy 0, policy_version 33099 (0.0008) [2023-12-27 05:53:39,034][20381] Updated weights for policy 1, policy_version 33041 (0.0006) [2023-12-27 05:53:39,094][20381] Updated weights for policy 1, policy_version 33051 (0.0006) [2023-12-27 05:53:39,152][20381] Updated weights for policy 1, policy_version 33061 (0.0008) [2023-12-27 05:53:39,196][20347] Updated weights for policy 0, policy_version 33109 (0.0008) [2023-12-27 05:53:39,212][20381] Updated weights for policy 1, policy_version 33071 (0.0007) [2023-12-27 05:53:39,256][20347] Updated weights for policy 0, policy_version 33119 (0.0007) [2023-12-27 05:53:39,315][20347] Updated weights for policy 0, policy_version 33129 (0.0006) [2023-12-27 05:53:39,900][20381] Updated weights for policy 1, policy_version 33081 (0.0006) [2023-12-27 05:53:39,963][20381] Updated weights for policy 1, policy_version 33091 (0.0009) [2023-12-27 05:53:40,019][20381] Updated weights for policy 1, policy_version 33101 (0.0009) [2023-12-27 05:53:40,050][20347] Updated weights for policy 0, policy_version 33139 (0.0008) [2023-12-27 05:53:40,095][20347] Updated weights for policy 0, policy_version 33149 (0.0006) [2023-12-27 05:53:40,142][20347] Updated weights for policy 0, policy_version 33159 (0.0006) [2023-12-27 05:53:40,778][20381] Updated weights for policy 1, policy_version 33111 (0.0009) [2023-12-27 05:53:40,828][20381] Updated weights for policy 1, policy_version 33121 (0.0010) [2023-12-27 05:53:40,840][20347] Updated weights for policy 0, policy_version 33169 (0.0008) [2023-12-27 05:53:40,873][20381] Updated weights for policy 1, policy_version 33131 (0.0006) [2023-12-27 05:53:40,890][20347] Updated weights for policy 0, policy_version 33179 (0.0007) [2023-12-27 05:53:40,948][20347] Updated weights for policy 0, policy_version 33189 (0.0008) [2023-12-27 05:53:41,012][20347] Updated weights for policy 0, policy_version 33199 (0.0007) [2023-12-27 05:53:41,597][20381] Updated weights for policy 1, policy_version 33141 (0.0006) [2023-12-27 05:53:41,668][20381] Updated weights for policy 1, policy_version 33151 (0.0011) [2023-12-27 05:53:41,728][20381] Updated weights for policy 1, policy_version 33161 (0.0009) [2023-12-27 05:53:41,783][20347] Updated weights for policy 0, policy_version 33209 (0.0006) [2023-12-27 05:53:41,844][20347] Updated weights for policy 0, policy_version 33219 (0.0005) [2023-12-27 05:53:41,906][20347] Updated weights for policy 0, policy_version 33229 (0.0007) [2023-12-27 05:53:42,345][20381] Updated weights for policy 1, policy_version 33171 (0.0009) [2023-12-27 05:53:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 17006592. Throughput: 0: 10474.1, 1: 10409.1. Samples: 17000600. Policy #0 lag: (min: 13.0, avg: 25.4, max: 45.0) [2023-12-27 05:53:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:42,412][20381] Updated weights for policy 1, policy_version 33181 (0.0010) [2023-12-27 05:53:42,471][20381] Updated weights for policy 1, policy_version 33191 (0.0008) [2023-12-27 05:53:42,594][20347] Updated weights for policy 0, policy_version 33239 (0.0009) [2023-12-27 05:53:42,655][20347] Updated weights for policy 0, policy_version 33249 (0.0010) [2023-12-27 05:53:42,716][20347] Updated weights for policy 0, policy_version 33259 (0.0010) [2023-12-27 05:53:43,176][20381] Updated weights for policy 1, policy_version 33201 (0.0011) [2023-12-27 05:53:43,224][20381] Updated weights for policy 1, policy_version 33211 (0.0010) [2023-12-27 05:53:43,281][20381] Updated weights for policy 1, policy_version 33221 (0.0010) [2023-12-27 05:53:43,322][20347] Updated weights for policy 0, policy_version 33269 (0.0007) [2023-12-27 05:53:43,336][20381] Updated weights for policy 1, policy_version 33231 (0.0010) [2023-12-27 05:53:43,376][20347] Updated weights for policy 0, policy_version 33279 (0.0006) [2023-12-27 05:53:43,437][20347] Updated weights for policy 0, policy_version 33289 (0.0007) [2023-12-27 05:53:44,018][20381] Updated weights for policy 1, policy_version 33241 (0.0010) [2023-12-27 05:53:44,069][20381] Updated weights for policy 1, policy_version 33251 (0.0006) [2023-12-27 05:53:44,097][20347] Updated weights for policy 0, policy_version 33299 (0.0007) [2023-12-27 05:53:44,122][20381] Updated weights for policy 1, policy_version 33261 (0.0010) [2023-12-27 05:53:44,153][20347] Updated weights for policy 0, policy_version 33309 (0.0006) [2023-12-27 05:53:44,215][20347] Updated weights for policy 0, policy_version 33319 (0.0007) [2023-12-27 05:53:44,723][20381] Updated weights for policy 1, policy_version 33271 (0.0007) [2023-12-27 05:53:44,784][20381] Updated weights for policy 1, policy_version 33281 (0.0011) [2023-12-27 05:53:44,848][20381] Updated weights for policy 1, policy_version 33291 (0.0011) [2023-12-27 05:53:44,929][20347] Updated weights for policy 0, policy_version 33329 (0.0007) [2023-12-27 05:53:44,989][20347] Updated weights for policy 0, policy_version 33339 (0.0005) [2023-12-27 05:53:45,052][20347] Updated weights for policy 0, policy_version 33349 (0.0007) [2023-12-27 05:53:45,107][20347] Updated weights for policy 0, policy_version 33359 (0.0007) [2023-12-27 05:53:45,529][20381] Updated weights for policy 1, policy_version 33301 (0.0011) [2023-12-27 05:53:45,584][20381] Updated weights for policy 1, policy_version 33311 (0.0010) [2023-12-27 05:53:45,647][20381] Updated weights for policy 1, policy_version 33321 (0.0011) [2023-12-27 05:53:45,748][20347] Updated weights for policy 0, policy_version 33369 (0.0006) [2023-12-27 05:53:45,805][20347] Updated weights for policy 0, policy_version 33379 (0.0005) [2023-12-27 05:53:45,876][20347] Updated weights for policy 0, policy_version 33389 (0.0006) [2023-12-27 05:53:46,287][20381] Updated weights for policy 1, policy_version 33331 (0.0009) [2023-12-27 05:53:46,339][20381] Updated weights for policy 1, policy_version 33341 (0.0005) [2023-12-27 05:53:46,389][20381] Updated weights for policy 1, policy_version 33351 (0.0008) [2023-12-27 05:53:46,411][20347] Updated weights for policy 0, policy_version 33399 (0.0010) [2023-12-27 05:53:46,458][20347] Updated weights for policy 0, policy_version 33409 (0.0010) [2023-12-27 05:53:46,508][20347] Updated weights for policy 0, policy_version 33419 (0.0010) [2023-12-27 05:53:47,076][20381] Updated weights for policy 1, policy_version 33361 (0.0010) [2023-12-27 05:53:47,120][20347] Updated weights for policy 0, policy_version 33429 (0.0008) [2023-12-27 05:53:47,138][20381] Updated weights for policy 1, policy_version 33371 (0.0005) [2023-12-27 05:53:47,177][20347] Updated weights for policy 0, policy_version 33439 (0.0010) [2023-12-27 05:53:47,190][20381] Updated weights for policy 1, policy_version 33381 (0.0005) [2023-12-27 05:53:47,231][20347] Updated weights for policy 0, policy_version 33449 (0.0010) [2023-12-27 05:53:47,241][20381] Updated weights for policy 1, policy_version 33391 (0.0005) [2023-12-27 05:53:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 17121280. Throughput: 0: 10451.8, 1: 10421.0. Samples: 17126572. Policy #0 lag: (min: 13.0, avg: 25.4, max: 45.0) [2023-12-27 05:53:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:47,826][20381] Updated weights for policy 1, policy_version 33401 (0.0007) [2023-12-27 05:53:47,887][20381] Updated weights for policy 1, policy_version 33411 (0.0007) [2023-12-27 05:53:47,913][20347] Updated weights for policy 0, policy_version 33459 (0.0010) [2023-12-27 05:53:47,945][20381] Updated weights for policy 1, policy_version 33421 (0.0008) [2023-12-27 05:53:47,967][20347] Updated weights for policy 0, policy_version 33469 (0.0010) [2023-12-27 05:53:48,031][20347] Updated weights for policy 0, policy_version 33479 (0.0010) [2023-12-27 05:53:48,551][20381] Updated weights for policy 1, policy_version 33431 (0.0008) [2023-12-27 05:53:48,608][20381] Updated weights for policy 1, policy_version 33441 (0.0008) [2023-12-27 05:53:48,670][20381] Updated weights for policy 1, policy_version 33451 (0.0006) [2023-12-27 05:53:48,713][20347] Updated weights for policy 0, policy_version 33489 (0.0009) [2023-12-27 05:53:48,775][20347] Updated weights for policy 0, policy_version 33499 (0.0009) [2023-12-27 05:53:48,831][20347] Updated weights for policy 0, policy_version 33509 (0.0008) [2023-12-27 05:53:48,896][20347] Updated weights for policy 0, policy_version 33519 (0.0006) [2023-12-27 05:53:49,307][20381] Updated weights for policy 1, policy_version 33461 (0.0008) [2023-12-27 05:53:49,374][20381] Updated weights for policy 1, policy_version 33471 (0.0006) [2023-12-27 05:53:49,436][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-27 05:53:49,519][20347] Updated weights for policy 0, policy_version 33529 (0.0010) [2023-12-27 05:53:49,568][20347] Updated weights for policy 0, policy_version 33539 (0.0010) [2023-12-27 05:53:49,621][20347] Updated weights for policy 0, policy_version 33549 (0.0010) [2023-12-27 05:53:49,995][20381] Updated weights for policy 1, policy_version 33481 (0.0006) [2023-12-27 05:53:50,059][20381] Updated weights for policy 1, policy_version 33491 (0.0007) [2023-12-27 05:53:50,116][20381] Updated weights for policy 1, policy_version 33501 (0.0007) [2023-12-27 05:53:50,184][20381] Updated weights for policy 1, policy_version 33511 (0.0008) [2023-12-27 05:53:50,340][20347] Updated weights for policy 0, policy_version 33559 (0.0007) [2023-12-27 05:53:50,398][20347] Updated weights for policy 0, policy_version 33569 (0.0008) [2023-12-27 05:53:50,450][20347] Updated weights for policy 0, policy_version 33579 (0.0010) [2023-12-27 05:53:50,819][20381] Updated weights for policy 1, policy_version 33521 (0.0006) [2023-12-27 05:53:50,874][20381] Updated weights for policy 1, policy_version 33531 (0.0005) [2023-12-27 05:53:50,931][20381] Updated weights for policy 1, policy_version 33541 (0.0006) [2023-12-27 05:53:51,130][20347] Updated weights for policy 0, policy_version 33589 (0.0009) [2023-12-27 05:53:51,195][20347] Updated weights for policy 0, policy_version 33599 (0.0011) [2023-12-27 05:53:51,255][20347] Updated weights for policy 0, policy_version 33609 (0.0010) [2023-12-27 05:53:51,563][20381] Updated weights for policy 1, policy_version 33551 (0.0006) [2023-12-27 05:53:51,625][20381] Updated weights for policy 1, policy_version 33561 (0.0009) [2023-12-27 05:53:51,688][20381] Updated weights for policy 1, policy_version 33571 (0.0011) [2023-12-27 05:53:51,903][20347] Updated weights for policy 0, policy_version 33619 (0.0009) [2023-12-27 05:53:51,966][20347] Updated weights for policy 0, policy_version 33629 (0.0008) [2023-12-27 05:53:52,024][20347] Updated weights for policy 0, policy_version 33639 (0.0007) [2023-12-27 05:53:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 17219584. Throughput: 0: 10403.9, 1: 10433.9. Samples: 17190332. Policy #0 lag: (min: 13.0, avg: 25.4, max: 45.0) [2023-12-27 05:53:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000033648_8617984.pth... [2023-12-27 05:53:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000032464_8314880.pth [2023-12-27 05:53:52,375][20381] Updated weights for policy 1, policy_version 33581 (0.0011) [2023-12-27 05:53:52,435][20381] Updated weights for policy 1, policy_version 33591 (0.0010) [2023-12-27 05:53:52,493][20381] Updated weights for policy 1, policy_version 33601 (0.0008) [2023-12-27 05:53:52,534][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000033608_8609792.pth... [2023-12-27 05:53:52,537][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000032336_8282112.pth [2023-12-27 05:53:52,730][20347] Updated weights for policy 0, policy_version 33649 (0.0007) [2023-12-27 05:53:52,783][20347] Updated weights for policy 0, policy_version 33659 (0.0008) [2023-12-27 05:53:52,831][20347] Updated weights for policy 0, policy_version 33669 (0.0008) [2023-12-27 05:53:52,884][20347] Updated weights for policy 0, policy_version 33679 (0.0008) [2023-12-27 05:53:53,198][20381] Updated weights for policy 1, policy_version 33611 (0.0009) [2023-12-27 05:53:53,247][20381] Updated weights for policy 1, policy_version 33621 (0.0010) [2023-12-27 05:53:53,293][20381] Updated weights for policy 1, policy_version 33631 (0.0010) [2023-12-27 05:53:53,522][20347] Updated weights for policy 0, policy_version 33689 (0.0007) [2023-12-27 05:53:53,580][20347] Updated weights for policy 0, policy_version 33699 (0.0007) [2023-12-27 05:53:53,645][20347] Updated weights for policy 0, policy_version 33709 (0.0008) [2023-12-27 05:53:54,038][20381] Updated weights for policy 1, policy_version 33641 (0.0010) [2023-12-27 05:53:54,098][20381] Updated weights for policy 1, policy_version 33651 (0.0007) [2023-12-27 05:53:54,157][20381] Updated weights for policy 1, policy_version 33661 (0.0007) [2023-12-27 05:53:54,216][20381] Updated weights for policy 1, policy_version 33671 (0.0007) [2023-12-27 05:53:54,222][20347] Updated weights for policy 0, policy_version 33719 (0.0007) [2023-12-27 05:53:54,269][20347] Updated weights for policy 0, policy_version 33729 (0.0007) [2023-12-27 05:53:54,319][20347] Updated weights for policy 0, policy_version 33739 (0.0006) [2023-12-27 05:53:54,904][20381] Updated weights for policy 1, policy_version 33681 (0.0010) [2023-12-27 05:53:54,953][20381] Updated weights for policy 1, policy_version 33691 (0.0010) [2023-12-27 05:53:54,984][20347] Updated weights for policy 0, policy_version 33749 (0.0008) [2023-12-27 05:53:55,014][20381] Updated weights for policy 1, policy_version 33701 (0.0011) [2023-12-27 05:53:55,049][20347] Updated weights for policy 0, policy_version 33759 (0.0008) [2023-12-27 05:53:55,114][20347] Updated weights for policy 0, policy_version 33769 (0.0008) [2023-12-27 05:53:55,583][20381] Updated weights for policy 1, policy_version 33711 (0.0011) [2023-12-27 05:53:55,631][20381] Updated weights for policy 1, policy_version 33721 (0.0008) [2023-12-27 05:53:55,692][20381] Updated weights for policy 1, policy_version 33731 (0.0005) [2023-12-27 05:53:55,753][20347] Updated weights for policy 0, policy_version 33779 (0.0008) [2023-12-27 05:53:55,805][20347] Updated weights for policy 0, policy_version 33789 (0.0010) [2023-12-27 05:53:55,857][20347] Updated weights for policy 0, policy_version 33799 (0.0010) [2023-12-27 05:53:56,280][20381] Updated weights for policy 1, policy_version 33741 (0.0008) [2023-12-27 05:53:56,333][20381] Updated weights for policy 1, policy_version 33751 (0.0007) [2023-12-27 05:53:56,380][20381] Updated weights for policy 1, policy_version 33761 (0.0005) [2023-12-27 05:53:56,507][20347] Updated weights for policy 0, policy_version 33809 (0.0009) [2023-12-27 05:53:56,575][20347] Updated weights for policy 0, policy_version 33819 (0.0005) [2023-12-27 05:53:56,630][20347] Updated weights for policy 0, policy_version 33829 (0.0005) [2023-12-27 05:53:56,689][20347] Updated weights for policy 0, policy_version 33839 (0.0005) [2023-12-27 05:53:57,024][20381] Updated weights for policy 1, policy_version 33771 (0.0007) [2023-12-27 05:53:57,075][20381] Updated weights for policy 1, policy_version 33781 (0.0010) [2023-12-27 05:53:57,129][20381] Updated weights for policy 1, policy_version 33791 (0.0010) [2023-12-27 05:53:57,175][20347] Updated weights for policy 0, policy_version 33849 (0.0009) [2023-12-27 05:53:57,225][20347] Updated weights for policy 0, policy_version 33859 (0.0010) [2023-12-27 05:53:57,276][20347] Updated weights for policy 0, policy_version 33869 (0.0010) [2023-12-27 05:53:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20966.0). Total num frames: 17334272. Throughput: 0: 10383.1, 1: 10505.6. Samples: 17315928. Policy #0 lag: (min: 13.0, avg: 25.4, max: 45.0) [2023-12-27 05:53:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:53:57,757][20381] Updated weights for policy 1, policy_version 33801 (0.0010) [2023-12-27 05:53:57,820][20381] Updated weights for policy 1, policy_version 33811 (0.0009) [2023-12-27 05:53:57,875][20381] Updated weights for policy 1, policy_version 33821 (0.0009) [2023-12-27 05:53:57,930][20381] Updated weights for policy 1, policy_version 33831 (0.0006) [2023-12-27 05:53:57,933][20347] Updated weights for policy 0, policy_version 33879 (0.0010) [2023-12-27 05:53:57,986][20347] Updated weights for policy 0, policy_version 33889 (0.0010) [2023-12-27 05:53:58,034][20347] Updated weights for policy 0, policy_version 33899 (0.0006) [2023-12-27 05:53:58,604][20381] Updated weights for policy 1, policy_version 33841 (0.0008) [2023-12-27 05:53:58,667][20381] Updated weights for policy 1, policy_version 33851 (0.0010) [2023-12-27 05:53:58,731][20381] Updated weights for policy 1, policy_version 33861 (0.0011) [2023-12-27 05:53:58,749][20347] Updated weights for policy 0, policy_version 33909 (0.0007) [2023-12-27 05:53:58,818][20347] Updated weights for policy 0, policy_version 33919 (0.0010) [2023-12-27 05:53:58,884][20347] Updated weights for policy 0, policy_version 33929 (0.0009) [2023-12-27 05:53:59,422][20381] Updated weights for policy 1, policy_version 33871 (0.0011) [2023-12-27 05:53:59,470][20381] Updated weights for policy 1, policy_version 33881 (0.0007) [2023-12-27 05:53:59,518][20381] Updated weights for policy 1, policy_version 33891 (0.0007) [2023-12-27 05:53:59,523][20347] Updated weights for policy 0, policy_version 33939 (0.0009) [2023-12-27 05:53:59,583][20347] Updated weights for policy 0, policy_version 33949 (0.0006) [2023-12-27 05:53:59,647][20347] Updated weights for policy 0, policy_version 33959 (0.0006) [2023-12-27 05:54:00,255][20381] Updated weights for policy 1, policy_version 33901 (0.0006) [2023-12-27 05:54:00,276][20347] Updated weights for policy 0, policy_version 33969 (0.0006) [2023-12-27 05:54:00,310][20381] Updated weights for policy 1, policy_version 33911 (0.0006) [2023-12-27 05:54:00,339][20347] Updated weights for policy 0, policy_version 33979 (0.0007) [2023-12-27 05:54:00,369][20381] Updated weights for policy 1, policy_version 33921 (0.0008) [2023-12-27 05:54:00,393][20347] Updated weights for policy 0, policy_version 33989 (0.0006) [2023-12-27 05:54:00,444][20347] Updated weights for policy 0, policy_version 33999 (0.0005) [2023-12-27 05:54:01,005][20347] Updated weights for policy 0, policy_version 34009 (0.0005) [2023-12-27 05:54:01,066][20347] Updated weights for policy 0, policy_version 34019 (0.0006) [2023-12-27 05:54:01,116][20381] Updated weights for policy 1, policy_version 33931 (0.0006) [2023-12-27 05:54:01,133][20347] Updated weights for policy 0, policy_version 34029 (0.0006) [2023-12-27 05:54:01,173][20381] Updated weights for policy 1, policy_version 33941 (0.0007) [2023-12-27 05:54:01,226][20381] Updated weights for policy 1, policy_version 33951 (0.0006) [2023-12-27 05:54:01,779][20347] Updated weights for policy 0, policy_version 34039 (0.0007) [2023-12-27 05:54:01,834][20347] Updated weights for policy 0, policy_version 34049 (0.0008) [2023-12-27 05:54:01,886][20347] Updated weights for policy 0, policy_version 34059 (0.0007) [2023-12-27 05:54:01,914][20381] Updated weights for policy 1, policy_version 33961 (0.0007) [2023-12-27 05:54:01,976][20381] Updated weights for policy 1, policy_version 33971 (0.0008) [2023-12-27 05:54:02,027][20381] Updated weights for policy 1, policy_version 33981 (0.0007) [2023-12-27 05:54:02,091][20381] Updated weights for policy 1, policy_version 33991 (0.0007) [2023-12-27 05:54:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 17432576. Throughput: 0: 10383.7, 1: 10501.9. Samples: 17442088. Policy #0 lag: (min: 29.0, avg: 36.0, max: 61.0) [2023-12-27 05:54:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:02,643][20347] Updated weights for policy 0, policy_version 34069 (0.0008) [2023-12-27 05:54:02,703][20347] Updated weights for policy 0, policy_version 34079 (0.0007) [2023-12-27 05:54:02,716][20381] Updated weights for policy 1, policy_version 34001 (0.0006) [2023-12-27 05:54:02,758][20347] Updated weights for policy 0, policy_version 34089 (0.0006) [2023-12-27 05:54:02,771][20381] Updated weights for policy 1, policy_version 34011 (0.0005) [2023-12-27 05:54:02,824][20381] Updated weights for policy 1, policy_version 34021 (0.0006) [2023-12-27 05:54:03,458][20347] Updated weights for policy 0, policy_version 34099 (0.0006) [2023-12-27 05:54:03,470][20381] Updated weights for policy 1, policy_version 34031 (0.0006) [2023-12-27 05:54:03,516][20347] Updated weights for policy 0, policy_version 34109 (0.0007) [2023-12-27 05:54:03,521][20381] Updated weights for policy 1, policy_version 34041 (0.0006) [2023-12-27 05:54:03,567][20347] Updated weights for policy 0, policy_version 34119 (0.0006) [2023-12-27 05:54:03,581][20381] Updated weights for policy 1, policy_version 34051 (0.0007) [2023-12-27 05:54:04,253][20381] Updated weights for policy 1, policy_version 34061 (0.0006) [2023-12-27 05:54:04,262][20347] Updated weights for policy 0, policy_version 34129 (0.0007) [2023-12-27 05:54:04,314][20381] Updated weights for policy 1, policy_version 34071 (0.0006) [2023-12-27 05:54:04,327][20347] Updated weights for policy 0, policy_version 34139 (0.0007) [2023-12-27 05:54:04,370][20381] Updated weights for policy 1, policy_version 34081 (0.0008) [2023-12-27 05:54:04,384][20347] Updated weights for policy 0, policy_version 34149 (0.0006) [2023-12-27 05:54:04,431][20347] Updated weights for policy 0, policy_version 34159 (0.0008) [2023-12-27 05:54:05,062][20381] Updated weights for policy 1, policy_version 34091 (0.0007) [2023-12-27 05:54:05,097][20347] Updated weights for policy 0, policy_version 34169 (0.0007) [2023-12-27 05:54:05,113][20381] Updated weights for policy 1, policy_version 34101 (0.0006) [2023-12-27 05:54:05,149][20347] Updated weights for policy 0, policy_version 34179 (0.0005) [2023-12-27 05:54:05,172][20381] Updated weights for policy 1, policy_version 34111 (0.0007) [2023-12-27 05:54:05,203][20347] Updated weights for policy 0, policy_version 34189 (0.0006) [2023-12-27 05:54:05,829][20381] Updated weights for policy 1, policy_version 34121 (0.0006) [2023-12-27 05:54:05,882][20347] Updated weights for policy 0, policy_version 34199 (0.0008) [2023-12-27 05:54:05,888][20381] Updated weights for policy 1, policy_version 34131 (0.0006) [2023-12-27 05:54:05,943][20347] Updated weights for policy 0, policy_version 34209 (0.0006) [2023-12-27 05:54:05,948][20381] Updated weights for policy 1, policy_version 34141 (0.0006) [2023-12-27 05:54:06,005][20347] Updated weights for policy 0, policy_version 34219 (0.0005) [2023-12-27 05:54:06,011][20381] Updated weights for policy 1, policy_version 34151 (0.0006) [2023-12-27 05:54:06,629][20381] Updated weights for policy 1, policy_version 34161 (0.0008) [2023-12-27 05:54:06,688][20381] Updated weights for policy 1, policy_version 34171 (0.0007) [2023-12-27 05:54:06,709][20347] Updated weights for policy 0, policy_version 34229 (0.0006) [2023-12-27 05:54:06,743][20381] Updated weights for policy 1, policy_version 34181 (0.0007) [2023-12-27 05:54:06,760][20347] Updated weights for policy 0, policy_version 34239 (0.0005) [2023-12-27 05:54:06,816][20347] Updated weights for policy 0, policy_version 34249 (0.0007) [2023-12-27 05:54:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20938.2). Total num frames: 17530880. Throughput: 0: 10345.0, 1: 10523.0. Samples: 17503964. Policy #0 lag: (min: 29.0, avg: 36.0, max: 61.0) [2023-12-27 05:54:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000034184_8757248.pth... [2023-12-27 05:54:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000034256_8773632.pth... [2023-12-27 05:54:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000032976_8445952.pth [2023-12-27 05:54:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000033040_8462336.pth [2023-12-27 05:54:07,436][20381] Updated weights for policy 1, policy_version 34191 (0.0008) [2023-12-27 05:54:07,501][20381] Updated weights for policy 1, policy_version 34201 (0.0007) [2023-12-27 05:54:07,527][20347] Updated weights for policy 0, policy_version 34259 (0.0007) [2023-12-27 05:54:07,552][20381] Updated weights for policy 1, policy_version 34211 (0.0006) [2023-12-27 05:54:07,573][20347] Updated weights for policy 0, policy_version 34269 (0.0006) [2023-12-27 05:54:07,626][20347] Updated weights for policy 0, policy_version 34279 (0.0009) [2023-12-27 05:54:08,143][20381] Updated weights for policy 1, policy_version 34221 (0.0006) [2023-12-27 05:54:08,197][20381] Updated weights for policy 1, policy_version 34231 (0.0008) [2023-12-27 05:54:08,262][20381] Updated weights for policy 1, policy_version 34241 (0.0008) [2023-12-27 05:54:08,374][20347] Updated weights for policy 0, policy_version 34289 (0.0008) [2023-12-27 05:54:08,431][20347] Updated weights for policy 0, policy_version 34299 (0.0009) [2023-12-27 05:54:08,487][20347] Updated weights for policy 0, policy_version 34309 (0.0008) [2023-12-27 05:54:08,541][20347] Updated weights for policy 0, policy_version 34319 (0.0007) [2023-12-27 05:54:08,957][20381] Updated weights for policy 1, policy_version 34251 (0.0007) [2023-12-27 05:54:09,019][20381] Updated weights for policy 1, policy_version 34261 (0.0006) [2023-12-27 05:54:09,077][20381] Updated weights for policy 1, policy_version 34271 (0.0010) [2023-12-27 05:54:09,272][20347] Updated weights for policy 0, policy_version 34329 (0.0008) [2023-12-27 05:54:09,335][20347] Updated weights for policy 0, policy_version 34339 (0.0007) [2023-12-27 05:54:09,400][20347] Updated weights for policy 0, policy_version 34349 (0.0007) [2023-12-27 05:54:09,733][20381] Updated weights for policy 1, policy_version 34281 (0.0006) [2023-12-27 05:54:09,789][20381] Updated weights for policy 1, policy_version 34291 (0.0007) [2023-12-27 05:54:09,851][20381] Updated weights for policy 1, policy_version 34301 (0.0006) [2023-12-27 05:54:09,913][20381] Updated weights for policy 1, policy_version 34311 (0.0008) [2023-12-27 05:54:10,114][20347] Updated weights for policy 0, policy_version 34359 (0.0009) [2023-12-27 05:54:10,181][20347] Updated weights for policy 0, policy_version 34369 (0.0009) [2023-12-27 05:54:10,239][20347] Updated weights for policy 0, policy_version 34379 (0.0008) [2023-12-27 05:54:10,590][20381] Updated weights for policy 1, policy_version 34321 (0.0007) [2023-12-27 05:54:10,650][20381] Updated weights for policy 1, policy_version 34331 (0.0008) [2023-12-27 05:54:10,711][20381] Updated weights for policy 1, policy_version 34341 (0.0008) [2023-12-27 05:54:10,880][20347] Updated weights for policy 0, policy_version 34389 (0.0007) [2023-12-27 05:54:10,933][20347] Updated weights for policy 0, policy_version 34399 (0.0006) [2023-12-27 05:54:10,985][20347] Updated weights for policy 0, policy_version 34409 (0.0008) [2023-12-27 05:54:11,502][20381] Updated weights for policy 1, policy_version 34351 (0.0006) [2023-12-27 05:54:11,565][20381] Updated weights for policy 1, policy_version 34361 (0.0007) [2023-12-27 05:54:11,626][20381] Updated weights for policy 1, policy_version 34371 (0.0007) [2023-12-27 05:54:11,688][20347] Updated weights for policy 0, policy_version 34419 (0.0008) [2023-12-27 05:54:11,752][20347] Updated weights for policy 0, policy_version 34429 (0.0009) [2023-12-27 05:54:11,812][20347] Updated weights for policy 0, policy_version 34439 (0.0007) [2023-12-27 05:54:12,277][20381] Updated weights for policy 1, policy_version 34381 (0.0008) [2023-12-27 05:54:12,337][20381] Updated weights for policy 1, policy_version 34391 (0.0008) [2023-12-27 05:54:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 17629184. Throughput: 0: 10311.6, 1: 10528.4. Samples: 17625552. Policy #0 lag: (min: 29.0, avg: 36.0, max: 61.0) [2023-12-27 05:54:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:12,396][20381] Updated weights for policy 1, policy_version 34401 (0.0009) [2023-12-27 05:54:12,455][20347] Updated weights for policy 0, policy_version 34449 (0.0007) [2023-12-27 05:54:12,511][20347] Updated weights for policy 0, policy_version 34459 (0.0006) [2023-12-27 05:54:12,577][20347] Updated weights for policy 0, policy_version 34469 (0.0008) [2023-12-27 05:54:12,636][20347] Updated weights for policy 0, policy_version 34479 (0.0008) [2023-12-27 05:54:13,047][20381] Updated weights for policy 1, policy_version 34411 (0.0008) [2023-12-27 05:54:13,111][20381] Updated weights for policy 1, policy_version 34421 (0.0007) [2023-12-27 05:54:13,167][20381] Updated weights for policy 1, policy_version 34431 (0.0007) [2023-12-27 05:54:13,267][20347] Updated weights for policy 0, policy_version 34489 (0.0007) [2023-12-27 05:54:13,321][20347] Updated weights for policy 0, policy_version 34499 (0.0008) [2023-12-27 05:54:13,382][20347] Updated weights for policy 0, policy_version 34509 (0.0007) [2023-12-27 05:54:13,869][20381] Updated weights for policy 1, policy_version 34441 (0.0008) [2023-12-27 05:54:13,928][20381] Updated weights for policy 1, policy_version 34451 (0.0007) [2023-12-27 05:54:13,972][20381] Updated weights for policy 1, policy_version 34461 (0.0007) [2023-12-27 05:54:13,993][20347] Updated weights for policy 0, policy_version 34519 (0.0006) [2023-12-27 05:54:14,018][20381] Updated weights for policy 1, policy_version 34471 (0.0005) [2023-12-27 05:54:14,046][20347] Updated weights for policy 0, policy_version 34529 (0.0007) [2023-12-27 05:54:14,108][20347] Updated weights for policy 0, policy_version 34539 (0.0008) [2023-12-27 05:54:14,661][20381] Updated weights for policy 1, policy_version 34481 (0.0007) [2023-12-27 05:54:14,711][20381] Updated weights for policy 1, policy_version 34491 (0.0008) [2023-12-27 05:54:14,761][20381] Updated weights for policy 1, policy_version 34501 (0.0008) [2023-12-27 05:54:14,812][20347] Updated weights for policy 0, policy_version 34549 (0.0007) [2023-12-27 05:54:14,868][20347] Updated weights for policy 0, policy_version 34559 (0.0007) [2023-12-27 05:54:14,928][20347] Updated weights for policy 0, policy_version 34569 (0.0007) [2023-12-27 05:54:15,490][20381] Updated weights for policy 1, policy_version 34511 (0.0006) [2023-12-27 05:54:15,552][20381] Updated weights for policy 1, policy_version 34521 (0.0008) [2023-12-27 05:54:15,596][20347] Updated weights for policy 0, policy_version 34579 (0.0009) [2023-12-27 05:54:15,603][20381] Updated weights for policy 1, policy_version 34531 (0.0007) [2023-12-27 05:54:15,650][20347] Updated weights for policy 0, policy_version 34589 (0.0007) [2023-12-27 05:54:15,702][20347] Updated weights for policy 0, policy_version 34599 (0.0008) [2023-12-27 05:54:16,269][20381] Updated weights for policy 1, policy_version 34541 (0.0007) [2023-12-27 05:54:16,327][20381] Updated weights for policy 1, policy_version 34551 (0.0008) [2023-12-27 05:54:16,365][20347] Updated weights for policy 0, policy_version 34609 (0.0007) [2023-12-27 05:54:16,389][20381] Updated weights for policy 1, policy_version 34561 (0.0006) [2023-12-27 05:54:16,423][20347] Updated weights for policy 0, policy_version 34619 (0.0006) [2023-12-27 05:54:16,481][20347] Updated weights for policy 0, policy_version 34629 (0.0006) [2023-12-27 05:54:16,529][20347] Updated weights for policy 0, policy_version 34639 (0.0008) [2023-12-27 05:54:16,958][20381] Updated weights for policy 1, policy_version 34571 (0.0006) [2023-12-27 05:54:17,026][20381] Updated weights for policy 1, policy_version 34581 (0.0006) [2023-12-27 05:54:17,088][20381] Updated weights for policy 1, policy_version 34591 (0.0007) [2023-12-27 05:54:17,165][20347] Updated weights for policy 0, policy_version 34649 (0.0006) [2023-12-27 05:54:17,218][20347] Updated weights for policy 0, policy_version 34659 (0.0005) [2023-12-27 05:54:17,266][20347] Updated weights for policy 0, policy_version 34669 (0.0009) [2023-12-27 05:54:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20938.2). Total num frames: 17743872. Throughput: 0: 10347.2, 1: 10461.6. Samples: 17750092. Policy #0 lag: (min: 29.0, avg: 36.0, max: 61.0) [2023-12-27 05:54:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:17,685][20381] Updated weights for policy 1, policy_version 34601 (0.0007) [2023-12-27 05:54:17,749][20381] Updated weights for policy 1, policy_version 34611 (0.0008) [2023-12-27 05:54:17,807][20381] Updated weights for policy 1, policy_version 34621 (0.0008) [2023-12-27 05:54:17,866][20381] Updated weights for policy 1, policy_version 34631 (0.0007) [2023-12-27 05:54:17,960][20347] Updated weights for policy 0, policy_version 34679 (0.0006) [2023-12-27 05:54:18,013][20347] Updated weights for policy 0, policy_version 34689 (0.0006) [2023-12-27 05:54:18,061][20347] Updated weights for policy 0, policy_version 34699 (0.0007) [2023-12-27 05:54:18,586][20381] Updated weights for policy 1, policy_version 34641 (0.0008) [2023-12-27 05:54:18,642][20381] Updated weights for policy 1, policy_version 34651 (0.0008) [2023-12-27 05:54:18,698][20381] Updated weights for policy 1, policy_version 34661 (0.0008) [2023-12-27 05:54:18,731][20347] Updated weights for policy 0, policy_version 34709 (0.0007) [2023-12-27 05:54:18,786][20347] Updated weights for policy 0, policy_version 34719 (0.0007) [2023-12-27 05:54:18,841][20347] Updated weights for policy 0, policy_version 34729 (0.0008) [2023-12-27 05:54:19,371][20381] Updated weights for policy 1, policy_version 34671 (0.0008) [2023-12-27 05:54:19,428][20381] Updated weights for policy 1, policy_version 34681 (0.0007) [2023-12-27 05:54:19,429][20347] Updated weights for policy 0, policy_version 34739 (0.0008) [2023-12-27 05:54:19,480][20381] Updated weights for policy 1, policy_version 34691 (0.0006) [2023-12-27 05:54:19,490][20347] Updated weights for policy 0, policy_version 34749 (0.0007) [2023-12-27 05:54:19,554][20347] Updated weights for policy 0, policy_version 34759 (0.0007) [2023-12-27 05:54:20,225][20381] Updated weights for policy 1, policy_version 34701 (0.0007) [2023-12-27 05:54:20,242][20347] Updated weights for policy 0, policy_version 34769 (0.0008) [2023-12-27 05:54:20,286][20381] Updated weights for policy 1, policy_version 34711 (0.0008) [2023-12-27 05:54:20,306][20347] Updated weights for policy 0, policy_version 34779 (0.0007) [2023-12-27 05:54:20,350][20381] Updated weights for policy 1, policy_version 34721 (0.0007) [2023-12-27 05:54:20,370][20347] Updated weights for policy 0, policy_version 34789 (0.0006) [2023-12-27 05:54:20,429][20347] Updated weights for policy 0, policy_version 34799 (0.0005) [2023-12-27 05:54:21,066][20381] Updated weights for policy 1, policy_version 34731 (0.0007) [2023-12-27 05:54:21,091][20347] Updated weights for policy 0, policy_version 34809 (0.0006) [2023-12-27 05:54:21,126][20381] Updated weights for policy 1, policy_version 34741 (0.0007) [2023-12-27 05:54:21,153][20347] Updated weights for policy 0, policy_version 34819 (0.0006) [2023-12-27 05:54:21,184][20381] Updated weights for policy 1, policy_version 34751 (0.0007) [2023-12-27 05:54:21,214][20347] Updated weights for policy 0, policy_version 34829 (0.0005) [2023-12-27 05:54:21,881][20347] Updated weights for policy 0, policy_version 34839 (0.0007) [2023-12-27 05:54:21,926][20381] Updated weights for policy 1, policy_version 34761 (0.0008) [2023-12-27 05:54:21,936][20347] Updated weights for policy 0, policy_version 34849 (0.0008) [2023-12-27 05:54:21,976][20381] Updated weights for policy 1, policy_version 34771 (0.0010) [2023-12-27 05:54:21,992][20347] Updated weights for policy 0, policy_version 34859 (0.0006) [2023-12-27 05:54:22,031][20381] Updated weights for policy 1, policy_version 34781 (0.0011) [2023-12-27 05:54:22,090][20381] Updated weights for policy 1, policy_version 34791 (0.0011) [2023-12-27 05:54:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 17842176. Throughput: 0: 10343.8, 1: 10418.4. Samples: 17811564. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 05:54:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000034792_8912896.pth... [2023-12-27 05:54:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000034864_8929280.pth... [2023-12-27 05:54:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000033608_8609792.pth [2023-12-27 05:54:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000033648_8617984.pth [2023-12-27 05:54:22,636][20347] Updated weights for policy 0, policy_version 34869 (0.0007) [2023-12-27 05:54:22,693][20347] Updated weights for policy 0, policy_version 34879 (0.0007) [2023-12-27 05:54:22,748][20347] Updated weights for policy 0, policy_version 34889 (0.0007) [2023-12-27 05:54:22,837][20381] Updated weights for policy 1, policy_version 34801 (0.0011) [2023-12-27 05:54:22,890][20381] Updated weights for policy 1, policy_version 34811 (0.0011) [2023-12-27 05:54:22,947][20381] Updated weights for policy 1, policy_version 34821 (0.0011) [2023-12-27 05:54:23,481][20347] Updated weights for policy 0, policy_version 34899 (0.0007) [2023-12-27 05:54:23,524][20347] Updated weights for policy 0, policy_version 34909 (0.0007) [2023-12-27 05:54:23,545][20381] Updated weights for policy 1, policy_version 34831 (0.0007) [2023-12-27 05:54:23,569][20347] Updated weights for policy 0, policy_version 34919 (0.0005) [2023-12-27 05:54:23,590][20381] Updated weights for policy 1, policy_version 34841 (0.0005) [2023-12-27 05:54:23,635][20381] Updated weights for policy 1, policy_version 34851 (0.0007) [2023-12-27 05:54:24,304][20347] Updated weights for policy 0, policy_version 34929 (0.0006) [2023-12-27 05:54:24,329][20381] Updated weights for policy 1, policy_version 34861 (0.0009) [2023-12-27 05:54:24,360][20347] Updated weights for policy 0, policy_version 34939 (0.0007) [2023-12-27 05:54:24,388][20381] Updated weights for policy 1, policy_version 34871 (0.0006) [2023-12-27 05:54:24,419][20347] Updated weights for policy 0, policy_version 34949 (0.0006) [2023-12-27 05:54:24,444][20381] Updated weights for policy 1, policy_version 34881 (0.0006) [2023-12-27 05:54:24,474][20347] Updated weights for policy 0, policy_version 34959 (0.0008) [2023-12-27 05:54:25,131][20381] Updated weights for policy 1, policy_version 34891 (0.0006) [2023-12-27 05:54:25,155][20347] Updated weights for policy 0, policy_version 34969 (0.0010) [2023-12-27 05:54:25,183][20381] Updated weights for policy 1, policy_version 34901 (0.0005) [2023-12-27 05:54:25,207][20347] Updated weights for policy 0, policy_version 34979 (0.0010) [2023-12-27 05:54:25,239][20381] Updated weights for policy 1, policy_version 34911 (0.0006) [2023-12-27 05:54:25,263][20347] Updated weights for policy 0, policy_version 34989 (0.0011) [2023-12-27 05:54:25,928][20381] Updated weights for policy 1, policy_version 34921 (0.0006) [2023-12-27 05:54:25,959][20347] Updated weights for policy 0, policy_version 34999 (0.0007) [2023-12-27 05:54:25,984][20381] Updated weights for policy 1, policy_version 34931 (0.0010) [2023-12-27 05:54:26,011][20347] Updated weights for policy 0, policy_version 35009 (0.0005) [2023-12-27 05:54:26,047][20381] Updated weights for policy 1, policy_version 34941 (0.0010) [2023-12-27 05:54:26,061][20347] Updated weights for policy 0, policy_version 35019 (0.0005) [2023-12-27 05:54:26,099][20381] Updated weights for policy 1, policy_version 34951 (0.0010) [2023-12-27 05:54:26,584][20347] Updated weights for policy 0, policy_version 35029 (0.0008) [2023-12-27 05:54:26,646][20347] Updated weights for policy 0, policy_version 35039 (0.0010) [2023-12-27 05:54:26,706][20347] Updated weights for policy 0, policy_version 35049 (0.0010) [2023-12-27 05:54:26,727][20381] Updated weights for policy 1, policy_version 34961 (0.0007) [2023-12-27 05:54:26,782][20381] Updated weights for policy 1, policy_version 34971 (0.0007) [2023-12-27 05:54:26,837][20381] Updated weights for policy 1, policy_version 34981 (0.0010) [2023-12-27 05:54:27,352][20347] Updated weights for policy 0, policy_version 35059 (0.0008) [2023-12-27 05:54:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 17940480. Throughput: 0: 10362.0, 1: 10355.6. Samples: 17932896. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 05:54:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:27,405][20347] Updated weights for policy 0, policy_version 35069 (0.0005) [2023-12-27 05:54:27,465][20347] Updated weights for policy 0, policy_version 35079 (0.0007) [2023-12-27 05:54:27,496][20381] Updated weights for policy 1, policy_version 34991 (0.0007) [2023-12-27 05:54:27,549][20381] Updated weights for policy 1, policy_version 35001 (0.0007) [2023-12-27 05:54:27,595][20381] Updated weights for policy 1, policy_version 35011 (0.0006) [2023-12-27 05:54:28,029][20347] Updated weights for policy 0, policy_version 35089 (0.0010) [2023-12-27 05:54:28,083][20347] Updated weights for policy 0, policy_version 35099 (0.0006) [2023-12-27 05:54:28,133][20347] Updated weights for policy 0, policy_version 35109 (0.0006) [2023-12-27 05:54:28,179][20381] Updated weights for policy 1, policy_version 35021 (0.0005) [2023-12-27 05:54:28,186][20347] Updated weights for policy 0, policy_version 35119 (0.0007) [2023-12-27 05:54:28,230][20381] Updated weights for policy 1, policy_version 35031 (0.0005) [2023-12-27 05:54:28,276][20381] Updated weights for policy 1, policy_version 35041 (0.0005) [2023-12-27 05:54:28,809][20347] Updated weights for policy 0, policy_version 35129 (0.0005) [2023-12-27 05:54:28,831][20381] Updated weights for policy 1, policy_version 35051 (0.0005) [2023-12-27 05:54:28,853][20347] Updated weights for policy 0, policy_version 35139 (0.0005) [2023-12-27 05:54:28,880][20381] Updated weights for policy 1, policy_version 35061 (0.0007) [2023-12-27 05:54:28,904][20347] Updated weights for policy 0, policy_version 35149 (0.0007) [2023-12-27 05:54:28,930][20381] Updated weights for policy 1, policy_version 35071 (0.0007) [2023-12-27 05:54:29,555][20381] Updated weights for policy 1, policy_version 35081 (0.0007) [2023-12-27 05:54:29,604][20347] Updated weights for policy 0, policy_version 35159 (0.0006) [2023-12-27 05:54:29,613][20381] Updated weights for policy 1, policy_version 35091 (0.0006) [2023-12-27 05:54:29,653][20347] Updated weights for policy 0, policy_version 35169 (0.0005) [2023-12-27 05:54:29,670][20381] Updated weights for policy 1, policy_version 35101 (0.0006) [2023-12-27 05:54:29,702][20347] Updated weights for policy 0, policy_version 35179 (0.0006) [2023-12-27 05:54:29,723][20381] Updated weights for policy 1, policy_version 35111 (0.0006) [2023-12-27 05:54:30,369][20347] Updated weights for policy 0, policy_version 35189 (0.0006) [2023-12-27 05:54:30,431][20347] Updated weights for policy 0, policy_version 35199 (0.0005) [2023-12-27 05:54:30,475][20381] Updated weights for policy 1, policy_version 35121 (0.0007) [2023-12-27 05:54:30,489][20347] Updated weights for policy 0, policy_version 35209 (0.0006) [2023-12-27 05:54:30,530][20381] Updated weights for policy 1, policy_version 35131 (0.0007) [2023-12-27 05:54:30,584][20381] Updated weights for policy 1, policy_version 35141 (0.0007) [2023-12-27 05:54:31,053][20347] Updated weights for policy 0, policy_version 35219 (0.0008) [2023-12-27 05:54:31,115][20347] Updated weights for policy 0, policy_version 35229 (0.0008) [2023-12-27 05:54:31,175][20347] Updated weights for policy 0, policy_version 35239 (0.0007) [2023-12-27 05:54:31,234][20381] Updated weights for policy 1, policy_version 35151 (0.0008) [2023-12-27 05:54:31,289][20381] Updated weights for policy 1, policy_version 35161 (0.0008) [2023-12-27 05:54:31,350][20381] Updated weights for policy 1, policy_version 35171 (0.0009) [2023-12-27 05:54:31,813][20347] Updated weights for policy 0, policy_version 35249 (0.0007) [2023-12-27 05:54:31,861][20347] Updated weights for policy 0, policy_version 35259 (0.0007) [2023-12-27 05:54:31,907][20347] Updated weights for policy 0, policy_version 35269 (0.0006) [2023-12-27 05:54:31,951][20347] Updated weights for policy 0, policy_version 35279 (0.0005) [2023-12-27 05:54:32,090][20381] Updated weights for policy 1, policy_version 35181 (0.0008) [2023-12-27 05:54:32,146][20381] Updated weights for policy 1, policy_version 35191 (0.0008) [2023-12-27 05:54:32,210][20381] Updated weights for policy 1, policy_version 35201 (0.0008) [2023-12-27 05:54:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 18055168. Throughput: 0: 10421.7, 1: 10384.2. Samples: 18062836. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 05:54:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:32,585][20347] Updated weights for policy 0, policy_version 35289 (0.0007) [2023-12-27 05:54:32,640][20347] Updated weights for policy 0, policy_version 35299 (0.0008) [2023-12-27 05:54:32,696][20347] Updated weights for policy 0, policy_version 35309 (0.0008) [2023-12-27 05:54:32,807][20381] Updated weights for policy 1, policy_version 35211 (0.0005) [2023-12-27 05:54:32,860][20381] Updated weights for policy 1, policy_version 35221 (0.0005) [2023-12-27 05:54:32,913][20381] Updated weights for policy 1, policy_version 35231 (0.0005) [2023-12-27 05:54:33,373][20347] Updated weights for policy 0, policy_version 35319 (0.0006) [2023-12-27 05:54:33,413][20381] Updated weights for policy 1, policy_version 35241 (0.0005) [2023-12-27 05:54:33,433][20347] Updated weights for policy 0, policy_version 35329 (0.0006) [2023-12-27 05:54:33,476][20381] Updated weights for policy 1, policy_version 35251 (0.0005) [2023-12-27 05:54:33,489][20347] Updated weights for policy 0, policy_version 35339 (0.0006) [2023-12-27 05:54:33,535][20381] Updated weights for policy 1, policy_version 35261 (0.0005) [2023-12-27 05:54:33,590][20381] Updated weights for policy 1, policy_version 35271 (0.0005) [2023-12-27 05:54:34,095][20347] Updated weights for policy 0, policy_version 35349 (0.0006) [2023-12-27 05:54:34,145][20347] Updated weights for policy 0, policy_version 35359 (0.0008) [2023-12-27 05:54:34,199][20381] Updated weights for policy 1, policy_version 35281 (0.0006) [2023-12-27 05:54:34,201][20347] Updated weights for policy 0, policy_version 35369 (0.0007) [2023-12-27 05:54:34,255][20381] Updated weights for policy 1, policy_version 35291 (0.0007) [2023-12-27 05:54:34,312][20381] Updated weights for policy 1, policy_version 35301 (0.0005) [2023-12-27 05:54:34,908][20347] Updated weights for policy 0, policy_version 35379 (0.0006) [2023-12-27 05:54:34,935][20381] Updated weights for policy 1, policy_version 35311 (0.0006) [2023-12-27 05:54:34,967][20347] Updated weights for policy 0, policy_version 35389 (0.0007) [2023-12-27 05:54:34,991][20381] Updated weights for policy 1, policy_version 35321 (0.0008) [2023-12-27 05:54:35,030][20347] Updated weights for policy 0, policy_version 35399 (0.0008) [2023-12-27 05:54:35,051][20381] Updated weights for policy 1, policy_version 35331 (0.0008) [2023-12-27 05:54:35,722][20347] Updated weights for policy 0, policy_version 35409 (0.0006) [2023-12-27 05:54:35,728][20381] Updated weights for policy 1, policy_version 35341 (0.0006) [2023-12-27 05:54:35,772][20347] Updated weights for policy 0, policy_version 35419 (0.0008) [2023-12-27 05:54:35,778][20381] Updated weights for policy 1, policy_version 35351 (0.0007) [2023-12-27 05:54:35,821][20347] Updated weights for policy 0, policy_version 35429 (0.0008) [2023-12-27 05:54:35,829][20381] Updated weights for policy 1, policy_version 35361 (0.0005) [2023-12-27 05:54:35,871][20347] Updated weights for policy 0, policy_version 35439 (0.0007) [2023-12-27 05:54:36,462][20381] Updated weights for policy 1, policy_version 35371 (0.0005) [2023-12-27 05:54:36,521][20381] Updated weights for policy 1, policy_version 35381 (0.0006) [2023-12-27 05:54:36,581][20381] Updated weights for policy 1, policy_version 35391 (0.0006) [2023-12-27 05:54:36,650][20347] Updated weights for policy 0, policy_version 35449 (0.0007) [2023-12-27 05:54:36,702][20347] Updated weights for policy 0, policy_version 35459 (0.0007) [2023-12-27 05:54:36,754][20347] Updated weights for policy 0, policy_version 35469 (0.0008) [2023-12-27 05:54:37,162][20381] Updated weights for policy 1, policy_version 35401 (0.0006) [2023-12-27 05:54:37,227][20381] Updated weights for policy 1, policy_version 35411 (0.0009) [2023-12-27 05:54:37,278][20381] Updated weights for policy 1, policy_version 35421 (0.0006) [2023-12-27 05:54:37,329][20381] Updated weights for policy 1, policy_version 35431 (0.0008) [2023-12-27 05:54:37,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.7, 300 sec: 20938.2). Total num frames: 18161664. Throughput: 0: 10445.5, 1: 10412.7. Samples: 18128944. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 05:54:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000035432_9076736.pth... [2023-12-27 05:54:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000035472_9084928.pth... [2023-12-27 05:54:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000034184_8757248.pth [2023-12-27 05:54:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000034256_8773632.pth [2023-12-27 05:54:37,511][20347] Updated weights for policy 0, policy_version 35479 (0.0008) [2023-12-27 05:54:37,570][20347] Updated weights for policy 0, policy_version 35489 (0.0008) [2023-12-27 05:54:37,622][20347] Updated weights for policy 0, policy_version 35499 (0.0008) [2023-12-27 05:54:37,926][20381] Updated weights for policy 1, policy_version 35441 (0.0007) [2023-12-27 05:54:37,976][20381] Updated weights for policy 1, policy_version 35451 (0.0006) [2023-12-27 05:54:38,032][20381] Updated weights for policy 1, policy_version 35461 (0.0008) [2023-12-27 05:54:38,268][20347] Updated weights for policy 0, policy_version 35509 (0.0007) [2023-12-27 05:54:38,328][20347] Updated weights for policy 0, policy_version 35519 (0.0006) [2023-12-27 05:54:38,387][20347] Updated weights for policy 0, policy_version 35529 (0.0006) [2023-12-27 05:54:38,768][20381] Updated weights for policy 1, policy_version 35471 (0.0008) [2023-12-27 05:54:38,827][20381] Updated weights for policy 1, policy_version 35481 (0.0007) [2023-12-27 05:54:38,888][20381] Updated weights for policy 1, policy_version 35491 (0.0007) [2023-12-27 05:54:39,001][20347] Updated weights for policy 0, policy_version 35539 (0.0006) [2023-12-27 05:54:39,065][20347] Updated weights for policy 0, policy_version 35549 (0.0008) [2023-12-27 05:54:39,122][20347] Updated weights for policy 0, policy_version 35559 (0.0006) [2023-12-27 05:54:39,660][20381] Updated weights for policy 1, policy_version 35501 (0.0007) [2023-12-27 05:54:39,717][20381] Updated weights for policy 1, policy_version 35511 (0.0008) [2023-12-27 05:54:39,776][20381] Updated weights for policy 1, policy_version 35521 (0.0007) [2023-12-27 05:54:39,931][20347] Updated weights for policy 0, policy_version 35569 (0.0006) [2023-12-27 05:54:39,988][20347] Updated weights for policy 0, policy_version 35579 (0.0009) [2023-12-27 05:54:40,044][20347] Updated weights for policy 0, policy_version 35589 (0.0008) [2023-12-27 05:54:40,110][20347] Updated weights for policy 0, policy_version 35599 (0.0008) [2023-12-27 05:54:40,477][20381] Updated weights for policy 1, policy_version 35531 (0.0007) [2023-12-27 05:54:40,526][20381] Updated weights for policy 1, policy_version 35541 (0.0008) [2023-12-27 05:54:40,580][20381] Updated weights for policy 1, policy_version 35551 (0.0007) [2023-12-27 05:54:40,773][20347] Updated weights for policy 0, policy_version 35609 (0.0005) [2023-12-27 05:54:40,833][20347] Updated weights for policy 0, policy_version 35619 (0.0005) [2023-12-27 05:54:40,894][20347] Updated weights for policy 0, policy_version 35629 (0.0007) [2023-12-27 05:54:41,270][20381] Updated weights for policy 1, policy_version 35561 (0.0007) [2023-12-27 05:54:41,327][20381] Updated weights for policy 1, policy_version 35571 (0.0008) [2023-12-27 05:54:41,391][20381] Updated weights for policy 1, policy_version 35581 (0.0008) [2023-12-27 05:54:41,456][20381] Updated weights for policy 1, policy_version 35591 (0.0009) [2023-12-27 05:54:41,569][20347] Updated weights for policy 0, policy_version 35639 (0.0008) [2023-12-27 05:54:41,630][20347] Updated weights for policy 0, policy_version 35649 (0.0007) [2023-12-27 05:54:41,696][20347] Updated weights for policy 0, policy_version 35659 (0.0009) [2023-12-27 05:54:42,173][20381] Updated weights for policy 1, policy_version 35601 (0.0006) [2023-12-27 05:54:42,228][20381] Updated weights for policy 1, policy_version 35611 (0.0006) [2023-12-27 05:54:42,288][20381] Updated weights for policy 1, policy_version 35621 (0.0008) [2023-12-27 05:54:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 18259968. Throughput: 0: 10375.8, 1: 10389.7. Samples: 18250376. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 05:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:42,476][20347] Updated weights for policy 0, policy_version 35669 (0.0008) [2023-12-27 05:54:42,538][20347] Updated weights for policy 0, policy_version 35679 (0.0008) [2023-12-27 05:54:42,599][20347] Updated weights for policy 0, policy_version 35689 (0.0010) [2023-12-27 05:54:42,930][20381] Updated weights for policy 1, policy_version 35631 (0.0007) [2023-12-27 05:54:42,975][20381] Updated weights for policy 1, policy_version 35641 (0.0007) [2023-12-27 05:54:43,023][20381] Updated weights for policy 1, policy_version 35651 (0.0007) [2023-12-27 05:54:43,252][20347] Updated weights for policy 0, policy_version 35699 (0.0009) [2023-12-27 05:54:43,307][20347] Updated weights for policy 0, policy_version 35709 (0.0005) [2023-12-27 05:54:43,364][20347] Updated weights for policy 0, policy_version 35719 (0.0005) [2023-12-27 05:54:43,726][20381] Updated weights for policy 1, policy_version 35661 (0.0007) [2023-12-27 05:54:43,777][20381] Updated weights for policy 1, policy_version 35671 (0.0005) [2023-12-27 05:54:43,834][20381] Updated weights for policy 1, policy_version 35681 (0.0005) [2023-12-27 05:54:43,983][20347] Updated weights for policy 0, policy_version 35729 (0.0006) [2023-12-27 05:54:44,044][20347] Updated weights for policy 0, policy_version 35739 (0.0010) [2023-12-27 05:54:44,109][20347] Updated weights for policy 0, policy_version 35749 (0.0011) [2023-12-27 05:54:44,162][20347] Updated weights for policy 0, policy_version 35759 (0.0010) [2023-12-27 05:54:44,417][20381] Updated weights for policy 1, policy_version 35691 (0.0005) [2023-12-27 05:54:44,473][20381] Updated weights for policy 1, policy_version 35701 (0.0007) [2023-12-27 05:54:44,520][20381] Updated weights for policy 1, policy_version 35711 (0.0007) [2023-12-27 05:54:44,851][20347] Updated weights for policy 0, policy_version 35769 (0.0006) [2023-12-27 05:54:44,911][20347] Updated weights for policy 0, policy_version 35779 (0.0011) [2023-12-27 05:54:44,973][20347] Updated weights for policy 0, policy_version 35789 (0.0010) [2023-12-27 05:54:45,187][20381] Updated weights for policy 1, policy_version 35721 (0.0007) [2023-12-27 05:54:45,243][20381] Updated weights for policy 1, policy_version 35731 (0.0005) [2023-12-27 05:54:45,295][20381] Updated weights for policy 1, policy_version 35741 (0.0005) [2023-12-27 05:54:45,353][20381] Updated weights for policy 1, policy_version 35751 (0.0005) [2023-12-27 05:54:45,647][20347] Updated weights for policy 0, policy_version 35799 (0.0010) [2023-12-27 05:54:45,707][20347] Updated weights for policy 0, policy_version 35809 (0.0010) [2023-12-27 05:54:45,762][20347] Updated weights for policy 0, policy_version 35819 (0.0010) [2023-12-27 05:54:45,984][20381] Updated weights for policy 1, policy_version 35761 (0.0008) [2023-12-27 05:54:46,032][20381] Updated weights for policy 1, policy_version 35771 (0.0007) [2023-12-27 05:54:46,086][20381] Updated weights for policy 1, policy_version 35781 (0.0008) [2023-12-27 05:54:46,376][20347] Updated weights for policy 0, policy_version 35829 (0.0009) [2023-12-27 05:54:46,432][20347] Updated weights for policy 0, policy_version 35839 (0.0008) [2023-12-27 05:54:46,487][20347] Updated weights for policy 0, policy_version 35849 (0.0008) [2023-12-27 05:54:46,839][20381] Updated weights for policy 1, policy_version 35791 (0.0009) [2023-12-27 05:54:46,884][20381] Updated weights for policy 1, policy_version 35801 (0.0008) [2023-12-27 05:54:46,943][20381] Updated weights for policy 1, policy_version 35811 (0.0007) [2023-12-27 05:54:47,053][20347] Updated weights for policy 0, policy_version 35859 (0.0008) [2023-12-27 05:54:47,111][20347] Updated weights for policy 0, policy_version 35869 (0.0010) [2023-12-27 05:54:47,168][20347] Updated weights for policy 0, policy_version 35879 (0.0010) [2023-12-27 05:54:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 18366464. Throughput: 0: 10321.3, 1: 10405.6. Samples: 18374800. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 05:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:47,680][20381] Updated weights for policy 1, policy_version 35821 (0.0007) [2023-12-27 05:54:47,738][20381] Updated weights for policy 1, policy_version 35831 (0.0006) [2023-12-27 05:54:47,796][20381] Updated weights for policy 1, policy_version 35841 (0.0005) [2023-12-27 05:54:47,827][20347] Updated weights for policy 0, policy_version 35889 (0.0007) [2023-12-27 05:54:47,876][20347] Updated weights for policy 0, policy_version 35899 (0.0005) [2023-12-27 05:54:47,943][20347] Updated weights for policy 0, policy_version 35909 (0.0009) [2023-12-27 05:54:48,000][20347] Updated weights for policy 0, policy_version 35919 (0.0010) [2023-12-27 05:54:48,317][20381] Updated weights for policy 1, policy_version 35851 (0.0005) [2023-12-27 05:54:48,383][20381] Updated weights for policy 1, policy_version 35861 (0.0007) [2023-12-27 05:54:48,445][20381] Updated weights for policy 1, policy_version 35871 (0.0006) [2023-12-27 05:54:48,589][20347] Updated weights for policy 0, policy_version 35929 (0.0010) [2023-12-27 05:54:48,638][20347] Updated weights for policy 0, policy_version 35939 (0.0010) [2023-12-27 05:54:48,684][20347] Updated weights for policy 0, policy_version 35949 (0.0010) [2023-12-27 05:54:49,095][20381] Updated weights for policy 1, policy_version 35881 (0.0007) [2023-12-27 05:54:49,155][20381] Updated weights for policy 1, policy_version 35891 (0.0009) [2023-12-27 05:54:49,209][20381] Updated weights for policy 1, policy_version 35901 (0.0009) [2023-12-27 05:54:49,269][20381] Updated weights for policy 1, policy_version 35911 (0.0008) [2023-12-27 05:54:49,351][20347] Updated weights for policy 0, policy_version 35959 (0.0009) [2023-12-27 05:54:49,405][20347] Updated weights for policy 0, policy_version 35969 (0.0008) [2023-12-27 05:54:49,466][20347] Updated weights for policy 0, policy_version 35979 (0.0008) [2023-12-27 05:54:49,941][20381] Updated weights for policy 1, policy_version 35921 (0.0006) [2023-12-27 05:54:49,998][20381] Updated weights for policy 1, policy_version 35931 (0.0005) [2023-12-27 05:54:50,050][20381] Updated weights for policy 1, policy_version 35941 (0.0006) [2023-12-27 05:54:50,120][20347] Updated weights for policy 0, policy_version 35989 (0.0009) [2023-12-27 05:54:50,169][20347] Updated weights for policy 0, policy_version 35999 (0.0010) [2023-12-27 05:54:50,231][20347] Updated weights for policy 0, policy_version 36009 (0.0010) [2023-12-27 05:54:50,614][20381] Updated weights for policy 1, policy_version 35951 (0.0006) [2023-12-27 05:54:50,673][20381] Updated weights for policy 1, policy_version 35961 (0.0009) [2023-12-27 05:54:50,720][20381] Updated weights for policy 1, policy_version 35971 (0.0010) [2023-12-27 05:54:50,937][20347] Updated weights for policy 0, policy_version 36019 (0.0010) [2023-12-27 05:54:50,985][20347] Updated weights for policy 0, policy_version 36029 (0.0008) [2023-12-27 05:54:51,045][20347] Updated weights for policy 0, policy_version 36039 (0.0008) [2023-12-27 05:54:51,465][20381] Updated weights for policy 1, policy_version 35981 (0.0008) [2023-12-27 05:54:51,513][20381] Updated weights for policy 1, policy_version 35991 (0.0005) [2023-12-27 05:54:51,570][20381] Updated weights for policy 1, policy_version 36001 (0.0005) [2023-12-27 05:54:51,789][20347] Updated weights for policy 0, policy_version 36049 (0.0010) [2023-12-27 05:54:51,849][20347] Updated weights for policy 0, policy_version 36059 (0.0006) [2023-12-27 05:54:51,908][20347] Updated weights for policy 0, policy_version 36069 (0.0007) [2023-12-27 05:54:51,963][20347] Updated weights for policy 0, policy_version 36079 (0.0008) [2023-12-27 05:54:52,280][20381] Updated weights for policy 1, policy_version 36011 (0.0007) [2023-12-27 05:54:52,343][20381] Updated weights for policy 1, policy_version 36021 (0.0008) [2023-12-27 05:54:52,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20753.0, 300 sec: 20882.6). Total num frames: 18464768. Throughput: 0: 10363.5, 1: 10421.5. Samples: 18439292. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 05:54:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000036080_9240576.pth... [2023-12-27 05:54:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000034864_8929280.pth [2023-12-27 05:54:52,407][20381] Updated weights for policy 1, policy_version 36031 (0.0008) [2023-12-27 05:54:52,450][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000036040_9232384.pth... [2023-12-27 05:54:52,453][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000034792_8912896.pth [2023-12-27 05:54:52,564][20347] Updated weights for policy 0, policy_version 36089 (0.0008) [2023-12-27 05:54:52,623][20347] Updated weights for policy 0, policy_version 36099 (0.0008) [2023-12-27 05:54:52,676][20347] Updated weights for policy 0, policy_version 36109 (0.0007) [2023-12-27 05:54:53,114][20381] Updated weights for policy 1, policy_version 36041 (0.0008) [2023-12-27 05:54:53,172][20381] Updated weights for policy 1, policy_version 36051 (0.0008) [2023-12-27 05:54:53,224][20381] Updated weights for policy 1, policy_version 36061 (0.0007) [2023-12-27 05:54:53,276][20381] Updated weights for policy 1, policy_version 36071 (0.0008) [2023-12-27 05:54:53,316][20347] Updated weights for policy 0, policy_version 36119 (0.0007) [2023-12-27 05:54:53,360][20347] Updated weights for policy 0, policy_version 36129 (0.0008) [2023-12-27 05:54:53,410][20347] Updated weights for policy 0, policy_version 36139 (0.0008) [2023-12-27 05:54:53,933][20381] Updated weights for policy 1, policy_version 36081 (0.0007) [2023-12-27 05:54:53,985][20381] Updated weights for policy 1, policy_version 36091 (0.0008) [2023-12-27 05:54:54,030][20381] Updated weights for policy 1, policy_version 36101 (0.0007) [2023-12-27 05:54:54,142][20347] Updated weights for policy 0, policy_version 36149 (0.0009) [2023-12-27 05:54:54,206][20347] Updated weights for policy 0, policy_version 36159 (0.0010) [2023-12-27 05:54:54,273][20347] Updated weights for policy 0, policy_version 36169 (0.0010) [2023-12-27 05:54:54,696][20381] Updated weights for policy 1, policy_version 36111 (0.0006) [2023-12-27 05:54:54,754][20381] Updated weights for policy 1, policy_version 36121 (0.0005) [2023-12-27 05:54:54,802][20381] Updated weights for policy 1, policy_version 36131 (0.0005) [2023-12-27 05:54:55,012][20347] Updated weights for policy 0, policy_version 36179 (0.0010) [2023-12-27 05:54:55,074][20347] Updated weights for policy 0, policy_version 36189 (0.0010) [2023-12-27 05:54:55,126][20347] Updated weights for policy 0, policy_version 36199 (0.0010) [2023-12-27 05:54:55,414][20381] Updated weights for policy 1, policy_version 36141 (0.0007) [2023-12-27 05:54:55,475][20381] Updated weights for policy 1, policy_version 36151 (0.0007) [2023-12-27 05:54:55,536][20381] Updated weights for policy 1, policy_version 36161 (0.0007) [2023-12-27 05:54:55,810][20347] Updated weights for policy 0, policy_version 36209 (0.0010) [2023-12-27 05:54:55,867][20347] Updated weights for policy 0, policy_version 36219 (0.0010) [2023-12-27 05:54:55,921][20347] Updated weights for policy 0, policy_version 36229 (0.0010) [2023-12-27 05:54:55,979][20347] Updated weights for policy 0, policy_version 36239 (0.0010) [2023-12-27 05:54:56,137][20381] Updated weights for policy 1, policy_version 36171 (0.0006) [2023-12-27 05:54:56,190][20381] Updated weights for policy 1, policy_version 36181 (0.0009) [2023-12-27 05:54:56,237][20381] Updated weights for policy 1, policy_version 36191 (0.0009) [2023-12-27 05:54:56,646][20347] Updated weights for policy 0, policy_version 36249 (0.0010) [2023-12-27 05:54:56,690][20347] Updated weights for policy 0, policy_version 36259 (0.0010) [2023-12-27 05:54:56,744][20347] Updated weights for policy 0, policy_version 36269 (0.0010) [2023-12-27 05:54:56,795][20381] Updated weights for policy 1, policy_version 36201 (0.0005) [2023-12-27 05:54:56,846][20381] Updated weights for policy 1, policy_version 36211 (0.0005) [2023-12-27 05:54:56,891][20381] Updated weights for policy 1, policy_version 36221 (0.0006) [2023-12-27 05:54:56,936][20381] Updated weights for policy 1, policy_version 36231 (0.0006) [2023-12-27 05:54:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20882.6). Total num frames: 18571264. Throughput: 0: 10391.4, 1: 10446.9. Samples: 18563280. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 05:54:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:54:57,420][20347] Updated weights for policy 0, policy_version 36279 (0.0007) [2023-12-27 05:54:57,469][20347] Updated weights for policy 0, policy_version 36289 (0.0006) [2023-12-27 05:54:57,526][20347] Updated weights for policy 0, policy_version 36299 (0.0010) [2023-12-27 05:54:57,634][20381] Updated weights for policy 1, policy_version 36241 (0.0006) [2023-12-27 05:54:57,686][20381] Updated weights for policy 1, policy_version 36251 (0.0008) [2023-12-27 05:54:57,745][20381] Updated weights for policy 1, policy_version 36261 (0.0006) [2023-12-27 05:54:58,138][20347] Updated weights for policy 0, policy_version 36309 (0.0010) [2023-12-27 05:54:58,199][20347] Updated weights for policy 0, policy_version 36319 (0.0009) [2023-12-27 05:54:58,257][20347] Updated weights for policy 0, policy_version 36329 (0.0006) [2023-12-27 05:54:58,318][20381] Updated weights for policy 1, policy_version 36271 (0.0007) [2023-12-27 05:54:58,385][20381] Updated weights for policy 1, policy_version 36281 (0.0008) [2023-12-27 05:54:58,443][20381] Updated weights for policy 1, policy_version 36291 (0.0008) [2023-12-27 05:54:59,059][20347] Updated weights for policy 0, policy_version 36339 (0.0007) [2023-12-27 05:54:59,122][20347] Updated weights for policy 0, policy_version 36349 (0.0008) [2023-12-27 05:54:59,179][20347] Updated weights for policy 0, policy_version 36359 (0.0007) [2023-12-27 05:54:59,191][20381] Updated weights for policy 1, policy_version 36301 (0.0008) [2023-12-27 05:54:59,259][20381] Updated weights for policy 1, policy_version 36311 (0.0008) [2023-12-27 05:54:59,324][20381] Updated weights for policy 1, policy_version 36321 (0.0008) [2023-12-27 05:54:59,805][20347] Updated weights for policy 0, policy_version 36369 (0.0007) [2023-12-27 05:54:59,868][20347] Updated weights for policy 0, policy_version 36379 (0.0008) [2023-12-27 05:54:59,924][20347] Updated weights for policy 0, policy_version 36389 (0.0008) [2023-12-27 05:54:59,982][20347] Updated weights for policy 0, policy_version 36399 (0.0007) [2023-12-27 05:55:00,021][20381] Updated weights for policy 1, policy_version 36331 (0.0007) [2023-12-27 05:55:00,076][20381] Updated weights for policy 1, policy_version 36341 (0.0009) [2023-12-27 05:55:00,136][20381] Updated weights for policy 1, policy_version 36351 (0.0010) [2023-12-27 05:55:00,626][20347] Updated weights for policy 0, policy_version 36409 (0.0006) [2023-12-27 05:55:00,685][20347] Updated weights for policy 0, policy_version 36419 (0.0007) [2023-12-27 05:55:00,709][20381] Updated weights for policy 1, policy_version 36361 (0.0010) [2023-12-27 05:55:00,745][20347] Updated weights for policy 0, policy_version 36429 (0.0006) [2023-12-27 05:55:00,756][20381] Updated weights for policy 1, policy_version 36371 (0.0010) [2023-12-27 05:55:00,812][20381] Updated weights for policy 1, policy_version 36381 (0.0009) [2023-12-27 05:55:00,867][20381] Updated weights for policy 1, policy_version 36391 (0.0005) [2023-12-27 05:55:01,364][20347] Updated weights for policy 0, policy_version 36439 (0.0006) [2023-12-27 05:55:01,430][20347] Updated weights for policy 0, policy_version 36449 (0.0008) [2023-12-27 05:55:01,489][20347] Updated weights for policy 0, policy_version 36459 (0.0007) [2023-12-27 05:55:01,545][20381] Updated weights for policy 1, policy_version 36401 (0.0010) [2023-12-27 05:55:01,604][20381] Updated weights for policy 1, policy_version 36411 (0.0011) [2023-12-27 05:55:01,666][20381] Updated weights for policy 1, policy_version 36421 (0.0009) [2023-12-27 05:55:02,204][20347] Updated weights for policy 0, policy_version 36469 (0.0007) [2023-12-27 05:55:02,260][20347] Updated weights for policy 0, policy_version 36479 (0.0007) [2023-12-27 05:55:02,325][20347] Updated weights for policy 0, policy_version 36489 (0.0007) [2023-12-27 05:55:02,328][20381] Updated weights for policy 1, policy_version 36431 (0.0008) [2023-12-27 05:55:02,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20882.7). Total num frames: 18669568. Throughput: 0: 10381.0, 1: 10485.0. Samples: 18689060. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 05:55:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:02,391][20381] Updated weights for policy 1, policy_version 36441 (0.0007) [2023-12-27 05:55:02,453][20381] Updated weights for policy 1, policy_version 36451 (0.0007) [2023-12-27 05:55:02,926][20347] Updated weights for policy 0, policy_version 36499 (0.0006) [2023-12-27 05:55:02,971][20347] Updated weights for policy 0, policy_version 36509 (0.0005) [2023-12-27 05:55:03,032][20347] Updated weights for policy 0, policy_version 36519 (0.0008) [2023-12-27 05:55:03,100][20381] Updated weights for policy 1, policy_version 36461 (0.0010) [2023-12-27 05:55:03,143][20381] Updated weights for policy 1, policy_version 36471 (0.0010) [2023-12-27 05:55:03,194][20381] Updated weights for policy 1, policy_version 36481 (0.0010) [2023-12-27 05:55:03,677][20347] Updated weights for policy 0, policy_version 36529 (0.0007) [2023-12-27 05:55:03,738][20347] Updated weights for policy 0, policy_version 36539 (0.0005) [2023-12-27 05:55:03,756][20381] Updated weights for policy 1, policy_version 36491 (0.0008) [2023-12-27 05:55:03,799][20347] Updated weights for policy 0, policy_version 36549 (0.0005) [2023-12-27 05:55:03,811][20381] Updated weights for policy 1, policy_version 36501 (0.0005) [2023-12-27 05:55:03,862][20347] Updated weights for policy 0, policy_version 36559 (0.0007) [2023-12-27 05:55:03,868][20381] Updated weights for policy 1, policy_version 36511 (0.0006) [2023-12-27 05:55:04,501][20347] Updated weights for policy 0, policy_version 36569 (0.0006) [2023-12-27 05:55:04,555][20347] Updated weights for policy 0, policy_version 36579 (0.0008) [2023-12-27 05:55:04,556][20381] Updated weights for policy 1, policy_version 36521 (0.0007) [2023-12-27 05:55:04,605][20347] Updated weights for policy 0, policy_version 36589 (0.0007) [2023-12-27 05:55:04,609][20381] Updated weights for policy 1, policy_version 36531 (0.0009) [2023-12-27 05:55:04,658][20381] Updated weights for policy 1, policy_version 36541 (0.0010) [2023-12-27 05:55:04,703][20381] Updated weights for policy 1, policy_version 36551 (0.0010) [2023-12-27 05:55:05,233][20347] Updated weights for policy 0, policy_version 36599 (0.0007) [2023-12-27 05:55:05,296][20347] Updated weights for policy 0, policy_version 36609 (0.0008) [2023-12-27 05:55:05,351][20347] Updated weights for policy 0, policy_version 36619 (0.0007) [2023-12-27 05:55:05,396][20381] Updated weights for policy 1, policy_version 36561 (0.0007) [2023-12-27 05:55:05,451][20381] Updated weights for policy 1, policy_version 36571 (0.0008) [2023-12-27 05:55:05,510][20381] Updated weights for policy 1, policy_version 36581 (0.0007) [2023-12-27 05:55:05,927][20347] Updated weights for policy 0, policy_version 36629 (0.0006) [2023-12-27 05:55:05,992][20347] Updated weights for policy 0, policy_version 36639 (0.0006) [2023-12-27 05:55:06,047][20347] Updated weights for policy 0, policy_version 36649 (0.0005) [2023-12-27 05:55:06,254][20381] Updated weights for policy 1, policy_version 36591 (0.0008) [2023-12-27 05:55:06,312][20381] Updated weights for policy 1, policy_version 36601 (0.0009) [2023-12-27 05:55:06,374][20381] Updated weights for policy 1, policy_version 36611 (0.0008) [2023-12-27 05:55:06,669][20347] Updated weights for policy 0, policy_version 36659 (0.0006) [2023-12-27 05:55:06,725][20347] Updated weights for policy 0, policy_version 36669 (0.0007) [2023-12-27 05:55:06,777][20347] Updated weights for policy 0, policy_version 36679 (0.0006) [2023-12-27 05:55:07,069][20381] Updated weights for policy 1, policy_version 36621 (0.0007) [2023-12-27 05:55:07,114][20381] Updated weights for policy 1, policy_version 36631 (0.0006) [2023-12-27 05:55:07,163][20381] Updated weights for policy 1, policy_version 36641 (0.0006) [2023-12-27 05:55:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 18784256. Throughput: 0: 10406.6, 1: 10518.2. Samples: 18753180. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 05:55:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000036688_9396224.pth... [2023-12-27 05:55:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000036648_9388032.pth... [2023-12-27 05:55:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000035472_9084928.pth [2023-12-27 05:55:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000035432_9076736.pth [2023-12-27 05:55:07,490][20347] Updated weights for policy 0, policy_version 36689 (0.0007) [2023-12-27 05:55:07,536][20347] Updated weights for policy 0, policy_version 36699 (0.0007) [2023-12-27 05:55:07,585][20347] Updated weights for policy 0, policy_version 36709 (0.0007) [2023-12-27 05:55:07,641][20347] Updated weights for policy 0, policy_version 36719 (0.0008) [2023-12-27 05:55:07,800][20381] Updated weights for policy 1, policy_version 36651 (0.0006) [2023-12-27 05:55:07,851][20381] Updated weights for policy 1, policy_version 36661 (0.0010) [2023-12-27 05:55:07,911][20381] Updated weights for policy 1, policy_version 36671 (0.0010) [2023-12-27 05:55:08,356][20347] Updated weights for policy 0, policy_version 36729 (0.0008) [2023-12-27 05:55:08,406][20347] Updated weights for policy 0, policy_version 36739 (0.0007) [2023-12-27 05:55:08,468][20347] Updated weights for policy 0, policy_version 36749 (0.0008) [2023-12-27 05:55:08,549][20381] Updated weights for policy 1, policy_version 36681 (0.0010) [2023-12-27 05:55:08,605][20381] Updated weights for policy 1, policy_version 36691 (0.0010) [2023-12-27 05:55:08,664][20381] Updated weights for policy 1, policy_version 36701 (0.0010) [2023-12-27 05:55:08,721][20381] Updated weights for policy 1, policy_version 36711 (0.0009) [2023-12-27 05:55:09,151][20347] Updated weights for policy 0, policy_version 36759 (0.0006) [2023-12-27 05:55:09,214][20347] Updated weights for policy 0, policy_version 36769 (0.0006) [2023-12-27 05:55:09,273][20347] Updated weights for policy 0, policy_version 36779 (0.0008) [2023-12-27 05:55:09,357][20381] Updated weights for policy 1, policy_version 36721 (0.0011) [2023-12-27 05:55:09,422][20381] Updated weights for policy 1, policy_version 36731 (0.0010) [2023-12-27 05:55:09,481][20381] Updated weights for policy 1, policy_version 36741 (0.0011) [2023-12-27 05:55:09,956][20347] Updated weights for policy 0, policy_version 36789 (0.0008) [2023-12-27 05:55:10,011][20347] Updated weights for policy 0, policy_version 36799 (0.0008) [2023-12-27 05:55:10,073][20347] Updated weights for policy 0, policy_version 36809 (0.0008) [2023-12-27 05:55:10,178][20381] Updated weights for policy 1, policy_version 36751 (0.0011) [2023-12-27 05:55:10,238][20381] Updated weights for policy 1, policy_version 36761 (0.0011) [2023-12-27 05:55:10,296][20381] Updated weights for policy 1, policy_version 36771 (0.0011) [2023-12-27 05:55:10,805][20347] Updated weights for policy 0, policy_version 36819 (0.0008) [2023-12-27 05:55:10,867][20347] Updated weights for policy 0, policy_version 36829 (0.0010) [2023-12-27 05:55:10,929][20347] Updated weights for policy 0, policy_version 36839 (0.0009) [2023-12-27 05:55:11,016][20381] Updated weights for policy 1, policy_version 36781 (0.0008) [2023-12-27 05:55:11,082][20381] Updated weights for policy 1, policy_version 36791 (0.0008) [2023-12-27 05:55:11,148][20381] Updated weights for policy 1, policy_version 36801 (0.0008) [2023-12-27 05:55:11,673][20347] Updated weights for policy 0, policy_version 36849 (0.0008) [2023-12-27 05:55:11,739][20347] Updated weights for policy 0, policy_version 36859 (0.0007) [2023-12-27 05:55:11,797][20347] Updated weights for policy 0, policy_version 36869 (0.0007) [2023-12-27 05:55:11,838][20381] Updated weights for policy 1, policy_version 36811 (0.0007) [2023-12-27 05:55:11,862][20347] Updated weights for policy 0, policy_version 36879 (0.0007) [2023-12-27 05:55:11,896][20381] Updated weights for policy 1, policy_version 36821 (0.0006) [2023-12-27 05:55:11,957][20381] Updated weights for policy 1, policy_version 36831 (0.0008) [2023-12-27 05:55:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 18882560. Throughput: 0: 10402.1, 1: 10537.2. Samples: 18875168. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-12-27 05:55:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:12,554][20347] Updated weights for policy 0, policy_version 36889 (0.0006) [2023-12-27 05:55:12,613][20347] Updated weights for policy 0, policy_version 36899 (0.0005) [2023-12-27 05:55:12,671][20347] Updated weights for policy 0, policy_version 36909 (0.0005) [2023-12-27 05:55:12,713][20381] Updated weights for policy 1, policy_version 36841 (0.0008) [2023-12-27 05:55:12,765][20381] Updated weights for policy 1, policy_version 36851 (0.0008) [2023-12-27 05:55:12,823][20381] Updated weights for policy 1, policy_version 36861 (0.0005) [2023-12-27 05:55:12,884][20381] Updated weights for policy 1, policy_version 36871 (0.0005) [2023-12-27 05:55:13,297][20347] Updated weights for policy 0, policy_version 36919 (0.0008) [2023-12-27 05:55:13,348][20347] Updated weights for policy 0, policy_version 36929 (0.0007) [2023-12-27 05:55:13,394][20347] Updated weights for policy 0, policy_version 36939 (0.0007) [2023-12-27 05:55:13,430][20381] Updated weights for policy 1, policy_version 36881 (0.0007) [2023-12-27 05:55:13,484][20381] Updated weights for policy 1, policy_version 36891 (0.0007) [2023-12-27 05:55:13,536][20381] Updated weights for policy 1, policy_version 36901 (0.0008) [2023-12-27 05:55:14,097][20347] Updated weights for policy 0, policy_version 36949 (0.0006) [2023-12-27 05:55:14,160][20347] Updated weights for policy 0, policy_version 36959 (0.0009) [2023-12-27 05:55:14,173][20381] Updated weights for policy 1, policy_version 36911 (0.0006) [2023-12-27 05:55:14,215][20347] Updated weights for policy 0, policy_version 36969 (0.0010) [2023-12-27 05:55:14,219][20381] Updated weights for policy 1, policy_version 36921 (0.0007) [2023-12-27 05:55:14,268][20381] Updated weights for policy 1, policy_version 36931 (0.0007) [2023-12-27 05:55:14,842][20347] Updated weights for policy 0, policy_version 36979 (0.0010) [2023-12-27 05:55:14,907][20347] Updated weights for policy 0, policy_version 36989 (0.0007) [2023-12-27 05:55:14,973][20347] Updated weights for policy 0, policy_version 36999 (0.0008) [2023-12-27 05:55:15,017][20381] Updated weights for policy 1, policy_version 36941 (0.0009) [2023-12-27 05:55:15,073][20381] Updated weights for policy 1, policy_version 36951 (0.0007) [2023-12-27 05:55:15,131][20381] Updated weights for policy 1, policy_version 36961 (0.0008) [2023-12-27 05:55:15,538][20347] Updated weights for policy 0, policy_version 37009 (0.0008) [2023-12-27 05:55:15,599][20347] Updated weights for policy 0, policy_version 37019 (0.0008) [2023-12-27 05:55:15,656][20347] Updated weights for policy 0, policy_version 37029 (0.0005) [2023-12-27 05:55:15,717][20347] Updated weights for policy 0, policy_version 37039 (0.0008) [2023-12-27 05:55:15,861][20381] Updated weights for policy 1, policy_version 36971 (0.0007) [2023-12-27 05:55:15,922][20381] Updated weights for policy 1, policy_version 36981 (0.0009) [2023-12-27 05:55:15,982][20381] Updated weights for policy 1, policy_version 36991 (0.0011) [2023-12-27 05:55:16,363][20347] Updated weights for policy 0, policy_version 37049 (0.0009) [2023-12-27 05:55:16,419][20347] Updated weights for policy 0, policy_version 37059 (0.0005) [2023-12-27 05:55:16,481][20347] Updated weights for policy 0, policy_version 37069 (0.0005) [2023-12-27 05:55:16,617][20381] Updated weights for policy 1, policy_version 37001 (0.0010) [2023-12-27 05:55:16,677][20381] Updated weights for policy 1, policy_version 37011 (0.0006) [2023-12-27 05:55:16,726][20381] Updated weights for policy 1, policy_version 37021 (0.0010) [2023-12-27 05:55:16,774][20381] Updated weights for policy 1, policy_version 37031 (0.0008) [2023-12-27 05:55:17,076][20347] Updated weights for policy 0, policy_version 37079 (0.0009) [2023-12-27 05:55:17,123][20347] Updated weights for policy 0, policy_version 37089 (0.0010) [2023-12-27 05:55:17,174][20347] Updated weights for policy 0, policy_version 37099 (0.0010) [2023-12-27 05:55:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 18989056. Throughput: 0: 10336.3, 1: 10498.0. Samples: 19000380. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-12-27 05:55:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:17,410][20381] Updated weights for policy 1, policy_version 37041 (0.0008) [2023-12-27 05:55:17,464][20381] Updated weights for policy 1, policy_version 37051 (0.0007) [2023-12-27 05:55:17,514][20381] Updated weights for policy 1, policy_version 37061 (0.0007) [2023-12-27 05:55:17,868][20347] Updated weights for policy 0, policy_version 37109 (0.0010) [2023-12-27 05:55:17,922][20347] Updated weights for policy 0, policy_version 37119 (0.0010) [2023-12-27 05:55:17,978][20347] Updated weights for policy 0, policy_version 37129 (0.0008) [2023-12-27 05:55:18,200][20381] Updated weights for policy 1, policy_version 37071 (0.0006) [2023-12-27 05:55:18,251][20381] Updated weights for policy 1, policy_version 37081 (0.0005) [2023-12-27 05:55:18,305][20381] Updated weights for policy 1, policy_version 37091 (0.0005) [2023-12-27 05:55:18,645][20347] Updated weights for policy 0, policy_version 37139 (0.0007) [2023-12-27 05:55:18,693][20347] Updated weights for policy 0, policy_version 37149 (0.0008) [2023-12-27 05:55:18,745][20347] Updated weights for policy 0, policy_version 37159 (0.0008) [2023-12-27 05:55:18,951][20381] Updated weights for policy 1, policy_version 37101 (0.0007) [2023-12-27 05:55:18,998][20381] Updated weights for policy 1, policy_version 37111 (0.0007) [2023-12-27 05:55:19,049][20381] Updated weights for policy 1, policy_version 37121 (0.0007) [2023-12-27 05:55:19,400][20347] Updated weights for policy 0, policy_version 37169 (0.0007) [2023-12-27 05:55:19,455][20347] Updated weights for policy 0, policy_version 37179 (0.0006) [2023-12-27 05:55:19,514][20347] Updated weights for policy 0, policy_version 37189 (0.0007) [2023-12-27 05:55:19,572][20347] Updated weights for policy 0, policy_version 37199 (0.0008) [2023-12-27 05:55:19,783][20381] Updated weights for policy 1, policy_version 37131 (0.0008) [2023-12-27 05:55:19,839][20381] Updated weights for policy 1, policy_version 37141 (0.0008) [2023-12-27 05:55:19,896][20381] Updated weights for policy 1, policy_version 37151 (0.0007) [2023-12-27 05:55:20,275][20347] Updated weights for policy 0, policy_version 37209 (0.0010) [2023-12-27 05:55:20,335][20347] Updated weights for policy 0, policy_version 37219 (0.0010) [2023-12-27 05:55:20,393][20347] Updated weights for policy 0, policy_version 37229 (0.0010) [2023-12-27 05:55:20,549][20381] Updated weights for policy 1, policy_version 37161 (0.0008) [2023-12-27 05:55:20,611][20381] Updated weights for policy 1, policy_version 37171 (0.0007) [2023-12-27 05:55:20,675][20381] Updated weights for policy 1, policy_version 37181 (0.0006) [2023-12-27 05:55:20,736][20381] Updated weights for policy 1, policy_version 37191 (0.0005) [2023-12-27 05:55:21,086][20347] Updated weights for policy 0, policy_version 37239 (0.0009) [2023-12-27 05:55:21,155][20347] Updated weights for policy 0, policy_version 37249 (0.0008) [2023-12-27 05:55:21,214][20347] Updated weights for policy 0, policy_version 37259 (0.0008) [2023-12-27 05:55:21,350][20381] Updated weights for policy 1, policy_version 37201 (0.0007) [2023-12-27 05:55:21,414][20381] Updated weights for policy 1, policy_version 37211 (0.0008) [2023-12-27 05:55:21,474][20381] Updated weights for policy 1, policy_version 37221 (0.0007) [2023-12-27 05:55:21,931][20347] Updated weights for policy 0, policy_version 37269 (0.0008) [2023-12-27 05:55:21,996][20347] Updated weights for policy 0, policy_version 37279 (0.0008) [2023-12-27 05:55:22,052][20347] Updated weights for policy 0, policy_version 37289 (0.0007) [2023-12-27 05:55:22,156][20381] Updated weights for policy 1, policy_version 37231 (0.0010) [2023-12-27 05:55:22,202][20381] Updated weights for policy 1, policy_version 37241 (0.0010) [2023-12-27 05:55:22,261][20381] Updated weights for policy 1, policy_version 37251 (0.0011) [2023-12-27 05:55:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 19095552. Throughput: 0: 10317.6, 1: 10412.0. Samples: 19061776. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-12-27 05:55:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000037256_9543680.pth... [2023-12-27 05:55:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000037296_9551872.pth... [2023-12-27 05:55:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000036040_9232384.pth [2023-12-27 05:55:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000036080_9240576.pth [2023-12-27 05:55:22,757][20347] Updated weights for policy 0, policy_version 37299 (0.0008) [2023-12-27 05:55:22,805][20347] Updated weights for policy 0, policy_version 37309 (0.0007) [2023-12-27 05:55:22,860][20347] Updated weights for policy 0, policy_version 37319 (0.0008) [2023-12-27 05:55:22,938][20381] Updated weights for policy 1, policy_version 37261 (0.0010) [2023-12-27 05:55:22,996][20381] Updated weights for policy 1, policy_version 37271 (0.0006) [2023-12-27 05:55:23,057][20381] Updated weights for policy 1, policy_version 37281 (0.0006) [2023-12-27 05:55:23,480][20347] Updated weights for policy 0, policy_version 37329 (0.0007) [2023-12-27 05:55:23,537][20347] Updated weights for policy 0, policy_version 37339 (0.0007) [2023-12-27 05:55:23,580][20347] Updated weights for policy 0, policy_version 37349 (0.0007) [2023-12-27 05:55:23,625][20347] Updated weights for policy 0, policy_version 37359 (0.0007) [2023-12-27 05:55:23,676][20381] Updated weights for policy 1, policy_version 37291 (0.0008) [2023-12-27 05:55:23,723][20381] Updated weights for policy 1, policy_version 37301 (0.0010) [2023-12-27 05:55:23,776][20381] Updated weights for policy 1, policy_version 37311 (0.0008) [2023-12-27 05:55:24,229][20347] Updated weights for policy 0, policy_version 37369 (0.0008) [2023-12-27 05:55:24,280][20347] Updated weights for policy 0, policy_version 37379 (0.0010) [2023-12-27 05:55:24,330][20347] Updated weights for policy 0, policy_version 37389 (0.0005) [2023-12-27 05:55:24,454][20381] Updated weights for policy 1, policy_version 37321 (0.0009) [2023-12-27 05:55:24,513][20381] Updated weights for policy 1, policy_version 37331 (0.0008) [2023-12-27 05:55:24,559][20381] Updated weights for policy 1, policy_version 37341 (0.0008) [2023-12-27 05:55:24,609][20381] Updated weights for policy 1, policy_version 37351 (0.0008) [2023-12-27 05:55:24,996][20347] Updated weights for policy 0, policy_version 37399 (0.0009) [2023-12-27 05:55:25,062][20347] Updated weights for policy 0, policy_version 37409 (0.0010) [2023-12-27 05:55:25,124][20347] Updated weights for policy 0, policy_version 37419 (0.0010) [2023-12-27 05:55:25,306][20381] Updated weights for policy 1, policy_version 37361 (0.0006) [2023-12-27 05:55:25,351][20381] Updated weights for policy 1, policy_version 37371 (0.0005) [2023-12-27 05:55:25,407][20381] Updated weights for policy 1, policy_version 37381 (0.0005) [2023-12-27 05:55:25,776][20347] Updated weights for policy 0, policy_version 37429 (0.0010) [2023-12-27 05:55:25,833][20347] Updated weights for policy 0, policy_version 37440 (0.0010) [2023-12-27 05:55:25,890][20347] Updated weights for policy 0, policy_version 37450 (0.0009) [2023-12-27 05:55:25,997][20381] Updated weights for policy 1, policy_version 37391 (0.0006) [2023-12-27 05:55:26,041][20381] Updated weights for policy 1, policy_version 37401 (0.0005) [2023-12-27 05:55:26,094][20381] Updated weights for policy 1, policy_version 37411 (0.0005) [2023-12-27 05:55:26,554][20347] Updated weights for policy 0, policy_version 37460 (0.0010) [2023-12-27 05:55:26,619][20347] Updated weights for policy 0, policy_version 37470 (0.0008) [2023-12-27 05:55:26,658][20381] Updated weights for policy 1, policy_version 37421 (0.0008) [2023-12-27 05:55:26,672][20347] Updated weights for policy 0, policy_version 37480 (0.0010) [2023-12-27 05:55:26,706][20381] Updated weights for policy 1, policy_version 37431 (0.0010) [2023-12-27 05:55:26,756][20381] Updated weights for policy 1, policy_version 37441 (0.0010) [2023-12-27 05:55:27,314][20347] Updated weights for policy 0, policy_version 37490 (0.0009) [2023-12-27 05:55:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 19193856. Throughput: 0: 10376.7, 1: 10455.8. Samples: 19187840. Policy #0 lag: (min: 27.0, avg: 32.1, max: 59.0) [2023-12-27 05:55:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:27,366][20347] Updated weights for policy 0, policy_version 37500 (0.0008) [2023-12-27 05:55:27,380][20381] Updated weights for policy 1, policy_version 37451 (0.0010) [2023-12-27 05:55:27,422][20347] Updated weights for policy 0, policy_version 37510 (0.0007) [2023-12-27 05:55:27,428][20381] Updated weights for policy 1, policy_version 37461 (0.0010) [2023-12-27 05:55:27,470][20347] Updated weights for policy 0, policy_version 37520 (0.0005) [2023-12-27 05:55:27,474][20381] Updated weights for policy 1, policy_version 37471 (0.0009) [2023-12-27 05:55:28,034][20381] Updated weights for policy 1, policy_version 37481 (0.0005) [2023-12-27 05:55:28,098][20381] Updated weights for policy 1, policy_version 37491 (0.0005) [2023-12-27 05:55:28,123][20347] Updated weights for policy 0, policy_version 37530 (0.0006) [2023-12-27 05:55:28,151][20381] Updated weights for policy 1, policy_version 37501 (0.0006) [2023-12-27 05:55:28,179][20347] Updated weights for policy 0, policy_version 37540 (0.0006) [2023-12-27 05:55:28,208][20381] Updated weights for policy 1, policy_version 37511 (0.0005) [2023-12-27 05:55:28,238][20347] Updated weights for policy 0, policy_version 37550 (0.0007) [2023-12-27 05:55:28,844][20381] Updated weights for policy 1, policy_version 37521 (0.0007) [2023-12-27 05:55:28,895][20381] Updated weights for policy 1, policy_version 37531 (0.0008) [2023-12-27 05:55:28,915][20347] Updated weights for policy 0, policy_version 37560 (0.0007) [2023-12-27 05:55:28,948][20381] Updated weights for policy 1, policy_version 37541 (0.0006) [2023-12-27 05:55:28,968][20347] Updated weights for policy 0, policy_version 37570 (0.0006) [2023-12-27 05:55:29,023][20347] Updated weights for policy 0, policy_version 37580 (0.0008) [2023-12-27 05:55:29,620][20381] Updated weights for policy 1, policy_version 37551 (0.0005) [2023-12-27 05:55:29,673][20381] Updated weights for policy 1, policy_version 37561 (0.0005) [2023-12-27 05:55:29,729][20381] Updated weights for policy 1, policy_version 37571 (0.0006) [2023-12-27 05:55:29,752][20347] Updated weights for policy 0, policy_version 37590 (0.0007) [2023-12-27 05:55:29,805][20347] Updated weights for policy 0, policy_version 37600 (0.0008) [2023-12-27 05:55:29,867][20347] Updated weights for policy 0, policy_version 37610 (0.0007) [2023-12-27 05:55:30,367][20381] Updated weights for policy 1, policy_version 37581 (0.0008) [2023-12-27 05:55:30,421][20381] Updated weights for policy 1, policy_version 37591 (0.0010) [2023-12-27 05:55:30,469][20381] Updated weights for policy 1, policy_version 37601 (0.0010) [2023-12-27 05:55:30,588][20347] Updated weights for policy 0, policy_version 37620 (0.0007) [2023-12-27 05:55:30,635][20347] Updated weights for policy 0, policy_version 37630 (0.0008) [2023-12-27 05:55:30,695][20347] Updated weights for policy 0, policy_version 37640 (0.0005) [2023-12-27 05:55:31,107][20381] Updated weights for policy 1, policy_version 37611 (0.0008) [2023-12-27 05:55:31,166][20381] Updated weights for policy 1, policy_version 37621 (0.0007) [2023-12-27 05:55:31,222][20381] Updated weights for policy 1, policy_version 37631 (0.0006) [2023-12-27 05:55:31,332][20347] Updated weights for policy 0, policy_version 37650 (0.0006) [2023-12-27 05:55:31,397][20347] Updated weights for policy 0, policy_version 37660 (0.0008) [2023-12-27 05:55:31,462][20347] Updated weights for policy 0, policy_version 37670 (0.0008) [2023-12-27 05:55:31,526][20347] Updated weights for policy 0, policy_version 37680 (0.0007) [2023-12-27 05:55:31,915][20381] Updated weights for policy 1, policy_version 37641 (0.0010) [2023-12-27 05:55:31,967][20381] Updated weights for policy 1, policy_version 37651 (0.0011) [2023-12-27 05:55:32,018][20381] Updated weights for policy 1, policy_version 37661 (0.0009) [2023-12-27 05:55:32,073][20381] Updated weights for policy 1, policy_version 37671 (0.0010) [2023-12-27 05:55:32,224][20347] Updated weights for policy 0, policy_version 37690 (0.0005) [2023-12-27 05:55:32,288][20347] Updated weights for policy 0, policy_version 37700 (0.0009) [2023-12-27 05:55:32,353][20347] Updated weights for policy 0, policy_version 37710 (0.0009) [2023-12-27 05:55:32,368][19170] Fps is (10 sec: 21287.5, 60 sec: 20887.7, 300 sec: 20937.8). Total num frames: 19308544. Throughput: 0: 10349.8, 1: 10545.5. Samples: 19315208. Policy #0 lag: (min: 7.0, avg: 9.4, max: 39.0) [2023-12-27 05:55:32,369][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:32,771][20381] Updated weights for policy 1, policy_version 37681 (0.0008) [2023-12-27 05:55:32,821][20381] Updated weights for policy 1, policy_version 37691 (0.0005) [2023-12-27 05:55:32,876][20381] Updated weights for policy 1, policy_version 37701 (0.0005) [2023-12-27 05:55:33,006][20347] Updated weights for policy 0, policy_version 37720 (0.0008) [2023-12-27 05:55:33,062][20347] Updated weights for policy 0, policy_version 37730 (0.0007) [2023-12-27 05:55:33,119][20347] Updated weights for policy 0, policy_version 37740 (0.0007) [2023-12-27 05:55:33,439][20381] Updated weights for policy 1, policy_version 37711 (0.0005) [2023-12-27 05:55:33,497][20381] Updated weights for policy 1, policy_version 37721 (0.0005) [2023-12-27 05:55:33,545][20381] Updated weights for policy 1, policy_version 37731 (0.0005) [2023-12-27 05:55:33,790][20347] Updated weights for policy 0, policy_version 37750 (0.0007) [2023-12-27 05:55:33,856][20347] Updated weights for policy 0, policy_version 37760 (0.0006) [2023-12-27 05:55:33,911][20347] Updated weights for policy 0, policy_version 37770 (0.0006) [2023-12-27 05:55:34,117][20381] Updated weights for policy 1, policy_version 37741 (0.0007) [2023-12-27 05:55:34,175][20381] Updated weights for policy 1, policy_version 37751 (0.0009) [2023-12-27 05:55:34,226][20381] Updated weights for policy 1, policy_version 37761 (0.0008) [2023-12-27 05:55:34,507][20347] Updated weights for policy 0, policy_version 37780 (0.0008) [2023-12-27 05:55:34,563][20347] Updated weights for policy 0, policy_version 37790 (0.0010) [2023-12-27 05:55:34,620][20347] Updated weights for policy 0, policy_version 37800 (0.0008) [2023-12-27 05:55:34,881][20381] Updated weights for policy 1, policy_version 37771 (0.0007) [2023-12-27 05:55:34,945][20381] Updated weights for policy 1, policy_version 37781 (0.0008) [2023-12-27 05:55:35,004][20381] Updated weights for policy 1, policy_version 37791 (0.0008) [2023-12-27 05:55:35,259][20347] Updated weights for policy 0, policy_version 37810 (0.0007) [2023-12-27 05:55:35,310][20347] Updated weights for policy 0, policy_version 37820 (0.0006) [2023-12-27 05:55:35,360][20347] Updated weights for policy 0, policy_version 37830 (0.0008) [2023-12-27 05:55:35,410][20347] Updated weights for policy 0, policy_version 37840 (0.0010) [2023-12-27 05:55:35,666][20381] Updated weights for policy 1, policy_version 37801 (0.0008) [2023-12-27 05:55:35,713][20381] Updated weights for policy 1, policy_version 37811 (0.0009) [2023-12-27 05:55:35,764][20381] Updated weights for policy 1, policy_version 37821 (0.0005) [2023-12-27 05:55:35,811][20381] Updated weights for policy 1, policy_version 37831 (0.0005) [2023-12-27 05:55:36,064][20347] Updated weights for policy 0, policy_version 37850 (0.0007) [2023-12-27 05:55:36,127][20347] Updated weights for policy 0, policy_version 37860 (0.0007) [2023-12-27 05:55:36,186][20347] Updated weights for policy 0, policy_version 37870 (0.0007) [2023-12-27 05:55:36,496][20381] Updated weights for policy 1, policy_version 37841 (0.0010) [2023-12-27 05:55:36,550][20381] Updated weights for policy 1, policy_version 37851 (0.0011) [2023-12-27 05:55:36,606][20381] Updated weights for policy 1, policy_version 37861 (0.0010) [2023-12-27 05:55:36,821][20347] Updated weights for policy 0, policy_version 37880 (0.0008) [2023-12-27 05:55:36,879][20347] Updated weights for policy 0, policy_version 37890 (0.0007) [2023-12-27 05:55:36,937][20347] Updated weights for policy 0, policy_version 37900 (0.0008) [2023-12-27 05:55:37,212][20381] Updated weights for policy 1, policy_version 37871 (0.0007) [2023-12-27 05:55:37,272][20381] Updated weights for policy 1, policy_version 37881 (0.0008) [2023-12-27 05:55:37,331][20381] Updated weights for policy 1, policy_version 37891 (0.0008) [2023-12-27 05:55:37,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20966.0). Total num frames: 19415040. Throughput: 0: 10337.0, 1: 10550.7. Samples: 19379232. Policy #0 lag: (min: 7.0, avg: 9.4, max: 39.0) [2023-12-27 05:55:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000037896_9707520.pth... [2023-12-27 05:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000037904_9707520.pth... [2023-12-27 05:55:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000036688_9396224.pth [2023-12-27 05:55:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000036648_9388032.pth [2023-12-27 05:55:37,645][20347] Updated weights for policy 0, policy_version 37910 (0.0007) [2023-12-27 05:55:37,705][20347] Updated weights for policy 0, policy_version 37920 (0.0006) [2023-12-27 05:55:37,761][20347] Updated weights for policy 0, policy_version 37930 (0.0008) [2023-12-27 05:55:38,027][20381] Updated weights for policy 1, policy_version 37901 (0.0009) [2023-12-27 05:55:38,082][20381] Updated weights for policy 1, policy_version 37911 (0.0010) [2023-12-27 05:55:38,139][20381] Updated weights for policy 1, policy_version 37921 (0.0010) [2023-12-27 05:55:38,385][20347] Updated weights for policy 0, policy_version 37940 (0.0007) [2023-12-27 05:55:38,449][20347] Updated weights for policy 0, policy_version 37950 (0.0009) [2023-12-27 05:55:38,505][20347] Updated weights for policy 0, policy_version 37960 (0.0006) [2023-12-27 05:55:38,731][20381] Updated weights for policy 1, policy_version 37931 (0.0007) [2023-12-27 05:55:38,785][20381] Updated weights for policy 1, policy_version 37941 (0.0007) [2023-12-27 05:55:38,847][20381] Updated weights for policy 1, policy_version 37951 (0.0007) [2023-12-27 05:55:39,163][20347] Updated weights for policy 0, policy_version 37970 (0.0010) [2023-12-27 05:55:39,226][20347] Updated weights for policy 0, policy_version 37980 (0.0011) [2023-12-27 05:55:39,293][20347] Updated weights for policy 0, policy_version 37990 (0.0011) [2023-12-27 05:55:39,364][20347] Updated weights for policy 0, policy_version 38000 (0.0010) [2023-12-27 05:55:39,530][20381] Updated weights for policy 1, policy_version 37961 (0.0007) [2023-12-27 05:55:39,586][20381] Updated weights for policy 1, policy_version 37971 (0.0008) [2023-12-27 05:55:39,646][20381] Updated weights for policy 1, policy_version 37981 (0.0007) [2023-12-27 05:55:39,701][20381] Updated weights for policy 1, policy_version 37991 (0.0007) [2023-12-27 05:55:40,043][20347] Updated weights for policy 0, policy_version 38010 (0.0007) [2023-12-27 05:55:40,113][20347] Updated weights for policy 0, policy_version 38020 (0.0006) [2023-12-27 05:55:40,171][20347] Updated weights for policy 0, policy_version 38030 (0.0005) [2023-12-27 05:55:40,381][20381] Updated weights for policy 1, policy_version 38001 (0.0007) [2023-12-27 05:55:40,436][20381] Updated weights for policy 1, policy_version 38011 (0.0007) [2023-12-27 05:55:40,490][20381] Updated weights for policy 1, policy_version 38021 (0.0010) [2023-12-27 05:55:40,766][20347] Updated weights for policy 0, policy_version 38040 (0.0005) [2023-12-27 05:55:40,830][20347] Updated weights for policy 0, policy_version 38050 (0.0005) [2023-12-27 05:55:40,891][20347] Updated weights for policy 0, policy_version 38060 (0.0005) [2023-12-27 05:55:41,215][20381] Updated weights for policy 1, policy_version 38031 (0.0007) [2023-12-27 05:55:41,279][20381] Updated weights for policy 1, policy_version 38041 (0.0007) [2023-12-27 05:55:41,345][20381] Updated weights for policy 1, policy_version 38051 (0.0008) [2023-12-27 05:55:41,500][20347] Updated weights for policy 0, policy_version 38070 (0.0007) [2023-12-27 05:55:41,565][20347] Updated weights for policy 0, policy_version 38080 (0.0008) [2023-12-27 05:55:41,635][20347] Updated weights for policy 0, policy_version 38090 (0.0007) [2023-12-27 05:55:42,014][20381] Updated weights for policy 1, policy_version 38061 (0.0007) [2023-12-27 05:55:42,085][20381] Updated weights for policy 1, policy_version 38071 (0.0006) [2023-12-27 05:55:42,147][20381] Updated weights for policy 1, policy_version 38081 (0.0006) [2023-12-27 05:55:42,363][19170] Fps is (10 sec: 20491.1, 60 sec: 20889.5, 300 sec: 20910.4). Total num frames: 19513344. Throughput: 0: 10412.0, 1: 10531.2. Samples: 19505724. Policy #0 lag: (min: 7.0, avg: 9.4, max: 39.0) [2023-12-27 05:55:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:42,395][20347] Updated weights for policy 0, policy_version 38100 (0.0007) [2023-12-27 05:55:42,452][20347] Updated weights for policy 0, policy_version 38110 (0.0007) [2023-12-27 05:55:42,512][20347] Updated weights for policy 0, policy_version 38120 (0.0008) [2023-12-27 05:55:42,782][20381] Updated weights for policy 1, policy_version 38091 (0.0008) [2023-12-27 05:55:42,846][20381] Updated weights for policy 1, policy_version 38101 (0.0008) [2023-12-27 05:55:42,899][20381] Updated weights for policy 1, policy_version 38111 (0.0008) [2023-12-27 05:55:43,208][20347] Updated weights for policy 0, policy_version 38130 (0.0008) [2023-12-27 05:55:43,264][20347] Updated weights for policy 0, policy_version 38140 (0.0007) [2023-12-27 05:55:43,318][20347] Updated weights for policy 0, policy_version 38150 (0.0007) [2023-12-27 05:55:43,375][20347] Updated weights for policy 0, policy_version 38160 (0.0008) [2023-12-27 05:55:43,593][20381] Updated weights for policy 1, policy_version 38121 (0.0008) [2023-12-27 05:55:43,637][20381] Updated weights for policy 1, policy_version 38131 (0.0007) [2023-12-27 05:55:43,690][20381] Updated weights for policy 1, policy_version 38141 (0.0008) [2023-12-27 05:55:43,735][20381] Updated weights for policy 1, policy_version 38151 (0.0009) [2023-12-27 05:55:44,041][20347] Updated weights for policy 0, policy_version 38170 (0.0006) [2023-12-27 05:55:44,086][20347] Updated weights for policy 0, policy_version 38180 (0.0005) [2023-12-27 05:55:44,143][20347] Updated weights for policy 0, policy_version 38190 (0.0005) [2023-12-27 05:55:44,449][20381] Updated weights for policy 1, policy_version 38161 (0.0005) [2023-12-27 05:55:44,507][20381] Updated weights for policy 1, policy_version 38171 (0.0006) [2023-12-27 05:55:44,563][20381] Updated weights for policy 1, policy_version 38181 (0.0007) [2023-12-27 05:55:44,821][20347] Updated weights for policy 0, policy_version 38200 (0.0007) [2023-12-27 05:55:44,874][20347] Updated weights for policy 0, policy_version 38210 (0.0007) [2023-12-27 05:55:44,923][20347] Updated weights for policy 0, policy_version 38220 (0.0005) [2023-12-27 05:55:45,240][20381] Updated weights for policy 1, policy_version 38191 (0.0007) [2023-12-27 05:55:45,286][20381] Updated weights for policy 1, policy_version 38201 (0.0007) [2023-12-27 05:55:45,333][20381] Updated weights for policy 1, policy_version 38211 (0.0010) [2023-12-27 05:55:45,499][20347] Updated weights for policy 0, policy_version 38230 (0.0007) [2023-12-27 05:55:45,542][20347] Updated weights for policy 0, policy_version 38240 (0.0005) [2023-12-27 05:55:45,593][20347] Updated weights for policy 0, policy_version 38250 (0.0005) [2023-12-27 05:55:46,048][20381] Updated weights for policy 1, policy_version 38221 (0.0008) [2023-12-27 05:55:46,109][20381] Updated weights for policy 1, policy_version 38231 (0.0005) [2023-12-27 05:55:46,167][20381] Updated weights for policy 1, policy_version 38241 (0.0005) [2023-12-27 05:55:46,258][20347] Updated weights for policy 0, policy_version 38260 (0.0006) [2023-12-27 05:55:46,305][20347] Updated weights for policy 0, policy_version 38270 (0.0008) [2023-12-27 05:55:46,358][20347] Updated weights for policy 0, policy_version 38280 (0.0008) [2023-12-27 05:55:46,799][20381] Updated weights for policy 1, policy_version 38251 (0.0006) [2023-12-27 05:55:46,853][20381] Updated weights for policy 1, policy_version 38261 (0.0009) [2023-12-27 05:55:46,908][20381] Updated weights for policy 1, policy_version 38271 (0.0005) [2023-12-27 05:55:46,993][20347] Updated weights for policy 0, policy_version 38290 (0.0007) [2023-12-27 05:55:47,054][20347] Updated weights for policy 0, policy_version 38300 (0.0005) [2023-12-27 05:55:47,115][20347] Updated weights for policy 0, policy_version 38310 (0.0005) [2023-12-27 05:55:47,171][20347] Updated weights for policy 0, policy_version 38320 (0.0005) [2023-12-27 05:55:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 19619840. Throughput: 0: 10420.4, 1: 10466.9. Samples: 19628988. Policy #0 lag: (min: 7.0, avg: 9.4, max: 39.0) [2023-12-27 05:55:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:47,599][20381] Updated weights for policy 1, policy_version 38281 (0.0009) [2023-12-27 05:55:47,656][20381] Updated weights for policy 1, policy_version 38291 (0.0010) [2023-12-27 05:55:47,713][20381] Updated weights for policy 1, policy_version 38301 (0.0010) [2023-12-27 05:55:47,767][20347] Updated weights for policy 0, policy_version 38330 (0.0009) [2023-12-27 05:55:47,780][20381] Updated weights for policy 1, policy_version 38311 (0.0010) [2023-12-27 05:55:47,823][20347] Updated weights for policy 0, policy_version 38340 (0.0006) [2023-12-27 05:55:47,875][20347] Updated weights for policy 0, policy_version 38350 (0.0008) [2023-12-27 05:55:48,397][20381] Updated weights for policy 1, policy_version 38321 (0.0008) [2023-12-27 05:55:48,457][20381] Updated weights for policy 1, policy_version 38331 (0.0008) [2023-12-27 05:55:48,515][20381] Updated weights for policy 1, policy_version 38341 (0.0007) [2023-12-27 05:55:48,533][20347] Updated weights for policy 0, policy_version 38360 (0.0011) [2023-12-27 05:55:48,582][20347] Updated weights for policy 0, policy_version 38370 (0.0010) [2023-12-27 05:55:48,634][20347] Updated weights for policy 0, policy_version 38380 (0.0010) [2023-12-27 05:55:49,204][20381] Updated weights for policy 1, policy_version 38351 (0.0006) [2023-12-27 05:55:49,265][20381] Updated weights for policy 1, policy_version 38361 (0.0007) [2023-12-27 05:55:49,327][20381] Updated weights for policy 1, policy_version 38371 (0.0007) [2023-12-27 05:55:49,330][20347] Updated weights for policy 0, policy_version 38390 (0.0009) [2023-12-27 05:55:49,389][20347] Updated weights for policy 0, policy_version 38400 (0.0010) [2023-12-27 05:55:49,454][20347] Updated weights for policy 0, policy_version 38410 (0.0010) [2023-12-27 05:55:49,983][20381] Updated weights for policy 1, policy_version 38381 (0.0007) [2023-12-27 05:55:50,041][20381] Updated weights for policy 1, policy_version 38391 (0.0007) [2023-12-27 05:55:50,096][20347] Updated weights for policy 0, policy_version 38420 (0.0009) [2023-12-27 05:55:50,105][20381] Updated weights for policy 1, policy_version 38401 (0.0008) [2023-12-27 05:55:50,150][20347] Updated weights for policy 0, policy_version 38430 (0.0008) [2023-12-27 05:55:50,202][20347] Updated weights for policy 0, policy_version 38440 (0.0010) [2023-12-27 05:55:50,812][20381] Updated weights for policy 1, policy_version 38411 (0.0007) [2023-12-27 05:55:50,871][20381] Updated weights for policy 1, policy_version 38421 (0.0007) [2023-12-27 05:55:50,882][20347] Updated weights for policy 0, policy_version 38450 (0.0010) [2023-12-27 05:55:50,927][20347] Updated weights for policy 0, policy_version 38460 (0.0010) [2023-12-27 05:55:50,938][20381] Updated weights for policy 1, policy_version 38431 (0.0006) [2023-12-27 05:55:50,982][20347] Updated weights for policy 0, policy_version 38470 (0.0008) [2023-12-27 05:55:51,047][20347] Updated weights for policy 0, policy_version 38480 (0.0010) [2023-12-27 05:55:51,616][20381] Updated weights for policy 1, policy_version 38441 (0.0007) [2023-12-27 05:55:51,666][20381] Updated weights for policy 1, policy_version 38451 (0.0008) [2023-12-27 05:55:51,713][20381] Updated weights for policy 1, policy_version 38461 (0.0005) [2023-12-27 05:55:51,763][20347] Updated weights for policy 0, policy_version 38490 (0.0007) [2023-12-27 05:55:51,778][20381] Updated weights for policy 1, policy_version 38471 (0.0008) [2023-12-27 05:55:51,832][20347] Updated weights for policy 0, policy_version 38500 (0.0008) [2023-12-27 05:55:51,894][20347] Updated weights for policy 0, policy_version 38510 (0.0009) [2023-12-27 05:55:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20882.7). Total num frames: 19718144. Throughput: 0: 10400.3, 1: 10440.3. Samples: 19691008. Policy #0 lag: (min: 7.0, avg: 9.4, max: 39.0) [2023-12-27 05:55:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000038512_9863168.pth... [2023-12-27 05:55:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000037296_9551872.pth [2023-12-27 05:55:52,396][20381] Updated weights for policy 1, policy_version 38481 (0.0006) [2023-12-27 05:55:52,451][20381] Updated weights for policy 1, policy_version 38491 (0.0006) [2023-12-27 05:55:52,506][20381] Updated weights for policy 1, policy_version 38501 (0.0007) [2023-12-27 05:55:52,521][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000038504_9863168.pth... [2023-12-27 05:55:52,524][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000037256_9543680.pth [2023-12-27 05:55:52,570][20347] Updated weights for policy 0, policy_version 38520 (0.0006) [2023-12-27 05:55:52,627][20347] Updated weights for policy 0, policy_version 38530 (0.0005) [2023-12-27 05:55:52,689][20347] Updated weights for policy 0, policy_version 38540 (0.0005) [2023-12-27 05:55:53,198][20347] Updated weights for policy 0, policy_version 38550 (0.0005) [2023-12-27 05:55:53,242][20381] Updated weights for policy 1, policy_version 38511 (0.0007) [2023-12-27 05:55:53,247][20347] Updated weights for policy 0, policy_version 38560 (0.0005) [2023-12-27 05:55:53,293][20381] Updated weights for policy 1, policy_version 38521 (0.0007) [2023-12-27 05:55:53,298][20347] Updated weights for policy 0, policy_version 38570 (0.0007) [2023-12-27 05:55:53,342][20381] Updated weights for policy 1, policy_version 38531 (0.0006) [2023-12-27 05:55:53,875][20347] Updated weights for policy 0, policy_version 38580 (0.0010) [2023-12-27 05:55:53,920][20347] Updated weights for policy 0, policy_version 38590 (0.0010) [2023-12-27 05:55:53,965][20347] Updated weights for policy 0, policy_version 38600 (0.0010) [2023-12-27 05:55:54,085][20381] Updated weights for policy 1, policy_version 38541 (0.0008) [2023-12-27 05:55:54,139][20381] Updated weights for policy 1, policy_version 38551 (0.0008) [2023-12-27 05:55:54,188][20381] Updated weights for policy 1, policy_version 38561 (0.0007) [2023-12-27 05:55:54,638][20347] Updated weights for policy 0, policy_version 38610 (0.0009) [2023-12-27 05:55:54,702][20347] Updated weights for policy 0, policy_version 38620 (0.0007) [2023-12-27 05:55:54,760][20347] Updated weights for policy 0, policy_version 38630 (0.0007) [2023-12-27 05:55:54,817][20347] Updated weights for policy 0, policy_version 38640 (0.0007) [2023-12-27 05:55:54,910][20381] Updated weights for policy 1, policy_version 38571 (0.0008) [2023-12-27 05:55:54,976][20381] Updated weights for policy 1, policy_version 38581 (0.0008) [2023-12-27 05:55:55,036][20381] Updated weights for policy 1, policy_version 38591 (0.0008) [2023-12-27 05:55:55,481][20347] Updated weights for policy 0, policy_version 38650 (0.0007) [2023-12-27 05:55:55,536][20347] Updated weights for policy 0, policy_version 38660 (0.0007) [2023-12-27 05:55:55,588][20347] Updated weights for policy 0, policy_version 38670 (0.0008) [2023-12-27 05:55:55,729][20381] Updated weights for policy 1, policy_version 38601 (0.0008) [2023-12-27 05:55:55,777][20381] Updated weights for policy 1, policy_version 38611 (0.0007) [2023-12-27 05:55:55,832][20381] Updated weights for policy 1, policy_version 38621 (0.0007) [2023-12-27 05:55:55,889][20381] Updated weights for policy 1, policy_version 38631 (0.0007) [2023-12-27 05:55:56,215][20347] Updated weights for policy 0, policy_version 38680 (0.0007) [2023-12-27 05:55:56,264][20347] Updated weights for policy 0, policy_version 38690 (0.0007) [2023-12-27 05:55:56,309][20347] Updated weights for policy 0, policy_version 38700 (0.0008) [2023-12-27 05:55:56,557][20381] Updated weights for policy 1, policy_version 38641 (0.0007) [2023-12-27 05:55:56,602][20381] Updated weights for policy 1, policy_version 38651 (0.0008) [2023-12-27 05:55:56,649][20381] Updated weights for policy 1, policy_version 38661 (0.0008) [2023-12-27 05:55:56,994][20347] Updated weights for policy 0, policy_version 38710 (0.0007) [2023-12-27 05:55:57,058][20347] Updated weights for policy 0, policy_version 38720 (0.0009) [2023-12-27 05:55:57,115][20347] Updated weights for policy 0, policy_version 38730 (0.0010) [2023-12-27 05:55:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 19824640. Throughput: 0: 10459.7, 1: 10417.8. Samples: 19814656. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:55:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:55:57,366][20381] Updated weights for policy 1, policy_version 38671 (0.0009) [2023-12-27 05:55:57,418][20381] Updated weights for policy 1, policy_version 38681 (0.0007) [2023-12-27 05:55:57,470][20381] Updated weights for policy 1, policy_version 38691 (0.0010) [2023-12-27 05:55:57,730][20347] Updated weights for policy 0, policy_version 38740 (0.0008) [2023-12-27 05:55:57,778][20347] Updated weights for policy 0, policy_version 38750 (0.0006) [2023-12-27 05:55:57,822][20347] Updated weights for policy 0, policy_version 38760 (0.0010) [2023-12-27 05:55:58,116][20381] Updated weights for policy 1, policy_version 38701 (0.0008) [2023-12-27 05:55:58,181][20381] Updated weights for policy 1, policy_version 38711 (0.0007) [2023-12-27 05:55:58,246][20381] Updated weights for policy 1, policy_version 38721 (0.0008) [2023-12-27 05:55:58,518][20347] Updated weights for policy 0, policy_version 38770 (0.0010) [2023-12-27 05:55:58,575][20347] Updated weights for policy 0, policy_version 38780 (0.0010) [2023-12-27 05:55:58,644][20347] Updated weights for policy 0, policy_version 38790 (0.0009) [2023-12-27 05:55:58,705][20347] Updated weights for policy 0, policy_version 38800 (0.0007) [2023-12-27 05:55:59,019][20381] Updated weights for policy 1, policy_version 38731 (0.0008) [2023-12-27 05:55:59,077][20381] Updated weights for policy 1, policy_version 38741 (0.0007) [2023-12-27 05:55:59,142][20381] Updated weights for policy 1, policy_version 38751 (0.0008) [2023-12-27 05:55:59,400][20347] Updated weights for policy 0, policy_version 38810 (0.0008) [2023-12-27 05:55:59,454][20347] Updated weights for policy 0, policy_version 38820 (0.0007) [2023-12-27 05:55:59,512][20347] Updated weights for policy 0, policy_version 38830 (0.0006) [2023-12-27 05:55:59,860][20381] Updated weights for policy 1, policy_version 38761 (0.0008) [2023-12-27 05:55:59,917][20381] Updated weights for policy 1, policy_version 38771 (0.0008) [2023-12-27 05:55:59,973][20381] Updated weights for policy 1, policy_version 38781 (0.0008) [2023-12-27 05:56:00,020][20381] Updated weights for policy 1, policy_version 38791 (0.0008) [2023-12-27 05:56:00,146][20347] Updated weights for policy 0, policy_version 38840 (0.0006) [2023-12-27 05:56:00,214][20347] Updated weights for policy 0, policy_version 38850 (0.0005) [2023-12-27 05:56:00,281][20347] Updated weights for policy 0, policy_version 38860 (0.0005) [2023-12-27 05:56:00,732][20381] Updated weights for policy 1, policy_version 38801 (0.0008) [2023-12-27 05:56:00,795][20381] Updated weights for policy 1, policy_version 38811 (0.0005) [2023-12-27 05:56:00,847][20381] Updated weights for policy 1, policy_version 38821 (0.0007) [2023-12-27 05:56:00,864][20347] Updated weights for policy 0, policy_version 38870 (0.0005) [2023-12-27 05:56:00,918][20347] Updated weights for policy 0, policy_version 38880 (0.0007) [2023-12-27 05:56:00,977][20347] Updated weights for policy 0, policy_version 38890 (0.0008) [2023-12-27 05:56:01,511][20381] Updated weights for policy 1, policy_version 38831 (0.0008) [2023-12-27 05:56:01,566][20381] Updated weights for policy 1, policy_version 38841 (0.0008) [2023-12-27 05:56:01,617][20347] Updated weights for policy 0, policy_version 38900 (0.0008) [2023-12-27 05:56:01,621][20381] Updated weights for policy 1, policy_version 38851 (0.0006) [2023-12-27 05:56:01,684][20347] Updated weights for policy 0, policy_version 38910 (0.0008) [2023-12-27 05:56:01,745][20347] Updated weights for policy 0, policy_version 38920 (0.0007) [2023-12-27 05:56:02,284][20381] Updated weights for policy 1, policy_version 38861 (0.0007) [2023-12-27 05:56:02,337][20381] Updated weights for policy 1, policy_version 38871 (0.0010) [2023-12-27 05:56:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 19922944. Throughput: 0: 10455.1, 1: 10380.2. Samples: 19937968. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:56:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:02,401][20381] Updated weights for policy 1, policy_version 38881 (0.0006) [2023-12-27 05:56:02,474][20347] Updated weights for policy 0, policy_version 38930 (0.0007) [2023-12-27 05:56:02,530][20347] Updated weights for policy 0, policy_version 38940 (0.0007) [2023-12-27 05:56:02,585][20347] Updated weights for policy 0, policy_version 38950 (0.0008) [2023-12-27 05:56:02,649][20347] Updated weights for policy 0, policy_version 38960 (0.0007) [2023-12-27 05:56:02,982][20381] Updated weights for policy 1, policy_version 38891 (0.0007) [2023-12-27 05:56:03,042][20381] Updated weights for policy 1, policy_version 38901 (0.0011) [2023-12-27 05:56:03,098][20381] Updated weights for policy 1, policy_version 38911 (0.0011) [2023-12-27 05:56:03,268][20347] Updated weights for policy 0, policy_version 38970 (0.0009) [2023-12-27 05:56:03,323][20347] Updated weights for policy 0, policy_version 38980 (0.0010) [2023-12-27 05:56:03,374][20347] Updated weights for policy 0, policy_version 38990 (0.0005) [2023-12-27 05:56:03,710][20381] Updated weights for policy 1, policy_version 38921 (0.0011) [2023-12-27 05:56:03,775][20381] Updated weights for policy 1, policy_version 38931 (0.0011) [2023-12-27 05:56:03,831][20381] Updated weights for policy 1, policy_version 38941 (0.0011) [2023-12-27 05:56:03,891][20381] Updated weights for policy 1, policy_version 38951 (0.0011) [2023-12-27 05:56:03,996][20347] Updated weights for policy 0, policy_version 39000 (0.0006) [2023-12-27 05:56:04,051][20347] Updated weights for policy 0, policy_version 39010 (0.0008) [2023-12-27 05:56:04,112][20347] Updated weights for policy 0, policy_version 39020 (0.0011) [2023-12-27 05:56:04,595][20381] Updated weights for policy 1, policy_version 38961 (0.0009) [2023-12-27 05:56:04,651][20381] Updated weights for policy 1, policy_version 38971 (0.0011) [2023-12-27 05:56:04,708][20381] Updated weights for policy 1, policy_version 38981 (0.0011) [2023-12-27 05:56:04,753][20347] Updated weights for policy 0, policy_version 39030 (0.0010) [2023-12-27 05:56:04,802][20347] Updated weights for policy 0, policy_version 39040 (0.0010) [2023-12-27 05:56:04,857][20347] Updated weights for policy 0, policy_version 39050 (0.0010) [2023-12-27 05:56:05,388][20381] Updated weights for policy 1, policy_version 38991 (0.0011) [2023-12-27 05:56:05,447][20381] Updated weights for policy 1, policy_version 39001 (0.0011) [2023-12-27 05:56:05,488][20347] Updated weights for policy 0, policy_version 39060 (0.0008) [2023-12-27 05:56:05,507][20381] Updated weights for policy 1, policy_version 39011 (0.0011) [2023-12-27 05:56:05,537][20347] Updated weights for policy 0, policy_version 39070 (0.0005) [2023-12-27 05:56:05,588][20347] Updated weights for policy 0, policy_version 39080 (0.0005) [2023-12-27 05:56:06,179][20347] Updated weights for policy 0, policy_version 39090 (0.0007) [2023-12-27 05:56:06,180][20381] Updated weights for policy 1, policy_version 39021 (0.0011) [2023-12-27 05:56:06,234][20347] Updated weights for policy 0, policy_version 39100 (0.0010) [2023-12-27 05:56:06,241][20381] Updated weights for policy 1, policy_version 39031 (0.0011) [2023-12-27 05:56:06,290][20347] Updated weights for policy 0, policy_version 39110 (0.0010) [2023-12-27 05:56:06,297][20381] Updated weights for policy 1, policy_version 39041 (0.0009) [2023-12-27 05:56:06,346][20347] Updated weights for policy 0, policy_version 39120 (0.0010) [2023-12-27 05:56:06,977][20381] Updated weights for policy 1, policy_version 39051 (0.0008) [2023-12-27 05:56:07,032][20381] Updated weights for policy 1, policy_version 39061 (0.0010) [2023-12-27 05:56:07,050][20347] Updated weights for policy 0, policy_version 39130 (0.0010) [2023-12-27 05:56:07,084][20381] Updated weights for policy 1, policy_version 39071 (0.0010) [2023-12-27 05:56:07,102][20347] Updated weights for policy 0, policy_version 39140 (0.0010) [2023-12-27 05:56:07,160][20347] Updated weights for policy 0, policy_version 39150 (0.0010) [2023-12-27 05:56:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 20037632. Throughput: 0: 10483.8, 1: 10406.8. Samples: 20001856. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:56:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000039080_10010624.pth... [2023-12-27 05:56:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000039152_10027008.pth... [2023-12-27 05:56:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000037896_9707520.pth [2023-12-27 05:56:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000037904_9707520.pth [2023-12-27 05:56:07,766][20381] Updated weights for policy 1, policy_version 39081 (0.0010) [2023-12-27 05:56:07,769][20347] Updated weights for policy 0, policy_version 39160 (0.0006) [2023-12-27 05:56:07,819][20381] Updated weights for policy 1, policy_version 39091 (0.0010) [2023-12-27 05:56:07,820][20347] Updated weights for policy 0, policy_version 39170 (0.0006) [2023-12-27 05:56:07,873][20347] Updated weights for policy 0, policy_version 39180 (0.0005) [2023-12-27 05:56:07,874][20381] Updated weights for policy 1, policy_version 39101 (0.0010) [2023-12-27 05:56:07,923][20381] Updated weights for policy 1, policy_version 39111 (0.0010) [2023-12-27 05:56:08,463][20347] Updated weights for policy 0, policy_version 39190 (0.0007) [2023-12-27 05:56:08,516][20347] Updated weights for policy 0, policy_version 39200 (0.0008) [2023-12-27 05:56:08,570][20347] Updated weights for policy 0, policy_version 39210 (0.0008) [2023-12-27 05:56:08,632][20381] Updated weights for policy 1, policy_version 39121 (0.0010) [2023-12-27 05:56:08,694][20381] Updated weights for policy 1, policy_version 39131 (0.0011) [2023-12-27 05:56:08,755][20381] Updated weights for policy 1, policy_version 39141 (0.0010) [2023-12-27 05:56:09,191][20347] Updated weights for policy 0, policy_version 39220 (0.0006) [2023-12-27 05:56:09,252][20347] Updated weights for policy 0, policy_version 39230 (0.0009) [2023-12-27 05:56:09,310][20347] Updated weights for policy 0, policy_version 39240 (0.0008) [2023-12-27 05:56:09,440][20381] Updated weights for policy 1, policy_version 39151 (0.0009) [2023-12-27 05:56:09,500][20381] Updated weights for policy 1, policy_version 39161 (0.0008) [2023-12-27 05:56:09,553][20381] Updated weights for policy 1, policy_version 39171 (0.0008) [2023-12-27 05:56:10,009][20347] Updated weights for policy 0, policy_version 39250 (0.0008) [2023-12-27 05:56:10,066][20347] Updated weights for policy 0, policy_version 39260 (0.0008) [2023-12-27 05:56:10,124][20347] Updated weights for policy 0, policy_version 39270 (0.0008) [2023-12-27 05:56:10,188][20347] Updated weights for policy 0, policy_version 39280 (0.0007) [2023-12-27 05:56:10,297][20381] Updated weights for policy 1, policy_version 39181 (0.0009) [2023-12-27 05:56:10,359][20381] Updated weights for policy 1, policy_version 39191 (0.0008) [2023-12-27 05:56:10,406][20381] Updated weights for policy 1, policy_version 39201 (0.0007) [2023-12-27 05:56:10,909][20347] Updated weights for policy 0, policy_version 39290 (0.0007) [2023-12-27 05:56:10,958][20347] Updated weights for policy 0, policy_version 39300 (0.0007) [2023-12-27 05:56:11,010][20347] Updated weights for policy 0, policy_version 39310 (0.0007) [2023-12-27 05:56:11,103][20381] Updated weights for policy 1, policy_version 39211 (0.0007) [2023-12-27 05:56:11,165][20381] Updated weights for policy 1, policy_version 39221 (0.0011) [2023-12-27 05:56:11,222][20381] Updated weights for policy 1, policy_version 39231 (0.0010) [2023-12-27 05:56:11,747][20347] Updated weights for policy 0, policy_version 39320 (0.0009) [2023-12-27 05:56:11,801][20347] Updated weights for policy 0, policy_version 39330 (0.0008) [2023-12-27 05:56:11,858][20347] Updated weights for policy 0, policy_version 39340 (0.0007) [2023-12-27 05:56:11,925][20381] Updated weights for policy 1, policy_version 39241 (0.0008) [2023-12-27 05:56:11,980][20381] Updated weights for policy 1, policy_version 39251 (0.0008) [2023-12-27 05:56:12,042][20381] Updated weights for policy 1, policy_version 39261 (0.0008) [2023-12-27 05:56:12,111][20381] Updated weights for policy 1, policy_version 39271 (0.0008) [2023-12-27 05:56:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 20135936. Throughput: 0: 10519.8, 1: 10321.9. Samples: 20125716. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:56:12,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:12,614][20347] Updated weights for policy 0, policy_version 39350 (0.0008) [2023-12-27 05:56:12,673][20347] Updated weights for policy 0, policy_version 39360 (0.0009) [2023-12-27 05:56:12,731][20347] Updated weights for policy 0, policy_version 39370 (0.0009) [2023-12-27 05:56:12,937][20381] Updated weights for policy 1, policy_version 39281 (0.0009) [2023-12-27 05:56:12,991][20381] Updated weights for policy 1, policy_version 39291 (0.0006) [2023-12-27 05:56:13,046][20381] Updated weights for policy 1, policy_version 39301 (0.0007) [2023-12-27 05:56:13,540][20347] Updated weights for policy 0, policy_version 39380 (0.0009) [2023-12-27 05:56:13,592][20347] Updated weights for policy 0, policy_version 39390 (0.0007) [2023-12-27 05:56:13,600][20381] Updated weights for policy 1, policy_version 39311 (0.0009) [2023-12-27 05:56:13,649][20347] Updated weights for policy 0, policy_version 39400 (0.0008) [2023-12-27 05:56:13,658][20381] Updated weights for policy 1, policy_version 39321 (0.0006) [2023-12-27 05:56:13,714][20381] Updated weights for policy 1, policy_version 39331 (0.0005) [2023-12-27 05:56:14,215][20381] Updated weights for policy 1, policy_version 39341 (0.0005) [2023-12-27 05:56:14,268][20381] Updated weights for policy 1, policy_version 39351 (0.0005) [2023-12-27 05:56:14,324][20381] Updated weights for policy 1, policy_version 39361 (0.0005) [2023-12-27 05:56:14,385][20347] Updated weights for policy 0, policy_version 39410 (0.0008) [2023-12-27 05:56:14,432][20347] Updated weights for policy 0, policy_version 39420 (0.0008) [2023-12-27 05:56:14,486][20347] Updated weights for policy 0, policy_version 39430 (0.0007) [2023-12-27 05:56:14,542][20347] Updated weights for policy 0, policy_version 39440 (0.0007) [2023-12-27 05:56:14,935][20381] Updated weights for policy 1, policy_version 39371 (0.0007) [2023-12-27 05:56:14,985][20381] Updated weights for policy 1, policy_version 39381 (0.0010) [2023-12-27 05:56:15,038][20381] Updated weights for policy 1, policy_version 39391 (0.0010) [2023-12-27 05:56:15,240][20347] Updated weights for policy 0, policy_version 39450 (0.0007) [2023-12-27 05:56:15,301][20347] Updated weights for policy 0, policy_version 39460 (0.0007) [2023-12-27 05:56:15,362][20347] Updated weights for policy 0, policy_version 39470 (0.0007) [2023-12-27 05:56:15,744][20381] Updated weights for policy 1, policy_version 39401 (0.0010) [2023-12-27 05:56:15,798][20381] Updated weights for policy 1, policy_version 39411 (0.0010) [2023-12-27 05:56:15,856][20381] Updated weights for policy 1, policy_version 39421 (0.0010) [2023-12-27 05:56:15,905][20381] Updated weights for policy 1, policy_version 39431 (0.0010) [2023-12-27 05:56:15,984][20347] Updated weights for policy 0, policy_version 39480 (0.0006) [2023-12-27 05:56:16,052][20347] Updated weights for policy 0, policy_version 39490 (0.0006) [2023-12-27 05:56:16,113][20347] Updated weights for policy 0, policy_version 39500 (0.0006) [2023-12-27 05:56:16,599][20381] Updated weights for policy 1, policy_version 39441 (0.0010) [2023-12-27 05:56:16,658][20347] Updated weights for policy 0, policy_version 39510 (0.0009) [2023-12-27 05:56:16,660][20381] Updated weights for policy 1, policy_version 39451 (0.0011) [2023-12-27 05:56:16,707][20347] Updated weights for policy 0, policy_version 39520 (0.0010) [2023-12-27 05:56:16,715][20381] Updated weights for policy 1, policy_version 39461 (0.0010) [2023-12-27 05:56:16,753][20347] Updated weights for policy 0, policy_version 39530 (0.0008) [2023-12-27 05:56:17,333][20381] Updated weights for policy 1, policy_version 39471 (0.0010) [2023-12-27 05:56:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 20234240. Throughput: 0: 10494.7, 1: 10274.5. Samples: 20249708. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:56:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:17,368][20347] Updated weights for policy 0, policy_version 39540 (0.0006) [2023-12-27 05:56:17,398][20381] Updated weights for policy 1, policy_version 39481 (0.0011) [2023-12-27 05:56:17,422][20347] Updated weights for policy 0, policy_version 39550 (0.0007) [2023-12-27 05:56:17,456][20381] Updated weights for policy 1, policy_version 39491 (0.0010) [2023-12-27 05:56:17,469][20347] Updated weights for policy 0, policy_version 39560 (0.0008) [2023-12-27 05:56:18,069][20347] Updated weights for policy 0, policy_version 39570 (0.0005) [2023-12-27 05:56:18,129][20347] Updated weights for policy 0, policy_version 39580 (0.0005) [2023-12-27 05:56:18,140][20381] Updated weights for policy 1, policy_version 39501 (0.0011) [2023-12-27 05:56:18,187][20347] Updated weights for policy 0, policy_version 39590 (0.0005) [2023-12-27 05:56:18,198][20381] Updated weights for policy 1, policy_version 39511 (0.0011) [2023-12-27 05:56:18,237][20347] Updated weights for policy 0, policy_version 39600 (0.0005) [2023-12-27 05:56:18,250][20381] Updated weights for policy 1, policy_version 39521 (0.0010) [2023-12-27 05:56:18,847][20347] Updated weights for policy 0, policy_version 39610 (0.0008) [2023-12-27 05:56:18,901][20347] Updated weights for policy 0, policy_version 39620 (0.0008) [2023-12-27 05:56:18,903][20381] Updated weights for policy 1, policy_version 39531 (0.0009) [2023-12-27 05:56:18,959][20381] Updated weights for policy 1, policy_version 39541 (0.0007) [2023-12-27 05:56:18,959][20347] Updated weights for policy 0, policy_version 39630 (0.0006) [2023-12-27 05:56:19,010][20381] Updated weights for policy 1, policy_version 39551 (0.0008) [2023-12-27 05:56:19,608][20347] Updated weights for policy 0, policy_version 39640 (0.0007) [2023-12-27 05:56:19,671][20347] Updated weights for policy 0, policy_version 39650 (0.0008) [2023-12-27 05:56:19,672][20381] Updated weights for policy 1, policy_version 39561 (0.0007) [2023-12-27 05:56:19,729][20381] Updated weights for policy 1, policy_version 39571 (0.0006) [2023-12-27 05:56:19,730][20347] Updated weights for policy 0, policy_version 39660 (0.0008) [2023-12-27 05:56:19,793][20381] Updated weights for policy 1, policy_version 39581 (0.0006) [2023-12-27 05:56:19,858][20381] Updated weights for policy 1, policy_version 39591 (0.0008) [2023-12-27 05:56:20,478][20347] Updated weights for policy 0, policy_version 39670 (0.0007) [2023-12-27 05:56:20,531][20347] Updated weights for policy 0, policy_version 39680 (0.0008) [2023-12-27 05:56:20,542][20381] Updated weights for policy 1, policy_version 39601 (0.0006) [2023-12-27 05:56:20,583][20347] Updated weights for policy 0, policy_version 39690 (0.0007) [2023-12-27 05:56:20,604][20381] Updated weights for policy 1, policy_version 39611 (0.0007) [2023-12-27 05:56:20,658][20381] Updated weights for policy 1, policy_version 39621 (0.0007) [2023-12-27 05:56:21,310][20347] Updated weights for policy 0, policy_version 39700 (0.0006) [2023-12-27 05:56:21,341][20381] Updated weights for policy 1, policy_version 39631 (0.0007) [2023-12-27 05:56:21,375][20347] Updated weights for policy 0, policy_version 39710 (0.0006) [2023-12-27 05:56:21,412][20381] Updated weights for policy 1, policy_version 39641 (0.0008) [2023-12-27 05:56:21,440][20347] Updated weights for policy 0, policy_version 39720 (0.0007) [2023-12-27 05:56:21,469][20381] Updated weights for policy 1, policy_version 39651 (0.0007) [2023-12-27 05:56:22,126][20381] Updated weights for policy 1, policy_version 39661 (0.0007) [2023-12-27 05:56:22,150][20347] Updated weights for policy 0, policy_version 39730 (0.0009) [2023-12-27 05:56:22,184][20381] Updated weights for policy 1, policy_version 39671 (0.0008) [2023-12-27 05:56:22,199][20347] Updated weights for policy 0, policy_version 39740 (0.0010) [2023-12-27 05:56:22,245][20347] Updated weights for policy 0, policy_version 39750 (0.0010) [2023-12-27 05:56:22,246][20381] Updated weights for policy 1, policy_version 39681 (0.0007) [2023-12-27 05:56:22,305][20347] Updated weights for policy 0, policy_version 39760 (0.0007) [2023-12-27 05:56:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 20348928. Throughput: 0: 10513.5, 1: 10229.5. Samples: 20312668. Policy #0 lag: (min: 31.0, avg: 31.8, max: 56.0) [2023-12-27 05:56:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000039688_10166272.pth... [2023-12-27 05:56:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000039760_10182656.pth... [2023-12-27 05:56:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000038504_9863168.pth [2023-12-27 05:56:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000038512_9863168.pth [2023-12-27 05:56:22,954][20381] Updated weights for policy 1, policy_version 39691 (0.0007) [2023-12-27 05:56:23,013][20381] Updated weights for policy 1, policy_version 39701 (0.0007) [2023-12-27 05:56:23,032][20347] Updated weights for policy 0, policy_version 39770 (0.0006) [2023-12-27 05:56:23,069][20381] Updated weights for policy 1, policy_version 39711 (0.0006) [2023-12-27 05:56:23,090][20347] Updated weights for policy 0, policy_version 39781 (0.0007) [2023-12-27 05:56:23,152][20347] Updated weights for policy 0, policy_version 39791 (0.0007) [2023-12-27 05:56:23,676][20381] Updated weights for policy 1, policy_version 39721 (0.0006) [2023-12-27 05:56:23,738][20381] Updated weights for policy 1, policy_version 39731 (0.0007) [2023-12-27 05:56:23,754][20347] Updated weights for policy 0, policy_version 39801 (0.0007) [2023-12-27 05:56:23,787][20381] Updated weights for policy 1, policy_version 39741 (0.0005) [2023-12-27 05:56:23,802][20347] Updated weights for policy 0, policy_version 39811 (0.0008) [2023-12-27 05:56:23,842][20381] Updated weights for policy 1, policy_version 39751 (0.0006) [2023-12-27 05:56:23,854][20347] Updated weights for policy 0, policy_version 39821 (0.0007) [2023-12-27 05:56:24,496][20381] Updated weights for policy 1, policy_version 39761 (0.0006) [2023-12-27 05:56:24,505][20347] Updated weights for policy 0, policy_version 39831 (0.0007) [2023-12-27 05:56:24,553][20381] Updated weights for policy 1, policy_version 39771 (0.0008) [2023-12-27 05:56:24,560][20347] Updated weights for policy 0, policy_version 39841 (0.0005) [2023-12-27 05:56:24,606][20347] Updated weights for policy 0, policy_version 39851 (0.0006) [2023-12-27 05:56:24,607][20381] Updated weights for policy 1, policy_version 39781 (0.0007) [2023-12-27 05:56:25,245][20347] Updated weights for policy 0, policy_version 39861 (0.0008) [2023-12-27 05:56:25,267][20381] Updated weights for policy 1, policy_version 39791 (0.0006) [2023-12-27 05:56:25,301][20347] Updated weights for policy 0, policy_version 39871 (0.0006) [2023-12-27 05:56:25,327][20381] Updated weights for policy 1, policy_version 39801 (0.0008) [2023-12-27 05:56:25,351][20347] Updated weights for policy 0, policy_version 39881 (0.0005) [2023-12-27 05:56:25,379][20381] Updated weights for policy 1, policy_version 39811 (0.0007) [2023-12-27 05:56:25,945][20347] Updated weights for policy 0, policy_version 39891 (0.0008) [2023-12-27 05:56:26,006][20347] Updated weights for policy 0, policy_version 39901 (0.0005) [2023-12-27 05:56:26,059][20347] Updated weights for policy 0, policy_version 39911 (0.0009) [2023-12-27 05:56:26,120][20381] Updated weights for policy 1, policy_version 39821 (0.0008) [2023-12-27 05:56:26,169][20381] Updated weights for policy 1, policy_version 39831 (0.0007) [2023-12-27 05:56:26,218][20381] Updated weights for policy 1, policy_version 39841 (0.0008) [2023-12-27 05:56:26,720][20347] Updated weights for policy 0, policy_version 39921 (0.0010) [2023-12-27 05:56:26,776][20347] Updated weights for policy 0, policy_version 39931 (0.0010) [2023-12-27 05:56:26,824][20347] Updated weights for policy 0, policy_version 39941 (0.0010) [2023-12-27 05:56:26,870][20347] Updated weights for policy 0, policy_version 39951 (0.0010) [2023-12-27 05:56:26,896][20381] Updated weights for policy 1, policy_version 39851 (0.0008) [2023-12-27 05:56:26,940][20381] Updated weights for policy 1, policy_version 39861 (0.0007) [2023-12-27 05:56:26,984][20381] Updated weights for policy 1, policy_version 39871 (0.0007) [2023-12-27 05:56:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 20447232. Throughput: 0: 10495.7, 1: 10199.7. Samples: 20437012. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:56:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:27,552][20347] Updated weights for policy 0, policy_version 39961 (0.0010) [2023-12-27 05:56:27,602][20347] Updated weights for policy 0, policy_version 39971 (0.0010) [2023-12-27 05:56:27,614][20381] Updated weights for policy 1, policy_version 39881 (0.0007) [2023-12-27 05:56:27,653][20347] Updated weights for policy 0, policy_version 39981 (0.0010) [2023-12-27 05:56:27,670][20381] Updated weights for policy 1, policy_version 39891 (0.0006) [2023-12-27 05:56:27,724][20381] Updated weights for policy 1, policy_version 39901 (0.0007) [2023-12-27 05:56:27,780][20381] Updated weights for policy 1, policy_version 39911 (0.0007) [2023-12-27 05:56:28,277][20347] Updated weights for policy 0, policy_version 39991 (0.0008) [2023-12-27 05:56:28,340][20347] Updated weights for policy 0, policy_version 40001 (0.0006) [2023-12-27 05:56:28,403][20347] Updated weights for policy 0, policy_version 40011 (0.0007) [2023-12-27 05:56:28,463][20381] Updated weights for policy 1, policy_version 39921 (0.0009) [2023-12-27 05:56:28,521][20381] Updated weights for policy 1, policy_version 39931 (0.0006) [2023-12-27 05:56:28,580][20381] Updated weights for policy 1, policy_version 39941 (0.0010) [2023-12-27 05:56:29,081][20347] Updated weights for policy 0, policy_version 40021 (0.0006) [2023-12-27 05:56:29,137][20347] Updated weights for policy 0, policy_version 40031 (0.0007) [2023-12-27 05:56:29,166][20381] Updated weights for policy 1, policy_version 39951 (0.0009) [2023-12-27 05:56:29,198][20347] Updated weights for policy 0, policy_version 40041 (0.0008) [2023-12-27 05:56:29,230][20381] Updated weights for policy 1, policy_version 39961 (0.0008) [2023-12-27 05:56:29,286][20381] Updated weights for policy 1, policy_version 39971 (0.0006) [2023-12-27 05:56:29,850][20347] Updated weights for policy 0, policy_version 40051 (0.0008) [2023-12-27 05:56:29,912][20347] Updated weights for policy 0, policy_version 40061 (0.0007) [2023-12-27 05:56:29,967][20347] Updated weights for policy 0, policy_version 40071 (0.0006) [2023-12-27 05:56:29,975][20381] Updated weights for policy 1, policy_version 39981 (0.0009) [2023-12-27 05:56:30,031][20381] Updated weights for policy 1, policy_version 39991 (0.0010) [2023-12-27 05:56:30,087][20381] Updated weights for policy 1, policy_version 40001 (0.0011) [2023-12-27 05:56:30,632][20347] Updated weights for policy 0, policy_version 40081 (0.0005) [2023-12-27 05:56:30,683][20347] Updated weights for policy 0, policy_version 40092 (0.0007) [2023-12-27 05:56:30,724][20381] Updated weights for policy 1, policy_version 40011 (0.0009) [2023-12-27 05:56:30,735][20347] Updated weights for policy 0, policy_version 40102 (0.0006) [2023-12-27 05:56:30,785][20381] Updated weights for policy 1, policy_version 40021 (0.0009) [2023-12-27 05:56:30,786][20347] Updated weights for policy 0, policy_version 40112 (0.0006) [2023-12-27 05:56:30,843][20381] Updated weights for policy 1, policy_version 40031 (0.0008) [2023-12-27 05:56:31,441][20347] Updated weights for policy 0, policy_version 40122 (0.0007) [2023-12-27 05:56:31,449][20381] Updated weights for policy 1, policy_version 40041 (0.0007) [2023-12-27 05:56:31,496][20347] Updated weights for policy 0, policy_version 40132 (0.0006) [2023-12-27 05:56:31,503][20381] Updated weights for policy 1, policy_version 40051 (0.0008) [2023-12-27 05:56:31,558][20347] Updated weights for policy 0, policy_version 40142 (0.0008) [2023-12-27 05:56:31,564][20381] Updated weights for policy 1, policy_version 40061 (0.0005) [2023-12-27 05:56:31,617][20381] Updated weights for policy 1, policy_version 40071 (0.0008) [2023-12-27 05:56:32,250][20347] Updated weights for policy 0, policy_version 40152 (0.0008) [2023-12-27 05:56:32,309][20347] Updated weights for policy 0, policy_version 40162 (0.0008) [2023-12-27 05:56:32,319][20381] Updated weights for policy 1, policy_version 40081 (0.0006) [2023-12-27 05:56:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20618.4, 300 sec: 20854.9). Total num frames: 20545536. Throughput: 0: 10493.1, 1: 10268.6. Samples: 20563268. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:56:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:32,370][20347] Updated weights for policy 0, policy_version 40172 (0.0006) [2023-12-27 05:56:32,372][20381] Updated weights for policy 1, policy_version 40091 (0.0007) [2023-12-27 05:56:32,420][20381] Updated weights for policy 1, policy_version 40101 (0.0007) [2023-12-27 05:56:33,032][20347] Updated weights for policy 0, policy_version 40182 (0.0006) [2023-12-27 05:56:33,084][20381] Updated weights for policy 1, policy_version 40111 (0.0009) [2023-12-27 05:56:33,086][20347] Updated weights for policy 0, policy_version 40192 (0.0006) [2023-12-27 05:56:33,134][20381] Updated weights for policy 1, policy_version 40121 (0.0010) [2023-12-27 05:56:33,140][20347] Updated weights for policy 0, policy_version 40202 (0.0006) [2023-12-27 05:56:33,182][20381] Updated weights for policy 1, policy_version 40131 (0.0007) [2023-12-27 05:56:33,798][20381] Updated weights for policy 1, policy_version 40141 (0.0007) [2023-12-27 05:56:33,829][20347] Updated weights for policy 0, policy_version 40212 (0.0007) [2023-12-27 05:56:33,856][20381] Updated weights for policy 1, policy_version 40151 (0.0005) [2023-12-27 05:56:33,886][20347] Updated weights for policy 0, policy_version 40222 (0.0007) [2023-12-27 05:56:33,911][20381] Updated weights for policy 1, policy_version 40161 (0.0006) [2023-12-27 05:56:33,937][20347] Updated weights for policy 0, policy_version 40232 (0.0009) [2023-12-27 05:56:34,488][20381] Updated weights for policy 1, policy_version 40171 (0.0007) [2023-12-27 05:56:34,545][20381] Updated weights for policy 1, policy_version 40181 (0.0008) [2023-12-27 05:56:34,609][20381] Updated weights for policy 1, policy_version 40191 (0.0007) [2023-12-27 05:56:34,681][20347] Updated weights for policy 0, policy_version 40242 (0.0006) [2023-12-27 05:56:34,735][20347] Updated weights for policy 0, policy_version 40252 (0.0008) [2023-12-27 05:56:34,788][20347] Updated weights for policy 0, policy_version 40262 (0.0005) [2023-12-27 05:56:34,837][20347] Updated weights for policy 0, policy_version 40272 (0.0005) [2023-12-27 05:56:35,295][20381] Updated weights for policy 1, policy_version 40201 (0.0007) [2023-12-27 05:56:35,346][20381] Updated weights for policy 1, policy_version 40211 (0.0008) [2023-12-27 05:56:35,398][20381] Updated weights for policy 1, policy_version 40221 (0.0009) [2023-12-27 05:56:35,452][20381] Updated weights for policy 1, policy_version 40231 (0.0007) [2023-12-27 05:56:35,474][20347] Updated weights for policy 0, policy_version 40282 (0.0009) [2023-12-27 05:56:35,529][20347] Updated weights for policy 0, policy_version 40292 (0.0010) [2023-12-27 05:56:35,593][20347] Updated weights for policy 0, policy_version 40302 (0.0010) [2023-12-27 05:56:36,060][20381] Updated weights for policy 1, policy_version 40241 (0.0005) [2023-12-27 05:56:36,117][20381] Updated weights for policy 1, policy_version 40251 (0.0006) [2023-12-27 05:56:36,173][20381] Updated weights for policy 1, policy_version 40261 (0.0008) [2023-12-27 05:56:36,229][20347] Updated weights for policy 0, policy_version 40312 (0.0011) [2023-12-27 05:56:36,292][20347] Updated weights for policy 0, policy_version 40322 (0.0011) [2023-12-27 05:56:36,342][20347] Updated weights for policy 0, policy_version 40332 (0.0010) [2023-12-27 05:56:36,828][20381] Updated weights for policy 1, policy_version 40271 (0.0009) [2023-12-27 05:56:36,882][20381] Updated weights for policy 1, policy_version 40281 (0.0010) [2023-12-27 05:56:36,939][20381] Updated weights for policy 1, policy_version 40291 (0.0009) [2023-12-27 05:56:37,055][20347] Updated weights for policy 0, policy_version 40342 (0.0010) [2023-12-27 05:56:37,109][20347] Updated weights for policy 0, policy_version 40352 (0.0010) [2023-12-27 05:56:37,163][20347] Updated weights for policy 0, policy_version 40362 (0.0010) [2023-12-27 05:56:37,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 20660224. Throughput: 0: 10467.2, 1: 10312.2. Samples: 20626084. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:56:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000040296_10321920.pth... [2023-12-27 05:56:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000040368_10338304.pth... [2023-12-27 05:56:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000039080_10010624.pth [2023-12-27 05:56:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000039152_10027008.pth [2023-12-27 05:56:37,595][20381] Updated weights for policy 1, policy_version 40301 (0.0009) [2023-12-27 05:56:37,675][20381] Updated weights for policy 1, policy_version 40311 (0.0010) [2023-12-27 05:56:37,731][20381] Updated weights for policy 1, policy_version 40321 (0.0009) [2023-12-27 05:56:37,752][20347] Updated weights for policy 0, policy_version 40372 (0.0008) [2023-12-27 05:56:37,809][20347] Updated weights for policy 0, policy_version 40382 (0.0005) [2023-12-27 05:56:37,861][20347] Updated weights for policy 0, policy_version 40392 (0.0006) [2023-12-27 05:56:38,366][20381] Updated weights for policy 1, policy_version 40331 (0.0006) [2023-12-27 05:56:38,423][20381] Updated weights for policy 1, policy_version 40341 (0.0007) [2023-12-27 05:56:38,480][20381] Updated weights for policy 1, policy_version 40351 (0.0008) [2023-12-27 05:56:38,521][20347] Updated weights for policy 0, policy_version 40402 (0.0009) [2023-12-27 05:56:38,576][20347] Updated weights for policy 0, policy_version 40412 (0.0007) [2023-12-27 05:56:38,631][20347] Updated weights for policy 0, policy_version 40422 (0.0007) [2023-12-27 05:56:38,679][20347] Updated weights for policy 0, policy_version 40432 (0.0008) [2023-12-27 05:56:39,103][20381] Updated weights for policy 1, policy_version 40361 (0.0007) [2023-12-27 05:56:39,157][20381] Updated weights for policy 1, policy_version 40371 (0.0007) [2023-12-27 05:56:39,223][20381] Updated weights for policy 1, policy_version 40381 (0.0006) [2023-12-27 05:56:39,272][20381] Updated weights for policy 1, policy_version 40391 (0.0007) [2023-12-27 05:56:39,380][20347] Updated weights for policy 0, policy_version 40442 (0.0011) [2023-12-27 05:56:39,447][20347] Updated weights for policy 0, policy_version 40452 (0.0009) [2023-12-27 05:56:39,512][20347] Updated weights for policy 0, policy_version 40462 (0.0008) [2023-12-27 05:56:39,952][20381] Updated weights for policy 1, policy_version 40401 (0.0007) [2023-12-27 05:56:40,000][20381] Updated weights for policy 1, policy_version 40411 (0.0007) [2023-12-27 05:56:40,060][20381] Updated weights for policy 1, policy_version 40421 (0.0008) [2023-12-27 05:56:40,161][20347] Updated weights for policy 0, policy_version 40472 (0.0010) [2023-12-27 05:56:40,210][20347] Updated weights for policy 0, policy_version 40482 (0.0010) [2023-12-27 05:56:40,263][20347] Updated weights for policy 0, policy_version 40492 (0.0010) [2023-12-27 05:56:40,769][20381] Updated weights for policy 1, policy_version 40431 (0.0007) [2023-12-27 05:56:40,820][20381] Updated weights for policy 1, policy_version 40441 (0.0007) [2023-12-27 05:56:40,867][20381] Updated weights for policy 1, policy_version 40451 (0.0007) [2023-12-27 05:56:40,954][20347] Updated weights for policy 0, policy_version 40502 (0.0010) [2023-12-27 05:56:41,003][20347] Updated weights for policy 0, policy_version 40512 (0.0010) [2023-12-27 05:56:41,066][20347] Updated weights for policy 0, policy_version 40522 (0.0010) [2023-12-27 05:56:41,595][20381] Updated weights for policy 1, policy_version 40461 (0.0006) [2023-12-27 05:56:41,659][20381] Updated weights for policy 1, policy_version 40471 (0.0008) [2023-12-27 05:56:41,716][20381] Updated weights for policy 1, policy_version 40481 (0.0007) [2023-12-27 05:56:41,813][20347] Updated weights for policy 0, policy_version 40532 (0.0011) [2023-12-27 05:56:41,861][20347] Updated weights for policy 0, policy_version 40542 (0.0009) [2023-12-27 05:56:41,914][20347] Updated weights for policy 0, policy_version 40552 (0.0007) [2023-12-27 05:56:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 20758528. Throughput: 0: 10441.2, 1: 10372.1. Samples: 20751252. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:56:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:42,388][20381] Updated weights for policy 1, policy_version 40491 (0.0008) [2023-12-27 05:56:42,445][20381] Updated weights for policy 1, policy_version 40501 (0.0008) [2023-12-27 05:56:42,507][20381] Updated weights for policy 1, policy_version 40511 (0.0008) [2023-12-27 05:56:42,588][20347] Updated weights for policy 0, policy_version 40562 (0.0005) [2023-12-27 05:56:42,647][20347] Updated weights for policy 0, policy_version 40572 (0.0006) [2023-12-27 05:56:42,703][20347] Updated weights for policy 0, policy_version 40582 (0.0006) [2023-12-27 05:56:42,753][20347] Updated weights for policy 0, policy_version 40592 (0.0008) [2023-12-27 05:56:43,254][20381] Updated weights for policy 1, policy_version 40521 (0.0008) [2023-12-27 05:56:43,307][20381] Updated weights for policy 1, policy_version 40531 (0.0010) [2023-12-27 05:56:43,356][20381] Updated weights for policy 1, policy_version 40541 (0.0010) [2023-12-27 05:56:43,378][20347] Updated weights for policy 0, policy_version 40602 (0.0006) [2023-12-27 05:56:43,411][20381] Updated weights for policy 1, policy_version 40551 (0.0010) [2023-12-27 05:56:43,434][20347] Updated weights for policy 0, policy_version 40612 (0.0007) [2023-12-27 05:56:43,488][20347] Updated weights for policy 0, policy_version 40622 (0.0007) [2023-12-27 05:56:44,119][20347] Updated weights for policy 0, policy_version 40632 (0.0005) [2023-12-27 05:56:44,124][20381] Updated weights for policy 1, policy_version 40561 (0.0011) [2023-12-27 05:56:44,174][20347] Updated weights for policy 0, policy_version 40642 (0.0006) [2023-12-27 05:56:44,181][20381] Updated weights for policy 1, policy_version 40571 (0.0009) [2023-12-27 05:56:44,234][20347] Updated weights for policy 0, policy_version 40652 (0.0007) [2023-12-27 05:56:44,239][20381] Updated weights for policy 1, policy_version 40581 (0.0007) [2023-12-27 05:56:44,876][20381] Updated weights for policy 1, policy_version 40591 (0.0007) [2023-12-27 05:56:44,883][20347] Updated weights for policy 0, policy_version 40662 (0.0007) [2023-12-27 05:56:44,934][20347] Updated weights for policy 0, policy_version 40672 (0.0007) [2023-12-27 05:56:44,941][20381] Updated weights for policy 1, policy_version 40601 (0.0008) [2023-12-27 05:56:44,983][20347] Updated weights for policy 0, policy_version 40682 (0.0007) [2023-12-27 05:56:44,997][20381] Updated weights for policy 1, policy_version 40611 (0.0006) [2023-12-27 05:56:45,668][20381] Updated weights for policy 1, policy_version 40621 (0.0006) [2023-12-27 05:56:45,719][20381] Updated weights for policy 1, policy_version 40631 (0.0007) [2023-12-27 05:56:45,720][20347] Updated weights for policy 0, policy_version 40692 (0.0006) [2023-12-27 05:56:45,772][20347] Updated weights for policy 0, policy_version 40702 (0.0005) [2023-12-27 05:56:45,777][20381] Updated weights for policy 1, policy_version 40641 (0.0006) [2023-12-27 05:56:45,824][20347] Updated weights for policy 0, policy_version 40712 (0.0007) [2023-12-27 05:56:46,456][20381] Updated weights for policy 1, policy_version 40651 (0.0007) [2023-12-27 05:56:46,511][20381] Updated weights for policy 1, policy_version 40661 (0.0007) [2023-12-27 05:56:46,538][20347] Updated weights for policy 0, policy_version 40722 (0.0007) [2023-12-27 05:56:46,555][20381] Updated weights for policy 1, policy_version 40671 (0.0007) [2023-12-27 05:56:46,595][20347] Updated weights for policy 0, policy_version 40732 (0.0006) [2023-12-27 05:56:46,649][20347] Updated weights for policy 0, policy_version 40742 (0.0007) [2023-12-27 05:56:46,697][20347] Updated weights for policy 0, policy_version 40752 (0.0007) [2023-12-27 05:56:47,223][20381] Updated weights for policy 1, policy_version 40681 (0.0007) [2023-12-27 05:56:47,278][20381] Updated weights for policy 1, policy_version 40691 (0.0007) [2023-12-27 05:56:47,335][20381] Updated weights for policy 1, policy_version 40701 (0.0008) [2023-12-27 05:56:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 20856832. Throughput: 0: 10408.7, 1: 10379.9. Samples: 20873456. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 05:56:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:47,374][20347] Updated weights for policy 0, policy_version 40762 (0.0006) [2023-12-27 05:56:47,386][20381] Updated weights for policy 1, policy_version 40711 (0.0006) [2023-12-27 05:56:47,430][20347] Updated weights for policy 0, policy_version 40772 (0.0007) [2023-12-27 05:56:47,485][20347] Updated weights for policy 0, policy_version 40782 (0.0007) [2023-12-27 05:56:48,048][20381] Updated weights for policy 1, policy_version 40721 (0.0008) [2023-12-27 05:56:48,095][20381] Updated weights for policy 1, policy_version 40731 (0.0007) [2023-12-27 05:56:48,123][20347] Updated weights for policy 0, policy_version 40792 (0.0006) [2023-12-27 05:56:48,144][20381] Updated weights for policy 1, policy_version 40741 (0.0005) [2023-12-27 05:56:48,179][20347] Updated weights for policy 0, policy_version 40802 (0.0006) [2023-12-27 05:56:48,237][20347] Updated weights for policy 0, policy_version 40812 (0.0005) [2023-12-27 05:56:48,852][20381] Updated weights for policy 1, policy_version 40751 (0.0008) [2023-12-27 05:56:48,888][20347] Updated weights for policy 0, policy_version 40822 (0.0005) [2023-12-27 05:56:48,915][20381] Updated weights for policy 1, policy_version 40761 (0.0008) [2023-12-27 05:56:48,947][20347] Updated weights for policy 0, policy_version 40832 (0.0005) [2023-12-27 05:56:48,970][20381] Updated weights for policy 1, policy_version 40771 (0.0007) [2023-12-27 05:56:49,004][20347] Updated weights for policy 0, policy_version 40842 (0.0007) [2023-12-27 05:56:49,590][20381] Updated weights for policy 1, policy_version 40781 (0.0008) [2023-12-27 05:56:49,649][20381] Updated weights for policy 1, policy_version 40791 (0.0011) [2023-12-27 05:56:49,677][20347] Updated weights for policy 0, policy_version 40852 (0.0008) [2023-12-27 05:56:49,708][20381] Updated weights for policy 1, policy_version 40801 (0.0011) [2023-12-27 05:56:49,735][20347] Updated weights for policy 0, policy_version 40862 (0.0005) [2023-12-27 05:56:49,790][20347] Updated weights for policy 0, policy_version 40872 (0.0006) [2023-12-27 05:56:50,333][20381] Updated weights for policy 1, policy_version 40811 (0.0010) [2023-12-27 05:56:50,386][20381] Updated weights for policy 1, policy_version 40821 (0.0005) [2023-12-27 05:56:50,435][20381] Updated weights for policy 1, policy_version 40831 (0.0008) [2023-12-27 05:56:50,456][20347] Updated weights for policy 0, policy_version 40882 (0.0007) [2023-12-27 05:56:50,499][20347] Updated weights for policy 0, policy_version 40892 (0.0008) [2023-12-27 05:56:50,549][20347] Updated weights for policy 0, policy_version 40902 (0.0007) [2023-12-27 05:56:50,603][20347] Updated weights for policy 0, policy_version 40912 (0.0007) [2023-12-27 05:56:51,093][20381] Updated weights for policy 1, policy_version 40841 (0.0010) [2023-12-27 05:56:51,156][20381] Updated weights for policy 1, policy_version 40851 (0.0008) [2023-12-27 05:56:51,215][20381] Updated weights for policy 1, policy_version 40861 (0.0010) [2023-12-27 05:56:51,274][20381] Updated weights for policy 1, policy_version 40871 (0.0011) [2023-12-27 05:56:51,285][20347] Updated weights for policy 0, policy_version 40922 (0.0006) [2023-12-27 05:56:51,346][20347] Updated weights for policy 0, policy_version 40932 (0.0007) [2023-12-27 05:56:51,402][20347] Updated weights for policy 0, policy_version 40942 (0.0006) [2023-12-27 05:56:51,940][20381] Updated weights for policy 1, policy_version 40881 (0.0008) [2023-12-27 05:56:51,995][20381] Updated weights for policy 1, policy_version 40891 (0.0009) [2023-12-27 05:56:52,043][20347] Updated weights for policy 0, policy_version 40952 (0.0006) [2023-12-27 05:56:52,049][20381] Updated weights for policy 1, policy_version 40901 (0.0008) [2023-12-27 05:56:52,095][20347] Updated weights for policy 0, policy_version 40962 (0.0005) [2023-12-27 05:56:52,143][20347] Updated weights for policy 0, policy_version 40972 (0.0005) [2023-12-27 05:56:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 20971520. Throughput: 0: 10399.6, 1: 10368.2. Samples: 20936408. Policy #0 lag: (min: 0.0, avg: 27.5, max: 32.0) [2023-12-27 05:56:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000040904_10477568.pth... [2023-12-27 05:56:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000040976_10493952.pth... [2023-12-27 05:56:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000039760_10182656.pth [2023-12-27 05:56:52,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000039688_10166272.pth [2023-12-27 05:56:52,745][20381] Updated weights for policy 1, policy_version 40911 (0.0010) [2023-12-27 05:56:52,766][20347] Updated weights for policy 0, policy_version 40982 (0.0007) [2023-12-27 05:56:52,804][20381] Updated weights for policy 1, policy_version 40921 (0.0011) [2023-12-27 05:56:52,813][20347] Updated weights for policy 0, policy_version 40992 (0.0008) [2023-12-27 05:56:52,859][20347] Updated weights for policy 0, policy_version 41002 (0.0007) [2023-12-27 05:56:52,863][20381] Updated weights for policy 1, policy_version 40931 (0.0009) [2023-12-27 05:56:53,491][20381] Updated weights for policy 1, policy_version 40941 (0.0006) [2023-12-27 05:56:53,548][20381] Updated weights for policy 1, policy_version 40951 (0.0006) [2023-12-27 05:56:53,567][20347] Updated weights for policy 0, policy_version 41012 (0.0007) [2023-12-27 05:56:53,606][20381] Updated weights for policy 1, policy_version 40961 (0.0006) [2023-12-27 05:56:53,622][20347] Updated weights for policy 0, policy_version 41022 (0.0007) [2023-12-27 05:56:53,676][20347] Updated weights for policy 0, policy_version 41032 (0.0007) [2023-12-27 05:56:54,206][20381] Updated weights for policy 1, policy_version 40971 (0.0006) [2023-12-27 05:56:54,263][20381] Updated weights for policy 1, policy_version 40981 (0.0007) [2023-12-27 05:56:54,318][20381] Updated weights for policy 1, policy_version 40991 (0.0008) [2023-12-27 05:56:54,358][20347] Updated weights for policy 0, policy_version 41042 (0.0007) [2023-12-27 05:56:54,414][20347] Updated weights for policy 0, policy_version 41052 (0.0006) [2023-12-27 05:56:54,473][20347] Updated weights for policy 0, policy_version 41062 (0.0006) [2023-12-27 05:56:54,530][20347] Updated weights for policy 0, policy_version 41072 (0.0007) [2023-12-27 05:56:54,958][20381] Updated weights for policy 1, policy_version 41001 (0.0008) [2023-12-27 05:56:55,017][20381] Updated weights for policy 1, policy_version 41011 (0.0006) [2023-12-27 05:56:55,073][20381] Updated weights for policy 1, policy_version 41021 (0.0005) [2023-12-27 05:56:55,123][20347] Updated weights for policy 0, policy_version 41082 (0.0007) [2023-12-27 05:56:55,130][20381] Updated weights for policy 1, policy_version 41031 (0.0005) [2023-12-27 05:56:55,177][20347] Updated weights for policy 0, policy_version 41092 (0.0006) [2023-12-27 05:56:55,229][20347] Updated weights for policy 0, policy_version 41102 (0.0008) [2023-12-27 05:56:55,812][20381] Updated weights for policy 1, policy_version 41041 (0.0008) [2023-12-27 05:56:55,860][20381] Updated weights for policy 1, policy_version 41051 (0.0008) [2023-12-27 05:56:55,907][20381] Updated weights for policy 1, policy_version 41061 (0.0006) [2023-12-27 05:56:55,916][20347] Updated weights for policy 0, policy_version 41112 (0.0006) [2023-12-27 05:56:55,973][20347] Updated weights for policy 0, policy_version 41122 (0.0008) [2023-12-27 05:56:56,023][20347] Updated weights for policy 0, policy_version 41132 (0.0009) [2023-12-27 05:56:56,541][20381] Updated weights for policy 1, policy_version 41071 (0.0005) [2023-12-27 05:56:56,601][20381] Updated weights for policy 1, policy_version 41081 (0.0007) [2023-12-27 05:56:56,655][20381] Updated weights for policy 1, policy_version 41091 (0.0007) [2023-12-27 05:56:56,675][20347] Updated weights for policy 0, policy_version 41142 (0.0007) [2023-12-27 05:56:56,752][20347] Updated weights for policy 0, policy_version 41152 (0.0007) [2023-12-27 05:56:56,809][20347] Updated weights for policy 0, policy_version 41162 (0.0007) [2023-12-27 05:56:57,320][20381] Updated weights for policy 1, policy_version 41101 (0.0006) [2023-12-27 05:56:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 21069824. Throughput: 0: 10378.1, 1: 10451.7. Samples: 21063052. Policy #0 lag: (min: 0.0, avg: 27.5, max: 32.0) [2023-12-27 05:56:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:56:57,375][20381] Updated weights for policy 1, policy_version 41111 (0.0006) [2023-12-27 05:56:57,418][20381] Updated weights for policy 1, policy_version 41121 (0.0005) [2023-12-27 05:56:57,433][20347] Updated weights for policy 0, policy_version 41172 (0.0006) [2023-12-27 05:56:57,493][20347] Updated weights for policy 0, policy_version 41182 (0.0005) [2023-12-27 05:56:57,549][20347] Updated weights for policy 0, policy_version 41192 (0.0007) [2023-12-27 05:56:58,092][20381] Updated weights for policy 1, policy_version 41131 (0.0006) [2023-12-27 05:56:58,137][20381] Updated weights for policy 1, policy_version 41141 (0.0008) [2023-12-27 05:56:58,168][20347] Updated weights for policy 0, policy_version 41202 (0.0006) [2023-12-27 05:56:58,203][20381] Updated weights for policy 1, policy_version 41151 (0.0010) [2023-12-27 05:56:58,231][20347] Updated weights for policy 0, policy_version 41212 (0.0008) [2023-12-27 05:56:58,292][20347] Updated weights for policy 0, policy_version 41222 (0.0008) [2023-12-27 05:56:58,360][20347] Updated weights for policy 0, policy_version 41232 (0.0008) [2023-12-27 05:56:59,008][20381] Updated weights for policy 1, policy_version 41161 (0.0011) [2023-12-27 05:56:59,068][20381] Updated weights for policy 1, policy_version 41171 (0.0010) [2023-12-27 05:56:59,088][20347] Updated weights for policy 0, policy_version 41242 (0.0005) [2023-12-27 05:56:59,128][20381] Updated weights for policy 1, policy_version 41181 (0.0010) [2023-12-27 05:56:59,151][20347] Updated weights for policy 0, policy_version 41252 (0.0006) [2023-12-27 05:56:59,190][20381] Updated weights for policy 1, policy_version 41191 (0.0010) [2023-12-27 05:56:59,211][20347] Updated weights for policy 0, policy_version 41262 (0.0006) [2023-12-27 05:56:59,866][20381] Updated weights for policy 1, policy_version 41201 (0.0010) [2023-12-27 05:56:59,924][20381] Updated weights for policy 1, policy_version 41211 (0.0010) [2023-12-27 05:56:59,973][20347] Updated weights for policy 0, policy_version 41272 (0.0007) [2023-12-27 05:56:59,985][20381] Updated weights for policy 1, policy_version 41221 (0.0010) [2023-12-27 05:57:00,028][20347] Updated weights for policy 0, policy_version 41282 (0.0008) [2023-12-27 05:57:00,082][20347] Updated weights for policy 0, policy_version 41292 (0.0007) [2023-12-27 05:57:00,683][20381] Updated weights for policy 1, policy_version 41231 (0.0010) [2023-12-27 05:57:00,742][20381] Updated weights for policy 1, policy_version 41241 (0.0010) [2023-12-27 05:57:00,790][20347] Updated weights for policy 0, policy_version 41302 (0.0006) [2023-12-27 05:57:00,792][20381] Updated weights for policy 1, policy_version 41251 (0.0010) [2023-12-27 05:57:00,841][20347] Updated weights for policy 0, policy_version 41312 (0.0006) [2023-12-27 05:57:00,893][20347] Updated weights for policy 0, policy_version 41322 (0.0007) [2023-12-27 05:57:01,466][20381] Updated weights for policy 1, policy_version 41261 (0.0010) [2023-12-27 05:57:01,530][20381] Updated weights for policy 1, policy_version 41271 (0.0010) [2023-12-27 05:57:01,581][20381] Updated weights for policy 1, policy_version 41281 (0.0010) [2023-12-27 05:57:01,611][20347] Updated weights for policy 0, policy_version 41332 (0.0007) [2023-12-27 05:57:01,669][20347] Updated weights for policy 0, policy_version 41342 (0.0007) [2023-12-27 05:57:01,729][20347] Updated weights for policy 0, policy_version 41352 (0.0008) [2023-12-27 05:57:02,259][20381] Updated weights for policy 1, policy_version 41291 (0.0009) [2023-12-27 05:57:02,313][20381] Updated weights for policy 1, policy_version 41301 (0.0010) [2023-12-27 05:57:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 21168128. Throughput: 0: 10374.6, 1: 10379.2. Samples: 21183628. Policy #0 lag: (min: 0.0, avg: 27.5, max: 32.0) [2023-12-27 05:57:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:02,374][20381] Updated weights for policy 1, policy_version 41311 (0.0010) [2023-12-27 05:57:02,436][20347] Updated weights for policy 0, policy_version 41362 (0.0007) [2023-12-27 05:57:02,495][20347] Updated weights for policy 0, policy_version 41372 (0.0007) [2023-12-27 05:57:02,551][20347] Updated weights for policy 0, policy_version 41382 (0.0007) [2023-12-27 05:57:02,604][20347] Updated weights for policy 0, policy_version 41392 (0.0007) [2023-12-27 05:57:03,068][20381] Updated weights for policy 1, policy_version 41321 (0.0010) [2023-12-27 05:57:03,119][20381] Updated weights for policy 1, policy_version 41331 (0.0010) [2023-12-27 05:57:03,179][20381] Updated weights for policy 1, policy_version 41341 (0.0010) [2023-12-27 05:57:03,233][20381] Updated weights for policy 1, policy_version 41351 (0.0010) [2023-12-27 05:57:03,292][20347] Updated weights for policy 0, policy_version 41402 (0.0007) [2023-12-27 05:57:03,345][20347] Updated weights for policy 0, policy_version 41412 (0.0007) [2023-12-27 05:57:03,388][20347] Updated weights for policy 0, policy_version 41422 (0.0007) [2023-12-27 05:57:03,868][20381] Updated weights for policy 1, policy_version 41361 (0.0008) [2023-12-27 05:57:03,924][20381] Updated weights for policy 1, policy_version 41371 (0.0008) [2023-12-27 05:57:03,980][20381] Updated weights for policy 1, policy_version 41381 (0.0007) [2023-12-27 05:57:04,082][20347] Updated weights for policy 0, policy_version 41432 (0.0008) [2023-12-27 05:57:04,146][20347] Updated weights for policy 0, policy_version 41442 (0.0008) [2023-12-27 05:57:04,200][20347] Updated weights for policy 0, policy_version 41452 (0.0009) [2023-12-27 05:57:04,604][20381] Updated weights for policy 1, policy_version 41391 (0.0007) [2023-12-27 05:57:04,656][20381] Updated weights for policy 1, policy_version 41401 (0.0007) [2023-12-27 05:57:04,708][20381] Updated weights for policy 1, policy_version 41411 (0.0007) [2023-12-27 05:57:04,902][20347] Updated weights for policy 0, policy_version 41462 (0.0009) [2023-12-27 05:57:04,954][20347] Updated weights for policy 0, policy_version 41472 (0.0008) [2023-12-27 05:57:05,003][20347] Updated weights for policy 0, policy_version 41482 (0.0009) [2023-12-27 05:57:05,312][20381] Updated weights for policy 1, policy_version 41421 (0.0008) [2023-12-27 05:57:05,365][20381] Updated weights for policy 1, policy_version 41431 (0.0008) [2023-12-27 05:57:05,411][20381] Updated weights for policy 1, policy_version 41441 (0.0008) [2023-12-27 05:57:05,729][20347] Updated weights for policy 0, policy_version 41492 (0.0009) [2023-12-27 05:57:05,777][20347] Updated weights for policy 0, policy_version 41502 (0.0008) [2023-12-27 05:57:05,837][20347] Updated weights for policy 0, policy_version 41512 (0.0007) [2023-12-27 05:57:06,124][20381] Updated weights for policy 1, policy_version 41451 (0.0008) [2023-12-27 05:57:06,177][20381] Updated weights for policy 1, policy_version 41461 (0.0007) [2023-12-27 05:57:06,231][20381] Updated weights for policy 1, policy_version 41471 (0.0007) [2023-12-27 05:57:06,558][20347] Updated weights for policy 0, policy_version 41522 (0.0007) [2023-12-27 05:57:06,618][20347] Updated weights for policy 0, policy_version 41532 (0.0007) [2023-12-27 05:57:06,677][20347] Updated weights for policy 0, policy_version 41542 (0.0007) [2023-12-27 05:57:06,738][20347] Updated weights for policy 0, policy_version 41552 (0.0008) [2023-12-27 05:57:06,924][20381] Updated weights for policy 1, policy_version 41481 (0.0006) [2023-12-27 05:57:06,988][20381] Updated weights for policy 1, policy_version 41491 (0.0008) [2023-12-27 05:57:07,052][20381] Updated weights for policy 1, policy_version 41501 (0.0008) [2023-12-27 05:57:07,121][20381] Updated weights for policy 1, policy_version 41511 (0.0008) [2023-12-27 05:57:07,362][20347] Updated weights for policy 0, policy_version 41562 (0.0008) [2023-12-27 05:57:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20854.9). Total num frames: 21274624. Throughput: 0: 10323.4, 1: 10415.6. Samples: 21245920. Policy #0 lag: (min: 0.0, avg: 27.5, max: 32.0) [2023-12-27 05:57:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000041512_10633216.pth... [2023-12-27 05:57:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000040296_10321920.pth [2023-12-27 05:57:07,424][20347] Updated weights for policy 0, policy_version 41572 (0.0008) [2023-12-27 05:57:07,482][20347] Updated weights for policy 0, policy_version 41582 (0.0008) [2023-12-27 05:57:07,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000041584_10649600.pth... [2023-12-27 05:57:07,493][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000040368_10338304.pth [2023-12-27 05:57:07,829][20381] Updated weights for policy 1, policy_version 41521 (0.0008) [2023-12-27 05:57:07,889][20381] Updated weights for policy 1, policy_version 41531 (0.0010) [2023-12-27 05:57:07,945][20381] Updated weights for policy 1, policy_version 41541 (0.0010) [2023-12-27 05:57:08,203][20347] Updated weights for policy 0, policy_version 41592 (0.0008) [2023-12-27 05:57:08,251][20347] Updated weights for policy 0, policy_version 41602 (0.0008) [2023-12-27 05:57:08,297][20347] Updated weights for policy 0, policy_version 41612 (0.0007) [2023-12-27 05:57:08,544][20381] Updated weights for policy 1, policy_version 41551 (0.0009) [2023-12-27 05:57:08,594][20381] Updated weights for policy 1, policy_version 41561 (0.0006) [2023-12-27 05:57:08,649][20381] Updated weights for policy 1, policy_version 41571 (0.0006) [2023-12-27 05:57:09,037][20347] Updated weights for policy 0, policy_version 41622 (0.0009) [2023-12-27 05:57:09,080][20347] Updated weights for policy 0, policy_version 41632 (0.0008) [2023-12-27 05:57:09,139][20347] Updated weights for policy 0, policy_version 41642 (0.0007) [2023-12-27 05:57:09,267][20381] Updated weights for policy 1, policy_version 41581 (0.0006) [2023-12-27 05:57:09,327][20381] Updated weights for policy 1, policy_version 41591 (0.0006) [2023-12-27 05:57:09,388][20381] Updated weights for policy 1, policy_version 41601 (0.0008) [2023-12-27 05:57:09,947][20347] Updated weights for policy 0, policy_version 41652 (0.0007) [2023-12-27 05:57:09,987][20381] Updated weights for policy 1, policy_version 41611 (0.0009) [2023-12-27 05:57:09,997][20347] Updated weights for policy 0, policy_version 41662 (0.0005) [2023-12-27 05:57:10,045][20381] Updated weights for policy 1, policy_version 41621 (0.0011) [2023-12-27 05:57:10,057][20347] Updated weights for policy 0, policy_version 41672 (0.0006) [2023-12-27 05:57:10,105][20381] Updated weights for policy 1, policy_version 41631 (0.0011) [2023-12-27 05:57:10,711][20347] Updated weights for policy 0, policy_version 41682 (0.0006) [2023-12-27 05:57:10,770][20347] Updated weights for policy 0, policy_version 41692 (0.0008) [2023-12-27 05:57:10,817][20381] Updated weights for policy 1, policy_version 41641 (0.0011) [2023-12-27 05:57:10,826][20347] Updated weights for policy 0, policy_version 41702 (0.0007) [2023-12-27 05:57:10,876][20381] Updated weights for policy 1, policy_version 41651 (0.0010) [2023-12-27 05:57:10,881][20347] Updated weights for policy 0, policy_version 41712 (0.0005) [2023-12-27 05:57:10,931][20381] Updated weights for policy 1, policy_version 41661 (0.0010) [2023-12-27 05:57:10,992][20381] Updated weights for policy 1, policy_version 41671 (0.0010) [2023-12-27 05:57:11,660][20381] Updated weights for policy 1, policy_version 41681 (0.0008) [2023-12-27 05:57:11,669][20347] Updated weights for policy 0, policy_version 41722 (0.0007) [2023-12-27 05:57:11,728][20381] Updated weights for policy 1, policy_version 41691 (0.0008) [2023-12-27 05:57:11,739][20347] Updated weights for policy 0, policy_version 41732 (0.0008) [2023-12-27 05:57:11,787][20381] Updated weights for policy 1, policy_version 41701 (0.0008) [2023-12-27 05:57:11,808][20347] Updated weights for policy 0, policy_version 41742 (0.0008) [2023-12-27 05:57:12,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 21372928. Throughput: 0: 10245.2, 1: 10433.3. Samples: 21367548. Policy #0 lag: (min: 0.0, avg: 27.5, max: 32.0) [2023-12-27 05:57:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:12,408][20347] Updated weights for policy 0, policy_version 41752 (0.0007) [2023-12-27 05:57:12,476][20347] Updated weights for policy 0, policy_version 41762 (0.0005) [2023-12-27 05:57:12,543][20381] Updated weights for policy 1, policy_version 41711 (0.0008) [2023-12-27 05:57:12,545][20347] Updated weights for policy 0, policy_version 41772 (0.0005) [2023-12-27 05:57:12,595][20381] Updated weights for policy 1, policy_version 41721 (0.0008) [2023-12-27 05:57:12,643][20381] Updated weights for policy 1, policy_version 41731 (0.0008) [2023-12-27 05:57:13,109][20347] Updated weights for policy 0, policy_version 41782 (0.0008) [2023-12-27 05:57:13,156][20347] Updated weights for policy 0, policy_version 41792 (0.0010) [2023-12-27 05:57:13,204][20347] Updated weights for policy 0, policy_version 41802 (0.0010) [2023-12-27 05:57:13,327][20381] Updated weights for policy 1, policy_version 41741 (0.0007) [2023-12-27 05:57:13,371][20381] Updated weights for policy 1, policy_version 41751 (0.0005) [2023-12-27 05:57:13,425][20381] Updated weights for policy 1, policy_version 41761 (0.0005) [2023-12-27 05:57:13,887][20347] Updated weights for policy 0, policy_version 41812 (0.0008) [2023-12-27 05:57:13,951][20347] Updated weights for policy 0, policy_version 41822 (0.0007) [2023-12-27 05:57:14,005][20347] Updated weights for policy 0, policy_version 41832 (0.0007) [2023-12-27 05:57:14,106][20381] Updated weights for policy 1, policy_version 41771 (0.0007) [2023-12-27 05:57:14,164][20381] Updated weights for policy 1, policy_version 41781 (0.0008) [2023-12-27 05:57:14,220][20381] Updated weights for policy 1, policy_version 41791 (0.0007) [2023-12-27 05:57:14,712][20347] Updated weights for policy 0, policy_version 41842 (0.0007) [2023-12-27 05:57:14,767][20347] Updated weights for policy 0, policy_version 41852 (0.0008) [2023-12-27 05:57:14,830][20347] Updated weights for policy 0, policy_version 41862 (0.0008) [2023-12-27 05:57:14,894][20347] Updated weights for policy 0, policy_version 41872 (0.0008) [2023-12-27 05:57:14,909][20381] Updated weights for policy 1, policy_version 41801 (0.0006) [2023-12-27 05:57:14,972][20381] Updated weights for policy 1, policy_version 41811 (0.0008) [2023-12-27 05:57:15,040][20381] Updated weights for policy 1, policy_version 41821 (0.0008) [2023-12-27 05:57:15,109][20381] Updated weights for policy 1, policy_version 41831 (0.0007) [2023-12-27 05:57:15,541][20347] Updated weights for policy 0, policy_version 41882 (0.0010) [2023-12-27 05:57:15,600][20347] Updated weights for policy 0, policy_version 41892 (0.0010) [2023-12-27 05:57:15,660][20347] Updated weights for policy 0, policy_version 41902 (0.0010) [2023-12-27 05:57:15,815][20381] Updated weights for policy 1, policy_version 41841 (0.0008) [2023-12-27 05:57:15,860][20381] Updated weights for policy 1, policy_version 41851 (0.0008) [2023-12-27 05:57:15,911][20381] Updated weights for policy 1, policy_version 41861 (0.0009) [2023-12-27 05:57:16,308][20347] Updated weights for policy 0, policy_version 41912 (0.0010) [2023-12-27 05:57:16,363][20347] Updated weights for policy 0, policy_version 41922 (0.0010) [2023-12-27 05:57:16,414][20347] Updated weights for policy 0, policy_version 41932 (0.0010) [2023-12-27 05:57:16,533][20381] Updated weights for policy 1, policy_version 41871 (0.0007) [2023-12-27 05:57:16,585][20381] Updated weights for policy 1, policy_version 41881 (0.0005) [2023-12-27 05:57:16,639][20381] Updated weights for policy 1, policy_version 41891 (0.0009) [2023-12-27 05:57:17,098][20347] Updated weights for policy 0, policy_version 41942 (0.0010) [2023-12-27 05:57:17,150][20347] Updated weights for policy 0, policy_version 41952 (0.0010) [2023-12-27 05:57:17,187][20381] Updated weights for policy 1, policy_version 41901 (0.0005) [2023-12-27 05:57:17,204][20347] Updated weights for policy 0, policy_version 41962 (0.0010) [2023-12-27 05:57:17,246][20381] Updated weights for policy 1, policy_version 41911 (0.0005) [2023-12-27 05:57:17,305][20381] Updated weights for policy 1, policy_version 41921 (0.0005) [2023-12-27 05:57:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 21487616. Throughput: 0: 10237.8, 1: 10400.4. Samples: 21491984. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:17,821][20347] Updated weights for policy 0, policy_version 41972 (0.0008) [2023-12-27 05:57:17,882][20347] Updated weights for policy 0, policy_version 41982 (0.0007) [2023-12-27 05:57:17,898][20381] Updated weights for policy 1, policy_version 41931 (0.0009) [2023-12-27 05:57:17,940][20347] Updated weights for policy 0, policy_version 41992 (0.0010) [2023-12-27 05:57:17,956][20381] Updated weights for policy 1, policy_version 41941 (0.0010) [2023-12-27 05:57:18,011][20381] Updated weights for policy 1, policy_version 41951 (0.0010) [2023-12-27 05:57:18,594][20381] Updated weights for policy 1, policy_version 41961 (0.0010) [2023-12-27 05:57:18,640][20347] Updated weights for policy 0, policy_version 42002 (0.0010) [2023-12-27 05:57:18,659][20381] Updated weights for policy 1, policy_version 41971 (0.0006) [2023-12-27 05:57:18,699][20347] Updated weights for policy 0, policy_version 42012 (0.0007) [2023-12-27 05:57:18,716][20381] Updated weights for policy 1, policy_version 41981 (0.0008) [2023-12-27 05:57:18,756][20347] Updated weights for policy 0, policy_version 42022 (0.0006) [2023-12-27 05:57:18,772][20381] Updated weights for policy 1, policy_version 41991 (0.0010) [2023-12-27 05:57:18,809][20347] Updated weights for policy 0, policy_version 42032 (0.0007) [2023-12-27 05:57:19,348][20381] Updated weights for policy 1, policy_version 42001 (0.0008) [2023-12-27 05:57:19,411][20381] Updated weights for policy 1, policy_version 42011 (0.0008) [2023-12-27 05:57:19,464][20381] Updated weights for policy 1, policy_version 42021 (0.0009) [2023-12-27 05:57:19,550][20347] Updated weights for policy 0, policy_version 42042 (0.0008) [2023-12-27 05:57:19,606][20347] Updated weights for policy 0, policy_version 42052 (0.0008) [2023-12-27 05:57:19,663][20347] Updated weights for policy 0, policy_version 42062 (0.0009) [2023-12-27 05:57:20,179][20381] Updated weights for policy 1, policy_version 42031 (0.0009) [2023-12-27 05:57:20,240][20381] Updated weights for policy 1, policy_version 42041 (0.0008) [2023-12-27 05:57:20,296][20381] Updated weights for policy 1, policy_version 42051 (0.0008) [2023-12-27 05:57:20,411][20347] Updated weights for policy 0, policy_version 42072 (0.0007) [2023-12-27 05:57:20,466][20347] Updated weights for policy 0, policy_version 42082 (0.0007) [2023-12-27 05:57:20,522][20347] Updated weights for policy 0, policy_version 42092 (0.0008) [2023-12-27 05:57:20,997][20381] Updated weights for policy 1, policy_version 42061 (0.0008) [2023-12-27 05:57:21,065][20381] Updated weights for policy 1, policy_version 42071 (0.0008) [2023-12-27 05:57:21,120][20381] Updated weights for policy 1, policy_version 42081 (0.0008) [2023-12-27 05:57:21,227][20347] Updated weights for policy 0, policy_version 42102 (0.0007) [2023-12-27 05:57:21,289][20347] Updated weights for policy 0, policy_version 42112 (0.0008) [2023-12-27 05:57:21,356][20347] Updated weights for policy 0, policy_version 42122 (0.0008) [2023-12-27 05:57:21,812][20381] Updated weights for policy 1, policy_version 42091 (0.0008) [2023-12-27 05:57:21,863][20381] Updated weights for policy 1, policy_version 42101 (0.0007) [2023-12-27 05:57:21,916][20381] Updated weights for policy 1, policy_version 42111 (0.0008) [2023-12-27 05:57:22,073][20347] Updated weights for policy 0, policy_version 42132 (0.0008) [2023-12-27 05:57:22,134][20347] Updated weights for policy 0, policy_version 42142 (0.0008) [2023-12-27 05:57:22,199][20347] Updated weights for policy 0, policy_version 42152 (0.0007) [2023-12-27 05:57:22,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 21585920. Throughput: 0: 10222.6, 1: 10410.0. Samples: 21554548. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000042120_10788864.pth... [2023-12-27 05:57:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000042160_10797056.pth... [2023-12-27 05:57:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000040904_10477568.pth [2023-12-27 05:57:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000040976_10493952.pth [2023-12-27 05:57:22,585][20381] Updated weights for policy 1, policy_version 42121 (0.0006) [2023-12-27 05:57:22,645][20381] Updated weights for policy 1, policy_version 42131 (0.0011) [2023-12-27 05:57:22,701][20381] Updated weights for policy 1, policy_version 42141 (0.0011) [2023-12-27 05:57:22,750][20381] Updated weights for policy 1, policy_version 42151 (0.0007) [2023-12-27 05:57:22,907][20347] Updated weights for policy 0, policy_version 42162 (0.0007) [2023-12-27 05:57:22,960][20347] Updated weights for policy 0, policy_version 42172 (0.0007) [2023-12-27 05:57:23,013][20347] Updated weights for policy 0, policy_version 42182 (0.0008) [2023-12-27 05:57:23,061][20347] Updated weights for policy 0, policy_version 42192 (0.0007) [2023-12-27 05:57:23,392][20381] Updated weights for policy 1, policy_version 42161 (0.0009) [2023-12-27 05:57:23,444][20381] Updated weights for policy 1, policy_version 42171 (0.0008) [2023-12-27 05:57:23,499][20381] Updated weights for policy 1, policy_version 42181 (0.0008) [2023-12-27 05:57:23,784][20347] Updated weights for policy 0, policy_version 42202 (0.0007) [2023-12-27 05:57:23,840][20347] Updated weights for policy 0, policy_version 42212 (0.0007) [2023-12-27 05:57:23,889][20347] Updated weights for policy 0, policy_version 42222 (0.0008) [2023-12-27 05:57:24,173][20381] Updated weights for policy 1, policy_version 42191 (0.0008) [2023-12-27 05:57:24,221][20381] Updated weights for policy 1, policy_version 42201 (0.0008) [2023-12-27 05:57:24,279][20381] Updated weights for policy 1, policy_version 42211 (0.0008) [2023-12-27 05:57:24,602][20347] Updated weights for policy 0, policy_version 42232 (0.0007) [2023-12-27 05:57:24,658][20347] Updated weights for policy 0, policy_version 42242 (0.0008) [2023-12-27 05:57:24,717][20347] Updated weights for policy 0, policy_version 42252 (0.0008) [2023-12-27 05:57:24,955][20381] Updated weights for policy 1, policy_version 42221 (0.0007) [2023-12-27 05:57:25,018][20381] Updated weights for policy 1, policy_version 42231 (0.0005) [2023-12-27 05:57:25,080][20381] Updated weights for policy 1, policy_version 42241 (0.0005) [2023-12-27 05:57:25,435][20347] Updated weights for policy 0, policy_version 42262 (0.0007) [2023-12-27 05:57:25,494][20347] Updated weights for policy 0, policy_version 42272 (0.0007) [2023-12-27 05:57:25,553][20347] Updated weights for policy 0, policy_version 42282 (0.0008) [2023-12-27 05:57:25,698][20381] Updated weights for policy 1, policy_version 42251 (0.0007) [2023-12-27 05:57:25,762][20381] Updated weights for policy 1, policy_version 42261 (0.0010) [2023-12-27 05:57:25,817][20381] Updated weights for policy 1, policy_version 42271 (0.0010) [2023-12-27 05:57:26,252][20347] Updated weights for policy 0, policy_version 42292 (0.0007) [2023-12-27 05:57:26,306][20347] Updated weights for policy 0, policy_version 42302 (0.0005) [2023-12-27 05:57:26,354][20347] Updated weights for policy 0, policy_version 42312 (0.0005) [2023-12-27 05:57:26,415][20381] Updated weights for policy 1, policy_version 42281 (0.0010) [2023-12-27 05:57:26,471][20381] Updated weights for policy 1, policy_version 42291 (0.0005) [2023-12-27 05:57:26,531][20381] Updated weights for policy 1, policy_version 42301 (0.0006) [2023-12-27 05:57:26,582][20381] Updated weights for policy 1, policy_version 42311 (0.0006) [2023-12-27 05:57:26,979][20347] Updated weights for policy 0, policy_version 42322 (0.0006) [2023-12-27 05:57:27,026][20347] Updated weights for policy 0, policy_version 42332 (0.0005) [2023-12-27 05:57:27,074][20347] Updated weights for policy 0, policy_version 42342 (0.0005) [2023-12-27 05:57:27,122][20347] Updated weights for policy 0, policy_version 42352 (0.0005) [2023-12-27 05:57:27,140][20381] Updated weights for policy 1, policy_version 42321 (0.0008) [2023-12-27 05:57:27,187][20381] Updated weights for policy 1, policy_version 42331 (0.0005) [2023-12-27 05:57:27,239][20381] Updated weights for policy 1, policy_version 42341 (0.0010) [2023-12-27 05:57:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 21692416. Throughput: 0: 10135.6, 1: 10407.3. Samples: 21675688. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:27,778][20347] Updated weights for policy 0, policy_version 42362 (0.0007) [2023-12-27 05:57:27,830][20347] Updated weights for policy 0, policy_version 42372 (0.0007) [2023-12-27 05:57:27,889][20381] Updated weights for policy 1, policy_version 42351 (0.0008) [2023-12-27 05:57:27,892][20347] Updated weights for policy 0, policy_version 42382 (0.0007) [2023-12-27 05:57:27,953][20381] Updated weights for policy 1, policy_version 42361 (0.0005) [2023-12-27 05:57:28,018][20381] Updated weights for policy 1, policy_version 42371 (0.0007) [2023-12-27 05:57:28,568][20347] Updated weights for policy 0, policy_version 42392 (0.0008) [2023-12-27 05:57:28,619][20347] Updated weights for policy 0, policy_version 42402 (0.0007) [2023-12-27 05:57:28,656][20381] Updated weights for policy 1, policy_version 42381 (0.0008) [2023-12-27 05:57:28,674][20347] Updated weights for policy 0, policy_version 42412 (0.0006) [2023-12-27 05:57:28,713][20381] Updated weights for policy 1, policy_version 42391 (0.0007) [2023-12-27 05:57:28,769][20381] Updated weights for policy 1, policy_version 42401 (0.0008) [2023-12-27 05:57:29,366][20347] Updated weights for policy 0, policy_version 42422 (0.0007) [2023-12-27 05:57:29,426][20347] Updated weights for policy 0, policy_version 42432 (0.0006) [2023-12-27 05:57:29,479][20381] Updated weights for policy 1, policy_version 42411 (0.0007) [2023-12-27 05:57:29,485][20347] Updated weights for policy 0, policy_version 42442 (0.0007) [2023-12-27 05:57:29,536][20381] Updated weights for policy 1, policy_version 42421 (0.0009) [2023-12-27 05:57:29,588][20381] Updated weights for policy 1, policy_version 42431 (0.0010) [2023-12-27 05:57:30,151][20347] Updated weights for policy 0, policy_version 42452 (0.0008) [2023-12-27 05:57:30,216][20347] Updated weights for policy 0, policy_version 42462 (0.0010) [2023-12-27 05:57:30,268][20347] Updated weights for policy 0, policy_version 42472 (0.0010) [2023-12-27 05:57:30,275][20381] Updated weights for policy 1, policy_version 42441 (0.0011) [2023-12-27 05:57:30,330][20381] Updated weights for policy 1, policy_version 42451 (0.0011) [2023-12-27 05:57:30,391][20381] Updated weights for policy 1, policy_version 42461 (0.0009) [2023-12-27 05:57:30,455][20381] Updated weights for policy 1, policy_version 42471 (0.0006) [2023-12-27 05:57:30,937][20347] Updated weights for policy 0, policy_version 42482 (0.0010) [2023-12-27 05:57:30,987][20347] Updated weights for policy 0, policy_version 42492 (0.0007) [2023-12-27 05:57:31,041][20347] Updated weights for policy 0, policy_version 42502 (0.0007) [2023-12-27 05:57:31,064][20381] Updated weights for policy 1, policy_version 42481 (0.0008) [2023-12-27 05:57:31,101][20347] Updated weights for policy 0, policy_version 42512 (0.0007) [2023-12-27 05:57:31,121][20381] Updated weights for policy 1, policy_version 42491 (0.0007) [2023-12-27 05:57:31,182][20381] Updated weights for policy 1, policy_version 42501 (0.0010) [2023-12-27 05:57:31,812][20347] Updated weights for policy 0, policy_version 42522 (0.0010) [2023-12-27 05:57:31,835][20381] Updated weights for policy 1, policy_version 42511 (0.0009) [2023-12-27 05:57:31,861][20347] Updated weights for policy 0, policy_version 42532 (0.0010) [2023-12-27 05:57:31,892][20381] Updated weights for policy 1, policy_version 42521 (0.0007) [2023-12-27 05:57:31,906][20347] Updated weights for policy 0, policy_version 42542 (0.0010) [2023-12-27 05:57:31,953][20381] Updated weights for policy 1, policy_version 42531 (0.0008) [2023-12-27 05:57:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 21790720. Throughput: 0: 10144.0, 1: 10495.2. Samples: 21802220. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:32,554][20347] Updated weights for policy 0, policy_version 42552 (0.0006) [2023-12-27 05:57:32,614][20381] Updated weights for policy 1, policy_version 42541 (0.0006) [2023-12-27 05:57:32,617][20347] Updated weights for policy 0, policy_version 42562 (0.0006) [2023-12-27 05:57:32,667][20381] Updated weights for policy 1, policy_version 42551 (0.0005) [2023-12-27 05:57:32,675][20347] Updated weights for policy 0, policy_version 42572 (0.0010) [2023-12-27 05:57:32,721][20381] Updated weights for policy 1, policy_version 42561 (0.0007) [2023-12-27 05:57:33,226][20347] Updated weights for policy 0, policy_version 42582 (0.0006) [2023-12-27 05:57:33,287][20347] Updated weights for policy 0, policy_version 42592 (0.0008) [2023-12-27 05:57:33,341][20347] Updated weights for policy 0, policy_version 42602 (0.0010) [2023-12-27 05:57:33,393][20381] Updated weights for policy 1, policy_version 42571 (0.0008) [2023-12-27 05:57:33,440][20381] Updated weights for policy 1, policy_version 42581 (0.0007) [2023-12-27 05:57:33,486][20381] Updated weights for policy 1, policy_version 42591 (0.0008) [2023-12-27 05:57:33,946][20347] Updated weights for policy 0, policy_version 42612 (0.0009) [2023-12-27 05:57:33,997][20347] Updated weights for policy 0, policy_version 42622 (0.0008) [2023-12-27 05:57:34,056][20347] Updated weights for policy 0, policy_version 42632 (0.0008) [2023-12-27 05:57:34,210][20381] Updated weights for policy 1, policy_version 42601 (0.0007) [2023-12-27 05:57:34,268][20381] Updated weights for policy 1, policy_version 42611 (0.0009) [2023-12-27 05:57:34,328][20381] Updated weights for policy 1, policy_version 42621 (0.0009) [2023-12-27 05:57:34,384][20381] Updated weights for policy 1, policy_version 42631 (0.0007) [2023-12-27 05:57:34,666][20347] Updated weights for policy 0, policy_version 42642 (0.0007) [2023-12-27 05:57:34,722][20347] Updated weights for policy 0, policy_version 42652 (0.0007) [2023-12-27 05:57:34,773][20347] Updated weights for policy 0, policy_version 42663 (0.0007) [2023-12-27 05:57:35,090][20381] Updated weights for policy 1, policy_version 42641 (0.0005) [2023-12-27 05:57:35,141][20381] Updated weights for policy 1, policy_version 42651 (0.0005) [2023-12-27 05:57:35,205][20381] Updated weights for policy 1, policy_version 42661 (0.0005) [2023-12-27 05:57:35,446][20347] Updated weights for policy 0, policy_version 42673 (0.0006) [2023-12-27 05:57:35,504][20347] Updated weights for policy 0, policy_version 42683 (0.0005) [2023-12-27 05:57:35,561][20347] Updated weights for policy 0, policy_version 42693 (0.0005) [2023-12-27 05:57:35,614][20347] Updated weights for policy 0, policy_version 42703 (0.0005) [2023-12-27 05:57:35,860][20381] Updated weights for policy 1, policy_version 42671 (0.0007) [2023-12-27 05:57:35,916][20381] Updated weights for policy 1, policy_version 42681 (0.0008) [2023-12-27 05:57:35,960][20381] Updated weights for policy 1, policy_version 42691 (0.0007) [2023-12-27 05:57:36,208][20347] Updated weights for policy 0, policy_version 42713 (0.0008) [2023-12-27 05:57:36,265][20347] Updated weights for policy 0, policy_version 42723 (0.0008) [2023-12-27 05:57:36,322][20347] Updated weights for policy 0, policy_version 42733 (0.0007) [2023-12-27 05:57:36,727][20381] Updated weights for policy 1, policy_version 42701 (0.0008) [2023-12-27 05:57:36,786][20381] Updated weights for policy 1, policy_version 42711 (0.0008) [2023-12-27 05:57:36,841][20381] Updated weights for policy 1, policy_version 42721 (0.0007) [2023-12-27 05:57:36,975][20347] Updated weights for policy 0, policy_version 42743 (0.0009) [2023-12-27 05:57:37,028][20347] Updated weights for policy 0, policy_version 42753 (0.0008) [2023-12-27 05:57:37,080][20347] Updated weights for policy 0, policy_version 42763 (0.0005) [2023-12-27 05:57:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 21897216. Throughput: 0: 10189.5, 1: 10470.1. Samples: 21866092. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000042768_10952704.pth... [2023-12-27 05:57:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000042728_10944512.pth... [2023-12-27 05:57:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000041512_10633216.pth [2023-12-27 05:57:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000041584_10649600.pth [2023-12-27 05:57:37,592][20381] Updated weights for policy 1, policy_version 42731 (0.0007) [2023-12-27 05:57:37,652][20381] Updated weights for policy 1, policy_version 42741 (0.0008) [2023-12-27 05:57:37,665][20347] Updated weights for policy 0, policy_version 42773 (0.0007) [2023-12-27 05:57:37,713][20381] Updated weights for policy 1, policy_version 42751 (0.0006) [2023-12-27 05:57:37,722][20347] Updated weights for policy 0, policy_version 42783 (0.0010) [2023-12-27 05:57:37,775][20347] Updated weights for policy 0, policy_version 42793 (0.0011) [2023-12-27 05:57:38,400][20347] Updated weights for policy 0, policy_version 42803 (0.0009) [2023-12-27 05:57:38,423][20381] Updated weights for policy 1, policy_version 42761 (0.0006) [2023-12-27 05:57:38,457][20347] Updated weights for policy 0, policy_version 42813 (0.0007) [2023-12-27 05:57:38,489][20381] Updated weights for policy 1, policy_version 42771 (0.0007) [2023-12-27 05:57:38,510][20347] Updated weights for policy 0, policy_version 42823 (0.0009) [2023-12-27 05:57:38,540][20381] Updated weights for policy 1, policy_version 42781 (0.0006) [2023-12-27 05:57:38,599][20381] Updated weights for policy 1, policy_version 42791 (0.0006) [2023-12-27 05:57:39,141][20381] Updated weights for policy 1, policy_version 42801 (0.0007) [2023-12-27 05:57:39,195][20381] Updated weights for policy 1, policy_version 42811 (0.0008) [2023-12-27 05:57:39,236][20347] Updated weights for policy 0, policy_version 42833 (0.0008) [2023-12-27 05:57:39,261][20381] Updated weights for policy 1, policy_version 42821 (0.0007) [2023-12-27 05:57:39,296][20347] Updated weights for policy 0, policy_version 42843 (0.0006) [2023-12-27 05:57:39,357][20347] Updated weights for policy 0, policy_version 42853 (0.0007) [2023-12-27 05:57:39,422][20347] Updated weights for policy 0, policy_version 42863 (0.0009) [2023-12-27 05:57:39,905][20381] Updated weights for policy 1, policy_version 42831 (0.0008) [2023-12-27 05:57:39,970][20381] Updated weights for policy 1, policy_version 42841 (0.0008) [2023-12-27 05:57:40,034][20381] Updated weights for policy 1, policy_version 42851 (0.0008) [2023-12-27 05:57:40,074][20347] Updated weights for policy 0, policy_version 42873 (0.0007) [2023-12-27 05:57:40,138][20347] Updated weights for policy 0, policy_version 42883 (0.0008) [2023-12-27 05:57:40,202][20347] Updated weights for policy 0, policy_version 42893 (0.0008) [2023-12-27 05:57:40,707][20381] Updated weights for policy 1, policy_version 42861 (0.0007) [2023-12-27 05:57:40,760][20381] Updated weights for policy 1, policy_version 42871 (0.0008) [2023-12-27 05:57:40,815][20381] Updated weights for policy 1, policy_version 42881 (0.0008) [2023-12-27 05:57:40,935][20347] Updated weights for policy 0, policy_version 42903 (0.0008) [2023-12-27 05:57:40,994][20347] Updated weights for policy 0, policy_version 42913 (0.0007) [2023-12-27 05:57:41,064][20347] Updated weights for policy 0, policy_version 42923 (0.0008) [2023-12-27 05:57:41,550][20381] Updated weights for policy 1, policy_version 42891 (0.0007) [2023-12-27 05:57:41,600][20381] Updated weights for policy 1, policy_version 42901 (0.0008) [2023-12-27 05:57:41,660][20381] Updated weights for policy 1, policy_version 42911 (0.0008) [2023-12-27 05:57:41,768][20347] Updated weights for policy 0, policy_version 42933 (0.0008) [2023-12-27 05:57:41,823][20347] Updated weights for policy 0, policy_version 42943 (0.0007) [2023-12-27 05:57:41,886][20347] Updated weights for policy 0, policy_version 42953 (0.0007) [2023-12-27 05:57:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 21995520. Throughput: 0: 10185.6, 1: 10396.7. Samples: 21989256. Policy #0 lag: (min: 6.0, avg: 10.8, max: 38.0) [2023-12-27 05:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:42,368][20381] Updated weights for policy 1, policy_version 42921 (0.0008) [2023-12-27 05:57:42,427][20381] Updated weights for policy 1, policy_version 42931 (0.0008) [2023-12-27 05:57:42,487][20381] Updated weights for policy 1, policy_version 42941 (0.0008) [2023-12-27 05:57:42,546][20381] Updated weights for policy 1, policy_version 42951 (0.0008) [2023-12-27 05:57:42,599][20347] Updated weights for policy 0, policy_version 42963 (0.0007) [2023-12-27 05:57:42,661][20347] Updated weights for policy 0, policy_version 42973 (0.0008) [2023-12-27 05:57:42,716][20347] Updated weights for policy 0, policy_version 42983 (0.0008) [2023-12-27 05:57:43,261][20381] Updated weights for policy 1, policy_version 42961 (0.0008) [2023-12-27 05:57:43,317][20381] Updated weights for policy 1, policy_version 42971 (0.0008) [2023-12-27 05:57:43,368][20381] Updated weights for policy 1, policy_version 42981 (0.0007) [2023-12-27 05:57:43,409][20347] Updated weights for policy 0, policy_version 42993 (0.0007) [2023-12-27 05:57:43,463][20347] Updated weights for policy 0, policy_version 43003 (0.0008) [2023-12-27 05:57:43,516][20347] Updated weights for policy 0, policy_version 43013 (0.0008) [2023-12-27 05:57:43,562][20347] Updated weights for policy 0, policy_version 43023 (0.0007) [2023-12-27 05:57:44,018][20381] Updated weights for policy 1, policy_version 42991 (0.0010) [2023-12-27 05:57:44,066][20381] Updated weights for policy 1, policy_version 43001 (0.0010) [2023-12-27 05:57:44,114][20381] Updated weights for policy 1, policy_version 43011 (0.0010) [2023-12-27 05:57:44,283][20347] Updated weights for policy 0, policy_version 43033 (0.0008) [2023-12-27 05:57:44,337][20347] Updated weights for policy 0, policy_version 43043 (0.0007) [2023-12-27 05:57:44,390][20347] Updated weights for policy 0, policy_version 43053 (0.0006) [2023-12-27 05:57:44,772][20381] Updated weights for policy 1, policy_version 43021 (0.0010) [2023-12-27 05:57:44,836][20381] Updated weights for policy 1, policy_version 43031 (0.0008) [2023-12-27 05:57:44,900][20381] Updated weights for policy 1, policy_version 43041 (0.0007) [2023-12-27 05:57:45,072][20347] Updated weights for policy 0, policy_version 43063 (0.0006) [2023-12-27 05:57:45,131][20347] Updated weights for policy 0, policy_version 43073 (0.0008) [2023-12-27 05:57:45,195][20347] Updated weights for policy 0, policy_version 43083 (0.0008) [2023-12-27 05:57:45,629][20381] Updated weights for policy 1, policy_version 43051 (0.0007) [2023-12-27 05:57:45,688][20381] Updated weights for policy 1, policy_version 43061 (0.0006) [2023-12-27 05:57:45,745][20381] Updated weights for policy 1, policy_version 43071 (0.0008) [2023-12-27 05:57:45,879][20347] Updated weights for policy 0, policy_version 43093 (0.0008) [2023-12-27 05:57:45,932][20347] Updated weights for policy 0, policy_version 43103 (0.0007) [2023-12-27 05:57:45,988][20347] Updated weights for policy 0, policy_version 43113 (0.0007) [2023-12-27 05:57:46,450][20381] Updated weights for policy 1, policy_version 43081 (0.0008) [2023-12-27 05:57:46,510][20381] Updated weights for policy 1, policy_version 43091 (0.0008) [2023-12-27 05:57:46,571][20381] Updated weights for policy 1, policy_version 43101 (0.0009) [2023-12-27 05:57:46,598][20347] Updated weights for policy 0, policy_version 43123 (0.0007) [2023-12-27 05:57:46,625][20381] Updated weights for policy 1, policy_version 43111 (0.0008) [2023-12-27 05:57:46,657][20347] Updated weights for policy 0, policy_version 43133 (0.0006) [2023-12-27 05:57:46,714][20347] Updated weights for policy 0, policy_version 43143 (0.0007) [2023-12-27 05:57:47,328][20381] Updated weights for policy 1, policy_version 43121 (0.0007) [2023-12-27 05:57:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 22093824. Throughput: 0: 10186.0, 1: 10382.6. Samples: 22109212. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-12-27 05:57:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:47,373][20347] Updated weights for policy 0, policy_version 43153 (0.0007) [2023-12-27 05:57:47,379][20381] Updated weights for policy 1, policy_version 43131 (0.0007) [2023-12-27 05:57:47,427][20381] Updated weights for policy 1, policy_version 43141 (0.0005) [2023-12-27 05:57:47,428][20347] Updated weights for policy 0, policy_version 43163 (0.0007) [2023-12-27 05:57:47,479][20347] Updated weights for policy 0, policy_version 43173 (0.0007) [2023-12-27 05:57:47,536][20347] Updated weights for policy 0, policy_version 43183 (0.0010) [2023-12-27 05:57:48,082][20381] Updated weights for policy 1, policy_version 43151 (0.0006) [2023-12-27 05:57:48,139][20381] Updated weights for policy 1, policy_version 43161 (0.0005) [2023-12-27 05:57:48,188][20347] Updated weights for policy 0, policy_version 43193 (0.0007) [2023-12-27 05:57:48,202][20381] Updated weights for policy 1, policy_version 43171 (0.0006) [2023-12-27 05:57:48,237][20347] Updated weights for policy 0, policy_version 43203 (0.0008) [2023-12-27 05:57:48,288][20347] Updated weights for policy 0, policy_version 43213 (0.0010) [2023-12-27 05:57:48,866][20381] Updated weights for policy 1, policy_version 43181 (0.0009) [2023-12-27 05:57:48,921][20381] Updated weights for policy 1, policy_version 43191 (0.0010) [2023-12-27 05:57:48,950][20347] Updated weights for policy 0, policy_version 43223 (0.0007) [2023-12-27 05:57:48,970][20381] Updated weights for policy 1, policy_version 43201 (0.0006) [2023-12-27 05:57:49,011][20347] Updated weights for policy 0, policy_version 43233 (0.0008) [2023-12-27 05:57:49,062][20347] Updated weights for policy 0, policy_version 43243 (0.0007) [2023-12-27 05:57:49,650][20381] Updated weights for policy 1, policy_version 43211 (0.0008) [2023-12-27 05:57:49,699][20381] Updated weights for policy 1, policy_version 43221 (0.0010) [2023-12-27 05:57:49,754][20381] Updated weights for policy 1, policy_version 43231 (0.0011) [2023-12-27 05:57:49,760][20347] Updated weights for policy 0, policy_version 43253 (0.0007) [2023-12-27 05:57:49,816][20347] Updated weights for policy 0, policy_version 43263 (0.0007) [2023-12-27 05:57:49,877][20347] Updated weights for policy 0, policy_version 43273 (0.0007) [2023-12-27 05:57:50,466][20381] Updated weights for policy 1, policy_version 43241 (0.0010) [2023-12-27 05:57:50,496][20347] Updated weights for policy 0, policy_version 43283 (0.0006) [2023-12-27 05:57:50,520][20381] Updated weights for policy 1, policy_version 43251 (0.0010) [2023-12-27 05:57:50,553][20347] Updated weights for policy 0, policy_version 43293 (0.0005) [2023-12-27 05:57:50,582][20381] Updated weights for policy 1, policy_version 43261 (0.0010) [2023-12-27 05:57:50,618][20347] Updated weights for policy 0, policy_version 43303 (0.0007) [2023-12-27 05:57:50,643][20381] Updated weights for policy 1, policy_version 43271 (0.0007) [2023-12-27 05:57:51,290][20347] Updated weights for policy 0, policy_version 43313 (0.0007) [2023-12-27 05:57:51,292][20381] Updated weights for policy 1, policy_version 43281 (0.0008) [2023-12-27 05:57:51,344][20347] Updated weights for policy 0, policy_version 43323 (0.0006) [2023-12-27 05:57:51,351][20381] Updated weights for policy 1, policy_version 43291 (0.0006) [2023-12-27 05:57:51,400][20347] Updated weights for policy 0, policy_version 43333 (0.0008) [2023-12-27 05:57:51,407][20381] Updated weights for policy 1, policy_version 43301 (0.0007) [2023-12-27 05:57:51,449][20347] Updated weights for policy 0, policy_version 43343 (0.0008) [2023-12-27 05:57:52,014][20381] Updated weights for policy 1, policy_version 43311 (0.0007) [2023-12-27 05:57:52,073][20381] Updated weights for policy 1, policy_version 43321 (0.0008) [2023-12-27 05:57:52,132][20381] Updated weights for policy 1, policy_version 43331 (0.0008) [2023-12-27 05:57:52,184][20347] Updated weights for policy 0, policy_version 43353 (0.0007) [2023-12-27 05:57:52,236][20347] Updated weights for policy 0, policy_version 43363 (0.0008) [2023-12-27 05:57:52,296][20347] Updated weights for policy 0, policy_version 43373 (0.0010) [2023-12-27 05:57:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 22208512. Throughput: 0: 10205.6, 1: 10348.2. Samples: 22170840. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-12-27 05:57:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000043336_11100160.pth... [2023-12-27 05:57:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000043376_11108352.pth... [2023-12-27 05:57:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000042120_10788864.pth [2023-12-27 05:57:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000042160_10797056.pth [2023-12-27 05:57:52,804][20381] Updated weights for policy 1, policy_version 43341 (0.0007) [2023-12-27 05:57:52,858][20381] Updated weights for policy 1, policy_version 43351 (0.0008) [2023-12-27 05:57:52,917][20381] Updated weights for policy 1, policy_version 43361 (0.0006) [2023-12-27 05:57:52,952][20347] Updated weights for policy 0, policy_version 43383 (0.0007) [2023-12-27 05:57:53,007][20347] Updated weights for policy 0, policy_version 43393 (0.0007) [2023-12-27 05:57:53,060][20347] Updated weights for policy 0, policy_version 43403 (0.0007) [2023-12-27 05:57:53,575][20381] Updated weights for policy 1, policy_version 43371 (0.0007) [2023-12-27 05:57:53,623][20381] Updated weights for policy 1, policy_version 43381 (0.0008) [2023-12-27 05:57:53,670][20381] Updated weights for policy 1, policy_version 43391 (0.0007) [2023-12-27 05:57:53,684][20347] Updated weights for policy 0, policy_version 43413 (0.0007) [2023-12-27 05:57:53,739][20347] Updated weights for policy 0, policy_version 43423 (0.0006) [2023-12-27 05:57:53,791][20347] Updated weights for policy 0, policy_version 43433 (0.0008) [2023-12-27 05:57:54,318][20381] Updated weights for policy 1, policy_version 43401 (0.0006) [2023-12-27 05:57:54,374][20381] Updated weights for policy 1, policy_version 43411 (0.0008) [2023-12-27 05:57:54,391][20347] Updated weights for policy 0, policy_version 43443 (0.0007) [2023-12-27 05:57:54,428][20381] Updated weights for policy 1, policy_version 43421 (0.0005) [2023-12-27 05:57:54,451][20347] Updated weights for policy 0, policy_version 43453 (0.0007) [2023-12-27 05:57:54,480][20381] Updated weights for policy 1, policy_version 43431 (0.0006) [2023-12-27 05:57:54,508][20347] Updated weights for policy 0, policy_version 43463 (0.0007) [2023-12-27 05:57:55,082][20381] Updated weights for policy 1, policy_version 43441 (0.0005) [2023-12-27 05:57:55,129][20381] Updated weights for policy 1, policy_version 43451 (0.0005) [2023-12-27 05:57:55,144][20347] Updated weights for policy 0, policy_version 43473 (0.0008) [2023-12-27 05:57:55,177][20381] Updated weights for policy 1, policy_version 43461 (0.0005) [2023-12-27 05:57:55,205][20347] Updated weights for policy 0, policy_version 43483 (0.0008) [2023-12-27 05:57:55,269][20347] Updated weights for policy 0, policy_version 43493 (0.0005) [2023-12-27 05:57:55,325][20347] Updated weights for policy 0, policy_version 43503 (0.0008) [2023-12-27 05:57:55,826][20381] Updated weights for policy 1, policy_version 43471 (0.0008) [2023-12-27 05:57:55,879][20381] Updated weights for policy 1, policy_version 43481 (0.0008) [2023-12-27 05:57:55,926][20381] Updated weights for policy 1, policy_version 43491 (0.0007) [2023-12-27 05:57:55,951][20347] Updated weights for policy 0, policy_version 43513 (0.0006) [2023-12-27 05:57:56,001][20347] Updated weights for policy 0, policy_version 43523 (0.0007) [2023-12-27 05:57:56,063][20347] Updated weights for policy 0, policy_version 43533 (0.0005) [2023-12-27 05:57:56,580][20381] Updated weights for policy 1, policy_version 43501 (0.0007) [2023-12-27 05:57:56,635][20381] Updated weights for policy 1, policy_version 43511 (0.0008) [2023-12-27 05:57:56,681][20381] Updated weights for policy 1, policy_version 43521 (0.0009) [2023-12-27 05:57:56,718][20347] Updated weights for policy 0, policy_version 43543 (0.0006) [2023-12-27 05:57:56,768][20347] Updated weights for policy 0, policy_version 43553 (0.0007) [2023-12-27 05:57:56,825][20347] Updated weights for policy 0, policy_version 43563 (0.0007) [2023-12-27 05:57:57,368][19170] Fps is (10 sec: 21286.5, 60 sec: 20614.5, 300 sec: 20798.9). Total num frames: 22306816. Throughput: 0: 10310.7, 1: 10399.7. Samples: 22299636. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-12-27 05:57:57,369][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:57:57,373][20381] Updated weights for policy 1, policy_version 43531 (0.0009) [2023-12-27 05:57:57,412][20347] Updated weights for policy 0, policy_version 43573 (0.0009) [2023-12-27 05:57:57,427][20381] Updated weights for policy 1, policy_version 43541 (0.0007) [2023-12-27 05:57:57,468][20347] Updated weights for policy 0, policy_version 43583 (0.0007) [2023-12-27 05:57:57,471][20381] Updated weights for policy 1, policy_version 43551 (0.0006) [2023-12-27 05:57:57,522][20347] Updated weights for policy 0, policy_version 43593 (0.0008) [2023-12-27 05:57:58,056][20381] Updated weights for policy 1, policy_version 43561 (0.0005) [2023-12-27 05:57:58,117][20381] Updated weights for policy 1, policy_version 43571 (0.0006) [2023-12-27 05:57:58,146][20347] Updated weights for policy 0, policy_version 43603 (0.0006) [2023-12-27 05:57:58,175][20381] Updated weights for policy 1, policy_version 43581 (0.0008) [2023-12-27 05:57:58,209][20347] Updated weights for policy 0, policy_version 43613 (0.0006) [2023-12-27 05:57:58,233][20381] Updated weights for policy 1, policy_version 43591 (0.0006) [2023-12-27 05:57:58,268][20347] Updated weights for policy 0, policy_version 43623 (0.0007) [2023-12-27 05:57:58,935][20381] Updated weights for policy 1, policy_version 43601 (0.0008) [2023-12-27 05:57:58,984][20381] Updated weights for policy 1, policy_version 43611 (0.0008) [2023-12-27 05:57:59,028][20347] Updated weights for policy 0, policy_version 43633 (0.0007) [2023-12-27 05:57:59,032][20381] Updated weights for policy 1, policy_version 43621 (0.0008) [2023-12-27 05:57:59,075][20347] Updated weights for policy 0, policy_version 43643 (0.0005) [2023-12-27 05:57:59,129][20347] Updated weights for policy 0, policy_version 43653 (0.0005) [2023-12-27 05:57:59,186][20347] Updated weights for policy 0, policy_version 43663 (0.0006) [2023-12-27 05:57:59,718][20381] Updated weights for policy 1, policy_version 43631 (0.0007) [2023-12-27 05:57:59,766][20381] Updated weights for policy 1, policy_version 43641 (0.0007) [2023-12-27 05:57:59,821][20381] Updated weights for policy 1, policy_version 43651 (0.0007) [2023-12-27 05:57:59,883][20347] Updated weights for policy 0, policy_version 43673 (0.0007) [2023-12-27 05:57:59,946][20347] Updated weights for policy 0, policy_version 43683 (0.0008) [2023-12-27 05:57:59,998][20347] Updated weights for policy 0, policy_version 43693 (0.0007) [2023-12-27 05:58:00,515][20381] Updated weights for policy 1, policy_version 43661 (0.0008) [2023-12-27 05:58:00,565][20381] Updated weights for policy 1, policy_version 43671 (0.0007) [2023-12-27 05:58:00,612][20381] Updated weights for policy 1, policy_version 43681 (0.0007) [2023-12-27 05:58:00,690][20347] Updated weights for policy 0, policy_version 43703 (0.0007) [2023-12-27 05:58:00,748][20347] Updated weights for policy 0, policy_version 43713 (0.0007) [2023-12-27 05:58:00,805][20347] Updated weights for policy 0, policy_version 43723 (0.0007) [2023-12-27 05:58:01,337][20381] Updated weights for policy 1, policy_version 43691 (0.0008) [2023-12-27 05:58:01,399][20381] Updated weights for policy 1, policy_version 43701 (0.0007) [2023-12-27 05:58:01,451][20381] Updated weights for policy 1, policy_version 43711 (0.0005) [2023-12-27 05:58:01,492][20347] Updated weights for policy 0, policy_version 43733 (0.0007) [2023-12-27 05:58:01,540][20347] Updated weights for policy 0, policy_version 43743 (0.0008) [2023-12-27 05:58:01,604][20347] Updated weights for policy 0, policy_version 43753 (0.0009) [2023-12-27 05:58:02,098][20381] Updated weights for policy 1, policy_version 43721 (0.0006) [2023-12-27 05:58:02,161][20381] Updated weights for policy 1, policy_version 43731 (0.0007) [2023-12-27 05:58:02,221][20381] Updated weights for policy 1, policy_version 43741 (0.0008) [2023-12-27 05:58:02,240][20347] Updated weights for policy 0, policy_version 43763 (0.0009) [2023-12-27 05:58:02,283][20381] Updated weights for policy 1, policy_version 43751 (0.0007) [2023-12-27 05:58:02,300][20347] Updated weights for policy 0, policy_version 43773 (0.0006) [2023-12-27 05:58:02,359][20347] Updated weights for policy 0, policy_version 43783 (0.0007) [2023-12-27 05:58:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 22413312. Throughput: 0: 10323.5, 1: 10391.3. Samples: 22424148. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-12-27 05:58:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:02,887][20381] Updated weights for policy 1, policy_version 43761 (0.0010) [2023-12-27 05:58:02,923][20347] Updated weights for policy 0, policy_version 43793 (0.0007) [2023-12-27 05:58:02,946][20381] Updated weights for policy 1, policy_version 43771 (0.0010) [2023-12-27 05:58:02,978][20347] Updated weights for policy 0, policy_version 43803 (0.0007) [2023-12-27 05:58:02,997][20381] Updated weights for policy 1, policy_version 43781 (0.0010) [2023-12-27 05:58:03,042][20347] Updated weights for policy 0, policy_version 43813 (0.0008) [2023-12-27 05:58:03,103][20347] Updated weights for policy 0, policy_version 43823 (0.0008) [2023-12-27 05:58:03,586][20381] Updated weights for policy 1, policy_version 43791 (0.0007) [2023-12-27 05:58:03,641][20381] Updated weights for policy 1, policy_version 43801 (0.0007) [2023-12-27 05:58:03,651][20347] Updated weights for policy 0, policy_version 43833 (0.0008) [2023-12-27 05:58:03,693][20381] Updated weights for policy 1, policy_version 43811 (0.0005) [2023-12-27 05:58:03,713][20347] Updated weights for policy 0, policy_version 43843 (0.0007) [2023-12-27 05:58:03,773][20347] Updated weights for policy 0, policy_version 43853 (0.0008) [2023-12-27 05:58:04,395][20381] Updated weights for policy 1, policy_version 43821 (0.0008) [2023-12-27 05:58:04,431][20347] Updated weights for policy 0, policy_version 43863 (0.0006) [2023-12-27 05:58:04,451][20381] Updated weights for policy 1, policy_version 43831 (0.0011) [2023-12-27 05:58:04,484][20347] Updated weights for policy 0, policy_version 43873 (0.0008) [2023-12-27 05:58:04,507][20381] Updated weights for policy 1, policy_version 43841 (0.0011) [2023-12-27 05:58:04,536][20347] Updated weights for policy 0, policy_version 43883 (0.0008) [2023-12-27 05:58:05,186][20381] Updated weights for policy 1, policy_version 43851 (0.0010) [2023-12-27 05:58:05,241][20381] Updated weights for policy 1, policy_version 43861 (0.0010) [2023-12-27 05:58:05,241][20347] Updated weights for policy 0, policy_version 43893 (0.0008) [2023-12-27 05:58:05,293][20347] Updated weights for policy 0, policy_version 43903 (0.0006) [2023-12-27 05:58:05,295][20381] Updated weights for policy 1, policy_version 43871 (0.0010) [2023-12-27 05:58:05,340][20347] Updated weights for policy 0, policy_version 43913 (0.0005) [2023-12-27 05:58:05,975][20381] Updated weights for policy 1, policy_version 43881 (0.0010) [2023-12-27 05:58:06,024][20347] Updated weights for policy 0, policy_version 43923 (0.0008) [2023-12-27 05:58:06,026][20381] Updated weights for policy 1, policy_version 43891 (0.0010) [2023-12-27 05:58:06,077][20381] Updated weights for policy 1, policy_version 43901 (0.0010) [2023-12-27 05:58:06,082][20347] Updated weights for policy 0, policy_version 43933 (0.0010) [2023-12-27 05:58:06,127][20381] Updated weights for policy 1, policy_version 43911 (0.0010) [2023-12-27 05:58:06,134][20347] Updated weights for policy 0, policy_version 43943 (0.0009) [2023-12-27 05:58:06,809][20347] Updated weights for policy 0, policy_version 43953 (0.0007) [2023-12-27 05:58:06,824][20381] Updated weights for policy 1, policy_version 43921 (0.0010) [2023-12-27 05:58:06,873][20347] Updated weights for policy 0, policy_version 43963 (0.0011) [2023-12-27 05:58:06,881][20381] Updated weights for policy 1, policy_version 43931 (0.0010) [2023-12-27 05:58:06,927][20347] Updated weights for policy 0, policy_version 43973 (0.0011) [2023-12-27 05:58:06,939][20381] Updated weights for policy 1, policy_version 43941 (0.0011) [2023-12-27 05:58:06,988][20347] Updated weights for policy 0, policy_version 43983 (0.0011) [2023-12-27 05:58:07,362][19170] Fps is (10 sec: 21311.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 22519808. Throughput: 0: 10407.4, 1: 10366.2. Samples: 22489356. Policy #0 lag: (min: 31.0, avg: 31.7, max: 47.0) [2023-12-27 05:58:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000043944_11255808.pth... [2023-12-27 05:58:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000043984_11264000.pth... [2023-12-27 05:58:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000042768_10952704.pth [2023-12-27 05:58:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000042728_10944512.pth [2023-12-27 05:58:07,653][20381] Updated weights for policy 1, policy_version 43951 (0.0007) [2023-12-27 05:58:07,672][20347] Updated weights for policy 0, policy_version 43993 (0.0010) [2023-12-27 05:58:07,702][20381] Updated weights for policy 1, policy_version 43961 (0.0009) [2023-12-27 05:58:07,731][20347] Updated weights for policy 0, policy_version 44003 (0.0010) [2023-12-27 05:58:07,759][20381] Updated weights for policy 1, policy_version 43971 (0.0006) [2023-12-27 05:58:07,773][20347] Updated weights for policy 0, policy_version 44013 (0.0010) [2023-12-27 05:58:08,401][20381] Updated weights for policy 1, policy_version 43981 (0.0006) [2023-12-27 05:58:08,406][20347] Updated weights for policy 0, policy_version 44023 (0.0008) [2023-12-27 05:58:08,458][20381] Updated weights for policy 1, policy_version 43991 (0.0007) [2023-12-27 05:58:08,460][20347] Updated weights for policy 0, policy_version 44033 (0.0006) [2023-12-27 05:58:08,511][20381] Updated weights for policy 1, policy_version 44001 (0.0006) [2023-12-27 05:58:08,516][20347] Updated weights for policy 0, policy_version 44043 (0.0007) [2023-12-27 05:58:09,110][20381] Updated weights for policy 1, policy_version 44011 (0.0006) [2023-12-27 05:58:09,162][20381] Updated weights for policy 1, policy_version 44021 (0.0008) [2023-12-27 05:58:09,198][20347] Updated weights for policy 0, policy_version 44053 (0.0006) [2023-12-27 05:58:09,212][20381] Updated weights for policy 1, policy_version 44031 (0.0006) [2023-12-27 05:58:09,258][20347] Updated weights for policy 0, policy_version 44063 (0.0008) [2023-12-27 05:58:09,312][20347] Updated weights for policy 0, policy_version 44073 (0.0005) [2023-12-27 05:58:09,938][20381] Updated weights for policy 1, policy_version 44041 (0.0007) [2023-12-27 05:58:09,999][20381] Updated weights for policy 1, policy_version 44051 (0.0007) [2023-12-27 05:58:10,034][20347] Updated weights for policy 0, policy_version 44083 (0.0008) [2023-12-27 05:58:10,066][20381] Updated weights for policy 1, policy_version 44061 (0.0007) [2023-12-27 05:58:10,103][20347] Updated weights for policy 0, policy_version 44093 (0.0006) [2023-12-27 05:58:10,125][20381] Updated weights for policy 1, policy_version 44071 (0.0006) [2023-12-27 05:58:10,166][20347] Updated weights for policy 0, policy_version 44103 (0.0007) [2023-12-27 05:58:10,818][20381] Updated weights for policy 1, policy_version 44081 (0.0008) [2023-12-27 05:58:10,883][20381] Updated weights for policy 1, policy_version 44091 (0.0010) [2023-12-27 05:58:10,902][20347] Updated weights for policy 0, policy_version 44113 (0.0008) [2023-12-27 05:58:10,946][20381] Updated weights for policy 1, policy_version 44101 (0.0008) [2023-12-27 05:58:10,967][20347] Updated weights for policy 0, policy_version 44123 (0.0006) [2023-12-27 05:58:11,028][20347] Updated weights for policy 0, policy_version 44133 (0.0008) [2023-12-27 05:58:11,086][20347] Updated weights for policy 0, policy_version 44143 (0.0009) [2023-12-27 05:58:11,641][20381] Updated weights for policy 1, policy_version 44111 (0.0007) [2023-12-27 05:58:11,698][20381] Updated weights for policy 1, policy_version 44121 (0.0006) [2023-12-27 05:58:11,765][20381] Updated weights for policy 1, policy_version 44131 (0.0007) [2023-12-27 05:58:11,849][20347] Updated weights for policy 0, policy_version 44153 (0.0009) [2023-12-27 05:58:11,916][20347] Updated weights for policy 0, policy_version 44163 (0.0008) [2023-12-27 05:58:11,976][20347] Updated weights for policy 0, policy_version 44173 (0.0006) [2023-12-27 05:58:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 22618112. Throughput: 0: 10436.6, 1: 10333.4. Samples: 22610332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:12,442][20381] Updated weights for policy 1, policy_version 44141 (0.0008) [2023-12-27 05:58:12,499][20381] Updated weights for policy 1, policy_version 44151 (0.0008) [2023-12-27 05:58:12,555][20381] Updated weights for policy 1, policy_version 44161 (0.0006) [2023-12-27 05:58:12,568][20347] Updated weights for policy 0, policy_version 44183 (0.0007) [2023-12-27 05:58:12,623][20347] Updated weights for policy 0, policy_version 44193 (0.0006) [2023-12-27 05:58:12,686][20347] Updated weights for policy 0, policy_version 44203 (0.0008) [2023-12-27 05:58:13,159][20381] Updated weights for policy 1, policy_version 44171 (0.0007) [2023-12-27 05:58:13,212][20381] Updated weights for policy 1, policy_version 44181 (0.0008) [2023-12-27 05:58:13,272][20381] Updated weights for policy 1, policy_version 44191 (0.0008) [2023-12-27 05:58:13,277][20347] Updated weights for policy 0, policy_version 44213 (0.0006) [2023-12-27 05:58:13,325][20347] Updated weights for policy 0, policy_version 44223 (0.0006) [2023-12-27 05:58:13,376][20347] Updated weights for policy 0, policy_version 44233 (0.0005) [2023-12-27 05:58:13,896][20381] Updated weights for policy 1, policy_version 44201 (0.0007) [2023-12-27 05:58:13,929][20347] Updated weights for policy 0, policy_version 44243 (0.0005) [2023-12-27 05:58:13,947][20381] Updated weights for policy 1, policy_version 44211 (0.0005) [2023-12-27 05:58:13,979][20347] Updated weights for policy 0, policy_version 44253 (0.0006) [2023-12-27 05:58:13,997][20381] Updated weights for policy 1, policy_version 44221 (0.0005) [2023-12-27 05:58:14,028][20347] Updated weights for policy 0, policy_version 44263 (0.0007) [2023-12-27 05:58:14,052][20381] Updated weights for policy 1, policy_version 44231 (0.0005) [2023-12-27 05:58:14,665][20381] Updated weights for policy 1, policy_version 44241 (0.0005) [2023-12-27 05:58:14,714][20381] Updated weights for policy 1, policy_version 44251 (0.0007) [2023-12-27 05:58:14,746][20347] Updated weights for policy 0, policy_version 44273 (0.0008) [2023-12-27 05:58:14,767][20381] Updated weights for policy 1, policy_version 44261 (0.0007) [2023-12-27 05:58:14,799][20347] Updated weights for policy 0, policy_version 44283 (0.0008) [2023-12-27 05:58:14,850][20347] Updated weights for policy 0, policy_version 44293 (0.0005) [2023-12-27 05:58:14,915][20347] Updated weights for policy 0, policy_version 44303 (0.0006) [2023-12-27 05:58:15,435][20381] Updated weights for policy 1, policy_version 44271 (0.0007) [2023-12-27 05:58:15,493][20381] Updated weights for policy 1, policy_version 44281 (0.0009) [2023-12-27 05:58:15,566][20381] Updated weights for policy 1, policy_version 44291 (0.0008) [2023-12-27 05:58:15,598][20347] Updated weights for policy 0, policy_version 44313 (0.0007) [2023-12-27 05:58:15,648][20347] Updated weights for policy 0, policy_version 44323 (0.0009) [2023-12-27 05:58:15,700][20347] Updated weights for policy 0, policy_version 44333 (0.0008) [2023-12-27 05:58:16,156][20381] Updated weights for policy 1, policy_version 44301 (0.0007) [2023-12-27 05:58:16,212][20381] Updated weights for policy 1, policy_version 44311 (0.0005) [2023-12-27 05:58:16,267][20381] Updated weights for policy 1, policy_version 44321 (0.0005) [2023-12-27 05:58:16,438][20347] Updated weights for policy 0, policy_version 44343 (0.0008) [2023-12-27 05:58:16,497][20347] Updated weights for policy 0, policy_version 44353 (0.0008) [2023-12-27 05:58:16,558][20347] Updated weights for policy 0, policy_version 44363 (0.0008) [2023-12-27 05:58:16,893][20381] Updated weights for policy 1, policy_version 44331 (0.0006) [2023-12-27 05:58:16,958][20381] Updated weights for policy 1, policy_version 44341 (0.0006) [2023-12-27 05:58:17,018][20381] Updated weights for policy 1, policy_version 44351 (0.0006) [2023-12-27 05:58:17,214][20347] Updated weights for policy 0, policy_version 44373 (0.0007) [2023-12-27 05:58:17,267][20347] Updated weights for policy 0, policy_version 44383 (0.0009) [2023-12-27 05:58:17,317][20347] Updated weights for policy 0, policy_version 44393 (0.0008) [2023-12-27 05:58:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 22732800. Throughput: 0: 10456.6, 1: 10360.3. Samples: 22738984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:17,580][20381] Updated weights for policy 1, policy_version 44361 (0.0007) [2023-12-27 05:58:17,637][20381] Updated weights for policy 1, policy_version 44371 (0.0008) [2023-12-27 05:58:17,695][20381] Updated weights for policy 1, policy_version 44381 (0.0008) [2023-12-27 05:58:17,752][20381] Updated weights for policy 1, policy_version 44391 (0.0008) [2023-12-27 05:58:17,995][20347] Updated weights for policy 0, policy_version 44403 (0.0009) [2023-12-27 05:58:18,052][20347] Updated weights for policy 0, policy_version 44413 (0.0008) [2023-12-27 05:58:18,108][20347] Updated weights for policy 0, policy_version 44423 (0.0007) [2023-12-27 05:58:18,488][20381] Updated weights for policy 1, policy_version 44401 (0.0007) [2023-12-27 05:58:18,532][20381] Updated weights for policy 1, policy_version 44411 (0.0008) [2023-12-27 05:58:18,581][20381] Updated weights for policy 1, policy_version 44421 (0.0007) [2023-12-27 05:58:18,733][20347] Updated weights for policy 0, policy_version 44433 (0.0007) [2023-12-27 05:58:18,785][20347] Updated weights for policy 0, policy_version 44443 (0.0008) [2023-12-27 05:58:18,839][20347] Updated weights for policy 0, policy_version 44453 (0.0008) [2023-12-27 05:58:18,886][20347] Updated weights for policy 0, policy_version 44463 (0.0008) [2023-12-27 05:58:19,292][20381] Updated weights for policy 1, policy_version 44431 (0.0006) [2023-12-27 05:58:19,358][20381] Updated weights for policy 1, policy_version 44441 (0.0009) [2023-12-27 05:58:19,412][20381] Updated weights for policy 1, policy_version 44451 (0.0010) [2023-12-27 05:58:19,646][20347] Updated weights for policy 0, policy_version 44473 (0.0007) [2023-12-27 05:58:19,706][20347] Updated weights for policy 0, policy_version 44483 (0.0007) [2023-12-27 05:58:19,766][20347] Updated weights for policy 0, policy_version 44493 (0.0007) [2023-12-27 05:58:20,106][20381] Updated weights for policy 1, policy_version 44461 (0.0007) [2023-12-27 05:58:20,162][20381] Updated weights for policy 1, policy_version 44471 (0.0006) [2023-12-27 05:58:20,223][20381] Updated weights for policy 1, policy_version 44481 (0.0005) [2023-12-27 05:58:20,523][20347] Updated weights for policy 0, policy_version 44503 (0.0007) [2023-12-27 05:58:20,589][20347] Updated weights for policy 0, policy_version 44513 (0.0008) [2023-12-27 05:58:20,647][20347] Updated weights for policy 0, policy_version 44523 (0.0006) [2023-12-27 05:58:20,811][20381] Updated weights for policy 1, policy_version 44491 (0.0006) [2023-12-27 05:58:20,868][20381] Updated weights for policy 1, policy_version 44501 (0.0007) [2023-12-27 05:58:20,926][20381] Updated weights for policy 1, policy_version 44511 (0.0006) [2023-12-27 05:58:21,334][20347] Updated weights for policy 0, policy_version 44533 (0.0006) [2023-12-27 05:58:21,407][20347] Updated weights for policy 0, policy_version 44543 (0.0009) [2023-12-27 05:58:21,466][20347] Updated weights for policy 0, policy_version 44553 (0.0008) [2023-12-27 05:58:21,601][20381] Updated weights for policy 1, policy_version 44521 (0.0009) [2023-12-27 05:58:21,666][20381] Updated weights for policy 1, policy_version 44531 (0.0007) [2023-12-27 05:58:21,720][20381] Updated weights for policy 1, policy_version 44541 (0.0008) [2023-12-27 05:58:21,792][20381] Updated weights for policy 1, policy_version 44551 (0.0008) [2023-12-27 05:58:22,112][20347] Updated weights for policy 0, policy_version 44563 (0.0007) [2023-12-27 05:58:22,172][20347] Updated weights for policy 0, policy_version 44573 (0.0006) [2023-12-27 05:58:22,226][20347] Updated weights for policy 0, policy_version 44583 (0.0007) [2023-12-27 05:58:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 22831104. Throughput: 0: 10382.2, 1: 10361.3. Samples: 22799548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000044552_11411456.pth... [2023-12-27 05:58:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000044592_11419648.pth... [2023-12-27 05:58:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000043336_11100160.pth [2023-12-27 05:58:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000043376_11108352.pth [2023-12-27 05:58:22,448][20381] Updated weights for policy 1, policy_version 44561 (0.0007) [2023-12-27 05:58:22,513][20381] Updated weights for policy 1, policy_version 44571 (0.0005) [2023-12-27 05:58:22,572][20381] Updated weights for policy 1, policy_version 44581 (0.0006) [2023-12-27 05:58:22,962][20347] Updated weights for policy 0, policy_version 44593 (0.0008) [2023-12-27 05:58:23,017][20347] Updated weights for policy 0, policy_version 44603 (0.0008) [2023-12-27 05:58:23,074][20347] Updated weights for policy 0, policy_version 44613 (0.0007) [2023-12-27 05:58:23,128][20347] Updated weights for policy 0, policy_version 44623 (0.0007) [2023-12-27 05:58:23,254][20381] Updated weights for policy 1, policy_version 44591 (0.0009) [2023-12-27 05:58:23,298][20381] Updated weights for policy 1, policy_version 44601 (0.0010) [2023-12-27 05:58:23,344][20381] Updated weights for policy 1, policy_version 44611 (0.0007) [2023-12-27 05:58:23,688][20347] Updated weights for policy 0, policy_version 44633 (0.0006) [2023-12-27 05:58:23,745][20347] Updated weights for policy 0, policy_version 44643 (0.0005) [2023-12-27 05:58:23,793][20347] Updated weights for policy 0, policy_version 44653 (0.0005) [2023-12-27 05:58:23,930][20381] Updated weights for policy 1, policy_version 44621 (0.0005) [2023-12-27 05:58:23,979][20381] Updated weights for policy 1, policy_version 44631 (0.0007) [2023-12-27 05:58:24,028][20381] Updated weights for policy 1, policy_version 44641 (0.0007) [2023-12-27 05:58:24,404][20347] Updated weights for policy 0, policy_version 44663 (0.0007) [2023-12-27 05:58:24,456][20347] Updated weights for policy 0, policy_version 44673 (0.0007) [2023-12-27 05:58:24,515][20347] Updated weights for policy 0, policy_version 44683 (0.0008) [2023-12-27 05:58:24,708][20381] Updated weights for policy 1, policy_version 44651 (0.0005) [2023-12-27 05:58:24,765][20381] Updated weights for policy 1, policy_version 44661 (0.0005) [2023-12-27 05:58:24,824][20381] Updated weights for policy 1, policy_version 44671 (0.0005) [2023-12-27 05:58:25,101][20347] Updated weights for policy 0, policy_version 44693 (0.0006) [2023-12-27 05:58:25,149][20347] Updated weights for policy 0, policy_version 44703 (0.0005) [2023-12-27 05:58:25,209][20347] Updated weights for policy 0, policy_version 44713 (0.0005) [2023-12-27 05:58:25,387][20381] Updated weights for policy 1, policy_version 44681 (0.0007) [2023-12-27 05:58:25,447][20381] Updated weights for policy 1, policy_version 44691 (0.0010) [2023-12-27 05:58:25,505][20381] Updated weights for policy 1, policy_version 44701 (0.0005) [2023-12-27 05:58:25,559][20381] Updated weights for policy 1, policy_version 44711 (0.0005) [2023-12-27 05:58:25,790][20347] Updated weights for policy 0, policy_version 44723 (0.0007) [2023-12-27 05:58:25,848][20347] Updated weights for policy 0, policy_version 44733 (0.0010) [2023-12-27 05:58:25,902][20347] Updated weights for policy 0, policy_version 44743 (0.0010) [2023-12-27 05:58:26,149][20381] Updated weights for policy 1, policy_version 44721 (0.0007) [2023-12-27 05:58:26,200][20381] Updated weights for policy 1, policy_version 44731 (0.0008) [2023-12-27 05:58:26,254][20381] Updated weights for policy 1, policy_version 44741 (0.0006) [2023-12-27 05:58:26,513][20347] Updated weights for policy 0, policy_version 44753 (0.0010) [2023-12-27 05:58:26,571][20347] Updated weights for policy 0, policy_version 44763 (0.0011) [2023-12-27 05:58:26,631][20347] Updated weights for policy 0, policy_version 44773 (0.0010) [2023-12-27 05:58:26,691][20347] Updated weights for policy 0, policy_version 44783 (0.0010) [2023-12-27 05:58:26,866][20381] Updated weights for policy 1, policy_version 44751 (0.0005) [2023-12-27 05:58:26,920][20381] Updated weights for policy 1, policy_version 44761 (0.0005) [2023-12-27 05:58:26,968][20381] Updated weights for policy 1, policy_version 44771 (0.0005) [2023-12-27 05:58:27,355][20347] Updated weights for policy 0, policy_version 44793 (0.0010) [2023-12-27 05:58:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 22937600. Throughput: 0: 10412.5, 1: 10471.7. Samples: 22929040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:27,412][20347] Updated weights for policy 0, policy_version 44803 (0.0010) [2023-12-27 05:58:27,468][20347] Updated weights for policy 0, policy_version 44813 (0.0010) [2023-12-27 05:58:27,560][20381] Updated weights for policy 1, policy_version 44781 (0.0008) [2023-12-27 05:58:27,611][20381] Updated weights for policy 1, policy_version 44791 (0.0010) [2023-12-27 05:58:27,652][20381] Updated weights for policy 1, policy_version 44801 (0.0010) [2023-12-27 05:58:28,118][20347] Updated weights for policy 0, policy_version 44823 (0.0010) [2023-12-27 05:58:28,176][20347] Updated weights for policy 0, policy_version 44833 (0.0010) [2023-12-27 05:58:28,228][20347] Updated weights for policy 0, policy_version 44843 (0.0010) [2023-12-27 05:58:28,333][20381] Updated weights for policy 1, policy_version 44811 (0.0010) [2023-12-27 05:58:28,389][20381] Updated weights for policy 1, policy_version 44821 (0.0007) [2023-12-27 05:58:28,451][20381] Updated weights for policy 1, policy_version 44831 (0.0008) [2023-12-27 05:58:28,901][20347] Updated weights for policy 0, policy_version 44853 (0.0010) [2023-12-27 05:58:28,969][20347] Updated weights for policy 0, policy_version 44863 (0.0010) [2023-12-27 05:58:29,022][20347] Updated weights for policy 0, policy_version 44873 (0.0007) [2023-12-27 05:58:29,120][20381] Updated weights for policy 1, policy_version 44841 (0.0010) [2023-12-27 05:58:29,175][20381] Updated weights for policy 1, policy_version 44851 (0.0005) [2023-12-27 05:58:29,236][20381] Updated weights for policy 1, policy_version 44861 (0.0009) [2023-12-27 05:58:29,295][20381] Updated weights for policy 1, policy_version 44871 (0.0010) [2023-12-27 05:58:29,679][20347] Updated weights for policy 0, policy_version 44883 (0.0007) [2023-12-27 05:58:29,743][20347] Updated weights for policy 0, policy_version 44893 (0.0010) [2023-12-27 05:58:29,801][20347] Updated weights for policy 0, policy_version 44903 (0.0010) [2023-12-27 05:58:29,942][20381] Updated weights for policy 1, policy_version 44881 (0.0008) [2023-12-27 05:58:30,004][20381] Updated weights for policy 1, policy_version 44891 (0.0008) [2023-12-27 05:58:30,062][20381] Updated weights for policy 1, policy_version 44901 (0.0007) [2023-12-27 05:58:30,455][20347] Updated weights for policy 0, policy_version 44913 (0.0009) [2023-12-27 05:58:30,508][20347] Updated weights for policy 0, policy_version 44923 (0.0010) [2023-12-27 05:58:30,557][20347] Updated weights for policy 0, policy_version 44933 (0.0010) [2023-12-27 05:58:30,603][20347] Updated weights for policy 0, policy_version 44943 (0.0010) [2023-12-27 05:58:30,647][20381] Updated weights for policy 1, policy_version 44911 (0.0007) [2023-12-27 05:58:30,704][20381] Updated weights for policy 1, policy_version 44921 (0.0007) [2023-12-27 05:58:30,753][20381] Updated weights for policy 1, policy_version 44931 (0.0006) [2023-12-27 05:58:31,284][20347] Updated weights for policy 0, policy_version 44953 (0.0007) [2023-12-27 05:58:31,346][20347] Updated weights for policy 0, policy_version 44963 (0.0007) [2023-12-27 05:58:31,402][20347] Updated weights for policy 0, policy_version 44973 (0.0008) [2023-12-27 05:58:31,426][20381] Updated weights for policy 1, policy_version 44941 (0.0006) [2023-12-27 05:58:31,481][20381] Updated weights for policy 1, policy_version 44951 (0.0007) [2023-12-27 05:58:31,536][20381] Updated weights for policy 1, policy_version 44961 (0.0008) [2023-12-27 05:58:32,073][20347] Updated weights for policy 0, policy_version 44983 (0.0007) [2023-12-27 05:58:32,128][20347] Updated weights for policy 0, policy_version 44993 (0.0005) [2023-12-27 05:58:32,190][20347] Updated weights for policy 0, policy_version 45003 (0.0007) [2023-12-27 05:58:32,210][20381] Updated weights for policy 1, policy_version 44971 (0.0006) [2023-12-27 05:58:32,260][20381] Updated weights for policy 1, policy_version 44981 (0.0006) [2023-12-27 05:58:32,312][20381] Updated weights for policy 1, policy_version 44991 (0.0008) [2023-12-27 05:58:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 23044096. Throughput: 0: 10461.3, 1: 10584.5. Samples: 23056276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:32,794][20347] Updated weights for policy 0, policy_version 45013 (0.0010) [2023-12-27 05:58:32,850][20347] Updated weights for policy 0, policy_version 45023 (0.0008) [2023-12-27 05:58:32,910][20347] Updated weights for policy 0, policy_version 45033 (0.0008) [2023-12-27 05:58:33,002][20381] Updated weights for policy 1, policy_version 45001 (0.0007) [2023-12-27 05:58:33,061][20381] Updated weights for policy 1, policy_version 45011 (0.0007) [2023-12-27 05:58:33,106][20381] Updated weights for policy 1, policy_version 45021 (0.0008) [2023-12-27 05:58:33,154][20381] Updated weights for policy 1, policy_version 45031 (0.0008) [2023-12-27 05:58:33,506][20347] Updated weights for policy 0, policy_version 45043 (0.0007) [2023-12-27 05:58:33,556][20347] Updated weights for policy 0, policy_version 45053 (0.0007) [2023-12-27 05:58:33,602][20347] Updated weights for policy 0, policy_version 45063 (0.0007) [2023-12-27 05:58:33,853][20381] Updated weights for policy 1, policy_version 45041 (0.0007) [2023-12-27 05:58:33,901][20381] Updated weights for policy 1, policy_version 45051 (0.0008) [2023-12-27 05:58:33,946][20381] Updated weights for policy 1, policy_version 45061 (0.0008) [2023-12-27 05:58:34,239][20347] Updated weights for policy 0, policy_version 45073 (0.0007) [2023-12-27 05:58:34,301][20347] Updated weights for policy 0, policy_version 45083 (0.0010) [2023-12-27 05:58:34,356][20347] Updated weights for policy 0, policy_version 45093 (0.0010) [2023-12-27 05:58:34,415][20347] Updated weights for policy 0, policy_version 45103 (0.0010) [2023-12-27 05:58:34,641][20381] Updated weights for policy 1, policy_version 45071 (0.0008) [2023-12-27 05:58:34,691][20381] Updated weights for policy 1, policy_version 45081 (0.0005) [2023-12-27 05:58:34,741][20381] Updated weights for policy 1, policy_version 45091 (0.0008) [2023-12-27 05:58:35,039][20347] Updated weights for policy 0, policy_version 45113 (0.0006) [2023-12-27 05:58:35,096][20347] Updated weights for policy 0, policy_version 45123 (0.0005) [2023-12-27 05:58:35,141][20347] Updated weights for policy 0, policy_version 45133 (0.0005) [2023-12-27 05:58:35,386][20381] Updated weights for policy 1, policy_version 45101 (0.0009) [2023-12-27 05:58:35,441][20381] Updated weights for policy 1, policy_version 45111 (0.0011) [2023-12-27 05:58:35,488][20381] Updated weights for policy 1, policy_version 45121 (0.0005) [2023-12-27 05:58:35,711][20347] Updated weights for policy 0, policy_version 45143 (0.0007) [2023-12-27 05:58:35,769][20347] Updated weights for policy 0, policy_version 45153 (0.0007) [2023-12-27 05:58:35,824][20347] Updated weights for policy 0, policy_version 45163 (0.0007) [2023-12-27 05:58:36,072][20381] Updated weights for policy 1, policy_version 45131 (0.0007) [2023-12-27 05:58:36,135][20381] Updated weights for policy 1, policy_version 45141 (0.0011) [2023-12-27 05:58:36,192][20381] Updated weights for policy 1, policy_version 45151 (0.0009) [2023-12-27 05:58:36,498][20347] Updated weights for policy 0, policy_version 45173 (0.0007) [2023-12-27 05:58:36,558][20347] Updated weights for policy 0, policy_version 45183 (0.0005) [2023-12-27 05:58:36,623][20347] Updated weights for policy 0, policy_version 45193 (0.0006) [2023-12-27 05:58:36,876][20381] Updated weights for policy 1, policy_version 45161 (0.0010) [2023-12-27 05:58:36,925][20381] Updated weights for policy 1, policy_version 45171 (0.0011) [2023-12-27 05:58:36,977][20381] Updated weights for policy 1, policy_version 45181 (0.0009) [2023-12-27 05:58:37,022][20381] Updated weights for policy 1, policy_version 45191 (0.0010) [2023-12-27 05:58:37,214][20347] Updated weights for policy 0, policy_version 45203 (0.0007) [2023-12-27 05:58:37,268][20347] Updated weights for policy 0, policy_version 45213 (0.0010) [2023-12-27 05:58:37,316][20347] Updated weights for policy 0, policy_version 45223 (0.0010) [2023-12-27 05:58:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.7, 300 sec: 20827.1). Total num frames: 23150592. Throughput: 0: 10520.2, 1: 10577.7. Samples: 23120244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 05:58:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000045192_11575296.pth... [2023-12-27 05:58:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000043944_11255808.pth [2023-12-27 05:58:37,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000045232_11583488.pth... [2023-12-27 05:58:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000043984_11264000.pth [2023-12-27 05:58:37,625][20381] Updated weights for policy 1, policy_version 45201 (0.0011) [2023-12-27 05:58:37,671][20381] Updated weights for policy 1, policy_version 45211 (0.0010) [2023-12-27 05:58:37,720][20381] Updated weights for policy 1, policy_version 45221 (0.0010) [2023-12-27 05:58:38,005][20347] Updated weights for policy 0, policy_version 45233 (0.0010) [2023-12-27 05:58:38,054][20347] Updated weights for policy 0, policy_version 45243 (0.0010) [2023-12-27 05:58:38,109][20347] Updated weights for policy 0, policy_version 45253 (0.0010) [2023-12-27 05:58:38,161][20347] Updated weights for policy 0, policy_version 45263 (0.0010) [2023-12-27 05:58:38,422][20381] Updated weights for policy 1, policy_version 45231 (0.0011) [2023-12-27 05:58:38,472][20381] Updated weights for policy 1, policy_version 45241 (0.0006) [2023-12-27 05:58:38,517][20381] Updated weights for policy 1, policy_version 45251 (0.0005) [2023-12-27 05:58:38,862][20347] Updated weights for policy 0, policy_version 45273 (0.0010) [2023-12-27 05:58:38,916][20347] Updated weights for policy 0, policy_version 45283 (0.0010) [2023-12-27 05:58:38,977][20347] Updated weights for policy 0, policy_version 45293 (0.0010) [2023-12-27 05:58:39,213][20381] Updated weights for policy 1, policy_version 45261 (0.0008) [2023-12-27 05:58:39,274][20381] Updated weights for policy 1, policy_version 45271 (0.0011) [2023-12-27 05:58:39,323][20381] Updated weights for policy 1, policy_version 45281 (0.0010) [2023-12-27 05:58:39,708][20347] Updated weights for policy 0, policy_version 45303 (0.0009) [2023-12-27 05:58:39,767][20347] Updated weights for policy 0, policy_version 45313 (0.0008) [2023-12-27 05:58:39,828][20347] Updated weights for policy 0, policy_version 45323 (0.0008) [2023-12-27 05:58:40,032][20381] Updated weights for policy 1, policy_version 45291 (0.0011) [2023-12-27 05:58:40,091][20381] Updated weights for policy 1, policy_version 45301 (0.0006) [2023-12-27 05:58:40,152][20381] Updated weights for policy 1, policy_version 45311 (0.0006) [2023-12-27 05:58:40,602][20347] Updated weights for policy 0, policy_version 45333 (0.0007) [2023-12-27 05:58:40,660][20347] Updated weights for policy 0, policy_version 45343 (0.0007) [2023-12-27 05:58:40,705][20347] Updated weights for policy 0, policy_version 45353 (0.0010) [2023-12-27 05:58:40,830][20381] Updated weights for policy 1, policy_version 45321 (0.0007) [2023-12-27 05:58:40,895][20381] Updated weights for policy 1, policy_version 45331 (0.0010) [2023-12-27 05:58:40,959][20381] Updated weights for policy 1, policy_version 45341 (0.0010) [2023-12-27 05:58:41,015][20381] Updated weights for policy 1, policy_version 45351 (0.0008) [2023-12-27 05:58:41,455][20347] Updated weights for policy 0, policy_version 45363 (0.0009) [2023-12-27 05:58:41,521][20347] Updated weights for policy 0, policy_version 45373 (0.0007) [2023-12-27 05:58:41,586][20347] Updated weights for policy 0, policy_version 45383 (0.0008) [2023-12-27 05:58:41,757][20381] Updated weights for policy 1, policy_version 45361 (0.0007) [2023-12-27 05:58:41,820][20381] Updated weights for policy 1, policy_version 45371 (0.0007) [2023-12-27 05:58:41,881][20381] Updated weights for policy 1, policy_version 45381 (0.0008) [2023-12-27 05:58:42,354][20347] Updated weights for policy 0, policy_version 45393 (0.0007) [2023-12-27 05:58:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 23248896. Throughput: 0: 10456.8, 1: 10535.0. Samples: 23244140. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:58:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:42,418][20347] Updated weights for policy 0, policy_version 45403 (0.0008) [2023-12-27 05:58:42,462][20381] Updated weights for policy 1, policy_version 45391 (0.0007) [2023-12-27 05:58:42,475][20347] Updated weights for policy 0, policy_version 45413 (0.0006) [2023-12-27 05:58:42,527][20381] Updated weights for policy 1, policy_version 45401 (0.0007) [2023-12-27 05:58:42,538][20347] Updated weights for policy 0, policy_version 45423 (0.0008) [2023-12-27 05:58:42,591][20381] Updated weights for policy 1, policy_version 45411 (0.0006) [2023-12-27 05:58:43,158][20381] Updated weights for policy 1, policy_version 45421 (0.0006) [2023-12-27 05:58:43,204][20347] Updated weights for policy 0, policy_version 45433 (0.0006) [2023-12-27 05:58:43,218][20381] Updated weights for policy 1, policy_version 45431 (0.0005) [2023-12-27 05:58:43,259][20347] Updated weights for policy 0, policy_version 45443 (0.0005) [2023-12-27 05:58:43,271][20381] Updated weights for policy 1, policy_version 45441 (0.0005) [2023-12-27 05:58:43,308][20347] Updated weights for policy 0, policy_version 45453 (0.0005) [2023-12-27 05:58:43,864][20381] Updated weights for policy 1, policy_version 45451 (0.0006) [2023-12-27 05:58:43,914][20381] Updated weights for policy 1, policy_version 45461 (0.0006) [2023-12-27 05:58:43,915][20347] Updated weights for policy 0, policy_version 45463 (0.0009) [2023-12-27 05:58:43,964][20347] Updated weights for policy 0, policy_version 45473 (0.0010) [2023-12-27 05:58:43,969][20381] Updated weights for policy 1, policy_version 45471 (0.0005) [2023-12-27 05:58:44,010][20347] Updated weights for policy 0, policy_version 45483 (0.0010) [2023-12-27 05:58:44,693][20381] Updated weights for policy 1, policy_version 45481 (0.0006) [2023-12-27 05:58:44,713][20347] Updated weights for policy 0, policy_version 45493 (0.0009) [2023-12-27 05:58:44,749][20381] Updated weights for policy 1, policy_version 45491 (0.0009) [2023-12-27 05:58:44,769][20347] Updated weights for policy 0, policy_version 45503 (0.0010) [2023-12-27 05:58:44,809][20381] Updated weights for policy 1, policy_version 45501 (0.0008) [2023-12-27 05:58:44,833][20347] Updated weights for policy 0, policy_version 45513 (0.0008) [2023-12-27 05:58:44,868][20381] Updated weights for policy 1, policy_version 45511 (0.0006) [2023-12-27 05:58:45,541][20347] Updated weights for policy 0, policy_version 45523 (0.0010) [2023-12-27 05:58:45,593][20381] Updated weights for policy 1, policy_version 45521 (0.0007) [2023-12-27 05:58:45,599][20347] Updated weights for policy 0, policy_version 45533 (0.0010) [2023-12-27 05:58:45,648][20347] Updated weights for policy 0, policy_version 45543 (0.0010) [2023-12-27 05:58:45,649][20381] Updated weights for policy 1, policy_version 45531 (0.0005) [2023-12-27 05:58:45,697][20381] Updated weights for policy 1, policy_version 45541 (0.0005) [2023-12-27 05:58:46,333][20347] Updated weights for policy 0, policy_version 45553 (0.0010) [2023-12-27 05:58:46,387][20347] Updated weights for policy 0, policy_version 45563 (0.0010) [2023-12-27 05:58:46,389][20381] Updated weights for policy 1, policy_version 45551 (0.0006) [2023-12-27 05:58:46,442][20347] Updated weights for policy 0, policy_version 45573 (0.0010) [2023-12-27 05:58:46,448][20381] Updated weights for policy 1, policy_version 45561 (0.0005) [2023-12-27 05:58:46,493][20347] Updated weights for policy 0, policy_version 45583 (0.0010) [2023-12-27 05:58:46,499][20381] Updated weights for policy 1, policy_version 45571 (0.0005) [2023-12-27 05:58:47,180][20347] Updated weights for policy 0, policy_version 45593 (0.0010) [2023-12-27 05:58:47,199][20381] Updated weights for policy 1, policy_version 45581 (0.0008) [2023-12-27 05:58:47,235][20347] Updated weights for policy 0, policy_version 45603 (0.0010) [2023-12-27 05:58:47,258][20381] Updated weights for policy 1, policy_version 45591 (0.0006) [2023-12-27 05:58:47,294][20347] Updated weights for policy 0, policy_version 45613 (0.0010) [2023-12-27 05:58:47,312][20381] Updated weights for policy 1, policy_version 45601 (0.0006) [2023-12-27 05:58:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21162.6, 300 sec: 20827.1). Total num frames: 23363584. Throughput: 0: 10403.4, 1: 10554.4. Samples: 23367252. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:58:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:47,935][20347] Updated weights for policy 0, policy_version 45623 (0.0006) [2023-12-27 05:58:47,996][20347] Updated weights for policy 0, policy_version 45633 (0.0007) [2023-12-27 05:58:48,052][20347] Updated weights for policy 0, policy_version 45643 (0.0010) [2023-12-27 05:58:48,061][20381] Updated weights for policy 1, policy_version 45611 (0.0007) [2023-12-27 05:58:48,114][20381] Updated weights for policy 1, policy_version 45621 (0.0005) [2023-12-27 05:58:48,171][20381] Updated weights for policy 1, policy_version 45631 (0.0006) [2023-12-27 05:58:48,689][20347] Updated weights for policy 0, policy_version 45653 (0.0011) [2023-12-27 05:58:48,747][20347] Updated weights for policy 0, policy_version 45663 (0.0010) [2023-12-27 05:58:48,800][20347] Updated weights for policy 0, policy_version 45673 (0.0005) [2023-12-27 05:58:48,871][20381] Updated weights for policy 1, policy_version 45641 (0.0007) [2023-12-27 05:58:48,934][20381] Updated weights for policy 1, policy_version 45651 (0.0007) [2023-12-27 05:58:49,000][20381] Updated weights for policy 1, policy_version 45661 (0.0007) [2023-12-27 05:58:49,062][20381] Updated weights for policy 1, policy_version 45671 (0.0007) [2023-12-27 05:58:49,509][20347] Updated weights for policy 0, policy_version 45683 (0.0010) [2023-12-27 05:58:49,564][20347] Updated weights for policy 0, policy_version 45693 (0.0010) [2023-12-27 05:58:49,625][20347] Updated weights for policy 0, policy_version 45703 (0.0010) [2023-12-27 05:58:49,705][20381] Updated weights for policy 1, policy_version 45681 (0.0007) [2023-12-27 05:58:49,756][20381] Updated weights for policy 1, policy_version 45691 (0.0007) [2023-12-27 05:58:49,808][20381] Updated weights for policy 1, policy_version 45701 (0.0007) [2023-12-27 05:58:50,315][20347] Updated weights for policy 0, policy_version 45713 (0.0010) [2023-12-27 05:58:50,375][20347] Updated weights for policy 0, policy_version 45723 (0.0007) [2023-12-27 05:58:50,434][20347] Updated weights for policy 0, policy_version 45733 (0.0008) [2023-12-27 05:58:50,498][20347] Updated weights for policy 0, policy_version 45743 (0.0009) [2023-12-27 05:58:50,508][20381] Updated weights for policy 1, policy_version 45711 (0.0009) [2023-12-27 05:58:50,565][20381] Updated weights for policy 1, policy_version 45721 (0.0010) [2023-12-27 05:58:50,624][20381] Updated weights for policy 1, policy_version 45731 (0.0010) [2023-12-27 05:58:51,173][20347] Updated weights for policy 0, policy_version 45753 (0.0006) [2023-12-27 05:58:51,227][20347] Updated weights for policy 0, policy_version 45763 (0.0006) [2023-12-27 05:58:51,253][20381] Updated weights for policy 1, policy_version 45741 (0.0008) [2023-12-27 05:58:51,290][20347] Updated weights for policy 0, policy_version 45773 (0.0007) [2023-12-27 05:58:51,313][20381] Updated weights for policy 1, policy_version 45751 (0.0011) [2023-12-27 05:58:51,378][20381] Updated weights for policy 1, policy_version 45761 (0.0008) [2023-12-27 05:58:51,953][20347] Updated weights for policy 0, policy_version 45783 (0.0008) [2023-12-27 05:58:52,013][20347] Updated weights for policy 0, policy_version 45793 (0.0007) [2023-12-27 05:58:52,047][20381] Updated weights for policy 1, policy_version 45771 (0.0006) [2023-12-27 05:58:52,069][20347] Updated weights for policy 0, policy_version 45803 (0.0007) [2023-12-27 05:58:52,093][20381] Updated weights for policy 1, policy_version 45781 (0.0008) [2023-12-27 05:58:52,148][20381] Updated weights for policy 1, policy_version 45791 (0.0008) [2023-12-27 05:58:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 23461888. Throughput: 0: 10347.8, 1: 10496.1. Samples: 23427332. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:58:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000045800_11730944.pth... [2023-12-27 05:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000045808_11730944.pth... [2023-12-27 05:58:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000044552_11411456.pth [2023-12-27 05:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000044592_11419648.pth [2023-12-27 05:58:52,718][20347] Updated weights for policy 0, policy_version 45813 (0.0006) [2023-12-27 05:58:52,764][20347] Updated weights for policy 0, policy_version 45823 (0.0005) [2023-12-27 05:58:52,817][20347] Updated weights for policy 0, policy_version 45833 (0.0005) [2023-12-27 05:58:52,865][20381] Updated weights for policy 1, policy_version 45801 (0.0007) [2023-12-27 05:58:52,910][20381] Updated weights for policy 1, policy_version 45811 (0.0008) [2023-12-27 05:58:52,959][20381] Updated weights for policy 1, policy_version 45821 (0.0006) [2023-12-27 05:58:53,014][20381] Updated weights for policy 1, policy_version 45831 (0.0008) [2023-12-27 05:58:53,338][20347] Updated weights for policy 0, policy_version 45843 (0.0005) [2023-12-27 05:58:53,388][20347] Updated weights for policy 0, policy_version 45853 (0.0005) [2023-12-27 05:58:53,451][20347] Updated weights for policy 0, policy_version 45863 (0.0005) [2023-12-27 05:58:53,785][20381] Updated weights for policy 1, policy_version 45841 (0.0010) [2023-12-27 05:58:53,841][20381] Updated weights for policy 1, policy_version 45851 (0.0005) [2023-12-27 05:58:53,890][20381] Updated weights for policy 1, policy_version 45861 (0.0005) [2023-12-27 05:58:54,016][20347] Updated weights for policy 0, policy_version 45873 (0.0005) [2023-12-27 05:58:54,072][20347] Updated weights for policy 0, policy_version 45883 (0.0005) [2023-12-27 05:58:54,126][20347] Updated weights for policy 0, policy_version 45893 (0.0005) [2023-12-27 05:58:54,179][20347] Updated weights for policy 0, policy_version 45903 (0.0008) [2023-12-27 05:58:54,482][20381] Updated weights for policy 1, policy_version 45871 (0.0006) [2023-12-27 05:58:54,547][20381] Updated weights for policy 1, policy_version 45881 (0.0006) [2023-12-27 05:58:54,611][20381] Updated weights for policy 1, policy_version 45891 (0.0007) [2023-12-27 05:58:54,827][20347] Updated weights for policy 0, policy_version 45913 (0.0009) [2023-12-27 05:58:54,880][20347] Updated weights for policy 0, policy_version 45923 (0.0006) [2023-12-27 05:58:54,938][20347] Updated weights for policy 0, policy_version 45933 (0.0007) [2023-12-27 05:58:55,185][20381] Updated weights for policy 1, policy_version 45901 (0.0008) [2023-12-27 05:58:55,232][20381] Updated weights for policy 1, policy_version 45911 (0.0007) [2023-12-27 05:58:55,284][20381] Updated weights for policy 1, policy_version 45921 (0.0007) [2023-12-27 05:58:55,569][20347] Updated weights for policy 0, policy_version 45943 (0.0007) [2023-12-27 05:58:55,629][20347] Updated weights for policy 0, policy_version 45953 (0.0006) [2023-12-27 05:58:55,697][20347] Updated weights for policy 0, policy_version 45963 (0.0007) [2023-12-27 05:58:55,961][20381] Updated weights for policy 1, policy_version 45931 (0.0007) [2023-12-27 05:58:56,018][20381] Updated weights for policy 1, policy_version 45941 (0.0007) [2023-12-27 05:58:56,077][20381] Updated weights for policy 1, policy_version 45951 (0.0008) [2023-12-27 05:58:56,308][20347] Updated weights for policy 0, policy_version 45973 (0.0006) [2023-12-27 05:58:56,360][20347] Updated weights for policy 0, policy_version 45983 (0.0009) [2023-12-27 05:58:56,416][20347] Updated weights for policy 0, policy_version 45993 (0.0007) [2023-12-27 05:58:56,740][20381] Updated weights for policy 1, policy_version 45961 (0.0006) [2023-12-27 05:58:56,785][20381] Updated weights for policy 1, policy_version 45971 (0.0008) [2023-12-27 05:58:56,830][20381] Updated weights for policy 1, policy_version 45981 (0.0007) [2023-12-27 05:58:56,873][20381] Updated weights for policy 1, policy_version 45991 (0.0005) [2023-12-27 05:58:57,136][20347] Updated weights for policy 0, policy_version 46003 (0.0008) [2023-12-27 05:58:57,190][20347] Updated weights for policy 0, policy_version 46013 (0.0008) [2023-12-27 05:58:57,244][20347] Updated weights for policy 0, policy_version 46023 (0.0006) [2023-12-27 05:58:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21028.2, 300 sec: 20799.3). Total num frames: 23568384. Throughput: 0: 10476.1, 1: 10535.9. Samples: 23555872. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:58:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:58:57,488][20381] Updated weights for policy 1, policy_version 46001 (0.0005) [2023-12-27 05:58:57,533][20381] Updated weights for policy 1, policy_version 46011 (0.0005) [2023-12-27 05:58:57,575][20381] Updated weights for policy 1, policy_version 46021 (0.0005) [2023-12-27 05:58:57,906][20347] Updated weights for policy 0, policy_version 46033 (0.0006) [2023-12-27 05:58:57,966][20347] Updated weights for policy 0, policy_version 46043 (0.0010) [2023-12-27 05:58:58,011][20347] Updated weights for policy 0, policy_version 46053 (0.0010) [2023-12-27 05:58:58,064][20347] Updated weights for policy 0, policy_version 46063 (0.0007) [2023-12-27 05:58:58,212][20381] Updated weights for policy 1, policy_version 46031 (0.0010) [2023-12-27 05:58:58,260][20381] Updated weights for policy 1, policy_version 46041 (0.0007) [2023-12-27 05:58:58,310][20381] Updated weights for policy 1, policy_version 46051 (0.0007) [2023-12-27 05:58:58,772][20347] Updated weights for policy 0, policy_version 46073 (0.0010) [2023-12-27 05:58:58,836][20347] Updated weights for policy 0, policy_version 46083 (0.0008) [2023-12-27 05:58:58,906][20347] Updated weights for policy 0, policy_version 46093 (0.0008) [2023-12-27 05:58:59,163][20381] Updated weights for policy 1, policy_version 46061 (0.0008) [2023-12-27 05:58:59,227][20381] Updated weights for policy 1, policy_version 46071 (0.0008) [2023-12-27 05:58:59,281][20381] Updated weights for policy 1, policy_version 46081 (0.0008) [2023-12-27 05:58:59,560][20347] Updated weights for policy 0, policy_version 46103 (0.0006) [2023-12-27 05:58:59,618][20347] Updated weights for policy 0, policy_version 46113 (0.0006) [2023-12-27 05:58:59,671][20347] Updated weights for policy 0, policy_version 46123 (0.0007) [2023-12-27 05:58:59,985][20381] Updated weights for policy 1, policy_version 46091 (0.0008) [2023-12-27 05:59:00,035][20381] Updated weights for policy 1, policy_version 46101 (0.0007) [2023-12-27 05:59:00,091][20381] Updated weights for policy 1, policy_version 46111 (0.0007) [2023-12-27 05:59:00,270][20347] Updated weights for policy 0, policy_version 46133 (0.0005) [2023-12-27 05:59:00,327][20347] Updated weights for policy 0, policy_version 46143 (0.0006) [2023-12-27 05:59:00,375][20347] Updated weights for policy 0, policy_version 46153 (0.0007) [2023-12-27 05:59:00,795][20381] Updated weights for policy 1, policy_version 46121 (0.0008) [2023-12-27 05:59:00,842][20381] Updated weights for policy 1, policy_version 46131 (0.0010) [2023-12-27 05:59:00,900][20381] Updated weights for policy 1, policy_version 46141 (0.0010) [2023-12-27 05:59:00,958][20381] Updated weights for policy 1, policy_version 46151 (0.0008) [2023-12-27 05:59:00,975][20347] Updated weights for policy 0, policy_version 46163 (0.0007) [2023-12-27 05:59:01,032][20347] Updated weights for policy 0, policy_version 46173 (0.0008) [2023-12-27 05:59:01,091][20347] Updated weights for policy 0, policy_version 46183 (0.0008) [2023-12-27 05:59:01,638][20381] Updated weights for policy 1, policy_version 46161 (0.0008) [2023-12-27 05:59:01,696][20381] Updated weights for policy 1, policy_version 46171 (0.0008) [2023-12-27 05:59:01,765][20381] Updated weights for policy 1, policy_version 46181 (0.0008) [2023-12-27 05:59:01,804][20347] Updated weights for policy 0, policy_version 46193 (0.0008) [2023-12-27 05:59:01,861][20347] Updated weights for policy 0, policy_version 46203 (0.0008) [2023-12-27 05:59:01,912][20347] Updated weights for policy 0, policy_version 46213 (0.0008) [2023-12-27 05:59:01,961][20347] Updated weights for policy 0, policy_version 46223 (0.0008) [2023-12-27 05:59:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 23666688. Throughput: 0: 10451.9, 1: 10433.4. Samples: 23678824. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:59:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:02,441][20381] Updated weights for policy 1, policy_version 46191 (0.0008) [2023-12-27 05:59:02,502][20381] Updated weights for policy 1, policy_version 46201 (0.0009) [2023-12-27 05:59:02,566][20381] Updated weights for policy 1, policy_version 46211 (0.0008) [2023-12-27 05:59:02,604][20347] Updated weights for policy 0, policy_version 46233 (0.0005) [2023-12-27 05:59:02,655][20347] Updated weights for policy 0, policy_version 46243 (0.0005) [2023-12-27 05:59:02,708][20347] Updated weights for policy 0, policy_version 46253 (0.0005) [2023-12-27 05:59:03,229][20381] Updated weights for policy 1, policy_version 46221 (0.0006) [2023-12-27 05:59:03,281][20381] Updated weights for policy 1, policy_version 46231 (0.0005) [2023-12-27 05:59:03,341][20381] Updated weights for policy 1, policy_version 46241 (0.0007) [2023-12-27 05:59:03,369][20347] Updated weights for policy 0, policy_version 46263 (0.0010) [2023-12-27 05:59:03,414][20347] Updated weights for policy 0, policy_version 46273 (0.0010) [2023-12-27 05:59:03,460][20347] Updated weights for policy 0, policy_version 46283 (0.0010) [2023-12-27 05:59:03,956][20381] Updated weights for policy 1, policy_version 46251 (0.0006) [2023-12-27 05:59:04,014][20381] Updated weights for policy 1, policy_version 46261 (0.0009) [2023-12-27 05:59:04,102][20381] Updated weights for policy 1, policy_version 46271 (0.0010) [2023-12-27 05:59:04,128][20347] Updated weights for policy 0, policy_version 46293 (0.0010) [2023-12-27 05:59:04,187][20347] Updated weights for policy 0, policy_version 46303 (0.0011) [2023-12-27 05:59:04,249][20347] Updated weights for policy 0, policy_version 46313 (0.0007) [2023-12-27 05:59:04,755][20381] Updated weights for policy 1, policy_version 46281 (0.0010) [2023-12-27 05:59:04,804][20381] Updated weights for policy 1, policy_version 46291 (0.0007) [2023-12-27 05:59:04,852][20381] Updated weights for policy 1, policy_version 46301 (0.0005) [2023-12-27 05:59:04,900][20381] Updated weights for policy 1, policy_version 46311 (0.0006) [2023-12-27 05:59:04,939][20347] Updated weights for policy 0, policy_version 46323 (0.0007) [2023-12-27 05:59:04,992][20347] Updated weights for policy 0, policy_version 46333 (0.0008) [2023-12-27 05:59:05,049][20347] Updated weights for policy 0, policy_version 46343 (0.0006) [2023-12-27 05:59:05,501][20381] Updated weights for policy 1, policy_version 46321 (0.0007) [2023-12-27 05:59:05,546][20381] Updated weights for policy 1, policy_version 46331 (0.0005) [2023-12-27 05:59:05,592][20381] Updated weights for policy 1, policy_version 46341 (0.0005) [2023-12-27 05:59:05,702][20347] Updated weights for policy 0, policy_version 46353 (0.0007) [2023-12-27 05:59:05,759][20347] Updated weights for policy 0, policy_version 46363 (0.0007) [2023-12-27 05:59:05,806][20347] Updated weights for policy 0, policy_version 46373 (0.0009) [2023-12-27 05:59:05,863][20347] Updated weights for policy 0, policy_version 46383 (0.0010) [2023-12-27 05:59:06,251][20381] Updated weights for policy 1, policy_version 46351 (0.0008) [2023-12-27 05:59:06,302][20381] Updated weights for policy 1, policy_version 46361 (0.0008) [2023-12-27 05:59:06,358][20381] Updated weights for policy 1, policy_version 46371 (0.0006) [2023-12-27 05:59:06,502][20347] Updated weights for policy 0, policy_version 46393 (0.0008) [2023-12-27 05:59:06,554][20347] Updated weights for policy 0, policy_version 46403 (0.0008) [2023-12-27 05:59:06,614][20347] Updated weights for policy 0, policy_version 46413 (0.0008) [2023-12-27 05:59:06,962][20381] Updated weights for policy 1, policy_version 46381 (0.0007) [2023-12-27 05:59:07,015][20381] Updated weights for policy 1, policy_version 46391 (0.0008) [2023-12-27 05:59:07,064][20381] Updated weights for policy 1, policy_version 46401 (0.0009) [2023-12-27 05:59:07,236][20347] Updated weights for policy 0, policy_version 46423 (0.0007) [2023-12-27 05:59:07,298][20347] Updated weights for policy 0, policy_version 46433 (0.0008) [2023-12-27 05:59:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 23773184. Throughput: 0: 10486.9, 1: 10468.7. Samples: 23742544. Policy #0 lag: (min: 31.0, avg: 32.0, max: 56.0) [2023-12-27 05:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:07,364][20347] Updated weights for policy 0, policy_version 46443 (0.0008) [2023-12-27 05:59:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000046408_11886592.pth... [2023-12-27 05:59:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000045192_11575296.pth [2023-12-27 05:59:07,392][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000046448_11894784.pth... [2023-12-27 05:59:07,396][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000045232_11583488.pth [2023-12-27 05:59:07,767][20381] Updated weights for policy 1, policy_version 46411 (0.0008) [2023-12-27 05:59:07,812][20381] Updated weights for policy 1, policy_version 46421 (0.0009) [2023-12-27 05:59:07,864][20381] Updated weights for policy 1, policy_version 46431 (0.0007) [2023-12-27 05:59:08,025][20347] Updated weights for policy 0, policy_version 46453 (0.0008) [2023-12-27 05:59:08,081][20347] Updated weights for policy 0, policy_version 46463 (0.0007) [2023-12-27 05:59:08,138][20347] Updated weights for policy 0, policy_version 46473 (0.0007) [2023-12-27 05:59:08,560][20381] Updated weights for policy 1, policy_version 46441 (0.0007) [2023-12-27 05:59:08,621][20381] Updated weights for policy 1, policy_version 46451 (0.0008) [2023-12-27 05:59:08,682][20381] Updated weights for policy 1, policy_version 46461 (0.0007) [2023-12-27 05:59:08,744][20381] Updated weights for policy 1, policy_version 46471 (0.0005) [2023-12-27 05:59:08,857][20347] Updated weights for policy 0, policy_version 46483 (0.0007) [2023-12-27 05:59:08,913][20347] Updated weights for policy 0, policy_version 46493 (0.0007) [2023-12-27 05:59:08,968][20347] Updated weights for policy 0, policy_version 46503 (0.0008) [2023-12-27 05:59:09,389][20381] Updated weights for policy 1, policy_version 46481 (0.0006) [2023-12-27 05:59:09,452][20381] Updated weights for policy 1, policy_version 46491 (0.0008) [2023-12-27 05:59:09,510][20381] Updated weights for policy 1, policy_version 46501 (0.0007) [2023-12-27 05:59:09,723][20347] Updated weights for policy 0, policy_version 46513 (0.0007) [2023-12-27 05:59:09,771][20347] Updated weights for policy 0, policy_version 46523 (0.0007) [2023-12-27 05:59:09,825][20347] Updated weights for policy 0, policy_version 46533 (0.0006) [2023-12-27 05:59:09,890][20347] Updated weights for policy 0, policy_version 46543 (0.0006) [2023-12-27 05:59:10,151][20381] Updated weights for policy 1, policy_version 46511 (0.0008) [2023-12-27 05:59:10,213][20381] Updated weights for policy 1, policy_version 46521 (0.0007) [2023-12-27 05:59:10,273][20381] Updated weights for policy 1, policy_version 46531 (0.0008) [2023-12-27 05:59:10,574][20347] Updated weights for policy 0, policy_version 46553 (0.0009) [2023-12-27 05:59:10,629][20347] Updated weights for policy 0, policy_version 46563 (0.0008) [2023-12-27 05:59:10,679][20347] Updated weights for policy 0, policy_version 46573 (0.0006) [2023-12-27 05:59:10,824][20381] Updated weights for policy 1, policy_version 46541 (0.0006) [2023-12-27 05:59:10,881][20381] Updated weights for policy 1, policy_version 46551 (0.0005) [2023-12-27 05:59:10,934][20381] Updated weights for policy 1, policy_version 46561 (0.0005) [2023-12-27 05:59:11,415][20347] Updated weights for policy 0, policy_version 46583 (0.0009) [2023-12-27 05:59:11,475][20347] Updated weights for policy 0, policy_version 46593 (0.0011) [2023-12-27 05:59:11,536][20347] Updated weights for policy 0, policy_version 46603 (0.0009) [2023-12-27 05:59:11,601][20381] Updated weights for policy 1, policy_version 46571 (0.0006) [2023-12-27 05:59:11,659][20381] Updated weights for policy 1, policy_version 46581 (0.0007) [2023-12-27 05:59:11,716][20381] Updated weights for policy 1, policy_version 46591 (0.0006) [2023-12-27 05:59:12,234][20347] Updated weights for policy 0, policy_version 46613 (0.0010) [2023-12-27 05:59:12,298][20347] Updated weights for policy 0, policy_version 46623 (0.0010) [2023-12-27 05:59:12,362][20347] Updated weights for policy 0, policy_version 46633 (0.0011) [2023-12-27 05:59:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 23871488. Throughput: 0: 10419.1, 1: 10449.2. Samples: 23868116. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:12,401][20381] Updated weights for policy 1, policy_version 46601 (0.0008) [2023-12-27 05:59:12,468][20381] Updated weights for policy 1, policy_version 46611 (0.0005) [2023-12-27 05:59:12,534][20381] Updated weights for policy 1, policy_version 46621 (0.0007) [2023-12-27 05:59:12,594][20381] Updated weights for policy 1, policy_version 46631 (0.0007) [2023-12-27 05:59:13,048][20347] Updated weights for policy 0, policy_version 46643 (0.0009) [2023-12-27 05:59:13,109][20347] Updated weights for policy 0, policy_version 46653 (0.0007) [2023-12-27 05:59:13,167][20347] Updated weights for policy 0, policy_version 46663 (0.0007) [2023-12-27 05:59:13,192][20381] Updated weights for policy 1, policy_version 46641 (0.0006) [2023-12-27 05:59:13,245][20381] Updated weights for policy 1, policy_version 46651 (0.0007) [2023-12-27 05:59:13,295][20381] Updated weights for policy 1, policy_version 46661 (0.0008) [2023-12-27 05:59:13,783][20347] Updated weights for policy 0, policy_version 46673 (0.0007) [2023-12-27 05:59:13,837][20347] Updated weights for policy 0, policy_version 46683 (0.0005) [2023-12-27 05:59:13,883][20347] Updated weights for policy 0, policy_version 46693 (0.0005) [2023-12-27 05:59:13,902][20381] Updated weights for policy 1, policy_version 46671 (0.0006) [2023-12-27 05:59:13,933][20347] Updated weights for policy 0, policy_version 46703 (0.0005) [2023-12-27 05:59:13,961][20381] Updated weights for policy 1, policy_version 46681 (0.0005) [2023-12-27 05:59:14,014][20381] Updated weights for policy 1, policy_version 46691 (0.0007) [2023-12-27 05:59:14,474][20347] Updated weights for policy 0, policy_version 46713 (0.0007) [2023-12-27 05:59:14,531][20347] Updated weights for policy 0, policy_version 46723 (0.0010) [2023-12-27 05:59:14,589][20347] Updated weights for policy 0, policy_version 46733 (0.0010) [2023-12-27 05:59:14,665][20381] Updated weights for policy 1, policy_version 46701 (0.0011) [2023-12-27 05:59:14,714][20381] Updated weights for policy 1, policy_version 46711 (0.0010) [2023-12-27 05:59:14,762][20381] Updated weights for policy 1, policy_version 46721 (0.0010) [2023-12-27 05:59:15,300][20347] Updated weights for policy 0, policy_version 46743 (0.0010) [2023-12-27 05:59:15,355][20347] Updated weights for policy 0, policy_version 46753 (0.0010) [2023-12-27 05:59:15,409][20347] Updated weights for policy 0, policy_version 46763 (0.0010) [2023-12-27 05:59:15,529][20381] Updated weights for policy 1, policy_version 46731 (0.0008) [2023-12-27 05:59:15,581][20381] Updated weights for policy 1, policy_version 46741 (0.0008) [2023-12-27 05:59:15,630][20381] Updated weights for policy 1, policy_version 46751 (0.0007) [2023-12-27 05:59:16,053][20347] Updated weights for policy 0, policy_version 46773 (0.0010) [2023-12-27 05:59:16,102][20347] Updated weights for policy 0, policy_version 46783 (0.0010) [2023-12-27 05:59:16,160][20347] Updated weights for policy 0, policy_version 46793 (0.0010) [2023-12-27 05:59:16,249][20381] Updated weights for policy 1, policy_version 46761 (0.0008) [2023-12-27 05:59:16,308][20381] Updated weights for policy 1, policy_version 46771 (0.0011) [2023-12-27 05:59:16,361][20381] Updated weights for policy 1, policy_version 46781 (0.0011) [2023-12-27 05:59:16,410][20381] Updated weights for policy 1, policy_version 46791 (0.0010) [2023-12-27 05:59:16,853][20347] Updated weights for policy 0, policy_version 46803 (0.0010) [2023-12-27 05:59:16,907][20347] Updated weights for policy 0, policy_version 46813 (0.0010) [2023-12-27 05:59:16,971][20347] Updated weights for policy 0, policy_version 46823 (0.0010) [2023-12-27 05:59:17,013][20381] Updated weights for policy 1, policy_version 46801 (0.0006) [2023-12-27 05:59:17,068][20381] Updated weights for policy 1, policy_version 46811 (0.0007) [2023-12-27 05:59:17,124][20381] Updated weights for policy 1, policy_version 46821 (0.0007) [2023-12-27 05:59:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 23986176. Throughput: 0: 10416.5, 1: 10421.7. Samples: 23993996. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:17,630][20347] Updated weights for policy 0, policy_version 46833 (0.0010) [2023-12-27 05:59:17,690][20347] Updated weights for policy 0, policy_version 46843 (0.0010) [2023-12-27 05:59:17,707][20381] Updated weights for policy 1, policy_version 46831 (0.0006) [2023-12-27 05:59:17,744][20347] Updated weights for policy 0, policy_version 46853 (0.0010) [2023-12-27 05:59:17,756][20381] Updated weights for policy 1, policy_version 46841 (0.0009) [2023-12-27 05:59:17,799][20347] Updated weights for policy 0, policy_version 46863 (0.0010) [2023-12-27 05:59:17,803][20381] Updated weights for policy 1, policy_version 46851 (0.0005) [2023-12-27 05:59:18,372][20381] Updated weights for policy 1, policy_version 46861 (0.0007) [2023-12-27 05:59:18,427][20381] Updated weights for policy 1, policy_version 46871 (0.0009) [2023-12-27 05:59:18,429][20347] Updated weights for policy 0, policy_version 46873 (0.0007) [2023-12-27 05:59:18,482][20381] Updated weights for policy 1, policy_version 46881 (0.0006) [2023-12-27 05:59:18,484][20347] Updated weights for policy 0, policy_version 46883 (0.0010) [2023-12-27 05:59:18,538][20347] Updated weights for policy 0, policy_version 46893 (0.0010) [2023-12-27 05:59:19,219][20381] Updated weights for policy 1, policy_version 46891 (0.0006) [2023-12-27 05:59:19,258][20347] Updated weights for policy 0, policy_version 46903 (0.0010) [2023-12-27 05:59:19,282][20381] Updated weights for policy 1, policy_version 46901 (0.0008) [2023-12-27 05:59:19,315][20347] Updated weights for policy 0, policy_version 46913 (0.0010) [2023-12-27 05:59:19,345][20381] Updated weights for policy 1, policy_version 46911 (0.0008) [2023-12-27 05:59:19,380][20347] Updated weights for policy 0, policy_version 46923 (0.0011) [2023-12-27 05:59:20,059][20347] Updated weights for policy 0, policy_version 46933 (0.0009) [2023-12-27 05:59:20,097][20381] Updated weights for policy 1, policy_version 46921 (0.0008) [2023-12-27 05:59:20,118][20347] Updated weights for policy 0, policy_version 46943 (0.0006) [2023-12-27 05:59:20,161][20381] Updated weights for policy 1, policy_version 46931 (0.0008) [2023-12-27 05:59:20,180][20347] Updated weights for policy 0, policy_version 46953 (0.0007) [2023-12-27 05:59:20,215][20381] Updated weights for policy 1, policy_version 46941 (0.0008) [2023-12-27 05:59:20,271][20381] Updated weights for policy 1, policy_version 46951 (0.0007) [2023-12-27 05:59:20,888][20347] Updated weights for policy 0, policy_version 46963 (0.0008) [2023-12-27 05:59:20,945][20347] Updated weights for policy 0, policy_version 46973 (0.0011) [2023-12-27 05:59:20,983][20381] Updated weights for policy 1, policy_version 46961 (0.0009) [2023-12-27 05:59:21,005][20347] Updated weights for policy 0, policy_version 46983 (0.0008) [2023-12-27 05:59:21,044][20381] Updated weights for policy 1, policy_version 46971 (0.0008) [2023-12-27 05:59:21,106][20381] Updated weights for policy 1, policy_version 46981 (0.0006) [2023-12-27 05:59:21,677][20347] Updated weights for policy 0, policy_version 46993 (0.0010) [2023-12-27 05:59:21,745][20347] Updated weights for policy 0, policy_version 47003 (0.0011) [2023-12-27 05:59:21,789][20381] Updated weights for policy 1, policy_version 46991 (0.0008) [2023-12-27 05:59:21,817][20347] Updated weights for policy 0, policy_version 47013 (0.0011) [2023-12-27 05:59:21,852][20381] Updated weights for policy 1, policy_version 47001 (0.0006) [2023-12-27 05:59:21,879][20347] Updated weights for policy 0, policy_version 47023 (0.0011) [2023-12-27 05:59:21,911][20381] Updated weights for policy 1, policy_version 47011 (0.0008) [2023-12-27 05:59:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 24084480. Throughput: 0: 10359.7, 1: 10437.0. Samples: 24056100. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000047016_12042240.pth... [2023-12-27 05:59:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000047024_12042240.pth... [2023-12-27 05:59:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000045808_11730944.pth [2023-12-27 05:59:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000045800_11730944.pth [2023-12-27 05:59:22,537][20347] Updated weights for policy 0, policy_version 47033 (0.0006) [2023-12-27 05:59:22,549][20381] Updated weights for policy 1, policy_version 47021 (0.0008) [2023-12-27 05:59:22,596][20347] Updated weights for policy 0, policy_version 47043 (0.0009) [2023-12-27 05:59:22,607][20381] Updated weights for policy 1, policy_version 47031 (0.0008) [2023-12-27 05:59:22,658][20347] Updated weights for policy 0, policy_version 47053 (0.0010) [2023-12-27 05:59:22,667][20381] Updated weights for policy 1, policy_version 47041 (0.0006) [2023-12-27 05:59:23,300][20381] Updated weights for policy 1, policy_version 47051 (0.0005) [2023-12-27 05:59:23,302][20347] Updated weights for policy 0, policy_version 47063 (0.0010) [2023-12-27 05:59:23,350][20347] Updated weights for policy 0, policy_version 47073 (0.0010) [2023-12-27 05:59:23,359][20381] Updated weights for policy 1, policy_version 47061 (0.0005) [2023-12-27 05:59:23,398][20347] Updated weights for policy 0, policy_version 47083 (0.0010) [2023-12-27 05:59:23,408][20381] Updated weights for policy 1, policy_version 47071 (0.0005) [2023-12-27 05:59:24,091][20381] Updated weights for policy 1, policy_version 47081 (0.0007) [2023-12-27 05:59:24,100][20347] Updated weights for policy 0, policy_version 47093 (0.0010) [2023-12-27 05:59:24,143][20381] Updated weights for policy 1, policy_version 47091 (0.0006) [2023-12-27 05:59:24,155][20347] Updated weights for policy 0, policy_version 47103 (0.0010) [2023-12-27 05:59:24,203][20381] Updated weights for policy 1, policy_version 47101 (0.0008) [2023-12-27 05:59:24,241][20347] Updated weights for policy 0, policy_version 47113 (0.0010) [2023-12-27 05:59:24,261][20381] Updated weights for policy 1, policy_version 47111 (0.0007) [2023-12-27 05:59:24,875][20347] Updated weights for policy 0, policy_version 47123 (0.0009) [2023-12-27 05:59:24,920][20381] Updated weights for policy 1, policy_version 47121 (0.0006) [2023-12-27 05:59:24,926][20347] Updated weights for policy 0, policy_version 47133 (0.0005) [2023-12-27 05:59:24,969][20381] Updated weights for policy 1, policy_version 47131 (0.0005) [2023-12-27 05:59:24,970][20347] Updated weights for policy 0, policy_version 47143 (0.0005) [2023-12-27 05:59:25,024][20381] Updated weights for policy 1, policy_version 47141 (0.0006) [2023-12-27 05:59:25,559][20347] Updated weights for policy 0, policy_version 47153 (0.0008) [2023-12-27 05:59:25,615][20381] Updated weights for policy 1, policy_version 47151 (0.0010) [2023-12-27 05:59:25,618][20347] Updated weights for policy 0, policy_version 47163 (0.0005) [2023-12-27 05:59:25,670][20347] Updated weights for policy 0, policy_version 47173 (0.0005) [2023-12-27 05:59:25,674][20381] Updated weights for policy 1, policy_version 47161 (0.0011) [2023-12-27 05:59:25,722][20347] Updated weights for policy 0, policy_version 47183 (0.0009) [2023-12-27 05:59:25,732][20381] Updated weights for policy 1, policy_version 47171 (0.0010) [2023-12-27 05:59:26,350][20381] Updated weights for policy 1, policy_version 47181 (0.0009) [2023-12-27 05:59:26,378][20347] Updated weights for policy 0, policy_version 47193 (0.0009) [2023-12-27 05:59:26,415][20381] Updated weights for policy 1, policy_version 47191 (0.0010) [2023-12-27 05:59:26,436][20347] Updated weights for policy 0, policy_version 47203 (0.0010) [2023-12-27 05:59:26,470][20381] Updated weights for policy 1, policy_version 47201 (0.0007) [2023-12-27 05:59:26,491][20347] Updated weights for policy 0, policy_version 47213 (0.0010) [2023-12-27 05:59:27,112][20381] Updated weights for policy 1, policy_version 47211 (0.0006) [2023-12-27 05:59:27,155][20347] Updated weights for policy 0, policy_version 47223 (0.0010) [2023-12-27 05:59:27,170][20381] Updated weights for policy 1, policy_version 47221 (0.0005) [2023-12-27 05:59:27,217][20347] Updated weights for policy 0, policy_version 47233 (0.0008) [2023-12-27 05:59:27,229][20381] Updated weights for policy 1, policy_version 47231 (0.0005) [2023-12-27 05:59:27,272][20347] Updated weights for policy 0, policy_version 47243 (0.0007) [2023-12-27 05:59:27,363][19170] Fps is (10 sec: 21298.6, 60 sec: 21026.0, 300 sec: 20827.1). Total num frames: 24199168. Throughput: 0: 10382.0, 1: 10466.5. Samples: 24182328. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:27,816][20381] Updated weights for policy 1, policy_version 47241 (0.0005) [2023-12-27 05:59:27,864][20381] Updated weights for policy 1, policy_version 47251 (0.0005) [2023-12-27 05:59:27,918][20381] Updated weights for policy 1, policy_version 47261 (0.0005) [2023-12-27 05:59:27,951][20347] Updated weights for policy 0, policy_version 47253 (0.0008) [2023-12-27 05:59:27,978][20381] Updated weights for policy 1, policy_version 47271 (0.0006) [2023-12-27 05:59:28,009][20347] Updated weights for policy 0, policy_version 47263 (0.0007) [2023-12-27 05:59:28,065][20347] Updated weights for policy 0, policy_version 47273 (0.0007) [2023-12-27 05:59:28,592][20381] Updated weights for policy 1, policy_version 47281 (0.0008) [2023-12-27 05:59:28,641][20381] Updated weights for policy 1, policy_version 47291 (0.0007) [2023-12-27 05:59:28,685][20381] Updated weights for policy 1, policy_version 47301 (0.0007) [2023-12-27 05:59:28,750][20347] Updated weights for policy 0, policy_version 47283 (0.0007) [2023-12-27 05:59:28,802][20347] Updated weights for policy 0, policy_version 47293 (0.0007) [2023-12-27 05:59:28,857][20347] Updated weights for policy 0, policy_version 47303 (0.0007) [2023-12-27 05:59:29,371][20381] Updated weights for policy 1, policy_version 47311 (0.0008) [2023-12-27 05:59:29,426][20381] Updated weights for policy 1, policy_version 47321 (0.0008) [2023-12-27 05:59:29,483][20381] Updated weights for policy 1, policy_version 47331 (0.0007) [2023-12-27 05:59:29,561][20347] Updated weights for policy 0, policy_version 47313 (0.0007) [2023-12-27 05:59:29,616][20347] Updated weights for policy 0, policy_version 47323 (0.0010) [2023-12-27 05:59:29,664][20347] Updated weights for policy 0, policy_version 47333 (0.0010) [2023-12-27 05:59:29,719][20347] Updated weights for policy 0, policy_version 47343 (0.0010) [2023-12-27 05:59:30,162][20381] Updated weights for policy 1, policy_version 47341 (0.0007) [2023-12-27 05:59:30,224][20381] Updated weights for policy 1, policy_version 47351 (0.0008) [2023-12-27 05:59:30,280][20381] Updated weights for policy 1, policy_version 47361 (0.0007) [2023-12-27 05:59:30,391][20347] Updated weights for policy 0, policy_version 47353 (0.0010) [2023-12-27 05:59:30,443][20347] Updated weights for policy 0, policy_version 47363 (0.0010) [2023-12-27 05:59:30,493][20347] Updated weights for policy 0, policy_version 47373 (0.0010) [2023-12-27 05:59:31,004][20381] Updated weights for policy 1, policy_version 47371 (0.0007) [2023-12-27 05:59:31,063][20381] Updated weights for policy 1, policy_version 47381 (0.0007) [2023-12-27 05:59:31,078][20347] Updated weights for policy 0, policy_version 47383 (0.0009) [2023-12-27 05:59:31,121][20381] Updated weights for policy 1, policy_version 47391 (0.0007) [2023-12-27 05:59:31,138][20347] Updated weights for policy 0, policy_version 47393 (0.0006) [2023-12-27 05:59:31,200][20347] Updated weights for policy 0, policy_version 47403 (0.0008) [2023-12-27 05:59:31,843][20347] Updated weights for policy 0, policy_version 47413 (0.0006) [2023-12-27 05:59:31,845][20381] Updated weights for policy 1, policy_version 47401 (0.0006) [2023-12-27 05:59:31,901][20347] Updated weights for policy 0, policy_version 47423 (0.0006) [2023-12-27 05:59:31,902][20381] Updated weights for policy 1, policy_version 47411 (0.0007) [2023-12-27 05:59:31,955][20347] Updated weights for policy 0, policy_version 47433 (0.0005) [2023-12-27 05:59:31,958][20381] Updated weights for policy 1, policy_version 47421 (0.0008) [2023-12-27 05:59:32,015][20381] Updated weights for policy 1, policy_version 47431 (0.0008) [2023-12-27 05:59:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 24297472. Throughput: 0: 10439.0, 1: 10439.2. Samples: 24306772. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:32,542][20347] Updated weights for policy 0, policy_version 47443 (0.0006) [2023-12-27 05:59:32,602][20347] Updated weights for policy 0, policy_version 47453 (0.0007) [2023-12-27 05:59:32,657][20347] Updated weights for policy 0, policy_version 47463 (0.0007) [2023-12-27 05:59:32,715][20381] Updated weights for policy 1, policy_version 47441 (0.0009) [2023-12-27 05:59:32,765][20381] Updated weights for policy 1, policy_version 47451 (0.0008) [2023-12-27 05:59:32,814][20381] Updated weights for policy 1, policy_version 47461 (0.0007) [2023-12-27 05:59:33,207][20347] Updated weights for policy 0, policy_version 47473 (0.0006) [2023-12-27 05:59:33,266][20347] Updated weights for policy 0, policy_version 47483 (0.0007) [2023-12-27 05:59:33,326][20347] Updated weights for policy 0, policy_version 47493 (0.0008) [2023-12-27 05:59:33,373][20381] Updated weights for policy 1, policy_version 47471 (0.0005) [2023-12-27 05:59:33,382][20347] Updated weights for policy 0, policy_version 47503 (0.0007) [2023-12-27 05:59:33,420][20381] Updated weights for policy 1, policy_version 47481 (0.0007) [2023-12-27 05:59:33,471][20381] Updated weights for policy 1, policy_version 47491 (0.0007) [2023-12-27 05:59:34,056][20347] Updated weights for policy 0, policy_version 47513 (0.0007) [2023-12-27 05:59:34,101][20347] Updated weights for policy 0, policy_version 47523 (0.0007) [2023-12-27 05:59:34,152][20381] Updated weights for policy 1, policy_version 47501 (0.0008) [2023-12-27 05:59:34,159][20347] Updated weights for policy 0, policy_version 47533 (0.0008) [2023-12-27 05:59:34,213][20381] Updated weights for policy 1, policy_version 47511 (0.0009) [2023-12-27 05:59:34,277][20381] Updated weights for policy 1, policy_version 47521 (0.0007) [2023-12-27 05:59:34,885][20347] Updated weights for policy 0, policy_version 47543 (0.0006) [2023-12-27 05:59:34,938][20347] Updated weights for policy 0, policy_version 47553 (0.0006) [2023-12-27 05:59:34,963][20381] Updated weights for policy 1, policy_version 47531 (0.0007) [2023-12-27 05:59:34,995][20347] Updated weights for policy 0, policy_version 47563 (0.0006) [2023-12-27 05:59:35,019][20381] Updated weights for policy 1, policy_version 47541 (0.0006) [2023-12-27 05:59:35,079][20381] Updated weights for policy 1, policy_version 47551 (0.0007) [2023-12-27 05:59:35,690][20347] Updated weights for policy 0, policy_version 47573 (0.0006) [2023-12-27 05:59:35,698][20381] Updated weights for policy 1, policy_version 47561 (0.0008) [2023-12-27 05:59:35,741][20347] Updated weights for policy 0, policy_version 47583 (0.0006) [2023-12-27 05:59:35,749][20381] Updated weights for policy 1, policy_version 47571 (0.0006) [2023-12-27 05:59:35,800][20347] Updated weights for policy 0, policy_version 47593 (0.0007) [2023-12-27 05:59:35,803][20381] Updated weights for policy 1, policy_version 47581 (0.0007) [2023-12-27 05:59:35,855][20381] Updated weights for policy 1, policy_version 47591 (0.0006) [2023-12-27 05:59:36,494][20347] Updated weights for policy 0, policy_version 47603 (0.0006) [2023-12-27 05:59:36,552][20347] Updated weights for policy 0, policy_version 47613 (0.0008) [2023-12-27 05:59:36,560][20381] Updated weights for policy 1, policy_version 47601 (0.0006) [2023-12-27 05:59:36,612][20347] Updated weights for policy 0, policy_version 47623 (0.0008) [2023-12-27 05:59:36,623][20381] Updated weights for policy 1, policy_version 47611 (0.0009) [2023-12-27 05:59:36,685][20381] Updated weights for policy 1, policy_version 47621 (0.0011) [2023-12-27 05:59:37,220][20381] Updated weights for policy 1, policy_version 47631 (0.0007) [2023-12-27 05:59:37,269][20381] Updated weights for policy 1, policy_version 47641 (0.0009) [2023-12-27 05:59:37,330][20381] Updated weights for policy 1, policy_version 47651 (0.0011) [2023-12-27 05:59:37,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 24403968. Throughput: 0: 10473.8, 1: 10503.8. Samples: 24371324. Policy #0 lag: (min: 19.0, avg: 24.4, max: 51.0) [2023-12-27 05:59:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000047656_12206080.pth... [2023-12-27 05:59:37,371][20347] Updated weights for policy 0, policy_version 47633 (0.0007) [2023-12-27 05:59:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000046408_11886592.pth [2023-12-27 05:59:37,438][20347] Updated weights for policy 0, policy_version 47643 (0.0007) [2023-12-27 05:59:37,492][20347] Updated weights for policy 0, policy_version 47653 (0.0007) [2023-12-27 05:59:37,543][20347] Updated weights for policy 0, policy_version 47663 (0.0008) [2023-12-27 05:59:37,546][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000047664_12206080.pth... [2023-12-27 05:59:37,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000046448_11894784.pth [2023-12-27 05:59:37,983][20381] Updated weights for policy 1, policy_version 47661 (0.0011) [2023-12-27 05:59:38,038][20381] Updated weights for policy 1, policy_version 47671 (0.0006) [2023-12-27 05:59:38,097][20381] Updated weights for policy 1, policy_version 47681 (0.0009) [2023-12-27 05:59:38,209][20347] Updated weights for policy 0, policy_version 47673 (0.0005) [2023-12-27 05:59:38,267][20347] Updated weights for policy 0, policy_version 47683 (0.0006) [2023-12-27 05:59:38,323][20347] Updated weights for policy 0, policy_version 47693 (0.0008) [2023-12-27 05:59:38,756][20381] Updated weights for policy 1, policy_version 47691 (0.0011) [2023-12-27 05:59:38,813][20381] Updated weights for policy 1, policy_version 47701 (0.0011) [2023-12-27 05:59:38,872][20381] Updated weights for policy 1, policy_version 47711 (0.0011) [2023-12-27 05:59:38,986][20347] Updated weights for policy 0, policy_version 47703 (0.0009) [2023-12-27 05:59:39,038][20347] Updated weights for policy 0, policy_version 47713 (0.0010) [2023-12-27 05:59:39,096][20347] Updated weights for policy 0, policy_version 47723 (0.0010) [2023-12-27 05:59:39,508][20381] Updated weights for policy 1, policy_version 47721 (0.0010) [2023-12-27 05:59:39,569][20381] Updated weights for policy 1, policy_version 47731 (0.0011) [2023-12-27 05:59:39,625][20381] Updated weights for policy 1, policy_version 47741 (0.0011) [2023-12-27 05:59:39,693][20381] Updated weights for policy 1, policy_version 47751 (0.0011) [2023-12-27 05:59:39,782][20347] Updated weights for policy 0, policy_version 47733 (0.0008) [2023-12-27 05:59:39,849][20347] Updated weights for policy 0, policy_version 47743 (0.0008) [2023-12-27 05:59:39,914][20347] Updated weights for policy 0, policy_version 47753 (0.0009) [2023-12-27 05:59:40,415][20381] Updated weights for policy 1, policy_version 47761 (0.0011) [2023-12-27 05:59:40,471][20381] Updated weights for policy 1, policy_version 47771 (0.0011) [2023-12-27 05:59:40,523][20381] Updated weights for policy 1, policy_version 47781 (0.0011) [2023-12-27 05:59:40,587][20347] Updated weights for policy 0, policy_version 47763 (0.0010) [2023-12-27 05:59:40,654][20347] Updated weights for policy 0, policy_version 47773 (0.0008) [2023-12-27 05:59:40,704][20347] Updated weights for policy 0, policy_version 47783 (0.0010) [2023-12-27 05:59:41,143][20381] Updated weights for policy 1, policy_version 47791 (0.0009) [2023-12-27 05:59:41,199][20381] Updated weights for policy 1, policy_version 47801 (0.0008) [2023-12-27 05:59:41,258][20381] Updated weights for policy 1, policy_version 47811 (0.0007) [2023-12-27 05:59:41,416][20347] Updated weights for policy 0, policy_version 47793 (0.0010) [2023-12-27 05:59:41,476][20347] Updated weights for policy 0, policy_version 47803 (0.0006) [2023-12-27 05:59:41,543][20347] Updated weights for policy 0, policy_version 47813 (0.0008) [2023-12-27 05:59:41,604][20347] Updated weights for policy 0, policy_version 47823 (0.0007) [2023-12-27 05:59:42,040][20381] Updated weights for policy 1, policy_version 47821 (0.0009) [2023-12-27 05:59:42,099][20381] Updated weights for policy 1, policy_version 47831 (0.0009) [2023-12-27 05:59:42,159][20381] Updated weights for policy 1, policy_version 47841 (0.0010) [2023-12-27 05:59:42,199][20347] Updated weights for policy 0, policy_version 47833 (0.0007) [2023-12-27 05:59:42,260][20347] Updated weights for policy 0, policy_version 47843 (0.0006) [2023-12-27 05:59:42,324][20347] Updated weights for policy 0, policy_version 47853 (0.0008) [2023-12-27 05:59:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 24510464. Throughput: 0: 10339.5, 1: 10530.9. Samples: 24495036. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:59:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:42,831][20381] Updated weights for policy 1, policy_version 47851 (0.0007) [2023-12-27 05:59:42,884][20381] Updated weights for policy 1, policy_version 47861 (0.0008) [2023-12-27 05:59:42,935][20381] Updated weights for policy 1, policy_version 47871 (0.0005) [2023-12-27 05:59:43,036][20347] Updated weights for policy 0, policy_version 47863 (0.0009) [2023-12-27 05:59:43,100][20347] Updated weights for policy 0, policy_version 47873 (0.0008) [2023-12-27 05:59:43,164][20347] Updated weights for policy 0, policy_version 47883 (0.0008) [2023-12-27 05:59:43,511][20381] Updated weights for policy 1, policy_version 47881 (0.0007) [2023-12-27 05:59:43,574][20381] Updated weights for policy 1, policy_version 47891 (0.0007) [2023-12-27 05:59:43,632][20381] Updated weights for policy 1, policy_version 47901 (0.0007) [2023-12-27 05:59:43,686][20381] Updated weights for policy 1, policy_version 47911 (0.0006) [2023-12-27 05:59:43,853][20347] Updated weights for policy 0, policy_version 47893 (0.0008) [2023-12-27 05:59:43,905][20347] Updated weights for policy 0, policy_version 47903 (0.0008) [2023-12-27 05:59:43,969][20347] Updated weights for policy 0, policy_version 47913 (0.0007) [2023-12-27 05:59:44,351][20381] Updated weights for policy 1, policy_version 47921 (0.0009) [2023-12-27 05:59:44,402][20381] Updated weights for policy 1, policy_version 47931 (0.0008) [2023-12-27 05:59:44,452][20381] Updated weights for policy 1, policy_version 47941 (0.0005) [2023-12-27 05:59:44,599][20347] Updated weights for policy 0, policy_version 47923 (0.0006) [2023-12-27 05:59:44,653][20347] Updated weights for policy 0, policy_version 47933 (0.0007) [2023-12-27 05:59:44,712][20347] Updated weights for policy 0, policy_version 47943 (0.0008) [2023-12-27 05:59:45,094][20381] Updated weights for policy 1, policy_version 47951 (0.0007) [2023-12-27 05:59:45,151][20381] Updated weights for policy 1, policy_version 47961 (0.0008) [2023-12-27 05:59:45,212][20381] Updated weights for policy 1, policy_version 47971 (0.0008) [2023-12-27 05:59:45,446][20347] Updated weights for policy 0, policy_version 47953 (0.0007) [2023-12-27 05:59:45,507][20347] Updated weights for policy 0, policy_version 47963 (0.0008) [2023-12-27 05:59:45,569][20347] Updated weights for policy 0, policy_version 47973 (0.0009) [2023-12-27 05:59:45,623][20347] Updated weights for policy 0, policy_version 47983 (0.0008) [2023-12-27 05:59:45,879][20381] Updated weights for policy 1, policy_version 47981 (0.0007) [2023-12-27 05:59:45,924][20381] Updated weights for policy 1, policy_version 47991 (0.0008) [2023-12-27 05:59:45,970][20381] Updated weights for policy 1, policy_version 48001 (0.0007) [2023-12-27 05:59:46,306][20347] Updated weights for policy 0, policy_version 47993 (0.0009) [2023-12-27 05:59:46,366][20347] Updated weights for policy 0, policy_version 48003 (0.0010) [2023-12-27 05:59:46,414][20347] Updated weights for policy 0, policy_version 48013 (0.0010) [2023-12-27 05:59:46,666][20381] Updated weights for policy 1, policy_version 48011 (0.0007) [2023-12-27 05:59:46,721][20381] Updated weights for policy 1, policy_version 48021 (0.0010) [2023-12-27 05:59:46,776][20381] Updated weights for policy 1, policy_version 48031 (0.0008) [2023-12-27 05:59:47,053][20347] Updated weights for policy 0, policy_version 48023 (0.0009) [2023-12-27 05:59:47,112][20347] Updated weights for policy 0, policy_version 48033 (0.0008) [2023-12-27 05:59:47,174][20347] Updated weights for policy 0, policy_version 48043 (0.0010) [2023-12-27 05:59:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 24608768. Throughput: 0: 10333.6, 1: 10567.4. Samples: 24619368. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:59:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:47,389][20381] Updated weights for policy 1, policy_version 48041 (0.0009) [2023-12-27 05:59:47,441][20381] Updated weights for policy 1, policy_version 48051 (0.0010) [2023-12-27 05:59:47,489][20381] Updated weights for policy 1, policy_version 48061 (0.0010) [2023-12-27 05:59:47,534][20381] Updated weights for policy 1, policy_version 48071 (0.0010) [2023-12-27 05:59:47,741][20347] Updated weights for policy 0, policy_version 48053 (0.0009) [2023-12-27 05:59:47,792][20347] Updated weights for policy 0, policy_version 48063 (0.0010) [2023-12-27 05:59:47,850][20347] Updated weights for policy 0, policy_version 48073 (0.0008) [2023-12-27 05:59:48,202][20381] Updated weights for policy 1, policy_version 48081 (0.0010) [2023-12-27 05:59:48,249][20381] Updated weights for policy 1, policy_version 48091 (0.0010) [2023-12-27 05:59:48,300][20381] Updated weights for policy 1, policy_version 48101 (0.0005) [2023-12-27 05:59:48,430][20347] Updated weights for policy 0, policy_version 48083 (0.0006) [2023-12-27 05:59:48,490][20347] Updated weights for policy 0, policy_version 48093 (0.0008) [2023-12-27 05:59:48,550][20347] Updated weights for policy 0, policy_version 48103 (0.0010) [2023-12-27 05:59:48,974][20381] Updated weights for policy 1, policy_version 48111 (0.0009) [2023-12-27 05:59:49,028][20381] Updated weights for policy 1, policy_version 48121 (0.0010) [2023-12-27 05:59:49,084][20381] Updated weights for policy 1, policy_version 48131 (0.0011) [2023-12-27 05:59:49,249][20347] Updated weights for policy 0, policy_version 48113 (0.0010) [2023-12-27 05:59:49,311][20347] Updated weights for policy 0, policy_version 48123 (0.0006) [2023-12-27 05:59:49,379][20347] Updated weights for policy 0, policy_version 48133 (0.0009) [2023-12-27 05:59:49,439][20347] Updated weights for policy 0, policy_version 48143 (0.0010) [2023-12-27 05:59:49,792][20381] Updated weights for policy 1, policy_version 48141 (0.0011) [2023-12-27 05:59:49,856][20381] Updated weights for policy 1, policy_version 48151 (0.0010) [2023-12-27 05:59:49,920][20381] Updated weights for policy 1, policy_version 48161 (0.0008) [2023-12-27 05:59:50,173][20347] Updated weights for policy 0, policy_version 48153 (0.0008) [2023-12-27 05:59:50,233][20347] Updated weights for policy 0, policy_version 48163 (0.0008) [2023-12-27 05:59:50,293][20347] Updated weights for policy 0, policy_version 48173 (0.0007) [2023-12-27 05:59:50,595][20381] Updated weights for policy 1, policy_version 48171 (0.0009) [2023-12-27 05:59:50,652][20381] Updated weights for policy 1, policy_version 48181 (0.0011) [2023-12-27 05:59:50,714][20381] Updated weights for policy 1, policy_version 48191 (0.0006) [2023-12-27 05:59:51,024][20347] Updated weights for policy 0, policy_version 48183 (0.0008) [2023-12-27 05:59:51,088][20347] Updated weights for policy 0, policy_version 48193 (0.0007) [2023-12-27 05:59:51,151][20347] Updated weights for policy 0, policy_version 48203 (0.0007) [2023-12-27 05:59:51,355][20381] Updated weights for policy 1, policy_version 48201 (0.0009) [2023-12-27 05:59:51,412][20381] Updated weights for policy 1, policy_version 48211 (0.0009) [2023-12-27 05:59:51,468][20381] Updated weights for policy 1, policy_version 48221 (0.0011) [2023-12-27 05:59:51,526][20381] Updated weights for policy 1, policy_version 48231 (0.0010) [2023-12-27 05:59:51,874][20347] Updated weights for policy 0, policy_version 48213 (0.0008) [2023-12-27 05:59:51,935][20347] Updated weights for policy 0, policy_version 48223 (0.0008) [2023-12-27 05:59:51,995][20347] Updated weights for policy 0, policy_version 48233 (0.0007) [2023-12-27 05:59:52,152][20381] Updated weights for policy 1, policy_version 48241 (0.0006) [2023-12-27 05:59:52,199][20381] Updated weights for policy 1, policy_version 48251 (0.0005) [2023-12-27 05:59:52,252][20381] Updated weights for policy 1, policy_version 48261 (0.0005) [2023-12-27 05:59:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 24715264. Throughput: 0: 10327.1, 1: 10534.0. Samples: 24681296. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:59:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000048264_12361728.pth... [2023-12-27 05:59:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000048240_12353536.pth... [2023-12-27 05:59:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000047024_12042240.pth [2023-12-27 05:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000047016_12042240.pth [2023-12-27 05:59:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000048240_12353536.pth [2023-12-27 05:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000048264_12361728.pth [2023-12-27 05:59:52,704][20347] Updated weights for policy 0, policy_version 48243 (0.0007) [2023-12-27 05:59:52,760][20347] Updated weights for policy 0, policy_version 48253 (0.0008) [2023-12-27 05:59:52,824][20347] Updated weights for policy 0, policy_version 48263 (0.0008) [2023-12-27 05:59:52,932][20381] Updated weights for policy 1, policy_version 48271 (0.0008) [2023-12-27 05:59:52,979][20381] Updated weights for policy 1, policy_version 48281 (0.0010) [2023-12-27 05:59:53,025][20381] Updated weights for policy 1, policy_version 48291 (0.0008) [2023-12-27 05:59:53,426][20347] Updated weights for policy 0, policy_version 48273 (0.0008) [2023-12-27 05:59:53,480][20347] Updated weights for policy 0, policy_version 48283 (0.0007) [2023-12-27 05:59:53,534][20347] Updated weights for policy 0, policy_version 48293 (0.0007) [2023-12-27 05:59:53,586][20347] Updated weights for policy 0, policy_version 48303 (0.0005) [2023-12-27 05:59:53,763][20381] Updated weights for policy 1, policy_version 48301 (0.0007) [2023-12-27 05:59:53,808][20381] Updated weights for policy 1, policy_version 48311 (0.0007) [2023-12-27 05:59:53,856][20381] Updated weights for policy 1, policy_version 48321 (0.0007) [2023-12-27 05:59:54,223][20347] Updated weights for policy 0, policy_version 48313 (0.0005) [2023-12-27 05:59:54,279][20347] Updated weights for policy 0, policy_version 48323 (0.0005) [2023-12-27 05:59:54,337][20347] Updated weights for policy 0, policy_version 48333 (0.0006) [2023-12-27 05:59:54,560][20381] Updated weights for policy 1, policy_version 48331 (0.0007) [2023-12-27 05:59:54,614][20381] Updated weights for policy 1, policy_version 48341 (0.0008) [2023-12-27 05:59:54,664][20381] Updated weights for policy 1, policy_version 48351 (0.0009) [2023-12-27 05:59:54,843][20347] Updated weights for policy 0, policy_version 48343 (0.0005) [2023-12-27 05:59:54,897][20347] Updated weights for policy 0, policy_version 48353 (0.0005) [2023-12-27 05:59:54,949][20347] Updated weights for policy 0, policy_version 48363 (0.0007) [2023-12-27 05:59:55,411][20381] Updated weights for policy 1, policy_version 48361 (0.0009) [2023-12-27 05:59:55,468][20381] Updated weights for policy 1, policy_version 48371 (0.0008) [2023-12-27 05:59:55,522][20381] Updated weights for policy 1, policy_version 48381 (0.0008) [2023-12-27 05:59:55,565][20347] Updated weights for policy 0, policy_version 48373 (0.0007) [2023-12-27 05:59:55,578][20381] Updated weights for policy 1, policy_version 48391 (0.0006) [2023-12-27 05:59:55,618][20347] Updated weights for policy 0, policy_version 48383 (0.0007) [2023-12-27 05:59:55,678][20347] Updated weights for policy 0, policy_version 48393 (0.0007) [2023-12-27 05:59:56,273][20347] Updated weights for policy 0, policy_version 48403 (0.0007) [2023-12-27 05:59:56,306][20381] Updated weights for policy 1, policy_version 48401 (0.0007) [2023-12-27 05:59:56,323][20347] Updated weights for policy 0, policy_version 48413 (0.0006) [2023-12-27 05:59:56,355][20381] Updated weights for policy 1, policy_version 48411 (0.0006) [2023-12-27 05:59:56,380][20347] Updated weights for policy 0, policy_version 48423 (0.0006) [2023-12-27 05:59:56,414][20381] Updated weights for policy 1, policy_version 48421 (0.0005) [2023-12-27 05:59:57,029][20347] Updated weights for policy 0, policy_version 48433 (0.0006) [2023-12-27 05:59:57,084][20347] Updated weights for policy 0, policy_version 48443 (0.0007) [2023-12-27 05:59:57,109][20381] Updated weights for policy 1, policy_version 48431 (0.0006) [2023-12-27 05:59:57,134][20347] Updated weights for policy 0, policy_version 48453 (0.0006) [2023-12-27 05:59:57,154][20381] Updated weights for policy 1, policy_version 48441 (0.0006) [2023-12-27 05:59:57,182][20347] Updated weights for policy 0, policy_version 48463 (0.0006) [2023-12-27 05:59:57,200][20381] Updated weights for policy 1, policy_version 48451 (0.0006) [2023-12-27 05:59:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 24821760. Throughput: 0: 10393.1, 1: 10444.3. Samples: 24805800. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 05:59:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 05:59:57,839][20347] Updated weights for policy 0, policy_version 48473 (0.0006) [2023-12-27 05:59:57,871][20381] Updated weights for policy 1, policy_version 48461 (0.0009) [2023-12-27 05:59:57,882][20347] Updated weights for policy 0, policy_version 48483 (0.0009) [2023-12-27 05:59:57,929][20381] Updated weights for policy 1, policy_version 48471 (0.0010) [2023-12-27 05:59:57,930][20347] Updated weights for policy 0, policy_version 48493 (0.0005) [2023-12-27 05:59:57,979][20381] Updated weights for policy 1, policy_version 48481 (0.0010) [2023-12-27 05:59:58,599][20347] Updated weights for policy 0, policy_version 48503 (0.0007) [2023-12-27 05:59:58,658][20347] Updated weights for policy 0, policy_version 48513 (0.0008) [2023-12-27 05:59:58,693][20381] Updated weights for policy 1, policy_version 48491 (0.0009) [2023-12-27 05:59:58,720][20347] Updated weights for policy 0, policy_version 48523 (0.0008) [2023-12-27 05:59:58,758][20381] Updated weights for policy 1, policy_version 48501 (0.0008) [2023-12-27 05:59:58,821][20381] Updated weights for policy 1, policy_version 48511 (0.0008) [2023-12-27 05:59:59,507][20347] Updated weights for policy 0, policy_version 48533 (0.0009) [2023-12-27 05:59:59,527][20381] Updated weights for policy 1, policy_version 48521 (0.0007) [2023-12-27 05:59:59,556][20347] Updated weights for policy 0, policy_version 48543 (0.0010) [2023-12-27 05:59:59,578][20381] Updated weights for policy 1, policy_version 48531 (0.0011) [2023-12-27 05:59:59,608][20347] Updated weights for policy 0, policy_version 48553 (0.0010) [2023-12-27 05:59:59,635][20381] Updated weights for policy 1, policy_version 48541 (0.0011) [2023-12-27 05:59:59,684][20381] Updated weights for policy 1, policy_version 48551 (0.0011) [2023-12-27 06:00:00,200][20347] Updated weights for policy 0, policy_version 48563 (0.0009) [2023-12-27 06:00:00,251][20347] Updated weights for policy 0, policy_version 48573 (0.0005) [2023-12-27 06:00:00,306][20347] Updated weights for policy 0, policy_version 48583 (0.0005) [2023-12-27 06:00:00,360][20381] Updated weights for policy 1, policy_version 48561 (0.0011) [2023-12-27 06:00:00,411][20381] Updated weights for policy 1, policy_version 48571 (0.0010) [2023-12-27 06:00:00,463][20381] Updated weights for policy 1, policy_version 48581 (0.0010) [2023-12-27 06:00:00,916][20347] Updated weights for policy 0, policy_version 48593 (0.0007) [2023-12-27 06:00:00,967][20347] Updated weights for policy 0, policy_version 48603 (0.0005) [2023-12-27 06:00:01,018][20347] Updated weights for policy 0, policy_version 48613 (0.0006) [2023-12-27 06:00:01,073][20347] Updated weights for policy 0, policy_version 48623 (0.0008) [2023-12-27 06:00:01,190][20381] Updated weights for policy 1, policy_version 48591 (0.0011) [2023-12-27 06:00:01,256][20381] Updated weights for policy 1, policy_version 48601 (0.0011) [2023-12-27 06:00:01,318][20381] Updated weights for policy 1, policy_version 48611 (0.0010) [2023-12-27 06:00:01,768][20347] Updated weights for policy 0, policy_version 48633 (0.0006) [2023-12-27 06:00:01,824][20347] Updated weights for policy 0, policy_version 48643 (0.0006) [2023-12-27 06:00:01,868][20347] Updated weights for policy 0, policy_version 48653 (0.0006) [2023-12-27 06:00:02,001][20381] Updated weights for policy 1, policy_version 48621 (0.0010) [2023-12-27 06:00:02,068][20381] Updated weights for policy 1, policy_version 48631 (0.0009) [2023-12-27 06:00:02,130][20381] Updated weights for policy 1, policy_version 48641 (0.0007) [2023-12-27 06:00:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 24920064. Throughput: 0: 10415.6, 1: 10366.2. Samples: 24929176. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 06:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:02,490][20347] Updated weights for policy 0, policy_version 48663 (0.0006) [2023-12-27 06:00:02,553][20347] Updated weights for policy 0, policy_version 48673 (0.0005) [2023-12-27 06:00:02,615][20347] Updated weights for policy 0, policy_version 48683 (0.0005) [2023-12-27 06:00:02,785][20381] Updated weights for policy 1, policy_version 48651 (0.0008) [2023-12-27 06:00:02,839][20381] Updated weights for policy 1, policy_version 48661 (0.0005) [2023-12-27 06:00:02,905][20381] Updated weights for policy 1, policy_version 48671 (0.0005) [2023-12-27 06:00:03,262][20347] Updated weights for policy 0, policy_version 48693 (0.0006) [2023-12-27 06:00:03,321][20347] Updated weights for policy 0, policy_version 48703 (0.0008) [2023-12-27 06:00:03,375][20347] Updated weights for policy 0, policy_version 48713 (0.0007) [2023-12-27 06:00:03,547][20381] Updated weights for policy 1, policy_version 48681 (0.0007) [2023-12-27 06:00:03,592][20381] Updated weights for policy 1, policy_version 48691 (0.0008) [2023-12-27 06:00:03,636][20381] Updated weights for policy 1, policy_version 48701 (0.0007) [2023-12-27 06:00:03,682][20381] Updated weights for policy 1, policy_version 48711 (0.0008) [2023-12-27 06:00:04,037][20347] Updated weights for policy 0, policy_version 48723 (0.0007) [2023-12-27 06:00:04,109][20347] Updated weights for policy 0, policy_version 48733 (0.0007) [2023-12-27 06:00:04,171][20347] Updated weights for policy 0, policy_version 48743 (0.0008) [2023-12-27 06:00:04,419][20381] Updated weights for policy 1, policy_version 48721 (0.0007) [2023-12-27 06:00:04,483][20381] Updated weights for policy 1, policy_version 48731 (0.0007) [2023-12-27 06:00:04,552][20381] Updated weights for policy 1, policy_version 48741 (0.0007) [2023-12-27 06:00:04,786][20347] Updated weights for policy 0, policy_version 48753 (0.0007) [2023-12-27 06:00:04,831][20347] Updated weights for policy 0, policy_version 48763 (0.0005) [2023-12-27 06:00:04,878][20347] Updated weights for policy 0, policy_version 48773 (0.0005) [2023-12-27 06:00:04,922][20347] Updated weights for policy 0, policy_version 48783 (0.0005) [2023-12-27 06:00:05,203][20381] Updated weights for policy 1, policy_version 48751 (0.0008) [2023-12-27 06:00:05,262][20381] Updated weights for policy 1, policy_version 48761 (0.0010) [2023-12-27 06:00:05,323][20381] Updated weights for policy 1, policy_version 48771 (0.0010) [2023-12-27 06:00:05,565][20347] Updated weights for policy 0, policy_version 48793 (0.0005) [2023-12-27 06:00:05,608][20347] Updated weights for policy 0, policy_version 48803 (0.0005) [2023-12-27 06:00:05,664][20347] Updated weights for policy 0, policy_version 48813 (0.0006) [2023-12-27 06:00:05,922][20381] Updated weights for policy 1, policy_version 48781 (0.0010) [2023-12-27 06:00:05,976][20381] Updated weights for policy 1, policy_version 48791 (0.0010) [2023-12-27 06:00:06,024][20381] Updated weights for policy 1, policy_version 48801 (0.0010) [2023-12-27 06:00:06,320][20347] Updated weights for policy 0, policy_version 48823 (0.0005) [2023-12-27 06:00:06,382][20347] Updated weights for policy 0, policy_version 48833 (0.0005) [2023-12-27 06:00:06,439][20347] Updated weights for policy 0, policy_version 48843 (0.0007) [2023-12-27 06:00:06,711][20381] Updated weights for policy 1, policy_version 48811 (0.0007) [2023-12-27 06:00:06,777][20381] Updated weights for policy 1, policy_version 48821 (0.0011) [2023-12-27 06:00:06,828][20381] Updated weights for policy 1, policy_version 48831 (0.0011) [2023-12-27 06:00:07,101][20347] Updated weights for policy 0, policy_version 48853 (0.0008) [2023-12-27 06:00:07,155][20347] Updated weights for policy 0, policy_version 48863 (0.0005) [2023-12-27 06:00:07,207][20347] Updated weights for policy 0, policy_version 48873 (0.0006) [2023-12-27 06:00:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 25026560. Throughput: 0: 10441.3, 1: 10369.3. Samples: 24992572. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 06:00:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000048880_12517376.pth... [2023-12-27 06:00:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000048840_12509184.pth... [2023-12-27 06:00:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000047664_12206080.pth [2023-12-27 06:00:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000047656_12206080.pth [2023-12-27 06:00:07,492][20381] Updated weights for policy 1, policy_version 48841 (0.0011) [2023-12-27 06:00:07,540][20381] Updated weights for policy 1, policy_version 48851 (0.0010) [2023-12-27 06:00:07,588][20381] Updated weights for policy 1, policy_version 48861 (0.0010) [2023-12-27 06:00:07,636][20381] Updated weights for policy 1, policy_version 48871 (0.0010) [2023-12-27 06:00:07,878][20347] Updated weights for policy 0, policy_version 48883 (0.0006) [2023-12-27 06:00:07,935][20347] Updated weights for policy 0, policy_version 48893 (0.0007) [2023-12-27 06:00:07,997][20347] Updated weights for policy 0, policy_version 48903 (0.0007) [2023-12-27 06:00:08,373][20381] Updated weights for policy 1, policy_version 48881 (0.0008) [2023-12-27 06:00:08,431][20381] Updated weights for policy 1, policy_version 48891 (0.0008) [2023-12-27 06:00:08,490][20381] Updated weights for policy 1, policy_version 48901 (0.0008) [2023-12-27 06:00:08,647][20347] Updated weights for policy 0, policy_version 48913 (0.0007) [2023-12-27 06:00:08,710][20347] Updated weights for policy 0, policy_version 48923 (0.0008) [2023-12-27 06:00:08,764][20347] Updated weights for policy 0, policy_version 48933 (0.0009) [2023-12-27 06:00:08,822][20347] Updated weights for policy 0, policy_version 48943 (0.0007) [2023-12-27 06:00:09,230][20381] Updated weights for policy 1, policy_version 48911 (0.0007) [2023-12-27 06:00:09,298][20381] Updated weights for policy 1, policy_version 48921 (0.0008) [2023-12-27 06:00:09,367][20381] Updated weights for policy 1, policy_version 48931 (0.0007) [2023-12-27 06:00:09,530][20347] Updated weights for policy 0, policy_version 48953 (0.0007) [2023-12-27 06:00:09,590][20347] Updated weights for policy 0, policy_version 48963 (0.0008) [2023-12-27 06:00:09,649][20347] Updated weights for policy 0, policy_version 48973 (0.0008) [2023-12-27 06:00:10,148][20381] Updated weights for policy 1, policy_version 48941 (0.0008) [2023-12-27 06:00:10,203][20381] Updated weights for policy 1, policy_version 48952 (0.0009) [2023-12-27 06:00:10,261][20381] Updated weights for policy 1, policy_version 48962 (0.0009) [2023-12-27 06:00:10,343][20347] Updated weights for policy 0, policy_version 48983 (0.0007) [2023-12-27 06:00:10,405][20347] Updated weights for policy 0, policy_version 48993 (0.0006) [2023-12-27 06:00:10,468][20347] Updated weights for policy 0, policy_version 49003 (0.0005) [2023-12-27 06:00:10,958][20381] Updated weights for policy 1, policy_version 48972 (0.0008) [2023-12-27 06:00:11,022][20381] Updated weights for policy 1, policy_version 48982 (0.0008) [2023-12-27 06:00:11,086][20347] Updated weights for policy 0, policy_version 49013 (0.0006) [2023-12-27 06:00:11,088][20381] Updated weights for policy 1, policy_version 48992 (0.0007) [2023-12-27 06:00:11,148][20347] Updated weights for policy 0, policy_version 49023 (0.0008) [2023-12-27 06:00:11,212][20347] Updated weights for policy 0, policy_version 49033 (0.0008) [2023-12-27 06:00:11,824][20381] Updated weights for policy 1, policy_version 49002 (0.0007) [2023-12-27 06:00:11,885][20381] Updated weights for policy 1, policy_version 49012 (0.0006) [2023-12-27 06:00:11,901][20347] Updated weights for policy 0, policy_version 49043 (0.0008) [2023-12-27 06:00:11,942][20381] Updated weights for policy 1, policy_version 49022 (0.0007) [2023-12-27 06:00:11,961][20347] Updated weights for policy 0, policy_version 49053 (0.0009) [2023-12-27 06:00:12,005][20381] Updated weights for policy 1, policy_version 49032 (0.0006) [2023-12-27 06:00:12,020][20347] Updated weights for policy 0, policy_version 49063 (0.0009) [2023-12-27 06:00:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 25124864. Throughput: 0: 10448.6, 1: 10270.2. Samples: 25114668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:12,683][20381] Updated weights for policy 1, policy_version 49042 (0.0008) [2023-12-27 06:00:12,736][20347] Updated weights for policy 0, policy_version 49073 (0.0011) [2023-12-27 06:00:12,745][20381] Updated weights for policy 1, policy_version 49052 (0.0007) [2023-12-27 06:00:12,791][20347] Updated weights for policy 0, policy_version 49083 (0.0010) [2023-12-27 06:00:12,804][20381] Updated weights for policy 1, policy_version 49062 (0.0006) [2023-12-27 06:00:12,854][20347] Updated weights for policy 0, policy_version 49093 (0.0010) [2023-12-27 06:00:12,913][20347] Updated weights for policy 0, policy_version 49103 (0.0009) [2023-12-27 06:00:13,491][20347] Updated weights for policy 0, policy_version 49113 (0.0006) [2023-12-27 06:00:13,545][20381] Updated weights for policy 1, policy_version 49072 (0.0006) [2023-12-27 06:00:13,553][20347] Updated weights for policy 0, policy_version 49123 (0.0007) [2023-12-27 06:00:13,605][20381] Updated weights for policy 1, policy_version 49082 (0.0006) [2023-12-27 06:00:13,612][20347] Updated weights for policy 0, policy_version 49133 (0.0007) [2023-12-27 06:00:13,658][20381] Updated weights for policy 1, policy_version 49092 (0.0006) [2023-12-27 06:00:14,250][20347] Updated weights for policy 0, policy_version 49143 (0.0010) [2023-12-27 06:00:14,315][20347] Updated weights for policy 0, policy_version 49153 (0.0010) [2023-12-27 06:00:14,371][20347] Updated weights for policy 0, policy_version 49163 (0.0010) [2023-12-27 06:00:14,378][20381] Updated weights for policy 1, policy_version 49102 (0.0006) [2023-12-27 06:00:14,433][20381] Updated weights for policy 1, policy_version 49112 (0.0008) [2023-12-27 06:00:14,491][20381] Updated weights for policy 1, policy_version 49122 (0.0008) [2023-12-27 06:00:15,020][20347] Updated weights for policy 0, policy_version 49173 (0.0008) [2023-12-27 06:00:15,084][20347] Updated weights for policy 0, policy_version 49183 (0.0006) [2023-12-27 06:00:15,148][20347] Updated weights for policy 0, policy_version 49193 (0.0006) [2023-12-27 06:00:15,240][20381] Updated weights for policy 1, policy_version 49132 (0.0006) [2023-12-27 06:00:15,303][20381] Updated weights for policy 1, policy_version 49142 (0.0007) [2023-12-27 06:00:15,363][20381] Updated weights for policy 1, policy_version 49152 (0.0007) [2023-12-27 06:00:15,760][20347] Updated weights for policy 0, policy_version 49203 (0.0007) [2023-12-27 06:00:15,807][20347] Updated weights for policy 0, policy_version 49213 (0.0009) [2023-12-27 06:00:15,864][20347] Updated weights for policy 0, policy_version 49223 (0.0009) [2023-12-27 06:00:16,014][20381] Updated weights for policy 1, policy_version 49162 (0.0006) [2023-12-27 06:00:16,068][20381] Updated weights for policy 1, policy_version 49172 (0.0007) [2023-12-27 06:00:16,124][20381] Updated weights for policy 1, policy_version 49182 (0.0007) [2023-12-27 06:00:16,191][20381] Updated weights for policy 1, policy_version 49192 (0.0007) [2023-12-27 06:00:16,606][20347] Updated weights for policy 0, policy_version 49233 (0.0008) [2023-12-27 06:00:16,674][20347] Updated weights for policy 0, policy_version 49243 (0.0006) [2023-12-27 06:00:16,740][20347] Updated weights for policy 0, policy_version 49253 (0.0008) [2023-12-27 06:00:16,804][20347] Updated weights for policy 0, policy_version 49263 (0.0008) [2023-12-27 06:00:16,893][20381] Updated weights for policy 1, policy_version 49202 (0.0008) [2023-12-27 06:00:16,951][20381] Updated weights for policy 1, policy_version 49212 (0.0008) [2023-12-27 06:00:17,000][20381] Updated weights for policy 1, policy_version 49222 (0.0005) [2023-12-27 06:00:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 25223168. Throughput: 0: 10424.3, 1: 10191.0. Samples: 25234460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:17,446][20347] Updated weights for policy 0, policy_version 49273 (0.0008) [2023-12-27 06:00:17,495][20347] Updated weights for policy 0, policy_version 49283 (0.0007) [2023-12-27 06:00:17,546][20347] Updated weights for policy 0, policy_version 49293 (0.0007) [2023-12-27 06:00:17,697][20381] Updated weights for policy 1, policy_version 49232 (0.0009) [2023-12-27 06:00:17,753][20381] Updated weights for policy 1, policy_version 49242 (0.0010) [2023-12-27 06:00:17,815][20381] Updated weights for policy 1, policy_version 49252 (0.0006) [2023-12-27 06:00:18,203][20347] Updated weights for policy 0, policy_version 49303 (0.0006) [2023-12-27 06:00:18,264][20347] Updated weights for policy 0, policy_version 49313 (0.0005) [2023-12-27 06:00:18,327][20347] Updated weights for policy 0, policy_version 49323 (0.0011) [2023-12-27 06:00:18,462][20381] Updated weights for policy 1, policy_version 49262 (0.0007) [2023-12-27 06:00:18,525][20381] Updated weights for policy 1, policy_version 49272 (0.0008) [2023-12-27 06:00:18,588][20381] Updated weights for policy 1, policy_version 49282 (0.0008) [2023-12-27 06:00:18,981][20347] Updated weights for policy 0, policy_version 49333 (0.0009) [2023-12-27 06:00:19,043][20347] Updated weights for policy 0, policy_version 49343 (0.0007) [2023-12-27 06:00:19,101][20347] Updated weights for policy 0, policy_version 49353 (0.0007) [2023-12-27 06:00:19,294][20381] Updated weights for policy 1, policy_version 49292 (0.0008) [2023-12-27 06:00:19,348][20381] Updated weights for policy 1, policy_version 49302 (0.0008) [2023-12-27 06:00:19,418][20381] Updated weights for policy 1, policy_version 49312 (0.0009) [2023-12-27 06:00:19,825][20347] Updated weights for policy 0, policy_version 49363 (0.0008) [2023-12-27 06:00:19,892][20347] Updated weights for policy 0, policy_version 49373 (0.0008) [2023-12-27 06:00:19,958][20347] Updated weights for policy 0, policy_version 49383 (0.0010) [2023-12-27 06:00:20,225][20381] Updated weights for policy 1, policy_version 49322 (0.0008) [2023-12-27 06:00:20,289][20381] Updated weights for policy 1, policy_version 49332 (0.0008) [2023-12-27 06:00:20,358][20381] Updated weights for policy 1, policy_version 49342 (0.0008) [2023-12-27 06:00:20,424][20381] Updated weights for policy 1, policy_version 49352 (0.0009) [2023-12-27 06:00:20,582][20347] Updated weights for policy 0, policy_version 49393 (0.0006) [2023-12-27 06:00:20,659][20347] Updated weights for policy 0, policy_version 49403 (0.0009) [2023-12-27 06:00:20,724][20347] Updated weights for policy 0, policy_version 49413 (0.0008) [2023-12-27 06:00:20,785][20347] Updated weights for policy 0, policy_version 49423 (0.0008) [2023-12-27 06:00:21,189][20381] Updated weights for policy 1, policy_version 49362 (0.0008) [2023-12-27 06:00:21,251][20381] Updated weights for policy 1, policy_version 49372 (0.0008) [2023-12-27 06:00:21,312][20381] Updated weights for policy 1, policy_version 49382 (0.0009) [2023-12-27 06:00:21,563][20347] Updated weights for policy 0, policy_version 49433 (0.0008) [2023-12-27 06:00:21,623][20347] Updated weights for policy 0, policy_version 49443 (0.0008) [2023-12-27 06:00:21,690][20347] Updated weights for policy 0, policy_version 49453 (0.0007) [2023-12-27 06:00:22,070][20381] Updated weights for policy 1, policy_version 49392 (0.0008) [2023-12-27 06:00:22,133][20381] Updated weights for policy 1, policy_version 49402 (0.0009) [2023-12-27 06:00:22,195][20381] Updated weights for policy 1, policy_version 49412 (0.0008) [2023-12-27 06:00:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 25321472. Throughput: 0: 10386.1, 1: 10138.9. Samples: 25294944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000049416_12656640.pth... [2023-12-27 06:00:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000048264_12361728.pth [2023-12-27 06:00:22,405][20347] Updated weights for policy 0, policy_version 49463 (0.0007) [2023-12-27 06:00:22,466][20347] Updated weights for policy 0, policy_version 49473 (0.0007) [2023-12-27 06:00:22,518][20347] Updated weights for policy 0, policy_version 49483 (0.0005) [2023-12-27 06:00:22,547][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000049488_12673024.pth... [2023-12-27 06:00:22,552][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000048240_12353536.pth [2023-12-27 06:00:22,901][20381] Updated weights for policy 1, policy_version 49422 (0.0008) [2023-12-27 06:00:22,958][20381] Updated weights for policy 1, policy_version 49432 (0.0008) [2023-12-27 06:00:23,012][20381] Updated weights for policy 1, policy_version 49442 (0.0008) [2023-12-27 06:00:23,155][20347] Updated weights for policy 0, policy_version 49493 (0.0005) [2023-12-27 06:00:23,203][20347] Updated weights for policy 0, policy_version 49503 (0.0005) [2023-12-27 06:00:23,258][20347] Updated weights for policy 0, policy_version 49513 (0.0005) [2023-12-27 06:00:23,687][20381] Updated weights for policy 1, policy_version 49452 (0.0008) [2023-12-27 06:00:23,743][20381] Updated weights for policy 1, policy_version 49462 (0.0010) [2023-12-27 06:00:23,786][20347] Updated weights for policy 0, policy_version 49523 (0.0005) [2023-12-27 06:00:23,798][20381] Updated weights for policy 1, policy_version 49472 (0.0008) [2023-12-27 06:00:23,841][20347] Updated weights for policy 0, policy_version 49533 (0.0005) [2023-12-27 06:00:23,896][20347] Updated weights for policy 0, policy_version 49543 (0.0005) [2023-12-27 06:00:24,454][20381] Updated weights for policy 1, policy_version 49482 (0.0006) [2023-12-27 06:00:24,505][20381] Updated weights for policy 1, policy_version 49492 (0.0010) [2023-12-27 06:00:24,534][20347] Updated weights for policy 0, policy_version 49553 (0.0005) [2023-12-27 06:00:24,563][20381] Updated weights for policy 1, policy_version 49502 (0.0009) [2023-12-27 06:00:24,593][20347] Updated weights for policy 0, policy_version 49563 (0.0007) [2023-12-27 06:00:24,617][20381] Updated weights for policy 1, policy_version 49512 (0.0006) [2023-12-27 06:00:24,654][20347] Updated weights for policy 0, policy_version 49573 (0.0008) [2023-12-27 06:00:24,713][20347] Updated weights for policy 0, policy_version 49583 (0.0008) [2023-12-27 06:00:25,142][20381] Updated weights for policy 1, policy_version 49522 (0.0005) [2023-12-27 06:00:25,197][20381] Updated weights for policy 1, policy_version 49532 (0.0005) [2023-12-27 06:00:25,253][20381] Updated weights for policy 1, policy_version 49542 (0.0005) [2023-12-27 06:00:25,413][20347] Updated weights for policy 0, policy_version 49593 (0.0008) [2023-12-27 06:00:25,476][20347] Updated weights for policy 0, policy_version 49603 (0.0008) [2023-12-27 06:00:25,535][20347] Updated weights for policy 0, policy_version 49613 (0.0008) [2023-12-27 06:00:25,807][20381] Updated weights for policy 1, policy_version 49552 (0.0007) [2023-12-27 06:00:25,856][20381] Updated weights for policy 1, policy_version 49562 (0.0008) [2023-12-27 06:00:25,915][20381] Updated weights for policy 1, policy_version 49572 (0.0008) [2023-12-27 06:00:26,157][20347] Updated weights for policy 0, policy_version 49623 (0.0006) [2023-12-27 06:00:26,209][20347] Updated weights for policy 0, policy_version 49633 (0.0005) [2023-12-27 06:00:26,271][20347] Updated weights for policy 0, policy_version 49643 (0.0007) [2023-12-27 06:00:26,649][20381] Updated weights for policy 1, policy_version 49582 (0.0006) [2023-12-27 06:00:26,697][20381] Updated weights for policy 1, policy_version 49592 (0.0005) [2023-12-27 06:00:26,763][20381] Updated weights for policy 1, policy_version 49602 (0.0005) [2023-12-27 06:00:26,932][20347] Updated weights for policy 0, policy_version 49653 (0.0007) [2023-12-27 06:00:26,986][20347] Updated weights for policy 0, policy_version 49663 (0.0008) [2023-12-27 06:00:27,043][20347] Updated weights for policy 0, policy_version 49673 (0.0007) [2023-12-27 06:00:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20744.2). Total num frames: 25427968. Throughput: 0: 10436.1, 1: 10086.8. Samples: 25418568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:27,403][20381] Updated weights for policy 1, policy_version 49612 (0.0007) [2023-12-27 06:00:27,458][20381] Updated weights for policy 1, policy_version 49622 (0.0007) [2023-12-27 06:00:27,508][20381] Updated weights for policy 1, policy_version 49632 (0.0008) [2023-12-27 06:00:27,736][20347] Updated weights for policy 0, policy_version 49683 (0.0008) [2023-12-27 06:00:27,793][20347] Updated weights for policy 0, policy_version 49693 (0.0006) [2023-12-27 06:00:27,841][20347] Updated weights for policy 0, policy_version 49703 (0.0005) [2023-12-27 06:00:28,200][20381] Updated weights for policy 1, policy_version 49642 (0.0007) [2023-12-27 06:00:28,255][20381] Updated weights for policy 1, policy_version 49652 (0.0005) [2023-12-27 06:00:28,311][20381] Updated weights for policy 1, policy_version 49662 (0.0005) [2023-12-27 06:00:28,372][20381] Updated weights for policy 1, policy_version 49672 (0.0008) [2023-12-27 06:00:28,417][20347] Updated weights for policy 0, policy_version 49713 (0.0006) [2023-12-27 06:00:28,466][20347] Updated weights for policy 0, policy_version 49723 (0.0009) [2023-12-27 06:00:28,514][20347] Updated weights for policy 0, policy_version 49733 (0.0009) [2023-12-27 06:00:28,564][20347] Updated weights for policy 0, policy_version 49743 (0.0008) [2023-12-27 06:00:28,882][20381] Updated weights for policy 1, policy_version 49682 (0.0005) [2023-12-27 06:00:28,936][20381] Updated weights for policy 1, policy_version 49692 (0.0006) [2023-12-27 06:00:28,982][20381] Updated weights for policy 1, policy_version 49702 (0.0005) [2023-12-27 06:00:29,259][20347] Updated weights for policy 0, policy_version 49753 (0.0007) [2023-12-27 06:00:29,309][20347] Updated weights for policy 0, policy_version 49763 (0.0008) [2023-12-27 06:00:29,377][20347] Updated weights for policy 0, policy_version 49773 (0.0008) [2023-12-27 06:00:29,648][20381] Updated weights for policy 1, policy_version 49712 (0.0008) [2023-12-27 06:00:29,708][20381] Updated weights for policy 1, policy_version 49722 (0.0008) [2023-12-27 06:00:29,769][20381] Updated weights for policy 1, policy_version 49732 (0.0008) [2023-12-27 06:00:30,027][20347] Updated weights for policy 0, policy_version 49783 (0.0007) [2023-12-27 06:00:30,080][20347] Updated weights for policy 0, policy_version 49793 (0.0008) [2023-12-27 06:00:30,129][20347] Updated weights for policy 0, policy_version 49803 (0.0007) [2023-12-27 06:00:30,499][20381] Updated weights for policy 1, policy_version 49742 (0.0008) [2023-12-27 06:00:30,548][20381] Updated weights for policy 1, policy_version 49752 (0.0007) [2023-12-27 06:00:30,598][20381] Updated weights for policy 1, policy_version 49762 (0.0008) [2023-12-27 06:00:30,775][20347] Updated weights for policy 0, policy_version 49813 (0.0007) [2023-12-27 06:00:30,832][20347] Updated weights for policy 0, policy_version 49823 (0.0007) [2023-12-27 06:00:30,886][20347] Updated weights for policy 0, policy_version 49833 (0.0007) [2023-12-27 06:00:31,318][20381] Updated weights for policy 1, policy_version 49772 (0.0008) [2023-12-27 06:00:31,376][20381] Updated weights for policy 1, policy_version 49782 (0.0009) [2023-12-27 06:00:31,430][20381] Updated weights for policy 1, policy_version 49792 (0.0008) [2023-12-27 06:00:31,569][20347] Updated weights for policy 0, policy_version 49843 (0.0007) [2023-12-27 06:00:31,629][20347] Updated weights for policy 0, policy_version 49853 (0.0008) [2023-12-27 06:00:31,685][20347] Updated weights for policy 0, policy_version 49863 (0.0007) [2023-12-27 06:00:32,129][20381] Updated weights for policy 1, policy_version 49802 (0.0008) [2023-12-27 06:00:32,184][20381] Updated weights for policy 1, policy_version 49812 (0.0009) [2023-12-27 06:00:32,239][20381] Updated weights for policy 1, policy_version 49822 (0.0008) [2023-12-27 06:00:32,294][20381] Updated weights for policy 1, policy_version 49832 (0.0010) [2023-12-27 06:00:32,337][20347] Updated weights for policy 0, policy_version 49873 (0.0006) [2023-12-27 06:00:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 25534464. Throughput: 0: 10476.2, 1: 10068.6. Samples: 25543884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:32,393][20347] Updated weights for policy 0, policy_version 49883 (0.0007) [2023-12-27 06:00:32,441][20347] Updated weights for policy 0, policy_version 49893 (0.0007) [2023-12-27 06:00:32,497][20347] Updated weights for policy 0, policy_version 49903 (0.0005) [2023-12-27 06:00:32,942][20381] Updated weights for policy 1, policy_version 49842 (0.0006) [2023-12-27 06:00:32,991][20381] Updated weights for policy 1, policy_version 49852 (0.0008) [2023-12-27 06:00:33,034][20381] Updated weights for policy 1, policy_version 49862 (0.0005) [2023-12-27 06:00:33,149][20347] Updated weights for policy 0, policy_version 49913 (0.0007) [2023-12-27 06:00:33,205][20347] Updated weights for policy 0, policy_version 49923 (0.0007) [2023-12-27 06:00:33,261][20347] Updated weights for policy 0, policy_version 49933 (0.0008) [2023-12-27 06:00:33,605][20381] Updated weights for policy 1, policy_version 49872 (0.0009) [2023-12-27 06:00:33,650][20381] Updated weights for policy 1, policy_version 49882 (0.0010) [2023-12-27 06:00:33,695][20381] Updated weights for policy 1, policy_version 49892 (0.0010) [2023-12-27 06:00:33,984][20347] Updated weights for policy 0, policy_version 49943 (0.0007) [2023-12-27 06:00:34,043][20347] Updated weights for policy 0, policy_version 49953 (0.0008) [2023-12-27 06:00:34,106][20347] Updated weights for policy 0, policy_version 49963 (0.0008) [2023-12-27 06:00:34,371][20381] Updated weights for policy 1, policy_version 49902 (0.0010) [2023-12-27 06:00:34,430][20381] Updated weights for policy 1, policy_version 49912 (0.0010) [2023-12-27 06:00:34,487][20381] Updated weights for policy 1, policy_version 49922 (0.0009) [2023-12-27 06:00:34,784][20347] Updated weights for policy 0, policy_version 49973 (0.0006) [2023-12-27 06:00:34,843][20347] Updated weights for policy 0, policy_version 49983 (0.0005) [2023-12-27 06:00:34,902][20347] Updated weights for policy 0, policy_version 49993 (0.0005) [2023-12-27 06:00:35,150][20381] Updated weights for policy 1, policy_version 49932 (0.0011) [2023-12-27 06:00:35,206][20381] Updated weights for policy 1, policy_version 49942 (0.0006) [2023-12-27 06:00:35,261][20381] Updated weights for policy 1, policy_version 49952 (0.0006) [2023-12-27 06:00:35,468][20347] Updated weights for policy 0, policy_version 50003 (0.0006) [2023-12-27 06:00:35,524][20347] Updated weights for policy 0, policy_version 50013 (0.0007) [2023-12-27 06:00:35,578][20347] Updated weights for policy 0, policy_version 50023 (0.0007) [2023-12-27 06:00:35,935][20381] Updated weights for policy 1, policy_version 49962 (0.0010) [2023-12-27 06:00:35,990][20381] Updated weights for policy 1, policy_version 49972 (0.0011) [2023-12-27 06:00:36,054][20381] Updated weights for policy 1, policy_version 49982 (0.0011) [2023-12-27 06:00:36,115][20381] Updated weights for policy 1, policy_version 49992 (0.0010) [2023-12-27 06:00:36,274][20347] Updated weights for policy 0, policy_version 50033 (0.0007) [2023-12-27 06:00:36,330][20347] Updated weights for policy 0, policy_version 50043 (0.0008) [2023-12-27 06:00:36,379][20347] Updated weights for policy 0, policy_version 50053 (0.0008) [2023-12-27 06:00:36,434][20347] Updated weights for policy 0, policy_version 50063 (0.0006) [2023-12-27 06:00:36,771][20381] Updated weights for policy 1, policy_version 50002 (0.0008) [2023-12-27 06:00:36,828][20381] Updated weights for policy 1, policy_version 50012 (0.0011) [2023-12-27 06:00:36,877][20381] Updated weights for policy 1, policy_version 50022 (0.0010) [2023-12-27 06:00:37,135][20347] Updated weights for policy 0, policy_version 50073 (0.0007) [2023-12-27 06:00:37,199][20347] Updated weights for policy 0, policy_version 50083 (0.0009) [2023-12-27 06:00:37,263][20347] Updated weights for policy 0, policy_version 50093 (0.0007) [2023-12-27 06:00:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 25640960. Throughput: 0: 10466.1, 1: 10132.5. Samples: 25608236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:00:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000050024_12812288.pth... [2023-12-27 06:00:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000050096_12828672.pth... [2023-12-27 06:00:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000048880_12517376.pth [2023-12-27 06:00:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000048840_12509184.pth [2023-12-27 06:00:37,542][20381] Updated weights for policy 1, policy_version 50032 (0.0010) [2023-12-27 06:00:37,597][20381] Updated weights for policy 1, policy_version 50042 (0.0010) [2023-12-27 06:00:37,658][20381] Updated weights for policy 1, policy_version 50052 (0.0006) [2023-12-27 06:00:37,956][20347] Updated weights for policy 0, policy_version 50103 (0.0008) [2023-12-27 06:00:38,021][20347] Updated weights for policy 0, policy_version 50113 (0.0008) [2023-12-27 06:00:38,081][20347] Updated weights for policy 0, policy_version 50123 (0.0010) [2023-12-27 06:00:38,288][20381] Updated weights for policy 1, policy_version 50062 (0.0009) [2023-12-27 06:00:38,350][20381] Updated weights for policy 1, policy_version 50072 (0.0010) [2023-12-27 06:00:38,405][20381] Updated weights for policy 1, policy_version 50082 (0.0009) [2023-12-27 06:00:38,762][20347] Updated weights for policy 0, policy_version 50133 (0.0008) [2023-12-27 06:00:38,820][20347] Updated weights for policy 0, policy_version 50143 (0.0005) [2023-12-27 06:00:38,882][20347] Updated weights for policy 0, policy_version 50153 (0.0006) [2023-12-27 06:00:39,043][20381] Updated weights for policy 1, policy_version 50092 (0.0008) [2023-12-27 06:00:39,089][20381] Updated weights for policy 1, policy_version 50102 (0.0011) [2023-12-27 06:00:39,135][20381] Updated weights for policy 1, policy_version 50112 (0.0007) [2023-12-27 06:00:39,485][20347] Updated weights for policy 0, policy_version 50163 (0.0006) [2023-12-27 06:00:39,537][20347] Updated weights for policy 0, policy_version 50173 (0.0008) [2023-12-27 06:00:39,585][20347] Updated weights for policy 0, policy_version 50183 (0.0008) [2023-12-27 06:00:39,841][20381] Updated weights for policy 1, policy_version 50122 (0.0010) [2023-12-27 06:00:39,896][20381] Updated weights for policy 1, policy_version 50132 (0.0008) [2023-12-27 06:00:39,959][20381] Updated weights for policy 1, policy_version 50142 (0.0008) [2023-12-27 06:00:40,022][20381] Updated weights for policy 1, policy_version 50152 (0.0007) [2023-12-27 06:00:40,335][20347] Updated weights for policy 0, policy_version 50193 (0.0008) [2023-12-27 06:00:40,398][20347] Updated weights for policy 0, policy_version 50203 (0.0010) [2023-12-27 06:00:40,464][20347] Updated weights for policy 0, policy_version 50213 (0.0010) [2023-12-27 06:00:40,520][20347] Updated weights for policy 0, policy_version 50223 (0.0010) [2023-12-27 06:00:40,683][20381] Updated weights for policy 1, policy_version 50162 (0.0008) [2023-12-27 06:00:40,740][20381] Updated weights for policy 1, policy_version 50172 (0.0007) [2023-12-27 06:00:40,800][20381] Updated weights for policy 1, policy_version 50182 (0.0007) [2023-12-27 06:00:41,183][20347] Updated weights for policy 0, policy_version 50233 (0.0008) [2023-12-27 06:00:41,237][20347] Updated weights for policy 0, policy_version 50243 (0.0008) [2023-12-27 06:00:41,303][20347] Updated weights for policy 0, policy_version 50253 (0.0008) [2023-12-27 06:00:41,521][20381] Updated weights for policy 1, policy_version 50192 (0.0007) [2023-12-27 06:00:41,585][20381] Updated weights for policy 1, policy_version 50202 (0.0008) [2023-12-27 06:00:41,649][20381] Updated weights for policy 1, policy_version 50212 (0.0007) [2023-12-27 06:00:42,031][20347] Updated weights for policy 0, policy_version 50263 (0.0008) [2023-12-27 06:00:42,081][20347] Updated weights for policy 0, policy_version 50273 (0.0008) [2023-12-27 06:00:42,137][20347] Updated weights for policy 0, policy_version 50283 (0.0007) [2023-12-27 06:00:42,353][20381] Updated weights for policy 1, policy_version 50222 (0.0007) [2023-12-27 06:00:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 25739264. Throughput: 0: 10411.0, 1: 10162.8. Samples: 25731624. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:00:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:42,413][20381] Updated weights for policy 1, policy_version 50232 (0.0008) [2023-12-27 06:00:42,457][20381] Updated weights for policy 1, policy_version 50242 (0.0007) [2023-12-27 06:00:42,824][20347] Updated weights for policy 0, policy_version 50293 (0.0007) [2023-12-27 06:00:42,874][20347] Updated weights for policy 0, policy_version 50303 (0.0008) [2023-12-27 06:00:42,919][20347] Updated weights for policy 0, policy_version 50313 (0.0008) [2023-12-27 06:00:43,108][20381] Updated weights for policy 1, policy_version 50252 (0.0005) [2023-12-27 06:00:43,157][20381] Updated weights for policy 1, policy_version 50262 (0.0005) [2023-12-27 06:00:43,209][20381] Updated weights for policy 1, policy_version 50272 (0.0005) [2023-12-27 06:00:43,609][20347] Updated weights for policy 0, policy_version 50323 (0.0008) [2023-12-27 06:00:43,667][20347] Updated weights for policy 0, policy_version 50333 (0.0006) [2023-12-27 06:00:43,720][20347] Updated weights for policy 0, policy_version 50343 (0.0006) [2023-12-27 06:00:43,780][20381] Updated weights for policy 1, policy_version 50282 (0.0006) [2023-12-27 06:00:43,828][20381] Updated weights for policy 1, policy_version 50292 (0.0010) [2023-12-27 06:00:43,875][20381] Updated weights for policy 1, policy_version 50302 (0.0006) [2023-12-27 06:00:43,936][20381] Updated weights for policy 1, policy_version 50312 (0.0005) [2023-12-27 06:00:44,400][20347] Updated weights for policy 0, policy_version 50353 (0.0007) [2023-12-27 06:00:44,451][20347] Updated weights for policy 0, policy_version 50363 (0.0007) [2023-12-27 06:00:44,502][20347] Updated weights for policy 0, policy_version 50373 (0.0009) [2023-12-27 06:00:44,541][20381] Updated weights for policy 1, policy_version 50322 (0.0005) [2023-12-27 06:00:44,558][20347] Updated weights for policy 0, policy_version 50383 (0.0008) [2023-12-27 06:00:44,604][20381] Updated weights for policy 1, policy_version 50332 (0.0010) [2023-12-27 06:00:44,657][20381] Updated weights for policy 1, policy_version 50342 (0.0010) [2023-12-27 06:00:45,239][20347] Updated weights for policy 0, policy_version 50393 (0.0007) [2023-12-27 06:00:45,297][20347] Updated weights for policy 0, policy_version 50403 (0.0007) [2023-12-27 06:00:45,325][20381] Updated weights for policy 1, policy_version 50352 (0.0010) [2023-12-27 06:00:45,346][20347] Updated weights for policy 0, policy_version 50413 (0.0006) [2023-12-27 06:00:45,380][20381] Updated weights for policy 1, policy_version 50362 (0.0010) [2023-12-27 06:00:45,435][20381] Updated weights for policy 1, policy_version 50372 (0.0010) [2023-12-27 06:00:45,992][20347] Updated weights for policy 0, policy_version 50423 (0.0009) [2023-12-27 06:00:46,010][20381] Updated weights for policy 1, policy_version 50382 (0.0005) [2023-12-27 06:00:46,052][20347] Updated weights for policy 0, policy_version 50433 (0.0010) [2023-12-27 06:00:46,069][20381] Updated weights for policy 1, policy_version 50392 (0.0005) [2023-12-27 06:00:46,113][20347] Updated weights for policy 0, policy_version 50443 (0.0007) [2023-12-27 06:00:46,135][20381] Updated weights for policy 1, policy_version 50402 (0.0007) [2023-12-27 06:00:46,682][20347] Updated weights for policy 0, policy_version 50453 (0.0005) [2023-12-27 06:00:46,737][20381] Updated weights for policy 1, policy_version 50412 (0.0007) [2023-12-27 06:00:46,738][20347] Updated weights for policy 0, policy_version 50463 (0.0008) [2023-12-27 06:00:46,788][20347] Updated weights for policy 0, policy_version 50473 (0.0007) [2023-12-27 06:00:46,793][20381] Updated weights for policy 1, policy_version 50422 (0.0005) [2023-12-27 06:00:46,849][20381] Updated weights for policy 1, policy_version 50432 (0.0006) [2023-12-27 06:00:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 25845760. Throughput: 0: 10381.9, 1: 10278.0. Samples: 25858872. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:00:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:47,382][20347] Updated weights for policy 0, policy_version 50483 (0.0008) [2023-12-27 06:00:47,443][20347] Updated weights for policy 0, policy_version 50493 (0.0010) [2023-12-27 06:00:47,500][20347] Updated weights for policy 0, policy_version 50503 (0.0009) [2023-12-27 06:00:47,532][20381] Updated weights for policy 1, policy_version 50442 (0.0005) [2023-12-27 06:00:47,582][20381] Updated weights for policy 1, policy_version 50452 (0.0006) [2023-12-27 06:00:47,645][20381] Updated weights for policy 1, policy_version 50462 (0.0005) [2023-12-27 06:00:47,695][20381] Updated weights for policy 1, policy_version 50472 (0.0007) [2023-12-27 06:00:48,170][20347] Updated weights for policy 0, policy_version 50513 (0.0010) [2023-12-27 06:00:48,231][20347] Updated weights for policy 0, policy_version 50523 (0.0007) [2023-12-27 06:00:48,292][20347] Updated weights for policy 0, policy_version 50533 (0.0007) [2023-12-27 06:00:48,355][20347] Updated weights for policy 0, policy_version 50543 (0.0007) [2023-12-27 06:00:48,368][20381] Updated weights for policy 1, policy_version 50482 (0.0007) [2023-12-27 06:00:48,430][20381] Updated weights for policy 1, policy_version 50492 (0.0009) [2023-12-27 06:00:48,489][20381] Updated weights for policy 1, policy_version 50502 (0.0008) [2023-12-27 06:00:49,022][20347] Updated weights for policy 0, policy_version 50553 (0.0006) [2023-12-27 06:00:49,070][20347] Updated weights for policy 0, policy_version 50563 (0.0005) [2023-12-27 06:00:49,127][20347] Updated weights for policy 0, policy_version 50573 (0.0006) [2023-12-27 06:00:49,192][20381] Updated weights for policy 1, policy_version 50512 (0.0008) [2023-12-27 06:00:49,254][20381] Updated weights for policy 1, policy_version 50522 (0.0007) [2023-12-27 06:00:49,304][20381] Updated weights for policy 1, policy_version 50532 (0.0009) [2023-12-27 06:00:49,712][20347] Updated weights for policy 0, policy_version 50583 (0.0007) [2023-12-27 06:00:49,775][20347] Updated weights for policy 0, policy_version 50593 (0.0007) [2023-12-27 06:00:49,838][20347] Updated weights for policy 0, policy_version 50603 (0.0006) [2023-12-27 06:00:50,078][20381] Updated weights for policy 1, policy_version 50542 (0.0007) [2023-12-27 06:00:50,134][20381] Updated weights for policy 1, policy_version 50552 (0.0008) [2023-12-27 06:00:50,186][20381] Updated weights for policy 1, policy_version 50562 (0.0008) [2023-12-27 06:00:50,557][20347] Updated weights for policy 0, policy_version 50613 (0.0008) [2023-12-27 06:00:50,625][20347] Updated weights for policy 0, policy_version 50623 (0.0008) [2023-12-27 06:00:50,678][20347] Updated weights for policy 0, policy_version 50633 (0.0008) [2023-12-27 06:00:50,792][20381] Updated weights for policy 1, policy_version 50572 (0.0007) [2023-12-27 06:00:50,846][20381] Updated weights for policy 1, policy_version 50582 (0.0006) [2023-12-27 06:00:50,903][20381] Updated weights for policy 1, policy_version 50592 (0.0006) [2023-12-27 06:00:51,391][20347] Updated weights for policy 0, policy_version 50643 (0.0008) [2023-12-27 06:00:51,457][20347] Updated weights for policy 0, policy_version 50653 (0.0007) [2023-12-27 06:00:51,520][20347] Updated weights for policy 0, policy_version 50663 (0.0007) [2023-12-27 06:00:51,574][20381] Updated weights for policy 1, policy_version 50602 (0.0006) [2023-12-27 06:00:51,635][20381] Updated weights for policy 1, policy_version 50612 (0.0007) [2023-12-27 06:00:51,690][20381] Updated weights for policy 1, policy_version 50622 (0.0007) [2023-12-27 06:00:51,758][20381] Updated weights for policy 1, policy_version 50632 (0.0009) [2023-12-27 06:00:52,240][20347] Updated weights for policy 0, policy_version 50673 (0.0007) [2023-12-27 06:00:52,302][20347] Updated weights for policy 0, policy_version 50683 (0.0008) [2023-12-27 06:00:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 25944064. Throughput: 0: 10385.2, 1: 10228.7. Samples: 25920200. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:00:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:52,365][20347] Updated weights for policy 0, policy_version 50693 (0.0007) [2023-12-27 06:00:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000050632_12967936.pth... [2023-12-27 06:00:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000049416_12656640.pth [2023-12-27 06:00:52,428][20347] Updated weights for policy 0, policy_version 50703 (0.0006) [2023-12-27 06:00:52,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000050704_12984320.pth... [2023-12-27 06:00:52,433][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000049488_12673024.pth [2023-12-27 06:00:52,479][20381] Updated weights for policy 1, policy_version 50642 (0.0008) [2023-12-27 06:00:52,544][20381] Updated weights for policy 1, policy_version 50652 (0.0009) [2023-12-27 06:00:52,609][20381] Updated weights for policy 1, policy_version 50662 (0.0008) [2023-12-27 06:00:53,122][20347] Updated weights for policy 0, policy_version 50713 (0.0010) [2023-12-27 06:00:53,180][20347] Updated weights for policy 0, policy_version 50723 (0.0010) [2023-12-27 06:00:53,243][20347] Updated weights for policy 0, policy_version 50733 (0.0009) [2023-12-27 06:00:53,247][20381] Updated weights for policy 1, policy_version 50672 (0.0009) [2023-12-27 06:00:53,305][20381] Updated weights for policy 1, policy_version 50682 (0.0008) [2023-12-27 06:00:53,357][20381] Updated weights for policy 1, policy_version 50692 (0.0008) [2023-12-27 06:00:53,917][20347] Updated weights for policy 0, policy_version 50743 (0.0010) [2023-12-27 06:00:53,965][20347] Updated weights for policy 0, policy_version 50753 (0.0010) [2023-12-27 06:00:54,013][20347] Updated weights for policy 0, policy_version 50763 (0.0010) [2023-12-27 06:00:54,054][20381] Updated weights for policy 1, policy_version 50702 (0.0007) [2023-12-27 06:00:54,098][20381] Updated weights for policy 1, policy_version 50712 (0.0007) [2023-12-27 06:00:54,142][20381] Updated weights for policy 1, policy_version 50722 (0.0008) [2023-12-27 06:00:54,641][20347] Updated weights for policy 0, policy_version 50773 (0.0009) [2023-12-27 06:00:54,687][20347] Updated weights for policy 0, policy_version 50783 (0.0008) [2023-12-27 06:00:54,733][20347] Updated weights for policy 0, policy_version 50793 (0.0008) [2023-12-27 06:00:54,882][20381] Updated weights for policy 1, policy_version 50732 (0.0007) [2023-12-27 06:00:54,932][20381] Updated weights for policy 1, policy_version 50742 (0.0008) [2023-12-27 06:00:54,985][20381] Updated weights for policy 1, policy_version 50752 (0.0007) [2023-12-27 06:00:55,419][20347] Updated weights for policy 0, policy_version 50803 (0.0007) [2023-12-27 06:00:55,485][20347] Updated weights for policy 0, policy_version 50813 (0.0006) [2023-12-27 06:00:55,544][20347] Updated weights for policy 0, policy_version 50823 (0.0005) [2023-12-27 06:00:55,586][20381] Updated weights for policy 1, policy_version 50762 (0.0007) [2023-12-27 06:00:55,644][20381] Updated weights for policy 1, policy_version 50772 (0.0011) [2023-12-27 06:00:55,706][20381] Updated weights for policy 1, policy_version 50782 (0.0011) [2023-12-27 06:00:55,765][20381] Updated weights for policy 1, policy_version 50792 (0.0011) [2023-12-27 06:00:56,165][20347] Updated weights for policy 0, policy_version 50833 (0.0007) [2023-12-27 06:00:56,216][20347] Updated weights for policy 0, policy_version 50843 (0.0010) [2023-12-27 06:00:56,264][20347] Updated weights for policy 0, policy_version 50853 (0.0010) [2023-12-27 06:00:56,309][20347] Updated weights for policy 0, policy_version 50863 (0.0010) [2023-12-27 06:00:56,456][20381] Updated weights for policy 1, policy_version 50802 (0.0005) [2023-12-27 06:00:56,513][20381] Updated weights for policy 1, policy_version 50812 (0.0005) [2023-12-27 06:00:56,573][20381] Updated weights for policy 1, policy_version 50822 (0.0005) [2023-12-27 06:00:56,997][20347] Updated weights for policy 0, policy_version 50873 (0.0010) [2023-12-27 06:00:57,056][20381] Updated weights for policy 1, policy_version 50832 (0.0005) [2023-12-27 06:00:57,058][20347] Updated weights for policy 0, policy_version 50883 (0.0010) [2023-12-27 06:00:57,104][20347] Updated weights for policy 0, policy_version 50893 (0.0009) [2023-12-27 06:00:57,116][20381] Updated weights for policy 1, policy_version 50842 (0.0006) [2023-12-27 06:00:57,165][20381] Updated weights for policy 1, policy_version 50852 (0.0011) [2023-12-27 06:00:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 26058752. Throughput: 0: 10345.6, 1: 10292.9. Samples: 26043400. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:00:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:00:57,779][20381] Updated weights for policy 1, policy_version 50862 (0.0007) [2023-12-27 06:00:57,782][20347] Updated weights for policy 0, policy_version 50903 (0.0008) [2023-12-27 06:00:57,830][20347] Updated weights for policy 0, policy_version 50913 (0.0007) [2023-12-27 06:00:57,831][20381] Updated weights for policy 1, policy_version 50872 (0.0005) [2023-12-27 06:00:57,886][20347] Updated weights for policy 0, policy_version 50923 (0.0007) [2023-12-27 06:00:57,887][20381] Updated weights for policy 1, policy_version 50882 (0.0005) [2023-12-27 06:00:58,475][20381] Updated weights for policy 1, policy_version 50892 (0.0006) [2023-12-27 06:00:58,535][20381] Updated weights for policy 1, policy_version 50902 (0.0008) [2023-12-27 06:00:58,602][20381] Updated weights for policy 1, policy_version 50912 (0.0008) [2023-12-27 06:00:58,663][20347] Updated weights for policy 0, policy_version 50933 (0.0007) [2023-12-27 06:00:58,732][20347] Updated weights for policy 0, policy_version 50943 (0.0008) [2023-12-27 06:00:58,794][20347] Updated weights for policy 0, policy_version 50953 (0.0008) [2023-12-27 06:00:59,335][20381] Updated weights for policy 1, policy_version 50922 (0.0008) [2023-12-27 06:00:59,396][20381] Updated weights for policy 1, policy_version 50932 (0.0008) [2023-12-27 06:00:59,457][20381] Updated weights for policy 1, policy_version 50942 (0.0008) [2023-12-27 06:00:59,509][20381] Updated weights for policy 1, policy_version 50952 (0.0008) [2023-12-27 06:00:59,567][20347] Updated weights for policy 0, policy_version 50963 (0.0007) [2023-12-27 06:00:59,627][20347] Updated weights for policy 0, policy_version 50973 (0.0007) [2023-12-27 06:00:59,686][20347] Updated weights for policy 0, policy_version 50983 (0.0008) [2023-12-27 06:01:00,186][20381] Updated weights for policy 1, policy_version 50962 (0.0007) [2023-12-27 06:01:00,231][20381] Updated weights for policy 1, policy_version 50972 (0.0007) [2023-12-27 06:01:00,277][20381] Updated weights for policy 1, policy_version 50982 (0.0007) [2023-12-27 06:01:00,366][20347] Updated weights for policy 0, policy_version 50993 (0.0008) [2023-12-27 06:01:00,422][20347] Updated weights for policy 0, policy_version 51003 (0.0006) [2023-12-27 06:01:00,470][20347] Updated weights for policy 0, policy_version 51013 (0.0006) [2023-12-27 06:01:00,522][20347] Updated weights for policy 0, policy_version 51023 (0.0007) [2023-12-27 06:01:00,996][20381] Updated weights for policy 1, policy_version 50992 (0.0008) [2023-12-27 06:01:01,054][20381] Updated weights for policy 1, policy_version 51002 (0.0009) [2023-12-27 06:01:01,106][20381] Updated weights for policy 1, policy_version 51012 (0.0007) [2023-12-27 06:01:01,132][20347] Updated weights for policy 0, policy_version 51033 (0.0008) [2023-12-27 06:01:01,199][20347] Updated weights for policy 0, policy_version 51043 (0.0008) [2023-12-27 06:01:01,260][20347] Updated weights for policy 0, policy_version 51053 (0.0010) [2023-12-27 06:01:01,791][20381] Updated weights for policy 1, policy_version 51022 (0.0007) [2023-12-27 06:01:01,842][20381] Updated weights for policy 1, policy_version 51032 (0.0007) [2023-12-27 06:01:01,897][20381] Updated weights for policy 1, policy_version 51042 (0.0008) [2023-12-27 06:01:01,951][20347] Updated weights for policy 0, policy_version 51063 (0.0007) [2023-12-27 06:01:02,001][20347] Updated weights for policy 0, policy_version 51073 (0.0005) [2023-12-27 06:01:02,045][20347] Updated weights for policy 0, policy_version 51083 (0.0005) [2023-12-27 06:01:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 26157056. Throughput: 0: 10324.6, 1: 10399.3. Samples: 26167036. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:01:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:02,619][20381] Updated weights for policy 1, policy_version 51052 (0.0007) [2023-12-27 06:01:02,654][20347] Updated weights for policy 0, policy_version 51093 (0.0006) [2023-12-27 06:01:02,679][20381] Updated weights for policy 1, policy_version 51062 (0.0007) [2023-12-27 06:01:02,710][20347] Updated weights for policy 0, policy_version 51103 (0.0005) [2023-12-27 06:01:02,728][20381] Updated weights for policy 1, policy_version 51072 (0.0008) [2023-12-27 06:01:02,772][20347] Updated weights for policy 0, policy_version 51113 (0.0006) [2023-12-27 06:01:03,356][20347] Updated weights for policy 0, policy_version 51123 (0.0008) [2023-12-27 06:01:03,410][20347] Updated weights for policy 0, policy_version 51133 (0.0010) [2023-12-27 06:01:03,464][20381] Updated weights for policy 1, policy_version 51082 (0.0007) [2023-12-27 06:01:03,467][20347] Updated weights for policy 0, policy_version 51143 (0.0010) [2023-12-27 06:01:03,516][20381] Updated weights for policy 1, policy_version 51092 (0.0005) [2023-12-27 06:01:03,570][20381] Updated weights for policy 1, policy_version 51102 (0.0007) [2023-12-27 06:01:03,624][20381] Updated weights for policy 1, policy_version 51112 (0.0007) [2023-12-27 06:01:04,090][20347] Updated weights for policy 0, policy_version 51153 (0.0010) [2023-12-27 06:01:04,152][20347] Updated weights for policy 0, policy_version 51163 (0.0010) [2023-12-27 06:01:04,210][20347] Updated weights for policy 0, policy_version 51173 (0.0010) [2023-12-27 06:01:04,270][20347] Updated weights for policy 0, policy_version 51183 (0.0010) [2023-12-27 06:01:04,364][20381] Updated weights for policy 1, policy_version 51122 (0.0008) [2023-12-27 06:01:04,420][20381] Updated weights for policy 1, policy_version 51132 (0.0008) [2023-12-27 06:01:04,479][20381] Updated weights for policy 1, policy_version 51142 (0.0008) [2023-12-27 06:01:04,959][20347] Updated weights for policy 0, policy_version 51193 (0.0009) [2023-12-27 06:01:05,016][20347] Updated weights for policy 0, policy_version 51203 (0.0005) [2023-12-27 06:01:05,079][20347] Updated weights for policy 0, policy_version 51213 (0.0005) [2023-12-27 06:01:05,148][20381] Updated weights for policy 1, policy_version 51152 (0.0007) [2023-12-27 06:01:05,209][20381] Updated weights for policy 1, policy_version 51162 (0.0005) [2023-12-27 06:01:05,269][20381] Updated weights for policy 1, policy_version 51172 (0.0005) [2023-12-27 06:01:05,742][20347] Updated weights for policy 0, policy_version 51223 (0.0007) [2023-12-27 06:01:05,795][20347] Updated weights for policy 0, policy_version 51233 (0.0008) [2023-12-27 06:01:05,831][20381] Updated weights for policy 1, policy_version 51182 (0.0005) [2023-12-27 06:01:05,852][20347] Updated weights for policy 0, policy_version 51243 (0.0007) [2023-12-27 06:01:05,881][20381] Updated weights for policy 1, policy_version 51192 (0.0005) [2023-12-27 06:01:05,929][20381] Updated weights for policy 1, policy_version 51202 (0.0006) [2023-12-27 06:01:06,583][20381] Updated weights for policy 1, policy_version 51212 (0.0008) [2023-12-27 06:01:06,593][20347] Updated weights for policy 0, policy_version 51253 (0.0008) [2023-12-27 06:01:06,641][20381] Updated weights for policy 1, policy_version 51222 (0.0007) [2023-12-27 06:01:06,644][20347] Updated weights for policy 0, policy_version 51263 (0.0007) [2023-12-27 06:01:06,694][20381] Updated weights for policy 1, policy_version 51232 (0.0007) [2023-12-27 06:01:06,704][20347] Updated weights for policy 0, policy_version 51273 (0.0007) [2023-12-27 06:01:07,347][20381] Updated weights for policy 1, policy_version 51242 (0.0007) [2023-12-27 06:01:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 26255360. Throughput: 0: 10370.9, 1: 10396.5. Samples: 26229472. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:01:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:07,374][20347] Updated weights for policy 0, policy_version 51283 (0.0007) [2023-12-27 06:01:07,405][20381] Updated weights for policy 1, policy_version 51252 (0.0006) [2023-12-27 06:01:07,430][20347] Updated weights for policy 0, policy_version 51293 (0.0006) [2023-12-27 06:01:07,462][20381] Updated weights for policy 1, policy_version 51262 (0.0005) [2023-12-27 06:01:07,484][20347] Updated weights for policy 0, policy_version 51303 (0.0009) [2023-12-27 06:01:07,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000051272_13131776.pth... [2023-12-27 06:01:07,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000050024_12812288.pth [2023-12-27 06:01:07,515][20381] Updated weights for policy 1, policy_version 51272 (0.0007) [2023-12-27 06:01:07,535][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000051312_13139968.pth... [2023-12-27 06:01:07,538][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000050096_12828672.pth [2023-12-27 06:01:08,116][20381] Updated weights for policy 1, policy_version 51282 (0.0008) [2023-12-27 06:01:08,141][20347] Updated weights for policy 0, policy_version 51313 (0.0006) [2023-12-27 06:01:08,164][20381] Updated weights for policy 1, policy_version 51292 (0.0006) [2023-12-27 06:01:08,191][20347] Updated weights for policy 0, policy_version 51323 (0.0010) [2023-12-27 06:01:08,213][20381] Updated weights for policy 1, policy_version 51302 (0.0006) [2023-12-27 06:01:08,248][20347] Updated weights for policy 0, policy_version 51333 (0.0010) [2023-12-27 06:01:08,305][20347] Updated weights for policy 0, policy_version 51343 (0.0010) [2023-12-27 06:01:08,935][20381] Updated weights for policy 1, policy_version 51312 (0.0007) [2023-12-27 06:01:08,952][20347] Updated weights for policy 0, policy_version 51353 (0.0010) [2023-12-27 06:01:08,995][20381] Updated weights for policy 1, policy_version 51322 (0.0005) [2023-12-27 06:01:09,011][20347] Updated weights for policy 0, policy_version 51363 (0.0010) [2023-12-27 06:01:09,049][20381] Updated weights for policy 1, policy_version 51332 (0.0006) [2023-12-27 06:01:09,066][20347] Updated weights for policy 0, policy_version 51373 (0.0011) [2023-12-27 06:01:09,709][20381] Updated weights for policy 1, policy_version 51342 (0.0006) [2023-12-27 06:01:09,771][20381] Updated weights for policy 1, policy_version 51352 (0.0009) [2023-12-27 06:01:09,828][20347] Updated weights for policy 0, policy_version 51383 (0.0007) [2023-12-27 06:01:09,835][20381] Updated weights for policy 1, policy_version 51362 (0.0011) [2023-12-27 06:01:09,888][20347] Updated weights for policy 0, policy_version 51393 (0.0006) [2023-12-27 06:01:09,951][20347] Updated weights for policy 0, policy_version 51403 (0.0007) [2023-12-27 06:01:10,519][20381] Updated weights for policy 1, policy_version 51372 (0.0012) [2023-12-27 06:01:10,574][20381] Updated weights for policy 1, policy_version 51382 (0.0010) [2023-12-27 06:01:10,576][20347] Updated weights for policy 0, policy_version 51413 (0.0008) [2023-12-27 06:01:10,635][20381] Updated weights for policy 1, policy_version 51392 (0.0010) [2023-12-27 06:01:10,637][20347] Updated weights for policy 0, policy_version 51423 (0.0006) [2023-12-27 06:01:10,693][20347] Updated weights for policy 0, policy_version 51433 (0.0006) [2023-12-27 06:01:11,337][20381] Updated weights for policy 1, policy_version 51402 (0.0010) [2023-12-27 06:01:11,408][20381] Updated weights for policy 1, policy_version 51412 (0.0007) [2023-12-27 06:01:11,423][20347] Updated weights for policy 0, policy_version 51443 (0.0006) [2023-12-27 06:01:11,469][20381] Updated weights for policy 1, policy_version 51422 (0.0006) [2023-12-27 06:01:11,485][20347] Updated weights for policy 0, policy_version 51453 (0.0008) [2023-12-27 06:01:11,530][20381] Updated weights for policy 1, policy_version 51432 (0.0006) [2023-12-27 06:01:11,548][20347] Updated weights for policy 0, policy_version 51463 (0.0007) [2023-12-27 06:01:12,182][20381] Updated weights for policy 1, policy_version 51442 (0.0008) [2023-12-27 06:01:12,242][20381] Updated weights for policy 1, policy_version 51452 (0.0008) [2023-12-27 06:01:12,273][20347] Updated weights for policy 0, policy_version 51473 (0.0007) [2023-12-27 06:01:12,296][20381] Updated weights for policy 1, policy_version 51462 (0.0008) [2023-12-27 06:01:12,340][20347] Updated weights for policy 0, policy_version 51483 (0.0008) [2023-12-27 06:01:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 26361856. Throughput: 0: 10339.6, 1: 10445.6. Samples: 26353900. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:01:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:12,402][20347] Updated weights for policy 0, policy_version 51493 (0.0008) [2023-12-27 06:01:12,462][20347] Updated weights for policy 0, policy_version 51503 (0.0008) [2023-12-27 06:01:13,037][20381] Updated weights for policy 1, policy_version 51472 (0.0007) [2023-12-27 06:01:13,096][20381] Updated weights for policy 1, policy_version 51482 (0.0008) [2023-12-27 06:01:13,155][20347] Updated weights for policy 0, policy_version 51513 (0.0010) [2023-12-27 06:01:13,156][20381] Updated weights for policy 1, policy_version 51492 (0.0006) [2023-12-27 06:01:13,214][20347] Updated weights for policy 0, policy_version 51523 (0.0010) [2023-12-27 06:01:13,275][20347] Updated weights for policy 0, policy_version 51533 (0.0010) [2023-12-27 06:01:13,843][20381] Updated weights for policy 1, policy_version 51502 (0.0007) [2023-12-27 06:01:13,896][20381] Updated weights for policy 1, policy_version 51512 (0.0007) [2023-12-27 06:01:13,940][20347] Updated weights for policy 0, policy_version 51543 (0.0007) [2023-12-27 06:01:13,950][20381] Updated weights for policy 1, policy_version 51522 (0.0007) [2023-12-27 06:01:14,001][20347] Updated weights for policy 0, policy_version 51553 (0.0007) [2023-12-27 06:01:14,065][20347] Updated weights for policy 0, policy_version 51563 (0.0006) [2023-12-27 06:01:14,651][20381] Updated weights for policy 1, policy_version 51532 (0.0008) [2023-12-27 06:01:14,672][20347] Updated weights for policy 0, policy_version 51573 (0.0006) [2023-12-27 06:01:14,706][20381] Updated weights for policy 1, policy_version 51542 (0.0008) [2023-12-27 06:01:14,723][20347] Updated weights for policy 0, policy_version 51583 (0.0005) [2023-12-27 06:01:14,767][20381] Updated weights for policy 1, policy_version 51552 (0.0007) [2023-12-27 06:01:14,786][20347] Updated weights for policy 0, policy_version 51593 (0.0007) [2023-12-27 06:01:15,396][20347] Updated weights for policy 0, policy_version 51603 (0.0007) [2023-12-27 06:01:15,441][20381] Updated weights for policy 1, policy_version 51562 (0.0008) [2023-12-27 06:01:15,454][20347] Updated weights for policy 0, policy_version 51613 (0.0007) [2023-12-27 06:01:15,493][20381] Updated weights for policy 1, policy_version 51572 (0.0006) [2023-12-27 06:01:15,510][20347] Updated weights for policy 0, policy_version 51623 (0.0006) [2023-12-27 06:01:15,553][20381] Updated weights for policy 1, policy_version 51582 (0.0006) [2023-12-27 06:01:15,611][20381] Updated weights for policy 1, policy_version 51592 (0.0007) [2023-12-27 06:01:16,204][20347] Updated weights for policy 0, policy_version 51633 (0.0007) [2023-12-27 06:01:16,224][20381] Updated weights for policy 1, policy_version 51602 (0.0006) [2023-12-27 06:01:16,259][20347] Updated weights for policy 0, policy_version 51643 (0.0006) [2023-12-27 06:01:16,279][20381] Updated weights for policy 1, policy_version 51612 (0.0005) [2023-12-27 06:01:16,313][20347] Updated weights for policy 0, policy_version 51653 (0.0005) [2023-12-27 06:01:16,334][20381] Updated weights for policy 1, policy_version 51622 (0.0007) [2023-12-27 06:01:16,366][20347] Updated weights for policy 0, policy_version 51663 (0.0005) [2023-12-27 06:01:16,985][20347] Updated weights for policy 0, policy_version 51673 (0.0007) [2023-12-27 06:01:17,038][20347] Updated weights for policy 0, policy_version 51683 (0.0007) [2023-12-27 06:01:17,055][20381] Updated weights for policy 1, policy_version 51632 (0.0007) [2023-12-27 06:01:17,091][20347] Updated weights for policy 0, policy_version 51693 (0.0005) [2023-12-27 06:01:17,111][20381] Updated weights for policy 1, policy_version 51642 (0.0007) [2023-12-27 06:01:17,169][20381] Updated weights for policy 1, policy_version 51652 (0.0008) [2023-12-27 06:01:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 26468352. Throughput: 0: 10302.2, 1: 10406.8. Samples: 26475792. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:17,765][20347] Updated weights for policy 0, policy_version 51703 (0.0009) [2023-12-27 06:01:17,811][20347] Updated weights for policy 0, policy_version 51713 (0.0009) [2023-12-27 06:01:17,841][20381] Updated weights for policy 1, policy_version 51662 (0.0009) [2023-12-27 06:01:17,856][20347] Updated weights for policy 0, policy_version 51723 (0.0005) [2023-12-27 06:01:17,896][20381] Updated weights for policy 1, policy_version 51672 (0.0010) [2023-12-27 06:01:17,954][20381] Updated weights for policy 1, policy_version 51682 (0.0010) [2023-12-27 06:01:18,452][20347] Updated weights for policy 0, policy_version 51733 (0.0007) [2023-12-27 06:01:18,508][20347] Updated weights for policy 0, policy_version 51743 (0.0009) [2023-12-27 06:01:18,563][20347] Updated weights for policy 0, policy_version 51753 (0.0008) [2023-12-27 06:01:18,654][20381] Updated weights for policy 1, policy_version 51692 (0.0009) [2023-12-27 06:01:18,703][20381] Updated weights for policy 1, policy_version 51702 (0.0010) [2023-12-27 06:01:18,751][20381] Updated weights for policy 1, policy_version 51712 (0.0010) [2023-12-27 06:01:19,208][20347] Updated weights for policy 0, policy_version 51763 (0.0007) [2023-12-27 06:01:19,265][20347] Updated weights for policy 0, policy_version 51773 (0.0011) [2023-12-27 06:01:19,318][20347] Updated weights for policy 0, policy_version 51783 (0.0010) [2023-12-27 06:01:19,464][20381] Updated weights for policy 1, policy_version 51722 (0.0011) [2023-12-27 06:01:19,526][20381] Updated weights for policy 1, policy_version 51732 (0.0011) [2023-12-27 06:01:19,589][20381] Updated weights for policy 1, policy_version 51742 (0.0011) [2023-12-27 06:01:19,652][20381] Updated weights for policy 1, policy_version 51752 (0.0011) [2023-12-27 06:01:20,060][20347] Updated weights for policy 0, policy_version 51793 (0.0010) [2023-12-27 06:01:20,123][20347] Updated weights for policy 0, policy_version 51803 (0.0007) [2023-12-27 06:01:20,187][20347] Updated weights for policy 0, policy_version 51813 (0.0007) [2023-12-27 06:01:20,251][20347] Updated weights for policy 0, policy_version 51823 (0.0009) [2023-12-27 06:01:20,365][20381] Updated weights for policy 1, policy_version 51762 (0.0008) [2023-12-27 06:01:20,419][20381] Updated weights for policy 1, policy_version 51772 (0.0011) [2023-12-27 06:01:20,478][20381] Updated weights for policy 1, policy_version 51782 (0.0010) [2023-12-27 06:01:20,959][20347] Updated weights for policy 0, policy_version 51833 (0.0007) [2023-12-27 06:01:21,014][20347] Updated weights for policy 0, policy_version 51843 (0.0007) [2023-12-27 06:01:21,080][20347] Updated weights for policy 0, policy_version 51853 (0.0006) [2023-12-27 06:01:21,199][20381] Updated weights for policy 1, policy_version 51792 (0.0009) [2023-12-27 06:01:21,251][20381] Updated weights for policy 1, policy_version 51802 (0.0008) [2023-12-27 06:01:21,314][20381] Updated weights for policy 1, policy_version 51812 (0.0007) [2023-12-27 06:01:21,763][20347] Updated weights for policy 0, policy_version 51863 (0.0007) [2023-12-27 06:01:21,820][20347] Updated weights for policy 0, policy_version 51873 (0.0007) [2023-12-27 06:01:21,876][20347] Updated weights for policy 0, policy_version 51883 (0.0008) [2023-12-27 06:01:22,044][20381] Updated weights for policy 1, policy_version 51822 (0.0007) [2023-12-27 06:01:22,101][20381] Updated weights for policy 1, policy_version 51832 (0.0008) [2023-12-27 06:01:22,161][20381] Updated weights for policy 1, policy_version 51842 (0.0008) [2023-12-27 06:01:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 26566656. Throughput: 0: 10318.0, 1: 10333.8. Samples: 26537564. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:22,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000051848_13279232.pth... [2023-12-27 06:01:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000051888_13287424.pth... [2023-12-27 06:01:22,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000050704_12984320.pth [2023-12-27 06:01:22,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000050632_12967936.pth [2023-12-27 06:01:22,569][20347] Updated weights for policy 0, policy_version 51893 (0.0009) [2023-12-27 06:01:22,627][20347] Updated weights for policy 0, policy_version 51903 (0.0010) [2023-12-27 06:01:22,691][20347] Updated weights for policy 0, policy_version 51913 (0.0011) [2023-12-27 06:01:22,890][20381] Updated weights for policy 1, policy_version 51852 (0.0008) [2023-12-27 06:01:22,950][20381] Updated weights for policy 1, policy_version 51862 (0.0008) [2023-12-27 06:01:23,008][20381] Updated weights for policy 1, policy_version 51872 (0.0007) [2023-12-27 06:01:23,370][20347] Updated weights for policy 0, policy_version 51923 (0.0009) [2023-12-27 06:01:23,424][20347] Updated weights for policy 0, policy_version 51933 (0.0005) [2023-12-27 06:01:23,473][20347] Updated weights for policy 0, policy_version 51943 (0.0006) [2023-12-27 06:01:23,689][20381] Updated weights for policy 1, policy_version 51882 (0.0007) [2023-12-27 06:01:23,760][20381] Updated weights for policy 1, policy_version 51892 (0.0007) [2023-12-27 06:01:23,828][20381] Updated weights for policy 1, policy_version 51902 (0.0009) [2023-12-27 06:01:23,881][20381] Updated weights for policy 1, policy_version 51912 (0.0009) [2023-12-27 06:01:24,092][20347] Updated weights for policy 0, policy_version 51953 (0.0006) [2023-12-27 06:01:24,148][20347] Updated weights for policy 0, policy_version 51963 (0.0009) [2023-12-27 06:01:24,200][20347] Updated weights for policy 0, policy_version 51973 (0.0008) [2023-12-27 06:01:24,249][20347] Updated weights for policy 0, policy_version 51983 (0.0008) [2023-12-27 06:01:24,502][20381] Updated weights for policy 1, policy_version 51922 (0.0006) [2023-12-27 06:01:24,560][20381] Updated weights for policy 1, policy_version 51932 (0.0007) [2023-12-27 06:01:24,617][20381] Updated weights for policy 1, policy_version 51942 (0.0008) [2023-12-27 06:01:24,967][20347] Updated weights for policy 0, policy_version 51993 (0.0005) [2023-12-27 06:01:25,023][20347] Updated weights for policy 0, policy_version 52003 (0.0006) [2023-12-27 06:01:25,072][20347] Updated weights for policy 0, policy_version 52013 (0.0008) [2023-12-27 06:01:25,259][20381] Updated weights for policy 1, policy_version 51952 (0.0008) [2023-12-27 06:01:25,316][20381] Updated weights for policy 1, policy_version 51962 (0.0007) [2023-12-27 06:01:25,367][20381] Updated weights for policy 1, policy_version 51972 (0.0007) [2023-12-27 06:01:25,628][20347] Updated weights for policy 0, policy_version 52023 (0.0007) [2023-12-27 06:01:25,685][20347] Updated weights for policy 0, policy_version 52033 (0.0006) [2023-12-27 06:01:25,733][20347] Updated weights for policy 0, policy_version 52043 (0.0005) [2023-12-27 06:01:26,107][20381] Updated weights for policy 1, policy_version 51982 (0.0008) [2023-12-27 06:01:26,156][20381] Updated weights for policy 1, policy_version 51992 (0.0006) [2023-12-27 06:01:26,203][20381] Updated weights for policy 1, policy_version 52002 (0.0005) [2023-12-27 06:01:26,344][20347] Updated weights for policy 0, policy_version 52053 (0.0005) [2023-12-27 06:01:26,396][20347] Updated weights for policy 0, policy_version 52063 (0.0005) [2023-12-27 06:01:26,459][20347] Updated weights for policy 0, policy_version 52073 (0.0005) [2023-12-27 06:01:26,823][20381] Updated weights for policy 1, policy_version 52012 (0.0005) [2023-12-27 06:01:26,878][20381] Updated weights for policy 1, policy_version 52022 (0.0005) [2023-12-27 06:01:26,933][20381] Updated weights for policy 1, policy_version 52032 (0.0005) [2023-12-27 06:01:27,047][20347] Updated weights for policy 0, policy_version 52083 (0.0008) [2023-12-27 06:01:27,092][20347] Updated weights for policy 0, policy_version 52093 (0.0008) [2023-12-27 06:01:27,144][20347] Updated weights for policy 0, policy_version 52103 (0.0006) [2023-12-27 06:01:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 26673152. Throughput: 0: 10340.3, 1: 10277.2. Samples: 26659412. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:27,460][20381] Updated weights for policy 1, policy_version 52042 (0.0006) [2023-12-27 06:01:27,521][20381] Updated weights for policy 1, policy_version 52052 (0.0007) [2023-12-27 06:01:27,578][20381] Updated weights for policy 1, policy_version 52062 (0.0005) [2023-12-27 06:01:27,638][20381] Updated weights for policy 1, policy_version 52072 (0.0005) [2023-12-27 06:01:27,853][20347] Updated weights for policy 0, policy_version 52113 (0.0006) [2023-12-27 06:01:27,898][20347] Updated weights for policy 0, policy_version 52123 (0.0008) [2023-12-27 06:01:27,942][20347] Updated weights for policy 0, policy_version 52133 (0.0008) [2023-12-27 06:01:27,987][20347] Updated weights for policy 0, policy_version 52143 (0.0009) [2023-12-27 06:01:28,178][20381] Updated weights for policy 1, policy_version 52082 (0.0007) [2023-12-27 06:01:28,237][20381] Updated weights for policy 1, policy_version 52092 (0.0007) [2023-12-27 06:01:28,292][20381] Updated weights for policy 1, policy_version 52102 (0.0007) [2023-12-27 06:01:28,707][20347] Updated weights for policy 0, policy_version 52153 (0.0005) [2023-12-27 06:01:28,767][20347] Updated weights for policy 0, policy_version 52163 (0.0005) [2023-12-27 06:01:28,825][20347] Updated weights for policy 0, policy_version 52173 (0.0005) [2023-12-27 06:01:28,966][20381] Updated weights for policy 1, policy_version 52112 (0.0006) [2023-12-27 06:01:29,012][20381] Updated weights for policy 1, policy_version 52122 (0.0006) [2023-12-27 06:01:29,058][20381] Updated weights for policy 1, policy_version 52132 (0.0009) [2023-12-27 06:01:29,347][20347] Updated weights for policy 0, policy_version 52183 (0.0007) [2023-12-27 06:01:29,401][20347] Updated weights for policy 0, policy_version 52193 (0.0008) [2023-12-27 06:01:29,455][20347] Updated weights for policy 0, policy_version 52203 (0.0008) [2023-12-27 06:01:29,783][20381] Updated weights for policy 1, policy_version 52142 (0.0008) [2023-12-27 06:01:29,830][20381] Updated weights for policy 1, policy_version 52152 (0.0008) [2023-12-27 06:01:29,882][20381] Updated weights for policy 1, policy_version 52162 (0.0008) [2023-12-27 06:01:30,148][20347] Updated weights for policy 0, policy_version 52213 (0.0007) [2023-12-27 06:01:30,208][20347] Updated weights for policy 0, policy_version 52223 (0.0007) [2023-12-27 06:01:30,261][20347] Updated weights for policy 0, policy_version 52233 (0.0007) [2023-12-27 06:01:30,610][20381] Updated weights for policy 1, policy_version 52172 (0.0008) [2023-12-27 06:01:30,662][20381] Updated weights for policy 1, policy_version 52182 (0.0007) [2023-12-27 06:01:30,713][20381] Updated weights for policy 1, policy_version 52192 (0.0008) [2023-12-27 06:01:30,925][20347] Updated weights for policy 0, policy_version 52243 (0.0007) [2023-12-27 06:01:30,971][20347] Updated weights for policy 0, policy_version 52253 (0.0005) [2023-12-27 06:01:31,024][20347] Updated weights for policy 0, policy_version 52263 (0.0005) [2023-12-27 06:01:31,335][20381] Updated weights for policy 1, policy_version 52202 (0.0008) [2023-12-27 06:01:31,386][20381] Updated weights for policy 1, policy_version 52212 (0.0008) [2023-12-27 06:01:31,447][20381] Updated weights for policy 1, policy_version 52222 (0.0006) [2023-12-27 06:01:31,503][20381] Updated weights for policy 1, policy_version 52232 (0.0006) [2023-12-27 06:01:31,698][20347] Updated weights for policy 0, policy_version 52273 (0.0008) [2023-12-27 06:01:31,757][20347] Updated weights for policy 0, policy_version 52283 (0.0008) [2023-12-27 06:01:31,818][20347] Updated weights for policy 0, policy_version 52293 (0.0008) [2023-12-27 06:01:31,878][20347] Updated weights for policy 0, policy_version 52303 (0.0008) [2023-12-27 06:01:32,195][20381] Updated weights for policy 1, policy_version 52242 (0.0008) [2023-12-27 06:01:32,255][20381] Updated weights for policy 1, policy_version 52252 (0.0007) [2023-12-27 06:01:32,309][20381] Updated weights for policy 1, policy_version 52262 (0.0008) [2023-12-27 06:01:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 26779648. Throughput: 0: 10368.4, 1: 10275.3. Samples: 26787836. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:32,586][20347] Updated weights for policy 0, policy_version 52313 (0.0007) [2023-12-27 06:01:32,639][20347] Updated weights for policy 0, policy_version 52323 (0.0008) [2023-12-27 06:01:32,696][20347] Updated weights for policy 0, policy_version 52333 (0.0007) [2023-12-27 06:01:32,987][20381] Updated weights for policy 1, policy_version 52272 (0.0008) [2023-12-27 06:01:33,044][20381] Updated weights for policy 1, policy_version 52282 (0.0008) [2023-12-27 06:01:33,094][20381] Updated weights for policy 1, policy_version 52292 (0.0006) [2023-12-27 06:01:33,342][20347] Updated weights for policy 0, policy_version 52343 (0.0006) [2023-12-27 06:01:33,402][20347] Updated weights for policy 0, policy_version 52353 (0.0005) [2023-12-27 06:01:33,457][20347] Updated weights for policy 0, policy_version 52363 (0.0006) [2023-12-27 06:01:33,757][20381] Updated weights for policy 1, policy_version 52302 (0.0007) [2023-12-27 06:01:33,815][20381] Updated weights for policy 1, policy_version 52312 (0.0005) [2023-12-27 06:01:33,868][20381] Updated weights for policy 1, policy_version 52322 (0.0005) [2023-12-27 06:01:34,052][20347] Updated weights for policy 0, policy_version 52373 (0.0006) [2023-12-27 06:01:34,102][20347] Updated weights for policy 0, policy_version 52383 (0.0007) [2023-12-27 06:01:34,157][20347] Updated weights for policy 0, policy_version 52393 (0.0007) [2023-12-27 06:01:34,478][20381] Updated weights for policy 1, policy_version 52332 (0.0007) [2023-12-27 06:01:34,531][20381] Updated weights for policy 1, policy_version 52342 (0.0010) [2023-12-27 06:01:34,583][20381] Updated weights for policy 1, policy_version 52352 (0.0010) [2023-12-27 06:01:34,789][20347] Updated weights for policy 0, policy_version 52403 (0.0007) [2023-12-27 06:01:34,837][20347] Updated weights for policy 0, policy_version 52413 (0.0007) [2023-12-27 06:01:34,896][20347] Updated weights for policy 0, policy_version 52423 (0.0005) [2023-12-27 06:01:35,263][20381] Updated weights for policy 1, policy_version 52362 (0.0010) [2023-12-27 06:01:35,323][20381] Updated weights for policy 1, policy_version 52372 (0.0008) [2023-12-27 06:01:35,384][20381] Updated weights for policy 1, policy_version 52382 (0.0008) [2023-12-27 06:01:35,434][20381] Updated weights for policy 1, policy_version 52392 (0.0009) [2023-12-27 06:01:35,468][20347] Updated weights for policy 0, policy_version 52433 (0.0005) [2023-12-27 06:01:35,527][20347] Updated weights for policy 0, policy_version 52443 (0.0005) [2023-12-27 06:01:35,578][20347] Updated weights for policy 0, policy_version 52453 (0.0005) [2023-12-27 06:01:35,637][20347] Updated weights for policy 0, policy_version 52463 (0.0006) [2023-12-27 06:01:36,098][20381] Updated weights for policy 1, policy_version 52402 (0.0008) [2023-12-27 06:01:36,160][20381] Updated weights for policy 1, policy_version 52412 (0.0007) [2023-12-27 06:01:36,184][20347] Updated weights for policy 0, policy_version 52473 (0.0008) [2023-12-27 06:01:36,212][20381] Updated weights for policy 1, policy_version 52422 (0.0007) [2023-12-27 06:01:36,236][20347] Updated weights for policy 0, policy_version 52483 (0.0008) [2023-12-27 06:01:36,288][20347] Updated weights for policy 0, policy_version 52493 (0.0008) [2023-12-27 06:01:36,851][20381] Updated weights for policy 1, policy_version 52432 (0.0007) [2023-12-27 06:01:36,898][20381] Updated weights for policy 1, policy_version 52442 (0.0008) [2023-12-27 06:01:36,951][20381] Updated weights for policy 1, policy_version 52452 (0.0008) [2023-12-27 06:01:37,051][20347] Updated weights for policy 0, policy_version 52503 (0.0008) [2023-12-27 06:01:37,103][20347] Updated weights for policy 0, policy_version 52513 (0.0007) [2023-12-27 06:01:37,166][20347] Updated weights for policy 0, policy_version 52523 (0.0008) [2023-12-27 06:01:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 26886144. Throughput: 0: 10377.6, 1: 10322.9. Samples: 26851720. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000052456_13434880.pth... [2023-12-27 06:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000052528_13451264.pth... [2023-12-27 06:01:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000051272_13131776.pth [2023-12-27 06:01:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000051312_13139968.pth [2023-12-27 06:01:37,656][20381] Updated weights for policy 1, policy_version 52462 (0.0006) [2023-12-27 06:01:37,711][20381] Updated weights for policy 1, policy_version 52472 (0.0005) [2023-12-27 06:01:37,771][20381] Updated weights for policy 1, policy_version 52482 (0.0006) [2023-12-27 06:01:37,858][20347] Updated weights for policy 0, policy_version 52533 (0.0006) [2023-12-27 06:01:37,915][20347] Updated weights for policy 0, policy_version 52543 (0.0007) [2023-12-27 06:01:37,968][20347] Updated weights for policy 0, policy_version 52553 (0.0008) [2023-12-27 06:01:38,396][20381] Updated weights for policy 1, policy_version 52492 (0.0007) [2023-12-27 06:01:38,449][20381] Updated weights for policy 1, policy_version 52502 (0.0011) [2023-12-27 06:01:38,505][20381] Updated weights for policy 1, policy_version 52512 (0.0010) [2023-12-27 06:01:38,644][20347] Updated weights for policy 0, policy_version 52563 (0.0007) [2023-12-27 06:01:38,698][20347] Updated weights for policy 0, policy_version 52573 (0.0007) [2023-12-27 06:01:38,759][20347] Updated weights for policy 0, policy_version 52583 (0.0007) [2023-12-27 06:01:39,183][20381] Updated weights for policy 1, policy_version 52522 (0.0010) [2023-12-27 06:01:39,247][20381] Updated weights for policy 1, policy_version 52532 (0.0007) [2023-12-27 06:01:39,309][20381] Updated weights for policy 1, policy_version 52542 (0.0010) [2023-12-27 06:01:39,375][20381] Updated weights for policy 1, policy_version 52552 (0.0010) [2023-12-27 06:01:39,410][20347] Updated weights for policy 0, policy_version 52593 (0.0007) [2023-12-27 06:01:39,470][20347] Updated weights for policy 0, policy_version 52603 (0.0005) [2023-12-27 06:01:39,529][20347] Updated weights for policy 0, policy_version 52613 (0.0005) [2023-12-27 06:01:39,591][20347] Updated weights for policy 0, policy_version 52623 (0.0006) [2023-12-27 06:01:40,053][20381] Updated weights for policy 1, policy_version 52562 (0.0010) [2023-12-27 06:01:40,117][20381] Updated weights for policy 1, policy_version 52572 (0.0011) [2023-12-27 06:01:40,177][20381] Updated weights for policy 1, policy_version 52582 (0.0011) [2023-12-27 06:01:40,291][20347] Updated weights for policy 0, policy_version 52633 (0.0007) [2023-12-27 06:01:40,349][20347] Updated weights for policy 0, policy_version 52643 (0.0007) [2023-12-27 06:01:40,414][20347] Updated weights for policy 0, policy_version 52653 (0.0007) [2023-12-27 06:01:40,851][20381] Updated weights for policy 1, policy_version 52592 (0.0006) [2023-12-27 06:01:40,898][20381] Updated weights for policy 1, policy_version 52602 (0.0010) [2023-12-27 06:01:40,946][20381] Updated weights for policy 1, policy_version 52612 (0.0010) [2023-12-27 06:01:41,003][20347] Updated weights for policy 0, policy_version 52663 (0.0006) [2023-12-27 06:01:41,061][20347] Updated weights for policy 0, policy_version 52673 (0.0008) [2023-12-27 06:01:41,126][20347] Updated weights for policy 0, policy_version 52683 (0.0007) [2023-12-27 06:01:41,619][20381] Updated weights for policy 1, policy_version 52622 (0.0011) [2023-12-27 06:01:41,687][20381] Updated weights for policy 1, policy_version 52632 (0.0006) [2023-12-27 06:01:41,746][20381] Updated weights for policy 1, policy_version 52642 (0.0008) [2023-12-27 06:01:41,829][20347] Updated weights for policy 0, policy_version 52693 (0.0009) [2023-12-27 06:01:41,879][20347] Updated weights for policy 0, policy_version 52703 (0.0007) [2023-12-27 06:01:41,925][20347] Updated weights for policy 0, policy_version 52713 (0.0006) [2023-12-27 06:01:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 26984448. Throughput: 0: 10441.7, 1: 10315.1. Samples: 26977456. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:42,417][20381] Updated weights for policy 1, policy_version 52652 (0.0008) [2023-12-27 06:01:42,476][20381] Updated weights for policy 1, policy_version 52662 (0.0011) [2023-12-27 06:01:42,535][20381] Updated weights for policy 1, policy_version 52672 (0.0011) [2023-12-27 06:01:42,660][20347] Updated weights for policy 0, policy_version 52723 (0.0007) [2023-12-27 06:01:42,714][20347] Updated weights for policy 0, policy_version 52733 (0.0010) [2023-12-27 06:01:42,767][20347] Updated weights for policy 0, policy_version 52743 (0.0010) [2023-12-27 06:01:43,178][20381] Updated weights for policy 1, policy_version 52682 (0.0009) [2023-12-27 06:01:43,238][20381] Updated weights for policy 1, policy_version 52692 (0.0005) [2023-12-27 06:01:43,295][20381] Updated weights for policy 1, policy_version 52702 (0.0008) [2023-12-27 06:01:43,342][20381] Updated weights for policy 1, policy_version 52712 (0.0010) [2023-12-27 06:01:43,451][20347] Updated weights for policy 0, policy_version 52753 (0.0010) [2023-12-27 06:01:43,501][20347] Updated weights for policy 0, policy_version 52763 (0.0007) [2023-12-27 06:01:43,548][20347] Updated weights for policy 0, policy_version 52773 (0.0007) [2023-12-27 06:01:43,594][20347] Updated weights for policy 0, policy_version 52783 (0.0007) [2023-12-27 06:01:44,037][20381] Updated weights for policy 1, policy_version 52722 (0.0010) [2023-12-27 06:01:44,092][20381] Updated weights for policy 1, policy_version 52732 (0.0011) [2023-12-27 06:01:44,148][20381] Updated weights for policy 1, policy_version 52742 (0.0011) [2023-12-27 06:01:44,216][20347] Updated weights for policy 0, policy_version 52793 (0.0007) [2023-12-27 06:01:44,265][20347] Updated weights for policy 0, policy_version 52803 (0.0008) [2023-12-27 06:01:44,322][20347] Updated weights for policy 0, policy_version 52813 (0.0007) [2023-12-27 06:01:44,772][20381] Updated weights for policy 1, policy_version 52752 (0.0009) [2023-12-27 06:01:44,828][20381] Updated weights for policy 1, policy_version 52762 (0.0011) [2023-12-27 06:01:44,888][20381] Updated weights for policy 1, policy_version 52772 (0.0011) [2023-12-27 06:01:45,050][20347] Updated weights for policy 0, policy_version 52823 (0.0008) [2023-12-27 06:01:45,097][20347] Updated weights for policy 0, policy_version 52833 (0.0007) [2023-12-27 06:01:45,152][20347] Updated weights for policy 0, policy_version 52843 (0.0007) [2023-12-27 06:01:45,572][20381] Updated weights for policy 1, policy_version 52782 (0.0010) [2023-12-27 06:01:45,623][20381] Updated weights for policy 1, policy_version 52792 (0.0010) [2023-12-27 06:01:45,681][20381] Updated weights for policy 1, policy_version 52802 (0.0010) [2023-12-27 06:01:45,759][20347] Updated weights for policy 0, policy_version 52853 (0.0009) [2023-12-27 06:01:45,818][20347] Updated weights for policy 0, policy_version 52863 (0.0007) [2023-12-27 06:01:45,875][20347] Updated weights for policy 0, policy_version 52873 (0.0008) [2023-12-27 06:01:46,331][20381] Updated weights for policy 1, policy_version 52812 (0.0010) [2023-12-27 06:01:46,390][20381] Updated weights for policy 1, policy_version 52822 (0.0007) [2023-12-27 06:01:46,450][20381] Updated weights for policy 1, policy_version 52832 (0.0009) [2023-12-27 06:01:46,465][20347] Updated weights for policy 0, policy_version 52883 (0.0009) [2023-12-27 06:01:46,521][20347] Updated weights for policy 0, policy_version 52893 (0.0006) [2023-12-27 06:01:46,581][20347] Updated weights for policy 0, policy_version 52903 (0.0006) [2023-12-27 06:01:47,115][20381] Updated weights for policy 1, policy_version 52842 (0.0010) [2023-12-27 06:01:47,169][20381] Updated weights for policy 1, policy_version 52852 (0.0010) [2023-12-27 06:01:47,174][20347] Updated weights for policy 0, policy_version 52913 (0.0005) [2023-12-27 06:01:47,215][20381] Updated weights for policy 1, policy_version 52862 (0.0010) [2023-12-27 06:01:47,224][20347] Updated weights for policy 0, policy_version 52923 (0.0005) [2023-12-27 06:01:47,259][20381] Updated weights for policy 1, policy_version 52872 (0.0010) [2023-12-27 06:01:47,273][20347] Updated weights for policy 0, policy_version 52933 (0.0006) [2023-12-27 06:01:47,324][20347] Updated weights for policy 0, policy_version 52943 (0.0007) [2023-12-27 06:01:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 27099136. Throughput: 0: 10498.2, 1: 10289.7. Samples: 27102492. Policy #0 lag: (min: 2.0, avg: 25.0, max: 34.0) [2023-12-27 06:01:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:47,953][20381] Updated weights for policy 1, policy_version 52882 (0.0011) [2023-12-27 06:01:47,966][20347] Updated weights for policy 0, policy_version 52953 (0.0006) [2023-12-27 06:01:48,001][20381] Updated weights for policy 1, policy_version 52892 (0.0010) [2023-12-27 06:01:48,011][20347] Updated weights for policy 0, policy_version 52963 (0.0005) [2023-12-27 06:01:48,050][20381] Updated weights for policy 1, policy_version 52902 (0.0010) [2023-12-27 06:01:48,066][20347] Updated weights for policy 0, policy_version 52973 (0.0006) [2023-12-27 06:01:48,712][20347] Updated weights for policy 0, policy_version 52983 (0.0009) [2023-12-27 06:01:48,762][20347] Updated weights for policy 0, policy_version 52993 (0.0010) [2023-12-27 06:01:48,779][20381] Updated weights for policy 1, policy_version 52912 (0.0011) [2023-12-27 06:01:48,811][20347] Updated weights for policy 0, policy_version 53003 (0.0010) [2023-12-27 06:01:48,829][20381] Updated weights for policy 1, policy_version 52922 (0.0011) [2023-12-27 06:01:48,883][20381] Updated weights for policy 1, policy_version 52932 (0.0010) [2023-12-27 06:01:49,519][20347] Updated weights for policy 0, policy_version 53013 (0.0008) [2023-12-27 06:01:49,590][20347] Updated weights for policy 0, policy_version 53023 (0.0006) [2023-12-27 06:01:49,601][20381] Updated weights for policy 1, policy_version 52942 (0.0010) [2023-12-27 06:01:49,650][20381] Updated weights for policy 1, policy_version 52952 (0.0006) [2023-12-27 06:01:49,655][20347] Updated weights for policy 0, policy_version 53033 (0.0010) [2023-12-27 06:01:49,702][20381] Updated weights for policy 1, policy_version 52962 (0.0008) [2023-12-27 06:01:50,235][20347] Updated weights for policy 0, policy_version 53043 (0.0008) [2023-12-27 06:01:50,305][20347] Updated weights for policy 0, policy_version 53053 (0.0005) [2023-12-27 06:01:50,366][20347] Updated weights for policy 0, policy_version 53063 (0.0007) [2023-12-27 06:01:50,451][20381] Updated weights for policy 1, policy_version 52972 (0.0007) [2023-12-27 06:01:50,503][20381] Updated weights for policy 1, policy_version 52982 (0.0010) [2023-12-27 06:01:50,551][20381] Updated weights for policy 1, policy_version 52992 (0.0010) [2023-12-27 06:01:51,081][20347] Updated weights for policy 0, policy_version 53073 (0.0008) [2023-12-27 06:01:51,168][20347] Updated weights for policy 0, policy_version 53083 (0.0007) [2023-12-27 06:01:51,185][20381] Updated weights for policy 1, policy_version 53002 (0.0009) [2023-12-27 06:01:51,219][20347] Updated weights for policy 0, policy_version 53093 (0.0005) [2023-12-27 06:01:51,248][20381] Updated weights for policy 1, policy_version 53012 (0.0010) [2023-12-27 06:01:51,277][20347] Updated weights for policy 0, policy_version 53103 (0.0007) [2023-12-27 06:01:51,302][20381] Updated weights for policy 1, policy_version 53022 (0.0008) [2023-12-27 06:01:51,364][20381] Updated weights for policy 1, policy_version 53032 (0.0011) [2023-12-27 06:01:51,948][20347] Updated weights for policy 0, policy_version 53113 (0.0007) [2023-12-27 06:01:52,007][20347] Updated weights for policy 0, policy_version 53123 (0.0008) [2023-12-27 06:01:52,060][20347] Updated weights for policy 0, policy_version 53133 (0.0007) [2023-12-27 06:01:52,087][20381] Updated weights for policy 1, policy_version 53042 (0.0007) [2023-12-27 06:01:52,140][20381] Updated weights for policy 1, policy_version 53052 (0.0008) [2023-12-27 06:01:52,200][20381] Updated weights for policy 1, policy_version 53062 (0.0008) [2023-12-27 06:01:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 27197440. Throughput: 0: 10479.9, 1: 10287.7. Samples: 27164016. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:01:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000053064_13590528.pth... [2023-12-27 06:01:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000053136_13606912.pth... [2023-12-27 06:01:52,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000051848_13279232.pth [2023-12-27 06:01:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000051888_13287424.pth [2023-12-27 06:01:52,768][20347] Updated weights for policy 0, policy_version 53143 (0.0007) [2023-12-27 06:01:52,827][20347] Updated weights for policy 0, policy_version 53153 (0.0010) [2023-12-27 06:01:52,878][20347] Updated weights for policy 0, policy_version 53163 (0.0010) [2023-12-27 06:01:52,939][20381] Updated weights for policy 1, policy_version 53072 (0.0008) [2023-12-27 06:01:52,990][20381] Updated weights for policy 1, policy_version 53082 (0.0008) [2023-12-27 06:01:53,043][20381] Updated weights for policy 1, policy_version 53092 (0.0007) [2023-12-27 06:01:53,508][20347] Updated weights for policy 0, policy_version 53173 (0.0008) [2023-12-27 06:01:53,561][20347] Updated weights for policy 0, policy_version 53183 (0.0005) [2023-12-27 06:01:53,612][20347] Updated weights for policy 0, policy_version 53193 (0.0005) [2023-12-27 06:01:53,725][20381] Updated weights for policy 1, policy_version 53102 (0.0007) [2023-12-27 06:01:53,778][20381] Updated weights for policy 1, policy_version 53112 (0.0008) [2023-12-27 06:01:53,840][20381] Updated weights for policy 1, policy_version 53122 (0.0006) [2023-12-27 06:01:54,241][20347] Updated weights for policy 0, policy_version 53203 (0.0007) [2023-12-27 06:01:54,297][20347] Updated weights for policy 0, policy_version 53213 (0.0010) [2023-12-27 06:01:54,350][20347] Updated weights for policy 0, policy_version 53223 (0.0010) [2023-12-27 06:01:54,510][20381] Updated weights for policy 1, policy_version 53132 (0.0008) [2023-12-27 06:01:54,569][20381] Updated weights for policy 1, policy_version 53142 (0.0011) [2023-12-27 06:01:54,628][20381] Updated weights for policy 1, policy_version 53152 (0.0010) [2023-12-27 06:01:54,930][20347] Updated weights for policy 0, policy_version 53233 (0.0007) [2023-12-27 06:01:54,989][20347] Updated weights for policy 0, policy_version 53243 (0.0006) [2023-12-27 06:01:55,050][20347] Updated weights for policy 0, policy_version 53253 (0.0006) [2023-12-27 06:01:55,115][20347] Updated weights for policy 0, policy_version 53263 (0.0005) [2023-12-27 06:01:55,317][20381] Updated weights for policy 1, policy_version 53162 (0.0011) [2023-12-27 06:01:55,368][20381] Updated weights for policy 1, policy_version 53172 (0.0010) [2023-12-27 06:01:55,421][20381] Updated weights for policy 1, policy_version 53182 (0.0011) [2023-12-27 06:01:55,469][20381] Updated weights for policy 1, policy_version 53192 (0.0010) [2023-12-27 06:01:55,682][20347] Updated weights for policy 0, policy_version 53273 (0.0010) [2023-12-27 06:01:55,730][20347] Updated weights for policy 0, policy_version 53283 (0.0010) [2023-12-27 06:01:55,774][20347] Updated weights for policy 0, policy_version 53293 (0.0010) [2023-12-27 06:01:56,158][20381] Updated weights for policy 1, policy_version 53202 (0.0011) [2023-12-27 06:01:56,207][20381] Updated weights for policy 1, policy_version 53212 (0.0010) [2023-12-27 06:01:56,253][20381] Updated weights for policy 1, policy_version 53222 (0.0010) [2023-12-27 06:01:56,368][20347] Updated weights for policy 0, policy_version 53303 (0.0007) [2023-12-27 06:01:56,419][20347] Updated weights for policy 0, policy_version 53313 (0.0005) [2023-12-27 06:01:56,465][20347] Updated weights for policy 0, policy_version 53323 (0.0005) [2023-12-27 06:01:56,858][20381] Updated weights for policy 1, policy_version 53232 (0.0006) [2023-12-27 06:01:56,915][20381] Updated weights for policy 1, policy_version 53242 (0.0005) [2023-12-27 06:01:56,971][20381] Updated weights for policy 1, policy_version 53252 (0.0006) [2023-12-27 06:01:56,984][20347] Updated weights for policy 0, policy_version 53333 (0.0005) [2023-12-27 06:01:57,031][20347] Updated weights for policy 0, policy_version 53343 (0.0005) [2023-12-27 06:01:57,085][20347] Updated weights for policy 0, policy_version 53353 (0.0005) [2023-12-27 06:01:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 27303936. Throughput: 0: 10550.4, 1: 10223.8. Samples: 27288740. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:01:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:01:57,565][20381] Updated weights for policy 1, policy_version 53262 (0.0010) [2023-12-27 06:01:57,620][20381] Updated weights for policy 1, policy_version 53272 (0.0009) [2023-12-27 06:01:57,678][20381] Updated weights for policy 1, policy_version 53282 (0.0011) [2023-12-27 06:01:57,690][20347] Updated weights for policy 0, policy_version 53363 (0.0006) [2023-12-27 06:01:57,743][20347] Updated weights for policy 0, policy_version 53373 (0.0007) [2023-12-27 06:01:57,796][20347] Updated weights for policy 0, policy_version 53383 (0.0006) [2023-12-27 06:01:58,380][20381] Updated weights for policy 1, policy_version 53292 (0.0009) [2023-12-27 06:01:58,411][20347] Updated weights for policy 0, policy_version 53393 (0.0006) [2023-12-27 06:01:58,441][20381] Updated weights for policy 1, policy_version 53302 (0.0007) [2023-12-27 06:01:58,474][20347] Updated weights for policy 0, policy_version 53403 (0.0008) [2023-12-27 06:01:58,498][20381] Updated weights for policy 1, policy_version 53312 (0.0007) [2023-12-27 06:01:58,535][20347] Updated weights for policy 0, policy_version 53413 (0.0007) [2023-12-27 06:01:58,591][20347] Updated weights for policy 0, policy_version 53423 (0.0007) [2023-12-27 06:01:59,306][20381] Updated weights for policy 1, policy_version 53322 (0.0008) [2023-12-27 06:01:59,344][20347] Updated weights for policy 0, policy_version 53433 (0.0007) [2023-12-27 06:01:59,372][20381] Updated weights for policy 1, policy_version 53332 (0.0008) [2023-12-27 06:01:59,410][20347] Updated weights for policy 0, policy_version 53443 (0.0006) [2023-12-27 06:01:59,427][20381] Updated weights for policy 1, policy_version 53342 (0.0006) [2023-12-27 06:01:59,471][20347] Updated weights for policy 0, policy_version 53453 (0.0005) [2023-12-27 06:01:59,477][20381] Updated weights for policy 1, policy_version 53352 (0.0006) [2023-12-27 06:02:00,067][20347] Updated weights for policy 0, policy_version 53463 (0.0006) [2023-12-27 06:02:00,127][20347] Updated weights for policy 0, policy_version 53473 (0.0008) [2023-12-27 06:02:00,137][20381] Updated weights for policy 1, policy_version 53362 (0.0010) [2023-12-27 06:02:00,180][20347] Updated weights for policy 0, policy_version 53483 (0.0007) [2023-12-27 06:02:00,199][20381] Updated weights for policy 1, policy_version 53372 (0.0010) [2023-12-27 06:02:00,265][20381] Updated weights for policy 1, policy_version 53382 (0.0011) [2023-12-27 06:02:00,872][20347] Updated weights for policy 0, policy_version 53493 (0.0007) [2023-12-27 06:02:00,923][20347] Updated weights for policy 0, policy_version 53503 (0.0006) [2023-12-27 06:02:00,935][20381] Updated weights for policy 1, policy_version 53392 (0.0010) [2023-12-27 06:02:00,974][20347] Updated weights for policy 0, policy_version 53513 (0.0006) [2023-12-27 06:02:00,983][20381] Updated weights for policy 1, policy_version 53402 (0.0010) [2023-12-27 06:02:01,040][20381] Updated weights for policy 1, policy_version 53412 (0.0010) [2023-12-27 06:02:01,683][20347] Updated weights for policy 0, policy_version 53523 (0.0006) [2023-12-27 06:02:01,745][20347] Updated weights for policy 0, policy_version 53533 (0.0007) [2023-12-27 06:02:01,787][20381] Updated weights for policy 1, policy_version 53422 (0.0009) [2023-12-27 06:02:01,807][20347] Updated weights for policy 0, policy_version 53543 (0.0008) [2023-12-27 06:02:01,841][20381] Updated weights for policy 1, policy_version 53432 (0.0008) [2023-12-27 06:02:01,889][20381] Updated weights for policy 1, policy_version 53442 (0.0007) [2023-12-27 06:02:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 27402240. Throughput: 0: 10621.0, 1: 10256.9. Samples: 27415296. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:02:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:02,396][20347] Updated weights for policy 0, policy_version 53553 (0.0007) [2023-12-27 06:02:02,456][20347] Updated weights for policy 0, policy_version 53563 (0.0008) [2023-12-27 06:02:02,469][20381] Updated weights for policy 1, policy_version 53452 (0.0007) [2023-12-27 06:02:02,509][20347] Updated weights for policy 0, policy_version 53573 (0.0006) [2023-12-27 06:02:02,534][20381] Updated weights for policy 1, policy_version 53462 (0.0006) [2023-12-27 06:02:02,568][20347] Updated weights for policy 0, policy_version 53583 (0.0006) [2023-12-27 06:02:02,586][20381] Updated weights for policy 1, policy_version 53472 (0.0006) [2023-12-27 06:02:03,201][20347] Updated weights for policy 0, policy_version 53593 (0.0005) [2023-12-27 06:02:03,249][20347] Updated weights for policy 0, policy_version 53603 (0.0005) [2023-12-27 06:02:03,250][20381] Updated weights for policy 1, policy_version 53482 (0.0007) [2023-12-27 06:02:03,292][20347] Updated weights for policy 0, policy_version 53613 (0.0005) [2023-12-27 06:02:03,304][20381] Updated weights for policy 1, policy_version 53492 (0.0010) [2023-12-27 06:02:03,359][20381] Updated weights for policy 1, policy_version 53502 (0.0010) [2023-12-27 06:02:03,406][20381] Updated weights for policy 1, policy_version 53512 (0.0010) [2023-12-27 06:02:03,876][20347] Updated weights for policy 0, policy_version 53623 (0.0007) [2023-12-27 06:02:03,930][20347] Updated weights for policy 0, policy_version 53633 (0.0007) [2023-12-27 06:02:03,992][20347] Updated weights for policy 0, policy_version 53643 (0.0008) [2023-12-27 06:02:04,055][20381] Updated weights for policy 1, policy_version 53522 (0.0008) [2023-12-27 06:02:04,103][20381] Updated weights for policy 1, policy_version 53532 (0.0010) [2023-12-27 06:02:04,166][20381] Updated weights for policy 1, policy_version 53542 (0.0011) [2023-12-27 06:02:04,676][20347] Updated weights for policy 0, policy_version 53653 (0.0007) [2023-12-27 06:02:04,730][20347] Updated weights for policy 0, policy_version 53663 (0.0006) [2023-12-27 06:02:04,790][20347] Updated weights for policy 0, policy_version 53673 (0.0005) [2023-12-27 06:02:04,840][20381] Updated weights for policy 1, policy_version 53552 (0.0010) [2023-12-27 06:02:04,894][20381] Updated weights for policy 1, policy_version 53562 (0.0006) [2023-12-27 06:02:04,950][20381] Updated weights for policy 1, policy_version 53572 (0.0006) [2023-12-27 06:02:05,484][20381] Updated weights for policy 1, policy_version 53582 (0.0008) [2023-12-27 06:02:05,503][20347] Updated weights for policy 0, policy_version 53683 (0.0005) [2023-12-27 06:02:05,543][20381] Updated weights for policy 1, policy_version 53592 (0.0009) [2023-12-27 06:02:05,554][20347] Updated weights for policy 0, policy_version 53693 (0.0007) [2023-12-27 06:02:05,594][20381] Updated weights for policy 1, policy_version 53602 (0.0007) [2023-12-27 06:02:05,610][20347] Updated weights for policy 0, policy_version 53703 (0.0005) [2023-12-27 06:02:06,259][20381] Updated weights for policy 1, policy_version 53612 (0.0009) [2023-12-27 06:02:06,269][20347] Updated weights for policy 0, policy_version 53713 (0.0006) [2023-12-27 06:02:06,319][20381] Updated weights for policy 1, policy_version 53622 (0.0010) [2023-12-27 06:02:06,328][20347] Updated weights for policy 0, policy_version 53723 (0.0005) [2023-12-27 06:02:06,379][20381] Updated weights for policy 1, policy_version 53632 (0.0011) [2023-12-27 06:02:06,381][20347] Updated weights for policy 0, policy_version 53733 (0.0005) [2023-12-27 06:02:06,441][20347] Updated weights for policy 0, policy_version 53743 (0.0006) [2023-12-27 06:02:06,998][20381] Updated weights for policy 1, policy_version 53642 (0.0010) [2023-12-27 06:02:07,062][20381] Updated weights for policy 1, policy_version 53652 (0.0005) [2023-12-27 06:02:07,120][20381] Updated weights for policy 1, policy_version 53662 (0.0006) [2023-12-27 06:02:07,180][20381] Updated weights for policy 1, policy_version 53672 (0.0010) [2023-12-27 06:02:07,194][20347] Updated weights for policy 0, policy_version 53753 (0.0007) [2023-12-27 06:02:07,255][20347] Updated weights for policy 0, policy_version 53763 (0.0008) [2023-12-27 06:02:07,317][20347] Updated weights for policy 0, policy_version 53773 (0.0008) [2023-12-27 06:02:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 27516928. Throughput: 0: 10640.4, 1: 10319.9. Samples: 27480780. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:02:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000053776_13770752.pth... [2023-12-27 06:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000053672_13746176.pth... [2023-12-27 06:02:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000052528_13451264.pth [2023-12-27 06:02:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000052456_13434880.pth [2023-12-27 06:02:07,817][20381] Updated weights for policy 1, policy_version 53682 (0.0010) [2023-12-27 06:02:07,872][20381] Updated weights for policy 1, policy_version 53692 (0.0010) [2023-12-27 06:02:07,918][20381] Updated weights for policy 1, policy_version 53702 (0.0010) [2023-12-27 06:02:08,012][20347] Updated weights for policy 0, policy_version 53783 (0.0008) [2023-12-27 06:02:08,061][20347] Updated weights for policy 0, policy_version 53793 (0.0009) [2023-12-27 06:02:08,110][20347] Updated weights for policy 0, policy_version 53803 (0.0009) [2023-12-27 06:02:08,571][20381] Updated weights for policy 1, policy_version 53712 (0.0011) [2023-12-27 06:02:08,630][20381] Updated weights for policy 1, policy_version 53722 (0.0006) [2023-12-27 06:02:08,685][20381] Updated weights for policy 1, policy_version 53732 (0.0009) [2023-12-27 06:02:08,799][20347] Updated weights for policy 0, policy_version 53813 (0.0007) [2023-12-27 06:02:08,863][20347] Updated weights for policy 0, policy_version 53823 (0.0006) [2023-12-27 06:02:08,925][20347] Updated weights for policy 0, policy_version 53833 (0.0007) [2023-12-27 06:02:09,371][20381] Updated weights for policy 1, policy_version 53742 (0.0010) [2023-12-27 06:02:09,432][20381] Updated weights for policy 1, policy_version 53752 (0.0008) [2023-12-27 06:02:09,497][20381] Updated weights for policy 1, policy_version 53762 (0.0007) [2023-12-27 06:02:09,630][20347] Updated weights for policy 0, policy_version 53843 (0.0008) [2023-12-27 06:02:09,692][20347] Updated weights for policy 0, policy_version 53853 (0.0007) [2023-12-27 06:02:09,758][20347] Updated weights for policy 0, policy_version 53863 (0.0007) [2023-12-27 06:02:10,230][20381] Updated weights for policy 1, policy_version 53772 (0.0011) [2023-12-27 06:02:10,294][20381] Updated weights for policy 1, policy_version 53782 (0.0011) [2023-12-27 06:02:10,359][20381] Updated weights for policy 1, policy_version 53792 (0.0010) [2023-12-27 06:02:10,518][20347] Updated weights for policy 0, policy_version 53873 (0.0006) [2023-12-27 06:02:10,575][20347] Updated weights for policy 0, policy_version 53883 (0.0007) [2023-12-27 06:02:10,627][20347] Updated weights for policy 0, policy_version 53893 (0.0007) [2023-12-27 06:02:10,679][20347] Updated weights for policy 0, policy_version 53903 (0.0007) [2023-12-27 06:02:11,051][20381] Updated weights for policy 1, policy_version 53802 (0.0010) [2023-12-27 06:02:11,103][20381] Updated weights for policy 1, policy_version 53812 (0.0006) [2023-12-27 06:02:11,170][20381] Updated weights for policy 1, policy_version 53822 (0.0007) [2023-12-27 06:02:11,233][20381] Updated weights for policy 1, policy_version 53832 (0.0008) [2023-12-27 06:02:11,398][20347] Updated weights for policy 0, policy_version 53913 (0.0009) [2023-12-27 06:02:11,446][20347] Updated weights for policy 0, policy_version 53923 (0.0006) [2023-12-27 06:02:11,492][20347] Updated weights for policy 0, policy_version 53933 (0.0007) [2023-12-27 06:02:11,977][20381] Updated weights for policy 1, policy_version 53842 (0.0008) [2023-12-27 06:02:12,035][20381] Updated weights for policy 1, policy_version 53852 (0.0008) [2023-12-27 06:02:12,095][20381] Updated weights for policy 1, policy_version 53862 (0.0008) [2023-12-27 06:02:12,203][20347] Updated weights for policy 0, policy_version 53943 (0.0008) [2023-12-27 06:02:12,263][20347] Updated weights for policy 0, policy_version 53953 (0.0007) [2023-12-27 06:02:12,323][20347] Updated weights for policy 0, policy_version 53963 (0.0007) [2023-12-27 06:02:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 27615232. Throughput: 0: 10560.4, 1: 10374.7. Samples: 27601488. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:02:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:12,829][20381] Updated weights for policy 1, policy_version 53872 (0.0007) [2023-12-27 06:02:12,895][20381] Updated weights for policy 1, policy_version 53882 (0.0006) [2023-12-27 06:02:12,957][20381] Updated weights for policy 1, policy_version 53892 (0.0008) [2023-12-27 06:02:12,979][20347] Updated weights for policy 0, policy_version 53973 (0.0007) [2023-12-27 06:02:13,044][20347] Updated weights for policy 0, policy_version 53983 (0.0008) [2023-12-27 06:02:13,094][20347] Updated weights for policy 0, policy_version 53993 (0.0007) [2023-12-27 06:02:13,633][20381] Updated weights for policy 1, policy_version 53902 (0.0008) [2023-12-27 06:02:13,697][20381] Updated weights for policy 1, policy_version 53912 (0.0008) [2023-12-27 06:02:13,754][20381] Updated weights for policy 1, policy_version 53922 (0.0008) [2023-12-27 06:02:13,764][20347] Updated weights for policy 0, policy_version 54003 (0.0007) [2023-12-27 06:02:13,821][20347] Updated weights for policy 0, policy_version 54013 (0.0006) [2023-12-27 06:02:13,877][20347] Updated weights for policy 0, policy_version 54023 (0.0009) [2023-12-27 06:02:14,356][20381] Updated weights for policy 1, policy_version 53932 (0.0006) [2023-12-27 06:02:14,415][20381] Updated weights for policy 1, policy_version 53942 (0.0005) [2023-12-27 06:02:14,476][20381] Updated weights for policy 1, policy_version 53952 (0.0006) [2023-12-27 06:02:14,568][20347] Updated weights for policy 0, policy_version 54033 (0.0007) [2023-12-27 06:02:14,622][20347] Updated weights for policy 0, policy_version 54043 (0.0008) [2023-12-27 06:02:14,679][20347] Updated weights for policy 0, policy_version 54053 (0.0007) [2023-12-27 06:02:14,731][20347] Updated weights for policy 0, policy_version 54063 (0.0008) [2023-12-27 06:02:15,182][20381] Updated weights for policy 1, policy_version 53962 (0.0008) [2023-12-27 06:02:15,236][20381] Updated weights for policy 1, policy_version 53972 (0.0007) [2023-12-27 06:02:15,297][20381] Updated weights for policy 1, policy_version 53982 (0.0008) [2023-12-27 06:02:15,348][20381] Updated weights for policy 1, policy_version 53992 (0.0008) [2023-12-27 06:02:15,440][20347] Updated weights for policy 0, policy_version 54073 (0.0007) [2023-12-27 06:02:15,502][20347] Updated weights for policy 0, policy_version 54083 (0.0007) [2023-12-27 06:02:15,562][20347] Updated weights for policy 0, policy_version 54093 (0.0008) [2023-12-27 06:02:16,023][20381] Updated weights for policy 1, policy_version 54002 (0.0007) [2023-12-27 06:02:16,069][20381] Updated weights for policy 1, policy_version 54012 (0.0010) [2023-12-27 06:02:16,122][20381] Updated weights for policy 1, policy_version 54022 (0.0008) [2023-12-27 06:02:16,197][20347] Updated weights for policy 0, policy_version 54103 (0.0007) [2023-12-27 06:02:16,257][20347] Updated weights for policy 0, policy_version 54113 (0.0007) [2023-12-27 06:02:16,309][20347] Updated weights for policy 0, policy_version 54123 (0.0007) [2023-12-27 06:02:16,770][20381] Updated weights for policy 1, policy_version 54032 (0.0008) [2023-12-27 06:02:16,830][20381] Updated weights for policy 1, policy_version 54042 (0.0007) [2023-12-27 06:02:16,886][20381] Updated weights for policy 1, policy_version 54052 (0.0006) [2023-12-27 06:02:17,011][20347] Updated weights for policy 0, policy_version 54133 (0.0008) [2023-12-27 06:02:17,061][20347] Updated weights for policy 0, policy_version 54143 (0.0008) [2023-12-27 06:02:17,112][20347] Updated weights for policy 0, policy_version 54153 (0.0008) [2023-12-27 06:02:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 27713536. Throughput: 0: 10506.3, 1: 10283.8. Samples: 27723392. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 06:02:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:17,523][20381] Updated weights for policy 1, policy_version 54062 (0.0007) [2023-12-27 06:02:17,569][20381] Updated weights for policy 1, policy_version 54072 (0.0006) [2023-12-27 06:02:17,619][20381] Updated weights for policy 1, policy_version 54082 (0.0005) [2023-12-27 06:02:17,847][20347] Updated weights for policy 0, policy_version 54163 (0.0009) [2023-12-27 06:02:17,897][20347] Updated weights for policy 0, policy_version 54173 (0.0009) [2023-12-27 06:02:17,959][20347] Updated weights for policy 0, policy_version 54183 (0.0008) [2023-12-27 06:02:18,241][20381] Updated weights for policy 1, policy_version 54092 (0.0006) [2023-12-27 06:02:18,287][20381] Updated weights for policy 1, policy_version 54102 (0.0008) [2023-12-27 06:02:18,337][20381] Updated weights for policy 1, policy_version 54112 (0.0007) [2023-12-27 06:02:18,630][20347] Updated weights for policy 0, policy_version 54193 (0.0007) [2023-12-27 06:02:18,687][20347] Updated weights for policy 0, policy_version 54203 (0.0006) [2023-12-27 06:02:18,749][20347] Updated weights for policy 0, policy_version 54213 (0.0008) [2023-12-27 06:02:18,815][20347] Updated weights for policy 0, policy_version 54223 (0.0008) [2023-12-27 06:02:18,962][20381] Updated weights for policy 1, policy_version 54122 (0.0007) [2023-12-27 06:02:19,020][20381] Updated weights for policy 1, policy_version 54132 (0.0008) [2023-12-27 06:02:19,070][20381] Updated weights for policy 1, policy_version 54142 (0.0005) [2023-12-27 06:02:19,121][20381] Updated weights for policy 1, policy_version 54152 (0.0007) [2023-12-27 06:02:19,491][20347] Updated weights for policy 0, policy_version 54233 (0.0008) [2023-12-27 06:02:19,547][20347] Updated weights for policy 0, policy_version 54243 (0.0005) [2023-12-27 06:02:19,600][20347] Updated weights for policy 0, policy_version 54253 (0.0006) [2023-12-27 06:02:19,776][20381] Updated weights for policy 1, policy_version 54162 (0.0008) [2023-12-27 06:02:19,832][20381] Updated weights for policy 1, policy_version 54172 (0.0008) [2023-12-27 06:02:19,887][20381] Updated weights for policy 1, policy_version 54182 (0.0008) [2023-12-27 06:02:20,219][20347] Updated weights for policy 0, policy_version 54263 (0.0006) [2023-12-27 06:02:20,275][20347] Updated weights for policy 0, policy_version 54273 (0.0007) [2023-12-27 06:02:20,337][20347] Updated weights for policy 0, policy_version 54283 (0.0007) [2023-12-27 06:02:20,622][20381] Updated weights for policy 1, policy_version 54192 (0.0007) [2023-12-27 06:02:20,687][20381] Updated weights for policy 1, policy_version 54202 (0.0006) [2023-12-27 06:02:20,747][20381] Updated weights for policy 1, policy_version 54212 (0.0006) [2023-12-27 06:02:20,982][20347] Updated weights for policy 0, policy_version 54293 (0.0007) [2023-12-27 06:02:21,050][20347] Updated weights for policy 0, policy_version 54303 (0.0008) [2023-12-27 06:02:21,102][20347] Updated weights for policy 0, policy_version 54313 (0.0008) [2023-12-27 06:02:21,359][20381] Updated weights for policy 1, policy_version 54222 (0.0009) [2023-12-27 06:02:21,418][20381] Updated weights for policy 1, policy_version 54232 (0.0009) [2023-12-27 06:02:21,472][20381] Updated weights for policy 1, policy_version 54242 (0.0009) [2023-12-27 06:02:21,878][20347] Updated weights for policy 0, policy_version 54323 (0.0008) [2023-12-27 06:02:21,931][20347] Updated weights for policy 0, policy_version 54333 (0.0008) [2023-12-27 06:02:21,984][20347] Updated weights for policy 0, policy_version 54343 (0.0006) [2023-12-27 06:02:22,165][20381] Updated weights for policy 1, policy_version 54252 (0.0010) [2023-12-27 06:02:22,229][20381] Updated weights for policy 1, policy_version 54262 (0.0010) [2023-12-27 06:02:22,293][20381] Updated weights for policy 1, policy_version 54272 (0.0011) [2023-12-27 06:02:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 27820032. Throughput: 0: 10459.4, 1: 10301.8. Samples: 27785976. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000054280_13901824.pth... [2023-12-27 06:02:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000054352_13918208.pth... [2023-12-27 06:02:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000053064_13590528.pth [2023-12-27 06:02:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000053136_13606912.pth [2023-12-27 06:02:22,697][20347] Updated weights for policy 0, policy_version 54353 (0.0007) [2023-12-27 06:02:22,756][20347] Updated weights for policy 0, policy_version 54363 (0.0008) [2023-12-27 06:02:22,814][20347] Updated weights for policy 0, policy_version 54373 (0.0007) [2023-12-27 06:02:22,873][20347] Updated weights for policy 0, policy_version 54383 (0.0008) [2023-12-27 06:02:22,994][20381] Updated weights for policy 1, policy_version 54282 (0.0012) [2023-12-27 06:02:23,051][20381] Updated weights for policy 1, policy_version 54292 (0.0009) [2023-12-27 06:02:23,102][20381] Updated weights for policy 1, policy_version 54302 (0.0009) [2023-12-27 06:02:23,157][20381] Updated weights for policy 1, policy_version 54312 (0.0008) [2023-12-27 06:02:23,577][20347] Updated weights for policy 0, policy_version 54393 (0.0007) [2023-12-27 06:02:23,637][20347] Updated weights for policy 0, policy_version 54403 (0.0008) [2023-12-27 06:02:23,694][20347] Updated weights for policy 0, policy_version 54413 (0.0007) [2023-12-27 06:02:23,859][20381] Updated weights for policy 1, policy_version 54322 (0.0008) [2023-12-27 06:02:23,913][20381] Updated weights for policy 1, policy_version 54332 (0.0008) [2023-12-27 06:02:23,980][20381] Updated weights for policy 1, policy_version 54342 (0.0009) [2023-12-27 06:02:24,324][20347] Updated weights for policy 0, policy_version 54423 (0.0007) [2023-12-27 06:02:24,381][20347] Updated weights for policy 0, policy_version 54433 (0.0009) [2023-12-27 06:02:24,440][20347] Updated weights for policy 0, policy_version 54443 (0.0010) [2023-12-27 06:02:24,660][20381] Updated weights for policy 1, policy_version 54352 (0.0011) [2023-12-27 06:02:24,726][20381] Updated weights for policy 1, policy_version 54362 (0.0011) [2023-12-27 06:02:24,788][20381] Updated weights for policy 1, policy_version 54372 (0.0011) [2023-12-27 06:02:25,053][20347] Updated weights for policy 0, policy_version 54453 (0.0010) [2023-12-27 06:02:25,106][20347] Updated weights for policy 0, policy_version 54463 (0.0009) [2023-12-27 06:02:25,151][20347] Updated weights for policy 0, policy_version 54473 (0.0008) [2023-12-27 06:02:25,428][20381] Updated weights for policy 1, policy_version 54382 (0.0008) [2023-12-27 06:02:25,490][20381] Updated weights for policy 1, policy_version 54392 (0.0005) [2023-12-27 06:02:25,542][20381] Updated weights for policy 1, policy_version 54402 (0.0005) [2023-12-27 06:02:25,886][20347] Updated weights for policy 0, policy_version 54483 (0.0005) [2023-12-27 06:02:25,947][20347] Updated weights for policy 0, policy_version 54493 (0.0006) [2023-12-27 06:02:26,003][20347] Updated weights for policy 0, policy_version 54503 (0.0007) [2023-12-27 06:02:26,180][20381] Updated weights for policy 1, policy_version 54412 (0.0007) [2023-12-27 06:02:26,242][20381] Updated weights for policy 1, policy_version 54422 (0.0005) [2023-12-27 06:02:26,296][20381] Updated weights for policy 1, policy_version 54432 (0.0005) [2023-12-27 06:02:26,576][20347] Updated weights for policy 0, policy_version 54513 (0.0007) [2023-12-27 06:02:26,624][20347] Updated weights for policy 0, policy_version 54523 (0.0010) [2023-12-27 06:02:26,671][20347] Updated weights for policy 0, policy_version 54533 (0.0010) [2023-12-27 06:02:26,730][20347] Updated weights for policy 0, policy_version 54543 (0.0009) [2023-12-27 06:02:26,892][20381] Updated weights for policy 1, policy_version 54442 (0.0009) [2023-12-27 06:02:26,944][20381] Updated weights for policy 1, policy_version 54452 (0.0008) [2023-12-27 06:02:26,995][20381] Updated weights for policy 1, policy_version 54462 (0.0009) [2023-12-27 06:02:27,045][20381] Updated weights for policy 1, policy_version 54472 (0.0005) [2023-12-27 06:02:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 27918336. Throughput: 0: 10389.2, 1: 10295.9. Samples: 27908284. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:27,381][20347] Updated weights for policy 0, policy_version 54553 (0.0006) [2023-12-27 06:02:27,434][20347] Updated weights for policy 0, policy_version 54563 (0.0005) [2023-12-27 06:02:27,482][20347] Updated weights for policy 0, policy_version 54573 (0.0010) [2023-12-27 06:02:27,612][20381] Updated weights for policy 1, policy_version 54482 (0.0010) [2023-12-27 06:02:27,665][20381] Updated weights for policy 1, policy_version 54492 (0.0010) [2023-12-27 06:02:27,715][20381] Updated weights for policy 1, policy_version 54502 (0.0010) [2023-12-27 06:02:28,096][20347] Updated weights for policy 0, policy_version 54583 (0.0010) [2023-12-27 06:02:28,151][20347] Updated weights for policy 0, policy_version 54593 (0.0010) [2023-12-27 06:02:28,213][20347] Updated weights for policy 0, policy_version 54603 (0.0006) [2023-12-27 06:02:28,335][20381] Updated weights for policy 1, policy_version 54512 (0.0010) [2023-12-27 06:02:28,399][20381] Updated weights for policy 1, policy_version 54522 (0.0009) [2023-12-27 06:02:28,462][20381] Updated weights for policy 1, policy_version 54532 (0.0006) [2023-12-27 06:02:28,877][20347] Updated weights for policy 0, policy_version 54613 (0.0008) [2023-12-27 06:02:28,929][20347] Updated weights for policy 0, policy_version 54624 (0.0009) [2023-12-27 06:02:28,977][20347] Updated weights for policy 0, policy_version 54634 (0.0006) [2023-12-27 06:02:29,072][20381] Updated weights for policy 1, policy_version 54542 (0.0006) [2023-12-27 06:02:29,119][20381] Updated weights for policy 1, policy_version 54552 (0.0005) [2023-12-27 06:02:29,179][20381] Updated weights for policy 1, policy_version 54562 (0.0005) [2023-12-27 06:02:29,614][20347] Updated weights for policy 0, policy_version 54644 (0.0008) [2023-12-27 06:02:29,665][20347] Updated weights for policy 0, policy_version 54654 (0.0010) [2023-12-27 06:02:29,721][20347] Updated weights for policy 0, policy_version 54664 (0.0009) [2023-12-27 06:02:29,773][20381] Updated weights for policy 1, policy_version 54572 (0.0007) [2023-12-27 06:02:29,821][20381] Updated weights for policy 1, policy_version 54582 (0.0007) [2023-12-27 06:02:29,878][20381] Updated weights for policy 1, policy_version 54592 (0.0007) [2023-12-27 06:02:30,459][20347] Updated weights for policy 0, policy_version 54674 (0.0008) [2023-12-27 06:02:30,507][20381] Updated weights for policy 1, policy_version 54602 (0.0007) [2023-12-27 06:02:30,511][20347] Updated weights for policy 0, policy_version 54684 (0.0008) [2023-12-27 06:02:30,561][20347] Updated weights for policy 0, policy_version 54694 (0.0008) [2023-12-27 06:02:30,563][20381] Updated weights for policy 1, policy_version 54612 (0.0008) [2023-12-27 06:02:30,618][20347] Updated weights for policy 0, policy_version 54704 (0.0007) [2023-12-27 06:02:30,620][20381] Updated weights for policy 1, policy_version 54622 (0.0006) [2023-12-27 06:02:30,682][20381] Updated weights for policy 1, policy_version 54632 (0.0008) [2023-12-27 06:02:31,329][20347] Updated weights for policy 0, policy_version 54714 (0.0007) [2023-12-27 06:02:31,399][20347] Updated weights for policy 0, policy_version 54724 (0.0007) [2023-12-27 06:02:31,412][20381] Updated weights for policy 1, policy_version 54642 (0.0007) [2023-12-27 06:02:31,457][20347] Updated weights for policy 0, policy_version 54734 (0.0005) [2023-12-27 06:02:31,462][20381] Updated weights for policy 1, policy_version 54652 (0.0008) [2023-12-27 06:02:31,524][20381] Updated weights for policy 1, policy_version 54662 (0.0008) [2023-12-27 06:02:32,098][20347] Updated weights for policy 0, policy_version 54744 (0.0007) [2023-12-27 06:02:32,150][20347] Updated weights for policy 0, policy_version 54754 (0.0008) [2023-12-27 06:02:32,196][20381] Updated weights for policy 1, policy_version 54672 (0.0008) [2023-12-27 06:02:32,202][20347] Updated weights for policy 0, policy_version 54764 (0.0007) [2023-12-27 06:02:32,257][20381] Updated weights for policy 1, policy_version 54682 (0.0007) [2023-12-27 06:02:32,319][20381] Updated weights for policy 1, policy_version 54692 (0.0007) [2023-12-27 06:02:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 28033024. Throughput: 0: 10398.0, 1: 10380.8. Samples: 28037536. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:32,927][20347] Updated weights for policy 0, policy_version 54774 (0.0006) [2023-12-27 06:02:32,950][20381] Updated weights for policy 1, policy_version 54702 (0.0009) [2023-12-27 06:02:32,982][20347] Updated weights for policy 0, policy_version 54784 (0.0010) [2023-12-27 06:02:33,007][20381] Updated weights for policy 1, policy_version 54712 (0.0010) [2023-12-27 06:02:33,039][20347] Updated weights for policy 0, policy_version 54794 (0.0010) [2023-12-27 06:02:33,058][20381] Updated weights for policy 1, policy_version 54722 (0.0007) [2023-12-27 06:02:33,698][20347] Updated weights for policy 0, policy_version 54804 (0.0009) [2023-12-27 06:02:33,745][20381] Updated weights for policy 1, policy_version 54732 (0.0007) [2023-12-27 06:02:33,751][20347] Updated weights for policy 0, policy_version 54814 (0.0006) [2023-12-27 06:02:33,793][20381] Updated weights for policy 1, policy_version 54742 (0.0005) [2023-12-27 06:02:33,806][20347] Updated weights for policy 0, policy_version 54824 (0.0005) [2023-12-27 06:02:33,848][20381] Updated weights for policy 1, policy_version 54752 (0.0005) [2023-12-27 06:02:34,398][20347] Updated weights for policy 0, policy_version 54834 (0.0006) [2023-12-27 06:02:34,459][20347] Updated weights for policy 0, policy_version 54844 (0.0008) [2023-12-27 06:02:34,519][20347] Updated weights for policy 0, policy_version 54854 (0.0007) [2023-12-27 06:02:34,580][20347] Updated weights for policy 0, policy_version 54864 (0.0006) [2023-12-27 06:02:34,585][20381] Updated weights for policy 1, policy_version 54762 (0.0008) [2023-12-27 06:02:34,641][20381] Updated weights for policy 1, policy_version 54772 (0.0008) [2023-12-27 06:02:34,698][20381] Updated weights for policy 1, policy_version 54782 (0.0009) [2023-12-27 06:02:34,750][20381] Updated weights for policy 1, policy_version 54792 (0.0008) [2023-12-27 06:02:35,167][20347] Updated weights for policy 0, policy_version 54874 (0.0006) [2023-12-27 06:02:35,219][20347] Updated weights for policy 0, policy_version 54884 (0.0007) [2023-12-27 06:02:35,274][20347] Updated weights for policy 0, policy_version 54894 (0.0007) [2023-12-27 06:02:35,480][20381] Updated weights for policy 1, policy_version 54802 (0.0008) [2023-12-27 06:02:35,534][20381] Updated weights for policy 1, policy_version 54812 (0.0008) [2023-12-27 06:02:35,579][20381] Updated weights for policy 1, policy_version 54822 (0.0010) [2023-12-27 06:02:35,902][20347] Updated weights for policy 0, policy_version 54904 (0.0008) [2023-12-27 06:02:35,950][20347] Updated weights for policy 0, policy_version 54914 (0.0007) [2023-12-27 06:02:35,996][20347] Updated weights for policy 0, policy_version 54924 (0.0008) [2023-12-27 06:02:36,297][20381] Updated weights for policy 1, policy_version 54832 (0.0007) [2023-12-27 06:02:36,364][20381] Updated weights for policy 1, policy_version 54842 (0.0008) [2023-12-27 06:02:36,430][20381] Updated weights for policy 1, policy_version 54852 (0.0008) [2023-12-27 06:02:36,706][20347] Updated weights for policy 0, policy_version 54934 (0.0009) [2023-12-27 06:02:36,765][20347] Updated weights for policy 0, policy_version 54944 (0.0010) [2023-12-27 06:02:36,820][20347] Updated weights for policy 0, policy_version 54954 (0.0010) [2023-12-27 06:02:37,055][20381] Updated weights for policy 1, policy_version 54862 (0.0007) [2023-12-27 06:02:37,117][20381] Updated weights for policy 1, policy_version 54872 (0.0007) [2023-12-27 06:02:37,175][20381] Updated weights for policy 1, policy_version 54882 (0.0008) [2023-12-27 06:02:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 28131328. Throughput: 0: 10406.8, 1: 10387.9. Samples: 28099776. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000054888_14057472.pth... [2023-12-27 06:02:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000054960_14073856.pth... [2023-12-27 06:02:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000053672_13746176.pth [2023-12-27 06:02:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000053776_13770752.pth [2023-12-27 06:02:37,490][20347] Updated weights for policy 0, policy_version 54964 (0.0009) [2023-12-27 06:02:37,547][20347] Updated weights for policy 0, policy_version 54974 (0.0007) [2023-12-27 06:02:37,609][20347] Updated weights for policy 0, policy_version 54984 (0.0007) [2023-12-27 06:02:37,858][20381] Updated weights for policy 1, policy_version 54892 (0.0007) [2023-12-27 06:02:37,919][20381] Updated weights for policy 1, policy_version 54902 (0.0006) [2023-12-27 06:02:37,976][20381] Updated weights for policy 1, policy_version 54912 (0.0005) [2023-12-27 06:02:38,286][20347] Updated weights for policy 0, policy_version 54994 (0.0007) [2023-12-27 06:02:38,346][20347] Updated weights for policy 0, policy_version 55004 (0.0007) [2023-12-27 06:02:38,406][20347] Updated weights for policy 0, policy_version 55014 (0.0011) [2023-12-27 06:02:38,467][20347] Updated weights for policy 0, policy_version 55024 (0.0011) [2023-12-27 06:02:38,620][20381] Updated weights for policy 1, policy_version 54922 (0.0008) [2023-12-27 06:02:38,689][20381] Updated weights for policy 1, policy_version 54932 (0.0010) [2023-12-27 06:02:38,754][20381] Updated weights for policy 1, policy_version 54942 (0.0005) [2023-12-27 06:02:38,819][20381] Updated weights for policy 1, policy_version 54952 (0.0006) [2023-12-27 06:02:39,090][20347] Updated weights for policy 0, policy_version 55034 (0.0010) [2023-12-27 06:02:39,142][20347] Updated weights for policy 0, policy_version 55044 (0.0010) [2023-12-27 06:02:39,187][20347] Updated weights for policy 0, policy_version 55054 (0.0010) [2023-12-27 06:02:39,427][20381] Updated weights for policy 1, policy_version 54962 (0.0009) [2023-12-27 06:02:39,482][20381] Updated weights for policy 1, policy_version 54972 (0.0009) [2023-12-27 06:02:39,541][20381] Updated weights for policy 1, policy_version 54982 (0.0010) [2023-12-27 06:02:39,930][20347] Updated weights for policy 0, policy_version 55064 (0.0011) [2023-12-27 06:02:39,996][20347] Updated weights for policy 0, policy_version 55074 (0.0010) [2023-12-27 06:02:40,057][20347] Updated weights for policy 0, policy_version 55084 (0.0010) [2023-12-27 06:02:40,147][20381] Updated weights for policy 1, policy_version 54992 (0.0006) [2023-12-27 06:02:40,200][20381] Updated weights for policy 1, policy_version 55002 (0.0006) [2023-12-27 06:02:40,258][20381] Updated weights for policy 1, policy_version 55012 (0.0006) [2023-12-27 06:02:40,741][20347] Updated weights for policy 0, policy_version 55094 (0.0007) [2023-12-27 06:02:40,803][20347] Updated weights for policy 0, policy_version 55104 (0.0006) [2023-12-27 06:02:40,869][20381] Updated weights for policy 1, policy_version 55022 (0.0006) [2023-12-27 06:02:40,872][20347] Updated weights for policy 0, policy_version 55114 (0.0005) [2023-12-27 06:02:40,928][20381] Updated weights for policy 1, policy_version 55032 (0.0007) [2023-12-27 06:02:40,985][20381] Updated weights for policy 1, policy_version 55042 (0.0009) [2023-12-27 06:02:41,554][20347] Updated weights for policy 0, policy_version 55124 (0.0007) [2023-12-27 06:02:41,604][20381] Updated weights for policy 1, policy_version 55052 (0.0007) [2023-12-27 06:02:41,608][20347] Updated weights for policy 0, policy_version 55134 (0.0007) [2023-12-27 06:02:41,669][20381] Updated weights for policy 1, policy_version 55062 (0.0007) [2023-12-27 06:02:41,671][20347] Updated weights for policy 0, policy_version 55144 (0.0007) [2023-12-27 06:02:41,722][20381] Updated weights for policy 1, policy_version 55072 (0.0006) [2023-12-27 06:02:42,306][20347] Updated weights for policy 0, policy_version 55154 (0.0007) [2023-12-27 06:02:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 28229632. Throughput: 0: 10354.2, 1: 10441.1. Samples: 28224528. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:42,369][20347] Updated weights for policy 0, policy_version 55164 (0.0007) [2023-12-27 06:02:42,411][20381] Updated weights for policy 1, policy_version 55082 (0.0008) [2023-12-27 06:02:42,432][20347] Updated weights for policy 0, policy_version 55174 (0.0007) [2023-12-27 06:02:42,468][20381] Updated weights for policy 1, policy_version 55092 (0.0007) [2023-12-27 06:02:42,488][20347] Updated weights for policy 0, policy_version 55184 (0.0006) [2023-12-27 06:02:42,526][20381] Updated weights for policy 1, policy_version 55102 (0.0007) [2023-12-27 06:02:42,587][20381] Updated weights for policy 1, policy_version 55112 (0.0008) [2023-12-27 06:02:43,179][20381] Updated weights for policy 1, policy_version 55122 (0.0008) [2023-12-27 06:02:43,237][20347] Updated weights for policy 0, policy_version 55194 (0.0006) [2023-12-27 06:02:43,245][20381] Updated weights for policy 1, policy_version 55132 (0.0007) [2023-12-27 06:02:43,298][20347] Updated weights for policy 0, policy_version 55204 (0.0006) [2023-12-27 06:02:43,309][20381] Updated weights for policy 1, policy_version 55142 (0.0007) [2023-12-27 06:02:43,357][20347] Updated weights for policy 0, policy_version 55214 (0.0005) [2023-12-27 06:02:43,958][20381] Updated weights for policy 1, policy_version 55152 (0.0006) [2023-12-27 06:02:43,984][20347] Updated weights for policy 0, policy_version 55224 (0.0007) [2023-12-27 06:02:44,016][20381] Updated weights for policy 1, policy_version 55162 (0.0005) [2023-12-27 06:02:44,033][20347] Updated weights for policy 0, policy_version 55234 (0.0007) [2023-12-27 06:02:44,067][20381] Updated weights for policy 1, policy_version 55172 (0.0005) [2023-12-27 06:02:44,087][20347] Updated weights for policy 0, policy_version 55244 (0.0006) [2023-12-27 06:02:44,649][20381] Updated weights for policy 1, policy_version 55182 (0.0007) [2023-12-27 06:02:44,700][20381] Updated weights for policy 1, policy_version 55192 (0.0007) [2023-12-27 06:02:44,742][20347] Updated weights for policy 0, policy_version 55254 (0.0007) [2023-12-27 06:02:44,756][20381] Updated weights for policy 1, policy_version 55202 (0.0006) [2023-12-27 06:02:44,800][20347] Updated weights for policy 0, policy_version 55264 (0.0007) [2023-12-27 06:02:44,861][20347] Updated weights for policy 0, policy_version 55274 (0.0006) [2023-12-27 06:02:45,456][20347] Updated weights for policy 0, policy_version 55284 (0.0006) [2023-12-27 06:02:45,493][20381] Updated weights for policy 1, policy_version 55212 (0.0008) [2023-12-27 06:02:45,516][20347] Updated weights for policy 0, policy_version 55294 (0.0006) [2023-12-27 06:02:45,549][20381] Updated weights for policy 1, policy_version 55222 (0.0009) [2023-12-27 06:02:45,573][20347] Updated weights for policy 0, policy_version 55304 (0.0005) [2023-12-27 06:02:45,605][20381] Updated weights for policy 1, policy_version 55232 (0.0008) [2023-12-27 06:02:46,109][20347] Updated weights for policy 0, policy_version 55314 (0.0006) [2023-12-27 06:02:46,161][20347] Updated weights for policy 0, policy_version 55324 (0.0007) [2023-12-27 06:02:46,212][20347] Updated weights for policy 0, policy_version 55334 (0.0007) [2023-12-27 06:02:46,261][20347] Updated weights for policy 0, policy_version 55344 (0.0007) [2023-12-27 06:02:46,329][20381] Updated weights for policy 1, policy_version 55242 (0.0008) [2023-12-27 06:02:46,382][20381] Updated weights for policy 1, policy_version 55252 (0.0008) [2023-12-27 06:02:46,443][20381] Updated weights for policy 1, policy_version 55262 (0.0008) [2023-12-27 06:02:46,502][20381] Updated weights for policy 1, policy_version 55272 (0.0008) [2023-12-27 06:02:46,914][20347] Updated weights for policy 0, policy_version 55354 (0.0008) [2023-12-27 06:02:46,970][20347] Updated weights for policy 0, policy_version 55364 (0.0007) [2023-12-27 06:02:47,022][20347] Updated weights for policy 0, policy_version 55374 (0.0007) [2023-12-27 06:02:47,189][20381] Updated weights for policy 1, policy_version 55282 (0.0007) [2023-12-27 06:02:47,248][20381] Updated weights for policy 1, policy_version 55292 (0.0007) [2023-12-27 06:02:47,306][20381] Updated weights for policy 1, policy_version 55302 (0.0008) [2023-12-27 06:02:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 28344320. Throughput: 0: 10327.9, 1: 10445.3. Samples: 28350088. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:47,649][20347] Updated weights for policy 0, policy_version 55384 (0.0006) [2023-12-27 06:02:47,714][20347] Updated weights for policy 0, policy_version 55394 (0.0006) [2023-12-27 06:02:47,772][20347] Updated weights for policy 0, policy_version 55404 (0.0010) [2023-12-27 06:02:47,899][20381] Updated weights for policy 1, policy_version 55312 (0.0008) [2023-12-27 06:02:47,955][20381] Updated weights for policy 1, policy_version 55322 (0.0008) [2023-12-27 06:02:48,019][20381] Updated weights for policy 1, policy_version 55332 (0.0007) [2023-12-27 06:02:48,379][20347] Updated weights for policy 0, policy_version 55414 (0.0009) [2023-12-27 06:02:48,441][20347] Updated weights for policy 0, policy_version 55424 (0.0009) [2023-12-27 06:02:48,506][20347] Updated weights for policy 0, policy_version 55434 (0.0007) [2023-12-27 06:02:48,720][20381] Updated weights for policy 1, policy_version 55342 (0.0007) [2023-12-27 06:02:48,775][20381] Updated weights for policy 1, policy_version 55352 (0.0006) [2023-12-27 06:02:48,820][20381] Updated weights for policy 1, policy_version 55362 (0.0005) [2023-12-27 06:02:49,185][20347] Updated weights for policy 0, policy_version 55444 (0.0008) [2023-12-27 06:02:49,243][20347] Updated weights for policy 0, policy_version 55454 (0.0009) [2023-12-27 06:02:49,304][20347] Updated weights for policy 0, policy_version 55464 (0.0007) [2023-12-27 06:02:49,519][20381] Updated weights for policy 1, policy_version 55372 (0.0008) [2023-12-27 06:02:49,586][20381] Updated weights for policy 1, policy_version 55382 (0.0008) [2023-12-27 06:02:49,641][20381] Updated weights for policy 1, policy_version 55392 (0.0007) [2023-12-27 06:02:49,935][20347] Updated weights for policy 0, policy_version 55474 (0.0009) [2023-12-27 06:02:49,993][20347] Updated weights for policy 0, policy_version 55484 (0.0007) [2023-12-27 06:02:50,053][20347] Updated weights for policy 0, policy_version 55494 (0.0008) [2023-12-27 06:02:50,109][20347] Updated weights for policy 0, policy_version 55504 (0.0007) [2023-12-27 06:02:50,286][20381] Updated weights for policy 1, policy_version 55402 (0.0006) [2023-12-27 06:02:50,340][20381] Updated weights for policy 1, policy_version 55412 (0.0007) [2023-12-27 06:02:50,392][20381] Updated weights for policy 1, policy_version 55422 (0.0005) [2023-12-27 06:02:50,449][20381] Updated weights for policy 1, policy_version 55432 (0.0007) [2023-12-27 06:02:50,717][20347] Updated weights for policy 0, policy_version 55514 (0.0006) [2023-12-27 06:02:50,778][20347] Updated weights for policy 0, policy_version 55524 (0.0008) [2023-12-27 06:02:50,834][20347] Updated weights for policy 0, policy_version 55534 (0.0008) [2023-12-27 06:02:51,110][20381] Updated weights for policy 1, policy_version 55442 (0.0006) [2023-12-27 06:02:51,174][20381] Updated weights for policy 1, policy_version 55452 (0.0007) [2023-12-27 06:02:51,237][20381] Updated weights for policy 1, policy_version 55462 (0.0008) [2023-12-27 06:02:51,547][20347] Updated weights for policy 0, policy_version 55544 (0.0006) [2023-12-27 06:02:51,608][20347] Updated weights for policy 0, policy_version 55554 (0.0006) [2023-12-27 06:02:51,671][20347] Updated weights for policy 0, policy_version 55564 (0.0007) [2023-12-27 06:02:51,943][20381] Updated weights for policy 1, policy_version 55472 (0.0008) [2023-12-27 06:02:52,010][20381] Updated weights for policy 1, policy_version 55482 (0.0008) [2023-12-27 06:02:52,076][20381] Updated weights for policy 1, policy_version 55492 (0.0007) [2023-12-27 06:02:52,349][20347] Updated weights for policy 0, policy_version 55574 (0.0007) [2023-12-27 06:02:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.8). Total num frames: 28442624. Throughput: 0: 10317.7, 1: 10422.3. Samples: 28414080. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 06:02:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000055496_14213120.pth... [2023-12-27 06:02:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000054280_13901824.pth [2023-12-27 06:02:52,409][20347] Updated weights for policy 0, policy_version 55584 (0.0008) [2023-12-27 06:02:52,458][20347] Updated weights for policy 0, policy_version 55594 (0.0008) [2023-12-27 06:02:52,488][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000055600_14237696.pth... [2023-12-27 06:02:52,491][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000054352_13918208.pth [2023-12-27 06:02:52,697][20381] Updated weights for policy 1, policy_version 55502 (0.0007) [2023-12-27 06:02:52,751][20381] Updated weights for policy 1, policy_version 55512 (0.0007) [2023-12-27 06:02:52,796][20381] Updated weights for policy 1, policy_version 55522 (0.0007) [2023-12-27 06:02:53,157][20347] Updated weights for policy 0, policy_version 55604 (0.0007) [2023-12-27 06:02:53,220][20347] Updated weights for policy 0, policy_version 55614 (0.0006) [2023-12-27 06:02:53,282][20347] Updated weights for policy 0, policy_version 55624 (0.0005) [2023-12-27 06:02:53,532][20381] Updated weights for policy 1, policy_version 55532 (0.0008) [2023-12-27 06:02:53,582][20381] Updated weights for policy 1, policy_version 55542 (0.0008) [2023-12-27 06:02:53,634][20381] Updated weights for policy 1, policy_version 55552 (0.0008) [2023-12-27 06:02:53,900][20347] Updated weights for policy 0, policy_version 55634 (0.0006) [2023-12-27 06:02:53,963][20347] Updated weights for policy 0, policy_version 55644 (0.0005) [2023-12-27 06:02:54,027][20347] Updated weights for policy 0, policy_version 55654 (0.0006) [2023-12-27 06:02:54,080][20347] Updated weights for policy 0, policy_version 55664 (0.0007) [2023-12-27 06:02:54,345][20381] Updated weights for policy 1, policy_version 55562 (0.0007) [2023-12-27 06:02:54,390][20381] Updated weights for policy 1, policy_version 55572 (0.0007) [2023-12-27 06:02:54,445][20381] Updated weights for policy 1, policy_version 55582 (0.0008) [2023-12-27 06:02:54,501][20381] Updated weights for policy 1, policy_version 55592 (0.0007) [2023-12-27 06:02:54,748][20347] Updated weights for policy 0, policy_version 55674 (0.0007) [2023-12-27 06:02:54,809][20347] Updated weights for policy 0, policy_version 55684 (0.0007) [2023-12-27 06:02:54,865][20347] Updated weights for policy 0, policy_version 55694 (0.0008) [2023-12-27 06:02:55,174][20381] Updated weights for policy 1, policy_version 55602 (0.0008) [2023-12-27 06:02:55,233][20381] Updated weights for policy 1, policy_version 55612 (0.0008) [2023-12-27 06:02:55,286][20381] Updated weights for policy 1, policy_version 55622 (0.0007) [2023-12-27 06:02:55,558][20347] Updated weights for policy 0, policy_version 55704 (0.0007) [2023-12-27 06:02:55,614][20347] Updated weights for policy 0, policy_version 55714 (0.0008) [2023-12-27 06:02:55,676][20347] Updated weights for policy 0, policy_version 55724 (0.0007) [2023-12-27 06:02:55,994][20381] Updated weights for policy 1, policy_version 55632 (0.0009) [2023-12-27 06:02:56,053][20381] Updated weights for policy 1, policy_version 55642 (0.0008) [2023-12-27 06:02:56,117][20381] Updated weights for policy 1, policy_version 55652 (0.0008) [2023-12-27 06:02:56,304][20347] Updated weights for policy 0, policy_version 55734 (0.0007) [2023-12-27 06:02:56,367][20347] Updated weights for policy 0, policy_version 55744 (0.0005) [2023-12-27 06:02:56,425][20347] Updated weights for policy 0, policy_version 55754 (0.0009) [2023-12-27 06:02:56,849][20381] Updated weights for policy 1, policy_version 55662 (0.0008) [2023-12-27 06:02:56,903][20381] Updated weights for policy 1, policy_version 55672 (0.0007) [2023-12-27 06:02:56,966][20381] Updated weights for policy 1, policy_version 55682 (0.0006) [2023-12-27 06:02:57,000][20347] Updated weights for policy 0, policy_version 55764 (0.0007) [2023-12-27 06:02:57,056][20347] Updated weights for policy 0, policy_version 55774 (0.0010) [2023-12-27 06:02:57,108][20347] Updated weights for policy 0, policy_version 55784 (0.0010) [2023-12-27 06:02:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 28549120. Throughput: 0: 10380.5, 1: 10385.3. Samples: 28535952. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:02:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:02:57,648][20381] Updated weights for policy 1, policy_version 55692 (0.0010) [2023-12-27 06:02:57,698][20381] Updated weights for policy 1, policy_version 55702 (0.0010) [2023-12-27 06:02:57,746][20381] Updated weights for policy 1, policy_version 55712 (0.0010) [2023-12-27 06:02:57,797][20347] Updated weights for policy 0, policy_version 55794 (0.0010) [2023-12-27 06:02:57,856][20347] Updated weights for policy 0, policy_version 55804 (0.0007) [2023-12-27 06:02:57,909][20347] Updated weights for policy 0, policy_version 55814 (0.0007) [2023-12-27 06:02:57,971][20347] Updated weights for policy 0, policy_version 55824 (0.0008) [2023-12-27 06:02:58,525][20381] Updated weights for policy 1, policy_version 55722 (0.0010) [2023-12-27 06:02:58,583][20381] Updated weights for policy 1, policy_version 55732 (0.0008) [2023-12-27 06:02:58,643][20381] Updated weights for policy 1, policy_version 55742 (0.0006) [2023-12-27 06:02:58,676][20347] Updated weights for policy 0, policy_version 55834 (0.0007) [2023-12-27 06:02:58,701][20381] Updated weights for policy 1, policy_version 55752 (0.0006) [2023-12-27 06:02:58,739][20347] Updated weights for policy 0, policy_version 55844 (0.0007) [2023-12-27 06:02:58,796][20347] Updated weights for policy 0, policy_version 55854 (0.0008) [2023-12-27 06:02:59,443][20381] Updated weights for policy 1, policy_version 55762 (0.0008) [2023-12-27 06:02:59,452][20347] Updated weights for policy 0, policy_version 55864 (0.0007) [2023-12-27 06:02:59,497][20381] Updated weights for policy 1, policy_version 55772 (0.0008) [2023-12-27 06:02:59,508][20347] Updated weights for policy 0, policy_version 55874 (0.0006) [2023-12-27 06:02:59,551][20381] Updated weights for policy 1, policy_version 55782 (0.0006) [2023-12-27 06:02:59,570][20347] Updated weights for policy 0, policy_version 55884 (0.0010) [2023-12-27 06:03:00,189][20347] Updated weights for policy 0, policy_version 55894 (0.0007) [2023-12-27 06:03:00,246][20347] Updated weights for policy 0, policy_version 55904 (0.0006) [2023-12-27 06:03:00,258][20381] Updated weights for policy 1, policy_version 55792 (0.0007) [2023-12-27 06:03:00,301][20347] Updated weights for policy 0, policy_version 55914 (0.0005) [2023-12-27 06:03:00,315][20381] Updated weights for policy 1, policy_version 55802 (0.0008) [2023-12-27 06:03:00,376][20381] Updated weights for policy 1, policy_version 55812 (0.0008) [2023-12-27 06:03:00,863][20347] Updated weights for policy 0, policy_version 55924 (0.0007) [2023-12-27 06:03:00,915][20347] Updated weights for policy 0, policy_version 55934 (0.0010) [2023-12-27 06:03:00,965][20347] Updated weights for policy 0, policy_version 55944 (0.0010) [2023-12-27 06:03:01,026][20381] Updated weights for policy 1, policy_version 55822 (0.0010) [2023-12-27 06:03:01,085][20381] Updated weights for policy 1, policy_version 55832 (0.0010) [2023-12-27 06:03:01,142][20381] Updated weights for policy 1, policy_version 55842 (0.0010) [2023-12-27 06:03:01,638][20347] Updated weights for policy 0, policy_version 55954 (0.0010) [2023-12-27 06:03:01,700][20347] Updated weights for policy 0, policy_version 55964 (0.0010) [2023-12-27 06:03:01,766][20347] Updated weights for policy 0, policy_version 55974 (0.0010) [2023-12-27 06:03:01,824][20347] Updated weights for policy 0, policy_version 55984 (0.0010) [2023-12-27 06:03:01,835][20381] Updated weights for policy 1, policy_version 55852 (0.0008) [2023-12-27 06:03:01,884][20381] Updated weights for policy 1, policy_version 55862 (0.0005) [2023-12-27 06:03:01,949][20381] Updated weights for policy 1, policy_version 55872 (0.0005) [2023-12-27 06:03:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 28647424. Throughput: 0: 10450.5, 1: 10373.8. Samples: 28660484. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:02,515][20381] Updated weights for policy 1, policy_version 55882 (0.0006) [2023-12-27 06:03:02,531][20347] Updated weights for policy 0, policy_version 55994 (0.0006) [2023-12-27 06:03:02,571][20381] Updated weights for policy 1, policy_version 55892 (0.0011) [2023-12-27 06:03:02,587][20347] Updated weights for policy 0, policy_version 56004 (0.0005) [2023-12-27 06:03:02,627][20381] Updated weights for policy 1, policy_version 55902 (0.0011) [2023-12-27 06:03:02,643][20347] Updated weights for policy 0, policy_version 56014 (0.0005) [2023-12-27 06:03:02,675][20381] Updated weights for policy 1, policy_version 55912 (0.0010) [2023-12-27 06:03:03,251][20347] Updated weights for policy 0, policy_version 56024 (0.0007) [2023-12-27 06:03:03,278][20381] Updated weights for policy 1, policy_version 55922 (0.0006) [2023-12-27 06:03:03,304][20347] Updated weights for policy 0, policy_version 56034 (0.0007) [2023-12-27 06:03:03,333][20381] Updated weights for policy 1, policy_version 55932 (0.0006) [2023-12-27 06:03:03,353][20347] Updated weights for policy 0, policy_version 56044 (0.0007) [2023-12-27 06:03:03,388][20381] Updated weights for policy 1, policy_version 55942 (0.0005) [2023-12-27 06:03:03,972][20347] Updated weights for policy 0, policy_version 56054 (0.0007) [2023-12-27 06:03:04,023][20347] Updated weights for policy 0, policy_version 56064 (0.0007) [2023-12-27 06:03:04,044][20381] Updated weights for policy 1, policy_version 55952 (0.0005) [2023-12-27 06:03:04,080][20347] Updated weights for policy 0, policy_version 56074 (0.0006) [2023-12-27 06:03:04,104][20381] Updated weights for policy 1, policy_version 55962 (0.0007) [2023-12-27 06:03:04,167][20381] Updated weights for policy 1, policy_version 55972 (0.0008) [2023-12-27 06:03:04,704][20347] Updated weights for policy 0, policy_version 56084 (0.0005) [2023-12-27 06:03:04,766][20347] Updated weights for policy 0, policy_version 56094 (0.0009) [2023-12-27 06:03:04,828][20347] Updated weights for policy 0, policy_version 56104 (0.0010) [2023-12-27 06:03:04,862][20381] Updated weights for policy 1, policy_version 55982 (0.0006) [2023-12-27 06:03:04,909][20381] Updated weights for policy 1, policy_version 55992 (0.0007) [2023-12-27 06:03:04,955][20381] Updated weights for policy 1, policy_version 56002 (0.0007) [2023-12-27 06:03:05,467][20347] Updated weights for policy 0, policy_version 56114 (0.0009) [2023-12-27 06:03:05,512][20347] Updated weights for policy 0, policy_version 56124 (0.0007) [2023-12-27 06:03:05,561][20347] Updated weights for policy 0, policy_version 56134 (0.0008) [2023-12-27 06:03:05,614][20347] Updated weights for policy 0, policy_version 56144 (0.0007) [2023-12-27 06:03:05,650][20381] Updated weights for policy 1, policy_version 56012 (0.0007) [2023-12-27 06:03:05,698][20381] Updated weights for policy 1, policy_version 56022 (0.0007) [2023-12-27 06:03:05,747][20381] Updated weights for policy 1, policy_version 56032 (0.0007) [2023-12-27 06:03:06,321][20347] Updated weights for policy 0, policy_version 56154 (0.0005) [2023-12-27 06:03:06,379][20347] Updated weights for policy 0, policy_version 56164 (0.0006) [2023-12-27 06:03:06,417][20381] Updated weights for policy 1, policy_version 56042 (0.0008) [2023-12-27 06:03:06,431][20347] Updated weights for policy 0, policy_version 56174 (0.0005) [2023-12-27 06:03:06,479][20381] Updated weights for policy 1, policy_version 56052 (0.0009) [2023-12-27 06:03:06,536][20381] Updated weights for policy 1, policy_version 56062 (0.0007) [2023-12-27 06:03:06,596][20381] Updated weights for policy 1, policy_version 56072 (0.0007) [2023-12-27 06:03:07,051][20347] Updated weights for policy 0, policy_version 56184 (0.0006) [2023-12-27 06:03:07,114][20347] Updated weights for policy 0, policy_version 56194 (0.0007) [2023-12-27 06:03:07,174][20347] Updated weights for policy 0, policy_version 56204 (0.0008) [2023-12-27 06:03:07,294][20381] Updated weights for policy 1, policy_version 56082 (0.0008) [2023-12-27 06:03:07,349][20381] Updated weights for policy 1, policy_version 56092 (0.0008) [2023-12-27 06:03:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 28753920. Throughput: 0: 10493.3, 1: 10360.8. Samples: 28724408. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000056208_14393344.pth... [2023-12-27 06:03:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000054960_14073856.pth [2023-12-27 06:03:07,411][20381] Updated weights for policy 1, policy_version 56102 (0.0008) [2023-12-27 06:03:07,421][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000056104_14368768.pth... [2023-12-27 06:03:07,426][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000054888_14057472.pth [2023-12-27 06:03:07,845][20347] Updated weights for policy 0, policy_version 56214 (0.0007) [2023-12-27 06:03:07,899][20347] Updated weights for policy 0, policy_version 56224 (0.0007) [2023-12-27 06:03:07,954][20347] Updated weights for policy 0, policy_version 56234 (0.0007) [2023-12-27 06:03:08,108][20381] Updated weights for policy 1, policy_version 56112 (0.0008) [2023-12-27 06:03:08,164][20381] Updated weights for policy 1, policy_version 56122 (0.0008) [2023-12-27 06:03:08,224][20381] Updated weights for policy 1, policy_version 56132 (0.0008) [2023-12-27 06:03:08,605][20347] Updated weights for policy 0, policy_version 56244 (0.0008) [2023-12-27 06:03:08,659][20347] Updated weights for policy 0, policy_version 56254 (0.0006) [2023-12-27 06:03:08,719][20347] Updated weights for policy 0, policy_version 56264 (0.0006) [2023-12-27 06:03:08,924][20381] Updated weights for policy 1, policy_version 56142 (0.0008) [2023-12-27 06:03:08,976][20381] Updated weights for policy 1, policy_version 56152 (0.0007) [2023-12-27 06:03:09,032][20381] Updated weights for policy 1, policy_version 56162 (0.0007) [2023-12-27 06:03:09,372][20347] Updated weights for policy 0, policy_version 56274 (0.0007) [2023-12-27 06:03:09,434][20347] Updated weights for policy 0, policy_version 56284 (0.0008) [2023-12-27 06:03:09,501][20347] Updated weights for policy 0, policy_version 56294 (0.0008) [2023-12-27 06:03:09,560][20347] Updated weights for policy 0, policy_version 56304 (0.0008) [2023-12-27 06:03:09,760][20381] Updated weights for policy 1, policy_version 56172 (0.0008) [2023-12-27 06:03:09,825][20381] Updated weights for policy 1, policy_version 56182 (0.0008) [2023-12-27 06:03:09,885][20381] Updated weights for policy 1, policy_version 56192 (0.0009) [2023-12-27 06:03:10,240][20347] Updated weights for policy 0, policy_version 56314 (0.0008) [2023-12-27 06:03:10,299][20347] Updated weights for policy 0, policy_version 56324 (0.0008) [2023-12-27 06:03:10,358][20347] Updated weights for policy 0, policy_version 56334 (0.0008) [2023-12-27 06:03:10,578][20381] Updated weights for policy 1, policy_version 56202 (0.0010) [2023-12-27 06:03:10,626][20381] Updated weights for policy 1, policy_version 56212 (0.0008) [2023-12-27 06:03:10,672][20381] Updated weights for policy 1, policy_version 56222 (0.0007) [2023-12-27 06:03:10,720][20381] Updated weights for policy 1, policy_version 56232 (0.0007) [2023-12-27 06:03:11,020][20347] Updated weights for policy 0, policy_version 56344 (0.0007) [2023-12-27 06:03:11,083][20347] Updated weights for policy 0, policy_version 56354 (0.0007) [2023-12-27 06:03:11,145][20347] Updated weights for policy 0, policy_version 56364 (0.0007) [2023-12-27 06:03:11,450][20381] Updated weights for policy 1, policy_version 56242 (0.0007) [2023-12-27 06:03:11,511][20381] Updated weights for policy 1, policy_version 56252 (0.0006) [2023-12-27 06:03:11,570][20381] Updated weights for policy 1, policy_version 56262 (0.0006) [2023-12-27 06:03:11,938][20347] Updated weights for policy 0, policy_version 56374 (0.0008) [2023-12-27 06:03:11,999][20347] Updated weights for policy 0, policy_version 56384 (0.0008) [2023-12-27 06:03:12,062][20347] Updated weights for policy 0, policy_version 56394 (0.0008) [2023-12-27 06:03:12,182][20381] Updated weights for policy 1, policy_version 56272 (0.0007) [2023-12-27 06:03:12,233][20381] Updated weights for policy 1, policy_version 56282 (0.0008) [2023-12-27 06:03:12,294][20381] Updated weights for policy 1, policy_version 56292 (0.0007) [2023-12-27 06:03:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 28860416. Throughput: 0: 10530.2, 1: 10325.8. Samples: 28846804. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:12,765][20347] Updated weights for policy 0, policy_version 56404 (0.0011) [2023-12-27 06:03:12,817][20347] Updated weights for policy 0, policy_version 56414 (0.0011) [2023-12-27 06:03:12,875][20347] Updated weights for policy 0, policy_version 56424 (0.0010) [2023-12-27 06:03:12,993][20381] Updated weights for policy 1, policy_version 56302 (0.0009) [2023-12-27 06:03:13,042][20381] Updated weights for policy 1, policy_version 56312 (0.0009) [2023-12-27 06:03:13,095][20381] Updated weights for policy 1, policy_version 56322 (0.0010) [2023-12-27 06:03:13,510][20347] Updated weights for policy 0, policy_version 56434 (0.0007) [2023-12-27 06:03:13,568][20347] Updated weights for policy 0, policy_version 56444 (0.0008) [2023-12-27 06:03:13,625][20347] Updated weights for policy 0, policy_version 56454 (0.0008) [2023-12-27 06:03:13,685][20347] Updated weights for policy 0, policy_version 56464 (0.0008) [2023-12-27 06:03:13,766][20381] Updated weights for policy 1, policy_version 56332 (0.0009) [2023-12-27 06:03:13,820][20381] Updated weights for policy 1, policy_version 56342 (0.0008) [2023-12-27 06:03:13,876][20381] Updated weights for policy 1, policy_version 56352 (0.0007) [2023-12-27 06:03:14,399][20347] Updated weights for policy 0, policy_version 56474 (0.0008) [2023-12-27 06:03:14,447][20347] Updated weights for policy 0, policy_version 56484 (0.0007) [2023-12-27 06:03:14,498][20347] Updated weights for policy 0, policy_version 56494 (0.0008) [2023-12-27 06:03:14,499][20381] Updated weights for policy 1, policy_version 56362 (0.0007) [2023-12-27 06:03:14,562][20381] Updated weights for policy 1, policy_version 56372 (0.0005) [2023-12-27 06:03:14,622][20381] Updated weights for policy 1, policy_version 56382 (0.0005) [2023-12-27 06:03:14,680][20381] Updated weights for policy 1, policy_version 56392 (0.0005) [2023-12-27 06:03:15,216][20347] Updated weights for policy 0, policy_version 56504 (0.0007) [2023-12-27 06:03:15,275][20347] Updated weights for policy 0, policy_version 56514 (0.0006) [2023-12-27 06:03:15,285][20381] Updated weights for policy 1, policy_version 56402 (0.0007) [2023-12-27 06:03:15,332][20347] Updated weights for policy 0, policy_version 56524 (0.0005) [2023-12-27 06:03:15,341][20381] Updated weights for policy 1, policy_version 56412 (0.0006) [2023-12-27 06:03:15,398][20381] Updated weights for policy 1, policy_version 56422 (0.0007) [2023-12-27 06:03:16,027][20347] Updated weights for policy 0, policy_version 56534 (0.0006) [2023-12-27 06:03:16,065][20381] Updated weights for policy 1, policy_version 56432 (0.0008) [2023-12-27 06:03:16,087][20347] Updated weights for policy 0, policy_version 56544 (0.0005) [2023-12-27 06:03:16,120][20381] Updated weights for policy 1, policy_version 56442 (0.0007) [2023-12-27 06:03:16,149][20347] Updated weights for policy 0, policy_version 56554 (0.0006) [2023-12-27 06:03:16,172][20381] Updated weights for policy 1, policy_version 56452 (0.0007) [2023-12-27 06:03:16,764][20381] Updated weights for policy 1, policy_version 56462 (0.0007) [2023-12-27 06:03:16,781][20347] Updated weights for policy 0, policy_version 56564 (0.0005) [2023-12-27 06:03:16,822][20381] Updated weights for policy 1, policy_version 56472 (0.0005) [2023-12-27 06:03:16,830][20347] Updated weights for policy 0, policy_version 56574 (0.0006) [2023-12-27 06:03:16,871][20381] Updated weights for policy 1, policy_version 56482 (0.0008) [2023-12-27 06:03:16,885][20347] Updated weights for policy 0, policy_version 56584 (0.0005) [2023-12-27 06:03:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 28958720. Throughput: 0: 10448.4, 1: 10282.6. Samples: 28970436. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:17,517][20347] Updated weights for policy 0, policy_version 56594 (0.0005) [2023-12-27 06:03:17,549][20381] Updated weights for policy 1, policy_version 56492 (0.0006) [2023-12-27 06:03:17,570][20347] Updated weights for policy 0, policy_version 56604 (0.0007) [2023-12-27 06:03:17,605][20381] Updated weights for policy 1, policy_version 56502 (0.0005) [2023-12-27 06:03:17,624][20347] Updated weights for policy 0, policy_version 56614 (0.0009) [2023-12-27 06:03:17,661][20381] Updated weights for policy 1, policy_version 56512 (0.0008) [2023-12-27 06:03:17,677][20347] Updated weights for policy 0, policy_version 56624 (0.0005) [2023-12-27 06:03:18,264][20381] Updated weights for policy 1, policy_version 56522 (0.0007) [2023-12-27 06:03:18,313][20381] Updated weights for policy 1, policy_version 56532 (0.0006) [2023-12-27 06:03:18,350][20347] Updated weights for policy 0, policy_version 56634 (0.0010) [2023-12-27 06:03:18,377][20381] Updated weights for policy 1, policy_version 56542 (0.0008) [2023-12-27 06:03:18,412][20347] Updated weights for policy 0, policy_version 56644 (0.0010) [2023-12-27 06:03:18,435][20381] Updated weights for policy 1, policy_version 56552 (0.0007) [2023-12-27 06:03:18,470][20347] Updated weights for policy 0, policy_version 56654 (0.0010) [2023-12-27 06:03:19,031][20381] Updated weights for policy 1, policy_version 56562 (0.0007) [2023-12-27 06:03:19,079][20381] Updated weights for policy 1, policy_version 56572 (0.0008) [2023-12-27 06:03:19,124][20381] Updated weights for policy 1, policy_version 56582 (0.0007) [2023-12-27 06:03:19,145][20347] Updated weights for policy 0, policy_version 56664 (0.0010) [2023-12-27 06:03:19,193][20347] Updated weights for policy 0, policy_version 56674 (0.0010) [2023-12-27 06:03:19,255][20347] Updated weights for policy 0, policy_version 56684 (0.0010) [2023-12-27 06:03:19,826][20381] Updated weights for policy 1, policy_version 56592 (0.0007) [2023-12-27 06:03:19,885][20381] Updated weights for policy 1, policy_version 56602 (0.0007) [2023-12-27 06:03:19,944][20381] Updated weights for policy 1, policy_version 56612 (0.0006) [2023-12-27 06:03:20,012][20347] Updated weights for policy 0, policy_version 56694 (0.0011) [2023-12-27 06:03:20,073][20347] Updated weights for policy 0, policy_version 56704 (0.0010) [2023-12-27 06:03:20,131][20347] Updated weights for policy 0, policy_version 56714 (0.0010) [2023-12-27 06:03:20,569][20381] Updated weights for policy 1, policy_version 56622 (0.0007) [2023-12-27 06:03:20,628][20381] Updated weights for policy 1, policy_version 56632 (0.0008) [2023-12-27 06:03:20,676][20381] Updated weights for policy 1, policy_version 56642 (0.0007) [2023-12-27 06:03:20,835][20347] Updated weights for policy 0, policy_version 56724 (0.0010) [2023-12-27 06:03:20,891][20347] Updated weights for policy 0, policy_version 56734 (0.0008) [2023-12-27 06:03:20,955][20347] Updated weights for policy 0, policy_version 56744 (0.0008) [2023-12-27 06:03:21,354][20381] Updated weights for policy 1, policy_version 56652 (0.0008) [2023-12-27 06:03:21,414][20381] Updated weights for policy 1, policy_version 56662 (0.0006) [2023-12-27 06:03:21,470][20381] Updated weights for policy 1, policy_version 56672 (0.0005) [2023-12-27 06:03:21,701][20347] Updated weights for policy 0, policy_version 56754 (0.0009) [2023-12-27 06:03:21,765][20347] Updated weights for policy 0, policy_version 56764 (0.0008) [2023-12-27 06:03:21,823][20347] Updated weights for policy 0, policy_version 56774 (0.0006) [2023-12-27 06:03:21,872][20347] Updated weights for policy 0, policy_version 56784 (0.0006) [2023-12-27 06:03:22,161][20381] Updated weights for policy 1, policy_version 56682 (0.0006) [2023-12-27 06:03:22,215][20381] Updated weights for policy 1, policy_version 56692 (0.0009) [2023-12-27 06:03:22,274][20381] Updated weights for policy 1, policy_version 56702 (0.0007) [2023-12-27 06:03:22,328][20381] Updated weights for policy 1, policy_version 56712 (0.0008) [2023-12-27 06:03:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 29065216. Throughput: 0: 10408.2, 1: 10342.9. Samples: 29033580. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000056712_14524416.pth... [2023-12-27 06:03:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000056784_14540800.pth... [2023-12-27 06:03:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000055496_14213120.pth [2023-12-27 06:03:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000055600_14237696.pth [2023-12-27 06:03:22,491][20347] Updated weights for policy 0, policy_version 56794 (0.0010) [2023-12-27 06:03:22,537][20347] Updated weights for policy 0, policy_version 56804 (0.0010) [2023-12-27 06:03:22,584][20347] Updated weights for policy 0, policy_version 56814 (0.0010) [2023-12-27 06:03:23,006][20381] Updated weights for policy 1, policy_version 56722 (0.0005) [2023-12-27 06:03:23,065][20381] Updated weights for policy 1, policy_version 56732 (0.0006) [2023-12-27 06:03:23,118][20381] Updated weights for policy 1, policy_version 56742 (0.0005) [2023-12-27 06:03:23,316][20347] Updated weights for policy 0, policy_version 56824 (0.0006) [2023-12-27 06:03:23,368][20347] Updated weights for policy 0, policy_version 56834 (0.0008) [2023-12-27 06:03:23,427][20347] Updated weights for policy 0, policy_version 56844 (0.0010) [2023-12-27 06:03:23,683][20381] Updated weights for policy 1, policy_version 56752 (0.0006) [2023-12-27 06:03:23,734][20381] Updated weights for policy 1, policy_version 56762 (0.0005) [2023-12-27 06:03:23,785][20381] Updated weights for policy 1, policy_version 56772 (0.0006) [2023-12-27 06:03:24,026][20347] Updated weights for policy 0, policy_version 56854 (0.0007) [2023-12-27 06:03:24,079][20347] Updated weights for policy 0, policy_version 56864 (0.0005) [2023-12-27 06:03:24,132][20347] Updated weights for policy 0, policy_version 56874 (0.0007) [2023-12-27 06:03:24,362][20381] Updated weights for policy 1, policy_version 56782 (0.0007) [2023-12-27 06:03:24,406][20381] Updated weights for policy 1, policy_version 56792 (0.0010) [2023-12-27 06:03:24,452][20381] Updated weights for policy 1, policy_version 56802 (0.0010) [2023-12-27 06:03:24,813][20347] Updated weights for policy 0, policy_version 56884 (0.0008) [2023-12-27 06:03:24,865][20347] Updated weights for policy 0, policy_version 56894 (0.0007) [2023-12-27 06:03:24,919][20347] Updated weights for policy 0, policy_version 56904 (0.0008) [2023-12-27 06:03:25,100][20381] Updated weights for policy 1, policy_version 56812 (0.0008) [2023-12-27 06:03:25,161][20381] Updated weights for policy 1, policy_version 56822 (0.0006) [2023-12-27 06:03:25,213][20381] Updated weights for policy 1, policy_version 56832 (0.0005) [2023-12-27 06:03:25,588][20347] Updated weights for policy 0, policy_version 56914 (0.0008) [2023-12-27 06:03:25,645][20347] Updated weights for policy 0, policy_version 56924 (0.0010) [2023-12-27 06:03:25,709][20347] Updated weights for policy 0, policy_version 56934 (0.0010) [2023-12-27 06:03:25,765][20347] Updated weights for policy 0, policy_version 56944 (0.0010) [2023-12-27 06:03:25,791][20381] Updated weights for policy 1, policy_version 56842 (0.0006) [2023-12-27 06:03:25,847][20381] Updated weights for policy 1, policy_version 56852 (0.0010) [2023-12-27 06:03:25,896][20381] Updated weights for policy 1, policy_version 56862 (0.0010) [2023-12-27 06:03:25,950][20381] Updated weights for policy 1, policy_version 56872 (0.0009) [2023-12-27 06:03:26,436][20347] Updated weights for policy 0, policy_version 56954 (0.0010) [2023-12-27 06:03:26,496][20347] Updated weights for policy 0, policy_version 56964 (0.0010) [2023-12-27 06:03:26,546][20347] Updated weights for policy 0, policy_version 56974 (0.0010) [2023-12-27 06:03:26,592][20381] Updated weights for policy 1, policy_version 56882 (0.0007) [2023-12-27 06:03:26,636][20381] Updated weights for policy 1, policy_version 56892 (0.0007) [2023-12-27 06:03:26,682][20381] Updated weights for policy 1, policy_version 56902 (0.0007) [2023-12-27 06:03:27,227][20347] Updated weights for policy 0, policy_version 56984 (0.0010) [2023-12-27 06:03:27,287][20347] Updated weights for policy 0, policy_version 56994 (0.0011) [2023-12-27 06:03:27,346][20347] Updated weights for policy 0, policy_version 57004 (0.0010) [2023-12-27 06:03:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 29163520. Throughput: 0: 10410.2, 1: 10396.1. Samples: 29160812. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:27,395][20381] Updated weights for policy 1, policy_version 56912 (0.0008) [2023-12-27 06:03:27,448][20381] Updated weights for policy 1, policy_version 56922 (0.0008) [2023-12-27 06:03:27,510][20381] Updated weights for policy 1, policy_version 56932 (0.0006) [2023-12-27 06:03:28,017][20347] Updated weights for policy 0, policy_version 57014 (0.0010) [2023-12-27 06:03:28,063][20347] Updated weights for policy 0, policy_version 57024 (0.0010) [2023-12-27 06:03:28,114][20347] Updated weights for policy 0, policy_version 57034 (0.0010) [2023-12-27 06:03:28,199][20381] Updated weights for policy 1, policy_version 56942 (0.0009) [2023-12-27 06:03:28,247][20381] Updated weights for policy 1, policy_version 56952 (0.0010) [2023-12-27 06:03:28,296][20381] Updated weights for policy 1, policy_version 56962 (0.0010) [2023-12-27 06:03:28,768][20347] Updated weights for policy 0, policy_version 57044 (0.0009) [2023-12-27 06:03:28,822][20347] Updated weights for policy 0, policy_version 57054 (0.0007) [2023-12-27 06:03:28,876][20347] Updated weights for policy 0, policy_version 57064 (0.0007) [2023-12-27 06:03:29,034][20381] Updated weights for policy 1, policy_version 56972 (0.0011) [2023-12-27 06:03:29,085][20381] Updated weights for policy 1, policy_version 56982 (0.0010) [2023-12-27 06:03:29,140][20381] Updated weights for policy 1, policy_version 56992 (0.0011) [2023-12-27 06:03:29,559][20347] Updated weights for policy 0, policy_version 57074 (0.0008) [2023-12-27 06:03:29,609][20347] Updated weights for policy 0, policy_version 57084 (0.0010) [2023-12-27 06:03:29,673][20347] Updated weights for policy 0, policy_version 57094 (0.0010) [2023-12-27 06:03:29,733][20347] Updated weights for policy 0, policy_version 57104 (0.0010) [2023-12-27 06:03:29,802][20381] Updated weights for policy 1, policy_version 57002 (0.0010) [2023-12-27 06:03:29,864][20381] Updated weights for policy 1, policy_version 57012 (0.0007) [2023-12-27 06:03:29,919][20381] Updated weights for policy 1, policy_version 57022 (0.0008) [2023-12-27 06:03:29,983][20381] Updated weights for policy 1, policy_version 57032 (0.0006) [2023-12-27 06:03:30,424][20347] Updated weights for policy 0, policy_version 57114 (0.0006) [2023-12-27 06:03:30,479][20347] Updated weights for policy 0, policy_version 57124 (0.0005) [2023-12-27 06:03:30,531][20347] Updated weights for policy 0, policy_version 57134 (0.0006) [2023-12-27 06:03:30,634][20381] Updated weights for policy 1, policy_version 57042 (0.0007) [2023-12-27 06:03:30,688][20381] Updated weights for policy 1, policy_version 57052 (0.0007) [2023-12-27 06:03:30,742][20381] Updated weights for policy 1, policy_version 57062 (0.0007) [2023-12-27 06:03:31,190][20347] Updated weights for policy 0, policy_version 57144 (0.0006) [2023-12-27 06:03:31,247][20347] Updated weights for policy 0, policy_version 57154 (0.0007) [2023-12-27 06:03:31,311][20347] Updated weights for policy 0, policy_version 57164 (0.0005) [2023-12-27 06:03:31,477][20381] Updated weights for policy 1, policy_version 57072 (0.0007) [2023-12-27 06:03:31,546][20381] Updated weights for policy 1, policy_version 57082 (0.0005) [2023-12-27 06:03:31,607][20381] Updated weights for policy 1, policy_version 57092 (0.0006) [2023-12-27 06:03:31,972][20347] Updated weights for policy 0, policy_version 57174 (0.0007) [2023-12-27 06:03:32,031][20347] Updated weights for policy 0, policy_version 57184 (0.0007) [2023-12-27 06:03:32,091][20347] Updated weights for policy 0, policy_version 57194 (0.0005) [2023-12-27 06:03:32,224][20381] Updated weights for policy 1, policy_version 57102 (0.0008) [2023-12-27 06:03:32,284][20381] Updated weights for policy 1, policy_version 57112 (0.0007) [2023-12-27 06:03:32,343][20381] Updated weights for policy 1, policy_version 57122 (0.0007) [2023-12-27 06:03:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 29270016. Throughput: 0: 10375.3, 1: 10382.3. Samples: 29284180. Policy #0 lag: (min: 31.0, avg: 46.0, max: 63.0) [2023-12-27 06:03:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:32,749][20347] Updated weights for policy 0, policy_version 57204 (0.0006) [2023-12-27 06:03:32,799][20347] Updated weights for policy 0, policy_version 57214 (0.0006) [2023-12-27 06:03:32,854][20347] Updated weights for policy 0, policy_version 57224 (0.0008) [2023-12-27 06:03:32,982][20381] Updated weights for policy 1, policy_version 57132 (0.0008) [2023-12-27 06:03:33,026][20381] Updated weights for policy 1, policy_version 57142 (0.0008) [2023-12-27 06:03:33,078][20381] Updated weights for policy 1, policy_version 57152 (0.0008) [2023-12-27 06:03:33,461][20347] Updated weights for policy 0, policy_version 57234 (0.0007) [2023-12-27 06:03:33,512][20347] Updated weights for policy 0, policy_version 57244 (0.0007) [2023-12-27 06:03:33,562][20347] Updated weights for policy 0, policy_version 57254 (0.0007) [2023-12-27 06:03:33,607][20347] Updated weights for policy 0, policy_version 57264 (0.0007) [2023-12-27 06:03:33,811][20381] Updated weights for policy 1, policy_version 57162 (0.0008) [2023-12-27 06:03:33,871][20381] Updated weights for policy 1, policy_version 57172 (0.0008) [2023-12-27 06:03:33,928][20381] Updated weights for policy 1, policy_version 57182 (0.0008) [2023-12-27 06:03:33,977][20381] Updated weights for policy 1, policy_version 57192 (0.0008) [2023-12-27 06:03:34,321][20347] Updated weights for policy 0, policy_version 57274 (0.0008) [2023-12-27 06:03:34,371][20347] Updated weights for policy 0, policy_version 57284 (0.0007) [2023-12-27 06:03:34,431][20347] Updated weights for policy 0, policy_version 57294 (0.0007) [2023-12-27 06:03:34,627][20381] Updated weights for policy 1, policy_version 57202 (0.0008) [2023-12-27 06:03:34,680][20381] Updated weights for policy 1, policy_version 57212 (0.0008) [2023-12-27 06:03:34,732][20381] Updated weights for policy 1, policy_version 57222 (0.0007) [2023-12-27 06:03:35,138][20347] Updated weights for policy 0, policy_version 57304 (0.0008) [2023-12-27 06:03:35,186][20347] Updated weights for policy 0, policy_version 57314 (0.0008) [2023-12-27 06:03:35,238][20347] Updated weights for policy 0, policy_version 57324 (0.0009) [2023-12-27 06:03:35,340][20381] Updated weights for policy 1, policy_version 57232 (0.0006) [2023-12-27 06:03:35,397][20381] Updated weights for policy 1, policy_version 57242 (0.0005) [2023-12-27 06:03:35,454][20381] Updated weights for policy 1, policy_version 57252 (0.0005) [2023-12-27 06:03:35,995][20347] Updated weights for policy 0, policy_version 57334 (0.0009) [2023-12-27 06:03:36,026][20381] Updated weights for policy 1, policy_version 57262 (0.0006) [2023-12-27 06:03:36,046][20347] Updated weights for policy 0, policy_version 57344 (0.0006) [2023-12-27 06:03:36,071][20381] Updated weights for policy 1, policy_version 57272 (0.0005) [2023-12-27 06:03:36,099][20347] Updated weights for policy 0, policy_version 57354 (0.0007) [2023-12-27 06:03:36,128][20381] Updated weights for policy 1, policy_version 57282 (0.0007) [2023-12-27 06:03:36,814][20347] Updated weights for policy 0, policy_version 57364 (0.0007) [2023-12-27 06:03:36,836][20381] Updated weights for policy 1, policy_version 57292 (0.0008) [2023-12-27 06:03:36,868][20347] Updated weights for policy 0, policy_version 57374 (0.0007) [2023-12-27 06:03:36,894][20381] Updated weights for policy 1, policy_version 57302 (0.0008) [2023-12-27 06:03:36,926][20347] Updated weights for policy 0, policy_version 57384 (0.0007) [2023-12-27 06:03:36,953][20381] Updated weights for policy 1, policy_version 57312 (0.0007) [2023-12-27 06:03:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 29376512. Throughput: 0: 10351.5, 1: 10374.5. Samples: 29346752. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:03:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000057320_14680064.pth... [2023-12-27 06:03:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000057392_14696448.pth... [2023-12-27 06:03:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000056104_14368768.pth [2023-12-27 06:03:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000056208_14393344.pth [2023-12-27 06:03:37,554][20381] Updated weights for policy 1, policy_version 57322 (0.0007) [2023-12-27 06:03:37,613][20381] Updated weights for policy 1, policy_version 57332 (0.0005) [2023-12-27 06:03:37,663][20347] Updated weights for policy 0, policy_version 57394 (0.0007) [2023-12-27 06:03:37,679][20381] Updated weights for policy 1, policy_version 57342 (0.0007) [2023-12-27 06:03:37,715][20347] Updated weights for policy 0, policy_version 57404 (0.0006) [2023-12-27 06:03:37,741][20381] Updated weights for policy 1, policy_version 57352 (0.0007) [2023-12-27 06:03:37,760][20347] Updated weights for policy 0, policy_version 57414 (0.0008) [2023-12-27 06:03:37,824][20347] Updated weights for policy 0, policy_version 57424 (0.0007) [2023-12-27 06:03:38,395][20381] Updated weights for policy 1, policy_version 57362 (0.0008) [2023-12-27 06:03:38,457][20381] Updated weights for policy 1, policy_version 57372 (0.0008) [2023-12-27 06:03:38,517][20381] Updated weights for policy 1, policy_version 57382 (0.0007) [2023-12-27 06:03:38,519][20347] Updated weights for policy 0, policy_version 57434 (0.0006) [2023-12-27 06:03:38,577][20347] Updated weights for policy 0, policy_version 57444 (0.0007) [2023-12-27 06:03:38,634][20347] Updated weights for policy 0, policy_version 57454 (0.0008) [2023-12-27 06:03:39,186][20381] Updated weights for policy 1, policy_version 57392 (0.0010) [2023-12-27 06:03:39,251][20381] Updated weights for policy 1, policy_version 57402 (0.0011) [2023-12-27 06:03:39,305][20381] Updated weights for policy 1, policy_version 57412 (0.0011) [2023-12-27 06:03:39,352][20347] Updated weights for policy 0, policy_version 57464 (0.0009) [2023-12-27 06:03:39,414][20347] Updated weights for policy 0, policy_version 57474 (0.0007) [2023-12-27 06:03:39,469][20347] Updated weights for policy 0, policy_version 57484 (0.0008) [2023-12-27 06:03:40,080][20381] Updated weights for policy 1, policy_version 57422 (0.0009) [2023-12-27 06:03:40,139][20381] Updated weights for policy 1, policy_version 57432 (0.0007) [2023-12-27 06:03:40,202][20381] Updated weights for policy 1, policy_version 57442 (0.0007) [2023-12-27 06:03:40,217][20347] Updated weights for policy 0, policy_version 57494 (0.0009) [2023-12-27 06:03:40,277][20347] Updated weights for policy 0, policy_version 57504 (0.0008) [2023-12-27 06:03:40,339][20347] Updated weights for policy 0, policy_version 57514 (0.0010) [2023-12-27 06:03:40,922][20381] Updated weights for policy 1, policy_version 57452 (0.0007) [2023-12-27 06:03:40,931][20347] Updated weights for policy 0, policy_version 57524 (0.0007) [2023-12-27 06:03:40,985][20381] Updated weights for policy 1, policy_version 57462 (0.0006) [2023-12-27 06:03:40,993][20347] Updated weights for policy 0, policy_version 57534 (0.0005) [2023-12-27 06:03:41,043][20381] Updated weights for policy 1, policy_version 57472 (0.0006) [2023-12-27 06:03:41,059][20347] Updated weights for policy 0, policy_version 57544 (0.0009) [2023-12-27 06:03:41,759][20347] Updated weights for policy 0, policy_version 57554 (0.0009) [2023-12-27 06:03:41,764][20381] Updated weights for policy 1, policy_version 57482 (0.0007) [2023-12-27 06:03:41,815][20347] Updated weights for policy 0, policy_version 57564 (0.0010) [2023-12-27 06:03:41,831][20381] Updated weights for policy 1, policy_version 57492 (0.0006) [2023-12-27 06:03:41,874][20347] Updated weights for policy 0, policy_version 57574 (0.0010) [2023-12-27 06:03:41,894][20381] Updated weights for policy 1, policy_version 57502 (0.0006) [2023-12-27 06:03:41,936][20347] Updated weights for policy 0, policy_version 57584 (0.0010) [2023-12-27 06:03:41,953][20381] Updated weights for policy 1, policy_version 57512 (0.0009) [2023-12-27 06:03:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 29474816. Throughput: 0: 10300.3, 1: 10400.9. Samples: 29467508. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:03:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:42,635][20347] Updated weights for policy 0, policy_version 57594 (0.0011) [2023-12-27 06:03:42,652][20381] Updated weights for policy 1, policy_version 57522 (0.0005) [2023-12-27 06:03:42,698][20347] Updated weights for policy 0, policy_version 57604 (0.0011) [2023-12-27 06:03:42,711][20381] Updated weights for policy 1, policy_version 57532 (0.0005) [2023-12-27 06:03:42,758][20347] Updated weights for policy 0, policy_version 57614 (0.0010) [2023-12-27 06:03:42,774][20381] Updated weights for policy 1, policy_version 57542 (0.0007) [2023-12-27 06:03:43,413][20347] Updated weights for policy 0, policy_version 57624 (0.0008) [2023-12-27 06:03:43,465][20381] Updated weights for policy 1, policy_version 57552 (0.0009) [2023-12-27 06:03:43,471][20347] Updated weights for policy 0, policy_version 57634 (0.0009) [2023-12-27 06:03:43,517][20381] Updated weights for policy 1, policy_version 57562 (0.0006) [2023-12-27 06:03:43,526][20347] Updated weights for policy 0, policy_version 57644 (0.0006) [2023-12-27 06:03:43,571][20381] Updated weights for policy 1, policy_version 57572 (0.0006) [2023-12-27 06:03:44,207][20347] Updated weights for policy 0, policy_version 57654 (0.0007) [2023-12-27 06:03:44,233][20381] Updated weights for policy 1, policy_version 57582 (0.0008) [2023-12-27 06:03:44,261][20347] Updated weights for policy 0, policy_version 57664 (0.0007) [2023-12-27 06:03:44,279][20381] Updated weights for policy 1, policy_version 57592 (0.0006) [2023-12-27 06:03:44,321][20347] Updated weights for policy 0, policy_version 57674 (0.0007) [2023-12-27 06:03:44,344][20381] Updated weights for policy 1, policy_version 57602 (0.0006) [2023-12-27 06:03:45,008][20347] Updated weights for policy 0, policy_version 57684 (0.0007) [2023-12-27 06:03:45,049][20381] Updated weights for policy 1, policy_version 57612 (0.0007) [2023-12-27 06:03:45,068][20347] Updated weights for policy 0, policy_version 57694 (0.0010) [2023-12-27 06:03:45,106][20381] Updated weights for policy 1, policy_version 57622 (0.0006) [2023-12-27 06:03:45,123][20347] Updated weights for policy 0, policy_version 57704 (0.0008) [2023-12-27 06:03:45,164][20381] Updated weights for policy 1, policy_version 57632 (0.0008) [2023-12-27 06:03:45,735][20347] Updated weights for policy 0, policy_version 57714 (0.0008) [2023-12-27 06:03:45,792][20347] Updated weights for policy 0, policy_version 57724 (0.0007) [2023-12-27 06:03:45,797][20381] Updated weights for policy 1, policy_version 57642 (0.0008) [2023-12-27 06:03:45,842][20381] Updated weights for policy 1, policy_version 57652 (0.0006) [2023-12-27 06:03:45,844][20347] Updated weights for policy 0, policy_version 57734 (0.0006) [2023-12-27 06:03:45,894][20381] Updated weights for policy 1, policy_version 57662 (0.0005) [2023-12-27 06:03:45,896][20347] Updated weights for policy 0, policy_version 57744 (0.0006) [2023-12-27 06:03:45,941][20381] Updated weights for policy 1, policy_version 57672 (0.0008) [2023-12-27 06:03:46,520][20347] Updated weights for policy 0, policy_version 57754 (0.0007) [2023-12-27 06:03:46,567][20347] Updated weights for policy 0, policy_version 57764 (0.0007) [2023-12-27 06:03:46,623][20347] Updated weights for policy 0, policy_version 57774 (0.0007) [2023-12-27 06:03:46,629][20381] Updated weights for policy 1, policy_version 57682 (0.0007) [2023-12-27 06:03:46,689][20381] Updated weights for policy 1, policy_version 57692 (0.0007) [2023-12-27 06:03:46,740][20381] Updated weights for policy 1, policy_version 57702 (0.0007) [2023-12-27 06:03:47,285][20347] Updated weights for policy 0, policy_version 57784 (0.0007) [2023-12-27 06:03:47,341][20347] Updated weights for policy 0, policy_version 57794 (0.0007) [2023-12-27 06:03:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 29573120. Throughput: 0: 10254.8, 1: 10404.9. Samples: 29590172. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:03:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:47,404][20347] Updated weights for policy 0, policy_version 57804 (0.0007) [2023-12-27 06:03:47,432][20381] Updated weights for policy 1, policy_version 57712 (0.0007) [2023-12-27 06:03:47,500][20381] Updated weights for policy 1, policy_version 57722 (0.0007) [2023-12-27 06:03:47,557][20381] Updated weights for policy 1, policy_version 57732 (0.0007) [2023-12-27 06:03:47,986][20347] Updated weights for policy 0, policy_version 57814 (0.0007) [2023-12-27 06:03:48,046][20347] Updated weights for policy 0, policy_version 57824 (0.0008) [2023-12-27 06:03:48,104][20347] Updated weights for policy 0, policy_version 57834 (0.0007) [2023-12-27 06:03:48,242][20381] Updated weights for policy 1, policy_version 57742 (0.0007) [2023-12-27 06:03:48,300][20381] Updated weights for policy 1, policy_version 57752 (0.0007) [2023-12-27 06:03:48,363][20381] Updated weights for policy 1, policy_version 57762 (0.0008) [2023-12-27 06:03:48,763][20347] Updated weights for policy 0, policy_version 57844 (0.0007) [2023-12-27 06:03:48,824][20347] Updated weights for policy 0, policy_version 57854 (0.0007) [2023-12-27 06:03:48,884][20347] Updated weights for policy 0, policy_version 57864 (0.0008) [2023-12-27 06:03:49,070][20381] Updated weights for policy 1, policy_version 57772 (0.0008) [2023-12-27 06:03:49,130][20381] Updated weights for policy 1, policy_version 57782 (0.0008) [2023-12-27 06:03:49,196][20381] Updated weights for policy 1, policy_version 57792 (0.0008) [2023-12-27 06:03:49,538][20347] Updated weights for policy 0, policy_version 57874 (0.0008) [2023-12-27 06:03:49,594][20347] Updated weights for policy 0, policy_version 57884 (0.0008) [2023-12-27 06:03:49,659][20347] Updated weights for policy 0, policy_version 57894 (0.0007) [2023-12-27 06:03:49,716][20347] Updated weights for policy 0, policy_version 57904 (0.0005) [2023-12-27 06:03:49,925][20381] Updated weights for policy 1, policy_version 57802 (0.0008) [2023-12-27 06:03:49,994][20381] Updated weights for policy 1, policy_version 57812 (0.0008) [2023-12-27 06:03:50,052][20381] Updated weights for policy 1, policy_version 57822 (0.0008) [2023-12-27 06:03:50,111][20381] Updated weights for policy 1, policy_version 57832 (0.0007) [2023-12-27 06:03:50,399][20347] Updated weights for policy 0, policy_version 57914 (0.0008) [2023-12-27 06:03:50,454][20347] Updated weights for policy 0, policy_version 57924 (0.0008) [2023-12-27 06:03:50,499][20347] Updated weights for policy 0, policy_version 57934 (0.0008) [2023-12-27 06:03:50,797][20381] Updated weights for policy 1, policy_version 57842 (0.0008) [2023-12-27 06:03:50,857][20381] Updated weights for policy 1, policy_version 57852 (0.0008) [2023-12-27 06:03:50,914][20381] Updated weights for policy 1, policy_version 57862 (0.0007) [2023-12-27 06:03:51,176][20347] Updated weights for policy 0, policy_version 57944 (0.0008) [2023-12-27 06:03:51,237][20347] Updated weights for policy 0, policy_version 57954 (0.0009) [2023-12-27 06:03:51,295][20347] Updated weights for policy 0, policy_version 57964 (0.0007) [2023-12-27 06:03:51,633][20381] Updated weights for policy 1, policy_version 57872 (0.0008) [2023-12-27 06:03:51,696][20381] Updated weights for policy 1, policy_version 57882 (0.0007) [2023-12-27 06:03:51,763][20381] Updated weights for policy 1, policy_version 57892 (0.0007) [2023-12-27 06:03:52,013][20347] Updated weights for policy 0, policy_version 57974 (0.0008) [2023-12-27 06:03:52,076][20347] Updated weights for policy 0, policy_version 57984 (0.0005) [2023-12-27 06:03:52,137][20347] Updated weights for policy 0, policy_version 57994 (0.0006) [2023-12-27 06:03:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 29679616. Throughput: 0: 10232.1, 1: 10357.4. Samples: 29650936. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000058000_14852096.pth... [2023-12-27 06:03:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000056784_14540800.pth [2023-12-27 06:03:52,386][20381] Updated weights for policy 1, policy_version 57902 (0.0008) [2023-12-27 06:03:52,445][20381] Updated weights for policy 1, policy_version 57912 (0.0007) [2023-12-27 06:03:52,500][20381] Updated weights for policy 1, policy_version 57922 (0.0006) [2023-12-27 06:03:52,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000057928_14835712.pth... [2023-12-27 06:03:52,537][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000056712_14524416.pth [2023-12-27 06:03:52,822][20347] Updated weights for policy 0, policy_version 58004 (0.0008) [2023-12-27 06:03:52,869][20347] Updated weights for policy 0, policy_version 58014 (0.0008) [2023-12-27 06:03:52,921][20347] Updated weights for policy 0, policy_version 58024 (0.0007) [2023-12-27 06:03:53,144][20381] Updated weights for policy 1, policy_version 57932 (0.0007) [2023-12-27 06:03:53,199][20381] Updated weights for policy 1, policy_version 57942 (0.0005) [2023-12-27 06:03:53,248][20381] Updated weights for policy 1, policy_version 57952 (0.0005) [2023-12-27 06:03:53,597][20347] Updated weights for policy 0, policy_version 58034 (0.0007) [2023-12-27 06:03:53,659][20347] Updated weights for policy 0, policy_version 58044 (0.0005) [2023-12-27 06:03:53,715][20347] Updated weights for policy 0, policy_version 58054 (0.0006) [2023-12-27 06:03:53,760][20347] Updated weights for policy 0, policy_version 58064 (0.0008) [2023-12-27 06:03:53,918][20381] Updated weights for policy 1, policy_version 57962 (0.0007) [2023-12-27 06:03:53,969][20381] Updated weights for policy 1, policy_version 57972 (0.0007) [2023-12-27 06:03:54,018][20381] Updated weights for policy 1, policy_version 57982 (0.0010) [2023-12-27 06:03:54,062][20381] Updated weights for policy 1, policy_version 57992 (0.0008) [2023-12-27 06:03:54,298][20347] Updated weights for policy 0, policy_version 58074 (0.0008) [2023-12-27 06:03:54,351][20347] Updated weights for policy 0, policy_version 58084 (0.0007) [2023-12-27 06:03:54,416][20347] Updated weights for policy 0, policy_version 58094 (0.0005) [2023-12-27 06:03:54,818][20381] Updated weights for policy 1, policy_version 58002 (0.0007) [2023-12-27 06:03:54,879][20381] Updated weights for policy 1, policy_version 58012 (0.0008) [2023-12-27 06:03:54,945][20381] Updated weights for policy 1, policy_version 58022 (0.0008) [2023-12-27 06:03:55,002][20347] Updated weights for policy 0, policy_version 58104 (0.0007) [2023-12-27 06:03:55,062][20347] Updated weights for policy 0, policy_version 58114 (0.0008) [2023-12-27 06:03:55,121][20347] Updated weights for policy 0, policy_version 58124 (0.0008) [2023-12-27 06:03:55,653][20381] Updated weights for policy 1, policy_version 58032 (0.0009) [2023-12-27 06:03:55,705][20381] Updated weights for policy 1, policy_version 58042 (0.0008) [2023-12-27 06:03:55,741][20347] Updated weights for policy 0, policy_version 58134 (0.0006) [2023-12-27 06:03:55,757][20381] Updated weights for policy 1, policy_version 58052 (0.0008) [2023-12-27 06:03:55,793][20347] Updated weights for policy 0, policy_version 58144 (0.0006) [2023-12-27 06:03:55,845][20347] Updated weights for policy 0, policy_version 58154 (0.0010) [2023-12-27 06:03:56,424][20347] Updated weights for policy 0, policy_version 58164 (0.0010) [2023-12-27 06:03:56,481][20347] Updated weights for policy 0, policy_version 58174 (0.0007) [2023-12-27 06:03:56,489][20381] Updated weights for policy 1, policy_version 58062 (0.0008) [2023-12-27 06:03:56,526][20347] Updated weights for policy 0, policy_version 58184 (0.0007) [2023-12-27 06:03:56,547][20381] Updated weights for policy 1, policy_version 58072 (0.0008) [2023-12-27 06:03:56,603][20381] Updated weights for policy 1, policy_version 58082 (0.0007) [2023-12-27 06:03:57,152][20347] Updated weights for policy 0, policy_version 58194 (0.0005) [2023-12-27 06:03:57,203][20347] Updated weights for policy 0, policy_version 58204 (0.0005) [2023-12-27 06:03:57,249][20347] Updated weights for policy 0, policy_version 58214 (0.0005) [2023-12-27 06:03:57,297][20347] Updated weights for policy 0, policy_version 58224 (0.0005) [2023-12-27 06:03:57,320][20381] Updated weights for policy 1, policy_version 58092 (0.0007) [2023-12-27 06:03:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 29786112. Throughput: 0: 10280.9, 1: 10356.6. Samples: 29775492. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:03:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:03:57,366][20381] Updated weights for policy 1, policy_version 58102 (0.0007) [2023-12-27 06:03:57,419][20381] Updated weights for policy 1, policy_version 58112 (0.0008) [2023-12-27 06:03:57,876][20347] Updated weights for policy 0, policy_version 58234 (0.0010) [2023-12-27 06:03:57,923][20347] Updated weights for policy 0, policy_version 58244 (0.0010) [2023-12-27 06:03:57,976][20347] Updated weights for policy 0, policy_version 58254 (0.0011) [2023-12-27 06:03:58,161][20381] Updated weights for policy 1, policy_version 58122 (0.0009) [2023-12-27 06:03:58,220][20381] Updated weights for policy 1, policy_version 58132 (0.0008) [2023-12-27 06:03:58,282][20381] Updated weights for policy 1, policy_version 58142 (0.0008) [2023-12-27 06:03:58,348][20381] Updated weights for policy 1, policy_version 58152 (0.0008) [2023-12-27 06:03:58,719][20347] Updated weights for policy 0, policy_version 58264 (0.0008) [2023-12-27 06:03:58,802][20347] Updated weights for policy 0, policy_version 58274 (0.0008) [2023-12-27 06:03:58,866][20347] Updated weights for policy 0, policy_version 58284 (0.0008) [2023-12-27 06:03:59,098][20381] Updated weights for policy 1, policy_version 58162 (0.0005) [2023-12-27 06:03:59,152][20381] Updated weights for policy 1, policy_version 58172 (0.0005) [2023-12-27 06:03:59,205][20381] Updated weights for policy 1, policy_version 58182 (0.0005) [2023-12-27 06:03:59,600][20347] Updated weights for policy 0, policy_version 58294 (0.0010) [2023-12-27 06:03:59,650][20347] Updated weights for policy 0, policy_version 58304 (0.0011) [2023-12-27 06:03:59,707][20347] Updated weights for policy 0, policy_version 58314 (0.0008) [2023-12-27 06:03:59,878][20381] Updated weights for policy 1, policy_version 58192 (0.0008) [2023-12-27 06:03:59,960][20381] Updated weights for policy 1, policy_version 58202 (0.0007) [2023-12-27 06:04:00,024][20381] Updated weights for policy 1, policy_version 58212 (0.0007) [2023-12-27 06:04:00,419][20347] Updated weights for policy 0, policy_version 58324 (0.0007) [2023-12-27 06:04:00,479][20347] Updated weights for policy 0, policy_version 58334 (0.0007) [2023-12-27 06:04:00,545][20347] Updated weights for policy 0, policy_version 58344 (0.0007) [2023-12-27 06:04:00,615][20381] Updated weights for policy 1, policy_version 58222 (0.0007) [2023-12-27 06:04:00,664][20381] Updated weights for policy 1, policy_version 58232 (0.0008) [2023-12-27 06:04:00,711][20381] Updated weights for policy 1, policy_version 58242 (0.0007) [2023-12-27 06:04:01,186][20347] Updated weights for policy 0, policy_version 58354 (0.0007) [2023-12-27 06:04:01,235][20347] Updated weights for policy 0, policy_version 58364 (0.0008) [2023-12-27 06:04:01,295][20347] Updated weights for policy 0, policy_version 58374 (0.0008) [2023-12-27 06:04:01,349][20347] Updated weights for policy 0, policy_version 58384 (0.0007) [2023-12-27 06:04:01,434][20381] Updated weights for policy 1, policy_version 58252 (0.0007) [2023-12-27 06:04:01,494][20381] Updated weights for policy 1, policy_version 58262 (0.0008) [2023-12-27 06:04:01,553][20381] Updated weights for policy 1, policy_version 58272 (0.0009) [2023-12-27 06:04:02,028][20347] Updated weights for policy 0, policy_version 58394 (0.0006) [2023-12-27 06:04:02,083][20347] Updated weights for policy 0, policy_version 58404 (0.0009) [2023-12-27 06:04:02,135][20347] Updated weights for policy 0, policy_version 58414 (0.0008) [2023-12-27 06:04:02,204][20381] Updated weights for policy 1, policy_version 58282 (0.0008) [2023-12-27 06:04:02,256][20381] Updated weights for policy 1, policy_version 58292 (0.0008) [2023-12-27 06:04:02,312][20381] Updated weights for policy 1, policy_version 58302 (0.0007) [2023-12-27 06:04:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 29884416. Throughput: 0: 10326.2, 1: 10280.6. Samples: 29897744. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:04:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:02,368][20381] Updated weights for policy 1, policy_version 58312 (0.0007) [2023-12-27 06:04:02,832][20347] Updated weights for policy 0, policy_version 58424 (0.0010) [2023-12-27 06:04:02,879][20347] Updated weights for policy 0, policy_version 58434 (0.0010) [2023-12-27 06:04:02,930][20347] Updated weights for policy 0, policy_version 58444 (0.0010) [2023-12-27 06:04:03,052][20381] Updated weights for policy 1, policy_version 58322 (0.0008) [2023-12-27 06:04:03,095][20381] Updated weights for policy 1, policy_version 58332 (0.0007) [2023-12-27 06:04:03,145][20381] Updated weights for policy 1, policy_version 58342 (0.0007) [2023-12-27 06:04:03,624][20347] Updated weights for policy 0, policy_version 58454 (0.0010) [2023-12-27 06:04:03,677][20347] Updated weights for policy 0, policy_version 58464 (0.0010) [2023-12-27 06:04:03,730][20347] Updated weights for policy 0, policy_version 58474 (0.0010) [2023-12-27 06:04:03,851][20381] Updated weights for policy 1, policy_version 58352 (0.0008) [2023-12-27 06:04:03,914][20381] Updated weights for policy 1, policy_version 58362 (0.0008) [2023-12-27 06:04:03,974][20381] Updated weights for policy 1, policy_version 58372 (0.0008) [2023-12-27 06:04:04,364][20347] Updated weights for policy 0, policy_version 58484 (0.0010) [2023-12-27 06:04:04,423][20347] Updated weights for policy 0, policy_version 58494 (0.0010) [2023-12-27 06:04:04,481][20347] Updated weights for policy 0, policy_version 58504 (0.0008) [2023-12-27 06:04:04,540][20381] Updated weights for policy 1, policy_version 58382 (0.0006) [2023-12-27 06:04:04,605][20381] Updated weights for policy 1, policy_version 58392 (0.0005) [2023-12-27 06:04:04,652][20381] Updated weights for policy 1, policy_version 58402 (0.0007) [2023-12-27 06:04:05,117][20347] Updated weights for policy 0, policy_version 58514 (0.0005) [2023-12-27 06:04:05,170][20347] Updated weights for policy 0, policy_version 58524 (0.0005) [2023-12-27 06:04:05,216][20347] Updated weights for policy 0, policy_version 58534 (0.0005) [2023-12-27 06:04:05,258][20381] Updated weights for policy 1, policy_version 58412 (0.0006) [2023-12-27 06:04:05,275][20347] Updated weights for policy 0, policy_version 58544 (0.0010) [2023-12-27 06:04:05,310][20381] Updated weights for policy 1, policy_version 58422 (0.0006) [2023-12-27 06:04:05,370][20381] Updated weights for policy 1, policy_version 58432 (0.0006) [2023-12-27 06:04:05,813][20347] Updated weights for policy 0, policy_version 58554 (0.0010) [2023-12-27 06:04:05,857][20347] Updated weights for policy 0, policy_version 58564 (0.0010) [2023-12-27 06:04:05,908][20347] Updated weights for policy 0, policy_version 58574 (0.0010) [2023-12-27 06:04:05,984][20381] Updated weights for policy 1, policy_version 58442 (0.0006) [2023-12-27 06:04:06,038][20381] Updated weights for policy 1, policy_version 58452 (0.0007) [2023-12-27 06:04:06,088][20381] Updated weights for policy 1, policy_version 58462 (0.0007) [2023-12-27 06:04:06,165][20381] Updated weights for policy 1, policy_version 58472 (0.0007) [2023-12-27 06:04:06,617][20347] Updated weights for policy 0, policy_version 58584 (0.0010) [2023-12-27 06:04:06,676][20347] Updated weights for policy 0, policy_version 58594 (0.0010) [2023-12-27 06:04:06,725][20347] Updated weights for policy 0, policy_version 58604 (0.0010) [2023-12-27 06:04:06,858][20381] Updated weights for policy 1, policy_version 58482 (0.0008) [2023-12-27 06:04:06,921][20381] Updated weights for policy 1, policy_version 58492 (0.0008) [2023-12-27 06:04:06,987][20381] Updated weights for policy 1, policy_version 58502 (0.0008) [2023-12-27 06:04:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 29990912. Throughput: 0: 10346.2, 1: 10269.3. Samples: 29961272. Policy #0 lag: (min: 13.0, avg: 19.2, max: 45.0) [2023-12-27 06:04:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000058504_14983168.pth... [2023-12-27 06:04:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000057320_14680064.pth [2023-12-27 06:04:07,409][20347] Updated weights for policy 0, policy_version 58614 (0.0010) [2023-12-27 06:04:07,470][20347] Updated weights for policy 0, policy_version 58624 (0.0010) [2023-12-27 06:04:07,518][20347] Updated weights for policy 0, policy_version 58634 (0.0010) [2023-12-27 06:04:07,552][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000058640_15015936.pth... [2023-12-27 06:04:07,555][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000057392_14696448.pth [2023-12-27 06:04:07,624][20381] Updated weights for policy 1, policy_version 58512 (0.0007) [2023-12-27 06:04:07,694][20381] Updated weights for policy 1, policy_version 58522 (0.0007) [2023-12-27 06:04:07,751][20381] Updated weights for policy 1, policy_version 58532 (0.0006) [2023-12-27 06:04:08,191][20347] Updated weights for policy 0, policy_version 58644 (0.0009) [2023-12-27 06:04:08,246][20347] Updated weights for policy 0, policy_version 58654 (0.0008) [2023-12-27 06:04:08,304][20347] Updated weights for policy 0, policy_version 58664 (0.0008) [2023-12-27 06:04:08,381][20381] Updated weights for policy 1, policy_version 58542 (0.0007) [2023-12-27 06:04:08,435][20381] Updated weights for policy 1, policy_version 58552 (0.0008) [2023-12-27 06:04:08,501][20381] Updated weights for policy 1, policy_version 58562 (0.0007) [2023-12-27 06:04:08,999][20347] Updated weights for policy 0, policy_version 58674 (0.0008) [2023-12-27 06:04:09,061][20347] Updated weights for policy 0, policy_version 58684 (0.0006) [2023-12-27 06:04:09,116][20347] Updated weights for policy 0, policy_version 58694 (0.0006) [2023-12-27 06:04:09,116][20381] Updated weights for policy 1, policy_version 58572 (0.0007) [2023-12-27 06:04:09,164][20381] Updated weights for policy 1, policy_version 58582 (0.0008) [2023-12-27 06:04:09,182][20347] Updated weights for policy 0, policy_version 58704 (0.0007) [2023-12-27 06:04:09,218][20381] Updated weights for policy 1, policy_version 58592 (0.0007) [2023-12-27 06:04:09,789][20347] Updated weights for policy 0, policy_version 58714 (0.0006) [2023-12-27 06:04:09,855][20347] Updated weights for policy 0, policy_version 58724 (0.0007) [2023-12-27 06:04:09,917][20347] Updated weights for policy 0, policy_version 58734 (0.0008) [2023-12-27 06:04:09,949][20381] Updated weights for policy 1, policy_version 58602 (0.0008) [2023-12-27 06:04:10,008][20381] Updated weights for policy 1, policy_version 58612 (0.0010) [2023-12-27 06:04:10,064][20381] Updated weights for policy 1, policy_version 58622 (0.0010) [2023-12-27 06:04:10,110][20381] Updated weights for policy 1, policy_version 58632 (0.0008) [2023-12-27 06:04:10,597][20347] Updated weights for policy 0, policy_version 58744 (0.0007) [2023-12-27 06:04:10,652][20347] Updated weights for policy 0, policy_version 58754 (0.0008) [2023-12-27 06:04:10,696][20381] Updated weights for policy 1, policy_version 58642 (0.0011) [2023-12-27 06:04:10,705][20347] Updated weights for policy 0, policy_version 58764 (0.0006) [2023-12-27 06:04:10,753][20381] Updated weights for policy 1, policy_version 58652 (0.0011) [2023-12-27 06:04:10,814][20381] Updated weights for policy 1, policy_version 58662 (0.0011) [2023-12-27 06:04:11,463][20347] Updated weights for policy 0, policy_version 58774 (0.0007) [2023-12-27 06:04:11,529][20347] Updated weights for policy 0, policy_version 58784 (0.0012) [2023-12-27 06:04:11,543][20381] Updated weights for policy 1, policy_version 58672 (0.0008) [2023-12-27 06:04:11,590][20347] Updated weights for policy 0, policy_version 58794 (0.0007) [2023-12-27 06:04:11,600][20381] Updated weights for policy 1, policy_version 58682 (0.0006) [2023-12-27 06:04:11,663][20381] Updated weights for policy 1, policy_version 58692 (0.0007) [2023-12-27 06:04:12,287][20381] Updated weights for policy 1, policy_version 58702 (0.0007) [2023-12-27 06:04:12,300][20347] Updated weights for policy 0, policy_version 58804 (0.0009) [2023-12-27 06:04:12,350][20381] Updated weights for policy 1, policy_version 58712 (0.0007) [2023-12-27 06:04:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 30089216. Throughput: 0: 10364.6, 1: 10209.8. Samples: 30086660. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:12,368][20347] Updated weights for policy 0, policy_version 58814 (0.0008) [2023-12-27 06:04:12,412][20381] Updated weights for policy 1, policy_version 58722 (0.0007) [2023-12-27 06:04:12,428][20347] Updated weights for policy 0, policy_version 58824 (0.0010) [2023-12-27 06:04:13,051][20381] Updated weights for policy 1, policy_version 58732 (0.0006) [2023-12-27 06:04:13,105][20381] Updated weights for policy 1, policy_version 58742 (0.0008) [2023-12-27 06:04:13,106][20347] Updated weights for policy 0, policy_version 58834 (0.0011) [2023-12-27 06:04:13,157][20381] Updated weights for policy 1, policy_version 58752 (0.0010) [2023-12-27 06:04:13,158][20347] Updated weights for policy 0, policy_version 58844 (0.0010) [2023-12-27 06:04:13,209][20347] Updated weights for policy 0, policy_version 58854 (0.0010) [2023-12-27 06:04:13,260][20347] Updated weights for policy 0, policy_version 58864 (0.0010) [2023-12-27 06:04:13,857][20381] Updated weights for policy 1, policy_version 58762 (0.0007) [2023-12-27 06:04:13,916][20381] Updated weights for policy 1, policy_version 58772 (0.0006) [2023-12-27 06:04:13,927][20347] Updated weights for policy 0, policy_version 58874 (0.0009) [2023-12-27 06:04:13,971][20381] Updated weights for policy 1, policy_version 58782 (0.0010) [2023-12-27 06:04:13,991][20347] Updated weights for policy 0, policy_version 58884 (0.0007) [2023-12-27 06:04:14,031][20381] Updated weights for policy 1, policy_version 58792 (0.0010) [2023-12-27 06:04:14,042][20347] Updated weights for policy 0, policy_version 58894 (0.0010) [2023-12-27 06:04:14,699][20381] Updated weights for policy 1, policy_version 58802 (0.0008) [2023-12-27 06:04:14,735][20347] Updated weights for policy 0, policy_version 58904 (0.0010) [2023-12-27 06:04:14,757][20381] Updated weights for policy 1, policy_version 58812 (0.0007) [2023-12-27 06:04:14,793][20347] Updated weights for policy 0, policy_version 58914 (0.0009) [2023-12-27 06:04:14,816][20381] Updated weights for policy 1, policy_version 58822 (0.0009) [2023-12-27 06:04:14,851][20347] Updated weights for policy 0, policy_version 58924 (0.0007) [2023-12-27 06:04:15,501][20381] Updated weights for policy 1, policy_version 58832 (0.0010) [2023-12-27 06:04:15,515][20347] Updated weights for policy 0, policy_version 58934 (0.0008) [2023-12-27 06:04:15,552][20381] Updated weights for policy 1, policy_version 58842 (0.0010) [2023-12-27 06:04:15,575][20347] Updated weights for policy 0, policy_version 58944 (0.0006) [2023-12-27 06:04:15,607][20381] Updated weights for policy 1, policy_version 58852 (0.0010) [2023-12-27 06:04:15,636][20347] Updated weights for policy 0, policy_version 58954 (0.0009) [2023-12-27 06:04:16,213][20347] Updated weights for policy 0, policy_version 58964 (0.0007) [2023-12-27 06:04:16,269][20347] Updated weights for policy 0, policy_version 58974 (0.0007) [2023-12-27 06:04:16,285][20381] Updated weights for policy 1, policy_version 58862 (0.0008) [2023-12-27 06:04:16,321][20347] Updated weights for policy 0, policy_version 58984 (0.0005) [2023-12-27 06:04:16,336][20381] Updated weights for policy 1, policy_version 58872 (0.0007) [2023-12-27 06:04:16,385][20381] Updated weights for policy 1, policy_version 58882 (0.0007) [2023-12-27 06:04:16,966][20381] Updated weights for policy 1, policy_version 58892 (0.0008) [2023-12-27 06:04:17,021][20381] Updated weights for policy 1, policy_version 58902 (0.0008) [2023-12-27 06:04:17,045][20347] Updated weights for policy 0, policy_version 58994 (0.0005) [2023-12-27 06:04:17,076][20381] Updated weights for policy 1, policy_version 58912 (0.0010) [2023-12-27 06:04:17,100][20347] Updated weights for policy 0, policy_version 59004 (0.0005) [2023-12-27 06:04:17,159][20347] Updated weights for policy 0, policy_version 59014 (0.0007) [2023-12-27 06:04:17,206][20347] Updated weights for policy 0, policy_version 59024 (0.0007) [2023-12-27 06:04:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 30203904. Throughput: 0: 10348.6, 1: 10231.9. Samples: 30210304. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:17,717][20381] Updated weights for policy 1, policy_version 58922 (0.0010) [2023-12-27 06:04:17,775][20381] Updated weights for policy 1, policy_version 58932 (0.0010) [2023-12-27 06:04:17,823][20381] Updated weights for policy 1, policy_version 58942 (0.0010) [2023-12-27 06:04:17,876][20381] Updated weights for policy 1, policy_version 58952 (0.0010) [2023-12-27 06:04:17,920][20347] Updated weights for policy 0, policy_version 59034 (0.0008) [2023-12-27 06:04:17,992][20347] Updated weights for policy 0, policy_version 59044 (0.0008) [2023-12-27 06:04:18,050][20347] Updated weights for policy 0, policy_version 59054 (0.0007) [2023-12-27 06:04:18,560][20381] Updated weights for policy 1, policy_version 58962 (0.0010) [2023-12-27 06:04:18,621][20381] Updated weights for policy 1, policy_version 58972 (0.0010) [2023-12-27 06:04:18,681][20381] Updated weights for policy 1, policy_version 58982 (0.0010) [2023-12-27 06:04:18,703][20347] Updated weights for policy 0, policy_version 59064 (0.0008) [2023-12-27 06:04:18,762][20347] Updated weights for policy 0, policy_version 59074 (0.0008) [2023-12-27 06:04:18,815][20347] Updated weights for policy 0, policy_version 59084 (0.0007) [2023-12-27 06:04:19,389][20381] Updated weights for policy 1, policy_version 58992 (0.0009) [2023-12-27 06:04:19,448][20381] Updated weights for policy 1, policy_version 59002 (0.0007) [2023-12-27 06:04:19,512][20381] Updated weights for policy 1, policy_version 59012 (0.0007) [2023-12-27 06:04:19,562][20347] Updated weights for policy 0, policy_version 59094 (0.0007) [2023-12-27 06:04:19,617][20347] Updated weights for policy 0, policy_version 59104 (0.0007) [2023-12-27 06:04:19,674][20347] Updated weights for policy 0, policy_version 59114 (0.0008) [2023-12-27 06:04:20,188][20381] Updated weights for policy 1, policy_version 59022 (0.0008) [2023-12-27 06:04:20,239][20381] Updated weights for policy 1, policy_version 59032 (0.0008) [2023-12-27 06:04:20,295][20381] Updated weights for policy 1, policy_version 59042 (0.0007) [2023-12-27 06:04:20,426][20347] Updated weights for policy 0, policy_version 59124 (0.0007) [2023-12-27 06:04:20,481][20347] Updated weights for policy 0, policy_version 59134 (0.0008) [2023-12-27 06:04:20,531][20347] Updated weights for policy 0, policy_version 59144 (0.0008) [2023-12-27 06:04:20,979][20381] Updated weights for policy 1, policy_version 59052 (0.0008) [2023-12-27 06:04:21,042][20381] Updated weights for policy 1, policy_version 59062 (0.0008) [2023-12-27 06:04:21,100][20381] Updated weights for policy 1, policy_version 59072 (0.0008) [2023-12-27 06:04:21,210][20347] Updated weights for policy 0, policy_version 59154 (0.0008) [2023-12-27 06:04:21,275][20347] Updated weights for policy 0, policy_version 59164 (0.0011) [2023-12-27 06:04:21,337][20347] Updated weights for policy 0, policy_version 59174 (0.0011) [2023-12-27 06:04:21,404][20347] Updated weights for policy 0, policy_version 59184 (0.0009) [2023-12-27 06:04:21,904][20381] Updated weights for policy 1, policy_version 59082 (0.0008) [2023-12-27 06:04:21,967][20381] Updated weights for policy 1, policy_version 59092 (0.0007) [2023-12-27 06:04:22,029][20381] Updated weights for policy 1, policy_version 59102 (0.0008) [2023-12-27 06:04:22,089][20381] Updated weights for policy 1, policy_version 59112 (0.0008) [2023-12-27 06:04:22,107][20347] Updated weights for policy 0, policy_version 59194 (0.0006) [2023-12-27 06:04:22,173][20347] Updated weights for policy 0, policy_version 59204 (0.0005) [2023-12-27 06:04:22,243][20347] Updated weights for policy 0, policy_version 59214 (0.0005) [2023-12-27 06:04:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 30302208. Throughput: 0: 10302.9, 1: 10220.5. Samples: 30270304. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000059112_15138816.pth... [2023-12-27 06:04:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000059216_15163392.pth... [2023-12-27 06:04:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000057928_14835712.pth [2023-12-27 06:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000058000_14852096.pth [2023-12-27 06:04:22,803][20381] Updated weights for policy 1, policy_version 59122 (0.0008) [2023-12-27 06:04:22,852][20347] Updated weights for policy 0, policy_version 59224 (0.0007) [2023-12-27 06:04:22,868][20381] Updated weights for policy 1, policy_version 59132 (0.0006) [2023-12-27 06:04:22,908][20347] Updated weights for policy 0, policy_version 59234 (0.0006) [2023-12-27 06:04:22,933][20381] Updated weights for policy 1, policy_version 59142 (0.0007) [2023-12-27 06:04:22,967][20347] Updated weights for policy 0, policy_version 59244 (0.0007) [2023-12-27 06:04:23,608][20347] Updated weights for policy 0, policy_version 59254 (0.0006) [2023-12-27 06:04:23,618][20381] Updated weights for policy 1, policy_version 59152 (0.0006) [2023-12-27 06:04:23,661][20347] Updated weights for policy 0, policy_version 59264 (0.0005) [2023-12-27 06:04:23,663][20381] Updated weights for policy 1, policy_version 59162 (0.0007) [2023-12-27 06:04:23,711][20381] Updated weights for policy 1, policy_version 59172 (0.0005) [2023-12-27 06:04:23,716][20347] Updated weights for policy 0, policy_version 59274 (0.0006) [2023-12-27 06:04:24,315][20347] Updated weights for policy 0, policy_version 59284 (0.0007) [2023-12-27 06:04:24,364][20347] Updated weights for policy 0, policy_version 59294 (0.0006) [2023-12-27 06:04:24,373][20381] Updated weights for policy 1, policy_version 59182 (0.0008) [2023-12-27 06:04:24,417][20347] Updated weights for policy 0, policy_version 59304 (0.0005) [2023-12-27 06:04:24,419][20381] Updated weights for policy 1, policy_version 59192 (0.0010) [2023-12-27 06:04:24,478][20381] Updated weights for policy 1, policy_version 59202 (0.0010) [2023-12-27 06:04:25,112][20381] Updated weights for policy 1, policy_version 59212 (0.0010) [2023-12-27 06:04:25,122][20347] Updated weights for policy 0, policy_version 59314 (0.0005) [2023-12-27 06:04:25,166][20381] Updated weights for policy 1, policy_version 59222 (0.0010) [2023-12-27 06:04:25,175][20347] Updated weights for policy 0, policy_version 59324 (0.0005) [2023-12-27 06:04:25,225][20347] Updated weights for policy 0, policy_version 59334 (0.0006) [2023-12-27 06:04:25,227][20381] Updated weights for policy 1, policy_version 59232 (0.0010) [2023-12-27 06:04:25,283][20347] Updated weights for policy 0, policy_version 59344 (0.0007) [2023-12-27 06:04:25,773][20381] Updated weights for policy 1, policy_version 59242 (0.0006) [2023-12-27 06:04:25,827][20381] Updated weights for policy 1, policy_version 59252 (0.0010) [2023-12-27 06:04:25,874][20381] Updated weights for policy 1, policy_version 59262 (0.0010) [2023-12-27 06:04:25,899][20347] Updated weights for policy 0, policy_version 59354 (0.0005) [2023-12-27 06:04:25,931][20381] Updated weights for policy 1, policy_version 59272 (0.0010) [2023-12-27 06:04:25,959][20347] Updated weights for policy 0, policy_version 59364 (0.0005) [2023-12-27 06:04:26,019][20347] Updated weights for policy 0, policy_version 59374 (0.0006) [2023-12-27 06:04:26,606][20347] Updated weights for policy 0, policy_version 59384 (0.0007) [2023-12-27 06:04:26,625][20381] Updated weights for policy 1, policy_version 59282 (0.0010) [2023-12-27 06:04:26,668][20347] Updated weights for policy 0, policy_version 59394 (0.0007) [2023-12-27 06:04:26,674][20381] Updated weights for policy 1, policy_version 59292 (0.0010) [2023-12-27 06:04:26,719][20347] Updated weights for policy 0, policy_version 59404 (0.0007) [2023-12-27 06:04:26,730][20381] Updated weights for policy 1, policy_version 59302 (0.0011) [2023-12-27 06:04:27,340][20381] Updated weights for policy 1, policy_version 59312 (0.0008) [2023-12-27 06:04:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 30400512. Throughput: 0: 10387.1, 1: 10218.1. Samples: 30394740. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:27,396][20381] Updated weights for policy 1, policy_version 59322 (0.0007) [2023-12-27 06:04:27,404][20347] Updated weights for policy 0, policy_version 59414 (0.0005) [2023-12-27 06:04:27,439][20381] Updated weights for policy 1, policy_version 59332 (0.0006) [2023-12-27 06:04:27,452][20347] Updated weights for policy 0, policy_version 59424 (0.0005) [2023-12-27 06:04:27,502][20347] Updated weights for policy 0, policy_version 59434 (0.0005) [2023-12-27 06:04:28,036][20381] Updated weights for policy 1, policy_version 59342 (0.0007) [2023-12-27 06:04:28,096][20381] Updated weights for policy 1, policy_version 59352 (0.0007) [2023-12-27 06:04:28,146][20381] Updated weights for policy 1, policy_version 59362 (0.0007) [2023-12-27 06:04:28,171][20347] Updated weights for policy 0, policy_version 59444 (0.0007) [2023-12-27 06:04:28,223][20347] Updated weights for policy 0, policy_version 59454 (0.0007) [2023-12-27 06:04:28,281][20347] Updated weights for policy 0, policy_version 59464 (0.0008) [2023-12-27 06:04:28,740][20381] Updated weights for policy 1, policy_version 59372 (0.0007) [2023-12-27 06:04:28,803][20381] Updated weights for policy 1, policy_version 59382 (0.0005) [2023-12-27 06:04:28,856][20381] Updated weights for policy 1, policy_version 59392 (0.0006) [2023-12-27 06:04:29,044][20347] Updated weights for policy 0, policy_version 59474 (0.0008) [2023-12-27 06:04:29,102][20347] Updated weights for policy 0, policy_version 59484 (0.0008) [2023-12-27 06:04:29,162][20347] Updated weights for policy 0, policy_version 59494 (0.0007) [2023-12-27 06:04:29,225][20347] Updated weights for policy 0, policy_version 59504 (0.0006) [2023-12-27 06:04:29,489][20381] Updated weights for policy 1, policy_version 59402 (0.0005) [2023-12-27 06:04:29,548][20381] Updated weights for policy 1, policy_version 59412 (0.0006) [2023-12-27 06:04:29,611][20381] Updated weights for policy 1, policy_version 59422 (0.0006) [2023-12-27 06:04:29,678][20381] Updated weights for policy 1, policy_version 59432 (0.0007) [2023-12-27 06:04:29,950][20347] Updated weights for policy 0, policy_version 59514 (0.0006) [2023-12-27 06:04:30,014][20347] Updated weights for policy 0, policy_version 59524 (0.0007) [2023-12-27 06:04:30,076][20347] Updated weights for policy 0, policy_version 59534 (0.0006) [2023-12-27 06:04:30,266][20381] Updated weights for policy 1, policy_version 59442 (0.0008) [2023-12-27 06:04:30,331][20381] Updated weights for policy 1, policy_version 59452 (0.0009) [2023-12-27 06:04:30,388][20381] Updated weights for policy 1, policy_version 59462 (0.0007) [2023-12-27 06:04:30,730][20347] Updated weights for policy 0, policy_version 59544 (0.0006) [2023-12-27 06:04:30,775][20347] Updated weights for policy 0, policy_version 59554 (0.0005) [2023-12-27 06:04:30,824][20347] Updated weights for policy 0, policy_version 59564 (0.0005) [2023-12-27 06:04:31,023][20381] Updated weights for policy 1, policy_version 59472 (0.0010) [2023-12-27 06:04:31,078][20381] Updated weights for policy 1, policy_version 59482 (0.0010) [2023-12-27 06:04:31,140][20381] Updated weights for policy 1, policy_version 59492 (0.0010) [2023-12-27 06:04:31,471][20347] Updated weights for policy 0, policy_version 59574 (0.0005) [2023-12-27 06:04:31,528][20347] Updated weights for policy 0, policy_version 59584 (0.0008) [2023-12-27 06:04:31,576][20347] Updated weights for policy 0, policy_version 59594 (0.0007) [2023-12-27 06:04:31,862][20381] Updated weights for policy 1, policy_version 59502 (0.0008) [2023-12-27 06:04:31,916][20381] Updated weights for policy 1, policy_version 59512 (0.0008) [2023-12-27 06:04:31,970][20381] Updated weights for policy 1, policy_version 59522 (0.0008) [2023-12-27 06:04:32,277][20347] Updated weights for policy 0, policy_version 59604 (0.0008) [2023-12-27 06:04:32,340][20347] Updated weights for policy 0, policy_version 59614 (0.0006) [2023-12-27 06:04:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 30507008. Throughput: 0: 10373.5, 1: 10311.4. Samples: 30520988. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:32,403][20347] Updated weights for policy 0, policy_version 59624 (0.0006) [2023-12-27 06:04:32,662][20381] Updated weights for policy 1, policy_version 59532 (0.0009) [2023-12-27 06:04:32,720][20381] Updated weights for policy 1, policy_version 59542 (0.0010) [2023-12-27 06:04:32,778][20381] Updated weights for policy 1, policy_version 59552 (0.0009) [2023-12-27 06:04:33,041][20347] Updated weights for policy 0, policy_version 59634 (0.0006) [2023-12-27 06:04:33,097][20347] Updated weights for policy 0, policy_version 59644 (0.0007) [2023-12-27 06:04:33,148][20347] Updated weights for policy 0, policy_version 59654 (0.0007) [2023-12-27 06:04:33,195][20347] Updated weights for policy 0, policy_version 59664 (0.0007) [2023-12-27 06:04:33,369][20381] Updated weights for policy 1, policy_version 59562 (0.0005) [2023-12-27 06:04:33,421][20381] Updated weights for policy 1, policy_version 59572 (0.0005) [2023-12-27 06:04:33,469][20381] Updated weights for policy 1, policy_version 59582 (0.0005) [2023-12-27 06:04:33,519][20381] Updated weights for policy 1, policy_version 59592 (0.0007) [2023-12-27 06:04:33,910][20347] Updated weights for policy 0, policy_version 59674 (0.0007) [2023-12-27 06:04:33,956][20347] Updated weights for policy 0, policy_version 59684 (0.0008) [2023-12-27 06:04:34,007][20347] Updated weights for policy 0, policy_version 59694 (0.0007) [2023-12-27 06:04:34,158][20381] Updated weights for policy 1, policy_version 59602 (0.0010) [2023-12-27 06:04:34,206][20381] Updated weights for policy 1, policy_version 59612 (0.0007) [2023-12-27 06:04:34,265][20381] Updated weights for policy 1, policy_version 59622 (0.0008) [2023-12-27 06:04:34,736][20347] Updated weights for policy 0, policy_version 59704 (0.0007) [2023-12-27 06:04:34,797][20347] Updated weights for policy 0, policy_version 59714 (0.0007) [2023-12-27 06:04:34,848][20347] Updated weights for policy 0, policy_version 59724 (0.0007) [2023-12-27 06:04:34,965][20381] Updated weights for policy 1, policy_version 59632 (0.0007) [2023-12-27 06:04:35,010][20381] Updated weights for policy 1, policy_version 59642 (0.0007) [2023-12-27 06:04:35,056][20381] Updated weights for policy 1, policy_version 59652 (0.0007) [2023-12-27 06:04:35,534][20347] Updated weights for policy 0, policy_version 59734 (0.0007) [2023-12-27 06:04:35,584][20347] Updated weights for policy 0, policy_version 59744 (0.0007) [2023-12-27 06:04:35,638][20347] Updated weights for policy 0, policy_version 59754 (0.0007) [2023-12-27 06:04:35,768][20381] Updated weights for policy 1, policy_version 59662 (0.0007) [2023-12-27 06:04:35,820][20381] Updated weights for policy 1, policy_version 59672 (0.0007) [2023-12-27 06:04:35,870][20381] Updated weights for policy 1, policy_version 59682 (0.0008) [2023-12-27 06:04:36,367][20347] Updated weights for policy 0, policy_version 59764 (0.0008) [2023-12-27 06:04:36,426][20347] Updated weights for policy 0, policy_version 59774 (0.0008) [2023-12-27 06:04:36,489][20347] Updated weights for policy 0, policy_version 59784 (0.0008) [2023-12-27 06:04:36,570][20381] Updated weights for policy 1, policy_version 59692 (0.0007) [2023-12-27 06:04:36,620][20381] Updated weights for policy 1, policy_version 59702 (0.0008) [2023-12-27 06:04:36,667][20381] Updated weights for policy 1, policy_version 59712 (0.0008) [2023-12-27 06:04:37,199][20347] Updated weights for policy 0, policy_version 59794 (0.0008) [2023-12-27 06:04:37,255][20347] Updated weights for policy 0, policy_version 59804 (0.0008) [2023-12-27 06:04:37,310][20347] Updated weights for policy 0, policy_version 59814 (0.0007) [2023-12-27 06:04:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 30605312. Throughput: 0: 10339.7, 1: 10366.3. Samples: 30582708. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000059824_15319040.pth... [2023-12-27 06:04:37,368][20381] Updated weights for policy 1, policy_version 59722 (0.0007) [2023-12-27 06:04:37,370][20347] Updated weights for policy 0, policy_version 59824 (0.0007) [2023-12-27 06:04:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000058640_15015936.pth [2023-12-27 06:04:37,422][20381] Updated weights for policy 1, policy_version 59732 (0.0008) [2023-12-27 06:04:37,479][20381] Updated weights for policy 1, policy_version 59742 (0.0007) [2023-12-27 06:04:37,544][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000059752_15302656.pth... [2023-12-27 06:04:37,546][20381] Updated weights for policy 1, policy_version 59752 (0.0005) [2023-12-27 06:04:37,548][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000058504_14983168.pth [2023-12-27 06:04:38,083][20381] Updated weights for policy 1, policy_version 59762 (0.0006) [2023-12-27 06:04:38,117][20347] Updated weights for policy 0, policy_version 59834 (0.0008) [2023-12-27 06:04:38,134][20381] Updated weights for policy 1, policy_version 59772 (0.0006) [2023-12-27 06:04:38,173][20347] Updated weights for policy 0, policy_version 59844 (0.0005) [2023-12-27 06:04:38,196][20381] Updated weights for policy 1, policy_version 59782 (0.0006) [2023-12-27 06:04:38,229][20347] Updated weights for policy 0, policy_version 59854 (0.0006) [2023-12-27 06:04:38,844][20381] Updated weights for policy 1, policy_version 59792 (0.0007) [2023-12-27 06:04:38,898][20381] Updated weights for policy 1, policy_version 59802 (0.0006) [2023-12-27 06:04:38,916][20347] Updated weights for policy 0, policy_version 59864 (0.0006) [2023-12-27 06:04:38,954][20381] Updated weights for policy 1, policy_version 59812 (0.0007) [2023-12-27 06:04:38,971][20347] Updated weights for policy 0, policy_version 59874 (0.0005) [2023-12-27 06:04:39,026][20347] Updated weights for policy 0, policy_version 59884 (0.0007) [2023-12-27 06:04:39,662][20381] Updated weights for policy 1, policy_version 59822 (0.0007) [2023-12-27 06:04:39,718][20381] Updated weights for policy 1, policy_version 59832 (0.0007) [2023-12-27 06:04:39,733][20347] Updated weights for policy 0, policy_version 59894 (0.0007) [2023-12-27 06:04:39,775][20381] Updated weights for policy 1, policy_version 59842 (0.0008) [2023-12-27 06:04:39,782][20347] Updated weights for policy 0, policy_version 59904 (0.0006) [2023-12-27 06:04:39,845][20347] Updated weights for policy 0, policy_version 59914 (0.0007) [2023-12-27 06:04:40,384][20381] Updated weights for policy 1, policy_version 59852 (0.0008) [2023-12-27 06:04:40,440][20381] Updated weights for policy 1, policy_version 59862 (0.0008) [2023-12-27 06:04:40,498][20381] Updated weights for policy 1, policy_version 59872 (0.0007) [2023-12-27 06:04:40,581][20347] Updated weights for policy 0, policy_version 59924 (0.0008) [2023-12-27 06:04:40,636][20347] Updated weights for policy 0, policy_version 59934 (0.0008) [2023-12-27 06:04:40,695][20347] Updated weights for policy 0, policy_version 59944 (0.0008) [2023-12-27 06:04:41,138][20381] Updated weights for policy 1, policy_version 59882 (0.0008) [2023-12-27 06:04:41,200][20381] Updated weights for policy 1, policy_version 59892 (0.0011) [2023-12-27 06:04:41,261][20381] Updated weights for policy 1, policy_version 59902 (0.0010) [2023-12-27 06:04:41,329][20381] Updated weights for policy 1, policy_version 59912 (0.0010) [2023-12-27 06:04:41,357][20347] Updated weights for policy 0, policy_version 59954 (0.0007) [2023-12-27 06:04:41,416][20347] Updated weights for policy 0, policy_version 59964 (0.0007) [2023-12-27 06:04:41,467][20347] Updated weights for policy 0, policy_version 59974 (0.0007) [2023-12-27 06:04:41,529][20347] Updated weights for policy 0, policy_version 59984 (0.0009) [2023-12-27 06:04:42,059][20381] Updated weights for policy 1, policy_version 59922 (0.0009) [2023-12-27 06:04:42,117][20381] Updated weights for policy 1, policy_version 59932 (0.0008) [2023-12-27 06:04:42,181][20381] Updated weights for policy 1, policy_version 59942 (0.0008) [2023-12-27 06:04:42,206][20347] Updated weights for policy 0, policy_version 59994 (0.0007) [2023-12-27 06:04:42,260][20347] Updated weights for policy 0, policy_version 60004 (0.0009) [2023-12-27 06:04:42,325][20347] Updated weights for policy 0, policy_version 60014 (0.0008) [2023-12-27 06:04:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 30720000. Throughput: 0: 10221.4, 1: 10439.6. Samples: 30705240. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:42,900][20381] Updated weights for policy 1, policy_version 59952 (0.0011) [2023-12-27 06:04:42,960][20381] Updated weights for policy 1, policy_version 59962 (0.0010) [2023-12-27 06:04:42,962][20347] Updated weights for policy 0, policy_version 60024 (0.0008) [2023-12-27 06:04:43,011][20381] Updated weights for policy 1, policy_version 59972 (0.0005) [2023-12-27 06:04:43,019][20347] Updated weights for policy 0, policy_version 60034 (0.0008) [2023-12-27 06:04:43,070][20347] Updated weights for policy 0, policy_version 60044 (0.0008) [2023-12-27 06:04:43,700][20381] Updated weights for policy 1, policy_version 59982 (0.0009) [2023-12-27 06:04:43,706][20347] Updated weights for policy 0, policy_version 60054 (0.0006) [2023-12-27 06:04:43,752][20381] Updated weights for policy 1, policy_version 59992 (0.0010) [2023-12-27 06:04:43,757][20347] Updated weights for policy 0, policy_version 60064 (0.0005) [2023-12-27 06:04:43,800][20381] Updated weights for policy 1, policy_version 60002 (0.0010) [2023-12-27 06:04:43,809][20347] Updated weights for policy 0, policy_version 60074 (0.0005) [2023-12-27 06:04:44,461][20347] Updated weights for policy 0, policy_version 60084 (0.0006) [2023-12-27 06:04:44,499][20381] Updated weights for policy 1, policy_version 60012 (0.0011) [2023-12-27 06:04:44,515][20347] Updated weights for policy 0, policy_version 60094 (0.0005) [2023-12-27 06:04:44,558][20381] Updated weights for policy 1, policy_version 60022 (0.0011) [2023-12-27 06:04:44,577][20347] Updated weights for policy 0, policy_version 60104 (0.0005) [2023-12-27 06:04:44,618][20381] Updated weights for policy 1, policy_version 60032 (0.0011) [2023-12-27 06:04:45,215][20381] Updated weights for policy 1, policy_version 60042 (0.0009) [2023-12-27 06:04:45,273][20381] Updated weights for policy 1, policy_version 60052 (0.0007) [2023-12-27 06:04:45,304][20347] Updated weights for policy 0, policy_version 60114 (0.0006) [2023-12-27 06:04:45,334][20381] Updated weights for policy 1, policy_version 60062 (0.0006) [2023-12-27 06:04:45,355][20347] Updated weights for policy 0, policy_version 60124 (0.0006) [2023-12-27 06:04:45,392][20381] Updated weights for policy 1, policy_version 60072 (0.0010) [2023-12-27 06:04:45,400][20347] Updated weights for policy 0, policy_version 60134 (0.0005) [2023-12-27 06:04:45,444][20347] Updated weights for policy 0, policy_version 60144 (0.0005) [2023-12-27 06:04:46,026][20381] Updated weights for policy 1, policy_version 60082 (0.0006) [2023-12-27 06:04:46,039][20347] Updated weights for policy 0, policy_version 60154 (0.0005) [2023-12-27 06:04:46,082][20381] Updated weights for policy 1, policy_version 60092 (0.0011) [2023-12-27 06:04:46,094][20347] Updated weights for policy 0, policy_version 60164 (0.0006) [2023-12-27 06:04:46,138][20381] Updated weights for policy 1, policy_version 60102 (0.0010) [2023-12-27 06:04:46,142][20347] Updated weights for policy 0, policy_version 60174 (0.0005) [2023-12-27 06:04:46,720][20381] Updated weights for policy 1, policy_version 60112 (0.0006) [2023-12-27 06:04:46,769][20381] Updated weights for policy 1, policy_version 60122 (0.0006) [2023-12-27 06:04:46,819][20381] Updated weights for policy 1, policy_version 60132 (0.0006) [2023-12-27 06:04:46,839][20347] Updated weights for policy 0, policy_version 60184 (0.0005) [2023-12-27 06:04:46,884][20347] Updated weights for policy 0, policy_version 60194 (0.0008) [2023-12-27 06:04:46,932][20347] Updated weights for policy 0, policy_version 60204 (0.0010) [2023-12-27 06:04:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 30818304. Throughput: 0: 10245.9, 1: 10495.8. Samples: 30831120. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:04:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:47,397][20381] Updated weights for policy 1, policy_version 60142 (0.0008) [2023-12-27 06:04:47,442][20381] Updated weights for policy 1, policy_version 60152 (0.0008) [2023-12-27 06:04:47,492][20381] Updated weights for policy 1, policy_version 60162 (0.0007) [2023-12-27 06:04:47,642][20347] Updated weights for policy 0, policy_version 60214 (0.0009) [2023-12-27 06:04:47,703][20347] Updated weights for policy 0, policy_version 60224 (0.0008) [2023-12-27 06:04:47,753][20347] Updated weights for policy 0, policy_version 60234 (0.0007) [2023-12-27 06:04:48,106][20381] Updated weights for policy 1, policy_version 60172 (0.0006) [2023-12-27 06:04:48,161][20381] Updated weights for policy 1, policy_version 60182 (0.0008) [2023-12-27 06:04:48,222][20381] Updated weights for policy 1, policy_version 60192 (0.0006) [2023-12-27 06:04:48,472][20347] Updated weights for policy 0, policy_version 60244 (0.0007) [2023-12-27 06:04:48,543][20347] Updated weights for policy 0, policy_version 60254 (0.0008) [2023-12-27 06:04:48,597][20347] Updated weights for policy 0, policy_version 60264 (0.0007) [2023-12-27 06:04:48,880][20381] Updated weights for policy 1, policy_version 60202 (0.0008) [2023-12-27 06:04:48,932][20381] Updated weights for policy 1, policy_version 60212 (0.0006) [2023-12-27 06:04:48,985][20381] Updated weights for policy 1, policy_version 60222 (0.0005) [2023-12-27 06:04:49,044][20381] Updated weights for policy 1, policy_version 60232 (0.0005) [2023-12-27 06:04:49,298][20347] Updated weights for policy 0, policy_version 60274 (0.0008) [2023-12-27 06:04:49,359][20347] Updated weights for policy 0, policy_version 60284 (0.0008) [2023-12-27 06:04:49,414][20347] Updated weights for policy 0, policy_version 60294 (0.0007) [2023-12-27 06:04:49,465][20347] Updated weights for policy 0, policy_version 60304 (0.0006) [2023-12-27 06:04:49,663][20381] Updated weights for policy 1, policy_version 60242 (0.0008) [2023-12-27 06:04:49,726][20381] Updated weights for policy 1, policy_version 60252 (0.0008) [2023-12-27 06:04:49,787][20381] Updated weights for policy 1, policy_version 60262 (0.0008) [2023-12-27 06:04:50,172][20347] Updated weights for policy 0, policy_version 60314 (0.0008) [2023-12-27 06:04:50,224][20347] Updated weights for policy 0, policy_version 60324 (0.0008) [2023-12-27 06:04:50,283][20347] Updated weights for policy 0, policy_version 60334 (0.0007) [2023-12-27 06:04:50,496][20381] Updated weights for policy 1, policy_version 60272 (0.0006) [2023-12-27 06:04:50,547][20381] Updated weights for policy 1, policy_version 60282 (0.0005) [2023-12-27 06:04:50,603][20381] Updated weights for policy 1, policy_version 60292 (0.0006) [2023-12-27 06:04:50,993][20347] Updated weights for policy 0, policy_version 60344 (0.0006) [2023-12-27 06:04:51,061][20347] Updated weights for policy 0, policy_version 60354 (0.0007) [2023-12-27 06:04:51,125][20347] Updated weights for policy 0, policy_version 60364 (0.0006) [2023-12-27 06:04:51,314][20381] Updated weights for policy 1, policy_version 60302 (0.0006) [2023-12-27 06:04:51,380][20381] Updated weights for policy 1, policy_version 60312 (0.0007) [2023-12-27 06:04:51,444][20381] Updated weights for policy 1, policy_version 60322 (0.0007) [2023-12-27 06:04:51,755][20347] Updated weights for policy 0, policy_version 60374 (0.0007) [2023-12-27 06:04:51,813][20347] Updated weights for policy 0, policy_version 60384 (0.0008) [2023-12-27 06:04:51,873][20347] Updated weights for policy 0, policy_version 60394 (0.0009) [2023-12-27 06:04:52,059][20381] Updated weights for policy 1, policy_version 60332 (0.0005) [2023-12-27 06:04:52,112][20381] Updated weights for policy 1, policy_version 60342 (0.0006) [2023-12-27 06:04:52,173][20381] Updated weights for policy 1, policy_version 60352 (0.0008) [2023-12-27 06:04:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 30924800. Throughput: 0: 10199.0, 1: 10501.3. Samples: 30892788. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:04:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000060360_15458304.pth... [2023-12-27 06:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000060400_15466496.pth... [2023-12-27 06:04:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000059112_15138816.pth [2023-12-27 06:04:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000059216_15163392.pth [2023-12-27 06:04:52,609][20347] Updated weights for policy 0, policy_version 60404 (0.0007) [2023-12-27 06:04:52,670][20347] Updated weights for policy 0, policy_version 60414 (0.0008) [2023-12-27 06:04:52,730][20347] Updated weights for policy 0, policy_version 60424 (0.0008) [2023-12-27 06:04:52,785][20381] Updated weights for policy 1, policy_version 60362 (0.0007) [2023-12-27 06:04:52,840][20381] Updated weights for policy 1, policy_version 60372 (0.0007) [2023-12-27 06:04:52,895][20381] Updated weights for policy 1, policy_version 60382 (0.0008) [2023-12-27 06:04:52,953][20381] Updated weights for policy 1, policy_version 60392 (0.0006) [2023-12-27 06:04:53,442][20347] Updated weights for policy 0, policy_version 60434 (0.0008) [2023-12-27 06:04:53,494][20347] Updated weights for policy 0, policy_version 60444 (0.0009) [2023-12-27 06:04:53,547][20347] Updated weights for policy 0, policy_version 60454 (0.0006) [2023-12-27 06:04:53,599][20347] Updated weights for policy 0, policy_version 60464 (0.0010) [2023-12-27 06:04:53,631][20381] Updated weights for policy 1, policy_version 60402 (0.0006) [2023-12-27 06:04:53,691][20381] Updated weights for policy 1, policy_version 60412 (0.0007) [2023-12-27 06:04:53,745][20381] Updated weights for policy 1, policy_version 60422 (0.0007) [2023-12-27 06:04:54,194][20347] Updated weights for policy 0, policy_version 60474 (0.0010) [2023-12-27 06:04:54,243][20347] Updated weights for policy 0, policy_version 60484 (0.0006) [2023-12-27 06:04:54,295][20347] Updated weights for policy 0, policy_version 60494 (0.0005) [2023-12-27 06:04:54,481][20381] Updated weights for policy 1, policy_version 60432 (0.0008) [2023-12-27 06:04:54,537][20381] Updated weights for policy 1, policy_version 60442 (0.0007) [2023-12-27 06:04:54,593][20381] Updated weights for policy 1, policy_version 60452 (0.0008) [2023-12-27 06:04:54,858][20347] Updated weights for policy 0, policy_version 60504 (0.0005) [2023-12-27 06:04:54,918][20347] Updated weights for policy 0, policy_version 60514 (0.0009) [2023-12-27 06:04:54,980][20347] Updated weights for policy 0, policy_version 60524 (0.0008) [2023-12-27 06:04:55,349][20381] Updated weights for policy 1, policy_version 60462 (0.0008) [2023-12-27 06:04:55,422][20381] Updated weights for policy 1, policy_version 60472 (0.0008) [2023-12-27 06:04:55,485][20381] Updated weights for policy 1, policy_version 60482 (0.0008) [2023-12-27 06:04:55,521][20347] Updated weights for policy 0, policy_version 60534 (0.0006) [2023-12-27 06:04:55,572][20347] Updated weights for policy 0, policy_version 60544 (0.0005) [2023-12-27 06:04:55,621][20347] Updated weights for policy 0, policy_version 60554 (0.0005) [2023-12-27 06:04:56,178][20347] Updated weights for policy 0, policy_version 60564 (0.0005) [2023-12-27 06:04:56,202][20381] Updated weights for policy 1, policy_version 60492 (0.0007) [2023-12-27 06:04:56,234][20347] Updated weights for policy 0, policy_version 60574 (0.0005) [2023-12-27 06:04:56,262][20381] Updated weights for policy 1, policy_version 60502 (0.0008) [2023-12-27 06:04:56,282][20347] Updated weights for policy 0, policy_version 60584 (0.0005) [2023-12-27 06:04:56,317][20381] Updated weights for policy 1, policy_version 60512 (0.0007) [2023-12-27 06:04:56,870][20347] Updated weights for policy 0, policy_version 60594 (0.0006) [2023-12-27 06:04:56,914][20347] Updated weights for policy 0, policy_version 60604 (0.0010) [2023-12-27 06:04:56,962][20347] Updated weights for policy 0, policy_version 60614 (0.0010) [2023-12-27 06:04:56,989][20381] Updated weights for policy 1, policy_version 60522 (0.0007) [2023-12-27 06:04:57,012][20347] Updated weights for policy 0, policy_version 60624 (0.0010) [2023-12-27 06:04:57,044][20381] Updated weights for policy 1, policy_version 60532 (0.0006) [2023-12-27 06:04:57,099][20381] Updated weights for policy 1, policy_version 60542 (0.0009) [2023-12-27 06:04:57,144][20381] Updated weights for policy 1, policy_version 60552 (0.0010) [2023-12-27 06:04:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 31031296. Throughput: 0: 10245.3, 1: 10435.3. Samples: 31017288. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:04:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:04:57,657][20347] Updated weights for policy 0, policy_version 60634 (0.0005) [2023-12-27 06:04:57,703][20347] Updated weights for policy 0, policy_version 60644 (0.0005) [2023-12-27 06:04:57,760][20347] Updated weights for policy 0, policy_version 60654 (0.0005) [2023-12-27 06:04:57,838][20381] Updated weights for policy 1, policy_version 60562 (0.0010) [2023-12-27 06:04:57,895][20381] Updated weights for policy 1, policy_version 60572 (0.0010) [2023-12-27 06:04:57,945][20381] Updated weights for policy 1, policy_version 60582 (0.0010) [2023-12-27 06:04:58,282][20347] Updated weights for policy 0, policy_version 60664 (0.0007) [2023-12-27 06:04:58,346][20347] Updated weights for policy 0, policy_version 60674 (0.0007) [2023-12-27 06:04:58,406][20347] Updated weights for policy 0, policy_version 60684 (0.0007) [2023-12-27 06:04:58,687][20381] Updated weights for policy 1, policy_version 60592 (0.0010) [2023-12-27 06:04:58,749][20381] Updated weights for policy 1, policy_version 60602 (0.0010) [2023-12-27 06:04:58,810][20381] Updated weights for policy 1, policy_version 60612 (0.0006) [2023-12-27 06:04:59,176][20347] Updated weights for policy 0, policy_version 60694 (0.0006) [2023-12-27 06:04:59,237][20347] Updated weights for policy 0, policy_version 60704 (0.0007) [2023-12-27 06:04:59,302][20347] Updated weights for policy 0, policy_version 60714 (0.0008) [2023-12-27 06:04:59,428][20381] Updated weights for policy 1, policy_version 60622 (0.0006) [2023-12-27 06:04:59,476][20381] Updated weights for policy 1, policy_version 60632 (0.0005) [2023-12-27 06:04:59,530][20381] Updated weights for policy 1, policy_version 60642 (0.0005) [2023-12-27 06:05:00,072][20347] Updated weights for policy 0, policy_version 60724 (0.0007) [2023-12-27 06:05:00,125][20347] Updated weights for policy 0, policy_version 60734 (0.0007) [2023-12-27 06:05:00,144][20381] Updated weights for policy 1, policy_version 60652 (0.0006) [2023-12-27 06:05:00,186][20347] Updated weights for policy 0, policy_version 60744 (0.0008) [2023-12-27 06:05:00,200][20381] Updated weights for policy 1, policy_version 60662 (0.0006) [2023-12-27 06:05:00,257][20381] Updated weights for policy 1, policy_version 60672 (0.0006) [2023-12-27 06:05:00,857][20347] Updated weights for policy 0, policy_version 60754 (0.0008) [2023-12-27 06:05:00,906][20381] Updated weights for policy 1, policy_version 60682 (0.0008) [2023-12-27 06:05:00,913][20347] Updated weights for policy 0, policy_version 60764 (0.0008) [2023-12-27 06:05:00,960][20347] Updated weights for policy 0, policy_version 60774 (0.0008) [2023-12-27 06:05:00,971][20381] Updated weights for policy 1, policy_version 60692 (0.0010) [2023-12-27 06:05:01,007][20347] Updated weights for policy 0, policy_version 60784 (0.0005) [2023-12-27 06:05:01,021][20381] Updated weights for policy 1, policy_version 60702 (0.0010) [2023-12-27 06:05:01,076][20381] Updated weights for policy 1, policy_version 60712 (0.0011) [2023-12-27 06:05:01,773][20347] Updated weights for policy 0, policy_version 60794 (0.0008) [2023-12-27 06:05:01,813][20381] Updated weights for policy 1, policy_version 60722 (0.0009) [2023-12-27 06:05:01,827][20347] Updated weights for policy 0, policy_version 60804 (0.0007) [2023-12-27 06:05:01,866][20381] Updated weights for policy 1, policy_version 60732 (0.0009) [2023-12-27 06:05:01,882][20347] Updated weights for policy 0, policy_version 60814 (0.0006) [2023-12-27 06:05:01,924][20381] Updated weights for policy 1, policy_version 60742 (0.0010) [2023-12-27 06:05:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 31129600. Throughput: 0: 10259.5, 1: 10424.3. Samples: 31141072. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:05:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:02,591][20347] Updated weights for policy 0, policy_version 60824 (0.0006) [2023-12-27 06:05:02,605][20381] Updated weights for policy 1, policy_version 60752 (0.0006) [2023-12-27 06:05:02,655][20347] Updated weights for policy 0, policy_version 60834 (0.0008) [2023-12-27 06:05:02,660][20381] Updated weights for policy 1, policy_version 60762 (0.0007) [2023-12-27 06:05:02,720][20347] Updated weights for policy 0, policy_version 60844 (0.0008) [2023-12-27 06:05:02,720][20381] Updated weights for policy 1, policy_version 60772 (0.0009) [2023-12-27 06:05:03,244][20347] Updated weights for policy 0, policy_version 60854 (0.0006) [2023-12-27 06:05:03,270][20381] Updated weights for policy 1, policy_version 60782 (0.0006) [2023-12-27 06:05:03,308][20347] Updated weights for policy 0, policy_version 60864 (0.0008) [2023-12-27 06:05:03,323][20381] Updated weights for policy 1, policy_version 60792 (0.0005) [2023-12-27 06:05:03,374][20381] Updated weights for policy 1, policy_version 60802 (0.0007) [2023-12-27 06:05:03,376][20347] Updated weights for policy 0, policy_version 60874 (0.0008) [2023-12-27 06:05:03,939][20347] Updated weights for policy 0, policy_version 60884 (0.0007) [2023-12-27 06:05:03,987][20347] Updated weights for policy 0, policy_version 60894 (0.0008) [2023-12-27 06:05:04,038][20347] Updated weights for policy 0, policy_version 60904 (0.0007) [2023-12-27 06:05:04,078][20381] Updated weights for policy 1, policy_version 60812 (0.0005) [2023-12-27 06:05:04,139][20381] Updated weights for policy 1, policy_version 60822 (0.0006) [2023-12-27 06:05:04,201][20381] Updated weights for policy 1, policy_version 60832 (0.0006) [2023-12-27 06:05:04,747][20347] Updated weights for policy 0, policy_version 60914 (0.0007) [2023-12-27 06:05:04,809][20347] Updated weights for policy 0, policy_version 60924 (0.0008) [2023-12-27 06:05:04,826][20381] Updated weights for policy 1, policy_version 60842 (0.0007) [2023-12-27 06:05:04,867][20347] Updated weights for policy 0, policy_version 60934 (0.0008) [2023-12-27 06:05:04,869][20381] Updated weights for policy 1, policy_version 60852 (0.0005) [2023-12-27 06:05:04,916][20347] Updated weights for policy 0, policy_version 60944 (0.0008) [2023-12-27 06:05:04,918][20381] Updated weights for policy 1, policy_version 60862 (0.0005) [2023-12-27 06:05:04,967][20381] Updated weights for policy 1, policy_version 60872 (0.0007) [2023-12-27 06:05:05,565][20381] Updated weights for policy 1, policy_version 60882 (0.0005) [2023-12-27 06:05:05,620][20347] Updated weights for policy 0, policy_version 60954 (0.0008) [2023-12-27 06:05:05,622][20381] Updated weights for policy 1, policy_version 60892 (0.0005) [2023-12-27 06:05:05,674][20347] Updated weights for policy 0, policy_version 60964 (0.0005) [2023-12-27 06:05:05,677][20381] Updated weights for policy 1, policy_version 60902 (0.0008) [2023-12-27 06:05:05,735][20347] Updated weights for policy 0, policy_version 60974 (0.0006) [2023-12-27 06:05:06,340][20347] Updated weights for policy 0, policy_version 60984 (0.0007) [2023-12-27 06:05:06,365][20381] Updated weights for policy 1, policy_version 60912 (0.0008) [2023-12-27 06:05:06,402][20347] Updated weights for policy 0, policy_version 60994 (0.0006) [2023-12-27 06:05:06,432][20381] Updated weights for policy 1, policy_version 60922 (0.0007) [2023-12-27 06:05:06,462][20347] Updated weights for policy 0, policy_version 61004 (0.0006) [2023-12-27 06:05:06,493][20381] Updated weights for policy 1, policy_version 60932 (0.0008) [2023-12-27 06:05:07,063][20347] Updated weights for policy 0, policy_version 61014 (0.0007) [2023-12-27 06:05:07,102][20381] Updated weights for policy 1, policy_version 60942 (0.0008) [2023-12-27 06:05:07,122][20347] Updated weights for policy 0, policy_version 61024 (0.0007) [2023-12-27 06:05:07,153][20381] Updated weights for policy 1, policy_version 60952 (0.0005) [2023-12-27 06:05:07,175][20347] Updated weights for policy 0, policy_version 61034 (0.0007) [2023-12-27 06:05:07,209][20381] Updated weights for policy 1, policy_version 60962 (0.0007) [2023-12-27 06:05:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 31244288. Throughput: 0: 10330.9, 1: 10464.4. Samples: 31206096. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:05:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000060968_15613952.pth... [2023-12-27 06:05:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000061040_15630336.pth... [2023-12-27 06:05:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000059824_15319040.pth [2023-12-27 06:05:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000059752_15302656.pth [2023-12-27 06:05:07,719][20347] Updated weights for policy 0, policy_version 61044 (0.0007) [2023-12-27 06:05:07,785][20347] Updated weights for policy 0, policy_version 61054 (0.0006) [2023-12-27 06:05:07,812][20381] Updated weights for policy 1, policy_version 60972 (0.0008) [2023-12-27 06:05:07,841][20347] Updated weights for policy 0, policy_version 61064 (0.0006) [2023-12-27 06:05:07,864][20381] Updated weights for policy 1, policy_version 60982 (0.0007) [2023-12-27 06:05:07,922][20381] Updated weights for policy 1, policy_version 60992 (0.0008) [2023-12-27 06:05:08,565][20381] Updated weights for policy 1, policy_version 61002 (0.0008) [2023-12-27 06:05:08,568][20347] Updated weights for policy 0, policy_version 61074 (0.0007) [2023-12-27 06:05:08,629][20381] Updated weights for policy 1, policy_version 61012 (0.0009) [2023-12-27 06:05:08,632][20347] Updated weights for policy 0, policy_version 61084 (0.0005) [2023-12-27 06:05:08,687][20347] Updated weights for policy 0, policy_version 61094 (0.0008) [2023-12-27 06:05:08,689][20381] Updated weights for policy 1, policy_version 61022 (0.0007) [2023-12-27 06:05:08,749][20347] Updated weights for policy 0, policy_version 61104 (0.0007) [2023-12-27 06:05:08,751][20381] Updated weights for policy 1, policy_version 61032 (0.0006) [2023-12-27 06:05:09,392][20381] Updated weights for policy 1, policy_version 61042 (0.0008) [2023-12-27 06:05:09,398][20347] Updated weights for policy 0, policy_version 61114 (0.0007) [2023-12-27 06:05:09,453][20381] Updated weights for policy 1, policy_version 61052 (0.0008) [2023-12-27 06:05:09,460][20347] Updated weights for policy 0, policy_version 61124 (0.0006) [2023-12-27 06:05:09,513][20381] Updated weights for policy 1, policy_version 61062 (0.0007) [2023-12-27 06:05:09,514][20347] Updated weights for policy 0, policy_version 61134 (0.0006) [2023-12-27 06:05:10,153][20347] Updated weights for policy 0, policy_version 61144 (0.0006) [2023-12-27 06:05:10,184][20381] Updated weights for policy 1, policy_version 61072 (0.0007) [2023-12-27 06:05:10,214][20347] Updated weights for policy 0, policy_version 61154 (0.0006) [2023-12-27 06:05:10,248][20381] Updated weights for policy 1, policy_version 61082 (0.0007) [2023-12-27 06:05:10,273][20347] Updated weights for policy 0, policy_version 61164 (0.0006) [2023-12-27 06:05:10,312][20381] Updated weights for policy 1, policy_version 61092 (0.0005) [2023-12-27 06:05:10,864][20347] Updated weights for policy 0, policy_version 61174 (0.0006) [2023-12-27 06:05:10,928][20347] Updated weights for policy 0, policy_version 61184 (0.0010) [2023-12-27 06:05:10,945][20381] Updated weights for policy 1, policy_version 61102 (0.0006) [2023-12-27 06:05:10,980][20347] Updated weights for policy 0, policy_version 61194 (0.0010) [2023-12-27 06:05:10,997][20381] Updated weights for policy 1, policy_version 61112 (0.0006) [2023-12-27 06:05:11,060][20381] Updated weights for policy 1, policy_version 61122 (0.0008) [2023-12-27 06:05:11,625][20347] Updated weights for policy 0, policy_version 61204 (0.0010) [2023-12-27 06:05:11,693][20347] Updated weights for policy 0, policy_version 61214 (0.0007) [2023-12-27 06:05:11,759][20347] Updated weights for policy 0, policy_version 61224 (0.0012) [2023-12-27 06:05:11,809][20381] Updated weights for policy 1, policy_version 61132 (0.0006) [2023-12-27 06:05:11,871][20381] Updated weights for policy 1, policy_version 61142 (0.0008) [2023-12-27 06:05:11,933][20381] Updated weights for policy 1, policy_version 61152 (0.0007) [2023-12-27 06:05:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 31342592. Throughput: 0: 10369.9, 1: 10520.0. Samples: 31334784. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:05:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:12,475][20347] Updated weights for policy 0, policy_version 61234 (0.0010) [2023-12-27 06:05:12,535][20347] Updated weights for policy 0, policy_version 61244 (0.0007) [2023-12-27 06:05:12,590][20347] Updated weights for policy 0, policy_version 61254 (0.0007) [2023-12-27 06:05:12,638][20381] Updated weights for policy 1, policy_version 61162 (0.0008) [2023-12-27 06:05:12,648][20347] Updated weights for policy 0, policy_version 61264 (0.0007) [2023-12-27 06:05:12,699][20381] Updated weights for policy 1, policy_version 61172 (0.0007) [2023-12-27 06:05:12,764][20381] Updated weights for policy 1, policy_version 61182 (0.0007) [2023-12-27 06:05:12,818][20381] Updated weights for policy 1, policy_version 61192 (0.0007) [2023-12-27 06:05:13,324][20347] Updated weights for policy 0, policy_version 61274 (0.0011) [2023-12-27 06:05:13,390][20347] Updated weights for policy 0, policy_version 61284 (0.0009) [2023-12-27 06:05:13,437][20347] Updated weights for policy 0, policy_version 61294 (0.0009) [2023-12-27 06:05:13,483][20381] Updated weights for policy 1, policy_version 61202 (0.0005) [2023-12-27 06:05:13,540][20381] Updated weights for policy 1, policy_version 61212 (0.0005) [2023-12-27 06:05:13,599][20381] Updated weights for policy 1, policy_version 61222 (0.0006) [2023-12-27 06:05:14,082][20347] Updated weights for policy 0, policy_version 61304 (0.0009) [2023-12-27 06:05:14,138][20347] Updated weights for policy 0, policy_version 61314 (0.0008) [2023-12-27 06:05:14,193][20347] Updated weights for policy 0, policy_version 61324 (0.0007) [2023-12-27 06:05:14,218][20381] Updated weights for policy 1, policy_version 61232 (0.0009) [2023-12-27 06:05:14,280][20381] Updated weights for policy 1, policy_version 61242 (0.0008) [2023-12-27 06:05:14,346][20381] Updated weights for policy 1, policy_version 61252 (0.0008) [2023-12-27 06:05:14,904][20347] Updated weights for policy 0, policy_version 61334 (0.0008) [2023-12-27 06:05:14,963][20347] Updated weights for policy 0, policy_version 61344 (0.0007) [2023-12-27 06:05:15,020][20347] Updated weights for policy 0, policy_version 61354 (0.0007) [2023-12-27 06:05:15,044][20381] Updated weights for policy 1, policy_version 61262 (0.0007) [2023-12-27 06:05:15,102][20381] Updated weights for policy 1, policy_version 61272 (0.0007) [2023-12-27 06:05:15,162][20381] Updated weights for policy 1, policy_version 61282 (0.0008) [2023-12-27 06:05:15,711][20347] Updated weights for policy 0, policy_version 61364 (0.0007) [2023-12-27 06:05:15,756][20347] Updated weights for policy 0, policy_version 61374 (0.0006) [2023-12-27 06:05:15,812][20347] Updated weights for policy 0, policy_version 61384 (0.0005) [2023-12-27 06:05:15,874][20381] Updated weights for policy 1, policy_version 61292 (0.0008) [2023-12-27 06:05:15,921][20381] Updated weights for policy 1, policy_version 61302 (0.0010) [2023-12-27 06:05:15,980][20381] Updated weights for policy 1, policy_version 61312 (0.0008) [2023-12-27 06:05:16,420][20347] Updated weights for policy 0, policy_version 61394 (0.0006) [2023-12-27 06:05:16,473][20347] Updated weights for policy 0, policy_version 61404 (0.0007) [2023-12-27 06:05:16,528][20347] Updated weights for policy 0, policy_version 61414 (0.0007) [2023-12-27 06:05:16,592][20347] Updated weights for policy 0, policy_version 61424 (0.0007) [2023-12-27 06:05:16,704][20381] Updated weights for policy 1, policy_version 61322 (0.0008) [2023-12-27 06:05:16,764][20381] Updated weights for policy 1, policy_version 61332 (0.0007) [2023-12-27 06:05:16,821][20381] Updated weights for policy 1, policy_version 61342 (0.0007) [2023-12-27 06:05:16,872][20381] Updated weights for policy 1, policy_version 61352 (0.0007) [2023-12-27 06:05:17,259][20347] Updated weights for policy 0, policy_version 61434 (0.0008) [2023-12-27 06:05:17,305][20347] Updated weights for policy 0, policy_version 61444 (0.0007) [2023-12-27 06:05:17,352][20347] Updated weights for policy 0, policy_version 61454 (0.0007) [2023-12-27 06:05:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 31440896. Throughput: 0: 10377.4, 1: 10407.5. Samples: 31456308. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:05:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:17,554][20381] Updated weights for policy 1, policy_version 61362 (0.0011) [2023-12-27 06:05:17,602][20381] Updated weights for policy 1, policy_version 61372 (0.0010) [2023-12-27 06:05:17,659][20381] Updated weights for policy 1, policy_version 61382 (0.0010) [2023-12-27 06:05:18,070][20347] Updated weights for policy 0, policy_version 61464 (0.0007) [2023-12-27 06:05:18,120][20347] Updated weights for policy 0, policy_version 61474 (0.0007) [2023-12-27 06:05:18,172][20347] Updated weights for policy 0, policy_version 61484 (0.0007) [2023-12-27 06:05:18,359][20381] Updated weights for policy 1, policy_version 61392 (0.0009) [2023-12-27 06:05:18,417][20381] Updated weights for policy 1, policy_version 61402 (0.0005) [2023-12-27 06:05:18,479][20381] Updated weights for policy 1, policy_version 61412 (0.0007) [2023-12-27 06:05:18,865][20347] Updated weights for policy 0, policy_version 61494 (0.0008) [2023-12-27 06:05:18,927][20347] Updated weights for policy 0, policy_version 61504 (0.0008) [2023-12-27 06:05:18,983][20347] Updated weights for policy 0, policy_version 61514 (0.0008) [2023-12-27 06:05:19,141][20381] Updated weights for policy 1, policy_version 61422 (0.0006) [2023-12-27 06:05:19,202][20381] Updated weights for policy 1, policy_version 61432 (0.0007) [2023-12-27 06:05:19,268][20381] Updated weights for policy 1, policy_version 61442 (0.0008) [2023-12-27 06:05:19,694][20347] Updated weights for policy 0, policy_version 61524 (0.0007) [2023-12-27 06:05:19,752][20347] Updated weights for policy 0, policy_version 61534 (0.0008) [2023-12-27 06:05:19,813][20347] Updated weights for policy 0, policy_version 61544 (0.0006) [2023-12-27 06:05:19,941][20381] Updated weights for policy 1, policy_version 61452 (0.0005) [2023-12-27 06:05:20,005][20381] Updated weights for policy 1, policy_version 61462 (0.0006) [2023-12-27 06:05:20,059][20381] Updated weights for policy 1, policy_version 61472 (0.0008) [2023-12-27 06:05:20,496][20347] Updated weights for policy 0, policy_version 61554 (0.0008) [2023-12-27 06:05:20,554][20347] Updated weights for policy 0, policy_version 61564 (0.0007) [2023-12-27 06:05:20,617][20347] Updated weights for policy 0, policy_version 61574 (0.0007) [2023-12-27 06:05:20,682][20347] Updated weights for policy 0, policy_version 61584 (0.0008) [2023-12-27 06:05:20,763][20381] Updated weights for policy 1, policy_version 61482 (0.0007) [2023-12-27 06:05:20,810][20381] Updated weights for policy 1, policy_version 61492 (0.0005) [2023-12-27 06:05:20,864][20381] Updated weights for policy 1, policy_version 61502 (0.0008) [2023-12-27 06:05:20,921][20381] Updated weights for policy 1, policy_version 61512 (0.0007) [2023-12-27 06:05:21,377][20347] Updated weights for policy 0, policy_version 61594 (0.0009) [2023-12-27 06:05:21,438][20347] Updated weights for policy 0, policy_version 61604 (0.0008) [2023-12-27 06:05:21,493][20347] Updated weights for policy 0, policy_version 61614 (0.0006) [2023-12-27 06:05:21,628][20381] Updated weights for policy 1, policy_version 61522 (0.0009) [2023-12-27 06:05:21,692][20381] Updated weights for policy 1, policy_version 61532 (0.0008) [2023-12-27 06:05:21,764][20381] Updated weights for policy 1, policy_version 61542 (0.0009) [2023-12-27 06:05:22,197][20347] Updated weights for policy 0, policy_version 61624 (0.0008) [2023-12-27 06:05:22,260][20347] Updated weights for policy 0, policy_version 61634 (0.0007) [2023-12-27 06:05:22,326][20347] Updated weights for policy 0, policy_version 61644 (0.0009) [2023-12-27 06:05:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 31547392. Throughput: 0: 10370.7, 1: 10380.5. Samples: 31516508. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) [2023-12-27 06:05:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000061544_15761408.pth... [2023-12-27 06:05:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000061648_15785984.pth... [2023-12-27 06:05:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000060360_15458304.pth [2023-12-27 06:05:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000060400_15466496.pth [2023-12-27 06:05:22,448][20381] Updated weights for policy 1, policy_version 61552 (0.0008) [2023-12-27 06:05:22,508][20381] Updated weights for policy 1, policy_version 61562 (0.0008) [2023-12-27 06:05:22,568][20381] Updated weights for policy 1, policy_version 61572 (0.0008) [2023-12-27 06:05:23,032][20347] Updated weights for policy 0, policy_version 61654 (0.0007) [2023-12-27 06:05:23,086][20347] Updated weights for policy 0, policy_version 61664 (0.0009) [2023-12-27 06:05:23,139][20347] Updated weights for policy 0, policy_version 61674 (0.0009) [2023-12-27 06:05:23,271][20381] Updated weights for policy 1, policy_version 61582 (0.0006) [2023-12-27 06:05:23,314][20381] Updated weights for policy 1, policy_version 61592 (0.0007) [2023-12-27 06:05:23,366][20381] Updated weights for policy 1, policy_version 61602 (0.0007) [2023-12-27 06:05:23,815][20347] Updated weights for policy 0, policy_version 61684 (0.0008) [2023-12-27 06:05:23,860][20347] Updated weights for policy 0, policy_version 61694 (0.0007) [2023-12-27 06:05:23,905][20347] Updated weights for policy 0, policy_version 61704 (0.0008) [2023-12-27 06:05:24,064][20381] Updated weights for policy 1, policy_version 61612 (0.0007) [2023-12-27 06:05:24,121][20381] Updated weights for policy 1, policy_version 61622 (0.0008) [2023-12-27 06:05:24,172][20381] Updated weights for policy 1, policy_version 61632 (0.0008) [2023-12-27 06:05:24,649][20347] Updated weights for policy 0, policy_version 61714 (0.0008) [2023-12-27 06:05:24,705][20347] Updated weights for policy 0, policy_version 61724 (0.0007) [2023-12-27 06:05:24,754][20347] Updated weights for policy 0, policy_version 61734 (0.0007) [2023-12-27 06:05:24,768][20381] Updated weights for policy 1, policy_version 61642 (0.0008) [2023-12-27 06:05:24,810][20347] Updated weights for policy 0, policy_version 61744 (0.0007) [2023-12-27 06:05:24,822][20381] Updated weights for policy 1, policy_version 61652 (0.0007) [2023-12-27 06:05:24,875][20381] Updated weights for policy 1, policy_version 61662 (0.0008) [2023-12-27 06:05:24,927][20381] Updated weights for policy 1, policy_version 61672 (0.0007) [2023-12-27 06:05:25,486][20347] Updated weights for policy 0, policy_version 61754 (0.0007) [2023-12-27 06:05:25,530][20347] Updated weights for policy 0, policy_version 61764 (0.0009) [2023-12-27 06:05:25,575][20347] Updated weights for policy 0, policy_version 61774 (0.0009) [2023-12-27 06:05:25,626][20381] Updated weights for policy 1, policy_version 61682 (0.0009) [2023-12-27 06:05:25,675][20381] Updated weights for policy 1, policy_version 61692 (0.0008) [2023-12-27 06:05:25,728][20381] Updated weights for policy 1, policy_version 61702 (0.0006) [2023-12-27 06:05:26,152][20347] Updated weights for policy 0, policy_version 61784 (0.0006) [2023-12-27 06:05:26,198][20347] Updated weights for policy 0, policy_version 61794 (0.0006) [2023-12-27 06:05:26,250][20347] Updated weights for policy 0, policy_version 61804 (0.0007) [2023-12-27 06:05:26,360][20381] Updated weights for policy 1, policy_version 61712 (0.0009) [2023-12-27 06:05:26,407][20381] Updated weights for policy 1, policy_version 61722 (0.0010) [2023-12-27 06:05:26,459][20381] Updated weights for policy 1, policy_version 61732 (0.0010) [2023-12-27 06:05:26,946][20347] Updated weights for policy 0, policy_version 61814 (0.0007) [2023-12-27 06:05:26,997][20347] Updated weights for policy 0, policy_version 61824 (0.0007) [2023-12-27 06:05:27,047][20347] Updated weights for policy 0, policy_version 61834 (0.0007) [2023-12-27 06:05:27,107][20381] Updated weights for policy 1, policy_version 61742 (0.0007) [2023-12-27 06:05:27,162][20381] Updated weights for policy 1, policy_version 61752 (0.0006) [2023-12-27 06:05:27,216][20381] Updated weights for policy 1, policy_version 61762 (0.0010) [2023-12-27 06:05:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 31653888. Throughput: 0: 10416.9, 1: 10344.7. Samples: 31639512. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:27,765][20347] Updated weights for policy 0, policy_version 61844 (0.0007) [2023-12-27 06:05:27,818][20347] Updated weights for policy 0, policy_version 61854 (0.0007) [2023-12-27 06:05:27,849][20381] Updated weights for policy 1, policy_version 61772 (0.0008) [2023-12-27 06:05:27,870][20347] Updated weights for policy 0, policy_version 61864 (0.0006) [2023-12-27 06:05:27,897][20381] Updated weights for policy 1, policy_version 61782 (0.0005) [2023-12-27 06:05:27,945][20381] Updated weights for policy 1, policy_version 61792 (0.0006) [2023-12-27 06:05:28,563][20347] Updated weights for policy 0, policy_version 61874 (0.0008) [2023-12-27 06:05:28,611][20381] Updated weights for policy 1, policy_version 61802 (0.0007) [2023-12-27 06:05:28,623][20347] Updated weights for policy 0, policy_version 61884 (0.0007) [2023-12-27 06:05:28,672][20381] Updated weights for policy 1, policy_version 61812 (0.0006) [2023-12-27 06:05:28,686][20347] Updated weights for policy 0, policy_version 61894 (0.0006) [2023-12-27 06:05:28,728][20381] Updated weights for policy 1, policy_version 61822 (0.0008) [2023-12-27 06:05:28,742][20347] Updated weights for policy 0, policy_version 61904 (0.0005) [2023-12-27 06:05:28,777][20381] Updated weights for policy 1, policy_version 61832 (0.0010) [2023-12-27 06:05:29,333][20347] Updated weights for policy 0, policy_version 61914 (0.0007) [2023-12-27 06:05:29,360][20381] Updated weights for policy 1, policy_version 61842 (0.0007) [2023-12-27 06:05:29,393][20347] Updated weights for policy 0, policy_version 61924 (0.0006) [2023-12-27 06:05:29,418][20381] Updated weights for policy 1, policy_version 61852 (0.0007) [2023-12-27 06:05:29,448][20347] Updated weights for policy 0, policy_version 61934 (0.0007) [2023-12-27 06:05:29,474][20381] Updated weights for policy 1, policy_version 61862 (0.0005) [2023-12-27 06:05:30,109][20347] Updated weights for policy 0, policy_version 61944 (0.0007) [2023-12-27 06:05:30,132][20381] Updated weights for policy 1, policy_version 61872 (0.0006) [2023-12-27 06:05:30,163][20347] Updated weights for policy 0, policy_version 61954 (0.0006) [2023-12-27 06:05:30,188][20381] Updated weights for policy 1, policy_version 61882 (0.0008) [2023-12-27 06:05:30,216][20347] Updated weights for policy 0, policy_version 61964 (0.0005) [2023-12-27 06:05:30,239][20381] Updated weights for policy 1, policy_version 61892 (0.0007) [2023-12-27 06:05:30,905][20381] Updated weights for policy 1, policy_version 61902 (0.0009) [2023-12-27 06:05:30,922][20347] Updated weights for policy 0, policy_version 61974 (0.0005) [2023-12-27 06:05:30,955][20381] Updated weights for policy 1, policy_version 61912 (0.0010) [2023-12-27 06:05:30,976][20347] Updated weights for policy 0, policy_version 61984 (0.0005) [2023-12-27 06:05:31,002][20381] Updated weights for policy 1, policy_version 61922 (0.0011) [2023-12-27 06:05:31,036][20347] Updated weights for policy 0, policy_version 61994 (0.0007) [2023-12-27 06:05:31,772][20381] Updated weights for policy 1, policy_version 61932 (0.0011) [2023-12-27 06:05:31,790][20347] Updated weights for policy 0, policy_version 62004 (0.0007) [2023-12-27 06:05:31,822][20381] Updated weights for policy 1, policy_version 61942 (0.0011) [2023-12-27 06:05:31,850][20347] Updated weights for policy 0, policy_version 62014 (0.0006) [2023-12-27 06:05:31,873][20381] Updated weights for policy 1, policy_version 61952 (0.0011) [2023-12-27 06:05:31,913][20347] Updated weights for policy 0, policy_version 62024 (0.0006) [2023-12-27 06:05:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 31752192. Throughput: 0: 10363.3, 1: 10337.4. Samples: 31762652. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:32,618][20347] Updated weights for policy 0, policy_version 62034 (0.0007) [2023-12-27 06:05:32,632][20381] Updated weights for policy 1, policy_version 61962 (0.0010) [2023-12-27 06:05:32,680][20347] Updated weights for policy 0, policy_version 62044 (0.0008) [2023-12-27 06:05:32,689][20381] Updated weights for policy 1, policy_version 61972 (0.0008) [2023-12-27 06:05:32,729][20347] Updated weights for policy 0, policy_version 62054 (0.0006) [2023-12-27 06:05:32,742][20381] Updated weights for policy 1, policy_version 61982 (0.0007) [2023-12-27 06:05:32,779][20347] Updated weights for policy 0, policy_version 62064 (0.0006) [2023-12-27 06:05:32,797][20381] Updated weights for policy 1, policy_version 61992 (0.0006) [2023-12-27 06:05:33,438][20347] Updated weights for policy 0, policy_version 62074 (0.0006) [2023-12-27 06:05:33,438][20381] Updated weights for policy 1, policy_version 62002 (0.0010) [2023-12-27 06:05:33,484][20347] Updated weights for policy 0, policy_version 62084 (0.0006) [2023-12-27 06:05:33,493][20381] Updated weights for policy 1, policy_version 62012 (0.0010) [2023-12-27 06:05:33,527][20347] Updated weights for policy 0, policy_version 62094 (0.0006) [2023-12-27 06:05:33,549][20381] Updated weights for policy 1, policy_version 62022 (0.0010) [2023-12-27 06:05:34,146][20381] Updated weights for policy 1, policy_version 62032 (0.0007) [2023-12-27 06:05:34,186][20347] Updated weights for policy 0, policy_version 62104 (0.0007) [2023-12-27 06:05:34,207][20381] Updated weights for policy 1, policy_version 62042 (0.0006) [2023-12-27 06:05:34,244][20347] Updated weights for policy 0, policy_version 62114 (0.0006) [2023-12-27 06:05:34,266][20381] Updated weights for policy 1, policy_version 62052 (0.0007) [2023-12-27 06:05:34,304][20347] Updated weights for policy 0, policy_version 62124 (0.0007) [2023-12-27 06:05:34,905][20381] Updated weights for policy 1, policy_version 62062 (0.0008) [2023-12-27 06:05:34,938][20347] Updated weights for policy 0, policy_version 62134 (0.0006) [2023-12-27 06:05:34,958][20381] Updated weights for policy 1, policy_version 62072 (0.0007) [2023-12-27 06:05:34,986][20347] Updated weights for policy 0, policy_version 62144 (0.0006) [2023-12-27 06:05:35,019][20381] Updated weights for policy 1, policy_version 62082 (0.0006) [2023-12-27 06:05:35,040][20347] Updated weights for policy 0, policy_version 62154 (0.0005) [2023-12-27 06:05:35,669][20347] Updated weights for policy 0, policy_version 62164 (0.0007) [2023-12-27 06:05:35,696][20381] Updated weights for policy 1, policy_version 62092 (0.0007) [2023-12-27 06:05:35,723][20347] Updated weights for policy 0, policy_version 62174 (0.0010) [2023-12-27 06:05:35,762][20381] Updated weights for policy 1, policy_version 62102 (0.0008) [2023-12-27 06:05:35,780][20347] Updated weights for policy 0, policy_version 62184 (0.0009) [2023-12-27 06:05:35,821][20381] Updated weights for policy 1, policy_version 62112 (0.0010) [2023-12-27 06:05:36,431][20347] Updated weights for policy 0, policy_version 62194 (0.0009) [2023-12-27 06:05:36,490][20347] Updated weights for policy 0, policy_version 62204 (0.0008) [2023-12-27 06:05:36,514][20381] Updated weights for policy 1, policy_version 62122 (0.0006) [2023-12-27 06:05:36,542][20347] Updated weights for policy 0, policy_version 62214 (0.0010) [2023-12-27 06:05:36,578][20381] Updated weights for policy 1, policy_version 62132 (0.0006) [2023-12-27 06:05:36,594][20347] Updated weights for policy 0, policy_version 62224 (0.0010) [2023-12-27 06:05:36,640][20381] Updated weights for policy 1, policy_version 62142 (0.0007) [2023-12-27 06:05:36,700][20381] Updated weights for policy 1, policy_version 62152 (0.0008) [2023-12-27 06:05:37,249][20347] Updated weights for policy 0, policy_version 62234 (0.0010) [2023-12-27 06:05:37,307][20347] Updated weights for policy 0, policy_version 62244 (0.0010) [2023-12-27 06:05:37,359][20347] Updated weights for policy 0, policy_version 62254 (0.0010) [2023-12-27 06:05:37,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 31850496. Throughput: 0: 10414.7, 1: 10325.1. Samples: 31826076. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:37,368][20381] Updated weights for policy 1, policy_version 62162 (0.0007) [2023-12-27 06:05:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000062256_15941632.pth... [2023-12-27 06:05:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000061040_15630336.pth [2023-12-27 06:05:37,424][20381] Updated weights for policy 1, policy_version 62172 (0.0008) [2023-12-27 06:05:37,477][20381] Updated weights for policy 1, policy_version 62182 (0.0009) [2023-12-27 06:05:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000062184_15925248.pth... [2023-12-27 06:05:37,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000060968_15613952.pth [2023-12-27 06:05:37,996][20347] Updated weights for policy 0, policy_version 62264 (0.0006) [2023-12-27 06:05:38,051][20347] Updated weights for policy 0, policy_version 62274 (0.0005) [2023-12-27 06:05:38,105][20347] Updated weights for policy 0, policy_version 62284 (0.0007) [2023-12-27 06:05:38,179][20381] Updated weights for policy 1, policy_version 62192 (0.0006) [2023-12-27 06:05:38,237][20381] Updated weights for policy 1, policy_version 62202 (0.0007) [2023-12-27 06:05:38,291][20381] Updated weights for policy 1, policy_version 62212 (0.0007) [2023-12-27 06:05:38,751][20347] Updated weights for policy 0, policy_version 62294 (0.0009) [2023-12-27 06:05:38,800][20347] Updated weights for policy 0, policy_version 62304 (0.0010) [2023-12-27 06:05:38,855][20347] Updated weights for policy 0, policy_version 62314 (0.0010) [2023-12-27 06:05:38,976][20381] Updated weights for policy 1, policy_version 62222 (0.0008) [2023-12-27 06:05:39,038][20381] Updated weights for policy 1, policy_version 62232 (0.0007) [2023-12-27 06:05:39,106][20381] Updated weights for policy 1, policy_version 62242 (0.0008) [2023-12-27 06:05:39,507][20347] Updated weights for policy 0, policy_version 62324 (0.0008) [2023-12-27 06:05:39,574][20347] Updated weights for policy 0, policy_version 62334 (0.0008) [2023-12-27 06:05:39,634][20347] Updated weights for policy 0, policy_version 62344 (0.0007) [2023-12-27 06:05:39,835][20381] Updated weights for policy 1, policy_version 62252 (0.0007) [2023-12-27 06:05:39,896][20381] Updated weights for policy 1, policy_version 62262 (0.0006) [2023-12-27 06:05:39,962][20381] Updated weights for policy 1, policy_version 62272 (0.0011) [2023-12-27 06:05:40,336][20347] Updated weights for policy 0, policy_version 62354 (0.0008) [2023-12-27 06:05:40,393][20347] Updated weights for policy 0, policy_version 62364 (0.0007) [2023-12-27 06:05:40,442][20347] Updated weights for policy 0, policy_version 62374 (0.0008) [2023-12-27 06:05:40,493][20347] Updated weights for policy 0, policy_version 62384 (0.0008) [2023-12-27 06:05:40,625][20381] Updated weights for policy 1, policy_version 62282 (0.0011) [2023-12-27 06:05:40,676][20381] Updated weights for policy 1, policy_version 62292 (0.0011) [2023-12-27 06:05:40,719][20381] Updated weights for policy 1, policy_version 62302 (0.0011) [2023-12-27 06:05:40,775][20381] Updated weights for policy 1, policy_version 62312 (0.0011) [2023-12-27 06:05:41,215][20347] Updated weights for policy 0, policy_version 62394 (0.0008) [2023-12-27 06:05:41,277][20347] Updated weights for policy 0, policy_version 62404 (0.0008) [2023-12-27 06:05:41,329][20347] Updated weights for policy 0, policy_version 62414 (0.0007) [2023-12-27 06:05:41,528][20381] Updated weights for policy 1, policy_version 62322 (0.0008) [2023-12-27 06:05:41,588][20381] Updated weights for policy 1, policy_version 62332 (0.0011) [2023-12-27 06:05:41,652][20381] Updated weights for policy 1, policy_version 62342 (0.0008) [2023-12-27 06:05:42,042][20347] Updated weights for policy 0, policy_version 62424 (0.0008) [2023-12-27 06:05:42,090][20347] Updated weights for policy 0, policy_version 62434 (0.0007) [2023-12-27 06:05:42,145][20347] Updated weights for policy 0, policy_version 62444 (0.0007) [2023-12-27 06:05:42,317][20381] Updated weights for policy 1, policy_version 62352 (0.0007) [2023-12-27 06:05:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 31956992. Throughput: 0: 10381.7, 1: 10317.7. Samples: 31948760. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:42,380][20381] Updated weights for policy 1, policy_version 62362 (0.0007) [2023-12-27 06:05:42,434][20381] Updated weights for policy 1, policy_version 62372 (0.0007) [2023-12-27 06:05:42,878][20347] Updated weights for policy 0, policy_version 62454 (0.0007) [2023-12-27 06:05:42,939][20347] Updated weights for policy 0, policy_version 62464 (0.0007) [2023-12-27 06:05:42,983][20347] Updated weights for policy 0, policy_version 62474 (0.0007) [2023-12-27 06:05:43,088][20381] Updated weights for policy 1, policy_version 62382 (0.0005) [2023-12-27 06:05:43,144][20381] Updated weights for policy 1, policy_version 62392 (0.0005) [2023-12-27 06:05:43,199][20381] Updated weights for policy 1, policy_version 62402 (0.0006) [2023-12-27 06:05:43,654][20347] Updated weights for policy 0, policy_version 62484 (0.0008) [2023-12-27 06:05:43,711][20347] Updated weights for policy 0, policy_version 62494 (0.0008) [2023-12-27 06:05:43,763][20347] Updated weights for policy 0, policy_version 62504 (0.0006) [2023-12-27 06:05:43,834][20381] Updated weights for policy 1, policy_version 62412 (0.0007) [2023-12-27 06:05:43,897][20381] Updated weights for policy 1, policy_version 62422 (0.0009) [2023-12-27 06:05:43,952][20381] Updated weights for policy 1, policy_version 62432 (0.0007) [2023-12-27 06:05:44,352][20347] Updated weights for policy 0, policy_version 62514 (0.0007) [2023-12-27 06:05:44,401][20347] Updated weights for policy 0, policy_version 62524 (0.0005) [2023-12-27 06:05:44,449][20347] Updated weights for policy 0, policy_version 62534 (0.0005) [2023-12-27 06:05:44,494][20347] Updated weights for policy 0, policy_version 62544 (0.0005) [2023-12-27 06:05:44,618][20381] Updated weights for policy 1, policy_version 62442 (0.0009) [2023-12-27 06:05:44,666][20381] Updated weights for policy 1, policy_version 62452 (0.0010) [2023-12-27 06:05:44,715][20381] Updated weights for policy 1, policy_version 62462 (0.0010) [2023-12-27 06:05:44,767][20381] Updated weights for policy 1, policy_version 62472 (0.0010) [2023-12-27 06:05:45,196][20347] Updated weights for policy 0, policy_version 62554 (0.0010) [2023-12-27 06:05:45,246][20347] Updated weights for policy 0, policy_version 62564 (0.0010) [2023-12-27 06:05:45,298][20347] Updated weights for policy 0, policy_version 62574 (0.0010) [2023-12-27 06:05:45,492][20381] Updated weights for policy 1, policy_version 62482 (0.0005) [2023-12-27 06:05:45,541][20381] Updated weights for policy 1, policy_version 62492 (0.0007) [2023-12-27 06:05:45,590][20381] Updated weights for policy 1, policy_version 62502 (0.0010) [2023-12-27 06:05:45,945][20347] Updated weights for policy 0, policy_version 62584 (0.0006) [2023-12-27 06:05:45,996][20347] Updated weights for policy 0, policy_version 62594 (0.0005) [2023-12-27 06:05:46,044][20347] Updated weights for policy 0, policy_version 62604 (0.0005) [2023-12-27 06:05:46,234][20381] Updated weights for policy 1, policy_version 62512 (0.0007) [2023-12-27 06:05:46,283][20381] Updated weights for policy 1, policy_version 62522 (0.0008) [2023-12-27 06:05:46,334][20381] Updated weights for policy 1, policy_version 62532 (0.0008) [2023-12-27 06:05:46,703][20347] Updated weights for policy 0, policy_version 62614 (0.0010) [2023-12-27 06:05:46,758][20347] Updated weights for policy 0, policy_version 62624 (0.0010) [2023-12-27 06:05:46,819][20347] Updated weights for policy 0, policy_version 62634 (0.0010) [2023-12-27 06:05:46,900][20381] Updated weights for policy 1, policy_version 62542 (0.0007) [2023-12-27 06:05:46,951][20381] Updated weights for policy 1, policy_version 62552 (0.0005) [2023-12-27 06:05:47,007][20381] Updated weights for policy 1, policy_version 62562 (0.0007) [2023-12-27 06:05:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 32063488. Throughput: 0: 10379.1, 1: 10351.4. Samples: 32073948. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:47,498][20347] Updated weights for policy 0, policy_version 62644 (0.0008) [2023-12-27 06:05:47,545][20347] Updated weights for policy 0, policy_version 62654 (0.0008) [2023-12-27 06:05:47,590][20347] Updated weights for policy 0, policy_version 62664 (0.0010) [2023-12-27 06:05:47,668][20381] Updated weights for policy 1, policy_version 62572 (0.0007) [2023-12-27 06:05:47,716][20381] Updated weights for policy 1, policy_version 62582 (0.0010) [2023-12-27 06:05:47,765][20381] Updated weights for policy 1, policy_version 62592 (0.0010) [2023-12-27 06:05:48,256][20347] Updated weights for policy 0, policy_version 62674 (0.0010) [2023-12-27 06:05:48,307][20347] Updated weights for policy 0, policy_version 62684 (0.0010) [2023-12-27 06:05:48,371][20347] Updated weights for policy 0, policy_version 62694 (0.0009) [2023-12-27 06:05:48,431][20347] Updated weights for policy 0, policy_version 62704 (0.0006) [2023-12-27 06:05:48,484][20381] Updated weights for policy 1, policy_version 62602 (0.0010) [2023-12-27 06:05:48,542][20381] Updated weights for policy 1, policy_version 62612 (0.0010) [2023-12-27 06:05:48,591][20381] Updated weights for policy 1, policy_version 62622 (0.0010) [2023-12-27 06:05:48,636][20381] Updated weights for policy 1, policy_version 62632 (0.0010) [2023-12-27 06:05:49,003][20347] Updated weights for policy 0, policy_version 62714 (0.0010) [2023-12-27 06:05:49,055][20347] Updated weights for policy 0, policy_version 62724 (0.0010) [2023-12-27 06:05:49,107][20347] Updated weights for policy 0, policy_version 62734 (0.0006) [2023-12-27 06:05:49,343][20381] Updated weights for policy 1, policy_version 62642 (0.0011) [2023-12-27 06:05:49,405][20381] Updated weights for policy 1, policy_version 62652 (0.0010) [2023-12-27 06:05:49,465][20381] Updated weights for policy 1, policy_version 62662 (0.0010) [2023-12-27 06:05:49,752][20347] Updated weights for policy 0, policy_version 62744 (0.0006) [2023-12-27 06:05:49,810][20347] Updated weights for policy 0, policy_version 62754 (0.0008) [2023-12-27 06:05:49,869][20347] Updated weights for policy 0, policy_version 62764 (0.0008) [2023-12-27 06:05:50,167][20381] Updated weights for policy 1, policy_version 62672 (0.0010) [2023-12-27 06:05:50,224][20381] Updated weights for policy 1, policy_version 62682 (0.0010) [2023-12-27 06:05:50,279][20381] Updated weights for policy 1, policy_version 62692 (0.0010) [2023-12-27 06:05:50,511][20347] Updated weights for policy 0, policy_version 62774 (0.0006) [2023-12-27 06:05:50,568][20347] Updated weights for policy 0, policy_version 62784 (0.0006) [2023-12-27 06:05:50,628][20347] Updated weights for policy 0, policy_version 62794 (0.0007) [2023-12-27 06:05:50,956][20381] Updated weights for policy 1, policy_version 62702 (0.0010) [2023-12-27 06:05:51,012][20381] Updated weights for policy 1, policy_version 62712 (0.0008) [2023-12-27 06:05:51,075][20381] Updated weights for policy 1, policy_version 62722 (0.0007) [2023-12-27 06:05:51,324][20347] Updated weights for policy 0, policy_version 62804 (0.0007) [2023-12-27 06:05:51,387][20347] Updated weights for policy 0, policy_version 62814 (0.0008) [2023-12-27 06:05:51,446][20347] Updated weights for policy 0, policy_version 62824 (0.0007) [2023-12-27 06:05:51,687][20381] Updated weights for policy 1, policy_version 62732 (0.0007) [2023-12-27 06:05:51,749][20381] Updated weights for policy 1, policy_version 62742 (0.0007) [2023-12-27 06:05:51,807][20381] Updated weights for policy 1, policy_version 62752 (0.0007) [2023-12-27 06:05:52,192][20347] Updated weights for policy 0, policy_version 62834 (0.0008) [2023-12-27 06:05:52,246][20347] Updated weights for policy 0, policy_version 62844 (0.0007) [2023-12-27 06:05:52,301][20347] Updated weights for policy 0, policy_version 62854 (0.0008) [2023-12-27 06:05:52,353][20347] Updated weights for policy 0, policy_version 62864 (0.0007) [2023-12-27 06:05:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 32169984. Throughput: 0: 10385.6, 1: 10295.8. Samples: 32136756. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000062760_16072704.pth... [2023-12-27 06:05:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000062864_16097280.pth... [2023-12-27 06:05:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000061544_15761408.pth [2023-12-27 06:05:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000061648_15785984.pth [2023-12-27 06:05:52,430][20381] Updated weights for policy 1, policy_version 62762 (0.0007) [2023-12-27 06:05:52,493][20381] Updated weights for policy 1, policy_version 62772 (0.0008) [2023-12-27 06:05:52,552][20381] Updated weights for policy 1, policy_version 62782 (0.0007) [2023-12-27 06:05:52,616][20381] Updated weights for policy 1, policy_version 62792 (0.0008) [2023-12-27 06:05:53,047][20347] Updated weights for policy 0, policy_version 62874 (0.0008) [2023-12-27 06:05:53,094][20347] Updated weights for policy 0, policy_version 62884 (0.0008) [2023-12-27 06:05:53,147][20347] Updated weights for policy 0, policy_version 62894 (0.0008) [2023-12-27 06:05:53,328][20381] Updated weights for policy 1, policy_version 62802 (0.0008) [2023-12-27 06:05:53,386][20381] Updated weights for policy 1, policy_version 62812 (0.0008) [2023-12-27 06:05:53,438][20381] Updated weights for policy 1, policy_version 62822 (0.0007) [2023-12-27 06:05:53,851][20347] Updated weights for policy 0, policy_version 62904 (0.0008) [2023-12-27 06:05:53,902][20347] Updated weights for policy 0, policy_version 62914 (0.0007) [2023-12-27 06:05:53,948][20347] Updated weights for policy 0, policy_version 62924 (0.0007) [2023-12-27 06:05:54,114][20381] Updated weights for policy 1, policy_version 62832 (0.0008) [2023-12-27 06:05:54,178][20381] Updated weights for policy 1, policy_version 62842 (0.0008) [2023-12-27 06:05:54,235][20381] Updated weights for policy 1, policy_version 62852 (0.0008) [2023-12-27 06:05:54,634][20347] Updated weights for policy 0, policy_version 62934 (0.0006) [2023-12-27 06:05:54,682][20347] Updated weights for policy 0, policy_version 62944 (0.0006) [2023-12-27 06:05:54,727][20347] Updated weights for policy 0, policy_version 62954 (0.0006) [2023-12-27 06:05:54,952][20381] Updated weights for policy 1, policy_version 62862 (0.0007) [2023-12-27 06:05:55,001][20381] Updated weights for policy 1, policy_version 62872 (0.0007) [2023-12-27 06:05:55,047][20381] Updated weights for policy 1, policy_version 62882 (0.0007) [2023-12-27 06:05:55,365][20347] Updated weights for policy 0, policy_version 62964 (0.0007) [2023-12-27 06:05:55,423][20347] Updated weights for policy 0, policy_version 62974 (0.0010) [2023-12-27 06:05:55,480][20347] Updated weights for policy 0, policy_version 62984 (0.0010) [2023-12-27 06:05:55,765][20381] Updated weights for policy 1, policy_version 62892 (0.0008) [2023-12-27 06:05:55,818][20381] Updated weights for policy 1, policy_version 62902 (0.0008) [2023-12-27 06:05:55,867][20381] Updated weights for policy 1, policy_version 62912 (0.0008) [2023-12-27 06:05:56,062][20347] Updated weights for policy 0, policy_version 62994 (0.0007) [2023-12-27 06:05:56,117][20347] Updated weights for policy 0, policy_version 63004 (0.0007) [2023-12-27 06:05:56,175][20347] Updated weights for policy 0, policy_version 63014 (0.0005) [2023-12-27 06:05:56,230][20347] Updated weights for policy 0, policy_version 63024 (0.0006) [2023-12-27 06:05:56,550][20381] Updated weights for policy 1, policy_version 62922 (0.0007) [2023-12-27 06:05:56,600][20381] Updated weights for policy 1, policy_version 62932 (0.0005) [2023-12-27 06:05:56,645][20381] Updated weights for policy 1, policy_version 62942 (0.0005) [2023-12-27 06:05:56,688][20381] Updated weights for policy 1, policy_version 62952 (0.0005) [2023-12-27 06:05:56,755][20347] Updated weights for policy 0, policy_version 63034 (0.0008) [2023-12-27 06:05:56,801][20347] Updated weights for policy 0, policy_version 63044 (0.0007) [2023-12-27 06:05:56,844][20347] Updated weights for policy 0, policy_version 63054 (0.0005) [2023-12-27 06:05:57,333][20381] Updated weights for policy 1, policy_version 62962 (0.0006) [2023-12-27 06:05:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 32268288. Throughput: 0: 10328.4, 1: 10235.9. Samples: 32260176. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:05:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:05:57,387][20381] Updated weights for policy 1, policy_version 62972 (0.0005) [2023-12-27 06:05:57,440][20381] Updated weights for policy 1, policy_version 62982 (0.0007) [2023-12-27 06:05:57,478][20347] Updated weights for policy 0, policy_version 63064 (0.0006) [2023-12-27 06:05:57,535][20347] Updated weights for policy 0, policy_version 63074 (0.0005) [2023-12-27 06:05:57,595][20347] Updated weights for policy 0, policy_version 63084 (0.0005) [2023-12-27 06:05:58,123][20381] Updated weights for policy 1, policy_version 62992 (0.0008) [2023-12-27 06:05:58,182][20347] Updated weights for policy 0, policy_version 63094 (0.0007) [2023-12-27 06:05:58,185][20381] Updated weights for policy 1, policy_version 63002 (0.0007) [2023-12-27 06:05:58,239][20347] Updated weights for policy 0, policy_version 63104 (0.0007) [2023-12-27 06:05:58,243][20381] Updated weights for policy 1, policy_version 63012 (0.0008) [2023-12-27 06:05:58,311][20347] Updated weights for policy 0, policy_version 63114 (0.0007) [2023-12-27 06:05:59,024][20381] Updated weights for policy 1, policy_version 63022 (0.0009) [2023-12-27 06:05:59,076][20347] Updated weights for policy 0, policy_version 63124 (0.0008) [2023-12-27 06:05:59,080][20381] Updated weights for policy 1, policy_version 63032 (0.0006) [2023-12-27 06:05:59,133][20347] Updated weights for policy 0, policy_version 63134 (0.0007) [2023-12-27 06:05:59,145][20381] Updated weights for policy 1, policy_version 63042 (0.0007) [2023-12-27 06:05:59,189][20347] Updated weights for policy 0, policy_version 63144 (0.0006) [2023-12-27 06:05:59,885][20381] Updated weights for policy 1, policy_version 63052 (0.0008) [2023-12-27 06:05:59,919][20347] Updated weights for policy 0, policy_version 63154 (0.0008) [2023-12-27 06:05:59,945][20381] Updated weights for policy 1, policy_version 63062 (0.0008) [2023-12-27 06:05:59,983][20347] Updated weights for policy 0, policy_version 63164 (0.0007) [2023-12-27 06:05:59,999][20381] Updated weights for policy 1, policy_version 63072 (0.0008) [2023-12-27 06:06:00,047][20347] Updated weights for policy 0, policy_version 63174 (0.0007) [2023-12-27 06:06:00,109][20347] Updated weights for policy 0, policy_version 63184 (0.0008) [2023-12-27 06:06:00,578][20381] Updated weights for policy 1, policy_version 63082 (0.0006) [2023-12-27 06:06:00,630][20381] Updated weights for policy 1, policy_version 63092 (0.0006) [2023-12-27 06:06:00,676][20381] Updated weights for policy 1, policy_version 63102 (0.0006) [2023-12-27 06:06:00,696][20347] Updated weights for policy 0, policy_version 63194 (0.0010) [2023-12-27 06:06:00,721][20381] Updated weights for policy 1, policy_version 63112 (0.0007) [2023-12-27 06:06:00,745][20347] Updated weights for policy 0, policy_version 63204 (0.0010) [2023-12-27 06:06:00,796][20347] Updated weights for policy 0, policy_version 63214 (0.0010) [2023-12-27 06:06:01,450][20381] Updated weights for policy 1, policy_version 63122 (0.0009) [2023-12-27 06:06:01,457][20347] Updated weights for policy 0, policy_version 63224 (0.0010) [2023-12-27 06:06:01,511][20381] Updated weights for policy 1, policy_version 63132 (0.0008) [2023-12-27 06:06:01,517][20347] Updated weights for policy 0, policy_version 63234 (0.0010) [2023-12-27 06:06:01,573][20381] Updated weights for policy 1, policy_version 63142 (0.0006) [2023-12-27 06:06:01,578][20347] Updated weights for policy 0, policy_version 63244 (0.0012) [2023-12-27 06:06:02,186][20381] Updated weights for policy 1, policy_version 63152 (0.0006) [2023-12-27 06:06:02,246][20381] Updated weights for policy 1, policy_version 63162 (0.0008) [2023-12-27 06:06:02,303][20347] Updated weights for policy 0, policy_version 63254 (0.0009) [2023-12-27 06:06:02,305][20381] Updated weights for policy 1, policy_version 63172 (0.0007) [2023-12-27 06:06:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 32374784. Throughput: 0: 10362.6, 1: 10287.6. Samples: 32385568. Policy #0 lag: (min: 40.0, avg: 47.7, max: 48.0) [2023-12-27 06:06:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:02,372][20347] Updated weights for policy 0, policy_version 63264 (0.0006) [2023-12-27 06:06:02,431][20347] Updated weights for policy 0, policy_version 63274 (0.0007) [2023-12-27 06:06:03,014][20381] Updated weights for policy 1, policy_version 63182 (0.0008) [2023-12-27 06:06:03,047][20347] Updated weights for policy 0, policy_version 63284 (0.0011) [2023-12-27 06:06:03,068][20381] Updated weights for policy 1, policy_version 63192 (0.0007) [2023-12-27 06:06:03,106][20347] Updated weights for policy 0, policy_version 63294 (0.0010) [2023-12-27 06:06:03,116][20381] Updated weights for policy 1, policy_version 63202 (0.0006) [2023-12-27 06:06:03,169][20347] Updated weights for policy 0, policy_version 63304 (0.0011) [2023-12-27 06:06:03,758][20347] Updated weights for policy 0, policy_version 63314 (0.0009) [2023-12-27 06:06:03,819][20347] Updated weights for policy 0, policy_version 63324 (0.0005) [2023-12-27 06:06:03,860][20381] Updated weights for policy 1, policy_version 63212 (0.0008) [2023-12-27 06:06:03,880][20347] Updated weights for policy 0, policy_version 63334 (0.0010) [2023-12-27 06:06:03,918][20381] Updated weights for policy 1, policy_version 63222 (0.0006) [2023-12-27 06:06:03,940][20347] Updated weights for policy 0, policy_version 63344 (0.0010) [2023-12-27 06:06:03,974][20381] Updated weights for policy 1, policy_version 63232 (0.0007) [2023-12-27 06:06:04,567][20347] Updated weights for policy 0, policy_version 63354 (0.0008) [2023-12-27 06:06:04,618][20347] Updated weights for policy 0, policy_version 63364 (0.0006) [2023-12-27 06:06:04,674][20381] Updated weights for policy 1, policy_version 63242 (0.0007) [2023-12-27 06:06:04,675][20347] Updated weights for policy 0, policy_version 63374 (0.0007) [2023-12-27 06:06:04,727][20381] Updated weights for policy 1, policy_version 63252 (0.0008) [2023-12-27 06:06:04,780][20381] Updated weights for policy 1, policy_version 63262 (0.0008) [2023-12-27 06:06:04,826][20381] Updated weights for policy 1, policy_version 63272 (0.0007) [2023-12-27 06:06:05,379][20347] Updated weights for policy 0, policy_version 63384 (0.0007) [2023-12-27 06:06:05,431][20347] Updated weights for policy 0, policy_version 63394 (0.0008) [2023-12-27 06:06:05,480][20347] Updated weights for policy 0, policy_version 63404 (0.0009) [2023-12-27 06:06:05,537][20381] Updated weights for policy 1, policy_version 63282 (0.0005) [2023-12-27 06:06:05,589][20381] Updated weights for policy 1, policy_version 63292 (0.0005) [2023-12-27 06:06:05,645][20381] Updated weights for policy 1, policy_version 63302 (0.0005) [2023-12-27 06:06:06,213][20347] Updated weights for policy 0, policy_version 63414 (0.0008) [2023-12-27 06:06:06,272][20347] Updated weights for policy 0, policy_version 63424 (0.0009) [2023-12-27 06:06:06,278][20381] Updated weights for policy 1, policy_version 63312 (0.0005) [2023-12-27 06:06:06,328][20347] Updated weights for policy 0, policy_version 63434 (0.0009) [2023-12-27 06:06:06,334][20381] Updated weights for policy 1, policy_version 63322 (0.0007) [2023-12-27 06:06:06,399][20381] Updated weights for policy 1, policy_version 63332 (0.0006) [2023-12-27 06:06:07,014][20347] Updated weights for policy 0, policy_version 63444 (0.0011) [2023-12-27 06:06:07,067][20381] Updated weights for policy 1, policy_version 63342 (0.0007) [2023-12-27 06:06:07,072][20347] Updated weights for policy 0, policy_version 63454 (0.0010) [2023-12-27 06:06:07,124][20381] Updated weights for policy 1, policy_version 63352 (0.0006) [2023-12-27 06:06:07,125][20347] Updated weights for policy 0, policy_version 63464 (0.0008) [2023-12-27 06:06:07,187][20381] Updated weights for policy 1, policy_version 63362 (0.0008) [2023-12-27 06:06:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 32481280. Throughput: 0: 10422.1, 1: 10250.8. Samples: 32446792. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000063368_16228352.pth... [2023-12-27 06:06:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000063472_16252928.pth... [2023-12-27 06:06:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000062184_15925248.pth [2023-12-27 06:06:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000062256_15941632.pth [2023-12-27 06:06:07,684][20347] Updated weights for policy 0, policy_version 63474 (0.0008) [2023-12-27 06:06:07,741][20347] Updated weights for policy 0, policy_version 63484 (0.0005) [2023-12-27 06:06:07,792][20347] Updated weights for policy 0, policy_version 63494 (0.0009) [2023-12-27 06:06:07,840][20381] Updated weights for policy 1, policy_version 63372 (0.0008) [2023-12-27 06:06:07,849][20347] Updated weights for policy 0, policy_version 63504 (0.0010) [2023-12-27 06:06:07,886][20381] Updated weights for policy 1, policy_version 63382 (0.0009) [2023-12-27 06:06:07,941][20381] Updated weights for policy 1, policy_version 63392 (0.0005) [2023-12-27 06:06:08,489][20347] Updated weights for policy 0, policy_version 63514 (0.0010) [2023-12-27 06:06:08,547][20347] Updated weights for policy 0, policy_version 63524 (0.0010) [2023-12-27 06:06:08,608][20381] Updated weights for policy 1, policy_version 63402 (0.0006) [2023-12-27 06:06:08,609][20347] Updated weights for policy 0, policy_version 63534 (0.0010) [2023-12-27 06:06:08,669][20381] Updated weights for policy 1, policy_version 63412 (0.0011) [2023-12-27 06:06:08,718][20381] Updated weights for policy 1, policy_version 63422 (0.0011) [2023-12-27 06:06:08,776][20381] Updated weights for policy 1, policy_version 63432 (0.0008) [2023-12-27 06:06:09,283][20347] Updated weights for policy 0, policy_version 63544 (0.0011) [2023-12-27 06:06:09,339][20347] Updated weights for policy 0, policy_version 63554 (0.0011) [2023-12-27 06:06:09,403][20347] Updated weights for policy 0, policy_version 63564 (0.0008) [2023-12-27 06:06:09,479][20381] Updated weights for policy 1, policy_version 63442 (0.0005) [2023-12-27 06:06:09,536][20381] Updated weights for policy 1, policy_version 63452 (0.0007) [2023-12-27 06:06:09,590][20381] Updated weights for policy 1, policy_version 63462 (0.0007) [2023-12-27 06:06:10,080][20347] Updated weights for policy 0, policy_version 63574 (0.0010) [2023-12-27 06:06:10,139][20347] Updated weights for policy 0, policy_version 63584 (0.0010) [2023-12-27 06:06:10,194][20347] Updated weights for policy 0, policy_version 63594 (0.0011) [2023-12-27 06:06:10,226][20381] Updated weights for policy 1, policy_version 63472 (0.0006) [2023-12-27 06:06:10,272][20381] Updated weights for policy 1, policy_version 63482 (0.0007) [2023-12-27 06:06:10,323][20381] Updated weights for policy 1, policy_version 63492 (0.0006) [2023-12-27 06:06:10,883][20347] Updated weights for policy 0, policy_version 63604 (0.0010) [2023-12-27 06:06:10,939][20347] Updated weights for policy 0, policy_version 63614 (0.0010) [2023-12-27 06:06:10,944][20381] Updated weights for policy 1, policy_version 63502 (0.0006) [2023-12-27 06:06:10,991][20347] Updated weights for policy 0, policy_version 63624 (0.0010) [2023-12-27 06:06:11,004][20381] Updated weights for policy 1, policy_version 63512 (0.0006) [2023-12-27 06:06:11,069][20381] Updated weights for policy 1, policy_version 63522 (0.0006) [2023-12-27 06:06:11,681][20347] Updated weights for policy 0, policy_version 63634 (0.0009) [2023-12-27 06:06:11,746][20347] Updated weights for policy 0, policy_version 63644 (0.0007) [2023-12-27 06:06:11,758][20381] Updated weights for policy 1, policy_version 63532 (0.0006) [2023-12-27 06:06:11,806][20347] Updated weights for policy 0, policy_version 63654 (0.0007) [2023-12-27 06:06:11,815][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000007 [2023-12-27 06:06:11,824][20381] Updated weights for policy 1, policy_version 63542 (0.0006) [2023-12-27 06:06:11,891][20381] Updated weights for policy 1, policy_version 63552 (0.0007) [2023-12-27 06:06:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 32579584. Throughput: 0: 10436.9, 1: 10280.4. Samples: 32571792. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:12,518][20347] Updated weights for policy 0, policy_version 63664 (0.0007) [2023-12-27 06:06:12,577][20347] Updated weights for policy 0, policy_version 63674 (0.0007) [2023-12-27 06:06:12,586][20381] Updated weights for policy 1, policy_version 63562 (0.0007) [2023-12-27 06:06:12,633][20347] Updated weights for policy 0, policy_version 63684 (0.0008) [2023-12-27 06:06:12,646][20381] Updated weights for policy 1, policy_version 63572 (0.0007) [2023-12-27 06:06:12,708][20381] Updated weights for policy 1, policy_version 63582 (0.0008) [2023-12-27 06:06:12,768][20381] Updated weights for policy 1, policy_version 63592 (0.0008) [2023-12-27 06:06:13,298][20347] Updated weights for policy 0, policy_version 63694 (0.0007) [2023-12-27 06:06:13,358][20347] Updated weights for policy 0, policy_version 63704 (0.0008) [2023-12-27 06:06:13,370][20381] Updated weights for policy 1, policy_version 63602 (0.0006) [2023-12-27 06:06:13,426][20347] Updated weights for policy 0, policy_version 63714 (0.0008) [2023-12-27 06:06:13,426][20381] Updated weights for policy 1, policy_version 63612 (0.0009) [2023-12-27 06:06:13,480][20381] Updated weights for policy 1, policy_version 63622 (0.0006) [2023-12-27 06:06:14,024][20381] Updated weights for policy 1, policy_version 63632 (0.0007) [2023-12-27 06:06:14,088][20381] Updated weights for policy 1, policy_version 63642 (0.0008) [2023-12-27 06:06:14,091][20347] Updated weights for policy 0, policy_version 63724 (0.0008) [2023-12-27 06:06:14,144][20381] Updated weights for policy 1, policy_version 63652 (0.0007) [2023-12-27 06:06:14,146][20347] Updated weights for policy 0, policy_version 63734 (0.0005) [2023-12-27 06:06:14,212][20347] Updated weights for policy 0, policy_version 63744 (0.0007) [2023-12-27 06:06:14,822][20381] Updated weights for policy 1, policy_version 63662 (0.0005) [2023-12-27 06:06:14,836][20347] Updated weights for policy 0, policy_version 63754 (0.0008) [2023-12-27 06:06:14,885][20381] Updated weights for policy 1, policy_version 63672 (0.0008) [2023-12-27 06:06:14,896][20347] Updated weights for policy 0, policy_version 63764 (0.0008) [2023-12-27 06:06:14,944][20381] Updated weights for policy 1, policy_version 63682 (0.0007) [2023-12-27 06:06:14,953][20347] Updated weights for policy 0, policy_version 63774 (0.0007) [2023-12-27 06:06:15,019][20347] Updated weights for policy 0, policy_version 63784 (0.0008) [2023-12-27 06:06:15,554][20381] Updated weights for policy 1, policy_version 63692 (0.0007) [2023-12-27 06:06:15,604][20381] Updated weights for policy 1, policy_version 63702 (0.0007) [2023-12-27 06:06:15,654][20381] Updated weights for policy 1, policy_version 63712 (0.0005) [2023-12-27 06:06:15,726][20347] Updated weights for policy 0, policy_version 63794 (0.0007) [2023-12-27 06:06:15,779][20347] Updated weights for policy 0, policy_version 63804 (0.0008) [2023-12-27 06:06:15,839][20347] Updated weights for policy 0, policy_version 63814 (0.0006) [2023-12-27 06:06:16,312][20381] Updated weights for policy 1, policy_version 63722 (0.0006) [2023-12-27 06:06:16,366][20381] Updated weights for policy 1, policy_version 63732 (0.0006) [2023-12-27 06:06:16,420][20381] Updated weights for policy 1, policy_version 63742 (0.0010) [2023-12-27 06:06:16,454][20347] Updated weights for policy 0, policy_version 63824 (0.0007) [2023-12-27 06:06:16,471][20381] Updated weights for policy 1, policy_version 63752 (0.0005) [2023-12-27 06:06:16,510][20347] Updated weights for policy 0, policy_version 63834 (0.0008) [2023-12-27 06:06:16,568][20347] Updated weights for policy 0, policy_version 63844 (0.0007) [2023-12-27 06:06:17,135][20381] Updated weights for policy 1, policy_version 63762 (0.0010) [2023-12-27 06:06:17,186][20381] Updated weights for policy 1, policy_version 63772 (0.0010) [2023-12-27 06:06:17,245][20381] Updated weights for policy 1, policy_version 63782 (0.0011) [2023-12-27 06:06:17,264][20347] Updated weights for policy 0, policy_version 63854 (0.0007) [2023-12-27 06:06:17,319][20347] Updated weights for policy 0, policy_version 63864 (0.0007) [2023-12-27 06:06:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 32686080. Throughput: 0: 10467.2, 1: 10303.8. Samples: 32697348. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:17,366][20347] Updated weights for policy 0, policy_version 63874 (0.0007) [2023-12-27 06:06:17,947][20381] Updated weights for policy 1, policy_version 63792 (0.0008) [2023-12-27 06:06:17,998][20381] Updated weights for policy 1, policy_version 63802 (0.0007) [2023-12-27 06:06:18,054][20381] Updated weights for policy 1, policy_version 63812 (0.0005) [2023-12-27 06:06:18,060][20347] Updated weights for policy 0, policy_version 63884 (0.0007) [2023-12-27 06:06:18,114][20347] Updated weights for policy 0, policy_version 63894 (0.0008) [2023-12-27 06:06:18,166][20347] Updated weights for policy 0, policy_version 63904 (0.0006) [2023-12-27 06:06:18,738][20381] Updated weights for policy 1, policy_version 63822 (0.0007) [2023-12-27 06:06:18,764][20347] Updated weights for policy 0, policy_version 63914 (0.0006) [2023-12-27 06:06:18,795][20381] Updated weights for policy 1, policy_version 63832 (0.0008) [2023-12-27 06:06:18,826][20347] Updated weights for policy 0, policy_version 63924 (0.0007) [2023-12-27 06:06:18,848][20381] Updated weights for policy 1, policy_version 63842 (0.0007) [2023-12-27 06:06:18,890][20347] Updated weights for policy 0, policy_version 63934 (0.0007) [2023-12-27 06:06:18,951][20347] Updated weights for policy 0, policy_version 63944 (0.0007) [2023-12-27 06:06:19,577][20347] Updated weights for policy 0, policy_version 63954 (0.0007) [2023-12-27 06:06:19,603][20381] Updated weights for policy 1, policy_version 63852 (0.0006) [2023-12-27 06:06:19,635][20347] Updated weights for policy 0, policy_version 63964 (0.0007) [2023-12-27 06:06:19,656][20381] Updated weights for policy 1, policy_version 63862 (0.0007) [2023-12-27 06:06:19,690][20347] Updated weights for policy 0, policy_version 63974 (0.0007) [2023-12-27 06:06:19,707][20381] Updated weights for policy 1, policy_version 63872 (0.0007) [2023-12-27 06:06:20,385][20347] Updated weights for policy 0, policy_version 63984 (0.0007) [2023-12-27 06:06:20,433][20381] Updated weights for policy 1, policy_version 63882 (0.0008) [2023-12-27 06:06:20,444][20347] Updated weights for policy 0, policy_version 63994 (0.0007) [2023-12-27 06:06:20,489][20381] Updated weights for policy 1, policy_version 63892 (0.0008) [2023-12-27 06:06:20,492][20347] Updated weights for policy 0, policy_version 64004 (0.0007) [2023-12-27 06:06:20,537][20381] Updated weights for policy 1, policy_version 63902 (0.0006) [2023-12-27 06:06:20,588][20381] Updated weights for policy 1, policy_version 63912 (0.0008) [2023-12-27 06:06:21,142][20347] Updated weights for policy 0, policy_version 64014 (0.0009) [2023-12-27 06:06:21,203][20347] Updated weights for policy 0, policy_version 64024 (0.0011) [2023-12-27 06:06:21,263][20347] Updated weights for policy 0, policy_version 64034 (0.0011) [2023-12-27 06:06:21,394][20381] Updated weights for policy 1, policy_version 63923 (0.0007) [2023-12-27 06:06:21,459][20381] Updated weights for policy 1, policy_version 63933 (0.0007) [2023-12-27 06:06:21,527][20381] Updated weights for policy 1, policy_version 63943 (0.0008) [2023-12-27 06:06:21,979][20347] Updated weights for policy 0, policy_version 64044 (0.0008) [2023-12-27 06:06:22,027][20347] Updated weights for policy 0, policy_version 64054 (0.0007) [2023-12-27 06:06:22,075][20347] Updated weights for policy 0, policy_version 64064 (0.0008) [2023-12-27 06:06:22,211][20381] Updated weights for policy 1, policy_version 63953 (0.0007) [2023-12-27 06:06:22,275][20381] Updated weights for policy 1, policy_version 63963 (0.0008) [2023-12-27 06:06:22,328][20381] Updated weights for policy 1, policy_version 63973 (0.0006) [2023-12-27 06:06:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 32792576. Throughput: 0: 10467.5, 1: 10254.0. Samples: 32758540. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000063976_16384000.pth... [2023-12-27 06:06:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000064072_16408576.pth... [2023-12-27 06:06:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000062760_16072704.pth [2023-12-27 06:06:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000062864_16097280.pth [2023-12-27 06:06:22,778][20347] Updated weights for policy 0, policy_version 64074 (0.0007) [2023-12-27 06:06:22,834][20347] Updated weights for policy 0, policy_version 64084 (0.0008) [2023-12-27 06:06:22,894][20347] Updated weights for policy 0, policy_version 64094 (0.0007) [2023-12-27 06:06:22,953][20347] Updated weights for policy 0, policy_version 64104 (0.0008) [2023-12-27 06:06:23,039][20381] Updated weights for policy 1, policy_version 63983 (0.0007) [2023-12-27 06:06:23,100][20381] Updated weights for policy 1, policy_version 63993 (0.0007) [2023-12-27 06:06:23,159][20381] Updated weights for policy 1, policy_version 64003 (0.0007) [2023-12-27 06:06:23,648][20347] Updated weights for policy 0, policy_version 64114 (0.0005) [2023-12-27 06:06:23,712][20347] Updated weights for policy 0, policy_version 64124 (0.0006) [2023-12-27 06:06:23,776][20347] Updated weights for policy 0, policy_version 64134 (0.0006) [2023-12-27 06:06:23,850][20381] Updated weights for policy 1, policy_version 64013 (0.0006) [2023-12-27 06:06:23,907][20381] Updated weights for policy 1, policy_version 64023 (0.0008) [2023-12-27 06:06:23,965][20381] Updated weights for policy 1, policy_version 64033 (0.0008) [2023-12-27 06:06:24,449][20347] Updated weights for policy 0, policy_version 64144 (0.0008) [2023-12-27 06:06:24,503][20347] Updated weights for policy 0, policy_version 64154 (0.0007) [2023-12-27 06:06:24,551][20347] Updated weights for policy 0, policy_version 64164 (0.0007) [2023-12-27 06:06:24,592][20381] Updated weights for policy 1, policy_version 64043 (0.0008) [2023-12-27 06:06:24,646][20381] Updated weights for policy 1, policy_version 64053 (0.0007) [2023-12-27 06:06:24,695][20381] Updated weights for policy 1, policy_version 64063 (0.0007) [2023-12-27 06:06:25,237][20347] Updated weights for policy 0, policy_version 64174 (0.0007) [2023-12-27 06:06:25,290][20347] Updated weights for policy 0, policy_version 64184 (0.0007) [2023-12-27 06:06:25,339][20381] Updated weights for policy 1, policy_version 64073 (0.0005) [2023-12-27 06:06:25,348][20347] Updated weights for policy 0, policy_version 64194 (0.0007) [2023-12-27 06:06:25,392][20381] Updated weights for policy 1, policy_version 64083 (0.0006) [2023-12-27 06:06:25,439][20381] Updated weights for policy 1, policy_version 64093 (0.0007) [2023-12-27 06:06:25,487][20381] Updated weights for policy 1, policy_version 64103 (0.0008) [2023-12-27 06:06:26,031][20347] Updated weights for policy 0, policy_version 64204 (0.0006) [2023-12-27 06:06:26,090][20347] Updated weights for policy 0, policy_version 64214 (0.0008) [2023-12-27 06:06:26,147][20347] Updated weights for policy 0, policy_version 64224 (0.0007) [2023-12-27 06:06:26,182][20381] Updated weights for policy 1, policy_version 64113 (0.0007) [2023-12-27 06:06:26,230][20381] Updated weights for policy 1, policy_version 64123 (0.0007) [2023-12-27 06:06:26,281][20381] Updated weights for policy 1, policy_version 64133 (0.0008) [2023-12-27 06:06:26,782][20347] Updated weights for policy 0, policy_version 64234 (0.0006) [2023-12-27 06:06:26,839][20347] Updated weights for policy 0, policy_version 64244 (0.0007) [2023-12-27 06:06:26,893][20347] Updated weights for policy 0, policy_version 64254 (0.0007) [2023-12-27 06:06:26,940][20347] Updated weights for policy 0, policy_version 64264 (0.0007) [2023-12-27 06:06:26,985][20381] Updated weights for policy 1, policy_version 64143 (0.0007) [2023-12-27 06:06:27,028][20381] Updated weights for policy 1, policy_version 64153 (0.0007) [2023-12-27 06:06:27,096][20381] Updated weights for policy 1, policy_version 64163 (0.0007) [2023-12-27 06:06:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 32890880. Throughput: 0: 10414.7, 1: 10268.5. Samples: 32879504. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:27,601][20347] Updated weights for policy 0, policy_version 64274 (0.0007) [2023-12-27 06:06:27,651][20347] Updated weights for policy 0, policy_version 64284 (0.0007) [2023-12-27 06:06:27,706][20347] Updated weights for policy 0, policy_version 64295 (0.0010) [2023-12-27 06:06:27,775][20381] Updated weights for policy 1, policy_version 64173 (0.0007) [2023-12-27 06:06:27,835][20381] Updated weights for policy 1, policy_version 64183 (0.0008) [2023-12-27 06:06:27,897][20381] Updated weights for policy 1, policy_version 64193 (0.0008) [2023-12-27 06:06:28,336][20347] Updated weights for policy 0, policy_version 64305 (0.0006) [2023-12-27 06:06:28,395][20347] Updated weights for policy 0, policy_version 64315 (0.0007) [2023-12-27 06:06:28,458][20347] Updated weights for policy 0, policy_version 64325 (0.0006) [2023-12-27 06:06:28,623][20381] Updated weights for policy 1, policy_version 64203 (0.0008) [2023-12-27 06:06:28,679][20381] Updated weights for policy 1, policy_version 64213 (0.0008) [2023-12-27 06:06:28,731][20381] Updated weights for policy 1, policy_version 64223 (0.0008) [2023-12-27 06:06:29,024][20347] Updated weights for policy 0, policy_version 64335 (0.0007) [2023-12-27 06:06:29,084][20347] Updated weights for policy 0, policy_version 64345 (0.0008) [2023-12-27 06:06:29,142][20347] Updated weights for policy 0, policy_version 64355 (0.0008) [2023-12-27 06:06:29,457][20381] Updated weights for policy 1, policy_version 64233 (0.0008) [2023-12-27 06:06:29,515][20381] Updated weights for policy 1, policy_version 64243 (0.0011) [2023-12-27 06:06:29,569][20381] Updated weights for policy 1, policy_version 64253 (0.0010) [2023-12-27 06:06:29,624][20381] Updated weights for policy 1, policy_version 64263 (0.0010) [2023-12-27 06:06:29,745][20347] Updated weights for policy 0, policy_version 64365 (0.0006) [2023-12-27 06:06:29,801][20347] Updated weights for policy 0, policy_version 64375 (0.0007) [2023-12-27 06:06:29,866][20347] Updated weights for policy 0, policy_version 64385 (0.0008) [2023-12-27 06:06:30,322][20381] Updated weights for policy 1, policy_version 64273 (0.0006) [2023-12-27 06:06:30,385][20381] Updated weights for policy 1, policy_version 64283 (0.0007) [2023-12-27 06:06:30,441][20381] Updated weights for policy 1, policy_version 64293 (0.0006) [2023-12-27 06:06:30,540][20347] Updated weights for policy 0, policy_version 64395 (0.0008) [2023-12-27 06:06:30,601][20347] Updated weights for policy 0, policy_version 64405 (0.0008) [2023-12-27 06:06:30,664][20347] Updated weights for policy 0, policy_version 64415 (0.0007) [2023-12-27 06:06:31,109][20381] Updated weights for policy 1, policy_version 64303 (0.0007) [2023-12-27 06:06:31,174][20381] Updated weights for policy 1, policy_version 64313 (0.0007) [2023-12-27 06:06:31,231][20381] Updated weights for policy 1, policy_version 64323 (0.0008) [2023-12-27 06:06:31,236][20347] Updated weights for policy 0, policy_version 64425 (0.0006) [2023-12-27 06:06:31,294][20347] Updated weights for policy 0, policy_version 64435 (0.0006) [2023-12-27 06:06:31,354][20347] Updated weights for policy 0, policy_version 64445 (0.0006) [2023-12-27 06:06:31,414][20347] Updated weights for policy 0, policy_version 64455 (0.0006) [2023-12-27 06:06:31,955][20381] Updated weights for policy 1, policy_version 64333 (0.0008) [2023-12-27 06:06:32,008][20381] Updated weights for policy 1, policy_version 64343 (0.0007) [2023-12-27 06:06:32,034][20347] Updated weights for policy 0, policy_version 64465 (0.0006) [2023-12-27 06:06:32,062][20381] Updated weights for policy 1, policy_version 64353 (0.0007) [2023-12-27 06:06:32,097][20347] Updated weights for policy 0, policy_version 64475 (0.0006) [2023-12-27 06:06:32,159][20347] Updated weights for policy 0, policy_version 64485 (0.0007) [2023-12-27 06:06:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 32997376. Throughput: 0: 10499.8, 1: 10181.3. Samples: 33004600. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:32,719][20381] Updated weights for policy 1, policy_version 64363 (0.0008) [2023-12-27 06:06:32,781][20381] Updated weights for policy 1, policy_version 64373 (0.0008) [2023-12-27 06:06:32,842][20381] Updated weights for policy 1, policy_version 64383 (0.0006) [2023-12-27 06:06:32,849][20347] Updated weights for policy 0, policy_version 64495 (0.0007) [2023-12-27 06:06:32,906][20347] Updated weights for policy 0, policy_version 64505 (0.0008) [2023-12-27 06:06:32,966][20347] Updated weights for policy 0, policy_version 64515 (0.0009) [2023-12-27 06:06:33,444][20381] Updated weights for policy 1, policy_version 64393 (0.0006) [2023-12-27 06:06:33,510][20381] Updated weights for policy 1, policy_version 64403 (0.0008) [2023-12-27 06:06:33,566][20381] Updated weights for policy 1, policy_version 64413 (0.0008) [2023-12-27 06:06:33,609][20347] Updated weights for policy 0, policy_version 64525 (0.0008) [2023-12-27 06:06:33,625][20381] Updated weights for policy 1, policy_version 64423 (0.0008) [2023-12-27 06:06:33,666][20347] Updated weights for policy 0, policy_version 64535 (0.0007) [2023-12-27 06:06:33,718][20347] Updated weights for policy 0, policy_version 64545 (0.0007) [2023-12-27 06:06:34,340][20381] Updated weights for policy 1, policy_version 64433 (0.0007) [2023-12-27 06:06:34,397][20381] Updated weights for policy 1, policy_version 64443 (0.0008) [2023-12-27 06:06:34,403][20347] Updated weights for policy 0, policy_version 64555 (0.0006) [2023-12-27 06:06:34,461][20381] Updated weights for policy 1, policy_version 64453 (0.0008) [2023-12-27 06:06:34,461][20347] Updated weights for policy 0, policy_version 64565 (0.0007) [2023-12-27 06:06:34,524][20347] Updated weights for policy 0, policy_version 64575 (0.0008) [2023-12-27 06:06:35,099][20381] Updated weights for policy 1, policy_version 64463 (0.0007) [2023-12-27 06:06:35,153][20381] Updated weights for policy 1, policy_version 64473 (0.0008) [2023-12-27 06:06:35,209][20381] Updated weights for policy 1, policy_version 64483 (0.0005) [2023-12-27 06:06:35,259][20347] Updated weights for policy 0, policy_version 64585 (0.0008) [2023-12-27 06:06:35,314][20347] Updated weights for policy 0, policy_version 64595 (0.0008) [2023-12-27 06:06:35,365][20347] Updated weights for policy 0, policy_version 64605 (0.0008) [2023-12-27 06:06:35,414][20347] Updated weights for policy 0, policy_version 64615 (0.0007) [2023-12-27 06:06:35,874][20381] Updated weights for policy 1, policy_version 64493 (0.0007) [2023-12-27 06:06:35,932][20381] Updated weights for policy 1, policy_version 64503 (0.0008) [2023-12-27 06:06:35,983][20381] Updated weights for policy 1, policy_version 64513 (0.0007) [2023-12-27 06:06:36,066][20347] Updated weights for policy 0, policy_version 64625 (0.0007) [2023-12-27 06:06:36,128][20347] Updated weights for policy 0, policy_version 64635 (0.0007) [2023-12-27 06:06:36,181][20347] Updated weights for policy 0, policy_version 64645 (0.0006) [2023-12-27 06:06:36,714][20381] Updated weights for policy 1, policy_version 64523 (0.0008) [2023-12-27 06:06:36,773][20381] Updated weights for policy 1, policy_version 64533 (0.0008) [2023-12-27 06:06:36,804][20347] Updated weights for policy 0, policy_version 64655 (0.0006) [2023-12-27 06:06:36,833][20381] Updated weights for policy 1, policy_version 64543 (0.0006) [2023-12-27 06:06:36,861][20347] Updated weights for policy 0, policy_version 64665 (0.0006) [2023-12-27 06:06:36,922][20347] Updated weights for policy 0, policy_version 64675 (0.0008) [2023-12-27 06:06:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 33095680. Throughput: 0: 10441.5, 1: 10211.4. Samples: 33066136. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000064552_16531456.pth... [2023-12-27 06:06:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000064680_16564224.pth... [2023-12-27 06:06:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000063368_16228352.pth [2023-12-27 06:06:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000063472_16252928.pth [2023-12-27 06:06:37,451][20381] Updated weights for policy 1, policy_version 64553 (0.0008) [2023-12-27 06:06:37,504][20381] Updated weights for policy 1, policy_version 64563 (0.0008) [2023-12-27 06:06:37,562][20381] Updated weights for policy 1, policy_version 64573 (0.0010) [2023-12-27 06:06:37,618][20381] Updated weights for policy 1, policy_version 64583 (0.0011) [2023-12-27 06:06:37,620][20347] Updated weights for policy 0, policy_version 64685 (0.0009) [2023-12-27 06:06:37,681][20347] Updated weights for policy 0, policy_version 64695 (0.0007) [2023-12-27 06:06:37,742][20347] Updated weights for policy 0, policy_version 64705 (0.0007) [2023-12-27 06:06:38,278][20381] Updated weights for policy 1, policy_version 64593 (0.0010) [2023-12-27 06:06:38,339][20381] Updated weights for policy 1, policy_version 64603 (0.0009) [2023-12-27 06:06:38,399][20381] Updated weights for policy 1, policy_version 64613 (0.0006) [2023-12-27 06:06:38,467][20347] Updated weights for policy 0, policy_version 64715 (0.0008) [2023-12-27 06:06:38,520][20347] Updated weights for policy 0, policy_version 64725 (0.0008) [2023-12-27 06:06:38,579][20347] Updated weights for policy 0, policy_version 64735 (0.0008) [2023-12-27 06:06:39,062][20381] Updated weights for policy 1, policy_version 64623 (0.0007) [2023-12-27 06:06:39,111][20381] Updated weights for policy 1, policy_version 64633 (0.0007) [2023-12-27 06:06:39,165][20381] Updated weights for policy 1, policy_version 64643 (0.0010) [2023-12-27 06:06:39,287][20347] Updated weights for policy 0, policy_version 64745 (0.0007) [2023-12-27 06:06:39,335][20347] Updated weights for policy 0, policy_version 64755 (0.0007) [2023-12-27 06:06:39,399][20347] Updated weights for policy 0, policy_version 64765 (0.0007) [2023-12-27 06:06:39,457][20347] Updated weights for policy 0, policy_version 64775 (0.0008) [2023-12-27 06:06:39,855][20381] Updated weights for policy 1, policy_version 64653 (0.0009) [2023-12-27 06:06:39,916][20381] Updated weights for policy 1, policy_version 64663 (0.0007) [2023-12-27 06:06:39,968][20381] Updated weights for policy 1, policy_version 64673 (0.0005) [2023-12-27 06:06:40,190][20347] Updated weights for policy 0, policy_version 64785 (0.0010) [2023-12-27 06:06:40,249][20347] Updated weights for policy 0, policy_version 64795 (0.0010) [2023-12-27 06:06:40,315][20347] Updated weights for policy 0, policy_version 64805 (0.0009) [2023-12-27 06:06:40,643][20381] Updated weights for policy 1, policy_version 64683 (0.0008) [2023-12-27 06:06:40,704][20381] Updated weights for policy 1, policy_version 64693 (0.0011) [2023-12-27 06:06:40,763][20381] Updated weights for policy 1, policy_version 64703 (0.0011) [2023-12-27 06:06:40,922][20347] Updated weights for policy 0, policy_version 64815 (0.0007) [2023-12-27 06:06:40,981][20347] Updated weights for policy 0, policy_version 64825 (0.0006) [2023-12-27 06:06:41,044][20347] Updated weights for policy 0, policy_version 64835 (0.0009) [2023-12-27 06:06:41,439][20381] Updated weights for policy 1, policy_version 64713 (0.0010) [2023-12-27 06:06:41,496][20381] Updated weights for policy 1, policy_version 64723 (0.0008) [2023-12-27 06:06:41,555][20381] Updated weights for policy 1, policy_version 64733 (0.0008) [2023-12-27 06:06:41,615][20381] Updated weights for policy 1, policy_version 64743 (0.0008) [2023-12-27 06:06:41,678][20347] Updated weights for policy 0, policy_version 64845 (0.0010) [2023-12-27 06:06:41,740][20347] Updated weights for policy 0, policy_version 64855 (0.0010) [2023-12-27 06:06:41,798][20347] Updated weights for policy 0, policy_version 64865 (0.0009) [2023-12-27 06:06:42,351][20381] Updated weights for policy 1, policy_version 64753 (0.0007) [2023-12-27 06:06:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 33193984. Throughput: 0: 10419.3, 1: 10224.9. Samples: 33189164. Policy #0 lag: (min: 6.0, avg: 20.4, max: 38.0) [2023-12-27 06:06:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:42,411][20381] Updated weights for policy 1, policy_version 64763 (0.0006) [2023-12-27 06:06:42,455][20347] Updated weights for policy 0, policy_version 64875 (0.0006) [2023-12-27 06:06:42,464][20381] Updated weights for policy 1, policy_version 64773 (0.0006) [2023-12-27 06:06:42,513][20347] Updated weights for policy 0, policy_version 64885 (0.0008) [2023-12-27 06:06:42,576][20347] Updated weights for policy 0, policy_version 64895 (0.0007) [2023-12-27 06:06:43,155][20381] Updated weights for policy 1, policy_version 64783 (0.0009) [2023-12-27 06:06:43,201][20381] Updated weights for policy 1, policy_version 64793 (0.0010) [2023-12-27 06:06:43,203][20347] Updated weights for policy 0, policy_version 64905 (0.0008) [2023-12-27 06:06:43,252][20381] Updated weights for policy 1, policy_version 64803 (0.0010) [2023-12-27 06:06:43,261][20347] Updated weights for policy 0, policy_version 64915 (0.0010) [2023-12-27 06:06:43,318][20347] Updated weights for policy 0, policy_version 64925 (0.0010) [2023-12-27 06:06:43,369][20347] Updated weights for policy 0, policy_version 64935 (0.0010) [2023-12-27 06:06:43,828][20381] Updated weights for policy 1, policy_version 64813 (0.0008) [2023-12-27 06:06:43,883][20381] Updated weights for policy 1, policy_version 64823 (0.0009) [2023-12-27 06:06:43,934][20381] Updated weights for policy 1, policy_version 64833 (0.0010) [2023-12-27 06:06:44,007][20347] Updated weights for policy 0, policy_version 64945 (0.0010) [2023-12-27 06:06:44,056][20347] Updated weights for policy 0, policy_version 64955 (0.0010) [2023-12-27 06:06:44,112][20347] Updated weights for policy 0, policy_version 64965 (0.0010) [2023-12-27 06:06:44,625][20381] Updated weights for policy 1, policy_version 64843 (0.0010) [2023-12-27 06:06:44,685][20381] Updated weights for policy 1, policy_version 64853 (0.0010) [2023-12-27 06:06:44,743][20381] Updated weights for policy 1, policy_version 64863 (0.0010) [2023-12-27 06:06:44,749][20347] Updated weights for policy 0, policy_version 64975 (0.0010) [2023-12-27 06:06:44,808][20347] Updated weights for policy 0, policy_version 64985 (0.0010) [2023-12-27 06:06:44,861][20347] Updated weights for policy 0, policy_version 64995 (0.0009) [2023-12-27 06:06:45,587][20381] Updated weights for policy 1, policy_version 64873 (0.0009) [2023-12-27 06:06:45,646][20381] Updated weights for policy 1, policy_version 64883 (0.0009) [2023-12-27 06:06:45,647][20347] Updated weights for policy 0, policy_version 65005 (0.0007) [2023-12-27 06:06:45,702][20381] Updated weights for policy 1, policy_version 64893 (0.0011) [2023-12-27 06:06:45,734][20347] Updated weights for policy 0, policy_version 65015 (0.0005) [2023-12-27 06:06:45,757][20381] Updated weights for policy 1, policy_version 64903 (0.0007) [2023-12-27 06:06:45,794][20347] Updated weights for policy 0, policy_version 65025 (0.0005) [2023-12-27 06:06:46,349][20347] Updated weights for policy 0, policy_version 65035 (0.0007) [2023-12-27 06:06:46,386][20381] Updated weights for policy 1, policy_version 64913 (0.0007) [2023-12-27 06:06:46,410][20347] Updated weights for policy 0, policy_version 65045 (0.0010) [2023-12-27 06:06:46,435][20381] Updated weights for policy 1, policy_version 64923 (0.0010) [2023-12-27 06:06:46,458][20347] Updated weights for policy 0, policy_version 65055 (0.0010) [2023-12-27 06:06:46,483][20381] Updated weights for policy 1, policy_version 64933 (0.0010) [2023-12-27 06:06:47,133][20347] Updated weights for policy 0, policy_version 65065 (0.0010) [2023-12-27 06:06:47,145][20381] Updated weights for policy 1, policy_version 64943 (0.0010) [2023-12-27 06:06:47,191][20347] Updated weights for policy 0, policy_version 65075 (0.0011) [2023-12-27 06:06:47,205][20381] Updated weights for policy 1, policy_version 64953 (0.0011) [2023-12-27 06:06:47,242][20347] Updated weights for policy 0, policy_version 65085 (0.0009) [2023-12-27 06:06:47,265][20381] Updated weights for policy 1, policy_version 64963 (0.0010) [2023-12-27 06:06:47,302][20347] Updated weights for policy 0, policy_version 65095 (0.0009) [2023-12-27 06:06:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 33308672. Throughput: 0: 10402.2, 1: 10190.4. Samples: 33312236. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:06:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:47,885][20347] Updated weights for policy 0, policy_version 65105 (0.0010) [2023-12-27 06:06:47,922][20381] Updated weights for policy 1, policy_version 64973 (0.0010) [2023-12-27 06:06:47,940][20347] Updated weights for policy 0, policy_version 65115 (0.0010) [2023-12-27 06:06:47,974][20381] Updated weights for policy 1, policy_version 64983 (0.0010) [2023-12-27 06:06:47,994][20347] Updated weights for policy 0, policy_version 65125 (0.0010) [2023-12-27 06:06:48,026][20381] Updated weights for policy 1, policy_version 64993 (0.0010) [2023-12-27 06:06:48,672][20381] Updated weights for policy 1, policy_version 65003 (0.0009) [2023-12-27 06:06:48,676][20347] Updated weights for policy 0, policy_version 65135 (0.0010) [2023-12-27 06:06:48,732][20381] Updated weights for policy 1, policy_version 65013 (0.0005) [2023-12-27 06:06:48,738][20347] Updated weights for policy 0, policy_version 65145 (0.0010) [2023-12-27 06:06:48,793][20381] Updated weights for policy 1, policy_version 65023 (0.0007) [2023-12-27 06:06:48,799][20347] Updated weights for policy 0, policy_version 65155 (0.0009) [2023-12-27 06:06:49,429][20381] Updated weights for policy 1, policy_version 65033 (0.0010) [2023-12-27 06:06:49,491][20347] Updated weights for policy 0, policy_version 65165 (0.0009) [2023-12-27 06:06:49,493][20381] Updated weights for policy 1, policy_version 65043 (0.0008) [2023-12-27 06:06:49,538][20347] Updated weights for policy 0, policy_version 65175 (0.0007) [2023-12-27 06:06:49,557][20381] Updated weights for policy 1, policy_version 65053 (0.0008) [2023-12-27 06:06:49,585][20347] Updated weights for policy 0, policy_version 65185 (0.0007) [2023-12-27 06:06:49,619][20381] Updated weights for policy 1, policy_version 65063 (0.0008) [2023-12-27 06:06:50,253][20347] Updated weights for policy 0, policy_version 65195 (0.0008) [2023-12-27 06:06:50,274][20381] Updated weights for policy 1, policy_version 65073 (0.0006) [2023-12-27 06:06:50,310][20347] Updated weights for policy 0, policy_version 65205 (0.0010) [2023-12-27 06:06:50,331][20381] Updated weights for policy 1, policy_version 65083 (0.0005) [2023-12-27 06:06:50,370][20347] Updated weights for policy 0, policy_version 65215 (0.0010) [2023-12-27 06:06:50,387][20381] Updated weights for policy 1, policy_version 65093 (0.0005) [2023-12-27 06:06:50,978][20347] Updated weights for policy 0, policy_version 65225 (0.0010) [2023-12-27 06:06:51,037][20347] Updated weights for policy 0, policy_version 65235 (0.0011) [2023-12-27 06:06:51,041][20381] Updated weights for policy 1, policy_version 65103 (0.0006) [2023-12-27 06:06:51,097][20347] Updated weights for policy 0, policy_version 65245 (0.0007) [2023-12-27 06:06:51,105][20381] Updated weights for policy 1, policy_version 65113 (0.0007) [2023-12-27 06:06:51,160][20347] Updated weights for policy 0, policy_version 65255 (0.0010) [2023-12-27 06:06:51,167][20381] Updated weights for policy 1, policy_version 65123 (0.0007) [2023-12-27 06:06:51,802][20347] Updated weights for policy 0, policy_version 65265 (0.0008) [2023-12-27 06:06:51,858][20347] Updated weights for policy 0, policy_version 65275 (0.0009) [2023-12-27 06:06:51,858][20381] Updated weights for policy 1, policy_version 65133 (0.0009) [2023-12-27 06:06:51,914][20347] Updated weights for policy 0, policy_version 65285 (0.0007) [2023-12-27 06:06:51,915][20381] Updated weights for policy 1, policy_version 65143 (0.0006) [2023-12-27 06:06:51,972][20381] Updated weights for policy 1, policy_version 65153 (0.0006) [2023-12-27 06:06:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 33406976. Throughput: 0: 10391.5, 1: 10251.2. Samples: 33375716. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:06:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000065160_16687104.pth... [2023-12-27 06:06:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000065288_16719872.pth... [2023-12-27 06:06:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000063976_16384000.pth [2023-12-27 06:06:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000064072_16408576.pth [2023-12-27 06:06:52,622][20381] Updated weights for policy 1, policy_version 65163 (0.0008) [2023-12-27 06:06:52,678][20381] Updated weights for policy 1, policy_version 65173 (0.0008) [2023-12-27 06:06:52,689][20347] Updated weights for policy 0, policy_version 65295 (0.0007) [2023-12-27 06:06:52,735][20381] Updated weights for policy 1, policy_version 65183 (0.0005) [2023-12-27 06:06:52,754][20347] Updated weights for policy 0, policy_version 65305 (0.0006) [2023-12-27 06:06:52,824][20347] Updated weights for policy 0, policy_version 65315 (0.0005) [2023-12-27 06:06:53,356][20381] Updated weights for policy 1, policy_version 65193 (0.0006) [2023-12-27 06:06:53,409][20381] Updated weights for policy 1, policy_version 65203 (0.0008) [2023-12-27 06:06:53,469][20381] Updated weights for policy 1, policy_version 65213 (0.0008) [2023-12-27 06:06:53,482][20347] Updated weights for policy 0, policy_version 65325 (0.0006) [2023-12-27 06:06:53,517][20381] Updated weights for policy 1, policy_version 65223 (0.0007) [2023-12-27 06:06:53,542][20347] Updated weights for policy 0, policy_version 65335 (0.0006) [2023-12-27 06:06:53,598][20347] Updated weights for policy 0, policy_version 65345 (0.0008) [2023-12-27 06:06:54,194][20347] Updated weights for policy 0, policy_version 65355 (0.0007) [2023-12-27 06:06:54,219][20381] Updated weights for policy 1, policy_version 65233 (0.0010) [2023-12-27 06:06:54,252][20347] Updated weights for policy 0, policy_version 65365 (0.0005) [2023-12-27 06:06:54,279][20381] Updated weights for policy 1, policy_version 65243 (0.0010) [2023-12-27 06:06:54,315][20347] Updated weights for policy 0, policy_version 65375 (0.0006) [2023-12-27 06:06:54,335][20381] Updated weights for policy 1, policy_version 65253 (0.0010) [2023-12-27 06:06:54,969][20381] Updated weights for policy 1, policy_version 65263 (0.0009) [2023-12-27 06:06:54,974][20347] Updated weights for policy 0, policy_version 65385 (0.0006) [2023-12-27 06:06:55,019][20381] Updated weights for policy 1, policy_version 65273 (0.0006) [2023-12-27 06:06:55,035][20347] Updated weights for policy 0, policy_version 65395 (0.0007) [2023-12-27 06:06:55,074][20381] Updated weights for policy 1, policy_version 65283 (0.0007) [2023-12-27 06:06:55,090][20347] Updated weights for policy 0, policy_version 65405 (0.0010) [2023-12-27 06:06:55,145][20347] Updated weights for policy 0, policy_version 65415 (0.0008) [2023-12-27 06:06:55,777][20381] Updated weights for policy 1, policy_version 65293 (0.0007) [2023-12-27 06:06:55,782][20347] Updated weights for policy 0, policy_version 65425 (0.0007) [2023-12-27 06:06:55,825][20381] Updated weights for policy 1, policy_version 65303 (0.0005) [2023-12-27 06:06:55,827][20347] Updated weights for policy 0, policy_version 65435 (0.0007) [2023-12-27 06:06:55,873][20381] Updated weights for policy 1, policy_version 65313 (0.0005) [2023-12-27 06:06:55,879][20347] Updated weights for policy 0, policy_version 65445 (0.0007) [2023-12-27 06:06:56,426][20381] Updated weights for policy 1, policy_version 65323 (0.0007) [2023-12-27 06:06:56,471][20381] Updated weights for policy 1, policy_version 65333 (0.0007) [2023-12-27 06:06:56,514][20381] Updated weights for policy 1, policy_version 65343 (0.0006) [2023-12-27 06:06:56,634][20347] Updated weights for policy 0, policy_version 65455 (0.0007) [2023-12-27 06:06:56,691][20347] Updated weights for policy 0, policy_version 65465 (0.0007) [2023-12-27 06:06:56,736][20347] Updated weights for policy 0, policy_version 65475 (0.0007) [2023-12-27 06:06:57,158][20381] Updated weights for policy 1, policy_version 65353 (0.0005) [2023-12-27 06:06:57,206][20381] Updated weights for policy 1, policy_version 65363 (0.0008) [2023-12-27 06:06:57,254][20381] Updated weights for policy 1, policy_version 65373 (0.0007) [2023-12-27 06:06:57,307][20381] Updated weights for policy 1, policy_version 65383 (0.0007) [2023-12-27 06:06:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 33513472. Throughput: 0: 10394.5, 1: 10253.7. Samples: 33500960. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:06:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:06:57,443][20347] Updated weights for policy 0, policy_version 65485 (0.0008) [2023-12-27 06:06:57,496][20347] Updated weights for policy 0, policy_version 65495 (0.0006) [2023-12-27 06:06:57,551][20347] Updated weights for policy 0, policy_version 65505 (0.0005) [2023-12-27 06:06:57,921][20381] Updated weights for policy 1, policy_version 65393 (0.0006) [2023-12-27 06:06:57,978][20381] Updated weights for policy 1, policy_version 65403 (0.0005) [2023-12-27 06:06:58,040][20381] Updated weights for policy 1, policy_version 65413 (0.0008) [2023-12-27 06:06:58,124][20347] Updated weights for policy 0, policy_version 65515 (0.0006) [2023-12-27 06:06:58,180][20347] Updated weights for policy 0, policy_version 65525 (0.0008) [2023-12-27 06:06:58,236][20347] Updated weights for policy 0, policy_version 65535 (0.0006) [2023-12-27 06:06:58,754][20381] Updated weights for policy 1, policy_version 65423 (0.0009) [2023-12-27 06:06:58,829][20381] Updated weights for policy 1, policy_version 65434 (0.0011) [2023-12-27 06:06:58,884][20381] Updated weights for policy 1, policy_version 65444 (0.0007) [2023-12-27 06:06:59,014][20347] Updated weights for policy 0, policy_version 65545 (0.0007) [2023-12-27 06:06:59,073][20347] Updated weights for policy 0, policy_version 65555 (0.0008) [2023-12-27 06:06:59,137][20347] Updated weights for policy 0, policy_version 65565 (0.0007) [2023-12-27 06:06:59,189][20347] Updated weights for policy 0, policy_version 65575 (0.0006) [2023-12-27 06:06:59,557][20381] Updated weights for policy 1, policy_version 65454 (0.0006) [2023-12-27 06:06:59,618][20381] Updated weights for policy 1, policy_version 65464 (0.0006) [2023-12-27 06:06:59,664][20381] Updated weights for policy 1, policy_version 65474 (0.0006) [2023-12-27 06:06:59,877][20347] Updated weights for policy 0, policy_version 65585 (0.0008) [2023-12-27 06:06:59,937][20347] Updated weights for policy 0, policy_version 65595 (0.0008) [2023-12-27 06:06:59,992][20347] Updated weights for policy 0, policy_version 65605 (0.0007) [2023-12-27 06:07:00,301][20381] Updated weights for policy 1, policy_version 65484 (0.0009) [2023-12-27 06:07:00,362][20381] Updated weights for policy 1, policy_version 65494 (0.0010) [2023-12-27 06:07:00,421][20381] Updated weights for policy 1, policy_version 65504 (0.0008) [2023-12-27 06:07:00,649][20347] Updated weights for policy 0, policy_version 65615 (0.0006) [2023-12-27 06:07:00,710][20347] Updated weights for policy 0, policy_version 65625 (0.0007) [2023-12-27 06:07:00,759][20347] Updated weights for policy 0, policy_version 65635 (0.0007) [2023-12-27 06:07:01,094][20381] Updated weights for policy 1, policy_version 65514 (0.0007) [2023-12-27 06:07:01,152][20381] Updated weights for policy 1, policy_version 65524 (0.0008) [2023-12-27 06:07:01,209][20381] Updated weights for policy 1, policy_version 65534 (0.0011) [2023-12-27 06:07:01,277][20381] Updated weights for policy 1, policy_version 65544 (0.0010) [2023-12-27 06:07:01,489][20347] Updated weights for policy 0, policy_version 65645 (0.0008) [2023-12-27 06:07:01,549][20347] Updated weights for policy 0, policy_version 65655 (0.0006) [2023-12-27 06:07:01,612][20347] Updated weights for policy 0, policy_version 65665 (0.0007) [2023-12-27 06:07:01,920][20381] Updated weights for policy 1, policy_version 65554 (0.0005) [2023-12-27 06:07:01,978][20381] Updated weights for policy 1, policy_version 65564 (0.0008) [2023-12-27 06:07:02,026][20381] Updated weights for policy 1, policy_version 65574 (0.0010) [2023-12-27 06:07:02,284][20347] Updated weights for policy 0, policy_version 65675 (0.0008) [2023-12-27 06:07:02,335][20347] Updated weights for policy 0, policy_version 65685 (0.0009) [2023-12-27 06:07:02,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 33611776. Throughput: 0: 10357.8, 1: 10247.3. Samples: 33624584. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:07:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:02,405][20347] Updated weights for policy 0, policy_version 65695 (0.0008) [2023-12-27 06:07:02,629][20381] Updated weights for policy 1, policy_version 65584 (0.0010) [2023-12-27 06:07:02,677][20381] Updated weights for policy 1, policy_version 65594 (0.0010) [2023-12-27 06:07:02,734][20381] Updated weights for policy 1, policy_version 65604 (0.0011) [2023-12-27 06:07:03,018][20347] Updated weights for policy 0, policy_version 65705 (0.0008) [2023-12-27 06:07:03,066][20347] Updated weights for policy 0, policy_version 65715 (0.0005) [2023-12-27 06:07:03,112][20347] Updated weights for policy 0, policy_version 65725 (0.0005) [2023-12-27 06:07:03,157][20347] Updated weights for policy 0, policy_version 65735 (0.0005) [2023-12-27 06:07:03,369][20381] Updated weights for policy 1, policy_version 65614 (0.0009) [2023-12-27 06:07:03,424][20381] Updated weights for policy 1, policy_version 65624 (0.0007) [2023-12-27 06:07:03,482][20381] Updated weights for policy 1, policy_version 65634 (0.0005) [2023-12-27 06:07:03,760][20347] Updated weights for policy 0, policy_version 65745 (0.0007) [2023-12-27 06:07:03,822][20347] Updated weights for policy 0, policy_version 65755 (0.0008) [2023-12-27 06:07:03,882][20347] Updated weights for policy 0, policy_version 65765 (0.0008) [2023-12-27 06:07:04,049][20381] Updated weights for policy 1, policy_version 65644 (0.0005) [2023-12-27 06:07:04,115][20381] Updated weights for policy 1, policy_version 65654 (0.0006) [2023-12-27 06:07:04,175][20381] Updated weights for policy 1, policy_version 65664 (0.0010) [2023-12-27 06:07:04,533][20347] Updated weights for policy 0, policy_version 65775 (0.0006) [2023-12-27 06:07:04,582][20347] Updated weights for policy 0, policy_version 65785 (0.0008) [2023-12-27 06:07:04,638][20347] Updated weights for policy 0, policy_version 65795 (0.0008) [2023-12-27 06:07:04,840][20381] Updated weights for policy 1, policy_version 65674 (0.0008) [2023-12-27 06:07:04,896][20381] Updated weights for policy 1, policy_version 65684 (0.0009) [2023-12-27 06:07:04,955][20381] Updated weights for policy 1, policy_version 65694 (0.0011) [2023-12-27 06:07:05,011][20381] Updated weights for policy 1, policy_version 65704 (0.0011) [2023-12-27 06:07:05,346][20347] Updated weights for policy 0, policy_version 65805 (0.0008) [2023-12-27 06:07:05,397][20347] Updated weights for policy 0, policy_version 65815 (0.0008) [2023-12-27 06:07:05,450][20347] Updated weights for policy 0, policy_version 65825 (0.0007) [2023-12-27 06:07:05,594][20381] Updated weights for policy 1, policy_version 65714 (0.0008) [2023-12-27 06:07:05,639][20381] Updated weights for policy 1, policy_version 65724 (0.0009) [2023-12-27 06:07:05,693][20381] Updated weights for policy 1, policy_version 65734 (0.0008) [2023-12-27 06:07:06,045][20347] Updated weights for policy 0, policy_version 65835 (0.0008) [2023-12-27 06:07:06,110][20347] Updated weights for policy 0, policy_version 65845 (0.0007) [2023-12-27 06:07:06,166][20347] Updated weights for policy 0, policy_version 65855 (0.0010) [2023-12-27 06:07:06,450][20381] Updated weights for policy 1, policy_version 65744 (0.0006) [2023-12-27 06:07:06,512][20381] Updated weights for policy 1, policy_version 65754 (0.0006) [2023-12-27 06:07:06,573][20381] Updated weights for policy 1, policy_version 65764 (0.0006) [2023-12-27 06:07:06,782][20347] Updated weights for policy 0, policy_version 65865 (0.0010) [2023-12-27 06:07:06,834][20347] Updated weights for policy 0, policy_version 65875 (0.0005) [2023-12-27 06:07:06,897][20347] Updated weights for policy 0, policy_version 65885 (0.0006) [2023-12-27 06:07:06,950][20347] Updated weights for policy 0, policy_version 65895 (0.0007) [2023-12-27 06:07:07,158][20381] Updated weights for policy 1, policy_version 65774 (0.0007) [2023-12-27 06:07:07,204][20381] Updated weights for policy 1, policy_version 65784 (0.0008) [2023-12-27 06:07:07,260][20381] Updated weights for policy 1, policy_version 65794 (0.0006) [2023-12-27 06:07:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 33726464. Throughput: 0: 10379.9, 1: 10333.1. Samples: 33690628. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:07:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000065896_16875520.pth... [2023-12-27 06:07:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000065800_16850944.pth... [2023-12-27 06:07:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000064552_16531456.pth [2023-12-27 06:07:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000064680_16564224.pth [2023-12-27 06:07:07,575][20347] Updated weights for policy 0, policy_version 65905 (0.0006) [2023-12-27 06:07:07,628][20347] Updated weights for policy 0, policy_version 65915 (0.0005) [2023-12-27 06:07:07,686][20347] Updated weights for policy 0, policy_version 65925 (0.0009) [2023-12-27 06:07:07,936][20381] Updated weights for policy 1, policy_version 65804 (0.0005) [2023-12-27 06:07:07,995][20381] Updated weights for policy 1, policy_version 65814 (0.0005) [2023-12-27 06:07:08,046][20381] Updated weights for policy 1, policy_version 65824 (0.0007) [2023-12-27 06:07:08,313][20347] Updated weights for policy 0, policy_version 65935 (0.0008) [2023-12-27 06:07:08,375][20347] Updated weights for policy 0, policy_version 65945 (0.0007) [2023-12-27 06:07:08,429][20347] Updated weights for policy 0, policy_version 65955 (0.0007) [2023-12-27 06:07:08,729][20381] Updated weights for policy 1, policy_version 65834 (0.0007) [2023-12-27 06:07:08,789][20381] Updated weights for policy 1, policy_version 65844 (0.0008) [2023-12-27 06:07:08,849][20381] Updated weights for policy 1, policy_version 65854 (0.0008) [2023-12-27 06:07:08,909][20381] Updated weights for policy 1, policy_version 65864 (0.0008) [2023-12-27 06:07:09,116][20347] Updated weights for policy 0, policy_version 65965 (0.0006) [2023-12-27 06:07:09,174][20347] Updated weights for policy 0, policy_version 65975 (0.0005) [2023-12-27 06:07:09,219][20347] Updated weights for policy 0, policy_version 65985 (0.0006) [2023-12-27 06:07:09,596][20381] Updated weights for policy 1, policy_version 65874 (0.0008) [2023-12-27 06:07:09,643][20381] Updated weights for policy 1, policy_version 65884 (0.0007) [2023-12-27 06:07:09,706][20381] Updated weights for policy 1, policy_version 65894 (0.0005) [2023-12-27 06:07:09,905][20347] Updated weights for policy 0, policy_version 65995 (0.0006) [2023-12-27 06:07:09,966][20347] Updated weights for policy 0, policy_version 66005 (0.0007) [2023-12-27 06:07:10,028][20347] Updated weights for policy 0, policy_version 66015 (0.0007) [2023-12-27 06:07:10,391][20381] Updated weights for policy 1, policy_version 65904 (0.0008) [2023-12-27 06:07:10,446][20381] Updated weights for policy 1, policy_version 65914 (0.0008) [2023-12-27 06:07:10,492][20381] Updated weights for policy 1, policy_version 65924 (0.0008) [2023-12-27 06:07:10,728][20347] Updated weights for policy 0, policy_version 66025 (0.0008) [2023-12-27 06:07:10,783][20347] Updated weights for policy 0, policy_version 66035 (0.0007) [2023-12-27 06:07:10,834][20347] Updated weights for policy 0, policy_version 66045 (0.0008) [2023-12-27 06:07:10,882][20347] Updated weights for policy 0, policy_version 66055 (0.0007) [2023-12-27 06:07:11,085][20381] Updated weights for policy 1, policy_version 65934 (0.0007) [2023-12-27 06:07:11,141][20381] Updated weights for policy 1, policy_version 65944 (0.0007) [2023-12-27 06:07:11,199][20381] Updated weights for policy 1, policy_version 65954 (0.0008) [2023-12-27 06:07:11,619][20347] Updated weights for policy 0, policy_version 66065 (0.0007) [2023-12-27 06:07:11,678][20347] Updated weights for policy 0, policy_version 66075 (0.0008) [2023-12-27 06:07:11,742][20347] Updated weights for policy 0, policy_version 66085 (0.0008) [2023-12-27 06:07:11,979][20381] Updated weights for policy 1, policy_version 65964 (0.0007) [2023-12-27 06:07:12,034][20381] Updated weights for policy 1, policy_version 65974 (0.0008) [2023-12-27 06:07:12,088][20381] Updated weights for policy 1, policy_version 65984 (0.0008) [2023-12-27 06:07:12,360][20347] Updated weights for policy 0, policy_version 66095 (0.0007) [2023-12-27 06:07:12,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 33824768. Throughput: 0: 10421.5, 1: 10381.1. Samples: 33815620. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:07:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:12,423][20347] Updated weights for policy 0, policy_version 66105 (0.0007) [2023-12-27 06:07:12,483][20347] Updated weights for policy 0, policy_version 66115 (0.0008) [2023-12-27 06:07:12,695][20381] Updated weights for policy 1, policy_version 65994 (0.0007) [2023-12-27 06:07:12,758][20381] Updated weights for policy 1, policy_version 66004 (0.0011) [2023-12-27 06:07:12,820][20381] Updated weights for policy 1, policy_version 66014 (0.0011) [2023-12-27 06:07:12,872][20381] Updated weights for policy 1, policy_version 66024 (0.0010) [2023-12-27 06:07:13,151][20347] Updated weights for policy 0, policy_version 66125 (0.0006) [2023-12-27 06:07:13,202][20347] Updated weights for policy 0, policy_version 66135 (0.0005) [2023-12-27 06:07:13,259][20347] Updated weights for policy 0, policy_version 66145 (0.0005) [2023-12-27 06:07:13,544][20381] Updated weights for policy 1, policy_version 66034 (0.0008) [2023-12-27 06:07:13,599][20381] Updated weights for policy 1, policy_version 66044 (0.0008) [2023-12-27 06:07:13,655][20381] Updated weights for policy 1, policy_version 66054 (0.0008) [2023-12-27 06:07:13,853][20347] Updated weights for policy 0, policy_version 66155 (0.0005) [2023-12-27 06:07:13,910][20347] Updated weights for policy 0, policy_version 66165 (0.0007) [2023-12-27 06:07:13,967][20347] Updated weights for policy 0, policy_version 66175 (0.0008) [2023-12-27 06:07:14,271][20381] Updated weights for policy 1, policy_version 66064 (0.0006) [2023-12-27 06:07:14,319][20381] Updated weights for policy 1, policy_version 66074 (0.0005) [2023-12-27 06:07:14,380][20381] Updated weights for policy 1, policy_version 66084 (0.0006) [2023-12-27 06:07:14,680][20347] Updated weights for policy 0, policy_version 66185 (0.0007) [2023-12-27 06:07:14,746][20347] Updated weights for policy 0, policy_version 66195 (0.0008) [2023-12-27 06:07:14,804][20347] Updated weights for policy 0, policy_version 66205 (0.0008) [2023-12-27 06:07:14,853][20347] Updated weights for policy 0, policy_version 66215 (0.0007) [2023-12-27 06:07:15,057][20381] Updated weights for policy 1, policy_version 66094 (0.0008) [2023-12-27 06:07:15,119][20381] Updated weights for policy 1, policy_version 66104 (0.0008) [2023-12-27 06:07:15,186][20381] Updated weights for policy 1, policy_version 66114 (0.0009) [2023-12-27 06:07:15,569][20347] Updated weights for policy 0, policy_version 66225 (0.0008) [2023-12-27 06:07:15,623][20347] Updated weights for policy 0, policy_version 66235 (0.0007) [2023-12-27 06:07:15,673][20347] Updated weights for policy 0, policy_version 66245 (0.0008) [2023-12-27 06:07:15,872][20381] Updated weights for policy 1, policy_version 66124 (0.0009) [2023-12-27 06:07:15,925][20381] Updated weights for policy 1, policy_version 66134 (0.0008) [2023-12-27 06:07:15,978][20381] Updated weights for policy 1, policy_version 66144 (0.0008) [2023-12-27 06:07:16,385][20347] Updated weights for policy 0, policy_version 66255 (0.0009) [2023-12-27 06:07:16,441][20347] Updated weights for policy 0, policy_version 66265 (0.0010) [2023-12-27 06:07:16,491][20347] Updated weights for policy 0, policy_version 66275 (0.0010) [2023-12-27 06:07:16,645][20381] Updated weights for policy 1, policy_version 66154 (0.0008) [2023-12-27 06:07:16,707][20381] Updated weights for policy 1, policy_version 66164 (0.0007) [2023-12-27 06:07:16,769][20381] Updated weights for policy 1, policy_version 66174 (0.0008) [2023-12-27 06:07:16,818][20381] Updated weights for policy 1, policy_version 66184 (0.0007) [2023-12-27 06:07:17,192][20347] Updated weights for policy 0, policy_version 66285 (0.0008) [2023-12-27 06:07:17,250][20347] Updated weights for policy 0, policy_version 66295 (0.0006) [2023-12-27 06:07:17,310][20347] Updated weights for policy 0, policy_version 66305 (0.0007) [2023-12-27 06:07:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 33931264. Throughput: 0: 10327.6, 1: 10436.0. Samples: 33938960. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:07:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:17,497][20381] Updated weights for policy 1, policy_version 66194 (0.0010) [2023-12-27 06:07:17,554][20381] Updated weights for policy 1, policy_version 66204 (0.0010) [2023-12-27 06:07:17,602][20381] Updated weights for policy 1, policy_version 66214 (0.0010) [2023-12-27 06:07:17,956][20347] Updated weights for policy 0, policy_version 66315 (0.0007) [2023-12-27 06:07:18,016][20347] Updated weights for policy 0, policy_version 66325 (0.0006) [2023-12-27 06:07:18,075][20347] Updated weights for policy 0, policy_version 66335 (0.0007) [2023-12-27 06:07:18,283][20381] Updated weights for policy 1, policy_version 66224 (0.0010) [2023-12-27 06:07:18,338][20381] Updated weights for policy 1, policy_version 66234 (0.0010) [2023-12-27 06:07:18,396][20381] Updated weights for policy 1, policy_version 66244 (0.0008) [2023-12-27 06:07:18,736][20347] Updated weights for policy 0, policy_version 66345 (0.0007) [2023-12-27 06:07:18,799][20347] Updated weights for policy 0, policy_version 66355 (0.0007) [2023-12-27 06:07:18,861][20347] Updated weights for policy 0, policy_version 66365 (0.0007) [2023-12-27 06:07:18,921][20347] Updated weights for policy 0, policy_version 66375 (0.0006) [2023-12-27 06:07:19,112][20381] Updated weights for policy 1, policy_version 66254 (0.0008) [2023-12-27 06:07:19,169][20381] Updated weights for policy 1, policy_version 66264 (0.0010) [2023-12-27 06:07:19,221][20381] Updated weights for policy 1, policy_version 66274 (0.0010) [2023-12-27 06:07:19,625][20347] Updated weights for policy 0, policy_version 66385 (0.0008) [2023-12-27 06:07:19,671][20347] Updated weights for policy 0, policy_version 66395 (0.0007) [2023-12-27 06:07:19,728][20347] Updated weights for policy 0, policy_version 66405 (0.0008) [2023-12-27 06:07:19,928][20381] Updated weights for policy 1, policy_version 66284 (0.0009) [2023-12-27 06:07:19,991][20381] Updated weights for policy 1, policy_version 66294 (0.0008) [2023-12-27 06:07:20,058][20381] Updated weights for policy 1, policy_version 66304 (0.0007) [2023-12-27 06:07:20,436][20347] Updated weights for policy 0, policy_version 66415 (0.0009) [2023-12-27 06:07:20,495][20347] Updated weights for policy 0, policy_version 66425 (0.0008) [2023-12-27 06:07:20,548][20347] Updated weights for policy 0, policy_version 66435 (0.0007) [2023-12-27 06:07:20,714][20381] Updated weights for policy 1, policy_version 66314 (0.0008) [2023-12-27 06:07:20,778][20381] Updated weights for policy 1, policy_version 66324 (0.0009) [2023-12-27 06:07:20,841][20381] Updated weights for policy 1, policy_version 66334 (0.0008) [2023-12-27 06:07:20,898][20381] Updated weights for policy 1, policy_version 66344 (0.0008) [2023-12-27 06:07:21,211][20347] Updated weights for policy 0, policy_version 66445 (0.0006) [2023-12-27 06:07:21,269][20347] Updated weights for policy 0, policy_version 66455 (0.0007) [2023-12-27 06:07:21,321][20347] Updated weights for policy 0, policy_version 66465 (0.0007) [2023-12-27 06:07:21,661][20381] Updated weights for policy 1, policy_version 66354 (0.0007) [2023-12-27 06:07:21,724][20381] Updated weights for policy 1, policy_version 66364 (0.0008) [2023-12-27 06:07:21,784][20381] Updated weights for policy 1, policy_version 66374 (0.0008) [2023-12-27 06:07:21,967][20347] Updated weights for policy 0, policy_version 66475 (0.0008) [2023-12-27 06:07:22,028][20347] Updated weights for policy 0, policy_version 66485 (0.0005) [2023-12-27 06:07:22,089][20347] Updated weights for policy 0, policy_version 66495 (0.0005) [2023-12-27 06:07:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 34029568. Throughput: 0: 10338.6, 1: 10410.3. Samples: 33999840. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 06:07:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000066376_16998400.pth... [2023-12-27 06:07:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000066504_17031168.pth... [2023-12-27 06:07:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000065160_16687104.pth [2023-12-27 06:07:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000065288_16719872.pth [2023-12-27 06:07:22,540][20381] Updated weights for policy 1, policy_version 66384 (0.0008) [2023-12-27 06:07:22,602][20381] Updated weights for policy 1, policy_version 66394 (0.0008) [2023-12-27 06:07:22,643][20347] Updated weights for policy 0, policy_version 66505 (0.0005) [2023-12-27 06:07:22,663][20381] Updated weights for policy 1, policy_version 66404 (0.0008) [2023-12-27 06:07:22,697][20347] Updated weights for policy 0, policy_version 66515 (0.0008) [2023-12-27 06:07:22,751][20347] Updated weights for policy 0, policy_version 66525 (0.0007) [2023-12-27 06:07:22,803][20347] Updated weights for policy 0, policy_version 66535 (0.0008) [2023-12-27 06:07:23,393][20381] Updated weights for policy 1, policy_version 66414 (0.0008) [2023-12-27 06:07:23,446][20381] Updated weights for policy 1, policy_version 66424 (0.0007) [2023-12-27 06:07:23,461][20347] Updated weights for policy 0, policy_version 66545 (0.0009) [2023-12-27 06:07:23,501][20381] Updated weights for policy 1, policy_version 66434 (0.0006) [2023-12-27 06:07:23,520][20347] Updated weights for policy 0, policy_version 66555 (0.0008) [2023-12-27 06:07:23,564][20347] Updated weights for policy 0, policy_version 66565 (0.0007) [2023-12-27 06:07:24,075][20381] Updated weights for policy 1, policy_version 66444 (0.0008) [2023-12-27 06:07:24,129][20381] Updated weights for policy 1, policy_version 66454 (0.0005) [2023-12-27 06:07:24,189][20381] Updated weights for policy 1, policy_version 66464 (0.0006) [2023-12-27 06:07:24,313][20347] Updated weights for policy 0, policy_version 66575 (0.0008) [2023-12-27 06:07:24,374][20347] Updated weights for policy 0, policy_version 66585 (0.0007) [2023-12-27 06:07:24,422][20347] Updated weights for policy 0, policy_version 66595 (0.0007) [2023-12-27 06:07:24,852][20381] Updated weights for policy 1, policy_version 66474 (0.0008) [2023-12-27 06:07:24,906][20381] Updated weights for policy 1, policy_version 66484 (0.0007) [2023-12-27 06:07:24,963][20381] Updated weights for policy 1, policy_version 66494 (0.0007) [2023-12-27 06:07:25,030][20381] Updated weights for policy 1, policy_version 66504 (0.0006) [2023-12-27 06:07:25,071][20347] Updated weights for policy 0, policy_version 66605 (0.0008) [2023-12-27 06:07:25,122][20347] Updated weights for policy 0, policy_version 66615 (0.0010) [2023-12-27 06:07:25,173][20347] Updated weights for policy 0, policy_version 66625 (0.0010) [2023-12-27 06:07:25,727][20381] Updated weights for policy 1, policy_version 66514 (0.0008) [2023-12-27 06:07:25,765][20347] Updated weights for policy 0, policy_version 66635 (0.0009) [2023-12-27 06:07:25,782][20381] Updated weights for policy 1, policy_version 66524 (0.0006) [2023-12-27 06:07:25,828][20347] Updated weights for policy 0, policy_version 66645 (0.0007) [2023-12-27 06:07:25,845][20381] Updated weights for policy 1, policy_version 66534 (0.0006) [2023-12-27 06:07:25,887][20347] Updated weights for policy 0, policy_version 66655 (0.0007) [2023-12-27 06:07:26,515][20381] Updated weights for policy 1, policy_version 66544 (0.0006) [2023-12-27 06:07:26,545][20347] Updated weights for policy 0, policy_version 66665 (0.0008) [2023-12-27 06:07:26,572][20381] Updated weights for policy 1, policy_version 66554 (0.0006) [2023-12-27 06:07:26,595][20347] Updated weights for policy 0, policy_version 66675 (0.0010) [2023-12-27 06:07:26,619][20381] Updated weights for policy 1, policy_version 66564 (0.0005) [2023-12-27 06:07:26,646][20347] Updated weights for policy 0, policy_version 66685 (0.0010) [2023-12-27 06:07:26,690][20347] Updated weights for policy 0, policy_version 66695 (0.0010) [2023-12-27 06:07:27,251][20381] Updated weights for policy 1, policy_version 66574 (0.0006) [2023-12-27 06:07:27,309][20381] Updated weights for policy 1, policy_version 66584 (0.0006) [2023-12-27 06:07:27,348][20347] Updated weights for policy 0, policy_version 66705 (0.0006) [2023-12-27 06:07:27,358][20381] Updated weights for policy 1, policy_version 66594 (0.0006) [2023-12-27 06:07:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 34127872. Throughput: 0: 10400.7, 1: 10365.8. Samples: 34123652. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:27,400][20347] Updated weights for policy 0, policy_version 66715 (0.0007) [2023-12-27 06:07:27,456][20347] Updated weights for policy 0, policy_version 66725 (0.0009) [2023-12-27 06:07:27,954][20381] Updated weights for policy 1, policy_version 66604 (0.0005) [2023-12-27 06:07:27,999][20381] Updated weights for policy 1, policy_version 66614 (0.0005) [2023-12-27 06:07:28,054][20381] Updated weights for policy 1, policy_version 66624 (0.0005) [2023-12-27 06:07:28,064][20347] Updated weights for policy 0, policy_version 66735 (0.0007) [2023-12-27 06:07:28,114][20347] Updated weights for policy 0, policy_version 66745 (0.0007) [2023-12-27 06:07:28,159][20347] Updated weights for policy 0, policy_version 66755 (0.0007) [2023-12-27 06:07:28,652][20381] Updated weights for policy 1, policy_version 66634 (0.0006) [2023-12-27 06:07:28,715][20381] Updated weights for policy 1, policy_version 66644 (0.0008) [2023-12-27 06:07:28,771][20381] Updated weights for policy 1, policy_version 66654 (0.0007) [2023-12-27 06:07:28,782][20347] Updated weights for policy 0, policy_version 66765 (0.0007) [2023-12-27 06:07:28,829][20381] Updated weights for policy 1, policy_version 66664 (0.0006) [2023-12-27 06:07:28,838][20347] Updated weights for policy 0, policy_version 66775 (0.0005) [2023-12-27 06:07:28,892][20347] Updated weights for policy 0, policy_version 66785 (0.0007) [2023-12-27 06:07:29,459][20381] Updated weights for policy 1, policy_version 66674 (0.0008) [2023-12-27 06:07:29,505][20381] Updated weights for policy 1, policy_version 66684 (0.0008) [2023-12-27 06:07:29,555][20381] Updated weights for policy 1, policy_version 66694 (0.0008) [2023-12-27 06:07:29,583][20347] Updated weights for policy 0, policy_version 66795 (0.0007) [2023-12-27 06:07:29,640][20347] Updated weights for policy 0, policy_version 66805 (0.0006) [2023-12-27 06:07:29,701][20347] Updated weights for policy 0, policy_version 66815 (0.0005) [2023-12-27 06:07:30,251][20381] Updated weights for policy 1, policy_version 66704 (0.0008) [2023-12-27 06:07:30,263][20347] Updated weights for policy 0, policy_version 66825 (0.0006) [2023-12-27 06:07:30,306][20381] Updated weights for policy 1, policy_version 66714 (0.0007) [2023-12-27 06:07:30,319][20347] Updated weights for policy 0, policy_version 66835 (0.0006) [2023-12-27 06:07:30,361][20381] Updated weights for policy 1, policy_version 66724 (0.0006) [2023-12-27 06:07:30,376][20347] Updated weights for policy 0, policy_version 66845 (0.0006) [2023-12-27 06:07:30,438][20347] Updated weights for policy 0, policy_version 66855 (0.0006) [2023-12-27 06:07:31,012][20381] Updated weights for policy 1, policy_version 66734 (0.0008) [2023-12-27 06:07:31,072][20381] Updated weights for policy 1, policy_version 66744 (0.0008) [2023-12-27 06:07:31,122][20347] Updated weights for policy 0, policy_version 66865 (0.0006) [2023-12-27 06:07:31,132][20381] Updated weights for policy 1, policy_version 66754 (0.0006) [2023-12-27 06:07:31,188][20347] Updated weights for policy 0, policy_version 66875 (0.0008) [2023-12-27 06:07:31,255][20347] Updated weights for policy 0, policy_version 66885 (0.0008) [2023-12-27 06:07:31,815][20381] Updated weights for policy 1, policy_version 66764 (0.0007) [2023-12-27 06:07:31,868][20381] Updated weights for policy 1, policy_version 66774 (0.0006) [2023-12-27 06:07:31,878][20347] Updated weights for policy 0, policy_version 66895 (0.0006) [2023-12-27 06:07:31,914][20381] Updated weights for policy 1, policy_version 66784 (0.0005) [2023-12-27 06:07:31,937][20347] Updated weights for policy 0, policy_version 66905 (0.0008) [2023-12-27 06:07:31,996][20347] Updated weights for policy 0, policy_version 66915 (0.0008) [2023-12-27 06:07:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 34242560. Throughput: 0: 10441.5, 1: 10447.1. Samples: 34252224. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:32,610][20381] Updated weights for policy 1, policy_version 66794 (0.0009) [2023-12-27 06:07:32,657][20381] Updated weights for policy 1, policy_version 66804 (0.0007) [2023-12-27 06:07:32,698][20347] Updated weights for policy 0, policy_version 66925 (0.0007) [2023-12-27 06:07:32,707][20381] Updated weights for policy 1, policy_version 66814 (0.0007) [2023-12-27 06:07:32,754][20347] Updated weights for policy 0, policy_version 66935 (0.0006) [2023-12-27 06:07:32,762][20381] Updated weights for policy 1, policy_version 66824 (0.0006) [2023-12-27 06:07:32,815][20347] Updated weights for policy 0, policy_version 66945 (0.0008) [2023-12-27 06:07:33,416][20381] Updated weights for policy 1, policy_version 66834 (0.0011) [2023-12-27 06:07:33,458][20347] Updated weights for policy 0, policy_version 66955 (0.0007) [2023-12-27 06:07:33,462][20381] Updated weights for policy 1, policy_version 66844 (0.0010) [2023-12-27 06:07:33,503][20347] Updated weights for policy 0, policy_version 66965 (0.0006) [2023-12-27 06:07:33,509][20381] Updated weights for policy 1, policy_version 66854 (0.0010) [2023-12-27 06:07:33,548][20347] Updated weights for policy 0, policy_version 66975 (0.0010) [2023-12-27 06:07:34,175][20347] Updated weights for policy 0, policy_version 66985 (0.0010) [2023-12-27 06:07:34,195][20381] Updated weights for policy 1, policy_version 66864 (0.0008) [2023-12-27 06:07:34,227][20347] Updated weights for policy 0, policy_version 66995 (0.0009) [2023-12-27 06:07:34,245][20381] Updated weights for policy 1, policy_version 66874 (0.0009) [2023-12-27 06:07:34,280][20347] Updated weights for policy 0, policy_version 67005 (0.0009) [2023-12-27 06:07:34,308][20381] Updated weights for policy 1, policy_version 66884 (0.0009) [2023-12-27 06:07:34,342][20347] Updated weights for policy 0, policy_version 67015 (0.0009) [2023-12-27 06:07:35,035][20347] Updated weights for policy 0, policy_version 67025 (0.0008) [2023-12-27 06:07:35,056][20381] Updated weights for policy 1, policy_version 66894 (0.0010) [2023-12-27 06:07:35,091][20347] Updated weights for policy 0, policy_version 67035 (0.0006) [2023-12-27 06:07:35,111][20381] Updated weights for policy 1, policy_version 66904 (0.0011) [2023-12-27 06:07:35,145][20347] Updated weights for policy 0, policy_version 67045 (0.0005) [2023-12-27 06:07:35,163][20381] Updated weights for policy 1, policy_version 66914 (0.0010) [2023-12-27 06:07:35,839][20381] Updated weights for policy 1, policy_version 66924 (0.0009) [2023-12-27 06:07:35,853][20347] Updated weights for policy 0, policy_version 67055 (0.0007) [2023-12-27 06:07:35,900][20381] Updated weights for policy 1, policy_version 66934 (0.0006) [2023-12-27 06:07:35,908][20347] Updated weights for policy 0, policy_version 67065 (0.0007) [2023-12-27 06:07:35,962][20381] Updated weights for policy 1, policy_version 66944 (0.0009) [2023-12-27 06:07:35,967][20347] Updated weights for policy 0, policy_version 67075 (0.0008) [2023-12-27 06:07:36,657][20347] Updated weights for policy 0, policy_version 67085 (0.0007) [2023-12-27 06:07:36,663][20381] Updated weights for policy 1, policy_version 66954 (0.0010) [2023-12-27 06:07:36,716][20381] Updated weights for policy 1, policy_version 66964 (0.0006) [2023-12-27 06:07:36,718][20347] Updated weights for policy 0, policy_version 67095 (0.0007) [2023-12-27 06:07:36,774][20381] Updated weights for policy 1, policy_version 66974 (0.0006) [2023-12-27 06:07:36,783][20347] Updated weights for policy 0, policy_version 67105 (0.0007) [2023-12-27 06:07:36,830][20381] Updated weights for policy 1, policy_version 66984 (0.0006) [2023-12-27 06:07:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 34340864. Throughput: 0: 10436.7, 1: 10415.9. Samples: 34314080. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000066984_17154048.pth... [2023-12-27 06:07:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000067112_17186816.pth... [2023-12-27 06:07:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000065800_16850944.pth [2023-12-27 06:07:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000065896_16875520.pth [2023-12-27 06:07:37,475][20347] Updated weights for policy 0, policy_version 67115 (0.0007) [2023-12-27 06:07:37,495][20381] Updated weights for policy 1, policy_version 66994 (0.0006) [2023-12-27 06:07:37,532][20347] Updated weights for policy 0, policy_version 67125 (0.0007) [2023-12-27 06:07:37,553][20381] Updated weights for policy 1, policy_version 67004 (0.0006) [2023-12-27 06:07:37,589][20347] Updated weights for policy 0, policy_version 67135 (0.0006) [2023-12-27 06:07:37,603][20381] Updated weights for policy 1, policy_version 67014 (0.0005) [2023-12-27 06:07:38,227][20381] Updated weights for policy 1, policy_version 67024 (0.0005) [2023-12-27 06:07:38,260][20347] Updated weights for policy 0, policy_version 67145 (0.0007) [2023-12-27 06:07:38,283][20381] Updated weights for policy 1, policy_version 67034 (0.0008) [2023-12-27 06:07:38,319][20347] Updated weights for policy 0, policy_version 67155 (0.0006) [2023-12-27 06:07:38,346][20381] Updated weights for policy 1, policy_version 67044 (0.0007) [2023-12-27 06:07:38,385][20347] Updated weights for policy 0, policy_version 67165 (0.0008) [2023-12-27 06:07:38,446][20347] Updated weights for policy 0, policy_version 67175 (0.0008) [2023-12-27 06:07:38,987][20381] Updated weights for policy 1, policy_version 67054 (0.0008) [2023-12-27 06:07:39,040][20381] Updated weights for policy 1, policy_version 67064 (0.0008) [2023-12-27 06:07:39,096][20381] Updated weights for policy 1, policy_version 67074 (0.0008) [2023-12-27 06:07:39,147][20347] Updated weights for policy 0, policy_version 67185 (0.0006) [2023-12-27 06:07:39,199][20347] Updated weights for policy 0, policy_version 67195 (0.0007) [2023-12-27 06:07:39,263][20347] Updated weights for policy 0, policy_version 67205 (0.0008) [2023-12-27 06:07:39,802][20381] Updated weights for policy 1, policy_version 67084 (0.0006) [2023-12-27 06:07:39,872][20381] Updated weights for policy 1, policy_version 67094 (0.0008) [2023-12-27 06:07:39,935][20381] Updated weights for policy 1, policy_version 67104 (0.0008) [2023-12-27 06:07:39,984][20347] Updated weights for policy 0, policy_version 67215 (0.0007) [2023-12-27 06:07:40,041][20347] Updated weights for policy 0, policy_version 67225 (0.0008) [2023-12-27 06:07:40,099][20347] Updated weights for policy 0, policy_version 67235 (0.0008) [2023-12-27 06:07:40,526][20381] Updated weights for policy 1, policy_version 67114 (0.0007) [2023-12-27 06:07:40,583][20381] Updated weights for policy 1, policy_version 67124 (0.0008) [2023-12-27 06:07:40,639][20381] Updated weights for policy 1, policy_version 67134 (0.0008) [2023-12-27 06:07:40,695][20381] Updated weights for policy 1, policy_version 67144 (0.0007) [2023-12-27 06:07:40,851][20347] Updated weights for policy 0, policy_version 67245 (0.0008) [2023-12-27 06:07:40,914][20347] Updated weights for policy 0, policy_version 67255 (0.0008) [2023-12-27 06:07:40,972][20347] Updated weights for policy 0, policy_version 67265 (0.0007) [2023-12-27 06:07:41,389][20381] Updated weights for policy 1, policy_version 67154 (0.0009) [2023-12-27 06:07:41,450][20381] Updated weights for policy 1, policy_version 67164 (0.0008) [2023-12-27 06:07:41,510][20381] Updated weights for policy 1, policy_version 67174 (0.0008) [2023-12-27 06:07:41,648][20347] Updated weights for policy 0, policy_version 67275 (0.0007) [2023-12-27 06:07:41,709][20347] Updated weights for policy 0, policy_version 67285 (0.0007) [2023-12-27 06:07:41,777][20347] Updated weights for policy 0, policy_version 67295 (0.0008) [2023-12-27 06:07:42,246][20381] Updated weights for policy 1, policy_version 67184 (0.0007) [2023-12-27 06:07:42,313][20381] Updated weights for policy 1, policy_version 67194 (0.0008) [2023-12-27 06:07:42,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 34439168. Throughput: 0: 10359.8, 1: 10399.0. Samples: 34435108. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:42,378][20381] Updated weights for policy 1, policy_version 67204 (0.0008) [2023-12-27 06:07:42,475][20347] Updated weights for policy 0, policy_version 67305 (0.0007) [2023-12-27 06:07:42,530][20347] Updated weights for policy 0, policy_version 67315 (0.0007) [2023-12-27 06:07:42,582][20347] Updated weights for policy 0, policy_version 67325 (0.0008) [2023-12-27 06:07:42,645][20347] Updated weights for policy 0, policy_version 67335 (0.0008) [2023-12-27 06:07:43,110][20381] Updated weights for policy 1, policy_version 67214 (0.0008) [2023-12-27 06:07:43,158][20381] Updated weights for policy 1, policy_version 67224 (0.0008) [2023-12-27 06:07:43,210][20381] Updated weights for policy 1, policy_version 67234 (0.0007) [2023-12-27 06:07:43,295][20347] Updated weights for policy 0, policy_version 67345 (0.0008) [2023-12-27 06:07:43,352][20347] Updated weights for policy 0, policy_version 67355 (0.0008) [2023-12-27 06:07:43,398][20347] Updated weights for policy 0, policy_version 67365 (0.0007) [2023-12-27 06:07:43,927][20381] Updated weights for policy 1, policy_version 67244 (0.0009) [2023-12-27 06:07:43,981][20381] Updated weights for policy 1, policy_version 67254 (0.0010) [2023-12-27 06:07:44,034][20347] Updated weights for policy 0, policy_version 67375 (0.0005) [2023-12-27 06:07:44,039][20381] Updated weights for policy 1, policy_version 67264 (0.0010) [2023-12-27 06:07:44,081][20347] Updated weights for policy 0, policy_version 67385 (0.0006) [2023-12-27 06:07:44,142][20347] Updated weights for policy 0, policy_version 67395 (0.0008) [2023-12-27 06:07:44,720][20381] Updated weights for policy 1, policy_version 67274 (0.0010) [2023-12-27 06:07:44,774][20381] Updated weights for policy 1, policy_version 67284 (0.0007) [2023-12-27 06:07:44,828][20381] Updated weights for policy 1, policy_version 67294 (0.0007) [2023-12-27 06:07:44,840][20347] Updated weights for policy 0, policy_version 67405 (0.0007) [2023-12-27 06:07:44,887][20381] Updated weights for policy 1, policy_version 67304 (0.0008) [2023-12-27 06:07:44,901][20347] Updated weights for policy 0, policy_version 67415 (0.0007) [2023-12-27 06:07:44,964][20347] Updated weights for policy 0, policy_version 67425 (0.0008) [2023-12-27 06:07:45,574][20381] Updated weights for policy 1, policy_version 67314 (0.0009) [2023-12-27 06:07:45,622][20381] Updated weights for policy 1, policy_version 67324 (0.0008) [2023-12-27 06:07:45,632][20347] Updated weights for policy 0, policy_version 67435 (0.0007) [2023-12-27 06:07:45,672][20381] Updated weights for policy 1, policy_version 67334 (0.0007) [2023-12-27 06:07:45,684][20347] Updated weights for policy 0, policy_version 67445 (0.0005) [2023-12-27 06:07:45,736][20347] Updated weights for policy 0, policy_version 67455 (0.0006) [2023-12-27 06:07:46,372][20381] Updated weights for policy 1, policy_version 67344 (0.0007) [2023-12-27 06:07:46,382][20347] Updated weights for policy 0, policy_version 67465 (0.0006) [2023-12-27 06:07:46,426][20381] Updated weights for policy 1, policy_version 67354 (0.0007) [2023-12-27 06:07:46,432][20347] Updated weights for policy 0, policy_version 67475 (0.0006) [2023-12-27 06:07:46,475][20381] Updated weights for policy 1, policy_version 67364 (0.0005) [2023-12-27 06:07:46,481][20347] Updated weights for policy 0, policy_version 67485 (0.0006) [2023-12-27 06:07:46,536][20347] Updated weights for policy 0, policy_version 67495 (0.0006) [2023-12-27 06:07:47,187][20381] Updated weights for policy 1, policy_version 67374 (0.0008) [2023-12-27 06:07:47,192][20347] Updated weights for policy 0, policy_version 67505 (0.0005) [2023-12-27 06:07:47,236][20381] Updated weights for policy 1, policy_version 67384 (0.0010) [2023-12-27 06:07:47,247][20347] Updated weights for policy 0, policy_version 67515 (0.0006) [2023-12-27 06:07:47,292][20381] Updated weights for policy 1, policy_version 67394 (0.0005) [2023-12-27 06:07:47,307][20347] Updated weights for policy 0, policy_version 67525 (0.0005) [2023-12-27 06:07:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 34553856. Throughput: 0: 10403.4, 1: 10304.5. Samples: 34556436. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:47,904][20347] Updated weights for policy 0, policy_version 67535 (0.0009) [2023-12-27 06:07:47,906][20381] Updated weights for policy 1, policy_version 67404 (0.0007) [2023-12-27 06:07:47,957][20347] Updated weights for policy 0, policy_version 67545 (0.0010) [2023-12-27 06:07:47,961][20381] Updated weights for policy 1, policy_version 67414 (0.0011) [2023-12-27 06:07:48,013][20347] Updated weights for policy 0, policy_version 67555 (0.0010) [2023-12-27 06:07:48,017][20381] Updated weights for policy 1, policy_version 67424 (0.0011) [2023-12-27 06:07:48,704][20381] Updated weights for policy 1, policy_version 67434 (0.0010) [2023-12-27 06:07:48,743][20347] Updated weights for policy 0, policy_version 67565 (0.0007) [2023-12-27 06:07:48,759][20381] Updated weights for policy 1, policy_version 67444 (0.0011) [2023-12-27 06:07:48,802][20347] Updated weights for policy 0, policy_version 67575 (0.0006) [2023-12-27 06:07:48,814][20381] Updated weights for policy 1, policy_version 67454 (0.0011) [2023-12-27 06:07:48,857][20347] Updated weights for policy 0, policy_version 67585 (0.0007) [2023-12-27 06:07:48,870][20381] Updated weights for policy 1, policy_version 67464 (0.0010) [2023-12-27 06:07:49,529][20347] Updated weights for policy 0, policy_version 67595 (0.0007) [2023-12-27 06:07:49,577][20381] Updated weights for policy 1, policy_version 67474 (0.0008) [2023-12-27 06:07:49,585][20347] Updated weights for policy 0, policy_version 67605 (0.0007) [2023-12-27 06:07:49,635][20381] Updated weights for policy 1, policy_version 67484 (0.0008) [2023-12-27 06:07:49,641][20347] Updated weights for policy 0, policy_version 67615 (0.0006) [2023-12-27 06:07:49,696][20381] Updated weights for policy 1, policy_version 67494 (0.0008) [2023-12-27 06:07:50,320][20347] Updated weights for policy 0, policy_version 67625 (0.0007) [2023-12-27 06:07:50,388][20347] Updated weights for policy 0, policy_version 67635 (0.0007) [2023-12-27 06:07:50,396][20381] Updated weights for policy 1, policy_version 67504 (0.0011) [2023-12-27 06:07:50,443][20347] Updated weights for policy 0, policy_version 67645 (0.0005) [2023-12-27 06:07:50,452][20381] Updated weights for policy 1, policy_version 67514 (0.0011) [2023-12-27 06:07:50,503][20347] Updated weights for policy 0, policy_version 67655 (0.0005) [2023-12-27 06:07:50,505][20381] Updated weights for policy 1, policy_version 67524 (0.0010) [2023-12-27 06:07:51,071][20347] Updated weights for policy 0, policy_version 67665 (0.0006) [2023-12-27 06:07:51,133][20347] Updated weights for policy 0, policy_version 67675 (0.0007) [2023-12-27 06:07:51,196][20381] Updated weights for policy 1, policy_version 67534 (0.0011) [2023-12-27 06:07:51,197][20347] Updated weights for policy 0, policy_version 67685 (0.0006) [2023-12-27 06:07:51,249][20381] Updated weights for policy 1, policy_version 67544 (0.0011) [2023-12-27 06:07:51,309][20381] Updated weights for policy 1, policy_version 67554 (0.0007) [2023-12-27 06:07:51,921][20347] Updated weights for policy 0, policy_version 67695 (0.0006) [2023-12-27 06:07:51,978][20347] Updated weights for policy 0, policy_version 67705 (0.0007) [2023-12-27 06:07:52,003][20381] Updated weights for policy 1, policy_version 67564 (0.0008) [2023-12-27 06:07:52,036][20347] Updated weights for policy 0, policy_version 67715 (0.0007) [2023-12-27 06:07:52,056][20381] Updated weights for policy 1, policy_version 67574 (0.0006) [2023-12-27 06:07:52,112][20381] Updated weights for policy 1, policy_version 67584 (0.0007) [2023-12-27 06:07:52,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 34652160. Throughput: 0: 10367.7, 1: 10243.3. Samples: 34618120. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000067592_17309696.pth... [2023-12-27 06:07:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000067720_17342464.pth... [2023-12-27 06:07:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000066376_16998400.pth [2023-12-27 06:07:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000066504_17031168.pth [2023-12-27 06:07:52,692][20347] Updated weights for policy 0, policy_version 67725 (0.0008) [2023-12-27 06:07:52,750][20347] Updated weights for policy 0, policy_version 67735 (0.0010) [2023-12-27 06:07:52,805][20347] Updated weights for policy 0, policy_version 67745 (0.0010) [2023-12-27 06:07:52,835][20381] Updated weights for policy 1, policy_version 67594 (0.0008) [2023-12-27 06:07:52,881][20381] Updated weights for policy 1, policy_version 67604 (0.0006) [2023-12-27 06:07:52,930][20381] Updated weights for policy 1, policy_version 67614 (0.0005) [2023-12-27 06:07:52,984][20381] Updated weights for policy 1, policy_version 67624 (0.0007) [2023-12-27 06:07:53,444][20347] Updated weights for policy 0, policy_version 67755 (0.0010) [2023-12-27 06:07:53,497][20347] Updated weights for policy 0, policy_version 67765 (0.0010) [2023-12-27 06:07:53,554][20347] Updated weights for policy 0, policy_version 67775 (0.0010) [2023-12-27 06:07:53,695][20381] Updated weights for policy 1, policy_version 67634 (0.0007) [2023-12-27 06:07:53,741][20381] Updated weights for policy 1, policy_version 67644 (0.0007) [2023-12-27 06:07:53,790][20381] Updated weights for policy 1, policy_version 67654 (0.0007) [2023-12-27 06:07:54,230][20347] Updated weights for policy 0, policy_version 67785 (0.0009) [2023-12-27 06:07:54,295][20347] Updated weights for policy 0, policy_version 67795 (0.0005) [2023-12-27 06:07:54,355][20347] Updated weights for policy 0, policy_version 67805 (0.0009) [2023-12-27 06:07:54,406][20347] Updated weights for policy 0, policy_version 67815 (0.0010) [2023-12-27 06:07:54,416][20381] Updated weights for policy 1, policy_version 67664 (0.0006) [2023-12-27 06:07:54,480][20381] Updated weights for policy 1, policy_version 67674 (0.0005) [2023-12-27 06:07:54,534][20381] Updated weights for policy 1, policy_version 67684 (0.0010) [2023-12-27 06:07:54,978][20347] Updated weights for policy 0, policy_version 67825 (0.0011) [2023-12-27 06:07:55,034][20347] Updated weights for policy 0, policy_version 67835 (0.0010) [2023-12-27 06:07:55,082][20347] Updated weights for policy 0, policy_version 67845 (0.0010) [2023-12-27 06:07:55,170][20381] Updated weights for policy 1, policy_version 67694 (0.0010) [2023-12-27 06:07:55,221][20381] Updated weights for policy 1, policy_version 67704 (0.0010) [2023-12-27 06:07:55,275][20381] Updated weights for policy 1, policy_version 67714 (0.0010) [2023-12-27 06:07:55,736][20347] Updated weights for policy 0, policy_version 67855 (0.0010) [2023-12-27 06:07:55,800][20347] Updated weights for policy 0, policy_version 67865 (0.0010) [2023-12-27 06:07:55,851][20347] Updated weights for policy 0, policy_version 67875 (0.0010) [2023-12-27 06:07:55,876][20381] Updated weights for policy 1, policy_version 67724 (0.0008) [2023-12-27 06:07:55,926][20381] Updated weights for policy 1, policy_version 67734 (0.0005) [2023-12-27 06:07:55,977][20381] Updated weights for policy 1, policy_version 67744 (0.0005) [2023-12-27 06:07:56,535][20347] Updated weights for policy 0, policy_version 67885 (0.0010) [2023-12-27 06:07:56,587][20347] Updated weights for policy 0, policy_version 67895 (0.0010) [2023-12-27 06:07:56,602][20381] Updated weights for policy 1, policy_version 67754 (0.0005) [2023-12-27 06:07:56,642][20347] Updated weights for policy 0, policy_version 67905 (0.0010) [2023-12-27 06:07:56,657][20381] Updated weights for policy 1, policy_version 67764 (0.0006) [2023-12-27 06:07:56,709][20381] Updated weights for policy 1, policy_version 67774 (0.0005) [2023-12-27 06:07:56,766][20381] Updated weights for policy 1, policy_version 67784 (0.0007) [2023-12-27 06:07:57,258][20347] Updated weights for policy 0, policy_version 67915 (0.0010) [2023-12-27 06:07:57,314][20347] Updated weights for policy 0, policy_version 67925 (0.0009) [2023-12-27 06:07:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 34750464. Throughput: 0: 10387.3, 1: 10258.3. Samples: 34744672. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:07:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:07:57,370][20347] Updated weights for policy 0, policy_version 67935 (0.0009) [2023-12-27 06:07:57,405][20381] Updated weights for policy 1, policy_version 67794 (0.0010) [2023-12-27 06:07:57,460][20381] Updated weights for policy 1, policy_version 67804 (0.0010) [2023-12-27 06:07:57,509][20381] Updated weights for policy 1, policy_version 67814 (0.0010) [2023-12-27 06:07:57,973][20347] Updated weights for policy 0, policy_version 67945 (0.0010) [2023-12-27 06:07:58,024][20347] Updated weights for policy 0, policy_version 67955 (0.0010) [2023-12-27 06:07:58,077][20347] Updated weights for policy 0, policy_version 67965 (0.0010) [2023-12-27 06:07:58,131][20347] Updated weights for policy 0, policy_version 67975 (0.0010) [2023-12-27 06:07:58,189][20381] Updated weights for policy 1, policy_version 67824 (0.0009) [2023-12-27 06:07:58,240][20381] Updated weights for policy 1, policy_version 67834 (0.0008) [2023-12-27 06:07:58,299][20381] Updated weights for policy 1, policy_version 67844 (0.0008) [2023-12-27 06:07:58,838][20347] Updated weights for policy 0, policy_version 67985 (0.0007) [2023-12-27 06:07:58,909][20347] Updated weights for policy 0, policy_version 67995 (0.0009) [2023-12-27 06:07:58,971][20347] Updated weights for policy 0, policy_version 68005 (0.0005) [2023-12-27 06:07:59,080][20381] Updated weights for policy 1, policy_version 67854 (0.0007) [2023-12-27 06:07:59,130][20381] Updated weights for policy 1, policy_version 67864 (0.0009) [2023-12-27 06:07:59,189][20381] Updated weights for policy 1, policy_version 67874 (0.0008) [2023-12-27 06:07:59,678][20347] Updated weights for policy 0, policy_version 68015 (0.0006) [2023-12-27 06:07:59,729][20347] Updated weights for policy 0, policy_version 68025 (0.0005) [2023-12-27 06:07:59,779][20347] Updated weights for policy 0, policy_version 68035 (0.0005) [2023-12-27 06:07:59,899][20381] Updated weights for policy 1, policy_version 67884 (0.0007) [2023-12-27 06:07:59,963][20381] Updated weights for policy 1, policy_version 67894 (0.0008) [2023-12-27 06:08:00,023][20381] Updated weights for policy 1, policy_version 67904 (0.0009) [2023-12-27 06:08:00,417][20347] Updated weights for policy 0, policy_version 68045 (0.0008) [2023-12-27 06:08:00,475][20347] Updated weights for policy 0, policy_version 68055 (0.0006) [2023-12-27 06:08:00,525][20347] Updated weights for policy 0, policy_version 68065 (0.0005) [2023-12-27 06:08:00,731][20381] Updated weights for policy 1, policy_version 67914 (0.0007) [2023-12-27 06:08:00,784][20381] Updated weights for policy 1, policy_version 67924 (0.0007) [2023-12-27 06:08:00,837][20381] Updated weights for policy 1, policy_version 67934 (0.0006) [2023-12-27 06:08:00,891][20381] Updated weights for policy 1, policy_version 67944 (0.0008) [2023-12-27 06:08:01,130][20347] Updated weights for policy 0, policy_version 68075 (0.0006) [2023-12-27 06:08:01,182][20347] Updated weights for policy 0, policy_version 68085 (0.0005) [2023-12-27 06:08:01,234][20347] Updated weights for policy 0, policy_version 68095 (0.0005) [2023-12-27 06:08:01,597][20381] Updated weights for policy 1, policy_version 67954 (0.0006) [2023-12-27 06:08:01,663][20381] Updated weights for policy 1, policy_version 67964 (0.0006) [2023-12-27 06:08:01,733][20381] Updated weights for policy 1, policy_version 67974 (0.0007) [2023-12-27 06:08:01,870][20347] Updated weights for policy 0, policy_version 68105 (0.0008) [2023-12-27 06:08:01,930][20347] Updated weights for policy 0, policy_version 68115 (0.0005) [2023-12-27 06:08:01,991][20347] Updated weights for policy 0, policy_version 68125 (0.0005) [2023-12-27 06:08:02,048][20347] Updated weights for policy 0, policy_version 68135 (0.0008) [2023-12-27 06:08:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 34856960. Throughput: 0: 10457.0, 1: 10217.5. Samples: 34869312. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:08:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:02,374][20381] Updated weights for policy 1, policy_version 67984 (0.0008) [2023-12-27 06:08:02,432][20381] Updated weights for policy 1, policy_version 67994 (0.0009) [2023-12-27 06:08:02,492][20381] Updated weights for policy 1, policy_version 68004 (0.0009) [2023-12-27 06:08:02,689][20347] Updated weights for policy 0, policy_version 68145 (0.0007) [2023-12-27 06:08:02,744][20347] Updated weights for policy 0, policy_version 68155 (0.0008) [2023-12-27 06:08:02,802][20347] Updated weights for policy 0, policy_version 68165 (0.0007) [2023-12-27 06:08:03,186][20381] Updated weights for policy 1, policy_version 68014 (0.0008) [2023-12-27 06:08:03,241][20381] Updated weights for policy 1, policy_version 68024 (0.0008) [2023-12-27 06:08:03,295][20381] Updated weights for policy 1, policy_version 68034 (0.0008) [2023-12-27 06:08:03,435][20347] Updated weights for policy 0, policy_version 68175 (0.0010) [2023-12-27 06:08:03,493][20347] Updated weights for policy 0, policy_version 68185 (0.0010) [2023-12-27 06:08:03,541][20347] Updated weights for policy 0, policy_version 68195 (0.0010) [2023-12-27 06:08:03,994][20381] Updated weights for policy 1, policy_version 68044 (0.0008) [2023-12-27 06:08:04,056][20381] Updated weights for policy 1, policy_version 68054 (0.0007) [2023-12-27 06:08:04,112][20381] Updated weights for policy 1, policy_version 68064 (0.0006) [2023-12-27 06:08:04,223][20347] Updated weights for policy 0, policy_version 68205 (0.0010) [2023-12-27 06:08:04,279][20347] Updated weights for policy 0, policy_version 68215 (0.0010) [2023-12-27 06:08:04,338][20347] Updated weights for policy 0, policy_version 68225 (0.0010) [2023-12-27 06:08:04,804][20381] Updated weights for policy 1, policy_version 68074 (0.0007) [2023-12-27 06:08:04,854][20381] Updated weights for policy 1, policy_version 68084 (0.0005) [2023-12-27 06:08:04,914][20381] Updated weights for policy 1, policy_version 68094 (0.0006) [2023-12-27 06:08:04,967][20381] Updated weights for policy 1, policy_version 68104 (0.0008) [2023-12-27 06:08:05,033][20347] Updated weights for policy 0, policy_version 68235 (0.0010) [2023-12-27 06:08:05,088][20347] Updated weights for policy 0, policy_version 68245 (0.0010) [2023-12-27 06:08:05,143][20347] Updated weights for policy 0, policy_version 68255 (0.0010) [2023-12-27 06:08:05,657][20381] Updated weights for policy 1, policy_version 68114 (0.0008) [2023-12-27 06:08:05,704][20381] Updated weights for policy 1, policy_version 68124 (0.0009) [2023-12-27 06:08:05,758][20381] Updated weights for policy 1, policy_version 68134 (0.0008) [2023-12-27 06:08:05,782][20347] Updated weights for policy 0, policy_version 68265 (0.0010) [2023-12-27 06:08:05,849][20347] Updated weights for policy 0, policy_version 68275 (0.0005) [2023-12-27 06:08:05,912][20347] Updated weights for policy 0, policy_version 68285 (0.0005) [2023-12-27 06:08:05,972][20347] Updated weights for policy 0, policy_version 68295 (0.0006) [2023-12-27 06:08:06,511][20381] Updated weights for policy 1, policy_version 68144 (0.0007) [2023-12-27 06:08:06,573][20381] Updated weights for policy 1, policy_version 68154 (0.0008) [2023-12-27 06:08:06,633][20347] Updated weights for policy 0, policy_version 68305 (0.0010) [2023-12-27 06:08:06,640][20381] Updated weights for policy 1, policy_version 68164 (0.0008) [2023-12-27 06:08:06,694][20347] Updated weights for policy 0, policy_version 68315 (0.0011) [2023-12-27 06:08:06,752][20347] Updated weights for policy 0, policy_version 68325 (0.0010) [2023-12-27 06:08:07,339][20347] Updated weights for policy 0, policy_version 68335 (0.0010) [2023-12-27 06:08:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 34955264. Throughput: 0: 10465.9, 1: 10211.5. Samples: 34930320. Policy #0 lag: (min: 22.0, avg: 27.2, max: 54.0) [2023-12-27 06:08:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:07,363][20381] Updated weights for policy 1, policy_version 68174 (0.0006) [2023-12-27 06:08:07,395][20347] Updated weights for policy 0, policy_version 68345 (0.0010) [2023-12-27 06:08:07,416][20381] Updated weights for policy 1, policy_version 68184 (0.0005) [2023-12-27 06:08:07,453][20347] Updated weights for policy 0, policy_version 68355 (0.0008) [2023-12-27 06:08:07,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000068360_17506304.pth... [2023-12-27 06:08:07,475][20381] Updated weights for policy 1, policy_version 68194 (0.0007) [2023-12-27 06:08:07,478][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000067112_17186816.pth [2023-12-27 06:08:07,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000068200_17465344.pth... [2023-12-27 06:08:07,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000066984_17154048.pth [2023-12-27 06:08:08,110][20347] Updated weights for policy 0, policy_version 68365 (0.0008) [2023-12-27 06:08:08,140][20381] Updated weights for policy 1, policy_version 68204 (0.0007) [2023-12-27 06:08:08,167][20347] Updated weights for policy 0, policy_version 68375 (0.0010) [2023-12-27 06:08:08,194][20381] Updated weights for policy 1, policy_version 68214 (0.0009) [2023-12-27 06:08:08,221][20347] Updated weights for policy 0, policy_version 68385 (0.0010) [2023-12-27 06:08:08,248][20381] Updated weights for policy 1, policy_version 68224 (0.0008) [2023-12-27 06:08:08,896][20381] Updated weights for policy 1, policy_version 68234 (0.0007) [2023-12-27 06:08:08,907][20347] Updated weights for policy 0, policy_version 68395 (0.0010) [2023-12-27 06:08:08,943][20381] Updated weights for policy 1, policy_version 68244 (0.0006) [2023-12-27 06:08:08,965][20347] Updated weights for policy 0, policy_version 68405 (0.0007) [2023-12-27 06:08:08,996][20381] Updated weights for policy 1, policy_version 68254 (0.0007) [2023-12-27 06:08:09,020][20347] Updated weights for policy 0, policy_version 68415 (0.0007) [2023-12-27 06:08:09,046][20381] Updated weights for policy 1, policy_version 68264 (0.0008) [2023-12-27 06:08:09,758][20381] Updated weights for policy 1, policy_version 68274 (0.0007) [2023-12-27 06:08:09,759][20347] Updated weights for policy 0, policy_version 68425 (0.0006) [2023-12-27 06:08:09,820][20381] Updated weights for policy 1, policy_version 68284 (0.0007) [2023-12-27 06:08:09,820][20347] Updated weights for policy 0, policy_version 68435 (0.0007) [2023-12-27 06:08:09,879][20381] Updated weights for policy 1, policy_version 68294 (0.0007) [2023-12-27 06:08:09,884][20347] Updated weights for policy 0, policy_version 68445 (0.0010) [2023-12-27 06:08:09,949][20347] Updated weights for policy 0, policy_version 68455 (0.0012) [2023-12-27 06:08:10,588][20381] Updated weights for policy 1, policy_version 68304 (0.0006) [2023-12-27 06:08:10,602][20347] Updated weights for policy 0, policy_version 68465 (0.0011) [2023-12-27 06:08:10,653][20381] Updated weights for policy 1, policy_version 68314 (0.0006) [2023-12-27 06:08:10,661][20347] Updated weights for policy 0, policy_version 68475 (0.0006) [2023-12-27 06:08:10,711][20381] Updated weights for policy 1, policy_version 68324 (0.0005) [2023-12-27 06:08:10,720][20347] Updated weights for policy 0, policy_version 68485 (0.0010) [2023-12-27 06:08:11,337][20381] Updated weights for policy 1, policy_version 68334 (0.0007) [2023-12-27 06:08:11,383][20347] Updated weights for policy 0, policy_version 68495 (0.0008) [2023-12-27 06:08:11,400][20381] Updated weights for policy 1, policy_version 68344 (0.0008) [2023-12-27 06:08:11,432][20347] Updated weights for policy 0, policy_version 68505 (0.0008) [2023-12-27 06:08:11,454][20381] Updated weights for policy 1, policy_version 68354 (0.0007) [2023-12-27 06:08:11,491][20347] Updated weights for policy 0, policy_version 68515 (0.0008) [2023-12-27 06:08:12,153][20347] Updated weights for policy 0, policy_version 68525 (0.0007) [2023-12-27 06:08:12,213][20347] Updated weights for policy 0, policy_version 68535 (0.0007) [2023-12-27 06:08:12,224][20381] Updated weights for policy 1, policy_version 68364 (0.0007) [2023-12-27 06:08:12,272][20347] Updated weights for policy 0, policy_version 68545 (0.0007) [2023-12-27 06:08:12,291][20381] Updated weights for policy 1, policy_version 68374 (0.0007) [2023-12-27 06:08:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 35061760. Throughput: 0: 10425.5, 1: 10229.6. Samples: 35053132. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:12,364][20381] Updated weights for policy 1, policy_version 68384 (0.0007) [2023-12-27 06:08:12,867][20347] Updated weights for policy 0, policy_version 68555 (0.0007) [2023-12-27 06:08:12,917][20347] Updated weights for policy 0, policy_version 68565 (0.0008) [2023-12-27 06:08:12,965][20347] Updated weights for policy 0, policy_version 68575 (0.0007) [2023-12-27 06:08:13,069][20381] Updated weights for policy 1, policy_version 68394 (0.0007) [2023-12-27 06:08:13,116][20381] Updated weights for policy 1, policy_version 68404 (0.0007) [2023-12-27 06:08:13,173][20381] Updated weights for policy 1, policy_version 68414 (0.0007) [2023-12-27 06:08:13,229][20381] Updated weights for policy 1, policy_version 68424 (0.0008) [2023-12-27 06:08:13,668][20347] Updated weights for policy 0, policy_version 68585 (0.0006) [2023-12-27 06:08:13,733][20347] Updated weights for policy 0, policy_version 68595 (0.0010) [2023-12-27 06:08:13,781][20347] Updated weights for policy 0, policy_version 68605 (0.0007) [2023-12-27 06:08:13,830][20381] Updated weights for policy 1, policy_version 68434 (0.0005) [2023-12-27 06:08:13,843][20347] Updated weights for policy 0, policy_version 68615 (0.0006) [2023-12-27 06:08:13,893][20381] Updated weights for policy 1, policy_version 68444 (0.0005) [2023-12-27 06:08:13,944][20381] Updated weights for policy 1, policy_version 68454 (0.0009) [2023-12-27 06:08:14,421][20347] Updated weights for policy 0, policy_version 68625 (0.0007) [2023-12-27 06:08:14,475][20347] Updated weights for policy 0, policy_version 68635 (0.0007) [2023-12-27 06:08:14,520][20381] Updated weights for policy 1, policy_version 68464 (0.0010) [2023-12-27 06:08:14,532][20347] Updated weights for policy 0, policy_version 68645 (0.0007) [2023-12-27 06:08:14,578][20381] Updated weights for policy 1, policy_version 68474 (0.0010) [2023-12-27 06:08:14,632][20381] Updated weights for policy 1, policy_version 68484 (0.0010) [2023-12-27 06:08:15,199][20347] Updated weights for policy 0, policy_version 68655 (0.0006) [2023-12-27 06:08:15,258][20347] Updated weights for policy 0, policy_version 68665 (0.0005) [2023-12-27 06:08:15,314][20347] Updated weights for policy 0, policy_version 68675 (0.0005) [2023-12-27 06:08:15,343][20381] Updated weights for policy 1, policy_version 68494 (0.0011) [2023-12-27 06:08:15,409][20381] Updated weights for policy 1, policy_version 68504 (0.0011) [2023-12-27 06:08:15,471][20381] Updated weights for policy 1, policy_version 68514 (0.0011) [2023-12-27 06:08:15,911][20347] Updated weights for policy 0, policy_version 68685 (0.0005) [2023-12-27 06:08:15,968][20347] Updated weights for policy 0, policy_version 68695 (0.0005) [2023-12-27 06:08:16,022][20347] Updated weights for policy 0, policy_version 68705 (0.0010) [2023-12-27 06:08:16,140][20381] Updated weights for policy 1, policy_version 68524 (0.0010) [2023-12-27 06:08:16,199][20381] Updated weights for policy 1, policy_version 68534 (0.0011) [2023-12-27 06:08:16,257][20381] Updated weights for policy 1, policy_version 68544 (0.0010) [2023-12-27 06:08:16,645][20347] Updated weights for policy 0, policy_version 68715 (0.0009) [2023-12-27 06:08:16,691][20347] Updated weights for policy 0, policy_version 68725 (0.0005) [2023-12-27 06:08:16,740][20347] Updated weights for policy 0, policy_version 68735 (0.0008) [2023-12-27 06:08:16,934][20381] Updated weights for policy 1, policy_version 68554 (0.0010) [2023-12-27 06:08:16,992][20381] Updated weights for policy 1, policy_version 68564 (0.0010) [2023-12-27 06:08:17,053][20381] Updated weights for policy 1, policy_version 68574 (0.0010) [2023-12-27 06:08:17,110][20381] Updated weights for policy 1, policy_version 68584 (0.0010) [2023-12-27 06:08:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 35168256. Throughput: 0: 10434.0, 1: 10166.5. Samples: 35179248. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:17,373][20347] Updated weights for policy 0, policy_version 68745 (0.0010) [2023-12-27 06:08:17,428][20347] Updated weights for policy 0, policy_version 68755 (0.0005) [2023-12-27 06:08:17,482][20347] Updated weights for policy 0, policy_version 68765 (0.0005) [2023-12-27 06:08:17,530][20347] Updated weights for policy 0, policy_version 68775 (0.0009) [2023-12-27 06:08:17,704][20381] Updated weights for policy 1, policy_version 68594 (0.0010) [2023-12-27 06:08:17,766][20381] Updated weights for policy 1, policy_version 68604 (0.0008) [2023-12-27 06:08:17,823][20381] Updated weights for policy 1, policy_version 68614 (0.0010) [2023-12-27 06:08:18,186][20347] Updated weights for policy 0, policy_version 68785 (0.0006) [2023-12-27 06:08:18,246][20347] Updated weights for policy 0, policy_version 68795 (0.0010) [2023-12-27 06:08:18,306][20347] Updated weights for policy 0, policy_version 68805 (0.0010) [2023-12-27 06:08:18,495][20381] Updated weights for policy 1, policy_version 68624 (0.0011) [2023-12-27 06:08:18,551][20381] Updated weights for policy 1, policy_version 68634 (0.0011) [2023-12-27 06:08:18,616][20381] Updated weights for policy 1, policy_version 68644 (0.0011) [2023-12-27 06:08:18,963][20347] Updated weights for policy 0, policy_version 68815 (0.0007) [2023-12-27 06:08:19,028][20347] Updated weights for policy 0, policy_version 68825 (0.0006) [2023-12-27 06:08:19,088][20347] Updated weights for policy 0, policy_version 68835 (0.0006) [2023-12-27 06:08:19,327][20381] Updated weights for policy 1, policy_version 68654 (0.0011) [2023-12-27 06:08:19,385][20381] Updated weights for policy 1, policy_version 68664 (0.0008) [2023-12-27 06:08:19,433][20381] Updated weights for policy 1, policy_version 68674 (0.0008) [2023-12-27 06:08:19,725][20347] Updated weights for policy 0, policy_version 68845 (0.0008) [2023-12-27 06:08:19,787][20347] Updated weights for policy 0, policy_version 68855 (0.0011) [2023-12-27 06:08:19,847][20347] Updated weights for policy 0, policy_version 68865 (0.0008) [2023-12-27 06:08:20,245][20381] Updated weights for policy 1, policy_version 68684 (0.0008) [2023-12-27 06:08:20,301][20381] Updated weights for policy 1, policy_version 68694 (0.0008) [2023-12-27 06:08:20,358][20381] Updated weights for policy 1, policy_version 68704 (0.0007) [2023-12-27 06:08:20,588][20347] Updated weights for policy 0, policy_version 68875 (0.0008) [2023-12-27 06:08:20,648][20347] Updated weights for policy 0, policy_version 68885 (0.0007) [2023-12-27 06:08:20,721][20347] Updated weights for policy 0, policy_version 68895 (0.0008) [2023-12-27 06:08:21,046][20381] Updated weights for policy 1, policy_version 68714 (0.0007) [2023-12-27 06:08:21,101][20381] Updated weights for policy 1, policy_version 68724 (0.0008) [2023-12-27 06:08:21,167][20381] Updated weights for policy 1, policy_version 68734 (0.0008) [2023-12-27 06:08:21,212][20381] Updated weights for policy 1, policy_version 68744 (0.0007) [2023-12-27 06:08:21,393][20347] Updated weights for policy 0, policy_version 68905 (0.0006) [2023-12-27 06:08:21,455][20347] Updated weights for policy 0, policy_version 68915 (0.0007) [2023-12-27 06:08:21,484][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-12-27 06:08:21,947][20381] Updated weights for policy 1, policy_version 68754 (0.0008) [2023-12-27 06:08:22,007][20381] Updated weights for policy 1, policy_version 68764 (0.0007) [2023-12-27 06:08:22,076][20381] Updated weights for policy 1, policy_version 68774 (0.0008) [2023-12-27 06:08:22,223][20347] Updated weights for policy 0, policy_version 68925 (0.0008) [2023-12-27 06:08:22,288][20347] Updated weights for policy 0, policy_version 68935 (0.0006) [2023-12-27 06:08:22,349][20347] Updated weights for policy 0, policy_version 68945 (0.0007) [2023-12-27 06:08:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 35266560. Throughput: 0: 10425.6, 1: 10153.9. Samples: 35240156. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000068776_17612800.pth... [2023-12-27 06:08:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000067592_17309696.pth [2023-12-27 06:08:22,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000068952_17661952.pth... [2023-12-27 06:08:22,394][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000067720_17342464.pth [2023-12-27 06:08:22,783][20381] Updated weights for policy 1, policy_version 68784 (0.0008) [2023-12-27 06:08:22,847][20381] Updated weights for policy 1, policy_version 68794 (0.0009) [2023-12-27 06:08:22,915][20381] Updated weights for policy 1, policy_version 68804 (0.0009) [2023-12-27 06:08:22,990][20347] Updated weights for policy 0, policy_version 68955 (0.0007) [2023-12-27 06:08:23,047][20347] Updated weights for policy 0, policy_version 68965 (0.0007) [2023-12-27 06:08:23,106][20347] Updated weights for policy 0, policy_version 68975 (0.0007) [2023-12-27 06:08:23,549][20381] Updated weights for policy 1, policy_version 68814 (0.0008) [2023-12-27 06:08:23,601][20381] Updated weights for policy 1, policy_version 68824 (0.0007) [2023-12-27 06:08:23,649][20381] Updated weights for policy 1, policy_version 68834 (0.0007) [2023-12-27 06:08:23,809][20347] Updated weights for policy 0, policy_version 68985 (0.0007) [2023-12-27 06:08:23,865][20347] Updated weights for policy 0, policy_version 68995 (0.0008) [2023-12-27 06:08:23,917][20347] Updated weights for policy 0, policy_version 69005 (0.0008) [2023-12-27 06:08:23,962][20347] Updated weights for policy 0, policy_version 69015 (0.0008) [2023-12-27 06:08:24,230][20381] Updated weights for policy 1, policy_version 68844 (0.0008) [2023-12-27 06:08:24,287][20381] Updated weights for policy 1, policy_version 68854 (0.0008) [2023-12-27 06:08:24,342][20381] Updated weights for policy 1, policy_version 68864 (0.0007) [2023-12-27 06:08:24,655][20347] Updated weights for policy 0, policy_version 69025 (0.0008) [2023-12-27 06:08:24,705][20347] Updated weights for policy 0, policy_version 69035 (0.0008) [2023-12-27 06:08:24,751][20347] Updated weights for policy 0, policy_version 69045 (0.0006) [2023-12-27 06:08:24,952][20381] Updated weights for policy 1, policy_version 68874 (0.0007) [2023-12-27 06:08:25,015][20381] Updated weights for policy 1, policy_version 68884 (0.0008) [2023-12-27 06:08:25,066][20381] Updated weights for policy 1, policy_version 68894 (0.0010) [2023-12-27 06:08:25,114][20381] Updated weights for policy 1, policy_version 68904 (0.0010) [2023-12-27 06:08:25,412][20347] Updated weights for policy 0, policy_version 69055 (0.0009) [2023-12-27 06:08:25,457][20347] Updated weights for policy 0, policy_version 69065 (0.0010) [2023-12-27 06:08:25,509][20347] Updated weights for policy 0, policy_version 69075 (0.0010) [2023-12-27 06:08:25,676][20381] Updated weights for policy 1, policy_version 68914 (0.0010) [2023-12-27 06:08:25,734][20381] Updated weights for policy 1, policy_version 68924 (0.0010) [2023-12-27 06:08:25,797][20381] Updated weights for policy 1, policy_version 68934 (0.0010) [2023-12-27 06:08:26,224][20347] Updated weights for policy 0, policy_version 69085 (0.0010) [2023-12-27 06:08:26,279][20347] Updated weights for policy 0, policy_version 69095 (0.0010) [2023-12-27 06:08:26,331][20347] Updated weights for policy 0, policy_version 69105 (0.0010) [2023-12-27 06:08:26,363][20381] Updated weights for policy 1, policy_version 68944 (0.0006) [2023-12-27 06:08:26,411][20381] Updated weights for policy 1, policy_version 68954 (0.0007) [2023-12-27 06:08:26,458][20381] Updated weights for policy 1, policy_version 68964 (0.0007) [2023-12-27 06:08:27,024][20347] Updated weights for policy 0, policy_version 69115 (0.0009) [2023-12-27 06:08:27,087][20347] Updated weights for policy 0, policy_version 69125 (0.0006) [2023-12-27 06:08:27,146][20347] Updated weights for policy 0, policy_version 69135 (0.0007) [2023-12-27 06:08:27,172][20381] Updated weights for policy 1, policy_version 68974 (0.0008) [2023-12-27 06:08:27,219][20381] Updated weights for policy 1, policy_version 68984 (0.0005) [2023-12-27 06:08:27,280][20381] Updated weights for policy 1, policy_version 68994 (0.0006) [2023-12-27 06:08:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 35381248. Throughput: 0: 10479.5, 1: 10198.9. Samples: 35365632. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:27,799][20347] Updated weights for policy 0, policy_version 69145 (0.0010) [2023-12-27 06:08:27,861][20347] Updated weights for policy 0, policy_version 69155 (0.0008) [2023-12-27 06:08:27,911][20347] Updated weights for policy 0, policy_version 69165 (0.0007) [2023-12-27 06:08:27,956][20381] Updated weights for policy 1, policy_version 69004 (0.0006) [2023-12-27 06:08:27,961][20347] Updated weights for policy 0, policy_version 69175 (0.0008) [2023-12-27 06:08:28,012][20381] Updated weights for policy 1, policy_version 69014 (0.0006) [2023-12-27 06:08:28,054][20381] Updated weights for policy 1, policy_version 69024 (0.0010) [2023-12-27 06:08:28,578][20347] Updated weights for policy 0, policy_version 69185 (0.0007) [2023-12-27 06:08:28,632][20347] Updated weights for policy 0, policy_version 69195 (0.0005) [2023-12-27 06:08:28,686][20381] Updated weights for policy 1, policy_version 69034 (0.0010) [2023-12-27 06:08:28,692][20347] Updated weights for policy 0, policy_version 69205 (0.0006) [2023-12-27 06:08:28,742][20381] Updated weights for policy 1, policy_version 69044 (0.0010) [2023-12-27 06:08:28,797][20381] Updated weights for policy 1, policy_version 69054 (0.0010) [2023-12-27 06:08:28,849][20381] Updated weights for policy 1, policy_version 69064 (0.0010) [2023-12-27 06:08:29,227][20347] Updated weights for policy 0, policy_version 69215 (0.0006) [2023-12-27 06:08:29,286][20347] Updated weights for policy 0, policy_version 69225 (0.0007) [2023-12-27 06:08:29,344][20347] Updated weights for policy 0, policy_version 69235 (0.0007) [2023-12-27 06:08:29,465][20381] Updated weights for policy 1, policy_version 69074 (0.0010) [2023-12-27 06:08:29,515][20381] Updated weights for policy 1, policy_version 69084 (0.0010) [2023-12-27 06:08:29,570][20381] Updated weights for policy 1, policy_version 69094 (0.0011) [2023-12-27 06:08:30,031][20347] Updated weights for policy 0, policy_version 69245 (0.0008) [2023-12-27 06:08:30,084][20347] Updated weights for policy 0, policy_version 69255 (0.0010) [2023-12-27 06:08:30,141][20347] Updated weights for policy 0, policy_version 69265 (0.0009) [2023-12-27 06:08:30,267][20381] Updated weights for policy 1, policy_version 69104 (0.0010) [2023-12-27 06:08:30,328][20381] Updated weights for policy 1, policy_version 69114 (0.0010) [2023-12-27 06:08:30,384][20381] Updated weights for policy 1, policy_version 69124 (0.0009) [2023-12-27 06:08:30,794][20347] Updated weights for policy 0, policy_version 69275 (0.0009) [2023-12-27 06:08:30,840][20347] Updated weights for policy 0, policy_version 69285 (0.0007) [2023-12-27 06:08:30,888][20347] Updated weights for policy 0, policy_version 69295 (0.0008) [2023-12-27 06:08:31,057][20381] Updated weights for policy 1, policy_version 69134 (0.0008) [2023-12-27 06:08:31,117][20381] Updated weights for policy 1, policy_version 69144 (0.0008) [2023-12-27 06:08:31,176][20381] Updated weights for policy 1, policy_version 69154 (0.0008) [2023-12-27 06:08:31,634][20347] Updated weights for policy 0, policy_version 69305 (0.0008) [2023-12-27 06:08:31,692][20347] Updated weights for policy 0, policy_version 69315 (0.0008) [2023-12-27 06:08:31,749][20347] Updated weights for policy 0, policy_version 69325 (0.0008) [2023-12-27 06:08:31,810][20347] Updated weights for policy 0, policy_version 69336 (0.0008) [2023-12-27 06:08:31,865][20381] Updated weights for policy 1, policy_version 69164 (0.0008) [2023-12-27 06:08:31,917][20381] Updated weights for policy 1, policy_version 69174 (0.0007) [2023-12-27 06:08:31,970][20381] Updated weights for policy 1, policy_version 69184 (0.0006) [2023-12-27 06:08:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 35479552. Throughput: 0: 10494.8, 1: 10266.6. Samples: 35490700. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:32,534][20347] Updated weights for policy 0, policy_version 69346 (0.0006) [2023-12-27 06:08:32,597][20347] Updated weights for policy 0, policy_version 69356 (0.0006) [2023-12-27 06:08:32,655][20347] Updated weights for policy 0, policy_version 69366 (0.0007) [2023-12-27 06:08:32,674][20381] Updated weights for policy 1, policy_version 69194 (0.0008) [2023-12-27 06:08:32,720][20381] Updated weights for policy 1, policy_version 69204 (0.0009) [2023-12-27 06:08:32,769][20381] Updated weights for policy 1, policy_version 69214 (0.0008) [2023-12-27 06:08:32,816][20381] Updated weights for policy 1, policy_version 69224 (0.0007) [2023-12-27 06:08:33,263][20347] Updated weights for policy 0, policy_version 69376 (0.0006) [2023-12-27 06:08:33,319][20347] Updated weights for policy 0, policy_version 69386 (0.0005) [2023-12-27 06:08:33,367][20347] Updated weights for policy 0, policy_version 69396 (0.0005) [2023-12-27 06:08:33,543][20381] Updated weights for policy 1, policy_version 69234 (0.0007) [2023-12-27 06:08:33,593][20381] Updated weights for policy 1, policy_version 69244 (0.0007) [2023-12-27 06:08:33,653][20381] Updated weights for policy 1, policy_version 69254 (0.0008) [2023-12-27 06:08:33,954][20347] Updated weights for policy 0, policy_version 69406 (0.0006) [2023-12-27 06:08:34,004][20347] Updated weights for policy 0, policy_version 69416 (0.0005) [2023-12-27 06:08:34,054][20347] Updated weights for policy 0, policy_version 69426 (0.0005) [2023-12-27 06:08:34,392][20381] Updated weights for policy 1, policy_version 69264 (0.0008) [2023-12-27 06:08:34,447][20381] Updated weights for policy 1, policy_version 69274 (0.0009) [2023-12-27 06:08:34,503][20381] Updated weights for policy 1, policy_version 69284 (0.0007) [2023-12-27 06:08:34,707][20347] Updated weights for policy 0, policy_version 69436 (0.0006) [2023-12-27 06:08:34,763][20347] Updated weights for policy 0, policy_version 69446 (0.0007) [2023-12-27 06:08:34,818][20347] Updated weights for policy 0, policy_version 69456 (0.0007) [2023-12-27 06:08:35,140][20381] Updated weights for policy 1, policy_version 69294 (0.0006) [2023-12-27 06:08:35,198][20381] Updated weights for policy 1, policy_version 69304 (0.0005) [2023-12-27 06:08:35,245][20381] Updated weights for policy 1, policy_version 69314 (0.0005) [2023-12-27 06:08:35,561][20347] Updated weights for policy 0, policy_version 69466 (0.0007) [2023-12-27 06:08:35,613][20347] Updated weights for policy 0, policy_version 69476 (0.0008) [2023-12-27 06:08:35,668][20347] Updated weights for policy 0, policy_version 69486 (0.0008) [2023-12-27 06:08:35,718][20347] Updated weights for policy 0, policy_version 69496 (0.0007) [2023-12-27 06:08:35,882][20381] Updated weights for policy 1, policy_version 69324 (0.0006) [2023-12-27 06:08:35,928][20381] Updated weights for policy 1, policy_version 69334 (0.0008) [2023-12-27 06:08:35,974][20381] Updated weights for policy 1, policy_version 69344 (0.0008) [2023-12-27 06:08:36,412][20347] Updated weights for policy 0, policy_version 69506 (0.0008) [2023-12-27 06:08:36,464][20347] Updated weights for policy 0, policy_version 69516 (0.0006) [2023-12-27 06:08:36,519][20347] Updated weights for policy 0, policy_version 69526 (0.0008) [2023-12-27 06:08:36,668][20381] Updated weights for policy 1, policy_version 69354 (0.0008) [2023-12-27 06:08:36,729][20381] Updated weights for policy 1, policy_version 69364 (0.0006) [2023-12-27 06:08:36,786][20381] Updated weights for policy 1, policy_version 69374 (0.0005) [2023-12-27 06:08:36,849][20381] Updated weights for policy 1, policy_version 69384 (0.0007) [2023-12-27 06:08:37,208][20347] Updated weights for policy 0, policy_version 69536 (0.0007) [2023-12-27 06:08:37,267][20347] Updated weights for policy 0, policy_version 69546 (0.0007) [2023-12-27 06:08:37,327][20347] Updated weights for policy 0, policy_version 69556 (0.0009) [2023-12-27 06:08:37,363][19170] Fps is (10 sec: 20479.1, 60 sec: 20752.9, 300 sec: 20716.0). Total num frames: 35586048. Throughput: 0: 10514.8, 1: 10259.2. Samples: 35552960. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000069560_17817600.pth... [2023-12-27 06:08:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000068360_17506304.pth [2023-12-27 06:08:37,416][20381] Updated weights for policy 1, policy_version 69394 (0.0007) [2023-12-27 06:08:37,478][20381] Updated weights for policy 1, policy_version 69404 (0.0008) [2023-12-27 06:08:37,541][20381] Updated weights for policy 1, policy_version 69414 (0.0006) [2023-12-27 06:08:37,551][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000069416_17776640.pth... [2023-12-27 06:08:37,555][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000068200_17465344.pth [2023-12-27 06:08:38,064][20347] Updated weights for policy 0, policy_version 69566 (0.0008) [2023-12-27 06:08:38,124][20347] Updated weights for policy 0, policy_version 69576 (0.0006) [2023-12-27 06:08:38,126][20381] Updated weights for policy 1, policy_version 69424 (0.0009) [2023-12-27 06:08:38,181][20347] Updated weights for policy 0, policy_version 69586 (0.0007) [2023-12-27 06:08:38,193][20381] Updated weights for policy 1, policy_version 69434 (0.0010) [2023-12-27 06:08:38,253][20381] Updated weights for policy 1, policy_version 69444 (0.0009) [2023-12-27 06:08:38,887][20347] Updated weights for policy 0, policy_version 69596 (0.0008) [2023-12-27 06:08:38,923][20381] Updated weights for policy 1, policy_version 69454 (0.0008) [2023-12-27 06:08:38,942][20347] Updated weights for policy 0, policy_version 69606 (0.0005) [2023-12-27 06:08:38,973][20381] Updated weights for policy 1, policy_version 69464 (0.0008) [2023-12-27 06:08:38,990][20347] Updated weights for policy 0, policy_version 69616 (0.0005) [2023-12-27 06:08:39,021][20381] Updated weights for policy 1, policy_version 69474 (0.0008) [2023-12-27 06:08:39,659][20347] Updated weights for policy 0, policy_version 69626 (0.0007) [2023-12-27 06:08:39,713][20347] Updated weights for policy 0, policy_version 69636 (0.0007) [2023-12-27 06:08:39,769][20347] Updated weights for policy 0, policy_version 69646 (0.0008) [2023-12-27 06:08:39,783][20381] Updated weights for policy 1, policy_version 69484 (0.0006) [2023-12-27 06:08:39,827][20347] Updated weights for policy 0, policy_version 69656 (0.0006) [2023-12-27 06:08:39,847][20381] Updated weights for policy 1, policy_version 69494 (0.0007) [2023-12-27 06:08:39,906][20381] Updated weights for policy 1, policy_version 69504 (0.0007) [2023-12-27 06:08:40,529][20347] Updated weights for policy 0, policy_version 69666 (0.0008) [2023-12-27 06:08:40,575][20347] Updated weights for policy 0, policy_version 69676 (0.0007) [2023-12-27 06:08:40,622][20347] Updated weights for policy 0, policy_version 69686 (0.0007) [2023-12-27 06:08:40,623][20381] Updated weights for policy 1, policy_version 69514 (0.0008) [2023-12-27 06:08:40,678][20381] Updated weights for policy 1, policy_version 69524 (0.0007) [2023-12-27 06:08:40,738][20381] Updated weights for policy 1, policy_version 69534 (0.0007) [2023-12-27 06:08:40,797][20381] Updated weights for policy 1, policy_version 69544 (0.0006) [2023-12-27 06:08:41,319][20347] Updated weights for policy 0, policy_version 69696 (0.0008) [2023-12-27 06:08:41,381][20347] Updated weights for policy 0, policy_version 69706 (0.0008) [2023-12-27 06:08:41,439][20347] Updated weights for policy 0, policy_version 69716 (0.0007) [2023-12-27 06:08:41,503][20381] Updated weights for policy 1, policy_version 69554 (0.0010) [2023-12-27 06:08:41,565][20381] Updated weights for policy 1, policy_version 69564 (0.0010) [2023-12-27 06:08:41,626][20381] Updated weights for policy 1, policy_version 69574 (0.0010) [2023-12-27 06:08:42,119][20347] Updated weights for policy 0, policy_version 69726 (0.0009) [2023-12-27 06:08:42,179][20347] Updated weights for policy 0, policy_version 69736 (0.0009) [2023-12-27 06:08:42,229][20381] Updated weights for policy 1, policy_version 69584 (0.0006) [2023-12-27 06:08:42,240][20347] Updated weights for policy 0, policy_version 69746 (0.0008) [2023-12-27 06:08:42,289][20381] Updated weights for policy 1, policy_version 69594 (0.0009) [2023-12-27 06:08:42,349][20381] Updated weights for policy 1, policy_version 69604 (0.0010) [2023-12-27 06:08:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 35684352. Throughput: 0: 10438.6, 1: 10237.6. Samples: 35675104. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:42,882][20347] Updated weights for policy 0, policy_version 69756 (0.0008) [2023-12-27 06:08:42,936][20347] Updated weights for policy 0, policy_version 69766 (0.0006) [2023-12-27 06:08:42,992][20347] Updated weights for policy 0, policy_version 69776 (0.0005) [2023-12-27 06:08:43,002][20381] Updated weights for policy 1, policy_version 69614 (0.0007) [2023-12-27 06:08:43,053][20381] Updated weights for policy 1, policy_version 69624 (0.0010) [2023-12-27 06:08:43,104][20381] Updated weights for policy 1, policy_version 69634 (0.0010) [2023-12-27 06:08:43,556][20347] Updated weights for policy 0, policy_version 69786 (0.0005) [2023-12-27 06:08:43,615][20347] Updated weights for policy 0, policy_version 69796 (0.0005) [2023-12-27 06:08:43,671][20347] Updated weights for policy 0, policy_version 69806 (0.0005) [2023-12-27 06:08:43,719][20381] Updated weights for policy 1, policy_version 69644 (0.0008) [2023-12-27 06:08:43,734][20347] Updated weights for policy 0, policy_version 69816 (0.0006) [2023-12-27 06:08:43,779][20381] Updated weights for policy 1, policy_version 69654 (0.0006) [2023-12-27 06:08:43,833][20381] Updated weights for policy 1, policy_version 69664 (0.0006) [2023-12-27 06:08:44,320][20347] Updated weights for policy 0, policy_version 69826 (0.0007) [2023-12-27 06:08:44,377][20347] Updated weights for policy 0, policy_version 69836 (0.0006) [2023-12-27 06:08:44,439][20347] Updated weights for policy 0, policy_version 69846 (0.0007) [2023-12-27 06:08:44,441][20381] Updated weights for policy 1, policy_version 69674 (0.0005) [2023-12-27 06:08:44,493][20381] Updated weights for policy 1, policy_version 69684 (0.0006) [2023-12-27 06:08:44,549][20381] Updated weights for policy 1, policy_version 69694 (0.0008) [2023-12-27 06:08:44,611][20381] Updated weights for policy 1, policy_version 69704 (0.0007) [2023-12-27 06:08:45,117][20347] Updated weights for policy 0, policy_version 69856 (0.0006) [2023-12-27 06:08:45,182][20347] Updated weights for policy 0, policy_version 69866 (0.0008) [2023-12-27 06:08:45,241][20347] Updated weights for policy 0, policy_version 69876 (0.0008) [2023-12-27 06:08:45,294][20381] Updated weights for policy 1, policy_version 69714 (0.0011) [2023-12-27 06:08:45,347][20381] Updated weights for policy 1, policy_version 69724 (0.0011) [2023-12-27 06:08:45,402][20381] Updated weights for policy 1, policy_version 69734 (0.0011) [2023-12-27 06:08:45,859][20347] Updated weights for policy 0, policy_version 69886 (0.0007) [2023-12-27 06:08:45,910][20347] Updated weights for policy 0, policy_version 69896 (0.0010) [2023-12-27 06:08:45,958][20347] Updated weights for policy 0, policy_version 69906 (0.0010) [2023-12-27 06:08:46,035][20381] Updated weights for policy 1, policy_version 69744 (0.0010) [2023-12-27 06:08:46,088][20381] Updated weights for policy 1, policy_version 69754 (0.0007) [2023-12-27 06:08:46,143][20381] Updated weights for policy 1, policy_version 69764 (0.0006) [2023-12-27 06:08:46,603][20347] Updated weights for policy 0, policy_version 69916 (0.0010) [2023-12-27 06:08:46,667][20347] Updated weights for policy 0, policy_version 69926 (0.0010) [2023-12-27 06:08:46,728][20347] Updated weights for policy 0, policy_version 69936 (0.0010) [2023-12-27 06:08:46,733][20381] Updated weights for policy 1, policy_version 69774 (0.0005) [2023-12-27 06:08:46,790][20381] Updated weights for policy 1, policy_version 69784 (0.0008) [2023-12-27 06:08:46,846][20381] Updated weights for policy 1, policy_version 69794 (0.0007) [2023-12-27 06:08:47,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 35790848. Throughput: 0: 10438.0, 1: 10359.3. Samples: 35805192. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) [2023-12-27 06:08:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:47,379][20347] Updated weights for policy 0, policy_version 69946 (0.0009) [2023-12-27 06:08:47,426][20347] Updated weights for policy 0, policy_version 69956 (0.0005) [2023-12-27 06:08:47,429][20381] Updated weights for policy 1, policy_version 69804 (0.0007) [2023-12-27 06:08:47,477][20347] Updated weights for policy 0, policy_version 69966 (0.0006) [2023-12-27 06:08:47,485][20381] Updated weights for policy 1, policy_version 69814 (0.0007) [2023-12-27 06:08:47,532][20347] Updated weights for policy 0, policy_version 69976 (0.0009) [2023-12-27 06:08:47,537][20381] Updated weights for policy 1, policy_version 69824 (0.0005) [2023-12-27 06:08:48,137][20347] Updated weights for policy 0, policy_version 69986 (0.0010) [2023-12-27 06:08:48,155][20381] Updated weights for policy 1, policy_version 69834 (0.0005) [2023-12-27 06:08:48,195][20347] Updated weights for policy 0, policy_version 69996 (0.0010) [2023-12-27 06:08:48,201][20381] Updated weights for policy 1, policy_version 69844 (0.0006) [2023-12-27 06:08:48,249][20347] Updated weights for policy 0, policy_version 70006 (0.0008) [2023-12-27 06:08:48,261][20381] Updated weights for policy 1, policy_version 69854 (0.0006) [2023-12-27 06:08:48,319][20381] Updated weights for policy 1, policy_version 69864 (0.0006) [2023-12-27 06:08:48,935][20347] Updated weights for policy 0, policy_version 70016 (0.0007) [2023-12-27 06:08:48,968][20381] Updated weights for policy 1, policy_version 69874 (0.0010) [2023-12-27 06:08:48,994][20347] Updated weights for policy 0, policy_version 70026 (0.0006) [2023-12-27 06:08:49,025][20381] Updated weights for policy 1, policy_version 69884 (0.0010) [2023-12-27 06:08:49,052][20347] Updated weights for policy 0, policy_version 70036 (0.0005) [2023-12-27 06:08:49,079][20381] Updated weights for policy 1, policy_version 69894 (0.0010) [2023-12-27 06:08:49,758][20381] Updated weights for policy 1, policy_version 69904 (0.0009) [2023-12-27 06:08:49,759][20347] Updated weights for policy 0, policy_version 70046 (0.0006) [2023-12-27 06:08:49,817][20347] Updated weights for policy 0, policy_version 70056 (0.0006) [2023-12-27 06:08:49,819][20381] Updated weights for policy 1, policy_version 69914 (0.0010) [2023-12-27 06:08:49,884][20381] Updated weights for policy 1, policy_version 69924 (0.0009) [2023-12-27 06:08:49,885][20347] Updated weights for policy 0, policy_version 70066 (0.0007) [2023-12-27 06:08:50,589][20347] Updated weights for policy 0, policy_version 70076 (0.0007) [2023-12-27 06:08:50,598][20381] Updated weights for policy 1, policy_version 69934 (0.0011) [2023-12-27 06:08:50,639][20347] Updated weights for policy 0, policy_version 70086 (0.0005) [2023-12-27 06:08:50,659][20381] Updated weights for policy 1, policy_version 69944 (0.0011) [2023-12-27 06:08:50,683][20347] Updated weights for policy 0, policy_version 70096 (0.0006) [2023-12-27 06:08:50,719][20381] Updated weights for policy 1, policy_version 69954 (0.0011) [2023-12-27 06:08:51,335][20347] Updated weights for policy 0, policy_version 70106 (0.0006) [2023-12-27 06:08:51,403][20347] Updated weights for policy 0, policy_version 70116 (0.0006) [2023-12-27 06:08:51,418][20381] Updated weights for policy 1, policy_version 69964 (0.0011) [2023-12-27 06:08:51,458][20347] Updated weights for policy 0, policy_version 70126 (0.0006) [2023-12-27 06:08:51,474][20381] Updated weights for policy 1, policy_version 69974 (0.0011) [2023-12-27 06:08:51,517][20347] Updated weights for policy 0, policy_version 70136 (0.0007) [2023-12-27 06:08:51,533][20381] Updated weights for policy 1, policy_version 69984 (0.0011) [2023-12-27 06:08:52,203][20347] Updated weights for policy 0, policy_version 70146 (0.0008) [2023-12-27 06:08:52,205][20381] Updated weights for policy 1, policy_version 69994 (0.0010) [2023-12-27 06:08:52,253][20347] Updated weights for policy 0, policy_version 70156 (0.0007) [2023-12-27 06:08:52,262][20381] Updated weights for policy 1, policy_version 70004 (0.0006) [2023-12-27 06:08:52,314][20347] Updated weights for policy 0, policy_version 70166 (0.0007) [2023-12-27 06:08:52,319][20381] Updated weights for policy 1, policy_version 70014 (0.0006) [2023-12-27 06:08:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 35897344. Throughput: 0: 10438.2, 1: 10385.7. Samples: 35867396. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:08:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000070168_17973248.pth... [2023-12-27 06:08:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000068952_17661952.pth [2023-12-27 06:08:52,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000070024_17932288.pth... [2023-12-27 06:08:52,380][20381] Updated weights for policy 1, policy_version 70024 (0.0007) [2023-12-27 06:08:52,384][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000068776_17612800.pth [2023-12-27 06:08:53,010][20347] Updated weights for policy 0, policy_version 70176 (0.0008) [2023-12-27 06:08:53,066][20381] Updated weights for policy 1, policy_version 70034 (0.0006) [2023-12-27 06:08:53,071][20347] Updated weights for policy 0, policy_version 70186 (0.0006) [2023-12-27 06:08:53,124][20381] Updated weights for policy 1, policy_version 70044 (0.0006) [2023-12-27 06:08:53,130][20347] Updated weights for policy 0, policy_version 70196 (0.0006) [2023-12-27 06:08:53,184][20381] Updated weights for policy 1, policy_version 70054 (0.0005) [2023-12-27 06:08:53,744][20347] Updated weights for policy 0, policy_version 70206 (0.0006) [2023-12-27 06:08:53,751][20381] Updated weights for policy 1, policy_version 70064 (0.0005) [2023-12-27 06:08:53,800][20347] Updated weights for policy 0, policy_version 70216 (0.0007) [2023-12-27 06:08:53,809][20381] Updated weights for policy 1, policy_version 70074 (0.0005) [2023-12-27 06:08:53,855][20347] Updated weights for policy 0, policy_version 70226 (0.0006) [2023-12-27 06:08:53,857][20381] Updated weights for policy 1, policy_version 70084 (0.0005) [2023-12-27 06:08:54,525][20347] Updated weights for policy 0, policy_version 70236 (0.0006) [2023-12-27 06:08:54,530][20381] Updated weights for policy 1, policy_version 70094 (0.0005) [2023-12-27 06:08:54,574][20347] Updated weights for policy 0, policy_version 70246 (0.0005) [2023-12-27 06:08:54,592][20381] Updated weights for policy 1, policy_version 70104 (0.0006) [2023-12-27 06:08:54,620][20347] Updated weights for policy 0, policy_version 70256 (0.0005) [2023-12-27 06:08:54,653][20381] Updated weights for policy 1, policy_version 70114 (0.0005) [2023-12-27 06:08:55,257][20347] Updated weights for policy 0, policy_version 70266 (0.0006) [2023-12-27 06:08:55,292][20381] Updated weights for policy 1, policy_version 70124 (0.0006) [2023-12-27 06:08:55,313][20347] Updated weights for policy 0, policy_version 70276 (0.0006) [2023-12-27 06:08:55,356][20381] Updated weights for policy 1, policy_version 70134 (0.0006) [2023-12-27 06:08:55,366][20347] Updated weights for policy 0, policy_version 70286 (0.0005) [2023-12-27 06:08:55,418][20381] Updated weights for policy 1, policy_version 70144 (0.0007) [2023-12-27 06:08:55,421][20347] Updated weights for policy 0, policy_version 70296 (0.0005) [2023-12-27 06:08:56,034][20347] Updated weights for policy 0, policy_version 70307 (0.0008) [2023-12-27 06:08:56,079][20381] Updated weights for policy 1, policy_version 70154 (0.0007) [2023-12-27 06:08:56,082][20347] Updated weights for policy 0, policy_version 70317 (0.0007) [2023-12-27 06:08:56,130][20381] Updated weights for policy 1, policy_version 70164 (0.0005) [2023-12-27 06:08:56,137][20347] Updated weights for policy 0, policy_version 70327 (0.0007) [2023-12-27 06:08:56,182][20381] Updated weights for policy 1, policy_version 70174 (0.0005) [2023-12-27 06:08:56,245][20381] Updated weights for policy 1, policy_version 70184 (0.0005) [2023-12-27 06:08:56,807][20381] Updated weights for policy 1, policy_version 70194 (0.0008) [2023-12-27 06:08:56,850][20347] Updated weights for policy 0, policy_version 70337 (0.0006) [2023-12-27 06:08:56,861][20381] Updated weights for policy 1, policy_version 70204 (0.0007) [2023-12-27 06:08:56,905][20347] Updated weights for policy 0, policy_version 70347 (0.0007) [2023-12-27 06:08:56,913][20381] Updated weights for policy 1, policy_version 70214 (0.0005) [2023-12-27 06:08:56,955][20347] Updated weights for policy 0, policy_version 70357 (0.0006) [2023-12-27 06:08:57,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 36003840. Throughput: 0: 10460.0, 1: 10433.6. Samples: 35993352. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:08:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:08:57,550][20381] Updated weights for policy 1, policy_version 70224 (0.0007) [2023-12-27 06:08:57,594][20381] Updated weights for policy 1, policy_version 70234 (0.0007) [2023-12-27 06:08:57,640][20381] Updated weights for policy 1, policy_version 70244 (0.0006) [2023-12-27 06:08:57,653][20347] Updated weights for policy 0, policy_version 70367 (0.0006) [2023-12-27 06:08:57,701][20347] Updated weights for policy 0, policy_version 70377 (0.0007) [2023-12-27 06:08:57,748][20347] Updated weights for policy 0, policy_version 70387 (0.0008) [2023-12-27 06:08:58,364][20381] Updated weights for policy 1, policy_version 70254 (0.0007) [2023-12-27 06:08:58,428][20381] Updated weights for policy 1, policy_version 70264 (0.0008) [2023-12-27 06:08:58,486][20381] Updated weights for policy 1, policy_version 70274 (0.0008) [2023-12-27 06:08:58,493][20347] Updated weights for policy 0, policy_version 70397 (0.0007) [2023-12-27 06:08:58,554][20347] Updated weights for policy 0, policy_version 70407 (0.0007) [2023-12-27 06:08:58,615][20347] Updated weights for policy 0, policy_version 70417 (0.0008) [2023-12-27 06:08:59,194][20381] Updated weights for policy 1, policy_version 70284 (0.0009) [2023-12-27 06:08:59,250][20381] Updated weights for policy 1, policy_version 70294 (0.0010) [2023-12-27 06:08:59,309][20381] Updated weights for policy 1, policy_version 70304 (0.0008) [2023-12-27 06:08:59,335][20347] Updated weights for policy 0, policy_version 70427 (0.0008) [2023-12-27 06:08:59,393][20347] Updated weights for policy 0, policy_version 70437 (0.0008) [2023-12-27 06:08:59,450][20347] Updated weights for policy 0, policy_version 70447 (0.0008) [2023-12-27 06:08:59,969][20381] Updated weights for policy 1, policy_version 70314 (0.0009) [2023-12-27 06:09:00,018][20381] Updated weights for policy 1, policy_version 70324 (0.0010) [2023-12-27 06:09:00,069][20381] Updated weights for policy 1, policy_version 70334 (0.0010) [2023-12-27 06:09:00,118][20381] Updated weights for policy 1, policy_version 70344 (0.0010) [2023-12-27 06:09:00,125][20347] Updated weights for policy 0, policy_version 70457 (0.0008) [2023-12-27 06:09:00,187][20347] Updated weights for policy 0, policy_version 70467 (0.0008) [2023-12-27 06:09:00,250][20347] Updated weights for policy 0, policy_version 70477 (0.0008) [2023-12-27 06:09:00,302][20347] Updated weights for policy 0, policy_version 70487 (0.0006) [2023-12-27 06:09:00,830][20381] Updated weights for policy 1, policy_version 70354 (0.0010) [2023-12-27 06:09:00,882][20347] Updated weights for policy 0, policy_version 70497 (0.0005) [2023-12-27 06:09:00,888][20381] Updated weights for policy 1, policy_version 70364 (0.0010) [2023-12-27 06:09:00,927][20347] Updated weights for policy 0, policy_version 70507 (0.0005) [2023-12-27 06:09:00,939][20381] Updated weights for policy 1, policy_version 70374 (0.0010) [2023-12-27 06:09:00,975][20347] Updated weights for policy 0, policy_version 70517 (0.0006) [2023-12-27 06:09:01,621][20381] Updated weights for policy 1, policy_version 70384 (0.0007) [2023-12-27 06:09:01,676][20381] Updated weights for policy 1, policy_version 70394 (0.0008) [2023-12-27 06:09:01,679][20347] Updated weights for policy 0, policy_version 70527 (0.0009) [2023-12-27 06:09:01,734][20381] Updated weights for policy 1, policy_version 70404 (0.0008) [2023-12-27 06:09:01,736][20347] Updated weights for policy 0, policy_version 70537 (0.0010) [2023-12-27 06:09:01,798][20347] Updated weights for policy 0, policy_version 70547 (0.0010) [2023-12-27 06:09:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 36102144. Throughput: 0: 10367.5, 1: 10444.4. Samples: 36115788. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:02,431][20347] Updated weights for policy 0, policy_version 70557 (0.0009) [2023-12-27 06:09:02,437][20381] Updated weights for policy 1, policy_version 70414 (0.0008) [2023-12-27 06:09:02,481][20347] Updated weights for policy 0, policy_version 70567 (0.0008) [2023-12-27 06:09:02,491][20381] Updated weights for policy 1, policy_version 70424 (0.0010) [2023-12-27 06:09:02,535][20347] Updated weights for policy 0, policy_version 70577 (0.0008) [2023-12-27 06:09:02,548][20381] Updated weights for policy 1, policy_version 70434 (0.0010) [2023-12-27 06:09:03,212][20347] Updated weights for policy 0, policy_version 70587 (0.0007) [2023-12-27 06:09:03,225][20381] Updated weights for policy 1, policy_version 70444 (0.0010) [2023-12-27 06:09:03,268][20347] Updated weights for policy 0, policy_version 70597 (0.0005) [2023-12-27 06:09:03,278][20381] Updated weights for policy 1, policy_version 70454 (0.0010) [2023-12-27 06:09:03,321][20347] Updated weights for policy 0, policy_version 70607 (0.0005) [2023-12-27 06:09:03,330][20381] Updated weights for policy 1, policy_version 70464 (0.0010) [2023-12-27 06:09:03,866][20347] Updated weights for policy 0, policy_version 70617 (0.0006) [2023-12-27 06:09:03,902][20381] Updated weights for policy 1, policy_version 70474 (0.0009) [2023-12-27 06:09:03,927][20347] Updated weights for policy 0, policy_version 70627 (0.0010) [2023-12-27 06:09:03,953][20381] Updated weights for policy 1, policy_version 70484 (0.0006) [2023-12-27 06:09:03,992][20347] Updated weights for policy 0, policy_version 70637 (0.0011) [2023-12-27 06:09:04,009][20381] Updated weights for policy 1, policy_version 70494 (0.0005) [2023-12-27 06:09:04,057][20347] Updated weights for policy 0, policy_version 70647 (0.0011) [2023-12-27 06:09:04,063][20381] Updated weights for policy 1, policy_version 70504 (0.0006) [2023-12-27 06:09:04,663][20347] Updated weights for policy 0, policy_version 70657 (0.0006) [2023-12-27 06:09:04,701][20381] Updated weights for policy 1, policy_version 70514 (0.0007) [2023-12-27 06:09:04,708][20347] Updated weights for policy 0, policy_version 70667 (0.0006) [2023-12-27 06:09:04,750][20381] Updated weights for policy 1, policy_version 70524 (0.0008) [2023-12-27 06:09:04,762][20347] Updated weights for policy 0, policy_version 70677 (0.0005) [2023-12-27 06:09:04,796][20381] Updated weights for policy 1, policy_version 70534 (0.0007) [2023-12-27 06:09:05,352][20347] Updated weights for policy 0, policy_version 70687 (0.0005) [2023-12-27 06:09:05,410][20347] Updated weights for policy 0, policy_version 70697 (0.0009) [2023-12-27 06:09:05,459][20347] Updated weights for policy 0, policy_version 70707 (0.0010) [2023-12-27 06:09:05,532][20381] Updated weights for policy 1, policy_version 70544 (0.0008) [2023-12-27 06:09:05,587][20381] Updated weights for policy 1, policy_version 70554 (0.0008) [2023-12-27 06:09:05,639][20381] Updated weights for policy 1, policy_version 70564 (0.0006) [2023-12-27 06:09:06,078][20347] Updated weights for policy 0, policy_version 70717 (0.0008) [2023-12-27 06:09:06,143][20347] Updated weights for policy 0, policy_version 70727 (0.0010) [2023-12-27 06:09:06,200][20347] Updated weights for policy 0, policy_version 70737 (0.0010) [2023-12-27 06:09:06,373][20381] Updated weights for policy 1, policy_version 70574 (0.0007) [2023-12-27 06:09:06,430][20381] Updated weights for policy 1, policy_version 70584 (0.0008) [2023-12-27 06:09:06,492][20381] Updated weights for policy 1, policy_version 70594 (0.0007) [2023-12-27 06:09:06,813][20347] Updated weights for policy 0, policy_version 70747 (0.0009) [2023-12-27 06:09:06,859][20347] Updated weights for policy 0, policy_version 70757 (0.0010) [2023-12-27 06:09:06,910][20347] Updated weights for policy 0, policy_version 70767 (0.0010) [2023-12-27 06:09:07,157][20381] Updated weights for policy 1, policy_version 70604 (0.0007) [2023-12-27 06:09:07,219][20381] Updated weights for policy 1, policy_version 70614 (0.0006) [2023-12-27 06:09:07,273][20381] Updated weights for policy 1, policy_version 70624 (0.0005) [2023-12-27 06:09:07,363][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 36216832. Throughput: 0: 10423.8, 1: 10489.3. Samples: 36181248. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000070632_18087936.pth... [2023-12-27 06:09:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000070776_18128896.pth... [2023-12-27 06:09:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000069416_17776640.pth [2023-12-27 06:09:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000069560_17817600.pth [2023-12-27 06:09:07,596][20347] Updated weights for policy 0, policy_version 70777 (0.0007) [2023-12-27 06:09:07,655][20347] Updated weights for policy 0, policy_version 70787 (0.0010) [2023-12-27 06:09:07,715][20347] Updated weights for policy 0, policy_version 70797 (0.0010) [2023-12-27 06:09:07,772][20347] Updated weights for policy 0, policy_version 70807 (0.0010) [2023-12-27 06:09:07,916][20381] Updated weights for policy 1, policy_version 70634 (0.0006) [2023-12-27 06:09:07,976][20381] Updated weights for policy 1, policy_version 70644 (0.0007) [2023-12-27 06:09:08,030][20381] Updated weights for policy 1, policy_version 70654 (0.0007) [2023-12-27 06:09:08,091][20381] Updated weights for policy 1, policy_version 70664 (0.0007) [2023-12-27 06:09:08,423][20347] Updated weights for policy 0, policy_version 70817 (0.0010) [2023-12-27 06:09:08,483][20347] Updated weights for policy 0, policy_version 70827 (0.0009) [2023-12-27 06:09:08,542][20347] Updated weights for policy 0, policy_version 70837 (0.0007) [2023-12-27 06:09:08,798][20381] Updated weights for policy 1, policy_version 70674 (0.0008) [2023-12-27 06:09:08,852][20381] Updated weights for policy 1, policy_version 70684 (0.0007) [2023-12-27 06:09:08,899][20381] Updated weights for policy 1, policy_version 70694 (0.0007) [2023-12-27 06:09:09,247][20347] Updated weights for policy 0, policy_version 70847 (0.0009) [2023-12-27 06:09:09,313][20347] Updated weights for policy 0, policy_version 70857 (0.0008) [2023-12-27 06:09:09,385][20347] Updated weights for policy 0, policy_version 70867 (0.0011) [2023-12-27 06:09:09,607][20381] Updated weights for policy 1, policy_version 70704 (0.0007) [2023-12-27 06:09:09,665][20381] Updated weights for policy 1, policy_version 70714 (0.0008) [2023-12-27 06:09:09,728][20381] Updated weights for policy 1, policy_version 70724 (0.0008) [2023-12-27 06:09:10,049][20347] Updated weights for policy 0, policy_version 70877 (0.0009) [2023-12-27 06:09:10,105][20347] Updated weights for policy 0, policy_version 70887 (0.0008) [2023-12-27 06:09:10,157][20347] Updated weights for policy 0, policy_version 70897 (0.0008) [2023-12-27 06:09:10,412][20381] Updated weights for policy 1, policy_version 70734 (0.0007) [2023-12-27 06:09:10,470][20381] Updated weights for policy 1, policy_version 70744 (0.0007) [2023-12-27 06:09:10,522][20381] Updated weights for policy 1, policy_version 70754 (0.0007) [2023-12-27 06:09:10,844][20347] Updated weights for policy 0, policy_version 70907 (0.0009) [2023-12-27 06:09:10,902][20347] Updated weights for policy 0, policy_version 70917 (0.0010) [2023-12-27 06:09:10,957][20347] Updated weights for policy 0, policy_version 70927 (0.0010) [2023-12-27 06:09:11,255][20381] Updated weights for policy 1, policy_version 70764 (0.0007) [2023-12-27 06:09:11,321][20381] Updated weights for policy 1, policy_version 70774 (0.0007) [2023-12-27 06:09:11,387][20381] Updated weights for policy 1, policy_version 70784 (0.0008) [2023-12-27 06:09:11,669][20347] Updated weights for policy 0, policy_version 70937 (0.0007) [2023-12-27 06:09:11,734][20347] Updated weights for policy 0, policy_version 70947 (0.0009) [2023-12-27 06:09:11,806][20347] Updated weights for policy 0, policy_version 70957 (0.0008) [2023-12-27 06:09:11,870][20347] Updated weights for policy 0, policy_version 70967 (0.0008) [2023-12-27 06:09:12,093][20381] Updated weights for policy 1, policy_version 70794 (0.0007) [2023-12-27 06:09:12,142][20381] Updated weights for policy 1, policy_version 70804 (0.0005) [2023-12-27 06:09:12,194][20381] Updated weights for policy 1, policy_version 70814 (0.0005) [2023-12-27 06:09:12,253][20381] Updated weights for policy 1, policy_version 70824 (0.0008) [2023-12-27 06:09:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 36315136. Throughput: 0: 10456.3, 1: 10390.6. Samples: 36303740. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:12,640][20347] Updated weights for policy 0, policy_version 70977 (0.0008) [2023-12-27 06:09:12,698][20347] Updated weights for policy 0, policy_version 70987 (0.0009) [2023-12-27 06:09:12,750][20347] Updated weights for policy 0, policy_version 70997 (0.0010) [2023-12-27 06:09:12,894][20381] Updated weights for policy 1, policy_version 70834 (0.0011) [2023-12-27 06:09:12,951][20381] Updated weights for policy 1, policy_version 70844 (0.0007) [2023-12-27 06:09:12,999][20381] Updated weights for policy 1, policy_version 70854 (0.0007) [2023-12-27 06:09:13,370][20347] Updated weights for policy 0, policy_version 71007 (0.0007) [2023-12-27 06:09:13,426][20347] Updated weights for policy 0, policy_version 71017 (0.0009) [2023-12-27 06:09:13,482][20347] Updated weights for policy 0, policy_version 71027 (0.0009) [2023-12-27 06:09:13,658][20381] Updated weights for policy 1, policy_version 70864 (0.0005) [2023-12-27 06:09:13,710][20381] Updated weights for policy 1, policy_version 70874 (0.0005) [2023-12-27 06:09:13,758][20381] Updated weights for policy 1, policy_version 70884 (0.0005) [2023-12-27 06:09:14,057][20347] Updated weights for policy 0, policy_version 71037 (0.0008) [2023-12-27 06:09:14,113][20347] Updated weights for policy 0, policy_version 71047 (0.0005) [2023-12-27 06:09:14,163][20347] Updated weights for policy 0, policy_version 71057 (0.0007) [2023-12-27 06:09:14,278][20381] Updated weights for policy 1, policy_version 70894 (0.0006) [2023-12-27 06:09:14,337][20381] Updated weights for policy 1, policy_version 70904 (0.0007) [2023-12-27 06:09:14,397][20381] Updated weights for policy 1, policy_version 70914 (0.0008) [2023-12-27 06:09:14,830][20347] Updated weights for policy 0, policy_version 71067 (0.0010) [2023-12-27 06:09:14,895][20347] Updated weights for policy 0, policy_version 71077 (0.0008) [2023-12-27 06:09:14,955][20347] Updated weights for policy 0, policy_version 71087 (0.0008) [2023-12-27 06:09:15,059][20381] Updated weights for policy 1, policy_version 70924 (0.0007) [2023-12-27 06:09:15,123][20381] Updated weights for policy 1, policy_version 70934 (0.0008) [2023-12-27 06:09:15,185][20381] Updated weights for policy 1, policy_version 70944 (0.0008) [2023-12-27 06:09:15,627][20347] Updated weights for policy 0, policy_version 71097 (0.0008) [2023-12-27 06:09:15,681][20347] Updated weights for policy 0, policy_version 71107 (0.0010) [2023-12-27 06:09:15,733][20347] Updated weights for policy 0, policy_version 71117 (0.0010) [2023-12-27 06:09:15,784][20347] Updated weights for policy 0, policy_version 71127 (0.0010) [2023-12-27 06:09:15,832][20381] Updated weights for policy 1, policy_version 70954 (0.0007) [2023-12-27 06:09:15,880][20381] Updated weights for policy 1, policy_version 70964 (0.0008) [2023-12-27 06:09:15,929][20381] Updated weights for policy 1, policy_version 70974 (0.0009) [2023-12-27 06:09:15,976][20381] Updated weights for policy 1, policy_version 70984 (0.0007) [2023-12-27 06:09:16,440][20347] Updated weights for policy 0, policy_version 71137 (0.0010) [2023-12-27 06:09:16,504][20347] Updated weights for policy 0, policy_version 71147 (0.0009) [2023-12-27 06:09:16,565][20347] Updated weights for policy 0, policy_version 71157 (0.0006) [2023-12-27 06:09:16,721][20381] Updated weights for policy 1, policy_version 70994 (0.0006) [2023-12-27 06:09:16,785][20381] Updated weights for policy 1, policy_version 71004 (0.0009) [2023-12-27 06:09:16,842][20381] Updated weights for policy 1, policy_version 71014 (0.0011) [2023-12-27 06:09:17,134][20347] Updated weights for policy 0, policy_version 71167 (0.0009) [2023-12-27 06:09:17,180][20347] Updated weights for policy 0, policy_version 71177 (0.0009) [2023-12-27 06:09:17,236][20347] Updated weights for policy 0, policy_version 71187 (0.0005) [2023-12-27 06:09:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 36421632. Throughput: 0: 10451.0, 1: 10441.6. Samples: 36430868. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:17,407][20381] Updated weights for policy 1, policy_version 71024 (0.0006) [2023-12-27 06:09:17,464][20381] Updated weights for policy 1, policy_version 71034 (0.0005) [2023-12-27 06:09:17,516][20381] Updated weights for policy 1, policy_version 71044 (0.0006) [2023-12-27 06:09:17,875][20347] Updated weights for policy 0, policy_version 71197 (0.0007) [2023-12-27 06:09:17,924][20347] Updated weights for policy 0, policy_version 71207 (0.0008) [2023-12-27 06:09:17,972][20347] Updated weights for policy 0, policy_version 71217 (0.0007) [2023-12-27 06:09:18,113][20381] Updated weights for policy 1, policy_version 71054 (0.0007) [2023-12-27 06:09:18,166][20381] Updated weights for policy 1, policy_version 71064 (0.0005) [2023-12-27 06:09:18,217][20381] Updated weights for policy 1, policy_version 71074 (0.0010) [2023-12-27 06:09:18,618][20347] Updated weights for policy 0, policy_version 71227 (0.0007) [2023-12-27 06:09:18,672][20347] Updated weights for policy 0, policy_version 71237 (0.0007) [2023-12-27 06:09:18,731][20347] Updated weights for policy 0, policy_version 71247 (0.0006) [2023-12-27 06:09:18,878][20381] Updated weights for policy 1, policy_version 71084 (0.0010) [2023-12-27 06:09:18,933][20381] Updated weights for policy 1, policy_version 71094 (0.0010) [2023-12-27 06:09:18,987][20381] Updated weights for policy 1, policy_version 71104 (0.0010) [2023-12-27 06:09:19,403][20347] Updated weights for policy 0, policy_version 71257 (0.0005) [2023-12-27 06:09:19,458][20347] Updated weights for policy 0, policy_version 71267 (0.0005) [2023-12-27 06:09:19,521][20347] Updated weights for policy 0, policy_version 71277 (0.0006) [2023-12-27 06:09:19,579][20347] Updated weights for policy 0, policy_version 71287 (0.0007) [2023-12-27 06:09:19,594][20381] Updated weights for policy 1, policy_version 71114 (0.0010) [2023-12-27 06:09:19,654][20381] Updated weights for policy 1, policy_version 71124 (0.0008) [2023-12-27 06:09:19,710][20381] Updated weights for policy 1, policy_version 71134 (0.0008) [2023-12-27 06:09:19,770][20381] Updated weights for policy 1, policy_version 71144 (0.0008) [2023-12-27 06:09:20,217][20347] Updated weights for policy 0, policy_version 71297 (0.0006) [2023-12-27 06:09:20,282][20347] Updated weights for policy 0, policy_version 71307 (0.0007) [2023-12-27 06:09:20,344][20347] Updated weights for policy 0, policy_version 71317 (0.0008) [2023-12-27 06:09:20,512][20381] Updated weights for policy 1, policy_version 71154 (0.0006) [2023-12-27 06:09:20,569][20381] Updated weights for policy 1, policy_version 71164 (0.0008) [2023-12-27 06:09:20,623][20381] Updated weights for policy 1, policy_version 71174 (0.0008) [2023-12-27 06:09:21,010][20347] Updated weights for policy 0, policy_version 71327 (0.0008) [2023-12-27 06:09:21,075][20347] Updated weights for policy 0, policy_version 71337 (0.0009) [2023-12-27 06:09:21,134][20347] Updated weights for policy 0, policy_version 71347 (0.0009) [2023-12-27 06:09:21,319][20381] Updated weights for policy 1, policy_version 71184 (0.0009) [2023-12-27 06:09:21,387][20381] Updated weights for policy 1, policy_version 71194 (0.0008) [2023-12-27 06:09:21,457][20381] Updated weights for policy 1, policy_version 71204 (0.0008) [2023-12-27 06:09:21,836][20347] Updated weights for policy 0, policy_version 71357 (0.0008) [2023-12-27 06:09:21,896][20347] Updated weights for policy 0, policy_version 71367 (0.0008) [2023-12-27 06:09:21,964][20347] Updated weights for policy 0, policy_version 71377 (0.0008) [2023-12-27 06:09:22,145][20381] Updated weights for policy 1, policy_version 71214 (0.0007) [2023-12-27 06:09:22,206][20381] Updated weights for policy 1, policy_version 71224 (0.0008) [2023-12-27 06:09:22,262][20381] Updated weights for policy 1, policy_version 71234 (0.0007) [2023-12-27 06:09:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 36528128. Throughput: 0: 10449.0, 1: 10482.3. Samples: 36494860. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000071240_18243584.pth... [2023-12-27 06:09:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000071384_18284544.pth... [2023-12-27 06:09:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000070024_17932288.pth [2023-12-27 06:09:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000070168_17973248.pth [2023-12-27 06:09:22,644][20347] Updated weights for policy 0, policy_version 71387 (0.0008) [2023-12-27 06:09:22,700][20347] Updated weights for policy 0, policy_version 71397 (0.0008) [2023-12-27 06:09:22,749][20347] Updated weights for policy 0, policy_version 71407 (0.0008) [2023-12-27 06:09:22,998][20381] Updated weights for policy 1, policy_version 71244 (0.0007) [2023-12-27 06:09:23,062][20381] Updated weights for policy 1, policy_version 71254 (0.0006) [2023-12-27 06:09:23,120][20381] Updated weights for policy 1, policy_version 71264 (0.0007) [2023-12-27 06:09:23,424][20347] Updated weights for policy 0, policy_version 71417 (0.0008) [2023-12-27 06:09:23,483][20347] Updated weights for policy 0, policy_version 71427 (0.0008) [2023-12-27 06:09:23,544][20347] Updated weights for policy 0, policy_version 71437 (0.0007) [2023-12-27 06:09:23,605][20347] Updated weights for policy 0, policy_version 71447 (0.0005) [2023-12-27 06:09:23,676][20381] Updated weights for policy 1, policy_version 71274 (0.0005) [2023-12-27 06:09:23,734][20381] Updated weights for policy 1, policy_version 71284 (0.0006) [2023-12-27 06:09:23,787][20381] Updated weights for policy 1, policy_version 71294 (0.0008) [2023-12-27 06:09:23,838][20381] Updated weights for policy 1, policy_version 71304 (0.0006) [2023-12-27 06:09:24,208][20347] Updated weights for policy 0, policy_version 71457 (0.0005) [2023-12-27 06:09:24,257][20347] Updated weights for policy 0, policy_version 71467 (0.0005) [2023-12-27 06:09:24,307][20347] Updated weights for policy 0, policy_version 71477 (0.0005) [2023-12-27 06:09:24,535][20381] Updated weights for policy 1, policy_version 71314 (0.0007) [2023-12-27 06:09:24,582][20381] Updated weights for policy 1, policy_version 71324 (0.0009) [2023-12-27 06:09:24,627][20381] Updated weights for policy 1, policy_version 71334 (0.0008) [2023-12-27 06:09:24,940][20347] Updated weights for policy 0, policy_version 71487 (0.0010) [2023-12-27 06:09:24,994][20347] Updated weights for policy 0, policy_version 71497 (0.0010) [2023-12-27 06:09:25,045][20347] Updated weights for policy 0, policy_version 71507 (0.0009) [2023-12-27 06:09:25,303][20381] Updated weights for policy 1, policy_version 71344 (0.0008) [2023-12-27 06:09:25,346][20381] Updated weights for policy 1, policy_version 71354 (0.0007) [2023-12-27 06:09:25,391][20381] Updated weights for policy 1, policy_version 71364 (0.0007) [2023-12-27 06:09:25,734][20347] Updated weights for policy 0, policy_version 71517 (0.0010) [2023-12-27 06:09:25,789][20347] Updated weights for policy 0, policy_version 71527 (0.0010) [2023-12-27 06:09:25,840][20347] Updated weights for policy 0, policy_version 71537 (0.0010) [2023-12-27 06:09:26,017][20381] Updated weights for policy 1, policy_version 71374 (0.0006) [2023-12-27 06:09:26,074][20381] Updated weights for policy 1, policy_version 71384 (0.0009) [2023-12-27 06:09:26,131][20381] Updated weights for policy 1, policy_version 71394 (0.0010) [2023-12-27 06:09:26,470][20347] Updated weights for policy 0, policy_version 71547 (0.0009) [2023-12-27 06:09:26,518][20347] Updated weights for policy 0, policy_version 71557 (0.0005) [2023-12-27 06:09:26,571][20347] Updated weights for policy 0, policy_version 71567 (0.0005) [2023-12-27 06:09:26,823][20381] Updated weights for policy 1, policy_version 71404 (0.0008) [2023-12-27 06:09:26,870][20381] Updated weights for policy 1, policy_version 71414 (0.0005) [2023-12-27 06:09:26,921][20381] Updated weights for policy 1, policy_version 71424 (0.0005) [2023-12-27 06:09:27,176][20347] Updated weights for policy 0, policy_version 71577 (0.0006) [2023-12-27 06:09:27,224][20347] Updated weights for policy 0, policy_version 71587 (0.0010) [2023-12-27 06:09:27,272][20347] Updated weights for policy 0, policy_version 71597 (0.0010) [2023-12-27 06:09:27,323][20347] Updated weights for policy 0, policy_version 71607 (0.0009) [2023-12-27 06:09:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 36634624. Throughput: 0: 10495.2, 1: 10480.9. Samples: 36619028. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:27,543][20381] Updated weights for policy 1, policy_version 71434 (0.0005) [2023-12-27 06:09:27,594][20381] Updated weights for policy 1, policy_version 71444 (0.0008) [2023-12-27 06:09:27,648][20381] Updated weights for policy 1, policy_version 71454 (0.0006) [2023-12-27 06:09:27,712][20381] Updated weights for policy 1, policy_version 71464 (0.0005) [2023-12-27 06:09:27,908][20347] Updated weights for policy 0, policy_version 71617 (0.0005) [2023-12-27 06:09:27,957][20347] Updated weights for policy 0, policy_version 71627 (0.0005) [2023-12-27 06:09:28,005][20347] Updated weights for policy 0, policy_version 71637 (0.0006) [2023-12-27 06:09:28,333][20381] Updated weights for policy 1, policy_version 71474 (0.0008) [2023-12-27 06:09:28,396][20381] Updated weights for policy 1, policy_version 71484 (0.0006) [2023-12-27 06:09:28,464][20381] Updated weights for policy 1, policy_version 71494 (0.0006) [2023-12-27 06:09:28,648][20347] Updated weights for policy 0, policy_version 71647 (0.0006) [2023-12-27 06:09:28,709][20347] Updated weights for policy 0, policy_version 71657 (0.0007) [2023-12-27 06:09:28,774][20347] Updated weights for policy 0, policy_version 71667 (0.0007) [2023-12-27 06:09:29,092][20381] Updated weights for policy 1, policy_version 71504 (0.0007) [2023-12-27 06:09:29,140][20381] Updated weights for policy 1, policy_version 71514 (0.0007) [2023-12-27 06:09:29,192][20381] Updated weights for policy 1, policy_version 71524 (0.0008) [2023-12-27 06:09:29,423][20347] Updated weights for policy 0, policy_version 71677 (0.0007) [2023-12-27 06:09:29,482][20347] Updated weights for policy 0, policy_version 71687 (0.0007) [2023-12-27 06:09:29,537][20347] Updated weights for policy 0, policy_version 71697 (0.0008) [2023-12-27 06:09:29,904][20381] Updated weights for policy 1, policy_version 71534 (0.0007) [2023-12-27 06:09:29,961][20381] Updated weights for policy 1, policy_version 71544 (0.0008) [2023-12-27 06:09:30,017][20381] Updated weights for policy 1, policy_version 71554 (0.0008) [2023-12-27 06:09:30,205][20347] Updated weights for policy 0, policy_version 71707 (0.0007) [2023-12-27 06:09:30,254][20347] Updated weights for policy 0, policy_version 71717 (0.0005) [2023-12-27 06:09:30,304][20347] Updated weights for policy 0, policy_version 71727 (0.0005) [2023-12-27 06:09:30,742][20381] Updated weights for policy 1, policy_version 71564 (0.0008) [2023-12-27 06:09:30,791][20381] Updated weights for policy 1, policy_version 71574 (0.0008) [2023-12-27 06:09:30,839][20381] Updated weights for policy 1, policy_version 71584 (0.0009) [2023-12-27 06:09:30,866][20347] Updated weights for policy 0, policy_version 71737 (0.0007) [2023-12-27 06:09:30,920][20347] Updated weights for policy 0, policy_version 71747 (0.0005) [2023-12-27 06:09:30,971][20347] Updated weights for policy 0, policy_version 71757 (0.0005) [2023-12-27 06:09:31,024][20347] Updated weights for policy 0, policy_version 71767 (0.0006) [2023-12-27 06:09:31,518][20381] Updated weights for policy 1, policy_version 71594 (0.0007) [2023-12-27 06:09:31,582][20381] Updated weights for policy 1, policy_version 71604 (0.0008) [2023-12-27 06:09:31,645][20381] Updated weights for policy 1, policy_version 71614 (0.0008) [2023-12-27 06:09:31,706][20381] Updated weights for policy 1, policy_version 71624 (0.0008) [2023-12-27 06:09:31,708][20347] Updated weights for policy 0, policy_version 71777 (0.0006) [2023-12-27 06:09:31,756][20347] Updated weights for policy 0, policy_version 71787 (0.0008) [2023-12-27 06:09:31,806][20347] Updated weights for policy 0, policy_version 71797 (0.0007) [2023-12-27 06:09:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 36732928. Throughput: 0: 10526.1, 1: 10401.8. Samples: 36746944. Policy #0 lag: (min: 41.0, avg: 47.9, max: 48.0) [2023-12-27 06:09:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:32,395][20381] Updated weights for policy 1, policy_version 71634 (0.0006) [2023-12-27 06:09:32,458][20381] Updated weights for policy 1, policy_version 71644 (0.0006) [2023-12-27 06:09:32,515][20381] Updated weights for policy 1, policy_version 71654 (0.0006) [2023-12-27 06:09:32,539][20347] Updated weights for policy 0, policy_version 71807 (0.0007) [2023-12-27 06:09:32,599][20347] Updated weights for policy 0, policy_version 71817 (0.0006) [2023-12-27 06:09:32,662][20347] Updated weights for policy 0, policy_version 71827 (0.0005) [2023-12-27 06:09:33,168][20381] Updated weights for policy 1, policy_version 71664 (0.0007) [2023-12-27 06:09:33,177][20347] Updated weights for policy 0, policy_version 71837 (0.0006) [2023-12-27 06:09:33,216][20381] Updated weights for policy 1, policy_version 71674 (0.0005) [2023-12-27 06:09:33,229][20347] Updated weights for policy 0, policy_version 71847 (0.0006) [2023-12-27 06:09:33,267][20381] Updated weights for policy 1, policy_version 71684 (0.0005) [2023-12-27 06:09:33,280][20347] Updated weights for policy 0, policy_version 71857 (0.0006) [2023-12-27 06:09:33,912][20381] Updated weights for policy 1, policy_version 71694 (0.0005) [2023-12-27 06:09:33,968][20381] Updated weights for policy 1, policy_version 71704 (0.0005) [2023-12-27 06:09:34,010][20347] Updated weights for policy 0, policy_version 71867 (0.0007) [2023-12-27 06:09:34,026][20381] Updated weights for policy 1, policy_version 71714 (0.0009) [2023-12-27 06:09:34,065][20347] Updated weights for policy 0, policy_version 71877 (0.0006) [2023-12-27 06:09:34,125][20347] Updated weights for policy 0, policy_version 71887 (0.0006) [2023-12-27 06:09:34,564][20381] Updated weights for policy 1, policy_version 71724 (0.0008) [2023-12-27 06:09:34,621][20381] Updated weights for policy 1, policy_version 71734 (0.0005) [2023-12-27 06:09:34,678][20381] Updated weights for policy 1, policy_version 71744 (0.0005) [2023-12-27 06:09:34,822][20347] Updated weights for policy 0, policy_version 71897 (0.0008) [2023-12-27 06:09:34,883][20347] Updated weights for policy 0, policy_version 71907 (0.0009) [2023-12-27 06:09:34,939][20347] Updated weights for policy 0, policy_version 71917 (0.0007) [2023-12-27 06:09:35,000][20347] Updated weights for policy 0, policy_version 71927 (0.0006) [2023-12-27 06:09:35,319][20381] Updated weights for policy 1, policy_version 71754 (0.0009) [2023-12-27 06:09:35,375][20381] Updated weights for policy 1, policy_version 71764 (0.0005) [2023-12-27 06:09:35,431][20381] Updated weights for policy 1, policy_version 71774 (0.0005) [2023-12-27 06:09:35,497][20381] Updated weights for policy 1, policy_version 71784 (0.0005) [2023-12-27 06:09:35,622][20347] Updated weights for policy 0, policy_version 71937 (0.0007) [2023-12-27 06:09:35,673][20347] Updated weights for policy 0, policy_version 71947 (0.0008) [2023-12-27 06:09:35,729][20347] Updated weights for policy 0, policy_version 71957 (0.0006) [2023-12-27 06:09:36,113][20381] Updated weights for policy 1, policy_version 71794 (0.0010) [2023-12-27 06:09:36,175][20381] Updated weights for policy 1, policy_version 71804 (0.0007) [2023-12-27 06:09:36,236][20381] Updated weights for policy 1, policy_version 71814 (0.0011) [2023-12-27 06:09:36,429][20347] Updated weights for policy 0, policy_version 71967 (0.0007) [2023-12-27 06:09:36,488][20347] Updated weights for policy 0, policy_version 71977 (0.0007) [2023-12-27 06:09:36,550][20347] Updated weights for policy 0, policy_version 71987 (0.0007) [2023-12-27 06:09:36,888][20381] Updated weights for policy 1, policy_version 71824 (0.0006) [2023-12-27 06:09:36,948][20381] Updated weights for policy 1, policy_version 71834 (0.0005) [2023-12-27 06:09:37,010][20381] Updated weights for policy 1, policy_version 71844 (0.0005) [2023-12-27 06:09:37,221][20347] Updated weights for policy 0, policy_version 71997 (0.0007) [2023-12-27 06:09:37,270][20347] Updated weights for policy 0, policy_version 72007 (0.0005) [2023-12-27 06:09:37,320][20347] Updated weights for policy 0, policy_version 72017 (0.0007) [2023-12-27 06:09:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.3, 300 sec: 20771.6). Total num frames: 36847616. Throughput: 0: 10537.5, 1: 10435.6. Samples: 36811184. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:09:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000071848_18399232.pth... [2023-12-27 06:09:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000072024_18448384.pth... [2023-12-27 06:09:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000070632_18087936.pth [2023-12-27 06:09:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000070776_18128896.pth [2023-12-27 06:09:37,595][20381] Updated weights for policy 1, policy_version 71854 (0.0009) [2023-12-27 06:09:37,653][20381] Updated weights for policy 1, policy_version 71864 (0.0011) [2023-12-27 06:09:37,717][20381] Updated weights for policy 1, policy_version 71874 (0.0010) [2023-12-27 06:09:37,995][20347] Updated weights for policy 0, policy_version 72027 (0.0007) [2023-12-27 06:09:38,057][20347] Updated weights for policy 0, policy_version 72037 (0.0007) [2023-12-27 06:09:38,109][20347] Updated weights for policy 0, policy_version 72047 (0.0007) [2023-12-27 06:09:38,384][20381] Updated weights for policy 1, policy_version 71884 (0.0011) [2023-12-27 06:09:38,433][20381] Updated weights for policy 1, policy_version 71894 (0.0011) [2023-12-27 06:09:38,490][20381] Updated weights for policy 1, policy_version 71904 (0.0010) [2023-12-27 06:09:38,857][20347] Updated weights for policy 0, policy_version 72057 (0.0006) [2023-12-27 06:09:38,913][20347] Updated weights for policy 0, policy_version 72067 (0.0007) [2023-12-27 06:09:38,968][20347] Updated weights for policy 0, policy_version 72077 (0.0007) [2023-12-27 06:09:39,019][20347] Updated weights for policy 0, policy_version 72087 (0.0007) [2023-12-27 06:09:39,177][20381] Updated weights for policy 1, policy_version 71914 (0.0010) [2023-12-27 06:09:39,238][20381] Updated weights for policy 1, policy_version 71924 (0.0008) [2023-12-27 06:09:39,298][20381] Updated weights for policy 1, policy_version 71934 (0.0008) [2023-12-27 06:09:39,359][20381] Updated weights for policy 1, policy_version 71944 (0.0008) [2023-12-27 06:09:39,726][20347] Updated weights for policy 0, policy_version 72097 (0.0009) [2023-12-27 06:09:39,784][20347] Updated weights for policy 0, policy_version 72107 (0.0008) [2023-12-27 06:09:39,846][20347] Updated weights for policy 0, policy_version 72117 (0.0007) [2023-12-27 06:09:40,124][20381] Updated weights for policy 1, policy_version 71954 (0.0007) [2023-12-27 06:09:40,178][20381] Updated weights for policy 1, policy_version 71964 (0.0007) [2023-12-27 06:09:40,241][20381] Updated weights for policy 1, policy_version 71974 (0.0006) [2023-12-27 06:09:40,599][20347] Updated weights for policy 0, policy_version 72127 (0.0007) [2023-12-27 06:09:40,659][20347] Updated weights for policy 0, policy_version 72137 (0.0008) [2023-12-27 06:09:40,712][20347] Updated weights for policy 0, policy_version 72147 (0.0008) [2023-12-27 06:09:40,903][20381] Updated weights for policy 1, policy_version 71984 (0.0008) [2023-12-27 06:09:40,961][20381] Updated weights for policy 1, policy_version 71994 (0.0007) [2023-12-27 06:09:41,018][20381] Updated weights for policy 1, policy_version 72004 (0.0008) [2023-12-27 06:09:41,431][20347] Updated weights for policy 0, policy_version 72157 (0.0006) [2023-12-27 06:09:41,497][20347] Updated weights for policy 0, policy_version 72167 (0.0005) [2023-12-27 06:09:41,562][20347] Updated weights for policy 0, policy_version 72177 (0.0006) [2023-12-27 06:09:41,692][20381] Updated weights for policy 1, policy_version 72014 (0.0008) [2023-12-27 06:09:41,757][20381] Updated weights for policy 1, policy_version 72024 (0.0009) [2023-12-27 06:09:41,809][20381] Updated weights for policy 1, policy_version 72034 (0.0008) [2023-12-27 06:09:42,206][20347] Updated weights for policy 0, policy_version 72187 (0.0007) [2023-12-27 06:09:42,260][20347] Updated weights for policy 0, policy_version 72197 (0.0008) [2023-12-27 06:09:42,313][20347] Updated weights for policy 0, policy_version 72207 (0.0007) [2023-12-27 06:09:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 36937728. Throughput: 0: 10466.8, 1: 10429.1. Samples: 36933660. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:09:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:42,538][20381] Updated weights for policy 1, policy_version 72044 (0.0007) [2023-12-27 06:09:42,593][20381] Updated weights for policy 1, policy_version 72054 (0.0005) [2023-12-27 06:09:42,659][20381] Updated weights for policy 1, policy_version 72064 (0.0006) [2023-12-27 06:09:43,081][20347] Updated weights for policy 0, policy_version 72217 (0.0007) [2023-12-27 06:09:43,134][20347] Updated weights for policy 0, policy_version 72227 (0.0006) [2023-12-27 06:09:43,189][20381] Updated weights for policy 1, policy_version 72074 (0.0007) [2023-12-27 06:09:43,191][20347] Updated weights for policy 0, policy_version 72237 (0.0006) [2023-12-27 06:09:43,247][20381] Updated weights for policy 1, policy_version 72084 (0.0006) [2023-12-27 06:09:43,249][20347] Updated weights for policy 0, policy_version 72247 (0.0006) [2023-12-27 06:09:43,303][20381] Updated weights for policy 1, policy_version 72094 (0.0008) [2023-12-27 06:09:43,359][20381] Updated weights for policy 1, policy_version 72104 (0.0006) [2023-12-27 06:09:43,825][20347] Updated weights for policy 0, policy_version 72257 (0.0007) [2023-12-27 06:09:43,882][20347] Updated weights for policy 0, policy_version 72267 (0.0007) [2023-12-27 06:09:43,937][20347] Updated weights for policy 0, policy_version 72277 (0.0009) [2023-12-27 06:09:43,997][20381] Updated weights for policy 1, policy_version 72114 (0.0007) [2023-12-27 06:09:44,056][20381] Updated weights for policy 1, policy_version 72124 (0.0007) [2023-12-27 06:09:44,112][20381] Updated weights for policy 1, policy_version 72134 (0.0007) [2023-12-27 06:09:44,614][20347] Updated weights for policy 0, policy_version 72287 (0.0009) [2023-12-27 06:09:44,668][20347] Updated weights for policy 0, policy_version 72297 (0.0010) [2023-12-27 06:09:44,723][20347] Updated weights for policy 0, policy_version 72307 (0.0010) [2023-12-27 06:09:44,759][20381] Updated weights for policy 1, policy_version 72144 (0.0005) [2023-12-27 06:09:44,820][20381] Updated weights for policy 1, policy_version 72154 (0.0008) [2023-12-27 06:09:44,882][20381] Updated weights for policy 1, policy_version 72164 (0.0007) [2023-12-27 06:09:45,420][20347] Updated weights for policy 0, policy_version 72317 (0.0009) [2023-12-27 06:09:45,477][20347] Updated weights for policy 0, policy_version 72327 (0.0008) [2023-12-27 06:09:45,530][20347] Updated weights for policy 0, policy_version 72337 (0.0008) [2023-12-27 06:09:45,577][20381] Updated weights for policy 1, policy_version 72174 (0.0007) [2023-12-27 06:09:45,632][20381] Updated weights for policy 1, policy_version 72184 (0.0007) [2023-12-27 06:09:45,687][20381] Updated weights for policy 1, policy_version 72194 (0.0008) [2023-12-27 06:09:46,189][20347] Updated weights for policy 0, policy_version 72347 (0.0006) [2023-12-27 06:09:46,244][20347] Updated weights for policy 0, policy_version 72357 (0.0005) [2023-12-27 06:09:46,297][20347] Updated weights for policy 0, policy_version 72367 (0.0005) [2023-12-27 06:09:46,401][20381] Updated weights for policy 1, policy_version 72204 (0.0008) [2023-12-27 06:09:46,454][20381] Updated weights for policy 1, policy_version 72214 (0.0007) [2023-12-27 06:09:46,505][20381] Updated weights for policy 1, policy_version 72224 (0.0007) [2023-12-27 06:09:46,968][20347] Updated weights for policy 0, policy_version 72377 (0.0008) [2023-12-27 06:09:47,027][20347] Updated weights for policy 0, policy_version 72387 (0.0007) [2023-12-27 06:09:47,084][20347] Updated weights for policy 0, policy_version 72397 (0.0007) [2023-12-27 06:09:47,134][20347] Updated weights for policy 0, policy_version 72407 (0.0008) [2023-12-27 06:09:47,159][20381] Updated weights for policy 1, policy_version 72234 (0.0007) [2023-12-27 06:09:47,218][20381] Updated weights for policy 1, policy_version 72244 (0.0008) [2023-12-27 06:09:47,273][20381] Updated weights for policy 1, policy_version 72254 (0.0007) [2023-12-27 06:09:47,328][20381] Updated weights for policy 1, policy_version 72264 (0.0007) [2023-12-27 06:09:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 37052416. Throughput: 0: 10478.4, 1: 10447.1. Samples: 37057436. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:09:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:47,869][20347] Updated weights for policy 0, policy_version 72417 (0.0008) [2023-12-27 06:09:47,925][20381] Updated weights for policy 1, policy_version 72274 (0.0010) [2023-12-27 06:09:47,927][20347] Updated weights for policy 0, policy_version 72427 (0.0007) [2023-12-27 06:09:47,973][20381] Updated weights for policy 1, policy_version 72284 (0.0010) [2023-12-27 06:09:47,982][20347] Updated weights for policy 0, policy_version 72437 (0.0006) [2023-12-27 06:09:48,028][20381] Updated weights for policy 1, policy_version 72294 (0.0010) [2023-12-27 06:09:48,647][20347] Updated weights for policy 0, policy_version 72447 (0.0010) [2023-12-27 06:09:48,664][20381] Updated weights for policy 1, policy_version 72304 (0.0007) [2023-12-27 06:09:48,699][20347] Updated weights for policy 0, policy_version 72457 (0.0010) [2023-12-27 06:09:48,723][20381] Updated weights for policy 1, policy_version 72314 (0.0007) [2023-12-27 06:09:48,752][20347] Updated weights for policy 0, policy_version 72467 (0.0010) [2023-12-27 06:09:48,779][20381] Updated weights for policy 1, policy_version 72324 (0.0008) [2023-12-27 06:09:49,406][20347] Updated weights for policy 0, policy_version 72477 (0.0010) [2023-12-27 06:09:49,470][20347] Updated weights for policy 0, policy_version 72487 (0.0006) [2023-12-27 06:09:49,476][20381] Updated weights for policy 1, policy_version 72334 (0.0009) [2023-12-27 06:09:49,532][20381] Updated weights for policy 1, policy_version 72344 (0.0010) [2023-12-27 06:09:49,538][20347] Updated weights for policy 0, policy_version 72497 (0.0005) [2023-12-27 06:09:49,590][20381] Updated weights for policy 1, policy_version 72354 (0.0011) [2023-12-27 06:09:50,142][20347] Updated weights for policy 0, policy_version 72507 (0.0007) [2023-12-27 06:09:50,198][20347] Updated weights for policy 0, policy_version 72517 (0.0010) [2023-12-27 06:09:50,239][20381] Updated weights for policy 1, policy_version 72364 (0.0010) [2023-12-27 06:09:50,251][20347] Updated weights for policy 0, policy_version 72527 (0.0010) [2023-12-27 06:09:50,303][20381] Updated weights for policy 1, policy_version 72374 (0.0009) [2023-12-27 06:09:50,356][20381] Updated weights for policy 1, policy_version 72384 (0.0010) [2023-12-27 06:09:50,938][20347] Updated weights for policy 0, policy_version 72537 (0.0010) [2023-12-27 06:09:50,993][20347] Updated weights for policy 0, policy_version 72547 (0.0009) [2023-12-27 06:09:51,038][20381] Updated weights for policy 1, policy_version 72394 (0.0010) [2023-12-27 06:09:51,053][20347] Updated weights for policy 0, policy_version 72557 (0.0010) [2023-12-27 06:09:51,099][20381] Updated weights for policy 1, policy_version 72404 (0.0007) [2023-12-27 06:09:51,115][20347] Updated weights for policy 0, policy_version 72567 (0.0010) [2023-12-27 06:09:51,156][20381] Updated weights for policy 1, policy_version 72414 (0.0007) [2023-12-27 06:09:51,208][20381] Updated weights for policy 1, policy_version 72424 (0.0008) [2023-12-27 06:09:51,813][20347] Updated weights for policy 0, policy_version 72577 (0.0010) [2023-12-27 06:09:51,868][20347] Updated weights for policy 0, policy_version 72587 (0.0010) [2023-12-27 06:09:51,926][20347] Updated weights for policy 0, policy_version 72597 (0.0011) [2023-12-27 06:09:51,946][20381] Updated weights for policy 1, policy_version 72434 (0.0006) [2023-12-27 06:09:52,004][20381] Updated weights for policy 1, policy_version 72444 (0.0008) [2023-12-27 06:09:52,057][20381] Updated weights for policy 1, policy_version 72454 (0.0007) [2023-12-27 06:09:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 37150720. Throughput: 0: 10418.8, 1: 10454.0. Samples: 37120520. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:09:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000072456_18554880.pth... [2023-12-27 06:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000072600_18595840.pth... [2023-12-27 06:09:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000071240_18243584.pth [2023-12-27 06:09:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000071384_18284544.pth [2023-12-27 06:09:52,614][20347] Updated weights for policy 0, policy_version 72607 (0.0010) [2023-12-27 06:09:52,678][20347] Updated weights for policy 0, policy_version 72617 (0.0010) [2023-12-27 06:09:52,716][20381] Updated weights for policy 1, policy_version 72464 (0.0006) [2023-12-27 06:09:52,740][20347] Updated weights for policy 0, policy_version 72627 (0.0008) [2023-12-27 06:09:52,772][20381] Updated weights for policy 1, policy_version 72474 (0.0007) [2023-12-27 06:09:52,829][20381] Updated weights for policy 1, policy_version 72484 (0.0007) [2023-12-27 06:09:53,357][20347] Updated weights for policy 0, policy_version 72637 (0.0007) [2023-12-27 06:09:53,402][20347] Updated weights for policy 0, policy_version 72647 (0.0007) [2023-12-27 06:09:53,451][20347] Updated weights for policy 0, policy_version 72657 (0.0007) [2023-12-27 06:09:53,509][20381] Updated weights for policy 1, policy_version 72494 (0.0007) [2023-12-27 06:09:53,558][20381] Updated weights for policy 1, policy_version 72504 (0.0007) [2023-12-27 06:09:53,614][20381] Updated weights for policy 1, policy_version 72514 (0.0008) [2023-12-27 06:09:54,158][20347] Updated weights for policy 0, policy_version 72667 (0.0007) [2023-12-27 06:09:54,210][20347] Updated weights for policy 0, policy_version 72677 (0.0008) [2023-12-27 06:09:54,265][20347] Updated weights for policy 0, policy_version 72687 (0.0007) [2023-12-27 06:09:54,293][20381] Updated weights for policy 1, policy_version 72524 (0.0008) [2023-12-27 06:09:54,344][20381] Updated weights for policy 1, policy_version 72534 (0.0008) [2023-12-27 06:09:54,403][20381] Updated weights for policy 1, policy_version 72544 (0.0010) [2023-12-27 06:09:54,914][20347] Updated weights for policy 0, policy_version 72697 (0.0008) [2023-12-27 06:09:54,970][20347] Updated weights for policy 0, policy_version 72707 (0.0010) [2023-12-27 06:09:55,033][20347] Updated weights for policy 0, policy_version 72717 (0.0011) [2023-12-27 06:09:55,046][20381] Updated weights for policy 1, policy_version 72554 (0.0010) [2023-12-27 06:09:55,104][20381] Updated weights for policy 1, policy_version 72564 (0.0008) [2023-12-27 06:09:55,105][20347] Updated weights for policy 0, policy_version 72727 (0.0010) [2023-12-27 06:09:55,156][20381] Updated weights for policy 1, policy_version 72574 (0.0006) [2023-12-27 06:09:55,212][20381] Updated weights for policy 1, policy_version 72584 (0.0005) [2023-12-27 06:09:55,707][20347] Updated weights for policy 0, policy_version 72737 (0.0006) [2023-12-27 06:09:55,762][20347] Updated weights for policy 0, policy_version 72747 (0.0005) [2023-12-27 06:09:55,818][20381] Updated weights for policy 1, policy_version 72594 (0.0009) [2023-12-27 06:09:55,828][20347] Updated weights for policy 0, policy_version 72757 (0.0005) [2023-12-27 06:09:55,867][20381] Updated weights for policy 1, policy_version 72604 (0.0010) [2023-12-27 06:09:55,924][20381] Updated weights for policy 1, policy_version 72614 (0.0010) [2023-12-27 06:09:56,456][20347] Updated weights for policy 0, policy_version 72767 (0.0006) [2023-12-27 06:09:56,506][20347] Updated weights for policy 0, policy_version 72777 (0.0005) [2023-12-27 06:09:56,559][20347] Updated weights for policy 0, policy_version 72787 (0.0009) [2023-12-27 06:09:56,568][20381] Updated weights for policy 1, policy_version 72624 (0.0010) [2023-12-27 06:09:56,613][20381] Updated weights for policy 1, policy_version 72634 (0.0010) [2023-12-27 06:09:56,666][20381] Updated weights for policy 1, policy_version 72644 (0.0010) [2023-12-27 06:09:57,177][20347] Updated weights for policy 0, policy_version 72797 (0.0008) [2023-12-27 06:09:57,226][20347] Updated weights for policy 0, policy_version 72807 (0.0005) [2023-12-27 06:09:57,280][20347] Updated weights for policy 0, policy_version 72817 (0.0006) [2023-12-27 06:09:57,354][20381] Updated weights for policy 1, policy_version 72654 (0.0010) [2023-12-27 06:09:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 37257216. Throughput: 0: 10416.1, 1: 10498.4. Samples: 37244892. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:09:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:09:57,401][20381] Updated weights for policy 1, policy_version 72664 (0.0007) [2023-12-27 06:09:57,451][20381] Updated weights for policy 1, policy_version 72674 (0.0007) [2023-12-27 06:09:57,878][20347] Updated weights for policy 0, policy_version 72827 (0.0008) [2023-12-27 06:09:57,940][20347] Updated weights for policy 0, policy_version 72837 (0.0010) [2023-12-27 06:09:57,987][20347] Updated weights for policy 0, policy_version 72847 (0.0010) [2023-12-27 06:09:58,160][20381] Updated weights for policy 1, policy_version 72684 (0.0008) [2023-12-27 06:09:58,217][20381] Updated weights for policy 1, policy_version 72694 (0.0008) [2023-12-27 06:09:58,275][20381] Updated weights for policy 1, policy_version 72704 (0.0007) [2023-12-27 06:09:58,720][20347] Updated weights for policy 0, policy_version 72857 (0.0010) [2023-12-27 06:09:58,789][20347] Updated weights for policy 0, policy_version 72867 (0.0012) [2023-12-27 06:09:58,856][20347] Updated weights for policy 0, policy_version 72877 (0.0007) [2023-12-27 06:09:58,929][20347] Updated weights for policy 0, policy_version 72887 (0.0008) [2023-12-27 06:09:59,058][20381] Updated weights for policy 1, policy_version 72714 (0.0008) [2023-12-27 06:09:59,109][20381] Updated weights for policy 1, policy_version 72724 (0.0007) [2023-12-27 06:09:59,156][20381] Updated weights for policy 1, policy_version 72734 (0.0007) [2023-12-27 06:09:59,210][20381] Updated weights for policy 1, policy_version 72744 (0.0007) [2023-12-27 06:09:59,619][20347] Updated weights for policy 0, policy_version 72897 (0.0008) [2023-12-27 06:09:59,666][20347] Updated weights for policy 0, policy_version 72907 (0.0007) [2023-12-27 06:09:59,719][20347] Updated weights for policy 0, policy_version 72917 (0.0007) [2023-12-27 06:09:59,966][20381] Updated weights for policy 1, policy_version 72754 (0.0008) [2023-12-27 06:10:00,026][20381] Updated weights for policy 1, policy_version 72764 (0.0006) [2023-12-27 06:10:00,085][20381] Updated weights for policy 1, policy_version 72774 (0.0007) [2023-12-27 06:10:00,418][20347] Updated weights for policy 0, policy_version 72927 (0.0006) [2023-12-27 06:10:00,481][20347] Updated weights for policy 0, policy_version 72937 (0.0006) [2023-12-27 06:10:00,537][20347] Updated weights for policy 0, policy_version 72947 (0.0005) [2023-12-27 06:10:00,693][20381] Updated weights for policy 1, policy_version 72784 (0.0005) [2023-12-27 06:10:00,740][20381] Updated weights for policy 1, policy_version 72794 (0.0005) [2023-12-27 06:10:00,793][20381] Updated weights for policy 1, policy_version 72804 (0.0005) [2023-12-27 06:10:01,146][20347] Updated weights for policy 0, policy_version 72957 (0.0006) [2023-12-27 06:10:01,206][20347] Updated weights for policy 0, policy_version 72967 (0.0006) [2023-12-27 06:10:01,258][20347] Updated weights for policy 0, policy_version 72977 (0.0007) [2023-12-27 06:10:01,433][20381] Updated weights for policy 1, policy_version 72814 (0.0008) [2023-12-27 06:10:01,488][20381] Updated weights for policy 1, policy_version 72824 (0.0005) [2023-12-27 06:10:01,551][20381] Updated weights for policy 1, policy_version 72834 (0.0007) [2023-12-27 06:10:01,896][20347] Updated weights for policy 0, policy_version 72987 (0.0005) [2023-12-27 06:10:01,953][20347] Updated weights for policy 0, policy_version 72997 (0.0010) [2023-12-27 06:10:02,003][20347] Updated weights for policy 0, policy_version 73007 (0.0010) [2023-12-27 06:10:02,231][20381] Updated weights for policy 1, policy_version 72844 (0.0009) [2023-12-27 06:10:02,284][20381] Updated weights for policy 1, policy_version 72854 (0.0008) [2023-12-27 06:10:02,335][20381] Updated weights for policy 1, policy_version 72864 (0.0005) [2023-12-27 06:10:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 37355520. Throughput: 0: 10417.7, 1: 10429.1. Samples: 37368976. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:10:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:02,655][20347] Updated weights for policy 0, policy_version 73017 (0.0009) [2023-12-27 06:10:02,707][20347] Updated weights for policy 0, policy_version 73027 (0.0007) [2023-12-27 06:10:02,762][20347] Updated weights for policy 0, policy_version 73037 (0.0007) [2023-12-27 06:10:02,818][20347] Updated weights for policy 0, policy_version 73047 (0.0008) [2023-12-27 06:10:02,986][20381] Updated weights for policy 1, policy_version 72874 (0.0008) [2023-12-27 06:10:03,032][20381] Updated weights for policy 1, policy_version 72884 (0.0009) [2023-12-27 06:10:03,086][20381] Updated weights for policy 1, policy_version 72894 (0.0010) [2023-12-27 06:10:03,143][20381] Updated weights for policy 1, policy_version 72904 (0.0010) [2023-12-27 06:10:03,473][20347] Updated weights for policy 0, policy_version 73057 (0.0005) [2023-12-27 06:10:03,519][20347] Updated weights for policy 0, policy_version 73067 (0.0006) [2023-12-27 06:10:03,569][20347] Updated weights for policy 0, policy_version 73077 (0.0007) [2023-12-27 06:10:03,818][20381] Updated weights for policy 1, policy_version 72914 (0.0010) [2023-12-27 06:10:03,876][20381] Updated weights for policy 1, policy_version 72924 (0.0009) [2023-12-27 06:10:03,932][20381] Updated weights for policy 1, policy_version 72934 (0.0008) [2023-12-27 06:10:04,201][20347] Updated weights for policy 0, policy_version 73087 (0.0008) [2023-12-27 06:10:04,256][20347] Updated weights for policy 0, policy_version 73097 (0.0006) [2023-12-27 06:10:04,305][20347] Updated weights for policy 0, policy_version 73107 (0.0008) [2023-12-27 06:10:04,622][20381] Updated weights for policy 1, policy_version 72944 (0.0007) [2023-12-27 06:10:04,680][20381] Updated weights for policy 1, policy_version 72954 (0.0006) [2023-12-27 06:10:04,739][20381] Updated weights for policy 1, policy_version 72964 (0.0005) [2023-12-27 06:10:04,904][20347] Updated weights for policy 0, policy_version 73117 (0.0006) [2023-12-27 06:10:04,954][20347] Updated weights for policy 0, policy_version 73127 (0.0008) [2023-12-27 06:10:05,006][20347] Updated weights for policy 0, policy_version 73137 (0.0007) [2023-12-27 06:10:05,296][20381] Updated weights for policy 1, policy_version 72974 (0.0006) [2023-12-27 06:10:05,347][20381] Updated weights for policy 1, policy_version 72984 (0.0005) [2023-12-27 06:10:05,408][20381] Updated weights for policy 1, policy_version 72994 (0.0006) [2023-12-27 06:10:05,642][20347] Updated weights for policy 0, policy_version 73147 (0.0007) [2023-12-27 06:10:05,691][20347] Updated weights for policy 0, policy_version 73157 (0.0009) [2023-12-27 06:10:05,744][20347] Updated weights for policy 0, policy_version 73167 (0.0006) [2023-12-27 06:10:05,980][20381] Updated weights for policy 1, policy_version 73004 (0.0007) [2023-12-27 06:10:06,032][20381] Updated weights for policy 1, policy_version 73014 (0.0009) [2023-12-27 06:10:06,081][20381] Updated weights for policy 1, policy_version 73024 (0.0008) [2023-12-27 06:10:06,376][20347] Updated weights for policy 0, policy_version 73177 (0.0006) [2023-12-27 06:10:06,422][20347] Updated weights for policy 0, policy_version 73187 (0.0010) [2023-12-27 06:10:06,468][20347] Updated weights for policy 0, policy_version 73197 (0.0010) [2023-12-27 06:10:06,518][20347] Updated weights for policy 0, policy_version 73207 (0.0010) [2023-12-27 06:10:06,801][20381] Updated weights for policy 1, policy_version 73034 (0.0008) [2023-12-27 06:10:06,859][20381] Updated weights for policy 1, policy_version 73044 (0.0010) [2023-12-27 06:10:06,917][20381] Updated weights for policy 1, policy_version 73054 (0.0010) [2023-12-27 06:10:06,976][20381] Updated weights for policy 1, policy_version 73064 (0.0006) [2023-12-27 06:10:07,157][20347] Updated weights for policy 0, policy_version 73217 (0.0010) [2023-12-27 06:10:07,212][20347] Updated weights for policy 0, policy_version 73227 (0.0010) [2023-12-27 06:10:07,260][20347] Updated weights for policy 0, policy_version 73237 (0.0009) [2023-12-27 06:10:07,363][19170] Fps is (10 sec: 21298.0, 60 sec: 20889.5, 300 sec: 20771.5). Total num frames: 37470208. Throughput: 0: 10440.2, 1: 10428.6. Samples: 37433968. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:10:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000073064_18710528.pth... [2023-12-27 06:10:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000073240_18759680.pth... [2023-12-27 06:10:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000071848_18399232.pth [2023-12-27 06:10:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000072024_18448384.pth [2023-12-27 06:10:07,612][20381] Updated weights for policy 1, policy_version 73074 (0.0008) [2023-12-27 06:10:07,665][20381] Updated weights for policy 1, policy_version 73084 (0.0010) [2023-12-27 06:10:07,726][20381] Updated weights for policy 1, policy_version 73094 (0.0009) [2023-12-27 06:10:07,861][20347] Updated weights for policy 0, policy_version 73247 (0.0006) [2023-12-27 06:10:07,909][20347] Updated weights for policy 0, policy_version 73257 (0.0007) [2023-12-27 06:10:07,963][20347] Updated weights for policy 0, policy_version 73267 (0.0010) [2023-12-27 06:10:08,371][20381] Updated weights for policy 1, policy_version 73104 (0.0009) [2023-12-27 06:10:08,427][20381] Updated weights for policy 1, policy_version 73114 (0.0011) [2023-12-27 06:10:08,485][20381] Updated weights for policy 1, policy_version 73124 (0.0009) [2023-12-27 06:10:08,637][20347] Updated weights for policy 0, policy_version 73277 (0.0010) [2023-12-27 06:10:08,698][20347] Updated weights for policy 0, policy_version 73287 (0.0010) [2023-12-27 06:10:08,759][20347] Updated weights for policy 0, policy_version 73297 (0.0010) [2023-12-27 06:10:09,175][20381] Updated weights for policy 1, policy_version 73134 (0.0009) [2023-12-27 06:10:09,240][20381] Updated weights for policy 1, policy_version 73144 (0.0008) [2023-12-27 06:10:09,303][20381] Updated weights for policy 1, policy_version 73154 (0.0006) [2023-12-27 06:10:09,504][20347] Updated weights for policy 0, policy_version 73307 (0.0010) [2023-12-27 06:10:09,561][20347] Updated weights for policy 0, policy_version 73317 (0.0008) [2023-12-27 06:10:09,625][20347] Updated weights for policy 0, policy_version 73327 (0.0008) [2023-12-27 06:10:09,968][20381] Updated weights for policy 1, policy_version 73164 (0.0008) [2023-12-27 06:10:10,034][20381] Updated weights for policy 1, policy_version 73174 (0.0007) [2023-12-27 06:10:10,102][20381] Updated weights for policy 1, policy_version 73184 (0.0006) [2023-12-27 06:10:10,301][20347] Updated weights for policy 0, policy_version 73337 (0.0008) [2023-12-27 06:10:10,361][20347] Updated weights for policy 0, policy_version 73347 (0.0007) [2023-12-27 06:10:10,422][20347] Updated weights for policy 0, policy_version 73357 (0.0007) [2023-12-27 06:10:10,483][20347] Updated weights for policy 0, policy_version 73367 (0.0006) [2023-12-27 06:10:10,769][20381] Updated weights for policy 1, policy_version 73194 (0.0007) [2023-12-27 06:10:10,818][20381] Updated weights for policy 1, policy_version 73204 (0.0009) [2023-12-27 06:10:10,878][20381] Updated weights for policy 1, policy_version 73214 (0.0008) [2023-12-27 06:10:10,927][20381] Updated weights for policy 1, policy_version 73224 (0.0008) [2023-12-27 06:10:11,166][20347] Updated weights for policy 0, policy_version 73377 (0.0008) [2023-12-27 06:10:11,227][20347] Updated weights for policy 0, policy_version 73387 (0.0009) [2023-12-27 06:10:11,283][20347] Updated weights for policy 0, policy_version 73397 (0.0007) [2023-12-27 06:10:11,633][20381] Updated weights for policy 1, policy_version 73234 (0.0008) [2023-12-27 06:10:11,698][20381] Updated weights for policy 1, policy_version 73244 (0.0009) [2023-12-27 06:10:11,765][20381] Updated weights for policy 1, policy_version 73254 (0.0008) [2023-12-27 06:10:11,985][20347] Updated weights for policy 0, policy_version 73407 (0.0006) [2023-12-27 06:10:12,041][20347] Updated weights for policy 0, policy_version 73417 (0.0007) [2023-12-27 06:10:12,107][20347] Updated weights for policy 0, policy_version 73427 (0.0007) [2023-12-27 06:10:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 37568512. Throughput: 0: 10467.4, 1: 10419.9. Samples: 37558956. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:10:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:12,438][20381] Updated weights for policy 1, policy_version 73264 (0.0008) [2023-12-27 06:10:12,513][20381] Updated weights for policy 1, policy_version 73274 (0.0007) [2023-12-27 06:10:12,567][20381] Updated weights for policy 1, policy_version 73284 (0.0008) [2023-12-27 06:10:12,785][20347] Updated weights for policy 0, policy_version 73437 (0.0007) [2023-12-27 06:10:12,841][20347] Updated weights for policy 0, policy_version 73447 (0.0007) [2023-12-27 06:10:12,889][20347] Updated weights for policy 0, policy_version 73457 (0.0008) [2023-12-27 06:10:13,259][20381] Updated weights for policy 1, policy_version 73294 (0.0007) [2023-12-27 06:10:13,317][20381] Updated weights for policy 1, policy_version 73304 (0.0006) [2023-12-27 06:10:13,380][20381] Updated weights for policy 1, policy_version 73314 (0.0006) [2023-12-27 06:10:13,553][20347] Updated weights for policy 0, policy_version 73467 (0.0008) [2023-12-27 06:10:13,612][20347] Updated weights for policy 0, policy_version 73477 (0.0008) [2023-12-27 06:10:13,671][20347] Updated weights for policy 0, policy_version 73487 (0.0007) [2023-12-27 06:10:13,928][20381] Updated weights for policy 1, policy_version 73324 (0.0009) [2023-12-27 06:10:13,976][20381] Updated weights for policy 1, policy_version 73334 (0.0006) [2023-12-27 06:10:14,026][20381] Updated weights for policy 1, policy_version 73344 (0.0005) [2023-12-27 06:10:14,410][20347] Updated weights for policy 0, policy_version 73497 (0.0007) [2023-12-27 06:10:14,464][20347] Updated weights for policy 0, policy_version 73507 (0.0005) [2023-12-27 06:10:14,528][20347] Updated weights for policy 0, policy_version 73517 (0.0008) [2023-12-27 06:10:14,565][20381] Updated weights for policy 1, policy_version 73354 (0.0006) [2023-12-27 06:10:14,583][20347] Updated weights for policy 0, policy_version 73527 (0.0010) [2023-12-27 06:10:14,628][20381] Updated weights for policy 1, policy_version 73364 (0.0011) [2023-12-27 06:10:14,687][20381] Updated weights for policy 1, policy_version 73374 (0.0011) [2023-12-27 06:10:14,746][20381] Updated weights for policy 1, policy_version 73384 (0.0011) [2023-12-27 06:10:15,336][20347] Updated weights for policy 0, policy_version 73537 (0.0007) [2023-12-27 06:10:15,382][20381] Updated weights for policy 1, policy_version 73394 (0.0010) [2023-12-27 06:10:15,397][20347] Updated weights for policy 0, policy_version 73547 (0.0006) [2023-12-27 06:10:15,437][20381] Updated weights for policy 1, policy_version 73404 (0.0005) [2023-12-27 06:10:15,454][20347] Updated weights for policy 0, policy_version 73557 (0.0008) [2023-12-27 06:10:15,494][20381] Updated weights for policy 1, policy_version 73414 (0.0009) [2023-12-27 06:10:16,087][20347] Updated weights for policy 0, policy_version 73567 (0.0009) [2023-12-27 06:10:16,118][20381] Updated weights for policy 1, policy_version 73424 (0.0006) [2023-12-27 06:10:16,147][20347] Updated weights for policy 0, policy_version 73577 (0.0007) [2023-12-27 06:10:16,174][20381] Updated weights for policy 1, policy_version 73434 (0.0006) [2023-12-27 06:10:16,202][20347] Updated weights for policy 0, policy_version 73587 (0.0009) [2023-12-27 06:10:16,223][20381] Updated weights for policy 1, policy_version 73444 (0.0005) [2023-12-27 06:10:16,772][20347] Updated weights for policy 0, policy_version 73597 (0.0010) [2023-12-27 06:10:16,821][20347] Updated weights for policy 0, policy_version 73607 (0.0005) [2023-12-27 06:10:16,829][20381] Updated weights for policy 1, policy_version 73454 (0.0009) [2023-12-27 06:10:16,875][20347] Updated weights for policy 0, policy_version 73617 (0.0009) [2023-12-27 06:10:16,881][20381] Updated weights for policy 1, policy_version 73464 (0.0010) [2023-12-27 06:10:16,929][20381] Updated weights for policy 1, policy_version 73474 (0.0010) [2023-12-27 06:10:17,362][19170] Fps is (10 sec: 20481.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 37675008. Throughput: 0: 10361.8, 1: 10495.6. Samples: 37685524. Policy #0 lag: (min: 19.0, avg: 27.3, max: 51.0) [2023-12-27 06:10:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:17,467][20347] Updated weights for policy 0, policy_version 73627 (0.0010) [2023-12-27 06:10:17,528][20347] Updated weights for policy 0, policy_version 73637 (0.0010) [2023-12-27 06:10:17,578][20381] Updated weights for policy 1, policy_version 73484 (0.0010) [2023-12-27 06:10:17,586][20347] Updated weights for policy 0, policy_version 73647 (0.0010) [2023-12-27 06:10:17,625][20381] Updated weights for policy 1, policy_version 73494 (0.0010) [2023-12-27 06:10:17,677][20381] Updated weights for policy 1, policy_version 73504 (0.0010) [2023-12-27 06:10:18,190][20347] Updated weights for policy 0, policy_version 73657 (0.0008) [2023-12-27 06:10:18,246][20347] Updated weights for policy 0, policy_version 73667 (0.0005) [2023-12-27 06:10:18,297][20347] Updated weights for policy 0, policy_version 73677 (0.0005) [2023-12-27 06:10:18,345][20381] Updated weights for policy 1, policy_version 73514 (0.0010) [2023-12-27 06:10:18,364][20347] Updated weights for policy 0, policy_version 73687 (0.0007) [2023-12-27 06:10:18,397][20381] Updated weights for policy 1, policy_version 73524 (0.0006) [2023-12-27 06:10:18,451][20381] Updated weights for policy 1, policy_version 73534 (0.0011) [2023-12-27 06:10:18,505][20381] Updated weights for policy 1, policy_version 73544 (0.0008) [2023-12-27 06:10:19,027][20347] Updated weights for policy 0, policy_version 73697 (0.0007) [2023-12-27 06:10:19,072][20381] Updated weights for policy 1, policy_version 73554 (0.0007) [2023-12-27 06:10:19,078][20347] Updated weights for policy 0, policy_version 73707 (0.0006) [2023-12-27 06:10:19,125][20381] Updated weights for policy 1, policy_version 73564 (0.0011) [2023-12-27 06:10:19,130][20347] Updated weights for policy 0, policy_version 73717 (0.0005) [2023-12-27 06:10:19,181][20381] Updated weights for policy 1, policy_version 73574 (0.0011) [2023-12-27 06:10:19,849][20347] Updated weights for policy 0, policy_version 73727 (0.0008) [2023-12-27 06:10:19,860][20381] Updated weights for policy 1, policy_version 73584 (0.0011) [2023-12-27 06:10:19,906][20347] Updated weights for policy 0, policy_version 73737 (0.0007) [2023-12-27 06:10:19,911][20381] Updated weights for policy 1, policy_version 73594 (0.0011) [2023-12-27 06:10:19,975][20381] Updated weights for policy 1, policy_version 73604 (0.0010) [2023-12-27 06:10:19,979][20347] Updated weights for policy 0, policy_version 73747 (0.0010) [2023-12-27 06:10:20,639][20347] Updated weights for policy 0, policy_version 73757 (0.0010) [2023-12-27 06:10:20,699][20381] Updated weights for policy 1, policy_version 73614 (0.0008) [2023-12-27 06:10:20,702][20347] Updated weights for policy 0, policy_version 73767 (0.0010) [2023-12-27 06:10:20,758][20347] Updated weights for policy 0, policy_version 73777 (0.0010) [2023-12-27 06:10:20,758][20381] Updated weights for policy 1, policy_version 73624 (0.0006) [2023-12-27 06:10:20,818][20381] Updated weights for policy 1, policy_version 73634 (0.0010) [2023-12-27 06:10:21,490][20347] Updated weights for policy 0, policy_version 73787 (0.0010) [2023-12-27 06:10:21,541][20347] Updated weights for policy 0, policy_version 73797 (0.0008) [2023-12-27 06:10:21,558][20381] Updated weights for policy 1, policy_version 73644 (0.0009) [2023-12-27 06:10:21,600][20347] Updated weights for policy 0, policy_version 73807 (0.0008) [2023-12-27 06:10:21,611][20381] Updated weights for policy 1, policy_version 73654 (0.0008) [2023-12-27 06:10:21,675][20381] Updated weights for policy 1, policy_version 73664 (0.0008) [2023-12-27 06:10:22,343][20347] Updated weights for policy 0, policy_version 73817 (0.0007) [2023-12-27 06:10:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 37773312. Throughput: 0: 10361.0, 1: 10486.0. Samples: 37749296. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000073672_18866176.pth... [2023-12-27 06:10:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000072456_18554880.pth [2023-12-27 06:10:22,402][20381] Updated weights for policy 1, policy_version 73674 (0.0008) [2023-12-27 06:10:22,410][20347] Updated weights for policy 0, policy_version 73827 (0.0007) [2023-12-27 06:10:22,460][20381] Updated weights for policy 1, policy_version 73684 (0.0008) [2023-12-27 06:10:22,461][20347] Updated weights for policy 0, policy_version 73837 (0.0006) [2023-12-27 06:10:22,514][20347] Updated weights for policy 0, policy_version 73847 (0.0005) [2023-12-27 06:10:22,517][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000073848_18915328.pth... [2023-12-27 06:10:22,519][20381] Updated weights for policy 1, policy_version 73694 (0.0007) [2023-12-27 06:10:22,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000072600_18595840.pth [2023-12-27 06:10:22,573][20381] Updated weights for policy 1, policy_version 73704 (0.0009) [2023-12-27 06:10:23,138][20347] Updated weights for policy 0, policy_version 73857 (0.0006) [2023-12-27 06:10:23,190][20347] Updated weights for policy 0, policy_version 73867 (0.0007) [2023-12-27 06:10:23,199][20381] Updated weights for policy 1, policy_version 73714 (0.0011) [2023-12-27 06:10:23,243][20347] Updated weights for policy 0, policy_version 73877 (0.0007) [2023-12-27 06:10:23,262][20381] Updated weights for policy 1, policy_version 73724 (0.0009) [2023-12-27 06:10:23,309][20381] Updated weights for policy 1, policy_version 73734 (0.0005) [2023-12-27 06:10:23,841][20381] Updated weights for policy 1, policy_version 73744 (0.0005) [2023-12-27 06:10:23,898][20381] Updated weights for policy 1, policy_version 73754 (0.0010) [2023-12-27 06:10:23,948][20381] Updated weights for policy 1, policy_version 73764 (0.0011) [2023-12-27 06:10:23,962][20347] Updated weights for policy 0, policy_version 73887 (0.0008) [2023-12-27 06:10:24,013][20347] Updated weights for policy 0, policy_version 73897 (0.0007) [2023-12-27 06:10:24,060][20347] Updated weights for policy 0, policy_version 73907 (0.0007) [2023-12-27 06:10:24,557][20381] Updated weights for policy 1, policy_version 73774 (0.0007) [2023-12-27 06:10:24,603][20381] Updated weights for policy 1, policy_version 73784 (0.0008) [2023-12-27 06:10:24,655][20381] Updated weights for policy 1, policy_version 73794 (0.0010) [2023-12-27 06:10:24,793][20347] Updated weights for policy 0, policy_version 73917 (0.0007) [2023-12-27 06:10:24,860][20347] Updated weights for policy 0, policy_version 73927 (0.0005) [2023-12-27 06:10:24,912][20347] Updated weights for policy 0, policy_version 73937 (0.0007) [2023-12-27 06:10:25,333][20381] Updated weights for policy 1, policy_version 73804 (0.0011) [2023-12-27 06:10:25,392][20381] Updated weights for policy 1, policy_version 73814 (0.0011) [2023-12-27 06:10:25,441][20381] Updated weights for policy 1, policy_version 73824 (0.0010) [2023-12-27 06:10:25,569][20347] Updated weights for policy 0, policy_version 73947 (0.0007) [2023-12-27 06:10:25,624][20347] Updated weights for policy 0, policy_version 73957 (0.0007) [2023-12-27 06:10:25,678][20347] Updated weights for policy 0, policy_version 73967 (0.0007) [2023-12-27 06:10:26,117][20381] Updated weights for policy 1, policy_version 73834 (0.0010) [2023-12-27 06:10:26,166][20381] Updated weights for policy 1, policy_version 73844 (0.0010) [2023-12-27 06:10:26,217][20381] Updated weights for policy 1, policy_version 73854 (0.0007) [2023-12-27 06:10:26,269][20381] Updated weights for policy 1, policy_version 73864 (0.0006) [2023-12-27 06:10:26,330][20347] Updated weights for policy 0, policy_version 73977 (0.0007) [2023-12-27 06:10:26,386][20347] Updated weights for policy 0, policy_version 73987 (0.0007) [2023-12-27 06:10:26,439][20347] Updated weights for policy 0, policy_version 73997 (0.0007) [2023-12-27 06:10:26,495][20347] Updated weights for policy 0, policy_version 74007 (0.0007) [2023-12-27 06:10:26,919][20381] Updated weights for policy 1, policy_version 73874 (0.0005) [2023-12-27 06:10:26,971][20381] Updated weights for policy 1, policy_version 73884 (0.0007) [2023-12-27 06:10:27,023][20381] Updated weights for policy 1, policy_version 73894 (0.0008) [2023-12-27 06:10:27,160][20347] Updated weights for policy 0, policy_version 74017 (0.0008) [2023-12-27 06:10:27,218][20347] Updated weights for policy 0, policy_version 74027 (0.0008) [2023-12-27 06:10:27,275][20347] Updated weights for policy 0, policy_version 74037 (0.0008) [2023-12-27 06:10:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 37888000. Throughput: 0: 10372.0, 1: 10499.6. Samples: 37872884. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:27,614][20381] Updated weights for policy 1, policy_version 73904 (0.0006) [2023-12-27 06:10:27,669][20381] Updated weights for policy 1, policy_version 73914 (0.0007) [2023-12-27 06:10:27,733][20381] Updated weights for policy 1, policy_version 73924 (0.0007) [2023-12-27 06:10:27,988][20347] Updated weights for policy 0, policy_version 74047 (0.0007) [2023-12-27 06:10:28,042][20347] Updated weights for policy 0, policy_version 74057 (0.0007) [2023-12-27 06:10:28,103][20347] Updated weights for policy 0, policy_version 74067 (0.0008) [2023-12-27 06:10:28,337][20381] Updated weights for policy 1, policy_version 73934 (0.0008) [2023-12-27 06:10:28,397][20381] Updated weights for policy 1, policy_version 73944 (0.0010) [2023-12-27 06:10:28,454][20381] Updated weights for policy 1, policy_version 73954 (0.0007) [2023-12-27 06:10:28,736][20347] Updated weights for policy 0, policy_version 74077 (0.0007) [2023-12-27 06:10:28,789][20347] Updated weights for policy 0, policy_version 74087 (0.0007) [2023-12-27 06:10:28,840][20347] Updated weights for policy 0, policy_version 74097 (0.0007) [2023-12-27 06:10:29,133][20381] Updated weights for policy 1, policy_version 73964 (0.0008) [2023-12-27 06:10:29,186][20381] Updated weights for policy 1, policy_version 73974 (0.0008) [2023-12-27 06:10:29,245][20381] Updated weights for policy 1, policy_version 73984 (0.0009) [2023-12-27 06:10:29,498][20347] Updated weights for policy 0, policy_version 74107 (0.0007) [2023-12-27 06:10:29,545][20347] Updated weights for policy 0, policy_version 74117 (0.0005) [2023-12-27 06:10:29,594][20347] Updated weights for policy 0, policy_version 74127 (0.0008) [2023-12-27 06:10:29,965][20381] Updated weights for policy 1, policy_version 73994 (0.0008) [2023-12-27 06:10:30,023][20381] Updated weights for policy 1, policy_version 74004 (0.0008) [2023-12-27 06:10:30,077][20381] Updated weights for policy 1, policy_version 74014 (0.0008) [2023-12-27 06:10:30,127][20381] Updated weights for policy 1, policy_version 74024 (0.0008) [2023-12-27 06:10:30,252][20347] Updated weights for policy 0, policy_version 74137 (0.0007) [2023-12-27 06:10:30,304][20347] Updated weights for policy 0, policy_version 74147 (0.0007) [2023-12-27 06:10:30,366][20347] Updated weights for policy 0, policy_version 74157 (0.0007) [2023-12-27 06:10:30,420][20347] Updated weights for policy 0, policy_version 74167 (0.0008) [2023-12-27 06:10:30,815][20381] Updated weights for policy 1, policy_version 74034 (0.0005) [2023-12-27 06:10:30,875][20381] Updated weights for policy 1, policy_version 74044 (0.0005) [2023-12-27 06:10:30,927][20381] Updated weights for policy 1, policy_version 74054 (0.0005) [2023-12-27 06:10:31,113][20347] Updated weights for policy 0, policy_version 74177 (0.0008) [2023-12-27 06:10:31,176][20347] Updated weights for policy 0, policy_version 74187 (0.0008) [2023-12-27 06:10:31,227][20347] Updated weights for policy 0, policy_version 74197 (0.0008) [2023-12-27 06:10:31,515][20381] Updated weights for policy 1, policy_version 74064 (0.0006) [2023-12-27 06:10:31,568][20381] Updated weights for policy 1, policy_version 74074 (0.0005) [2023-12-27 06:10:31,631][20381] Updated weights for policy 1, policy_version 74084 (0.0007) [2023-12-27 06:10:31,972][20347] Updated weights for policy 0, policy_version 74207 (0.0008) [2023-12-27 06:10:32,025][20347] Updated weights for policy 0, policy_version 74217 (0.0009) [2023-12-27 06:10:32,074][20347] Updated weights for policy 0, policy_version 74227 (0.0008) [2023-12-27 06:10:32,173][20381] Updated weights for policy 1, policy_version 74094 (0.0007) [2023-12-27 06:10:32,232][20381] Updated weights for policy 1, policy_version 74104 (0.0005) [2023-12-27 06:10:32,288][20381] Updated weights for policy 1, policy_version 74114 (0.0007) [2023-12-27 06:10:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 37994496. Throughput: 0: 10374.9, 1: 10559.5. Samples: 37999488. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:32,718][20347] Updated weights for policy 0, policy_version 74237 (0.0007) [2023-12-27 06:10:32,766][20347] Updated weights for policy 0, policy_version 74247 (0.0007) [2023-12-27 06:10:32,811][20347] Updated weights for policy 0, policy_version 74257 (0.0008) [2023-12-27 06:10:32,924][20381] Updated weights for policy 1, policy_version 74124 (0.0008) [2023-12-27 06:10:32,975][20381] Updated weights for policy 1, policy_version 74134 (0.0007) [2023-12-27 06:10:33,022][20381] Updated weights for policy 1, policy_version 74144 (0.0007) [2023-12-27 06:10:33,447][20347] Updated weights for policy 0, policy_version 74267 (0.0006) [2023-12-27 06:10:33,494][20347] Updated weights for policy 0, policy_version 74277 (0.0005) [2023-12-27 06:10:33,546][20347] Updated weights for policy 0, policy_version 74287 (0.0006) [2023-12-27 06:10:33,717][20381] Updated weights for policy 1, policy_version 74154 (0.0007) [2023-12-27 06:10:33,763][20381] Updated weights for policy 1, policy_version 74164 (0.0007) [2023-12-27 06:10:33,810][20381] Updated weights for policy 1, policy_version 74174 (0.0007) [2023-12-27 06:10:33,856][20381] Updated weights for policy 1, policy_version 74184 (0.0007) [2023-12-27 06:10:34,148][20347] Updated weights for policy 0, policy_version 74297 (0.0006) [2023-12-27 06:10:34,205][20347] Updated weights for policy 0, policy_version 74307 (0.0008) [2023-12-27 06:10:34,258][20347] Updated weights for policy 0, policy_version 74317 (0.0007) [2023-12-27 06:10:34,321][20347] Updated weights for policy 0, policy_version 74327 (0.0008) [2023-12-27 06:10:34,497][20381] Updated weights for policy 1, policy_version 74194 (0.0008) [2023-12-27 06:10:34,557][20381] Updated weights for policy 1, policy_version 74204 (0.0007) [2023-12-27 06:10:34,619][20381] Updated weights for policy 1, policy_version 74214 (0.0008) [2023-12-27 06:10:35,018][20347] Updated weights for policy 0, policy_version 74337 (0.0008) [2023-12-27 06:10:35,077][20347] Updated weights for policy 0, policy_version 74347 (0.0005) [2023-12-27 06:10:35,131][20347] Updated weights for policy 0, policy_version 74357 (0.0005) [2023-12-27 06:10:35,223][20381] Updated weights for policy 1, policy_version 74224 (0.0007) [2023-12-27 06:10:35,285][20381] Updated weights for policy 1, policy_version 74234 (0.0007) [2023-12-27 06:10:35,348][20381] Updated weights for policy 1, policy_version 74244 (0.0006) [2023-12-27 06:10:35,707][20347] Updated weights for policy 0, policy_version 74367 (0.0006) [2023-12-27 06:10:35,760][20347] Updated weights for policy 0, policy_version 74377 (0.0007) [2023-12-27 06:10:35,816][20347] Updated weights for policy 0, policy_version 74387 (0.0007) [2023-12-27 06:10:36,060][20381] Updated weights for policy 1, policy_version 74254 (0.0009) [2023-12-27 06:10:36,118][20381] Updated weights for policy 1, policy_version 74264 (0.0007) [2023-12-27 06:10:36,178][20381] Updated weights for policy 1, policy_version 74274 (0.0011) [2023-12-27 06:10:36,494][20347] Updated weights for policy 0, policy_version 74397 (0.0008) [2023-12-27 06:10:36,543][20347] Updated weights for policy 0, policy_version 74407 (0.0006) [2023-12-27 06:10:36,599][20347] Updated weights for policy 0, policy_version 74417 (0.0007) [2023-12-27 06:10:36,883][20381] Updated weights for policy 1, policy_version 74284 (0.0011) [2023-12-27 06:10:36,950][20381] Updated weights for policy 1, policy_version 74294 (0.0011) [2023-12-27 06:10:37,010][20381] Updated weights for policy 1, policy_version 74304 (0.0011) [2023-12-27 06:10:37,218][20347] Updated weights for policy 0, policy_version 74427 (0.0007) [2023-12-27 06:10:37,265][20347] Updated weights for policy 0, policy_version 74437 (0.0007) [2023-12-27 06:10:37,310][20347] Updated weights for policy 0, policy_version 74447 (0.0007) [2023-12-27 06:10:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 38100992. Throughput: 0: 10403.8, 1: 10570.8. Samples: 38064372. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000074456_19070976.pth... [2023-12-27 06:10:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000074312_19030016.pth... [2023-12-27 06:10:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000073240_18759680.pth [2023-12-27 06:10:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000073064_18710528.pth [2023-12-27 06:10:37,677][20381] Updated weights for policy 1, policy_version 74314 (0.0011) [2023-12-27 06:10:37,733][20381] Updated weights for policy 1, policy_version 74324 (0.0011) [2023-12-27 06:10:37,788][20381] Updated weights for policy 1, policy_version 74334 (0.0010) [2023-12-27 06:10:37,849][20381] Updated weights for policy 1, policy_version 74344 (0.0010) [2023-12-27 06:10:38,017][20347] Updated weights for policy 0, policy_version 74457 (0.0006) [2023-12-27 06:10:38,071][20347] Updated weights for policy 0, policy_version 74467 (0.0008) [2023-12-27 06:10:38,116][20347] Updated weights for policy 0, policy_version 74477 (0.0009) [2023-12-27 06:10:38,165][20347] Updated weights for policy 0, policy_version 74487 (0.0008) [2023-12-27 06:10:38,464][20381] Updated weights for policy 1, policy_version 74354 (0.0010) [2023-12-27 06:10:38,528][20381] Updated weights for policy 1, policy_version 74364 (0.0007) [2023-12-27 06:10:38,590][20381] Updated weights for policy 1, policy_version 74374 (0.0008) [2023-12-27 06:10:38,938][20347] Updated weights for policy 0, policy_version 74497 (0.0008) [2023-12-27 06:10:38,999][20347] Updated weights for policy 0, policy_version 74507 (0.0008) [2023-12-27 06:10:39,060][20347] Updated weights for policy 0, policy_version 74517 (0.0008) [2023-12-27 06:10:39,279][20381] Updated weights for policy 1, policy_version 74384 (0.0007) [2023-12-27 06:10:39,339][20381] Updated weights for policy 1, policy_version 74394 (0.0008) [2023-12-27 06:10:39,413][20381] Updated weights for policy 1, policy_version 74404 (0.0010) [2023-12-27 06:10:39,791][20347] Updated weights for policy 0, policy_version 74527 (0.0008) [2023-12-27 06:10:39,856][20347] Updated weights for policy 0, policy_version 74537 (0.0007) [2023-12-27 06:10:39,923][20347] Updated weights for policy 0, policy_version 74547 (0.0008) [2023-12-27 06:10:40,105][20381] Updated weights for policy 1, policy_version 74414 (0.0008) [2023-12-27 06:10:40,161][20381] Updated weights for policy 1, policy_version 74424 (0.0005) [2023-12-27 06:10:40,214][20381] Updated weights for policy 1, policy_version 74434 (0.0007) [2023-12-27 06:10:40,663][20347] Updated weights for policy 0, policy_version 74557 (0.0007) [2023-12-27 06:10:40,727][20347] Updated weights for policy 0, policy_version 74567 (0.0006) [2023-12-27 06:10:40,781][20347] Updated weights for policy 0, policy_version 74577 (0.0007) [2023-12-27 06:10:40,899][20381] Updated weights for policy 1, policy_version 74444 (0.0007) [2023-12-27 06:10:40,959][20381] Updated weights for policy 1, policy_version 74454 (0.0009) [2023-12-27 06:10:41,019][20381] Updated weights for policy 1, policy_version 74464 (0.0008) [2023-12-27 06:10:41,447][20347] Updated weights for policy 0, policy_version 74587 (0.0007) [2023-12-27 06:10:41,510][20347] Updated weights for policy 0, policy_version 74597 (0.0008) [2023-12-27 06:10:41,572][20347] Updated weights for policy 0, policy_version 74607 (0.0007) [2023-12-27 06:10:41,783][20381] Updated weights for policy 1, policy_version 74474 (0.0008) [2023-12-27 06:10:41,842][20381] Updated weights for policy 1, policy_version 74484 (0.0008) [2023-12-27 06:10:41,898][20381] Updated weights for policy 1, policy_version 74494 (0.0008) [2023-12-27 06:10:41,957][20381] Updated weights for policy 1, policy_version 74504 (0.0008) [2023-12-27 06:10:42,302][20347] Updated weights for policy 0, policy_version 74617 (0.0008) [2023-12-27 06:10:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 38191104. Throughput: 0: 10375.8, 1: 10519.0. Samples: 38185160. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:42,363][20347] Updated weights for policy 0, policy_version 74627 (0.0008) [2023-12-27 06:10:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:42,415][20347] Updated weights for policy 0, policy_version 74637 (0.0008) [2023-12-27 06:10:42,467][20347] Updated weights for policy 0, policy_version 74647 (0.0008) [2023-12-27 06:10:42,586][20381] Updated weights for policy 1, policy_version 74514 (0.0007) [2023-12-27 06:10:42,646][20381] Updated weights for policy 1, policy_version 74524 (0.0008) [2023-12-27 06:10:42,698][20381] Updated weights for policy 1, policy_version 74534 (0.0008) [2023-12-27 06:10:43,174][20347] Updated weights for policy 0, policy_version 74657 (0.0006) [2023-12-27 06:10:43,221][20347] Updated weights for policy 0, policy_version 74667 (0.0009) [2023-12-27 06:10:43,266][20347] Updated weights for policy 0, policy_version 74677 (0.0010) [2023-12-27 06:10:43,400][20381] Updated weights for policy 1, policy_version 74544 (0.0010) [2023-12-27 06:10:43,464][20381] Updated weights for policy 1, policy_version 74554 (0.0010) [2023-12-27 06:10:43,518][20381] Updated weights for policy 1, policy_version 74564 (0.0009) [2023-12-27 06:10:43,886][20347] Updated weights for policy 0, policy_version 74687 (0.0010) [2023-12-27 06:10:43,937][20347] Updated weights for policy 0, policy_version 74697 (0.0010) [2023-12-27 06:10:43,981][20347] Updated weights for policy 0, policy_version 74707 (0.0010) [2023-12-27 06:10:44,040][20381] Updated weights for policy 1, policy_version 74574 (0.0005) [2023-12-27 06:10:44,085][20381] Updated weights for policy 1, policy_version 74584 (0.0006) [2023-12-27 06:10:44,139][20381] Updated weights for policy 1, policy_version 74594 (0.0010) [2023-12-27 06:10:44,648][20347] Updated weights for policy 0, policy_version 74717 (0.0010) [2023-12-27 06:10:44,690][20381] Updated weights for policy 1, policy_version 74604 (0.0009) [2023-12-27 06:10:44,703][20347] Updated weights for policy 0, policy_version 74727 (0.0010) [2023-12-27 06:10:44,738][20381] Updated weights for policy 1, policy_version 74614 (0.0006) [2023-12-27 06:10:44,758][20347] Updated weights for policy 0, policy_version 74737 (0.0010) [2023-12-27 06:10:44,797][20381] Updated weights for policy 1, policy_version 74624 (0.0009) [2023-12-27 06:10:45,392][20347] Updated weights for policy 0, policy_version 74747 (0.0009) [2023-12-27 06:10:45,443][20347] Updated weights for policy 0, policy_version 74757 (0.0006) [2023-12-27 06:10:45,475][20381] Updated weights for policy 1, policy_version 74634 (0.0009) [2023-12-27 06:10:45,501][20347] Updated weights for policy 0, policy_version 74767 (0.0006) [2023-12-27 06:10:45,538][20381] Updated weights for policy 1, policy_version 74644 (0.0006) [2023-12-27 06:10:45,602][20381] Updated weights for policy 1, policy_version 74654 (0.0007) [2023-12-27 06:10:45,658][20381] Updated weights for policy 1, policy_version 74664 (0.0005) [2023-12-27 06:10:46,086][20347] Updated weights for policy 0, policy_version 74777 (0.0006) [2023-12-27 06:10:46,136][20347] Updated weights for policy 0, policy_version 74787 (0.0005) [2023-12-27 06:10:46,184][20347] Updated weights for policy 0, policy_version 74797 (0.0006) [2023-12-27 06:10:46,198][20381] Updated weights for policy 1, policy_version 74674 (0.0007) [2023-12-27 06:10:46,235][20347] Updated weights for policy 0, policy_version 74807 (0.0006) [2023-12-27 06:10:46,267][20381] Updated weights for policy 1, policy_version 74684 (0.0007) [2023-12-27 06:10:46,333][20381] Updated weights for policy 1, policy_version 74694 (0.0007) [2023-12-27 06:10:46,855][20347] Updated weights for policy 0, policy_version 74817 (0.0006) [2023-12-27 06:10:46,914][20347] Updated weights for policy 0, policy_version 74827 (0.0005) [2023-12-27 06:10:46,976][20347] Updated weights for policy 0, policy_version 74837 (0.0005) [2023-12-27 06:10:47,063][20381] Updated weights for policy 1, policy_version 74704 (0.0008) [2023-12-27 06:10:47,115][20381] Updated weights for policy 1, policy_version 74714 (0.0008) [2023-12-27 06:10:47,177][20381] Updated weights for policy 1, policy_version 74724 (0.0008) [2023-12-27 06:10:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 38305792. Throughput: 0: 10399.4, 1: 10585.8. Samples: 38313308. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:47,528][20347] Updated weights for policy 0, policy_version 74847 (0.0005) [2023-12-27 06:10:47,586][20347] Updated weights for policy 0, policy_version 74857 (0.0007) [2023-12-27 06:10:47,646][20347] Updated weights for policy 0, policy_version 74867 (0.0008) [2023-12-27 06:10:47,807][20381] Updated weights for policy 1, policy_version 74734 (0.0008) [2023-12-27 06:10:47,866][20381] Updated weights for policy 1, policy_version 74744 (0.0008) [2023-12-27 06:10:47,919][20381] Updated weights for policy 1, policy_version 74754 (0.0007) [2023-12-27 06:10:48,291][20347] Updated weights for policy 0, policy_version 74877 (0.0006) [2023-12-27 06:10:48,354][20347] Updated weights for policy 0, policy_version 74887 (0.0006) [2023-12-27 06:10:48,417][20347] Updated weights for policy 0, policy_version 74897 (0.0006) [2023-12-27 06:10:48,612][20381] Updated weights for policy 1, policy_version 74764 (0.0009) [2023-12-27 06:10:48,665][20381] Updated weights for policy 1, policy_version 74774 (0.0007) [2023-12-27 06:10:48,717][20381] Updated weights for policy 1, policy_version 74784 (0.0007) [2023-12-27 06:10:49,000][20347] Updated weights for policy 0, policy_version 74907 (0.0006) [2023-12-27 06:10:49,054][20347] Updated weights for policy 0, policy_version 74917 (0.0007) [2023-12-27 06:10:49,119][20347] Updated weights for policy 0, policy_version 74927 (0.0005) [2023-12-27 06:10:49,431][20381] Updated weights for policy 1, policy_version 74794 (0.0009) [2023-12-27 06:10:49,486][20381] Updated weights for policy 1, policy_version 74804 (0.0011) [2023-12-27 06:10:49,535][20381] Updated weights for policy 1, policy_version 74814 (0.0011) [2023-12-27 06:10:49,585][20381] Updated weights for policy 1, policy_version 74824 (0.0011) [2023-12-27 06:10:49,776][20347] Updated weights for policy 0, policy_version 74937 (0.0006) [2023-12-27 06:10:49,830][20347] Updated weights for policy 0, policy_version 74947 (0.0007) [2023-12-27 06:10:49,897][20347] Updated weights for policy 0, policy_version 74957 (0.0007) [2023-12-27 06:10:49,964][20347] Updated weights for policy 0, policy_version 74967 (0.0007) [2023-12-27 06:10:50,318][20381] Updated weights for policy 1, policy_version 74834 (0.0006) [2023-12-27 06:10:50,384][20381] Updated weights for policy 1, policy_version 74844 (0.0005) [2023-12-27 06:10:50,433][20381] Updated weights for policy 1, policy_version 74854 (0.0005) [2023-12-27 06:10:50,572][20347] Updated weights for policy 0, policy_version 74977 (0.0007) [2023-12-27 06:10:50,630][20347] Updated weights for policy 0, policy_version 74987 (0.0007) [2023-12-27 06:10:50,690][20347] Updated weights for policy 0, policy_version 74997 (0.0008) [2023-12-27 06:10:51,096][20381] Updated weights for policy 1, policy_version 74864 (0.0007) [2023-12-27 06:10:51,155][20381] Updated weights for policy 1, policy_version 74874 (0.0007) [2023-12-27 06:10:51,213][20381] Updated weights for policy 1, policy_version 74884 (0.0008) [2023-12-27 06:10:51,319][20347] Updated weights for policy 0, policy_version 75007 (0.0006) [2023-12-27 06:10:51,377][20347] Updated weights for policy 0, policy_version 75017 (0.0009) [2023-12-27 06:10:51,431][20347] Updated weights for policy 0, policy_version 75027 (0.0009) [2023-12-27 06:10:51,884][20381] Updated weights for policy 1, policy_version 74894 (0.0006) [2023-12-27 06:10:51,944][20381] Updated weights for policy 1, policy_version 74904 (0.0007) [2023-12-27 06:10:51,999][20381] Updated weights for policy 1, policy_version 74914 (0.0008) [2023-12-27 06:10:52,119][20347] Updated weights for policy 0, policy_version 75037 (0.0010) [2023-12-27 06:10:52,174][20347] Updated weights for policy 0, policy_version 75047 (0.0010) [2023-12-27 06:10:52,238][20347] Updated weights for policy 0, policy_version 75057 (0.0010) [2023-12-27 06:10:52,362][19170] Fps is (10 sec: 22118.4, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 38412288. Throughput: 0: 10390.1, 1: 10552.1. Samples: 38376356. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000074920_19185664.pth... [2023-12-27 06:10:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000075064_19226624.pth... [2023-12-27 06:10:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000073672_18866176.pth [2023-12-27 06:10:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000073848_18915328.pth [2023-12-27 06:10:52,712][20381] Updated weights for policy 1, policy_version 74924 (0.0008) [2023-12-27 06:10:52,773][20381] Updated weights for policy 1, policy_version 74934 (0.0007) [2023-12-27 06:10:52,818][20381] Updated weights for policy 1, policy_version 74944 (0.0007) [2023-12-27 06:10:52,941][20347] Updated weights for policy 0, policy_version 75067 (0.0008) [2023-12-27 06:10:52,996][20347] Updated weights for policy 0, policy_version 75077 (0.0007) [2023-12-27 06:10:53,053][20347] Updated weights for policy 0, policy_version 75087 (0.0007) [2023-12-27 06:10:53,508][20381] Updated weights for policy 1, policy_version 74954 (0.0007) [2023-12-27 06:10:53,561][20381] Updated weights for policy 1, policy_version 74964 (0.0008) [2023-12-27 06:10:53,617][20381] Updated weights for policy 1, policy_version 74974 (0.0007) [2023-12-27 06:10:53,672][20381] Updated weights for policy 1, policy_version 74984 (0.0010) [2023-12-27 06:10:53,765][20347] Updated weights for policy 0, policy_version 75097 (0.0008) [2023-12-27 06:10:53,821][20347] Updated weights for policy 0, policy_version 75107 (0.0007) [2023-12-27 06:10:53,883][20347] Updated weights for policy 0, policy_version 75117 (0.0007) [2023-12-27 06:10:53,942][20347] Updated weights for policy 0, policy_version 75127 (0.0008) [2023-12-27 06:10:54,302][20381] Updated weights for policy 1, policy_version 74994 (0.0011) [2023-12-27 06:10:54,361][20381] Updated weights for policy 1, policy_version 75004 (0.0011) [2023-12-27 06:10:54,420][20381] Updated weights for policy 1, policy_version 75014 (0.0011) [2023-12-27 06:10:54,635][20347] Updated weights for policy 0, policy_version 75137 (0.0008) [2023-12-27 06:10:54,682][20347] Updated weights for policy 0, policy_version 75147 (0.0007) [2023-12-27 06:10:54,730][20347] Updated weights for policy 0, policy_version 75157 (0.0007) [2023-12-27 06:10:55,076][20381] Updated weights for policy 1, policy_version 75024 (0.0009) [2023-12-27 06:10:55,131][20381] Updated weights for policy 1, policy_version 75034 (0.0011) [2023-12-27 06:10:55,183][20381] Updated weights for policy 1, policy_version 75044 (0.0010) [2023-12-27 06:10:55,373][20347] Updated weights for policy 0, policy_version 75167 (0.0010) [2023-12-27 06:10:55,431][20347] Updated weights for policy 0, policy_version 75177 (0.0010) [2023-12-27 06:10:55,489][20347] Updated weights for policy 0, policy_version 75187 (0.0010) [2023-12-27 06:10:55,831][20381] Updated weights for policy 1, policy_version 75054 (0.0009) [2023-12-27 06:10:55,882][20381] Updated weights for policy 1, policy_version 75064 (0.0010) [2023-12-27 06:10:55,930][20381] Updated weights for policy 1, policy_version 75074 (0.0010) [2023-12-27 06:10:56,107][20347] Updated weights for policy 0, policy_version 75197 (0.0008) [2023-12-27 06:10:56,158][20347] Updated weights for policy 0, policy_version 75207 (0.0005) [2023-12-27 06:10:56,218][20347] Updated weights for policy 0, policy_version 75217 (0.0006) [2023-12-27 06:10:56,610][20381] Updated weights for policy 1, policy_version 75084 (0.0008) [2023-12-27 06:10:56,663][20381] Updated weights for policy 1, policy_version 75094 (0.0005) [2023-12-27 06:10:56,716][20381] Updated weights for policy 1, policy_version 75104 (0.0006) [2023-12-27 06:10:56,780][20347] Updated weights for policy 0, policy_version 75227 (0.0007) [2023-12-27 06:10:56,824][20347] Updated weights for policy 0, policy_version 75237 (0.0008) [2023-12-27 06:10:56,876][20347] Updated weights for policy 0, policy_version 75247 (0.0005) [2023-12-27 06:10:57,330][20381] Updated weights for policy 1, policy_version 75114 (0.0006) [2023-12-27 06:10:57,366][19170] Fps is (10 sec: 20472.9, 60 sec: 20888.4, 300 sec: 20799.1). Total num frames: 38510592. Throughput: 0: 10377.5, 1: 10560.4. Samples: 38501232. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:10:57,367][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:10:57,403][20381] Updated weights for policy 1, policy_version 75124 (0.0006) [2023-12-27 06:10:57,456][20381] Updated weights for policy 1, policy_version 75134 (0.0005) [2023-12-27 06:10:57,518][20381] Updated weights for policy 1, policy_version 75144 (0.0005) [2023-12-27 06:10:57,531][20347] Updated weights for policy 0, policy_version 75257 (0.0009) [2023-12-27 06:10:57,581][20347] Updated weights for policy 0, policy_version 75267 (0.0010) [2023-12-27 06:10:57,629][20347] Updated weights for policy 0, policy_version 75277 (0.0010) [2023-12-27 06:10:57,676][20347] Updated weights for policy 0, policy_version 75287 (0.0010) [2023-12-27 06:10:58,074][20381] Updated weights for policy 1, policy_version 75154 (0.0005) [2023-12-27 06:10:58,131][20381] Updated weights for policy 1, policy_version 75164 (0.0006) [2023-12-27 06:10:58,195][20381] Updated weights for policy 1, policy_version 75174 (0.0007) [2023-12-27 06:10:58,296][20347] Updated weights for policy 0, policy_version 75297 (0.0007) [2023-12-27 06:10:58,361][20347] Updated weights for policy 0, policy_version 75307 (0.0007) [2023-12-27 06:10:58,424][20347] Updated weights for policy 0, policy_version 75317 (0.0008) [2023-12-27 06:10:58,917][20381] Updated weights for policy 1, policy_version 75184 (0.0009) [2023-12-27 06:10:58,974][20381] Updated weights for policy 1, policy_version 75194 (0.0009) [2023-12-27 06:10:59,033][20381] Updated weights for policy 1, policy_version 75204 (0.0009) [2023-12-27 06:10:59,245][20347] Updated weights for policy 0, policy_version 75327 (0.0010) [2023-12-27 06:10:59,303][20347] Updated weights for policy 0, policy_version 75337 (0.0010) [2023-12-27 06:10:59,364][20347] Updated weights for policy 0, policy_version 75347 (0.0007) [2023-12-27 06:10:59,808][20381] Updated weights for policy 1, policy_version 75214 (0.0009) [2023-12-27 06:10:59,866][20381] Updated weights for policy 1, policy_version 75224 (0.0006) [2023-12-27 06:10:59,934][20381] Updated weights for policy 1, policy_version 75234 (0.0008) [2023-12-27 06:10:59,977][20347] Updated weights for policy 0, policy_version 75357 (0.0006) [2023-12-27 06:11:00,038][20347] Updated weights for policy 0, policy_version 75367 (0.0007) [2023-12-27 06:11:00,090][20347] Updated weights for policy 0, policy_version 75377 (0.0008) [2023-12-27 06:11:00,627][20381] Updated weights for policy 1, policy_version 75244 (0.0005) [2023-12-27 06:11:00,682][20381] Updated weights for policy 1, policy_version 75254 (0.0005) [2023-12-27 06:11:00,728][20381] Updated weights for policy 1, policy_version 75264 (0.0007) [2023-12-27 06:11:00,782][20347] Updated weights for policy 0, policy_version 75387 (0.0008) [2023-12-27 06:11:00,840][20347] Updated weights for policy 0, policy_version 75397 (0.0008) [2023-12-27 06:11:00,891][20347] Updated weights for policy 0, policy_version 75407 (0.0008) [2023-12-27 06:11:01,455][20381] Updated weights for policy 1, policy_version 75274 (0.0008) [2023-12-27 06:11:01,511][20381] Updated weights for policy 1, policy_version 75284 (0.0008) [2023-12-27 06:11:01,559][20347] Updated weights for policy 0, policy_version 75417 (0.0007) [2023-12-27 06:11:01,573][20381] Updated weights for policy 1, policy_version 75294 (0.0007) [2023-12-27 06:11:01,616][20347] Updated weights for policy 0, policy_version 75427 (0.0006) [2023-12-27 06:11:01,631][20381] Updated weights for policy 1, policy_version 75304 (0.0006) [2023-12-27 06:11:01,679][20347] Updated weights for policy 0, policy_version 75437 (0.0006) [2023-12-27 06:11:01,742][20347] Updated weights for policy 0, policy_version 75447 (0.0008) [2023-12-27 06:11:02,355][20381] Updated weights for policy 1, policy_version 75314 (0.0006) [2023-12-27 06:11:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 38608896. Throughput: 0: 10426.6, 1: 10458.9. Samples: 38625372. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 06:11:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:02,387][20347] Updated weights for policy 0, policy_version 75457 (0.0008) [2023-12-27 06:11:02,411][20381] Updated weights for policy 1, policy_version 75324 (0.0007) [2023-12-27 06:11:02,442][20347] Updated weights for policy 0, policy_version 75467 (0.0006) [2023-12-27 06:11:02,462][20381] Updated weights for policy 1, policy_version 75334 (0.0005) [2023-12-27 06:11:02,498][20347] Updated weights for policy 0, policy_version 75477 (0.0008) [2023-12-27 06:11:03,037][20381] Updated weights for policy 1, policy_version 75344 (0.0005) [2023-12-27 06:11:03,089][20381] Updated weights for policy 1, policy_version 75354 (0.0005) [2023-12-27 06:11:03,147][20381] Updated weights for policy 1, policy_version 75364 (0.0006) [2023-12-27 06:11:03,215][20347] Updated weights for policy 0, policy_version 75487 (0.0008) [2023-12-27 06:11:03,264][20347] Updated weights for policy 0, policy_version 75497 (0.0005) [2023-12-27 06:11:03,322][20347] Updated weights for policy 0, policy_version 75507 (0.0005) [2023-12-27 06:11:03,763][20381] Updated weights for policy 1, policy_version 75374 (0.0008) [2023-12-27 06:11:03,819][20381] Updated weights for policy 1, policy_version 75384 (0.0007) [2023-12-27 06:11:03,878][20381] Updated weights for policy 1, policy_version 75394 (0.0008) [2023-12-27 06:11:03,917][20347] Updated weights for policy 0, policy_version 75517 (0.0007) [2023-12-27 06:11:03,984][20347] Updated weights for policy 0, policy_version 75527 (0.0007) [2023-12-27 06:11:04,052][20347] Updated weights for policy 0, policy_version 75537 (0.0007) [2023-12-27 06:11:04,600][20381] Updated weights for policy 1, policy_version 75404 (0.0007) [2023-12-27 06:11:04,658][20381] Updated weights for policy 1, policy_version 75414 (0.0008) [2023-12-27 06:11:04,717][20381] Updated weights for policy 1, policy_version 75424 (0.0007) [2023-12-27 06:11:04,741][20347] Updated weights for policy 0, policy_version 75547 (0.0010) [2023-12-27 06:11:04,795][20347] Updated weights for policy 0, policy_version 75557 (0.0010) [2023-12-27 06:11:04,843][20347] Updated weights for policy 0, policy_version 75567 (0.0010) [2023-12-27 06:11:05,395][20381] Updated weights for policy 1, policy_version 75434 (0.0005) [2023-12-27 06:11:05,442][20381] Updated weights for policy 1, policy_version 75444 (0.0007) [2023-12-27 06:11:05,496][20381] Updated weights for policy 1, policy_version 75454 (0.0007) [2023-12-27 06:11:05,530][20347] Updated weights for policy 0, policy_version 75577 (0.0010) [2023-12-27 06:11:05,542][20381] Updated weights for policy 1, policy_version 75464 (0.0007) [2023-12-27 06:11:05,584][20347] Updated weights for policy 0, policy_version 75587 (0.0010) [2023-12-27 06:11:05,644][20347] Updated weights for policy 0, policy_version 75597 (0.0010) [2023-12-27 06:11:05,694][20347] Updated weights for policy 0, policy_version 75607 (0.0010) [2023-12-27 06:11:06,263][20381] Updated weights for policy 1, policy_version 75474 (0.0007) [2023-12-27 06:11:06,328][20381] Updated weights for policy 1, policy_version 75484 (0.0008) [2023-12-27 06:11:06,380][20347] Updated weights for policy 0, policy_version 75617 (0.0010) [2023-12-27 06:11:06,390][20381] Updated weights for policy 1, policy_version 75494 (0.0005) [2023-12-27 06:11:06,436][20347] Updated weights for policy 0, policy_version 75627 (0.0010) [2023-12-27 06:11:06,494][20347] Updated weights for policy 0, policy_version 75637 (0.0010) [2023-12-27 06:11:07,071][20381] Updated weights for policy 1, policy_version 75504 (0.0007) [2023-12-27 06:11:07,122][20381] Updated weights for policy 1, policy_version 75514 (0.0007) [2023-12-27 06:11:07,176][20381] Updated weights for policy 1, policy_version 75524 (0.0007) [2023-12-27 06:11:07,182][20347] Updated weights for policy 0, policy_version 75647 (0.0010) [2023-12-27 06:11:07,234][20347] Updated weights for policy 0, policy_version 75657 (0.0010) [2023-12-27 06:11:07,290][20347] Updated weights for policy 0, policy_version 75667 (0.0010) [2023-12-27 06:11:07,362][19170] Fps is (10 sec: 21306.2, 60 sec: 20889.7, 300 sec: 20827.1). Total num frames: 38723584. Throughput: 0: 10402.2, 1: 10446.8. Samples: 38687504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000075672_19382272.pth... [2023-12-27 06:11:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000075528_19341312.pth... [2023-12-27 06:11:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000074456_19070976.pth [2023-12-27 06:11:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000074312_19030016.pth [2023-12-27 06:11:07,885][20381] Updated weights for policy 1, policy_version 75534 (0.0008) [2023-12-27 06:11:07,942][20381] Updated weights for policy 1, policy_version 75544 (0.0007) [2023-12-27 06:11:07,972][20347] Updated weights for policy 0, policy_version 75677 (0.0010) [2023-12-27 06:11:08,002][20381] Updated weights for policy 1, policy_version 75554 (0.0006) [2023-12-27 06:11:08,021][20347] Updated weights for policy 0, policy_version 75687 (0.0010) [2023-12-27 06:11:08,070][20347] Updated weights for policy 0, policy_version 75697 (0.0010) [2023-12-27 06:11:08,687][20381] Updated weights for policy 1, policy_version 75564 (0.0007) [2023-12-27 06:11:08,738][20381] Updated weights for policy 1, policy_version 75574 (0.0008) [2023-12-27 06:11:08,785][20381] Updated weights for policy 1, policy_version 75584 (0.0007) [2023-12-27 06:11:08,792][20347] Updated weights for policy 0, policy_version 75707 (0.0010) [2023-12-27 06:11:08,846][20347] Updated weights for policy 0, policy_version 75717 (0.0010) [2023-12-27 06:11:08,904][20347] Updated weights for policy 0, policy_version 75727 (0.0010) [2023-12-27 06:11:09,501][20381] Updated weights for policy 1, policy_version 75594 (0.0007) [2023-12-27 06:11:09,561][20381] Updated weights for policy 1, policy_version 75604 (0.0007) [2023-12-27 06:11:09,606][20347] Updated weights for policy 0, policy_version 75737 (0.0010) [2023-12-27 06:11:09,616][20381] Updated weights for policy 1, policy_version 75614 (0.0007) [2023-12-27 06:11:09,660][20347] Updated weights for policy 0, policy_version 75747 (0.0011) [2023-12-27 06:11:09,674][20381] Updated weights for policy 1, policy_version 75624 (0.0006) [2023-12-27 06:11:09,714][20347] Updated weights for policy 0, policy_version 75757 (0.0010) [2023-12-27 06:11:09,777][20347] Updated weights for policy 0, policy_version 75767 (0.0010) [2023-12-27 06:11:10,408][20381] Updated weights for policy 1, policy_version 75634 (0.0007) [2023-12-27 06:11:10,460][20381] Updated weights for policy 1, policy_version 75644 (0.0009) [2023-12-27 06:11:10,482][20347] Updated weights for policy 0, policy_version 75777 (0.0010) [2023-12-27 06:11:10,510][20381] Updated weights for policy 1, policy_version 75654 (0.0006) [2023-12-27 06:11:10,539][20347] Updated weights for policy 0, policy_version 75787 (0.0010) [2023-12-27 06:11:10,598][20347] Updated weights for policy 0, policy_version 75797 (0.0010) [2023-12-27 06:11:11,227][20381] Updated weights for policy 1, policy_version 75664 (0.0007) [2023-12-27 06:11:11,286][20347] Updated weights for policy 0, policy_version 75807 (0.0011) [2023-12-27 06:11:11,290][20381] Updated weights for policy 1, policy_version 75674 (0.0007) [2023-12-27 06:11:11,347][20347] Updated weights for policy 0, policy_version 75817 (0.0011) [2023-12-27 06:11:11,348][20381] Updated weights for policy 1, policy_version 75684 (0.0006) [2023-12-27 06:11:11,415][20347] Updated weights for policy 0, policy_version 75827 (0.0011) [2023-12-27 06:11:12,112][20381] Updated weights for policy 1, policy_version 75694 (0.0007) [2023-12-27 06:11:12,153][20347] Updated weights for policy 0, policy_version 75837 (0.0011) [2023-12-27 06:11:12,165][20381] Updated weights for policy 1, policy_version 75704 (0.0007) [2023-12-27 06:11:12,214][20347] Updated weights for policy 0, policy_version 75847 (0.0011) [2023-12-27 06:11:12,220][20381] Updated weights for policy 1, policy_version 75714 (0.0006) [2023-12-27 06:11:12,280][20347] Updated weights for policy 0, policy_version 75857 (0.0011) [2023-12-27 06:11:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 38821888. Throughput: 0: 10395.8, 1: 10361.2. Samples: 38806948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:12,994][20381] Updated weights for policy 1, policy_version 75724 (0.0007) [2023-12-27 06:11:12,997][20347] Updated weights for policy 0, policy_version 75867 (0.0011) [2023-12-27 06:11:13,046][20381] Updated weights for policy 1, policy_version 75734 (0.0005) [2023-12-27 06:11:13,052][20347] Updated weights for policy 0, policy_version 75877 (0.0010) [2023-12-27 06:11:13,102][20381] Updated weights for policy 1, policy_version 75744 (0.0005) [2023-12-27 06:11:13,107][20347] Updated weights for policy 0, policy_version 75887 (0.0010) [2023-12-27 06:11:13,787][20347] Updated weights for policy 0, policy_version 75897 (0.0010) [2023-12-27 06:11:13,796][20381] Updated weights for policy 1, policy_version 75754 (0.0006) [2023-12-27 06:11:13,844][20347] Updated weights for policy 0, policy_version 75907 (0.0010) [2023-12-27 06:11:13,860][20381] Updated weights for policy 1, policy_version 75764 (0.0005) [2023-12-27 06:11:13,901][20347] Updated weights for policy 0, policy_version 75917 (0.0010) [2023-12-27 06:11:13,913][20381] Updated weights for policy 1, policy_version 75774 (0.0006) [2023-12-27 06:11:13,957][20347] Updated weights for policy 0, policy_version 75927 (0.0005) [2023-12-27 06:11:13,968][20381] Updated weights for policy 1, policy_version 75784 (0.0005) [2023-12-27 06:11:14,581][20347] Updated weights for policy 0, policy_version 75937 (0.0010) [2023-12-27 06:11:14,590][20381] Updated weights for policy 1, policy_version 75794 (0.0005) [2023-12-27 06:11:14,635][20347] Updated weights for policy 0, policy_version 75947 (0.0010) [2023-12-27 06:11:14,645][20381] Updated weights for policy 1, policy_version 75804 (0.0005) [2023-12-27 06:11:14,693][20347] Updated weights for policy 0, policy_version 75957 (0.0010) [2023-12-27 06:11:14,698][20381] Updated weights for policy 1, policy_version 75814 (0.0005) [2023-12-27 06:11:15,319][20381] Updated weights for policy 1, policy_version 75824 (0.0010) [2023-12-27 06:11:15,378][20381] Updated weights for policy 1, policy_version 75834 (0.0010) [2023-12-27 06:11:15,397][20347] Updated weights for policy 0, policy_version 75967 (0.0009) [2023-12-27 06:11:15,433][20381] Updated weights for policy 1, policy_version 75844 (0.0011) [2023-12-27 06:11:15,454][20347] Updated weights for policy 0, policy_version 75977 (0.0007) [2023-12-27 06:11:15,513][20347] Updated weights for policy 0, policy_version 75987 (0.0007) [2023-12-27 06:11:16,040][20381] Updated weights for policy 1, policy_version 75854 (0.0007) [2023-12-27 06:11:16,092][20381] Updated weights for policy 1, policy_version 75864 (0.0005) [2023-12-27 06:11:16,146][20381] Updated weights for policy 1, policy_version 75874 (0.0005) [2023-12-27 06:11:16,148][20347] Updated weights for policy 0, policy_version 75997 (0.0007) [2023-12-27 06:11:16,200][20347] Updated weights for policy 0, policy_version 76007 (0.0007) [2023-12-27 06:11:16,251][20347] Updated weights for policy 0, policy_version 76017 (0.0005) [2023-12-27 06:11:16,661][20381] Updated weights for policy 1, policy_version 75884 (0.0005) [2023-12-27 06:11:16,713][20381] Updated weights for policy 1, policy_version 75894 (0.0007) [2023-12-27 06:11:16,762][20381] Updated weights for policy 1, policy_version 75904 (0.0010) [2023-12-27 06:11:16,950][20347] Updated weights for policy 0, policy_version 76027 (0.0008) [2023-12-27 06:11:17,005][20347] Updated weights for policy 0, policy_version 76037 (0.0007) [2023-12-27 06:11:17,054][20347] Updated weights for policy 0, policy_version 76047 (0.0007) [2023-12-27 06:11:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 38920192. Throughput: 0: 10377.9, 1: 10320.4. Samples: 38930912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:17,445][20381] Updated weights for policy 1, policy_version 75914 (0.0011) [2023-12-27 06:11:17,507][20381] Updated weights for policy 1, policy_version 75924 (0.0010) [2023-12-27 06:11:17,571][20381] Updated weights for policy 1, policy_version 75934 (0.0011) [2023-12-27 06:11:17,630][20381] Updated weights for policy 1, policy_version 75944 (0.0011) [2023-12-27 06:11:17,669][20347] Updated weights for policy 0, policy_version 76057 (0.0007) [2023-12-27 06:11:17,725][20347] Updated weights for policy 0, policy_version 76067 (0.0010) [2023-12-27 06:11:17,785][20347] Updated weights for policy 0, policy_version 76077 (0.0010) [2023-12-27 06:11:17,841][20347] Updated weights for policy 0, policy_version 76087 (0.0010) [2023-12-27 06:11:18,334][20381] Updated weights for policy 1, policy_version 75954 (0.0007) [2023-12-27 06:11:18,394][20381] Updated weights for policy 1, policy_version 75964 (0.0008) [2023-12-27 06:11:18,456][20381] Updated weights for policy 1, policy_version 75974 (0.0007) [2023-12-27 06:11:18,546][20347] Updated weights for policy 0, policy_version 76097 (0.0008) [2023-12-27 06:11:18,604][20347] Updated weights for policy 0, policy_version 76107 (0.0008) [2023-12-27 06:11:18,670][20347] Updated weights for policy 0, policy_version 76117 (0.0006) [2023-12-27 06:11:19,106][20381] Updated weights for policy 1, policy_version 75984 (0.0007) [2023-12-27 06:11:19,157][20381] Updated weights for policy 1, policy_version 75994 (0.0006) [2023-12-27 06:11:19,217][20381] Updated weights for policy 1, policy_version 76004 (0.0006) [2023-12-27 06:11:19,266][20347] Updated weights for policy 0, policy_version 76127 (0.0009) [2023-12-27 06:11:19,324][20347] Updated weights for policy 0, policy_version 76137 (0.0010) [2023-12-27 06:11:19,386][20347] Updated weights for policy 0, policy_version 76147 (0.0010) [2023-12-27 06:11:19,866][20381] Updated weights for policy 1, policy_version 76014 (0.0006) [2023-12-27 06:11:19,927][20381] Updated weights for policy 1, policy_version 76024 (0.0007) [2023-12-27 06:11:19,993][20381] Updated weights for policy 1, policy_version 76034 (0.0006) [2023-12-27 06:11:20,072][20347] Updated weights for policy 0, policy_version 76157 (0.0009) [2023-12-27 06:11:20,136][20347] Updated weights for policy 0, policy_version 76167 (0.0008) [2023-12-27 06:11:20,194][20347] Updated weights for policy 0, policy_version 76177 (0.0008) [2023-12-27 06:11:20,683][20381] Updated weights for policy 1, policy_version 76044 (0.0007) [2023-12-27 06:11:20,731][20381] Updated weights for policy 1, policy_version 76054 (0.0007) [2023-12-27 06:11:20,786][20381] Updated weights for policy 1, policy_version 76064 (0.0008) [2023-12-27 06:11:20,833][20347] Updated weights for policy 0, policy_version 76187 (0.0007) [2023-12-27 06:11:20,890][20347] Updated weights for policy 0, policy_version 76197 (0.0005) [2023-12-27 06:11:20,952][20347] Updated weights for policy 0, policy_version 76207 (0.0006) [2023-12-27 06:11:21,544][20381] Updated weights for policy 1, policy_version 76074 (0.0009) [2023-12-27 06:11:21,607][20381] Updated weights for policy 1, policy_version 76084 (0.0006) [2023-12-27 06:11:21,635][20347] Updated weights for policy 0, policy_version 76217 (0.0006) [2023-12-27 06:11:21,692][20381] Updated weights for policy 1, policy_version 76094 (0.0006) [2023-12-27 06:11:21,698][20347] Updated weights for policy 0, policy_version 76227 (0.0008) [2023-12-27 06:11:21,755][20381] Updated weights for policy 1, policy_version 76104 (0.0008) [2023-12-27 06:11:21,757][20347] Updated weights for policy 0, policy_version 76237 (0.0007) [2023-12-27 06:11:21,816][20347] Updated weights for policy 0, policy_version 76247 (0.0008) [2023-12-27 06:11:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 39018496. Throughput: 0: 10362.9, 1: 10276.6. Samples: 38993152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000076248_19529728.pth... [2023-12-27 06:11:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000075064_19226624.pth [2023-12-27 06:11:22,436][20381] Updated weights for policy 1, policy_version 76114 (0.0007) [2023-12-27 06:11:22,498][20381] Updated weights for policy 1, policy_version 76124 (0.0006) [2023-12-27 06:11:22,513][20347] Updated weights for policy 0, policy_version 76257 (0.0008) [2023-12-27 06:11:22,556][20381] Updated weights for policy 1, policy_version 76134 (0.0006) [2023-12-27 06:11:22,566][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000076136_19496960.pth... [2023-12-27 06:11:22,569][20347] Updated weights for policy 0, policy_version 76267 (0.0005) [2023-12-27 06:11:22,570][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000074920_19185664.pth [2023-12-27 06:11:22,626][20347] Updated weights for policy 0, policy_version 76277 (0.0007) [2023-12-27 06:11:23,159][20381] Updated weights for policy 1, policy_version 76144 (0.0007) [2023-12-27 06:11:23,219][20381] Updated weights for policy 1, policy_version 76154 (0.0006) [2023-12-27 06:11:23,275][20381] Updated weights for policy 1, policy_version 76164 (0.0005) [2023-12-27 06:11:23,366][20347] Updated weights for policy 0, policy_version 76287 (0.0008) [2023-12-27 06:11:23,418][20347] Updated weights for policy 0, policy_version 76297 (0.0007) [2023-12-27 06:11:23,466][20347] Updated weights for policy 0, policy_version 76307 (0.0007) [2023-12-27 06:11:23,916][20381] Updated weights for policy 1, policy_version 76174 (0.0006) [2023-12-27 06:11:23,971][20381] Updated weights for policy 1, policy_version 76184 (0.0006) [2023-12-27 06:11:24,022][20381] Updated weights for policy 1, policy_version 76194 (0.0006) [2023-12-27 06:11:24,150][20347] Updated weights for policy 0, policy_version 76317 (0.0009) [2023-12-27 06:11:24,204][20347] Updated weights for policy 0, policy_version 76327 (0.0010) [2023-12-27 06:11:24,261][20347] Updated weights for policy 0, policy_version 76337 (0.0010) [2023-12-27 06:11:24,619][20381] Updated weights for policy 1, policy_version 76204 (0.0007) [2023-12-27 06:11:24,667][20381] Updated weights for policy 1, policy_version 76214 (0.0005) [2023-12-27 06:11:24,715][20381] Updated weights for policy 1, policy_version 76224 (0.0007) [2023-12-27 06:11:24,854][20347] Updated weights for policy 0, policy_version 76347 (0.0008) [2023-12-27 06:11:24,905][20347] Updated weights for policy 0, policy_version 76357 (0.0005) [2023-12-27 06:11:24,957][20347] Updated weights for policy 0, policy_version 76367 (0.0005) [2023-12-27 06:11:25,396][20381] Updated weights for policy 1, policy_version 76234 (0.0007) [2023-12-27 06:11:25,452][20381] Updated weights for policy 1, policy_version 76244 (0.0008) [2023-12-27 06:11:25,506][20381] Updated weights for policy 1, policy_version 76254 (0.0008) [2023-12-27 06:11:25,568][20381] Updated weights for policy 1, policy_version 76264 (0.0008) [2023-12-27 06:11:25,608][20347] Updated weights for policy 0, policy_version 76377 (0.0006) [2023-12-27 06:11:25,670][20347] Updated weights for policy 0, policy_version 76387 (0.0006) [2023-12-27 06:11:25,723][20347] Updated weights for policy 0, policy_version 76397 (0.0006) [2023-12-27 06:11:25,772][20347] Updated weights for policy 0, policy_version 76407 (0.0010) [2023-12-27 06:11:26,257][20381] Updated weights for policy 1, policy_version 76274 (0.0007) [2023-12-27 06:11:26,325][20381] Updated weights for policy 1, policy_version 76284 (0.0007) [2023-12-27 06:11:26,383][20381] Updated weights for policy 1, policy_version 76294 (0.0007) [2023-12-27 06:11:26,392][20347] Updated weights for policy 0, policy_version 76417 (0.0010) [2023-12-27 06:11:26,441][20347] Updated weights for policy 0, policy_version 76427 (0.0008) [2023-12-27 06:11:26,494][20347] Updated weights for policy 0, policy_version 76437 (0.0005) [2023-12-27 06:11:27,067][20381] Updated weights for policy 1, policy_version 76304 (0.0006) [2023-12-27 06:11:27,076][20347] Updated weights for policy 0, policy_version 76447 (0.0006) [2023-12-27 06:11:27,121][20381] Updated weights for policy 1, policy_version 76314 (0.0007) [2023-12-27 06:11:27,139][20347] Updated weights for policy 0, policy_version 76457 (0.0006) [2023-12-27 06:11:27,173][20381] Updated weights for policy 1, policy_version 76324 (0.0006) [2023-12-27 06:11:27,198][20347] Updated weights for policy 0, policy_version 76467 (0.0006) [2023-12-27 06:11:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 39133184. Throughput: 0: 10394.5, 1: 10326.2. Samples: 39117592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:27,790][20347] Updated weights for policy 0, policy_version 76477 (0.0006) [2023-12-27 06:11:27,844][20347] Updated weights for policy 0, policy_version 76487 (0.0006) [2023-12-27 06:11:27,890][20381] Updated weights for policy 1, policy_version 76334 (0.0006) [2023-12-27 06:11:27,899][20347] Updated weights for policy 0, policy_version 76497 (0.0006) [2023-12-27 06:11:27,946][20381] Updated weights for policy 1, policy_version 76344 (0.0007) [2023-12-27 06:11:28,006][20381] Updated weights for policy 1, policy_version 76354 (0.0008) [2023-12-27 06:11:28,504][20347] Updated weights for policy 0, policy_version 76507 (0.0006) [2023-12-27 06:11:28,560][20347] Updated weights for policy 0, policy_version 76517 (0.0008) [2023-12-27 06:11:28,619][20347] Updated weights for policy 0, policy_version 76527 (0.0008) [2023-12-27 06:11:28,690][20381] Updated weights for policy 1, policy_version 76364 (0.0007) [2023-12-27 06:11:28,748][20381] Updated weights for policy 1, policy_version 76374 (0.0006) [2023-12-27 06:11:28,803][20381] Updated weights for policy 1, policy_version 76384 (0.0007) [2023-12-27 06:11:29,226][20347] Updated weights for policy 0, policy_version 76537 (0.0007) [2023-12-27 06:11:29,279][20347] Updated weights for policy 0, policy_version 76547 (0.0007) [2023-12-27 06:11:29,336][20347] Updated weights for policy 0, policy_version 76557 (0.0007) [2023-12-27 06:11:29,400][20347] Updated weights for policy 0, policy_version 76567 (0.0008) [2023-12-27 06:11:29,497][20381] Updated weights for policy 1, policy_version 76394 (0.0006) [2023-12-27 06:11:29,550][20381] Updated weights for policy 1, policy_version 76404 (0.0009) [2023-12-27 06:11:29,603][20381] Updated weights for policy 1, policy_version 76414 (0.0009) [2023-12-27 06:11:29,655][20381] Updated weights for policy 1, policy_version 76424 (0.0008) [2023-12-27 06:11:29,982][20347] Updated weights for policy 0, policy_version 76577 (0.0006) [2023-12-27 06:11:30,046][20347] Updated weights for policy 0, policy_version 76587 (0.0007) [2023-12-27 06:11:30,110][20347] Updated weights for policy 0, policy_version 76597 (0.0007) [2023-12-27 06:11:30,422][20381] Updated weights for policy 1, policy_version 76434 (0.0008) [2023-12-27 06:11:30,486][20381] Updated weights for policy 1, policy_version 76444 (0.0009) [2023-12-27 06:11:30,537][20381] Updated weights for policy 1, policy_version 76454 (0.0008) [2023-12-27 06:11:30,717][20347] Updated weights for policy 0, policy_version 76607 (0.0006) [2023-12-27 06:11:30,765][20347] Updated weights for policy 0, policy_version 76617 (0.0005) [2023-12-27 06:11:30,815][20347] Updated weights for policy 0, policy_version 76627 (0.0005) [2023-12-27 06:11:31,285][20381] Updated weights for policy 1, policy_version 76464 (0.0008) [2023-12-27 06:11:31,337][20381] Updated weights for policy 1, policy_version 76474 (0.0007) [2023-12-27 06:11:31,403][20381] Updated weights for policy 1, policy_version 76484 (0.0008) [2023-12-27 06:11:31,410][20347] Updated weights for policy 0, policy_version 76637 (0.0007) [2023-12-27 06:11:31,466][20347] Updated weights for policy 0, policy_version 76647 (0.0005) [2023-12-27 06:11:31,526][20347] Updated weights for policy 0, policy_version 76657 (0.0007) [2023-12-27 06:11:32,060][20381] Updated weights for policy 1, policy_version 76494 (0.0006) [2023-12-27 06:11:32,110][20381] Updated weights for policy 1, policy_version 76504 (0.0005) [2023-12-27 06:11:32,171][20381] Updated weights for policy 1, policy_version 76514 (0.0005) [2023-12-27 06:11:32,210][20347] Updated weights for policy 0, policy_version 76667 (0.0007) [2023-12-27 06:11:32,268][20347] Updated weights for policy 0, policy_version 76677 (0.0006) [2023-12-27 06:11:32,330][20347] Updated weights for policy 0, policy_version 76687 (0.0007) [2023-12-27 06:11:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 39231488. Throughput: 0: 10461.1, 1: 10213.3. Samples: 39243656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:32,810][20381] Updated weights for policy 1, policy_version 76524 (0.0005) [2023-12-27 06:11:32,865][20381] Updated weights for policy 1, policy_version 76534 (0.0007) [2023-12-27 06:11:32,917][20381] Updated weights for policy 1, policy_version 76544 (0.0007) [2023-12-27 06:11:33,010][20347] Updated weights for policy 0, policy_version 76697 (0.0008) [2023-12-27 06:11:33,058][20347] Updated weights for policy 0, policy_version 76707 (0.0008) [2023-12-27 06:11:33,105][20347] Updated weights for policy 0, policy_version 76717 (0.0007) [2023-12-27 06:11:33,151][20347] Updated weights for policy 0, policy_version 76727 (0.0007) [2023-12-27 06:11:33,596][20381] Updated weights for policy 1, policy_version 76554 (0.0006) [2023-12-27 06:11:33,647][20381] Updated weights for policy 1, policy_version 76564 (0.0005) [2023-12-27 06:11:33,705][20381] Updated weights for policy 1, policy_version 76574 (0.0005) [2023-12-27 06:11:33,763][20381] Updated weights for policy 1, policy_version 76584 (0.0007) [2023-12-27 06:11:33,788][20347] Updated weights for policy 0, policy_version 76737 (0.0006) [2023-12-27 06:11:33,841][20347] Updated weights for policy 0, policy_version 76747 (0.0007) [2023-12-27 06:11:33,895][20347] Updated weights for policy 0, policy_version 76757 (0.0010) [2023-12-27 06:11:34,452][20381] Updated weights for policy 1, policy_version 76594 (0.0008) [2023-12-27 06:11:34,507][20381] Updated weights for policy 1, policy_version 76604 (0.0007) [2023-12-27 06:11:34,552][20347] Updated weights for policy 0, policy_version 76767 (0.0010) [2023-12-27 06:11:34,568][20381] Updated weights for policy 1, policy_version 76614 (0.0007) [2023-12-27 06:11:34,610][20347] Updated weights for policy 0, policy_version 76777 (0.0010) [2023-12-27 06:11:34,666][20347] Updated weights for policy 0, policy_version 76787 (0.0010) [2023-12-27 06:11:35,269][20347] Updated weights for policy 0, policy_version 76797 (0.0010) [2023-12-27 06:11:35,309][20381] Updated weights for policy 1, policy_version 76624 (0.0005) [2023-12-27 06:11:35,321][20347] Updated weights for policy 0, policy_version 76807 (0.0010) [2023-12-27 06:11:35,367][20381] Updated weights for policy 1, policy_version 76634 (0.0005) [2023-12-27 06:11:35,373][20347] Updated weights for policy 0, policy_version 76817 (0.0010) [2023-12-27 06:11:35,416][20381] Updated weights for policy 1, policy_version 76644 (0.0006) [2023-12-27 06:11:35,981][20347] Updated weights for policy 0, policy_version 76827 (0.0009) [2023-12-27 06:11:36,040][20347] Updated weights for policy 0, policy_version 76837 (0.0005) [2023-12-27 06:11:36,099][20347] Updated weights for policy 0, policy_version 76847 (0.0006) [2023-12-27 06:11:36,158][20381] Updated weights for policy 1, policy_version 76654 (0.0006) [2023-12-27 06:11:36,211][20381] Updated weights for policy 1, policy_version 76664 (0.0007) [2023-12-27 06:11:36,272][20381] Updated weights for policy 1, policy_version 76674 (0.0008) [2023-12-27 06:11:36,665][20347] Updated weights for policy 0, policy_version 76857 (0.0010) [2023-12-27 06:11:36,718][20347] Updated weights for policy 0, policy_version 76867 (0.0006) [2023-12-27 06:11:36,773][20347] Updated weights for policy 0, policy_version 76877 (0.0005) [2023-12-27 06:11:36,830][20347] Updated weights for policy 0, policy_version 76887 (0.0007) [2023-12-27 06:11:37,019][20381] Updated weights for policy 1, policy_version 76684 (0.0008) [2023-12-27 06:11:37,071][20381] Updated weights for policy 1, policy_version 76694 (0.0008) [2023-12-27 06:11:37,122][20381] Updated weights for policy 1, policy_version 76704 (0.0008) [2023-12-27 06:11:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 39337984. Throughput: 0: 10448.9, 1: 10199.7. Samples: 39305544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000076712_19644416.pth... [2023-12-27 06:11:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000076888_19693568.pth... [2023-12-27 06:11:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000075528_19341312.pth [2023-12-27 06:11:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000075672_19382272.pth [2023-12-27 06:11:37,441][20347] Updated weights for policy 0, policy_version 76897 (0.0011) [2023-12-27 06:11:37,494][20347] Updated weights for policy 0, policy_version 76907 (0.0008) [2023-12-27 06:11:37,548][20347] Updated weights for policy 0, policy_version 76917 (0.0005) [2023-12-27 06:11:37,843][20381] Updated weights for policy 1, policy_version 76714 (0.0008) [2023-12-27 06:11:37,900][20381] Updated weights for policy 1, policy_version 76724 (0.0008) [2023-12-27 06:11:37,956][20381] Updated weights for policy 1, policy_version 76734 (0.0008) [2023-12-27 06:11:38,009][20381] Updated weights for policy 1, policy_version 76744 (0.0009) [2023-12-27 06:11:38,110][20347] Updated weights for policy 0, policy_version 76927 (0.0005) [2023-12-27 06:11:38,192][20347] Updated weights for policy 0, policy_version 76937 (0.0005) [2023-12-27 06:11:38,251][20347] Updated weights for policy 0, policy_version 76947 (0.0008) [2023-12-27 06:11:38,775][20381] Updated weights for policy 1, policy_version 76754 (0.0008) [2023-12-27 06:11:38,843][20381] Updated weights for policy 1, policy_version 76764 (0.0008) [2023-12-27 06:11:38,885][20347] Updated weights for policy 0, policy_version 76957 (0.0008) [2023-12-27 06:11:38,899][20381] Updated weights for policy 1, policy_version 76774 (0.0006) [2023-12-27 06:11:38,945][20347] Updated weights for policy 0, policy_version 76967 (0.0010) [2023-12-27 06:11:39,005][20347] Updated weights for policy 0, policy_version 76977 (0.0007) [2023-12-27 06:11:39,624][20381] Updated weights for policy 1, policy_version 76784 (0.0008) [2023-12-27 06:11:39,679][20347] Updated weights for policy 0, policy_version 76987 (0.0007) [2023-12-27 06:11:39,686][20381] Updated weights for policy 1, policy_version 76794 (0.0009) [2023-12-27 06:11:39,737][20347] Updated weights for policy 0, policy_version 76997 (0.0006) [2023-12-27 06:11:39,742][20381] Updated weights for policy 1, policy_version 76804 (0.0008) [2023-12-27 06:11:39,798][20347] Updated weights for policy 0, policy_version 77007 (0.0006) [2023-12-27 06:11:40,442][20381] Updated weights for policy 1, policy_version 76814 (0.0007) [2023-12-27 06:11:40,492][20347] Updated weights for policy 0, policy_version 77017 (0.0008) [2023-12-27 06:11:40,499][20381] Updated weights for policy 1, policy_version 76824 (0.0007) [2023-12-27 06:11:40,551][20347] Updated weights for policy 0, policy_version 77027 (0.0006) [2023-12-27 06:11:40,557][20381] Updated weights for policy 1, policy_version 76834 (0.0006) [2023-12-27 06:11:40,605][20347] Updated weights for policy 0, policy_version 77037 (0.0006) [2023-12-27 06:11:40,662][20347] Updated weights for policy 0, policy_version 77047 (0.0008) [2023-12-27 06:11:41,266][20381] Updated weights for policy 1, policy_version 76844 (0.0006) [2023-12-27 06:11:41,327][20381] Updated weights for policy 1, policy_version 76854 (0.0007) [2023-12-27 06:11:41,383][20347] Updated weights for policy 0, policy_version 77057 (0.0009) [2023-12-27 06:11:41,396][20381] Updated weights for policy 1, policy_version 76864 (0.0008) [2023-12-27 06:11:41,440][20347] Updated weights for policy 0, policy_version 77067 (0.0008) [2023-12-27 06:11:41,497][20347] Updated weights for policy 0, policy_version 77077 (0.0010) [2023-12-27 06:11:42,066][20381] Updated weights for policy 1, policy_version 76874 (0.0009) [2023-12-27 06:11:42,119][20381] Updated weights for policy 1, policy_version 76884 (0.0011) [2023-12-27 06:11:42,172][20381] Updated weights for policy 1, policy_version 76894 (0.0011) [2023-12-27 06:11:42,175][20347] Updated weights for policy 0, policy_version 77087 (0.0010) [2023-12-27 06:11:42,227][20381] Updated weights for policy 1, policy_version 76904 (0.0007) [2023-12-27 06:11:42,230][20347] Updated weights for policy 0, policy_version 77097 (0.0010) [2023-12-27 06:11:42,294][20347] Updated weights for policy 0, policy_version 77107 (0.0011) [2023-12-27 06:11:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 39444480. Throughput: 0: 10518.9, 1: 10099.5. Samples: 39428992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:42,887][20381] Updated weights for policy 1, policy_version 76914 (0.0008) [2023-12-27 06:11:42,943][20381] Updated weights for policy 1, policy_version 76924 (0.0006) [2023-12-27 06:11:42,967][20347] Updated weights for policy 0, policy_version 77117 (0.0009) [2023-12-27 06:11:43,000][20381] Updated weights for policy 1, policy_version 76934 (0.0006) [2023-12-27 06:11:43,029][20347] Updated weights for policy 0, policy_version 77127 (0.0006) [2023-12-27 06:11:43,089][20347] Updated weights for policy 0, policy_version 77137 (0.0005) [2023-12-27 06:11:43,741][20347] Updated weights for policy 0, policy_version 77147 (0.0007) [2023-12-27 06:11:43,755][20381] Updated weights for policy 1, policy_version 76944 (0.0008) [2023-12-27 06:11:43,798][20347] Updated weights for policy 0, policy_version 77157 (0.0008) [2023-12-27 06:11:43,815][20381] Updated weights for policy 1, policy_version 76954 (0.0007) [2023-12-27 06:11:43,858][20347] Updated weights for policy 0, policy_version 77167 (0.0006) [2023-12-27 06:11:43,878][20381] Updated weights for policy 1, policy_version 76964 (0.0006) [2023-12-27 06:11:44,541][20347] Updated weights for policy 0, policy_version 77177 (0.0005) [2023-12-27 06:11:44,578][20381] Updated weights for policy 1, policy_version 76974 (0.0006) [2023-12-27 06:11:44,588][20347] Updated weights for policy 0, policy_version 77187 (0.0006) [2023-12-27 06:11:44,631][20381] Updated weights for policy 1, policy_version 76984 (0.0007) [2023-12-27 06:11:44,643][20347] Updated weights for policy 0, policy_version 77197 (0.0005) [2023-12-27 06:11:44,686][20381] Updated weights for policy 1, policy_version 76994 (0.0005) [2023-12-27 06:11:44,699][20347] Updated weights for policy 0, policy_version 77207 (0.0007) [2023-12-27 06:11:45,345][20381] Updated weights for policy 1, policy_version 77004 (0.0006) [2023-12-27 06:11:45,396][20381] Updated weights for policy 1, policy_version 77014 (0.0006) [2023-12-27 06:11:45,439][20347] Updated weights for policy 0, policy_version 77217 (0.0007) [2023-12-27 06:11:45,451][20381] Updated weights for policy 1, policy_version 77024 (0.0010) [2023-12-27 06:11:45,496][20347] Updated weights for policy 0, policy_version 77227 (0.0006) [2023-12-27 06:11:45,550][20347] Updated weights for policy 0, policy_version 77237 (0.0007) [2023-12-27 06:11:46,055][20381] Updated weights for policy 1, policy_version 77034 (0.0010) [2023-12-27 06:11:46,103][20381] Updated weights for policy 1, policy_version 77044 (0.0007) [2023-12-27 06:11:46,151][20381] Updated weights for policy 1, policy_version 77054 (0.0007) [2023-12-27 06:11:46,195][20347] Updated weights for policy 0, policy_version 77247 (0.0007) [2023-12-27 06:11:46,207][20381] Updated weights for policy 1, policy_version 77064 (0.0005) [2023-12-27 06:11:46,242][20347] Updated weights for policy 0, policy_version 77257 (0.0008) [2023-12-27 06:11:46,287][20347] Updated weights for policy 0, policy_version 77267 (0.0005) [2023-12-27 06:11:46,879][20347] Updated weights for policy 0, policy_version 77277 (0.0007) [2023-12-27 06:11:46,889][20381] Updated weights for policy 1, policy_version 77074 (0.0006) [2023-12-27 06:11:46,931][20347] Updated weights for policy 0, policy_version 77287 (0.0008) [2023-12-27 06:11:46,944][20381] Updated weights for policy 1, policy_version 77084 (0.0006) [2023-12-27 06:11:46,986][20347] Updated weights for policy 0, policy_version 77297 (0.0006) [2023-12-27 06:11:46,999][20381] Updated weights for policy 1, policy_version 77094 (0.0006) [2023-12-27 06:11:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 39542784. Throughput: 0: 10478.1, 1: 10105.2. Samples: 39551620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:47,597][20347] Updated weights for policy 0, policy_version 77307 (0.0006) [2023-12-27 06:11:47,658][20347] Updated weights for policy 0, policy_version 77317 (0.0006) [2023-12-27 06:11:47,682][20381] Updated weights for policy 1, policy_version 77104 (0.0008) [2023-12-27 06:11:47,720][20347] Updated weights for policy 0, policy_version 77327 (0.0006) [2023-12-27 06:11:47,740][20381] Updated weights for policy 1, policy_version 77114 (0.0007) [2023-12-27 06:11:47,795][20381] Updated weights for policy 1, policy_version 77124 (0.0006) [2023-12-27 06:11:48,379][20347] Updated weights for policy 0, policy_version 77337 (0.0006) [2023-12-27 06:11:48,438][20347] Updated weights for policy 0, policy_version 77347 (0.0005) [2023-12-27 06:11:48,497][20347] Updated weights for policy 0, policy_version 77357 (0.0006) [2023-12-27 06:11:48,500][20381] Updated weights for policy 1, policy_version 77134 (0.0008) [2023-12-27 06:11:48,550][20347] Updated weights for policy 0, policy_version 77367 (0.0009) [2023-12-27 06:11:48,558][20381] Updated weights for policy 1, policy_version 77144 (0.0010) [2023-12-27 06:11:48,612][20381] Updated weights for policy 1, policy_version 77154 (0.0010) [2023-12-27 06:11:49,243][20347] Updated weights for policy 0, policy_version 77377 (0.0008) [2023-12-27 06:11:49,256][20381] Updated weights for policy 1, policy_version 77164 (0.0010) [2023-12-27 06:11:49,302][20347] Updated weights for policy 0, policy_version 77387 (0.0006) [2023-12-27 06:11:49,318][20381] Updated weights for policy 1, policy_version 77174 (0.0010) [2023-12-27 06:11:49,358][20347] Updated weights for policy 0, policy_version 77397 (0.0008) [2023-12-27 06:11:49,380][20381] Updated weights for policy 1, policy_version 77184 (0.0009) [2023-12-27 06:11:50,071][20347] Updated weights for policy 0, policy_version 77407 (0.0007) [2023-12-27 06:11:50,095][20381] Updated weights for policy 1, policy_version 77194 (0.0010) [2023-12-27 06:11:50,134][20347] Updated weights for policy 0, policy_version 77417 (0.0005) [2023-12-27 06:11:50,145][20381] Updated weights for policy 1, policy_version 77204 (0.0011) [2023-12-27 06:11:50,193][20347] Updated weights for policy 0, policy_version 77427 (0.0007) [2023-12-27 06:11:50,195][20381] Updated weights for policy 1, policy_version 77214 (0.0010) [2023-12-27 06:11:50,253][20381] Updated weights for policy 1, policy_version 77224 (0.0010) [2023-12-27 06:11:50,870][20347] Updated weights for policy 0, policy_version 77437 (0.0008) [2023-12-27 06:11:50,919][20381] Updated weights for policy 1, policy_version 77234 (0.0010) [2023-12-27 06:11:50,923][20347] Updated weights for policy 0, policy_version 77447 (0.0010) [2023-12-27 06:11:50,972][20347] Updated weights for policy 0, policy_version 77457 (0.0010) [2023-12-27 06:11:50,978][20381] Updated weights for policy 1, policy_version 77244 (0.0010) [2023-12-27 06:11:51,038][20381] Updated weights for policy 1, policy_version 77254 (0.0009) [2023-12-27 06:11:51,744][20381] Updated weights for policy 1, policy_version 77264 (0.0010) [2023-12-27 06:11:51,764][20347] Updated weights for policy 0, policy_version 77467 (0.0009) [2023-12-27 06:11:51,806][20381] Updated weights for policy 1, policy_version 77274 (0.0010) [2023-12-27 06:11:51,818][20347] Updated weights for policy 0, policy_version 77477 (0.0005) [2023-12-27 06:11:51,865][20381] Updated weights for policy 1, policy_version 77284 (0.0011) [2023-12-27 06:11:51,875][20347] Updated weights for policy 0, policy_version 77487 (0.0005) [2023-12-27 06:11:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 39641088. Throughput: 0: 10505.8, 1: 10072.7. Samples: 39613536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:11:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000077288_19791872.pth... [2023-12-27 06:11:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000077496_19849216.pth... [2023-12-27 06:11:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000076136_19496960.pth [2023-12-27 06:11:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000076248_19529728.pth [2023-12-27 06:11:52,484][20347] Updated weights for policy 0, policy_version 77497 (0.0007) [2023-12-27 06:11:52,543][20347] Updated weights for policy 0, policy_version 77507 (0.0007) [2023-12-27 06:11:52,547][20381] Updated weights for policy 1, policy_version 77294 (0.0010) [2023-12-27 06:11:52,606][20381] Updated weights for policy 1, policy_version 77304 (0.0008) [2023-12-27 06:11:52,606][20347] Updated weights for policy 0, policy_version 77517 (0.0007) [2023-12-27 06:11:52,661][20347] Updated weights for policy 0, policy_version 77527 (0.0007) [2023-12-27 06:11:52,666][20381] Updated weights for policy 1, policy_version 77314 (0.0006) [2023-12-27 06:11:53,259][20381] Updated weights for policy 1, policy_version 77324 (0.0009) [2023-12-27 06:11:53,320][20381] Updated weights for policy 1, policy_version 77334 (0.0005) [2023-12-27 06:11:53,365][20347] Updated weights for policy 0, policy_version 77537 (0.0006) [2023-12-27 06:11:53,368][20381] Updated weights for policy 1, policy_version 77344 (0.0010) [2023-12-27 06:11:53,416][20347] Updated weights for policy 0, policy_version 77547 (0.0005) [2023-12-27 06:11:53,472][20347] Updated weights for policy 0, policy_version 77557 (0.0007) [2023-12-27 06:11:54,006][20381] Updated weights for policy 1, policy_version 77354 (0.0010) [2023-12-27 06:11:54,052][20381] Updated weights for policy 1, policy_version 77364 (0.0007) [2023-12-27 06:11:54,101][20381] Updated weights for policy 1, policy_version 77374 (0.0006) [2023-12-27 06:11:54,154][20381] Updated weights for policy 1, policy_version 77384 (0.0008) [2023-12-27 06:11:54,171][20347] Updated weights for policy 0, policy_version 77567 (0.0007) [2023-12-27 06:11:54,218][20347] Updated weights for policy 0, policy_version 77577 (0.0007) [2023-12-27 06:11:54,265][20347] Updated weights for policy 0, policy_version 77587 (0.0007) [2023-12-27 06:11:54,802][20381] Updated weights for policy 1, policy_version 77394 (0.0008) [2023-12-27 06:11:54,856][20381] Updated weights for policy 1, policy_version 77404 (0.0010) [2023-12-27 06:11:54,926][20381] Updated weights for policy 1, policy_version 77414 (0.0010) [2023-12-27 06:11:54,996][20347] Updated weights for policy 0, policy_version 77597 (0.0007) [2023-12-27 06:11:55,049][20347] Updated weights for policy 0, policy_version 77607 (0.0008) [2023-12-27 06:11:55,096][20347] Updated weights for policy 0, policy_version 77617 (0.0006) [2023-12-27 06:11:55,499][20381] Updated weights for policy 1, policy_version 77424 (0.0006) [2023-12-27 06:11:55,550][20381] Updated weights for policy 1, policy_version 77434 (0.0005) [2023-12-27 06:11:55,600][20381] Updated weights for policy 1, policy_version 77444 (0.0005) [2023-12-27 06:11:55,873][20347] Updated weights for policy 0, policy_version 77627 (0.0008) [2023-12-27 06:11:55,929][20347] Updated weights for policy 0, policy_version 77637 (0.0007) [2023-12-27 06:11:55,983][20347] Updated weights for policy 0, policy_version 77647 (0.0007) [2023-12-27 06:11:56,178][20381] Updated weights for policy 1, policy_version 77454 (0.0008) [2023-12-27 06:11:56,233][20381] Updated weights for policy 1, policy_version 77464 (0.0010) [2023-12-27 06:11:56,278][20381] Updated weights for policy 1, policy_version 77474 (0.0010) [2023-12-27 06:11:56,694][20347] Updated weights for policy 0, policy_version 77657 (0.0009) [2023-12-27 06:11:56,750][20347] Updated weights for policy 0, policy_version 77667 (0.0007) [2023-12-27 06:11:56,802][20347] Updated weights for policy 0, policy_version 77677 (0.0007) [2023-12-27 06:11:56,858][20347] Updated weights for policy 0, policy_version 77687 (0.0005) [2023-12-27 06:11:56,942][20381] Updated weights for policy 1, policy_version 77484 (0.0008) [2023-12-27 06:11:56,988][20381] Updated weights for policy 1, policy_version 77494 (0.0005) [2023-12-27 06:11:57,042][20381] Updated weights for policy 1, policy_version 77504 (0.0006) [2023-12-27 06:11:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20617.7, 300 sec: 20799.4). Total num frames: 39747584. Throughput: 0: 10467.1, 1: 10210.6. Samples: 39737444. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:11:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:11:57,508][20347] Updated weights for policy 0, policy_version 77697 (0.0007) [2023-12-27 06:11:57,558][20347] Updated weights for policy 0, policy_version 77707 (0.0007) [2023-12-27 06:11:57,609][20381] Updated weights for policy 1, policy_version 77514 (0.0005) [2023-12-27 06:11:57,616][20347] Updated weights for policy 0, policy_version 77717 (0.0005) [2023-12-27 06:11:57,658][20381] Updated weights for policy 1, policy_version 77524 (0.0005) [2023-12-27 06:11:57,713][20381] Updated weights for policy 1, policy_version 77534 (0.0005) [2023-12-27 06:11:57,761][20381] Updated weights for policy 1, policy_version 77544 (0.0005) [2023-12-27 06:11:58,299][20381] Updated weights for policy 1, policy_version 77554 (0.0010) [2023-12-27 06:11:58,300][20347] Updated weights for policy 0, policy_version 77727 (0.0007) [2023-12-27 06:11:58,364][20381] Updated weights for policy 1, policy_version 77564 (0.0009) [2023-12-27 06:11:58,365][20347] Updated weights for policy 0, policy_version 77737 (0.0007) [2023-12-27 06:11:58,425][20381] Updated weights for policy 1, policy_version 77574 (0.0008) [2023-12-27 06:11:58,428][20347] Updated weights for policy 0, policy_version 77747 (0.0007) [2023-12-27 06:11:59,152][20347] Updated weights for policy 0, policy_version 77757 (0.0007) [2023-12-27 06:11:59,161][20381] Updated weights for policy 1, policy_version 77584 (0.0007) [2023-12-27 06:11:59,210][20347] Updated weights for policy 0, policy_version 77767 (0.0007) [2023-12-27 06:11:59,222][20381] Updated weights for policy 1, policy_version 77594 (0.0006) [2023-12-27 06:11:59,276][20347] Updated weights for policy 0, policy_version 77777 (0.0007) [2023-12-27 06:11:59,289][20381] Updated weights for policy 1, policy_version 77604 (0.0008) [2023-12-27 06:11:59,921][20381] Updated weights for policy 1, policy_version 77614 (0.0008) [2023-12-27 06:11:59,976][20381] Updated weights for policy 1, policy_version 77624 (0.0008) [2023-12-27 06:11:59,990][20347] Updated weights for policy 0, policy_version 77787 (0.0007) [2023-12-27 06:12:00,038][20381] Updated weights for policy 1, policy_version 77634 (0.0008) [2023-12-27 06:12:00,049][20347] Updated weights for policy 0, policy_version 77797 (0.0007) [2023-12-27 06:12:00,104][20347] Updated weights for policy 0, policy_version 77807 (0.0006) [2023-12-27 06:12:00,724][20347] Updated weights for policy 0, policy_version 77817 (0.0006) [2023-12-27 06:12:00,768][20381] Updated weights for policy 1, policy_version 77644 (0.0009) [2023-12-27 06:12:00,774][20347] Updated weights for policy 0, policy_version 77827 (0.0006) [2023-12-27 06:12:00,817][20381] Updated weights for policy 1, policy_version 77654 (0.0005) [2023-12-27 06:12:00,822][20347] Updated weights for policy 0, policy_version 77837 (0.0005) [2023-12-27 06:12:00,870][20347] Updated weights for policy 0, policy_version 77847 (0.0006) [2023-12-27 06:12:00,871][20381] Updated weights for policy 1, policy_version 77664 (0.0006) [2023-12-27 06:12:01,551][20347] Updated weights for policy 0, policy_version 77857 (0.0005) [2023-12-27 06:12:01,561][20381] Updated weights for policy 1, policy_version 77674 (0.0008) [2023-12-27 06:12:01,609][20347] Updated weights for policy 0, policy_version 77867 (0.0005) [2023-12-27 06:12:01,621][20381] Updated weights for policy 1, policy_version 77684 (0.0009) [2023-12-27 06:12:01,666][20347] Updated weights for policy 0, policy_version 77877 (0.0007) [2023-12-27 06:12:01,684][20381] Updated weights for policy 1, policy_version 77694 (0.0008) [2023-12-27 06:12:01,743][20381] Updated weights for policy 1, policy_version 77704 (0.0009) [2023-12-27 06:12:02,325][20347] Updated weights for policy 0, policy_version 77887 (0.0007) [2023-12-27 06:12:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 39845888. Throughput: 0: 10482.0, 1: 10204.7. Samples: 39861812. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:02,387][20347] Updated weights for policy 0, policy_version 77897 (0.0008) [2023-12-27 06:12:02,400][20381] Updated weights for policy 1, policy_version 77714 (0.0006) [2023-12-27 06:12:02,443][20347] Updated weights for policy 0, policy_version 77907 (0.0006) [2023-12-27 06:12:02,458][20381] Updated weights for policy 1, policy_version 77724 (0.0006) [2023-12-27 06:12:02,518][20381] Updated weights for policy 1, policy_version 77734 (0.0005) [2023-12-27 06:12:03,127][20347] Updated weights for policy 0, policy_version 77917 (0.0006) [2023-12-27 06:12:03,137][20381] Updated weights for policy 1, policy_version 77744 (0.0007) [2023-12-27 06:12:03,182][20347] Updated weights for policy 0, policy_version 77927 (0.0005) [2023-12-27 06:12:03,191][20381] Updated weights for policy 1, policy_version 77754 (0.0007) [2023-12-27 06:12:03,242][20347] Updated weights for policy 0, policy_version 77937 (0.0006) [2023-12-27 06:12:03,258][20381] Updated weights for policy 1, policy_version 77764 (0.0006) [2023-12-27 06:12:03,880][20381] Updated weights for policy 1, policy_version 77774 (0.0006) [2023-12-27 06:12:03,923][20347] Updated weights for policy 0, policy_version 77947 (0.0006) [2023-12-27 06:12:03,933][20381] Updated weights for policy 1, policy_version 77784 (0.0008) [2023-12-27 06:12:03,983][20347] Updated weights for policy 0, policy_version 77957 (0.0005) [2023-12-27 06:12:03,990][20381] Updated weights for policy 1, policy_version 77794 (0.0009) [2023-12-27 06:12:04,044][20347] Updated weights for policy 0, policy_version 77967 (0.0006) [2023-12-27 06:12:04,614][20381] Updated weights for policy 1, policy_version 77804 (0.0007) [2023-12-27 06:12:04,678][20381] Updated weights for policy 1, policy_version 77814 (0.0005) [2023-12-27 06:12:04,705][20347] Updated weights for policy 0, policy_version 77977 (0.0005) [2023-12-27 06:12:04,736][20381] Updated weights for policy 1, policy_version 77824 (0.0008) [2023-12-27 06:12:04,766][20347] Updated weights for policy 0, policy_version 77987 (0.0007) [2023-12-27 06:12:04,819][20347] Updated weights for policy 0, policy_version 77997 (0.0006) [2023-12-27 06:12:04,876][20347] Updated weights for policy 0, policy_version 78007 (0.0008) [2023-12-27 06:12:05,387][20381] Updated weights for policy 1, policy_version 77834 (0.0009) [2023-12-27 06:12:05,434][20381] Updated weights for policy 1, policy_version 77844 (0.0007) [2023-12-27 06:12:05,487][20381] Updated weights for policy 1, policy_version 77854 (0.0007) [2023-12-27 06:12:05,518][20347] Updated weights for policy 0, policy_version 78017 (0.0005) [2023-12-27 06:12:05,547][20381] Updated weights for policy 1, policy_version 77864 (0.0006) [2023-12-27 06:12:05,571][20347] Updated weights for policy 0, policy_version 78027 (0.0005) [2023-12-27 06:12:05,620][20347] Updated weights for policy 0, policy_version 78037 (0.0005) [2023-12-27 06:12:06,168][20347] Updated weights for policy 0, policy_version 78047 (0.0006) [2023-12-27 06:12:06,236][20347] Updated weights for policy 0, policy_version 78057 (0.0009) [2023-12-27 06:12:06,240][20381] Updated weights for policy 1, policy_version 77874 (0.0008) [2023-12-27 06:12:06,291][20381] Updated weights for policy 1, policy_version 77884 (0.0006) [2023-12-27 06:12:06,292][20347] Updated weights for policy 0, policy_version 78067 (0.0007) [2023-12-27 06:12:06,349][20381] Updated weights for policy 1, policy_version 77894 (0.0007) [2023-12-27 06:12:06,931][20347] Updated weights for policy 0, policy_version 78077 (0.0009) [2023-12-27 06:12:06,974][20381] Updated weights for policy 1, policy_version 77904 (0.0008) [2023-12-27 06:12:06,985][20347] Updated weights for policy 0, policy_version 78087 (0.0005) [2023-12-27 06:12:07,020][20381] Updated weights for policy 1, policy_version 77914 (0.0007) [2023-12-27 06:12:07,043][20347] Updated weights for policy 0, policy_version 78097 (0.0005) [2023-12-27 06:12:07,069][20381] Updated weights for policy 1, policy_version 77924 (0.0007) [2023-12-27 06:12:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 39960576. Throughput: 0: 10446.7, 1: 10249.6. Samples: 39924492. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000077928_19955712.pth... [2023-12-27 06:12:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000078104_20004864.pth... [2023-12-27 06:12:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000076712_19644416.pth [2023-12-27 06:12:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000076888_19693568.pth [2023-12-27 06:12:07,650][20347] Updated weights for policy 0, policy_version 78107 (0.0005) [2023-12-27 06:12:07,704][20347] Updated weights for policy 0, policy_version 78117 (0.0005) [2023-12-27 06:12:07,761][20347] Updated weights for policy 0, policy_version 78127 (0.0005) [2023-12-27 06:12:07,812][20381] Updated weights for policy 1, policy_version 77934 (0.0006) [2023-12-27 06:12:07,857][20381] Updated weights for policy 1, policy_version 77944 (0.0008) [2023-12-27 06:12:07,907][20381] Updated weights for policy 1, policy_version 77954 (0.0005) [2023-12-27 06:12:08,334][20347] Updated weights for policy 0, policy_version 78137 (0.0006) [2023-12-27 06:12:08,396][20347] Updated weights for policy 0, policy_version 78147 (0.0008) [2023-12-27 06:12:08,460][20347] Updated weights for policy 0, policy_version 78157 (0.0006) [2023-12-27 06:12:08,514][20347] Updated weights for policy 0, policy_version 78167 (0.0005) [2023-12-27 06:12:08,593][20381] Updated weights for policy 1, policy_version 77964 (0.0010) [2023-12-27 06:12:08,648][20381] Updated weights for policy 1, policy_version 77974 (0.0011) [2023-12-27 06:12:08,710][20381] Updated weights for policy 1, policy_version 77984 (0.0011) [2023-12-27 06:12:09,140][20347] Updated weights for policy 0, policy_version 78177 (0.0007) [2023-12-27 06:12:09,199][20347] Updated weights for policy 0, policy_version 78187 (0.0008) [2023-12-27 06:12:09,257][20347] Updated weights for policy 0, policy_version 78197 (0.0008) [2023-12-27 06:12:09,414][20381] Updated weights for policy 1, policy_version 77994 (0.0010) [2023-12-27 06:12:09,470][20381] Updated weights for policy 1, policy_version 78004 (0.0011) [2023-12-27 06:12:09,524][20381] Updated weights for policy 1, policy_version 78014 (0.0010) [2023-12-27 06:12:09,578][20381] Updated weights for policy 1, policy_version 78024 (0.0010) [2023-12-27 06:12:10,066][20347] Updated weights for policy 0, policy_version 78207 (0.0007) [2023-12-27 06:12:10,125][20347] Updated weights for policy 0, policy_version 78217 (0.0008) [2023-12-27 06:12:10,183][20347] Updated weights for policy 0, policy_version 78227 (0.0008) [2023-12-27 06:12:10,298][20381] Updated weights for policy 1, policy_version 78034 (0.0011) [2023-12-27 06:12:10,355][20381] Updated weights for policy 1, policy_version 78044 (0.0011) [2023-12-27 06:12:10,416][20381] Updated weights for policy 1, policy_version 78054 (0.0010) [2023-12-27 06:12:10,871][20347] Updated weights for policy 0, policy_version 78237 (0.0008) [2023-12-27 06:12:10,925][20347] Updated weights for policy 0, policy_version 78247 (0.0010) [2023-12-27 06:12:10,983][20347] Updated weights for policy 0, policy_version 78257 (0.0010) [2023-12-27 06:12:11,071][20381] Updated weights for policy 1, policy_version 78064 (0.0008) [2023-12-27 06:12:11,122][20381] Updated weights for policy 1, policy_version 78074 (0.0006) [2023-12-27 06:12:11,181][20381] Updated weights for policy 1, policy_version 78084 (0.0009) [2023-12-27 06:12:11,710][20347] Updated weights for policy 0, policy_version 78267 (0.0009) [2023-12-27 06:12:11,784][20347] Updated weights for policy 0, policy_version 78277 (0.0009) [2023-12-27 06:12:11,849][20347] Updated weights for policy 0, policy_version 78287 (0.0006) [2023-12-27 06:12:11,916][20381] Updated weights for policy 1, policy_version 78094 (0.0006) [2023-12-27 06:12:11,976][20381] Updated weights for policy 1, policy_version 78104 (0.0007) [2023-12-27 06:12:12,040][20381] Updated weights for policy 1, policy_version 78114 (0.0006) [2023-12-27 06:12:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 40058880. Throughput: 0: 10486.3, 1: 10222.0. Samples: 40049468. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:12,524][20347] Updated weights for policy 0, policy_version 78297 (0.0010) [2023-12-27 06:12:12,584][20347] Updated weights for policy 0, policy_version 78307 (0.0010) [2023-12-27 06:12:12,643][20347] Updated weights for policy 0, policy_version 78317 (0.0010) [2023-12-27 06:12:12,689][20381] Updated weights for policy 1, policy_version 78124 (0.0006) [2023-12-27 06:12:12,699][20347] Updated weights for policy 0, policy_version 78327 (0.0010) [2023-12-27 06:12:12,749][20381] Updated weights for policy 1, policy_version 78134 (0.0006) [2023-12-27 06:12:12,803][20381] Updated weights for policy 1, policy_version 78144 (0.0008) [2023-12-27 06:12:13,318][20347] Updated weights for policy 0, policy_version 78337 (0.0010) [2023-12-27 06:12:13,363][20347] Updated weights for policy 0, policy_version 78347 (0.0010) [2023-12-27 06:12:13,421][20347] Updated weights for policy 0, policy_version 78357 (0.0010) [2023-12-27 06:12:13,438][20381] Updated weights for policy 1, policy_version 78154 (0.0007) [2023-12-27 06:12:13,483][20381] Updated weights for policy 1, policy_version 78164 (0.0007) [2023-12-27 06:12:13,529][20381] Updated weights for policy 1, policy_version 78174 (0.0007) [2023-12-27 06:12:13,579][20381] Updated weights for policy 1, policy_version 78184 (0.0007) [2023-12-27 06:12:14,143][20347] Updated weights for policy 0, policy_version 78367 (0.0009) [2023-12-27 06:12:14,201][20347] Updated weights for policy 0, policy_version 78377 (0.0008) [2023-12-27 06:12:14,263][20347] Updated weights for policy 0, policy_version 78387 (0.0007) [2023-12-27 06:12:14,301][20381] Updated weights for policy 1, policy_version 78194 (0.0006) [2023-12-27 06:12:14,357][20381] Updated weights for policy 1, policy_version 78204 (0.0007) [2023-12-27 06:12:14,424][20381] Updated weights for policy 1, policy_version 78214 (0.0005) [2023-12-27 06:12:14,995][20381] Updated weights for policy 1, policy_version 78224 (0.0007) [2023-12-27 06:12:15,013][20347] Updated weights for policy 0, policy_version 78397 (0.0007) [2023-12-27 06:12:15,057][20381] Updated weights for policy 1, policy_version 78234 (0.0007) [2023-12-27 06:12:15,067][20347] Updated weights for policy 0, policy_version 78407 (0.0006) [2023-12-27 06:12:15,116][20381] Updated weights for policy 1, policy_version 78244 (0.0006) [2023-12-27 06:12:15,127][20347] Updated weights for policy 0, policy_version 78417 (0.0007) [2023-12-27 06:12:15,778][20347] Updated weights for policy 0, policy_version 78427 (0.0007) [2023-12-27 06:12:15,821][20381] Updated weights for policy 1, policy_version 78254 (0.0008) [2023-12-27 06:12:15,838][20347] Updated weights for policy 0, policy_version 78437 (0.0006) [2023-12-27 06:12:15,878][20381] Updated weights for policy 1, policy_version 78264 (0.0008) [2023-12-27 06:12:15,898][20347] Updated weights for policy 0, policy_version 78447 (0.0006) [2023-12-27 06:12:15,938][20381] Updated weights for policy 1, policy_version 78274 (0.0007) [2023-12-27 06:12:16,565][20347] Updated weights for policy 0, policy_version 78457 (0.0006) [2023-12-27 06:12:16,605][20381] Updated weights for policy 1, policy_version 78284 (0.0007) [2023-12-27 06:12:16,614][20347] Updated weights for policy 0, policy_version 78467 (0.0007) [2023-12-27 06:12:16,654][20381] Updated weights for policy 1, policy_version 78294 (0.0008) [2023-12-27 06:12:16,667][20347] Updated weights for policy 0, policy_version 78477 (0.0007) [2023-12-27 06:12:16,707][20381] Updated weights for policy 1, policy_version 78304 (0.0005) [2023-12-27 06:12:16,726][20347] Updated weights for policy 0, policy_version 78487 (0.0007) [2023-12-27 06:12:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 40157184. Throughput: 0: 10350.7, 1: 10282.5. Samples: 40172152. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:17,377][20381] Updated weights for policy 1, policy_version 78314 (0.0008) [2023-12-27 06:12:17,422][20347] Updated weights for policy 0, policy_version 78497 (0.0009) [2023-12-27 06:12:17,426][20381] Updated weights for policy 1, policy_version 78324 (0.0010) [2023-12-27 06:12:17,467][20347] Updated weights for policy 0, policy_version 78507 (0.0007) [2023-12-27 06:12:17,481][20381] Updated weights for policy 1, policy_version 78334 (0.0010) [2023-12-27 06:12:17,518][20347] Updated weights for policy 0, policy_version 78517 (0.0008) [2023-12-27 06:12:17,530][20381] Updated weights for policy 1, policy_version 78344 (0.0010) [2023-12-27 06:12:18,123][20347] Updated weights for policy 0, policy_version 78527 (0.0006) [2023-12-27 06:12:18,180][20347] Updated weights for policy 0, policy_version 78537 (0.0007) [2023-12-27 06:12:18,220][20381] Updated weights for policy 1, policy_version 78354 (0.0010) [2023-12-27 06:12:18,233][20347] Updated weights for policy 0, policy_version 78547 (0.0006) [2023-12-27 06:12:18,269][20381] Updated weights for policy 1, policy_version 78364 (0.0010) [2023-12-27 06:12:18,323][20381] Updated weights for policy 1, policy_version 78374 (0.0011) [2023-12-27 06:12:18,913][20347] Updated weights for policy 0, policy_version 78557 (0.0007) [2023-12-27 06:12:18,967][20347] Updated weights for policy 0, policy_version 78567 (0.0007) [2023-12-27 06:12:19,022][20347] Updated weights for policy 0, policy_version 78577 (0.0006) [2023-12-27 06:12:19,035][20381] Updated weights for policy 1, policy_version 78384 (0.0010) [2023-12-27 06:12:19,085][20381] Updated weights for policy 1, policy_version 78394 (0.0010) [2023-12-27 06:12:19,133][20381] Updated weights for policy 1, policy_version 78404 (0.0010) [2023-12-27 06:12:19,757][20347] Updated weights for policy 0, policy_version 78587 (0.0006) [2023-12-27 06:12:19,809][20347] Updated weights for policy 0, policy_version 78597 (0.0008) [2023-12-27 06:12:19,815][20381] Updated weights for policy 1, policy_version 78414 (0.0011) [2023-12-27 06:12:19,872][20347] Updated weights for policy 0, policy_version 78607 (0.0007) [2023-12-27 06:12:19,878][20381] Updated weights for policy 1, policy_version 78424 (0.0008) [2023-12-27 06:12:19,927][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000002 [2023-12-27 06:12:20,591][20347] Updated weights for policy 0, policy_version 78617 (0.0006) [2023-12-27 06:12:20,601][20381] Updated weights for policy 1, policy_version 78434 (0.0010) [2023-12-27 06:12:20,651][20347] Updated weights for policy 0, policy_version 78627 (0.0006) [2023-12-27 06:12:20,659][20381] Updated weights for policy 1, policy_version 78444 (0.0008) [2023-12-27 06:12:20,710][20347] Updated weights for policy 0, policy_version 78637 (0.0009) [2023-12-27 06:12:20,713][20381] Updated weights for policy 1, policy_version 78454 (0.0007) [2023-12-27 06:12:20,760][20381] Updated weights for policy 1, policy_version 78464 (0.0006) [2023-12-27 06:12:20,772][20347] Updated weights for policy 0, policy_version 78647 (0.0009) [2023-12-27 06:12:21,500][20381] Updated weights for policy 1, policy_version 78474 (0.0009) [2023-12-27 06:12:21,532][20347] Updated weights for policy 0, policy_version 78657 (0.0007) [2023-12-27 06:12:21,558][20381] Updated weights for policy 1, policy_version 78484 (0.0009) [2023-12-27 06:12:21,583][20347] Updated weights for policy 0, policy_version 78667 (0.0006) [2023-12-27 06:12:21,618][20381] Updated weights for policy 1, policy_version 78494 (0.0009) [2023-12-27 06:12:21,638][20347] Updated weights for policy 0, policy_version 78677 (0.0007) [2023-12-27 06:12:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 40255488. Throughput: 0: 10324.4, 1: 10288.2. Samples: 40233108. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:22,377][20347] Updated weights for policy 0, policy_version 78687 (0.0007) [2023-12-27 06:12:22,392][20381] Updated weights for policy 1, policy_version 78504 (0.0008) [2023-12-27 06:12:22,440][20347] Updated weights for policy 0, policy_version 78697 (0.0007) [2023-12-27 06:12:22,451][20381] Updated weights for policy 1, policy_version 78514 (0.0006) [2023-12-27 06:12:22,500][20347] Updated weights for policy 0, policy_version 78707 (0.0007) [2023-12-27 06:12:22,513][20381] Updated weights for policy 1, policy_version 78524 (0.0006) [2023-12-27 06:12:22,530][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000078712_20160512.pth... [2023-12-27 06:12:22,534][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000077496_19849216.pth [2023-12-27 06:12:22,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000078528_20111360.pth... [2023-12-27 06:12:22,541][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000077288_19791872.pth [2023-12-27 06:12:23,141][20347] Updated weights for policy 0, policy_version 78717 (0.0007) [2023-12-27 06:12:23,197][20381] Updated weights for policy 1, policy_version 78534 (0.0006) [2023-12-27 06:12:23,200][20347] Updated weights for policy 0, policy_version 78727 (0.0005) [2023-12-27 06:12:23,247][20381] Updated weights for policy 1, policy_version 78544 (0.0005) [2023-12-27 06:12:23,251][20347] Updated weights for policy 0, policy_version 78737 (0.0010) [2023-12-27 06:12:23,308][20381] Updated weights for policy 1, policy_version 78554 (0.0006) [2023-12-27 06:12:23,826][20347] Updated weights for policy 0, policy_version 78747 (0.0010) [2023-12-27 06:12:23,879][20347] Updated weights for policy 0, policy_version 78757 (0.0008) [2023-12-27 06:12:23,880][20381] Updated weights for policy 1, policy_version 78564 (0.0010) [2023-12-27 06:12:23,932][20381] Updated weights for policy 1, policy_version 78574 (0.0010) [2023-12-27 06:12:23,933][20347] Updated weights for policy 0, policy_version 78767 (0.0006) [2023-12-27 06:12:23,984][20381] Updated weights for policy 1, policy_version 78584 (0.0007) [2023-12-27 06:12:24,497][20347] Updated weights for policy 0, policy_version 78777 (0.0006) [2023-12-27 06:12:24,547][20347] Updated weights for policy 0, policy_version 78787 (0.0010) [2023-12-27 06:12:24,592][20347] Updated weights for policy 0, policy_version 78797 (0.0010) [2023-12-27 06:12:24,611][20381] Updated weights for policy 1, policy_version 78594 (0.0007) [2023-12-27 06:12:24,644][20347] Updated weights for policy 0, policy_version 78807 (0.0010) [2023-12-27 06:12:24,662][20381] Updated weights for policy 1, policy_version 78604 (0.0010) [2023-12-27 06:12:24,717][20381] Updated weights for policy 1, policy_version 78614 (0.0010) [2023-12-27 06:12:24,781][20381] Updated weights for policy 1, policy_version 78624 (0.0010) [2023-12-27 06:12:25,298][20347] Updated weights for policy 0, policy_version 78817 (0.0010) [2023-12-27 06:12:25,346][20347] Updated weights for policy 0, policy_version 78827 (0.0010) [2023-12-27 06:12:25,399][20347] Updated weights for policy 0, policy_version 78837 (0.0010) [2023-12-27 06:12:25,467][20381] Updated weights for policy 1, policy_version 78634 (0.0010) [2023-12-27 06:12:25,528][20381] Updated weights for policy 1, policy_version 78644 (0.0010) [2023-12-27 06:12:25,584][20381] Updated weights for policy 1, policy_version 78654 (0.0011) [2023-12-27 06:12:26,011][20347] Updated weights for policy 0, policy_version 78847 (0.0010) [2023-12-27 06:12:26,070][20347] Updated weights for policy 0, policy_version 78857 (0.0010) [2023-12-27 06:12:26,118][20347] Updated weights for policy 0, policy_version 78867 (0.0010) [2023-12-27 06:12:26,270][20381] Updated weights for policy 1, policy_version 78664 (0.0011) [2023-12-27 06:12:26,318][20381] Updated weights for policy 1, policy_version 78674 (0.0010) [2023-12-27 06:12:26,369][20381] Updated weights for policy 1, policy_version 78684 (0.0007) [2023-12-27 06:12:26,753][20347] Updated weights for policy 0, policy_version 78877 (0.0010) [2023-12-27 06:12:26,816][20347] Updated weights for policy 0, policy_version 78887 (0.0010) [2023-12-27 06:12:26,872][20347] Updated weights for policy 0, policy_version 78897 (0.0010) [2023-12-27 06:12:26,921][20381] Updated weights for policy 1, policy_version 78694 (0.0006) [2023-12-27 06:12:26,979][20381] Updated weights for policy 1, policy_version 78704 (0.0005) [2023-12-27 06:12:27,046][20381] Updated weights for policy 1, policy_version 78714 (0.0005) [2023-12-27 06:12:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 40370176. Throughput: 0: 10278.8, 1: 10365.5. Samples: 40357988. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:27,531][20347] Updated weights for policy 0, policy_version 78907 (0.0009) [2023-12-27 06:12:27,589][20347] Updated weights for policy 0, policy_version 78917 (0.0008) [2023-12-27 06:12:27,643][20347] Updated weights for policy 0, policy_version 78927 (0.0007) [2023-12-27 06:12:27,648][20381] Updated weights for policy 1, policy_version 78724 (0.0006) [2023-12-27 06:12:27,700][20381] Updated weights for policy 1, policy_version 78734 (0.0006) [2023-12-27 06:12:27,748][20381] Updated weights for policy 1, policy_version 78744 (0.0007) [2023-12-27 06:12:28,249][20347] Updated weights for policy 0, policy_version 78937 (0.0007) [2023-12-27 06:12:28,301][20347] Updated weights for policy 0, policy_version 78947 (0.0010) [2023-12-27 06:12:28,355][20381] Updated weights for policy 1, policy_version 78754 (0.0007) [2023-12-27 06:12:28,365][20347] Updated weights for policy 0, policy_version 78957 (0.0010) [2023-12-27 06:12:28,410][20381] Updated weights for policy 1, policy_version 78764 (0.0006) [2023-12-27 06:12:28,423][20347] Updated weights for policy 0, policy_version 78967 (0.0010) [2023-12-27 06:12:28,466][20381] Updated weights for policy 1, policy_version 78774 (0.0006) [2023-12-27 06:12:28,519][20381] Updated weights for policy 1, policy_version 78784 (0.0008) [2023-12-27 06:12:29,040][20347] Updated weights for policy 0, policy_version 78977 (0.0006) [2023-12-27 06:12:29,098][20347] Updated weights for policy 0, policy_version 78987 (0.0005) [2023-12-27 06:12:29,141][20381] Updated weights for policy 1, policy_version 78794 (0.0006) [2023-12-27 06:12:29,153][20347] Updated weights for policy 0, policy_version 78997 (0.0005) [2023-12-27 06:12:29,185][20381] Updated weights for policy 1, policy_version 78804 (0.0009) [2023-12-27 06:12:29,237][20381] Updated weights for policy 1, policy_version 78814 (0.0008) [2023-12-27 06:12:29,807][20347] Updated weights for policy 0, policy_version 79007 (0.0009) [2023-12-27 06:12:29,868][20347] Updated weights for policy 0, policy_version 79017 (0.0011) [2023-12-27 06:12:29,924][20381] Updated weights for policy 1, policy_version 78824 (0.0006) [2023-12-27 06:12:29,930][20347] Updated weights for policy 0, policy_version 79027 (0.0010) [2023-12-27 06:12:29,986][20381] Updated weights for policy 1, policy_version 78834 (0.0005) [2023-12-27 06:12:30,047][20381] Updated weights for policy 1, policy_version 78844 (0.0007) [2023-12-27 06:12:30,595][20347] Updated weights for policy 0, policy_version 79037 (0.0007) [2023-12-27 06:12:30,646][20347] Updated weights for policy 0, policy_version 79047 (0.0007) [2023-12-27 06:12:30,701][20347] Updated weights for policy 0, policy_version 79057 (0.0008) [2023-12-27 06:12:30,737][20381] Updated weights for policy 1, policy_version 78854 (0.0007) [2023-12-27 06:12:30,780][20381] Updated weights for policy 1, policy_version 78864 (0.0007) [2023-12-27 06:12:30,830][20381] Updated weights for policy 1, policy_version 78874 (0.0007) [2023-12-27 06:12:31,400][20347] Updated weights for policy 0, policy_version 79067 (0.0008) [2023-12-27 06:12:31,450][20347] Updated weights for policy 0, policy_version 79077 (0.0005) [2023-12-27 06:12:31,501][20347] Updated weights for policy 0, policy_version 79087 (0.0007) [2023-12-27 06:12:31,529][20381] Updated weights for policy 1, policy_version 78884 (0.0007) [2023-12-27 06:12:31,581][20381] Updated weights for policy 1, policy_version 78894 (0.0007) [2023-12-27 06:12:31,642][20381] Updated weights for policy 1, policy_version 78904 (0.0007) [2023-12-27 06:12:32,138][20347] Updated weights for policy 0, policy_version 79097 (0.0007) [2023-12-27 06:12:32,184][20347] Updated weights for policy 0, policy_version 79107 (0.0008) [2023-12-27 06:12:32,237][20347] Updated weights for policy 0, policy_version 79117 (0.0007) [2023-12-27 06:12:32,295][20347] Updated weights for policy 0, policy_version 79127 (0.0008) [2023-12-27 06:12:32,346][20381] Updated weights for policy 1, policy_version 78914 (0.0007) [2023-12-27 06:12:32,362][19170] Fps is (10 sec: 22118.8, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 40476672. Throughput: 0: 10326.3, 1: 10439.0. Samples: 40486052. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:32,412][20381] Updated weights for policy 1, policy_version 78924 (0.0008) [2023-12-27 06:12:32,479][20381] Updated weights for policy 1, policy_version 78934 (0.0006) [2023-12-27 06:12:32,532][20381] Updated weights for policy 1, policy_version 78944 (0.0007) [2023-12-27 06:12:32,988][20347] Updated weights for policy 0, policy_version 79137 (0.0007) [2023-12-27 06:12:33,038][20347] Updated weights for policy 0, policy_version 79147 (0.0007) [2023-12-27 06:12:33,100][20347] Updated weights for policy 0, policy_version 79157 (0.0008) [2023-12-27 06:12:33,153][20381] Updated weights for policy 1, policy_version 78954 (0.0006) [2023-12-27 06:12:33,207][20381] Updated weights for policy 1, policy_version 78964 (0.0005) [2023-12-27 06:12:33,258][20381] Updated weights for policy 1, policy_version 78974 (0.0005) [2023-12-27 06:12:33,795][20347] Updated weights for policy 0, policy_version 79167 (0.0008) [2023-12-27 06:12:33,844][20347] Updated weights for policy 0, policy_version 79177 (0.0007) [2023-12-27 06:12:33,854][20381] Updated weights for policy 1, policy_version 78984 (0.0005) [2023-12-27 06:12:33,894][20347] Updated weights for policy 0, policy_version 79187 (0.0006) [2023-12-27 06:12:33,899][20381] Updated weights for policy 1, policy_version 78994 (0.0006) [2023-12-27 06:12:33,953][20381] Updated weights for policy 1, policy_version 79004 (0.0005) [2023-12-27 06:12:34,526][20381] Updated weights for policy 1, policy_version 79014 (0.0007) [2023-12-27 06:12:34,582][20381] Updated weights for policy 1, policy_version 79024 (0.0007) [2023-12-27 06:12:34,629][20347] Updated weights for policy 0, policy_version 79197 (0.0006) [2023-12-27 06:12:34,640][20381] Updated weights for policy 1, policy_version 79034 (0.0008) [2023-12-27 06:12:34,684][20347] Updated weights for policy 0, policy_version 79207 (0.0007) [2023-12-27 06:12:34,740][20347] Updated weights for policy 0, policy_version 79217 (0.0007) [2023-12-27 06:12:35,353][20381] Updated weights for policy 1, policy_version 79044 (0.0006) [2023-12-27 06:12:35,397][20347] Updated weights for policy 0, policy_version 79227 (0.0007) [2023-12-27 06:12:35,414][20381] Updated weights for policy 1, policy_version 79054 (0.0008) [2023-12-27 06:12:35,453][20347] Updated weights for policy 0, policy_version 79237 (0.0005) [2023-12-27 06:12:35,476][20381] Updated weights for policy 1, policy_version 79064 (0.0007) [2023-12-27 06:12:35,506][20347] Updated weights for policy 0, policy_version 79247 (0.0005) [2023-12-27 06:12:36,099][20347] Updated weights for policy 0, policy_version 79257 (0.0007) [2023-12-27 06:12:36,158][20347] Updated weights for policy 0, policy_version 79267 (0.0007) [2023-12-27 06:12:36,167][20381] Updated weights for policy 1, policy_version 79074 (0.0006) [2023-12-27 06:12:36,221][20347] Updated weights for policy 0, policy_version 79277 (0.0006) [2023-12-27 06:12:36,223][20381] Updated weights for policy 1, policy_version 79084 (0.0008) [2023-12-27 06:12:36,272][20347] Updated weights for policy 0, policy_version 79287 (0.0005) [2023-12-27 06:12:36,279][20381] Updated weights for policy 1, policy_version 79094 (0.0007) [2023-12-27 06:12:36,327][20381] Updated weights for policy 1, policy_version 79104 (0.0008) [2023-12-27 06:12:36,915][20347] Updated weights for policy 0, policy_version 79297 (0.0010) [2023-12-27 06:12:36,975][20347] Updated weights for policy 0, policy_version 79307 (0.0009) [2023-12-27 06:12:37,034][20347] Updated weights for policy 0, policy_version 79317 (0.0008) [2023-12-27 06:12:37,059][20381] Updated weights for policy 1, policy_version 79114 (0.0006) [2023-12-27 06:12:37,124][20381] Updated weights for policy 1, policy_version 79124 (0.0006) [2023-12-27 06:12:37,188][20381] Updated weights for policy 1, policy_version 79134 (0.0007) [2023-12-27 06:12:37,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 40583168. Throughput: 0: 10290.2, 1: 10489.5. Samples: 40548632. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000079136_20267008.pth... [2023-12-27 06:12:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000079320_20316160.pth... [2023-12-27 06:12:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000077928_19955712.pth [2023-12-27 06:12:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000078104_20004864.pth [2023-12-27 06:12:37,646][20347] Updated weights for policy 0, policy_version 79327 (0.0007) [2023-12-27 06:12:37,706][20347] Updated weights for policy 0, policy_version 79337 (0.0006) [2023-12-27 06:12:37,730][20381] Updated weights for policy 1, policy_version 79144 (0.0006) [2023-12-27 06:12:37,754][20347] Updated weights for policy 0, policy_version 79347 (0.0006) [2023-12-27 06:12:37,792][20381] Updated weights for policy 1, policy_version 79154 (0.0007) [2023-12-27 06:12:37,848][20381] Updated weights for policy 1, policy_version 79164 (0.0005) [2023-12-27 06:12:38,368][20347] Updated weights for policy 0, policy_version 79357 (0.0006) [2023-12-27 06:12:38,424][20347] Updated weights for policy 0, policy_version 79367 (0.0008) [2023-12-27 06:12:38,481][20347] Updated weights for policy 0, policy_version 79377 (0.0006) [2023-12-27 06:12:38,489][20381] Updated weights for policy 1, policy_version 79174 (0.0007) [2023-12-27 06:12:38,553][20381] Updated weights for policy 1, policy_version 79184 (0.0008) [2023-12-27 06:12:38,614][20381] Updated weights for policy 1, policy_version 79194 (0.0008) [2023-12-27 06:12:39,145][20347] Updated weights for policy 0, policy_version 79387 (0.0006) [2023-12-27 06:12:39,206][20347] Updated weights for policy 0, policy_version 79397 (0.0008) [2023-12-27 06:12:39,265][20347] Updated weights for policy 0, policy_version 79407 (0.0008) [2023-12-27 06:12:39,318][20381] Updated weights for policy 1, policy_version 79204 (0.0008) [2023-12-27 06:12:39,377][20381] Updated weights for policy 1, policy_version 79214 (0.0008) [2023-12-27 06:12:39,443][20381] Updated weights for policy 1, policy_version 79224 (0.0007) [2023-12-27 06:12:40,064][20347] Updated weights for policy 0, policy_version 79417 (0.0008) [2023-12-27 06:12:40,129][20347] Updated weights for policy 0, policy_version 79427 (0.0009) [2023-12-27 06:12:40,134][20381] Updated weights for policy 1, policy_version 79234 (0.0006) [2023-12-27 06:12:40,191][20347] Updated weights for policy 0, policy_version 79437 (0.0007) [2023-12-27 06:12:40,198][20381] Updated weights for policy 1, policy_version 79244 (0.0006) [2023-12-27 06:12:40,251][20381] Updated weights for policy 1, policy_version 79254 (0.0006) [2023-12-27 06:12:40,257][20347] Updated weights for policy 0, policy_version 79447 (0.0007) [2023-12-27 06:12:40,301][20381] Updated weights for policy 1, policy_version 79264 (0.0007) [2023-12-27 06:12:40,928][20347] Updated weights for policy 0, policy_version 79457 (0.0007) [2023-12-27 06:12:40,971][20381] Updated weights for policy 1, policy_version 79274 (0.0005) [2023-12-27 06:12:40,987][20347] Updated weights for policy 0, policy_version 79467 (0.0007) [2023-12-27 06:12:41,030][20381] Updated weights for policy 1, policy_version 79284 (0.0006) [2023-12-27 06:12:41,046][20347] Updated weights for policy 0, policy_version 79477 (0.0007) [2023-12-27 06:12:41,097][20381] Updated weights for policy 1, policy_version 79294 (0.0007) [2023-12-27 06:12:41,771][20381] Updated weights for policy 1, policy_version 79304 (0.0008) [2023-12-27 06:12:41,810][20347] Updated weights for policy 0, policy_version 79487 (0.0008) [2023-12-27 06:12:41,831][20381] Updated weights for policy 1, policy_version 79314 (0.0006) [2023-12-27 06:12:41,871][20347] Updated weights for policy 0, policy_version 79497 (0.0007) [2023-12-27 06:12:41,893][20381] Updated weights for policy 1, policy_version 79324 (0.0007) [2023-12-27 06:12:41,926][20347] Updated weights for policy 0, policy_version 79507 (0.0007) [2023-12-27 06:12:42,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 40681472. Throughput: 0: 10390.1, 1: 10396.8. Samples: 40672856. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-12-27 06:12:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:42,543][20381] Updated weights for policy 1, policy_version 79334 (0.0009) [2023-12-27 06:12:42,608][20381] Updated weights for policy 1, policy_version 79344 (0.0011) [2023-12-27 06:12:42,656][20347] Updated weights for policy 0, policy_version 79517 (0.0007) [2023-12-27 06:12:42,658][20381] Updated weights for policy 1, policy_version 79354 (0.0011) [2023-12-27 06:12:42,716][20347] Updated weights for policy 0, policy_version 79527 (0.0005) [2023-12-27 06:12:42,774][20347] Updated weights for policy 0, policy_version 79537 (0.0007) [2023-12-27 06:12:43,350][20381] Updated weights for policy 1, policy_version 79364 (0.0011) [2023-12-27 06:12:43,410][20381] Updated weights for policy 1, policy_version 79374 (0.0011) [2023-12-27 06:12:43,468][20381] Updated weights for policy 1, policy_version 79384 (0.0011) [2023-12-27 06:12:43,477][20347] Updated weights for policy 0, policy_version 79547 (0.0007) [2023-12-27 06:12:43,530][20347] Updated weights for policy 0, policy_version 79557 (0.0006) [2023-12-27 06:12:43,579][20347] Updated weights for policy 0, policy_version 79567 (0.0007) [2023-12-27 06:12:44,121][20381] Updated weights for policy 1, policy_version 79394 (0.0010) [2023-12-27 06:12:44,167][20381] Updated weights for policy 1, policy_version 79404 (0.0009) [2023-12-27 06:12:44,227][20381] Updated weights for policy 1, policy_version 79414 (0.0009) [2023-12-27 06:12:44,279][20381] Updated weights for policy 1, policy_version 79424 (0.0008) [2023-12-27 06:12:44,296][20347] Updated weights for policy 0, policy_version 79577 (0.0007) [2023-12-27 06:12:44,353][20347] Updated weights for policy 0, policy_version 79587 (0.0007) [2023-12-27 06:12:44,410][20347] Updated weights for policy 0, policy_version 79597 (0.0008) [2023-12-27 06:12:44,472][20347] Updated weights for policy 0, policy_version 79607 (0.0007) [2023-12-27 06:12:44,992][20381] Updated weights for policy 1, policy_version 79434 (0.0009) [2023-12-27 06:12:45,053][20381] Updated weights for policy 1, policy_version 79444 (0.0009) [2023-12-27 06:12:45,107][20381] Updated weights for policy 1, policy_version 79454 (0.0009) [2023-12-27 06:12:45,164][20347] Updated weights for policy 0, policy_version 79617 (0.0007) [2023-12-27 06:12:45,223][20347] Updated weights for policy 0, policy_version 79627 (0.0007) [2023-12-27 06:12:45,273][20347] Updated weights for policy 0, policy_version 79637 (0.0008) [2023-12-27 06:12:45,815][20381] Updated weights for policy 1, policy_version 79464 (0.0007) [2023-12-27 06:12:45,862][20381] Updated weights for policy 1, policy_version 79474 (0.0009) [2023-12-27 06:12:45,915][20381] Updated weights for policy 1, policy_version 79484 (0.0008) [2023-12-27 06:12:45,953][20347] Updated weights for policy 0, policy_version 79647 (0.0006) [2023-12-27 06:12:46,012][20347] Updated weights for policy 0, policy_version 79657 (0.0006) [2023-12-27 06:12:46,075][20347] Updated weights for policy 0, policy_version 79667 (0.0007) [2023-12-27 06:12:46,617][20381] Updated weights for policy 1, policy_version 79494 (0.0009) [2023-12-27 06:12:46,668][20381] Updated weights for policy 1, policy_version 79504 (0.0010) [2023-12-27 06:12:46,716][20381] Updated weights for policy 1, policy_version 79514 (0.0010) [2023-12-27 06:12:46,716][20347] Updated weights for policy 0, policy_version 79677 (0.0009) [2023-12-27 06:12:46,763][20347] Updated weights for policy 0, policy_version 79687 (0.0008) [2023-12-27 06:12:46,815][20347] Updated weights for policy 0, policy_version 79697 (0.0007) [2023-12-27 06:12:47,336][20381] Updated weights for policy 1, policy_version 79524 (0.0005) [2023-12-27 06:12:47,362][19170] Fps is (10 sec: 19661.5, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 40779776. Throughput: 0: 10353.6, 1: 10361.4. Samples: 40793988. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:12:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:47,399][20381] Updated weights for policy 1, policy_version 79534 (0.0005) [2023-12-27 06:12:47,456][20347] Updated weights for policy 0, policy_version 79707 (0.0006) [2023-12-27 06:12:47,457][20381] Updated weights for policy 1, policy_version 79544 (0.0007) [2023-12-27 06:12:47,509][20347] Updated weights for policy 0, policy_version 79717 (0.0007) [2023-12-27 06:12:47,562][20347] Updated weights for policy 0, policy_version 79727 (0.0007) [2023-12-27 06:12:48,075][20381] Updated weights for policy 1, policy_version 79554 (0.0008) [2023-12-27 06:12:48,133][20381] Updated weights for policy 1, policy_version 79564 (0.0011) [2023-12-27 06:12:48,187][20381] Updated weights for policy 1, policy_version 79574 (0.0010) [2023-12-27 06:12:48,239][20381] Updated weights for policy 1, policy_version 79584 (0.0009) [2023-12-27 06:12:48,284][20347] Updated weights for policy 0, policy_version 79737 (0.0007) [2023-12-27 06:12:48,349][20347] Updated weights for policy 0, policy_version 79747 (0.0007) [2023-12-27 06:12:48,412][20347] Updated weights for policy 0, policy_version 79757 (0.0008) [2023-12-27 06:12:48,471][20347] Updated weights for policy 0, policy_version 79767 (0.0007) [2023-12-27 06:12:48,905][20381] Updated weights for policy 1, policy_version 79594 (0.0005) [2023-12-27 06:12:48,956][20381] Updated weights for policy 1, policy_version 79604 (0.0005) [2023-12-27 06:12:49,013][20381] Updated weights for policy 1, policy_version 79614 (0.0005) [2023-12-27 06:12:49,172][20347] Updated weights for policy 0, policy_version 79777 (0.0010) [2023-12-27 06:12:49,228][20347] Updated weights for policy 0, policy_version 79787 (0.0010) [2023-12-27 06:12:49,285][20347] Updated weights for policy 0, policy_version 79797 (0.0006) [2023-12-27 06:12:49,598][20381] Updated weights for policy 1, policy_version 79624 (0.0010) [2023-12-27 06:12:49,653][20381] Updated weights for policy 1, policy_version 79634 (0.0011) [2023-12-27 06:12:49,705][20381] Updated weights for policy 1, policy_version 79644 (0.0011) [2023-12-27 06:12:49,967][20347] Updated weights for policy 0, policy_version 79807 (0.0009) [2023-12-27 06:12:50,034][20347] Updated weights for policy 0, policy_version 79817 (0.0010) [2023-12-27 06:12:50,098][20347] Updated weights for policy 0, policy_version 79827 (0.0008) [2023-12-27 06:12:50,373][20381] Updated weights for policy 1, policy_version 79654 (0.0011) [2023-12-27 06:12:50,436][20381] Updated weights for policy 1, policy_version 79664 (0.0011) [2023-12-27 06:12:50,481][20381] Updated weights for policy 1, policy_version 79674 (0.0010) [2023-12-27 06:12:50,755][20347] Updated weights for policy 0, policy_version 79837 (0.0009) [2023-12-27 06:12:50,808][20347] Updated weights for policy 0, policy_version 79847 (0.0007) [2023-12-27 06:12:50,862][20347] Updated weights for policy 0, policy_version 79857 (0.0007) [2023-12-27 06:12:51,179][20381] Updated weights for policy 1, policy_version 79684 (0.0010) [2023-12-27 06:12:51,238][20381] Updated weights for policy 1, policy_version 79694 (0.0007) [2023-12-27 06:12:51,294][20381] Updated weights for policy 1, policy_version 79704 (0.0008) [2023-12-27 06:12:51,560][20347] Updated weights for policy 0, policy_version 79867 (0.0007) [2023-12-27 06:12:51,627][20347] Updated weights for policy 0, policy_version 79877 (0.0008) [2023-12-27 06:12:51,680][20347] Updated weights for policy 0, policy_version 79887 (0.0005) [2023-12-27 06:12:52,024][20381] Updated weights for policy 1, policy_version 79714 (0.0007) [2023-12-27 06:12:52,084][20381] Updated weights for policy 1, policy_version 79724 (0.0008) [2023-12-27 06:12:52,141][20381] Updated weights for policy 1, policy_version 79734 (0.0010) [2023-12-27 06:12:52,197][20381] Updated weights for policy 1, policy_version 79744 (0.0009) [2023-12-27 06:12:52,290][20347] Updated weights for policy 0, policy_version 79897 (0.0006) [2023-12-27 06:12:52,344][20347] Updated weights for policy 0, policy_version 79907 (0.0009) [2023-12-27 06:12:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 40886272. Throughput: 0: 10354.3, 1: 10367.9. Samples: 40856988. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:12:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:12:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000079744_20422656.pth... [2023-12-27 06:12:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000078528_20111360.pth [2023-12-27 06:12:52,404][20347] Updated weights for policy 0, policy_version 79917 (0.0009) [2023-12-27 06:12:52,457][20347] Updated weights for policy 0, policy_version 79927 (0.0009) [2023-12-27 06:12:52,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000079928_20471808.pth... [2023-12-27 06:12:52,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000078712_20160512.pth [2023-12-27 06:12:53,024][20381] Updated weights for policy 1, policy_version 79754 (0.0011) [2023-12-27 06:12:53,075][20381] Updated weights for policy 1, policy_version 79764 (0.0010) [2023-12-27 06:12:53,130][20381] Updated weights for policy 1, policy_version 79774 (0.0010) [2023-12-27 06:12:53,236][20347] Updated weights for policy 0, policy_version 79937 (0.0007) [2023-12-27 06:12:53,288][20347] Updated weights for policy 0, policy_version 79947 (0.0008) [2023-12-27 06:12:53,335][20347] Updated weights for policy 0, policy_version 79957 (0.0007) [2023-12-27 06:12:53,809][20381] Updated weights for policy 1, policy_version 79784 (0.0010) [2023-12-27 06:12:53,859][20381] Updated weights for policy 1, policy_version 79794 (0.0008) [2023-12-27 06:12:53,917][20381] Updated weights for policy 1, policy_version 79804 (0.0009) [2023-12-27 06:12:54,052][20347] Updated weights for policy 0, policy_version 79967 (0.0006) [2023-12-27 06:12:54,111][20347] Updated weights for policy 0, policy_version 79977 (0.0005) [2023-12-27 06:12:54,165][20347] Updated weights for policy 0, policy_version 79987 (0.0005) [2023-12-27 06:12:54,596][20381] Updated weights for policy 1, policy_version 79814 (0.0006) [2023-12-27 06:12:54,658][20381] Updated weights for policy 1, policy_version 79824 (0.0005) [2023-12-27 06:12:54,717][20381] Updated weights for policy 1, policy_version 79834 (0.0008) [2023-12-27 06:12:54,722][20347] Updated weights for policy 0, policy_version 79997 (0.0005) [2023-12-27 06:12:54,766][20347] Updated weights for policy 0, policy_version 80007 (0.0006) [2023-12-27 06:12:54,819][20347] Updated weights for policy 0, policy_version 80017 (0.0007) [2023-12-27 06:12:55,324][20381] Updated weights for policy 1, policy_version 79844 (0.0007) [2023-12-27 06:12:55,379][20381] Updated weights for policy 1, policy_version 79854 (0.0006) [2023-12-27 06:12:55,433][20381] Updated weights for policy 1, policy_version 79864 (0.0010) [2023-12-27 06:12:55,515][20347] Updated weights for policy 0, policy_version 80027 (0.0006) [2023-12-27 06:12:55,575][20347] Updated weights for policy 0, policy_version 80037 (0.0007) [2023-12-27 06:12:55,620][20347] Updated weights for policy 0, policy_version 80047 (0.0007) [2023-12-27 06:12:56,112][20381] Updated weights for policy 1, policy_version 79874 (0.0010) [2023-12-27 06:12:56,163][20381] Updated weights for policy 1, policy_version 79884 (0.0010) [2023-12-27 06:12:56,195][20347] Updated weights for policy 0, policy_version 80057 (0.0006) [2023-12-27 06:12:56,218][20381] Updated weights for policy 1, policy_version 79894 (0.0010) [2023-12-27 06:12:56,249][20347] Updated weights for policy 0, policy_version 80067 (0.0005) [2023-12-27 06:12:56,270][20381] Updated weights for policy 1, policy_version 79904 (0.0010) [2023-12-27 06:12:56,304][20347] Updated weights for policy 0, policy_version 80077 (0.0005) [2023-12-27 06:12:56,368][20347] Updated weights for policy 0, policy_version 80087 (0.0006) [2023-12-27 06:12:56,948][20347] Updated weights for policy 0, policy_version 80097 (0.0008) [2023-12-27 06:12:56,960][20381] Updated weights for policy 1, policy_version 79914 (0.0006) [2023-12-27 06:12:57,008][20381] Updated weights for policy 1, policy_version 79924 (0.0006) [2023-12-27 06:12:57,015][20347] Updated weights for policy 0, policy_version 80107 (0.0007) [2023-12-27 06:12:57,060][20381] Updated weights for policy 1, policy_version 79934 (0.0006) [2023-12-27 06:12:57,062][20347] Updated weights for policy 0, policy_version 80117 (0.0006) [2023-12-27 06:12:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 40992768. Throughput: 0: 10310.1, 1: 10342.8. Samples: 40978848. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:12:57,363][19170] Avg episode reward: [(0, '990.830'), (1, '1000.000')] [2023-12-27 06:12:57,632][20381] Updated weights for policy 1, policy_version 79944 (0.0005) [2023-12-27 06:12:57,652][20347] Updated weights for policy 0, policy_version 80127 (0.0008) [2023-12-27 06:12:57,688][20381] Updated weights for policy 1, policy_version 79954 (0.0006) [2023-12-27 06:12:57,707][20347] Updated weights for policy 0, policy_version 80137 (0.0006) [2023-12-27 06:12:57,743][20381] Updated weights for policy 1, policy_version 79964 (0.0007) [2023-12-27 06:12:57,756][20347] Updated weights for policy 0, policy_version 80147 (0.0005) [2023-12-27 06:12:58,403][20381] Updated weights for policy 1, policy_version 79974 (0.0008) [2023-12-27 06:12:58,441][20347] Updated weights for policy 0, policy_version 80157 (0.0006) [2023-12-27 06:12:58,469][20381] Updated weights for policy 1, policy_version 79984 (0.0010) [2023-12-27 06:12:58,469][20291] KL-divergence is very high: 132.1630 [2023-12-27 06:12:58,506][20347] Updated weights for policy 0, policy_version 80167 (0.0007) [2023-12-27 06:12:58,520][20291] KL-divergence is very high: 133.7742 [2023-12-27 06:12:58,527][20291] KL-divergence is very high: 111.2012 [2023-12-27 06:12:58,534][20381] Updated weights for policy 1, policy_version 79994 (0.0010) [2023-12-27 06:12:58,568][20347] Updated weights for policy 0, policy_version 80177 (0.0006) [2023-12-27 06:12:59,343][20381] Updated weights for policy 1, policy_version 80004 (0.0008) [2023-12-27 06:12:59,368][20347] Updated weights for policy 0, policy_version 80187 (0.0007) [2023-12-27 06:12:59,400][20381] Updated weights for policy 1, policy_version 80014 (0.0008) [2023-12-27 06:12:59,426][20347] Updated weights for policy 0, policy_version 80197 (0.0008) [2023-12-27 06:12:59,445][20381] Updated weights for policy 1, policy_version 80024 (0.0005) [2023-12-27 06:12:59,487][20347] Updated weights for policy 0, policy_version 80207 (0.0007) [2023-12-27 06:12:59,523][20291] KL-divergence is very high: 104.0378 [2023-12-27 06:13:00,114][20381] Updated weights for policy 1, policy_version 80034 (0.0007) [2023-12-27 06:13:00,171][20381] Updated weights for policy 1, policy_version 80044 (0.0010) [2023-12-27 06:13:00,179][20347] Updated weights for policy 0, policy_version 80217 (0.0008) [2023-12-27 06:13:00,226][20291] KL-divergence is very high: 159.7152 [2023-12-27 06:13:00,226][20381] Updated weights for policy 1, policy_version 80054 (0.0010) [2023-12-27 06:13:00,238][20347] Updated weights for policy 0, policy_version 80227 (0.0008) [2023-12-27 06:13:00,275][20291] KL-divergence is very high: 239.5522 [2023-12-27 06:13:00,279][20381] Updated weights for policy 1, policy_version 80064 (0.0009) [2023-12-27 06:13:00,289][20291] KL-divergence is very high: 202.3634 [2023-12-27 06:13:00,300][20347] Updated weights for policy 0, policy_version 80237 (0.0006) [2023-12-27 06:13:00,325][20291] KL-divergence is very high: 106.8140 [2023-12-27 06:13:00,340][20291] KL-divergence is very high: 176.7942 [2023-12-27 06:13:00,358][20291] KL-divergence is very high: 219.2020 [2023-12-27 06:13:00,363][20347] Updated weights for policy 0, policy_version 80247 (0.0007) [2023-12-27 06:13:00,901][20381] Updated weights for policy 1, policy_version 80074 (0.0005) [2023-12-27 06:13:00,949][20381] Updated weights for policy 1, policy_version 80084 (0.0005) [2023-12-27 06:13:00,995][20381] Updated weights for policy 1, policy_version 80094 (0.0009) [2023-12-27 06:13:01,009][20347] Updated weights for policy 0, policy_version 80257 (0.0009) [2023-12-27 06:13:01,068][20347] Updated weights for policy 0, policy_version 80267 (0.0009) [2023-12-27 06:13:01,133][20347] Updated weights for policy 0, policy_version 80277 (0.0008) [2023-12-27 06:13:01,670][20381] Updated weights for policy 1, policy_version 80104 (0.0007) [2023-12-27 06:13:01,733][20381] Updated weights for policy 1, policy_version 80114 (0.0009) [2023-12-27 06:13:01,788][20381] Updated weights for policy 1, policy_version 80124 (0.0009) [2023-12-27 06:13:01,836][20347] Updated weights for policy 0, policy_version 80287 (0.0006) [2023-12-27 06:13:01,890][20347] Updated weights for policy 0, policy_version 80297 (0.0007) [2023-12-27 06:13:01,947][20347] Updated weights for policy 0, policy_version 80307 (0.0007) [2023-12-27 06:13:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 41091072. Throughput: 0: 10340.9, 1: 10357.7. Samples: 41103588. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:02,363][19170] Avg episode reward: [(0, '601.550'), (1, '1000.000')] [2023-12-27 06:13:02,478][20381] Updated weights for policy 1, policy_version 80134 (0.0009) [2023-12-27 06:13:02,536][20381] Updated weights for policy 1, policy_version 80144 (0.0006) [2023-12-27 06:13:02,595][20381] Updated weights for policy 1, policy_version 80154 (0.0005) [2023-12-27 06:13:02,607][20347] Updated weights for policy 0, policy_version 80317 (0.0009) [2023-12-27 06:13:02,655][20347] Updated weights for policy 0, policy_version 80327 (0.0010) [2023-12-27 06:13:02,708][20347] Updated weights for policy 0, policy_version 80337 (0.0010) [2023-12-27 06:13:03,283][20347] Updated weights for policy 0, policy_version 80347 (0.0007) [2023-12-27 06:13:03,305][20381] Updated weights for policy 1, policy_version 80164 (0.0006) [2023-12-27 06:13:03,343][20347] Updated weights for policy 0, policy_version 80357 (0.0007) [2023-12-27 06:13:03,367][20381] Updated weights for policy 1, policy_version 80174 (0.0007) [2023-12-27 06:13:03,398][20347] Updated weights for policy 0, policy_version 80367 (0.0005) [2023-12-27 06:13:03,420][20381] Updated weights for policy 1, policy_version 80184 (0.0008) [2023-12-27 06:13:03,981][20347] Updated weights for policy 0, policy_version 80377 (0.0009) [2023-12-27 06:13:04,032][20347] Updated weights for policy 0, policy_version 80387 (0.0005) [2023-12-27 06:13:04,086][20347] Updated weights for policy 0, policy_version 80397 (0.0005) [2023-12-27 06:13:04,145][20381] Updated weights for policy 1, policy_version 80194 (0.0007) [2023-12-27 06:13:04,152][20347] Updated weights for policy 0, policy_version 80407 (0.0009) [2023-12-27 06:13:04,202][20381] Updated weights for policy 1, policy_version 80204 (0.0007) [2023-12-27 06:13:04,253][20381] Updated weights for policy 1, policy_version 80214 (0.0007) [2023-12-27 06:13:04,314][20381] Updated weights for policy 1, policy_version 80224 (0.0007) [2023-12-27 06:13:04,786][20347] Updated weights for policy 0, policy_version 80417 (0.0006) [2023-12-27 06:13:04,835][20347] Updated weights for policy 0, policy_version 80427 (0.0005) [2023-12-27 06:13:04,892][20347] Updated weights for policy 0, policy_version 80437 (0.0007) [2023-12-27 06:13:04,950][20381] Updated weights for policy 1, policy_version 80234 (0.0010) [2023-12-27 06:13:05,009][20381] Updated weights for policy 1, policy_version 80244 (0.0011) [2023-12-27 06:13:05,068][20381] Updated weights for policy 1, policy_version 80254 (0.0011) [2023-12-27 06:13:05,517][20347] Updated weights for policy 0, policy_version 80447 (0.0010) [2023-12-27 06:13:05,579][20347] Updated weights for policy 0, policy_version 80457 (0.0010) [2023-12-27 06:13:05,642][20347] Updated weights for policy 0, policy_version 80467 (0.0010) [2023-12-27 06:13:05,687][20381] Updated weights for policy 1, policy_version 80264 (0.0010) [2023-12-27 06:13:05,735][20381] Updated weights for policy 1, policy_version 80274 (0.0010) [2023-12-27 06:13:05,793][20381] Updated weights for policy 1, policy_version 80284 (0.0008) [2023-12-27 06:13:06,222][20347] Updated weights for policy 0, policy_version 80477 (0.0007) [2023-12-27 06:13:06,278][20347] Updated weights for policy 0, policy_version 80487 (0.0008) [2023-12-27 06:13:06,337][20347] Updated weights for policy 0, policy_version 80497 (0.0007) [2023-12-27 06:13:06,479][20381] Updated weights for policy 1, policy_version 80294 (0.0009) [2023-12-27 06:13:06,540][20381] Updated weights for policy 1, policy_version 80304 (0.0011) [2023-12-27 06:13:06,600][20381] Updated weights for policy 1, policy_version 80314 (0.0011) [2023-12-27 06:13:07,036][20347] Updated weights for policy 0, policy_version 80507 (0.0008) [2023-12-27 06:13:07,098][20347] Updated weights for policy 0, policy_version 80517 (0.0010) [2023-12-27 06:13:07,157][20347] Updated weights for policy 0, policy_version 80527 (0.0011) [2023-12-27 06:13:07,286][20381] Updated weights for policy 1, policy_version 80324 (0.0011) [2023-12-27 06:13:07,341][20381] Updated weights for policy 1, policy_version 80334 (0.0010) [2023-12-27 06:13:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 41197568. Throughput: 0: 10400.6, 1: 10360.1. Samples: 41167344. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:07,364][19170] Avg episode reward: [(0, '540.040'), (1, '1000.000')] [2023-12-27 06:13:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000080536_20627456.pth... [2023-12-27 06:13:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000079320_20316160.pth [2023-12-27 06:13:07,391][20381] Updated weights for policy 1, policy_version 80344 (0.0008) [2023-12-27 06:13:07,430][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000080352_20578304.pth... [2023-12-27 06:13:07,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000079136_20267008.pth [2023-12-27 06:13:07,758][20347] Updated weights for policy 0, policy_version 80537 (0.0010) [2023-12-27 06:13:07,817][20347] Updated weights for policy 0, policy_version 80547 (0.0007) [2023-12-27 06:13:07,873][20347] Updated weights for policy 0, policy_version 80557 (0.0007) [2023-12-27 06:13:07,933][20347] Updated weights for policy 0, policy_version 80567 (0.0005) [2023-12-27 06:13:08,026][20381] Updated weights for policy 1, policy_version 80354 (0.0010) [2023-12-27 06:13:08,084][20381] Updated weights for policy 1, policy_version 80364 (0.0011) [2023-12-27 06:13:08,133][20381] Updated weights for policy 1, policy_version 80374 (0.0011) [2023-12-27 06:13:08,181][20381] Updated weights for policy 1, policy_version 80384 (0.0010) [2023-12-27 06:13:08,540][20347] Updated weights for policy 0, policy_version 80577 (0.0007) [2023-12-27 06:13:08,591][20347] Updated weights for policy 0, policy_version 80587 (0.0007) [2023-12-27 06:13:08,647][20347] Updated weights for policy 0, policy_version 80597 (0.0007) [2023-12-27 06:13:08,893][20381] Updated weights for policy 1, policy_version 80394 (0.0011) [2023-12-27 06:13:08,951][20381] Updated weights for policy 1, policy_version 80404 (0.0010) [2023-12-27 06:13:09,006][20381] Updated weights for policy 1, policy_version 80414 (0.0010) [2023-12-27 06:13:09,351][20347] Updated weights for policy 0, policy_version 80607 (0.0007) [2023-12-27 06:13:09,410][20347] Updated weights for policy 0, policy_version 80617 (0.0008) [2023-12-27 06:13:09,473][20347] Updated weights for policy 0, policy_version 80627 (0.0007) [2023-12-27 06:13:09,712][20381] Updated weights for policy 1, policy_version 80424 (0.0011) [2023-12-27 06:13:09,765][20381] Updated weights for policy 1, policy_version 80434 (0.0011) [2023-12-27 06:13:09,815][20381] Updated weights for policy 1, policy_version 80444 (0.0010) [2023-12-27 06:13:10,087][20347] Updated weights for policy 0, policy_version 80637 (0.0007) [2023-12-27 06:13:10,151][20347] Updated weights for policy 0, policy_version 80647 (0.0007) [2023-12-27 06:13:10,214][20347] Updated weights for policy 0, policy_version 80657 (0.0007) [2023-12-27 06:13:10,528][20381] Updated weights for policy 1, policy_version 80454 (0.0011) [2023-12-27 06:13:10,587][20381] Updated weights for policy 1, policy_version 80464 (0.0011) [2023-12-27 06:13:10,645][20381] Updated weights for policy 1, policy_version 80474 (0.0010) [2023-12-27 06:13:10,881][20347] Updated weights for policy 0, policy_version 80667 (0.0008) [2023-12-27 06:13:10,936][20347] Updated weights for policy 0, policy_version 80677 (0.0008) [2023-12-27 06:13:11,000][20347] Updated weights for policy 0, policy_version 80687 (0.0007) [2023-12-27 06:13:11,340][20381] Updated weights for policy 1, policy_version 80484 (0.0011) [2023-12-27 06:13:11,403][20381] Updated weights for policy 1, policy_version 80494 (0.0010) [2023-12-27 06:13:11,456][20381] Updated weights for policy 1, policy_version 80504 (0.0010) [2023-12-27 06:13:11,719][20347] Updated weights for policy 0, policy_version 80697 (0.0007) [2023-12-27 06:13:11,783][20347] Updated weights for policy 0, policy_version 80707 (0.0006) [2023-12-27 06:13:11,838][20347] Updated weights for policy 0, policy_version 80717 (0.0005) [2023-12-27 06:13:11,901][20347] Updated weights for policy 0, policy_version 80727 (0.0006) [2023-12-27 06:13:12,126][20381] Updated weights for policy 1, policy_version 80514 (0.0009) [2023-12-27 06:13:12,182][20381] Updated weights for policy 1, policy_version 80524 (0.0011) [2023-12-27 06:13:12,244][20381] Updated weights for policy 1, policy_version 80534 (0.0011) [2023-12-27 06:13:12,312][20381] Updated weights for policy 1, policy_version 80544 (0.0011) [2023-12-27 06:13:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 41304064. Throughput: 0: 10393.9, 1: 10361.4. Samples: 41291980. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:12,363][19170] Avg episode reward: [(0, '571.030'), (1, '1000.000')] [2023-12-27 06:13:12,510][20347] Updated weights for policy 0, policy_version 80737 (0.0007) [2023-12-27 06:13:12,565][20347] Updated weights for policy 0, policy_version 80747 (0.0008) [2023-12-27 06:13:12,618][20347] Updated weights for policy 0, policy_version 80757 (0.0007) [2023-12-27 06:13:12,996][20381] Updated weights for policy 1, policy_version 80554 (0.0010) [2023-12-27 06:13:13,051][20381] Updated weights for policy 1, policy_version 80564 (0.0010) [2023-12-27 06:13:13,102][20381] Updated weights for policy 1, policy_version 80574 (0.0010) [2023-12-27 06:13:13,274][20347] Updated weights for policy 0, policy_version 80767 (0.0007) [2023-12-27 06:13:13,326][20347] Updated weights for policy 0, policy_version 80777 (0.0008) [2023-12-27 06:13:13,381][20347] Updated weights for policy 0, policy_version 80787 (0.0010) [2023-12-27 06:13:13,770][20381] Updated weights for policy 1, policy_version 80584 (0.0011) [2023-12-27 06:13:13,826][20381] Updated weights for policy 1, policy_version 80594 (0.0011) [2023-12-27 06:13:13,890][20381] Updated weights for policy 1, policy_version 80604 (0.0011) [2023-12-27 06:13:14,054][20347] Updated weights for policy 0, policy_version 80797 (0.0008) [2023-12-27 06:13:14,113][20347] Updated weights for policy 0, policy_version 80807 (0.0006) [2023-12-27 06:13:14,168][20347] Updated weights for policy 0, policy_version 80817 (0.0010) [2023-12-27 06:13:14,540][20381] Updated weights for policy 1, policy_version 80614 (0.0007) [2023-12-27 06:13:14,590][20381] Updated weights for policy 1, policy_version 80624 (0.0005) [2023-12-27 06:13:14,645][20381] Updated weights for policy 1, policy_version 80634 (0.0005) [2023-12-27 06:13:14,837][20347] Updated weights for policy 0, policy_version 80827 (0.0009) [2023-12-27 06:13:14,895][20347] Updated weights for policy 0, policy_version 80837 (0.0011) [2023-12-27 06:13:14,959][20347] Updated weights for policy 0, policy_version 80847 (0.0009) [2023-12-27 06:13:15,253][20381] Updated weights for policy 1, policy_version 80644 (0.0007) [2023-12-27 06:13:15,311][20381] Updated weights for policy 1, policy_version 80654 (0.0010) [2023-12-27 06:13:15,376][20381] Updated weights for policy 1, policy_version 80664 (0.0010) [2023-12-27 06:13:15,612][20347] Updated weights for policy 0, policy_version 80857 (0.0008) [2023-12-27 06:13:15,672][20347] Updated weights for policy 0, policy_version 80867 (0.0007) [2023-12-27 06:13:15,730][20347] Updated weights for policy 0, policy_version 80877 (0.0008) [2023-12-27 06:13:15,787][20347] Updated weights for policy 0, policy_version 80887 (0.0007) [2023-12-27 06:13:16,059][20381] Updated weights for policy 1, policy_version 80674 (0.0010) [2023-12-27 06:13:16,112][20381] Updated weights for policy 1, policy_version 80684 (0.0011) [2023-12-27 06:13:16,162][20381] Updated weights for policy 1, policy_version 80694 (0.0011) [2023-12-27 06:13:16,217][20381] Updated weights for policy 1, policy_version 80704 (0.0010) [2023-12-27 06:13:16,498][20347] Updated weights for policy 0, policy_version 80897 (0.0007) [2023-12-27 06:13:16,560][20347] Updated weights for policy 0, policy_version 80907 (0.0007) [2023-12-27 06:13:16,612][20347] Updated weights for policy 0, policy_version 80917 (0.0007) [2023-12-27 06:13:16,896][20381] Updated weights for policy 1, policy_version 80714 (0.0006) [2023-12-27 06:13:16,959][20381] Updated weights for policy 1, policy_version 80724 (0.0009) [2023-12-27 06:13:17,010][20381] Updated weights for policy 1, policy_version 80734 (0.0005) [2023-12-27 06:13:17,251][20347] Updated weights for policy 0, policy_version 80927 (0.0007) [2023-12-27 06:13:17,312][20347] Updated weights for policy 0, policy_version 80937 (0.0005) [2023-12-27 06:13:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 41402368. Throughput: 0: 10357.6, 1: 10325.7. Samples: 41416808. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:17,363][19170] Avg episode reward: [(0, '618.700'), (1, '1000.000')] [2023-12-27 06:13:17,374][20347] Updated weights for policy 0, policy_version 80947 (0.0006) [2023-12-27 06:13:17,560][20381] Updated weights for policy 1, policy_version 80744 (0.0005) [2023-12-27 06:13:17,603][20381] Updated weights for policy 1, policy_version 80754 (0.0005) [2023-12-27 06:13:17,656][20381] Updated weights for policy 1, policy_version 80764 (0.0005) [2023-12-27 06:13:18,044][20347] Updated weights for policy 0, policy_version 80957 (0.0007) [2023-12-27 06:13:18,100][20347] Updated weights for policy 0, policy_version 80967 (0.0008) [2023-12-27 06:13:18,146][20347] Updated weights for policy 0, policy_version 80977 (0.0007) [2023-12-27 06:13:18,209][20381] Updated weights for policy 1, policy_version 80774 (0.0006) [2023-12-27 06:13:18,260][20381] Updated weights for policy 1, policy_version 80784 (0.0008) [2023-12-27 06:13:18,305][20381] Updated weights for policy 1, policy_version 80794 (0.0006) [2023-12-27 06:13:18,829][20347] Updated weights for policy 0, policy_version 80987 (0.0008) [2023-12-27 06:13:18,888][20347] Updated weights for policy 0, policy_version 80997 (0.0008) [2023-12-27 06:13:18,946][20347] Updated weights for policy 0, policy_version 81007 (0.0007) [2023-12-27 06:13:19,019][20381] Updated weights for policy 1, policy_version 80804 (0.0008) [2023-12-27 06:13:19,067][20381] Updated weights for policy 1, policy_version 80814 (0.0007) [2023-12-27 06:13:19,119][20381] Updated weights for policy 1, policy_version 80824 (0.0007) [2023-12-27 06:13:19,639][20347] Updated weights for policy 0, policy_version 81017 (0.0007) [2023-12-27 06:13:19,700][20347] Updated weights for policy 0, policy_version 81027 (0.0008) [2023-12-27 06:13:19,750][20347] Updated weights for policy 0, policy_version 81037 (0.0007) [2023-12-27 06:13:19,784][20381] Updated weights for policy 1, policy_version 80834 (0.0008) [2023-12-27 06:13:19,809][20347] Updated weights for policy 0, policy_version 81047 (0.0008) [2023-12-27 06:13:19,848][20381] Updated weights for policy 1, policy_version 80844 (0.0007) [2023-12-27 06:13:19,907][20381] Updated weights for policy 1, policy_version 80854 (0.0007) [2023-12-27 06:13:19,970][20381] Updated weights for policy 1, policy_version 80864 (0.0008) [2023-12-27 06:13:20,481][20347] Updated weights for policy 0, policy_version 81057 (0.0007) [2023-12-27 06:13:20,532][20347] Updated weights for policy 0, policy_version 81067 (0.0008) [2023-12-27 06:13:20,594][20347] Updated weights for policy 0, policy_version 81077 (0.0008) [2023-12-27 06:13:20,682][20381] Updated weights for policy 1, policy_version 80874 (0.0008) [2023-12-27 06:13:20,736][20381] Updated weights for policy 1, policy_version 80884 (0.0007) [2023-12-27 06:13:20,786][20381] Updated weights for policy 1, policy_version 80894 (0.0008) [2023-12-27 06:13:21,325][20347] Updated weights for policy 0, policy_version 81087 (0.0008) [2023-12-27 06:13:21,391][20347] Updated weights for policy 0, policy_version 81097 (0.0007) [2023-12-27 06:13:21,455][20347] Updated weights for policy 0, policy_version 81107 (0.0007) [2023-12-27 06:13:21,491][20381] Updated weights for policy 1, policy_version 80904 (0.0008) [2023-12-27 06:13:21,555][20381] Updated weights for policy 1, policy_version 80914 (0.0008) [2023-12-27 06:13:21,614][20381] Updated weights for policy 1, policy_version 80924 (0.0008) [2023-12-27 06:13:22,227][20381] Updated weights for policy 1, policy_version 80934 (0.0008) [2023-12-27 06:13:22,236][20347] Updated weights for policy 0, policy_version 81117 (0.0007) [2023-12-27 06:13:22,291][20381] Updated weights for policy 1, policy_version 80944 (0.0006) [2023-12-27 06:13:22,296][20347] Updated weights for policy 0, policy_version 81127 (0.0007) [2023-12-27 06:13:22,350][20381] Updated weights for policy 1, policy_version 80954 (0.0007) [2023-12-27 06:13:22,352][20347] Updated weights for policy 0, policy_version 81137 (0.0005) [2023-12-27 06:13:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 41500672. Throughput: 0: 10362.3, 1: 10317.7. Samples: 41479228. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:22,363][19170] Avg episode reward: [(0, '740.950'), (1, '1000.000')] [2023-12-27 06:13:22,382][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000080960_20733952.pth... [2023-12-27 06:13:22,386][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000079744_20422656.pth [2023-12-27 06:13:22,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000081144_20783104.pth... [2023-12-27 06:13:22,394][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000079928_20471808.pth [2023-12-27 06:13:22,953][20381] Updated weights for policy 1, policy_version 80964 (0.0006) [2023-12-27 06:13:23,003][20381] Updated weights for policy 1, policy_version 80974 (0.0005) [2023-12-27 06:13:23,061][20381] Updated weights for policy 1, policy_version 80984 (0.0005) [2023-12-27 06:13:23,071][20347] Updated weights for policy 0, policy_version 81147 (0.0006) [2023-12-27 06:13:23,131][20347] Updated weights for policy 0, policy_version 81157 (0.0005) [2023-12-27 06:13:23,179][20347] Updated weights for policy 0, policy_version 81167 (0.0006) [2023-12-27 06:13:23,656][20381] Updated weights for policy 1, policy_version 80994 (0.0006) [2023-12-27 06:13:23,718][20381] Updated weights for policy 1, policy_version 81004 (0.0008) [2023-12-27 06:13:23,778][20381] Updated weights for policy 1, policy_version 81014 (0.0008) [2023-12-27 06:13:23,787][20347] Updated weights for policy 0, policy_version 81177 (0.0007) [2023-12-27 06:13:23,841][20381] Updated weights for policy 1, policy_version 81024 (0.0007) [2023-12-27 06:13:23,849][20347] Updated weights for policy 0, policy_version 81187 (0.0006) [2023-12-27 06:13:23,908][20347] Updated weights for policy 0, policy_version 81197 (0.0008) [2023-12-27 06:13:23,967][20347] Updated weights for policy 0, policy_version 81207 (0.0007) [2023-12-27 06:13:24,502][20381] Updated weights for policy 1, policy_version 81034 (0.0007) [2023-12-27 06:13:24,550][20381] Updated weights for policy 1, policy_version 81044 (0.0008) [2023-12-27 06:13:24,605][20381] Updated weights for policy 1, policy_version 81054 (0.0007) [2023-12-27 06:13:24,655][20347] Updated weights for policy 0, policy_version 81217 (0.0008) [2023-12-27 06:13:24,717][20347] Updated weights for policy 0, policy_version 81227 (0.0008) [2023-12-27 06:13:24,773][20347] Updated weights for policy 0, policy_version 81237 (0.0008) [2023-12-27 06:13:25,226][20381] Updated weights for policy 1, policy_version 81064 (0.0007) [2023-12-27 06:13:25,275][20381] Updated weights for policy 1, policy_version 81074 (0.0007) [2023-12-27 06:13:25,326][20381] Updated weights for policy 1, policy_version 81084 (0.0008) [2023-12-27 06:13:25,470][20347] Updated weights for policy 0, policy_version 81247 (0.0006) [2023-12-27 06:13:25,526][20347] Updated weights for policy 0, policy_version 81257 (0.0007) [2023-12-27 06:13:25,581][20347] Updated weights for policy 0, policy_version 81267 (0.0007) [2023-12-27 06:13:26,074][20381] Updated weights for policy 1, policy_version 81094 (0.0007) [2023-12-27 06:13:26,135][20381] Updated weights for policy 1, policy_version 81104 (0.0008) [2023-12-27 06:13:26,169][20347] Updated weights for policy 0, policy_version 81277 (0.0007) [2023-12-27 06:13:26,193][20381] Updated weights for policy 1, policy_version 81114 (0.0006) [2023-12-27 06:13:26,218][20347] Updated weights for policy 0, policy_version 81287 (0.0006) [2023-12-27 06:13:26,267][20347] Updated weights for policy 0, policy_version 81297 (0.0007) [2023-12-27 06:13:26,875][20381] Updated weights for policy 1, policy_version 81124 (0.0006) [2023-12-27 06:13:26,927][20381] Updated weights for policy 1, policy_version 81134 (0.0008) [2023-12-27 06:13:26,963][20347] Updated weights for policy 0, policy_version 81307 (0.0007) [2023-12-27 06:13:26,977][20381] Updated weights for policy 1, policy_version 81144 (0.0006) [2023-12-27 06:13:27,008][20347] Updated weights for policy 0, policy_version 81317 (0.0006) [2023-12-27 06:13:27,065][20347] Updated weights for policy 0, policy_version 81327 (0.0007) [2023-12-27 06:13:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 41615360. Throughput: 0: 10311.0, 1: 10369.0. Samples: 41603456. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:27,363][19170] Avg episode reward: [(0, '892.560'), (1, '1000.000')] [2023-12-27 06:13:27,645][20381] Updated weights for policy 1, policy_version 81154 (0.0006) [2023-12-27 06:13:27,673][20347] Updated weights for policy 0, policy_version 81337 (0.0007) [2023-12-27 06:13:27,693][20381] Updated weights for policy 1, policy_version 81164 (0.0007) [2023-12-27 06:13:27,721][20347] Updated weights for policy 0, policy_version 81347 (0.0005) [2023-12-27 06:13:27,741][20381] Updated weights for policy 1, policy_version 81174 (0.0005) [2023-12-27 06:13:27,769][20347] Updated weights for policy 0, policy_version 81357 (0.0006) [2023-12-27 06:13:27,790][20381] Updated weights for policy 1, policy_version 81184 (0.0005) [2023-12-27 06:13:27,824][20347] Updated weights for policy 0, policy_version 81367 (0.0006) [2023-12-27 06:13:28,400][20381] Updated weights for policy 1, policy_version 81194 (0.0008) [2023-12-27 06:13:28,462][20381] Updated weights for policy 1, policy_version 81204 (0.0007) [2023-12-27 06:13:28,514][20381] Updated weights for policy 1, policy_version 81214 (0.0005) [2023-12-27 06:13:28,536][20347] Updated weights for policy 0, policy_version 81377 (0.0005) [2023-12-27 06:13:28,585][20347] Updated weights for policy 0, policy_version 81387 (0.0007) [2023-12-27 06:13:28,646][20347] Updated weights for policy 0, policy_version 81397 (0.0007) [2023-12-27 06:13:29,103][20381] Updated weights for policy 1, policy_version 81224 (0.0007) [2023-12-27 06:13:29,162][20381] Updated weights for policy 1, policy_version 81234 (0.0007) [2023-12-27 06:13:29,216][20381] Updated weights for policy 1, policy_version 81244 (0.0006) [2023-12-27 06:13:29,374][20347] Updated weights for policy 0, policy_version 81407 (0.0008) [2023-12-27 06:13:29,436][20347] Updated weights for policy 0, policy_version 81417 (0.0007) [2023-12-27 06:13:29,504][20347] Updated weights for policy 0, policy_version 81427 (0.0009) [2023-12-27 06:13:29,861][20381] Updated weights for policy 1, policy_version 81254 (0.0007) [2023-12-27 06:13:29,926][20381] Updated weights for policy 1, policy_version 81264 (0.0008) [2023-12-27 06:13:29,986][20381] Updated weights for policy 1, policy_version 81274 (0.0007) [2023-12-27 06:13:30,245][20347] Updated weights for policy 0, policy_version 81437 (0.0007) [2023-12-27 06:13:30,295][20347] Updated weights for policy 0, policy_version 81447 (0.0007) [2023-12-27 06:13:30,346][20347] Updated weights for policy 0, policy_version 81457 (0.0006) [2023-12-27 06:13:30,607][20381] Updated weights for policy 1, policy_version 81284 (0.0007) [2023-12-27 06:13:30,656][20381] Updated weights for policy 1, policy_version 81294 (0.0008) [2023-12-27 06:13:30,707][20381] Updated weights for policy 1, policy_version 81304 (0.0007) [2023-12-27 06:13:30,997][20347] Updated weights for policy 0, policy_version 81467 (0.0008) [2023-12-27 06:13:31,060][20347] Updated weights for policy 0, policy_version 81477 (0.0010) [2023-12-27 06:13:31,101][20347] Updated weights for policy 0, policy_version 81487 (0.0010) [2023-12-27 06:13:31,404][20381] Updated weights for policy 1, policy_version 81314 (0.0007) [2023-12-27 06:13:31,459][20381] Updated weights for policy 1, policy_version 81324 (0.0005) [2023-12-27 06:13:31,512][20381] Updated weights for policy 1, policy_version 81334 (0.0005) [2023-12-27 06:13:31,566][20381] Updated weights for policy 1, policy_version 81344 (0.0005) [2023-12-27 06:13:31,776][20347] Updated weights for policy 0, policy_version 81497 (0.0009) [2023-12-27 06:13:31,826][20347] Updated weights for policy 0, policy_version 81507 (0.0008) [2023-12-27 06:13:31,874][20347] Updated weights for policy 0, policy_version 81517 (0.0008) [2023-12-27 06:13:31,928][20347] Updated weights for policy 0, policy_version 81527 (0.0008) [2023-12-27 06:13:32,223][20381] Updated weights for policy 1, policy_version 81354 (0.0008) [2023-12-27 06:13:32,275][20381] Updated weights for policy 1, policy_version 81364 (0.0008) [2023-12-27 06:13:32,323][20381] Updated weights for policy 1, policy_version 81374 (0.0008) [2023-12-27 06:13:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 41721856. Throughput: 0: 10341.0, 1: 10434.2. Samples: 41728872. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:13:32,363][19170] Avg episode reward: [(0, '979.010'), (1, '1000.000')] [2023-12-27 06:13:32,623][20347] Updated weights for policy 0, policy_version 81537 (0.0005) [2023-12-27 06:13:32,677][20347] Updated weights for policy 0, policy_version 81547 (0.0005) [2023-12-27 06:13:32,723][20347] Updated weights for policy 0, policy_version 81557 (0.0005) [2023-12-27 06:13:33,087][20381] Updated weights for policy 1, policy_version 81384 (0.0008) [2023-12-27 06:13:33,140][20381] Updated weights for policy 1, policy_version 81394 (0.0007) [2023-12-27 06:13:33,189][20381] Updated weights for policy 1, policy_version 81404 (0.0008) [2023-12-27 06:13:33,343][20347] Updated weights for policy 0, policy_version 81567 (0.0005) [2023-12-27 06:13:33,393][20347] Updated weights for policy 0, policy_version 81577 (0.0005) [2023-12-27 06:13:33,453][20347] Updated weights for policy 0, policy_version 81587 (0.0005) [2023-12-27 06:13:33,853][20381] Updated weights for policy 1, policy_version 81414 (0.0007) [2023-12-27 06:13:33,899][20381] Updated weights for policy 1, policy_version 81424 (0.0006) [2023-12-27 06:13:33,956][20381] Updated weights for policy 1, policy_version 81434 (0.0005) [2023-12-27 06:13:34,032][20347] Updated weights for policy 0, policy_version 81597 (0.0006) [2023-12-27 06:13:34,088][20347] Updated weights for policy 0, policy_version 81607 (0.0005) [2023-12-27 06:13:34,141][20347] Updated weights for policy 0, policy_version 81617 (0.0005) [2023-12-27 06:13:34,563][20381] Updated weights for policy 1, policy_version 81444 (0.0008) [2023-12-27 06:13:34,618][20381] Updated weights for policy 1, policy_version 81454 (0.0005) [2023-12-27 06:13:34,673][20381] Updated weights for policy 1, policy_version 81464 (0.0005) [2023-12-27 06:13:34,826][20347] Updated weights for policy 0, policy_version 81627 (0.0007) [2023-12-27 06:13:34,870][20347] Updated weights for policy 0, policy_version 81637 (0.0008) [2023-12-27 06:13:34,932][20347] Updated weights for policy 0, policy_version 81647 (0.0007) [2023-12-27 06:13:35,235][20381] Updated weights for policy 1, policy_version 81474 (0.0006) [2023-12-27 06:13:35,284][20381] Updated weights for policy 1, policy_version 81484 (0.0009) [2023-12-27 06:13:35,337][20381] Updated weights for policy 1, policy_version 81494 (0.0008) [2023-12-27 06:13:35,386][20381] Updated weights for policy 1, policy_version 81504 (0.0008) [2023-12-27 06:13:35,517][20347] Updated weights for policy 0, policy_version 81657 (0.0007) [2023-12-27 06:13:35,574][20347] Updated weights for policy 0, policy_version 81667 (0.0005) [2023-12-27 06:13:35,623][20347] Updated weights for policy 0, policy_version 81677 (0.0005) [2023-12-27 06:13:35,680][20347] Updated weights for policy 0, policy_version 81687 (0.0007) [2023-12-27 06:13:36,060][20381] Updated weights for policy 1, policy_version 81514 (0.0005) [2023-12-27 06:13:36,116][20381] Updated weights for policy 1, policy_version 81524 (0.0007) [2023-12-27 06:13:36,167][20381] Updated weights for policy 1, policy_version 81534 (0.0008) [2023-12-27 06:13:36,353][20347] Updated weights for policy 0, policy_version 81697 (0.0007) [2023-12-27 06:13:36,408][20347] Updated weights for policy 0, policy_version 81707 (0.0010) [2023-12-27 06:13:36,462][20347] Updated weights for policy 0, policy_version 81717 (0.0010) [2023-12-27 06:13:36,846][20381] Updated weights for policy 1, policy_version 81544 (0.0010) [2023-12-27 06:13:36,895][20381] Updated weights for policy 1, policy_version 81554 (0.0010) [2023-12-27 06:13:36,951][20381] Updated weights for policy 1, policy_version 81564 (0.0011) [2023-12-27 06:13:37,134][20347] Updated weights for policy 0, policy_version 81727 (0.0009) [2023-12-27 06:13:37,186][20347] Updated weights for policy 0, policy_version 81737 (0.0010) [2023-12-27 06:13:37,244][20347] Updated weights for policy 0, policy_version 81747 (0.0010) [2023-12-27 06:13:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 41828352. Throughput: 0: 10383.9, 1: 10419.2. Samples: 41793124. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:13:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:13:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000081568_20889600.pth... [2023-12-27 06:13:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000081752_20938752.pth... [2023-12-27 06:13:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000080536_20627456.pth [2023-12-27 06:13:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000080352_20578304.pth [2023-12-27 06:13:37,619][20381] Updated weights for policy 1, policy_version 81574 (0.0011) [2023-12-27 06:13:37,674][20381] Updated weights for policy 1, policy_version 81584 (0.0010) [2023-12-27 06:13:37,727][20381] Updated weights for policy 1, policy_version 81594 (0.0008) [2023-12-27 06:13:37,931][20347] Updated weights for policy 0, policy_version 81757 (0.0010) [2023-12-27 06:13:37,979][20347] Updated weights for policy 0, policy_version 81767 (0.0010) [2023-12-27 06:13:38,031][20347] Updated weights for policy 0, policy_version 81777 (0.0010) [2023-12-27 06:13:38,395][20381] Updated weights for policy 1, policy_version 81604 (0.0008) [2023-12-27 06:13:38,457][20381] Updated weights for policy 1, policy_version 81614 (0.0011) [2023-12-27 06:13:38,509][20381] Updated weights for policy 1, policy_version 81624 (0.0011) [2023-12-27 06:13:38,717][20347] Updated weights for policy 0, policy_version 81787 (0.0010) [2023-12-27 06:13:38,781][20347] Updated weights for policy 0, policy_version 81797 (0.0010) [2023-12-27 06:13:38,830][20347] Updated weights for policy 0, policy_version 81807 (0.0010) [2023-12-27 06:13:39,102][20381] Updated weights for policy 1, policy_version 81634 (0.0010) [2023-12-27 06:13:39,159][20381] Updated weights for policy 1, policy_version 81644 (0.0010) [2023-12-27 06:13:39,221][20381] Updated weights for policy 1, policy_version 81654 (0.0009) [2023-12-27 06:13:39,281][20381] Updated weights for policy 1, policy_version 81664 (0.0008) [2023-12-27 06:13:39,500][20347] Updated weights for policy 0, policy_version 81817 (0.0010) [2023-12-27 06:13:39,558][20347] Updated weights for policy 0, policy_version 81827 (0.0007) [2023-12-27 06:13:39,615][20347] Updated weights for policy 0, policy_version 81837 (0.0008) [2023-12-27 06:13:39,674][20347] Updated weights for policy 0, policy_version 81847 (0.0007) [2023-12-27 06:13:39,947][20381] Updated weights for policy 1, policy_version 81674 (0.0010) [2023-12-27 06:13:40,009][20381] Updated weights for policy 1, policy_version 81684 (0.0008) [2023-12-27 06:13:40,066][20381] Updated weights for policy 1, policy_version 81694 (0.0007) [2023-12-27 06:13:40,458][20347] Updated weights for policy 0, policy_version 81857 (0.0007) [2023-12-27 06:13:40,516][20347] Updated weights for policy 0, policy_version 81867 (0.0007) [2023-12-27 06:13:40,579][20347] Updated weights for policy 0, policy_version 81877 (0.0008) [2023-12-27 06:13:40,689][20381] Updated weights for policy 1, policy_version 81704 (0.0010) [2023-12-27 06:13:40,745][20381] Updated weights for policy 1, policy_version 81714 (0.0006) [2023-12-27 06:13:40,803][20381] Updated weights for policy 1, policy_version 81724 (0.0006) [2023-12-27 06:13:41,296][20347] Updated weights for policy 0, policy_version 81887 (0.0007) [2023-12-27 06:13:41,362][20347] Updated weights for policy 0, policy_version 81897 (0.0007) [2023-12-27 06:13:41,426][20347] Updated weights for policy 0, policy_version 81907 (0.0009) [2023-12-27 06:13:41,519][20381] Updated weights for policy 1, policy_version 81734 (0.0007) [2023-12-27 06:13:41,571][20381] Updated weights for policy 1, policy_version 81744 (0.0007) [2023-12-27 06:13:41,637][20381] Updated weights for policy 1, policy_version 81754 (0.0010) [2023-12-27 06:13:42,024][20347] Updated weights for policy 0, policy_version 81917 (0.0007) [2023-12-27 06:13:42,078][20347] Updated weights for policy 0, policy_version 81927 (0.0008) [2023-12-27 06:13:42,128][20347] Updated weights for policy 0, policy_version 81937 (0.0009) [2023-12-27 06:13:42,301][20381] Updated weights for policy 1, policy_version 81764 (0.0009) [2023-12-27 06:13:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 41926656. Throughput: 0: 10374.9, 1: 10498.8. Samples: 41918168. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:13:42,363][19170] Avg episode reward: [(0, '993.500'), (1, '1000.000')] [2023-12-27 06:13:42,364][20381] Updated weights for policy 1, policy_version 81774 (0.0008) [2023-12-27 06:13:42,425][20381] Updated weights for policy 1, policy_version 81784 (0.0006) [2023-12-27 06:13:42,785][20347] Updated weights for policy 0, policy_version 81947 (0.0006) [2023-12-27 06:13:42,845][20347] Updated weights for policy 0, policy_version 81957 (0.0006) [2023-12-27 06:13:42,891][20347] Updated weights for policy 0, policy_version 81967 (0.0005) [2023-12-27 06:13:43,113][20381] Updated weights for policy 1, policy_version 81794 (0.0005) [2023-12-27 06:13:43,167][20381] Updated weights for policy 1, policy_version 81804 (0.0005) [2023-12-27 06:13:43,215][20381] Updated weights for policy 1, policy_version 81814 (0.0007) [2023-12-27 06:13:43,260][20381] Updated weights for policy 1, policy_version 81824 (0.0009) [2023-12-27 06:13:43,484][20347] Updated weights for policy 0, policy_version 81977 (0.0008) [2023-12-27 06:13:43,548][20347] Updated weights for policy 0, policy_version 81987 (0.0010) [2023-12-27 06:13:43,614][20347] Updated weights for policy 0, policy_version 81997 (0.0010) [2023-12-27 06:13:43,680][20347] Updated weights for policy 0, policy_version 82007 (0.0007) [2023-12-27 06:13:43,883][20381] Updated weights for policy 1, policy_version 81834 (0.0010) [2023-12-27 06:13:43,937][20381] Updated weights for policy 1, policy_version 81844 (0.0010) [2023-12-27 06:13:43,989][20381] Updated weights for policy 1, policy_version 81854 (0.0010) [2023-12-27 06:13:44,321][20347] Updated weights for policy 0, policy_version 82017 (0.0010) [2023-12-27 06:13:44,369][20347] Updated weights for policy 0, policy_version 82027 (0.0010) [2023-12-27 06:13:44,414][20347] Updated weights for policy 0, policy_version 82037 (0.0010) [2023-12-27 06:13:44,627][20381] Updated weights for policy 1, policy_version 81864 (0.0010) [2023-12-27 06:13:44,686][20381] Updated weights for policy 1, policy_version 81874 (0.0010) [2023-12-27 06:13:44,744][20381] Updated weights for policy 1, policy_version 81884 (0.0005) [2023-12-27 06:13:45,100][20347] Updated weights for policy 0, policy_version 82047 (0.0007) [2023-12-27 06:13:45,161][20347] Updated weights for policy 0, policy_version 82057 (0.0010) [2023-12-27 06:13:45,219][20347] Updated weights for policy 0, policy_version 82067 (0.0010) [2023-12-27 06:13:45,485][20381] Updated weights for policy 1, policy_version 81894 (0.0006) [2023-12-27 06:13:45,542][20381] Updated weights for policy 1, policy_version 81904 (0.0006) [2023-12-27 06:13:45,592][20381] Updated weights for policy 1, policy_version 81914 (0.0005) [2023-12-27 06:13:45,880][20347] Updated weights for policy 0, policy_version 82077 (0.0009) [2023-12-27 06:13:45,933][20347] Updated weights for policy 0, policy_version 82087 (0.0006) [2023-12-27 06:13:45,990][20347] Updated weights for policy 0, policy_version 82097 (0.0006) [2023-12-27 06:13:46,129][20381] Updated weights for policy 1, policy_version 81924 (0.0006) [2023-12-27 06:13:46,184][20381] Updated weights for policy 1, policy_version 81934 (0.0007) [2023-12-27 06:13:46,239][20381] Updated weights for policy 1, policy_version 81944 (0.0008) [2023-12-27 06:13:46,573][20347] Updated weights for policy 0, policy_version 82107 (0.0008) [2023-12-27 06:13:46,623][20347] Updated weights for policy 0, policy_version 82117 (0.0005) [2023-12-27 06:13:46,671][20347] Updated weights for policy 0, policy_version 82127 (0.0005) [2023-12-27 06:13:46,921][20381] Updated weights for policy 1, policy_version 81954 (0.0008) [2023-12-27 06:13:46,968][20381] Updated weights for policy 1, policy_version 81964 (0.0005) [2023-12-27 06:13:47,019][20381] Updated weights for policy 1, policy_version 81974 (0.0005) [2023-12-27 06:13:47,085][20381] Updated weights for policy 1, policy_version 81984 (0.0006) [2023-12-27 06:13:47,237][20347] Updated weights for policy 0, policy_version 82137 (0.0005) [2023-12-27 06:13:47,295][20347] Updated weights for policy 0, policy_version 82147 (0.0006) [2023-12-27 06:13:47,358][20347] Updated weights for policy 0, policy_version 82157 (0.0005) [2023-12-27 06:13:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 42033152. Throughput: 0: 10430.7, 1: 10524.0. Samples: 42046552. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:13:47,363][19170] Avg episode reward: [(0, '931.350'), (1, '1000.000')] [2023-12-27 06:13:47,417][20347] Updated weights for policy 0, policy_version 82167 (0.0006) [2023-12-27 06:13:47,649][20381] Updated weights for policy 1, policy_version 81994 (0.0008) [2023-12-27 06:13:47,701][20381] Updated weights for policy 1, policy_version 82004 (0.0008) [2023-12-27 06:13:47,751][20381] Updated weights for policy 1, policy_version 82014 (0.0006) [2023-12-27 06:13:48,043][20347] Updated weights for policy 0, policy_version 82177 (0.0007) [2023-12-27 06:13:48,102][20347] Updated weights for policy 0, policy_version 82187 (0.0006) [2023-12-27 06:13:48,157][20347] Updated weights for policy 0, policy_version 82197 (0.0007) [2023-12-27 06:13:48,475][20381] Updated weights for policy 1, policy_version 82024 (0.0006) [2023-12-27 06:13:48,537][20381] Updated weights for policy 1, policy_version 82034 (0.0006) [2023-12-27 06:13:48,595][20381] Updated weights for policy 1, policy_version 82044 (0.0010) [2023-12-27 06:13:48,865][20347] Updated weights for policy 0, policy_version 82207 (0.0007) [2023-12-27 06:13:48,919][20347] Updated weights for policy 0, policy_version 82217 (0.0008) [2023-12-27 06:13:48,980][20347] Updated weights for policy 0, policy_version 82227 (0.0007) [2023-12-27 06:13:49,212][20381] Updated weights for policy 1, policy_version 82054 (0.0010) [2023-12-27 06:13:49,275][20381] Updated weights for policy 1, policy_version 82064 (0.0009) [2023-12-27 06:13:49,339][20381] Updated weights for policy 1, policy_version 82074 (0.0009) [2023-12-27 06:13:49,690][20347] Updated weights for policy 0, policy_version 82237 (0.0008) [2023-12-27 06:13:49,745][20347] Updated weights for policy 0, policy_version 82247 (0.0007) [2023-12-27 06:13:49,797][20347] Updated weights for policy 0, policy_version 82257 (0.0007) [2023-12-27 06:13:49,953][20381] Updated weights for policy 1, policy_version 82084 (0.0008) [2023-12-27 06:13:50,016][20381] Updated weights for policy 1, policy_version 82094 (0.0007) [2023-12-27 06:13:50,076][20381] Updated weights for policy 1, policy_version 82104 (0.0007) [2023-12-27 06:13:50,534][20347] Updated weights for policy 0, policy_version 82267 (0.0008) [2023-12-27 06:13:50,592][20347] Updated weights for policy 0, policy_version 82277 (0.0007) [2023-12-27 06:13:50,650][20347] Updated weights for policy 0, policy_version 82287 (0.0007) [2023-12-27 06:13:50,693][20381] Updated weights for policy 1, policy_version 82114 (0.0008) [2023-12-27 06:13:50,749][20381] Updated weights for policy 1, policy_version 82124 (0.0007) [2023-12-27 06:13:50,809][20381] Updated weights for policy 1, policy_version 82134 (0.0006) [2023-12-27 06:13:50,868][20381] Updated weights for policy 1, policy_version 82144 (0.0006) [2023-12-27 06:13:51,337][20347] Updated weights for policy 0, policy_version 82297 (0.0006) [2023-12-27 06:13:51,397][20347] Updated weights for policy 0, policy_version 82307 (0.0007) [2023-12-27 06:13:51,452][20347] Updated weights for policy 0, policy_version 82317 (0.0008) [2023-12-27 06:13:51,507][20347] Updated weights for policy 0, policy_version 82327 (0.0007) [2023-12-27 06:13:51,548][20381] Updated weights for policy 1, policy_version 82154 (0.0007) [2023-12-27 06:13:51,602][20381] Updated weights for policy 1, policy_version 82164 (0.0007) [2023-12-27 06:13:51,665][20381] Updated weights for policy 1, policy_version 82174 (0.0008) [2023-12-27 06:13:52,179][20347] Updated weights for policy 0, policy_version 82337 (0.0008) [2023-12-27 06:13:52,236][20347] Updated weights for policy 0, policy_version 82347 (0.0008) [2023-12-27 06:13:52,293][20347] Updated weights for policy 0, policy_version 82357 (0.0007) [2023-12-27 06:13:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20799.4). Total num frames: 42139648. Throughput: 0: 10364.9, 1: 10560.1. Samples: 42108960. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:13:52,363][19170] Avg episode reward: [(0, '937.850'), (1, '1000.000')] [2023-12-27 06:13:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000082360_21094400.pth... [2023-12-27 06:13:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000081144_20783104.pth [2023-12-27 06:13:52,377][20381] Updated weights for policy 1, policy_version 82184 (0.0007) [2023-12-27 06:13:52,436][20381] Updated weights for policy 1, policy_version 82194 (0.0007) [2023-12-27 06:13:52,488][20381] Updated weights for policy 1, policy_version 82204 (0.0007) [2023-12-27 06:13:52,507][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000082208_21053440.pth... [2023-12-27 06:13:52,510][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000080960_20733952.pth [2023-12-27 06:13:52,921][20347] Updated weights for policy 0, policy_version 82367 (0.0006) [2023-12-27 06:13:52,978][20347] Updated weights for policy 0, policy_version 82377 (0.0006) [2023-12-27 06:13:53,033][20347] Updated weights for policy 0, policy_version 82387 (0.0006) [2023-12-27 06:13:53,183][20381] Updated weights for policy 1, policy_version 82214 (0.0006) [2023-12-27 06:13:53,227][20381] Updated weights for policy 1, policy_version 82224 (0.0005) [2023-12-27 06:13:53,271][20381] Updated weights for policy 1, policy_version 82234 (0.0005) [2023-12-27 06:13:53,635][20347] Updated weights for policy 0, policy_version 82397 (0.0005) [2023-12-27 06:13:53,692][20347] Updated weights for policy 0, policy_version 82407 (0.0006) [2023-12-27 06:13:53,751][20347] Updated weights for policy 0, policy_version 82417 (0.0008) [2023-12-27 06:13:53,829][20381] Updated weights for policy 1, policy_version 82244 (0.0005) [2023-12-27 06:13:53,884][20381] Updated weights for policy 1, policy_version 82254 (0.0008) [2023-12-27 06:13:53,940][20381] Updated weights for policy 1, policy_version 82264 (0.0008) [2023-12-27 06:13:54,381][20347] Updated weights for policy 0, policy_version 82427 (0.0009) [2023-12-27 06:13:54,436][20347] Updated weights for policy 0, policy_version 82437 (0.0008) [2023-12-27 06:13:54,481][20347] Updated weights for policy 0, policy_version 82447 (0.0010) [2023-12-27 06:13:54,624][20381] Updated weights for policy 1, policy_version 82274 (0.0008) [2023-12-27 06:13:54,672][20381] Updated weights for policy 1, policy_version 82284 (0.0005) [2023-12-27 06:13:54,718][20381] Updated weights for policy 1, policy_version 82294 (0.0005) [2023-12-27 06:13:54,774][20381] Updated weights for policy 1, policy_version 82304 (0.0005) [2023-12-27 06:13:55,111][20347] Updated weights for policy 0, policy_version 82457 (0.0010) [2023-12-27 06:13:55,163][20347] Updated weights for policy 0, policy_version 82467 (0.0005) [2023-12-27 06:13:55,223][20347] Updated weights for policy 0, policy_version 82477 (0.0006) [2023-12-27 06:13:55,286][20347] Updated weights for policy 0, policy_version 82487 (0.0006) [2023-12-27 06:13:55,393][20381] Updated weights for policy 1, policy_version 82314 (0.0010) [2023-12-27 06:13:55,452][20381] Updated weights for policy 1, policy_version 82324 (0.0010) [2023-12-27 06:13:55,513][20381] Updated weights for policy 1, policy_version 82334 (0.0010) [2023-12-27 06:13:55,856][20347] Updated weights for policy 0, policy_version 82497 (0.0006) [2023-12-27 06:13:55,919][20347] Updated weights for policy 0, policy_version 82507 (0.0005) [2023-12-27 06:13:55,978][20347] Updated weights for policy 0, policy_version 82517 (0.0007) [2023-12-27 06:13:56,164][20381] Updated weights for policy 1, policy_version 82344 (0.0006) [2023-12-27 06:13:56,218][20381] Updated weights for policy 1, policy_version 82354 (0.0010) [2023-12-27 06:13:56,275][20381] Updated weights for policy 1, policy_version 82364 (0.0009) [2023-12-27 06:13:56,519][20347] Updated weights for policy 0, policy_version 82527 (0.0007) [2023-12-27 06:13:56,561][20347] Updated weights for policy 0, policy_version 82537 (0.0007) [2023-12-27 06:13:56,607][20347] Updated weights for policy 0, policy_version 82547 (0.0007) [2023-12-27 06:13:56,912][20381] Updated weights for policy 1, policy_version 82374 (0.0008) [2023-12-27 06:13:56,966][20381] Updated weights for policy 1, policy_version 82384 (0.0010) [2023-12-27 06:13:57,011][20381] Updated weights for policy 1, policy_version 82394 (0.0010) [2023-12-27 06:13:57,285][20347] Updated weights for policy 0, policy_version 82557 (0.0007) [2023-12-27 06:13:57,329][20347] Updated weights for policy 0, policy_version 82567 (0.0005) [2023-12-27 06:13:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 42246144. Throughput: 0: 10386.6, 1: 10623.7. Samples: 42237444. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:13:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:13:57,378][20347] Updated weights for policy 0, policy_version 82577 (0.0006) [2023-12-27 06:13:57,712][20381] Updated weights for policy 1, policy_version 82404 (0.0010) [2023-12-27 06:13:57,770][20381] Updated weights for policy 1, policy_version 82414 (0.0007) [2023-12-27 06:13:57,830][20381] Updated weights for policy 1, policy_version 82424 (0.0010) [2023-12-27 06:13:58,019][20347] Updated weights for policy 0, policy_version 82587 (0.0006) [2023-12-27 06:13:58,068][20347] Updated weights for policy 0, policy_version 82597 (0.0005) [2023-12-27 06:13:58,116][20347] Updated weights for policy 0, policy_version 82607 (0.0005) [2023-12-27 06:13:58,501][20381] Updated weights for policy 1, policy_version 82434 (0.0010) [2023-12-27 06:13:58,565][20381] Updated weights for policy 1, policy_version 82444 (0.0008) [2023-12-27 06:13:58,630][20381] Updated weights for policy 1, policy_version 82454 (0.0011) [2023-12-27 06:13:58,691][20381] Updated weights for policy 1, policy_version 82464 (0.0009) [2023-12-27 06:13:58,847][20347] Updated weights for policy 0, policy_version 82617 (0.0006) [2023-12-27 06:13:58,903][20347] Updated weights for policy 0, policy_version 82627 (0.0005) [2023-12-27 06:13:58,962][20347] Updated weights for policy 0, policy_version 82637 (0.0007) [2023-12-27 06:13:59,018][20347] Updated weights for policy 0, policy_version 82647 (0.0008) [2023-12-27 06:13:59,424][20381] Updated weights for policy 1, policy_version 82474 (0.0007) [2023-12-27 06:13:59,489][20381] Updated weights for policy 1, policy_version 82484 (0.0007) [2023-12-27 06:13:59,541][20381] Updated weights for policy 1, policy_version 82494 (0.0008) [2023-12-27 06:13:59,740][20347] Updated weights for policy 0, policy_version 82657 (0.0007) [2023-12-27 06:13:59,796][20347] Updated weights for policy 0, policy_version 82667 (0.0007) [2023-12-27 06:13:59,860][20347] Updated weights for policy 0, policy_version 82677 (0.0007) [2023-12-27 06:14:00,200][20381] Updated weights for policy 1, policy_version 82504 (0.0008) [2023-12-27 06:14:00,260][20381] Updated weights for policy 1, policy_version 82514 (0.0007) [2023-12-27 06:14:00,320][20381] Updated weights for policy 1, policy_version 82524 (0.0007) [2023-12-27 06:14:00,575][20347] Updated weights for policy 0, policy_version 82687 (0.0006) [2023-12-27 06:14:00,631][20347] Updated weights for policy 0, policy_version 82697 (0.0005) [2023-12-27 06:14:00,684][20347] Updated weights for policy 0, policy_version 82707 (0.0005) [2023-12-27 06:14:00,935][20381] Updated weights for policy 1, policy_version 82534 (0.0006) [2023-12-27 06:14:00,985][20381] Updated weights for policy 1, policy_version 82544 (0.0005) [2023-12-27 06:14:01,049][20381] Updated weights for policy 1, policy_version 82554 (0.0008) [2023-12-27 06:14:01,282][20347] Updated weights for policy 0, policy_version 82717 (0.0006) [2023-12-27 06:14:01,344][20347] Updated weights for policy 0, policy_version 82727 (0.0006) [2023-12-27 06:14:01,409][20347] Updated weights for policy 0, policy_version 82737 (0.0008) [2023-12-27 06:14:01,712][20381] Updated weights for policy 1, policy_version 82564 (0.0008) [2023-12-27 06:14:01,768][20381] Updated weights for policy 1, policy_version 82574 (0.0005) [2023-12-27 06:14:01,825][20381] Updated weights for policy 1, policy_version 82584 (0.0005) [2023-12-27 06:14:02,083][20347] Updated weights for policy 0, policy_version 82747 (0.0007) [2023-12-27 06:14:02,133][20347] Updated weights for policy 0, policy_version 82757 (0.0008) [2023-12-27 06:14:02,179][20347] Updated weights for policy 0, policy_version 82767 (0.0008) [2023-12-27 06:14:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 42352640. Throughput: 0: 10413.2, 1: 10593.8. Samples: 42362120. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:14:02,363][19170] Avg episode reward: [(0, '997.870'), (1, '1000.000')] [2023-12-27 06:14:02,461][20381] Updated weights for policy 1, policy_version 82594 (0.0007) [2023-12-27 06:14:02,510][20381] Updated weights for policy 1, policy_version 82604 (0.0008) [2023-12-27 06:14:02,564][20381] Updated weights for policy 1, policy_version 82614 (0.0008) [2023-12-27 06:14:02,615][20381] Updated weights for policy 1, policy_version 82624 (0.0011) [2023-12-27 06:14:02,830][20347] Updated weights for policy 0, policy_version 82777 (0.0008) [2023-12-27 06:14:02,880][20347] Updated weights for policy 0, policy_version 82787 (0.0005) [2023-12-27 06:14:02,941][20347] Updated weights for policy 0, policy_version 82797 (0.0005) [2023-12-27 06:14:02,996][20347] Updated weights for policy 0, policy_version 82807 (0.0007) [2023-12-27 06:14:03,324][20381] Updated weights for policy 1, policy_version 82634 (0.0007) [2023-12-27 06:14:03,373][20381] Updated weights for policy 1, policy_version 82644 (0.0009) [2023-12-27 06:14:03,425][20381] Updated weights for policy 1, policy_version 82654 (0.0007) [2023-12-27 06:14:03,557][20347] Updated weights for policy 0, policy_version 82817 (0.0007) [2023-12-27 06:14:03,606][20347] Updated weights for policy 0, policy_version 82827 (0.0006) [2023-12-27 06:14:03,667][20347] Updated weights for policy 0, policy_version 82837 (0.0005) [2023-12-27 06:14:04,183][20381] Updated weights for policy 1, policy_version 82664 (0.0006) [2023-12-27 06:14:04,250][20381] Updated weights for policy 1, policy_version 82674 (0.0006) [2023-12-27 06:14:04,309][20381] Updated weights for policy 1, policy_version 82684 (0.0006) [2023-12-27 06:14:04,358][20347] Updated weights for policy 0, policy_version 82847 (0.0007) [2023-12-27 06:14:04,423][20347] Updated weights for policy 0, policy_version 82857 (0.0008) [2023-12-27 06:14:04,471][20347] Updated weights for policy 0, policy_version 82867 (0.0007) [2023-12-27 06:14:04,846][20381] Updated weights for policy 1, policy_version 82694 (0.0005) [2023-12-27 06:14:04,911][20381] Updated weights for policy 1, policy_version 82704 (0.0005) [2023-12-27 06:14:04,971][20381] Updated weights for policy 1, policy_version 82714 (0.0006) [2023-12-27 06:14:05,093][20347] Updated weights for policy 0, policy_version 82877 (0.0008) [2023-12-27 06:14:05,143][20347] Updated weights for policy 0, policy_version 82887 (0.0009) [2023-12-27 06:14:05,193][20347] Updated weights for policy 0, policy_version 82897 (0.0008) [2023-12-27 06:14:05,483][20381] Updated weights for policy 1, policy_version 82724 (0.0005) [2023-12-27 06:14:05,530][20381] Updated weights for policy 1, policy_version 82734 (0.0007) [2023-12-27 06:14:05,575][20381] Updated weights for policy 1, policy_version 82744 (0.0007) [2023-12-27 06:14:05,925][20347] Updated weights for policy 0, policy_version 82907 (0.0008) [2023-12-27 06:14:05,982][20347] Updated weights for policy 0, policy_version 82917 (0.0008) [2023-12-27 06:14:06,038][20347] Updated weights for policy 0, policy_version 82927 (0.0008) [2023-12-27 06:14:06,164][20381] Updated weights for policy 1, policy_version 82754 (0.0008) [2023-12-27 06:14:06,214][20381] Updated weights for policy 1, policy_version 82764 (0.0010) [2023-12-27 06:14:06,264][20381] Updated weights for policy 1, policy_version 82774 (0.0010) [2023-12-27 06:14:06,318][20381] Updated weights for policy 1, policy_version 82784 (0.0010) [2023-12-27 06:14:06,762][20347] Updated weights for policy 0, policy_version 82937 (0.0008) [2023-12-27 06:14:06,824][20347] Updated weights for policy 0, policy_version 82947 (0.0007) [2023-12-27 06:14:06,882][20347] Updated weights for policy 0, policy_version 82957 (0.0007) [2023-12-27 06:14:06,929][20347] Updated weights for policy 0, policy_version 82967 (0.0007) [2023-12-27 06:14:07,041][20381] Updated weights for policy 1, policy_version 82794 (0.0008) [2023-12-27 06:14:07,106][20381] Updated weights for policy 1, policy_version 82804 (0.0007) [2023-12-27 06:14:07,158][20381] Updated weights for policy 1, policy_version 82814 (0.0008) [2023-12-27 06:14:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 42459136. Throughput: 0: 10477.1, 1: 10587.0. Samples: 42427116. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:14:07,363][19170] Avg episode reward: [(0, '997.100'), (1, '1000.000')] [2023-12-27 06:14:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000082968_21250048.pth... [2023-12-27 06:14:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000082816_21209088.pth... [2023-12-27 06:14:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000081752_20938752.pth [2023-12-27 06:14:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000081568_20889600.pth [2023-12-27 06:14:07,589][20347] Updated weights for policy 0, policy_version 82977 (0.0006) [2023-12-27 06:14:07,649][20347] Updated weights for policy 0, policy_version 82987 (0.0005) [2023-12-27 06:14:07,698][20347] Updated weights for policy 0, policy_version 82997 (0.0006) [2023-12-27 06:14:07,855][20381] Updated weights for policy 1, policy_version 82824 (0.0011) [2023-12-27 06:14:07,910][20381] Updated weights for policy 1, policy_version 82834 (0.0011) [2023-12-27 06:14:07,966][20381] Updated weights for policy 1, policy_version 82844 (0.0010) [2023-12-27 06:14:08,323][20347] Updated weights for policy 0, policy_version 83007 (0.0006) [2023-12-27 06:14:08,383][20347] Updated weights for policy 0, policy_version 83017 (0.0006) [2023-12-27 06:14:08,443][20347] Updated weights for policy 0, policy_version 83027 (0.0007) [2023-12-27 06:14:08,650][20381] Updated weights for policy 1, policy_version 82854 (0.0009) [2023-12-27 06:14:08,707][20381] Updated weights for policy 1, policy_version 82864 (0.0009) [2023-12-27 06:14:08,770][20381] Updated weights for policy 1, policy_version 82874 (0.0011) [2023-12-27 06:14:09,021][20347] Updated weights for policy 0, policy_version 83037 (0.0006) [2023-12-27 06:14:09,073][20347] Updated weights for policy 0, policy_version 83047 (0.0005) [2023-12-27 06:14:09,134][20347] Updated weights for policy 0, policy_version 83057 (0.0005) [2023-12-27 06:14:09,447][20381] Updated weights for policy 1, policy_version 82884 (0.0011) [2023-12-27 06:14:09,498][20381] Updated weights for policy 1, policy_version 82894 (0.0010) [2023-12-27 06:14:09,551][20381] Updated weights for policy 1, policy_version 82904 (0.0010) [2023-12-27 06:14:09,785][20347] Updated weights for policy 0, policy_version 83067 (0.0005) [2023-12-27 06:14:09,852][20347] Updated weights for policy 0, policy_version 83077 (0.0007) [2023-12-27 06:14:09,916][20347] Updated weights for policy 0, policy_version 83087 (0.0007) [2023-12-27 06:14:10,206][20381] Updated weights for policy 1, policy_version 82914 (0.0011) [2023-12-27 06:14:10,268][20381] Updated weights for policy 1, policy_version 82924 (0.0011) [2023-12-27 06:14:10,323][20381] Updated weights for policy 1, policy_version 82934 (0.0010) [2023-12-27 06:14:10,382][20381] Updated weights for policy 1, policy_version 82944 (0.0011) [2023-12-27 06:14:10,544][20347] Updated weights for policy 0, policy_version 83097 (0.0006) [2023-12-27 06:14:10,595][20347] Updated weights for policy 0, policy_version 83107 (0.0006) [2023-12-27 06:14:10,646][20347] Updated weights for policy 0, policy_version 83117 (0.0005) [2023-12-27 06:14:10,695][20347] Updated weights for policy 0, policy_version 83127 (0.0005) [2023-12-27 06:14:11,024][20381] Updated weights for policy 1, policy_version 82954 (0.0011) [2023-12-27 06:14:11,086][20381] Updated weights for policy 1, policy_version 82964 (0.0009) [2023-12-27 06:14:11,132][20381] Updated weights for policy 1, policy_version 82974 (0.0010) [2023-12-27 06:14:11,298][20347] Updated weights for policy 0, policy_version 83137 (0.0008) [2023-12-27 06:14:11,360][20347] Updated weights for policy 0, policy_version 83147 (0.0008) [2023-12-27 06:14:11,419][20347] Updated weights for policy 0, policy_version 83157 (0.0009) [2023-12-27 06:14:11,858][20381] Updated weights for policy 1, policy_version 82984 (0.0010) [2023-12-27 06:14:11,915][20381] Updated weights for policy 1, policy_version 82994 (0.0011) [2023-12-27 06:14:11,980][20381] Updated weights for policy 1, policy_version 83004 (0.0011) [2023-12-27 06:14:12,127][20347] Updated weights for policy 0, policy_version 83167 (0.0007) [2023-12-27 06:14:12,182][20347] Updated weights for policy 0, policy_version 83177 (0.0010) [2023-12-27 06:14:12,235][20347] Updated weights for policy 0, policy_version 83187 (0.0009) [2023-12-27 06:14:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 42565632. Throughput: 0: 10533.1, 1: 10564.5. Samples: 42552848. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:14:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:12,651][20381] Updated weights for policy 1, policy_version 83014 (0.0011) [2023-12-27 06:14:12,694][20381] Updated weights for policy 1, policy_version 83024 (0.0007) [2023-12-27 06:14:12,745][20381] Updated weights for policy 1, policy_version 83034 (0.0008) [2023-12-27 06:14:12,904][20347] Updated weights for policy 0, policy_version 83197 (0.0005) [2023-12-27 06:14:12,967][20347] Updated weights for policy 0, policy_version 83207 (0.0005) [2023-12-27 06:14:13,028][20347] Updated weights for policy 0, policy_version 83217 (0.0005) [2023-12-27 06:14:13,525][20381] Updated weights for policy 1, policy_version 83044 (0.0007) [2023-12-27 06:14:13,566][20347] Updated weights for policy 0, policy_version 83227 (0.0005) [2023-12-27 06:14:13,573][20381] Updated weights for policy 1, policy_version 83054 (0.0007) [2023-12-27 06:14:13,623][20347] Updated weights for policy 0, policy_version 83237 (0.0005) [2023-12-27 06:14:13,624][20381] Updated weights for policy 1, policy_version 83064 (0.0007) [2023-12-27 06:14:13,677][20347] Updated weights for policy 0, policy_version 83247 (0.0005) [2023-12-27 06:14:14,246][20347] Updated weights for policy 0, policy_version 83257 (0.0005) [2023-12-27 06:14:14,289][20381] Updated weights for policy 1, policy_version 83074 (0.0008) [2023-12-27 06:14:14,298][20347] Updated weights for policy 0, policy_version 83267 (0.0007) [2023-12-27 06:14:14,344][20381] Updated weights for policy 1, policy_version 83084 (0.0007) [2023-12-27 06:14:14,352][20347] Updated weights for policy 0, policy_version 83277 (0.0005) [2023-12-27 06:14:14,397][20381] Updated weights for policy 1, policy_version 83094 (0.0008) [2023-12-27 06:14:14,405][20347] Updated weights for policy 0, policy_version 83287 (0.0010) [2023-12-27 06:14:14,449][20381] Updated weights for policy 1, policy_version 83104 (0.0009) [2023-12-27 06:14:15,097][20347] Updated weights for policy 0, policy_version 83297 (0.0010) [2023-12-27 06:14:15,146][20347] Updated weights for policy 0, policy_version 83307 (0.0007) [2023-12-27 06:14:15,161][20381] Updated weights for policy 1, policy_version 83114 (0.0011) [2023-12-27 06:14:15,204][20347] Updated weights for policy 0, policy_version 83317 (0.0007) [2023-12-27 06:14:15,218][20381] Updated weights for policy 1, policy_version 83124 (0.0011) [2023-12-27 06:14:15,275][20381] Updated weights for policy 1, policy_version 83134 (0.0011) [2023-12-27 06:14:15,752][20347] Updated weights for policy 0, policy_version 83327 (0.0007) [2023-12-27 06:14:15,800][20347] Updated weights for policy 0, policy_version 83337 (0.0007) [2023-12-27 06:14:15,852][20347] Updated weights for policy 0, policy_version 83347 (0.0008) [2023-12-27 06:14:15,971][20381] Updated weights for policy 1, policy_version 83144 (0.0007) [2023-12-27 06:14:16,030][20381] Updated weights for policy 1, policy_version 83154 (0.0009) [2023-12-27 06:14:16,088][20381] Updated weights for policy 1, policy_version 83164 (0.0011) [2023-12-27 06:14:16,492][20347] Updated weights for policy 0, policy_version 83357 (0.0007) [2023-12-27 06:14:16,543][20347] Updated weights for policy 0, policy_version 83367 (0.0010) [2023-12-27 06:14:16,594][20347] Updated weights for policy 0, policy_version 83377 (0.0010) [2023-12-27 06:14:16,746][20381] Updated weights for policy 1, policy_version 83174 (0.0010) [2023-12-27 06:14:16,803][20381] Updated weights for policy 1, policy_version 83184 (0.0010) [2023-12-27 06:14:16,860][20381] Updated weights for policy 1, policy_version 83194 (0.0010) [2023-12-27 06:14:17,287][20347] Updated weights for policy 0, policy_version 83387 (0.0009) [2023-12-27 06:14:17,339][20347] Updated weights for policy 0, policy_version 83397 (0.0009) [2023-12-27 06:14:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 42663936. Throughput: 0: 10632.0, 1: 10477.6. Samples: 42678804. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:14:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:17,393][20347] Updated weights for policy 0, policy_version 83407 (0.0010) [2023-12-27 06:14:17,484][20381] Updated weights for policy 1, policy_version 83204 (0.0008) [2023-12-27 06:14:17,541][20381] Updated weights for policy 1, policy_version 83214 (0.0006) [2023-12-27 06:14:17,589][20381] Updated weights for policy 1, policy_version 83224 (0.0008) [2023-12-27 06:14:17,999][20347] Updated weights for policy 0, policy_version 83417 (0.0010) [2023-12-27 06:14:18,056][20347] Updated weights for policy 0, policy_version 83427 (0.0008) [2023-12-27 06:14:18,116][20347] Updated weights for policy 0, policy_version 83437 (0.0009) [2023-12-27 06:14:18,183][20347] Updated weights for policy 0, policy_version 83447 (0.0008) [2023-12-27 06:14:18,253][20381] Updated weights for policy 1, policy_version 83234 (0.0007) [2023-12-27 06:14:18,318][20381] Updated weights for policy 1, policy_version 83244 (0.0006) [2023-12-27 06:14:18,379][20381] Updated weights for policy 1, policy_version 83254 (0.0008) [2023-12-27 06:14:18,441][20381] Updated weights for policy 1, policy_version 83264 (0.0008) [2023-12-27 06:14:18,863][20347] Updated weights for policy 0, policy_version 83457 (0.0008) [2023-12-27 06:14:18,929][20347] Updated weights for policy 0, policy_version 83467 (0.0008) [2023-12-27 06:14:18,982][20347] Updated weights for policy 0, policy_version 83477 (0.0008) [2023-12-27 06:14:19,173][20381] Updated weights for policy 1, policy_version 83274 (0.0008) [2023-12-27 06:14:19,233][20381] Updated weights for policy 1, policy_version 83284 (0.0008) [2023-12-27 06:14:19,290][20381] Updated weights for policy 1, policy_version 83294 (0.0011) [2023-12-27 06:14:19,636][20347] Updated weights for policy 0, policy_version 83487 (0.0007) [2023-12-27 06:14:19,696][20347] Updated weights for policy 0, policy_version 83497 (0.0006) [2023-12-27 06:14:19,758][20347] Updated weights for policy 0, policy_version 83507 (0.0008) [2023-12-27 06:14:20,046][20381] Updated weights for policy 1, policy_version 83304 (0.0008) [2023-12-27 06:14:20,113][20381] Updated weights for policy 1, policy_version 83314 (0.0011) [2023-12-27 06:14:20,175][20381] Updated weights for policy 1, policy_version 83324 (0.0011) [2023-12-27 06:14:20,398][20347] Updated weights for policy 0, policy_version 83517 (0.0010) [2023-12-27 06:14:20,458][20347] Updated weights for policy 0, policy_version 83527 (0.0010) [2023-12-27 06:14:20,513][20347] Updated weights for policy 0, policy_version 83537 (0.0010) [2023-12-27 06:14:20,820][20381] Updated weights for policy 1, policy_version 83334 (0.0009) [2023-12-27 06:14:20,886][20381] Updated weights for policy 1, policy_version 83344 (0.0007) [2023-12-27 06:14:20,944][20381] Updated weights for policy 1, policy_version 83354 (0.0008) [2023-12-27 06:14:21,222][20347] Updated weights for policy 0, policy_version 83547 (0.0007) [2023-12-27 06:14:21,281][20347] Updated weights for policy 0, policy_version 83557 (0.0008) [2023-12-27 06:14:21,336][20347] Updated weights for policy 0, policy_version 83567 (0.0008) [2023-12-27 06:14:21,582][20381] Updated weights for policy 1, policy_version 83364 (0.0007) [2023-12-27 06:14:21,648][20381] Updated weights for policy 1, policy_version 83374 (0.0008) [2023-12-27 06:14:21,705][20381] Updated weights for policy 1, policy_version 83384 (0.0008) [2023-12-27 06:14:22,041][20347] Updated weights for policy 0, policy_version 83577 (0.0007) [2023-12-27 06:14:22,096][20347] Updated weights for policy 0, policy_version 83587 (0.0007) [2023-12-27 06:14:22,142][20347] Updated weights for policy 0, policy_version 83597 (0.0007) [2023-12-27 06:14:22,197][20347] Updated weights for policy 0, policy_version 83607 (0.0005) [2023-12-27 06:14:22,319][20381] Updated weights for policy 1, policy_version 83394 (0.0006) [2023-12-27 06:14:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 21162.7, 300 sec: 20799.3). Total num frames: 42770432. Throughput: 0: 10617.3, 1: 10424.2. Samples: 42739996. Policy #0 lag: (min: 1.0, avg: 26.5, max: 33.0) [2023-12-27 06:14:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000083608_21413888.pth... [2023-12-27 06:14:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000082360_21094400.pth [2023-12-27 06:14:22,383][20381] Updated weights for policy 1, policy_version 83404 (0.0007) [2023-12-27 06:14:22,444][20381] Updated weights for policy 1, policy_version 83414 (0.0008) [2023-12-27 06:14:22,498][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000083424_21364736.pth... [2023-12-27 06:14:22,500][20381] Updated weights for policy 1, policy_version 83424 (0.0007) [2023-12-27 06:14:22,501][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000082208_21053440.pth [2023-12-27 06:14:22,879][20347] Updated weights for policy 0, policy_version 83617 (0.0007) [2023-12-27 06:14:22,938][20347] Updated weights for policy 0, policy_version 83627 (0.0008) [2023-12-27 06:14:22,988][20347] Updated weights for policy 0, policy_version 83637 (0.0009) [2023-12-27 06:14:23,123][20381] Updated weights for policy 1, policy_version 83434 (0.0008) [2023-12-27 06:14:23,174][20381] Updated weights for policy 1, policy_version 83444 (0.0007) [2023-12-27 06:14:23,226][20381] Updated weights for policy 1, policy_version 83454 (0.0008) [2023-12-27 06:14:23,591][20347] Updated weights for policy 0, policy_version 83647 (0.0008) [2023-12-27 06:14:23,652][20347] Updated weights for policy 0, policy_version 83657 (0.0007) [2023-12-27 06:14:23,710][20347] Updated weights for policy 0, policy_version 83667 (0.0008) [2023-12-27 06:14:23,900][20381] Updated weights for policy 1, policy_version 83464 (0.0007) [2023-12-27 06:14:23,955][20381] Updated weights for policy 1, policy_version 83474 (0.0007) [2023-12-27 06:14:24,008][20381] Updated weights for policy 1, policy_version 83484 (0.0008) [2023-12-27 06:14:24,268][20347] Updated weights for policy 0, policy_version 83677 (0.0007) [2023-12-27 06:14:24,328][20347] Updated weights for policy 0, policy_version 83687 (0.0008) [2023-12-27 06:14:24,389][20347] Updated weights for policy 0, policy_version 83697 (0.0007) [2023-12-27 06:14:24,714][20381] Updated weights for policy 1, policy_version 83494 (0.0008) [2023-12-27 06:14:24,770][20381] Updated weights for policy 1, policy_version 83504 (0.0008) [2023-12-27 06:14:24,823][20381] Updated weights for policy 1, policy_version 83514 (0.0007) [2023-12-27 06:14:25,036][20347] Updated weights for policy 0, policy_version 83707 (0.0008) [2023-12-27 06:14:25,085][20347] Updated weights for policy 0, policy_version 83717 (0.0007) [2023-12-27 06:14:25,148][20347] Updated weights for policy 0, policy_version 83727 (0.0008) [2023-12-27 06:14:25,412][20381] Updated weights for policy 1, policy_version 83524 (0.0008) [2023-12-27 06:14:25,456][20381] Updated weights for policy 1, policy_version 83534 (0.0007) [2023-12-27 06:14:25,501][20381] Updated weights for policy 1, policy_version 83544 (0.0008) [2023-12-27 06:14:25,858][20347] Updated weights for policy 0, policy_version 83737 (0.0008) [2023-12-27 06:14:25,919][20347] Updated weights for policy 0, policy_version 83747 (0.0007) [2023-12-27 06:14:25,970][20347] Updated weights for policy 0, policy_version 83757 (0.0008) [2023-12-27 06:14:26,019][20347] Updated weights for policy 0, policy_version 83767 (0.0008) [2023-12-27 06:14:26,212][20381] Updated weights for policy 1, policy_version 83554 (0.0008) [2023-12-27 06:14:26,271][20381] Updated weights for policy 1, policy_version 83564 (0.0008) [2023-12-27 06:14:26,332][20381] Updated weights for policy 1, policy_version 83574 (0.0008) [2023-12-27 06:14:26,387][20381] Updated weights for policy 1, policy_version 83584 (0.0008) [2023-12-27 06:14:26,708][20347] Updated weights for policy 0, policy_version 83777 (0.0007) [2023-12-27 06:14:26,756][20347] Updated weights for policy 0, policy_version 83787 (0.0007) [2023-12-27 06:14:26,811][20347] Updated weights for policy 0, policy_version 83797 (0.0007) [2023-12-27 06:14:27,073][20381] Updated weights for policy 1, policy_version 83594 (0.0008) [2023-12-27 06:14:27,131][20381] Updated weights for policy 1, policy_version 83604 (0.0007) [2023-12-27 06:14:27,184][20381] Updated weights for policy 1, policy_version 83614 (0.0008) [2023-12-27 06:14:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 42876928. Throughput: 0: 10637.3, 1: 10429.3. Samples: 42866160. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:27,514][20347] Updated weights for policy 0, policy_version 83807 (0.0007) [2023-12-27 06:14:27,567][20347] Updated weights for policy 0, policy_version 83817 (0.0007) [2023-12-27 06:14:27,619][20347] Updated weights for policy 0, policy_version 83827 (0.0007) [2023-12-27 06:14:27,870][20381] Updated weights for policy 1, policy_version 83624 (0.0008) [2023-12-27 06:14:27,924][20381] Updated weights for policy 1, policy_version 83634 (0.0007) [2023-12-27 06:14:27,975][20381] Updated weights for policy 1, policy_version 83644 (0.0008) [2023-12-27 06:14:28,326][20347] Updated weights for policy 0, policy_version 83837 (0.0008) [2023-12-27 06:14:28,381][20347] Updated weights for policy 0, policy_version 83847 (0.0007) [2023-12-27 06:14:28,436][20347] Updated weights for policy 0, policy_version 83857 (0.0005) [2023-12-27 06:14:28,673][20381] Updated weights for policy 1, policy_version 83654 (0.0008) [2023-12-27 06:14:28,722][20381] Updated weights for policy 1, policy_version 83664 (0.0009) [2023-12-27 06:14:28,783][20381] Updated weights for policy 1, policy_version 83674 (0.0008) [2023-12-27 06:14:29,017][20347] Updated weights for policy 0, policy_version 83867 (0.0006) [2023-12-27 06:14:29,076][20347] Updated weights for policy 0, policy_version 83877 (0.0006) [2023-12-27 06:14:29,129][20347] Updated weights for policy 0, policy_version 83887 (0.0006) [2023-12-27 06:14:29,486][20381] Updated weights for policy 1, policy_version 83684 (0.0008) [2023-12-27 06:14:29,547][20381] Updated weights for policy 1, policy_version 83694 (0.0008) [2023-12-27 06:14:29,608][20381] Updated weights for policy 1, policy_version 83704 (0.0007) [2023-12-27 06:14:29,728][20347] Updated weights for policy 0, policy_version 83897 (0.0010) [2023-12-27 06:14:29,782][20347] Updated weights for policy 0, policy_version 83907 (0.0007) [2023-12-27 06:14:29,846][20347] Updated weights for policy 0, policy_version 83917 (0.0008) [2023-12-27 06:14:29,901][20347] Updated weights for policy 0, policy_version 83927 (0.0006) [2023-12-27 06:14:30,247][20381] Updated weights for policy 1, policy_version 83714 (0.0008) [2023-12-27 06:14:30,308][20381] Updated weights for policy 1, policy_version 83724 (0.0005) [2023-12-27 06:14:30,362][20381] Updated weights for policy 1, policy_version 83734 (0.0006) [2023-12-27 06:14:30,419][20381] Updated weights for policy 1, policy_version 83744 (0.0005) [2023-12-27 06:14:30,552][20347] Updated weights for policy 0, policy_version 83937 (0.0005) [2023-12-27 06:14:30,603][20347] Updated weights for policy 0, policy_version 83947 (0.0005) [2023-12-27 06:14:30,657][20347] Updated weights for policy 0, policy_version 83957 (0.0005) [2023-12-27 06:14:31,059][20381] Updated weights for policy 1, policy_version 83754 (0.0007) [2023-12-27 06:14:31,124][20381] Updated weights for policy 1, policy_version 83764 (0.0009) [2023-12-27 06:14:31,185][20381] Updated weights for policy 1, policy_version 83774 (0.0007) [2023-12-27 06:14:31,252][20347] Updated weights for policy 0, policy_version 83967 (0.0007) [2023-12-27 06:14:31,315][20347] Updated weights for policy 0, policy_version 83977 (0.0007) [2023-12-27 06:14:31,380][20347] Updated weights for policy 0, policy_version 83987 (0.0008) [2023-12-27 06:14:31,877][20381] Updated weights for policy 1, policy_version 83784 (0.0008) [2023-12-27 06:14:31,930][20381] Updated weights for policy 1, policy_version 83794 (0.0008) [2023-12-27 06:14:31,980][20381] Updated weights for policy 1, policy_version 83804 (0.0008) [2023-12-27 06:14:32,114][20347] Updated weights for policy 0, policy_version 83997 (0.0007) [2023-12-27 06:14:32,162][20347] Updated weights for policy 0, policy_version 84007 (0.0008) [2023-12-27 06:14:32,207][20347] Updated weights for policy 0, policy_version 84017 (0.0008) [2023-12-27 06:14:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 20799.4). Total num frames: 42983424. Throughput: 0: 10617.2, 1: 10384.3. Samples: 42991616. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:32,588][20381] Updated weights for policy 1, policy_version 83814 (0.0009) [2023-12-27 06:14:32,646][20381] Updated weights for policy 1, policy_version 83824 (0.0010) [2023-12-27 06:14:32,704][20381] Updated weights for policy 1, policy_version 83834 (0.0010) [2023-12-27 06:14:32,917][20347] Updated weights for policy 0, policy_version 84027 (0.0009) [2023-12-27 06:14:32,979][20347] Updated weights for policy 0, policy_version 84037 (0.0010) [2023-12-27 06:14:33,034][20347] Updated weights for policy 0, policy_version 84047 (0.0010) [2023-12-27 06:14:33,354][20381] Updated weights for policy 1, policy_version 83844 (0.0009) [2023-12-27 06:14:33,414][20381] Updated weights for policy 1, policy_version 83854 (0.0008) [2023-12-27 06:14:33,468][20381] Updated weights for policy 1, policy_version 83864 (0.0007) [2023-12-27 06:14:33,649][20347] Updated weights for policy 0, policy_version 84057 (0.0010) [2023-12-27 06:14:33,703][20347] Updated weights for policy 0, policy_version 84067 (0.0007) [2023-12-27 06:14:33,760][20347] Updated weights for policy 0, policy_version 84077 (0.0010) [2023-12-27 06:14:33,820][20347] Updated weights for policy 0, policy_version 84087 (0.0010) [2023-12-27 06:14:34,102][20381] Updated weights for policy 1, policy_version 83874 (0.0008) [2023-12-27 06:14:34,162][20381] Updated weights for policy 1, policy_version 83884 (0.0007) [2023-12-27 06:14:34,220][20381] Updated weights for policy 1, policy_version 83894 (0.0007) [2023-12-27 06:14:34,272][20381] Updated weights for policy 1, policy_version 83904 (0.0008) [2023-12-27 06:14:34,510][20347] Updated weights for policy 0, policy_version 84097 (0.0008) [2023-12-27 06:14:34,572][20347] Updated weights for policy 0, policy_version 84107 (0.0010) [2023-12-27 06:14:34,633][20347] Updated weights for policy 0, policy_version 84117 (0.0009) [2023-12-27 06:14:34,941][20381] Updated weights for policy 1, policy_version 83914 (0.0005) [2023-12-27 06:14:34,992][20381] Updated weights for policy 1, policy_version 83924 (0.0008) [2023-12-27 06:14:35,041][20381] Updated weights for policy 1, policy_version 83934 (0.0008) [2023-12-27 06:14:35,170][20347] Updated weights for policy 0, policy_version 84127 (0.0005) [2023-12-27 06:14:35,223][20347] Updated weights for policy 0, policy_version 84137 (0.0005) [2023-12-27 06:14:35,281][20347] Updated weights for policy 0, policy_version 84147 (0.0007) [2023-12-27 06:14:35,687][20381] Updated weights for policy 1, policy_version 83944 (0.0008) [2023-12-27 06:14:35,738][20381] Updated weights for policy 1, policy_version 83954 (0.0005) [2023-12-27 06:14:35,794][20381] Updated weights for policy 1, policy_version 83964 (0.0007) [2023-12-27 06:14:35,926][20347] Updated weights for policy 0, policy_version 84157 (0.0008) [2023-12-27 06:14:35,972][20347] Updated weights for policy 0, policy_version 84167 (0.0005) [2023-12-27 06:14:36,026][20347] Updated weights for policy 0, policy_version 84177 (0.0007) [2023-12-27 06:14:36,478][20381] Updated weights for policy 1, policy_version 83974 (0.0007) [2023-12-27 06:14:36,536][20381] Updated weights for policy 1, policy_version 83984 (0.0008) [2023-12-27 06:14:36,591][20381] Updated weights for policy 1, policy_version 83994 (0.0006) [2023-12-27 06:14:36,668][20347] Updated weights for policy 0, policy_version 84187 (0.0007) [2023-12-27 06:14:36,720][20347] Updated weights for policy 0, policy_version 84197 (0.0009) [2023-12-27 06:14:36,780][20347] Updated weights for policy 0, policy_version 84207 (0.0007) [2023-12-27 06:14:37,167][20381] Updated weights for policy 1, policy_version 84004 (0.0006) [2023-12-27 06:14:37,213][20381] Updated weights for policy 1, policy_version 84014 (0.0008) [2023-12-27 06:14:37,264][20381] Updated weights for policy 1, policy_version 84024 (0.0008) [2023-12-27 06:14:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 43089920. Throughput: 0: 10651.7, 1: 10383.0. Samples: 43055524. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:14:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000084032_21520384.pth... [2023-12-27 06:14:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000084216_21569536.pth... [2023-12-27 06:14:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000082968_21250048.pth [2023-12-27 06:14:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000082816_21209088.pth [2023-12-27 06:14:37,495][20347] Updated weights for policy 0, policy_version 84217 (0.0007) [2023-12-27 06:14:37,557][20347] Updated weights for policy 0, policy_version 84227 (0.0006) [2023-12-27 06:14:37,619][20347] Updated weights for policy 0, policy_version 84237 (0.0006) [2023-12-27 06:14:37,678][20347] Updated weights for policy 0, policy_version 84247 (0.0006) [2023-12-27 06:14:37,901][20381] Updated weights for policy 1, policy_version 84034 (0.0008) [2023-12-27 06:14:37,957][20381] Updated weights for policy 1, policy_version 84044 (0.0007) [2023-12-27 06:14:38,010][20381] Updated weights for policy 1, policy_version 84054 (0.0008) [2023-12-27 06:14:38,066][20381] Updated weights for policy 1, policy_version 84064 (0.0008) [2023-12-27 06:14:38,273][20347] Updated weights for policy 0, policy_version 84257 (0.0006) [2023-12-27 06:14:38,332][20347] Updated weights for policy 0, policy_version 84267 (0.0007) [2023-12-27 06:14:38,392][20347] Updated weights for policy 0, policy_version 84277 (0.0006) [2023-12-27 06:14:38,776][20381] Updated weights for policy 1, policy_version 84074 (0.0007) [2023-12-27 06:14:38,831][20381] Updated weights for policy 1, policy_version 84084 (0.0008) [2023-12-27 06:14:38,893][20381] Updated weights for policy 1, policy_version 84094 (0.0007) [2023-12-27 06:14:39,083][20347] Updated weights for policy 0, policy_version 84287 (0.0007) [2023-12-27 06:14:39,143][20347] Updated weights for policy 0, policy_version 84297 (0.0008) [2023-12-27 06:14:39,194][20347] Updated weights for policy 0, policy_version 84307 (0.0007) [2023-12-27 06:14:39,673][20381] Updated weights for policy 1, policy_version 84104 (0.0007) [2023-12-27 06:14:39,723][20381] Updated weights for policy 1, policy_version 84114 (0.0007) [2023-12-27 06:14:39,777][20381] Updated weights for policy 1, policy_version 84124 (0.0008) [2023-12-27 06:14:39,949][20347] Updated weights for policy 0, policy_version 84317 (0.0008) [2023-12-27 06:14:40,011][20347] Updated weights for policy 0, policy_version 84327 (0.0008) [2023-12-27 06:14:40,073][20347] Updated weights for policy 0, policy_version 84337 (0.0008) [2023-12-27 06:14:40,490][20381] Updated weights for policy 1, policy_version 84134 (0.0009) [2023-12-27 06:14:40,541][20381] Updated weights for policy 1, policy_version 84144 (0.0010) [2023-12-27 06:14:40,594][20381] Updated weights for policy 1, policy_version 84154 (0.0010) [2023-12-27 06:14:40,777][20347] Updated weights for policy 0, policy_version 84347 (0.0007) [2023-12-27 06:14:40,831][20347] Updated weights for policy 0, policy_version 84357 (0.0008) [2023-12-27 06:14:40,881][20347] Updated weights for policy 0, policy_version 84367 (0.0007) [2023-12-27 06:14:41,306][20381] Updated weights for policy 1, policy_version 84164 (0.0009) [2023-12-27 06:14:41,370][20381] Updated weights for policy 1, policy_version 84174 (0.0010) [2023-12-27 06:14:41,433][20381] Updated weights for policy 1, policy_version 84184 (0.0007) [2023-12-27 06:14:41,581][20347] Updated weights for policy 0, policy_version 84377 (0.0007) [2023-12-27 06:14:41,647][20347] Updated weights for policy 0, policy_version 84387 (0.0007) [2023-12-27 06:14:41,713][20347] Updated weights for policy 0, policy_version 84397 (0.0008) [2023-12-27 06:14:41,763][20347] Updated weights for policy 0, policy_version 84407 (0.0007) [2023-12-27 06:14:42,135][20381] Updated weights for policy 1, policy_version 84194 (0.0010) [2023-12-27 06:14:42,187][20381] Updated weights for policy 1, policy_version 84204 (0.0010) [2023-12-27 06:14:42,232][20381] Updated weights for policy 1, policy_version 84214 (0.0010) [2023-12-27 06:14:42,296][20381] Updated weights for policy 1, policy_version 84224 (0.0011) [2023-12-27 06:14:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 43188224. Throughput: 0: 10573.3, 1: 10330.6. Samples: 43178124. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:42,363][19170] Avg episode reward: [(0, '994.780'), (1, '1000.000')] [2023-12-27 06:14:42,406][20347] Updated weights for policy 0, policy_version 84417 (0.0009) [2023-12-27 06:14:42,470][20347] Updated weights for policy 0, policy_version 84427 (0.0008) [2023-12-27 06:14:42,534][20347] Updated weights for policy 0, policy_version 84437 (0.0008) [2023-12-27 06:14:42,971][20381] Updated weights for policy 1, policy_version 84234 (0.0010) [2023-12-27 06:14:43,016][20381] Updated weights for policy 1, policy_version 84244 (0.0010) [2023-12-27 06:14:43,068][20381] Updated weights for policy 1, policy_version 84254 (0.0010) [2023-12-27 06:14:43,114][20347] Updated weights for policy 0, policy_version 84447 (0.0007) [2023-12-27 06:14:43,166][20347] Updated weights for policy 0, policy_version 84457 (0.0008) [2023-12-27 06:14:43,219][20347] Updated weights for policy 0, policy_version 84467 (0.0006) [2023-12-27 06:14:43,673][20381] Updated weights for policy 1, policy_version 84264 (0.0010) [2023-12-27 06:14:43,721][20381] Updated weights for policy 1, policy_version 84274 (0.0010) [2023-12-27 06:14:43,772][20381] Updated weights for policy 1, policy_version 84284 (0.0010) [2023-12-27 06:14:43,779][20347] Updated weights for policy 0, policy_version 84477 (0.0005) [2023-12-27 06:14:43,834][20347] Updated weights for policy 0, policy_version 84487 (0.0005) [2023-12-27 06:14:43,880][20347] Updated weights for policy 0, policy_version 84497 (0.0005) [2023-12-27 06:14:44,392][20381] Updated weights for policy 1, policy_version 84294 (0.0007) [2023-12-27 06:14:44,404][20347] Updated weights for policy 0, policy_version 84507 (0.0007) [2023-12-27 06:14:44,444][20381] Updated weights for policy 1, policy_version 84304 (0.0008) [2023-12-27 06:14:44,459][20347] Updated weights for policy 0, policy_version 84517 (0.0010) [2023-12-27 06:14:44,493][20381] Updated weights for policy 1, policy_version 84314 (0.0007) [2023-12-27 06:14:44,510][20347] Updated weights for policy 0, policy_version 84527 (0.0010) [2023-12-27 06:14:45,181][20381] Updated weights for policy 1, policy_version 84324 (0.0011) [2023-12-27 06:14:45,205][20347] Updated weights for policy 0, policy_version 84537 (0.0007) [2023-12-27 06:14:45,236][20381] Updated weights for policy 1, policy_version 84334 (0.0010) [2023-12-27 06:14:45,262][20347] Updated weights for policy 0, policy_version 84547 (0.0006) [2023-12-27 06:14:45,295][20381] Updated weights for policy 1, policy_version 84344 (0.0010) [2023-12-27 06:14:45,321][20347] Updated weights for policy 0, policy_version 84557 (0.0005) [2023-12-27 06:14:45,385][20347] Updated weights for policy 0, policy_version 84567 (0.0010) [2023-12-27 06:14:45,979][20381] Updated weights for policy 1, policy_version 84354 (0.0010) [2023-12-27 06:14:46,022][20347] Updated weights for policy 0, policy_version 84577 (0.0010) [2023-12-27 06:14:46,024][20381] Updated weights for policy 1, policy_version 84364 (0.0010) [2023-12-27 06:14:46,069][20381] Updated weights for policy 1, policy_version 84374 (0.0010) [2023-12-27 06:14:46,079][20347] Updated weights for policy 0, policy_version 84587 (0.0010) [2023-12-27 06:14:46,117][20381] Updated weights for policy 1, policy_version 84384 (0.0010) [2023-12-27 06:14:46,137][20347] Updated weights for policy 0, policy_version 84597 (0.0010) [2023-12-27 06:14:46,797][20347] Updated weights for policy 0, policy_version 84607 (0.0008) [2023-12-27 06:14:46,846][20381] Updated weights for policy 1, policy_version 84394 (0.0007) [2023-12-27 06:14:46,862][20347] Updated weights for policy 0, policy_version 84617 (0.0009) [2023-12-27 06:14:46,895][20381] Updated weights for policy 1, policy_version 84404 (0.0008) [2023-12-27 06:14:46,910][20347] Updated weights for policy 0, policy_version 84627 (0.0010) [2023-12-27 06:14:46,952][20381] Updated weights for policy 1, policy_version 84414 (0.0008) [2023-12-27 06:14:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 43294720. Throughput: 0: 10628.3, 1: 10371.1. Samples: 43307092. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:47,363][19170] Avg episode reward: [(0, '958.720'), (1, '1000.000')] [2023-12-27 06:14:47,521][20381] Updated weights for policy 1, policy_version 84424 (0.0005) [2023-12-27 06:14:47,565][20347] Updated weights for policy 0, policy_version 84637 (0.0010) [2023-12-27 06:14:47,570][20381] Updated weights for policy 1, policy_version 84434 (0.0008) [2023-12-27 06:14:47,626][20347] Updated weights for policy 0, policy_version 84647 (0.0010) [2023-12-27 06:14:47,630][20381] Updated weights for policy 1, policy_version 84444 (0.0005) [2023-12-27 06:14:47,686][20347] Updated weights for policy 0, policy_version 84657 (0.0008) [2023-12-27 06:14:48,279][20381] Updated weights for policy 1, policy_version 84454 (0.0009) [2023-12-27 06:14:48,333][20381] Updated weights for policy 1, policy_version 84464 (0.0006) [2023-12-27 06:14:48,350][20347] Updated weights for policy 0, policy_version 84667 (0.0007) [2023-12-27 06:14:48,390][20381] Updated weights for policy 1, policy_version 84474 (0.0007) [2023-12-27 06:14:48,409][20347] Updated weights for policy 0, policy_version 84677 (0.0006) [2023-12-27 06:14:48,467][20347] Updated weights for policy 0, policy_version 84687 (0.0007) [2023-12-27 06:14:48,992][20381] Updated weights for policy 1, policy_version 84484 (0.0008) [2023-12-27 06:14:49,042][20381] Updated weights for policy 1, policy_version 84494 (0.0011) [2023-12-27 06:14:49,089][20381] Updated weights for policy 1, policy_version 84504 (0.0011) [2023-12-27 06:14:49,182][20347] Updated weights for policy 0, policy_version 84697 (0.0009) [2023-12-27 06:14:49,241][20347] Updated weights for policy 0, policy_version 84707 (0.0007) [2023-12-27 06:14:49,306][20347] Updated weights for policy 0, policy_version 84717 (0.0008) [2023-12-27 06:14:49,371][20347] Updated weights for policy 0, policy_version 84727 (0.0008) [2023-12-27 06:14:49,815][20381] Updated weights for policy 1, policy_version 84514 (0.0007) [2023-12-27 06:14:49,876][20381] Updated weights for policy 1, policy_version 84524 (0.0007) [2023-12-27 06:14:49,942][20381] Updated weights for policy 1, policy_version 84534 (0.0007) [2023-12-27 06:14:49,999][20381] Updated weights for policy 1, policy_version 84544 (0.0009) [2023-12-27 06:14:50,105][20347] Updated weights for policy 0, policy_version 84737 (0.0005) [2023-12-27 06:14:50,161][20347] Updated weights for policy 0, policy_version 84747 (0.0005) [2023-12-27 06:14:50,212][20347] Updated weights for policy 0, policy_version 84757 (0.0007) [2023-12-27 06:14:50,722][20381] Updated weights for policy 1, policy_version 84554 (0.0009) [2023-12-27 06:14:50,781][20381] Updated weights for policy 1, policy_version 84564 (0.0011) [2023-12-27 06:14:50,835][20347] Updated weights for policy 0, policy_version 84767 (0.0006) [2023-12-27 06:14:50,844][20381] Updated weights for policy 1, policy_version 84574 (0.0011) [2023-12-27 06:14:50,891][20347] Updated weights for policy 0, policy_version 84777 (0.0005) [2023-12-27 06:14:50,940][20347] Updated weights for policy 0, policy_version 84787 (0.0005) [2023-12-27 06:14:51,510][20381] Updated weights for policy 1, policy_version 84584 (0.0011) [2023-12-27 06:14:51,566][20381] Updated weights for policy 1, policy_version 84594 (0.0010) [2023-12-27 06:14:51,567][20347] Updated weights for policy 0, policy_version 84797 (0.0005) [2023-12-27 06:14:51,626][20381] Updated weights for policy 1, policy_version 84604 (0.0007) [2023-12-27 06:14:51,632][20347] Updated weights for policy 0, policy_version 84807 (0.0007) [2023-12-27 06:14:51,700][20347] Updated weights for policy 0, policy_version 84817 (0.0008) [2023-12-27 06:14:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 43393024. Throughput: 0: 10551.9, 1: 10359.8. Samples: 43368144. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:52,363][19170] Avg episode reward: [(0, '930.710'), (1, '1000.000')] [2023-12-27 06:14:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000084824_21725184.pth... [2023-12-27 06:14:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000083608_21413888.pth [2023-12-27 06:14:52,383][20381] Updated weights for policy 1, policy_version 84614 (0.0007) [2023-12-27 06:14:52,407][20347] Updated weights for policy 0, policy_version 84827 (0.0008) [2023-12-27 06:14:52,445][20381] Updated weights for policy 1, policy_version 84624 (0.0007) [2023-12-27 06:14:52,462][20347] Updated weights for policy 0, policy_version 84837 (0.0005) [2023-12-27 06:14:52,511][20381] Updated weights for policy 1, policy_version 84634 (0.0008) [2023-12-27 06:14:52,517][20347] Updated weights for policy 0, policy_version 84847 (0.0005) [2023-12-27 06:14:52,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000084640_21676032.pth... [2023-12-27 06:14:52,551][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000083424_21364736.pth [2023-12-27 06:14:53,204][20381] Updated weights for policy 1, policy_version 84644 (0.0008) [2023-12-27 06:14:53,220][20347] Updated weights for policy 0, policy_version 84857 (0.0006) [2023-12-27 06:14:53,257][20381] Updated weights for policy 1, policy_version 84654 (0.0007) [2023-12-27 06:14:53,267][20347] Updated weights for policy 0, policy_version 84867 (0.0008) [2023-12-27 06:14:53,310][20381] Updated weights for policy 1, policy_version 84664 (0.0010) [2023-12-27 06:14:53,312][20347] Updated weights for policy 0, policy_version 84877 (0.0005) [2023-12-27 06:14:53,357][20347] Updated weights for policy 0, policy_version 84887 (0.0006) [2023-12-27 06:14:53,951][20381] Updated weights for policy 1, policy_version 84674 (0.0010) [2023-12-27 06:14:54,013][20381] Updated weights for policy 1, policy_version 84684 (0.0006) [2023-12-27 06:14:54,064][20347] Updated weights for policy 0, policy_version 84897 (0.0010) [2023-12-27 06:14:54,072][20381] Updated weights for policy 1, policy_version 84694 (0.0008) [2023-12-27 06:14:54,123][20347] Updated weights for policy 0, policy_version 84907 (0.0010) [2023-12-27 06:14:54,135][20381] Updated weights for policy 1, policy_version 84704 (0.0006) [2023-12-27 06:14:54,175][20347] Updated weights for policy 0, policy_version 84917 (0.0006) [2023-12-27 06:14:54,714][20347] Updated weights for policy 0, policy_version 84927 (0.0006) [2023-12-27 06:14:54,767][20347] Updated weights for policy 0, policy_version 84937 (0.0005) [2023-12-27 06:14:54,820][20347] Updated weights for policy 0, policy_version 84947 (0.0006) [2023-12-27 06:14:54,841][20381] Updated weights for policy 1, policy_version 84714 (0.0006) [2023-12-27 06:14:54,896][20381] Updated weights for policy 1, policy_version 84724 (0.0008) [2023-12-27 06:14:54,945][20381] Updated weights for policy 1, policy_version 84734 (0.0008) [2023-12-27 06:14:55,473][20347] Updated weights for policy 0, policy_version 84957 (0.0007) [2023-12-27 06:14:55,517][20347] Updated weights for policy 0, policy_version 84967 (0.0007) [2023-12-27 06:14:55,563][20347] Updated weights for policy 0, policy_version 84977 (0.0008) [2023-12-27 06:14:55,641][20381] Updated weights for policy 1, policy_version 84744 (0.0008) [2023-12-27 06:14:55,681][20381] Updated weights for policy 1, policy_version 84754 (0.0007) [2023-12-27 06:14:55,731][20381] Updated weights for policy 1, policy_version 84764 (0.0008) [2023-12-27 06:14:56,203][20347] Updated weights for policy 0, policy_version 84987 (0.0007) [2023-12-27 06:14:56,253][20347] Updated weights for policy 0, policy_version 84997 (0.0005) [2023-12-27 06:14:56,310][20347] Updated weights for policy 0, policy_version 85007 (0.0005) [2023-12-27 06:14:56,475][20381] Updated weights for policy 1, policy_version 84774 (0.0008) [2023-12-27 06:14:56,523][20381] Updated weights for policy 1, policy_version 84784 (0.0007) [2023-12-27 06:14:56,575][20381] Updated weights for policy 1, policy_version 84794 (0.0008) [2023-12-27 06:14:56,925][20347] Updated weights for policy 0, policy_version 85017 (0.0007) [2023-12-27 06:14:56,973][20347] Updated weights for policy 0, policy_version 85027 (0.0010) [2023-12-27 06:14:57,018][20347] Updated weights for policy 0, policy_version 85037 (0.0010) [2023-12-27 06:14:57,063][20347] Updated weights for policy 0, policy_version 85047 (0.0007) [2023-12-27 06:14:57,235][20381] Updated weights for policy 1, policy_version 84804 (0.0008) [2023-12-27 06:14:57,296][20381] Updated weights for policy 1, policy_version 84814 (0.0007) [2023-12-27 06:14:57,355][20381] Updated weights for policy 1, policy_version 84824 (0.0006) [2023-12-27 06:14:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 43499520. Throughput: 0: 10555.6, 1: 10297.3. Samples: 43491232. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:14:57,363][19170] Avg episode reward: [(0, '963.980'), (1, '1000.000')] [2023-12-27 06:14:57,758][20347] Updated weights for policy 0, policy_version 85057 (0.0008) [2023-12-27 06:14:57,814][20347] Updated weights for policy 0, policy_version 85067 (0.0006) [2023-12-27 06:14:57,872][20347] Updated weights for policy 0, policy_version 85077 (0.0005) [2023-12-27 06:14:58,019][20381] Updated weights for policy 1, policy_version 84834 (0.0005) [2023-12-27 06:14:58,076][20381] Updated weights for policy 1, policy_version 84844 (0.0008) [2023-12-27 06:14:58,128][20381] Updated weights for policy 1, policy_version 84854 (0.0009) [2023-12-27 06:14:58,185][20381] Updated weights for policy 1, policy_version 84864 (0.0008) [2023-12-27 06:14:58,516][20347] Updated weights for policy 0, policy_version 85087 (0.0007) [2023-12-27 06:14:58,583][20347] Updated weights for policy 0, policy_version 85097 (0.0008) [2023-12-27 06:14:58,645][20347] Updated weights for policy 0, policy_version 85107 (0.0008) [2023-12-27 06:14:58,965][20381] Updated weights for policy 1, policy_version 84874 (0.0009) [2023-12-27 06:14:59,034][20381] Updated weights for policy 1, policy_version 84884 (0.0008) [2023-12-27 06:14:59,098][20381] Updated weights for policy 1, policy_version 84894 (0.0007) [2023-12-27 06:14:59,405][20347] Updated weights for policy 0, policy_version 85117 (0.0006) [2023-12-27 06:14:59,463][20347] Updated weights for policy 0, policy_version 85127 (0.0007) [2023-12-27 06:14:59,524][20347] Updated weights for policy 0, policy_version 85137 (0.0008) [2023-12-27 06:14:59,792][20381] Updated weights for policy 1, policy_version 84904 (0.0008) [2023-12-27 06:14:59,855][20381] Updated weights for policy 1, policy_version 84914 (0.0006) [2023-12-27 06:14:59,909][20381] Updated weights for policy 1, policy_version 84924 (0.0008) [2023-12-27 06:15:00,212][20347] Updated weights for policy 0, policy_version 85147 (0.0008) [2023-12-27 06:15:00,261][20347] Updated weights for policy 0, policy_version 85157 (0.0009) [2023-12-27 06:15:00,308][20347] Updated weights for policy 0, policy_version 85167 (0.0008) [2023-12-27 06:15:00,555][20381] Updated weights for policy 1, policy_version 84934 (0.0006) [2023-12-27 06:15:00,611][20381] Updated weights for policy 1, policy_version 84944 (0.0007) [2023-12-27 06:15:00,670][20381] Updated weights for policy 1, policy_version 84954 (0.0006) [2023-12-27 06:15:01,007][20347] Updated weights for policy 0, policy_version 85177 (0.0008) [2023-12-27 06:15:01,067][20347] Updated weights for policy 0, policy_version 85187 (0.0008) [2023-12-27 06:15:01,126][20347] Updated weights for policy 0, policy_version 85197 (0.0007) [2023-12-27 06:15:01,184][20347] Updated weights for policy 0, policy_version 85207 (0.0009) [2023-12-27 06:15:01,291][20381] Updated weights for policy 1, policy_version 84964 (0.0006) [2023-12-27 06:15:01,350][20381] Updated weights for policy 1, policy_version 84974 (0.0008) [2023-12-27 06:15:01,418][20381] Updated weights for policy 1, policy_version 84984 (0.0008) [2023-12-27 06:15:01,814][20347] Updated weights for policy 0, policy_version 85217 (0.0006) [2023-12-27 06:15:01,874][20347] Updated weights for policy 0, policy_version 85227 (0.0006) [2023-12-27 06:15:01,933][20347] Updated weights for policy 0, policy_version 85237 (0.0005) [2023-12-27 06:15:02,125][20381] Updated weights for policy 1, policy_version 84994 (0.0008) [2023-12-27 06:15:02,196][20381] Updated weights for policy 1, policy_version 85004 (0.0006) [2023-12-27 06:15:02,261][20381] Updated weights for policy 1, policy_version 85014 (0.0007) [2023-12-27 06:15:02,326][20381] Updated weights for policy 1, policy_version 85024 (0.0007) [2023-12-27 06:15:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 43606016. Throughput: 0: 10488.0, 1: 10290.7. Samples: 43613844. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:15:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:15:02,587][20347] Updated weights for policy 0, policy_version 85247 (0.0006) [2023-12-27 06:15:02,642][20347] Updated weights for policy 0, policy_version 85257 (0.0006) [2023-12-27 06:15:02,695][20347] Updated weights for policy 0, policy_version 85267 (0.0005) [2023-12-27 06:15:02,981][20381] Updated weights for policy 1, policy_version 85034 (0.0011) [2023-12-27 06:15:03,026][20381] Updated weights for policy 1, policy_version 85044 (0.0010) [2023-12-27 06:15:03,080][20381] Updated weights for policy 1, policy_version 85054 (0.0010) [2023-12-27 06:15:03,290][20347] Updated weights for policy 0, policy_version 85277 (0.0006) [2023-12-27 06:15:03,341][20347] Updated weights for policy 0, policy_version 85287 (0.0007) [2023-12-27 06:15:03,387][20347] Updated weights for policy 0, policy_version 85297 (0.0007) [2023-12-27 06:15:03,789][20381] Updated weights for policy 1, policy_version 85064 (0.0007) [2023-12-27 06:15:03,844][20381] Updated weights for policy 1, policy_version 85074 (0.0006) [2023-12-27 06:15:03,903][20381] Updated weights for policy 1, policy_version 85084 (0.0008) [2023-12-27 06:15:04,107][20347] Updated weights for policy 0, policy_version 85307 (0.0007) [2023-12-27 06:15:04,161][20347] Updated weights for policy 0, policy_version 85317 (0.0007) [2023-12-27 06:15:04,219][20347] Updated weights for policy 0, policy_version 85327 (0.0007) [2023-12-27 06:15:04,585][20381] Updated weights for policy 1, policy_version 85094 (0.0010) [2023-12-27 06:15:04,645][20381] Updated weights for policy 1, policy_version 85104 (0.0011) [2023-12-27 06:15:04,696][20381] Updated weights for policy 1, policy_version 85114 (0.0011) [2023-12-27 06:15:04,949][20347] Updated weights for policy 0, policy_version 85337 (0.0007) [2023-12-27 06:15:05,013][20347] Updated weights for policy 0, policy_version 85347 (0.0007) [2023-12-27 06:15:05,069][20347] Updated weights for policy 0, policy_version 85357 (0.0007) [2023-12-27 06:15:05,130][20347] Updated weights for policy 0, policy_version 85367 (0.0007) [2023-12-27 06:15:05,341][20381] Updated weights for policy 1, policy_version 85124 (0.0009) [2023-12-27 06:15:05,404][20381] Updated weights for policy 1, policy_version 85134 (0.0005) [2023-12-27 06:15:05,465][20381] Updated weights for policy 1, policy_version 85144 (0.0010) [2023-12-27 06:15:05,766][20347] Updated weights for policy 0, policy_version 85377 (0.0008) [2023-12-27 06:15:05,812][20347] Updated weights for policy 0, policy_version 85387 (0.0007) [2023-12-27 06:15:05,862][20347] Updated weights for policy 0, policy_version 85397 (0.0007) [2023-12-27 06:15:06,077][20381] Updated weights for policy 1, policy_version 85154 (0.0010) [2023-12-27 06:15:06,138][20381] Updated weights for policy 1, policy_version 85164 (0.0010) [2023-12-27 06:15:06,194][20381] Updated weights for policy 1, policy_version 85174 (0.0007) [2023-12-27 06:15:06,264][20381] Updated weights for policy 1, policy_version 85184 (0.0008) [2023-12-27 06:15:06,566][20347] Updated weights for policy 0, policy_version 85407 (0.0006) [2023-12-27 06:15:06,618][20347] Updated weights for policy 0, policy_version 85417 (0.0007) [2023-12-27 06:15:06,672][20347] Updated weights for policy 0, policy_version 85427 (0.0008) [2023-12-27 06:15:06,890][20381] Updated weights for policy 1, policy_version 85194 (0.0010) [2023-12-27 06:15:06,947][20381] Updated weights for policy 1, policy_version 85204 (0.0009) [2023-12-27 06:15:06,997][20381] Updated weights for policy 1, policy_version 85214 (0.0011) [2023-12-27 06:15:07,336][20347] Updated weights for policy 0, policy_version 85437 (0.0006) [2023-12-27 06:15:07,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 43704320. Throughput: 0: 10473.8, 1: 10305.5. Samples: 43675068. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:15:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:15:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000085216_21823488.pth... [2023-12-27 06:15:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000084032_21520384.pth [2023-12-27 06:15:07,391][20347] Updated weights for policy 0, policy_version 85447 (0.0005) [2023-12-27 06:15:07,447][20347] Updated weights for policy 0, policy_version 85457 (0.0005) [2023-12-27 06:15:07,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000085464_21889024.pth... [2023-12-27 06:15:07,490][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000084216_21569536.pth [2023-12-27 06:15:07,684][20381] Updated weights for policy 1, policy_version 85224 (0.0009) [2023-12-27 06:15:07,747][20381] Updated weights for policy 1, policy_version 85234 (0.0008) [2023-12-27 06:15:07,808][20381] Updated weights for policy 1, policy_version 85244 (0.0005) [2023-12-27 06:15:08,036][20347] Updated weights for policy 0, policy_version 85467 (0.0006) [2023-12-27 06:15:08,097][20347] Updated weights for policy 0, policy_version 85477 (0.0007) [2023-12-27 06:15:08,158][20347] Updated weights for policy 0, policy_version 85487 (0.0007) [2023-12-27 06:15:08,415][20381] Updated weights for policy 1, policy_version 85254 (0.0010) [2023-12-27 06:15:08,471][20381] Updated weights for policy 1, policy_version 85264 (0.0011) [2023-12-27 06:15:08,524][20381] Updated weights for policy 1, policy_version 85274 (0.0009) [2023-12-27 06:15:08,826][20347] Updated weights for policy 0, policy_version 85497 (0.0007) [2023-12-27 06:15:08,880][20347] Updated weights for policy 0, policy_version 85507 (0.0006) [2023-12-27 06:15:08,928][20347] Updated weights for policy 0, policy_version 85517 (0.0008) [2023-12-27 06:15:08,991][20347] Updated weights for policy 0, policy_version 85527 (0.0008) [2023-12-27 06:15:09,204][20381] Updated weights for policy 1, policy_version 85284 (0.0009) [2023-12-27 06:15:09,267][20381] Updated weights for policy 1, policy_version 85294 (0.0011) [2023-12-27 06:15:09,327][20381] Updated weights for policy 1, policy_version 85304 (0.0011) [2023-12-27 06:15:09,739][20347] Updated weights for policy 0, policy_version 85537 (0.0006) [2023-12-27 06:15:09,810][20347] Updated weights for policy 0, policy_version 85547 (0.0006) [2023-12-27 06:15:09,879][20347] Updated weights for policy 0, policy_version 85557 (0.0007) [2023-12-27 06:15:10,080][20381] Updated weights for policy 1, policy_version 85314 (0.0010) [2023-12-27 06:15:10,143][20381] Updated weights for policy 1, policy_version 85324 (0.0006) [2023-12-27 06:15:10,207][20381] Updated weights for policy 1, policy_version 85334 (0.0006) [2023-12-27 06:15:10,263][20381] Updated weights for policy 1, policy_version 85344 (0.0006) [2023-12-27 06:15:10,612][20347] Updated weights for policy 0, policy_version 85567 (0.0007) [2023-12-27 06:15:10,665][20347] Updated weights for policy 0, policy_version 85577 (0.0008) [2023-12-27 06:15:10,712][20347] Updated weights for policy 0, policy_version 85587 (0.0008) [2023-12-27 06:15:10,939][20381] Updated weights for policy 1, policy_version 85354 (0.0007) [2023-12-27 06:15:10,996][20381] Updated weights for policy 1, policy_version 85364 (0.0008) [2023-12-27 06:15:11,064][20381] Updated weights for policy 1, policy_version 85374 (0.0009) [2023-12-27 06:15:11,477][20347] Updated weights for policy 0, policy_version 85597 (0.0007) [2023-12-27 06:15:11,537][20347] Updated weights for policy 0, policy_version 85607 (0.0008) [2023-12-27 06:15:11,601][20347] Updated weights for policy 0, policy_version 85617 (0.0008) [2023-12-27 06:15:11,836][20381] Updated weights for policy 1, policy_version 85384 (0.0007) [2023-12-27 06:15:11,904][20381] Updated weights for policy 1, policy_version 85394 (0.0009) [2023-12-27 06:15:11,961][20381] Updated weights for policy 1, policy_version 85404 (0.0011) [2023-12-27 06:15:12,362][20347] Updated weights for policy 0, policy_version 85627 (0.0008) [2023-12-27 06:15:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 43802624. Throughput: 0: 10434.1, 1: 10264.8. Samples: 43797612. Policy #0 lag: (min: 25.0, avg: 51.7, max: 57.0) [2023-12-27 06:15:12,363][19170] Avg episode reward: [(0, '689.440'), (1, '1000.000')] [2023-12-27 06:15:12,425][20347] Updated weights for policy 0, policy_version 85637 (0.0008) [2023-12-27 06:15:12,485][20347] Updated weights for policy 0, policy_version 85647 (0.0008) [2023-12-27 06:15:12,694][20381] Updated weights for policy 1, policy_version 85414 (0.0009) [2023-12-27 06:15:12,759][20381] Updated weights for policy 1, policy_version 85424 (0.0008) [2023-12-27 06:15:12,821][20381] Updated weights for policy 1, policy_version 85434 (0.0008) [2023-12-27 06:15:13,124][20347] Updated weights for policy 0, policy_version 85657 (0.0007) [2023-12-27 06:15:13,183][20347] Updated weights for policy 0, policy_version 85667 (0.0005) [2023-12-27 06:15:13,247][20347] Updated weights for policy 0, policy_version 85677 (0.0008) [2023-12-27 06:15:13,310][20347] Updated weights for policy 0, policy_version 85687 (0.0008) [2023-12-27 06:15:13,472][20381] Updated weights for policy 1, policy_version 85444 (0.0007) [2023-12-27 06:15:13,528][20381] Updated weights for policy 1, policy_version 85454 (0.0006) [2023-12-27 06:15:13,581][20381] Updated weights for policy 1, policy_version 85464 (0.0006) [2023-12-27 06:15:14,023][20347] Updated weights for policy 0, policy_version 85697 (0.0008) [2023-12-27 06:15:14,081][20347] Updated weights for policy 0, policy_version 85707 (0.0007) [2023-12-27 06:15:14,137][20347] Updated weights for policy 0, policy_version 85717 (0.0008) [2023-12-27 06:15:14,236][20381] Updated weights for policy 1, policy_version 85474 (0.0006) [2023-12-27 06:15:14,296][20381] Updated weights for policy 1, policy_version 85484 (0.0008) [2023-12-27 06:15:14,359][20381] Updated weights for policy 1, policy_version 85494 (0.0008) [2023-12-27 06:15:14,417][20381] Updated weights for policy 1, policy_version 85504 (0.0008) [2023-12-27 06:15:14,844][20291] KL-divergence is very high: 103.5585 [2023-12-27 06:15:14,848][20347] Updated weights for policy 0, policy_version 85727 (0.0006) [2023-12-27 06:15:14,856][20291] KL-divergence is very high: 112.9618 [2023-12-27 06:15:14,908][20347] Updated weights for policy 0, policy_version 85737 (0.0005) [2023-12-27 06:15:14,923][20291] KL-divergence is very high: 103.2801 [2023-12-27 06:15:14,941][20291] KL-divergence is very high: 145.0557 [2023-12-27 06:15:14,971][20347] Updated weights for policy 0, policy_version 85747 (0.0007) [2023-12-27 06:15:14,989][20291] KL-divergence is very high: 115.0954 [2023-12-27 06:15:15,133][20381] Updated weights for policy 1, policy_version 85514 (0.0008) [2023-12-27 06:15:15,197][20381] Updated weights for policy 1, policy_version 85524 (0.0009) [2023-12-27 06:15:15,256][20381] Updated weights for policy 1, policy_version 85534 (0.0008) [2023-12-27 06:15:15,584][20291] KL-divergence is very high: 105.2193 [2023-12-27 06:15:15,600][20291] KL-divergence is very high: 113.5501 [2023-12-27 06:15:15,606][20347] Updated weights for policy 0, policy_version 85757 (0.0007) [2023-12-27 06:15:15,626][20291] KL-divergence is very high: 113.2264 [2023-12-27 06:15:15,643][20291] KL-divergence is very high: 109.2995 [2023-12-27 06:15:15,660][20347] Updated weights for policy 0, policy_version 85767 (0.0007) [2023-12-27 06:15:15,670][20291] KL-divergence is very high: 108.3934 [2023-12-27 06:15:15,686][20291] KL-divergence is very high: 102.5906 [2023-12-27 06:15:15,709][20347] Updated weights for policy 0, policy_version 85777 (0.0006) [2023-12-27 06:15:15,710][20291] KL-divergence is very high: 123.1668 [2023-12-27 06:15:15,724][20291] KL-divergence is very high: 101.5310 [2023-12-27 06:15:16,009][20381] Updated weights for policy 1, policy_version 85544 (0.0008) [2023-12-27 06:15:16,069][20381] Updated weights for policy 1, policy_version 85554 (0.0008) [2023-12-27 06:15:16,127][20381] Updated weights for policy 1, policy_version 85564 (0.0007) [2023-12-27 06:15:16,354][20347] Updated weights for policy 0, policy_version 85787 (0.0006) [2023-12-27 06:15:16,410][20347] Updated weights for policy 0, policy_version 85797 (0.0008) [2023-12-27 06:15:16,474][20347] Updated weights for policy 0, policy_version 85807 (0.0008) [2023-12-27 06:15:16,854][20381] Updated weights for policy 1, policy_version 85574 (0.0008) [2023-12-27 06:15:16,920][20381] Updated weights for policy 1, policy_version 85584 (0.0006) [2023-12-27 06:15:16,976][20381] Updated weights for policy 1, policy_version 85594 (0.0005) [2023-12-27 06:15:17,151][20347] Updated weights for policy 0, policy_version 85817 (0.0008) [2023-12-27 06:15:17,200][20347] Updated weights for policy 0, policy_version 85827 (0.0009) [2023-12-27 06:15:17,258][20347] Updated weights for policy 0, policy_version 85837 (0.0010) [2023-12-27 06:15:17,314][20347] Updated weights for policy 0, policy_version 85847 (0.0010) [2023-12-27 06:15:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 43909120. Throughput: 0: 10358.2, 1: 10206.4. Samples: 43917024. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:17,363][19170] Avg episode reward: [(0, '336.630'), (1, '1000.000')] [2023-12-27 06:15:17,648][20381] Updated weights for policy 1, policy_version 85604 (0.0006) [2023-12-27 06:15:17,708][20381] Updated weights for policy 1, policy_version 85614 (0.0007) [2023-12-27 06:15:17,765][20381] Updated weights for policy 1, policy_version 85624 (0.0007) [2023-12-27 06:15:17,929][20347] Updated weights for policy 0, policy_version 85857 (0.0010) [2023-12-27 06:15:17,991][20347] Updated weights for policy 0, policy_version 85867 (0.0009) [2023-12-27 06:15:18,053][20347] Updated weights for policy 0, policy_version 85877 (0.0010) [2023-12-27 06:15:18,448][20381] Updated weights for policy 1, policy_version 85634 (0.0008) [2023-12-27 06:15:18,504][20381] Updated weights for policy 1, policy_version 85644 (0.0007) [2023-12-27 06:15:18,568][20381] Updated weights for policy 1, policy_version 85654 (0.0008) [2023-12-27 06:15:18,629][20381] Updated weights for policy 1, policy_version 85664 (0.0008) [2023-12-27 06:15:18,744][20347] Updated weights for policy 0, policy_version 85887 (0.0010) [2023-12-27 06:15:18,814][20347] Updated weights for policy 0, policy_version 85897 (0.0005) [2023-12-27 06:15:18,881][20347] Updated weights for policy 0, policy_version 85907 (0.0005) [2023-12-27 06:15:19,314][20381] Updated weights for policy 1, policy_version 85674 (0.0010) [2023-12-27 06:15:19,375][20381] Updated weights for policy 1, policy_version 85684 (0.0010) [2023-12-27 06:15:19,437][20381] Updated weights for policy 1, policy_version 85694 (0.0006) [2023-12-27 06:15:19,565][20347] Updated weights for policy 0, policy_version 85917 (0.0007) [2023-12-27 06:15:19,616][20347] Updated weights for policy 0, policy_version 85927 (0.0008) [2023-12-27 06:15:19,675][20347] Updated weights for policy 0, policy_version 85937 (0.0007) [2023-12-27 06:15:20,147][20381] Updated weights for policy 1, policy_version 85704 (0.0008) [2023-12-27 06:15:20,202][20381] Updated weights for policy 1, policy_version 85714 (0.0008) [2023-12-27 06:15:20,254][20381] Updated weights for policy 1, policy_version 85724 (0.0008) [2023-12-27 06:15:20,400][20347] Updated weights for policy 0, policy_version 85947 (0.0010) [2023-12-27 06:15:20,467][20347] Updated weights for policy 0, policy_version 85957 (0.0008) [2023-12-27 06:15:20,524][20347] Updated weights for policy 0, policy_version 85967 (0.0005) [2023-12-27 06:15:20,976][20381] Updated weights for policy 1, policy_version 85734 (0.0008) [2023-12-27 06:15:21,041][20381] Updated weights for policy 1, policy_version 85744 (0.0007) [2023-12-27 06:15:21,108][20381] Updated weights for policy 1, policy_version 85754 (0.0008) [2023-12-27 06:15:21,269][20347] Updated weights for policy 0, policy_version 85977 (0.0007) [2023-12-27 06:15:21,334][20347] Updated weights for policy 0, policy_version 85987 (0.0008) [2023-12-27 06:15:21,402][20347] Updated weights for policy 0, policy_version 85997 (0.0008) [2023-12-27 06:15:21,459][20347] Updated weights for policy 0, policy_version 86007 (0.0008) [2023-12-27 06:15:21,811][20381] Updated weights for policy 1, policy_version 85764 (0.0008) [2023-12-27 06:15:21,879][20381] Updated weights for policy 1, policy_version 85774 (0.0008) [2023-12-27 06:15:21,939][20381] Updated weights for policy 1, policy_version 85784 (0.0007) [2023-12-27 06:15:22,085][20347] Updated weights for policy 0, policy_version 86017 (0.0005) [2023-12-27 06:15:22,138][20347] Updated weights for policy 0, policy_version 86027 (0.0009) [2023-12-27 06:15:22,188][20347] Updated weights for policy 0, policy_version 86037 (0.0010) [2023-12-27 06:15:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 44007424. Throughput: 0: 10322.2, 1: 10155.6. Samples: 43977024. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:22,363][19170] Avg episode reward: [(0, '365.350'), (1, '1000.000')] [2023-12-27 06:15:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000085792_21970944.pth... [2023-12-27 06:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000086040_22036480.pth... [2023-12-27 06:15:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000084640_21676032.pth [2023-12-27 06:15:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000084824_21725184.pth [2023-12-27 06:15:22,665][20381] Updated weights for policy 1, policy_version 85794 (0.0008) [2023-12-27 06:15:22,722][20381] Updated weights for policy 1, policy_version 85804 (0.0008) [2023-12-27 06:15:22,779][20381] Updated weights for policy 1, policy_version 85814 (0.0008) [2023-12-27 06:15:22,832][20381] Updated weights for policy 1, policy_version 85824 (0.0007) [2023-12-27 06:15:22,894][20347] Updated weights for policy 0, policy_version 86047 (0.0007) [2023-12-27 06:15:22,953][20347] Updated weights for policy 0, policy_version 86057 (0.0006) [2023-12-27 06:15:23,021][20347] Updated weights for policy 0, policy_version 86067 (0.0006) [2023-12-27 06:15:23,494][20381] Updated weights for policy 1, policy_version 85834 (0.0007) [2023-12-27 06:15:23,564][20381] Updated weights for policy 1, policy_version 85844 (0.0005) [2023-12-27 06:15:23,583][20347] Updated weights for policy 0, policy_version 86077 (0.0006) [2023-12-27 06:15:23,627][20381] Updated weights for policy 1, policy_version 85854 (0.0008) [2023-12-27 06:15:23,642][20347] Updated weights for policy 0, policy_version 86087 (0.0008) [2023-12-27 06:15:23,689][20347] Updated weights for policy 0, policy_version 86097 (0.0007) [2023-12-27 06:15:24,236][20381] Updated weights for policy 1, policy_version 85864 (0.0007) [2023-12-27 06:15:24,297][20381] Updated weights for policy 1, policy_version 85874 (0.0008) [2023-12-27 06:15:24,355][20381] Updated weights for policy 1, policy_version 85884 (0.0008) [2023-12-27 06:15:24,400][20347] Updated weights for policy 0, policy_version 86107 (0.0008) [2023-12-27 06:15:24,454][20347] Updated weights for policy 0, policy_version 86117 (0.0010) [2023-12-27 06:15:24,506][20347] Updated weights for policy 0, policy_version 86127 (0.0010) [2023-12-27 06:15:24,940][20381] Updated weights for policy 1, policy_version 85894 (0.0011) [2023-12-27 06:15:24,996][20381] Updated weights for policy 1, policy_version 85904 (0.0007) [2023-12-27 06:15:25,046][20381] Updated weights for policy 1, policy_version 85914 (0.0008) [2023-12-27 06:15:25,148][20347] Updated weights for policy 0, policy_version 86137 (0.0010) [2023-12-27 06:15:25,194][20347] Updated weights for policy 0, policy_version 86147 (0.0009) [2023-12-27 06:15:25,250][20347] Updated weights for policy 0, policy_version 86157 (0.0007) [2023-12-27 06:15:25,310][20347] Updated weights for policy 0, policy_version 86167 (0.0005) [2023-12-27 06:15:25,684][20381] Updated weights for policy 1, policy_version 85924 (0.0008) [2023-12-27 06:15:25,740][20381] Updated weights for policy 1, policy_version 85934 (0.0008) [2023-12-27 06:15:25,796][20381] Updated weights for policy 1, policy_version 85944 (0.0007) [2023-12-27 06:15:25,889][20347] Updated weights for policy 0, policy_version 86177 (0.0006) [2023-12-27 06:15:25,945][20347] Updated weights for policy 0, policy_version 86187 (0.0008) [2023-12-27 06:15:26,010][20347] Updated weights for policy 0, policy_version 86197 (0.0008) [2023-12-27 06:15:26,472][20381] Updated weights for policy 1, policy_version 85954 (0.0006) [2023-12-27 06:15:26,526][20381] Updated weights for policy 1, policy_version 85964 (0.0008) [2023-12-27 06:15:26,581][20381] Updated weights for policy 1, policy_version 85974 (0.0007) [2023-12-27 06:15:26,620][20347] Updated weights for policy 0, policy_version 86207 (0.0006) [2023-12-27 06:15:26,645][20381] Updated weights for policy 1, policy_version 85984 (0.0006) [2023-12-27 06:15:26,670][20347] Updated weights for policy 0, policy_version 86217 (0.0007) [2023-12-27 06:15:26,723][20347] Updated weights for policy 0, policy_version 86227 (0.0007) [2023-12-27 06:15:27,235][20381] Updated weights for policy 1, policy_version 85994 (0.0010) [2023-12-27 06:15:27,291][20381] Updated weights for policy 1, policy_version 86004 (0.0008) [2023-12-27 06:15:27,317][20347] Updated weights for policy 0, policy_version 86237 (0.0007) [2023-12-27 06:15:27,336][20381] Updated weights for policy 1, policy_version 86014 (0.0007) [2023-12-27 06:15:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 44113920. Throughput: 0: 10354.6, 1: 10166.0. Samples: 44101548. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:27,363][19170] Avg episode reward: [(0, '777.350'), (1, '1000.000')] [2023-12-27 06:15:27,366][20347] Updated weights for policy 0, policy_version 86247 (0.0006) [2023-12-27 06:15:27,413][20347] Updated weights for policy 0, policy_version 86257 (0.0007) [2023-12-27 06:15:28,033][20381] Updated weights for policy 1, policy_version 86024 (0.0006) [2023-12-27 06:15:28,042][20347] Updated weights for policy 0, policy_version 86267 (0.0008) [2023-12-27 06:15:28,086][20381] Updated weights for policy 1, policy_version 86034 (0.0005) [2023-12-27 06:15:28,100][20347] Updated weights for policy 0, policy_version 86277 (0.0010) [2023-12-27 06:15:28,138][20381] Updated weights for policy 1, policy_version 86044 (0.0006) [2023-12-27 06:15:28,155][20347] Updated weights for policy 0, policy_version 86287 (0.0010) [2023-12-27 06:15:28,780][20347] Updated weights for policy 0, policy_version 86297 (0.0010) [2023-12-27 06:15:28,811][20381] Updated weights for policy 1, policy_version 86054 (0.0006) [2023-12-27 06:15:28,829][20347] Updated weights for policy 0, policy_version 86307 (0.0010) [2023-12-27 06:15:28,867][20381] Updated weights for policy 1, policy_version 86064 (0.0005) [2023-12-27 06:15:28,885][20347] Updated weights for policy 0, policy_version 86317 (0.0008) [2023-12-27 06:15:28,914][20381] Updated weights for policy 1, policy_version 86074 (0.0006) [2023-12-27 06:15:28,935][20347] Updated weights for policy 0, policy_version 86327 (0.0008) [2023-12-27 06:15:29,540][20347] Updated weights for policy 0, policy_version 86337 (0.0005) [2023-12-27 06:15:29,602][20347] Updated weights for policy 0, policy_version 86347 (0.0005) [2023-12-27 06:15:29,658][20347] Updated weights for policy 0, policy_version 86357 (0.0007) [2023-12-27 06:15:29,667][20381] Updated weights for policy 1, policy_version 86084 (0.0007) [2023-12-27 06:15:29,725][20381] Updated weights for policy 1, policy_version 86094 (0.0007) [2023-12-27 06:15:29,788][20381] Updated weights for policy 1, policy_version 86104 (0.0006) [2023-12-27 06:15:30,276][20347] Updated weights for policy 0, policy_version 86367 (0.0008) [2023-12-27 06:15:30,328][20347] Updated weights for policy 0, policy_version 86377 (0.0007) [2023-12-27 06:15:30,378][20347] Updated weights for policy 0, policy_version 86387 (0.0007) [2023-12-27 06:15:30,442][20381] Updated weights for policy 1, policy_version 86114 (0.0008) [2023-12-27 06:15:30,492][20381] Updated weights for policy 1, policy_version 86124 (0.0006) [2023-12-27 06:15:30,548][20381] Updated weights for policy 1, policy_version 86134 (0.0008) [2023-12-27 06:15:30,606][20381] Updated weights for policy 1, policy_version 86144 (0.0007) [2023-12-27 06:15:31,061][20347] Updated weights for policy 0, policy_version 86397 (0.0009) [2023-12-27 06:15:31,121][20347] Updated weights for policy 0, policy_version 86407 (0.0006) [2023-12-27 06:15:31,192][20347] Updated weights for policy 0, policy_version 86417 (0.0008) [2023-12-27 06:15:31,317][20381] Updated weights for policy 1, policy_version 86154 (0.0007) [2023-12-27 06:15:31,372][20381] Updated weights for policy 1, policy_version 86164 (0.0008) [2023-12-27 06:15:31,431][20381] Updated weights for policy 1, policy_version 86174 (0.0008) [2023-12-27 06:15:31,884][20347] Updated weights for policy 0, policy_version 86427 (0.0007) [2023-12-27 06:15:31,938][20347] Updated weights for policy 0, policy_version 86437 (0.0007) [2023-12-27 06:15:31,990][20347] Updated weights for policy 0, policy_version 86447 (0.0008) [2023-12-27 06:15:32,094][20381] Updated weights for policy 1, policy_version 86184 (0.0008) [2023-12-27 06:15:32,144][20381] Updated weights for policy 1, policy_version 86194 (0.0008) [2023-12-27 06:15:32,193][20381] Updated weights for policy 1, policy_version 86204 (0.0007) [2023-12-27 06:15:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 44220416. Throughput: 0: 10345.5, 1: 10114.2. Samples: 44227780. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:32,363][19170] Avg episode reward: [(0, '976.230'), (1, '1000.000')] [2023-12-27 06:15:32,608][20347] Updated weights for policy 0, policy_version 86457 (0.0008) [2023-12-27 06:15:32,657][20347] Updated weights for policy 0, policy_version 86467 (0.0008) [2023-12-27 06:15:32,709][20347] Updated weights for policy 0, policy_version 86477 (0.0007) [2023-12-27 06:15:32,763][20347] Updated weights for policy 0, policy_version 86487 (0.0007) [2023-12-27 06:15:32,917][20381] Updated weights for policy 1, policy_version 86214 (0.0007) [2023-12-27 06:15:32,966][20381] Updated weights for policy 1, policy_version 86224 (0.0008) [2023-12-27 06:15:33,017][20381] Updated weights for policy 1, policy_version 86234 (0.0008) [2023-12-27 06:15:33,467][20347] Updated weights for policy 0, policy_version 86497 (0.0008) [2023-12-27 06:15:33,516][20347] Updated weights for policy 0, policy_version 86507 (0.0007) [2023-12-27 06:15:33,558][20347] Updated weights for policy 0, policy_version 86517 (0.0007) [2023-12-27 06:15:33,713][20381] Updated weights for policy 1, policy_version 86244 (0.0007) [2023-12-27 06:15:33,758][20381] Updated weights for policy 1, policy_version 86254 (0.0007) [2023-12-27 06:15:33,807][20381] Updated weights for policy 1, policy_version 86264 (0.0007) [2023-12-27 06:15:34,258][20347] Updated weights for policy 0, policy_version 86527 (0.0008) [2023-12-27 06:15:34,319][20347] Updated weights for policy 0, policy_version 86537 (0.0008) [2023-12-27 06:15:34,387][20347] Updated weights for policy 0, policy_version 86547 (0.0008) [2023-12-27 06:15:34,496][20381] Updated weights for policy 1, policy_version 86274 (0.0008) [2023-12-27 06:15:34,552][20381] Updated weights for policy 1, policy_version 86284 (0.0007) [2023-12-27 06:15:34,615][20381] Updated weights for policy 1, policy_version 86294 (0.0008) [2023-12-27 06:15:34,678][20381] Updated weights for policy 1, policy_version 86304 (0.0009) [2023-12-27 06:15:35,007][20347] Updated weights for policy 0, policy_version 86557 (0.0007) [2023-12-27 06:15:35,067][20347] Updated weights for policy 0, policy_version 86567 (0.0007) [2023-12-27 06:15:35,125][20347] Updated weights for policy 0, policy_version 86577 (0.0008) [2023-12-27 06:15:35,344][20381] Updated weights for policy 1, policy_version 86314 (0.0006) [2023-12-27 06:15:35,402][20381] Updated weights for policy 1, policy_version 86324 (0.0011) [2023-12-27 06:15:35,454][20381] Updated weights for policy 1, policy_version 86334 (0.0010) [2023-12-27 06:15:35,818][20347] Updated weights for policy 0, policy_version 86587 (0.0008) [2023-12-27 06:15:35,872][20347] Updated weights for policy 0, policy_version 86597 (0.0010) [2023-12-27 06:15:35,926][20347] Updated weights for policy 0, policy_version 86607 (0.0010) [2023-12-27 06:15:36,031][20381] Updated weights for policy 1, policy_version 86344 (0.0006) [2023-12-27 06:15:36,081][20381] Updated weights for policy 1, policy_version 86354 (0.0006) [2023-12-27 06:15:36,153][20381] Updated weights for policy 1, policy_version 86364 (0.0010) [2023-12-27 06:15:36,675][20347] Updated weights for policy 0, policy_version 86617 (0.0010) [2023-12-27 06:15:36,729][20347] Updated weights for policy 0, policy_version 86627 (0.0007) [2023-12-27 06:15:36,795][20347] Updated weights for policy 0, policy_version 86637 (0.0007) [2023-12-27 06:15:36,816][20381] Updated weights for policy 1, policy_version 86374 (0.0007) [2023-12-27 06:15:36,854][20347] Updated weights for policy 0, policy_version 86647 (0.0008) [2023-12-27 06:15:36,855][20291] KL-divergence is very high: 111.3359 [2023-12-27 06:15:36,873][20381] Updated weights for policy 1, policy_version 86384 (0.0008) [2023-12-27 06:15:36,929][20381] Updated weights for policy 1, policy_version 86394 (0.0008) [2023-12-27 06:15:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 44318720. Throughput: 0: 10392.6, 1: 10094.1. Samples: 44290040. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:37,363][19170] Avg episode reward: [(0, '590.420'), (1, '1000.000')] [2023-12-27 06:15:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000086400_22126592.pth... [2023-12-27 06:15:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000086648_22192128.pth... [2023-12-27 06:15:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000085216_21823488.pth [2023-12-27 06:15:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000085464_21889024.pth [2023-12-27 06:15:37,455][20347] Updated weights for policy 0, policy_version 86657 (0.0008) [2023-12-27 06:15:37,511][20347] Updated weights for policy 0, policy_version 86667 (0.0007) [2023-12-27 06:15:37,578][20347] Updated weights for policy 0, policy_version 86677 (0.0007) [2023-12-27 06:15:37,676][20381] Updated weights for policy 1, policy_version 86404 (0.0008) [2023-12-27 06:15:37,727][20381] Updated weights for policy 1, policy_version 86414 (0.0008) [2023-12-27 06:15:37,778][20381] Updated weights for policy 1, policy_version 86424 (0.0008) [2023-12-27 06:15:38,173][20347] Updated weights for policy 0, policy_version 86687 (0.0010) [2023-12-27 06:15:38,221][20347] Updated weights for policy 0, policy_version 86697 (0.0010) [2023-12-27 06:15:38,267][20347] Updated weights for policy 0, policy_version 86707 (0.0008) [2023-12-27 06:15:38,496][20381] Updated weights for policy 1, policy_version 86434 (0.0009) [2023-12-27 06:15:38,552][20381] Updated weights for policy 1, policy_version 86444 (0.0011) [2023-12-27 06:15:38,613][20381] Updated weights for policy 1, policy_version 86454 (0.0007) [2023-12-27 06:15:38,671][20381] Updated weights for policy 1, policy_version 86464 (0.0006) [2023-12-27 06:15:38,937][20347] Updated weights for policy 0, policy_version 86717 (0.0005) [2023-12-27 06:15:38,993][20347] Updated weights for policy 0, policy_version 86727 (0.0005) [2023-12-27 06:15:39,054][20347] Updated weights for policy 0, policy_version 86737 (0.0010) [2023-12-27 06:15:39,315][20381] Updated weights for policy 1, policy_version 86474 (0.0007) [2023-12-27 06:15:39,378][20381] Updated weights for policy 1, policy_version 86484 (0.0007) [2023-12-27 06:15:39,443][20381] Updated weights for policy 1, policy_version 86494 (0.0007) [2023-12-27 06:15:39,716][20347] Updated weights for policy 0, policy_version 86747 (0.0009) [2023-12-27 06:15:39,780][20347] Updated weights for policy 0, policy_version 86757 (0.0006) [2023-12-27 06:15:39,859][20347] Updated weights for policy 0, policy_version 86767 (0.0008) [2023-12-27 06:15:40,152][20381] Updated weights for policy 1, policy_version 86504 (0.0010) [2023-12-27 06:15:40,209][20381] Updated weights for policy 1, policy_version 86514 (0.0011) [2023-12-27 06:15:40,271][20381] Updated weights for policy 1, policy_version 86524 (0.0010) [2023-12-27 06:15:40,532][20347] Updated weights for policy 0, policy_version 86777 (0.0008) [2023-12-27 06:15:40,583][20347] Updated weights for policy 0, policy_version 86787 (0.0010) [2023-12-27 06:15:40,638][20347] Updated weights for policy 0, policy_version 86797 (0.0010) [2023-12-27 06:15:40,703][20347] Updated weights for policy 0, policy_version 86807 (0.0010) [2023-12-27 06:15:40,899][20381] Updated weights for policy 1, policy_version 86534 (0.0009) [2023-12-27 06:15:40,961][20381] Updated weights for policy 1, policy_version 86544 (0.0011) [2023-12-27 06:15:41,021][20381] Updated weights for policy 1, policy_version 86554 (0.0011) [2023-12-27 06:15:41,400][20347] Updated weights for policy 0, policy_version 86817 (0.0010) [2023-12-27 06:15:41,453][20347] Updated weights for policy 0, policy_version 86827 (0.0011) [2023-12-27 06:15:41,505][20347] Updated weights for policy 0, policy_version 86837 (0.0010) [2023-12-27 06:15:41,742][20381] Updated weights for policy 1, policy_version 86564 (0.0009) [2023-12-27 06:15:41,800][20381] Updated weights for policy 1, policy_version 86574 (0.0011) [2023-12-27 06:15:41,854][20381] Updated weights for policy 1, policy_version 86584 (0.0011) [2023-12-27 06:15:42,208][20347] Updated weights for policy 0, policy_version 86847 (0.0009) [2023-12-27 06:15:42,264][20347] Updated weights for policy 0, policy_version 86857 (0.0007) [2023-12-27 06:15:42,320][20347] Updated weights for policy 0, policy_version 86867 (0.0008) [2023-12-27 06:15:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 44425216. Throughput: 0: 10356.6, 1: 10129.0. Samples: 44413084. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:42,363][19170] Avg episode reward: [(0, '572.080'), (1, '1000.000')] [2023-12-27 06:15:42,570][20381] Updated weights for policy 1, policy_version 86594 (0.0010) [2023-12-27 06:15:42,641][20381] Updated weights for policy 1, policy_version 86604 (0.0011) [2023-12-27 06:15:42,700][20381] Updated weights for policy 1, policy_version 86614 (0.0011) [2023-12-27 06:15:42,752][20381] Updated weights for policy 1, policy_version 86624 (0.0011) [2023-12-27 06:15:42,986][20347] Updated weights for policy 0, policy_version 86877 (0.0008) [2023-12-27 06:15:43,037][20347] Updated weights for policy 0, policy_version 86887 (0.0008) [2023-12-27 06:15:43,087][20347] Updated weights for policy 0, policy_version 86897 (0.0006) [2023-12-27 06:15:43,420][20381] Updated weights for policy 1, policy_version 86634 (0.0010) [2023-12-27 06:15:43,478][20381] Updated weights for policy 1, policy_version 86644 (0.0010) [2023-12-27 06:15:43,532][20381] Updated weights for policy 1, policy_version 86654 (0.0010) [2023-12-27 06:15:43,723][20347] Updated weights for policy 0, policy_version 86907 (0.0006) [2023-12-27 06:15:43,789][20347] Updated weights for policy 0, policy_version 86917 (0.0005) [2023-12-27 06:15:43,837][20347] Updated weights for policy 0, policy_version 86927 (0.0005) [2023-12-27 06:15:44,167][20381] Updated weights for policy 1, policy_version 86664 (0.0010) [2023-12-27 06:15:44,225][20381] Updated weights for policy 1, policy_version 86674 (0.0010) [2023-12-27 06:15:44,278][20381] Updated weights for policy 1, policy_version 86684 (0.0008) [2023-12-27 06:15:44,387][20347] Updated weights for policy 0, policy_version 86937 (0.0005) [2023-12-27 06:15:44,441][20347] Updated weights for policy 0, policy_version 86947 (0.0010) [2023-12-27 06:15:44,492][20347] Updated weights for policy 0, policy_version 86957 (0.0010) [2023-12-27 06:15:44,543][20347] Updated weights for policy 0, policy_version 86967 (0.0010) [2023-12-27 06:15:44,901][20381] Updated weights for policy 1, policy_version 86694 (0.0009) [2023-12-27 06:15:44,963][20381] Updated weights for policy 1, policy_version 86704 (0.0007) [2023-12-27 06:15:45,024][20381] Updated weights for policy 1, policy_version 86714 (0.0006) [2023-12-27 06:15:45,243][20347] Updated weights for policy 0, policy_version 86977 (0.0010) [2023-12-27 06:15:45,288][20347] Updated weights for policy 0, policy_version 86987 (0.0010) [2023-12-27 06:15:45,340][20347] Updated weights for policy 0, policy_version 86997 (0.0010) [2023-12-27 06:15:45,642][20381] Updated weights for policy 1, policy_version 86724 (0.0009) [2023-12-27 06:15:45,691][20381] Updated weights for policy 1, policy_version 86734 (0.0007) [2023-12-27 06:15:45,749][20381] Updated weights for policy 1, policy_version 86744 (0.0005) [2023-12-27 06:15:46,029][20347] Updated weights for policy 0, policy_version 87007 (0.0010) [2023-12-27 06:15:46,032][20291] KL-divergence is very high: 114.8828 [2023-12-27 06:15:46,039][20291] KL-divergence is very high: 118.6433 [2023-12-27 06:15:46,045][20291] KL-divergence is very high: 137.7358 [2023-12-27 06:15:46,051][20291] KL-divergence is very high: 115.0626 [2023-12-27 06:15:46,057][20291] KL-divergence is very high: 112.5675 [2023-12-27 06:15:46,086][20347] Updated weights for policy 0, policy_version 87017 (0.0010) [2023-12-27 06:15:46,134][20347] Updated weights for policy 0, policy_version 87027 (0.0010) [2023-12-27 06:15:46,299][20381] Updated weights for policy 1, policy_version 86754 (0.0006) [2023-12-27 06:15:46,348][20381] Updated weights for policy 1, policy_version 86764 (0.0011) [2023-12-27 06:15:46,406][20381] Updated weights for policy 1, policy_version 86774 (0.0010) [2023-12-27 06:15:46,469][20381] Updated weights for policy 1, policy_version 86784 (0.0010) [2023-12-27 06:15:46,730][20291] KL-divergence is very high: 102.6554 [2023-12-27 06:15:46,753][20347] Updated weights for policy 0, policy_version 87037 (0.0006) [2023-12-27 06:15:46,813][20347] Updated weights for policy 0, policy_version 87047 (0.0005) [2023-12-27 06:15:46,869][20347] Updated weights for policy 0, policy_version 87057 (0.0005) [2023-12-27 06:15:47,180][20381] Updated weights for policy 1, policy_version 86794 (0.0011) [2023-12-27 06:15:47,233][20381] Updated weights for policy 1, policy_version 86804 (0.0010) [2023-12-27 06:15:47,278][20381] Updated weights for policy 1, policy_version 86814 (0.0010) [2023-12-27 06:15:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 44531712. Throughput: 0: 10393.2, 1: 10193.8. Samples: 44540260. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:47,364][19170] Avg episode reward: [(0, '583.480'), (1, '1000.000')] [2023-12-27 06:15:47,437][20347] Updated weights for policy 0, policy_version 87067 (0.0007) [2023-12-27 06:15:47,483][20347] Updated weights for policy 0, policy_version 87077 (0.0010) [2023-12-27 06:15:47,538][20347] Updated weights for policy 0, policy_version 87087 (0.0010) [2023-12-27 06:15:47,915][20381] Updated weights for policy 1, policy_version 86824 (0.0010) [2023-12-27 06:15:47,967][20381] Updated weights for policy 1, policy_version 86834 (0.0008) [2023-12-27 06:15:48,014][20381] Updated weights for policy 1, policy_version 86844 (0.0006) [2023-12-27 06:15:48,199][20347] Updated weights for policy 0, policy_version 87097 (0.0010) [2023-12-27 06:15:48,259][20347] Updated weights for policy 0, policy_version 87107 (0.0005) [2023-12-27 06:15:48,311][20347] Updated weights for policy 0, policy_version 87117 (0.0005) [2023-12-27 06:15:48,375][20347] Updated weights for policy 0, policy_version 87127 (0.0008) [2023-12-27 06:15:48,724][20381] Updated weights for policy 1, policy_version 86854 (0.0008) [2023-12-27 06:15:48,777][20381] Updated weights for policy 1, policy_version 86864 (0.0008) [2023-12-27 06:15:48,829][20381] Updated weights for policy 1, policy_version 86874 (0.0008) [2023-12-27 06:15:48,937][20347] Updated weights for policy 0, policy_version 87137 (0.0005) [2023-12-27 06:15:48,994][20347] Updated weights for policy 0, policy_version 87147 (0.0008) [2023-12-27 06:15:49,044][20347] Updated weights for policy 0, policy_version 87157 (0.0008) [2023-12-27 06:15:49,585][20381] Updated weights for policy 1, policy_version 86884 (0.0007) [2023-12-27 06:15:49,637][20347] Updated weights for policy 0, policy_version 87167 (0.0006) [2023-12-27 06:15:49,639][20381] Updated weights for policy 1, policy_version 86894 (0.0007) [2023-12-27 06:15:49,690][20347] Updated weights for policy 0, policy_version 87177 (0.0005) [2023-12-27 06:15:49,691][20381] Updated weights for policy 1, policy_version 86904 (0.0008) [2023-12-27 06:15:49,750][20347] Updated weights for policy 0, policy_version 87187 (0.0006) [2023-12-27 06:15:50,371][20347] Updated weights for policy 0, policy_version 87197 (0.0007) [2023-12-27 06:15:50,392][20381] Updated weights for policy 1, policy_version 86914 (0.0009) [2023-12-27 06:15:50,431][20347] Updated weights for policy 0, policy_version 87207 (0.0007) [2023-12-27 06:15:50,444][20381] Updated weights for policy 1, policy_version 86924 (0.0005) [2023-12-27 06:15:50,486][20347] Updated weights for policy 0, policy_version 87217 (0.0007) [2023-12-27 06:15:50,498][20381] Updated weights for policy 1, policy_version 86934 (0.0006) [2023-12-27 06:15:50,549][20381] Updated weights for policy 1, policy_version 86944 (0.0006) [2023-12-27 06:15:51,185][20347] Updated weights for policy 0, policy_version 87227 (0.0007) [2023-12-27 06:15:51,197][20381] Updated weights for policy 1, policy_version 86954 (0.0008) [2023-12-27 06:15:51,235][20347] Updated weights for policy 0, policy_version 87237 (0.0010) [2023-12-27 06:15:51,261][20381] Updated weights for policy 1, policy_version 86964 (0.0008) [2023-12-27 06:15:51,295][20347] Updated weights for policy 0, policy_version 87247 (0.0010) [2023-12-27 06:15:51,326][20381] Updated weights for policy 1, policy_version 86974 (0.0009) [2023-12-27 06:15:52,032][20381] Updated weights for policy 1, policy_version 86984 (0.0009) [2023-12-27 06:15:52,037][20347] Updated weights for policy 0, policy_version 87257 (0.0011) [2023-12-27 06:15:52,091][20347] Updated weights for policy 0, policy_version 87267 (0.0010) [2023-12-27 06:15:52,091][20381] Updated weights for policy 1, policy_version 86994 (0.0011) [2023-12-27 06:15:52,147][20381] Updated weights for policy 1, policy_version 87004 (0.0006) [2023-12-27 06:15:52,150][20347] Updated weights for policy 0, policy_version 87277 (0.0011) [2023-12-27 06:15:52,208][20347] Updated weights for policy 0, policy_version 87287 (0.0010) [2023-12-27 06:15:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.8). Total num frames: 44638208. Throughput: 0: 10461.1, 1: 10186.9. Samples: 44604224. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:52,363][19170] Avg episode reward: [(0, '236.950'), (1, '1000.000')] [2023-12-27 06:15:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000087008_22282240.pth... [2023-12-27 06:15:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000087288_22355968.pth... [2023-12-27 06:15:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000086040_22036480.pth [2023-12-27 06:15:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000085792_21970944.pth [2023-12-27 06:15:52,822][20347] Updated weights for policy 0, policy_version 87297 (0.0010) [2023-12-27 06:15:52,833][20381] Updated weights for policy 1, policy_version 87014 (0.0009) [2023-12-27 06:15:52,874][20347] Updated weights for policy 0, policy_version 87307 (0.0010) [2023-12-27 06:15:52,883][20381] Updated weights for policy 1, policy_version 87024 (0.0011) [2023-12-27 06:15:52,923][20347] Updated weights for policy 0, policy_version 87317 (0.0010) [2023-12-27 06:15:52,933][20381] Updated weights for policy 1, policy_version 87034 (0.0011) [2023-12-27 06:15:53,577][20381] Updated weights for policy 1, policy_version 87044 (0.0011) [2023-12-27 06:15:53,607][20347] Updated weights for policy 0, policy_version 87327 (0.0010) [2023-12-27 06:15:53,636][20381] Updated weights for policy 1, policy_version 87054 (0.0011) [2023-12-27 06:15:53,668][20347] Updated weights for policy 0, policy_version 87337 (0.0010) [2023-12-27 06:15:53,694][20381] Updated weights for policy 1, policy_version 87064 (0.0011) [2023-12-27 06:15:53,721][20347] Updated weights for policy 0, policy_version 87347 (0.0010) [2023-12-27 06:15:54,361][20347] Updated weights for policy 0, policy_version 87357 (0.0008) [2023-12-27 06:15:54,382][20381] Updated weights for policy 1, policy_version 87074 (0.0011) [2023-12-27 06:15:54,412][20347] Updated weights for policy 0, policy_version 87367 (0.0007) [2023-12-27 06:15:54,435][20381] Updated weights for policy 1, policy_version 87084 (0.0006) [2023-12-27 06:15:54,459][20347] Updated weights for policy 0, policy_version 87377 (0.0008) [2023-12-27 06:15:54,486][20381] Updated weights for policy 1, policy_version 87094 (0.0005) [2023-12-27 06:15:54,537][20381] Updated weights for policy 1, policy_version 87104 (0.0010) [2023-12-27 06:15:55,113][20347] Updated weights for policy 0, policy_version 87387 (0.0007) [2023-12-27 06:15:55,169][20347] Updated weights for policy 0, policy_version 87397 (0.0006) [2023-12-27 06:15:55,182][20381] Updated weights for policy 1, policy_version 87114 (0.0011) [2023-12-27 06:15:55,222][20347] Updated weights for policy 0, policy_version 87407 (0.0006) [2023-12-27 06:15:55,238][20381] Updated weights for policy 1, policy_version 87124 (0.0008) [2023-12-27 06:15:55,292][20381] Updated weights for policy 1, policy_version 87134 (0.0008) [2023-12-27 06:15:55,890][20381] Updated weights for policy 1, policy_version 87144 (0.0007) [2023-12-27 06:15:55,893][20347] Updated weights for policy 0, policy_version 87417 (0.0006) [2023-12-27 06:15:55,946][20347] Updated weights for policy 0, policy_version 87427 (0.0006) [2023-12-27 06:15:55,947][20381] Updated weights for policy 1, policy_version 87154 (0.0007) [2023-12-27 06:15:56,005][20381] Updated weights for policy 1, policy_version 87164 (0.0006) [2023-12-27 06:15:56,006][20347] Updated weights for policy 0, policy_version 87437 (0.0005) [2023-12-27 06:15:56,053][20347] Updated weights for policy 0, policy_version 87447 (0.0005) [2023-12-27 06:15:56,550][20381] Updated weights for policy 1, policy_version 87174 (0.0006) [2023-12-27 06:15:56,606][20381] Updated weights for policy 1, policy_version 87184 (0.0007) [2023-12-27 06:15:56,650][20347] Updated weights for policy 0, policy_version 87457 (0.0006) [2023-12-27 06:15:56,667][20381] Updated weights for policy 1, policy_version 87194 (0.0006) [2023-12-27 06:15:56,697][20347] Updated weights for policy 0, policy_version 87467 (0.0005) [2023-12-27 06:15:56,748][20347] Updated weights for policy 0, policy_version 87477 (0.0005) [2023-12-27 06:15:57,335][20347] Updated weights for policy 0, policy_version 87487 (0.0005) [2023-12-27 06:15:57,337][20381] Updated weights for policy 1, policy_version 87204 (0.0005) [2023-12-27 06:15:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 44736512. Throughput: 0: 10491.7, 1: 10221.8. Samples: 44729716. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:15:57,363][19170] Avg episode reward: [(0, '381.440'), (1, '1000.000')] [2023-12-27 06:15:57,384][20347] Updated weights for policy 0, policy_version 87497 (0.0005) [2023-12-27 06:15:57,392][20381] Updated weights for policy 1, policy_version 87214 (0.0006) [2023-12-27 06:15:57,431][20347] Updated weights for policy 0, policy_version 87507 (0.0006) [2023-12-27 06:15:57,449][20381] Updated weights for policy 1, policy_version 87224 (0.0006) [2023-12-27 06:15:58,063][20381] Updated weights for policy 1, policy_version 87234 (0.0006) [2023-12-27 06:15:58,082][20347] Updated weights for policy 0, policy_version 87517 (0.0006) [2023-12-27 06:15:58,114][20381] Updated weights for policy 1, policy_version 87244 (0.0005) [2023-12-27 06:15:58,135][20347] Updated weights for policy 0, policy_version 87527 (0.0007) [2023-12-27 06:15:58,174][20381] Updated weights for policy 1, policy_version 87254 (0.0007) [2023-12-27 06:15:58,199][20347] Updated weights for policy 0, policy_version 87537 (0.0007) [2023-12-27 06:15:58,236][20381] Updated weights for policy 1, policy_version 87264 (0.0008) [2023-12-27 06:15:58,964][20347] Updated weights for policy 0, policy_version 87547 (0.0006) [2023-12-27 06:15:58,978][20381] Updated weights for policy 1, policy_version 87274 (0.0008) [2023-12-27 06:15:59,011][20347] Updated weights for policy 0, policy_version 87557 (0.0006) [2023-12-27 06:15:59,028][20381] Updated weights for policy 1, policy_version 87284 (0.0006) [2023-12-27 06:15:59,067][20347] Updated weights for policy 0, policy_version 87567 (0.0006) [2023-12-27 06:15:59,087][20381] Updated weights for policy 1, policy_version 87294 (0.0007) [2023-12-27 06:15:59,720][20347] Updated weights for policy 0, policy_version 87577 (0.0007) [2023-12-27 06:15:59,744][20381] Updated weights for policy 1, policy_version 87304 (0.0007) [2023-12-27 06:15:59,773][20347] Updated weights for policy 0, policy_version 87587 (0.0007) [2023-12-27 06:15:59,801][20381] Updated weights for policy 1, policy_version 87314 (0.0006) [2023-12-27 06:15:59,834][20347] Updated weights for policy 0, policy_version 87597 (0.0006) [2023-12-27 06:15:59,863][20381] Updated weights for policy 1, policy_version 87324 (0.0008) [2023-12-27 06:15:59,887][20347] Updated weights for policy 0, policy_version 87607 (0.0007) [2023-12-27 06:16:00,472][20381] Updated weights for policy 1, policy_version 87334 (0.0006) [2023-12-27 06:16:00,525][20381] Updated weights for policy 1, policy_version 87344 (0.0008) [2023-12-27 06:16:00,531][20347] Updated weights for policy 0, policy_version 87617 (0.0006) [2023-12-27 06:16:00,581][20381] Updated weights for policy 1, policy_version 87354 (0.0007) [2023-12-27 06:16:00,582][20347] Updated weights for policy 0, policy_version 87627 (0.0008) [2023-12-27 06:16:00,624][20347] Updated weights for policy 0, policy_version 87637 (0.0007) [2023-12-27 06:16:01,287][20381] Updated weights for policy 1, policy_version 87364 (0.0008) [2023-12-27 06:16:01,313][20347] Updated weights for policy 0, policy_version 87647 (0.0008) [2023-12-27 06:16:01,350][20381] Updated weights for policy 1, policy_version 87374 (0.0006) [2023-12-27 06:16:01,379][20347] Updated weights for policy 0, policy_version 87657 (0.0008) [2023-12-27 06:16:01,413][20381] Updated weights for policy 1, policy_version 87384 (0.0008) [2023-12-27 06:16:01,439][20347] Updated weights for policy 0, policy_version 87667 (0.0008) [2023-12-27 06:16:02,026][20381] Updated weights for policy 1, policy_version 87394 (0.0008) [2023-12-27 06:16:02,083][20381] Updated weights for policy 1, policy_version 87404 (0.0011) [2023-12-27 06:16:02,139][20381] Updated weights for policy 1, policy_version 87414 (0.0010) [2023-12-27 06:16:02,163][20347] Updated weights for policy 0, policy_version 87677 (0.0007) [2023-12-27 06:16:02,194][20381] Updated weights for policy 1, policy_version 87424 (0.0009) [2023-12-27 06:16:02,214][20347] Updated weights for policy 0, policy_version 87687 (0.0007) [2023-12-27 06:16:02,272][20347] Updated weights for policy 0, policy_version 87697 (0.0007) [2023-12-27 06:16:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 44851200. Throughput: 0: 10547.6, 1: 10315.0. Samples: 44855840. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:16:02,363][19170] Avg episode reward: [(0, '333.040'), (1, '1000.000')] [2023-12-27 06:16:02,883][20381] Updated weights for policy 1, policy_version 87434 (0.0008) [2023-12-27 06:16:02,935][20381] Updated weights for policy 1, policy_version 87444 (0.0010) [2023-12-27 06:16:02,978][20347] Updated weights for policy 0, policy_version 87707 (0.0008) [2023-12-27 06:16:02,991][20381] Updated weights for policy 1, policy_version 87454 (0.0010) [2023-12-27 06:16:03,034][20347] Updated weights for policy 0, policy_version 87717 (0.0006) [2023-12-27 06:16:03,085][20347] Updated weights for policy 0, policy_version 87727 (0.0007) [2023-12-27 06:16:03,678][20381] Updated weights for policy 1, policy_version 87464 (0.0010) [2023-12-27 06:16:03,735][20381] Updated weights for policy 1, policy_version 87474 (0.0010) [2023-12-27 06:16:03,754][20347] Updated weights for policy 0, policy_version 87737 (0.0007) [2023-12-27 06:16:03,784][20381] Updated weights for policy 1, policy_version 87484 (0.0010) [2023-12-27 06:16:03,805][20347] Updated weights for policy 0, policy_version 87747 (0.0010) [2023-12-27 06:16:03,868][20347] Updated weights for policy 0, policy_version 87757 (0.0010) [2023-12-27 06:16:03,927][20347] Updated weights for policy 0, policy_version 87767 (0.0009) [2023-12-27 06:16:04,502][20381] Updated weights for policy 1, policy_version 87494 (0.0010) [2023-12-27 06:16:04,553][20381] Updated weights for policy 1, policy_version 87504 (0.0011) [2023-12-27 06:16:04,612][20381] Updated weights for policy 1, policy_version 87514 (0.0011) [2023-12-27 06:16:04,614][20347] Updated weights for policy 0, policy_version 87777 (0.0010) [2023-12-27 06:16:04,667][20347] Updated weights for policy 0, policy_version 87787 (0.0010) [2023-12-27 06:16:04,720][20347] Updated weights for policy 0, policy_version 87797 (0.0008) [2023-12-27 06:16:05,277][20381] Updated weights for policy 1, policy_version 87524 (0.0011) [2023-12-27 06:16:05,335][20381] Updated weights for policy 1, policy_version 87534 (0.0010) [2023-12-27 06:16:05,400][20381] Updated weights for policy 1, policy_version 87544 (0.0011) [2023-12-27 06:16:05,408][20347] Updated weights for policy 0, policy_version 87807 (0.0005) [2023-12-27 06:16:05,460][20347] Updated weights for policy 0, policy_version 87817 (0.0009) [2023-12-27 06:16:05,513][20347] Updated weights for policy 0, policy_version 87827 (0.0010) [2023-12-27 06:16:06,073][20381] Updated weights for policy 1, policy_version 87554 (0.0011) [2023-12-27 06:16:06,130][20381] Updated weights for policy 1, policy_version 87564 (0.0011) [2023-12-27 06:16:06,176][20347] Updated weights for policy 0, policy_version 87837 (0.0010) [2023-12-27 06:16:06,195][20381] Updated weights for policy 1, policy_version 87574 (0.0011) [2023-12-27 06:16:06,231][20347] Updated weights for policy 0, policy_version 87847 (0.0010) [2023-12-27 06:16:06,259][20381] Updated weights for policy 1, policy_version 87584 (0.0010) [2023-12-27 06:16:06,289][20347] Updated weights for policy 0, policy_version 87857 (0.0010) [2023-12-27 06:16:06,858][20381] Updated weights for policy 1, policy_version 87594 (0.0005) [2023-12-27 06:16:06,912][20381] Updated weights for policy 1, policy_version 87604 (0.0006) [2023-12-27 06:16:06,974][20381] Updated weights for policy 1, policy_version 87614 (0.0005) [2023-12-27 06:16:06,994][20347] Updated weights for policy 0, policy_version 87867 (0.0010) [2023-12-27 06:16:07,050][20347] Updated weights for policy 0, policy_version 87877 (0.0010) [2023-12-27 06:16:07,095][20347] Updated weights for policy 0, policy_version 87887 (0.0005) [2023-12-27 06:16:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 44949504. Throughput: 0: 10539.2, 1: 10344.3. Samples: 44916784. Policy #0 lag: (min: 20.0, avg: 26.0, max: 52.0) [2023-12-27 06:16:07,363][19170] Avg episode reward: [(0, '218.090'), (1, '1000.000')] [2023-12-27 06:16:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000087616_22437888.pth... [2023-12-27 06:16:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000087896_22511616.pth... [2023-12-27 06:16:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000086400_22126592.pth [2023-12-27 06:16:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000086648_22192128.pth [2023-12-27 06:16:07,586][20381] Updated weights for policy 1, policy_version 87624 (0.0010) [2023-12-27 06:16:07,632][20381] Updated weights for policy 1, policy_version 87634 (0.0010) [2023-12-27 06:16:07,683][20381] Updated weights for policy 1, policy_version 87644 (0.0010) [2023-12-27 06:16:07,748][20347] Updated weights for policy 0, policy_version 87897 (0.0005) [2023-12-27 06:16:07,807][20347] Updated weights for policy 0, policy_version 87907 (0.0005) [2023-12-27 06:16:07,864][20347] Updated weights for policy 0, policy_version 87917 (0.0005) [2023-12-27 06:16:07,918][20347] Updated weights for policy 0, policy_version 87927 (0.0007) [2023-12-27 06:16:08,380][20381] Updated weights for policy 1, policy_version 87654 (0.0009) [2023-12-27 06:16:08,441][20381] Updated weights for policy 1, policy_version 87664 (0.0008) [2023-12-27 06:16:08,492][20381] Updated weights for policy 1, policy_version 87674 (0.0008) [2023-12-27 06:16:08,597][20347] Updated weights for policy 0, policy_version 87937 (0.0010) [2023-12-27 06:16:08,655][20347] Updated weights for policy 0, policy_version 87947 (0.0010) [2023-12-27 06:16:08,715][20347] Updated weights for policy 0, policy_version 87957 (0.0010) [2023-12-27 06:16:09,189][20381] Updated weights for policy 1, policy_version 87684 (0.0008) [2023-12-27 06:16:09,255][20381] Updated weights for policy 1, policy_version 87694 (0.0011) [2023-12-27 06:16:09,318][20381] Updated weights for policy 1, policy_version 87704 (0.0011) [2023-12-27 06:16:09,443][20347] Updated weights for policy 0, policy_version 87967 (0.0008) [2023-12-27 06:16:09,497][20347] Updated weights for policy 0, policy_version 87977 (0.0010) [2023-12-27 06:16:09,558][20347] Updated weights for policy 0, policy_version 87987 (0.0008) [2023-12-27 06:16:09,990][20381] Updated weights for policy 1, policy_version 87714 (0.0010) [2023-12-27 06:16:10,036][20381] Updated weights for policy 1, policy_version 87724 (0.0006) [2023-12-27 06:16:10,090][20381] Updated weights for policy 1, policy_version 87734 (0.0007) [2023-12-27 06:16:10,142][20381] Updated weights for policy 1, policy_version 87744 (0.0005) [2023-12-27 06:16:10,292][20347] Updated weights for policy 0, policy_version 87997 (0.0007) [2023-12-27 06:16:10,359][20347] Updated weights for policy 0, policy_version 88007 (0.0008) [2023-12-27 06:16:10,424][20347] Updated weights for policy 0, policy_version 88017 (0.0007) [2023-12-27 06:16:10,744][20381] Updated weights for policy 1, policy_version 87754 (0.0006) [2023-12-27 06:16:10,797][20381] Updated weights for policy 1, policy_version 87764 (0.0007) [2023-12-27 06:16:10,846][20381] Updated weights for policy 1, policy_version 87774 (0.0007) [2023-12-27 06:16:11,069][20347] Updated weights for policy 0, policy_version 88027 (0.0007) [2023-12-27 06:16:11,139][20347] Updated weights for policy 0, policy_version 88037 (0.0007) [2023-12-27 06:16:11,196][20347] Updated weights for policy 0, policy_version 88047 (0.0009) [2023-12-27 06:16:11,513][20381] Updated weights for policy 1, policy_version 87784 (0.0009) [2023-12-27 06:16:11,569][20381] Updated weights for policy 1, policy_version 87794 (0.0010) [2023-12-27 06:16:11,638][20381] Updated weights for policy 1, policy_version 87804 (0.0010) [2023-12-27 06:16:11,927][20347] Updated weights for policy 0, policy_version 88057 (0.0007) [2023-12-27 06:16:11,994][20347] Updated weights for policy 0, policy_version 88067 (0.0007) [2023-12-27 06:16:12,057][20347] Updated weights for policy 0, policy_version 88077 (0.0006) [2023-12-27 06:16:12,120][20347] Updated weights for policy 0, policy_version 88087 (0.0006) [2023-12-27 06:16:12,347][20381] Updated weights for policy 1, policy_version 87814 (0.0010) [2023-12-27 06:16:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 45047808. Throughput: 0: 10513.7, 1: 10366.4. Samples: 45041152. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:12,363][19170] Avg episode reward: [(0, '386.900'), (1, '1000.000')] [2023-12-27 06:16:12,407][20381] Updated weights for policy 1, policy_version 87824 (0.0007) [2023-12-27 06:16:12,467][20381] Updated weights for policy 1, policy_version 87834 (0.0006) [2023-12-27 06:16:12,770][20347] Updated weights for policy 0, policy_version 88097 (0.0006) [2023-12-27 06:16:12,832][20347] Updated weights for policy 0, policy_version 88107 (0.0008) [2023-12-27 06:16:12,888][20347] Updated weights for policy 0, policy_version 88117 (0.0007) [2023-12-27 06:16:13,113][20381] Updated weights for policy 1, policy_version 87844 (0.0007) [2023-12-27 06:16:13,157][20381] Updated weights for policy 1, policy_version 87854 (0.0008) [2023-12-27 06:16:13,207][20381] Updated weights for policy 1, policy_version 87864 (0.0008) [2023-12-27 06:16:13,460][20347] Updated weights for policy 0, policy_version 88127 (0.0006) [2023-12-27 06:16:13,517][20347] Updated weights for policy 0, policy_version 88137 (0.0005) [2023-12-27 06:16:13,567][20347] Updated weights for policy 0, policy_version 88147 (0.0005) [2023-12-27 06:16:13,905][20381] Updated weights for policy 1, policy_version 87874 (0.0008) [2023-12-27 06:16:13,956][20381] Updated weights for policy 1, policy_version 87884 (0.0007) [2023-12-27 06:16:14,002][20381] Updated weights for policy 1, policy_version 87894 (0.0005) [2023-12-27 06:16:14,050][20381] Updated weights for policy 1, policy_version 87904 (0.0007) [2023-12-27 06:16:14,156][20347] Updated weights for policy 0, policy_version 88157 (0.0007) [2023-12-27 06:16:14,204][20347] Updated weights for policy 0, policy_version 88167 (0.0007) [2023-12-27 06:16:14,255][20347] Updated weights for policy 0, policy_version 88177 (0.0007) [2023-12-27 06:16:14,727][20381] Updated weights for policy 1, policy_version 87914 (0.0006) [2023-12-27 06:16:14,784][20381] Updated weights for policy 1, policy_version 87924 (0.0006) [2023-12-27 06:16:14,831][20381] Updated weights for policy 1, policy_version 87934 (0.0007) [2023-12-27 06:16:14,985][20347] Updated weights for policy 0, policy_version 88187 (0.0007) [2023-12-27 06:16:15,051][20347] Updated weights for policy 0, policy_version 88197 (0.0008) [2023-12-27 06:16:15,108][20347] Updated weights for policy 0, policy_version 88207 (0.0008) [2023-12-27 06:16:15,486][20381] Updated weights for policy 1, policy_version 87944 (0.0008) [2023-12-27 06:16:15,545][20381] Updated weights for policy 1, policy_version 87954 (0.0007) [2023-12-27 06:16:15,598][20381] Updated weights for policy 1, policy_version 87964 (0.0008) [2023-12-27 06:16:15,741][20347] Updated weights for policy 0, policy_version 88217 (0.0008) [2023-12-27 06:16:15,790][20347] Updated weights for policy 0, policy_version 88227 (0.0005) [2023-12-27 06:16:15,849][20347] Updated weights for policy 0, policy_version 88237 (0.0008) [2023-12-27 06:16:15,906][20347] Updated weights for policy 0, policy_version 88247 (0.0010) [2023-12-27 06:16:16,217][20381] Updated weights for policy 1, policy_version 87974 (0.0006) [2023-12-27 06:16:16,273][20381] Updated weights for policy 1, policy_version 87984 (0.0005) [2023-12-27 06:16:16,330][20381] Updated weights for policy 1, policy_version 87994 (0.0006) [2023-12-27 06:16:16,573][20347] Updated weights for policy 0, policy_version 88257 (0.0010) [2023-12-27 06:16:16,627][20347] Updated weights for policy 0, policy_version 88267 (0.0010) [2023-12-27 06:16:16,685][20347] Updated weights for policy 0, policy_version 88277 (0.0010) [2023-12-27 06:16:16,890][20381] Updated weights for policy 1, policy_version 88004 (0.0006) [2023-12-27 06:16:16,972][20381] Updated weights for policy 1, policy_version 88014 (0.0007) [2023-12-27 06:16:17,023][20381] Updated weights for policy 1, policy_version 88024 (0.0007) [2023-12-27 06:16:17,355][20347] Updated weights for policy 0, policy_version 88287 (0.0008) [2023-12-27 06:16:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 45154304. Throughput: 0: 10451.1, 1: 10406.4. Samples: 45166368. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:17,363][19170] Avg episode reward: [(0, '184.420'), (1, '1000.000')] [2023-12-27 06:16:17,441][20347] Updated weights for policy 0, policy_version 88297 (0.0006) [2023-12-27 06:16:17,487][20347] Updated weights for policy 0, policy_version 88307 (0.0007) [2023-12-27 06:16:17,645][20381] Updated weights for policy 1, policy_version 88034 (0.0007) [2023-12-27 06:16:17,694][20381] Updated weights for policy 1, policy_version 88044 (0.0005) [2023-12-27 06:16:17,749][20381] Updated weights for policy 1, policy_version 88054 (0.0010) [2023-12-27 06:16:17,800][20381] Updated weights for policy 1, policy_version 88064 (0.0010) [2023-12-27 06:16:18,086][20347] Updated weights for policy 0, policy_version 88317 (0.0008) [2023-12-27 06:16:18,143][20347] Updated weights for policy 0, policy_version 88327 (0.0008) [2023-12-27 06:16:18,194][20347] Updated weights for policy 0, policy_version 88337 (0.0007) [2023-12-27 06:16:18,490][20381] Updated weights for policy 1, policy_version 88074 (0.0011) [2023-12-27 06:16:18,549][20381] Updated weights for policy 1, policy_version 88084 (0.0011) [2023-12-27 06:16:18,611][20381] Updated weights for policy 1, policy_version 88094 (0.0011) [2023-12-27 06:16:18,877][20347] Updated weights for policy 0, policy_version 88347 (0.0007) [2023-12-27 06:16:18,928][20347] Updated weights for policy 0, policy_version 88357 (0.0007) [2023-12-27 06:16:18,986][20347] Updated weights for policy 0, policy_version 88367 (0.0007) [2023-12-27 06:16:19,264][20381] Updated weights for policy 1, policy_version 88104 (0.0008) [2023-12-27 06:16:19,319][20381] Updated weights for policy 1, policy_version 88114 (0.0006) [2023-12-27 06:16:19,385][20381] Updated weights for policy 1, policy_version 88124 (0.0007) [2023-12-27 06:16:19,746][20347] Updated weights for policy 0, policy_version 88377 (0.0008) [2023-12-27 06:16:19,814][20347] Updated weights for policy 0, policy_version 88387 (0.0009) [2023-12-27 06:16:19,881][20347] Updated weights for policy 0, policy_version 88397 (0.0008) [2023-12-27 06:16:19,947][20347] Updated weights for policy 0, policy_version 88407 (0.0010) [2023-12-27 06:16:20,087][20381] Updated weights for policy 1, policy_version 88134 (0.0007) [2023-12-27 06:16:20,145][20381] Updated weights for policy 1, policy_version 88144 (0.0008) [2023-12-27 06:16:20,198][20381] Updated weights for policy 1, policy_version 88154 (0.0008) [2023-12-27 06:16:20,610][20347] Updated weights for policy 0, policy_version 88417 (0.0008) [2023-12-27 06:16:20,662][20347] Updated weights for policy 0, policy_version 88427 (0.0008) [2023-12-27 06:16:20,725][20347] Updated weights for policy 0, policy_version 88437 (0.0007) [2023-12-27 06:16:20,893][20381] Updated weights for policy 1, policy_version 88164 (0.0008) [2023-12-27 06:16:20,951][20381] Updated weights for policy 1, policy_version 88174 (0.0008) [2023-12-27 06:16:21,011][20381] Updated weights for policy 1, policy_version 88184 (0.0008) [2023-12-27 06:16:21,456][20347] Updated weights for policy 0, policy_version 88447 (0.0007) [2023-12-27 06:16:21,522][20347] Updated weights for policy 0, policy_version 88457 (0.0008) [2023-12-27 06:16:21,592][20347] Updated weights for policy 0, policy_version 88467 (0.0006) [2023-12-27 06:16:21,725][20381] Updated weights for policy 1, policy_version 88194 (0.0009) [2023-12-27 06:16:21,795][20381] Updated weights for policy 1, policy_version 88204 (0.0008) [2023-12-27 06:16:21,852][20381] Updated weights for policy 1, policy_version 88214 (0.0009) [2023-12-27 06:16:21,907][20381] Updated weights for policy 1, policy_version 88224 (0.0009) [2023-12-27 06:16:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 45252608. Throughput: 0: 10412.8, 1: 10410.0. Samples: 45227068. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:22,363][19170] Avg episode reward: [(0, '188.440'), (1, '1000.000')] [2023-12-27 06:16:22,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000088224_22593536.pth... [2023-12-27 06:16:22,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000087008_22282240.pth [2023-12-27 06:16:22,388][20347] Updated weights for policy 0, policy_version 88477 (0.0009) [2023-12-27 06:16:22,448][20347] Updated weights for policy 0, policy_version 88487 (0.0011) [2023-12-27 06:16:22,505][20347] Updated weights for policy 0, policy_version 88497 (0.0010) [2023-12-27 06:16:22,537][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000088504_22667264.pth... [2023-12-27 06:16:22,540][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000087288_22355968.pth [2023-12-27 06:16:22,695][20381] Updated weights for policy 1, policy_version 88234 (0.0009) [2023-12-27 06:16:22,756][20381] Updated weights for policy 1, policy_version 88244 (0.0007) [2023-12-27 06:16:22,821][20381] Updated weights for policy 1, policy_version 88254 (0.0006) [2023-12-27 06:16:23,108][20347] Updated weights for policy 0, policy_version 88507 (0.0008) [2023-12-27 06:16:23,155][20347] Updated weights for policy 0, policy_version 88517 (0.0005) [2023-12-27 06:16:23,210][20347] Updated weights for policy 0, policy_version 88527 (0.0005) [2023-12-27 06:16:23,519][20381] Updated weights for policy 1, policy_version 88264 (0.0006) [2023-12-27 06:16:23,572][20381] Updated weights for policy 1, policy_version 88274 (0.0006) [2023-12-27 06:16:23,635][20381] Updated weights for policy 1, policy_version 88284 (0.0010) [2023-12-27 06:16:23,859][20347] Updated weights for policy 0, policy_version 88537 (0.0007) [2023-12-27 06:16:23,913][20347] Updated weights for policy 0, policy_version 88547 (0.0005) [2023-12-27 06:16:23,968][20347] Updated weights for policy 0, policy_version 88557 (0.0006) [2023-12-27 06:16:24,017][20347] Updated weights for policy 0, policy_version 88567 (0.0008) [2023-12-27 06:16:24,286][20381] Updated weights for policy 1, policy_version 88294 (0.0009) [2023-12-27 06:16:24,341][20381] Updated weights for policy 1, policy_version 88304 (0.0008) [2023-12-27 06:16:24,393][20381] Updated weights for policy 1, policy_version 88314 (0.0008) [2023-12-27 06:16:24,666][20347] Updated weights for policy 0, policy_version 88577 (0.0005) [2023-12-27 06:16:24,717][20347] Updated weights for policy 0, policy_version 88587 (0.0005) [2023-12-27 06:16:24,762][20347] Updated weights for policy 0, policy_version 88597 (0.0005) [2023-12-27 06:16:25,048][20381] Updated weights for policy 1, policy_version 88324 (0.0006) [2023-12-27 06:16:25,104][20381] Updated weights for policy 1, policy_version 88334 (0.0007) [2023-12-27 06:16:25,153][20381] Updated weights for policy 1, policy_version 88344 (0.0008) [2023-12-27 06:16:25,418][20347] Updated weights for policy 0, policy_version 88607 (0.0007) [2023-12-27 06:16:25,480][20347] Updated weights for policy 0, policy_version 88617 (0.0008) [2023-12-27 06:16:25,533][20347] Updated weights for policy 0, policy_version 88627 (0.0006) [2023-12-27 06:16:25,828][20381] Updated weights for policy 1, policy_version 88354 (0.0007) [2023-12-27 06:16:25,892][20381] Updated weights for policy 1, policy_version 88364 (0.0008) [2023-12-27 06:16:25,944][20381] Updated weights for policy 1, policy_version 88374 (0.0008) [2023-12-27 06:16:25,989][20381] Updated weights for policy 1, policy_version 88384 (0.0006) [2023-12-27 06:16:26,141][20347] Updated weights for policy 0, policy_version 88637 (0.0007) [2023-12-27 06:16:26,190][20347] Updated weights for policy 0, policy_version 88647 (0.0008) [2023-12-27 06:16:26,245][20347] Updated weights for policy 0, policy_version 88657 (0.0008) [2023-12-27 06:16:26,646][20381] Updated weights for policy 1, policy_version 88394 (0.0006) [2023-12-27 06:16:26,699][20381] Updated weights for policy 1, policy_version 88404 (0.0008) [2023-12-27 06:16:26,751][20381] Updated weights for policy 1, policy_version 88414 (0.0007) [2023-12-27 06:16:26,895][20347] Updated weights for policy 0, policy_version 88667 (0.0007) [2023-12-27 06:16:26,951][20347] Updated weights for policy 0, policy_version 88677 (0.0008) [2023-12-27 06:16:27,019][20347] Updated weights for policy 0, policy_version 88687 (0.0007) [2023-12-27 06:16:27,360][20381] Updated weights for policy 1, policy_version 88424 (0.0008) [2023-12-27 06:16:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 45359104. Throughput: 0: 10432.5, 1: 10392.7. Samples: 45350220. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:27,363][19170] Avg episode reward: [(0, '252.560'), (1, '1000.000')] [2023-12-27 06:16:27,413][20381] Updated weights for policy 1, policy_version 88434 (0.0008) [2023-12-27 06:16:27,472][20381] Updated weights for policy 1, policy_version 88444 (0.0008) [2023-12-27 06:16:27,642][20347] Updated weights for policy 0, policy_version 88697 (0.0007) [2023-12-27 06:16:27,703][20347] Updated weights for policy 0, policy_version 88707 (0.0005) [2023-12-27 06:16:27,760][20347] Updated weights for policy 0, policy_version 88717 (0.0005) [2023-12-27 06:16:27,819][20347] Updated weights for policy 0, policy_version 88727 (0.0005) [2023-12-27 06:16:28,209][20381] Updated weights for policy 1, policy_version 88454 (0.0008) [2023-12-27 06:16:28,263][20381] Updated weights for policy 1, policy_version 88464 (0.0007) [2023-12-27 06:16:28,315][20381] Updated weights for policy 1, policy_version 88474 (0.0007) [2023-12-27 06:16:28,368][20347] Updated weights for policy 0, policy_version 88737 (0.0006) [2023-12-27 06:16:28,417][20347] Updated weights for policy 0, policy_version 88747 (0.0006) [2023-12-27 06:16:28,474][20347] Updated weights for policy 0, policy_version 88757 (0.0008) [2023-12-27 06:16:28,904][20381] Updated weights for policy 1, policy_version 88484 (0.0007) [2023-12-27 06:16:28,961][20381] Updated weights for policy 1, policy_version 88494 (0.0005) [2023-12-27 06:16:29,021][20381] Updated weights for policy 1, policy_version 88504 (0.0008) [2023-12-27 06:16:29,129][20347] Updated weights for policy 0, policy_version 88767 (0.0008) [2023-12-27 06:16:29,183][20347] Updated weights for policy 0, policy_version 88777 (0.0007) [2023-12-27 06:16:29,242][20347] Updated weights for policy 0, policy_version 88787 (0.0008) [2023-12-27 06:16:29,681][20381] Updated weights for policy 1, policy_version 88514 (0.0008) [2023-12-27 06:16:29,736][20381] Updated weights for policy 1, policy_version 88524 (0.0007) [2023-12-27 06:16:29,792][20381] Updated weights for policy 1, policy_version 88534 (0.0008) [2023-12-27 06:16:29,853][20381] Updated weights for policy 1, policy_version 88544 (0.0008) [2023-12-27 06:16:29,942][20347] Updated weights for policy 0, policy_version 88797 (0.0007) [2023-12-27 06:16:29,992][20347] Updated weights for policy 0, policy_version 88807 (0.0009) [2023-12-27 06:16:30,047][20347] Updated weights for policy 0, policy_version 88817 (0.0007) [2023-12-27 06:16:30,555][20381] Updated weights for policy 1, policy_version 88554 (0.0007) [2023-12-27 06:16:30,600][20381] Updated weights for policy 1, policy_version 88564 (0.0007) [2023-12-27 06:16:30,654][20381] Updated weights for policy 1, policy_version 88574 (0.0008) [2023-12-27 06:16:30,732][20347] Updated weights for policy 0, policy_version 88827 (0.0008) [2023-12-27 06:16:30,781][20347] Updated weights for policy 0, policy_version 88837 (0.0006) [2023-12-27 06:16:30,827][20347] Updated weights for policy 0, policy_version 88847 (0.0005) [2023-12-27 06:16:31,380][20381] Updated weights for policy 1, policy_version 88584 (0.0008) [2023-12-27 06:16:31,437][20381] Updated weights for policy 1, policy_version 88594 (0.0008) [2023-12-27 06:16:31,493][20381] Updated weights for policy 1, policy_version 88604 (0.0008) [2023-12-27 06:16:31,514][20347] Updated weights for policy 0, policy_version 88857 (0.0005) [2023-12-27 06:16:31,574][20347] Updated weights for policy 0, policy_version 88867 (0.0007) [2023-12-27 06:16:31,627][20347] Updated weights for policy 0, policy_version 88877 (0.0006) [2023-12-27 06:16:31,683][20347] Updated weights for policy 0, policy_version 88887 (0.0006) [2023-12-27 06:16:32,129][20381] Updated weights for policy 1, policy_version 88614 (0.0006) [2023-12-27 06:16:32,186][20381] Updated weights for policy 1, policy_version 88624 (0.0005) [2023-12-27 06:16:32,241][20381] Updated weights for policy 1, policy_version 88634 (0.0005) [2023-12-27 06:16:32,357][20347] Updated weights for policy 0, policy_version 88897 (0.0007) [2023-12-27 06:16:32,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 45465600. Throughput: 0: 10422.7, 1: 10379.0. Samples: 45476336. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:32,363][19170] Avg episode reward: [(0, '73.970'), (1, '1000.000')] [2023-12-27 06:16:32,418][20347] Updated weights for policy 0, policy_version 88907 (0.0006) [2023-12-27 06:16:32,473][20347] Updated weights for policy 0, policy_version 88917 (0.0006) [2023-12-27 06:16:32,902][20381] Updated weights for policy 1, policy_version 88644 (0.0008) [2023-12-27 06:16:32,960][20381] Updated weights for policy 1, policy_version 88654 (0.0010) [2023-12-27 06:16:33,015][20381] Updated weights for policy 1, policy_version 88664 (0.0010) [2023-12-27 06:16:33,055][20347] Updated weights for policy 0, policy_version 88927 (0.0005) [2023-12-27 06:16:33,129][20347] Updated weights for policy 0, policy_version 88937 (0.0006) [2023-12-27 06:16:33,183][20347] Updated weights for policy 0, policy_version 88947 (0.0007) [2023-12-27 06:16:33,630][20381] Updated weights for policy 1, policy_version 88674 (0.0009) [2023-12-27 06:16:33,688][20381] Updated weights for policy 1, policy_version 88684 (0.0005) [2023-12-27 06:16:33,749][20381] Updated weights for policy 1, policy_version 88694 (0.0007) [2023-12-27 06:16:33,799][20347] Updated weights for policy 0, policy_version 88957 (0.0007) [2023-12-27 06:16:33,808][20381] Updated weights for policy 1, policy_version 88704 (0.0005) [2023-12-27 06:16:33,856][20347] Updated weights for policy 0, policy_version 88967 (0.0005) [2023-12-27 06:16:33,909][20347] Updated weights for policy 0, policy_version 88977 (0.0007) [2023-12-27 06:16:34,324][20381] Updated weights for policy 1, policy_version 88714 (0.0006) [2023-12-27 06:16:34,391][20381] Updated weights for policy 1, policy_version 88724 (0.0011) [2023-12-27 06:16:34,453][20381] Updated weights for policy 1, policy_version 88734 (0.0011) [2023-12-27 06:16:34,629][20347] Updated weights for policy 0, policy_version 88987 (0.0008) [2023-12-27 06:16:34,688][20347] Updated weights for policy 0, policy_version 88997 (0.0007) [2023-12-27 06:16:34,744][20347] Updated weights for policy 0, policy_version 89007 (0.0007) [2023-12-27 06:16:35,122][20381] Updated weights for policy 1, policy_version 88744 (0.0009) [2023-12-27 06:16:35,189][20381] Updated weights for policy 1, policy_version 88754 (0.0011) [2023-12-27 06:16:35,256][20381] Updated weights for policy 1, policy_version 88764 (0.0011) [2023-12-27 06:16:35,402][20347] Updated weights for policy 0, policy_version 89017 (0.0007) [2023-12-27 06:16:35,454][20347] Updated weights for policy 0, policy_version 89027 (0.0008) [2023-12-27 06:16:35,507][20347] Updated weights for policy 0, policy_version 89037 (0.0008) [2023-12-27 06:16:35,570][20347] Updated weights for policy 0, policy_version 89047 (0.0007) [2023-12-27 06:16:35,944][20381] Updated weights for policy 1, policy_version 88774 (0.0010) [2023-12-27 06:16:36,004][20381] Updated weights for policy 1, policy_version 88784 (0.0010) [2023-12-27 06:16:36,064][20381] Updated weights for policy 1, policy_version 88794 (0.0010) [2023-12-27 06:16:36,226][20347] Updated weights for policy 0, policy_version 89057 (0.0008) [2023-12-27 06:16:36,286][20347] Updated weights for policy 0, policy_version 89067 (0.0007) [2023-12-27 06:16:36,344][20347] Updated weights for policy 0, policy_version 89077 (0.0007) [2023-12-27 06:16:36,709][20381] Updated weights for policy 1, policy_version 88804 (0.0010) [2023-12-27 06:16:36,773][20381] Updated weights for policy 1, policy_version 88814 (0.0010) [2023-12-27 06:16:36,838][20381] Updated weights for policy 1, policy_version 88824 (0.0007) [2023-12-27 06:16:36,989][20347] Updated weights for policy 0, policy_version 89087 (0.0006) [2023-12-27 06:16:37,047][20347] Updated weights for policy 0, policy_version 89097 (0.0006) [2023-12-27 06:16:37,099][20347] Updated weights for policy 0, policy_version 89107 (0.0007) [2023-12-27 06:16:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 45572096. Throughput: 0: 10363.9, 1: 10432.9. Samples: 45540080. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:37,363][19170] Avg episode reward: [(0, '45.460'), (1, '1000.000')] [2023-12-27 06:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000089112_22822912.pth... [2023-12-27 06:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000088832_22749184.pth... [2023-12-27 06:16:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000087896_22511616.pth [2023-12-27 06:16:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000087616_22437888.pth [2023-12-27 06:16:37,451][20381] Updated weights for policy 1, policy_version 88834 (0.0008) [2023-12-27 06:16:37,510][20381] Updated weights for policy 1, policy_version 88844 (0.0005) [2023-12-27 06:16:37,567][20381] Updated weights for policy 1, policy_version 88854 (0.0007) [2023-12-27 06:16:37,624][20381] Updated weights for policy 1, policy_version 88864 (0.0008) [2023-12-27 06:16:37,815][20347] Updated weights for policy 0, policy_version 89117 (0.0008) [2023-12-27 06:16:37,865][20347] Updated weights for policy 0, policy_version 89127 (0.0007) [2023-12-27 06:16:37,916][20347] Updated weights for policy 0, policy_version 89137 (0.0007) [2023-12-27 06:16:38,249][20381] Updated weights for policy 1, policy_version 88874 (0.0008) [2023-12-27 06:16:38,308][20381] Updated weights for policy 1, policy_version 88884 (0.0007) [2023-12-27 06:16:38,376][20381] Updated weights for policy 1, policy_version 88894 (0.0009) [2023-12-27 06:16:38,624][20347] Updated weights for policy 0, policy_version 89147 (0.0008) [2023-12-27 06:16:38,685][20347] Updated weights for policy 0, policy_version 89157 (0.0006) [2023-12-27 06:16:38,742][20347] Updated weights for policy 0, policy_version 89167 (0.0005) [2023-12-27 06:16:39,057][20381] Updated weights for policy 1, policy_version 88904 (0.0008) [2023-12-27 06:16:39,122][20381] Updated weights for policy 1, policy_version 88914 (0.0007) [2023-12-27 06:16:39,173][20381] Updated weights for policy 1, policy_version 88924 (0.0007) [2023-12-27 06:16:39,414][20347] Updated weights for policy 0, policy_version 89177 (0.0008) [2023-12-27 06:16:39,479][20347] Updated weights for policy 0, policy_version 89187 (0.0011) [2023-12-27 06:16:39,538][20347] Updated weights for policy 0, policy_version 89197 (0.0010) [2023-12-27 06:16:39,595][20347] Updated weights for policy 0, policy_version 89207 (0.0010) [2023-12-27 06:16:39,913][20381] Updated weights for policy 1, policy_version 88934 (0.0007) [2023-12-27 06:16:39,985][20381] Updated weights for policy 1, policy_version 88944 (0.0008) [2023-12-27 06:16:40,052][20381] Updated weights for policy 1, policy_version 88954 (0.0008) [2023-12-27 06:16:40,314][20347] Updated weights for policy 0, policy_version 89217 (0.0010) [2023-12-27 06:16:40,374][20347] Updated weights for policy 0, policy_version 89227 (0.0010) [2023-12-27 06:16:40,424][20347] Updated weights for policy 0, policy_version 89237 (0.0011) [2023-12-27 06:16:40,680][20381] Updated weights for policy 1, policy_version 88964 (0.0008) [2023-12-27 06:16:40,736][20381] Updated weights for policy 1, policy_version 88974 (0.0007) [2023-12-27 06:16:40,793][20381] Updated weights for policy 1, policy_version 88984 (0.0005) [2023-12-27 06:16:41,036][20347] Updated weights for policy 0, policy_version 89247 (0.0009) [2023-12-27 06:16:41,099][20347] Updated weights for policy 0, policy_version 89257 (0.0010) [2023-12-27 06:16:41,166][20347] Updated weights for policy 0, policy_version 89267 (0.0010) [2023-12-27 06:16:41,453][20381] Updated weights for policy 1, policy_version 88994 (0.0006) [2023-12-27 06:16:41,515][20381] Updated weights for policy 1, policy_version 89004 (0.0007) [2023-12-27 06:16:41,579][20381] Updated weights for policy 1, policy_version 89014 (0.0005) [2023-12-27 06:16:41,646][20381] Updated weights for policy 1, policy_version 89024 (0.0007) [2023-12-27 06:16:41,907][20347] Updated weights for policy 0, policy_version 89277 (0.0009) [2023-12-27 06:16:41,975][20347] Updated weights for policy 0, policy_version 89287 (0.0010) [2023-12-27 06:16:42,040][20347] Updated weights for policy 0, policy_version 89297 (0.0011) [2023-12-27 06:16:42,327][20381] Updated weights for policy 1, policy_version 89034 (0.0006) [2023-12-27 06:16:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 45670400. Throughput: 0: 10356.4, 1: 10407.2. Samples: 45664080. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:42,363][19170] Avg episode reward: [(0, '47.810'), (1, '1000.000')] [2023-12-27 06:16:42,391][20381] Updated weights for policy 1, policy_version 89044 (0.0009) [2023-12-27 06:16:42,437][20381] Updated weights for policy 1, policy_version 89054 (0.0006) [2023-12-27 06:16:42,734][20347] Updated weights for policy 0, policy_version 89307 (0.0010) [2023-12-27 06:16:42,793][20347] Updated weights for policy 0, policy_version 89317 (0.0010) [2023-12-27 06:16:42,853][20347] Updated weights for policy 0, policy_version 89327 (0.0010) [2023-12-27 06:16:43,112][20381] Updated weights for policy 1, policy_version 89064 (0.0007) [2023-12-27 06:16:43,159][20381] Updated weights for policy 1, policy_version 89074 (0.0008) [2023-12-27 06:16:43,208][20381] Updated weights for policy 1, policy_version 89084 (0.0007) [2023-12-27 06:16:43,473][20347] Updated weights for policy 0, policy_version 89337 (0.0010) [2023-12-27 06:16:43,528][20347] Updated weights for policy 0, policy_version 89347 (0.0005) [2023-12-27 06:16:43,590][20347] Updated weights for policy 0, policy_version 89357 (0.0010) [2023-12-27 06:16:43,650][20347] Updated weights for policy 0, policy_version 89367 (0.0010) [2023-12-27 06:16:43,948][20381] Updated weights for policy 1, policy_version 89094 (0.0007) [2023-12-27 06:16:43,995][20381] Updated weights for policy 1, policy_version 89104 (0.0007) [2023-12-27 06:16:44,046][20381] Updated weights for policy 1, policy_version 89114 (0.0007) [2023-12-27 06:16:44,286][20347] Updated weights for policy 0, policy_version 89377 (0.0010) [2023-12-27 06:16:44,340][20347] Updated weights for policy 0, policy_version 89387 (0.0010) [2023-12-27 06:16:44,395][20347] Updated weights for policy 0, policy_version 89397 (0.0009) [2023-12-27 06:16:44,826][20381] Updated weights for policy 1, policy_version 89124 (0.0008) [2023-12-27 06:16:44,892][20381] Updated weights for policy 1, policy_version 89134 (0.0007) [2023-12-27 06:16:44,966][20381] Updated weights for policy 1, policy_version 89144 (0.0006) [2023-12-27 06:16:44,996][20347] Updated weights for policy 0, policy_version 89407 (0.0009) [2023-12-27 06:16:45,059][20347] Updated weights for policy 0, policy_version 89417 (0.0009) [2023-12-27 06:16:45,122][20347] Updated weights for policy 0, policy_version 89427 (0.0006) [2023-12-27 06:16:45,682][20381] Updated weights for policy 1, policy_version 89154 (0.0007) [2023-12-27 06:16:45,720][20347] Updated weights for policy 0, policy_version 89437 (0.0007) [2023-12-27 06:16:45,738][20381] Updated weights for policy 1, policy_version 89164 (0.0008) [2023-12-27 06:16:45,771][20347] Updated weights for policy 0, policy_version 89447 (0.0005) [2023-12-27 06:16:45,785][20381] Updated weights for policy 1, policy_version 89174 (0.0007) [2023-12-27 06:16:45,819][20347] Updated weights for policy 0, policy_version 89457 (0.0005) [2023-12-27 06:16:45,831][20381] Updated weights for policy 1, policy_version 89184 (0.0008) [2023-12-27 06:16:46,423][20347] Updated weights for policy 0, policy_version 89467 (0.0010) [2023-12-27 06:16:46,472][20347] Updated weights for policy 0, policy_version 89477 (0.0010) [2023-12-27 06:16:46,522][20347] Updated weights for policy 0, policy_version 89487 (0.0010) [2023-12-27 06:16:46,583][20381] Updated weights for policy 1, policy_version 89194 (0.0006) [2023-12-27 06:16:46,627][20381] Updated weights for policy 1, policy_version 89204 (0.0007) [2023-12-27 06:16:46,685][20381] Updated weights for policy 1, policy_version 89214 (0.0008) [2023-12-27 06:16:47,216][20347] Updated weights for policy 0, policy_version 89497 (0.0010) [2023-12-27 06:16:47,270][20347] Updated weights for policy 0, policy_version 89507 (0.0010) [2023-12-27 06:16:47,332][20347] Updated weights for policy 0, policy_version 89517 (0.0007) [2023-12-27 06:16:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 45768704. Throughput: 0: 10369.2, 1: 10303.9. Samples: 45786132. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:47,364][19170] Avg episode reward: [(0, '57.330'), (1, '1000.000')] [2023-12-27 06:16:47,379][20347] Updated weights for policy 0, policy_version 89527 (0.0010) [2023-12-27 06:16:47,388][20381] Updated weights for policy 1, policy_version 89224 (0.0006) [2023-12-27 06:16:47,446][20381] Updated weights for policy 1, policy_version 89234 (0.0008) [2023-12-27 06:16:47,504][20381] Updated weights for policy 1, policy_version 89244 (0.0008) [2023-12-27 06:16:48,004][20347] Updated weights for policy 0, policy_version 89537 (0.0010) [2023-12-27 06:16:48,057][20347] Updated weights for policy 0, policy_version 89547 (0.0010) [2023-12-27 06:16:48,107][20347] Updated weights for policy 0, policy_version 89557 (0.0010) [2023-12-27 06:16:48,210][20381] Updated weights for policy 1, policy_version 89254 (0.0007) [2023-12-27 06:16:48,258][20381] Updated weights for policy 1, policy_version 89264 (0.0008) [2023-12-27 06:16:48,316][20381] Updated weights for policy 1, policy_version 89274 (0.0006) [2023-12-27 06:16:48,808][20347] Updated weights for policy 0, policy_version 89567 (0.0010) [2023-12-27 06:16:48,869][20347] Updated weights for policy 0, policy_version 89577 (0.0010) [2023-12-27 06:16:48,922][20347] Updated weights for policy 0, policy_version 89587 (0.0010) [2023-12-27 06:16:49,044][20381] Updated weights for policy 1, policy_version 89284 (0.0007) [2023-12-27 06:16:49,104][20381] Updated weights for policy 1, policy_version 89294 (0.0008) [2023-12-27 06:16:49,162][20381] Updated weights for policy 1, policy_version 89304 (0.0008) [2023-12-27 06:16:49,552][20347] Updated weights for policy 0, policy_version 89597 (0.0009) [2023-12-27 06:16:49,610][20347] Updated weights for policy 0, policy_version 89607 (0.0010) [2023-12-27 06:16:49,664][20347] Updated weights for policy 0, policy_version 89617 (0.0010) [2023-12-27 06:16:49,871][20381] Updated weights for policy 1, policy_version 89314 (0.0007) [2023-12-27 06:16:49,937][20381] Updated weights for policy 1, policy_version 89324 (0.0008) [2023-12-27 06:16:49,998][20381] Updated weights for policy 1, policy_version 89334 (0.0005) [2023-12-27 06:16:50,053][20381] Updated weights for policy 1, policy_version 89344 (0.0007) [2023-12-27 06:16:50,342][20347] Updated weights for policy 0, policy_version 89627 (0.0009) [2023-12-27 06:16:50,403][20347] Updated weights for policy 0, policy_version 89637 (0.0007) [2023-12-27 06:16:50,462][20347] Updated weights for policy 0, policy_version 89647 (0.0009) [2023-12-27 06:16:50,751][20381] Updated weights for policy 1, policy_version 89354 (0.0009) [2023-12-27 06:16:50,812][20381] Updated weights for policy 1, policy_version 89364 (0.0008) [2023-12-27 06:16:50,872][20381] Updated weights for policy 1, policy_version 89374 (0.0007) [2023-12-27 06:16:51,181][20347] Updated weights for policy 0, policy_version 89657 (0.0007) [2023-12-27 06:16:51,238][20347] Updated weights for policy 0, policy_version 89667 (0.0007) [2023-12-27 06:16:51,295][20347] Updated weights for policy 0, policy_version 89677 (0.0008) [2023-12-27 06:16:51,351][20347] Updated weights for policy 0, policy_version 89687 (0.0008) [2023-12-27 06:16:51,564][20381] Updated weights for policy 1, policy_version 89384 (0.0008) [2023-12-27 06:16:51,615][20381] Updated weights for policy 1, policy_version 89394 (0.0008) [2023-12-27 06:16:51,676][20381] Updated weights for policy 1, policy_version 89404 (0.0006) [2023-12-27 06:16:52,027][20347] Updated weights for policy 0, policy_version 89697 (0.0007) [2023-12-27 06:16:52,092][20347] Updated weights for policy 0, policy_version 89707 (0.0007) [2023-12-27 06:16:52,152][20347] Updated weights for policy 0, policy_version 89717 (0.0007) [2023-12-27 06:16:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20771.6). Total num frames: 45875200. Throughput: 0: 10402.1, 1: 10266.3. Samples: 45846864. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:52,363][19170] Avg episode reward: [(0, '67.020'), (1, '1000.000')] [2023-12-27 06:16:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000089408_22896640.pth... [2023-12-27 06:16:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000089720_22978560.pth... [2023-12-27 06:16:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000088504_22667264.pth [2023-12-27 06:16:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000088224_22593536.pth [2023-12-27 06:16:52,438][20381] Updated weights for policy 1, policy_version 89414 (0.0008) [2023-12-27 06:16:52,509][20381] Updated weights for policy 1, policy_version 89424 (0.0007) [2023-12-27 06:16:52,575][20381] Updated weights for policy 1, policy_version 89434 (0.0005) [2023-12-27 06:16:52,846][20347] Updated weights for policy 0, policy_version 89727 (0.0009) [2023-12-27 06:16:52,907][20347] Updated weights for policy 0, policy_version 89737 (0.0011) [2023-12-27 06:16:52,960][20347] Updated weights for policy 0, policy_version 89747 (0.0010) [2023-12-27 06:16:53,212][20381] Updated weights for policy 1, policy_version 89444 (0.0006) [2023-12-27 06:16:53,271][20381] Updated weights for policy 1, policy_version 89454 (0.0008) [2023-12-27 06:16:53,337][20381] Updated weights for policy 1, policy_version 89464 (0.0008) [2023-12-27 06:16:53,643][20347] Updated weights for policy 0, policy_version 89757 (0.0009) [2023-12-27 06:16:53,688][20347] Updated weights for policy 0, policy_version 89767 (0.0007) [2023-12-27 06:16:53,741][20347] Updated weights for policy 0, policy_version 89777 (0.0007) [2023-12-27 06:16:54,048][20381] Updated weights for policy 1, policy_version 89474 (0.0007) [2023-12-27 06:16:54,096][20381] Updated weights for policy 1, policy_version 89484 (0.0007) [2023-12-27 06:16:54,155][20381] Updated weights for policy 1, policy_version 89494 (0.0009) [2023-12-27 06:16:54,212][20381] Updated weights for policy 1, policy_version 89504 (0.0008) [2023-12-27 06:16:54,452][20347] Updated weights for policy 0, policy_version 89787 (0.0007) [2023-12-27 06:16:54,509][20347] Updated weights for policy 0, policy_version 89797 (0.0005) [2023-12-27 06:16:54,558][20347] Updated weights for policy 0, policy_version 89807 (0.0007) [2023-12-27 06:16:54,945][20381] Updated weights for policy 1, policy_version 89514 (0.0008) [2023-12-27 06:16:55,010][20381] Updated weights for policy 1, policy_version 89524 (0.0008) [2023-12-27 06:16:55,069][20381] Updated weights for policy 1, policy_version 89534 (0.0008) [2023-12-27 06:16:55,161][20347] Updated weights for policy 0, policy_version 89817 (0.0008) [2023-12-27 06:16:55,209][20347] Updated weights for policy 0, policy_version 89827 (0.0006) [2023-12-27 06:16:55,254][20347] Updated weights for policy 0, policy_version 89837 (0.0005) [2023-12-27 06:16:55,302][20347] Updated weights for policy 0, policy_version 89847 (0.0005) [2023-12-27 06:16:55,812][20381] Updated weights for policy 1, policy_version 89544 (0.0009) [2023-12-27 06:16:55,858][20381] Updated weights for policy 1, policy_version 89554 (0.0007) [2023-12-27 06:16:55,873][20347] Updated weights for policy 0, policy_version 89857 (0.0005) [2023-12-27 06:16:55,909][20381] Updated weights for policy 1, policy_version 89564 (0.0007) [2023-12-27 06:16:55,922][20347] Updated weights for policy 0, policy_version 89867 (0.0005) [2023-12-27 06:16:55,976][20347] Updated weights for policy 0, policy_version 89877 (0.0007) [2023-12-27 06:16:56,595][20347] Updated weights for policy 0, policy_version 89887 (0.0007) [2023-12-27 06:16:56,648][20347] Updated weights for policy 0, policy_version 89897 (0.0007) [2023-12-27 06:16:56,665][20381] Updated weights for policy 1, policy_version 89574 (0.0007) [2023-12-27 06:16:56,704][20347] Updated weights for policy 0, policy_version 89907 (0.0007) [2023-12-27 06:16:56,712][20381] Updated weights for policy 1, policy_version 89584 (0.0009) [2023-12-27 06:16:56,766][20381] Updated weights for policy 1, policy_version 89594 (0.0007) [2023-12-27 06:16:57,352][20347] Updated weights for policy 0, policy_version 89917 (0.0006) [2023-12-27 06:16:57,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 45973504. Throughput: 0: 10437.0, 1: 10154.8. Samples: 45967788. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:16:57,364][19170] Avg episode reward: [(0, '83.130'), (1, '1000.000')] [2023-12-27 06:16:57,408][20347] Updated weights for policy 0, policy_version 89927 (0.0007) [2023-12-27 06:16:57,461][20381] Updated weights for policy 1, policy_version 89604 (0.0007) [2023-12-27 06:16:57,464][20347] Updated weights for policy 0, policy_version 89937 (0.0007) [2023-12-27 06:16:57,516][20381] Updated weights for policy 1, policy_version 89614 (0.0006) [2023-12-27 06:16:57,573][20381] Updated weights for policy 1, policy_version 89624 (0.0007) [2023-12-27 06:16:58,077][20347] Updated weights for policy 0, policy_version 89947 (0.0009) [2023-12-27 06:16:58,134][20347] Updated weights for policy 0, policy_version 89957 (0.0008) [2023-12-27 06:16:58,202][20347] Updated weights for policy 0, policy_version 89967 (0.0009) [2023-12-27 06:16:58,276][20381] Updated weights for policy 1, policy_version 89634 (0.0007) [2023-12-27 06:16:58,337][20381] Updated weights for policy 1, policy_version 89644 (0.0008) [2023-12-27 06:16:58,401][20381] Updated weights for policy 1, policy_version 89654 (0.0007) [2023-12-27 06:16:58,463][20381] Updated weights for policy 1, policy_version 89664 (0.0008) [2023-12-27 06:16:58,925][20347] Updated weights for policy 0, policy_version 89977 (0.0007) [2023-12-27 06:16:58,981][20347] Updated weights for policy 0, policy_version 89987 (0.0008) [2023-12-27 06:16:59,043][20347] Updated weights for policy 0, policy_version 89997 (0.0008) [2023-12-27 06:16:59,091][20347] Updated weights for policy 0, policy_version 90007 (0.0007) [2023-12-27 06:16:59,200][20381] Updated weights for policy 1, policy_version 89674 (0.0008) [2023-12-27 06:16:59,277][20381] Updated weights for policy 1, policy_version 89684 (0.0008) [2023-12-27 06:16:59,335][20381] Updated weights for policy 1, policy_version 89694 (0.0006) [2023-12-27 06:16:59,730][20347] Updated weights for policy 0, policy_version 90017 (0.0006) [2023-12-27 06:16:59,780][20347] Updated weights for policy 0, policy_version 90027 (0.0007) [2023-12-27 06:16:59,840][20347] Updated weights for policy 0, policy_version 90037 (0.0010) [2023-12-27 06:16:59,963][20381] Updated weights for policy 1, policy_version 89704 (0.0007) [2023-12-27 06:17:00,016][20381] Updated weights for policy 1, policy_version 89714 (0.0009) [2023-12-27 06:17:00,071][20381] Updated weights for policy 1, policy_version 89724 (0.0011) [2023-12-27 06:17:00,480][20347] Updated weights for policy 0, policy_version 90047 (0.0007) [2023-12-27 06:17:00,542][20347] Updated weights for policy 0, policy_version 90057 (0.0007) [2023-12-27 06:17:00,587][20347] Updated weights for policy 0, policy_version 90067 (0.0007) [2023-12-27 06:17:00,741][20381] Updated weights for policy 1, policy_version 89734 (0.0009) [2023-12-27 06:17:00,798][20381] Updated weights for policy 1, policy_version 89744 (0.0010) [2023-12-27 06:17:00,859][20381] Updated weights for policy 1, policy_version 89754 (0.0008) [2023-12-27 06:17:01,218][20347] Updated weights for policy 0, policy_version 90077 (0.0008) [2023-12-27 06:17:01,275][20347] Updated weights for policy 0, policy_version 90087 (0.0007) [2023-12-27 06:17:01,332][20347] Updated weights for policy 0, policy_version 90097 (0.0007) [2023-12-27 06:17:01,495][20381] Updated weights for policy 1, policy_version 89764 (0.0008) [2023-12-27 06:17:01,554][20381] Updated weights for policy 1, policy_version 89774 (0.0010) [2023-12-27 06:17:01,613][20381] Updated weights for policy 1, policy_version 89784 (0.0010) [2023-12-27 06:17:02,049][20347] Updated weights for policy 0, policy_version 90107 (0.0008) [2023-12-27 06:17:02,107][20347] Updated weights for policy 0, policy_version 90117 (0.0007) [2023-12-27 06:17:02,160][20347] Updated weights for policy 0, policy_version 90127 (0.0007) [2023-12-27 06:17:02,272][20381] Updated weights for policy 1, policy_version 89794 (0.0011) [2023-12-27 06:17:02,330][20381] Updated weights for policy 1, policy_version 89804 (0.0011) [2023-12-27 06:17:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 46080000. Throughput: 0: 10453.7, 1: 10119.3. Samples: 46092156. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 06:17:02,363][19170] Avg episode reward: [(0, '161.180'), (1, '1000.000')] [2023-12-27 06:17:02,385][20381] Updated weights for policy 1, policy_version 89814 (0.0012) [2023-12-27 06:17:02,433][20381] Updated weights for policy 1, policy_version 89824 (0.0010) [2023-12-27 06:17:02,851][20347] Updated weights for policy 0, policy_version 90137 (0.0007) [2023-12-27 06:17:02,898][20347] Updated weights for policy 0, policy_version 90147 (0.0007) [2023-12-27 06:17:02,948][20347] Updated weights for policy 0, policy_version 90157 (0.0007) [2023-12-27 06:17:02,997][20347] Updated weights for policy 0, policy_version 90167 (0.0007) [2023-12-27 06:17:03,078][20381] Updated weights for policy 1, policy_version 89834 (0.0005) [2023-12-27 06:17:03,134][20381] Updated weights for policy 1, policy_version 89844 (0.0006) [2023-12-27 06:17:03,193][20381] Updated weights for policy 1, policy_version 89854 (0.0007) [2023-12-27 06:17:03,667][20347] Updated weights for policy 0, policy_version 90177 (0.0009) [2023-12-27 06:17:03,711][20347] Updated weights for policy 0, policy_version 90187 (0.0008) [2023-12-27 06:17:03,754][20347] Updated weights for policy 0, policy_version 90197 (0.0006) [2023-12-27 06:17:03,761][20381] Updated weights for policy 1, policy_version 89864 (0.0006) [2023-12-27 06:17:03,820][20381] Updated weights for policy 1, policy_version 89874 (0.0008) [2023-12-27 06:17:03,879][20381] Updated weights for policy 1, policy_version 89884 (0.0008) [2023-12-27 06:17:04,445][20347] Updated weights for policy 0, policy_version 90207 (0.0006) [2023-12-27 06:17:04,489][20381] Updated weights for policy 1, policy_version 89894 (0.0009) [2023-12-27 06:17:04,506][20347] Updated weights for policy 0, policy_version 90217 (0.0007) [2023-12-27 06:17:04,548][20381] Updated weights for policy 1, policy_version 89904 (0.0008) [2023-12-27 06:17:04,559][20347] Updated weights for policy 0, policy_version 90227 (0.0005) [2023-12-27 06:17:04,604][20381] Updated weights for policy 1, policy_version 89914 (0.0007) [2023-12-27 06:17:05,117][20347] Updated weights for policy 0, policy_version 90237 (0.0005) [2023-12-27 06:17:05,176][20347] Updated weights for policy 0, policy_version 90247 (0.0006) [2023-12-27 06:17:05,202][20381] Updated weights for policy 1, policy_version 89924 (0.0009) [2023-12-27 06:17:05,227][20347] Updated weights for policy 0, policy_version 90257 (0.0007) [2023-12-27 06:17:05,256][20381] Updated weights for policy 1, policy_version 89934 (0.0011) [2023-12-27 06:17:05,321][20381] Updated weights for policy 1, policy_version 89944 (0.0006) [2023-12-27 06:17:05,906][20347] Updated weights for policy 0, policy_version 90267 (0.0006) [2023-12-27 06:17:05,964][20347] Updated weights for policy 0, policy_version 90277 (0.0007) [2023-12-27 06:17:05,993][20381] Updated weights for policy 1, policy_version 89954 (0.0008) [2023-12-27 06:17:06,020][20347] Updated weights for policy 0, policy_version 90287 (0.0008) [2023-12-27 06:17:06,045][20381] Updated weights for policy 1, policy_version 89964 (0.0006) [2023-12-27 06:17:06,102][20381] Updated weights for policy 1, policy_version 89974 (0.0007) [2023-12-27 06:17:06,167][20381] Updated weights for policy 1, policy_version 89984 (0.0008) [2023-12-27 06:17:06,768][20347] Updated weights for policy 0, policy_version 90297 (0.0008) [2023-12-27 06:17:06,829][20347] Updated weights for policy 0, policy_version 90307 (0.0007) [2023-12-27 06:17:06,864][20381] Updated weights for policy 1, policy_version 89994 (0.0009) [2023-12-27 06:17:06,878][20347] Updated weights for policy 0, policy_version 90317 (0.0008) [2023-12-27 06:17:06,919][20381] Updated weights for policy 1, policy_version 90004 (0.0006) [2023-12-27 06:17:06,932][20347] Updated weights for policy 0, policy_version 90327 (0.0007) [2023-12-27 06:17:06,969][20381] Updated weights for policy 1, policy_version 90014 (0.0005) [2023-12-27 06:17:07,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 46186496. Throughput: 0: 10497.9, 1: 10189.8. Samples: 46158016. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:07,364][19170] Avg episode reward: [(0, '92.170'), (1, '1000.000')] [2023-12-27 06:17:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000090016_23052288.pth... [2023-12-27 06:17:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000090328_23134208.pth... [2023-12-27 06:17:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000089112_22822912.pth [2023-12-27 06:17:07,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000088832_22749184.pth [2023-12-27 06:17:07,537][20381] Updated weights for policy 1, policy_version 90024 (0.0007) [2023-12-27 06:17:07,585][20381] Updated weights for policy 1, policy_version 90034 (0.0008) [2023-12-27 06:17:07,637][20381] Updated weights for policy 1, policy_version 90044 (0.0007) [2023-12-27 06:17:07,677][20347] Updated weights for policy 0, policy_version 90337 (0.0007) [2023-12-27 06:17:07,738][20347] Updated weights for policy 0, policy_version 90347 (0.0007) [2023-12-27 06:17:07,794][20347] Updated weights for policy 0, policy_version 90357 (0.0007) [2023-12-27 06:17:08,322][20381] Updated weights for policy 1, policy_version 90054 (0.0009) [2023-12-27 06:17:08,387][20381] Updated weights for policy 1, policy_version 90064 (0.0008) [2023-12-27 06:17:08,441][20381] Updated weights for policy 1, policy_version 90074 (0.0011) [2023-12-27 06:17:08,463][20347] Updated weights for policy 0, policy_version 90367 (0.0007) [2023-12-27 06:17:08,512][20347] Updated weights for policy 0, policy_version 90377 (0.0007) [2023-12-27 06:17:08,560][20347] Updated weights for policy 0, policy_version 90387 (0.0007) [2023-12-27 06:17:09,041][20381] Updated weights for policy 1, policy_version 90084 (0.0005) [2023-12-27 06:17:09,096][20381] Updated weights for policy 1, policy_version 90094 (0.0007) [2023-12-27 06:17:09,151][20381] Updated weights for policy 1, policy_version 90104 (0.0008) [2023-12-27 06:17:09,316][20347] Updated weights for policy 0, policy_version 90397 (0.0007) [2023-12-27 06:17:09,382][20347] Updated weights for policy 0, policy_version 90407 (0.0009) [2023-12-27 06:17:09,448][20347] Updated weights for policy 0, policy_version 90417 (0.0008) [2023-12-27 06:17:09,803][20381] Updated weights for policy 1, policy_version 90114 (0.0006) [2023-12-27 06:17:09,866][20381] Updated weights for policy 1, policy_version 90124 (0.0008) [2023-12-27 06:17:09,922][20381] Updated weights for policy 1, policy_version 90134 (0.0008) [2023-12-27 06:17:09,985][20381] Updated weights for policy 1, policy_version 90144 (0.0006) [2023-12-27 06:17:10,170][20347] Updated weights for policy 0, policy_version 90427 (0.0008) [2023-12-27 06:17:10,233][20347] Updated weights for policy 0, policy_version 90437 (0.0008) [2023-12-27 06:17:10,289][20347] Updated weights for policy 0, policy_version 90447 (0.0008) [2023-12-27 06:17:10,624][20381] Updated weights for policy 1, policy_version 90154 (0.0007) [2023-12-27 06:17:10,680][20381] Updated weights for policy 1, policy_version 90164 (0.0008) [2023-12-27 06:17:10,743][20381] Updated weights for policy 1, policy_version 90174 (0.0008) [2023-12-27 06:17:10,977][20347] Updated weights for policy 0, policy_version 90457 (0.0007) [2023-12-27 06:17:11,039][20347] Updated weights for policy 0, policy_version 90467 (0.0007) [2023-12-27 06:17:11,095][20347] Updated weights for policy 0, policy_version 90477 (0.0007) [2023-12-27 06:17:11,165][20347] Updated weights for policy 0, policy_version 90487 (0.0007) [2023-12-27 06:17:11,456][20381] Updated weights for policy 1, policy_version 90184 (0.0008) [2023-12-27 06:17:11,515][20381] Updated weights for policy 1, policy_version 90194 (0.0008) [2023-12-27 06:17:11,574][20381] Updated weights for policy 1, policy_version 90204 (0.0008) [2023-12-27 06:17:11,901][20347] Updated weights for policy 0, policy_version 90497 (0.0008) [2023-12-27 06:17:11,958][20347] Updated weights for policy 0, policy_version 90507 (0.0007) [2023-12-27 06:17:12,018][20347] Updated weights for policy 0, policy_version 90517 (0.0007) [2023-12-27 06:17:12,327][20381] Updated weights for policy 1, policy_version 90214 (0.0008) [2023-12-27 06:17:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 46284800. Throughput: 0: 10406.6, 1: 10252.2. Samples: 46279872. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:12,363][19170] Avg episode reward: [(0, '200.290'), (1, '1000.000')] [2023-12-27 06:17:12,382][20381] Updated weights for policy 1, policy_version 90224 (0.0006) [2023-12-27 06:17:12,449][20381] Updated weights for policy 1, policy_version 90234 (0.0008) [2023-12-27 06:17:12,712][20347] Updated weights for policy 0, policy_version 90527 (0.0008) [2023-12-27 06:17:12,768][20347] Updated weights for policy 0, policy_version 90537 (0.0005) [2023-12-27 06:17:12,836][20347] Updated weights for policy 0, policy_version 90547 (0.0006) [2023-12-27 06:17:13,027][20381] Updated weights for policy 1, policy_version 90244 (0.0008) [2023-12-27 06:17:13,077][20381] Updated weights for policy 1, policy_version 90254 (0.0008) [2023-12-27 06:17:13,125][20381] Updated weights for policy 1, policy_version 90264 (0.0007) [2023-12-27 06:17:13,413][20347] Updated weights for policy 0, policy_version 90557 (0.0005) [2023-12-27 06:17:13,471][20347] Updated weights for policy 0, policy_version 90567 (0.0006) [2023-12-27 06:17:13,527][20347] Updated weights for policy 0, policy_version 90577 (0.0008) [2023-12-27 06:17:13,758][20381] Updated weights for policy 1, policy_version 90274 (0.0007) [2023-12-27 06:17:13,822][20381] Updated weights for policy 1, policy_version 90284 (0.0009) [2023-12-27 06:17:13,878][20381] Updated weights for policy 1, policy_version 90294 (0.0007) [2023-12-27 06:17:13,936][20381] Updated weights for policy 1, policy_version 90304 (0.0006) [2023-12-27 06:17:14,107][20347] Updated weights for policy 0, policy_version 90587 (0.0005) [2023-12-27 06:17:14,158][20347] Updated weights for policy 0, policy_version 90597 (0.0007) [2023-12-27 06:17:14,204][20347] Updated weights for policy 0, policy_version 90607 (0.0006) [2023-12-27 06:17:14,566][20381] Updated weights for policy 1, policy_version 90314 (0.0005) [2023-12-27 06:17:14,617][20381] Updated weights for policy 1, policy_version 90324 (0.0006) [2023-12-27 06:17:14,666][20381] Updated weights for policy 1, policy_version 90334 (0.0007) [2023-12-27 06:17:14,851][20347] Updated weights for policy 0, policy_version 90617 (0.0006) [2023-12-27 06:17:14,912][20347] Updated weights for policy 0, policy_version 90627 (0.0010) [2023-12-27 06:17:14,961][20347] Updated weights for policy 0, policy_version 90637 (0.0010) [2023-12-27 06:17:15,011][20347] Updated weights for policy 0, policy_version 90647 (0.0010) [2023-12-27 06:17:15,371][20381] Updated weights for policy 1, policy_version 90344 (0.0007) [2023-12-27 06:17:15,421][20381] Updated weights for policy 1, policy_version 90354 (0.0005) [2023-12-27 06:17:15,470][20381] Updated weights for policy 1, policy_version 90364 (0.0006) [2023-12-27 06:17:15,711][20347] Updated weights for policy 0, policy_version 90657 (0.0008) [2023-12-27 06:17:15,771][20347] Updated weights for policy 0, policy_version 90667 (0.0007) [2023-12-27 06:17:15,828][20347] Updated weights for policy 0, policy_version 90677 (0.0009) [2023-12-27 06:17:16,067][20381] Updated weights for policy 1, policy_version 90374 (0.0007) [2023-12-27 06:17:16,131][20381] Updated weights for policy 1, policy_version 90384 (0.0008) [2023-12-27 06:17:16,192][20381] Updated weights for policy 1, policy_version 90394 (0.0007) [2023-12-27 06:17:16,409][20347] Updated weights for policy 0, policy_version 90687 (0.0009) [2023-12-27 06:17:16,457][20347] Updated weights for policy 0, policy_version 90697 (0.0010) [2023-12-27 06:17:16,502][20347] Updated weights for policy 0, policy_version 90707 (0.0010) [2023-12-27 06:17:16,888][20381] Updated weights for policy 1, policy_version 90404 (0.0008) [2023-12-27 06:17:16,947][20381] Updated weights for policy 1, policy_version 90414 (0.0007) [2023-12-27 06:17:16,997][20381] Updated weights for policy 1, policy_version 90424 (0.0010) [2023-12-27 06:17:17,121][20347] Updated weights for policy 0, policy_version 90717 (0.0007) [2023-12-27 06:17:17,170][20347] Updated weights for policy 0, policy_version 90727 (0.0005) [2023-12-27 06:17:17,227][20347] Updated weights for policy 0, policy_version 90737 (0.0005) [2023-12-27 06:17:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 46399488. Throughput: 0: 10418.0, 1: 10248.5. Samples: 46406324. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:17,363][19170] Avg episode reward: [(0, '136.100'), (1, '1000.000')] [2023-12-27 06:17:17,653][20381] Updated weights for policy 1, policy_version 90434 (0.0009) [2023-12-27 06:17:17,715][20381] Updated weights for policy 1, policy_version 90444 (0.0005) [2023-12-27 06:17:17,771][20381] Updated weights for policy 1, policy_version 90454 (0.0005) [2023-12-27 06:17:17,801][20347] Updated weights for policy 0, policy_version 90747 (0.0007) [2023-12-27 06:17:17,823][20381] Updated weights for policy 1, policy_version 90464 (0.0007) [2023-12-27 06:17:17,857][20347] Updated weights for policy 0, policy_version 90757 (0.0005) [2023-12-27 06:17:17,914][20347] Updated weights for policy 0, policy_version 90767 (0.0010) [2023-12-27 06:17:18,390][20381] Updated weights for policy 1, policy_version 90474 (0.0011) [2023-12-27 06:17:18,443][20381] Updated weights for policy 1, policy_version 90484 (0.0010) [2023-12-27 06:17:18,498][20381] Updated weights for policy 1, policy_version 90494 (0.0010) [2023-12-27 06:17:18,590][20347] Updated weights for policy 0, policy_version 90777 (0.0010) [2023-12-27 06:17:18,657][20347] Updated weights for policy 0, policy_version 90787 (0.0008) [2023-12-27 06:17:18,721][20347] Updated weights for policy 0, policy_version 90797 (0.0008) [2023-12-27 06:17:18,779][20347] Updated weights for policy 0, policy_version 90807 (0.0006) [2023-12-27 06:17:19,191][20381] Updated weights for policy 1, policy_version 90504 (0.0011) [2023-12-27 06:17:19,257][20381] Updated weights for policy 1, policy_version 90514 (0.0010) [2023-12-27 06:17:19,312][20381] Updated weights for policy 1, policy_version 90524 (0.0010) [2023-12-27 06:17:19,427][20347] Updated weights for policy 0, policy_version 90817 (0.0007) [2023-12-27 06:17:19,490][20347] Updated weights for policy 0, policy_version 90827 (0.0008) [2023-12-27 06:17:19,546][20347] Updated weights for policy 0, policy_version 90837 (0.0007) [2023-12-27 06:17:19,981][20381] Updated weights for policy 1, policy_version 90534 (0.0007) [2023-12-27 06:17:20,041][20381] Updated weights for policy 1, policy_version 90544 (0.0007) [2023-12-27 06:17:20,105][20381] Updated weights for policy 1, policy_version 90554 (0.0007) [2023-12-27 06:17:20,232][20347] Updated weights for policy 0, policy_version 90847 (0.0006) [2023-12-27 06:17:20,292][20347] Updated weights for policy 0, policy_version 90857 (0.0006) [2023-12-27 06:17:20,354][20347] Updated weights for policy 0, policy_version 90867 (0.0007) [2023-12-27 06:17:20,755][20381] Updated weights for policy 1, policy_version 90564 (0.0006) [2023-12-27 06:17:20,816][20381] Updated weights for policy 1, policy_version 90574 (0.0006) [2023-12-27 06:17:20,874][20381] Updated weights for policy 1, policy_version 90584 (0.0006) [2023-12-27 06:17:21,044][20347] Updated weights for policy 0, policy_version 90877 (0.0009) [2023-12-27 06:17:21,100][20347] Updated weights for policy 0, policy_version 90887 (0.0008) [2023-12-27 06:17:21,160][20347] Updated weights for policy 0, policy_version 90897 (0.0007) [2023-12-27 06:17:21,505][20381] Updated weights for policy 1, policy_version 90594 (0.0005) [2023-12-27 06:17:21,565][20381] Updated weights for policy 1, policy_version 90604 (0.0008) [2023-12-27 06:17:21,627][20381] Updated weights for policy 1, policy_version 90614 (0.0006) [2023-12-27 06:17:21,694][20381] Updated weights for policy 1, policy_version 90624 (0.0008) [2023-12-27 06:17:21,899][20347] Updated weights for policy 0, policy_version 90907 (0.0008) [2023-12-27 06:17:21,958][20347] Updated weights for policy 0, policy_version 90917 (0.0007) [2023-12-27 06:17:22,025][20347] Updated weights for policy 0, policy_version 90927 (0.0008) [2023-12-27 06:17:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 46497792. Throughput: 0: 10438.1, 1: 10237.8. Samples: 46470500. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:22,363][19170] Avg episode reward: [(0, '465.820'), (1, '1000.000')] [2023-12-27 06:17:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000090936_23289856.pth... [2023-12-27 06:17:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000089720_22978560.pth [2023-12-27 06:17:22,396][20381] Updated weights for policy 1, policy_version 90634 (0.0008) [2023-12-27 06:17:22,446][20381] Updated weights for policy 1, policy_version 90644 (0.0008) [2023-12-27 06:17:22,495][20381] Updated weights for policy 1, policy_version 90654 (0.0007) [2023-12-27 06:17:22,504][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000090656_23216128.pth... [2023-12-27 06:17:22,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000089408_22896640.pth [2023-12-27 06:17:22,744][20347] Updated weights for policy 0, policy_version 90937 (0.0007) [2023-12-27 06:17:22,804][20347] Updated weights for policy 0, policy_version 90947 (0.0008) [2023-12-27 06:17:22,862][20347] Updated weights for policy 0, policy_version 90957 (0.0008) [2023-12-27 06:17:22,920][20347] Updated weights for policy 0, policy_version 90967 (0.0008) [2023-12-27 06:17:23,171][20381] Updated weights for policy 1, policy_version 90664 (0.0007) [2023-12-27 06:17:23,220][20381] Updated weights for policy 1, policy_version 90674 (0.0008) [2023-12-27 06:17:23,276][20381] Updated weights for policy 1, policy_version 90684 (0.0008) [2023-12-27 06:17:23,509][20347] Updated weights for policy 0, policy_version 90977 (0.0007) [2023-12-27 06:17:23,565][20347] Updated weights for policy 0, policy_version 90987 (0.0010) [2023-12-27 06:17:23,625][20347] Updated weights for policy 0, policy_version 90997 (0.0007) [2023-12-27 06:17:23,986][20381] Updated weights for policy 1, policy_version 90694 (0.0009) [2023-12-27 06:17:24,042][20381] Updated weights for policy 1, policy_version 90704 (0.0007) [2023-12-27 06:17:24,100][20381] Updated weights for policy 1, policy_version 90714 (0.0007) [2023-12-27 06:17:24,292][20347] Updated weights for policy 0, policy_version 91007 (0.0006) [2023-12-27 06:17:24,350][20347] Updated weights for policy 0, policy_version 91017 (0.0008) [2023-12-27 06:17:24,407][20347] Updated weights for policy 0, policy_version 91027 (0.0008) [2023-12-27 06:17:24,828][20381] Updated weights for policy 1, policy_version 90724 (0.0008) [2023-12-27 06:17:24,893][20381] Updated weights for policy 1, policy_version 90734 (0.0009) [2023-12-27 06:17:24,952][20381] Updated weights for policy 1, policy_version 90744 (0.0007) [2023-12-27 06:17:25,000][20347] Updated weights for policy 0, policy_version 91037 (0.0007) [2023-12-27 06:17:25,069][20347] Updated weights for policy 0, policy_version 91047 (0.0005) [2023-12-27 06:17:25,135][20347] Updated weights for policy 0, policy_version 91057 (0.0006) [2023-12-27 06:17:25,579][20381] Updated weights for policy 1, policy_version 90754 (0.0006) [2023-12-27 06:17:25,634][20381] Updated weights for policy 1, policy_version 90764 (0.0010) [2023-12-27 06:17:25,688][20347] Updated weights for policy 0, policy_version 91067 (0.0005) [2023-12-27 06:17:25,692][20381] Updated weights for policy 1, policy_version 90774 (0.0011) [2023-12-27 06:17:25,746][20347] Updated weights for policy 0, policy_version 91077 (0.0009) [2023-12-27 06:17:25,751][20381] Updated weights for policy 1, policy_version 90784 (0.0011) [2023-12-27 06:17:25,793][20347] Updated weights for policy 0, policy_version 91087 (0.0010) [2023-12-27 06:17:26,359][20381] Updated weights for policy 1, policy_version 90794 (0.0005) [2023-12-27 06:17:26,415][20381] Updated weights for policy 1, policy_version 90804 (0.0005) [2023-12-27 06:17:26,436][20347] Updated weights for policy 0, policy_version 91097 (0.0010) [2023-12-27 06:17:26,471][20381] Updated weights for policy 1, policy_version 90814 (0.0006) [2023-12-27 06:17:26,491][20347] Updated weights for policy 0, policy_version 91107 (0.0006) [2023-12-27 06:17:26,537][20347] Updated weights for policy 0, policy_version 91117 (0.0005) [2023-12-27 06:17:26,582][20347] Updated weights for policy 0, policy_version 91127 (0.0005) [2023-12-27 06:17:27,027][20381] Updated weights for policy 1, policy_version 90824 (0.0009) [2023-12-27 06:17:27,078][20381] Updated weights for policy 1, policy_version 90834 (0.0010) [2023-12-27 06:17:27,138][20381] Updated weights for policy 1, policy_version 90844 (0.0010) [2023-12-27 06:17:27,222][20347] Updated weights for policy 0, policy_version 91137 (0.0010) [2023-12-27 06:17:27,280][20347] Updated weights for policy 0, policy_version 91147 (0.0007) [2023-12-27 06:17:27,343][20347] Updated weights for policy 0, policy_version 91157 (0.0007) [2023-12-27 06:17:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 46612480. Throughput: 0: 10452.5, 1: 10231.2. Samples: 46594844. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:27,363][19170] Avg episode reward: [(0, '867.260'), (1, '1000.000')] [2023-12-27 06:17:27,706][20381] Updated weights for policy 1, policy_version 90854 (0.0010) [2023-12-27 06:17:27,760][20381] Updated weights for policy 1, policy_version 90864 (0.0010) [2023-12-27 06:17:27,823][20381] Updated weights for policy 1, policy_version 90874 (0.0010) [2023-12-27 06:17:28,018][20347] Updated weights for policy 0, policy_version 91167 (0.0008) [2023-12-27 06:17:28,064][20347] Updated weights for policy 0, policy_version 91177 (0.0007) [2023-12-27 06:17:28,111][20347] Updated weights for policy 0, policy_version 91187 (0.0007) [2023-12-27 06:17:28,495][20381] Updated weights for policy 1, policy_version 90884 (0.0010) [2023-12-27 06:17:28,547][20381] Updated weights for policy 1, policy_version 90894 (0.0010) [2023-12-27 06:17:28,601][20381] Updated weights for policy 1, policy_version 90904 (0.0010) [2023-12-27 06:17:28,815][20347] Updated weights for policy 0, policy_version 91197 (0.0008) [2023-12-27 06:17:28,868][20347] Updated weights for policy 0, policy_version 91207 (0.0007) [2023-12-27 06:17:28,916][20347] Updated weights for policy 0, policy_version 91217 (0.0007) [2023-12-27 06:17:29,281][20381] Updated weights for policy 1, policy_version 90914 (0.0010) [2023-12-27 06:17:29,342][20381] Updated weights for policy 1, policy_version 90924 (0.0008) [2023-12-27 06:17:29,399][20381] Updated weights for policy 1, policy_version 90934 (0.0007) [2023-12-27 06:17:29,463][20381] Updated weights for policy 1, policy_version 90944 (0.0005) [2023-12-27 06:17:29,568][20347] Updated weights for policy 0, policy_version 91227 (0.0007) [2023-12-27 06:17:29,613][20347] Updated weights for policy 0, policy_version 91237 (0.0010) [2023-12-27 06:17:29,664][20347] Updated weights for policy 0, policy_version 91247 (0.0010) [2023-12-27 06:17:30,088][20381] Updated weights for policy 1, policy_version 90954 (0.0005) [2023-12-27 06:17:30,132][20381] Updated weights for policy 1, policy_version 90964 (0.0005) [2023-12-27 06:17:30,179][20381] Updated weights for policy 1, policy_version 90974 (0.0005) [2023-12-27 06:17:30,346][20347] Updated weights for policy 0, policy_version 91257 (0.0010) [2023-12-27 06:17:30,399][20347] Updated weights for policy 0, policy_version 91267 (0.0008) [2023-12-27 06:17:30,447][20347] Updated weights for policy 0, policy_version 91277 (0.0007) [2023-12-27 06:17:30,512][20347] Updated weights for policy 0, policy_version 91287 (0.0006) [2023-12-27 06:17:30,727][20381] Updated weights for policy 1, policy_version 90984 (0.0007) [2023-12-27 06:17:30,773][20381] Updated weights for policy 1, policy_version 90994 (0.0005) [2023-12-27 06:17:30,823][20381] Updated weights for policy 1, policy_version 91004 (0.0006) [2023-12-27 06:17:31,206][20347] Updated weights for policy 0, policy_version 91297 (0.0008) [2023-12-27 06:17:31,270][20347] Updated weights for policy 0, policy_version 91307 (0.0008) [2023-12-27 06:17:31,329][20347] Updated weights for policy 0, policy_version 91317 (0.0008) [2023-12-27 06:17:31,508][20381] Updated weights for policy 1, policy_version 91014 (0.0008) [2023-12-27 06:17:31,566][20381] Updated weights for policy 1, policy_version 91024 (0.0008) [2023-12-27 06:17:31,629][20381] Updated weights for policy 1, policy_version 91034 (0.0007) [2023-12-27 06:17:32,021][20347] Updated weights for policy 0, policy_version 91327 (0.0008) [2023-12-27 06:17:32,083][20347] Updated weights for policy 0, policy_version 91337 (0.0010) [2023-12-27 06:17:32,134][20347] Updated weights for policy 0, policy_version 91347 (0.0010) [2023-12-27 06:17:32,192][20381] Updated weights for policy 1, policy_version 91044 (0.0006) [2023-12-27 06:17:32,243][20381] Updated weights for policy 1, policy_version 91054 (0.0005) [2023-12-27 06:17:32,303][20381] Updated weights for policy 1, policy_version 91064 (0.0007) [2023-12-27 06:17:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 46718976. Throughput: 0: 10423.9, 1: 10416.1. Samples: 46723932. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:32,802][20347] Updated weights for policy 0, policy_version 91357 (0.0006) [2023-12-27 06:17:32,857][20347] Updated weights for policy 0, policy_version 91367 (0.0010) [2023-12-27 06:17:32,917][20347] Updated weights for policy 0, policy_version 91377 (0.0010) [2023-12-27 06:17:32,976][20381] Updated weights for policy 1, policy_version 91074 (0.0008) [2023-12-27 06:17:33,026][20381] Updated weights for policy 1, policy_version 91084 (0.0010) [2023-12-27 06:17:33,092][20381] Updated weights for policy 1, policy_version 91094 (0.0010) [2023-12-27 06:17:33,149][20381] Updated weights for policy 1, policy_version 91104 (0.0010) [2023-12-27 06:17:33,561][20347] Updated weights for policy 0, policy_version 91387 (0.0009) [2023-12-27 06:17:33,608][20347] Updated weights for policy 0, policy_version 91397 (0.0005) [2023-12-27 06:17:33,654][20347] Updated weights for policy 0, policy_version 91407 (0.0008) [2023-12-27 06:17:33,810][20381] Updated weights for policy 1, policy_version 91114 (0.0010) [2023-12-27 06:17:33,864][20381] Updated weights for policy 1, policy_version 91124 (0.0010) [2023-12-27 06:17:33,909][20381] Updated weights for policy 1, policy_version 91134 (0.0010) [2023-12-27 06:17:34,260][20347] Updated weights for policy 0, policy_version 91417 (0.0010) [2023-12-27 06:17:34,319][20347] Updated weights for policy 0, policy_version 91427 (0.0010) [2023-12-27 06:17:34,375][20347] Updated weights for policy 0, policy_version 91437 (0.0006) [2023-12-27 06:17:34,433][20347] Updated weights for policy 0, policy_version 91447 (0.0005) [2023-12-27 06:17:34,625][20381] Updated weights for policy 1, policy_version 91144 (0.0010) [2023-12-27 06:17:34,684][20381] Updated weights for policy 1, policy_version 91154 (0.0010) [2023-12-27 06:17:34,746][20381] Updated weights for policy 1, policy_version 91164 (0.0011) [2023-12-27 06:17:35,058][20347] Updated weights for policy 0, policy_version 91457 (0.0008) [2023-12-27 06:17:35,115][20347] Updated weights for policy 0, policy_version 91467 (0.0009) [2023-12-27 06:17:35,171][20347] Updated weights for policy 0, policy_version 91477 (0.0007) [2023-12-27 06:17:35,428][20381] Updated weights for policy 1, policy_version 91174 (0.0010) [2023-12-27 06:17:35,496][20381] Updated weights for policy 1, policy_version 91184 (0.0011) [2023-12-27 06:17:35,553][20381] Updated weights for policy 1, policy_version 91194 (0.0010) [2023-12-27 06:17:35,830][20347] Updated weights for policy 0, policy_version 91487 (0.0007) [2023-12-27 06:17:35,886][20347] Updated weights for policy 0, policy_version 91497 (0.0007) [2023-12-27 06:17:35,945][20347] Updated weights for policy 0, policy_version 91507 (0.0007) [2023-12-27 06:17:36,248][20381] Updated weights for policy 1, policy_version 91204 (0.0011) [2023-12-27 06:17:36,308][20381] Updated weights for policy 1, policy_version 91214 (0.0011) [2023-12-27 06:17:36,371][20381] Updated weights for policy 1, policy_version 91224 (0.0011) [2023-12-27 06:17:36,661][20347] Updated weights for policy 0, policy_version 91517 (0.0009) [2023-12-27 06:17:36,719][20347] Updated weights for policy 0, policy_version 91527 (0.0007) [2023-12-27 06:17:36,777][20347] Updated weights for policy 0, policy_version 91537 (0.0007) [2023-12-27 06:17:37,039][20381] Updated weights for policy 1, policy_version 91234 (0.0011) [2023-12-27 06:17:37,084][20381] Updated weights for policy 1, policy_version 91244 (0.0009) [2023-12-27 06:17:37,141][20381] Updated weights for policy 1, policy_version 91254 (0.0007) [2023-12-27 06:17:37,191][20381] Updated weights for policy 1, policy_version 91264 (0.0010) [2023-12-27 06:17:37,351][20347] Updated weights for policy 0, policy_version 91547 (0.0007) [2023-12-27 06:17:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 46817280. Throughput: 0: 10446.1, 1: 10444.3. Samples: 46786928. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000091264_23371776.pth... [2023-12-27 06:17:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000090016_23052288.pth [2023-12-27 06:17:37,416][20347] Updated weights for policy 0, policy_version 91557 (0.0007) [2023-12-27 06:17:37,468][20347] Updated weights for policy 0, policy_version 91567 (0.0007) [2023-12-27 06:17:37,512][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000091576_23453696.pth... [2023-12-27 06:17:37,515][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000090328_23134208.pth [2023-12-27 06:17:37,914][20381] Updated weights for policy 1, policy_version 91274 (0.0011) [2023-12-27 06:17:37,973][20381] Updated weights for policy 1, policy_version 91284 (0.0010) [2023-12-27 06:17:38,029][20381] Updated weights for policy 1, policy_version 91294 (0.0011) [2023-12-27 06:17:38,184][20347] Updated weights for policy 0, policy_version 91577 (0.0007) [2023-12-27 06:17:38,235][20347] Updated weights for policy 0, policy_version 91587 (0.0007) [2023-12-27 06:17:38,281][20347] Updated weights for policy 0, policy_version 91597 (0.0007) [2023-12-27 06:17:38,335][20347] Updated weights for policy 0, policy_version 91607 (0.0006) [2023-12-27 06:17:38,712][20381] Updated weights for policy 1, policy_version 91304 (0.0010) [2023-12-27 06:17:38,774][20381] Updated weights for policy 1, policy_version 91314 (0.0010) [2023-12-27 06:17:38,832][20381] Updated weights for policy 1, policy_version 91324 (0.0010) [2023-12-27 06:17:39,045][20347] Updated weights for policy 0, policy_version 91617 (0.0007) [2023-12-27 06:17:39,089][20347] Updated weights for policy 0, policy_version 91627 (0.0007) [2023-12-27 06:17:39,144][20347] Updated weights for policy 0, policy_version 91637 (0.0007) [2023-12-27 06:17:39,559][20381] Updated weights for policy 1, policy_version 91334 (0.0009) [2023-12-27 06:17:39,615][20381] Updated weights for policy 1, policy_version 91344 (0.0007) [2023-12-27 06:17:39,675][20381] Updated weights for policy 1, policy_version 91354 (0.0008) [2023-12-27 06:17:39,851][20347] Updated weights for policy 0, policy_version 91647 (0.0009) [2023-12-27 06:17:39,909][20347] Updated weights for policy 0, policy_version 91657 (0.0008) [2023-12-27 06:17:39,979][20347] Updated weights for policy 0, policy_version 91667 (0.0009) [2023-12-27 06:17:40,281][20381] Updated weights for policy 1, policy_version 91364 (0.0008) [2023-12-27 06:17:40,344][20381] Updated weights for policy 1, policy_version 91374 (0.0008) [2023-12-27 06:17:40,403][20381] Updated weights for policy 1, policy_version 91384 (0.0007) [2023-12-27 06:17:40,695][20347] Updated weights for policy 0, policy_version 91677 (0.0008) [2023-12-27 06:17:40,745][20347] Updated weights for policy 0, policy_version 91687 (0.0008) [2023-12-27 06:17:40,794][20347] Updated weights for policy 0, policy_version 91697 (0.0008) [2023-12-27 06:17:41,115][20381] Updated weights for policy 1, policy_version 91394 (0.0009) [2023-12-27 06:17:41,186][20381] Updated weights for policy 1, policy_version 91404 (0.0008) [2023-12-27 06:17:41,254][20381] Updated weights for policy 1, policy_version 91414 (0.0009) [2023-12-27 06:17:41,320][20381] Updated weights for policy 1, policy_version 91424 (0.0007) [2023-12-27 06:17:41,434][20347] Updated weights for policy 0, policy_version 91707 (0.0006) [2023-12-27 06:17:41,491][20347] Updated weights for policy 0, policy_version 91717 (0.0009) [2023-12-27 06:17:41,553][20347] Updated weights for policy 0, policy_version 91727 (0.0008) [2023-12-27 06:17:41,983][20381] Updated weights for policy 1, policy_version 91434 (0.0008) [2023-12-27 06:17:42,050][20381] Updated weights for policy 1, policy_version 91444 (0.0008) [2023-12-27 06:17:42,108][20381] Updated weights for policy 1, policy_version 91454 (0.0008) [2023-12-27 06:17:42,296][20347] Updated weights for policy 0, policy_version 91737 (0.0008) [2023-12-27 06:17:42,358][20347] Updated weights for policy 0, policy_version 91747 (0.0008) [2023-12-27 06:17:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 46915584. Throughput: 0: 10398.0, 1: 10501.7. Samples: 46908272. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:42,421][20347] Updated weights for policy 0, policy_version 91757 (0.0007) [2023-12-27 06:17:42,465][20347] Updated weights for policy 0, policy_version 91767 (0.0007) [2023-12-27 06:17:42,796][20381] Updated weights for policy 1, policy_version 91464 (0.0008) [2023-12-27 06:17:42,860][20381] Updated weights for policy 1, policy_version 91474 (0.0008) [2023-12-27 06:17:42,925][20381] Updated weights for policy 1, policy_version 91484 (0.0008) [2023-12-27 06:17:43,165][20347] Updated weights for policy 0, policy_version 91777 (0.0007) [2023-12-27 06:17:43,216][20347] Updated weights for policy 0, policy_version 91787 (0.0007) [2023-12-27 06:17:43,275][20347] Updated weights for policy 0, policy_version 91797 (0.0007) [2023-12-27 06:17:43,553][20381] Updated weights for policy 1, policy_version 91494 (0.0006) [2023-12-27 06:17:43,620][20381] Updated weights for policy 1, policy_version 91504 (0.0008) [2023-12-27 06:17:43,681][20381] Updated weights for policy 1, policy_version 91514 (0.0006) [2023-12-27 06:17:43,910][20347] Updated weights for policy 0, policy_version 91807 (0.0006) [2023-12-27 06:17:43,972][20347] Updated weights for policy 0, policy_version 91817 (0.0006) [2023-12-27 06:17:44,020][20347] Updated weights for policy 0, policy_version 91827 (0.0008) [2023-12-27 06:17:44,277][20381] Updated weights for policy 1, policy_version 91524 (0.0007) [2023-12-27 06:17:44,344][20381] Updated weights for policy 1, policy_version 91534 (0.0005) [2023-12-27 06:17:44,412][20381] Updated weights for policy 1, policy_version 91544 (0.0006) [2023-12-27 06:17:44,683][20347] Updated weights for policy 0, policy_version 91837 (0.0008) [2023-12-27 06:17:44,737][20347] Updated weights for policy 0, policy_version 91847 (0.0007) [2023-12-27 06:17:44,802][20347] Updated weights for policy 0, policy_version 91857 (0.0007) [2023-12-27 06:17:45,006][20381] Updated weights for policy 1, policy_version 91554 (0.0006) [2023-12-27 06:17:45,068][20381] Updated weights for policy 1, policy_version 91564 (0.0009) [2023-12-27 06:17:45,130][20381] Updated weights for policy 1, policy_version 91574 (0.0008) [2023-12-27 06:17:45,188][20381] Updated weights for policy 1, policy_version 91584 (0.0009) [2023-12-27 06:17:45,365][20347] Updated weights for policy 0, policy_version 91867 (0.0006) [2023-12-27 06:17:45,422][20347] Updated weights for policy 0, policy_version 91877 (0.0006) [2023-12-27 06:17:45,480][20347] Updated weights for policy 0, policy_version 91887 (0.0006) [2023-12-27 06:17:45,929][20381] Updated weights for policy 1, policy_version 91594 (0.0007) [2023-12-27 06:17:45,978][20381] Updated weights for policy 1, policy_version 91604 (0.0007) [2023-12-27 06:17:46,036][20381] Updated weights for policy 1, policy_version 91614 (0.0008) [2023-12-27 06:17:46,104][20347] Updated weights for policy 0, policy_version 91897 (0.0006) [2023-12-27 06:17:46,156][20347] Updated weights for policy 0, policy_version 91907 (0.0008) [2023-12-27 06:17:46,206][20347] Updated weights for policy 0, policy_version 91917 (0.0007) [2023-12-27 06:17:46,254][20347] Updated weights for policy 0, policy_version 91927 (0.0007) [2023-12-27 06:17:46,642][20381] Updated weights for policy 1, policy_version 91624 (0.0006) [2023-12-27 06:17:46,693][20381] Updated weights for policy 1, policy_version 91634 (0.0010) [2023-12-27 06:17:46,742][20381] Updated weights for policy 1, policy_version 91644 (0.0010) [2023-12-27 06:17:46,968][20347] Updated weights for policy 0, policy_version 91937 (0.0006) [2023-12-27 06:17:47,027][20347] Updated weights for policy 0, policy_version 91947 (0.0006) [2023-12-27 06:17:47,083][20347] Updated weights for policy 0, policy_version 91957 (0.0005) [2023-12-27 06:17:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 47022080. Throughput: 0: 10386.1, 1: 10531.3. Samples: 47033436. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:47,411][20381] Updated weights for policy 1, policy_version 91654 (0.0010) [2023-12-27 06:17:47,463][20381] Updated weights for policy 1, policy_version 91664 (0.0010) [2023-12-27 06:17:47,517][20381] Updated weights for policy 1, policy_version 91674 (0.0010) [2023-12-27 06:17:47,695][20347] Updated weights for policy 0, policy_version 91967 (0.0006) [2023-12-27 06:17:47,755][20347] Updated weights for policy 0, policy_version 91977 (0.0006) [2023-12-27 06:17:47,809][20347] Updated weights for policy 0, policy_version 91987 (0.0007) [2023-12-27 06:17:48,215][20381] Updated weights for policy 1, policy_version 91684 (0.0009) [2023-12-27 06:17:48,273][20381] Updated weights for policy 1, policy_version 91694 (0.0011) [2023-12-27 06:17:48,318][20381] Updated weights for policy 1, policy_version 91704 (0.0010) [2023-12-27 06:17:48,375][20347] Updated weights for policy 0, policy_version 91997 (0.0007) [2023-12-27 06:17:48,438][20347] Updated weights for policy 0, policy_version 92007 (0.0007) [2023-12-27 06:17:48,502][20347] Updated weights for policy 0, policy_version 92017 (0.0006) [2023-12-27 06:17:49,055][20381] Updated weights for policy 1, policy_version 91714 (0.0010) [2023-12-27 06:17:49,115][20381] Updated weights for policy 1, policy_version 91724 (0.0008) [2023-12-27 06:17:49,116][20347] Updated weights for policy 0, policy_version 92027 (0.0007) [2023-12-27 06:17:49,172][20381] Updated weights for policy 1, policy_version 91734 (0.0006) [2023-12-27 06:17:49,175][20347] Updated weights for policy 0, policy_version 92037 (0.0007) [2023-12-27 06:17:49,230][20347] Updated weights for policy 0, policy_version 92047 (0.0008) [2023-12-27 06:17:49,240][20381] Updated weights for policy 1, policy_version 91744 (0.0006) [2023-12-27 06:17:49,850][20381] Updated weights for policy 1, policy_version 91754 (0.0007) [2023-12-27 06:17:49,913][20381] Updated weights for policy 1, policy_version 91764 (0.0006) [2023-12-27 06:17:49,959][20347] Updated weights for policy 0, policy_version 92057 (0.0006) [2023-12-27 06:17:49,983][20381] Updated weights for policy 1, policy_version 91774 (0.0008) [2023-12-27 06:17:50,010][20347] Updated weights for policy 0, policy_version 92067 (0.0006) [2023-12-27 06:17:50,067][20347] Updated weights for policy 0, policy_version 92077 (0.0006) [2023-12-27 06:17:50,123][20347] Updated weights for policy 0, policy_version 92087 (0.0007) [2023-12-27 06:17:50,619][20381] Updated weights for policy 1, policy_version 91784 (0.0008) [2023-12-27 06:17:50,681][20381] Updated weights for policy 1, policy_version 91794 (0.0008) [2023-12-27 06:17:50,744][20381] Updated weights for policy 1, policy_version 91804 (0.0008) [2023-12-27 06:17:50,761][20347] Updated weights for policy 0, policy_version 92097 (0.0008) [2023-12-27 06:17:50,824][20347] Updated weights for policy 0, policy_version 92107 (0.0008) [2023-12-27 06:17:50,882][20347] Updated weights for policy 0, policy_version 92117 (0.0007) [2023-12-27 06:17:51,446][20381] Updated weights for policy 1, policy_version 91814 (0.0006) [2023-12-27 06:17:51,496][20381] Updated weights for policy 1, policy_version 91824 (0.0005) [2023-12-27 06:17:51,550][20381] Updated weights for policy 1, policy_version 91834 (0.0006) [2023-12-27 06:17:51,560][20347] Updated weights for policy 0, policy_version 92127 (0.0006) [2023-12-27 06:17:51,628][20347] Updated weights for policy 0, policy_version 92137 (0.0007) [2023-12-27 06:17:51,687][20347] Updated weights for policy 0, policy_version 92147 (0.0006) [2023-12-27 06:17:52,247][20381] Updated weights for policy 1, policy_version 91844 (0.0007) [2023-12-27 06:17:52,315][20381] Updated weights for policy 1, policy_version 91854 (0.0008) [2023-12-27 06:17:52,316][20347] Updated weights for policy 0, policy_version 92157 (0.0006) [2023-12-27 06:17:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 47120384. Throughput: 0: 10393.9, 1: 10452.1. Samples: 47096084. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:52,373][20347] Updated weights for policy 0, policy_version 92167 (0.0007) [2023-12-27 06:17:52,382][20381] Updated weights for policy 1, policy_version 91864 (0.0008) [2023-12-27 06:17:52,424][20347] Updated weights for policy 0, policy_version 92177 (0.0006) [2023-12-27 06:17:52,428][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000091872_23527424.pth... [2023-12-27 06:17:52,432][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000090656_23216128.pth [2023-12-27 06:17:52,459][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000092184_23609344.pth... [2023-12-27 06:17:52,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000090936_23289856.pth [2023-12-27 06:17:53,078][20381] Updated weights for policy 1, policy_version 91874 (0.0007) [2023-12-27 06:17:53,111][20347] Updated weights for policy 0, policy_version 92187 (0.0007) [2023-12-27 06:17:53,135][20381] Updated weights for policy 1, policy_version 91884 (0.0007) [2023-12-27 06:17:53,166][20347] Updated weights for policy 0, policy_version 92197 (0.0005) [2023-12-27 06:17:53,189][20381] Updated weights for policy 1, policy_version 91894 (0.0006) [2023-12-27 06:17:53,222][20347] Updated weights for policy 0, policy_version 92207 (0.0006) [2023-12-27 06:17:53,246][20381] Updated weights for policy 1, policy_version 91904 (0.0007) [2023-12-27 06:17:53,884][20381] Updated weights for policy 1, policy_version 91914 (0.0008) [2023-12-27 06:17:53,910][20347] Updated weights for policy 0, policy_version 92217 (0.0006) [2023-12-27 06:17:53,935][20381] Updated weights for policy 1, policy_version 91924 (0.0006) [2023-12-27 06:17:53,969][20347] Updated weights for policy 0, policy_version 92227 (0.0007) [2023-12-27 06:17:53,983][20381] Updated weights for policy 1, policy_version 91934 (0.0005) [2023-12-27 06:17:54,024][20347] Updated weights for policy 0, policy_version 92237 (0.0007) [2023-12-27 06:17:54,080][20347] Updated weights for policy 0, policy_version 92247 (0.0008) [2023-12-27 06:17:54,699][20381] Updated weights for policy 1, policy_version 91944 (0.0006) [2023-12-27 06:17:54,728][20347] Updated weights for policy 0, policy_version 92257 (0.0006) [2023-12-27 06:17:54,753][20381] Updated weights for policy 1, policy_version 91954 (0.0006) [2023-12-27 06:17:54,785][20347] Updated weights for policy 0, policy_version 92267 (0.0006) [2023-12-27 06:17:54,810][20381] Updated weights for policy 1, policy_version 91964 (0.0006) [2023-12-27 06:17:54,843][20347] Updated weights for policy 0, policy_version 92277 (0.0006) [2023-12-27 06:17:55,438][20381] Updated weights for policy 1, policy_version 91974 (0.0006) [2023-12-27 06:17:55,499][20381] Updated weights for policy 1, policy_version 91984 (0.0005) [2023-12-27 06:17:55,549][20347] Updated weights for policy 0, policy_version 92287 (0.0007) [2023-12-27 06:17:55,558][20381] Updated weights for policy 1, policy_version 91994 (0.0006) [2023-12-27 06:17:55,604][20347] Updated weights for policy 0, policy_version 92297 (0.0006) [2023-12-27 06:17:55,661][20347] Updated weights for policy 0, policy_version 92307 (0.0007) [2023-12-27 06:17:56,204][20381] Updated weights for policy 1, policy_version 92004 (0.0006) [2023-12-27 06:17:56,251][20381] Updated weights for policy 1, policy_version 92014 (0.0008) [2023-12-27 06:17:56,300][20381] Updated weights for policy 1, policy_version 92024 (0.0008) [2023-12-27 06:17:56,348][20347] Updated weights for policy 0, policy_version 92317 (0.0007) [2023-12-27 06:17:56,399][20347] Updated weights for policy 0, policy_version 92327 (0.0007) [2023-12-27 06:17:56,447][20347] Updated weights for policy 0, policy_version 92337 (0.0008) [2023-12-27 06:17:57,002][20381] Updated weights for policy 1, policy_version 92034 (0.0008) [2023-12-27 06:17:57,048][20381] Updated weights for policy 1, policy_version 92044 (0.0008) [2023-12-27 06:17:57,102][20381] Updated weights for policy 1, policy_version 92054 (0.0006) [2023-12-27 06:17:57,111][20347] Updated weights for policy 0, policy_version 92347 (0.0008) [2023-12-27 06:17:57,162][20381] Updated weights for policy 1, policy_version 92064 (0.0005) [2023-12-27 06:17:57,165][20347] Updated weights for policy 0, policy_version 92357 (0.0008) [2023-12-27 06:17:57,224][20347] Updated weights for policy 0, policy_version 92367 (0.0007) [2023-12-27 06:17:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 47235072. Throughput: 0: 10457.5, 1: 10419.0. Samples: 47219312. Policy #0 lag: (min: 3.0, avg: 11.5, max: 35.0) [2023-12-27 06:17:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:17:57,792][20381] Updated weights for policy 1, policy_version 92074 (0.0007) [2023-12-27 06:17:57,850][20381] Updated weights for policy 1, policy_version 92084 (0.0007) [2023-12-27 06:17:57,901][20381] Updated weights for policy 1, policy_version 92094 (0.0007) [2023-12-27 06:17:57,906][20347] Updated weights for policy 0, policy_version 92377 (0.0008) [2023-12-27 06:17:57,956][20347] Updated weights for policy 0, policy_version 92387 (0.0007) [2023-12-27 06:17:58,003][20347] Updated weights for policy 0, policy_version 92397 (0.0007) [2023-12-27 06:17:58,051][20347] Updated weights for policy 0, policy_version 92407 (0.0006) [2023-12-27 06:17:58,602][20381] Updated weights for policy 1, policy_version 92104 (0.0008) [2023-12-27 06:17:58,663][20381] Updated weights for policy 1, policy_version 92114 (0.0007) [2023-12-27 06:17:58,729][20381] Updated weights for policy 1, policy_version 92124 (0.0008) [2023-12-27 06:17:58,841][20347] Updated weights for policy 0, policy_version 92417 (0.0007) [2023-12-27 06:17:58,911][20347] Updated weights for policy 0, policy_version 92427 (0.0008) [2023-12-27 06:17:58,975][20347] Updated weights for policy 0, policy_version 92437 (0.0008) [2023-12-27 06:17:59,491][20381] Updated weights for policy 1, policy_version 92134 (0.0008) [2023-12-27 06:17:59,542][20381] Updated weights for policy 1, policy_version 92144 (0.0007) [2023-12-27 06:17:59,606][20381] Updated weights for policy 1, policy_version 92154 (0.0007) [2023-12-27 06:17:59,662][20347] Updated weights for policy 0, policy_version 92447 (0.0006) [2023-12-27 06:17:59,718][20347] Updated weights for policy 0, policy_version 92457 (0.0005) [2023-12-27 06:17:59,781][20347] Updated weights for policy 0, policy_version 92467 (0.0005) [2023-12-27 06:18:00,337][20381] Updated weights for policy 1, policy_version 92164 (0.0009) [2023-12-27 06:18:00,371][20347] Updated weights for policy 0, policy_version 92477 (0.0006) [2023-12-27 06:18:00,397][20381] Updated weights for policy 1, policy_version 92174 (0.0006) [2023-12-27 06:18:00,425][20347] Updated weights for policy 0, policy_version 92487 (0.0005) [2023-12-27 06:18:00,463][20381] Updated weights for policy 1, policy_version 92184 (0.0005) [2023-12-27 06:18:00,472][20347] Updated weights for policy 0, policy_version 92497 (0.0005) [2023-12-27 06:18:01,041][20381] Updated weights for policy 1, policy_version 92194 (0.0006) [2023-12-27 06:18:01,107][20381] Updated weights for policy 1, policy_version 92204 (0.0009) [2023-12-27 06:18:01,142][20347] Updated weights for policy 0, policy_version 92507 (0.0006) [2023-12-27 06:18:01,170][20381] Updated weights for policy 1, policy_version 92214 (0.0008) [2023-12-27 06:18:01,206][20347] Updated weights for policy 0, policy_version 92517 (0.0006) [2023-12-27 06:18:01,231][20381] Updated weights for policy 1, policy_version 92224 (0.0006) [2023-12-27 06:18:01,269][20347] Updated weights for policy 0, policy_version 92527 (0.0007) [2023-12-27 06:18:01,859][20381] Updated weights for policy 1, policy_version 92234 (0.0008) [2023-12-27 06:18:01,863][20347] Updated weights for policy 0, policy_version 92537 (0.0006) [2023-12-27 06:18:01,907][20381] Updated weights for policy 1, policy_version 92244 (0.0007) [2023-12-27 06:18:01,908][20347] Updated weights for policy 0, policy_version 92547 (0.0005) [2023-12-27 06:18:01,962][20381] Updated weights for policy 1, policy_version 92254 (0.0007) [2023-12-27 06:18:01,962][20347] Updated weights for policy 0, policy_version 92557 (0.0006) [2023-12-27 06:18:02,009][20347] Updated weights for policy 0, policy_version 92567 (0.0008) [2023-12-27 06:18:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 47333376. Throughput: 0: 10425.4, 1: 10382.0. Samples: 47342660. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:02,645][20347] Updated weights for policy 0, policy_version 92577 (0.0006) [2023-12-27 06:18:02,680][20381] Updated weights for policy 1, policy_version 92264 (0.0008) [2023-12-27 06:18:02,702][20347] Updated weights for policy 0, policy_version 92587 (0.0005) [2023-12-27 06:18:02,733][20381] Updated weights for policy 1, policy_version 92274 (0.0007) [2023-12-27 06:18:02,750][20347] Updated weights for policy 0, policy_version 92597 (0.0005) [2023-12-27 06:18:02,780][20381] Updated weights for policy 1, policy_version 92284 (0.0008) [2023-12-27 06:18:03,365][20347] Updated weights for policy 0, policy_version 92607 (0.0007) [2023-12-27 06:18:03,410][20347] Updated weights for policy 0, policy_version 92617 (0.0007) [2023-12-27 06:18:03,458][20347] Updated weights for policy 0, policy_version 92627 (0.0007) [2023-12-27 06:18:03,474][20381] Updated weights for policy 1, policy_version 92294 (0.0006) [2023-12-27 06:18:03,528][20381] Updated weights for policy 1, policy_version 92304 (0.0007) [2023-12-27 06:18:03,574][20381] Updated weights for policy 1, policy_version 92314 (0.0008) [2023-12-27 06:18:04,064][20347] Updated weights for policy 0, policy_version 92637 (0.0007) [2023-12-27 06:18:04,119][20347] Updated weights for policy 0, policy_version 92647 (0.0008) [2023-12-27 06:18:04,169][20347] Updated weights for policy 0, policy_version 92657 (0.0008) [2023-12-27 06:18:04,325][20381] Updated weights for policy 1, policy_version 92324 (0.0009) [2023-12-27 06:18:04,378][20381] Updated weights for policy 1, policy_version 92334 (0.0008) [2023-12-27 06:18:04,425][20381] Updated weights for policy 1, policy_version 92344 (0.0008) [2023-12-27 06:18:04,884][20347] Updated weights for policy 0, policy_version 92667 (0.0008) [2023-12-27 06:18:04,942][20347] Updated weights for policy 0, policy_version 92677 (0.0008) [2023-12-27 06:18:04,993][20347] Updated weights for policy 0, policy_version 92687 (0.0007) [2023-12-27 06:18:05,139][20381] Updated weights for policy 1, policy_version 92354 (0.0008) [2023-12-27 06:18:05,189][20381] Updated weights for policy 1, policy_version 92364 (0.0006) [2023-12-27 06:18:05,245][20381] Updated weights for policy 1, policy_version 92374 (0.0005) [2023-12-27 06:18:05,301][20381] Updated weights for policy 1, policy_version 92384 (0.0005) [2023-12-27 06:18:05,703][20347] Updated weights for policy 0, policy_version 92697 (0.0008) [2023-12-27 06:18:05,754][20347] Updated weights for policy 0, policy_version 92707 (0.0008) [2023-12-27 06:18:05,798][20347] Updated weights for policy 0, policy_version 92717 (0.0007) [2023-12-27 06:18:05,846][20347] Updated weights for policy 0, policy_version 92727 (0.0007) [2023-12-27 06:18:05,899][20381] Updated weights for policy 1, policy_version 92394 (0.0008) [2023-12-27 06:18:05,958][20381] Updated weights for policy 1, policy_version 92404 (0.0008) [2023-12-27 06:18:06,011][20381] Updated weights for policy 1, policy_version 92414 (0.0008) [2023-12-27 06:18:06,579][20347] Updated weights for policy 0, policy_version 92737 (0.0008) [2023-12-27 06:18:06,637][20347] Updated weights for policy 0, policy_version 92747 (0.0008) [2023-12-27 06:18:06,696][20347] Updated weights for policy 0, policy_version 92757 (0.0008) [2023-12-27 06:18:06,764][20381] Updated weights for policy 1, policy_version 92424 (0.0008) [2023-12-27 06:18:06,810][20381] Updated weights for policy 1, policy_version 92434 (0.0007) [2023-12-27 06:18:06,860][20381] Updated weights for policy 1, policy_version 92444 (0.0007) [2023-12-27 06:18:07,335][20347] Updated weights for policy 0, policy_version 92767 (0.0008) [2023-12-27 06:18:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 47431680. Throughput: 0: 10436.1, 1: 10329.6. Samples: 47404960. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000092448_23674880.pth... [2023-12-27 06:18:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000091264_23371776.pth [2023-12-27 06:18:07,394][20347] Updated weights for policy 0, policy_version 92777 (0.0008) [2023-12-27 06:18:07,445][20347] Updated weights for policy 0, policy_version 92787 (0.0008) [2023-12-27 06:18:07,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000092792_23764992.pth... [2023-12-27 06:18:07,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000091576_23453696.pth [2023-12-27 06:18:07,605][20381] Updated weights for policy 1, policy_version 92454 (0.0008) [2023-12-27 06:18:07,668][20381] Updated weights for policy 1, policy_version 92464 (0.0008) [2023-12-27 06:18:07,725][20381] Updated weights for policy 1, policy_version 92474 (0.0007) [2023-12-27 06:18:08,092][20347] Updated weights for policy 0, policy_version 92797 (0.0007) [2023-12-27 06:18:08,153][20347] Updated weights for policy 0, policy_version 92807 (0.0008) [2023-12-27 06:18:08,214][20347] Updated weights for policy 0, policy_version 92817 (0.0008) [2023-12-27 06:18:08,463][20381] Updated weights for policy 1, policy_version 92484 (0.0008) [2023-12-27 06:18:08,514][20381] Updated weights for policy 1, policy_version 92494 (0.0008) [2023-12-27 06:18:08,564][20381] Updated weights for policy 1, policy_version 92504 (0.0008) [2023-12-27 06:18:08,879][20347] Updated weights for policy 0, policy_version 92827 (0.0009) [2023-12-27 06:18:08,935][20347] Updated weights for policy 0, policy_version 92837 (0.0008) [2023-12-27 06:18:08,988][20347] Updated weights for policy 0, policy_version 92847 (0.0005) [2023-12-27 06:18:09,317][20381] Updated weights for policy 1, policy_version 92514 (0.0008) [2023-12-27 06:18:09,384][20381] Updated weights for policy 1, policy_version 92524 (0.0008) [2023-12-27 06:18:09,444][20381] Updated weights for policy 1, policy_version 92534 (0.0009) [2023-12-27 06:18:09,501][20381] Updated weights for policy 1, policy_version 92544 (0.0007) [2023-12-27 06:18:09,676][20347] Updated weights for policy 0, policy_version 92857 (0.0005) [2023-12-27 06:18:09,729][20347] Updated weights for policy 0, policy_version 92867 (0.0006) [2023-12-27 06:18:09,788][20347] Updated weights for policy 0, policy_version 92877 (0.0007) [2023-12-27 06:18:09,850][20347] Updated weights for policy 0, policy_version 92887 (0.0007) [2023-12-27 06:18:10,192][20381] Updated weights for policy 1, policy_version 92554 (0.0007) [2023-12-27 06:18:10,253][20381] Updated weights for policy 1, policy_version 92564 (0.0006) [2023-12-27 06:18:10,314][20381] Updated weights for policy 1, policy_version 92574 (0.0008) [2023-12-27 06:18:10,544][20347] Updated weights for policy 0, policy_version 92897 (0.0007) [2023-12-27 06:18:10,595][20347] Updated weights for policy 0, policy_version 92907 (0.0008) [2023-12-27 06:18:10,639][20347] Updated weights for policy 0, policy_version 92917 (0.0007) [2023-12-27 06:18:10,939][20381] Updated weights for policy 1, policy_version 92584 (0.0006) [2023-12-27 06:18:10,985][20381] Updated weights for policy 1, policy_version 92594 (0.0008) [2023-12-27 06:18:11,040][20381] Updated weights for policy 1, policy_version 92604 (0.0007) [2023-12-27 06:18:11,313][20347] Updated weights for policy 0, policy_version 92927 (0.0007) [2023-12-27 06:18:11,381][20347] Updated weights for policy 0, policy_version 92937 (0.0008) [2023-12-27 06:18:11,453][20347] Updated weights for policy 0, policy_version 92947 (0.0007) [2023-12-27 06:18:11,829][20381] Updated weights for policy 1, policy_version 92614 (0.0007) [2023-12-27 06:18:11,897][20381] Updated weights for policy 1, policy_version 92624 (0.0009) [2023-12-27 06:18:11,962][20381] Updated weights for policy 1, policy_version 92634 (0.0008) [2023-12-27 06:18:12,085][20347] Updated weights for policy 0, policy_version 92957 (0.0005) [2023-12-27 06:18:12,147][20347] Updated weights for policy 0, policy_version 92967 (0.0007) [2023-12-27 06:18:12,196][20347] Updated weights for policy 0, policy_version 92977 (0.0007) [2023-12-27 06:18:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 47538176. Throughput: 0: 10410.9, 1: 10284.0. Samples: 47526116. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:12,659][20381] Updated weights for policy 1, policy_version 92644 (0.0007) [2023-12-27 06:18:12,714][20381] Updated weights for policy 1, policy_version 92654 (0.0009) [2023-12-27 06:18:12,770][20381] Updated weights for policy 1, policy_version 92664 (0.0008) [2023-12-27 06:18:12,804][20347] Updated weights for policy 0, policy_version 92987 (0.0007) [2023-12-27 06:18:12,854][20347] Updated weights for policy 0, policy_version 92997 (0.0009) [2023-12-27 06:18:12,905][20347] Updated weights for policy 0, policy_version 93007 (0.0007) [2023-12-27 06:18:13,453][20381] Updated weights for policy 1, policy_version 92674 (0.0007) [2023-12-27 06:18:13,516][20381] Updated weights for policy 1, policy_version 92684 (0.0005) [2023-12-27 06:18:13,543][20347] Updated weights for policy 0, policy_version 93017 (0.0007) [2023-12-27 06:18:13,570][20381] Updated weights for policy 1, policy_version 92694 (0.0005) [2023-12-27 06:18:13,594][20347] Updated weights for policy 0, policy_version 93027 (0.0008) [2023-12-27 06:18:13,621][20381] Updated weights for policy 1, policy_version 92704 (0.0005) [2023-12-27 06:18:13,654][20347] Updated weights for policy 0, policy_version 93037 (0.0008) [2023-12-27 06:18:13,712][20347] Updated weights for policy 0, policy_version 93047 (0.0007) [2023-12-27 06:18:14,159][20381] Updated weights for policy 1, policy_version 92714 (0.0005) [2023-12-27 06:18:14,217][20381] Updated weights for policy 1, policy_version 92724 (0.0006) [2023-12-27 06:18:14,264][20381] Updated weights for policy 1, policy_version 92734 (0.0005) [2023-12-27 06:18:14,401][20347] Updated weights for policy 0, policy_version 93057 (0.0006) [2023-12-27 06:18:14,450][20347] Updated weights for policy 0, policy_version 93067 (0.0006) [2023-12-27 06:18:14,501][20347] Updated weights for policy 0, policy_version 93077 (0.0005) [2023-12-27 06:18:14,907][20381] Updated weights for policy 1, policy_version 92744 (0.0010) [2023-12-27 06:18:14,965][20381] Updated weights for policy 1, policy_version 92754 (0.0009) [2023-12-27 06:18:15,029][20381] Updated weights for policy 1, policy_version 92764 (0.0011) [2023-12-27 06:18:15,141][20347] Updated weights for policy 0, policy_version 93087 (0.0007) [2023-12-27 06:18:15,198][20347] Updated weights for policy 0, policy_version 93097 (0.0008) [2023-12-27 06:18:15,257][20347] Updated weights for policy 0, policy_version 93107 (0.0008) [2023-12-27 06:18:15,644][20381] Updated weights for policy 1, policy_version 92774 (0.0009) [2023-12-27 06:18:15,701][20381] Updated weights for policy 1, policy_version 92784 (0.0007) [2023-12-27 06:18:15,759][20381] Updated weights for policy 1, policy_version 92794 (0.0008) [2023-12-27 06:18:15,946][20347] Updated weights for policy 0, policy_version 93117 (0.0007) [2023-12-27 06:18:16,006][20347] Updated weights for policy 0, policy_version 93127 (0.0007) [2023-12-27 06:18:16,066][20347] Updated weights for policy 0, policy_version 93137 (0.0007) [2023-12-27 06:18:16,438][20381] Updated weights for policy 1, policy_version 92804 (0.0011) [2023-12-27 06:18:16,492][20381] Updated weights for policy 1, policy_version 92814 (0.0010) [2023-12-27 06:18:16,552][20381] Updated weights for policy 1, policy_version 92824 (0.0010) [2023-12-27 06:18:16,653][20347] Updated weights for policy 0, policy_version 93147 (0.0007) [2023-12-27 06:18:16,702][20347] Updated weights for policy 0, policy_version 93157 (0.0008) [2023-12-27 06:18:16,755][20347] Updated weights for policy 0, policy_version 93167 (0.0008) [2023-12-27 06:18:17,181][20381] Updated weights for policy 1, policy_version 92834 (0.0010) [2023-12-27 06:18:17,239][20381] Updated weights for policy 1, policy_version 92844 (0.0010) [2023-12-27 06:18:17,294][20381] Updated weights for policy 1, policy_version 92854 (0.0010) [2023-12-27 06:18:17,352][20381] Updated weights for policy 1, policy_version 92864 (0.0010) [2023-12-27 06:18:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 47644672. Throughput: 0: 10441.4, 1: 10199.5. Samples: 47652772. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:17,486][20347] Updated weights for policy 0, policy_version 93177 (0.0008) [2023-12-27 06:18:17,541][20347] Updated weights for policy 0, policy_version 93187 (0.0008) [2023-12-27 06:18:17,597][20347] Updated weights for policy 0, policy_version 93197 (0.0007) [2023-12-27 06:18:17,652][20347] Updated weights for policy 0, policy_version 93207 (0.0007) [2023-12-27 06:18:17,926][20381] Updated weights for policy 1, policy_version 92874 (0.0010) [2023-12-27 06:18:17,984][20381] Updated weights for policy 1, policy_version 92884 (0.0009) [2023-12-27 06:18:18,046][20381] Updated weights for policy 1, policy_version 92894 (0.0009) [2023-12-27 06:18:18,375][20347] Updated weights for policy 0, policy_version 93217 (0.0007) [2023-12-27 06:18:18,428][20347] Updated weights for policy 0, policy_version 93227 (0.0008) [2023-12-27 06:18:18,491][20347] Updated weights for policy 0, policy_version 93237 (0.0008) [2023-12-27 06:18:18,676][20381] Updated weights for policy 1, policy_version 92904 (0.0008) [2023-12-27 06:18:18,737][20381] Updated weights for policy 1, policy_version 92914 (0.0008) [2023-12-27 06:18:18,796][20381] Updated weights for policy 1, policy_version 92924 (0.0009) [2023-12-27 06:18:19,213][20347] Updated weights for policy 0, policy_version 93247 (0.0008) [2023-12-27 06:18:19,275][20347] Updated weights for policy 0, policy_version 93257 (0.0008) [2023-12-27 06:18:19,326][20347] Updated weights for policy 0, policy_version 93267 (0.0008) [2023-12-27 06:18:19,406][20381] Updated weights for policy 1, policy_version 92934 (0.0007) [2023-12-27 06:18:19,463][20381] Updated weights for policy 1, policy_version 92944 (0.0008) [2023-12-27 06:18:19,527][20381] Updated weights for policy 1, policy_version 92954 (0.0007) [2023-12-27 06:18:20,077][20347] Updated weights for policy 0, policy_version 93277 (0.0008) [2023-12-27 06:18:20,129][20347] Updated weights for policy 0, policy_version 93287 (0.0007) [2023-12-27 06:18:20,186][20347] Updated weights for policy 0, policy_version 93297 (0.0008) [2023-12-27 06:18:20,246][20381] Updated weights for policy 1, policy_version 92964 (0.0007) [2023-12-27 06:18:20,311][20381] Updated weights for policy 1, policy_version 92974 (0.0008) [2023-12-27 06:18:20,376][20381] Updated weights for policy 1, policy_version 92984 (0.0008) [2023-12-27 06:18:20,903][20347] Updated weights for policy 0, policy_version 93307 (0.0007) [2023-12-27 06:18:20,963][20347] Updated weights for policy 0, policy_version 93317 (0.0008) [2023-12-27 06:18:21,028][20347] Updated weights for policy 0, policy_version 93327 (0.0008) [2023-12-27 06:18:21,047][20381] Updated weights for policy 1, policy_version 92994 (0.0007) [2023-12-27 06:18:21,107][20381] Updated weights for policy 1, policy_version 93004 (0.0007) [2023-12-27 06:18:21,167][20381] Updated weights for policy 1, policy_version 93014 (0.0008) [2023-12-27 06:18:21,225][20381] Updated weights for policy 1, policy_version 93024 (0.0008) [2023-12-27 06:18:21,716][20347] Updated weights for policy 0, policy_version 93337 (0.0006) [2023-12-27 06:18:21,773][20347] Updated weights for policy 0, policy_version 93347 (0.0009) [2023-12-27 06:18:21,821][20347] Updated weights for policy 0, policy_version 93357 (0.0008) [2023-12-27 06:18:21,876][20347] Updated weights for policy 0, policy_version 93367 (0.0008) [2023-12-27 06:18:21,969][20381] Updated weights for policy 1, policy_version 93034 (0.0006) [2023-12-27 06:18:22,034][20381] Updated weights for policy 1, policy_version 93044 (0.0006) [2023-12-27 06:18:22,095][20381] Updated weights for policy 1, policy_version 93054 (0.0007) [2023-12-27 06:18:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 47742976. Throughput: 0: 10360.9, 1: 10237.9. Samples: 47713872. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000093368_23912448.pth... [2023-12-27 06:18:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000093056_23830528.pth... [2023-12-27 06:18:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000091872_23527424.pth [2023-12-27 06:18:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000092184_23609344.pth [2023-12-27 06:18:22,620][20347] Updated weights for policy 0, policy_version 93377 (0.0008) [2023-12-27 06:18:22,673][20347] Updated weights for policy 0, policy_version 93387 (0.0008) [2023-12-27 06:18:22,728][20347] Updated weights for policy 0, policy_version 93397 (0.0007) [2023-12-27 06:18:22,749][20381] Updated weights for policy 1, policy_version 93064 (0.0007) [2023-12-27 06:18:22,808][20381] Updated weights for policy 1, policy_version 93074 (0.0008) [2023-12-27 06:18:22,872][20381] Updated weights for policy 1, policy_version 93084 (0.0007) [2023-12-27 06:18:23,466][20347] Updated weights for policy 0, policy_version 93407 (0.0008) [2023-12-27 06:18:23,503][20381] Updated weights for policy 1, policy_version 93094 (0.0006) [2023-12-27 06:18:23,516][20347] Updated weights for policy 0, policy_version 93417 (0.0007) [2023-12-27 06:18:23,548][20381] Updated weights for policy 1, policy_version 93104 (0.0005) [2023-12-27 06:18:23,564][20347] Updated weights for policy 0, policy_version 93427 (0.0006) [2023-12-27 06:18:23,596][20381] Updated weights for policy 1, policy_version 93114 (0.0005) [2023-12-27 06:18:24,288][20347] Updated weights for policy 0, policy_version 93437 (0.0007) [2023-12-27 06:18:24,290][20381] Updated weights for policy 1, policy_version 93124 (0.0007) [2023-12-27 06:18:24,335][20381] Updated weights for policy 1, policy_version 93134 (0.0005) [2023-12-27 06:18:24,347][20347] Updated weights for policy 0, policy_version 93447 (0.0007) [2023-12-27 06:18:24,391][20381] Updated weights for policy 1, policy_version 93144 (0.0005) [2023-12-27 06:18:24,403][20347] Updated weights for policy 0, policy_version 93457 (0.0008) [2023-12-27 06:18:25,026][20381] Updated weights for policy 1, policy_version 93154 (0.0006) [2023-12-27 06:18:25,072][20347] Updated weights for policy 0, policy_version 93467 (0.0007) [2023-12-27 06:18:25,080][20381] Updated weights for policy 1, policy_version 93164 (0.0010) [2023-12-27 06:18:25,131][20381] Updated weights for policy 1, policy_version 93174 (0.0010) [2023-12-27 06:18:25,132][20347] Updated weights for policy 0, policy_version 93477 (0.0005) [2023-12-27 06:18:25,182][20381] Updated weights for policy 1, policy_version 93184 (0.0010) [2023-12-27 06:18:25,186][20347] Updated weights for policy 0, policy_version 93487 (0.0007) [2023-12-27 06:18:25,791][20347] Updated weights for policy 0, policy_version 93497 (0.0005) [2023-12-27 06:18:25,843][20347] Updated weights for policy 0, policy_version 93507 (0.0007) [2023-12-27 06:18:25,857][20381] Updated weights for policy 1, policy_version 93194 (0.0007) [2023-12-27 06:18:25,872][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-12-27 06:18:25,914][20381] Updated weights for policy 1, policy_version 93204 (0.0007) [2023-12-27 06:18:25,963][20381] Updated weights for policy 1, policy_version 93214 (0.0007) [2023-12-27 06:18:26,531][20347] Updated weights for policy 0, policy_version 93517 (0.0006) [2023-12-27 06:18:26,596][20347] Updated weights for policy 0, policy_version 93527 (0.0005) [2023-12-27 06:18:26,605][20381] Updated weights for policy 1, policy_version 93224 (0.0006) [2023-12-27 06:18:26,654][20381] Updated weights for policy 1, policy_version 93234 (0.0005) [2023-12-27 06:18:26,655][20347] Updated weights for policy 0, policy_version 93537 (0.0005) [2023-12-27 06:18:26,709][20381] Updated weights for policy 1, policy_version 93244 (0.0005) [2023-12-27 06:18:27,275][20347] Updated weights for policy 0, policy_version 93547 (0.0007) [2023-12-27 06:18:27,324][20347] Updated weights for policy 0, policy_version 93557 (0.0010) [2023-12-27 06:18:27,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20479.9, 300 sec: 20743.8). Total num frames: 47841280. Throughput: 0: 10354.7, 1: 10258.1. Samples: 47835852. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:27,376][20347] Updated weights for policy 0, policy_version 93567 (0.0010) [2023-12-27 06:18:27,378][20381] Updated weights for policy 1, policy_version 93254 (0.0005) [2023-12-27 06:18:27,429][20381] Updated weights for policy 1, policy_version 93264 (0.0005) [2023-12-27 06:18:27,479][20381] Updated weights for policy 1, policy_version 93274 (0.0007) [2023-12-27 06:18:28,062][20347] Updated weights for policy 0, policy_version 93577 (0.0010) [2023-12-27 06:18:28,115][20347] Updated weights for policy 0, policy_version 93587 (0.0010) [2023-12-27 06:18:28,162][20381] Updated weights for policy 1, policy_version 93284 (0.0007) [2023-12-27 06:18:28,172][20347] Updated weights for policy 0, policy_version 93597 (0.0010) [2023-12-27 06:18:28,215][20381] Updated weights for policy 1, policy_version 93294 (0.0005) [2023-12-27 06:18:28,229][20347] Updated weights for policy 0, policy_version 93607 (0.0010) [2023-12-27 06:18:28,262][20381] Updated weights for policy 1, policy_version 93304 (0.0006) [2023-12-27 06:18:28,899][20347] Updated weights for policy 0, policy_version 93617 (0.0010) [2023-12-27 06:18:28,907][20381] Updated weights for policy 1, policy_version 93314 (0.0008) [2023-12-27 06:18:28,950][20347] Updated weights for policy 0, policy_version 93627 (0.0010) [2023-12-27 06:18:28,957][20381] Updated weights for policy 1, policy_version 93324 (0.0007) [2023-12-27 06:18:29,006][20347] Updated weights for policy 0, policy_version 93637 (0.0010) [2023-12-27 06:18:29,013][20381] Updated weights for policy 1, policy_version 93334 (0.0005) [2023-12-27 06:18:29,079][20381] Updated weights for policy 1, policy_version 93344 (0.0005) [2023-12-27 06:18:29,676][20381] Updated weights for policy 1, policy_version 93354 (0.0008) [2023-12-27 06:18:29,683][20347] Updated weights for policy 0, policy_version 93647 (0.0010) [2023-12-27 06:18:29,731][20381] Updated weights for policy 1, policy_version 93364 (0.0011) [2023-12-27 06:18:29,734][20347] Updated weights for policy 0, policy_version 93657 (0.0010) [2023-12-27 06:18:29,780][20381] Updated weights for policy 1, policy_version 93374 (0.0011) [2023-12-27 06:18:29,789][20347] Updated weights for policy 0, policy_version 93667 (0.0010) [2023-12-27 06:18:30,437][20347] Updated weights for policy 0, policy_version 93677 (0.0010) [2023-12-27 06:18:30,441][20381] Updated weights for policy 1, policy_version 93384 (0.0010) [2023-12-27 06:18:30,485][20347] Updated weights for policy 0, policy_version 93687 (0.0008) [2023-12-27 06:18:30,498][20381] Updated weights for policy 1, policy_version 93394 (0.0010) [2023-12-27 06:18:30,536][20347] Updated weights for policy 0, policy_version 93697 (0.0006) [2023-12-27 06:18:30,546][20381] Updated weights for policy 1, policy_version 93404 (0.0010) [2023-12-27 06:18:31,199][20347] Updated weights for policy 0, policy_version 93707 (0.0007) [2023-12-27 06:18:31,209][20381] Updated weights for policy 1, policy_version 93414 (0.0009) [2023-12-27 06:18:31,254][20347] Updated weights for policy 0, policy_version 93717 (0.0006) [2023-12-27 06:18:31,265][20381] Updated weights for policy 1, policy_version 93424 (0.0008) [2023-12-27 06:18:31,314][20347] Updated weights for policy 0, policy_version 93727 (0.0006) [2023-12-27 06:18:31,317][20381] Updated weights for policy 1, policy_version 93434 (0.0008) [2023-12-27 06:18:31,997][20381] Updated weights for policy 1, policy_version 93444 (0.0007) [2023-12-27 06:18:32,015][20347] Updated weights for policy 0, policy_version 93737 (0.0007) [2023-12-27 06:18:32,055][20381] Updated weights for policy 1, policy_version 93454 (0.0005) [2023-12-27 06:18:32,070][20347] Updated weights for policy 0, policy_version 93747 (0.0007) [2023-12-27 06:18:32,106][20381] Updated weights for policy 1, policy_version 93464 (0.0005) [2023-12-27 06:18:32,120][20347] Updated weights for policy 0, policy_version 93757 (0.0006) [2023-12-27 06:18:32,172][20347] Updated weights for policy 0, policy_version 93767 (0.0006) [2023-12-27 06:18:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 47955968. Throughput: 0: 10366.5, 1: 10288.6. Samples: 47962916. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:18:32,738][20381] Updated weights for policy 1, policy_version 93474 (0.0006) [2023-12-27 06:18:32,792][20381] Updated weights for policy 1, policy_version 93484 (0.0007) [2023-12-27 06:18:32,838][20347] Updated weights for policy 0, policy_version 93777 (0.0007) [2023-12-27 06:18:32,852][20381] Updated weights for policy 1, policy_version 93494 (0.0005) [2023-12-27 06:18:32,894][20347] Updated weights for policy 0, policy_version 93787 (0.0006) [2023-12-27 06:18:32,906][20381] Updated weights for policy 1, policy_version 93504 (0.0005) [2023-12-27 06:18:32,947][20347] Updated weights for policy 0, policy_version 93797 (0.0007) [2023-12-27 06:18:33,596][20347] Updated weights for policy 0, policy_version 93807 (0.0007) [2023-12-27 06:18:33,606][20381] Updated weights for policy 1, policy_version 93514 (0.0007) [2023-12-27 06:18:33,652][20347] Updated weights for policy 0, policy_version 93817 (0.0005) [2023-12-27 06:18:33,664][20381] Updated weights for policy 1, policy_version 93524 (0.0007) [2023-12-27 06:18:33,702][20347] Updated weights for policy 0, policy_version 93827 (0.0005) [2023-12-27 06:18:33,726][20381] Updated weights for policy 1, policy_version 93534 (0.0008) [2023-12-27 06:18:34,238][20347] Updated weights for policy 0, policy_version 93837 (0.0006) [2023-12-27 06:18:34,304][20347] Updated weights for policy 0, policy_version 93847 (0.0008) [2023-12-27 06:18:34,354][20347] Updated weights for policy 0, policy_version 93857 (0.0006) [2023-12-27 06:18:34,378][20381] Updated weights for policy 1, policy_version 93544 (0.0006) [2023-12-27 06:18:34,406][20292] KL-divergence is very high: 139.8796 [2023-12-27 06:18:34,436][20292] KL-divergence is very high: 105.6885 [2023-12-27 06:18:34,441][20381] Updated weights for policy 1, policy_version 93554 (0.0006) [2023-12-27 06:18:34,454][20292] KL-divergence is very high: 192.1882 [2023-12-27 06:18:34,465][20292] KL-divergence is very high: 151.1350 [2023-12-27 06:18:34,504][20381] Updated weights for policy 1, policy_version 93564 (0.0006) [2023-12-27 06:18:34,946][20347] Updated weights for policy 0, policy_version 93867 (0.0007) [2023-12-27 06:18:35,000][20347] Updated weights for policy 0, policy_version 93877 (0.0007) [2023-12-27 06:18:35,011][20292] KL-divergence is very high: 167.9870 [2023-12-27 06:18:35,017][20292] KL-divergence is very high: 153.6141 [2023-12-27 06:18:35,023][20292] KL-divergence is very high: 159.0904 [2023-12-27 06:18:35,030][20292] KL-divergence is very high: 236.6002 [2023-12-27 06:18:35,041][20381] Updated weights for policy 1, policy_version 93574 (0.0006) [2023-12-27 06:18:35,048][20292] KL-divergence is very high: 134.8346 [2023-12-27 06:18:35,054][20292] KL-divergence is very high: 130.9287 [2023-12-27 06:18:35,055][20347] Updated weights for policy 0, policy_version 93887 (0.0006) [2023-12-27 06:18:35,059][20292] KL-divergence is very high: 203.9130 [2023-12-27 06:18:35,064][20292] KL-divergence is very high: 187.4506 [2023-12-27 06:18:35,070][20292] KL-divergence is very high: 188.5914 [2023-12-27 06:18:35,075][20292] KL-divergence is very high: 231.8136 [2023-12-27 06:18:35,092][20292] KL-divergence is very high: 243.7714 [2023-12-27 06:18:35,097][20381] Updated weights for policy 1, policy_version 93584 (0.0005) [2023-12-27 06:18:35,098][20292] KL-divergence is very high: 230.0602 [2023-12-27 06:18:35,103][20292] KL-divergence is very high: 147.2020 [2023-12-27 06:18:35,109][20292] KL-divergence is very high: 195.8689 [2023-12-27 06:18:35,116][20292] KL-divergence is very high: 245.1969 [2023-12-27 06:18:35,122][20292] KL-divergence is very high: 282.1790 [2023-12-27 06:18:35,139][20292] KL-divergence is very high: 309.5609 [2023-12-27 06:18:35,145][20292] KL-divergence is very high: 299.9322 [2023-12-27 06:18:35,152][20292] KL-divergence is very high: 178.7557 [2023-12-27 06:18:35,158][20292] KL-divergence is very high: 173.1250 [2023-12-27 06:18:35,159][20381] Updated weights for policy 1, policy_version 93594 (0.0007) [2023-12-27 06:18:35,164][20292] KL-divergence is very high: 306.1620 [2023-12-27 06:18:35,170][20292] KL-divergence is very high: 430.6422 [2023-12-27 06:18:35,189][20292] KL-divergence is very high: 214.9098 [2023-12-27 06:18:35,708][20347] Updated weights for policy 0, policy_version 93897 (0.0007) [2023-12-27 06:18:35,758][20347] Updated weights for policy 0, policy_version 93907 (0.0009) [2023-12-27 06:18:35,799][20381] Updated weights for policy 1, policy_version 93604 (0.0007) [2023-12-27 06:18:35,807][20347] Updated weights for policy 0, policy_version 93917 (0.0008) [2023-12-27 06:18:35,864][20381] Updated weights for policy 1, policy_version 93614 (0.0006) [2023-12-27 06:18:35,864][20347] Updated weights for policy 0, policy_version 93927 (0.0007) [2023-12-27 06:18:35,925][20381] Updated weights for policy 1, policy_version 93624 (0.0006) [2023-12-27 06:18:36,556][20347] Updated weights for policy 0, policy_version 93937 (0.0008) [2023-12-27 06:18:36,582][20381] Updated weights for policy 1, policy_version 93634 (0.0007) [2023-12-27 06:18:36,613][20347] Updated weights for policy 0, policy_version 93947 (0.0007) [2023-12-27 06:18:36,646][20381] Updated weights for policy 1, policy_version 93644 (0.0005) [2023-12-27 06:18:36,671][20347] Updated weights for policy 0, policy_version 93957 (0.0006) [2023-12-27 06:18:36,706][20381] Updated weights for policy 1, policy_version 93654 (0.0007) [2023-12-27 06:18:36,765][20381] Updated weights for policy 1, policy_version 93664 (0.0006) [2023-12-27 06:18:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 48054272. Throughput: 0: 10394.2, 1: 10326.8. Samples: 48028532. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '210.150')] [2023-12-27 06:18:37,374][20381] Updated weights for policy 1, policy_version 93674 (0.0007) [2023-12-27 06:18:37,384][20347] Updated weights for policy 0, policy_version 93967 (0.0006) [2023-12-27 06:18:37,432][20381] Updated weights for policy 1, policy_version 93684 (0.0006) [2023-12-27 06:18:37,437][20347] Updated weights for policy 0, policy_version 93977 (0.0005) [2023-12-27 06:18:37,482][20381] Updated weights for policy 1, policy_version 93694 (0.0006) [2023-12-27 06:18:37,484][20347] Updated weights for policy 0, policy_version 93987 (0.0006) [2023-12-27 06:18:37,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000093696_23994368.pth... [2023-12-27 06:18:37,496][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000092448_23674880.pth [2023-12-27 06:18:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000093992_24076288.pth... [2023-12-27 06:18:37,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000092792_23764992.pth [2023-12-27 06:18:38,079][20347] Updated weights for policy 0, policy_version 93997 (0.0006) [2023-12-27 06:18:38,132][20347] Updated weights for policy 0, policy_version 94007 (0.0005) [2023-12-27 06:18:38,185][20347] Updated weights for policy 0, policy_version 94017 (0.0008) [2023-12-27 06:18:38,219][20381] Updated weights for policy 1, policy_version 93704 (0.0008) [2023-12-27 06:18:38,270][20381] Updated weights for policy 1, policy_version 93714 (0.0007) [2023-12-27 06:18:38,330][20381] Updated weights for policy 1, policy_version 93724 (0.0007) [2023-12-27 06:18:38,925][20347] Updated weights for policy 0, policy_version 94027 (0.0008) [2023-12-27 06:18:38,965][20381] Updated weights for policy 1, policy_version 93734 (0.0006) [2023-12-27 06:18:38,972][20347] Updated weights for policy 0, policy_version 94037 (0.0008) [2023-12-27 06:18:39,019][20381] Updated weights for policy 1, policy_version 93744 (0.0007) [2023-12-27 06:18:39,021][20347] Updated weights for policy 0, policy_version 94047 (0.0008) [2023-12-27 06:18:39,069][20381] Updated weights for policy 1, policy_version 93754 (0.0006) [2023-12-27 06:18:39,766][20347] Updated weights for policy 0, policy_version 94057 (0.0005) [2023-12-27 06:18:39,779][20381] Updated weights for policy 1, policy_version 93764 (0.0008) [2023-12-27 06:18:39,831][20347] Updated weights for policy 0, policy_version 94067 (0.0006) [2023-12-27 06:18:39,842][20381] Updated weights for policy 1, policy_version 93774 (0.0009) [2023-12-27 06:18:39,893][20347] Updated weights for policy 0, policy_version 94077 (0.0007) [2023-12-27 06:18:39,899][20381] Updated weights for policy 1, policy_version 93784 (0.0005) [2023-12-27 06:18:39,953][20347] Updated weights for policy 0, policy_version 94087 (0.0007) [2023-12-27 06:18:40,578][20381] Updated weights for policy 1, policy_version 93794 (0.0006) [2023-12-27 06:18:40,591][20347] Updated weights for policy 0, policy_version 94097 (0.0005) [2023-12-27 06:18:40,635][20381] Updated weights for policy 1, policy_version 93804 (0.0007) [2023-12-27 06:18:40,646][20347] Updated weights for policy 0, policy_version 94107 (0.0006) [2023-12-27 06:18:40,695][20381] Updated weights for policy 1, policy_version 93814 (0.0007) [2023-12-27 06:18:40,703][20347] Updated weights for policy 0, policy_version 94117 (0.0007) [2023-12-27 06:18:40,753][20381] Updated weights for policy 1, policy_version 93824 (0.0006) [2023-12-27 06:18:41,332][20347] Updated weights for policy 0, policy_version 94127 (0.0008) [2023-12-27 06:18:41,404][20347] Updated weights for policy 0, policy_version 94137 (0.0007) [2023-12-27 06:18:41,435][20381] Updated weights for policy 1, policy_version 93834 (0.0009) [2023-12-27 06:18:41,455][20347] Updated weights for policy 0, policy_version 94147 (0.0007) [2023-12-27 06:18:41,491][20381] Updated weights for policy 1, policy_version 93844 (0.0007) [2023-12-27 06:18:41,551][20381] Updated weights for policy 1, policy_version 93854 (0.0005) [2023-12-27 06:18:42,199][20381] Updated weights for policy 1, policy_version 93864 (0.0006) [2023-12-27 06:18:42,208][20347] Updated weights for policy 0, policy_version 94157 (0.0010) [2023-12-27 06:18:42,251][20381] Updated weights for policy 1, policy_version 93874 (0.0008) [2023-12-27 06:18:42,266][20347] Updated weights for policy 0, policy_version 94167 (0.0010) [2023-12-27 06:18:42,311][20381] Updated weights for policy 1, policy_version 93884 (0.0008) [2023-12-27 06:18:42,327][20347] Updated weights for policy 0, policy_version 94177 (0.0010) [2023-12-27 06:18:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 48160768. Throughput: 0: 10391.0, 1: 10330.0. Samples: 48151760. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '195.870')] [2023-12-27 06:18:42,974][20381] Updated weights for policy 1, policy_version 93894 (0.0005) [2023-12-27 06:18:43,001][20347] Updated weights for policy 0, policy_version 94187 (0.0008) [2023-12-27 06:18:43,023][20381] Updated weights for policy 1, policy_version 93904 (0.0005) [2023-12-27 06:18:43,050][20347] Updated weights for policy 0, policy_version 94197 (0.0005) [2023-12-27 06:18:43,072][20381] Updated weights for policy 1, policy_version 93914 (0.0005) [2023-12-27 06:18:43,107][20347] Updated weights for policy 0, policy_version 94207 (0.0005) [2023-12-27 06:18:43,677][20381] Updated weights for policy 1, policy_version 93924 (0.0006) [2023-12-27 06:18:43,710][20347] Updated weights for policy 0, policy_version 94217 (0.0005) [2023-12-27 06:18:43,731][20381] Updated weights for policy 1, policy_version 93934 (0.0007) [2023-12-27 06:18:43,766][20347] Updated weights for policy 0, policy_version 94227 (0.0005) [2023-12-27 06:18:43,788][20381] Updated weights for policy 1, policy_version 93944 (0.0008) [2023-12-27 06:18:43,816][20347] Updated weights for policy 0, policy_version 94237 (0.0009) [2023-12-27 06:18:43,861][20347] Updated weights for policy 0, policy_version 94247 (0.0010) [2023-12-27 06:18:44,424][20381] Updated weights for policy 1, policy_version 93954 (0.0006) [2023-12-27 06:18:44,470][20381] Updated weights for policy 1, policy_version 93964 (0.0006) [2023-12-27 06:18:44,526][20381] Updated weights for policy 1, policy_version 93974 (0.0007) [2023-12-27 06:18:44,533][20347] Updated weights for policy 0, policy_version 94257 (0.0010) [2023-12-27 06:18:44,584][20381] Updated weights for policy 1, policy_version 93984 (0.0007) [2023-12-27 06:18:44,588][20347] Updated weights for policy 0, policy_version 94267 (0.0010) [2023-12-27 06:18:44,646][20347] Updated weights for policy 0, policy_version 94277 (0.0010) [2023-12-27 06:18:45,243][20381] Updated weights for policy 1, policy_version 93994 (0.0005) [2023-12-27 06:18:45,282][20292] KL-divergence is very high: 152.4200 [2023-12-27 06:18:45,288][20292] KL-divergence is very high: 140.5552 [2023-12-27 06:18:45,299][20381] Updated weights for policy 1, policy_version 94004 (0.0006) [2023-12-27 06:18:45,307][20347] Updated weights for policy 0, policy_version 94287 (0.0008) [2023-12-27 06:18:45,330][20292] KL-divergence is very high: 107.8939 [2023-12-27 06:18:45,349][20381] Updated weights for policy 1, policy_version 94014 (0.0007) [2023-12-27 06:18:45,364][20347] Updated weights for policy 0, policy_version 94297 (0.0008) [2023-12-27 06:18:45,418][20347] Updated weights for policy 0, policy_version 94307 (0.0007) [2023-12-27 06:18:45,960][20381] Updated weights for policy 1, policy_version 94024 (0.0005) [2023-12-27 06:18:46,011][20292] KL-divergence is very high: 101.1665 [2023-12-27 06:18:46,022][20381] Updated weights for policy 1, policy_version 94034 (0.0006) [2023-12-27 06:18:46,071][20292] KL-divergence is very high: 106.4155 [2023-12-27 06:18:46,084][20381] Updated weights for policy 1, policy_version 94044 (0.0008) [2023-12-27 06:18:46,095][20292] KL-divergence is very high: 121.6010 [2023-12-27 06:18:46,108][20292] KL-divergence is very high: 126.2749 [2023-12-27 06:18:46,115][20347] Updated weights for policy 0, policy_version 94317 (0.0007) [2023-12-27 06:18:46,173][20347] Updated weights for policy 0, policy_version 94327 (0.0008) [2023-12-27 06:18:46,228][20347] Updated weights for policy 0, policy_version 94337 (0.0007) [2023-12-27 06:18:46,636][20381] Updated weights for policy 1, policy_version 94054 (0.0005) [2023-12-27 06:18:46,692][20381] Updated weights for policy 1, policy_version 94064 (0.0006) [2023-12-27 06:18:46,742][20381] Updated weights for policy 1, policy_version 94074 (0.0006) [2023-12-27 06:18:46,898][20347] Updated weights for policy 0, policy_version 94347 (0.0006) [2023-12-27 06:18:46,962][20347] Updated weights for policy 0, policy_version 94357 (0.0005) [2023-12-27 06:18:47,028][20347] Updated weights for policy 0, policy_version 94367 (0.0005) [2023-12-27 06:18:47,329][20381] Updated weights for policy 1, policy_version 94084 (0.0009) [2023-12-27 06:18:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 48267264. Throughput: 0: 10385.2, 1: 10438.7. Samples: 48279732. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '204.740')] [2023-12-27 06:18:47,390][20381] Updated weights for policy 1, policy_version 94094 (0.0008) [2023-12-27 06:18:47,397][20292] KL-divergence is very high: 100.8648 [2023-12-27 06:18:47,456][20381] Updated weights for policy 1, policy_version 94104 (0.0008) [2023-12-27 06:18:47,533][20347] Updated weights for policy 0, policy_version 94377 (0.0007) [2023-12-27 06:18:47,592][20347] Updated weights for policy 0, policy_version 94387 (0.0005) [2023-12-27 06:18:47,648][20347] Updated weights for policy 0, policy_version 94397 (0.0005) [2023-12-27 06:18:47,706][20347] Updated weights for policy 0, policy_version 94407 (0.0006) [2023-12-27 06:18:48,089][20381] Updated weights for policy 1, policy_version 94114 (0.0009) [2023-12-27 06:18:48,138][20381] Updated weights for policy 1, policy_version 94124 (0.0006) [2023-12-27 06:18:48,196][20381] Updated weights for policy 1, policy_version 94134 (0.0005) [2023-12-27 06:18:48,251][20381] Updated weights for policy 1, policy_version 94144 (0.0008) [2023-12-27 06:18:48,355][20347] Updated weights for policy 0, policy_version 94417 (0.0007) [2023-12-27 06:18:48,411][20347] Updated weights for policy 0, policy_version 94427 (0.0007) [2023-12-27 06:18:48,473][20347] Updated weights for policy 0, policy_version 94437 (0.0007) [2023-12-27 06:18:48,934][20381] Updated weights for policy 1, policy_version 94154 (0.0008) [2023-12-27 06:18:48,994][20381] Updated weights for policy 1, policy_version 94164 (0.0008) [2023-12-27 06:18:49,049][20381] Updated weights for policy 1, policy_version 94174 (0.0008) [2023-12-27 06:18:49,113][20347] Updated weights for policy 0, policy_version 94447 (0.0010) [2023-12-27 06:18:49,161][20347] Updated weights for policy 0, policy_version 94457 (0.0007) [2023-12-27 06:18:49,218][20347] Updated weights for policy 0, policy_version 94467 (0.0006) [2023-12-27 06:18:49,767][20381] Updated weights for policy 1, policy_version 94184 (0.0007) [2023-12-27 06:18:49,818][20381] Updated weights for policy 1, policy_version 94194 (0.0007) [2023-12-27 06:18:49,878][20381] Updated weights for policy 1, policy_version 94204 (0.0008) [2023-12-27 06:18:49,908][20347] Updated weights for policy 0, policy_version 94477 (0.0009) [2023-12-27 06:18:49,969][20347] Updated weights for policy 0, policy_version 94487 (0.0010) [2023-12-27 06:18:50,019][20347] Updated weights for policy 0, policy_version 94497 (0.0007) [2023-12-27 06:18:50,553][20381] Updated weights for policy 1, policy_version 94214 (0.0007) [2023-12-27 06:18:50,611][20381] Updated weights for policy 1, policy_version 94224 (0.0008) [2023-12-27 06:18:50,672][20381] Updated weights for policy 1, policy_version 94234 (0.0006) [2023-12-27 06:18:50,685][20347] Updated weights for policy 0, policy_version 94507 (0.0009) [2023-12-27 06:18:50,741][20347] Updated weights for policy 0, policy_version 94517 (0.0010) [2023-12-27 06:18:50,798][20347] Updated weights for policy 0, policy_version 94527 (0.0009) [2023-12-27 06:18:51,338][20381] Updated weights for policy 1, policy_version 94244 (0.0007) [2023-12-27 06:18:51,394][20381] Updated weights for policy 1, policy_version 94254 (0.0007) [2023-12-27 06:18:51,444][20381] Updated weights for policy 1, policy_version 94264 (0.0007) [2023-12-27 06:18:51,488][20347] Updated weights for policy 0, policy_version 94537 (0.0010) [2023-12-27 06:18:51,546][20347] Updated weights for policy 0, policy_version 94547 (0.0008) [2023-12-27 06:18:51,594][20347] Updated weights for policy 0, policy_version 94557 (0.0008) [2023-12-27 06:18:51,656][20347] Updated weights for policy 0, policy_version 94567 (0.0008) [2023-12-27 06:18:52,129][20381] Updated weights for policy 1, policy_version 94274 (0.0008) [2023-12-27 06:18:52,181][20381] Updated weights for policy 1, policy_version 94284 (0.0007) [2023-12-27 06:18:52,233][20381] Updated weights for policy 1, policy_version 94294 (0.0007) [2023-12-27 06:18:52,292][20381] Updated weights for policy 1, policy_version 94304 (0.0007) [2023-12-27 06:18:52,342][20347] Updated weights for policy 0, policy_version 94577 (0.0007) [2023-12-27 06:18:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 48373760. Throughput: 0: 10382.6, 1: 10454.1. Samples: 48342608. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '289.120')] [2023-12-27 06:18:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000094304_24150016.pth... [2023-12-27 06:18:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000093056_23830528.pth [2023-12-27 06:18:52,400][20347] Updated weights for policy 0, policy_version 94587 (0.0006) [2023-12-27 06:18:52,453][20347] Updated weights for policy 0, policy_version 94597 (0.0009) [2023-12-27 06:18:52,470][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000094600_24231936.pth... [2023-12-27 06:18:52,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000093368_23912448.pth [2023-12-27 06:18:52,890][20381] Updated weights for policy 1, policy_version 94314 (0.0006) [2023-12-27 06:18:52,945][20381] Updated weights for policy 1, policy_version 94324 (0.0008) [2023-12-27 06:18:53,000][20381] Updated weights for policy 1, policy_version 94334 (0.0007) [2023-12-27 06:18:53,170][20347] Updated weights for policy 0, policy_version 94607 (0.0006) [2023-12-27 06:18:53,232][20347] Updated weights for policy 0, policy_version 94617 (0.0005) [2023-12-27 06:18:53,291][20347] Updated weights for policy 0, policy_version 94627 (0.0005) [2023-12-27 06:18:53,716][20381] Updated weights for policy 1, policy_version 94344 (0.0008) [2023-12-27 06:18:53,778][20381] Updated weights for policy 1, policy_version 94354 (0.0008) [2023-12-27 06:18:53,832][20381] Updated weights for policy 1, policy_version 94364 (0.0008) [2023-12-27 06:18:53,889][20347] Updated weights for policy 0, policy_version 94637 (0.0006) [2023-12-27 06:18:53,950][20347] Updated weights for policy 0, policy_version 94647 (0.0006) [2023-12-27 06:18:54,013][20347] Updated weights for policy 0, policy_version 94657 (0.0006) [2023-12-27 06:18:54,553][20381] Updated weights for policy 1, policy_version 94374 (0.0009) [2023-12-27 06:18:54,576][20347] Updated weights for policy 0, policy_version 94667 (0.0005) [2023-12-27 06:18:54,611][20381] Updated weights for policy 1, policy_version 94384 (0.0007) [2023-12-27 06:18:54,628][20347] Updated weights for policy 0, policy_version 94677 (0.0006) [2023-12-27 06:18:54,667][20381] Updated weights for policy 1, policy_version 94394 (0.0006) [2023-12-27 06:18:54,672][20347] Updated weights for policy 0, policy_version 94687 (0.0007) [2023-12-27 06:18:55,311][20347] Updated weights for policy 0, policy_version 94697 (0.0008) [2023-12-27 06:18:55,356][20347] Updated weights for policy 0, policy_version 94707 (0.0010) [2023-12-27 06:18:55,376][20381] Updated weights for policy 1, policy_version 94404 (0.0007) [2023-12-27 06:18:55,407][20347] Updated weights for policy 0, policy_version 94717 (0.0010) [2023-12-27 06:18:55,425][20381] Updated weights for policy 1, policy_version 94414 (0.0010) [2023-12-27 06:18:55,468][20347] Updated weights for policy 0, policy_version 94727 (0.0010) [2023-12-27 06:18:55,476][20381] Updated weights for policy 1, policy_version 94424 (0.0006) [2023-12-27 06:18:56,059][20347] Updated weights for policy 0, policy_version 94737 (0.0010) [2023-12-27 06:18:56,123][20347] Updated weights for policy 0, policy_version 94747 (0.0010) [2023-12-27 06:18:56,172][20381] Updated weights for policy 1, policy_version 94434 (0.0010) [2023-12-27 06:18:56,178][20347] Updated weights for policy 0, policy_version 94757 (0.0010) [2023-12-27 06:18:56,224][20381] Updated weights for policy 1, policy_version 94444 (0.0010) [2023-12-27 06:18:56,281][20381] Updated weights for policy 1, policy_version 94454 (0.0010) [2023-12-27 06:18:56,344][20381] Updated weights for policy 1, policy_version 94464 (0.0011) [2023-12-27 06:18:56,809][20347] Updated weights for policy 0, policy_version 94767 (0.0010) [2023-12-27 06:18:56,867][20347] Updated weights for policy 0, policy_version 94777 (0.0010) [2023-12-27 06:18:56,930][20347] Updated weights for policy 0, policy_version 94787 (0.0011) [2023-12-27 06:18:57,019][20381] Updated weights for policy 1, policy_version 94474 (0.0005) [2023-12-27 06:18:57,075][20381] Updated weights for policy 1, policy_version 94484 (0.0005) [2023-12-27 06:18:57,134][20381] Updated weights for policy 1, policy_version 94494 (0.0008) [2023-12-27 06:18:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 48480256. Throughput: 0: 10451.0, 1: 10490.9. Samples: 48468500. Policy #0 lag: (min: 31.0, avg: 31.4, max: 46.0) [2023-12-27 06:18:57,363][19170] Avg episode reward: [(0, '859.990'), (1, '230.990')] [2023-12-27 06:18:57,563][20347] Updated weights for policy 0, policy_version 94797 (0.0008) [2023-12-27 06:18:57,621][20347] Updated weights for policy 0, policy_version 94807 (0.0005) [2023-12-27 06:18:57,674][20347] Updated weights for policy 0, policy_version 94817 (0.0005) [2023-12-27 06:18:57,770][20381] Updated weights for policy 1, policy_version 94504 (0.0008) [2023-12-27 06:18:57,822][20381] Updated weights for policy 1, policy_version 94514 (0.0008) [2023-12-27 06:18:57,876][20381] Updated weights for policy 1, policy_version 94524 (0.0005) [2023-12-27 06:18:58,240][20347] Updated weights for policy 0, policy_version 94827 (0.0007) [2023-12-27 06:18:58,303][20347] Updated weights for policy 0, policy_version 94837 (0.0007) [2023-12-27 06:18:58,363][20347] Updated weights for policy 0, policy_version 94847 (0.0007) [2023-12-27 06:18:58,588][20381] Updated weights for policy 1, policy_version 94534 (0.0008) [2023-12-27 06:18:58,646][20381] Updated weights for policy 1, policy_version 94544 (0.0008) [2023-12-27 06:18:58,704][20381] Updated weights for policy 1, policy_version 94554 (0.0007) [2023-12-27 06:18:59,039][20347] Updated weights for policy 0, policy_version 94857 (0.0007) [2023-12-27 06:18:59,090][20347] Updated weights for policy 0, policy_version 94867 (0.0005) [2023-12-27 06:18:59,148][20347] Updated weights for policy 0, policy_version 94877 (0.0007) [2023-12-27 06:18:59,209][20347] Updated weights for policy 0, policy_version 94887 (0.0008) [2023-12-27 06:18:59,410][20381] Updated weights for policy 1, policy_version 94564 (0.0009) [2023-12-27 06:18:59,475][20381] Updated weights for policy 1, policy_version 94574 (0.0010) [2023-12-27 06:18:59,529][20381] Updated weights for policy 1, policy_version 94584 (0.0010) [2023-12-27 06:18:59,990][20347] Updated weights for policy 0, policy_version 94897 (0.0008) [2023-12-27 06:19:00,056][20347] Updated weights for policy 0, policy_version 94907 (0.0008) [2023-12-27 06:19:00,104][20347] Updated weights for policy 0, policy_version 94917 (0.0007) [2023-12-27 06:19:00,225][20381] Updated weights for policy 1, policy_version 94594 (0.0010) [2023-12-27 06:19:00,278][20381] Updated weights for policy 1, policy_version 94604 (0.0011) [2023-12-27 06:19:00,325][20381] Updated weights for policy 1, policy_version 94614 (0.0010) [2023-12-27 06:19:00,380][20381] Updated weights for policy 1, policy_version 94624 (0.0009) [2023-12-27 06:19:00,697][20347] Updated weights for policy 0, policy_version 94927 (0.0009) [2023-12-27 06:19:00,743][20347] Updated weights for policy 0, policy_version 94937 (0.0008) [2023-12-27 06:19:00,791][20347] Updated weights for policy 0, policy_version 94947 (0.0007) [2023-12-27 06:19:00,978][20381] Updated weights for policy 1, policy_version 94634 (0.0007) [2023-12-27 06:19:01,028][20381] Updated weights for policy 1, policy_version 94644 (0.0008) [2023-12-27 06:19:01,088][20381] Updated weights for policy 1, policy_version 94654 (0.0007) [2023-12-27 06:19:01,476][20347] Updated weights for policy 0, policy_version 94957 (0.0009) [2023-12-27 06:19:01,536][20347] Updated weights for policy 0, policy_version 94967 (0.0010) [2023-12-27 06:19:01,588][20347] Updated weights for policy 0, policy_version 94977 (0.0010) [2023-12-27 06:19:01,792][20381] Updated weights for policy 1, policy_version 94664 (0.0007) [2023-12-27 06:19:01,849][20381] Updated weights for policy 1, policy_version 94674 (0.0006) [2023-12-27 06:19:01,915][20381] Updated weights for policy 1, policy_version 94684 (0.0007) [2023-12-27 06:19:02,289][20347] Updated weights for policy 0, policy_version 94987 (0.0011) [2023-12-27 06:19:02,349][20347] Updated weights for policy 0, policy_version 94997 (0.0008) [2023-12-27 06:19:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 48578560. Throughput: 0: 10433.3, 1: 10452.8. Samples: 48592648. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:02,363][19170] Avg episode reward: [(0, '630.580'), (1, '226.310')] [2023-12-27 06:19:02,411][20347] Updated weights for policy 0, policy_version 95007 (0.0008) [2023-12-27 06:19:02,555][20381] Updated weights for policy 1, policy_version 94694 (0.0008) [2023-12-27 06:19:02,607][20381] Updated weights for policy 1, policy_version 94704 (0.0008) [2023-12-27 06:19:02,667][20381] Updated weights for policy 1, policy_version 94714 (0.0008) [2023-12-27 06:19:03,014][20347] Updated weights for policy 0, policy_version 95017 (0.0007) [2023-12-27 06:19:03,061][20347] Updated weights for policy 0, policy_version 95027 (0.0007) [2023-12-27 06:19:03,121][20347] Updated weights for policy 0, policy_version 95037 (0.0005) [2023-12-27 06:19:03,182][20347] Updated weights for policy 0, policy_version 95047 (0.0005) [2023-12-27 06:19:03,372][20381] Updated weights for policy 1, policy_version 94724 (0.0007) [2023-12-27 06:19:03,413][20381] Updated weights for policy 1, policy_version 94734 (0.0007) [2023-12-27 06:19:03,458][20381] Updated weights for policy 1, policy_version 94744 (0.0008) [2023-12-27 06:19:03,747][20347] Updated weights for policy 0, policy_version 95057 (0.0007) [2023-12-27 06:19:03,797][20347] Updated weights for policy 0, policy_version 95067 (0.0008) [2023-12-27 06:19:03,857][20347] Updated weights for policy 0, policy_version 95077 (0.0008) [2023-12-27 06:19:04,135][20381] Updated weights for policy 1, policy_version 94754 (0.0008) [2023-12-27 06:19:04,191][20381] Updated weights for policy 1, policy_version 94764 (0.0008) [2023-12-27 06:19:04,240][20381] Updated weights for policy 1, policy_version 94774 (0.0008) [2023-12-27 06:19:04,290][20381] Updated weights for policy 1, policy_version 94784 (0.0007) [2023-12-27 06:19:04,552][20347] Updated weights for policy 0, policy_version 95087 (0.0008) [2023-12-27 06:19:04,611][20347] Updated weights for policy 0, policy_version 95097 (0.0007) [2023-12-27 06:19:04,668][20347] Updated weights for policy 0, policy_version 95107 (0.0007) [2023-12-27 06:19:04,899][20381] Updated weights for policy 1, policy_version 94794 (0.0005) [2023-12-27 06:19:04,948][20381] Updated weights for policy 1, policy_version 94804 (0.0005) [2023-12-27 06:19:05,008][20381] Updated weights for policy 1, policy_version 94814 (0.0008) [2023-12-27 06:19:05,354][20347] Updated weights for policy 0, policy_version 95117 (0.0007) [2023-12-27 06:19:05,408][20347] Updated weights for policy 0, policy_version 95127 (0.0007) [2023-12-27 06:19:05,460][20347] Updated weights for policy 0, policy_version 95137 (0.0007) [2023-12-27 06:19:05,645][20381] Updated weights for policy 1, policy_version 94824 (0.0008) [2023-12-27 06:19:05,694][20381] Updated weights for policy 1, policy_version 94834 (0.0007) [2023-12-27 06:19:05,745][20381] Updated weights for policy 1, policy_version 94844 (0.0007) [2023-12-27 06:19:06,026][20347] Updated weights for policy 0, policy_version 95147 (0.0006) [2023-12-27 06:19:06,081][20347] Updated weights for policy 0, policy_version 95157 (0.0008) [2023-12-27 06:19:06,140][20347] Updated weights for policy 0, policy_version 95167 (0.0008) [2023-12-27 06:19:06,450][20381] Updated weights for policy 1, policy_version 94854 (0.0007) [2023-12-27 06:19:06,516][20381] Updated weights for policy 1, policy_version 94864 (0.0007) [2023-12-27 06:19:06,572][20381] Updated weights for policy 1, policy_version 94874 (0.0008) [2023-12-27 06:19:06,886][20347] Updated weights for policy 0, policy_version 95177 (0.0008) [2023-12-27 06:19:06,938][20347] Updated weights for policy 0, policy_version 95187 (0.0008) [2023-12-27 06:19:06,994][20347] Updated weights for policy 0, policy_version 95197 (0.0007) [2023-12-27 06:19:07,052][20347] Updated weights for policy 0, policy_version 95207 (0.0005) [2023-12-27 06:19:07,182][20381] Updated weights for policy 1, policy_version 94884 (0.0007) [2023-12-27 06:19:07,227][20381] Updated weights for policy 1, policy_version 94894 (0.0009) [2023-12-27 06:19:07,274][20381] Updated weights for policy 1, policy_version 94904 (0.0010) [2023-12-27 06:19:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 48693248. Throughput: 0: 10506.6, 1: 10439.2. Samples: 48656440. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:07,364][19170] Avg episode reward: [(0, '755.300'), (1, '156.200')] [2023-12-27 06:19:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000094912_24305664.pth... [2023-12-27 06:19:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000095208_24387584.pth... [2023-12-27 06:19:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000093696_23994368.pth [2023-12-27 06:19:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000093992_24076288.pth [2023-12-27 06:19:07,620][20347] Updated weights for policy 0, policy_version 95217 (0.0006) [2023-12-27 06:19:07,671][20347] Updated weights for policy 0, policy_version 95227 (0.0007) [2023-12-27 06:19:07,727][20347] Updated weights for policy 0, policy_version 95237 (0.0008) [2023-12-27 06:19:07,999][20381] Updated weights for policy 1, policy_version 94914 (0.0009) [2023-12-27 06:19:08,062][20381] Updated weights for policy 1, policy_version 94924 (0.0009) [2023-12-27 06:19:08,121][20381] Updated weights for policy 1, policy_version 94934 (0.0008) [2023-12-27 06:19:08,183][20381] Updated weights for policy 1, policy_version 94944 (0.0007) [2023-12-27 06:19:08,372][20347] Updated weights for policy 0, policy_version 95247 (0.0008) [2023-12-27 06:19:08,429][20347] Updated weights for policy 0, policy_version 95257 (0.0008) [2023-12-27 06:19:08,486][20347] Updated weights for policy 0, policy_version 95267 (0.0008) [2023-12-27 06:19:08,879][20381] Updated weights for policy 1, policy_version 94954 (0.0008) [2023-12-27 06:19:08,948][20381] Updated weights for policy 1, policy_version 94964 (0.0006) [2023-12-27 06:19:09,007][20381] Updated weights for policy 1, policy_version 94974 (0.0007) [2023-12-27 06:19:09,130][20347] Updated weights for policy 0, policy_version 95277 (0.0006) [2023-12-27 06:19:09,185][20347] Updated weights for policy 0, policy_version 95287 (0.0007) [2023-12-27 06:19:09,250][20347] Updated weights for policy 0, policy_version 95297 (0.0007) [2023-12-27 06:19:09,705][20381] Updated weights for policy 1, policy_version 94984 (0.0008) [2023-12-27 06:19:09,770][20381] Updated weights for policy 1, policy_version 94994 (0.0008) [2023-12-27 06:19:09,831][20381] Updated weights for policy 1, policy_version 95004 (0.0008) [2023-12-27 06:19:09,922][20347] Updated weights for policy 0, policy_version 95307 (0.0006) [2023-12-27 06:19:09,990][20347] Updated weights for policy 0, policy_version 95317 (0.0008) [2023-12-27 06:19:10,056][20347] Updated weights for policy 0, policy_version 95327 (0.0007) [2023-12-27 06:19:10,533][20381] Updated weights for policy 1, policy_version 95014 (0.0007) [2023-12-27 06:19:10,597][20381] Updated weights for policy 1, policy_version 95024 (0.0006) [2023-12-27 06:19:10,664][20381] Updated weights for policy 1, policy_version 95034 (0.0008) [2023-12-27 06:19:10,778][20347] Updated weights for policy 0, policy_version 95337 (0.0008) [2023-12-27 06:19:10,841][20347] Updated weights for policy 0, policy_version 95347 (0.0007) [2023-12-27 06:19:10,900][20347] Updated weights for policy 0, policy_version 95357 (0.0005) [2023-12-27 06:19:10,957][20347] Updated weights for policy 0, policy_version 95367 (0.0006) [2023-12-27 06:19:11,355][20381] Updated weights for policy 1, policy_version 95044 (0.0008) [2023-12-27 06:19:11,420][20381] Updated weights for policy 1, policy_version 95054 (0.0006) [2023-12-27 06:19:11,481][20381] Updated weights for policy 1, policy_version 95064 (0.0008) [2023-12-27 06:19:11,596][20347] Updated weights for policy 0, policy_version 95377 (0.0005) [2023-12-27 06:19:11,661][20347] Updated weights for policy 0, policy_version 95387 (0.0009) [2023-12-27 06:19:11,724][20347] Updated weights for policy 0, policy_version 95397 (0.0008) [2023-12-27 06:19:12,251][20381] Updated weights for policy 1, policy_version 95074 (0.0009) [2023-12-27 06:19:12,313][20347] Updated weights for policy 0, policy_version 95407 (0.0006) [2023-12-27 06:19:12,316][20381] Updated weights for policy 1, policy_version 95084 (0.0008) [2023-12-27 06:19:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 48783360. Throughput: 0: 10572.7, 1: 10408.7. Samples: 48780012. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:12,363][19170] Avg episode reward: [(0, '289.600'), (1, '121.600')] [2023-12-27 06:19:12,378][20347] Updated weights for policy 0, policy_version 95417 (0.0006) [2023-12-27 06:19:12,382][20381] Updated weights for policy 1, policy_version 95094 (0.0008) [2023-12-27 06:19:12,437][20347] Updated weights for policy 0, policy_version 95427 (0.0005) [2023-12-27 06:19:12,439][20381] Updated weights for policy 1, policy_version 95104 (0.0006) [2023-12-27 06:19:13,089][20347] Updated weights for policy 0, policy_version 95437 (0.0009) [2023-12-27 06:19:13,131][20381] Updated weights for policy 1, policy_version 95114 (0.0006) [2023-12-27 06:19:13,143][20347] Updated weights for policy 0, policy_version 95447 (0.0010) [2023-12-27 06:19:13,187][20381] Updated weights for policy 1, policy_version 95124 (0.0006) [2023-12-27 06:19:13,196][20347] Updated weights for policy 0, policy_version 95457 (0.0010) [2023-12-27 06:19:13,243][20381] Updated weights for policy 1, policy_version 95134 (0.0008) [2023-12-27 06:19:13,760][20347] Updated weights for policy 0, policy_version 95467 (0.0006) [2023-12-27 06:19:13,808][20347] Updated weights for policy 0, policy_version 95477 (0.0010) [2023-12-27 06:19:13,856][20347] Updated weights for policy 0, policy_version 95487 (0.0010) [2023-12-27 06:19:13,943][20381] Updated weights for policy 1, policy_version 95144 (0.0008) [2023-12-27 06:19:13,993][20381] Updated weights for policy 1, policy_version 95154 (0.0007) [2023-12-27 06:19:14,043][20381] Updated weights for policy 1, policy_version 95164 (0.0008) [2023-12-27 06:19:14,543][20347] Updated weights for policy 0, policy_version 95497 (0.0010) [2023-12-27 06:19:14,600][20347] Updated weights for policy 0, policy_version 95507 (0.0010) [2023-12-27 06:19:14,659][20347] Updated weights for policy 0, policy_version 95517 (0.0007) [2023-12-27 06:19:14,719][20347] Updated weights for policy 0, policy_version 95527 (0.0008) [2023-12-27 06:19:14,754][20381] Updated weights for policy 1, policy_version 95174 (0.0007) [2023-12-27 06:19:14,815][20381] Updated weights for policy 1, policy_version 95184 (0.0006) [2023-12-27 06:19:14,872][20381] Updated weights for policy 1, policy_version 95194 (0.0008) [2023-12-27 06:19:15,310][20347] Updated weights for policy 0, policy_version 95537 (0.0006) [2023-12-27 06:19:15,366][20347] Updated weights for policy 0, policy_version 95547 (0.0005) [2023-12-27 06:19:15,423][20347] Updated weights for policy 0, policy_version 95557 (0.0009) [2023-12-27 06:19:15,613][20381] Updated weights for policy 1, policy_version 95204 (0.0009) [2023-12-27 06:19:15,659][20381] Updated weights for policy 1, policy_version 95214 (0.0010) [2023-12-27 06:19:15,723][20381] Updated weights for policy 1, policy_version 95224 (0.0008) [2023-12-27 06:19:16,036][20347] Updated weights for policy 0, policy_version 95567 (0.0010) [2023-12-27 06:19:16,089][20347] Updated weights for policy 0, policy_version 95577 (0.0010) [2023-12-27 06:19:16,150][20347] Updated weights for policy 0, policy_version 95587 (0.0010) [2023-12-27 06:19:16,361][20381] Updated weights for policy 1, policy_version 95234 (0.0010) [2023-12-27 06:19:16,418][20381] Updated weights for policy 1, policy_version 95244 (0.0005) [2023-12-27 06:19:16,479][20381] Updated weights for policy 1, policy_version 95254 (0.0007) [2023-12-27 06:19:16,533][20381] Updated weights for policy 1, policy_version 95264 (0.0007) [2023-12-27 06:19:16,812][20347] Updated weights for policy 0, policy_version 95597 (0.0009) [2023-12-27 06:19:16,862][20347] Updated weights for policy 0, policy_version 95607 (0.0008) [2023-12-27 06:19:16,922][20347] Updated weights for policy 0, policy_version 95617 (0.0008) [2023-12-27 06:19:17,106][20381] Updated weights for policy 1, policy_version 95274 (0.0007) [2023-12-27 06:19:17,161][20381] Updated weights for policy 1, policy_version 95284 (0.0008) [2023-12-27 06:19:17,220][20381] Updated weights for policy 1, policy_version 95294 (0.0007) [2023-12-27 06:19:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 48898048. Throughput: 0: 10608.3, 1: 10330.0. Samples: 48905140. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:17,363][19170] Avg episode reward: [(0, '283.460'), (1, '181.950')] [2023-12-27 06:19:17,667][20347] Updated weights for policy 0, policy_version 95627 (0.0007) [2023-12-27 06:19:17,715][20347] Updated weights for policy 0, policy_version 95637 (0.0007) [2023-12-27 06:19:17,761][20347] Updated weights for policy 0, policy_version 95647 (0.0008) [2023-12-27 06:19:17,837][20381] Updated weights for policy 1, policy_version 95304 (0.0007) [2023-12-27 06:19:17,886][20381] Updated weights for policy 1, policy_version 95314 (0.0006) [2023-12-27 06:19:17,938][20381] Updated weights for policy 1, policy_version 95324 (0.0005) [2023-12-27 06:19:18,455][20347] Updated weights for policy 0, policy_version 95657 (0.0007) [2023-12-27 06:19:18,511][20347] Updated weights for policy 0, policy_version 95667 (0.0008) [2023-12-27 06:19:18,574][20347] Updated weights for policy 0, policy_version 95677 (0.0008) [2023-12-27 06:19:18,613][20381] Updated weights for policy 1, policy_version 95334 (0.0006) [2023-12-27 06:19:18,635][20347] Updated weights for policy 0, policy_version 95687 (0.0005) [2023-12-27 06:19:18,667][20381] Updated weights for policy 1, policy_version 95344 (0.0006) [2023-12-27 06:19:18,719][20381] Updated weights for policy 1, policy_version 95354 (0.0005) [2023-12-27 06:19:19,278][20347] Updated weights for policy 0, policy_version 95697 (0.0006) [2023-12-27 06:19:19,334][20347] Updated weights for policy 0, policy_version 95707 (0.0006) [2023-12-27 06:19:19,369][20381] Updated weights for policy 1, policy_version 95364 (0.0006) [2023-12-27 06:19:19,397][20347] Updated weights for policy 0, policy_version 95717 (0.0007) [2023-12-27 06:19:19,423][20381] Updated weights for policy 1, policy_version 95374 (0.0008) [2023-12-27 06:19:19,474][20381] Updated weights for policy 1, policy_version 95384 (0.0008) [2023-12-27 06:19:20,104][20347] Updated weights for policy 0, policy_version 95727 (0.0007) [2023-12-27 06:19:20,157][20347] Updated weights for policy 0, policy_version 95737 (0.0007) [2023-12-27 06:19:20,184][20381] Updated weights for policy 1, policy_version 95394 (0.0008) [2023-12-27 06:19:20,212][20347] Updated weights for policy 0, policy_version 95747 (0.0008) [2023-12-27 06:19:20,244][20381] Updated weights for policy 1, policy_version 95404 (0.0007) [2023-12-27 06:19:20,294][20381] Updated weights for policy 1, policy_version 95414 (0.0007) [2023-12-27 06:19:20,341][20381] Updated weights for policy 1, policy_version 95424 (0.0006) [2023-12-27 06:19:20,955][20347] Updated weights for policy 0, policy_version 95757 (0.0007) [2023-12-27 06:19:21,011][20347] Updated weights for policy 0, policy_version 95767 (0.0008) [2023-12-27 06:19:21,072][20347] Updated weights for policy 0, policy_version 95777 (0.0008) [2023-12-27 06:19:21,077][20381] Updated weights for policy 1, policy_version 95434 (0.0007) [2023-12-27 06:19:21,135][20381] Updated weights for policy 1, policy_version 95444 (0.0008) [2023-12-27 06:19:21,191][20381] Updated weights for policy 1, policy_version 95454 (0.0008) [2023-12-27 06:19:21,781][20347] Updated weights for policy 0, policy_version 95787 (0.0006) [2023-12-27 06:19:21,844][20347] Updated weights for policy 0, policy_version 95797 (0.0008) [2023-12-27 06:19:21,902][20381] Updated weights for policy 1, policy_version 95464 (0.0007) [2023-12-27 06:19:21,904][20347] Updated weights for policy 0, policy_version 95807 (0.0008) [2023-12-27 06:19:21,961][20381] Updated weights for policy 1, policy_version 95474 (0.0007) [2023-12-27 06:19:22,017][20381] Updated weights for policy 1, policy_version 95484 (0.0008) [2023-12-27 06:19:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 48996352. Throughput: 0: 10538.0, 1: 10330.7. Samples: 48967620. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:22,363][19170] Avg episode reward: [(0, '333.980'), (1, '284.970')] [2023-12-27 06:19:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000095488_24453120.pth... [2023-12-27 06:19:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000095816_24543232.pth... [2023-12-27 06:19:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000094304_24150016.pth [2023-12-27 06:19:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000094600_24231936.pth [2023-12-27 06:19:22,611][20347] Updated weights for policy 0, policy_version 95817 (0.0006) [2023-12-27 06:19:22,672][20347] Updated weights for policy 0, policy_version 95827 (0.0007) [2023-12-27 06:19:22,709][20381] Updated weights for policy 1, policy_version 95494 (0.0006) [2023-12-27 06:19:22,726][20347] Updated weights for policy 0, policy_version 95837 (0.0009) [2023-12-27 06:19:22,758][20381] Updated weights for policy 1, policy_version 95504 (0.0006) [2023-12-27 06:19:22,773][20347] Updated weights for policy 0, policy_version 95847 (0.0011) [2023-12-27 06:19:22,813][20381] Updated weights for policy 1, policy_version 95514 (0.0008) [2023-12-27 06:19:23,463][20347] Updated weights for policy 0, policy_version 95857 (0.0006) [2023-12-27 06:19:23,475][20381] Updated weights for policy 1, policy_version 95524 (0.0008) [2023-12-27 06:19:23,512][20347] Updated weights for policy 0, policy_version 95867 (0.0005) [2023-12-27 06:19:23,523][20381] Updated weights for policy 1, policy_version 95534 (0.0007) [2023-12-27 06:19:23,565][20347] Updated weights for policy 0, policy_version 95877 (0.0009) [2023-12-27 06:19:23,577][20381] Updated weights for policy 1, policy_version 95544 (0.0009) [2023-12-27 06:19:24,182][20347] Updated weights for policy 0, policy_version 95887 (0.0010) [2023-12-27 06:19:24,226][20381] Updated weights for policy 1, policy_version 95554 (0.0007) [2023-12-27 06:19:24,232][20347] Updated weights for policy 0, policy_version 95897 (0.0007) [2023-12-27 06:19:24,282][20381] Updated weights for policy 1, policy_version 95564 (0.0006) [2023-12-27 06:19:24,287][20347] Updated weights for policy 0, policy_version 95907 (0.0006) [2023-12-27 06:19:24,337][20381] Updated weights for policy 1, policy_version 95574 (0.0006) [2023-12-27 06:19:24,392][20381] Updated weights for policy 1, policy_version 95584 (0.0007) [2023-12-27 06:19:24,993][20347] Updated weights for policy 0, policy_version 95917 (0.0006) [2023-12-27 06:19:25,010][20381] Updated weights for policy 1, policy_version 95594 (0.0006) [2023-12-27 06:19:25,046][20347] Updated weights for policy 0, policy_version 95927 (0.0007) [2023-12-27 06:19:25,070][20381] Updated weights for policy 1, policy_version 95604 (0.0009) [2023-12-27 06:19:25,092][20347] Updated weights for policy 0, policy_version 95937 (0.0008) [2023-12-27 06:19:25,121][20381] Updated weights for policy 1, policy_version 95614 (0.0007) [2023-12-27 06:19:25,770][20347] Updated weights for policy 0, policy_version 95947 (0.0006) [2023-12-27 06:19:25,777][20381] Updated weights for policy 1, policy_version 95624 (0.0010) [2023-12-27 06:19:25,824][20347] Updated weights for policy 0, policy_version 95957 (0.0006) [2023-12-27 06:19:25,826][20381] Updated weights for policy 1, policy_version 95634 (0.0010) [2023-12-27 06:19:25,876][20347] Updated weights for policy 0, policy_version 95967 (0.0005) [2023-12-27 06:19:25,881][20381] Updated weights for policy 1, policy_version 95644 (0.0010) [2023-12-27 06:19:26,464][20381] Updated weights for policy 1, policy_version 95654 (0.0010) [2023-12-27 06:19:26,519][20381] Updated weights for policy 1, policy_version 95664 (0.0008) [2023-12-27 06:19:26,559][20347] Updated weights for policy 0, policy_version 95977 (0.0007) [2023-12-27 06:19:26,575][20381] Updated weights for policy 1, policy_version 95674 (0.0006) [2023-12-27 06:19:26,614][20347] Updated weights for policy 0, policy_version 95987 (0.0010) [2023-12-27 06:19:26,674][20347] Updated weights for policy 0, policy_version 95997 (0.0006) [2023-12-27 06:19:26,735][20347] Updated weights for policy 0, policy_version 96007 (0.0007) [2023-12-27 06:19:27,191][20381] Updated weights for policy 1, policy_version 95684 (0.0007) [2023-12-27 06:19:27,252][20381] Updated weights for policy 1, policy_version 95694 (0.0009) [2023-12-27 06:19:27,301][20381] Updated weights for policy 1, policy_version 95704 (0.0008) [2023-12-27 06:19:27,342][20347] Updated weights for policy 0, policy_version 96017 (0.0008) [2023-12-27 06:19:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 49102848. Throughput: 0: 10507.7, 1: 10343.0. Samples: 49090044. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:27,363][19170] Avg episode reward: [(0, '673.570'), (1, '254.490')] [2023-12-27 06:19:27,395][20347] Updated weights for policy 0, policy_version 96027 (0.0009) [2023-12-27 06:19:27,452][20347] Updated weights for policy 0, policy_version 96037 (0.0007) [2023-12-27 06:19:27,947][20381] Updated weights for policy 1, policy_version 95714 (0.0006) [2023-12-27 06:19:27,994][20381] Updated weights for policy 1, policy_version 95724 (0.0005) [2023-12-27 06:19:28,038][20381] Updated weights for policy 1, policy_version 95734 (0.0005) [2023-12-27 06:19:28,099][20381] Updated weights for policy 1, policy_version 95744 (0.0007) [2023-12-27 06:19:28,106][20347] Updated weights for policy 0, policy_version 96047 (0.0006) [2023-12-27 06:19:28,161][20347] Updated weights for policy 0, policy_version 96057 (0.0008) [2023-12-27 06:19:28,217][20347] Updated weights for policy 0, policy_version 96067 (0.0007) [2023-12-27 06:19:28,801][20381] Updated weights for policy 1, policy_version 95754 (0.0011) [2023-12-27 06:19:28,861][20381] Updated weights for policy 1, policy_version 95764 (0.0011) [2023-12-27 06:19:28,905][20347] Updated weights for policy 0, policy_version 96077 (0.0007) [2023-12-27 06:19:28,915][20381] Updated weights for policy 1, policy_version 95774 (0.0008) [2023-12-27 06:19:28,965][20347] Updated weights for policy 0, policy_version 96087 (0.0009) [2023-12-27 06:19:29,017][20347] Updated weights for policy 0, policy_version 96097 (0.0009) [2023-12-27 06:19:29,499][20381] Updated weights for policy 1, policy_version 95784 (0.0007) [2023-12-27 06:19:29,556][20381] Updated weights for policy 1, policy_version 95794 (0.0007) [2023-12-27 06:19:29,614][20381] Updated weights for policy 1, policy_version 95804 (0.0008) [2023-12-27 06:19:29,739][20347] Updated weights for policy 0, policy_version 96108 (0.0008) [2023-12-27 06:19:29,788][20347] Updated weights for policy 0, policy_version 96118 (0.0008) [2023-12-27 06:19:29,851][20347] Updated weights for policy 0, policy_version 96128 (0.0008) [2023-12-27 06:19:30,315][20381] Updated weights for policy 1, policy_version 95814 (0.0007) [2023-12-27 06:19:30,362][20381] Updated weights for policy 1, policy_version 95824 (0.0007) [2023-12-27 06:19:30,414][20381] Updated weights for policy 1, policy_version 95834 (0.0007) [2023-12-27 06:19:30,565][20347] Updated weights for policy 0, policy_version 96138 (0.0008) [2023-12-27 06:19:30,616][20347] Updated weights for policy 0, policy_version 96148 (0.0005) [2023-12-27 06:19:30,664][20347] Updated weights for policy 0, policy_version 96158 (0.0007) [2023-12-27 06:19:30,707][20347] Updated weights for policy 0, policy_version 96168 (0.0007) [2023-12-27 06:19:31,163][20381] Updated weights for policy 1, policy_version 95844 (0.0008) [2023-12-27 06:19:31,218][20381] Updated weights for policy 1, policy_version 95854 (0.0007) [2023-12-27 06:19:31,274][20381] Updated weights for policy 1, policy_version 95864 (0.0008) [2023-12-27 06:19:31,351][20347] Updated weights for policy 0, policy_version 96178 (0.0007) [2023-12-27 06:19:31,409][20347] Updated weights for policy 0, policy_version 96188 (0.0008) [2023-12-27 06:19:31,464][20347] Updated weights for policy 0, policy_version 96198 (0.0008) [2023-12-27 06:19:31,979][20381] Updated weights for policy 1, policy_version 95874 (0.0007) [2023-12-27 06:19:32,030][20381] Updated weights for policy 1, policy_version 95884 (0.0010) [2023-12-27 06:19:32,082][20381] Updated weights for policy 1, policy_version 95894 (0.0007) [2023-12-27 06:19:32,144][20381] Updated weights for policy 1, policy_version 95904 (0.0008) [2023-12-27 06:19:32,175][20347] Updated weights for policy 0, policy_version 96208 (0.0008) [2023-12-27 06:19:32,226][20347] Updated weights for policy 0, policy_version 96218 (0.0007) [2023-12-27 06:19:32,284][20347] Updated weights for policy 0, policy_version 96228 (0.0007) [2023-12-27 06:19:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 49209344. Throughput: 0: 10502.3, 1: 10270.5. Samples: 49214508. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:32,363][19170] Avg episode reward: [(0, '931.110'), (1, '228.890')] [2023-12-27 06:19:32,819][20381] Updated weights for policy 1, policy_version 95914 (0.0010) [2023-12-27 06:19:32,869][20381] Updated weights for policy 1, policy_version 95924 (0.0007) [2023-12-27 06:19:32,919][20381] Updated weights for policy 1, policy_version 95934 (0.0005) [2023-12-27 06:19:32,990][20347] Updated weights for policy 0, policy_version 96238 (0.0009) [2023-12-27 06:19:33,051][20347] Updated weights for policy 0, policy_version 96248 (0.0009) [2023-12-27 06:19:33,109][20347] Updated weights for policy 0, policy_version 96258 (0.0006) [2023-12-27 06:19:33,530][20381] Updated weights for policy 1, policy_version 95944 (0.0010) [2023-12-27 06:19:33,577][20381] Updated weights for policy 1, policy_version 95954 (0.0010) [2023-12-27 06:19:33,628][20381] Updated weights for policy 1, policy_version 95964 (0.0010) [2023-12-27 06:19:33,678][20347] Updated weights for policy 0, policy_version 96268 (0.0010) [2023-12-27 06:19:33,722][20347] Updated weights for policy 0, policy_version 96278 (0.0010) [2023-12-27 06:19:33,778][20347] Updated weights for policy 0, policy_version 96288 (0.0010) [2023-12-27 06:19:34,324][20381] Updated weights for policy 1, policy_version 95974 (0.0009) [2023-12-27 06:19:34,408][20381] Updated weights for policy 1, policy_version 95984 (0.0008) [2023-12-27 06:19:34,467][20381] Updated weights for policy 1, policy_version 95994 (0.0007) [2023-12-27 06:19:34,486][20347] Updated weights for policy 0, policy_version 96298 (0.0009) [2023-12-27 06:19:34,538][20347] Updated weights for policy 0, policy_version 96308 (0.0005) [2023-12-27 06:19:34,598][20347] Updated weights for policy 0, policy_version 96318 (0.0006) [2023-12-27 06:19:34,647][20347] Updated weights for policy 0, policy_version 96328 (0.0006) [2023-12-27 06:19:35,122][20381] Updated weights for policy 1, policy_version 96004 (0.0007) [2023-12-27 06:19:35,182][20381] Updated weights for policy 1, policy_version 96014 (0.0005) [2023-12-27 06:19:35,218][20347] Updated weights for policy 0, policy_version 96338 (0.0005) [2023-12-27 06:19:35,243][20381] Updated weights for policy 1, policy_version 96024 (0.0005) [2023-12-27 06:19:35,271][20347] Updated weights for policy 0, policy_version 96348 (0.0005) [2023-12-27 06:19:35,319][20347] Updated weights for policy 0, policy_version 96358 (0.0007) [2023-12-27 06:19:35,750][20381] Updated weights for policy 1, policy_version 96034 (0.0006) [2023-12-27 06:19:35,813][20381] Updated weights for policy 1, policy_version 96044 (0.0006) [2023-12-27 06:19:35,869][20381] Updated weights for policy 1, policy_version 96054 (0.0009) [2023-12-27 06:19:35,882][20347] Updated weights for policy 0, policy_version 96368 (0.0007) [2023-12-27 06:19:35,922][20381] Updated weights for policy 1, policy_version 96064 (0.0006) [2023-12-27 06:19:35,938][20347] Updated weights for policy 0, policy_version 96378 (0.0006) [2023-12-27 06:19:35,993][20347] Updated weights for policy 0, policy_version 96388 (0.0008) [2023-12-27 06:19:36,564][20381] Updated weights for policy 1, policy_version 96074 (0.0007) [2023-12-27 06:19:36,614][20381] Updated weights for policy 1, policy_version 96084 (0.0008) [2023-12-27 06:19:36,675][20381] Updated weights for policy 1, policy_version 96094 (0.0007) [2023-12-27 06:19:36,694][20347] Updated weights for policy 0, policy_version 96398 (0.0008) [2023-12-27 06:19:36,758][20347] Updated weights for policy 0, policy_version 96408 (0.0007) [2023-12-27 06:19:36,819][20347] Updated weights for policy 0, policy_version 96418 (0.0007) [2023-12-27 06:19:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 49307648. Throughput: 0: 10500.1, 1: 10301.1. Samples: 49278664. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:37,363][19170] Avg episode reward: [(0, '991.450'), (1, '328.170')] [2023-12-27 06:19:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000096424_24698880.pth... [2023-12-27 06:19:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000095208_24387584.pth [2023-12-27 06:19:37,403][20381] Updated weights for policy 1, policy_version 96104 (0.0007) [2023-12-27 06:19:37,430][20347] Updated weights for policy 0, policy_version 96428 (0.0008) [2023-12-27 06:19:37,466][20381] Updated weights for policy 1, policy_version 96114 (0.0006) [2023-12-27 06:19:37,476][20347] Updated weights for policy 0, policy_version 96438 (0.0005) [2023-12-27 06:19:37,529][20381] Updated weights for policy 1, policy_version 96124 (0.0007) [2023-12-27 06:19:37,534][20347] Updated weights for policy 0, policy_version 96448 (0.0008) [2023-12-27 06:19:37,551][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000096128_24616960.pth... [2023-12-27 06:19:37,556][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000094912_24305664.pth [2023-12-27 06:19:38,150][20347] Updated weights for policy 0, policy_version 96458 (0.0008) [2023-12-27 06:19:38,212][20347] Updated weights for policy 0, policy_version 96468 (0.0006) [2023-12-27 06:19:38,241][20381] Updated weights for policy 1, policy_version 96134 (0.0006) [2023-12-27 06:19:38,272][20347] Updated weights for policy 0, policy_version 96478 (0.0010) [2023-12-27 06:19:38,292][20381] Updated weights for policy 1, policy_version 96144 (0.0005) [2023-12-27 06:19:38,337][20347] Updated weights for policy 0, policy_version 96488 (0.0009) [2023-12-27 06:19:38,360][20381] Updated weights for policy 1, policy_version 96154 (0.0007) [2023-12-27 06:19:38,943][20347] Updated weights for policy 0, policy_version 96498 (0.0007) [2023-12-27 06:19:38,988][20381] Updated weights for policy 1, policy_version 96164 (0.0005) [2023-12-27 06:19:38,990][20347] Updated weights for policy 0, policy_version 96508 (0.0007) [2023-12-27 06:19:39,033][20381] Updated weights for policy 1, policy_version 96174 (0.0006) [2023-12-27 06:19:39,036][20347] Updated weights for policy 0, policy_version 96518 (0.0006) [2023-12-27 06:19:39,089][20381] Updated weights for policy 1, policy_version 96184 (0.0007) [2023-12-27 06:19:39,742][20381] Updated weights for policy 1, policy_version 96194 (0.0008) [2023-12-27 06:19:39,789][20347] Updated weights for policy 0, policy_version 96528 (0.0006) [2023-12-27 06:19:39,808][20381] Updated weights for policy 1, policy_version 96204 (0.0008) [2023-12-27 06:19:39,853][20347] Updated weights for policy 0, policy_version 96538 (0.0008) [2023-12-27 06:19:39,871][20381] Updated weights for policy 1, policy_version 96214 (0.0008) [2023-12-27 06:19:39,917][20347] Updated weights for policy 0, policy_version 96548 (0.0006) [2023-12-27 06:19:39,927][20381] Updated weights for policy 1, policy_version 96224 (0.0007) [2023-12-27 06:19:40,569][20381] Updated weights for policy 1, policy_version 96234 (0.0005) [2023-12-27 06:19:40,605][20347] Updated weights for policy 0, policy_version 96558 (0.0009) [2023-12-27 06:19:40,619][20381] Updated weights for policy 1, policy_version 96244 (0.0006) [2023-12-27 06:19:40,658][20347] Updated weights for policy 0, policy_version 96568 (0.0010) [2023-12-27 06:19:40,673][20381] Updated weights for policy 1, policy_version 96254 (0.0005) [2023-12-27 06:19:40,713][20347] Updated weights for policy 0, policy_version 96578 (0.0010) [2023-12-27 06:19:41,349][20381] Updated weights for policy 1, policy_version 96264 (0.0010) [2023-12-27 06:19:41,364][20347] Updated weights for policy 0, policy_version 96588 (0.0007) [2023-12-27 06:19:41,421][20381] Updated weights for policy 1, policy_version 96274 (0.0010) [2023-12-27 06:19:41,435][20347] Updated weights for policy 0, policy_version 96598 (0.0006) [2023-12-27 06:19:41,481][20381] Updated weights for policy 1, policy_version 96284 (0.0008) [2023-12-27 06:19:41,494][20347] Updated weights for policy 0, policy_version 96608 (0.0006) [2023-12-27 06:19:42,167][20381] Updated weights for policy 1, policy_version 96294 (0.0010) [2023-12-27 06:19:42,215][20347] Updated weights for policy 0, policy_version 96618 (0.0008) [2023-12-27 06:19:42,226][20381] Updated weights for policy 1, policy_version 96304 (0.0011) [2023-12-27 06:19:42,279][20347] Updated weights for policy 0, policy_version 96628 (0.0008) [2023-12-27 06:19:42,283][20381] Updated weights for policy 1, policy_version 96314 (0.0011) [2023-12-27 06:19:42,343][20347] Updated weights for policy 0, policy_version 96638 (0.0006) [2023-12-27 06:19:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 49414144. Throughput: 0: 10481.6, 1: 10351.5. Samples: 49405992. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:42,363][19170] Avg episode reward: [(0, '998.640'), (1, '597.320')] [2023-12-27 06:19:42,417][20347] Updated weights for policy 0, policy_version 96648 (0.0008) [2023-12-27 06:19:42,976][20381] Updated weights for policy 1, policy_version 96324 (0.0011) [2023-12-27 06:19:42,998][20347] Updated weights for policy 0, policy_version 96658 (0.0005) [2023-12-27 06:19:43,026][20381] Updated weights for policy 1, policy_version 96334 (0.0006) [2023-12-27 06:19:43,045][20347] Updated weights for policy 0, policy_version 96668 (0.0007) [2023-12-27 06:19:43,070][20381] Updated weights for policy 1, policy_version 96344 (0.0006) [2023-12-27 06:19:43,091][20347] Updated weights for policy 0, policy_version 96678 (0.0007) [2023-12-27 06:19:43,629][20381] Updated weights for policy 1, policy_version 96354 (0.0006) [2023-12-27 06:19:43,679][20381] Updated weights for policy 1, policy_version 96364 (0.0008) [2023-12-27 06:19:43,728][20381] Updated weights for policy 1, policy_version 96374 (0.0007) [2023-12-27 06:19:43,769][20347] Updated weights for policy 0, policy_version 96688 (0.0005) [2023-12-27 06:19:43,775][20381] Updated weights for policy 1, policy_version 96384 (0.0006) [2023-12-27 06:19:43,826][20347] Updated weights for policy 0, policy_version 96698 (0.0007) [2023-12-27 06:19:43,873][20347] Updated weights for policy 0, policy_version 96708 (0.0007) [2023-12-27 06:19:44,474][20381] Updated weights for policy 1, policy_version 96394 (0.0007) [2023-12-27 06:19:44,527][20347] Updated weights for policy 0, policy_version 96718 (0.0009) [2023-12-27 06:19:44,529][20381] Updated weights for policy 1, policy_version 96404 (0.0007) [2023-12-27 06:19:44,576][20381] Updated weights for policy 1, policy_version 96414 (0.0006) [2023-12-27 06:19:44,578][20347] Updated weights for policy 0, policy_version 96728 (0.0008) [2023-12-27 06:19:44,637][20347] Updated weights for policy 0, policy_version 96738 (0.0008) [2023-12-27 06:19:45,297][20381] Updated weights for policy 1, policy_version 96424 (0.0008) [2023-12-27 06:19:45,354][20347] Updated weights for policy 0, policy_version 96748 (0.0009) [2023-12-27 06:19:45,356][20381] Updated weights for policy 1, policy_version 96435 (0.0009) [2023-12-27 06:19:45,399][20347] Updated weights for policy 0, policy_version 96758 (0.0007) [2023-12-27 06:19:45,418][20381] Updated weights for policy 1, policy_version 96445 (0.0005) [2023-12-27 06:19:45,445][20347] Updated weights for policy 0, policy_version 96768 (0.0007) [2023-12-27 06:19:45,987][20381] Updated weights for policy 1, policy_version 96455 (0.0005) [2023-12-27 06:19:46,046][20381] Updated weights for policy 1, policy_version 96465 (0.0006) [2023-12-27 06:19:46,085][20347] Updated weights for policy 0, policy_version 96778 (0.0009) [2023-12-27 06:19:46,098][20381] Updated weights for policy 1, policy_version 96475 (0.0006) [2023-12-27 06:19:46,149][20347] Updated weights for policy 0, policy_version 96788 (0.0010) [2023-12-27 06:19:46,209][20347] Updated weights for policy 0, policy_version 96798 (0.0010) [2023-12-27 06:19:46,269][20347] Updated weights for policy 0, policy_version 96808 (0.0010) [2023-12-27 06:19:46,728][20381] Updated weights for policy 1, policy_version 96485 (0.0008) [2023-12-27 06:19:46,789][20381] Updated weights for policy 1, policy_version 96495 (0.0010) [2023-12-27 06:19:46,854][20381] Updated weights for policy 1, policy_version 96505 (0.0010) [2023-12-27 06:19:46,870][20347] Updated weights for policy 0, policy_version 96818 (0.0006) [2023-12-27 06:19:46,914][20347] Updated weights for policy 0, policy_version 96828 (0.0006) [2023-12-27 06:19:46,964][20347] Updated weights for policy 0, policy_version 96838 (0.0007) [2023-12-27 06:19:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 49520640. Throughput: 0: 10473.1, 1: 10390.3. Samples: 49531504. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '333.480')] [2023-12-27 06:19:47,505][20381] Updated weights for policy 1, policy_version 96515 (0.0010) [2023-12-27 06:19:47,560][20381] Updated weights for policy 1, policy_version 96525 (0.0010) [2023-12-27 06:19:47,621][20381] Updated weights for policy 1, policy_version 96535 (0.0010) [2023-12-27 06:19:47,649][20347] Updated weights for policy 0, policy_version 96848 (0.0006) [2023-12-27 06:19:47,707][20347] Updated weights for policy 0, policy_version 96858 (0.0007) [2023-12-27 06:19:47,761][20347] Updated weights for policy 0, policy_version 96868 (0.0007) [2023-12-27 06:19:48,254][20381] Updated weights for policy 1, policy_version 96545 (0.0010) [2023-12-27 06:19:48,315][20381] Updated weights for policy 1, policy_version 96555 (0.0006) [2023-12-27 06:19:48,379][20381] Updated weights for policy 1, policy_version 96565 (0.0009) [2023-12-27 06:19:48,436][20381] Updated weights for policy 1, policy_version 96575 (0.0008) [2023-12-27 06:19:48,483][20347] Updated weights for policy 0, policy_version 96878 (0.0007) [2023-12-27 06:19:48,541][20347] Updated weights for policy 0, policy_version 96888 (0.0007) [2023-12-27 06:19:48,602][20347] Updated weights for policy 0, policy_version 96898 (0.0007) [2023-12-27 06:19:49,043][20381] Updated weights for policy 1, policy_version 96585 (0.0005) [2023-12-27 06:19:49,105][20381] Updated weights for policy 1, policy_version 96595 (0.0005) [2023-12-27 06:19:49,171][20381] Updated weights for policy 1, policy_version 96605 (0.0006) [2023-12-27 06:19:49,362][20347] Updated weights for policy 0, policy_version 96908 (0.0008) [2023-12-27 06:19:49,409][20347] Updated weights for policy 0, policy_version 96918 (0.0008) [2023-12-27 06:19:49,456][20347] Updated weights for policy 0, policy_version 96928 (0.0008) [2023-12-27 06:19:49,797][20381] Updated weights for policy 1, policy_version 96615 (0.0006) [2023-12-27 06:19:49,861][20381] Updated weights for policy 1, policy_version 96625 (0.0007) [2023-12-27 06:19:49,919][20381] Updated weights for policy 1, policy_version 96635 (0.0008) [2023-12-27 06:19:50,200][20347] Updated weights for policy 0, policy_version 96938 (0.0008) [2023-12-27 06:19:50,257][20347] Updated weights for policy 0, policy_version 96948 (0.0007) [2023-12-27 06:19:50,311][20347] Updated weights for policy 0, policy_version 96958 (0.0005) [2023-12-27 06:19:50,362][20347] Updated weights for policy 0, policy_version 96968 (0.0005) [2023-12-27 06:19:50,624][20381] Updated weights for policy 1, policy_version 96645 (0.0007) [2023-12-27 06:19:50,686][20381] Updated weights for policy 1, policy_version 96655 (0.0007) [2023-12-27 06:19:50,740][20381] Updated weights for policy 1, policy_version 96665 (0.0008) [2023-12-27 06:19:51,015][20347] Updated weights for policy 0, policy_version 96978 (0.0010) [2023-12-27 06:19:51,075][20347] Updated weights for policy 0, policy_version 96988 (0.0008) [2023-12-27 06:19:51,143][20347] Updated weights for policy 0, policy_version 96998 (0.0007) [2023-12-27 06:19:51,427][20381] Updated weights for policy 1, policy_version 96675 (0.0008) [2023-12-27 06:19:51,482][20381] Updated weights for policy 1, policy_version 96685 (0.0007) [2023-12-27 06:19:51,542][20381] Updated weights for policy 1, policy_version 96695 (0.0008) [2023-12-27 06:19:51,806][20347] Updated weights for policy 0, policy_version 97008 (0.0008) [2023-12-27 06:19:51,859][20347] Updated weights for policy 0, policy_version 97018 (0.0010) [2023-12-27 06:19:51,917][20347] Updated weights for policy 0, policy_version 97028 (0.0010) [2023-12-27 06:19:52,275][20381] Updated weights for policy 1, policy_version 96705 (0.0007) [2023-12-27 06:19:52,336][20381] Updated weights for policy 1, policy_version 96715 (0.0007) [2023-12-27 06:19:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 49618944. Throughput: 0: 10407.3, 1: 10400.8. Samples: 49592796. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '722.880')] [2023-12-27 06:19:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000097032_24854528.pth... [2023-12-27 06:19:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000095816_24543232.pth [2023-12-27 06:19:52,372][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000097032_24854528.pth [2023-12-27 06:19:52,400][20381] Updated weights for policy 1, policy_version 96725 (0.0007) [2023-12-27 06:19:52,450][20381] Updated weights for policy 1, policy_version 96735 (0.0007) [2023-12-27 06:19:52,453][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000096736_24772608.pth... [2023-12-27 06:19:52,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000095488_24453120.pth [2023-12-27 06:19:52,456][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000096736_24772608.pth [2023-12-27 06:19:52,603][20347] Updated weights for policy 0, policy_version 97038 (0.0007) [2023-12-27 06:19:52,654][20347] Updated weights for policy 0, policy_version 97048 (0.0006) [2023-12-27 06:19:52,704][20347] Updated weights for policy 0, policy_version 97058 (0.0010) [2023-12-27 06:19:53,166][20381] Updated weights for policy 1, policy_version 96745 (0.0007) [2023-12-27 06:19:53,222][20381] Updated weights for policy 1, policy_version 96755 (0.0007) [2023-12-27 06:19:53,281][20381] Updated weights for policy 1, policy_version 96765 (0.0006) [2023-12-27 06:19:53,380][20347] Updated weights for policy 0, policy_version 97068 (0.0010) [2023-12-27 06:19:53,440][20347] Updated weights for policy 0, policy_version 97078 (0.0010) [2023-12-27 06:19:53,495][20347] Updated weights for policy 0, policy_version 97088 (0.0010) [2023-12-27 06:19:53,819][20381] Updated weights for policy 1, policy_version 96775 (0.0006) [2023-12-27 06:19:53,878][20381] Updated weights for policy 1, policy_version 96785 (0.0005) [2023-12-27 06:19:53,926][20381] Updated weights for policy 1, policy_version 96795 (0.0005) [2023-12-27 06:19:54,167][20347] Updated weights for policy 0, policy_version 97098 (0.0010) [2023-12-27 06:19:54,221][20347] Updated weights for policy 0, policy_version 97108 (0.0010) [2023-12-27 06:19:54,269][20347] Updated weights for policy 0, policy_version 97118 (0.0010) [2023-12-27 06:19:54,323][20347] Updated weights for policy 0, policy_version 97128 (0.0010) [2023-12-27 06:19:54,533][20381] Updated weights for policy 1, policy_version 96805 (0.0006) [2023-12-27 06:19:54,588][20381] Updated weights for policy 1, policy_version 96815 (0.0008) [2023-12-27 06:19:54,647][20381] Updated weights for policy 1, policy_version 96825 (0.0008) [2023-12-27 06:19:55,002][20347] Updated weights for policy 0, policy_version 97138 (0.0010) [2023-12-27 06:19:55,054][20347] Updated weights for policy 0, policy_version 97148 (0.0010) [2023-12-27 06:19:55,103][20347] Updated weights for policy 0, policy_version 97158 (0.0007) [2023-12-27 06:19:55,302][20381] Updated weights for policy 1, policy_version 96835 (0.0008) [2023-12-27 06:19:55,356][20381] Updated weights for policy 1, policy_version 96845 (0.0006) [2023-12-27 06:19:55,415][20381] Updated weights for policy 1, policy_version 96855 (0.0007) [2023-12-27 06:19:55,785][20347] Updated weights for policy 0, policy_version 97168 (0.0010) [2023-12-27 06:19:55,837][20347] Updated weights for policy 0, policy_version 97178 (0.0010) [2023-12-27 06:19:55,898][20347] Updated weights for policy 0, policy_version 97188 (0.0010) [2023-12-27 06:19:56,096][20381] Updated weights for policy 1, policy_version 96865 (0.0007) [2023-12-27 06:19:56,150][20381] Updated weights for policy 1, policy_version 96875 (0.0007) [2023-12-27 06:19:56,196][20381] Updated weights for policy 1, policy_version 96885 (0.0006) [2023-12-27 06:19:56,246][20381] Updated weights for policy 1, policy_version 96895 (0.0007) [2023-12-27 06:19:56,554][20347] Updated weights for policy 0, policy_version 97198 (0.0008) [2023-12-27 06:19:56,601][20347] Updated weights for policy 0, policy_version 97208 (0.0009) [2023-12-27 06:19:56,651][20347] Updated weights for policy 0, policy_version 97218 (0.0010) [2023-12-27 06:19:57,013][20381] Updated weights for policy 1, policy_version 96905 (0.0008) [2023-12-27 06:19:57,064][20381] Updated weights for policy 1, policy_version 96915 (0.0009) [2023-12-27 06:19:57,114][20381] Updated weights for policy 1, policy_version 96925 (0.0008) [2023-12-27 06:19:57,210][20347] Updated weights for policy 0, policy_version 97228 (0.0005) [2023-12-27 06:19:57,256][20347] Updated weights for policy 0, policy_version 97238 (0.0005) [2023-12-27 06:19:57,316][20347] Updated weights for policy 0, policy_version 97248 (0.0006) [2023-12-27 06:19:57,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 49733632. Throughput: 0: 10370.5, 1: 10434.5. Samples: 49716244. Policy #0 lag: (min: 22.0, avg: 24.6, max: 54.0) [2023-12-27 06:19:57,364][19170] Avg episode reward: [(0, '952.530'), (1, '964.100')] [2023-12-27 06:19:57,739][20381] Updated weights for policy 1, policy_version 96935 (0.0007) [2023-12-27 06:19:57,802][20381] Updated weights for policy 1, policy_version 96945 (0.0005) [2023-12-27 06:19:57,850][20381] Updated weights for policy 1, policy_version 96955 (0.0005) [2023-12-27 06:19:57,937][20347] Updated weights for policy 0, policy_version 97258 (0.0007) [2023-12-27 06:19:57,991][20347] Updated weights for policy 0, policy_version 97268 (0.0010) [2023-12-27 06:19:58,046][20347] Updated weights for policy 0, policy_version 97278 (0.0005) [2023-12-27 06:19:58,096][20347] Updated weights for policy 0, policy_version 97288 (0.0005) [2023-12-27 06:19:58,531][20381] Updated weights for policy 1, policy_version 96965 (0.0006) [2023-12-27 06:19:58,593][20381] Updated weights for policy 1, policy_version 96975 (0.0008) [2023-12-27 06:19:58,656][20381] Updated weights for policy 1, policy_version 96985 (0.0008) [2023-12-27 06:19:58,786][20347] Updated weights for policy 0, policy_version 97298 (0.0008) [2023-12-27 06:19:58,855][20347] Updated weights for policy 0, policy_version 97308 (0.0009) [2023-12-27 06:19:58,919][20347] Updated weights for policy 0, policy_version 97318 (0.0007) [2023-12-27 06:19:59,468][20381] Updated weights for policy 1, policy_version 96995 (0.0007) [2023-12-27 06:19:59,513][20381] Updated weights for policy 1, policy_version 97005 (0.0007) [2023-12-27 06:19:59,564][20381] Updated weights for policy 1, policy_version 97015 (0.0009) [2023-12-27 06:19:59,636][20347] Updated weights for policy 0, policy_version 97328 (0.0006) [2023-12-27 06:19:59,694][20347] Updated weights for policy 0, policy_version 97338 (0.0006) [2023-12-27 06:19:59,744][20347] Updated weights for policy 0, policy_version 97348 (0.0007) [2023-12-27 06:20:00,255][20381] Updated weights for policy 1, policy_version 97025 (0.0008) [2023-12-27 06:20:00,302][20381] Updated weights for policy 1, policy_version 97035 (0.0005) [2023-12-27 06:20:00,359][20381] Updated weights for policy 1, policy_version 97045 (0.0005) [2023-12-27 06:20:00,408][20347] Updated weights for policy 0, policy_version 97358 (0.0010) [2023-12-27 06:20:00,417][20381] Updated weights for policy 1, policy_version 97055 (0.0005) [2023-12-27 06:20:00,456][20347] Updated weights for policy 0, policy_version 97368 (0.0010) [2023-12-27 06:20:00,513][20347] Updated weights for policy 0, policy_version 97378 (0.0010) [2023-12-27 06:20:00,981][20381] Updated weights for policy 1, policy_version 97065 (0.0005) [2023-12-27 06:20:01,036][20381] Updated weights for policy 1, policy_version 97075 (0.0006) [2023-12-27 06:20:01,097][20381] Updated weights for policy 1, policy_version 97085 (0.0007) [2023-12-27 06:20:01,203][20347] Updated weights for policy 0, policy_version 97388 (0.0010) [2023-12-27 06:20:01,267][20347] Updated weights for policy 0, policy_version 97398 (0.0010) [2023-12-27 06:20:01,328][20347] Updated weights for policy 0, policy_version 97408 (0.0010) [2023-12-27 06:20:01,710][20381] Updated weights for policy 1, policy_version 97095 (0.0006) [2023-12-27 06:20:01,773][20381] Updated weights for policy 1, policy_version 97105 (0.0008) [2023-12-27 06:20:01,819][20381] Updated weights for policy 1, policy_version 97115 (0.0008) [2023-12-27 06:20:02,006][20347] Updated weights for policy 0, policy_version 97418 (0.0007) [2023-12-27 06:20:02,062][20347] Updated weights for policy 0, policy_version 97428 (0.0007) [2023-12-27 06:20:02,114][20347] Updated weights for policy 0, policy_version 97438 (0.0008) [2023-12-27 06:20:02,163][20347] Updated weights for policy 0, policy_version 97448 (0.0007) [2023-12-27 06:20:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 49831936. Throughput: 0: 10329.1, 1: 10478.4. Samples: 49841476. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:02,363][19170] Avg episode reward: [(0, '942.960'), (1, '982.010')] [2023-12-27 06:20:02,532][20381] Updated weights for policy 1, policy_version 97125 (0.0009) [2023-12-27 06:20:02,592][20381] Updated weights for policy 1, policy_version 97135 (0.0009) [2023-12-27 06:20:02,643][20381] Updated weights for policy 1, policy_version 97145 (0.0010) [2023-12-27 06:20:02,791][20347] Updated weights for policy 0, policy_version 97458 (0.0008) [2023-12-27 06:20:02,847][20347] Updated weights for policy 0, policy_version 97468 (0.0006) [2023-12-27 06:20:02,898][20347] Updated weights for policy 0, policy_version 97478 (0.0008) [2023-12-27 06:20:03,303][20381] Updated weights for policy 1, policy_version 97155 (0.0007) [2023-12-27 06:20:03,357][20381] Updated weights for policy 1, policy_version 97165 (0.0007) [2023-12-27 06:20:03,415][20381] Updated weights for policy 1, policy_version 97175 (0.0006) [2023-12-27 06:20:03,520][20347] Updated weights for policy 0, policy_version 97488 (0.0006) [2023-12-27 06:20:03,580][20347] Updated weights for policy 0, policy_version 97498 (0.0006) [2023-12-27 06:20:03,634][20347] Updated weights for policy 0, policy_version 97508 (0.0007) [2023-12-27 06:20:04,081][20381] Updated weights for policy 1, policy_version 97185 (0.0007) [2023-12-27 06:20:04,143][20381] Updated weights for policy 1, policy_version 97195 (0.0008) [2023-12-27 06:20:04,204][20381] Updated weights for policy 1, policy_version 97205 (0.0006) [2023-12-27 06:20:04,266][20381] Updated weights for policy 1, policy_version 97215 (0.0006) [2023-12-27 06:20:04,319][20347] Updated weights for policy 0, policy_version 97518 (0.0008) [2023-12-27 06:20:04,373][20347] Updated weights for policy 0, policy_version 97528 (0.0008) [2023-12-27 06:20:04,427][20347] Updated weights for policy 0, policy_version 97538 (0.0009) [2023-12-27 06:20:04,896][20381] Updated weights for policy 1, policy_version 97225 (0.0006) [2023-12-27 06:20:04,956][20381] Updated weights for policy 1, policy_version 97235 (0.0005) [2023-12-27 06:20:05,009][20381] Updated weights for policy 1, policy_version 97245 (0.0005) [2023-12-27 06:20:05,063][20347] Updated weights for policy 0, policy_version 97548 (0.0007) [2023-12-27 06:20:05,111][20347] Updated weights for policy 0, policy_version 97558 (0.0010) [2023-12-27 06:20:05,163][20347] Updated weights for policy 0, policy_version 97568 (0.0010) [2023-12-27 06:20:05,513][20381] Updated weights for policy 1, policy_version 97255 (0.0007) [2023-12-27 06:20:05,566][20381] Updated weights for policy 1, policy_version 97265 (0.0008) [2023-12-27 06:20:05,616][20381] Updated weights for policy 1, policy_version 97275 (0.0007) [2023-12-27 06:20:05,821][20347] Updated weights for policy 0, policy_version 97578 (0.0011) [2023-12-27 06:20:05,879][20347] Updated weights for policy 0, policy_version 97588 (0.0010) [2023-12-27 06:20:05,939][20347] Updated weights for policy 0, policy_version 97598 (0.0010) [2023-12-27 06:20:05,990][20347] Updated weights for policy 0, policy_version 97608 (0.0010) [2023-12-27 06:20:06,314][20381] Updated weights for policy 1, policy_version 97285 (0.0007) [2023-12-27 06:20:06,368][20381] Updated weights for policy 1, policy_version 97295 (0.0005) [2023-12-27 06:20:06,427][20381] Updated weights for policy 1, policy_version 97305 (0.0006) [2023-12-27 06:20:06,686][20347] Updated weights for policy 0, policy_version 97618 (0.0010) [2023-12-27 06:20:06,747][20347] Updated weights for policy 0, policy_version 97628 (0.0010) [2023-12-27 06:20:06,813][20347] Updated weights for policy 0, policy_version 97638 (0.0010) [2023-12-27 06:20:07,085][20381] Updated weights for policy 1, policy_version 97315 (0.0007) [2023-12-27 06:20:07,147][20381] Updated weights for policy 1, policy_version 97325 (0.0008) [2023-12-27 06:20:07,209][20381] Updated weights for policy 1, policy_version 97335 (0.0008) [2023-12-27 06:20:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 49938432. Throughput: 0: 10365.2, 1: 10483.4. Samples: 49905812. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:07,363][19170] Avg episode reward: [(0, '972.920'), (1, '1000.000')] [2023-12-27 06:20:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000097640_25010176.pth... [2023-12-27 06:20:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000097344_24928256.pth... [2023-12-27 06:20:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000096424_24698880.pth [2023-12-27 06:20:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000096128_24616960.pth [2023-12-27 06:20:07,552][20347] Updated weights for policy 0, policy_version 97648 (0.0006) [2023-12-27 06:20:07,617][20347] Updated weights for policy 0, policy_version 97658 (0.0005) [2023-12-27 06:20:07,676][20347] Updated weights for policy 0, policy_version 97668 (0.0010) [2023-12-27 06:20:07,875][20381] Updated weights for policy 1, policy_version 97345 (0.0007) [2023-12-27 06:20:07,933][20381] Updated weights for policy 1, policy_version 97355 (0.0006) [2023-12-27 06:20:07,994][20381] Updated weights for policy 1, policy_version 97365 (0.0006) [2023-12-27 06:20:08,052][20381] Updated weights for policy 1, policy_version 97375 (0.0006) [2023-12-27 06:20:08,342][20347] Updated weights for policy 0, policy_version 97678 (0.0010) [2023-12-27 06:20:08,404][20347] Updated weights for policy 0, policy_version 97688 (0.0010) [2023-12-27 06:20:08,462][20347] Updated weights for policy 0, policy_version 97698 (0.0010) [2023-12-27 06:20:08,623][20381] Updated weights for policy 1, policy_version 97385 (0.0005) [2023-12-27 06:20:08,672][20381] Updated weights for policy 1, policy_version 97395 (0.0005) [2023-12-27 06:20:08,722][20381] Updated weights for policy 1, policy_version 97405 (0.0008) [2023-12-27 06:20:09,097][20347] Updated weights for policy 0, policy_version 97708 (0.0009) [2023-12-27 06:20:09,143][20347] Updated weights for policy 0, policy_version 97718 (0.0010) [2023-12-27 06:20:09,196][20347] Updated weights for policy 0, policy_version 97728 (0.0008) [2023-12-27 06:20:09,434][20381] Updated weights for policy 1, policy_version 97415 (0.0009) [2023-12-27 06:20:09,482][20381] Updated weights for policy 1, policy_version 97425 (0.0008) [2023-12-27 06:20:09,529][20381] Updated weights for policy 1, policy_version 97435 (0.0008) [2023-12-27 06:20:09,879][20347] Updated weights for policy 0, policy_version 97738 (0.0009) [2023-12-27 06:20:09,946][20347] Updated weights for policy 0, policy_version 97748 (0.0008) [2023-12-27 06:20:10,012][20347] Updated weights for policy 0, policy_version 97758 (0.0008) [2023-12-27 06:20:10,078][20347] Updated weights for policy 0, policy_version 97768 (0.0006) [2023-12-27 06:20:10,268][20381] Updated weights for policy 1, policy_version 97445 (0.0008) [2023-12-27 06:20:10,320][20381] Updated weights for policy 1, policy_version 97455 (0.0007) [2023-12-27 06:20:10,383][20381] Updated weights for policy 1, policy_version 97465 (0.0008) [2023-12-27 06:20:10,726][20347] Updated weights for policy 0, policy_version 97778 (0.0007) [2023-12-27 06:20:10,780][20347] Updated weights for policy 0, policy_version 97788 (0.0010) [2023-12-27 06:20:10,834][20347] Updated weights for policy 0, policy_version 97798 (0.0010) [2023-12-27 06:20:11,112][20381] Updated weights for policy 1, policy_version 97475 (0.0007) [2023-12-27 06:20:11,179][20381] Updated weights for policy 1, policy_version 97485 (0.0009) [2023-12-27 06:20:11,240][20381] Updated weights for policy 1, policy_version 97495 (0.0007) [2023-12-27 06:20:11,544][20347] Updated weights for policy 0, policy_version 97808 (0.0006) [2023-12-27 06:20:11,604][20347] Updated weights for policy 0, policy_version 97818 (0.0006) [2023-12-27 06:20:11,667][20347] Updated weights for policy 0, policy_version 97828 (0.0008) [2023-12-27 06:20:12,000][20381] Updated weights for policy 1, policy_version 97505 (0.0008) [2023-12-27 06:20:12,059][20381] Updated weights for policy 1, policy_version 97515 (0.0008) [2023-12-27 06:20:12,119][20381] Updated weights for policy 1, policy_version 97525 (0.0008) [2023-12-27 06:20:12,175][20381] Updated weights for policy 1, policy_version 97535 (0.0008) [2023-12-27 06:20:12,357][20347] Updated weights for policy 0, policy_version 97838 (0.0007) [2023-12-27 06:20:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 50036736. Throughput: 0: 10396.3, 1: 10461.8. Samples: 50028656. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:20:12,421][20347] Updated weights for policy 0, policy_version 97848 (0.0008) [2023-12-27 06:20:12,486][20347] Updated weights for policy 0, policy_version 97858 (0.0008) [2023-12-27 06:20:12,908][20381] Updated weights for policy 1, policy_version 97545 (0.0007) [2023-12-27 06:20:12,957][20381] Updated weights for policy 1, policy_version 97555 (0.0007) [2023-12-27 06:20:13,008][20381] Updated weights for policy 1, policy_version 97565 (0.0006) [2023-12-27 06:20:13,147][20347] Updated weights for policy 0, policy_version 97868 (0.0005) [2023-12-27 06:20:13,198][20347] Updated weights for policy 0, policy_version 97878 (0.0005) [2023-12-27 06:20:13,248][20347] Updated weights for policy 0, policy_version 97888 (0.0005) [2023-12-27 06:20:13,710][20381] Updated weights for policy 1, policy_version 97575 (0.0007) [2023-12-27 06:20:13,759][20381] Updated weights for policy 1, policy_version 97585 (0.0009) [2023-12-27 06:20:13,768][20347] Updated weights for policy 0, policy_version 97898 (0.0005) [2023-12-27 06:20:13,810][20381] Updated weights for policy 1, policy_version 97595 (0.0006) [2023-12-27 06:20:13,819][20347] Updated weights for policy 0, policy_version 97908 (0.0006) [2023-12-27 06:20:13,871][20347] Updated weights for policy 0, policy_version 97918 (0.0007) [2023-12-27 06:20:13,923][20347] Updated weights for policy 0, policy_version 97928 (0.0007) [2023-12-27 06:20:14,530][20381] Updated weights for policy 1, policy_version 97605 (0.0007) [2023-12-27 06:20:14,584][20381] Updated weights for policy 1, policy_version 97615 (0.0008) [2023-12-27 06:20:14,633][20347] Updated weights for policy 0, policy_version 97938 (0.0006) [2023-12-27 06:20:14,636][20381] Updated weights for policy 1, policy_version 97625 (0.0005) [2023-12-27 06:20:14,689][20347] Updated weights for policy 0, policy_version 97948 (0.0006) [2023-12-27 06:20:14,749][20347] Updated weights for policy 0, policy_version 97958 (0.0008) [2023-12-27 06:20:15,332][20381] Updated weights for policy 1, policy_version 97635 (0.0006) [2023-12-27 06:20:15,377][20381] Updated weights for policy 1, policy_version 97645 (0.0008) [2023-12-27 06:20:15,431][20381] Updated weights for policy 1, policy_version 97655 (0.0008) [2023-12-27 06:20:15,447][20347] Updated weights for policy 0, policy_version 97968 (0.0008) [2023-12-27 06:20:15,496][20347] Updated weights for policy 0, policy_version 97978 (0.0005) [2023-12-27 06:20:15,546][20347] Updated weights for policy 0, policy_version 97988 (0.0005) [2023-12-27 06:20:16,129][20347] Updated weights for policy 0, policy_version 97998 (0.0010) [2023-12-27 06:20:16,137][20381] Updated weights for policy 1, policy_version 97665 (0.0008) [2023-12-27 06:20:16,187][20347] Updated weights for policy 0, policy_version 98008 (0.0010) [2023-12-27 06:20:16,192][20381] Updated weights for policy 1, policy_version 97675 (0.0010) [2023-12-27 06:20:16,248][20347] Updated weights for policy 0, policy_version 98018 (0.0010) [2023-12-27 06:20:16,251][20381] Updated weights for policy 1, policy_version 97685 (0.0010) [2023-12-27 06:20:16,312][20381] Updated weights for policy 1, policy_version 97695 (0.0011) [2023-12-27 06:20:16,883][20347] Updated weights for policy 0, policy_version 98028 (0.0009) [2023-12-27 06:20:16,911][20381] Updated weights for policy 1, policy_version 97705 (0.0007) [2023-12-27 06:20:16,938][20347] Updated weights for policy 0, policy_version 98038 (0.0010) [2023-12-27 06:20:16,970][20381] Updated weights for policy 1, policy_version 97715 (0.0006) [2023-12-27 06:20:16,998][20347] Updated weights for policy 0, policy_version 98048 (0.0010) [2023-12-27 06:20:17,025][20381] Updated weights for policy 1, policy_version 97725 (0.0006) [2023-12-27 06:20:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 50143232. Throughput: 0: 10429.4, 1: 10408.3. Samples: 50152204. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:17,363][19170] Avg episode reward: [(0, '964.980'), (1, '1000.000')] [2023-12-27 06:20:17,540][20347] Updated weights for policy 0, policy_version 98058 (0.0009) [2023-12-27 06:20:17,601][20347] Updated weights for policy 0, policy_version 98068 (0.0006) [2023-12-27 06:20:17,650][20347] Updated weights for policy 0, policy_version 98078 (0.0010) [2023-12-27 06:20:17,695][20347] Updated weights for policy 0, policy_version 98088 (0.0008) [2023-12-27 06:20:17,698][20381] Updated weights for policy 1, policy_version 97735 (0.0006) [2023-12-27 06:20:17,758][20381] Updated weights for policy 1, policy_version 97745 (0.0009) [2023-12-27 06:20:17,818][20381] Updated weights for policy 1, policy_version 97755 (0.0005) [2023-12-27 06:20:18,298][20347] Updated weights for policy 0, policy_version 98098 (0.0006) [2023-12-27 06:20:18,354][20347] Updated weights for policy 0, policy_version 98108 (0.0007) [2023-12-27 06:20:18,408][20347] Updated weights for policy 0, policy_version 98118 (0.0008) [2023-12-27 06:20:18,438][20381] Updated weights for policy 1, policy_version 97765 (0.0007) [2023-12-27 06:20:18,498][20381] Updated weights for policy 1, policy_version 97775 (0.0008) [2023-12-27 06:20:18,560][20381] Updated weights for policy 1, policy_version 97785 (0.0008) [2023-12-27 06:20:19,071][20347] Updated weights for policy 0, policy_version 98128 (0.0007) [2023-12-27 06:20:19,130][20347] Updated weights for policy 0, policy_version 98138 (0.0006) [2023-12-27 06:20:19,189][20347] Updated weights for policy 0, policy_version 98148 (0.0006) [2023-12-27 06:20:19,292][20381] Updated weights for policy 1, policy_version 97795 (0.0007) [2023-12-27 06:20:19,358][20381] Updated weights for policy 1, policy_version 97805 (0.0008) [2023-12-27 06:20:19,419][20381] Updated weights for policy 1, policy_version 97815 (0.0008) [2023-12-27 06:20:19,847][20347] Updated weights for policy 0, policy_version 98158 (0.0008) [2023-12-27 06:20:19,907][20347] Updated weights for policy 0, policy_version 98168 (0.0010) [2023-12-27 06:20:19,972][20347] Updated weights for policy 0, policy_version 98178 (0.0008) [2023-12-27 06:20:20,157][20381] Updated weights for policy 1, policy_version 97825 (0.0008) [2023-12-27 06:20:20,215][20381] Updated weights for policy 1, policy_version 97835 (0.0008) [2023-12-27 06:20:20,277][20381] Updated weights for policy 1, policy_version 97845 (0.0007) [2023-12-27 06:20:20,333][20381] Updated weights for policy 1, policy_version 97855 (0.0008) [2023-12-27 06:20:20,635][20347] Updated weights for policy 0, policy_version 98188 (0.0010) [2023-12-27 06:20:20,689][20347] Updated weights for policy 0, policy_version 98198 (0.0010) [2023-12-27 06:20:20,747][20347] Updated weights for policy 0, policy_version 98208 (0.0010) [2023-12-27 06:20:21,074][20381] Updated weights for policy 1, policy_version 97865 (0.0008) [2023-12-27 06:20:21,137][20381] Updated weights for policy 1, policy_version 97875 (0.0008) [2023-12-27 06:20:21,201][20381] Updated weights for policy 1, policy_version 97885 (0.0008) [2023-12-27 06:20:21,350][20347] Updated weights for policy 0, policy_version 98218 (0.0006) [2023-12-27 06:20:21,423][20347] Updated weights for policy 0, policy_version 98228 (0.0011) [2023-12-27 06:20:21,488][20347] Updated weights for policy 0, policy_version 98238 (0.0010) [2023-12-27 06:20:21,548][20347] Updated weights for policy 0, policy_version 98248 (0.0009) [2023-12-27 06:20:21,947][20381] Updated weights for policy 1, policy_version 97895 (0.0006) [2023-12-27 06:20:22,013][20381] Updated weights for policy 1, policy_version 97905 (0.0007) [2023-12-27 06:20:22,075][20381] Updated weights for policy 1, policy_version 97915 (0.0006) [2023-12-27 06:20:22,249][20347] Updated weights for policy 0, policy_version 98258 (0.0007) [2023-12-27 06:20:22,306][20347] Updated weights for policy 0, policy_version 98268 (0.0007) [2023-12-27 06:20:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 50241536. Throughput: 0: 10441.4, 1: 10368.7. Samples: 50215116. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:22,363][19170] Avg episode reward: [(0, '845.010'), (1, '1000.000')] [2023-12-27 06:20:22,368][20347] Updated weights for policy 0, policy_version 98278 (0.0007) [2023-12-27 06:20:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000097920_25075712.pth... [2023-12-27 06:20:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000096736_24772608.pth [2023-12-27 06:20:22,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000098280_25174016.pth... [2023-12-27 06:20:22,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000097032_24854528.pth [2023-12-27 06:20:22,759][20381] Updated weights for policy 1, policy_version 97925 (0.0007) [2023-12-27 06:20:22,823][20381] Updated weights for policy 1, policy_version 97935 (0.0008) [2023-12-27 06:20:22,883][20381] Updated weights for policy 1, policy_version 97945 (0.0007) [2023-12-27 06:20:22,948][20347] Updated weights for policy 0, policy_version 98288 (0.0006) [2023-12-27 06:20:23,004][20347] Updated weights for policy 0, policy_version 98298 (0.0005) [2023-12-27 06:20:23,060][20347] Updated weights for policy 0, policy_version 98308 (0.0005) [2023-12-27 06:20:23,455][20381] Updated weights for policy 1, policy_version 97955 (0.0007) [2023-12-27 06:20:23,500][20381] Updated weights for policy 1, policy_version 97965 (0.0005) [2023-12-27 06:20:23,558][20381] Updated weights for policy 1, policy_version 97975 (0.0005) [2023-12-27 06:20:23,606][20347] Updated weights for policy 0, policy_version 98318 (0.0005) [2023-12-27 06:20:23,659][20347] Updated weights for policy 0, policy_version 98328 (0.0005) [2023-12-27 06:20:23,713][20347] Updated weights for policy 0, policy_version 98338 (0.0005) [2023-12-27 06:20:24,172][20381] Updated weights for policy 1, policy_version 97985 (0.0005) [2023-12-27 06:20:24,233][20381] Updated weights for policy 1, policy_version 97995 (0.0007) [2023-12-27 06:20:24,289][20381] Updated weights for policy 1, policy_version 98005 (0.0008) [2023-12-27 06:20:24,316][20347] Updated weights for policy 0, policy_version 98348 (0.0007) [2023-12-27 06:20:24,343][20381] Updated weights for policy 1, policy_version 98015 (0.0005) [2023-12-27 06:20:24,366][20347] Updated weights for policy 0, policy_version 98358 (0.0010) [2023-12-27 06:20:24,428][20347] Updated weights for policy 0, policy_version 98368 (0.0007) [2023-12-27 06:20:25,009][20381] Updated weights for policy 1, policy_version 98025 (0.0010) [2023-12-27 06:20:25,064][20381] Updated weights for policy 1, policy_version 98035 (0.0010) [2023-12-27 06:20:25,100][20347] Updated weights for policy 0, policy_version 98378 (0.0008) [2023-12-27 06:20:25,118][20381] Updated weights for policy 1, policy_version 98045 (0.0010) [2023-12-27 06:20:25,153][20347] Updated weights for policy 0, policy_version 98388 (0.0010) [2023-12-27 06:20:25,204][20347] Updated weights for policy 0, policy_version 98398 (0.0009) [2023-12-27 06:20:25,259][20347] Updated weights for policy 0, policy_version 98408 (0.0007) [2023-12-27 06:20:25,770][20381] Updated weights for policy 1, policy_version 98055 (0.0007) [2023-12-27 06:20:25,829][20381] Updated weights for policy 1, policy_version 98065 (0.0005) [2023-12-27 06:20:25,876][20381] Updated weights for policy 1, policy_version 98075 (0.0005) [2023-12-27 06:20:25,921][20347] Updated weights for policy 0, policy_version 98418 (0.0010) [2023-12-27 06:20:25,970][20347] Updated weights for policy 0, policy_version 98428 (0.0010) [2023-12-27 06:20:26,018][20347] Updated weights for policy 0, policy_version 98438 (0.0010) [2023-12-27 06:20:26,400][20381] Updated weights for policy 1, policy_version 98085 (0.0005) [2023-12-27 06:20:26,445][20381] Updated weights for policy 1, policy_version 98095 (0.0005) [2023-12-27 06:20:26,489][20381] Updated weights for policy 1, policy_version 98105 (0.0005) [2023-12-27 06:20:26,720][20347] Updated weights for policy 0, policy_version 98448 (0.0010) [2023-12-27 06:20:26,770][20347] Updated weights for policy 0, policy_version 98458 (0.0009) [2023-12-27 06:20:26,822][20347] Updated weights for policy 0, policy_version 98468 (0.0006) [2023-12-27 06:20:27,083][20381] Updated weights for policy 1, policy_version 98115 (0.0006) [2023-12-27 06:20:27,131][20381] Updated weights for policy 1, policy_version 98125 (0.0007) [2023-12-27 06:20:27,183][20381] Updated weights for policy 1, policy_version 98135 (0.0007) [2023-12-27 06:20:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 50356224. Throughput: 0: 10466.2, 1: 10356.6. Samples: 50343020. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:27,363][19170] Avg episode reward: [(0, '807.920'), (1, '1000.000')] [2023-12-27 06:20:27,467][20347] Updated weights for policy 0, policy_version 98478 (0.0008) [2023-12-27 06:20:27,527][20347] Updated weights for policy 0, policy_version 98488 (0.0010) [2023-12-27 06:20:27,586][20347] Updated weights for policy 0, policy_version 98498 (0.0010) [2023-12-27 06:20:27,890][20381] Updated weights for policy 1, policy_version 98145 (0.0007) [2023-12-27 06:20:27,947][20381] Updated weights for policy 1, policy_version 98155 (0.0007) [2023-12-27 06:20:28,000][20381] Updated weights for policy 1, policy_version 98165 (0.0007) [2023-12-27 06:20:28,051][20381] Updated weights for policy 1, policy_version 98175 (0.0008) [2023-12-27 06:20:28,246][20347] Updated weights for policy 0, policy_version 98508 (0.0010) [2023-12-27 06:20:28,297][20347] Updated weights for policy 0, policy_version 98518 (0.0010) [2023-12-27 06:20:28,357][20347] Updated weights for policy 0, policy_version 98528 (0.0009) [2023-12-27 06:20:28,698][20381] Updated weights for policy 1, policy_version 98185 (0.0007) [2023-12-27 06:20:28,755][20381] Updated weights for policy 1, policy_version 98195 (0.0007) [2023-12-27 06:20:28,815][20381] Updated weights for policy 1, policy_version 98205 (0.0008) [2023-12-27 06:20:28,986][20347] Updated weights for policy 0, policy_version 98538 (0.0007) [2023-12-27 06:20:29,038][20347] Updated weights for policy 0, policy_version 98548 (0.0010) [2023-12-27 06:20:29,089][20347] Updated weights for policy 0, policy_version 98558 (0.0010) [2023-12-27 06:20:29,134][20347] Updated weights for policy 0, policy_version 98568 (0.0010) [2023-12-27 06:20:29,531][20381] Updated weights for policy 1, policy_version 98215 (0.0008) [2023-12-27 06:20:29,599][20381] Updated weights for policy 1, policy_version 98225 (0.0008) [2023-12-27 06:20:29,661][20381] Updated weights for policy 1, policy_version 98235 (0.0007) [2023-12-27 06:20:29,846][20347] Updated weights for policy 0, policy_version 98578 (0.0007) [2023-12-27 06:20:29,900][20347] Updated weights for policy 0, policy_version 98588 (0.0009) [2023-12-27 06:20:29,966][20347] Updated weights for policy 0, policy_version 98598 (0.0007) [2023-12-27 06:20:30,348][20381] Updated weights for policy 1, policy_version 98245 (0.0007) [2023-12-27 06:20:30,399][20381] Updated weights for policy 1, policy_version 98255 (0.0007) [2023-12-27 06:20:30,452][20381] Updated weights for policy 1, policy_version 98265 (0.0006) [2023-12-27 06:20:30,613][20347] Updated weights for policy 0, policy_version 98608 (0.0007) [2023-12-27 06:20:30,672][20347] Updated weights for policy 0, policy_version 98618 (0.0008) [2023-12-27 06:20:30,723][20347] Updated weights for policy 0, policy_version 98628 (0.0006) [2023-12-27 06:20:31,163][20381] Updated weights for policy 1, policy_version 98275 (0.0007) [2023-12-27 06:20:31,227][20381] Updated weights for policy 1, policy_version 98285 (0.0006) [2023-12-27 06:20:31,290][20381] Updated weights for policy 1, policy_version 98295 (0.0007) [2023-12-27 06:20:31,410][20347] Updated weights for policy 0, policy_version 98638 (0.0007) [2023-12-27 06:20:31,466][20347] Updated weights for policy 0, policy_version 98648 (0.0008) [2023-12-27 06:20:31,516][20347] Updated weights for policy 0, policy_version 98658 (0.0007) [2023-12-27 06:20:31,930][20381] Updated weights for policy 1, policy_version 98305 (0.0006) [2023-12-27 06:20:31,991][20381] Updated weights for policy 1, policy_version 98315 (0.0007) [2023-12-27 06:20:32,054][20381] Updated weights for policy 1, policy_version 98325 (0.0009) [2023-12-27 06:20:32,113][20381] Updated weights for policy 1, policy_version 98335 (0.0008) [2023-12-27 06:20:32,195][20347] Updated weights for policy 0, policy_version 98668 (0.0008) [2023-12-27 06:20:32,244][20347] Updated weights for policy 0, policy_version 98678 (0.0007) [2023-12-27 06:20:32,297][20347] Updated weights for policy 0, policy_version 98688 (0.0007) [2023-12-27 06:20:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 50462720. Throughput: 0: 10465.8, 1: 10334.9. Samples: 50467532. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:32,363][19170] Avg episode reward: [(0, '303.210'), (1, '1000.000')] [2023-12-27 06:20:32,773][20381] Updated weights for policy 1, policy_version 98345 (0.0008) [2023-12-27 06:20:32,831][20381] Updated weights for policy 1, policy_version 98355 (0.0007) [2023-12-27 06:20:32,886][20381] Updated weights for policy 1, policy_version 98365 (0.0008) [2023-12-27 06:20:32,984][20347] Updated weights for policy 0, policy_version 98698 (0.0008) [2023-12-27 06:20:33,041][20347] Updated weights for policy 0, policy_version 98708 (0.0007) [2023-12-27 06:20:33,087][20347] Updated weights for policy 0, policy_version 98718 (0.0007) [2023-12-27 06:20:33,132][20347] Updated weights for policy 0, policy_version 98728 (0.0007) [2023-12-27 06:20:33,562][20381] Updated weights for policy 1, policy_version 98375 (0.0007) [2023-12-27 06:20:33,615][20381] Updated weights for policy 1, policy_version 98385 (0.0007) [2023-12-27 06:20:33,666][20381] Updated weights for policy 1, policy_version 98395 (0.0007) [2023-12-27 06:20:33,837][20347] Updated weights for policy 0, policy_version 98738 (0.0007) [2023-12-27 06:20:33,885][20347] Updated weights for policy 0, policy_version 98748 (0.0008) [2023-12-27 06:20:33,929][20347] Updated weights for policy 0, policy_version 98758 (0.0007) [2023-12-27 06:20:34,361][20381] Updated weights for policy 1, policy_version 98405 (0.0007) [2023-12-27 06:20:34,421][20381] Updated weights for policy 1, policy_version 98415 (0.0008) [2023-12-27 06:20:34,481][20381] Updated weights for policy 1, policy_version 98425 (0.0007) [2023-12-27 06:20:34,611][20347] Updated weights for policy 0, policy_version 98768 (0.0008) [2023-12-27 06:20:34,667][20347] Updated weights for policy 0, policy_version 98778 (0.0007) [2023-12-27 06:20:34,723][20347] Updated weights for policy 0, policy_version 98788 (0.0008) [2023-12-27 06:20:35,180][20381] Updated weights for policy 1, policy_version 98435 (0.0007) [2023-12-27 06:20:35,235][20381] Updated weights for policy 1, policy_version 98445 (0.0009) [2023-12-27 06:20:35,287][20381] Updated weights for policy 1, policy_version 98455 (0.0008) [2023-12-27 06:20:35,392][20347] Updated weights for policy 0, policy_version 98798 (0.0008) [2023-12-27 06:20:35,441][20347] Updated weights for policy 0, policy_version 98808 (0.0008) [2023-12-27 06:20:35,490][20347] Updated weights for policy 0, policy_version 98818 (0.0008) [2023-12-27 06:20:35,959][20381] Updated weights for policy 1, policy_version 98465 (0.0007) [2023-12-27 06:20:36,008][20381] Updated weights for policy 1, policy_version 98475 (0.0008) [2023-12-27 06:20:36,063][20381] Updated weights for policy 1, policy_version 98485 (0.0008) [2023-12-27 06:20:36,121][20381] Updated weights for policy 1, policy_version 98495 (0.0008) [2023-12-27 06:20:36,206][20347] Updated weights for policy 0, policy_version 98828 (0.0008) [2023-12-27 06:20:36,269][20347] Updated weights for policy 0, policy_version 98838 (0.0008) [2023-12-27 06:20:36,329][20347] Updated weights for policy 0, policy_version 98848 (0.0007) [2023-12-27 06:20:36,783][20381] Updated weights for policy 1, policy_version 98505 (0.0008) [2023-12-27 06:20:36,841][20381] Updated weights for policy 1, policy_version 98515 (0.0007) [2023-12-27 06:20:36,896][20381] Updated weights for policy 1, policy_version 98525 (0.0008) [2023-12-27 06:20:37,038][20347] Updated weights for policy 0, policy_version 98858 (0.0007) [2023-12-27 06:20:37,095][20347] Updated weights for policy 0, policy_version 98868 (0.0008) [2023-12-27 06:20:37,159][20347] Updated weights for policy 0, policy_version 98878 (0.0007) [2023-12-27 06:20:37,222][20347] Updated weights for policy 0, policy_version 98888 (0.0009) [2023-12-27 06:20:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 50561024. Throughput: 0: 10505.4, 1: 10294.9. Samples: 50528812. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:37,363][19170] Avg episode reward: [(0, '247.470'), (1, '1000.000')] [2023-12-27 06:20:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000098528_25231360.pth... [2023-12-27 06:20:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000098888_25329664.pth... [2023-12-27 06:20:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000097344_24928256.pth [2023-12-27 06:20:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000097640_25010176.pth [2023-12-27 06:20:37,621][20381] Updated weights for policy 1, policy_version 98535 (0.0007) [2023-12-27 06:20:37,677][20381] Updated weights for policy 1, policy_version 98545 (0.0007) [2023-12-27 06:20:37,734][20381] Updated weights for policy 1, policy_version 98555 (0.0009) [2023-12-27 06:20:37,921][20347] Updated weights for policy 0, policy_version 98898 (0.0008) [2023-12-27 06:20:37,980][20347] Updated weights for policy 0, policy_version 98908 (0.0008) [2023-12-27 06:20:38,038][20347] Updated weights for policy 0, policy_version 98918 (0.0007) [2023-12-27 06:20:38,426][20381] Updated weights for policy 1, policy_version 98565 (0.0007) [2023-12-27 06:20:38,482][20381] Updated weights for policy 1, policy_version 98575 (0.0008) [2023-12-27 06:20:38,541][20381] Updated weights for policy 1, policy_version 98585 (0.0007) [2023-12-27 06:20:38,744][20347] Updated weights for policy 0, policy_version 98928 (0.0007) [2023-12-27 06:20:38,797][20347] Updated weights for policy 0, policy_version 98938 (0.0008) [2023-12-27 06:20:38,854][20347] Updated weights for policy 0, policy_version 98948 (0.0008) [2023-12-27 06:20:39,241][20381] Updated weights for policy 1, policy_version 98595 (0.0007) [2023-12-27 06:20:39,303][20381] Updated weights for policy 1, policy_version 98605 (0.0008) [2023-12-27 06:20:39,373][20381] Updated weights for policy 1, policy_version 98615 (0.0008) [2023-12-27 06:20:39,557][20347] Updated weights for policy 0, policy_version 98958 (0.0007) [2023-12-27 06:20:39,618][20347] Updated weights for policy 0, policy_version 98968 (0.0007) [2023-12-27 06:20:39,682][20347] Updated weights for policy 0, policy_version 98978 (0.0008) [2023-12-27 06:20:40,056][20381] Updated weights for policy 1, policy_version 98625 (0.0008) [2023-12-27 06:20:40,115][20381] Updated weights for policy 1, policy_version 98635 (0.0010) [2023-12-27 06:20:40,177][20381] Updated weights for policy 1, policy_version 98645 (0.0010) [2023-12-27 06:20:40,235][20381] Updated weights for policy 1, policy_version 98655 (0.0010) [2023-12-27 06:20:40,311][20347] Updated weights for policy 0, policy_version 98988 (0.0009) [2023-12-27 06:20:40,371][20347] Updated weights for policy 0, policy_version 98998 (0.0006) [2023-12-27 06:20:40,428][20347] Updated weights for policy 0, policy_version 99008 (0.0008) [2023-12-27 06:20:40,925][20381] Updated weights for policy 1, policy_version 98665 (0.0010) [2023-12-27 06:20:40,978][20381] Updated weights for policy 1, policy_version 98675 (0.0011) [2023-12-27 06:20:41,031][20347] Updated weights for policy 0, policy_version 99018 (0.0008) [2023-12-27 06:20:41,036][20381] Updated weights for policy 1, policy_version 98685 (0.0010) [2023-12-27 06:20:41,087][20347] Updated weights for policy 0, policy_version 99028 (0.0007) [2023-12-27 06:20:41,144][20347] Updated weights for policy 0, policy_version 99038 (0.0005) [2023-12-27 06:20:41,207][20347] Updated weights for policy 0, policy_version 99048 (0.0007) [2023-12-27 06:20:41,741][20381] Updated weights for policy 1, policy_version 98695 (0.0008) [2023-12-27 06:20:41,796][20381] Updated weights for policy 1, policy_version 98705 (0.0007) [2023-12-27 06:20:41,852][20381] Updated weights for policy 1, policy_version 98715 (0.0007) [2023-12-27 06:20:41,910][20347] Updated weights for policy 0, policy_version 99058 (0.0009) [2023-12-27 06:20:41,969][20347] Updated weights for policy 0, policy_version 99068 (0.0008) [2023-12-27 06:20:42,025][20347] Updated weights for policy 0, policy_version 99078 (0.0008) [2023-12-27 06:20:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 50659328. Throughput: 0: 10496.5, 1: 10252.0. Samples: 50649924. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:42,363][19170] Avg episode reward: [(0, '355.640'), (1, '1000.000')] [2023-12-27 06:20:42,522][20381] Updated weights for policy 1, policy_version 98725 (0.0008) [2023-12-27 06:20:42,581][20381] Updated weights for policy 1, policy_version 98735 (0.0008) [2023-12-27 06:20:42,634][20381] Updated weights for policy 1, policy_version 98745 (0.0008) [2023-12-27 06:20:42,707][20347] Updated weights for policy 0, policy_version 99088 (0.0007) [2023-12-27 06:20:42,771][20347] Updated weights for policy 0, policy_version 99098 (0.0007) [2023-12-27 06:20:42,833][20347] Updated weights for policy 0, policy_version 99108 (0.0009) [2023-12-27 06:20:43,298][20381] Updated weights for policy 1, policy_version 98755 (0.0007) [2023-12-27 06:20:43,351][20381] Updated weights for policy 1, policy_version 98765 (0.0005) [2023-12-27 06:20:43,402][20381] Updated weights for policy 1, policy_version 98775 (0.0006) [2023-12-27 06:20:43,552][20347] Updated weights for policy 0, policy_version 99118 (0.0007) [2023-12-27 06:20:43,617][20347] Updated weights for policy 0, policy_version 99128 (0.0008) [2023-12-27 06:20:43,677][20347] Updated weights for policy 0, policy_version 99138 (0.0007) [2023-12-27 06:20:44,006][20381] Updated weights for policy 1, policy_version 98785 (0.0005) [2023-12-27 06:20:44,065][20381] Updated weights for policy 1, policy_version 98795 (0.0010) [2023-12-27 06:20:44,120][20381] Updated weights for policy 1, policy_version 98805 (0.0010) [2023-12-27 06:20:44,185][20381] Updated weights for policy 1, policy_version 98815 (0.0010) [2023-12-27 06:20:44,360][20347] Updated weights for policy 0, policy_version 99148 (0.0008) [2023-12-27 06:20:44,418][20347] Updated weights for policy 0, policy_version 99158 (0.0007) [2023-12-27 06:20:44,472][20347] Updated weights for policy 0, policy_version 99168 (0.0007) [2023-12-27 06:20:44,857][20381] Updated weights for policy 1, policy_version 98825 (0.0011) [2023-12-27 06:20:44,922][20381] Updated weights for policy 1, policy_version 98835 (0.0011) [2023-12-27 06:20:44,979][20381] Updated weights for policy 1, policy_version 98845 (0.0011) [2023-12-27 06:20:45,148][20347] Updated weights for policy 0, policy_version 99178 (0.0007) [2023-12-27 06:20:45,202][20347] Updated weights for policy 0, policy_version 99188 (0.0008) [2023-12-27 06:20:45,258][20347] Updated weights for policy 0, policy_version 99198 (0.0010) [2023-12-27 06:20:45,318][20347] Updated weights for policy 0, policy_version 99208 (0.0008) [2023-12-27 06:20:45,673][20381] Updated weights for policy 1, policy_version 98855 (0.0010) [2023-12-27 06:20:45,728][20381] Updated weights for policy 1, policy_version 98865 (0.0010) [2023-12-27 06:20:45,779][20381] Updated weights for policy 1, policy_version 98875 (0.0010) [2023-12-27 06:20:45,916][20347] Updated weights for policy 0, policy_version 99218 (0.0005) [2023-12-27 06:20:45,977][20347] Updated weights for policy 0, policy_version 99228 (0.0005) [2023-12-27 06:20:46,037][20347] Updated weights for policy 0, policy_version 99238 (0.0005) [2023-12-27 06:20:46,370][20381] Updated weights for policy 1, policy_version 98885 (0.0008) [2023-12-27 06:20:46,427][20381] Updated weights for policy 1, policy_version 98895 (0.0005) [2023-12-27 06:20:46,482][20381] Updated weights for policy 1, policy_version 98905 (0.0005) [2023-12-27 06:20:46,600][20347] Updated weights for policy 0, policy_version 99248 (0.0006) [2023-12-27 06:20:46,655][20347] Updated weights for policy 0, policy_version 99258 (0.0005) [2023-12-27 06:20:46,719][20347] Updated weights for policy 0, policy_version 99268 (0.0005) [2023-12-27 06:20:47,039][20381] Updated weights for policy 1, policy_version 98915 (0.0005) [2023-12-27 06:20:47,098][20381] Updated weights for policy 1, policy_version 98925 (0.0005) [2023-12-27 06:20:47,160][20381] Updated weights for policy 1, policy_version 98935 (0.0006) [2023-12-27 06:20:47,349][20347] Updated weights for policy 0, policy_version 99278 (0.0007) [2023-12-27 06:20:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 50765824. Throughput: 0: 10483.8, 1: 10294.4. Samples: 50776496. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:47,363][19170] Avg episode reward: [(0, '197.820'), (1, '1000.000')] [2023-12-27 06:20:47,409][20347] Updated weights for policy 0, policy_version 99288 (0.0007) [2023-12-27 06:20:47,465][20347] Updated weights for policy 0, policy_version 99298 (0.0007) [2023-12-27 06:20:47,713][20381] Updated weights for policy 1, policy_version 98945 (0.0007) [2023-12-27 06:20:47,775][20381] Updated weights for policy 1, policy_version 98955 (0.0005) [2023-12-27 06:20:47,826][20381] Updated weights for policy 1, policy_version 98965 (0.0005) [2023-12-27 06:20:47,885][20381] Updated weights for policy 1, policy_version 98975 (0.0006) [2023-12-27 06:20:48,110][20347] Updated weights for policy 0, policy_version 99308 (0.0006) [2023-12-27 06:20:48,159][20347] Updated weights for policy 0, policy_version 99318 (0.0007) [2023-12-27 06:20:48,210][20347] Updated weights for policy 0, policy_version 99328 (0.0007) [2023-12-27 06:20:48,464][20381] Updated weights for policy 1, policy_version 98985 (0.0008) [2023-12-27 06:20:48,513][20381] Updated weights for policy 1, policy_version 98995 (0.0009) [2023-12-27 06:20:48,571][20381] Updated weights for policy 1, policy_version 99005 (0.0007) [2023-12-27 06:20:48,965][20347] Updated weights for policy 0, policy_version 99338 (0.0008) [2023-12-27 06:20:49,019][20347] Updated weights for policy 0, policy_version 99348 (0.0007) [2023-12-27 06:20:49,078][20347] Updated weights for policy 0, policy_version 99358 (0.0007) [2023-12-27 06:20:49,140][20347] Updated weights for policy 0, policy_version 99368 (0.0006) [2023-12-27 06:20:49,217][20381] Updated weights for policy 1, policy_version 99015 (0.0007) [2023-12-27 06:20:49,281][20381] Updated weights for policy 1, policy_version 99025 (0.0008) [2023-12-27 06:20:49,346][20381] Updated weights for policy 1, policy_version 99035 (0.0008) [2023-12-27 06:20:49,830][20347] Updated weights for policy 0, policy_version 99378 (0.0008) [2023-12-27 06:20:49,889][20347] Updated weights for policy 0, policy_version 99388 (0.0006) [2023-12-27 06:20:49,951][20347] Updated weights for policy 0, policy_version 99398 (0.0007) [2023-12-27 06:20:50,084][20381] Updated weights for policy 1, policy_version 99045 (0.0008) [2023-12-27 06:20:50,140][20381] Updated weights for policy 1, policy_version 99055 (0.0007) [2023-12-27 06:20:50,188][20381] Updated weights for policy 1, policy_version 99065 (0.0008) [2023-12-27 06:20:50,635][20347] Updated weights for policy 0, policy_version 99408 (0.0007) [2023-12-27 06:20:50,691][20347] Updated weights for policy 0, policy_version 99418 (0.0007) [2023-12-27 06:20:50,746][20347] Updated weights for policy 0, policy_version 99428 (0.0007) [2023-12-27 06:20:50,918][20381] Updated weights for policy 1, policy_version 99075 (0.0008) [2023-12-27 06:20:50,968][20381] Updated weights for policy 1, policy_version 99085 (0.0008) [2023-12-27 06:20:51,020][20381] Updated weights for policy 1, policy_version 99095 (0.0008) [2023-12-27 06:20:51,426][20347] Updated weights for policy 0, policy_version 99438 (0.0007) [2023-12-27 06:20:51,483][20347] Updated weights for policy 0, policy_version 99448 (0.0008) [2023-12-27 06:20:51,542][20347] Updated weights for policy 0, policy_version 99458 (0.0007) [2023-12-27 06:20:51,733][20381] Updated weights for policy 1, policy_version 99105 (0.0008) [2023-12-27 06:20:51,791][20381] Updated weights for policy 1, policy_version 99115 (0.0008) [2023-12-27 06:20:51,837][20381] Updated weights for policy 1, policy_version 99125 (0.0009) [2023-12-27 06:20:51,890][20381] Updated weights for policy 1, policy_version 99135 (0.0007) [2023-12-27 06:20:52,223][20347] Updated weights for policy 0, policy_version 99468 (0.0008) [2023-12-27 06:20:52,281][20347] Updated weights for policy 0, policy_version 99478 (0.0010) [2023-12-27 06:20:52,337][20347] Updated weights for policy 0, policy_version 99488 (0.0010) [2023-12-27 06:20:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 50864128. Throughput: 0: 10431.1, 1: 10289.1. Samples: 50838220. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:52,363][19170] Avg episode reward: [(0, '240.690'), (1, '1000.000')] [2023-12-27 06:20:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000099136_25387008.pth... [2023-12-27 06:20:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000097920_25075712.pth [2023-12-27 06:20:52,386][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000099496_25485312.pth... [2023-12-27 06:20:52,389][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000098280_25174016.pth [2023-12-27 06:20:52,621][20381] Updated weights for policy 1, policy_version 99145 (0.0007) [2023-12-27 06:20:52,678][20381] Updated weights for policy 1, policy_version 99155 (0.0008) [2023-12-27 06:20:52,741][20381] Updated weights for policy 1, policy_version 99165 (0.0008) [2023-12-27 06:20:53,036][20347] Updated weights for policy 0, policy_version 99498 (0.0009) [2023-12-27 06:20:53,099][20347] Updated weights for policy 0, policy_version 99508 (0.0007) [2023-12-27 06:20:53,151][20347] Updated weights for policy 0, policy_version 99518 (0.0009) [2023-12-27 06:20:53,208][20347] Updated weights for policy 0, policy_version 99528 (0.0008) [2023-12-27 06:20:53,351][20381] Updated weights for policy 1, policy_version 99175 (0.0007) [2023-12-27 06:20:53,403][20381] Updated weights for policy 1, policy_version 99185 (0.0008) [2023-12-27 06:20:53,449][20381] Updated weights for policy 1, policy_version 99195 (0.0008) [2023-12-27 06:20:53,870][20347] Updated weights for policy 0, policy_version 99538 (0.0007) [2023-12-27 06:20:53,925][20347] Updated weights for policy 0, policy_version 99548 (0.0008) [2023-12-27 06:20:53,982][20347] Updated weights for policy 0, policy_version 99558 (0.0008) [2023-12-27 06:20:54,061][20381] Updated weights for policy 1, policy_version 99205 (0.0005) [2023-12-27 06:20:54,128][20381] Updated weights for policy 1, policy_version 99215 (0.0006) [2023-12-27 06:20:54,176][20381] Updated weights for policy 1, policy_version 99225 (0.0005) [2023-12-27 06:20:54,647][20347] Updated weights for policy 0, policy_version 99568 (0.0007) [2023-12-27 06:20:54,691][20347] Updated weights for policy 0, policy_version 99578 (0.0007) [2023-12-27 06:20:54,734][20347] Updated weights for policy 0, policy_version 99588 (0.0007) [2023-12-27 06:20:54,792][20381] Updated weights for policy 1, policy_version 99235 (0.0006) [2023-12-27 06:20:54,844][20381] Updated weights for policy 1, policy_version 99245 (0.0007) [2023-12-27 06:20:54,890][20381] Updated weights for policy 1, policy_version 99255 (0.0008) [2023-12-27 06:20:55,385][20347] Updated weights for policy 0, policy_version 99598 (0.0005) [2023-12-27 06:20:55,452][20347] Updated weights for policy 0, policy_version 99608 (0.0005) [2023-12-27 06:20:55,515][20347] Updated weights for policy 0, policy_version 99618 (0.0006) [2023-12-27 06:20:55,645][20381] Updated weights for policy 1, policy_version 99265 (0.0007) [2023-12-27 06:20:55,697][20381] Updated weights for policy 1, policy_version 99275 (0.0008) [2023-12-27 06:20:55,747][20381] Updated weights for policy 1, policy_version 99285 (0.0008) [2023-12-27 06:20:55,799][20381] Updated weights for policy 1, policy_version 99295 (0.0007) [2023-12-27 06:20:56,093][20347] Updated weights for policy 0, policy_version 99628 (0.0007) [2023-12-27 06:20:56,153][20347] Updated weights for policy 0, policy_version 99638 (0.0010) [2023-12-27 06:20:56,208][20347] Updated weights for policy 0, policy_version 99648 (0.0006) [2023-12-27 06:20:56,517][20381] Updated weights for policy 1, policy_version 99305 (0.0007) [2023-12-27 06:20:56,571][20381] Updated weights for policy 1, policy_version 99315 (0.0006) [2023-12-27 06:20:56,615][20381] Updated weights for policy 1, policy_version 99325 (0.0007) [2023-12-27 06:20:56,840][20347] Updated weights for policy 0, policy_version 99658 (0.0006) [2023-12-27 06:20:56,895][20347] Updated weights for policy 0, policy_version 99668 (0.0008) [2023-12-27 06:20:56,951][20347] Updated weights for policy 0, policy_version 99678 (0.0008) [2023-12-27 06:20:57,007][20347] Updated weights for policy 0, policy_version 99688 (0.0008) [2023-12-27 06:20:57,258][20381] Updated weights for policy 1, policy_version 99335 (0.0006) [2023-12-27 06:20:57,312][20381] Updated weights for policy 1, policy_version 99345 (0.0007) [2023-12-27 06:20:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 50970624. Throughput: 0: 10466.4, 1: 10292.6. Samples: 50962812. Policy #0 lag: (min: 1.0, avg: 22.0, max: 33.0) [2023-12-27 06:20:57,363][19170] Avg episode reward: [(0, '316.550'), (1, '1000.000')] [2023-12-27 06:20:57,366][20381] Updated weights for policy 1, policy_version 99355 (0.0007) [2023-12-27 06:20:57,651][20347] Updated weights for policy 0, policy_version 99698 (0.0008) [2023-12-27 06:20:57,696][20347] Updated weights for policy 0, policy_version 99708 (0.0008) [2023-12-27 06:20:57,749][20347] Updated weights for policy 0, policy_version 99718 (0.0008) [2023-12-27 06:20:58,004][20381] Updated weights for policy 1, policy_version 99365 (0.0007) [2023-12-27 06:20:58,055][20381] Updated weights for policy 1, policy_version 99375 (0.0005) [2023-12-27 06:20:58,109][20381] Updated weights for policy 1, policy_version 99385 (0.0005) [2023-12-27 06:20:58,494][20347] Updated weights for policy 0, policy_version 99728 (0.0008) [2023-12-27 06:20:58,557][20347] Updated weights for policy 0, policy_version 99738 (0.0007) [2023-12-27 06:20:58,618][20347] Updated weights for policy 0, policy_version 99748 (0.0006) [2023-12-27 06:20:58,841][20381] Updated weights for policy 1, policy_version 99395 (0.0006) [2023-12-27 06:20:58,909][20381] Updated weights for policy 1, policy_version 99405 (0.0008) [2023-12-27 06:20:58,975][20381] Updated weights for policy 1, policy_version 99415 (0.0008) [2023-12-27 06:20:59,385][20347] Updated weights for policy 0, policy_version 99758 (0.0007) [2023-12-27 06:20:59,442][20347] Updated weights for policy 0, policy_version 99768 (0.0009) [2023-12-27 06:20:59,496][20347] Updated weights for policy 0, policy_version 99778 (0.0009) [2023-12-27 06:20:59,749][20381] Updated weights for policy 1, policy_version 99425 (0.0008) [2023-12-27 06:20:59,813][20381] Updated weights for policy 1, policy_version 99435 (0.0006) [2023-12-27 06:20:59,875][20381] Updated weights for policy 1, policy_version 99445 (0.0007) [2023-12-27 06:20:59,931][20381] Updated weights for policy 1, policy_version 99455 (0.0008) [2023-12-27 06:21:00,170][20347] Updated weights for policy 0, policy_version 99788 (0.0008) [2023-12-27 06:21:00,223][20347] Updated weights for policy 0, policy_version 99798 (0.0007) [2023-12-27 06:21:00,285][20347] Updated weights for policy 0, policy_version 99808 (0.0007) [2023-12-27 06:21:00,572][20381] Updated weights for policy 1, policy_version 99465 (0.0007) [2023-12-27 06:21:00,628][20381] Updated weights for policy 1, policy_version 99475 (0.0008) [2023-12-27 06:21:00,687][20381] Updated weights for policy 1, policy_version 99485 (0.0008) [2023-12-27 06:21:00,959][20347] Updated weights for policy 0, policy_version 99818 (0.0007) [2023-12-27 06:21:01,016][20347] Updated weights for policy 0, policy_version 99828 (0.0007) [2023-12-27 06:21:01,081][20347] Updated weights for policy 0, policy_version 99838 (0.0005) [2023-12-27 06:21:01,149][20347] Updated weights for policy 0, policy_version 99848 (0.0006) [2023-12-27 06:21:01,410][20381] Updated weights for policy 1, policy_version 99495 (0.0009) [2023-12-27 06:21:01,466][20381] Updated weights for policy 1, policy_version 99505 (0.0009) [2023-12-27 06:21:01,514][20381] Updated weights for policy 1, policy_version 99515 (0.0008) [2023-12-27 06:21:01,806][20347] Updated weights for policy 0, policy_version 99858 (0.0006) [2023-12-27 06:21:01,860][20347] Updated weights for policy 0, policy_version 99868 (0.0007) [2023-12-27 06:21:01,918][20347] Updated weights for policy 0, policy_version 99878 (0.0008) [2023-12-27 06:21:02,218][20381] Updated weights for policy 1, policy_version 99525 (0.0008) [2023-12-27 06:21:02,272][20381] Updated weights for policy 1, policy_version 99535 (0.0009) [2023-12-27 06:21:02,333][20381] Updated weights for policy 1, policy_version 99545 (0.0009) [2023-12-27 06:21:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 51068928. Throughput: 0: 10397.1, 1: 10289.7. Samples: 51083108. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:02,363][19170] Avg episode reward: [(0, '769.860'), (1, '1000.000')] [2023-12-27 06:21:02,553][20347] Updated weights for policy 0, policy_version 99888 (0.0007) [2023-12-27 06:21:02,611][20347] Updated weights for policy 0, policy_version 99898 (0.0008) [2023-12-27 06:21:02,659][20347] Updated weights for policy 0, policy_version 99908 (0.0009) [2023-12-27 06:21:03,118][20381] Updated weights for policy 1, policy_version 99555 (0.0008) [2023-12-27 06:21:03,176][20381] Updated weights for policy 1, policy_version 99565 (0.0007) [2023-12-27 06:21:03,227][20347] Updated weights for policy 0, policy_version 99918 (0.0008) [2023-12-27 06:21:03,231][20381] Updated weights for policy 1, policy_version 99575 (0.0007) [2023-12-27 06:21:03,272][20347] Updated weights for policy 0, policy_version 99928 (0.0010) [2023-12-27 06:21:03,317][20347] Updated weights for policy 0, policy_version 99938 (0.0010) [2023-12-27 06:21:03,911][20381] Updated weights for policy 1, policy_version 99585 (0.0006) [2023-12-27 06:21:03,975][20381] Updated weights for policy 1, policy_version 99595 (0.0007) [2023-12-27 06:21:04,012][20347] Updated weights for policy 0, policy_version 99948 (0.0010) [2023-12-27 06:21:04,034][20381] Updated weights for policy 1, policy_version 99605 (0.0006) [2023-12-27 06:21:04,072][20347] Updated weights for policy 0, policy_version 99958 (0.0010) [2023-12-27 06:21:04,092][20381] Updated weights for policy 1, policy_version 99615 (0.0005) [2023-12-27 06:21:04,127][20347] Updated weights for policy 0, policy_version 99968 (0.0010) [2023-12-27 06:21:04,787][20347] Updated weights for policy 0, policy_version 99978 (0.0010) [2023-12-27 06:21:04,797][20381] Updated weights for policy 1, policy_version 99625 (0.0007) [2023-12-27 06:21:04,838][20347] Updated weights for policy 0, policy_version 99988 (0.0010) [2023-12-27 06:21:04,849][20381] Updated weights for policy 1, policy_version 99635 (0.0006) [2023-12-27 06:21:04,889][20347] Updated weights for policy 0, policy_version 99998 (0.0010) [2023-12-27 06:21:04,899][20381] Updated weights for policy 1, policy_version 99645 (0.0007) [2023-12-27 06:21:04,934][20347] Updated weights for policy 0, policy_version 100008 (0.0010) [2023-12-27 06:21:05,606][20347] Updated weights for policy 0, policy_version 100018 (0.0010) [2023-12-27 06:21:05,607][20381] Updated weights for policy 1, policy_version 99655 (0.0006) [2023-12-27 06:21:05,666][20381] Updated weights for policy 1, policy_version 99665 (0.0006) [2023-12-27 06:21:05,671][20347] Updated weights for policy 0, policy_version 100028 (0.0010) [2023-12-27 06:21:05,719][20347] Updated weights for policy 0, policy_version 100038 (0.0009) [2023-12-27 06:21:05,723][20381] Updated weights for policy 1, policy_version 99675 (0.0005) [2023-12-27 06:21:06,274][20381] Updated weights for policy 1, policy_version 99685 (0.0007) [2023-12-27 06:21:06,338][20381] Updated weights for policy 1, policy_version 99695 (0.0006) [2023-12-27 06:21:06,356][20347] Updated weights for policy 0, policy_version 100048 (0.0005) [2023-12-27 06:21:06,399][20381] Updated weights for policy 1, policy_version 99705 (0.0006) [2023-12-27 06:21:06,412][20347] Updated weights for policy 0, policy_version 100058 (0.0005) [2023-12-27 06:21:06,473][20347] Updated weights for policy 0, policy_version 100068 (0.0006) [2023-12-27 06:21:06,978][20381] Updated weights for policy 1, policy_version 99715 (0.0006) [2023-12-27 06:21:07,027][20381] Updated weights for policy 1, policy_version 99725 (0.0006) [2023-12-27 06:21:07,087][20381] Updated weights for policy 1, policy_version 99735 (0.0007) [2023-12-27 06:21:07,124][20347] Updated weights for policy 0, policy_version 100078 (0.0008) [2023-12-27 06:21:07,181][20347] Updated weights for policy 0, policy_version 100088 (0.0007) [2023-12-27 06:21:07,238][20347] Updated weights for policy 0, policy_version 100098 (0.0008) [2023-12-27 06:21:07,363][19170] Fps is (10 sec: 21298.3, 60 sec: 20752.9, 300 sec: 20799.3). Total num frames: 51183616. Throughput: 0: 10385.4, 1: 10268.1. Samples: 51144532. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000099744_25542656.pth... [2023-12-27 06:21:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000100104_25640960.pth... [2023-12-27 06:21:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000098528_25231360.pth [2023-12-27 06:21:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000098888_25329664.pth [2023-12-27 06:21:07,744][20381] Updated weights for policy 1, policy_version 99745 (0.0006) [2023-12-27 06:21:07,803][20381] Updated weights for policy 1, policy_version 99755 (0.0005) [2023-12-27 06:21:07,860][20381] Updated weights for policy 1, policy_version 99765 (0.0007) [2023-12-27 06:21:07,920][20381] Updated weights for policy 1, policy_version 99775 (0.0008) [2023-12-27 06:21:07,942][20347] Updated weights for policy 0, policy_version 100108 (0.0007) [2023-12-27 06:21:07,997][20347] Updated weights for policy 0, policy_version 100118 (0.0007) [2023-12-27 06:21:08,055][20347] Updated weights for policy 0, policy_version 100128 (0.0006) [2023-12-27 06:21:08,609][20381] Updated weights for policy 1, policy_version 99785 (0.0008) [2023-12-27 06:21:08,666][20381] Updated weights for policy 1, policy_version 99795 (0.0008) [2023-12-27 06:21:08,715][20381] Updated weights for policy 1, policy_version 99805 (0.0007) [2023-12-27 06:21:08,726][20347] Updated weights for policy 0, policy_version 100138 (0.0006) [2023-12-27 06:21:08,778][20347] Updated weights for policy 0, policy_version 100148 (0.0008) [2023-12-27 06:21:08,830][20347] Updated weights for policy 0, policy_version 100158 (0.0007) [2023-12-27 06:21:08,882][20347] Updated weights for policy 0, policy_version 100168 (0.0008) [2023-12-27 06:21:09,425][20381] Updated weights for policy 1, policy_version 99815 (0.0008) [2023-12-27 06:21:09,480][20381] Updated weights for policy 1, policy_version 99825 (0.0009) [2023-12-27 06:21:09,538][20381] Updated weights for policy 1, policy_version 99835 (0.0008) [2023-12-27 06:21:09,548][20347] Updated weights for policy 0, policy_version 100178 (0.0005) [2023-12-27 06:21:09,612][20347] Updated weights for policy 0, policy_version 100188 (0.0006) [2023-12-27 06:21:09,672][20347] Updated weights for policy 0, policy_version 100198 (0.0008) [2023-12-27 06:21:10,215][20381] Updated weights for policy 1, policy_version 99845 (0.0008) [2023-12-27 06:21:10,272][20381] Updated weights for policy 1, policy_version 99855 (0.0008) [2023-12-27 06:21:10,322][20381] Updated weights for policy 1, policy_version 99865 (0.0008) [2023-12-27 06:21:10,372][20347] Updated weights for policy 0, policy_version 100208 (0.0006) [2023-12-27 06:21:10,457][20347] Updated weights for policy 0, policy_version 100218 (0.0008) [2023-12-27 06:21:10,505][20347] Updated weights for policy 0, policy_version 100228 (0.0009) [2023-12-27 06:21:10,952][20381] Updated weights for policy 1, policy_version 99875 (0.0007) [2023-12-27 06:21:11,014][20381] Updated weights for policy 1, policy_version 99885 (0.0006) [2023-12-27 06:21:11,075][20347] Updated weights for policy 0, policy_version 100238 (0.0006) [2023-12-27 06:21:11,078][20381] Updated weights for policy 1, policy_version 99895 (0.0008) [2023-12-27 06:21:11,136][20347] Updated weights for policy 0, policy_version 100248 (0.0007) [2023-12-27 06:21:11,198][20347] Updated weights for policy 0, policy_version 100258 (0.0008) [2023-12-27 06:21:11,790][20381] Updated weights for policy 1, policy_version 99905 (0.0007) [2023-12-27 06:21:11,856][20381] Updated weights for policy 1, policy_version 99915 (0.0007) [2023-12-27 06:21:11,884][20347] Updated weights for policy 0, policy_version 100268 (0.0007) [2023-12-27 06:21:11,916][20381] Updated weights for policy 1, policy_version 99925 (0.0008) [2023-12-27 06:21:11,941][20347] Updated weights for policy 0, policy_version 100278 (0.0006) [2023-12-27 06:21:11,970][20381] Updated weights for policy 1, policy_version 99935 (0.0006) [2023-12-27 06:21:11,997][20347] Updated weights for policy 0, policy_version 100288 (0.0006) [2023-12-27 06:21:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 51281920. Throughput: 0: 10346.4, 1: 10281.8. Samples: 51271288. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:12,653][20381] Updated weights for policy 1, policy_version 99945 (0.0010) [2023-12-27 06:21:12,702][20381] Updated weights for policy 1, policy_version 99955 (0.0010) [2023-12-27 06:21:12,721][20347] Updated weights for policy 0, policy_version 100298 (0.0008) [2023-12-27 06:21:12,754][20381] Updated weights for policy 1, policy_version 99965 (0.0010) [2023-12-27 06:21:12,779][20347] Updated weights for policy 0, policy_version 100308 (0.0008) [2023-12-27 06:21:12,837][20347] Updated weights for policy 0, policy_version 100318 (0.0010) [2023-12-27 06:21:12,905][20347] Updated weights for policy 0, policy_version 100328 (0.0007) [2023-12-27 06:21:13,417][20381] Updated weights for policy 1, policy_version 99975 (0.0010) [2023-12-27 06:21:13,475][20381] Updated weights for policy 1, policy_version 99985 (0.0009) [2023-12-27 06:21:13,533][20381] Updated weights for policy 1, policy_version 99995 (0.0010) [2023-12-27 06:21:13,557][20347] Updated weights for policy 0, policy_version 100338 (0.0010) [2023-12-27 06:21:13,607][20347] Updated weights for policy 0, policy_version 100348 (0.0010) [2023-12-27 06:21:13,660][20347] Updated weights for policy 0, policy_version 100358 (0.0007) [2023-12-27 06:21:14,215][20381] Updated weights for policy 1, policy_version 100005 (0.0009) [2023-12-27 06:21:14,266][20381] Updated weights for policy 1, policy_version 100015 (0.0008) [2023-12-27 06:21:14,322][20381] Updated weights for policy 1, policy_version 100025 (0.0007) [2023-12-27 06:21:14,343][20347] Updated weights for policy 0, policy_version 100368 (0.0009) [2023-12-27 06:21:14,399][20347] Updated weights for policy 0, policy_version 100378 (0.0010) [2023-12-27 06:21:14,451][20347] Updated weights for policy 0, policy_version 100388 (0.0010) [2023-12-27 06:21:15,030][20381] Updated weights for policy 1, policy_version 100035 (0.0006) [2023-12-27 06:21:15,093][20381] Updated weights for policy 1, policy_version 100045 (0.0008) [2023-12-27 06:21:15,147][20347] Updated weights for policy 0, policy_version 100398 (0.0008) [2023-12-27 06:21:15,153][20381] Updated weights for policy 1, policy_version 100055 (0.0006) [2023-12-27 06:21:15,208][20347] Updated weights for policy 0, policy_version 100408 (0.0007) [2023-12-27 06:21:15,267][20347] Updated weights for policy 0, policy_version 100418 (0.0008) [2023-12-27 06:21:15,800][20381] Updated weights for policy 1, policy_version 100065 (0.0006) [2023-12-27 06:21:15,855][20381] Updated weights for policy 1, policy_version 100075 (0.0005) [2023-12-27 06:21:15,913][20381] Updated weights for policy 1, policy_version 100085 (0.0008) [2023-12-27 06:21:15,969][20381] Updated weights for policy 1, policy_version 100095 (0.0008) [2023-12-27 06:21:15,975][20347] Updated weights for policy 0, policy_version 100428 (0.0009) [2023-12-27 06:21:16,031][20347] Updated weights for policy 0, policy_version 100438 (0.0010) [2023-12-27 06:21:16,085][20347] Updated weights for policy 0, policy_version 100448 (0.0010) [2023-12-27 06:21:16,577][20381] Updated weights for policy 1, policy_version 100105 (0.0011) [2023-12-27 06:21:16,625][20381] Updated weights for policy 1, policy_version 100115 (0.0010) [2023-12-27 06:21:16,674][20381] Updated weights for policy 1, policy_version 100125 (0.0007) [2023-12-27 06:21:16,719][20347] Updated weights for policy 0, policy_version 100458 (0.0009) [2023-12-27 06:21:16,779][20347] Updated weights for policy 0, policy_version 100468 (0.0005) [2023-12-27 06:21:16,844][20347] Updated weights for policy 0, policy_version 100478 (0.0008) [2023-12-27 06:21:16,900][20347] Updated weights for policy 0, policy_version 100488 (0.0008) [2023-12-27 06:21:17,230][20381] Updated weights for policy 1, policy_version 100135 (0.0005) [2023-12-27 06:21:17,290][20381] Updated weights for policy 1, policy_version 100145 (0.0005) [2023-12-27 06:21:17,349][20381] Updated weights for policy 1, policy_version 100155 (0.0005) [2023-12-27 06:21:17,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 51380224. Throughput: 0: 10316.1, 1: 10307.4. Samples: 51395588. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:17,538][20347] Updated weights for policy 0, policy_version 100498 (0.0008) [2023-12-27 06:21:17,582][20347] Updated weights for policy 0, policy_version 100508 (0.0007) [2023-12-27 06:21:17,632][20347] Updated weights for policy 0, policy_version 100518 (0.0009) [2023-12-27 06:21:17,923][20381] Updated weights for policy 1, policy_version 100165 (0.0008) [2023-12-27 06:21:17,982][20381] Updated weights for policy 1, policy_version 100175 (0.0007) [2023-12-27 06:21:18,039][20381] Updated weights for policy 1, policy_version 100185 (0.0007) [2023-12-27 06:21:18,339][20347] Updated weights for policy 0, policy_version 100528 (0.0007) [2023-12-27 06:21:18,394][20347] Updated weights for policy 0, policy_version 100538 (0.0008) [2023-12-27 06:21:18,450][20347] Updated weights for policy 0, policy_version 100548 (0.0009) [2023-12-27 06:21:18,748][20381] Updated weights for policy 1, policy_version 100195 (0.0008) [2023-12-27 06:21:18,805][20381] Updated weights for policy 1, policy_version 100205 (0.0006) [2023-12-27 06:21:18,867][20381] Updated weights for policy 1, policy_version 100215 (0.0007) [2023-12-27 06:21:19,134][20347] Updated weights for policy 0, policy_version 100558 (0.0007) [2023-12-27 06:21:19,189][20347] Updated weights for policy 0, policy_version 100568 (0.0005) [2023-12-27 06:21:19,248][20347] Updated weights for policy 0, policy_version 100578 (0.0010) [2023-12-27 06:21:19,534][20381] Updated weights for policy 1, policy_version 100225 (0.0007) [2023-12-27 06:21:19,598][20381] Updated weights for policy 1, policy_version 100235 (0.0006) [2023-12-27 06:21:19,665][20381] Updated weights for policy 1, policy_version 100245 (0.0007) [2023-12-27 06:21:19,726][20381] Updated weights for policy 1, policy_version 100255 (0.0008) [2023-12-27 06:21:19,940][20347] Updated weights for policy 0, policy_version 100588 (0.0006) [2023-12-27 06:21:20,004][20347] Updated weights for policy 0, policy_version 100598 (0.0008) [2023-12-27 06:21:20,061][20347] Updated weights for policy 0, policy_version 100608 (0.0009) [2023-12-27 06:21:20,400][20381] Updated weights for policy 1, policy_version 100265 (0.0008) [2023-12-27 06:21:20,462][20381] Updated weights for policy 1, policy_version 100275 (0.0008) [2023-12-27 06:21:20,513][20381] Updated weights for policy 1, policy_version 100285 (0.0007) [2023-12-27 06:21:20,710][20347] Updated weights for policy 0, policy_version 100618 (0.0008) [2023-12-27 06:21:20,773][20347] Updated weights for policy 0, policy_version 100628 (0.0010) [2023-12-27 06:21:20,836][20347] Updated weights for policy 0, policy_version 100638 (0.0010) [2023-12-27 06:21:20,899][20347] Updated weights for policy 0, policy_version 100648 (0.0010) [2023-12-27 06:21:21,197][20381] Updated weights for policy 1, policy_version 100295 (0.0008) [2023-12-27 06:21:21,257][20381] Updated weights for policy 1, policy_version 100305 (0.0008) [2023-12-27 06:21:21,318][20381] Updated weights for policy 1, policy_version 100315 (0.0009) [2023-12-27 06:21:21,592][20347] Updated weights for policy 0, policy_version 100658 (0.0010) [2023-12-27 06:21:21,646][20347] Updated weights for policy 0, policy_version 100668 (0.0009) [2023-12-27 06:21:21,709][20347] Updated weights for policy 0, policy_version 100678 (0.0010) [2023-12-27 06:21:22,044][20381] Updated weights for policy 1, policy_version 100325 (0.0008) [2023-12-27 06:21:22,103][20381] Updated weights for policy 1, policy_version 100335 (0.0007) [2023-12-27 06:21:22,157][20381] Updated weights for policy 1, policy_version 100345 (0.0007) [2023-12-27 06:21:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 51486720. Throughput: 0: 10312.5, 1: 10319.5. Samples: 51457252. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000100352_25698304.pth... [2023-12-27 06:21:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000099136_25387008.pth [2023-12-27 06:21:22,395][20347] Updated weights for policy 0, policy_version 100688 (0.0011) [2023-12-27 06:21:22,449][20347] Updated weights for policy 0, policy_version 100698 (0.0009) [2023-12-27 06:21:22,507][20347] Updated weights for policy 0, policy_version 100708 (0.0005) [2023-12-27 06:21:22,531][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000100712_25796608.pth... [2023-12-27 06:21:22,536][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000099496_25485312.pth [2023-12-27 06:21:22,900][20381] Updated weights for policy 1, policy_version 100355 (0.0007) [2023-12-27 06:21:22,963][20381] Updated weights for policy 1, policy_version 100365 (0.0006) [2023-12-27 06:21:23,022][20381] Updated weights for policy 1, policy_version 100375 (0.0006) [2023-12-27 06:21:23,183][20347] Updated weights for policy 0, policy_version 100718 (0.0009) [2023-12-27 06:21:23,234][20347] Updated weights for policy 0, policy_version 100728 (0.0010) [2023-12-27 06:21:23,288][20347] Updated weights for policy 0, policy_version 100738 (0.0010) [2023-12-27 06:21:23,603][20381] Updated weights for policy 1, policy_version 100385 (0.0006) [2023-12-27 06:21:23,649][20381] Updated weights for policy 1, policy_version 100395 (0.0005) [2023-12-27 06:21:23,699][20381] Updated weights for policy 1, policy_version 100405 (0.0006) [2023-12-27 06:21:23,748][20381] Updated weights for policy 1, policy_version 100415 (0.0005) [2023-12-27 06:21:23,939][20347] Updated weights for policy 0, policy_version 100748 (0.0008) [2023-12-27 06:21:23,990][20347] Updated weights for policy 0, policy_version 100758 (0.0006) [2023-12-27 06:21:24,049][20347] Updated weights for policy 0, policy_version 100768 (0.0006) [2023-12-27 06:21:24,346][20381] Updated weights for policy 1, policy_version 100425 (0.0005) [2023-12-27 06:21:24,407][20381] Updated weights for policy 1, policy_version 100435 (0.0006) [2023-12-27 06:21:24,465][20381] Updated weights for policy 1, policy_version 100445 (0.0006) [2023-12-27 06:21:24,750][20347] Updated weights for policy 0, policy_version 100778 (0.0006) [2023-12-27 06:21:24,815][20347] Updated weights for policy 0, policy_version 100788 (0.0005) [2023-12-27 06:21:24,870][20347] Updated weights for policy 0, policy_version 100798 (0.0008) [2023-12-27 06:21:24,919][20347] Updated weights for policy 0, policy_version 100808 (0.0007) [2023-12-27 06:21:25,106][20381] Updated weights for policy 1, policy_version 100455 (0.0006) [2023-12-27 06:21:25,168][20381] Updated weights for policy 1, policy_version 100465 (0.0008) [2023-12-27 06:21:25,215][20381] Updated weights for policy 1, policy_version 100475 (0.0005) [2023-12-27 06:21:25,550][20347] Updated weights for policy 0, policy_version 100818 (0.0008) [2023-12-27 06:21:25,601][20347] Updated weights for policy 0, policy_version 100828 (0.0007) [2023-12-27 06:21:25,652][20347] Updated weights for policy 0, policy_version 100838 (0.0008) [2023-12-27 06:21:25,789][20381] Updated weights for policy 1, policy_version 100485 (0.0006) [2023-12-27 06:21:25,839][20381] Updated weights for policy 1, policy_version 100495 (0.0006) [2023-12-27 06:21:25,889][20381] Updated weights for policy 1, policy_version 100505 (0.0005) [2023-12-27 06:21:26,272][20347] Updated weights for policy 0, policy_version 100848 (0.0005) [2023-12-27 06:21:26,335][20347] Updated weights for policy 0, policy_version 100858 (0.0005) [2023-12-27 06:21:26,395][20347] Updated weights for policy 0, policy_version 100868 (0.0006) [2023-12-27 06:21:26,443][20381] Updated weights for policy 1, policy_version 100515 (0.0005) [2023-12-27 06:21:26,502][20381] Updated weights for policy 1, policy_version 100525 (0.0005) [2023-12-27 06:21:26,556][20381] Updated weights for policy 1, policy_version 100535 (0.0005) [2023-12-27 06:21:27,049][20347] Updated weights for policy 0, policy_version 100878 (0.0007) [2023-12-27 06:21:27,069][20381] Updated weights for policy 1, policy_version 100545 (0.0005) [2023-12-27 06:21:27,099][20347] Updated weights for policy 0, policy_version 100888 (0.0005) [2023-12-27 06:21:27,122][20381] Updated weights for policy 1, policy_version 100555 (0.0005) [2023-12-27 06:21:27,154][20347] Updated weights for policy 0, policy_version 100898 (0.0005) [2023-12-27 06:21:27,177][20381] Updated weights for policy 1, policy_version 100565 (0.0005) [2023-12-27 06:21:27,232][20381] Updated weights for policy 1, policy_version 100575 (0.0005) [2023-12-27 06:21:27,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 51601408. Throughput: 0: 10329.4, 1: 10429.8. Samples: 51584088. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:27,710][20347] Updated weights for policy 0, policy_version 100908 (0.0005) [2023-12-27 06:21:27,763][20347] Updated weights for policy 0, policy_version 100918 (0.0006) [2023-12-27 06:21:27,806][20381] Updated weights for policy 1, policy_version 100585 (0.0006) [2023-12-27 06:21:27,820][20347] Updated weights for policy 0, policy_version 100928 (0.0008) [2023-12-27 06:21:27,855][20381] Updated weights for policy 1, policy_version 100595 (0.0007) [2023-12-27 06:21:27,902][20381] Updated weights for policy 1, policy_version 100605 (0.0005) [2023-12-27 06:21:28,520][20347] Updated weights for policy 0, policy_version 100938 (0.0007) [2023-12-27 06:21:28,541][20381] Updated weights for policy 1, policy_version 100615 (0.0009) [2023-12-27 06:21:28,575][20347] Updated weights for policy 0, policy_version 100948 (0.0010) [2023-12-27 06:21:28,596][20381] Updated weights for policy 1, policy_version 100625 (0.0010) [2023-12-27 06:21:28,633][20347] Updated weights for policy 0, policy_version 100958 (0.0010) [2023-12-27 06:21:28,652][20381] Updated weights for policy 1, policy_version 100635 (0.0010) [2023-12-27 06:21:28,690][20347] Updated weights for policy 0, policy_version 100968 (0.0010) [2023-12-27 06:21:29,223][20381] Updated weights for policy 1, policy_version 100645 (0.0009) [2023-12-27 06:21:29,282][20381] Updated weights for policy 1, policy_version 100655 (0.0010) [2023-12-27 06:21:29,328][20347] Updated weights for policy 0, policy_version 100978 (0.0007) [2023-12-27 06:21:29,345][20381] Updated weights for policy 1, policy_version 100665 (0.0010) [2023-12-27 06:21:29,393][20347] Updated weights for policy 0, policy_version 100988 (0.0008) [2023-12-27 06:21:29,447][20347] Updated weights for policy 0, policy_version 100998 (0.0006) [2023-12-27 06:21:29,886][20381] Updated weights for policy 1, policy_version 100675 (0.0007) [2023-12-27 06:21:29,954][20381] Updated weights for policy 1, policy_version 100685 (0.0008) [2023-12-27 06:21:30,016][20381] Updated weights for policy 1, policy_version 100695 (0.0010) [2023-12-27 06:21:30,143][20347] Updated weights for policy 0, policy_version 101008 (0.0007) [2023-12-27 06:21:30,204][20347] Updated weights for policy 0, policy_version 101018 (0.0007) [2023-12-27 06:21:30,255][20347] Updated weights for policy 0, policy_version 101028 (0.0007) [2023-12-27 06:21:30,698][20381] Updated weights for policy 1, policy_version 100705 (0.0010) [2023-12-27 06:21:30,748][20381] Updated weights for policy 1, policy_version 100715 (0.0010) [2023-12-27 06:21:30,793][20381] Updated weights for policy 1, policy_version 100725 (0.0010) [2023-12-27 06:21:30,840][20381] Updated weights for policy 1, policy_version 100735 (0.0010) [2023-12-27 06:21:30,943][20347] Updated weights for policy 0, policy_version 101038 (0.0007) [2023-12-27 06:21:30,990][20347] Updated weights for policy 0, policy_version 101048 (0.0007) [2023-12-27 06:21:31,048][20347] Updated weights for policy 0, policy_version 101058 (0.0008) [2023-12-27 06:21:31,564][20381] Updated weights for policy 1, policy_version 100745 (0.0007) [2023-12-27 06:21:31,618][20381] Updated weights for policy 1, policy_version 100755 (0.0007) [2023-12-27 06:21:31,674][20381] Updated weights for policy 1, policy_version 100765 (0.0006) [2023-12-27 06:21:31,816][20347] Updated weights for policy 0, policy_version 101068 (0.0007) [2023-12-27 06:21:31,879][20347] Updated weights for policy 0, policy_version 101078 (0.0009) [2023-12-27 06:21:31,937][20347] Updated weights for policy 0, policy_version 101088 (0.0008) [2023-12-27 06:21:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 51699712. Throughput: 0: 10314.8, 1: 10484.0. Samples: 51712444. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:32,397][20381] Updated weights for policy 1, policy_version 100775 (0.0011) [2023-12-27 06:21:32,461][20381] Updated weights for policy 1, policy_version 100785 (0.0010) [2023-12-27 06:21:32,519][20381] Updated weights for policy 1, policy_version 100795 (0.0010) [2023-12-27 06:21:32,660][20347] Updated weights for policy 0, policy_version 101098 (0.0007) [2023-12-27 06:21:32,716][20347] Updated weights for policy 0, policy_version 101108 (0.0008) [2023-12-27 06:21:32,775][20347] Updated weights for policy 0, policy_version 101118 (0.0007) [2023-12-27 06:21:32,834][20347] Updated weights for policy 0, policy_version 101128 (0.0007) [2023-12-27 06:21:33,187][20381] Updated weights for policy 1, policy_version 100805 (0.0009) [2023-12-27 06:21:33,250][20381] Updated weights for policy 1, policy_version 100815 (0.0008) [2023-12-27 06:21:33,318][20381] Updated weights for policy 1, policy_version 100825 (0.0007) [2023-12-27 06:21:33,502][20347] Updated weights for policy 0, policy_version 101138 (0.0008) [2023-12-27 06:21:33,549][20347] Updated weights for policy 0, policy_version 101148 (0.0009) [2023-12-27 06:21:33,605][20347] Updated weights for policy 0, policy_version 101158 (0.0009) [2023-12-27 06:21:33,876][20381] Updated weights for policy 1, policy_version 100835 (0.0007) [2023-12-27 06:21:33,929][20381] Updated weights for policy 1, policy_version 100845 (0.0005) [2023-12-27 06:21:33,986][20381] Updated weights for policy 1, policy_version 100855 (0.0005) [2023-12-27 06:21:34,332][20347] Updated weights for policy 0, policy_version 101168 (0.0008) [2023-12-27 06:21:34,398][20347] Updated weights for policy 0, policy_version 101178 (0.0008) [2023-12-27 06:21:34,454][20347] Updated weights for policy 0, policy_version 101188 (0.0007) [2023-12-27 06:21:34,607][20381] Updated weights for policy 1, policy_version 100865 (0.0005) [2023-12-27 06:21:34,673][20381] Updated weights for policy 1, policy_version 100875 (0.0007) [2023-12-27 06:21:34,736][20381] Updated weights for policy 1, policy_version 100885 (0.0008) [2023-12-27 06:21:34,790][20381] Updated weights for policy 1, policy_version 100895 (0.0008) [2023-12-27 06:21:35,041][20347] Updated weights for policy 0, policy_version 101198 (0.0006) [2023-12-27 06:21:35,097][20347] Updated weights for policy 0, policy_version 101208 (0.0006) [2023-12-27 06:21:35,152][20347] Updated weights for policy 0, policy_version 101218 (0.0008) [2023-12-27 06:21:35,523][20381] Updated weights for policy 1, policy_version 100905 (0.0008) [2023-12-27 06:21:35,579][20381] Updated weights for policy 1, policy_version 100915 (0.0005) [2023-12-27 06:21:35,635][20381] Updated weights for policy 1, policy_version 100925 (0.0007) [2023-12-27 06:21:35,851][20347] Updated weights for policy 0, policy_version 101228 (0.0009) [2023-12-27 06:21:35,899][20347] Updated weights for policy 0, policy_version 101238 (0.0010) [2023-12-27 06:21:35,946][20347] Updated weights for policy 0, policy_version 101248 (0.0010) [2023-12-27 06:21:36,304][20381] Updated weights for policy 1, policy_version 100935 (0.0006) [2023-12-27 06:21:36,356][20381] Updated weights for policy 1, policy_version 100945 (0.0005) [2023-12-27 06:21:36,408][20381] Updated weights for policy 1, policy_version 100955 (0.0005) [2023-12-27 06:21:36,638][20347] Updated weights for policy 0, policy_version 101258 (0.0009) [2023-12-27 06:21:36,697][20347] Updated weights for policy 0, policy_version 101268 (0.0008) [2023-12-27 06:21:36,750][20347] Updated weights for policy 0, policy_version 101278 (0.0008) [2023-12-27 06:21:36,806][20347] Updated weights for policy 0, policy_version 101288 (0.0006) [2023-12-27 06:21:37,031][20381] Updated weights for policy 1, policy_version 100965 (0.0007) [2023-12-27 06:21:37,091][20381] Updated weights for policy 1, policy_version 100975 (0.0007) [2023-12-27 06:21:37,147][20381] Updated weights for policy 1, policy_version 100985 (0.0008) [2023-12-27 06:21:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 51806208. Throughput: 0: 10343.7, 1: 10478.4. Samples: 51775212. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000100992_25862144.pth... [2023-12-27 06:21:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000099744_25542656.pth [2023-12-27 06:21:37,426][20347] Updated weights for policy 0, policy_version 101298 (0.0005) [2023-12-27 06:21:37,489][20347] Updated weights for policy 0, policy_version 101308 (0.0005) [2023-12-27 06:21:37,546][20347] Updated weights for policy 0, policy_version 101318 (0.0005) [2023-12-27 06:21:37,556][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000101320_25952256.pth... [2023-12-27 06:21:37,560][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000100104_25640960.pth [2023-12-27 06:21:37,915][20381] Updated weights for policy 1, policy_version 100995 (0.0008) [2023-12-27 06:21:37,979][20381] Updated weights for policy 1, policy_version 101005 (0.0008) [2023-12-27 06:21:38,040][20381] Updated weights for policy 1, policy_version 101015 (0.0008) [2023-12-27 06:21:38,083][20347] Updated weights for policy 0, policy_version 101328 (0.0006) [2023-12-27 06:21:38,145][20347] Updated weights for policy 0, policy_version 101338 (0.0006) [2023-12-27 06:21:38,200][20347] Updated weights for policy 0, policy_version 101348 (0.0007) [2023-12-27 06:21:38,716][20381] Updated weights for policy 1, policy_version 101025 (0.0008) [2023-12-27 06:21:38,775][20381] Updated weights for policy 1, policy_version 101035 (0.0009) [2023-12-27 06:21:38,788][20347] Updated weights for policy 0, policy_version 101358 (0.0007) [2023-12-27 06:21:38,834][20347] Updated weights for policy 0, policy_version 101368 (0.0006) [2023-12-27 06:21:38,837][20381] Updated weights for policy 1, policy_version 101045 (0.0007) [2023-12-27 06:21:38,895][20381] Updated weights for policy 1, policy_version 101055 (0.0008) [2023-12-27 06:21:38,899][20347] Updated weights for policy 0, policy_version 101378 (0.0005) [2023-12-27 06:21:39,459][20347] Updated weights for policy 0, policy_version 101388 (0.0006) [2023-12-27 06:21:39,514][20347] Updated weights for policy 0, policy_version 101398 (0.0007) [2023-12-27 06:21:39,570][20347] Updated weights for policy 0, policy_version 101408 (0.0007) [2023-12-27 06:21:39,639][20381] Updated weights for policy 1, policy_version 101065 (0.0007) [2023-12-27 06:21:39,698][20381] Updated weights for policy 1, policy_version 101075 (0.0008) [2023-12-27 06:21:39,749][20381] Updated weights for policy 1, policy_version 101085 (0.0008) [2023-12-27 06:21:40,284][20347] Updated weights for policy 0, policy_version 101418 (0.0007) [2023-12-27 06:21:40,347][20347] Updated weights for policy 0, policy_version 101428 (0.0006) [2023-12-27 06:21:40,409][20347] Updated weights for policy 0, policy_version 101438 (0.0007) [2023-12-27 06:21:40,445][20381] Updated weights for policy 1, policy_version 101095 (0.0007) [2023-12-27 06:21:40,463][20347] Updated weights for policy 0, policy_version 101448 (0.0006) [2023-12-27 06:21:40,505][20381] Updated weights for policy 1, policy_version 101105 (0.0007) [2023-12-27 06:21:40,558][20381] Updated weights for policy 1, policy_version 101115 (0.0006) [2023-12-27 06:21:41,126][20347] Updated weights for policy 0, policy_version 101458 (0.0007) [2023-12-27 06:21:41,185][20347] Updated weights for policy 0, policy_version 101468 (0.0008) [2023-12-27 06:21:41,246][20347] Updated weights for policy 0, policy_version 101478 (0.0007) [2023-12-27 06:21:41,256][20381] Updated weights for policy 1, policy_version 101125 (0.0008) [2023-12-27 06:21:41,319][20381] Updated weights for policy 1, policy_version 101135 (0.0011) [2023-12-27 06:21:41,382][20381] Updated weights for policy 1, policy_version 101145 (0.0011) [2023-12-27 06:21:41,977][20347] Updated weights for policy 0, policy_version 101488 (0.0007) [2023-12-27 06:21:42,044][20347] Updated weights for policy 0, policy_version 101498 (0.0007) [2023-12-27 06:21:42,067][20381] Updated weights for policy 1, policy_version 101155 (0.0008) [2023-12-27 06:21:42,108][20347] Updated weights for policy 0, policy_version 101508 (0.0005) [2023-12-27 06:21:42,119][20381] Updated weights for policy 1, policy_version 101165 (0.0007) [2023-12-27 06:21:42,178][20381] Updated weights for policy 1, policy_version 101175 (0.0009) [2023-12-27 06:21:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 51912704. Throughput: 0: 10383.8, 1: 10436.5. Samples: 51899724. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:42,749][20347] Updated weights for policy 0, policy_version 101518 (0.0010) [2023-12-27 06:21:42,807][20347] Updated weights for policy 0, policy_version 101528 (0.0010) [2023-12-27 06:21:42,849][20381] Updated weights for policy 1, policy_version 101185 (0.0010) [2023-12-27 06:21:42,870][20347] Updated weights for policy 0, policy_version 101538 (0.0011) [2023-12-27 06:21:42,906][20381] Updated weights for policy 1, policy_version 101195 (0.0008) [2023-12-27 06:21:42,961][20381] Updated weights for policy 1, policy_version 101205 (0.0006) [2023-12-27 06:21:43,012][20381] Updated weights for policy 1, policy_version 101215 (0.0005) [2023-12-27 06:21:43,445][20347] Updated weights for policy 0, policy_version 101548 (0.0006) [2023-12-27 06:21:43,503][20347] Updated weights for policy 0, policy_version 101558 (0.0007) [2023-12-27 06:21:43,552][20347] Updated weights for policy 0, policy_version 101568 (0.0008) [2023-12-27 06:21:43,685][20381] Updated weights for policy 1, policy_version 101225 (0.0007) [2023-12-27 06:21:43,740][20381] Updated weights for policy 1, policy_version 101235 (0.0008) [2023-12-27 06:21:43,800][20381] Updated weights for policy 1, policy_version 101245 (0.0008) [2023-12-27 06:21:44,154][20347] Updated weights for policy 0, policy_version 101578 (0.0007) [2023-12-27 06:21:44,206][20347] Updated weights for policy 0, policy_version 101588 (0.0005) [2023-12-27 06:21:44,260][20347] Updated weights for policy 0, policy_version 101598 (0.0008) [2023-12-27 06:21:44,308][20347] Updated weights for policy 0, policy_version 101608 (0.0007) [2023-12-27 06:21:44,525][20381] Updated weights for policy 1, policy_version 101255 (0.0007) [2023-12-27 06:21:44,581][20381] Updated weights for policy 1, policy_version 101265 (0.0008) [2023-12-27 06:21:44,637][20381] Updated weights for policy 1, policy_version 101275 (0.0008) [2023-12-27 06:21:44,979][20347] Updated weights for policy 0, policy_version 101618 (0.0008) [2023-12-27 06:21:45,039][20347] Updated weights for policy 0, policy_version 101628 (0.0008) [2023-12-27 06:21:45,094][20347] Updated weights for policy 0, policy_version 101638 (0.0009) [2023-12-27 06:21:45,318][20381] Updated weights for policy 1, policy_version 101285 (0.0008) [2023-12-27 06:21:45,382][20381] Updated weights for policy 1, policy_version 101295 (0.0008) [2023-12-27 06:21:45,442][20381] Updated weights for policy 1, policy_version 101305 (0.0007) [2023-12-27 06:21:45,762][20347] Updated weights for policy 0, policy_version 101648 (0.0005) [2023-12-27 06:21:45,819][20347] Updated weights for policy 0, policy_version 101658 (0.0005) [2023-12-27 06:21:45,875][20347] Updated weights for policy 0, policy_version 101668 (0.0006) [2023-12-27 06:21:46,142][20381] Updated weights for policy 1, policy_version 101315 (0.0007) [2023-12-27 06:21:46,195][20381] Updated weights for policy 1, policy_version 101325 (0.0005) [2023-12-27 06:21:46,254][20381] Updated weights for policy 1, policy_version 101335 (0.0006) [2023-12-27 06:21:46,456][20347] Updated weights for policy 0, policy_version 101678 (0.0007) [2023-12-27 06:21:46,506][20347] Updated weights for policy 0, policy_version 101688 (0.0005) [2023-12-27 06:21:46,566][20347] Updated weights for policy 0, policy_version 101698 (0.0006) [2023-12-27 06:21:46,808][20381] Updated weights for policy 1, policy_version 101345 (0.0005) [2023-12-27 06:21:46,866][20381] Updated weights for policy 1, policy_version 101355 (0.0009) [2023-12-27 06:21:46,914][20381] Updated weights for policy 1, policy_version 101365 (0.0010) [2023-12-27 06:21:46,963][20381] Updated weights for policy 1, policy_version 101375 (0.0010) [2023-12-27 06:21:47,098][20347] Updated weights for policy 0, policy_version 101708 (0.0005) [2023-12-27 06:21:47,159][20347] Updated weights for policy 0, policy_version 101718 (0.0005) [2023-12-27 06:21:47,219][20347] Updated weights for policy 0, policy_version 101728 (0.0008) [2023-12-27 06:21:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 52019200. Throughput: 0: 10498.7, 1: 10475.9. Samples: 52026968. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:47,608][20381] Updated weights for policy 1, policy_version 101385 (0.0011) [2023-12-27 06:21:47,670][20381] Updated weights for policy 1, policy_version 101395 (0.0006) [2023-12-27 06:21:47,730][20381] Updated weights for policy 1, policy_version 101405 (0.0005) [2023-12-27 06:21:47,790][20347] Updated weights for policy 0, policy_version 101738 (0.0010) [2023-12-27 06:21:47,850][20347] Updated weights for policy 0, policy_version 101748 (0.0011) [2023-12-27 06:21:47,909][20347] Updated weights for policy 0, policy_version 101758 (0.0009) [2023-12-27 06:21:47,958][20347] Updated weights for policy 0, policy_version 101768 (0.0006) [2023-12-27 06:21:48,289][20381] Updated weights for policy 1, policy_version 101415 (0.0007) [2023-12-27 06:21:48,349][20381] Updated weights for policy 1, policy_version 101425 (0.0007) [2023-12-27 06:21:48,420][20381] Updated weights for policy 1, policy_version 101435 (0.0008) [2023-12-27 06:21:48,597][20347] Updated weights for policy 0, policy_version 101778 (0.0010) [2023-12-27 06:21:48,657][20347] Updated weights for policy 0, policy_version 101788 (0.0011) [2023-12-27 06:21:48,712][20347] Updated weights for policy 0, policy_version 101798 (0.0010) [2023-12-27 06:21:49,115][20381] Updated weights for policy 1, policy_version 101445 (0.0009) [2023-12-27 06:21:49,181][20381] Updated weights for policy 1, policy_version 101455 (0.0009) [2023-12-27 06:21:49,235][20381] Updated weights for policy 1, policy_version 101465 (0.0008) [2023-12-27 06:21:49,411][20347] Updated weights for policy 0, policy_version 101808 (0.0007) [2023-12-27 06:21:49,465][20347] Updated weights for policy 0, policy_version 101818 (0.0009) [2023-12-27 06:21:49,525][20347] Updated weights for policy 0, policy_version 101828 (0.0010) [2023-12-27 06:21:49,907][20381] Updated weights for policy 1, policy_version 101475 (0.0007) [2023-12-27 06:21:49,971][20381] Updated weights for policy 1, policy_version 101485 (0.0007) [2023-12-27 06:21:50,033][20381] Updated weights for policy 1, policy_version 101495 (0.0008) [2023-12-27 06:21:50,195][20347] Updated weights for policy 0, policy_version 101838 (0.0010) [2023-12-27 06:21:50,256][20347] Updated weights for policy 0, policy_version 101848 (0.0010) [2023-12-27 06:21:50,315][20347] Updated weights for policy 0, policy_version 101858 (0.0011) [2023-12-27 06:21:50,752][20381] Updated weights for policy 1, policy_version 101505 (0.0007) [2023-12-27 06:21:50,811][20381] Updated weights for policy 1, policy_version 101515 (0.0007) [2023-12-27 06:21:50,874][20381] Updated weights for policy 1, policy_version 101525 (0.0005) [2023-12-27 06:21:50,922][20381] Updated weights for policy 1, policy_version 101535 (0.0005) [2023-12-27 06:21:50,923][20347] Updated weights for policy 0, policy_version 101868 (0.0010) [2023-12-27 06:21:50,982][20347] Updated weights for policy 0, policy_version 101878 (0.0010) [2023-12-27 06:21:51,040][20347] Updated weights for policy 0, policy_version 101888 (0.0010) [2023-12-27 06:21:51,620][20381] Updated weights for policy 1, policy_version 101545 (0.0009) [2023-12-27 06:21:51,678][20381] Updated weights for policy 1, policy_version 101555 (0.0009) [2023-12-27 06:21:51,743][20381] Updated weights for policy 1, policy_version 101565 (0.0009) [2023-12-27 06:21:51,848][20347] Updated weights for policy 0, policy_version 101898 (0.0010) [2023-12-27 06:21:51,901][20347] Updated weights for policy 0, policy_version 101908 (0.0009) [2023-12-27 06:21:51,957][20347] Updated weights for policy 0, policy_version 101918 (0.0007) [2023-12-27 06:21:52,018][20347] Updated weights for policy 0, policy_version 101928 (0.0008) [2023-12-27 06:21:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 52117504. Throughput: 0: 10484.0, 1: 10525.7. Samples: 52089960. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000101568_26009600.pth... [2023-12-27 06:21:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000101928_26107904.pth... [2023-12-27 06:21:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000100352_25698304.pth [2023-12-27 06:21:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000100712_25796608.pth [2023-12-27 06:21:52,503][20381] Updated weights for policy 1, policy_version 101575 (0.0008) [2023-12-27 06:21:52,563][20381] Updated weights for policy 1, policy_version 101585 (0.0008) [2023-12-27 06:21:52,619][20381] Updated weights for policy 1, policy_version 101595 (0.0007) [2023-12-27 06:21:52,661][20347] Updated weights for policy 0, policy_version 101938 (0.0010) [2023-12-27 06:21:52,716][20347] Updated weights for policy 0, policy_version 101948 (0.0010) [2023-12-27 06:21:52,774][20347] Updated weights for policy 0, policy_version 101958 (0.0010) [2023-12-27 06:21:53,334][20381] Updated weights for policy 1, policy_version 101605 (0.0007) [2023-12-27 06:21:53,394][20381] Updated weights for policy 1, policy_version 101615 (0.0008) [2023-12-27 06:21:53,440][20347] Updated weights for policy 0, policy_version 101968 (0.0007) [2023-12-27 06:21:53,453][20381] Updated weights for policy 1, policy_version 101625 (0.0005) [2023-12-27 06:21:53,499][20347] Updated weights for policy 0, policy_version 101978 (0.0006) [2023-12-27 06:21:53,553][20347] Updated weights for policy 0, policy_version 101988 (0.0008) [2023-12-27 06:21:54,096][20381] Updated weights for policy 1, policy_version 101635 (0.0005) [2023-12-27 06:21:54,150][20381] Updated weights for policy 1, policy_version 101645 (0.0007) [2023-12-27 06:21:54,209][20347] Updated weights for policy 0, policy_version 101998 (0.0007) [2023-12-27 06:21:54,210][20381] Updated weights for policy 1, policy_version 101655 (0.0007) [2023-12-27 06:21:54,264][20347] Updated weights for policy 0, policy_version 102008 (0.0008) [2023-12-27 06:21:54,312][20347] Updated weights for policy 0, policy_version 102018 (0.0007) [2023-12-27 06:21:54,914][20381] Updated weights for policy 1, policy_version 101665 (0.0007) [2023-12-27 06:21:54,934][20347] Updated weights for policy 0, policy_version 102028 (0.0007) [2023-12-27 06:21:54,972][20381] Updated weights for policy 1, policy_version 101675 (0.0006) [2023-12-27 06:21:54,988][20347] Updated weights for policy 0, policy_version 102038 (0.0006) [2023-12-27 06:21:55,027][20381] Updated weights for policy 1, policy_version 101685 (0.0006) [2023-12-27 06:21:55,048][20347] Updated weights for policy 0, policy_version 102048 (0.0008) [2023-12-27 06:21:55,083][20381] Updated weights for policy 1, policy_version 101695 (0.0005) [2023-12-27 06:21:55,612][20381] Updated weights for policy 1, policy_version 101705 (0.0005) [2023-12-27 06:21:55,673][20381] Updated weights for policy 1, policy_version 101715 (0.0007) [2023-12-27 06:21:55,726][20347] Updated weights for policy 0, policy_version 102058 (0.0008) [2023-12-27 06:21:55,730][20381] Updated weights for policy 1, policy_version 101725 (0.0008) [2023-12-27 06:21:55,780][20347] Updated weights for policy 0, policy_version 102068 (0.0010) [2023-12-27 06:21:55,834][20347] Updated weights for policy 0, policy_version 102078 (0.0010) [2023-12-27 06:21:55,887][20347] Updated weights for policy 0, policy_version 102088 (0.0010) [2023-12-27 06:21:56,391][20381] Updated weights for policy 1, policy_version 101735 (0.0007) [2023-12-27 06:21:56,439][20381] Updated weights for policy 1, policy_version 101745 (0.0007) [2023-12-27 06:21:56,489][20381] Updated weights for policy 1, policy_version 101755 (0.0007) [2023-12-27 06:21:56,558][20347] Updated weights for policy 0, policy_version 102098 (0.0010) [2023-12-27 06:21:56,610][20347] Updated weights for policy 0, policy_version 102108 (0.0010) [2023-12-27 06:21:56,661][20347] Updated weights for policy 0, policy_version 102118 (0.0010) [2023-12-27 06:21:57,199][20381] Updated weights for policy 1, policy_version 101765 (0.0007) [2023-12-27 06:21:57,252][20381] Updated weights for policy 1, policy_version 101775 (0.0007) [2023-12-27 06:21:57,292][20347] Updated weights for policy 0, policy_version 102128 (0.0009) [2023-12-27 06:21:57,309][20381] Updated weights for policy 1, policy_version 101785 (0.0006) [2023-12-27 06:21:57,347][20347] Updated weights for policy 0, policy_version 102138 (0.0010) [2023-12-27 06:21:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 52224000. Throughput: 0: 10454.3, 1: 10475.8. Samples: 52213144. Policy #0 lag: (min: 31.0, avg: 41.9, max: 63.0) [2023-12-27 06:21:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:21:57,394][20347] Updated weights for policy 0, policy_version 102148 (0.0010) [2023-12-27 06:21:58,002][20381] Updated weights for policy 1, policy_version 101795 (0.0008) [2023-12-27 06:21:58,048][20381] Updated weights for policy 1, policy_version 101805 (0.0006) [2023-12-27 06:21:58,055][20347] Updated weights for policy 0, policy_version 102158 (0.0010) [2023-12-27 06:21:58,096][20381] Updated weights for policy 1, policy_version 101815 (0.0005) [2023-12-27 06:21:58,109][20347] Updated weights for policy 0, policy_version 102168 (0.0010) [2023-12-27 06:21:58,168][20347] Updated weights for policy 0, policy_version 102178 (0.0009) [2023-12-27 06:21:58,870][20381] Updated weights for policy 1, policy_version 101825 (0.0006) [2023-12-27 06:21:58,894][20347] Updated weights for policy 0, policy_version 102188 (0.0008) [2023-12-27 06:21:58,934][20381] Updated weights for policy 1, policy_version 101835 (0.0008) [2023-12-27 06:21:58,951][20347] Updated weights for policy 0, policy_version 102198 (0.0006) [2023-12-27 06:21:58,992][20381] Updated weights for policy 1, policy_version 101845 (0.0007) [2023-12-27 06:21:59,007][20347] Updated weights for policy 0, policy_version 102208 (0.0005) [2023-12-27 06:21:59,053][20381] Updated weights for policy 1, policy_version 101855 (0.0007) [2023-12-27 06:21:59,630][20347] Updated weights for policy 0, policy_version 102218 (0.0007) [2023-12-27 06:21:59,688][20347] Updated weights for policy 0, policy_version 102228 (0.0007) [2023-12-27 06:21:59,741][20381] Updated weights for policy 1, policy_version 101865 (0.0010) [2023-12-27 06:21:59,744][20347] Updated weights for policy 0, policy_version 102238 (0.0008) [2023-12-27 06:21:59,789][20381] Updated weights for policy 1, policy_version 101875 (0.0010) [2023-12-27 06:21:59,794][20347] Updated weights for policy 0, policy_version 102248 (0.0010) [2023-12-27 06:21:59,849][20381] Updated weights for policy 1, policy_version 101885 (0.0007) [2023-12-27 06:22:00,400][20347] Updated weights for policy 0, policy_version 102258 (0.0008) [2023-12-27 06:22:00,462][20347] Updated weights for policy 0, policy_version 102268 (0.0008) [2023-12-27 06:22:00,511][20381] Updated weights for policy 1, policy_version 101895 (0.0006) [2023-12-27 06:22:00,513][20347] Updated weights for policy 0, policy_version 102278 (0.0007) [2023-12-27 06:22:00,567][20381] Updated weights for policy 1, policy_version 101905 (0.0008) [2023-12-27 06:22:00,627][20381] Updated weights for policy 1, policy_version 101915 (0.0007) [2023-12-27 06:22:01,114][20347] Updated weights for policy 0, policy_version 102288 (0.0007) [2023-12-27 06:22:01,176][20347] Updated weights for policy 0, policy_version 102298 (0.0008) [2023-12-27 06:22:01,228][20347] Updated weights for policy 0, policy_version 102308 (0.0007) [2023-12-27 06:22:01,271][20381] Updated weights for policy 1, policy_version 101925 (0.0009) [2023-12-27 06:22:01,323][20381] Updated weights for policy 1, policy_version 101935 (0.0008) [2023-12-27 06:22:01,412][20381] Updated weights for policy 1, policy_version 101945 (0.0008) [2023-12-27 06:22:01,961][20347] Updated weights for policy 0, policy_version 102318 (0.0007) [2023-12-27 06:22:02,017][20347] Updated weights for policy 0, policy_version 102328 (0.0006) [2023-12-27 06:22:02,066][20347] Updated weights for policy 0, policy_version 102338 (0.0005) [2023-12-27 06:22:02,081][20381] Updated weights for policy 1, policy_version 101955 (0.0009) [2023-12-27 06:22:02,135][20381] Updated weights for policy 1, policy_version 101965 (0.0007) [2023-12-27 06:22:02,187][20381] Updated weights for policy 1, policy_version 101975 (0.0006) [2023-12-27 06:22:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 52330496. Throughput: 0: 10511.2, 1: 10403.6. Samples: 52336752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:02,684][20347] Updated weights for policy 0, policy_version 102348 (0.0005) [2023-12-27 06:22:02,746][20347] Updated weights for policy 0, policy_version 102358 (0.0006) [2023-12-27 06:22:02,816][20347] Updated weights for policy 0, policy_version 102368 (0.0006) [2023-12-27 06:22:02,875][20381] Updated weights for policy 1, policy_version 101985 (0.0006) [2023-12-27 06:22:02,931][20381] Updated weights for policy 1, policy_version 101995 (0.0008) [2023-12-27 06:22:02,993][20381] Updated weights for policy 1, policy_version 102005 (0.0007) [2023-12-27 06:22:03,048][20381] Updated weights for policy 1, policy_version 102015 (0.0006) [2023-12-27 06:22:03,391][20347] Updated weights for policy 0, policy_version 102378 (0.0009) [2023-12-27 06:22:03,447][20347] Updated weights for policy 0, policy_version 102388 (0.0008) [2023-12-27 06:22:03,505][20347] Updated weights for policy 0, policy_version 102398 (0.0007) [2023-12-27 06:22:03,553][20347] Updated weights for policy 0, policy_version 102408 (0.0007) [2023-12-27 06:22:03,709][20381] Updated weights for policy 1, policy_version 102025 (0.0006) [2023-12-27 06:22:03,768][20381] Updated weights for policy 1, policy_version 102035 (0.0005) [2023-12-27 06:22:03,828][20381] Updated weights for policy 1, policy_version 102045 (0.0009) [2023-12-27 06:22:04,275][20347] Updated weights for policy 0, policy_version 102418 (0.0006) [2023-12-27 06:22:04,325][20347] Updated weights for policy 0, policy_version 102428 (0.0008) [2023-12-27 06:22:04,378][20347] Updated weights for policy 0, policy_version 102438 (0.0008) [2023-12-27 06:22:04,435][20381] Updated weights for policy 1, policy_version 102055 (0.0007) [2023-12-27 06:22:04,498][20381] Updated weights for policy 1, policy_version 102065 (0.0007) [2023-12-27 06:22:04,557][20381] Updated weights for policy 1, policy_version 102075 (0.0006) [2023-12-27 06:22:05,057][20347] Updated weights for policy 0, policy_version 102448 (0.0006) [2023-12-27 06:22:05,108][20347] Updated weights for policy 0, policy_version 102458 (0.0005) [2023-12-27 06:22:05,160][20347] Updated weights for policy 0, policy_version 102468 (0.0006) [2023-12-27 06:22:05,243][20381] Updated weights for policy 1, policy_version 102085 (0.0008) [2023-12-27 06:22:05,298][20381] Updated weights for policy 1, policy_version 102095 (0.0007) [2023-12-27 06:22:05,349][20381] Updated weights for policy 1, policy_version 102105 (0.0007) [2023-12-27 06:22:05,771][20347] Updated weights for policy 0, policy_version 102478 (0.0008) [2023-12-27 06:22:05,824][20347] Updated weights for policy 0, policy_version 102488 (0.0005) [2023-12-27 06:22:05,880][20347] Updated weights for policy 0, policy_version 102498 (0.0005) [2023-12-27 06:22:06,037][20381] Updated weights for policy 1, policy_version 102115 (0.0008) [2023-12-27 06:22:06,101][20381] Updated weights for policy 1, policy_version 102125 (0.0008) [2023-12-27 06:22:06,157][20381] Updated weights for policy 1, policy_version 102135 (0.0008) [2023-12-27 06:22:06,517][20347] Updated weights for policy 0, policy_version 102508 (0.0006) [2023-12-27 06:22:06,581][20347] Updated weights for policy 0, policy_version 102518 (0.0007) [2023-12-27 06:22:06,640][20347] Updated weights for policy 0, policy_version 102528 (0.0008) [2023-12-27 06:22:06,863][20381] Updated weights for policy 1, policy_version 102145 (0.0007) [2023-12-27 06:22:06,920][20381] Updated weights for policy 1, policy_version 102155 (0.0008) [2023-12-27 06:22:06,979][20381] Updated weights for policy 1, policy_version 102165 (0.0007) [2023-12-27 06:22:07,041][20381] Updated weights for policy 1, policy_version 102175 (0.0008) [2023-12-27 06:22:07,306][20347] Updated weights for policy 0, policy_version 102538 (0.0007) [2023-12-27 06:22:07,362][20347] Updated weights for policy 0, policy_version 102548 (0.0005) [2023-12-27 06:22:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 52428800. Throughput: 0: 10541.9, 1: 10420.5. Samples: 52400560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000102176_26165248.pth... [2023-12-27 06:22:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000100992_25862144.pth [2023-12-27 06:22:07,415][20347] Updated weights for policy 0, policy_version 102558 (0.0005) [2023-12-27 06:22:07,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000102568_26271744.pth... [2023-12-27 06:22:07,462][20347] Updated weights for policy 0, policy_version 102568 (0.0005) [2023-12-27 06:22:07,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000101320_25952256.pth [2023-12-27 06:22:07,676][20381] Updated weights for policy 1, policy_version 102185 (0.0006) [2023-12-27 06:22:07,741][20381] Updated weights for policy 1, policy_version 102195 (0.0005) [2023-12-27 06:22:07,804][20381] Updated weights for policy 1, policy_version 102205 (0.0007) [2023-12-27 06:22:08,058][20347] Updated weights for policy 0, policy_version 102578 (0.0010) [2023-12-27 06:22:08,118][20347] Updated weights for policy 0, policy_version 102588 (0.0011) [2023-12-27 06:22:08,179][20347] Updated weights for policy 0, policy_version 102598 (0.0010) [2023-12-27 06:22:08,440][20381] Updated weights for policy 1, policy_version 102215 (0.0006) [2023-12-27 06:22:08,500][20381] Updated weights for policy 1, policy_version 102225 (0.0008) [2023-12-27 06:22:08,560][20381] Updated weights for policy 1, policy_version 102235 (0.0008) [2023-12-27 06:22:08,797][20347] Updated weights for policy 0, policy_version 102608 (0.0007) [2023-12-27 06:22:08,852][20347] Updated weights for policy 0, policy_version 102618 (0.0007) [2023-12-27 06:22:08,902][20347] Updated weights for policy 0, policy_version 102628 (0.0008) [2023-12-27 06:22:09,241][20381] Updated weights for policy 1, policy_version 102245 (0.0007) [2023-12-27 06:22:09,307][20381] Updated weights for policy 1, policy_version 102255 (0.0009) [2023-12-27 06:22:09,371][20381] Updated weights for policy 1, policy_version 102265 (0.0007) [2023-12-27 06:22:09,589][20347] Updated weights for policy 0, policy_version 102638 (0.0007) [2023-12-27 06:22:09,642][20347] Updated weights for policy 0, policy_version 102648 (0.0007) [2023-12-27 06:22:09,696][20347] Updated weights for policy 0, policy_version 102658 (0.0007) [2023-12-27 06:22:10,040][20381] Updated weights for policy 1, policy_version 102275 (0.0008) [2023-12-27 06:22:10,097][20381] Updated weights for policy 1, policy_version 102285 (0.0006) [2023-12-27 06:22:10,147][20381] Updated weights for policy 1, policy_version 102295 (0.0006) [2023-12-27 06:22:10,453][20347] Updated weights for policy 0, policy_version 102668 (0.0008) [2023-12-27 06:22:10,513][20347] Updated weights for policy 0, policy_version 102678 (0.0007) [2023-12-27 06:22:10,577][20347] Updated weights for policy 0, policy_version 102688 (0.0008) [2023-12-27 06:22:10,732][20381] Updated weights for policy 1, policy_version 102305 (0.0005) [2023-12-27 06:22:10,787][20381] Updated weights for policy 1, policy_version 102315 (0.0008) [2023-12-27 06:22:10,842][20381] Updated weights for policy 1, policy_version 102325 (0.0007) [2023-12-27 06:22:10,897][20381] Updated weights for policy 1, policy_version 102335 (0.0007) [2023-12-27 06:22:11,252][20347] Updated weights for policy 0, policy_version 102698 (0.0008) [2023-12-27 06:22:11,312][20347] Updated weights for policy 0, policy_version 102708 (0.0010) [2023-12-27 06:22:11,380][20347] Updated weights for policy 0, policy_version 102718 (0.0010) [2023-12-27 06:22:11,436][20347] Updated weights for policy 0, policy_version 102728 (0.0010) [2023-12-27 06:22:11,651][20381] Updated weights for policy 1, policy_version 102345 (0.0008) [2023-12-27 06:22:11,708][20381] Updated weights for policy 1, policy_version 102355 (0.0007) [2023-12-27 06:22:11,775][20381] Updated weights for policy 1, policy_version 102365 (0.0008) [2023-12-27 06:22:12,142][20347] Updated weights for policy 0, policy_version 102738 (0.0008) [2023-12-27 06:22:12,199][20347] Updated weights for policy 0, policy_version 102748 (0.0008) [2023-12-27 06:22:12,258][20347] Updated weights for policy 0, policy_version 102758 (0.0007) [2023-12-27 06:22:12,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 52535296. Throughput: 0: 10565.5, 1: 10343.3. Samples: 52524988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:12,513][20381] Updated weights for policy 1, policy_version 102375 (0.0008) [2023-12-27 06:22:12,560][20381] Updated weights for policy 1, policy_version 102385 (0.0009) [2023-12-27 06:22:12,614][20381] Updated weights for policy 1, policy_version 102395 (0.0009) [2023-12-27 06:22:12,898][20347] Updated weights for policy 0, policy_version 102768 (0.0010) [2023-12-27 06:22:12,945][20347] Updated weights for policy 0, policy_version 102778 (0.0009) [2023-12-27 06:22:12,996][20347] Updated weights for policy 0, policy_version 102788 (0.0005) [2023-12-27 06:22:13,316][20381] Updated weights for policy 1, policy_version 102405 (0.0007) [2023-12-27 06:22:13,366][20381] Updated weights for policy 1, policy_version 102415 (0.0005) [2023-12-27 06:22:13,424][20381] Updated weights for policy 1, policy_version 102425 (0.0007) [2023-12-27 06:22:13,579][20347] Updated weights for policy 0, policy_version 102798 (0.0008) [2023-12-27 06:22:13,636][20347] Updated weights for policy 0, policy_version 102808 (0.0010) [2023-12-27 06:22:13,694][20347] Updated weights for policy 0, policy_version 102818 (0.0008) [2023-12-27 06:22:14,031][20381] Updated weights for policy 1, policy_version 102435 (0.0008) [2023-12-27 06:22:14,076][20381] Updated weights for policy 1, policy_version 102445 (0.0010) [2023-12-27 06:22:14,130][20381] Updated weights for policy 1, policy_version 102455 (0.0010) [2023-12-27 06:22:14,330][20347] Updated weights for policy 0, policy_version 102828 (0.0008) [2023-12-27 06:22:14,384][20347] Updated weights for policy 0, policy_version 102838 (0.0009) [2023-12-27 06:22:14,440][20347] Updated weights for policy 0, policy_version 102848 (0.0010) [2023-12-27 06:22:14,814][20381] Updated weights for policy 1, policy_version 102465 (0.0010) [2023-12-27 06:22:14,877][20381] Updated weights for policy 1, policy_version 102475 (0.0008) [2023-12-27 06:22:14,937][20381] Updated weights for policy 1, policy_version 102485 (0.0008) [2023-12-27 06:22:14,997][20381] Updated weights for policy 1, policy_version 102495 (0.0008) [2023-12-27 06:22:15,081][20347] Updated weights for policy 0, policy_version 102858 (0.0009) [2023-12-27 06:22:15,141][20347] Updated weights for policy 0, policy_version 102868 (0.0005) [2023-12-27 06:22:15,203][20347] Updated weights for policy 0, policy_version 102878 (0.0006) [2023-12-27 06:22:15,266][20347] Updated weights for policy 0, policy_version 102888 (0.0006) [2023-12-27 06:22:15,709][20381] Updated weights for policy 1, policy_version 102505 (0.0010) [2023-12-27 06:22:15,768][20381] Updated weights for policy 1, policy_version 102515 (0.0011) [2023-12-27 06:22:15,821][20381] Updated weights for policy 1, policy_version 102525 (0.0011) [2023-12-27 06:22:15,873][20347] Updated weights for policy 0, policy_version 102898 (0.0006) [2023-12-27 06:22:15,925][20347] Updated weights for policy 0, policy_version 102908 (0.0005) [2023-12-27 06:22:15,990][20347] Updated weights for policy 0, policy_version 102918 (0.0005) [2023-12-27 06:22:16,495][20381] Updated weights for policy 1, policy_version 102535 (0.0011) [2023-12-27 06:22:16,550][20381] Updated weights for policy 1, policy_version 102545 (0.0011) [2023-12-27 06:22:16,558][20347] Updated weights for policy 0, policy_version 102928 (0.0005) [2023-12-27 06:22:16,606][20381] Updated weights for policy 1, policy_version 102555 (0.0011) [2023-12-27 06:22:16,610][20347] Updated weights for policy 0, policy_version 102938 (0.0005) [2023-12-27 06:22:16,660][20347] Updated weights for policy 0, policy_version 102948 (0.0007) [2023-12-27 06:22:17,312][20381] Updated weights for policy 1, policy_version 102565 (0.0009) [2023-12-27 06:22:17,345][20347] Updated weights for policy 0, policy_version 102958 (0.0009) [2023-12-27 06:22:17,360][20381] Updated weights for policy 1, policy_version 102575 (0.0008) [2023-12-27 06:22:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 52633600. Throughput: 0: 10642.6, 1: 10205.4. Samples: 52650604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:17,400][20347] Updated weights for policy 0, policy_version 102968 (0.0006) [2023-12-27 06:22:17,412][20381] Updated weights for policy 1, policy_version 102585 (0.0007) [2023-12-27 06:22:17,459][20347] Updated weights for policy 0, policy_version 102978 (0.0005) [2023-12-27 06:22:18,006][20347] Updated weights for policy 0, policy_version 102988 (0.0006) [2023-12-27 06:22:18,062][20347] Updated weights for policy 0, policy_version 102998 (0.0007) [2023-12-27 06:22:18,092][20381] Updated weights for policy 1, policy_version 102595 (0.0009) [2023-12-27 06:22:18,124][20347] Updated weights for policy 0, policy_version 103008 (0.0006) [2023-12-27 06:22:18,151][20381] Updated weights for policy 1, policy_version 102605 (0.0008) [2023-12-27 06:22:18,202][20381] Updated weights for policy 1, policy_version 102615 (0.0007) [2023-12-27 06:22:18,756][20347] Updated weights for policy 0, policy_version 103018 (0.0005) [2023-12-27 06:22:18,820][20347] Updated weights for policy 0, policy_version 103028 (0.0006) [2023-12-27 06:22:18,884][20347] Updated weights for policy 0, policy_version 103038 (0.0008) [2023-12-27 06:22:18,908][20381] Updated weights for policy 1, policy_version 102625 (0.0007) [2023-12-27 06:22:18,939][20347] Updated weights for policy 0, policy_version 103048 (0.0008) [2023-12-27 06:22:18,964][20381] Updated weights for policy 1, policy_version 102635 (0.0006) [2023-12-27 06:22:19,013][20381] Updated weights for policy 1, policy_version 102645 (0.0008) [2023-12-27 06:22:19,061][20381] Updated weights for policy 1, policy_version 102655 (0.0007) [2023-12-27 06:22:19,616][20347] Updated weights for policy 0, policy_version 103058 (0.0007) [2023-12-27 06:22:19,685][20347] Updated weights for policy 0, policy_version 103068 (0.0008) [2023-12-27 06:22:19,758][20347] Updated weights for policy 0, policy_version 103078 (0.0007) [2023-12-27 06:22:19,824][20381] Updated weights for policy 1, policy_version 102665 (0.0007) [2023-12-27 06:22:19,885][20381] Updated weights for policy 1, policy_version 102675 (0.0008) [2023-12-27 06:22:19,950][20381] Updated weights for policy 1, policy_version 102685 (0.0007) [2023-12-27 06:22:20,376][20347] Updated weights for policy 0, policy_version 103088 (0.0009) [2023-12-27 06:22:20,424][20347] Updated weights for policy 0, policy_version 103098 (0.0010) [2023-12-27 06:22:20,469][20347] Updated weights for policy 0, policy_version 103108 (0.0010) [2023-12-27 06:22:20,654][20381] Updated weights for policy 1, policy_version 102695 (0.0008) [2023-12-27 06:22:20,722][20381] Updated weights for policy 1, policy_version 102705 (0.0009) [2023-12-27 06:22:20,786][20381] Updated weights for policy 1, policy_version 102715 (0.0008) [2023-12-27 06:22:21,188][20347] Updated weights for policy 0, policy_version 103118 (0.0009) [2023-12-27 06:22:21,244][20347] Updated weights for policy 0, policy_version 103128 (0.0010) [2023-12-27 06:22:21,303][20347] Updated weights for policy 0, policy_version 103138 (0.0008) [2023-12-27 06:22:21,468][20381] Updated weights for policy 1, policy_version 102725 (0.0008) [2023-12-27 06:22:21,527][20381] Updated weights for policy 1, policy_version 102735 (0.0007) [2023-12-27 06:22:21,586][20381] Updated weights for policy 1, policy_version 102745 (0.0008) [2023-12-27 06:22:21,996][20347] Updated weights for policy 0, policy_version 103148 (0.0007) [2023-12-27 06:22:22,053][20347] Updated weights for policy 0, policy_version 103158 (0.0007) [2023-12-27 06:22:22,109][20347] Updated weights for policy 0, policy_version 103168 (0.0007) [2023-12-27 06:22:22,304][20381] Updated weights for policy 1, policy_version 102755 (0.0008) [2023-12-27 06:22:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 52740096. Throughput: 0: 10665.6, 1: 10151.4. Samples: 52711976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000103176_26427392.pth... [2023-12-27 06:22:22,370][20381] Updated weights for policy 1, policy_version 102765 (0.0008) [2023-12-27 06:22:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000101928_26107904.pth [2023-12-27 06:22:22,434][20381] Updated weights for policy 1, policy_version 102775 (0.0007) [2023-12-27 06:22:22,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000102784_26320896.pth... [2023-12-27 06:22:22,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000101568_26009600.pth [2023-12-27 06:22:22,753][20347] Updated weights for policy 0, policy_version 103178 (0.0007) [2023-12-27 06:22:22,810][20347] Updated weights for policy 0, policy_version 103188 (0.0008) [2023-12-27 06:22:22,860][20347] Updated weights for policy 0, policy_version 103198 (0.0008) [2023-12-27 06:22:22,912][20347] Updated weights for policy 0, policy_version 103208 (0.0008) [2023-12-27 06:22:23,110][20381] Updated weights for policy 1, policy_version 102785 (0.0007) [2023-12-27 06:22:23,166][20381] Updated weights for policy 1, policy_version 102795 (0.0007) [2023-12-27 06:22:23,216][20381] Updated weights for policy 1, policy_version 102805 (0.0008) [2023-12-27 06:22:23,266][20381] Updated weights for policy 1, policy_version 102815 (0.0007) [2023-12-27 06:22:23,657][20347] Updated weights for policy 0, policy_version 103218 (0.0007) [2023-12-27 06:22:23,708][20347] Updated weights for policy 0, policy_version 103228 (0.0007) [2023-12-27 06:22:23,759][20347] Updated weights for policy 0, policy_version 103238 (0.0007) [2023-12-27 06:22:23,994][20381] Updated weights for policy 1, policy_version 102825 (0.0010) [2023-12-27 06:22:24,049][20381] Updated weights for policy 1, policy_version 102835 (0.0011) [2023-12-27 06:22:24,110][20381] Updated weights for policy 1, policy_version 102845 (0.0010) [2023-12-27 06:22:24,362][20347] Updated weights for policy 0, policy_version 103248 (0.0008) [2023-12-27 06:22:24,415][20347] Updated weights for policy 0, policy_version 103258 (0.0010) [2023-12-27 06:22:24,478][20347] Updated weights for policy 0, policy_version 103268 (0.0005) [2023-12-27 06:22:24,789][20381] Updated weights for policy 1, policy_version 102855 (0.0007) [2023-12-27 06:22:24,840][20381] Updated weights for policy 1, policy_version 102865 (0.0005) [2023-12-27 06:22:24,891][20381] Updated weights for policy 1, policy_version 102875 (0.0005) [2023-12-27 06:22:25,104][20347] Updated weights for policy 0, policy_version 103278 (0.0007) [2023-12-27 06:22:25,157][20347] Updated weights for policy 0, policy_version 103288 (0.0005) [2023-12-27 06:22:25,216][20347] Updated weights for policy 0, policy_version 103298 (0.0005) [2023-12-27 06:22:25,486][20381] Updated weights for policy 1, policy_version 102885 (0.0005) [2023-12-27 06:22:25,543][20381] Updated weights for policy 1, policy_version 102895 (0.0006) [2023-12-27 06:22:25,597][20381] Updated weights for policy 1, policy_version 102905 (0.0007) [2023-12-27 06:22:25,821][20347] Updated weights for policy 0, policy_version 103308 (0.0007) [2023-12-27 06:22:25,876][20347] Updated weights for policy 0, policy_version 103318 (0.0010) [2023-12-27 06:22:25,930][20347] Updated weights for policy 0, policy_version 103328 (0.0010) [2023-12-27 06:22:26,218][20381] Updated weights for policy 1, policy_version 102915 (0.0007) [2023-12-27 06:22:26,265][20381] Updated weights for policy 1, policy_version 102925 (0.0007) [2023-12-27 06:22:26,308][20381] Updated weights for policy 1, policy_version 102935 (0.0007) [2023-12-27 06:22:26,572][20347] Updated weights for policy 0, policy_version 103338 (0.0010) [2023-12-27 06:22:26,632][20347] Updated weights for policy 0, policy_version 103348 (0.0010) [2023-12-27 06:22:26,690][20347] Updated weights for policy 0, policy_version 103358 (0.0008) [2023-12-27 06:22:26,748][20347] Updated weights for policy 0, policy_version 103368 (0.0005) [2023-12-27 06:22:26,972][20381] Updated weights for policy 1, policy_version 102945 (0.0007) [2023-12-27 06:22:27,034][20381] Updated weights for policy 1, policy_version 102955 (0.0007) [2023-12-27 06:22:27,077][20381] Updated weights for policy 1, policy_version 102965 (0.0007) [2023-12-27 06:22:27,124][20381] Updated weights for policy 1, policy_version 102975 (0.0007) [2023-12-27 06:22:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 52846592. Throughput: 0: 10631.1, 1: 10193.0. Samples: 52836808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:27,369][20347] Updated weights for policy 0, policy_version 103378 (0.0005) [2023-12-27 06:22:27,420][20347] Updated weights for policy 0, policy_version 103388 (0.0009) [2023-12-27 06:22:27,479][20347] Updated weights for policy 0, policy_version 103398 (0.0010) [2023-12-27 06:22:27,833][20381] Updated weights for policy 1, policy_version 102985 (0.0007) [2023-12-27 06:22:27,885][20381] Updated weights for policy 1, policy_version 102995 (0.0007) [2023-12-27 06:22:27,935][20381] Updated weights for policy 1, policy_version 103005 (0.0007) [2023-12-27 06:22:28,130][20347] Updated weights for policy 0, policy_version 103408 (0.0010) [2023-12-27 06:22:28,177][20347] Updated weights for policy 0, policy_version 103418 (0.0010) [2023-12-27 06:22:28,232][20347] Updated weights for policy 0, policy_version 103428 (0.0008) [2023-12-27 06:22:28,636][20381] Updated weights for policy 1, policy_version 103015 (0.0007) [2023-12-27 06:22:28,694][20381] Updated weights for policy 1, policy_version 103025 (0.0007) [2023-12-27 06:22:28,748][20381] Updated weights for policy 1, policy_version 103035 (0.0007) [2023-12-27 06:22:28,927][20347] Updated weights for policy 0, policy_version 103438 (0.0008) [2023-12-27 06:22:28,984][20347] Updated weights for policy 0, policy_version 103448 (0.0010) [2023-12-27 06:22:29,037][20347] Updated weights for policy 0, policy_version 103458 (0.0010) [2023-12-27 06:22:29,423][20381] Updated weights for policy 1, policy_version 103045 (0.0007) [2023-12-27 06:22:29,477][20381] Updated weights for policy 1, policy_version 103055 (0.0008) [2023-12-27 06:22:29,535][20381] Updated weights for policy 1, policy_version 103065 (0.0007) [2023-12-27 06:22:29,750][20347] Updated weights for policy 0, policy_version 103468 (0.0010) [2023-12-27 06:22:29,804][20347] Updated weights for policy 0, policy_version 103478 (0.0010) [2023-12-27 06:22:29,864][20347] Updated weights for policy 0, policy_version 103488 (0.0010) [2023-12-27 06:22:30,238][20381] Updated weights for policy 1, policy_version 103075 (0.0007) [2023-12-27 06:22:30,295][20381] Updated weights for policy 1, policy_version 103085 (0.0007) [2023-12-27 06:22:30,349][20381] Updated weights for policy 1, policy_version 103095 (0.0007) [2023-12-27 06:22:30,522][20347] Updated weights for policy 0, policy_version 103498 (0.0010) [2023-12-27 06:22:30,571][20347] Updated weights for policy 0, policy_version 103508 (0.0007) [2023-12-27 06:22:30,625][20347] Updated weights for policy 0, policy_version 103518 (0.0005) [2023-12-27 06:22:30,668][20347] Updated weights for policy 0, policy_version 103528 (0.0010) [2023-12-27 06:22:31,050][20381] Updated weights for policy 1, policy_version 103105 (0.0008) [2023-12-27 06:22:31,113][20381] Updated weights for policy 1, policy_version 103115 (0.0007) [2023-12-27 06:22:31,174][20381] Updated weights for policy 1, policy_version 103125 (0.0008) [2023-12-27 06:22:31,227][20381] Updated weights for policy 1, policy_version 103135 (0.0007) [2023-12-27 06:22:31,313][20347] Updated weights for policy 0, policy_version 103538 (0.0008) [2023-12-27 06:22:31,378][20347] Updated weights for policy 0, policy_version 103548 (0.0009) [2023-12-27 06:22:31,439][20347] Updated weights for policy 0, policy_version 103558 (0.0008) [2023-12-27 06:22:31,894][20381] Updated weights for policy 1, policy_version 103145 (0.0006) [2023-12-27 06:22:31,951][20381] Updated weights for policy 1, policy_version 103155 (0.0006) [2023-12-27 06:22:32,006][20381] Updated weights for policy 1, policy_version 103165 (0.0007) [2023-12-27 06:22:32,175][20347] Updated weights for policy 0, policy_version 103568 (0.0010) [2023-12-27 06:22:32,232][20347] Updated weights for policy 0, policy_version 103578 (0.0010) [2023-12-27 06:22:32,300][20347] Updated weights for policy 0, policy_version 103588 (0.0010) [2023-12-27 06:22:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 52953088. Throughput: 0: 10557.6, 1: 10176.1. Samples: 52959984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:32,686][20381] Updated weights for policy 1, policy_version 103175 (0.0005) [2023-12-27 06:22:32,736][20381] Updated weights for policy 1, policy_version 103185 (0.0006) [2023-12-27 06:22:32,790][20381] Updated weights for policy 1, policy_version 103195 (0.0008) [2023-12-27 06:22:32,957][20347] Updated weights for policy 0, policy_version 103598 (0.0009) [2023-12-27 06:22:33,008][20347] Updated weights for policy 0, policy_version 103608 (0.0010) [2023-12-27 06:22:33,055][20347] Updated weights for policy 0, policy_version 103618 (0.0010) [2023-12-27 06:22:33,347][20381] Updated weights for policy 1, policy_version 103205 (0.0007) [2023-12-27 06:22:33,394][20381] Updated weights for policy 1, policy_version 103215 (0.0008) [2023-12-27 06:22:33,444][20381] Updated weights for policy 1, policy_version 103225 (0.0010) [2023-12-27 06:22:33,681][20347] Updated weights for policy 0, policy_version 103628 (0.0010) [2023-12-27 06:22:33,742][20347] Updated weights for policy 0, policy_version 103638 (0.0011) [2023-12-27 06:22:33,810][20347] Updated weights for policy 0, policy_version 103648 (0.0011) [2023-12-27 06:22:34,107][20381] Updated weights for policy 1, policy_version 103235 (0.0007) [2023-12-27 06:22:34,165][20381] Updated weights for policy 1, policy_version 103245 (0.0006) [2023-12-27 06:22:34,222][20381] Updated weights for policy 1, policy_version 103255 (0.0008) [2023-12-27 06:22:34,440][20347] Updated weights for policy 0, policy_version 103658 (0.0007) [2023-12-27 06:22:34,505][20347] Updated weights for policy 0, policy_version 103668 (0.0007) [2023-12-27 06:22:34,561][20347] Updated weights for policy 0, policy_version 103678 (0.0007) [2023-12-27 06:22:34,613][20347] Updated weights for policy 0, policy_version 103688 (0.0007) [2023-12-27 06:22:34,928][20381] Updated weights for policy 1, policy_version 103265 (0.0008) [2023-12-27 06:22:34,981][20381] Updated weights for policy 1, policy_version 103275 (0.0007) [2023-12-27 06:22:35,034][20381] Updated weights for policy 1, policy_version 103285 (0.0007) [2023-12-27 06:22:35,085][20381] Updated weights for policy 1, policy_version 103295 (0.0006) [2023-12-27 06:22:35,308][20347] Updated weights for policy 0, policy_version 103698 (0.0007) [2023-12-27 06:22:35,363][20347] Updated weights for policy 0, policy_version 103708 (0.0007) [2023-12-27 06:22:35,411][20347] Updated weights for policy 0, policy_version 103718 (0.0007) [2023-12-27 06:22:35,743][20381] Updated weights for policy 1, policy_version 103305 (0.0007) [2023-12-27 06:22:35,789][20381] Updated weights for policy 1, policy_version 103315 (0.0008) [2023-12-27 06:22:35,838][20381] Updated weights for policy 1, policy_version 103325 (0.0008) [2023-12-27 06:22:36,066][20347] Updated weights for policy 0, policy_version 103728 (0.0008) [2023-12-27 06:22:36,117][20347] Updated weights for policy 0, policy_version 103738 (0.0007) [2023-12-27 06:22:36,169][20347] Updated weights for policy 0, policy_version 103748 (0.0007) [2023-12-27 06:22:36,624][20381] Updated weights for policy 1, policy_version 103335 (0.0008) [2023-12-27 06:22:36,686][20381] Updated weights for policy 1, policy_version 103345 (0.0007) [2023-12-27 06:22:36,747][20381] Updated weights for policy 1, policy_version 103355 (0.0009) [2023-12-27 06:22:36,763][20347] Updated weights for policy 0, policy_version 103758 (0.0008) [2023-12-27 06:22:36,824][20347] Updated weights for policy 0, policy_version 103768 (0.0008) [2023-12-27 06:22:36,883][20347] Updated weights for policy 0, policy_version 103778 (0.0007) [2023-12-27 06:22:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 53051392. Throughput: 0: 10551.0, 1: 10203.6. Samples: 53023916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000103360_26468352.pth... [2023-12-27 06:22:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000103784_26583040.pth... [2023-12-27 06:22:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000102176_26165248.pth [2023-12-27 06:22:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000102568_26271744.pth [2023-12-27 06:22:37,437][20381] Updated weights for policy 1, policy_version 103365 (0.0010) [2023-12-27 06:22:37,499][20381] Updated weights for policy 1, policy_version 103375 (0.0006) [2023-12-27 06:22:37,532][20347] Updated weights for policy 0, policy_version 103788 (0.0005) [2023-12-27 06:22:37,561][20381] Updated weights for policy 1, policy_version 103385 (0.0006) [2023-12-27 06:22:37,584][20347] Updated weights for policy 0, policy_version 103798 (0.0005) [2023-12-27 06:22:37,636][20347] Updated weights for policy 0, policy_version 103808 (0.0005) [2023-12-27 06:22:38,188][20381] Updated weights for policy 1, policy_version 103395 (0.0007) [2023-12-27 06:22:38,195][20347] Updated weights for policy 0, policy_version 103818 (0.0005) [2023-12-27 06:22:38,235][20381] Updated weights for policy 1, policy_version 103405 (0.0009) [2023-12-27 06:22:38,255][20347] Updated weights for policy 0, policy_version 103828 (0.0007) [2023-12-27 06:22:38,299][20381] Updated weights for policy 1, policy_version 103415 (0.0006) [2023-12-27 06:22:38,317][20347] Updated weights for policy 0, policy_version 103838 (0.0009) [2023-12-27 06:22:38,386][20347] Updated weights for policy 0, policy_version 103848 (0.0010) [2023-12-27 06:22:39,023][20381] Updated weights for policy 1, policy_version 103425 (0.0010) [2023-12-27 06:22:39,070][20347] Updated weights for policy 0, policy_version 103858 (0.0006) [2023-12-27 06:22:39,079][20381] Updated weights for policy 1, policy_version 103435 (0.0008) [2023-12-27 06:22:39,124][20347] Updated weights for policy 0, policy_version 103868 (0.0006) [2023-12-27 06:22:39,132][20381] Updated weights for policy 1, policy_version 103445 (0.0007) [2023-12-27 06:22:39,175][20347] Updated weights for policy 0, policy_version 103878 (0.0006) [2023-12-27 06:22:39,191][20381] Updated weights for policy 1, policy_version 103455 (0.0008) [2023-12-27 06:22:39,846][20381] Updated weights for policy 1, policy_version 103465 (0.0008) [2023-12-27 06:22:39,852][20347] Updated weights for policy 0, policy_version 103888 (0.0008) [2023-12-27 06:22:39,902][20381] Updated weights for policy 1, policy_version 103475 (0.0006) [2023-12-27 06:22:39,913][20347] Updated weights for policy 0, policy_version 103898 (0.0008) [2023-12-27 06:22:39,961][20381] Updated weights for policy 1, policy_version 103485 (0.0009) [2023-12-27 06:22:39,970][20347] Updated weights for policy 0, policy_version 103908 (0.0009) [2023-12-27 06:22:40,627][20381] Updated weights for policy 1, policy_version 103495 (0.0006) [2023-12-27 06:22:40,678][20381] Updated weights for policy 1, policy_version 103505 (0.0010) [2023-12-27 06:22:40,727][20381] Updated weights for policy 1, policy_version 103515 (0.0010) [2023-12-27 06:22:40,752][20347] Updated weights for policy 0, policy_version 103918 (0.0007) [2023-12-27 06:22:40,806][20347] Updated weights for policy 0, policy_version 103928 (0.0007) [2023-12-27 06:22:40,856][20347] Updated weights for policy 0, policy_version 103938 (0.0009) [2023-12-27 06:22:41,395][20381] Updated weights for policy 1, policy_version 103525 (0.0007) [2023-12-27 06:22:41,455][20381] Updated weights for policy 1, policy_version 103535 (0.0008) [2023-12-27 06:22:41,509][20381] Updated weights for policy 1, policy_version 103545 (0.0006) [2023-12-27 06:22:41,565][20347] Updated weights for policy 0, policy_version 103948 (0.0008) [2023-12-27 06:22:41,630][20347] Updated weights for policy 0, policy_version 103958 (0.0008) [2023-12-27 06:22:41,691][20347] Updated weights for policy 0, policy_version 103968 (0.0006) [2023-12-27 06:22:42,209][20381] Updated weights for policy 1, policy_version 103555 (0.0006) [2023-12-27 06:22:42,272][20381] Updated weights for policy 1, policy_version 103565 (0.0006) [2023-12-27 06:22:42,327][20381] Updated weights for policy 1, policy_version 103575 (0.0007) [2023-12-27 06:22:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 53149696. Throughput: 0: 10570.1, 1: 10200.0. Samples: 53147800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:42,401][20347] Updated weights for policy 0, policy_version 103978 (0.0007) [2023-12-27 06:22:42,464][20347] Updated weights for policy 0, policy_version 103988 (0.0005) [2023-12-27 06:22:42,524][20347] Updated weights for policy 0, policy_version 103998 (0.0008) [2023-12-27 06:22:42,584][20347] Updated weights for policy 0, policy_version 104008 (0.0008) [2023-12-27 06:22:42,931][20381] Updated weights for policy 1, policy_version 103585 (0.0007) [2023-12-27 06:22:42,984][20381] Updated weights for policy 1, policy_version 103595 (0.0007) [2023-12-27 06:22:43,040][20381] Updated weights for policy 1, policy_version 103605 (0.0007) [2023-12-27 06:22:43,097][20381] Updated weights for policy 1, policy_version 103615 (0.0007) [2023-12-27 06:22:43,274][20347] Updated weights for policy 0, policy_version 104018 (0.0008) [2023-12-27 06:22:43,326][20347] Updated weights for policy 0, policy_version 104028 (0.0007) [2023-12-27 06:22:43,379][20347] Updated weights for policy 0, policy_version 104038 (0.0006) [2023-12-27 06:22:43,755][20381] Updated weights for policy 1, policy_version 103625 (0.0005) [2023-12-27 06:22:43,813][20381] Updated weights for policy 1, policy_version 103635 (0.0005) [2023-12-27 06:22:43,871][20381] Updated weights for policy 1, policy_version 103645 (0.0005) [2023-12-27 06:22:43,974][20347] Updated weights for policy 0, policy_version 104048 (0.0007) [2023-12-27 06:22:44,024][20347] Updated weights for policy 0, policy_version 104058 (0.0006) [2023-12-27 06:22:44,091][20347] Updated weights for policy 0, policy_version 104068 (0.0007) [2023-12-27 06:22:44,483][20381] Updated weights for policy 1, policy_version 103655 (0.0007) [2023-12-27 06:22:44,542][20381] Updated weights for policy 1, policy_version 103665 (0.0008) [2023-12-27 06:22:44,591][20381] Updated weights for policy 1, policy_version 103675 (0.0007) [2023-12-27 06:22:44,775][20347] Updated weights for policy 0, policy_version 104078 (0.0008) [2023-12-27 06:22:44,837][20347] Updated weights for policy 0, policy_version 104088 (0.0007) [2023-12-27 06:22:44,886][20347] Updated weights for policy 0, policy_version 104098 (0.0009) [2023-12-27 06:22:45,230][20381] Updated weights for policy 1, policy_version 103685 (0.0008) [2023-12-27 06:22:45,286][20381] Updated weights for policy 1, policy_version 103695 (0.0008) [2023-12-27 06:22:45,339][20381] Updated weights for policy 1, policy_version 103705 (0.0008) [2023-12-27 06:22:45,568][20347] Updated weights for policy 0, policy_version 104108 (0.0006) [2023-12-27 06:22:45,623][20347] Updated weights for policy 0, policy_version 104118 (0.0005) [2023-12-27 06:22:45,672][20347] Updated weights for policy 0, policy_version 104128 (0.0007) [2023-12-27 06:22:46,061][20381] Updated weights for policy 1, policy_version 103715 (0.0007) [2023-12-27 06:22:46,107][20381] Updated weights for policy 1, policy_version 103725 (0.0010) [2023-12-27 06:22:46,159][20381] Updated weights for policy 1, policy_version 103735 (0.0009) [2023-12-27 06:22:46,232][20347] Updated weights for policy 0, policy_version 104138 (0.0007) [2023-12-27 06:22:46,280][20347] Updated weights for policy 0, policy_version 104148 (0.0007) [2023-12-27 06:22:46,326][20347] Updated weights for policy 0, policy_version 104158 (0.0005) [2023-12-27 06:22:46,370][20347] Updated weights for policy 0, policy_version 104168 (0.0005) [2023-12-27 06:22:46,794][20381] Updated weights for policy 1, policy_version 103745 (0.0007) [2023-12-27 06:22:46,854][20381] Updated weights for policy 1, policy_version 103755 (0.0008) [2023-12-27 06:22:46,914][20381] Updated weights for policy 1, policy_version 103765 (0.0008) [2023-12-27 06:22:46,970][20381] Updated weights for policy 1, policy_version 103775 (0.0008) [2023-12-27 06:22:47,071][20347] Updated weights for policy 0, policy_version 104178 (0.0006) [2023-12-27 06:22:47,128][20347] Updated weights for policy 0, policy_version 104188 (0.0006) [2023-12-27 06:22:47,172][20347] Updated weights for policy 0, policy_version 104198 (0.0006) [2023-12-27 06:22:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 53264384. Throughput: 0: 10547.1, 1: 10258.5. Samples: 53273004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:47,654][20381] Updated weights for policy 1, policy_version 103785 (0.0008) [2023-12-27 06:22:47,713][20381] Updated weights for policy 1, policy_version 103795 (0.0008) [2023-12-27 06:22:47,774][20381] Updated weights for policy 1, policy_version 103805 (0.0008) [2023-12-27 06:22:47,809][20347] Updated weights for policy 0, policy_version 104208 (0.0007) [2023-12-27 06:22:47,859][20347] Updated weights for policy 0, policy_version 104218 (0.0008) [2023-12-27 06:22:47,910][20347] Updated weights for policy 0, policy_version 104229 (0.0007) [2023-12-27 06:22:48,456][20381] Updated weights for policy 1, policy_version 103815 (0.0007) [2023-12-27 06:22:48,513][20381] Updated weights for policy 1, policy_version 103825 (0.0008) [2023-12-27 06:22:48,569][20381] Updated weights for policy 1, policy_version 103835 (0.0008) [2023-12-27 06:22:48,633][20347] Updated weights for policy 0, policy_version 104239 (0.0007) [2023-12-27 06:22:48,692][20347] Updated weights for policy 0, policy_version 104249 (0.0008) [2023-12-27 06:22:48,756][20347] Updated weights for policy 0, policy_version 104259 (0.0008) [2023-12-27 06:22:49,201][20381] Updated weights for policy 1, policy_version 103845 (0.0007) [2023-12-27 06:22:49,264][20381] Updated weights for policy 1, policy_version 103855 (0.0007) [2023-12-27 06:22:49,328][20381] Updated weights for policy 1, policy_version 103865 (0.0007) [2023-12-27 06:22:49,490][20347] Updated weights for policy 0, policy_version 104269 (0.0008) [2023-12-27 06:22:49,544][20347] Updated weights for policy 0, policy_version 104279 (0.0008) [2023-12-27 06:22:49,594][20347] Updated weights for policy 0, policy_version 104289 (0.0008) [2023-12-27 06:22:50,008][20381] Updated weights for policy 1, policy_version 103875 (0.0008) [2023-12-27 06:22:50,070][20381] Updated weights for policy 1, policy_version 103885 (0.0008) [2023-12-27 06:22:50,133][20381] Updated weights for policy 1, policy_version 103895 (0.0008) [2023-12-27 06:22:50,267][20347] Updated weights for policy 0, policy_version 104299 (0.0007) [2023-12-27 06:22:50,327][20347] Updated weights for policy 0, policy_version 104309 (0.0007) [2023-12-27 06:22:50,384][20347] Updated weights for policy 0, policy_version 104319 (0.0007) [2023-12-27 06:22:50,862][20381] Updated weights for policy 1, policy_version 103905 (0.0008) [2023-12-27 06:22:50,922][20381] Updated weights for policy 1, policy_version 103915 (0.0008) [2023-12-27 06:22:50,981][20381] Updated weights for policy 1, policy_version 103925 (0.0008) [2023-12-27 06:22:51,044][20381] Updated weights for policy 1, policy_version 103935 (0.0009) [2023-12-27 06:22:51,104][20347] Updated weights for policy 0, policy_version 104329 (0.0008) [2023-12-27 06:22:51,162][20347] Updated weights for policy 0, policy_version 104339 (0.0011) [2023-12-27 06:22:51,216][20347] Updated weights for policy 0, policy_version 104349 (0.0008) [2023-12-27 06:22:51,272][20347] Updated weights for policy 0, policy_version 104359 (0.0010) [2023-12-27 06:22:51,828][20381] Updated weights for policy 1, policy_version 103945 (0.0008) [2023-12-27 06:22:51,880][20381] Updated weights for policy 1, policy_version 103955 (0.0007) [2023-12-27 06:22:51,929][20347] Updated weights for policy 0, policy_version 104369 (0.0010) [2023-12-27 06:22:51,936][20381] Updated weights for policy 1, policy_version 103965 (0.0008) [2023-12-27 06:22:51,983][20347] Updated weights for policy 0, policy_version 104379 (0.0009) [2023-12-27 06:22:52,045][20347] Updated weights for policy 0, policy_version 104389 (0.0009) [2023-12-27 06:22:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 53362688. Throughput: 0: 10494.2, 1: 10237.5. Samples: 53333488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000103968_26624000.pth... [2023-12-27 06:22:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000104392_26738688.pth... [2023-12-27 06:22:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000103176_26427392.pth [2023-12-27 06:22:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000102784_26320896.pth [2023-12-27 06:22:52,631][20381] Updated weights for policy 1, policy_version 103975 (0.0008) [2023-12-27 06:22:52,681][20381] Updated weights for policy 1, policy_version 103985 (0.0008) [2023-12-27 06:22:52,732][20347] Updated weights for policy 0, policy_version 104399 (0.0007) [2023-12-27 06:22:52,743][20381] Updated weights for policy 1, policy_version 103995 (0.0006) [2023-12-27 06:22:52,790][20347] Updated weights for policy 0, policy_version 104409 (0.0005) [2023-12-27 06:22:52,845][20347] Updated weights for policy 0, policy_version 104419 (0.0005) [2023-12-27 06:22:53,452][20347] Updated weights for policy 0, policy_version 104429 (0.0007) [2023-12-27 06:22:53,463][20381] Updated weights for policy 1, policy_version 104005 (0.0009) [2023-12-27 06:22:53,517][20347] Updated weights for policy 0, policy_version 104439 (0.0007) [2023-12-27 06:22:53,526][20381] Updated weights for policy 1, policy_version 104015 (0.0007) [2023-12-27 06:22:53,580][20347] Updated weights for policy 0, policy_version 104449 (0.0006) [2023-12-27 06:22:53,588][20381] Updated weights for policy 1, policy_version 104025 (0.0008) [2023-12-27 06:22:54,172][20347] Updated weights for policy 0, policy_version 104459 (0.0006) [2023-12-27 06:22:54,227][20347] Updated weights for policy 0, policy_version 104469 (0.0007) [2023-12-27 06:22:54,275][20381] Updated weights for policy 1, policy_version 104035 (0.0007) [2023-12-27 06:22:54,285][20347] Updated weights for policy 0, policy_version 104479 (0.0007) [2023-12-27 06:22:54,331][20381] Updated weights for policy 1, policy_version 104045 (0.0008) [2023-12-27 06:22:54,387][20381] Updated weights for policy 1, policy_version 104055 (0.0007) [2023-12-27 06:22:54,898][20347] Updated weights for policy 0, policy_version 104489 (0.0006) [2023-12-27 06:22:54,944][20347] Updated weights for policy 0, policy_version 104499 (0.0007) [2023-12-27 06:22:55,003][20347] Updated weights for policy 0, policy_version 104509 (0.0006) [2023-12-27 06:22:55,067][20347] Updated weights for policy 0, policy_version 104519 (0.0007) [2023-12-27 06:22:55,076][20381] Updated weights for policy 1, policy_version 104065 (0.0008) [2023-12-27 06:22:55,134][20381] Updated weights for policy 1, policy_version 104075 (0.0010) [2023-12-27 06:22:55,189][20381] Updated weights for policy 1, policy_version 104085 (0.0010) [2023-12-27 06:22:55,247][20381] Updated weights for policy 1, policy_version 104095 (0.0010) [2023-12-27 06:22:55,720][20347] Updated weights for policy 0, policy_version 104529 (0.0005) [2023-12-27 06:22:55,766][20347] Updated weights for policy 0, policy_version 104539 (0.0005) [2023-12-27 06:22:55,827][20347] Updated weights for policy 0, policy_version 104549 (0.0005) [2023-12-27 06:22:55,861][20381] Updated weights for policy 1, policy_version 104105 (0.0011) [2023-12-27 06:22:55,922][20381] Updated weights for policy 1, policy_version 104115 (0.0010) [2023-12-27 06:22:55,972][20381] Updated weights for policy 1, policy_version 104125 (0.0007) [2023-12-27 06:22:56,464][20347] Updated weights for policy 0, policy_version 104559 (0.0009) [2023-12-27 06:22:56,518][20347] Updated weights for policy 0, policy_version 104569 (0.0010) [2023-12-27 06:22:56,531][20381] Updated weights for policy 1, policy_version 104135 (0.0005) [2023-12-27 06:22:56,572][20347] Updated weights for policy 0, policy_version 104579 (0.0010) [2023-12-27 06:22:56,587][20381] Updated weights for policy 1, policy_version 104145 (0.0005) [2023-12-27 06:22:56,648][20381] Updated weights for policy 1, policy_version 104155 (0.0005) [2023-12-27 06:22:57,136][20347] Updated weights for policy 0, policy_version 104589 (0.0006) [2023-12-27 06:22:57,147][20381] Updated weights for policy 1, policy_version 104165 (0.0005) [2023-12-27 06:22:57,187][20347] Updated weights for policy 0, policy_version 104599 (0.0006) [2023-12-27 06:22:57,194][20381] Updated weights for policy 1, policy_version 104175 (0.0005) [2023-12-27 06:22:57,244][20381] Updated weights for policy 1, policy_version 104185 (0.0005) [2023-12-27 06:22:57,244][20347] Updated weights for policy 0, policy_version 104609 (0.0010) [2023-12-27 06:22:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 53477376. Throughput: 0: 10524.5, 1: 10220.6. Samples: 53458516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:22:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:22:57,769][20381] Updated weights for policy 1, policy_version 104195 (0.0005) [2023-12-27 06:22:57,826][20381] Updated weights for policy 1, policy_version 104205 (0.0005) [2023-12-27 06:22:57,875][20381] Updated weights for policy 1, policy_version 104215 (0.0006) [2023-12-27 06:22:57,887][20347] Updated weights for policy 0, policy_version 104619 (0.0010) [2023-12-27 06:22:57,942][20347] Updated weights for policy 0, policy_version 104629 (0.0006) [2023-12-27 06:22:57,989][20347] Updated weights for policy 0, policy_version 104639 (0.0010) [2023-12-27 06:22:58,553][20381] Updated weights for policy 1, policy_version 104225 (0.0010) [2023-12-27 06:22:58,617][20381] Updated weights for policy 1, policy_version 104235 (0.0009) [2023-12-27 06:22:58,677][20381] Updated weights for policy 1, policy_version 104245 (0.0009) [2023-12-27 06:22:58,684][20347] Updated weights for policy 0, policy_version 104649 (0.0010) [2023-12-27 06:22:58,737][20381] Updated weights for policy 1, policy_version 104255 (0.0010) [2023-12-27 06:22:58,740][20347] Updated weights for policy 0, policy_version 104659 (0.0007) [2023-12-27 06:22:58,817][20347] Updated weights for policy 0, policy_version 104669 (0.0007) [2023-12-27 06:22:58,885][20347] Updated weights for policy 0, policy_version 104679 (0.0008) [2023-12-27 06:22:59,472][20381] Updated weights for policy 1, policy_version 104265 (0.0007) [2023-12-27 06:22:59,521][20381] Updated weights for policy 1, policy_version 104275 (0.0011) [2023-12-27 06:22:59,531][20347] Updated weights for policy 0, policy_version 104689 (0.0006) [2023-12-27 06:22:59,574][20381] Updated weights for policy 1, policy_version 104285 (0.0011) [2023-12-27 06:22:59,584][20347] Updated weights for policy 0, policy_version 104699 (0.0005) [2023-12-27 06:22:59,642][20347] Updated weights for policy 0, policy_version 104709 (0.0005) [2023-12-27 06:23:00,239][20381] Updated weights for policy 1, policy_version 104295 (0.0010) [2023-12-27 06:23:00,285][20381] Updated weights for policy 1, policy_version 104305 (0.0010) [2023-12-27 06:23:00,297][20347] Updated weights for policy 0, policy_version 104719 (0.0008) [2023-12-27 06:23:00,331][20381] Updated weights for policy 1, policy_version 104315 (0.0010) [2023-12-27 06:23:00,348][20347] Updated weights for policy 0, policy_version 104729 (0.0006) [2023-12-27 06:23:00,389][20347] Updated weights for policy 0, policy_version 104739 (0.0005) [2023-12-27 06:23:00,928][20381] Updated weights for policy 1, policy_version 104325 (0.0006) [2023-12-27 06:23:00,956][20347] Updated weights for policy 0, policy_version 104749 (0.0005) [2023-12-27 06:23:00,978][20381] Updated weights for policy 1, policy_version 104335 (0.0005) [2023-12-27 06:23:01,013][20347] Updated weights for policy 0, policy_version 104759 (0.0006) [2023-12-27 06:23:01,032][20381] Updated weights for policy 1, policy_version 104345 (0.0006) [2023-12-27 06:23:01,075][20347] Updated weights for policy 0, policy_version 104769 (0.0008) [2023-12-27 06:23:01,651][20381] Updated weights for policy 1, policy_version 104355 (0.0009) [2023-12-27 06:23:01,709][20381] Updated weights for policy 1, policy_version 104365 (0.0008) [2023-12-27 06:23:01,770][20381] Updated weights for policy 1, policy_version 104375 (0.0011) [2023-12-27 06:23:01,780][20347] Updated weights for policy 0, policy_version 104779 (0.0006) [2023-12-27 06:23:01,824][20347] Updated weights for policy 0, policy_version 104789 (0.0005) [2023-12-27 06:23:01,875][20347] Updated weights for policy 0, policy_version 104799 (0.0007) [2023-12-27 06:23:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 53575680. Throughput: 0: 10479.8, 1: 10349.8. Samples: 53587936. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:02,490][20381] Updated weights for policy 1, policy_version 104385 (0.0010) [2023-12-27 06:23:02,545][20381] Updated weights for policy 1, policy_version 104395 (0.0007) [2023-12-27 06:23:02,599][20381] Updated weights for policy 1, policy_version 104405 (0.0007) [2023-12-27 06:23:02,606][20347] Updated weights for policy 0, policy_version 104809 (0.0007) [2023-12-27 06:23:02,655][20381] Updated weights for policy 1, policy_version 104415 (0.0007) [2023-12-27 06:23:02,664][20347] Updated weights for policy 0, policy_version 104819 (0.0006) [2023-12-27 06:23:02,715][20347] Updated weights for policy 0, policy_version 104829 (0.0007) [2023-12-27 06:23:02,768][20347] Updated weights for policy 0, policy_version 104839 (0.0007) [2023-12-27 06:23:03,299][20381] Updated weights for policy 1, policy_version 104425 (0.0005) [2023-12-27 06:23:03,352][20381] Updated weights for policy 1, policy_version 104435 (0.0005) [2023-12-27 06:23:03,405][20381] Updated weights for policy 1, policy_version 104445 (0.0005) [2023-12-27 06:23:03,485][20347] Updated weights for policy 0, policy_version 104849 (0.0006) [2023-12-27 06:23:03,533][20347] Updated weights for policy 0, policy_version 104859 (0.0005) [2023-12-27 06:23:03,578][20347] Updated weights for policy 0, policy_version 104869 (0.0005) [2023-12-27 06:23:03,957][20381] Updated weights for policy 1, policy_version 104455 (0.0010) [2023-12-27 06:23:04,009][20381] Updated weights for policy 1, policy_version 104465 (0.0011) [2023-12-27 06:23:04,065][20381] Updated weights for policy 1, policy_version 104475 (0.0011) [2023-12-27 06:23:04,150][20347] Updated weights for policy 0, policy_version 104879 (0.0007) [2023-12-27 06:23:04,216][20347] Updated weights for policy 0, policy_version 104889 (0.0007) [2023-12-27 06:23:04,280][20347] Updated weights for policy 0, policy_version 104899 (0.0007) [2023-12-27 06:23:04,773][20381] Updated weights for policy 1, policy_version 104485 (0.0009) [2023-12-27 06:23:04,837][20381] Updated weights for policy 1, policy_version 104495 (0.0007) [2023-12-27 06:23:04,895][20381] Updated weights for policy 1, policy_version 104505 (0.0011) [2023-12-27 06:23:04,956][20347] Updated weights for policy 0, policy_version 104909 (0.0007) [2023-12-27 06:23:05,012][20347] Updated weights for policy 0, policy_version 104919 (0.0007) [2023-12-27 06:23:05,067][20347] Updated weights for policy 0, policy_version 104929 (0.0007) [2023-12-27 06:23:05,605][20381] Updated weights for policy 1, policy_version 104515 (0.0010) [2023-12-27 06:23:05,653][20381] Updated weights for policy 1, policy_version 104525 (0.0008) [2023-12-27 06:23:05,703][20381] Updated weights for policy 1, policy_version 104535 (0.0009) [2023-12-27 06:23:05,752][20347] Updated weights for policy 0, policy_version 104939 (0.0007) [2023-12-27 06:23:05,796][20347] Updated weights for policy 0, policy_version 104949 (0.0007) [2023-12-27 06:23:05,847][20347] Updated weights for policy 0, policy_version 104959 (0.0007) [2023-12-27 06:23:06,397][20381] Updated weights for policy 1, policy_version 104545 (0.0010) [2023-12-27 06:23:06,462][20381] Updated weights for policy 1, policy_version 104555 (0.0011) [2023-12-27 06:23:06,522][20381] Updated weights for policy 1, policy_version 104565 (0.0007) [2023-12-27 06:23:06,575][20381] Updated weights for policy 1, policy_version 104575 (0.0006) [2023-12-27 06:23:06,583][20347] Updated weights for policy 0, policy_version 104969 (0.0007) [2023-12-27 06:23:06,641][20347] Updated weights for policy 0, policy_version 104979 (0.0007) [2023-12-27 06:23:06,696][20347] Updated weights for policy 0, policy_version 104989 (0.0007) [2023-12-27 06:23:06,759][20347] Updated weights for policy 0, policy_version 104999 (0.0007) [2023-12-27 06:23:07,232][20381] Updated weights for policy 1, policy_version 104585 (0.0011) [2023-12-27 06:23:07,284][20381] Updated weights for policy 1, policy_version 104595 (0.0009) [2023-12-27 06:23:07,337][20381] Updated weights for policy 1, policy_version 104605 (0.0008) [2023-12-27 06:23:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 53682176. Throughput: 0: 10461.7, 1: 10405.0. Samples: 53650980. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000104608_26787840.pth... [2023-12-27 06:23:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000105000_26894336.pth... [2023-12-27 06:23:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000103360_26468352.pth [2023-12-27 06:23:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000103784_26583040.pth [2023-12-27 06:23:07,453][20347] Updated weights for policy 0, policy_version 105009 (0.0007) [2023-12-27 06:23:07,503][20347] Updated weights for policy 0, policy_version 105019 (0.0007) [2023-12-27 06:23:07,550][20347] Updated weights for policy 0, policy_version 105029 (0.0007) [2023-12-27 06:23:08,010][20381] Updated weights for policy 1, policy_version 104615 (0.0007) [2023-12-27 06:23:08,061][20381] Updated weights for policy 1, policy_version 104625 (0.0005) [2023-12-27 06:23:08,109][20381] Updated weights for policy 1, policy_version 104635 (0.0009) [2023-12-27 06:23:08,222][20347] Updated weights for policy 0, policy_version 105039 (0.0007) [2023-12-27 06:23:08,272][20347] Updated weights for policy 0, policy_version 105049 (0.0007) [2023-12-27 06:23:08,316][20347] Updated weights for policy 0, policy_version 105059 (0.0007) [2023-12-27 06:23:08,793][20381] Updated weights for policy 1, policy_version 104645 (0.0008) [2023-12-27 06:23:08,851][20381] Updated weights for policy 1, policy_version 104655 (0.0007) [2023-12-27 06:23:08,903][20381] Updated weights for policy 1, policy_version 104665 (0.0010) [2023-12-27 06:23:09,043][20347] Updated weights for policy 0, policy_version 105069 (0.0008) [2023-12-27 06:23:09,103][20347] Updated weights for policy 0, policy_version 105079 (0.0008) [2023-12-27 06:23:09,151][20347] Updated weights for policy 0, policy_version 105089 (0.0010) [2023-12-27 06:23:09,529][20381] Updated weights for policy 1, policy_version 104675 (0.0009) [2023-12-27 06:23:09,588][20381] Updated weights for policy 1, policy_version 104685 (0.0007) [2023-12-27 06:23:09,647][20381] Updated weights for policy 1, policy_version 104695 (0.0006) [2023-12-27 06:23:09,849][20347] Updated weights for policy 0, policy_version 105099 (0.0010) [2023-12-27 06:23:09,904][20347] Updated weights for policy 0, policy_version 105109 (0.0010) [2023-12-27 06:23:09,971][20347] Updated weights for policy 0, policy_version 105119 (0.0008) [2023-12-27 06:23:10,333][20381] Updated weights for policy 1, policy_version 104705 (0.0006) [2023-12-27 06:23:10,388][20381] Updated weights for policy 1, policy_version 104715 (0.0007) [2023-12-27 06:23:10,445][20381] Updated weights for policy 1, policy_version 104725 (0.0009) [2023-12-27 06:23:10,503][20381] Updated weights for policy 1, policy_version 104735 (0.0008) [2023-12-27 06:23:10,559][20347] Updated weights for policy 0, policy_version 105129 (0.0007) [2023-12-27 06:23:10,618][20347] Updated weights for policy 0, policy_version 105139 (0.0010) [2023-12-27 06:23:10,678][20347] Updated weights for policy 0, policy_version 105149 (0.0010) [2023-12-27 06:23:10,735][20347] Updated weights for policy 0, policy_version 105159 (0.0008) [2023-12-27 06:23:11,152][20381] Updated weights for policy 1, policy_version 104745 (0.0007) [2023-12-27 06:23:11,214][20381] Updated weights for policy 1, policy_version 104755 (0.0009) [2023-12-27 06:23:11,273][20381] Updated weights for policy 1, policy_version 104765 (0.0008) [2023-12-27 06:23:11,408][20347] Updated weights for policy 0, policy_version 105169 (0.0009) [2023-12-27 06:23:11,457][20347] Updated weights for policy 0, policy_version 105179 (0.0008) [2023-12-27 06:23:11,507][20347] Updated weights for policy 0, policy_version 105189 (0.0007) [2023-12-27 06:23:11,958][20381] Updated weights for policy 1, policy_version 104775 (0.0007) [2023-12-27 06:23:12,018][20381] Updated weights for policy 1, policy_version 104785 (0.0007) [2023-12-27 06:23:12,065][20381] Updated weights for policy 1, policy_version 104795 (0.0007) [2023-12-27 06:23:12,238][20347] Updated weights for policy 0, policy_version 105199 (0.0010) [2023-12-27 06:23:12,295][20347] Updated weights for policy 0, policy_version 105209 (0.0010) [2023-12-27 06:23:12,357][20347] Updated weights for policy 0, policy_version 105219 (0.0010) [2023-12-27 06:23:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 53780480. Throughput: 0: 10413.1, 1: 10407.9. Samples: 53773756. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:12,815][20381] Updated weights for policy 1, policy_version 104805 (0.0009) [2023-12-27 06:23:12,870][20381] Updated weights for policy 1, policy_version 104815 (0.0009) [2023-12-27 06:23:12,924][20381] Updated weights for policy 1, policy_version 104825 (0.0006) [2023-12-27 06:23:12,949][20347] Updated weights for policy 0, policy_version 105229 (0.0006) [2023-12-27 06:23:13,000][20347] Updated weights for policy 0, policy_version 105239 (0.0008) [2023-12-27 06:23:13,047][20347] Updated weights for policy 0, policy_version 105249 (0.0006) [2023-12-27 06:23:13,551][20381] Updated weights for policy 1, policy_version 104835 (0.0005) [2023-12-27 06:23:13,599][20381] Updated weights for policy 1, policy_version 104845 (0.0005) [2023-12-27 06:23:13,633][20347] Updated weights for policy 0, policy_version 105259 (0.0005) [2023-12-27 06:23:13,657][20381] Updated weights for policy 1, policy_version 104855 (0.0005) [2023-12-27 06:23:13,679][20347] Updated weights for policy 0, policy_version 105269 (0.0006) [2023-12-27 06:23:13,727][20347] Updated weights for policy 0, policy_version 105279 (0.0009) [2023-12-27 06:23:14,296][20347] Updated weights for policy 0, policy_version 105289 (0.0009) [2023-12-27 06:23:14,298][20381] Updated weights for policy 1, policy_version 104865 (0.0007) [2023-12-27 06:23:14,350][20381] Updated weights for policy 1, policy_version 104875 (0.0011) [2023-12-27 06:23:14,357][20347] Updated weights for policy 0, policy_version 105299 (0.0006) [2023-12-27 06:23:14,405][20381] Updated weights for policy 1, policy_version 104885 (0.0010) [2023-12-27 06:23:14,418][20347] Updated weights for policy 0, policy_version 105309 (0.0008) [2023-12-27 06:23:14,464][20381] Updated weights for policy 1, policy_version 104895 (0.0011) [2023-12-27 06:23:14,471][20347] Updated weights for policy 0, policy_version 105319 (0.0008) [2023-12-27 06:23:15,035][20347] Updated weights for policy 0, policy_version 105329 (0.0010) [2023-12-27 06:23:15,090][20347] Updated weights for policy 0, policy_version 105339 (0.0010) [2023-12-27 06:23:15,151][20347] Updated weights for policy 0, policy_version 105349 (0.0011) [2023-12-27 06:23:15,167][20381] Updated weights for policy 1, policy_version 104905 (0.0011) [2023-12-27 06:23:15,228][20381] Updated weights for policy 1, policy_version 104915 (0.0006) [2023-12-27 06:23:15,287][20381] Updated weights for policy 1, policy_version 104925 (0.0006) [2023-12-27 06:23:15,749][20347] Updated weights for policy 0, policy_version 105359 (0.0007) [2023-12-27 06:23:15,811][20347] Updated weights for policy 0, policy_version 105369 (0.0005) [2023-12-27 06:23:15,875][20347] Updated weights for policy 0, policy_version 105379 (0.0005) [2023-12-27 06:23:15,877][20381] Updated weights for policy 1, policy_version 104935 (0.0006) [2023-12-27 06:23:15,926][20381] Updated weights for policy 1, policy_version 104945 (0.0008) [2023-12-27 06:23:15,974][20381] Updated weights for policy 1, policy_version 104955 (0.0007) [2023-12-27 06:23:16,388][20347] Updated weights for policy 0, policy_version 105389 (0.0005) [2023-12-27 06:23:16,434][20347] Updated weights for policy 0, policy_version 105399 (0.0005) [2023-12-27 06:23:16,485][20347] Updated weights for policy 0, policy_version 105409 (0.0007) [2023-12-27 06:23:16,510][20381] Updated weights for policy 1, policy_version 104965 (0.0006) [2023-12-27 06:23:16,565][20381] Updated weights for policy 1, policy_version 104975 (0.0007) [2023-12-27 06:23:16,620][20381] Updated weights for policy 1, policy_version 104985 (0.0008) [2023-12-27 06:23:17,165][20347] Updated weights for policy 0, policy_version 105419 (0.0006) [2023-12-27 06:23:17,227][20347] Updated weights for policy 0, policy_version 105429 (0.0007) [2023-12-27 06:23:17,247][20381] Updated weights for policy 1, policy_version 104995 (0.0007) [2023-12-27 06:23:17,278][20347] Updated weights for policy 0, policy_version 105439 (0.0005) [2023-12-27 06:23:17,302][20381] Updated weights for policy 1, policy_version 105005 (0.0006) [2023-12-27 06:23:17,351][20381] Updated weights for policy 1, policy_version 105015 (0.0006) [2023-12-27 06:23:17,363][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 53895168. Throughput: 0: 10524.8, 1: 10485.2. Samples: 53905436. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:17,969][20347] Updated weights for policy 0, policy_version 105449 (0.0006) [2023-12-27 06:23:18,036][20347] Updated weights for policy 0, policy_version 105459 (0.0006) [2023-12-27 06:23:18,052][20381] Updated weights for policy 1, policy_version 105025 (0.0007) [2023-12-27 06:23:18,084][20347] Updated weights for policy 0, policy_version 105469 (0.0007) [2023-12-27 06:23:18,119][20381] Updated weights for policy 1, policy_version 105035 (0.0007) [2023-12-27 06:23:18,149][20347] Updated weights for policy 0, policy_version 105479 (0.0007) [2023-12-27 06:23:18,175][20381] Updated weights for policy 1, policy_version 105045 (0.0007) [2023-12-27 06:23:18,229][20381] Updated weights for policy 1, policy_version 105055 (0.0008) [2023-12-27 06:23:18,780][20347] Updated weights for policy 0, policy_version 105489 (0.0010) [2023-12-27 06:23:18,829][20347] Updated weights for policy 0, policy_version 105499 (0.0010) [2023-12-27 06:23:18,878][20347] Updated weights for policy 0, policy_version 105509 (0.0009) [2023-12-27 06:23:18,936][20381] Updated weights for policy 1, policy_version 105065 (0.0006) [2023-12-27 06:23:18,986][20381] Updated weights for policy 1, policy_version 105075 (0.0007) [2023-12-27 06:23:19,032][20381] Updated weights for policy 1, policy_version 105085 (0.0007) [2023-12-27 06:23:19,601][20347] Updated weights for policy 0, policy_version 105519 (0.0006) [2023-12-27 06:23:19,647][20347] Updated weights for policy 0, policy_version 105529 (0.0010) [2023-12-27 06:23:19,706][20347] Updated weights for policy 0, policy_version 105539 (0.0010) [2023-12-27 06:23:19,711][20381] Updated weights for policy 1, policy_version 105095 (0.0007) [2023-12-27 06:23:19,769][20381] Updated weights for policy 1, policy_version 105105 (0.0011) [2023-12-27 06:23:19,835][20381] Updated weights for policy 1, policy_version 105115 (0.0010) [2023-12-27 06:23:20,440][20347] Updated weights for policy 0, policy_version 105549 (0.0009) [2023-12-27 06:23:20,482][20381] Updated weights for policy 1, policy_version 105125 (0.0008) [2023-12-27 06:23:20,496][20347] Updated weights for policy 0, policy_version 105559 (0.0008) [2023-12-27 06:23:20,533][20381] Updated weights for policy 1, policy_version 105135 (0.0010) [2023-12-27 06:23:20,553][20347] Updated weights for policy 0, policy_version 105569 (0.0009) [2023-12-27 06:23:20,587][20381] Updated weights for policy 1, policy_version 105145 (0.0009) [2023-12-27 06:23:21,233][20347] Updated weights for policy 0, policy_version 105579 (0.0009) [2023-12-27 06:23:21,276][20381] Updated weights for policy 1, policy_version 105155 (0.0008) [2023-12-27 06:23:21,297][20347] Updated weights for policy 0, policy_version 105589 (0.0010) [2023-12-27 06:23:21,334][20381] Updated weights for policy 1, policy_version 105165 (0.0011) [2023-12-27 06:23:21,358][20347] Updated weights for policy 0, policy_version 105599 (0.0010) [2023-12-27 06:23:21,397][20381] Updated weights for policy 1, policy_version 105175 (0.0009) [2023-12-27 06:23:22,080][20347] Updated weights for policy 0, policy_version 105609 (0.0007) [2023-12-27 06:23:22,132][20381] Updated weights for policy 1, policy_version 105185 (0.0008) [2023-12-27 06:23:22,141][20347] Updated weights for policy 0, policy_version 105619 (0.0007) [2023-12-27 06:23:22,191][20347] Updated weights for policy 0, policy_version 105629 (0.0005) [2023-12-27 06:23:22,195][20381] Updated weights for policy 1, policy_version 105195 (0.0011) [2023-12-27 06:23:22,251][20347] Updated weights for policy 0, policy_version 105639 (0.0006) [2023-12-27 06:23:22,258][20381] Updated weights for policy 1, policy_version 105205 (0.0010) [2023-12-27 06:23:22,322][20381] Updated weights for policy 1, policy_version 105215 (0.0009) [2023-12-27 06:23:22,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 54001664. Throughput: 0: 10508.3, 1: 10430.2. Samples: 53966152. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000105216_26943488.pth... [2023-12-27 06:23:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000105640_27058176.pth... [2023-12-27 06:23:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000103968_26624000.pth [2023-12-27 06:23:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000104392_26738688.pth [2023-12-27 06:23:22,920][20347] Updated weights for policy 0, policy_version 105649 (0.0007) [2023-12-27 06:23:22,979][20381] Updated weights for policy 1, policy_version 105225 (0.0008) [2023-12-27 06:23:22,980][20347] Updated weights for policy 0, policy_version 105659 (0.0007) [2023-12-27 06:23:23,028][20381] Updated weights for policy 1, policy_version 105235 (0.0005) [2023-12-27 06:23:23,043][20347] Updated weights for policy 0, policy_version 105669 (0.0007) [2023-12-27 06:23:23,086][20381] Updated weights for policy 1, policy_version 105245 (0.0008) [2023-12-27 06:23:23,669][20347] Updated weights for policy 0, policy_version 105679 (0.0007) [2023-12-27 06:23:23,734][20347] Updated weights for policy 0, policy_version 105689 (0.0007) [2023-12-27 06:23:23,785][20347] Updated weights for policy 0, policy_version 105699 (0.0006) [2023-12-27 06:23:23,787][20381] Updated weights for policy 1, policy_version 105255 (0.0007) [2023-12-27 06:23:23,845][20381] Updated weights for policy 1, policy_version 105265 (0.0007) [2023-12-27 06:23:23,906][20381] Updated weights for policy 1, policy_version 105275 (0.0008) [2023-12-27 06:23:24,435][20347] Updated weights for policy 0, policy_version 105709 (0.0008) [2023-12-27 06:23:24,493][20347] Updated weights for policy 0, policy_version 105719 (0.0010) [2023-12-27 06:23:24,551][20347] Updated weights for policy 0, policy_version 105729 (0.0010) [2023-12-27 06:23:24,611][20381] Updated weights for policy 1, policy_version 105285 (0.0008) [2023-12-27 06:23:24,665][20381] Updated weights for policy 1, policy_version 105295 (0.0007) [2023-12-27 06:23:24,712][20381] Updated weights for policy 1, policy_version 105305 (0.0007) [2023-12-27 06:23:25,180][20347] Updated weights for policy 0, policy_version 105739 (0.0009) [2023-12-27 06:23:25,228][20347] Updated weights for policy 0, policy_version 105749 (0.0010) [2023-12-27 06:23:25,276][20347] Updated weights for policy 0, policy_version 105759 (0.0010) [2023-12-27 06:23:25,406][20381] Updated weights for policy 1, policy_version 105315 (0.0007) [2023-12-27 06:23:25,459][20381] Updated weights for policy 1, policy_version 105325 (0.0007) [2023-12-27 06:23:25,519][20381] Updated weights for policy 1, policy_version 105335 (0.0007) [2023-12-27 06:23:25,870][20347] Updated weights for policy 0, policy_version 105769 (0.0009) [2023-12-27 06:23:25,931][20347] Updated weights for policy 0, policy_version 105779 (0.0005) [2023-12-27 06:23:25,989][20347] Updated weights for policy 0, policy_version 105789 (0.0007) [2023-12-27 06:23:26,044][20347] Updated weights for policy 0, policy_version 105799 (0.0005) [2023-12-27 06:23:26,254][20381] Updated weights for policy 1, policy_version 105345 (0.0008) [2023-12-27 06:23:26,313][20381] Updated weights for policy 1, policy_version 105355 (0.0007) [2023-12-27 06:23:26,370][20381] Updated weights for policy 1, policy_version 105365 (0.0009) [2023-12-27 06:23:26,424][20381] Updated weights for policy 1, policy_version 105375 (0.0008) [2023-12-27 06:23:26,599][20347] Updated weights for policy 0, policy_version 105809 (0.0005) [2023-12-27 06:23:26,651][20347] Updated weights for policy 0, policy_version 105819 (0.0005) [2023-12-27 06:23:26,711][20347] Updated weights for policy 0, policy_version 105829 (0.0006) [2023-12-27 06:23:27,147][20381] Updated weights for policy 1, policy_version 105385 (0.0006) [2023-12-27 06:23:27,197][20381] Updated weights for policy 1, policy_version 105395 (0.0005) [2023-12-27 06:23:27,250][20381] Updated weights for policy 1, policy_version 105405 (0.0006) [2023-12-27 06:23:27,255][20347] Updated weights for policy 0, policy_version 105839 (0.0006) [2023-12-27 06:23:27,307][20347] Updated weights for policy 0, policy_version 105849 (0.0007) [2023-12-27 06:23:27,354][20347] Updated weights for policy 0, policy_version 105859 (0.0005) [2023-12-27 06:23:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 54099968. Throughput: 0: 10507.4, 1: 10392.3. Samples: 54088284. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:27,870][20381] Updated weights for policy 1, policy_version 105415 (0.0005) [2023-12-27 06:23:27,879][20347] Updated weights for policy 0, policy_version 105869 (0.0006) [2023-12-27 06:23:27,928][20381] Updated weights for policy 1, policy_version 105425 (0.0005) [2023-12-27 06:23:27,932][20347] Updated weights for policy 0, policy_version 105879 (0.0007) [2023-12-27 06:23:27,983][20347] Updated weights for policy 0, policy_version 105889 (0.0007) [2023-12-27 06:23:27,985][20381] Updated weights for policy 1, policy_version 105435 (0.0006) [2023-12-27 06:23:28,614][20381] Updated weights for policy 1, policy_version 105445 (0.0007) [2023-12-27 06:23:28,672][20381] Updated weights for policy 1, policy_version 105455 (0.0007) [2023-12-27 06:23:28,685][20347] Updated weights for policy 0, policy_version 105899 (0.0007) [2023-12-27 06:23:28,730][20381] Updated weights for policy 1, policy_version 105465 (0.0006) [2023-12-27 06:23:28,740][20347] Updated weights for policy 0, policy_version 105909 (0.0006) [2023-12-27 06:23:28,794][20347] Updated weights for policy 0, policy_version 105919 (0.0006) [2023-12-27 06:23:29,444][20381] Updated weights for policy 1, policy_version 105475 (0.0007) [2023-12-27 06:23:29,474][20347] Updated weights for policy 0, policy_version 105929 (0.0007) [2023-12-27 06:23:29,494][20381] Updated weights for policy 1, policy_version 105485 (0.0008) [2023-12-27 06:23:29,527][20347] Updated weights for policy 0, policy_version 105939 (0.0005) [2023-12-27 06:23:29,554][20381] Updated weights for policy 1, policy_version 105495 (0.0007) [2023-12-27 06:23:29,582][20347] Updated weights for policy 0, policy_version 105949 (0.0007) [2023-12-27 06:23:29,643][20347] Updated weights for policy 0, policy_version 105959 (0.0007) [2023-12-27 06:23:30,247][20347] Updated weights for policy 0, policy_version 105969 (0.0005) [2023-12-27 06:23:30,259][20381] Updated weights for policy 1, policy_version 105505 (0.0006) [2023-12-27 06:23:30,308][20347] Updated weights for policy 0, policy_version 105979 (0.0006) [2023-12-27 06:23:30,313][20381] Updated weights for policy 1, policy_version 105515 (0.0011) [2023-12-27 06:23:30,365][20381] Updated weights for policy 1, policy_version 105525 (0.0009) [2023-12-27 06:23:30,367][20347] Updated weights for policy 0, policy_version 105989 (0.0006) [2023-12-27 06:23:30,423][20381] Updated weights for policy 1, policy_version 105535 (0.0010) [2023-12-27 06:23:30,959][20347] Updated weights for policy 0, policy_version 105999 (0.0009) [2023-12-27 06:23:31,011][20347] Updated weights for policy 0, policy_version 106009 (0.0006) [2023-12-27 06:23:31,071][20347] Updated weights for policy 0, policy_version 106019 (0.0007) [2023-12-27 06:23:31,118][20381] Updated weights for policy 1, policy_version 105545 (0.0009) [2023-12-27 06:23:31,176][20381] Updated weights for policy 1, policy_version 105555 (0.0006) [2023-12-27 06:23:31,235][20381] Updated weights for policy 1, policy_version 105565 (0.0008) [2023-12-27 06:23:31,744][20347] Updated weights for policy 0, policy_version 106029 (0.0008) [2023-12-27 06:23:31,796][20347] Updated weights for policy 0, policy_version 106039 (0.0009) [2023-12-27 06:23:31,866][20347] Updated weights for policy 0, policy_version 106049 (0.0009) [2023-12-27 06:23:31,890][20381] Updated weights for policy 1, policy_version 105575 (0.0008) [2023-12-27 06:23:31,944][20381] Updated weights for policy 1, policy_version 105585 (0.0006) [2023-12-27 06:23:32,002][20381] Updated weights for policy 1, policy_version 105595 (0.0009) [2023-12-27 06:23:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 54206464. Throughput: 0: 10600.3, 1: 10353.4. Samples: 54215920. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:23:32,498][20347] Updated weights for policy 0, policy_version 106059 (0.0007) [2023-12-27 06:23:32,561][20347] Updated weights for policy 0, policy_version 106069 (0.0006) [2023-12-27 06:23:32,627][20347] Updated weights for policy 0, policy_version 106079 (0.0005) [2023-12-27 06:23:32,730][20381] Updated weights for policy 1, policy_version 105605 (0.0008) [2023-12-27 06:23:32,779][20381] Updated weights for policy 1, policy_version 105615 (0.0007) [2023-12-27 06:23:32,831][20381] Updated weights for policy 1, policy_version 105625 (0.0007) [2023-12-27 06:23:33,258][20347] Updated weights for policy 0, policy_version 106089 (0.0005) [2023-12-27 06:23:33,312][20347] Updated weights for policy 0, policy_version 106099 (0.0006) [2023-12-27 06:23:33,369][20347] Updated weights for policy 0, policy_version 106109 (0.0006) [2023-12-27 06:23:33,429][20347] Updated weights for policy 0, policy_version 106119 (0.0006) [2023-12-27 06:23:33,510][20381] Updated weights for policy 1, policy_version 105635 (0.0007) [2023-12-27 06:23:33,574][20381] Updated weights for policy 1, policy_version 105645 (0.0005) [2023-12-27 06:23:33,634][20381] Updated weights for policy 1, policy_version 105655 (0.0005) [2023-12-27 06:23:34,026][20347] Updated weights for policy 0, policy_version 106129 (0.0006) [2023-12-27 06:23:34,088][20347] Updated weights for policy 0, policy_version 106139 (0.0008) [2023-12-27 06:23:34,151][20347] Updated weights for policy 0, policy_version 106149 (0.0008) [2023-12-27 06:23:34,168][20381] Updated weights for policy 1, policy_version 105665 (0.0006) [2023-12-27 06:23:34,220][20381] Updated weights for policy 1, policy_version 105675 (0.0007) [2023-12-27 06:23:34,271][20381] Updated weights for policy 1, policy_version 105685 (0.0008) [2023-12-27 06:23:34,328][20381] Updated weights for policy 1, policy_version 105695 (0.0008) [2023-12-27 06:23:34,844][20347] Updated weights for policy 0, policy_version 106159 (0.0007) [2023-12-27 06:23:34,905][20347] Updated weights for policy 0, policy_version 106169 (0.0008) [2023-12-27 06:23:34,989][20347] Updated weights for policy 0, policy_version 106179 (0.0007) [2023-12-27 06:23:35,070][20381] Updated weights for policy 1, policy_version 105705 (0.0010) [2023-12-27 06:23:35,123][20381] Updated weights for policy 1, policy_version 105715 (0.0006) [2023-12-27 06:23:35,175][20381] Updated weights for policy 1, policy_version 105725 (0.0005) [2023-12-27 06:23:35,602][20347] Updated weights for policy 0, policy_version 106189 (0.0008) [2023-12-27 06:23:35,663][20347] Updated weights for policy 0, policy_version 106199 (0.0007) [2023-12-27 06:23:35,715][20347] Updated weights for policy 0, policy_version 106209 (0.0005) [2023-12-27 06:23:35,850][20381] Updated weights for policy 1, policy_version 105735 (0.0007) [2023-12-27 06:23:35,899][20381] Updated weights for policy 1, policy_version 105745 (0.0008) [2023-12-27 06:23:35,947][20381] Updated weights for policy 1, policy_version 105755 (0.0007) [2023-12-27 06:23:36,308][20347] Updated weights for policy 0, policy_version 106219 (0.0005) [2023-12-27 06:23:36,366][20347] Updated weights for policy 0, policy_version 106229 (0.0008) [2023-12-27 06:23:36,425][20347] Updated weights for policy 0, policy_version 106239 (0.0007) [2023-12-27 06:23:36,685][20381] Updated weights for policy 1, policy_version 105765 (0.0006) [2023-12-27 06:23:36,738][20381] Updated weights for policy 1, policy_version 105775 (0.0008) [2023-12-27 06:23:36,793][20381] Updated weights for policy 1, policy_version 105785 (0.0008) [2023-12-27 06:23:37,016][20347] Updated weights for policy 0, policy_version 106249 (0.0006) [2023-12-27 06:23:37,070][20347] Updated weights for policy 0, policy_version 106259 (0.0005) [2023-12-27 06:23:37,132][20347] Updated weights for policy 0, policy_version 106269 (0.0006) [2023-12-27 06:23:37,190][20347] Updated weights for policy 0, policy_version 106279 (0.0005) [2023-12-27 06:23:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 54312960. Throughput: 0: 10643.8, 1: 10364.7. Samples: 54278872. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:37,363][19170] Avg episode reward: [(0, '836.710'), (1, '1000.000')] [2023-12-27 06:23:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000106280_27222016.pth... [2023-12-27 06:23:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000105792_27090944.pth... [2023-12-27 06:23:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000105000_26894336.pth [2023-12-27 06:23:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000104608_26787840.pth [2023-12-27 06:23:37,461][20381] Updated weights for policy 1, policy_version 105795 (0.0008) [2023-12-27 06:23:37,510][20381] Updated weights for policy 1, policy_version 105805 (0.0009) [2023-12-27 06:23:37,558][20381] Updated weights for policy 1, policy_version 105815 (0.0008) [2023-12-27 06:23:37,721][20347] Updated weights for policy 0, policy_version 106289 (0.0006) [2023-12-27 06:23:37,774][20347] Updated weights for policy 0, policy_version 106299 (0.0007) [2023-12-27 06:23:37,821][20347] Updated weights for policy 0, policy_version 106309 (0.0008) [2023-12-27 06:23:38,302][20381] Updated weights for policy 1, policy_version 105825 (0.0008) [2023-12-27 06:23:38,361][20381] Updated weights for policy 1, policy_version 105835 (0.0007) [2023-12-27 06:23:38,421][20381] Updated weights for policy 1, policy_version 105845 (0.0007) [2023-12-27 06:23:38,474][20381] Updated weights for policy 1, policy_version 105855 (0.0008) [2023-12-27 06:23:38,500][20347] Updated weights for policy 0, policy_version 106319 (0.0008) [2023-12-27 06:23:38,562][20347] Updated weights for policy 0, policy_version 106329 (0.0008) [2023-12-27 06:23:38,620][20347] Updated weights for policy 0, policy_version 106339 (0.0008) [2023-12-27 06:23:39,242][20381] Updated weights for policy 1, policy_version 105865 (0.0007) [2023-12-27 06:23:39,264][20347] Updated weights for policy 0, policy_version 106349 (0.0007) [2023-12-27 06:23:39,302][20381] Updated weights for policy 1, policy_version 105875 (0.0006) [2023-12-27 06:23:39,323][20347] Updated weights for policy 0, policy_version 106359 (0.0007) [2023-12-27 06:23:39,367][20381] Updated weights for policy 1, policy_version 105885 (0.0007) [2023-12-27 06:23:39,387][20347] Updated weights for policy 0, policy_version 106369 (0.0007) [2023-12-27 06:23:39,965][20381] Updated weights for policy 1, policy_version 105895 (0.0008) [2023-12-27 06:23:40,018][20381] Updated weights for policy 1, policy_version 105905 (0.0007) [2023-12-27 06:23:40,080][20381] Updated weights for policy 1, policy_version 105915 (0.0007) [2023-12-27 06:23:40,155][20347] Updated weights for policy 0, policy_version 106379 (0.0007) [2023-12-27 06:23:40,217][20347] Updated weights for policy 0, policy_version 106389 (0.0005) [2023-12-27 06:23:40,276][20347] Updated weights for policy 0, policy_version 106399 (0.0007) [2023-12-27 06:23:40,773][20381] Updated weights for policy 1, policy_version 105925 (0.0009) [2023-12-27 06:23:40,834][20381] Updated weights for policy 1, policy_version 105935 (0.0008) [2023-12-27 06:23:40,896][20381] Updated weights for policy 1, policy_version 105945 (0.0008) [2023-12-27 06:23:40,922][20347] Updated weights for policy 0, policy_version 106409 (0.0007) [2023-12-27 06:23:40,982][20347] Updated weights for policy 0, policy_version 106419 (0.0007) [2023-12-27 06:23:41,040][20347] Updated weights for policy 0, policy_version 106429 (0.0008) [2023-12-27 06:23:41,096][20347] Updated weights for policy 0, policy_version 106439 (0.0008) [2023-12-27 06:23:41,565][20381] Updated weights for policy 1, policy_version 105955 (0.0008) [2023-12-27 06:23:41,619][20381] Updated weights for policy 1, policy_version 105965 (0.0008) [2023-12-27 06:23:41,683][20381] Updated weights for policy 1, policy_version 105975 (0.0008) [2023-12-27 06:23:41,862][20347] Updated weights for policy 0, policy_version 106449 (0.0009) [2023-12-27 06:23:41,915][20347] Updated weights for policy 0, policy_version 106459 (0.0007) [2023-12-27 06:23:41,968][20347] Updated weights for policy 0, policy_version 106469 (0.0009) [2023-12-27 06:23:42,338][20381] Updated weights for policy 1, policy_version 105985 (0.0008) [2023-12-27 06:23:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 54411264. Throughput: 0: 10642.1, 1: 10357.6. Samples: 54403500. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:42,363][19170] Avg episode reward: [(0, '862.420'), (1, '1000.000')] [2023-12-27 06:23:42,394][20381] Updated weights for policy 1, policy_version 105995 (0.0007) [2023-12-27 06:23:42,452][20381] Updated weights for policy 1, policy_version 106005 (0.0007) [2023-12-27 06:23:42,510][20381] Updated weights for policy 1, policy_version 106015 (0.0008) [2023-12-27 06:23:42,703][20347] Updated weights for policy 0, policy_version 106479 (0.0010) [2023-12-27 06:23:42,759][20347] Updated weights for policy 0, policy_version 106489 (0.0011) [2023-12-27 06:23:42,821][20347] Updated weights for policy 0, policy_version 106499 (0.0011) [2023-12-27 06:23:43,105][20381] Updated weights for policy 1, policy_version 106025 (0.0005) [2023-12-27 06:23:43,160][20381] Updated weights for policy 1, policy_version 106035 (0.0005) [2023-12-27 06:23:43,226][20381] Updated weights for policy 1, policy_version 106045 (0.0005) [2023-12-27 06:23:43,439][20347] Updated weights for policy 0, policy_version 106509 (0.0010) [2023-12-27 06:23:43,499][20347] Updated weights for policy 0, policy_version 106519 (0.0010) [2023-12-27 06:23:43,560][20347] Updated weights for policy 0, policy_version 106529 (0.0010) [2023-12-27 06:23:43,831][20381] Updated weights for policy 1, policy_version 106055 (0.0009) [2023-12-27 06:23:43,882][20381] Updated weights for policy 1, policy_version 106065 (0.0010) [2023-12-27 06:23:43,933][20381] Updated weights for policy 1, policy_version 106075 (0.0010) [2023-12-27 06:23:44,210][20347] Updated weights for policy 0, policy_version 106539 (0.0010) [2023-12-27 06:23:44,268][20347] Updated weights for policy 0, policy_version 106549 (0.0010) [2023-12-27 06:23:44,324][20347] Updated weights for policy 0, policy_version 106559 (0.0010) [2023-12-27 06:23:44,549][20381] Updated weights for policy 1, policy_version 106085 (0.0011) [2023-12-27 06:23:44,594][20381] Updated weights for policy 1, policy_version 106095 (0.0005) [2023-12-27 06:23:44,642][20381] Updated weights for policy 1, policy_version 106105 (0.0005) [2023-12-27 06:23:44,988][20347] Updated weights for policy 0, policy_version 106569 (0.0010) [2023-12-27 06:23:45,046][20347] Updated weights for policy 0, policy_version 106579 (0.0010) [2023-12-27 06:23:45,098][20347] Updated weights for policy 0, policy_version 106589 (0.0010) [2023-12-27 06:23:45,157][20347] Updated weights for policy 0, policy_version 106599 (0.0010) [2023-12-27 06:23:45,321][20381] Updated weights for policy 1, policy_version 106115 (0.0007) [2023-12-27 06:23:45,370][20381] Updated weights for policy 1, policy_version 106125 (0.0009) [2023-12-27 06:23:45,422][20381] Updated weights for policy 1, policy_version 106135 (0.0005) [2023-12-27 06:23:45,827][20347] Updated weights for policy 0, policy_version 106609 (0.0010) [2023-12-27 06:23:45,872][20347] Updated weights for policy 0, policy_version 106619 (0.0010) [2023-12-27 06:23:45,928][20347] Updated weights for policy 0, policy_version 106629 (0.0010) [2023-12-27 06:23:46,058][20381] Updated weights for policy 1, policy_version 106145 (0.0006) [2023-12-27 06:23:46,103][20381] Updated weights for policy 1, policy_version 106155 (0.0007) [2023-12-27 06:23:46,150][20381] Updated weights for policy 1, policy_version 106165 (0.0008) [2023-12-27 06:23:46,203][20381] Updated weights for policy 1, policy_version 106175 (0.0010) [2023-12-27 06:23:46,493][20347] Updated weights for policy 0, policy_version 106639 (0.0009) [2023-12-27 06:23:46,547][20347] Updated weights for policy 0, policy_version 106649 (0.0005) [2023-12-27 06:23:46,596][20347] Updated weights for policy 0, policy_version 106659 (0.0005) [2023-12-27 06:23:46,832][20381] Updated weights for policy 1, policy_version 106185 (0.0006) [2023-12-27 06:23:46,880][20381] Updated weights for policy 1, policy_version 106195 (0.0005) [2023-12-27 06:23:46,936][20381] Updated weights for policy 1, policy_version 106205 (0.0005) [2023-12-27 06:23:47,137][20347] Updated weights for policy 0, policy_version 106669 (0.0005) [2023-12-27 06:23:47,183][20347] Updated weights for policy 0, policy_version 106679 (0.0005) [2023-12-27 06:23:47,239][20347] Updated weights for policy 0, policy_version 106689 (0.0010) [2023-12-27 06:23:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 54525952. Throughput: 0: 10649.3, 1: 10344.3. Samples: 54532648. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:47,363][19170] Avg episode reward: [(0, '902.290'), (1, '1000.000')] [2023-12-27 06:23:47,511][20381] Updated weights for policy 1, policy_version 106215 (0.0007) [2023-12-27 06:23:47,555][20381] Updated weights for policy 1, policy_version 106225 (0.0007) [2023-12-27 06:23:47,602][20381] Updated weights for policy 1, policy_version 106235 (0.0005) [2023-12-27 06:23:47,926][20347] Updated weights for policy 0, policy_version 106699 (0.0009) [2023-12-27 06:23:47,982][20347] Updated weights for policy 0, policy_version 106709 (0.0007) [2023-12-27 06:23:48,029][20347] Updated weights for policy 0, policy_version 106719 (0.0007) [2023-12-27 06:23:48,194][20381] Updated weights for policy 1, policy_version 106245 (0.0007) [2023-12-27 06:23:48,238][20381] Updated weights for policy 1, policy_version 106255 (0.0007) [2023-12-27 06:23:48,283][20381] Updated weights for policy 1, policy_version 106265 (0.0007) [2023-12-27 06:23:48,647][20347] Updated weights for policy 0, policy_version 106729 (0.0005) [2023-12-27 06:23:48,710][20347] Updated weights for policy 0, policy_version 106739 (0.0006) [2023-12-27 06:23:48,772][20347] Updated weights for policy 0, policy_version 106749 (0.0005) [2023-12-27 06:23:48,833][20347] Updated weights for policy 0, policy_version 106759 (0.0005) [2023-12-27 06:23:49,074][20381] Updated weights for policy 1, policy_version 106275 (0.0007) [2023-12-27 06:23:49,124][20381] Updated weights for policy 1, policy_version 106285 (0.0005) [2023-12-27 06:23:49,172][20381] Updated weights for policy 1, policy_version 106295 (0.0005) [2023-12-27 06:23:49,429][20347] Updated weights for policy 0, policy_version 106769 (0.0007) [2023-12-27 06:23:49,484][20347] Updated weights for policy 0, policy_version 106779 (0.0007) [2023-12-27 06:23:49,545][20347] Updated weights for policy 0, policy_version 106789 (0.0008) [2023-12-27 06:23:49,879][20381] Updated weights for policy 1, policy_version 106305 (0.0006) [2023-12-27 06:23:49,938][20381] Updated weights for policy 1, policy_version 106315 (0.0011) [2023-12-27 06:23:50,004][20381] Updated weights for policy 1, policy_version 106325 (0.0011) [2023-12-27 06:23:50,063][20381] Updated weights for policy 1, policy_version 106335 (0.0011) [2023-12-27 06:23:50,201][20347] Updated weights for policy 0, policy_version 106799 (0.0007) [2023-12-27 06:23:50,251][20347] Updated weights for policy 0, policy_version 106809 (0.0005) [2023-12-27 06:23:50,305][20347] Updated weights for policy 0, policy_version 106819 (0.0005) [2023-12-27 06:23:50,756][20381] Updated weights for policy 1, policy_version 106345 (0.0011) [2023-12-27 06:23:50,811][20381] Updated weights for policy 1, policy_version 106355 (0.0010) [2023-12-27 06:23:50,864][20381] Updated weights for policy 1, policy_version 106365 (0.0010) [2023-12-27 06:23:50,881][20347] Updated weights for policy 0, policy_version 106829 (0.0008) [2023-12-27 06:23:50,942][20347] Updated weights for policy 0, policy_version 106839 (0.0010) [2023-12-27 06:23:50,999][20347] Updated weights for policy 0, policy_version 106849 (0.0010) [2023-12-27 06:23:51,573][20381] Updated weights for policy 1, policy_version 106375 (0.0008) [2023-12-27 06:23:51,634][20381] Updated weights for policy 1, policy_version 106385 (0.0007) [2023-12-27 06:23:51,687][20381] Updated weights for policy 1, policy_version 106395 (0.0008) [2023-12-27 06:23:51,757][20347] Updated weights for policy 0, policy_version 106859 (0.0011) [2023-12-27 06:23:51,800][20347] Updated weights for policy 0, policy_version 106869 (0.0006) [2023-12-27 06:23:51,846][20347] Updated weights for policy 0, policy_version 106879 (0.0007) [2023-12-27 06:23:52,337][20381] Updated weights for policy 1, policy_version 106405 (0.0008) [2023-12-27 06:23:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 54624256. Throughput: 0: 10673.3, 1: 10327.5. Samples: 54596016. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:52,363][19170] Avg episode reward: [(0, '903.700'), (1, '1000.000')] [2023-12-27 06:23:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000106888_27377664.pth... [2023-12-27 06:23:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000105640_27058176.pth [2023-12-27 06:23:52,404][20381] Updated weights for policy 1, policy_version 106415 (0.0008) [2023-12-27 06:23:52,471][20381] Updated weights for policy 1, policy_version 106425 (0.0008) [2023-12-27 06:23:52,509][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000106432_27254784.pth... [2023-12-27 06:23:52,512][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000105216_26943488.pth [2023-12-27 06:23:52,517][20347] Updated weights for policy 0, policy_version 106889 (0.0010) [2023-12-27 06:23:52,570][20347] Updated weights for policy 0, policy_version 106899 (0.0008) [2023-12-27 06:23:52,631][20347] Updated weights for policy 0, policy_version 106909 (0.0008) [2023-12-27 06:23:52,691][20347] Updated weights for policy 0, policy_version 106919 (0.0008) [2023-12-27 06:23:53,193][20381] Updated weights for policy 1, policy_version 106435 (0.0008) [2023-12-27 06:23:53,258][20381] Updated weights for policy 1, policy_version 106445 (0.0008) [2023-12-27 06:23:53,274][20347] Updated weights for policy 0, policy_version 106929 (0.0005) [2023-12-27 06:23:53,313][20381] Updated weights for policy 1, policy_version 106455 (0.0008) [2023-12-27 06:23:53,328][20347] Updated weights for policy 0, policy_version 106939 (0.0005) [2023-12-27 06:23:53,385][20347] Updated weights for policy 0, policy_version 106949 (0.0005) [2023-12-27 06:23:53,941][20347] Updated weights for policy 0, policy_version 106959 (0.0007) [2023-12-27 06:23:53,976][20381] Updated weights for policy 1, policy_version 106465 (0.0007) [2023-12-27 06:23:54,001][20347] Updated weights for policy 0, policy_version 106969 (0.0007) [2023-12-27 06:23:54,037][20381] Updated weights for policy 1, policy_version 106475 (0.0007) [2023-12-27 06:23:54,064][20347] Updated weights for policy 0, policy_version 106979 (0.0007) [2023-12-27 06:23:54,099][20381] Updated weights for policy 1, policy_version 106485 (0.0007) [2023-12-27 06:23:54,153][20381] Updated weights for policy 1, policy_version 106495 (0.0008) [2023-12-27 06:23:54,719][20347] Updated weights for policy 0, policy_version 106989 (0.0007) [2023-12-27 06:23:54,766][20347] Updated weights for policy 0, policy_version 106999 (0.0007) [2023-12-27 06:23:54,811][20347] Updated weights for policy 0, policy_version 107009 (0.0008) [2023-12-27 06:23:54,847][20381] Updated weights for policy 1, policy_version 106505 (0.0006) [2023-12-27 06:23:54,907][20381] Updated weights for policy 1, policy_version 106515 (0.0005) [2023-12-27 06:23:54,969][20381] Updated weights for policy 1, policy_version 106525 (0.0007) [2023-12-27 06:23:55,486][20347] Updated weights for policy 0, policy_version 107019 (0.0006) [2023-12-27 06:23:55,534][20381] Updated weights for policy 1, policy_version 106535 (0.0007) [2023-12-27 06:23:55,545][20347] Updated weights for policy 0, policy_version 107029 (0.0005) [2023-12-27 06:23:55,588][20381] Updated weights for policy 1, policy_version 106545 (0.0005) [2023-12-27 06:23:55,608][20347] Updated weights for policy 0, policy_version 107039 (0.0006) [2023-12-27 06:23:55,655][20381] Updated weights for policy 1, policy_version 106555 (0.0006) [2023-12-27 06:23:56,216][20381] Updated weights for policy 1, policy_version 106565 (0.0006) [2023-12-27 06:23:56,273][20381] Updated weights for policy 1, policy_version 106575 (0.0010) [2023-12-27 06:23:56,285][20347] Updated weights for policy 0, policy_version 107049 (0.0007) [2023-12-27 06:23:56,334][20381] Updated weights for policy 1, policy_version 106585 (0.0011) [2023-12-27 06:23:56,337][20347] Updated weights for policy 0, policy_version 107059 (0.0005) [2023-12-27 06:23:56,388][20347] Updated weights for policy 0, policy_version 107069 (0.0006) [2023-12-27 06:23:56,454][20347] Updated weights for policy 0, policy_version 107079 (0.0008) [2023-12-27 06:23:56,865][20381] Updated weights for policy 1, policy_version 106595 (0.0009) [2023-12-27 06:23:56,917][20381] Updated weights for policy 1, policy_version 106605 (0.0008) [2023-12-27 06:23:56,971][20381] Updated weights for policy 1, policy_version 106615 (0.0010) [2023-12-27 06:23:57,103][20347] Updated weights for policy 0, policy_version 107089 (0.0007) [2023-12-27 06:23:57,155][20347] Updated weights for policy 0, policy_version 107099 (0.0008) [2023-12-27 06:23:57,204][20347] Updated weights for policy 0, policy_version 107109 (0.0005) [2023-12-27 06:23:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20882.7). Total num frames: 54738944. Throughput: 0: 10757.5, 1: 10352.8. Samples: 54723720. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 06:23:57,363][19170] Avg episode reward: [(0, '994.590'), (1, '1000.000')] [2023-12-27 06:23:57,621][20381] Updated weights for policy 1, policy_version 106625 (0.0010) [2023-12-27 06:23:57,668][20381] Updated weights for policy 1, policy_version 106635 (0.0010) [2023-12-27 06:23:57,729][20381] Updated weights for policy 1, policy_version 106645 (0.0009) [2023-12-27 06:23:57,768][20347] Updated weights for policy 0, policy_version 107119 (0.0005) [2023-12-27 06:23:57,786][20381] Updated weights for policy 1, policy_version 106655 (0.0010) [2023-12-27 06:23:57,824][20347] Updated weights for policy 0, policy_version 107129 (0.0006) [2023-12-27 06:23:57,875][20347] Updated weights for policy 0, policy_version 107139 (0.0007) [2023-12-27 06:23:58,370][20381] Updated weights for policy 1, policy_version 106665 (0.0007) [2023-12-27 06:23:58,438][20381] Updated weights for policy 1, policy_version 106675 (0.0006) [2023-12-27 06:23:58,507][20381] Updated weights for policy 1, policy_version 106685 (0.0007) [2023-12-27 06:23:58,611][20347] Updated weights for policy 0, policy_version 107149 (0.0008) [2023-12-27 06:23:58,677][20347] Updated weights for policy 0, policy_version 107159 (0.0007) [2023-12-27 06:23:58,735][20347] Updated weights for policy 0, policy_version 107169 (0.0008) [2023-12-27 06:23:59,201][20381] Updated weights for policy 1, policy_version 106695 (0.0007) [2023-12-27 06:23:59,261][20381] Updated weights for policy 1, policy_version 106705 (0.0006) [2023-12-27 06:23:59,323][20381] Updated weights for policy 1, policy_version 106715 (0.0006) [2023-12-27 06:23:59,495][20347] Updated weights for policy 0, policy_version 107179 (0.0008) [2023-12-27 06:23:59,555][20347] Updated weights for policy 0, policy_version 107189 (0.0007) [2023-12-27 06:23:59,614][20347] Updated weights for policy 0, policy_version 107199 (0.0008) [2023-12-27 06:24:00,047][20381] Updated weights for policy 1, policy_version 106725 (0.0007) [2023-12-27 06:24:00,109][20381] Updated weights for policy 1, policy_version 106735 (0.0007) [2023-12-27 06:24:00,167][20381] Updated weights for policy 1, policy_version 106745 (0.0007) [2023-12-27 06:24:00,290][20347] Updated weights for policy 0, policy_version 107209 (0.0007) [2023-12-27 06:24:00,346][20347] Updated weights for policy 0, policy_version 107219 (0.0008) [2023-12-27 06:24:00,403][20347] Updated weights for policy 0, policy_version 107229 (0.0008) [2023-12-27 06:24:00,454][20347] Updated weights for policy 0, policy_version 107239 (0.0007) [2023-12-27 06:24:00,792][20381] Updated weights for policy 1, policy_version 106755 (0.0006) [2023-12-27 06:24:00,837][20381] Updated weights for policy 1, policy_version 106765 (0.0008) [2023-12-27 06:24:00,882][20381] Updated weights for policy 1, policy_version 106775 (0.0007) [2023-12-27 06:24:01,034][20347] Updated weights for policy 0, policy_version 107249 (0.0007) [2023-12-27 06:24:01,094][20347] Updated weights for policy 0, policy_version 107259 (0.0008) [2023-12-27 06:24:01,155][20347] Updated weights for policy 0, policy_version 107269 (0.0007) [2023-12-27 06:24:01,605][20381] Updated weights for policy 1, policy_version 106785 (0.0006) [2023-12-27 06:24:01,665][20381] Updated weights for policy 1, policy_version 106795 (0.0008) [2023-12-27 06:24:01,733][20381] Updated weights for policy 1, policy_version 106805 (0.0007) [2023-12-27 06:24:01,792][20381] Updated weights for policy 1, policy_version 106815 (0.0008) [2023-12-27 06:24:01,841][20347] Updated weights for policy 0, policy_version 107279 (0.0008) [2023-12-27 06:24:01,899][20347] Updated weights for policy 0, policy_version 107289 (0.0008) [2023-12-27 06:24:01,950][20347] Updated weights for policy 0, policy_version 107299 (0.0009) [2023-12-27 06:24:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 54837248. Throughput: 0: 10646.5, 1: 10313.6. Samples: 54848636. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:02,471][20381] Updated weights for policy 1, policy_version 106825 (0.0007) [2023-12-27 06:24:02,516][20381] Updated weights for policy 1, policy_version 106835 (0.0006) [2023-12-27 06:24:02,563][20381] Updated weights for policy 1, policy_version 106845 (0.0005) [2023-12-27 06:24:02,649][20347] Updated weights for policy 0, policy_version 107309 (0.0011) [2023-12-27 06:24:02,700][20347] Updated weights for policy 0, policy_version 107319 (0.0010) [2023-12-27 06:24:02,758][20347] Updated weights for policy 0, policy_version 107329 (0.0010) [2023-12-27 06:24:03,174][20381] Updated weights for policy 1, policy_version 106855 (0.0005) [2023-12-27 06:24:03,227][20381] Updated weights for policy 1, policy_version 106865 (0.0006) [2023-12-27 06:24:03,280][20381] Updated weights for policy 1, policy_version 106875 (0.0006) [2023-12-27 06:24:03,410][20347] Updated weights for policy 0, policy_version 107339 (0.0009) [2023-12-27 06:24:03,466][20347] Updated weights for policy 0, policy_version 107349 (0.0005) [2023-12-27 06:24:03,515][20347] Updated weights for policy 0, policy_version 107359 (0.0009) [2023-12-27 06:24:03,860][20381] Updated weights for policy 1, policy_version 106885 (0.0009) [2023-12-27 06:24:03,917][20381] Updated weights for policy 1, policy_version 106895 (0.0010) [2023-12-27 06:24:03,974][20381] Updated weights for policy 1, policy_version 106905 (0.0008) [2023-12-27 06:24:04,128][20347] Updated weights for policy 0, policy_version 107369 (0.0010) [2023-12-27 06:24:04,182][20347] Updated weights for policy 0, policy_version 107379 (0.0011) [2023-12-27 06:24:04,235][20347] Updated weights for policy 0, policy_version 107389 (0.0009) [2023-12-27 06:24:04,295][20347] Updated weights for policy 0, policy_version 107399 (0.0006) [2023-12-27 06:24:04,608][20381] Updated weights for policy 1, policy_version 106915 (0.0007) [2023-12-27 06:24:04,653][20381] Updated weights for policy 1, policy_version 106925 (0.0010) [2023-12-27 06:24:04,702][20381] Updated weights for policy 1, policy_version 106935 (0.0011) [2023-12-27 06:24:04,845][20347] Updated weights for policy 0, policy_version 107409 (0.0005) [2023-12-27 06:24:04,898][20347] Updated weights for policy 0, policy_version 107419 (0.0010) [2023-12-27 06:24:04,949][20347] Updated weights for policy 0, policy_version 107429 (0.0008) [2023-12-27 06:24:05,405][20381] Updated weights for policy 1, policy_version 106945 (0.0010) [2023-12-27 06:24:05,450][20381] Updated weights for policy 1, policy_version 106955 (0.0008) [2023-12-27 06:24:05,504][20381] Updated weights for policy 1, policy_version 106965 (0.0010) [2023-12-27 06:24:05,552][20381] Updated weights for policy 1, policy_version 106975 (0.0010) [2023-12-27 06:24:05,580][20347] Updated weights for policy 0, policy_version 107439 (0.0009) [2023-12-27 06:24:05,628][20347] Updated weights for policy 0, policy_version 107449 (0.0010) [2023-12-27 06:24:05,682][20347] Updated weights for policy 0, policy_version 107459 (0.0010) [2023-12-27 06:24:06,225][20381] Updated weights for policy 1, policy_version 106985 (0.0011) [2023-12-27 06:24:06,291][20381] Updated weights for policy 1, policy_version 106995 (0.0010) [2023-12-27 06:24:06,296][20347] Updated weights for policy 0, policy_version 107469 (0.0010) [2023-12-27 06:24:06,352][20381] Updated weights for policy 1, policy_version 107005 (0.0008) [2023-12-27 06:24:06,363][20347] Updated weights for policy 0, policy_version 107479 (0.0011) [2023-12-27 06:24:06,431][20347] Updated weights for policy 0, policy_version 107489 (0.0010) [2023-12-27 06:24:07,052][20347] Updated weights for policy 0, policy_version 107499 (0.0009) [2023-12-27 06:24:07,077][20381] Updated weights for policy 1, policy_version 107015 (0.0006) [2023-12-27 06:24:07,106][20347] Updated weights for policy 0, policy_version 107509 (0.0007) [2023-12-27 06:24:07,133][20381] Updated weights for policy 1, policy_version 107025 (0.0009) [2023-12-27 06:24:07,162][20347] Updated weights for policy 0, policy_version 107519 (0.0006) [2023-12-27 06:24:07,192][20381] Updated weights for policy 1, policy_version 107035 (0.0011) [2023-12-27 06:24:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21162.7, 300 sec: 20910.4). Total num frames: 54951936. Throughput: 0: 10715.8, 1: 10376.6. Samples: 54915308. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000107040_27410432.pth... [2023-12-27 06:24:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000107528_27541504.pth... [2023-12-27 06:24:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000105792_27090944.pth [2023-12-27 06:24:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000106280_27222016.pth [2023-12-27 06:24:07,824][20347] Updated weights for policy 0, policy_version 107529 (0.0006) [2023-12-27 06:24:07,884][20347] Updated weights for policy 0, policy_version 107539 (0.0006) [2023-12-27 06:24:07,888][20381] Updated weights for policy 1, policy_version 107045 (0.0011) [2023-12-27 06:24:07,935][20347] Updated weights for policy 0, policy_version 107549 (0.0005) [2023-12-27 06:24:07,950][20381] Updated weights for policy 1, policy_version 107055 (0.0011) [2023-12-27 06:24:07,994][20347] Updated weights for policy 0, policy_version 107559 (0.0006) [2023-12-27 06:24:08,010][20381] Updated weights for policy 1, policy_version 107065 (0.0011) [2023-12-27 06:24:08,626][20347] Updated weights for policy 0, policy_version 107569 (0.0010) [2023-12-27 06:24:08,684][20347] Updated weights for policy 0, policy_version 107579 (0.0010) [2023-12-27 06:24:08,702][20381] Updated weights for policy 1, policy_version 107075 (0.0011) [2023-12-27 06:24:08,740][20347] Updated weights for policy 0, policy_version 107589 (0.0010) [2023-12-27 06:24:08,757][20381] Updated weights for policy 1, policy_version 107085 (0.0006) [2023-12-27 06:24:08,812][20381] Updated weights for policy 1, policy_version 107095 (0.0007) [2023-12-27 06:24:09,429][20347] Updated weights for policy 0, policy_version 107599 (0.0008) [2023-12-27 06:24:09,483][20347] Updated weights for policy 0, policy_version 107609 (0.0010) [2023-12-27 06:24:09,496][20381] Updated weights for policy 1, policy_version 107105 (0.0008) [2023-12-27 06:24:09,540][20347] Updated weights for policy 0, policy_version 107619 (0.0010) [2023-12-27 06:24:09,561][20381] Updated weights for policy 1, policy_version 107115 (0.0006) [2023-12-27 06:24:09,628][20381] Updated weights for policy 1, policy_version 107125 (0.0007) [2023-12-27 06:24:09,690][20381] Updated weights for policy 1, policy_version 107135 (0.0009) [2023-12-27 06:24:10,211][20347] Updated weights for policy 0, policy_version 107629 (0.0009) [2023-12-27 06:24:10,269][20347] Updated weights for policy 0, policy_version 107639 (0.0008) [2023-12-27 06:24:10,329][20347] Updated weights for policy 0, policy_version 107649 (0.0008) [2023-12-27 06:24:10,401][20381] Updated weights for policy 1, policy_version 107145 (0.0007) [2023-12-27 06:24:10,463][20381] Updated weights for policy 1, policy_version 107155 (0.0007) [2023-12-27 06:24:10,523][20381] Updated weights for policy 1, policy_version 107165 (0.0009) [2023-12-27 06:24:10,955][20347] Updated weights for policy 0, policy_version 107659 (0.0007) [2023-12-27 06:24:11,007][20347] Updated weights for policy 0, policy_version 107669 (0.0008) [2023-12-27 06:24:11,076][20347] Updated weights for policy 0, policy_version 107679 (0.0009) [2023-12-27 06:24:11,188][20381] Updated weights for policy 1, policy_version 107175 (0.0008) [2023-12-27 06:24:11,243][20381] Updated weights for policy 1, policy_version 107185 (0.0007) [2023-12-27 06:24:11,303][20381] Updated weights for policy 1, policy_version 107195 (0.0007) [2023-12-27 06:24:11,795][20347] Updated weights for policy 0, policy_version 107689 (0.0008) [2023-12-27 06:24:11,849][20347] Updated weights for policy 0, policy_version 107699 (0.0007) [2023-12-27 06:24:11,903][20347] Updated weights for policy 0, policy_version 107709 (0.0007) [2023-12-27 06:24:11,959][20347] Updated weights for policy 0, policy_version 107719 (0.0008) [2023-12-27 06:24:12,033][20381] Updated weights for policy 1, policy_version 107205 (0.0007) [2023-12-27 06:24:12,099][20381] Updated weights for policy 1, policy_version 107215 (0.0008) [2023-12-27 06:24:12,159][20381] Updated weights for policy 1, policy_version 107225 (0.0008) [2023-12-27 06:24:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.7, 300 sec: 20854.9). Total num frames: 55050240. Throughput: 0: 10744.4, 1: 10396.4. Samples: 55039620. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:12,651][20347] Updated weights for policy 0, policy_version 107729 (0.0007) [2023-12-27 06:24:12,719][20347] Updated weights for policy 0, policy_version 107739 (0.0008) [2023-12-27 06:24:12,786][20347] Updated weights for policy 0, policy_version 107749 (0.0006) [2023-12-27 06:24:12,797][20381] Updated weights for policy 1, policy_version 107235 (0.0007) [2023-12-27 06:24:12,858][20381] Updated weights for policy 1, policy_version 107245 (0.0006) [2023-12-27 06:24:12,918][20381] Updated weights for policy 1, policy_version 107255 (0.0006) [2023-12-27 06:24:13,478][20381] Updated weights for policy 1, policy_version 107265 (0.0006) [2023-12-27 06:24:13,506][20347] Updated weights for policy 0, policy_version 107759 (0.0007) [2023-12-27 06:24:13,522][20381] Updated weights for policy 1, policy_version 107275 (0.0006) [2023-12-27 06:24:13,565][20347] Updated weights for policy 0, policy_version 107769 (0.0006) [2023-12-27 06:24:13,569][20381] Updated weights for policy 1, policy_version 107285 (0.0007) [2023-12-27 06:24:13,625][20347] Updated weights for policy 0, policy_version 107779 (0.0007) [2023-12-27 06:24:13,625][20381] Updated weights for policy 1, policy_version 107295 (0.0006) [2023-12-27 06:24:14,196][20381] Updated weights for policy 1, policy_version 107305 (0.0006) [2023-12-27 06:24:14,237][20347] Updated weights for policy 0, policy_version 107789 (0.0006) [2023-12-27 06:24:14,247][20381] Updated weights for policy 1, policy_version 107315 (0.0008) [2023-12-27 06:24:14,298][20347] Updated weights for policy 0, policy_version 107799 (0.0005) [2023-12-27 06:24:14,305][20381] Updated weights for policy 1, policy_version 107325 (0.0010) [2023-12-27 06:24:14,358][20347] Updated weights for policy 0, policy_version 107809 (0.0008) [2023-12-27 06:24:14,899][20381] Updated weights for policy 1, policy_version 107335 (0.0007) [2023-12-27 06:24:14,957][20381] Updated weights for policy 1, policy_version 107345 (0.0006) [2023-12-27 06:24:14,998][20347] Updated weights for policy 0, policy_version 107819 (0.0010) [2023-12-27 06:24:15,018][20381] Updated weights for policy 1, policy_version 107355 (0.0011) [2023-12-27 06:24:15,060][20347] Updated weights for policy 0, policy_version 107829 (0.0010) [2023-12-27 06:24:15,119][20347] Updated weights for policy 0, policy_version 107839 (0.0011) [2023-12-27 06:24:15,674][20381] Updated weights for policy 1, policy_version 107365 (0.0011) [2023-12-27 06:24:15,708][20347] Updated weights for policy 0, policy_version 107849 (0.0010) [2023-12-27 06:24:15,727][20381] Updated weights for policy 1, policy_version 107375 (0.0010) [2023-12-27 06:24:15,768][20347] Updated weights for policy 0, policy_version 107859 (0.0007) [2023-12-27 06:24:15,780][20381] Updated weights for policy 1, policy_version 107385 (0.0008) [2023-12-27 06:24:15,824][20347] Updated weights for policy 0, policy_version 107869 (0.0010) [2023-12-27 06:24:15,888][20347] Updated weights for policy 0, policy_version 107879 (0.0009) [2023-12-27 06:24:16,468][20381] Updated weights for policy 1, policy_version 107395 (0.0006) [2023-12-27 06:24:16,517][20381] Updated weights for policy 1, policy_version 107405 (0.0008) [2023-12-27 06:24:16,534][20347] Updated weights for policy 0, policy_version 107889 (0.0010) [2023-12-27 06:24:16,565][20381] Updated weights for policy 1, policy_version 107415 (0.0008) [2023-12-27 06:24:16,583][20347] Updated weights for policy 0, policy_version 107899 (0.0010) [2023-12-27 06:24:16,637][20347] Updated weights for policy 0, policy_version 107909 (0.0010) [2023-12-27 06:24:17,281][20381] Updated weights for policy 1, policy_version 107425 (0.0006) [2023-12-27 06:24:17,318][20347] Updated weights for policy 0, policy_version 107919 (0.0010) [2023-12-27 06:24:17,338][20381] Updated weights for policy 1, policy_version 107435 (0.0007) [2023-12-27 06:24:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 55148544. Throughput: 0: 10634.7, 1: 10470.1. Samples: 55165636. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:17,369][20347] Updated weights for policy 0, policy_version 107929 (0.0010) [2023-12-27 06:24:17,389][20381] Updated weights for policy 1, policy_version 107445 (0.0005) [2023-12-27 06:24:17,426][20347] Updated weights for policy 0, policy_version 107939 (0.0010) [2023-12-27 06:24:17,439][20381] Updated weights for policy 1, policy_version 107455 (0.0005) [2023-12-27 06:24:18,059][20381] Updated weights for policy 1, policy_version 107465 (0.0005) [2023-12-27 06:24:18,108][20347] Updated weights for policy 0, policy_version 107949 (0.0010) [2023-12-27 06:24:18,113][20381] Updated weights for policy 1, policy_version 107475 (0.0009) [2023-12-27 06:24:18,162][20347] Updated weights for policy 0, policy_version 107959 (0.0010) [2023-12-27 06:24:18,162][20381] Updated weights for policy 1, policy_version 107485 (0.0010) [2023-12-27 06:24:18,218][20347] Updated weights for policy 0, policy_version 107969 (0.0010) [2023-12-27 06:24:18,812][20381] Updated weights for policy 1, policy_version 107495 (0.0008) [2023-12-27 06:24:18,864][20381] Updated weights for policy 1, policy_version 107505 (0.0007) [2023-12-27 06:24:18,889][20347] Updated weights for policy 0, policy_version 107979 (0.0010) [2023-12-27 06:24:18,914][20381] Updated weights for policy 1, policy_version 107515 (0.0006) [2023-12-27 06:24:18,941][20347] Updated weights for policy 0, policy_version 107989 (0.0008) [2023-12-27 06:24:18,992][20347] Updated weights for policy 0, policy_version 107999 (0.0009) [2023-12-27 06:24:19,631][20347] Updated weights for policy 0, policy_version 108009 (0.0010) [2023-12-27 06:24:19,655][20381] Updated weights for policy 1, policy_version 107525 (0.0006) [2023-12-27 06:24:19,690][20347] Updated weights for policy 0, policy_version 108019 (0.0009) [2023-12-27 06:24:19,708][20381] Updated weights for policy 1, policy_version 107535 (0.0006) [2023-12-27 06:24:19,744][20347] Updated weights for policy 0, policy_version 108029 (0.0009) [2023-12-27 06:24:19,766][20381] Updated weights for policy 1, policy_version 107545 (0.0006) [2023-12-27 06:24:19,797][20347] Updated weights for policy 0, policy_version 108039 (0.0008) [2023-12-27 06:24:20,456][20381] Updated weights for policy 1, policy_version 107555 (0.0008) [2023-12-27 06:24:20,503][20381] Updated weights for policy 1, policy_version 107565 (0.0011) [2023-12-27 06:24:20,509][20347] Updated weights for policy 0, policy_version 108049 (0.0006) [2023-12-27 06:24:20,553][20381] Updated weights for policy 1, policy_version 107575 (0.0011) [2023-12-27 06:24:20,566][20347] Updated weights for policy 0, policy_version 108059 (0.0006) [2023-12-27 06:24:20,628][20347] Updated weights for policy 0, policy_version 108069 (0.0006) [2023-12-27 06:24:21,260][20381] Updated weights for policy 1, policy_version 107585 (0.0011) [2023-12-27 06:24:21,318][20381] Updated weights for policy 1, policy_version 107595 (0.0010) [2023-12-27 06:24:21,359][20347] Updated weights for policy 0, policy_version 108079 (0.0007) [2023-12-27 06:24:21,383][20381] Updated weights for policy 1, policy_version 107605 (0.0009) [2023-12-27 06:24:21,422][20347] Updated weights for policy 0, policy_version 108089 (0.0008) [2023-12-27 06:24:21,448][20381] Updated weights for policy 1, policy_version 107615 (0.0009) [2023-12-27 06:24:21,483][20347] Updated weights for policy 0, policy_version 108099 (0.0008) [2023-12-27 06:24:22,175][20347] Updated weights for policy 0, policy_version 108109 (0.0007) [2023-12-27 06:24:22,187][20381] Updated weights for policy 1, policy_version 107625 (0.0006) [2023-12-27 06:24:22,223][20347] Updated weights for policy 0, policy_version 108119 (0.0005) [2023-12-27 06:24:22,241][20381] Updated weights for policy 1, policy_version 107635 (0.0010) [2023-12-27 06:24:22,278][20347] Updated weights for policy 0, policy_version 108129 (0.0006) [2023-12-27 06:24:22,300][20381] Updated weights for policy 1, policy_version 107645 (0.0011) [2023-12-27 06:24:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20882.6). Total num frames: 55263232. Throughput: 0: 10613.0, 1: 10475.4. Samples: 55227856. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000107648_27566080.pth... [2023-12-27 06:24:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000108136_27697152.pth... [2023-12-27 06:24:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000106432_27254784.pth [2023-12-27 06:24:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000106888_27377664.pth [2023-12-27 06:24:23,003][20381] Updated weights for policy 1, policy_version 107655 (0.0011) [2023-12-27 06:24:23,019][20347] Updated weights for policy 0, policy_version 108139 (0.0007) [2023-12-27 06:24:23,068][20381] Updated weights for policy 1, policy_version 107665 (0.0011) [2023-12-27 06:24:23,075][20347] Updated weights for policy 0, policy_version 108149 (0.0006) [2023-12-27 06:24:23,123][20381] Updated weights for policy 1, policy_version 107675 (0.0011) [2023-12-27 06:24:23,137][20347] Updated weights for policy 0, policy_version 108159 (0.0007) [2023-12-27 06:24:23,830][20381] Updated weights for policy 1, policy_version 107685 (0.0011) [2023-12-27 06:24:23,832][20347] Updated weights for policy 0, policy_version 108169 (0.0007) [2023-12-27 06:24:23,883][20381] Updated weights for policy 1, policy_version 107695 (0.0010) [2023-12-27 06:24:23,892][20347] Updated weights for policy 0, policy_version 108179 (0.0007) [2023-12-27 06:24:23,933][20381] Updated weights for policy 1, policy_version 107705 (0.0011) [2023-12-27 06:24:23,949][20347] Updated weights for policy 0, policy_version 108189 (0.0006) [2023-12-27 06:24:24,007][20347] Updated weights for policy 0, policy_version 108199 (0.0008) [2023-12-27 06:24:24,660][20381] Updated weights for policy 1, policy_version 107715 (0.0009) [2023-12-27 06:24:24,697][20347] Updated weights for policy 0, policy_version 108209 (0.0006) [2023-12-27 06:24:24,714][20381] Updated weights for policy 1, policy_version 107725 (0.0009) [2023-12-27 06:24:24,754][20347] Updated weights for policy 0, policy_version 108219 (0.0007) [2023-12-27 06:24:24,769][20381] Updated weights for policy 1, policy_version 107735 (0.0011) [2023-12-27 06:24:24,811][20347] Updated weights for policy 0, policy_version 108229 (0.0007) [2023-12-27 06:24:25,463][20381] Updated weights for policy 1, policy_version 107745 (0.0010) [2023-12-27 06:24:25,494][20347] Updated weights for policy 0, policy_version 108239 (0.0007) [2023-12-27 06:24:25,521][20381] Updated weights for policy 1, policy_version 107755 (0.0011) [2023-12-27 06:24:25,546][20347] Updated weights for policy 0, policy_version 108249 (0.0006) [2023-12-27 06:24:25,579][20381] Updated weights for policy 1, policy_version 107765 (0.0011) [2023-12-27 06:24:25,594][20347] Updated weights for policy 0, policy_version 108259 (0.0007) [2023-12-27 06:24:25,637][20381] Updated weights for policy 1, policy_version 107775 (0.0011) [2023-12-27 06:24:26,310][20381] Updated weights for policy 1, policy_version 107785 (0.0010) [2023-12-27 06:24:26,318][20347] Updated weights for policy 0, policy_version 108269 (0.0006) [2023-12-27 06:24:26,367][20347] Updated weights for policy 0, policy_version 108279 (0.0006) [2023-12-27 06:24:26,372][20381] Updated weights for policy 1, policy_version 107795 (0.0010) [2023-12-27 06:24:26,417][20347] Updated weights for policy 0, policy_version 108289 (0.0007) [2023-12-27 06:24:26,418][20381] Updated weights for policy 1, policy_version 107805 (0.0010) [2023-12-27 06:24:27,061][20347] Updated weights for policy 0, policy_version 108299 (0.0006) [2023-12-27 06:24:27,066][20381] Updated weights for policy 1, policy_version 107815 (0.0009) [2023-12-27 06:24:27,120][20381] Updated weights for policy 1, policy_version 107825 (0.0007) [2023-12-27 06:24:27,120][20347] Updated weights for policy 0, policy_version 108309 (0.0005) [2023-12-27 06:24:27,175][20381] Updated weights for policy 1, policy_version 107835 (0.0006) [2023-12-27 06:24:27,185][20347] Updated weights for policy 0, policy_version 108319 (0.0005) [2023-12-27 06:24:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 55361536. Throughput: 0: 10502.9, 1: 10455.6. Samples: 55346636. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:27,696][20347] Updated weights for policy 0, policy_version 108329 (0.0005) [2023-12-27 06:24:27,744][20347] Updated weights for policy 0, policy_version 108339 (0.0007) [2023-12-27 06:24:27,748][20381] Updated weights for policy 1, policy_version 107845 (0.0006) [2023-12-27 06:24:27,791][20347] Updated weights for policy 0, policy_version 108349 (0.0008) [2023-12-27 06:24:27,795][20381] Updated weights for policy 1, policy_version 107855 (0.0006) [2023-12-27 06:24:27,837][20347] Updated weights for policy 0, policy_version 108359 (0.0006) [2023-12-27 06:24:27,844][20381] Updated weights for policy 1, policy_version 107865 (0.0010) [2023-12-27 06:24:28,475][20347] Updated weights for policy 0, policy_version 108369 (0.0005) [2023-12-27 06:24:28,530][20347] Updated weights for policy 0, policy_version 108379 (0.0005) [2023-12-27 06:24:28,555][20381] Updated weights for policy 1, policy_version 107875 (0.0010) [2023-12-27 06:24:28,583][20347] Updated weights for policy 0, policy_version 108389 (0.0007) [2023-12-27 06:24:28,616][20381] Updated weights for policy 1, policy_version 107885 (0.0011) [2023-12-27 06:24:28,676][20381] Updated weights for policy 1, policy_version 107895 (0.0006) [2023-12-27 06:24:29,153][20347] Updated weights for policy 0, policy_version 108399 (0.0007) [2023-12-27 06:24:29,202][20347] Updated weights for policy 0, policy_version 108409 (0.0005) [2023-12-27 06:24:29,263][20347] Updated weights for policy 0, policy_version 108419 (0.0006) [2023-12-27 06:24:29,329][20381] Updated weights for policy 1, policy_version 107905 (0.0005) [2023-12-27 06:24:29,397][20381] Updated weights for policy 1, policy_version 107915 (0.0008) [2023-12-27 06:24:29,461][20381] Updated weights for policy 1, policy_version 107925 (0.0006) [2023-12-27 06:24:29,514][20381] Updated weights for policy 1, policy_version 107935 (0.0008) [2023-12-27 06:24:29,901][20347] Updated weights for policy 0, policy_version 108429 (0.0006) [2023-12-27 06:24:29,960][20347] Updated weights for policy 0, policy_version 108439 (0.0007) [2023-12-27 06:24:30,018][20347] Updated weights for policy 0, policy_version 108449 (0.0008) [2023-12-27 06:24:30,131][20381] Updated weights for policy 1, policy_version 107945 (0.0005) [2023-12-27 06:24:30,190][20381] Updated weights for policy 1, policy_version 107955 (0.0005) [2023-12-27 06:24:30,240][20381] Updated weights for policy 1, policy_version 107965 (0.0005) [2023-12-27 06:24:30,734][20347] Updated weights for policy 0, policy_version 108459 (0.0006) [2023-12-27 06:24:30,784][20347] Updated weights for policy 0, policy_version 108469 (0.0005) [2023-12-27 06:24:30,806][20381] Updated weights for policy 1, policy_version 107975 (0.0007) [2023-12-27 06:24:30,837][20347] Updated weights for policy 0, policy_version 108479 (0.0005) [2023-12-27 06:24:30,864][20381] Updated weights for policy 1, policy_version 107985 (0.0010) [2023-12-27 06:24:30,926][20381] Updated weights for policy 1, policy_version 107995 (0.0009) [2023-12-27 06:24:31,489][20347] Updated weights for policy 0, policy_version 108489 (0.0005) [2023-12-27 06:24:31,547][20347] Updated weights for policy 0, policy_version 108499 (0.0006) [2023-12-27 06:24:31,588][20381] Updated weights for policy 1, policy_version 108005 (0.0009) [2023-12-27 06:24:31,604][20347] Updated weights for policy 0, policy_version 108509 (0.0005) [2023-12-27 06:24:31,648][20381] Updated weights for policy 1, policy_version 108015 (0.0011) [2023-12-27 06:24:31,666][20347] Updated weights for policy 0, policy_version 108519 (0.0006) [2023-12-27 06:24:31,703][20381] Updated weights for policy 1, policy_version 108025 (0.0011) [2023-12-27 06:24:32,353][20347] Updated weights for policy 0, policy_version 108529 (0.0008) [2023-12-27 06:24:32,360][20381] Updated weights for policy 1, policy_version 108035 (0.0010) [2023-12-27 06:24:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 55459840. Throughput: 0: 10543.7, 1: 10428.4. Samples: 55476396. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:32,413][20347] Updated weights for policy 0, policy_version 108539 (0.0006) [2023-12-27 06:24:32,418][20381] Updated weights for policy 1, policy_version 108045 (0.0009) [2023-12-27 06:24:32,468][20347] Updated weights for policy 0, policy_version 108549 (0.0005) [2023-12-27 06:24:32,476][20381] Updated weights for policy 1, policy_version 108055 (0.0011) [2023-12-27 06:24:33,058][20347] Updated weights for policy 0, policy_version 108559 (0.0005) [2023-12-27 06:24:33,074][20381] Updated weights for policy 1, policy_version 108065 (0.0008) [2023-12-27 06:24:33,113][20347] Updated weights for policy 0, policy_version 108569 (0.0006) [2023-12-27 06:24:33,127][20381] Updated weights for policy 1, policy_version 108075 (0.0005) [2023-12-27 06:24:33,170][20347] Updated weights for policy 0, policy_version 108579 (0.0007) [2023-12-27 06:24:33,187][20381] Updated weights for policy 1, policy_version 108085 (0.0007) [2023-12-27 06:24:33,232][20381] Updated weights for policy 1, policy_version 108095 (0.0010) [2023-12-27 06:24:33,827][20347] Updated weights for policy 0, policy_version 108589 (0.0006) [2023-12-27 06:24:33,875][20347] Updated weights for policy 0, policy_version 108599 (0.0007) [2023-12-27 06:24:33,894][20381] Updated weights for policy 1, policy_version 108105 (0.0010) [2023-12-27 06:24:33,923][20347] Updated weights for policy 0, policy_version 108609 (0.0006) [2023-12-27 06:24:33,947][20381] Updated weights for policy 1, policy_version 108115 (0.0011) [2023-12-27 06:24:33,998][20381] Updated weights for policy 1, policy_version 108125 (0.0010) [2023-12-27 06:24:34,583][20381] Updated weights for policy 1, policy_version 108135 (0.0008) [2023-12-27 06:24:34,644][20381] Updated weights for policy 1, policy_version 108145 (0.0005) [2023-12-27 06:24:34,669][20347] Updated weights for policy 0, policy_version 108619 (0.0006) [2023-12-27 06:24:34,704][20381] Updated weights for policy 1, policy_version 108155 (0.0006) [2023-12-27 06:24:34,734][20347] Updated weights for policy 0, policy_version 108629 (0.0007) [2023-12-27 06:24:34,792][20347] Updated weights for policy 0, policy_version 108639 (0.0008) [2023-12-27 06:24:35,365][20347] Updated weights for policy 0, policy_version 108649 (0.0006) [2023-12-27 06:24:35,368][20381] Updated weights for policy 1, policy_version 108165 (0.0007) [2023-12-27 06:24:35,415][20381] Updated weights for policy 1, policy_version 108175 (0.0008) [2023-12-27 06:24:35,417][20347] Updated weights for policy 0, policy_version 108659 (0.0007) [2023-12-27 06:24:35,462][20381] Updated weights for policy 1, policy_version 108185 (0.0007) [2023-12-27 06:24:35,476][20347] Updated weights for policy 0, policy_version 108669 (0.0007) [2023-12-27 06:24:35,533][20347] Updated weights for policy 0, policy_version 108679 (0.0007) [2023-12-27 06:24:36,194][20381] Updated weights for policy 1, policy_version 108195 (0.0007) [2023-12-27 06:24:36,235][20347] Updated weights for policy 0, policy_version 108689 (0.0007) [2023-12-27 06:24:36,241][20381] Updated weights for policy 1, policy_version 108205 (0.0006) [2023-12-27 06:24:36,293][20381] Updated weights for policy 1, policy_version 108215 (0.0007) [2023-12-27 06:24:36,295][20347] Updated weights for policy 0, policy_version 108699 (0.0006) [2023-12-27 06:24:36,352][20347] Updated weights for policy 0, policy_version 108709 (0.0006) [2023-12-27 06:24:36,967][20381] Updated weights for policy 1, policy_version 108225 (0.0008) [2023-12-27 06:24:37,028][20381] Updated weights for policy 1, policy_version 108235 (0.0006) [2023-12-27 06:24:37,081][20381] Updated weights for policy 1, policy_version 108245 (0.0010) [2023-12-27 06:24:37,086][20347] Updated weights for policy 0, policy_version 108719 (0.0006) [2023-12-27 06:24:37,135][20347] Updated weights for policy 0, policy_version 108729 (0.0006) [2023-12-27 06:24:37,136][20381] Updated weights for policy 1, policy_version 108255 (0.0010) [2023-12-27 06:24:37,194][20347] Updated weights for policy 0, policy_version 108739 (0.0007) [2023-12-27 06:24:37,363][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 55574528. Throughput: 0: 10514.9, 1: 10472.8. Samples: 55540464. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000108744_27852800.pth... [2023-12-27 06:24:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000108256_27721728.pth... [2023-12-27 06:24:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000107528_27541504.pth [2023-12-27 06:24:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000107040_27410432.pth [2023-12-27 06:24:37,735][20381] Updated weights for policy 1, policy_version 108265 (0.0007) [2023-12-27 06:24:37,792][20381] Updated weights for policy 1, policy_version 108275 (0.0007) [2023-12-27 06:24:37,850][20381] Updated weights for policy 1, policy_version 108285 (0.0008) [2023-12-27 06:24:37,927][20347] Updated weights for policy 0, policy_version 108749 (0.0008) [2023-12-27 06:24:37,981][20347] Updated weights for policy 0, policy_version 108759 (0.0007) [2023-12-27 06:24:38,032][20347] Updated weights for policy 0, policy_version 108769 (0.0006) [2023-12-27 06:24:38,546][20381] Updated weights for policy 1, policy_version 108295 (0.0009) [2023-12-27 06:24:38,601][20381] Updated weights for policy 1, policy_version 108305 (0.0010) [2023-12-27 06:24:38,653][20381] Updated weights for policy 1, policy_version 108315 (0.0010) [2023-12-27 06:24:38,707][20347] Updated weights for policy 0, policy_version 108779 (0.0006) [2023-12-27 06:24:38,757][20347] Updated weights for policy 0, policy_version 108789 (0.0007) [2023-12-27 06:24:38,812][20347] Updated weights for policy 0, policy_version 108799 (0.0007) [2023-12-27 06:24:39,363][20381] Updated weights for policy 1, policy_version 108325 (0.0010) [2023-12-27 06:24:39,429][20381] Updated weights for policy 1, policy_version 108335 (0.0011) [2023-12-27 06:24:39,491][20381] Updated weights for policy 1, policy_version 108345 (0.0011) [2023-12-27 06:24:39,538][20347] Updated weights for policy 0, policy_version 108809 (0.0007) [2023-12-27 06:24:39,599][20347] Updated weights for policy 0, policy_version 108819 (0.0008) [2023-12-27 06:24:39,654][20347] Updated weights for policy 0, policy_version 108829 (0.0007) [2023-12-27 06:24:39,716][20347] Updated weights for policy 0, policy_version 108839 (0.0007) [2023-12-27 06:24:40,194][20381] Updated weights for policy 1, policy_version 108355 (0.0010) [2023-12-27 06:24:40,252][20381] Updated weights for policy 1, policy_version 108365 (0.0010) [2023-12-27 06:24:40,313][20381] Updated weights for policy 1, policy_version 108375 (0.0010) [2023-12-27 06:24:40,413][20347] Updated weights for policy 0, policy_version 108849 (0.0007) [2023-12-27 06:24:40,467][20347] Updated weights for policy 0, policy_version 108859 (0.0007) [2023-12-27 06:24:40,522][20347] Updated weights for policy 0, policy_version 108869 (0.0008) [2023-12-27 06:24:40,991][20381] Updated weights for policy 1, policy_version 108385 (0.0010) [2023-12-27 06:24:41,054][20381] Updated weights for policy 1, policy_version 108395 (0.0007) [2023-12-27 06:24:41,108][20381] Updated weights for policy 1, policy_version 108405 (0.0008) [2023-12-27 06:24:41,172][20381] Updated weights for policy 1, policy_version 108415 (0.0009) [2023-12-27 06:24:41,224][20347] Updated weights for policy 0, policy_version 108879 (0.0007) [2023-12-27 06:24:41,286][20347] Updated weights for policy 0, policy_version 108889 (0.0008) [2023-12-27 06:24:41,344][20347] Updated weights for policy 0, policy_version 108899 (0.0007) [2023-12-27 06:24:41,843][20381] Updated weights for policy 1, policy_version 108425 (0.0007) [2023-12-27 06:24:41,906][20381] Updated weights for policy 1, policy_version 108435 (0.0006) [2023-12-27 06:24:41,972][20381] Updated weights for policy 1, policy_version 108445 (0.0006) [2023-12-27 06:24:42,118][20347] Updated weights for policy 0, policy_version 108909 (0.0007) [2023-12-27 06:24:42,174][20347] Updated weights for policy 0, policy_version 108919 (0.0008) [2023-12-27 06:24:42,233][20347] Updated weights for policy 0, policy_version 108929 (0.0007) [2023-12-27 06:24:42,363][19170] Fps is (10 sec: 21298.7, 60 sec: 21026.0, 300 sec: 20854.9). Total num frames: 55672832. Throughput: 0: 10397.7, 1: 10414.5. Samples: 55660276. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:42,608][20381] Updated weights for policy 1, policy_version 108455 (0.0007) [2023-12-27 06:24:42,666][20381] Updated weights for policy 1, policy_version 108465 (0.0006) [2023-12-27 06:24:42,723][20381] Updated weights for policy 1, policy_version 108475 (0.0006) [2023-12-27 06:24:42,921][20347] Updated weights for policy 0, policy_version 108939 (0.0007) [2023-12-27 06:24:42,980][20347] Updated weights for policy 0, policy_version 108949 (0.0007) [2023-12-27 06:24:43,049][20347] Updated weights for policy 0, policy_version 108959 (0.0007) [2023-12-27 06:24:43,361][20381] Updated weights for policy 1, policy_version 108485 (0.0005) [2023-12-27 06:24:43,423][20381] Updated weights for policy 1, policy_version 108495 (0.0008) [2023-12-27 06:24:43,486][20381] Updated weights for policy 1, policy_version 108505 (0.0008) [2023-12-27 06:24:43,686][20347] Updated weights for policy 0, policy_version 108969 (0.0006) [2023-12-27 06:24:43,745][20347] Updated weights for policy 0, policy_version 108979 (0.0008) [2023-12-27 06:24:43,800][20347] Updated weights for policy 0, policy_version 108989 (0.0007) [2023-12-27 06:24:43,857][20347] Updated weights for policy 0, policy_version 108999 (0.0008) [2023-12-27 06:24:44,131][20381] Updated weights for policy 1, policy_version 108515 (0.0007) [2023-12-27 06:24:44,185][20381] Updated weights for policy 1, policy_version 108525 (0.0010) [2023-12-27 06:24:44,241][20381] Updated weights for policy 1, policy_version 108535 (0.0011) [2023-12-27 06:24:44,486][20347] Updated weights for policy 0, policy_version 109009 (0.0007) [2023-12-27 06:24:44,537][20347] Updated weights for policy 0, policy_version 109019 (0.0007) [2023-12-27 06:24:44,590][20347] Updated weights for policy 0, policy_version 109029 (0.0007) [2023-12-27 06:24:44,877][20381] Updated weights for policy 1, policy_version 108545 (0.0010) [2023-12-27 06:24:44,937][20381] Updated weights for policy 1, policy_version 108555 (0.0006) [2023-12-27 06:24:44,993][20381] Updated weights for policy 1, policy_version 108565 (0.0005) [2023-12-27 06:24:45,056][20381] Updated weights for policy 1, policy_version 108575 (0.0006) [2023-12-27 06:24:45,296][20347] Updated weights for policy 0, policy_version 109039 (0.0006) [2023-12-27 06:24:45,350][20347] Updated weights for policy 0, policy_version 109049 (0.0005) [2023-12-27 06:24:45,407][20347] Updated weights for policy 0, policy_version 109059 (0.0006) [2023-12-27 06:24:45,689][20381] Updated weights for policy 1, policy_version 108585 (0.0006) [2023-12-27 06:24:45,761][20381] Updated weights for policy 1, policy_version 108595 (0.0005) [2023-12-27 06:24:45,826][20381] Updated weights for policy 1, policy_version 108605 (0.0006) [2023-12-27 06:24:46,040][20347] Updated weights for policy 0, policy_version 109069 (0.0008) [2023-12-27 06:24:46,092][20347] Updated weights for policy 0, policy_version 109079 (0.0008) [2023-12-27 06:24:46,147][20347] Updated weights for policy 0, policy_version 109089 (0.0008) [2023-12-27 06:24:46,403][20381] Updated weights for policy 1, policy_version 108615 (0.0005) [2023-12-27 06:24:46,461][20381] Updated weights for policy 1, policy_version 108625 (0.0006) [2023-12-27 06:24:46,507][20381] Updated weights for policy 1, policy_version 108635 (0.0008) [2023-12-27 06:24:46,807][20347] Updated weights for policy 0, policy_version 109099 (0.0007) [2023-12-27 06:24:46,855][20347] Updated weights for policy 0, policy_version 109109 (0.0005) [2023-12-27 06:24:46,905][20347] Updated weights for policy 0, policy_version 109119 (0.0005) [2023-12-27 06:24:47,129][20381] Updated weights for policy 1, policy_version 108645 (0.0007) [2023-12-27 06:24:47,188][20381] Updated weights for policy 1, policy_version 108655 (0.0006) [2023-12-27 06:24:47,240][20381] Updated weights for policy 1, policy_version 108665 (0.0008) [2023-12-27 06:24:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 55779328. Throughput: 0: 10380.5, 1: 10471.2. Samples: 55786960. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:47,535][20347] Updated weights for policy 0, policy_version 109129 (0.0007) [2023-12-27 06:24:47,580][20347] Updated weights for policy 0, policy_version 109139 (0.0010) [2023-12-27 06:24:47,634][20347] Updated weights for policy 0, policy_version 109149 (0.0010) [2023-12-27 06:24:47,683][20347] Updated weights for policy 0, policy_version 109159 (0.0010) [2023-12-27 06:24:47,901][20381] Updated weights for policy 1, policy_version 108675 (0.0006) [2023-12-27 06:24:47,958][20381] Updated weights for policy 1, policy_version 108685 (0.0007) [2023-12-27 06:24:48,018][20381] Updated weights for policy 1, policy_version 108695 (0.0007) [2023-12-27 06:24:48,372][20347] Updated weights for policy 0, policy_version 109169 (0.0007) [2023-12-27 06:24:48,428][20347] Updated weights for policy 0, policy_version 109179 (0.0008) [2023-12-27 06:24:48,479][20347] Updated weights for policy 0, policy_version 109189 (0.0008) [2023-12-27 06:24:48,709][20381] Updated weights for policy 1, policy_version 108705 (0.0008) [2023-12-27 06:24:48,771][20381] Updated weights for policy 1, policy_version 108715 (0.0005) [2023-12-27 06:24:48,830][20381] Updated weights for policy 1, policy_version 108725 (0.0005) [2023-12-27 06:24:48,893][20381] Updated weights for policy 1, policy_version 108735 (0.0006) [2023-12-27 06:24:49,156][20347] Updated weights for policy 0, policy_version 109199 (0.0009) [2023-12-27 06:24:49,211][20347] Updated weights for policy 0, policy_version 109209 (0.0010) [2023-12-27 06:24:49,273][20347] Updated weights for policy 0, policy_version 109219 (0.0008) [2023-12-27 06:24:49,487][20381] Updated weights for policy 1, policy_version 108745 (0.0007) [2023-12-27 06:24:49,548][20381] Updated weights for policy 1, policy_version 108755 (0.0008) [2023-12-27 06:24:49,606][20381] Updated weights for policy 1, policy_version 108765 (0.0008) [2023-12-27 06:24:49,909][20347] Updated weights for policy 0, policy_version 109229 (0.0009) [2023-12-27 06:24:49,965][20347] Updated weights for policy 0, policy_version 109239 (0.0009) [2023-12-27 06:24:50,019][20347] Updated weights for policy 0, policy_version 109249 (0.0010) [2023-12-27 06:24:50,263][20381] Updated weights for policy 1, policy_version 108775 (0.0008) [2023-12-27 06:24:50,325][20381] Updated weights for policy 1, policy_version 108785 (0.0007) [2023-12-27 06:24:50,383][20381] Updated weights for policy 1, policy_version 108795 (0.0007) [2023-12-27 06:24:50,749][20347] Updated weights for policy 0, policy_version 109259 (0.0010) [2023-12-27 06:24:50,801][20347] Updated weights for policy 0, policy_version 109269 (0.0010) [2023-12-27 06:24:50,850][20347] Updated weights for policy 0, policy_version 109279 (0.0010) [2023-12-27 06:24:51,074][20381] Updated weights for policy 1, policy_version 108805 (0.0007) [2023-12-27 06:24:51,138][20381] Updated weights for policy 1, policy_version 108815 (0.0007) [2023-12-27 06:24:51,203][20381] Updated weights for policy 1, policy_version 108825 (0.0008) [2023-12-27 06:24:51,557][20347] Updated weights for policy 0, policy_version 109289 (0.0010) [2023-12-27 06:24:51,619][20347] Updated weights for policy 0, policy_version 109299 (0.0006) [2023-12-27 06:24:51,682][20347] Updated weights for policy 0, policy_version 109309 (0.0007) [2023-12-27 06:24:51,746][20347] Updated weights for policy 0, policy_version 109319 (0.0006) [2023-12-27 06:24:51,911][20381] Updated weights for policy 1, policy_version 108835 (0.0008) [2023-12-27 06:24:51,976][20381] Updated weights for policy 1, policy_version 108845 (0.0010) [2023-12-27 06:24:52,042][20381] Updated weights for policy 1, policy_version 108855 (0.0007) [2023-12-27 06:24:52,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 55877632. Throughput: 0: 10333.3, 1: 10432.4. Samples: 55849764. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000108864_27877376.pth... [2023-12-27 06:24:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000107648_27566080.pth [2023-12-27 06:24:52,415][20347] Updated weights for policy 0, policy_version 109329 (0.0006) [2023-12-27 06:24:52,477][20347] Updated weights for policy 0, policy_version 109339 (0.0010) [2023-12-27 06:24:52,532][20347] Updated weights for policy 0, policy_version 109349 (0.0010) [2023-12-27 06:24:52,548][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000109352_28008448.pth... [2023-12-27 06:24:52,551][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000108136_27697152.pth [2023-12-27 06:24:52,687][20381] Updated weights for policy 1, policy_version 108865 (0.0010) [2023-12-27 06:24:52,749][20381] Updated weights for policy 1, policy_version 108875 (0.0007) [2023-12-27 06:24:52,813][20381] Updated weights for policy 1, policy_version 108885 (0.0007) [2023-12-27 06:24:52,874][20381] Updated weights for policy 1, policy_version 108895 (0.0008) [2023-12-27 06:24:53,191][20347] Updated weights for policy 0, policy_version 109359 (0.0010) [2023-12-27 06:24:53,246][20347] Updated weights for policy 0, policy_version 109369 (0.0010) [2023-12-27 06:24:53,301][20347] Updated weights for policy 0, policy_version 109379 (0.0010) [2023-12-27 06:24:53,533][20381] Updated weights for policy 1, policy_version 108905 (0.0007) [2023-12-27 06:24:53,600][20381] Updated weights for policy 1, policy_version 108915 (0.0007) [2023-12-27 06:24:53,645][20381] Updated weights for policy 1, policy_version 108925 (0.0006) [2023-12-27 06:24:53,992][20347] Updated weights for policy 0, policy_version 109389 (0.0010) [2023-12-27 06:24:54,047][20347] Updated weights for policy 0, policy_version 109399 (0.0010) [2023-12-27 06:24:54,110][20347] Updated weights for policy 0, policy_version 109409 (0.0010) [2023-12-27 06:24:54,225][20381] Updated weights for policy 1, policy_version 108935 (0.0005) [2023-12-27 06:24:54,283][20381] Updated weights for policy 1, policy_version 108945 (0.0006) [2023-12-27 06:24:54,338][20381] Updated weights for policy 1, policy_version 108955 (0.0007) [2023-12-27 06:24:54,839][20347] Updated weights for policy 0, policy_version 109419 (0.0008) [2023-12-27 06:24:54,898][20347] Updated weights for policy 0, policy_version 109429 (0.0010) [2023-12-27 06:24:54,949][20347] Updated weights for policy 0, policy_version 109439 (0.0010) [2023-12-27 06:24:55,015][20381] Updated weights for policy 1, policy_version 108965 (0.0006) [2023-12-27 06:24:55,070][20381] Updated weights for policy 1, policy_version 108975 (0.0008) [2023-12-27 06:24:55,124][20381] Updated weights for policy 1, policy_version 108985 (0.0007) [2023-12-27 06:24:55,624][20347] Updated weights for policy 0, policy_version 109449 (0.0011) [2023-12-27 06:24:55,685][20347] Updated weights for policy 0, policy_version 109459 (0.0010) [2023-12-27 06:24:55,743][20347] Updated weights for policy 0, policy_version 109469 (0.0010) [2023-12-27 06:24:55,805][20347] Updated weights for policy 0, policy_version 109479 (0.0010) [2023-12-27 06:24:55,810][20381] Updated weights for policy 1, policy_version 108995 (0.0007) [2023-12-27 06:24:55,867][20381] Updated weights for policy 1, policy_version 109005 (0.0006) [2023-12-27 06:24:55,920][20381] Updated weights for policy 1, policy_version 109015 (0.0006) [2023-12-27 06:24:56,488][20347] Updated weights for policy 0, policy_version 109489 (0.0008) [2023-12-27 06:24:56,547][20347] Updated weights for policy 0, policy_version 109499 (0.0007) [2023-12-27 06:24:56,599][20347] Updated weights for policy 0, policy_version 109509 (0.0008) [2023-12-27 06:24:56,605][20381] Updated weights for policy 1, policy_version 109025 (0.0008) [2023-12-27 06:24:56,657][20381] Updated weights for policy 1, policy_version 109035 (0.0007) [2023-12-27 06:24:56,709][20381] Updated weights for policy 1, policy_version 109045 (0.0008) [2023-12-27 06:24:56,767][20381] Updated weights for policy 1, policy_version 109055 (0.0008) [2023-12-27 06:24:57,241][20347] Updated weights for policy 0, policy_version 109519 (0.0008) [2023-12-27 06:24:57,298][20347] Updated weights for policy 0, policy_version 109529 (0.0008) [2023-12-27 06:24:57,358][20347] Updated weights for policy 0, policy_version 109539 (0.0008) [2023-12-27 06:24:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 55975936. Throughput: 0: 10256.1, 1: 10455.8. Samples: 55971656. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-12-27 06:24:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:24:57,480][20381] Updated weights for policy 1, policy_version 109065 (0.0008) [2023-12-27 06:24:57,527][20381] Updated weights for policy 1, policy_version 109075 (0.0007) [2023-12-27 06:24:57,578][20381] Updated weights for policy 1, policy_version 109085 (0.0008) [2023-12-27 06:24:58,027][20347] Updated weights for policy 0, policy_version 109549 (0.0007) [2023-12-27 06:24:58,076][20347] Updated weights for policy 0, policy_version 109559 (0.0008) [2023-12-27 06:24:58,126][20347] Updated weights for policy 0, policy_version 109569 (0.0007) [2023-12-27 06:24:58,296][20381] Updated weights for policy 1, policy_version 109095 (0.0007) [2023-12-27 06:24:58,353][20381] Updated weights for policy 1, policy_version 109105 (0.0007) [2023-12-27 06:24:58,414][20381] Updated weights for policy 1, policy_version 109115 (0.0007) [2023-12-27 06:24:58,923][20347] Updated weights for policy 0, policy_version 109579 (0.0008) [2023-12-27 06:24:58,982][20347] Updated weights for policy 0, policy_version 109589 (0.0007) [2023-12-27 06:24:59,038][20347] Updated weights for policy 0, policy_version 109599 (0.0008) [2023-12-27 06:24:59,195][20381] Updated weights for policy 1, policy_version 109125 (0.0009) [2023-12-27 06:24:59,261][20381] Updated weights for policy 1, policy_version 109135 (0.0009) [2023-12-27 06:24:59,310][20381] Updated weights for policy 1, policy_version 109145 (0.0012) [2023-12-27 06:24:59,699][20347] Updated weights for policy 0, policy_version 109609 (0.0008) [2023-12-27 06:24:59,745][20347] Updated weights for policy 0, policy_version 109619 (0.0008) [2023-12-27 06:24:59,794][20347] Updated weights for policy 0, policy_version 109629 (0.0007) [2023-12-27 06:24:59,848][20347] Updated weights for policy 0, policy_version 109639 (0.0008) [2023-12-27 06:25:00,053][20381] Updated weights for policy 1, policy_version 109155 (0.0008) [2023-12-27 06:25:00,106][20381] Updated weights for policy 1, policy_version 109165 (0.0008) [2023-12-27 06:25:00,160][20381] Updated weights for policy 1, policy_version 109175 (0.0008) [2023-12-27 06:25:00,507][20347] Updated weights for policy 0, policy_version 109649 (0.0006) [2023-12-27 06:25:00,563][20347] Updated weights for policy 0, policy_version 109659 (0.0009) [2023-12-27 06:25:00,620][20347] Updated weights for policy 0, policy_version 109669 (0.0007) [2023-12-27 06:25:00,778][20381] Updated weights for policy 1, policy_version 109185 (0.0008) [2023-12-27 06:25:00,827][20381] Updated weights for policy 1, policy_version 109195 (0.0005) [2023-12-27 06:25:00,874][20381] Updated weights for policy 1, policy_version 109205 (0.0005) [2023-12-27 06:25:00,923][20381] Updated weights for policy 1, policy_version 109215 (0.0006) [2023-12-27 06:25:01,238][20347] Updated weights for policy 0, policy_version 109679 (0.0006) [2023-12-27 06:25:01,296][20347] Updated weights for policy 0, policy_version 109689 (0.0007) [2023-12-27 06:25:01,361][20347] Updated weights for policy 0, policy_version 109699 (0.0007) [2023-12-27 06:25:01,599][20381] Updated weights for policy 1, policy_version 109225 (0.0007) [2023-12-27 06:25:01,659][20381] Updated weights for policy 1, policy_version 109235 (0.0008) [2023-12-27 06:25:01,707][20381] Updated weights for policy 1, policy_version 109245 (0.0008) [2023-12-27 06:25:02,052][20347] Updated weights for policy 0, policy_version 109709 (0.0009) [2023-12-27 06:25:02,114][20347] Updated weights for policy 0, policy_version 109719 (0.0011) [2023-12-27 06:25:02,172][20347] Updated weights for policy 0, policy_version 109729 (0.0010) [2023-12-27 06:25:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 56082432. Throughput: 0: 10253.5, 1: 10373.3. Samples: 56093844. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:02,369][20381] Updated weights for policy 1, policy_version 109255 (0.0010) [2023-12-27 06:25:02,423][20381] Updated weights for policy 1, policy_version 109265 (0.0006) [2023-12-27 06:25:02,476][20381] Updated weights for policy 1, policy_version 109275 (0.0005) [2023-12-27 06:25:02,850][20347] Updated weights for policy 0, policy_version 109739 (0.0010) [2023-12-27 06:25:02,906][20347] Updated weights for policy 0, policy_version 109749 (0.0010) [2023-12-27 06:25:02,956][20347] Updated weights for policy 0, policy_version 109759 (0.0010) [2023-12-27 06:25:03,088][20381] Updated weights for policy 1, policy_version 109285 (0.0005) [2023-12-27 06:25:03,143][20381] Updated weights for policy 1, policy_version 109295 (0.0006) [2023-12-27 06:25:03,202][20381] Updated weights for policy 1, policy_version 109305 (0.0005) [2023-12-27 06:25:03,549][20347] Updated weights for policy 0, policy_version 109769 (0.0010) [2023-12-27 06:25:03,600][20347] Updated weights for policy 0, policy_version 109779 (0.0007) [2023-12-27 06:25:03,644][20347] Updated weights for policy 0, policy_version 109789 (0.0010) [2023-12-27 06:25:03,688][20347] Updated weights for policy 0, policy_version 109799 (0.0010) [2023-12-27 06:25:03,774][20381] Updated weights for policy 1, policy_version 109315 (0.0007) [2023-12-27 06:25:03,823][20381] Updated weights for policy 1, policy_version 109325 (0.0010) [2023-12-27 06:25:03,880][20381] Updated weights for policy 1, policy_version 109335 (0.0011) [2023-12-27 06:25:04,306][20347] Updated weights for policy 0, policy_version 109809 (0.0010) [2023-12-27 06:25:04,362][20347] Updated weights for policy 0, policy_version 109819 (0.0010) [2023-12-27 06:25:04,420][20347] Updated weights for policy 0, policy_version 109829 (0.0010) [2023-12-27 06:25:04,588][20381] Updated weights for policy 1, policy_version 109345 (0.0011) [2023-12-27 06:25:04,644][20381] Updated weights for policy 1, policy_version 109355 (0.0011) [2023-12-27 06:25:04,706][20381] Updated weights for policy 1, policy_version 109365 (0.0011) [2023-12-27 06:25:04,757][20381] Updated weights for policy 1, policy_version 109375 (0.0011) [2023-12-27 06:25:05,107][20347] Updated weights for policy 0, policy_version 109839 (0.0010) [2023-12-27 06:25:05,168][20347] Updated weights for policy 0, policy_version 109849 (0.0010) [2023-12-27 06:25:05,225][20347] Updated weights for policy 0, policy_version 109859 (0.0010) [2023-12-27 06:25:05,399][20381] Updated weights for policy 1, policy_version 109385 (0.0007) [2023-12-27 06:25:05,449][20381] Updated weights for policy 1, policy_version 109395 (0.0008) [2023-12-27 06:25:05,501][20381] Updated weights for policy 1, policy_version 109405 (0.0007) [2023-12-27 06:25:05,799][20347] Updated weights for policy 0, policy_version 109869 (0.0008) [2023-12-27 06:25:05,851][20347] Updated weights for policy 0, policy_version 109879 (0.0005) [2023-12-27 06:25:05,910][20347] Updated weights for policy 0, policy_version 109889 (0.0011) [2023-12-27 06:25:06,226][20381] Updated weights for policy 1, policy_version 109415 (0.0007) [2023-12-27 06:25:06,273][20381] Updated weights for policy 1, policy_version 109425 (0.0006) [2023-12-27 06:25:06,323][20381] Updated weights for policy 1, policy_version 109435 (0.0006) [2023-12-27 06:25:06,569][20347] Updated weights for policy 0, policy_version 109899 (0.0009) [2023-12-27 06:25:06,633][20347] Updated weights for policy 0, policy_version 109909 (0.0006) [2023-12-27 06:25:06,691][20347] Updated weights for policy 0, policy_version 109919 (0.0008) [2023-12-27 06:25:07,030][20381] Updated weights for policy 1, policy_version 109445 (0.0008) [2023-12-27 06:25:07,088][20381] Updated weights for policy 1, policy_version 109455 (0.0008) [2023-12-27 06:25:07,151][20381] Updated weights for policy 1, policy_version 109465 (0.0008) [2023-12-27 06:25:07,306][20347] Updated weights for policy 0, policy_version 109929 (0.0007) [2023-12-27 06:25:07,362][20347] Updated weights for policy 0, policy_version 109939 (0.0008) [2023-12-27 06:25:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 56188928. Throughput: 0: 10288.5, 1: 10395.3. Samples: 56158628. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000109472_28033024.pth... [2023-12-27 06:25:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000108256_27721728.pth [2023-12-27 06:25:07,422][20347] Updated weights for policy 0, policy_version 109949 (0.0008) [2023-12-27 06:25:07,478][20347] Updated weights for policy 0, policy_version 109959 (0.0007) [2023-12-27 06:25:07,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000109960_28164096.pth... [2023-12-27 06:25:07,488][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000108744_27852800.pth [2023-12-27 06:25:07,776][20381] Updated weights for policy 1, policy_version 109475 (0.0007) [2023-12-27 06:25:07,835][20381] Updated weights for policy 1, policy_version 109485 (0.0007) [2023-12-27 06:25:07,880][20381] Updated weights for policy 1, policy_version 109495 (0.0009) [2023-12-27 06:25:08,144][20347] Updated weights for policy 0, policy_version 109969 (0.0006) [2023-12-27 06:25:08,201][20347] Updated weights for policy 0, policy_version 109979 (0.0006) [2023-12-27 06:25:08,258][20347] Updated weights for policy 0, policy_version 109989 (0.0005) [2023-12-27 06:25:08,634][20381] Updated weights for policy 1, policy_version 109505 (0.0008) [2023-12-27 06:25:08,703][20381] Updated weights for policy 1, policy_version 109515 (0.0008) [2023-12-27 06:25:08,767][20381] Updated weights for policy 1, policy_version 109525 (0.0008) [2023-12-27 06:25:08,827][20381] Updated weights for policy 1, policy_version 109535 (0.0008) [2023-12-27 06:25:08,905][20347] Updated weights for policy 0, policy_version 109999 (0.0007) [2023-12-27 06:25:08,955][20347] Updated weights for policy 0, policy_version 110009 (0.0008) [2023-12-27 06:25:09,014][20347] Updated weights for policy 0, policy_version 110019 (0.0008) [2023-12-27 06:25:09,546][20381] Updated weights for policy 1, policy_version 109545 (0.0007) [2023-12-27 06:25:09,605][20381] Updated weights for policy 1, policy_version 109555 (0.0008) [2023-12-27 06:25:09,665][20381] Updated weights for policy 1, policy_version 109565 (0.0007) [2023-12-27 06:25:09,700][20347] Updated weights for policy 0, policy_version 110029 (0.0007) [2023-12-27 06:25:09,758][20347] Updated weights for policy 0, policy_version 110039 (0.0008) [2023-12-27 06:25:09,822][20347] Updated weights for policy 0, policy_version 110049 (0.0007) [2023-12-27 06:25:10,348][20381] Updated weights for policy 1, policy_version 109575 (0.0007) [2023-12-27 06:25:10,405][20381] Updated weights for policy 1, policy_version 109585 (0.0008) [2023-12-27 06:25:10,460][20381] Updated weights for policy 1, policy_version 109595 (0.0007) [2023-12-27 06:25:10,521][20347] Updated weights for policy 0, policy_version 110059 (0.0009) [2023-12-27 06:25:10,585][20347] Updated weights for policy 0, policy_version 110069 (0.0010) [2023-12-27 06:25:10,646][20347] Updated weights for policy 0, policy_version 110079 (0.0010) [2023-12-27 06:25:11,183][20381] Updated weights for policy 1, policy_version 109605 (0.0008) [2023-12-27 06:25:11,245][20381] Updated weights for policy 1, policy_version 109615 (0.0009) [2023-12-27 06:25:11,310][20347] Updated weights for policy 0, policy_version 110089 (0.0010) [2023-12-27 06:25:11,312][20381] Updated weights for policy 1, policy_version 109625 (0.0008) [2023-12-27 06:25:11,384][20347] Updated weights for policy 0, policy_version 110099 (0.0007) [2023-12-27 06:25:11,438][20347] Updated weights for policy 0, policy_version 110109 (0.0009) [2023-12-27 06:25:11,497][20347] Updated weights for policy 0, policy_version 110119 (0.0008) [2023-12-27 06:25:11,988][20381] Updated weights for policy 1, policy_version 109635 (0.0007) [2023-12-27 06:25:12,042][20381] Updated weights for policy 1, policy_version 109645 (0.0006) [2023-12-27 06:25:12,101][20381] Updated weights for policy 1, policy_version 109655 (0.0005) [2023-12-27 06:25:12,239][20347] Updated weights for policy 0, policy_version 110129 (0.0007) [2023-12-27 06:25:12,306][20347] Updated weights for policy 0, policy_version 110139 (0.0008) [2023-12-27 06:25:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 56287232. Throughput: 0: 10362.2, 1: 10396.5. Samples: 56280772. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:12,371][20347] Updated weights for policy 0, policy_version 110149 (0.0007) [2023-12-27 06:25:12,708][20381] Updated weights for policy 1, policy_version 109665 (0.0006) [2023-12-27 06:25:12,765][20381] Updated weights for policy 1, policy_version 109675 (0.0008) [2023-12-27 06:25:12,821][20381] Updated weights for policy 1, policy_version 109685 (0.0009) [2023-12-27 06:25:12,873][20381] Updated weights for policy 1, policy_version 109695 (0.0008) [2023-12-27 06:25:13,043][20347] Updated weights for policy 0, policy_version 110159 (0.0006) [2023-12-27 06:25:13,105][20347] Updated weights for policy 0, policy_version 110169 (0.0005) [2023-12-27 06:25:13,156][20347] Updated weights for policy 0, policy_version 110179 (0.0006) [2023-12-27 06:25:13,557][20381] Updated weights for policy 1, policy_version 109705 (0.0008) [2023-12-27 06:25:13,612][20381] Updated weights for policy 1, policy_version 109715 (0.0007) [2023-12-27 06:25:13,657][20381] Updated weights for policy 1, policy_version 109725 (0.0007) [2023-12-27 06:25:13,789][20347] Updated weights for policy 0, policy_version 110189 (0.0006) [2023-12-27 06:25:13,849][20347] Updated weights for policy 0, policy_version 110199 (0.0005) [2023-12-27 06:25:13,899][20347] Updated weights for policy 0, policy_version 110209 (0.0005) [2023-12-27 06:25:14,346][20381] Updated weights for policy 1, policy_version 109735 (0.0008) [2023-12-27 06:25:14,411][20381] Updated weights for policy 1, policy_version 109745 (0.0009) [2023-12-27 06:25:14,455][20347] Updated weights for policy 0, policy_version 110219 (0.0005) [2023-12-27 06:25:14,462][20381] Updated weights for policy 1, policy_version 109755 (0.0008) [2023-12-27 06:25:14,512][20347] Updated weights for policy 0, policy_version 110229 (0.0006) [2023-12-27 06:25:14,557][20347] Updated weights for policy 0, policy_version 110239 (0.0007) [2023-12-27 06:25:15,125][20381] Updated weights for policy 1, policy_version 109765 (0.0007) [2023-12-27 06:25:15,185][20381] Updated weights for policy 1, policy_version 109775 (0.0008) [2023-12-27 06:25:15,245][20381] Updated weights for policy 1, policy_version 109785 (0.0008) [2023-12-27 06:25:15,278][20347] Updated weights for policy 0, policy_version 110249 (0.0007) [2023-12-27 06:25:15,332][20347] Updated weights for policy 0, policy_version 110259 (0.0007) [2023-12-27 06:25:15,389][20347] Updated weights for policy 0, policy_version 110269 (0.0007) [2023-12-27 06:25:15,445][20347] Updated weights for policy 0, policy_version 110279 (0.0007) [2023-12-27 06:25:15,895][20381] Updated weights for policy 1, policy_version 109795 (0.0010) [2023-12-27 06:25:15,950][20381] Updated weights for policy 1, policy_version 109805 (0.0010) [2023-12-27 06:25:16,004][20381] Updated weights for policy 1, policy_version 109815 (0.0010) [2023-12-27 06:25:16,104][20347] Updated weights for policy 0, policy_version 110289 (0.0007) [2023-12-27 06:25:16,152][20347] Updated weights for policy 0, policy_version 110299 (0.0005) [2023-12-27 06:25:16,204][20347] Updated weights for policy 0, policy_version 110309 (0.0007) [2023-12-27 06:25:16,692][20381] Updated weights for policy 1, policy_version 109825 (0.0010) [2023-12-27 06:25:16,753][20381] Updated weights for policy 1, policy_version 109835 (0.0009) [2023-12-27 06:25:16,808][20381] Updated weights for policy 1, policy_version 109845 (0.0011) [2023-12-27 06:25:16,874][20381] Updated weights for policy 1, policy_version 109855 (0.0005) [2023-12-27 06:25:16,881][20347] Updated weights for policy 0, policy_version 110319 (0.0006) [2023-12-27 06:25:16,933][20347] Updated weights for policy 0, policy_version 110329 (0.0007) [2023-12-27 06:25:16,981][20347] Updated weights for policy 0, policy_version 110339 (0.0007) [2023-12-27 06:25:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 56393728. Throughput: 0: 10286.8, 1: 10353.3. Samples: 56405196. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:17,515][20381] Updated weights for policy 1, policy_version 109865 (0.0010) [2023-12-27 06:25:17,570][20381] Updated weights for policy 1, policy_version 109875 (0.0010) [2023-12-27 06:25:17,631][20381] Updated weights for policy 1, policy_version 109885 (0.0010) [2023-12-27 06:25:17,647][20347] Updated weights for policy 0, policy_version 110349 (0.0009) [2023-12-27 06:25:17,703][20347] Updated weights for policy 0, policy_version 110359 (0.0007) [2023-12-27 06:25:17,754][20347] Updated weights for policy 0, policy_version 110369 (0.0005) [2023-12-27 06:25:18,227][20381] Updated weights for policy 1, policy_version 109895 (0.0011) [2023-12-27 06:25:18,282][20381] Updated weights for policy 1, policy_version 109905 (0.0010) [2023-12-27 06:25:18,342][20381] Updated weights for policy 1, policy_version 109915 (0.0011) [2023-12-27 06:25:18,395][20347] Updated weights for policy 0, policy_version 110379 (0.0006) [2023-12-27 06:25:18,460][20347] Updated weights for policy 0, policy_version 110389 (0.0010) [2023-12-27 06:25:18,528][20347] Updated weights for policy 0, policy_version 110399 (0.0009) [2023-12-27 06:25:18,995][20381] Updated weights for policy 1, policy_version 109925 (0.0008) [2023-12-27 06:25:19,059][20381] Updated weights for policy 1, policy_version 109935 (0.0011) [2023-12-27 06:25:19,119][20381] Updated weights for policy 1, policy_version 109945 (0.0008) [2023-12-27 06:25:19,127][20347] Updated weights for policy 0, policy_version 110409 (0.0011) [2023-12-27 06:25:19,181][20347] Updated weights for policy 0, policy_version 110419 (0.0010) [2023-12-27 06:25:19,233][20347] Updated weights for policy 0, policy_version 110429 (0.0010) [2023-12-27 06:25:19,288][20347] Updated weights for policy 0, policy_version 110439 (0.0009) [2023-12-27 06:25:19,729][20381] Updated weights for policy 1, policy_version 109955 (0.0005) [2023-12-27 06:25:19,786][20381] Updated weights for policy 1, policy_version 109965 (0.0005) [2023-12-27 06:25:19,846][20381] Updated weights for policy 1, policy_version 109975 (0.0006) [2023-12-27 06:25:20,077][20347] Updated weights for policy 0, policy_version 110449 (0.0006) [2023-12-27 06:25:20,134][20347] Updated weights for policy 0, policy_version 110459 (0.0005) [2023-12-27 06:25:20,196][20347] Updated weights for policy 0, policy_version 110469 (0.0006) [2023-12-27 06:25:20,519][20381] Updated weights for policy 1, policy_version 109985 (0.0007) [2023-12-27 06:25:20,576][20381] Updated weights for policy 1, policy_version 109995 (0.0006) [2023-12-27 06:25:20,636][20381] Updated weights for policy 1, policy_version 110005 (0.0008) [2023-12-27 06:25:20,697][20381] Updated weights for policy 1, policy_version 110015 (0.0008) [2023-12-27 06:25:20,776][20347] Updated weights for policy 0, policy_version 110479 (0.0007) [2023-12-27 06:25:20,842][20347] Updated weights for policy 0, policy_version 110489 (0.0007) [2023-12-27 06:25:20,900][20347] Updated weights for policy 0, policy_version 110499 (0.0007) [2023-12-27 06:25:21,358][20381] Updated weights for policy 1, policy_version 110025 (0.0008) [2023-12-27 06:25:21,417][20381] Updated weights for policy 1, policy_version 110035 (0.0008) [2023-12-27 06:25:21,469][20381] Updated weights for policy 1, policy_version 110045 (0.0010) [2023-12-27 06:25:21,595][20347] Updated weights for policy 0, policy_version 110509 (0.0008) [2023-12-27 06:25:21,668][20347] Updated weights for policy 0, policy_version 110519 (0.0008) [2023-12-27 06:25:21,734][20347] Updated weights for policy 0, policy_version 110529 (0.0008) [2023-12-27 06:25:22,302][20381] Updated weights for policy 1, policy_version 110055 (0.0008) [2023-12-27 06:25:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 56492032. Throughput: 0: 10295.8, 1: 10334.1. Samples: 56468812. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:22,366][20381] Updated weights for policy 1, policy_version 110065 (0.0007) [2023-12-27 06:25:22,387][20347] Updated weights for policy 0, policy_version 110539 (0.0007) [2023-12-27 06:25:22,420][20381] Updated weights for policy 1, policy_version 110075 (0.0006) [2023-12-27 06:25:22,443][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000110080_28188672.pth... [2023-12-27 06:25:22,446][20347] Updated weights for policy 0, policy_version 110549 (0.0006) [2023-12-27 06:25:22,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000108864_27877376.pth [2023-12-27 06:25:22,502][20347] Updated weights for policy 0, policy_version 110559 (0.0007) [2023-12-27 06:25:22,547][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000110568_28319744.pth... [2023-12-27 06:25:22,551][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000109352_28008448.pth [2023-12-27 06:25:23,152][20381] Updated weights for policy 1, policy_version 110085 (0.0007) [2023-12-27 06:25:23,184][20347] Updated weights for policy 0, policy_version 110569 (0.0008) [2023-12-27 06:25:23,202][20381] Updated weights for policy 1, policy_version 110095 (0.0006) [2023-12-27 06:25:23,241][20347] Updated weights for policy 0, policy_version 110579 (0.0007) [2023-12-27 06:25:23,255][20381] Updated weights for policy 1, policy_version 110105 (0.0006) [2023-12-27 06:25:23,299][20347] Updated weights for policy 0, policy_version 110589 (0.0006) [2023-12-27 06:25:23,358][20347] Updated weights for policy 0, policy_version 110599 (0.0007) [2023-12-27 06:25:23,956][20381] Updated weights for policy 1, policy_version 110115 (0.0006) [2023-12-27 06:25:24,015][20381] Updated weights for policy 1, policy_version 110125 (0.0006) [2023-12-27 06:25:24,036][20347] Updated weights for policy 0, policy_version 110609 (0.0010) [2023-12-27 06:25:24,071][20381] Updated weights for policy 1, policy_version 110135 (0.0005) [2023-12-27 06:25:24,092][20347] Updated weights for policy 0, policy_version 110619 (0.0010) [2023-12-27 06:25:24,152][20347] Updated weights for policy 0, policy_version 110629 (0.0008) [2023-12-27 06:25:24,747][20381] Updated weights for policy 1, policy_version 110145 (0.0007) [2023-12-27 06:25:24,793][20347] Updated weights for policy 0, policy_version 110639 (0.0011) [2023-12-27 06:25:24,796][20381] Updated weights for policy 1, policy_version 110155 (0.0010) [2023-12-27 06:25:24,849][20381] Updated weights for policy 1, policy_version 110165 (0.0010) [2023-12-27 06:25:24,852][20347] Updated weights for policy 0, policy_version 110649 (0.0010) [2023-12-27 06:25:24,901][20381] Updated weights for policy 1, policy_version 110175 (0.0010) [2023-12-27 06:25:24,912][20347] Updated weights for policy 0, policy_version 110659 (0.0010) [2023-12-27 06:25:25,525][20381] Updated weights for policy 1, policy_version 110185 (0.0008) [2023-12-27 06:25:25,582][20347] Updated weights for policy 0, policy_version 110669 (0.0011) [2023-12-27 06:25:25,596][20381] Updated weights for policy 1, policy_version 110195 (0.0009) [2023-12-27 06:25:25,634][20347] Updated weights for policy 0, policy_version 110679 (0.0010) [2023-12-27 06:25:25,661][20381] Updated weights for policy 1, policy_version 110205 (0.0007) [2023-12-27 06:25:25,681][20347] Updated weights for policy 0, policy_version 110689 (0.0007) [2023-12-27 06:25:26,288][20381] Updated weights for policy 1, policy_version 110215 (0.0009) [2023-12-27 06:25:26,328][20347] Updated weights for policy 0, policy_version 110699 (0.0005) [2023-12-27 06:25:26,349][20381] Updated weights for policy 1, policy_version 110225 (0.0008) [2023-12-27 06:25:26,376][20347] Updated weights for policy 0, policy_version 110709 (0.0005) [2023-12-27 06:25:26,400][20381] Updated weights for policy 1, policy_version 110235 (0.0006) [2023-12-27 06:25:26,423][20347] Updated weights for policy 0, policy_version 110719 (0.0010) [2023-12-27 06:25:26,967][20381] Updated weights for policy 1, policy_version 110245 (0.0006) [2023-12-27 06:25:27,014][20381] Updated weights for policy 1, policy_version 110255 (0.0005) [2023-12-27 06:25:27,055][20347] Updated weights for policy 0, policy_version 110729 (0.0010) [2023-12-27 06:25:27,068][20381] Updated weights for policy 1, policy_version 110265 (0.0009) [2023-12-27 06:25:27,112][20347] Updated weights for policy 0, policy_version 110739 (0.0010) [2023-12-27 06:25:27,165][20347] Updated weights for policy 0, policy_version 110749 (0.0010) [2023-12-27 06:25:27,224][20347] Updated weights for policy 0, policy_version 110759 (0.0010) [2023-12-27 06:25:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 56606720. Throughput: 0: 10348.0, 1: 10327.3. Samples: 56590664. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:27,705][20381] Updated weights for policy 1, policy_version 110275 (0.0010) [2023-12-27 06:25:27,766][20381] Updated weights for policy 1, policy_version 110285 (0.0011) [2023-12-27 06:25:27,815][20381] Updated weights for policy 1, policy_version 110295 (0.0010) [2023-12-27 06:25:27,855][20347] Updated weights for policy 0, policy_version 110769 (0.0010) [2023-12-27 06:25:27,913][20347] Updated weights for policy 0, policy_version 110779 (0.0010) [2023-12-27 06:25:27,970][20347] Updated weights for policy 0, policy_version 110789 (0.0010) [2023-12-27 06:25:28,476][20381] Updated weights for policy 1, policy_version 110305 (0.0008) [2023-12-27 06:25:28,542][20381] Updated weights for policy 1, policy_version 110315 (0.0011) [2023-12-27 06:25:28,610][20381] Updated weights for policy 1, policy_version 110325 (0.0011) [2023-12-27 06:25:28,652][20347] Updated weights for policy 0, policy_version 110799 (0.0008) [2023-12-27 06:25:28,675][20381] Updated weights for policy 1, policy_version 110335 (0.0011) [2023-12-27 06:25:28,715][20347] Updated weights for policy 0, policy_version 110809 (0.0008) [2023-12-27 06:25:28,774][20347] Updated weights for policy 0, policy_version 110819 (0.0010) [2023-12-27 06:25:29,309][20381] Updated weights for policy 1, policy_version 110345 (0.0007) [2023-12-27 06:25:29,375][20381] Updated weights for policy 1, policy_version 110355 (0.0008) [2023-12-27 06:25:29,389][20347] Updated weights for policy 0, policy_version 110829 (0.0008) [2023-12-27 06:25:29,425][20381] Updated weights for policy 1, policy_version 110365 (0.0006) [2023-12-27 06:25:29,451][20347] Updated weights for policy 0, policy_version 110839 (0.0010) [2023-12-27 06:25:29,511][20347] Updated weights for policy 0, policy_version 110849 (0.0010) [2023-12-27 06:25:30,112][20381] Updated weights for policy 1, policy_version 110375 (0.0008) [2023-12-27 06:25:30,166][20381] Updated weights for policy 1, policy_version 110385 (0.0007) [2023-12-27 06:25:30,209][20347] Updated weights for policy 0, policy_version 110859 (0.0010) [2023-12-27 06:25:30,221][20381] Updated weights for policy 1, policy_version 110395 (0.0008) [2023-12-27 06:25:30,261][20347] Updated weights for policy 0, policy_version 110869 (0.0010) [2023-12-27 06:25:30,313][20347] Updated weights for policy 0, policy_version 110879 (0.0010) [2023-12-27 06:25:30,920][20347] Updated weights for policy 0, policy_version 110889 (0.0010) [2023-12-27 06:25:30,930][20381] Updated weights for policy 1, policy_version 110405 (0.0007) [2023-12-27 06:25:30,980][20347] Updated weights for policy 0, policy_version 110899 (0.0005) [2023-12-27 06:25:30,983][20381] Updated weights for policy 1, policy_version 110415 (0.0008) [2023-12-27 06:25:31,037][20347] Updated weights for policy 0, policy_version 110909 (0.0007) [2023-12-27 06:25:31,041][20381] Updated weights for policy 1, policy_version 110425 (0.0008) [2023-12-27 06:25:31,097][20347] Updated weights for policy 0, policy_version 110919 (0.0010) [2023-12-27 06:25:31,757][20381] Updated weights for policy 1, policy_version 110435 (0.0006) [2023-12-27 06:25:31,787][20347] Updated weights for policy 0, policy_version 110929 (0.0010) [2023-12-27 06:25:31,810][20381] Updated weights for policy 1, policy_version 110445 (0.0006) [2023-12-27 06:25:31,844][20347] Updated weights for policy 0, policy_version 110939 (0.0010) [2023-12-27 06:25:31,863][20381] Updated weights for policy 1, policy_version 110455 (0.0008) [2023-12-27 06:25:31,902][20347] Updated weights for policy 0, policy_version 110949 (0.0010) [2023-12-27 06:25:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 56705024. Throughput: 0: 10396.7, 1: 10259.7. Samples: 56716500. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:32,574][20347] Updated weights for policy 0, policy_version 110959 (0.0010) [2023-12-27 06:25:32,588][20381] Updated weights for policy 1, policy_version 110465 (0.0006) [2023-12-27 06:25:32,626][20347] Updated weights for policy 0, policy_version 110969 (0.0010) [2023-12-27 06:25:32,643][20381] Updated weights for policy 1, policy_version 110475 (0.0005) [2023-12-27 06:25:32,677][20347] Updated weights for policy 0, policy_version 110979 (0.0010) [2023-12-27 06:25:32,697][20381] Updated weights for policy 1, policy_version 110485 (0.0005) [2023-12-27 06:25:32,751][20381] Updated weights for policy 1, policy_version 110495 (0.0007) [2023-12-27 06:25:33,338][20381] Updated weights for policy 1, policy_version 110505 (0.0006) [2023-12-27 06:25:33,376][20347] Updated weights for policy 0, policy_version 110989 (0.0010) [2023-12-27 06:25:33,390][20381] Updated weights for policy 1, policy_version 110515 (0.0005) [2023-12-27 06:25:33,430][20347] Updated weights for policy 0, policy_version 110999 (0.0010) [2023-12-27 06:25:33,446][20381] Updated weights for policy 1, policy_version 110525 (0.0005) [2023-12-27 06:25:33,481][20347] Updated weights for policy 0, policy_version 111009 (0.0010) [2023-12-27 06:25:34,009][20381] Updated weights for policy 1, policy_version 110535 (0.0006) [2023-12-27 06:25:34,074][20381] Updated weights for policy 1, policy_version 110545 (0.0005) [2023-12-27 06:25:34,104][20347] Updated weights for policy 0, policy_version 111019 (0.0009) [2023-12-27 06:25:34,131][20381] Updated weights for policy 1, policy_version 110555 (0.0007) [2023-12-27 06:25:34,164][20347] Updated weights for policy 0, policy_version 111029 (0.0006) [2023-12-27 06:25:34,227][20347] Updated weights for policy 0, policy_version 111039 (0.0008) [2023-12-27 06:25:34,804][20347] Updated weights for policy 0, policy_version 111049 (0.0007) [2023-12-27 06:25:34,859][20381] Updated weights for policy 1, policy_version 110565 (0.0006) [2023-12-27 06:25:34,860][20347] Updated weights for policy 0, policy_version 111059 (0.0005) [2023-12-27 06:25:34,915][20381] Updated weights for policy 1, policy_version 110575 (0.0006) [2023-12-27 06:25:34,922][20347] Updated weights for policy 0, policy_version 111069 (0.0006) [2023-12-27 06:25:34,969][20381] Updated weights for policy 1, policy_version 110585 (0.0007) [2023-12-27 06:25:34,986][20347] Updated weights for policy 0, policy_version 111079 (0.0006) [2023-12-27 06:25:35,520][20347] Updated weights for policy 0, policy_version 111089 (0.0005) [2023-12-27 06:25:35,585][20347] Updated weights for policy 0, policy_version 111099 (0.0005) [2023-12-27 06:25:35,651][20347] Updated weights for policy 0, policy_version 111109 (0.0005) [2023-12-27 06:25:35,728][20381] Updated weights for policy 1, policy_version 110595 (0.0008) [2023-12-27 06:25:35,776][20381] Updated weights for policy 1, policy_version 110605 (0.0007) [2023-12-27 06:25:35,820][20381] Updated weights for policy 1, policy_version 110615 (0.0008) [2023-12-27 06:25:36,272][20347] Updated weights for policy 0, policy_version 111119 (0.0009) [2023-12-27 06:25:36,330][20347] Updated weights for policy 0, policy_version 111129 (0.0008) [2023-12-27 06:25:36,392][20347] Updated weights for policy 0, policy_version 111139 (0.0006) [2023-12-27 06:25:36,527][20381] Updated weights for policy 1, policy_version 110625 (0.0007) [2023-12-27 06:25:36,586][20381] Updated weights for policy 1, policy_version 110635 (0.0006) [2023-12-27 06:25:36,650][20381] Updated weights for policy 1, policy_version 110645 (0.0008) [2023-12-27 06:25:36,708][20381] Updated weights for policy 1, policy_version 110655 (0.0008) [2023-12-27 06:25:37,024][20347] Updated weights for policy 0, policy_version 111149 (0.0007) [2023-12-27 06:25:37,090][20347] Updated weights for policy 0, policy_version 111159 (0.0007) [2023-12-27 06:25:37,145][20347] Updated weights for policy 0, policy_version 111169 (0.0007) [2023-12-27 06:25:37,356][20381] Updated weights for policy 1, policy_version 110665 (0.0008) [2023-12-27 06:25:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 56811520. Throughput: 0: 10414.2, 1: 10271.1. Samples: 56780612. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000111176_28475392.pth... [2023-12-27 06:25:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000109960_28164096.pth [2023-12-27 06:25:37,411][20381] Updated weights for policy 1, policy_version 110675 (0.0005) [2023-12-27 06:25:37,469][20381] Updated weights for policy 1, policy_version 110685 (0.0005) [2023-12-27 06:25:37,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000110688_28344320.pth... [2023-12-27 06:25:37,490][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000109472_28033024.pth [2023-12-27 06:25:37,825][20347] Updated weights for policy 0, policy_version 111179 (0.0007) [2023-12-27 06:25:37,889][20347] Updated weights for policy 0, policy_version 111189 (0.0006) [2023-12-27 06:25:37,947][20347] Updated weights for policy 0, policy_version 111199 (0.0007) [2023-12-27 06:25:38,141][20381] Updated weights for policy 1, policy_version 110695 (0.0006) [2023-12-27 06:25:38,202][20381] Updated weights for policy 1, policy_version 110705 (0.0008) [2023-12-27 06:25:38,256][20381] Updated weights for policy 1, policy_version 110715 (0.0007) [2023-12-27 06:25:38,673][20347] Updated weights for policy 0, policy_version 111209 (0.0007) [2023-12-27 06:25:38,737][20347] Updated weights for policy 0, policy_version 111219 (0.0006) [2023-12-27 06:25:38,798][20347] Updated weights for policy 0, policy_version 111229 (0.0005) [2023-12-27 06:25:38,862][20347] Updated weights for policy 0, policy_version 111239 (0.0005) [2023-12-27 06:25:38,938][20381] Updated weights for policy 1, policy_version 110725 (0.0008) [2023-12-27 06:25:39,002][20381] Updated weights for policy 1, policy_version 110735 (0.0008) [2023-12-27 06:25:39,064][20381] Updated weights for policy 1, policy_version 110745 (0.0006) [2023-12-27 06:25:39,523][20347] Updated weights for policy 0, policy_version 111249 (0.0008) [2023-12-27 06:25:39,581][20347] Updated weights for policy 0, policy_version 111259 (0.0008) [2023-12-27 06:25:39,634][20347] Updated weights for policy 0, policy_version 111269 (0.0008) [2023-12-27 06:25:39,771][20381] Updated weights for policy 1, policy_version 110755 (0.0009) [2023-12-27 06:25:39,841][20381] Updated weights for policy 1, policy_version 110765 (0.0009) [2023-12-27 06:25:39,908][20381] Updated weights for policy 1, policy_version 110775 (0.0008) [2023-12-27 06:25:40,403][20347] Updated weights for policy 0, policy_version 111279 (0.0008) [2023-12-27 06:25:40,462][20347] Updated weights for policy 0, policy_version 111289 (0.0007) [2023-12-27 06:25:40,515][20347] Updated weights for policy 0, policy_version 111299 (0.0008) [2023-12-27 06:25:40,586][20381] Updated weights for policy 1, policy_version 110785 (0.0007) [2023-12-27 06:25:40,632][20381] Updated weights for policy 1, policy_version 110795 (0.0005) [2023-12-27 06:25:40,679][20381] Updated weights for policy 1, policy_version 110805 (0.0005) [2023-12-27 06:25:40,734][20381] Updated weights for policy 1, policy_version 110815 (0.0007) [2023-12-27 06:25:41,215][20347] Updated weights for policy 0, policy_version 111309 (0.0009) [2023-12-27 06:25:41,274][20347] Updated weights for policy 0, policy_version 111319 (0.0007) [2023-12-27 06:25:41,332][20347] Updated weights for policy 0, policy_version 111329 (0.0008) [2023-12-27 06:25:41,464][20381] Updated weights for policy 1, policy_version 110825 (0.0009) [2023-12-27 06:25:41,525][20381] Updated weights for policy 1, policy_version 110835 (0.0007) [2023-12-27 06:25:41,585][20381] Updated weights for policy 1, policy_version 110845 (0.0006) [2023-12-27 06:25:42,029][20347] Updated weights for policy 0, policy_version 111339 (0.0007) [2023-12-27 06:25:42,085][20347] Updated weights for policy 0, policy_version 111349 (0.0008) [2023-12-27 06:25:42,143][20347] Updated weights for policy 0, policy_version 111359 (0.0007) [2023-12-27 06:25:42,244][20381] Updated weights for policy 1, policy_version 110855 (0.0007) [2023-12-27 06:25:42,310][20381] Updated weights for policy 1, policy_version 110865 (0.0009) [2023-12-27 06:25:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.7, 300 sec: 20827.1). Total num frames: 56909824. Throughput: 0: 10446.4, 1: 10245.6. Samples: 56902796. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:42,373][20381] Updated weights for policy 1, policy_version 110875 (0.0009) [2023-12-27 06:25:42,822][20347] Updated weights for policy 0, policy_version 111369 (0.0007) [2023-12-27 06:25:42,884][20347] Updated weights for policy 0, policy_version 111379 (0.0007) [2023-12-27 06:25:42,938][20347] Updated weights for policy 0, policy_version 111389 (0.0006) [2023-12-27 06:25:43,001][20347] Updated weights for policy 0, policy_version 111399 (0.0005) [2023-12-27 06:25:43,060][20381] Updated weights for policy 1, policy_version 110885 (0.0008) [2023-12-27 06:25:43,109][20381] Updated weights for policy 1, policy_version 110895 (0.0007) [2023-12-27 06:25:43,152][20381] Updated weights for policy 1, policy_version 110905 (0.0006) [2023-12-27 06:25:43,658][20347] Updated weights for policy 0, policy_version 111409 (0.0007) [2023-12-27 06:25:43,716][20347] Updated weights for policy 0, policy_version 111419 (0.0006) [2023-12-27 06:25:43,775][20347] Updated weights for policy 0, policy_version 111429 (0.0008) [2023-12-27 06:25:43,853][20381] Updated weights for policy 1, policy_version 110915 (0.0006) [2023-12-27 06:25:43,905][20381] Updated weights for policy 1, policy_version 110925 (0.0008) [2023-12-27 06:25:43,960][20381] Updated weights for policy 1, policy_version 110935 (0.0008) [2023-12-27 06:25:44,449][20347] Updated weights for policy 0, policy_version 111439 (0.0006) [2023-12-27 06:25:44,498][20347] Updated weights for policy 0, policy_version 111449 (0.0005) [2023-12-27 06:25:44,548][20347] Updated weights for policy 0, policy_version 111459 (0.0007) [2023-12-27 06:25:44,633][20381] Updated weights for policy 1, policy_version 110945 (0.0008) [2023-12-27 06:25:44,692][20381] Updated weights for policy 1, policy_version 110955 (0.0007) [2023-12-27 06:25:44,737][20381] Updated weights for policy 1, policy_version 110965 (0.0007) [2023-12-27 06:25:44,796][20381] Updated weights for policy 1, policy_version 110975 (0.0008) [2023-12-27 06:25:45,153][20347] Updated weights for policy 0, policy_version 111469 (0.0007) [2023-12-27 06:25:45,210][20347] Updated weights for policy 0, policy_version 111479 (0.0008) [2023-12-27 06:25:45,270][20347] Updated weights for policy 0, policy_version 111489 (0.0007) [2023-12-27 06:25:45,492][20381] Updated weights for policy 1, policy_version 110985 (0.0008) [2023-12-27 06:25:45,548][20381] Updated weights for policy 1, policy_version 110995 (0.0007) [2023-12-27 06:25:45,605][20381] Updated weights for policy 1, policy_version 111005 (0.0007) [2023-12-27 06:25:45,932][20347] Updated weights for policy 0, policy_version 111499 (0.0008) [2023-12-27 06:25:45,988][20347] Updated weights for policy 0, policy_version 111509 (0.0007) [2023-12-27 06:25:46,038][20347] Updated weights for policy 0, policy_version 111519 (0.0008) [2023-12-27 06:25:46,277][20381] Updated weights for policy 1, policy_version 111015 (0.0006) [2023-12-27 06:25:46,348][20381] Updated weights for policy 1, policy_version 111025 (0.0005) [2023-12-27 06:25:46,408][20381] Updated weights for policy 1, policy_version 111035 (0.0008) [2023-12-27 06:25:46,750][20347] Updated weights for policy 0, policy_version 111529 (0.0008) [2023-12-27 06:25:46,808][20347] Updated weights for policy 0, policy_version 111539 (0.0007) [2023-12-27 06:25:46,864][20347] Updated weights for policy 0, policy_version 111549 (0.0007) [2023-12-27 06:25:46,920][20347] Updated weights for policy 0, policy_version 111559 (0.0005) [2023-12-27 06:25:47,052][20381] Updated weights for policy 1, policy_version 111045 (0.0008) [2023-12-27 06:25:47,105][20381] Updated weights for policy 1, policy_version 111055 (0.0008) [2023-12-27 06:25:47,160][20381] Updated weights for policy 1, policy_version 111065 (0.0007) [2023-12-27 06:25:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 57016320. Throughput: 0: 10435.8, 1: 10247.2. Samples: 57024580. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:47,563][20347] Updated weights for policy 0, policy_version 111569 (0.0007) [2023-12-27 06:25:47,612][20347] Updated weights for policy 0, policy_version 111579 (0.0008) [2023-12-27 06:25:47,664][20347] Updated weights for policy 0, policy_version 111589 (0.0008) [2023-12-27 06:25:47,835][20381] Updated weights for policy 1, policy_version 111075 (0.0008) [2023-12-27 06:25:47,883][20381] Updated weights for policy 1, policy_version 111085 (0.0007) [2023-12-27 06:25:47,932][20381] Updated weights for policy 1, policy_version 111095 (0.0007) [2023-12-27 06:25:48,295][20347] Updated weights for policy 0, policy_version 111599 (0.0006) [2023-12-27 06:25:48,356][20347] Updated weights for policy 0, policy_version 111609 (0.0007) [2023-12-27 06:25:48,405][20347] Updated weights for policy 0, policy_version 111619 (0.0008) [2023-12-27 06:25:48,655][20381] Updated weights for policy 1, policy_version 111105 (0.0007) [2023-12-27 06:25:48,716][20381] Updated weights for policy 1, policy_version 111115 (0.0005) [2023-12-27 06:25:48,785][20381] Updated weights for policy 1, policy_version 111125 (0.0009) [2023-12-27 06:25:48,855][20381] Updated weights for policy 1, policy_version 111135 (0.0008) [2023-12-27 06:25:49,095][20347] Updated weights for policy 0, policy_version 111629 (0.0008) [2023-12-27 06:25:49,155][20347] Updated weights for policy 0, policy_version 111639 (0.0010) [2023-12-27 06:25:49,216][20347] Updated weights for policy 0, policy_version 111649 (0.0010) [2023-12-27 06:25:49,565][20381] Updated weights for policy 1, policy_version 111145 (0.0007) [2023-12-27 06:25:49,635][20381] Updated weights for policy 1, policy_version 111155 (0.0007) [2023-12-27 06:25:49,692][20381] Updated weights for policy 1, policy_version 111165 (0.0008) [2023-12-27 06:25:49,926][20347] Updated weights for policy 0, policy_version 111659 (0.0010) [2023-12-27 06:25:49,985][20347] Updated weights for policy 0, policy_version 111669 (0.0007) [2023-12-27 06:25:50,043][20347] Updated weights for policy 0, policy_version 111679 (0.0005) [2023-12-27 06:25:50,353][20381] Updated weights for policy 1, policy_version 111175 (0.0008) [2023-12-27 06:25:50,416][20381] Updated weights for policy 1, policy_version 111185 (0.0008) [2023-12-27 06:25:50,479][20381] Updated weights for policy 1, policy_version 111195 (0.0008) [2023-12-27 06:25:50,663][20347] Updated weights for policy 0, policy_version 111689 (0.0007) [2023-12-27 06:25:50,719][20347] Updated weights for policy 0, policy_version 111699 (0.0009) [2023-12-27 06:25:50,772][20347] Updated weights for policy 0, policy_version 111709 (0.0009) [2023-12-27 06:25:50,821][20347] Updated weights for policy 0, policy_version 111719 (0.0008) [2023-12-27 06:25:51,273][20381] Updated weights for policy 1, policy_version 111205 (0.0008) [2023-12-27 06:25:51,341][20381] Updated weights for policy 1, policy_version 111215 (0.0008) [2023-12-27 06:25:51,405][20381] Updated weights for policy 1, policy_version 111225 (0.0007) [2023-12-27 06:25:51,543][20347] Updated weights for policy 0, policy_version 111729 (0.0006) [2023-12-27 06:25:51,598][20347] Updated weights for policy 0, policy_version 111739 (0.0007) [2023-12-27 06:25:51,664][20347] Updated weights for policy 0, policy_version 111749 (0.0008) [2023-12-27 06:25:52,095][20381] Updated weights for policy 1, policy_version 111235 (0.0008) [2023-12-27 06:25:52,160][20381] Updated weights for policy 1, policy_version 111245 (0.0008) [2023-12-27 06:25:52,224][20381] Updated weights for policy 1, policy_version 111255 (0.0008) [2023-12-27 06:25:52,271][20347] Updated weights for policy 0, policy_version 111759 (0.0007) [2023-12-27 06:25:52,332][20347] Updated weights for policy 0, policy_version 111769 (0.0008) [2023-12-27 06:25:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 57114624. Throughput: 0: 10413.2, 1: 10196.4. Samples: 57086056. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000111264_28491776.pth... [2023-12-27 06:25:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000110080_28188672.pth [2023-12-27 06:25:52,401][20347] Updated weights for policy 0, policy_version 111779 (0.0008) [2023-12-27 06:25:52,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000111784_28631040.pth... [2023-12-27 06:25:52,430][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000110568_28319744.pth [2023-12-27 06:25:52,950][20381] Updated weights for policy 1, policy_version 111265 (0.0007) [2023-12-27 06:25:53,009][20381] Updated weights for policy 1, policy_version 111275 (0.0008) [2023-12-27 06:25:53,047][20347] Updated weights for policy 0, policy_version 111789 (0.0007) [2023-12-27 06:25:53,069][20381] Updated weights for policy 1, policy_version 111285 (0.0007) [2023-12-27 06:25:53,106][20347] Updated weights for policy 0, policy_version 111799 (0.0007) [2023-12-27 06:25:53,126][20381] Updated weights for policy 1, policy_version 111295 (0.0006) [2023-12-27 06:25:53,162][20347] Updated weights for policy 0, policy_version 111809 (0.0006) [2023-12-27 06:25:53,716][20381] Updated weights for policy 1, policy_version 111305 (0.0005) [2023-12-27 06:25:53,764][20381] Updated weights for policy 1, policy_version 111315 (0.0005) [2023-12-27 06:25:53,820][20381] Updated weights for policy 1, policy_version 111325 (0.0006) [2023-12-27 06:25:53,898][20347] Updated weights for policy 0, policy_version 111819 (0.0008) [2023-12-27 06:25:53,960][20347] Updated weights for policy 0, policy_version 111829 (0.0006) [2023-12-27 06:25:54,012][20347] Updated weights for policy 0, policy_version 111839 (0.0007) [2023-12-27 06:25:54,468][20381] Updated weights for policy 1, policy_version 111335 (0.0009) [2023-12-27 06:25:54,529][20381] Updated weights for policy 1, policy_version 111345 (0.0011) [2023-12-27 06:25:54,585][20381] Updated weights for policy 1, policy_version 111355 (0.0011) [2023-12-27 06:25:54,642][20347] Updated weights for policy 0, policy_version 111849 (0.0006) [2023-12-27 06:25:54,705][20347] Updated weights for policy 0, policy_version 111859 (0.0006) [2023-12-27 06:25:54,770][20347] Updated weights for policy 0, policy_version 111869 (0.0006) [2023-12-27 06:25:54,829][20347] Updated weights for policy 0, policy_version 111879 (0.0006) [2023-12-27 06:25:55,279][20381] Updated weights for policy 1, policy_version 111365 (0.0011) [2023-12-27 06:25:55,332][20381] Updated weights for policy 1, policy_version 111375 (0.0011) [2023-12-27 06:25:55,391][20381] Updated weights for policy 1, policy_version 111385 (0.0010) [2023-12-27 06:25:55,452][20347] Updated weights for policy 0, policy_version 111889 (0.0008) [2023-12-27 06:25:55,505][20347] Updated weights for policy 0, policy_version 111899 (0.0006) [2023-12-27 06:25:55,563][20347] Updated weights for policy 0, policy_version 111909 (0.0005) [2023-12-27 06:25:56,050][20381] Updated weights for policy 1, policy_version 111395 (0.0011) [2023-12-27 06:25:56,107][20381] Updated weights for policy 1, policy_version 111405 (0.0011) [2023-12-27 06:25:56,143][20347] Updated weights for policy 0, policy_version 111919 (0.0008) [2023-12-27 06:25:56,159][20381] Updated weights for policy 1, policy_version 111415 (0.0010) [2023-12-27 06:25:56,195][20347] Updated weights for policy 0, policy_version 111929 (0.0010) [2023-12-27 06:25:56,246][20347] Updated weights for policy 0, policy_version 111939 (0.0010) [2023-12-27 06:25:56,818][20381] Updated weights for policy 1, policy_version 111425 (0.0010) [2023-12-27 06:25:56,887][20381] Updated weights for policy 1, policy_version 111435 (0.0006) [2023-12-27 06:25:56,945][20381] Updated weights for policy 1, policy_version 111445 (0.0008) [2023-12-27 06:25:56,958][20347] Updated weights for policy 0, policy_version 111949 (0.0008) [2023-12-27 06:25:57,000][20381] Updated weights for policy 1, policy_version 111455 (0.0009) [2023-12-27 06:25:57,011][20347] Updated weights for policy 0, policy_version 111959 (0.0007) [2023-12-27 06:25:57,060][20347] Updated weights for policy 0, policy_version 111969 (0.0008) [2023-12-27 06:25:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 57221120. Throughput: 0: 10427.8, 1: 10219.6. Samples: 57209904. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:25:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:25:57,565][20381] Updated weights for policy 1, policy_version 111465 (0.0006) [2023-12-27 06:25:57,620][20381] Updated weights for policy 1, policy_version 111475 (0.0005) [2023-12-27 06:25:57,669][20381] Updated weights for policy 1, policy_version 111485 (0.0007) [2023-12-27 06:25:57,689][20347] Updated weights for policy 0, policy_version 111979 (0.0007) [2023-12-27 06:25:57,736][20347] Updated weights for policy 0, policy_version 111989 (0.0008) [2023-12-27 06:25:57,784][20347] Updated weights for policy 0, policy_version 111999 (0.0007) [2023-12-27 06:25:58,263][20381] Updated weights for policy 1, policy_version 111495 (0.0008) [2023-12-27 06:25:58,319][20381] Updated weights for policy 1, policy_version 111505 (0.0008) [2023-12-27 06:25:58,385][20381] Updated weights for policy 1, policy_version 111515 (0.0008) [2023-12-27 06:25:58,428][20347] Updated weights for policy 0, policy_version 112009 (0.0006) [2023-12-27 06:25:58,484][20347] Updated weights for policy 0, policy_version 112019 (0.0008) [2023-12-27 06:25:58,539][20347] Updated weights for policy 0, policy_version 112029 (0.0007) [2023-12-27 06:25:58,598][20347] Updated weights for policy 0, policy_version 112039 (0.0008) [2023-12-27 06:25:59,148][20381] Updated weights for policy 1, policy_version 111525 (0.0008) [2023-12-27 06:25:59,207][20381] Updated weights for policy 1, policy_version 111535 (0.0008) [2023-12-27 06:25:59,278][20381] Updated weights for policy 1, policy_version 111545 (0.0008) [2023-12-27 06:25:59,447][20347] Updated weights for policy 0, policy_version 112049 (0.0008) [2023-12-27 06:25:59,508][20347] Updated weights for policy 0, policy_version 112059 (0.0008) [2023-12-27 06:25:59,568][20347] Updated weights for policy 0, policy_version 112069 (0.0007) [2023-12-27 06:26:00,018][20381] Updated weights for policy 1, policy_version 111555 (0.0007) [2023-12-27 06:26:00,074][20381] Updated weights for policy 1, policy_version 111565 (0.0006) [2023-12-27 06:26:00,135][20381] Updated weights for policy 1, policy_version 111575 (0.0006) [2023-12-27 06:26:00,279][20347] Updated weights for policy 0, policy_version 112079 (0.0007) [2023-12-27 06:26:00,338][20347] Updated weights for policy 0, policy_version 112089 (0.0008) [2023-12-27 06:26:00,396][20347] Updated weights for policy 0, policy_version 112099 (0.0008) [2023-12-27 06:26:00,766][20381] Updated weights for policy 1, policy_version 111585 (0.0006) [2023-12-27 06:26:00,823][20381] Updated weights for policy 1, policy_version 111595 (0.0005) [2023-12-27 06:26:00,873][20381] Updated weights for policy 1, policy_version 111605 (0.0005) [2023-12-27 06:26:00,932][20381] Updated weights for policy 1, policy_version 111615 (0.0008) [2023-12-27 06:26:01,099][20347] Updated weights for policy 0, policy_version 112109 (0.0008) [2023-12-27 06:26:01,156][20347] Updated weights for policy 0, policy_version 112119 (0.0007) [2023-12-27 06:26:01,212][20347] Updated weights for policy 0, policy_version 112129 (0.0007) [2023-12-27 06:26:01,592][20381] Updated weights for policy 1, policy_version 111625 (0.0005) [2023-12-27 06:26:01,657][20381] Updated weights for policy 1, policy_version 111635 (0.0008) [2023-12-27 06:26:01,712][20381] Updated weights for policy 1, policy_version 111645 (0.0008) [2023-12-27 06:26:01,974][20347] Updated weights for policy 0, policy_version 112139 (0.0008) [2023-12-27 06:26:02,033][20347] Updated weights for policy 0, policy_version 112149 (0.0008) [2023-12-27 06:26:02,089][20347] Updated weights for policy 0, policy_version 112159 (0.0007) [2023-12-27 06:26:02,337][20381] Updated weights for policy 1, policy_version 111655 (0.0008) [2023-12-27 06:26:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 57319424. Throughput: 0: 10363.1, 1: 10232.2. Samples: 57331984. Policy #0 lag: (min: 12.0, avg: 18.3, max: 44.0) [2023-12-27 06:26:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:02,401][20381] Updated weights for policy 1, policy_version 111665 (0.0006) [2023-12-27 06:26:02,461][20381] Updated weights for policy 1, policy_version 111675 (0.0006) [2023-12-27 06:26:02,781][20347] Updated weights for policy 0, policy_version 112169 (0.0007) [2023-12-27 06:26:02,838][20347] Updated weights for policy 0, policy_version 112179 (0.0008) [2023-12-27 06:26:02,895][20347] Updated weights for policy 0, policy_version 112189 (0.0008) [2023-12-27 06:26:02,951][20347] Updated weights for policy 0, policy_version 112199 (0.0007) [2023-12-27 06:26:03,069][20381] Updated weights for policy 1, policy_version 111685 (0.0006) [2023-12-27 06:26:03,127][20381] Updated weights for policy 1, policy_version 111695 (0.0005) [2023-12-27 06:26:03,181][20381] Updated weights for policy 1, policy_version 111705 (0.0008) [2023-12-27 06:26:03,630][20347] Updated weights for policy 0, policy_version 112209 (0.0005) [2023-12-27 06:26:03,677][20347] Updated weights for policy 0, policy_version 112219 (0.0005) [2023-12-27 06:26:03,732][20347] Updated weights for policy 0, policy_version 112229 (0.0005) [2023-12-27 06:26:03,755][20381] Updated weights for policy 1, policy_version 111715 (0.0010) [2023-12-27 06:26:03,806][20381] Updated weights for policy 1, policy_version 111725 (0.0010) [2023-12-27 06:26:03,866][20381] Updated weights for policy 1, policy_version 111735 (0.0011) [2023-12-27 06:26:04,379][20347] Updated weights for policy 0, policy_version 112239 (0.0007) [2023-12-27 06:26:04,445][20347] Updated weights for policy 0, policy_version 112249 (0.0007) [2023-12-27 06:26:04,476][20381] Updated weights for policy 1, policy_version 111745 (0.0010) [2023-12-27 06:26:04,504][20347] Updated weights for policy 0, policy_version 112259 (0.0007) [2023-12-27 06:26:04,529][20381] Updated weights for policy 1, policy_version 111755 (0.0010) [2023-12-27 06:26:04,582][20381] Updated weights for policy 1, policy_version 111765 (0.0010) [2023-12-27 06:26:04,645][20381] Updated weights for policy 1, policy_version 111775 (0.0011) [2023-12-27 06:26:05,083][20347] Updated weights for policy 0, policy_version 112269 (0.0005) [2023-12-27 06:26:05,141][20347] Updated weights for policy 0, policy_version 112279 (0.0007) [2023-12-27 06:26:05,198][20347] Updated weights for policy 0, policy_version 112289 (0.0008) [2023-12-27 06:26:05,328][20381] Updated weights for policy 1, policy_version 111785 (0.0010) [2023-12-27 06:26:05,376][20381] Updated weights for policy 1, policy_version 111795 (0.0010) [2023-12-27 06:26:05,430][20381] Updated weights for policy 1, policy_version 111805 (0.0010) [2023-12-27 06:26:05,875][20347] Updated weights for policy 0, policy_version 112299 (0.0009) [2023-12-27 06:26:05,934][20347] Updated weights for policy 0, policy_version 112309 (0.0008) [2023-12-27 06:26:05,984][20347] Updated weights for policy 0, policy_version 112319 (0.0008) [2023-12-27 06:26:06,094][20381] Updated weights for policy 1, policy_version 111815 (0.0008) [2023-12-27 06:26:06,153][20381] Updated weights for policy 1, policy_version 111825 (0.0007) [2023-12-27 06:26:06,214][20381] Updated weights for policy 1, policy_version 111835 (0.0008) [2023-12-27 06:26:06,710][20347] Updated weights for policy 0, policy_version 112329 (0.0007) [2023-12-27 06:26:06,756][20347] Updated weights for policy 0, policy_version 112339 (0.0010) [2023-12-27 06:26:06,810][20347] Updated weights for policy 0, policy_version 112349 (0.0007) [2023-12-27 06:26:06,830][20381] Updated weights for policy 1, policy_version 111845 (0.0006) [2023-12-27 06:26:06,869][20347] Updated weights for policy 0, policy_version 112359 (0.0005) [2023-12-27 06:26:06,882][20381] Updated weights for policy 1, policy_version 111855 (0.0005) [2023-12-27 06:26:06,941][20381] Updated weights for policy 1, policy_version 111865 (0.0006) [2023-12-27 06:26:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 57425920. Throughput: 0: 10370.5, 1: 10244.3. Samples: 57396476. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000112360_28778496.pth... [2023-12-27 06:26:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000111872_28647424.pth... [2023-12-27 06:26:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000111176_28475392.pth [2023-12-27 06:26:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000110688_28344320.pth [2023-12-27 06:26:07,557][20347] Updated weights for policy 0, policy_version 112369 (0.0007) [2023-12-27 06:26:07,578][20381] Updated weights for policy 1, policy_version 111875 (0.0005) [2023-12-27 06:26:07,608][20347] Updated weights for policy 0, policy_version 112379 (0.0006) [2023-12-27 06:26:07,650][20381] Updated weights for policy 1, policy_version 111885 (0.0006) [2023-12-27 06:26:07,658][20347] Updated weights for policy 0, policy_version 112389 (0.0005) [2023-12-27 06:26:07,710][20381] Updated weights for policy 1, policy_version 111895 (0.0007) [2023-12-27 06:26:08,346][20347] Updated weights for policy 0, policy_version 112399 (0.0008) [2023-12-27 06:26:08,367][20381] Updated weights for policy 1, policy_version 111905 (0.0005) [2023-12-27 06:26:08,407][20347] Updated weights for policy 0, policy_version 112409 (0.0007) [2023-12-27 06:26:08,424][20381] Updated weights for policy 1, policy_version 111915 (0.0006) [2023-12-27 06:26:08,459][20347] Updated weights for policy 0, policy_version 112419 (0.0006) [2023-12-27 06:26:08,479][20381] Updated weights for policy 1, policy_version 111925 (0.0005) [2023-12-27 06:26:08,537][20381] Updated weights for policy 1, policy_version 111935 (0.0007) [2023-12-27 06:26:09,150][20347] Updated weights for policy 0, policy_version 112429 (0.0008) [2023-12-27 06:26:09,185][20381] Updated weights for policy 1, policy_version 111945 (0.0007) [2023-12-27 06:26:09,206][20347] Updated weights for policy 0, policy_version 112439 (0.0010) [2023-12-27 06:26:09,250][20381] Updated weights for policy 1, policy_version 111955 (0.0007) [2023-12-27 06:26:09,267][20347] Updated weights for policy 0, policy_version 112449 (0.0010) [2023-12-27 06:26:09,305][20381] Updated weights for policy 1, policy_version 111965 (0.0009) [2023-12-27 06:26:09,989][20347] Updated weights for policy 0, policy_version 112459 (0.0011) [2023-12-27 06:26:10,046][20347] Updated weights for policy 0, policy_version 112469 (0.0011) [2023-12-27 06:26:10,055][20381] Updated weights for policy 1, policy_version 111975 (0.0006) [2023-12-27 06:26:10,113][20347] Updated weights for policy 0, policy_version 112479 (0.0011) [2023-12-27 06:26:10,122][20381] Updated weights for policy 1, policy_version 111985 (0.0006) [2023-12-27 06:26:10,188][20381] Updated weights for policy 1, policy_version 111995 (0.0006) [2023-12-27 06:26:10,810][20347] Updated weights for policy 0, policy_version 112489 (0.0011) [2023-12-27 06:26:10,872][20347] Updated weights for policy 0, policy_version 112499 (0.0010) [2023-12-27 06:26:10,894][20381] Updated weights for policy 1, policy_version 112005 (0.0009) [2023-12-27 06:26:10,928][20347] Updated weights for policy 0, policy_version 112509 (0.0010) [2023-12-27 06:26:10,941][20381] Updated weights for policy 1, policy_version 112015 (0.0009) [2023-12-27 06:26:10,990][20381] Updated weights for policy 1, policy_version 112025 (0.0006) [2023-12-27 06:26:10,990][20347] Updated weights for policy 0, policy_version 112519 (0.0011) [2023-12-27 06:26:11,688][20347] Updated weights for policy 0, policy_version 112529 (0.0010) [2023-12-27 06:26:11,726][20381] Updated weights for policy 1, policy_version 112035 (0.0008) [2023-12-27 06:26:11,758][20347] Updated weights for policy 0, policy_version 112539 (0.0009) [2023-12-27 06:26:11,787][20381] Updated weights for policy 1, policy_version 112045 (0.0006) [2023-12-27 06:26:11,820][20347] Updated weights for policy 0, policy_version 112549 (0.0007) [2023-12-27 06:26:11,849][20381] Updated weights for policy 1, policy_version 112055 (0.0006) [2023-12-27 06:26:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 57524224. Throughput: 0: 10320.1, 1: 10274.9. Samples: 57517436. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:12,518][20381] Updated weights for policy 1, policy_version 112065 (0.0006) [2023-12-27 06:26:12,542][20347] Updated weights for policy 0, policy_version 112559 (0.0010) [2023-12-27 06:26:12,570][20381] Updated weights for policy 1, policy_version 112075 (0.0005) [2023-12-27 06:26:12,594][20347] Updated weights for policy 0, policy_version 112569 (0.0010) [2023-12-27 06:26:12,619][20381] Updated weights for policy 1, policy_version 112085 (0.0005) [2023-12-27 06:26:12,647][20347] Updated weights for policy 0, policy_version 112579 (0.0010) [2023-12-27 06:26:12,671][20381] Updated weights for policy 1, policy_version 112095 (0.0005) [2023-12-27 06:26:13,343][20347] Updated weights for policy 0, policy_version 112589 (0.0010) [2023-12-27 06:26:13,385][20381] Updated weights for policy 1, policy_version 112105 (0.0008) [2023-12-27 06:26:13,392][20347] Updated weights for policy 0, policy_version 112599 (0.0010) [2023-12-27 06:26:13,444][20381] Updated weights for policy 1, policy_version 112115 (0.0007) [2023-12-27 06:26:13,446][20347] Updated weights for policy 0, policy_version 112609 (0.0010) [2023-12-27 06:26:13,507][20381] Updated weights for policy 1, policy_version 112125 (0.0008) [2023-12-27 06:26:14,059][20347] Updated weights for policy 0, policy_version 112619 (0.0007) [2023-12-27 06:26:14,114][20347] Updated weights for policy 0, policy_version 112629 (0.0008) [2023-12-27 06:26:14,168][20347] Updated weights for policy 0, policy_version 112639 (0.0008) [2023-12-27 06:26:14,190][20381] Updated weights for policy 1, policy_version 112135 (0.0006) [2023-12-27 06:26:14,247][20381] Updated weights for policy 1, policy_version 112145 (0.0005) [2023-12-27 06:26:14,305][20381] Updated weights for policy 1, policy_version 112155 (0.0008) [2023-12-27 06:26:14,841][20347] Updated weights for policy 0, policy_version 112649 (0.0006) [2023-12-27 06:26:14,887][20347] Updated weights for policy 0, policy_version 112659 (0.0008) [2023-12-27 06:26:14,941][20381] Updated weights for policy 1, policy_version 112165 (0.0006) [2023-12-27 06:26:14,943][20347] Updated weights for policy 0, policy_version 112669 (0.0007) [2023-12-27 06:26:14,998][20347] Updated weights for policy 0, policy_version 112679 (0.0006) [2023-12-27 06:26:15,000][20381] Updated weights for policy 1, policy_version 112175 (0.0006) [2023-12-27 06:26:15,061][20381] Updated weights for policy 1, policy_version 112185 (0.0007) [2023-12-27 06:26:15,686][20381] Updated weights for policy 1, policy_version 112195 (0.0008) [2023-12-27 06:26:15,687][20347] Updated weights for policy 0, policy_version 112689 (0.0007) [2023-12-27 06:26:15,737][20381] Updated weights for policy 1, policy_version 112205 (0.0006) [2023-12-27 06:26:15,738][20347] Updated weights for policy 0, policy_version 112699 (0.0007) [2023-12-27 06:26:15,790][20381] Updated weights for policy 1, policy_version 112215 (0.0005) [2023-12-27 06:26:15,795][20347] Updated weights for policy 0, policy_version 112709 (0.0008) [2023-12-27 06:26:16,453][20347] Updated weights for policy 0, policy_version 112719 (0.0007) [2023-12-27 06:26:16,457][20381] Updated weights for policy 1, policy_version 112225 (0.0006) [2023-12-27 06:26:16,502][20381] Updated weights for policy 1, policy_version 112235 (0.0005) [2023-12-27 06:26:16,516][20347] Updated weights for policy 0, policy_version 112729 (0.0006) [2023-12-27 06:26:16,551][20381] Updated weights for policy 1, policy_version 112245 (0.0007) [2023-12-27 06:26:16,568][20347] Updated weights for policy 0, policy_version 112739 (0.0005) [2023-12-27 06:26:16,602][20381] Updated weights for policy 1, policy_version 112255 (0.0007) [2023-12-27 06:26:17,209][20347] Updated weights for policy 0, policy_version 112749 (0.0006) [2023-12-27 06:26:17,266][20347] Updated weights for policy 0, policy_version 112759 (0.0005) [2023-12-27 06:26:17,283][20381] Updated weights for policy 1, policy_version 112265 (0.0007) [2023-12-27 06:26:17,322][20347] Updated weights for policy 0, policy_version 112769 (0.0005) [2023-12-27 06:26:17,328][20381] Updated weights for policy 1, policy_version 112275 (0.0007) [2023-12-27 06:26:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 57630720. Throughput: 0: 10283.6, 1: 10279.7. Samples: 57641844. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:17,373][20381] Updated weights for policy 1, policy_version 112285 (0.0006) [2023-12-27 06:26:17,970][20347] Updated weights for policy 0, policy_version 112779 (0.0007) [2023-12-27 06:26:17,984][20381] Updated weights for policy 1, policy_version 112295 (0.0006) [2023-12-27 06:26:18,020][20347] Updated weights for policy 0, policy_version 112789 (0.0008) [2023-12-27 06:26:18,039][20381] Updated weights for policy 1, policy_version 112305 (0.0005) [2023-12-27 06:26:18,074][20347] Updated weights for policy 0, policy_version 112799 (0.0007) [2023-12-27 06:26:18,089][20381] Updated weights for policy 1, policy_version 112315 (0.0005) [2023-12-27 06:26:18,710][20381] Updated weights for policy 1, policy_version 112325 (0.0006) [2023-12-27 06:26:18,769][20381] Updated weights for policy 1, policy_version 112335 (0.0005) [2023-12-27 06:26:18,776][20347] Updated weights for policy 0, policy_version 112809 (0.0008) [2023-12-27 06:26:18,827][20381] Updated weights for policy 1, policy_version 112345 (0.0006) [2023-12-27 06:26:18,837][20347] Updated weights for policy 0, policy_version 112819 (0.0007) [2023-12-27 06:26:18,890][20347] Updated weights for policy 0, policy_version 112829 (0.0007) [2023-12-27 06:26:18,946][20347] Updated weights for policy 0, policy_version 112839 (0.0008) [2023-12-27 06:26:19,361][20381] Updated weights for policy 1, policy_version 112355 (0.0006) [2023-12-27 06:26:19,418][20381] Updated weights for policy 1, policy_version 112365 (0.0006) [2023-12-27 06:26:19,475][20381] Updated weights for policy 1, policy_version 112375 (0.0005) [2023-12-27 06:26:19,739][20347] Updated weights for policy 0, policy_version 112849 (0.0007) [2023-12-27 06:26:19,795][20347] Updated weights for policy 0, policy_version 112859 (0.0008) [2023-12-27 06:26:19,854][20347] Updated weights for policy 0, policy_version 112869 (0.0007) [2023-12-27 06:26:20,164][20381] Updated weights for policy 1, policy_version 112385 (0.0007) [2023-12-27 06:26:20,226][20381] Updated weights for policy 1, policy_version 112395 (0.0008) [2023-12-27 06:26:20,282][20381] Updated weights for policy 1, policy_version 112405 (0.0008) [2023-12-27 06:26:20,341][20381] Updated weights for policy 1, policy_version 112415 (0.0008) [2023-12-27 06:26:20,512][20347] Updated weights for policy 0, policy_version 112879 (0.0008) [2023-12-27 06:26:20,575][20347] Updated weights for policy 0, policy_version 112889 (0.0008) [2023-12-27 06:26:20,636][20347] Updated weights for policy 0, policy_version 112899 (0.0008) [2023-12-27 06:26:20,990][20381] Updated weights for policy 1, policy_version 112425 (0.0008) [2023-12-27 06:26:21,057][20381] Updated weights for policy 1, policy_version 112435 (0.0010) [2023-12-27 06:26:21,123][20381] Updated weights for policy 1, policy_version 112445 (0.0008) [2023-12-27 06:26:21,373][20347] Updated weights for policy 0, policy_version 112909 (0.0008) [2023-12-27 06:26:21,426][20347] Updated weights for policy 0, policy_version 112919 (0.0008) [2023-12-27 06:26:21,476][20347] Updated weights for policy 0, policy_version 112929 (0.0008) [2023-12-27 06:26:21,834][20381] Updated weights for policy 1, policy_version 112455 (0.0008) [2023-12-27 06:26:21,894][20381] Updated weights for policy 1, policy_version 112465 (0.0007) [2023-12-27 06:26:21,954][20381] Updated weights for policy 1, policy_version 112475 (0.0007) [2023-12-27 06:26:22,231][20347] Updated weights for policy 0, policy_version 112939 (0.0008) [2023-12-27 06:26:22,298][20347] Updated weights for policy 0, policy_version 112949 (0.0009) [2023-12-27 06:26:22,353][20347] Updated weights for policy 0, policy_version 112959 (0.0009) [2023-12-27 06:26:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 57729024. Throughput: 0: 10217.4, 1: 10314.3. Samples: 57704532. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000112480_28803072.pth... [2023-12-27 06:26:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000111264_28491776.pth [2023-12-27 06:26:22,404][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000112968_28934144.pth... [2023-12-27 06:26:22,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000111784_28631040.pth [2023-12-27 06:26:22,672][20381] Updated weights for policy 1, policy_version 112485 (0.0009) [2023-12-27 06:26:22,731][20381] Updated weights for policy 1, policy_version 112495 (0.0011) [2023-12-27 06:26:22,793][20381] Updated weights for policy 1, policy_version 112505 (0.0011) [2023-12-27 06:26:22,984][20347] Updated weights for policy 0, policy_version 112969 (0.0009) [2023-12-27 06:26:23,039][20347] Updated weights for policy 0, policy_version 112979 (0.0005) [2023-12-27 06:26:23,090][20347] Updated weights for policy 0, policy_version 112989 (0.0006) [2023-12-27 06:26:23,139][20347] Updated weights for policy 0, policy_version 112999 (0.0006) [2023-12-27 06:26:23,484][20381] Updated weights for policy 1, policy_version 112515 (0.0011) [2023-12-27 06:26:23,538][20381] Updated weights for policy 1, policy_version 112525 (0.0011) [2023-12-27 06:26:23,593][20381] Updated weights for policy 1, policy_version 112535 (0.0010) [2023-12-27 06:26:23,772][20347] Updated weights for policy 0, policy_version 113009 (0.0005) [2023-12-27 06:26:23,830][20347] Updated weights for policy 0, policy_version 113019 (0.0007) [2023-12-27 06:26:23,894][20347] Updated weights for policy 0, policy_version 113029 (0.0006) [2023-12-27 06:26:24,277][20381] Updated weights for policy 1, policy_version 112545 (0.0011) [2023-12-27 06:26:24,341][20381] Updated weights for policy 1, policy_version 112555 (0.0011) [2023-12-27 06:26:24,392][20381] Updated weights for policy 1, policy_version 112565 (0.0010) [2023-12-27 06:26:24,440][20381] Updated weights for policy 1, policy_version 112575 (0.0010) [2023-12-27 06:26:24,520][20347] Updated weights for policy 0, policy_version 113039 (0.0007) [2023-12-27 06:26:24,578][20347] Updated weights for policy 0, policy_version 113049 (0.0005) [2023-12-27 06:26:24,640][20347] Updated weights for policy 0, policy_version 113059 (0.0005) [2023-12-27 06:26:25,133][20381] Updated weights for policy 1, policy_version 112585 (0.0010) [2023-12-27 06:26:25,189][20381] Updated weights for policy 1, policy_version 112595 (0.0007) [2023-12-27 06:26:25,239][20381] Updated weights for policy 1, policy_version 112605 (0.0009) [2023-12-27 06:26:25,255][20347] Updated weights for policy 0, policy_version 113069 (0.0008) [2023-12-27 06:26:25,319][20347] Updated weights for policy 0, policy_version 113079 (0.0007) [2023-12-27 06:26:25,374][20347] Updated weights for policy 0, policy_version 113089 (0.0007) [2023-12-27 06:26:25,909][20381] Updated weights for policy 1, policy_version 112615 (0.0011) [2023-12-27 06:26:25,966][20381] Updated weights for policy 1, policy_version 112625 (0.0011) [2023-12-27 06:26:26,015][20381] Updated weights for policy 1, policy_version 112635 (0.0011) [2023-12-27 06:26:26,050][20347] Updated weights for policy 0, policy_version 113099 (0.0006) [2023-12-27 06:26:26,101][20347] Updated weights for policy 0, policy_version 113109 (0.0007) [2023-12-27 06:26:26,159][20347] Updated weights for policy 0, policy_version 113119 (0.0008) [2023-12-27 06:26:26,713][20381] Updated weights for policy 1, policy_version 112645 (0.0009) [2023-12-27 06:26:26,763][20381] Updated weights for policy 1, policy_version 112655 (0.0005) [2023-12-27 06:26:26,819][20381] Updated weights for policy 1, policy_version 112665 (0.0005) [2023-12-27 06:26:26,846][20347] Updated weights for policy 0, policy_version 113129 (0.0006) [2023-12-27 06:26:26,900][20347] Updated weights for policy 0, policy_version 113139 (0.0006) [2023-12-27 06:26:26,952][20347] Updated weights for policy 0, policy_version 113149 (0.0008) [2023-12-27 06:26:26,997][20347] Updated weights for policy 0, policy_version 113159 (0.0007) [2023-12-27 06:26:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 57835520. Throughput: 0: 10223.6, 1: 10309.4. Samples: 57826784. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:27,421][20381] Updated weights for policy 1, policy_version 112675 (0.0006) [2023-12-27 06:26:27,471][20381] Updated weights for policy 1, policy_version 112685 (0.0007) [2023-12-27 06:26:27,530][20381] Updated weights for policy 1, policy_version 112695 (0.0006) [2023-12-27 06:26:27,728][20347] Updated weights for policy 0, policy_version 113169 (0.0008) [2023-12-27 06:26:27,781][20347] Updated weights for policy 0, policy_version 113179 (0.0009) [2023-12-27 06:26:27,833][20347] Updated weights for policy 0, policy_version 113189 (0.0006) [2023-12-27 06:26:28,107][20381] Updated weights for policy 1, policy_version 112705 (0.0006) [2023-12-27 06:26:28,162][20381] Updated weights for policy 1, policy_version 112715 (0.0005) [2023-12-27 06:26:28,221][20381] Updated weights for policy 1, policy_version 112725 (0.0005) [2023-12-27 06:26:28,269][20381] Updated weights for policy 1, policy_version 112735 (0.0005) [2023-12-27 06:26:28,570][20347] Updated weights for policy 0, policy_version 113199 (0.0008) [2023-12-27 06:26:28,631][20347] Updated weights for policy 0, policy_version 113209 (0.0007) [2023-12-27 06:26:28,689][20347] Updated weights for policy 0, policy_version 113219 (0.0008) [2023-12-27 06:26:28,850][20381] Updated weights for policy 1, policy_version 112745 (0.0005) [2023-12-27 06:26:28,901][20381] Updated weights for policy 1, policy_version 112755 (0.0006) [2023-12-27 06:26:28,956][20381] Updated weights for policy 1, policy_version 112765 (0.0010) [2023-12-27 06:26:29,415][20347] Updated weights for policy 0, policy_version 113229 (0.0007) [2023-12-27 06:26:29,476][20347] Updated weights for policy 0, policy_version 113239 (0.0005) [2023-12-27 06:26:29,526][20347] Updated weights for policy 0, policy_version 113249 (0.0005) [2023-12-27 06:26:29,638][20381] Updated weights for policy 1, policy_version 112775 (0.0009) [2023-12-27 06:26:29,697][20381] Updated weights for policy 1, policy_version 112785 (0.0008) [2023-12-27 06:26:29,757][20381] Updated weights for policy 1, policy_version 112795 (0.0007) [2023-12-27 06:26:30,128][20347] Updated weights for policy 0, policy_version 113259 (0.0006) [2023-12-27 06:26:30,196][20347] Updated weights for policy 0, policy_version 113269 (0.0007) [2023-12-27 06:26:30,264][20347] Updated weights for policy 0, policy_version 113279 (0.0005) [2023-12-27 06:26:30,446][20381] Updated weights for policy 1, policy_version 112805 (0.0007) [2023-12-27 06:26:30,497][20381] Updated weights for policy 1, policy_version 112815 (0.0005) [2023-12-27 06:26:30,553][20381] Updated weights for policy 1, policy_version 112825 (0.0005) [2023-12-27 06:26:30,963][20347] Updated weights for policy 0, policy_version 113289 (0.0006) [2023-12-27 06:26:31,015][20347] Updated weights for policy 0, policy_version 113299 (0.0007) [2023-12-27 06:26:31,081][20347] Updated weights for policy 0, policy_version 113309 (0.0008) [2023-12-27 06:26:31,128][20381] Updated weights for policy 1, policy_version 112835 (0.0007) [2023-12-27 06:26:31,144][20347] Updated weights for policy 0, policy_version 113319 (0.0007) [2023-12-27 06:26:31,183][20381] Updated weights for policy 1, policy_version 112845 (0.0008) [2023-12-27 06:26:31,233][20381] Updated weights for policy 1, policy_version 112855 (0.0005) [2023-12-27 06:26:31,812][20347] Updated weights for policy 0, policy_version 113329 (0.0006) [2023-12-27 06:26:31,874][20347] Updated weights for policy 0, policy_version 113339 (0.0006) [2023-12-27 06:26:31,928][20347] Updated weights for policy 0, policy_version 113349 (0.0005) [2023-12-27 06:26:31,961][20381] Updated weights for policy 1, policy_version 112865 (0.0007) [2023-12-27 06:26:32,018][20381] Updated weights for policy 1, policy_version 112875 (0.0007) [2023-12-27 06:26:32,075][20381] Updated weights for policy 1, policy_version 112885 (0.0007) [2023-12-27 06:26:32,128][20381] Updated weights for policy 1, policy_version 112895 (0.0009) [2023-12-27 06:26:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 57942016. Throughput: 0: 10196.4, 1: 10393.2. Samples: 57951116. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:32,536][20347] Updated weights for policy 0, policy_version 113359 (0.0006) [2023-12-27 06:26:32,592][20347] Updated weights for policy 0, policy_version 113369 (0.0005) [2023-12-27 06:26:32,648][20347] Updated weights for policy 0, policy_version 113379 (0.0005) [2023-12-27 06:26:32,837][20381] Updated weights for policy 1, policy_version 112905 (0.0008) [2023-12-27 06:26:32,890][20381] Updated weights for policy 1, policy_version 112915 (0.0008) [2023-12-27 06:26:32,947][20381] Updated weights for policy 1, policy_version 112925 (0.0008) [2023-12-27 06:26:33,243][20347] Updated weights for policy 0, policy_version 113389 (0.0006) [2023-12-27 06:26:33,297][20347] Updated weights for policy 0, policy_version 113399 (0.0007) [2023-12-27 06:26:33,351][20347] Updated weights for policy 0, policy_version 113409 (0.0007) [2023-12-27 06:26:33,661][20381] Updated weights for policy 1, policy_version 112935 (0.0007) [2023-12-27 06:26:33,713][20381] Updated weights for policy 1, policy_version 112945 (0.0008) [2023-12-27 06:26:33,764][20381] Updated weights for policy 1, policy_version 112955 (0.0008) [2023-12-27 06:26:34,036][20347] Updated weights for policy 0, policy_version 113419 (0.0007) [2023-12-27 06:26:34,095][20347] Updated weights for policy 0, policy_version 113429 (0.0008) [2023-12-27 06:26:34,146][20347] Updated weights for policy 0, policy_version 113439 (0.0007) [2023-12-27 06:26:34,440][20381] Updated weights for policy 1, policy_version 112965 (0.0007) [2023-12-27 06:26:34,494][20381] Updated weights for policy 1, policy_version 112975 (0.0008) [2023-12-27 06:26:34,549][20381] Updated weights for policy 1, policy_version 112985 (0.0008) [2023-12-27 06:26:34,825][20347] Updated weights for policy 0, policy_version 113449 (0.0007) [2023-12-27 06:26:34,881][20347] Updated weights for policy 0, policy_version 113459 (0.0006) [2023-12-27 06:26:34,946][20347] Updated weights for policy 0, policy_version 113469 (0.0006) [2023-12-27 06:26:35,005][20347] Updated weights for policy 0, policy_version 113479 (0.0007) [2023-12-27 06:26:35,268][20381] Updated weights for policy 1, policy_version 112995 (0.0006) [2023-12-27 06:26:35,323][20381] Updated weights for policy 1, policy_version 113005 (0.0008) [2023-12-27 06:26:35,381][20381] Updated weights for policy 1, policy_version 113015 (0.0005) [2023-12-27 06:26:35,642][20347] Updated weights for policy 0, policy_version 113489 (0.0006) [2023-12-27 06:26:35,692][20347] Updated weights for policy 0, policy_version 113499 (0.0005) [2023-12-27 06:26:35,738][20347] Updated weights for policy 0, policy_version 113509 (0.0005) [2023-12-27 06:26:36,063][20381] Updated weights for policy 1, policy_version 113025 (0.0006) [2023-12-27 06:26:36,128][20381] Updated weights for policy 1, policy_version 113035 (0.0008) [2023-12-27 06:26:36,188][20381] Updated weights for policy 1, policy_version 113045 (0.0008) [2023-12-27 06:26:36,247][20381] Updated weights for policy 1, policy_version 113055 (0.0008) [2023-12-27 06:26:36,398][20347] Updated weights for policy 0, policy_version 113519 (0.0007) [2023-12-27 06:26:36,459][20347] Updated weights for policy 0, policy_version 113529 (0.0008) [2023-12-27 06:26:36,511][20347] Updated weights for policy 0, policy_version 113539 (0.0006) [2023-12-27 06:26:36,864][20381] Updated weights for policy 1, policy_version 113065 (0.0007) [2023-12-27 06:26:36,915][20381] Updated weights for policy 1, policy_version 113075 (0.0008) [2023-12-27 06:26:36,966][20381] Updated weights for policy 1, policy_version 113085 (0.0008) [2023-12-27 06:26:37,173][20347] Updated weights for policy 0, policy_version 113549 (0.0006) [2023-12-27 06:26:37,230][20347] Updated weights for policy 0, policy_version 113559 (0.0005) [2023-12-27 06:26:37,277][20347] Updated weights for policy 0, policy_version 113569 (0.0005) [2023-12-27 06:26:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 58048512. Throughput: 0: 10208.4, 1: 10392.9. Samples: 58013116. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000113088_28958720.pth... [2023-12-27 06:26:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000113576_29089792.pth... [2023-12-27 06:26:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000111872_28647424.pth [2023-12-27 06:26:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000112360_28778496.pth [2023-12-27 06:26:37,712][20381] Updated weights for policy 1, policy_version 113095 (0.0009) [2023-12-27 06:26:37,759][20381] Updated weights for policy 1, policy_version 113105 (0.0007) [2023-12-27 06:26:37,808][20381] Updated weights for policy 1, policy_version 113115 (0.0008) [2023-12-27 06:26:37,902][20347] Updated weights for policy 0, policy_version 113579 (0.0006) [2023-12-27 06:26:37,948][20347] Updated weights for policy 0, policy_version 113589 (0.0008) [2023-12-27 06:26:37,995][20347] Updated weights for policy 0, policy_version 113599 (0.0007) [2023-12-27 06:26:38,516][20381] Updated weights for policy 1, policy_version 113125 (0.0008) [2023-12-27 06:26:38,572][20381] Updated weights for policy 1, policy_version 113135 (0.0008) [2023-12-27 06:26:38,629][20381] Updated weights for policy 1, policy_version 113145 (0.0009) [2023-12-27 06:26:38,670][20347] Updated weights for policy 0, policy_version 113609 (0.0008) [2023-12-27 06:26:38,729][20347] Updated weights for policy 0, policy_version 113619 (0.0008) [2023-12-27 06:26:38,792][20347] Updated weights for policy 0, policy_version 113629 (0.0008) [2023-12-27 06:26:38,845][20347] Updated weights for policy 0, policy_version 113639 (0.0008) [2023-12-27 06:26:39,396][20381] Updated weights for policy 1, policy_version 113155 (0.0008) [2023-12-27 06:26:39,459][20381] Updated weights for policy 1, policy_version 113165 (0.0007) [2023-12-27 06:26:39,511][20347] Updated weights for policy 0, policy_version 113649 (0.0007) [2023-12-27 06:26:39,517][20381] Updated weights for policy 1, policy_version 113175 (0.0006) [2023-12-27 06:26:39,571][20347] Updated weights for policy 0, policy_version 113659 (0.0009) [2023-12-27 06:26:39,637][20347] Updated weights for policy 0, policy_version 113669 (0.0008) [2023-12-27 06:26:40,144][20381] Updated weights for policy 1, policy_version 113185 (0.0007) [2023-12-27 06:26:40,205][20381] Updated weights for policy 1, policy_version 113195 (0.0007) [2023-12-27 06:26:40,264][20381] Updated weights for policy 1, policy_version 113205 (0.0008) [2023-12-27 06:26:40,332][20381] Updated weights for policy 1, policy_version 113215 (0.0007) [2023-12-27 06:26:40,365][20347] Updated weights for policy 0, policy_version 113679 (0.0007) [2023-12-27 06:26:40,426][20347] Updated weights for policy 0, policy_version 113689 (0.0009) [2023-12-27 06:26:40,481][20347] Updated weights for policy 0, policy_version 113699 (0.0007) [2023-12-27 06:26:40,933][20381] Updated weights for policy 1, policy_version 113225 (0.0006) [2023-12-27 06:26:40,983][20381] Updated weights for policy 1, policy_version 113235 (0.0007) [2023-12-27 06:26:41,049][20381] Updated weights for policy 1, policy_version 113245 (0.0008) [2023-12-27 06:26:41,199][20347] Updated weights for policy 0, policy_version 113709 (0.0007) [2023-12-27 06:26:41,269][20347] Updated weights for policy 0, policy_version 113719 (0.0008) [2023-12-27 06:26:41,338][20347] Updated weights for policy 0, policy_version 113729 (0.0008) [2023-12-27 06:26:41,770][20381] Updated weights for policy 1, policy_version 113255 (0.0008) [2023-12-27 06:26:41,832][20381] Updated weights for policy 1, policy_version 113265 (0.0007) [2023-12-27 06:26:41,894][20381] Updated weights for policy 1, policy_version 113275 (0.0008) [2023-12-27 06:26:42,091][20347] Updated weights for policy 0, policy_version 113739 (0.0009) [2023-12-27 06:26:42,149][20347] Updated weights for policy 0, policy_version 113749 (0.0008) [2023-12-27 06:26:42,214][20347] Updated weights for policy 0, policy_version 113759 (0.0008) [2023-12-27 06:26:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 58146816. Throughput: 0: 10179.5, 1: 10399.2. Samples: 58135944. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:42,558][20381] Updated weights for policy 1, policy_version 113285 (0.0008) [2023-12-27 06:26:42,626][20381] Updated weights for policy 1, policy_version 113295 (0.0010) [2023-12-27 06:26:42,683][20381] Updated weights for policy 1, policy_version 113305 (0.0010) [2023-12-27 06:26:42,874][20347] Updated weights for policy 0, policy_version 113769 (0.0007) [2023-12-27 06:26:42,941][20347] Updated weights for policy 0, policy_version 113779 (0.0007) [2023-12-27 06:26:43,004][20347] Updated weights for policy 0, policy_version 113789 (0.0006) [2023-12-27 06:26:43,071][20347] Updated weights for policy 0, policy_version 113799 (0.0007) [2023-12-27 06:26:43,317][20381] Updated weights for policy 1, policy_version 113315 (0.0009) [2023-12-27 06:26:43,370][20381] Updated weights for policy 1, policy_version 113325 (0.0008) [2023-12-27 06:26:43,423][20381] Updated weights for policy 1, policy_version 113335 (0.0008) [2023-12-27 06:26:43,728][20347] Updated weights for policy 0, policy_version 113809 (0.0006) [2023-12-27 06:26:43,796][20347] Updated weights for policy 0, policy_version 113819 (0.0006) [2023-12-27 06:26:43,844][20347] Updated weights for policy 0, policy_version 113829 (0.0010) [2023-12-27 06:26:44,069][20381] Updated weights for policy 1, policy_version 113345 (0.0007) [2023-12-27 06:26:44,127][20381] Updated weights for policy 1, policy_version 113355 (0.0010) [2023-12-27 06:26:44,181][20381] Updated weights for policy 1, policy_version 113365 (0.0010) [2023-12-27 06:26:44,235][20381] Updated weights for policy 1, policy_version 113375 (0.0010) [2023-12-27 06:26:44,468][20347] Updated weights for policy 0, policy_version 113839 (0.0010) [2023-12-27 06:26:44,527][20347] Updated weights for policy 0, policy_version 113849 (0.0009) [2023-12-27 06:26:44,585][20347] Updated weights for policy 0, policy_version 113859 (0.0008) [2023-12-27 06:26:44,909][20381] Updated weights for policy 1, policy_version 113385 (0.0008) [2023-12-27 06:26:44,966][20381] Updated weights for policy 1, policy_version 113395 (0.0006) [2023-12-27 06:26:45,022][20381] Updated weights for policy 1, policy_version 113405 (0.0006) [2023-12-27 06:26:45,256][20347] Updated weights for policy 0, policy_version 113869 (0.0010) [2023-12-27 06:26:45,305][20347] Updated weights for policy 0, policy_version 113879 (0.0010) [2023-12-27 06:26:45,356][20347] Updated weights for policy 0, policy_version 113889 (0.0010) [2023-12-27 06:26:45,656][20381] Updated weights for policy 1, policy_version 113415 (0.0007) [2023-12-27 06:26:45,715][20381] Updated weights for policy 1, policy_version 113425 (0.0007) [2023-12-27 06:26:45,769][20381] Updated weights for policy 1, policy_version 113435 (0.0007) [2023-12-27 06:26:46,060][20347] Updated weights for policy 0, policy_version 113899 (0.0010) [2023-12-27 06:26:46,123][20347] Updated weights for policy 0, policy_version 113909 (0.0010) [2023-12-27 06:26:46,177][20347] Updated weights for policy 0, policy_version 113919 (0.0010) [2023-12-27 06:26:46,402][20381] Updated weights for policy 1, policy_version 113445 (0.0007) [2023-12-27 06:26:46,447][20381] Updated weights for policy 1, policy_version 113455 (0.0007) [2023-12-27 06:26:46,491][20381] Updated weights for policy 1, policy_version 113465 (0.0007) [2023-12-27 06:26:46,822][20347] Updated weights for policy 0, policy_version 113929 (0.0010) [2023-12-27 06:26:46,873][20347] Updated weights for policy 0, policy_version 113939 (0.0010) [2023-12-27 06:26:46,927][20347] Updated weights for policy 0, policy_version 113949 (0.0010) [2023-12-27 06:26:46,979][20347] Updated weights for policy 0, policy_version 113959 (0.0010) [2023-12-27 06:26:47,141][20381] Updated weights for policy 1, policy_version 113475 (0.0007) [2023-12-27 06:26:47,199][20381] Updated weights for policy 1, policy_version 113485 (0.0008) [2023-12-27 06:26:47,251][20381] Updated weights for policy 1, policy_version 113495 (0.0010) [2023-12-27 06:26:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 58253312. Throughput: 0: 10209.0, 1: 10402.7. Samples: 58259508. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:47,673][20347] Updated weights for policy 0, policy_version 113969 (0.0010) [2023-12-27 06:26:47,724][20347] Updated weights for policy 0, policy_version 113979 (0.0010) [2023-12-27 06:26:47,775][20347] Updated weights for policy 0, policy_version 113989 (0.0010) [2023-12-27 06:26:47,945][20381] Updated weights for policy 1, policy_version 113505 (0.0010) [2023-12-27 06:26:47,998][20381] Updated weights for policy 1, policy_version 113515 (0.0009) [2023-12-27 06:26:48,050][20381] Updated weights for policy 1, policy_version 113525 (0.0005) [2023-12-27 06:26:48,103][20381] Updated weights for policy 1, policy_version 113535 (0.0005) [2023-12-27 06:26:48,479][20347] Updated weights for policy 0, policy_version 113999 (0.0010) [2023-12-27 06:26:48,529][20347] Updated weights for policy 0, policy_version 114009 (0.0009) [2023-12-27 06:26:48,587][20347] Updated weights for policy 0, policy_version 114019 (0.0008) [2023-12-27 06:26:48,737][20381] Updated weights for policy 1, policy_version 113545 (0.0008) [2023-12-27 06:26:48,796][20381] Updated weights for policy 1, policy_version 113555 (0.0009) [2023-12-27 06:26:48,846][20381] Updated weights for policy 1, policy_version 113565 (0.0010) [2023-12-27 06:26:49,258][20347] Updated weights for policy 0, policy_version 114029 (0.0007) [2023-12-27 06:26:49,306][20347] Updated weights for policy 0, policy_version 114039 (0.0008) [2023-12-27 06:26:49,359][20347] Updated weights for policy 0, policy_version 114049 (0.0008) [2023-12-27 06:26:49,493][20381] Updated weights for policy 1, policy_version 113575 (0.0009) [2023-12-27 06:26:49,546][20381] Updated weights for policy 1, policy_version 113585 (0.0005) [2023-12-27 06:26:49,599][20381] Updated weights for policy 1, policy_version 113595 (0.0009) [2023-12-27 06:26:50,011][20347] Updated weights for policy 0, policy_version 114059 (0.0006) [2023-12-27 06:26:50,066][20347] Updated weights for policy 0, policy_version 114069 (0.0007) [2023-12-27 06:26:50,115][20347] Updated weights for policy 0, policy_version 114079 (0.0010) [2023-12-27 06:26:50,304][20381] Updated weights for policy 1, policy_version 113605 (0.0011) [2023-12-27 06:26:50,363][20381] Updated weights for policy 1, policy_version 113615 (0.0009) [2023-12-27 06:26:50,420][20381] Updated weights for policy 1, policy_version 113625 (0.0005) [2023-12-27 06:26:50,731][20347] Updated weights for policy 0, policy_version 114089 (0.0010) [2023-12-27 06:26:50,785][20347] Updated weights for policy 0, policy_version 114099 (0.0007) [2023-12-27 06:26:50,837][20347] Updated weights for policy 0, policy_version 114109 (0.0006) [2023-12-27 06:26:50,897][20347] Updated weights for policy 0, policy_version 114119 (0.0005) [2023-12-27 06:26:51,067][20381] Updated weights for policy 1, policy_version 113635 (0.0006) [2023-12-27 06:26:51,135][20381] Updated weights for policy 1, policy_version 113645 (0.0007) [2023-12-27 06:26:51,194][20381] Updated weights for policy 1, policy_version 113655 (0.0011) [2023-12-27 06:26:51,546][20347] Updated weights for policy 0, policy_version 114129 (0.0010) [2023-12-27 06:26:51,608][20347] Updated weights for policy 0, policy_version 114139 (0.0010) [2023-12-27 06:26:51,671][20347] Updated weights for policy 0, policy_version 114149 (0.0007) [2023-12-27 06:26:51,870][20381] Updated weights for policy 1, policy_version 113665 (0.0010) [2023-12-27 06:26:51,932][20381] Updated weights for policy 1, policy_version 113675 (0.0008) [2023-12-27 06:26:51,990][20381] Updated weights for policy 1, policy_version 113685 (0.0006) [2023-12-27 06:26:52,040][20381] Updated weights for policy 1, policy_version 113695 (0.0008) [2023-12-27 06:26:52,298][20347] Updated weights for policy 0, policy_version 114159 (0.0006) [2023-12-27 06:26:52,357][20347] Updated weights for policy 0, policy_version 114169 (0.0007) [2023-12-27 06:26:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 58351616. Throughput: 0: 10200.4, 1: 10374.9. Samples: 58322360. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000113696_29114368.pth... [2023-12-27 06:26:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000112480_28803072.pth [2023-12-27 06:26:52,417][20347] Updated weights for policy 0, policy_version 114179 (0.0007) [2023-12-27 06:26:52,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000114184_29245440.pth... [2023-12-27 06:26:52,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000112968_28934144.pth [2023-12-27 06:26:52,644][20381] Updated weights for policy 1, policy_version 113705 (0.0006) [2023-12-27 06:26:52,699][20381] Updated weights for policy 1, policy_version 113715 (0.0005) [2023-12-27 06:26:52,759][20381] Updated weights for policy 1, policy_version 113725 (0.0006) [2023-12-27 06:26:53,047][20347] Updated weights for policy 0, policy_version 114189 (0.0006) [2023-12-27 06:26:53,094][20347] Updated weights for policy 0, policy_version 114199 (0.0005) [2023-12-27 06:26:53,139][20347] Updated weights for policy 0, policy_version 114209 (0.0005) [2023-12-27 06:26:53,376][20381] Updated weights for policy 1, policy_version 113735 (0.0007) [2023-12-27 06:26:53,425][20381] Updated weights for policy 1, policy_version 113745 (0.0010) [2023-12-27 06:26:53,474][20381] Updated weights for policy 1, policy_version 113755 (0.0008) [2023-12-27 06:26:53,735][20347] Updated weights for policy 0, policy_version 114219 (0.0005) [2023-12-27 06:26:53,793][20347] Updated weights for policy 0, policy_version 114229 (0.0007) [2023-12-27 06:26:53,850][20347] Updated weights for policy 0, policy_version 114239 (0.0008) [2023-12-27 06:26:54,135][20381] Updated weights for policy 1, policy_version 113765 (0.0008) [2023-12-27 06:26:54,189][20381] Updated weights for policy 1, policy_version 113775 (0.0008) [2023-12-27 06:26:54,241][20381] Updated weights for policy 1, policy_version 113785 (0.0007) [2023-12-27 06:26:54,566][20347] Updated weights for policy 0, policy_version 114249 (0.0007) [2023-12-27 06:26:54,632][20347] Updated weights for policy 0, policy_version 114259 (0.0007) [2023-12-27 06:26:54,688][20347] Updated weights for policy 0, policy_version 114269 (0.0007) [2023-12-27 06:26:54,741][20347] Updated weights for policy 0, policy_version 114279 (0.0007) [2023-12-27 06:26:54,900][20381] Updated weights for policy 1, policy_version 113795 (0.0007) [2023-12-27 06:26:54,952][20381] Updated weights for policy 1, policy_version 113805 (0.0008) [2023-12-27 06:26:55,016][20381] Updated weights for policy 1, policy_version 113815 (0.0008) [2023-12-27 06:26:55,426][20347] Updated weights for policy 0, policy_version 114289 (0.0008) [2023-12-27 06:26:55,482][20347] Updated weights for policy 0, policy_version 114299 (0.0008) [2023-12-27 06:26:55,536][20347] Updated weights for policy 0, policy_version 114309 (0.0007) [2023-12-27 06:26:55,675][20381] Updated weights for policy 1, policy_version 113825 (0.0007) [2023-12-27 06:26:55,727][20381] Updated weights for policy 1, policy_version 113835 (0.0005) [2023-12-27 06:26:55,788][20381] Updated weights for policy 1, policy_version 113845 (0.0007) [2023-12-27 06:26:55,852][20381] Updated weights for policy 1, policy_version 113855 (0.0007) [2023-12-27 06:26:56,309][20347] Updated weights for policy 0, policy_version 114319 (0.0009) [2023-12-27 06:26:56,361][20347] Updated weights for policy 0, policy_version 114329 (0.0010) [2023-12-27 06:26:56,409][20347] Updated weights for policy 0, policy_version 114339 (0.0006) [2023-12-27 06:26:56,472][20381] Updated weights for policy 1, policy_version 113865 (0.0007) [2023-12-27 06:26:56,540][20381] Updated weights for policy 1, policy_version 113875 (0.0007) [2023-12-27 06:26:56,589][20381] Updated weights for policy 1, policy_version 113885 (0.0007) [2023-12-27 06:26:57,082][20347] Updated weights for policy 0, policy_version 114349 (0.0006) [2023-12-27 06:26:57,147][20347] Updated weights for policy 0, policy_version 114359 (0.0005) [2023-12-27 06:26:57,196][20347] Updated weights for policy 0, policy_version 114369 (0.0007) [2023-12-27 06:26:57,263][20381] Updated weights for policy 1, policy_version 113895 (0.0006) [2023-12-27 06:26:57,319][20381] Updated weights for policy 1, policy_version 113905 (0.0006) [2023-12-27 06:26:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 58458112. Throughput: 0: 10276.3, 1: 10434.5. Samples: 58449420. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:26:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:26:57,375][20381] Updated weights for policy 1, policy_version 113915 (0.0006) [2023-12-27 06:26:57,795][20347] Updated weights for policy 0, policy_version 114379 (0.0006) [2023-12-27 06:26:57,849][20347] Updated weights for policy 0, policy_version 114389 (0.0006) [2023-12-27 06:26:57,904][20347] Updated weights for policy 0, policy_version 114399 (0.0005) [2023-12-27 06:26:58,052][20381] Updated weights for policy 1, policy_version 113925 (0.0008) [2023-12-27 06:26:58,100][20381] Updated weights for policy 1, policy_version 113935 (0.0008) [2023-12-27 06:26:58,157][20381] Updated weights for policy 1, policy_version 113945 (0.0009) [2023-12-27 06:26:58,526][20347] Updated weights for policy 0, policy_version 114409 (0.0005) [2023-12-27 06:26:58,593][20347] Updated weights for policy 0, policy_version 114419 (0.0009) [2023-12-27 06:26:58,650][20347] Updated weights for policy 0, policy_version 114429 (0.0007) [2023-12-27 06:26:58,715][20347] Updated weights for policy 0, policy_version 114439 (0.0008) [2023-12-27 06:26:58,911][20381] Updated weights for policy 1, policy_version 113955 (0.0008) [2023-12-27 06:26:58,972][20381] Updated weights for policy 1, policy_version 113965 (0.0008) [2023-12-27 06:26:59,034][20381] Updated weights for policy 1, policy_version 113975 (0.0007) [2023-12-27 06:26:59,459][20347] Updated weights for policy 0, policy_version 114449 (0.0009) [2023-12-27 06:26:59,507][20347] Updated weights for policy 0, policy_version 114459 (0.0010) [2023-12-27 06:26:59,561][20347] Updated weights for policy 0, policy_version 114469 (0.0010) [2023-12-27 06:26:59,770][20381] Updated weights for policy 1, policy_version 113985 (0.0007) [2023-12-27 06:26:59,830][20381] Updated weights for policy 1, policy_version 113995 (0.0005) [2023-12-27 06:26:59,885][20381] Updated weights for policy 1, policy_version 114005 (0.0006) [2023-12-27 06:26:59,945][20381] Updated weights for policy 1, policy_version 114015 (0.0008) [2023-12-27 06:27:00,212][20347] Updated weights for policy 0, policy_version 114479 (0.0007) [2023-12-27 06:27:00,258][20347] Updated weights for policy 0, policy_version 114489 (0.0006) [2023-12-27 06:27:00,312][20347] Updated weights for policy 0, policy_version 114499 (0.0008) [2023-12-27 06:27:00,541][20381] Updated weights for policy 1, policy_version 114025 (0.0007) [2023-12-27 06:27:00,585][20381] Updated weights for policy 1, policy_version 114035 (0.0008) [2023-12-27 06:27:00,638][20381] Updated weights for policy 1, policy_version 114045 (0.0007) [2023-12-27 06:27:00,983][20347] Updated weights for policy 0, policy_version 114509 (0.0007) [2023-12-27 06:27:01,044][20347] Updated weights for policy 0, policy_version 114519 (0.0006) [2023-12-27 06:27:01,104][20347] Updated weights for policy 0, policy_version 114529 (0.0007) [2023-12-27 06:27:01,361][20381] Updated weights for policy 1, policy_version 114055 (0.0008) [2023-12-27 06:27:01,417][20381] Updated weights for policy 1, policy_version 114065 (0.0008) [2023-12-27 06:27:01,465][20381] Updated weights for policy 1, policy_version 114075 (0.0008) [2023-12-27 06:27:01,801][20347] Updated weights for policy 0, policy_version 114539 (0.0007) [2023-12-27 06:27:01,864][20347] Updated weights for policy 0, policy_version 114549 (0.0007) [2023-12-27 06:27:01,931][20347] Updated weights for policy 0, policy_version 114559 (0.0005) [2023-12-27 06:27:02,147][20381] Updated weights for policy 1, policy_version 114085 (0.0008) [2023-12-27 06:27:02,201][20381] Updated weights for policy 1, policy_version 114095 (0.0008) [2023-12-27 06:27:02,264][20381] Updated weights for policy 1, policy_version 114105 (0.0009) [2023-12-27 06:27:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 58564608. Throughput: 0: 10285.1, 1: 10403.4. Samples: 58572824. Policy #0 lag: (min: 5.0, avg: 30.1, max: 32.0) [2023-12-27 06:27:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:02,577][20347] Updated weights for policy 0, policy_version 114569 (0.0006) [2023-12-27 06:27:02,638][20347] Updated weights for policy 0, policy_version 114579 (0.0008) [2023-12-27 06:27:02,688][20347] Updated weights for policy 0, policy_version 114589 (0.0009) [2023-12-27 06:27:02,752][20347] Updated weights for policy 0, policy_version 114599 (0.0008) [2023-12-27 06:27:02,869][20381] Updated weights for policy 1, policy_version 114115 (0.0007) [2023-12-27 06:27:02,937][20381] Updated weights for policy 1, policy_version 114125 (0.0005) [2023-12-27 06:27:02,996][20381] Updated weights for policy 1, policy_version 114135 (0.0005) [2023-12-27 06:27:03,439][20347] Updated weights for policy 0, policy_version 114609 (0.0008) [2023-12-27 06:27:03,486][20347] Updated weights for policy 0, policy_version 114619 (0.0008) [2023-12-27 06:27:03,538][20347] Updated weights for policy 0, policy_version 114629 (0.0007) [2023-12-27 06:27:03,616][20381] Updated weights for policy 1, policy_version 114145 (0.0006) [2023-12-27 06:27:03,668][20381] Updated weights for policy 1, policy_version 114155 (0.0005) [2023-12-27 06:27:03,719][20381] Updated weights for policy 1, policy_version 114165 (0.0005) [2023-12-27 06:27:03,775][20381] Updated weights for policy 1, policy_version 114175 (0.0008) [2023-12-27 06:27:04,278][20347] Updated weights for policy 0, policy_version 114639 (0.0008) [2023-12-27 06:27:04,356][20347] Updated weights for policy 0, policy_version 114649 (0.0008) [2023-12-27 06:27:04,413][20347] Updated weights for policy 0, policy_version 114659 (0.0008) [2023-12-27 06:27:04,437][20381] Updated weights for policy 1, policy_version 114185 (0.0005) [2023-12-27 06:27:04,486][20381] Updated weights for policy 1, policy_version 114195 (0.0007) [2023-12-27 06:27:04,536][20381] Updated weights for policy 1, policy_version 114205 (0.0008) [2023-12-27 06:27:05,089][20347] Updated weights for policy 0, policy_version 114669 (0.0008) [2023-12-27 06:27:05,147][20347] Updated weights for policy 0, policy_version 114679 (0.0007) [2023-12-27 06:27:05,202][20381] Updated weights for policy 1, policy_version 114215 (0.0007) [2023-12-27 06:27:05,204][20347] Updated weights for policy 0, policy_version 114689 (0.0006) [2023-12-27 06:27:05,255][20381] Updated weights for policy 1, policy_version 114225 (0.0005) [2023-12-27 06:27:05,305][20381] Updated weights for policy 1, policy_version 114235 (0.0006) [2023-12-27 06:27:05,911][20347] Updated weights for policy 0, policy_version 114699 (0.0007) [2023-12-27 06:27:05,971][20347] Updated weights for policy 0, policy_version 114709 (0.0006) [2023-12-27 06:27:05,974][20381] Updated weights for policy 1, policy_version 114245 (0.0009) [2023-12-27 06:27:06,018][20381] Updated weights for policy 1, policy_version 114255 (0.0010) [2023-12-27 06:27:06,026][20347] Updated weights for policy 0, policy_version 114719 (0.0005) [2023-12-27 06:27:06,066][20381] Updated weights for policy 1, policy_version 114265 (0.0010) [2023-12-27 06:27:06,683][20347] Updated weights for policy 0, policy_version 114729 (0.0005) [2023-12-27 06:27:06,733][20347] Updated weights for policy 0, policy_version 114739 (0.0007) [2023-12-27 06:27:06,781][20347] Updated weights for policy 0, policy_version 114749 (0.0007) [2023-12-27 06:27:06,783][20381] Updated weights for policy 1, policy_version 114275 (0.0011) [2023-12-27 06:27:06,830][20347] Updated weights for policy 0, policy_version 114759 (0.0005) [2023-12-27 06:27:06,832][20381] Updated weights for policy 1, policy_version 114285 (0.0011) [2023-12-27 06:27:06,887][20381] Updated weights for policy 1, policy_version 114295 (0.0010) [2023-12-27 06:27:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 58662912. Throughput: 0: 10288.6, 1: 10383.2. Samples: 58634764. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000114304_29270016.pth... [2023-12-27 06:27:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000114760_29392896.pth... [2023-12-27 06:27:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000113088_28958720.pth [2023-12-27 06:27:07,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000113576_29089792.pth [2023-12-27 06:27:07,548][20347] Updated weights for policy 0, policy_version 114769 (0.0007) [2023-12-27 06:27:07,596][20381] Updated weights for policy 1, policy_version 114305 (0.0011) [2023-12-27 06:27:07,599][20347] Updated weights for policy 0, policy_version 114779 (0.0007) [2023-12-27 06:27:07,652][20381] Updated weights for policy 1, policy_version 114315 (0.0009) [2023-12-27 06:27:07,655][20347] Updated weights for policy 0, policy_version 114789 (0.0008) [2023-12-27 06:27:07,705][20381] Updated weights for policy 1, policy_version 114325 (0.0008) [2023-12-27 06:27:07,756][20381] Updated weights for policy 1, policy_version 114335 (0.0005) [2023-12-27 06:27:08,258][20347] Updated weights for policy 0, policy_version 114799 (0.0010) [2023-12-27 06:27:08,306][20347] Updated weights for policy 0, policy_version 114809 (0.0010) [2023-12-27 06:27:08,310][20381] Updated weights for policy 1, policy_version 114345 (0.0010) [2023-12-27 06:27:08,366][20347] Updated weights for policy 0, policy_version 114819 (0.0011) [2023-12-27 06:27:08,376][20381] Updated weights for policy 1, policy_version 114355 (0.0011) [2023-12-27 06:27:08,437][20381] Updated weights for policy 1, policy_version 114365 (0.0011) [2023-12-27 06:27:09,034][20347] Updated weights for policy 0, policy_version 114829 (0.0008) [2023-12-27 06:27:09,087][20347] Updated weights for policy 0, policy_version 114839 (0.0010) [2023-12-27 06:27:09,112][20381] Updated weights for policy 1, policy_version 114376 (0.0011) [2023-12-27 06:27:09,146][20347] Updated weights for policy 0, policy_version 114849 (0.0010) [2023-12-27 06:27:09,170][20381] Updated weights for policy 1, policy_version 114386 (0.0007) [2023-12-27 06:27:09,229][20381] Updated weights for policy 1, policy_version 114396 (0.0009) [2023-12-27 06:27:09,866][20347] Updated weights for policy 0, policy_version 114859 (0.0007) [2023-12-27 06:27:09,924][20381] Updated weights for policy 1, policy_version 114406 (0.0010) [2023-12-27 06:27:09,930][20347] Updated weights for policy 0, policy_version 114869 (0.0007) [2023-12-27 06:27:09,989][20347] Updated weights for policy 0, policy_version 114879 (0.0007) [2023-12-27 06:27:09,991][20381] Updated weights for policy 1, policy_version 114416 (0.0007) [2023-12-27 06:27:10,055][20381] Updated weights for policy 1, policy_version 114426 (0.0008) [2023-12-27 06:27:10,609][20381] Updated weights for policy 1, policy_version 114436 (0.0006) [2023-12-27 06:27:10,673][20381] Updated weights for policy 1, policy_version 114446 (0.0010) [2023-12-27 06:27:10,678][20347] Updated weights for policy 0, policy_version 114889 (0.0007) [2023-12-27 06:27:10,732][20381] Updated weights for policy 1, policy_version 114456 (0.0010) [2023-12-27 06:27:10,738][20347] Updated weights for policy 0, policy_version 114899 (0.0006) [2023-12-27 06:27:10,801][20347] Updated weights for policy 0, policy_version 114909 (0.0006) [2023-12-27 06:27:10,865][20347] Updated weights for policy 0, policy_version 114919 (0.0006) [2023-12-27 06:27:11,419][20381] Updated weights for policy 1, policy_version 114466 (0.0010) [2023-12-27 06:27:11,475][20381] Updated weights for policy 1, policy_version 114476 (0.0008) [2023-12-27 06:27:11,532][20381] Updated weights for policy 1, policy_version 114486 (0.0008) [2023-12-27 06:27:11,539][20347] Updated weights for policy 0, policy_version 114929 (0.0007) [2023-12-27 06:27:11,591][20347] Updated weights for policy 0, policy_version 114939 (0.0007) [2023-12-27 06:27:11,592][20381] Updated weights for policy 1, policy_version 114496 (0.0008) [2023-12-27 06:27:11,654][20347] Updated weights for policy 0, policy_version 114949 (0.0008) [2023-12-27 06:27:12,327][20347] Updated weights for policy 0, policy_version 114959 (0.0006) [2023-12-27 06:27:12,345][20381] Updated weights for policy 1, policy_version 114506 (0.0008) [2023-12-27 06:27:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 58761216. Throughput: 0: 10263.7, 1: 10455.6. Samples: 58759156. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:12,384][20347] Updated weights for policy 0, policy_version 114969 (0.0007) [2023-12-27 06:27:12,405][20381] Updated weights for policy 1, policy_version 114516 (0.0008) [2023-12-27 06:27:12,436][20347] Updated weights for policy 0, policy_version 114979 (0.0007) [2023-12-27 06:27:12,461][20381] Updated weights for policy 1, policy_version 114526 (0.0007) [2023-12-27 06:27:13,096][20347] Updated weights for policy 0, policy_version 114989 (0.0007) [2023-12-27 06:27:13,165][20347] Updated weights for policy 0, policy_version 114999 (0.0008) [2023-12-27 06:27:13,189][20381] Updated weights for policy 1, policy_version 114536 (0.0007) [2023-12-27 06:27:13,220][20347] Updated weights for policy 0, policy_version 115009 (0.0007) [2023-12-27 06:27:13,248][20381] Updated weights for policy 1, policy_version 114546 (0.0007) [2023-12-27 06:27:13,296][20381] Updated weights for policy 1, policy_version 114556 (0.0008) [2023-12-27 06:27:13,806][20347] Updated weights for policy 0, policy_version 115019 (0.0007) [2023-12-27 06:27:13,858][20347] Updated weights for policy 0, policy_version 115029 (0.0008) [2023-12-27 06:27:13,919][20347] Updated weights for policy 0, policy_version 115039 (0.0008) [2023-12-27 06:27:13,967][20381] Updated weights for policy 1, policy_version 114566 (0.0006) [2023-12-27 06:27:14,018][20381] Updated weights for policy 1, policy_version 114576 (0.0005) [2023-12-27 06:27:14,066][20381] Updated weights for policy 1, policy_version 114586 (0.0005) [2023-12-27 06:27:14,608][20347] Updated weights for policy 0, policy_version 115049 (0.0007) [2023-12-27 06:27:14,662][20347] Updated weights for policy 0, policy_version 115059 (0.0008) [2023-12-27 06:27:14,665][20381] Updated weights for policy 1, policy_version 114596 (0.0008) [2023-12-27 06:27:14,721][20347] Updated weights for policy 0, policy_version 115069 (0.0006) [2023-12-27 06:27:14,723][20381] Updated weights for policy 1, policy_version 114606 (0.0010) [2023-12-27 06:27:14,776][20347] Updated weights for policy 0, policy_version 115079 (0.0007) [2023-12-27 06:27:14,789][20381] Updated weights for policy 1, policy_version 114616 (0.0007) [2023-12-27 06:27:15,428][20347] Updated weights for policy 0, policy_version 115089 (0.0006) [2023-12-27 06:27:15,478][20347] Updated weights for policy 0, policy_version 115099 (0.0006) [2023-12-27 06:27:15,479][20381] Updated weights for policy 1, policy_version 114626 (0.0011) [2023-12-27 06:27:15,539][20347] Updated weights for policy 0, policy_version 115109 (0.0007) [2023-12-27 06:27:15,541][20381] Updated weights for policy 1, policy_version 114636 (0.0011) [2023-12-27 06:27:15,603][20381] Updated weights for policy 1, policy_version 114646 (0.0011) [2023-12-27 06:27:15,664][20381] Updated weights for policy 1, policy_version 114656 (0.0010) [2023-12-27 06:27:16,077][20347] Updated weights for policy 0, policy_version 115119 (0.0006) [2023-12-27 06:27:16,135][20347] Updated weights for policy 0, policy_version 115129 (0.0005) [2023-12-27 06:27:16,186][20347] Updated weights for policy 0, policy_version 115139 (0.0005) [2023-12-27 06:27:16,360][20381] Updated weights for policy 1, policy_version 114666 (0.0009) [2023-12-27 06:27:16,417][20381] Updated weights for policy 1, policy_version 114676 (0.0009) [2023-12-27 06:27:16,473][20381] Updated weights for policy 1, policy_version 114686 (0.0008) [2023-12-27 06:27:16,765][20347] Updated weights for policy 0, policy_version 115149 (0.0005) [2023-12-27 06:27:16,815][20347] Updated weights for policy 0, policy_version 115159 (0.0005) [2023-12-27 06:27:16,868][20347] Updated weights for policy 0, policy_version 115169 (0.0006) [2023-12-27 06:27:17,057][20381] Updated weights for policy 1, policy_version 114696 (0.0006) [2023-12-27 06:27:17,109][20381] Updated weights for policy 1, policy_version 114706 (0.0006) [2023-12-27 06:27:17,167][20381] Updated weights for policy 1, policy_version 114716 (0.0006) [2023-12-27 06:27:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 58875904. Throughput: 0: 10368.6, 1: 10385.5. Samples: 58885052. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:17,581][20347] Updated weights for policy 0, policy_version 115179 (0.0008) [2023-12-27 06:27:17,638][20347] Updated weights for policy 0, policy_version 115189 (0.0007) [2023-12-27 06:27:17,683][20347] Updated weights for policy 0, policy_version 115199 (0.0007) [2023-12-27 06:27:17,733][20381] Updated weights for policy 1, policy_version 114726 (0.0006) [2023-12-27 06:27:17,798][20381] Updated weights for policy 1, policy_version 114736 (0.0005) [2023-12-27 06:27:17,862][20381] Updated weights for policy 1, policy_version 114746 (0.0005) [2023-12-27 06:27:18,370][20347] Updated weights for policy 0, policy_version 115209 (0.0008) [2023-12-27 06:27:18,421][20347] Updated weights for policy 0, policy_version 115219 (0.0008) [2023-12-27 06:27:18,479][20347] Updated weights for policy 0, policy_version 115229 (0.0008) [2023-12-27 06:27:18,480][20381] Updated weights for policy 1, policy_version 114757 (0.0007) [2023-12-27 06:27:18,531][20347] Updated weights for policy 0, policy_version 115239 (0.0009) [2023-12-27 06:27:18,541][20381] Updated weights for policy 1, policy_version 114767 (0.0005) [2023-12-27 06:27:18,602][20381] Updated weights for policy 1, policy_version 114777 (0.0005) [2023-12-27 06:27:19,168][20381] Updated weights for policy 1, policy_version 114787 (0.0006) [2023-12-27 06:27:19,182][20347] Updated weights for policy 0, policy_version 115249 (0.0006) [2023-12-27 06:27:19,227][20381] Updated weights for policy 1, policy_version 114797 (0.0011) [2023-12-27 06:27:19,242][20347] Updated weights for policy 0, policy_version 115259 (0.0007) [2023-12-27 06:27:19,285][20381] Updated weights for policy 1, policy_version 114807 (0.0010) [2023-12-27 06:27:19,296][20347] Updated weights for policy 0, policy_version 115269 (0.0010) [2023-12-27 06:27:19,902][20347] Updated weights for policy 0, policy_version 115279 (0.0010) [2023-12-27 06:27:19,956][20347] Updated weights for policy 0, policy_version 115289 (0.0010) [2023-12-27 06:27:19,982][20381] Updated weights for policy 1, policy_version 114817 (0.0010) [2023-12-27 06:27:20,010][20347] Updated weights for policy 0, policy_version 115299 (0.0006) [2023-12-27 06:27:20,038][20381] Updated weights for policy 1, policy_version 114827 (0.0008) [2023-12-27 06:27:20,096][20381] Updated weights for policy 1, policy_version 114837 (0.0005) [2023-12-27 06:27:20,149][20381] Updated weights for policy 1, policy_version 114847 (0.0007) [2023-12-27 06:27:20,735][20347] Updated weights for policy 0, policy_version 115309 (0.0007) [2023-12-27 06:27:20,791][20381] Updated weights for policy 1, policy_version 114857 (0.0010) [2023-12-27 06:27:20,800][20347] Updated weights for policy 0, policy_version 115319 (0.0006) [2023-12-27 06:27:20,848][20381] Updated weights for policy 1, policy_version 114867 (0.0011) [2023-12-27 06:27:20,853][20347] Updated weights for policy 0, policy_version 115329 (0.0005) [2023-12-27 06:27:20,901][20381] Updated weights for policy 1, policy_version 114877 (0.0011) [2023-12-27 06:27:21,585][20347] Updated weights for policy 0, policy_version 115339 (0.0006) [2023-12-27 06:27:21,615][20381] Updated weights for policy 1, policy_version 114887 (0.0011) [2023-12-27 06:27:21,646][20347] Updated weights for policy 0, policy_version 115349 (0.0007) [2023-12-27 06:27:21,685][20381] Updated weights for policy 1, policy_version 114897 (0.0010) [2023-12-27 06:27:21,709][20347] Updated weights for policy 0, policy_version 115359 (0.0006) [2023-12-27 06:27:21,747][20381] Updated weights for policy 1, policy_version 114907 (0.0011) [2023-12-27 06:27:22,362][20381] Updated weights for policy 1, policy_version 114917 (0.0008) [2023-12-27 06:27:22,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 58974208. Throughput: 0: 10350.1, 1: 10453.8. Samples: 58949292. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000115368_29548544.pth... [2023-12-27 06:27:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000114184_29245440.pth [2023-12-27 06:27:22,418][20381] Updated weights for policy 1, policy_version 114927 (0.0007) [2023-12-27 06:27:22,457][20347] Updated weights for policy 0, policy_version 115369 (0.0006) [2023-12-27 06:27:22,471][20381] Updated weights for policy 1, policy_version 114937 (0.0006) [2023-12-27 06:27:22,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000114944_29433856.pth... [2023-12-27 06:27:22,515][20347] Updated weights for policy 0, policy_version 115379 (0.0006) [2023-12-27 06:27:22,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000113696_29114368.pth [2023-12-27 06:27:22,570][20347] Updated weights for policy 0, policy_version 115389 (0.0007) [2023-12-27 06:27:22,633][20347] Updated weights for policy 0, policy_version 115399 (0.0008) [2023-12-27 06:27:23,167][20381] Updated weights for policy 1, policy_version 114947 (0.0007) [2023-12-27 06:27:23,216][20381] Updated weights for policy 1, policy_version 114957 (0.0009) [2023-12-27 06:27:23,260][20381] Updated weights for policy 1, policy_version 114967 (0.0008) [2023-12-27 06:27:23,294][20347] Updated weights for policy 0, policy_version 115409 (0.0005) [2023-12-27 06:27:23,346][20347] Updated weights for policy 0, policy_version 115419 (0.0005) [2023-12-27 06:27:23,394][20347] Updated weights for policy 0, policy_version 115429 (0.0005) [2023-12-27 06:27:23,988][20381] Updated weights for policy 1, policy_version 114977 (0.0008) [2023-12-27 06:27:24,016][20347] Updated weights for policy 0, policy_version 115439 (0.0005) [2023-12-27 06:27:24,040][20381] Updated weights for policy 1, policy_version 114987 (0.0006) [2023-12-27 06:27:24,071][20347] Updated weights for policy 0, policy_version 115449 (0.0006) [2023-12-27 06:27:24,088][20381] Updated weights for policy 1, policy_version 114997 (0.0005) [2023-12-27 06:27:24,123][20347] Updated weights for policy 0, policy_version 115459 (0.0005) [2023-12-27 06:27:24,143][20381] Updated weights for policy 1, policy_version 115007 (0.0007) [2023-12-27 06:27:24,771][20347] Updated weights for policy 0, policy_version 115469 (0.0006) [2023-12-27 06:27:24,823][20347] Updated weights for policy 0, policy_version 115479 (0.0005) [2023-12-27 06:27:24,853][20381] Updated weights for policy 1, policy_version 115017 (0.0008) [2023-12-27 06:27:24,870][20347] Updated weights for policy 0, policy_version 115489 (0.0005) [2023-12-27 06:27:24,905][20381] Updated weights for policy 1, policy_version 115027 (0.0007) [2023-12-27 06:27:24,953][20381] Updated weights for policy 1, policy_version 115037 (0.0007) [2023-12-27 06:27:25,562][20347] Updated weights for policy 0, policy_version 115499 (0.0005) [2023-12-27 06:27:25,582][20381] Updated weights for policy 1, policy_version 115047 (0.0006) [2023-12-27 06:27:25,618][20347] Updated weights for policy 0, policy_version 115509 (0.0005) [2023-12-27 06:27:25,641][20381] Updated weights for policy 1, policy_version 115057 (0.0006) [2023-12-27 06:27:25,679][20347] Updated weights for policy 0, policy_version 115519 (0.0005) [2023-12-27 06:27:25,703][20381] Updated weights for policy 1, policy_version 115067 (0.0007) [2023-12-27 06:27:26,295][20347] Updated weights for policy 0, policy_version 115529 (0.0005) [2023-12-27 06:27:26,352][20347] Updated weights for policy 0, policy_version 115539 (0.0008) [2023-12-27 06:27:26,386][20381] Updated weights for policy 1, policy_version 115077 (0.0007) [2023-12-27 06:27:26,407][20347] Updated weights for policy 0, policy_version 115549 (0.0007) [2023-12-27 06:27:26,435][20381] Updated weights for policy 1, policy_version 115087 (0.0005) [2023-12-27 06:27:26,465][20347] Updated weights for policy 0, policy_version 115559 (0.0007) [2023-12-27 06:27:26,486][20381] Updated weights for policy 1, policy_version 115097 (0.0006) [2023-12-27 06:27:27,092][20347] Updated weights for policy 0, policy_version 115569 (0.0007) [2023-12-27 06:27:27,142][20381] Updated weights for policy 1, policy_version 115107 (0.0009) [2023-12-27 06:27:27,146][20347] Updated weights for policy 0, policy_version 115579 (0.0007) [2023-12-27 06:27:27,200][20381] Updated weights for policy 1, policy_version 115117 (0.0011) [2023-12-27 06:27:27,202][20347] Updated weights for policy 0, policy_version 115589 (0.0005) [2023-12-27 06:27:27,259][20381] Updated weights for policy 1, policy_version 115127 (0.0010) [2023-12-27 06:27:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 59088896. Throughput: 0: 10324.5, 1: 10474.8. Samples: 59071916. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:27,878][20347] Updated weights for policy 0, policy_version 115599 (0.0007) [2023-12-27 06:27:27,895][20381] Updated weights for policy 1, policy_version 115137 (0.0010) [2023-12-27 06:27:27,919][20347] Updated weights for policy 0, policy_version 115609 (0.0007) [2023-12-27 06:27:27,950][20381] Updated weights for policy 1, policy_version 115147 (0.0005) [2023-12-27 06:27:27,968][20347] Updated weights for policy 0, policy_version 115619 (0.0007) [2023-12-27 06:27:27,996][20381] Updated weights for policy 1, policy_version 115157 (0.0005) [2023-12-27 06:27:28,045][20381] Updated weights for policy 1, policy_version 115167 (0.0005) [2023-12-27 06:27:28,650][20381] Updated weights for policy 1, policy_version 115177 (0.0008) [2023-12-27 06:27:28,706][20347] Updated weights for policy 0, policy_version 115629 (0.0007) [2023-12-27 06:27:28,708][20381] Updated weights for policy 1, policy_version 115187 (0.0006) [2023-12-27 06:27:28,761][20381] Updated weights for policy 1, policy_version 115197 (0.0005) [2023-12-27 06:27:28,762][20347] Updated weights for policy 0, policy_version 115639 (0.0007) [2023-12-27 06:27:28,820][20347] Updated weights for policy 0, policy_version 115649 (0.0007) [2023-12-27 06:27:29,395][20347] Updated weights for policy 0, policy_version 115659 (0.0007) [2023-12-27 06:27:29,455][20347] Updated weights for policy 0, policy_version 115669 (0.0007) [2023-12-27 06:27:29,492][20381] Updated weights for policy 1, policy_version 115207 (0.0007) [2023-12-27 06:27:29,509][20347] Updated weights for policy 0, policy_version 115679 (0.0009) [2023-12-27 06:27:29,544][20381] Updated weights for policy 1, policy_version 115217 (0.0006) [2023-12-27 06:27:29,594][20381] Updated weights for policy 1, policy_version 115227 (0.0005) [2023-12-27 06:27:30,168][20347] Updated weights for policy 0, policy_version 115689 (0.0010) [2023-12-27 06:27:30,222][20347] Updated weights for policy 0, policy_version 115699 (0.0009) [2023-12-27 06:27:30,257][20381] Updated weights for policy 1, policy_version 115237 (0.0008) [2023-12-27 06:27:30,277][20347] Updated weights for policy 0, policy_version 115709 (0.0010) [2023-12-27 06:27:30,309][20381] Updated weights for policy 1, policy_version 115247 (0.0010) [2023-12-27 06:27:30,333][20347] Updated weights for policy 0, policy_version 115719 (0.0010) [2023-12-27 06:27:30,357][20381] Updated weights for policy 1, policy_version 115257 (0.0010) [2023-12-27 06:27:30,944][20381] Updated weights for policy 1, policy_version 115267 (0.0010) [2023-12-27 06:27:30,975][20347] Updated weights for policy 0, policy_version 115729 (0.0006) [2023-12-27 06:27:30,992][20381] Updated weights for policy 1, policy_version 115277 (0.0010) [2023-12-27 06:27:31,031][20347] Updated weights for policy 0, policy_version 115739 (0.0006) [2023-12-27 06:27:31,048][20381] Updated weights for policy 1, policy_version 115287 (0.0010) [2023-12-27 06:27:31,079][20347] Updated weights for policy 0, policy_version 115749 (0.0007) [2023-12-27 06:27:31,675][20381] Updated weights for policy 1, policy_version 115297 (0.0010) [2023-12-27 06:27:31,729][20347] Updated weights for policy 0, policy_version 115759 (0.0006) [2023-12-27 06:27:31,730][20381] Updated weights for policy 1, policy_version 115307 (0.0008) [2023-12-27 06:27:31,784][20347] Updated weights for policy 0, policy_version 115769 (0.0006) [2023-12-27 06:27:31,786][20381] Updated weights for policy 1, policy_version 115317 (0.0006) [2023-12-27 06:27:31,840][20381] Updated weights for policy 1, policy_version 115327 (0.0005) [2023-12-27 06:27:31,842][20347] Updated weights for policy 0, policy_version 115779 (0.0007) [2023-12-27 06:27:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 59187200. Throughput: 0: 10394.6, 1: 10503.1. Samples: 59199900. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:32,475][20381] Updated weights for policy 1, policy_version 115337 (0.0008) [2023-12-27 06:27:32,521][20347] Updated weights for policy 0, policy_version 115789 (0.0006) [2023-12-27 06:27:32,527][20381] Updated weights for policy 1, policy_version 115347 (0.0006) [2023-12-27 06:27:32,578][20347] Updated weights for policy 0, policy_version 115799 (0.0006) [2023-12-27 06:27:32,580][20381] Updated weights for policy 1, policy_version 115357 (0.0006) [2023-12-27 06:27:32,644][20347] Updated weights for policy 0, policy_version 115809 (0.0008) [2023-12-27 06:27:33,166][20381] Updated weights for policy 1, policy_version 115367 (0.0005) [2023-12-27 06:27:33,209][20381] Updated weights for policy 1, policy_version 115377 (0.0005) [2023-12-27 06:27:33,259][20381] Updated weights for policy 1, policy_version 115387 (0.0005) [2023-12-27 06:27:33,328][20347] Updated weights for policy 0, policy_version 115819 (0.0007) [2023-12-27 06:27:33,381][20347] Updated weights for policy 0, policy_version 115829 (0.0005) [2023-12-27 06:27:33,440][20347] Updated weights for policy 0, policy_version 115839 (0.0005) [2023-12-27 06:27:33,848][20381] Updated weights for policy 1, policy_version 115397 (0.0007) [2023-12-27 06:27:33,900][20381] Updated weights for policy 1, policy_version 115407 (0.0005) [2023-12-27 06:27:33,956][20381] Updated weights for policy 1, policy_version 115417 (0.0006) [2023-12-27 06:27:34,000][20347] Updated weights for policy 0, policy_version 115849 (0.0006) [2023-12-27 06:27:34,056][20347] Updated weights for policy 0, policy_version 115859 (0.0010) [2023-12-27 06:27:34,113][20347] Updated weights for policy 0, policy_version 115869 (0.0008) [2023-12-27 06:27:34,174][20347] Updated weights for policy 0, policy_version 115879 (0.0010) [2023-12-27 06:27:34,639][20381] Updated weights for policy 1, policy_version 115427 (0.0010) [2023-12-27 06:27:34,692][20381] Updated weights for policy 1, policy_version 115437 (0.0011) [2023-12-27 06:27:34,745][20381] Updated weights for policy 1, policy_version 115447 (0.0010) [2023-12-27 06:27:34,850][20347] Updated weights for policy 0, policy_version 115889 (0.0010) [2023-12-27 06:27:34,909][20347] Updated weights for policy 0, policy_version 115899 (0.0010) [2023-12-27 06:27:34,966][20347] Updated weights for policy 0, policy_version 115909 (0.0007) [2023-12-27 06:27:35,379][20381] Updated weights for policy 1, policy_version 115457 (0.0010) [2023-12-27 06:27:35,435][20381] Updated weights for policy 1, policy_version 115467 (0.0005) [2023-12-27 06:27:35,480][20381] Updated weights for policy 1, policy_version 115477 (0.0008) [2023-12-27 06:27:35,528][20381] Updated weights for policy 1, policy_version 115487 (0.0010) [2023-12-27 06:27:35,663][20347] Updated weights for policy 0, policy_version 115919 (0.0006) [2023-12-27 06:27:35,713][20347] Updated weights for policy 0, policy_version 115929 (0.0005) [2023-12-27 06:27:35,771][20347] Updated weights for policy 0, policy_version 115939 (0.0006) [2023-12-27 06:27:36,195][20381] Updated weights for policy 1, policy_version 115497 (0.0009) [2023-12-27 06:27:36,247][20381] Updated weights for policy 1, policy_version 115507 (0.0008) [2023-12-27 06:27:36,295][20381] Updated weights for policy 1, policy_version 115517 (0.0010) [2023-12-27 06:27:36,436][20347] Updated weights for policy 0, policy_version 115949 (0.0008) [2023-12-27 06:27:36,492][20347] Updated weights for policy 0, policy_version 115959 (0.0008) [2023-12-27 06:27:36,556][20347] Updated weights for policy 0, policy_version 115969 (0.0008) [2023-12-27 06:27:37,021][20381] Updated weights for policy 1, policy_version 115527 (0.0008) [2023-12-27 06:27:37,079][20381] Updated weights for policy 1, policy_version 115537 (0.0008) [2023-12-27 06:27:37,130][20381] Updated weights for policy 1, policy_version 115547 (0.0007) [2023-12-27 06:27:37,227][20347] Updated weights for policy 0, policy_version 115979 (0.0008) [2023-12-27 06:27:37,285][20347] Updated weights for policy 0, policy_version 115989 (0.0007) [2023-12-27 06:27:37,347][20347] Updated weights for policy 0, policy_version 115999 (0.0007) [2023-12-27 06:27:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 59293696. Throughput: 0: 10403.6, 1: 10528.0. Samples: 59264284. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000115552_29589504.pth... [2023-12-27 06:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000114304_29270016.pth [2023-12-27 06:27:37,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000116008_29712384.pth... [2023-12-27 06:27:37,404][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000114760_29392896.pth [2023-12-27 06:27:37,790][20381] Updated weights for policy 1, policy_version 115557 (0.0008) [2023-12-27 06:27:37,850][20381] Updated weights for policy 1, policy_version 115567 (0.0007) [2023-12-27 06:27:37,910][20381] Updated weights for policy 1, policy_version 115577 (0.0007) [2023-12-27 06:27:38,056][20347] Updated weights for policy 0, policy_version 116009 (0.0008) [2023-12-27 06:27:38,115][20347] Updated weights for policy 0, policy_version 116019 (0.0008) [2023-12-27 06:27:38,165][20347] Updated weights for policy 0, policy_version 116029 (0.0008) [2023-12-27 06:27:38,221][20347] Updated weights for policy 0, policy_version 116039 (0.0007) [2023-12-27 06:27:38,568][20381] Updated weights for policy 1, policy_version 115587 (0.0008) [2023-12-27 06:27:38,624][20381] Updated weights for policy 1, policy_version 115597 (0.0008) [2023-12-27 06:27:38,673][20381] Updated weights for policy 1, policy_version 115607 (0.0008) [2023-12-27 06:27:38,873][20347] Updated weights for policy 0, policy_version 116049 (0.0007) [2023-12-27 06:27:38,929][20347] Updated weights for policy 0, policy_version 116059 (0.0008) [2023-12-27 06:27:38,978][20347] Updated weights for policy 0, policy_version 116069 (0.0008) [2023-12-27 06:27:39,305][20381] Updated weights for policy 1, policy_version 115617 (0.0008) [2023-12-27 06:27:39,369][20381] Updated weights for policy 1, policy_version 115627 (0.0008) [2023-12-27 06:27:39,432][20381] Updated weights for policy 1, policy_version 115637 (0.0008) [2023-12-27 06:27:39,489][20381] Updated weights for policy 1, policy_version 115647 (0.0007) [2023-12-27 06:27:39,717][20347] Updated weights for policy 0, policy_version 116079 (0.0006) [2023-12-27 06:27:39,783][20347] Updated weights for policy 0, policy_version 116089 (0.0006) [2023-12-27 06:27:39,851][20347] Updated weights for policy 0, policy_version 116099 (0.0008) [2023-12-27 06:27:40,176][20381] Updated weights for policy 1, policy_version 115657 (0.0007) [2023-12-27 06:27:40,238][20381] Updated weights for policy 1, policy_version 115667 (0.0007) [2023-12-27 06:27:40,294][20381] Updated weights for policy 1, policy_version 115677 (0.0006) [2023-12-27 06:27:40,551][20347] Updated weights for policy 0, policy_version 116109 (0.0009) [2023-12-27 06:27:40,609][20347] Updated weights for policy 0, policy_version 116119 (0.0010) [2023-12-27 06:27:40,665][20347] Updated weights for policy 0, policy_version 116129 (0.0010) [2023-12-27 06:27:40,950][20381] Updated weights for policy 1, policy_version 115687 (0.0007) [2023-12-27 06:27:41,012][20381] Updated weights for policy 1, policy_version 115697 (0.0008) [2023-12-27 06:27:41,083][20381] Updated weights for policy 1, policy_version 115707 (0.0008) [2023-12-27 06:27:41,404][20347] Updated weights for policy 0, policy_version 116139 (0.0010) [2023-12-27 06:27:41,464][20347] Updated weights for policy 0, policy_version 116149 (0.0010) [2023-12-27 06:27:41,529][20347] Updated weights for policy 0, policy_version 116159 (0.0010) [2023-12-27 06:27:41,786][20381] Updated weights for policy 1, policy_version 115717 (0.0009) [2023-12-27 06:27:41,836][20381] Updated weights for policy 1, policy_version 115727 (0.0010) [2023-12-27 06:27:41,884][20381] Updated weights for policy 1, policy_version 115737 (0.0007) [2023-12-27 06:27:42,223][20347] Updated weights for policy 0, policy_version 116169 (0.0010) [2023-12-27 06:27:42,286][20347] Updated weights for policy 0, policy_version 116179 (0.0011) [2023-12-27 06:27:42,343][20347] Updated weights for policy 0, policy_version 116189 (0.0010) [2023-12-27 06:27:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 59392000. Throughput: 0: 10340.2, 1: 10483.7. Samples: 59386496. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:42,410][20347] Updated weights for policy 0, policy_version 116199 (0.0010) [2023-12-27 06:27:42,570][20381] Updated weights for policy 1, policy_version 115747 (0.0009) [2023-12-27 06:27:42,618][20381] Updated weights for policy 1, policy_version 115757 (0.0010) [2023-12-27 06:27:42,673][20381] Updated weights for policy 1, policy_version 115767 (0.0010) [2023-12-27 06:27:43,062][20347] Updated weights for policy 0, policy_version 116209 (0.0008) [2023-12-27 06:27:43,115][20347] Updated weights for policy 0, policy_version 116219 (0.0008) [2023-12-27 06:27:43,167][20347] Updated weights for policy 0, policy_version 116229 (0.0010) [2023-12-27 06:27:43,338][20381] Updated weights for policy 1, policy_version 115777 (0.0011) [2023-12-27 06:27:43,392][20381] Updated weights for policy 1, policy_version 115787 (0.0010) [2023-12-27 06:27:43,442][20381] Updated weights for policy 1, policy_version 115797 (0.0011) [2023-12-27 06:27:43,505][20381] Updated weights for policy 1, policy_version 115807 (0.0011) [2023-12-27 06:27:43,800][20347] Updated weights for policy 0, policy_version 116239 (0.0009) [2023-12-27 06:27:43,858][20347] Updated weights for policy 0, policy_version 116249 (0.0010) [2023-12-27 06:27:43,919][20347] Updated weights for policy 0, policy_version 116259 (0.0010) [2023-12-27 06:27:44,167][20381] Updated weights for policy 1, policy_version 115817 (0.0011) [2023-12-27 06:27:44,230][20381] Updated weights for policy 1, policy_version 115827 (0.0010) [2023-12-27 06:27:44,291][20381] Updated weights for policy 1, policy_version 115837 (0.0011) [2023-12-27 06:27:44,510][20347] Updated weights for policy 0, policy_version 116269 (0.0008) [2023-12-27 06:27:44,567][20347] Updated weights for policy 0, policy_version 116279 (0.0008) [2023-12-27 06:27:44,631][20347] Updated weights for policy 0, policy_version 116289 (0.0008) [2023-12-27 06:27:45,013][20381] Updated weights for policy 1, policy_version 115847 (0.0011) [2023-12-27 06:27:45,075][20381] Updated weights for policy 1, policy_version 115857 (0.0011) [2023-12-27 06:27:45,136][20381] Updated weights for policy 1, policy_version 115867 (0.0011) [2023-12-27 06:27:45,298][20347] Updated weights for policy 0, policy_version 116299 (0.0010) [2023-12-27 06:27:45,350][20347] Updated weights for policy 0, policy_version 116309 (0.0010) [2023-12-27 06:27:45,408][20347] Updated weights for policy 0, policy_version 116319 (0.0010) [2023-12-27 06:27:45,722][20381] Updated weights for policy 1, policy_version 115877 (0.0008) [2023-12-27 06:27:45,775][20381] Updated weights for policy 1, policy_version 115887 (0.0006) [2023-12-27 06:27:45,831][20381] Updated weights for policy 1, policy_version 115897 (0.0005) [2023-12-27 06:27:46,096][20347] Updated weights for policy 0, policy_version 116329 (0.0010) [2023-12-27 06:27:46,144][20347] Updated weights for policy 0, policy_version 116339 (0.0010) [2023-12-27 06:27:46,201][20347] Updated weights for policy 0, policy_version 116349 (0.0010) [2023-12-27 06:27:46,259][20347] Updated weights for policy 0, policy_version 116359 (0.0010) [2023-12-27 06:27:46,356][20381] Updated weights for policy 1, policy_version 115907 (0.0005) [2023-12-27 06:27:46,418][20381] Updated weights for policy 1, policy_version 115917 (0.0010) [2023-12-27 06:27:46,477][20381] Updated weights for policy 1, policy_version 115927 (0.0011) [2023-12-27 06:27:46,836][20347] Updated weights for policy 0, policy_version 116369 (0.0010) [2023-12-27 06:27:46,884][20347] Updated weights for policy 0, policy_version 116379 (0.0010) [2023-12-27 06:27:46,942][20347] Updated weights for policy 0, policy_version 116389 (0.0010) [2023-12-27 06:27:47,084][20381] Updated weights for policy 1, policy_version 115937 (0.0011) [2023-12-27 06:27:47,129][20381] Updated weights for policy 1, policy_version 115947 (0.0010) [2023-12-27 06:27:47,178][20381] Updated weights for policy 1, policy_version 115957 (0.0010) [2023-12-27 06:27:47,226][20381] Updated weights for policy 1, policy_version 115967 (0.0010) [2023-12-27 06:27:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.7, 300 sec: 20827.1). Total num frames: 59506688. Throughput: 0: 10353.7, 1: 10558.5. Samples: 59513872. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:47,560][20347] Updated weights for policy 0, policy_version 116399 (0.0010) [2023-12-27 06:27:47,610][20347] Updated weights for policy 0, policy_version 116409 (0.0010) [2023-12-27 06:27:47,671][20347] Updated weights for policy 0, policy_version 116419 (0.0010) [2023-12-27 06:27:47,885][20381] Updated weights for policy 1, policy_version 115977 (0.0006) [2023-12-27 06:27:47,940][20381] Updated weights for policy 1, policy_version 115987 (0.0005) [2023-12-27 06:27:48,000][20381] Updated weights for policy 1, policy_version 115997 (0.0006) [2023-12-27 06:27:48,281][20347] Updated weights for policy 0, policy_version 116429 (0.0008) [2023-12-27 06:27:48,349][20347] Updated weights for policy 0, policy_version 116439 (0.0006) [2023-12-27 06:27:48,416][20347] Updated weights for policy 0, policy_version 116449 (0.0006) [2023-12-27 06:27:48,579][20381] Updated weights for policy 1, policy_version 116007 (0.0009) [2023-12-27 06:27:48,635][20381] Updated weights for policy 1, policy_version 116017 (0.0011) [2023-12-27 06:27:48,694][20381] Updated weights for policy 1, policy_version 116027 (0.0011) [2023-12-27 06:27:49,045][20347] Updated weights for policy 0, policy_version 116459 (0.0010) [2023-12-27 06:27:49,095][20347] Updated weights for policy 0, policy_version 116469 (0.0010) [2023-12-27 06:27:49,153][20347] Updated weights for policy 0, policy_version 116479 (0.0010) [2023-12-27 06:27:49,314][20381] Updated weights for policy 1, policy_version 116037 (0.0011) [2023-12-27 06:27:49,377][20381] Updated weights for policy 1, policy_version 116047 (0.0009) [2023-12-27 06:27:49,427][20381] Updated weights for policy 1, policy_version 116057 (0.0011) [2023-12-27 06:27:49,768][20347] Updated weights for policy 0, policy_version 116489 (0.0009) [2023-12-27 06:27:49,827][20347] Updated weights for policy 0, policy_version 116499 (0.0009) [2023-12-27 06:27:49,886][20347] Updated weights for policy 0, policy_version 116509 (0.0007) [2023-12-27 06:27:49,946][20347] Updated weights for policy 0, policy_version 116519 (0.0006) [2023-12-27 06:27:50,173][20381] Updated weights for policy 1, policy_version 116067 (0.0009) [2023-12-27 06:27:50,231][20381] Updated weights for policy 1, policy_version 116077 (0.0007) [2023-12-27 06:27:50,288][20381] Updated weights for policy 1, policy_version 116087 (0.0005) [2023-12-27 06:27:50,618][20347] Updated weights for policy 0, policy_version 116529 (0.0011) [2023-12-27 06:27:50,677][20347] Updated weights for policy 0, policy_version 116539 (0.0010) [2023-12-27 06:27:50,735][20347] Updated weights for policy 0, policy_version 116549 (0.0010) [2023-12-27 06:27:50,943][20381] Updated weights for policy 1, policy_version 116097 (0.0006) [2023-12-27 06:27:50,999][20381] Updated weights for policy 1, policy_version 116107 (0.0008) [2023-12-27 06:27:51,055][20381] Updated weights for policy 1, policy_version 116117 (0.0009) [2023-12-27 06:27:51,107][20381] Updated weights for policy 1, policy_version 116127 (0.0007) [2023-12-27 06:27:51,415][20347] Updated weights for policy 0, policy_version 116559 (0.0008) [2023-12-27 06:27:51,463][20347] Updated weights for policy 0, policy_version 116569 (0.0007) [2023-12-27 06:27:51,518][20347] Updated weights for policy 0, policy_version 116579 (0.0008) [2023-12-27 06:27:51,834][20381] Updated weights for policy 1, policy_version 116137 (0.0009) [2023-12-27 06:27:51,889][20381] Updated weights for policy 1, policy_version 116147 (0.0010) [2023-12-27 06:27:51,945][20381] Updated weights for policy 1, policy_version 116157 (0.0010) [2023-12-27 06:27:52,180][20347] Updated weights for policy 0, policy_version 116589 (0.0007) [2023-12-27 06:27:52,236][20347] Updated weights for policy 0, policy_version 116599 (0.0007) [2023-12-27 06:27:52,290][20347] Updated weights for policy 0, policy_version 116609 (0.0008) [2023-12-27 06:27:52,362][19170] Fps is (10 sec: 22118.4, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 59613184. Throughput: 0: 10428.8, 1: 10554.3. Samples: 59578996. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000116160_29745152.pth... [2023-12-27 06:27:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000116616_29868032.pth... [2023-12-27 06:27:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000115368_29548544.pth [2023-12-27 06:27:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000114944_29433856.pth [2023-12-27 06:27:52,633][20381] Updated weights for policy 1, policy_version 116167 (0.0010) [2023-12-27 06:27:52,691][20381] Updated weights for policy 1, policy_version 116177 (0.0011) [2023-12-27 06:27:52,746][20381] Updated weights for policy 1, policy_version 116187 (0.0008) [2023-12-27 06:27:52,936][20347] Updated weights for policy 0, policy_version 116619 (0.0007) [2023-12-27 06:27:52,992][20347] Updated weights for policy 0, policy_version 116629 (0.0006) [2023-12-27 06:27:53,051][20347] Updated weights for policy 0, policy_version 116639 (0.0006) [2023-12-27 06:27:53,351][20381] Updated weights for policy 1, policy_version 116197 (0.0005) [2023-12-27 06:27:53,401][20381] Updated weights for policy 1, policy_version 116207 (0.0005) [2023-12-27 06:27:53,451][20381] Updated weights for policy 1, policy_version 116217 (0.0005) [2023-12-27 06:27:53,712][20347] Updated weights for policy 0, policy_version 116649 (0.0005) [2023-12-27 06:27:53,772][20347] Updated weights for policy 0, policy_version 116659 (0.0007) [2023-12-27 06:27:53,817][20347] Updated weights for policy 0, policy_version 116669 (0.0009) [2023-12-27 06:27:53,868][20347] Updated weights for policy 0, policy_version 116679 (0.0008) [2023-12-27 06:27:54,029][20381] Updated weights for policy 1, policy_version 116227 (0.0007) [2023-12-27 06:27:54,087][20381] Updated weights for policy 1, policy_version 116237 (0.0007) [2023-12-27 06:27:54,145][20381] Updated weights for policy 1, policy_version 116247 (0.0005) [2023-12-27 06:27:54,508][20347] Updated weights for policy 0, policy_version 116689 (0.0005) [2023-12-27 06:27:54,554][20347] Updated weights for policy 0, policy_version 116699 (0.0005) [2023-12-27 06:27:54,607][20347] Updated weights for policy 0, policy_version 116709 (0.0005) [2023-12-27 06:27:54,726][20381] Updated weights for policy 1, policy_version 116257 (0.0006) [2023-12-27 06:27:54,788][20381] Updated weights for policy 1, policy_version 116267 (0.0006) [2023-12-27 06:27:54,841][20381] Updated weights for policy 1, policy_version 116277 (0.0005) [2023-12-27 06:27:54,900][20381] Updated weights for policy 1, policy_version 116287 (0.0010) [2023-12-27 06:27:55,205][20347] Updated weights for policy 0, policy_version 116719 (0.0007) [2023-12-27 06:27:55,255][20347] Updated weights for policy 0, policy_version 116729 (0.0008) [2023-12-27 06:27:55,300][20347] Updated weights for policy 0, policy_version 116739 (0.0007) [2023-12-27 06:27:55,553][20381] Updated weights for policy 1, policy_version 116297 (0.0011) [2023-12-27 06:27:55,613][20381] Updated weights for policy 1, policy_version 116307 (0.0008) [2023-12-27 06:27:55,685][20381] Updated weights for policy 1, policy_version 116317 (0.0008) [2023-12-27 06:27:56,045][20347] Updated weights for policy 0, policy_version 116749 (0.0008) [2023-12-27 06:27:56,097][20347] Updated weights for policy 0, policy_version 116759 (0.0008) [2023-12-27 06:27:56,154][20347] Updated weights for policy 0, policy_version 116769 (0.0008) [2023-12-27 06:27:56,244][20381] Updated weights for policy 1, policy_version 116327 (0.0006) [2023-12-27 06:27:56,302][20381] Updated weights for policy 1, policy_version 116337 (0.0007) [2023-12-27 06:27:56,367][20381] Updated weights for policy 1, policy_version 116347 (0.0008) [2023-12-27 06:27:56,895][20347] Updated weights for policy 0, policy_version 116779 (0.0009) [2023-12-27 06:27:56,945][20347] Updated weights for policy 0, policy_version 116789 (0.0008) [2023-12-27 06:27:56,976][20381] Updated weights for policy 1, policy_version 116357 (0.0007) [2023-12-27 06:27:57,001][20347] Updated weights for policy 0, policy_version 116799 (0.0008) [2023-12-27 06:27:57,036][20381] Updated weights for policy 1, policy_version 116367 (0.0005) [2023-12-27 06:27:57,083][20381] Updated weights for policy 1, policy_version 116377 (0.0007) [2023-12-27 06:27:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 59719680. Throughput: 0: 10468.2, 1: 10584.7. Samples: 59706540. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:27:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:27:57,663][20347] Updated weights for policy 0, policy_version 116809 (0.0007) [2023-12-27 06:27:57,691][20381] Updated weights for policy 1, policy_version 116387 (0.0009) [2023-12-27 06:27:57,728][20347] Updated weights for policy 0, policy_version 116819 (0.0006) [2023-12-27 06:27:57,744][20381] Updated weights for policy 1, policy_version 116397 (0.0010) [2023-12-27 06:27:57,800][20347] Updated weights for policy 0, policy_version 116829 (0.0005) [2023-12-27 06:27:57,800][20381] Updated weights for policy 1, policy_version 116407 (0.0007) [2023-12-27 06:27:57,870][20347] Updated weights for policy 0, policy_version 116839 (0.0006) [2023-12-27 06:27:58,440][20347] Updated weights for policy 0, policy_version 116849 (0.0006) [2023-12-27 06:27:58,494][20347] Updated weights for policy 0, policy_version 116859 (0.0007) [2023-12-27 06:27:58,496][20381] Updated weights for policy 1, policy_version 116417 (0.0006) [2023-12-27 06:27:58,555][20347] Updated weights for policy 0, policy_version 116869 (0.0006) [2023-12-27 06:27:58,557][20381] Updated weights for policy 1, policy_version 116427 (0.0011) [2023-12-27 06:27:58,624][20381] Updated weights for policy 1, policy_version 116437 (0.0011) [2023-12-27 06:27:58,688][20381] Updated weights for policy 1, policy_version 116447 (0.0010) [2023-12-27 06:27:59,314][20347] Updated weights for policy 0, policy_version 116879 (0.0007) [2023-12-27 06:27:59,375][20347] Updated weights for policy 0, policy_version 116889 (0.0008) [2023-12-27 06:27:59,432][20347] Updated weights for policy 0, policy_version 116899 (0.0007) [2023-12-27 06:27:59,451][20381] Updated weights for policy 1, policy_version 116457 (0.0010) [2023-12-27 06:27:59,505][20381] Updated weights for policy 1, policy_version 116467 (0.0010) [2023-12-27 06:27:59,560][20381] Updated weights for policy 1, policy_version 116477 (0.0010) [2023-12-27 06:28:00,132][20347] Updated weights for policy 0, policy_version 116909 (0.0007) [2023-12-27 06:28:00,184][20347] Updated weights for policy 0, policy_version 116919 (0.0007) [2023-12-27 06:28:00,241][20347] Updated weights for policy 0, policy_version 116929 (0.0008) [2023-12-27 06:28:00,243][20381] Updated weights for policy 1, policy_version 116487 (0.0007) [2023-12-27 06:28:00,297][20381] Updated weights for policy 1, policy_version 116497 (0.0010) [2023-12-27 06:28:00,345][20381] Updated weights for policy 1, policy_version 116507 (0.0010) [2023-12-27 06:28:00,934][20347] Updated weights for policy 0, policy_version 116939 (0.0009) [2023-12-27 06:28:00,985][20347] Updated weights for policy 0, policy_version 116949 (0.0010) [2023-12-27 06:28:01,031][20381] Updated weights for policy 1, policy_version 116517 (0.0009) [2023-12-27 06:28:01,041][20347] Updated weights for policy 0, policy_version 116959 (0.0010) [2023-12-27 06:28:01,091][20381] Updated weights for policy 1, policy_version 116527 (0.0005) [2023-12-27 06:28:01,150][20381] Updated weights for policy 1, policy_version 116537 (0.0008) [2023-12-27 06:28:01,672][20347] Updated weights for policy 0, policy_version 116969 (0.0010) [2023-12-27 06:28:01,739][20347] Updated weights for policy 0, policy_version 116979 (0.0010) [2023-12-27 06:28:01,798][20347] Updated weights for policy 0, policy_version 116989 (0.0010) [2023-12-27 06:28:01,848][20381] Updated weights for policy 1, policy_version 116547 (0.0007) [2023-12-27 06:28:01,856][20347] Updated weights for policy 0, policy_version 116999 (0.0010) [2023-12-27 06:28:01,909][20381] Updated weights for policy 1, policy_version 116557 (0.0005) [2023-12-27 06:28:01,968][20381] Updated weights for policy 1, policy_version 116567 (0.0005) [2023-12-27 06:28:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 59817984. Throughput: 0: 10374.7, 1: 10600.2. Samples: 59828920. Policy #0 lag: (min: 7.0, avg: 10.4, max: 39.0) [2023-12-27 06:28:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:02,554][20347] Updated weights for policy 0, policy_version 117009 (0.0007) [2023-12-27 06:28:02,556][20381] Updated weights for policy 1, policy_version 116577 (0.0006) [2023-12-27 06:28:02,604][20381] Updated weights for policy 1, policy_version 116587 (0.0007) [2023-12-27 06:28:02,605][20347] Updated weights for policy 0, policy_version 117019 (0.0005) [2023-12-27 06:28:02,650][20381] Updated weights for policy 1, policy_version 116597 (0.0007) [2023-12-27 06:28:02,660][20347] Updated weights for policy 0, policy_version 117029 (0.0005) [2023-12-27 06:28:02,701][20381] Updated weights for policy 1, policy_version 116607 (0.0008) [2023-12-27 06:28:03,322][20381] Updated weights for policy 1, policy_version 116617 (0.0005) [2023-12-27 06:28:03,350][20347] Updated weights for policy 0, policy_version 117039 (0.0007) [2023-12-27 06:28:03,371][20381] Updated weights for policy 1, policy_version 116627 (0.0005) [2023-12-27 06:28:03,397][20347] Updated weights for policy 0, policy_version 117049 (0.0007) [2023-12-27 06:28:03,420][20381] Updated weights for policy 1, policy_version 116637 (0.0009) [2023-12-27 06:28:03,448][20347] Updated weights for policy 0, policy_version 117059 (0.0006) [2023-12-27 06:28:03,994][20381] Updated weights for policy 1, policy_version 116647 (0.0008) [2023-12-27 06:28:04,049][20381] Updated weights for policy 1, policy_version 116657 (0.0006) [2023-12-27 06:28:04,100][20381] Updated weights for policy 1, policy_version 116667 (0.0009) [2023-12-27 06:28:04,205][20347] Updated weights for policy 0, policy_version 117069 (0.0007) [2023-12-27 06:28:04,266][20347] Updated weights for policy 0, policy_version 117079 (0.0008) [2023-12-27 06:28:04,325][20347] Updated weights for policy 0, policy_version 117089 (0.0008) [2023-12-27 06:28:04,709][20381] Updated weights for policy 1, policy_version 116677 (0.0011) [2023-12-27 06:28:04,758][20381] Updated weights for policy 1, policy_version 116687 (0.0010) [2023-12-27 06:28:04,814][20381] Updated weights for policy 1, policy_version 116697 (0.0011) [2023-12-27 06:28:04,986][20347] Updated weights for policy 0, policy_version 117099 (0.0007) [2023-12-27 06:28:05,044][20347] Updated weights for policy 0, policy_version 117109 (0.0007) [2023-12-27 06:28:05,105][20347] Updated weights for policy 0, policy_version 117119 (0.0010) [2023-12-27 06:28:05,462][20381] Updated weights for policy 1, policy_version 116707 (0.0010) [2023-12-27 06:28:05,511][20381] Updated weights for policy 1, policy_version 116717 (0.0010) [2023-12-27 06:28:05,566][20381] Updated weights for policy 1, policy_version 116727 (0.0011) [2023-12-27 06:28:05,736][20347] Updated weights for policy 0, policy_version 117129 (0.0010) [2023-12-27 06:28:05,795][20347] Updated weights for policy 0, policy_version 117139 (0.0010) [2023-12-27 06:28:05,842][20347] Updated weights for policy 0, policy_version 117149 (0.0010) [2023-12-27 06:28:05,891][20347] Updated weights for policy 0, policy_version 117159 (0.0010) [2023-12-27 06:28:06,293][20381] Updated weights for policy 1, policy_version 116737 (0.0010) [2023-12-27 06:28:06,351][20381] Updated weights for policy 1, policy_version 116747 (0.0008) [2023-12-27 06:28:06,416][20381] Updated weights for policy 1, policy_version 116757 (0.0009) [2023-12-27 06:28:06,476][20381] Updated weights for policy 1, policy_version 116767 (0.0011) [2023-12-27 06:28:06,608][20347] Updated weights for policy 0, policy_version 117169 (0.0008) [2023-12-27 06:28:06,667][20347] Updated weights for policy 0, policy_version 117179 (0.0007) [2023-12-27 06:28:06,722][20347] Updated weights for policy 0, policy_version 117189 (0.0008) [2023-12-27 06:28:07,119][20381] Updated weights for policy 1, policy_version 116777 (0.0011) [2023-12-27 06:28:07,177][20381] Updated weights for policy 1, policy_version 116787 (0.0011) [2023-12-27 06:28:07,228][20381] Updated weights for policy 1, policy_version 116797 (0.0011) [2023-12-27 06:28:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 59924480. Throughput: 0: 10355.2, 1: 10616.7. Samples: 59893028. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000116800_29908992.pth... [2023-12-27 06:28:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000115552_29589504.pth [2023-12-27 06:28:07,380][20347] Updated weights for policy 0, policy_version 117199 (0.0006) [2023-12-27 06:28:07,433][20347] Updated weights for policy 0, policy_version 117209 (0.0005) [2023-12-27 06:28:07,485][20347] Updated weights for policy 0, policy_version 117219 (0.0005) [2023-12-27 06:28:07,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000117224_30023680.pth... [2023-12-27 06:28:07,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000116008_29712384.pth [2023-12-27 06:28:07,921][20381] Updated weights for policy 1, policy_version 116807 (0.0011) [2023-12-27 06:28:07,976][20381] Updated weights for policy 1, policy_version 116817 (0.0011) [2023-12-27 06:28:08,028][20381] Updated weights for policy 1, policy_version 116827 (0.0010) [2023-12-27 06:28:08,045][20347] Updated weights for policy 0, policy_version 117229 (0.0007) [2023-12-27 06:28:08,106][20347] Updated weights for policy 0, policy_version 117239 (0.0010) [2023-12-27 06:28:08,165][20347] Updated weights for policy 0, policy_version 117249 (0.0010) [2023-12-27 06:28:08,658][20381] Updated weights for policy 1, policy_version 116837 (0.0011) [2023-12-27 06:28:08,711][20381] Updated weights for policy 1, policy_version 116847 (0.0007) [2023-12-27 06:28:08,762][20347] Updated weights for policy 0, policy_version 117259 (0.0007) [2023-12-27 06:28:08,771][20381] Updated weights for policy 1, policy_version 116857 (0.0006) [2023-12-27 06:28:08,813][20347] Updated weights for policy 0, policy_version 117269 (0.0006) [2023-12-27 06:28:08,877][20347] Updated weights for policy 0, policy_version 117279 (0.0005) [2023-12-27 06:28:09,429][20381] Updated weights for policy 1, policy_version 116867 (0.0007) [2023-12-27 06:28:09,467][20347] Updated weights for policy 0, policy_version 117289 (0.0006) [2023-12-27 06:28:09,493][20381] Updated weights for policy 1, policy_version 116877 (0.0008) [2023-12-27 06:28:09,527][20347] Updated weights for policy 0, policy_version 117299 (0.0008) [2023-12-27 06:28:09,552][20381] Updated weights for policy 1, policy_version 116887 (0.0007) [2023-12-27 06:28:09,582][20347] Updated weights for policy 0, policy_version 117309 (0.0008) [2023-12-27 06:28:09,640][20347] Updated weights for policy 0, policy_version 117319 (0.0007) [2023-12-27 06:28:10,206][20381] Updated weights for policy 1, policy_version 116897 (0.0008) [2023-12-27 06:28:10,261][20381] Updated weights for policy 1, policy_version 116907 (0.0008) [2023-12-27 06:28:10,321][20381] Updated weights for policy 1, policy_version 116917 (0.0007) [2023-12-27 06:28:10,331][20347] Updated weights for policy 0, policy_version 117329 (0.0006) [2023-12-27 06:28:10,381][20381] Updated weights for policy 1, policy_version 116927 (0.0007) [2023-12-27 06:28:10,385][20347] Updated weights for policy 0, policy_version 117339 (0.0006) [2023-12-27 06:28:10,437][20347] Updated weights for policy 0, policy_version 117349 (0.0006) [2023-12-27 06:28:10,979][20381] Updated weights for policy 1, policy_version 116937 (0.0008) [2023-12-27 06:28:11,021][20347] Updated weights for policy 0, policy_version 117359 (0.0007) [2023-12-27 06:28:11,041][20381] Updated weights for policy 1, policy_version 116947 (0.0008) [2023-12-27 06:28:11,076][20347] Updated weights for policy 0, policy_version 117369 (0.0006) [2023-12-27 06:28:11,101][20381] Updated weights for policy 1, policy_version 116957 (0.0007) [2023-12-27 06:28:11,135][20347] Updated weights for policy 0, policy_version 117379 (0.0006) [2023-12-27 06:28:11,839][20381] Updated weights for policy 1, policy_version 116967 (0.0008) [2023-12-27 06:28:11,879][20347] Updated weights for policy 0, policy_version 117389 (0.0006) [2023-12-27 06:28:11,901][20381] Updated weights for policy 1, policy_version 116977 (0.0007) [2023-12-27 06:28:11,939][20347] Updated weights for policy 0, policy_version 117399 (0.0007) [2023-12-27 06:28:11,959][20381] Updated weights for policy 1, policy_version 116987 (0.0006) [2023-12-27 06:28:11,995][20347] Updated weights for policy 0, policy_version 117409 (0.0006) [2023-12-27 06:28:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.7, 300 sec: 20799.4). Total num frames: 60030976. Throughput: 0: 10441.2, 1: 10629.3. Samples: 60020092. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:12,576][20381] Updated weights for policy 1, policy_version 116997 (0.0005) [2023-12-27 06:28:12,640][20381] Updated weights for policy 1, policy_version 117007 (0.0005) [2023-12-27 06:28:12,706][20381] Updated weights for policy 1, policy_version 117017 (0.0005) [2023-12-27 06:28:12,752][20347] Updated weights for policy 0, policy_version 117419 (0.0007) [2023-12-27 06:28:12,814][20347] Updated weights for policy 0, policy_version 117429 (0.0007) [2023-12-27 06:28:12,870][20347] Updated weights for policy 0, policy_version 117439 (0.0008) [2023-12-27 06:28:13,236][20381] Updated weights for policy 1, policy_version 117027 (0.0006) [2023-12-27 06:28:13,289][20381] Updated weights for policy 1, policy_version 117037 (0.0006) [2023-12-27 06:28:13,346][20381] Updated weights for policy 1, policy_version 117047 (0.0008) [2023-12-27 06:28:13,613][20347] Updated weights for policy 0, policy_version 117449 (0.0008) [2023-12-27 06:28:13,674][20347] Updated weights for policy 0, policy_version 117459 (0.0007) [2023-12-27 06:28:13,734][20347] Updated weights for policy 0, policy_version 117469 (0.0005) [2023-12-27 06:28:13,796][20347] Updated weights for policy 0, policy_version 117479 (0.0007) [2023-12-27 06:28:13,987][20381] Updated weights for policy 1, policy_version 117057 (0.0008) [2023-12-27 06:28:14,048][20381] Updated weights for policy 1, policy_version 117067 (0.0008) [2023-12-27 06:28:14,105][20381] Updated weights for policy 1, policy_version 117077 (0.0007) [2023-12-27 06:28:14,159][20381] Updated weights for policy 1, policy_version 117087 (0.0008) [2023-12-27 06:28:14,412][20347] Updated weights for policy 0, policy_version 117489 (0.0007) [2023-12-27 06:28:14,471][20347] Updated weights for policy 0, policy_version 117499 (0.0008) [2023-12-27 06:28:14,522][20347] Updated weights for policy 0, policy_version 117509 (0.0005) [2023-12-27 06:28:14,916][20381] Updated weights for policy 1, policy_version 117097 (0.0008) [2023-12-27 06:28:14,977][20381] Updated weights for policy 1, policy_version 117107 (0.0008) [2023-12-27 06:28:15,035][20381] Updated weights for policy 1, policy_version 117117 (0.0008) [2023-12-27 06:28:15,214][20347] Updated weights for policy 0, policy_version 117519 (0.0007) [2023-12-27 06:28:15,266][20347] Updated weights for policy 0, policy_version 117529 (0.0007) [2023-12-27 06:28:15,316][20347] Updated weights for policy 0, policy_version 117539 (0.0008) [2023-12-27 06:28:15,703][20381] Updated weights for policy 1, policy_version 117127 (0.0007) [2023-12-27 06:28:15,750][20381] Updated weights for policy 1, policy_version 117137 (0.0010) [2023-12-27 06:28:15,807][20381] Updated weights for policy 1, policy_version 117147 (0.0007) [2023-12-27 06:28:15,967][20347] Updated weights for policy 0, policy_version 117549 (0.0007) [2023-12-27 06:28:16,020][20347] Updated weights for policy 0, policy_version 117559 (0.0008) [2023-12-27 06:28:16,068][20347] Updated weights for policy 0, policy_version 117569 (0.0010) [2023-12-27 06:28:16,431][20381] Updated weights for policy 1, policy_version 117157 (0.0007) [2023-12-27 06:28:16,491][20381] Updated weights for policy 1, policy_version 117167 (0.0008) [2023-12-27 06:28:16,549][20381] Updated weights for policy 1, policy_version 117177 (0.0008) [2023-12-27 06:28:16,786][20347] Updated weights for policy 0, policy_version 117579 (0.0007) [2023-12-27 06:28:16,838][20347] Updated weights for policy 0, policy_version 117589 (0.0008) [2023-12-27 06:28:16,886][20347] Updated weights for policy 0, policy_version 117599 (0.0007) [2023-12-27 06:28:17,250][20381] Updated weights for policy 1, policy_version 117187 (0.0008) [2023-12-27 06:28:17,305][20381] Updated weights for policy 1, policy_version 117197 (0.0008) [2023-12-27 06:28:17,358][20381] Updated weights for policy 1, policy_version 117207 (0.0007) [2023-12-27 06:28:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 60129280. Throughput: 0: 10352.1, 1: 10596.1. Samples: 60142568. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:17,577][20347] Updated weights for policy 0, policy_version 117609 (0.0007) [2023-12-27 06:28:17,636][20347] Updated weights for policy 0, policy_version 117619 (0.0007) [2023-12-27 06:28:17,694][20347] Updated weights for policy 0, policy_version 117629 (0.0007) [2023-12-27 06:28:17,748][20347] Updated weights for policy 0, policy_version 117639 (0.0008) [2023-12-27 06:28:18,041][20381] Updated weights for policy 1, policy_version 117217 (0.0007) [2023-12-27 06:28:18,091][20381] Updated weights for policy 1, policy_version 117227 (0.0005) [2023-12-27 06:28:18,138][20381] Updated weights for policy 1, policy_version 117237 (0.0005) [2023-12-27 06:28:18,196][20381] Updated weights for policy 1, policy_version 117247 (0.0008) [2023-12-27 06:28:18,452][20347] Updated weights for policy 0, policy_version 117649 (0.0007) [2023-12-27 06:28:18,496][20347] Updated weights for policy 0, policy_version 117659 (0.0007) [2023-12-27 06:28:18,550][20347] Updated weights for policy 0, policy_version 117669 (0.0007) [2023-12-27 06:28:18,820][20381] Updated weights for policy 1, policy_version 117257 (0.0006) [2023-12-27 06:28:18,875][20381] Updated weights for policy 1, policy_version 117267 (0.0007) [2023-12-27 06:28:18,934][20381] Updated weights for policy 1, policy_version 117277 (0.0011) [2023-12-27 06:28:19,282][20347] Updated weights for policy 0, policy_version 117679 (0.0006) [2023-12-27 06:28:19,332][20347] Updated weights for policy 0, policy_version 117689 (0.0007) [2023-12-27 06:28:19,388][20347] Updated weights for policy 0, policy_version 117699 (0.0007) [2023-12-27 06:28:19,598][20381] Updated weights for policy 1, policy_version 117287 (0.0011) [2023-12-27 06:28:19,658][20381] Updated weights for policy 1, policy_version 117297 (0.0011) [2023-12-27 06:28:19,717][20381] Updated weights for policy 1, policy_version 117307 (0.0011) [2023-12-27 06:28:20,084][20347] Updated weights for policy 0, policy_version 117709 (0.0008) [2023-12-27 06:28:20,145][20347] Updated weights for policy 0, policy_version 117719 (0.0008) [2023-12-27 06:28:20,203][20347] Updated weights for policy 0, policy_version 117729 (0.0005) [2023-12-27 06:28:20,332][20381] Updated weights for policy 1, policy_version 117317 (0.0011) [2023-12-27 06:28:20,391][20381] Updated weights for policy 1, policy_version 117327 (0.0011) [2023-12-27 06:28:20,451][20381] Updated weights for policy 1, policy_version 117337 (0.0011) [2023-12-27 06:28:20,870][20347] Updated weights for policy 0, policy_version 117739 (0.0006) [2023-12-27 06:28:20,923][20347] Updated weights for policy 0, policy_version 117749 (0.0007) [2023-12-27 06:28:20,981][20347] Updated weights for policy 0, policy_version 117759 (0.0007) [2023-12-27 06:28:21,110][20381] Updated weights for policy 1, policy_version 117347 (0.0007) [2023-12-27 06:28:21,173][20381] Updated weights for policy 1, policy_version 117357 (0.0010) [2023-12-27 06:28:21,235][20381] Updated weights for policy 1, policy_version 117367 (0.0010) [2023-12-27 06:28:21,689][20347] Updated weights for policy 0, policy_version 117769 (0.0008) [2023-12-27 06:28:21,753][20347] Updated weights for policy 0, policy_version 117779 (0.0007) [2023-12-27 06:28:21,818][20347] Updated weights for policy 0, policy_version 117789 (0.0007) [2023-12-27 06:28:21,875][20347] Updated weights for policy 0, policy_version 117799 (0.0005) [2023-12-27 06:28:22,015][20381] Updated weights for policy 1, policy_version 117377 (0.0011) [2023-12-27 06:28:22,074][20381] Updated weights for policy 1, policy_version 117387 (0.0009) [2023-12-27 06:28:22,133][20381] Updated weights for policy 1, policy_version 117397 (0.0007) [2023-12-27 06:28:22,192][20381] Updated weights for policy 1, policy_version 117407 (0.0006) [2023-12-27 06:28:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 60235776. Throughput: 0: 10315.1, 1: 10573.3. Samples: 60204260. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000117408_30064640.pth... [2023-12-27 06:28:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000117800_30171136.pth... [2023-12-27 06:28:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000116160_29745152.pth [2023-12-27 06:28:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000116616_29868032.pth [2023-12-27 06:28:22,534][20347] Updated weights for policy 0, policy_version 117809 (0.0007) [2023-12-27 06:28:22,595][20347] Updated weights for policy 0, policy_version 117819 (0.0008) [2023-12-27 06:28:22,654][20347] Updated weights for policy 0, policy_version 117829 (0.0006) [2023-12-27 06:28:22,801][20381] Updated weights for policy 1, policy_version 117417 (0.0008) [2023-12-27 06:28:22,867][20381] Updated weights for policy 1, policy_version 117427 (0.0008) [2023-12-27 06:28:22,928][20381] Updated weights for policy 1, policy_version 117437 (0.0008) [2023-12-27 06:28:23,353][20347] Updated weights for policy 0, policy_version 117839 (0.0007) [2023-12-27 06:28:23,411][20347] Updated weights for policy 0, policy_version 117849 (0.0006) [2023-12-27 06:28:23,461][20347] Updated weights for policy 0, policy_version 117859 (0.0005) [2023-12-27 06:28:23,614][20381] Updated weights for policy 1, policy_version 117447 (0.0007) [2023-12-27 06:28:23,669][20381] Updated weights for policy 1, policy_version 117457 (0.0005) [2023-12-27 06:28:23,725][20381] Updated weights for policy 1, policy_version 117467 (0.0005) [2023-12-27 06:28:24,082][20347] Updated weights for policy 0, policy_version 117869 (0.0009) [2023-12-27 06:28:24,143][20347] Updated weights for policy 0, policy_version 117879 (0.0010) [2023-12-27 06:28:24,200][20347] Updated weights for policy 0, policy_version 117889 (0.0010) [2023-12-27 06:28:24,309][20381] Updated weights for policy 1, policy_version 117477 (0.0006) [2023-12-27 06:28:24,362][20381] Updated weights for policy 1, policy_version 117487 (0.0007) [2023-12-27 06:28:24,425][20381] Updated weights for policy 1, policy_version 117497 (0.0008) [2023-12-27 06:28:24,793][20347] Updated weights for policy 0, policy_version 117899 (0.0009) [2023-12-27 06:28:24,850][20347] Updated weights for policy 0, policy_version 117909 (0.0008) [2023-12-27 06:28:24,907][20347] Updated weights for policy 0, policy_version 117919 (0.0010) [2023-12-27 06:28:25,046][20381] Updated weights for policy 1, policy_version 117507 (0.0008) [2023-12-27 06:28:25,099][20381] Updated weights for policy 1, policy_version 117517 (0.0007) [2023-12-27 06:28:25,147][20381] Updated weights for policy 1, policy_version 117527 (0.0009) [2023-12-27 06:28:25,483][20347] Updated weights for policy 0, policy_version 117929 (0.0009) [2023-12-27 06:28:25,539][20347] Updated weights for policy 0, policy_version 117939 (0.0005) [2023-12-27 06:28:25,593][20347] Updated weights for policy 0, policy_version 117949 (0.0005) [2023-12-27 06:28:25,654][20347] Updated weights for policy 0, policy_version 117959 (0.0005) [2023-12-27 06:28:25,900][20381] Updated weights for policy 1, policy_version 117537 (0.0007) [2023-12-27 06:28:25,949][20381] Updated weights for policy 1, policy_version 117547 (0.0007) [2023-12-27 06:28:26,002][20381] Updated weights for policy 1, policy_version 117557 (0.0008) [2023-12-27 06:28:26,054][20381] Updated weights for policy 1, policy_version 117567 (0.0007) [2023-12-27 06:28:26,222][20347] Updated weights for policy 0, policy_version 117969 (0.0009) [2023-12-27 06:28:26,275][20347] Updated weights for policy 0, policy_version 117979 (0.0010) [2023-12-27 06:28:26,332][20347] Updated weights for policy 0, policy_version 117989 (0.0010) [2023-12-27 06:28:26,805][20381] Updated weights for policy 1, policy_version 117577 (0.0008) [2023-12-27 06:28:26,864][20381] Updated weights for policy 1, policy_version 117587 (0.0008) [2023-12-27 06:28:26,920][20347] Updated weights for policy 0, policy_version 117999 (0.0010) [2023-12-27 06:28:26,921][20381] Updated weights for policy 1, policy_version 117597 (0.0007) [2023-12-27 06:28:26,971][20347] Updated weights for policy 0, policy_version 118009 (0.0010) [2023-12-27 06:28:27,029][20347] Updated weights for policy 0, policy_version 118019 (0.0007) [2023-12-27 06:28:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 60342272. Throughput: 0: 10410.0, 1: 10560.9. Samples: 60330188. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:27,563][20381] Updated weights for policy 1, policy_version 117607 (0.0008) [2023-12-27 06:28:27,620][20381] Updated weights for policy 1, policy_version 117617 (0.0005) [2023-12-27 06:28:27,638][20347] Updated weights for policy 0, policy_version 118029 (0.0005) [2023-12-27 06:28:27,672][20381] Updated weights for policy 1, policy_version 117627 (0.0006) [2023-12-27 06:28:27,690][20347] Updated weights for policy 0, policy_version 118039 (0.0005) [2023-12-27 06:28:27,745][20347] Updated weights for policy 0, policy_version 118049 (0.0005) [2023-12-27 06:28:28,267][20381] Updated weights for policy 1, policy_version 117637 (0.0006) [2023-12-27 06:28:28,310][20381] Updated weights for policy 1, policy_version 117647 (0.0005) [2023-12-27 06:28:28,364][20381] Updated weights for policy 1, policy_version 117657 (0.0006) [2023-12-27 06:28:28,382][20347] Updated weights for policy 0, policy_version 118059 (0.0007) [2023-12-27 06:28:28,433][20347] Updated weights for policy 0, policy_version 118069 (0.0010) [2023-12-27 06:28:28,484][20347] Updated weights for policy 0, policy_version 118079 (0.0010) [2023-12-27 06:28:29,002][20381] Updated weights for policy 1, policy_version 117667 (0.0006) [2023-12-27 06:28:29,057][20381] Updated weights for policy 1, policy_version 117677 (0.0008) [2023-12-27 06:28:29,108][20381] Updated weights for policy 1, policy_version 117687 (0.0007) [2023-12-27 06:28:29,177][20347] Updated weights for policy 0, policy_version 118089 (0.0010) [2023-12-27 06:28:29,234][20347] Updated weights for policy 0, policy_version 118099 (0.0009) [2023-12-27 06:28:29,292][20347] Updated weights for policy 0, policy_version 118109 (0.0008) [2023-12-27 06:28:29,353][20347] Updated weights for policy 0, policy_version 118119 (0.0008) [2023-12-27 06:28:29,727][20381] Updated weights for policy 1, policy_version 117697 (0.0007) [2023-12-27 06:28:29,787][20381] Updated weights for policy 1, policy_version 117707 (0.0008) [2023-12-27 06:28:29,847][20381] Updated weights for policy 1, policy_version 117717 (0.0008) [2023-12-27 06:28:29,904][20381] Updated weights for policy 1, policy_version 117727 (0.0006) [2023-12-27 06:28:30,058][20347] Updated weights for policy 0, policy_version 118129 (0.0010) [2023-12-27 06:28:30,109][20347] Updated weights for policy 0, policy_version 118139 (0.0008) [2023-12-27 06:28:30,161][20347] Updated weights for policy 0, policy_version 118149 (0.0009) [2023-12-27 06:28:30,532][20381] Updated weights for policy 1, policy_version 117737 (0.0010) [2023-12-27 06:28:30,583][20381] Updated weights for policy 1, policy_version 117747 (0.0010) [2023-12-27 06:28:30,649][20381] Updated weights for policy 1, policy_version 117757 (0.0005) [2023-12-27 06:28:30,866][20347] Updated weights for policy 0, policy_version 118159 (0.0010) [2023-12-27 06:28:30,916][20347] Updated weights for policy 0, policy_version 118169 (0.0010) [2023-12-27 06:28:30,964][20347] Updated weights for policy 0, policy_version 118179 (0.0010) [2023-12-27 06:28:31,290][20381] Updated weights for policy 1, policy_version 117767 (0.0008) [2023-12-27 06:28:31,348][20381] Updated weights for policy 1, policy_version 117777 (0.0007) [2023-12-27 06:28:31,415][20381] Updated weights for policy 1, policy_version 117787 (0.0010) [2023-12-27 06:28:31,680][20347] Updated weights for policy 0, policy_version 118189 (0.0009) [2023-12-27 06:28:31,742][20347] Updated weights for policy 0, policy_version 118199 (0.0008) [2023-12-27 06:28:31,808][20347] Updated weights for policy 0, policy_version 118209 (0.0007) [2023-12-27 06:28:32,121][20381] Updated weights for policy 1, policy_version 117797 (0.0009) [2023-12-27 06:28:32,179][20381] Updated weights for policy 1, policy_version 117807 (0.0008) [2023-12-27 06:28:32,241][20381] Updated weights for policy 1, policy_version 117817 (0.0007) [2023-12-27 06:28:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 60448768. Throughput: 0: 10390.8, 1: 10555.8. Samples: 60456472. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:32,523][20347] Updated weights for policy 0, policy_version 118219 (0.0007) [2023-12-27 06:28:32,581][20347] Updated weights for policy 0, policy_version 118229 (0.0008) [2023-12-27 06:28:32,634][20347] Updated weights for policy 0, policy_version 118239 (0.0007) [2023-12-27 06:28:32,863][20381] Updated weights for policy 1, policy_version 117827 (0.0007) [2023-12-27 06:28:32,920][20381] Updated weights for policy 1, policy_version 117837 (0.0010) [2023-12-27 06:28:32,982][20381] Updated weights for policy 1, policy_version 117847 (0.0011) [2023-12-27 06:28:33,338][20347] Updated weights for policy 0, policy_version 118249 (0.0007) [2023-12-27 06:28:33,390][20347] Updated weights for policy 0, policy_version 118259 (0.0007) [2023-12-27 06:28:33,434][20347] Updated weights for policy 0, policy_version 118269 (0.0007) [2023-12-27 06:28:33,481][20347] Updated weights for policy 0, policy_version 118279 (0.0007) [2023-12-27 06:28:33,627][20381] Updated weights for policy 1, policy_version 117857 (0.0010) [2023-12-27 06:28:33,686][20381] Updated weights for policy 1, policy_version 117867 (0.0010) [2023-12-27 06:28:33,740][20381] Updated weights for policy 1, policy_version 117877 (0.0010) [2023-12-27 06:28:33,806][20381] Updated weights for policy 1, policy_version 117887 (0.0010) [2023-12-27 06:28:34,163][20347] Updated weights for policy 0, policy_version 118289 (0.0007) [2023-12-27 06:28:34,224][20347] Updated weights for policy 0, policy_version 118299 (0.0007) [2023-12-27 06:28:34,285][20347] Updated weights for policy 0, policy_version 118309 (0.0007) [2023-12-27 06:28:34,500][20381] Updated weights for policy 1, policy_version 117897 (0.0011) [2023-12-27 06:28:34,564][20381] Updated weights for policy 1, policy_version 117907 (0.0011) [2023-12-27 06:28:34,622][20381] Updated weights for policy 1, policy_version 117917 (0.0008) [2023-12-27 06:28:34,976][20347] Updated weights for policy 0, policy_version 118319 (0.0007) [2023-12-27 06:28:35,032][20347] Updated weights for policy 0, policy_version 118329 (0.0008) [2023-12-27 06:28:35,085][20347] Updated weights for policy 0, policy_version 118339 (0.0008) [2023-12-27 06:28:35,181][20381] Updated weights for policy 1, policy_version 117927 (0.0006) [2023-12-27 06:28:35,239][20381] Updated weights for policy 1, policy_version 117937 (0.0007) [2023-12-27 06:28:35,296][20381] Updated weights for policy 1, policy_version 117947 (0.0010) [2023-12-27 06:28:35,770][20347] Updated weights for policy 0, policy_version 118349 (0.0007) [2023-12-27 06:28:35,820][20347] Updated weights for policy 0, policy_version 118359 (0.0007) [2023-12-27 06:28:35,867][20347] Updated weights for policy 0, policy_version 118369 (0.0007) [2023-12-27 06:28:35,906][20381] Updated weights for policy 1, policy_version 117957 (0.0010) [2023-12-27 06:28:35,967][20381] Updated weights for policy 1, policy_version 117967 (0.0010) [2023-12-27 06:28:36,027][20381] Updated weights for policy 1, policy_version 117977 (0.0010) [2023-12-27 06:28:36,528][20347] Updated weights for policy 0, policy_version 118379 (0.0007) [2023-12-27 06:28:36,589][20347] Updated weights for policy 0, policy_version 118389 (0.0007) [2023-12-27 06:28:36,644][20347] Updated weights for policy 0, policy_version 118399 (0.0009) [2023-12-27 06:28:36,687][20381] Updated weights for policy 1, policy_version 117987 (0.0009) [2023-12-27 06:28:36,739][20381] Updated weights for policy 1, policy_version 117997 (0.0006) [2023-12-27 06:28:36,791][20381] Updated weights for policy 1, policy_version 118007 (0.0007) [2023-12-27 06:28:37,359][20347] Updated weights for policy 0, policy_version 118409 (0.0009) [2023-12-27 06:28:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 60547072. Throughput: 0: 10323.4, 1: 10567.0. Samples: 60519068. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000118016_30220288.pth... [2023-12-27 06:28:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000116800_29908992.pth [2023-12-27 06:28:37,399][20381] Updated weights for policy 1, policy_version 118017 (0.0007) [2023-12-27 06:28:37,421][20347] Updated weights for policy 0, policy_version 118419 (0.0007) [2023-12-27 06:28:37,457][20381] Updated weights for policy 1, policy_version 118027 (0.0007) [2023-12-27 06:28:37,478][20347] Updated weights for policy 0, policy_version 118429 (0.0006) [2023-12-27 06:28:37,515][20381] Updated weights for policy 1, policy_version 118037 (0.0007) [2023-12-27 06:28:37,540][20347] Updated weights for policy 0, policy_version 118439 (0.0008) [2023-12-27 06:28:37,542][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000118440_30334976.pth... [2023-12-27 06:28:37,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000117224_30023680.pth [2023-12-27 06:28:37,574][20381] Updated weights for policy 1, policy_version 118047 (0.0007) [2023-12-27 06:28:38,199][20347] Updated weights for policy 0, policy_version 118449 (0.0007) [2023-12-27 06:28:38,239][20381] Updated weights for policy 1, policy_version 118057 (0.0006) [2023-12-27 06:28:38,245][20347] Updated weights for policy 0, policy_version 118459 (0.0006) [2023-12-27 06:28:38,289][20381] Updated weights for policy 1, policy_version 118067 (0.0005) [2023-12-27 06:28:38,291][20347] Updated weights for policy 0, policy_version 118469 (0.0006) [2023-12-27 06:28:38,351][20381] Updated weights for policy 1, policy_version 118077 (0.0007) [2023-12-27 06:28:39,013][20347] Updated weights for policy 0, policy_version 118479 (0.0007) [2023-12-27 06:28:39,023][20381] Updated weights for policy 1, policy_version 118087 (0.0006) [2023-12-27 06:28:39,071][20347] Updated weights for policy 0, policy_version 118489 (0.0007) [2023-12-27 06:28:39,079][20381] Updated weights for policy 1, policy_version 118097 (0.0005) [2023-12-27 06:28:39,133][20347] Updated weights for policy 0, policy_version 118499 (0.0007) [2023-12-27 06:28:39,135][20381] Updated weights for policy 1, policy_version 118107 (0.0006) [2023-12-27 06:28:39,819][20381] Updated weights for policy 1, policy_version 118117 (0.0008) [2023-12-27 06:28:39,844][20347] Updated weights for policy 0, policy_version 118509 (0.0006) [2023-12-27 06:28:39,885][20381] Updated weights for policy 1, policy_version 118127 (0.0006) [2023-12-27 06:28:39,902][20347] Updated weights for policy 0, policy_version 118519 (0.0006) [2023-12-27 06:28:39,950][20381] Updated weights for policy 1, policy_version 118137 (0.0007) [2023-12-27 06:28:39,963][20347] Updated weights for policy 0, policy_version 118529 (0.0006) [2023-12-27 06:28:40,613][20381] Updated weights for policy 1, policy_version 118147 (0.0008) [2023-12-27 06:28:40,666][20381] Updated weights for policy 1, policy_version 118157 (0.0010) [2023-12-27 06:28:40,689][20347] Updated weights for policy 0, policy_version 118539 (0.0006) [2023-12-27 06:28:40,724][20381] Updated weights for policy 1, policy_version 118167 (0.0011) [2023-12-27 06:28:40,738][20347] Updated weights for policy 0, policy_version 118549 (0.0005) [2023-12-27 06:28:40,794][20347] Updated weights for policy 0, policy_version 118559 (0.0006) [2023-12-27 06:28:41,423][20381] Updated weights for policy 1, policy_version 118177 (0.0011) [2023-12-27 06:28:41,486][20381] Updated weights for policy 1, policy_version 118187 (0.0008) [2023-12-27 06:28:41,516][20347] Updated weights for policy 0, policy_version 118569 (0.0008) [2023-12-27 06:28:41,548][20381] Updated weights for policy 1, policy_version 118197 (0.0007) [2023-12-27 06:28:41,566][20347] Updated weights for policy 0, policy_version 118579 (0.0006) [2023-12-27 06:28:41,614][20381] Updated weights for policy 1, policy_version 118207 (0.0008) [2023-12-27 06:28:41,622][20347] Updated weights for policy 0, policy_version 118589 (0.0006) [2023-12-27 06:28:41,674][20347] Updated weights for policy 0, policy_version 118599 (0.0008) [2023-12-27 06:28:42,260][20381] Updated weights for policy 1, policy_version 118217 (0.0007) [2023-12-27 06:28:42,326][20381] Updated weights for policy 1, policy_version 118227 (0.0007) [2023-12-27 06:28:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 60645376. Throughput: 0: 10259.8, 1: 10517.5. Samples: 60641516. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:42,394][20381] Updated weights for policy 1, policy_version 118237 (0.0007) [2023-12-27 06:28:42,435][20347] Updated weights for policy 0, policy_version 118609 (0.0006) [2023-12-27 06:28:42,495][20347] Updated weights for policy 0, policy_version 118619 (0.0005) [2023-12-27 06:28:42,543][20347] Updated weights for policy 0, policy_version 118629 (0.0007) [2023-12-27 06:28:43,114][20381] Updated weights for policy 1, policy_version 118247 (0.0008) [2023-12-27 06:28:43,174][20381] Updated weights for policy 1, policy_version 118257 (0.0008) [2023-12-27 06:28:43,196][20347] Updated weights for policy 0, policy_version 118639 (0.0006) [2023-12-27 06:28:43,235][20381] Updated weights for policy 1, policy_version 118267 (0.0007) [2023-12-27 06:28:43,257][20347] Updated weights for policy 0, policy_version 118649 (0.0007) [2023-12-27 06:28:43,302][20347] Updated weights for policy 0, policy_version 118659 (0.0006) [2023-12-27 06:28:43,889][20381] Updated weights for policy 1, policy_version 118277 (0.0007) [2023-12-27 06:28:43,940][20381] Updated weights for policy 1, policy_version 118287 (0.0010) [2023-12-27 06:28:43,942][20347] Updated weights for policy 0, policy_version 118669 (0.0005) [2023-12-27 06:28:43,991][20347] Updated weights for policy 0, policy_version 118679 (0.0007) [2023-12-27 06:28:43,994][20381] Updated weights for policy 1, policy_version 118297 (0.0006) [2023-12-27 06:28:44,032][20347] Updated weights for policy 0, policy_version 118689 (0.0007) [2023-12-27 06:28:44,637][20381] Updated weights for policy 1, policy_version 118307 (0.0010) [2023-12-27 06:28:44,698][20381] Updated weights for policy 1, policy_version 118317 (0.0011) [2023-12-27 06:28:44,724][20347] Updated weights for policy 0, policy_version 118699 (0.0007) [2023-12-27 06:28:44,762][20381] Updated weights for policy 1, policy_version 118327 (0.0011) [2023-12-27 06:28:44,791][20347] Updated weights for policy 0, policy_version 118709 (0.0008) [2023-12-27 06:28:44,848][20347] Updated weights for policy 0, policy_version 118719 (0.0008) [2023-12-27 06:28:45,495][20347] Updated weights for policy 0, policy_version 118729 (0.0006) [2023-12-27 06:28:45,504][20381] Updated weights for policy 1, policy_version 118338 (0.0009) [2023-12-27 06:28:45,551][20347] Updated weights for policy 0, policy_version 118739 (0.0005) [2023-12-27 06:28:45,565][20381] Updated weights for policy 1, policy_version 118348 (0.0006) [2023-12-27 06:28:45,611][20347] Updated weights for policy 0, policy_version 118749 (0.0006) [2023-12-27 06:28:45,625][20381] Updated weights for policy 1, policy_version 118358 (0.0006) [2023-12-27 06:28:45,671][20347] Updated weights for policy 0, policy_version 118759 (0.0006) [2023-12-27 06:28:45,679][20381] Updated weights for policy 1, policy_version 118368 (0.0007) [2023-12-27 06:28:46,346][20381] Updated weights for policy 1, policy_version 118378 (0.0005) [2023-12-27 06:28:46,354][20347] Updated weights for policy 0, policy_version 118769 (0.0007) [2023-12-27 06:28:46,401][20381] Updated weights for policy 1, policy_version 118388 (0.0005) [2023-12-27 06:28:46,410][20347] Updated weights for policy 0, policy_version 118779 (0.0008) [2023-12-27 06:28:46,453][20381] Updated weights for policy 1, policy_version 118398 (0.0007) [2023-12-27 06:28:46,461][20347] Updated weights for policy 0, policy_version 118789 (0.0008) [2023-12-27 06:28:47,081][20381] Updated weights for policy 1, policy_version 118408 (0.0008) [2023-12-27 06:28:47,137][20381] Updated weights for policy 1, policy_version 118418 (0.0008) [2023-12-27 06:28:47,165][20347] Updated weights for policy 0, policy_version 118799 (0.0007) [2023-12-27 06:28:47,186][20381] Updated weights for policy 1, policy_version 118428 (0.0008) [2023-12-27 06:28:47,225][20347] Updated weights for policy 0, policy_version 118809 (0.0006) [2023-12-27 06:28:47,285][20347] Updated weights for policy 0, policy_version 118819 (0.0007) [2023-12-27 06:28:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.5, 300 sec: 20799.4). Total num frames: 60760064. Throughput: 0: 10269.0, 1: 10491.5. Samples: 60763140. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:47,851][20381] Updated weights for policy 1, policy_version 118438 (0.0010) [2023-12-27 06:28:47,904][20347] Updated weights for policy 0, policy_version 118829 (0.0007) [2023-12-27 06:28:47,915][20381] Updated weights for policy 1, policy_version 118448 (0.0005) [2023-12-27 06:28:47,966][20347] Updated weights for policy 0, policy_version 118839 (0.0006) [2023-12-27 06:28:47,976][20381] Updated weights for policy 1, policy_version 118458 (0.0005) [2023-12-27 06:28:48,017][20347] Updated weights for policy 0, policy_version 118849 (0.0007) [2023-12-27 06:28:48,630][20381] Updated weights for policy 1, policy_version 118468 (0.0008) [2023-12-27 06:28:48,692][20381] Updated weights for policy 1, policy_version 118478 (0.0007) [2023-12-27 06:28:48,726][20347] Updated weights for policy 0, policy_version 118859 (0.0007) [2023-12-27 06:28:48,740][20381] Updated weights for policy 1, policy_version 118488 (0.0007) [2023-12-27 06:28:48,779][20347] Updated weights for policy 0, policy_version 118869 (0.0008) [2023-12-27 06:28:48,824][20347] Updated weights for policy 0, policy_version 118879 (0.0007) [2023-12-27 06:28:49,444][20381] Updated weights for policy 1, policy_version 118498 (0.0008) [2023-12-27 06:28:49,500][20381] Updated weights for policy 1, policy_version 118508 (0.0007) [2023-12-27 06:28:49,537][20347] Updated weights for policy 0, policy_version 118889 (0.0008) [2023-12-27 06:28:49,554][20381] Updated weights for policy 1, policy_version 118518 (0.0007) [2023-12-27 06:28:49,587][20347] Updated weights for policy 0, policy_version 118899 (0.0005) [2023-12-27 06:28:49,610][20381] Updated weights for policy 1, policy_version 118528 (0.0008) [2023-12-27 06:28:49,638][20347] Updated weights for policy 0, policy_version 118909 (0.0007) [2023-12-27 06:28:49,684][20347] Updated weights for policy 0, policy_version 118919 (0.0007) [2023-12-27 06:28:50,361][20347] Updated weights for policy 0, policy_version 118929 (0.0006) [2023-12-27 06:28:50,371][20381] Updated weights for policy 1, policy_version 118538 (0.0008) [2023-12-27 06:28:50,410][20347] Updated weights for policy 0, policy_version 118939 (0.0005) [2023-12-27 06:28:50,426][20381] Updated weights for policy 1, policy_version 118548 (0.0007) [2023-12-27 06:28:50,457][20347] Updated weights for policy 0, policy_version 118949 (0.0007) [2023-12-27 06:28:50,474][20381] Updated weights for policy 1, policy_version 118558 (0.0006) [2023-12-27 06:28:51,115][20347] Updated weights for policy 0, policy_version 118959 (0.0007) [2023-12-27 06:28:51,170][20347] Updated weights for policy 0, policy_version 118969 (0.0007) [2023-12-27 06:28:51,194][20381] Updated weights for policy 1, policy_version 118568 (0.0006) [2023-12-27 06:28:51,223][20347] Updated weights for policy 0, policy_version 118979 (0.0005) [2023-12-27 06:28:51,253][20381] Updated weights for policy 1, policy_version 118578 (0.0006) [2023-12-27 06:28:51,314][20381] Updated weights for policy 1, policy_version 118588 (0.0008) [2023-12-27 06:28:51,925][20347] Updated weights for policy 0, policy_version 118989 (0.0009) [2023-12-27 06:28:51,975][20381] Updated weights for policy 1, policy_version 118598 (0.0007) [2023-12-27 06:28:51,988][20347] Updated weights for policy 0, policy_version 118999 (0.0006) [2023-12-27 06:28:52,025][20381] Updated weights for policy 1, policy_version 118608 (0.0007) [2023-12-27 06:28:52,045][20347] Updated weights for policy 0, policy_version 119009 (0.0007) [2023-12-27 06:28:52,083][20381] Updated weights for policy 1, policy_version 118618 (0.0006) [2023-12-27 06:28:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 60858368. Throughput: 0: 10274.7, 1: 10414.8. Samples: 60824052. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:28:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000118624_30375936.pth... [2023-12-27 06:28:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000119016_30482432.pth... [2023-12-27 06:28:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000117800_30171136.pth [2023-12-27 06:28:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000117408_30064640.pth [2023-12-27 06:28:52,696][20347] Updated weights for policy 0, policy_version 119019 (0.0007) [2023-12-27 06:28:52,742][20381] Updated weights for policy 1, policy_version 118628 (0.0006) [2023-12-27 06:28:52,760][20347] Updated weights for policy 0, policy_version 119029 (0.0005) [2023-12-27 06:28:52,797][20381] Updated weights for policy 1, policy_version 118638 (0.0007) [2023-12-27 06:28:52,818][20347] Updated weights for policy 0, policy_version 119039 (0.0009) [2023-12-27 06:28:52,858][20381] Updated weights for policy 1, policy_version 118648 (0.0008) [2023-12-27 06:28:53,408][20347] Updated weights for policy 0, policy_version 119049 (0.0006) [2023-12-27 06:28:53,466][20347] Updated weights for policy 0, policy_version 119059 (0.0007) [2023-12-27 06:28:53,498][20381] Updated weights for policy 1, policy_version 118658 (0.0008) [2023-12-27 06:28:53,520][20347] Updated weights for policy 0, policy_version 119069 (0.0005) [2023-12-27 06:28:53,557][20381] Updated weights for policy 1, policy_version 118668 (0.0006) [2023-12-27 06:28:53,574][20347] Updated weights for policy 0, policy_version 119079 (0.0006) [2023-12-27 06:28:53,617][20381] Updated weights for policy 1, policy_version 118678 (0.0007) [2023-12-27 06:28:53,672][20381] Updated weights for policy 1, policy_version 118688 (0.0008) [2023-12-27 06:28:54,196][20347] Updated weights for policy 0, policy_version 119089 (0.0010) [2023-12-27 06:28:54,250][20347] Updated weights for policy 0, policy_version 119099 (0.0010) [2023-12-27 06:28:54,312][20347] Updated weights for policy 0, policy_version 119109 (0.0010) [2023-12-27 06:28:54,362][20381] Updated weights for policy 1, policy_version 118698 (0.0010) [2023-12-27 06:28:54,404][20381] Updated weights for policy 1, policy_version 118708 (0.0010) [2023-12-27 06:28:54,458][20381] Updated weights for policy 1, policy_version 118718 (0.0010) [2023-12-27 06:28:54,980][20347] Updated weights for policy 0, policy_version 119119 (0.0008) [2023-12-27 06:28:55,030][20347] Updated weights for policy 0, policy_version 119129 (0.0010) [2023-12-27 06:28:55,079][20347] Updated weights for policy 0, policy_version 119139 (0.0010) [2023-12-27 06:28:55,124][20381] Updated weights for policy 1, policy_version 118728 (0.0006) [2023-12-27 06:28:55,178][20381] Updated weights for policy 1, policy_version 118738 (0.0005) [2023-12-27 06:28:55,237][20381] Updated weights for policy 1, policy_version 118748 (0.0005) [2023-12-27 06:28:55,736][20347] Updated weights for policy 0, policy_version 119149 (0.0009) [2023-12-27 06:28:55,787][20347] Updated weights for policy 0, policy_version 119159 (0.0008) [2023-12-27 06:28:55,845][20347] Updated weights for policy 0, policy_version 119169 (0.0009) [2023-12-27 06:28:55,875][20381] Updated weights for policy 1, policy_version 118758 (0.0005) [2023-12-27 06:28:55,921][20381] Updated weights for policy 1, policy_version 118768 (0.0005) [2023-12-27 06:28:55,968][20381] Updated weights for policy 1, policy_version 118778 (0.0006) [2023-12-27 06:28:56,456][20347] Updated weights for policy 0, policy_version 119179 (0.0007) [2023-12-27 06:28:56,509][20347] Updated weights for policy 0, policy_version 119189 (0.0010) [2023-12-27 06:28:56,564][20381] Updated weights for policy 1, policy_version 118788 (0.0010) [2023-12-27 06:28:56,569][20347] Updated weights for policy 0, policy_version 119199 (0.0010) [2023-12-27 06:28:56,620][20381] Updated weights for policy 1, policy_version 118798 (0.0011) [2023-12-27 06:28:56,676][20381] Updated weights for policy 1, policy_version 118808 (0.0006) [2023-12-27 06:28:57,234][20347] Updated weights for policy 0, policy_version 119209 (0.0011) [2023-12-27 06:28:57,292][20381] Updated weights for policy 1, policy_version 118818 (0.0005) [2023-12-27 06:28:57,294][20347] Updated weights for policy 0, policy_version 119219 (0.0010) [2023-12-27 06:28:57,309][20292] KL-divergence is very high: 132.2063 [2023-12-27 06:28:57,346][20381] Updated weights for policy 1, policy_version 118828 (0.0007) [2023-12-27 06:28:57,349][20347] Updated weights for policy 0, policy_version 119229 (0.0010) [2023-12-27 06:28:57,352][20292] KL-divergence is very high: 182.2248 [2023-12-27 06:28:57,359][20292] KL-divergence is very high: 284.5633 [2023-12-27 06:28:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 60956672. Throughput: 0: 10248.8, 1: 10408.3. Samples: 60949660. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:28:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '724.660')] [2023-12-27 06:28:57,385][20292] KL-divergence is very high: 117.2076 [2023-12-27 06:28:57,395][20292] KL-divergence is very high: 177.7344 [2023-12-27 06:28:57,400][20292] KL-divergence is very high: 231.6406 [2023-12-27 06:28:57,400][20347] Updated weights for policy 0, policy_version 119239 (0.0010) [2023-12-27 06:28:57,402][20381] Updated weights for policy 1, policy_version 118838 (0.0006) [2023-12-27 06:28:57,410][20292] KL-divergence is very high: 194.4348 [2023-12-27 06:28:57,415][20292] KL-divergence is very high: 210.7102 [2023-12-27 06:28:57,420][20292] KL-divergence is very high: 151.2842 [2023-12-27 06:28:57,452][20381] Updated weights for policy 1, policy_version 118848 (0.0007) [2023-12-27 06:28:57,993][20292] KL-divergence is very high: 267.4476 [2023-12-27 06:28:57,999][20292] KL-divergence is very high: 1128.5292 [2023-12-27 06:28:58,005][20292] KL-divergence is very high: 584.5060 [2023-12-27 06:28:58,010][20292] KL-divergence is very high: 343.9464 [2023-12-27 06:28:58,016][20292] KL-divergence is very high: 269.6596 [2023-12-27 06:28:58,022][20292] KL-divergence is very high: 160.6027 [2023-12-27 06:28:58,037][20292] KL-divergence is very high: 162.7824 [2023-12-27 06:28:58,043][20292] KL-divergence is very high: 617.8777 [2023-12-27 06:28:58,047][20381] Updated weights for policy 1, policy_version 118858 (0.0006) [2023-12-27 06:28:58,048][20292] KL-divergence is very high: 368.1576 [2023-12-27 06:28:58,053][20292] KL-divergence is very high: 297.2290 [2023-12-27 06:28:58,059][20292] KL-divergence is very high: 254.3307 [2023-12-27 06:28:58,061][20347] Updated weights for policy 0, policy_version 119249 (0.0008) [2023-12-27 06:28:58,063][20292] KL-divergence is very high: 104.3158 [2023-12-27 06:28:58,078][20292] KL-divergence is very high: 183.5876 [2023-12-27 06:28:58,084][20292] KL-divergence is very high: 594.3633 [2023-12-27 06:28:58,088][20292] KL-divergence is very high: 311.8260 [2023-12-27 06:28:58,094][20292] KL-divergence is very high: 227.9101 [2023-12-27 06:28:58,098][20381] Updated weights for policy 1, policy_version 118868 (0.0010) [2023-12-27 06:28:58,099][20292] KL-divergence is very high: 233.0697 [2023-12-27 06:28:58,104][20292] KL-divergence is very high: 109.2920 [2023-12-27 06:28:58,112][20347] Updated weights for policy 0, policy_version 119259 (0.0009) [2023-12-27 06:28:58,118][20292] KL-divergence is very high: 244.0676 [2023-12-27 06:28:58,123][20292] KL-divergence is very high: 661.9631 [2023-12-27 06:28:58,128][20292] KL-divergence is very high: 372.7006 [2023-12-27 06:28:58,133][20292] KL-divergence is very high: 245.6031 [2023-12-27 06:28:58,139][20292] KL-divergence is very high: 284.6706 [2023-12-27 06:28:58,144][20292] KL-divergence is very high: 130.6323 [2023-12-27 06:28:58,151][20381] Updated weights for policy 1, policy_version 118878 (0.0010) [2023-12-27 06:28:58,174][20347] Updated weights for policy 0, policy_version 119269 (0.0006) [2023-12-27 06:28:58,891][20381] Updated weights for policy 1, policy_version 118888 (0.0009) [2023-12-27 06:28:58,927][20292] KL-divergence is very high: 160.6906 [2023-12-27 06:28:58,940][20292] KL-divergence is very high: 118.9225 [2023-12-27 06:28:58,951][20381] Updated weights for policy 1, policy_version 118898 (0.0011) [2023-12-27 06:28:59,001][20347] Updated weights for policy 0, policy_version 119279 (0.0010) [2023-12-27 06:28:59,012][20381] Updated weights for policy 1, policy_version 118908 (0.0011) [2023-12-27 06:28:59,062][20347] Updated weights for policy 0, policy_version 119289 (0.0010) [2023-12-27 06:28:59,124][20347] Updated weights for policy 0, policy_version 119299 (0.0010) [2023-12-27 06:28:59,689][20292] KL-divergence is very high: 130.4690 [2023-12-27 06:28:59,717][20381] Updated weights for policy 1, policy_version 118918 (0.0009) [2023-12-27 06:28:59,772][20381] Updated weights for policy 1, policy_version 118928 (0.0008) [2023-12-27 06:28:59,815][20347] Updated weights for policy 0, policy_version 119309 (0.0008) [2023-12-27 06:28:59,828][20381] Updated weights for policy 1, policy_version 118938 (0.0008) [2023-12-27 06:28:59,874][20347] Updated weights for policy 0, policy_version 119319 (0.0008) [2023-12-27 06:28:59,936][20347] Updated weights for policy 0, policy_version 119329 (0.0008) [2023-12-27 06:29:00,441][20381] Updated weights for policy 1, policy_version 118948 (0.0007) [2023-12-27 06:29:00,499][20381] Updated weights for policy 1, policy_version 118958 (0.0006) [2023-12-27 06:29:00,552][20381] Updated weights for policy 1, policy_version 118968 (0.0005) [2023-12-27 06:29:00,690][20347] Updated weights for policy 0, policy_version 119339 (0.0008) [2023-12-27 06:29:00,734][20347] Updated weights for policy 0, policy_version 119349 (0.0007) [2023-12-27 06:29:00,793][20347] Updated weights for policy 0, policy_version 119359 (0.0007) [2023-12-27 06:29:01,129][20381] Updated weights for policy 1, policy_version 118978 (0.0006) [2023-12-27 06:29:01,189][20381] Updated weights for policy 1, policy_version 118988 (0.0008) [2023-12-27 06:29:01,248][20381] Updated weights for policy 1, policy_version 118998 (0.0007) [2023-12-27 06:29:01,307][20381] Updated weights for policy 1, policy_version 119008 (0.0008) [2023-12-27 06:29:01,506][20347] Updated weights for policy 0, policy_version 119369 (0.0008) [2023-12-27 06:29:01,565][20347] Updated weights for policy 0, policy_version 119379 (0.0008) [2023-12-27 06:29:01,628][20347] Updated weights for policy 0, policy_version 119389 (0.0008) [2023-12-27 06:29:01,679][20347] Updated weights for policy 0, policy_version 119399 (0.0006) [2023-12-27 06:29:01,983][20381] Updated weights for policy 1, policy_version 119018 (0.0005) [2023-12-27 06:29:02,055][20381] Updated weights for policy 1, policy_version 119028 (0.0008) [2023-12-27 06:29:02,115][20381] Updated weights for policy 1, policy_version 119038 (0.0006) [2023-12-27 06:29:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 61063168. Throughput: 0: 10252.5, 1: 10433.9. Samples: 61073452. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:29:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '174.320')] [2023-12-27 06:29:02,368][20347] Updated weights for policy 0, policy_version 119409 (0.0007) [2023-12-27 06:29:02,429][20347] Updated weights for policy 0, policy_version 119419 (0.0010) [2023-12-27 06:29:02,493][20347] Updated weights for policy 0, policy_version 119429 (0.0006) [2023-12-27 06:29:02,724][20381] Updated weights for policy 1, policy_version 119048 (0.0005) [2023-12-27 06:29:02,726][20292] KL-divergence is very high: 103.0685 [2023-12-27 06:29:02,787][20381] Updated weights for policy 1, policy_version 119058 (0.0006) [2023-12-27 06:29:02,848][20381] Updated weights for policy 1, policy_version 119068 (0.0007) [2023-12-27 06:29:03,099][20347] Updated weights for policy 0, policy_version 119439 (0.0009) [2023-12-27 06:29:03,144][20347] Updated weights for policy 0, policy_version 119449 (0.0010) [2023-12-27 06:29:03,195][20347] Updated weights for policy 0, policy_version 119459 (0.0010) [2023-12-27 06:29:03,403][20381] Updated weights for policy 1, policy_version 119078 (0.0006) [2023-12-27 06:29:03,427][20292] KL-divergence is very high: 110.0494 [2023-12-27 06:29:03,461][20381] Updated weights for policy 1, policy_version 119088 (0.0005) [2023-12-27 06:29:03,522][20381] Updated weights for policy 1, policy_version 119098 (0.0005) [2023-12-27 06:29:03,878][20347] Updated weights for policy 0, policy_version 119469 (0.0009) [2023-12-27 06:29:03,925][20347] Updated weights for policy 0, policy_version 119479 (0.0008) [2023-12-27 06:29:03,975][20347] Updated weights for policy 0, policy_version 119489 (0.0008) [2023-12-27 06:29:04,118][20381] Updated weights for policy 1, policy_version 119108 (0.0006) [2023-12-27 06:29:04,177][20381] Updated weights for policy 1, policy_version 119118 (0.0008) [2023-12-27 06:29:04,241][20381] Updated weights for policy 1, policy_version 119128 (0.0008) [2023-12-27 06:29:04,704][20347] Updated weights for policy 0, policy_version 119499 (0.0007) [2023-12-27 06:29:04,766][20347] Updated weights for policy 0, policy_version 119509 (0.0005) [2023-12-27 06:29:04,795][20381] Updated weights for policy 1, policy_version 119138 (0.0009) [2023-12-27 06:29:04,820][20347] Updated weights for policy 0, policy_version 119519 (0.0005) [2023-12-27 06:29:04,842][20381] Updated weights for policy 1, policy_version 119148 (0.0010) [2023-12-27 06:29:04,901][20381] Updated weights for policy 1, policy_version 119158 (0.0005) [2023-12-27 06:29:04,949][20381] Updated weights for policy 1, policy_version 119168 (0.0005) [2023-12-27 06:29:05,420][20347] Updated weights for policy 0, policy_version 119529 (0.0009) [2023-12-27 06:29:05,471][20347] Updated weights for policy 0, policy_version 119539 (0.0007) [2023-12-27 06:29:05,527][20347] Updated weights for policy 0, policy_version 119549 (0.0005) [2023-12-27 06:29:05,587][20347] Updated weights for policy 0, policy_version 119559 (0.0010) [2023-12-27 06:29:05,621][20381] Updated weights for policy 1, policy_version 119178 (0.0006) [2023-12-27 06:29:05,668][20292] KL-divergence is very high: 105.2492 [2023-12-27 06:29:05,687][20381] Updated weights for policy 1, policy_version 119188 (0.0009) [2023-12-27 06:29:05,715][20292] KL-divergence is very high: 145.9029 [2023-12-27 06:29:05,721][20292] KL-divergence is very high: 114.8725 [2023-12-27 06:29:05,748][20381] Updated weights for policy 1, policy_version 119198 (0.0010) [2023-12-27 06:29:06,216][20347] Updated weights for policy 0, policy_version 119569 (0.0006) [2023-12-27 06:29:06,273][20347] Updated weights for policy 0, policy_version 119579 (0.0005) [2023-12-27 06:29:06,330][20347] Updated weights for policy 0, policy_version 119589 (0.0006) [2023-12-27 06:29:06,344][20292] KL-divergence is very high: 112.4805 [2023-12-27 06:29:06,351][20292] KL-divergence is very high: 144.3437 [2023-12-27 06:29:06,382][20381] Updated weights for policy 1, policy_version 119208 (0.0010) [2023-12-27 06:29:06,428][20381] Updated weights for policy 1, policy_version 119218 (0.0010) [2023-12-27 06:29:06,480][20381] Updated weights for policy 1, policy_version 119228 (0.0010) [2023-12-27 06:29:06,946][20347] Updated weights for policy 0, policy_version 119599 (0.0005) [2023-12-27 06:29:06,997][20347] Updated weights for policy 0, policy_version 119609 (0.0005) [2023-12-27 06:29:07,046][20347] Updated weights for policy 0, policy_version 119619 (0.0005) [2023-12-27 06:29:07,142][20381] Updated weights for policy 1, policy_version 119238 (0.0007) [2023-12-27 06:29:07,193][20381] Updated weights for policy 1, policy_version 119248 (0.0007) [2023-12-27 06:29:07,257][20381] Updated weights for policy 1, policy_version 119258 (0.0007) [2023-12-27 06:29:07,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 61177856. Throughput: 0: 10281.5, 1: 10507.0. Samples: 61139740. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 06:29:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '118.780')] [2023-12-27 06:29:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000119264_30539776.pth... [2023-12-27 06:29:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000119624_30638080.pth... [2023-12-27 06:29:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000118440_30334976.pth [2023-12-27 06:29:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000118016_30220288.pth [2023-12-27 06:29:07,657][20347] Updated weights for policy 0, policy_version 119629 (0.0008) [2023-12-27 06:29:07,713][20347] Updated weights for policy 0, policy_version 119639 (0.0005) [2023-12-27 06:29:07,769][20347] Updated weights for policy 0, policy_version 119649 (0.0005) [2023-12-27 06:29:07,876][20381] Updated weights for policy 1, policy_version 119268 (0.0008) [2023-12-27 06:29:07,922][20381] Updated weights for policy 1, policy_version 119278 (0.0005) [2023-12-27 06:29:07,972][20381] Updated weights for policy 1, policy_version 119288 (0.0006) [2023-12-27 06:29:08,372][20347] Updated weights for policy 0, policy_version 119659 (0.0006) [2023-12-27 06:29:08,422][20347] Updated weights for policy 0, policy_version 119669 (0.0008) [2023-12-27 06:29:08,471][20347] Updated weights for policy 0, policy_version 119679 (0.0008) [2023-12-27 06:29:08,664][20381] Updated weights for policy 1, policy_version 119298 (0.0010) [2023-12-27 06:29:08,713][20381] Updated weights for policy 1, policy_version 119308 (0.0010) [2023-12-27 06:29:08,772][20381] Updated weights for policy 1, policy_version 119318 (0.0009) [2023-12-27 06:29:08,828][20381] Updated weights for policy 1, policy_version 119328 (0.0010) [2023-12-27 06:29:09,199][20347] Updated weights for policy 0, policy_version 119689 (0.0008) [2023-12-27 06:29:09,263][20347] Updated weights for policy 0, policy_version 119699 (0.0006) [2023-12-27 06:29:09,314][20347] Updated weights for policy 0, policy_version 119709 (0.0007) [2023-12-27 06:29:09,378][20347] Updated weights for policy 0, policy_version 119719 (0.0006) [2023-12-27 06:29:09,563][20381] Updated weights for policy 1, policy_version 119338 (0.0008) [2023-12-27 06:29:09,585][20292] KL-divergence is very high: 145.3317 [2023-12-27 06:29:09,591][20292] KL-divergence is very high: 139.5961 [2023-12-27 06:29:09,621][20381] Updated weights for policy 1, policy_version 119348 (0.0007) [2023-12-27 06:29:09,632][20292] KL-divergence is very high: 101.9392 [2023-12-27 06:29:09,639][20292] KL-divergence is very high: 139.6578 [2023-12-27 06:29:09,645][20292] KL-divergence is very high: 144.8612 [2023-12-27 06:29:09,684][20381] Updated weights for policy 1, policy_version 119358 (0.0008) [2023-12-27 06:29:10,017][20347] Updated weights for policy 0, policy_version 119729 (0.0008) [2023-12-27 06:29:10,082][20347] Updated weights for policy 0, policy_version 119739 (0.0006) [2023-12-27 06:29:10,137][20347] Updated weights for policy 0, policy_version 119749 (0.0008) [2023-12-27 06:29:10,325][20292] KL-divergence is very high: 119.8153 [2023-12-27 06:29:10,332][20292] KL-divergence is very high: 125.7969 [2023-12-27 06:29:10,337][20292] KL-divergence is very high: 107.1817 [2023-12-27 06:29:10,368][20292] KL-divergence is very high: 117.3200 [2023-12-27 06:29:10,375][20381] Updated weights for policy 1, policy_version 119368 (0.0008) [2023-12-27 06:29:10,375][20292] KL-divergence is very high: 129.0248 [2023-12-27 06:29:10,415][20292] KL-divergence is very high: 114.5954 [2023-12-27 06:29:10,421][20292] KL-divergence is very high: 111.3312 [2023-12-27 06:29:10,434][20381] Updated weights for policy 1, policy_version 119378 (0.0008) [2023-12-27 06:29:10,462][20292] KL-divergence is very high: 138.4936 [2023-12-27 06:29:10,468][20292] KL-divergence is very high: 137.9334 [2023-12-27 06:29:10,493][20381] Updated weights for policy 1, policy_version 119388 (0.0008) [2023-12-27 06:29:10,517][20292] KL-divergence is very high: 142.6606 [2023-12-27 06:29:10,787][20347] Updated weights for policy 0, policy_version 119759 (0.0008) [2023-12-27 06:29:10,844][20347] Updated weights for policy 0, policy_version 119769 (0.0005) [2023-12-27 06:29:10,904][20347] Updated weights for policy 0, policy_version 119779 (0.0006) [2023-12-27 06:29:11,097][20381] Updated weights for policy 1, policy_version 119398 (0.0008) [2023-12-27 06:29:11,161][20381] Updated weights for policy 1, policy_version 119408 (0.0008) [2023-12-27 06:29:11,203][20292] KL-divergence is very high: 125.2703 [2023-12-27 06:29:11,209][20292] KL-divergence is very high: 130.8471 [2023-12-27 06:29:11,221][20381] Updated weights for policy 1, policy_version 119418 (0.0008) [2023-12-27 06:29:11,595][20347] Updated weights for policy 0, policy_version 119789 (0.0009) [2023-12-27 06:29:11,661][20347] Updated weights for policy 0, policy_version 119799 (0.0008) [2023-12-27 06:29:11,724][20347] Updated weights for policy 0, policy_version 119809 (0.0008) [2023-12-27 06:29:12,008][20381] Updated weights for policy 1, policy_version 119428 (0.0007) [2023-12-27 06:29:12,065][20381] Updated weights for policy 1, policy_version 119438 (0.0007) [2023-12-27 06:29:12,123][20381] Updated weights for policy 1, policy_version 119448 (0.0007) [2023-12-27 06:29:12,168][20292] KL-divergence is very high: 102.1713 [2023-12-27 06:29:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 61276160. Throughput: 0: 10292.9, 1: 10514.5. Samples: 61266520. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.440')] [2023-12-27 06:29:12,420][20347] Updated weights for policy 0, policy_version 119819 (0.0010) [2023-12-27 06:29:12,484][20347] Updated weights for policy 0, policy_version 119829 (0.0008) [2023-12-27 06:29:12,545][20347] Updated weights for policy 0, policy_version 119839 (0.0008) [2023-12-27 06:29:12,759][20381] Updated weights for policy 1, policy_version 119458 (0.0006) [2023-12-27 06:29:12,815][20381] Updated weights for policy 1, policy_version 119468 (0.0006) [2023-12-27 06:29:12,870][20381] Updated weights for policy 1, policy_version 119478 (0.0005) [2023-12-27 06:29:12,932][20381] Updated weights for policy 1, policy_version 119488 (0.0005) [2023-12-27 06:29:13,215][20347] Updated weights for policy 0, policy_version 119849 (0.0008) [2023-12-27 06:29:13,272][20347] Updated weights for policy 0, policy_version 119859 (0.0010) [2023-12-27 06:29:13,320][20347] Updated weights for policy 0, policy_version 119869 (0.0010) [2023-12-27 06:29:13,375][20347] Updated weights for policy 0, policy_version 119879 (0.0010) [2023-12-27 06:29:13,552][20381] Updated weights for policy 1, policy_version 119498 (0.0007) [2023-12-27 06:29:13,609][20381] Updated weights for policy 1, policy_version 119508 (0.0007) [2023-12-27 06:29:13,663][20381] Updated weights for policy 1, policy_version 119518 (0.0007) [2023-12-27 06:29:14,011][20347] Updated weights for policy 0, policy_version 119889 (0.0010) [2023-12-27 06:29:14,070][20347] Updated weights for policy 0, policy_version 119899 (0.0010) [2023-12-27 06:29:14,133][20347] Updated weights for policy 0, policy_version 119909 (0.0010) [2023-12-27 06:29:14,395][20381] Updated weights for policy 1, policy_version 119528 (0.0009) [2023-12-27 06:29:14,452][20381] Updated weights for policy 1, policy_version 119538 (0.0008) [2023-12-27 06:29:14,515][20381] Updated weights for policy 1, policy_version 119548 (0.0008) [2023-12-27 06:29:14,758][20347] Updated weights for policy 0, policy_version 119919 (0.0009) [2023-12-27 06:29:14,812][20347] Updated weights for policy 0, policy_version 119929 (0.0010) [2023-12-27 06:29:14,860][20347] Updated weights for policy 0, policy_version 119939 (0.0009) [2023-12-27 06:29:15,245][20381] Updated weights for policy 1, policy_version 119558 (0.0007) [2023-12-27 06:29:15,305][20381] Updated weights for policy 1, policy_version 119568 (0.0008) [2023-12-27 06:29:15,362][20381] Updated weights for policy 1, policy_version 119578 (0.0008) [2023-12-27 06:29:15,541][20347] Updated weights for policy 0, policy_version 119949 (0.0008) [2023-12-27 06:29:15,597][20347] Updated weights for policy 0, policy_version 119959 (0.0008) [2023-12-27 06:29:15,659][20347] Updated weights for policy 0, policy_version 119969 (0.0010) [2023-12-27 06:29:16,015][20381] Updated weights for policy 1, policy_version 119588 (0.0007) [2023-12-27 06:29:16,067][20381] Updated weights for policy 1, policy_version 119598 (0.0007) [2023-12-27 06:29:16,115][20381] Updated weights for policy 1, policy_version 119608 (0.0007) [2023-12-27 06:29:16,326][20347] Updated weights for policy 0, policy_version 119979 (0.0009) [2023-12-27 06:29:16,381][20347] Updated weights for policy 0, policy_version 119989 (0.0008) [2023-12-27 06:29:16,439][20347] Updated weights for policy 0, policy_version 119999 (0.0008) [2023-12-27 06:29:16,847][20381] Updated weights for policy 1, policy_version 119618 (0.0007) [2023-12-27 06:29:16,893][20381] Updated weights for policy 1, policy_version 119628 (0.0010) [2023-12-27 06:29:16,946][20381] Updated weights for policy 1, policy_version 119638 (0.0009) [2023-12-27 06:29:17,002][20381] Updated weights for policy 1, policy_version 119648 (0.0009) [2023-12-27 06:29:17,060][20347] Updated weights for policy 0, policy_version 120009 (0.0007) [2023-12-27 06:29:17,115][20347] Updated weights for policy 0, policy_version 120019 (0.0007) [2023-12-27 06:29:17,176][20347] Updated weights for policy 0, policy_version 120029 (0.0008) [2023-12-27 06:29:17,236][20347] Updated weights for policy 0, policy_version 120039 (0.0005) [2023-12-27 06:29:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 61382656. Throughput: 0: 10294.8, 1: 10429.1. Samples: 61389048. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.540')] [2023-12-27 06:29:17,742][20381] Updated weights for policy 1, policy_version 119658 (0.0008) [2023-12-27 06:29:17,798][20381] Updated weights for policy 1, policy_version 119668 (0.0007) [2023-12-27 06:29:17,851][20381] Updated weights for policy 1, policy_version 119678 (0.0006) [2023-12-27 06:29:17,856][20347] Updated weights for policy 0, policy_version 120049 (0.0007) [2023-12-27 06:29:17,906][20347] Updated weights for policy 0, policy_version 120059 (0.0006) [2023-12-27 06:29:17,964][20347] Updated weights for policy 0, policy_version 120069 (0.0005) [2023-12-27 06:29:18,615][20381] Updated weights for policy 1, policy_version 119688 (0.0008) [2023-12-27 06:29:18,637][20347] Updated weights for policy 0, policy_version 120079 (0.0007) [2023-12-27 06:29:18,669][20381] Updated weights for policy 1, policy_version 119698 (0.0006) [2023-12-27 06:29:18,699][20347] Updated weights for policy 0, policy_version 120089 (0.0005) [2023-12-27 06:29:18,729][20381] Updated weights for policy 1, policy_version 119708 (0.0007) [2023-12-27 06:29:18,752][20347] Updated weights for policy 0, policy_version 120099 (0.0006) [2023-12-27 06:29:19,313][20347] Updated weights for policy 0, policy_version 120109 (0.0007) [2023-12-27 06:29:19,378][20347] Updated weights for policy 0, policy_version 120119 (0.0006) [2023-12-27 06:29:19,429][20347] Updated weights for policy 0, policy_version 120129 (0.0007) [2023-12-27 06:29:19,484][20381] Updated weights for policy 1, policy_version 119718 (0.0006) [2023-12-27 06:29:19,542][20381] Updated weights for policy 1, policy_version 119728 (0.0007) [2023-12-27 06:29:19,603][20381] Updated weights for policy 1, policy_version 119738 (0.0008) [2023-12-27 06:29:20,108][20347] Updated weights for policy 0, policy_version 120139 (0.0008) [2023-12-27 06:29:20,170][20347] Updated weights for policy 0, policy_version 120149 (0.0008) [2023-12-27 06:29:20,235][20347] Updated weights for policy 0, policy_version 120159 (0.0006) [2023-12-27 06:29:20,282][20381] Updated weights for policy 1, policy_version 119748 (0.0007) [2023-12-27 06:29:20,339][20381] Updated weights for policy 1, policy_version 119758 (0.0006) [2023-12-27 06:29:20,393][20381] Updated weights for policy 1, policy_version 119768 (0.0005) [2023-12-27 06:29:20,825][20347] Updated weights for policy 0, policy_version 120169 (0.0006) [2023-12-27 06:29:20,887][20347] Updated weights for policy 0, policy_version 120179 (0.0006) [2023-12-27 06:29:20,941][20347] Updated weights for policy 0, policy_version 120189 (0.0008) [2023-12-27 06:29:21,006][20347] Updated weights for policy 0, policy_version 120199 (0.0006) [2023-12-27 06:29:21,100][20381] Updated weights for policy 1, policy_version 119778 (0.0007) [2023-12-27 06:29:21,164][20381] Updated weights for policy 1, policy_version 119788 (0.0008) [2023-12-27 06:29:21,230][20381] Updated weights for policy 1, policy_version 119798 (0.0008) [2023-12-27 06:29:21,296][20381] Updated weights for policy 1, policy_version 119808 (0.0006) [2023-12-27 06:29:21,680][20347] Updated weights for policy 0, policy_version 120209 (0.0008) [2023-12-27 06:29:21,738][20347] Updated weights for policy 0, policy_version 120219 (0.0009) [2023-12-27 06:29:21,805][20347] Updated weights for policy 0, policy_version 120229 (0.0007) [2023-12-27 06:29:21,992][20381] Updated weights for policy 1, policy_version 119818 (0.0008) [2023-12-27 06:29:22,056][20381] Updated weights for policy 1, policy_version 119828 (0.0007) [2023-12-27 06:29:22,143][20381] Updated weights for policy 1, policy_version 119838 (0.0006) [2023-12-27 06:29:22,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 61480960. Throughput: 0: 10343.1, 1: 10343.3. Samples: 61449952. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.330')] [2023-12-27 06:29:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000120232_30793728.pth... [2023-12-27 06:29:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000119840_30687232.pth... [2023-12-27 06:29:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000118624_30375936.pth [2023-12-27 06:29:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000119016_30482432.pth [2023-12-27 06:29:22,547][20347] Updated weights for policy 0, policy_version 120239 (0.0006) [2023-12-27 06:29:22,603][20347] Updated weights for policy 0, policy_version 120249 (0.0005) [2023-12-27 06:29:22,664][20347] Updated weights for policy 0, policy_version 120259 (0.0006) [2023-12-27 06:29:22,773][20381] Updated weights for policy 1, policy_version 119848 (0.0010) [2023-12-27 06:29:22,822][20381] Updated weights for policy 1, policy_version 119858 (0.0010) [2023-12-27 06:29:22,874][20381] Updated weights for policy 1, policy_version 119868 (0.0010) [2023-12-27 06:29:23,212][20347] Updated weights for policy 0, policy_version 120269 (0.0005) [2023-12-27 06:29:23,262][20347] Updated weights for policy 0, policy_version 120279 (0.0005) [2023-12-27 06:29:23,315][20347] Updated weights for policy 0, policy_version 120289 (0.0005) [2023-12-27 06:29:23,507][20381] Updated weights for policy 1, policy_version 119878 (0.0011) [2023-12-27 06:29:23,559][20381] Updated weights for policy 1, policy_version 119888 (0.0010) [2023-12-27 06:29:23,608][20381] Updated weights for policy 1, policy_version 119898 (0.0010) [2023-12-27 06:29:23,892][20347] Updated weights for policy 0, policy_version 120299 (0.0007) [2023-12-27 06:29:23,955][20347] Updated weights for policy 0, policy_version 120309 (0.0010) [2023-12-27 06:29:24,019][20347] Updated weights for policy 0, policy_version 120319 (0.0010) [2023-12-27 06:29:24,268][20381] Updated weights for policy 1, policy_version 119908 (0.0010) [2023-12-27 06:29:24,315][20381] Updated weights for policy 1, policy_version 119918 (0.0008) [2023-12-27 06:29:24,370][20381] Updated weights for policy 1, policy_version 119928 (0.0007) [2023-12-27 06:29:24,641][20347] Updated weights for policy 0, policy_version 120329 (0.0010) [2023-12-27 06:29:24,690][20347] Updated weights for policy 0, policy_version 120339 (0.0005) [2023-12-27 06:29:24,743][20347] Updated weights for policy 0, policy_version 120349 (0.0005) [2023-12-27 06:29:24,801][20347] Updated weights for policy 0, policy_version 120359 (0.0007) [2023-12-27 06:29:25,040][20381] Updated weights for policy 1, policy_version 119938 (0.0006) [2023-12-27 06:29:25,092][20381] Updated weights for policy 1, policy_version 119948 (0.0005) [2023-12-27 06:29:25,146][20381] Updated weights for policy 1, policy_version 119958 (0.0007) [2023-12-27 06:29:25,199][20381] Updated weights for policy 1, policy_version 119968 (0.0010) [2023-12-27 06:29:25,395][20347] Updated weights for policy 0, policy_version 120369 (0.0007) [2023-12-27 06:29:25,457][20347] Updated weights for policy 0, policy_version 120379 (0.0005) [2023-12-27 06:29:25,517][20347] Updated weights for policy 0, policy_version 120389 (0.0006) [2023-12-27 06:29:25,868][20381] Updated weights for policy 1, policy_version 119978 (0.0005) [2023-12-27 06:29:25,929][20381] Updated weights for policy 1, policy_version 119988 (0.0008) [2023-12-27 06:29:25,995][20381] Updated weights for policy 1, policy_version 119998 (0.0006) [2023-12-27 06:29:26,056][20347] Updated weights for policy 0, policy_version 120399 (0.0008) [2023-12-27 06:29:26,115][20347] Updated weights for policy 0, policy_version 120409 (0.0008) [2023-12-27 06:29:26,176][20347] Updated weights for policy 0, policy_version 120419 (0.0006) [2023-12-27 06:29:26,627][20381] Updated weights for policy 1, policy_version 120008 (0.0007) [2023-12-27 06:29:26,680][20381] Updated weights for policy 1, policy_version 120018 (0.0007) [2023-12-27 06:29:26,736][20381] Updated weights for policy 1, policy_version 120028 (0.0006) [2023-12-27 06:29:26,769][20347] Updated weights for policy 0, policy_version 120429 (0.0008) [2023-12-27 06:29:26,829][20347] Updated weights for policy 0, policy_version 120439 (0.0009) [2023-12-27 06:29:26,888][20347] Updated weights for policy 0, policy_version 120449 (0.0010) [2023-12-27 06:29:27,362][20381] Updated weights for policy 1, policy_version 120038 (0.0008) [2023-12-27 06:29:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 61587456. Throughput: 0: 10483.8, 1: 10340.6. Samples: 61578612. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.180')] [2023-12-27 06:29:27,411][20381] Updated weights for policy 1, policy_version 120048 (0.0007) [2023-12-27 06:29:27,459][20381] Updated weights for policy 1, policy_version 120058 (0.0007) [2023-12-27 06:29:27,515][20347] Updated weights for policy 0, policy_version 120459 (0.0005) [2023-12-27 06:29:27,560][20347] Updated weights for policy 0, policy_version 120469 (0.0005) [2023-12-27 06:29:27,620][20347] Updated weights for policy 0, policy_version 120479 (0.0006) [2023-12-27 06:29:28,159][20381] Updated weights for policy 1, policy_version 120068 (0.0008) [2023-12-27 06:29:28,207][20381] Updated weights for policy 1, policy_version 120078 (0.0008) [2023-12-27 06:29:28,210][20347] Updated weights for policy 0, policy_version 120489 (0.0009) [2023-12-27 06:29:28,256][20347] Updated weights for policy 0, policy_version 120499 (0.0008) [2023-12-27 06:29:28,263][20381] Updated weights for policy 1, policy_version 120088 (0.0009) [2023-12-27 06:29:28,306][20347] Updated weights for policy 0, policy_version 120509 (0.0010) [2023-12-27 06:29:28,366][20347] Updated weights for policy 0, policy_version 120519 (0.0009) [2023-12-27 06:29:28,964][20381] Updated weights for policy 1, policy_version 120098 (0.0009) [2023-12-27 06:29:29,023][20381] Updated weights for policy 1, policy_version 120108 (0.0006) [2023-12-27 06:29:29,074][20347] Updated weights for policy 0, policy_version 120529 (0.0007) [2023-12-27 06:29:29,080][20381] Updated weights for policy 1, policy_version 120118 (0.0007) [2023-12-27 06:29:29,128][20381] Updated weights for policy 1, policy_version 120128 (0.0007) [2023-12-27 06:29:29,132][20347] Updated weights for policy 0, policy_version 120539 (0.0006) [2023-12-27 06:29:29,189][20347] Updated weights for policy 0, policy_version 120549 (0.0005) [2023-12-27 06:29:29,852][20347] Updated weights for policy 0, policy_version 120559 (0.0010) [2023-12-27 06:29:29,857][20381] Updated weights for policy 1, policy_version 120138 (0.0007) [2023-12-27 06:29:29,911][20347] Updated weights for policy 0, policy_version 120569 (0.0007) [2023-12-27 06:29:29,916][20381] Updated weights for policy 1, policy_version 120148 (0.0007) [2023-12-27 06:29:29,975][20347] Updated weights for policy 0, policy_version 120579 (0.0008) [2023-12-27 06:29:29,986][20381] Updated weights for policy 1, policy_version 120158 (0.0008) [2023-12-27 06:29:30,640][20381] Updated weights for policy 1, policy_version 120168 (0.0007) [2023-12-27 06:29:30,677][20347] Updated weights for policy 0, policy_version 120589 (0.0007) [2023-12-27 06:29:30,687][20381] Updated weights for policy 1, policy_version 120178 (0.0006) [2023-12-27 06:29:30,730][20347] Updated weights for policy 0, policy_version 120599 (0.0006) [2023-12-27 06:29:30,743][20381] Updated weights for policy 1, policy_version 120188 (0.0006) [2023-12-27 06:29:30,813][20347] Updated weights for policy 0, policy_version 120609 (0.0007) [2023-12-27 06:29:31,434][20347] Updated weights for policy 0, policy_version 120619 (0.0007) [2023-12-27 06:29:31,488][20347] Updated weights for policy 0, policy_version 120629 (0.0005) [2023-12-27 06:29:31,497][20381] Updated weights for policy 1, policy_version 120198 (0.0007) [2023-12-27 06:29:31,544][20347] Updated weights for policy 0, policy_version 120639 (0.0006) [2023-12-27 06:29:31,550][20381] Updated weights for policy 1, policy_version 120208 (0.0007) [2023-12-27 06:29:31,604][20381] Updated weights for policy 1, policy_version 120218 (0.0007) [2023-12-27 06:29:32,156][20347] Updated weights for policy 0, policy_version 120649 (0.0005) [2023-12-27 06:29:32,216][20347] Updated weights for policy 0, policy_version 120659 (0.0007) [2023-12-27 06:29:32,274][20347] Updated weights for policy 0, policy_version 120669 (0.0007) [2023-12-27 06:29:32,318][20381] Updated weights for policy 1, policy_version 120228 (0.0007) [2023-12-27 06:29:32,328][20347] Updated weights for policy 0, policy_version 120679 (0.0006) [2023-12-27 06:29:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 61693952. Throughput: 0: 10563.8, 1: 10325.0. Samples: 61703132. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.470')] [2023-12-27 06:29:32,375][20381] Updated weights for policy 1, policy_version 120238 (0.0007) [2023-12-27 06:29:32,433][20381] Updated weights for policy 1, policy_version 120248 (0.0007) [2023-12-27 06:29:32,947][20347] Updated weights for policy 0, policy_version 120689 (0.0007) [2023-12-27 06:29:33,008][20347] Updated weights for policy 0, policy_version 120699 (0.0005) [2023-12-27 06:29:33,070][20381] Updated weights for policy 1, policy_version 120258 (0.0007) [2023-12-27 06:29:33,075][20347] Updated weights for policy 0, policy_version 120709 (0.0005) [2023-12-27 06:29:33,119][20381] Updated weights for policy 1, policy_version 120268 (0.0009) [2023-12-27 06:29:33,173][20381] Updated weights for policy 1, policy_version 120278 (0.0010) [2023-12-27 06:29:33,235][20381] Updated weights for policy 1, policy_version 120288 (0.0010) [2023-12-27 06:29:33,628][20347] Updated weights for policy 0, policy_version 120719 (0.0007) [2023-12-27 06:29:33,677][20347] Updated weights for policy 0, policy_version 120729 (0.0006) [2023-12-27 06:29:33,730][20347] Updated weights for policy 0, policy_version 120739 (0.0005) [2023-12-27 06:29:33,944][20381] Updated weights for policy 1, policy_version 120298 (0.0011) [2023-12-27 06:29:33,997][20381] Updated weights for policy 1, policy_version 120308 (0.0007) [2023-12-27 06:29:34,053][20381] Updated weights for policy 1, policy_version 120318 (0.0005) [2023-12-27 06:29:34,333][20347] Updated weights for policy 0, policy_version 120749 (0.0008) [2023-12-27 06:29:34,394][20347] Updated weights for policy 0, policy_version 120759 (0.0010) [2023-12-27 06:29:34,457][20347] Updated weights for policy 0, policy_version 120769 (0.0007) [2023-12-27 06:29:34,704][20381] Updated weights for policy 1, policy_version 120328 (0.0007) [2023-12-27 06:29:34,764][20381] Updated weights for policy 1, policy_version 120338 (0.0006) [2023-12-27 06:29:34,822][20381] Updated weights for policy 1, policy_version 120348 (0.0006) [2023-12-27 06:29:35,140][20347] Updated weights for policy 0, policy_version 120779 (0.0010) [2023-12-27 06:29:35,197][20347] Updated weights for policy 0, policy_version 120789 (0.0010) [2023-12-27 06:29:35,252][20347] Updated weights for policy 0, policy_version 120799 (0.0010) [2023-12-27 06:29:35,448][20381] Updated weights for policy 1, policy_version 120358 (0.0007) [2023-12-27 06:29:35,507][20381] Updated weights for policy 1, policy_version 120368 (0.0007) [2023-12-27 06:29:35,565][20381] Updated weights for policy 1, policy_version 120378 (0.0007) [2023-12-27 06:29:35,925][20347] Updated weights for policy 0, policy_version 120809 (0.0010) [2023-12-27 06:29:35,979][20347] Updated weights for policy 0, policy_version 120819 (0.0010) [2023-12-27 06:29:36,033][20347] Updated weights for policy 0, policy_version 120829 (0.0010) [2023-12-27 06:29:36,091][20347] Updated weights for policy 0, policy_version 120839 (0.0010) [2023-12-27 06:29:36,237][20381] Updated weights for policy 1, policy_version 120388 (0.0008) [2023-12-27 06:29:36,286][20381] Updated weights for policy 1, policy_version 120398 (0.0007) [2023-12-27 06:29:36,339][20381] Updated weights for policy 1, policy_version 120408 (0.0008) [2023-12-27 06:29:36,793][20347] Updated weights for policy 0, policy_version 120849 (0.0010) [2023-12-27 06:29:36,852][20347] Updated weights for policy 0, policy_version 120859 (0.0010) [2023-12-27 06:29:36,908][20347] Updated weights for policy 0, policy_version 120869 (0.0010) [2023-12-27 06:29:37,079][20381] Updated weights for policy 1, policy_version 120418 (0.0009) [2023-12-27 06:29:37,132][20381] Updated weights for policy 1, policy_version 120428 (0.0011) [2023-12-27 06:29:37,179][20381] Updated weights for policy 1, policy_version 120438 (0.0010) [2023-12-27 06:29:37,232][20381] Updated weights for policy 1, policy_version 120448 (0.0011) [2023-12-27 06:29:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 61800448. Throughput: 0: 10623.2, 1: 10348.4. Samples: 61767776. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.590')] [2023-12-27 06:29:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000120448_30842880.pth... [2023-12-27 06:29:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000120872_30957568.pth... [2023-12-27 06:29:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000119264_30539776.pth [2023-12-27 06:29:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000119624_30638080.pth [2023-12-27 06:29:37,578][20347] Updated weights for policy 0, policy_version 120879 (0.0005) [2023-12-27 06:29:37,637][20347] Updated weights for policy 0, policy_version 120889 (0.0006) [2023-12-27 06:29:37,696][20347] Updated weights for policy 0, policy_version 120899 (0.0008) [2023-12-27 06:29:37,903][20381] Updated weights for policy 1, policy_version 120458 (0.0005) [2023-12-27 06:29:37,950][20381] Updated weights for policy 1, policy_version 120468 (0.0005) [2023-12-27 06:29:37,996][20381] Updated weights for policy 1, policy_version 120478 (0.0005) [2023-12-27 06:29:38,358][20347] Updated weights for policy 0, policy_version 120909 (0.0007) [2023-12-27 06:29:38,423][20347] Updated weights for policy 0, policy_version 120919 (0.0008) [2023-12-27 06:29:38,479][20347] Updated weights for policy 0, policy_version 120929 (0.0008) [2023-12-27 06:29:38,580][20381] Updated weights for policy 1, policy_version 120488 (0.0007) [2023-12-27 06:29:38,632][20381] Updated weights for policy 1, policy_version 120498 (0.0007) [2023-12-27 06:29:38,693][20381] Updated weights for policy 1, policy_version 120508 (0.0006) [2023-12-27 06:29:39,179][20347] Updated weights for policy 0, policy_version 120939 (0.0007) [2023-12-27 06:29:39,239][20347] Updated weights for policy 0, policy_version 120949 (0.0007) [2023-12-27 06:29:39,296][20381] Updated weights for policy 1, policy_version 120518 (0.0007) [2023-12-27 06:29:39,297][20347] Updated weights for policy 0, policy_version 120959 (0.0008) [2023-12-27 06:29:39,358][20381] Updated weights for policy 1, policy_version 120528 (0.0007) [2023-12-27 06:29:39,423][20381] Updated weights for policy 1, policy_version 120538 (0.0008) [2023-12-27 06:29:39,999][20347] Updated weights for policy 0, policy_version 120969 (0.0008) [2023-12-27 06:29:40,059][20347] Updated weights for policy 0, policy_version 120979 (0.0006) [2023-12-27 06:29:40,127][20347] Updated weights for policy 0, policy_version 120989 (0.0005) [2023-12-27 06:29:40,179][20381] Updated weights for policy 1, policy_version 120548 (0.0008) [2023-12-27 06:29:40,190][20347] Updated weights for policy 0, policy_version 120999 (0.0007) [2023-12-27 06:29:40,237][20381] Updated weights for policy 1, policy_version 120558 (0.0005) [2023-12-27 06:29:40,295][20381] Updated weights for policy 1, policy_version 120568 (0.0007) [2023-12-27 06:29:40,877][20347] Updated weights for policy 0, policy_version 121009 (0.0008) [2023-12-27 06:29:40,934][20347] Updated weights for policy 0, policy_version 121019 (0.0008) [2023-12-27 06:29:40,945][20381] Updated weights for policy 1, policy_version 120578 (0.0006) [2023-12-27 06:29:40,996][20347] Updated weights for policy 0, policy_version 121029 (0.0007) [2023-12-27 06:29:41,008][20381] Updated weights for policy 1, policy_version 120588 (0.0006) [2023-12-27 06:29:41,074][20381] Updated weights for policy 1, policy_version 120598 (0.0006) [2023-12-27 06:29:41,138][20381] Updated weights for policy 1, policy_version 120608 (0.0006) [2023-12-27 06:29:41,778][20347] Updated weights for policy 0, policy_version 121039 (0.0008) [2023-12-27 06:29:41,798][20381] Updated weights for policy 1, policy_version 120618 (0.0008) [2023-12-27 06:29:41,845][20347] Updated weights for policy 0, policy_version 121049 (0.0008) [2023-12-27 06:29:41,856][20381] Updated weights for policy 1, policy_version 120628 (0.0005) [2023-12-27 06:29:41,915][20347] Updated weights for policy 0, policy_version 121059 (0.0008) [2023-12-27 06:29:41,918][20381] Updated weights for policy 1, policy_version 120638 (0.0006) [2023-12-27 06:29:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 61898752. Throughput: 0: 10541.4, 1: 10352.0. Samples: 61889864. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.080')] [2023-12-27 06:29:42,614][20381] Updated weights for policy 1, policy_version 120648 (0.0008) [2023-12-27 06:29:42,629][20347] Updated weights for policy 0, policy_version 121069 (0.0009) [2023-12-27 06:29:42,669][20381] Updated weights for policy 1, policy_version 120658 (0.0007) [2023-12-27 06:29:42,683][20347] Updated weights for policy 0, policy_version 121079 (0.0007) [2023-12-27 06:29:42,726][20381] Updated weights for policy 1, policy_version 120668 (0.0009) [2023-12-27 06:29:42,729][20347] Updated weights for policy 0, policy_version 121089 (0.0006) [2023-12-27 06:29:43,406][20381] Updated weights for policy 1, policy_version 120678 (0.0008) [2023-12-27 06:29:43,442][20347] Updated weights for policy 0, policy_version 121099 (0.0009) [2023-12-27 06:29:43,458][20381] Updated weights for policy 1, policy_version 120688 (0.0006) [2023-12-27 06:29:43,500][20347] Updated weights for policy 0, policy_version 121109 (0.0007) [2023-12-27 06:29:43,512][20381] Updated weights for policy 1, policy_version 120698 (0.0008) [2023-12-27 06:29:43,556][20347] Updated weights for policy 0, policy_version 121119 (0.0006) [2023-12-27 06:29:44,115][20381] Updated weights for policy 1, policy_version 120708 (0.0006) [2023-12-27 06:29:44,170][20381] Updated weights for policy 1, policy_version 120718 (0.0008) [2023-12-27 06:29:44,226][20381] Updated weights for policy 1, policy_version 120728 (0.0007) [2023-12-27 06:29:44,263][20347] Updated weights for policy 0, policy_version 121129 (0.0007) [2023-12-27 06:29:44,318][20347] Updated weights for policy 0, policy_version 121139 (0.0007) [2023-12-27 06:29:44,377][20347] Updated weights for policy 0, policy_version 121149 (0.0007) [2023-12-27 06:29:44,431][20347] Updated weights for policy 0, policy_version 121159 (0.0008) [2023-12-27 06:29:44,904][20381] Updated weights for policy 1, policy_version 120738 (0.0007) [2023-12-27 06:29:44,959][20381] Updated weights for policy 1, policy_version 120748 (0.0007) [2023-12-27 06:29:45,020][20381] Updated weights for policy 1, policy_version 120758 (0.0007) [2023-12-27 06:29:45,080][20381] Updated weights for policy 1, policy_version 120768 (0.0007) [2023-12-27 06:29:45,156][20347] Updated weights for policy 0, policy_version 121169 (0.0008) [2023-12-27 06:29:45,216][20347] Updated weights for policy 0, policy_version 121179 (0.0009) [2023-12-27 06:29:45,277][20347] Updated weights for policy 0, policy_version 121189 (0.0008) [2023-12-27 06:29:45,758][20381] Updated weights for policy 1, policy_version 120778 (0.0006) [2023-12-27 06:29:45,817][20381] Updated weights for policy 1, policy_version 120788 (0.0007) [2023-12-27 06:29:45,872][20381] Updated weights for policy 1, policy_version 120798 (0.0008) [2023-12-27 06:29:45,977][20347] Updated weights for policy 0, policy_version 121199 (0.0008) [2023-12-27 06:29:46,043][20347] Updated weights for policy 0, policy_version 121209 (0.0009) [2023-12-27 06:29:46,106][20347] Updated weights for policy 0, policy_version 121219 (0.0007) [2023-12-27 06:29:46,543][20381] Updated weights for policy 1, policy_version 120808 (0.0006) [2023-12-27 06:29:46,607][20381] Updated weights for policy 1, policy_version 120818 (0.0006) [2023-12-27 06:29:46,657][20381] Updated weights for policy 1, policy_version 120828 (0.0006) [2023-12-27 06:29:46,831][20347] Updated weights for policy 0, policy_version 121229 (0.0008) [2023-12-27 06:29:46,897][20347] Updated weights for policy 0, policy_version 121239 (0.0008) [2023-12-27 06:29:46,963][20347] Updated weights for policy 0, policy_version 121249 (0.0009) [2023-12-27 06:29:47,256][20381] Updated weights for policy 1, policy_version 120838 (0.0008) [2023-12-27 06:29:47,308][20381] Updated weights for policy 1, policy_version 120848 (0.0008) [2023-12-27 06:29:47,360][20381] Updated weights for policy 1, policy_version 120858 (0.0008) [2023-12-27 06:29:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 61997056. Throughput: 0: 10493.2, 1: 10320.4. Samples: 62010060. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.500')] [2023-12-27 06:29:47,666][20347] Updated weights for policy 0, policy_version 121259 (0.0009) [2023-12-27 06:29:47,724][20347] Updated weights for policy 0, policy_version 121269 (0.0008) [2023-12-27 06:29:47,775][20347] Updated weights for policy 0, policy_version 121279 (0.0007) [2023-12-27 06:29:48,096][20381] Updated weights for policy 1, policy_version 120868 (0.0007) [2023-12-27 06:29:48,141][20381] Updated weights for policy 1, policy_version 120878 (0.0007) [2023-12-27 06:29:48,194][20381] Updated weights for policy 1, policy_version 120888 (0.0008) [2023-12-27 06:29:48,429][20347] Updated weights for policy 0, policy_version 121289 (0.0005) [2023-12-27 06:29:48,496][20347] Updated weights for policy 0, policy_version 121299 (0.0008) [2023-12-27 06:29:48,549][20347] Updated weights for policy 0, policy_version 121309 (0.0007) [2023-12-27 06:29:48,598][20347] Updated weights for policy 0, policy_version 121320 (0.0009) [2023-12-27 06:29:48,895][20381] Updated weights for policy 1, policy_version 120898 (0.0007) [2023-12-27 06:29:48,948][20381] Updated weights for policy 1, policy_version 120908 (0.0008) [2023-12-27 06:29:48,998][20381] Updated weights for policy 1, policy_version 120918 (0.0009) [2023-12-27 06:29:49,050][20381] Updated weights for policy 1, policy_version 120928 (0.0008) [2023-12-27 06:29:49,196][20347] Updated weights for policy 0, policy_version 121330 (0.0008) [2023-12-27 06:29:49,258][20347] Updated weights for policy 0, policy_version 121340 (0.0008) [2023-12-27 06:29:49,318][20347] Updated weights for policy 0, policy_version 121350 (0.0008) [2023-12-27 06:29:49,736][20381] Updated weights for policy 1, policy_version 120938 (0.0008) [2023-12-27 06:29:49,793][20381] Updated weights for policy 1, policy_version 120948 (0.0008) [2023-12-27 06:29:49,852][20381] Updated weights for policy 1, policy_version 120958 (0.0006) [2023-12-27 06:29:50,080][20347] Updated weights for policy 0, policy_version 121360 (0.0010) [2023-12-27 06:29:50,134][20347] Updated weights for policy 0, policy_version 121370 (0.0010) [2023-12-27 06:29:50,202][20347] Updated weights for policy 0, policy_version 121380 (0.0011) [2023-12-27 06:29:50,528][20381] Updated weights for policy 1, policy_version 120968 (0.0007) [2023-12-27 06:29:50,586][20381] Updated weights for policy 1, policy_version 120978 (0.0008) [2023-12-27 06:29:50,644][20381] Updated weights for policy 1, policy_version 120988 (0.0006) [2023-12-27 06:29:50,854][20347] Updated weights for policy 0, policy_version 121390 (0.0009) [2023-12-27 06:29:50,916][20347] Updated weights for policy 0, policy_version 121400 (0.0007) [2023-12-27 06:29:50,978][20347] Updated weights for policy 0, policy_version 121410 (0.0005) [2023-12-27 06:29:51,379][20381] Updated weights for policy 1, policy_version 120998 (0.0007) [2023-12-27 06:29:51,445][20381] Updated weights for policy 1, policy_version 121008 (0.0006) [2023-12-27 06:29:51,509][20381] Updated weights for policy 1, policy_version 121018 (0.0008) [2023-12-27 06:29:51,565][20347] Updated weights for policy 0, policy_version 121420 (0.0005) [2023-12-27 06:29:51,633][20347] Updated weights for policy 0, policy_version 121430 (0.0009) [2023-12-27 06:29:51,693][20347] Updated weights for policy 0, policy_version 121440 (0.0008) [2023-12-27 06:29:52,172][20381] Updated weights for policy 1, policy_version 121028 (0.0008) [2023-12-27 06:29:52,227][20381] Updated weights for policy 1, policy_version 121038 (0.0008) [2023-12-27 06:29:52,280][20381] Updated weights for policy 1, policy_version 121048 (0.0008) [2023-12-27 06:29:52,347][20347] Updated weights for policy 0, policy_version 121450 (0.0008) [2023-12-27 06:29:52,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 62103552. Throughput: 0: 10479.6, 1: 10211.9. Samples: 62070860. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.920')] [2023-12-27 06:29:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000121056_30998528.pth... [2023-12-27 06:29:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000119840_30687232.pth [2023-12-27 06:29:52,410][20347] Updated weights for policy 0, policy_version 121460 (0.0007) [2023-12-27 06:29:52,466][20347] Updated weights for policy 0, policy_version 121470 (0.0008) [2023-12-27 06:29:52,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000121480_31113216.pth... [2023-12-27 06:29:52,527][20347] Updated weights for policy 0, policy_version 121480 (0.0007) [2023-12-27 06:29:52,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000120232_30793728.pth [2023-12-27 06:29:52,932][20381] Updated weights for policy 1, policy_version 121058 (0.0008) [2023-12-27 06:29:52,986][20381] Updated weights for policy 1, policy_version 121068 (0.0010) [2023-12-27 06:29:53,037][20381] Updated weights for policy 1, policy_version 121078 (0.0010) [2023-12-27 06:29:53,083][20381] Updated weights for policy 1, policy_version 121088 (0.0010) [2023-12-27 06:29:53,247][20347] Updated weights for policy 0, policy_version 121490 (0.0007) [2023-12-27 06:29:53,298][20347] Updated weights for policy 0, policy_version 121500 (0.0009) [2023-12-27 06:29:53,344][20347] Updated weights for policy 0, policy_version 121510 (0.0009) [2023-12-27 06:29:53,643][20381] Updated weights for policy 1, policy_version 121098 (0.0007) [2023-12-27 06:29:53,696][20381] Updated weights for policy 1, policy_version 121108 (0.0011) [2023-12-27 06:29:53,751][20381] Updated weights for policy 1, policy_version 121118 (0.0010) [2023-12-27 06:29:54,012][20347] Updated weights for policy 0, policy_version 121520 (0.0006) [2023-12-27 06:29:54,064][20347] Updated weights for policy 0, policy_version 121530 (0.0006) [2023-12-27 06:29:54,118][20347] Updated weights for policy 0, policy_version 121540 (0.0007) [2023-12-27 06:29:54,394][20381] Updated weights for policy 1, policy_version 121128 (0.0008) [2023-12-27 06:29:54,445][20381] Updated weights for policy 1, policy_version 121138 (0.0008) [2023-12-27 06:29:54,501][20381] Updated weights for policy 1, policy_version 121148 (0.0008) [2023-12-27 06:29:54,805][20347] Updated weights for policy 0, policy_version 121550 (0.0008) [2023-12-27 06:29:54,859][20347] Updated weights for policy 0, policy_version 121560 (0.0007) [2023-12-27 06:29:54,919][20347] Updated weights for policy 0, policy_version 121570 (0.0006) [2023-12-27 06:29:55,211][20381] Updated weights for policy 1, policy_version 121158 (0.0010) [2023-12-27 06:29:55,263][20381] Updated weights for policy 1, policy_version 121168 (0.0011) [2023-12-27 06:29:55,314][20381] Updated weights for policy 1, policy_version 121178 (0.0010) [2023-12-27 06:29:55,517][20347] Updated weights for policy 0, policy_version 121580 (0.0006) [2023-12-27 06:29:55,571][20347] Updated weights for policy 0, policy_version 121590 (0.0007) [2023-12-27 06:29:55,626][20347] Updated weights for policy 0, policy_version 121600 (0.0007) [2023-12-27 06:29:56,013][20381] Updated weights for policy 1, policy_version 121188 (0.0008) [2023-12-27 06:29:56,070][20381] Updated weights for policy 1, policy_version 121198 (0.0005) [2023-12-27 06:29:56,134][20381] Updated weights for policy 1, policy_version 121208 (0.0005) [2023-12-27 06:29:56,275][20347] Updated weights for policy 0, policy_version 121610 (0.0008) [2023-12-27 06:29:56,328][20347] Updated weights for policy 0, policy_version 121620 (0.0007) [2023-12-27 06:29:56,380][20347] Updated weights for policy 0, policy_version 121630 (0.0005) [2023-12-27 06:29:56,430][20347] Updated weights for policy 0, policy_version 121640 (0.0005) [2023-12-27 06:29:56,726][20381] Updated weights for policy 1, policy_version 121218 (0.0006) [2023-12-27 06:29:56,786][20381] Updated weights for policy 1, policy_version 121228 (0.0010) [2023-12-27 06:29:56,837][20381] Updated weights for policy 1, policy_version 121238 (0.0007) [2023-12-27 06:29:56,882][20381] Updated weights for policy 1, policy_version 121248 (0.0010) [2023-12-27 06:29:57,025][20347] Updated weights for policy 0, policy_version 121650 (0.0007) [2023-12-27 06:29:57,084][20347] Updated weights for policy 0, policy_version 121660 (0.0007) [2023-12-27 06:29:57,134][20347] Updated weights for policy 0, policy_version 121670 (0.0007) [2023-12-27 06:29:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 62210048. Throughput: 0: 10427.9, 1: 10227.7. Samples: 62196024. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:29:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.850')] [2023-12-27 06:29:57,560][20381] Updated weights for policy 1, policy_version 121258 (0.0005) [2023-12-27 06:29:57,615][20381] Updated weights for policy 1, policy_version 121268 (0.0009) [2023-12-27 06:29:57,670][20381] Updated weights for policy 1, policy_version 121278 (0.0009) [2023-12-27 06:29:57,836][20347] Updated weights for policy 0, policy_version 121680 (0.0008) [2023-12-27 06:29:57,879][20347] Updated weights for policy 0, policy_version 121690 (0.0008) [2023-12-27 06:29:57,932][20347] Updated weights for policy 0, policy_version 121700 (0.0008) [2023-12-27 06:29:58,259][20381] Updated weights for policy 1, policy_version 121288 (0.0008) [2023-12-27 06:29:58,318][20381] Updated weights for policy 1, policy_version 121298 (0.0008) [2023-12-27 06:29:58,381][20381] Updated weights for policy 1, policy_version 121308 (0.0007) [2023-12-27 06:29:58,710][20347] Updated weights for policy 0, policy_version 121710 (0.0008) [2023-12-27 06:29:58,770][20347] Updated weights for policy 0, policy_version 121720 (0.0008) [2023-12-27 06:29:58,833][20347] Updated weights for policy 0, policy_version 121730 (0.0007) [2023-12-27 06:29:59,157][20381] Updated weights for policy 1, policy_version 121318 (0.0008) [2023-12-27 06:29:59,214][20381] Updated weights for policy 1, policy_version 121328 (0.0008) [2023-12-27 06:29:59,287][20381] Updated weights for policy 1, policy_version 121339 (0.0010) [2023-12-27 06:29:59,617][20347] Updated weights for policy 0, policy_version 121740 (0.0009) [2023-12-27 06:29:59,665][20347] Updated weights for policy 0, policy_version 121750 (0.0010) [2023-12-27 06:29:59,723][20347] Updated weights for policy 0, policy_version 121760 (0.0010) [2023-12-27 06:29:59,963][20381] Updated weights for policy 1, policy_version 121349 (0.0008) [2023-12-27 06:30:00,015][20381] Updated weights for policy 1, policy_version 121359 (0.0008) [2023-12-27 06:30:00,074][20381] Updated weights for policy 1, policy_version 121369 (0.0008) [2023-12-27 06:30:00,404][20347] Updated weights for policy 0, policy_version 121770 (0.0010) [2023-12-27 06:30:00,455][20347] Updated weights for policy 0, policy_version 121780 (0.0010) [2023-12-27 06:30:00,500][20347] Updated weights for policy 0, policy_version 121790 (0.0010) [2023-12-27 06:30:00,544][20347] Updated weights for policy 0, policy_version 121800 (0.0010) [2023-12-27 06:30:00,717][20381] Updated weights for policy 1, policy_version 121379 (0.0008) [2023-12-27 06:30:00,766][20381] Updated weights for policy 1, policy_version 121389 (0.0010) [2023-12-27 06:30:00,809][20381] Updated weights for policy 1, policy_version 121399 (0.0010) [2023-12-27 06:30:01,256][20347] Updated weights for policy 0, policy_version 121810 (0.0008) [2023-12-27 06:30:01,305][20347] Updated weights for policy 0, policy_version 121820 (0.0007) [2023-12-27 06:30:01,370][20347] Updated weights for policy 0, policy_version 121830 (0.0008) [2023-12-27 06:30:01,564][20381] Updated weights for policy 1, policy_version 121409 (0.0010) [2023-12-27 06:30:01,623][20381] Updated weights for policy 1, policy_version 121419 (0.0008) [2023-12-27 06:30:01,680][20381] Updated weights for policy 1, policy_version 121429 (0.0008) [2023-12-27 06:30:01,743][20381] Updated weights for policy 1, policy_version 121439 (0.0008) [2023-12-27 06:30:02,087][20347] Updated weights for policy 0, policy_version 121840 (0.0008) [2023-12-27 06:30:02,145][20347] Updated weights for policy 0, policy_version 121850 (0.0009) [2023-12-27 06:30:02,202][20347] Updated weights for policy 0, policy_version 121860 (0.0008) [2023-12-27 06:30:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 62308352. Throughput: 0: 10369.8, 1: 10255.0. Samples: 62317160. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:30:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.420')] [2023-12-27 06:30:02,415][20381] Updated weights for policy 1, policy_version 121449 (0.0006) [2023-12-27 06:30:02,469][20381] Updated weights for policy 1, policy_version 121459 (0.0008) [2023-12-27 06:30:02,530][20381] Updated weights for policy 1, policy_version 121469 (0.0007) [2023-12-27 06:30:02,939][20347] Updated weights for policy 0, policy_version 121870 (0.0008) [2023-12-27 06:30:02,984][20347] Updated weights for policy 0, policy_version 121880 (0.0008) [2023-12-27 06:30:03,030][20347] Updated weights for policy 0, policy_version 121890 (0.0007) [2023-12-27 06:30:03,107][20381] Updated weights for policy 1, policy_version 121479 (0.0007) [2023-12-27 06:30:03,151][20381] Updated weights for policy 1, policy_version 121489 (0.0008) [2023-12-27 06:30:03,202][20381] Updated weights for policy 1, policy_version 121499 (0.0008) [2023-12-27 06:30:03,640][20347] Updated weights for policy 0, policy_version 121900 (0.0007) [2023-12-27 06:30:03,690][20347] Updated weights for policy 0, policy_version 121910 (0.0005) [2023-12-27 06:30:03,748][20381] Updated weights for policy 1, policy_version 121509 (0.0005) [2023-12-27 06:30:03,755][20347] Updated weights for policy 0, policy_version 121920 (0.0005) [2023-12-27 06:30:03,801][20381] Updated weights for policy 1, policy_version 121519 (0.0005) [2023-12-27 06:30:03,862][20381] Updated weights for policy 1, policy_version 121529 (0.0007) [2023-12-27 06:30:04,437][20381] Updated weights for policy 1, policy_version 121539 (0.0007) [2023-12-27 06:30:04,447][20347] Updated weights for policy 0, policy_version 121930 (0.0006) [2023-12-27 06:30:04,494][20381] Updated weights for policy 1, policy_version 121549 (0.0006) [2023-12-27 06:30:04,510][20347] Updated weights for policy 0, policy_version 121940 (0.0006) [2023-12-27 06:30:04,549][20381] Updated weights for policy 1, policy_version 121559 (0.0006) [2023-12-27 06:30:04,565][20347] Updated weights for policy 0, policy_version 121950 (0.0007) [2023-12-27 06:30:04,625][20347] Updated weights for policy 0, policy_version 121960 (0.0008) [2023-12-27 06:30:05,194][20381] Updated weights for policy 1, policy_version 121569 (0.0007) [2023-12-27 06:30:05,258][20381] Updated weights for policy 1, policy_version 121579 (0.0009) [2023-12-27 06:30:05,319][20381] Updated weights for policy 1, policy_version 121589 (0.0007) [2023-12-27 06:30:05,350][20347] Updated weights for policy 0, policy_version 121970 (0.0007) [2023-12-27 06:30:05,376][20381] Updated weights for policy 1, policy_version 121599 (0.0006) [2023-12-27 06:30:05,400][20347] Updated weights for policy 0, policy_version 121980 (0.0007) [2023-12-27 06:30:05,446][20347] Updated weights for policy 0, policy_version 121990 (0.0005) [2023-12-27 06:30:06,032][20347] Updated weights for policy 0, policy_version 122000 (0.0006) [2023-12-27 06:30:06,037][20381] Updated weights for policy 1, policy_version 121609 (0.0008) [2023-12-27 06:30:06,084][20347] Updated weights for policy 0, policy_version 122010 (0.0008) [2023-12-27 06:30:06,093][20381] Updated weights for policy 1, policy_version 121619 (0.0006) [2023-12-27 06:30:06,140][20347] Updated weights for policy 0, policy_version 122020 (0.0010) [2023-12-27 06:30:06,153][20381] Updated weights for policy 1, policy_version 121629 (0.0008) [2023-12-27 06:30:06,759][20347] Updated weights for policy 0, policy_version 122030 (0.0010) [2023-12-27 06:30:06,816][20347] Updated weights for policy 0, policy_version 122040 (0.0005) [2023-12-27 06:30:06,871][20347] Updated weights for policy 0, policy_version 122050 (0.0005) [2023-12-27 06:30:06,905][20381] Updated weights for policy 1, policy_version 121639 (0.0007) [2023-12-27 06:30:06,966][20381] Updated weights for policy 1, policy_version 121649 (0.0006) [2023-12-27 06:30:07,026][20381] Updated weights for policy 1, policy_version 121659 (0.0006) [2023-12-27 06:30:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 62414848. Throughput: 0: 10325.6, 1: 10385.5. Samples: 62381952. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-12-27 06:30:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.630')] [2023-12-27 06:30:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000121664_31154176.pth... [2023-12-27 06:30:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000122056_31260672.pth... [2023-12-27 06:30:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000120448_30842880.pth [2023-12-27 06:30:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000120872_30957568.pth [2023-12-27 06:30:07,548][20347] Updated weights for policy 0, policy_version 122060 (0.0009) [2023-12-27 06:30:07,602][20347] Updated weights for policy 0, policy_version 122070 (0.0009) [2023-12-27 06:30:07,657][20347] Updated weights for policy 0, policy_version 122080 (0.0008) [2023-12-27 06:30:07,694][20381] Updated weights for policy 1, policy_version 121669 (0.0007) [2023-12-27 06:30:07,743][20381] Updated weights for policy 1, policy_version 121679 (0.0007) [2023-12-27 06:30:07,789][20381] Updated weights for policy 1, policy_version 121689 (0.0008) [2023-12-27 06:30:08,317][20347] Updated weights for policy 0, policy_version 122090 (0.0006) [2023-12-27 06:30:08,380][20347] Updated weights for policy 0, policy_version 122100 (0.0007) [2023-12-27 06:30:08,437][20347] Updated weights for policy 0, policy_version 122110 (0.0008) [2023-12-27 06:30:08,497][20347] Updated weights for policy 0, policy_version 122120 (0.0008) [2023-12-27 06:30:08,547][20381] Updated weights for policy 1, policy_version 121699 (0.0008) [2023-12-27 06:30:08,605][20381] Updated weights for policy 1, policy_version 121709 (0.0011) [2023-12-27 06:30:08,665][20381] Updated weights for policy 1, policy_version 121719 (0.0011) [2023-12-27 06:30:09,184][20347] Updated weights for policy 0, policy_version 122130 (0.0008) [2023-12-27 06:30:09,248][20347] Updated weights for policy 0, policy_version 122140 (0.0010) [2023-12-27 06:30:09,309][20347] Updated weights for policy 0, policy_version 122150 (0.0010) [2023-12-27 06:30:09,384][20381] Updated weights for policy 1, policy_version 121729 (0.0011) [2023-12-27 06:30:09,446][20381] Updated weights for policy 1, policy_version 121739 (0.0011) [2023-12-27 06:30:09,500][20381] Updated weights for policy 1, policy_version 121749 (0.0010) [2023-12-27 06:30:09,553][20381] Updated weights for policy 1, policy_version 121759 (0.0010) [2023-12-27 06:30:10,033][20347] Updated weights for policy 0, policy_version 122160 (0.0010) [2023-12-27 06:30:10,097][20347] Updated weights for policy 0, policy_version 122170 (0.0011) [2023-12-27 06:30:10,150][20347] Updated weights for policy 0, policy_version 122180 (0.0010) [2023-12-27 06:30:10,323][20381] Updated weights for policy 1, policy_version 121769 (0.0010) [2023-12-27 06:30:10,383][20381] Updated weights for policy 1, policy_version 121779 (0.0011) [2023-12-27 06:30:10,442][20381] Updated weights for policy 1, policy_version 121789 (0.0010) [2023-12-27 06:30:10,805][20347] Updated weights for policy 0, policy_version 122190 (0.0011) [2023-12-27 06:30:10,866][20347] Updated weights for policy 0, policy_version 122200 (0.0011) [2023-12-27 06:30:10,915][20347] Updated weights for policy 0, policy_version 122210 (0.0010) [2023-12-27 06:30:11,086][20381] Updated weights for policy 1, policy_version 121799 (0.0008) [2023-12-27 06:30:11,141][20381] Updated weights for policy 1, policy_version 121809 (0.0009) [2023-12-27 06:30:11,209][20381] Updated weights for policy 1, policy_version 121819 (0.0009) [2023-12-27 06:30:11,711][20347] Updated weights for policy 0, policy_version 122220 (0.0009) [2023-12-27 06:30:11,780][20347] Updated weights for policy 0, policy_version 122230 (0.0009) [2023-12-27 06:30:11,836][20347] Updated weights for policy 0, policy_version 122240 (0.0008) [2023-12-27 06:30:11,940][20381] Updated weights for policy 1, policy_version 121829 (0.0007) [2023-12-27 06:30:11,999][20381] Updated weights for policy 1, policy_version 121839 (0.0005) [2023-12-27 06:30:12,061][20381] Updated weights for policy 1, policy_version 121849 (0.0008) [2023-12-27 06:30:12,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 62513152. Throughput: 0: 10244.7, 1: 10294.8. Samples: 62502892. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.890')] [2023-12-27 06:30:12,554][20347] Updated weights for policy 0, policy_version 122250 (0.0009) [2023-12-27 06:30:12,612][20347] Updated weights for policy 0, policy_version 122260 (0.0008) [2023-12-27 06:30:12,675][20347] Updated weights for policy 0, policy_version 122270 (0.0008) [2023-12-27 06:30:12,723][20381] Updated weights for policy 1, policy_version 121859 (0.0007) [2023-12-27 06:30:12,736][20347] Updated weights for policy 0, policy_version 122280 (0.0007) [2023-12-27 06:30:12,777][20381] Updated weights for policy 1, policy_version 121869 (0.0007) [2023-12-27 06:30:12,839][20381] Updated weights for policy 1, policy_version 121879 (0.0008) [2023-12-27 06:30:13,363][20347] Updated weights for policy 0, policy_version 122290 (0.0007) [2023-12-27 06:30:13,414][20347] Updated weights for policy 0, policy_version 122300 (0.0010) [2023-12-27 06:30:13,463][20347] Updated weights for policy 0, policy_version 122310 (0.0010) [2023-12-27 06:30:13,568][20381] Updated weights for policy 1, policy_version 121889 (0.0007) [2023-12-27 06:30:13,621][20381] Updated weights for policy 1, policy_version 121899 (0.0007) [2023-12-27 06:30:13,676][20381] Updated weights for policy 1, policy_version 121909 (0.0007) [2023-12-27 06:30:13,733][20381] Updated weights for policy 1, policy_version 121919 (0.0007) [2023-12-27 06:30:14,138][20347] Updated weights for policy 0, policy_version 122320 (0.0008) [2023-12-27 06:30:14,193][20347] Updated weights for policy 0, policy_version 122330 (0.0008) [2023-12-27 06:30:14,245][20347] Updated weights for policy 0, policy_version 122340 (0.0008) [2023-12-27 06:30:14,434][20381] Updated weights for policy 1, policy_version 121929 (0.0008) [2023-12-27 06:30:14,490][20381] Updated weights for policy 1, policy_version 121939 (0.0007) [2023-12-27 06:30:14,543][20381] Updated weights for policy 1, policy_version 121949 (0.0010) [2023-12-27 06:30:14,939][20347] Updated weights for policy 0, policy_version 122350 (0.0006) [2023-12-27 06:30:14,992][20347] Updated weights for policy 0, policy_version 122360 (0.0008) [2023-12-27 06:30:15,045][20347] Updated weights for policy 0, policy_version 122370 (0.0007) [2023-12-27 06:30:15,272][20381] Updated weights for policy 1, policy_version 121959 (0.0010) [2023-12-27 06:30:15,328][20381] Updated weights for policy 1, policy_version 121969 (0.0011) [2023-12-27 06:30:15,386][20381] Updated weights for policy 1, policy_version 121979 (0.0010) [2023-12-27 06:30:15,771][20347] Updated weights for policy 0, policy_version 122380 (0.0007) [2023-12-27 06:30:15,826][20347] Updated weights for policy 0, policy_version 122390 (0.0007) [2023-12-27 06:30:15,888][20347] Updated weights for policy 0, policy_version 122400 (0.0007) [2023-12-27 06:30:16,099][20381] Updated weights for policy 1, policy_version 121989 (0.0011) [2023-12-27 06:30:16,162][20381] Updated weights for policy 1, policy_version 121999 (0.0011) [2023-12-27 06:30:16,222][20381] Updated weights for policy 1, policy_version 122009 (0.0011) [2023-12-27 06:30:16,605][20347] Updated weights for policy 0, policy_version 122410 (0.0007) [2023-12-27 06:30:16,653][20347] Updated weights for policy 0, policy_version 122420 (0.0008) [2023-12-27 06:30:16,712][20347] Updated weights for policy 0, policy_version 122430 (0.0010) [2023-12-27 06:30:16,773][20347] Updated weights for policy 0, policy_version 122440 (0.0010) [2023-12-27 06:30:16,819][20381] Updated weights for policy 1, policy_version 122019 (0.0010) [2023-12-27 06:30:16,874][20381] Updated weights for policy 1, policy_version 122029 (0.0007) [2023-12-27 06:30:16,927][20381] Updated weights for policy 1, policy_version 122039 (0.0005) [2023-12-27 06:30:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 62611456. Throughput: 0: 10126.2, 1: 10302.5. Samples: 62622424. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.150')] [2023-12-27 06:30:17,471][20347] Updated weights for policy 0, policy_version 122450 (0.0008) [2023-12-27 06:30:17,538][20347] Updated weights for policy 0, policy_version 122460 (0.0007) [2023-12-27 06:30:17,603][20347] Updated weights for policy 0, policy_version 122470 (0.0009) [2023-12-27 06:30:17,634][20381] Updated weights for policy 1, policy_version 122049 (0.0011) [2023-12-27 06:30:17,691][20381] Updated weights for policy 1, policy_version 122059 (0.0011) [2023-12-27 06:30:17,744][20381] Updated weights for policy 1, policy_version 122069 (0.0008) [2023-12-27 06:30:17,799][20381] Updated weights for policy 1, policy_version 122079 (0.0007) [2023-12-27 06:30:18,314][20347] Updated weights for policy 0, policy_version 122480 (0.0010) [2023-12-27 06:30:18,380][20347] Updated weights for policy 0, policy_version 122490 (0.0008) [2023-12-27 06:30:18,440][20347] Updated weights for policy 0, policy_version 122500 (0.0011) [2023-12-27 06:30:18,470][20381] Updated weights for policy 1, policy_version 122089 (0.0007) [2023-12-27 06:30:18,533][20381] Updated weights for policy 1, policy_version 122099 (0.0008) [2023-12-27 06:30:18,592][20381] Updated weights for policy 1, policy_version 122109 (0.0007) [2023-12-27 06:30:19,126][20347] Updated weights for policy 0, policy_version 122510 (0.0006) [2023-12-27 06:30:19,177][20347] Updated weights for policy 0, policy_version 122520 (0.0008) [2023-12-27 06:30:19,240][20347] Updated weights for policy 0, policy_version 122530 (0.0009) [2023-12-27 06:30:19,267][20381] Updated weights for policy 1, policy_version 122119 (0.0007) [2023-12-27 06:30:19,325][20381] Updated weights for policy 1, policy_version 122129 (0.0008) [2023-12-27 06:30:19,388][20381] Updated weights for policy 1, policy_version 122139 (0.0009) [2023-12-27 06:30:19,963][20347] Updated weights for policy 0, policy_version 122540 (0.0010) [2023-12-27 06:30:20,022][20347] Updated weights for policy 0, policy_version 122550 (0.0008) [2023-12-27 06:30:20,083][20347] Updated weights for policy 0, policy_version 122560 (0.0008) [2023-12-27 06:30:20,088][20381] Updated weights for policy 1, policy_version 122149 (0.0009) [2023-12-27 06:30:20,147][20381] Updated weights for policy 1, policy_version 122159 (0.0007) [2023-12-27 06:30:20,206][20381] Updated weights for policy 1, policy_version 122169 (0.0008) [2023-12-27 06:30:20,843][20381] Updated weights for policy 1, policy_version 122179 (0.0007) [2023-12-27 06:30:20,876][20347] Updated weights for policy 0, policy_version 122570 (0.0007) [2023-12-27 06:30:20,899][20381] Updated weights for policy 1, policy_version 122189 (0.0006) [2023-12-27 06:30:20,936][20347] Updated weights for policy 0, policy_version 122580 (0.0006) [2023-12-27 06:30:20,953][20381] Updated weights for policy 1, policy_version 122199 (0.0007) [2023-12-27 06:30:20,997][20347] Updated weights for policy 0, policy_version 122590 (0.0006) [2023-12-27 06:30:21,057][20347] Updated weights for policy 0, policy_version 122600 (0.0008) [2023-12-27 06:30:21,673][20381] Updated weights for policy 1, policy_version 122209 (0.0006) [2023-12-27 06:30:21,732][20381] Updated weights for policy 1, policy_version 122219 (0.0007) [2023-12-27 06:30:21,798][20347] Updated weights for policy 0, policy_version 122610 (0.0007) [2023-12-27 06:30:21,798][20381] Updated weights for policy 1, policy_version 122229 (0.0009) [2023-12-27 06:30:21,855][20347] Updated weights for policy 0, policy_version 122620 (0.0006) [2023-12-27 06:30:21,857][20381] Updated weights for policy 1, policy_version 122239 (0.0009) [2023-12-27 06:30:21,906][20347] Updated weights for policy 0, policy_version 122630 (0.0008) [2023-12-27 06:30:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 62709760. Throughput: 0: 10047.7, 1: 10263.3. Samples: 62681772. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.350')] [2023-12-27 06:30:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000122240_31301632.pth... [2023-12-27 06:30:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000122632_31408128.pth... [2023-12-27 06:30:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000121056_30998528.pth [2023-12-27 06:30:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000121480_31113216.pth [2023-12-27 06:30:22,583][20381] Updated weights for policy 1, policy_version 122249 (0.0006) [2023-12-27 06:30:22,601][20347] Updated weights for policy 0, policy_version 122640 (0.0007) [2023-12-27 06:30:22,646][20381] Updated weights for policy 1, policy_version 122259 (0.0007) [2023-12-27 06:30:22,652][20347] Updated weights for policy 0, policy_version 122650 (0.0008) [2023-12-27 06:30:22,703][20381] Updated weights for policy 1, policy_version 122269 (0.0006) [2023-12-27 06:30:22,711][20347] Updated weights for policy 0, policy_version 122660 (0.0006) [2023-12-27 06:30:23,337][20347] Updated weights for policy 0, policy_version 122670 (0.0006) [2023-12-27 06:30:23,397][20347] Updated weights for policy 0, policy_version 122680 (0.0008) [2023-12-27 06:30:23,430][20381] Updated weights for policy 1, policy_version 122279 (0.0005) [2023-12-27 06:30:23,448][20347] Updated weights for policy 0, policy_version 122690 (0.0007) [2023-12-27 06:30:23,484][20381] Updated weights for policy 1, policy_version 122289 (0.0006) [2023-12-27 06:30:23,534][20381] Updated weights for policy 1, policy_version 122299 (0.0008) [2023-12-27 06:30:24,100][20347] Updated weights for policy 0, policy_version 122700 (0.0006) [2023-12-27 06:30:24,154][20347] Updated weights for policy 0, policy_version 122710 (0.0005) [2023-12-27 06:30:24,191][20381] Updated weights for policy 1, policy_version 122309 (0.0010) [2023-12-27 06:30:24,202][20347] Updated weights for policy 0, policy_version 122720 (0.0005) [2023-12-27 06:30:24,247][20381] Updated weights for policy 1, policy_version 122319 (0.0011) [2023-12-27 06:30:24,295][20381] Updated weights for policy 1, policy_version 122329 (0.0010) [2023-12-27 06:30:24,757][20347] Updated weights for policy 0, policy_version 122730 (0.0005) [2023-12-27 06:30:24,815][20347] Updated weights for policy 0, policy_version 122740 (0.0005) [2023-12-27 06:30:24,869][20347] Updated weights for policy 0, policy_version 122750 (0.0005) [2023-12-27 06:30:24,924][20347] Updated weights for policy 0, policy_version 122760 (0.0006) [2023-12-27 06:30:24,986][20381] Updated weights for policy 1, policy_version 122339 (0.0008) [2023-12-27 06:30:25,040][20381] Updated weights for policy 1, policy_version 122349 (0.0007) [2023-12-27 06:30:25,103][20381] Updated weights for policy 1, policy_version 122359 (0.0005) [2023-12-27 06:30:25,564][20347] Updated weights for policy 0, policy_version 122770 (0.0008) [2023-12-27 06:30:25,626][20347] Updated weights for policy 0, policy_version 122780 (0.0008) [2023-12-27 06:30:25,678][20347] Updated weights for policy 0, policy_version 122790 (0.0007) [2023-12-27 06:30:25,679][20381] Updated weights for policy 1, policy_version 122369 (0.0006) [2023-12-27 06:30:25,730][20381] Updated weights for policy 1, policy_version 122379 (0.0007) [2023-12-27 06:30:25,776][20381] Updated weights for policy 1, policy_version 122389 (0.0007) [2023-12-27 06:30:25,825][20381] Updated weights for policy 1, policy_version 122399 (0.0008) [2023-12-27 06:30:26,294][20347] Updated weights for policy 0, policy_version 122800 (0.0005) [2023-12-27 06:30:26,347][20347] Updated weights for policy 0, policy_version 122810 (0.0005) [2023-12-27 06:30:26,395][20347] Updated weights for policy 0, policy_version 122820 (0.0005) [2023-12-27 06:30:26,498][20381] Updated weights for policy 1, policy_version 122409 (0.0008) [2023-12-27 06:30:26,552][20381] Updated weights for policy 1, policy_version 122419 (0.0007) [2023-12-27 06:30:26,604][20381] Updated weights for policy 1, policy_version 122429 (0.0008) [2023-12-27 06:30:26,990][20347] Updated weights for policy 0, policy_version 122830 (0.0008) [2023-12-27 06:30:27,045][20347] Updated weights for policy 0, policy_version 122840 (0.0010) [2023-12-27 06:30:27,096][20347] Updated weights for policy 0, policy_version 122850 (0.0010) [2023-12-27 06:30:27,175][20381] Updated weights for policy 1, policy_version 122439 (0.0008) [2023-12-27 06:30:27,232][20381] Updated weights for policy 1, policy_version 122449 (0.0007) [2023-12-27 06:30:27,283][20381] Updated weights for policy 1, policy_version 122459 (0.0007) [2023-12-27 06:30:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 62824448. Throughput: 0: 10089.9, 1: 10236.2. Samples: 62804540. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.840')] [2023-12-27 06:30:27,797][20347] Updated weights for policy 0, policy_version 122860 (0.0008) [2023-12-27 06:30:27,850][20347] Updated weights for policy 0, policy_version 122870 (0.0007) [2023-12-27 06:30:27,906][20347] Updated weights for policy 0, policy_version 122880 (0.0008) [2023-12-27 06:30:27,917][20381] Updated weights for policy 1, policy_version 122469 (0.0007) [2023-12-27 06:30:27,971][20381] Updated weights for policy 1, policy_version 122479 (0.0006) [2023-12-27 06:30:28,030][20381] Updated weights for policy 1, policy_version 122489 (0.0005) [2023-12-27 06:30:28,610][20347] Updated weights for policy 0, policy_version 122890 (0.0007) [2023-12-27 06:30:28,651][20381] Updated weights for policy 1, policy_version 122499 (0.0008) [2023-12-27 06:30:28,674][20347] Updated weights for policy 0, policy_version 122900 (0.0008) [2023-12-27 06:30:28,702][20381] Updated weights for policy 1, policy_version 122509 (0.0010) [2023-12-27 06:30:28,732][20347] Updated weights for policy 0, policy_version 122910 (0.0008) [2023-12-27 06:30:28,752][20381] Updated weights for policy 1, policy_version 122519 (0.0005) [2023-12-27 06:30:28,792][20347] Updated weights for policy 0, policy_version 122920 (0.0008) [2023-12-27 06:30:29,301][20381] Updated weights for policy 1, policy_version 122529 (0.0008) [2023-12-27 06:30:29,362][20381] Updated weights for policy 1, policy_version 122539 (0.0010) [2023-12-27 06:30:29,419][20381] Updated weights for policy 1, policy_version 122549 (0.0011) [2023-12-27 06:30:29,475][20381] Updated weights for policy 1, policy_version 122559 (0.0010) [2023-12-27 06:30:29,511][20347] Updated weights for policy 0, policy_version 122930 (0.0005) [2023-12-27 06:30:29,571][20347] Updated weights for policy 0, policy_version 122940 (0.0005) [2023-12-27 06:30:29,623][20347] Updated weights for policy 0, policy_version 122950 (0.0005) [2023-12-27 06:30:30,171][20381] Updated weights for policy 1, policy_version 122569 (0.0011) [2023-12-27 06:30:30,181][20347] Updated weights for policy 0, policy_version 122960 (0.0007) [2023-12-27 06:30:30,228][20381] Updated weights for policy 1, policy_version 122579 (0.0011) [2023-12-27 06:30:30,241][20347] Updated weights for policy 0, policy_version 122970 (0.0007) [2023-12-27 06:30:30,287][20381] Updated weights for policy 1, policy_version 122589 (0.0010) [2023-12-27 06:30:30,304][20347] Updated weights for policy 0, policy_version 122980 (0.0006) [2023-12-27 06:30:30,944][20347] Updated weights for policy 0, policy_version 122990 (0.0006) [2023-12-27 06:30:30,966][20381] Updated weights for policy 1, policy_version 122599 (0.0010) [2023-12-27 06:30:30,992][20347] Updated weights for policy 0, policy_version 123000 (0.0006) [2023-12-27 06:30:31,018][20381] Updated weights for policy 1, policy_version 122609 (0.0010) [2023-12-27 06:30:31,047][20347] Updated weights for policy 0, policy_version 123010 (0.0005) [2023-12-27 06:30:31,080][20381] Updated weights for policy 1, policy_version 122619 (0.0009) [2023-12-27 06:30:31,688][20347] Updated weights for policy 0, policy_version 123020 (0.0007) [2023-12-27 06:30:31,754][20347] Updated weights for policy 0, policy_version 123030 (0.0008) [2023-12-27 06:30:31,778][20381] Updated weights for policy 1, policy_version 122629 (0.0008) [2023-12-27 06:30:31,810][20347] Updated weights for policy 0, policy_version 123040 (0.0006) [2023-12-27 06:30:31,835][20381] Updated weights for policy 1, policy_version 122639 (0.0005) [2023-12-27 06:30:31,898][20381] Updated weights for policy 1, policy_version 122649 (0.0010) [2023-12-27 06:30:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20716.1). Total num frames: 62922752. Throughput: 0: 10240.5, 1: 10296.6. Samples: 62934232. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.290')] [2023-12-27 06:30:32,434][20347] Updated weights for policy 0, policy_version 123050 (0.0005) [2023-12-27 06:30:32,495][20347] Updated weights for policy 0, policy_version 123060 (0.0005) [2023-12-27 06:30:32,553][20347] Updated weights for policy 0, policy_version 123070 (0.0007) [2023-12-27 06:30:32,583][20381] Updated weights for policy 1, policy_version 122659 (0.0010) [2023-12-27 06:30:32,603][20347] Updated weights for policy 0, policy_version 123080 (0.0007) [2023-12-27 06:30:32,636][20381] Updated weights for policy 1, policy_version 122669 (0.0010) [2023-12-27 06:30:32,690][20381] Updated weights for policy 1, policy_version 122679 (0.0010) [2023-12-27 06:30:33,292][20347] Updated weights for policy 0, policy_version 123090 (0.0007) [2023-12-27 06:30:33,304][20381] Updated weights for policy 1, policy_version 122689 (0.0010) [2023-12-27 06:30:33,344][20347] Updated weights for policy 0, policy_version 123100 (0.0009) [2023-12-27 06:30:33,356][20381] Updated weights for policy 1, policy_version 122699 (0.0010) [2023-12-27 06:30:33,391][20347] Updated weights for policy 0, policy_version 123110 (0.0006) [2023-12-27 06:30:33,401][20381] Updated weights for policy 1, policy_version 122709 (0.0010) [2023-12-27 06:30:33,450][20381] Updated weights for policy 1, policy_version 122719 (0.0010) [2023-12-27 06:30:34,094][20381] Updated weights for policy 1, policy_version 122729 (0.0006) [2023-12-27 06:30:34,129][20347] Updated weights for policy 0, policy_version 123120 (0.0006) [2023-12-27 06:30:34,150][20381] Updated weights for policy 1, policy_version 122739 (0.0008) [2023-12-27 06:30:34,185][20347] Updated weights for policy 0, policy_version 123130 (0.0008) [2023-12-27 06:30:34,200][20381] Updated weights for policy 1, policy_version 122749 (0.0008) [2023-12-27 06:30:34,245][20347] Updated weights for policy 0, policy_version 123140 (0.0007) [2023-12-27 06:30:34,889][20381] Updated weights for policy 1, policy_version 122759 (0.0010) [2023-12-27 06:30:34,951][20381] Updated weights for policy 1, policy_version 122769 (0.0010) [2023-12-27 06:30:34,972][20347] Updated weights for policy 0, policy_version 123150 (0.0007) [2023-12-27 06:30:35,008][20381] Updated weights for policy 1, policy_version 122779 (0.0010) [2023-12-27 06:30:35,034][20347] Updated weights for policy 0, policy_version 123160 (0.0006) [2023-12-27 06:30:35,083][20347] Updated weights for policy 0, policy_version 123170 (0.0008) [2023-12-27 06:30:35,601][20381] Updated weights for policy 1, policy_version 122789 (0.0008) [2023-12-27 06:30:35,657][20381] Updated weights for policy 1, policy_version 122799 (0.0011) [2023-12-27 06:30:35,710][20381] Updated weights for policy 1, policy_version 122809 (0.0010) [2023-12-27 06:30:35,721][20347] Updated weights for policy 0, policy_version 123180 (0.0007) [2023-12-27 06:30:35,777][20347] Updated weights for policy 0, policy_version 123190 (0.0005) [2023-12-27 06:30:35,833][20347] Updated weights for policy 0, policy_version 123200 (0.0005) [2023-12-27 06:30:36,382][20381] Updated weights for policy 1, policy_version 122820 (0.0010) [2023-12-27 06:30:36,446][20381] Updated weights for policy 1, policy_version 122830 (0.0010) [2023-12-27 06:30:36,505][20347] Updated weights for policy 0, policy_version 123210 (0.0006) [2023-12-27 06:30:36,506][20381] Updated weights for policy 1, policy_version 122840 (0.0011) [2023-12-27 06:30:36,568][20347] Updated weights for policy 0, policy_version 123220 (0.0008) [2023-12-27 06:30:36,630][20347] Updated weights for policy 0, policy_version 123230 (0.0007) [2023-12-27 06:30:36,691][20347] Updated weights for policy 0, policy_version 123240 (0.0008) [2023-12-27 06:30:37,210][20381] Updated weights for policy 1, policy_version 122850 (0.0011) [2023-12-27 06:30:37,256][20381] Updated weights for policy 1, policy_version 122860 (0.0010) [2023-12-27 06:30:37,305][20381] Updated weights for policy 1, policy_version 122870 (0.0010) [2023-12-27 06:30:37,348][20347] Updated weights for policy 0, policy_version 123250 (0.0007) [2023-12-27 06:30:37,355][20381] Updated weights for policy 1, policy_version 122880 (0.0010) [2023-12-27 06:30:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 63029248. Throughput: 0: 10223.2, 1: 10347.4. Samples: 62996536. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.210')] [2023-12-27 06:30:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000122880_31465472.pth... [2023-12-27 06:30:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000121664_31154176.pth [2023-12-27 06:30:37,409][20347] Updated weights for policy 0, policy_version 123260 (0.0007) [2023-12-27 06:30:37,471][20347] Updated weights for policy 0, policy_version 123270 (0.0007) [2023-12-27 06:30:37,482][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000123272_31571968.pth... [2023-12-27 06:30:37,486][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000122056_31260672.pth [2023-12-27 06:30:38,050][20347] Updated weights for policy 0, policy_version 123280 (0.0006) [2023-12-27 06:30:38,067][20381] Updated weights for policy 1, policy_version 122890 (0.0010) [2023-12-27 06:30:38,102][20347] Updated weights for policy 0, policy_version 123290 (0.0005) [2023-12-27 06:30:38,122][20381] Updated weights for policy 1, policy_version 122900 (0.0010) [2023-12-27 06:30:38,153][20347] Updated weights for policy 0, policy_version 123300 (0.0005) [2023-12-27 06:30:38,177][20381] Updated weights for policy 1, policy_version 122910 (0.0010) [2023-12-27 06:30:38,820][20347] Updated weights for policy 0, policy_version 123310 (0.0007) [2023-12-27 06:30:38,879][20347] Updated weights for policy 0, policy_version 123320 (0.0008) [2023-12-27 06:30:38,893][20381] Updated weights for policy 1, policy_version 122920 (0.0011) [2023-12-27 06:30:38,937][20347] Updated weights for policy 0, policy_version 123330 (0.0006) [2023-12-27 06:30:38,953][20381] Updated weights for policy 1, policy_version 122930 (0.0011) [2023-12-27 06:30:39,003][20381] Updated weights for policy 1, policy_version 122940 (0.0011) [2023-12-27 06:30:39,626][20347] Updated weights for policy 0, policy_version 123340 (0.0007) [2023-12-27 06:30:39,682][20347] Updated weights for policy 0, policy_version 123350 (0.0007) [2023-12-27 06:30:39,715][20381] Updated weights for policy 1, policy_version 122950 (0.0011) [2023-12-27 06:30:39,743][20347] Updated weights for policy 0, policy_version 123360 (0.0006) [2023-12-27 06:30:39,776][20381] Updated weights for policy 1, policy_version 122960 (0.0011) [2023-12-27 06:30:39,844][20381] Updated weights for policy 1, policy_version 122970 (0.0010) [2023-12-27 06:30:40,401][20347] Updated weights for policy 0, policy_version 123370 (0.0007) [2023-12-27 06:30:40,454][20347] Updated weights for policy 0, policy_version 123380 (0.0010) [2023-12-27 06:30:40,503][20347] Updated weights for policy 0, policy_version 123390 (0.0010) [2023-12-27 06:30:40,556][20347] Updated weights for policy 0, policy_version 123400 (0.0010) [2023-12-27 06:30:40,590][20381] Updated weights for policy 1, policy_version 122980 (0.0007) [2023-12-27 06:30:40,651][20381] Updated weights for policy 1, policy_version 122990 (0.0005) [2023-12-27 06:30:40,714][20381] Updated weights for policy 1, policy_version 123000 (0.0005) [2023-12-27 06:30:41,308][20347] Updated weights for policy 0, policy_version 123410 (0.0006) [2023-12-27 06:30:41,313][20381] Updated weights for policy 1, policy_version 123010 (0.0006) [2023-12-27 06:30:41,372][20381] Updated weights for policy 1, policy_version 123020 (0.0007) [2023-12-27 06:30:41,373][20347] Updated weights for policy 0, policy_version 123420 (0.0006) [2023-12-27 06:30:41,434][20347] Updated weights for policy 0, policy_version 123430 (0.0009) [2023-12-27 06:30:41,437][20381] Updated weights for policy 1, policy_version 123030 (0.0007) [2023-12-27 06:30:41,505][20381] Updated weights for policy 1, policy_version 123040 (0.0006) [2023-12-27 06:30:42,097][20347] Updated weights for policy 0, policy_version 123440 (0.0007) [2023-12-27 06:30:42,156][20347] Updated weights for policy 0, policy_version 123450 (0.0007) [2023-12-27 06:30:42,211][20347] Updated weights for policy 0, policy_version 123460 (0.0006) [2023-12-27 06:30:42,230][20381] Updated weights for policy 1, policy_version 123050 (0.0007) [2023-12-27 06:30:42,286][20381] Updated weights for policy 1, policy_version 123060 (0.0008) [2023-12-27 06:30:42,354][20381] Updated weights for policy 1, policy_version 123070 (0.0009) [2023-12-27 06:30:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 63127552. Throughput: 0: 10224.9, 1: 10300.6. Samples: 63119672. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.300')] [2023-12-27 06:30:42,901][20347] Updated weights for policy 0, policy_version 123470 (0.0007) [2023-12-27 06:30:42,964][20347] Updated weights for policy 0, policy_version 123480 (0.0008) [2023-12-27 06:30:43,018][20347] Updated weights for policy 0, policy_version 123490 (0.0009) [2023-12-27 06:30:43,029][20381] Updated weights for policy 1, policy_version 123080 (0.0006) [2023-12-27 06:30:43,083][20381] Updated weights for policy 1, policy_version 123090 (0.0005) [2023-12-27 06:30:43,134][20381] Updated weights for policy 1, policy_version 123100 (0.0005) [2023-12-27 06:30:43,747][20347] Updated weights for policy 0, policy_version 123500 (0.0006) [2023-12-27 06:30:43,760][20381] Updated weights for policy 1, policy_version 123110 (0.0007) [2023-12-27 06:30:43,803][20347] Updated weights for policy 0, policy_version 123510 (0.0006) [2023-12-27 06:30:43,816][20381] Updated weights for policy 1, policy_version 123120 (0.0006) [2023-12-27 06:30:43,862][20347] Updated weights for policy 0, policy_version 123520 (0.0005) [2023-12-27 06:30:43,875][20381] Updated weights for policy 1, policy_version 123130 (0.0006) [2023-12-27 06:30:44,490][20347] Updated weights for policy 0, policy_version 123530 (0.0006) [2023-12-27 06:30:44,550][20347] Updated weights for policy 0, policy_version 123540 (0.0006) [2023-12-27 06:30:44,584][20381] Updated weights for policy 1, policy_version 123140 (0.0007) [2023-12-27 06:30:44,603][20347] Updated weights for policy 0, policy_version 123550 (0.0008) [2023-12-27 06:30:44,637][20381] Updated weights for policy 1, policy_version 123150 (0.0005) [2023-12-27 06:30:44,658][20347] Updated weights for policy 0, policy_version 123560 (0.0007) [2023-12-27 06:30:44,695][20381] Updated weights for policy 1, policy_version 123160 (0.0005) [2023-12-27 06:30:45,242][20381] Updated weights for policy 1, policy_version 123170 (0.0006) [2023-12-27 06:30:45,303][20381] Updated weights for policy 1, policy_version 123180 (0.0008) [2023-12-27 06:30:45,352][20381] Updated weights for policy 1, policy_version 123190 (0.0007) [2023-12-27 06:30:45,371][20347] Updated weights for policy 0, policy_version 123570 (0.0009) [2023-12-27 06:30:45,417][20381] Updated weights for policy 1, policy_version 123200 (0.0008) [2023-12-27 06:30:45,420][20347] Updated weights for policy 0, policy_version 123580 (0.0006) [2023-12-27 06:30:45,474][20347] Updated weights for policy 0, policy_version 123590 (0.0008) [2023-12-27 06:30:46,064][20381] Updated weights for policy 1, policy_version 123210 (0.0005) [2023-12-27 06:30:46,126][20381] Updated weights for policy 1, policy_version 123220 (0.0005) [2023-12-27 06:30:46,188][20381] Updated weights for policy 1, policy_version 123230 (0.0005) [2023-12-27 06:30:46,188][20347] Updated weights for policy 0, policy_version 123600 (0.0007) [2023-12-27 06:30:46,255][20347] Updated weights for policy 0, policy_version 123610 (0.0008) [2023-12-27 06:30:46,312][20347] Updated weights for policy 0, policy_version 123620 (0.0008) [2023-12-27 06:30:46,717][20381] Updated weights for policy 1, policy_version 123240 (0.0008) [2023-12-27 06:30:46,776][20381] Updated weights for policy 1, policy_version 123250 (0.0008) [2023-12-27 06:30:46,833][20381] Updated weights for policy 1, policy_version 123260 (0.0008) [2023-12-27 06:30:47,032][20347] Updated weights for policy 0, policy_version 123630 (0.0008) [2023-12-27 06:30:47,084][20347] Updated weights for policy 0, policy_version 123640 (0.0008) [2023-12-27 06:30:47,128][20347] Updated weights for policy 0, policy_version 123650 (0.0007) [2023-12-27 06:30:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 63234048. Throughput: 0: 10221.7, 1: 10374.2. Samples: 63243976. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.930')] [2023-12-27 06:30:47,412][20381] Updated weights for policy 1, policy_version 123270 (0.0008) [2023-12-27 06:30:47,463][20381] Updated weights for policy 1, policy_version 123280 (0.0007) [2023-12-27 06:30:47,517][20381] Updated weights for policy 1, policy_version 123290 (0.0006) [2023-12-27 06:30:47,844][20347] Updated weights for policy 0, policy_version 123660 (0.0007) [2023-12-27 06:30:47,904][20347] Updated weights for policy 0, policy_version 123670 (0.0008) [2023-12-27 06:30:47,958][20347] Updated weights for policy 0, policy_version 123680 (0.0008) [2023-12-27 06:30:48,139][20381] Updated weights for policy 1, policy_version 123300 (0.0008) [2023-12-27 06:30:48,193][20381] Updated weights for policy 1, policy_version 123310 (0.0008) [2023-12-27 06:30:48,248][20381] Updated weights for policy 1, policy_version 123320 (0.0008) [2023-12-27 06:30:48,610][20347] Updated weights for policy 0, policy_version 123690 (0.0007) [2023-12-27 06:30:48,671][20347] Updated weights for policy 0, policy_version 123700 (0.0008) [2023-12-27 06:30:48,734][20347] Updated weights for policy 0, policy_version 123710 (0.0007) [2023-12-27 06:30:48,793][20347] Updated weights for policy 0, policy_version 123720 (0.0006) [2023-12-27 06:30:49,056][20381] Updated weights for policy 1, policy_version 123330 (0.0008) [2023-12-27 06:30:49,108][20381] Updated weights for policy 1, policy_version 123340 (0.0008) [2023-12-27 06:30:49,160][20381] Updated weights for policy 1, policy_version 123350 (0.0007) [2023-12-27 06:30:49,215][20381] Updated weights for policy 1, policy_version 123360 (0.0007) [2023-12-27 06:30:49,501][20347] Updated weights for policy 0, policy_version 123730 (0.0006) [2023-12-27 06:30:49,561][20347] Updated weights for policy 0, policy_version 123740 (0.0008) [2023-12-27 06:30:49,621][20347] Updated weights for policy 0, policy_version 123750 (0.0009) [2023-12-27 06:30:49,955][20381] Updated weights for policy 1, policy_version 123370 (0.0008) [2023-12-27 06:30:50,009][20381] Updated weights for policy 1, policy_version 123380 (0.0007) [2023-12-27 06:30:50,066][20381] Updated weights for policy 1, policy_version 123390 (0.0008) [2023-12-27 06:30:50,332][20347] Updated weights for policy 0, policy_version 123760 (0.0007) [2023-12-27 06:30:50,389][20347] Updated weights for policy 0, policy_version 123770 (0.0007) [2023-12-27 06:30:50,453][20347] Updated weights for policy 0, policy_version 123780 (0.0005) [2023-12-27 06:30:50,786][20381] Updated weights for policy 1, policy_version 123400 (0.0011) [2023-12-27 06:30:50,835][20381] Updated weights for policy 1, policy_version 123410 (0.0010) [2023-12-27 06:30:50,891][20381] Updated weights for policy 1, policy_version 123420 (0.0010) [2023-12-27 06:30:51,120][20347] Updated weights for policy 0, policy_version 123790 (0.0007) [2023-12-27 06:30:51,177][20347] Updated weights for policy 0, policy_version 123800 (0.0008) [2023-12-27 06:30:51,230][20347] Updated weights for policy 0, policy_version 123810 (0.0008) [2023-12-27 06:30:51,607][20381] Updated weights for policy 1, policy_version 123430 (0.0010) [2023-12-27 06:30:51,666][20381] Updated weights for policy 1, policy_version 123440 (0.0011) [2023-12-27 06:30:51,726][20381] Updated weights for policy 1, policy_version 123450 (0.0010) [2023-12-27 06:30:51,969][20347] Updated weights for policy 0, policy_version 123820 (0.0007) [2023-12-27 06:30:52,024][20347] Updated weights for policy 0, policy_version 123830 (0.0007) [2023-12-27 06:30:52,077][20347] Updated weights for policy 0, policy_version 123840 (0.0007) [2023-12-27 06:30:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 63332352. Throughput: 0: 10212.2, 1: 10268.0. Samples: 63303560. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.640')] [2023-12-27 06:30:52,366][20381] Updated weights for policy 1, policy_version 123460 (0.0008) [2023-12-27 06:30:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000123848_31719424.pth... [2023-12-27 06:30:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000122632_31408128.pth [2023-12-27 06:30:52,427][20381] Updated weights for policy 1, policy_version 123470 (0.0010) [2023-12-27 06:30:52,488][20381] Updated weights for policy 1, policy_version 123480 (0.0005) [2023-12-27 06:30:52,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000123488_31621120.pth... [2023-12-27 06:30:52,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000122240_31301632.pth [2023-12-27 06:30:52,828][20347] Updated weights for policy 0, policy_version 123850 (0.0008) [2023-12-27 06:30:52,888][20347] Updated weights for policy 0, policy_version 123860 (0.0007) [2023-12-27 06:30:52,944][20347] Updated weights for policy 0, policy_version 123870 (0.0005) [2023-12-27 06:30:52,996][20347] Updated weights for policy 0, policy_version 123880 (0.0006) [2023-12-27 06:30:53,137][20381] Updated weights for policy 1, policy_version 123490 (0.0006) [2023-12-27 06:30:53,193][20381] Updated weights for policy 1, policy_version 123500 (0.0011) [2023-12-27 06:30:53,248][20381] Updated weights for policy 1, policy_version 123510 (0.0010) [2023-12-27 06:30:53,309][20381] Updated weights for policy 1, policy_version 123520 (0.0010) [2023-12-27 06:30:53,601][20347] Updated weights for policy 0, policy_version 123890 (0.0005) [2023-12-27 06:30:53,658][20347] Updated weights for policy 0, policy_version 123900 (0.0006) [2023-12-27 06:30:53,709][20347] Updated weights for policy 0, policy_version 123910 (0.0008) [2023-12-27 06:30:53,954][20381] Updated weights for policy 1, policy_version 123530 (0.0008) [2023-12-27 06:30:54,024][20381] Updated weights for policy 1, policy_version 123540 (0.0005) [2023-12-27 06:30:54,087][20381] Updated weights for policy 1, policy_version 123550 (0.0005) [2023-12-27 06:30:54,404][20347] Updated weights for policy 0, policy_version 123920 (0.0008) [2023-12-27 06:30:54,459][20347] Updated weights for policy 0, policy_version 123930 (0.0007) [2023-12-27 06:30:54,520][20347] Updated weights for policy 0, policy_version 123940 (0.0007) [2023-12-27 06:30:54,696][20381] Updated weights for policy 1, policy_version 123560 (0.0009) [2023-12-27 06:30:54,757][20381] Updated weights for policy 1, policy_version 123570 (0.0006) [2023-12-27 06:30:54,818][20381] Updated weights for policy 1, policy_version 123580 (0.0010) [2023-12-27 06:30:55,190][20347] Updated weights for policy 0, policy_version 123950 (0.0008) [2023-12-27 06:30:55,237][20347] Updated weights for policy 0, policy_version 123960 (0.0010) [2023-12-27 06:30:55,285][20347] Updated weights for policy 0, policy_version 123970 (0.0010) [2023-12-27 06:30:55,490][20381] Updated weights for policy 1, policy_version 123590 (0.0008) [2023-12-27 06:30:55,548][20381] Updated weights for policy 1, policy_version 123600 (0.0007) [2023-12-27 06:30:55,614][20381] Updated weights for policy 1, policy_version 123610 (0.0008) [2023-12-27 06:30:55,929][20347] Updated weights for policy 0, policy_version 123980 (0.0010) [2023-12-27 06:30:55,978][20347] Updated weights for policy 0, policy_version 123990 (0.0007) [2023-12-27 06:30:56,039][20347] Updated weights for policy 0, policy_version 124000 (0.0005) [2023-12-27 06:30:56,192][20381] Updated weights for policy 1, policy_version 123620 (0.0009) [2023-12-27 06:30:56,244][20381] Updated weights for policy 1, policy_version 123630 (0.0010) [2023-12-27 06:30:56,298][20381] Updated weights for policy 1, policy_version 123640 (0.0010) [2023-12-27 06:30:56,604][20347] Updated weights for policy 0, policy_version 124010 (0.0006) [2023-12-27 06:30:56,664][20347] Updated weights for policy 0, policy_version 124020 (0.0010) [2023-12-27 06:30:56,720][20347] Updated weights for policy 0, policy_version 124030 (0.0010) [2023-12-27 06:30:56,780][20347] Updated weights for policy 0, policy_version 124040 (0.0010) [2023-12-27 06:30:56,877][20381] Updated weights for policy 1, policy_version 123650 (0.0006) [2023-12-27 06:30:56,928][20381] Updated weights for policy 1, policy_version 123660 (0.0010) [2023-12-27 06:30:56,984][20381] Updated weights for policy 1, policy_version 123670 (0.0009) [2023-12-27 06:30:57,039][20381] Updated weights for policy 1, policy_version 123680 (0.0005) [2023-12-27 06:30:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 63438848. Throughput: 0: 10179.0, 1: 10368.3. Samples: 63427516. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:30:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.300')] [2023-12-27 06:30:57,442][20347] Updated weights for policy 0, policy_version 124050 (0.0008) [2023-12-27 06:30:57,490][20347] Updated weights for policy 0, policy_version 124060 (0.0005) [2023-12-27 06:30:57,547][20347] Updated weights for policy 0, policy_version 124070 (0.0008) [2023-12-27 06:30:57,608][20381] Updated weights for policy 1, policy_version 123690 (0.0007) [2023-12-27 06:30:57,663][20381] Updated weights for policy 1, policy_version 123700 (0.0010) [2023-12-27 06:30:57,715][20381] Updated weights for policy 1, policy_version 123710 (0.0010) [2023-12-27 06:30:58,206][20347] Updated weights for policy 0, policy_version 124080 (0.0010) [2023-12-27 06:30:58,262][20347] Updated weights for policy 0, policy_version 124090 (0.0010) [2023-12-27 06:30:58,318][20347] Updated weights for policy 0, policy_version 124100 (0.0010) [2023-12-27 06:30:58,421][20381] Updated weights for policy 1, policy_version 123720 (0.0008) [2023-12-27 06:30:58,483][20381] Updated weights for policy 1, policy_version 123730 (0.0007) [2023-12-27 06:30:58,546][20381] Updated weights for policy 1, policy_version 123740 (0.0008) [2023-12-27 06:30:59,058][20347] Updated weights for policy 0, policy_version 124110 (0.0008) [2023-12-27 06:30:59,116][20347] Updated weights for policy 0, policy_version 124120 (0.0007) [2023-12-27 06:30:59,173][20347] Updated weights for policy 0, policy_version 124130 (0.0008) [2023-12-27 06:30:59,306][20381] Updated weights for policy 1, policy_version 123750 (0.0007) [2023-12-27 06:30:59,365][20381] Updated weights for policy 1, policy_version 123760 (0.0007) [2023-12-27 06:30:59,422][20381] Updated weights for policy 1, policy_version 123770 (0.0008) [2023-12-27 06:30:59,900][20347] Updated weights for policy 0, policy_version 124140 (0.0009) [2023-12-27 06:30:59,967][20347] Updated weights for policy 0, policy_version 124150 (0.0011) [2023-12-27 06:31:00,036][20347] Updated weights for policy 0, policy_version 124160 (0.0006) [2023-12-27 06:31:00,138][20381] Updated weights for policy 1, policy_version 123780 (0.0008) [2023-12-27 06:31:00,197][20381] Updated weights for policy 1, policy_version 123790 (0.0007) [2023-12-27 06:31:00,256][20381] Updated weights for policy 1, policy_version 123800 (0.0007) [2023-12-27 06:31:00,599][20347] Updated weights for policy 0, policy_version 124170 (0.0006) [2023-12-27 06:31:00,658][20347] Updated weights for policy 0, policy_version 124180 (0.0006) [2023-12-27 06:31:00,716][20347] Updated weights for policy 0, policy_version 124190 (0.0005) [2023-12-27 06:31:00,774][20347] Updated weights for policy 0, policy_version 124200 (0.0005) [2023-12-27 06:31:01,005][20381] Updated weights for policy 1, policy_version 123810 (0.0008) [2023-12-27 06:31:01,063][20381] Updated weights for policy 1, policy_version 123820 (0.0008) [2023-12-27 06:31:01,117][20381] Updated weights for policy 1, policy_version 123830 (0.0007) [2023-12-27 06:31:01,185][20381] Updated weights for policy 1, policy_version 123840 (0.0007) [2023-12-27 06:31:01,372][20347] Updated weights for policy 0, policy_version 124210 (0.0011) [2023-12-27 06:31:01,429][20347] Updated weights for policy 0, policy_version 124220 (0.0008) [2023-12-27 06:31:01,477][20347] Updated weights for policy 0, policy_version 124230 (0.0008) [2023-12-27 06:31:01,891][20381] Updated weights for policy 1, policy_version 123850 (0.0006) [2023-12-27 06:31:01,941][20381] Updated weights for policy 1, policy_version 123860 (0.0007) [2023-12-27 06:31:01,989][20381] Updated weights for policy 1, policy_version 123870 (0.0005) [2023-12-27 06:31:02,197][20347] Updated weights for policy 0, policy_version 124240 (0.0008) [2023-12-27 06:31:02,259][20347] Updated weights for policy 0, policy_version 124250 (0.0007) [2023-12-27 06:31:02,314][20347] Updated weights for policy 0, policy_version 124260 (0.0008) [2023-12-27 06:31:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 63545344. Throughput: 0: 10264.5, 1: 10387.4. Samples: 63551760. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:31:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.340')] [2023-12-27 06:31:02,639][20381] Updated weights for policy 1, policy_version 123880 (0.0006) [2023-12-27 06:31:02,693][20381] Updated weights for policy 1, policy_version 123890 (0.0008) [2023-12-27 06:31:02,751][20381] Updated weights for policy 1, policy_version 123900 (0.0008) [2023-12-27 06:31:03,030][20347] Updated weights for policy 0, policy_version 124270 (0.0008) [2023-12-27 06:31:03,080][20347] Updated weights for policy 0, policy_version 124280 (0.0008) [2023-12-27 06:31:03,141][20347] Updated weights for policy 0, policy_version 124290 (0.0008) [2023-12-27 06:31:03,461][20381] Updated weights for policy 1, policy_version 123910 (0.0008) [2023-12-27 06:31:03,516][20381] Updated weights for policy 1, policy_version 123920 (0.0008) [2023-12-27 06:31:03,567][20381] Updated weights for policy 1, policy_version 123930 (0.0007) [2023-12-27 06:31:03,848][20347] Updated weights for policy 0, policy_version 124300 (0.0008) [2023-12-27 06:31:03,906][20347] Updated weights for policy 0, policy_version 124310 (0.0007) [2023-12-27 06:31:03,966][20347] Updated weights for policy 0, policy_version 124320 (0.0008) [2023-12-27 06:31:04,192][20381] Updated weights for policy 1, policy_version 123940 (0.0007) [2023-12-27 06:31:04,248][20381] Updated weights for policy 1, policy_version 123950 (0.0008) [2023-12-27 06:31:04,311][20381] Updated weights for policy 1, policy_version 123960 (0.0008) [2023-12-27 06:31:04,751][20347] Updated weights for policy 0, policy_version 124330 (0.0009) [2023-12-27 06:31:04,806][20347] Updated weights for policy 0, policy_version 124340 (0.0008) [2023-12-27 06:31:04,856][20347] Updated weights for policy 0, policy_version 124350 (0.0008) [2023-12-27 06:31:04,915][20347] Updated weights for policy 0, policy_version 124360 (0.0008) [2023-12-27 06:31:04,947][20381] Updated weights for policy 1, policy_version 123970 (0.0007) [2023-12-27 06:31:05,000][20381] Updated weights for policy 1, policy_version 123980 (0.0006) [2023-12-27 06:31:05,049][20381] Updated weights for policy 1, policy_version 123990 (0.0005) [2023-12-27 06:31:05,106][20381] Updated weights for policy 1, policy_version 124000 (0.0008) [2023-12-27 06:31:05,545][20347] Updated weights for policy 0, policy_version 124370 (0.0007) [2023-12-27 06:31:05,598][20347] Updated weights for policy 0, policy_version 124380 (0.0008) [2023-12-27 06:31:05,658][20347] Updated weights for policy 0, policy_version 124390 (0.0008) [2023-12-27 06:31:05,794][20381] Updated weights for policy 1, policy_version 124010 (0.0008) [2023-12-27 06:31:05,848][20381] Updated weights for policy 1, policy_version 124020 (0.0007) [2023-12-27 06:31:05,900][20381] Updated weights for policy 1, policy_version 124030 (0.0007) [2023-12-27 06:31:06,331][20347] Updated weights for policy 0, policy_version 124400 (0.0007) [2023-12-27 06:31:06,381][20347] Updated weights for policy 0, policy_version 124410 (0.0008) [2023-12-27 06:31:06,433][20347] Updated weights for policy 0, policy_version 124420 (0.0007) [2023-12-27 06:31:06,626][20381] Updated weights for policy 1, policy_version 124040 (0.0006) [2023-12-27 06:31:06,686][20381] Updated weights for policy 1, policy_version 124050 (0.0008) [2023-12-27 06:31:06,742][20381] Updated weights for policy 1, policy_version 124060 (0.0008) [2023-12-27 06:31:07,161][20347] Updated weights for policy 0, policy_version 124430 (0.0008) [2023-12-27 06:31:07,212][20347] Updated weights for policy 0, policy_version 124440 (0.0006) [2023-12-27 06:31:07,266][20347] Updated weights for policy 0, policy_version 124450 (0.0005) [2023-12-27 06:31:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 63643648. Throughput: 0: 10244.8, 1: 10416.2. Samples: 63611516. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:31:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.550')] [2023-12-27 06:31:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000124456_31875072.pth... [2023-12-27 06:31:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000123272_31571968.pth [2023-12-27 06:31:07,399][20381] Updated weights for policy 1, policy_version 124070 (0.0006) [2023-12-27 06:31:07,455][20381] Updated weights for policy 1, policy_version 124080 (0.0006) [2023-12-27 06:31:07,514][20381] Updated weights for policy 1, policy_version 124090 (0.0008) [2023-12-27 06:31:07,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000124096_31776768.pth... [2023-12-27 06:31:07,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000122880_31465472.pth [2023-12-27 06:31:07,879][20347] Updated weights for policy 0, policy_version 124460 (0.0006) [2023-12-27 06:31:07,926][20347] Updated weights for policy 0, policy_version 124470 (0.0008) [2023-12-27 06:31:07,990][20347] Updated weights for policy 0, policy_version 124480 (0.0007) [2023-12-27 06:31:08,179][20381] Updated weights for policy 1, policy_version 124100 (0.0006) [2023-12-27 06:31:08,233][20381] Updated weights for policy 1, policy_version 124110 (0.0005) [2023-12-27 06:31:08,293][20381] Updated weights for policy 1, policy_version 124120 (0.0006) [2023-12-27 06:31:08,588][20347] Updated weights for policy 0, policy_version 124490 (0.0005) [2023-12-27 06:31:08,648][20347] Updated weights for policy 0, policy_version 124500 (0.0005) [2023-12-27 06:31:08,705][20347] Updated weights for policy 0, policy_version 124510 (0.0006) [2023-12-27 06:31:08,765][20347] Updated weights for policy 0, policy_version 124520 (0.0007) [2023-12-27 06:31:08,894][20381] Updated weights for policy 1, policy_version 124130 (0.0007) [2023-12-27 06:31:08,944][20381] Updated weights for policy 1, policy_version 124140 (0.0011) [2023-12-27 06:31:08,990][20381] Updated weights for policy 1, policy_version 124150 (0.0010) [2023-12-27 06:31:09,049][20381] Updated weights for policy 1, policy_version 124160 (0.0011) [2023-12-27 06:31:09,382][20347] Updated weights for policy 0, policy_version 124530 (0.0006) [2023-12-27 06:31:09,450][20347] Updated weights for policy 0, policy_version 124540 (0.0008) [2023-12-27 06:31:09,513][20347] Updated weights for policy 0, policy_version 124550 (0.0007) [2023-12-27 06:31:09,790][20381] Updated weights for policy 1, policy_version 124170 (0.0008) [2023-12-27 06:31:09,863][20381] Updated weights for policy 1, policy_version 124180 (0.0007) [2023-12-27 06:31:09,923][20381] Updated weights for policy 1, policy_version 124190 (0.0007) [2023-12-27 06:31:10,174][20347] Updated weights for policy 0, policy_version 124560 (0.0008) [2023-12-27 06:31:10,228][20347] Updated weights for policy 0, policy_version 124570 (0.0007) [2023-12-27 06:31:10,278][20347] Updated weights for policy 0, policy_version 124580 (0.0005) [2023-12-27 06:31:10,646][20381] Updated weights for policy 1, policy_version 124200 (0.0009) [2023-12-27 06:31:10,709][20381] Updated weights for policy 1, policy_version 124210 (0.0008) [2023-12-27 06:31:10,771][20381] Updated weights for policy 1, policy_version 124220 (0.0007) [2023-12-27 06:31:10,979][20347] Updated weights for policy 0, policy_version 124590 (0.0008) [2023-12-27 06:31:11,033][20347] Updated weights for policy 0, policy_version 124600 (0.0008) [2023-12-27 06:31:11,094][20347] Updated weights for policy 0, policy_version 124610 (0.0007) [2023-12-27 06:31:11,475][20381] Updated weights for policy 1, policy_version 124230 (0.0008) [2023-12-27 06:31:11,526][20381] Updated weights for policy 1, policy_version 124240 (0.0008) [2023-12-27 06:31:11,582][20381] Updated weights for policy 1, policy_version 124250 (0.0008) [2023-12-27 06:31:11,825][20347] Updated weights for policy 0, policy_version 124620 (0.0007) [2023-12-27 06:31:11,876][20347] Updated weights for policy 0, policy_version 124630 (0.0007) [2023-12-27 06:31:11,934][20347] Updated weights for policy 0, policy_version 124640 (0.0007) [2023-12-27 06:31:12,321][20381] Updated weights for policy 1, policy_version 124260 (0.0007) [2023-12-27 06:31:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 63741952. Throughput: 0: 10276.3, 1: 10414.7. Samples: 63735636. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 06:31:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.720')] [2023-12-27 06:31:12,383][20381] Updated weights for policy 1, policy_version 124270 (0.0007) [2023-12-27 06:31:12,450][20381] Updated weights for policy 1, policy_version 124280 (0.0011) [2023-12-27 06:31:12,595][20347] Updated weights for policy 0, policy_version 124650 (0.0007) [2023-12-27 06:31:12,651][20347] Updated weights for policy 0, policy_version 124660 (0.0009) [2023-12-27 06:31:12,709][20347] Updated weights for policy 0, policy_version 124670 (0.0007) [2023-12-27 06:31:12,769][20347] Updated weights for policy 0, policy_version 124680 (0.0008) [2023-12-27 06:31:13,109][20381] Updated weights for policy 1, policy_version 124290 (0.0011) [2023-12-27 06:31:13,170][20381] Updated weights for policy 1, policy_version 124300 (0.0010) [2023-12-27 06:31:13,237][20381] Updated weights for policy 1, policy_version 124310 (0.0010) [2023-12-27 06:31:13,286][20381] Updated weights for policy 1, policy_version 124320 (0.0010) [2023-12-27 06:31:13,495][20347] Updated weights for policy 0, policy_version 124690 (0.0007) [2023-12-27 06:31:13,549][20347] Updated weights for policy 0, policy_version 124700 (0.0007) [2023-12-27 06:31:13,605][20347] Updated weights for policy 0, policy_version 124710 (0.0007) [2023-12-27 06:31:13,955][20381] Updated weights for policy 1, policy_version 124330 (0.0005) [2023-12-27 06:31:14,006][20381] Updated weights for policy 1, policy_version 124340 (0.0009) [2023-12-27 06:31:14,061][20381] Updated weights for policy 1, policy_version 124350 (0.0011) [2023-12-27 06:31:14,348][20347] Updated weights for policy 0, policy_version 124720 (0.0007) [2023-12-27 06:31:14,396][20347] Updated weights for policy 0, policy_version 124730 (0.0007) [2023-12-27 06:31:14,447][20347] Updated weights for policy 0, policy_version 124740 (0.0005) [2023-12-27 06:31:14,688][20381] Updated weights for policy 1, policy_version 124360 (0.0010) [2023-12-27 06:31:14,740][20381] Updated weights for policy 1, policy_version 124370 (0.0010) [2023-12-27 06:31:14,804][20381] Updated weights for policy 1, policy_version 124380 (0.0008) [2023-12-27 06:31:15,156][20347] Updated weights for policy 0, policy_version 124750 (0.0005) [2023-12-27 06:31:15,215][20347] Updated weights for policy 0, policy_version 124760 (0.0006) [2023-12-27 06:31:15,277][20347] Updated weights for policy 0, policy_version 124770 (0.0007) [2023-12-27 06:31:15,491][20381] Updated weights for policy 1, policy_version 124390 (0.0005) [2023-12-27 06:31:15,546][20381] Updated weights for policy 1, policy_version 124400 (0.0005) [2023-12-27 06:31:15,600][20381] Updated weights for policy 1, policy_version 124410 (0.0006) [2023-12-27 06:31:15,959][20347] Updated weights for policy 0, policy_version 124780 (0.0008) [2023-12-27 06:31:16,010][20347] Updated weights for policy 0, policy_version 124790 (0.0007) [2023-12-27 06:31:16,060][20347] Updated weights for policy 0, policy_version 124800 (0.0008) [2023-12-27 06:31:16,231][20381] Updated weights for policy 1, policy_version 124420 (0.0007) [2023-12-27 06:31:16,283][20381] Updated weights for policy 1, policy_version 124430 (0.0005) [2023-12-27 06:31:16,333][20381] Updated weights for policy 1, policy_version 124440 (0.0006) [2023-12-27 06:31:16,735][20347] Updated weights for policy 0, policy_version 124810 (0.0008) [2023-12-27 06:31:16,786][20347] Updated weights for policy 0, policy_version 124820 (0.0007) [2023-12-27 06:31:16,839][20347] Updated weights for policy 0, policy_version 124830 (0.0007) [2023-12-27 06:31:16,898][20347] Updated weights for policy 0, policy_version 124840 (0.0008) [2023-12-27 06:31:17,013][20381] Updated weights for policy 1, policy_version 124450 (0.0006) [2023-12-27 06:31:17,061][20381] Updated weights for policy 1, policy_version 124460 (0.0008) [2023-12-27 06:31:17,115][20381] Updated weights for policy 1, policy_version 124470 (0.0007) [2023-12-27 06:31:17,173][20381] Updated weights for policy 1, policy_version 124480 (0.0007) [2023-12-27 06:31:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 63848448. Throughput: 0: 10170.4, 1: 10324.8. Samples: 63856516. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.750')] [2023-12-27 06:31:17,557][20347] Updated weights for policy 0, policy_version 124850 (0.0010) [2023-12-27 06:31:17,607][20347] Updated weights for policy 0, policy_version 124860 (0.0010) [2023-12-27 06:31:17,659][20347] Updated weights for policy 0, policy_version 124870 (0.0010) [2023-12-27 06:31:17,893][20381] Updated weights for policy 1, policy_version 124490 (0.0007) [2023-12-27 06:31:17,938][20381] Updated weights for policy 1, policy_version 124500 (0.0008) [2023-12-27 06:31:17,989][20381] Updated weights for policy 1, policy_version 124510 (0.0005) [2023-12-27 06:31:18,352][20347] Updated weights for policy 0, policy_version 124880 (0.0010) [2023-12-27 06:31:18,414][20347] Updated weights for policy 0, policy_version 124890 (0.0007) [2023-12-27 06:31:18,468][20347] Updated weights for policy 0, policy_version 124900 (0.0009) [2023-12-27 06:31:18,679][20381] Updated weights for policy 1, policy_version 124520 (0.0008) [2023-12-27 06:31:18,728][20381] Updated weights for policy 1, policy_version 124530 (0.0008) [2023-12-27 06:31:18,781][20381] Updated weights for policy 1, policy_version 124540 (0.0007) [2023-12-27 06:31:19,153][20347] Updated weights for policy 0, policy_version 124910 (0.0008) [2023-12-27 06:31:19,209][20347] Updated weights for policy 0, policy_version 124920 (0.0007) [2023-12-27 06:31:19,276][20347] Updated weights for policy 0, policy_version 124930 (0.0008) [2023-12-27 06:31:19,530][20381] Updated weights for policy 1, policy_version 124550 (0.0008) [2023-12-27 06:31:19,591][20381] Updated weights for policy 1, policy_version 124560 (0.0009) [2023-12-27 06:31:19,649][20381] Updated weights for policy 1, policy_version 124570 (0.0009) [2023-12-27 06:31:19,984][20347] Updated weights for policy 0, policy_version 124940 (0.0008) [2023-12-27 06:31:20,045][20347] Updated weights for policy 0, policy_version 124950 (0.0007) [2023-12-27 06:31:20,100][20347] Updated weights for policy 0, policy_version 124960 (0.0006) [2023-12-27 06:31:20,441][20381] Updated weights for policy 1, policy_version 124580 (0.0008) [2023-12-27 06:31:20,501][20381] Updated weights for policy 1, policy_version 124590 (0.0008) [2023-12-27 06:31:20,556][20381] Updated weights for policy 1, policy_version 124600 (0.0007) [2023-12-27 06:31:20,800][20347] Updated weights for policy 0, policy_version 124970 (0.0007) [2023-12-27 06:31:20,862][20347] Updated weights for policy 0, policy_version 124980 (0.0009) [2023-12-27 06:31:20,924][20347] Updated weights for policy 0, policy_version 124990 (0.0008) [2023-12-27 06:31:20,987][20347] Updated weights for policy 0, policy_version 125000 (0.0008) [2023-12-27 06:31:21,279][20381] Updated weights for policy 1, policy_version 124610 (0.0007) [2023-12-27 06:31:21,344][20381] Updated weights for policy 1, policy_version 124620 (0.0006) [2023-12-27 06:31:21,409][20381] Updated weights for policy 1, policy_version 124630 (0.0008) [2023-12-27 06:31:21,469][20381] Updated weights for policy 1, policy_version 124640 (0.0008) [2023-12-27 06:31:21,756][20347] Updated weights for policy 0, policy_version 125010 (0.0007) [2023-12-27 06:31:21,824][20347] Updated weights for policy 0, policy_version 125020 (0.0006) [2023-12-27 06:31:21,890][20347] Updated weights for policy 0, policy_version 125030 (0.0007) [2023-12-27 06:31:22,167][20381] Updated weights for policy 1, policy_version 124650 (0.0007) [2023-12-27 06:31:22,218][20381] Updated weights for policy 1, policy_version 124660 (0.0006) [2023-12-27 06:31:22,281][20381] Updated weights for policy 1, policy_version 124670 (0.0007) [2023-12-27 06:31:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 63946752. Throughput: 0: 10180.8, 1: 10250.7. Samples: 63915952. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '23.750')] [2023-12-27 06:31:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000124672_31924224.pth... [2023-12-27 06:31:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000125032_32022528.pth... [2023-12-27 06:31:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000123848_31719424.pth [2023-12-27 06:31:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000123488_31621120.pth [2023-12-27 06:31:22,506][20347] Updated weights for policy 0, policy_version 125040 (0.0010) [2023-12-27 06:31:22,575][20347] Updated weights for policy 0, policy_version 125050 (0.0010) [2023-12-27 06:31:22,633][20347] Updated weights for policy 0, policy_version 125060 (0.0008) [2023-12-27 06:31:22,967][20381] Updated weights for policy 1, policy_version 124680 (0.0007) [2023-12-27 06:31:23,035][20381] Updated weights for policy 1, policy_version 124690 (0.0006) [2023-12-27 06:31:23,100][20381] Updated weights for policy 1, policy_version 124700 (0.0006) [2023-12-27 06:31:23,343][20347] Updated weights for policy 0, policy_version 125070 (0.0010) [2023-12-27 06:31:23,391][20347] Updated weights for policy 0, policy_version 125080 (0.0005) [2023-12-27 06:31:23,449][20347] Updated weights for policy 0, policy_version 125090 (0.0006) [2023-12-27 06:31:23,690][20381] Updated weights for policy 1, policy_version 124710 (0.0006) [2023-12-27 06:31:23,737][20381] Updated weights for policy 1, policy_version 124720 (0.0005) [2023-12-27 06:31:23,787][20381] Updated weights for policy 1, policy_version 124730 (0.0005) [2023-12-27 06:31:24,047][20347] Updated weights for policy 0, policy_version 125100 (0.0009) [2023-12-27 06:31:24,095][20347] Updated weights for policy 0, policy_version 125110 (0.0010) [2023-12-27 06:31:24,146][20347] Updated weights for policy 0, policy_version 125120 (0.0010) [2023-12-27 06:31:24,431][20381] Updated weights for policy 1, policy_version 124740 (0.0009) [2023-12-27 06:31:24,491][20381] Updated weights for policy 1, policy_version 124750 (0.0007) [2023-12-27 06:31:24,543][20381] Updated weights for policy 1, policy_version 124760 (0.0005) [2023-12-27 06:31:24,801][20347] Updated weights for policy 0, policy_version 125130 (0.0009) [2023-12-27 06:31:24,857][20347] Updated weights for policy 0, policy_version 125140 (0.0007) [2023-12-27 06:31:24,912][20347] Updated weights for policy 0, policy_version 125150 (0.0008) [2023-12-27 06:31:24,957][20347] Updated weights for policy 0, policy_version 125160 (0.0007) [2023-12-27 06:31:25,153][20381] Updated weights for policy 1, policy_version 124770 (0.0005) [2023-12-27 06:31:25,202][20381] Updated weights for policy 1, policy_version 124780 (0.0005) [2023-12-27 06:31:25,252][20381] Updated weights for policy 1, policy_version 124790 (0.0009) [2023-12-27 06:31:25,300][20381] Updated weights for policy 1, policy_version 124800 (0.0010) [2023-12-27 06:31:25,716][20347] Updated weights for policy 0, policy_version 125170 (0.0007) [2023-12-27 06:31:25,772][20347] Updated weights for policy 0, policy_version 125180 (0.0008) [2023-12-27 06:31:25,828][20347] Updated weights for policy 0, policy_version 125190 (0.0008) [2023-12-27 06:31:25,904][20381] Updated weights for policy 1, policy_version 124810 (0.0007) [2023-12-27 06:31:25,965][20381] Updated weights for policy 1, policy_version 124820 (0.0005) [2023-12-27 06:31:26,027][20381] Updated weights for policy 1, policy_version 124830 (0.0008) [2023-12-27 06:31:26,434][20347] Updated weights for policy 0, policy_version 125200 (0.0009) [2023-12-27 06:31:26,486][20347] Updated weights for policy 0, policy_version 125210 (0.0008) [2023-12-27 06:31:26,541][20347] Updated weights for policy 0, policy_version 125220 (0.0007) [2023-12-27 06:31:26,690][20381] Updated weights for policy 1, policy_version 124840 (0.0007) [2023-12-27 06:31:26,736][20381] Updated weights for policy 1, policy_version 124850 (0.0008) [2023-12-27 06:31:26,781][20381] Updated weights for policy 1, policy_version 124860 (0.0008) [2023-12-27 06:31:27,226][20347] Updated weights for policy 0, policy_version 125230 (0.0006) [2023-12-27 06:31:27,275][20347] Updated weights for policy 0, policy_version 125240 (0.0005) [2023-12-27 06:31:27,327][20347] Updated weights for policy 0, policy_version 125250 (0.0007) [2023-12-27 06:31:27,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 64053248. Throughput: 0: 10132.8, 1: 10300.2. Samples: 64039156. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '24.580')] [2023-12-27 06:31:27,415][20381] Updated weights for policy 1, policy_version 124870 (0.0008) [2023-12-27 06:31:27,473][20381] Updated weights for policy 1, policy_version 124880 (0.0007) [2023-12-27 06:31:27,527][20381] Updated weights for policy 1, policy_version 124890 (0.0005) [2023-12-27 06:31:27,966][20347] Updated weights for policy 0, policy_version 125260 (0.0007) [2023-12-27 06:31:28,018][20347] Updated weights for policy 0, policy_version 125270 (0.0008) [2023-12-27 06:31:28,075][20347] Updated weights for policy 0, policy_version 125280 (0.0010) [2023-12-27 06:31:28,139][20381] Updated weights for policy 1, policy_version 124900 (0.0006) [2023-12-27 06:31:28,194][20381] Updated weights for policy 1, policy_version 124910 (0.0006) [2023-12-27 06:31:28,245][20381] Updated weights for policy 1, policy_version 124920 (0.0008) [2023-12-27 06:31:28,736][20347] Updated weights for policy 0, policy_version 125290 (0.0010) [2023-12-27 06:31:28,782][20347] Updated weights for policy 0, policy_version 125300 (0.0008) [2023-12-27 06:31:28,842][20347] Updated weights for policy 0, policy_version 125310 (0.0009) [2023-12-27 06:31:28,901][20347] Updated weights for policy 0, policy_version 125320 (0.0010) [2023-12-27 06:31:28,936][20381] Updated weights for policy 1, policy_version 124930 (0.0011) [2023-12-27 06:31:28,993][20381] Updated weights for policy 1, policy_version 124940 (0.0010) [2023-12-27 06:31:29,051][20381] Updated weights for policy 1, policy_version 124950 (0.0010) [2023-12-27 06:31:29,105][20381] Updated weights for policy 1, policy_version 124960 (0.0010) [2023-12-27 06:31:29,591][20347] Updated weights for policy 0, policy_version 125330 (0.0006) [2023-12-27 06:31:29,637][20347] Updated weights for policy 0, policy_version 125340 (0.0005) [2023-12-27 06:31:29,697][20347] Updated weights for policy 0, policy_version 125350 (0.0010) [2023-12-27 06:31:29,795][20381] Updated weights for policy 1, policy_version 124970 (0.0011) [2023-12-27 06:31:29,852][20381] Updated weights for policy 1, policy_version 124980 (0.0011) [2023-12-27 06:31:29,907][20381] Updated weights for policy 1, policy_version 124990 (0.0011) [2023-12-27 06:31:30,412][20347] Updated weights for policy 0, policy_version 125360 (0.0009) [2023-12-27 06:31:30,460][20347] Updated weights for policy 0, policy_version 125370 (0.0008) [2023-12-27 06:31:30,501][20381] Updated weights for policy 1, policy_version 125000 (0.0006) [2023-12-27 06:31:30,513][20347] Updated weights for policy 0, policy_version 125380 (0.0007) [2023-12-27 06:31:30,554][20381] Updated weights for policy 1, policy_version 125010 (0.0005) [2023-12-27 06:31:30,605][20381] Updated weights for policy 1, policy_version 125020 (0.0009) [2023-12-27 06:31:31,143][20347] Updated weights for policy 0, policy_version 125390 (0.0009) [2023-12-27 06:31:31,195][20347] Updated weights for policy 0, policy_version 125400 (0.0009) [2023-12-27 06:31:31,257][20347] Updated weights for policy 0, policy_version 125410 (0.0009) [2023-12-27 06:31:31,262][20381] Updated weights for policy 1, policy_version 125030 (0.0009) [2023-12-27 06:31:31,315][20381] Updated weights for policy 1, policy_version 125040 (0.0009) [2023-12-27 06:31:31,379][20381] Updated weights for policy 1, policy_version 125050 (0.0010) [2023-12-27 06:31:31,978][20347] Updated weights for policy 0, policy_version 125420 (0.0008) [2023-12-27 06:31:32,032][20347] Updated weights for policy 0, policy_version 125430 (0.0005) [2023-12-27 06:31:32,093][20347] Updated weights for policy 0, policy_version 125440 (0.0006) [2023-12-27 06:31:32,115][20381] Updated weights for policy 1, policy_version 125060 (0.0007) [2023-12-27 06:31:32,167][20381] Updated weights for policy 1, policy_version 125070 (0.0005) [2023-12-27 06:31:32,218][20381] Updated weights for policy 1, policy_version 125080 (0.0005) [2023-12-27 06:31:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 64159744. Throughput: 0: 10192.7, 1: 10266.8. Samples: 64164652. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '24.660')] [2023-12-27 06:31:32,736][20347] Updated weights for policy 0, policy_version 125450 (0.0007) [2023-12-27 06:31:32,786][20347] Updated weights for policy 0, policy_version 125460 (0.0006) [2023-12-27 06:31:32,818][20381] Updated weights for policy 1, policy_version 125090 (0.0008) [2023-12-27 06:31:32,833][20347] Updated weights for policy 0, policy_version 125470 (0.0006) [2023-12-27 06:31:32,875][20381] Updated weights for policy 1, policy_version 125100 (0.0007) [2023-12-27 06:31:32,881][20347] Updated weights for policy 0, policy_version 125480 (0.0005) [2023-12-27 06:31:32,928][20381] Updated weights for policy 1, policy_version 125110 (0.0006) [2023-12-27 06:31:32,983][20381] Updated weights for policy 1, policy_version 125120 (0.0005) [2023-12-27 06:31:33,567][20347] Updated weights for policy 0, policy_version 125490 (0.0006) [2023-12-27 06:31:33,574][20381] Updated weights for policy 1, policy_version 125130 (0.0007) [2023-12-27 06:31:33,624][20347] Updated weights for policy 0, policy_version 125500 (0.0007) [2023-12-27 06:31:33,630][20381] Updated weights for policy 1, policy_version 125140 (0.0006) [2023-12-27 06:31:33,689][20381] Updated weights for policy 1, policy_version 125150 (0.0006) [2023-12-27 06:31:33,691][20347] Updated weights for policy 0, policy_version 125510 (0.0008) [2023-12-27 06:31:34,321][20347] Updated weights for policy 0, policy_version 125520 (0.0010) [2023-12-27 06:31:34,340][20381] Updated weights for policy 1, policy_version 125160 (0.0008) [2023-12-27 06:31:34,379][20347] Updated weights for policy 0, policy_version 125530 (0.0010) [2023-12-27 06:31:34,399][20381] Updated weights for policy 1, policy_version 125170 (0.0006) [2023-12-27 06:31:34,437][20347] Updated weights for policy 0, policy_version 125540 (0.0011) [2023-12-27 06:31:34,457][20381] Updated weights for policy 1, policy_version 125180 (0.0006) [2023-12-27 06:31:35,068][20347] Updated weights for policy 0, policy_version 125550 (0.0010) [2023-12-27 06:31:35,120][20347] Updated weights for policy 0, policy_version 125560 (0.0010) [2023-12-27 06:31:35,179][20347] Updated weights for policy 0, policy_version 125570 (0.0010) [2023-12-27 06:31:35,192][20381] Updated weights for policy 1, policy_version 125190 (0.0006) [2023-12-27 06:31:35,250][20381] Updated weights for policy 1, policy_version 125200 (0.0007) [2023-12-27 06:31:35,304][20381] Updated weights for policy 1, policy_version 125210 (0.0007) [2023-12-27 06:31:35,819][20347] Updated weights for policy 0, policy_version 125580 (0.0010) [2023-12-27 06:31:35,870][20347] Updated weights for policy 0, policy_version 125590 (0.0010) [2023-12-27 06:31:35,918][20381] Updated weights for policy 1, policy_version 125220 (0.0006) [2023-12-27 06:31:35,918][20347] Updated weights for policy 0, policy_version 125600 (0.0010) [2023-12-27 06:31:35,964][20381] Updated weights for policy 1, policy_version 125230 (0.0007) [2023-12-27 06:31:36,012][20381] Updated weights for policy 1, policy_version 125240 (0.0010) [2023-12-27 06:31:36,639][20347] Updated weights for policy 0, policy_version 125610 (0.0010) [2023-12-27 06:31:36,696][20381] Updated weights for policy 1, policy_version 125250 (0.0010) [2023-12-27 06:31:36,697][20347] Updated weights for policy 0, policy_version 125620 (0.0007) [2023-12-27 06:31:36,749][20381] Updated weights for policy 1, policy_version 125260 (0.0011) [2023-12-27 06:31:36,757][20347] Updated weights for policy 0, policy_version 125630 (0.0010) [2023-12-27 06:31:36,814][20381] Updated weights for policy 1, policy_version 125270 (0.0011) [2023-12-27 06:31:36,817][20347] Updated weights for policy 0, policy_version 125640 (0.0009) [2023-12-27 06:31:36,872][20381] Updated weights for policy 1, policy_version 125280 (0.0011) [2023-12-27 06:31:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 64258048. Throughput: 0: 10240.8, 1: 10316.4. Samples: 64228632. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '24.600')] [2023-12-27 06:31:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000125280_32079872.pth... [2023-12-27 06:31:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000125640_32178176.pth... [2023-12-27 06:31:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000124096_31776768.pth [2023-12-27 06:31:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000124456_31875072.pth [2023-12-27 06:31:37,487][20347] Updated weights for policy 0, policy_version 125650 (0.0010) [2023-12-27 06:31:37,491][20381] Updated weights for policy 1, policy_version 125290 (0.0010) [2023-12-27 06:31:37,533][20347] Updated weights for policy 0, policy_version 125660 (0.0010) [2023-12-27 06:31:37,554][20381] Updated weights for policy 1, policy_version 125300 (0.0010) [2023-12-27 06:31:37,581][20347] Updated weights for policy 0, policy_version 125670 (0.0010) [2023-12-27 06:31:37,613][20381] Updated weights for policy 1, policy_version 125310 (0.0010) [2023-12-27 06:31:38,240][20347] Updated weights for policy 0, policy_version 125680 (0.0010) [2023-12-27 06:31:38,297][20347] Updated weights for policy 0, policy_version 125690 (0.0010) [2023-12-27 06:31:38,307][20381] Updated weights for policy 1, policy_version 125320 (0.0010) [2023-12-27 06:31:38,357][20347] Updated weights for policy 0, policy_version 125700 (0.0009) [2023-12-27 06:31:38,367][20381] Updated weights for policy 1, policy_version 125330 (0.0010) [2023-12-27 06:31:38,424][20381] Updated weights for policy 1, policy_version 125340 (0.0011) [2023-12-27 06:31:39,050][20347] Updated weights for policy 0, policy_version 125710 (0.0010) [2023-12-27 06:31:39,112][20347] Updated weights for policy 0, policy_version 125720 (0.0010) [2023-12-27 06:31:39,132][20381] Updated weights for policy 1, policy_version 125350 (0.0007) [2023-12-27 06:31:39,172][20347] Updated weights for policy 0, policy_version 125730 (0.0010) [2023-12-27 06:31:39,186][20381] Updated weights for policy 1, policy_version 125360 (0.0006) [2023-12-27 06:31:39,244][20381] Updated weights for policy 1, policy_version 125370 (0.0008) [2023-12-27 06:31:39,934][20381] Updated weights for policy 1, policy_version 125380 (0.0007) [2023-12-27 06:31:39,987][20381] Updated weights for policy 1, policy_version 125390 (0.0006) [2023-12-27 06:31:40,030][20347] Updated weights for policy 0, policy_version 125740 (0.0008) [2023-12-27 06:31:40,047][20381] Updated weights for policy 1, policy_version 125400 (0.0006) [2023-12-27 06:31:40,085][20347] Updated weights for policy 0, policy_version 125750 (0.0008) [2023-12-27 06:31:40,142][20347] Updated weights for policy 0, policy_version 125760 (0.0009) [2023-12-27 06:31:40,676][20381] Updated weights for policy 1, policy_version 125410 (0.0007) [2023-12-27 06:31:40,733][20381] Updated weights for policy 1, policy_version 125420 (0.0008) [2023-12-27 06:31:40,786][20381] Updated weights for policy 1, policy_version 125430 (0.0007) [2023-12-27 06:31:40,840][20381] Updated weights for policy 1, policy_version 125440 (0.0005) [2023-12-27 06:31:40,902][20347] Updated weights for policy 0, policy_version 125770 (0.0008) [2023-12-27 06:31:40,953][20347] Updated weights for policy 0, policy_version 125780 (0.0010) [2023-12-27 06:31:41,010][20347] Updated weights for policy 0, policy_version 125790 (0.0008) [2023-12-27 06:31:41,073][20347] Updated weights for policy 0, policy_version 125800 (0.0008) [2023-12-27 06:31:41,466][20381] Updated weights for policy 1, policy_version 125450 (0.0006) [2023-12-27 06:31:41,522][20381] Updated weights for policy 1, policy_version 125460 (0.0006) [2023-12-27 06:31:41,574][20381] Updated weights for policy 1, policy_version 125470 (0.0006) [2023-12-27 06:31:41,792][20347] Updated weights for policy 0, policy_version 125810 (0.0009) [2023-12-27 06:31:41,844][20347] Updated weights for policy 0, policy_version 125820 (0.0008) [2023-12-27 06:31:41,898][20347] Updated weights for policy 0, policy_version 125830 (0.0007) [2023-12-27 06:31:42,299][20381] Updated weights for policy 1, policy_version 125480 (0.0008) [2023-12-27 06:31:42,361][20381] Updated weights for policy 1, policy_version 125490 (0.0008) [2023-12-27 06:31:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 64356352. Throughput: 0: 10204.4, 1: 10319.7. Samples: 64351104. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '24.480')] [2023-12-27 06:31:42,421][20381] Updated weights for policy 1, policy_version 125500 (0.0006) [2023-12-27 06:31:42,534][20347] Updated weights for policy 0, policy_version 125840 (0.0005) [2023-12-27 06:31:42,586][20347] Updated weights for policy 0, policy_version 125850 (0.0005) [2023-12-27 06:31:42,643][20347] Updated weights for policy 0, policy_version 125860 (0.0007) [2023-12-27 06:31:43,060][20381] Updated weights for policy 1, policy_version 125510 (0.0007) [2023-12-27 06:31:43,116][20381] Updated weights for policy 1, policy_version 125520 (0.0007) [2023-12-27 06:31:43,172][20381] Updated weights for policy 1, policy_version 125530 (0.0005) [2023-12-27 06:31:43,271][20347] Updated weights for policy 0, policy_version 125870 (0.0008) [2023-12-27 06:31:43,334][20347] Updated weights for policy 0, policy_version 125880 (0.0007) [2023-12-27 06:31:43,392][20347] Updated weights for policy 0, policy_version 125890 (0.0008) [2023-12-27 06:31:43,792][20381] Updated weights for policy 1, policy_version 125540 (0.0007) [2023-12-27 06:31:43,856][20381] Updated weights for policy 1, policy_version 125550 (0.0008) [2023-12-27 06:31:43,918][20381] Updated weights for policy 1, policy_version 125560 (0.0007) [2023-12-27 06:31:44,119][20347] Updated weights for policy 0, policy_version 125900 (0.0008) [2023-12-27 06:31:44,176][20347] Updated weights for policy 0, policy_version 125910 (0.0008) [2023-12-27 06:31:44,231][20347] Updated weights for policy 0, policy_version 125920 (0.0008) [2023-12-27 06:31:44,541][20381] Updated weights for policy 1, policy_version 125570 (0.0007) [2023-12-27 06:31:44,600][20381] Updated weights for policy 1, policy_version 125580 (0.0007) [2023-12-27 06:31:44,656][20381] Updated weights for policy 1, policy_version 125590 (0.0005) [2023-12-27 06:31:44,710][20381] Updated weights for policy 1, policy_version 125600 (0.0005) [2023-12-27 06:31:44,969][20347] Updated weights for policy 0, policy_version 125930 (0.0007) [2023-12-27 06:31:45,027][20347] Updated weights for policy 0, policy_version 125940 (0.0006) [2023-12-27 06:31:45,086][20347] Updated weights for policy 0, policy_version 125950 (0.0005) [2023-12-27 06:31:45,146][20347] Updated weights for policy 0, policy_version 125960 (0.0005) [2023-12-27 06:31:45,376][20381] Updated weights for policy 1, policy_version 125610 (0.0008) [2023-12-27 06:31:45,434][20381] Updated weights for policy 1, policy_version 125620 (0.0008) [2023-12-27 06:31:45,482][20381] Updated weights for policy 1, policy_version 125630 (0.0009) [2023-12-27 06:31:45,716][20347] Updated weights for policy 0, policy_version 125970 (0.0008) [2023-12-27 06:31:45,784][20347] Updated weights for policy 0, policy_version 125980 (0.0007) [2023-12-27 06:31:45,848][20347] Updated weights for policy 0, policy_version 125990 (0.0008) [2023-12-27 06:31:46,108][20381] Updated weights for policy 1, policy_version 125640 (0.0006) [2023-12-27 06:31:46,157][20381] Updated weights for policy 1, policy_version 125650 (0.0005) [2023-12-27 06:31:46,210][20381] Updated weights for policy 1, policy_version 125660 (0.0005) [2023-12-27 06:31:46,593][20347] Updated weights for policy 0, policy_version 126000 (0.0008) [2023-12-27 06:31:46,655][20347] Updated weights for policy 0, policy_version 126010 (0.0007) [2023-12-27 06:31:46,702][20347] Updated weights for policy 0, policy_version 126020 (0.0010) [2023-12-27 06:31:46,777][20381] Updated weights for policy 1, policy_version 125670 (0.0005) [2023-12-27 06:31:46,833][20381] Updated weights for policy 1, policy_version 125680 (0.0005) [2023-12-27 06:31:46,884][20381] Updated weights for policy 1, policy_version 125690 (0.0005) [2023-12-27 06:31:47,330][20347] Updated weights for policy 0, policy_version 126030 (0.0008) [2023-12-27 06:31:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 64462848. Throughput: 0: 10162.1, 1: 10394.9. Samples: 64476828. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '24.980')] [2023-12-27 06:31:47,385][20347] Updated weights for policy 0, policy_version 126040 (0.0007) [2023-12-27 06:31:47,442][20347] Updated weights for policy 0, policy_version 126050 (0.0008) [2023-12-27 06:31:47,538][20381] Updated weights for policy 1, policy_version 125700 (0.0005) [2023-12-27 06:31:47,598][20381] Updated weights for policy 1, policy_version 125710 (0.0005) [2023-12-27 06:31:47,661][20381] Updated weights for policy 1, policy_version 125720 (0.0005) [2023-12-27 06:31:48,120][20347] Updated weights for policy 0, policy_version 126060 (0.0007) [2023-12-27 06:31:48,177][20347] Updated weights for policy 0, policy_version 126070 (0.0006) [2023-12-27 06:31:48,232][20347] Updated weights for policy 0, policy_version 126080 (0.0005) [2023-12-27 06:31:48,310][20381] Updated weights for policy 1, policy_version 125730 (0.0006) [2023-12-27 06:31:48,377][20381] Updated weights for policy 1, policy_version 125740 (0.0008) [2023-12-27 06:31:48,439][20381] Updated weights for policy 1, policy_version 125750 (0.0006) [2023-12-27 06:31:48,502][20381] Updated weights for policy 1, policy_version 125760 (0.0007) [2023-12-27 06:31:48,914][20347] Updated weights for policy 0, policy_version 126090 (0.0007) [2023-12-27 06:31:48,965][20347] Updated weights for policy 0, policy_version 126100 (0.0007) [2023-12-27 06:31:49,014][20347] Updated weights for policy 0, policy_version 126110 (0.0007) [2023-12-27 06:31:49,072][20347] Updated weights for policy 0, policy_version 126120 (0.0007) [2023-12-27 06:31:49,154][20381] Updated weights for policy 1, policy_version 125770 (0.0008) [2023-12-27 06:31:49,203][20381] Updated weights for policy 1, policy_version 125780 (0.0005) [2023-12-27 06:31:49,272][20381] Updated weights for policy 1, policy_version 125790 (0.0010) [2023-12-27 06:31:49,833][20347] Updated weights for policy 0, policy_version 126130 (0.0008) [2023-12-27 06:31:49,894][20347] Updated weights for policy 0, policy_version 126140 (0.0007) [2023-12-27 06:31:49,958][20347] Updated weights for policy 0, policy_version 126150 (0.0007) [2023-12-27 06:31:49,972][20381] Updated weights for policy 1, policy_version 125800 (0.0010) [2023-12-27 06:31:50,027][20381] Updated weights for policy 1, policy_version 125810 (0.0011) [2023-12-27 06:31:50,088][20381] Updated weights for policy 1, policy_version 125820 (0.0011) [2023-12-27 06:31:50,729][20347] Updated weights for policy 0, policy_version 126160 (0.0008) [2023-12-27 06:31:50,789][20347] Updated weights for policy 0, policy_version 126170 (0.0008) [2023-12-27 06:31:50,792][20381] Updated weights for policy 1, policy_version 125830 (0.0009) [2023-12-27 06:31:50,852][20347] Updated weights for policy 0, policy_version 126180 (0.0007) [2023-12-27 06:31:50,855][20381] Updated weights for policy 1, policy_version 125840 (0.0008) [2023-12-27 06:31:50,917][20381] Updated weights for policy 1, policy_version 125850 (0.0007) [2023-12-27 06:31:51,543][20347] Updated weights for policy 0, policy_version 126190 (0.0008) [2023-12-27 06:31:51,599][20347] Updated weights for policy 0, policy_version 126200 (0.0011) [2023-12-27 06:31:51,603][20381] Updated weights for policy 1, policy_version 125860 (0.0009) [2023-12-27 06:31:51,662][20347] Updated weights for policy 0, policy_version 126210 (0.0011) [2023-12-27 06:31:51,663][20381] Updated weights for policy 1, policy_version 125870 (0.0008) [2023-12-27 06:31:51,720][20381] Updated weights for policy 1, policy_version 125880 (0.0010) [2023-12-27 06:31:52,349][20347] Updated weights for policy 0, policy_version 126220 (0.0010) [2023-12-27 06:31:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 64561152. Throughput: 0: 10189.9, 1: 10376.0. Samples: 64536984. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '25.070')] [2023-12-27 06:31:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000125888_32235520.pth... [2023-12-27 06:31:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000124672_31924224.pth [2023-12-27 06:31:52,417][20347] Updated weights for policy 0, policy_version 126230 (0.0006) [2023-12-27 06:31:52,419][20381] Updated weights for policy 1, policy_version 125890 (0.0011) [2023-12-27 06:31:52,475][20347] Updated weights for policy 0, policy_version 126240 (0.0005) [2023-12-27 06:31:52,476][20381] Updated weights for policy 1, policy_version 125900 (0.0011) [2023-12-27 06:31:52,524][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000126248_32333824.pth... [2023-12-27 06:31:52,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000125032_32022528.pth [2023-12-27 06:31:52,532][20381] Updated weights for policy 1, policy_version 125910 (0.0011) [2023-12-27 06:31:52,590][20381] Updated weights for policy 1, policy_version 125920 (0.0011) [2023-12-27 06:31:53,050][20347] Updated weights for policy 0, policy_version 126250 (0.0005) [2023-12-27 06:31:53,113][20347] Updated weights for policy 0, policy_version 126260 (0.0005) [2023-12-27 06:31:53,162][20347] Updated weights for policy 0, policy_version 126270 (0.0005) [2023-12-27 06:31:53,220][20347] Updated weights for policy 0, policy_version 126280 (0.0005) [2023-12-27 06:31:53,239][20381] Updated weights for policy 1, policy_version 125930 (0.0006) [2023-12-27 06:31:53,304][20381] Updated weights for policy 1, policy_version 125940 (0.0005) [2023-12-27 06:31:53,368][20381] Updated weights for policy 1, policy_version 125950 (0.0007) [2023-12-27 06:31:53,814][20347] Updated weights for policy 0, policy_version 126290 (0.0010) [2023-12-27 06:31:53,869][20347] Updated weights for policy 0, policy_version 126300 (0.0010) [2023-12-27 06:31:53,923][20347] Updated weights for policy 0, policy_version 126310 (0.0010) [2023-12-27 06:31:53,953][20381] Updated weights for policy 1, policy_version 125960 (0.0006) [2023-12-27 06:31:54,012][20381] Updated weights for policy 1, policy_version 125970 (0.0006) [2023-12-27 06:31:54,071][20381] Updated weights for policy 1, policy_version 125980 (0.0005) [2023-12-27 06:31:54,583][20347] Updated weights for policy 0, policy_version 126320 (0.0006) [2023-12-27 06:31:54,644][20347] Updated weights for policy 0, policy_version 126330 (0.0005) [2023-12-27 06:31:54,702][20347] Updated weights for policy 0, policy_version 126340 (0.0005) [2023-12-27 06:31:54,733][20381] Updated weights for policy 1, policy_version 125990 (0.0006) [2023-12-27 06:31:54,782][20381] Updated weights for policy 1, policy_version 126000 (0.0011) [2023-12-27 06:31:54,828][20381] Updated weights for policy 1, policy_version 126010 (0.0010) [2023-12-27 06:31:55,283][20347] Updated weights for policy 0, policy_version 126350 (0.0008) [2023-12-27 06:31:55,331][20347] Updated weights for policy 0, policy_version 126360 (0.0010) [2023-12-27 06:31:55,387][20347] Updated weights for policy 0, policy_version 126370 (0.0010) [2023-12-27 06:31:55,429][20381] Updated weights for policy 1, policy_version 126020 (0.0008) [2023-12-27 06:31:55,477][20381] Updated weights for policy 1, policy_version 126030 (0.0010) [2023-12-27 06:31:55,523][20381] Updated weights for policy 1, policy_version 126040 (0.0010) [2023-12-27 06:31:56,007][20347] Updated weights for policy 0, policy_version 126380 (0.0010) [2023-12-27 06:31:56,059][20347] Updated weights for policy 0, policy_version 126390 (0.0010) [2023-12-27 06:31:56,116][20347] Updated weights for policy 0, policy_version 126400 (0.0010) [2023-12-27 06:31:56,228][20381] Updated weights for policy 1, policy_version 126050 (0.0009) [2023-12-27 06:31:56,289][20381] Updated weights for policy 1, policy_version 126060 (0.0005) [2023-12-27 06:31:56,345][20381] Updated weights for policy 1, policy_version 126070 (0.0006) [2023-12-27 06:31:56,404][20381] Updated weights for policy 1, policy_version 126080 (0.0010) [2023-12-27 06:31:56,776][20347] Updated weights for policy 0, policy_version 126410 (0.0010) [2023-12-27 06:31:56,830][20347] Updated weights for policy 0, policy_version 126420 (0.0010) [2023-12-27 06:31:56,875][20347] Updated weights for policy 0, policy_version 126430 (0.0010) [2023-12-27 06:31:56,932][20347] Updated weights for policy 0, policy_version 126440 (0.0010) [2023-12-27 06:31:56,981][20381] Updated weights for policy 1, policy_version 126090 (0.0010) [2023-12-27 06:31:57,025][20381] Updated weights for policy 1, policy_version 126100 (0.0010) [2023-12-27 06:31:57,080][20381] Updated weights for policy 1, policy_version 126110 (0.0010) [2023-12-27 06:31:57,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.4, 300 sec: 20716.0). Total num frames: 64675840. Throughput: 0: 10199.5, 1: 10428.4. Samples: 64663896. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:31:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '25.620')] [2023-12-27 06:31:57,571][20347] Updated weights for policy 0, policy_version 126450 (0.0010) [2023-12-27 06:31:57,625][20347] Updated weights for policy 0, policy_version 126460 (0.0010) [2023-12-27 06:31:57,639][20381] Updated weights for policy 1, policy_version 126120 (0.0005) [2023-12-27 06:31:57,679][20347] Updated weights for policy 0, policy_version 126470 (0.0010) [2023-12-27 06:31:57,690][20381] Updated weights for policy 1, policy_version 126130 (0.0005) [2023-12-27 06:31:57,748][20381] Updated weights for policy 1, policy_version 126140 (0.0006) [2023-12-27 06:31:58,388][20347] Updated weights for policy 0, policy_version 126480 (0.0011) [2023-12-27 06:31:58,400][20381] Updated weights for policy 1, policy_version 126150 (0.0010) [2023-12-27 06:31:58,453][20347] Updated weights for policy 0, policy_version 126490 (0.0011) [2023-12-27 06:31:58,458][20381] Updated weights for policy 1, policy_version 126160 (0.0007) [2023-12-27 06:31:58,513][20347] Updated weights for policy 0, policy_version 126500 (0.0011) [2023-12-27 06:31:58,522][20381] Updated weights for policy 1, policy_version 126170 (0.0008) [2023-12-27 06:31:59,193][20381] Updated weights for policy 1, policy_version 126180 (0.0007) [2023-12-27 06:31:59,226][20347] Updated weights for policy 0, policy_version 126510 (0.0008) [2023-12-27 06:31:59,261][20381] Updated weights for policy 1, policy_version 126190 (0.0008) [2023-12-27 06:31:59,281][20347] Updated weights for policy 0, policy_version 126520 (0.0007) [2023-12-27 06:31:59,322][20381] Updated weights for policy 1, policy_version 126200 (0.0006) [2023-12-27 06:31:59,335][20347] Updated weights for policy 0, policy_version 126530 (0.0006) [2023-12-27 06:31:59,937][20347] Updated weights for policy 0, policy_version 126540 (0.0009) [2023-12-27 06:31:59,949][20381] Updated weights for policy 1, policy_version 126210 (0.0007) [2023-12-27 06:31:59,993][20347] Updated weights for policy 0, policy_version 126550 (0.0008) [2023-12-27 06:31:59,996][20381] Updated weights for policy 1, policy_version 126220 (0.0008) [2023-12-27 06:32:00,046][20347] Updated weights for policy 0, policy_version 126560 (0.0010) [2023-12-27 06:32:00,048][20381] Updated weights for policy 1, policy_version 126230 (0.0008) [2023-12-27 06:32:00,109][20381] Updated weights for policy 1, policy_version 126240 (0.0008) [2023-12-27 06:32:00,655][20347] Updated weights for policy 0, policy_version 126570 (0.0007) [2023-12-27 06:32:00,708][20347] Updated weights for policy 0, policy_version 126580 (0.0005) [2023-12-27 06:32:00,766][20347] Updated weights for policy 0, policy_version 126590 (0.0005) [2023-12-27 06:32:00,813][20347] Updated weights for policy 0, policy_version 126600 (0.0005) [2023-12-27 06:32:00,852][20381] Updated weights for policy 1, policy_version 126250 (0.0007) [2023-12-27 06:32:00,908][20381] Updated weights for policy 1, policy_version 126260 (0.0009) [2023-12-27 06:32:00,963][20381] Updated weights for policy 1, policy_version 126270 (0.0007) [2023-12-27 06:32:01,391][20347] Updated weights for policy 0, policy_version 126610 (0.0008) [2023-12-27 06:32:01,444][20347] Updated weights for policy 0, policy_version 126620 (0.0008) [2023-12-27 06:32:01,493][20347] Updated weights for policy 0, policy_version 126630 (0.0010) [2023-12-27 06:32:01,615][20381] Updated weights for policy 1, policy_version 126280 (0.0007) [2023-12-27 06:32:01,670][20381] Updated weights for policy 1, policy_version 126290 (0.0010) [2023-12-27 06:32:01,732][20381] Updated weights for policy 1, policy_version 126301 (0.0007) [2023-12-27 06:32:02,182][20347] Updated weights for policy 0, policy_version 126640 (0.0008) [2023-12-27 06:32:02,233][20347] Updated weights for policy 0, policy_version 126650 (0.0007) [2023-12-27 06:32:02,298][20347] Updated weights for policy 0, policy_version 126660 (0.0007) [2023-12-27 06:32:02,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 64782336. Throughput: 0: 10293.2, 1: 10483.1. Samples: 64791452. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:32:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '28.420')] [2023-12-27 06:32:02,391][20381] Updated weights for policy 1, policy_version 126311 (0.0007) [2023-12-27 06:32:02,439][20381] Updated weights for policy 1, policy_version 126321 (0.0007) [2023-12-27 06:32:02,490][20381] Updated weights for policy 1, policy_version 126331 (0.0007) [2023-12-27 06:32:02,998][20347] Updated weights for policy 0, policy_version 126670 (0.0007) [2023-12-27 06:32:03,054][20347] Updated weights for policy 0, policy_version 126680 (0.0008) [2023-12-27 06:32:03,107][20347] Updated weights for policy 0, policy_version 126690 (0.0006) [2023-12-27 06:32:03,179][20381] Updated weights for policy 1, policy_version 126341 (0.0008) [2023-12-27 06:32:03,235][20381] Updated weights for policy 1, policy_version 126351 (0.0008) [2023-12-27 06:32:03,293][20381] Updated weights for policy 1, policy_version 126361 (0.0007) [2023-12-27 06:32:03,737][20347] Updated weights for policy 0, policy_version 126700 (0.0006) [2023-12-27 06:32:03,799][20347] Updated weights for policy 0, policy_version 126710 (0.0007) [2023-12-27 06:32:03,852][20347] Updated weights for policy 0, policy_version 126720 (0.0008) [2023-12-27 06:32:03,911][20381] Updated weights for policy 1, policy_version 126371 (0.0008) [2023-12-27 06:32:03,960][20381] Updated weights for policy 1, policy_version 126381 (0.0007) [2023-12-27 06:32:04,009][20381] Updated weights for policy 1, policy_version 126391 (0.0008) [2023-12-27 06:32:04,559][20347] Updated weights for policy 0, policy_version 126730 (0.0008) [2023-12-27 06:32:04,617][20347] Updated weights for policy 0, policy_version 126740 (0.0008) [2023-12-27 06:32:04,671][20347] Updated weights for policy 0, policy_version 126750 (0.0008) [2023-12-27 06:32:04,711][20381] Updated weights for policy 1, policy_version 126401 (0.0007) [2023-12-27 06:32:04,721][20347] Updated weights for policy 0, policy_version 126760 (0.0008) [2023-12-27 06:32:04,765][20381] Updated weights for policy 1, policy_version 126411 (0.0008) [2023-12-27 06:32:04,818][20381] Updated weights for policy 1, policy_version 126421 (0.0008) [2023-12-27 06:32:04,874][20381] Updated weights for policy 1, policy_version 126431 (0.0007) [2023-12-27 06:32:05,456][20347] Updated weights for policy 0, policy_version 126770 (0.0008) [2023-12-27 06:32:05,508][20347] Updated weights for policy 0, policy_version 126780 (0.0008) [2023-12-27 06:32:05,561][20347] Updated weights for policy 0, policy_version 126790 (0.0006) [2023-12-27 06:32:05,574][20381] Updated weights for policy 1, policy_version 126441 (0.0006) [2023-12-27 06:32:05,633][20381] Updated weights for policy 1, policy_version 126451 (0.0007) [2023-12-27 06:32:05,691][20381] Updated weights for policy 1, policy_version 126461 (0.0008) [2023-12-27 06:32:06,234][20347] Updated weights for policy 0, policy_version 126800 (0.0006) [2023-12-27 06:32:06,291][20347] Updated weights for policy 0, policy_version 126810 (0.0008) [2023-12-27 06:32:06,348][20347] Updated weights for policy 0, policy_version 126820 (0.0007) [2023-12-27 06:32:06,479][20381] Updated weights for policy 1, policy_version 126471 (0.0007) [2023-12-27 06:32:06,534][20381] Updated weights for policy 1, policy_version 126481 (0.0008) [2023-12-27 06:32:06,587][20381] Updated weights for policy 1, policy_version 126491 (0.0008) [2023-12-27 06:32:07,048][20347] Updated weights for policy 0, policy_version 126830 (0.0006) [2023-12-27 06:32:07,104][20347] Updated weights for policy 0, policy_version 126840 (0.0007) [2023-12-27 06:32:07,158][20347] Updated weights for policy 0, policy_version 126850 (0.0008) [2023-12-27 06:32:07,287][20381] Updated weights for policy 1, policy_version 126501 (0.0008) [2023-12-27 06:32:07,343][20381] Updated weights for policy 1, policy_version 126511 (0.0008) [2023-12-27 06:32:07,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 64880640. Throughput: 0: 10287.5, 1: 10522.8. Samples: 64852412. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:32:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '29.940')] [2023-12-27 06:32:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000126856_32489472.pth... [2023-12-27 06:32:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000125640_32178176.pth [2023-12-27 06:32:07,404][20381] Updated weights for policy 1, policy_version 126521 (0.0007) [2023-12-27 06:32:07,446][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000126528_32399360.pth... [2023-12-27 06:32:07,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000125280_32079872.pth [2023-12-27 06:32:07,829][20347] Updated weights for policy 0, policy_version 126860 (0.0008) [2023-12-27 06:32:07,889][20347] Updated weights for policy 0, policy_version 126870 (0.0008) [2023-12-27 06:32:07,945][20347] Updated weights for policy 0, policy_version 126880 (0.0008) [2023-12-27 06:32:08,100][20381] Updated weights for policy 1, policy_version 126531 (0.0007) [2023-12-27 06:32:08,152][20381] Updated weights for policy 1, policy_version 126541 (0.0007) [2023-12-27 06:32:08,211][20381] Updated weights for policy 1, policy_version 126551 (0.0007) [2023-12-27 06:32:08,652][20347] Updated weights for policy 0, policy_version 126890 (0.0008) [2023-12-27 06:32:08,712][20347] Updated weights for policy 0, policy_version 126900 (0.0008) [2023-12-27 06:32:08,766][20347] Updated weights for policy 0, policy_version 126910 (0.0007) [2023-12-27 06:32:08,829][20347] Updated weights for policy 0, policy_version 126920 (0.0009) [2023-12-27 06:32:08,910][20381] Updated weights for policy 1, policy_version 126561 (0.0008) [2023-12-27 06:32:08,968][20381] Updated weights for policy 1, policy_version 126571 (0.0007) [2023-12-27 06:32:09,028][20381] Updated weights for policy 1, policy_version 126581 (0.0008) [2023-12-27 06:32:09,083][20381] Updated weights for policy 1, policy_version 126591 (0.0008) [2023-12-27 06:32:09,546][20347] Updated weights for policy 0, policy_version 126930 (0.0006) [2023-12-27 06:32:09,603][20347] Updated weights for policy 0, policy_version 126940 (0.0005) [2023-12-27 06:32:09,656][20347] Updated weights for policy 0, policy_version 126950 (0.0007) [2023-12-27 06:32:09,821][20381] Updated weights for policy 1, policy_version 126601 (0.0008) [2023-12-27 06:32:09,883][20381] Updated weights for policy 1, policy_version 126611 (0.0008) [2023-12-27 06:32:09,945][20381] Updated weights for policy 1, policy_version 126621 (0.0007) [2023-12-27 06:32:10,264][20347] Updated weights for policy 0, policy_version 126960 (0.0006) [2023-12-27 06:32:10,328][20347] Updated weights for policy 0, policy_version 126970 (0.0005) [2023-12-27 06:32:10,389][20347] Updated weights for policy 0, policy_version 126980 (0.0006) [2023-12-27 06:32:10,646][20381] Updated weights for policy 1, policy_version 126631 (0.0007) [2023-12-27 06:32:10,698][20381] Updated weights for policy 1, policy_version 126641 (0.0008) [2023-12-27 06:32:10,750][20381] Updated weights for policy 1, policy_version 126651 (0.0008) [2023-12-27 06:32:11,031][20347] Updated weights for policy 0, policy_version 126990 (0.0008) [2023-12-27 06:32:11,094][20347] Updated weights for policy 0, policy_version 127000 (0.0009) [2023-12-27 06:32:11,160][20347] Updated weights for policy 0, policy_version 127010 (0.0008) [2023-12-27 06:32:11,489][20381] Updated weights for policy 1, policy_version 126661 (0.0008) [2023-12-27 06:32:11,545][20381] Updated weights for policy 1, policy_version 126671 (0.0007) [2023-12-27 06:32:11,612][20381] Updated weights for policy 1, policy_version 126681 (0.0008) [2023-12-27 06:32:11,897][20347] Updated weights for policy 0, policy_version 127020 (0.0008) [2023-12-27 06:32:11,952][20347] Updated weights for policy 0, policy_version 127030 (0.0009) [2023-12-27 06:32:12,003][20347] Updated weights for policy 0, policy_version 127040 (0.0007) [2023-12-27 06:32:12,307][20381] Updated weights for policy 1, policy_version 126691 (0.0009) [2023-12-27 06:32:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 64978944. Throughput: 0: 10312.8, 1: 10431.9. Samples: 64972668. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-12-27 06:32:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '33.700')] [2023-12-27 06:32:12,363][20381] Updated weights for policy 1, policy_version 126701 (0.0009) [2023-12-27 06:32:12,425][20381] Updated weights for policy 1, policy_version 126711 (0.0008) [2023-12-27 06:32:12,727][20347] Updated weights for policy 0, policy_version 127050 (0.0007) [2023-12-27 06:32:12,777][20347] Updated weights for policy 0, policy_version 127060 (0.0008) [2023-12-27 06:32:12,824][20347] Updated weights for policy 0, policy_version 127070 (0.0010) [2023-12-27 06:32:12,876][20347] Updated weights for policy 0, policy_version 127080 (0.0008) [2023-12-27 06:32:13,093][20381] Updated weights for policy 1, policy_version 126721 (0.0009) [2023-12-27 06:32:13,148][20381] Updated weights for policy 1, policy_version 126731 (0.0011) [2023-12-27 06:32:13,204][20381] Updated weights for policy 1, policy_version 126741 (0.0010) [2023-12-27 06:32:13,269][20381] Updated weights for policy 1, policy_version 126751 (0.0009) [2023-12-27 06:32:13,588][20347] Updated weights for policy 0, policy_version 127090 (0.0008) [2023-12-27 06:32:13,646][20347] Updated weights for policy 0, policy_version 127100 (0.0008) [2023-12-27 06:32:13,709][20347] Updated weights for policy 0, policy_version 127110 (0.0007) [2023-12-27 06:32:13,946][20381] Updated weights for policy 1, policy_version 126761 (0.0010) [2023-12-27 06:32:14,011][20381] Updated weights for policy 1, policy_version 126771 (0.0011) [2023-12-27 06:32:14,068][20381] Updated weights for policy 1, policy_version 126781 (0.0010) [2023-12-27 06:32:14,412][20347] Updated weights for policy 0, policy_version 127120 (0.0008) [2023-12-27 06:32:14,468][20347] Updated weights for policy 0, policy_version 127130 (0.0007) [2023-12-27 06:32:14,526][20347] Updated weights for policy 0, policy_version 127140 (0.0007) [2023-12-27 06:32:14,749][20381] Updated weights for policy 1, policy_version 126791 (0.0011) [2023-12-27 06:32:14,816][20381] Updated weights for policy 1, policy_version 126801 (0.0011) [2023-12-27 06:32:14,883][20381] Updated weights for policy 1, policy_version 126811 (0.0011) [2023-12-27 06:32:15,152][20347] Updated weights for policy 0, policy_version 127150 (0.0006) [2023-12-27 06:32:15,212][20347] Updated weights for policy 0, policy_version 127160 (0.0005) [2023-12-27 06:32:15,269][20347] Updated weights for policy 0, policy_version 127170 (0.0007) [2023-12-27 06:32:15,498][20381] Updated weights for policy 1, policy_version 126821 (0.0009) [2023-12-27 06:32:15,562][20381] Updated weights for policy 1, policy_version 126831 (0.0007) [2023-12-27 06:32:15,617][20381] Updated weights for policy 1, policy_version 126841 (0.0006) [2023-12-27 06:32:15,834][20347] Updated weights for policy 0, policy_version 127180 (0.0006) [2023-12-27 06:32:15,889][20347] Updated weights for policy 0, policy_version 127190 (0.0005) [2023-12-27 06:32:15,942][20347] Updated weights for policy 0, policy_version 127200 (0.0005) [2023-12-27 06:32:16,160][20381] Updated weights for policy 1, policy_version 126851 (0.0005) [2023-12-27 06:32:16,221][20381] Updated weights for policy 1, policy_version 126861 (0.0010) [2023-12-27 06:32:16,275][20381] Updated weights for policy 1, policy_version 126871 (0.0008) [2023-12-27 06:32:16,615][20347] Updated weights for policy 0, policy_version 127210 (0.0008) [2023-12-27 06:32:16,670][20347] Updated weights for policy 0, policy_version 127220 (0.0008) [2023-12-27 06:32:16,722][20347] Updated weights for policy 0, policy_version 127230 (0.0008) [2023-12-27 06:32:16,774][20347] Updated weights for policy 0, policy_version 127240 (0.0008) [2023-12-27 06:32:16,883][20381] Updated weights for policy 1, policy_version 126881 (0.0006) [2023-12-27 06:32:16,942][20381] Updated weights for policy 1, policy_version 126891 (0.0008) [2023-12-27 06:32:16,993][20381] Updated weights for policy 1, policy_version 126901 (0.0006) [2023-12-27 06:32:17,044][20381] Updated weights for policy 1, policy_version 126911 (0.0005) [2023-12-27 06:32:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 65085440. Throughput: 0: 10285.2, 1: 10425.0. Samples: 65096616. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '37.310')] [2023-12-27 06:32:17,444][20347] Updated weights for policy 0, policy_version 127250 (0.0009) [2023-12-27 06:32:17,496][20347] Updated weights for policy 0, policy_version 127260 (0.0007) [2023-12-27 06:32:17,555][20347] Updated weights for policy 0, policy_version 127270 (0.0008) [2023-12-27 06:32:17,709][20381] Updated weights for policy 1, policy_version 126921 (0.0008) [2023-12-27 06:32:17,767][20381] Updated weights for policy 1, policy_version 126931 (0.0007) [2023-12-27 06:32:17,822][20381] Updated weights for policy 1, policy_version 126941 (0.0008) [2023-12-27 06:32:18,247][20347] Updated weights for policy 0, policy_version 127280 (0.0007) [2023-12-27 06:32:18,300][20347] Updated weights for policy 0, policy_version 127290 (0.0010) [2023-12-27 06:32:18,361][20347] Updated weights for policy 0, policy_version 127300 (0.0008) [2023-12-27 06:32:18,453][20381] Updated weights for policy 1, policy_version 126951 (0.0008) [2023-12-27 06:32:18,500][20381] Updated weights for policy 1, policy_version 126961 (0.0008) [2023-12-27 06:32:18,548][20381] Updated weights for policy 1, policy_version 126971 (0.0008) [2023-12-27 06:32:18,942][20347] Updated weights for policy 0, policy_version 127310 (0.0007) [2023-12-27 06:32:18,994][20347] Updated weights for policy 0, policy_version 127320 (0.0007) [2023-12-27 06:32:19,053][20347] Updated weights for policy 0, policy_version 127330 (0.0006) [2023-12-27 06:32:19,365][20381] Updated weights for policy 1, policy_version 126981 (0.0008) [2023-12-27 06:32:19,412][20381] Updated weights for policy 1, policy_version 126991 (0.0008) [2023-12-27 06:32:19,486][20381] Updated weights for policy 1, policy_version 127001 (0.0008) [2023-12-27 06:32:19,744][20347] Updated weights for policy 0, policy_version 127340 (0.0008) [2023-12-27 06:32:19,806][20347] Updated weights for policy 0, policy_version 127350 (0.0007) [2023-12-27 06:32:19,870][20347] Updated weights for policy 0, policy_version 127360 (0.0008) [2023-12-27 06:32:20,119][20381] Updated weights for policy 1, policy_version 127011 (0.0008) [2023-12-27 06:32:20,183][20381] Updated weights for policy 1, policy_version 127021 (0.0008) [2023-12-27 06:32:20,246][20381] Updated weights for policy 1, policy_version 127031 (0.0007) [2023-12-27 06:32:20,572][20347] Updated weights for policy 0, policy_version 127370 (0.0009) [2023-12-27 06:32:20,636][20347] Updated weights for policy 0, policy_version 127380 (0.0008) [2023-12-27 06:32:20,698][20347] Updated weights for policy 0, policy_version 127390 (0.0008) [2023-12-27 06:32:20,759][20347] Updated weights for policy 0, policy_version 127400 (0.0007) [2023-12-27 06:32:20,928][20381] Updated weights for policy 1, policy_version 127041 (0.0007) [2023-12-27 06:32:20,988][20381] Updated weights for policy 1, policy_version 127051 (0.0008) [2023-12-27 06:32:21,072][20381] Updated weights for policy 1, policy_version 127061 (0.0008) [2023-12-27 06:32:21,137][20381] Updated weights for policy 1, policy_version 127071 (0.0007) [2023-12-27 06:32:21,508][20347] Updated weights for policy 0, policy_version 127410 (0.0008) [2023-12-27 06:32:21,559][20347] Updated weights for policy 0, policy_version 127420 (0.0008) [2023-12-27 06:32:21,617][20347] Updated weights for policy 0, policy_version 127430 (0.0008) [2023-12-27 06:32:21,845][20381] Updated weights for policy 1, policy_version 127081 (0.0008) [2023-12-27 06:32:21,914][20381] Updated weights for policy 1, policy_version 127091 (0.0008) [2023-12-27 06:32:21,978][20381] Updated weights for policy 1, policy_version 127101 (0.0006) [2023-12-27 06:32:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 65183744. Throughput: 0: 10275.3, 1: 10392.9. Samples: 65158708. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '41.510')] [2023-12-27 06:32:22,368][20347] Updated weights for policy 0, policy_version 127440 (0.0010) [2023-12-27 06:32:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000127104_32546816.pth... [2023-12-27 06:32:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000125888_32235520.pth [2023-12-27 06:32:22,428][20347] Updated weights for policy 0, policy_version 127450 (0.0010) [2023-12-27 06:32:22,484][20347] Updated weights for policy 0, policy_version 127460 (0.0009) [2023-12-27 06:32:22,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000127464_32645120.pth... [2023-12-27 06:32:22,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000126248_32333824.pth [2023-12-27 06:32:22,627][20381] Updated weights for policy 1, policy_version 127111 (0.0006) [2023-12-27 06:32:22,690][20381] Updated weights for policy 1, policy_version 127121 (0.0007) [2023-12-27 06:32:22,782][20381] Updated weights for policy 1, policy_version 127131 (0.0008) [2023-12-27 06:32:23,160][20347] Updated weights for policy 0, policy_version 127470 (0.0009) [2023-12-27 06:32:23,208][20347] Updated weights for policy 0, policy_version 127480 (0.0010) [2023-12-27 06:32:23,252][20347] Updated weights for policy 0, policy_version 127490 (0.0010) [2023-12-27 06:32:23,351][20381] Updated weights for policy 1, policy_version 127141 (0.0008) [2023-12-27 06:32:23,399][20381] Updated weights for policy 1, policy_version 127151 (0.0007) [2023-12-27 06:32:23,448][20381] Updated weights for policy 1, policy_version 127161 (0.0005) [2023-12-27 06:32:23,921][20347] Updated weights for policy 0, policy_version 127500 (0.0010) [2023-12-27 06:32:23,979][20347] Updated weights for policy 0, policy_version 127510 (0.0010) [2023-12-27 06:32:24,037][20347] Updated weights for policy 0, policy_version 127520 (0.0010) [2023-12-27 06:32:24,090][20381] Updated weights for policy 1, policy_version 127171 (0.0007) [2023-12-27 06:32:24,142][20381] Updated weights for policy 1, policy_version 127181 (0.0011) [2023-12-27 06:32:24,201][20381] Updated weights for policy 1, policy_version 127191 (0.0010) [2023-12-27 06:32:24,667][20347] Updated weights for policy 0, policy_version 127530 (0.0010) [2023-12-27 06:32:24,718][20347] Updated weights for policy 0, policy_version 127540 (0.0010) [2023-12-27 06:32:24,766][20347] Updated weights for policy 0, policy_version 127550 (0.0010) [2023-12-27 06:32:24,824][20347] Updated weights for policy 0, policy_version 127560 (0.0006) [2023-12-27 06:32:24,885][20381] Updated weights for policy 1, policy_version 127201 (0.0008) [2023-12-27 06:32:24,933][20381] Updated weights for policy 1, policy_version 127211 (0.0010) [2023-12-27 06:32:24,984][20381] Updated weights for policy 1, policy_version 127221 (0.0010) [2023-12-27 06:32:25,038][20381] Updated weights for policy 1, policy_version 127231 (0.0010) [2023-12-27 06:32:25,500][20347] Updated weights for policy 0, policy_version 127570 (0.0008) [2023-12-27 06:32:25,548][20347] Updated weights for policy 0, policy_version 127580 (0.0005) [2023-12-27 06:32:25,594][20347] Updated weights for policy 0, policy_version 127590 (0.0008) [2023-12-27 06:32:25,682][20381] Updated weights for policy 1, policy_version 127241 (0.0008) [2023-12-27 06:32:25,740][20381] Updated weights for policy 1, policy_version 127251 (0.0007) [2023-12-27 06:32:25,796][20381] Updated weights for policy 1, policy_version 127261 (0.0008) [2023-12-27 06:32:26,155][20347] Updated weights for policy 0, policy_version 127600 (0.0006) [2023-12-27 06:32:26,199][20347] Updated weights for policy 0, policy_version 127610 (0.0007) [2023-12-27 06:32:26,243][20347] Updated weights for policy 0, policy_version 127620 (0.0007) [2023-12-27 06:32:26,511][20381] Updated weights for policy 1, policy_version 127271 (0.0007) [2023-12-27 06:32:26,564][20381] Updated weights for policy 1, policy_version 127281 (0.0007) [2023-12-27 06:32:26,615][20381] Updated weights for policy 1, policy_version 127291 (0.0010) [2023-12-27 06:32:26,910][20347] Updated weights for policy 0, policy_version 127630 (0.0006) [2023-12-27 06:32:26,960][20347] Updated weights for policy 0, policy_version 127640 (0.0005) [2023-12-27 06:32:27,009][20347] Updated weights for policy 0, policy_version 127650 (0.0007) [2023-12-27 06:32:27,257][20381] Updated weights for policy 1, policy_version 127301 (0.0008) [2023-12-27 06:32:27,319][20381] Updated weights for policy 1, policy_version 127311 (0.0006) [2023-12-27 06:32:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 65290240. Throughput: 0: 10324.0, 1: 10378.7. Samples: 65282724. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '44.450')] [2023-12-27 06:32:27,370][20381] Updated weights for policy 1, policy_version 127321 (0.0007) [2023-12-27 06:32:27,668][20347] Updated weights for policy 0, policy_version 127660 (0.0008) [2023-12-27 06:32:27,718][20347] Updated weights for policy 0, policy_version 127670 (0.0005) [2023-12-27 06:32:27,768][20347] Updated weights for policy 0, policy_version 127680 (0.0005) [2023-12-27 06:32:28,047][20381] Updated weights for policy 1, policy_version 127331 (0.0008) [2023-12-27 06:32:28,102][20381] Updated weights for policy 1, policy_version 127341 (0.0008) [2023-12-27 06:32:28,157][20381] Updated weights for policy 1, policy_version 127351 (0.0008) [2023-12-27 06:32:28,351][20347] Updated weights for policy 0, policy_version 127690 (0.0006) [2023-12-27 06:32:28,415][20347] Updated weights for policy 0, policy_version 127700 (0.0008) [2023-12-27 06:32:28,473][20347] Updated weights for policy 0, policy_version 127710 (0.0008) [2023-12-27 06:32:28,531][20347] Updated weights for policy 0, policy_version 127720 (0.0007) [2023-12-27 06:32:28,864][20381] Updated weights for policy 1, policy_version 127361 (0.0007) [2023-12-27 06:32:28,913][20381] Updated weights for policy 1, policy_version 127371 (0.0008) [2023-12-27 06:32:28,959][20381] Updated weights for policy 1, policy_version 127381 (0.0008) [2023-12-27 06:32:29,012][20381] Updated weights for policy 1, policy_version 127391 (0.0008) [2023-12-27 06:32:29,210][20347] Updated weights for policy 0, policy_version 127730 (0.0006) [2023-12-27 06:32:29,272][20347] Updated weights for policy 0, policy_version 127740 (0.0008) [2023-12-27 06:32:29,328][20347] Updated weights for policy 0, policy_version 127750 (0.0008) [2023-12-27 06:32:29,721][20381] Updated weights for policy 1, policy_version 127401 (0.0008) [2023-12-27 06:32:29,783][20381] Updated weights for policy 1, policy_version 127411 (0.0008) [2023-12-27 06:32:29,842][20381] Updated weights for policy 1, policy_version 127421 (0.0009) [2023-12-27 06:32:30,042][20347] Updated weights for policy 0, policy_version 127760 (0.0009) [2023-12-27 06:32:30,099][20347] Updated weights for policy 0, policy_version 127770 (0.0006) [2023-12-27 06:32:30,154][20347] Updated weights for policy 0, policy_version 127780 (0.0007) [2023-12-27 06:32:30,517][20381] Updated weights for policy 1, policy_version 127431 (0.0008) [2023-12-27 06:32:30,566][20381] Updated weights for policy 1, policy_version 127441 (0.0008) [2023-12-27 06:32:30,621][20381] Updated weights for policy 1, policy_version 127451 (0.0006) [2023-12-27 06:32:30,750][20347] Updated weights for policy 0, policy_version 127790 (0.0008) [2023-12-27 06:32:30,806][20347] Updated weights for policy 0, policy_version 127800 (0.0007) [2023-12-27 06:32:30,861][20347] Updated weights for policy 0, policy_version 127810 (0.0005) [2023-12-27 06:32:31,336][20381] Updated weights for policy 1, policy_version 127461 (0.0008) [2023-12-27 06:32:31,401][20381] Updated weights for policy 1, policy_version 127471 (0.0008) [2023-12-27 06:32:31,459][20381] Updated weights for policy 1, policy_version 127481 (0.0006) [2023-12-27 06:32:31,560][20347] Updated weights for policy 0, policy_version 127820 (0.0006) [2023-12-27 06:32:31,615][20347] Updated weights for policy 0, policy_version 127830 (0.0006) [2023-12-27 06:32:31,671][20347] Updated weights for policy 0, policy_version 127840 (0.0007) [2023-12-27 06:32:32,149][20381] Updated weights for policy 1, policy_version 127491 (0.0008) [2023-12-27 06:32:32,203][20381] Updated weights for policy 1, policy_version 127501 (0.0007) [2023-12-27 06:32:32,258][20381] Updated weights for policy 1, policy_version 127511 (0.0008) [2023-12-27 06:32:32,348][20347] Updated weights for policy 0, policy_version 127850 (0.0008) [2023-12-27 06:32:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 65396736. Throughput: 0: 10371.7, 1: 10289.3. Samples: 65406576. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '47.670')] [2023-12-27 06:32:32,408][20347] Updated weights for policy 0, policy_version 127860 (0.0008) [2023-12-27 06:32:32,465][20347] Updated weights for policy 0, policy_version 127870 (0.0007) [2023-12-27 06:32:32,524][20347] Updated weights for policy 0, policy_version 127880 (0.0008) [2023-12-27 06:32:32,912][20381] Updated weights for policy 1, policy_version 127521 (0.0008) [2023-12-27 06:32:32,971][20381] Updated weights for policy 1, policy_version 127531 (0.0008) [2023-12-27 06:32:33,024][20381] Updated weights for policy 1, policy_version 127541 (0.0010) [2023-12-27 06:32:33,077][20381] Updated weights for policy 1, policy_version 127551 (0.0008) [2023-12-27 06:32:33,170][20347] Updated weights for policy 0, policy_version 127890 (0.0007) [2023-12-27 06:32:33,214][20347] Updated weights for policy 0, policy_version 127900 (0.0006) [2023-12-27 06:32:33,269][20347] Updated weights for policy 0, policy_version 127910 (0.0005) [2023-12-27 06:32:33,795][20347] Updated weights for policy 0, policy_version 127920 (0.0005) [2023-12-27 06:32:33,838][20381] Updated weights for policy 1, policy_version 127561 (0.0008) [2023-12-27 06:32:33,846][20347] Updated weights for policy 0, policy_version 127930 (0.0005) [2023-12-27 06:32:33,888][20381] Updated weights for policy 1, policy_version 127571 (0.0008) [2023-12-27 06:32:33,899][20347] Updated weights for policy 0, policy_version 127940 (0.0005) [2023-12-27 06:32:33,942][20381] Updated weights for policy 1, policy_version 127581 (0.0008) [2023-12-27 06:32:34,474][20347] Updated weights for policy 0, policy_version 127950 (0.0006) [2023-12-27 06:32:34,532][20347] Updated weights for policy 0, policy_version 127960 (0.0008) [2023-12-27 06:32:34,582][20347] Updated weights for policy 0, policy_version 127970 (0.0009) [2023-12-27 06:32:34,661][20381] Updated weights for policy 1, policy_version 127591 (0.0007) [2023-12-27 06:32:34,720][20381] Updated weights for policy 1, policy_version 127601 (0.0009) [2023-12-27 06:32:34,776][20381] Updated weights for policy 1, policy_version 127611 (0.0008) [2023-12-27 06:32:35,289][20347] Updated weights for policy 0, policy_version 127980 (0.0007) [2023-12-27 06:32:35,334][20347] Updated weights for policy 0, policy_version 127990 (0.0007) [2023-12-27 06:32:35,385][20347] Updated weights for policy 0, policy_version 128000 (0.0007) [2023-12-27 06:32:35,481][20381] Updated weights for policy 1, policy_version 127621 (0.0006) [2023-12-27 06:32:35,542][20381] Updated weights for policy 1, policy_version 127631 (0.0006) [2023-12-27 06:32:35,602][20381] Updated weights for policy 1, policy_version 127641 (0.0011) [2023-12-27 06:32:36,122][20347] Updated weights for policy 0, policy_version 128010 (0.0007) [2023-12-27 06:32:36,175][20347] Updated weights for policy 0, policy_version 128020 (0.0007) [2023-12-27 06:32:36,234][20347] Updated weights for policy 0, policy_version 128030 (0.0007) [2023-12-27 06:32:36,259][20381] Updated weights for policy 1, policy_version 127651 (0.0010) [2023-12-27 06:32:36,294][20347] Updated weights for policy 0, policy_version 128040 (0.0007) [2023-12-27 06:32:36,315][20381] Updated weights for policy 1, policy_version 127661 (0.0008) [2023-12-27 06:32:36,375][20381] Updated weights for policy 1, policy_version 127671 (0.0008) [2023-12-27 06:32:36,999][20347] Updated weights for policy 0, policy_version 128050 (0.0010) [2023-12-27 06:32:37,020][20381] Updated weights for policy 1, policy_version 127681 (0.0007) [2023-12-27 06:32:37,054][20347] Updated weights for policy 0, policy_version 128060 (0.0010) [2023-12-27 06:32:37,082][20381] Updated weights for policy 1, policy_version 127691 (0.0006) [2023-12-27 06:32:37,113][20347] Updated weights for policy 0, policy_version 128070 (0.0009) [2023-12-27 06:32:37,145][20381] Updated weights for policy 1, policy_version 127701 (0.0007) [2023-12-27 06:32:37,204][20381] Updated weights for policy 1, policy_version 127711 (0.0008) [2023-12-27 06:32:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 65503232. Throughput: 0: 10450.6, 1: 10275.2. Samples: 65469648. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '51.460')] [2023-12-27 06:32:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000128072_32800768.pth... [2023-12-27 06:32:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000127712_32702464.pth... [2023-12-27 06:32:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000126528_32399360.pth [2023-12-27 06:32:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000126856_32489472.pth [2023-12-27 06:32:37,768][20347] Updated weights for policy 0, policy_version 128080 (0.0006) [2023-12-27 06:32:37,820][20347] Updated weights for policy 0, policy_version 128090 (0.0005) [2023-12-27 06:32:37,850][20381] Updated weights for policy 1, policy_version 127721 (0.0008) [2023-12-27 06:32:37,872][20347] Updated weights for policy 0, policy_version 128100 (0.0005) [2023-12-27 06:32:37,896][20381] Updated weights for policy 1, policy_version 127731 (0.0008) [2023-12-27 06:32:37,945][20381] Updated weights for policy 1, policy_version 127741 (0.0009) [2023-12-27 06:32:38,461][20347] Updated weights for policy 0, policy_version 128110 (0.0006) [2023-12-27 06:32:38,516][20347] Updated weights for policy 0, policy_version 128120 (0.0008) [2023-12-27 06:32:38,569][20347] Updated weights for policy 0, policy_version 128130 (0.0007) [2023-12-27 06:32:38,664][20381] Updated weights for policy 1, policy_version 127751 (0.0007) [2023-12-27 06:32:38,712][20381] Updated weights for policy 1, policy_version 127761 (0.0007) [2023-12-27 06:32:38,759][20381] Updated weights for policy 1, policy_version 127771 (0.0007) [2023-12-27 06:32:39,249][20347] Updated weights for policy 0, policy_version 128140 (0.0008) [2023-12-27 06:32:39,312][20347] Updated weights for policy 0, policy_version 128150 (0.0009) [2023-12-27 06:32:39,382][20347] Updated weights for policy 0, policy_version 128160 (0.0010) [2023-12-27 06:32:39,422][20381] Updated weights for policy 1, policy_version 127781 (0.0006) [2023-12-27 06:32:39,479][20381] Updated weights for policy 1, policy_version 127791 (0.0006) [2023-12-27 06:32:39,531][20381] Updated weights for policy 1, policy_version 127801 (0.0007) [2023-12-27 06:32:40,156][20347] Updated weights for policy 0, policy_version 128170 (0.0009) [2023-12-27 06:32:40,213][20347] Updated weights for policy 0, policy_version 128180 (0.0007) [2023-12-27 06:32:40,275][20347] Updated weights for policy 0, policy_version 128190 (0.0007) [2023-12-27 06:32:40,291][20381] Updated weights for policy 1, policy_version 127811 (0.0008) [2023-12-27 06:32:40,324][20347] Updated weights for policy 0, policy_version 128200 (0.0008) [2023-12-27 06:32:40,342][20381] Updated weights for policy 1, policy_version 127821 (0.0006) [2023-12-27 06:32:40,399][20381] Updated weights for policy 1, policy_version 127831 (0.0007) [2023-12-27 06:32:41,014][20381] Updated weights for policy 1, policy_version 127841 (0.0007) [2023-12-27 06:32:41,073][20381] Updated weights for policy 1, policy_version 127851 (0.0007) [2023-12-27 06:32:41,076][20347] Updated weights for policy 0, policy_version 128210 (0.0007) [2023-12-27 06:32:41,133][20381] Updated weights for policy 1, policy_version 127861 (0.0008) [2023-12-27 06:32:41,139][20347] Updated weights for policy 0, policy_version 128220 (0.0006) [2023-12-27 06:32:41,187][20381] Updated weights for policy 1, policy_version 127871 (0.0007) [2023-12-27 06:32:41,208][20347] Updated weights for policy 0, policy_version 128230 (0.0007) [2023-12-27 06:32:41,907][20381] Updated weights for policy 1, policy_version 127881 (0.0006) [2023-12-27 06:32:41,932][20347] Updated weights for policy 0, policy_version 128240 (0.0006) [2023-12-27 06:32:41,965][20381] Updated weights for policy 1, policy_version 127891 (0.0008) [2023-12-27 06:32:41,990][20347] Updated weights for policy 0, policy_version 128250 (0.0005) [2023-12-27 06:32:42,024][20381] Updated weights for policy 1, policy_version 127901 (0.0008) [2023-12-27 06:32:42,040][20347] Updated weights for policy 0, policy_version 128260 (0.0008) [2023-12-27 06:32:42,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 65601536. Throughput: 0: 10372.6, 1: 10241.1. Samples: 65591512. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '58.370')] [2023-12-27 06:32:42,698][20347] Updated weights for policy 0, policy_version 128270 (0.0009) [2023-12-27 06:32:42,743][20381] Updated weights for policy 1, policy_version 127911 (0.0008) [2023-12-27 06:32:42,753][20347] Updated weights for policy 0, policy_version 128280 (0.0006) [2023-12-27 06:32:42,803][20381] Updated weights for policy 1, policy_version 127921 (0.0006) [2023-12-27 06:32:42,817][20347] Updated weights for policy 0, policy_version 128290 (0.0007) [2023-12-27 06:32:42,866][20381] Updated weights for policy 1, policy_version 127931 (0.0007) [2023-12-27 06:32:43,461][20347] Updated weights for policy 0, policy_version 128300 (0.0007) [2023-12-27 06:32:43,521][20347] Updated weights for policy 0, policy_version 128310 (0.0008) [2023-12-27 06:32:43,572][20381] Updated weights for policy 1, policy_version 127941 (0.0009) [2023-12-27 06:32:43,588][20347] Updated weights for policy 0, policy_version 128320 (0.0007) [2023-12-27 06:32:43,621][20381] Updated weights for policy 1, policy_version 127951 (0.0005) [2023-12-27 06:32:43,680][20381] Updated weights for policy 1, policy_version 127961 (0.0008) [2023-12-27 06:32:44,277][20347] Updated weights for policy 0, policy_version 128330 (0.0007) [2023-12-27 06:32:44,325][20381] Updated weights for policy 1, policy_version 127971 (0.0008) [2023-12-27 06:32:44,339][20347] Updated weights for policy 0, policy_version 128340 (0.0007) [2023-12-27 06:32:44,377][20381] Updated weights for policy 1, policy_version 127981 (0.0007) [2023-12-27 06:32:44,383][20347] Updated weights for policy 0, policy_version 128350 (0.0006) [2023-12-27 06:32:44,430][20381] Updated weights for policy 1, policy_version 127991 (0.0007) [2023-12-27 06:32:44,433][20347] Updated weights for policy 0, policy_version 128360 (0.0007) [2023-12-27 06:32:45,149][20347] Updated weights for policy 0, policy_version 128370 (0.0008) [2023-12-27 06:32:45,151][20381] Updated weights for policy 1, policy_version 128001 (0.0007) [2023-12-27 06:32:45,212][20381] Updated weights for policy 1, policy_version 128011 (0.0006) [2023-12-27 06:32:45,214][20347] Updated weights for policy 0, policy_version 128380 (0.0007) [2023-12-27 06:32:45,269][20347] Updated weights for policy 0, policy_version 128390 (0.0007) [2023-12-27 06:32:45,275][20381] Updated weights for policy 1, policy_version 128021 (0.0006) [2023-12-27 06:32:45,337][20381] Updated weights for policy 1, policy_version 128031 (0.0008) [2023-12-27 06:32:45,974][20347] Updated weights for policy 0, policy_version 128400 (0.0008) [2023-12-27 06:32:45,991][20381] Updated weights for policy 1, policy_version 128041 (0.0007) [2023-12-27 06:32:46,020][20347] Updated weights for policy 0, policy_version 128410 (0.0005) [2023-12-27 06:32:46,045][20381] Updated weights for policy 1, policy_version 128051 (0.0005) [2023-12-27 06:32:46,072][20347] Updated weights for policy 0, policy_version 128420 (0.0005) [2023-12-27 06:32:46,100][20381] Updated weights for policy 1, policy_version 128061 (0.0005) [2023-12-27 06:32:46,689][20381] Updated weights for policy 1, policy_version 128071 (0.0008) [2023-12-27 06:32:46,748][20381] Updated weights for policy 1, policy_version 128081 (0.0007) [2023-12-27 06:32:46,780][20347] Updated weights for policy 0, policy_version 128430 (0.0006) [2023-12-27 06:32:46,804][20381] Updated weights for policy 1, policy_version 128091 (0.0008) [2023-12-27 06:32:46,836][20347] Updated weights for policy 0, policy_version 128440 (0.0007) [2023-12-27 06:32:46,896][20347] Updated weights for policy 0, policy_version 128450 (0.0008) [2023-12-27 06:32:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 65699840. Throughput: 0: 10281.8, 1: 10186.1. Samples: 65712508. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '62.370')] [2023-12-27 06:32:47,463][20381] Updated weights for policy 1, policy_version 128101 (0.0009) [2023-12-27 06:32:47,523][20381] Updated weights for policy 1, policy_version 128111 (0.0009) [2023-12-27 06:32:47,575][20381] Updated weights for policy 1, policy_version 128121 (0.0010) [2023-12-27 06:32:47,595][20347] Updated weights for policy 0, policy_version 128460 (0.0007) [2023-12-27 06:32:47,644][20347] Updated weights for policy 0, policy_version 128470 (0.0006) [2023-12-27 06:32:47,702][20347] Updated weights for policy 0, policy_version 128480 (0.0008) [2023-12-27 06:32:48,235][20381] Updated weights for policy 1, policy_version 128131 (0.0010) [2023-12-27 06:32:48,289][20381] Updated weights for policy 1, policy_version 128141 (0.0010) [2023-12-27 06:32:48,356][20381] Updated weights for policy 1, policy_version 128151 (0.0011) [2023-12-27 06:32:48,372][20347] Updated weights for policy 0, policy_version 128490 (0.0007) [2023-12-27 06:32:48,426][20347] Updated weights for policy 0, policy_version 128500 (0.0007) [2023-12-27 06:32:48,493][20347] Updated weights for policy 0, policy_version 128510 (0.0008) [2023-12-27 06:32:48,554][20347] Updated weights for policy 0, policy_version 128520 (0.0008) [2023-12-27 06:32:48,985][20381] Updated weights for policy 1, policy_version 128161 (0.0011) [2023-12-27 06:32:49,044][20381] Updated weights for policy 1, policy_version 128171 (0.0011) [2023-12-27 06:32:49,099][20381] Updated weights for policy 1, policy_version 128181 (0.0010) [2023-12-27 06:32:49,163][20381] Updated weights for policy 1, policy_version 128191 (0.0010) [2023-12-27 06:32:49,271][20347] Updated weights for policy 0, policy_version 128530 (0.0007) [2023-12-27 06:32:49,323][20347] Updated weights for policy 0, policy_version 128540 (0.0007) [2023-12-27 06:32:49,377][20347] Updated weights for policy 0, policy_version 128550 (0.0006) [2023-12-27 06:32:49,857][20381] Updated weights for policy 1, policy_version 128201 (0.0007) [2023-12-27 06:32:49,913][20381] Updated weights for policy 1, policy_version 128211 (0.0008) [2023-12-27 06:32:49,976][20381] Updated weights for policy 1, policy_version 128221 (0.0008) [2023-12-27 06:32:50,121][20347] Updated weights for policy 0, policy_version 128560 (0.0008) [2023-12-27 06:32:50,179][20347] Updated weights for policy 0, policy_version 128570 (0.0007) [2023-12-27 06:32:50,232][20347] Updated weights for policy 0, policy_version 128580 (0.0008) [2023-12-27 06:32:50,661][20381] Updated weights for policy 1, policy_version 128231 (0.0010) [2023-12-27 06:32:50,722][20381] Updated weights for policy 1, policy_version 128241 (0.0011) [2023-12-27 06:32:50,775][20381] Updated weights for policy 1, policy_version 128251 (0.0011) [2023-12-27 06:32:50,933][20347] Updated weights for policy 0, policy_version 128590 (0.0009) [2023-12-27 06:32:50,980][20347] Updated weights for policy 0, policy_version 128600 (0.0007) [2023-12-27 06:32:51,039][20347] Updated weights for policy 0, policy_version 128610 (0.0007) [2023-12-27 06:32:51,539][20381] Updated weights for policy 1, policy_version 128261 (0.0011) [2023-12-27 06:32:51,593][20381] Updated weights for policy 1, policy_version 128271 (0.0011) [2023-12-27 06:32:51,655][20381] Updated weights for policy 1, policy_version 128281 (0.0011) [2023-12-27 06:32:51,693][20347] Updated weights for policy 0, policy_version 128620 (0.0009) [2023-12-27 06:32:51,752][20347] Updated weights for policy 0, policy_version 128630 (0.0009) [2023-12-27 06:32:51,805][20347] Updated weights for policy 0, policy_version 128640 (0.0008) [2023-12-27 06:32:52,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 65798144. Throughput: 0: 10271.7, 1: 10183.5. Samples: 65772900. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '68.120')] [2023-12-27 06:32:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000128648_32948224.pth... [2023-12-27 06:32:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000127464_32645120.pth [2023-12-27 06:32:52,378][20381] Updated weights for policy 1, policy_version 128291 (0.0010) [2023-12-27 06:32:52,431][20381] Updated weights for policy 1, policy_version 128301 (0.0011) [2023-12-27 06:32:52,479][20347] Updated weights for policy 0, policy_version 128650 (0.0008) [2023-12-27 06:32:52,489][20381] Updated weights for policy 1, policy_version 128311 (0.0011) [2023-12-27 06:32:52,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000128320_32858112.pth... [2023-12-27 06:32:52,542][20347] Updated weights for policy 0, policy_version 128660 (0.0006) [2023-12-27 06:32:52,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000127104_32546816.pth [2023-12-27 06:32:52,601][20347] Updated weights for policy 0, policy_version 128670 (0.0007) [2023-12-27 06:32:52,660][20347] Updated weights for policy 0, policy_version 128680 (0.0007) [2023-12-27 06:32:53,210][20381] Updated weights for policy 1, policy_version 128321 (0.0010) [2023-12-27 06:32:53,272][20381] Updated weights for policy 1, policy_version 128331 (0.0011) [2023-12-27 06:32:53,327][20381] Updated weights for policy 1, policy_version 128341 (0.0010) [2023-12-27 06:32:53,343][20347] Updated weights for policy 0, policy_version 128690 (0.0006) [2023-12-27 06:32:53,382][20381] Updated weights for policy 1, policy_version 128351 (0.0010) [2023-12-27 06:32:53,399][20347] Updated weights for policy 0, policy_version 128700 (0.0006) [2023-12-27 06:32:53,448][20347] Updated weights for policy 0, policy_version 128710 (0.0007) [2023-12-27 06:32:54,067][20381] Updated weights for policy 1, policy_version 128361 (0.0011) [2023-12-27 06:32:54,105][20347] Updated weights for policy 0, policy_version 128720 (0.0007) [2023-12-27 06:32:54,113][20381] Updated weights for policy 1, policy_version 128371 (0.0011) [2023-12-27 06:32:54,163][20347] Updated weights for policy 0, policy_version 128730 (0.0005) [2023-12-27 06:32:54,165][20381] Updated weights for policy 1, policy_version 128381 (0.0010) [2023-12-27 06:32:54,213][20347] Updated weights for policy 0, policy_version 128740 (0.0006) [2023-12-27 06:32:54,816][20381] Updated weights for policy 1, policy_version 128391 (0.0007) [2023-12-27 06:32:54,871][20381] Updated weights for policy 1, policy_version 128401 (0.0008) [2023-12-27 06:32:54,920][20347] Updated weights for policy 0, policy_version 128750 (0.0006) [2023-12-27 06:32:54,930][20381] Updated weights for policy 1, policy_version 128411 (0.0007) [2023-12-27 06:32:54,979][20347] Updated weights for policy 0, policy_version 128760 (0.0007) [2023-12-27 06:32:55,036][20347] Updated weights for policy 0, policy_version 128770 (0.0008) [2023-12-27 06:32:55,487][20381] Updated weights for policy 1, policy_version 128421 (0.0008) [2023-12-27 06:32:55,543][20381] Updated weights for policy 1, policy_version 128431 (0.0008) [2023-12-27 06:32:55,591][20381] Updated weights for policy 1, policy_version 128441 (0.0008) [2023-12-27 06:32:55,691][20347] Updated weights for policy 0, policy_version 128780 (0.0009) [2023-12-27 06:32:55,746][20347] Updated weights for policy 0, policy_version 128790 (0.0010) [2023-12-27 06:32:55,797][20347] Updated weights for policy 0, policy_version 128800 (0.0009) [2023-12-27 06:32:56,226][20381] Updated weights for policy 1, policy_version 128451 (0.0008) [2023-12-27 06:32:56,282][20381] Updated weights for policy 1, policy_version 128461 (0.0007) [2023-12-27 06:32:56,331][20381] Updated weights for policy 1, policy_version 128471 (0.0007) [2023-12-27 06:32:56,494][20347] Updated weights for policy 0, policy_version 128810 (0.0009) [2023-12-27 06:32:56,538][20347] Updated weights for policy 0, policy_version 128820 (0.0010) [2023-12-27 06:32:56,595][20347] Updated weights for policy 0, policy_version 128830 (0.0010) [2023-12-27 06:32:56,650][20347] Updated weights for policy 0, policy_version 128840 (0.0010) [2023-12-27 06:32:57,022][20381] Updated weights for policy 1, policy_version 128481 (0.0007) [2023-12-27 06:32:57,081][20381] Updated weights for policy 1, policy_version 128491 (0.0007) [2023-12-27 06:32:57,130][20381] Updated weights for policy 1, policy_version 128501 (0.0006) [2023-12-27 06:32:57,177][20381] Updated weights for policy 1, policy_version 128511 (0.0008) [2023-12-27 06:32:57,297][20347] Updated weights for policy 0, policy_version 128850 (0.0008) [2023-12-27 06:32:57,358][20347] Updated weights for policy 0, policy_version 128860 (0.0008) [2023-12-27 06:32:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 65904640. Throughput: 0: 10254.6, 1: 10264.8. Samples: 65896040. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:32:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '78.760')] [2023-12-27 06:32:57,425][20347] Updated weights for policy 0, policy_version 128870 (0.0008) [2023-12-27 06:32:57,863][20381] Updated weights for policy 1, policy_version 128521 (0.0007) [2023-12-27 06:32:57,918][20381] Updated weights for policy 1, policy_version 128531 (0.0008) [2023-12-27 06:32:57,972][20381] Updated weights for policy 1, policy_version 128541 (0.0007) [2023-12-27 06:32:57,999][20347] Updated weights for policy 0, policy_version 128880 (0.0006) [2023-12-27 06:32:58,057][20347] Updated weights for policy 0, policy_version 128890 (0.0005) [2023-12-27 06:32:58,112][20347] Updated weights for policy 0, policy_version 128900 (0.0005) [2023-12-27 06:32:58,678][20381] Updated weights for policy 1, policy_version 128551 (0.0007) [2023-12-27 06:32:58,744][20381] Updated weights for policy 1, policy_version 128561 (0.0008) [2023-12-27 06:32:58,807][20381] Updated weights for policy 1, policy_version 128571 (0.0008) [2023-12-27 06:32:58,811][20347] Updated weights for policy 0, policy_version 128910 (0.0007) [2023-12-27 06:32:58,879][20347] Updated weights for policy 0, policy_version 128920 (0.0008) [2023-12-27 06:32:58,948][20347] Updated weights for policy 0, policy_version 128930 (0.0009) [2023-12-27 06:32:59,554][20381] Updated weights for policy 1, policy_version 128581 (0.0008) [2023-12-27 06:32:59,620][20381] Updated weights for policy 1, policy_version 128591 (0.0007) [2023-12-27 06:32:59,676][20381] Updated weights for policy 1, policy_version 128601 (0.0007) [2023-12-27 06:32:59,724][20347] Updated weights for policy 0, policy_version 128940 (0.0007) [2023-12-27 06:32:59,769][20347] Updated weights for policy 0, policy_version 128950 (0.0005) [2023-12-27 06:32:59,824][20347] Updated weights for policy 0, policy_version 128960 (0.0005) [2023-12-27 06:33:00,285][20381] Updated weights for policy 1, policy_version 128611 (0.0007) [2023-12-27 06:33:00,341][20381] Updated weights for policy 1, policy_version 128621 (0.0005) [2023-12-27 06:33:00,399][20381] Updated weights for policy 1, policy_version 128631 (0.0005) [2023-12-27 06:33:00,455][20347] Updated weights for policy 0, policy_version 128970 (0.0006) [2023-12-27 06:33:00,513][20347] Updated weights for policy 0, policy_version 128980 (0.0005) [2023-12-27 06:33:00,568][20347] Updated weights for policy 0, policy_version 128990 (0.0008) [2023-12-27 06:33:00,619][20347] Updated weights for policy 0, policy_version 129000 (0.0007) [2023-12-27 06:33:00,948][20381] Updated weights for policy 1, policy_version 128641 (0.0006) [2023-12-27 06:33:01,013][20381] Updated weights for policy 1, policy_version 128651 (0.0006) [2023-12-27 06:33:01,068][20381] Updated weights for policy 1, policy_version 128661 (0.0007) [2023-12-27 06:33:01,120][20381] Updated weights for policy 1, policy_version 128671 (0.0008) [2023-12-27 06:33:01,307][20347] Updated weights for policy 0, policy_version 129010 (0.0008) [2023-12-27 06:33:01,368][20347] Updated weights for policy 0, policy_version 129020 (0.0008) [2023-12-27 06:33:01,425][20347] Updated weights for policy 0, policy_version 129030 (0.0007) [2023-12-27 06:33:01,751][20381] Updated weights for policy 1, policy_version 128681 (0.0007) [2023-12-27 06:33:01,804][20381] Updated weights for policy 1, policy_version 128691 (0.0005) [2023-12-27 06:33:01,850][20381] Updated weights for policy 1, policy_version 128701 (0.0005) [2023-12-27 06:33:02,205][20347] Updated weights for policy 0, policy_version 129040 (0.0008) [2023-12-27 06:33:02,250][20347] Updated weights for policy 0, policy_version 129050 (0.0007) [2023-12-27 06:33:02,310][20347] Updated weights for policy 0, policy_version 129060 (0.0008) [2023-12-27 06:33:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 66011136. Throughput: 0: 10257.7, 1: 10260.8. Samples: 66019948. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:33:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '101.560')] [2023-12-27 06:33:02,479][20381] Updated weights for policy 1, policy_version 128711 (0.0005) [2023-12-27 06:33:02,536][20381] Updated weights for policy 1, policy_version 128721 (0.0008) [2023-12-27 06:33:02,581][20381] Updated weights for policy 1, policy_version 128731 (0.0007) [2023-12-27 06:33:02,989][20347] Updated weights for policy 0, policy_version 129070 (0.0006) [2023-12-27 06:33:03,043][20347] Updated weights for policy 0, policy_version 129080 (0.0006) [2023-12-27 06:33:03,100][20347] Updated weights for policy 0, policy_version 129090 (0.0005) [2023-12-27 06:33:03,166][20381] Updated weights for policy 1, policy_version 128741 (0.0005) [2023-12-27 06:33:03,221][20381] Updated weights for policy 1, policy_version 128751 (0.0007) [2023-12-27 06:33:03,278][20381] Updated weights for policy 1, policy_version 128761 (0.0005) [2023-12-27 06:33:03,747][20347] Updated weights for policy 0, policy_version 129100 (0.0010) [2023-12-27 06:33:03,783][20381] Updated weights for policy 1, policy_version 128771 (0.0006) [2023-12-27 06:33:03,800][20347] Updated weights for policy 0, policy_version 129110 (0.0010) [2023-12-27 06:33:03,831][20381] Updated weights for policy 1, policy_version 128781 (0.0005) [2023-12-27 06:33:03,854][20347] Updated weights for policy 0, policy_version 129120 (0.0010) [2023-12-27 06:33:03,891][20381] Updated weights for policy 1, policy_version 128791 (0.0008) [2023-12-27 06:33:04,522][20347] Updated weights for policy 0, policy_version 129130 (0.0010) [2023-12-27 06:33:04,571][20347] Updated weights for policy 0, policy_version 129140 (0.0010) [2023-12-27 06:33:04,577][20381] Updated weights for policy 1, policy_version 128801 (0.0007) [2023-12-27 06:33:04,623][20347] Updated weights for policy 0, policy_version 129150 (0.0010) [2023-12-27 06:33:04,632][20381] Updated weights for policy 1, policy_version 128811 (0.0005) [2023-12-27 06:33:04,676][20347] Updated weights for policy 0, policy_version 129160 (0.0007) [2023-12-27 06:33:04,691][20381] Updated weights for policy 1, policy_version 128821 (0.0006) [2023-12-27 06:33:04,743][20381] Updated weights for policy 1, policy_version 128831 (0.0008) [2023-12-27 06:33:05,272][20347] Updated weights for policy 0, policy_version 129170 (0.0005) [2023-12-27 06:33:05,320][20347] Updated weights for policy 0, policy_version 129180 (0.0008) [2023-12-27 06:33:05,367][20347] Updated weights for policy 0, policy_version 129190 (0.0005) [2023-12-27 06:33:05,378][20381] Updated weights for policy 1, policy_version 128841 (0.0008) [2023-12-27 06:33:05,429][20381] Updated weights for policy 1, policy_version 128851 (0.0008) [2023-12-27 06:33:05,486][20381] Updated weights for policy 1, policy_version 128861 (0.0005) [2023-12-27 06:33:05,960][20347] Updated weights for policy 0, policy_version 129200 (0.0009) [2023-12-27 06:33:06,017][20347] Updated weights for policy 0, policy_version 129210 (0.0010) [2023-12-27 06:33:06,077][20347] Updated weights for policy 0, policy_version 129220 (0.0005) [2023-12-27 06:33:06,077][20381] Updated weights for policy 1, policy_version 128871 (0.0005) [2023-12-27 06:33:06,140][20381] Updated weights for policy 1, policy_version 128881 (0.0008) [2023-12-27 06:33:06,206][20381] Updated weights for policy 1, policy_version 128891 (0.0008) [2023-12-27 06:33:06,751][20347] Updated weights for policy 0, policy_version 129230 (0.0010) [2023-12-27 06:33:06,812][20347] Updated weights for policy 0, policy_version 129240 (0.0011) [2023-12-27 06:33:06,880][20347] Updated weights for policy 0, policy_version 129250 (0.0011) [2023-12-27 06:33:06,941][20381] Updated weights for policy 1, policy_version 128901 (0.0009) [2023-12-27 06:33:06,994][20381] Updated weights for policy 1, policy_version 128911 (0.0010) [2023-12-27 06:33:07,047][20381] Updated weights for policy 1, policy_version 128921 (0.0011) [2023-12-27 06:33:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.4, 300 sec: 20632.7). Total num frames: 66117632. Throughput: 0: 10269.7, 1: 10332.8. Samples: 66085824. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:33:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '117.500')] [2023-12-27 06:33:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000128928_33013760.pth... [2023-12-27 06:33:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000129256_33103872.pth... [2023-12-27 06:33:07,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000128072_32800768.pth [2023-12-27 06:33:07,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000127712_32702464.pth [2023-12-27 06:33:07,562][20347] Updated weights for policy 0, policy_version 129260 (0.0008) [2023-12-27 06:33:07,620][20347] Updated weights for policy 0, policy_version 129270 (0.0005) [2023-12-27 06:33:07,676][20347] Updated weights for policy 0, policy_version 129280 (0.0005) [2023-12-27 06:33:07,739][20381] Updated weights for policy 1, policy_version 128931 (0.0009) [2023-12-27 06:33:07,797][20381] Updated weights for policy 1, policy_version 128941 (0.0010) [2023-12-27 06:33:07,853][20381] Updated weights for policy 1, policy_version 128951 (0.0010) [2023-12-27 06:33:08,318][20347] Updated weights for policy 0, policy_version 129290 (0.0009) [2023-12-27 06:33:08,384][20347] Updated weights for policy 0, policy_version 129300 (0.0010) [2023-12-27 06:33:08,446][20347] Updated weights for policy 0, policy_version 129310 (0.0006) [2023-12-27 06:33:08,505][20381] Updated weights for policy 1, policy_version 128961 (0.0010) [2023-12-27 06:33:08,509][20347] Updated weights for policy 0, policy_version 129320 (0.0007) [2023-12-27 06:33:08,571][20381] Updated weights for policy 1, policy_version 128971 (0.0007) [2023-12-27 06:33:08,638][20381] Updated weights for policy 1, policy_version 128981 (0.0008) [2023-12-27 06:33:08,693][20381] Updated weights for policy 1, policy_version 128991 (0.0010) [2023-12-27 06:33:09,121][20347] Updated weights for policy 0, policy_version 129330 (0.0010) [2023-12-27 06:33:09,173][20347] Updated weights for policy 0, policy_version 129340 (0.0010) [2023-12-27 06:33:09,228][20347] Updated weights for policy 0, policy_version 129350 (0.0010) [2023-12-27 06:33:09,337][20381] Updated weights for policy 1, policy_version 129001 (0.0008) [2023-12-27 06:33:09,396][20381] Updated weights for policy 1, policy_version 129011 (0.0011) [2023-12-27 06:33:09,457][20381] Updated weights for policy 1, policy_version 129021 (0.0008) [2023-12-27 06:33:09,967][20347] Updated weights for policy 0, policy_version 129360 (0.0011) [2023-12-27 06:33:10,027][20347] Updated weights for policy 0, policy_version 129370 (0.0011) [2023-12-27 06:33:10,081][20347] Updated weights for policy 0, policy_version 129380 (0.0011) [2023-12-27 06:33:10,143][20381] Updated weights for policy 1, policy_version 129031 (0.0006) [2023-12-27 06:33:10,199][20381] Updated weights for policy 1, policy_version 129041 (0.0008) [2023-12-27 06:33:10,250][20381] Updated weights for policy 1, policy_version 129051 (0.0007) [2023-12-27 06:33:10,810][20347] Updated weights for policy 0, policy_version 129390 (0.0009) [2023-12-27 06:33:10,867][20347] Updated weights for policy 0, policy_version 129400 (0.0007) [2023-12-27 06:33:10,867][20381] Updated weights for policy 1, policy_version 129061 (0.0007) [2023-12-27 06:33:10,923][20381] Updated weights for policy 1, policy_version 129071 (0.0005) [2023-12-27 06:33:10,931][20347] Updated weights for policy 0, policy_version 129410 (0.0008) [2023-12-27 06:33:10,973][20381] Updated weights for policy 1, policy_version 129081 (0.0005) [2023-12-27 06:33:11,627][20347] Updated weights for policy 0, policy_version 129420 (0.0008) [2023-12-27 06:33:11,667][20381] Updated weights for policy 1, policy_version 129091 (0.0006) [2023-12-27 06:33:11,691][20347] Updated weights for policy 0, policy_version 129430 (0.0008) [2023-12-27 06:33:11,733][20381] Updated weights for policy 1, policy_version 129101 (0.0007) [2023-12-27 06:33:11,754][20347] Updated weights for policy 0, policy_version 129440 (0.0008) [2023-12-27 06:33:11,803][20381] Updated weights for policy 1, policy_version 129111 (0.0007) [2023-12-27 06:33:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 66215936. Throughput: 0: 10278.1, 1: 10335.1. Samples: 66210316. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:33:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '138.010')] [2023-12-27 06:33:12,411][20347] Updated weights for policy 0, policy_version 129450 (0.0008) [2023-12-27 06:33:12,472][20347] Updated weights for policy 0, policy_version 129460 (0.0009) [2023-12-27 06:33:12,489][20381] Updated weights for policy 1, policy_version 129121 (0.0008) [2023-12-27 06:33:12,529][20347] Updated weights for policy 0, policy_version 129470 (0.0010) [2023-12-27 06:33:12,557][20381] Updated weights for policy 1, policy_version 129131 (0.0008) [2023-12-27 06:33:12,588][20347] Updated weights for policy 0, policy_version 129480 (0.0010) [2023-12-27 06:33:12,620][20381] Updated weights for policy 1, policy_version 129141 (0.0007) [2023-12-27 06:33:12,672][20381] Updated weights for policy 1, policy_version 129151 (0.0009) [2023-12-27 06:33:13,265][20381] Updated weights for policy 1, policy_version 129161 (0.0008) [2023-12-27 06:33:13,314][20347] Updated weights for policy 0, policy_version 129490 (0.0007) [2023-12-27 06:33:13,324][20381] Updated weights for policy 1, policy_version 129171 (0.0007) [2023-12-27 06:33:13,364][20347] Updated weights for policy 0, policy_version 129500 (0.0006) [2023-12-27 06:33:13,391][20381] Updated weights for policy 1, policy_version 129181 (0.0007) [2023-12-27 06:33:13,412][20347] Updated weights for policy 0, policy_version 129510 (0.0007) [2023-12-27 06:33:14,008][20381] Updated weights for policy 1, policy_version 129191 (0.0009) [2023-12-27 06:33:14,067][20381] Updated weights for policy 1, policy_version 129201 (0.0011) [2023-12-27 06:33:14,099][20347] Updated weights for policy 0, policy_version 129520 (0.0010) [2023-12-27 06:33:14,126][20381] Updated weights for policy 1, policy_version 129211 (0.0008) [2023-12-27 06:33:14,162][20347] Updated weights for policy 0, policy_version 129530 (0.0010) [2023-12-27 06:33:14,224][20347] Updated weights for policy 0, policy_version 129540 (0.0010) [2023-12-27 06:33:14,773][20381] Updated weights for policy 1, policy_version 129221 (0.0009) [2023-12-27 06:33:14,836][20381] Updated weights for policy 1, policy_version 129231 (0.0011) [2023-12-27 06:33:14,843][20347] Updated weights for policy 0, policy_version 129550 (0.0008) [2023-12-27 06:33:14,896][20381] Updated weights for policy 1, policy_version 129241 (0.0011) [2023-12-27 06:33:14,901][20347] Updated weights for policy 0, policy_version 129560 (0.0006) [2023-12-27 06:33:14,964][20347] Updated weights for policy 0, policy_version 129570 (0.0008) [2023-12-27 06:33:15,510][20347] Updated weights for policy 0, policy_version 129580 (0.0007) [2023-12-27 06:33:15,557][20381] Updated weights for policy 1, policy_version 129251 (0.0007) [2023-12-27 06:33:15,562][20347] Updated weights for policy 0, policy_version 129590 (0.0010) [2023-12-27 06:33:15,605][20381] Updated weights for policy 1, policy_version 129261 (0.0010) [2023-12-27 06:33:15,613][20347] Updated weights for policy 0, policy_version 129600 (0.0009) [2023-12-27 06:33:15,654][20381] Updated weights for policy 1, policy_version 129271 (0.0010) [2023-12-27 06:33:16,243][20347] Updated weights for policy 0, policy_version 129610 (0.0005) [2023-12-27 06:33:16,302][20347] Updated weights for policy 0, policy_version 129620 (0.0006) [2023-12-27 06:33:16,360][20347] Updated weights for policy 0, policy_version 129630 (0.0005) [2023-12-27 06:33:16,360][20381] Updated weights for policy 1, policy_version 129281 (0.0010) [2023-12-27 06:33:16,409][20381] Updated weights for policy 1, policy_version 129291 (0.0010) [2023-12-27 06:33:16,418][20347] Updated weights for policy 0, policy_version 129640 (0.0005) [2023-12-27 06:33:16,456][20381] Updated weights for policy 1, policy_version 129301 (0.0009) [2023-12-27 06:33:16,507][20381] Updated weights for policy 1, policy_version 129311 (0.0010) [2023-12-27 06:33:17,024][20347] Updated weights for policy 0, policy_version 129650 (0.0007) [2023-12-27 06:33:17,077][20347] Updated weights for policy 0, policy_version 129660 (0.0005) [2023-12-27 06:33:17,131][20347] Updated weights for policy 0, policy_version 129670 (0.0005) [2023-12-27 06:33:17,201][20381] Updated weights for policy 1, policy_version 129321 (0.0010) [2023-12-27 06:33:17,247][20381] Updated weights for policy 1, policy_version 129331 (0.0010) [2023-12-27 06:33:17,294][20381] Updated weights for policy 1, policy_version 129341 (0.0010) [2023-12-27 06:33:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 66330624. Throughput: 0: 10285.8, 1: 10371.4. Samples: 66336148. Policy #0 lag: (min: 4.0, avg: 13.1, max: 36.0) [2023-12-27 06:33:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '197.360')] [2023-12-27 06:33:17,721][20347] Updated weights for policy 0, policy_version 129680 (0.0005) [2023-12-27 06:33:17,770][20347] Updated weights for policy 0, policy_version 129690 (0.0005) [2023-12-27 06:33:17,816][20347] Updated weights for policy 0, policy_version 129700 (0.0005) [2023-12-27 06:33:18,018][20381] Updated weights for policy 1, policy_version 129351 (0.0011) [2023-12-27 06:33:18,075][20381] Updated weights for policy 1, policy_version 129361 (0.0010) [2023-12-27 06:33:18,133][20381] Updated weights for policy 1, policy_version 129371 (0.0010) [2023-12-27 06:33:18,485][20347] Updated weights for policy 0, policy_version 129710 (0.0009) [2023-12-27 06:33:18,540][20347] Updated weights for policy 0, policy_version 129720 (0.0008) [2023-12-27 06:33:18,595][20347] Updated weights for policy 0, policy_version 129730 (0.0008) [2023-12-27 06:33:18,837][20381] Updated weights for policy 1, policy_version 129381 (0.0011) [2023-12-27 06:33:18,895][20381] Updated weights for policy 1, policy_version 129391 (0.0011) [2023-12-27 06:33:18,957][20381] Updated weights for policy 1, policy_version 129401 (0.0010) [2023-12-27 06:33:19,292][20347] Updated weights for policy 0, policy_version 129740 (0.0007) [2023-12-27 06:33:19,356][20347] Updated weights for policy 0, policy_version 129750 (0.0011) [2023-12-27 06:33:19,417][20347] Updated weights for policy 0, policy_version 129760 (0.0010) [2023-12-27 06:33:19,627][20381] Updated weights for policy 1, policy_version 129411 (0.0010) [2023-12-27 06:33:19,691][20381] Updated weights for policy 1, policy_version 129421 (0.0010) [2023-12-27 06:33:19,749][20381] Updated weights for policy 1, policy_version 129431 (0.0011) [2023-12-27 06:33:20,105][20347] Updated weights for policy 0, policy_version 129770 (0.0010) [2023-12-27 06:33:20,166][20347] Updated weights for policy 0, policy_version 129780 (0.0010) [2023-12-27 06:33:20,228][20347] Updated weights for policy 0, policy_version 129790 (0.0011) [2023-12-27 06:33:20,288][20347] Updated weights for policy 0, policy_version 129800 (0.0008) [2023-12-27 06:33:20,448][20381] Updated weights for policy 1, policy_version 129441 (0.0011) [2023-12-27 06:33:20,509][20381] Updated weights for policy 1, policy_version 129451 (0.0011) [2023-12-27 06:33:20,571][20381] Updated weights for policy 1, policy_version 129461 (0.0011) [2023-12-27 06:33:20,635][20381] Updated weights for policy 1, policy_version 129471 (0.0006) [2023-12-27 06:33:20,983][20347] Updated weights for policy 0, policy_version 129810 (0.0007) [2023-12-27 06:33:21,044][20347] Updated weights for policy 0, policy_version 129820 (0.0008) [2023-12-27 06:33:21,109][20347] Updated weights for policy 0, policy_version 129830 (0.0007) [2023-12-27 06:33:21,397][20381] Updated weights for policy 1, policy_version 129482 (0.0013) [2023-12-27 06:33:21,461][20381] Updated weights for policy 1, policy_version 129492 (0.0008) [2023-12-27 06:33:21,520][20381] Updated weights for policy 1, policy_version 129502 (0.0008) [2023-12-27 06:33:21,870][20347] Updated weights for policy 0, policy_version 129840 (0.0007) [2023-12-27 06:33:21,926][20347] Updated weights for policy 0, policy_version 129850 (0.0007) [2023-12-27 06:33:21,985][20347] Updated weights for policy 0, policy_version 129860 (0.0008) [2023-12-27 06:33:22,251][20381] Updated weights for policy 1, policy_version 129512 (0.0008) [2023-12-27 06:33:22,318][20381] Updated weights for policy 1, policy_version 129522 (0.0008) [2023-12-27 06:33:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 66420736. Throughput: 0: 10238.8, 1: 10374.9. Samples: 66397260. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '291.240')] [2023-12-27 06:33:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000129864_33259520.pth... [2023-12-27 06:33:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000128648_32948224.pth [2023-12-27 06:33:22,379][20381] Updated weights for policy 1, policy_version 129532 (0.0007) [2023-12-27 06:33:22,399][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000129536_33169408.pth... [2023-12-27 06:33:22,402][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000128320_32858112.pth [2023-12-27 06:33:22,714][20347] Updated weights for policy 0, policy_version 129870 (0.0008) [2023-12-27 06:33:22,770][20347] Updated weights for policy 0, policy_version 129880 (0.0007) [2023-12-27 06:33:22,830][20347] Updated weights for policy 0, policy_version 129890 (0.0008) [2023-12-27 06:33:23,110][20381] Updated weights for policy 1, policy_version 129542 (0.0008) [2023-12-27 06:33:23,171][20381] Updated weights for policy 1, policy_version 129552 (0.0007) [2023-12-27 06:33:23,230][20381] Updated weights for policy 1, policy_version 129562 (0.0007) [2023-12-27 06:33:23,541][20347] Updated weights for policy 0, policy_version 129900 (0.0008) [2023-12-27 06:33:23,594][20347] Updated weights for policy 0, policy_version 129910 (0.0008) [2023-12-27 06:33:23,649][20347] Updated weights for policy 0, policy_version 129920 (0.0008) [2023-12-27 06:33:23,871][20381] Updated weights for policy 1, policy_version 129572 (0.0007) [2023-12-27 06:33:23,921][20381] Updated weights for policy 1, policy_version 129582 (0.0008) [2023-12-27 06:33:23,985][20381] Updated weights for policy 1, policy_version 129592 (0.0008) [2023-12-27 06:33:24,347][20347] Updated weights for policy 0, policy_version 129930 (0.0008) [2023-12-27 06:33:24,391][20347] Updated weights for policy 0, policy_version 129940 (0.0008) [2023-12-27 06:33:24,444][20347] Updated weights for policy 0, policy_version 129950 (0.0006) [2023-12-27 06:33:24,499][20347] Updated weights for policy 0, policy_version 129960 (0.0008) [2023-12-27 06:33:24,656][20381] Updated weights for policy 1, policy_version 129602 (0.0008) [2023-12-27 06:33:24,709][20381] Updated weights for policy 1, policy_version 129612 (0.0009) [2023-12-27 06:33:24,770][20381] Updated weights for policy 1, policy_version 129622 (0.0010) [2023-12-27 06:33:24,828][20381] Updated weights for policy 1, policy_version 129632 (0.0010) [2023-12-27 06:33:25,123][20347] Updated weights for policy 0, policy_version 129970 (0.0008) [2023-12-27 06:33:25,176][20347] Updated weights for policy 0, policy_version 129980 (0.0006) [2023-12-27 06:33:25,230][20347] Updated weights for policy 0, policy_version 129990 (0.0005) [2023-12-27 06:33:25,429][20381] Updated weights for policy 1, policy_version 129642 (0.0009) [2023-12-27 06:33:25,478][20381] Updated weights for policy 1, policy_version 129652 (0.0005) [2023-12-27 06:33:25,534][20381] Updated weights for policy 1, policy_version 129662 (0.0006) [2023-12-27 06:33:25,834][20347] Updated weights for policy 0, policy_version 130000 (0.0009) [2023-12-27 06:33:25,891][20347] Updated weights for policy 0, policy_version 130010 (0.0010) [2023-12-27 06:33:25,938][20347] Updated weights for policy 0, policy_version 130020 (0.0010) [2023-12-27 06:33:26,090][20381] Updated weights for policy 1, policy_version 129672 (0.0008) [2023-12-27 06:33:26,155][20381] Updated weights for policy 1, policy_version 129682 (0.0008) [2023-12-27 06:33:26,216][20381] Updated weights for policy 1, policy_version 129692 (0.0009) [2023-12-27 06:33:26,627][20347] Updated weights for policy 0, policy_version 130030 (0.0007) [2023-12-27 06:33:26,676][20347] Updated weights for policy 0, policy_version 130040 (0.0005) [2023-12-27 06:33:26,729][20347] Updated weights for policy 0, policy_version 130050 (0.0007) [2023-12-27 06:33:26,822][20381] Updated weights for policy 1, policy_version 129702 (0.0010) [2023-12-27 06:33:26,877][20381] Updated weights for policy 1, policy_version 129712 (0.0010) [2023-12-27 06:33:26,925][20381] Updated weights for policy 1, policy_version 129722 (0.0010) [2023-12-27 06:33:27,298][20347] Updated weights for policy 0, policy_version 130060 (0.0007) [2023-12-27 06:33:27,355][20347] Updated weights for policy 0, policy_version 130070 (0.0007) [2023-12-27 06:33:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 66527232. Throughput: 0: 10244.4, 1: 10377.6. Samples: 66519500. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '518.820')] [2023-12-27 06:33:27,418][20347] Updated weights for policy 0, policy_version 130080 (0.0008) [2023-12-27 06:33:27,619][20381] Updated weights for policy 1, policy_version 129732 (0.0010) [2023-12-27 06:33:27,671][20381] Updated weights for policy 1, policy_version 129742 (0.0009) [2023-12-27 06:33:27,727][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-27 06:33:27,728][20381] Updated weights for policy 1, policy_version 129752 (0.0006) [2023-12-27 06:33:28,051][20347] Updated weights for policy 0, policy_version 130090 (0.0007) [2023-12-27 06:33:28,106][20347] Updated weights for policy 0, policy_version 130100 (0.0005) [2023-12-27 06:33:28,150][20347] Updated weights for policy 0, policy_version 130110 (0.0006) [2023-12-27 06:33:28,199][20347] Updated weights for policy 0, policy_version 130120 (0.0006) [2023-12-27 06:33:28,408][20381] Updated weights for policy 1, policy_version 129762 (0.0006) [2023-12-27 06:33:28,470][20381] Updated weights for policy 1, policy_version 129772 (0.0006) [2023-12-27 06:33:28,518][20381] Updated weights for policy 1, policy_version 129782 (0.0006) [2023-12-27 06:33:28,898][20347] Updated weights for policy 0, policy_version 130130 (0.0007) [2023-12-27 06:33:28,958][20347] Updated weights for policy 0, policy_version 130140 (0.0006) [2023-12-27 06:33:29,019][20347] Updated weights for policy 0, policy_version 130150 (0.0010) [2023-12-27 06:33:29,181][20381] Updated weights for policy 1, policy_version 129792 (0.0009) [2023-12-27 06:33:29,235][20381] Updated weights for policy 1, policy_version 129802 (0.0007) [2023-12-27 06:33:29,292][20381] Updated weights for policy 1, policy_version 129812 (0.0007) [2023-12-27 06:33:29,663][20347] Updated weights for policy 0, policy_version 130160 (0.0006) [2023-12-27 06:33:29,717][20347] Updated weights for policy 0, policy_version 130170 (0.0005) [2023-12-27 06:33:29,777][20347] Updated weights for policy 0, policy_version 130180 (0.0010) [2023-12-27 06:33:29,987][20381] Updated weights for policy 1, policy_version 129822 (0.0008) [2023-12-27 06:33:30,053][20381] Updated weights for policy 1, policy_version 129832 (0.0008) [2023-12-27 06:33:30,120][20381] Updated weights for policy 1, policy_version 129842 (0.0008) [2023-12-27 06:33:30,455][20347] Updated weights for policy 0, policy_version 130190 (0.0009) [2023-12-27 06:33:30,506][20347] Updated weights for policy 0, policy_version 130200 (0.0007) [2023-12-27 06:33:30,558][20347] Updated weights for policy 0, policy_version 130210 (0.0007) [2023-12-27 06:33:30,729][20381] Updated weights for policy 1, policy_version 129852 (0.0007) [2023-12-27 06:33:30,787][20381] Updated weights for policy 1, policy_version 129862 (0.0007) [2023-12-27 06:33:30,838][20381] Updated weights for policy 1, policy_version 129872 (0.0007) [2023-12-27 06:33:31,303][20347] Updated weights for policy 0, policy_version 130220 (0.0007) [2023-12-27 06:33:31,364][20347] Updated weights for policy 0, policy_version 130230 (0.0008) [2023-12-27 06:33:31,423][20347] Updated weights for policy 0, policy_version 130240 (0.0007) [2023-12-27 06:33:31,481][20381] Updated weights for policy 1, policy_version 129882 (0.0007) [2023-12-27 06:33:31,533][20381] Updated weights for policy 1, policy_version 129892 (0.0006) [2023-12-27 06:33:31,581][20381] Updated weights for policy 1, policy_version 129902 (0.0007) [2023-12-27 06:33:31,637][20381] Updated weights for policy 1, policy_version 129912 (0.0008) [2023-12-27 06:33:32,156][20347] Updated weights for policy 0, policy_version 130250 (0.0007) [2023-12-27 06:33:32,210][20347] Updated weights for policy 0, policy_version 130260 (0.0007) [2023-12-27 06:33:32,243][20381] Updated weights for policy 1, policy_version 129922 (0.0008) [2023-12-27 06:33:32,269][20347] Updated weights for policy 0, policy_version 130270 (0.0007) [2023-12-27 06:33:32,300][20381] Updated weights for policy 1, policy_version 129932 (0.0008) [2023-12-27 06:33:32,332][20347] Updated weights for policy 0, policy_version 130280 (0.0008) [2023-12-27 06:33:32,355][20381] Updated weights for policy 1, policy_version 129942 (0.0006) [2023-12-27 06:33:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 66633728. Throughput: 0: 10300.7, 1: 10434.1. Samples: 66645576. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '806.090')] [2023-12-27 06:33:33,012][20381] Updated weights for policy 1, policy_version 129952 (0.0007) [2023-12-27 06:33:33,043][20347] Updated weights for policy 0, policy_version 130290 (0.0005) [2023-12-27 06:33:33,068][20381] Updated weights for policy 1, policy_version 129962 (0.0005) [2023-12-27 06:33:33,099][20347] Updated weights for policy 0, policy_version 130300 (0.0005) [2023-12-27 06:33:33,123][20381] Updated weights for policy 1, policy_version 129972 (0.0005) [2023-12-27 06:33:33,163][20347] Updated weights for policy 0, policy_version 130310 (0.0005) [2023-12-27 06:33:33,672][20381] Updated weights for policy 1, policy_version 129982 (0.0006) [2023-12-27 06:33:33,716][20381] Updated weights for policy 1, policy_version 129992 (0.0007) [2023-12-27 06:33:33,772][20381] Updated weights for policy 1, policy_version 130002 (0.0007) [2023-12-27 06:33:33,787][20347] Updated weights for policy 0, policy_version 130320 (0.0007) [2023-12-27 06:33:33,840][20347] Updated weights for policy 0, policy_version 130330 (0.0007) [2023-12-27 06:33:33,892][20347] Updated weights for policy 0, policy_version 130340 (0.0007) [2023-12-27 06:33:34,431][20381] Updated weights for policy 1, policy_version 130012 (0.0006) [2023-12-27 06:33:34,498][20381] Updated weights for policy 1, policy_version 130022 (0.0008) [2023-12-27 06:33:34,558][20381] Updated weights for policy 1, policy_version 130032 (0.0008) [2023-12-27 06:33:34,602][20347] Updated weights for policy 0, policy_version 130350 (0.0006) [2023-12-27 06:33:34,661][20347] Updated weights for policy 0, policy_version 130360 (0.0008) [2023-12-27 06:33:34,720][20347] Updated weights for policy 0, policy_version 130370 (0.0007) [2023-12-27 06:33:35,246][20381] Updated weights for policy 1, policy_version 130042 (0.0007) [2023-12-27 06:33:35,297][20381] Updated weights for policy 1, policy_version 130052 (0.0007) [2023-12-27 06:33:35,348][20381] Updated weights for policy 1, policy_version 130062 (0.0007) [2023-12-27 06:33:35,374][20347] Updated weights for policy 0, policy_version 130380 (0.0007) [2023-12-27 06:33:35,397][20381] Updated weights for policy 1, policy_version 130072 (0.0006) [2023-12-27 06:33:35,430][20347] Updated weights for policy 0, policy_version 130390 (0.0007) [2023-12-27 06:33:35,488][20347] Updated weights for policy 0, policy_version 130400 (0.0008) [2023-12-27 06:33:36,110][20347] Updated weights for policy 0, policy_version 130410 (0.0007) [2023-12-27 06:33:36,134][20381] Updated weights for policy 1, policy_version 130082 (0.0008) [2023-12-27 06:33:36,172][20347] Updated weights for policy 0, policy_version 130420 (0.0009) [2023-12-27 06:33:36,188][20381] Updated weights for policy 1, policy_version 130092 (0.0006) [2023-12-27 06:33:36,220][20347] Updated weights for policy 0, policy_version 130430 (0.0006) [2023-12-27 06:33:36,239][20381] Updated weights for policy 1, policy_version 130102 (0.0008) [2023-12-27 06:33:36,267][20347] Updated weights for policy 0, policy_version 130440 (0.0006) [2023-12-27 06:33:36,903][20347] Updated weights for policy 0, policy_version 130450 (0.0007) [2023-12-27 06:33:36,960][20347] Updated weights for policy 0, policy_version 130460 (0.0006) [2023-12-27 06:33:36,970][20381] Updated weights for policy 1, policy_version 130112 (0.0007) [2023-12-27 06:33:37,013][20347] Updated weights for policy 0, policy_version 130470 (0.0006) [2023-12-27 06:33:37,024][20381] Updated weights for policy 1, policy_version 130122 (0.0005) [2023-12-27 06:33:37,076][20381] Updated weights for policy 1, policy_version 130132 (0.0007) [2023-12-27 06:33:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 66740224. Throughput: 0: 10311.3, 1: 10471.4. Samples: 66708120. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:33:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000130136_33325056.pth... [2023-12-27 06:33:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000130472_33415168.pth... [2023-12-27 06:33:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000128928_33013760.pth [2023-12-27 06:33:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000129256_33103872.pth [2023-12-27 06:33:37,718][20347] Updated weights for policy 0, policy_version 130480 (0.0007) [2023-12-27 06:33:37,778][20347] Updated weights for policy 0, policy_version 130490 (0.0008) [2023-12-27 06:33:37,789][20381] Updated weights for policy 1, policy_version 130142 (0.0010) [2023-12-27 06:33:37,832][20347] Updated weights for policy 0, policy_version 130500 (0.0006) [2023-12-27 06:33:37,849][20381] Updated weights for policy 1, policy_version 130152 (0.0011) [2023-12-27 06:33:37,906][20381] Updated weights for policy 1, policy_version 130162 (0.0011) [2023-12-27 06:33:38,555][20347] Updated weights for policy 0, policy_version 130510 (0.0007) [2023-12-27 06:33:38,565][20381] Updated weights for policy 1, policy_version 130172 (0.0010) [2023-12-27 06:33:38,611][20347] Updated weights for policy 0, policy_version 130520 (0.0005) [2023-12-27 06:33:38,624][20381] Updated weights for policy 1, policy_version 130182 (0.0007) [2023-12-27 06:33:38,669][20347] Updated weights for policy 0, policy_version 130530 (0.0005) [2023-12-27 06:33:38,685][20381] Updated weights for policy 1, policy_version 130192 (0.0009) [2023-12-27 06:33:39,324][20381] Updated weights for policy 1, policy_version 130202 (0.0006) [2023-12-27 06:33:39,392][20381] Updated weights for policy 1, policy_version 130212 (0.0008) [2023-12-27 06:33:39,396][20347] Updated weights for policy 0, policy_version 130540 (0.0006) [2023-12-27 06:33:39,452][20381] Updated weights for policy 1, policy_version 130222 (0.0009) [2023-12-27 06:33:39,455][20347] Updated weights for policy 0, policy_version 130550 (0.0009) [2023-12-27 06:33:39,505][20347] Updated weights for policy 0, policy_version 130560 (0.0009) [2023-12-27 06:33:39,505][20381] Updated weights for policy 1, policy_version 130232 (0.0009) [2023-12-27 06:33:40,287][20381] Updated weights for policy 1, policy_version 130242 (0.0008) [2023-12-27 06:33:40,297][20347] Updated weights for policy 0, policy_version 130570 (0.0009) [2023-12-27 06:33:40,340][20381] Updated weights for policy 1, policy_version 130252 (0.0007) [2023-12-27 06:33:40,358][20347] Updated weights for policy 0, policy_version 130580 (0.0010) [2023-12-27 06:33:40,400][20381] Updated weights for policy 1, policy_version 130262 (0.0007) [2023-12-27 06:33:40,417][20347] Updated weights for policy 0, policy_version 130590 (0.0010) [2023-12-27 06:33:40,472][20347] Updated weights for policy 0, policy_version 130600 (0.0010) [2023-12-27 06:33:41,072][20381] Updated weights for policy 1, policy_version 130272 (0.0011) [2023-12-27 06:33:41,128][20381] Updated weights for policy 1, policy_version 130282 (0.0011) [2023-12-27 06:33:41,139][20347] Updated weights for policy 0, policy_version 130610 (0.0006) [2023-12-27 06:33:41,188][20381] Updated weights for policy 1, policy_version 130292 (0.0010) [2023-12-27 06:33:41,198][20347] Updated weights for policy 0, policy_version 130620 (0.0006) [2023-12-27 06:33:41,256][20347] Updated weights for policy 0, policy_version 130630 (0.0007) [2023-12-27 06:33:41,886][20381] Updated weights for policy 1, policy_version 130302 (0.0007) [2023-12-27 06:33:41,942][20381] Updated weights for policy 1, policy_version 130312 (0.0005) [2023-12-27 06:33:41,956][20347] Updated weights for policy 0, policy_version 130640 (0.0008) [2023-12-27 06:33:41,998][20381] Updated weights for policy 1, policy_version 130322 (0.0006) [2023-12-27 06:33:42,009][20347] Updated weights for policy 0, policy_version 130650 (0.0007) [2023-12-27 06:33:42,068][20347] Updated weights for policy 0, policy_version 130660 (0.0007) [2023-12-27 06:33:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 66838528. Throughput: 0: 10303.1, 1: 10413.2. Samples: 66828276. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:33:42,671][20381] Updated weights for policy 1, policy_version 130332 (0.0008) [2023-12-27 06:33:42,726][20381] Updated weights for policy 1, policy_version 130342 (0.0010) [2023-12-27 06:33:42,755][20347] Updated weights for policy 0, policy_version 130670 (0.0008) [2023-12-27 06:33:42,781][20381] Updated weights for policy 1, policy_version 130352 (0.0011) [2023-12-27 06:33:42,822][20347] Updated weights for policy 0, policy_version 130680 (0.0008) [2023-12-27 06:33:42,881][20347] Updated weights for policy 0, policy_version 130690 (0.0007) [2023-12-27 06:33:43,350][20381] Updated weights for policy 1, policy_version 130362 (0.0010) [2023-12-27 06:33:43,404][20381] Updated weights for policy 1, policy_version 130372 (0.0006) [2023-12-27 06:33:43,460][20381] Updated weights for policy 1, policy_version 130382 (0.0005) [2023-12-27 06:33:43,520][20381] Updated weights for policy 1, policy_version 130392 (0.0005) [2023-12-27 06:33:43,574][20347] Updated weights for policy 0, policy_version 130700 (0.0007) [2023-12-27 06:33:43,637][20347] Updated weights for policy 0, policy_version 130710 (0.0008) [2023-12-27 06:33:43,689][20347] Updated weights for policy 0, policy_version 130720 (0.0007) [2023-12-27 06:33:44,066][20381] Updated weights for policy 1, policy_version 130402 (0.0008) [2023-12-27 06:33:44,124][20381] Updated weights for policy 1, policy_version 130412 (0.0008) [2023-12-27 06:33:44,186][20381] Updated weights for policy 1, policy_version 130422 (0.0009) [2023-12-27 06:33:44,401][20347] Updated weights for policy 0, policy_version 130730 (0.0008) [2023-12-27 06:33:44,467][20347] Updated weights for policy 0, policy_version 130740 (0.0006) [2023-12-27 06:33:44,526][20347] Updated weights for policy 0, policy_version 130750 (0.0008) [2023-12-27 06:33:44,585][20347] Updated weights for policy 0, policy_version 130760 (0.0007) [2023-12-27 06:33:44,840][20381] Updated weights for policy 1, policy_version 130432 (0.0008) [2023-12-27 06:33:44,897][20381] Updated weights for policy 1, policy_version 130442 (0.0007) [2023-12-27 06:33:44,957][20381] Updated weights for policy 1, policy_version 130452 (0.0008) [2023-12-27 06:33:45,234][20347] Updated weights for policy 0, policy_version 130770 (0.0005) [2023-12-27 06:33:45,297][20347] Updated weights for policy 0, policy_version 130780 (0.0005) [2023-12-27 06:33:45,346][20347] Updated weights for policy 0, policy_version 130790 (0.0008) [2023-12-27 06:33:45,556][20381] Updated weights for policy 1, policy_version 130462 (0.0008) [2023-12-27 06:33:45,609][20381] Updated weights for policy 1, policy_version 130472 (0.0008) [2023-12-27 06:33:45,668][20381] Updated weights for policy 1, policy_version 130482 (0.0008) [2023-12-27 06:33:45,975][20347] Updated weights for policy 0, policy_version 130800 (0.0008) [2023-12-27 06:33:46,024][20347] Updated weights for policy 0, policy_version 130810 (0.0008) [2023-12-27 06:33:46,080][20347] Updated weights for policy 0, policy_version 130820 (0.0008) [2023-12-27 06:33:46,302][20381] Updated weights for policy 1, policy_version 130492 (0.0008) [2023-12-27 06:33:46,357][20381] Updated weights for policy 1, policy_version 130502 (0.0009) [2023-12-27 06:33:46,415][20381] Updated weights for policy 1, policy_version 130512 (0.0006) [2023-12-27 06:33:46,726][20347] Updated weights for policy 0, policy_version 130830 (0.0006) [2023-12-27 06:33:46,784][20347] Updated weights for policy 0, policy_version 130840 (0.0005) [2023-12-27 06:33:46,842][20347] Updated weights for policy 0, policy_version 130850 (0.0005) [2023-12-27 06:33:47,053][20381] Updated weights for policy 1, policy_version 130522 (0.0005) [2023-12-27 06:33:47,104][20381] Updated weights for policy 1, policy_version 130532 (0.0005) [2023-12-27 06:33:47,154][20381] Updated weights for policy 1, policy_version 130542 (0.0005) [2023-12-27 06:33:47,211][20381] Updated weights for policy 1, policy_version 130552 (0.0005) [2023-12-27 06:33:47,348][20347] Updated weights for policy 0, policy_version 130860 (0.0005) [2023-12-27 06:33:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 66945024. Throughput: 0: 10314.3, 1: 10481.3. Samples: 66955752. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:33:47,406][20347] Updated weights for policy 0, policy_version 130870 (0.0005) [2023-12-27 06:33:47,465][20347] Updated weights for policy 0, policy_version 130880 (0.0006) [2023-12-27 06:33:47,786][20381] Updated weights for policy 1, policy_version 130562 (0.0008) [2023-12-27 06:33:47,841][20381] Updated weights for policy 1, policy_version 130572 (0.0007) [2023-12-27 06:33:47,887][20381] Updated weights for policy 1, policy_version 130582 (0.0010) [2023-12-27 06:33:48,056][20347] Updated weights for policy 0, policy_version 130890 (0.0008) [2023-12-27 06:33:48,117][20347] Updated weights for policy 0, policy_version 130900 (0.0008) [2023-12-27 06:33:48,174][20347] Updated weights for policy 0, policy_version 130910 (0.0007) [2023-12-27 06:33:48,226][20347] Updated weights for policy 0, policy_version 130920 (0.0007) [2023-12-27 06:33:48,623][20381] Updated weights for policy 1, policy_version 130592 (0.0007) [2023-12-27 06:33:48,683][20381] Updated weights for policy 1, policy_version 130602 (0.0011) [2023-12-27 06:33:48,740][20381] Updated weights for policy 1, policy_version 130612 (0.0011) [2023-12-27 06:33:48,930][20347] Updated weights for policy 0, policy_version 130930 (0.0007) [2023-12-27 06:33:48,991][20347] Updated weights for policy 0, policy_version 130940 (0.0007) [2023-12-27 06:33:49,035][20347] Updated weights for policy 0, policy_version 130950 (0.0007) [2023-12-27 06:33:49,409][20381] Updated weights for policy 1, policy_version 130622 (0.0008) [2023-12-27 06:33:49,459][20381] Updated weights for policy 1, policy_version 130632 (0.0008) [2023-12-27 06:33:49,510][20381] Updated weights for policy 1, policy_version 130642 (0.0009) [2023-12-27 06:33:49,706][20347] Updated weights for policy 0, policy_version 130960 (0.0007) [2023-12-27 06:33:49,758][20347] Updated weights for policy 0, policy_version 130970 (0.0005) [2023-12-27 06:33:49,814][20347] Updated weights for policy 0, policy_version 130980 (0.0006) [2023-12-27 06:33:50,262][20381] Updated weights for policy 1, policy_version 130652 (0.0008) [2023-12-27 06:33:50,318][20381] Updated weights for policy 1, policy_version 130662 (0.0007) [2023-12-27 06:33:50,371][20381] Updated weights for policy 1, policy_version 130672 (0.0007) [2023-12-27 06:33:50,433][20347] Updated weights for policy 0, policy_version 130990 (0.0010) [2023-12-27 06:33:50,494][20347] Updated weights for policy 0, policy_version 131000 (0.0010) [2023-12-27 06:33:50,551][20347] Updated weights for policy 0, policy_version 131010 (0.0009) [2023-12-27 06:33:51,037][20381] Updated weights for policy 1, policy_version 130682 (0.0008) [2023-12-27 06:33:51,093][20381] Updated weights for policy 1, policy_version 130692 (0.0009) [2023-12-27 06:33:51,153][20381] Updated weights for policy 1, policy_version 130702 (0.0007) [2023-12-27 06:33:51,208][20381] Updated weights for policy 1, policy_version 130712 (0.0008) [2023-12-27 06:33:51,270][20347] Updated weights for policy 0, policy_version 131020 (0.0010) [2023-12-27 06:33:51,326][20347] Updated weights for policy 0, policy_version 131030 (0.0009) [2023-12-27 06:33:51,389][20347] Updated weights for policy 0, policy_version 131040 (0.0008) [2023-12-27 06:33:51,933][20381] Updated weights for policy 1, policy_version 130722 (0.0007) [2023-12-27 06:33:51,991][20381] Updated weights for policy 1, policy_version 130732 (0.0008) [2023-12-27 06:33:52,055][20381] Updated weights for policy 1, policy_version 130742 (0.0008) [2023-12-27 06:33:52,059][20347] Updated weights for policy 0, policy_version 131050 (0.0006) [2023-12-27 06:33:52,121][20347] Updated weights for policy 0, policy_version 131060 (0.0008) [2023-12-27 06:33:52,175][20347] Updated weights for policy 0, policy_version 131070 (0.0009) [2023-12-27 06:33:52,219][20347] Updated weights for policy 0, policy_version 131080 (0.0008) [2023-12-27 06:33:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 67051520. Throughput: 0: 10322.5, 1: 10402.1. Samples: 67018428. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:33:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000131080_33570816.pth... [2023-12-27 06:33:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000130744_33480704.pth... [2023-12-27 06:33:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000129864_33259520.pth [2023-12-27 06:33:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000129536_33169408.pth [2023-12-27 06:33:52,742][20381] Updated weights for policy 1, policy_version 130752 (0.0007) [2023-12-27 06:33:52,798][20381] Updated weights for policy 1, policy_version 130762 (0.0007) [2023-12-27 06:33:52,850][20381] Updated weights for policy 1, policy_version 130772 (0.0008) [2023-12-27 06:33:52,975][20347] Updated weights for policy 0, policy_version 131090 (0.0010) [2023-12-27 06:33:53,031][20347] Updated weights for policy 0, policy_version 131100 (0.0010) [2023-12-27 06:33:53,092][20347] Updated weights for policy 0, policy_version 131110 (0.0010) [2023-12-27 06:33:53,557][20381] Updated weights for policy 1, policy_version 130782 (0.0008) [2023-12-27 06:33:53,607][20381] Updated weights for policy 1, policy_version 130792 (0.0010) [2023-12-27 06:33:53,671][20381] Updated weights for policy 1, policy_version 130802 (0.0009) [2023-12-27 06:33:53,722][20347] Updated weights for policy 0, policy_version 131120 (0.0006) [2023-12-27 06:33:53,780][20347] Updated weights for policy 0, policy_version 131130 (0.0005) [2023-12-27 06:33:53,842][20347] Updated weights for policy 0, policy_version 131140 (0.0005) [2023-12-27 06:33:54,418][20347] Updated weights for policy 0, policy_version 131150 (0.0008) [2023-12-27 06:33:54,445][20381] Updated weights for policy 1, policy_version 130812 (0.0007) [2023-12-27 06:33:54,477][20347] Updated weights for policy 0, policy_version 131160 (0.0011) [2023-12-27 06:33:54,501][20381] Updated weights for policy 1, policy_version 130822 (0.0005) [2023-12-27 06:33:54,529][20347] Updated weights for policy 0, policy_version 131170 (0.0010) [2023-12-27 06:33:54,553][20381] Updated weights for policy 1, policy_version 130832 (0.0005) [2023-12-27 06:33:55,208][20347] Updated weights for policy 0, policy_version 131180 (0.0010) [2023-12-27 06:33:55,230][20381] Updated weights for policy 1, policy_version 130842 (0.0007) [2023-12-27 06:33:55,262][20347] Updated weights for policy 0, policy_version 131190 (0.0007) [2023-12-27 06:33:55,283][20381] Updated weights for policy 1, policy_version 130852 (0.0006) [2023-12-27 06:33:55,307][20347] Updated weights for policy 0, policy_version 131200 (0.0007) [2023-12-27 06:33:55,335][20381] Updated weights for policy 1, policy_version 130862 (0.0005) [2023-12-27 06:33:55,393][20381] Updated weights for policy 1, policy_version 130872 (0.0006) [2023-12-27 06:33:55,987][20347] Updated weights for policy 0, policy_version 131210 (0.0007) [2023-12-27 06:33:56,045][20347] Updated weights for policy 0, policy_version 131220 (0.0007) [2023-12-27 06:33:56,077][20381] Updated weights for policy 1, policy_version 130882 (0.0006) [2023-12-27 06:33:56,102][20347] Updated weights for policy 0, policy_version 131230 (0.0006) [2023-12-27 06:33:56,131][20381] Updated weights for policy 1, policy_version 130892 (0.0005) [2023-12-27 06:33:56,154][20347] Updated weights for policy 0, policy_version 131240 (0.0007) [2023-12-27 06:33:56,181][20381] Updated weights for policy 1, policy_version 130902 (0.0006) [2023-12-27 06:33:56,758][20381] Updated weights for policy 1, policy_version 130912 (0.0005) [2023-12-27 06:33:56,798][20347] Updated weights for policy 0, policy_version 131250 (0.0005) [2023-12-27 06:33:56,812][20381] Updated weights for policy 1, policy_version 130922 (0.0005) [2023-12-27 06:33:56,849][20347] Updated weights for policy 0, policy_version 131260 (0.0009) [2023-12-27 06:33:56,865][20381] Updated weights for policy 1, policy_version 130932 (0.0005) [2023-12-27 06:33:56,896][20347] Updated weights for policy 0, policy_version 131270 (0.0010) [2023-12-27 06:33:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 67149824. Throughput: 0: 10327.0, 1: 10327.8. Samples: 67139784. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:33:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:33:57,466][20347] Updated weights for policy 0, policy_version 131280 (0.0010) [2023-12-27 06:33:57,483][20381] Updated weights for policy 1, policy_version 130942 (0.0005) [2023-12-27 06:33:57,528][20347] Updated weights for policy 0, policy_version 131290 (0.0009) [2023-12-27 06:33:57,537][20381] Updated weights for policy 1, policy_version 130952 (0.0005) [2023-12-27 06:33:57,585][20381] Updated weights for policy 1, policy_version 130962 (0.0006) [2023-12-27 06:33:57,590][20347] Updated weights for policy 0, policy_version 131300 (0.0008) [2023-12-27 06:33:58,209][20347] Updated weights for policy 0, policy_version 131310 (0.0006) [2023-12-27 06:33:58,262][20347] Updated weights for policy 0, policy_version 131320 (0.0007) [2023-12-27 06:33:58,312][20381] Updated weights for policy 1, policy_version 130972 (0.0006) [2023-12-27 06:33:58,315][20347] Updated weights for policy 0, policy_version 131330 (0.0007) [2023-12-27 06:33:58,382][20381] Updated weights for policy 1, policy_version 130982 (0.0007) [2023-12-27 06:33:58,443][20381] Updated weights for policy 1, policy_version 130992 (0.0006) [2023-12-27 06:33:59,160][20347] Updated weights for policy 0, policy_version 131340 (0.0008) [2023-12-27 06:33:59,179][20381] Updated weights for policy 1, policy_version 131002 (0.0007) [2023-12-27 06:33:59,228][20347] Updated weights for policy 0, policy_version 131350 (0.0008) [2023-12-27 06:33:59,245][20381] Updated weights for policy 1, policy_version 131012 (0.0007) [2023-12-27 06:33:59,289][20347] Updated weights for policy 0, policy_version 131360 (0.0007) [2023-12-27 06:33:59,311][20381] Updated weights for policy 1, policy_version 131022 (0.0007) [2023-12-27 06:33:59,375][20381] Updated weights for policy 1, policy_version 131032 (0.0008) [2023-12-27 06:34:00,026][20381] Updated weights for policy 1, policy_version 131042 (0.0009) [2023-12-27 06:34:00,056][20347] Updated weights for policy 0, policy_version 131370 (0.0007) [2023-12-27 06:34:00,079][20381] Updated weights for policy 1, policy_version 131052 (0.0006) [2023-12-27 06:34:00,114][20347] Updated weights for policy 0, policy_version 131380 (0.0007) [2023-12-27 06:34:00,136][20381] Updated weights for policy 1, policy_version 131062 (0.0007) [2023-12-27 06:34:00,176][20347] Updated weights for policy 0, policy_version 131390 (0.0006) [2023-12-27 06:34:00,225][20347] Updated weights for policy 0, policy_version 131400 (0.0007) [2023-12-27 06:34:00,695][20381] Updated weights for policy 1, policy_version 131072 (0.0007) [2023-12-27 06:34:00,741][20381] Updated weights for policy 1, policy_version 131082 (0.0008) [2023-12-27 06:34:00,792][20381] Updated weights for policy 1, policy_version 131092 (0.0009) [2023-12-27 06:34:00,811][20347] Updated weights for policy 0, policy_version 131410 (0.0005) [2023-12-27 06:34:00,870][20347] Updated weights for policy 0, policy_version 131420 (0.0007) [2023-12-27 06:34:00,925][20347] Updated weights for policy 0, policy_version 131430 (0.0005) [2023-12-27 06:34:01,492][20381] Updated weights for policy 1, policy_version 131102 (0.0006) [2023-12-27 06:34:01,549][20381] Updated weights for policy 1, policy_version 131112 (0.0009) [2023-12-27 06:34:01,566][20347] Updated weights for policy 0, policy_version 131440 (0.0005) [2023-12-27 06:34:01,605][20381] Updated weights for policy 1, policy_version 131122 (0.0010) [2023-12-27 06:34:01,631][20347] Updated weights for policy 0, policy_version 131450 (0.0009) [2023-12-27 06:34:01,685][20347] Updated weights for policy 0, policy_version 131460 (0.0010) [2023-12-27 06:34:02,234][20381] Updated weights for policy 1, policy_version 131132 (0.0009) [2023-12-27 06:34:02,295][20381] Updated weights for policy 1, policy_version 131142 (0.0007) [2023-12-27 06:34:02,359][20381] Updated weights for policy 1, policy_version 131152 (0.0008) [2023-12-27 06:34:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 67248128. Throughput: 0: 10292.2, 1: 10365.1. Samples: 67265728. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:34:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:02,365][20347] Updated weights for policy 0, policy_version 131470 (0.0008) [2023-12-27 06:34:02,419][20347] Updated weights for policy 0, policy_version 131480 (0.0007) [2023-12-27 06:34:02,469][20347] Updated weights for policy 0, policy_version 131490 (0.0008) [2023-12-27 06:34:02,945][20381] Updated weights for policy 1, policy_version 131162 (0.0007) [2023-12-27 06:34:02,991][20381] Updated weights for policy 1, policy_version 131172 (0.0005) [2023-12-27 06:34:03,039][20347] Updated weights for policy 0, policy_version 131500 (0.0007) [2023-12-27 06:34:03,046][20381] Updated weights for policy 1, policy_version 131182 (0.0006) [2023-12-27 06:34:03,091][20347] Updated weights for policy 0, policy_version 131510 (0.0005) [2023-12-27 06:34:03,102][20381] Updated weights for policy 1, policy_version 131192 (0.0005) [2023-12-27 06:34:03,155][20347] Updated weights for policy 0, policy_version 131520 (0.0005) [2023-12-27 06:34:03,658][20381] Updated weights for policy 1, policy_version 131202 (0.0009) [2023-12-27 06:34:03,713][20347] Updated weights for policy 0, policy_version 131530 (0.0006) [2023-12-27 06:34:03,715][20381] Updated weights for policy 1, policy_version 131212 (0.0010) [2023-12-27 06:34:03,759][20381] Updated weights for policy 1, policy_version 131222 (0.0010) [2023-12-27 06:34:03,766][20347] Updated weights for policy 0, policy_version 131540 (0.0005) [2023-12-27 06:34:03,813][20347] Updated weights for policy 0, policy_version 131550 (0.0008) [2023-12-27 06:34:03,868][20347] Updated weights for policy 0, policy_version 131560 (0.0007) [2023-12-27 06:34:04,462][20381] Updated weights for policy 1, policy_version 131232 (0.0011) [2023-12-27 06:34:04,511][20347] Updated weights for policy 0, policy_version 131570 (0.0010) [2023-12-27 06:34:04,518][20381] Updated weights for policy 1, policy_version 131242 (0.0011) [2023-12-27 06:34:04,565][20347] Updated weights for policy 0, policy_version 131580 (0.0010) [2023-12-27 06:34:04,571][20381] Updated weights for policy 1, policy_version 131252 (0.0011) [2023-12-27 06:34:04,617][20347] Updated weights for policy 0, policy_version 131590 (0.0010) [2023-12-27 06:34:05,222][20381] Updated weights for policy 1, policy_version 131262 (0.0007) [2023-12-27 06:34:05,268][20381] Updated weights for policy 1, policy_version 131272 (0.0005) [2023-12-27 06:34:05,314][20347] Updated weights for policy 0, policy_version 131600 (0.0010) [2023-12-27 06:34:05,322][20381] Updated weights for policy 1, policy_version 131282 (0.0009) [2023-12-27 06:34:05,368][20347] Updated weights for policy 0, policy_version 131610 (0.0010) [2023-12-27 06:34:05,422][20347] Updated weights for policy 0, policy_version 131620 (0.0010) [2023-12-27 06:34:05,918][20381] Updated weights for policy 1, policy_version 131292 (0.0006) [2023-12-27 06:34:05,965][20381] Updated weights for policy 1, policy_version 131302 (0.0008) [2023-12-27 06:34:06,013][20381] Updated weights for policy 1, policy_version 131312 (0.0007) [2023-12-27 06:34:06,103][20347] Updated weights for policy 0, policy_version 131630 (0.0008) [2023-12-27 06:34:06,163][20347] Updated weights for policy 0, policy_version 131640 (0.0008) [2023-12-27 06:34:06,218][20347] Updated weights for policy 0, policy_version 131650 (0.0007) [2023-12-27 06:34:06,744][20381] Updated weights for policy 1, policy_version 131322 (0.0008) [2023-12-27 06:34:06,798][20381] Updated weights for policy 1, policy_version 131332 (0.0007) [2023-12-27 06:34:06,842][20381] Updated weights for policy 1, policy_version 131342 (0.0006) [2023-12-27 06:34:06,900][20381] Updated weights for policy 1, policy_version 131352 (0.0006) [2023-12-27 06:34:06,909][20347] Updated weights for policy 0, policy_version 131660 (0.0008) [2023-12-27 06:34:06,967][20347] Updated weights for policy 0, policy_version 131670 (0.0010) [2023-12-27 06:34:07,027][20347] Updated weights for policy 0, policy_version 131680 (0.0010) [2023-12-27 06:34:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 67362816. Throughput: 0: 10337.4, 1: 10441.0. Samples: 67332288. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:34:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000131352_33636352.pth... [2023-12-27 06:34:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000131688_33726464.pth... [2023-12-27 06:34:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000130136_33325056.pth [2023-12-27 06:34:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000130472_33415168.pth [2023-12-27 06:34:07,564][20381] Updated weights for policy 1, policy_version 131362 (0.0008) [2023-12-27 06:34:07,620][20381] Updated weights for policy 1, policy_version 131372 (0.0006) [2023-12-27 06:34:07,681][20381] Updated weights for policy 1, policy_version 131382 (0.0005) [2023-12-27 06:34:07,699][20347] Updated weights for policy 0, policy_version 131690 (0.0010) [2023-12-27 06:34:07,749][20347] Updated weights for policy 0, policy_version 131700 (0.0010) [2023-12-27 06:34:07,805][20347] Updated weights for policy 0, policy_version 131710 (0.0008) [2023-12-27 06:34:07,867][20347] Updated weights for policy 0, policy_version 131720 (0.0008) [2023-12-27 06:34:08,259][20381] Updated weights for policy 1, policy_version 131392 (0.0006) [2023-12-27 06:34:08,316][20381] Updated weights for policy 1, policy_version 131402 (0.0007) [2023-12-27 06:34:08,378][20381] Updated weights for policy 1, policy_version 131412 (0.0010) [2023-12-27 06:34:08,557][20347] Updated weights for policy 0, policy_version 131730 (0.0011) [2023-12-27 06:34:08,623][20347] Updated weights for policy 0, policy_version 131740 (0.0011) [2023-12-27 06:34:08,684][20347] Updated weights for policy 0, policy_version 131750 (0.0007) [2023-12-27 06:34:09,063][20381] Updated weights for policy 1, policy_version 131422 (0.0008) [2023-12-27 06:34:09,114][20381] Updated weights for policy 1, policy_version 131432 (0.0008) [2023-12-27 06:34:09,161][20381] Updated weights for policy 1, policy_version 131442 (0.0008) [2023-12-27 06:34:09,286][20347] Updated weights for policy 0, policy_version 131760 (0.0006) [2023-12-27 06:34:09,345][20347] Updated weights for policy 0, policy_version 131770 (0.0010) [2023-12-27 06:34:09,408][20347] Updated weights for policy 0, policy_version 131780 (0.0011) [2023-12-27 06:34:09,883][20381] Updated weights for policy 1, policy_version 131452 (0.0009) [2023-12-27 06:34:09,951][20381] Updated weights for policy 1, policy_version 131462 (0.0006) [2023-12-27 06:34:10,011][20381] Updated weights for policy 1, policy_version 131472 (0.0007) [2023-12-27 06:34:10,127][20347] Updated weights for policy 0, policy_version 131790 (0.0010) [2023-12-27 06:34:10,192][20347] Updated weights for policy 0, policy_version 131800 (0.0010) [2023-12-27 06:34:10,248][20347] Updated weights for policy 0, policy_version 131810 (0.0010) [2023-12-27 06:34:10,714][20381] Updated weights for policy 1, policy_version 131482 (0.0007) [2023-12-27 06:34:10,768][20381] Updated weights for policy 1, policy_version 131492 (0.0005) [2023-12-27 06:34:10,821][20381] Updated weights for policy 1, policy_version 131502 (0.0006) [2023-12-27 06:34:10,864][20347] Updated weights for policy 0, policy_version 131820 (0.0009) [2023-12-27 06:34:10,885][20381] Updated weights for policy 1, policy_version 131512 (0.0006) [2023-12-27 06:34:10,913][20347] Updated weights for policy 0, policy_version 131830 (0.0006) [2023-12-27 06:34:10,968][20347] Updated weights for policy 0, policy_version 131840 (0.0005) [2023-12-27 06:34:11,571][20381] Updated weights for policy 1, policy_version 131522 (0.0007) [2023-12-27 06:34:11,632][20381] Updated weights for policy 1, policy_version 131532 (0.0009) [2023-12-27 06:34:11,650][20347] Updated weights for policy 0, policy_version 131850 (0.0006) [2023-12-27 06:34:11,705][20381] Updated weights for policy 1, policy_version 131542 (0.0006) [2023-12-27 06:34:11,714][20347] Updated weights for policy 0, policy_version 131860 (0.0008) [2023-12-27 06:34:11,768][20347] Updated weights for policy 0, policy_version 131870 (0.0008) [2023-12-27 06:34:11,824][20347] Updated weights for policy 0, policy_version 131880 (0.0008) [2023-12-27 06:34:12,357][20381] Updated weights for policy 1, policy_version 131552 (0.0007) [2023-12-27 06:34:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 67461120. Throughput: 0: 10362.6, 1: 10452.3. Samples: 67456168. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:34:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:12,418][20381] Updated weights for policy 1, policy_version 131562 (0.0008) [2023-12-27 06:34:12,475][20381] Updated weights for policy 1, policy_version 131572 (0.0005) [2023-12-27 06:34:12,608][20347] Updated weights for policy 0, policy_version 131890 (0.0008) [2023-12-27 06:34:12,666][20347] Updated weights for policy 0, policy_version 131900 (0.0007) [2023-12-27 06:34:12,724][20347] Updated weights for policy 0, policy_version 131910 (0.0007) [2023-12-27 06:34:13,142][20381] Updated weights for policy 1, policy_version 131582 (0.0007) [2023-12-27 06:34:13,191][20381] Updated weights for policy 1, policy_version 131592 (0.0007) [2023-12-27 06:34:13,255][20381] Updated weights for policy 1, policy_version 131602 (0.0009) [2023-12-27 06:34:13,383][20347] Updated weights for policy 0, policy_version 131920 (0.0007) [2023-12-27 06:34:13,430][20347] Updated weights for policy 0, policy_version 131930 (0.0007) [2023-12-27 06:34:13,476][20347] Updated weights for policy 0, policy_version 131940 (0.0007) [2023-12-27 06:34:13,930][20381] Updated weights for policy 1, policy_version 131612 (0.0009) [2023-12-27 06:34:13,981][20381] Updated weights for policy 1, policy_version 131622 (0.0005) [2023-12-27 06:34:14,028][20381] Updated weights for policy 1, policy_version 131632 (0.0006) [2023-12-27 06:34:14,177][20347] Updated weights for policy 0, policy_version 131950 (0.0006) [2023-12-27 06:34:14,237][20347] Updated weights for policy 0, policy_version 131960 (0.0006) [2023-12-27 06:34:14,289][20347] Updated weights for policy 0, policy_version 131970 (0.0007) [2023-12-27 06:34:14,699][20381] Updated weights for policy 1, policy_version 131642 (0.0011) [2023-12-27 06:34:14,762][20381] Updated weights for policy 1, policy_version 131652 (0.0011) [2023-12-27 06:34:14,820][20381] Updated weights for policy 1, policy_version 131662 (0.0008) [2023-12-27 06:34:14,882][20381] Updated weights for policy 1, policy_version 131672 (0.0010) [2023-12-27 06:34:14,984][20347] Updated weights for policy 0, policy_version 131980 (0.0008) [2023-12-27 06:34:15,050][20347] Updated weights for policy 0, policy_version 131990 (0.0008) [2023-12-27 06:34:15,100][20347] Updated weights for policy 0, policy_version 132000 (0.0007) [2023-12-27 06:34:15,566][20381] Updated weights for policy 1, policy_version 131682 (0.0011) [2023-12-27 06:34:15,631][20381] Updated weights for policy 1, policy_version 131692 (0.0010) [2023-12-27 06:34:15,686][20381] Updated weights for policy 1, policy_version 131702 (0.0011) [2023-12-27 06:34:15,807][20347] Updated weights for policy 0, policy_version 132010 (0.0008) [2023-12-27 06:34:15,862][20347] Updated weights for policy 0, policy_version 132020 (0.0007) [2023-12-27 06:34:15,912][20347] Updated weights for policy 0, policy_version 132030 (0.0007) [2023-12-27 06:34:15,966][20347] Updated weights for policy 0, policy_version 132040 (0.0007) [2023-12-27 06:34:16,370][20381] Updated weights for policy 1, policy_version 131712 (0.0008) [2023-12-27 06:34:16,428][20381] Updated weights for policy 1, policy_version 131722 (0.0008) [2023-12-27 06:34:16,482][20381] Updated weights for policy 1, policy_version 131732 (0.0006) [2023-12-27 06:34:16,635][20347] Updated weights for policy 0, policy_version 132050 (0.0007) [2023-12-27 06:34:16,689][20347] Updated weights for policy 0, policy_version 132060 (0.0005) [2023-12-27 06:34:16,748][20347] Updated weights for policy 0, policy_version 132070 (0.0006) [2023-12-27 06:34:17,053][20381] Updated weights for policy 1, policy_version 131742 (0.0007) [2023-12-27 06:34:17,113][20381] Updated weights for policy 1, policy_version 131752 (0.0009) [2023-12-27 06:34:17,165][20381] Updated weights for policy 1, policy_version 131762 (0.0007) [2023-12-27 06:34:17,328][20347] Updated weights for policy 0, policy_version 132080 (0.0006) [2023-12-27 06:34:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 67567616. Throughput: 0: 10321.1, 1: 10408.0. Samples: 67578384. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 06:34:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:17,383][20347] Updated weights for policy 0, policy_version 132090 (0.0005) [2023-12-27 06:34:17,442][20347] Updated weights for policy 0, policy_version 132100 (0.0007) [2023-12-27 06:34:17,810][20381] Updated weights for policy 1, policy_version 131772 (0.0007) [2023-12-27 06:34:17,859][20381] Updated weights for policy 1, policy_version 131782 (0.0006) [2023-12-27 06:34:17,905][20381] Updated weights for policy 1, policy_version 131792 (0.0008) [2023-12-27 06:34:18,122][20347] Updated weights for policy 0, policy_version 132110 (0.0007) [2023-12-27 06:34:18,179][20347] Updated weights for policy 0, policy_version 132120 (0.0007) [2023-12-27 06:34:18,243][20347] Updated weights for policy 0, policy_version 132130 (0.0008) [2023-12-27 06:34:18,522][20381] Updated weights for policy 1, policy_version 131802 (0.0007) [2023-12-27 06:34:18,583][20381] Updated weights for policy 1, policy_version 131812 (0.0008) [2023-12-27 06:34:18,638][20381] Updated weights for policy 1, policy_version 131822 (0.0007) [2023-12-27 06:34:18,705][20381] Updated weights for policy 1, policy_version 131832 (0.0005) [2023-12-27 06:34:18,875][20347] Updated weights for policy 0, policy_version 132140 (0.0007) [2023-12-27 06:34:18,927][20347] Updated weights for policy 0, policy_version 132150 (0.0007) [2023-12-27 06:34:18,975][20347] Updated weights for policy 0, policy_version 132160 (0.0007) [2023-12-27 06:34:19,282][20381] Updated weights for policy 1, policy_version 131842 (0.0007) [2023-12-27 06:34:19,349][20381] Updated weights for policy 1, policy_version 131852 (0.0009) [2023-12-27 06:34:19,410][20381] Updated weights for policy 1, policy_version 131862 (0.0011) [2023-12-27 06:34:19,678][20347] Updated weights for policy 0, policy_version 132170 (0.0006) [2023-12-27 06:34:19,737][20347] Updated weights for policy 0, policy_version 132180 (0.0007) [2023-12-27 06:34:19,789][20347] Updated weights for policy 0, policy_version 132190 (0.0008) [2023-12-27 06:34:19,845][20347] Updated weights for policy 0, policy_version 132200 (0.0008) [2023-12-27 06:34:20,102][20381] Updated weights for policy 1, policy_version 131872 (0.0007) [2023-12-27 06:34:20,167][20381] Updated weights for policy 1, policy_version 131882 (0.0008) [2023-12-27 06:34:20,237][20381] Updated weights for policy 1, policy_version 131892 (0.0008) [2023-12-27 06:34:20,542][20347] Updated weights for policy 0, policy_version 132210 (0.0006) [2023-12-27 06:34:20,600][20347] Updated weights for policy 0, policy_version 132220 (0.0007) [2023-12-27 06:34:20,664][20347] Updated weights for policy 0, policy_version 132230 (0.0006) [2023-12-27 06:34:20,959][20381] Updated weights for policy 1, policy_version 131902 (0.0008) [2023-12-27 06:34:21,020][20381] Updated weights for policy 1, policy_version 131912 (0.0007) [2023-12-27 06:34:21,087][20381] Updated weights for policy 1, policy_version 131922 (0.0007) [2023-12-27 06:34:21,283][20347] Updated weights for policy 0, policy_version 132240 (0.0007) [2023-12-27 06:34:21,349][20347] Updated weights for policy 0, policy_version 132250 (0.0007) [2023-12-27 06:34:21,415][20347] Updated weights for policy 0, policy_version 132260 (0.0006) [2023-12-27 06:34:21,726][20381] Updated weights for policy 1, policy_version 131932 (0.0007) [2023-12-27 06:34:21,788][20381] Updated weights for policy 1, policy_version 131942 (0.0007) [2023-12-27 06:34:21,848][20381] Updated weights for policy 1, policy_version 131952 (0.0008) [2023-12-27 06:34:22,113][20347] Updated weights for policy 0, policy_version 132270 (0.0006) [2023-12-27 06:34:22,165][20347] Updated weights for policy 0, policy_version 132280 (0.0008) [2023-12-27 06:34:22,215][20347] Updated weights for policy 0, policy_version 132290 (0.0008) [2023-12-27 06:34:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 67674112. Throughput: 0: 10339.0, 1: 10422.0. Samples: 67642364. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000131960_33792000.pth... [2023-12-27 06:34:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000132296_33882112.pth... [2023-12-27 06:34:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000130744_33480704.pth [2023-12-27 06:34:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000131080_33570816.pth [2023-12-27 06:34:22,575][20381] Updated weights for policy 1, policy_version 131962 (0.0007) [2023-12-27 06:34:22,632][20381] Updated weights for policy 1, policy_version 131972 (0.0005) [2023-12-27 06:34:22,694][20381] Updated weights for policy 1, policy_version 131982 (0.0011) [2023-12-27 06:34:22,751][20381] Updated weights for policy 1, policy_version 131992 (0.0011) [2023-12-27 06:34:22,919][20347] Updated weights for policy 0, policy_version 132300 (0.0006) [2023-12-27 06:34:22,977][20347] Updated weights for policy 0, policy_version 132310 (0.0005) [2023-12-27 06:34:23,029][20347] Updated weights for policy 0, policy_version 132320 (0.0005) [2023-12-27 06:34:23,420][20381] Updated weights for policy 1, policy_version 132002 (0.0011) [2023-12-27 06:34:23,481][20381] Updated weights for policy 1, policy_version 132012 (0.0010) [2023-12-27 06:34:23,540][20381] Updated weights for policy 1, policy_version 132022 (0.0009) [2023-12-27 06:34:23,619][20347] Updated weights for policy 0, policy_version 132330 (0.0006) [2023-12-27 06:34:23,675][20347] Updated weights for policy 0, policy_version 132340 (0.0008) [2023-12-27 06:34:23,724][20347] Updated weights for policy 0, policy_version 132350 (0.0007) [2023-12-27 06:34:23,770][20347] Updated weights for policy 0, policy_version 132360 (0.0009) [2023-12-27 06:34:24,108][20381] Updated weights for policy 1, policy_version 132032 (0.0008) [2023-12-27 06:34:24,160][20381] Updated weights for policy 1, policy_version 132042 (0.0010) [2023-12-27 06:34:24,215][20381] Updated weights for policy 1, policy_version 132052 (0.0010) [2023-12-27 06:34:24,465][20347] Updated weights for policy 0, policy_version 132370 (0.0008) [2023-12-27 06:34:24,518][20347] Updated weights for policy 0, policy_version 132380 (0.0007) [2023-12-27 06:34:24,573][20347] Updated weights for policy 0, policy_version 132390 (0.0007) [2023-12-27 06:34:24,896][20381] Updated weights for policy 1, policy_version 132062 (0.0009) [2023-12-27 06:34:24,955][20381] Updated weights for policy 1, policy_version 132072 (0.0007) [2023-12-27 06:34:25,016][20381] Updated weights for policy 1, policy_version 132082 (0.0006) [2023-12-27 06:34:25,193][20347] Updated weights for policy 0, policy_version 132400 (0.0009) [2023-12-27 06:34:25,242][20347] Updated weights for policy 0, policy_version 132410 (0.0010) [2023-12-27 06:34:25,288][20347] Updated weights for policy 0, policy_version 132420 (0.0010) [2023-12-27 06:34:25,644][20381] Updated weights for policy 1, policy_version 132092 (0.0007) [2023-12-27 06:34:25,699][20381] Updated weights for policy 1, policy_version 132102 (0.0011) [2023-12-27 06:34:25,757][20381] Updated weights for policy 1, policy_version 132112 (0.0010) [2023-12-27 06:34:25,987][20347] Updated weights for policy 0, policy_version 132430 (0.0008) [2023-12-27 06:34:26,041][20347] Updated weights for policy 0, policy_version 132440 (0.0008) [2023-12-27 06:34:26,097][20347] Updated weights for policy 0, policy_version 132450 (0.0007) [2023-12-27 06:34:26,449][20381] Updated weights for policy 1, policy_version 132122 (0.0010) [2023-12-27 06:34:26,502][20381] Updated weights for policy 1, policy_version 132132 (0.0009) [2023-12-27 06:34:26,555][20381] Updated weights for policy 1, policy_version 132142 (0.0010) [2023-12-27 06:34:26,621][20381] Updated weights for policy 1, policy_version 132152 (0.0005) [2023-12-27 06:34:26,686][20347] Updated weights for policy 0, policy_version 132460 (0.0007) [2023-12-27 06:34:26,743][20347] Updated weights for policy 0, policy_version 132470 (0.0005) [2023-12-27 06:34:26,795][20347] Updated weights for policy 0, policy_version 132480 (0.0005) [2023-12-27 06:34:27,282][20381] Updated weights for policy 1, policy_version 132162 (0.0005) [2023-12-27 06:34:27,340][20381] Updated weights for policy 1, policy_version 132172 (0.0006) [2023-12-27 06:34:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 67772416. Throughput: 0: 10394.8, 1: 10470.0. Samples: 67767196. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:27,394][20381] Updated weights for policy 1, policy_version 132182 (0.0010) [2023-12-27 06:34:27,401][20347] Updated weights for policy 0, policy_version 132490 (0.0005) [2023-12-27 06:34:27,460][20347] Updated weights for policy 0, policy_version 132500 (0.0006) [2023-12-27 06:34:27,517][20347] Updated weights for policy 0, policy_version 132510 (0.0008) [2023-12-27 06:34:27,571][20347] Updated weights for policy 0, policy_version 132520 (0.0010) [2023-12-27 06:34:27,960][20381] Updated weights for policy 1, policy_version 132192 (0.0006) [2023-12-27 06:34:28,017][20381] Updated weights for policy 1, policy_version 132202 (0.0006) [2023-12-27 06:34:28,066][20381] Updated weights for policy 1, policy_version 132212 (0.0005) [2023-12-27 06:34:28,159][20347] Updated weights for policy 0, policy_version 132530 (0.0006) [2023-12-27 06:34:28,239][20347] Updated weights for policy 0, policy_version 132540 (0.0007) [2023-12-27 06:34:28,303][20347] Updated weights for policy 0, policy_version 132550 (0.0006) [2023-12-27 06:34:28,675][20381] Updated weights for policy 1, policy_version 132222 (0.0008) [2023-12-27 06:34:28,737][20381] Updated weights for policy 1, policy_version 132232 (0.0007) [2023-12-27 06:34:28,796][20381] Updated weights for policy 1, policy_version 132242 (0.0008) [2023-12-27 06:34:28,882][20347] Updated weights for policy 0, policy_version 132560 (0.0008) [2023-12-27 06:34:28,936][20347] Updated weights for policy 0, policy_version 132570 (0.0008) [2023-12-27 06:34:28,997][20347] Updated weights for policy 0, policy_version 132580 (0.0008) [2023-12-27 06:34:29,480][20381] Updated weights for policy 1, policy_version 132252 (0.0009) [2023-12-27 06:34:29,543][20381] Updated weights for policy 1, policy_version 132262 (0.0009) [2023-12-27 06:34:29,599][20381] Updated weights for policy 1, policy_version 132272 (0.0011) [2023-12-27 06:34:29,662][20347] Updated weights for policy 0, policy_version 132590 (0.0008) [2023-12-27 06:34:29,723][20347] Updated weights for policy 0, policy_version 132600 (0.0007) [2023-12-27 06:34:29,778][20347] Updated weights for policy 0, policy_version 132610 (0.0007) [2023-12-27 06:34:30,291][20381] Updated weights for policy 1, policy_version 132282 (0.0009) [2023-12-27 06:34:30,340][20381] Updated weights for policy 1, policy_version 132292 (0.0008) [2023-12-27 06:34:30,389][20381] Updated weights for policy 1, policy_version 132302 (0.0006) [2023-12-27 06:34:30,434][20381] Updated weights for policy 1, policy_version 132312 (0.0007) [2023-12-27 06:34:30,492][20347] Updated weights for policy 0, policy_version 132620 (0.0008) [2023-12-27 06:34:30,547][20347] Updated weights for policy 0, policy_version 132630 (0.0007) [2023-12-27 06:34:30,601][20347] Updated weights for policy 0, policy_version 132640 (0.0008) [2023-12-27 06:34:31,093][20381] Updated weights for policy 1, policy_version 132322 (0.0011) [2023-12-27 06:34:31,157][20381] Updated weights for policy 1, policy_version 132332 (0.0009) [2023-12-27 06:34:31,219][20381] Updated weights for policy 1, policy_version 132342 (0.0008) [2023-12-27 06:34:31,261][20347] Updated weights for policy 0, policy_version 132650 (0.0007) [2023-12-27 06:34:31,315][20347] Updated weights for policy 0, policy_version 132660 (0.0006) [2023-12-27 06:34:31,384][20347] Updated weights for policy 0, policy_version 132670 (0.0008) [2023-12-27 06:34:31,449][20347] Updated weights for policy 0, policy_version 132680 (0.0007) [2023-12-27 06:34:31,872][20381] Updated weights for policy 1, policy_version 132352 (0.0008) [2023-12-27 06:34:31,932][20381] Updated weights for policy 1, policy_version 132362 (0.0008) [2023-12-27 06:34:31,987][20381] Updated weights for policy 1, policy_version 132372 (0.0007) [2023-12-27 06:34:32,059][20347] Updated weights for policy 0, policy_version 132690 (0.0007) [2023-12-27 06:34:32,109][20347] Updated weights for policy 0, policy_version 132700 (0.0006) [2023-12-27 06:34:32,154][20347] Updated weights for policy 0, policy_version 132710 (0.0006) [2023-12-27 06:34:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 67887104. Throughput: 0: 10466.8, 1: 10419.8. Samples: 67895648. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:34:32,609][20381] Updated weights for policy 1, policy_version 132382 (0.0007) [2023-12-27 06:34:32,663][20381] Updated weights for policy 1, policy_version 132392 (0.0008) [2023-12-27 06:34:32,712][20381] Updated weights for policy 1, policy_version 132402 (0.0008) [2023-12-27 06:34:32,837][20347] Updated weights for policy 0, policy_version 132720 (0.0007) [2023-12-27 06:34:32,885][20347] Updated weights for policy 0, policy_version 132730 (0.0008) [2023-12-27 06:34:32,943][20347] Updated weights for policy 0, policy_version 132740 (0.0005) [2023-12-27 06:34:33,354][20381] Updated weights for policy 1, policy_version 132412 (0.0006) [2023-12-27 06:34:33,418][20381] Updated weights for policy 1, policy_version 132422 (0.0006) [2023-12-27 06:34:33,476][20381] Updated weights for policy 1, policy_version 132432 (0.0008) [2023-12-27 06:34:33,490][20347] Updated weights for policy 0, policy_version 132750 (0.0005) [2023-12-27 06:34:33,548][20347] Updated weights for policy 0, policy_version 132760 (0.0005) [2023-12-27 06:34:33,604][20347] Updated weights for policy 0, policy_version 132770 (0.0007) [2023-12-27 06:34:34,071][20381] Updated weights for policy 1, policy_version 132442 (0.0007) [2023-12-27 06:34:34,132][20381] Updated weights for policy 1, policy_version 132452 (0.0005) [2023-12-27 06:34:34,195][20381] Updated weights for policy 1, policy_version 132462 (0.0006) [2023-12-27 06:34:34,222][20347] Updated weights for policy 0, policy_version 132780 (0.0008) [2023-12-27 06:34:34,250][20381] Updated weights for policy 1, policy_version 132472 (0.0008) [2023-12-27 06:34:34,274][20347] Updated weights for policy 0, policy_version 132790 (0.0010) [2023-12-27 06:34:34,334][20347] Updated weights for policy 0, policy_version 132800 (0.0010) [2023-12-27 06:34:34,911][20381] Updated weights for policy 1, policy_version 132482 (0.0007) [2023-12-27 06:34:34,968][20381] Updated weights for policy 1, policy_version 132492 (0.0007) [2023-12-27 06:34:35,002][20347] Updated weights for policy 0, policy_version 132810 (0.0008) [2023-12-27 06:34:35,019][20381] Updated weights for policy 1, policy_version 132502 (0.0006) [2023-12-27 06:34:35,061][20347] Updated weights for policy 0, policy_version 132820 (0.0010) [2023-12-27 06:34:35,118][20347] Updated weights for policy 0, policy_version 132830 (0.0010) [2023-12-27 06:34:35,166][20347] Updated weights for policy 0, policy_version 132840 (0.0010) [2023-12-27 06:34:35,679][20381] Updated weights for policy 1, policy_version 132512 (0.0007) [2023-12-27 06:34:35,724][20381] Updated weights for policy 1, policy_version 132522 (0.0007) [2023-12-27 06:34:35,771][20381] Updated weights for policy 1, policy_version 132532 (0.0007) [2023-12-27 06:34:35,843][20347] Updated weights for policy 0, policy_version 132850 (0.0010) [2023-12-27 06:34:35,898][20347] Updated weights for policy 0, policy_version 132860 (0.0010) [2023-12-27 06:34:35,956][20347] Updated weights for policy 0, policy_version 132870 (0.0006) [2023-12-27 06:34:36,489][20381] Updated weights for policy 1, policy_version 132542 (0.0007) [2023-12-27 06:34:36,552][20381] Updated weights for policy 1, policy_version 132552 (0.0007) [2023-12-27 06:34:36,606][20381] Updated weights for policy 1, policy_version 132562 (0.0006) [2023-12-27 06:34:36,647][20347] Updated weights for policy 0, policy_version 132880 (0.0010) [2023-12-27 06:34:36,714][20347] Updated weights for policy 0, policy_version 132890 (0.0008) [2023-12-27 06:34:36,783][20347] Updated weights for policy 0, policy_version 132900 (0.0011) [2023-12-27 06:34:37,212][20381] Updated weights for policy 1, policy_version 132572 (0.0006) [2023-12-27 06:34:37,272][20381] Updated weights for policy 1, policy_version 132582 (0.0006) [2023-12-27 06:34:37,328][20381] Updated weights for policy 1, policy_version 132592 (0.0006) [2023-12-27 06:34:37,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 67985408. Throughput: 0: 10482.4, 1: 10465.1. Samples: 67961064. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:37,363][19170] Avg episode reward: [(0, '992.520'), (1, '1000.000')] [2023-12-27 06:34:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000132904_34037760.pth... [2023-12-27 06:34:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000132600_33955840.pth... [2023-12-27 06:34:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000131688_33726464.pth [2023-12-27 06:34:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000131352_33636352.pth [2023-12-27 06:34:37,420][20347] Updated weights for policy 0, policy_version 132910 (0.0010) [2023-12-27 06:34:37,427][20291] KL-divergence is very high: 148.7547 [2023-12-27 06:34:37,474][20291] KL-divergence is very high: 170.2921 [2023-12-27 06:34:37,480][20347] Updated weights for policy 0, policy_version 132920 (0.0010) [2023-12-27 06:34:37,536][20347] Updated weights for policy 0, policy_version 132930 (0.0011) [2023-12-27 06:34:37,937][20381] Updated weights for policy 1, policy_version 132602 (0.0007) [2023-12-27 06:34:38,004][20381] Updated weights for policy 1, policy_version 132612 (0.0006) [2023-12-27 06:34:38,070][20381] Updated weights for policy 1, policy_version 132622 (0.0007) [2023-12-27 06:34:38,133][20381] Updated weights for policy 1, policy_version 132632 (0.0007) [2023-12-27 06:34:38,215][20347] Updated weights for policy 0, policy_version 132940 (0.0008) [2023-12-27 06:34:38,252][20291] KL-divergence is very high: 264.3680 [2023-12-27 06:34:38,266][20347] Updated weights for policy 0, policy_version 132950 (0.0007) [2023-12-27 06:34:38,320][20347] Updated weights for policy 0, policy_version 132960 (0.0007) [2023-12-27 06:34:38,342][20291] KL-divergence is very high: 146.2122 [2023-12-27 06:34:38,788][20381] Updated weights for policy 1, policy_version 132642 (0.0008) [2023-12-27 06:34:38,841][20381] Updated weights for policy 1, policy_version 132652 (0.0008) [2023-12-27 06:34:38,892][20381] Updated weights for policy 1, policy_version 132662 (0.0007) [2023-12-27 06:34:39,037][20347] Updated weights for policy 0, policy_version 132970 (0.0009) [2023-12-27 06:34:39,071][20291] KL-divergence is very high: 108.5605 [2023-12-27 06:34:39,086][20347] Updated weights for policy 0, policy_version 132980 (0.0008) [2023-12-27 06:34:39,112][20291] KL-divergence is very high: 262.3512 [2023-12-27 06:34:39,122][20291] KL-divergence is very high: 215.3009 [2023-12-27 06:34:39,135][20347] Updated weights for policy 0, policy_version 132990 (0.0008) [2023-12-27 06:34:39,188][20347] Updated weights for policy 0, policy_version 133000 (0.0007) [2023-12-27 06:34:39,615][20381] Updated weights for policy 1, policy_version 132672 (0.0010) [2023-12-27 06:34:39,675][20381] Updated weights for policy 1, policy_version 132682 (0.0011) [2023-12-27 06:34:39,735][20381] Updated weights for policy 1, policy_version 132692 (0.0009) [2023-12-27 06:34:39,857][20291] KL-divergence is very high: 116.7187 [2023-12-27 06:34:39,868][20291] KL-divergence is very high: 151.8900 [2023-12-27 06:34:39,889][20291] KL-divergence is very high: 262.4217 [2023-12-27 06:34:39,902][20291] KL-divergence is very high: 173.9436 [2023-12-27 06:34:39,915][20347] Updated weights for policy 0, policy_version 133010 (0.0007) [2023-12-27 06:34:39,945][20291] KL-divergence is very high: 164.6447 [2023-12-27 06:34:39,969][20291] KL-divergence is very high: 150.5435 [2023-12-27 06:34:39,973][20347] Updated weights for policy 0, policy_version 133020 (0.0006) [2023-12-27 06:34:40,041][20347] Updated weights for policy 0, policy_version 133030 (0.0005) [2023-12-27 06:34:40,050][20291] KL-divergence is very high: 106.0155 [2023-12-27 06:34:40,425][20381] Updated weights for policy 1, policy_version 132702 (0.0011) [2023-12-27 06:34:40,487][20381] Updated weights for policy 1, policy_version 132712 (0.0010) [2023-12-27 06:34:40,546][20381] Updated weights for policy 1, policy_version 132722 (0.0010) [2023-12-27 06:34:40,617][20291] KL-divergence is very high: 165.0631 [2023-12-27 06:34:40,650][20347] Updated weights for policy 0, policy_version 133040 (0.0007) [2023-12-27 06:34:40,706][20347] Updated weights for policy 0, policy_version 133050 (0.0007) [2023-12-27 06:34:40,709][20291] KL-divergence is very high: 151.9649 [2023-12-27 06:34:40,758][20291] KL-divergence is very high: 143.8237 [2023-12-27 06:34:40,765][20347] Updated weights for policy 0, policy_version 133060 (0.0008) [2023-12-27 06:34:41,227][20381] Updated weights for policy 1, policy_version 132732 (0.0010) [2023-12-27 06:34:41,289][20381] Updated weights for policy 1, policy_version 132742 (0.0011) [2023-12-27 06:34:41,351][20381] Updated weights for policy 1, policy_version 132752 (0.0009) [2023-12-27 06:34:41,405][20291] KL-divergence is very high: 160.4878 [2023-12-27 06:34:41,412][20291] KL-divergence is very high: 193.4987 [2023-12-27 06:34:41,419][20291] KL-divergence is very high: 193.5524 [2023-12-27 06:34:41,425][20291] KL-divergence is very high: 130.1325 [2023-12-27 06:34:41,432][20291] KL-divergence is very high: 154.0552 [2023-12-27 06:34:41,437][20347] Updated weights for policy 0, policy_version 133070 (0.0009) [2023-12-27 06:34:41,482][20291] KL-divergence is very high: 129.9974 [2023-12-27 06:34:41,502][20347] Updated weights for policy 0, policy_version 133080 (0.0010) [2023-12-27 06:34:41,557][20291] KL-divergence is very high: 107.3292 [2023-12-27 06:34:41,564][20347] Updated weights for policy 0, policy_version 133090 (0.0010) [2023-12-27 06:34:42,063][20381] Updated weights for policy 1, policy_version 132762 (0.0009) [2023-12-27 06:34:42,111][20381] Updated weights for policy 1, policy_version 132772 (0.0007) [2023-12-27 06:34:42,165][20381] Updated weights for policy 1, policy_version 132782 (0.0007) [2023-12-27 06:34:42,226][20381] Updated weights for policy 1, policy_version 132792 (0.0008) [2023-12-27 06:34:42,273][20347] Updated weights for policy 0, policy_version 133100 (0.0011) [2023-12-27 06:34:42,333][20347] Updated weights for policy 0, policy_version 133110 (0.0010) [2023-12-27 06:34:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 68091904. Throughput: 0: 10467.5, 1: 10538.9. Samples: 68085068. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:42,363][19170] Avg episode reward: [(0, '278.490'), (1, '1000.000')] [2023-12-27 06:34:42,400][20347] Updated weights for policy 0, policy_version 133120 (0.0011) [2023-12-27 06:34:42,949][20381] Updated weights for policy 1, policy_version 132802 (0.0008) [2023-12-27 06:34:42,982][20347] Updated weights for policy 0, policy_version 133130 (0.0010) [2023-12-27 06:34:42,991][20381] Updated weights for policy 1, policy_version 132812 (0.0010) [2023-12-27 06:34:43,028][20347] Updated weights for policy 0, policy_version 133140 (0.0008) [2023-12-27 06:34:43,039][20381] Updated weights for policy 1, policy_version 132822 (0.0010) [2023-12-27 06:34:43,077][20347] Updated weights for policy 0, policy_version 133150 (0.0008) [2023-12-27 06:34:43,122][20347] Updated weights for policy 0, policy_version 133160 (0.0008) [2023-12-27 06:34:43,754][20381] Updated weights for policy 1, policy_version 132832 (0.0010) [2023-12-27 06:34:43,799][20381] Updated weights for policy 1, policy_version 132842 (0.0010) [2023-12-27 06:34:43,827][20347] Updated weights for policy 0, policy_version 133170 (0.0006) [2023-12-27 06:34:43,844][20381] Updated weights for policy 1, policy_version 132852 (0.0010) [2023-12-27 06:34:43,873][20347] Updated weights for policy 0, policy_version 133180 (0.0006) [2023-12-27 06:34:43,919][20347] Updated weights for policy 0, policy_version 133190 (0.0007) [2023-12-27 06:34:44,569][20381] Updated weights for policy 1, policy_version 132862 (0.0010) [2023-12-27 06:34:44,625][20381] Updated weights for policy 1, policy_version 132872 (0.0009) [2023-12-27 06:34:44,627][20347] Updated weights for policy 0, policy_version 133200 (0.0006) [2023-12-27 06:34:44,681][20381] Updated weights for policy 1, policy_version 132882 (0.0011) [2023-12-27 06:34:44,683][20347] Updated weights for policy 0, policy_version 133210 (0.0006) [2023-12-27 06:34:44,733][20347] Updated weights for policy 0, policy_version 133220 (0.0005) [2023-12-27 06:34:45,355][20381] Updated weights for policy 1, policy_version 132892 (0.0010) [2023-12-27 06:34:45,383][20347] Updated weights for policy 0, policy_version 133230 (0.0007) [2023-12-27 06:34:45,410][20381] Updated weights for policy 1, policy_version 132902 (0.0007) [2023-12-27 06:34:45,441][20347] Updated weights for policy 0, policy_version 133240 (0.0008) [2023-12-27 06:34:45,458][20381] Updated weights for policy 1, policy_version 132912 (0.0005) [2023-12-27 06:34:45,493][20347] Updated weights for policy 0, policy_version 133250 (0.0006) [2023-12-27 06:34:46,067][20381] Updated weights for policy 1, policy_version 132922 (0.0006) [2023-12-27 06:34:46,127][20381] Updated weights for policy 1, policy_version 132932 (0.0007) [2023-12-27 06:34:46,186][20381] Updated weights for policy 1, policy_version 132942 (0.0007) [2023-12-27 06:34:46,188][20347] Updated weights for policy 0, policy_version 133260 (0.0006) [2023-12-27 06:34:46,241][20381] Updated weights for policy 1, policy_version 132952 (0.0008) [2023-12-27 06:34:46,243][20347] Updated weights for policy 0, policy_version 133270 (0.0007) [2023-12-27 06:34:46,295][20347] Updated weights for policy 0, policy_version 133280 (0.0006) [2023-12-27 06:34:46,833][20381] Updated weights for policy 1, policy_version 132962 (0.0005) [2023-12-27 06:34:46,889][20381] Updated weights for policy 1, policy_version 132972 (0.0005) [2023-12-27 06:34:46,933][20381] Updated weights for policy 1, policy_version 132982 (0.0005) [2023-12-27 06:34:46,998][20347] Updated weights for policy 0, policy_version 133290 (0.0006) [2023-12-27 06:34:47,053][20347] Updated weights for policy 0, policy_version 133300 (0.0007) [2023-12-27 06:34:47,101][20347] Updated weights for policy 0, policy_version 133310 (0.0008) [2023-12-27 06:34:47,155][20347] Updated weights for policy 0, policy_version 133320 (0.0007) [2023-12-27 06:34:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 68198400. Throughput: 0: 10450.7, 1: 10507.0. Samples: 68208824. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:47,363][19170] Avg episode reward: [(0, '117.050'), (1, '1000.000')] [2023-12-27 06:34:47,450][20381] Updated weights for policy 1, policy_version 132992 (0.0006) [2023-12-27 06:34:47,507][20381] Updated weights for policy 1, policy_version 133002 (0.0005) [2023-12-27 06:34:47,571][20381] Updated weights for policy 1, policy_version 133012 (0.0005) [2023-12-27 06:34:47,878][20347] Updated weights for policy 0, policy_version 133330 (0.0008) [2023-12-27 06:34:47,939][20347] Updated weights for policy 0, policy_version 133340 (0.0007) [2023-12-27 06:34:47,997][20347] Updated weights for policy 0, policy_version 133350 (0.0007) [2023-12-27 06:34:48,073][20381] Updated weights for policy 1, policy_version 133022 (0.0007) [2023-12-27 06:34:48,129][20381] Updated weights for policy 1, policy_version 133032 (0.0007) [2023-12-27 06:34:48,188][20381] Updated weights for policy 1, policy_version 133042 (0.0008) [2023-12-27 06:34:48,731][20347] Updated weights for policy 0, policy_version 133360 (0.0007) [2023-12-27 06:34:48,787][20347] Updated weights for policy 0, policy_version 133370 (0.0007) [2023-12-27 06:34:48,823][20381] Updated weights for policy 1, policy_version 133052 (0.0007) [2023-12-27 06:34:48,847][20347] Updated weights for policy 0, policy_version 133380 (0.0006) [2023-12-27 06:34:48,877][20381] Updated weights for policy 1, policy_version 133062 (0.0006) [2023-12-27 06:34:48,932][20381] Updated weights for policy 1, policy_version 133072 (0.0009) [2023-12-27 06:34:49,472][20347] Updated weights for policy 0, policy_version 133390 (0.0007) [2023-12-27 06:34:49,528][20347] Updated weights for policy 0, policy_version 133400 (0.0007) [2023-12-27 06:34:49,583][20347] Updated weights for policy 0, policy_version 133410 (0.0005) [2023-12-27 06:34:49,682][20381] Updated weights for policy 1, policy_version 133082 (0.0009) [2023-12-27 06:34:49,769][20381] Updated weights for policy 1, policy_version 133092 (0.0008) [2023-12-27 06:34:49,824][20381] Updated weights for policy 1, policy_version 133102 (0.0008) [2023-12-27 06:34:49,883][20381] Updated weights for policy 1, policy_version 133112 (0.0007) [2023-12-27 06:34:50,256][20347] Updated weights for policy 0, policy_version 133420 (0.0007) [2023-12-27 06:34:50,305][20347] Updated weights for policy 0, policy_version 133430 (0.0007) [2023-12-27 06:34:50,362][20347] Updated weights for policy 0, policy_version 133440 (0.0005) [2023-12-27 06:34:50,525][20381] Updated weights for policy 1, policy_version 133122 (0.0009) [2023-12-27 06:34:50,587][20381] Updated weights for policy 1, policy_version 133132 (0.0008) [2023-12-27 06:34:50,646][20381] Updated weights for policy 1, policy_version 133142 (0.0007) [2023-12-27 06:34:50,973][20347] Updated weights for policy 0, policy_version 133450 (0.0007) [2023-12-27 06:34:51,039][20347] Updated weights for policy 0, policy_version 133460 (0.0008) [2023-12-27 06:34:51,102][20347] Updated weights for policy 0, policy_version 133470 (0.0006) [2023-12-27 06:34:51,166][20347] Updated weights for policy 0, policy_version 133480 (0.0008) [2023-12-27 06:34:51,359][20381] Updated weights for policy 1, policy_version 133152 (0.0007) [2023-12-27 06:34:51,416][20381] Updated weights for policy 1, policy_version 133162 (0.0008) [2023-12-27 06:34:51,467][20381] Updated weights for policy 1, policy_version 133172 (0.0008) [2023-12-27 06:34:51,887][20347] Updated weights for policy 0, policy_version 133490 (0.0008) [2023-12-27 06:34:51,934][20347] Updated weights for policy 0, policy_version 133500 (0.0009) [2023-12-27 06:34:51,988][20347] Updated weights for policy 0, policy_version 133510 (0.0007) [2023-12-27 06:34:52,166][20381] Updated weights for policy 1, policy_version 133182 (0.0008) [2023-12-27 06:34:52,229][20381] Updated weights for policy 1, policy_version 133192 (0.0008) [2023-12-27 06:34:52,289][20381] Updated weights for policy 1, policy_version 133202 (0.0007) [2023-12-27 06:34:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.7, 300 sec: 20660.5). Total num frames: 68304896. Throughput: 0: 10388.7, 1: 10488.1. Samples: 68271744. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:52,363][19170] Avg episode reward: [(0, '129.670'), (1, '1000.000')] [2023-12-27 06:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000133208_34111488.pth... [2023-12-27 06:34:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000133512_34193408.pth... [2023-12-27 06:34:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000131960_33792000.pth [2023-12-27 06:34:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000132296_33882112.pth [2023-12-27 06:34:52,716][20347] Updated weights for policy 0, policy_version 133520 (0.0007) [2023-12-27 06:34:52,764][20347] Updated weights for policy 0, policy_version 133530 (0.0007) [2023-12-27 06:34:52,817][20347] Updated weights for policy 0, policy_version 133540 (0.0008) [2023-12-27 06:34:52,961][20381] Updated weights for policy 1, policy_version 133212 (0.0007) [2023-12-27 06:34:53,019][20381] Updated weights for policy 1, policy_version 133222 (0.0007) [2023-12-27 06:34:53,079][20381] Updated weights for policy 1, policy_version 133232 (0.0008) [2023-12-27 06:34:53,528][20347] Updated weights for policy 0, policy_version 133550 (0.0008) [2023-12-27 06:34:53,589][20347] Updated weights for policy 0, policy_version 133560 (0.0007) [2023-12-27 06:34:53,647][20347] Updated weights for policy 0, policy_version 133570 (0.0007) [2023-12-27 06:34:53,770][20381] Updated weights for policy 1, policy_version 133242 (0.0008) [2023-12-27 06:34:53,820][20381] Updated weights for policy 1, policy_version 133252 (0.0007) [2023-12-27 06:34:53,874][20381] Updated weights for policy 1, policy_version 133262 (0.0008) [2023-12-27 06:34:53,922][20381] Updated weights for policy 1, policy_version 133272 (0.0008) [2023-12-27 06:34:54,291][20347] Updated weights for policy 0, policy_version 133580 (0.0007) [2023-12-27 06:34:54,350][20347] Updated weights for policy 0, policy_version 133590 (0.0007) [2023-12-27 06:34:54,404][20347] Updated weights for policy 0, policy_version 133600 (0.0006) [2023-12-27 06:34:54,617][20381] Updated weights for policy 1, policy_version 133282 (0.0007) [2023-12-27 06:34:54,670][20381] Updated weights for policy 1, policy_version 133292 (0.0007) [2023-12-27 06:34:54,727][20381] Updated weights for policy 1, policy_version 133302 (0.0007) [2023-12-27 06:34:55,099][20347] Updated weights for policy 0, policy_version 133610 (0.0007) [2023-12-27 06:34:55,155][20347] Updated weights for policy 0, policy_version 133620 (0.0010) [2023-12-27 06:34:55,207][20347] Updated weights for policy 0, policy_version 133630 (0.0010) [2023-12-27 06:34:55,259][20347] Updated weights for policy 0, policy_version 133640 (0.0010) [2023-12-27 06:34:55,433][20381] Updated weights for policy 1, policy_version 133312 (0.0008) [2023-12-27 06:34:55,493][20381] Updated weights for policy 1, policy_version 133322 (0.0008) [2023-12-27 06:34:55,553][20381] Updated weights for policy 1, policy_version 133332 (0.0007) [2023-12-27 06:34:55,886][20347] Updated weights for policy 0, policy_version 133650 (0.0005) [2023-12-27 06:34:55,947][20347] Updated weights for policy 0, policy_version 133660 (0.0005) [2023-12-27 06:34:55,995][20347] Updated weights for policy 0, policy_version 133670 (0.0005) [2023-12-27 06:34:56,194][20381] Updated weights for policy 1, policy_version 133342 (0.0005) [2023-12-27 06:34:56,246][20381] Updated weights for policy 1, policy_version 133352 (0.0005) [2023-12-27 06:34:56,305][20381] Updated weights for policy 1, policy_version 133362 (0.0006) [2023-12-27 06:34:56,634][20347] Updated weights for policy 0, policy_version 133680 (0.0006) [2023-12-27 06:34:56,683][20347] Updated weights for policy 0, policy_version 133690 (0.0008) [2023-12-27 06:34:56,732][20347] Updated weights for policy 0, policy_version 133700 (0.0007) [2023-12-27 06:34:56,876][20381] Updated weights for policy 1, policy_version 133372 (0.0005) [2023-12-27 06:34:56,929][20381] Updated weights for policy 1, policy_version 133382 (0.0005) [2023-12-27 06:34:56,987][20381] Updated weights for policy 1, policy_version 133392 (0.0005) [2023-12-27 06:34:57,350][20347] Updated weights for policy 0, policy_version 133710 (0.0006) [2023-12-27 06:34:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 68403200. Throughput: 0: 10388.6, 1: 10464.4. Samples: 68394552. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:34:57,363][19170] Avg episode reward: [(0, '96.600'), (1, '1000.000')] [2023-12-27 06:34:57,405][20347] Updated weights for policy 0, policy_version 133720 (0.0007) [2023-12-27 06:34:57,452][20347] Updated weights for policy 0, policy_version 133730 (0.0007) [2023-12-27 06:34:57,660][20381] Updated weights for policy 1, policy_version 133402 (0.0006) [2023-12-27 06:34:57,707][20381] Updated weights for policy 1, policy_version 133412 (0.0006) [2023-12-27 06:34:57,763][20381] Updated weights for policy 1, policy_version 133422 (0.0008) [2023-12-27 06:34:57,818][20381] Updated weights for policy 1, policy_version 133432 (0.0007) [2023-12-27 06:34:58,129][20347] Updated weights for policy 0, policy_version 133740 (0.0007) [2023-12-27 06:34:58,192][20347] Updated weights for policy 0, policy_version 133750 (0.0007) [2023-12-27 06:34:58,253][20347] Updated weights for policy 0, policy_version 133760 (0.0008) [2023-12-27 06:34:58,483][20381] Updated weights for policy 1, policy_version 133442 (0.0007) [2023-12-27 06:34:58,548][20381] Updated weights for policy 1, policy_version 133452 (0.0007) [2023-12-27 06:34:58,607][20381] Updated weights for policy 1, policy_version 133462 (0.0008) [2023-12-27 06:34:58,976][20347] Updated weights for policy 0, policy_version 133770 (0.0007) [2023-12-27 06:34:59,037][20347] Updated weights for policy 0, policy_version 133780 (0.0008) [2023-12-27 06:34:59,099][20347] Updated weights for policy 0, policy_version 133790 (0.0007) [2023-12-27 06:34:59,158][20347] Updated weights for policy 0, policy_version 133800 (0.0008) [2023-12-27 06:34:59,394][20381] Updated weights for policy 1, policy_version 133472 (0.0008) [2023-12-27 06:34:59,447][20381] Updated weights for policy 1, policy_version 133482 (0.0009) [2023-12-27 06:34:59,495][20381] Updated weights for policy 1, policy_version 133492 (0.0009) [2023-12-27 06:34:59,766][20347] Updated weights for policy 0, policy_version 133810 (0.0006) [2023-12-27 06:34:59,830][20347] Updated weights for policy 0, policy_version 133820 (0.0008) [2023-12-27 06:34:59,883][20347] Updated weights for policy 0, policy_version 133830 (0.0007) [2023-12-27 06:35:00,185][20381] Updated weights for policy 1, policy_version 133502 (0.0007) [2023-12-27 06:35:00,236][20381] Updated weights for policy 1, policy_version 133512 (0.0005) [2023-12-27 06:35:00,285][20381] Updated weights for policy 1, policy_version 133522 (0.0005) [2023-12-27 06:35:00,509][20347] Updated weights for policy 0, policy_version 133840 (0.0007) [2023-12-27 06:35:00,558][20347] Updated weights for policy 0, policy_version 133850 (0.0007) [2023-12-27 06:35:00,605][20347] Updated weights for policy 0, policy_version 133860 (0.0007) [2023-12-27 06:35:00,914][20381] Updated weights for policy 1, policy_version 133532 (0.0006) [2023-12-27 06:35:00,978][20381] Updated weights for policy 1, policy_version 133542 (0.0005) [2023-12-27 06:35:01,044][20381] Updated weights for policy 1, policy_version 133552 (0.0007) [2023-12-27 06:35:01,292][20347] Updated weights for policy 0, policy_version 133870 (0.0007) [2023-12-27 06:35:01,358][20347] Updated weights for policy 0, policy_version 133880 (0.0008) [2023-12-27 06:35:01,416][20347] Updated weights for policy 0, policy_version 133890 (0.0008) [2023-12-27 06:35:01,755][20381] Updated weights for policy 1, policy_version 133562 (0.0011) [2023-12-27 06:35:01,800][20381] Updated weights for policy 1, policy_version 133572 (0.0010) [2023-12-27 06:35:01,846][20381] Updated weights for policy 1, policy_version 133582 (0.0010) [2023-12-27 06:35:01,893][20381] Updated weights for policy 1, policy_version 133592 (0.0009) [2023-12-27 06:35:02,121][20347] Updated weights for policy 0, policy_version 133900 (0.0007) [2023-12-27 06:35:02,176][20347] Updated weights for policy 0, policy_version 133910 (0.0007) [2023-12-27 06:35:02,232][20347] Updated weights for policy 0, policy_version 133920 (0.0007) [2023-12-27 06:35:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.2, 300 sec: 20660.5). Total num frames: 68509696. Throughput: 0: 10436.6, 1: 10453.2. Samples: 68518424. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:35:02,363][19170] Avg episode reward: [(0, '151.330'), (1, '1000.000')] [2023-12-27 06:35:02,582][20381] Updated weights for policy 1, policy_version 133602 (0.0005) [2023-12-27 06:35:02,645][20381] Updated weights for policy 1, policy_version 133612 (0.0005) [2023-12-27 06:35:02,704][20381] Updated weights for policy 1, policy_version 133622 (0.0007) [2023-12-27 06:35:02,947][20347] Updated weights for policy 0, policy_version 133930 (0.0008) [2023-12-27 06:35:02,992][20347] Updated weights for policy 0, policy_version 133940 (0.0009) [2023-12-27 06:35:03,049][20347] Updated weights for policy 0, policy_version 133950 (0.0008) [2023-12-27 06:35:03,096][20347] Updated weights for policy 0, policy_version 133960 (0.0007) [2023-12-27 06:35:03,248][20381] Updated weights for policy 1, policy_version 133632 (0.0010) [2023-12-27 06:35:03,302][20381] Updated weights for policy 1, policy_version 133642 (0.0010) [2023-12-27 06:35:03,359][20381] Updated weights for policy 1, policy_version 133652 (0.0010) [2023-12-27 06:35:03,760][20347] Updated weights for policy 0, policy_version 133970 (0.0010) [2023-12-27 06:35:03,813][20347] Updated weights for policy 0, policy_version 133980 (0.0010) [2023-12-27 06:35:03,872][20347] Updated weights for policy 0, policy_version 133990 (0.0010) [2023-12-27 06:35:03,972][20381] Updated weights for policy 1, policy_version 133662 (0.0007) [2023-12-27 06:35:04,023][20381] Updated weights for policy 1, policy_version 133672 (0.0008) [2023-12-27 06:35:04,081][20381] Updated weights for policy 1, policy_version 133682 (0.0008) [2023-12-27 06:35:04,453][20347] Updated weights for policy 0, policy_version 134000 (0.0009) [2023-12-27 06:35:04,512][20347] Updated weights for policy 0, policy_version 134010 (0.0010) [2023-12-27 06:35:04,570][20347] Updated weights for policy 0, policy_version 134020 (0.0010) [2023-12-27 06:35:04,707][20381] Updated weights for policy 1, policy_version 133692 (0.0011) [2023-12-27 06:35:04,756][20381] Updated weights for policy 1, policy_version 133702 (0.0010) [2023-12-27 06:35:04,801][20381] Updated weights for policy 1, policy_version 133712 (0.0010) [2023-12-27 06:35:05,180][20347] Updated weights for policy 0, policy_version 134030 (0.0008) [2023-12-27 06:35:05,238][20347] Updated weights for policy 0, policy_version 134040 (0.0006) [2023-12-27 06:35:05,301][20347] Updated weights for policy 0, policy_version 134050 (0.0007) [2023-12-27 06:35:05,512][20381] Updated weights for policy 1, policy_version 133722 (0.0010) [2023-12-27 06:35:05,557][20381] Updated weights for policy 1, policy_version 133732 (0.0010) [2023-12-27 06:35:05,612][20381] Updated weights for policy 1, policy_version 133742 (0.0010) [2023-12-27 06:35:05,672][20381] Updated weights for policy 1, policy_version 133752 (0.0010) [2023-12-27 06:35:05,930][20347] Updated weights for policy 0, policy_version 134060 (0.0008) [2023-12-27 06:35:05,992][20347] Updated weights for policy 0, policy_version 134070 (0.0008) [2023-12-27 06:35:06,045][20347] Updated weights for policy 0, policy_version 134080 (0.0007) [2023-12-27 06:35:06,321][20381] Updated weights for policy 1, policy_version 133762 (0.0011) [2023-12-27 06:35:06,370][20381] Updated weights for policy 1, policy_version 133772 (0.0010) [2023-12-27 06:35:06,423][20381] Updated weights for policy 1, policy_version 133782 (0.0011) [2023-12-27 06:35:06,745][20347] Updated weights for policy 0, policy_version 134090 (0.0007) [2023-12-27 06:35:06,797][20347] Updated weights for policy 0, policy_version 134100 (0.0006) [2023-12-27 06:35:06,858][20347] Updated weights for policy 0, policy_version 134110 (0.0010) [2023-12-27 06:35:06,913][20347] Updated weights for policy 0, policy_version 134120 (0.0010) [2023-12-27 06:35:07,113][20381] Updated weights for policy 1, policy_version 133792 (0.0010) [2023-12-27 06:35:07,173][20381] Updated weights for policy 1, policy_version 133802 (0.0011) [2023-12-27 06:35:07,222][20381] Updated weights for policy 1, policy_version 133812 (0.0011) [2023-12-27 06:35:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 68616192. Throughput: 0: 10454.7, 1: 10459.4. Samples: 68583496. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:35:07,363][19170] Avg episode reward: [(0, '60.240'), (1, '1000.000')] [2023-12-27 06:35:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000133816_34267136.pth... [2023-12-27 06:35:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000134120_34349056.pth... [2023-12-27 06:35:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000132600_33955840.pth [2023-12-27 06:35:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000132904_34037760.pth [2023-12-27 06:35:07,494][20347] Updated weights for policy 0, policy_version 134130 (0.0005) [2023-12-27 06:35:07,553][20347] Updated weights for policy 0, policy_version 134140 (0.0005) [2023-12-27 06:35:07,610][20347] Updated weights for policy 0, policy_version 134150 (0.0006) [2023-12-27 06:35:07,896][20381] Updated weights for policy 1, policy_version 133822 (0.0007) [2023-12-27 06:35:07,957][20381] Updated weights for policy 1, policy_version 133832 (0.0005) [2023-12-27 06:35:08,024][20381] Updated weights for policy 1, policy_version 133842 (0.0006) [2023-12-27 06:35:08,207][20347] Updated weights for policy 0, policy_version 134160 (0.0008) [2023-12-27 06:35:08,254][20347] Updated weights for policy 0, policy_version 134170 (0.0007) [2023-12-27 06:35:08,301][20347] Updated weights for policy 0, policy_version 134180 (0.0007) [2023-12-27 06:35:08,675][20381] Updated weights for policy 1, policy_version 133852 (0.0008) [2023-12-27 06:35:08,727][20381] Updated weights for policy 1, policy_version 133862 (0.0010) [2023-12-27 06:35:08,779][20381] Updated weights for policy 1, policy_version 133872 (0.0010) [2023-12-27 06:35:08,983][20347] Updated weights for policy 0, policy_version 134190 (0.0008) [2023-12-27 06:35:09,044][20347] Updated weights for policy 0, policy_version 134200 (0.0007) [2023-12-27 06:35:09,106][20347] Updated weights for policy 0, policy_version 134210 (0.0007) [2023-12-27 06:35:09,461][20381] Updated weights for policy 1, policy_version 133882 (0.0010) [2023-12-27 06:35:09,521][20381] Updated weights for policy 1, policy_version 133892 (0.0007) [2023-12-27 06:35:09,578][20381] Updated weights for policy 1, policy_version 133902 (0.0008) [2023-12-27 06:35:09,625][20381] Updated weights for policy 1, policy_version 133912 (0.0007) [2023-12-27 06:35:09,863][20347] Updated weights for policy 0, policy_version 134220 (0.0008) [2023-12-27 06:35:09,923][20347] Updated weights for policy 0, policy_version 134230 (0.0007) [2023-12-27 06:35:09,985][20347] Updated weights for policy 0, policy_version 134240 (0.0008) [2023-12-27 06:35:10,352][20381] Updated weights for policy 1, policy_version 133922 (0.0008) [2023-12-27 06:35:10,414][20381] Updated weights for policy 1, policy_version 133932 (0.0009) [2023-12-27 06:35:10,480][20381] Updated weights for policy 1, policy_version 133942 (0.0008) [2023-12-27 06:35:10,708][20347] Updated weights for policy 0, policy_version 134250 (0.0006) [2023-12-27 06:35:10,773][20347] Updated weights for policy 0, policy_version 134260 (0.0007) [2023-12-27 06:35:10,832][20347] Updated weights for policy 0, policy_version 134270 (0.0007) [2023-12-27 06:35:10,890][20347] Updated weights for policy 0, policy_version 134280 (0.0007) [2023-12-27 06:35:11,175][20381] Updated weights for policy 1, policy_version 133952 (0.0008) [2023-12-27 06:35:11,229][20381] Updated weights for policy 1, policy_version 133962 (0.0008) [2023-12-27 06:35:11,292][20381] Updated weights for policy 1, policy_version 133972 (0.0007) [2023-12-27 06:35:11,597][20347] Updated weights for policy 0, policy_version 134290 (0.0008) [2023-12-27 06:35:11,666][20347] Updated weights for policy 0, policy_version 134300 (0.0009) [2023-12-27 06:35:11,732][20347] Updated weights for policy 0, policy_version 134310 (0.0007) [2023-12-27 06:35:11,982][20381] Updated weights for policy 1, policy_version 133982 (0.0009) [2023-12-27 06:35:12,029][20381] Updated weights for policy 1, policy_version 133992 (0.0007) [2023-12-27 06:35:12,084][20381] Updated weights for policy 1, policy_version 134002 (0.0006) [2023-12-27 06:35:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 68714496. Throughput: 0: 10438.1, 1: 10440.0. Samples: 68706708. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:35:12,363][19170] Avg episode reward: [(0, '150.790'), (1, '1000.000')] [2023-12-27 06:35:12,462][20347] Updated weights for policy 0, policy_version 134320 (0.0007) [2023-12-27 06:35:12,524][20347] Updated weights for policy 0, policy_version 134330 (0.0007) [2023-12-27 06:35:12,581][20347] Updated weights for policy 0, policy_version 134340 (0.0008) [2023-12-27 06:35:12,777][20381] Updated weights for policy 1, policy_version 134012 (0.0011) [2023-12-27 06:35:12,835][20381] Updated weights for policy 1, policy_version 134022 (0.0005) [2023-12-27 06:35:12,895][20381] Updated weights for policy 1, policy_version 134032 (0.0007) [2023-12-27 06:35:13,228][20347] Updated weights for policy 0, policy_version 134350 (0.0008) [2023-12-27 06:35:13,286][20347] Updated weights for policy 0, policy_version 134360 (0.0008) [2023-12-27 06:35:13,335][20347] Updated weights for policy 0, policy_version 134370 (0.0006) [2023-12-27 06:35:13,524][20381] Updated weights for policy 1, policy_version 134042 (0.0009) [2023-12-27 06:35:13,573][20381] Updated weights for policy 1, policy_version 134052 (0.0009) [2023-12-27 06:35:13,635][20381] Updated weights for policy 1, policy_version 134062 (0.0011) [2023-12-27 06:35:13,697][20381] Updated weights for policy 1, policy_version 134072 (0.0010) [2023-12-27 06:35:14,023][20347] Updated weights for policy 0, policy_version 134380 (0.0006) [2023-12-27 06:35:14,078][20347] Updated weights for policy 0, policy_version 134390 (0.0007) [2023-12-27 06:35:14,139][20347] Updated weights for policy 0, policy_version 134400 (0.0007) [2023-12-27 06:35:14,327][20381] Updated weights for policy 1, policy_version 134082 (0.0011) [2023-12-27 06:35:14,379][20381] Updated weights for policy 1, policy_version 134092 (0.0011) [2023-12-27 06:35:14,430][20381] Updated weights for policy 1, policy_version 134102 (0.0005) [2023-12-27 06:35:14,855][20347] Updated weights for policy 0, policy_version 134410 (0.0007) [2023-12-27 06:35:14,920][20347] Updated weights for policy 0, policy_version 134420 (0.0007) [2023-12-27 06:35:14,979][20347] Updated weights for policy 0, policy_version 134430 (0.0007) [2023-12-27 06:35:15,034][20381] Updated weights for policy 1, policy_version 134112 (0.0008) [2023-12-27 06:35:15,038][20347] Updated weights for policy 0, policy_version 134440 (0.0006) [2023-12-27 06:35:15,090][20381] Updated weights for policy 1, policy_version 134122 (0.0008) [2023-12-27 06:35:15,143][20381] Updated weights for policy 1, policy_version 134132 (0.0008) [2023-12-27 06:35:15,723][20347] Updated weights for policy 0, policy_version 134450 (0.0008) [2023-12-27 06:35:15,776][20347] Updated weights for policy 0, policy_version 134460 (0.0008) [2023-12-27 06:35:15,824][20347] Updated weights for policy 0, policy_version 134470 (0.0007) [2023-12-27 06:35:15,869][20381] Updated weights for policy 1, policy_version 134142 (0.0007) [2023-12-27 06:35:15,919][20381] Updated weights for policy 1, policy_version 134152 (0.0006) [2023-12-27 06:35:15,988][20381] Updated weights for policy 1, policy_version 134162 (0.0006) [2023-12-27 06:35:16,481][20347] Updated weights for policy 0, policy_version 134480 (0.0007) [2023-12-27 06:35:16,545][20347] Updated weights for policy 0, policy_version 134490 (0.0007) [2023-12-27 06:35:16,612][20347] Updated weights for policy 0, policy_version 134500 (0.0007) [2023-12-27 06:35:16,665][20381] Updated weights for policy 1, policy_version 134172 (0.0006) [2023-12-27 06:35:16,715][20381] Updated weights for policy 1, policy_version 134182 (0.0007) [2023-12-27 06:35:16,771][20381] Updated weights for policy 1, policy_version 134192 (0.0008) [2023-12-27 06:35:17,316][20347] Updated weights for policy 0, policy_version 134510 (0.0008) [2023-12-27 06:35:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 68812800. Throughput: 0: 10323.4, 1: 10419.8. Samples: 68829092. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:35:17,363][19170] Avg episode reward: [(0, '75.810'), (1, '1000.000')] [2023-12-27 06:35:17,374][20381] Updated weights for policy 1, policy_version 134202 (0.0007) [2023-12-27 06:35:17,386][20347] Updated weights for policy 0, policy_version 134520 (0.0008) [2023-12-27 06:35:17,426][20381] Updated weights for policy 1, policy_version 134212 (0.0006) [2023-12-27 06:35:17,442][20347] Updated weights for policy 0, policy_version 134530 (0.0007) [2023-12-27 06:35:17,478][20381] Updated weights for policy 1, policy_version 134222 (0.0008) [2023-12-27 06:35:17,523][20381] Updated weights for policy 1, policy_version 134232 (0.0008) [2023-12-27 06:35:18,166][20347] Updated weights for policy 0, policy_version 134540 (0.0007) [2023-12-27 06:35:18,187][20381] Updated weights for policy 1, policy_version 134242 (0.0007) [2023-12-27 06:35:18,223][20347] Updated weights for policy 0, policy_version 134550 (0.0006) [2023-12-27 06:35:18,244][20381] Updated weights for policy 1, policy_version 134252 (0.0006) [2023-12-27 06:35:18,277][20347] Updated weights for policy 0, policy_version 134560 (0.0005) [2023-12-27 06:35:18,300][20381] Updated weights for policy 1, policy_version 134262 (0.0006) [2023-12-27 06:35:18,985][20347] Updated weights for policy 0, policy_version 134570 (0.0006) [2023-12-27 06:35:19,016][20381] Updated weights for policy 1, policy_version 134272 (0.0008) [2023-12-27 06:35:19,043][20347] Updated weights for policy 0, policy_version 134580 (0.0007) [2023-12-27 06:35:19,061][20381] Updated weights for policy 1, policy_version 134282 (0.0005) [2023-12-27 06:35:19,098][20347] Updated weights for policy 0, policy_version 134590 (0.0006) [2023-12-27 06:35:19,111][20381] Updated weights for policy 1, policy_version 134292 (0.0006) [2023-12-27 06:35:19,148][20347] Updated weights for policy 0, policy_version 134600 (0.0006) [2023-12-27 06:35:19,806][20347] Updated weights for policy 0, policy_version 134610 (0.0006) [2023-12-27 06:35:19,860][20381] Updated weights for policy 1, policy_version 134302 (0.0007) [2023-12-27 06:35:19,873][20347] Updated weights for policy 0, policy_version 134620 (0.0007) [2023-12-27 06:35:19,917][20381] Updated weights for policy 1, policy_version 134312 (0.0006) [2023-12-27 06:35:19,943][20347] Updated weights for policy 0, policy_version 134630 (0.0008) [2023-12-27 06:35:19,983][20381] Updated weights for policy 1, policy_version 134322 (0.0007) [2023-12-27 06:35:20,588][20381] Updated weights for policy 1, policy_version 134332 (0.0007) [2023-12-27 06:35:20,605][20347] Updated weights for policy 0, policy_version 134640 (0.0009) [2023-12-27 06:35:20,647][20381] Updated weights for policy 1, policy_version 134342 (0.0006) [2023-12-27 06:35:20,657][20347] Updated weights for policy 0, policy_version 134650 (0.0010) [2023-12-27 06:35:20,705][20381] Updated weights for policy 1, policy_version 134352 (0.0006) [2023-12-27 06:35:20,708][20347] Updated weights for policy 0, policy_version 134660 (0.0010) [2023-12-27 06:35:21,356][20347] Updated weights for policy 0, policy_version 134670 (0.0011) [2023-12-27 06:35:21,417][20347] Updated weights for policy 0, policy_version 134680 (0.0008) [2023-12-27 06:35:21,451][20381] Updated weights for policy 1, policy_version 134362 (0.0007) [2023-12-27 06:35:21,480][20347] Updated weights for policy 0, policy_version 134690 (0.0010) [2023-12-27 06:35:21,501][20381] Updated weights for policy 1, policy_version 134372 (0.0006) [2023-12-27 06:35:21,557][20381] Updated weights for policy 1, policy_version 134382 (0.0007) [2023-12-27 06:35:21,615][20381] Updated weights for policy 1, policy_version 134392 (0.0009) [2023-12-27 06:35:22,178][20347] Updated weights for policy 0, policy_version 134700 (0.0010) [2023-12-27 06:35:22,224][20347] Updated weights for policy 0, policy_version 134710 (0.0010) [2023-12-27 06:35:22,277][20347] Updated weights for policy 0, policy_version 134720 (0.0010) [2023-12-27 06:35:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 68919296. Throughput: 0: 10254.0, 1: 10366.0. Samples: 68888960. Policy #0 lag: (min: 24.0, avg: 54.2, max: 56.0) [2023-12-27 06:35:22,363][19170] Avg episode reward: [(0, '36.970'), (1, '1000.000')] [2023-12-27 06:35:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000134728_34504704.pth... [2023-12-27 06:35:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000133512_34193408.pth [2023-12-27 06:35:22,376][20381] Updated weights for policy 1, policy_version 134402 (0.0011) [2023-12-27 06:35:22,439][20381] Updated weights for policy 1, policy_version 134412 (0.0006) [2023-12-27 06:35:22,509][20381] Updated weights for policy 1, policy_version 134422 (0.0010) [2023-12-27 06:35:22,520][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000134424_34422784.pth... [2023-12-27 06:35:22,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000133208_34111488.pth [2023-12-27 06:35:22,954][20347] Updated weights for policy 0, policy_version 134730 (0.0010) [2023-12-27 06:35:23,016][20347] Updated weights for policy 0, policy_version 134740 (0.0005) [2023-12-27 06:35:23,067][20347] Updated weights for policy 0, policy_version 134750 (0.0005) [2023-12-27 06:35:23,118][20347] Updated weights for policy 0, policy_version 134760 (0.0005) [2023-12-27 06:35:23,142][20381] Updated weights for policy 1, policy_version 134432 (0.0008) [2023-12-27 06:35:23,193][20381] Updated weights for policy 1, policy_version 134442 (0.0007) [2023-12-27 06:35:23,250][20381] Updated weights for policy 1, policy_version 134452 (0.0008) [2023-12-27 06:35:23,765][20347] Updated weights for policy 0, policy_version 134770 (0.0007) [2023-12-27 06:35:23,827][20347] Updated weights for policy 0, policy_version 134780 (0.0008) [2023-12-27 06:35:23,875][20347] Updated weights for policy 0, policy_version 134790 (0.0009) [2023-12-27 06:35:23,941][20381] Updated weights for policy 1, policy_version 134462 (0.0008) [2023-12-27 06:35:23,998][20381] Updated weights for policy 1, policy_version 134472 (0.0007) [2023-12-27 06:35:24,048][20381] Updated weights for policy 1, policy_version 134482 (0.0005) [2023-12-27 06:35:24,512][20347] Updated weights for policy 0, policy_version 134800 (0.0006) [2023-12-27 06:35:24,563][20347] Updated weights for policy 0, policy_version 134810 (0.0008) [2023-12-27 06:35:24,610][20347] Updated weights for policy 0, policy_version 134820 (0.0005) [2023-12-27 06:35:24,666][20381] Updated weights for policy 1, policy_version 134492 (0.0006) [2023-12-27 06:35:24,726][20381] Updated weights for policy 1, policy_version 134502 (0.0008) [2023-12-27 06:35:24,783][20381] Updated weights for policy 1, policy_version 134512 (0.0008) [2023-12-27 06:35:25,274][20347] Updated weights for policy 0, policy_version 134830 (0.0008) [2023-12-27 06:35:25,318][20347] Updated weights for policy 0, policy_version 134840 (0.0007) [2023-12-27 06:35:25,367][20347] Updated weights for policy 0, policy_version 134850 (0.0008) [2023-12-27 06:35:25,453][20381] Updated weights for policy 1, policy_version 134522 (0.0008) [2023-12-27 06:35:25,511][20381] Updated weights for policy 1, policy_version 134532 (0.0008) [2023-12-27 06:35:25,561][20381] Updated weights for policy 1, policy_version 134542 (0.0007) [2023-12-27 06:35:25,611][20381] Updated weights for policy 1, policy_version 134552 (0.0008) [2023-12-27 06:35:26,063][20347] Updated weights for policy 0, policy_version 134860 (0.0007) [2023-12-27 06:35:26,125][20347] Updated weights for policy 0, policy_version 134870 (0.0007) [2023-12-27 06:35:26,181][20347] Updated weights for policy 0, policy_version 134880 (0.0008) [2023-12-27 06:35:26,336][20381] Updated weights for policy 1, policy_version 134562 (0.0007) [2023-12-27 06:35:26,397][20381] Updated weights for policy 1, policy_version 134572 (0.0008) [2023-12-27 06:35:26,462][20381] Updated weights for policy 1, policy_version 134582 (0.0009) [2023-12-27 06:35:26,829][20347] Updated weights for policy 0, policy_version 134890 (0.0008) [2023-12-27 06:35:26,888][20347] Updated weights for policy 0, policy_version 134900 (0.0008) [2023-12-27 06:35:26,945][20347] Updated weights for policy 0, policy_version 134910 (0.0008) [2023-12-27 06:35:26,990][20347] Updated weights for policy 0, policy_version 134920 (0.0008) [2023-12-27 06:35:27,154][20381] Updated weights for policy 1, policy_version 134592 (0.0006) [2023-12-27 06:35:27,203][20381] Updated weights for policy 1, policy_version 134602 (0.0007) [2023-12-27 06:35:27,253][20381] Updated weights for policy 1, policy_version 134612 (0.0008) [2023-12-27 06:35:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 69025792. Throughput: 0: 10277.8, 1: 10341.8. Samples: 69012956. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:27,364][19170] Avg episode reward: [(0, '34.820'), (1, '1000.000')] [2023-12-27 06:35:27,698][20347] Updated weights for policy 0, policy_version 134930 (0.0010) [2023-12-27 06:35:27,753][20347] Updated weights for policy 0, policy_version 134940 (0.0010) [2023-12-27 06:35:27,804][20347] Updated weights for policy 0, policy_version 134950 (0.0010) [2023-12-27 06:35:27,947][20381] Updated weights for policy 1, policy_version 134622 (0.0008) [2023-12-27 06:35:28,004][20381] Updated weights for policy 1, policy_version 134632 (0.0007) [2023-12-27 06:35:28,062][20381] Updated weights for policy 1, policy_version 134642 (0.0007) [2023-12-27 06:35:28,500][20347] Updated weights for policy 0, policy_version 134960 (0.0008) [2023-12-27 06:35:28,558][20347] Updated weights for policy 0, policy_version 134970 (0.0010) [2023-12-27 06:35:28,607][20347] Updated weights for policy 0, policy_version 134980 (0.0010) [2023-12-27 06:35:28,713][20381] Updated weights for policy 1, policy_version 134652 (0.0007) [2023-12-27 06:35:28,762][20381] Updated weights for policy 1, policy_version 134662 (0.0007) [2023-12-27 06:35:28,825][20381] Updated weights for policy 1, policy_version 134672 (0.0008) [2023-12-27 06:35:29,301][20347] Updated weights for policy 0, policy_version 134990 (0.0010) [2023-12-27 06:35:29,358][20347] Updated weights for policy 0, policy_version 135000 (0.0011) [2023-12-27 06:35:29,407][20347] Updated weights for policy 0, policy_version 135010 (0.0010) [2023-12-27 06:35:29,549][20381] Updated weights for policy 1, policy_version 134682 (0.0008) [2023-12-27 06:35:29,595][20381] Updated weights for policy 1, policy_version 134692 (0.0007) [2023-12-27 06:35:29,646][20381] Updated weights for policy 1, policy_version 134702 (0.0008) [2023-12-27 06:35:29,698][20381] Updated weights for policy 1, policy_version 134712 (0.0008) [2023-12-27 06:35:30,092][20347] Updated weights for policy 0, policy_version 135020 (0.0008) [2023-12-27 06:35:30,141][20347] Updated weights for policy 0, policy_version 135030 (0.0006) [2023-12-27 06:35:30,186][20347] Updated weights for policy 0, policy_version 135040 (0.0005) [2023-12-27 06:35:30,443][20381] Updated weights for policy 1, policy_version 134722 (0.0011) [2023-12-27 06:35:30,501][20381] Updated weights for policy 1, policy_version 134732 (0.0011) [2023-12-27 06:35:30,556][20381] Updated weights for policy 1, policy_version 134742 (0.0011) [2023-12-27 06:35:30,793][20347] Updated weights for policy 0, policy_version 135050 (0.0006) [2023-12-27 06:35:30,848][20347] Updated weights for policy 0, policy_version 135060 (0.0008) [2023-12-27 06:35:30,910][20347] Updated weights for policy 0, policy_version 135070 (0.0008) [2023-12-27 06:35:30,968][20347] Updated weights for policy 0, policy_version 135080 (0.0007) [2023-12-27 06:35:31,169][20381] Updated weights for policy 1, policy_version 134752 (0.0008) [2023-12-27 06:35:31,221][20381] Updated weights for policy 1, policy_version 134762 (0.0008) [2023-12-27 06:35:31,275][20381] Updated weights for policy 1, policy_version 134772 (0.0008) [2023-12-27 06:35:31,577][20347] Updated weights for policy 0, policy_version 135090 (0.0005) [2023-12-27 06:35:31,643][20347] Updated weights for policy 0, policy_version 135100 (0.0007) [2023-12-27 06:35:31,706][20347] Updated weights for policy 0, policy_version 135110 (0.0007) [2023-12-27 06:35:31,994][20381] Updated weights for policy 1, policy_version 134782 (0.0007) [2023-12-27 06:35:32,048][20381] Updated weights for policy 1, policy_version 134792 (0.0006) [2023-12-27 06:35:32,115][20381] Updated weights for policy 1, policy_version 134802 (0.0006) [2023-12-27 06:35:32,324][20347] Updated weights for policy 0, policy_version 135120 (0.0009) [2023-12-27 06:35:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 69124096. Throughput: 0: 10306.5, 1: 10296.5. Samples: 69135956. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:32,363][19170] Avg episode reward: [(0, '36.730'), (1, '943.880')] [2023-12-27 06:35:32,388][20347] Updated weights for policy 0, policy_version 135130 (0.0007) [2023-12-27 06:35:32,441][20347] Updated weights for policy 0, policy_version 135140 (0.0007) [2023-12-27 06:35:32,768][20381] Updated weights for policy 1, policy_version 134812 (0.0007) [2023-12-27 06:35:32,833][20381] Updated weights for policy 1, policy_version 134822 (0.0011) [2023-12-27 06:35:32,852][20292] KL-divergence is very high: 105.9952 [2023-12-27 06:35:32,901][20381] Updated weights for policy 1, policy_version 134832 (0.0011) [2023-12-27 06:35:33,082][20347] Updated weights for policy 0, policy_version 135150 (0.0006) [2023-12-27 06:35:33,142][20347] Updated weights for policy 0, policy_version 135160 (0.0005) [2023-12-27 06:35:33,199][20347] Updated weights for policy 0, policy_version 135170 (0.0005) [2023-12-27 06:35:33,600][20381] Updated weights for policy 1, policy_version 134842 (0.0009) [2023-12-27 06:35:33,657][20381] Updated weights for policy 1, policy_version 134852 (0.0005) [2023-12-27 06:35:33,712][20381] Updated weights for policy 1, policy_version 134862 (0.0005) [2023-12-27 06:35:33,765][20347] Updated weights for policy 0, policy_version 135180 (0.0005) [2023-12-27 06:35:33,772][20381] Updated weights for policy 1, policy_version 134872 (0.0005) [2023-12-27 06:35:33,810][20347] Updated weights for policy 0, policy_version 135190 (0.0005) [2023-12-27 06:35:33,868][20347] Updated weights for policy 0, policy_version 135200 (0.0005) [2023-12-27 06:35:34,309][20381] Updated weights for policy 1, policy_version 134882 (0.0007) [2023-12-27 06:35:34,358][20381] Updated weights for policy 1, policy_version 134892 (0.0008) [2023-12-27 06:35:34,404][20381] Updated weights for policy 1, policy_version 134902 (0.0005) [2023-12-27 06:35:34,527][20347] Updated weights for policy 0, policy_version 135210 (0.0006) [2023-12-27 06:35:34,595][20347] Updated weights for policy 0, policy_version 135220 (0.0007) [2023-12-27 06:35:34,653][20347] Updated weights for policy 0, policy_version 135230 (0.0007) [2023-12-27 06:35:34,707][20347] Updated weights for policy 0, policy_version 135240 (0.0007) [2023-12-27 06:35:35,078][20381] Updated weights for policy 1, policy_version 134912 (0.0009) [2023-12-27 06:35:35,130][20381] Updated weights for policy 1, policy_version 134922 (0.0010) [2023-12-27 06:35:35,189][20381] Updated weights for policy 1, policy_version 134932 (0.0011) [2023-12-27 06:35:35,400][20347] Updated weights for policy 0, policy_version 135250 (0.0008) [2023-12-27 06:35:35,460][20347] Updated weights for policy 0, policy_version 135260 (0.0007) [2023-12-27 06:35:35,513][20347] Updated weights for policy 0, policy_version 135270 (0.0008) [2023-12-27 06:35:35,792][20381] Updated weights for policy 1, policy_version 134942 (0.0006) [2023-12-27 06:35:35,840][20381] Updated weights for policy 1, policy_version 134952 (0.0005) [2023-12-27 06:35:35,892][20381] Updated weights for policy 1, policy_version 134962 (0.0009) [2023-12-27 06:35:36,180][20347] Updated weights for policy 0, policy_version 135280 (0.0007) [2023-12-27 06:35:36,241][20347] Updated weights for policy 0, policy_version 135290 (0.0008) [2023-12-27 06:35:36,306][20347] Updated weights for policy 0, policy_version 135300 (0.0007) [2023-12-27 06:35:36,533][20381] Updated weights for policy 1, policy_version 134972 (0.0009) [2023-12-27 06:35:36,587][20381] Updated weights for policy 1, policy_version 134982 (0.0008) [2023-12-27 06:35:36,639][20381] Updated weights for policy 1, policy_version 134992 (0.0008) [2023-12-27 06:35:36,995][20347] Updated weights for policy 0, policy_version 135310 (0.0006) [2023-12-27 06:35:37,056][20347] Updated weights for policy 0, policy_version 135320 (0.0009) [2023-12-27 06:35:37,114][20347] Updated weights for policy 0, policy_version 135330 (0.0008) [2023-12-27 06:35:37,235][20381] Updated weights for policy 1, policy_version 135002 (0.0007) [2023-12-27 06:35:37,283][20381] Updated weights for policy 1, policy_version 135012 (0.0007) [2023-12-27 06:35:37,330][20381] Updated weights for policy 1, policy_version 135022 (0.0007) [2023-12-27 06:35:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 69230592. Throughput: 0: 10346.0, 1: 10277.7. Samples: 69199812. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:37,363][19170] Avg episode reward: [(0, '35.140'), (1, '933.820')] [2023-12-27 06:35:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000135336_34660352.pth... [2023-12-27 06:35:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000134120_34349056.pth [2023-12-27 06:35:37,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000135032_34578432.pth... [2023-12-27 06:35:37,378][20381] Updated weights for policy 1, policy_version 135032 (0.0008) [2023-12-27 06:35:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000133816_34267136.pth [2023-12-27 06:35:37,752][20347] Updated weights for policy 0, policy_version 135340 (0.0007) [2023-12-27 06:35:37,802][20347] Updated weights for policy 0, policy_version 135350 (0.0005) [2023-12-27 06:35:37,855][20347] Updated weights for policy 0, policy_version 135360 (0.0005) [2023-12-27 06:35:38,087][20381] Updated weights for policy 1, policy_version 135042 (0.0011) [2023-12-27 06:35:38,148][20381] Updated weights for policy 1, policy_version 135052 (0.0010) [2023-12-27 06:35:38,210][20381] Updated weights for policy 1, policy_version 135062 (0.0010) [2023-12-27 06:35:38,490][20347] Updated weights for policy 0, policy_version 135370 (0.0006) [2023-12-27 06:35:38,535][20347] Updated weights for policy 0, policy_version 135380 (0.0007) [2023-12-27 06:35:38,589][20347] Updated weights for policy 0, policy_version 135390 (0.0007) [2023-12-27 06:35:38,644][20347] Updated weights for policy 0, policy_version 135400 (0.0007) [2023-12-27 06:35:38,903][20381] Updated weights for policy 1, policy_version 135072 (0.0008) [2023-12-27 06:35:38,972][20381] Updated weights for policy 1, policy_version 135082 (0.0011) [2023-12-27 06:35:39,037][20381] Updated weights for policy 1, policy_version 135092 (0.0011) [2023-12-27 06:35:39,319][20347] Updated weights for policy 0, policy_version 135410 (0.0007) [2023-12-27 06:35:39,381][20347] Updated weights for policy 0, policy_version 135420 (0.0008) [2023-12-27 06:35:39,445][20347] Updated weights for policy 0, policy_version 135430 (0.0008) [2023-12-27 06:35:39,672][20381] Updated weights for policy 1, policy_version 135102 (0.0009) [2023-12-27 06:35:39,727][20381] Updated weights for policy 1, policy_version 135112 (0.0007) [2023-12-27 06:35:39,779][20381] Updated weights for policy 1, policy_version 135122 (0.0007) [2023-12-27 06:35:40,172][20347] Updated weights for policy 0, policy_version 135440 (0.0009) [2023-12-27 06:35:40,229][20347] Updated weights for policy 0, policy_version 135450 (0.0009) [2023-12-27 06:35:40,285][20347] Updated weights for policy 0, policy_version 135460 (0.0008) [2023-12-27 06:35:40,440][20381] Updated weights for policy 1, policy_version 135132 (0.0007) [2023-12-27 06:35:40,501][20381] Updated weights for policy 1, policy_version 135142 (0.0008) [2023-12-27 06:35:40,551][20381] Updated weights for policy 1, policy_version 135152 (0.0008) [2023-12-27 06:35:40,947][20347] Updated weights for policy 0, policy_version 135470 (0.0007) [2023-12-27 06:35:40,999][20347] Updated weights for policy 0, policy_version 135480 (0.0007) [2023-12-27 06:35:41,062][20347] Updated weights for policy 0, policy_version 135490 (0.0008) [2023-12-27 06:35:41,272][20381] Updated weights for policy 1, policy_version 135162 (0.0007) [2023-12-27 06:35:41,336][20381] Updated weights for policy 1, policy_version 135172 (0.0008) [2023-12-27 06:35:41,404][20381] Updated weights for policy 1, policy_version 135182 (0.0008) [2023-12-27 06:35:41,459][20381] Updated weights for policy 1, policy_version 135192 (0.0008) [2023-12-27 06:35:41,798][20347] Updated weights for policy 0, policy_version 135500 (0.0007) [2023-12-27 06:35:41,858][20347] Updated weights for policy 0, policy_version 135510 (0.0007) [2023-12-27 06:35:41,908][20347] Updated weights for policy 0, policy_version 135520 (0.0007) [2023-12-27 06:35:42,155][20381] Updated weights for policy 1, policy_version 135202 (0.0005) [2023-12-27 06:35:42,204][20381] Updated weights for policy 1, policy_version 135212 (0.0005) [2023-12-27 06:35:42,257][20381] Updated weights for policy 1, policy_version 135222 (0.0006) [2023-12-27 06:35:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 69337088. Throughput: 0: 10349.3, 1: 10328.7. Samples: 69325064. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:42,363][19170] Avg episode reward: [(0, '34.280'), (1, '973.590')] [2023-12-27 06:35:42,649][20347] Updated weights for policy 0, policy_version 135530 (0.0007) [2023-12-27 06:35:42,707][20347] Updated weights for policy 0, policy_version 135540 (0.0007) [2023-12-27 06:35:42,774][20347] Updated weights for policy 0, policy_version 135550 (0.0007) [2023-12-27 06:35:42,835][20347] Updated weights for policy 0, policy_version 135560 (0.0007) [2023-12-27 06:35:42,907][20381] Updated weights for policy 1, policy_version 135232 (0.0006) [2023-12-27 06:35:42,968][20381] Updated weights for policy 1, policy_version 135242 (0.0009) [2023-12-27 06:35:43,023][20381] Updated weights for policy 1, policy_version 135252 (0.0010) [2023-12-27 06:35:43,463][20347] Updated weights for policy 0, policy_version 135570 (0.0007) [2023-12-27 06:35:43,521][20347] Updated weights for policy 0, policy_version 135580 (0.0008) [2023-12-27 06:35:43,580][20347] Updated weights for policy 0, policy_version 135590 (0.0008) [2023-12-27 06:35:43,667][20381] Updated weights for policy 1, policy_version 135262 (0.0010) [2023-12-27 06:35:43,722][20381] Updated weights for policy 1, policy_version 135272 (0.0010) [2023-12-27 06:35:43,760][20292] KL-divergence is very high: 192.5060 [2023-12-27 06:35:43,766][20292] KL-divergence is very high: 266.4086 [2023-12-27 06:35:43,771][20292] KL-divergence is very high: 133.7063 [2023-12-27 06:35:43,778][20381] Updated weights for policy 1, policy_version 135282 (0.0010) [2023-12-27 06:35:44,209][20347] Updated weights for policy 0, policy_version 135600 (0.0007) [2023-12-27 06:35:44,259][20347] Updated weights for policy 0, policy_version 135610 (0.0007) [2023-12-27 06:35:44,313][20347] Updated weights for policy 0, policy_version 135620 (0.0007) [2023-12-27 06:35:44,453][20381] Updated weights for policy 1, policy_version 135292 (0.0009) [2023-12-27 06:35:44,509][20381] Updated weights for policy 1, policy_version 135302 (0.0010) [2023-12-27 06:35:44,564][20381] Updated weights for policy 1, policy_version 135312 (0.0010) [2023-12-27 06:35:45,036][20347] Updated weights for policy 0, policy_version 135630 (0.0008) [2023-12-27 06:35:45,099][20347] Updated weights for policy 0, policy_version 135640 (0.0008) [2023-12-27 06:35:45,167][20347] Updated weights for policy 0, policy_version 135650 (0.0008) [2023-12-27 06:35:45,182][20381] Updated weights for policy 1, policy_version 135322 (0.0010) [2023-12-27 06:35:45,241][20381] Updated weights for policy 1, policy_version 135332 (0.0006) [2023-12-27 06:35:45,304][20381] Updated weights for policy 1, policy_version 135342 (0.0008) [2023-12-27 06:35:45,366][20381] Updated weights for policy 1, policy_version 135352 (0.0009) [2023-12-27 06:35:45,806][20347] Updated weights for policy 0, policy_version 135660 (0.0007) [2023-12-27 06:35:45,858][20347] Updated weights for policy 0, policy_version 135670 (0.0008) [2023-12-27 06:35:45,906][20347] Updated weights for policy 0, policy_version 135680 (0.0007) [2023-12-27 06:35:46,006][20381] Updated weights for policy 1, policy_version 135362 (0.0008) [2023-12-27 06:35:46,068][20381] Updated weights for policy 1, policy_version 135372 (0.0008) [2023-12-27 06:35:46,128][20381] Updated weights for policy 1, policy_version 135382 (0.0007) [2023-12-27 06:35:46,614][20347] Updated weights for policy 0, policy_version 135690 (0.0006) [2023-12-27 06:35:46,668][20347] Updated weights for policy 0, policy_version 135700 (0.0008) [2023-12-27 06:35:46,730][20347] Updated weights for policy 0, policy_version 135710 (0.0007) [2023-12-27 06:35:46,782][20381] Updated weights for policy 1, policy_version 135392 (0.0007) [2023-12-27 06:35:46,784][20347] Updated weights for policy 0, policy_version 135720 (0.0007) [2023-12-27 06:35:46,832][20381] Updated weights for policy 1, policy_version 135402 (0.0007) [2023-12-27 06:35:46,881][20381] Updated weights for policy 1, policy_version 135412 (0.0010) [2023-12-27 06:35:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 69435392. Throughput: 0: 10294.1, 1: 10350.7. Samples: 69447444. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:47,364][19170] Avg episode reward: [(0, '34.930'), (1, '949.010')] [2023-12-27 06:35:47,493][20347] Updated weights for policy 0, policy_version 135730 (0.0008) [2023-12-27 06:35:47,549][20347] Updated weights for policy 0, policy_version 135740 (0.0006) [2023-12-27 06:35:47,550][20381] Updated weights for policy 1, policy_version 135422 (0.0010) [2023-12-27 06:35:47,604][20347] Updated weights for policy 0, policy_version 135750 (0.0005) [2023-12-27 06:35:47,612][20381] Updated weights for policy 1, policy_version 135432 (0.0011) [2023-12-27 06:35:47,672][20381] Updated weights for policy 1, policy_version 135442 (0.0010) [2023-12-27 06:35:48,314][20347] Updated weights for policy 0, policy_version 135760 (0.0008) [2023-12-27 06:35:48,366][20381] Updated weights for policy 1, policy_version 135452 (0.0011) [2023-12-27 06:35:48,384][20347] Updated weights for policy 0, policy_version 135770 (0.0008) [2023-12-27 06:35:48,425][20381] Updated weights for policy 1, policy_version 135462 (0.0010) [2023-12-27 06:35:48,435][20347] Updated weights for policy 0, policy_version 135780 (0.0006) [2023-12-27 06:35:48,481][20381] Updated weights for policy 1, policy_version 135472 (0.0011) [2023-12-27 06:35:49,131][20347] Updated weights for policy 0, policy_version 135790 (0.0007) [2023-12-27 06:35:49,156][20381] Updated weights for policy 1, policy_version 135482 (0.0011) [2023-12-27 06:35:49,183][20347] Updated weights for policy 0, policy_version 135800 (0.0006) [2023-12-27 06:35:49,211][20381] Updated weights for policy 1, policy_version 135492 (0.0010) [2023-12-27 06:35:49,245][20347] Updated weights for policy 0, policy_version 135810 (0.0007) [2023-12-27 06:35:49,282][20381] Updated weights for policy 1, policy_version 135502 (0.0013) [2023-12-27 06:35:49,338][20381] Updated weights for policy 1, policy_version 135512 (0.0010) [2023-12-27 06:35:49,963][20347] Updated weights for policy 0, policy_version 135820 (0.0007) [2023-12-27 06:35:50,017][20347] Updated weights for policy 0, policy_version 135830 (0.0007) [2023-12-27 06:35:50,042][20381] Updated weights for policy 1, policy_version 135522 (0.0007) [2023-12-27 06:35:50,076][20347] Updated weights for policy 0, policy_version 135840 (0.0008) [2023-12-27 06:35:50,097][20381] Updated weights for policy 1, policy_version 135532 (0.0006) [2023-12-27 06:35:50,152][20381] Updated weights for policy 1, policy_version 135542 (0.0006) [2023-12-27 06:35:50,763][20381] Updated weights for policy 1, policy_version 135552 (0.0007) [2023-12-27 06:35:50,819][20381] Updated weights for policy 1, policy_version 135562 (0.0007) [2023-12-27 06:35:50,830][20347] Updated weights for policy 0, policy_version 135850 (0.0007) [2023-12-27 06:35:50,869][20381] Updated weights for policy 1, policy_version 135572 (0.0006) [2023-12-27 06:35:50,885][20347] Updated weights for policy 0, policy_version 135860 (0.0006) [2023-12-27 06:35:50,941][20347] Updated weights for policy 0, policy_version 135870 (0.0007) [2023-12-27 06:35:50,993][20347] Updated weights for policy 0, policy_version 135880 (0.0008) [2023-12-27 06:35:51,517][20381] Updated weights for policy 1, policy_version 135582 (0.0006) [2023-12-27 06:35:51,574][20381] Updated weights for policy 1, policy_version 135592 (0.0005) [2023-12-27 06:35:51,637][20381] Updated weights for policy 1, policy_version 135602 (0.0008) [2023-12-27 06:35:51,770][20347] Updated weights for policy 0, policy_version 135890 (0.0008) [2023-12-27 06:35:51,834][20347] Updated weights for policy 0, policy_version 135900 (0.0007) [2023-12-27 06:35:51,889][20347] Updated weights for policy 0, policy_version 135910 (0.0007) [2023-12-27 06:35:52,321][20381] Updated weights for policy 1, policy_version 135612 (0.0008) [2023-12-27 06:35:52,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20660.5). Total num frames: 69533696. Throughput: 0: 10235.5, 1: 10284.0. Samples: 69506876. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:52,363][19170] Avg episode reward: [(0, '39.630'), (1, '960.820')] [2023-12-27 06:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000135912_34807808.pth... [2023-12-27 06:35:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000134728_34504704.pth [2023-12-27 06:35:52,381][20381] Updated weights for policy 1, policy_version 135622 (0.0011) [2023-12-27 06:35:52,441][20381] Updated weights for policy 1, policy_version 135632 (0.0011) [2023-12-27 06:35:52,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000135640_34734080.pth... [2023-12-27 06:35:52,486][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000134424_34422784.pth [2023-12-27 06:35:52,570][20347] Updated weights for policy 0, policy_version 135920 (0.0006) [2023-12-27 06:35:52,643][20347] Updated weights for policy 0, policy_version 135930 (0.0005) [2023-12-27 06:35:52,710][20347] Updated weights for policy 0, policy_version 135940 (0.0006) [2023-12-27 06:35:53,122][20381] Updated weights for policy 1, policy_version 135642 (0.0011) [2023-12-27 06:35:53,176][20381] Updated weights for policy 1, policy_version 135652 (0.0010) [2023-12-27 06:35:53,228][20381] Updated weights for policy 1, policy_version 135662 (0.0010) [2023-12-27 06:35:53,279][20347] Updated weights for policy 0, policy_version 135950 (0.0007) [2023-12-27 06:35:53,289][20381] Updated weights for policy 1, policy_version 135672 (0.0011) [2023-12-27 06:35:53,333][20347] Updated weights for policy 0, policy_version 135960 (0.0005) [2023-12-27 06:35:53,386][20347] Updated weights for policy 0, policy_version 135970 (0.0005) [2023-12-27 06:35:53,953][20347] Updated weights for policy 0, policy_version 135980 (0.0007) [2023-12-27 06:35:53,964][20381] Updated weights for policy 1, policy_version 135682 (0.0010) [2023-12-27 06:35:54,007][20347] Updated weights for policy 0, policy_version 135990 (0.0010) [2023-12-27 06:35:54,025][20381] Updated weights for policy 1, policy_version 135692 (0.0010) [2023-12-27 06:35:54,059][20347] Updated weights for policy 0, policy_version 136000 (0.0010) [2023-12-27 06:35:54,076][20381] Updated weights for policy 1, policy_version 135702 (0.0010) [2023-12-27 06:35:54,721][20381] Updated weights for policy 1, policy_version 135712 (0.0009) [2023-12-27 06:35:54,726][20347] Updated weights for policy 0, policy_version 136010 (0.0010) [2023-12-27 06:35:54,772][20381] Updated weights for policy 1, policy_version 135722 (0.0010) [2023-12-27 06:35:54,780][20347] Updated weights for policy 0, policy_version 136020 (0.0010) [2023-12-27 06:35:54,824][20381] Updated weights for policy 1, policy_version 135732 (0.0010) [2023-12-27 06:35:54,834][20347] Updated weights for policy 0, policy_version 136030 (0.0010) [2023-12-27 06:35:54,888][20347] Updated weights for policy 0, policy_version 136040 (0.0010) [2023-12-27 06:35:55,445][20381] Updated weights for policy 1, policy_version 135742 (0.0007) [2023-12-27 06:35:55,505][20381] Updated weights for policy 1, policy_version 135752 (0.0005) [2023-12-27 06:35:55,555][20381] Updated weights for policy 1, policy_version 135762 (0.0006) [2023-12-27 06:35:55,596][20347] Updated weights for policy 0, policy_version 136050 (0.0009) [2023-12-27 06:35:55,658][20347] Updated weights for policy 0, policy_version 136060 (0.0006) [2023-12-27 06:35:55,712][20347] Updated weights for policy 0, policy_version 136070 (0.0008) [2023-12-27 06:35:56,184][20381] Updated weights for policy 1, policy_version 135772 (0.0011) [2023-12-27 06:35:56,239][20381] Updated weights for policy 1, policy_version 135782 (0.0010) [2023-12-27 06:35:56,262][20347] Updated weights for policy 0, policy_version 136080 (0.0008) [2023-12-27 06:35:56,296][20381] Updated weights for policy 1, policy_version 135792 (0.0010) [2023-12-27 06:35:56,322][20347] Updated weights for policy 0, policy_version 136090 (0.0009) [2023-12-27 06:35:56,370][20347] Updated weights for policy 0, policy_version 136100 (0.0010) [2023-12-27 06:35:56,928][20381] Updated weights for policy 1, policy_version 135802 (0.0009) [2023-12-27 06:35:56,988][20381] Updated weights for policy 1, policy_version 135812 (0.0005) [2023-12-27 06:35:57,008][20347] Updated weights for policy 0, policy_version 136110 (0.0010) [2023-12-27 06:35:57,042][20381] Updated weights for policy 1, policy_version 135822 (0.0005) [2023-12-27 06:35:57,065][20347] Updated weights for policy 0, policy_version 136120 (0.0010) [2023-12-27 06:35:57,089][20381] Updated weights for policy 1, policy_version 135832 (0.0005) [2023-12-27 06:35:57,110][20347] Updated weights for policy 0, policy_version 136130 (0.0010) [2023-12-27 06:35:57,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 69648384. Throughput: 0: 10241.9, 1: 10350.0. Samples: 69633340. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:35:57,363][19170] Avg episode reward: [(0, '43.600'), (1, '966.290')] [2023-12-27 06:35:57,665][20381] Updated weights for policy 1, policy_version 135842 (0.0011) [2023-12-27 06:35:57,718][20381] Updated weights for policy 1, policy_version 135852 (0.0011) [2023-12-27 06:35:57,772][20381] Updated weights for policy 1, policy_version 135862 (0.0011) [2023-12-27 06:35:57,773][20347] Updated weights for policy 0, policy_version 136140 (0.0009) [2023-12-27 06:35:57,825][20347] Updated weights for policy 0, policy_version 136150 (0.0007) [2023-12-27 06:35:57,882][20347] Updated weights for policy 0, policy_version 136160 (0.0007) [2023-12-27 06:35:58,499][20381] Updated weights for policy 1, policy_version 135872 (0.0009) [2023-12-27 06:35:58,554][20381] Updated weights for policy 1, policy_version 135882 (0.0008) [2023-12-27 06:35:58,621][20381] Updated weights for policy 1, policy_version 135892 (0.0008) [2023-12-27 06:35:58,673][20347] Updated weights for policy 0, policy_version 136170 (0.0008) [2023-12-27 06:35:58,731][20347] Updated weights for policy 0, policy_version 136180 (0.0008) [2023-12-27 06:35:58,796][20347] Updated weights for policy 0, policy_version 136190 (0.0008) [2023-12-27 06:35:58,857][20347] Updated weights for policy 0, policy_version 136200 (0.0008) [2023-12-27 06:35:59,389][20381] Updated weights for policy 1, policy_version 135902 (0.0008) [2023-12-27 06:35:59,442][20381] Updated weights for policy 1, policy_version 135912 (0.0008) [2023-12-27 06:35:59,491][20381] Updated weights for policy 1, policy_version 135922 (0.0006) [2023-12-27 06:35:59,505][20347] Updated weights for policy 0, policy_version 136210 (0.0007) [2023-12-27 06:35:59,555][20347] Updated weights for policy 0, policy_version 136220 (0.0007) [2023-12-27 06:35:59,610][20347] Updated weights for policy 0, policy_version 136230 (0.0007) [2023-12-27 06:36:00,165][20381] Updated weights for policy 1, policy_version 135932 (0.0006) [2023-12-27 06:36:00,223][20381] Updated weights for policy 1, policy_version 135942 (0.0008) [2023-12-27 06:36:00,290][20381] Updated weights for policy 1, policy_version 135952 (0.0008) [2023-12-27 06:36:00,344][20347] Updated weights for policy 0, policy_version 136240 (0.0008) [2023-12-27 06:36:00,403][20347] Updated weights for policy 0, policy_version 136250 (0.0008) [2023-12-27 06:36:00,463][20347] Updated weights for policy 0, policy_version 136260 (0.0007) [2023-12-27 06:36:00,974][20381] Updated weights for policy 1, policy_version 135962 (0.0007) [2023-12-27 06:36:01,025][20381] Updated weights for policy 1, policy_version 135972 (0.0010) [2023-12-27 06:36:01,074][20347] Updated weights for policy 0, policy_version 136270 (0.0006) [2023-12-27 06:36:01,084][20381] Updated weights for policy 1, policy_version 135982 (0.0010) [2023-12-27 06:36:01,130][20347] Updated weights for policy 0, policy_version 136280 (0.0006) [2023-12-27 06:36:01,141][20381] Updated weights for policy 1, policy_version 135992 (0.0010) [2023-12-27 06:36:01,189][20347] Updated weights for policy 0, policy_version 136290 (0.0008) [2023-12-27 06:36:01,850][20381] Updated weights for policy 1, policy_version 136002 (0.0008) [2023-12-27 06:36:01,881][20347] Updated weights for policy 0, policy_version 136300 (0.0007) [2023-12-27 06:36:01,903][20381] Updated weights for policy 1, policy_version 136012 (0.0008) [2023-12-27 06:36:01,934][20347] Updated weights for policy 0, policy_version 136310 (0.0007) [2023-12-27 06:36:01,952][20381] Updated weights for policy 1, policy_version 136022 (0.0005) [2023-12-27 06:36:01,992][20347] Updated weights for policy 0, policy_version 136320 (0.0007) [2023-12-27 06:36:02,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 69746688. Throughput: 0: 10277.2, 1: 10310.1. Samples: 69755524. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:36:02,363][19170] Avg episode reward: [(0, '46.510'), (1, '914.190')] [2023-12-27 06:36:02,678][20381] Updated weights for policy 1, policy_version 136032 (0.0008) [2023-12-27 06:36:02,690][20347] Updated weights for policy 0, policy_version 136330 (0.0007) [2023-12-27 06:36:02,732][20381] Updated weights for policy 1, policy_version 136042 (0.0008) [2023-12-27 06:36:02,748][20347] Updated weights for policy 0, policy_version 136340 (0.0006) [2023-12-27 06:36:02,782][20381] Updated weights for policy 1, policy_version 136052 (0.0006) [2023-12-27 06:36:02,803][20347] Updated weights for policy 0, policy_version 136350 (0.0006) [2023-12-27 06:36:02,858][20347] Updated weights for policy 0, policy_version 136360 (0.0008) [2023-12-27 06:36:03,451][20381] Updated weights for policy 1, policy_version 136062 (0.0008) [2023-12-27 06:36:03,500][20381] Updated weights for policy 1, policy_version 136072 (0.0007) [2023-12-27 06:36:03,542][20347] Updated weights for policy 0, policy_version 136370 (0.0006) [2023-12-27 06:36:03,544][20381] Updated weights for policy 1, policy_version 136082 (0.0005) [2023-12-27 06:36:03,589][20347] Updated weights for policy 0, policy_version 136380 (0.0005) [2023-12-27 06:36:03,652][20347] Updated weights for policy 0, policy_version 136390 (0.0008) [2023-12-27 06:36:04,256][20381] Updated weights for policy 1, policy_version 136092 (0.0006) [2023-12-27 06:36:04,316][20381] Updated weights for policy 1, policy_version 136102 (0.0008) [2023-12-27 06:36:04,371][20347] Updated weights for policy 0, policy_version 136400 (0.0007) [2023-12-27 06:36:04,378][20381] Updated weights for policy 1, policy_version 136112 (0.0007) [2023-12-27 06:36:04,434][20347] Updated weights for policy 0, policy_version 136410 (0.0006) [2023-12-27 06:36:04,495][20347] Updated weights for policy 0, policy_version 136420 (0.0008) [2023-12-27 06:36:05,081][20381] Updated weights for policy 1, policy_version 136122 (0.0007) [2023-12-27 06:36:05,133][20381] Updated weights for policy 1, policy_version 136132 (0.0008) [2023-12-27 06:36:05,162][20347] Updated weights for policy 0, policy_version 136430 (0.0007) [2023-12-27 06:36:05,198][20381] Updated weights for policy 1, policy_version 136142 (0.0008) [2023-12-27 06:36:05,208][20347] Updated weights for policy 0, policy_version 136440 (0.0005) [2023-12-27 06:36:05,251][20381] Updated weights for policy 1, policy_version 136152 (0.0007) [2023-12-27 06:36:05,264][20347] Updated weights for policy 0, policy_version 136450 (0.0005) [2023-12-27 06:36:05,832][20347] Updated weights for policy 0, policy_version 136460 (0.0006) [2023-12-27 06:36:05,838][20381] Updated weights for policy 1, policy_version 136162 (0.0007) [2023-12-27 06:36:05,887][20381] Updated weights for policy 1, policy_version 136172 (0.0006) [2023-12-27 06:36:05,892][20347] Updated weights for policy 0, policy_version 136470 (0.0007) [2023-12-27 06:36:05,946][20381] Updated weights for policy 1, policy_version 136182 (0.0006) [2023-12-27 06:36:05,953][20347] Updated weights for policy 0, policy_version 136480 (0.0008) [2023-12-27 06:36:06,591][20381] Updated weights for policy 1, policy_version 136192 (0.0005) [2023-12-27 06:36:06,603][20347] Updated weights for policy 0, policy_version 136490 (0.0007) [2023-12-27 06:36:06,657][20381] Updated weights for policy 1, policy_version 136202 (0.0006) [2023-12-27 06:36:06,668][20347] Updated weights for policy 0, policy_version 136500 (0.0008) [2023-12-27 06:36:06,722][20381] Updated weights for policy 1, policy_version 136212 (0.0008) [2023-12-27 06:36:06,733][20347] Updated weights for policy 0, policy_version 136510 (0.0006) [2023-12-27 06:36:06,790][20347] Updated weights for policy 0, policy_version 136520 (0.0006) [2023-12-27 06:36:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 69844992. Throughput: 0: 10285.3, 1: 10315.6. Samples: 69816004. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:36:07,363][19170] Avg episode reward: [(0, '83.340'), (1, '919.640')] [2023-12-27 06:36:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000136216_34881536.pth... [2023-12-27 06:36:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000135032_34578432.pth [2023-12-27 06:36:07,392][20347] Updated weights for policy 0, policy_version 136530 (0.0005) [2023-12-27 06:36:07,432][20381] Updated weights for policy 1, policy_version 136222 (0.0009) [2023-12-27 06:36:07,454][20347] Updated weights for policy 0, policy_version 136540 (0.0007) [2023-12-27 06:36:07,488][20381] Updated weights for policy 1, policy_version 136232 (0.0006) [2023-12-27 06:36:07,517][20347] Updated weights for policy 0, policy_version 136550 (0.0010) [2023-12-27 06:36:07,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000136552_34971648.pth... [2023-12-27 06:36:07,528][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000135336_34660352.pth [2023-12-27 06:36:07,538][20381] Updated weights for policy 1, policy_version 136242 (0.0005) [2023-12-27 06:36:08,066][20381] Updated weights for policy 1, policy_version 136252 (0.0005) [2023-12-27 06:36:08,122][20381] Updated weights for policy 1, policy_version 136262 (0.0005) [2023-12-27 06:36:08,145][20347] Updated weights for policy 0, policy_version 136560 (0.0010) [2023-12-27 06:36:08,181][20381] Updated weights for policy 1, policy_version 136272 (0.0005) [2023-12-27 06:36:08,200][20347] Updated weights for policy 0, policy_version 136570 (0.0010) [2023-12-27 06:36:08,255][20347] Updated weights for policy 0, policy_version 136580 (0.0010) [2023-12-27 06:36:08,724][20381] Updated weights for policy 1, policy_version 136282 (0.0005) [2023-12-27 06:36:08,783][20381] Updated weights for policy 1, policy_version 136292 (0.0007) [2023-12-27 06:36:08,842][20381] Updated weights for policy 1, policy_version 136302 (0.0006) [2023-12-27 06:36:08,898][20381] Updated weights for policy 1, policy_version 136312 (0.0006) [2023-12-27 06:36:08,963][20347] Updated weights for policy 0, policy_version 136590 (0.0010) [2023-12-27 06:36:09,028][20347] Updated weights for policy 0, policy_version 136600 (0.0010) [2023-12-27 06:36:09,077][20347] Updated weights for policy 0, policy_version 136610 (0.0010) [2023-12-27 06:36:09,493][20381] Updated weights for policy 1, policy_version 136322 (0.0011) [2023-12-27 06:36:09,548][20381] Updated weights for policy 1, policy_version 136332 (0.0010) [2023-12-27 06:36:09,607][20381] Updated weights for policy 1, policy_version 136342 (0.0005) [2023-12-27 06:36:09,761][20347] Updated weights for policy 0, policy_version 136620 (0.0010) [2023-12-27 06:36:09,825][20347] Updated weights for policy 0, policy_version 136630 (0.0008) [2023-12-27 06:36:09,889][20347] Updated weights for policy 0, policy_version 136640 (0.0010) [2023-12-27 06:36:10,256][20381] Updated weights for policy 1, policy_version 136352 (0.0005) [2023-12-27 06:36:10,321][20381] Updated weights for policy 1, policy_version 136362 (0.0006) [2023-12-27 06:36:10,389][20381] Updated weights for policy 1, policy_version 136372 (0.0007) [2023-12-27 06:36:10,578][20347] Updated weights for policy 0, policy_version 136650 (0.0010) [2023-12-27 06:36:10,637][20347] Updated weights for policy 0, policy_version 136660 (0.0010) [2023-12-27 06:36:10,701][20347] Updated weights for policy 0, policy_version 136670 (0.0010) [2023-12-27 06:36:10,756][20347] Updated weights for policy 0, policy_version 136680 (0.0010) [2023-12-27 06:36:11,000][20381] Updated weights for policy 1, policy_version 136382 (0.0009) [2023-12-27 06:36:11,064][20381] Updated weights for policy 1, policy_version 136392 (0.0009) [2023-12-27 06:36:11,120][20381] Updated weights for policy 1, policy_version 136402 (0.0008) [2023-12-27 06:36:11,448][20347] Updated weights for policy 0, policy_version 136690 (0.0006) [2023-12-27 06:36:11,509][20347] Updated weights for policy 0, policy_version 136700 (0.0010) [2023-12-27 06:36:11,559][20347] Updated weights for policy 0, policy_version 136710 (0.0010) [2023-12-27 06:36:11,796][20381] Updated weights for policy 1, policy_version 136412 (0.0009) [2023-12-27 06:36:11,857][20381] Updated weights for policy 1, policy_version 136422 (0.0008) [2023-12-27 06:36:11,916][20381] Updated weights for policy 1, policy_version 136432 (0.0010) [2023-12-27 06:36:12,252][20347] Updated weights for policy 0, policy_version 136720 (0.0008) [2023-12-27 06:36:12,310][20347] Updated weights for policy 0, policy_version 136730 (0.0008) [2023-12-27 06:36:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 69951488. Throughput: 0: 10285.9, 1: 10423.8. Samples: 69944888. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:36:12,363][19170] Avg episode reward: [(0, '155.540'), (1, '1000.000')] [2023-12-27 06:36:12,373][20347] Updated weights for policy 0, policy_version 136740 (0.0007) [2023-12-27 06:36:12,646][20381] Updated weights for policy 1, policy_version 136442 (0.0011) [2023-12-27 06:36:12,702][20381] Updated weights for policy 1, policy_version 136452 (0.0011) [2023-12-27 06:36:12,762][20381] Updated weights for policy 1, policy_version 136462 (0.0009) [2023-12-27 06:36:12,816][20381] Updated weights for policy 1, policy_version 136472 (0.0006) [2023-12-27 06:36:13,129][20347] Updated weights for policy 0, policy_version 136750 (0.0006) [2023-12-27 06:36:13,181][20347] Updated weights for policy 0, policy_version 136760 (0.0006) [2023-12-27 06:36:13,230][20347] Updated weights for policy 0, policy_version 136770 (0.0009) [2023-12-27 06:36:13,334][20381] Updated weights for policy 1, policy_version 136482 (0.0005) [2023-12-27 06:36:13,383][20381] Updated weights for policy 1, policy_version 136492 (0.0005) [2023-12-27 06:36:13,434][20381] Updated weights for policy 1, policy_version 136502 (0.0005) [2023-12-27 06:36:13,965][20347] Updated weights for policy 0, policy_version 136780 (0.0009) [2023-12-27 06:36:13,971][20381] Updated weights for policy 1, policy_version 136512 (0.0005) [2023-12-27 06:36:14,025][20347] Updated weights for policy 0, policy_version 136790 (0.0007) [2023-12-27 06:36:14,032][20381] Updated weights for policy 1, policy_version 136522 (0.0005) [2023-12-27 06:36:14,090][20347] Updated weights for policy 0, policy_version 136800 (0.0008) [2023-12-27 06:36:14,091][20381] Updated weights for policy 1, policy_version 136532 (0.0006) [2023-12-27 06:36:14,660][20381] Updated weights for policy 1, policy_version 136542 (0.0007) [2023-12-27 06:36:14,707][20347] Updated weights for policy 0, policy_version 136810 (0.0008) [2023-12-27 06:36:14,717][20381] Updated weights for policy 1, policy_version 136552 (0.0007) [2023-12-27 06:36:14,760][20347] Updated weights for policy 0, policy_version 136820 (0.0006) [2023-12-27 06:36:14,779][20381] Updated weights for policy 1, policy_version 136562 (0.0006) [2023-12-27 06:36:14,823][20347] Updated weights for policy 0, policy_version 136830 (0.0007) [2023-12-27 06:36:14,887][20347] Updated weights for policy 0, policy_version 136840 (0.0007) [2023-12-27 06:36:15,402][20381] Updated weights for policy 1, policy_version 136572 (0.0007) [2023-12-27 06:36:15,455][20381] Updated weights for policy 1, policy_version 136582 (0.0008) [2023-12-27 06:36:15,506][20381] Updated weights for policy 1, policy_version 136592 (0.0008) [2023-12-27 06:36:15,578][20347] Updated weights for policy 0, policy_version 136850 (0.0006) [2023-12-27 06:36:15,633][20347] Updated weights for policy 0, policy_version 136860 (0.0008) [2023-12-27 06:36:15,689][20347] Updated weights for policy 0, policy_version 136870 (0.0007) [2023-12-27 06:36:16,226][20381] Updated weights for policy 1, policy_version 136602 (0.0008) [2023-12-27 06:36:16,285][20381] Updated weights for policy 1, policy_version 136612 (0.0007) [2023-12-27 06:36:16,344][20381] Updated weights for policy 1, policy_version 136622 (0.0007) [2023-12-27 06:36:16,390][20347] Updated weights for policy 0, policy_version 136880 (0.0010) [2023-12-27 06:36:16,392][20381] Updated weights for policy 1, policy_version 136632 (0.0005) [2023-12-27 06:36:16,442][20347] Updated weights for policy 0, policy_version 136890 (0.0010) [2023-12-27 06:36:16,500][20347] Updated weights for policy 0, policy_version 136900 (0.0010) [2023-12-27 06:36:17,102][20347] Updated weights for policy 0, policy_version 136910 (0.0007) [2023-12-27 06:36:17,105][20381] Updated weights for policy 1, policy_version 136642 (0.0007) [2023-12-27 06:36:17,156][20381] Updated weights for policy 1, policy_version 136652 (0.0006) [2023-12-27 06:36:17,159][20347] Updated weights for policy 0, policy_version 136920 (0.0009) [2023-12-27 06:36:17,208][20381] Updated weights for policy 1, policy_version 136662 (0.0008) [2023-12-27 06:36:17,216][20347] Updated weights for policy 0, policy_version 136930 (0.0009) [2023-12-27 06:36:17,362][19170] Fps is (10 sec: 22118.8, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 70066176. Throughput: 0: 10236.3, 1: 10524.9. Samples: 70070208. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:36:17,363][19170] Avg episode reward: [(0, '245.590'), (1, '1000.000')] [2023-12-27 06:36:17,818][20347] Updated weights for policy 0, policy_version 136940 (0.0009) [2023-12-27 06:36:17,863][20381] Updated weights for policy 1, policy_version 136672 (0.0009) [2023-12-27 06:36:17,866][20347] Updated weights for policy 0, policy_version 136950 (0.0010) [2023-12-27 06:36:17,909][20381] Updated weights for policy 1, policy_version 136682 (0.0006) [2023-12-27 06:36:17,914][20347] Updated weights for policy 0, policy_version 136960 (0.0010) [2023-12-27 06:36:17,955][20381] Updated weights for policy 1, policy_version 136692 (0.0006) [2023-12-27 06:36:18,547][20347] Updated weights for policy 0, policy_version 136970 (0.0010) [2023-12-27 06:36:18,614][20347] Updated weights for policy 0, policy_version 136980 (0.0010) [2023-12-27 06:36:18,635][20381] Updated weights for policy 1, policy_version 136702 (0.0006) [2023-12-27 06:36:18,673][20347] Updated weights for policy 0, policy_version 136990 (0.0011) [2023-12-27 06:36:18,700][20381] Updated weights for policy 1, policy_version 136712 (0.0005) [2023-12-27 06:36:18,729][20347] Updated weights for policy 0, policy_version 137000 (0.0010) [2023-12-27 06:36:18,757][20381] Updated weights for policy 1, policy_version 136722 (0.0005) [2023-12-27 06:36:19,397][20347] Updated weights for policy 0, policy_version 137010 (0.0007) [2023-12-27 06:36:19,438][20381] Updated weights for policy 1, policy_version 136732 (0.0007) [2023-12-27 06:36:19,448][20347] Updated weights for policy 0, policy_version 137020 (0.0007) [2023-12-27 06:36:19,502][20381] Updated weights for policy 1, policy_version 136742 (0.0006) [2023-12-27 06:36:19,507][20347] Updated weights for policy 0, policy_version 137030 (0.0007) [2023-12-27 06:36:19,560][20381] Updated weights for policy 1, policy_version 136752 (0.0009) [2023-12-27 06:36:20,195][20381] Updated weights for policy 1, policy_version 136762 (0.0008) [2023-12-27 06:36:20,217][20347] Updated weights for policy 0, policy_version 137040 (0.0007) [2023-12-27 06:36:20,256][20381] Updated weights for policy 1, policy_version 136772 (0.0006) [2023-12-27 06:36:20,266][20347] Updated weights for policy 0, policy_version 137050 (0.0005) [2023-12-27 06:36:20,323][20381] Updated weights for policy 1, policy_version 136782 (0.0006) [2023-12-27 06:36:20,328][20347] Updated weights for policy 0, policy_version 137060 (0.0007) [2023-12-27 06:36:20,379][20381] Updated weights for policy 1, policy_version 136792 (0.0007) [2023-12-27 06:36:21,042][20347] Updated weights for policy 0, policy_version 137070 (0.0007) [2023-12-27 06:36:21,068][20381] Updated weights for policy 1, policy_version 136802 (0.0007) [2023-12-27 06:36:21,104][20347] Updated weights for policy 0, policy_version 137080 (0.0007) [2023-12-27 06:36:21,132][20381] Updated weights for policy 1, policy_version 136812 (0.0007) [2023-12-27 06:36:21,166][20347] Updated weights for policy 0, policy_version 137090 (0.0006) [2023-12-27 06:36:21,199][20381] Updated weights for policy 1, policy_version 136822 (0.0010) [2023-12-27 06:36:21,888][20347] Updated weights for policy 0, policy_version 137100 (0.0007) [2023-12-27 06:36:21,910][20381] Updated weights for policy 1, policy_version 136832 (0.0011) [2023-12-27 06:36:21,946][20347] Updated weights for policy 0, policy_version 137110 (0.0007) [2023-12-27 06:36:21,973][20381] Updated weights for policy 1, policy_version 136842 (0.0011) [2023-12-27 06:36:22,006][20347] Updated weights for policy 0, policy_version 137120 (0.0005) [2023-12-27 06:36:22,033][20381] Updated weights for policy 1, policy_version 136852 (0.0011) [2023-12-27 06:36:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 70164480. Throughput: 0: 10215.4, 1: 10522.3. Samples: 70133008. Policy #0 lag: (min: 15.0, avg: 17.2, max: 47.0) [2023-12-27 06:36:22,363][19170] Avg episode reward: [(0, '361.920'), (1, '1000.000')] [2023-12-27 06:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000136856_35045376.pth... [2023-12-27 06:36:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000137128_35119104.pth... [2023-12-27 06:36:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000135912_34807808.pth [2023-12-27 06:36:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000135640_34734080.pth [2023-12-27 06:36:22,712][20347] Updated weights for policy 0, policy_version 137130 (0.0007) [2023-12-27 06:36:22,732][20381] Updated weights for policy 1, policy_version 136862 (0.0011) [2023-12-27 06:36:22,771][20347] Updated weights for policy 0, policy_version 137140 (0.0005) [2023-12-27 06:36:22,784][20381] Updated weights for policy 1, policy_version 136872 (0.0010) [2023-12-27 06:36:22,828][20347] Updated weights for policy 0, policy_version 137150 (0.0007) [2023-12-27 06:36:22,845][20381] Updated weights for policy 1, policy_version 136882 (0.0011) [2023-12-27 06:36:22,881][20347] Updated weights for policy 0, policy_version 137160 (0.0007) [2023-12-27 06:36:23,544][20381] Updated weights for policy 1, policy_version 136892 (0.0010) [2023-12-27 06:36:23,554][20347] Updated weights for policy 0, policy_version 137170 (0.0007) [2023-12-27 06:36:23,589][20381] Updated weights for policy 1, policy_version 136902 (0.0008) [2023-12-27 06:36:23,611][20347] Updated weights for policy 0, policy_version 137180 (0.0005) [2023-12-27 06:36:23,644][20381] Updated weights for policy 1, policy_version 136912 (0.0007) [2023-12-27 06:36:23,661][20347] Updated weights for policy 0, policy_version 137190 (0.0006) [2023-12-27 06:36:24,317][20347] Updated weights for policy 0, policy_version 137200 (0.0007) [2023-12-27 06:36:24,332][20381] Updated weights for policy 1, policy_version 136922 (0.0008) [2023-12-27 06:36:24,373][20347] Updated weights for policy 0, policy_version 137210 (0.0006) [2023-12-27 06:36:24,384][20381] Updated weights for policy 1, policy_version 136932 (0.0010) [2023-12-27 06:36:24,429][20347] Updated weights for policy 0, policy_version 137220 (0.0006) [2023-12-27 06:36:24,438][20381] Updated weights for policy 1, policy_version 136942 (0.0010) [2023-12-27 06:36:24,497][20381] Updated weights for policy 1, policy_version 136952 (0.0010) [2023-12-27 06:36:25,108][20347] Updated weights for policy 0, policy_version 137230 (0.0007) [2023-12-27 06:36:25,116][20381] Updated weights for policy 1, policy_version 136962 (0.0006) [2023-12-27 06:36:25,163][20347] Updated weights for policy 0, policy_version 137240 (0.0007) [2023-12-27 06:36:25,167][20381] Updated weights for policy 1, policy_version 136972 (0.0008) [2023-12-27 06:36:25,211][20347] Updated weights for policy 0, policy_version 137250 (0.0006) [2023-12-27 06:36:25,213][20381] Updated weights for policy 1, policy_version 136982 (0.0006) [2023-12-27 06:36:25,802][20381] Updated weights for policy 1, policy_version 136992 (0.0005) [2023-12-27 06:36:25,862][20381] Updated weights for policy 1, policy_version 137002 (0.0005) [2023-12-27 06:36:25,901][20347] Updated weights for policy 0, policy_version 137260 (0.0007) [2023-12-27 06:36:25,920][20381] Updated weights for policy 1, policy_version 137012 (0.0005) [2023-12-27 06:36:25,954][20347] Updated weights for policy 0, policy_version 137270 (0.0007) [2023-12-27 06:36:26,000][20347] Updated weights for policy 0, policy_version 137280 (0.0010) [2023-12-27 06:36:26,421][20381] Updated weights for policy 1, policy_version 137022 (0.0006) [2023-12-27 06:36:26,475][20381] Updated weights for policy 1, policy_version 137032 (0.0005) [2023-12-27 06:36:26,532][20381] Updated weights for policy 1, policy_version 137042 (0.0010) [2023-12-27 06:36:26,690][20347] Updated weights for policy 0, policy_version 137290 (0.0008) [2023-12-27 06:36:26,738][20347] Updated weights for policy 0, policy_version 137300 (0.0007) [2023-12-27 06:36:26,785][20347] Updated weights for policy 0, policy_version 137310 (0.0007) [2023-12-27 06:36:26,851][20347] Updated weights for policy 0, policy_version 137320 (0.0007) [2023-12-27 06:36:27,138][20381] Updated weights for policy 1, policy_version 137052 (0.0009) [2023-12-27 06:36:27,194][20381] Updated weights for policy 1, policy_version 137062 (0.0007) [2023-12-27 06:36:27,252][20381] Updated weights for policy 1, policy_version 137072 (0.0007) [2023-12-27 06:36:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 70270976. Throughput: 0: 10179.8, 1: 10510.1. Samples: 70256112. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:27,363][19170] Avg episode reward: [(0, '321.930'), (1, '1000.000')] [2023-12-27 06:36:27,560][20347] Updated weights for policy 0, policy_version 137330 (0.0007) [2023-12-27 06:36:27,613][20347] Updated weights for policy 0, policy_version 137340 (0.0007) [2023-12-27 06:36:27,666][20347] Updated weights for policy 0, policy_version 137350 (0.0007) [2023-12-27 06:36:27,889][20381] Updated weights for policy 1, policy_version 137082 (0.0010) [2023-12-27 06:36:27,944][20381] Updated weights for policy 1, policy_version 137092 (0.0011) [2023-12-27 06:36:28,006][20381] Updated weights for policy 1, policy_version 137102 (0.0011) [2023-12-27 06:36:28,064][20381] Updated weights for policy 1, policy_version 137112 (0.0008) [2023-12-27 06:36:28,390][20347] Updated weights for policy 0, policy_version 137360 (0.0006) [2023-12-27 06:36:28,456][20347] Updated weights for policy 0, policy_version 137370 (0.0007) [2023-12-27 06:36:28,521][20347] Updated weights for policy 0, policy_version 137380 (0.0007) [2023-12-27 06:36:28,716][20381] Updated weights for policy 1, policy_version 137122 (0.0005) [2023-12-27 06:36:28,777][20381] Updated weights for policy 1, policy_version 137132 (0.0005) [2023-12-27 06:36:28,838][20381] Updated weights for policy 1, policy_version 137142 (0.0006) [2023-12-27 06:36:29,183][20347] Updated weights for policy 0, policy_version 137390 (0.0008) [2023-12-27 06:36:29,255][20347] Updated weights for policy 0, policy_version 137400 (0.0007) [2023-12-27 06:36:29,312][20347] Updated weights for policy 0, policy_version 137410 (0.0008) [2023-12-27 06:36:29,492][20381] Updated weights for policy 1, policy_version 137152 (0.0010) [2023-12-27 06:36:29,556][20381] Updated weights for policy 1, policy_version 137162 (0.0011) [2023-12-27 06:36:29,612][20381] Updated weights for policy 1, policy_version 137172 (0.0010) [2023-12-27 06:36:30,067][20347] Updated weights for policy 0, policy_version 137420 (0.0007) [2023-12-27 06:36:30,127][20347] Updated weights for policy 0, policy_version 137430 (0.0006) [2023-12-27 06:36:30,187][20347] Updated weights for policy 0, policy_version 137440 (0.0006) [2023-12-27 06:36:30,341][20381] Updated weights for policy 1, policy_version 137182 (0.0011) [2023-12-27 06:36:30,403][20381] Updated weights for policy 1, policy_version 137192 (0.0007) [2023-12-27 06:36:30,473][20381] Updated weights for policy 1, policy_version 137202 (0.0007) [2023-12-27 06:36:30,785][20347] Updated weights for policy 0, policy_version 137450 (0.0007) [2023-12-27 06:36:30,835][20347] Updated weights for policy 0, policy_version 137460 (0.0005) [2023-12-27 06:36:30,883][20347] Updated weights for policy 0, policy_version 137470 (0.0005) [2023-12-27 06:36:30,934][20347] Updated weights for policy 0, policy_version 137480 (0.0006) [2023-12-27 06:36:31,133][20381] Updated weights for policy 1, policy_version 137212 (0.0010) [2023-12-27 06:36:31,188][20381] Updated weights for policy 1, policy_version 137222 (0.0009) [2023-12-27 06:36:31,252][20381] Updated weights for policy 1, policy_version 137232 (0.0007) [2023-12-27 06:36:31,563][20347] Updated weights for policy 0, policy_version 137490 (0.0007) [2023-12-27 06:36:31,618][20347] Updated weights for policy 0, policy_version 137500 (0.0006) [2023-12-27 06:36:31,667][20347] Updated weights for policy 0, policy_version 137510 (0.0006) [2023-12-27 06:36:31,935][20381] Updated weights for policy 1, policy_version 137242 (0.0008) [2023-12-27 06:36:32,001][20381] Updated weights for policy 1, policy_version 137252 (0.0011) [2023-12-27 06:36:32,060][20381] Updated weights for policy 1, policy_version 137262 (0.0011) [2023-12-27 06:36:32,117][20381] Updated weights for policy 1, policy_version 137272 (0.0011) [2023-12-27 06:36:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 70369280. Throughput: 0: 10223.5, 1: 10516.9. Samples: 70380760. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:32,363][19170] Avg episode reward: [(0, '530.740'), (1, '1000.000')] [2023-12-27 06:36:32,389][20347] Updated weights for policy 0, policy_version 137520 (0.0007) [2023-12-27 06:36:32,448][20347] Updated weights for policy 0, policy_version 137530 (0.0006) [2023-12-27 06:36:32,508][20347] Updated weights for policy 0, policy_version 137540 (0.0007) [2023-12-27 06:36:32,780][20381] Updated weights for policy 1, policy_version 137282 (0.0008) [2023-12-27 06:36:32,836][20381] Updated weights for policy 1, policy_version 137292 (0.0008) [2023-12-27 06:36:32,884][20381] Updated weights for policy 1, policy_version 137302 (0.0007) [2023-12-27 06:36:33,220][20347] Updated weights for policy 0, policy_version 137550 (0.0009) [2023-12-27 06:36:33,265][20347] Updated weights for policy 0, policy_version 137560 (0.0009) [2023-12-27 06:36:33,310][20347] Updated weights for policy 0, policy_version 137570 (0.0009) [2023-12-27 06:36:33,445][20381] Updated weights for policy 1, policy_version 137312 (0.0006) [2023-12-27 06:36:33,493][20381] Updated weights for policy 1, policy_version 137322 (0.0009) [2023-12-27 06:36:33,537][20381] Updated weights for policy 1, policy_version 137332 (0.0007) [2023-12-27 06:36:33,881][20347] Updated weights for policy 0, policy_version 137580 (0.0006) [2023-12-27 06:36:33,935][20347] Updated weights for policy 0, policy_version 137591 (0.0008) [2023-12-27 06:36:33,990][20347] Updated weights for policy 0, policy_version 137601 (0.0005) [2023-12-27 06:36:34,201][20381] Updated weights for policy 1, policy_version 137342 (0.0007) [2023-12-27 06:36:34,266][20381] Updated weights for policy 1, policy_version 137352 (0.0008) [2023-12-27 06:36:34,325][20381] Updated weights for policy 1, policy_version 137362 (0.0008) [2023-12-27 06:36:34,665][20347] Updated weights for policy 0, policy_version 137611 (0.0006) [2023-12-27 06:36:34,715][20347] Updated weights for policy 0, policy_version 137621 (0.0008) [2023-12-27 06:36:34,774][20347] Updated weights for policy 0, policy_version 137631 (0.0007) [2023-12-27 06:36:34,975][20381] Updated weights for policy 1, policy_version 137372 (0.0007) [2023-12-27 06:36:35,026][20381] Updated weights for policy 1, policy_version 137382 (0.0006) [2023-12-27 06:36:35,078][20381] Updated weights for policy 1, policy_version 137392 (0.0005) [2023-12-27 06:36:35,450][20347] Updated weights for policy 0, policy_version 137641 (0.0007) [2023-12-27 06:36:35,498][20347] Updated weights for policy 0, policy_version 137651 (0.0005) [2023-12-27 06:36:35,552][20347] Updated weights for policy 0, policy_version 137661 (0.0005) [2023-12-27 06:36:35,597][20381] Updated weights for policy 1, policy_version 137402 (0.0006) [2023-12-27 06:36:35,613][20347] Updated weights for policy 0, policy_version 137671 (0.0005) [2023-12-27 06:36:35,654][20381] Updated weights for policy 1, policy_version 137412 (0.0008) [2023-12-27 06:36:35,704][20381] Updated weights for policy 1, policy_version 137422 (0.0005) [2023-12-27 06:36:35,754][20381] Updated weights for policy 1, policy_version 137432 (0.0007) [2023-12-27 06:36:36,196][20347] Updated weights for policy 0, policy_version 137681 (0.0009) [2023-12-27 06:36:36,252][20347] Updated weights for policy 0, policy_version 137691 (0.0010) [2023-12-27 06:36:36,313][20347] Updated weights for policy 0, policy_version 137701 (0.0006) [2023-12-27 06:36:36,389][20381] Updated weights for policy 1, policy_version 137442 (0.0011) [2023-12-27 06:36:36,448][20381] Updated weights for policy 1, policy_version 137452 (0.0011) [2023-12-27 06:36:36,510][20381] Updated weights for policy 1, policy_version 137462 (0.0011) [2023-12-27 06:36:36,964][20347] Updated weights for policy 0, policy_version 137711 (0.0009) [2023-12-27 06:36:37,019][20347] Updated weights for policy 0, policy_version 137721 (0.0010) [2023-12-27 06:36:37,077][20347] Updated weights for policy 0, policy_version 137731 (0.0010) [2023-12-27 06:36:37,199][20381] Updated weights for policy 1, policy_version 137472 (0.0011) [2023-12-27 06:36:37,258][20381] Updated weights for policy 1, policy_version 137482 (0.0011) [2023-12-27 06:36:37,319][20381] Updated weights for policy 1, policy_version 137492 (0.0011) [2023-12-27 06:36:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 70483968. Throughput: 0: 10272.3, 1: 10579.7. Samples: 70445212. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:37,363][19170] Avg episode reward: [(0, '871.790'), (1, '1000.000')] [2023-12-27 06:36:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000137496_35209216.pth... [2023-12-27 06:36:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000137736_35274752.pth... [2023-12-27 06:36:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000136216_34881536.pth [2023-12-27 06:36:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000136552_34971648.pth [2023-12-27 06:36:37,756][20347] Updated weights for policy 0, policy_version 137741 (0.0010) [2023-12-27 06:36:37,803][20347] Updated weights for policy 0, policy_version 137751 (0.0010) [2023-12-27 06:36:37,858][20347] Updated weights for policy 0, policy_version 137761 (0.0010) [2023-12-27 06:36:37,977][20381] Updated weights for policy 1, policy_version 137502 (0.0007) [2023-12-27 06:36:38,030][20381] Updated weights for policy 1, policy_version 137512 (0.0009) [2023-12-27 06:36:38,084][20381] Updated weights for policy 1, policy_version 137522 (0.0010) [2023-12-27 06:36:38,566][20347] Updated weights for policy 0, policy_version 137771 (0.0010) [2023-12-27 06:36:38,618][20347] Updated weights for policy 0, policy_version 137781 (0.0010) [2023-12-27 06:36:38,666][20347] Updated weights for policy 0, policy_version 137791 (0.0010) [2023-12-27 06:36:38,743][20381] Updated weights for policy 1, policy_version 137532 (0.0011) [2023-12-27 06:36:38,800][20381] Updated weights for policy 1, policy_version 137542 (0.0010) [2023-12-27 06:36:38,855][20381] Updated weights for policy 1, policy_version 137552 (0.0005) [2023-12-27 06:36:39,367][20347] Updated weights for policy 0, policy_version 137801 (0.0010) [2023-12-27 06:36:39,431][20347] Updated weights for policy 0, policy_version 137811 (0.0008) [2023-12-27 06:36:39,489][20381] Updated weights for policy 1, policy_version 137562 (0.0006) [2023-12-27 06:36:39,494][20347] Updated weights for policy 0, policy_version 137821 (0.0008) [2023-12-27 06:36:39,547][20381] Updated weights for policy 1, policy_version 137572 (0.0007) [2023-12-27 06:36:39,558][20347] Updated weights for policy 0, policy_version 137831 (0.0007) [2023-12-27 06:36:39,611][20381] Updated weights for policy 1, policy_version 137582 (0.0009) [2023-12-27 06:36:39,675][20381] Updated weights for policy 1, policy_version 137592 (0.0006) [2023-12-27 06:36:40,174][20347] Updated weights for policy 0, policy_version 137841 (0.0005) [2023-12-27 06:36:40,238][20347] Updated weights for policy 0, policy_version 137851 (0.0005) [2023-12-27 06:36:40,299][20347] Updated weights for policy 0, policy_version 137861 (0.0007) [2023-12-27 06:36:40,379][20381] Updated weights for policy 1, policy_version 137602 (0.0008) [2023-12-27 06:36:40,429][20381] Updated weights for policy 1, policy_version 137612 (0.0008) [2023-12-27 06:36:40,482][20381] Updated weights for policy 1, policy_version 137622 (0.0008) [2023-12-27 06:36:40,964][20347] Updated weights for policy 0, policy_version 137871 (0.0007) [2023-12-27 06:36:41,009][20347] Updated weights for policy 0, policy_version 137881 (0.0007) [2023-12-27 06:36:41,071][20347] Updated weights for policy 0, policy_version 137891 (0.0008) [2023-12-27 06:36:41,234][20381] Updated weights for policy 1, policy_version 137632 (0.0006) [2023-12-27 06:36:41,295][20381] Updated weights for policy 1, policy_version 137642 (0.0007) [2023-12-27 06:36:41,357][20381] Updated weights for policy 1, policy_version 137652 (0.0008) [2023-12-27 06:36:41,818][20347] Updated weights for policy 0, policy_version 137901 (0.0008) [2023-12-27 06:36:41,877][20347] Updated weights for policy 0, policy_version 137911 (0.0010) [2023-12-27 06:36:41,936][20347] Updated weights for policy 0, policy_version 137921 (0.0011) [2023-12-27 06:36:42,081][20381] Updated weights for policy 1, policy_version 137662 (0.0007) [2023-12-27 06:36:42,141][20381] Updated weights for policy 1, policy_version 137672 (0.0007) [2023-12-27 06:36:42,197][20381] Updated weights for policy 1, policy_version 137682 (0.0008) [2023-12-27 06:36:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 70582272. Throughput: 0: 10276.4, 1: 10553.6. Samples: 70570692. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:42,363][19170] Avg episode reward: [(0, '990.340'), (1, '1000.000')] [2023-12-27 06:36:42,618][20347] Updated weights for policy 0, policy_version 137931 (0.0010) [2023-12-27 06:36:42,680][20347] Updated weights for policy 0, policy_version 137941 (0.0007) [2023-12-27 06:36:42,740][20347] Updated weights for policy 0, policy_version 137951 (0.0008) [2023-12-27 06:36:42,893][20381] Updated weights for policy 1, policy_version 137692 (0.0007) [2023-12-27 06:36:42,939][20381] Updated weights for policy 1, policy_version 137702 (0.0008) [2023-12-27 06:36:42,984][20381] Updated weights for policy 1, policy_version 137712 (0.0008) [2023-12-27 06:36:43,374][20347] Updated weights for policy 0, policy_version 137961 (0.0007) [2023-12-27 06:36:43,439][20347] Updated weights for policy 0, policy_version 137971 (0.0007) [2023-12-27 06:36:43,488][20347] Updated weights for policy 0, policy_version 137981 (0.0008) [2023-12-27 06:36:43,547][20347] Updated weights for policy 0, policy_version 137991 (0.0007) [2023-12-27 06:36:43,680][20381] Updated weights for policy 1, policy_version 137722 (0.0006) [2023-12-27 06:36:43,735][20381] Updated weights for policy 1, policy_version 137732 (0.0008) [2023-12-27 06:36:43,783][20381] Updated weights for policy 1, policy_version 137742 (0.0010) [2023-12-27 06:36:43,832][20381] Updated weights for policy 1, policy_version 137752 (0.0007) [2023-12-27 06:36:44,186][20347] Updated weights for policy 0, policy_version 138001 (0.0007) [2023-12-27 06:36:44,239][20347] Updated weights for policy 0, policy_version 138011 (0.0007) [2023-12-27 06:36:44,302][20347] Updated weights for policy 0, policy_version 138021 (0.0007) [2023-12-27 06:36:44,515][20381] Updated weights for policy 1, policy_version 137762 (0.0010) [2023-12-27 06:36:44,560][20381] Updated weights for policy 1, policy_version 137772 (0.0010) [2023-12-27 06:36:44,606][20381] Updated weights for policy 1, policy_version 137782 (0.0010) [2023-12-27 06:36:44,942][20347] Updated weights for policy 0, policy_version 138031 (0.0009) [2023-12-27 06:36:44,998][20347] Updated weights for policy 0, policy_version 138041 (0.0009) [2023-12-27 06:36:45,057][20347] Updated weights for policy 0, policy_version 138051 (0.0009) [2023-12-27 06:36:45,286][20381] Updated weights for policy 1, policy_version 137792 (0.0011) [2023-12-27 06:36:45,342][20381] Updated weights for policy 1, policy_version 137802 (0.0011) [2023-12-27 06:36:45,401][20381] Updated weights for policy 1, policy_version 137812 (0.0010) [2023-12-27 06:36:45,706][20347] Updated weights for policy 0, policy_version 138061 (0.0009) [2023-12-27 06:36:45,764][20347] Updated weights for policy 0, policy_version 138071 (0.0010) [2023-12-27 06:36:45,827][20347] Updated weights for policy 0, policy_version 138081 (0.0008) [2023-12-27 06:36:46,052][20381] Updated weights for policy 1, policy_version 137822 (0.0010) [2023-12-27 06:36:46,100][20381] Updated weights for policy 1, policy_version 137832 (0.0010) [2023-12-27 06:36:46,150][20381] Updated weights for policy 1, policy_version 137842 (0.0005) [2023-12-27 06:36:46,492][20347] Updated weights for policy 0, policy_version 138091 (0.0010) [2023-12-27 06:36:46,543][20347] Updated weights for policy 0, policy_version 138101 (0.0010) [2023-12-27 06:36:46,597][20347] Updated weights for policy 0, policy_version 138111 (0.0010) [2023-12-27 06:36:46,832][20381] Updated weights for policy 1, policy_version 137852 (0.0008) [2023-12-27 06:36:46,881][20381] Updated weights for policy 1, policy_version 137862 (0.0005) [2023-12-27 06:36:46,927][20381] Updated weights for policy 1, policy_version 137872 (0.0006) [2023-12-27 06:36:47,263][20347] Updated weights for policy 0, policy_version 138121 (0.0010) [2023-12-27 06:36:47,321][20347] Updated weights for policy 0, policy_version 138131 (0.0006) [2023-12-27 06:36:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 70680576. Throughput: 0: 10293.1, 1: 10571.3. Samples: 70694428. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:36:47,376][20347] Updated weights for policy 0, policy_version 138141 (0.0005) [2023-12-27 06:36:47,429][20347] Updated weights for policy 0, policy_version 138151 (0.0006) [2023-12-27 06:36:47,620][20381] Updated weights for policy 1, policy_version 137882 (0.0008) [2023-12-27 06:36:47,679][20381] Updated weights for policy 1, policy_version 137892 (0.0008) [2023-12-27 06:36:47,734][20381] Updated weights for policy 1, policy_version 137902 (0.0007) [2023-12-27 06:36:47,783][20381] Updated weights for policy 1, policy_version 137912 (0.0008) [2023-12-27 06:36:48,030][20347] Updated weights for policy 0, policy_version 138161 (0.0008) [2023-12-27 06:36:48,086][20347] Updated weights for policy 0, policy_version 138171 (0.0008) [2023-12-27 06:36:48,144][20347] Updated weights for policy 0, policy_version 138181 (0.0007) [2023-12-27 06:36:48,536][20381] Updated weights for policy 1, policy_version 137922 (0.0008) [2023-12-27 06:36:48,592][20381] Updated weights for policy 1, policy_version 137932 (0.0008) [2023-12-27 06:36:48,652][20381] Updated weights for policy 1, policy_version 137942 (0.0008) [2023-12-27 06:36:48,789][20347] Updated weights for policy 0, policy_version 138191 (0.0006) [2023-12-27 06:36:48,851][20347] Updated weights for policy 0, policy_version 138201 (0.0007) [2023-12-27 06:36:48,913][20347] Updated weights for policy 0, policy_version 138211 (0.0008) [2023-12-27 06:36:49,402][20381] Updated weights for policy 1, policy_version 137952 (0.0008) [2023-12-27 06:36:49,459][20381] Updated weights for policy 1, policy_version 137962 (0.0008) [2023-12-27 06:36:49,514][20381] Updated weights for policy 1, policy_version 137972 (0.0007) [2023-12-27 06:36:49,519][20347] Updated weights for policy 0, policy_version 138221 (0.0006) [2023-12-27 06:36:49,582][20347] Updated weights for policy 0, policy_version 138231 (0.0006) [2023-12-27 06:36:49,642][20347] Updated weights for policy 0, policy_version 138241 (0.0005) [2023-12-27 06:36:50,254][20347] Updated weights for policy 0, policy_version 138251 (0.0006) [2023-12-27 06:36:50,260][20381] Updated weights for policy 1, policy_version 137982 (0.0006) [2023-12-27 06:36:50,314][20347] Updated weights for policy 0, policy_version 138261 (0.0006) [2023-12-27 06:36:50,320][20381] Updated weights for policy 1, policy_version 137992 (0.0006) [2023-12-27 06:36:50,373][20347] Updated weights for policy 0, policy_version 138271 (0.0007) [2023-12-27 06:36:50,374][20381] Updated weights for policy 1, policy_version 138002 (0.0008) [2023-12-27 06:36:51,005][20347] Updated weights for policy 0, policy_version 138281 (0.0006) [2023-12-27 06:36:51,065][20381] Updated weights for policy 1, policy_version 138012 (0.0008) [2023-12-27 06:36:51,072][20347] Updated weights for policy 0, policy_version 138291 (0.0007) [2023-12-27 06:36:51,122][20381] Updated weights for policy 1, policy_version 138022 (0.0010) [2023-12-27 06:36:51,123][20347] Updated weights for policy 0, policy_version 138301 (0.0006) [2023-12-27 06:36:51,178][20381] Updated weights for policy 1, policy_version 138032 (0.0008) [2023-12-27 06:36:51,180][20347] Updated weights for policy 0, policy_version 138311 (0.0006) [2023-12-27 06:36:51,849][20347] Updated weights for policy 0, policy_version 138321 (0.0007) [2023-12-27 06:36:51,894][20381] Updated weights for policy 1, policy_version 138042 (0.0008) [2023-12-27 06:36:51,898][20347] Updated weights for policy 0, policy_version 138331 (0.0007) [2023-12-27 06:36:51,945][20381] Updated weights for policy 1, policy_version 138052 (0.0007) [2023-12-27 06:36:51,955][20347] Updated weights for policy 0, policy_version 138341 (0.0007) [2023-12-27 06:36:51,998][20381] Updated weights for policy 1, policy_version 138062 (0.0006) [2023-12-27 06:36:52,060][20381] Updated weights for policy 1, policy_version 138072 (0.0008) [2023-12-27 06:36:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 70787072. Throughput: 0: 10347.2, 1: 10533.2. Samples: 70755624. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:36:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000138072_35356672.pth... [2023-12-27 06:36:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000138344_35430400.pth... [2023-12-27 06:36:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000136856_35045376.pth [2023-12-27 06:36:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000137128_35119104.pth [2023-12-27 06:36:52,624][20347] Updated weights for policy 0, policy_version 138351 (0.0006) [2023-12-27 06:36:52,676][20347] Updated weights for policy 0, policy_version 138361 (0.0009) [2023-12-27 06:36:52,728][20347] Updated weights for policy 0, policy_version 138371 (0.0010) [2023-12-27 06:36:52,787][20381] Updated weights for policy 1, policy_version 138082 (0.0011) [2023-12-27 06:36:52,844][20381] Updated weights for policy 1, policy_version 138092 (0.0011) [2023-12-27 06:36:52,901][20381] Updated weights for policy 1, policy_version 138102 (0.0009) [2023-12-27 06:36:53,426][20347] Updated weights for policy 0, policy_version 138381 (0.0009) [2023-12-27 06:36:53,474][20347] Updated weights for policy 0, policy_version 138391 (0.0005) [2023-12-27 06:36:53,523][20347] Updated weights for policy 0, policy_version 138401 (0.0006) [2023-12-27 06:36:53,555][20381] Updated weights for policy 1, policy_version 138112 (0.0005) [2023-12-27 06:36:53,601][20381] Updated weights for policy 1, policy_version 138122 (0.0008) [2023-12-27 06:36:53,646][20381] Updated weights for policy 1, policy_version 138132 (0.0007) [2023-12-27 06:36:54,207][20347] Updated weights for policy 0, policy_version 138411 (0.0007) [2023-12-27 06:36:54,262][20347] Updated weights for policy 0, policy_version 138421 (0.0007) [2023-12-27 06:36:54,320][20347] Updated weights for policy 0, policy_version 138431 (0.0008) [2023-12-27 06:36:54,355][20381] Updated weights for policy 1, policy_version 138142 (0.0006) [2023-12-27 06:36:54,407][20381] Updated weights for policy 1, policy_version 138152 (0.0010) [2023-12-27 06:36:54,456][20381] Updated weights for policy 1, policy_version 138162 (0.0010) [2023-12-27 06:36:54,870][20347] Updated weights for policy 0, policy_version 138441 (0.0007) [2023-12-27 06:36:54,930][20347] Updated weights for policy 0, policy_version 138451 (0.0005) [2023-12-27 06:36:54,981][20347] Updated weights for policy 0, policy_version 138461 (0.0009) [2023-12-27 06:36:55,038][20347] Updated weights for policy 0, policy_version 138471 (0.0007) [2023-12-27 06:36:55,135][20381] Updated weights for policy 1, policy_version 138172 (0.0009) [2023-12-27 06:36:55,188][20381] Updated weights for policy 1, policy_version 138182 (0.0006) [2023-12-27 06:36:55,244][20381] Updated weights for policy 1, policy_version 138192 (0.0005) [2023-12-27 06:36:55,696][20347] Updated weights for policy 0, policy_version 138481 (0.0009) [2023-12-27 06:36:55,754][20347] Updated weights for policy 0, policy_version 138491 (0.0008) [2023-12-27 06:36:55,793][20381] Updated weights for policy 1, policy_version 138202 (0.0006) [2023-12-27 06:36:55,807][20347] Updated weights for policy 0, policy_version 138501 (0.0007) [2023-12-27 06:36:55,854][20381] Updated weights for policy 1, policy_version 138212 (0.0008) [2023-12-27 06:36:55,908][20381] Updated weights for policy 1, policy_version 138222 (0.0007) [2023-12-27 06:36:55,959][20381] Updated weights for policy 1, policy_version 138232 (0.0007) [2023-12-27 06:36:56,509][20347] Updated weights for policy 0, policy_version 138511 (0.0007) [2023-12-27 06:36:56,560][20347] Updated weights for policy 0, policy_version 138521 (0.0007) [2023-12-27 06:36:56,603][20347] Updated weights for policy 0, policy_version 138531 (0.0006) [2023-12-27 06:36:56,635][20381] Updated weights for policy 1, policy_version 138242 (0.0006) [2023-12-27 06:36:56,689][20381] Updated weights for policy 1, policy_version 138252 (0.0007) [2023-12-27 06:36:56,745][20381] Updated weights for policy 1, policy_version 138262 (0.0008) [2023-12-27 06:36:57,214][20347] Updated weights for policy 0, policy_version 138541 (0.0005) [2023-12-27 06:36:57,267][20347] Updated weights for policy 0, policy_version 138551 (0.0005) [2023-12-27 06:36:57,323][20347] Updated weights for policy 0, policy_version 138561 (0.0006) [2023-12-27 06:36:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 70893568. Throughput: 0: 10353.2, 1: 10431.9. Samples: 70880220. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:36:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:36:57,431][20381] Updated weights for policy 1, policy_version 138272 (0.0009) [2023-12-27 06:36:57,497][20381] Updated weights for policy 1, policy_version 138282 (0.0007) [2023-12-27 06:36:57,556][20381] Updated weights for policy 1, policy_version 138292 (0.0007) [2023-12-27 06:36:57,909][20347] Updated weights for policy 0, policy_version 138571 (0.0009) [2023-12-27 06:36:57,967][20347] Updated weights for policy 0, policy_version 138581 (0.0009) [2023-12-27 06:36:58,020][20347] Updated weights for policy 0, policy_version 138591 (0.0010) [2023-12-27 06:36:58,190][20381] Updated weights for policy 1, policy_version 138302 (0.0008) [2023-12-27 06:36:58,248][20381] Updated weights for policy 1, policy_version 138312 (0.0008) [2023-12-27 06:36:58,311][20381] Updated weights for policy 1, policy_version 138322 (0.0008) [2023-12-27 06:36:58,790][20347] Updated weights for policy 0, policy_version 138601 (0.0010) [2023-12-27 06:36:58,856][20347] Updated weights for policy 0, policy_version 138611 (0.0008) [2023-12-27 06:36:58,915][20347] Updated weights for policy 0, policy_version 138621 (0.0007) [2023-12-27 06:36:58,976][20347] Updated weights for policy 0, policy_version 138631 (0.0008) [2023-12-27 06:36:59,066][20381] Updated weights for policy 1, policy_version 138332 (0.0008) [2023-12-27 06:36:59,123][20381] Updated weights for policy 1, policy_version 138342 (0.0005) [2023-12-27 06:36:59,192][20381] Updated weights for policy 1, policy_version 138352 (0.0008) [2023-12-27 06:36:59,673][20347] Updated weights for policy 0, policy_version 138641 (0.0008) [2023-12-27 06:36:59,736][20347] Updated weights for policy 0, policy_version 138651 (0.0008) [2023-12-27 06:36:59,796][20347] Updated weights for policy 0, policy_version 138661 (0.0008) [2023-12-27 06:36:59,881][20381] Updated weights for policy 1, policy_version 138362 (0.0009) [2023-12-27 06:36:59,940][20381] Updated weights for policy 1, policy_version 138372 (0.0009) [2023-12-27 06:37:00,004][20381] Updated weights for policy 1, policy_version 138382 (0.0006) [2023-12-27 06:37:00,065][20381] Updated weights for policy 1, policy_version 138392 (0.0007) [2023-12-27 06:37:00,458][20347] Updated weights for policy 0, policy_version 138671 (0.0008) [2023-12-27 06:37:00,506][20347] Updated weights for policy 0, policy_version 138681 (0.0005) [2023-12-27 06:37:00,562][20347] Updated weights for policy 0, policy_version 138691 (0.0005) [2023-12-27 06:37:00,761][20381] Updated weights for policy 1, policy_version 138402 (0.0006) [2023-12-27 06:37:00,824][20381] Updated weights for policy 1, policy_version 138412 (0.0006) [2023-12-27 06:37:00,886][20381] Updated weights for policy 1, policy_version 138422 (0.0008) [2023-12-27 06:37:01,222][20347] Updated weights for policy 0, policy_version 138701 (0.0005) [2023-12-27 06:37:01,280][20347] Updated weights for policy 0, policy_version 138711 (0.0007) [2023-12-27 06:37:01,337][20347] Updated weights for policy 0, policy_version 138721 (0.0008) [2023-12-27 06:37:01,606][20381] Updated weights for policy 1, policy_version 138432 (0.0006) [2023-12-27 06:37:01,667][20381] Updated weights for policy 1, policy_version 138442 (0.0007) [2023-12-27 06:37:01,719][20381] Updated weights for policy 1, policy_version 138452 (0.0008) [2023-12-27 06:37:01,987][20347] Updated weights for policy 0, policy_version 138731 (0.0007) [2023-12-27 06:37:02,044][20347] Updated weights for policy 0, policy_version 138741 (0.0005) [2023-12-27 06:37:02,101][20347] Updated weights for policy 0, policy_version 138751 (0.0005) [2023-12-27 06:37:02,358][20381] Updated weights for policy 1, policy_version 138462 (0.0008) [2023-12-27 06:37:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 70991872. Throughput: 0: 10408.1, 1: 10324.6. Samples: 71003180. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:02,412][20381] Updated weights for policy 1, policy_version 138472 (0.0007) [2023-12-27 06:37:02,474][20381] Updated weights for policy 1, policy_version 138482 (0.0005) [2023-12-27 06:37:02,667][20347] Updated weights for policy 0, policy_version 138761 (0.0008) [2023-12-27 06:37:02,714][20347] Updated weights for policy 0, policy_version 138771 (0.0007) [2023-12-27 06:37:02,770][20347] Updated weights for policy 0, policy_version 138781 (0.0007) [2023-12-27 06:37:02,826][20347] Updated weights for policy 0, policy_version 138791 (0.0008) [2023-12-27 06:37:03,139][20381] Updated weights for policy 1, policy_version 138492 (0.0006) [2023-12-27 06:37:03,187][20381] Updated weights for policy 1, policy_version 138502 (0.0005) [2023-12-27 06:37:03,239][20381] Updated weights for policy 1, policy_version 138512 (0.0005) [2023-12-27 06:37:03,557][20347] Updated weights for policy 0, policy_version 138801 (0.0008) [2023-12-27 06:37:03,607][20347] Updated weights for policy 0, policy_version 138811 (0.0007) [2023-12-27 06:37:03,658][20347] Updated weights for policy 0, policy_version 138821 (0.0007) [2023-12-27 06:37:03,790][20381] Updated weights for policy 1, policy_version 138522 (0.0005) [2023-12-27 06:37:03,854][20381] Updated weights for policy 1, policy_version 138532 (0.0009) [2023-12-27 06:37:03,909][20381] Updated weights for policy 1, policy_version 138542 (0.0010) [2023-12-27 06:37:03,967][20381] Updated weights for policy 1, policy_version 138552 (0.0006) [2023-12-27 06:37:04,323][20347] Updated weights for policy 0, policy_version 138831 (0.0006) [2023-12-27 06:37:04,376][20347] Updated weights for policy 0, policy_version 138841 (0.0005) [2023-12-27 06:37:04,431][20347] Updated weights for policy 0, policy_version 138851 (0.0005) [2023-12-27 06:37:04,633][20381] Updated weights for policy 1, policy_version 138562 (0.0005) [2023-12-27 06:37:04,687][20381] Updated weights for policy 1, policy_version 138572 (0.0005) [2023-12-27 06:37:04,750][20381] Updated weights for policy 1, policy_version 138582 (0.0005) [2023-12-27 06:37:04,998][20347] Updated weights for policy 0, policy_version 138861 (0.0008) [2023-12-27 06:37:05,056][20347] Updated weights for policy 0, policy_version 138871 (0.0010) [2023-12-27 06:37:05,114][20347] Updated weights for policy 0, policy_version 138881 (0.0010) [2023-12-27 06:37:05,319][20381] Updated weights for policy 1, policy_version 138592 (0.0008) [2023-12-27 06:37:05,365][20381] Updated weights for policy 1, policy_version 138602 (0.0007) [2023-12-27 06:37:05,420][20381] Updated weights for policy 1, policy_version 138612 (0.0006) [2023-12-27 06:37:05,717][20347] Updated weights for policy 0, policy_version 138891 (0.0009) [2023-12-27 06:37:05,778][20347] Updated weights for policy 0, policy_version 138901 (0.0007) [2023-12-27 06:37:05,836][20347] Updated weights for policy 0, policy_version 138911 (0.0007) [2023-12-27 06:37:06,022][20381] Updated weights for policy 1, policy_version 138622 (0.0008) [2023-12-27 06:37:06,073][20381] Updated weights for policy 1, policy_version 138632 (0.0010) [2023-12-27 06:37:06,129][20381] Updated weights for policy 1, policy_version 138642 (0.0010) [2023-12-27 06:37:06,412][20347] Updated weights for policy 0, policy_version 138921 (0.0007) [2023-12-27 06:37:06,464][20347] Updated weights for policy 0, policy_version 138931 (0.0006) [2023-12-27 06:37:06,521][20347] Updated weights for policy 0, policy_version 138941 (0.0007) [2023-12-27 06:37:06,569][20347] Updated weights for policy 0, policy_version 138951 (0.0007) [2023-12-27 06:37:06,808][20381] Updated weights for policy 1, policy_version 138652 (0.0010) [2023-12-27 06:37:06,860][20381] Updated weights for policy 1, policy_version 138662 (0.0011) [2023-12-27 06:37:06,924][20381] Updated weights for policy 1, policy_version 138672 (0.0011) [2023-12-27 06:37:07,278][20347] Updated weights for policy 0, policy_version 138961 (0.0007) [2023-12-27 06:37:07,334][20347] Updated weights for policy 0, policy_version 138971 (0.0008) [2023-12-27 06:37:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 71098368. Throughput: 0: 10434.4, 1: 10364.7. Samples: 71068968. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000138680_35512320.pth... [2023-12-27 06:37:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000137496_35209216.pth [2023-12-27 06:37:07,385][20347] Updated weights for policy 0, policy_version 138981 (0.0007) [2023-12-27 06:37:07,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000138984_35594240.pth... [2023-12-27 06:37:07,404][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000137736_35274752.pth [2023-12-27 06:37:07,620][20381] Updated weights for policy 1, policy_version 138682 (0.0011) [2023-12-27 06:37:07,681][20381] Updated weights for policy 1, policy_version 138692 (0.0010) [2023-12-27 06:37:07,738][20381] Updated weights for policy 1, policy_version 138702 (0.0010) [2023-12-27 06:37:07,787][20381] Updated weights for policy 1, policy_version 138712 (0.0010) [2023-12-27 06:37:08,039][20347] Updated weights for policy 0, policy_version 138991 (0.0007) [2023-12-27 06:37:08,097][20347] Updated weights for policy 0, policy_version 139001 (0.0005) [2023-12-27 06:37:08,150][20347] Updated weights for policy 0, policy_version 139011 (0.0005) [2023-12-27 06:37:08,475][20381] Updated weights for policy 1, policy_version 138722 (0.0010) [2023-12-27 06:37:08,533][20381] Updated weights for policy 1, policy_version 138732 (0.0010) [2023-12-27 06:37:08,591][20381] Updated weights for policy 1, policy_version 138742 (0.0007) [2023-12-27 06:37:08,742][20347] Updated weights for policy 0, policy_version 139021 (0.0007) [2023-12-27 06:37:08,787][20347] Updated weights for policy 0, policy_version 139031 (0.0008) [2023-12-27 06:37:08,849][20347] Updated weights for policy 0, policy_version 139041 (0.0010) [2023-12-27 06:37:09,220][20381] Updated weights for policy 1, policy_version 138752 (0.0007) [2023-12-27 06:37:09,275][20381] Updated weights for policy 1, policy_version 138762 (0.0011) [2023-12-27 06:37:09,339][20381] Updated weights for policy 1, policy_version 138772 (0.0011) [2023-12-27 06:37:09,533][20347] Updated weights for policy 0, policy_version 139051 (0.0009) [2023-12-27 06:37:09,600][20347] Updated weights for policy 0, policy_version 139061 (0.0006) [2023-12-27 06:37:09,656][20347] Updated weights for policy 0, policy_version 139071 (0.0006) [2023-12-27 06:37:10,060][20381] Updated weights for policy 1, policy_version 138782 (0.0009) [2023-12-27 06:37:10,117][20381] Updated weights for policy 1, policy_version 138792 (0.0010) [2023-12-27 06:37:10,173][20381] Updated weights for policy 1, policy_version 138802 (0.0011) [2023-12-27 06:37:10,310][20347] Updated weights for policy 0, policy_version 139081 (0.0006) [2023-12-27 06:37:10,379][20347] Updated weights for policy 0, policy_version 139091 (0.0008) [2023-12-27 06:37:10,433][20347] Updated weights for policy 0, policy_version 139101 (0.0007) [2023-12-27 06:37:10,490][20347] Updated weights for policy 0, policy_version 139111 (0.0005) [2023-12-27 06:37:10,857][20381] Updated weights for policy 1, policy_version 138812 (0.0009) [2023-12-27 06:37:10,911][20381] Updated weights for policy 1, policy_version 138822 (0.0009) [2023-12-27 06:37:10,960][20381] Updated weights for policy 1, policy_version 138832 (0.0009) [2023-12-27 06:37:11,082][20347] Updated weights for policy 0, policy_version 139121 (0.0008) [2023-12-27 06:37:11,140][20347] Updated weights for policy 0, policy_version 139131 (0.0007) [2023-12-27 06:37:11,206][20347] Updated weights for policy 0, policy_version 139141 (0.0007) [2023-12-27 06:37:11,685][20381] Updated weights for policy 1, policy_version 138842 (0.0009) [2023-12-27 06:37:11,756][20381] Updated weights for policy 1, policy_version 138852 (0.0007) [2023-12-27 06:37:11,818][20381] Updated weights for policy 1, policy_version 138862 (0.0007) [2023-12-27 06:37:11,872][20381] Updated weights for policy 1, policy_version 138872 (0.0007) [2023-12-27 06:37:11,927][20347] Updated weights for policy 0, policy_version 139151 (0.0006) [2023-12-27 06:37:11,981][20347] Updated weights for policy 0, policy_version 139161 (0.0007) [2023-12-27 06:37:12,038][20347] Updated weights for policy 0, policy_version 139171 (0.0008) [2023-12-27 06:37:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 71204864. Throughput: 0: 10535.6, 1: 10332.7. Samples: 71195188. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:12,538][20381] Updated weights for policy 1, policy_version 138882 (0.0008) [2023-12-27 06:37:12,596][20381] Updated weights for policy 1, policy_version 138892 (0.0007) [2023-12-27 06:37:12,653][20381] Updated weights for policy 1, policy_version 138902 (0.0008) [2023-12-27 06:37:12,722][20347] Updated weights for policy 0, policy_version 139181 (0.0007) [2023-12-27 06:37:12,790][20347] Updated weights for policy 0, policy_version 139191 (0.0008) [2023-12-27 06:37:12,853][20347] Updated weights for policy 0, policy_version 139201 (0.0008) [2023-12-27 06:37:13,293][20381] Updated weights for policy 1, policy_version 138912 (0.0008) [2023-12-27 06:37:13,344][20381] Updated weights for policy 1, policy_version 138922 (0.0007) [2023-12-27 06:37:13,395][20381] Updated weights for policy 1, policy_version 138932 (0.0007) [2023-12-27 06:37:13,487][20347] Updated weights for policy 0, policy_version 139211 (0.0008) [2023-12-27 06:37:13,551][20347] Updated weights for policy 0, policy_version 139221 (0.0008) [2023-12-27 06:37:13,616][20347] Updated weights for policy 0, policy_version 139231 (0.0008) [2023-12-27 06:37:14,050][20381] Updated weights for policy 1, policy_version 138942 (0.0006) [2023-12-27 06:37:14,102][20381] Updated weights for policy 1, policy_version 138952 (0.0006) [2023-12-27 06:37:14,155][20381] Updated weights for policy 1, policy_version 138962 (0.0008) [2023-12-27 06:37:14,271][20347] Updated weights for policy 0, policy_version 139241 (0.0008) [2023-12-27 06:37:14,332][20347] Updated weights for policy 0, policy_version 139251 (0.0009) [2023-12-27 06:37:14,393][20347] Updated weights for policy 0, policy_version 139261 (0.0007) [2023-12-27 06:37:14,457][20347] Updated weights for policy 0, policy_version 139271 (0.0006) [2023-12-27 06:37:14,827][20381] Updated weights for policy 1, policy_version 138972 (0.0009) [2023-12-27 06:37:14,887][20381] Updated weights for policy 1, policy_version 138982 (0.0006) [2023-12-27 06:37:14,950][20381] Updated weights for policy 1, policy_version 138992 (0.0008) [2023-12-27 06:37:15,128][20347] Updated weights for policy 0, policy_version 139281 (0.0007) [2023-12-27 06:37:15,175][20347] Updated weights for policy 0, policy_version 139291 (0.0007) [2023-12-27 06:37:15,223][20347] Updated weights for policy 0, policy_version 139301 (0.0007) [2023-12-27 06:37:15,623][20381] Updated weights for policy 1, policy_version 139002 (0.0008) [2023-12-27 06:37:15,673][20381] Updated weights for policy 1, policy_version 139012 (0.0006) [2023-12-27 06:37:15,724][20381] Updated weights for policy 1, policy_version 139022 (0.0005) [2023-12-27 06:37:15,789][20381] Updated weights for policy 1, policy_version 139032 (0.0010) [2023-12-27 06:37:15,852][20347] Updated weights for policy 0, policy_version 139311 (0.0008) [2023-12-27 06:37:15,910][20347] Updated weights for policy 0, policy_version 139321 (0.0007) [2023-12-27 06:37:15,969][20347] Updated weights for policy 0, policy_version 139331 (0.0008) [2023-12-27 06:37:16,490][20381] Updated weights for policy 1, policy_version 139042 (0.0007) [2023-12-27 06:37:16,575][20381] Updated weights for policy 1, policy_version 139052 (0.0008) [2023-12-27 06:37:16,634][20381] Updated weights for policy 1, policy_version 139062 (0.0006) [2023-12-27 06:37:16,670][20347] Updated weights for policy 0, policy_version 139341 (0.0007) [2023-12-27 06:37:16,729][20347] Updated weights for policy 0, policy_version 139351 (0.0007) [2023-12-27 06:37:16,790][20347] Updated weights for policy 0, policy_version 139361 (0.0008) [2023-12-27 06:37:17,234][20381] Updated weights for policy 1, policy_version 139072 (0.0007) [2023-12-27 06:37:17,289][20381] Updated weights for policy 1, policy_version 139082 (0.0006) [2023-12-27 06:37:17,350][20381] Updated weights for policy 1, policy_version 139092 (0.0007) [2023-12-27 06:37:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 71303168. Throughput: 0: 10523.7, 1: 10324.7. Samples: 71318940. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:17,396][20347] Updated weights for policy 0, policy_version 139371 (0.0008) [2023-12-27 06:37:17,454][20347] Updated weights for policy 0, policy_version 139381 (0.0006) [2023-12-27 06:37:17,500][20347] Updated weights for policy 0, policy_version 139391 (0.0006) [2023-12-27 06:37:17,993][20381] Updated weights for policy 1, policy_version 139102 (0.0006) [2023-12-27 06:37:18,047][20381] Updated weights for policy 1, policy_version 139112 (0.0008) [2023-12-27 06:37:18,097][20381] Updated weights for policy 1, policy_version 139122 (0.0005) [2023-12-27 06:37:18,160][20347] Updated weights for policy 0, policy_version 139401 (0.0007) [2023-12-27 06:37:18,217][20347] Updated weights for policy 0, policy_version 139411 (0.0010) [2023-12-27 06:37:18,271][20347] Updated weights for policy 0, policy_version 139421 (0.0010) [2023-12-27 06:37:18,322][20347] Updated weights for policy 0, policy_version 139431 (0.0010) [2023-12-27 06:37:18,721][20381] Updated weights for policy 1, policy_version 139132 (0.0006) [2023-12-27 06:37:18,780][20381] Updated weights for policy 1, policy_version 139142 (0.0005) [2023-12-27 06:37:18,839][20381] Updated weights for policy 1, policy_version 139152 (0.0006) [2023-12-27 06:37:19,029][20347] Updated weights for policy 0, policy_version 139441 (0.0008) [2023-12-27 06:37:19,084][20347] Updated weights for policy 0, policy_version 139451 (0.0007) [2023-12-27 06:37:19,136][20347] Updated weights for policy 0, policy_version 139461 (0.0008) [2023-12-27 06:37:19,421][20381] Updated weights for policy 1, policy_version 139162 (0.0006) [2023-12-27 06:37:19,482][20381] Updated weights for policy 1, policy_version 139172 (0.0006) [2023-12-27 06:37:19,545][20381] Updated weights for policy 1, policy_version 139182 (0.0008) [2023-12-27 06:37:19,605][20381] Updated weights for policy 1, policy_version 139192 (0.0008) [2023-12-27 06:37:19,858][20347] Updated weights for policy 0, policy_version 139471 (0.0008) [2023-12-27 06:37:19,919][20347] Updated weights for policy 0, policy_version 139481 (0.0007) [2023-12-27 06:37:19,979][20347] Updated weights for policy 0, policy_version 139491 (0.0007) [2023-12-27 06:37:20,321][20381] Updated weights for policy 1, policy_version 139202 (0.0008) [2023-12-27 06:37:20,380][20381] Updated weights for policy 1, policy_version 139212 (0.0008) [2023-12-27 06:37:20,440][20381] Updated weights for policy 1, policy_version 139222 (0.0007) [2023-12-27 06:37:20,691][20347] Updated weights for policy 0, policy_version 139501 (0.0007) [2023-12-27 06:37:20,752][20347] Updated weights for policy 0, policy_version 139511 (0.0007) [2023-12-27 06:37:20,808][20347] Updated weights for policy 0, policy_version 139521 (0.0008) [2023-12-27 06:37:21,078][20381] Updated weights for policy 1, policy_version 139232 (0.0007) [2023-12-27 06:37:21,140][20381] Updated weights for policy 1, policy_version 139242 (0.0008) [2023-12-27 06:37:21,197][20381] Updated weights for policy 1, policy_version 139252 (0.0006) [2023-12-27 06:37:21,588][20347] Updated weights for policy 0, policy_version 139531 (0.0008) [2023-12-27 06:37:21,651][20347] Updated weights for policy 0, policy_version 139541 (0.0007) [2023-12-27 06:37:21,711][20347] Updated weights for policy 0, policy_version 139551 (0.0007) [2023-12-27 06:37:21,801][20381] Updated weights for policy 1, policy_version 139262 (0.0008) [2023-12-27 06:37:21,854][20381] Updated weights for policy 1, policy_version 139272 (0.0007) [2023-12-27 06:37:21,907][20381] Updated weights for policy 1, policy_version 139282 (0.0008) [2023-12-27 06:37:22,359][20347] Updated weights for policy 0, policy_version 139561 (0.0009) [2023-12-27 06:37:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 71409664. Throughput: 0: 10500.8, 1: 10299.4. Samples: 71381224. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000139288_35667968.pth... [2023-12-27 06:37:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000138072_35356672.pth [2023-12-27 06:37:22,416][20347] Updated weights for policy 0, policy_version 139571 (0.0007) [2023-12-27 06:37:22,463][20347] Updated weights for policy 0, policy_version 139581 (0.0008) [2023-12-27 06:37:22,516][20347] Updated weights for policy 0, policy_version 139591 (0.0009) [2023-12-27 06:37:22,519][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000139592_35749888.pth... [2023-12-27 06:37:22,522][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000138344_35430400.pth [2023-12-27 06:37:22,583][20381] Updated weights for policy 1, policy_version 139292 (0.0007) [2023-12-27 06:37:22,637][20381] Updated weights for policy 1, policy_version 139302 (0.0005) [2023-12-27 06:37:22,696][20381] Updated weights for policy 1, policy_version 139312 (0.0005) [2023-12-27 06:37:23,244][20381] Updated weights for policy 1, policy_version 139322 (0.0008) [2023-12-27 06:37:23,248][20347] Updated weights for policy 0, policy_version 139601 (0.0008) [2023-12-27 06:37:23,292][20381] Updated weights for policy 1, policy_version 139332 (0.0010) [2023-12-27 06:37:23,295][20347] Updated weights for policy 0, policy_version 139611 (0.0007) [2023-12-27 06:37:23,347][20381] Updated weights for policy 1, policy_version 139342 (0.0010) [2023-12-27 06:37:23,348][20347] Updated weights for policy 0, policy_version 139621 (0.0007) [2023-12-27 06:37:23,402][20381] Updated weights for policy 1, policy_version 139352 (0.0011) [2023-12-27 06:37:24,055][20347] Updated weights for policy 0, policy_version 139631 (0.0008) [2023-12-27 06:37:24,087][20381] Updated weights for policy 1, policy_version 139362 (0.0010) [2023-12-27 06:37:24,113][20347] Updated weights for policy 0, policy_version 139641 (0.0007) [2023-12-27 06:37:24,146][20381] Updated weights for policy 1, policy_version 139372 (0.0006) [2023-12-27 06:37:24,172][20347] Updated weights for policy 0, policy_version 139651 (0.0005) [2023-12-27 06:37:24,213][20381] Updated weights for policy 1, policy_version 139382 (0.0006) [2023-12-27 06:37:24,685][20347] Updated weights for policy 0, policy_version 139661 (0.0005) [2023-12-27 06:37:24,743][20347] Updated weights for policy 0, policy_version 139671 (0.0005) [2023-12-27 06:37:24,789][20347] Updated weights for policy 0, policy_version 139681 (0.0009) [2023-12-27 06:37:24,820][20381] Updated weights for policy 1, policy_version 139392 (0.0006) [2023-12-27 06:37:24,877][20381] Updated weights for policy 1, policy_version 139402 (0.0006) [2023-12-27 06:37:24,930][20381] Updated weights for policy 1, policy_version 139412 (0.0006) [2023-12-27 06:37:25,388][20347] Updated weights for policy 0, policy_version 139691 (0.0010) [2023-12-27 06:37:25,433][20347] Updated weights for policy 0, policy_version 139701 (0.0010) [2023-12-27 06:37:25,480][20347] Updated weights for policy 0, policy_version 139711 (0.0010) [2023-12-27 06:37:25,550][20381] Updated weights for policy 1, policy_version 139422 (0.0007) [2023-12-27 06:37:25,605][20381] Updated weights for policy 1, policy_version 139432 (0.0006) [2023-12-27 06:37:25,667][20381] Updated weights for policy 1, policy_version 139442 (0.0005) [2023-12-27 06:37:26,102][20347] Updated weights for policy 0, policy_version 139721 (0.0010) [2023-12-27 06:37:26,160][20347] Updated weights for policy 0, policy_version 139731 (0.0010) [2023-12-27 06:37:26,223][20347] Updated weights for policy 0, policy_version 139741 (0.0009) [2023-12-27 06:37:26,280][20347] Updated weights for policy 0, policy_version 139751 (0.0010) [2023-12-27 06:37:26,346][20381] Updated weights for policy 1, policy_version 139452 (0.0007) [2023-12-27 06:37:26,399][20381] Updated weights for policy 1, policy_version 139462 (0.0005) [2023-12-27 06:37:26,447][20381] Updated weights for policy 1, policy_version 139472 (0.0005) [2023-12-27 06:37:26,935][20347] Updated weights for policy 0, policy_version 139761 (0.0010) [2023-12-27 06:37:26,983][20347] Updated weights for policy 0, policy_version 139771 (0.0010) [2023-12-27 06:37:27,036][20347] Updated weights for policy 0, policy_version 139781 (0.0010) [2023-12-27 06:37:27,077][20381] Updated weights for policy 1, policy_version 139482 (0.0006) [2023-12-27 06:37:27,123][20381] Updated weights for policy 1, policy_version 139492 (0.0007) [2023-12-27 06:37:27,172][20381] Updated weights for policy 1, policy_version 139502 (0.0007) [2023-12-27 06:37:27,222][20381] Updated weights for policy 1, policy_version 139512 (0.0007) [2023-12-27 06:37:27,363][19170] Fps is (10 sec: 22118.1, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 71524352. Throughput: 0: 10500.1, 1: 10342.7. Samples: 71508616. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 06:37:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:27,719][20347] Updated weights for policy 0, policy_version 139791 (0.0009) [2023-12-27 06:37:27,772][20347] Updated weights for policy 0, policy_version 139801 (0.0005) [2023-12-27 06:37:27,822][20347] Updated weights for policy 0, policy_version 139811 (0.0005) [2023-12-27 06:37:27,884][20381] Updated weights for policy 1, policy_version 139522 (0.0009) [2023-12-27 06:37:27,932][20381] Updated weights for policy 1, policy_version 139532 (0.0007) [2023-12-27 06:37:27,983][20381] Updated weights for policy 1, policy_version 139542 (0.0008) [2023-12-27 06:37:28,421][20347] Updated weights for policy 0, policy_version 139821 (0.0008) [2023-12-27 06:37:28,474][20347] Updated weights for policy 0, policy_version 139831 (0.0010) [2023-12-27 06:37:28,526][20347] Updated weights for policy 0, policy_version 139841 (0.0010) [2023-12-27 06:37:28,669][20381] Updated weights for policy 1, policy_version 139552 (0.0008) [2023-12-27 06:37:28,732][20381] Updated weights for policy 1, policy_version 139562 (0.0007) [2023-12-27 06:37:28,785][20381] Updated weights for policy 1, policy_version 139572 (0.0007) [2023-12-27 06:37:29,216][20347] Updated weights for policy 0, policy_version 139851 (0.0009) [2023-12-27 06:37:29,273][20347] Updated weights for policy 0, policy_version 139861 (0.0007) [2023-12-27 06:37:29,332][20347] Updated weights for policy 0, policy_version 139871 (0.0006) [2023-12-27 06:37:29,470][20381] Updated weights for policy 1, policy_version 139582 (0.0008) [2023-12-27 06:37:29,537][20381] Updated weights for policy 1, policy_version 139592 (0.0007) [2023-12-27 06:37:29,598][20381] Updated weights for policy 1, policy_version 139602 (0.0008) [2023-12-27 06:37:29,989][20347] Updated weights for policy 0, policy_version 139881 (0.0008) [2023-12-27 06:37:30,051][20347] Updated weights for policy 0, policy_version 139891 (0.0007) [2023-12-27 06:37:30,099][20347] Updated weights for policy 0, policy_version 139901 (0.0006) [2023-12-27 06:37:30,154][20347] Updated weights for policy 0, policy_version 139911 (0.0005) [2023-12-27 06:37:30,318][20381] Updated weights for policy 1, policy_version 139612 (0.0007) [2023-12-27 06:37:30,383][20381] Updated weights for policy 1, policy_version 139622 (0.0008) [2023-12-27 06:37:30,451][20381] Updated weights for policy 1, policy_version 139632 (0.0009) [2023-12-27 06:37:30,753][20347] Updated weights for policy 0, policy_version 139921 (0.0008) [2023-12-27 06:37:30,830][20347] Updated weights for policy 0, policy_version 139931 (0.0007) [2023-12-27 06:37:30,879][20347] Updated weights for policy 0, policy_version 139941 (0.0007) [2023-12-27 06:37:31,132][20381] Updated weights for policy 1, policy_version 139642 (0.0008) [2023-12-27 06:37:31,186][20381] Updated weights for policy 1, policy_version 139652 (0.0009) [2023-12-27 06:37:31,251][20381] Updated weights for policy 1, policy_version 139662 (0.0006) [2023-12-27 06:37:31,303][20381] Updated weights for policy 1, policy_version 139672 (0.0008) [2023-12-27 06:37:31,559][20347] Updated weights for policy 0, policy_version 139951 (0.0010) [2023-12-27 06:37:31,619][20347] Updated weights for policy 0, policy_version 139961 (0.0012) [2023-12-27 06:37:31,679][20347] Updated weights for policy 0, policy_version 139971 (0.0007) [2023-12-27 06:37:32,053][20381] Updated weights for policy 1, policy_version 139682 (0.0008) [2023-12-27 06:37:32,109][20381] Updated weights for policy 1, policy_version 139692 (0.0006) [2023-12-27 06:37:32,168][20381] Updated weights for policy 1, policy_version 139702 (0.0006) [2023-12-27 06:37:32,344][20347] Updated weights for policy 0, policy_version 139981 (0.0008) [2023-12-27 06:37:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 71622656. Throughput: 0: 10526.3, 1: 10326.3. Samples: 71632796. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:32,403][20347] Updated weights for policy 0, policy_version 139991 (0.0009) [2023-12-27 06:37:32,449][20347] Updated weights for policy 0, policy_version 140001 (0.0010) [2023-12-27 06:37:32,827][20381] Updated weights for policy 1, policy_version 139712 (0.0007) [2023-12-27 06:37:32,881][20381] Updated weights for policy 1, policy_version 139722 (0.0008) [2023-12-27 06:37:32,926][20381] Updated weights for policy 1, policy_version 139732 (0.0007) [2023-12-27 06:37:33,131][20347] Updated weights for policy 0, policy_version 140011 (0.0010) [2023-12-27 06:37:33,179][20347] Updated weights for policy 0, policy_version 140021 (0.0010) [2023-12-27 06:37:33,226][20347] Updated weights for policy 0, policy_version 140031 (0.0010) [2023-12-27 06:37:33,616][20381] Updated weights for policy 1, policy_version 139742 (0.0006) [2023-12-27 06:37:33,667][20381] Updated weights for policy 1, policy_version 139752 (0.0005) [2023-12-27 06:37:33,710][20381] Updated weights for policy 1, policy_version 139762 (0.0005) [2023-12-27 06:37:33,841][20347] Updated weights for policy 0, policy_version 140041 (0.0010) [2023-12-27 06:37:33,889][20347] Updated weights for policy 0, policy_version 140051 (0.0005) [2023-12-27 06:37:33,935][20347] Updated weights for policy 0, policy_version 140061 (0.0005) [2023-12-27 06:37:33,990][20347] Updated weights for policy 0, policy_version 140071 (0.0006) [2023-12-27 06:37:34,382][20381] Updated weights for policy 1, policy_version 139772 (0.0007) [2023-12-27 06:37:34,429][20381] Updated weights for policy 1, policy_version 139782 (0.0008) [2023-12-27 06:37:34,483][20381] Updated weights for policy 1, policy_version 139792 (0.0008) [2023-12-27 06:37:34,655][20347] Updated weights for policy 0, policy_version 140081 (0.0007) [2023-12-27 06:37:34,715][20347] Updated weights for policy 0, policy_version 140091 (0.0007) [2023-12-27 06:37:34,773][20347] Updated weights for policy 0, policy_version 140101 (0.0006) [2023-12-27 06:37:35,168][20381] Updated weights for policy 1, policy_version 139802 (0.0007) [2023-12-27 06:37:35,229][20381] Updated weights for policy 1, policy_version 139812 (0.0006) [2023-12-27 06:37:35,283][20381] Updated weights for policy 1, policy_version 139822 (0.0005) [2023-12-27 06:37:35,346][20381] Updated weights for policy 1, policy_version 139832 (0.0005) [2023-12-27 06:37:35,371][20347] Updated weights for policy 0, policy_version 140111 (0.0008) [2023-12-27 06:37:35,425][20347] Updated weights for policy 0, policy_version 140121 (0.0008) [2023-12-27 06:37:35,477][20347] Updated weights for policy 0, policy_version 140131 (0.0009) [2023-12-27 06:37:35,890][20381] Updated weights for policy 1, policy_version 139842 (0.0007) [2023-12-27 06:37:35,948][20381] Updated weights for policy 1, policy_version 139852 (0.0007) [2023-12-27 06:37:36,000][20381] Updated weights for policy 1, policy_version 139862 (0.0006) [2023-12-27 06:37:36,169][20347] Updated weights for policy 0, policy_version 140141 (0.0006) [2023-12-27 06:37:36,228][20347] Updated weights for policy 0, policy_version 140151 (0.0008) [2023-12-27 06:37:36,278][20347] Updated weights for policy 0, policy_version 140161 (0.0008) [2023-12-27 06:37:36,680][20381] Updated weights for policy 1, policy_version 139872 (0.0007) [2023-12-27 06:37:36,736][20381] Updated weights for policy 1, policy_version 139882 (0.0008) [2023-12-27 06:37:36,795][20381] Updated weights for policy 1, policy_version 139892 (0.0008) [2023-12-27 06:37:36,967][20347] Updated weights for policy 0, policy_version 140171 (0.0007) [2023-12-27 06:37:37,016][20347] Updated weights for policy 0, policy_version 140181 (0.0005) [2023-12-27 06:37:37,073][20347] Updated weights for policy 0, policy_version 140191 (0.0005) [2023-12-27 06:37:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 71729152. Throughput: 0: 10516.6, 1: 10388.6. Samples: 71696352. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000139896_35823616.pth... [2023-12-27 06:37:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000140200_35905536.pth... [2023-12-27 06:37:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000138680_35512320.pth [2023-12-27 06:37:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000138984_35594240.pth [2023-12-27 06:37:37,533][20381] Updated weights for policy 1, policy_version 139902 (0.0006) [2023-12-27 06:37:37,590][20381] Updated weights for policy 1, policy_version 139912 (0.0007) [2023-12-27 06:37:37,619][20347] Updated weights for policy 0, policy_version 140201 (0.0005) [2023-12-27 06:37:37,648][20381] Updated weights for policy 1, policy_version 139922 (0.0007) [2023-12-27 06:37:37,666][20347] Updated weights for policy 0, policy_version 140211 (0.0005) [2023-12-27 06:37:37,720][20347] Updated weights for policy 0, policy_version 140221 (0.0008) [2023-12-27 06:37:37,766][20347] Updated weights for policy 0, policy_version 140231 (0.0008) [2023-12-27 06:37:38,324][20381] Updated weights for policy 1, policy_version 139932 (0.0008) [2023-12-27 06:37:38,383][20381] Updated weights for policy 1, policy_version 139942 (0.0007) [2023-12-27 06:37:38,446][20347] Updated weights for policy 0, policy_version 140241 (0.0006) [2023-12-27 06:37:38,450][20381] Updated weights for policy 1, policy_version 139952 (0.0007) [2023-12-27 06:37:38,506][20347] Updated weights for policy 0, policy_version 140251 (0.0005) [2023-12-27 06:37:38,560][20347] Updated weights for policy 0, policy_version 140261 (0.0005) [2023-12-27 06:37:39,128][20381] Updated weights for policy 1, policy_version 139962 (0.0006) [2023-12-27 06:37:39,186][20381] Updated weights for policy 1, policy_version 139972 (0.0007) [2023-12-27 06:37:39,199][20347] Updated weights for policy 0, policy_version 140271 (0.0006) [2023-12-27 06:37:39,249][20381] Updated weights for policy 1, policy_version 139982 (0.0008) [2023-12-27 06:37:39,260][20347] Updated weights for policy 0, policy_version 140281 (0.0006) [2023-12-27 06:37:39,304][20381] Updated weights for policy 1, policy_version 139992 (0.0006) [2023-12-27 06:37:39,319][20347] Updated weights for policy 0, policy_version 140291 (0.0009) [2023-12-27 06:37:40,017][20347] Updated weights for policy 0, policy_version 140301 (0.0008) [2023-12-27 06:37:40,022][20381] Updated weights for policy 1, policy_version 140002 (0.0007) [2023-12-27 06:37:40,076][20347] Updated weights for policy 0, policy_version 140311 (0.0007) [2023-12-27 06:37:40,081][20381] Updated weights for policy 1, policy_version 140012 (0.0006) [2023-12-27 06:37:40,132][20347] Updated weights for policy 0, policy_version 140321 (0.0005) [2023-12-27 06:37:40,143][20381] Updated weights for policy 1, policy_version 140022 (0.0007) [2023-12-27 06:37:40,747][20381] Updated weights for policy 1, policy_version 140032 (0.0008) [2023-12-27 06:37:40,801][20381] Updated weights for policy 1, policy_version 140042 (0.0008) [2023-12-27 06:37:40,850][20381] Updated weights for policy 1, policy_version 140052 (0.0009) [2023-12-27 06:37:40,860][20347] Updated weights for policy 0, policy_version 140331 (0.0006) [2023-12-27 06:37:40,919][20347] Updated weights for policy 0, policy_version 140341 (0.0007) [2023-12-27 06:37:40,971][20347] Updated weights for policy 0, policy_version 140351 (0.0007) [2023-12-27 06:37:41,589][20381] Updated weights for policy 1, policy_version 140062 (0.0008) [2023-12-27 06:37:41,653][20381] Updated weights for policy 1, policy_version 140072 (0.0007) [2023-12-27 06:37:41,723][20381] Updated weights for policy 1, policy_version 140082 (0.0007) [2023-12-27 06:37:41,730][20347] Updated weights for policy 0, policy_version 140361 (0.0008) [2023-12-27 06:37:41,793][20347] Updated weights for policy 0, policy_version 140371 (0.0005) [2023-12-27 06:37:41,854][20347] Updated weights for policy 0, policy_version 140381 (0.0007) [2023-12-27 06:37:41,914][20347] Updated weights for policy 0, policy_version 140391 (0.0006) [2023-12-27 06:37:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 71827456. Throughput: 0: 10509.7, 1: 10381.9. Samples: 71820340. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:42,439][20381] Updated weights for policy 1, policy_version 140092 (0.0007) [2023-12-27 06:37:42,497][20381] Updated weights for policy 1, policy_version 140102 (0.0007) [2023-12-27 06:37:42,525][20347] Updated weights for policy 0, policy_version 140401 (0.0005) [2023-12-27 06:37:42,553][20381] Updated weights for policy 1, policy_version 140112 (0.0006) [2023-12-27 06:37:42,578][20347] Updated weights for policy 0, policy_version 140411 (0.0005) [2023-12-27 06:37:42,629][20347] Updated weights for policy 0, policy_version 140421 (0.0007) [2023-12-27 06:37:43,154][20381] Updated weights for policy 1, policy_version 140122 (0.0006) [2023-12-27 06:37:43,210][20381] Updated weights for policy 1, policy_version 140132 (0.0007) [2023-12-27 06:37:43,258][20381] Updated weights for policy 1, policy_version 140142 (0.0009) [2023-12-27 06:37:43,306][20381] Updated weights for policy 1, policy_version 140152 (0.0009) [2023-12-27 06:37:43,341][20347] Updated weights for policy 0, policy_version 140431 (0.0007) [2023-12-27 06:37:43,401][20347] Updated weights for policy 0, policy_version 140441 (0.0009) [2023-12-27 06:37:43,459][20347] Updated weights for policy 0, policy_version 140451 (0.0006) [2023-12-27 06:37:44,000][20381] Updated weights for policy 1, policy_version 140162 (0.0010) [2023-12-27 06:37:44,049][20347] Updated weights for policy 0, policy_version 140461 (0.0006) [2023-12-27 06:37:44,067][20381] Updated weights for policy 1, policy_version 140172 (0.0010) [2023-12-27 06:37:44,103][20347] Updated weights for policy 0, policy_version 140471 (0.0007) [2023-12-27 06:37:44,132][20381] Updated weights for policy 1, policy_version 140182 (0.0010) [2023-12-27 06:37:44,156][20347] Updated weights for policy 0, policy_version 140481 (0.0006) [2023-12-27 06:37:44,761][20381] Updated weights for policy 1, policy_version 140192 (0.0008) [2023-12-27 06:37:44,825][20381] Updated weights for policy 1, policy_version 140202 (0.0006) [2023-12-27 06:37:44,826][20347] Updated weights for policy 0, policy_version 140491 (0.0009) [2023-12-27 06:37:44,886][20381] Updated weights for policy 1, policy_version 140212 (0.0006) [2023-12-27 06:37:44,887][20347] Updated weights for policy 0, policy_version 140501 (0.0006) [2023-12-27 06:37:44,947][20347] Updated weights for policy 0, policy_version 140511 (0.0006) [2023-12-27 06:37:45,559][20347] Updated weights for policy 0, policy_version 140521 (0.0007) [2023-12-27 06:37:45,577][20381] Updated weights for policy 1, policy_version 140222 (0.0006) [2023-12-27 06:37:45,613][20347] Updated weights for policy 0, policy_version 140531 (0.0006) [2023-12-27 06:37:45,636][20381] Updated weights for policy 1, policy_version 140232 (0.0006) [2023-12-27 06:37:45,657][20347] Updated weights for policy 0, policy_version 140541 (0.0006) [2023-12-27 06:37:45,690][20381] Updated weights for policy 1, policy_version 140242 (0.0007) [2023-12-27 06:37:45,703][20347] Updated weights for policy 0, policy_version 140551 (0.0006) [2023-12-27 06:37:46,322][20381] Updated weights for policy 1, policy_version 140252 (0.0008) [2023-12-27 06:37:46,383][20381] Updated weights for policy 1, policy_version 140262 (0.0009) [2023-12-27 06:37:46,395][20347] Updated weights for policy 0, policy_version 140561 (0.0006) [2023-12-27 06:37:46,442][20381] Updated weights for policy 1, policy_version 140272 (0.0007) [2023-12-27 06:37:46,456][20347] Updated weights for policy 0, policy_version 140571 (0.0006) [2023-12-27 06:37:46,508][20347] Updated weights for policy 0, policy_version 140581 (0.0006) [2023-12-27 06:37:47,086][20381] Updated weights for policy 1, policy_version 140282 (0.0008) [2023-12-27 06:37:47,137][20347] Updated weights for policy 0, policy_version 140591 (0.0007) [2023-12-27 06:37:47,143][20381] Updated weights for policy 1, policy_version 140292 (0.0007) [2023-12-27 06:37:47,186][20347] Updated weights for policy 0, policy_version 140601 (0.0005) [2023-12-27 06:37:47,195][20381] Updated weights for policy 1, policy_version 140302 (0.0006) [2023-12-27 06:37:47,235][20347] Updated weights for policy 0, policy_version 140611 (0.0006) [2023-12-27 06:37:47,244][20381] Updated weights for policy 1, policy_version 140312 (0.0005) [2023-12-27 06:37:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 71942144. Throughput: 0: 10523.0, 1: 10423.5. Samples: 71945772. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:47,898][20347] Updated weights for policy 0, policy_version 140621 (0.0006) [2023-12-27 06:37:47,900][20381] Updated weights for policy 1, policy_version 140322 (0.0007) [2023-12-27 06:37:47,948][20347] Updated weights for policy 0, policy_version 140631 (0.0006) [2023-12-27 06:37:47,954][20381] Updated weights for policy 1, policy_version 140332 (0.0005) [2023-12-27 06:37:47,998][20347] Updated weights for policy 0, policy_version 140641 (0.0007) [2023-12-27 06:37:48,007][20381] Updated weights for policy 1, policy_version 140342 (0.0006) [2023-12-27 06:37:48,677][20347] Updated weights for policy 0, policy_version 140651 (0.0007) [2023-12-27 06:37:48,706][20381] Updated weights for policy 1, policy_version 140352 (0.0006) [2023-12-27 06:37:48,736][20347] Updated weights for policy 0, policy_version 140661 (0.0007) [2023-12-27 06:37:48,753][20381] Updated weights for policy 1, policy_version 140362 (0.0007) [2023-12-27 06:37:48,794][20347] Updated weights for policy 0, policy_version 140671 (0.0007) [2023-12-27 06:37:48,815][20381] Updated weights for policy 1, policy_version 140372 (0.0006) [2023-12-27 06:37:49,510][20347] Updated weights for policy 0, policy_version 140681 (0.0007) [2023-12-27 06:37:49,535][20381] Updated weights for policy 1, policy_version 140382 (0.0008) [2023-12-27 06:37:49,572][20347] Updated weights for policy 0, policy_version 140691 (0.0005) [2023-12-27 06:37:49,590][20381] Updated weights for policy 1, policy_version 140392 (0.0010) [2023-12-27 06:37:49,634][20347] Updated weights for policy 0, policy_version 140701 (0.0005) [2023-12-27 06:37:49,648][20381] Updated weights for policy 1, policy_version 140402 (0.0010) [2023-12-27 06:37:49,691][20347] Updated weights for policy 0, policy_version 140711 (0.0006) [2023-12-27 06:37:50,316][20381] Updated weights for policy 1, policy_version 140412 (0.0011) [2023-12-27 06:37:50,380][20381] Updated weights for policy 1, policy_version 140422 (0.0010) [2023-12-27 06:37:50,383][20347] Updated weights for policy 0, policy_version 140721 (0.0006) [2023-12-27 06:37:50,431][20381] Updated weights for policy 1, policy_version 140432 (0.0008) [2023-12-27 06:37:50,437][20347] Updated weights for policy 0, policy_version 140731 (0.0006) [2023-12-27 06:37:50,488][20347] Updated weights for policy 0, policy_version 140741 (0.0008) [2023-12-27 06:37:51,017][20381] Updated weights for policy 1, policy_version 140442 (0.0006) [2023-12-27 06:37:51,076][20381] Updated weights for policy 1, policy_version 140452 (0.0008) [2023-12-27 06:37:51,133][20381] Updated weights for policy 1, policy_version 140462 (0.0008) [2023-12-27 06:37:51,179][20381] Updated weights for policy 1, policy_version 140472 (0.0006) [2023-12-27 06:37:51,265][20347] Updated weights for policy 0, policy_version 140751 (0.0008) [2023-12-27 06:37:51,325][20347] Updated weights for policy 0, policy_version 140761 (0.0007) [2023-12-27 06:37:51,389][20347] Updated weights for policy 0, policy_version 140771 (0.0009) [2023-12-27 06:37:51,890][20381] Updated weights for policy 1, policy_version 140482 (0.0012) [2023-12-27 06:37:51,950][20381] Updated weights for policy 1, policy_version 140492 (0.0011) [2023-12-27 06:37:52,013][20381] Updated weights for policy 1, policy_version 140502 (0.0010) [2023-12-27 06:37:52,117][20347] Updated weights for policy 0, policy_version 140781 (0.0008) [2023-12-27 06:37:52,174][20347] Updated weights for policy 0, policy_version 140791 (0.0008) [2023-12-27 06:37:52,228][20347] Updated weights for policy 0, policy_version 140801 (0.0008) [2023-12-27 06:37:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.7, 300 sec: 20799.3). Total num frames: 72040448. Throughput: 0: 10482.5, 1: 10367.4. Samples: 72007212. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000140504_35979264.pth... [2023-12-27 06:37:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000140808_36061184.pth... [2023-12-27 06:37:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000139592_35749888.pth [2023-12-27 06:37:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000139288_35667968.pth [2023-12-27 06:37:52,706][20381] Updated weights for policy 1, policy_version 140512 (0.0008) [2023-12-27 06:37:52,759][20381] Updated weights for policy 1, policy_version 140522 (0.0007) [2023-12-27 06:37:52,817][20381] Updated weights for policy 1, policy_version 140532 (0.0008) [2023-12-27 06:37:52,921][20347] Updated weights for policy 0, policy_version 140811 (0.0008) [2023-12-27 06:37:52,978][20347] Updated weights for policy 0, policy_version 140821 (0.0007) [2023-12-27 06:37:53,035][20347] Updated weights for policy 0, policy_version 140831 (0.0007) [2023-12-27 06:37:53,501][20381] Updated weights for policy 1, policy_version 140542 (0.0009) [2023-12-27 06:37:53,563][20381] Updated weights for policy 1, policy_version 140552 (0.0007) [2023-12-27 06:37:53,611][20381] Updated weights for policy 1, policy_version 140562 (0.0009) [2023-12-27 06:37:53,627][20347] Updated weights for policy 0, policy_version 140841 (0.0005) [2023-12-27 06:37:53,680][20347] Updated weights for policy 0, policy_version 140851 (0.0005) [2023-12-27 06:37:53,733][20347] Updated weights for policy 0, policy_version 140861 (0.0005) [2023-12-27 06:37:53,790][20347] Updated weights for policy 0, policy_version 140871 (0.0010) [2023-12-27 06:37:54,277][20381] Updated weights for policy 1, policy_version 140572 (0.0008) [2023-12-27 06:37:54,338][20381] Updated weights for policy 1, policy_version 140582 (0.0010) [2023-12-27 06:37:54,393][20381] Updated weights for policy 1, policy_version 140592 (0.0010) [2023-12-27 06:37:54,429][20347] Updated weights for policy 0, policy_version 140881 (0.0006) [2023-12-27 06:37:54,476][20347] Updated weights for policy 0, policy_version 140891 (0.0007) [2023-12-27 06:37:54,528][20347] Updated weights for policy 0, policy_version 140901 (0.0005) [2023-12-27 06:37:55,065][20381] Updated weights for policy 1, policy_version 140602 (0.0010) [2023-12-27 06:37:55,093][20347] Updated weights for policy 0, policy_version 140911 (0.0006) [2023-12-27 06:37:55,118][20381] Updated weights for policy 1, policy_version 140612 (0.0011) [2023-12-27 06:37:55,142][20347] Updated weights for policy 0, policy_version 140921 (0.0005) [2023-12-27 06:37:55,173][20381] Updated weights for policy 1, policy_version 140622 (0.0010) [2023-12-27 06:37:55,200][20347] Updated weights for policy 0, policy_version 140931 (0.0005) [2023-12-27 06:37:55,227][20381] Updated weights for policy 1, policy_version 140632 (0.0010) [2023-12-27 06:37:55,840][20381] Updated weights for policy 1, policy_version 140642 (0.0010) [2023-12-27 06:37:55,900][20381] Updated weights for policy 1, policy_version 140652 (0.0010) [2023-12-27 06:37:55,916][20347] Updated weights for policy 0, policy_version 140941 (0.0005) [2023-12-27 06:37:55,957][20381] Updated weights for policy 1, policy_version 140662 (0.0010) [2023-12-27 06:37:55,969][20347] Updated weights for policy 0, policy_version 140951 (0.0008) [2023-12-27 06:37:56,019][20347] Updated weights for policy 0, policy_version 140961 (0.0007) [2023-12-27 06:37:56,569][20381] Updated weights for policy 1, policy_version 140672 (0.0011) [2023-12-27 06:37:56,623][20381] Updated weights for policy 1, policy_version 140682 (0.0010) [2023-12-27 06:37:56,671][20381] Updated weights for policy 1, policy_version 140692 (0.0010) [2023-12-27 06:37:56,694][20347] Updated weights for policy 0, policy_version 140971 (0.0008) [2023-12-27 06:37:56,756][20347] Updated weights for policy 0, policy_version 140981 (0.0008) [2023-12-27 06:37:56,822][20347] Updated weights for policy 0, policy_version 140991 (0.0008) [2023-12-27 06:37:57,310][20381] Updated weights for policy 1, policy_version 140702 (0.0007) [2023-12-27 06:37:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 72138752. Throughput: 0: 10409.4, 1: 10405.3. Samples: 72131844. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:37:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:37:57,368][20381] Updated weights for policy 1, policy_version 140712 (0.0005) [2023-12-27 06:37:57,375][20347] Updated weights for policy 0, policy_version 141001 (0.0008) [2023-12-27 06:37:57,417][20381] Updated weights for policy 1, policy_version 140722 (0.0005) [2023-12-27 06:37:57,420][20347] Updated weights for policy 0, policy_version 141011 (0.0010) [2023-12-27 06:37:57,478][20347] Updated weights for policy 0, policy_version 141021 (0.0010) [2023-12-27 06:37:57,529][20347] Updated weights for policy 0, policy_version 141031 (0.0010) [2023-12-27 06:37:57,950][20381] Updated weights for policy 1, policy_version 140732 (0.0006) [2023-12-27 06:37:58,004][20381] Updated weights for policy 1, policy_version 140742 (0.0007) [2023-12-27 06:37:58,054][20381] Updated weights for policy 1, policy_version 140752 (0.0005) [2023-12-27 06:37:58,210][20347] Updated weights for policy 0, policy_version 141041 (0.0010) [2023-12-27 06:37:58,269][20347] Updated weights for policy 0, policy_version 141051 (0.0010) [2023-12-27 06:37:58,331][20347] Updated weights for policy 0, policy_version 141061 (0.0009) [2023-12-27 06:37:58,763][20381] Updated weights for policy 1, policy_version 140762 (0.0006) [2023-12-27 06:37:58,821][20381] Updated weights for policy 1, policy_version 140772 (0.0011) [2023-12-27 06:37:58,888][20381] Updated weights for policy 1, policy_version 140782 (0.0011) [2023-12-27 06:37:58,945][20381] Updated weights for policy 1, policy_version 140792 (0.0009) [2023-12-27 06:37:59,078][20347] Updated weights for policy 0, policy_version 141071 (0.0008) [2023-12-27 06:37:59,134][20347] Updated weights for policy 0, policy_version 141081 (0.0010) [2023-12-27 06:37:59,186][20347] Updated weights for policy 0, policy_version 141091 (0.0009) [2023-12-27 06:37:59,622][20381] Updated weights for policy 1, policy_version 140802 (0.0006) [2023-12-27 06:37:59,683][20381] Updated weights for policy 1, policy_version 140812 (0.0005) [2023-12-27 06:37:59,743][20381] Updated weights for policy 1, policy_version 140822 (0.0007) [2023-12-27 06:37:59,870][20347] Updated weights for policy 0, policy_version 141101 (0.0010) [2023-12-27 06:37:59,936][20347] Updated weights for policy 0, policy_version 141111 (0.0010) [2023-12-27 06:37:59,992][20347] Updated weights for policy 0, policy_version 141121 (0.0009) [2023-12-27 06:38:00,381][20381] Updated weights for policy 1, policy_version 140832 (0.0007) [2023-12-27 06:38:00,446][20381] Updated weights for policy 1, policy_version 140842 (0.0008) [2023-12-27 06:38:00,511][20381] Updated weights for policy 1, policy_version 140852 (0.0007) [2023-12-27 06:38:00,645][20347] Updated weights for policy 0, policy_version 141131 (0.0008) [2023-12-27 06:38:00,705][20347] Updated weights for policy 0, policy_version 141141 (0.0006) [2023-12-27 06:38:00,764][20347] Updated weights for policy 0, policy_version 141151 (0.0010) [2023-12-27 06:38:01,208][20381] Updated weights for policy 1, policy_version 140862 (0.0007) [2023-12-27 06:38:01,262][20381] Updated weights for policy 1, policy_version 140872 (0.0006) [2023-12-27 06:38:01,320][20381] Updated weights for policy 1, policy_version 140882 (0.0006) [2023-12-27 06:38:01,466][20347] Updated weights for policy 0, policy_version 141161 (0.0010) [2023-12-27 06:38:01,513][20347] Updated weights for policy 0, policy_version 141171 (0.0008) [2023-12-27 06:38:01,568][20347] Updated weights for policy 0, policy_version 141181 (0.0006) [2023-12-27 06:38:01,634][20347] Updated weights for policy 0, policy_version 141191 (0.0008) [2023-12-27 06:38:02,003][20381] Updated weights for policy 1, policy_version 140892 (0.0011) [2023-12-27 06:38:02,059][20381] Updated weights for policy 1, policy_version 140902 (0.0011) [2023-12-27 06:38:02,114][20381] Updated weights for policy 1, policy_version 140912 (0.0006) [2023-12-27 06:38:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 72245248. Throughput: 0: 10414.1, 1: 10427.5. Samples: 72256812. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:02,364][20347] Updated weights for policy 0, policy_version 141201 (0.0008) [2023-12-27 06:38:02,418][20347] Updated weights for policy 0, policy_version 141211 (0.0008) [2023-12-27 06:38:02,468][20347] Updated weights for policy 0, policy_version 141221 (0.0010) [2023-12-27 06:38:02,708][20381] Updated weights for policy 1, policy_version 140922 (0.0007) [2023-12-27 06:38:02,761][20381] Updated weights for policy 1, policy_version 140932 (0.0011) [2023-12-27 06:38:02,820][20381] Updated weights for policy 1, policy_version 140942 (0.0011) [2023-12-27 06:38:02,876][20381] Updated weights for policy 1, policy_version 140952 (0.0011) [2023-12-27 06:38:03,185][20347] Updated weights for policy 0, policy_version 141231 (0.0007) [2023-12-27 06:38:03,241][20347] Updated weights for policy 0, policy_version 141241 (0.0007) [2023-12-27 06:38:03,297][20347] Updated weights for policy 0, policy_version 141251 (0.0006) [2023-12-27 06:38:03,550][20381] Updated weights for policy 1, policy_version 140962 (0.0005) [2023-12-27 06:38:03,600][20381] Updated weights for policy 1, policy_version 140972 (0.0005) [2023-12-27 06:38:03,653][20381] Updated weights for policy 1, policy_version 140982 (0.0005) [2023-12-27 06:38:03,934][20347] Updated weights for policy 0, policy_version 141261 (0.0007) [2023-12-27 06:38:03,994][20347] Updated weights for policy 0, policy_version 141271 (0.0007) [2023-12-27 06:38:04,052][20347] Updated weights for policy 0, policy_version 141281 (0.0007) [2023-12-27 06:38:04,323][20381] Updated weights for policy 1, policy_version 140992 (0.0010) [2023-12-27 06:38:04,381][20381] Updated weights for policy 1, policy_version 141002 (0.0011) [2023-12-27 06:38:04,437][20381] Updated weights for policy 1, policy_version 141012 (0.0011) [2023-12-27 06:38:04,707][20347] Updated weights for policy 0, policy_version 141291 (0.0006) [2023-12-27 06:38:04,763][20347] Updated weights for policy 0, policy_version 141301 (0.0005) [2023-12-27 06:38:04,813][20347] Updated weights for policy 0, policy_version 141311 (0.0005) [2023-12-27 06:38:05,071][20381] Updated weights for policy 1, policy_version 141022 (0.0011) [2023-12-27 06:38:05,122][20381] Updated weights for policy 1, policy_version 141032 (0.0007) [2023-12-27 06:38:05,168][20381] Updated weights for policy 1, policy_version 141042 (0.0005) [2023-12-27 06:38:05,342][20347] Updated weights for policy 0, policy_version 141321 (0.0005) [2023-12-27 06:38:05,395][20347] Updated weights for policy 0, policy_version 141331 (0.0005) [2023-12-27 06:38:05,449][20347] Updated weights for policy 0, policy_version 141341 (0.0006) [2023-12-27 06:38:05,497][20347] Updated weights for policy 0, policy_version 141351 (0.0007) [2023-12-27 06:38:05,812][20381] Updated weights for policy 1, policy_version 141052 (0.0005) [2023-12-27 06:38:05,879][20381] Updated weights for policy 1, policy_version 141062 (0.0006) [2023-12-27 06:38:05,938][20381] Updated weights for policy 1, policy_version 141072 (0.0008) [2023-12-27 06:38:06,191][20347] Updated weights for policy 0, policy_version 141361 (0.0007) [2023-12-27 06:38:06,251][20347] Updated weights for policy 0, policy_version 141371 (0.0008) [2023-12-27 06:38:06,313][20347] Updated weights for policy 0, policy_version 141381 (0.0007) [2023-12-27 06:38:06,600][20381] Updated weights for policy 1, policy_version 141082 (0.0007) [2023-12-27 06:38:06,661][20381] Updated weights for policy 1, policy_version 141092 (0.0008) [2023-12-27 06:38:06,718][20381] Updated weights for policy 1, policy_version 141102 (0.0008) [2023-12-27 06:38:06,778][20381] Updated weights for policy 1, policy_version 141112 (0.0008) [2023-12-27 06:38:06,974][20347] Updated weights for policy 0, policy_version 141391 (0.0008) [2023-12-27 06:38:07,036][20347] Updated weights for policy 0, policy_version 141401 (0.0008) [2023-12-27 06:38:07,104][20347] Updated weights for policy 0, policy_version 141411 (0.0008) [2023-12-27 06:38:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 72351744. Throughput: 0: 10441.2, 1: 10436.0. Samples: 72320696. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000141416_36216832.pth... [2023-12-27 06:38:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000141112_36134912.pth... [2023-12-27 06:38:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000140200_35905536.pth [2023-12-27 06:38:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000139896_35823616.pth [2023-12-27 06:38:07,454][20381] Updated weights for policy 1, policy_version 141122 (0.0011) [2023-12-27 06:38:07,506][20381] Updated weights for policy 1, policy_version 141132 (0.0010) [2023-12-27 06:38:07,554][20381] Updated weights for policy 1, policy_version 141142 (0.0010) [2023-12-27 06:38:07,804][20347] Updated weights for policy 0, policy_version 141421 (0.0008) [2023-12-27 06:38:07,860][20347] Updated weights for policy 0, policy_version 141431 (0.0007) [2023-12-27 06:38:07,918][20347] Updated weights for policy 0, policy_version 141441 (0.0009) [2023-12-27 06:38:08,203][20381] Updated weights for policy 1, policy_version 141152 (0.0006) [2023-12-27 06:38:08,258][20381] Updated weights for policy 1, policy_version 141162 (0.0009) [2023-12-27 06:38:08,304][20381] Updated weights for policy 1, policy_version 141172 (0.0011) [2023-12-27 06:38:08,547][20347] Updated weights for policy 0, policy_version 141451 (0.0008) [2023-12-27 06:38:08,606][20347] Updated weights for policy 0, policy_version 141461 (0.0008) [2023-12-27 06:38:08,661][20347] Updated weights for policy 0, policy_version 141471 (0.0007) [2023-12-27 06:38:09,001][20381] Updated weights for policy 1, policy_version 141182 (0.0011) [2023-12-27 06:38:09,059][20381] Updated weights for policy 1, policy_version 141192 (0.0010) [2023-12-27 06:38:09,110][20381] Updated weights for policy 1, policy_version 141202 (0.0011) [2023-12-27 06:38:09,337][20347] Updated weights for policy 0, policy_version 141481 (0.0008) [2023-12-27 06:38:09,405][20347] Updated weights for policy 0, policy_version 141491 (0.0011) [2023-12-27 06:38:09,466][20347] Updated weights for policy 0, policy_version 141501 (0.0007) [2023-12-27 06:38:09,526][20347] Updated weights for policy 0, policy_version 141511 (0.0006) [2023-12-27 06:38:09,857][20381] Updated weights for policy 1, policy_version 141212 (0.0010) [2023-12-27 06:38:09,915][20381] Updated weights for policy 1, policy_version 141222 (0.0008) [2023-12-27 06:38:09,975][20381] Updated weights for policy 1, policy_version 141232 (0.0008) [2023-12-27 06:38:10,262][20347] Updated weights for policy 0, policy_version 141521 (0.0008) [2023-12-27 06:38:10,326][20347] Updated weights for policy 0, policy_version 141531 (0.0009) [2023-12-27 06:38:10,388][20347] Updated weights for policy 0, policy_version 141541 (0.0011) [2023-12-27 06:38:10,581][20381] Updated weights for policy 1, policy_version 141242 (0.0007) [2023-12-27 06:38:10,640][20381] Updated weights for policy 1, policy_version 141252 (0.0011) [2023-12-27 06:38:10,701][20381] Updated weights for policy 1, policy_version 141262 (0.0009) [2023-12-27 06:38:10,764][20381] Updated weights for policy 1, policy_version 141272 (0.0006) [2023-12-27 06:38:11,072][20347] Updated weights for policy 0, policy_version 141551 (0.0008) [2023-12-27 06:38:11,135][20347] Updated weights for policy 0, policy_version 141561 (0.0008) [2023-12-27 06:38:11,195][20347] Updated weights for policy 0, policy_version 141571 (0.0008) [2023-12-27 06:38:11,446][20381] Updated weights for policy 1, policy_version 141282 (0.0009) [2023-12-27 06:38:11,505][20381] Updated weights for policy 1, policy_version 141292 (0.0011) [2023-12-27 06:38:11,558][20381] Updated weights for policy 1, policy_version 141302 (0.0010) [2023-12-27 06:38:11,933][20347] Updated weights for policy 0, policy_version 141581 (0.0007) [2023-12-27 06:38:11,986][20347] Updated weights for policy 0, policy_version 141591 (0.0007) [2023-12-27 06:38:12,044][20347] Updated weights for policy 0, policy_version 141601 (0.0007) [2023-12-27 06:38:12,298][20381] Updated weights for policy 1, policy_version 141312 (0.0008) [2023-12-27 06:38:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 72450048. Throughput: 0: 10403.3, 1: 10382.8. Samples: 72443984. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:12,369][20381] Updated weights for policy 1, policy_version 141322 (0.0007) [2023-12-27 06:38:12,430][20381] Updated weights for policy 1, policy_version 141332 (0.0008) [2023-12-27 06:38:12,732][20347] Updated weights for policy 0, policy_version 141611 (0.0007) [2023-12-27 06:38:12,787][20347] Updated weights for policy 0, policy_version 141621 (0.0007) [2023-12-27 06:38:12,841][20347] Updated weights for policy 0, policy_version 141631 (0.0006) [2023-12-27 06:38:13,051][20381] Updated weights for policy 1, policy_version 141342 (0.0006) [2023-12-27 06:38:13,107][20381] Updated weights for policy 1, policy_version 141352 (0.0005) [2023-12-27 06:38:13,160][20381] Updated weights for policy 1, policy_version 141362 (0.0007) [2023-12-27 06:38:13,600][20347] Updated weights for policy 0, policy_version 141641 (0.0006) [2023-12-27 06:38:13,656][20347] Updated weights for policy 0, policy_version 141651 (0.0007) [2023-12-27 06:38:13,707][20381] Updated weights for policy 1, policy_version 141372 (0.0007) [2023-12-27 06:38:13,716][20347] Updated weights for policy 0, policy_version 141661 (0.0007) [2023-12-27 06:38:13,762][20381] Updated weights for policy 1, policy_version 141382 (0.0010) [2023-12-27 06:38:13,773][20347] Updated weights for policy 0, policy_version 141671 (0.0008) [2023-12-27 06:38:13,816][20381] Updated weights for policy 1, policy_version 141392 (0.0010) [2023-12-27 06:38:14,413][20347] Updated weights for policy 0, policy_version 141681 (0.0006) [2023-12-27 06:38:14,471][20347] Updated weights for policy 0, policy_version 141691 (0.0006) [2023-12-27 06:38:14,503][20381] Updated weights for policy 1, policy_version 141402 (0.0010) [2023-12-27 06:38:14,528][20347] Updated weights for policy 0, policy_version 141701 (0.0006) [2023-12-27 06:38:14,552][20381] Updated weights for policy 1, policy_version 141412 (0.0011) [2023-12-27 06:38:14,597][20381] Updated weights for policy 1, policy_version 141422 (0.0010) [2023-12-27 06:38:14,642][20381] Updated weights for policy 1, policy_version 141432 (0.0010) [2023-12-27 06:38:15,154][20347] Updated weights for policy 0, policy_version 141711 (0.0007) [2023-12-27 06:38:15,206][20347] Updated weights for policy 0, policy_version 141721 (0.0009) [2023-12-27 06:38:15,262][20347] Updated weights for policy 0, policy_version 141731 (0.0008) [2023-12-27 06:38:15,328][20381] Updated weights for policy 1, policy_version 141442 (0.0011) [2023-12-27 06:38:15,379][20381] Updated weights for policy 1, policy_version 141452 (0.0007) [2023-12-27 06:38:15,429][20381] Updated weights for policy 1, policy_version 141462 (0.0010) [2023-12-27 06:38:15,840][20347] Updated weights for policy 0, policy_version 141741 (0.0008) [2023-12-27 06:38:15,887][20347] Updated weights for policy 0, policy_version 141751 (0.0007) [2023-12-27 06:38:15,942][20347] Updated weights for policy 0, policy_version 141761 (0.0007) [2023-12-27 06:38:16,115][20381] Updated weights for policy 1, policy_version 141472 (0.0011) [2023-12-27 06:38:16,179][20381] Updated weights for policy 1, policy_version 141482 (0.0011) [2023-12-27 06:38:16,247][20381] Updated weights for policy 1, policy_version 141492 (0.0009) [2023-12-27 06:38:16,662][20347] Updated weights for policy 0, policy_version 141771 (0.0007) [2023-12-27 06:38:16,724][20347] Updated weights for policy 0, policy_version 141781 (0.0007) [2023-12-27 06:38:16,776][20347] Updated weights for policy 0, policy_version 141791 (0.0008) [2023-12-27 06:38:16,931][20381] Updated weights for policy 1, policy_version 141502 (0.0006) [2023-12-27 06:38:16,986][20381] Updated weights for policy 1, policy_version 141512 (0.0005) [2023-12-27 06:38:17,039][20381] Updated weights for policy 1, policy_version 141522 (0.0006) [2023-12-27 06:38:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 72556544. Throughput: 0: 10349.5, 1: 10424.7. Samples: 72567636. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:17,496][20347] Updated weights for policy 0, policy_version 141801 (0.0008) [2023-12-27 06:38:17,547][20347] Updated weights for policy 0, policy_version 141811 (0.0005) [2023-12-27 06:38:17,601][20347] Updated weights for policy 0, policy_version 141821 (0.0006) [2023-12-27 06:38:17,656][20347] Updated weights for policy 0, policy_version 141831 (0.0007) [2023-12-27 06:38:17,683][20381] Updated weights for policy 1, policy_version 141532 (0.0007) [2023-12-27 06:38:17,741][20381] Updated weights for policy 1, policy_version 141542 (0.0007) [2023-12-27 06:38:17,799][20381] Updated weights for policy 1, policy_version 141552 (0.0007) [2023-12-27 06:38:18,334][20347] Updated weights for policy 0, policy_version 141841 (0.0007) [2023-12-27 06:38:18,397][20347] Updated weights for policy 0, policy_version 141851 (0.0007) [2023-12-27 06:38:18,457][20347] Updated weights for policy 0, policy_version 141861 (0.0006) [2023-12-27 06:38:18,478][20381] Updated weights for policy 1, policy_version 141562 (0.0009) [2023-12-27 06:38:18,530][20381] Updated weights for policy 1, policy_version 141572 (0.0010) [2023-12-27 06:38:18,595][20381] Updated weights for policy 1, policy_version 141582 (0.0011) [2023-12-27 06:38:18,661][20381] Updated weights for policy 1, policy_version 141592 (0.0011) [2023-12-27 06:38:18,984][20347] Updated weights for policy 0, policy_version 141871 (0.0006) [2023-12-27 06:38:19,038][20347] Updated weights for policy 0, policy_version 141881 (0.0006) [2023-12-27 06:38:19,094][20347] Updated weights for policy 0, policy_version 141891 (0.0007) [2023-12-27 06:38:19,375][20381] Updated weights for policy 1, policy_version 141602 (0.0008) [2023-12-27 06:38:19,434][20381] Updated weights for policy 1, policy_version 141612 (0.0008) [2023-12-27 06:38:19,495][20381] Updated weights for policy 1, policy_version 141622 (0.0010) [2023-12-27 06:38:19,754][20347] Updated weights for policy 0, policy_version 141901 (0.0006) [2023-12-27 06:38:19,821][20347] Updated weights for policy 0, policy_version 141911 (0.0006) [2023-12-27 06:38:19,887][20347] Updated weights for policy 0, policy_version 141921 (0.0008) [2023-12-27 06:38:20,224][20381] Updated weights for policy 1, policy_version 141632 (0.0009) [2023-12-27 06:38:20,285][20381] Updated weights for policy 1, policy_version 141642 (0.0007) [2023-12-27 06:38:20,335][20381] Updated weights for policy 1, policy_version 141652 (0.0005) [2023-12-27 06:38:20,560][20347] Updated weights for policy 0, policy_version 141931 (0.0007) [2023-12-27 06:38:20,617][20347] Updated weights for policy 0, policy_version 141941 (0.0006) [2023-12-27 06:38:20,670][20347] Updated weights for policy 0, policy_version 141951 (0.0008) [2023-12-27 06:38:21,036][20381] Updated weights for policy 1, policy_version 141662 (0.0006) [2023-12-27 06:38:21,100][20381] Updated weights for policy 1, policy_version 141672 (0.0008) [2023-12-27 06:38:21,159][20381] Updated weights for policy 1, policy_version 141682 (0.0008) [2023-12-27 06:38:21,407][20347] Updated weights for policy 0, policy_version 141961 (0.0008) [2023-12-27 06:38:21,469][20347] Updated weights for policy 0, policy_version 141971 (0.0007) [2023-12-27 06:38:21,531][20347] Updated weights for policy 0, policy_version 141981 (0.0006) [2023-12-27 06:38:21,594][20347] Updated weights for policy 0, policy_version 141991 (0.0008) [2023-12-27 06:38:21,899][20381] Updated weights for policy 1, policy_version 141692 (0.0008) [2023-12-27 06:38:21,966][20381] Updated weights for policy 1, policy_version 141702 (0.0008) [2023-12-27 06:38:22,029][20381] Updated weights for policy 1, policy_version 141712 (0.0007) [2023-12-27 06:38:22,242][20347] Updated weights for policy 0, policy_version 142001 (0.0008) [2023-12-27 06:38:22,300][20347] Updated weights for policy 0, policy_version 142011 (0.0008) [2023-12-27 06:38:22,359][20347] Updated weights for policy 0, policy_version 142021 (0.0008) [2023-12-27 06:38:22,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 72654848. Throughput: 0: 10349.5, 1: 10383.5. Samples: 72629340. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000141720_36290560.pth... [2023-12-27 06:38:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000140504_35979264.pth [2023-12-27 06:38:22,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000142024_36372480.pth... [2023-12-27 06:38:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000140808_36061184.pth [2023-12-27 06:38:22,651][20381] Updated weights for policy 1, policy_version 141722 (0.0007) [2023-12-27 06:38:22,707][20381] Updated weights for policy 1, policy_version 141732 (0.0006) [2023-12-27 06:38:22,766][20381] Updated weights for policy 1, policy_version 141742 (0.0006) [2023-12-27 06:38:22,823][20381] Updated weights for policy 1, policy_version 141752 (0.0006) [2023-12-27 06:38:23,118][20347] Updated weights for policy 0, policy_version 142031 (0.0007) [2023-12-27 06:38:23,168][20347] Updated weights for policy 0, policy_version 142041 (0.0007) [2023-12-27 06:38:23,222][20347] Updated weights for policy 0, policy_version 142051 (0.0008) [2023-12-27 06:38:23,467][20381] Updated weights for policy 1, policy_version 141762 (0.0007) [2023-12-27 06:38:23,519][20381] Updated weights for policy 1, policy_version 141772 (0.0007) [2023-12-27 06:38:23,569][20381] Updated weights for policy 1, policy_version 141782 (0.0006) [2023-12-27 06:38:23,935][20347] Updated weights for policy 0, policy_version 142061 (0.0006) [2023-12-27 06:38:23,986][20347] Updated weights for policy 0, policy_version 142071 (0.0007) [2023-12-27 06:38:24,039][20347] Updated weights for policy 0, policy_version 142081 (0.0005) [2023-12-27 06:38:24,235][20381] Updated weights for policy 1, policy_version 141792 (0.0007) [2023-12-27 06:38:24,291][20381] Updated weights for policy 1, policy_version 141802 (0.0007) [2023-12-27 06:38:24,342][20381] Updated weights for policy 1, policy_version 141812 (0.0007) [2023-12-27 06:38:24,664][20347] Updated weights for policy 0, policy_version 142091 (0.0006) [2023-12-27 06:38:24,712][20347] Updated weights for policy 0, policy_version 142101 (0.0007) [2023-12-27 06:38:24,761][20347] Updated weights for policy 0, policy_version 142111 (0.0007) [2023-12-27 06:38:25,005][20381] Updated weights for policy 1, policy_version 141822 (0.0007) [2023-12-27 06:38:25,055][20381] Updated weights for policy 1, policy_version 141832 (0.0008) [2023-12-27 06:38:25,112][20381] Updated weights for policy 1, policy_version 141842 (0.0008) [2023-12-27 06:38:25,388][20347] Updated weights for policy 0, policy_version 142121 (0.0008) [2023-12-27 06:38:25,447][20347] Updated weights for policy 0, policy_version 142131 (0.0007) [2023-12-27 06:38:25,505][20347] Updated weights for policy 0, policy_version 142141 (0.0007) [2023-12-27 06:38:25,558][20347] Updated weights for policy 0, policy_version 142151 (0.0008) [2023-12-27 06:38:25,732][20381] Updated weights for policy 1, policy_version 141852 (0.0008) [2023-12-27 06:38:25,796][20381] Updated weights for policy 1, policy_version 141862 (0.0008) [2023-12-27 06:38:25,842][20381] Updated weights for policy 1, policy_version 141872 (0.0008) [2023-12-27 06:38:26,233][20347] Updated weights for policy 0, policy_version 142161 (0.0007) [2023-12-27 06:38:26,285][20347] Updated weights for policy 0, policy_version 142171 (0.0008) [2023-12-27 06:38:26,337][20347] Updated weights for policy 0, policy_version 142181 (0.0007) [2023-12-27 06:38:26,554][20381] Updated weights for policy 1, policy_version 141882 (0.0007) [2023-12-27 06:38:26,600][20381] Updated weights for policy 1, policy_version 141892 (0.0007) [2023-12-27 06:38:26,645][20381] Updated weights for policy 1, policy_version 141902 (0.0008) [2023-12-27 06:38:26,692][20381] Updated weights for policy 1, policy_version 141912 (0.0006) [2023-12-27 06:38:26,960][20347] Updated weights for policy 0, policy_version 142191 (0.0006) [2023-12-27 06:38:27,013][20347] Updated weights for policy 0, policy_version 142201 (0.0007) [2023-12-27 06:38:27,062][20347] Updated weights for policy 0, policy_version 142211 (0.0008) [2023-12-27 06:38:27,283][20381] Updated weights for policy 1, policy_version 141922 (0.0007) [2023-12-27 06:38:27,339][20381] Updated weights for policy 1, policy_version 141932 (0.0010) [2023-12-27 06:38:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 72761344. Throughput: 0: 10316.2, 1: 10398.5. Samples: 72752500. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:27,388][20381] Updated weights for policy 1, policy_version 141942 (0.0010) [2023-12-27 06:38:27,737][20347] Updated weights for policy 0, policy_version 142221 (0.0009) [2023-12-27 06:38:27,780][20347] Updated weights for policy 0, policy_version 142231 (0.0008) [2023-12-27 06:38:27,823][20347] Updated weights for policy 0, policy_version 142241 (0.0007) [2023-12-27 06:38:27,957][20381] Updated weights for policy 1, policy_version 141952 (0.0010) [2023-12-27 06:38:28,004][20381] Updated weights for policy 1, policy_version 141962 (0.0009) [2023-12-27 06:38:28,057][20381] Updated weights for policy 1, policy_version 141972 (0.0005) [2023-12-27 06:38:28,540][20347] Updated weights for policy 0, policy_version 142251 (0.0007) [2023-12-27 06:38:28,594][20347] Updated weights for policy 0, policy_version 142261 (0.0007) [2023-12-27 06:38:28,648][20347] Updated weights for policy 0, policy_version 142271 (0.0007) [2023-12-27 06:38:28,757][20381] Updated weights for policy 1, policy_version 141982 (0.0008) [2023-12-27 06:38:28,806][20381] Updated weights for policy 1, policy_version 141992 (0.0010) [2023-12-27 06:38:28,852][20381] Updated weights for policy 1, policy_version 142002 (0.0010) [2023-12-27 06:38:29,301][20347] Updated weights for policy 0, policy_version 142281 (0.0006) [2023-12-27 06:38:29,366][20347] Updated weights for policy 0, policy_version 142291 (0.0007) [2023-12-27 06:38:29,421][20347] Updated weights for policy 0, policy_version 142301 (0.0007) [2023-12-27 06:38:29,478][20347] Updated weights for policy 0, policy_version 142311 (0.0007) [2023-12-27 06:38:29,566][20381] Updated weights for policy 1, policy_version 142012 (0.0009) [2023-12-27 06:38:29,614][20381] Updated weights for policy 1, policy_version 142022 (0.0007) [2023-12-27 06:38:29,667][20381] Updated weights for policy 1, policy_version 142032 (0.0005) [2023-12-27 06:38:30,170][20347] Updated weights for policy 0, policy_version 142321 (0.0008) [2023-12-27 06:38:30,228][20347] Updated weights for policy 0, policy_version 142331 (0.0008) [2023-12-27 06:38:30,246][20381] Updated weights for policy 1, policy_version 142042 (0.0005) [2023-12-27 06:38:30,285][20347] Updated weights for policy 0, policy_version 142341 (0.0006) [2023-12-27 06:38:30,301][20381] Updated weights for policy 1, policy_version 142052 (0.0006) [2023-12-27 06:38:30,359][20381] Updated weights for policy 1, policy_version 142062 (0.0008) [2023-12-27 06:38:30,418][20381] Updated weights for policy 1, policy_version 142072 (0.0008) [2023-12-27 06:38:30,914][20347] Updated weights for policy 0, policy_version 142351 (0.0008) [2023-12-27 06:38:30,974][20347] Updated weights for policy 0, policy_version 142361 (0.0007) [2023-12-27 06:38:31,017][20381] Updated weights for policy 1, policy_version 142082 (0.0007) [2023-12-27 06:38:31,033][20347] Updated weights for policy 0, policy_version 142371 (0.0008) [2023-12-27 06:38:31,080][20381] Updated weights for policy 1, policy_version 142092 (0.0007) [2023-12-27 06:38:31,137][20381] Updated weights for policy 1, policy_version 142102 (0.0007) [2023-12-27 06:38:31,702][20347] Updated weights for policy 0, policy_version 142381 (0.0009) [2023-12-27 06:38:31,764][20347] Updated weights for policy 0, policy_version 142391 (0.0009) [2023-12-27 06:38:31,829][20347] Updated weights for policy 0, policy_version 142401 (0.0008) [2023-12-27 06:38:31,838][20381] Updated weights for policy 1, policy_version 142112 (0.0007) [2023-12-27 06:38:31,884][20381] Updated weights for policy 1, policy_version 142122 (0.0006) [2023-12-27 06:38:31,935][20381] Updated weights for policy 1, policy_version 142132 (0.0008) [2023-12-27 06:38:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 72867840. Throughput: 0: 10295.2, 1: 10469.5. Samples: 72880184. Policy #0 lag: (min: 19.0, avg: 27.5, max: 51.0) [2023-12-27 06:38:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:32,476][20347] Updated weights for policy 0, policy_version 142411 (0.0008) [2023-12-27 06:38:32,540][20347] Updated weights for policy 0, policy_version 142421 (0.0007) [2023-12-27 06:38:32,596][20347] Updated weights for policy 0, policy_version 142431 (0.0008) [2023-12-27 06:38:32,624][20381] Updated weights for policy 1, policy_version 142142 (0.0008) [2023-12-27 06:38:32,677][20381] Updated weights for policy 1, policy_version 142152 (0.0007) [2023-12-27 06:38:32,729][20381] Updated weights for policy 1, policy_version 142162 (0.0011) [2023-12-27 06:38:33,292][20347] Updated weights for policy 0, policy_version 142441 (0.0006) [2023-12-27 06:38:33,343][20347] Updated weights for policy 0, policy_version 142451 (0.0007) [2023-12-27 06:38:33,373][20381] Updated weights for policy 1, policy_version 142172 (0.0010) [2023-12-27 06:38:33,391][20347] Updated weights for policy 0, policy_version 142461 (0.0008) [2023-12-27 06:38:33,420][20381] Updated weights for policy 1, policy_version 142182 (0.0010) [2023-12-27 06:38:33,443][20347] Updated weights for policy 0, policy_version 142471 (0.0006) [2023-12-27 06:38:33,472][20381] Updated weights for policy 1, policy_version 142192 (0.0010) [2023-12-27 06:38:34,164][20347] Updated weights for policy 0, policy_version 142481 (0.0008) [2023-12-27 06:38:34,166][20381] Updated weights for policy 1, policy_version 142202 (0.0011) [2023-12-27 06:38:34,222][20347] Updated weights for policy 0, policy_version 142491 (0.0006) [2023-12-27 06:38:34,225][20381] Updated weights for policy 1, policy_version 142212 (0.0010) [2023-12-27 06:38:34,278][20347] Updated weights for policy 0, policy_version 142501 (0.0006) [2023-12-27 06:38:34,283][20381] Updated weights for policy 1, policy_version 142222 (0.0010) [2023-12-27 06:38:34,335][20381] Updated weights for policy 1, policy_version 142232 (0.0010) [2023-12-27 06:38:34,974][20381] Updated weights for policy 1, policy_version 142242 (0.0005) [2023-12-27 06:38:35,006][20347] Updated weights for policy 0, policy_version 142511 (0.0007) [2023-12-27 06:38:35,033][20381] Updated weights for policy 1, policy_version 142252 (0.0006) [2023-12-27 06:38:35,064][20347] Updated weights for policy 0, policy_version 142521 (0.0008) [2023-12-27 06:38:35,085][20381] Updated weights for policy 1, policy_version 142262 (0.0005) [2023-12-27 06:38:35,120][20347] Updated weights for policy 0, policy_version 142531 (0.0008) [2023-12-27 06:38:35,722][20381] Updated weights for policy 1, policy_version 142272 (0.0010) [2023-12-27 06:38:35,775][20381] Updated weights for policy 1, policy_version 142282 (0.0011) [2023-12-27 06:38:35,811][20347] Updated weights for policy 0, policy_version 142541 (0.0007) [2023-12-27 06:38:35,823][20381] Updated weights for policy 1, policy_version 142292 (0.0007) [2023-12-27 06:38:35,869][20347] Updated weights for policy 0, policy_version 142551 (0.0007) [2023-12-27 06:38:35,929][20347] Updated weights for policy 0, policy_version 142561 (0.0007) [2023-12-27 06:38:36,552][20381] Updated weights for policy 1, policy_version 142302 (0.0010) [2023-12-27 06:38:36,580][20347] Updated weights for policy 0, policy_version 142571 (0.0007) [2023-12-27 06:38:36,611][20381] Updated weights for policy 1, policy_version 142312 (0.0011) [2023-12-27 06:38:36,637][20347] Updated weights for policy 0, policy_version 142581 (0.0006) [2023-12-27 06:38:36,669][20381] Updated weights for policy 1, policy_version 142322 (0.0008) [2023-12-27 06:38:36,696][20347] Updated weights for policy 0, policy_version 142591 (0.0007) [2023-12-27 06:38:37,310][20381] Updated weights for policy 1, policy_version 142332 (0.0006) [2023-12-27 06:38:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 72966144. Throughput: 0: 10270.6, 1: 10491.8. Samples: 72941520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:38:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000142600_36519936.pth... [2023-12-27 06:38:37,370][20381] Updated weights for policy 1, policy_version 142342 (0.0006) [2023-12-27 06:38:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000141416_36216832.pth [2023-12-27 06:38:37,417][20347] Updated weights for policy 0, policy_version 142601 (0.0008) [2023-12-27 06:38:37,430][20381] Updated weights for policy 1, policy_version 142352 (0.0009) [2023-12-27 06:38:37,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000142360_36454400.pth... [2023-12-27 06:38:37,473][20347] Updated weights for policy 0, policy_version 142611 (0.0005) [2023-12-27 06:38:37,476][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000141112_36134912.pth [2023-12-27 06:38:37,526][20347] Updated weights for policy 0, policy_version 142621 (0.0007) [2023-12-27 06:38:37,588][20347] Updated weights for policy 0, policy_version 142631 (0.0007) [2023-12-27 06:38:38,076][20381] Updated weights for policy 1, policy_version 142362 (0.0010) [2023-12-27 06:38:38,128][20381] Updated weights for policy 1, policy_version 142372 (0.0006) [2023-12-27 06:38:38,180][20381] Updated weights for policy 1, policy_version 142382 (0.0007) [2023-12-27 06:38:38,233][20381] Updated weights for policy 1, policy_version 142392 (0.0007) [2023-12-27 06:38:38,259][20347] Updated weights for policy 0, policy_version 142641 (0.0005) [2023-12-27 06:38:38,307][20347] Updated weights for policy 0, policy_version 142651 (0.0005) [2023-12-27 06:38:38,371][20347] Updated weights for policy 0, policy_version 142661 (0.0008) [2023-12-27 06:38:38,906][20381] Updated weights for policy 1, policy_version 142402 (0.0006) [2023-12-27 06:38:38,975][20381] Updated weights for policy 1, policy_version 142412 (0.0010) [2023-12-27 06:38:39,039][20381] Updated weights for policy 1, policy_version 142422 (0.0009) [2023-12-27 06:38:39,043][20347] Updated weights for policy 0, policy_version 142671 (0.0006) [2023-12-27 06:38:39,092][20347] Updated weights for policy 0, policy_version 142681 (0.0005) [2023-12-27 06:38:39,156][20347] Updated weights for policy 0, policy_version 142691 (0.0008) [2023-12-27 06:38:39,731][20381] Updated weights for policy 1, policy_version 142432 (0.0006) [2023-12-27 06:38:39,782][20381] Updated weights for policy 1, policy_version 142442 (0.0006) [2023-12-27 06:38:39,851][20381] Updated weights for policy 1, policy_version 142452 (0.0007) [2023-12-27 06:38:39,909][20347] Updated weights for policy 0, policy_version 142701 (0.0008) [2023-12-27 06:38:39,973][20347] Updated weights for policy 0, policy_version 142711 (0.0007) [2023-12-27 06:38:40,034][20347] Updated weights for policy 0, policy_version 142721 (0.0008) [2023-12-27 06:38:40,484][20381] Updated weights for policy 1, policy_version 142462 (0.0008) [2023-12-27 06:38:40,540][20381] Updated weights for policy 1, policy_version 142472 (0.0008) [2023-12-27 06:38:40,589][20381] Updated weights for policy 1, policy_version 142482 (0.0008) [2023-12-27 06:38:40,700][20347] Updated weights for policy 0, policy_version 142731 (0.0008) [2023-12-27 06:38:40,755][20347] Updated weights for policy 0, policy_version 142741 (0.0010) [2023-12-27 06:38:40,814][20347] Updated weights for policy 0, policy_version 142751 (0.0010) [2023-12-27 06:38:41,307][20381] Updated weights for policy 1, policy_version 142492 (0.0008) [2023-12-27 06:38:41,370][20381] Updated weights for policy 1, policy_version 142502 (0.0008) [2023-12-27 06:38:41,435][20381] Updated weights for policy 1, policy_version 142512 (0.0008) [2023-12-27 06:38:41,464][20347] Updated weights for policy 0, policy_version 142761 (0.0010) [2023-12-27 06:38:41,526][20347] Updated weights for policy 0, policy_version 142771 (0.0007) [2023-12-27 06:38:41,586][20347] Updated weights for policy 0, policy_version 142781 (0.0008) [2023-12-27 06:38:41,649][20347] Updated weights for policy 0, policy_version 142791 (0.0008) [2023-12-27 06:38:42,191][20381] Updated weights for policy 1, policy_version 142522 (0.0007) [2023-12-27 06:38:42,243][20381] Updated weights for policy 1, policy_version 142532 (0.0008) [2023-12-27 06:38:42,285][20347] Updated weights for policy 0, policy_version 142801 (0.0008) [2023-12-27 06:38:42,303][20381] Updated weights for policy 1, policy_version 142542 (0.0007) [2023-12-27 06:38:42,345][20347] Updated weights for policy 0, policy_version 142811 (0.0006) [2023-12-27 06:38:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 73064448. Throughput: 0: 10246.9, 1: 10456.4. Samples: 73063492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:38:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:42,366][20381] Updated weights for policy 1, policy_version 142552 (0.0008) [2023-12-27 06:38:42,408][20347] Updated weights for policy 0, policy_version 142821 (0.0007) [2023-12-27 06:38:43,069][20381] Updated weights for policy 1, policy_version 142562 (0.0008) [2023-12-27 06:38:43,102][20347] Updated weights for policy 0, policy_version 142831 (0.0007) [2023-12-27 06:38:43,119][20381] Updated weights for policy 1, policy_version 142572 (0.0007) [2023-12-27 06:38:43,151][20347] Updated weights for policy 0, policy_version 142841 (0.0006) [2023-12-27 06:38:43,175][20381] Updated weights for policy 1, policy_version 142582 (0.0005) [2023-12-27 06:38:43,203][20347] Updated weights for policy 0, policy_version 142851 (0.0007) [2023-12-27 06:38:43,884][20347] Updated weights for policy 0, policy_version 142861 (0.0006) [2023-12-27 06:38:43,889][20381] Updated weights for policy 1, policy_version 142592 (0.0007) [2023-12-27 06:38:43,933][20347] Updated weights for policy 0, policy_version 142871 (0.0005) [2023-12-27 06:38:43,953][20381] Updated weights for policy 1, policy_version 142602 (0.0008) [2023-12-27 06:38:43,980][20347] Updated weights for policy 0, policy_version 142881 (0.0005) [2023-12-27 06:38:44,024][20381] Updated weights for policy 1, policy_version 142612 (0.0009) [2023-12-27 06:38:44,630][20347] Updated weights for policy 0, policy_version 142891 (0.0006) [2023-12-27 06:38:44,678][20347] Updated weights for policy 0, policy_version 142901 (0.0008) [2023-12-27 06:38:44,705][20381] Updated weights for policy 1, policy_version 142622 (0.0006) [2023-12-27 06:38:44,724][20347] Updated weights for policy 0, policy_version 142911 (0.0006) [2023-12-27 06:38:44,761][20381] Updated weights for policy 1, policy_version 142632 (0.0007) [2023-12-27 06:38:44,822][20381] Updated weights for policy 1, policy_version 142642 (0.0008) [2023-12-27 06:38:45,422][20347] Updated weights for policy 0, policy_version 142921 (0.0007) [2023-12-27 06:38:45,483][20347] Updated weights for policy 0, policy_version 142931 (0.0007) [2023-12-27 06:38:45,517][20381] Updated weights for policy 1, policy_version 142652 (0.0007) [2023-12-27 06:38:45,536][20347] Updated weights for policy 0, policy_version 142941 (0.0008) [2023-12-27 06:38:45,566][20381] Updated weights for policy 1, policy_version 142662 (0.0005) [2023-12-27 06:38:45,583][20347] Updated weights for policy 0, policy_version 142951 (0.0006) [2023-12-27 06:38:45,613][20381] Updated weights for policy 1, policy_version 142672 (0.0005) [2023-12-27 06:38:46,247][20347] Updated weights for policy 0, policy_version 142961 (0.0005) [2023-12-27 06:38:46,272][20381] Updated weights for policy 1, policy_version 142682 (0.0006) [2023-12-27 06:38:46,311][20347] Updated weights for policy 0, policy_version 142971 (0.0007) [2023-12-27 06:38:46,324][20381] Updated weights for policy 1, policy_version 142692 (0.0006) [2023-12-27 06:38:46,367][20347] Updated weights for policy 0, policy_version 142981 (0.0007) [2023-12-27 06:38:46,378][20381] Updated weights for policy 1, policy_version 142702 (0.0007) [2023-12-27 06:38:46,438][20381] Updated weights for policy 1, policy_version 142712 (0.0010) [2023-12-27 06:38:46,998][20347] Updated weights for policy 0, policy_version 142991 (0.0008) [2023-12-27 06:38:47,061][20347] Updated weights for policy 0, policy_version 143001 (0.0008) [2023-12-27 06:38:47,078][20381] Updated weights for policy 1, policy_version 142722 (0.0009) [2023-12-27 06:38:47,120][20347] Updated weights for policy 0, policy_version 143011 (0.0006) [2023-12-27 06:38:47,137][20381] Updated weights for policy 1, policy_version 142732 (0.0008) [2023-12-27 06:38:47,198][20381] Updated weights for policy 1, policy_version 142742 (0.0006) [2023-12-27 06:38:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 73179136. Throughput: 0: 10269.7, 1: 10377.7. Samples: 73185944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:38:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:47,771][20347] Updated weights for policy 0, policy_version 143021 (0.0005) [2023-12-27 06:38:47,829][20347] Updated weights for policy 0, policy_version 143031 (0.0007) [2023-12-27 06:38:47,842][20381] Updated weights for policy 1, policy_version 142752 (0.0006) [2023-12-27 06:38:47,892][20347] Updated weights for policy 0, policy_version 143041 (0.0008) [2023-12-27 06:38:47,905][20381] Updated weights for policy 1, policy_version 142762 (0.0005) [2023-12-27 06:38:47,953][20381] Updated weights for policy 1, policy_version 142772 (0.0005) [2023-12-27 06:38:48,543][20347] Updated weights for policy 0, policy_version 143051 (0.0009) [2023-12-27 06:38:48,573][20381] Updated weights for policy 1, policy_version 142782 (0.0006) [2023-12-27 06:38:48,597][20347] Updated weights for policy 0, policy_version 143061 (0.0008) [2023-12-27 06:38:48,637][20381] Updated weights for policy 1, policy_version 142792 (0.0005) [2023-12-27 06:38:48,650][20347] Updated weights for policy 0, policy_version 143071 (0.0006) [2023-12-27 06:38:48,702][20381] Updated weights for policy 1, policy_version 142802 (0.0005) [2023-12-27 06:38:49,221][20381] Updated weights for policy 1, policy_version 142812 (0.0006) [2023-12-27 06:38:49,284][20381] Updated weights for policy 1, policy_version 142822 (0.0007) [2023-12-27 06:38:49,348][20381] Updated weights for policy 1, policy_version 142832 (0.0008) [2023-12-27 06:38:49,369][20347] Updated weights for policy 0, policy_version 143081 (0.0007) [2023-12-27 06:38:49,431][20347] Updated weights for policy 0, policy_version 143091 (0.0007) [2023-12-27 06:38:49,491][20347] Updated weights for policy 0, policy_version 143101 (0.0008) [2023-12-27 06:38:49,552][20347] Updated weights for policy 0, policy_version 143111 (0.0008) [2023-12-27 06:38:50,031][20381] Updated weights for policy 1, policy_version 142842 (0.0007) [2023-12-27 06:38:50,094][20381] Updated weights for policy 1, policy_version 142852 (0.0005) [2023-12-27 06:38:50,147][20381] Updated weights for policy 1, policy_version 142862 (0.0005) [2023-12-27 06:38:50,196][20381] Updated weights for policy 1, policy_version 142872 (0.0007) [2023-12-27 06:38:50,216][20347] Updated weights for policy 0, policy_version 143121 (0.0006) [2023-12-27 06:38:50,269][20347] Updated weights for policy 0, policy_version 143131 (0.0007) [2023-12-27 06:38:50,320][20347] Updated weights for policy 0, policy_version 143141 (0.0007) [2023-12-27 06:38:50,842][20381] Updated weights for policy 1, policy_version 142882 (0.0006) [2023-12-27 06:38:50,897][20381] Updated weights for policy 1, policy_version 142892 (0.0006) [2023-12-27 06:38:50,947][20381] Updated weights for policy 1, policy_version 142902 (0.0006) [2023-12-27 06:38:50,968][20347] Updated weights for policy 0, policy_version 143151 (0.0008) [2023-12-27 06:38:51,031][20347] Updated weights for policy 0, policy_version 143161 (0.0007) [2023-12-27 06:38:51,095][20347] Updated weights for policy 0, policy_version 143171 (0.0008) [2023-12-27 06:38:51,584][20381] Updated weights for policy 1, policy_version 142912 (0.0007) [2023-12-27 06:38:51,638][20381] Updated weights for policy 1, policy_version 142922 (0.0008) [2023-12-27 06:38:51,689][20381] Updated weights for policy 1, policy_version 142932 (0.0007) [2023-12-27 06:38:51,749][20347] Updated weights for policy 0, policy_version 143181 (0.0008) [2023-12-27 06:38:51,813][20347] Updated weights for policy 0, policy_version 143191 (0.0007) [2023-12-27 06:38:51,880][20347] Updated weights for policy 0, policy_version 143201 (0.0007) [2023-12-27 06:38:52,356][20381] Updated weights for policy 1, policy_version 142942 (0.0007) [2023-12-27 06:38:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 73277440. Throughput: 0: 10248.9, 1: 10400.1. Samples: 73249908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:38:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000143208_36675584.pth... [2023-12-27 06:38:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000142024_36372480.pth [2023-12-27 06:38:52,419][20381] Updated weights for policy 1, policy_version 142952 (0.0006) [2023-12-27 06:38:52,486][20381] Updated weights for policy 1, policy_version 142962 (0.0008) [2023-12-27 06:38:52,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000142968_36610048.pth... [2023-12-27 06:38:52,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000141720_36290560.pth [2023-12-27 06:38:52,591][20347] Updated weights for policy 0, policy_version 143211 (0.0007) [2023-12-27 06:38:52,649][20347] Updated weights for policy 0, policy_version 143221 (0.0008) [2023-12-27 06:38:52,695][20347] Updated weights for policy 0, policy_version 143231 (0.0009) [2023-12-27 06:38:53,136][20381] Updated weights for policy 1, policy_version 142972 (0.0008) [2023-12-27 06:38:53,190][20381] Updated weights for policy 1, policy_version 142982 (0.0010) [2023-12-27 06:38:53,249][20381] Updated weights for policy 1, policy_version 142992 (0.0011) [2023-12-27 06:38:53,327][20347] Updated weights for policy 0, policy_version 143241 (0.0010) [2023-12-27 06:38:53,381][20347] Updated weights for policy 0, policy_version 143251 (0.0007) [2023-12-27 06:38:53,435][20347] Updated weights for policy 0, policy_version 143261 (0.0007) [2023-12-27 06:38:53,494][20347] Updated weights for policy 0, policy_version 143271 (0.0007) [2023-12-27 06:38:53,933][20381] Updated weights for policy 1, policy_version 143002 (0.0009) [2023-12-27 06:38:53,994][20381] Updated weights for policy 1, policy_version 143012 (0.0010) [2023-12-27 06:38:54,049][20381] Updated weights for policy 1, policy_version 143022 (0.0010) [2023-12-27 06:38:54,114][20381] Updated weights for policy 1, policy_version 143032 (0.0011) [2023-12-27 06:38:54,148][20347] Updated weights for policy 0, policy_version 143281 (0.0007) [2023-12-27 06:38:54,204][20347] Updated weights for policy 0, policy_version 143291 (0.0008) [2023-12-27 06:38:54,260][20347] Updated weights for policy 0, policy_version 143301 (0.0008) [2023-12-27 06:38:54,752][20381] Updated weights for policy 1, policy_version 143042 (0.0011) [2023-12-27 06:38:54,808][20381] Updated weights for policy 1, policy_version 143052 (0.0010) [2023-12-27 06:38:54,866][20381] Updated weights for policy 1, policy_version 143062 (0.0010) [2023-12-27 06:38:54,932][20347] Updated weights for policy 0, policy_version 143311 (0.0007) [2023-12-27 06:38:54,989][20347] Updated weights for policy 0, policy_version 143321 (0.0008) [2023-12-27 06:38:55,041][20347] Updated weights for policy 0, policy_version 143331 (0.0007) [2023-12-27 06:38:55,528][20381] Updated weights for policy 1, policy_version 143072 (0.0010) [2023-12-27 06:38:55,586][20381] Updated weights for policy 1, policy_version 143082 (0.0010) [2023-12-27 06:38:55,647][20381] Updated weights for policy 1, policy_version 143092 (0.0010) [2023-12-27 06:38:55,715][20347] Updated weights for policy 0, policy_version 143341 (0.0007) [2023-12-27 06:38:55,767][20347] Updated weights for policy 0, policy_version 143351 (0.0007) [2023-12-27 06:38:55,822][20347] Updated weights for policy 0, policy_version 143361 (0.0007) [2023-12-27 06:38:56,310][20381] Updated weights for policy 1, policy_version 143102 (0.0010) [2023-12-27 06:38:56,369][20381] Updated weights for policy 1, policy_version 143112 (0.0010) [2023-12-27 06:38:56,412][20347] Updated weights for policy 0, policy_version 143371 (0.0008) [2023-12-27 06:38:56,423][20381] Updated weights for policy 1, policy_version 143122 (0.0010) [2023-12-27 06:38:56,467][20347] Updated weights for policy 0, policy_version 143381 (0.0007) [2023-12-27 06:38:56,518][20347] Updated weights for policy 0, policy_version 143391 (0.0007) [2023-12-27 06:38:57,041][20381] Updated weights for policy 1, policy_version 143132 (0.0006) [2023-12-27 06:38:57,101][20381] Updated weights for policy 1, policy_version 143142 (0.0008) [2023-12-27 06:38:57,164][20381] Updated weights for policy 1, policy_version 143152 (0.0008) [2023-12-27 06:38:57,212][20347] Updated weights for policy 0, policy_version 143401 (0.0006) [2023-12-27 06:38:57,259][20347] Updated weights for policy 0, policy_version 143411 (0.0007) [2023-12-27 06:38:57,310][20347] Updated weights for policy 0, policy_version 143421 (0.0007) [2023-12-27 06:38:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 73383936. Throughput: 0: 10265.4, 1: 10407.8. Samples: 73374280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:38:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:38:57,370][20347] Updated weights for policy 0, policy_version 143431 (0.0006) [2023-12-27 06:38:57,833][20381] Updated weights for policy 1, policy_version 143162 (0.0009) [2023-12-27 06:38:57,877][20381] Updated weights for policy 1, policy_version 143172 (0.0010) [2023-12-27 06:38:57,922][20381] Updated weights for policy 1, policy_version 143182 (0.0010) [2023-12-27 06:38:57,966][20381] Updated weights for policy 1, policy_version 143192 (0.0010) [2023-12-27 06:38:58,044][20347] Updated weights for policy 0, policy_version 143441 (0.0006) [2023-12-27 06:38:58,092][20347] Updated weights for policy 0, policy_version 143451 (0.0005) [2023-12-27 06:38:58,155][20347] Updated weights for policy 0, policy_version 143461 (0.0006) [2023-12-27 06:38:58,692][20381] Updated weights for policy 1, policy_version 143202 (0.0008) [2023-12-27 06:38:58,757][20381] Updated weights for policy 1, policy_version 143212 (0.0008) [2023-12-27 06:38:58,820][20381] Updated weights for policy 1, policy_version 143222 (0.0008) [2023-12-27 06:38:58,908][20347] Updated weights for policy 0, policy_version 143471 (0.0008) [2023-12-27 06:38:58,968][20347] Updated weights for policy 0, policy_version 143481 (0.0006) [2023-12-27 06:38:59,023][20347] Updated weights for policy 0, policy_version 143491 (0.0007) [2023-12-27 06:38:59,564][20381] Updated weights for policy 1, policy_version 143232 (0.0006) [2023-12-27 06:38:59,626][20381] Updated weights for policy 1, policy_version 143242 (0.0007) [2023-12-27 06:38:59,674][20292] KL-divergence is very high: 294.8161 [2023-12-27 06:38:59,680][20381] Updated weights for policy 1, policy_version 143252 (0.0007) [2023-12-27 06:38:59,680][20292] KL-divergence is very high: 305.0884 [2023-12-27 06:38:59,686][20292] KL-divergence is very high: 181.6932 [2023-12-27 06:38:59,718][20347] Updated weights for policy 0, policy_version 143501 (0.0009) [2023-12-27 06:38:59,772][20347] Updated weights for policy 0, policy_version 143511 (0.0010) [2023-12-27 06:38:59,826][20347] Updated weights for policy 0, policy_version 143521 (0.0010) [2023-12-27 06:39:00,288][20292] KL-divergence is very high: 256.6544 [2023-12-27 06:39:00,293][20292] KL-divergence is very high: 517.1519 [2023-12-27 06:39:00,299][20292] KL-divergence is very high: 152.8954 [2023-12-27 06:39:00,304][20292] KL-divergence is very high: 333.9540 [2023-12-27 06:39:00,309][20292] KL-divergence is very high: 325.1510 [2023-12-27 06:39:00,316][20381] Updated weights for policy 1, policy_version 143262 (0.0007) [2023-12-27 06:39:00,327][20292] KL-divergence is very high: 125.3294 [2023-12-27 06:39:00,333][20292] KL-divergence is very high: 120.1677 [2023-12-27 06:39:00,349][20292] KL-divergence is very high: 159.5430 [2023-12-27 06:39:00,363][20381] Updated weights for policy 1, policy_version 143272 (0.0007) [2023-12-27 06:39:00,371][20292] KL-divergence is very high: 257.1382 [2023-12-27 06:39:00,377][20292] KL-divergence is very high: 373.9532 [2023-12-27 06:39:00,387][20292] KL-divergence is very high: 138.3759 [2023-12-27 06:39:00,392][20292] KL-divergence is very high: 147.7820 [2023-12-27 06:39:00,412][20292] KL-divergence is very high: 137.8333 [2023-12-27 06:39:00,418][20292] KL-divergence is very high: 131.0318 [2023-12-27 06:39:00,418][20381] Updated weights for policy 1, policy_version 143282 (0.0008) [2023-12-27 06:39:00,430][20292] KL-divergence is very high: 126.5123 [2023-12-27 06:39:00,436][20292] KL-divergence is very high: 181.3408 [2023-12-27 06:39:00,512][20347] Updated weights for policy 0, policy_version 143531 (0.0011) [2023-12-27 06:39:00,560][20347] Updated weights for policy 0, policy_version 143541 (0.0007) [2023-12-27 06:39:00,615][20347] Updated weights for policy 0, policy_version 143551 (0.0010) [2023-12-27 06:39:01,139][20381] Updated weights for policy 1, policy_version 143292 (0.0008) [2023-12-27 06:39:01,200][20381] Updated weights for policy 1, policy_version 143302 (0.0007) [2023-12-27 06:39:01,257][20381] Updated weights for policy 1, policy_version 143312 (0.0007) [2023-12-27 06:39:01,322][20347] Updated weights for policy 0, policy_version 143561 (0.0009) [2023-12-27 06:39:01,385][20347] Updated weights for policy 0, policy_version 143571 (0.0009) [2023-12-27 06:39:01,443][20347] Updated weights for policy 0, policy_version 143581 (0.0010) [2023-12-27 06:39:01,496][20347] Updated weights for policy 0, policy_version 143591 (0.0009) [2023-12-27 06:39:01,909][20381] Updated weights for policy 1, policy_version 143322 (0.0008) [2023-12-27 06:39:01,969][20381] Updated weights for policy 1, policy_version 143332 (0.0006) [2023-12-27 06:39:02,028][20381] Updated weights for policy 1, policy_version 143342 (0.0005) [2023-12-27 06:39:02,093][20381] Updated weights for policy 1, policy_version 143352 (0.0008) [2023-12-27 06:39:02,157][20347] Updated weights for policy 0, policy_version 143601 (0.0007) [2023-12-27 06:39:02,215][20347] Updated weights for policy 0, policy_version 143611 (0.0010) [2023-12-27 06:39:02,271][20347] Updated weights for policy 0, policy_version 143621 (0.0010) [2023-12-27 06:39:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 73490432. Throughput: 0: 10277.1, 1: 10370.3. Samples: 73496768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '817.110')] [2023-12-27 06:39:02,679][20292] KL-divergence is very high: 103.7531 [2023-12-27 06:39:02,684][20292] KL-divergence is very high: 105.6009 [2023-12-27 06:39:02,689][20292] KL-divergence is very high: 200.5562 [2023-12-27 06:39:02,695][20381] Updated weights for policy 1, policy_version 143362 (0.0007) [2023-12-27 06:39:02,740][20381] Updated weights for policy 1, policy_version 143372 (0.0007) [2023-12-27 06:39:02,787][20381] Updated weights for policy 1, policy_version 143382 (0.0009) [2023-12-27 06:39:02,919][20347] Updated weights for policy 0, policy_version 143631 (0.0010) [2023-12-27 06:39:02,976][20347] Updated weights for policy 0, policy_version 143641 (0.0011) [2023-12-27 06:39:03,035][20347] Updated weights for policy 0, policy_version 143651 (0.0010) [2023-12-27 06:39:03,490][20381] Updated weights for policy 1, policy_version 143392 (0.0007) [2023-12-27 06:39:03,519][20292] KL-divergence is very high: 169.0920 [2023-12-27 06:39:03,546][20381] Updated weights for policy 1, policy_version 143402 (0.0005) [2023-12-27 06:39:03,606][20381] Updated weights for policy 1, policy_version 143412 (0.0008) [2023-12-27 06:39:03,718][20347] Updated weights for policy 0, policy_version 143661 (0.0010) [2023-12-27 06:39:03,762][20347] Updated weights for policy 0, policy_version 143671 (0.0010) [2023-12-27 06:39:03,804][20347] Updated weights for policy 0, policy_version 143681 (0.0010) [2023-12-27 06:39:04,260][20381] Updated weights for policy 1, policy_version 143422 (0.0009) [2023-12-27 06:39:04,321][20381] Updated weights for policy 1, policy_version 143432 (0.0007) [2023-12-27 06:39:04,377][20381] Updated weights for policy 1, policy_version 143442 (0.0007) [2023-12-27 06:39:04,539][20347] Updated weights for policy 0, policy_version 143691 (0.0010) [2023-12-27 06:39:04,599][20347] Updated weights for policy 0, policy_version 143701 (0.0010) [2023-12-27 06:39:04,658][20347] Updated weights for policy 0, policy_version 143711 (0.0010) [2023-12-27 06:39:05,076][20381] Updated weights for policy 1, policy_version 143452 (0.0007) [2023-12-27 06:39:05,131][20381] Updated weights for policy 1, policy_version 143462 (0.0009) [2023-12-27 06:39:05,192][20381] Updated weights for policy 1, policy_version 143472 (0.0007) [2023-12-27 06:39:05,329][20347] Updated weights for policy 0, policy_version 143721 (0.0011) [2023-12-27 06:39:05,381][20347] Updated weights for policy 0, policy_version 143731 (0.0010) [2023-12-27 06:39:05,435][20347] Updated weights for policy 0, policy_version 143741 (0.0008) [2023-12-27 06:39:05,499][20347] Updated weights for policy 0, policy_version 143751 (0.0005) [2023-12-27 06:39:05,877][20381] Updated weights for policy 1, policy_version 143482 (0.0008) [2023-12-27 06:39:05,929][20381] Updated weights for policy 1, policy_version 143492 (0.0010) [2023-12-27 06:39:05,988][20381] Updated weights for policy 1, policy_version 143502 (0.0011) [2023-12-27 06:39:06,026][20347] Updated weights for policy 0, policy_version 143761 (0.0005) [2023-12-27 06:39:06,046][20381] Updated weights for policy 1, policy_version 143512 (0.0011) [2023-12-27 06:39:06,077][20347] Updated weights for policy 0, policy_version 143771 (0.0005) [2023-12-27 06:39:06,140][20347] Updated weights for policy 0, policy_version 143781 (0.0008) [2023-12-27 06:39:06,752][20381] Updated weights for policy 1, policy_version 143522 (0.0011) [2023-12-27 06:39:06,811][20381] Updated weights for policy 1, policy_version 143532 (0.0011) [2023-12-27 06:39:06,817][20347] Updated weights for policy 0, policy_version 143791 (0.0006) [2023-12-27 06:39:06,864][20381] Updated weights for policy 1, policy_version 143542 (0.0011) [2023-12-27 06:39:06,870][20347] Updated weights for policy 0, policy_version 143801 (0.0007) [2023-12-27 06:39:06,931][20347] Updated weights for policy 0, policy_version 143811 (0.0007) [2023-12-27 06:39:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 73588736. Throughput: 0: 10238.8, 1: 10401.6. Samples: 73558156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '785.760')] [2023-12-27 06:39:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000143544_36757504.pth... [2023-12-27 06:39:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000143816_36831232.pth... [2023-12-27 06:39:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000142360_36454400.pth [2023-12-27 06:39:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000142600_36519936.pth [2023-12-27 06:39:07,537][20381] Updated weights for policy 1, policy_version 143552 (0.0011) [2023-12-27 06:39:07,597][20381] Updated weights for policy 1, policy_version 143562 (0.0011) [2023-12-27 06:39:07,650][20381] Updated weights for policy 1, policy_version 143572 (0.0011) [2023-12-27 06:39:07,661][20347] Updated weights for policy 0, policy_version 143821 (0.0007) [2023-12-27 06:39:07,726][20347] Updated weights for policy 0, policy_version 143831 (0.0007) [2023-12-27 06:39:07,782][20347] Updated weights for policy 0, policy_version 143841 (0.0008) [2023-12-27 06:39:08,241][20381] Updated weights for policy 1, policy_version 143582 (0.0010) [2023-12-27 06:39:08,304][20381] Updated weights for policy 1, policy_version 143592 (0.0010) [2023-12-27 06:39:08,362][20381] Updated weights for policy 1, policy_version 143602 (0.0010) [2023-12-27 06:39:08,489][20347] Updated weights for policy 0, policy_version 143851 (0.0008) [2023-12-27 06:39:08,543][20347] Updated weights for policy 0, policy_version 143861 (0.0007) [2023-12-27 06:39:08,601][20347] Updated weights for policy 0, policy_version 143871 (0.0007) [2023-12-27 06:39:08,973][20381] Updated weights for policy 1, policy_version 143612 (0.0008) [2023-12-27 06:39:09,029][20381] Updated weights for policy 1, policy_version 143622 (0.0005) [2023-12-27 06:39:09,077][20381] Updated weights for policy 1, policy_version 143632 (0.0006) [2023-12-27 06:39:09,253][20347] Updated weights for policy 0, policy_version 143881 (0.0008) [2023-12-27 06:39:09,316][20347] Updated weights for policy 0, policy_version 143891 (0.0008) [2023-12-27 06:39:09,376][20347] Updated weights for policy 0, policy_version 143901 (0.0007) [2023-12-27 06:39:09,439][20347] Updated weights for policy 0, policy_version 143911 (0.0009) [2023-12-27 06:39:09,788][20381] Updated weights for policy 1, policy_version 143642 (0.0008) [2023-12-27 06:39:09,849][20381] Updated weights for policy 1, policy_version 143652 (0.0008) [2023-12-27 06:39:09,910][20381] Updated weights for policy 1, policy_version 143662 (0.0009) [2023-12-27 06:39:09,974][20381] Updated weights for policy 1, policy_version 143672 (0.0009) [2023-12-27 06:39:10,044][20347] Updated weights for policy 0, policy_version 143921 (0.0010) [2023-12-27 06:39:10,104][20347] Updated weights for policy 0, policy_version 143931 (0.0010) [2023-12-27 06:39:10,160][20347] Updated weights for policy 0, policy_version 143941 (0.0010) [2023-12-27 06:39:10,695][20381] Updated weights for policy 1, policy_version 143682 (0.0011) [2023-12-27 06:39:10,748][20381] Updated weights for policy 1, policy_version 143692 (0.0009) [2023-12-27 06:39:10,804][20381] Updated weights for policy 1, policy_version 143702 (0.0010) [2023-12-27 06:39:10,810][20347] Updated weights for policy 0, policy_version 143951 (0.0010) [2023-12-27 06:39:10,862][20347] Updated weights for policy 0, policy_version 143961 (0.0010) [2023-12-27 06:39:10,914][20347] Updated weights for policy 0, policy_version 143971 (0.0010) [2023-12-27 06:39:11,471][20381] Updated weights for policy 1, policy_version 143712 (0.0010) [2023-12-27 06:39:11,530][20381] Updated weights for policy 1, policy_version 143722 (0.0011) [2023-12-27 06:39:11,593][20347] Updated weights for policy 0, policy_version 143981 (0.0010) [2023-12-27 06:39:11,593][20381] Updated weights for policy 1, policy_version 143732 (0.0010) [2023-12-27 06:39:11,655][20347] Updated weights for policy 0, policy_version 143991 (0.0010) [2023-12-27 06:39:11,714][20347] Updated weights for policy 0, policy_version 144001 (0.0010) [2023-12-27 06:39:12,211][20381] Updated weights for policy 1, policy_version 143742 (0.0008) [2023-12-27 06:39:12,273][20381] Updated weights for policy 1, policy_version 143752 (0.0007) [2023-12-27 06:39:12,326][20381] Updated weights for policy 1, policy_version 143762 (0.0008) [2023-12-27 06:39:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 73687040. Throughput: 0: 10281.7, 1: 10391.0. Samples: 73682772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.320')] [2023-12-27 06:39:12,421][20347] Updated weights for policy 0, policy_version 144011 (0.0008) [2023-12-27 06:39:12,486][20347] Updated weights for policy 0, policy_version 144021 (0.0005) [2023-12-27 06:39:12,550][20347] Updated weights for policy 0, policy_version 144031 (0.0005) [2023-12-27 06:39:13,039][20381] Updated weights for policy 1, policy_version 143772 (0.0008) [2023-12-27 06:39:13,091][20381] Updated weights for policy 1, policy_version 143782 (0.0008) [2023-12-27 06:39:13,101][20347] Updated weights for policy 0, policy_version 144041 (0.0005) [2023-12-27 06:39:13,145][20381] Updated weights for policy 1, policy_version 143792 (0.0006) [2023-12-27 06:39:13,162][20347] Updated weights for policy 0, policy_version 144051 (0.0005) [2023-12-27 06:39:13,222][20347] Updated weights for policy 0, policy_version 144061 (0.0005) [2023-12-27 06:39:13,279][20347] Updated weights for policy 0, policy_version 144071 (0.0009) [2023-12-27 06:39:13,844][20347] Updated weights for policy 0, policy_version 144081 (0.0008) [2023-12-27 06:39:13,885][20381] Updated weights for policy 1, policy_version 143802 (0.0008) [2023-12-27 06:39:13,904][20347] Updated weights for policy 0, policy_version 144091 (0.0008) [2023-12-27 06:39:13,930][20381] Updated weights for policy 1, policy_version 143812 (0.0007) [2023-12-27 06:39:13,958][20347] Updated weights for policy 0, policy_version 144101 (0.0008) [2023-12-27 06:39:13,984][20381] Updated weights for policy 1, policy_version 143822 (0.0005) [2023-12-27 06:39:14,031][20381] Updated weights for policy 1, policy_version 143832 (0.0005) [2023-12-27 06:39:14,644][20347] Updated weights for policy 0, policy_version 144111 (0.0008) [2023-12-27 06:39:14,663][20381] Updated weights for policy 1, policy_version 143842 (0.0005) [2023-12-27 06:39:14,695][20347] Updated weights for policy 0, policy_version 144121 (0.0007) [2023-12-27 06:39:14,711][20381] Updated weights for policy 1, policy_version 143852 (0.0010) [2023-12-27 06:39:14,749][20347] Updated weights for policy 0, policy_version 144131 (0.0006) [2023-12-27 06:39:14,763][20381] Updated weights for policy 1, policy_version 143862 (0.0010) [2023-12-27 06:39:15,456][20381] Updated weights for policy 1, policy_version 143872 (0.0010) [2023-12-27 06:39:15,460][20347] Updated weights for policy 0, policy_version 144141 (0.0007) [2023-12-27 06:39:15,508][20347] Updated weights for policy 0, policy_version 144151 (0.0006) [2023-12-27 06:39:15,511][20381] Updated weights for policy 1, policy_version 143882 (0.0010) [2023-12-27 06:39:15,561][20347] Updated weights for policy 0, policy_version 144161 (0.0005) [2023-12-27 06:39:15,567][20381] Updated weights for policy 1, policy_version 143892 (0.0011) [2023-12-27 06:39:16,216][20381] Updated weights for policy 1, policy_version 143902 (0.0007) [2023-12-27 06:39:16,276][20381] Updated weights for policy 1, policy_version 143912 (0.0006) [2023-12-27 06:39:16,285][20347] Updated weights for policy 0, policy_version 144171 (0.0007) [2023-12-27 06:39:16,324][20381] Updated weights for policy 1, policy_version 143922 (0.0006) [2023-12-27 06:39:16,332][20347] Updated weights for policy 0, policy_version 144181 (0.0008) [2023-12-27 06:39:16,376][20347] Updated weights for policy 0, policy_version 144191 (0.0007) [2023-12-27 06:39:16,985][20381] Updated weights for policy 1, policy_version 143932 (0.0006) [2023-12-27 06:39:17,033][20381] Updated weights for policy 1, policy_version 143942 (0.0008) [2023-12-27 06:39:17,083][20347] Updated weights for policy 0, policy_version 144201 (0.0008) [2023-12-27 06:39:17,084][20381] Updated weights for policy 1, policy_version 143952 (0.0008) [2023-12-27 06:39:17,143][20347] Updated weights for policy 0, policy_version 144211 (0.0007) [2023-12-27 06:39:17,193][20347] Updated weights for policy 0, policy_version 144221 (0.0007) [2023-12-27 06:39:17,245][20347] Updated weights for policy 0, policy_version 144231 (0.0007) [2023-12-27 06:39:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 73801728. Throughput: 0: 10292.4, 1: 10316.5. Samples: 73807580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.340')] [2023-12-27 06:39:17,817][20381] Updated weights for policy 1, policy_version 143962 (0.0008) [2023-12-27 06:39:17,874][20381] Updated weights for policy 1, policy_version 143972 (0.0008) [2023-12-27 06:39:17,929][20381] Updated weights for policy 1, policy_version 143982 (0.0006) [2023-12-27 06:39:17,946][20347] Updated weights for policy 0, policy_version 144241 (0.0007) [2023-12-27 06:39:17,988][20381] Updated weights for policy 1, policy_version 143992 (0.0005) [2023-12-27 06:39:18,006][20347] Updated weights for policy 0, policy_version 144251 (0.0007) [2023-12-27 06:39:18,067][20347] Updated weights for policy 0, policy_version 144261 (0.0008) [2023-12-27 06:39:18,712][20381] Updated weights for policy 1, policy_version 144002 (0.0007) [2023-12-27 06:39:18,768][20381] Updated weights for policy 1, policy_version 144012 (0.0007) [2023-12-27 06:39:18,777][20347] Updated weights for policy 0, policy_version 144271 (0.0008) [2023-12-27 06:39:18,818][20381] Updated weights for policy 1, policy_version 144022 (0.0008) [2023-12-27 06:39:18,835][20347] Updated weights for policy 0, policy_version 144281 (0.0008) [2023-12-27 06:39:18,889][20347] Updated weights for policy 0, policy_version 144291 (0.0008) [2023-12-27 06:39:19,540][20347] Updated weights for policy 0, policy_version 144301 (0.0007) [2023-12-27 06:39:19,576][20381] Updated weights for policy 1, policy_version 144032 (0.0006) [2023-12-27 06:39:19,594][20347] Updated weights for policy 0, policy_version 144311 (0.0006) [2023-12-27 06:39:19,632][20381] Updated weights for policy 1, policy_version 144042 (0.0006) [2023-12-27 06:39:19,648][20347] Updated weights for policy 0, policy_version 144321 (0.0006) [2023-12-27 06:39:19,695][20381] Updated weights for policy 1, policy_version 144052 (0.0005) [2023-12-27 06:39:20,327][20381] Updated weights for policy 1, policy_version 144062 (0.0005) [2023-12-27 06:39:20,387][20381] Updated weights for policy 1, policy_version 144072 (0.0007) [2023-12-27 06:39:20,402][20347] Updated weights for policy 0, policy_version 144331 (0.0007) [2023-12-27 06:39:20,435][20381] Updated weights for policy 1, policy_version 144082 (0.0006) [2023-12-27 06:39:20,459][20347] Updated weights for policy 0, policy_version 144341 (0.0006) [2023-12-27 06:39:20,515][20347] Updated weights for policy 0, policy_version 144351 (0.0007) [2023-12-27 06:39:21,143][20381] Updated weights for policy 1, policy_version 144092 (0.0007) [2023-12-27 06:39:21,157][20347] Updated weights for policy 0, policy_version 144361 (0.0009) [2023-12-27 06:39:21,208][20381] Updated weights for policy 1, policy_version 144102 (0.0007) [2023-12-27 06:39:21,217][20347] Updated weights for policy 0, policy_version 144371 (0.0006) [2023-12-27 06:39:21,268][20381] Updated weights for policy 1, policy_version 144112 (0.0007) [2023-12-27 06:39:21,276][20347] Updated weights for policy 0, policy_version 144381 (0.0008) [2023-12-27 06:39:21,327][20347] Updated weights for policy 0, policy_version 144391 (0.0007) [2023-12-27 06:39:21,966][20381] Updated weights for policy 1, policy_version 144122 (0.0006) [2023-12-27 06:39:22,025][20347] Updated weights for policy 0, policy_version 144401 (0.0006) [2023-12-27 06:39:22,026][20381] Updated weights for policy 1, policy_version 144132 (0.0006) [2023-12-27 06:39:22,077][20381] Updated weights for policy 1, policy_version 144142 (0.0006) [2023-12-27 06:39:22,081][20347] Updated weights for policy 0, policy_version 144411 (0.0007) [2023-12-27 06:39:22,124][20381] Updated weights for policy 1, policy_version 144152 (0.0007) [2023-12-27 06:39:22,139][20347] Updated weights for policy 0, policy_version 144421 (0.0008) [2023-12-27 06:39:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 73900032. Throughput: 0: 10293.0, 1: 10278.2. Samples: 73867224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:39:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000144424_36986880.pth... [2023-12-27 06:39:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000144152_36913152.pth... [2023-12-27 06:39:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000142968_36610048.pth [2023-12-27 06:39:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000143208_36675584.pth [2023-12-27 06:39:22,785][20347] Updated weights for policy 0, policy_version 144431 (0.0007) [2023-12-27 06:39:22,845][20347] Updated weights for policy 0, policy_version 144441 (0.0006) [2023-12-27 06:39:22,864][20381] Updated weights for policy 1, policy_version 144162 (0.0008) [2023-12-27 06:39:22,905][20347] Updated weights for policy 0, policy_version 144451 (0.0008) [2023-12-27 06:39:22,926][20381] Updated weights for policy 1, policy_version 144172 (0.0008) [2023-12-27 06:39:22,981][20381] Updated weights for policy 1, policy_version 144182 (0.0008) [2023-12-27 06:39:23,546][20347] Updated weights for policy 0, policy_version 144461 (0.0008) [2023-12-27 06:39:23,602][20347] Updated weights for policy 0, policy_version 144471 (0.0005) [2023-12-27 06:39:23,663][20347] Updated weights for policy 0, policy_version 144481 (0.0005) [2023-12-27 06:39:23,693][20381] Updated weights for policy 1, policy_version 144192 (0.0008) [2023-12-27 06:39:23,751][20381] Updated weights for policy 1, policy_version 144202 (0.0008) [2023-12-27 06:39:23,804][20381] Updated weights for policy 1, policy_version 144212 (0.0008) [2023-12-27 06:39:24,185][20347] Updated weights for policy 0, policy_version 144491 (0.0006) [2023-12-27 06:39:24,238][20347] Updated weights for policy 0, policy_version 144501 (0.0007) [2023-12-27 06:39:24,293][20347] Updated weights for policy 0, policy_version 144511 (0.0008) [2023-12-27 06:39:24,548][20381] Updated weights for policy 1, policy_version 144222 (0.0008) [2023-12-27 06:39:24,599][20381] Updated weights for policy 1, policy_version 144232 (0.0007) [2023-12-27 06:39:24,660][20381] Updated weights for policy 1, policy_version 144242 (0.0005) [2023-12-27 06:39:25,021][20347] Updated weights for policy 0, policy_version 144521 (0.0007) [2023-12-27 06:39:25,076][20347] Updated weights for policy 0, policy_version 144531 (0.0008) [2023-12-27 06:39:25,130][20347] Updated weights for policy 0, policy_version 144541 (0.0005) [2023-12-27 06:39:25,182][20347] Updated weights for policy 0, policy_version 144551 (0.0005) [2023-12-27 06:39:25,282][20381] Updated weights for policy 1, policy_version 144252 (0.0006) [2023-12-27 06:39:25,347][20381] Updated weights for policy 1, policy_version 144262 (0.0008) [2023-12-27 06:39:25,398][20381] Updated weights for policy 1, policy_version 144272 (0.0008) [2023-12-27 06:39:25,786][20347] Updated weights for policy 0, policy_version 144561 (0.0007) [2023-12-27 06:39:25,838][20347] Updated weights for policy 0, policy_version 144571 (0.0007) [2023-12-27 06:39:25,893][20347] Updated weights for policy 0, policy_version 144581 (0.0007) [2023-12-27 06:39:26,113][20381] Updated weights for policy 1, policy_version 144282 (0.0006) [2023-12-27 06:39:26,162][20381] Updated weights for policy 1, policy_version 144292 (0.0008) [2023-12-27 06:39:26,218][20381] Updated weights for policy 1, policy_version 144302 (0.0008) [2023-12-27 06:39:26,264][20381] Updated weights for policy 1, policy_version 144312 (0.0007) [2023-12-27 06:39:26,451][20347] Updated weights for policy 0, policy_version 144591 (0.0006) [2023-12-27 06:39:26,502][20347] Updated weights for policy 0, policy_version 144601 (0.0005) [2023-12-27 06:39:26,548][20347] Updated weights for policy 0, policy_version 144611 (0.0005) [2023-12-27 06:39:26,873][20381] Updated weights for policy 1, policy_version 144322 (0.0008) [2023-12-27 06:39:26,925][20381] Updated weights for policy 1, policy_version 144332 (0.0008) [2023-12-27 06:39:26,973][20381] Updated weights for policy 1, policy_version 144342 (0.0007) [2023-12-27 06:39:27,181][20347] Updated weights for policy 0, policy_version 144621 (0.0007) [2023-12-27 06:39:27,225][20347] Updated weights for policy 0, policy_version 144631 (0.0007) [2023-12-27 06:39:27,270][20347] Updated weights for policy 0, policy_version 144641 (0.0008) [2023-12-27 06:39:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 74006528. Throughput: 0: 10364.5, 1: 10218.8. Samples: 73989740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:39:27,576][20381] Updated weights for policy 1, policy_version 144352 (0.0010) [2023-12-27 06:39:27,633][20381] Updated weights for policy 1, policy_version 144362 (0.0010) [2023-12-27 06:39:27,694][20381] Updated weights for policy 1, policy_version 144372 (0.0005) [2023-12-27 06:39:28,036][20347] Updated weights for policy 0, policy_version 144651 (0.0009) [2023-12-27 06:39:28,085][20347] Updated weights for policy 0, policy_version 144661 (0.0007) [2023-12-27 06:39:28,145][20347] Updated weights for policy 0, policy_version 144671 (0.0005) [2023-12-27 06:39:28,264][20381] Updated weights for policy 1, policy_version 144382 (0.0007) [2023-12-27 06:39:28,315][20381] Updated weights for policy 1, policy_version 144392 (0.0008) [2023-12-27 06:39:28,375][20381] Updated weights for policy 1, policy_version 144402 (0.0008) [2023-12-27 06:39:28,825][20347] Updated weights for policy 0, policy_version 144681 (0.0006) [2023-12-27 06:39:28,880][20347] Updated weights for policy 0, policy_version 144691 (0.0007) [2023-12-27 06:39:28,935][20347] Updated weights for policy 0, policy_version 144701 (0.0007) [2023-12-27 06:39:28,972][20381] Updated weights for policy 1, policy_version 144412 (0.0006) [2023-12-27 06:39:28,994][20347] Updated weights for policy 0, policy_version 144711 (0.0008) [2023-12-27 06:39:29,028][20381] Updated weights for policy 1, policy_version 144422 (0.0005) [2023-12-27 06:39:29,083][20381] Updated weights for policy 1, policy_version 144432 (0.0005) [2023-12-27 06:39:29,689][20381] Updated weights for policy 1, policy_version 144442 (0.0005) [2023-12-27 06:39:29,738][20381] Updated weights for policy 1, policy_version 144452 (0.0005) [2023-12-27 06:39:29,747][20347] Updated weights for policy 0, policy_version 144721 (0.0007) [2023-12-27 06:39:29,782][20381] Updated weights for policy 1, policy_version 144462 (0.0005) [2023-12-27 06:39:29,806][20347] Updated weights for policy 0, policy_version 144731 (0.0009) [2023-12-27 06:39:29,836][20381] Updated weights for policy 1, policy_version 144472 (0.0006) [2023-12-27 06:39:29,866][20347] Updated weights for policy 0, policy_version 144741 (0.0007) [2023-12-27 06:39:30,466][20381] Updated weights for policy 1, policy_version 144482 (0.0006) [2023-12-27 06:39:30,511][20381] Updated weights for policy 1, policy_version 144492 (0.0008) [2023-12-27 06:39:30,549][20347] Updated weights for policy 0, policy_version 144751 (0.0008) [2023-12-27 06:39:30,553][20381] Updated weights for policy 1, policy_version 144502 (0.0007) [2023-12-27 06:39:30,599][20347] Updated weights for policy 0, policy_version 144761 (0.0007) [2023-12-27 06:39:30,643][20347] Updated weights for policy 0, policy_version 144771 (0.0007) [2023-12-27 06:39:31,204][20381] Updated weights for policy 1, policy_version 144512 (0.0008) [2023-12-27 06:39:31,264][20381] Updated weights for policy 1, policy_version 144522 (0.0007) [2023-12-27 06:39:31,322][20381] Updated weights for policy 1, policy_version 144532 (0.0008) [2023-12-27 06:39:31,354][20347] Updated weights for policy 0, policy_version 144781 (0.0007) [2023-12-27 06:39:31,419][20347] Updated weights for policy 0, policy_version 144791 (0.0008) [2023-12-27 06:39:31,475][20347] Updated weights for policy 0, policy_version 144801 (0.0007) [2023-12-27 06:39:32,020][20381] Updated weights for policy 1, policy_version 144542 (0.0006) [2023-12-27 06:39:32,074][20381] Updated weights for policy 1, policy_version 144552 (0.0005) [2023-12-27 06:39:32,130][20381] Updated weights for policy 1, policy_version 144562 (0.0008) [2023-12-27 06:39:32,163][20347] Updated weights for policy 0, policy_version 144811 (0.0008) [2023-12-27 06:39:32,220][20347] Updated weights for policy 0, policy_version 144821 (0.0006) [2023-12-27 06:39:32,281][20347] Updated weights for policy 0, policy_version 144831 (0.0007) [2023-12-27 06:39:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 74113024. Throughput: 0: 10336.0, 1: 10373.2. Samples: 74117860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:39:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:39:32,773][20381] Updated weights for policy 1, policy_version 144572 (0.0007) [2023-12-27 06:39:32,828][20381] Updated weights for policy 1, policy_version 144582 (0.0007) [2023-12-27 06:39:32,878][20381] Updated weights for policy 1, policy_version 144592 (0.0006) [2023-12-27 06:39:32,944][20347] Updated weights for policy 0, policy_version 144841 (0.0007) [2023-12-27 06:39:33,004][20347] Updated weights for policy 0, policy_version 144851 (0.0006) [2023-12-27 06:39:33,062][20347] Updated weights for policy 0, policy_version 144861 (0.0006) [2023-12-27 06:39:33,121][20347] Updated weights for policy 0, policy_version 144871 (0.0005) [2023-12-27 06:39:33,438][20381] Updated weights for policy 1, policy_version 144602 (0.0005) [2023-12-27 06:39:33,482][20381] Updated weights for policy 1, policy_version 144612 (0.0005) [2023-12-27 06:39:33,532][20381] Updated weights for policy 1, policy_version 144622 (0.0005) [2023-12-27 06:39:33,584][20381] Updated weights for policy 1, policy_version 144632 (0.0006) [2023-12-27 06:39:33,718][20347] Updated weights for policy 0, policy_version 144881 (0.0006) [2023-12-27 06:39:33,781][20347] Updated weights for policy 0, policy_version 144891 (0.0006) [2023-12-27 06:39:33,839][20347] Updated weights for policy 0, policy_version 144901 (0.0009) [2023-12-27 06:39:34,167][20381] Updated weights for policy 1, policy_version 144642 (0.0008) [2023-12-27 06:39:34,225][20381] Updated weights for policy 1, policy_version 144652 (0.0008) [2023-12-27 06:39:34,283][20381] Updated weights for policy 1, policy_version 144662 (0.0007) [2023-12-27 06:39:34,501][20347] Updated weights for policy 0, policy_version 144911 (0.0008) [2023-12-27 06:39:34,559][20347] Updated weights for policy 0, policy_version 144921 (0.0008) [2023-12-27 06:39:34,619][20347] Updated weights for policy 0, policy_version 144931 (0.0008) [2023-12-27 06:39:34,947][20381] Updated weights for policy 1, policy_version 144672 (0.0006) [2023-12-27 06:39:35,005][20381] Updated weights for policy 1, policy_version 144682 (0.0005) [2023-12-27 06:39:35,054][20381] Updated weights for policy 1, policy_version 144692 (0.0011) [2023-12-27 06:39:35,353][20347] Updated weights for policy 0, policy_version 144941 (0.0008) [2023-12-27 06:39:35,403][20347] Updated weights for policy 0, policy_version 144951 (0.0007) [2023-12-27 06:39:35,463][20347] Updated weights for policy 0, policy_version 144961 (0.0007) [2023-12-27 06:39:35,605][20381] Updated weights for policy 1, policy_version 144702 (0.0009) [2023-12-27 06:39:35,666][20381] Updated weights for policy 1, policy_version 144712 (0.0011) [2023-12-27 06:39:35,721][20381] Updated weights for policy 1, policy_version 144722 (0.0011) [2023-12-27 06:39:36,207][20347] Updated weights for policy 0, policy_version 144971 (0.0008) [2023-12-27 06:39:36,273][20347] Updated weights for policy 0, policy_version 144981 (0.0006) [2023-12-27 06:39:36,328][20381] Updated weights for policy 1, policy_version 144732 (0.0009) [2023-12-27 06:39:36,334][20347] Updated weights for policy 0, policy_version 144991 (0.0007) [2023-12-27 06:39:36,390][20381] Updated weights for policy 1, policy_version 144742 (0.0009) [2023-12-27 06:39:36,450][20381] Updated weights for policy 1, policy_version 144752 (0.0010) [2023-12-27 06:39:36,983][20347] Updated weights for policy 0, policy_version 145001 (0.0008) [2023-12-27 06:39:37,042][20347] Updated weights for policy 0, policy_version 145011 (0.0011) [2023-12-27 06:39:37,069][20381] Updated weights for policy 1, policy_version 144762 (0.0006) [2023-12-27 06:39:37,101][20347] Updated weights for policy 0, policy_version 145021 (0.0011) [2023-12-27 06:39:37,128][20381] Updated weights for policy 1, policy_version 144772 (0.0010) [2023-12-27 06:39:37,157][20347] Updated weights for policy 0, policy_version 145031 (0.0010) [2023-12-27 06:39:37,184][20381] Updated weights for policy 1, policy_version 144782 (0.0010) [2023-12-27 06:39:37,240][20381] Updated weights for policy 1, policy_version 144792 (0.0011) [2023-12-27 06:39:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 74219520. Throughput: 0: 10344.6, 1: 10384.0. Samples: 74182692. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:39:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:39:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000144792_37076992.pth... [2023-12-27 06:39:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000145032_37142528.pth... [2023-12-27 06:39:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000143816_36831232.pth [2023-12-27 06:39:37,395][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000143544_36757504.pth [2023-12-27 06:39:37,773][20347] Updated weights for policy 0, policy_version 145041 (0.0006) [2023-12-27 06:39:37,836][20347] Updated weights for policy 0, policy_version 145051 (0.0005) [2023-12-27 06:39:37,888][20347] Updated weights for policy 0, policy_version 145061 (0.0005) [2023-12-27 06:39:37,932][20381] Updated weights for policy 1, policy_version 144802 (0.0007) [2023-12-27 06:39:37,981][20381] Updated weights for policy 1, policy_version 144812 (0.0007) [2023-12-27 06:39:38,038][20381] Updated weights for policy 1, policy_version 144822 (0.0008) [2023-12-27 06:39:38,442][20347] Updated weights for policy 0, policy_version 145071 (0.0007) [2023-12-27 06:39:38,508][20347] Updated weights for policy 0, policy_version 145081 (0.0008) [2023-12-27 06:39:38,580][20347] Updated weights for policy 0, policy_version 145091 (0.0008) [2023-12-27 06:39:38,748][20381] Updated weights for policy 1, policy_version 144832 (0.0010) [2023-12-27 06:39:38,802][20381] Updated weights for policy 1, policy_version 144842 (0.0011) [2023-12-27 06:39:38,851][20381] Updated weights for policy 1, policy_version 144852 (0.0010) [2023-12-27 06:39:39,246][20347] Updated weights for policy 0, policy_version 145101 (0.0007) [2023-12-27 06:39:39,299][20347] Updated weights for policy 0, policy_version 145111 (0.0006) [2023-12-27 06:39:39,362][20347] Updated weights for policy 0, policy_version 145121 (0.0007) [2023-12-27 06:39:39,556][20381] Updated weights for policy 1, policy_version 144862 (0.0009) [2023-12-27 06:39:39,611][20381] Updated weights for policy 1, policy_version 144872 (0.0007) [2023-12-27 06:39:39,673][20381] Updated weights for policy 1, policy_version 144882 (0.0008) [2023-12-27 06:39:40,053][20347] Updated weights for policy 0, policy_version 145131 (0.0006) [2023-12-27 06:39:40,117][20347] Updated weights for policy 0, policy_version 145141 (0.0006) [2023-12-27 06:39:40,183][20347] Updated weights for policy 0, policy_version 145151 (0.0010) [2023-12-27 06:39:40,392][20381] Updated weights for policy 1, policy_version 144892 (0.0008) [2023-12-27 06:39:40,451][20381] Updated weights for policy 1, policy_version 144902 (0.0008) [2023-12-27 06:39:40,503][20381] Updated weights for policy 1, policy_version 144912 (0.0008) [2023-12-27 06:39:40,848][20347] Updated weights for policy 0, policy_version 145161 (0.0010) [2023-12-27 06:39:40,908][20347] Updated weights for policy 0, policy_version 145171 (0.0010) [2023-12-27 06:39:40,965][20347] Updated weights for policy 0, policy_version 145181 (0.0010) [2023-12-27 06:39:41,022][20347] Updated weights for policy 0, policy_version 145191 (0.0007) [2023-12-27 06:39:41,215][20381] Updated weights for policy 1, policy_version 144922 (0.0008) [2023-12-27 06:39:41,277][20381] Updated weights for policy 1, policy_version 144932 (0.0009) [2023-12-27 06:39:41,342][20381] Updated weights for policy 1, policy_version 144942 (0.0009) [2023-12-27 06:39:41,405][20381] Updated weights for policy 1, policy_version 144952 (0.0010) [2023-12-27 06:39:41,700][20347] Updated weights for policy 0, policy_version 145201 (0.0010) [2023-12-27 06:39:41,766][20347] Updated weights for policy 0, policy_version 145211 (0.0010) [2023-12-27 06:39:41,825][20347] Updated weights for policy 0, policy_version 145221 (0.0010) [2023-12-27 06:39:42,070][20381] Updated weights for policy 1, policy_version 144962 (0.0006) [2023-12-27 06:39:42,130][20381] Updated weights for policy 1, policy_version 144972 (0.0005) [2023-12-27 06:39:42,188][20381] Updated weights for policy 1, policy_version 144982 (0.0005) [2023-12-27 06:39:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 74317824. Throughput: 0: 10345.8, 1: 10371.4. Samples: 74306552. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:39:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '988.930')] [2023-12-27 06:39:42,506][20347] Updated weights for policy 0, policy_version 145231 (0.0009) [2023-12-27 06:39:42,562][20347] Updated weights for policy 0, policy_version 145241 (0.0008) [2023-12-27 06:39:42,616][20347] Updated weights for policy 0, policy_version 145251 (0.0008) [2023-12-27 06:39:42,849][20381] Updated weights for policy 1, policy_version 144992 (0.0010) [2023-12-27 06:39:42,904][20381] Updated weights for policy 1, policy_version 145002 (0.0011) [2023-12-27 06:39:42,953][20381] Updated weights for policy 1, policy_version 145012 (0.0011) [2023-12-27 06:39:43,181][20347] Updated weights for policy 0, policy_version 145261 (0.0006) [2023-12-27 06:39:43,241][20347] Updated weights for policy 0, policy_version 145271 (0.0005) [2023-12-27 06:39:43,298][20347] Updated weights for policy 0, policy_version 145281 (0.0010) [2023-12-27 06:39:43,646][20381] Updated weights for policy 1, policy_version 145022 (0.0009) [2023-12-27 06:39:43,697][20381] Updated weights for policy 1, policy_version 145032 (0.0010) [2023-12-27 06:39:43,745][20381] Updated weights for policy 1, policy_version 145042 (0.0010) [2023-12-27 06:39:43,936][20347] Updated weights for policy 0, policy_version 145291 (0.0010) [2023-12-27 06:39:43,980][20347] Updated weights for policy 0, policy_version 145301 (0.0010) [2023-12-27 06:39:44,025][20347] Updated weights for policy 0, policy_version 145311 (0.0010) [2023-12-27 06:39:44,415][20381] Updated weights for policy 1, policy_version 145052 (0.0009) [2023-12-27 06:39:44,463][20381] Updated weights for policy 1, policy_version 145062 (0.0010) [2023-12-27 06:39:44,515][20381] Updated weights for policy 1, policy_version 145072 (0.0010) [2023-12-27 06:39:44,720][20347] Updated weights for policy 0, policy_version 145321 (0.0010) [2023-12-27 06:39:44,779][20347] Updated weights for policy 0, policy_version 145331 (0.0009) [2023-12-27 06:39:44,838][20347] Updated weights for policy 0, policy_version 145341 (0.0007) [2023-12-27 06:39:44,899][20347] Updated weights for policy 0, policy_version 145351 (0.0007) [2023-12-27 06:39:45,225][20381] Updated weights for policy 1, policy_version 145082 (0.0010) [2023-12-27 06:39:45,277][20381] Updated weights for policy 1, policy_version 145092 (0.0011) [2023-12-27 06:39:45,331][20381] Updated weights for policy 1, policy_version 145102 (0.0011) [2023-12-27 06:39:45,380][20381] Updated weights for policy 1, policy_version 145112 (0.0011) [2023-12-27 06:39:45,545][20347] Updated weights for policy 0, policy_version 145361 (0.0009) [2023-12-27 06:39:45,602][20347] Updated weights for policy 0, policy_version 145371 (0.0010) [2023-12-27 06:39:45,653][20347] Updated weights for policy 0, policy_version 145381 (0.0010) [2023-12-27 06:39:46,063][20381] Updated weights for policy 1, policy_version 145122 (0.0011) [2023-12-27 06:39:46,118][20381] Updated weights for policy 1, policy_version 145132 (0.0011) [2023-12-27 06:39:46,175][20381] Updated weights for policy 1, policy_version 145142 (0.0010) [2023-12-27 06:39:46,333][20347] Updated weights for policy 0, policy_version 145391 (0.0010) [2023-12-27 06:39:46,386][20347] Updated weights for policy 0, policy_version 145401 (0.0010) [2023-12-27 06:39:46,440][20347] Updated weights for policy 0, policy_version 145411 (0.0010) [2023-12-27 06:39:46,795][20381] Updated weights for policy 1, policy_version 145152 (0.0006) [2023-12-27 06:39:46,858][20381] Updated weights for policy 1, policy_version 145162 (0.0006) [2023-12-27 06:39:46,912][20381] Updated weights for policy 1, policy_version 145172 (0.0010) [2023-12-27 06:39:47,102][20347] Updated weights for policy 0, policy_version 145421 (0.0009) [2023-12-27 06:39:47,164][20347] Updated weights for policy 0, policy_version 145431 (0.0007) [2023-12-27 06:39:47,223][20347] Updated weights for policy 0, policy_version 145441 (0.0007) [2023-12-27 06:39:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 74424320. Throughput: 0: 10386.3, 1: 10389.4. Samples: 74431676. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:39:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.450')] [2023-12-27 06:39:47,575][20381] Updated weights for policy 1, policy_version 145182 (0.0009) [2023-12-27 06:39:47,630][20381] Updated weights for policy 1, policy_version 145192 (0.0006) [2023-12-27 06:39:47,682][20381] Updated weights for policy 1, policy_version 145202 (0.0007) [2023-12-27 06:39:47,913][20347] Updated weights for policy 0, policy_version 145451 (0.0007) [2023-12-27 06:39:47,972][20347] Updated weights for policy 0, policy_version 145461 (0.0005) [2023-12-27 06:39:48,035][20347] Updated weights for policy 0, policy_version 145471 (0.0010) [2023-12-27 06:39:48,278][20381] Updated weights for policy 1, policy_version 145212 (0.0006) [2023-12-27 06:39:48,345][20381] Updated weights for policy 1, policy_version 145222 (0.0006) [2023-12-27 06:39:48,407][20381] Updated weights for policy 1, policy_version 145232 (0.0008) [2023-12-27 06:39:48,644][20347] Updated weights for policy 0, policy_version 145481 (0.0010) [2023-12-27 06:39:48,705][20347] Updated weights for policy 0, policy_version 145491 (0.0006) [2023-12-27 06:39:48,764][20347] Updated weights for policy 0, policy_version 145501 (0.0006) [2023-12-27 06:39:48,820][20347] Updated weights for policy 0, policy_version 145511 (0.0007) [2023-12-27 06:39:49,084][20381] Updated weights for policy 1, policy_version 145242 (0.0008) [2023-12-27 06:39:49,150][20381] Updated weights for policy 1, policy_version 145252 (0.0008) [2023-12-27 06:39:49,207][20381] Updated weights for policy 1, policy_version 145262 (0.0006) [2023-12-27 06:39:49,264][20381] Updated weights for policy 1, policy_version 145272 (0.0006) [2023-12-27 06:39:49,436][20347] Updated weights for policy 0, policy_version 145521 (0.0009) [2023-12-27 06:39:49,486][20347] Updated weights for policy 0, policy_version 145531 (0.0008) [2023-12-27 06:39:49,533][20347] Updated weights for policy 0, policy_version 145541 (0.0008) [2023-12-27 06:39:49,922][20381] Updated weights for policy 1, policy_version 145282 (0.0008) [2023-12-27 06:39:49,982][20381] Updated weights for policy 1, policy_version 145292 (0.0008) [2023-12-27 06:39:50,036][20381] Updated weights for policy 1, policy_version 145302 (0.0006) [2023-12-27 06:39:50,293][20347] Updated weights for policy 0, policy_version 145551 (0.0008) [2023-12-27 06:39:50,352][20347] Updated weights for policy 0, policy_version 145561 (0.0008) [2023-12-27 06:39:50,407][20347] Updated weights for policy 0, policy_version 145571 (0.0007) [2023-12-27 06:39:50,722][20381] Updated weights for policy 1, policy_version 145312 (0.0007) [2023-12-27 06:39:50,779][20381] Updated weights for policy 1, policy_version 145322 (0.0008) [2023-12-27 06:39:50,836][20381] Updated weights for policy 1, policy_version 145332 (0.0008) [2023-12-27 06:39:51,097][20347] Updated weights for policy 0, policy_version 145581 (0.0007) [2023-12-27 06:39:51,159][20347] Updated weights for policy 0, policy_version 145591 (0.0007) [2023-12-27 06:39:51,218][20347] Updated weights for policy 0, policy_version 145601 (0.0007) [2023-12-27 06:39:51,520][20381] Updated weights for policy 1, policy_version 145342 (0.0008) [2023-12-27 06:39:51,581][20381] Updated weights for policy 1, policy_version 145352 (0.0008) [2023-12-27 06:39:51,645][20381] Updated weights for policy 1, policy_version 145362 (0.0007) [2023-12-27 06:39:51,976][20347] Updated weights for policy 0, policy_version 145611 (0.0007) [2023-12-27 06:39:52,026][20347] Updated weights for policy 0, policy_version 145621 (0.0007) [2023-12-27 06:39:52,077][20347] Updated weights for policy 0, policy_version 145631 (0.0007) [2023-12-27 06:39:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 74522624. Throughput: 0: 10391.1, 1: 10413.0. Samples: 74494340. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:39:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '966.020')] [2023-12-27 06:39:52,366][20381] Updated weights for policy 1, policy_version 145372 (0.0009) [2023-12-27 06:39:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000145640_37298176.pth... [2023-12-27 06:39:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000144424_36986880.pth [2023-12-27 06:39:52,373][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000145640_37298176.pth [2023-12-27 06:39:52,424][20381] Updated weights for policy 1, policy_version 145382 (0.0007) [2023-12-27 06:39:52,471][20381] Updated weights for policy 1, policy_version 145392 (0.0006) [2023-12-27 06:39:52,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000145400_37232640.pth... [2023-12-27 06:39:52,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000144152_36913152.pth [2023-12-27 06:39:52,522][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000145400_37232640.pth [2023-12-27 06:39:52,772][20347] Updated weights for policy 0, policy_version 145641 (0.0008) [2023-12-27 06:39:52,833][20347] Updated weights for policy 0, policy_version 145651 (0.0008) [2023-12-27 06:39:52,894][20347] Updated weights for policy 0, policy_version 145661 (0.0007) [2023-12-27 06:39:52,949][20347] Updated weights for policy 0, policy_version 145671 (0.0008) [2023-12-27 06:39:53,119][20381] Updated weights for policy 1, policy_version 145402 (0.0006) [2023-12-27 06:39:53,182][20381] Updated weights for policy 1, policy_version 145412 (0.0008) [2023-12-27 06:39:53,233][20381] Updated weights for policy 1, policy_version 145422 (0.0009) [2023-12-27 06:39:53,277][20381] Updated weights for policy 1, policy_version 145432 (0.0006) [2023-12-27 06:39:53,543][20347] Updated weights for policy 0, policy_version 145681 (0.0005) [2023-12-27 06:39:53,601][20347] Updated weights for policy 0, policy_version 145691 (0.0008) [2023-12-27 06:39:53,653][20347] Updated weights for policy 0, policy_version 145701 (0.0008) [2023-12-27 06:39:53,877][20381] Updated weights for policy 1, policy_version 145442 (0.0005) [2023-12-27 06:39:53,923][20381] Updated weights for policy 1, policy_version 145452 (0.0005) [2023-12-27 06:39:53,969][20381] Updated weights for policy 1, policy_version 145462 (0.0005) [2023-12-27 06:39:54,363][20347] Updated weights for policy 0, policy_version 145711 (0.0008) [2023-12-27 06:39:54,415][20347] Updated weights for policy 0, policy_version 145721 (0.0008) [2023-12-27 06:39:54,460][20347] Updated weights for policy 0, policy_version 145731 (0.0008) [2023-12-27 06:39:54,583][20381] Updated weights for policy 1, policy_version 145472 (0.0007) [2023-12-27 06:39:54,636][20381] Updated weights for policy 1, policy_version 145482 (0.0008) [2023-12-27 06:39:54,696][20381] Updated weights for policy 1, policy_version 145492 (0.0009) [2023-12-27 06:39:55,125][20347] Updated weights for policy 0, policy_version 145741 (0.0008) [2023-12-27 06:39:55,173][20347] Updated weights for policy 0, policy_version 145751 (0.0008) [2023-12-27 06:39:55,230][20347] Updated weights for policy 0, policy_version 145761 (0.0008) [2023-12-27 06:39:55,376][20381] Updated weights for policy 1, policy_version 145502 (0.0006) [2023-12-27 06:39:55,432][20381] Updated weights for policy 1, policy_version 145512 (0.0005) [2023-12-27 06:39:55,495][20381] Updated weights for policy 1, policy_version 145522 (0.0005) [2023-12-27 06:39:55,914][20347] Updated weights for policy 0, policy_version 145771 (0.0007) [2023-12-27 06:39:55,976][20347] Updated weights for policy 0, policy_version 145781 (0.0007) [2023-12-27 06:39:56,020][20347] Updated weights for policy 0, policy_version 145791 (0.0007) [2023-12-27 06:39:56,104][20381] Updated weights for policy 1, policy_version 145532 (0.0007) [2023-12-27 06:39:56,166][20381] Updated weights for policy 1, policy_version 145542 (0.0007) [2023-12-27 06:39:56,218][20381] Updated weights for policy 1, policy_version 145552 (0.0008) [2023-12-27 06:39:56,606][20347] Updated weights for policy 0, policy_version 145801 (0.0010) [2023-12-27 06:39:56,655][20347] Updated weights for policy 0, policy_version 145811 (0.0007) [2023-12-27 06:39:56,705][20347] Updated weights for policy 0, policy_version 145821 (0.0009) [2023-12-27 06:39:56,766][20347] Updated weights for policy 0, policy_version 145831 (0.0006) [2023-12-27 06:39:56,869][20381] Updated weights for policy 1, policy_version 145562 (0.0008) [2023-12-27 06:39:56,924][20381] Updated weights for policy 1, policy_version 145572 (0.0005) [2023-12-27 06:39:56,970][20381] Updated weights for policy 1, policy_version 145582 (0.0005) [2023-12-27 06:39:57,032][20381] Updated weights for policy 1, policy_version 145592 (0.0008) [2023-12-27 06:39:57,344][20347] Updated weights for policy 0, policy_version 145841 (0.0005) [2023-12-27 06:39:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 74629120. Throughput: 0: 10347.6, 1: 10438.8. Samples: 74618160. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:39:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '974.500')] [2023-12-27 06:39:57,402][20347] Updated weights for policy 0, policy_version 145851 (0.0005) [2023-12-27 06:39:57,456][20347] Updated weights for policy 0, policy_version 145861 (0.0005) [2023-12-27 06:39:57,670][20381] Updated weights for policy 1, policy_version 145602 (0.0008) [2023-12-27 06:39:57,719][20381] Updated weights for policy 1, policy_version 145612 (0.0007) [2023-12-27 06:39:57,768][20381] Updated weights for policy 1, policy_version 145622 (0.0007) [2023-12-27 06:39:58,023][20347] Updated weights for policy 0, policy_version 145871 (0.0007) [2023-12-27 06:39:58,080][20347] Updated weights for policy 0, policy_version 145881 (0.0006) [2023-12-27 06:39:58,167][20347] Updated weights for policy 0, policy_version 145891 (0.0007) [2023-12-27 06:39:58,471][20381] Updated weights for policy 1, policy_version 145632 (0.0008) [2023-12-27 06:39:58,530][20381] Updated weights for policy 1, policy_version 145642 (0.0008) [2023-12-27 06:39:58,588][20381] Updated weights for policy 1, policy_version 145652 (0.0008) [2023-12-27 06:39:58,876][20347] Updated weights for policy 0, policy_version 145901 (0.0007) [2023-12-27 06:39:58,921][20347] Updated weights for policy 0, policy_version 145911 (0.0008) [2023-12-27 06:39:58,976][20347] Updated weights for policy 0, policy_version 145921 (0.0007) [2023-12-27 06:39:59,338][20381] Updated weights for policy 1, policy_version 145662 (0.0007) [2023-12-27 06:39:59,402][20381] Updated weights for policy 1, policy_version 145672 (0.0007) [2023-12-27 06:39:59,461][20381] Updated weights for policy 1, policy_version 145682 (0.0007) [2023-12-27 06:39:59,707][20347] Updated weights for policy 0, policy_version 145931 (0.0008) [2023-12-27 06:39:59,758][20347] Updated weights for policy 0, policy_version 145941 (0.0009) [2023-12-27 06:39:59,802][20347] Updated weights for policy 0, policy_version 145951 (0.0006) [2023-12-27 06:40:00,119][20381] Updated weights for policy 1, policy_version 145692 (0.0007) [2023-12-27 06:40:00,174][20381] Updated weights for policy 1, policy_version 145702 (0.0008) [2023-12-27 06:40:00,225][20381] Updated weights for policy 1, policy_version 145712 (0.0008) [2023-12-27 06:40:00,463][20347] Updated weights for policy 0, policy_version 145961 (0.0007) [2023-12-27 06:40:00,521][20347] Updated weights for policy 0, policy_version 145971 (0.0005) [2023-12-27 06:40:00,573][20347] Updated weights for policy 0, policy_version 145981 (0.0005) [2023-12-27 06:40:00,629][20347] Updated weights for policy 0, policy_version 145991 (0.0005) [2023-12-27 06:40:00,963][20381] Updated weights for policy 1, policy_version 145722 (0.0009) [2023-12-27 06:40:01,010][20381] Updated weights for policy 1, policy_version 145732 (0.0010) [2023-12-27 06:40:01,068][20381] Updated weights for policy 1, policy_version 145742 (0.0011) [2023-12-27 06:40:01,116][20381] Updated weights for policy 1, policy_version 145752 (0.0010) [2023-12-27 06:40:01,174][20347] Updated weights for policy 0, policy_version 146001 (0.0007) [2023-12-27 06:40:01,231][20347] Updated weights for policy 0, policy_version 146011 (0.0008) [2023-12-27 06:40:01,286][20347] Updated weights for policy 0, policy_version 146021 (0.0007) [2023-12-27 06:40:01,767][20381] Updated weights for policy 1, policy_version 145762 (0.0008) [2023-12-27 06:40:01,821][20381] Updated weights for policy 1, policy_version 145772 (0.0007) [2023-12-27 06:40:01,877][20381] Updated weights for policy 1, policy_version 145782 (0.0008) [2023-12-27 06:40:02,021][20347] Updated weights for policy 0, policy_version 146031 (0.0006) [2023-12-27 06:40:02,082][20347] Updated weights for policy 0, policy_version 146041 (0.0009) [2023-12-27 06:40:02,141][20347] Updated weights for policy 0, policy_version 146051 (0.0009) [2023-12-27 06:40:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 74735616. Throughput: 0: 10391.2, 1: 10418.5. Samples: 74744016. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:02,597][20381] Updated weights for policy 1, policy_version 145792 (0.0008) [2023-12-27 06:40:02,649][20381] Updated weights for policy 1, policy_version 145802 (0.0008) [2023-12-27 06:40:02,702][20381] Updated weights for policy 1, policy_version 145812 (0.0007) [2023-12-27 06:40:02,707][20347] Updated weights for policy 0, policy_version 146061 (0.0007) [2023-12-27 06:40:02,756][20347] Updated weights for policy 0, policy_version 146071 (0.0005) [2023-12-27 06:40:02,809][20347] Updated weights for policy 0, policy_version 146081 (0.0006) [2023-12-27 06:40:03,337][20381] Updated weights for policy 1, policy_version 145822 (0.0006) [2023-12-27 06:40:03,389][20381] Updated weights for policy 1, policy_version 145832 (0.0005) [2023-12-27 06:40:03,436][20381] Updated weights for policy 1, policy_version 145842 (0.0006) [2023-12-27 06:40:03,496][20347] Updated weights for policy 0, policy_version 146091 (0.0006) [2023-12-27 06:40:03,555][20347] Updated weights for policy 0, policy_version 146101 (0.0008) [2023-12-27 06:40:03,612][20347] Updated weights for policy 0, policy_version 146111 (0.0008) [2023-12-27 06:40:04,056][20381] Updated weights for policy 1, policy_version 145852 (0.0008) [2023-12-27 06:40:04,115][20381] Updated weights for policy 1, policy_version 145862 (0.0007) [2023-12-27 06:40:04,175][20381] Updated weights for policy 1, policy_version 145872 (0.0008) [2023-12-27 06:40:04,207][20347] Updated weights for policy 0, policy_version 146121 (0.0006) [2023-12-27 06:40:04,264][20347] Updated weights for policy 0, policy_version 146131 (0.0008) [2023-12-27 06:40:04,329][20347] Updated weights for policy 0, policy_version 146141 (0.0009) [2023-12-27 06:40:04,394][20347] Updated weights for policy 0, policy_version 146151 (0.0008) [2023-12-27 06:40:04,839][20381] Updated weights for policy 1, policy_version 145882 (0.0009) [2023-12-27 06:40:04,886][20381] Updated weights for policy 1, policy_version 145892 (0.0010) [2023-12-27 06:40:04,936][20381] Updated weights for policy 1, policy_version 145902 (0.0010) [2023-12-27 06:40:04,990][20381] Updated weights for policy 1, policy_version 145912 (0.0008) [2023-12-27 06:40:05,038][20347] Updated weights for policy 0, policy_version 146161 (0.0006) [2023-12-27 06:40:05,088][20347] Updated weights for policy 0, policy_version 146171 (0.0005) [2023-12-27 06:40:05,150][20347] Updated weights for policy 0, policy_version 146181 (0.0006) [2023-12-27 06:40:05,670][20381] Updated weights for policy 1, policy_version 145922 (0.0008) [2023-12-27 06:40:05,719][20381] Updated weights for policy 1, policy_version 145932 (0.0007) [2023-12-27 06:40:05,771][20381] Updated weights for policy 1, policy_version 145942 (0.0008) [2023-12-27 06:40:05,818][20347] Updated weights for policy 0, policy_version 146191 (0.0006) [2023-12-27 06:40:05,870][20347] Updated weights for policy 0, policy_version 146201 (0.0005) [2023-12-27 06:40:05,926][20347] Updated weights for policy 0, policy_version 146211 (0.0005) [2023-12-27 06:40:06,527][20347] Updated weights for policy 0, policy_version 146221 (0.0006) [2023-12-27 06:40:06,529][20381] Updated weights for policy 1, policy_version 145952 (0.0006) [2023-12-27 06:40:06,583][20381] Updated weights for policy 1, policy_version 145962 (0.0006) [2023-12-27 06:40:06,588][20347] Updated weights for policy 0, policy_version 146231 (0.0006) [2023-12-27 06:40:06,645][20381] Updated weights for policy 1, policy_version 145972 (0.0006) [2023-12-27 06:40:06,654][20347] Updated weights for policy 0, policy_version 146241 (0.0007) [2023-12-27 06:40:07,272][20381] Updated weights for policy 1, policy_version 145982 (0.0007) [2023-12-27 06:40:07,279][20347] Updated weights for policy 0, policy_version 146251 (0.0006) [2023-12-27 06:40:07,325][20381] Updated weights for policy 1, policy_version 145992 (0.0005) [2023-12-27 06:40:07,340][20347] Updated weights for policy 0, policy_version 146261 (0.0007) [2023-12-27 06:40:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 74833920. Throughput: 0: 10455.7, 1: 10449.7. Samples: 74807964. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:07,375][20381] Updated weights for policy 1, policy_version 146002 (0.0006) [2023-12-27 06:40:07,403][20347] Updated weights for policy 0, policy_version 146271 (0.0007) [2023-12-27 06:40:07,406][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000146008_37388288.pth... [2023-12-27 06:40:07,410][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000144792_37076992.pth [2023-12-27 06:40:07,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000146280_37462016.pth... [2023-12-27 06:40:07,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000145032_37142528.pth [2023-12-27 06:40:08,012][20381] Updated weights for policy 1, policy_version 146012 (0.0007) [2023-12-27 06:40:08,024][20347] Updated weights for policy 0, policy_version 146281 (0.0008) [2023-12-27 06:40:08,064][20381] Updated weights for policy 1, policy_version 146022 (0.0007) [2023-12-27 06:40:08,080][20347] Updated weights for policy 0, policy_version 146291 (0.0007) [2023-12-27 06:40:08,113][20381] Updated weights for policy 1, policy_version 146032 (0.0007) [2023-12-27 06:40:08,134][20347] Updated weights for policy 0, policy_version 146301 (0.0006) [2023-12-27 06:40:08,186][20347] Updated weights for policy 0, policy_version 146311 (0.0005) [2023-12-27 06:40:08,833][20347] Updated weights for policy 0, policy_version 146321 (0.0010) [2023-12-27 06:40:08,842][20381] Updated weights for policy 1, policy_version 146042 (0.0007) [2023-12-27 06:40:08,895][20347] Updated weights for policy 0, policy_version 146331 (0.0008) [2023-12-27 06:40:08,896][20381] Updated weights for policy 1, policy_version 146052 (0.0006) [2023-12-27 06:40:08,947][20347] Updated weights for policy 0, policy_version 146341 (0.0007) [2023-12-27 06:40:08,949][20381] Updated weights for policy 1, policy_version 146062 (0.0006) [2023-12-27 06:40:09,003][20381] Updated weights for policy 1, policy_version 146072 (0.0007) [2023-12-27 06:40:09,728][20347] Updated weights for policy 0, policy_version 146351 (0.0008) [2023-12-27 06:40:09,750][20381] Updated weights for policy 1, policy_version 146082 (0.0010) [2023-12-27 06:40:09,784][20347] Updated weights for policy 0, policy_version 146361 (0.0008) [2023-12-27 06:40:09,805][20381] Updated weights for policy 1, policy_version 146092 (0.0011) [2023-12-27 06:40:09,843][20347] Updated weights for policy 0, policy_version 146371 (0.0008) [2023-12-27 06:40:09,865][20381] Updated weights for policy 1, policy_version 146102 (0.0011) [2023-12-27 06:40:10,581][20347] Updated weights for policy 0, policy_version 146381 (0.0007) [2023-12-27 06:40:10,590][20381] Updated weights for policy 1, policy_version 146112 (0.0009) [2023-12-27 06:40:10,642][20347] Updated weights for policy 0, policy_version 146391 (0.0006) [2023-12-27 06:40:10,646][20381] Updated weights for policy 1, policy_version 146122 (0.0008) [2023-12-27 06:40:10,705][20347] Updated weights for policy 0, policy_version 146401 (0.0007) [2023-12-27 06:40:10,710][20381] Updated weights for policy 1, policy_version 146132 (0.0008) [2023-12-27 06:40:11,353][20347] Updated weights for policy 0, policy_version 146411 (0.0007) [2023-12-27 06:40:11,412][20381] Updated weights for policy 1, policy_version 146142 (0.0008) [2023-12-27 06:40:11,421][20347] Updated weights for policy 0, policy_version 146421 (0.0008) [2023-12-27 06:40:11,464][20381] Updated weights for policy 1, policy_version 146152 (0.0007) [2023-12-27 06:40:11,480][20347] Updated weights for policy 0, policy_version 146431 (0.0008) [2023-12-27 06:40:11,514][20381] Updated weights for policy 1, policy_version 146162 (0.0007) [2023-12-27 06:40:12,197][20381] Updated weights for policy 1, policy_version 146172 (0.0009) [2023-12-27 06:40:12,211][20347] Updated weights for policy 0, policy_version 146441 (0.0008) [2023-12-27 06:40:12,248][20381] Updated weights for policy 1, policy_version 146182 (0.0009) [2023-12-27 06:40:12,267][20347] Updated weights for policy 0, policy_version 146451 (0.0007) [2023-12-27 06:40:12,308][20381] Updated weights for policy 1, policy_version 146192 (0.0006) [2023-12-27 06:40:12,325][20347] Updated weights for policy 0, policy_version 146461 (0.0008) [2023-12-27 06:40:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 74940416. Throughput: 0: 10417.1, 1: 10489.3. Samples: 74930532. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:12,384][20347] Updated weights for policy 0, policy_version 146471 (0.0008) [2023-12-27 06:40:12,892][20381] Updated weights for policy 1, policy_version 146202 (0.0007) [2023-12-27 06:40:12,955][20381] Updated weights for policy 1, policy_version 146212 (0.0008) [2023-12-27 06:40:13,007][20381] Updated weights for policy 1, policy_version 146222 (0.0007) [2023-12-27 06:40:13,065][20381] Updated weights for policy 1, policy_version 146232 (0.0008) [2023-12-27 06:40:13,118][20347] Updated weights for policy 0, policy_version 146481 (0.0008) [2023-12-27 06:40:13,179][20347] Updated weights for policy 0, policy_version 146491 (0.0008) [2023-12-27 06:40:13,234][20347] Updated weights for policy 0, policy_version 146501 (0.0008) [2023-12-27 06:40:13,682][20381] Updated weights for policy 1, policy_version 146242 (0.0005) [2023-12-27 06:40:13,752][20381] Updated weights for policy 1, policy_version 146252 (0.0005) [2023-12-27 06:40:13,815][20381] Updated weights for policy 1, policy_version 146262 (0.0005) [2023-12-27 06:40:13,895][20347] Updated weights for policy 0, policy_version 146511 (0.0006) [2023-12-27 06:40:13,954][20347] Updated weights for policy 0, policy_version 146521 (0.0006) [2023-12-27 06:40:14,019][20347] Updated weights for policy 0, policy_version 146531 (0.0007) [2023-12-27 06:40:14,355][20381] Updated weights for policy 1, policy_version 146272 (0.0005) [2023-12-27 06:40:14,400][20381] Updated weights for policy 1, policy_version 146282 (0.0007) [2023-12-27 06:40:14,446][20381] Updated weights for policy 1, policy_version 146292 (0.0008) [2023-12-27 06:40:14,674][20347] Updated weights for policy 0, policy_version 146541 (0.0007) [2023-12-27 06:40:14,724][20347] Updated weights for policy 0, policy_version 146551 (0.0007) [2023-12-27 06:40:14,782][20347] Updated weights for policy 0, policy_version 146561 (0.0008) [2023-12-27 06:40:15,186][20381] Updated weights for policy 1, policy_version 146302 (0.0009) [2023-12-27 06:40:15,241][20381] Updated weights for policy 1, policy_version 146312 (0.0010) [2023-12-27 06:40:15,300][20381] Updated weights for policy 1, policy_version 146322 (0.0010) [2023-12-27 06:40:15,500][20347] Updated weights for policy 0, policy_version 146571 (0.0007) [2023-12-27 06:40:15,560][20347] Updated weights for policy 0, policy_version 146581 (0.0007) [2023-12-27 06:40:15,625][20347] Updated weights for policy 0, policy_version 146591 (0.0009) [2023-12-27 06:40:15,917][20381] Updated weights for policy 1, policy_version 146332 (0.0008) [2023-12-27 06:40:15,969][20381] Updated weights for policy 1, policy_version 146342 (0.0005) [2023-12-27 06:40:16,018][20381] Updated weights for policy 1, policy_version 146352 (0.0005) [2023-12-27 06:40:16,347][20347] Updated weights for policy 0, policy_version 146601 (0.0008) [2023-12-27 06:40:16,395][20347] Updated weights for policy 0, policy_version 146611 (0.0008) [2023-12-27 06:40:16,440][20347] Updated weights for policy 0, policy_version 146621 (0.0008) [2023-12-27 06:40:16,492][20347] Updated weights for policy 0, policy_version 146631 (0.0008) [2023-12-27 06:40:16,571][20381] Updated weights for policy 1, policy_version 146362 (0.0006) [2023-12-27 06:40:16,630][20381] Updated weights for policy 1, policy_version 146372 (0.0011) [2023-12-27 06:40:16,681][20381] Updated weights for policy 1, policy_version 146382 (0.0010) [2023-12-27 06:40:16,736][20381] Updated weights for policy 1, policy_version 146392 (0.0010) [2023-12-27 06:40:17,194][20347] Updated weights for policy 0, policy_version 146641 (0.0007) [2023-12-27 06:40:17,238][20347] Updated weights for policy 0, policy_version 146651 (0.0008) [2023-12-27 06:40:17,288][20347] Updated weights for policy 0, policy_version 146661 (0.0006) [2023-12-27 06:40:17,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 75046912. Throughput: 0: 10369.3, 1: 10463.6. Samples: 75055344. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:17,415][20381] Updated weights for policy 1, policy_version 146402 (0.0011) [2023-12-27 06:40:17,474][20381] Updated weights for policy 1, policy_version 146412 (0.0011) [2023-12-27 06:40:17,533][20381] Updated weights for policy 1, policy_version 146422 (0.0011) [2023-12-27 06:40:18,017][20347] Updated weights for policy 0, policy_version 146671 (0.0008) [2023-12-27 06:40:18,066][20347] Updated weights for policy 0, policy_version 146681 (0.0007) [2023-12-27 06:40:18,116][20347] Updated weights for policy 0, policy_version 146691 (0.0007) [2023-12-27 06:40:18,187][20381] Updated weights for policy 1, policy_version 146432 (0.0010) [2023-12-27 06:40:18,236][20381] Updated weights for policy 1, policy_version 146442 (0.0010) [2023-12-27 06:40:18,281][20381] Updated weights for policy 1, policy_version 146452 (0.0010) [2023-12-27 06:40:18,813][20347] Updated weights for policy 0, policy_version 146701 (0.0008) [2023-12-27 06:40:18,865][20347] Updated weights for policy 0, policy_version 146711 (0.0008) [2023-12-27 06:40:18,920][20347] Updated weights for policy 0, policy_version 146721 (0.0008) [2023-12-27 06:40:18,948][20381] Updated weights for policy 1, policy_version 146462 (0.0008) [2023-12-27 06:40:19,004][20381] Updated weights for policy 1, policy_version 146472 (0.0005) [2023-12-27 06:40:19,060][20381] Updated weights for policy 1, policy_version 146482 (0.0005) [2023-12-27 06:40:19,682][20347] Updated weights for policy 0, policy_version 146731 (0.0008) [2023-12-27 06:40:19,702][20381] Updated weights for policy 1, policy_version 146492 (0.0005) [2023-12-27 06:40:19,742][20347] Updated weights for policy 0, policy_version 146741 (0.0007) [2023-12-27 06:40:19,766][20381] Updated weights for policy 1, policy_version 146502 (0.0009) [2023-12-27 06:40:19,802][20347] Updated weights for policy 0, policy_version 146751 (0.0005) [2023-12-27 06:40:19,829][20381] Updated weights for policy 1, policy_version 146512 (0.0009) [2023-12-27 06:40:20,465][20381] Updated weights for policy 1, policy_version 146522 (0.0010) [2023-12-27 06:40:20,513][20347] Updated weights for policy 0, policy_version 146761 (0.0007) [2023-12-27 06:40:20,520][20381] Updated weights for policy 1, policy_version 146532 (0.0007) [2023-12-27 06:40:20,569][20347] Updated weights for policy 0, policy_version 146771 (0.0006) [2023-12-27 06:40:20,579][20381] Updated weights for policy 1, policy_version 146542 (0.0006) [2023-12-27 06:40:20,628][20347] Updated weights for policy 0, policy_version 146781 (0.0007) [2023-12-27 06:40:20,634][20381] Updated weights for policy 1, policy_version 146552 (0.0007) [2023-12-27 06:40:20,687][20347] Updated weights for policy 0, policy_version 146791 (0.0006) [2023-12-27 06:40:21,337][20347] Updated weights for policy 0, policy_version 146801 (0.0007) [2023-12-27 06:40:21,379][20381] Updated weights for policy 1, policy_version 146562 (0.0008) [2023-12-27 06:40:21,403][20347] Updated weights for policy 0, policy_version 146811 (0.0007) [2023-12-27 06:40:21,434][20381] Updated weights for policy 1, policy_version 146572 (0.0006) [2023-12-27 06:40:21,458][20347] Updated weights for policy 0, policy_version 146821 (0.0008) [2023-12-27 06:40:21,482][20381] Updated weights for policy 1, policy_version 146582 (0.0006) [2023-12-27 06:40:22,134][20347] Updated weights for policy 0, policy_version 146831 (0.0005) [2023-12-27 06:40:22,192][20347] Updated weights for policy 0, policy_version 146841 (0.0005) [2023-12-27 06:40:22,230][20381] Updated weights for policy 1, policy_version 146592 (0.0007) [2023-12-27 06:40:22,247][20347] Updated weights for policy 0, policy_version 146851 (0.0006) [2023-12-27 06:40:22,290][20381] Updated weights for policy 1, policy_version 146602 (0.0007) [2023-12-27 06:40:22,342][20381] Updated weights for policy 1, policy_version 146612 (0.0008) [2023-12-27 06:40:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 75145216. Throughput: 0: 10319.9, 1: 10407.0. Samples: 75115404. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000146856_37609472.pth... [2023-12-27 06:40:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000146616_37543936.pth... [2023-12-27 06:40:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000145400_37232640.pth [2023-12-27 06:40:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000145640_37298176.pth [2023-12-27 06:40:22,937][20347] Updated weights for policy 0, policy_version 146861 (0.0007) [2023-12-27 06:40:22,992][20347] Updated weights for policy 0, policy_version 146871 (0.0007) [2023-12-27 06:40:23,016][20381] Updated weights for policy 1, policy_version 146622 (0.0007) [2023-12-27 06:40:23,056][20347] Updated weights for policy 0, policy_version 146881 (0.0006) [2023-12-27 06:40:23,066][20381] Updated weights for policy 1, policy_version 146632 (0.0006) [2023-12-27 06:40:23,112][20381] Updated weights for policy 1, policy_version 146642 (0.0007) [2023-12-27 06:40:23,761][20381] Updated weights for policy 1, policy_version 146652 (0.0008) [2023-12-27 06:40:23,767][20347] Updated weights for policy 0, policy_version 146891 (0.0008) [2023-12-27 06:40:23,816][20381] Updated weights for policy 1, policy_version 146662 (0.0007) [2023-12-27 06:40:23,821][20347] Updated weights for policy 0, policy_version 146901 (0.0006) [2023-12-27 06:40:23,873][20347] Updated weights for policy 0, policy_version 146911 (0.0005) [2023-12-27 06:40:23,873][20381] Updated weights for policy 1, policy_version 146672 (0.0007) [2023-12-27 06:40:24,434][20381] Updated weights for policy 1, policy_version 146682 (0.0008) [2023-12-27 06:40:24,487][20381] Updated weights for policy 1, policy_version 146692 (0.0008) [2023-12-27 06:40:24,533][20381] Updated weights for policy 1, policy_version 146702 (0.0007) [2023-12-27 06:40:24,557][20347] Updated weights for policy 0, policy_version 146921 (0.0006) [2023-12-27 06:40:24,582][20381] Updated weights for policy 1, policy_version 146712 (0.0005) [2023-12-27 06:40:24,615][20347] Updated weights for policy 0, policy_version 146931 (0.0007) [2023-12-27 06:40:24,677][20347] Updated weights for policy 0, policy_version 146941 (0.0009) [2023-12-27 06:40:24,736][20347] Updated weights for policy 0, policy_version 146951 (0.0008) [2023-12-27 06:40:25,253][20381] Updated weights for policy 1, policy_version 146722 (0.0007) [2023-12-27 06:40:25,301][20381] Updated weights for policy 1, policy_version 146732 (0.0008) [2023-12-27 06:40:25,356][20381] Updated weights for policy 1, policy_version 146742 (0.0008) [2023-12-27 06:40:25,415][20347] Updated weights for policy 0, policy_version 146961 (0.0007) [2023-12-27 06:40:25,466][20347] Updated weights for policy 0, policy_version 146971 (0.0007) [2023-12-27 06:40:25,517][20347] Updated weights for policy 0, policy_version 146981 (0.0008) [2023-12-27 06:40:26,003][20381] Updated weights for policy 1, policy_version 146752 (0.0008) [2023-12-27 06:40:26,055][20381] Updated weights for policy 1, policy_version 146762 (0.0008) [2023-12-27 06:40:26,113][20381] Updated weights for policy 1, policy_version 146772 (0.0007) [2023-12-27 06:40:26,234][20347] Updated weights for policy 0, policy_version 146991 (0.0008) [2023-12-27 06:40:26,285][20347] Updated weights for policy 0, policy_version 147001 (0.0007) [2023-12-27 06:40:26,330][20347] Updated weights for policy 0, policy_version 147011 (0.0008) [2023-12-27 06:40:26,767][20381] Updated weights for policy 1, policy_version 146782 (0.0007) [2023-12-27 06:40:26,815][20381] Updated weights for policy 1, policy_version 146792 (0.0008) [2023-12-27 06:40:26,860][20381] Updated weights for policy 1, policy_version 146802 (0.0008) [2023-12-27 06:40:27,041][20347] Updated weights for policy 0, policy_version 147021 (0.0008) [2023-12-27 06:40:27,091][20347] Updated weights for policy 0, policy_version 147031 (0.0008) [2023-12-27 06:40:27,142][20347] Updated weights for policy 0, policy_version 147041 (0.0008) [2023-12-27 06:40:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 75251712. Throughput: 0: 10299.2, 1: 10422.6. Samples: 75239036. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:27,473][20381] Updated weights for policy 1, policy_version 146812 (0.0008) [2023-12-27 06:40:27,519][20381] Updated weights for policy 1, policy_version 146822 (0.0007) [2023-12-27 06:40:27,567][20381] Updated weights for policy 1, policy_version 146832 (0.0007) [2023-12-27 06:40:27,838][20347] Updated weights for policy 0, policy_version 147051 (0.0006) [2023-12-27 06:40:27,887][20347] Updated weights for policy 0, policy_version 147061 (0.0008) [2023-12-27 06:40:27,937][20347] Updated weights for policy 0, policy_version 147071 (0.0006) [2023-12-27 06:40:28,290][20381] Updated weights for policy 1, policy_version 146842 (0.0008) [2023-12-27 06:40:28,350][20381] Updated weights for policy 1, policy_version 146852 (0.0007) [2023-12-27 06:40:28,405][20381] Updated weights for policy 1, policy_version 146862 (0.0008) [2023-12-27 06:40:28,458][20381] Updated weights for policy 1, policy_version 146872 (0.0007) [2023-12-27 06:40:28,574][20347] Updated weights for policy 0, policy_version 147081 (0.0010) [2023-12-27 06:40:28,622][20347] Updated weights for policy 0, policy_version 147091 (0.0010) [2023-12-27 06:40:28,673][20347] Updated weights for policy 0, policy_version 147101 (0.0010) [2023-12-27 06:40:28,740][20347] Updated weights for policy 0, policy_version 147111 (0.0010) [2023-12-27 06:40:29,111][20381] Updated weights for policy 1, policy_version 146882 (0.0007) [2023-12-27 06:40:29,169][20381] Updated weights for policy 1, policy_version 146892 (0.0009) [2023-12-27 06:40:29,216][20381] Updated weights for policy 1, policy_version 146902 (0.0008) [2023-12-27 06:40:29,342][20347] Updated weights for policy 0, policy_version 147121 (0.0007) [2023-12-27 06:40:29,405][20347] Updated weights for policy 0, policy_version 147131 (0.0008) [2023-12-27 06:40:29,472][20347] Updated weights for policy 0, policy_version 147141 (0.0005) [2023-12-27 06:40:29,874][20381] Updated weights for policy 1, policy_version 146912 (0.0008) [2023-12-27 06:40:29,926][20381] Updated weights for policy 1, policy_version 146922 (0.0008) [2023-12-27 06:40:29,981][20381] Updated weights for policy 1, policy_version 146932 (0.0008) [2023-12-27 06:40:30,106][20347] Updated weights for policy 0, policy_version 147151 (0.0007) [2023-12-27 06:40:30,158][20347] Updated weights for policy 0, policy_version 147161 (0.0008) [2023-12-27 06:40:30,214][20347] Updated weights for policy 0, policy_version 147171 (0.0008) [2023-12-27 06:40:30,662][20381] Updated weights for policy 1, policy_version 146942 (0.0007) [2023-12-27 06:40:30,710][20381] Updated weights for policy 1, policy_version 146952 (0.0008) [2023-12-27 06:40:30,758][20381] Updated weights for policy 1, policy_version 146962 (0.0007) [2023-12-27 06:40:30,917][20347] Updated weights for policy 0, policy_version 147181 (0.0007) [2023-12-27 06:40:30,971][20347] Updated weights for policy 0, policy_version 147191 (0.0007) [2023-12-27 06:40:31,027][20347] Updated weights for policy 0, policy_version 147201 (0.0007) [2023-12-27 06:40:31,416][20381] Updated weights for policy 1, policy_version 146972 (0.0007) [2023-12-27 06:40:31,475][20381] Updated weights for policy 1, policy_version 146982 (0.0008) [2023-12-27 06:40:31,535][20381] Updated weights for policy 1, policy_version 146992 (0.0007) [2023-12-27 06:40:31,772][20347] Updated weights for policy 0, policy_version 147211 (0.0007) [2023-12-27 06:40:31,831][20347] Updated weights for policy 0, policy_version 147221 (0.0005) [2023-12-27 06:40:31,885][20347] Updated weights for policy 0, policy_version 147231 (0.0005) [2023-12-27 06:40:32,199][20381] Updated weights for policy 1, policy_version 147002 (0.0008) [2023-12-27 06:40:32,261][20381] Updated weights for policy 1, policy_version 147012 (0.0008) [2023-12-27 06:40:32,314][20381] Updated weights for policy 1, policy_version 147022 (0.0007) [2023-12-27 06:40:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 75350016. Throughput: 0: 10276.1, 1: 10460.5. Samples: 75364820. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:32,377][20381] Updated weights for policy 1, policy_version 147032 (0.0008) [2023-12-27 06:40:32,510][20347] Updated weights for policy 0, policy_version 147241 (0.0006) [2023-12-27 06:40:32,571][20347] Updated weights for policy 0, policy_version 147251 (0.0008) [2023-12-27 06:40:32,633][20347] Updated weights for policy 0, policy_version 147261 (0.0008) [2023-12-27 06:40:32,694][20347] Updated weights for policy 0, policy_version 147271 (0.0008) [2023-12-27 06:40:33,074][20381] Updated weights for policy 1, policy_version 147042 (0.0009) [2023-12-27 06:40:33,128][20381] Updated weights for policy 1, policy_version 147052 (0.0008) [2023-12-27 06:40:33,176][20381] Updated weights for policy 1, policy_version 147062 (0.0007) [2023-12-27 06:40:33,257][20347] Updated weights for policy 0, policy_version 147281 (0.0007) [2023-12-27 06:40:33,323][20347] Updated weights for policy 0, policy_version 147291 (0.0008) [2023-12-27 06:40:33,391][20347] Updated weights for policy 0, policy_version 147301 (0.0008) [2023-12-27 06:40:33,922][20381] Updated weights for policy 1, policy_version 147072 (0.0008) [2023-12-27 06:40:33,937][20347] Updated weights for policy 0, policy_version 147311 (0.0008) [2023-12-27 06:40:33,976][20381] Updated weights for policy 1, policy_version 147082 (0.0006) [2023-12-27 06:40:33,996][20347] Updated weights for policy 0, policy_version 147321 (0.0006) [2023-12-27 06:40:34,031][20381] Updated weights for policy 1, policy_version 147092 (0.0006) [2023-12-27 06:40:34,048][20347] Updated weights for policy 0, policy_version 147331 (0.0005) [2023-12-27 06:40:34,719][20381] Updated weights for policy 1, policy_version 147102 (0.0006) [2023-12-27 06:40:34,772][20381] Updated weights for policy 1, policy_version 147112 (0.0009) [2023-12-27 06:40:34,794][20347] Updated weights for policy 0, policy_version 147341 (0.0006) [2023-12-27 06:40:34,819][20381] Updated weights for policy 1, policy_version 147122 (0.0008) [2023-12-27 06:40:34,851][20347] Updated weights for policy 0, policy_version 147351 (0.0010) [2023-12-27 06:40:34,912][20347] Updated weights for policy 0, policy_version 147361 (0.0010) [2023-12-27 06:40:35,462][20381] Updated weights for policy 1, policy_version 147132 (0.0006) [2023-12-27 06:40:35,516][20381] Updated weights for policy 1, policy_version 147142 (0.0006) [2023-12-27 06:40:35,566][20381] Updated weights for policy 1, policy_version 147152 (0.0007) [2023-12-27 06:40:35,569][20347] Updated weights for policy 0, policy_version 147371 (0.0009) [2023-12-27 06:40:35,632][20347] Updated weights for policy 0, policy_version 147381 (0.0007) [2023-12-27 06:40:35,693][20347] Updated weights for policy 0, policy_version 147391 (0.0010) [2023-12-27 06:40:36,320][20381] Updated weights for policy 1, policy_version 147162 (0.0009) [2023-12-27 06:40:36,352][20347] Updated weights for policy 0, policy_version 147401 (0.0011) [2023-12-27 06:40:36,374][20381] Updated weights for policy 1, policy_version 147172 (0.0008) [2023-12-27 06:40:36,409][20347] Updated weights for policy 0, policy_version 147411 (0.0009) [2023-12-27 06:40:36,431][20381] Updated weights for policy 1, policy_version 147182 (0.0008) [2023-12-27 06:40:36,467][20347] Updated weights for policy 0, policy_version 147421 (0.0005) [2023-12-27 06:40:36,488][20381] Updated weights for policy 1, policy_version 147192 (0.0007) [2023-12-27 06:40:36,530][20347] Updated weights for policy 0, policy_version 147431 (0.0005) [2023-12-27 06:40:37,074][20347] Updated weights for policy 0, policy_version 147441 (0.0005) [2023-12-27 06:40:37,137][20347] Updated weights for policy 0, policy_version 147451 (0.0006) [2023-12-27 06:40:37,143][20381] Updated weights for policy 1, policy_version 147202 (0.0007) [2023-12-27 06:40:37,194][20381] Updated weights for policy 1, policy_version 147212 (0.0006) [2023-12-27 06:40:37,196][20347] Updated weights for policy 0, policy_version 147461 (0.0006) [2023-12-27 06:40:37,254][20381] Updated weights for policy 1, policy_version 147222 (0.0008) [2023-12-27 06:40:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 75464704. Throughput: 0: 10310.1, 1: 10407.9. Samples: 75426648. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 06:40:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000147464_37765120.pth... [2023-12-27 06:40:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000147224_37699584.pth... [2023-12-27 06:40:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000146280_37462016.pth [2023-12-27 06:40:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000146008_37388288.pth [2023-12-27 06:40:37,769][20347] Updated weights for policy 0, policy_version 147471 (0.0005) [2023-12-27 06:40:37,822][20347] Updated weights for policy 0, policy_version 147481 (0.0010) [2023-12-27 06:40:37,878][20347] Updated weights for policy 0, policy_version 147491 (0.0010) [2023-12-27 06:40:37,883][20381] Updated weights for policy 1, policy_version 147232 (0.0006) [2023-12-27 06:40:37,943][20381] Updated weights for policy 1, policy_version 147242 (0.0006) [2023-12-27 06:40:37,995][20381] Updated weights for policy 1, policy_version 147252 (0.0007) [2023-12-27 06:40:38,564][20347] Updated weights for policy 0, policy_version 147501 (0.0010) [2023-12-27 06:40:38,612][20347] Updated weights for policy 0, policy_version 147511 (0.0010) [2023-12-27 06:40:38,629][20381] Updated weights for policy 1, policy_version 147262 (0.0007) [2023-12-27 06:40:38,661][20347] Updated weights for policy 0, policy_version 147521 (0.0010) [2023-12-27 06:40:38,681][20381] Updated weights for policy 1, policy_version 147272 (0.0005) [2023-12-27 06:40:38,725][20381] Updated weights for policy 1, policy_version 147282 (0.0006) [2023-12-27 06:40:39,328][20347] Updated weights for policy 0, policy_version 147531 (0.0009) [2023-12-27 06:40:39,350][20381] Updated weights for policy 1, policy_version 147292 (0.0008) [2023-12-27 06:40:39,391][20347] Updated weights for policy 0, policy_version 147541 (0.0008) [2023-12-27 06:40:39,421][20381] Updated weights for policy 1, policy_version 147302 (0.0008) [2023-12-27 06:40:39,457][20347] Updated weights for policy 0, policy_version 147551 (0.0007) [2023-12-27 06:40:39,485][20381] Updated weights for policy 1, policy_version 147312 (0.0007) [2023-12-27 06:40:40,149][20347] Updated weights for policy 0, policy_version 147561 (0.0006) [2023-12-27 06:40:40,165][20381] Updated weights for policy 1, policy_version 147322 (0.0008) [2023-12-27 06:40:40,205][20347] Updated weights for policy 0, policy_version 147571 (0.0006) [2023-12-27 06:40:40,222][20381] Updated weights for policy 1, policy_version 147332 (0.0006) [2023-12-27 06:40:40,265][20347] Updated weights for policy 0, policy_version 147581 (0.0006) [2023-12-27 06:40:40,286][20381] Updated weights for policy 1, policy_version 147342 (0.0006) [2023-12-27 06:40:40,324][20347] Updated weights for policy 0, policy_version 147591 (0.0007) [2023-12-27 06:40:40,350][20381] Updated weights for policy 1, policy_version 147352 (0.0007) [2023-12-27 06:40:40,999][20381] Updated weights for policy 1, policy_version 147362 (0.0007) [2023-12-27 06:40:41,042][20347] Updated weights for policy 0, policy_version 147601 (0.0007) [2023-12-27 06:40:41,061][20381] Updated weights for policy 1, policy_version 147372 (0.0008) [2023-12-27 06:40:41,099][20347] Updated weights for policy 0, policy_version 147611 (0.0007) [2023-12-27 06:40:41,112][20381] Updated weights for policy 1, policy_version 147382 (0.0008) [2023-12-27 06:40:41,165][20347] Updated weights for policy 0, policy_version 147621 (0.0008) [2023-12-27 06:40:41,809][20381] Updated weights for policy 1, policy_version 147392 (0.0006) [2023-12-27 06:40:41,872][20381] Updated weights for policy 1, policy_version 147402 (0.0007) [2023-12-27 06:40:41,895][20347] Updated weights for policy 0, policy_version 147631 (0.0007) [2023-12-27 06:40:41,935][20381] Updated weights for policy 1, policy_version 147412 (0.0006) [2023-12-27 06:40:41,952][20347] Updated weights for policy 0, policy_version 147641 (0.0006) [2023-12-27 06:40:42,007][20347] Updated weights for policy 0, policy_version 147651 (0.0007) [2023-12-27 06:40:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 75563008. Throughput: 0: 10359.4, 1: 10406.4. Samples: 75552620. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:42,587][20381] Updated weights for policy 1, policy_version 147422 (0.0006) [2023-12-27 06:40:42,647][20381] Updated weights for policy 1, policy_version 147432 (0.0008) [2023-12-27 06:40:42,705][20381] Updated weights for policy 1, policy_version 147442 (0.0007) [2023-12-27 06:40:42,714][20347] Updated weights for policy 0, policy_version 147661 (0.0008) [2023-12-27 06:40:42,775][20347] Updated weights for policy 0, policy_version 147671 (0.0009) [2023-12-27 06:40:42,832][20347] Updated weights for policy 0, policy_version 147681 (0.0008) [2023-12-27 06:40:43,306][20381] Updated weights for policy 1, policy_version 147452 (0.0007) [2023-12-27 06:40:43,353][20381] Updated weights for policy 1, policy_version 147462 (0.0007) [2023-12-27 06:40:43,409][20381] Updated weights for policy 1, policy_version 147472 (0.0008) [2023-12-27 06:40:43,531][20347] Updated weights for policy 0, policy_version 147691 (0.0007) [2023-12-27 06:40:43,587][20347] Updated weights for policy 0, policy_version 147701 (0.0007) [2023-12-27 06:40:43,641][20347] Updated weights for policy 0, policy_version 147711 (0.0007) [2023-12-27 06:40:44,073][20381] Updated weights for policy 1, policy_version 147482 (0.0008) [2023-12-27 06:40:44,125][20381] Updated weights for policy 1, policy_version 147492 (0.0007) [2023-12-27 06:40:44,190][20381] Updated weights for policy 1, policy_version 147502 (0.0008) [2023-12-27 06:40:44,253][20381] Updated weights for policy 1, policy_version 147512 (0.0008) [2023-12-27 06:40:44,334][20347] Updated weights for policy 0, policy_version 147721 (0.0007) [2023-12-27 06:40:44,393][20347] Updated weights for policy 0, policy_version 147731 (0.0008) [2023-12-27 06:40:44,441][20347] Updated weights for policy 0, policy_version 147741 (0.0007) [2023-12-27 06:40:44,490][20347] Updated weights for policy 0, policy_version 147751 (0.0008) [2023-12-27 06:40:44,908][20381] Updated weights for policy 1, policy_version 147522 (0.0005) [2023-12-27 06:40:44,973][20381] Updated weights for policy 1, policy_version 147532 (0.0007) [2023-12-27 06:40:45,032][20381] Updated weights for policy 1, policy_version 147542 (0.0009) [2023-12-27 06:40:45,174][20347] Updated weights for policy 0, policy_version 147761 (0.0006) [2023-12-27 06:40:45,236][20347] Updated weights for policy 0, policy_version 147771 (0.0005) [2023-12-27 06:40:45,299][20347] Updated weights for policy 0, policy_version 147781 (0.0006) [2023-12-27 06:40:45,721][20381] Updated weights for policy 1, policy_version 147552 (0.0007) [2023-12-27 06:40:45,774][20381] Updated weights for policy 1, policy_version 147562 (0.0007) [2023-12-27 06:40:45,830][20381] Updated weights for policy 1, policy_version 147572 (0.0008) [2023-12-27 06:40:45,920][20347] Updated weights for policy 0, policy_version 147791 (0.0007) [2023-12-27 06:40:45,975][20347] Updated weights for policy 0, policy_version 147801 (0.0007) [2023-12-27 06:40:46,024][20347] Updated weights for policy 0, policy_version 147811 (0.0008) [2023-12-27 06:40:46,473][20381] Updated weights for policy 1, policy_version 147582 (0.0006) [2023-12-27 06:40:46,539][20381] Updated weights for policy 1, policy_version 147592 (0.0005) [2023-12-27 06:40:46,594][20381] Updated weights for policy 1, policy_version 147602 (0.0005) [2023-12-27 06:40:46,731][20347] Updated weights for policy 0, policy_version 147821 (0.0009) [2023-12-27 06:40:46,788][20347] Updated weights for policy 0, policy_version 147831 (0.0007) [2023-12-27 06:40:46,840][20347] Updated weights for policy 0, policy_version 147841 (0.0008) [2023-12-27 06:40:47,141][20381] Updated weights for policy 1, policy_version 147612 (0.0005) [2023-12-27 06:40:47,202][20381] Updated weights for policy 1, policy_version 147622 (0.0005) [2023-12-27 06:40:47,258][20381] Updated weights for policy 1, policy_version 147632 (0.0005) [2023-12-27 06:40:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 75669504. Throughput: 0: 10245.0, 1: 10481.4. Samples: 75676704. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:40:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:47,602][20347] Updated weights for policy 0, policy_version 147851 (0.0009) [2023-12-27 06:40:47,663][20347] Updated weights for policy 0, policy_version 147861 (0.0008) [2023-12-27 06:40:47,719][20347] Updated weights for policy 0, policy_version 147871 (0.0006) [2023-12-27 06:40:47,791][20381] Updated weights for policy 1, policy_version 147642 (0.0007) [2023-12-27 06:40:47,847][20381] Updated weights for policy 1, policy_version 147652 (0.0011) [2023-12-27 06:40:47,900][20381] Updated weights for policy 1, policy_version 147662 (0.0007) [2023-12-27 06:40:47,953][20381] Updated weights for policy 1, policy_version 147672 (0.0009) [2023-12-27 06:40:48,294][20347] Updated weights for policy 0, policy_version 147881 (0.0005) [2023-12-27 06:40:48,352][20347] Updated weights for policy 0, policy_version 147891 (0.0006) [2023-12-27 06:40:48,412][20347] Updated weights for policy 0, policy_version 147901 (0.0006) [2023-12-27 06:40:48,470][20347] Updated weights for policy 0, policy_version 147911 (0.0006) [2023-12-27 06:40:48,574][20381] Updated weights for policy 1, policy_version 147682 (0.0011) [2023-12-27 06:40:48,631][20381] Updated weights for policy 1, policy_version 147692 (0.0010) [2023-12-27 06:40:48,685][20381] Updated weights for policy 1, policy_version 147702 (0.0007) [2023-12-27 06:40:49,087][20347] Updated weights for policy 0, policy_version 147921 (0.0009) [2023-12-27 06:40:49,135][20347] Updated weights for policy 0, policy_version 147931 (0.0010) [2023-12-27 06:40:49,180][20347] Updated weights for policy 0, policy_version 147941 (0.0010) [2023-12-27 06:40:49,270][20381] Updated weights for policy 1, policy_version 147712 (0.0007) [2023-12-27 06:40:49,325][20381] Updated weights for policy 1, policy_version 147722 (0.0006) [2023-12-27 06:40:49,385][20381] Updated weights for policy 1, policy_version 147732 (0.0010) [2023-12-27 06:40:49,852][20347] Updated weights for policy 0, policy_version 147951 (0.0009) [2023-12-27 06:40:49,917][20347] Updated weights for policy 0, policy_version 147961 (0.0007) [2023-12-27 06:40:49,983][20347] Updated weights for policy 0, policy_version 147971 (0.0007) [2023-12-27 06:40:50,008][20381] Updated weights for policy 1, policy_version 147742 (0.0006) [2023-12-27 06:40:50,066][20381] Updated weights for policy 1, policy_version 147752 (0.0011) [2023-12-27 06:40:50,126][20381] Updated weights for policy 1, policy_version 147762 (0.0007) [2023-12-27 06:40:50,678][20347] Updated weights for policy 0, policy_version 147981 (0.0008) [2023-12-27 06:40:50,749][20347] Updated weights for policy 0, policy_version 147991 (0.0008) [2023-12-27 06:40:50,773][20381] Updated weights for policy 1, policy_version 147772 (0.0006) [2023-12-27 06:40:50,809][20347] Updated weights for policy 0, policy_version 148001 (0.0009) [2023-12-27 06:40:50,832][20381] Updated weights for policy 1, policy_version 147782 (0.0010) [2023-12-27 06:40:50,892][20381] Updated weights for policy 1, policy_version 147792 (0.0011) [2023-12-27 06:40:51,530][20347] Updated weights for policy 0, policy_version 148011 (0.0006) [2023-12-27 06:40:51,584][20347] Updated weights for policy 0, policy_version 148021 (0.0006) [2023-12-27 06:40:51,621][20381] Updated weights for policy 1, policy_version 147802 (0.0011) [2023-12-27 06:40:51,638][20347] Updated weights for policy 0, policy_version 148031 (0.0007) [2023-12-27 06:40:51,673][20381] Updated weights for policy 1, policy_version 147812 (0.0007) [2023-12-27 06:40:51,732][20381] Updated weights for policy 1, policy_version 147822 (0.0008) [2023-12-27 06:40:51,796][20381] Updated weights for policy 1, policy_version 147832 (0.0008) [2023-12-27 06:40:52,287][20347] Updated weights for policy 0, policy_version 148041 (0.0007) [2023-12-27 06:40:52,345][20347] Updated weights for policy 0, policy_version 148051 (0.0005) [2023-12-27 06:40:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 75767808. Throughput: 0: 10224.0, 1: 10548.3. Samples: 75742720. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:40:52,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000147832_37855232.pth... [2023-12-27 06:40:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000146616_37543936.pth [2023-12-27 06:40:52,408][20347] Updated weights for policy 0, policy_version 148061 (0.0007) [2023-12-27 06:40:52,469][20347] Updated weights for policy 0, policy_version 148071 (0.0007) [2023-12-27 06:40:52,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000148072_37920768.pth... [2023-12-27 06:40:52,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000146856_37609472.pth [2023-12-27 06:40:52,518][20381] Updated weights for policy 1, policy_version 147842 (0.0007) [2023-12-27 06:40:52,574][20381] Updated weights for policy 1, policy_version 147852 (0.0008) [2023-12-27 06:40:52,637][20381] Updated weights for policy 1, policy_version 147862 (0.0007) [2023-12-27 06:40:53,109][20347] Updated weights for policy 0, policy_version 148081 (0.0006) [2023-12-27 06:40:53,167][20347] Updated weights for policy 0, policy_version 148091 (0.0010) [2023-12-27 06:40:53,218][20347] Updated weights for policy 0, policy_version 148101 (0.0010) [2023-12-27 06:40:53,318][20381] Updated weights for policy 1, policy_version 147872 (0.0007) [2023-12-27 06:40:53,375][20381] Updated weights for policy 1, policy_version 147882 (0.0007) [2023-12-27 06:40:53,431][20381] Updated weights for policy 1, policy_version 147892 (0.0007) [2023-12-27 06:40:53,896][20347] Updated weights for policy 0, policy_version 148111 (0.0010) [2023-12-27 06:40:53,945][20347] Updated weights for policy 0, policy_version 148121 (0.0010) [2023-12-27 06:40:54,001][20347] Updated weights for policy 0, policy_version 148131 (0.0010) [2023-12-27 06:40:54,038][20381] Updated weights for policy 1, policy_version 147902 (0.0006) [2023-12-27 06:40:54,088][20381] Updated weights for policy 1, policy_version 147912 (0.0007) [2023-12-27 06:40:54,137][20381] Updated weights for policy 1, policy_version 147922 (0.0007) [2023-12-27 06:40:54,649][20347] Updated weights for policy 0, policy_version 148141 (0.0008) [2023-12-27 06:40:54,698][20347] Updated weights for policy 0, policy_version 148151 (0.0005) [2023-12-27 06:40:54,748][20347] Updated weights for policy 0, policy_version 148161 (0.0005) [2023-12-27 06:40:54,764][20381] Updated weights for policy 1, policy_version 147932 (0.0007) [2023-12-27 06:40:54,829][20381] Updated weights for policy 1, policy_version 147942 (0.0005) [2023-12-27 06:40:54,899][20381] Updated weights for policy 1, policy_version 147952 (0.0005) [2023-12-27 06:40:55,351][20347] Updated weights for policy 0, policy_version 148171 (0.0007) [2023-12-27 06:40:55,414][20347] Updated weights for policy 0, policy_version 148181 (0.0005) [2023-12-27 06:40:55,422][20381] Updated weights for policy 1, policy_version 147962 (0.0006) [2023-12-27 06:40:55,467][20347] Updated weights for policy 0, policy_version 148191 (0.0009) [2023-12-27 06:40:55,475][20381] Updated weights for policy 1, policy_version 147972 (0.0007) [2023-12-27 06:40:55,535][20381] Updated weights for policy 1, policy_version 147982 (0.0007) [2023-12-27 06:40:55,592][20381] Updated weights for policy 1, policy_version 147992 (0.0007) [2023-12-27 06:40:56,043][20347] Updated weights for policy 0, policy_version 148201 (0.0010) [2023-12-27 06:40:56,108][20347] Updated weights for policy 0, policy_version 148211 (0.0010) [2023-12-27 06:40:56,168][20347] Updated weights for policy 0, policy_version 148221 (0.0010) [2023-12-27 06:40:56,224][20347] Updated weights for policy 0, policy_version 148231 (0.0010) [2023-12-27 06:40:56,258][20381] Updated weights for policy 1, policy_version 148002 (0.0007) [2023-12-27 06:40:56,309][20381] Updated weights for policy 1, policy_version 148012 (0.0007) [2023-12-27 06:40:56,369][20381] Updated weights for policy 1, policy_version 148022 (0.0008) [2023-12-27 06:40:56,881][20347] Updated weights for policy 0, policy_version 148241 (0.0006) [2023-12-27 06:40:56,937][20347] Updated weights for policy 0, policy_version 148251 (0.0005) [2023-12-27 06:40:56,993][20347] Updated weights for policy 0, policy_version 148261 (0.0007) [2023-12-27 06:40:57,073][20381] Updated weights for policy 1, policy_version 148032 (0.0009) [2023-12-27 06:40:57,135][20381] Updated weights for policy 1, policy_version 148042 (0.0008) [2023-12-27 06:40:57,188][20381] Updated weights for policy 1, policy_version 148052 (0.0007) [2023-12-27 06:40:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 75882496. Throughput: 0: 10262.5, 1: 10592.5. Samples: 75869008. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:40:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:40:57,614][20347] Updated weights for policy 0, policy_version 148271 (0.0007) [2023-12-27 06:40:57,674][20347] Updated weights for policy 0, policy_version 148281 (0.0005) [2023-12-27 06:40:57,735][20347] Updated weights for policy 0, policy_version 148291 (0.0006) [2023-12-27 06:40:57,798][20381] Updated weights for policy 1, policy_version 148062 (0.0006) [2023-12-27 06:40:57,854][20381] Updated weights for policy 1, policy_version 148072 (0.0005) [2023-12-27 06:40:57,902][20381] Updated weights for policy 1, policy_version 148082 (0.0005) [2023-12-27 06:40:58,338][20347] Updated weights for policy 0, policy_version 148301 (0.0007) [2023-12-27 06:40:58,393][20347] Updated weights for policy 0, policy_version 148311 (0.0008) [2023-12-27 06:40:58,452][20347] Updated weights for policy 0, policy_version 148321 (0.0008) [2023-12-27 06:40:58,560][20381] Updated weights for policy 1, policy_version 148092 (0.0006) [2023-12-27 06:40:58,623][20381] Updated weights for policy 1, policy_version 148103 (0.0007) [2023-12-27 06:40:58,680][20381] Updated weights for policy 1, policy_version 148113 (0.0011) [2023-12-27 06:40:59,206][20347] Updated weights for policy 0, policy_version 148331 (0.0008) [2023-12-27 06:40:59,266][20347] Updated weights for policy 0, policy_version 148341 (0.0008) [2023-12-27 06:40:59,326][20347] Updated weights for policy 0, policy_version 148351 (0.0008) [2023-12-27 06:40:59,471][20381] Updated weights for policy 1, policy_version 148123 (0.0009) [2023-12-27 06:40:59,526][20381] Updated weights for policy 1, policy_version 148133 (0.0006) [2023-12-27 06:40:59,579][20381] Updated weights for policy 1, policy_version 148143 (0.0005) [2023-12-27 06:41:00,019][20347] Updated weights for policy 0, policy_version 148361 (0.0007) [2023-12-27 06:41:00,079][20347] Updated weights for policy 0, policy_version 148371 (0.0008) [2023-12-27 06:41:00,127][20347] Updated weights for policy 0, policy_version 148381 (0.0008) [2023-12-27 06:41:00,181][20347] Updated weights for policy 0, policy_version 148391 (0.0006) [2023-12-27 06:41:00,226][20381] Updated weights for policy 1, policy_version 148153 (0.0006) [2023-12-27 06:41:00,281][20381] Updated weights for policy 1, policy_version 148163 (0.0008) [2023-12-27 06:41:00,331][20381] Updated weights for policy 1, policy_version 148173 (0.0008) [2023-12-27 06:41:00,394][20381] Updated weights for policy 1, policy_version 148183 (0.0006) [2023-12-27 06:41:00,841][20347] Updated weights for policy 0, policy_version 148401 (0.0008) [2023-12-27 06:41:00,893][20347] Updated weights for policy 0, policy_version 148411 (0.0007) [2023-12-27 06:41:00,951][20347] Updated weights for policy 0, policy_version 148421 (0.0009) [2023-12-27 06:41:01,055][20381] Updated weights for policy 1, policy_version 148193 (0.0007) [2023-12-27 06:41:01,112][20381] Updated weights for policy 1, policy_version 148203 (0.0007) [2023-12-27 06:41:01,172][20381] Updated weights for policy 1, policy_version 148213 (0.0009) [2023-12-27 06:41:01,660][20347] Updated weights for policy 0, policy_version 148431 (0.0009) [2023-12-27 06:41:01,718][20347] Updated weights for policy 0, policy_version 148441 (0.0008) [2023-12-27 06:41:01,777][20347] Updated weights for policy 0, policy_version 148451 (0.0008) [2023-12-27 06:41:01,847][20381] Updated weights for policy 1, policy_version 148223 (0.0008) [2023-12-27 06:41:01,898][20381] Updated weights for policy 1, policy_version 148233 (0.0007) [2023-12-27 06:41:01,945][20381] Updated weights for policy 1, policy_version 148243 (0.0009) [2023-12-27 06:41:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 75980800. Throughput: 0: 10313.0, 1: 10496.8. Samples: 75991780. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:02,443][20347] Updated weights for policy 0, policy_version 148461 (0.0009) [2023-12-27 06:41:02,498][20347] Updated weights for policy 0, policy_version 148471 (0.0008) [2023-12-27 06:41:02,546][20347] Updated weights for policy 0, policy_version 148481 (0.0010) [2023-12-27 06:41:02,612][20381] Updated weights for policy 1, policy_version 148253 (0.0009) [2023-12-27 06:41:02,664][20381] Updated weights for policy 1, policy_version 148263 (0.0011) [2023-12-27 06:41:02,726][20381] Updated weights for policy 1, policy_version 148273 (0.0007) [2023-12-27 06:41:03,232][20347] Updated weights for policy 0, policy_version 148491 (0.0010) [2023-12-27 06:41:03,289][20347] Updated weights for policy 0, policy_version 148501 (0.0010) [2023-12-27 06:41:03,344][20347] Updated weights for policy 0, policy_version 148511 (0.0009) [2023-12-27 06:41:03,427][20381] Updated weights for policy 1, policy_version 148283 (0.0008) [2023-12-27 06:41:03,475][20381] Updated weights for policy 1, policy_version 148293 (0.0009) [2023-12-27 06:41:03,528][20381] Updated weights for policy 1, policy_version 148303 (0.0005) [2023-12-27 06:41:04,010][20347] Updated weights for policy 0, policy_version 148521 (0.0008) [2023-12-27 06:41:04,072][20347] Updated weights for policy 0, policy_version 148531 (0.0010) [2023-12-27 06:41:04,128][20347] Updated weights for policy 0, policy_version 148541 (0.0010) [2023-12-27 06:41:04,186][20347] Updated weights for policy 0, policy_version 148551 (0.0010) [2023-12-27 06:41:04,214][20381] Updated weights for policy 1, policy_version 148313 (0.0009) [2023-12-27 06:41:04,270][20381] Updated weights for policy 1, policy_version 148323 (0.0007) [2023-12-27 06:41:04,323][20381] Updated weights for policy 1, policy_version 148333 (0.0008) [2023-12-27 06:41:04,375][20381] Updated weights for policy 1, policy_version 148343 (0.0007) [2023-12-27 06:41:04,866][20347] Updated weights for policy 0, policy_version 148561 (0.0010) [2023-12-27 06:41:04,914][20347] Updated weights for policy 0, policy_version 148571 (0.0010) [2023-12-27 06:41:04,966][20347] Updated weights for policy 0, policy_version 148581 (0.0010) [2023-12-27 06:41:05,033][20381] Updated weights for policy 1, policy_version 148353 (0.0006) [2023-12-27 06:41:05,093][20381] Updated weights for policy 1, policy_version 148363 (0.0006) [2023-12-27 06:41:05,155][20381] Updated weights for policy 1, policy_version 148373 (0.0007) [2023-12-27 06:41:05,646][20347] Updated weights for policy 0, policy_version 148591 (0.0010) [2023-12-27 06:41:05,696][20347] Updated weights for policy 0, policy_version 148601 (0.0010) [2023-12-27 06:41:05,747][20347] Updated weights for policy 0, policy_version 148611 (0.0007) [2023-12-27 06:41:05,809][20381] Updated weights for policy 1, policy_version 148383 (0.0008) [2023-12-27 06:41:05,856][20381] Updated weights for policy 1, policy_version 148393 (0.0005) [2023-12-27 06:41:05,914][20381] Updated weights for policy 1, policy_version 148403 (0.0005) [2023-12-27 06:41:06,462][20347] Updated weights for policy 0, policy_version 148621 (0.0008) [2023-12-27 06:41:06,521][20347] Updated weights for policy 0, policy_version 148631 (0.0009) [2023-12-27 06:41:06,522][20381] Updated weights for policy 1, policy_version 148413 (0.0007) [2023-12-27 06:41:06,579][20347] Updated weights for policy 0, policy_version 148641 (0.0008) [2023-12-27 06:41:06,585][20381] Updated weights for policy 1, policy_version 148423 (0.0009) [2023-12-27 06:41:06,646][20381] Updated weights for policy 1, policy_version 148433 (0.0010) [2023-12-27 06:41:07,240][20347] Updated weights for policy 0, policy_version 148651 (0.0008) [2023-12-27 06:41:07,301][20347] Updated weights for policy 0, policy_version 148661 (0.0010) [2023-12-27 06:41:07,349][20381] Updated weights for policy 1, policy_version 148443 (0.0009) [2023-12-27 06:41:07,359][20347] Updated weights for policy 0, policy_version 148671 (0.0010) [2023-12-27 06:41:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 76079104. Throughput: 0: 10350.6, 1: 10499.5. Samples: 76053656. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:07,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000148680_38076416.pth... [2023-12-27 06:41:07,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000147464_37765120.pth [2023-12-27 06:41:07,410][20381] Updated weights for policy 1, policy_version 148453 (0.0005) [2023-12-27 06:41:07,468][20381] Updated weights for policy 1, policy_version 148463 (0.0007) [2023-12-27 06:41:07,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000148472_38019072.pth... [2023-12-27 06:41:07,526][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000147224_37699584.pth [2023-12-27 06:41:07,940][20347] Updated weights for policy 0, policy_version 148681 (0.0010) [2023-12-27 06:41:07,994][20347] Updated weights for policy 0, policy_version 148691 (0.0006) [2023-12-27 06:41:08,046][20347] Updated weights for policy 0, policy_version 148701 (0.0007) [2023-12-27 06:41:08,093][20381] Updated weights for policy 1, policy_version 148473 (0.0009) [2023-12-27 06:41:08,106][20347] Updated weights for policy 0, policy_version 148711 (0.0005) [2023-12-27 06:41:08,144][20381] Updated weights for policy 1, policy_version 148483 (0.0010) [2023-12-27 06:41:08,195][20381] Updated weights for policy 1, policy_version 148493 (0.0010) [2023-12-27 06:41:08,251][20381] Updated weights for policy 1, policy_version 148503 (0.0010) [2023-12-27 06:41:08,726][20347] Updated weights for policy 0, policy_version 148721 (0.0009) [2023-12-27 06:41:08,788][20347] Updated weights for policy 0, policy_version 148731 (0.0010) [2023-12-27 06:41:08,849][20347] Updated weights for policy 0, policy_version 148741 (0.0010) [2023-12-27 06:41:08,913][20381] Updated weights for policy 1, policy_version 148513 (0.0009) [2023-12-27 06:41:08,969][20381] Updated weights for policy 1, policy_version 148523 (0.0007) [2023-12-27 06:41:09,018][20381] Updated weights for policy 1, policy_version 148533 (0.0010) [2023-12-27 06:41:09,550][20347] Updated weights for policy 0, policy_version 148751 (0.0010) [2023-12-27 06:41:09,616][20347] Updated weights for policy 0, policy_version 148761 (0.0009) [2023-12-27 06:41:09,672][20381] Updated weights for policy 1, policy_version 148543 (0.0008) [2023-12-27 06:41:09,676][20347] Updated weights for policy 0, policy_version 148771 (0.0006) [2023-12-27 06:41:09,736][20381] Updated weights for policy 1, policy_version 148553 (0.0009) [2023-12-27 06:41:09,793][20381] Updated weights for policy 1, policy_version 148563 (0.0009) [2023-12-27 06:41:10,378][20347] Updated weights for policy 0, policy_version 148781 (0.0007) [2023-12-27 06:41:10,437][20347] Updated weights for policy 0, policy_version 148791 (0.0008) [2023-12-27 06:41:10,483][20381] Updated weights for policy 1, policy_version 148573 (0.0011) [2023-12-27 06:41:10,494][20347] Updated weights for policy 0, policy_version 148801 (0.0009) [2023-12-27 06:41:10,535][20381] Updated weights for policy 1, policy_version 148583 (0.0010) [2023-12-27 06:41:10,589][20381] Updated weights for policy 1, policy_version 148593 (0.0010) [2023-12-27 06:41:11,211][20347] Updated weights for policy 0, policy_version 148811 (0.0010) [2023-12-27 06:41:11,273][20347] Updated weights for policy 0, policy_version 148821 (0.0011) [2023-12-27 06:41:11,305][20381] Updated weights for policy 1, policy_version 148603 (0.0009) [2023-12-27 06:41:11,330][20347] Updated weights for policy 0, policy_version 148831 (0.0009) [2023-12-27 06:41:11,374][20381] Updated weights for policy 1, policy_version 148613 (0.0008) [2023-12-27 06:41:11,433][20381] Updated weights for policy 1, policy_version 148623 (0.0006) [2023-12-27 06:41:12,032][20347] Updated weights for policy 0, policy_version 148841 (0.0009) [2023-12-27 06:41:12,087][20347] Updated weights for policy 0, policy_version 148851 (0.0007) [2023-12-27 06:41:12,121][20381] Updated weights for policy 1, policy_version 148633 (0.0006) [2023-12-27 06:41:12,144][20347] Updated weights for policy 0, policy_version 148861 (0.0008) [2023-12-27 06:41:12,172][20381] Updated weights for policy 1, policy_version 148643 (0.0011) [2023-12-27 06:41:12,197][20347] Updated weights for policy 0, policy_version 148871 (0.0007) [2023-12-27 06:41:12,230][20381] Updated weights for policy 1, policy_version 148653 (0.0011) [2023-12-27 06:41:12,296][20381] Updated weights for policy 1, policy_version 148663 (0.0011) [2023-12-27 06:41:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 76193792. Throughput: 0: 10379.0, 1: 10496.6. Samples: 76178432. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:12,921][20347] Updated weights for policy 0, policy_version 148881 (0.0007) [2023-12-27 06:41:12,972][20347] Updated weights for policy 0, policy_version 148891 (0.0007) [2023-12-27 06:41:13,022][20381] Updated weights for policy 1, policy_version 148673 (0.0010) [2023-12-27 06:41:13,024][20347] Updated weights for policy 0, policy_version 148901 (0.0007) [2023-12-27 06:41:13,072][20381] Updated weights for policy 1, policy_version 148683 (0.0010) [2023-12-27 06:41:13,129][20381] Updated weights for policy 1, policy_version 148693 (0.0011) [2023-12-27 06:41:13,683][20347] Updated weights for policy 0, policy_version 148911 (0.0006) [2023-12-27 06:41:13,735][20347] Updated weights for policy 0, policy_version 148921 (0.0006) [2023-12-27 06:41:13,789][20347] Updated weights for policy 0, policy_version 148931 (0.0005) [2023-12-27 06:41:13,841][20381] Updated weights for policy 1, policy_version 148703 (0.0011) [2023-12-27 06:41:13,894][20381] Updated weights for policy 1, policy_version 148713 (0.0010) [2023-12-27 06:41:13,947][20381] Updated weights for policy 1, policy_version 148723 (0.0010) [2023-12-27 06:41:14,356][20347] Updated weights for policy 0, policy_version 148941 (0.0005) [2023-12-27 06:41:14,406][20347] Updated weights for policy 0, policy_version 148951 (0.0006) [2023-12-27 06:41:14,466][20347] Updated weights for policy 0, policy_version 148961 (0.0007) [2023-12-27 06:41:14,649][20381] Updated weights for policy 1, policy_version 148733 (0.0009) [2023-12-27 06:41:14,697][20381] Updated weights for policy 1, policy_version 148743 (0.0007) [2023-12-27 06:41:14,756][20381] Updated weights for policy 1, policy_version 148753 (0.0008) [2023-12-27 06:41:15,122][20347] Updated weights for policy 0, policy_version 148971 (0.0010) [2023-12-27 06:41:15,181][20347] Updated weights for policy 0, policy_version 148981 (0.0010) [2023-12-27 06:41:15,244][20347] Updated weights for policy 0, policy_version 148991 (0.0010) [2023-12-27 06:41:15,475][20381] Updated weights for policy 1, policy_version 148763 (0.0008) [2023-12-27 06:41:15,525][20381] Updated weights for policy 1, policy_version 148773 (0.0011) [2023-12-27 06:41:15,579][20381] Updated weights for policy 1, policy_version 148783 (0.0010) [2023-12-27 06:41:15,876][20347] Updated weights for policy 0, policy_version 149001 (0.0010) [2023-12-27 06:41:15,924][20347] Updated weights for policy 0, policy_version 149011 (0.0006) [2023-12-27 06:41:15,976][20347] Updated weights for policy 0, policy_version 149021 (0.0010) [2023-12-27 06:41:16,036][20347] Updated weights for policy 0, policy_version 149031 (0.0005) [2023-12-27 06:41:16,240][20381] Updated weights for policy 1, policy_version 148793 (0.0010) [2023-12-27 06:41:16,291][20381] Updated weights for policy 1, policy_version 148803 (0.0005) [2023-12-27 06:41:16,342][20381] Updated weights for policy 1, policy_version 148813 (0.0005) [2023-12-27 06:41:16,392][20381] Updated weights for policy 1, policy_version 148823 (0.0005) [2023-12-27 06:41:16,573][20347] Updated weights for policy 0, policy_version 149041 (0.0007) [2023-12-27 06:41:16,623][20347] Updated weights for policy 0, policy_version 149051 (0.0007) [2023-12-27 06:41:16,680][20347] Updated weights for policy 0, policy_version 149061 (0.0007) [2023-12-27 06:41:17,014][20381] Updated weights for policy 1, policy_version 148833 (0.0010) [2023-12-27 06:41:17,063][20381] Updated weights for policy 1, policy_version 148843 (0.0010) [2023-12-27 06:41:17,111][20381] Updated weights for policy 1, policy_version 148853 (0.0010) [2023-12-27 06:41:17,322][20347] Updated weights for policy 0, policy_version 149071 (0.0009) [2023-12-27 06:41:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 76292096. Throughput: 0: 10428.6, 1: 10427.3. Samples: 76303336. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:17,383][20347] Updated weights for policy 0, policy_version 149081 (0.0010) [2023-12-27 06:41:17,437][20347] Updated weights for policy 0, policy_version 149091 (0.0010) [2023-12-27 06:41:17,838][20381] Updated weights for policy 1, policy_version 148863 (0.0008) [2023-12-27 06:41:17,890][20381] Updated weights for policy 1, policy_version 148873 (0.0007) [2023-12-27 06:41:17,942][20381] Updated weights for policy 1, policy_version 148883 (0.0007) [2023-12-27 06:41:18,114][20347] Updated weights for policy 0, policy_version 149101 (0.0010) [2023-12-27 06:41:18,172][20347] Updated weights for policy 0, policy_version 149111 (0.0010) [2023-12-27 06:41:18,236][20347] Updated weights for policy 0, policy_version 149121 (0.0010) [2023-12-27 06:41:18,634][20381] Updated weights for policy 1, policy_version 148893 (0.0007) [2023-12-27 06:41:18,693][20381] Updated weights for policy 1, policy_version 148903 (0.0007) [2023-12-27 06:41:18,742][20381] Updated weights for policy 1, policy_version 148913 (0.0008) [2023-12-27 06:41:18,904][20347] Updated weights for policy 0, policy_version 149131 (0.0011) [2023-12-27 06:41:18,953][20347] Updated weights for policy 0, policy_version 149141 (0.0010) [2023-12-27 06:41:19,001][20347] Updated weights for policy 0, policy_version 149151 (0.0010) [2023-12-27 06:41:19,458][20381] Updated weights for policy 1, policy_version 148923 (0.0008) [2023-12-27 06:41:19,519][20381] Updated weights for policy 1, policy_version 148933 (0.0007) [2023-12-27 06:41:19,579][20381] Updated weights for policy 1, policy_version 148943 (0.0007) [2023-12-27 06:41:19,707][20347] Updated weights for policy 0, policy_version 149161 (0.0010) [2023-12-27 06:41:19,768][20347] Updated weights for policy 0, policy_version 149171 (0.0010) [2023-12-27 06:41:19,830][20347] Updated weights for policy 0, policy_version 149181 (0.0010) [2023-12-27 06:41:19,887][20347] Updated weights for policy 0, policy_version 149191 (0.0010) [2023-12-27 06:41:20,289][20381] Updated weights for policy 1, policy_version 148953 (0.0008) [2023-12-27 06:41:20,348][20381] Updated weights for policy 1, policy_version 148963 (0.0007) [2023-12-27 06:41:20,405][20381] Updated weights for policy 1, policy_version 148973 (0.0006) [2023-12-27 06:41:20,463][20381] Updated weights for policy 1, policy_version 148983 (0.0005) [2023-12-27 06:41:20,570][20347] Updated weights for policy 0, policy_version 149201 (0.0011) [2023-12-27 06:41:20,627][20347] Updated weights for policy 0, policy_version 149211 (0.0011) [2023-12-27 06:41:20,683][20347] Updated weights for policy 0, policy_version 149221 (0.0006) [2023-12-27 06:41:21,147][20381] Updated weights for policy 1, policy_version 148993 (0.0007) [2023-12-27 06:41:21,211][20381] Updated weights for policy 1, policy_version 149003 (0.0008) [2023-12-27 06:41:21,272][20381] Updated weights for policy 1, policy_version 149013 (0.0007) [2023-12-27 06:41:21,373][20347] Updated weights for policy 0, policy_version 149231 (0.0008) [2023-12-27 06:41:21,438][20347] Updated weights for policy 0, policy_version 149241 (0.0008) [2023-12-27 06:41:21,497][20347] Updated weights for policy 0, policy_version 149251 (0.0010) [2023-12-27 06:41:21,999][20381] Updated weights for policy 1, policy_version 149023 (0.0008) [2023-12-27 06:41:22,063][20381] Updated weights for policy 1, policy_version 149033 (0.0008) [2023-12-27 06:41:22,131][20381] Updated weights for policy 1, policy_version 149043 (0.0008) [2023-12-27 06:41:22,233][20347] Updated weights for policy 0, policy_version 149261 (0.0010) [2023-12-27 06:41:22,297][20347] Updated weights for policy 0, policy_version 149271 (0.0011) [2023-12-27 06:41:22,359][20347] Updated weights for policy 0, policy_version 149281 (0.0011) [2023-12-27 06:41:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 76390400. Throughput: 0: 10378.8, 1: 10429.9. Samples: 76363044. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000149048_38166528.pth... [2023-12-27 06:41:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000147832_37855232.pth [2023-12-27 06:41:22,398][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000149288_38232064.pth... [2023-12-27 06:41:22,401][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000148072_37920768.pth [2023-12-27 06:41:22,863][20381] Updated weights for policy 1, policy_version 149053 (0.0008) [2023-12-27 06:41:22,921][20381] Updated weights for policy 1, policy_version 149063 (0.0007) [2023-12-27 06:41:22,982][20381] Updated weights for policy 1, policy_version 149073 (0.0005) [2023-12-27 06:41:23,055][20347] Updated weights for policy 0, policy_version 149291 (0.0008) [2023-12-27 06:41:23,117][20347] Updated weights for policy 0, policy_version 149301 (0.0005) [2023-12-27 06:41:23,177][20347] Updated weights for policy 0, policy_version 149311 (0.0006) [2023-12-27 06:41:23,634][20381] Updated weights for policy 1, policy_version 149083 (0.0007) [2023-12-27 06:41:23,688][20381] Updated weights for policy 1, policy_version 149093 (0.0007) [2023-12-27 06:41:23,747][20381] Updated weights for policy 1, policy_version 149103 (0.0008) [2023-12-27 06:41:23,771][20347] Updated weights for policy 0, policy_version 149321 (0.0010) [2023-12-27 06:41:23,819][20347] Updated weights for policy 0, policy_version 149331 (0.0005) [2023-12-27 06:41:23,872][20347] Updated weights for policy 0, policy_version 149341 (0.0009) [2023-12-27 06:41:23,922][20347] Updated weights for policy 0, policy_version 149351 (0.0007) [2023-12-27 06:41:24,464][20381] Updated weights for policy 1, policy_version 149113 (0.0007) [2023-12-27 06:41:24,518][20381] Updated weights for policy 1, policy_version 149123 (0.0006) [2023-12-27 06:41:24,527][20347] Updated weights for policy 0, policy_version 149361 (0.0007) [2023-12-27 06:41:24,567][20381] Updated weights for policy 1, policy_version 149133 (0.0006) [2023-12-27 06:41:24,582][20347] Updated weights for policy 0, policy_version 149371 (0.0006) [2023-12-27 06:41:24,617][20381] Updated weights for policy 1, policy_version 149143 (0.0007) [2023-12-27 06:41:24,640][20347] Updated weights for policy 0, policy_version 149381 (0.0006) [2023-12-27 06:41:25,322][20347] Updated weights for policy 0, policy_version 149391 (0.0006) [2023-12-27 06:41:25,324][20381] Updated weights for policy 1, policy_version 149153 (0.0007) [2023-12-27 06:41:25,373][20347] Updated weights for policy 0, policy_version 149401 (0.0010) [2023-12-27 06:41:25,375][20381] Updated weights for policy 1, policy_version 149163 (0.0005) [2023-12-27 06:41:25,425][20347] Updated weights for policy 0, policy_version 149411 (0.0009) [2023-12-27 06:41:25,433][20381] Updated weights for policy 1, policy_version 149173 (0.0006) [2023-12-27 06:41:26,073][20347] Updated weights for policy 0, policy_version 149421 (0.0008) [2023-12-27 06:41:26,131][20347] Updated weights for policy 0, policy_version 149431 (0.0010) [2023-12-27 06:41:26,134][20381] Updated weights for policy 1, policy_version 149183 (0.0006) [2023-12-27 06:41:26,187][20381] Updated weights for policy 1, policy_version 149193 (0.0006) [2023-12-27 06:41:26,190][20347] Updated weights for policy 0, policy_version 149441 (0.0010) [2023-12-27 06:41:26,240][20381] Updated weights for policy 1, policy_version 149203 (0.0007) [2023-12-27 06:41:26,781][20347] Updated weights for policy 0, policy_version 149451 (0.0010) [2023-12-27 06:41:26,833][20347] Updated weights for policy 0, policy_version 149461 (0.0010) [2023-12-27 06:41:26,885][20347] Updated weights for policy 0, policy_version 149471 (0.0010) [2023-12-27 06:41:26,946][20381] Updated weights for policy 1, policy_version 149213 (0.0007) [2023-12-27 06:41:26,994][20381] Updated weights for policy 1, policy_version 149223 (0.0007) [2023-12-27 06:41:27,045][20381] Updated weights for policy 1, policy_version 149233 (0.0008) [2023-12-27 06:41:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 76496896. Throughput: 0: 10362.9, 1: 10362.6. Samples: 76485268. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:27,520][20347] Updated weights for policy 0, policy_version 149481 (0.0010) [2023-12-27 06:41:27,568][20347] Updated weights for policy 0, policy_version 149491 (0.0007) [2023-12-27 06:41:27,619][20347] Updated weights for policy 0, policy_version 149501 (0.0005) [2023-12-27 06:41:27,676][20347] Updated weights for policy 0, policy_version 149511 (0.0005) [2023-12-27 06:41:27,718][20381] Updated weights for policy 1, policy_version 149243 (0.0007) [2023-12-27 06:41:27,777][20381] Updated weights for policy 1, policy_version 149253 (0.0005) [2023-12-27 06:41:27,836][20381] Updated weights for policy 1, policy_version 149263 (0.0005) [2023-12-27 06:41:28,337][20347] Updated weights for policy 0, policy_version 149521 (0.0007) [2023-12-27 06:41:28,399][20347] Updated weights for policy 0, policy_version 149531 (0.0007) [2023-12-27 06:41:28,410][20381] Updated weights for policy 1, policy_version 149273 (0.0006) [2023-12-27 06:41:28,454][20347] Updated weights for policy 0, policy_version 149541 (0.0007) [2023-12-27 06:41:28,465][20381] Updated weights for policy 1, policy_version 149283 (0.0008) [2023-12-27 06:41:28,521][20381] Updated weights for policy 1, policy_version 149293 (0.0007) [2023-12-27 06:41:28,569][20381] Updated weights for policy 1, policy_version 149303 (0.0005) [2023-12-27 06:41:29,092][20347] Updated weights for policy 0, policy_version 149551 (0.0009) [2023-12-27 06:41:29,140][20347] Updated weights for policy 0, policy_version 149561 (0.0010) [2023-12-27 06:41:29,195][20347] Updated weights for policy 0, policy_version 149571 (0.0010) [2023-12-27 06:41:29,218][20381] Updated weights for policy 1, policy_version 149313 (0.0006) [2023-12-27 06:41:29,278][20381] Updated weights for policy 1, policy_version 149323 (0.0011) [2023-12-27 06:41:29,334][20381] Updated weights for policy 1, policy_version 149333 (0.0011) [2023-12-27 06:41:29,798][20347] Updated weights for policy 0, policy_version 149581 (0.0010) [2023-12-27 06:41:29,861][20347] Updated weights for policy 0, policy_version 149591 (0.0009) [2023-12-27 06:41:29,923][20347] Updated weights for policy 0, policy_version 149601 (0.0010) [2023-12-27 06:41:30,016][20381] Updated weights for policy 1, policy_version 149343 (0.0009) [2023-12-27 06:41:30,068][20381] Updated weights for policy 1, policy_version 149353 (0.0010) [2023-12-27 06:41:30,122][20381] Updated weights for policy 1, policy_version 149363 (0.0010) [2023-12-27 06:41:30,626][20347] Updated weights for policy 0, policy_version 149611 (0.0010) [2023-12-27 06:41:30,684][20347] Updated weights for policy 0, policy_version 149621 (0.0010) [2023-12-27 06:41:30,747][20347] Updated weights for policy 0, policy_version 149631 (0.0010) [2023-12-27 06:41:30,753][20381] Updated weights for policy 1, policy_version 149373 (0.0008) [2023-12-27 06:41:30,813][20381] Updated weights for policy 1, policy_version 149383 (0.0005) [2023-12-27 06:41:30,868][20381] Updated weights for policy 1, policy_version 149393 (0.0010) [2023-12-27 06:41:31,442][20347] Updated weights for policy 0, policy_version 149641 (0.0010) [2023-12-27 06:41:31,499][20347] Updated weights for policy 0, policy_version 149651 (0.0005) [2023-12-27 06:41:31,542][20381] Updated weights for policy 1, policy_version 149403 (0.0010) [2023-12-27 06:41:31,555][20347] Updated weights for policy 0, policy_version 149661 (0.0006) [2023-12-27 06:41:31,595][20381] Updated weights for policy 1, policy_version 149413 (0.0005) [2023-12-27 06:41:31,612][20347] Updated weights for policy 0, policy_version 149671 (0.0006) [2023-12-27 06:41:31,660][20381] Updated weights for policy 1, policy_version 149423 (0.0007) [2023-12-27 06:41:32,224][20347] Updated weights for policy 0, policy_version 149681 (0.0005) [2023-12-27 06:41:32,264][20381] Updated weights for policy 1, policy_version 149433 (0.0006) [2023-12-27 06:41:32,278][20347] Updated weights for policy 0, policy_version 149691 (0.0007) [2023-12-27 06:41:32,318][20381] Updated weights for policy 1, policy_version 149443 (0.0006) [2023-12-27 06:41:32,330][20347] Updated weights for policy 0, policy_version 149701 (0.0008) [2023-12-27 06:41:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 76603392. Throughput: 0: 10462.4, 1: 10346.9. Samples: 76613124. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:32,382][20381] Updated weights for policy 1, policy_version 149453 (0.0007) [2023-12-27 06:41:32,436][20381] Updated weights for policy 1, policy_version 149463 (0.0007) [2023-12-27 06:41:32,998][20381] Updated weights for policy 1, policy_version 149473 (0.0007) [2023-12-27 06:41:33,027][20347] Updated weights for policy 0, policy_version 149711 (0.0009) [2023-12-27 06:41:33,048][20381] Updated weights for policy 1, policy_version 149483 (0.0006) [2023-12-27 06:41:33,087][20347] Updated weights for policy 0, policy_version 149721 (0.0010) [2023-12-27 06:41:33,107][20381] Updated weights for policy 1, policy_version 149493 (0.0005) [2023-12-27 06:41:33,143][20347] Updated weights for policy 0, policy_version 149731 (0.0010) [2023-12-27 06:41:33,751][20347] Updated weights for policy 0, policy_version 149741 (0.0008) [2023-12-27 06:41:33,795][20381] Updated weights for policy 1, policy_version 149503 (0.0006) [2023-12-27 06:41:33,801][20347] Updated weights for policy 0, policy_version 149751 (0.0010) [2023-12-27 06:41:33,840][20381] Updated weights for policy 1, policy_version 149513 (0.0006) [2023-12-27 06:41:33,849][20347] Updated weights for policy 0, policy_version 149761 (0.0006) [2023-12-27 06:41:33,885][20381] Updated weights for policy 1, policy_version 149523 (0.0005) [2023-12-27 06:41:34,537][20347] Updated weights for policy 0, policy_version 149771 (0.0006) [2023-12-27 06:41:34,581][20347] Updated weights for policy 0, policy_version 149781 (0.0008) [2023-12-27 06:41:34,607][20381] Updated weights for policy 1, policy_version 149533 (0.0007) [2023-12-27 06:41:34,630][20347] Updated weights for policy 0, policy_version 149791 (0.0008) [2023-12-27 06:41:34,649][20381] Updated weights for policy 1, policy_version 149543 (0.0005) [2023-12-27 06:41:34,696][20381] Updated weights for policy 1, policy_version 149553 (0.0006) [2023-12-27 06:41:35,305][20381] Updated weights for policy 1, policy_version 149563 (0.0007) [2023-12-27 06:41:35,360][20347] Updated weights for policy 0, policy_version 149801 (0.0008) [2023-12-27 06:41:35,365][20381] Updated weights for policy 1, policy_version 149573 (0.0005) [2023-12-27 06:41:35,406][20347] Updated weights for policy 0, policy_version 149811 (0.0010) [2023-12-27 06:41:35,421][20381] Updated weights for policy 1, policy_version 149583 (0.0007) [2023-12-27 06:41:35,457][20347] Updated weights for policy 0, policy_version 149821 (0.0010) [2023-12-27 06:41:35,511][20347] Updated weights for policy 0, policy_version 149831 (0.0010) [2023-12-27 06:41:35,993][20381] Updated weights for policy 1, policy_version 149593 (0.0006) [2023-12-27 06:41:36,060][20381] Updated weights for policy 1, policy_version 149603 (0.0007) [2023-12-27 06:41:36,117][20381] Updated weights for policy 1, policy_version 149613 (0.0008) [2023-12-27 06:41:36,171][20347] Updated weights for policy 0, policy_version 149841 (0.0006) [2023-12-27 06:41:36,172][20381] Updated weights for policy 1, policy_version 149623 (0.0006) [2023-12-27 06:41:36,234][20347] Updated weights for policy 0, policy_version 149851 (0.0007) [2023-12-27 06:41:36,295][20347] Updated weights for policy 0, policy_version 149861 (0.0007) [2023-12-27 06:41:36,862][20381] Updated weights for policy 1, policy_version 149633 (0.0007) [2023-12-27 06:41:36,924][20381] Updated weights for policy 1, policy_version 149643 (0.0006) [2023-12-27 06:41:36,987][20381] Updated weights for policy 1, policy_version 149653 (0.0005) [2023-12-27 06:41:37,003][20347] Updated weights for policy 0, policy_version 149871 (0.0009) [2023-12-27 06:41:37,062][20347] Updated weights for policy 0, policy_version 149881 (0.0010) [2023-12-27 06:41:37,118][20347] Updated weights for policy 0, policy_version 149891 (0.0010) [2023-12-27 06:41:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 76709888. Throughput: 0: 10448.2, 1: 10299.3. Samples: 76676356. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000149656_38322176.pth... [2023-12-27 06:41:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000149896_38387712.pth... [2023-12-27 06:41:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000148472_38019072.pth [2023-12-27 06:41:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000148680_38076416.pth [2023-12-27 06:41:37,667][20381] Updated weights for policy 1, policy_version 149663 (0.0007) [2023-12-27 06:41:37,734][20381] Updated weights for policy 1, policy_version 149673 (0.0007) [2023-12-27 06:41:37,749][20347] Updated weights for policy 0, policy_version 149901 (0.0008) [2023-12-27 06:41:37,792][20381] Updated weights for policy 1, policy_version 149683 (0.0007) [2023-12-27 06:41:37,797][20347] Updated weights for policy 0, policy_version 149911 (0.0007) [2023-12-27 06:41:37,854][20347] Updated weights for policy 0, policy_version 149921 (0.0010) [2023-12-27 06:41:38,474][20381] Updated weights for policy 1, policy_version 149693 (0.0007) [2023-12-27 06:41:38,505][20347] Updated weights for policy 0, policy_version 149931 (0.0009) [2023-12-27 06:41:38,537][20381] Updated weights for policy 1, policy_version 149703 (0.0008) [2023-12-27 06:41:38,554][20347] Updated weights for policy 0, policy_version 149941 (0.0006) [2023-12-27 06:41:38,590][20381] Updated weights for policy 1, policy_version 149713 (0.0006) [2023-12-27 06:41:38,606][20347] Updated weights for policy 0, policy_version 149951 (0.0009) [2023-12-27 06:41:39,264][20381] Updated weights for policy 1, policy_version 149723 (0.0006) [2023-12-27 06:41:39,280][20347] Updated weights for policy 0, policy_version 149961 (0.0010) [2023-12-27 06:41:39,318][20381] Updated weights for policy 1, policy_version 149733 (0.0007) [2023-12-27 06:41:39,344][20347] Updated weights for policy 0, policy_version 149971 (0.0009) [2023-12-27 06:41:39,383][20381] Updated weights for policy 1, policy_version 149743 (0.0008) [2023-12-27 06:41:39,412][20347] Updated weights for policy 0, policy_version 149981 (0.0007) [2023-12-27 06:41:39,468][20347] Updated weights for policy 0, policy_version 149991 (0.0007) [2023-12-27 06:41:40,125][20381] Updated weights for policy 1, policy_version 149753 (0.0009) [2023-12-27 06:41:40,174][20347] Updated weights for policy 0, policy_version 150001 (0.0006) [2023-12-27 06:41:40,185][20381] Updated weights for policy 1, policy_version 149763 (0.0008) [2023-12-27 06:41:40,231][20347] Updated weights for policy 0, policy_version 150011 (0.0009) [2023-12-27 06:41:40,245][20381] Updated weights for policy 1, policy_version 149773 (0.0007) [2023-12-27 06:41:40,289][20347] Updated weights for policy 0, policy_version 150021 (0.0007) [2023-12-27 06:41:40,306][20381] Updated weights for policy 1, policy_version 149783 (0.0006) [2023-12-27 06:41:40,903][20347] Updated weights for policy 0, policy_version 150031 (0.0007) [2023-12-27 06:41:40,965][20347] Updated weights for policy 0, policy_version 150041 (0.0007) [2023-12-27 06:41:41,022][20381] Updated weights for policy 1, policy_version 149793 (0.0008) [2023-12-27 06:41:41,024][20347] Updated weights for policy 0, policy_version 150051 (0.0008) [2023-12-27 06:41:41,081][20381] Updated weights for policy 1, policy_version 149803 (0.0007) [2023-12-27 06:41:41,146][20381] Updated weights for policy 1, policy_version 149813 (0.0007) [2023-12-27 06:41:41,718][20347] Updated weights for policy 0, policy_version 150061 (0.0008) [2023-12-27 06:41:41,781][20347] Updated weights for policy 0, policy_version 150071 (0.0008) [2023-12-27 06:41:41,843][20347] Updated weights for policy 0, policy_version 150081 (0.0007) [2023-12-27 06:41:41,845][20381] Updated weights for policy 1, policy_version 149823 (0.0007) [2023-12-27 06:41:41,899][20381] Updated weights for policy 1, policy_version 149833 (0.0006) [2023-12-27 06:41:41,946][20381] Updated weights for policy 1, policy_version 149843 (0.0006) [2023-12-27 06:41:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 76808192. Throughput: 0: 10414.8, 1: 10240.1. Samples: 76798480. Policy #0 lag: (min: 16.0, avg: 41.8, max: 48.0) [2023-12-27 06:41:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:42,580][20347] Updated weights for policy 0, policy_version 150091 (0.0007) [2023-12-27 06:41:42,612][20381] Updated weights for policy 1, policy_version 149853 (0.0006) [2023-12-27 06:41:42,641][20347] Updated weights for policy 0, policy_version 150101 (0.0006) [2023-12-27 06:41:42,672][20381] Updated weights for policy 1, policy_version 149863 (0.0007) [2023-12-27 06:41:42,697][20347] Updated weights for policy 0, policy_version 150111 (0.0005) [2023-12-27 06:41:42,729][20381] Updated weights for policy 1, policy_version 149873 (0.0006) [2023-12-27 06:41:43,321][20347] Updated weights for policy 0, policy_version 150121 (0.0006) [2023-12-27 06:41:43,372][20347] Updated weights for policy 0, policy_version 150131 (0.0008) [2023-12-27 06:41:43,417][20347] Updated weights for policy 0, policy_version 150141 (0.0007) [2023-12-27 06:41:43,442][20381] Updated weights for policy 1, policy_version 149883 (0.0006) [2023-12-27 06:41:43,471][20347] Updated weights for policy 0, policy_version 150151 (0.0006) [2023-12-27 06:41:43,495][20381] Updated weights for policy 1, policy_version 149893 (0.0006) [2023-12-27 06:41:43,551][20381] Updated weights for policy 1, policy_version 149903 (0.0007) [2023-12-27 06:41:44,194][20347] Updated weights for policy 0, policy_version 150161 (0.0008) [2023-12-27 06:41:44,252][20347] Updated weights for policy 0, policy_version 150171 (0.0007) [2023-12-27 06:41:44,264][20381] Updated weights for policy 1, policy_version 149913 (0.0007) [2023-12-27 06:41:44,309][20347] Updated weights for policy 0, policy_version 150181 (0.0006) [2023-12-27 06:41:44,319][20381] Updated weights for policy 1, policy_version 149923 (0.0006) [2023-12-27 06:41:44,375][20381] Updated weights for policy 1, policy_version 149933 (0.0007) [2023-12-27 06:41:44,427][20381] Updated weights for policy 1, policy_version 149943 (0.0008) [2023-12-27 06:41:44,977][20347] Updated weights for policy 0, policy_version 150191 (0.0008) [2023-12-27 06:41:45,034][20347] Updated weights for policy 0, policy_version 150201 (0.0011) [2023-12-27 06:41:45,094][20347] Updated weights for policy 0, policy_version 150211 (0.0010) [2023-12-27 06:41:45,151][20381] Updated weights for policy 1, policy_version 149953 (0.0008) [2023-12-27 06:41:45,207][20381] Updated weights for policy 1, policy_version 149963 (0.0007) [2023-12-27 06:41:45,259][20381] Updated weights for policy 1, policy_version 149973 (0.0007) [2023-12-27 06:41:45,762][20347] Updated weights for policy 0, policy_version 150221 (0.0010) [2023-12-27 06:41:45,812][20347] Updated weights for policy 0, policy_version 150231 (0.0010) [2023-12-27 06:41:45,864][20347] Updated weights for policy 0, policy_version 150241 (0.0010) [2023-12-27 06:41:45,958][20381] Updated weights for policy 1, policy_version 149983 (0.0009) [2023-12-27 06:41:46,009][20381] Updated weights for policy 1, policy_version 149993 (0.0009) [2023-12-27 06:41:46,062][20381] Updated weights for policy 1, policy_version 150003 (0.0009) [2023-12-27 06:41:46,438][20347] Updated weights for policy 0, policy_version 150251 (0.0008) [2023-12-27 06:41:46,490][20347] Updated weights for policy 0, policy_version 150261 (0.0006) [2023-12-27 06:41:46,541][20347] Updated weights for policy 0, policy_version 150271 (0.0006) [2023-12-27 06:41:46,834][20381] Updated weights for policy 1, policy_version 150013 (0.0007) [2023-12-27 06:41:46,887][20381] Updated weights for policy 1, policy_version 150023 (0.0009) [2023-12-27 06:41:46,947][20381] Updated weights for policy 1, policy_version 150033 (0.0008) [2023-12-27 06:41:47,097][20347] Updated weights for policy 0, policy_version 150281 (0.0006) [2023-12-27 06:41:47,149][20347] Updated weights for policy 0, policy_version 150291 (0.0005) [2023-12-27 06:41:47,194][20347] Updated weights for policy 0, policy_version 150301 (0.0005) [2023-12-27 06:41:47,253][20347] Updated weights for policy 0, policy_version 150311 (0.0007) [2023-12-27 06:41:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 76914688. Throughput: 0: 10459.1, 1: 10184.3. Samples: 76920736. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:41:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:47,695][20381] Updated weights for policy 1, policy_version 150043 (0.0008) [2023-12-27 06:41:47,755][20381] Updated weights for policy 1, policy_version 150053 (0.0007) [2023-12-27 06:41:47,813][20381] Updated weights for policy 1, policy_version 150063 (0.0005) [2023-12-27 06:41:47,854][20347] Updated weights for policy 0, policy_version 150321 (0.0005) [2023-12-27 06:41:47,907][20347] Updated weights for policy 0, policy_version 150331 (0.0006) [2023-12-27 06:41:47,963][20347] Updated weights for policy 0, policy_version 150341 (0.0008) [2023-12-27 06:41:48,396][20381] Updated weights for policy 1, policy_version 150073 (0.0005) [2023-12-27 06:41:48,462][20381] Updated weights for policy 1, policy_version 150083 (0.0008) [2023-12-27 06:41:48,509][20381] Updated weights for policy 1, policy_version 150093 (0.0009) [2023-12-27 06:41:48,551][20347] Updated weights for policy 0, policy_version 150351 (0.0009) [2023-12-27 06:41:48,564][20381] Updated weights for policy 1, policy_version 150103 (0.0008) [2023-12-27 06:41:48,609][20347] Updated weights for policy 0, policy_version 150361 (0.0010) [2023-12-27 06:41:48,664][20347] Updated weights for policy 0, policy_version 150371 (0.0010) [2023-12-27 06:41:49,252][20381] Updated weights for policy 1, policy_version 150113 (0.0008) [2023-12-27 06:41:49,311][20381] Updated weights for policy 1, policy_version 150123 (0.0007) [2023-12-27 06:41:49,364][20347] Updated weights for policy 0, policy_version 150381 (0.0010) [2023-12-27 06:41:49,373][20381] Updated weights for policy 1, policy_version 150133 (0.0008) [2023-12-27 06:41:49,423][20347] Updated weights for policy 0, policy_version 150391 (0.0007) [2023-12-27 06:41:49,472][20347] Updated weights for policy 0, policy_version 150401 (0.0006) [2023-12-27 06:41:50,018][20381] Updated weights for policy 1, policy_version 150143 (0.0007) [2023-12-27 06:41:50,066][20381] Updated weights for policy 1, policy_version 150153 (0.0007) [2023-12-27 06:41:50,123][20347] Updated weights for policy 0, policy_version 150411 (0.0005) [2023-12-27 06:41:50,127][20381] Updated weights for policy 1, policy_version 150163 (0.0008) [2023-12-27 06:41:50,180][20347] Updated weights for policy 0, policy_version 150421 (0.0006) [2023-12-27 06:41:50,237][20347] Updated weights for policy 0, policy_version 150431 (0.0005) [2023-12-27 06:41:50,845][20347] Updated weights for policy 0, policy_version 150441 (0.0006) [2023-12-27 06:41:50,850][20381] Updated weights for policy 1, policy_version 150173 (0.0009) [2023-12-27 06:41:50,903][20347] Updated weights for policy 0, policy_version 150451 (0.0010) [2023-12-27 06:41:50,912][20381] Updated weights for policy 1, policy_version 150183 (0.0007) [2023-12-27 06:41:50,958][20347] Updated weights for policy 0, policy_version 150461 (0.0010) [2023-12-27 06:41:50,968][20381] Updated weights for policy 1, policy_version 150193 (0.0006) [2023-12-27 06:41:51,011][20347] Updated weights for policy 0, policy_version 150471 (0.0011) [2023-12-27 06:41:51,697][20381] Updated weights for policy 1, policy_version 150203 (0.0006) [2023-12-27 06:41:51,726][20347] Updated weights for policy 0, policy_version 150481 (0.0008) [2023-12-27 06:41:51,757][20381] Updated weights for policy 1, policy_version 150213 (0.0008) [2023-12-27 06:41:51,784][20347] Updated weights for policy 0, policy_version 150491 (0.0008) [2023-12-27 06:41:51,809][20381] Updated weights for policy 1, policy_version 150223 (0.0006) [2023-12-27 06:41:51,839][20347] Updated weights for policy 0, policy_version 150501 (0.0007) [2023-12-27 06:41:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 77012992. Throughput: 0: 10497.0, 1: 10177.8. Samples: 76984024. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:41:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000150504_38543360.pth... [2023-12-27 06:41:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000150232_38469632.pth... [2023-12-27 06:41:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000149288_38232064.pth [2023-12-27 06:41:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000149048_38166528.pth [2023-12-27 06:41:52,446][20381] Updated weights for policy 1, policy_version 150233 (0.0007) [2023-12-27 06:41:52,505][20381] Updated weights for policy 1, policy_version 150243 (0.0007) [2023-12-27 06:41:52,554][20347] Updated weights for policy 0, policy_version 150511 (0.0010) [2023-12-27 06:41:52,561][20381] Updated weights for policy 1, policy_version 150253 (0.0007) [2023-12-27 06:41:52,616][20381] Updated weights for policy 1, policy_version 150263 (0.0007) [2023-12-27 06:41:52,623][20347] Updated weights for policy 0, policy_version 150521 (0.0010) [2023-12-27 06:41:52,688][20347] Updated weights for policy 0, policy_version 150531 (0.0008) [2023-12-27 06:41:53,278][20381] Updated weights for policy 1, policy_version 150273 (0.0007) [2023-12-27 06:41:53,280][20347] Updated weights for policy 0, policy_version 150541 (0.0007) [2023-12-27 06:41:53,337][20347] Updated weights for policy 0, policy_version 150551 (0.0005) [2023-12-27 06:41:53,339][20381] Updated weights for policy 1, policy_version 150283 (0.0006) [2023-12-27 06:41:53,392][20347] Updated weights for policy 0, policy_version 150561 (0.0005) [2023-12-27 06:41:53,399][20381] Updated weights for policy 1, policy_version 150293 (0.0008) [2023-12-27 06:41:53,988][20347] Updated weights for policy 0, policy_version 150571 (0.0006) [2023-12-27 06:41:54,018][20381] Updated weights for policy 1, policy_version 150303 (0.0011) [2023-12-27 06:41:54,040][20347] Updated weights for policy 0, policy_version 150581 (0.0006) [2023-12-27 06:41:54,068][20381] Updated weights for policy 1, policy_version 150313 (0.0011) [2023-12-27 06:41:54,099][20347] Updated weights for policy 0, policy_version 150591 (0.0007) [2023-12-27 06:41:54,123][20381] Updated weights for policy 1, policy_version 150323 (0.0011) [2023-12-27 06:41:54,790][20347] Updated weights for policy 0, policy_version 150601 (0.0007) [2023-12-27 06:41:54,814][20381] Updated weights for policy 1, policy_version 150333 (0.0007) [2023-12-27 06:41:54,841][20347] Updated weights for policy 0, policy_version 150611 (0.0006) [2023-12-27 06:41:54,870][20381] Updated weights for policy 1, policy_version 150343 (0.0005) [2023-12-27 06:41:54,900][20347] Updated weights for policy 0, policy_version 150621 (0.0009) [2023-12-27 06:41:54,930][20381] Updated weights for policy 1, policy_version 150353 (0.0007) [2023-12-27 06:41:54,959][20347] Updated weights for policy 0, policy_version 150631 (0.0006) [2023-12-27 06:41:55,601][20381] Updated weights for policy 1, policy_version 150363 (0.0006) [2023-12-27 06:41:55,633][20347] Updated weights for policy 0, policy_version 150641 (0.0006) [2023-12-27 06:41:55,658][20381] Updated weights for policy 1, policy_version 150373 (0.0006) [2023-12-27 06:41:55,691][20347] Updated weights for policy 0, policy_version 150651 (0.0006) [2023-12-27 06:41:55,708][20381] Updated weights for policy 1, policy_version 150383 (0.0007) [2023-12-27 06:41:55,744][20347] Updated weights for policy 0, policy_version 150661 (0.0007) [2023-12-27 06:41:56,359][20381] Updated weights for policy 1, policy_version 150393 (0.0007) [2023-12-27 06:41:56,417][20381] Updated weights for policy 1, policy_version 150403 (0.0006) [2023-12-27 06:41:56,454][20347] Updated weights for policy 0, policy_version 150671 (0.0007) [2023-12-27 06:41:56,469][20381] Updated weights for policy 1, policy_version 150413 (0.0006) [2023-12-27 06:41:56,509][20347] Updated weights for policy 0, policy_version 150681 (0.0008) [2023-12-27 06:41:56,524][20381] Updated weights for policy 1, policy_version 150423 (0.0006) [2023-12-27 06:41:56,557][20347] Updated weights for policy 0, policy_version 150691 (0.0007) [2023-12-27 06:41:57,140][20381] Updated weights for policy 1, policy_version 150433 (0.0008) [2023-12-27 06:41:57,188][20381] Updated weights for policy 1, policy_version 150443 (0.0008) [2023-12-27 06:41:57,245][20381] Updated weights for policy 1, policy_version 150453 (0.0007) [2023-12-27 06:41:57,261][20347] Updated weights for policy 0, policy_version 150701 (0.0008) [2023-12-27 06:41:57,313][20347] Updated weights for policy 0, policy_version 150711 (0.0007) [2023-12-27 06:41:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 77119488. Throughput: 0: 10517.8, 1: 10167.7. Samples: 77109280. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:41:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:41:57,370][20347] Updated weights for policy 0, policy_version 150721 (0.0005) [2023-12-27 06:41:57,906][20381] Updated weights for policy 1, policy_version 150463 (0.0007) [2023-12-27 06:41:57,955][20381] Updated weights for policy 1, policy_version 150473 (0.0007) [2023-12-27 06:41:58,003][20381] Updated weights for policy 1, policy_version 150483 (0.0007) [2023-12-27 06:41:58,043][20347] Updated weights for policy 0, policy_version 150731 (0.0008) [2023-12-27 06:41:58,098][20347] Updated weights for policy 0, policy_version 150741 (0.0007) [2023-12-27 06:41:58,154][20347] Updated weights for policy 0, policy_version 150751 (0.0007) [2023-12-27 06:41:58,676][20381] Updated weights for policy 1, policy_version 150493 (0.0007) [2023-12-27 06:41:58,739][20381] Updated weights for policy 1, policy_version 150503 (0.0007) [2023-12-27 06:41:58,800][20381] Updated weights for policy 1, policy_version 150513 (0.0009) [2023-12-27 06:41:58,942][20347] Updated weights for policy 0, policy_version 150761 (0.0009) [2023-12-27 06:41:59,001][20347] Updated weights for policy 0, policy_version 150771 (0.0006) [2023-12-27 06:41:59,060][20347] Updated weights for policy 0, policy_version 150781 (0.0007) [2023-12-27 06:41:59,116][20347] Updated weights for policy 0, policy_version 150791 (0.0008) [2023-12-27 06:41:59,555][20381] Updated weights for policy 1, policy_version 150523 (0.0008) [2023-12-27 06:41:59,608][20381] Updated weights for policy 1, policy_version 150533 (0.0006) [2023-12-27 06:41:59,659][20381] Updated weights for policy 1, policy_version 150543 (0.0005) [2023-12-27 06:41:59,860][20347] Updated weights for policy 0, policy_version 150801 (0.0008) [2023-12-27 06:41:59,911][20347] Updated weights for policy 0, policy_version 150811 (0.0007) [2023-12-27 06:41:59,972][20347] Updated weights for policy 0, policy_version 150821 (0.0007) [2023-12-27 06:42:00,329][20381] Updated weights for policy 1, policy_version 150553 (0.0006) [2023-12-27 06:42:00,391][20381] Updated weights for policy 1, policy_version 150563 (0.0008) [2023-12-27 06:42:00,437][20381] Updated weights for policy 1, policy_version 150573 (0.0008) [2023-12-27 06:42:00,481][20381] Updated weights for policy 1, policy_version 150583 (0.0007) [2023-12-27 06:42:00,637][20347] Updated weights for policy 0, policy_version 150831 (0.0007) [2023-12-27 06:42:00,692][20347] Updated weights for policy 0, policy_version 150841 (0.0005) [2023-12-27 06:42:00,744][20347] Updated weights for policy 0, policy_version 150851 (0.0005) [2023-12-27 06:42:01,207][20381] Updated weights for policy 1, policy_version 150593 (0.0007) [2023-12-27 06:42:01,259][20381] Updated weights for policy 1, policy_version 150603 (0.0007) [2023-12-27 06:42:01,309][20381] Updated weights for policy 1, policy_version 150613 (0.0008) [2023-12-27 06:42:01,370][20347] Updated weights for policy 0, policy_version 150861 (0.0006) [2023-12-27 06:42:01,420][20347] Updated weights for policy 0, policy_version 150871 (0.0006) [2023-12-27 06:42:01,473][20347] Updated weights for policy 0, policy_version 150881 (0.0006) [2023-12-27 06:42:02,047][20381] Updated weights for policy 1, policy_version 150623 (0.0008) [2023-12-27 06:42:02,103][20381] Updated weights for policy 1, policy_version 150633 (0.0008) [2023-12-27 06:42:02,154][20347] Updated weights for policy 0, policy_version 150891 (0.0007) [2023-12-27 06:42:02,162][20381] Updated weights for policy 1, policy_version 150643 (0.0008) [2023-12-27 06:42:02,210][20347] Updated weights for policy 0, policy_version 150901 (0.0007) [2023-12-27 06:42:02,266][20347] Updated weights for policy 0, policy_version 150911 (0.0007) [2023-12-27 06:42:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 77225984. Throughput: 0: 10419.3, 1: 10174.3. Samples: 77230044. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:42:02,846][20381] Updated weights for policy 1, policy_version 150653 (0.0005) [2023-12-27 06:42:02,905][20381] Updated weights for policy 1, policy_version 150663 (0.0005) [2023-12-27 06:42:02,926][20347] Updated weights for policy 0, policy_version 150921 (0.0008) [2023-12-27 06:42:02,960][20381] Updated weights for policy 1, policy_version 150673 (0.0007) [2023-12-27 06:42:02,981][20347] Updated weights for policy 0, policy_version 150931 (0.0010) [2023-12-27 06:42:03,037][20347] Updated weights for policy 0, policy_version 150941 (0.0010) [2023-12-27 06:42:03,088][20347] Updated weights for policy 0, policy_version 150951 (0.0010) [2023-12-27 06:42:03,639][20381] Updated weights for policy 1, policy_version 150683 (0.0006) [2023-12-27 06:42:03,690][20381] Updated weights for policy 1, policy_version 150693 (0.0007) [2023-12-27 06:42:03,742][20381] Updated weights for policy 1, policy_version 150703 (0.0007) [2023-12-27 06:42:03,768][20347] Updated weights for policy 0, policy_version 150961 (0.0010) [2023-12-27 06:42:03,825][20347] Updated weights for policy 0, policy_version 150971 (0.0010) [2023-12-27 06:42:03,882][20347] Updated weights for policy 0, policy_version 150981 (0.0010) [2023-12-27 06:42:04,465][20381] Updated weights for policy 1, policy_version 150713 (0.0007) [2023-12-27 06:42:04,522][20381] Updated weights for policy 1, policy_version 150723 (0.0007) [2023-12-27 06:42:04,579][20347] Updated weights for policy 0, policy_version 150991 (0.0010) [2023-12-27 06:42:04,585][20381] Updated weights for policy 1, policy_version 150733 (0.0006) [2023-12-27 06:42:04,633][20381] Updated weights for policy 1, policy_version 150743 (0.0008) [2023-12-27 06:42:04,635][20347] Updated weights for policy 0, policy_version 151001 (0.0011) [2023-12-27 06:42:04,690][20347] Updated weights for policy 0, policy_version 151011 (0.0010) [2023-12-27 06:42:05,336][20347] Updated weights for policy 0, policy_version 151021 (0.0010) [2023-12-27 06:42:05,337][20381] Updated weights for policy 1, policy_version 150753 (0.0005) [2023-12-27 06:42:05,393][20347] Updated weights for policy 0, policy_version 151031 (0.0010) [2023-12-27 06:42:05,423][20381] Updated weights for policy 1, policy_version 150763 (0.0007) [2023-12-27 06:42:05,450][20347] Updated weights for policy 0, policy_version 151041 (0.0010) [2023-12-27 06:42:05,474][20381] Updated weights for policy 1, policy_version 150773 (0.0005) [2023-12-27 06:42:06,103][20381] Updated weights for policy 1, policy_version 150783 (0.0006) [2023-12-27 06:42:06,125][20347] Updated weights for policy 0, policy_version 151051 (0.0010) [2023-12-27 06:42:06,164][20381] Updated weights for policy 1, policy_version 150793 (0.0011) [2023-12-27 06:42:06,188][20347] Updated weights for policy 0, policy_version 151061 (0.0010) [2023-12-27 06:42:06,224][20381] Updated weights for policy 1, policy_version 150803 (0.0006) [2023-12-27 06:42:06,247][20347] Updated weights for policy 0, policy_version 151071 (0.0010) [2023-12-27 06:42:06,813][20381] Updated weights for policy 1, policy_version 150813 (0.0008) [2023-12-27 06:42:06,861][20381] Updated weights for policy 1, policy_version 150823 (0.0009) [2023-12-27 06:42:06,907][20381] Updated weights for policy 1, policy_version 150833 (0.0005) [2023-12-27 06:42:06,940][20347] Updated weights for policy 0, policy_version 151081 (0.0011) [2023-12-27 06:42:07,009][20347] Updated weights for policy 0, policy_version 151091 (0.0010) [2023-12-27 06:42:07,073][20347] Updated weights for policy 0, policy_version 151101 (0.0010) [2023-12-27 06:42:07,131][20347] Updated weights for policy 0, policy_version 151111 (0.0010) [2023-12-27 06:42:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 77324288. Throughput: 0: 10432.6, 1: 10192.8. Samples: 77291188. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:42:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000150840_38625280.pth... [2023-12-27 06:42:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000151112_38699008.pth... [2023-12-27 06:42:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000149656_38322176.pth [2023-12-27 06:42:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000149896_38387712.pth [2023-12-27 06:42:07,571][20381] Updated weights for policy 1, policy_version 150843 (0.0007) [2023-12-27 06:42:07,627][20381] Updated weights for policy 1, policy_version 150853 (0.0008) [2023-12-27 06:42:07,685][20381] Updated weights for policy 1, policy_version 150863 (0.0008) [2023-12-27 06:42:07,686][20347] Updated weights for policy 0, policy_version 151121 (0.0006) [2023-12-27 06:42:07,734][20347] Updated weights for policy 0, policy_version 151131 (0.0008) [2023-12-27 06:42:07,785][20347] Updated weights for policy 0, policy_version 151141 (0.0010) [2023-12-27 06:42:08,371][20347] Updated weights for policy 0, policy_version 151151 (0.0010) [2023-12-27 06:42:08,375][20381] Updated weights for policy 1, policy_version 150873 (0.0009) [2023-12-27 06:42:08,426][20347] Updated weights for policy 0, policy_version 151161 (0.0010) [2023-12-27 06:42:08,430][20381] Updated weights for policy 1, policy_version 150883 (0.0008) [2023-12-27 06:42:08,482][20347] Updated weights for policy 0, policy_version 151171 (0.0010) [2023-12-27 06:42:08,483][20381] Updated weights for policy 1, policy_version 150893 (0.0011) [2023-12-27 06:42:08,540][20381] Updated weights for policy 1, policy_version 150903 (0.0010) [2023-12-27 06:42:09,106][20347] Updated weights for policy 0, policy_version 151181 (0.0010) [2023-12-27 06:42:09,164][20347] Updated weights for policy 0, policy_version 151191 (0.0010) [2023-12-27 06:42:09,228][20347] Updated weights for policy 0, policy_version 151201 (0.0010) [2023-12-27 06:42:09,229][20381] Updated weights for policy 1, policy_version 150913 (0.0010) [2023-12-27 06:42:09,288][20381] Updated weights for policy 1, policy_version 150923 (0.0007) [2023-12-27 06:42:09,357][20381] Updated weights for policy 1, policy_version 150933 (0.0011) [2023-12-27 06:42:09,950][20347] Updated weights for policy 0, policy_version 151211 (0.0009) [2023-12-27 06:42:09,972][20381] Updated weights for policy 1, policy_version 150943 (0.0007) [2023-12-27 06:42:10,009][20347] Updated weights for policy 0, policy_version 151221 (0.0007) [2023-12-27 06:42:10,030][20381] Updated weights for policy 1, policy_version 150953 (0.0006) [2023-12-27 06:42:10,060][20347] Updated weights for policy 0, policy_version 151231 (0.0006) [2023-12-27 06:42:10,082][20381] Updated weights for policy 1, policy_version 150963 (0.0007) [2023-12-27 06:42:10,771][20381] Updated weights for policy 1, policy_version 150973 (0.0007) [2023-12-27 06:42:10,795][20347] Updated weights for policy 0, policy_version 151241 (0.0007) [2023-12-27 06:42:10,827][20381] Updated weights for policy 1, policy_version 150983 (0.0006) [2023-12-27 06:42:10,848][20347] Updated weights for policy 0, policy_version 151251 (0.0006) [2023-12-27 06:42:10,883][20381] Updated weights for policy 1, policy_version 150993 (0.0007) [2023-12-27 06:42:10,905][20347] Updated weights for policy 0, policy_version 151261 (0.0006) [2023-12-27 06:42:10,962][20347] Updated weights for policy 0, policy_version 151271 (0.0007) [2023-12-27 06:42:11,618][20381] Updated weights for policy 1, policy_version 151003 (0.0007) [2023-12-27 06:42:11,679][20381] Updated weights for policy 1, policy_version 151013 (0.0011) [2023-12-27 06:42:11,681][20347] Updated weights for policy 0, policy_version 151281 (0.0006) [2023-12-27 06:42:11,744][20347] Updated weights for policy 0, policy_version 151291 (0.0007) [2023-12-27 06:42:11,747][20381] Updated weights for policy 1, policy_version 151023 (0.0010) [2023-12-27 06:42:11,798][20347] Updated weights for policy 0, policy_version 151301 (0.0006) [2023-12-27 06:42:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 77422592. Throughput: 0: 10431.8, 1: 10255.4. Samples: 77416192. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '999.200')] [2023-12-27 06:42:12,428][20381] Updated weights for policy 1, policy_version 151033 (0.0011) [2023-12-27 06:42:12,485][20381] Updated weights for policy 1, policy_version 151043 (0.0009) [2023-12-27 06:42:12,541][20381] Updated weights for policy 1, policy_version 151053 (0.0011) [2023-12-27 06:42:12,542][20347] Updated weights for policy 0, policy_version 151311 (0.0006) [2023-12-27 06:42:12,590][20381] Updated weights for policy 1, policy_version 151063 (0.0010) [2023-12-27 06:42:12,597][20347] Updated weights for policy 0, policy_version 151321 (0.0007) [2023-12-27 06:42:12,663][20347] Updated weights for policy 0, policy_version 151331 (0.0007) [2023-12-27 06:42:13,277][20347] Updated weights for policy 0, policy_version 151341 (0.0007) [2023-12-27 06:42:13,294][20381] Updated weights for policy 1, policy_version 151073 (0.0011) [2023-12-27 06:42:13,326][20347] Updated weights for policy 0, policy_version 151351 (0.0005) [2023-12-27 06:42:13,349][20381] Updated weights for policy 1, policy_version 151083 (0.0011) [2023-12-27 06:42:13,388][20347] Updated weights for policy 0, policy_version 151361 (0.0005) [2023-12-27 06:42:13,404][20381] Updated weights for policy 1, policy_version 151093 (0.0009) [2023-12-27 06:42:13,960][20347] Updated weights for policy 0, policy_version 151371 (0.0006) [2023-12-27 06:42:14,012][20347] Updated weights for policy 0, policy_version 151381 (0.0006) [2023-12-27 06:42:14,025][20381] Updated weights for policy 1, policy_version 151103 (0.0009) [2023-12-27 06:42:14,047][20292] KL-divergence is very high: 442.1157 [2023-12-27 06:42:14,059][20292] KL-divergence is very high: 340.5273 [2023-12-27 06:42:14,063][20347] Updated weights for policy 0, policy_version 151391 (0.0006) [2023-12-27 06:42:14,074][20381] Updated weights for policy 1, policy_version 151113 (0.0010) [2023-12-27 06:42:14,127][20381] Updated weights for policy 1, policy_version 151123 (0.0010) [2023-12-27 06:42:14,661][20347] Updated weights for policy 0, policy_version 151401 (0.0005) [2023-12-27 06:42:14,716][20347] Updated weights for policy 0, policy_version 151411 (0.0005) [2023-12-27 06:42:14,774][20347] Updated weights for policy 0, policy_version 151421 (0.0007) [2023-12-27 06:42:14,812][20381] Updated weights for policy 1, policy_version 151133 (0.0009) [2023-12-27 06:42:14,836][20347] Updated weights for policy 0, policy_version 151431 (0.0007) [2023-12-27 06:42:14,872][20381] Updated weights for policy 1, policy_version 151143 (0.0009) [2023-12-27 06:42:14,934][20381] Updated weights for policy 1, policy_version 151153 (0.0007) [2023-12-27 06:42:15,477][20347] Updated weights for policy 0, policy_version 151441 (0.0008) [2023-12-27 06:42:15,534][20347] Updated weights for policy 0, policy_version 151451 (0.0007) [2023-12-27 06:42:15,593][20347] Updated weights for policy 0, policy_version 151461 (0.0008) [2023-12-27 06:42:15,622][20381] Updated weights for policy 1, policy_version 151163 (0.0008) [2023-12-27 06:42:15,681][20381] Updated weights for policy 1, policy_version 151173 (0.0008) [2023-12-27 06:42:15,733][20381] Updated weights for policy 1, policy_version 151183 (0.0008) [2023-12-27 06:42:16,240][20347] Updated weights for policy 0, policy_version 151471 (0.0008) [2023-12-27 06:42:16,299][20347] Updated weights for policy 0, policy_version 151481 (0.0008) [2023-12-27 06:42:16,358][20347] Updated weights for policy 0, policy_version 151491 (0.0007) [2023-12-27 06:42:16,404][20381] Updated weights for policy 1, policy_version 151193 (0.0007) [2023-12-27 06:42:16,456][20381] Updated weights for policy 1, policy_version 151203 (0.0008) [2023-12-27 06:42:16,509][20381] Updated weights for policy 1, policy_version 151213 (0.0007) [2023-12-27 06:42:16,569][20381] Updated weights for policy 1, policy_version 151223 (0.0008) [2023-12-27 06:42:16,990][20347] Updated weights for policy 0, policy_version 151501 (0.0006) [2023-12-27 06:42:17,051][20347] Updated weights for policy 0, policy_version 151511 (0.0005) [2023-12-27 06:42:17,115][20347] Updated weights for policy 0, policy_version 151521 (0.0008) [2023-12-27 06:42:17,211][20381] Updated weights for policy 1, policy_version 151233 (0.0009) [2023-12-27 06:42:17,263][20381] Updated weights for policy 1, policy_version 151243 (0.0009) [2023-12-27 06:42:17,311][20381] Updated weights for policy 1, policy_version 151253 (0.0008) [2023-12-27 06:42:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 77537280. Throughput: 0: 10422.3, 1: 10205.5. Samples: 77541376. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '973.380')] [2023-12-27 06:42:17,731][20347] Updated weights for policy 0, policy_version 151531 (0.0005) [2023-12-27 06:42:17,783][20347] Updated weights for policy 0, policy_version 151541 (0.0008) [2023-12-27 06:42:17,838][20347] Updated weights for policy 0, policy_version 151551 (0.0010) [2023-12-27 06:42:17,943][20381] Updated weights for policy 1, policy_version 151263 (0.0007) [2023-12-27 06:42:17,999][20381] Updated weights for policy 1, policy_version 151273 (0.0006) [2023-12-27 06:42:18,064][20381] Updated weights for policy 1, policy_version 151283 (0.0005) [2023-12-27 06:42:18,515][20347] Updated weights for policy 0, policy_version 151561 (0.0010) [2023-12-27 06:42:18,576][20347] Updated weights for policy 0, policy_version 151571 (0.0011) [2023-12-27 06:42:18,637][20347] Updated weights for policy 0, policy_version 151581 (0.0010) [2023-12-27 06:42:18,692][20381] Updated weights for policy 1, policy_version 151293 (0.0008) [2023-12-27 06:42:18,695][20347] Updated weights for policy 0, policy_version 151591 (0.0010) [2023-12-27 06:42:18,747][20381] Updated weights for policy 1, policy_version 151303 (0.0008) [2023-12-27 06:42:18,809][20381] Updated weights for policy 1, policy_version 151313 (0.0007) [2023-12-27 06:42:19,298][20347] Updated weights for policy 0, policy_version 151601 (0.0006) [2023-12-27 06:42:19,362][20347] Updated weights for policy 0, policy_version 151611 (0.0008) [2023-12-27 06:42:19,413][20347] Updated weights for policy 0, policy_version 151621 (0.0008) [2023-12-27 06:42:19,488][20292] KL-divergence is very high: 110.8829 [2023-12-27 06:42:19,500][20381] Updated weights for policy 1, policy_version 151323 (0.0007) [2023-12-27 06:42:19,534][20292] KL-divergence is very high: 147.4748 [2023-12-27 06:42:19,540][20292] KL-divergence is very high: 136.2191 [2023-12-27 06:42:19,559][20381] Updated weights for policy 1, policy_version 151333 (0.0009) [2023-12-27 06:42:19,615][20381] Updated weights for policy 1, policy_version 151343 (0.0009) [2023-12-27 06:42:19,624][20292] KL-divergence is very high: 150.5970 [2023-12-27 06:42:19,630][20292] KL-divergence is very high: 103.6960 [2023-12-27 06:42:20,048][20347] Updated weights for policy 0, policy_version 151631 (0.0006) [2023-12-27 06:42:20,116][20347] Updated weights for policy 0, policy_version 151641 (0.0007) [2023-12-27 06:42:20,174][20347] Updated weights for policy 0, policy_version 151651 (0.0006) [2023-12-27 06:42:20,428][20381] Updated weights for policy 1, policy_version 151353 (0.0008) [2023-12-27 06:42:20,491][20381] Updated weights for policy 1, policy_version 151363 (0.0007) [2023-12-27 06:42:20,542][20292] KL-divergence is very high: 115.9012 [2023-12-27 06:42:20,546][20381] Updated weights for policy 1, policy_version 151373 (0.0006) [2023-12-27 06:42:20,573][20292] KL-divergence is very high: 111.0732 [2023-12-27 06:42:20,580][20292] KL-divergence is very high: 215.5974 [2023-12-27 06:42:20,609][20381] Updated weights for policy 1, policy_version 151383 (0.0007) [2023-12-27 06:42:20,814][20347] Updated weights for policy 0, policy_version 151661 (0.0006) [2023-12-27 06:42:20,883][20347] Updated weights for policy 0, policy_version 151671 (0.0005) [2023-12-27 06:42:20,939][20347] Updated weights for policy 0, policy_version 151681 (0.0007) [2023-12-27 06:42:21,207][20292] KL-divergence is very high: 114.8153 [2023-12-27 06:42:21,220][20292] KL-divergence is very high: 112.7117 [2023-12-27 06:42:21,239][20381] Updated weights for policy 1, policy_version 151393 (0.0006) [2023-12-27 06:42:21,305][20381] Updated weights for policy 1, policy_version 151403 (0.0007) [2023-12-27 06:42:21,347][20292] KL-divergence is very high: 106.4637 [2023-12-27 06:42:21,353][20292] KL-divergence is very high: 113.5281 [2023-12-27 06:42:21,374][20381] Updated weights for policy 1, policy_version 151413 (0.0009) [2023-12-27 06:42:21,679][20347] Updated weights for policy 0, policy_version 151691 (0.0008) [2023-12-27 06:42:21,740][20347] Updated weights for policy 0, policy_version 151701 (0.0008) [2023-12-27 06:42:21,802][20347] Updated weights for policy 0, policy_version 151711 (0.0007) [2023-12-27 06:42:22,025][20381] Updated weights for policy 1, policy_version 151423 (0.0007) [2023-12-27 06:42:22,078][20381] Updated weights for policy 1, policy_version 151433 (0.0008) [2023-12-27 06:42:22,128][20292] KL-divergence is very high: 103.7859 [2023-12-27 06:42:22,133][20381] Updated weights for policy 1, policy_version 151443 (0.0007) [2023-12-27 06:42:22,134][20292] KL-divergence is very high: 113.9311 [2023-12-27 06:42:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 77635584. Throughput: 0: 10447.9, 1: 10172.5. Samples: 77604272. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '274.010')] [2023-12-27 06:42:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000151448_38780928.pth... [2023-12-27 06:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000151720_38854656.pth... [2023-12-27 06:42:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000150232_38469632.pth [2023-12-27 06:42:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000150504_38543360.pth [2023-12-27 06:42:22,552][20347] Updated weights for policy 0, policy_version 151721 (0.0008) [2023-12-27 06:42:22,600][20347] Updated weights for policy 0, policy_version 151731 (0.0008) [2023-12-27 06:42:22,652][20347] Updated weights for policy 0, policy_version 151741 (0.0007) [2023-12-27 06:42:22,711][20347] Updated weights for policy 0, policy_version 151751 (0.0007) [2023-12-27 06:42:22,793][20381] Updated weights for policy 1, policy_version 151453 (0.0009) [2023-12-27 06:42:22,846][20381] Updated weights for policy 1, policy_version 151463 (0.0010) [2023-12-27 06:42:22,898][20381] Updated weights for policy 1, policy_version 151473 (0.0010) [2023-12-27 06:42:23,323][20347] Updated weights for policy 0, policy_version 151761 (0.0005) [2023-12-27 06:42:23,376][20347] Updated weights for policy 0, policy_version 151771 (0.0006) [2023-12-27 06:42:23,424][20347] Updated weights for policy 0, policy_version 151781 (0.0007) [2023-12-27 06:42:23,592][20381] Updated weights for policy 1, policy_version 151483 (0.0011) [2023-12-27 06:42:23,642][20381] Updated weights for policy 1, policy_version 151493 (0.0010) [2023-12-27 06:42:23,694][20381] Updated weights for policy 1, policy_version 151503 (0.0010) [2023-12-27 06:42:23,991][20347] Updated weights for policy 0, policy_version 151791 (0.0007) [2023-12-27 06:42:24,047][20347] Updated weights for policy 0, policy_version 151801 (0.0007) [2023-12-27 06:42:24,109][20347] Updated weights for policy 0, policy_version 151811 (0.0005) [2023-12-27 06:42:24,391][20381] Updated weights for policy 1, policy_version 151513 (0.0011) [2023-12-27 06:42:24,448][20381] Updated weights for policy 1, policy_version 151523 (0.0008) [2023-12-27 06:42:24,503][20381] Updated weights for policy 1, policy_version 151533 (0.0006) [2023-12-27 06:42:24,555][20381] Updated weights for policy 1, policy_version 151543 (0.0010) [2023-12-27 06:42:24,739][20347] Updated weights for policy 0, policy_version 151821 (0.0006) [2023-12-27 06:42:24,801][20347] Updated weights for policy 0, policy_version 151831 (0.0007) [2023-12-27 06:42:24,860][20347] Updated weights for policy 0, policy_version 151841 (0.0007) [2023-12-27 06:42:25,227][20381] Updated weights for policy 1, policy_version 151553 (0.0010) [2023-12-27 06:42:25,275][20381] Updated weights for policy 1, policy_version 151563 (0.0010) [2023-12-27 06:42:25,323][20381] Updated weights for policy 1, policy_version 151573 (0.0010) [2023-12-27 06:42:25,543][20347] Updated weights for policy 0, policy_version 151851 (0.0008) [2023-12-27 06:42:25,618][20347] Updated weights for policy 0, policy_version 151861 (0.0007) [2023-12-27 06:42:25,668][20347] Updated weights for policy 0, policy_version 151871 (0.0007) [2023-12-27 06:42:25,978][20381] Updated weights for policy 1, policy_version 151583 (0.0009) [2023-12-27 06:42:26,030][20381] Updated weights for policy 1, policy_version 151593 (0.0007) [2023-12-27 06:42:26,087][20381] Updated weights for policy 1, policy_version 151603 (0.0007) [2023-12-27 06:42:26,370][20347] Updated weights for policy 0, policy_version 151881 (0.0008) [2023-12-27 06:42:26,415][20347] Updated weights for policy 0, policy_version 151891 (0.0009) [2023-12-27 06:42:26,465][20347] Updated weights for policy 0, policy_version 151901 (0.0008) [2023-12-27 06:42:26,513][20347] Updated weights for policy 0, policy_version 151911 (0.0009) [2023-12-27 06:42:26,683][20381] Updated weights for policy 1, policy_version 151613 (0.0008) [2023-12-27 06:42:26,741][20381] Updated weights for policy 1, policy_version 151623 (0.0007) [2023-12-27 06:42:26,791][20381] Updated weights for policy 1, policy_version 151633 (0.0008) [2023-12-27 06:42:27,233][20347] Updated weights for policy 0, policy_version 151921 (0.0007) [2023-12-27 06:42:27,280][20347] Updated weights for policy 0, policy_version 151931 (0.0007) [2023-12-27 06:42:27,334][20347] Updated weights for policy 0, policy_version 151941 (0.0008) [2023-12-27 06:42:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 77742080. Throughput: 0: 10448.3, 1: 10204.2. Samples: 77727840. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '328.460')] [2023-12-27 06:42:27,467][20381] Updated weights for policy 1, policy_version 151643 (0.0007) [2023-12-27 06:42:27,517][20381] Updated weights for policy 1, policy_version 151653 (0.0005) [2023-12-27 06:42:27,573][20381] Updated weights for policy 1, policy_version 151663 (0.0005) [2023-12-27 06:42:28,035][20347] Updated weights for policy 0, policy_version 151951 (0.0007) [2023-12-27 06:42:28,085][20347] Updated weights for policy 0, policy_version 151961 (0.0008) [2023-12-27 06:42:28,135][20347] Updated weights for policy 0, policy_version 151971 (0.0006) [2023-12-27 06:42:28,153][20381] Updated weights for policy 1, policy_version 151673 (0.0005) [2023-12-27 06:42:28,210][20381] Updated weights for policy 1, policy_version 151683 (0.0005) [2023-12-27 06:42:28,257][20381] Updated weights for policy 1, policy_version 151693 (0.0005) [2023-12-27 06:42:28,305][20381] Updated weights for policy 1, policy_version 151703 (0.0005) [2023-12-27 06:42:28,835][20381] Updated weights for policy 1, policy_version 151713 (0.0007) [2023-12-27 06:42:28,883][20381] Updated weights for policy 1, policy_version 151723 (0.0008) [2023-12-27 06:42:28,890][20347] Updated weights for policy 0, policy_version 151981 (0.0007) [2023-12-27 06:42:28,939][20381] Updated weights for policy 1, policy_version 151733 (0.0007) [2023-12-27 06:42:28,952][20347] Updated weights for policy 0, policy_version 151991 (0.0007) [2023-12-27 06:42:29,005][20347] Updated weights for policy 0, policy_version 152001 (0.0007) [2023-12-27 06:42:29,674][20381] Updated weights for policy 1, policy_version 151743 (0.0006) [2023-12-27 06:42:29,696][20347] Updated weights for policy 0, policy_version 152011 (0.0007) [2023-12-27 06:42:29,732][20381] Updated weights for policy 1, policy_version 151753 (0.0006) [2023-12-27 06:42:29,749][20347] Updated weights for policy 0, policy_version 152021 (0.0006) [2023-12-27 06:42:29,789][20381] Updated weights for policy 1, policy_version 151763 (0.0006) [2023-12-27 06:42:29,805][20347] Updated weights for policy 0, policy_version 152031 (0.0006) [2023-12-27 06:42:30,477][20381] Updated weights for policy 1, policy_version 151773 (0.0007) [2023-12-27 06:42:30,517][20347] Updated weights for policy 0, policy_version 152041 (0.0006) [2023-12-27 06:42:30,528][20381] Updated weights for policy 1, policy_version 151783 (0.0007) [2023-12-27 06:42:30,572][20347] Updated weights for policy 0, policy_version 152051 (0.0007) [2023-12-27 06:42:30,579][20381] Updated weights for policy 1, policy_version 151793 (0.0005) [2023-12-27 06:42:30,628][20347] Updated weights for policy 0, policy_version 152061 (0.0006) [2023-12-27 06:42:30,690][20347] Updated weights for policy 0, policy_version 152071 (0.0006) [2023-12-27 06:42:31,240][20381] Updated weights for policy 1, policy_version 151803 (0.0010) [2023-12-27 06:42:31,295][20381] Updated weights for policy 1, policy_version 151813 (0.0010) [2023-12-27 06:42:31,357][20381] Updated weights for policy 1, policy_version 151823 (0.0010) [2023-12-27 06:42:31,358][20347] Updated weights for policy 0, policy_version 152081 (0.0007) [2023-12-27 06:42:31,424][20347] Updated weights for policy 0, policy_version 152091 (0.0006) [2023-12-27 06:42:31,476][20347] Updated weights for policy 0, policy_version 152101 (0.0007) [2023-12-27 06:42:32,044][20381] Updated weights for policy 1, policy_version 151833 (0.0009) [2023-12-27 06:42:32,093][20381] Updated weights for policy 1, policy_version 151843 (0.0007) [2023-12-27 06:42:32,145][20381] Updated weights for policy 1, policy_version 151853 (0.0008) [2023-12-27 06:42:32,165][20347] Updated weights for policy 0, policy_version 152111 (0.0006) [2023-12-27 06:42:32,197][20381] Updated weights for policy 1, policy_version 151863 (0.0007) [2023-12-27 06:42:32,219][20347] Updated weights for policy 0, policy_version 152121 (0.0006) [2023-12-27 06:42:32,279][20347] Updated weights for policy 0, policy_version 152131 (0.0007) [2023-12-27 06:42:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 77848576. Throughput: 0: 10364.6, 1: 10336.6. Samples: 77852292. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '675.250')] [2023-12-27 06:42:32,897][20381] Updated weights for policy 1, policy_version 151873 (0.0007) [2023-12-27 06:42:32,957][20381] Updated weights for policy 1, policy_version 151883 (0.0005) [2023-12-27 06:42:32,974][20347] Updated weights for policy 0, policy_version 152141 (0.0007) [2023-12-27 06:42:33,009][20381] Updated weights for policy 1, policy_version 151893 (0.0005) [2023-12-27 06:42:33,023][20347] Updated weights for policy 0, policy_version 152151 (0.0007) [2023-12-27 06:42:33,067][20347] Updated weights for policy 0, policy_version 152161 (0.0008) [2023-12-27 06:42:33,571][20381] Updated weights for policy 1, policy_version 151903 (0.0009) [2023-12-27 06:42:33,628][20381] Updated weights for policy 1, policy_version 151913 (0.0010) [2023-12-27 06:42:33,676][20381] Updated weights for policy 1, policy_version 151923 (0.0010) [2023-12-27 06:42:33,830][20347] Updated weights for policy 0, policy_version 152171 (0.0008) [2023-12-27 06:42:33,887][20347] Updated weights for policy 0, policy_version 152181 (0.0007) [2023-12-27 06:42:33,948][20347] Updated weights for policy 0, policy_version 152191 (0.0007) [2023-12-27 06:42:34,333][20381] Updated weights for policy 1, policy_version 151933 (0.0008) [2023-12-27 06:42:34,388][20381] Updated weights for policy 1, policy_version 151943 (0.0007) [2023-12-27 06:42:34,448][20381] Updated weights for policy 1, policy_version 151953 (0.0007) [2023-12-27 06:42:34,664][20347] Updated weights for policy 0, policy_version 152201 (0.0007) [2023-12-27 06:42:34,715][20347] Updated weights for policy 0, policy_version 152211 (0.0010) [2023-12-27 06:42:34,768][20347] Updated weights for policy 0, policy_version 152221 (0.0009) [2023-12-27 06:42:34,821][20347] Updated weights for policy 0, policy_version 152231 (0.0008) [2023-12-27 06:42:35,002][20381] Updated weights for policy 1, policy_version 151963 (0.0008) [2023-12-27 06:42:35,058][20381] Updated weights for policy 1, policy_version 151973 (0.0005) [2023-12-27 06:42:35,113][20381] Updated weights for policy 1, policy_version 151983 (0.0005) [2023-12-27 06:42:35,589][20347] Updated weights for policy 0, policy_version 152241 (0.0007) [2023-12-27 06:42:35,634][20347] Updated weights for policy 0, policy_version 152251 (0.0009) [2023-12-27 06:42:35,639][20381] Updated weights for policy 1, policy_version 151993 (0.0006) [2023-12-27 06:42:35,687][20347] Updated weights for policy 0, policy_version 152261 (0.0006) [2023-12-27 06:42:35,689][20381] Updated weights for policy 1, policy_version 152003 (0.0006) [2023-12-27 06:42:35,744][20381] Updated weights for policy 1, policy_version 152013 (0.0007) [2023-12-27 06:42:35,797][20381] Updated weights for policy 1, policy_version 152023 (0.0007) [2023-12-27 06:42:36,411][20381] Updated weights for policy 1, policy_version 152033 (0.0008) [2023-12-27 06:42:36,444][20347] Updated weights for policy 0, policy_version 152271 (0.0006) [2023-12-27 06:42:36,470][20381] Updated weights for policy 1, policy_version 152043 (0.0006) [2023-12-27 06:42:36,501][20347] Updated weights for policy 0, policy_version 152281 (0.0007) [2023-12-27 06:42:36,528][20381] Updated weights for policy 1, policy_version 152053 (0.0008) [2023-12-27 06:42:36,558][20347] Updated weights for policy 0, policy_version 152291 (0.0009) [2023-12-27 06:42:37,225][20381] Updated weights for policy 1, policy_version 152063 (0.0008) [2023-12-27 06:42:37,231][20347] Updated weights for policy 0, policy_version 152301 (0.0006) [2023-12-27 06:42:37,282][20381] Updated weights for policy 1, policy_version 152073 (0.0005) [2023-12-27 06:42:37,283][20347] Updated weights for policy 0, policy_version 152311 (0.0010) [2023-12-27 06:42:37,328][20381] Updated weights for policy 1, policy_version 152083 (0.0005) [2023-12-27 06:42:37,329][20347] Updated weights for policy 0, policy_version 152321 (0.0010) [2023-12-27 06:42:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 77955072. Throughput: 0: 10290.0, 1: 10404.6. Samples: 77915276. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '624.960')] [2023-12-27 06:42:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000152088_38944768.pth... [2023-12-27 06:42:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000152328_39010304.pth... [2023-12-27 06:42:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000151112_38699008.pth [2023-12-27 06:42:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000150840_38625280.pth [2023-12-27 06:42:37,943][20347] Updated weights for policy 0, policy_version 152331 (0.0010) [2023-12-27 06:42:37,986][20381] Updated weights for policy 1, policy_version 152093 (0.0005) [2023-12-27 06:42:37,995][20347] Updated weights for policy 0, policy_version 152341 (0.0009) [2023-12-27 06:42:38,038][20381] Updated weights for policy 1, policy_version 152103 (0.0005) [2023-12-27 06:42:38,055][20347] Updated weights for policy 0, policy_version 152351 (0.0006) [2023-12-27 06:42:38,086][20381] Updated weights for policy 1, policy_version 152113 (0.0008) [2023-12-27 06:42:38,671][20347] Updated weights for policy 0, policy_version 152361 (0.0008) [2023-12-27 06:42:38,720][20347] Updated weights for policy 0, policy_version 152371 (0.0007) [2023-12-27 06:42:38,774][20347] Updated weights for policy 0, policy_version 152381 (0.0008) [2023-12-27 06:42:38,778][20381] Updated weights for policy 1, policy_version 152123 (0.0009) [2023-12-27 06:42:38,833][20347] Updated weights for policy 0, policy_version 152391 (0.0006) [2023-12-27 06:42:38,839][20381] Updated weights for policy 1, policy_version 152133 (0.0007) [2023-12-27 06:42:38,889][20381] Updated weights for policy 1, policy_version 152143 (0.0008) [2023-12-27 06:42:39,563][20381] Updated weights for policy 1, policy_version 152153 (0.0007) [2023-12-27 06:42:39,608][20347] Updated weights for policy 0, policy_version 152401 (0.0008) [2023-12-27 06:42:39,619][20381] Updated weights for policy 1, policy_version 152163 (0.0006) [2023-12-27 06:42:39,670][20381] Updated weights for policy 1, policy_version 152173 (0.0009) [2023-12-27 06:42:39,672][20347] Updated weights for policy 0, policy_version 152411 (0.0006) [2023-12-27 06:42:39,725][20381] Updated weights for policy 1, policy_version 152183 (0.0008) [2023-12-27 06:42:39,736][20347] Updated weights for policy 0, policy_version 152421 (0.0006) [2023-12-27 06:42:40,374][20347] Updated weights for policy 0, policy_version 152431 (0.0009) [2023-12-27 06:42:40,416][20381] Updated weights for policy 1, policy_version 152193 (0.0010) [2023-12-27 06:42:40,431][20347] Updated weights for policy 0, policy_version 152441 (0.0008) [2023-12-27 06:42:40,478][20381] Updated weights for policy 1, policy_version 152203 (0.0011) [2023-12-27 06:42:40,488][20347] Updated weights for policy 0, policy_version 152451 (0.0008) [2023-12-27 06:42:40,542][20381] Updated weights for policy 1, policy_version 152213 (0.0011) [2023-12-27 06:42:41,144][20347] Updated weights for policy 0, policy_version 152461 (0.0008) [2023-12-27 06:42:41,198][20381] Updated weights for policy 1, policy_version 152223 (0.0011) [2023-12-27 06:42:41,198][20347] Updated weights for policy 0, policy_version 152471 (0.0009) [2023-12-27 06:42:41,261][20347] Updated weights for policy 0, policy_version 152481 (0.0011) [2023-12-27 06:42:41,263][20381] Updated weights for policy 1, policy_version 152233 (0.0010) [2023-12-27 06:42:41,321][20381] Updated weights for policy 1, policy_version 152243 (0.0011) [2023-12-27 06:42:41,983][20347] Updated weights for policy 0, policy_version 152491 (0.0010) [2023-12-27 06:42:41,990][20381] Updated weights for policy 1, policy_version 152253 (0.0008) [2023-12-27 06:42:42,044][20381] Updated weights for policy 1, policy_version 152263 (0.0006) [2023-12-27 06:42:42,045][20347] Updated weights for policy 0, policy_version 152501 (0.0010) [2023-12-27 06:42:42,108][20381] Updated weights for policy 1, policy_version 152273 (0.0007) [2023-12-27 06:42:42,113][20347] Updated weights for policy 0, policy_version 152511 (0.0009) [2023-12-27 06:42:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 78053376. Throughput: 0: 10249.9, 1: 10423.8. Samples: 78039596. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '595.420')] [2023-12-27 06:42:42,772][20381] Updated weights for policy 1, policy_version 152283 (0.0007) [2023-12-27 06:42:42,825][20381] Updated weights for policy 1, policy_version 152293 (0.0007) [2023-12-27 06:42:42,830][20347] Updated weights for policy 0, policy_version 152521 (0.0008) [2023-12-27 06:42:42,877][20381] Updated weights for policy 1, policy_version 152303 (0.0009) [2023-12-27 06:42:42,882][20347] Updated weights for policy 0, policy_version 152531 (0.0005) [2023-12-27 06:42:42,931][20347] Updated weights for policy 0, policy_version 152541 (0.0005) [2023-12-27 06:42:42,985][20347] Updated weights for policy 0, policy_version 152551 (0.0007) [2023-12-27 06:42:43,560][20381] Updated weights for policy 1, policy_version 152313 (0.0010) [2023-12-27 06:42:43,614][20381] Updated weights for policy 1, policy_version 152323 (0.0010) [2023-12-27 06:42:43,669][20381] Updated weights for policy 1, policy_version 152333 (0.0010) [2023-12-27 06:42:43,678][20347] Updated weights for policy 0, policy_version 152561 (0.0006) [2023-12-27 06:42:43,724][20381] Updated weights for policy 1, policy_version 152343 (0.0010) [2023-12-27 06:42:43,731][20347] Updated weights for policy 0, policy_version 152571 (0.0005) [2023-12-27 06:42:43,783][20347] Updated weights for policy 0, policy_version 152581 (0.0005) [2023-12-27 06:42:44,387][20381] Updated weights for policy 1, policy_version 152353 (0.0010) [2023-12-27 06:42:44,437][20347] Updated weights for policy 0, policy_version 152591 (0.0006) [2023-12-27 06:42:44,450][20381] Updated weights for policy 1, policy_version 152363 (0.0011) [2023-12-27 06:42:44,485][20347] Updated weights for policy 0, policy_version 152601 (0.0005) [2023-12-27 06:42:44,505][20381] Updated weights for policy 1, policy_version 152373 (0.0011) [2023-12-27 06:42:44,544][20347] Updated weights for policy 0, policy_version 152611 (0.0006) [2023-12-27 06:42:45,173][20347] Updated weights for policy 0, policy_version 152621 (0.0007) [2023-12-27 06:42:45,222][20381] Updated weights for policy 1, policy_version 152383 (0.0008) [2023-12-27 06:42:45,230][20347] Updated weights for policy 0, policy_version 152631 (0.0007) [2023-12-27 06:42:45,281][20381] Updated weights for policy 1, policy_version 152393 (0.0010) [2023-12-27 06:42:45,295][20347] Updated weights for policy 0, policy_version 152641 (0.0006) [2023-12-27 06:42:45,340][20381] Updated weights for policy 1, policy_version 152403 (0.0011) [2023-12-27 06:42:45,856][20347] Updated weights for policy 0, policy_version 152651 (0.0007) [2023-12-27 06:42:45,907][20347] Updated weights for policy 0, policy_version 152661 (0.0010) [2023-12-27 06:42:45,959][20347] Updated weights for policy 0, policy_version 152671 (0.0010) [2023-12-27 06:42:46,000][20381] Updated weights for policy 1, policy_version 152413 (0.0009) [2023-12-27 06:42:46,055][20381] Updated weights for policy 1, policy_version 152423 (0.0011) [2023-12-27 06:42:46,113][20381] Updated weights for policy 1, policy_version 152433 (0.0011) [2023-12-27 06:42:46,610][20347] Updated weights for policy 0, policy_version 152681 (0.0010) [2023-12-27 06:42:46,673][20347] Updated weights for policy 0, policy_version 152691 (0.0005) [2023-12-27 06:42:46,702][20381] Updated weights for policy 1, policy_version 152443 (0.0009) [2023-12-27 06:42:46,734][20347] Updated weights for policy 0, policy_version 152701 (0.0005) [2023-12-27 06:42:46,752][20381] Updated weights for policy 1, policy_version 152453 (0.0008) [2023-12-27 06:42:46,791][20347] Updated weights for policy 0, policy_version 152711 (0.0010) [2023-12-27 06:42:46,805][20381] Updated weights for policy 1, policy_version 152463 (0.0008) [2023-12-27 06:42:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 78151680. Throughput: 0: 10311.8, 1: 10464.9. Samples: 78164996. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) [2023-12-27 06:42:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '380.870')] [2023-12-27 06:42:47,405][20381] Updated weights for policy 1, policy_version 152473 (0.0007) [2023-12-27 06:42:47,428][20347] Updated weights for policy 0, policy_version 152721 (0.0005) [2023-12-27 06:42:47,456][20381] Updated weights for policy 1, policy_version 152483 (0.0009) [2023-12-27 06:42:47,477][20347] Updated weights for policy 0, policy_version 152731 (0.0009) [2023-12-27 06:42:47,504][20381] Updated weights for policy 1, policy_version 152493 (0.0006) [2023-12-27 06:42:47,538][20347] Updated weights for policy 0, policy_version 152741 (0.0010) [2023-12-27 06:42:47,561][20381] Updated weights for policy 1, policy_version 152503 (0.0005) [2023-12-27 06:42:48,182][20381] Updated weights for policy 1, policy_version 152513 (0.0006) [2023-12-27 06:42:48,186][20347] Updated weights for policy 0, policy_version 152751 (0.0010) [2023-12-27 06:42:48,237][20347] Updated weights for policy 0, policy_version 152761 (0.0010) [2023-12-27 06:42:48,240][20381] Updated weights for policy 1, policy_version 152523 (0.0006) [2023-12-27 06:42:48,285][20347] Updated weights for policy 0, policy_version 152771 (0.0010) [2023-12-27 06:42:48,297][20381] Updated weights for policy 1, policy_version 152533 (0.0006) [2023-12-27 06:42:48,958][20381] Updated weights for policy 1, policy_version 152543 (0.0007) [2023-12-27 06:42:48,965][20347] Updated weights for policy 0, policy_version 152781 (0.0010) [2023-12-27 06:42:49,008][20381] Updated weights for policy 1, policy_version 152553 (0.0011) [2023-12-27 06:42:49,023][20347] Updated weights for policy 0, policy_version 152791 (0.0010) [2023-12-27 06:42:49,063][20381] Updated weights for policy 1, policy_version 152563 (0.0010) [2023-12-27 06:42:49,082][20347] Updated weights for policy 0, policy_version 152801 (0.0010) [2023-12-27 06:42:49,663][20381] Updated weights for policy 1, policy_version 152573 (0.0011) [2023-12-27 06:42:49,728][20381] Updated weights for policy 1, policy_version 152583 (0.0009) [2023-12-27 06:42:49,767][20347] Updated weights for policy 0, policy_version 152811 (0.0010) [2023-12-27 06:42:49,784][20381] Updated weights for policy 1, policy_version 152593 (0.0005) [2023-12-27 06:42:49,827][20347] Updated weights for policy 0, policy_version 152821 (0.0010) [2023-12-27 06:42:49,887][20347] Updated weights for policy 0, policy_version 152831 (0.0006) [2023-12-27 06:42:50,396][20381] Updated weights for policy 1, policy_version 152603 (0.0006) [2023-12-27 06:42:50,447][20381] Updated weights for policy 1, policy_version 152613 (0.0006) [2023-12-27 06:42:50,495][20347] Updated weights for policy 0, policy_version 152841 (0.0007) [2023-12-27 06:42:50,497][20381] Updated weights for policy 1, policy_version 152623 (0.0008) [2023-12-27 06:42:50,543][20347] Updated weights for policy 0, policy_version 152851 (0.0010) [2023-12-27 06:42:50,610][20347] Updated weights for policy 0, policy_version 152861 (0.0008) [2023-12-27 06:42:50,675][20347] Updated weights for policy 0, policy_version 152871 (0.0009) [2023-12-27 06:42:51,068][20381] Updated weights for policy 1, policy_version 152633 (0.0006) [2023-12-27 06:42:51,123][20381] Updated weights for policy 1, policy_version 152643 (0.0008) [2023-12-27 06:42:51,176][20381] Updated weights for policy 1, policy_version 152653 (0.0008) [2023-12-27 06:42:51,229][20381] Updated weights for policy 1, policy_version 152663 (0.0007) [2023-12-27 06:42:51,302][20347] Updated weights for policy 0, policy_version 152881 (0.0008) [2023-12-27 06:42:51,366][20347] Updated weights for policy 0, policy_version 152891 (0.0008) [2023-12-27 06:42:51,423][20347] Updated weights for policy 0, policy_version 152901 (0.0007) [2023-12-27 06:42:52,002][20381] Updated weights for policy 1, policy_version 152673 (0.0007) [2023-12-27 06:42:52,055][20381] Updated weights for policy 1, policy_version 152683 (0.0007) [2023-12-27 06:42:52,065][20347] Updated weights for policy 0, policy_version 152911 (0.0007) [2023-12-27 06:42:52,104][20381] Updated weights for policy 1, policy_version 152693 (0.0006) [2023-12-27 06:42:52,110][20347] Updated weights for policy 0, policy_version 152921 (0.0005) [2023-12-27 06:42:52,159][20347] Updated weights for policy 0, policy_version 152931 (0.0007) [2023-12-27 06:42:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 78266368. Throughput: 0: 10320.0, 1: 10525.5. Samples: 78229236. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:42:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '245.970')] [2023-12-27 06:42:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000152696_39100416.pth... [2023-12-27 06:42:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000152936_39165952.pth... [2023-12-27 06:42:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000151448_38780928.pth [2023-12-27 06:42:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000151720_38854656.pth [2023-12-27 06:42:52,845][20381] Updated weights for policy 1, policy_version 152703 (0.0007) [2023-12-27 06:42:52,849][20347] Updated weights for policy 0, policy_version 152941 (0.0006) [2023-12-27 06:42:52,899][20347] Updated weights for policy 0, policy_version 152951 (0.0005) [2023-12-27 06:42:52,905][20381] Updated weights for policy 1, policy_version 152713 (0.0008) [2023-12-27 06:42:52,954][20347] Updated weights for policy 0, policy_version 152961 (0.0005) [2023-12-27 06:42:52,960][20381] Updated weights for policy 1, policy_version 152723 (0.0007) [2023-12-27 06:42:53,568][20381] Updated weights for policy 1, policy_version 152733 (0.0006) [2023-12-27 06:42:53,619][20381] Updated weights for policy 1, policy_version 152743 (0.0006) [2023-12-27 06:42:53,659][20347] Updated weights for policy 0, policy_version 152971 (0.0006) [2023-12-27 06:42:53,669][20381] Updated weights for policy 1, policy_version 152753 (0.0007) [2023-12-27 06:42:53,702][20347] Updated weights for policy 0, policy_version 152981 (0.0005) [2023-12-27 06:42:53,747][20347] Updated weights for policy 0, policy_version 152991 (0.0007) [2023-12-27 06:42:54,311][20381] Updated weights for policy 1, policy_version 152763 (0.0006) [2023-12-27 06:42:54,359][20381] Updated weights for policy 1, policy_version 152773 (0.0007) [2023-12-27 06:42:54,412][20347] Updated weights for policy 0, policy_version 153001 (0.0007) [2023-12-27 06:42:54,417][20381] Updated weights for policy 1, policy_version 152783 (0.0006) [2023-12-27 06:42:54,467][20347] Updated weights for policy 0, policy_version 153011 (0.0006) [2023-12-27 06:42:54,522][20347] Updated weights for policy 0, policy_version 153021 (0.0007) [2023-12-27 06:42:54,574][20347] Updated weights for policy 0, policy_version 153031 (0.0007) [2023-12-27 06:42:55,098][20381] Updated weights for policy 1, policy_version 152793 (0.0006) [2023-12-27 06:42:55,158][20381] Updated weights for policy 1, policy_version 152803 (0.0006) [2023-12-27 06:42:55,216][20381] Updated weights for policy 1, policy_version 152813 (0.0006) [2023-12-27 06:42:55,242][20347] Updated weights for policy 0, policy_version 153041 (0.0007) [2023-12-27 06:42:55,273][20381] Updated weights for policy 1, policy_version 152823 (0.0005) [2023-12-27 06:42:55,299][20347] Updated weights for policy 0, policy_version 153051 (0.0007) [2023-12-27 06:42:55,357][20347] Updated weights for policy 0, policy_version 153061 (0.0008) [2023-12-27 06:42:55,836][20381] Updated weights for policy 1, policy_version 152833 (0.0008) [2023-12-27 06:42:55,897][20381] Updated weights for policy 1, policy_version 152843 (0.0008) [2023-12-27 06:42:55,950][20381] Updated weights for policy 1, policy_version 152853 (0.0007) [2023-12-27 06:42:56,095][20347] Updated weights for policy 0, policy_version 153071 (0.0007) [2023-12-27 06:42:56,146][20347] Updated weights for policy 0, policy_version 153081 (0.0007) [2023-12-27 06:42:56,202][20347] Updated weights for policy 0, policy_version 153091 (0.0006) [2023-12-27 06:42:56,643][20381] Updated weights for policy 1, policy_version 152863 (0.0010) [2023-12-27 06:42:56,697][20381] Updated weights for policy 1, policy_version 152873 (0.0010) [2023-12-27 06:42:56,754][20381] Updated weights for policy 1, policy_version 152884 (0.0011) [2023-12-27 06:42:56,817][20347] Updated weights for policy 0, policy_version 153101 (0.0007) [2023-12-27 06:42:56,862][20347] Updated weights for policy 0, policy_version 153111 (0.0005) [2023-12-27 06:42:56,914][20347] Updated weights for policy 0, policy_version 153121 (0.0005) [2023-12-27 06:42:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 78364672. Throughput: 0: 10317.4, 1: 10545.7. Samples: 78355032. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:42:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '312.660')] [2023-12-27 06:42:57,420][20381] Updated weights for policy 1, policy_version 152894 (0.0007) [2023-12-27 06:42:57,465][20347] Updated weights for policy 0, policy_version 153131 (0.0006) [2023-12-27 06:42:57,475][20381] Updated weights for policy 1, policy_version 152904 (0.0007) [2023-12-27 06:42:57,529][20381] Updated weights for policy 1, policy_version 152914 (0.0009) [2023-12-27 06:42:57,529][20347] Updated weights for policy 0, policy_version 153141 (0.0010) [2023-12-27 06:42:57,582][20347] Updated weights for policy 0, policy_version 153151 (0.0010) [2023-12-27 06:42:58,104][20347] Updated weights for policy 0, policy_version 153161 (0.0008) [2023-12-27 06:42:58,162][20347] Updated weights for policy 0, policy_version 153171 (0.0006) [2023-12-27 06:42:58,175][20381] Updated weights for policy 1, policy_version 152924 (0.0007) [2023-12-27 06:42:58,221][20347] Updated weights for policy 0, policy_version 153181 (0.0008) [2023-12-27 06:42:58,232][20381] Updated weights for policy 1, policy_version 152934 (0.0008) [2023-12-27 06:42:58,280][20347] Updated weights for policy 0, policy_version 153191 (0.0008) [2023-12-27 06:42:58,288][20381] Updated weights for policy 1, policy_version 152944 (0.0006) [2023-12-27 06:42:59,013][20347] Updated weights for policy 0, policy_version 153201 (0.0007) [2023-12-27 06:42:59,057][20381] Updated weights for policy 1, policy_version 152954 (0.0008) [2023-12-27 06:42:59,069][20347] Updated weights for policy 0, policy_version 153211 (0.0007) [2023-12-27 06:42:59,113][20381] Updated weights for policy 1, policy_version 152964 (0.0008) [2023-12-27 06:42:59,130][20347] Updated weights for policy 0, policy_version 153221 (0.0006) [2023-12-27 06:42:59,169][20381] Updated weights for policy 1, policy_version 152974 (0.0009) [2023-12-27 06:42:59,230][20381] Updated weights for policy 1, policy_version 152984 (0.0007) [2023-12-27 06:42:59,860][20347] Updated weights for policy 0, policy_version 153231 (0.0008) [2023-12-27 06:42:59,920][20347] Updated weights for policy 0, policy_version 153241 (0.0011) [2023-12-27 06:42:59,955][20381] Updated weights for policy 1, policy_version 152994 (0.0006) [2023-12-27 06:42:59,973][20347] Updated weights for policy 0, policy_version 153251 (0.0010) [2023-12-27 06:43:00,009][20381] Updated weights for policy 1, policy_version 153004 (0.0006) [2023-12-27 06:43:00,065][20381] Updated weights for policy 1, policy_version 153014 (0.0008) [2023-12-27 06:43:00,606][20347] Updated weights for policy 0, policy_version 153261 (0.0008) [2023-12-27 06:43:00,651][20347] Updated weights for policy 0, policy_version 153271 (0.0010) [2023-12-27 06:43:00,698][20347] Updated weights for policy 0, policy_version 153281 (0.0010) [2023-12-27 06:43:00,718][20381] Updated weights for policy 1, policy_version 153024 (0.0006) [2023-12-27 06:43:00,763][20381] Updated weights for policy 1, policy_version 153034 (0.0005) [2023-12-27 06:43:00,813][20381] Updated weights for policy 1, policy_version 153044 (0.0005) [2023-12-27 06:43:01,345][20347] Updated weights for policy 0, policy_version 153291 (0.0010) [2023-12-27 06:43:01,404][20347] Updated weights for policy 0, policy_version 153301 (0.0008) [2023-12-27 06:43:01,455][20347] Updated weights for policy 0, policy_version 153311 (0.0005) [2023-12-27 06:43:01,472][20381] Updated weights for policy 1, policy_version 153054 (0.0007) [2023-12-27 06:43:01,526][20381] Updated weights for policy 1, policy_version 153064 (0.0007) [2023-12-27 06:43:01,579][20381] Updated weights for policy 1, policy_version 153074 (0.0008) [2023-12-27 06:43:02,155][20347] Updated weights for policy 0, policy_version 153321 (0.0009) [2023-12-27 06:43:02,198][20381] Updated weights for policy 1, policy_version 153084 (0.0007) [2023-12-27 06:43:02,205][20347] Updated weights for policy 0, policy_version 153331 (0.0007) [2023-12-27 06:43:02,254][20381] Updated weights for policy 1, policy_version 153094 (0.0005) [2023-12-27 06:43:02,257][20347] Updated weights for policy 0, policy_version 153341 (0.0007) [2023-12-27 06:43:02,304][20381] Updated weights for policy 1, policy_version 153104 (0.0006) [2023-12-27 06:43:02,313][20347] Updated weights for policy 0, policy_version 153351 (0.0006) [2023-12-27 06:43:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 78479360. Throughput: 0: 10335.8, 1: 10560.5. Samples: 78481712. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '324.080')] [2023-12-27 06:43:02,943][20347] Updated weights for policy 0, policy_version 153361 (0.0010) [2023-12-27 06:43:02,971][20381] Updated weights for policy 1, policy_version 153114 (0.0008) [2023-12-27 06:43:02,993][20347] Updated weights for policy 0, policy_version 153371 (0.0010) [2023-12-27 06:43:03,024][20381] Updated weights for policy 1, policy_version 153124 (0.0006) [2023-12-27 06:43:03,038][20347] Updated weights for policy 0, policy_version 153381 (0.0010) [2023-12-27 06:43:03,086][20381] Updated weights for policy 1, policy_version 153134 (0.0007) [2023-12-27 06:43:03,154][20381] Updated weights for policy 1, policy_version 153144 (0.0007) [2023-12-27 06:43:03,688][20347] Updated weights for policy 0, policy_version 153391 (0.0007) [2023-12-27 06:43:03,738][20381] Updated weights for policy 1, policy_version 153154 (0.0006) [2023-12-27 06:43:03,747][20347] Updated weights for policy 0, policy_version 153401 (0.0006) [2023-12-27 06:43:03,795][20381] Updated weights for policy 1, policy_version 153164 (0.0007) [2023-12-27 06:43:03,803][20347] Updated weights for policy 0, policy_version 153411 (0.0006) [2023-12-27 06:43:03,854][20381] Updated weights for policy 1, policy_version 153174 (0.0008) [2023-12-27 06:43:04,470][20381] Updated weights for policy 1, policy_version 153184 (0.0008) [2023-12-27 06:43:04,484][20347] Updated weights for policy 0, policy_version 153421 (0.0009) [2023-12-27 06:43:04,525][20381] Updated weights for policy 1, policy_version 153194 (0.0005) [2023-12-27 06:43:04,535][20347] Updated weights for policy 0, policy_version 153431 (0.0010) [2023-12-27 06:43:04,584][20381] Updated weights for policy 1, policy_version 153204 (0.0005) [2023-12-27 06:43:04,589][20347] Updated weights for policy 0, policy_version 153441 (0.0005) [2023-12-27 06:43:05,180][20381] Updated weights for policy 1, policy_version 153214 (0.0006) [2023-12-27 06:43:05,231][20381] Updated weights for policy 1, policy_version 153224 (0.0006) [2023-12-27 06:43:05,255][20347] Updated weights for policy 0, policy_version 153451 (0.0009) [2023-12-27 06:43:05,278][20381] Updated weights for policy 1, policy_version 153234 (0.0007) [2023-12-27 06:43:05,300][20347] Updated weights for policy 0, policy_version 153461 (0.0010) [2023-12-27 06:43:05,350][20347] Updated weights for policy 0, policy_version 153471 (0.0010) [2023-12-27 06:43:05,884][20381] Updated weights for policy 1, policy_version 153244 (0.0010) [2023-12-27 06:43:05,930][20381] Updated weights for policy 1, policy_version 153254 (0.0009) [2023-12-27 06:43:05,973][20381] Updated weights for policy 1, policy_version 153264 (0.0005) [2023-12-27 06:43:06,049][20347] Updated weights for policy 0, policy_version 153481 (0.0010) [2023-12-27 06:43:06,111][20347] Updated weights for policy 0, policy_version 153491 (0.0010) [2023-12-27 06:43:06,168][20347] Updated weights for policy 0, policy_version 153501 (0.0009) [2023-12-27 06:43:06,227][20347] Updated weights for policy 0, policy_version 153511 (0.0007) [2023-12-27 06:43:06,660][20381] Updated weights for policy 1, policy_version 153274 (0.0006) [2023-12-27 06:43:06,709][20381] Updated weights for policy 1, policy_version 153284 (0.0011) [2023-12-27 06:43:06,758][20381] Updated weights for policy 1, policy_version 153294 (0.0010) [2023-12-27 06:43:06,822][20381] Updated weights for policy 1, policy_version 153304 (0.0011) [2023-12-27 06:43:06,898][20347] Updated weights for policy 0, policy_version 153521 (0.0005) [2023-12-27 06:43:06,960][20347] Updated weights for policy 0, policy_version 153531 (0.0007) [2023-12-27 06:43:07,015][20347] Updated weights for policy 0, policy_version 153541 (0.0010) [2023-12-27 06:43:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 78577664. Throughput: 0: 10322.8, 1: 10614.2. Samples: 78546436. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '387.950')] [2023-12-27 06:43:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000153304_39256064.pth... [2023-12-27 06:43:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000153544_39321600.pth... [2023-12-27 06:43:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000152088_38944768.pth [2023-12-27 06:43:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000152328_39010304.pth [2023-12-27 06:43:07,489][20381] Updated weights for policy 1, policy_version 153314 (0.0011) [2023-12-27 06:43:07,548][20381] Updated weights for policy 1, policy_version 153324 (0.0006) [2023-12-27 06:43:07,610][20381] Updated weights for policy 1, policy_version 153334 (0.0005) [2023-12-27 06:43:07,641][20347] Updated weights for policy 0, policy_version 153551 (0.0007) [2023-12-27 06:43:07,700][20347] Updated weights for policy 0, policy_version 153561 (0.0005) [2023-12-27 06:43:07,762][20347] Updated weights for policy 0, policy_version 153571 (0.0008) [2023-12-27 06:43:08,160][20381] Updated weights for policy 1, policy_version 153344 (0.0010) [2023-12-27 06:43:08,213][20381] Updated weights for policy 1, policy_version 153354 (0.0010) [2023-12-27 06:43:08,275][20381] Updated weights for policy 1, policy_version 153364 (0.0011) [2023-12-27 06:43:08,414][20347] Updated weights for policy 0, policy_version 153581 (0.0010) [2023-12-27 06:43:08,476][20347] Updated weights for policy 0, policy_version 153591 (0.0010) [2023-12-27 06:43:08,527][20347] Updated weights for policy 0, policy_version 153601 (0.0010) [2023-12-27 06:43:08,962][20381] Updated weights for policy 1, policy_version 153374 (0.0011) [2023-12-27 06:43:09,019][20381] Updated weights for policy 1, policy_version 153384 (0.0010) [2023-12-27 06:43:09,080][20381] Updated weights for policy 1, policy_version 153394 (0.0011) [2023-12-27 06:43:09,210][20347] Updated weights for policy 0, policy_version 153611 (0.0011) [2023-12-27 06:43:09,269][20347] Updated weights for policy 0, policy_version 153621 (0.0010) [2023-12-27 06:43:09,320][20347] Updated weights for policy 0, policy_version 153631 (0.0008) [2023-12-27 06:43:09,760][20381] Updated weights for policy 1, policy_version 153404 (0.0011) [2023-12-27 06:43:09,823][20381] Updated weights for policy 1, policy_version 153414 (0.0011) [2023-12-27 06:43:09,885][20381] Updated weights for policy 1, policy_version 153424 (0.0011) [2023-12-27 06:43:10,084][20347] Updated weights for policy 0, policy_version 153641 (0.0008) [2023-12-27 06:43:10,144][20347] Updated weights for policy 0, policy_version 153651 (0.0011) [2023-12-27 06:43:10,201][20347] Updated weights for policy 0, policy_version 153661 (0.0008) [2023-12-27 06:43:10,260][20347] Updated weights for policy 0, policy_version 153671 (0.0007) [2023-12-27 06:43:10,582][20381] Updated weights for policy 1, policy_version 153434 (0.0010) [2023-12-27 06:43:10,630][20381] Updated weights for policy 1, policy_version 153444 (0.0009) [2023-12-27 06:43:10,681][20381] Updated weights for policy 1, policy_version 153454 (0.0008) [2023-12-27 06:43:10,731][20381] Updated weights for policy 1, policy_version 153464 (0.0008) [2023-12-27 06:43:10,935][20347] Updated weights for policy 0, policy_version 153681 (0.0006) [2023-12-27 06:43:10,997][20347] Updated weights for policy 0, policy_version 153691 (0.0005) [2023-12-27 06:43:11,057][20347] Updated weights for policy 0, policy_version 153701 (0.0007) [2023-12-27 06:43:11,436][20381] Updated weights for policy 1, policy_version 153474 (0.0008) [2023-12-27 06:43:11,497][20381] Updated weights for policy 1, policy_version 153484 (0.0008) [2023-12-27 06:43:11,555][20381] Updated weights for policy 1, policy_version 153494 (0.0007) [2023-12-27 06:43:11,791][20347] Updated weights for policy 0, policy_version 153711 (0.0007) [2023-12-27 06:43:11,842][20347] Updated weights for policy 0, policy_version 153721 (0.0010) [2023-12-27 06:43:11,896][20347] Updated weights for policy 0, policy_version 153731 (0.0009) [2023-12-27 06:43:12,251][20381] Updated weights for policy 1, policy_version 153504 (0.0007) [2023-12-27 06:43:12,312][20381] Updated weights for policy 1, policy_version 153514 (0.0006) [2023-12-27 06:43:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 78675968. Throughput: 0: 10313.4, 1: 10638.6. Samples: 78670680. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '413.970')] [2023-12-27 06:43:12,375][20381] Updated weights for policy 1, policy_version 153524 (0.0008) [2023-12-27 06:43:12,644][20347] Updated weights for policy 0, policy_version 153741 (0.0008) [2023-12-27 06:43:12,699][20347] Updated weights for policy 0, policy_version 153751 (0.0008) [2023-12-27 06:43:12,757][20347] Updated weights for policy 0, policy_version 153761 (0.0007) [2023-12-27 06:43:13,054][20381] Updated weights for policy 1, policy_version 153534 (0.0007) [2023-12-27 06:43:13,107][20381] Updated weights for policy 1, policy_version 153544 (0.0007) [2023-12-27 06:43:13,160][20381] Updated weights for policy 1, policy_version 153554 (0.0008) [2023-12-27 06:43:13,388][20347] Updated weights for policy 0, policy_version 153771 (0.0006) [2023-12-27 06:43:13,439][20347] Updated weights for policy 0, policy_version 153781 (0.0008) [2023-12-27 06:43:13,497][20347] Updated weights for policy 0, policy_version 153791 (0.0008) [2023-12-27 06:43:13,849][20381] Updated weights for policy 1, policy_version 153564 (0.0008) [2023-12-27 06:43:13,910][20381] Updated weights for policy 1, policy_version 153574 (0.0008) [2023-12-27 06:43:13,972][20381] Updated weights for policy 1, policy_version 153584 (0.0008) [2023-12-27 06:43:14,173][20347] Updated weights for policy 0, policy_version 153801 (0.0007) [2023-12-27 06:43:14,225][20347] Updated weights for policy 0, policy_version 153811 (0.0007) [2023-12-27 06:43:14,278][20347] Updated weights for policy 0, policy_version 153821 (0.0007) [2023-12-27 06:43:14,335][20347] Updated weights for policy 0, policy_version 153831 (0.0005) [2023-12-27 06:43:14,676][20381] Updated weights for policy 1, policy_version 153594 (0.0008) [2023-12-27 06:43:14,735][20381] Updated weights for policy 1, policy_version 153604 (0.0008) [2023-12-27 06:43:14,800][20381] Updated weights for policy 1, policy_version 153614 (0.0009) [2023-12-27 06:43:14,853][20381] Updated weights for policy 1, policy_version 153624 (0.0008) [2023-12-27 06:43:14,894][20347] Updated weights for policy 0, policy_version 153841 (0.0008) [2023-12-27 06:43:14,955][20347] Updated weights for policy 0, policy_version 153851 (0.0005) [2023-12-27 06:43:15,014][20347] Updated weights for policy 0, policy_version 153861 (0.0006) [2023-12-27 06:43:15,522][20381] Updated weights for policy 1, policy_version 153634 (0.0007) [2023-12-27 06:43:15,583][20381] Updated weights for policy 1, policy_version 153644 (0.0008) [2023-12-27 06:43:15,638][20381] Updated weights for policy 1, policy_version 153654 (0.0008) [2023-12-27 06:43:15,694][20347] Updated weights for policy 0, policy_version 153871 (0.0008) [2023-12-27 06:43:15,754][20347] Updated weights for policy 0, policy_version 153881 (0.0008) [2023-12-27 06:43:15,812][20347] Updated weights for policy 0, policy_version 153891 (0.0008) [2023-12-27 06:43:16,271][20381] Updated weights for policy 1, policy_version 153664 (0.0006) [2023-12-27 06:43:16,325][20381] Updated weights for policy 1, policy_version 153674 (0.0005) [2023-12-27 06:43:16,379][20381] Updated weights for policy 1, policy_version 153684 (0.0006) [2023-12-27 06:43:16,449][20347] Updated weights for policy 0, policy_version 153901 (0.0006) [2023-12-27 06:43:16,509][20347] Updated weights for policy 0, policy_version 153911 (0.0005) [2023-12-27 06:43:16,574][20347] Updated weights for policy 0, policy_version 153921 (0.0005) [2023-12-27 06:43:16,965][20381] Updated weights for policy 1, policy_version 153694 (0.0008) [2023-12-27 06:43:17,010][20381] Updated weights for policy 1, policy_version 153704 (0.0006) [2023-12-27 06:43:17,056][20381] Updated weights for policy 1, policy_version 153714 (0.0010) [2023-12-27 06:43:17,148][20347] Updated weights for policy 0, policy_version 153931 (0.0006) [2023-12-27 06:43:17,199][20347] Updated weights for policy 0, policy_version 153941 (0.0007) [2023-12-27 06:43:17,254][20347] Updated weights for policy 0, policy_version 153951 (0.0007) [2023-12-27 06:43:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 78790656. Throughput: 0: 10388.1, 1: 10576.3. Samples: 78795688. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '587.770')] [2023-12-27 06:43:17,676][20381] Updated weights for policy 1, policy_version 153724 (0.0008) [2023-12-27 06:43:17,729][20381] Updated weights for policy 1, policy_version 153734 (0.0005) [2023-12-27 06:43:17,786][20381] Updated weights for policy 1, policy_version 153744 (0.0005) [2023-12-27 06:43:18,011][20347] Updated weights for policy 0, policy_version 153961 (0.0007) [2023-12-27 06:43:18,077][20347] Updated weights for policy 0, policy_version 153971 (0.0008) [2023-12-27 06:43:18,136][20347] Updated weights for policy 0, policy_version 153981 (0.0008) [2023-12-27 06:43:18,187][20347] Updated weights for policy 0, policy_version 153991 (0.0007) [2023-12-27 06:43:18,361][20381] Updated weights for policy 1, policy_version 153754 (0.0006) [2023-12-27 06:43:18,422][20381] Updated weights for policy 1, policy_version 153764 (0.0008) [2023-12-27 06:43:18,482][20381] Updated weights for policy 1, policy_version 153774 (0.0008) [2023-12-27 06:43:18,541][20381] Updated weights for policy 1, policy_version 153784 (0.0007) [2023-12-27 06:43:18,899][20347] Updated weights for policy 0, policy_version 154001 (0.0008) [2023-12-27 06:43:18,962][20347] Updated weights for policy 0, policy_version 154011 (0.0007) [2023-12-27 06:43:19,028][20347] Updated weights for policy 0, policy_version 154021 (0.0007) [2023-12-27 06:43:19,281][20381] Updated weights for policy 1, policy_version 153794 (0.0008) [2023-12-27 06:43:19,340][20381] Updated weights for policy 1, policy_version 153804 (0.0008) [2023-12-27 06:43:19,403][20381] Updated weights for policy 1, policy_version 153814 (0.0007) [2023-12-27 06:43:19,683][20347] Updated weights for policy 0, policy_version 154031 (0.0008) [2023-12-27 06:43:19,739][20347] Updated weights for policy 0, policy_version 154041 (0.0007) [2023-12-27 06:43:19,791][20347] Updated weights for policy 0, policy_version 154051 (0.0008) [2023-12-27 06:43:20,110][20381] Updated weights for policy 1, policy_version 153824 (0.0008) [2023-12-27 06:43:20,167][20381] Updated weights for policy 1, policy_version 153834 (0.0008) [2023-12-27 06:43:20,224][20381] Updated weights for policy 1, policy_version 153844 (0.0007) [2023-12-27 06:43:20,480][20347] Updated weights for policy 0, policy_version 154061 (0.0007) [2023-12-27 06:43:20,539][20347] Updated weights for policy 0, policy_version 154071 (0.0005) [2023-12-27 06:43:20,602][20347] Updated weights for policy 0, policy_version 154081 (0.0006) [2023-12-27 06:43:20,976][20381] Updated weights for policy 1, policy_version 153854 (0.0008) [2023-12-27 06:43:21,031][20381] Updated weights for policy 1, policy_version 153864 (0.0008) [2023-12-27 06:43:21,092][20381] Updated weights for policy 1, policy_version 153874 (0.0008) [2023-12-27 06:43:21,249][20347] Updated weights for policy 0, policy_version 154091 (0.0006) [2023-12-27 06:43:21,311][20347] Updated weights for policy 0, policy_version 154101 (0.0007) [2023-12-27 06:43:21,375][20347] Updated weights for policy 0, policy_version 154111 (0.0009) [2023-12-27 06:43:21,908][20381] Updated weights for policy 1, policy_version 153884 (0.0008) [2023-12-27 06:43:21,964][20381] Updated weights for policy 1, policy_version 153894 (0.0008) [2023-12-27 06:43:22,027][20381] Updated weights for policy 1, policy_version 153904 (0.0008) [2023-12-27 06:43:22,069][20347] Updated weights for policy 0, policy_version 154121 (0.0007) [2023-12-27 06:43:22,140][20347] Updated weights for policy 0, policy_version 154131 (0.0008) [2023-12-27 06:43:22,202][20347] Updated weights for policy 0, policy_version 154141 (0.0008) [2023-12-27 06:43:22,251][20347] Updated weights for policy 0, policy_version 154151 (0.0008) [2023-12-27 06:43:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 78888960. Throughput: 0: 10394.2, 1: 10511.2. Samples: 78856028. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '601.150')] [2023-12-27 06:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000153912_39411712.pth... [2023-12-27 06:43:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000154152_39477248.pth... [2023-12-27 06:43:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000152936_39165952.pth [2023-12-27 06:43:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000152696_39100416.pth [2023-12-27 06:43:22,693][20381] Updated weights for policy 1, policy_version 153914 (0.0008) [2023-12-27 06:43:22,745][20381] Updated weights for policy 1, policy_version 153924 (0.0008) [2023-12-27 06:43:22,795][20381] Updated weights for policy 1, policy_version 153934 (0.0008) [2023-12-27 06:43:22,856][20381] Updated weights for policy 1, policy_version 153944 (0.0008) [2023-12-27 06:43:22,919][20291] KL-divergence is very high: 112.0121 [2023-12-27 06:43:22,924][20291] KL-divergence is very high: 139.0183 [2023-12-27 06:43:22,939][20347] Updated weights for policy 0, policy_version 154161 (0.0007) [2023-12-27 06:43:23,002][20347] Updated weights for policy 0, policy_version 154171 (0.0008) [2023-12-27 06:43:23,060][20347] Updated weights for policy 0, policy_version 154181 (0.0006) [2023-12-27 06:43:23,570][20381] Updated weights for policy 1, policy_version 153954 (0.0007) [2023-12-27 06:43:23,579][20291] KL-divergence is very high: 149.4989 [2023-12-27 06:43:23,588][20291] KL-divergence is very high: 191.1229 [2023-12-27 06:43:23,598][20291] KL-divergence is very high: 301.9505 [2023-12-27 06:43:23,605][20291] KL-divergence is very high: 250.0758 [2023-12-27 06:43:23,612][20291] KL-divergence is very high: 178.5738 [2023-12-27 06:43:23,625][20381] Updated weights for policy 1, policy_version 153964 (0.0008) [2023-12-27 06:43:23,625][20291] KL-divergence is very high: 102.6806 [2023-12-27 06:43:23,630][20347] Updated weights for policy 0, policy_version 154191 (0.0006) [2023-12-27 06:43:23,632][20291] KL-divergence is very high: 136.1816 [2023-12-27 06:43:23,640][20291] KL-divergence is very high: 123.1870 [2023-12-27 06:43:23,647][20291] KL-divergence is very high: 100.3411 [2023-12-27 06:43:23,654][20291] KL-divergence is very high: 168.2769 [2023-12-27 06:43:23,661][20291] KL-divergence is very high: 160.4966 [2023-12-27 06:43:23,668][20291] KL-divergence is very high: 130.8691 [2023-12-27 06:43:23,679][20381] Updated weights for policy 1, policy_version 153974 (0.0008) [2023-12-27 06:43:23,681][20291] KL-divergence is very high: 255.0370 [2023-12-27 06:43:23,694][20291] KL-divergence is very high: 253.0804 [2023-12-27 06:43:23,699][20347] Updated weights for policy 0, policy_version 154201 (0.0006) [2023-12-27 06:43:23,700][20291] KL-divergence is very high: 183.1567 [2023-12-27 06:43:23,706][20291] KL-divergence is very high: 145.2537 [2023-12-27 06:43:23,711][20291] KL-divergence is very high: 211.1081 [2023-12-27 06:43:23,716][20291] KL-divergence is very high: 468.2932 [2023-12-27 06:43:23,721][20291] KL-divergence is very high: 251.5767 [2023-12-27 06:43:23,726][20291] KL-divergence is very high: 350.0454 [2023-12-27 06:43:23,731][20291] KL-divergence is very high: 115.3990 [2023-12-27 06:43:23,736][20291] KL-divergence is very high: 153.5948 [2023-12-27 06:43:23,742][20291] KL-divergence is very high: 203.7360 [2023-12-27 06:43:23,747][20291] KL-divergence is very high: 452.7145 [2023-12-27 06:43:23,751][20347] Updated weights for policy 0, policy_version 154211 (0.0006) [2023-12-27 06:43:23,753][20291] KL-divergence is very high: 328.4789 [2023-12-27 06:43:23,758][20291] KL-divergence is very high: 404.4820 [2023-12-27 06:43:23,764][20291] KL-divergence is very high: 186.0364 [2023-12-27 06:43:23,769][20291] KL-divergence is very high: 111.6172 [2023-12-27 06:43:24,352][20291] KL-divergence is very high: 136.2420 [2023-12-27 06:43:24,357][20347] Updated weights for policy 0, policy_version 154221 (0.0007) [2023-12-27 06:43:24,380][20291] KL-divergence is very high: 160.1176 [2023-12-27 06:43:24,398][20291] KL-divergence is very high: 151.4594 [2023-12-27 06:43:24,409][20381] Updated weights for policy 1, policy_version 153984 (0.0007) [2023-12-27 06:43:24,414][20347] Updated weights for policy 0, policy_version 154231 (0.0007) [2023-12-27 06:43:24,466][20347] Updated weights for policy 0, policy_version 154241 (0.0006) [2023-12-27 06:43:24,468][20381] Updated weights for policy 1, policy_version 153994 (0.0006) [2023-12-27 06:43:24,524][20381] Updated weights for policy 1, policy_version 154004 (0.0007) [2023-12-27 06:43:25,115][20347] Updated weights for policy 0, policy_version 154251 (0.0007) [2023-12-27 06:43:25,154][20291] KL-divergence is very high: 116.0310 [2023-12-27 06:43:25,163][20347] Updated weights for policy 0, policy_version 154261 (0.0005) [2023-12-27 06:43:25,220][20347] Updated weights for policy 0, policy_version 154271 (0.0010) [2023-12-27 06:43:25,238][20381] Updated weights for policy 1, policy_version 154014 (0.0007) [2023-12-27 06:43:25,293][20381] Updated weights for policy 1, policy_version 154024 (0.0007) [2023-12-27 06:43:25,346][20381] Updated weights for policy 1, policy_version 154034 (0.0007) [2023-12-27 06:43:25,809][20347] Updated weights for policy 0, policy_version 154281 (0.0010) [2023-12-27 06:43:25,863][20347] Updated weights for policy 0, policy_version 154291 (0.0010) [2023-12-27 06:43:25,916][20347] Updated weights for policy 0, policy_version 154301 (0.0010) [2023-12-27 06:43:25,966][20347] Updated weights for policy 0, policy_version 154311 (0.0008) [2023-12-27 06:43:26,067][20381] Updated weights for policy 1, policy_version 154044 (0.0008) [2023-12-27 06:43:26,118][20381] Updated weights for policy 1, policy_version 154054 (0.0009) [2023-12-27 06:43:26,188][20381] Updated weights for policy 1, policy_version 154064 (0.0007) [2023-12-27 06:43:26,563][20347] Updated weights for policy 0, policy_version 154321 (0.0006) [2023-12-27 06:43:26,614][20347] Updated weights for policy 0, policy_version 154331 (0.0006) [2023-12-27 06:43:26,626][20291] KL-divergence is very high: 131.2015 [2023-12-27 06:43:26,632][20291] KL-divergence is very high: 561.7503 [2023-12-27 06:43:26,637][20291] KL-divergence is very high: 479.3776 [2023-12-27 06:43:26,674][20347] Updated weights for policy 0, policy_version 154341 (0.0005) [2023-12-27 06:43:26,885][20381] Updated weights for policy 1, policy_version 154074 (0.0007) [2023-12-27 06:43:26,941][20381] Updated weights for policy 1, policy_version 154084 (0.0008) [2023-12-27 06:43:26,997][20381] Updated weights for policy 1, policy_version 154094 (0.0008) [2023-12-27 06:43:27,050][20381] Updated weights for policy 1, policy_version 154104 (0.0008) [2023-12-27 06:43:27,262][20291] KL-divergence is very high: 646.2632 [2023-12-27 06:43:27,267][20291] KL-divergence is very high: 127.8050 [2023-12-27 06:43:27,272][20291] KL-divergence is very high: 158.9062 [2023-12-27 06:43:27,278][20291] KL-divergence is very high: 240.7091 [2023-12-27 06:43:27,289][20347] Updated weights for policy 0, policy_version 154351 (0.0006) [2023-12-27 06:43:27,303][20291] KL-divergence is very high: 233.7749 [2023-12-27 06:43:27,314][20291] KL-divergence is very high: 126.1969 [2023-12-27 06:43:27,320][20291] KL-divergence is very high: 199.2769 [2023-12-27 06:43:27,346][20347] Updated weights for policy 0, policy_version 154361 (0.0006) [2023-12-27 06:43:27,351][20291] KL-divergence is very high: 570.2434 [2023-12-27 06:43:27,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 78987264. Throughput: 0: 10467.5, 1: 10397.5. Samples: 78978524. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:27,363][19170] Avg episode reward: [(0, '714.730'), (1, '594.890')] [2023-12-27 06:43:27,398][20291] KL-divergence is very high: 744.8844 [2023-12-27 06:43:27,403][20347] Updated weights for policy 0, policy_version 154371 (0.0007) [2023-12-27 06:43:27,409][20291] KL-divergence is very high: 170.6464 [2023-12-27 06:43:27,783][20381] Updated weights for policy 1, policy_version 154114 (0.0008) [2023-12-27 06:43:27,830][20381] Updated weights for policy 1, policy_version 154124 (0.0010) [2023-12-27 06:43:27,878][20381] Updated weights for policy 1, policy_version 154134 (0.0006) [2023-12-27 06:43:28,033][20291] KL-divergence is very high: 173.3163 [2023-12-27 06:43:28,056][20347] Updated weights for policy 0, policy_version 154381 (0.0009) [2023-12-27 06:43:28,110][20347] Updated weights for policy 0, policy_version 154391 (0.0010) [2023-12-27 06:43:28,118][20291] KL-divergence is very high: 150.3844 [2023-12-27 06:43:28,132][20291] KL-divergence is very high: 200.4108 [2023-12-27 06:43:28,156][20291] KL-divergence is very high: 165.8636 [2023-12-27 06:43:28,157][20347] Updated weights for policy 0, policy_version 154401 (0.0010) [2023-12-27 06:43:28,499][20381] Updated weights for policy 1, policy_version 154144 (0.0010) [2023-12-27 06:43:28,555][20381] Updated weights for policy 1, policy_version 154154 (0.0011) [2023-12-27 06:43:28,614][20381] Updated weights for policy 1, policy_version 154164 (0.0011) [2023-12-27 06:43:28,820][20291] KL-divergence is very high: 105.9023 [2023-12-27 06:43:28,825][20291] KL-divergence is very high: 123.4998 [2023-12-27 06:43:28,832][20291] KL-divergence is very high: 120.7994 [2023-12-27 06:43:28,833][20347] Updated weights for policy 0, policy_version 154411 (0.0010) [2023-12-27 06:43:28,888][20347] Updated weights for policy 0, policy_version 154421 (0.0011) [2023-12-27 06:43:28,943][20347] Updated weights for policy 0, policy_version 154431 (0.0010) [2023-12-27 06:43:29,247][20381] Updated weights for policy 1, policy_version 154174 (0.0009) [2023-12-27 06:43:29,292][20381] Updated weights for policy 1, policy_version 154184 (0.0008) [2023-12-27 06:43:29,357][20381] Updated weights for policy 1, policy_version 154194 (0.0009) [2023-12-27 06:43:29,549][20347] Updated weights for policy 0, policy_version 154441 (0.0008) [2023-12-27 06:43:29,600][20291] KL-divergence is very high: 180.7289 [2023-12-27 06:43:29,605][20347] Updated weights for policy 0, policy_version 154451 (0.0005) [2023-12-27 06:43:29,623][20291] KL-divergence is very high: 356.9460 [2023-12-27 06:43:29,634][20291] KL-divergence is very high: 277.8769 [2023-12-27 06:43:29,639][20291] KL-divergence is very high: 220.3277 [2023-12-27 06:43:29,646][20291] KL-divergence is very high: 300.5490 [2023-12-27 06:43:29,663][20347] Updated weights for policy 0, policy_version 154461 (0.0006) [2023-12-27 06:43:29,717][20347] Updated weights for policy 0, policy_version 154471 (0.0007) [2023-12-27 06:43:30,067][20381] Updated weights for policy 1, policy_version 154204 (0.0009) [2023-12-27 06:43:30,119][20381] Updated weights for policy 1, policy_version 154214 (0.0010) [2023-12-27 06:43:30,178][20381] Updated weights for policy 1, policy_version 154224 (0.0010) [2023-12-27 06:43:30,326][20347] Updated weights for policy 0, policy_version 154481 (0.0006) [2023-12-27 06:43:30,381][20347] Updated weights for policy 0, policy_version 154491 (0.0008) [2023-12-27 06:43:30,430][20347] Updated weights for policy 0, policy_version 154501 (0.0007) [2023-12-27 06:43:30,802][20381] Updated weights for policy 1, policy_version 154234 (0.0009) [2023-12-27 06:43:30,858][20381] Updated weights for policy 1, policy_version 154244 (0.0005) [2023-12-27 06:43:30,915][20381] Updated weights for policy 1, policy_version 154254 (0.0005) [2023-12-27 06:43:30,972][20381] Updated weights for policy 1, policy_version 154264 (0.0006) [2023-12-27 06:43:31,172][20347] Updated weights for policy 0, policy_version 154511 (0.0008) [2023-12-27 06:43:31,222][20347] Updated weights for policy 0, policy_version 154521 (0.0007) [2023-12-27 06:43:31,280][20347] Updated weights for policy 0, policy_version 154531 (0.0007) [2023-12-27 06:43:31,602][20381] Updated weights for policy 1, policy_version 154274 (0.0008) [2023-12-27 06:43:31,664][20381] Updated weights for policy 1, policy_version 154284 (0.0007) [2023-12-27 06:43:31,729][20381] Updated weights for policy 1, policy_version 154294 (0.0007) [2023-12-27 06:43:31,986][20347] Updated weights for policy 0, policy_version 154541 (0.0007) [2023-12-27 06:43:32,038][20347] Updated weights for policy 0, policy_version 154551 (0.0005) [2023-12-27 06:43:32,089][20347] Updated weights for policy 0, policy_version 154561 (0.0006) [2023-12-27 06:43:32,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 79093760. Throughput: 0: 10492.2, 1: 10409.4. Samples: 79105568. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:32,363][19170] Avg episode reward: [(0, '737.250'), (1, '781.730')] [2023-12-27 06:43:32,403][20381] Updated weights for policy 1, policy_version 154304 (0.0006) [2023-12-27 06:43:32,462][20381] Updated weights for policy 1, policy_version 154314 (0.0008) [2023-12-27 06:43:32,517][20381] Updated weights for policy 1, policy_version 154324 (0.0007) [2023-12-27 06:43:32,737][20347] Updated weights for policy 0, policy_version 154571 (0.0006) [2023-12-27 06:43:32,792][20347] Updated weights for policy 0, policy_version 154581 (0.0007) [2023-12-27 06:43:32,850][20347] Updated weights for policy 0, policy_version 154591 (0.0008) [2023-12-27 06:43:33,232][20381] Updated weights for policy 1, policy_version 154334 (0.0009) [2023-12-27 06:43:33,294][20381] Updated weights for policy 1, policy_version 154344 (0.0007) [2023-12-27 06:43:33,351][20381] Updated weights for policy 1, policy_version 154354 (0.0009) [2023-12-27 06:43:33,392][20347] Updated weights for policy 0, policy_version 154601 (0.0005) [2023-12-27 06:43:33,452][20347] Updated weights for policy 0, policy_version 154611 (0.0005) [2023-12-27 06:43:33,502][20347] Updated weights for policy 0, policy_version 154621 (0.0005) [2023-12-27 06:43:33,524][20291] KL-divergence is very high: 118.1287 [2023-12-27 06:43:33,542][20291] KL-divergence is very high: 102.9468 [2023-12-27 06:43:33,560][20347] Updated weights for policy 0, policy_version 154631 (0.0006) [2023-12-27 06:43:34,055][20381] Updated weights for policy 1, policy_version 154364 (0.0008) [2023-12-27 06:43:34,113][20381] Updated weights for policy 1, policy_version 154374 (0.0007) [2023-12-27 06:43:34,164][20347] Updated weights for policy 0, policy_version 154641 (0.0006) [2023-12-27 06:43:34,175][20381] Updated weights for policy 1, policy_version 154384 (0.0008) [2023-12-27 06:43:34,220][20347] Updated weights for policy 0, policy_version 154651 (0.0007) [2023-12-27 06:43:34,269][20347] Updated weights for policy 0, policy_version 154661 (0.0007) [2023-12-27 06:43:34,835][20381] Updated weights for policy 1, policy_version 154394 (0.0006) [2023-12-27 06:43:34,884][20381] Updated weights for policy 1, policy_version 154404 (0.0008) [2023-12-27 06:43:34,932][20381] Updated weights for policy 1, policy_version 154414 (0.0008) [2023-12-27 06:43:34,979][20381] Updated weights for policy 1, policy_version 154424 (0.0008) [2023-12-27 06:43:34,992][20347] Updated weights for policy 0, policy_version 154671 (0.0006) [2023-12-27 06:43:35,051][20347] Updated weights for policy 0, policy_version 154681 (0.0006) [2023-12-27 06:43:35,109][20347] Updated weights for policy 0, policy_version 154691 (0.0005) [2023-12-27 06:43:35,712][20381] Updated weights for policy 1, policy_version 154434 (0.0007) [2023-12-27 06:43:35,714][20347] Updated weights for policy 0, policy_version 154701 (0.0007) [2023-12-27 06:43:35,764][20381] Updated weights for policy 1, policy_version 154444 (0.0005) [2023-12-27 06:43:35,766][20347] Updated weights for policy 0, policy_version 154711 (0.0005) [2023-12-27 06:43:35,818][20381] Updated weights for policy 1, policy_version 154454 (0.0010) [2023-12-27 06:43:35,824][20347] Updated weights for policy 0, policy_version 154721 (0.0005) [2023-12-27 06:43:36,504][20347] Updated weights for policy 0, policy_version 154731 (0.0006) [2023-12-27 06:43:36,532][20381] Updated weights for policy 1, policy_version 154464 (0.0008) [2023-12-27 06:43:36,561][20347] Updated weights for policy 0, policy_version 154741 (0.0006) [2023-12-27 06:43:36,590][20381] Updated weights for policy 1, policy_version 154474 (0.0009) [2023-12-27 06:43:36,621][20347] Updated weights for policy 0, policy_version 154751 (0.0009) [2023-12-27 06:43:36,643][20381] Updated weights for policy 1, policy_version 154484 (0.0010) [2023-12-27 06:43:37,318][20347] Updated weights for policy 0, policy_version 154761 (0.0005) [2023-12-27 06:43:37,343][20381] Updated weights for policy 1, policy_version 154494 (0.0010) [2023-12-27 06:43:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 79192064. Throughput: 0: 10535.1, 1: 10330.2. Samples: 79168176. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:37,363][19170] Avg episode reward: [(0, '583.580'), (1, '941.430')] [2023-12-27 06:43:37,383][20347] Updated weights for policy 0, policy_version 154771 (0.0007) [2023-12-27 06:43:37,396][20381] Updated weights for policy 1, policy_version 154504 (0.0011) [2023-12-27 06:43:37,439][20347] Updated weights for policy 0, policy_version 154781 (0.0006) [2023-12-27 06:43:37,440][20291] KL-divergence is very high: 123.6214 [2023-12-27 06:43:37,456][20381] Updated weights for policy 1, policy_version 154514 (0.0011) [2023-12-27 06:43:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000154520_39567360.pth... [2023-12-27 06:43:37,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000153304_39256064.pth [2023-12-27 06:43:37,492][20291] KL-divergence is very high: 104.0073 [2023-12-27 06:43:37,499][20347] Updated weights for policy 0, policy_version 154791 (0.0005) [2023-12-27 06:43:37,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000154792_39641088.pth... [2023-12-27 06:43:37,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000153544_39321600.pth [2023-12-27 06:43:38,118][20381] Updated weights for policy 1, policy_version 154524 (0.0010) [2023-12-27 06:43:38,166][20381] Updated weights for policy 1, policy_version 154534 (0.0010) [2023-12-27 06:43:38,175][20291] KL-divergence is very high: 108.2841 [2023-12-27 06:43:38,198][20347] Updated weights for policy 0, policy_version 154801 (0.0006) [2023-12-27 06:43:38,198][20291] KL-divergence is very high: 108.3839 [2023-12-27 06:43:38,203][20291] KL-divergence is very high: 267.7455 [2023-12-27 06:43:38,213][20291] KL-divergence is very high: 441.7686 [2023-12-27 06:43:38,219][20381] Updated weights for policy 1, policy_version 154544 (0.0010) [2023-12-27 06:43:38,224][20291] KL-divergence is very high: 112.9807 [2023-12-27 06:43:38,248][20347] Updated weights for policy 0, policy_version 154811 (0.0006) [2023-12-27 06:43:38,284][20291] KL-divergence is very high: 206.6173 [2023-12-27 06:43:38,295][20291] KL-divergence is very high: 309.1355 [2023-12-27 06:43:38,300][20347] Updated weights for policy 0, policy_version 154821 (0.0006) [2023-12-27 06:43:38,307][20291] KL-divergence is very high: 117.2988 [2023-12-27 06:43:38,956][20291] KL-divergence is very high: 191.1595 [2023-12-27 06:43:38,958][20381] Updated weights for policy 1, policy_version 154554 (0.0010) [2023-12-27 06:43:38,963][20291] KL-divergence is very high: 162.5526 [2023-12-27 06:43:38,975][20291] KL-divergence is very high: 168.0232 [2023-12-27 06:43:38,998][20347] Updated weights for policy 0, policy_version 154831 (0.0008) [2023-12-27 06:43:39,004][20291] KL-divergence is very high: 169.3074 [2023-12-27 06:43:39,009][20291] KL-divergence is very high: 132.8815 [2023-12-27 06:43:39,012][20381] Updated weights for policy 1, policy_version 154564 (0.0011) [2023-12-27 06:43:39,021][20291] KL-divergence is very high: 298.4730 [2023-12-27 06:43:39,050][20291] KL-divergence is very high: 673.2722 [2023-12-27 06:43:39,054][20347] Updated weights for policy 0, policy_version 154841 (0.0009) [2023-12-27 06:43:39,056][20291] KL-divergence is very high: 411.0688 [2023-12-27 06:43:39,068][20381] Updated weights for policy 1, policy_version 154574 (0.0011) [2023-12-27 06:43:39,068][20291] KL-divergence is very high: 187.0463 [2023-12-27 06:43:39,103][20291] KL-divergence is very high: 193.8317 [2023-12-27 06:43:39,115][20347] Updated weights for policy 0, policy_version 154851 (0.0007) [2023-12-27 06:43:39,127][20381] Updated weights for policy 1, policy_version 154584 (0.0011) [2023-12-27 06:43:39,775][20347] Updated weights for policy 0, policy_version 154861 (0.0007) [2023-12-27 06:43:39,841][20347] Updated weights for policy 0, policy_version 154871 (0.0008) [2023-12-27 06:43:39,845][20381] Updated weights for policy 1, policy_version 154594 (0.0009) [2023-12-27 06:43:39,898][20291] KL-divergence is very high: 119.3424 [2023-12-27 06:43:39,904][20347] Updated weights for policy 0, policy_version 154881 (0.0006) [2023-12-27 06:43:39,910][20381] Updated weights for policy 1, policy_version 154604 (0.0011) [2023-12-27 06:43:39,974][20381] Updated weights for policy 1, policy_version 154614 (0.0010) [2023-12-27 06:43:40,613][20347] Updated weights for policy 0, policy_version 154891 (0.0007) [2023-12-27 06:43:40,618][20381] Updated weights for policy 1, policy_version 154624 (0.0011) [2023-12-27 06:43:40,668][20347] Updated weights for policy 0, policy_version 154901 (0.0006) [2023-12-27 06:43:40,673][20381] Updated weights for policy 1, policy_version 154634 (0.0008) [2023-12-27 06:43:40,726][20347] Updated weights for policy 0, policy_version 154911 (0.0007) [2023-12-27 06:43:40,728][20381] Updated weights for policy 1, policy_version 154644 (0.0007) [2023-12-27 06:43:41,355][20381] Updated weights for policy 1, policy_version 154654 (0.0009) [2023-12-27 06:43:41,422][20381] Updated weights for policy 1, policy_version 154664 (0.0008) [2023-12-27 06:43:41,457][20347] Updated weights for policy 0, policy_version 154921 (0.0010) [2023-12-27 06:43:41,481][20291] KL-divergence is very high: 102.3833 [2023-12-27 06:43:41,489][20381] Updated weights for policy 1, policy_version 154674 (0.0008) [2023-12-27 06:43:41,511][20291] KL-divergence is very high: 162.0124 [2023-12-27 06:43:41,517][20347] Updated weights for policy 0, policy_version 154931 (0.0007) [2023-12-27 06:43:41,527][20291] KL-divergence is very high: 159.0116 [2023-12-27 06:43:41,556][20291] KL-divergence is very high: 144.7674 [2023-12-27 06:43:41,573][20347] Updated weights for policy 0, policy_version 154941 (0.0008) [2023-12-27 06:43:41,575][20291] KL-divergence is very high: 116.4287 [2023-12-27 06:43:41,636][20347] Updated weights for policy 0, policy_version 154951 (0.0008) [2023-12-27 06:43:42,207][20381] Updated weights for policy 1, policy_version 154684 (0.0008) [2023-12-27 06:43:42,264][20381] Updated weights for policy 1, policy_version 154694 (0.0007) [2023-12-27 06:43:42,327][20381] Updated weights for policy 1, policy_version 154704 (0.0007) [2023-12-27 06:43:42,333][20347] Updated weights for policy 0, policy_version 154961 (0.0010) [2023-12-27 06:43:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 79290368. Throughput: 0: 10489.0, 1: 10275.3. Samples: 79289424. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:42,363][19170] Avg episode reward: [(0, '530.940'), (1, '967.540')] [2023-12-27 06:43:42,396][20347] Updated weights for policy 0, policy_version 154971 (0.0010) [2023-12-27 06:43:42,454][20347] Updated weights for policy 0, policy_version 154981 (0.0010) [2023-12-27 06:43:43,063][20381] Updated weights for policy 1, policy_version 154714 (0.0008) [2023-12-27 06:43:43,122][20381] Updated weights for policy 1, policy_version 154724 (0.0008) [2023-12-27 06:43:43,153][20347] Updated weights for policy 0, policy_version 154991 (0.0010) [2023-12-27 06:43:43,175][20381] Updated weights for policy 1, policy_version 154734 (0.0006) [2023-12-27 06:43:43,219][20347] Updated weights for policy 0, policy_version 155001 (0.0011) [2023-12-27 06:43:43,236][20381] Updated weights for policy 1, policy_version 154744 (0.0007) [2023-12-27 06:43:43,274][20347] Updated weights for policy 0, policy_version 155011 (0.0010) [2023-12-27 06:43:43,926][20347] Updated weights for policy 0, policy_version 155021 (0.0008) [2023-12-27 06:43:43,940][20381] Updated weights for policy 1, policy_version 154754 (0.0006) [2023-12-27 06:43:43,981][20347] Updated weights for policy 0, policy_version 155031 (0.0007) [2023-12-27 06:43:43,993][20381] Updated weights for policy 1, policy_version 154764 (0.0008) [2023-12-27 06:43:44,036][20347] Updated weights for policy 0, policy_version 155041 (0.0006) [2023-12-27 06:43:44,045][20381] Updated weights for policy 1, policy_version 154774 (0.0007) [2023-12-27 06:43:44,695][20381] Updated weights for policy 1, policy_version 154784 (0.0006) [2023-12-27 06:43:44,710][20347] Updated weights for policy 0, policy_version 155051 (0.0007) [2023-12-27 06:43:44,755][20381] Updated weights for policy 1, policy_version 154794 (0.0005) [2023-12-27 06:43:44,764][20347] Updated weights for policy 0, policy_version 155061 (0.0007) [2023-12-27 06:43:44,821][20381] Updated weights for policy 1, policy_version 154804 (0.0007) [2023-12-27 06:43:44,827][20347] Updated weights for policy 0, policy_version 155071 (0.0009) [2023-12-27 06:43:45,495][20381] Updated weights for policy 1, policy_version 154814 (0.0007) [2023-12-27 06:43:45,517][20347] Updated weights for policy 0, policy_version 155081 (0.0010) [2023-12-27 06:43:45,548][20381] Updated weights for policy 1, policy_version 154824 (0.0006) [2023-12-27 06:43:45,578][20347] Updated weights for policy 0, policy_version 155091 (0.0010) [2023-12-27 06:43:45,607][20381] Updated weights for policy 1, policy_version 154834 (0.0007) [2023-12-27 06:43:45,637][20347] Updated weights for policy 0, policy_version 155101 (0.0010) [2023-12-27 06:43:45,691][20347] Updated weights for policy 0, policy_version 155111 (0.0010) [2023-12-27 06:43:46,318][20347] Updated weights for policy 0, policy_version 155121 (0.0006) [2023-12-27 06:43:46,320][20381] Updated weights for policy 1, policy_version 154844 (0.0008) [2023-12-27 06:43:46,373][20347] Updated weights for policy 0, policy_version 155131 (0.0009) [2023-12-27 06:43:46,374][20381] Updated weights for policy 1, policy_version 154854 (0.0006) [2023-12-27 06:43:46,426][20381] Updated weights for policy 1, policy_version 154864 (0.0005) [2023-12-27 06:43:46,428][20347] Updated weights for policy 0, policy_version 155141 (0.0010) [2023-12-27 06:43:47,050][20347] Updated weights for policy 0, policy_version 155151 (0.0009) [2023-12-27 06:43:47,104][20347] Updated weights for policy 0, policy_version 155161 (0.0010) [2023-12-27 06:43:47,134][20381] Updated weights for policy 1, policy_version 154874 (0.0007) [2023-12-27 06:43:47,161][20347] Updated weights for policy 0, policy_version 155171 (0.0010) [2023-12-27 06:43:47,196][20381] Updated weights for policy 1, policy_version 154884 (0.0006) [2023-12-27 06:43:47,253][20381] Updated weights for policy 1, policy_version 154894 (0.0007) [2023-12-27 06:43:47,313][20381] Updated weights for policy 1, policy_version 154904 (0.0008) [2023-12-27 06:43:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 79405056. Throughput: 0: 10419.4, 1: 10210.3. Samples: 79410052. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:47,364][19170] Avg episode reward: [(0, '826.550'), (1, '998.680')] [2023-12-27 06:43:47,835][20347] Updated weights for policy 0, policy_version 155181 (0.0010) [2023-12-27 06:43:47,894][20347] Updated weights for policy 0, policy_version 155191 (0.0009) [2023-12-27 06:43:47,955][20347] Updated weights for policy 0, policy_version 155201 (0.0010) [2023-12-27 06:43:47,967][20381] Updated weights for policy 1, policy_version 154914 (0.0005) [2023-12-27 06:43:48,019][20381] Updated weights for policy 1, policy_version 154924 (0.0005) [2023-12-27 06:43:48,077][20381] Updated weights for policy 1, policy_version 154934 (0.0006) [2023-12-27 06:43:48,606][20347] Updated weights for policy 0, policy_version 155211 (0.0010) [2023-12-27 06:43:48,655][20347] Updated weights for policy 0, policy_version 155221 (0.0010) [2023-12-27 06:43:48,707][20347] Updated weights for policy 0, policy_version 155231 (0.0008) [2023-12-27 06:43:48,727][20381] Updated weights for policy 1, policy_version 154944 (0.0007) [2023-12-27 06:43:48,778][20381] Updated weights for policy 1, policy_version 154954 (0.0008) [2023-12-27 06:43:48,837][20381] Updated weights for policy 1, policy_version 154964 (0.0007) [2023-12-27 06:43:49,397][20347] Updated weights for policy 0, policy_version 155241 (0.0005) [2023-12-27 06:43:49,454][20347] Updated weights for policy 0, policy_version 155251 (0.0010) [2023-12-27 06:43:49,514][20347] Updated weights for policy 0, policy_version 155261 (0.0008) [2023-12-27 06:43:49,531][20381] Updated weights for policy 1, policy_version 154974 (0.0007) [2023-12-27 06:43:49,566][20347] Updated weights for policy 0, policy_version 155271 (0.0010) [2023-12-27 06:43:49,583][20381] Updated weights for policy 1, policy_version 154984 (0.0006) [2023-12-27 06:43:49,633][20381] Updated weights for policy 1, policy_version 154994 (0.0007) [2023-12-27 06:43:50,170][20347] Updated weights for policy 0, policy_version 155281 (0.0011) [2023-12-27 06:43:50,228][20347] Updated weights for policy 0, policy_version 155291 (0.0011) [2023-12-27 06:43:50,285][20347] Updated weights for policy 0, policy_version 155301 (0.0010) [2023-12-27 06:43:50,337][20381] Updated weights for policy 1, policy_version 155004 (0.0008) [2023-12-27 06:43:50,397][20381] Updated weights for policy 1, policy_version 155014 (0.0011) [2023-12-27 06:43:50,452][20381] Updated weights for policy 1, policy_version 155024 (0.0010) [2023-12-27 06:43:50,959][20347] Updated weights for policy 0, policy_version 155311 (0.0007) [2023-12-27 06:43:51,009][20347] Updated weights for policy 0, policy_version 155321 (0.0006) [2023-12-27 06:43:51,074][20347] Updated weights for policy 0, policy_version 155331 (0.0009) [2023-12-27 06:43:51,152][20381] Updated weights for policy 1, policy_version 155034 (0.0006) [2023-12-27 06:43:51,210][20381] Updated weights for policy 1, policy_version 155044 (0.0007) [2023-12-27 06:43:51,268][20381] Updated weights for policy 1, policy_version 155054 (0.0011) [2023-12-27 06:43:51,327][20381] Updated weights for policy 1, policy_version 155064 (0.0011) [2023-12-27 06:43:51,789][20347] Updated weights for policy 0, policy_version 155341 (0.0008) [2023-12-27 06:43:51,856][20347] Updated weights for policy 0, policy_version 155351 (0.0005) [2023-12-27 06:43:51,922][20347] Updated weights for policy 0, policy_version 155361 (0.0006) [2023-12-27 06:43:52,036][20381] Updated weights for policy 1, policy_version 155074 (0.0008) [2023-12-27 06:43:52,104][20381] Updated weights for policy 1, policy_version 155084 (0.0007) [2023-12-27 06:43:52,158][20381] Updated weights for policy 1, policy_version 155094 (0.0006) [2023-12-27 06:43:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 79503360. Throughput: 0: 10420.8, 1: 10156.1. Samples: 79472396. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:43:52,363][19170] Avg episode reward: [(0, '981.650'), (1, '1000.000')] [2023-12-27 06:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000155096_39714816.pth... [2023-12-27 06:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000155368_39788544.pth... [2023-12-27 06:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000153912_39411712.pth [2023-12-27 06:43:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000154152_39477248.pth [2023-12-27 06:43:52,606][20347] Updated weights for policy 0, policy_version 155371 (0.0008) [2023-12-27 06:43:52,660][20347] Updated weights for policy 0, policy_version 155381 (0.0007) [2023-12-27 06:43:52,727][20347] Updated weights for policy 0, policy_version 155391 (0.0006) [2023-12-27 06:43:52,752][20381] Updated weights for policy 1, policy_version 155104 (0.0009) [2023-12-27 06:43:52,811][20381] Updated weights for policy 1, policy_version 155114 (0.0010) [2023-12-27 06:43:52,872][20381] Updated weights for policy 1, policy_version 155124 (0.0010) [2023-12-27 06:43:53,406][20347] Updated weights for policy 0, policy_version 155401 (0.0006) [2023-12-27 06:43:53,457][20381] Updated weights for policy 1, policy_version 155134 (0.0007) [2023-12-27 06:43:53,459][20347] Updated weights for policy 0, policy_version 155411 (0.0005) [2023-12-27 06:43:53,507][20347] Updated weights for policy 0, policy_version 155421 (0.0006) [2023-12-27 06:43:53,508][20381] Updated weights for policy 1, policy_version 155144 (0.0005) [2023-12-27 06:43:53,556][20347] Updated weights for policy 0, policy_version 155431 (0.0008) [2023-12-27 06:43:53,556][20381] Updated weights for policy 1, policy_version 155154 (0.0007) [2023-12-27 06:43:54,098][20381] Updated weights for policy 1, policy_version 155164 (0.0007) [2023-12-27 06:43:54,148][20381] Updated weights for policy 1, policy_version 155174 (0.0007) [2023-12-27 06:43:54,202][20381] Updated weights for policy 1, policy_version 155184 (0.0007) [2023-12-27 06:43:54,234][20347] Updated weights for policy 0, policy_version 155441 (0.0006) [2023-12-27 06:43:54,282][20347] Updated weights for policy 0, policy_version 155451 (0.0007) [2023-12-27 06:43:54,337][20347] Updated weights for policy 0, policy_version 155461 (0.0007) [2023-12-27 06:43:54,894][20381] Updated weights for policy 1, policy_version 155194 (0.0006) [2023-12-27 06:43:54,951][20381] Updated weights for policy 1, policy_version 155204 (0.0007) [2023-12-27 06:43:54,975][20347] Updated weights for policy 0, policy_version 155471 (0.0009) [2023-12-27 06:43:55,006][20381] Updated weights for policy 1, policy_version 155214 (0.0009) [2023-12-27 06:43:55,031][20347] Updated weights for policy 0, policy_version 155481 (0.0010) [2023-12-27 06:43:55,056][20381] Updated weights for policy 1, policy_version 155224 (0.0005) [2023-12-27 06:43:55,083][20347] Updated weights for policy 0, policy_version 155491 (0.0010) [2023-12-27 06:43:55,698][20347] Updated weights for policy 0, policy_version 155501 (0.0008) [2023-12-27 06:43:55,750][20347] Updated weights for policy 0, policy_version 155511 (0.0005) [2023-12-27 06:43:55,752][20381] Updated weights for policy 1, policy_version 155234 (0.0007) [2023-12-27 06:43:55,804][20347] Updated weights for policy 0, policy_version 155521 (0.0005) [2023-12-27 06:43:55,806][20381] Updated weights for policy 1, policy_version 155244 (0.0006) [2023-12-27 06:43:55,862][20381] Updated weights for policy 1, policy_version 155254 (0.0006) [2023-12-27 06:43:56,442][20347] Updated weights for policy 0, policy_version 155531 (0.0006) [2023-12-27 06:43:56,490][20347] Updated weights for policy 0, policy_version 155541 (0.0007) [2023-12-27 06:43:56,541][20347] Updated weights for policy 0, policy_version 155551 (0.0007) [2023-12-27 06:43:56,569][20381] Updated weights for policy 1, policy_version 155264 (0.0006) [2023-12-27 06:43:56,621][20381] Updated weights for policy 1, policy_version 155274 (0.0006) [2023-12-27 06:43:56,671][20381] Updated weights for policy 1, policy_version 155284 (0.0007) [2023-12-27 06:43:57,228][20347] Updated weights for policy 0, policy_version 155561 (0.0005) [2023-12-27 06:43:57,286][20347] Updated weights for policy 0, policy_version 155571 (0.0005) [2023-12-27 06:43:57,335][20381] Updated weights for policy 1, policy_version 155294 (0.0006) [2023-12-27 06:43:57,338][20347] Updated weights for policy 0, policy_version 155581 (0.0006) [2023-12-27 06:43:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 79601664. Throughput: 0: 10436.2, 1: 10170.7. Samples: 79597988. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:43:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:43:57,383][20381] Updated weights for policy 1, policy_version 155304 (0.0007) [2023-12-27 06:43:57,386][20347] Updated weights for policy 0, policy_version 155591 (0.0005) [2023-12-27 06:43:57,431][20381] Updated weights for policy 1, policy_version 155314 (0.0009) [2023-12-27 06:43:57,947][20347] Updated weights for policy 0, policy_version 155601 (0.0007) [2023-12-27 06:43:57,998][20347] Updated weights for policy 0, policy_version 155611 (0.0007) [2023-12-27 06:43:58,051][20347] Updated weights for policy 0, policy_version 155621 (0.0008) [2023-12-27 06:43:58,173][20381] Updated weights for policy 1, policy_version 155324 (0.0008) [2023-12-27 06:43:58,233][20381] Updated weights for policy 1, policy_version 155334 (0.0008) [2023-12-27 06:43:58,295][20381] Updated weights for policy 1, policy_version 155344 (0.0008) [2023-12-27 06:43:58,768][20347] Updated weights for policy 0, policy_version 155631 (0.0007) [2023-12-27 06:43:58,828][20347] Updated weights for policy 0, policy_version 155641 (0.0009) [2023-12-27 06:43:58,889][20347] Updated weights for policy 0, policy_version 155651 (0.0007) [2023-12-27 06:43:58,970][20381] Updated weights for policy 1, policy_version 155354 (0.0007) [2023-12-27 06:43:59,021][20381] Updated weights for policy 1, policy_version 155364 (0.0008) [2023-12-27 06:43:59,066][20381] Updated weights for policy 1, policy_version 155374 (0.0009) [2023-12-27 06:43:59,118][20381] Updated weights for policy 1, policy_version 155384 (0.0009) [2023-12-27 06:43:59,550][20347] Updated weights for policy 0, policy_version 155661 (0.0009) [2023-12-27 06:43:59,605][20347] Updated weights for policy 0, policy_version 155671 (0.0008) [2023-12-27 06:43:59,654][20347] Updated weights for policy 0, policy_version 155681 (0.0008) [2023-12-27 06:43:59,799][20381] Updated weights for policy 1, policy_version 155394 (0.0005) [2023-12-27 06:43:59,863][20381] Updated weights for policy 1, policy_version 155404 (0.0007) [2023-12-27 06:43:59,923][20381] Updated weights for policy 1, policy_version 155414 (0.0007) [2023-12-27 06:44:00,384][20347] Updated weights for policy 0, policy_version 155691 (0.0008) [2023-12-27 06:44:00,432][20347] Updated weights for policy 0, policy_version 155701 (0.0009) [2023-12-27 06:44:00,477][20347] Updated weights for policy 0, policy_version 155711 (0.0006) [2023-12-27 06:44:00,523][20381] Updated weights for policy 1, policy_version 155424 (0.0009) [2023-12-27 06:44:00,580][20381] Updated weights for policy 1, policy_version 155434 (0.0010) [2023-12-27 06:44:00,641][20381] Updated weights for policy 1, policy_version 155444 (0.0010) [2023-12-27 06:44:01,163][20347] Updated weights for policy 0, policy_version 155721 (0.0006) [2023-12-27 06:44:01,221][20347] Updated weights for policy 0, policy_version 155731 (0.0010) [2023-12-27 06:44:01,280][20347] Updated weights for policy 0, policy_version 155741 (0.0007) [2023-12-27 06:44:01,289][20381] Updated weights for policy 1, policy_version 155454 (0.0010) [2023-12-27 06:44:01,336][20347] Updated weights for policy 0, policy_version 155751 (0.0006) [2023-12-27 06:44:01,350][20381] Updated weights for policy 1, policy_version 155464 (0.0010) [2023-12-27 06:44:01,409][20381] Updated weights for policy 1, policy_version 155474 (0.0009) [2023-12-27 06:44:01,997][20347] Updated weights for policy 0, policy_version 155761 (0.0007) [2023-12-27 06:44:02,055][20381] Updated weights for policy 1, policy_version 155484 (0.0008) [2023-12-27 06:44:02,060][20347] Updated weights for policy 0, policy_version 155771 (0.0007) [2023-12-27 06:44:02,112][20347] Updated weights for policy 0, policy_version 155781 (0.0005) [2023-12-27 06:44:02,117][20381] Updated weights for policy 1, policy_version 155494 (0.0007) [2023-12-27 06:44:02,173][20381] Updated weights for policy 1, policy_version 155504 (0.0007) [2023-12-27 06:44:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 79716352. Throughput: 0: 10413.4, 1: 10186.9. Samples: 79722704. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:02,793][20381] Updated weights for policy 1, policy_version 155514 (0.0007) [2023-12-27 06:44:02,816][20347] Updated weights for policy 0, policy_version 155791 (0.0006) [2023-12-27 06:44:02,841][20381] Updated weights for policy 1, policy_version 155524 (0.0006) [2023-12-27 06:44:02,864][20347] Updated weights for policy 0, policy_version 155801 (0.0006) [2023-12-27 06:44:02,892][20381] Updated weights for policy 1, policy_version 155534 (0.0005) [2023-12-27 06:44:02,912][20347] Updated weights for policy 0, policy_version 155811 (0.0005) [2023-12-27 06:44:02,943][20381] Updated weights for policy 1, policy_version 155544 (0.0005) [2023-12-27 06:44:03,585][20347] Updated weights for policy 0, policy_version 155821 (0.0006) [2023-12-27 06:44:03,603][20381] Updated weights for policy 1, policy_version 155554 (0.0006) [2023-12-27 06:44:03,632][20347] Updated weights for policy 0, policy_version 155831 (0.0005) [2023-12-27 06:44:03,652][20381] Updated weights for policy 1, policy_version 155564 (0.0006) [2023-12-27 06:44:03,685][20347] Updated weights for policy 0, policy_version 155841 (0.0005) [2023-12-27 06:44:03,703][20381] Updated weights for policy 1, policy_version 155574 (0.0006) [2023-12-27 06:44:04,309][20381] Updated weights for policy 1, policy_version 155584 (0.0007) [2023-12-27 06:44:04,365][20381] Updated weights for policy 1, policy_version 155594 (0.0008) [2023-12-27 06:44:04,395][20347] Updated weights for policy 0, policy_version 155851 (0.0007) [2023-12-27 06:44:04,420][20381] Updated weights for policy 1, policy_version 155604 (0.0008) [2023-12-27 06:44:04,446][20347] Updated weights for policy 0, policy_version 155861 (0.0007) [2023-12-27 06:44:04,503][20347] Updated weights for policy 0, policy_version 155871 (0.0007) [2023-12-27 06:44:05,050][20381] Updated weights for policy 1, policy_version 155614 (0.0006) [2023-12-27 06:44:05,107][20381] Updated weights for policy 1, policy_version 155624 (0.0005) [2023-12-27 06:44:05,162][20381] Updated weights for policy 1, policy_version 155634 (0.0006) [2023-12-27 06:44:05,241][20347] Updated weights for policy 0, policy_version 155881 (0.0008) [2023-12-27 06:44:05,299][20347] Updated weights for policy 0, policy_version 155891 (0.0008) [2023-12-27 06:44:05,347][20347] Updated weights for policy 0, policy_version 155901 (0.0008) [2023-12-27 06:44:05,398][20347] Updated weights for policy 0, policy_version 155911 (0.0008) [2023-12-27 06:44:05,802][20381] Updated weights for policy 1, policy_version 155644 (0.0008) [2023-12-27 06:44:05,844][20381] Updated weights for policy 1, policy_version 155654 (0.0007) [2023-12-27 06:44:05,890][20381] Updated weights for policy 1, policy_version 155664 (0.0007) [2023-12-27 06:44:06,057][20347] Updated weights for policy 0, policy_version 155921 (0.0008) [2023-12-27 06:44:06,111][20347] Updated weights for policy 0, policy_version 155931 (0.0007) [2023-12-27 06:44:06,173][20347] Updated weights for policy 0, policy_version 155941 (0.0007) [2023-12-27 06:44:06,629][20381] Updated weights for policy 1, policy_version 155674 (0.0007) [2023-12-27 06:44:06,691][20381] Updated weights for policy 1, policy_version 155684 (0.0005) [2023-12-27 06:44:06,752][20381] Updated weights for policy 1, policy_version 155694 (0.0005) [2023-12-27 06:44:06,801][20381] Updated weights for policy 1, policy_version 155704 (0.0005) [2023-12-27 06:44:06,876][20347] Updated weights for policy 0, policy_version 155951 (0.0007) [2023-12-27 06:44:06,941][20347] Updated weights for policy 0, policy_version 155961 (0.0007) [2023-12-27 06:44:06,995][20347] Updated weights for policy 0, policy_version 155971 (0.0007) [2023-12-27 06:44:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 79814656. Throughput: 0: 10426.4, 1: 10230.7. Samples: 79785596. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000155704_39870464.pth... [2023-12-27 06:44:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000155976_39944192.pth... [2023-12-27 06:44:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000154520_39567360.pth [2023-12-27 06:44:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000154792_39641088.pth [2023-12-27 06:44:07,438][20381] Updated weights for policy 1, policy_version 155714 (0.0007) [2023-12-27 06:44:07,488][20381] Updated weights for policy 1, policy_version 155724 (0.0008) [2023-12-27 06:44:07,540][20381] Updated weights for policy 1, policy_version 155734 (0.0007) [2023-12-27 06:44:07,637][20347] Updated weights for policy 0, policy_version 155981 (0.0007) [2023-12-27 06:44:07,694][20347] Updated weights for policy 0, policy_version 155991 (0.0007) [2023-12-27 06:44:07,751][20347] Updated weights for policy 0, policy_version 156001 (0.0007) [2023-12-27 06:44:08,220][20381] Updated weights for policy 1, policy_version 155744 (0.0005) [2023-12-27 06:44:08,273][20381] Updated weights for policy 1, policy_version 155754 (0.0005) [2023-12-27 06:44:08,329][20381] Updated weights for policy 1, policy_version 155764 (0.0006) [2023-12-27 06:44:08,455][20347] Updated weights for policy 0, policy_version 156011 (0.0008) [2023-12-27 06:44:08,514][20347] Updated weights for policy 0, policy_version 156021 (0.0008) [2023-12-27 06:44:08,577][20347] Updated weights for policy 0, policy_version 156031 (0.0008) [2023-12-27 06:44:08,914][20381] Updated weights for policy 1, policy_version 155774 (0.0009) [2023-12-27 06:44:08,960][20381] Updated weights for policy 1, policy_version 155784 (0.0008) [2023-12-27 06:44:09,009][20381] Updated weights for policy 1, policy_version 155794 (0.0006) [2023-12-27 06:44:09,192][20347] Updated weights for policy 0, policy_version 156041 (0.0008) [2023-12-27 06:44:09,250][20347] Updated weights for policy 0, policy_version 156051 (0.0008) [2023-12-27 06:44:09,304][20347] Updated weights for policy 0, policy_version 156061 (0.0007) [2023-12-27 06:44:09,362][20347] Updated weights for policy 0, policy_version 156071 (0.0008) [2023-12-27 06:44:09,706][20381] Updated weights for policy 1, policy_version 155804 (0.0006) [2023-12-27 06:44:09,766][20381] Updated weights for policy 1, policy_version 155814 (0.0009) [2023-12-27 06:44:09,823][20381] Updated weights for policy 1, policy_version 155824 (0.0009) [2023-12-27 06:44:10,005][20347] Updated weights for policy 0, policy_version 156081 (0.0008) [2023-12-27 06:44:10,060][20347] Updated weights for policy 0, policy_version 156091 (0.0008) [2023-12-27 06:44:10,119][20347] Updated weights for policy 0, policy_version 156101 (0.0009) [2023-12-27 06:44:10,559][20381] Updated weights for policy 1, policy_version 155834 (0.0008) [2023-12-27 06:44:10,629][20381] Updated weights for policy 1, policy_version 155844 (0.0008) [2023-12-27 06:44:10,684][20381] Updated weights for policy 1, policy_version 155854 (0.0005) [2023-12-27 06:44:10,737][20381] Updated weights for policy 1, policy_version 155864 (0.0006) [2023-12-27 06:44:10,809][20347] Updated weights for policy 0, policy_version 156111 (0.0007) [2023-12-27 06:44:10,871][20347] Updated weights for policy 0, policy_version 156121 (0.0005) [2023-12-27 06:44:10,923][20347] Updated weights for policy 0, policy_version 156131 (0.0007) [2023-12-27 06:44:11,417][20381] Updated weights for policy 1, policy_version 155874 (0.0008) [2023-12-27 06:44:11,475][20381] Updated weights for policy 1, policy_version 155884 (0.0008) [2023-12-27 06:44:11,529][20381] Updated weights for policy 1, policy_version 155894 (0.0008) [2023-12-27 06:44:11,595][20347] Updated weights for policy 0, policy_version 156141 (0.0009) [2023-12-27 06:44:11,660][20347] Updated weights for policy 0, policy_version 156151 (0.0006) [2023-12-27 06:44:11,716][20347] Updated weights for policy 0, policy_version 156161 (0.0007) [2023-12-27 06:44:12,204][20381] Updated weights for policy 1, policy_version 155904 (0.0007) [2023-12-27 06:44:12,263][20381] Updated weights for policy 1, policy_version 155914 (0.0006) [2023-12-27 06:44:12,316][20381] Updated weights for policy 1, policy_version 155924 (0.0007) [2023-12-27 06:44:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 79921152. Throughput: 0: 10367.1, 1: 10337.1. Samples: 79910208. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:12,424][20347] Updated weights for policy 0, policy_version 156171 (0.0009) [2023-12-27 06:44:12,483][20347] Updated weights for policy 0, policy_version 156181 (0.0008) [2023-12-27 06:44:12,535][20347] Updated weights for policy 0, policy_version 156191 (0.0007) [2023-12-27 06:44:13,044][20381] Updated weights for policy 1, policy_version 155934 (0.0008) [2023-12-27 06:44:13,093][20381] Updated weights for policy 1, policy_version 155944 (0.0008) [2023-12-27 06:44:13,151][20381] Updated weights for policy 1, policy_version 155954 (0.0008) [2023-12-27 06:44:13,173][20347] Updated weights for policy 0, policy_version 156201 (0.0008) [2023-12-27 06:44:13,232][20347] Updated weights for policy 0, policy_version 156211 (0.0010) [2023-12-27 06:44:13,287][20347] Updated weights for policy 0, policy_version 156221 (0.0009) [2023-12-27 06:44:13,342][20347] Updated weights for policy 0, policy_version 156231 (0.0010) [2023-12-27 06:44:13,770][20381] Updated weights for policy 1, policy_version 155964 (0.0009) [2023-12-27 06:44:13,825][20381] Updated weights for policy 1, policy_version 155974 (0.0011) [2023-12-27 06:44:13,874][20381] Updated weights for policy 1, policy_version 155984 (0.0010) [2023-12-27 06:44:14,037][20347] Updated weights for policy 0, policy_version 156241 (0.0008) [2023-12-27 06:44:14,101][20347] Updated weights for policy 0, policy_version 156251 (0.0007) [2023-12-27 06:44:14,159][20347] Updated weights for policy 0, policy_version 156261 (0.0007) [2023-12-27 06:44:14,575][20381] Updated weights for policy 1, policy_version 155994 (0.0010) [2023-12-27 06:44:14,636][20381] Updated weights for policy 1, policy_version 156004 (0.0010) [2023-12-27 06:44:14,687][20381] Updated weights for policy 1, policy_version 156014 (0.0010) [2023-12-27 06:44:14,734][20381] Updated weights for policy 1, policy_version 156024 (0.0009) [2023-12-27 06:44:14,808][20347] Updated weights for policy 0, policy_version 156271 (0.0008) [2023-12-27 06:44:14,865][20347] Updated weights for policy 0, policy_version 156281 (0.0008) [2023-12-27 06:44:14,925][20347] Updated weights for policy 0, policy_version 156291 (0.0008) [2023-12-27 06:44:15,431][20381] Updated weights for policy 1, policy_version 156034 (0.0011) [2023-12-27 06:44:15,490][20381] Updated weights for policy 1, policy_version 156044 (0.0011) [2023-12-27 06:44:15,550][20381] Updated weights for policy 1, policy_version 156054 (0.0011) [2023-12-27 06:44:15,643][20347] Updated weights for policy 0, policy_version 156301 (0.0007) [2023-12-27 06:44:15,694][20347] Updated weights for policy 0, policy_version 156311 (0.0007) [2023-12-27 06:44:15,749][20347] Updated weights for policy 0, policy_version 156321 (0.0007) [2023-12-27 06:44:16,228][20381] Updated weights for policy 1, policy_version 156064 (0.0010) [2023-12-27 06:44:16,272][20381] Updated weights for policy 1, policy_version 156074 (0.0010) [2023-12-27 06:44:16,316][20381] Updated weights for policy 1, policy_version 156084 (0.0010) [2023-12-27 06:44:16,415][20347] Updated weights for policy 0, policy_version 156331 (0.0007) [2023-12-27 06:44:16,476][20347] Updated weights for policy 0, policy_version 156341 (0.0006) [2023-12-27 06:44:16,531][20347] Updated weights for policy 0, policy_version 156351 (0.0005) [2023-12-27 06:44:16,973][20381] Updated weights for policy 1, policy_version 156094 (0.0008) [2023-12-27 06:44:17,040][20381] Updated weights for policy 1, policy_version 156104 (0.0008) [2023-12-27 06:44:17,099][20381] Updated weights for policy 1, policy_version 156114 (0.0008) [2023-12-27 06:44:17,129][20347] Updated weights for policy 0, policy_version 156361 (0.0005) [2023-12-27 06:44:17,198][20347] Updated weights for policy 0, policy_version 156371 (0.0005) [2023-12-27 06:44:17,254][20347] Updated weights for policy 0, policy_version 156381 (0.0005) [2023-12-27 06:44:17,310][20347] Updated weights for policy 0, policy_version 156391 (0.0005) [2023-12-27 06:44:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 80027648. Throughput: 0: 10315.4, 1: 10288.6. Samples: 80032748. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:17,817][20381] Updated weights for policy 1, policy_version 156124 (0.0007) [2023-12-27 06:44:17,857][20347] Updated weights for policy 0, policy_version 156401 (0.0009) [2023-12-27 06:44:17,870][20381] Updated weights for policy 1, policy_version 156134 (0.0005) [2023-12-27 06:44:17,918][20347] Updated weights for policy 0, policy_version 156411 (0.0008) [2023-12-27 06:44:17,924][20381] Updated weights for policy 1, policy_version 156144 (0.0005) [2023-12-27 06:44:17,978][20347] Updated weights for policy 0, policy_version 156421 (0.0007) [2023-12-27 06:44:18,604][20381] Updated weights for policy 1, policy_version 156154 (0.0006) [2023-12-27 06:44:18,649][20347] Updated weights for policy 0, policy_version 156431 (0.0008) [2023-12-27 06:44:18,662][20381] Updated weights for policy 1, policy_version 156164 (0.0008) [2023-12-27 06:44:18,699][20347] Updated weights for policy 0, policy_version 156441 (0.0006) [2023-12-27 06:44:18,731][20381] Updated weights for policy 1, policy_version 156174 (0.0006) [2023-12-27 06:44:18,755][20347] Updated weights for policy 0, policy_version 156451 (0.0007) [2023-12-27 06:44:18,780][20381] Updated weights for policy 1, policy_version 156184 (0.0006) [2023-12-27 06:44:19,450][20347] Updated weights for policy 0, policy_version 156461 (0.0006) [2023-12-27 06:44:19,473][20381] Updated weights for policy 1, policy_version 156194 (0.0006) [2023-12-27 06:44:19,518][20347] Updated weights for policy 0, policy_version 156471 (0.0007) [2023-12-27 06:44:19,540][20381] Updated weights for policy 1, policy_version 156204 (0.0007) [2023-12-27 06:44:19,579][20347] Updated weights for policy 0, policy_version 156481 (0.0006) [2023-12-27 06:44:19,601][20381] Updated weights for policy 1, policy_version 156214 (0.0008) [2023-12-27 06:44:20,248][20381] Updated weights for policy 1, policy_version 156224 (0.0008) [2023-12-27 06:44:20,304][20347] Updated weights for policy 0, policy_version 156491 (0.0006) [2023-12-27 06:44:20,309][20381] Updated weights for policy 1, policy_version 156234 (0.0008) [2023-12-27 06:44:20,361][20347] Updated weights for policy 0, policy_version 156501 (0.0007) [2023-12-27 06:44:20,372][20381] Updated weights for policy 1, policy_version 156244 (0.0008) [2023-12-27 06:44:20,420][20347] Updated weights for policy 0, policy_version 156511 (0.0008) [2023-12-27 06:44:21,080][20381] Updated weights for policy 1, policy_version 156254 (0.0007) [2023-12-27 06:44:21,136][20347] Updated weights for policy 0, policy_version 156521 (0.0008) [2023-12-27 06:44:21,148][20381] Updated weights for policy 1, policy_version 156264 (0.0010) [2023-12-27 06:44:21,199][20347] Updated weights for policy 0, policy_version 156531 (0.0006) [2023-12-27 06:44:21,205][20381] Updated weights for policy 1, policy_version 156274 (0.0011) [2023-12-27 06:44:21,262][20347] Updated weights for policy 0, policy_version 156541 (0.0006) [2023-12-27 06:44:21,329][20347] Updated weights for policy 0, policy_version 156551 (0.0009) [2023-12-27 06:44:21,964][20381] Updated weights for policy 1, policy_version 156284 (0.0010) [2023-12-27 06:44:22,024][20381] Updated weights for policy 1, policy_version 156294 (0.0008) [2023-12-27 06:44:22,068][20347] Updated weights for policy 0, policy_version 156561 (0.0007) [2023-12-27 06:44:22,087][20381] Updated weights for policy 1, policy_version 156304 (0.0007) [2023-12-27 06:44:22,124][20347] Updated weights for policy 0, policy_version 156571 (0.0006) [2023-12-27 06:44:22,184][20347] Updated weights for policy 0, policy_version 156581 (0.0006) [2023-12-27 06:44:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 80125952. Throughput: 0: 10284.6, 1: 10313.4. Samples: 80095084. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000156312_40026112.pth... [2023-12-27 06:44:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000156584_40099840.pth... [2023-12-27 06:44:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000155096_39714816.pth [2023-12-27 06:44:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000155368_39788544.pth [2023-12-27 06:44:22,811][20381] Updated weights for policy 1, policy_version 156314 (0.0006) [2023-12-27 06:44:22,859][20347] Updated weights for policy 0, policy_version 156591 (0.0007) [2023-12-27 06:44:22,870][20381] Updated weights for policy 1, policy_version 156324 (0.0006) [2023-12-27 06:44:22,909][20347] Updated weights for policy 0, policy_version 156601 (0.0007) [2023-12-27 06:44:22,927][20381] Updated weights for policy 1, policy_version 156334 (0.0006) [2023-12-27 06:44:22,966][20347] Updated weights for policy 0, policy_version 156611 (0.0008) [2023-12-27 06:44:22,980][20381] Updated weights for policy 1, policy_version 156344 (0.0006) [2023-12-27 06:44:23,656][20381] Updated weights for policy 1, policy_version 156354 (0.0011) [2023-12-27 06:44:23,672][20347] Updated weights for policy 0, policy_version 156621 (0.0007) [2023-12-27 06:44:23,701][20381] Updated weights for policy 1, policy_version 156364 (0.0010) [2023-12-27 06:44:23,729][20347] Updated weights for policy 0, policy_version 156631 (0.0007) [2023-12-27 06:44:23,760][20381] Updated weights for policy 1, policy_version 156374 (0.0011) [2023-12-27 06:44:23,784][20347] Updated weights for policy 0, policy_version 156641 (0.0006) [2023-12-27 06:44:24,323][20381] Updated weights for policy 1, policy_version 156384 (0.0006) [2023-12-27 06:44:24,374][20381] Updated weights for policy 1, policy_version 156394 (0.0005) [2023-12-27 06:44:24,421][20381] Updated weights for policy 1, policy_version 156404 (0.0005) [2023-12-27 06:44:24,538][20347] Updated weights for policy 0, policy_version 156651 (0.0008) [2023-12-27 06:44:24,594][20347] Updated weights for policy 0, policy_version 156661 (0.0008) [2023-12-27 06:44:24,642][20347] Updated weights for policy 0, policy_version 156671 (0.0007) [2023-12-27 06:44:25,006][20381] Updated weights for policy 1, policy_version 156414 (0.0005) [2023-12-27 06:44:25,052][20381] Updated weights for policy 1, policy_version 156424 (0.0005) [2023-12-27 06:44:25,101][20381] Updated weights for policy 1, policy_version 156434 (0.0006) [2023-12-27 06:44:25,351][20347] Updated weights for policy 0, policy_version 156681 (0.0008) [2023-12-27 06:44:25,400][20347] Updated weights for policy 0, policy_version 156691 (0.0008) [2023-12-27 06:44:25,451][20347] Updated weights for policy 0, policy_version 156701 (0.0007) [2023-12-27 06:44:25,503][20347] Updated weights for policy 0, policy_version 156711 (0.0008) [2023-12-27 06:44:25,758][20381] Updated weights for policy 1, policy_version 156444 (0.0007) [2023-12-27 06:44:25,812][20381] Updated weights for policy 1, policy_version 156454 (0.0005) [2023-12-27 06:44:25,862][20381] Updated weights for policy 1, policy_version 156464 (0.0005) [2023-12-27 06:44:26,199][20347] Updated weights for policy 0, policy_version 156721 (0.0007) [2023-12-27 06:44:26,244][20347] Updated weights for policy 0, policy_version 156731 (0.0009) [2023-12-27 06:44:26,292][20347] Updated weights for policy 0, policy_version 156741 (0.0008) [2023-12-27 06:44:26,437][20381] Updated weights for policy 1, policy_version 156474 (0.0005) [2023-12-27 06:44:26,498][20381] Updated weights for policy 1, policy_version 156484 (0.0005) [2023-12-27 06:44:26,561][20381] Updated weights for policy 1, policy_version 156494 (0.0005) [2023-12-27 06:44:26,614][20381] Updated weights for policy 1, policy_version 156504 (0.0005) [2023-12-27 06:44:26,983][20347] Updated weights for policy 0, policy_version 156751 (0.0008) [2023-12-27 06:44:27,038][20347] Updated weights for policy 0, policy_version 156761 (0.0007) [2023-12-27 06:44:27,099][20347] Updated weights for policy 0, policy_version 156771 (0.0008) [2023-12-27 06:44:27,171][20381] Updated weights for policy 1, policy_version 156514 (0.0010) [2023-12-27 06:44:27,219][20381] Updated weights for policy 1, policy_version 156524 (0.0010) [2023-12-27 06:44:27,264][20381] Updated weights for policy 1, policy_version 156534 (0.0010) [2023-12-27 06:44:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 80232448. Throughput: 0: 10242.8, 1: 10353.3. Samples: 80216252. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:27,795][20347] Updated weights for policy 0, policy_version 156781 (0.0008) [2023-12-27 06:44:27,839][20347] Updated weights for policy 0, policy_version 156791 (0.0006) [2023-12-27 06:44:27,901][20347] Updated weights for policy 0, policy_version 156801 (0.0005) [2023-12-27 06:44:27,970][20381] Updated weights for policy 1, policy_version 156544 (0.0008) [2023-12-27 06:44:28,018][20381] Updated weights for policy 1, policy_version 156554 (0.0007) [2023-12-27 06:44:28,068][20381] Updated weights for policy 1, policy_version 156564 (0.0008) [2023-12-27 06:44:28,474][20347] Updated weights for policy 0, policy_version 156811 (0.0005) [2023-12-27 06:44:28,535][20347] Updated weights for policy 0, policy_version 156821 (0.0010) [2023-12-27 06:44:28,586][20347] Updated weights for policy 0, policy_version 156831 (0.0010) [2023-12-27 06:44:28,760][20381] Updated weights for policy 1, policy_version 156574 (0.0006) [2023-12-27 06:44:28,826][20381] Updated weights for policy 1, policy_version 156584 (0.0005) [2023-12-27 06:44:28,887][20381] Updated weights for policy 1, policy_version 156594 (0.0005) [2023-12-27 06:44:29,261][20347] Updated weights for policy 0, policy_version 156841 (0.0010) [2023-12-27 06:44:29,310][20347] Updated weights for policy 0, policy_version 156851 (0.0007) [2023-12-27 06:44:29,377][20347] Updated weights for policy 0, policy_version 156861 (0.0006) [2023-12-27 06:44:29,439][20347] Updated weights for policy 0, policy_version 156871 (0.0006) [2023-12-27 06:44:29,475][20381] Updated weights for policy 1, policy_version 156604 (0.0007) [2023-12-27 06:44:29,529][20381] Updated weights for policy 1, policy_version 156614 (0.0009) [2023-12-27 06:44:29,588][20381] Updated weights for policy 1, policy_version 156624 (0.0007) [2023-12-27 06:44:30,045][20347] Updated weights for policy 0, policy_version 156881 (0.0006) [2023-12-27 06:44:30,110][20347] Updated weights for policy 0, policy_version 156891 (0.0005) [2023-12-27 06:44:30,168][20347] Updated weights for policy 0, policy_version 156901 (0.0007) [2023-12-27 06:44:30,265][20381] Updated weights for policy 1, policy_version 156634 (0.0008) [2023-12-27 06:44:30,323][20381] Updated weights for policy 1, policy_version 156644 (0.0010) [2023-12-27 06:44:30,384][20381] Updated weights for policy 1, policy_version 156654 (0.0007) [2023-12-27 06:44:30,443][20381] Updated weights for policy 1, policy_version 156664 (0.0007) [2023-12-27 06:44:30,814][20347] Updated weights for policy 0, policy_version 156911 (0.0008) [2023-12-27 06:44:30,884][20347] Updated weights for policy 0, policy_version 156921 (0.0008) [2023-12-27 06:44:30,936][20347] Updated weights for policy 0, policy_version 156931 (0.0010) [2023-12-27 06:44:31,087][20381] Updated weights for policy 1, policy_version 156674 (0.0009) [2023-12-27 06:44:31,143][20381] Updated weights for policy 1, policy_version 156684 (0.0007) [2023-12-27 06:44:31,200][20381] Updated weights for policy 1, policy_version 156694 (0.0006) [2023-12-27 06:44:31,643][20347] Updated weights for policy 0, policy_version 156941 (0.0009) [2023-12-27 06:44:31,705][20347] Updated weights for policy 0, policy_version 156951 (0.0005) [2023-12-27 06:44:31,765][20347] Updated weights for policy 0, policy_version 156961 (0.0007) [2023-12-27 06:44:31,917][20381] Updated weights for policy 1, policy_version 156704 (0.0008) [2023-12-27 06:44:31,974][20381] Updated weights for policy 1, policy_version 156714 (0.0006) [2023-12-27 06:44:32,048][20381] Updated weights for policy 1, policy_version 156724 (0.0005) [2023-12-27 06:44:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 80330752. Throughput: 0: 10292.7, 1: 10450.8. Samples: 80343508. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:32,409][20347] Updated weights for policy 0, policy_version 156971 (0.0006) [2023-12-27 06:44:32,464][20347] Updated weights for policy 0, policy_version 156981 (0.0006) [2023-12-27 06:44:32,525][20347] Updated weights for policy 0, policy_version 156991 (0.0006) [2023-12-27 06:44:32,721][20381] Updated weights for policy 1, policy_version 156734 (0.0006) [2023-12-27 06:44:32,773][20381] Updated weights for policy 1, policy_version 156744 (0.0008) [2023-12-27 06:44:32,822][20381] Updated weights for policy 1, policy_version 156754 (0.0007) [2023-12-27 06:44:33,104][20347] Updated weights for policy 0, policy_version 157001 (0.0007) [2023-12-27 06:44:33,148][20347] Updated weights for policy 0, policy_version 157011 (0.0005) [2023-12-27 06:44:33,207][20347] Updated weights for policy 0, policy_version 157021 (0.0005) [2023-12-27 06:44:33,267][20347] Updated weights for policy 0, policy_version 157031 (0.0005) [2023-12-27 06:44:33,512][20381] Updated weights for policy 1, policy_version 156764 (0.0007) [2023-12-27 06:44:33,561][20381] Updated weights for policy 1, policy_version 156774 (0.0009) [2023-12-27 06:44:33,624][20381] Updated weights for policy 1, policy_version 156784 (0.0010) [2023-12-27 06:44:33,911][20347] Updated weights for policy 0, policy_version 157041 (0.0007) [2023-12-27 06:44:33,967][20347] Updated weights for policy 0, policy_version 157051 (0.0007) [2023-12-27 06:44:34,020][20347] Updated weights for policy 0, policy_version 157061 (0.0005) [2023-12-27 06:44:34,292][20381] Updated weights for policy 1, policy_version 156794 (0.0007) [2023-12-27 06:44:34,354][20381] Updated weights for policy 1, policy_version 156804 (0.0011) [2023-12-27 06:44:34,410][20381] Updated weights for policy 1, policy_version 156814 (0.0010) [2023-12-27 06:44:34,463][20381] Updated weights for policy 1, policy_version 156824 (0.0011) [2023-12-27 06:44:34,694][20347] Updated weights for policy 0, policy_version 157071 (0.0007) [2023-12-27 06:44:34,752][20347] Updated weights for policy 0, policy_version 157081 (0.0007) [2023-12-27 06:44:34,815][20347] Updated weights for policy 0, policy_version 157091 (0.0007) [2023-12-27 06:44:35,174][20381] Updated weights for policy 1, policy_version 156834 (0.0007) [2023-12-27 06:44:35,236][20381] Updated weights for policy 1, policy_version 156844 (0.0008) [2023-12-27 06:44:35,295][20381] Updated weights for policy 1, policy_version 156854 (0.0007) [2023-12-27 06:44:35,437][20347] Updated weights for policy 0, policy_version 157101 (0.0007) [2023-12-27 06:44:35,488][20347] Updated weights for policy 0, policy_version 157111 (0.0005) [2023-12-27 06:44:35,546][20347] Updated weights for policy 0, policy_version 157121 (0.0005) [2023-12-27 06:44:36,005][20381] Updated weights for policy 1, policy_version 156864 (0.0007) [2023-12-27 06:44:36,061][20381] Updated weights for policy 1, policy_version 156874 (0.0006) [2023-12-27 06:44:36,088][20347] Updated weights for policy 0, policy_version 157131 (0.0007) [2023-12-27 06:44:36,110][20381] Updated weights for policy 1, policy_version 156884 (0.0006) [2023-12-27 06:44:36,154][20347] Updated weights for policy 0, policy_version 157141 (0.0007) [2023-12-27 06:44:36,209][20347] Updated weights for policy 0, policy_version 157151 (0.0009) [2023-12-27 06:44:36,782][20381] Updated weights for policy 1, policy_version 156894 (0.0008) [2023-12-27 06:44:36,839][20381] Updated weights for policy 1, policy_version 156904 (0.0007) [2023-12-27 06:44:36,881][20347] Updated weights for policy 0, policy_version 157161 (0.0011) [2023-12-27 06:44:36,898][20381] Updated weights for policy 1, policy_version 156914 (0.0007) [2023-12-27 06:44:36,940][20347] Updated weights for policy 0, policy_version 157171 (0.0010) [2023-12-27 06:44:36,998][20347] Updated weights for policy 0, policy_version 157181 (0.0010) [2023-12-27 06:44:37,050][20347] Updated weights for policy 0, policy_version 157191 (0.0006) [2023-12-27 06:44:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 80437248. Throughput: 0: 10290.4, 1: 10441.0. Samples: 80405308. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000156920_40181760.pth... [2023-12-27 06:44:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000157192_40255488.pth... [2023-12-27 06:44:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000155704_39870464.pth [2023-12-27 06:44:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000155976_39944192.pth [2023-12-27 06:44:37,522][20381] Updated weights for policy 1, policy_version 156924 (0.0008) [2023-12-27 06:44:37,570][20381] Updated weights for policy 1, policy_version 156934 (0.0010) [2023-12-27 06:44:37,617][20381] Updated weights for policy 1, policy_version 156944 (0.0006) [2023-12-27 06:44:37,680][20347] Updated weights for policy 0, policy_version 157201 (0.0009) [2023-12-27 06:44:37,737][20347] Updated weights for policy 0, policy_version 157211 (0.0011) [2023-12-27 06:44:37,796][20347] Updated weights for policy 0, policy_version 157221 (0.0010) [2023-12-27 06:44:38,212][20381] Updated weights for policy 1, policy_version 156954 (0.0008) [2023-12-27 06:44:38,264][20381] Updated weights for policy 1, policy_version 156964 (0.0006) [2023-12-27 06:44:38,314][20381] Updated weights for policy 1, policy_version 156974 (0.0005) [2023-12-27 06:44:38,374][20381] Updated weights for policy 1, policy_version 156984 (0.0007) [2023-12-27 06:44:38,419][20347] Updated weights for policy 0, policy_version 157231 (0.0007) [2023-12-27 06:44:38,470][20347] Updated weights for policy 0, policy_version 157241 (0.0006) [2023-12-27 06:44:38,523][20347] Updated weights for policy 0, policy_version 157251 (0.0007) [2023-12-27 06:44:39,052][20381] Updated weights for policy 1, policy_version 156994 (0.0007) [2023-12-27 06:44:39,104][20381] Updated weights for policy 1, policy_version 157004 (0.0007) [2023-12-27 06:44:39,154][20381] Updated weights for policy 1, policy_version 157014 (0.0008) [2023-12-27 06:44:39,185][20347] Updated weights for policy 0, policy_version 157261 (0.0007) [2023-12-27 06:44:39,250][20347] Updated weights for policy 0, policy_version 157271 (0.0006) [2023-12-27 06:44:39,312][20347] Updated weights for policy 0, policy_version 157281 (0.0005) [2023-12-27 06:44:39,843][20381] Updated weights for policy 1, policy_version 157024 (0.0007) [2023-12-27 06:44:39,902][20381] Updated weights for policy 1, policy_version 157034 (0.0008) [2023-12-27 06:44:39,957][20381] Updated weights for policy 1, policy_version 157044 (0.0007) [2023-12-27 06:44:39,991][20347] Updated weights for policy 0, policy_version 157291 (0.0006) [2023-12-27 06:44:40,045][20347] Updated weights for policy 0, policy_version 157301 (0.0006) [2023-12-27 06:44:40,098][20347] Updated weights for policy 0, policy_version 157311 (0.0006) [2023-12-27 06:44:40,651][20381] Updated weights for policy 1, policy_version 157054 (0.0009) [2023-12-27 06:44:40,705][20381] Updated weights for policy 1, policy_version 157064 (0.0010) [2023-12-27 06:44:40,759][20381] Updated weights for policy 1, policy_version 157074 (0.0010) [2023-12-27 06:44:40,769][20347] Updated weights for policy 0, policy_version 157321 (0.0006) [2023-12-27 06:44:40,826][20347] Updated weights for policy 0, policy_version 157331 (0.0007) [2023-12-27 06:44:40,874][20347] Updated weights for policy 0, policy_version 157341 (0.0007) [2023-12-27 06:44:40,927][20347] Updated weights for policy 0, policy_version 157351 (0.0008) [2023-12-27 06:44:41,463][20381] Updated weights for policy 1, policy_version 157084 (0.0009) [2023-12-27 06:44:41,524][20381] Updated weights for policy 1, policy_version 157094 (0.0008) [2023-12-27 06:44:41,578][20381] Updated weights for policy 1, policy_version 157104 (0.0008) [2023-12-27 06:44:41,667][20347] Updated weights for policy 0, policy_version 157361 (0.0009) [2023-12-27 06:44:41,728][20347] Updated weights for policy 0, policy_version 157371 (0.0009) [2023-12-27 06:44:41,790][20347] Updated weights for policy 0, policy_version 157381 (0.0007) [2023-12-27 06:44:42,264][20381] Updated weights for policy 1, policy_version 157114 (0.0008) [2023-12-27 06:44:42,328][20381] Updated weights for policy 1, policy_version 157124 (0.0011) [2023-12-27 06:44:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 80535552. Throughput: 0: 10330.1, 1: 10420.0. Samples: 80531744. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:42,390][20381] Updated weights for policy 1, policy_version 157134 (0.0011) [2023-12-27 06:44:42,447][20381] Updated weights for policy 1, policy_version 157144 (0.0010) [2023-12-27 06:44:42,448][20347] Updated weights for policy 0, policy_version 157391 (0.0006) [2023-12-27 06:44:42,507][20347] Updated weights for policy 0, policy_version 157401 (0.0007) [2023-12-27 06:44:42,564][20347] Updated weights for policy 0, policy_version 157411 (0.0006) [2023-12-27 06:44:43,123][20381] Updated weights for policy 1, policy_version 157154 (0.0009) [2023-12-27 06:44:43,167][20347] Updated weights for policy 0, policy_version 157421 (0.0006) [2023-12-27 06:44:43,178][20381] Updated weights for policy 1, policy_version 157164 (0.0006) [2023-12-27 06:44:43,215][20347] Updated weights for policy 0, policy_version 157431 (0.0006) [2023-12-27 06:44:43,227][20381] Updated weights for policy 1, policy_version 157174 (0.0010) [2023-12-27 06:44:43,272][20347] Updated weights for policy 0, policy_version 157441 (0.0009) [2023-12-27 06:44:43,869][20347] Updated weights for policy 0, policy_version 157451 (0.0006) [2023-12-27 06:44:43,878][20381] Updated weights for policy 1, policy_version 157184 (0.0010) [2023-12-27 06:44:43,921][20347] Updated weights for policy 0, policy_version 157461 (0.0005) [2023-12-27 06:44:43,924][20381] Updated weights for policy 1, policy_version 157194 (0.0010) [2023-12-27 06:44:43,968][20347] Updated weights for policy 0, policy_version 157471 (0.0005) [2023-12-27 06:44:43,973][20381] Updated weights for policy 1, policy_version 157204 (0.0010) [2023-12-27 06:44:44,652][20347] Updated weights for policy 0, policy_version 157481 (0.0007) [2023-12-27 06:44:44,690][20381] Updated weights for policy 1, policy_version 157214 (0.0009) [2023-12-27 06:44:44,707][20347] Updated weights for policy 0, policy_version 157491 (0.0008) [2023-12-27 06:44:44,747][20381] Updated weights for policy 1, policy_version 157224 (0.0007) [2023-12-27 06:44:44,761][20347] Updated weights for policy 0, policy_version 157501 (0.0007) [2023-12-27 06:44:44,811][20381] Updated weights for policy 1, policy_version 157234 (0.0011) [2023-12-27 06:44:44,817][20347] Updated weights for policy 0, policy_version 157511 (0.0007) [2023-12-27 06:44:45,497][20381] Updated weights for policy 1, policy_version 157244 (0.0011) [2023-12-27 06:44:45,551][20381] Updated weights for policy 1, policy_version 157254 (0.0011) [2023-12-27 06:44:45,552][20347] Updated weights for policy 0, policy_version 157521 (0.0005) [2023-12-27 06:44:45,607][20381] Updated weights for policy 1, policy_version 157264 (0.0011) [2023-12-27 06:44:45,616][20347] Updated weights for policy 0, policy_version 157531 (0.0007) [2023-12-27 06:44:45,670][20347] Updated weights for policy 0, policy_version 157541 (0.0006) [2023-12-27 06:44:46,233][20381] Updated weights for policy 1, policy_version 157274 (0.0011) [2023-12-27 06:44:46,268][20347] Updated weights for policy 0, policy_version 157551 (0.0006) [2023-12-27 06:44:46,283][20381] Updated weights for policy 1, policy_version 157284 (0.0011) [2023-12-27 06:44:46,318][20347] Updated weights for policy 0, policy_version 157561 (0.0010) [2023-12-27 06:44:46,332][20381] Updated weights for policy 1, policy_version 157294 (0.0010) [2023-12-27 06:44:46,367][20347] Updated weights for policy 0, policy_version 157571 (0.0010) [2023-12-27 06:44:46,380][20381] Updated weights for policy 1, policy_version 157304 (0.0010) [2023-12-27 06:44:47,054][20347] Updated weights for policy 0, policy_version 157581 (0.0008) [2023-12-27 06:44:47,091][20381] Updated weights for policy 1, policy_version 157314 (0.0009) [2023-12-27 06:44:47,099][20347] Updated weights for policy 0, policy_version 157591 (0.0005) [2023-12-27 06:44:47,146][20347] Updated weights for policy 0, policy_version 157601 (0.0007) [2023-12-27 06:44:47,147][20381] Updated weights for policy 1, policy_version 157324 (0.0011) [2023-12-27 06:44:47,196][20381] Updated weights for policy 1, policy_version 157334 (0.0010) [2023-12-27 06:44:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 80650240. Throughput: 0: 10348.0, 1: 10387.4. Samples: 80655800. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:47,826][20381] Updated weights for policy 1, policy_version 157344 (0.0011) [2023-12-27 06:44:47,862][20347] Updated weights for policy 0, policy_version 157611 (0.0007) [2023-12-27 06:44:47,877][20381] Updated weights for policy 1, policy_version 157354 (0.0007) [2023-12-27 06:44:47,918][20347] Updated weights for policy 0, policy_version 157621 (0.0008) [2023-12-27 06:44:47,928][20381] Updated weights for policy 1, policy_version 157364 (0.0005) [2023-12-27 06:44:47,964][20347] Updated weights for policy 0, policy_version 157631 (0.0008) [2023-12-27 06:44:48,555][20381] Updated weights for policy 1, policy_version 157374 (0.0006) [2023-12-27 06:44:48,616][20381] Updated weights for policy 1, policy_version 157384 (0.0008) [2023-12-27 06:44:48,642][20347] Updated weights for policy 0, policy_version 157641 (0.0009) [2023-12-27 06:44:48,676][20381] Updated weights for policy 1, policy_version 157394 (0.0008) [2023-12-27 06:44:48,701][20347] Updated weights for policy 0, policy_version 157651 (0.0006) [2023-12-27 06:44:48,762][20347] Updated weights for policy 0, policy_version 157661 (0.0005) [2023-12-27 06:44:48,821][20347] Updated weights for policy 0, policy_version 157671 (0.0005) [2023-12-27 06:44:49,308][20381] Updated weights for policy 1, policy_version 157404 (0.0008) [2023-12-27 06:44:49,374][20381] Updated weights for policy 1, policy_version 157414 (0.0007) [2023-12-27 06:44:49,437][20381] Updated weights for policy 1, policy_version 157424 (0.0007) [2023-12-27 06:44:49,454][20347] Updated weights for policy 0, policy_version 157681 (0.0006) [2023-12-27 06:44:49,504][20347] Updated weights for policy 0, policy_version 157691 (0.0006) [2023-12-27 06:44:49,566][20347] Updated weights for policy 0, policy_version 157701 (0.0006) [2023-12-27 06:44:50,127][20381] Updated weights for policy 1, policy_version 157434 (0.0006) [2023-12-27 06:44:50,190][20381] Updated weights for policy 1, policy_version 157444 (0.0009) [2023-12-27 06:44:50,233][20347] Updated weights for policy 0, policy_version 157711 (0.0007) [2023-12-27 06:44:50,246][20381] Updated weights for policy 1, policy_version 157454 (0.0006) [2023-12-27 06:44:50,289][20347] Updated weights for policy 0, policy_version 157721 (0.0008) [2023-12-27 06:44:50,302][20381] Updated weights for policy 1, policy_version 157464 (0.0006) [2023-12-27 06:44:50,349][20347] Updated weights for policy 0, policy_version 157731 (0.0007) [2023-12-27 06:44:50,956][20381] Updated weights for policy 1, policy_version 157474 (0.0007) [2023-12-27 06:44:51,017][20381] Updated weights for policy 1, policy_version 157484 (0.0008) [2023-12-27 06:44:51,032][20347] Updated weights for policy 0, policy_version 157741 (0.0007) [2023-12-27 06:44:51,080][20381] Updated weights for policy 1, policy_version 157494 (0.0007) [2023-12-27 06:44:51,096][20347] Updated weights for policy 0, policy_version 157751 (0.0008) [2023-12-27 06:44:51,154][20347] Updated weights for policy 0, policy_version 157761 (0.0008) [2023-12-27 06:44:51,756][20381] Updated weights for policy 1, policy_version 157504 (0.0007) [2023-12-27 06:44:51,818][20381] Updated weights for policy 1, policy_version 157514 (0.0007) [2023-12-27 06:44:51,874][20381] Updated weights for policy 1, policy_version 157524 (0.0008) [2023-12-27 06:44:51,899][20347] Updated weights for policy 0, policy_version 157771 (0.0008) [2023-12-27 06:44:51,948][20347] Updated weights for policy 0, policy_version 157781 (0.0010) [2023-12-27 06:44:52,004][20347] Updated weights for policy 0, policy_version 157791 (0.0010) [2023-12-27 06:44:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 80748544. Throughput: 0: 10369.4, 1: 10360.8. Samples: 80718456. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000157528_40337408.pth... [2023-12-27 06:44:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000157800_40411136.pth... [2023-12-27 06:44:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000156312_40026112.pth [2023-12-27 06:44:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000156584_40099840.pth [2023-12-27 06:44:52,453][20381] Updated weights for policy 1, policy_version 157534 (0.0006) [2023-12-27 06:44:52,512][20381] Updated weights for policy 1, policy_version 157544 (0.0005) [2023-12-27 06:44:52,571][20381] Updated weights for policy 1, policy_version 157554 (0.0005) [2023-12-27 06:44:52,673][20347] Updated weights for policy 0, policy_version 157801 (0.0010) [2023-12-27 06:44:52,742][20347] Updated weights for policy 0, policy_version 157811 (0.0010) [2023-12-27 06:44:52,819][20347] Updated weights for policy 0, policy_version 157821 (0.0010) [2023-12-27 06:44:52,878][20347] Updated weights for policy 0, policy_version 157831 (0.0011) [2023-12-27 06:44:53,211][20381] Updated weights for policy 1, policy_version 157564 (0.0005) [2023-12-27 06:44:53,277][20381] Updated weights for policy 1, policy_version 157574 (0.0005) [2023-12-27 06:44:53,341][20381] Updated weights for policy 1, policy_version 157584 (0.0005) [2023-12-27 06:44:53,482][20347] Updated weights for policy 0, policy_version 157841 (0.0011) [2023-12-27 06:44:53,533][20347] Updated weights for policy 0, policy_version 157851 (0.0010) [2023-12-27 06:44:53,586][20347] Updated weights for policy 0, policy_version 157861 (0.0011) [2023-12-27 06:44:53,895][20381] Updated weights for policy 1, policy_version 157594 (0.0006) [2023-12-27 06:44:53,949][20381] Updated weights for policy 1, policy_version 157604 (0.0007) [2023-12-27 06:44:54,000][20381] Updated weights for policy 1, policy_version 157614 (0.0007) [2023-12-27 06:44:54,058][20381] Updated weights for policy 1, policy_version 157624 (0.0007) [2023-12-27 06:44:54,291][20347] Updated weights for policy 0, policy_version 157871 (0.0007) [2023-12-27 06:44:54,341][20347] Updated weights for policy 0, policy_version 157881 (0.0007) [2023-12-27 06:44:54,385][20347] Updated weights for policy 0, policy_version 157891 (0.0009) [2023-12-27 06:44:54,730][20381] Updated weights for policy 1, policy_version 157634 (0.0011) [2023-12-27 06:44:54,775][20381] Updated weights for policy 1, policy_version 157644 (0.0010) [2023-12-27 06:44:54,824][20381] Updated weights for policy 1, policy_version 157654 (0.0010) [2023-12-27 06:44:55,081][20347] Updated weights for policy 0, policy_version 157901 (0.0008) [2023-12-27 06:44:55,131][20347] Updated weights for policy 0, policy_version 157911 (0.0007) [2023-12-27 06:44:55,179][20347] Updated weights for policy 0, policy_version 157921 (0.0007) [2023-12-27 06:44:55,533][20381] Updated weights for policy 1, policy_version 157664 (0.0011) [2023-12-27 06:44:55,586][20381] Updated weights for policy 1, policy_version 157674 (0.0011) [2023-12-27 06:44:55,638][20381] Updated weights for policy 1, policy_version 157684 (0.0010) [2023-12-27 06:44:55,810][20347] Updated weights for policy 0, policy_version 157931 (0.0006) [2023-12-27 06:44:55,863][20347] Updated weights for policy 0, policy_version 157941 (0.0005) [2023-12-27 06:44:55,910][20347] Updated weights for policy 0, policy_version 157951 (0.0007) [2023-12-27 06:44:56,273][20381] Updated weights for policy 1, policy_version 157694 (0.0010) [2023-12-27 06:44:56,328][20381] Updated weights for policy 1, policy_version 157704 (0.0010) [2023-12-27 06:44:56,373][20381] Updated weights for policy 1, policy_version 157714 (0.0010) [2023-12-27 06:44:56,560][20347] Updated weights for policy 0, policy_version 157961 (0.0007) [2023-12-27 06:44:56,613][20347] Updated weights for policy 0, policy_version 157971 (0.0007) [2023-12-27 06:44:56,670][20347] Updated weights for policy 0, policy_version 157981 (0.0008) [2023-12-27 06:44:56,720][20347] Updated weights for policy 0, policy_version 157991 (0.0007) [2023-12-27 06:44:57,052][20381] Updated weights for policy 1, policy_version 157724 (0.0010) [2023-12-27 06:44:57,096][20381] Updated weights for policy 1, policy_version 157734 (0.0010) [2023-12-27 06:44:57,144][20381] Updated weights for policy 1, policy_version 157744 (0.0010) [2023-12-27 06:44:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 80855040. Throughput: 0: 10363.6, 1: 10400.3. Samples: 80844580. Policy #0 lag: (min: 6.0, avg: 16.1, max: 38.0) [2023-12-27 06:44:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:44:57,396][20347] Updated weights for policy 0, policy_version 158001 (0.0006) [2023-12-27 06:44:57,445][20347] Updated weights for policy 0, policy_version 158011 (0.0006) [2023-12-27 06:44:57,489][20347] Updated weights for policy 0, policy_version 158021 (0.0008) [2023-12-27 06:44:57,815][20381] Updated weights for policy 1, policy_version 157754 (0.0008) [2023-12-27 06:44:57,869][20381] Updated weights for policy 1, policy_version 157764 (0.0010) [2023-12-27 06:44:57,922][20381] Updated weights for policy 1, policy_version 157774 (0.0010) [2023-12-27 06:44:57,979][20381] Updated weights for policy 1, policy_version 157784 (0.0010) [2023-12-27 06:44:58,170][20347] Updated weights for policy 0, policy_version 158031 (0.0008) [2023-12-27 06:44:58,229][20347] Updated weights for policy 0, policy_version 158041 (0.0007) [2023-12-27 06:44:58,296][20347] Updated weights for policy 0, policy_version 158051 (0.0005) [2023-12-27 06:44:58,692][20381] Updated weights for policy 1, policy_version 157794 (0.0010) [2023-12-27 06:44:58,758][20381] Updated weights for policy 1, policy_version 157804 (0.0007) [2023-12-27 06:44:58,828][20381] Updated weights for policy 1, policy_version 157814 (0.0010) [2023-12-27 06:44:59,104][20347] Updated weights for policy 0, policy_version 158061 (0.0007) [2023-12-27 06:44:59,158][20347] Updated weights for policy 0, policy_version 158071 (0.0007) [2023-12-27 06:44:59,231][20347] Updated weights for policy 0, policy_version 158081 (0.0008) [2023-12-27 06:44:59,569][20381] Updated weights for policy 1, policy_version 157824 (0.0008) [2023-12-27 06:44:59,623][20381] Updated weights for policy 1, policy_version 157834 (0.0008) [2023-12-27 06:44:59,675][20381] Updated weights for policy 1, policy_version 157844 (0.0008) [2023-12-27 06:44:59,901][20347] Updated weights for policy 0, policy_version 158091 (0.0007) [2023-12-27 06:44:59,953][20347] Updated weights for policy 0, policy_version 158101 (0.0007) [2023-12-27 06:45:00,006][20347] Updated weights for policy 0, policy_version 158111 (0.0008) [2023-12-27 06:45:00,370][20381] Updated weights for policy 1, policy_version 157854 (0.0007) [2023-12-27 06:45:00,423][20381] Updated weights for policy 1, policy_version 157864 (0.0009) [2023-12-27 06:45:00,478][20381] Updated weights for policy 1, policy_version 157874 (0.0009) [2023-12-27 06:45:00,619][20347] Updated weights for policy 0, policy_version 158121 (0.0007) [2023-12-27 06:45:00,681][20347] Updated weights for policy 0, policy_version 158131 (0.0008) [2023-12-27 06:45:00,736][20347] Updated weights for policy 0, policy_version 158141 (0.0006) [2023-12-27 06:45:00,795][20347] Updated weights for policy 0, policy_version 158151 (0.0008) [2023-12-27 06:45:01,198][20381] Updated weights for policy 1, policy_version 157884 (0.0008) [2023-12-27 06:45:01,258][20381] Updated weights for policy 1, policy_version 157894 (0.0006) [2023-12-27 06:45:01,316][20381] Updated weights for policy 1, policy_version 157904 (0.0007) [2023-12-27 06:45:01,567][20347] Updated weights for policy 0, policy_version 158161 (0.0007) [2023-12-27 06:45:01,629][20347] Updated weights for policy 0, policy_version 158171 (0.0008) [2023-12-27 06:45:01,682][20347] Updated weights for policy 0, policy_version 158181 (0.0008) [2023-12-27 06:45:02,024][20381] Updated weights for policy 1, policy_version 157914 (0.0008) [2023-12-27 06:45:02,089][20381] Updated weights for policy 1, policy_version 157924 (0.0007) [2023-12-27 06:45:02,148][20381] Updated weights for policy 1, policy_version 157934 (0.0007) [2023-12-27 06:45:02,200][20381] Updated weights for policy 1, policy_version 157944 (0.0008) [2023-12-27 06:45:02,347][20347] Updated weights for policy 0, policy_version 158191 (0.0010) [2023-12-27 06:45:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 80953344. Throughput: 0: 10339.5, 1: 10351.7. Samples: 80963852. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:02,404][20347] Updated weights for policy 0, policy_version 158201 (0.0007) [2023-12-27 06:45:02,461][20347] Updated weights for policy 0, policy_version 158211 (0.0006) [2023-12-27 06:45:02,808][20381] Updated weights for policy 1, policy_version 157954 (0.0005) [2023-12-27 06:45:02,868][20381] Updated weights for policy 1, policy_version 157965 (0.0007) [2023-12-27 06:45:02,920][20381] Updated weights for policy 1, policy_version 157975 (0.0005) [2023-12-27 06:45:03,103][20347] Updated weights for policy 0, policy_version 158221 (0.0007) [2023-12-27 06:45:03,155][20347] Updated weights for policy 0, policy_version 158231 (0.0006) [2023-12-27 06:45:03,211][20347] Updated weights for policy 0, policy_version 158241 (0.0007) [2023-12-27 06:45:03,487][20381] Updated weights for policy 1, policy_version 157985 (0.0005) [2023-12-27 06:45:03,541][20381] Updated weights for policy 1, policy_version 157995 (0.0005) [2023-12-27 06:45:03,602][20381] Updated weights for policy 1, policy_version 158005 (0.0006) [2023-12-27 06:45:03,843][20347] Updated weights for policy 0, policy_version 158251 (0.0009) [2023-12-27 06:45:03,905][20347] Updated weights for policy 0, policy_version 158261 (0.0008) [2023-12-27 06:45:03,969][20347] Updated weights for policy 0, policy_version 158271 (0.0008) [2023-12-27 06:45:04,143][20381] Updated weights for policy 1, policy_version 158015 (0.0007) [2023-12-27 06:45:04,191][20381] Updated weights for policy 1, policy_version 158025 (0.0006) [2023-12-27 06:45:04,246][20381] Updated weights for policy 1, policy_version 158035 (0.0006) [2023-12-27 06:45:04,652][20347] Updated weights for policy 0, policy_version 158281 (0.0008) [2023-12-27 06:45:04,710][20347] Updated weights for policy 0, policy_version 158291 (0.0006) [2023-12-27 06:45:04,763][20347] Updated weights for policy 0, policy_version 158301 (0.0006) [2023-12-27 06:45:04,814][20347] Updated weights for policy 0, policy_version 158311 (0.0007) [2023-12-27 06:45:04,954][20381] Updated weights for policy 1, policy_version 158045 (0.0007) [2023-12-27 06:45:05,016][20381] Updated weights for policy 1, policy_version 158055 (0.0011) [2023-12-27 06:45:05,072][20381] Updated weights for policy 1, policy_version 158065 (0.0011) [2023-12-27 06:45:05,434][20347] Updated weights for policy 0, policy_version 158321 (0.0006) [2023-12-27 06:45:05,493][20347] Updated weights for policy 0, policy_version 158331 (0.0005) [2023-12-27 06:45:05,544][20347] Updated weights for policy 0, policy_version 158341 (0.0005) [2023-12-27 06:45:05,741][20381] Updated weights for policy 1, policy_version 158075 (0.0010) [2023-12-27 06:45:05,797][20381] Updated weights for policy 1, policy_version 158085 (0.0009) [2023-12-27 06:45:05,843][20381] Updated weights for policy 1, policy_version 158095 (0.0006) [2023-12-27 06:45:06,245][20347] Updated weights for policy 0, policy_version 158351 (0.0007) [2023-12-27 06:45:06,298][20347] Updated weights for policy 0, policy_version 158361 (0.0009) [2023-12-27 06:45:06,357][20347] Updated weights for policy 0, policy_version 158371 (0.0006) [2023-12-27 06:45:06,477][20381] Updated weights for policy 1, policy_version 158105 (0.0005) [2023-12-27 06:45:06,537][20381] Updated weights for policy 1, policy_version 158115 (0.0007) [2023-12-27 06:45:06,595][20381] Updated weights for policy 1, policy_version 158125 (0.0008) [2023-12-27 06:45:06,655][20381] Updated weights for policy 1, policy_version 158135 (0.0008) [2023-12-27 06:45:07,001][20347] Updated weights for policy 0, policy_version 158381 (0.0006) [2023-12-27 06:45:07,068][20347] Updated weights for policy 0, policy_version 158391 (0.0007) [2023-12-27 06:45:07,123][20347] Updated weights for policy 0, policy_version 158401 (0.0007) [2023-12-27 06:45:07,295][20381] Updated weights for policy 1, policy_version 158145 (0.0006) [2023-12-27 06:45:07,348][20381] Updated weights for policy 1, policy_version 158155 (0.0007) [2023-12-27 06:45:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 81059840. Throughput: 0: 10335.6, 1: 10425.1. Samples: 81029312. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000158408_40566784.pth... [2023-12-27 06:45:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000157192_40255488.pth [2023-12-27 06:45:07,398][20381] Updated weights for policy 1, policy_version 158165 (0.0007) [2023-12-27 06:45:07,414][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000158168_40501248.pth... [2023-12-27 06:45:07,417][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000156920_40181760.pth [2023-12-27 06:45:07,787][20347] Updated weights for policy 0, policy_version 158411 (0.0007) [2023-12-27 06:45:07,843][20347] Updated weights for policy 0, policy_version 158421 (0.0007) [2023-12-27 06:45:07,892][20347] Updated weights for policy 0, policy_version 158431 (0.0007) [2023-12-27 06:45:08,023][20381] Updated weights for policy 1, policy_version 158175 (0.0007) [2023-12-27 06:45:08,081][20381] Updated weights for policy 1, policy_version 158185 (0.0007) [2023-12-27 06:45:08,137][20381] Updated weights for policy 1, policy_version 158195 (0.0008) [2023-12-27 06:45:08,618][20347] Updated weights for policy 0, policy_version 158441 (0.0008) [2023-12-27 06:45:08,673][20347] Updated weights for policy 0, policy_version 158451 (0.0007) [2023-12-27 06:45:08,721][20347] Updated weights for policy 0, policy_version 158461 (0.0008) [2023-12-27 06:45:08,775][20347] Updated weights for policy 0, policy_version 158471 (0.0007) [2023-12-27 06:45:08,837][20381] Updated weights for policy 1, policy_version 158205 (0.0008) [2023-12-27 06:45:08,900][20381] Updated weights for policy 1, policy_version 158215 (0.0008) [2023-12-27 06:45:08,957][20381] Updated weights for policy 1, policy_version 158225 (0.0007) [2023-12-27 06:45:09,513][20347] Updated weights for policy 0, policy_version 158481 (0.0007) [2023-12-27 06:45:09,581][20347] Updated weights for policy 0, policy_version 158491 (0.0008) [2023-12-27 06:45:09,644][20381] Updated weights for policy 1, policy_version 158235 (0.0006) [2023-12-27 06:45:09,645][20347] Updated weights for policy 0, policy_version 158501 (0.0008) [2023-12-27 06:45:09,703][20381] Updated weights for policy 1, policy_version 158245 (0.0006) [2023-12-27 06:45:09,768][20381] Updated weights for policy 1, policy_version 158255 (0.0007) [2023-12-27 06:45:10,368][20347] Updated weights for policy 0, policy_version 158511 (0.0008) [2023-12-27 06:45:10,414][20381] Updated weights for policy 1, policy_version 158265 (0.0008) [2023-12-27 06:45:10,419][20347] Updated weights for policy 0, policy_version 158521 (0.0007) [2023-12-27 06:45:10,473][20381] Updated weights for policy 1, policy_version 158275 (0.0010) [2023-12-27 06:45:10,482][20347] Updated weights for policy 0, policy_version 158531 (0.0005) [2023-12-27 06:45:10,535][20381] Updated weights for policy 1, policy_version 158285 (0.0011) [2023-12-27 06:45:10,593][20381] Updated weights for policy 1, policy_version 158295 (0.0011) [2023-12-27 06:45:11,204][20347] Updated weights for policy 0, policy_version 158541 (0.0006) [2023-12-27 06:45:11,250][20381] Updated weights for policy 1, policy_version 158305 (0.0010) [2023-12-27 06:45:11,268][20347] Updated weights for policy 0, policy_version 158551 (0.0007) [2023-12-27 06:45:11,316][20381] Updated weights for policy 1, policy_version 158315 (0.0009) [2023-12-27 06:45:11,325][20347] Updated weights for policy 0, policy_version 158561 (0.0007) [2023-12-27 06:45:11,382][20381] Updated weights for policy 1, policy_version 158325 (0.0009) [2023-12-27 06:45:11,973][20347] Updated weights for policy 0, policy_version 158571 (0.0007) [2023-12-27 06:45:12,039][20347] Updated weights for policy 0, policy_version 158581 (0.0008) [2023-12-27 06:45:12,098][20347] Updated weights for policy 0, policy_version 158591 (0.0007) [2023-12-27 06:45:12,111][20381] Updated weights for policy 1, policy_version 158335 (0.0008) [2023-12-27 06:45:12,172][20381] Updated weights for policy 1, policy_version 158345 (0.0008) [2023-12-27 06:45:12,226][20381] Updated weights for policy 1, policy_version 158355 (0.0008) [2023-12-27 06:45:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 81166336. Throughput: 0: 10379.3, 1: 10426.7. Samples: 81152520. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:12,825][20347] Updated weights for policy 0, policy_version 158601 (0.0006) [2023-12-27 06:45:12,888][20347] Updated weights for policy 0, policy_version 158611 (0.0007) [2023-12-27 06:45:12,937][20381] Updated weights for policy 1, policy_version 158365 (0.0007) [2023-12-27 06:45:12,950][20347] Updated weights for policy 0, policy_version 158621 (0.0008) [2023-12-27 06:45:12,987][20381] Updated weights for policy 1, policy_version 158375 (0.0007) [2023-12-27 06:45:13,014][20347] Updated weights for policy 0, policy_version 158631 (0.0008) [2023-12-27 06:45:13,039][20381] Updated weights for policy 1, policy_version 158385 (0.0008) [2023-12-27 06:45:13,670][20381] Updated weights for policy 1, policy_version 158395 (0.0007) [2023-12-27 06:45:13,709][20347] Updated weights for policy 0, policy_version 158641 (0.0008) [2023-12-27 06:45:13,734][20381] Updated weights for policy 1, policy_version 158405 (0.0006) [2023-12-27 06:45:13,768][20347] Updated weights for policy 0, policy_version 158651 (0.0008) [2023-12-27 06:45:13,790][20381] Updated weights for policy 1, policy_version 158415 (0.0007) [2023-12-27 06:45:13,821][20347] Updated weights for policy 0, policy_version 158661 (0.0007) [2023-12-27 06:45:14,395][20381] Updated weights for policy 1, policy_version 158425 (0.0006) [2023-12-27 06:45:14,447][20381] Updated weights for policy 1, policy_version 158435 (0.0008) [2023-12-27 06:45:14,511][20381] Updated weights for policy 1, policy_version 158445 (0.0006) [2023-12-27 06:45:14,546][20347] Updated weights for policy 0, policy_version 158671 (0.0009) [2023-12-27 06:45:14,560][20381] Updated weights for policy 1, policy_version 158455 (0.0006) [2023-12-27 06:45:14,600][20347] Updated weights for policy 0, policy_version 158681 (0.0010) [2023-12-27 06:45:14,656][20347] Updated weights for policy 0, policy_version 158691 (0.0011) [2023-12-27 06:45:15,292][20381] Updated weights for policy 1, policy_version 158465 (0.0007) [2023-12-27 06:45:15,349][20381] Updated weights for policy 1, policy_version 158475 (0.0007) [2023-12-27 06:45:15,362][20347] Updated weights for policy 0, policy_version 158701 (0.0011) [2023-12-27 06:45:15,409][20381] Updated weights for policy 1, policy_version 158485 (0.0006) [2023-12-27 06:45:15,419][20347] Updated weights for policy 0, policy_version 158711 (0.0011) [2023-12-27 06:45:15,475][20347] Updated weights for policy 0, policy_version 158721 (0.0011) [2023-12-27 06:45:16,057][20381] Updated weights for policy 1, policy_version 158495 (0.0007) [2023-12-27 06:45:16,120][20381] Updated weights for policy 1, policy_version 158505 (0.0011) [2023-12-27 06:45:16,157][20347] Updated weights for policy 0, policy_version 158731 (0.0010) [2023-12-27 06:45:16,180][20381] Updated weights for policy 1, policy_version 158515 (0.0011) [2023-12-27 06:45:16,214][20347] Updated weights for policy 0, policy_version 158741 (0.0007) [2023-12-27 06:45:16,269][20347] Updated weights for policy 0, policy_version 158751 (0.0009) [2023-12-27 06:45:16,885][20381] Updated weights for policy 1, policy_version 158525 (0.0011) [2023-12-27 06:45:16,929][20347] Updated weights for policy 0, policy_version 158761 (0.0008) [2023-12-27 06:45:16,955][20381] Updated weights for policy 1, policy_version 158535 (0.0011) [2023-12-27 06:45:16,995][20347] Updated weights for policy 0, policy_version 158771 (0.0008) [2023-12-27 06:45:17,020][20381] Updated weights for policy 1, policy_version 158545 (0.0006) [2023-12-27 06:45:17,058][20347] Updated weights for policy 0, policy_version 158781 (0.0007) [2023-12-27 06:45:17,108][20347] Updated weights for policy 0, policy_version 158791 (0.0007) [2023-12-27 06:45:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 81264640. Throughput: 0: 10284.0, 1: 10362.6. Samples: 81272604. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:17,665][20381] Updated weights for policy 1, policy_version 158555 (0.0008) [2023-12-27 06:45:17,720][20381] Updated weights for policy 1, policy_version 158565 (0.0009) [2023-12-27 06:45:17,777][20381] Updated weights for policy 1, policy_version 158575 (0.0009) [2023-12-27 06:45:17,812][20347] Updated weights for policy 0, policy_version 158801 (0.0005) [2023-12-27 06:45:17,865][20347] Updated weights for policy 0, policy_version 158811 (0.0007) [2023-12-27 06:45:17,911][20347] Updated weights for policy 0, policy_version 158821 (0.0007) [2023-12-27 06:45:18,458][20381] Updated weights for policy 1, policy_version 158585 (0.0010) [2023-12-27 06:45:18,523][20381] Updated weights for policy 1, policy_version 158595 (0.0009) [2023-12-27 06:45:18,574][20381] Updated weights for policy 1, policy_version 158605 (0.0008) [2023-12-27 06:45:18,626][20347] Updated weights for policy 0, policy_version 158831 (0.0007) [2023-12-27 06:45:18,639][20381] Updated weights for policy 1, policy_version 158615 (0.0007) [2023-12-27 06:45:18,690][20347] Updated weights for policy 0, policy_version 158841 (0.0007) [2023-12-27 06:45:18,753][20347] Updated weights for policy 0, policy_version 158851 (0.0007) [2023-12-27 06:45:19,343][20381] Updated weights for policy 1, policy_version 158625 (0.0009) [2023-12-27 06:45:19,402][20381] Updated weights for policy 1, policy_version 158635 (0.0010) [2023-12-27 06:45:19,426][20347] Updated weights for policy 0, policy_version 158861 (0.0007) [2023-12-27 06:45:19,457][20381] Updated weights for policy 1, policy_version 158645 (0.0010) [2023-12-27 06:45:19,479][20347] Updated weights for policy 0, policy_version 158871 (0.0006) [2023-12-27 06:45:19,540][20347] Updated weights for policy 0, policy_version 158881 (0.0007) [2023-12-27 06:45:20,205][20381] Updated weights for policy 1, policy_version 158655 (0.0007) [2023-12-27 06:45:20,209][20347] Updated weights for policy 0, policy_version 158891 (0.0006) [2023-12-27 06:45:20,265][20381] Updated weights for policy 1, policy_version 158665 (0.0010) [2023-12-27 06:45:20,267][20347] Updated weights for policy 0, policy_version 158901 (0.0005) [2023-12-27 06:45:20,323][20347] Updated weights for policy 0, policy_version 158911 (0.0005) [2023-12-27 06:45:20,324][20381] Updated weights for policy 1, policy_version 158675 (0.0011) [2023-12-27 06:45:21,003][20381] Updated weights for policy 1, policy_version 158685 (0.0011) [2023-12-27 06:45:21,039][20347] Updated weights for policy 0, policy_version 158921 (0.0007) [2023-12-27 06:45:21,069][20381] Updated weights for policy 1, policy_version 158695 (0.0010) [2023-12-27 06:45:21,110][20347] Updated weights for policy 0, policy_version 158931 (0.0011) [2023-12-27 06:45:21,132][20381] Updated weights for policy 1, policy_version 158705 (0.0010) [2023-12-27 06:45:21,177][20347] Updated weights for policy 0, policy_version 158941 (0.0010) [2023-12-27 06:45:21,243][20347] Updated weights for policy 0, policy_version 158951 (0.0010) [2023-12-27 06:45:21,855][20381] Updated weights for policy 1, policy_version 158715 (0.0011) [2023-12-27 06:45:21,913][20381] Updated weights for policy 1, policy_version 158725 (0.0010) [2023-12-27 06:45:21,929][20347] Updated weights for policy 0, policy_version 158961 (0.0009) [2023-12-27 06:45:21,972][20381] Updated weights for policy 1, policy_version 158735 (0.0007) [2023-12-27 06:45:21,981][20347] Updated weights for policy 0, policy_version 158971 (0.0009) [2023-12-27 06:45:22,039][20347] Updated weights for policy 0, policy_version 158981 (0.0008) [2023-12-27 06:45:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 81362944. Throughput: 0: 10256.8, 1: 10355.9. Samples: 81332880. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000158984_40714240.pth... [2023-12-27 06:45:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000158744_40648704.pth... [2023-12-27 06:45:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000157800_40411136.pth [2023-12-27 06:45:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000157528_40337408.pth [2023-12-27 06:45:22,615][20381] Updated weights for policy 1, policy_version 158745 (0.0011) [2023-12-27 06:45:22,676][20381] Updated weights for policy 1, policy_version 158755 (0.0010) [2023-12-27 06:45:22,739][20381] Updated weights for policy 1, policy_version 158765 (0.0011) [2023-12-27 06:45:22,799][20381] Updated weights for policy 1, policy_version 158775 (0.0011) [2023-12-27 06:45:22,807][20347] Updated weights for policy 0, policy_version 158991 (0.0007) [2023-12-27 06:45:22,863][20347] Updated weights for policy 0, policy_version 159001 (0.0007) [2023-12-27 06:45:22,922][20347] Updated weights for policy 0, policy_version 159011 (0.0007) [2023-12-27 06:45:23,384][20381] Updated weights for policy 1, policy_version 158785 (0.0006) [2023-12-27 06:45:23,435][20381] Updated weights for policy 1, policy_version 158795 (0.0005) [2023-12-27 06:45:23,489][20381] Updated weights for policy 1, policy_version 158805 (0.0005) [2023-12-27 06:45:23,684][20347] Updated weights for policy 0, policy_version 159021 (0.0008) [2023-12-27 06:45:23,734][20347] Updated weights for policy 0, policy_version 159031 (0.0007) [2023-12-27 06:45:23,793][20347] Updated weights for policy 0, policy_version 159041 (0.0008) [2023-12-27 06:45:24,077][20381] Updated weights for policy 1, policy_version 158815 (0.0009) [2023-12-27 06:45:24,139][20381] Updated weights for policy 1, policy_version 158825 (0.0010) [2023-12-27 06:45:24,200][20381] Updated weights for policy 1, policy_version 158835 (0.0011) [2023-12-27 06:45:24,503][20347] Updated weights for policy 0, policy_version 159051 (0.0008) [2023-12-27 06:45:24,571][20347] Updated weights for policy 0, policy_version 159061 (0.0009) [2023-12-27 06:45:24,627][20347] Updated weights for policy 0, policy_version 159071 (0.0008) [2023-12-27 06:45:24,774][20381] Updated weights for policy 1, policy_version 158845 (0.0008) [2023-12-27 06:45:24,828][20381] Updated weights for policy 1, policy_version 158855 (0.0006) [2023-12-27 06:45:24,892][20381] Updated weights for policy 1, policy_version 158865 (0.0008) [2023-12-27 06:45:25,334][20347] Updated weights for policy 0, policy_version 159081 (0.0009) [2023-12-27 06:45:25,391][20347] Updated weights for policy 0, policy_version 159091 (0.0009) [2023-12-27 06:45:25,444][20347] Updated weights for policy 0, policy_version 159101 (0.0007) [2023-12-27 06:45:25,491][20347] Updated weights for policy 0, policy_version 159111 (0.0007) [2023-12-27 06:45:25,529][20381] Updated weights for policy 1, policy_version 158875 (0.0009) [2023-12-27 06:45:25,580][20381] Updated weights for policy 1, policy_version 158885 (0.0009) [2023-12-27 06:45:25,634][20381] Updated weights for policy 1, policy_version 158895 (0.0010) [2023-12-27 06:45:26,185][20347] Updated weights for policy 0, policy_version 159121 (0.0007) [2023-12-27 06:45:26,241][20347] Updated weights for policy 0, policy_version 159131 (0.0006) [2023-12-27 06:45:26,278][20381] Updated weights for policy 1, policy_version 158905 (0.0010) [2023-12-27 06:45:26,295][20347] Updated weights for policy 0, policy_version 159141 (0.0005) [2023-12-27 06:45:26,336][20381] Updated weights for policy 1, policy_version 158915 (0.0010) [2023-12-27 06:45:26,393][20381] Updated weights for policy 1, policy_version 158925 (0.0010) [2023-12-27 06:45:26,447][20381] Updated weights for policy 1, policy_version 158935 (0.0010) [2023-12-27 06:45:26,854][20347] Updated weights for policy 0, policy_version 159151 (0.0007) [2023-12-27 06:45:26,902][20347] Updated weights for policy 0, policy_version 159161 (0.0006) [2023-12-27 06:45:26,950][20347] Updated weights for policy 0, policy_version 159171 (0.0006) [2023-12-27 06:45:27,109][20381] Updated weights for policy 1, policy_version 158945 (0.0010) [2023-12-27 06:45:27,160][20381] Updated weights for policy 1, policy_version 158955 (0.0010) [2023-12-27 06:45:27,219][20381] Updated weights for policy 1, policy_version 158965 (0.0010) [2023-12-27 06:45:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 81469440. Throughput: 0: 10120.5, 1: 10404.5. Samples: 81455372. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:27,556][20347] Updated weights for policy 0, policy_version 159181 (0.0007) [2023-12-27 06:45:27,607][20347] Updated weights for policy 0, policy_version 159191 (0.0007) [2023-12-27 06:45:27,659][20347] Updated weights for policy 0, policy_version 159201 (0.0007) [2023-12-27 06:45:27,848][20381] Updated weights for policy 1, policy_version 158975 (0.0010) [2023-12-27 06:45:27,903][20381] Updated weights for policy 1, policy_version 158985 (0.0010) [2023-12-27 06:45:27,963][20381] Updated weights for policy 1, policy_version 158995 (0.0010) [2023-12-27 06:45:28,318][20347] Updated weights for policy 0, policy_version 159211 (0.0006) [2023-12-27 06:45:28,383][20347] Updated weights for policy 0, policy_version 159221 (0.0008) [2023-12-27 06:45:28,443][20347] Updated weights for policy 0, policy_version 159231 (0.0008) [2023-12-27 06:45:28,625][20381] Updated weights for policy 1, policy_version 159005 (0.0011) [2023-12-27 06:45:28,681][20381] Updated weights for policy 1, policy_version 159015 (0.0009) [2023-12-27 06:45:28,745][20381] Updated weights for policy 1, policy_version 159025 (0.0005) [2023-12-27 06:45:29,032][20347] Updated weights for policy 0, policy_version 159241 (0.0007) [2023-12-27 06:45:29,085][20347] Updated weights for policy 0, policy_version 159251 (0.0005) [2023-12-27 06:45:29,138][20347] Updated weights for policy 0, policy_version 159261 (0.0010) [2023-12-27 06:45:29,194][20347] Updated weights for policy 0, policy_version 159271 (0.0010) [2023-12-27 06:45:29,365][20381] Updated weights for policy 1, policy_version 159035 (0.0007) [2023-12-27 06:45:29,427][20381] Updated weights for policy 1, policy_version 159045 (0.0011) [2023-12-27 06:45:29,488][20381] Updated weights for policy 1, policy_version 159055 (0.0010) [2023-12-27 06:45:29,853][20347] Updated weights for policy 0, policy_version 159281 (0.0010) [2023-12-27 06:45:29,908][20347] Updated weights for policy 0, policy_version 159291 (0.0010) [2023-12-27 06:45:29,968][20347] Updated weights for policy 0, policy_version 159301 (0.0009) [2023-12-27 06:45:30,172][20381] Updated weights for policy 1, policy_version 159065 (0.0010) [2023-12-27 06:45:30,220][20381] Updated weights for policy 1, policy_version 159075 (0.0010) [2023-12-27 06:45:30,274][20381] Updated weights for policy 1, policy_version 159085 (0.0010) [2023-12-27 06:45:30,322][20381] Updated weights for policy 1, policy_version 159095 (0.0010) [2023-12-27 06:45:30,657][20347] Updated weights for policy 0, policy_version 159311 (0.0008) [2023-12-27 06:45:30,704][20347] Updated weights for policy 0, policy_version 159321 (0.0008) [2023-12-27 06:45:30,755][20347] Updated weights for policy 0, policy_version 159331 (0.0008) [2023-12-27 06:45:31,023][20381] Updated weights for policy 1, policy_version 159105 (0.0011) [2023-12-27 06:45:31,081][20381] Updated weights for policy 1, policy_version 159115 (0.0011) [2023-12-27 06:45:31,149][20381] Updated weights for policy 1, policy_version 159125 (0.0011) [2023-12-27 06:45:31,452][20347] Updated weights for policy 0, policy_version 159341 (0.0006) [2023-12-27 06:45:31,511][20347] Updated weights for policy 0, policy_version 159351 (0.0006) [2023-12-27 06:45:31,566][20347] Updated weights for policy 0, policy_version 159361 (0.0006) [2023-12-27 06:45:31,852][20381] Updated weights for policy 1, policy_version 159135 (0.0008) [2023-12-27 06:45:31,913][20381] Updated weights for policy 1, policy_version 159145 (0.0007) [2023-12-27 06:45:31,969][20381] Updated weights for policy 1, policy_version 159155 (0.0009) [2023-12-27 06:45:32,202][20347] Updated weights for policy 0, policy_version 159371 (0.0007) [2023-12-27 06:45:32,262][20347] Updated weights for policy 0, policy_version 159381 (0.0007) [2023-12-27 06:45:32,316][20347] Updated weights for policy 0, policy_version 159391 (0.0008) [2023-12-27 06:45:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 81567744. Throughput: 0: 10168.0, 1: 10427.9. Samples: 81582612. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:32,666][20381] Updated weights for policy 1, policy_version 159165 (0.0009) [2023-12-27 06:45:32,719][20381] Updated weights for policy 1, policy_version 159175 (0.0008) [2023-12-27 06:45:32,779][20381] Updated weights for policy 1, policy_version 159185 (0.0008) [2023-12-27 06:45:32,955][20347] Updated weights for policy 0, policy_version 159401 (0.0007) [2023-12-27 06:45:33,017][20347] Updated weights for policy 0, policy_version 159411 (0.0008) [2023-12-27 06:45:33,079][20347] Updated weights for policy 0, policy_version 159421 (0.0005) [2023-12-27 06:45:33,132][20347] Updated weights for policy 0, policy_version 159431 (0.0008) [2023-12-27 06:45:33,383][20381] Updated weights for policy 1, policy_version 159195 (0.0008) [2023-12-27 06:45:33,446][20381] Updated weights for policy 1, policy_version 159206 (0.0007) [2023-12-27 06:45:33,496][20381] Updated weights for policy 1, policy_version 159216 (0.0005) [2023-12-27 06:45:33,742][20347] Updated weights for policy 0, policy_version 159441 (0.0009) [2023-12-27 06:45:33,803][20347] Updated weights for policy 0, policy_version 159451 (0.0007) [2023-12-27 06:45:33,863][20347] Updated weights for policy 0, policy_version 159461 (0.0005) [2023-12-27 06:45:34,054][20381] Updated weights for policy 1, policy_version 159226 (0.0005) [2023-12-27 06:45:34,112][20381] Updated weights for policy 1, policy_version 159236 (0.0005) [2023-12-27 06:45:34,162][20381] Updated weights for policy 1, policy_version 159246 (0.0006) [2023-12-27 06:45:34,222][20381] Updated weights for policy 1, policy_version 159256 (0.0010) [2023-12-27 06:45:34,498][20347] Updated weights for policy 0, policy_version 159471 (0.0006) [2023-12-27 06:45:34,557][20347] Updated weights for policy 0, policy_version 159481 (0.0007) [2023-12-27 06:45:34,618][20347] Updated weights for policy 0, policy_version 159491 (0.0006) [2023-12-27 06:45:34,836][20381] Updated weights for policy 1, policy_version 159266 (0.0011) [2023-12-27 06:45:34,895][20381] Updated weights for policy 1, policy_version 159276 (0.0011) [2023-12-27 06:45:34,954][20381] Updated weights for policy 1, policy_version 159286 (0.0011) [2023-12-27 06:45:35,302][20347] Updated weights for policy 0, policy_version 159501 (0.0006) [2023-12-27 06:45:35,365][20347] Updated weights for policy 0, policy_version 159511 (0.0008) [2023-12-27 06:45:35,423][20347] Updated weights for policy 0, policy_version 159521 (0.0007) [2023-12-27 06:45:35,568][20381] Updated weights for policy 1, policy_version 159296 (0.0008) [2023-12-27 06:45:35,620][20381] Updated weights for policy 1, policy_version 159306 (0.0007) [2023-12-27 06:45:35,684][20381] Updated weights for policy 1, policy_version 159316 (0.0008) [2023-12-27 06:45:36,035][20347] Updated weights for policy 0, policy_version 159531 (0.0008) [2023-12-27 06:45:36,089][20347] Updated weights for policy 0, policy_version 159541 (0.0005) [2023-12-27 06:45:36,152][20347] Updated weights for policy 0, policy_version 159551 (0.0009) [2023-12-27 06:45:36,377][20381] Updated weights for policy 1, policy_version 159326 (0.0008) [2023-12-27 06:45:36,443][20381] Updated weights for policy 1, policy_version 159336 (0.0008) [2023-12-27 06:45:36,505][20381] Updated weights for policy 1, policy_version 159346 (0.0007) [2023-12-27 06:45:36,818][20347] Updated weights for policy 0, policy_version 159561 (0.0007) [2023-12-27 06:45:36,867][20347] Updated weights for policy 0, policy_version 159571 (0.0007) [2023-12-27 06:45:36,915][20347] Updated weights for policy 0, policy_version 159581 (0.0007) [2023-12-27 06:45:36,966][20347] Updated weights for policy 0, policy_version 159591 (0.0007) [2023-12-27 06:45:37,137][20381] Updated weights for policy 1, policy_version 159356 (0.0007) [2023-12-27 06:45:37,200][20381] Updated weights for policy 1, policy_version 159366 (0.0005) [2023-12-27 06:45:37,256][20381] Updated weights for policy 1, policy_version 159376 (0.0007) [2023-12-27 06:45:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 81682432. Throughput: 0: 10189.6, 1: 10458.8. Samples: 81647636. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000159384_40812544.pth... [2023-12-27 06:45:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000159592_40869888.pth... [2023-12-27 06:45:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000158168_40501248.pth [2023-12-27 06:45:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000158408_40566784.pth [2023-12-27 06:45:37,737][20347] Updated weights for policy 0, policy_version 159601 (0.0007) [2023-12-27 06:45:37,787][20347] Updated weights for policy 0, policy_version 159611 (0.0011) [2023-12-27 06:45:37,843][20347] Updated weights for policy 0, policy_version 159621 (0.0010) [2023-12-27 06:45:37,844][20381] Updated weights for policy 1, policy_version 159386 (0.0007) [2023-12-27 06:45:37,904][20381] Updated weights for policy 1, policy_version 159396 (0.0005) [2023-12-27 06:45:37,966][20381] Updated weights for policy 1, policy_version 159406 (0.0006) [2023-12-27 06:45:38,034][20381] Updated weights for policy 1, policy_version 159416 (0.0008) [2023-12-27 06:45:38,517][20347] Updated weights for policy 0, policy_version 159631 (0.0007) [2023-12-27 06:45:38,570][20347] Updated weights for policy 0, policy_version 159641 (0.0010) [2023-12-27 06:45:38,623][20347] Updated weights for policy 0, policy_version 159651 (0.0010) [2023-12-27 06:45:38,695][20381] Updated weights for policy 1, policy_version 159426 (0.0011) [2023-12-27 06:45:38,750][20381] Updated weights for policy 1, policy_version 159436 (0.0011) [2023-12-27 06:45:38,808][20381] Updated weights for policy 1, policy_version 159446 (0.0010) [2023-12-27 06:45:39,286][20347] Updated weights for policy 0, policy_version 159661 (0.0010) [2023-12-27 06:45:39,354][20347] Updated weights for policy 0, policy_version 159671 (0.0010) [2023-12-27 06:45:39,412][20347] Updated weights for policy 0, policy_version 159681 (0.0007) [2023-12-27 06:45:39,467][20381] Updated weights for policy 1, policy_version 159456 (0.0008) [2023-12-27 06:45:39,522][20381] Updated weights for policy 1, policy_version 159466 (0.0007) [2023-12-27 06:45:39,581][20381] Updated weights for policy 1, policy_version 159476 (0.0009) [2023-12-27 06:45:40,115][20347] Updated weights for policy 0, policy_version 159691 (0.0007) [2023-12-27 06:45:40,167][20347] Updated weights for policy 0, policy_version 159701 (0.0007) [2023-12-27 06:45:40,228][20347] Updated weights for policy 0, policy_version 159711 (0.0008) [2023-12-27 06:45:40,295][20381] Updated weights for policy 1, policy_version 159486 (0.0010) [2023-12-27 06:45:40,361][20381] Updated weights for policy 1, policy_version 159496 (0.0011) [2023-12-27 06:45:40,420][20381] Updated weights for policy 1, policy_version 159506 (0.0011) [2023-12-27 06:45:40,894][20347] Updated weights for policy 0, policy_version 159721 (0.0008) [2023-12-27 06:45:40,947][20347] Updated weights for policy 0, policy_version 159731 (0.0006) [2023-12-27 06:45:41,001][20347] Updated weights for policy 0, policy_version 159741 (0.0005) [2023-12-27 06:45:41,067][20347] Updated weights for policy 0, policy_version 159751 (0.0009) [2023-12-27 06:45:41,104][20381] Updated weights for policy 1, policy_version 159516 (0.0010) [2023-12-27 06:45:41,168][20381] Updated weights for policy 1, policy_version 159526 (0.0008) [2023-12-27 06:45:41,232][20381] Updated weights for policy 1, policy_version 159536 (0.0007) [2023-12-27 06:45:41,775][20347] Updated weights for policy 0, policy_version 159761 (0.0007) [2023-12-27 06:45:41,832][20347] Updated weights for policy 0, policy_version 159771 (0.0007) [2023-12-27 06:45:41,881][20347] Updated weights for policy 0, policy_version 159781 (0.0007) [2023-12-27 06:45:41,978][20381] Updated weights for policy 1, policy_version 159546 (0.0007) [2023-12-27 06:45:42,034][20381] Updated weights for policy 1, policy_version 159556 (0.0011) [2023-12-27 06:45:42,096][20381] Updated weights for policy 1, policy_version 159566 (0.0011) [2023-12-27 06:45:42,148][20381] Updated weights for policy 1, policy_version 159576 (0.0010) [2023-12-27 06:45:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 81780736. Throughput: 0: 10183.7, 1: 10405.5. Samples: 81771096. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:42,571][20347] Updated weights for policy 0, policy_version 159791 (0.0007) [2023-12-27 06:45:42,622][20347] Updated weights for policy 0, policy_version 159801 (0.0007) [2023-12-27 06:45:42,684][20347] Updated weights for policy 0, policy_version 159811 (0.0007) [2023-12-27 06:45:42,847][20381] Updated weights for policy 1, policy_version 159586 (0.0007) [2023-12-27 06:45:42,901][20381] Updated weights for policy 1, policy_version 159596 (0.0008) [2023-12-27 06:45:42,952][20381] Updated weights for policy 1, policy_version 159606 (0.0007) [2023-12-27 06:45:43,364][20347] Updated weights for policy 0, policy_version 159821 (0.0006) [2023-12-27 06:45:43,417][20347] Updated weights for policy 0, policy_version 159831 (0.0005) [2023-12-27 06:45:43,468][20347] Updated weights for policy 0, policy_version 159841 (0.0006) [2023-12-27 06:45:43,606][20381] Updated weights for policy 1, policy_version 159616 (0.0006) [2023-12-27 06:45:43,657][20381] Updated weights for policy 1, policy_version 159626 (0.0005) [2023-12-27 06:45:43,712][20381] Updated weights for policy 1, policy_version 159636 (0.0005) [2023-12-27 06:45:44,187][20347] Updated weights for policy 0, policy_version 159851 (0.0008) [2023-12-27 06:45:44,232][20381] Updated weights for policy 1, policy_version 159646 (0.0006) [2023-12-27 06:45:44,240][20347] Updated weights for policy 0, policy_version 159861 (0.0008) [2023-12-27 06:45:44,276][20381] Updated weights for policy 1, policy_version 159656 (0.0005) [2023-12-27 06:45:44,294][20347] Updated weights for policy 0, policy_version 159871 (0.0007) [2023-12-27 06:45:44,321][20381] Updated weights for policy 1, policy_version 159666 (0.0005) [2023-12-27 06:45:44,930][20381] Updated weights for policy 1, policy_version 159676 (0.0009) [2023-12-27 06:45:44,989][20381] Updated weights for policy 1, policy_version 159686 (0.0011) [2023-12-27 06:45:44,994][20347] Updated weights for policy 0, policy_version 159881 (0.0007) [2023-12-27 06:45:45,049][20381] Updated weights for policy 1, policy_version 159696 (0.0011) [2023-12-27 06:45:45,055][20347] Updated weights for policy 0, policy_version 159891 (0.0006) [2023-12-27 06:45:45,101][20347] Updated weights for policy 0, policy_version 159901 (0.0006) [2023-12-27 06:45:45,153][20347] Updated weights for policy 0, policy_version 159911 (0.0007) [2023-12-27 06:45:45,732][20381] Updated weights for policy 1, policy_version 159706 (0.0011) [2023-12-27 06:45:45,786][20381] Updated weights for policy 1, policy_version 159716 (0.0010) [2023-12-27 06:45:45,814][20347] Updated weights for policy 0, policy_version 159921 (0.0006) [2023-12-27 06:45:45,838][20381] Updated weights for policy 1, policy_version 159726 (0.0011) [2023-12-27 06:45:45,865][20347] Updated weights for policy 0, policy_version 159931 (0.0007) [2023-12-27 06:45:45,887][20381] Updated weights for policy 1, policy_version 159736 (0.0010) [2023-12-27 06:45:45,922][20347] Updated weights for policy 0, policy_version 159941 (0.0006) [2023-12-27 06:45:46,572][20381] Updated weights for policy 1, policy_version 159746 (0.0009) [2023-12-27 06:45:46,622][20347] Updated weights for policy 0, policy_version 159951 (0.0008) [2023-12-27 06:45:46,625][20381] Updated weights for policy 1, policy_version 159756 (0.0008) [2023-12-27 06:45:46,679][20381] Updated weights for policy 1, policy_version 159766 (0.0007) [2023-12-27 06:45:46,680][20347] Updated weights for policy 0, policy_version 159961 (0.0007) [2023-12-27 06:45:46,736][20347] Updated weights for policy 0, policy_version 159971 (0.0008) [2023-12-27 06:45:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.1, 300 sec: 20716.0). Total num frames: 81879040. Throughput: 0: 10177.2, 1: 10500.4. Samples: 81894340. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:47,392][20381] Updated weights for policy 1, policy_version 159776 (0.0006) [2023-12-27 06:45:47,396][20347] Updated weights for policy 0, policy_version 159981 (0.0007) [2023-12-27 06:45:47,447][20381] Updated weights for policy 1, policy_version 159786 (0.0006) [2023-12-27 06:45:47,458][20347] Updated weights for policy 0, policy_version 159991 (0.0007) [2023-12-27 06:45:47,507][20381] Updated weights for policy 1, policy_version 159796 (0.0006) [2023-12-27 06:45:47,513][20347] Updated weights for policy 0, policy_version 160001 (0.0007) [2023-12-27 06:45:48,124][20381] Updated weights for policy 1, policy_version 159806 (0.0005) [2023-12-27 06:45:48,185][20381] Updated weights for policy 1, policy_version 159816 (0.0006) [2023-12-27 06:45:48,215][20347] Updated weights for policy 0, policy_version 160011 (0.0008) [2023-12-27 06:45:48,244][20381] Updated weights for policy 1, policy_version 159826 (0.0006) [2023-12-27 06:45:48,274][20347] Updated weights for policy 0, policy_version 160021 (0.0007) [2023-12-27 06:45:48,337][20347] Updated weights for policy 0, policy_version 160031 (0.0008) [2023-12-27 06:45:48,888][20381] Updated weights for policy 1, policy_version 159836 (0.0006) [2023-12-27 06:45:48,940][20381] Updated weights for policy 1, policy_version 159846 (0.0007) [2023-12-27 06:45:48,988][20381] Updated weights for policy 1, policy_version 159856 (0.0007) [2023-12-27 06:45:49,027][20347] Updated weights for policy 0, policy_version 160041 (0.0008) [2023-12-27 06:45:49,089][20347] Updated weights for policy 0, policy_version 160051 (0.0008) [2023-12-27 06:45:49,144][20347] Updated weights for policy 0, policy_version 160061 (0.0007) [2023-12-27 06:45:49,202][20347] Updated weights for policy 0, policy_version 160071 (0.0007) [2023-12-27 06:45:49,705][20381] Updated weights for policy 1, policy_version 159866 (0.0007) [2023-12-27 06:45:49,762][20381] Updated weights for policy 1, policy_version 159876 (0.0008) [2023-12-27 06:45:49,823][20381] Updated weights for policy 1, policy_version 159886 (0.0009) [2023-12-27 06:45:49,887][20381] Updated weights for policy 1, policy_version 159896 (0.0007) [2023-12-27 06:45:49,940][20347] Updated weights for policy 0, policy_version 160081 (0.0007) [2023-12-27 06:45:50,007][20347] Updated weights for policy 0, policy_version 160091 (0.0008) [2023-12-27 06:45:50,065][20347] Updated weights for policy 0, policy_version 160101 (0.0008) [2023-12-27 06:45:50,572][20381] Updated weights for policy 1, policy_version 159906 (0.0007) [2023-12-27 06:45:50,635][20381] Updated weights for policy 1, policy_version 159916 (0.0008) [2023-12-27 06:45:50,695][20381] Updated weights for policy 1, policy_version 159926 (0.0007) [2023-12-27 06:45:50,754][20347] Updated weights for policy 0, policy_version 160111 (0.0007) [2023-12-27 06:45:50,814][20347] Updated weights for policy 0, policy_version 160121 (0.0008) [2023-12-27 06:45:50,864][20347] Updated weights for policy 0, policy_version 160131 (0.0008) [2023-12-27 06:45:51,393][20381] Updated weights for policy 1, policy_version 159936 (0.0008) [2023-12-27 06:45:51,441][20381] Updated weights for policy 1, policy_version 159946 (0.0009) [2023-12-27 06:45:51,494][20381] Updated weights for policy 1, policy_version 159956 (0.0007) [2023-12-27 06:45:51,591][20347] Updated weights for policy 0, policy_version 160141 (0.0008) [2023-12-27 06:45:51,655][20347] Updated weights for policy 0, policy_version 160151 (0.0008) [2023-12-27 06:45:51,715][20347] Updated weights for policy 0, policy_version 160161 (0.0009) [2023-12-27 06:45:52,236][20381] Updated weights for policy 1, policy_version 159966 (0.0008) [2023-12-27 06:45:52,297][20381] Updated weights for policy 1, policy_version 159976 (0.0006) [2023-12-27 06:45:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 81977344. Throughput: 0: 10136.0, 1: 10429.1. Samples: 81954740. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:52,363][20381] Updated weights for policy 1, policy_version 159986 (0.0007) [2023-12-27 06:45:52,365][20347] Updated weights for policy 0, policy_version 160171 (0.0007) [2023-12-27 06:45:52,394][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000159992_40968192.pth... [2023-12-27 06:45:52,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000158744_40648704.pth [2023-12-27 06:45:52,422][20347] Updated weights for policy 0, policy_version 160181 (0.0007) [2023-12-27 06:45:52,472][20347] Updated weights for policy 0, policy_version 160191 (0.0007) [2023-12-27 06:45:52,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000160200_41025536.pth... [2023-12-27 06:45:52,527][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000158984_40714240.pth [2023-12-27 06:45:53,003][20381] Updated weights for policy 1, policy_version 159996 (0.0009) [2023-12-27 06:45:53,052][20381] Updated weights for policy 1, policy_version 160006 (0.0010) [2023-12-27 06:45:53,111][20381] Updated weights for policy 1, policy_version 160016 (0.0005) [2023-12-27 06:45:53,196][20347] Updated weights for policy 0, policy_version 160201 (0.0007) [2023-12-27 06:45:53,254][20347] Updated weights for policy 0, policy_version 160211 (0.0010) [2023-12-27 06:45:53,304][20347] Updated weights for policy 0, policy_version 160221 (0.0010) [2023-12-27 06:45:53,352][20347] Updated weights for policy 0, policy_version 160231 (0.0010) [2023-12-27 06:45:53,736][20381] Updated weights for policy 1, policy_version 160026 (0.0008) [2023-12-27 06:45:53,800][20381] Updated weights for policy 1, policy_version 160036 (0.0010) [2023-12-27 06:45:53,868][20381] Updated weights for policy 1, policy_version 160046 (0.0008) [2023-12-27 06:45:53,933][20381] Updated weights for policy 1, policy_version 160056 (0.0005) [2023-12-27 06:45:53,999][20347] Updated weights for policy 0, policy_version 160241 (0.0010) [2023-12-27 06:45:54,051][20347] Updated weights for policy 0, policy_version 160251 (0.0010) [2023-12-27 06:45:54,098][20347] Updated weights for policy 0, policy_version 160261 (0.0010) [2023-12-27 06:45:54,513][20381] Updated weights for policy 1, policy_version 160066 (0.0006) [2023-12-27 06:45:54,562][20381] Updated weights for policy 1, policy_version 160076 (0.0009) [2023-12-27 06:45:54,610][20381] Updated weights for policy 1, policy_version 160086 (0.0007) [2023-12-27 06:45:54,776][20347] Updated weights for policy 0, policy_version 160271 (0.0010) [2023-12-27 06:45:54,825][20347] Updated weights for policy 0, policy_version 160281 (0.0010) [2023-12-27 06:45:54,870][20347] Updated weights for policy 0, policy_version 160291 (0.0010) [2023-12-27 06:45:55,245][20381] Updated weights for policy 1, policy_version 160096 (0.0008) [2023-12-27 06:45:55,291][20381] Updated weights for policy 1, policy_version 160106 (0.0008) [2023-12-27 06:45:55,338][20381] Updated weights for policy 1, policy_version 160116 (0.0008) [2023-12-27 06:45:55,501][20347] Updated weights for policy 0, policy_version 160301 (0.0009) [2023-12-27 06:45:55,546][20347] Updated weights for policy 0, policy_version 160311 (0.0007) [2023-12-27 06:45:55,594][20347] Updated weights for policy 0, policy_version 160321 (0.0007) [2023-12-27 06:45:56,053][20381] Updated weights for policy 1, policy_version 160126 (0.0007) [2023-12-27 06:45:56,108][20381] Updated weights for policy 1, policy_version 160136 (0.0005) [2023-12-27 06:45:56,162][20381] Updated weights for policy 1, policy_version 160146 (0.0006) [2023-12-27 06:45:56,296][20347] Updated weights for policy 0, policy_version 160331 (0.0006) [2023-12-27 06:45:56,348][20347] Updated weights for policy 0, policy_version 160341 (0.0008) [2023-12-27 06:45:56,393][20347] Updated weights for policy 0, policy_version 160351 (0.0008) [2023-12-27 06:45:56,835][20381] Updated weights for policy 1, policy_version 160156 (0.0008) [2023-12-27 06:45:56,891][20381] Updated weights for policy 1, policy_version 160166 (0.0009) [2023-12-27 06:45:56,939][20381] Updated weights for policy 1, policy_version 160176 (0.0008) [2023-12-27 06:45:57,007][20347] Updated weights for policy 0, policy_version 160361 (0.0007) [2023-12-27 06:45:57,072][20347] Updated weights for policy 0, policy_version 160371 (0.0008) [2023-12-27 06:45:57,131][20347] Updated weights for policy 0, policy_version 160381 (0.0008) [2023-12-27 06:45:57,185][20347] Updated weights for policy 0, policy_version 160391 (0.0008) [2023-12-27 06:45:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 82092032. Throughput: 0: 10167.4, 1: 10425.3. Samples: 82079188. Policy #0 lag: (min: 15.0, avg: 17.7, max: 47.0) [2023-12-27 06:45:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:45:57,602][20381] Updated weights for policy 1, policy_version 160186 (0.0008) [2023-12-27 06:45:57,655][20381] Updated weights for policy 1, policy_version 160196 (0.0007) [2023-12-27 06:45:57,711][20381] Updated weights for policy 1, policy_version 160206 (0.0007) [2023-12-27 06:45:57,761][20381] Updated weights for policy 1, policy_version 160216 (0.0007) [2023-12-27 06:45:57,869][20347] Updated weights for policy 0, policy_version 160401 (0.0010) [2023-12-27 06:45:57,925][20347] Updated weights for policy 0, policy_version 160411 (0.0010) [2023-12-27 06:45:57,971][20347] Updated weights for policy 0, policy_version 160421 (0.0010) [2023-12-27 06:45:58,468][20381] Updated weights for policy 1, policy_version 160226 (0.0008) [2023-12-27 06:45:58,530][20381] Updated weights for policy 1, policy_version 160236 (0.0008) [2023-12-27 06:45:58,587][20381] Updated weights for policy 1, policy_version 160246 (0.0009) [2023-12-27 06:45:58,665][20347] Updated weights for policy 0, policy_version 160431 (0.0009) [2023-12-27 06:45:58,726][20347] Updated weights for policy 0, policy_version 160441 (0.0009) [2023-12-27 06:45:58,786][20347] Updated weights for policy 0, policy_version 160451 (0.0009) [2023-12-27 06:45:59,359][20381] Updated weights for policy 1, policy_version 160256 (0.0008) [2023-12-27 06:45:59,415][20381] Updated weights for policy 1, policy_version 160266 (0.0007) [2023-12-27 06:45:59,468][20381] Updated weights for policy 1, policy_version 160276 (0.0005) [2023-12-27 06:45:59,609][20347] Updated weights for policy 0, policy_version 160461 (0.0008) [2023-12-27 06:45:59,665][20347] Updated weights for policy 0, policy_version 160471 (0.0005) [2023-12-27 06:45:59,719][20347] Updated weights for policy 0, policy_version 160481 (0.0005) [2023-12-27 06:46:00,149][20381] Updated weights for policy 1, policy_version 160286 (0.0006) [2023-12-27 06:46:00,208][20381] Updated weights for policy 1, policy_version 160296 (0.0008) [2023-12-27 06:46:00,266][20381] Updated weights for policy 1, policy_version 160306 (0.0007) [2023-12-27 06:46:00,340][20347] Updated weights for policy 0, policy_version 160491 (0.0008) [2023-12-27 06:46:00,393][20347] Updated weights for policy 0, policy_version 160501 (0.0010) [2023-12-27 06:46:00,453][20347] Updated weights for policy 0, policy_version 160511 (0.0010) [2023-12-27 06:46:00,925][20381] Updated weights for policy 1, policy_version 160316 (0.0008) [2023-12-27 06:46:00,970][20381] Updated weights for policy 1, policy_version 160326 (0.0010) [2023-12-27 06:46:01,028][20381] Updated weights for policy 1, policy_version 160336 (0.0010) [2023-12-27 06:46:01,080][20347] Updated weights for policy 0, policy_version 160521 (0.0009) [2023-12-27 06:46:01,144][20347] Updated weights for policy 0, policy_version 160531 (0.0008) [2023-12-27 06:46:01,198][20347] Updated weights for policy 0, policy_version 160541 (0.0010) [2023-12-27 06:46:01,250][20347] Updated weights for policy 0, policy_version 160551 (0.0010) [2023-12-27 06:46:01,685][20381] Updated weights for policy 1, policy_version 160346 (0.0009) [2023-12-27 06:46:01,748][20381] Updated weights for policy 1, policy_version 160356 (0.0008) [2023-12-27 06:46:01,802][20381] Updated weights for policy 1, policy_version 160366 (0.0008) [2023-12-27 06:46:01,855][20381] Updated weights for policy 1, policy_version 160376 (0.0008) [2023-12-27 06:46:01,978][20347] Updated weights for policy 0, policy_version 160561 (0.0010) [2023-12-27 06:46:02,031][20347] Updated weights for policy 0, policy_version 160571 (0.0006) [2023-12-27 06:46:02,085][20347] Updated weights for policy 0, policy_version 160581 (0.0006) [2023-12-27 06:46:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 82190336. Throughput: 0: 10204.2, 1: 10423.4. Samples: 82200844. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:02,536][20381] Updated weights for policy 1, policy_version 160386 (0.0007) [2023-12-27 06:46:02,588][20381] Updated weights for policy 1, policy_version 160396 (0.0005) [2023-12-27 06:46:02,638][20381] Updated weights for policy 1, policy_version 160406 (0.0010) [2023-12-27 06:46:02,773][20347] Updated weights for policy 0, policy_version 160591 (0.0010) [2023-12-27 06:46:02,840][20347] Updated weights for policy 0, policy_version 160601 (0.0010) [2023-12-27 06:46:02,901][20347] Updated weights for policy 0, policy_version 160611 (0.0010) [2023-12-27 06:46:03,339][20381] Updated weights for policy 1, policy_version 160416 (0.0011) [2023-12-27 06:46:03,399][20381] Updated weights for policy 1, policy_version 160426 (0.0011) [2023-12-27 06:46:03,453][20381] Updated weights for policy 1, policy_version 160436 (0.0011) [2023-12-27 06:46:03,499][20347] Updated weights for policy 0, policy_version 160621 (0.0010) [2023-12-27 06:46:03,554][20347] Updated weights for policy 0, policy_version 160631 (0.0010) [2023-12-27 06:46:03,613][20347] Updated weights for policy 0, policy_version 160641 (0.0010) [2023-12-27 06:46:04,185][20381] Updated weights for policy 1, policy_version 160446 (0.0009) [2023-12-27 06:46:04,250][20381] Updated weights for policy 1, policy_version 160456 (0.0008) [2023-12-27 06:46:04,283][20347] Updated weights for policy 0, policy_version 160651 (0.0009) [2023-12-27 06:46:04,308][20381] Updated weights for policy 1, policy_version 160466 (0.0007) [2023-12-27 06:46:04,339][20347] Updated weights for policy 0, policy_version 160661 (0.0007) [2023-12-27 06:46:04,394][20347] Updated weights for policy 0, policy_version 160671 (0.0007) [2023-12-27 06:46:04,951][20347] Updated weights for policy 0, policy_version 160681 (0.0008) [2023-12-27 06:46:05,015][20347] Updated weights for policy 0, policy_version 160691 (0.0007) [2023-12-27 06:46:05,044][20381] Updated weights for policy 1, policy_version 160476 (0.0008) [2023-12-27 06:46:05,063][20347] Updated weights for policy 0, policy_version 160701 (0.0007) [2023-12-27 06:46:05,111][20381] Updated weights for policy 1, policy_version 160486 (0.0010) [2023-12-27 06:46:05,111][20347] Updated weights for policy 0, policy_version 160711 (0.0007) [2023-12-27 06:46:05,174][20381] Updated weights for policy 1, policy_version 160496 (0.0009) [2023-12-27 06:46:05,775][20347] Updated weights for policy 0, policy_version 160721 (0.0010) [2023-12-27 06:46:05,829][20347] Updated weights for policy 0, policy_version 160731 (0.0007) [2023-12-27 06:46:05,839][20381] Updated weights for policy 1, policy_version 160506 (0.0006) [2023-12-27 06:46:05,882][20347] Updated weights for policy 0, policy_version 160741 (0.0005) [2023-12-27 06:46:05,888][20381] Updated weights for policy 1, policy_version 160516 (0.0008) [2023-12-27 06:46:05,935][20381] Updated weights for policy 1, policy_version 160526 (0.0007) [2023-12-27 06:46:05,983][20381] Updated weights for policy 1, policy_version 160536 (0.0008) [2023-12-27 06:46:06,483][20347] Updated weights for policy 0, policy_version 160751 (0.0008) [2023-12-27 06:46:06,542][20347] Updated weights for policy 0, policy_version 160761 (0.0011) [2023-12-27 06:46:06,602][20347] Updated weights for policy 0, policy_version 160771 (0.0011) [2023-12-27 06:46:06,757][20381] Updated weights for policy 1, policy_version 160546 (0.0007) [2023-12-27 06:46:06,811][20381] Updated weights for policy 1, policy_version 160556 (0.0007) [2023-12-27 06:46:06,868][20381] Updated weights for policy 1, policy_version 160566 (0.0008) [2023-12-27 06:46:07,222][20347] Updated weights for policy 0, policy_version 160781 (0.0008) [2023-12-27 06:46:07,277][20347] Updated weights for policy 0, policy_version 160791 (0.0005) [2023-12-27 06:46:07,335][20347] Updated weights for policy 0, policy_version 160801 (0.0007) [2023-12-27 06:46:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20660.5). Total num frames: 82288640. Throughput: 0: 10251.3, 1: 10406.6. Samples: 82262492. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000160568_41115648.pth... [2023-12-27 06:46:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000160808_41181184.pth... [2023-12-27 06:46:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000159592_40869888.pth [2023-12-27 06:46:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000159384_40812544.pth [2023-12-27 06:46:07,609][20381] Updated weights for policy 1, policy_version 160576 (0.0007) [2023-12-27 06:46:07,661][20381] Updated weights for policy 1, policy_version 160586 (0.0007) [2023-12-27 06:46:07,706][20381] Updated weights for policy 1, policy_version 160596 (0.0009) [2023-12-27 06:46:07,864][20347] Updated weights for policy 0, policy_version 160811 (0.0007) [2023-12-27 06:46:07,919][20347] Updated weights for policy 0, policy_version 160821 (0.0010) [2023-12-27 06:46:07,972][20347] Updated weights for policy 0, policy_version 160831 (0.0010) [2023-12-27 06:46:08,362][20381] Updated weights for policy 1, policy_version 160606 (0.0009) [2023-12-27 06:46:08,418][20381] Updated weights for policy 1, policy_version 160616 (0.0005) [2023-12-27 06:46:08,474][20381] Updated weights for policy 1, policy_version 160626 (0.0005) [2023-12-27 06:46:08,665][20347] Updated weights for policy 0, policy_version 160841 (0.0010) [2023-12-27 06:46:08,726][20347] Updated weights for policy 0, policy_version 160851 (0.0010) [2023-12-27 06:46:08,785][20347] Updated weights for policy 0, policy_version 160861 (0.0010) [2023-12-27 06:46:08,854][20347] Updated weights for policy 0, policy_version 160871 (0.0010) [2023-12-27 06:46:09,052][20381] Updated weights for policy 1, policy_version 160636 (0.0006) [2023-12-27 06:46:09,111][20381] Updated weights for policy 1, policy_version 160646 (0.0007) [2023-12-27 06:46:09,172][20381] Updated weights for policy 1, policy_version 160656 (0.0007) [2023-12-27 06:46:09,532][20347] Updated weights for policy 0, policy_version 160881 (0.0008) [2023-12-27 06:46:09,597][20347] Updated weights for policy 0, policy_version 160891 (0.0009) [2023-12-27 06:46:09,660][20347] Updated weights for policy 0, policy_version 160901 (0.0010) [2023-12-27 06:46:09,891][20381] Updated weights for policy 1, policy_version 160666 (0.0007) [2023-12-27 06:46:09,954][20381] Updated weights for policy 1, policy_version 160676 (0.0008) [2023-12-27 06:46:10,014][20381] Updated weights for policy 1, policy_version 160686 (0.0008) [2023-12-27 06:46:10,078][20381] Updated weights for policy 1, policy_version 160696 (0.0008) [2023-12-27 06:46:10,316][20347] Updated weights for policy 0, policy_version 160911 (0.0008) [2023-12-27 06:46:10,382][20347] Updated weights for policy 0, policy_version 160921 (0.0009) [2023-12-27 06:46:10,443][20347] Updated weights for policy 0, policy_version 160931 (0.0008) [2023-12-27 06:46:10,730][20381] Updated weights for policy 1, policy_version 160706 (0.0008) [2023-12-27 06:46:10,782][20381] Updated weights for policy 1, policy_version 160716 (0.0007) [2023-12-27 06:46:10,841][20381] Updated weights for policy 1, policy_version 160726 (0.0008) [2023-12-27 06:46:11,038][20347] Updated weights for policy 0, policy_version 160941 (0.0009) [2023-12-27 06:46:11,099][20347] Updated weights for policy 0, policy_version 160951 (0.0011) [2023-12-27 06:46:11,163][20347] Updated weights for policy 0, policy_version 160961 (0.0011) [2023-12-27 06:46:11,560][20381] Updated weights for policy 1, policy_version 160736 (0.0008) [2023-12-27 06:46:11,622][20381] Updated weights for policy 1, policy_version 160746 (0.0007) [2023-12-27 06:46:11,683][20381] Updated weights for policy 1, policy_version 160756 (0.0008) [2023-12-27 06:46:11,902][20347] Updated weights for policy 0, policy_version 160971 (0.0010) [2023-12-27 06:46:11,961][20347] Updated weights for policy 0, policy_version 160981 (0.0009) [2023-12-27 06:46:12,022][20347] Updated weights for policy 0, policy_version 160991 (0.0010) [2023-12-27 06:46:12,281][20381] Updated weights for policy 1, policy_version 160766 (0.0007) [2023-12-27 06:46:12,347][20381] Updated weights for policy 1, policy_version 160776 (0.0009) [2023-12-27 06:46:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 82395136. Throughput: 0: 10398.1, 1: 10316.4. Samples: 82387520. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:12,405][20381] Updated weights for policy 1, policy_version 160786 (0.0010) [2023-12-27 06:46:12,679][20347] Updated weights for policy 0, policy_version 161001 (0.0010) [2023-12-27 06:46:12,738][20347] Updated weights for policy 0, policy_version 161011 (0.0011) [2023-12-27 06:46:12,794][20347] Updated weights for policy 0, policy_version 161021 (0.0011) [2023-12-27 06:46:12,854][20347] Updated weights for policy 0, policy_version 161031 (0.0008) [2023-12-27 06:46:13,062][20381] Updated weights for policy 1, policy_version 160796 (0.0008) [2023-12-27 06:46:13,117][20381] Updated weights for policy 1, policy_version 160806 (0.0005) [2023-12-27 06:46:13,171][20381] Updated weights for policy 1, policy_version 160816 (0.0007) [2023-12-27 06:46:13,541][20347] Updated weights for policy 0, policy_version 161041 (0.0009) [2023-12-27 06:46:13,590][20347] Updated weights for policy 0, policy_version 161051 (0.0009) [2023-12-27 06:46:13,654][20347] Updated weights for policy 0, policy_version 161061 (0.0008) [2023-12-27 06:46:13,756][20381] Updated weights for policy 1, policy_version 160826 (0.0007) [2023-12-27 06:46:13,811][20381] Updated weights for policy 1, policy_version 160836 (0.0006) [2023-12-27 06:46:13,871][20381] Updated weights for policy 1, policy_version 160846 (0.0008) [2023-12-27 06:46:13,931][20381] Updated weights for policy 1, policy_version 160856 (0.0006) [2023-12-27 06:46:14,322][20347] Updated weights for policy 0, policy_version 161071 (0.0008) [2023-12-27 06:46:14,385][20347] Updated weights for policy 0, policy_version 161081 (0.0007) [2023-12-27 06:46:14,431][20347] Updated weights for policy 0, policy_version 161091 (0.0007) [2023-12-27 06:46:14,511][20381] Updated weights for policy 1, policy_version 160866 (0.0006) [2023-12-27 06:46:14,558][20381] Updated weights for policy 1, policy_version 160876 (0.0007) [2023-12-27 06:46:14,611][20381] Updated weights for policy 1, policy_version 160886 (0.0008) [2023-12-27 06:46:15,136][20347] Updated weights for policy 0, policy_version 161101 (0.0008) [2023-12-27 06:46:15,195][20347] Updated weights for policy 0, policy_version 161111 (0.0008) [2023-12-27 06:46:15,250][20347] Updated weights for policy 0, policy_version 161121 (0.0007) [2023-12-27 06:46:15,274][20381] Updated weights for policy 1, policy_version 160896 (0.0007) [2023-12-27 06:46:15,329][20381] Updated weights for policy 1, policy_version 160906 (0.0007) [2023-12-27 06:46:15,378][20381] Updated weights for policy 1, policy_version 160916 (0.0008) [2023-12-27 06:46:15,836][20347] Updated weights for policy 0, policy_version 161131 (0.0006) [2023-12-27 06:46:15,891][20347] Updated weights for policy 0, policy_version 161141 (0.0005) [2023-12-27 06:46:15,944][20347] Updated weights for policy 0, policy_version 161151 (0.0006) [2023-12-27 06:46:16,122][20381] Updated weights for policy 1, policy_version 160926 (0.0007) [2023-12-27 06:46:16,181][20381] Updated weights for policy 1, policy_version 160936 (0.0008) [2023-12-27 06:46:16,241][20381] Updated weights for policy 1, policy_version 160946 (0.0008) [2023-12-27 06:46:16,549][20347] Updated weights for policy 0, policy_version 161161 (0.0007) [2023-12-27 06:46:16,601][20347] Updated weights for policy 0, policy_version 161171 (0.0010) [2023-12-27 06:46:16,668][20347] Updated weights for policy 0, policy_version 161181 (0.0005) [2023-12-27 06:46:16,728][20347] Updated weights for policy 0, policy_version 161191 (0.0006) [2023-12-27 06:46:16,964][20381] Updated weights for policy 1, policy_version 160956 (0.0008) [2023-12-27 06:46:17,014][20381] Updated weights for policy 1, policy_version 160966 (0.0006) [2023-12-27 06:46:17,067][20381] Updated weights for policy 1, policy_version 160976 (0.0005) [2023-12-27 06:46:17,300][20347] Updated weights for policy 0, policy_version 161201 (0.0005) [2023-12-27 06:46:17,362][20347] Updated weights for policy 0, policy_version 161211 (0.0006) [2023-12-27 06:46:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 82501632. Throughput: 0: 10352.4, 1: 10335.9. Samples: 82513588. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:17,427][20347] Updated weights for policy 0, policy_version 161221 (0.0007) [2023-12-27 06:46:17,738][20381] Updated weights for policy 1, policy_version 160986 (0.0006) [2023-12-27 06:46:17,798][20381] Updated weights for policy 1, policy_version 160996 (0.0007) [2023-12-27 06:46:17,858][20381] Updated weights for policy 1, policy_version 161006 (0.0007) [2023-12-27 06:46:17,921][20381] Updated weights for policy 1, policy_version 161016 (0.0008) [2023-12-27 06:46:18,008][20347] Updated weights for policy 0, policy_version 161231 (0.0008) [2023-12-27 06:46:18,061][20347] Updated weights for policy 0, policy_version 161241 (0.0010) [2023-12-27 06:46:18,123][20347] Updated weights for policy 0, policy_version 161251 (0.0008) [2023-12-27 06:46:18,612][20381] Updated weights for policy 1, policy_version 161026 (0.0007) [2023-12-27 06:46:18,671][20381] Updated weights for policy 1, policy_version 161036 (0.0007) [2023-12-27 06:46:18,730][20381] Updated weights for policy 1, policy_version 161046 (0.0007) [2023-12-27 06:46:18,810][20347] Updated weights for policy 0, policy_version 161261 (0.0006) [2023-12-27 06:46:18,864][20347] Updated weights for policy 0, policy_version 161271 (0.0010) [2023-12-27 06:46:18,917][20347] Updated weights for policy 0, policy_version 161281 (0.0011) [2023-12-27 06:46:19,500][20381] Updated weights for policy 1, policy_version 161056 (0.0007) [2023-12-27 06:46:19,560][20381] Updated weights for policy 1, policy_version 161066 (0.0008) [2023-12-27 06:46:19,622][20381] Updated weights for policy 1, policy_version 161076 (0.0007) [2023-12-27 06:46:19,632][20347] Updated weights for policy 0, policy_version 161291 (0.0010) [2023-12-27 06:46:19,686][20347] Updated weights for policy 0, policy_version 161301 (0.0010) [2023-12-27 06:46:19,742][20347] Updated weights for policy 0, policy_version 161311 (0.0010) [2023-12-27 06:46:20,345][20381] Updated weights for policy 1, policy_version 161086 (0.0007) [2023-12-27 06:46:20,397][20381] Updated weights for policy 1, policy_version 161096 (0.0009) [2023-12-27 06:46:20,452][20347] Updated weights for policy 0, policy_version 161321 (0.0010) [2023-12-27 06:46:20,459][20381] Updated weights for policy 1, policy_version 161106 (0.0008) [2023-12-27 06:46:20,514][20347] Updated weights for policy 0, policy_version 161331 (0.0010) [2023-12-27 06:46:20,579][20347] Updated weights for policy 0, policy_version 161341 (0.0010) [2023-12-27 06:46:20,635][20347] Updated weights for policy 0, policy_version 161351 (0.0010) [2023-12-27 06:46:21,157][20381] Updated weights for policy 1, policy_version 161116 (0.0008) [2023-12-27 06:46:21,213][20381] Updated weights for policy 1, policy_version 161126 (0.0007) [2023-12-27 06:46:21,274][20381] Updated weights for policy 1, policy_version 161136 (0.0007) [2023-12-27 06:46:21,301][20347] Updated weights for policy 0, policy_version 161361 (0.0010) [2023-12-27 06:46:21,373][20347] Updated weights for policy 0, policy_version 161371 (0.0010) [2023-12-27 06:46:21,438][20347] Updated weights for policy 0, policy_version 161381 (0.0008) [2023-12-27 06:46:21,958][20381] Updated weights for policy 1, policy_version 161146 (0.0007) [2023-12-27 06:46:22,020][20381] Updated weights for policy 1, policy_version 161156 (0.0008) [2023-12-27 06:46:22,072][20381] Updated weights for policy 1, policy_version 161166 (0.0008) [2023-12-27 06:46:22,123][20381] Updated weights for policy 1, policy_version 161176 (0.0007) [2023-12-27 06:46:22,140][20347] Updated weights for policy 0, policy_version 161391 (0.0010) [2023-12-27 06:46:22,199][20347] Updated weights for policy 0, policy_version 161401 (0.0010) [2023-12-27 06:46:22,263][20347] Updated weights for policy 0, policy_version 161411 (0.0011) [2023-12-27 06:46:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 82608128. Throughput: 0: 10341.0, 1: 10239.6. Samples: 82573764. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000161176_41271296.pth... [2023-12-27 06:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000161416_41336832.pth... [2023-12-27 06:46:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000159992_40968192.pth [2023-12-27 06:46:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000160200_41025536.pth [2023-12-27 06:46:22,836][20381] Updated weights for policy 1, policy_version 161186 (0.0008) [2023-12-27 06:46:22,889][20381] Updated weights for policy 1, policy_version 161196 (0.0007) [2023-12-27 06:46:22,939][20381] Updated weights for policy 1, policy_version 161206 (0.0008) [2023-12-27 06:46:22,954][20347] Updated weights for policy 0, policy_version 161421 (0.0010) [2023-12-27 06:46:23,007][20347] Updated weights for policy 0, policy_version 161431 (0.0008) [2023-12-27 06:46:23,062][20347] Updated weights for policy 0, policy_version 161441 (0.0008) [2023-12-27 06:46:23,637][20381] Updated weights for policy 1, policy_version 161216 (0.0008) [2023-12-27 06:46:23,682][20381] Updated weights for policy 1, policy_version 161226 (0.0008) [2023-12-27 06:46:23,739][20381] Updated weights for policy 1, policy_version 161236 (0.0008) [2023-12-27 06:46:23,758][20347] Updated weights for policy 0, policy_version 161451 (0.0008) [2023-12-27 06:46:23,814][20347] Updated weights for policy 0, policy_version 161461 (0.0005) [2023-12-27 06:46:23,872][20347] Updated weights for policy 0, policy_version 161471 (0.0005) [2023-12-27 06:46:24,383][20347] Updated weights for policy 0, policy_version 161481 (0.0005) [2023-12-27 06:46:24,451][20347] Updated weights for policy 0, policy_version 161491 (0.0006) [2023-12-27 06:46:24,504][20381] Updated weights for policy 1, policy_version 161246 (0.0009) [2023-12-27 06:46:24,507][20347] Updated weights for policy 0, policy_version 161501 (0.0006) [2023-12-27 06:46:24,553][20381] Updated weights for policy 1, policy_version 161256 (0.0007) [2023-12-27 06:46:24,567][20347] Updated weights for policy 0, policy_version 161511 (0.0006) [2023-12-27 06:46:24,600][20381] Updated weights for policy 1, policy_version 161266 (0.0007) [2023-12-27 06:46:25,158][20347] Updated weights for policy 0, policy_version 161521 (0.0007) [2023-12-27 06:46:25,208][20347] Updated weights for policy 0, policy_version 161531 (0.0008) [2023-12-27 06:46:25,256][20347] Updated weights for policy 0, policy_version 161541 (0.0007) [2023-12-27 06:46:25,315][20381] Updated weights for policy 1, policy_version 161276 (0.0007) [2023-12-27 06:46:25,370][20381] Updated weights for policy 1, policy_version 161286 (0.0008) [2023-12-27 06:46:25,432][20381] Updated weights for policy 1, policy_version 161296 (0.0008) [2023-12-27 06:46:25,911][20347] Updated weights for policy 0, policy_version 161551 (0.0008) [2023-12-27 06:46:25,960][20347] Updated weights for policy 0, policy_version 161561 (0.0008) [2023-12-27 06:46:26,017][20347] Updated weights for policy 0, policy_version 161571 (0.0006) [2023-12-27 06:46:26,133][20381] Updated weights for policy 1, policy_version 161306 (0.0007) [2023-12-27 06:46:26,183][20381] Updated weights for policy 1, policy_version 161316 (0.0010) [2023-12-27 06:46:26,230][20381] Updated weights for policy 1, policy_version 161326 (0.0009) [2023-12-27 06:46:26,291][20381] Updated weights for policy 1, policy_version 161336 (0.0009) [2023-12-27 06:46:26,571][20347] Updated weights for policy 0, policy_version 161581 (0.0006) [2023-12-27 06:46:26,618][20347] Updated weights for policy 0, policy_version 161591 (0.0006) [2023-12-27 06:46:26,664][20347] Updated weights for policy 0, policy_version 161601 (0.0007) [2023-12-27 06:46:27,042][20381] Updated weights for policy 1, policy_version 161346 (0.0008) [2023-12-27 06:46:27,089][20381] Updated weights for policy 1, policy_version 161356 (0.0006) [2023-12-27 06:46:27,140][20381] Updated weights for policy 1, policy_version 161366 (0.0007) [2023-12-27 06:46:27,222][20347] Updated weights for policy 0, policy_version 161611 (0.0007) [2023-12-27 06:46:27,282][20347] Updated weights for policy 0, policy_version 161621 (0.0006) [2023-12-27 06:46:27,343][20347] Updated weights for policy 0, policy_version 161631 (0.0006) [2023-12-27 06:46:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 82706432. Throughput: 0: 10376.1, 1: 10168.0. Samples: 82695580. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:27,790][20381] Updated weights for policy 1, policy_version 161376 (0.0005) [2023-12-27 06:46:27,855][20381] Updated weights for policy 1, policy_version 161386 (0.0005) [2023-12-27 06:46:27,919][20381] Updated weights for policy 1, policy_version 161396 (0.0005) [2023-12-27 06:46:27,943][20347] Updated weights for policy 0, policy_version 161641 (0.0005) [2023-12-27 06:46:27,991][20347] Updated weights for policy 0, policy_version 161651 (0.0005) [2023-12-27 06:46:28,045][20347] Updated weights for policy 0, policy_version 161661 (0.0005) [2023-12-27 06:46:28,101][20347] Updated weights for policy 0, policy_version 161671 (0.0006) [2023-12-27 06:46:28,454][20381] Updated weights for policy 1, policy_version 161406 (0.0005) [2023-12-27 06:46:28,514][20381] Updated weights for policy 1, policy_version 161416 (0.0005) [2023-12-27 06:46:28,576][20381] Updated weights for policy 1, policy_version 161426 (0.0006) [2023-12-27 06:46:28,744][20347] Updated weights for policy 0, policy_version 161681 (0.0010) [2023-12-27 06:46:28,801][20347] Updated weights for policy 0, policy_version 161691 (0.0010) [2023-12-27 06:46:28,858][20347] Updated weights for policy 0, policy_version 161701 (0.0010) [2023-12-27 06:46:29,227][20381] Updated weights for policy 1, policy_version 161436 (0.0008) [2023-12-27 06:46:29,285][20381] Updated weights for policy 1, policy_version 161446 (0.0007) [2023-12-27 06:46:29,346][20381] Updated weights for policy 1, policy_version 161456 (0.0008) [2023-12-27 06:46:29,503][20347] Updated weights for policy 0, policy_version 161711 (0.0010) [2023-12-27 06:46:29,560][20347] Updated weights for policy 0, policy_version 161721 (0.0010) [2023-12-27 06:46:29,611][20347] Updated weights for policy 0, policy_version 161731 (0.0010) [2023-12-27 06:46:30,033][20381] Updated weights for policy 1, policy_version 161466 (0.0008) [2023-12-27 06:46:30,085][20381] Updated weights for policy 1, policy_version 161476 (0.0007) [2023-12-27 06:46:30,142][20381] Updated weights for policy 1, policy_version 161486 (0.0007) [2023-12-27 06:46:30,193][20381] Updated weights for policy 1, policy_version 161496 (0.0007) [2023-12-27 06:46:30,292][20347] Updated weights for policy 0, policy_version 161741 (0.0010) [2023-12-27 06:46:30,352][20347] Updated weights for policy 0, policy_version 161751 (0.0010) [2023-12-27 06:46:30,409][20347] Updated weights for policy 0, policy_version 161761 (0.0010) [2023-12-27 06:46:30,888][20381] Updated weights for policy 1, policy_version 161506 (0.0007) [2023-12-27 06:46:30,940][20381] Updated weights for policy 1, policy_version 161516 (0.0007) [2023-12-27 06:46:30,993][20381] Updated weights for policy 1, policy_version 161526 (0.0007) [2023-12-27 06:46:31,072][20347] Updated weights for policy 0, policy_version 161771 (0.0010) [2023-12-27 06:46:31,125][20347] Updated weights for policy 0, policy_version 161781 (0.0010) [2023-12-27 06:46:31,188][20347] Updated weights for policy 0, policy_version 161791 (0.0010) [2023-12-27 06:46:31,702][20381] Updated weights for policy 1, policy_version 161536 (0.0007) [2023-12-27 06:46:31,764][20381] Updated weights for policy 1, policy_version 161546 (0.0007) [2023-12-27 06:46:31,822][20381] Updated weights for policy 1, policy_version 161556 (0.0007) [2023-12-27 06:46:31,882][20347] Updated weights for policy 0, policy_version 161801 (0.0010) [2023-12-27 06:46:31,934][20347] Updated weights for policy 0, policy_version 161811 (0.0008) [2023-12-27 06:46:31,993][20347] Updated weights for policy 0, policy_version 161821 (0.0007) [2023-12-27 06:46:32,046][20347] Updated weights for policy 0, policy_version 161831 (0.0008) [2023-12-27 06:46:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 82812928. Throughput: 0: 10490.1, 1: 10147.2. Samples: 82823016. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:32,501][20381] Updated weights for policy 1, policy_version 161566 (0.0007) [2023-12-27 06:46:32,545][20381] Updated weights for policy 1, policy_version 161576 (0.0007) [2023-12-27 06:46:32,595][20381] Updated weights for policy 1, policy_version 161586 (0.0007) [2023-12-27 06:46:32,727][20347] Updated weights for policy 0, policy_version 161841 (0.0008) [2023-12-27 06:46:32,776][20347] Updated weights for policy 0, policy_version 161851 (0.0008) [2023-12-27 06:46:32,828][20347] Updated weights for policy 0, policy_version 161861 (0.0008) [2023-12-27 06:46:33,309][20381] Updated weights for policy 1, policy_version 161596 (0.0007) [2023-12-27 06:46:33,354][20381] Updated weights for policy 1, policy_version 161606 (0.0009) [2023-12-27 06:46:33,401][20381] Updated weights for policy 1, policy_version 161616 (0.0008) [2023-12-27 06:46:33,456][20347] Updated weights for policy 0, policy_version 161871 (0.0007) [2023-12-27 06:46:33,501][20347] Updated weights for policy 0, policy_version 161881 (0.0007) [2023-12-27 06:46:33,544][20347] Updated weights for policy 0, policy_version 161891 (0.0005) [2023-12-27 06:46:34,133][20381] Updated weights for policy 1, policy_version 161626 (0.0007) [2023-12-27 06:46:34,138][20347] Updated weights for policy 0, policy_version 161901 (0.0006) [2023-12-27 06:46:34,191][20347] Updated weights for policy 0, policy_version 161911 (0.0008) [2023-12-27 06:46:34,197][20381] Updated weights for policy 1, policy_version 161636 (0.0008) [2023-12-27 06:46:34,244][20347] Updated weights for policy 0, policy_version 161921 (0.0006) [2023-12-27 06:46:34,254][20381] Updated weights for policy 1, policy_version 161646 (0.0007) [2023-12-27 06:46:34,317][20381] Updated weights for policy 1, policy_version 161656 (0.0007) [2023-12-27 06:46:34,985][20381] Updated weights for policy 1, policy_version 161666 (0.0007) [2023-12-27 06:46:34,987][20347] Updated weights for policy 0, policy_version 161931 (0.0008) [2023-12-27 06:46:35,048][20381] Updated weights for policy 1, policy_version 161676 (0.0007) [2023-12-27 06:46:35,049][20347] Updated weights for policy 0, policy_version 161941 (0.0006) [2023-12-27 06:46:35,100][20381] Updated weights for policy 1, policy_version 161686 (0.0005) [2023-12-27 06:46:35,110][20347] Updated weights for policy 0, policy_version 161951 (0.0006) [2023-12-27 06:46:35,679][20347] Updated weights for policy 0, policy_version 161961 (0.0006) [2023-12-27 06:46:35,747][20347] Updated weights for policy 0, policy_version 161971 (0.0009) [2023-12-27 06:46:35,813][20347] Updated weights for policy 0, policy_version 161981 (0.0006) [2023-12-27 06:46:35,834][20381] Updated weights for policy 1, policy_version 161696 (0.0007) [2023-12-27 06:46:35,870][20347] Updated weights for policy 0, policy_version 161991 (0.0005) [2023-12-27 06:46:35,895][20381] Updated weights for policy 1, policy_version 161706 (0.0007) [2023-12-27 06:46:35,958][20381] Updated weights for policy 1, policy_version 161716 (0.0007) [2023-12-27 06:46:36,494][20347] Updated weights for policy 0, policy_version 162001 (0.0005) [2023-12-27 06:46:36,544][20347] Updated weights for policy 0, policy_version 162011 (0.0008) [2023-12-27 06:46:36,592][20347] Updated weights for policy 0, policy_version 162021 (0.0005) [2023-12-27 06:46:36,704][20381] Updated weights for policy 1, policy_version 161726 (0.0008) [2023-12-27 06:46:36,761][20381] Updated weights for policy 1, policy_version 161736 (0.0009) [2023-12-27 06:46:36,816][20381] Updated weights for policy 1, policy_version 161746 (0.0008) [2023-12-27 06:46:37,202][20347] Updated weights for policy 0, policy_version 162031 (0.0009) [2023-12-27 06:46:37,267][20347] Updated weights for policy 0, policy_version 162041 (0.0010) [2023-12-27 06:46:37,323][20347] Updated weights for policy 0, policy_version 162051 (0.0010) [2023-12-27 06:46:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 82919424. Throughput: 0: 10548.8, 1: 10122.5. Samples: 82884948. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000162056_41500672.pth... [2023-12-27 06:46:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000161752_41418752.pth... [2023-12-27 06:46:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000160808_41181184.pth [2023-12-27 06:46:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000160568_41115648.pth [2023-12-27 06:46:37,473][20381] Updated weights for policy 1, policy_version 161756 (0.0009) [2023-12-27 06:46:37,525][20381] Updated weights for policy 1, policy_version 161766 (0.0009) [2023-12-27 06:46:37,575][20381] Updated weights for policy 1, policy_version 161776 (0.0007) [2023-12-27 06:46:37,999][20347] Updated weights for policy 0, policy_version 162061 (0.0010) [2023-12-27 06:46:38,066][20347] Updated weights for policy 0, policy_version 162071 (0.0010) [2023-12-27 06:46:38,121][20347] Updated weights for policy 0, policy_version 162081 (0.0010) [2023-12-27 06:46:38,290][20381] Updated weights for policy 1, policy_version 161786 (0.0008) [2023-12-27 06:46:38,350][20381] Updated weights for policy 1, policy_version 161796 (0.0011) [2023-12-27 06:46:38,407][20381] Updated weights for policy 1, policy_version 161806 (0.0011) [2023-12-27 06:46:38,464][20381] Updated weights for policy 1, policy_version 161816 (0.0011) [2023-12-27 06:46:38,775][20347] Updated weights for policy 0, policy_version 162091 (0.0009) [2023-12-27 06:46:38,827][20347] Updated weights for policy 0, policy_version 162101 (0.0006) [2023-12-27 06:46:38,886][20347] Updated weights for policy 0, policy_version 162111 (0.0011) [2023-12-27 06:46:39,180][20381] Updated weights for policy 1, policy_version 161826 (0.0008) [2023-12-27 06:46:39,244][20381] Updated weights for policy 1, policy_version 161836 (0.0006) [2023-12-27 06:46:39,313][20381] Updated weights for policy 1, policy_version 161846 (0.0010) [2023-12-27 06:46:39,544][20347] Updated weights for policy 0, policy_version 162121 (0.0010) [2023-12-27 06:46:39,600][20347] Updated weights for policy 0, policy_version 162131 (0.0007) [2023-12-27 06:46:39,663][20347] Updated weights for policy 0, policy_version 162141 (0.0005) [2023-12-27 06:46:39,724][20347] Updated weights for policy 0, policy_version 162151 (0.0008) [2023-12-27 06:46:39,989][20381] Updated weights for policy 1, policy_version 161856 (0.0008) [2023-12-27 06:46:40,042][20381] Updated weights for policy 1, policy_version 161866 (0.0007) [2023-12-27 06:46:40,095][20381] Updated weights for policy 1, policy_version 161876 (0.0008) [2023-12-27 06:46:40,347][20347] Updated weights for policy 0, policy_version 162161 (0.0006) [2023-12-27 06:46:40,407][20347] Updated weights for policy 0, policy_version 162171 (0.0007) [2023-12-27 06:46:40,462][20347] Updated weights for policy 0, policy_version 162181 (0.0007) [2023-12-27 06:46:40,786][20381] Updated weights for policy 1, policy_version 161886 (0.0009) [2023-12-27 06:46:40,842][20381] Updated weights for policy 1, policy_version 161896 (0.0011) [2023-12-27 06:46:40,897][20381] Updated weights for policy 1, policy_version 161906 (0.0011) [2023-12-27 06:46:41,135][20347] Updated weights for policy 0, policy_version 162191 (0.0007) [2023-12-27 06:46:41,202][20347] Updated weights for policy 0, policy_version 162201 (0.0008) [2023-12-27 06:46:41,259][20347] Updated weights for policy 0, policy_version 162211 (0.0009) [2023-12-27 06:46:41,577][20381] Updated weights for policy 1, policy_version 161916 (0.0009) [2023-12-27 06:46:41,637][20381] Updated weights for policy 1, policy_version 161926 (0.0008) [2023-12-27 06:46:41,692][20381] Updated weights for policy 1, policy_version 161936 (0.0008) [2023-12-27 06:46:41,966][20347] Updated weights for policy 0, policy_version 162221 (0.0007) [2023-12-27 06:46:42,020][20347] Updated weights for policy 0, policy_version 162231 (0.0007) [2023-12-27 06:46:42,075][20347] Updated weights for policy 0, policy_version 162241 (0.0009) [2023-12-27 06:46:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 83017728. Throughput: 0: 10600.0, 1: 10047.6. Samples: 83008332. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:42,367][20381] Updated weights for policy 1, policy_version 161946 (0.0008) [2023-12-27 06:46:42,423][20381] Updated weights for policy 1, policy_version 161956 (0.0008) [2023-12-27 06:46:42,479][20381] Updated weights for policy 1, policy_version 161966 (0.0008) [2023-12-27 06:46:42,538][20381] Updated weights for policy 1, policy_version 161976 (0.0006) [2023-12-27 06:46:42,795][20347] Updated weights for policy 0, policy_version 162251 (0.0009) [2023-12-27 06:46:42,847][20347] Updated weights for policy 0, policy_version 162261 (0.0008) [2023-12-27 06:46:42,895][20347] Updated weights for policy 0, policy_version 162271 (0.0008) [2023-12-27 06:46:43,175][20381] Updated weights for policy 1, policy_version 161986 (0.0008) [2023-12-27 06:46:43,236][20381] Updated weights for policy 1, policy_version 161996 (0.0006) [2023-12-27 06:46:43,289][20381] Updated weights for policy 1, policy_version 162006 (0.0006) [2023-12-27 06:46:43,591][20347] Updated weights for policy 0, policy_version 162281 (0.0007) [2023-12-27 06:46:43,641][20347] Updated weights for policy 0, policy_version 162291 (0.0009) [2023-12-27 06:46:43,687][20347] Updated weights for policy 0, policy_version 162301 (0.0008) [2023-12-27 06:46:43,741][20347] Updated weights for policy 0, policy_version 162311 (0.0006) [2023-12-27 06:46:43,870][20381] Updated weights for policy 1, policy_version 162016 (0.0005) [2023-12-27 06:46:43,924][20381] Updated weights for policy 1, policy_version 162026 (0.0006) [2023-12-27 06:46:43,979][20381] Updated weights for policy 1, policy_version 162036 (0.0006) [2023-12-27 06:46:44,469][20347] Updated weights for policy 0, policy_version 162321 (0.0006) [2023-12-27 06:46:44,531][20347] Updated weights for policy 0, policy_version 162331 (0.0006) [2023-12-27 06:46:44,586][20347] Updated weights for policy 0, policy_version 162341 (0.0005) [2023-12-27 06:46:44,608][20381] Updated weights for policy 1, policy_version 162046 (0.0008) [2023-12-27 06:46:44,662][20381] Updated weights for policy 1, policy_version 162056 (0.0008) [2023-12-27 06:46:44,717][20381] Updated weights for policy 1, policy_version 162066 (0.0008) [2023-12-27 06:46:45,210][20347] Updated weights for policy 0, policy_version 162351 (0.0006) [2023-12-27 06:46:45,266][20347] Updated weights for policy 0, policy_version 162361 (0.0008) [2023-12-27 06:46:45,325][20347] Updated weights for policy 0, policy_version 162371 (0.0007) [2023-12-27 06:46:45,450][20381] Updated weights for policy 1, policy_version 162076 (0.0008) [2023-12-27 06:46:45,501][20381] Updated weights for policy 1, policy_version 162086 (0.0009) [2023-12-27 06:46:45,554][20381] Updated weights for policy 1, policy_version 162096 (0.0008) [2023-12-27 06:46:45,948][20347] Updated weights for policy 0, policy_version 162381 (0.0008) [2023-12-27 06:46:46,003][20347] Updated weights for policy 0, policy_version 162391 (0.0006) [2023-12-27 06:46:46,055][20347] Updated weights for policy 0, policy_version 162401 (0.0005) [2023-12-27 06:46:46,252][20381] Updated weights for policy 1, policy_version 162106 (0.0008) [2023-12-27 06:46:46,304][20381] Updated weights for policy 1, policy_version 162116 (0.0007) [2023-12-27 06:46:46,356][20381] Updated weights for policy 1, policy_version 162126 (0.0007) [2023-12-27 06:46:46,416][20381] Updated weights for policy 1, policy_version 162136 (0.0006) [2023-12-27 06:46:46,740][20347] Updated weights for policy 0, policy_version 162411 (0.0006) [2023-12-27 06:46:46,796][20347] Updated weights for policy 0, policy_version 162421 (0.0006) [2023-12-27 06:46:46,844][20347] Updated weights for policy 0, policy_version 162431 (0.0005) [2023-12-27 06:46:47,035][20381] Updated weights for policy 1, policy_version 162146 (0.0005) [2023-12-27 06:46:47,088][20381] Updated weights for policy 1, policy_version 162156 (0.0005) [2023-12-27 06:46:47,138][20381] Updated weights for policy 1, policy_version 162166 (0.0007) [2023-12-27 06:46:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 83124224. Throughput: 0: 10611.6, 1: 10125.0. Samples: 83133996. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:47,510][20347] Updated weights for policy 0, policy_version 162441 (0.0005) [2023-12-27 06:46:47,564][20347] Updated weights for policy 0, policy_version 162451 (0.0008) [2023-12-27 06:46:47,620][20347] Updated weights for policy 0, policy_version 162461 (0.0007) [2023-12-27 06:46:47,677][20347] Updated weights for policy 0, policy_version 162471 (0.0005) [2023-12-27 06:46:47,678][20381] Updated weights for policy 1, policy_version 162176 (0.0007) [2023-12-27 06:46:47,741][20381] Updated weights for policy 1, policy_version 162186 (0.0009) [2023-12-27 06:46:47,798][20381] Updated weights for policy 1, policy_version 162196 (0.0007) [2023-12-27 06:46:48,272][20347] Updated weights for policy 0, policy_version 162481 (0.0008) [2023-12-27 06:46:48,327][20347] Updated weights for policy 0, policy_version 162491 (0.0009) [2023-12-27 06:46:48,378][20381] Updated weights for policy 1, policy_version 162206 (0.0007) [2023-12-27 06:46:48,384][20347] Updated weights for policy 0, policy_version 162501 (0.0006) [2023-12-27 06:46:48,432][20381] Updated weights for policy 1, policy_version 162216 (0.0007) [2023-12-27 06:46:48,481][20381] Updated weights for policy 1, policy_version 162226 (0.0006) [2023-12-27 06:46:49,042][20347] Updated weights for policy 0, policy_version 162511 (0.0008) [2023-12-27 06:46:49,069][20381] Updated weights for policy 1, policy_version 162236 (0.0006) [2023-12-27 06:46:49,098][20347] Updated weights for policy 0, policy_version 162521 (0.0005) [2023-12-27 06:46:49,125][20381] Updated weights for policy 1, policy_version 162246 (0.0006) [2023-12-27 06:46:49,152][20347] Updated weights for policy 0, policy_version 162531 (0.0006) [2023-12-27 06:46:49,187][20381] Updated weights for policy 1, policy_version 162256 (0.0006) [2023-12-27 06:46:49,855][20381] Updated weights for policy 1, policy_version 162266 (0.0009) [2023-12-27 06:46:49,892][20347] Updated weights for policy 0, policy_version 162541 (0.0005) [2023-12-27 06:46:49,906][20381] Updated weights for policy 1, policy_version 162276 (0.0006) [2023-12-27 06:46:49,961][20347] Updated weights for policy 0, policy_version 162551 (0.0009) [2023-12-27 06:46:49,968][20381] Updated weights for policy 1, policy_version 162286 (0.0008) [2023-12-27 06:46:50,022][20347] Updated weights for policy 0, policy_version 162561 (0.0007) [2023-12-27 06:46:50,024][20381] Updated weights for policy 1, policy_version 162296 (0.0006) [2023-12-27 06:46:50,664][20347] Updated weights for policy 0, policy_version 162571 (0.0007) [2023-12-27 06:46:50,725][20347] Updated weights for policy 0, policy_version 162581 (0.0008) [2023-12-27 06:46:50,786][20381] Updated weights for policy 1, policy_version 162306 (0.0009) [2023-12-27 06:46:50,787][20347] Updated weights for policy 0, policy_version 162591 (0.0008) [2023-12-27 06:46:50,839][20381] Updated weights for policy 1, policy_version 162316 (0.0007) [2023-12-27 06:46:50,892][20381] Updated weights for policy 1, policy_version 162326 (0.0008) [2023-12-27 06:46:51,549][20347] Updated weights for policy 0, policy_version 162601 (0.0008) [2023-12-27 06:46:51,609][20347] Updated weights for policy 0, policy_version 162611 (0.0007) [2023-12-27 06:46:51,615][20381] Updated weights for policy 1, policy_version 162336 (0.0009) [2023-12-27 06:46:51,674][20347] Updated weights for policy 0, policy_version 162621 (0.0007) [2023-12-27 06:46:51,674][20381] Updated weights for policy 1, policy_version 162346 (0.0008) [2023-12-27 06:46:51,728][20347] Updated weights for policy 0, policy_version 162631 (0.0007) [2023-12-27 06:46:51,734][20381] Updated weights for policy 1, policy_version 162356 (0.0007) [2023-12-27 06:46:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 83222528. Throughput: 0: 10560.4, 1: 10208.9. Samples: 83197108. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000162360_41574400.pth... [2023-12-27 06:46:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000162632_41648128.pth... [2023-12-27 06:46:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000161176_41271296.pth [2023-12-27 06:46:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000161416_41336832.pth [2023-12-27 06:46:52,437][20347] Updated weights for policy 0, policy_version 162641 (0.0007) [2023-12-27 06:46:52,442][20381] Updated weights for policy 1, policy_version 162366 (0.0007) [2023-12-27 06:46:52,492][20347] Updated weights for policy 0, policy_version 162651 (0.0006) [2023-12-27 06:46:52,498][20381] Updated weights for policy 1, policy_version 162376 (0.0005) [2023-12-27 06:46:52,552][20347] Updated weights for policy 0, policy_version 162661 (0.0007) [2023-12-27 06:46:52,558][20381] Updated weights for policy 1, policy_version 162386 (0.0006) [2023-12-27 06:46:53,227][20381] Updated weights for policy 1, policy_version 162396 (0.0007) [2023-12-27 06:46:53,256][20347] Updated weights for policy 0, policy_version 162671 (0.0007) [2023-12-27 06:46:53,281][20381] Updated weights for policy 1, policy_version 162406 (0.0009) [2023-12-27 06:46:53,303][20347] Updated weights for policy 0, policy_version 162681 (0.0008) [2023-12-27 06:46:53,335][20381] Updated weights for policy 1, policy_version 162416 (0.0008) [2023-12-27 06:46:53,349][20347] Updated weights for policy 0, policy_version 162691 (0.0006) [2023-12-27 06:46:53,905][20381] Updated weights for policy 1, policy_version 162426 (0.0006) [2023-12-27 06:46:53,961][20381] Updated weights for policy 1, policy_version 162436 (0.0009) [2023-12-27 06:46:54,014][20381] Updated weights for policy 1, policy_version 162446 (0.0009) [2023-12-27 06:46:54,068][20381] Updated weights for policy 1, policy_version 162456 (0.0005) [2023-12-27 06:46:54,082][20347] Updated weights for policy 0, policy_version 162701 (0.0007) [2023-12-27 06:46:54,133][20347] Updated weights for policy 0, policy_version 162711 (0.0007) [2023-12-27 06:46:54,193][20347] Updated weights for policy 0, policy_version 162721 (0.0009) [2023-12-27 06:46:54,722][20381] Updated weights for policy 1, policy_version 162466 (0.0008) [2023-12-27 06:46:54,775][20381] Updated weights for policy 1, policy_version 162476 (0.0008) [2023-12-27 06:46:54,829][20381] Updated weights for policy 1, policy_version 162486 (0.0010) [2023-12-27 06:46:54,873][20347] Updated weights for policy 0, policy_version 162731 (0.0008) [2023-12-27 06:46:54,933][20347] Updated weights for policy 0, policy_version 162741 (0.0007) [2023-12-27 06:46:54,992][20347] Updated weights for policy 0, policy_version 162751 (0.0007) [2023-12-27 06:46:55,549][20347] Updated weights for policy 0, policy_version 162761 (0.0005) [2023-12-27 06:46:55,573][20381] Updated weights for policy 1, policy_version 162496 (0.0010) [2023-12-27 06:46:55,600][20347] Updated weights for policy 0, policy_version 162771 (0.0005) [2023-12-27 06:46:55,627][20381] Updated weights for policy 1, policy_version 162506 (0.0010) [2023-12-27 06:46:55,658][20347] Updated weights for policy 0, policy_version 162781 (0.0005) [2023-12-27 06:46:55,682][20381] Updated weights for policy 1, policy_version 162516 (0.0010) [2023-12-27 06:46:55,713][20347] Updated weights for policy 0, policy_version 162791 (0.0006) [2023-12-27 06:46:56,268][20381] Updated weights for policy 1, policy_version 162526 (0.0007) [2023-12-27 06:46:56,317][20381] Updated weights for policy 1, policy_version 162536 (0.0005) [2023-12-27 06:46:56,372][20381] Updated weights for policy 1, policy_version 162546 (0.0007) [2023-12-27 06:46:56,390][20347] Updated weights for policy 0, policy_version 162801 (0.0007) [2023-12-27 06:46:56,455][20347] Updated weights for policy 0, policy_version 162811 (0.0006) [2023-12-27 06:46:56,508][20347] Updated weights for policy 0, policy_version 162821 (0.0005) [2023-12-27 06:46:57,029][20381] Updated weights for policy 1, policy_version 162556 (0.0006) [2023-12-27 06:46:57,072][20381] Updated weights for policy 1, policy_version 162566 (0.0010) [2023-12-27 06:46:57,130][20381] Updated weights for policy 1, policy_version 162576 (0.0010) [2023-12-27 06:46:57,140][20347] Updated weights for policy 0, policy_version 162831 (0.0005) [2023-12-27 06:46:57,195][20347] Updated weights for policy 0, policy_version 162841 (0.0006) [2023-12-27 06:46:57,245][20347] Updated weights for policy 0, policy_version 162851 (0.0007) [2023-12-27 06:46:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 83337216. Throughput: 0: 10494.6, 1: 10249.0. Samples: 83320980. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:46:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:46:57,736][20381] Updated weights for policy 1, policy_version 162586 (0.0010) [2023-12-27 06:46:57,789][20381] Updated weights for policy 1, policy_version 162596 (0.0006) [2023-12-27 06:46:57,844][20381] Updated weights for policy 1, policy_version 162606 (0.0005) [2023-12-27 06:46:57,892][20381] Updated weights for policy 1, policy_version 162616 (0.0005) [2023-12-27 06:46:57,959][20347] Updated weights for policy 0, policy_version 162861 (0.0008) [2023-12-27 06:46:58,012][20347] Updated weights for policy 0, policy_version 162871 (0.0008) [2023-12-27 06:46:58,066][20347] Updated weights for policy 0, policy_version 162881 (0.0005) [2023-12-27 06:46:58,523][20381] Updated weights for policy 1, policy_version 162626 (0.0008) [2023-12-27 06:46:58,583][20381] Updated weights for policy 1, policy_version 162636 (0.0008) [2023-12-27 06:46:58,639][20381] Updated weights for policy 1, policy_version 162646 (0.0008) [2023-12-27 06:46:58,763][20347] Updated weights for policy 0, policy_version 162891 (0.0007) [2023-12-27 06:46:58,834][20347] Updated weights for policy 0, policy_version 162901 (0.0010) [2023-12-27 06:46:58,901][20347] Updated weights for policy 0, policy_version 162911 (0.0009) [2023-12-27 06:46:59,390][20381] Updated weights for policy 1, policy_version 162656 (0.0008) [2023-12-27 06:46:59,445][20381] Updated weights for policy 1, policy_version 162666 (0.0008) [2023-12-27 06:46:59,494][20381] Updated weights for policy 1, policy_version 162676 (0.0009) [2023-12-27 06:46:59,606][20347] Updated weights for policy 0, policy_version 162921 (0.0006) [2023-12-27 06:46:59,652][20347] Updated weights for policy 0, policy_version 162931 (0.0008) [2023-12-27 06:46:59,699][20347] Updated weights for policy 0, policy_version 162941 (0.0007) [2023-12-27 06:46:59,746][20347] Updated weights for policy 0, policy_version 162951 (0.0008) [2023-12-27 06:47:00,230][20381] Updated weights for policy 1, policy_version 162686 (0.0011) [2023-12-27 06:47:00,293][20381] Updated weights for policy 1, policy_version 162696 (0.0011) [2023-12-27 06:47:00,354][20381] Updated weights for policy 1, policy_version 162706 (0.0010) [2023-12-27 06:47:00,428][20347] Updated weights for policy 0, policy_version 162961 (0.0007) [2023-12-27 06:47:00,485][20347] Updated weights for policy 0, policy_version 162971 (0.0008) [2023-12-27 06:47:00,534][20347] Updated weights for policy 0, policy_version 162981 (0.0007) [2023-12-27 06:47:01,010][20381] Updated weights for policy 1, policy_version 162716 (0.0010) [2023-12-27 06:47:01,068][20381] Updated weights for policy 1, policy_version 162726 (0.0011) [2023-12-27 06:47:01,121][20381] Updated weights for policy 1, policy_version 162736 (0.0010) [2023-12-27 06:47:01,202][20347] Updated weights for policy 0, policy_version 162991 (0.0007) [2023-12-27 06:47:01,258][20347] Updated weights for policy 0, policy_version 163001 (0.0006) [2023-12-27 06:47:01,318][20347] Updated weights for policy 0, policy_version 163011 (0.0006) [2023-12-27 06:47:01,792][20381] Updated weights for policy 1, policy_version 162746 (0.0009) [2023-12-27 06:47:01,843][20381] Updated weights for policy 1, policy_version 162756 (0.0006) [2023-12-27 06:47:01,900][20381] Updated weights for policy 1, policy_version 162766 (0.0006) [2023-12-27 06:47:01,962][20381] Updated weights for policy 1, policy_version 162776 (0.0007) [2023-12-27 06:47:02,021][20347] Updated weights for policy 0, policy_version 163021 (0.0008) [2023-12-27 06:47:02,074][20347] Updated weights for policy 0, policy_version 163031 (0.0008) [2023-12-27 06:47:02,134][20347] Updated weights for policy 0, policy_version 163041 (0.0008) [2023-12-27 06:47:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 83435520. Throughput: 0: 10440.2, 1: 10224.3. Samples: 83443488. Policy #0 lag: (min: 7.0, avg: 19.9, max: 39.0) [2023-12-27 06:47:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:02,574][20381] Updated weights for policy 1, policy_version 162786 (0.0006) [2023-12-27 06:47:02,620][20381] Updated weights for policy 1, policy_version 162796 (0.0010) [2023-12-27 06:47:02,669][20381] Updated weights for policy 1, policy_version 162806 (0.0009) [2023-12-27 06:47:02,811][20347] Updated weights for policy 0, policy_version 163051 (0.0008) [2023-12-27 06:47:02,862][20347] Updated weights for policy 0, policy_version 163061 (0.0007) [2023-12-27 06:47:02,917][20347] Updated weights for policy 0, policy_version 163071 (0.0007) [2023-12-27 06:47:03,386][20381] Updated weights for policy 1, policy_version 162816 (0.0007) [2023-12-27 06:47:03,431][20381] Updated weights for policy 1, policy_version 162826 (0.0007) [2023-12-27 06:47:03,476][20381] Updated weights for policy 1, policy_version 162836 (0.0007) [2023-12-27 06:47:03,620][20347] Updated weights for policy 0, policy_version 163081 (0.0008) [2023-12-27 06:47:03,669][20347] Updated weights for policy 0, policy_version 163091 (0.0007) [2023-12-27 06:47:03,720][20347] Updated weights for policy 0, policy_version 163101 (0.0007) [2023-12-27 06:47:03,768][20347] Updated weights for policy 0, policy_version 163111 (0.0008) [2023-12-27 06:47:04,122][20381] Updated weights for policy 1, policy_version 162846 (0.0008) [2023-12-27 06:47:04,183][20381] Updated weights for policy 1, policy_version 162856 (0.0008) [2023-12-27 06:47:04,237][20381] Updated weights for policy 1, policy_version 162866 (0.0008) [2023-12-27 06:47:04,499][20347] Updated weights for policy 0, policy_version 163121 (0.0007) [2023-12-27 06:47:04,560][20347] Updated weights for policy 0, policy_version 163131 (0.0007) [2023-12-27 06:47:04,607][20347] Updated weights for policy 0, policy_version 163141 (0.0006) [2023-12-27 06:47:04,876][20381] Updated weights for policy 1, policy_version 162876 (0.0006) [2023-12-27 06:47:04,937][20381] Updated weights for policy 1, policy_version 162886 (0.0005) [2023-12-27 06:47:04,996][20381] Updated weights for policy 1, policy_version 162896 (0.0005) [2023-12-27 06:47:05,225][20347] Updated weights for policy 0, policy_version 163151 (0.0007) [2023-12-27 06:47:05,275][20347] Updated weights for policy 0, policy_version 163161 (0.0008) [2023-12-27 06:47:05,325][20347] Updated weights for policy 0, policy_version 163171 (0.0007) [2023-12-27 06:47:05,541][20381] Updated weights for policy 1, policy_version 162906 (0.0008) [2023-12-27 06:47:05,589][20381] Updated weights for policy 1, policy_version 162916 (0.0008) [2023-12-27 06:47:05,637][20381] Updated weights for policy 1, policy_version 162926 (0.0008) [2023-12-27 06:47:05,689][20381] Updated weights for policy 1, policy_version 162936 (0.0008) [2023-12-27 06:47:06,112][20347] Updated weights for policy 0, policy_version 163181 (0.0008) [2023-12-27 06:47:06,180][20347] Updated weights for policy 0, policy_version 163191 (0.0006) [2023-12-27 06:47:06,237][20347] Updated weights for policy 0, policy_version 163201 (0.0008) [2023-12-27 06:47:06,456][20381] Updated weights for policy 1, policy_version 162946 (0.0008) [2023-12-27 06:47:06,512][20381] Updated weights for policy 1, policy_version 162956 (0.0008) [2023-12-27 06:47:06,561][20381] Updated weights for policy 1, policy_version 162966 (0.0007) [2023-12-27 06:47:06,867][20347] Updated weights for policy 0, policy_version 163211 (0.0008) [2023-12-27 06:47:06,926][20347] Updated weights for policy 0, policy_version 163221 (0.0007) [2023-12-27 06:47:06,993][20347] Updated weights for policy 0, policy_version 163231 (0.0007) [2023-12-27 06:47:07,266][20381] Updated weights for policy 1, policy_version 162976 (0.0006) [2023-12-27 06:47:07,317][20381] Updated weights for policy 1, policy_version 162986 (0.0006) [2023-12-27 06:47:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 83533824. Throughput: 0: 10426.9, 1: 10303.0. Samples: 83506612. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000163240_41803776.pth... [2023-12-27 06:47:07,368][20381] Updated weights for policy 1, policy_version 162996 (0.0008) [2023-12-27 06:47:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000162056_41500672.pth [2023-12-27 06:47:07,389][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000163000_41738240.pth... [2023-12-27 06:47:07,395][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000161752_41418752.pth [2023-12-27 06:47:07,681][20347] Updated weights for policy 0, policy_version 163241 (0.0007) [2023-12-27 06:47:07,739][20347] Updated weights for policy 0, policy_version 163251 (0.0008) [2023-12-27 06:47:07,795][20347] Updated weights for policy 0, policy_version 163262 (0.0008) [2023-12-27 06:47:07,844][20347] Updated weights for policy 0, policy_version 163272 (0.0010) [2023-12-27 06:47:08,039][20381] Updated weights for policy 1, policy_version 163006 (0.0008) [2023-12-27 06:47:08,093][20381] Updated weights for policy 1, policy_version 163016 (0.0007) [2023-12-27 06:47:08,149][20381] Updated weights for policy 1, policy_version 163026 (0.0006) [2023-12-27 06:47:08,496][20347] Updated weights for policy 0, policy_version 163282 (0.0005) [2023-12-27 06:47:08,555][20347] Updated weights for policy 0, policy_version 163292 (0.0006) [2023-12-27 06:47:08,612][20347] Updated weights for policy 0, policy_version 163302 (0.0008) [2023-12-27 06:47:08,805][20381] Updated weights for policy 1, policy_version 163036 (0.0006) [2023-12-27 06:47:08,861][20381] Updated weights for policy 1, policy_version 163046 (0.0010) [2023-12-27 06:47:08,923][20381] Updated weights for policy 1, policy_version 163056 (0.0010) [2023-12-27 06:47:09,231][20347] Updated weights for policy 0, policy_version 163312 (0.0011) [2023-12-27 06:47:09,293][20347] Updated weights for policy 0, policy_version 163322 (0.0011) [2023-12-27 06:47:09,355][20347] Updated weights for policy 0, policy_version 163332 (0.0010) [2023-12-27 06:47:09,550][20381] Updated weights for policy 1, policy_version 163066 (0.0010) [2023-12-27 06:47:09,598][20381] Updated weights for policy 1, policy_version 163076 (0.0007) [2023-12-27 06:47:09,653][20381] Updated weights for policy 1, policy_version 163086 (0.0008) [2023-12-27 06:47:09,709][20381] Updated weights for policy 1, policy_version 163096 (0.0008) [2023-12-27 06:47:10,127][20347] Updated weights for policy 0, policy_version 163342 (0.0006) [2023-12-27 06:47:10,194][20347] Updated weights for policy 0, policy_version 163352 (0.0008) [2023-12-27 06:47:10,250][20347] Updated weights for policy 0, policy_version 163362 (0.0008) [2023-12-27 06:47:10,420][20381] Updated weights for policy 1, policy_version 163106 (0.0008) [2023-12-27 06:47:10,465][20381] Updated weights for policy 1, policy_version 163116 (0.0009) [2023-12-27 06:47:10,509][20381] Updated weights for policy 1, policy_version 163126 (0.0008) [2023-12-27 06:47:10,835][20347] Updated weights for policy 0, policy_version 163372 (0.0008) [2023-12-27 06:47:10,889][20347] Updated weights for policy 0, policy_version 163382 (0.0009) [2023-12-27 06:47:10,954][20347] Updated weights for policy 0, policy_version 163392 (0.0010) [2023-12-27 06:47:11,287][20381] Updated weights for policy 1, policy_version 163136 (0.0009) [2023-12-27 06:47:11,354][20381] Updated weights for policy 1, policy_version 163146 (0.0008) [2023-12-27 06:47:11,419][20381] Updated weights for policy 1, policy_version 163156 (0.0008) [2023-12-27 06:47:11,696][20347] Updated weights for policy 0, policy_version 163402 (0.0010) [2023-12-27 06:47:11,756][20347] Updated weights for policy 0, policy_version 163412 (0.0008) [2023-12-27 06:47:11,815][20347] Updated weights for policy 0, policy_version 163422 (0.0008) [2023-12-27 06:47:11,874][20347] Updated weights for policy 0, policy_version 163432 (0.0008) [2023-12-27 06:47:12,021][20381] Updated weights for policy 1, policy_version 163166 (0.0006) [2023-12-27 06:47:12,081][20381] Updated weights for policy 1, policy_version 163176 (0.0005) [2023-12-27 06:47:12,132][20381] Updated weights for policy 1, policy_version 163186 (0.0007) [2023-12-27 06:47:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 83640320. Throughput: 0: 10399.2, 1: 10358.0. Samples: 83629652. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:12,588][20347] Updated weights for policy 0, policy_version 163442 (0.0005) [2023-12-27 06:47:12,638][20347] Updated weights for policy 0, policy_version 163452 (0.0008) [2023-12-27 06:47:12,695][20347] Updated weights for policy 0, policy_version 163462 (0.0008) [2023-12-27 06:47:12,841][20381] Updated weights for policy 1, policy_version 163196 (0.0009) [2023-12-27 06:47:12,887][20381] Updated weights for policy 1, policy_version 163206 (0.0009) [2023-12-27 06:47:12,934][20381] Updated weights for policy 1, policy_version 163216 (0.0009) [2023-12-27 06:47:13,270][20347] Updated weights for policy 0, policy_version 163472 (0.0006) [2023-12-27 06:47:13,326][20347] Updated weights for policy 0, policy_version 163482 (0.0005) [2023-12-27 06:47:13,377][20347] Updated weights for policy 0, policy_version 163492 (0.0006) [2023-12-27 06:47:13,503][20381] Updated weights for policy 1, policy_version 163226 (0.0007) [2023-12-27 06:47:13,562][20381] Updated weights for policy 1, policy_version 163236 (0.0007) [2023-12-27 06:47:13,606][20381] Updated weights for policy 1, policy_version 163246 (0.0008) [2023-12-27 06:47:13,669][20381] Updated weights for policy 1, policy_version 163256 (0.0008) [2023-12-27 06:47:13,895][20347] Updated weights for policy 0, policy_version 163502 (0.0005) [2023-12-27 06:47:13,949][20347] Updated weights for policy 0, policy_version 163512 (0.0005) [2023-12-27 06:47:14,005][20347] Updated weights for policy 0, policy_version 163522 (0.0005) [2023-12-27 06:47:14,260][20381] Updated weights for policy 1, policy_version 163266 (0.0005) [2023-12-27 06:47:14,316][20381] Updated weights for policy 1, policy_version 163276 (0.0005) [2023-12-27 06:47:14,369][20381] Updated weights for policy 1, policy_version 163286 (0.0006) [2023-12-27 06:47:14,558][20347] Updated weights for policy 0, policy_version 163532 (0.0006) [2023-12-27 06:47:14,607][20347] Updated weights for policy 0, policy_version 163542 (0.0007) [2023-12-27 06:47:14,658][20347] Updated weights for policy 0, policy_version 163552 (0.0007) [2023-12-27 06:47:15,018][20381] Updated weights for policy 1, policy_version 163296 (0.0008) [2023-12-27 06:47:15,073][20381] Updated weights for policy 1, policy_version 163306 (0.0007) [2023-12-27 06:47:15,134][20381] Updated weights for policy 1, policy_version 163316 (0.0008) [2023-12-27 06:47:15,368][20347] Updated weights for policy 0, policy_version 163562 (0.0007) [2023-12-27 06:47:15,425][20347] Updated weights for policy 0, policy_version 163572 (0.0005) [2023-12-27 06:47:15,483][20347] Updated weights for policy 0, policy_version 163582 (0.0007) [2023-12-27 06:47:15,532][20347] Updated weights for policy 0, policy_version 163592 (0.0007) [2023-12-27 06:47:15,854][20381] Updated weights for policy 1, policy_version 163326 (0.0006) [2023-12-27 06:47:15,908][20381] Updated weights for policy 1, policy_version 163336 (0.0005) [2023-12-27 06:47:15,960][20381] Updated weights for policy 1, policy_version 163346 (0.0005) [2023-12-27 06:47:16,184][20347] Updated weights for policy 0, policy_version 163602 (0.0009) [2023-12-27 06:47:16,247][20347] Updated weights for policy 0, policy_version 163612 (0.0007) [2023-12-27 06:47:16,311][20347] Updated weights for policy 0, policy_version 163622 (0.0007) [2023-12-27 06:47:16,555][20381] Updated weights for policy 1, policy_version 163356 (0.0007) [2023-12-27 06:47:16,608][20381] Updated weights for policy 1, policy_version 163366 (0.0008) [2023-12-27 06:47:16,664][20381] Updated weights for policy 1, policy_version 163376 (0.0011) [2023-12-27 06:47:16,974][20347] Updated weights for policy 0, policy_version 163632 (0.0007) [2023-12-27 06:47:17,018][20347] Updated weights for policy 0, policy_version 163642 (0.0007) [2023-12-27 06:47:17,066][20347] Updated weights for policy 0, policy_version 163652 (0.0007) [2023-12-27 06:47:17,327][20381] Updated weights for policy 1, policy_version 163386 (0.0009) [2023-12-27 06:47:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 83746816. Throughput: 0: 10374.7, 1: 10407.6. Samples: 83758220. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:17,380][20381] Updated weights for policy 1, policy_version 163396 (0.0005) [2023-12-27 06:47:17,434][20381] Updated weights for policy 1, policy_version 163406 (0.0005) [2023-12-27 06:47:17,488][20381] Updated weights for policy 1, policy_version 163416 (0.0006) [2023-12-27 06:47:17,806][20347] Updated weights for policy 0, policy_version 163662 (0.0007) [2023-12-27 06:47:17,861][20347] Updated weights for policy 0, policy_version 163672 (0.0007) [2023-12-27 06:47:17,919][20347] Updated weights for policy 0, policy_version 163682 (0.0008) [2023-12-27 06:47:18,083][20381] Updated weights for policy 1, policy_version 163426 (0.0006) [2023-12-27 06:47:18,137][20381] Updated weights for policy 1, policy_version 163436 (0.0010) [2023-12-27 06:47:18,196][20381] Updated weights for policy 1, policy_version 163446 (0.0011) [2023-12-27 06:47:18,594][20347] Updated weights for policy 0, policy_version 163692 (0.0009) [2023-12-27 06:47:18,649][20347] Updated weights for policy 0, policy_version 163702 (0.0010) [2023-12-27 06:47:18,708][20347] Updated weights for policy 0, policy_version 163712 (0.0010) [2023-12-27 06:47:18,911][20381] Updated weights for policy 1, policy_version 163456 (0.0008) [2023-12-27 06:47:18,966][20381] Updated weights for policy 1, policy_version 163466 (0.0007) [2023-12-27 06:47:19,024][20381] Updated weights for policy 1, policy_version 163476 (0.0007) [2023-12-27 06:47:19,384][20347] Updated weights for policy 0, policy_version 163722 (0.0010) [2023-12-27 06:47:19,438][20347] Updated weights for policy 0, policy_version 163732 (0.0010) [2023-12-27 06:47:19,496][20347] Updated weights for policy 0, policy_version 163742 (0.0011) [2023-12-27 06:47:19,547][20347] Updated weights for policy 0, policy_version 163752 (0.0009) [2023-12-27 06:47:19,762][20381] Updated weights for policy 1, policy_version 163486 (0.0007) [2023-12-27 06:47:19,827][20381] Updated weights for policy 1, policy_version 163496 (0.0008) [2023-12-27 06:47:19,885][20381] Updated weights for policy 1, policy_version 163506 (0.0007) [2023-12-27 06:47:20,235][20347] Updated weights for policy 0, policy_version 163762 (0.0006) [2023-12-27 06:47:20,289][20347] Updated weights for policy 0, policy_version 163772 (0.0006) [2023-12-27 06:47:20,379][20347] Updated weights for policy 0, policy_version 163782 (0.0006) [2023-12-27 06:47:20,600][20381] Updated weights for policy 1, policy_version 163516 (0.0008) [2023-12-27 06:47:20,652][20381] Updated weights for policy 1, policy_version 163526 (0.0008) [2023-12-27 06:47:20,710][20381] Updated weights for policy 1, policy_version 163536 (0.0008) [2023-12-27 06:47:20,990][20347] Updated weights for policy 0, policy_version 163792 (0.0009) [2023-12-27 06:47:21,054][20347] Updated weights for policy 0, policy_version 163802 (0.0009) [2023-12-27 06:47:21,112][20347] Updated weights for policy 0, policy_version 163812 (0.0009) [2023-12-27 06:47:21,552][20381] Updated weights for policy 1, policy_version 163546 (0.0008) [2023-12-27 06:47:21,602][20381] Updated weights for policy 1, policy_version 163556 (0.0009) [2023-12-27 06:47:21,661][20381] Updated weights for policy 1, policy_version 163566 (0.0009) [2023-12-27 06:47:21,726][20381] Updated weights for policy 1, policy_version 163576 (0.0009) [2023-12-27 06:47:21,959][20347] Updated weights for policy 0, policy_version 163822 (0.0008) [2023-12-27 06:47:22,012][20347] Updated weights for policy 0, policy_version 163832 (0.0009) [2023-12-27 06:47:22,064][20347] Updated weights for policy 0, policy_version 163842 (0.0009) [2023-12-27 06:47:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 83845120. Throughput: 0: 10327.1, 1: 10440.5. Samples: 83819492. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000163576_41885696.pth... [2023-12-27 06:47:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000163848_41959424.pth... [2023-12-27 06:47:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000162360_41574400.pth [2023-12-27 06:47:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000162632_41648128.pth [2023-12-27 06:47:22,506][20381] Updated weights for policy 1, policy_version 163586 (0.0008) [2023-12-27 06:47:22,567][20381] Updated weights for policy 1, policy_version 163596 (0.0008) [2023-12-27 06:47:22,622][20381] Updated weights for policy 1, policy_version 163606 (0.0008) [2023-12-27 06:47:22,774][20347] Updated weights for policy 0, policy_version 163852 (0.0007) [2023-12-27 06:47:22,840][20347] Updated weights for policy 0, policy_version 163862 (0.0007) [2023-12-27 06:47:22,892][20347] Updated weights for policy 0, policy_version 163872 (0.0007) [2023-12-27 06:47:23,335][20381] Updated weights for policy 1, policy_version 163616 (0.0008) [2023-12-27 06:47:23,394][20381] Updated weights for policy 1, policy_version 163626 (0.0008) [2023-12-27 06:47:23,442][20381] Updated weights for policy 1, policy_version 163636 (0.0008) [2023-12-27 06:47:23,517][20347] Updated weights for policy 0, policy_version 163882 (0.0007) [2023-12-27 06:47:23,570][20347] Updated weights for policy 0, policy_version 163892 (0.0005) [2023-12-27 06:47:23,628][20347] Updated weights for policy 0, policy_version 163902 (0.0006) [2023-12-27 06:47:23,691][20347] Updated weights for policy 0, policy_version 163912 (0.0006) [2023-12-27 06:47:24,120][20381] Updated weights for policy 1, policy_version 163646 (0.0008) [2023-12-27 06:47:24,174][20381] Updated weights for policy 1, policy_version 163656 (0.0008) [2023-12-27 06:47:24,228][20381] Updated weights for policy 1, policy_version 163666 (0.0006) [2023-12-27 06:47:24,242][20347] Updated weights for policy 0, policy_version 163922 (0.0008) [2023-12-27 06:47:24,287][20347] Updated weights for policy 0, policy_version 163932 (0.0009) [2023-12-27 06:47:24,345][20347] Updated weights for policy 0, policy_version 163942 (0.0010) [2023-12-27 06:47:24,861][20381] Updated weights for policy 1, policy_version 163676 (0.0007) [2023-12-27 06:47:24,918][20381] Updated weights for policy 1, policy_version 163686 (0.0007) [2023-12-27 06:47:24,971][20381] Updated weights for policy 1, policy_version 163696 (0.0007) [2023-12-27 06:47:24,978][20347] Updated weights for policy 0, policy_version 163952 (0.0007) [2023-12-27 06:47:25,037][20347] Updated weights for policy 0, policy_version 163962 (0.0006) [2023-12-27 06:47:25,089][20347] Updated weights for policy 0, policy_version 163972 (0.0007) [2023-12-27 06:47:25,658][20381] Updated weights for policy 1, policy_version 163706 (0.0007) [2023-12-27 06:47:25,696][20347] Updated weights for policy 0, policy_version 163982 (0.0006) [2023-12-27 06:47:25,713][20381] Updated weights for policy 1, policy_version 163716 (0.0008) [2023-12-27 06:47:25,741][20347] Updated weights for policy 0, policy_version 163992 (0.0006) [2023-12-27 06:47:25,765][20381] Updated weights for policy 1, policy_version 163726 (0.0008) [2023-12-27 06:47:25,788][20347] Updated weights for policy 0, policy_version 164002 (0.0008) [2023-12-27 06:47:25,810][20381] Updated weights for policy 1, policy_version 163736 (0.0009) [2023-12-27 06:47:26,355][20347] Updated weights for policy 0, policy_version 164012 (0.0008) [2023-12-27 06:47:26,412][20347] Updated weights for policy 0, policy_version 164022 (0.0005) [2023-12-27 06:47:26,473][20347] Updated weights for policy 0, policy_version 164032 (0.0005) [2023-12-27 06:47:26,521][20381] Updated weights for policy 1, policy_version 163746 (0.0007) [2023-12-27 06:47:26,574][20381] Updated weights for policy 1, policy_version 163756 (0.0008) [2023-12-27 06:47:26,625][20381] Updated weights for policy 1, policy_version 163766 (0.0008) [2023-12-27 06:47:27,016][20347] Updated weights for policy 0, policy_version 164042 (0.0008) [2023-12-27 06:47:27,075][20347] Updated weights for policy 0, policy_version 164052 (0.0008) [2023-12-27 06:47:27,126][20347] Updated weights for policy 0, policy_version 164062 (0.0005) [2023-12-27 06:47:27,174][20347] Updated weights for policy 0, policy_version 164072 (0.0005) [2023-12-27 06:47:27,353][20381] Updated weights for policy 1, policy_version 163776 (0.0007) [2023-12-27 06:47:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 83951616. Throughput: 0: 10314.1, 1: 10415.0. Samples: 83941140. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:27,397][20381] Updated weights for policy 1, policy_version 163786 (0.0007) [2023-12-27 06:47:27,443][20381] Updated weights for policy 1, policy_version 163796 (0.0007) [2023-12-27 06:47:27,785][20347] Updated weights for policy 0, policy_version 164082 (0.0005) [2023-12-27 06:47:27,828][20347] Updated weights for policy 0, policy_version 164092 (0.0005) [2023-12-27 06:47:27,884][20347] Updated weights for policy 0, policy_version 164102 (0.0006) [2023-12-27 06:47:28,174][20381] Updated weights for policy 1, policy_version 163806 (0.0008) [2023-12-27 06:47:28,222][20381] Updated weights for policy 1, policy_version 163816 (0.0006) [2023-12-27 06:47:28,267][20381] Updated weights for policy 1, policy_version 163826 (0.0005) [2023-12-27 06:47:28,494][20347] Updated weights for policy 0, policy_version 164112 (0.0006) [2023-12-27 06:47:28,548][20347] Updated weights for policy 0, policy_version 164122 (0.0007) [2023-12-27 06:47:28,605][20347] Updated weights for policy 0, policy_version 164132 (0.0008) [2023-12-27 06:47:28,915][20381] Updated weights for policy 1, policy_version 163836 (0.0006) [2023-12-27 06:47:28,977][20381] Updated weights for policy 1, policy_version 163846 (0.0008) [2023-12-27 06:47:29,034][20381] Updated weights for policy 1, policy_version 163856 (0.0007) [2023-12-27 06:47:29,278][20347] Updated weights for policy 0, policy_version 164142 (0.0007) [2023-12-27 06:47:29,336][20347] Updated weights for policy 0, policy_version 164152 (0.0009) [2023-12-27 06:47:29,389][20347] Updated weights for policy 0, policy_version 164162 (0.0008) [2023-12-27 06:47:29,651][20381] Updated weights for policy 1, policy_version 163866 (0.0007) [2023-12-27 06:47:29,703][20381] Updated weights for policy 1, policy_version 163876 (0.0006) [2023-12-27 06:47:29,752][20381] Updated weights for policy 1, policy_version 163886 (0.0008) [2023-12-27 06:47:29,808][20381] Updated weights for policy 1, policy_version 163896 (0.0007) [2023-12-27 06:47:30,090][20347] Updated weights for policy 0, policy_version 164172 (0.0007) [2023-12-27 06:47:30,146][20347] Updated weights for policy 0, policy_version 164182 (0.0008) [2023-12-27 06:47:30,204][20347] Updated weights for policy 0, policy_version 164192 (0.0007) [2023-12-27 06:47:30,495][20381] Updated weights for policy 1, policy_version 163906 (0.0006) [2023-12-27 06:47:30,549][20381] Updated weights for policy 1, policy_version 163916 (0.0008) [2023-12-27 06:47:30,606][20381] Updated weights for policy 1, policy_version 163926 (0.0006) [2023-12-27 06:47:30,856][20347] Updated weights for policy 0, policy_version 164202 (0.0007) [2023-12-27 06:47:30,917][20347] Updated weights for policy 0, policy_version 164212 (0.0007) [2023-12-27 06:47:30,968][20347] Updated weights for policy 0, policy_version 164222 (0.0007) [2023-12-27 06:47:31,016][20347] Updated weights for policy 0, policy_version 164232 (0.0006) [2023-12-27 06:47:31,287][20381] Updated weights for policy 1, policy_version 163936 (0.0008) [2023-12-27 06:47:31,343][20381] Updated weights for policy 1, policy_version 163946 (0.0008) [2023-12-27 06:47:31,408][20381] Updated weights for policy 1, policy_version 163956 (0.0007) [2023-12-27 06:47:31,693][20347] Updated weights for policy 0, policy_version 164242 (0.0008) [2023-12-27 06:47:31,752][20347] Updated weights for policy 0, policy_version 164252 (0.0006) [2023-12-27 06:47:31,810][20347] Updated weights for policy 0, policy_version 164262 (0.0005) [2023-12-27 06:47:32,097][20381] Updated weights for policy 1, policy_version 163966 (0.0009) [2023-12-27 06:47:32,155][20381] Updated weights for policy 1, policy_version 163976 (0.0008) [2023-12-27 06:47:32,217][20381] Updated weights for policy 1, policy_version 163986 (0.0009) [2023-12-27 06:47:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 84058112. Throughput: 0: 10399.1, 1: 10348.9. Samples: 84067656. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:32,443][20347] Updated weights for policy 0, policy_version 164272 (0.0007) [2023-12-27 06:47:32,498][20347] Updated weights for policy 0, policy_version 164282 (0.0007) [2023-12-27 06:47:32,551][20347] Updated weights for policy 0, policy_version 164292 (0.0007) [2023-12-27 06:47:32,889][20381] Updated weights for policy 1, policy_version 163996 (0.0010) [2023-12-27 06:47:32,937][20381] Updated weights for policy 1, policy_version 164006 (0.0008) [2023-12-27 06:47:32,985][20381] Updated weights for policy 1, policy_version 164016 (0.0007) [2023-12-27 06:47:33,263][20347] Updated weights for policy 0, policy_version 164302 (0.0008) [2023-12-27 06:47:33,317][20347] Updated weights for policy 0, policy_version 164312 (0.0007) [2023-12-27 06:47:33,368][20347] Updated weights for policy 0, policy_version 164322 (0.0008) [2023-12-27 06:47:33,624][20381] Updated weights for policy 1, policy_version 164026 (0.0007) [2023-12-27 06:47:33,674][20381] Updated weights for policy 1, policy_version 164036 (0.0007) [2023-12-27 06:47:33,724][20381] Updated weights for policy 1, policy_version 164046 (0.0005) [2023-12-27 06:47:33,772][20381] Updated weights for policy 1, policy_version 164056 (0.0005) [2023-12-27 06:47:34,080][20347] Updated weights for policy 0, policy_version 164332 (0.0008) [2023-12-27 06:47:34,138][20347] Updated weights for policy 0, policy_version 164342 (0.0008) [2023-12-27 06:47:34,198][20347] Updated weights for policy 0, policy_version 164352 (0.0009) [2023-12-27 06:47:34,306][20381] Updated weights for policy 1, policy_version 164066 (0.0008) [2023-12-27 06:47:34,362][20381] Updated weights for policy 1, policy_version 164076 (0.0008) [2023-12-27 06:47:34,412][20381] Updated weights for policy 1, policy_version 164086 (0.0007) [2023-12-27 06:47:34,868][20347] Updated weights for policy 0, policy_version 164362 (0.0007) [2023-12-27 06:47:34,913][20347] Updated weights for policy 0, policy_version 164372 (0.0007) [2023-12-27 06:47:34,961][20347] Updated weights for policy 0, policy_version 164382 (0.0008) [2023-12-27 06:47:35,022][20347] Updated weights for policy 0, policy_version 164392 (0.0007) [2023-12-27 06:47:35,047][20381] Updated weights for policy 1, policy_version 164096 (0.0007) [2023-12-27 06:47:35,102][20381] Updated weights for policy 1, policy_version 164106 (0.0008) [2023-12-27 06:47:35,158][20381] Updated weights for policy 1, policy_version 164116 (0.0008) [2023-12-27 06:47:35,728][20347] Updated weights for policy 0, policy_version 164402 (0.0007) [2023-12-27 06:47:35,780][20347] Updated weights for policy 0, policy_version 164412 (0.0007) [2023-12-27 06:47:35,832][20347] Updated weights for policy 0, policy_version 164422 (0.0005) [2023-12-27 06:47:35,858][20381] Updated weights for policy 1, policy_version 164126 (0.0008) [2023-12-27 06:47:35,915][20381] Updated weights for policy 1, policy_version 164136 (0.0009) [2023-12-27 06:47:35,962][20381] Updated weights for policy 1, policy_version 164146 (0.0007) [2023-12-27 06:47:36,470][20347] Updated weights for policy 0, policy_version 164432 (0.0008) [2023-12-27 06:47:36,527][20347] Updated weights for policy 0, policy_version 164442 (0.0010) [2023-12-27 06:47:36,583][20347] Updated weights for policy 0, policy_version 164452 (0.0011) [2023-12-27 06:47:36,734][20381] Updated weights for policy 1, policy_version 164156 (0.0007) [2023-12-27 06:47:36,792][20381] Updated weights for policy 1, policy_version 164166 (0.0008) [2023-12-27 06:47:36,849][20381] Updated weights for policy 1, policy_version 164176 (0.0008) [2023-12-27 06:47:37,231][20347] Updated weights for policy 0, policy_version 164462 (0.0007) [2023-12-27 06:47:37,292][20347] Updated weights for policy 0, policy_version 164472 (0.0007) [2023-12-27 06:47:37,352][20347] Updated weights for policy 0, policy_version 164482 (0.0011) [2023-12-27 06:47:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 84156416. Throughput: 0: 10390.9, 1: 10356.9. Samples: 84130760. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000164184_42041344.pth... [2023-12-27 06:47:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000163000_41738240.pth [2023-12-27 06:47:37,387][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000164488_42123264.pth... [2023-12-27 06:47:37,391][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000163240_41803776.pth [2023-12-27 06:47:37,595][20381] Updated weights for policy 1, policy_version 164186 (0.0008) [2023-12-27 06:47:37,651][20381] Updated weights for policy 1, policy_version 164196 (0.0005) [2023-12-27 06:47:37,704][20381] Updated weights for policy 1, policy_version 164206 (0.0005) [2023-12-27 06:47:37,754][20381] Updated weights for policy 1, policy_version 164216 (0.0008) [2023-12-27 06:47:37,963][20347] Updated weights for policy 0, policy_version 164492 (0.0010) [2023-12-27 06:47:38,018][20347] Updated weights for policy 0, policy_version 164502 (0.0010) [2023-12-27 06:47:38,070][20347] Updated weights for policy 0, policy_version 164512 (0.0010) [2023-12-27 06:47:38,465][20381] Updated weights for policy 1, policy_version 164226 (0.0007) [2023-12-27 06:47:38,519][20381] Updated weights for policy 1, policy_version 164236 (0.0008) [2023-12-27 06:47:38,577][20381] Updated weights for policy 1, policy_version 164246 (0.0007) [2023-12-27 06:47:38,754][20347] Updated weights for policy 0, policy_version 164522 (0.0010) [2023-12-27 06:47:38,806][20347] Updated weights for policy 0, policy_version 164532 (0.0010) [2023-12-27 06:47:38,865][20347] Updated weights for policy 0, policy_version 164542 (0.0010) [2023-12-27 06:47:38,929][20347] Updated weights for policy 0, policy_version 164552 (0.0007) [2023-12-27 06:47:39,277][20381] Updated weights for policy 1, policy_version 164256 (0.0010) [2023-12-27 06:47:39,331][20381] Updated weights for policy 1, policy_version 164266 (0.0011) [2023-12-27 06:47:39,391][20381] Updated weights for policy 1, policy_version 164276 (0.0008) [2023-12-27 06:47:39,651][20347] Updated weights for policy 0, policy_version 164562 (0.0010) [2023-12-27 06:47:39,711][20347] Updated weights for policy 0, policy_version 164572 (0.0009) [2023-12-27 06:47:39,768][20347] Updated weights for policy 0, policy_version 164582 (0.0009) [2023-12-27 06:47:40,154][20381] Updated weights for policy 1, policy_version 164286 (0.0007) [2023-12-27 06:47:40,214][20381] Updated weights for policy 1, policy_version 164296 (0.0007) [2023-12-27 06:47:40,268][20381] Updated weights for policy 1, policy_version 164306 (0.0008) [2023-12-27 06:47:40,435][20347] Updated weights for policy 0, policy_version 164592 (0.0007) [2023-12-27 06:47:40,488][20347] Updated weights for policy 0, policy_version 164602 (0.0007) [2023-12-27 06:47:40,544][20347] Updated weights for policy 0, policy_version 164612 (0.0009) [2023-12-27 06:47:40,947][20381] Updated weights for policy 1, policy_version 164316 (0.0008) [2023-12-27 06:47:40,999][20381] Updated weights for policy 1, policy_version 164326 (0.0006) [2023-12-27 06:47:41,062][20381] Updated weights for policy 1, policy_version 164336 (0.0008) [2023-12-27 06:47:41,242][20347] Updated weights for policy 0, policy_version 164622 (0.0007) [2023-12-27 06:47:41,305][20347] Updated weights for policy 0, policy_version 164632 (0.0007) [2023-12-27 06:47:41,366][20347] Updated weights for policy 0, policy_version 164642 (0.0006) [2023-12-27 06:47:41,764][20381] Updated weights for policy 1, policy_version 164346 (0.0007) [2023-12-27 06:47:41,819][20381] Updated weights for policy 1, policy_version 164356 (0.0007) [2023-12-27 06:47:41,882][20381] Updated weights for policy 1, policy_version 164366 (0.0007) [2023-12-27 06:47:41,944][20381] Updated weights for policy 1, policy_version 164376 (0.0008) [2023-12-27 06:47:42,064][20347] Updated weights for policy 0, policy_version 164652 (0.0008) [2023-12-27 06:47:42,123][20347] Updated weights for policy 0, policy_version 164662 (0.0006) [2023-12-27 06:47:42,177][20347] Updated weights for policy 0, policy_version 164672 (0.0005) [2023-12-27 06:47:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 84262912. Throughput: 0: 10408.1, 1: 10278.2. Samples: 84251864. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:42,632][20381] Updated weights for policy 1, policy_version 164386 (0.0005) [2023-12-27 06:47:42,690][20381] Updated weights for policy 1, policy_version 164396 (0.0005) [2023-12-27 06:47:42,742][20381] Updated weights for policy 1, policy_version 164406 (0.0005) [2023-12-27 06:47:42,826][20347] Updated weights for policy 0, policy_version 164682 (0.0007) [2023-12-27 06:47:42,887][20347] Updated weights for policy 0, policy_version 164692 (0.0006) [2023-12-27 06:47:42,949][20347] Updated weights for policy 0, policy_version 164702 (0.0010) [2023-12-27 06:47:42,997][20347] Updated weights for policy 0, policy_version 164712 (0.0010) [2023-12-27 06:47:43,301][20381] Updated weights for policy 1, policy_version 164416 (0.0009) [2023-12-27 06:47:43,349][20381] Updated weights for policy 1, policy_version 164426 (0.0010) [2023-12-27 06:47:43,397][20381] Updated weights for policy 1, policy_version 164436 (0.0010) [2023-12-27 06:47:43,609][20347] Updated weights for policy 0, policy_version 164722 (0.0007) [2023-12-27 06:47:43,662][20347] Updated weights for policy 0, policy_version 164732 (0.0007) [2023-12-27 06:47:43,719][20347] Updated weights for policy 0, policy_version 164742 (0.0007) [2023-12-27 06:47:44,035][20381] Updated weights for policy 1, policy_version 164446 (0.0009) [2023-12-27 06:47:44,093][20381] Updated weights for policy 1, policy_version 164456 (0.0007) [2023-12-27 06:47:44,153][20381] Updated weights for policy 1, policy_version 164466 (0.0005) [2023-12-27 06:47:44,402][20347] Updated weights for policy 0, policy_version 164752 (0.0007) [2023-12-27 06:47:44,453][20347] Updated weights for policy 0, policy_version 164762 (0.0007) [2023-12-27 06:47:44,510][20347] Updated weights for policy 0, policy_version 164772 (0.0005) [2023-12-27 06:47:44,797][20381] Updated weights for policy 1, policy_version 164476 (0.0007) [2023-12-27 06:47:44,852][20381] Updated weights for policy 1, policy_version 164486 (0.0006) [2023-12-27 06:47:44,917][20381] Updated weights for policy 1, policy_version 164496 (0.0009) [2023-12-27 06:47:45,201][20347] Updated weights for policy 0, policy_version 164782 (0.0006) [2023-12-27 06:47:45,249][20347] Updated weights for policy 0, policy_version 164792 (0.0008) [2023-12-27 06:47:45,299][20347] Updated weights for policy 0, policy_version 164802 (0.0009) [2023-12-27 06:47:45,474][20381] Updated weights for policy 1, policy_version 164506 (0.0010) [2023-12-27 06:47:45,526][20381] Updated weights for policy 1, policy_version 164516 (0.0005) [2023-12-27 06:47:45,570][20381] Updated weights for policy 1, policy_version 164526 (0.0005) [2023-12-27 06:47:45,623][20381] Updated weights for policy 1, policy_version 164536 (0.0008) [2023-12-27 06:47:46,044][20347] Updated weights for policy 0, policy_version 164812 (0.0008) [2023-12-27 06:47:46,094][20347] Updated weights for policy 0, policy_version 164822 (0.0007) [2023-12-27 06:47:46,142][20347] Updated weights for policy 0, policy_version 164832 (0.0007) [2023-12-27 06:47:46,245][20381] Updated weights for policy 1, policy_version 164546 (0.0011) [2023-12-27 06:47:46,309][20381] Updated weights for policy 1, policy_version 164556 (0.0011) [2023-12-27 06:47:46,372][20381] Updated weights for policy 1, policy_version 164566 (0.0010) [2023-12-27 06:47:46,804][20347] Updated weights for policy 0, policy_version 164842 (0.0007) [2023-12-27 06:47:46,853][20347] Updated weights for policy 0, policy_version 164852 (0.0007) [2023-12-27 06:47:46,910][20347] Updated weights for policy 0, policy_version 164862 (0.0007) [2023-12-27 06:47:46,960][20347] Updated weights for policy 0, policy_version 164872 (0.0007) [2023-12-27 06:47:47,036][20381] Updated weights for policy 1, policy_version 164576 (0.0010) [2023-12-27 06:47:47,084][20381] Updated weights for policy 1, policy_version 164586 (0.0010) [2023-12-27 06:47:47,142][20381] Updated weights for policy 1, policy_version 164596 (0.0011) [2023-12-27 06:47:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 84369408. Throughput: 0: 10420.3, 1: 10342.2. Samples: 84377800. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:47,606][20347] Updated weights for policy 0, policy_version 164882 (0.0006) [2023-12-27 06:47:47,659][20347] Updated weights for policy 0, policy_version 164892 (0.0008) [2023-12-27 06:47:47,717][20347] Updated weights for policy 0, policy_version 164902 (0.0008) [2023-12-27 06:47:47,742][20381] Updated weights for policy 1, policy_version 164606 (0.0010) [2023-12-27 06:47:47,798][20381] Updated weights for policy 1, policy_version 164616 (0.0011) [2023-12-27 06:47:47,854][20381] Updated weights for policy 1, policy_version 164626 (0.0011) [2023-12-27 06:47:48,354][20347] Updated weights for policy 0, policy_version 164912 (0.0008) [2023-12-27 06:47:48,412][20347] Updated weights for policy 0, policy_version 164922 (0.0008) [2023-12-27 06:47:48,466][20347] Updated weights for policy 0, policy_version 164932 (0.0007) [2023-12-27 06:47:48,500][20381] Updated weights for policy 1, policy_version 164636 (0.0011) [2023-12-27 06:47:48,559][20381] Updated weights for policy 1, policy_version 164646 (0.0011) [2023-12-27 06:47:48,617][20381] Updated weights for policy 1, policy_version 164656 (0.0011) [2023-12-27 06:47:49,089][20347] Updated weights for policy 0, policy_version 164942 (0.0006) [2023-12-27 06:47:49,151][20347] Updated weights for policy 0, policy_version 164952 (0.0007) [2023-12-27 06:47:49,210][20347] Updated weights for policy 0, policy_version 164962 (0.0008) [2023-12-27 06:47:49,225][20381] Updated weights for policy 1, policy_version 164666 (0.0010) [2023-12-27 06:47:49,283][20381] Updated weights for policy 1, policy_version 164676 (0.0009) [2023-12-27 06:47:49,344][20381] Updated weights for policy 1, policy_version 164686 (0.0011) [2023-12-27 06:47:49,404][20381] Updated weights for policy 1, policy_version 164696 (0.0010) [2023-12-27 06:47:49,943][20347] Updated weights for policy 0, policy_version 164972 (0.0009) [2023-12-27 06:47:50,000][20347] Updated weights for policy 0, policy_version 164982 (0.0007) [2023-12-27 06:47:50,011][20381] Updated weights for policy 1, policy_version 164706 (0.0010) [2023-12-27 06:47:50,059][20347] Updated weights for policy 0, policy_version 164992 (0.0007) [2023-12-27 06:47:50,067][20381] Updated weights for policy 1, policy_version 164716 (0.0010) [2023-12-27 06:47:50,123][20381] Updated weights for policy 1, policy_version 164726 (0.0010) [2023-12-27 06:47:50,728][20347] Updated weights for policy 0, policy_version 165002 (0.0006) [2023-12-27 06:47:50,789][20347] Updated weights for policy 0, policy_version 165012 (0.0009) [2023-12-27 06:47:50,795][20381] Updated weights for policy 1, policy_version 164736 (0.0011) [2023-12-27 06:47:50,845][20347] Updated weights for policy 0, policy_version 165022 (0.0010) [2023-12-27 06:47:50,854][20381] Updated weights for policy 1, policy_version 164746 (0.0010) [2023-12-27 06:47:50,909][20347] Updated weights for policy 0, policy_version 165032 (0.0006) [2023-12-27 06:47:50,913][20381] Updated weights for policy 1, policy_version 164756 (0.0011) [2023-12-27 06:47:51,621][20347] Updated weights for policy 0, policy_version 165042 (0.0009) [2023-12-27 06:47:51,656][20381] Updated weights for policy 1, policy_version 164766 (0.0009) [2023-12-27 06:47:51,679][20347] Updated weights for policy 0, policy_version 165052 (0.0009) [2023-12-27 06:47:51,717][20381] Updated weights for policy 1, policy_version 164776 (0.0009) [2023-12-27 06:47:51,745][20347] Updated weights for policy 0, policy_version 165062 (0.0008) [2023-12-27 06:47:51,777][20381] Updated weights for policy 1, policy_version 164786 (0.0007) [2023-12-27 06:47:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 84467712. Throughput: 0: 10440.8, 1: 10351.3. Samples: 84442256. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000164792_42196992.pth... [2023-12-27 06:47:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000165064_42270720.pth... [2023-12-27 06:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000163576_41885696.pth [2023-12-27 06:47:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000163848_41959424.pth [2023-12-27 06:47:52,485][20347] Updated weights for policy 0, policy_version 165072 (0.0008) [2023-12-27 06:47:52,493][20381] Updated weights for policy 1, policy_version 164796 (0.0008) [2023-12-27 06:47:52,549][20347] Updated weights for policy 0, policy_version 165082 (0.0006) [2023-12-27 06:47:52,554][20381] Updated weights for policy 1, policy_version 164806 (0.0011) [2023-12-27 06:47:52,602][20347] Updated weights for policy 0, policy_version 165092 (0.0008) [2023-12-27 06:47:52,621][20381] Updated weights for policy 1, policy_version 164816 (0.0011) [2023-12-27 06:47:53,237][20347] Updated weights for policy 0, policy_version 165102 (0.0006) [2023-12-27 06:47:53,293][20347] Updated weights for policy 0, policy_version 165112 (0.0005) [2023-12-27 06:47:53,302][20381] Updated weights for policy 1, policy_version 164826 (0.0011) [2023-12-27 06:47:53,338][20347] Updated weights for policy 0, policy_version 165122 (0.0005) [2023-12-27 06:47:53,359][20381] Updated weights for policy 1, policy_version 164836 (0.0010) [2023-12-27 06:47:53,407][20381] Updated weights for policy 1, policy_version 164846 (0.0010) [2023-12-27 06:47:53,464][20381] Updated weights for policy 1, policy_version 164856 (0.0010) [2023-12-27 06:47:53,952][20347] Updated weights for policy 0, policy_version 165132 (0.0006) [2023-12-27 06:47:54,010][20347] Updated weights for policy 0, policy_version 165142 (0.0006) [2023-12-27 06:47:54,064][20347] Updated weights for policy 0, policy_version 165152 (0.0007) [2023-12-27 06:47:54,134][20381] Updated weights for policy 1, policy_version 164866 (0.0011) [2023-12-27 06:47:54,186][20381] Updated weights for policy 1, policy_version 164876 (0.0011) [2023-12-27 06:47:54,250][20381] Updated weights for policy 1, policy_version 164886 (0.0011) [2023-12-27 06:47:54,789][20347] Updated weights for policy 0, policy_version 165162 (0.0006) [2023-12-27 06:47:54,838][20381] Updated weights for policy 1, policy_version 164896 (0.0006) [2023-12-27 06:47:54,842][20347] Updated weights for policy 0, policy_version 165172 (0.0009) [2023-12-27 06:47:54,899][20347] Updated weights for policy 0, policy_version 165182 (0.0008) [2023-12-27 06:47:54,902][20381] Updated weights for policy 1, policy_version 164906 (0.0005) [2023-12-27 06:47:54,956][20347] Updated weights for policy 0, policy_version 165192 (0.0008) [2023-12-27 06:47:54,965][20381] Updated weights for policy 1, policy_version 164916 (0.0007) [2023-12-27 06:47:55,524][20381] Updated weights for policy 1, policy_version 164926 (0.0009) [2023-12-27 06:47:55,575][20381] Updated weights for policy 1, policy_version 164936 (0.0007) [2023-12-27 06:47:55,617][20347] Updated weights for policy 0, policy_version 165202 (0.0008) [2023-12-27 06:47:55,635][20381] Updated weights for policy 1, policy_version 164946 (0.0006) [2023-12-27 06:47:55,667][20347] Updated weights for policy 0, policy_version 165212 (0.0008) [2023-12-27 06:47:55,715][20347] Updated weights for policy 0, policy_version 165222 (0.0007) [2023-12-27 06:47:56,240][20381] Updated weights for policy 1, policy_version 164956 (0.0007) [2023-12-27 06:47:56,285][20381] Updated weights for policy 1, policy_version 164966 (0.0005) [2023-12-27 06:47:56,344][20381] Updated weights for policy 1, policy_version 164976 (0.0005) [2023-12-27 06:47:56,463][20347] Updated weights for policy 0, policy_version 165232 (0.0007) [2023-12-27 06:47:56,509][20347] Updated weights for policy 0, policy_version 165242 (0.0007) [2023-12-27 06:47:56,553][20347] Updated weights for policy 0, policy_version 165252 (0.0007) [2023-12-27 06:47:56,881][20381] Updated weights for policy 1, policy_version 164986 (0.0006) [2023-12-27 06:47:56,928][20381] Updated weights for policy 1, policy_version 164996 (0.0010) [2023-12-27 06:47:56,973][20381] Updated weights for policy 1, policy_version 165006 (0.0010) [2023-12-27 06:47:57,027][20381] Updated weights for policy 1, policy_version 165016 (0.0010) [2023-12-27 06:47:57,246][20347] Updated weights for policy 0, policy_version 165262 (0.0006) [2023-12-27 06:47:57,304][20347] Updated weights for policy 0, policy_version 165272 (0.0005) [2023-12-27 06:47:57,359][20347] Updated weights for policy 0, policy_version 165282 (0.0009) [2023-12-27 06:47:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 84574208. Throughput: 0: 10412.1, 1: 10395.4. Samples: 84565988. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:47:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:47:57,691][20381] Updated weights for policy 1, policy_version 165026 (0.0006) [2023-12-27 06:47:57,743][20381] Updated weights for policy 1, policy_version 165036 (0.0008) [2023-12-27 06:47:57,791][20381] Updated weights for policy 1, policy_version 165046 (0.0009) [2023-12-27 06:47:57,883][20347] Updated weights for policy 0, policy_version 165292 (0.0008) [2023-12-27 06:47:57,948][20347] Updated weights for policy 0, policy_version 165302 (0.0005) [2023-12-27 06:47:58,004][20347] Updated weights for policy 0, policy_version 165312 (0.0005) [2023-12-27 06:47:58,447][20381] Updated weights for policy 1, policy_version 165056 (0.0007) [2023-12-27 06:47:58,512][20381] Updated weights for policy 1, policy_version 165066 (0.0007) [2023-12-27 06:47:58,576][20381] Updated weights for policy 1, policy_version 165076 (0.0007) [2023-12-27 06:47:58,639][20347] Updated weights for policy 0, policy_version 165322 (0.0006) [2023-12-27 06:47:58,698][20347] Updated weights for policy 0, policy_version 165332 (0.0010) [2023-12-27 06:47:58,767][20347] Updated weights for policy 0, policy_version 165342 (0.0007) [2023-12-27 06:47:58,828][20347] Updated weights for policy 0, policy_version 165352 (0.0008) [2023-12-27 06:47:59,288][20381] Updated weights for policy 1, policy_version 165086 (0.0008) [2023-12-27 06:47:59,340][20381] Updated weights for policy 1, policy_version 165096 (0.0007) [2023-12-27 06:47:59,402][20381] Updated weights for policy 1, policy_version 165106 (0.0008) [2023-12-27 06:47:59,575][20347] Updated weights for policy 0, policy_version 165362 (0.0007) [2023-12-27 06:47:59,633][20347] Updated weights for policy 0, policy_version 165372 (0.0005) [2023-12-27 06:47:59,690][20347] Updated weights for policy 0, policy_version 165382 (0.0005) [2023-12-27 06:48:00,159][20381] Updated weights for policy 1, policy_version 165116 (0.0007) [2023-12-27 06:48:00,211][20381] Updated weights for policy 1, policy_version 165126 (0.0008) [2023-12-27 06:48:00,257][20381] Updated weights for policy 1, policy_version 165136 (0.0007) [2023-12-27 06:48:00,331][20347] Updated weights for policy 0, policy_version 165392 (0.0006) [2023-12-27 06:48:00,388][20347] Updated weights for policy 0, policy_version 165402 (0.0007) [2023-12-27 06:48:00,436][20347] Updated weights for policy 0, policy_version 165412 (0.0007) [2023-12-27 06:48:00,996][20381] Updated weights for policy 1, policy_version 165146 (0.0008) [2023-12-27 06:48:01,036][20347] Updated weights for policy 0, policy_version 165422 (0.0008) [2023-12-27 06:48:01,063][20381] Updated weights for policy 1, policy_version 165156 (0.0009) [2023-12-27 06:48:01,093][20347] Updated weights for policy 0, policy_version 165432 (0.0007) [2023-12-27 06:48:01,124][20381] Updated weights for policy 1, policy_version 165166 (0.0006) [2023-12-27 06:48:01,148][20347] Updated weights for policy 0, policy_version 165442 (0.0008) [2023-12-27 06:48:01,192][20381] Updated weights for policy 1, policy_version 165176 (0.0008) [2023-12-27 06:48:01,848][20347] Updated weights for policy 0, policy_version 165452 (0.0007) [2023-12-27 06:48:01,869][20381] Updated weights for policy 1, policy_version 165186 (0.0010) [2023-12-27 06:48:01,902][20347] Updated weights for policy 0, policy_version 165462 (0.0005) [2023-12-27 06:48:01,923][20381] Updated weights for policy 1, policy_version 165196 (0.0010) [2023-12-27 06:48:01,958][20347] Updated weights for policy 0, policy_version 165472 (0.0005) [2023-12-27 06:48:01,978][20381] Updated weights for policy 1, policy_version 165206 (0.0011) [2023-12-27 06:48:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 84680704. Throughput: 0: 10384.9, 1: 10331.9. Samples: 84690476. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:48:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:02,610][20381] Updated weights for policy 1, policy_version 165216 (0.0010) [2023-12-27 06:48:02,659][20381] Updated weights for policy 1, policy_version 165226 (0.0010) [2023-12-27 06:48:02,661][20347] Updated weights for policy 0, policy_version 165482 (0.0008) [2023-12-27 06:48:02,716][20381] Updated weights for policy 1, policy_version 165236 (0.0011) [2023-12-27 06:48:02,718][20347] Updated weights for policy 0, policy_version 165492 (0.0007) [2023-12-27 06:48:02,766][20347] Updated weights for policy 0, policy_version 165502 (0.0006) [2023-12-27 06:48:02,817][20347] Updated weights for policy 0, policy_version 165512 (0.0007) [2023-12-27 06:48:03,357][20381] Updated weights for policy 1, policy_version 165246 (0.0007) [2023-12-27 06:48:03,402][20381] Updated weights for policy 1, policy_version 165256 (0.0005) [2023-12-27 06:48:03,447][20381] Updated weights for policy 1, policy_version 165266 (0.0005) [2023-12-27 06:48:03,513][20347] Updated weights for policy 0, policy_version 165522 (0.0006) [2023-12-27 06:48:03,567][20347] Updated weights for policy 0, policy_version 165532 (0.0008) [2023-12-27 06:48:03,617][20347] Updated weights for policy 0, policy_version 165542 (0.0010) [2023-12-27 06:48:04,050][20381] Updated weights for policy 1, policy_version 165276 (0.0008) [2023-12-27 06:48:04,107][20381] Updated weights for policy 1, policy_version 165286 (0.0011) [2023-12-27 06:48:04,163][20381] Updated weights for policy 1, policy_version 165296 (0.0011) [2023-12-27 06:48:04,329][20347] Updated weights for policy 0, policy_version 165552 (0.0005) [2023-12-27 06:48:04,386][20347] Updated weights for policy 0, policy_version 165562 (0.0006) [2023-12-27 06:48:04,445][20347] Updated weights for policy 0, policy_version 165572 (0.0008) [2023-12-27 06:48:04,851][20381] Updated weights for policy 1, policy_version 165306 (0.0011) [2023-12-27 06:48:04,908][20381] Updated weights for policy 1, policy_version 165316 (0.0010) [2023-12-27 06:48:04,960][20381] Updated weights for policy 1, policy_version 165326 (0.0010) [2023-12-27 06:48:05,009][20381] Updated weights for policy 1, policy_version 165336 (0.0010) [2023-12-27 06:48:05,032][20347] Updated weights for policy 0, policy_version 165582 (0.0006) [2023-12-27 06:48:05,090][20347] Updated weights for policy 0, policy_version 165592 (0.0005) [2023-12-27 06:48:05,152][20347] Updated weights for policy 0, policy_version 165602 (0.0006) [2023-12-27 06:48:05,664][20347] Updated weights for policy 0, policy_version 165612 (0.0005) [2023-12-27 06:48:05,708][20381] Updated weights for policy 1, policy_version 165346 (0.0010) [2023-12-27 06:48:05,713][20347] Updated weights for policy 0, policy_version 165622 (0.0005) [2023-12-27 06:48:05,768][20381] Updated weights for policy 1, policy_version 165356 (0.0005) [2023-12-27 06:48:05,769][20347] Updated weights for policy 0, policy_version 165632 (0.0006) [2023-12-27 06:48:05,826][20381] Updated weights for policy 1, policy_version 165366 (0.0008) [2023-12-27 06:48:06,333][20347] Updated weights for policy 0, policy_version 165642 (0.0007) [2023-12-27 06:48:06,403][20347] Updated weights for policy 0, policy_version 165652 (0.0007) [2023-12-27 06:48:06,458][20381] Updated weights for policy 1, policy_version 165376 (0.0011) [2023-12-27 06:48:06,460][20347] Updated weights for policy 0, policy_version 165662 (0.0006) [2023-12-27 06:48:06,513][20381] Updated weights for policy 1, policy_version 165386 (0.0009) [2023-12-27 06:48:06,517][20347] Updated weights for policy 0, policy_version 165672 (0.0006) [2023-12-27 06:48:06,569][20381] Updated weights for policy 1, policy_version 165396 (0.0011) [2023-12-27 06:48:07,169][20347] Updated weights for policy 0, policy_version 165682 (0.0007) [2023-12-27 06:48:07,231][20347] Updated weights for policy 0, policy_version 165692 (0.0007) [2023-12-27 06:48:07,281][20381] Updated weights for policy 1, policy_version 165406 (0.0008) [2023-12-27 06:48:07,291][20347] Updated weights for policy 0, policy_version 165702 (0.0007) [2023-12-27 06:48:07,337][20381] Updated weights for policy 1, policy_version 165416 (0.0007) [2023-12-27 06:48:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 84787200. Throughput: 0: 10415.6, 1: 10351.1. Samples: 84753992. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 06:48:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000165704_42434560.pth... [2023-12-27 06:48:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000164488_42123264.pth [2023-12-27 06:48:07,396][20381] Updated weights for policy 1, policy_version 165426 (0.0008) [2023-12-27 06:48:07,431][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000165432_42360832.pth... [2023-12-27 06:48:07,435][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000164184_42041344.pth [2023-12-27 06:48:07,983][20347] Updated weights for policy 0, policy_version 165712 (0.0008) [2023-12-27 06:48:08,041][20347] Updated weights for policy 0, policy_version 165722 (0.0007) [2023-12-27 06:48:08,047][20381] Updated weights for policy 1, policy_version 165436 (0.0006) [2023-12-27 06:48:08,097][20347] Updated weights for policy 0, policy_version 165732 (0.0006) [2023-12-27 06:48:08,099][20381] Updated weights for policy 1, policy_version 165446 (0.0005) [2023-12-27 06:48:08,158][20381] Updated weights for policy 1, policy_version 165456 (0.0007) [2023-12-27 06:48:08,765][20347] Updated weights for policy 0, policy_version 165742 (0.0007) [2023-12-27 06:48:08,829][20347] Updated weights for policy 0, policy_version 165752 (0.0008) [2023-12-27 06:48:08,886][20347] Updated weights for policy 0, policy_version 165762 (0.0008) [2023-12-27 06:48:08,891][20381] Updated weights for policy 1, policy_version 165466 (0.0007) [2023-12-27 06:48:08,951][20381] Updated weights for policy 1, policy_version 165476 (0.0006) [2023-12-27 06:48:09,003][20381] Updated weights for policy 1, policy_version 165486 (0.0007) [2023-12-27 06:48:09,060][20381] Updated weights for policy 1, policy_version 165496 (0.0010) [2023-12-27 06:48:09,560][20347] Updated weights for policy 0, policy_version 165772 (0.0007) [2023-12-27 06:48:09,621][20347] Updated weights for policy 0, policy_version 165782 (0.0007) [2023-12-27 06:48:09,685][20347] Updated weights for policy 0, policy_version 165792 (0.0007) [2023-12-27 06:48:09,738][20381] Updated weights for policy 1, policy_version 165506 (0.0011) [2023-12-27 06:48:09,793][20381] Updated weights for policy 1, policy_version 165516 (0.0011) [2023-12-27 06:48:09,856][20381] Updated weights for policy 1, policy_version 165526 (0.0008) [2023-12-27 06:48:09,864][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-27 06:48:10,346][20347] Updated weights for policy 0, policy_version 165802 (0.0009) [2023-12-27 06:48:10,408][20347] Updated weights for policy 0, policy_version 165812 (0.0011) [2023-12-27 06:48:10,472][20347] Updated weights for policy 0, policy_version 165822 (0.0006) [2023-12-27 06:48:10,535][20347] Updated weights for policy 0, policy_version 165832 (0.0005) [2023-12-27 06:48:10,590][20381] Updated weights for policy 1, policy_version 165536 (0.0010) [2023-12-27 06:48:10,642][20381] Updated weights for policy 1, policy_version 165546 (0.0008) [2023-12-27 06:48:10,700][20381] Updated weights for policy 1, policy_version 165556 (0.0008) [2023-12-27 06:48:11,086][20347] Updated weights for policy 0, policy_version 165842 (0.0008) [2023-12-27 06:48:11,157][20347] Updated weights for policy 0, policy_version 165852 (0.0008) [2023-12-27 06:48:11,216][20347] Updated weights for policy 0, policy_version 165862 (0.0007) [2023-12-27 06:48:11,397][20381] Updated weights for policy 1, policy_version 165566 (0.0008) [2023-12-27 06:48:11,464][20381] Updated weights for policy 1, policy_version 165576 (0.0007) [2023-12-27 06:48:11,527][20381] Updated weights for policy 1, policy_version 165586 (0.0005) [2023-12-27 06:48:11,927][20347] Updated weights for policy 0, policy_version 165872 (0.0008) [2023-12-27 06:48:11,992][20347] Updated weights for policy 0, policy_version 165882 (0.0006) [2023-12-27 06:48:12,050][20347] Updated weights for policy 0, policy_version 165892 (0.0010) [2023-12-27 06:48:12,166][20381] Updated weights for policy 1, policy_version 165596 (0.0007) [2023-12-27 06:48:12,224][20381] Updated weights for policy 1, policy_version 165606 (0.0008) [2023-12-27 06:48:12,287][20381] Updated weights for policy 1, policy_version 165616 (0.0008) [2023-12-27 06:48:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 84893696. Throughput: 0: 10451.4, 1: 10409.4. Samples: 84879876. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:12,655][20347] Updated weights for policy 0, policy_version 165902 (0.0010) [2023-12-27 06:48:12,707][20347] Updated weights for policy 0, policy_version 165912 (0.0010) [2023-12-27 06:48:12,759][20347] Updated weights for policy 0, policy_version 165922 (0.0010) [2023-12-27 06:48:13,000][20381] Updated weights for policy 1, policy_version 165626 (0.0009) [2023-12-27 06:48:13,055][20381] Updated weights for policy 1, policy_version 165636 (0.0006) [2023-12-27 06:48:13,111][20381] Updated weights for policy 1, policy_version 165646 (0.0006) [2023-12-27 06:48:13,160][20381] Updated weights for policy 1, policy_version 165656 (0.0008) [2023-12-27 06:48:13,384][20347] Updated weights for policy 0, policy_version 165932 (0.0008) [2023-12-27 06:48:13,445][20347] Updated weights for policy 0, policy_version 165942 (0.0008) [2023-12-27 06:48:13,499][20347] Updated weights for policy 0, policy_version 165952 (0.0007) [2023-12-27 06:48:13,846][20381] Updated weights for policy 1, policy_version 165666 (0.0007) [2023-12-27 06:48:13,893][20381] Updated weights for policy 1, policy_version 165676 (0.0008) [2023-12-27 06:48:13,941][20381] Updated weights for policy 1, policy_version 165686 (0.0008) [2023-12-27 06:48:14,152][20347] Updated weights for policy 0, policy_version 165962 (0.0008) [2023-12-27 06:48:14,211][20347] Updated weights for policy 0, policy_version 165972 (0.0009) [2023-12-27 06:48:14,265][20347] Updated weights for policy 0, policy_version 165982 (0.0007) [2023-12-27 06:48:14,317][20347] Updated weights for policy 0, policy_version 165992 (0.0007) [2023-12-27 06:48:14,623][20381] Updated weights for policy 1, policy_version 165696 (0.0008) [2023-12-27 06:48:14,679][20381] Updated weights for policy 1, policy_version 165706 (0.0007) [2023-12-27 06:48:14,726][20381] Updated weights for policy 1, policy_version 165716 (0.0005) [2023-12-27 06:48:15,012][20347] Updated weights for policy 0, policy_version 166002 (0.0006) [2023-12-27 06:48:15,073][20347] Updated weights for policy 0, policy_version 166012 (0.0006) [2023-12-27 06:48:15,130][20347] Updated weights for policy 0, policy_version 166022 (0.0007) [2023-12-27 06:48:15,486][20381] Updated weights for policy 1, policy_version 165726 (0.0007) [2023-12-27 06:48:15,542][20381] Updated weights for policy 1, policy_version 165736 (0.0007) [2023-12-27 06:48:15,599][20381] Updated weights for policy 1, policy_version 165746 (0.0008) [2023-12-27 06:48:15,690][20347] Updated weights for policy 0, policy_version 166032 (0.0007) [2023-12-27 06:48:15,752][20347] Updated weights for policy 0, policy_version 166042 (0.0008) [2023-12-27 06:48:15,807][20347] Updated weights for policy 0, policy_version 166052 (0.0008) [2023-12-27 06:48:16,279][20381] Updated weights for policy 1, policy_version 165756 (0.0007) [2023-12-27 06:48:16,345][20381] Updated weights for policy 1, policy_version 165766 (0.0006) [2023-12-27 06:48:16,404][20381] Updated weights for policy 1, policy_version 165776 (0.0006) [2023-12-27 06:48:16,463][20347] Updated weights for policy 0, policy_version 166062 (0.0006) [2023-12-27 06:48:16,525][20347] Updated weights for policy 0, policy_version 166072 (0.0005) [2023-12-27 06:48:16,589][20347] Updated weights for policy 0, policy_version 166082 (0.0005) [2023-12-27 06:48:17,002][20381] Updated weights for policy 1, policy_version 165786 (0.0008) [2023-12-27 06:48:17,066][20381] Updated weights for policy 1, policy_version 165796 (0.0007) [2023-12-27 06:48:17,096][20347] Updated weights for policy 0, policy_version 166092 (0.0005) [2023-12-27 06:48:17,114][20381] Updated weights for policy 1, policy_version 165806 (0.0010) [2023-12-27 06:48:17,157][20347] Updated weights for policy 0, policy_version 166102 (0.0008) [2023-12-27 06:48:17,171][20381] Updated weights for policy 1, policy_version 165816 (0.0010) [2023-12-27 06:48:17,214][20347] Updated weights for policy 0, policy_version 166112 (0.0010) [2023-12-27 06:48:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.1). Total num frames: 85000192. Throughput: 0: 10442.1, 1: 10403.4. Samples: 85005704. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:17,787][20381] Updated weights for policy 1, policy_version 165826 (0.0006) [2023-12-27 06:48:17,842][20381] Updated weights for policy 1, policy_version 165836 (0.0006) [2023-12-27 06:48:17,876][20347] Updated weights for policy 0, policy_version 166122 (0.0010) [2023-12-27 06:48:17,898][20381] Updated weights for policy 1, policy_version 165846 (0.0007) [2023-12-27 06:48:17,935][20347] Updated weights for policy 0, policy_version 166132 (0.0009) [2023-12-27 06:48:17,986][20347] Updated weights for policy 0, policy_version 166142 (0.0005) [2023-12-27 06:48:18,043][20347] Updated weights for policy 0, policy_version 166152 (0.0006) [2023-12-27 06:48:18,541][20381] Updated weights for policy 1, policy_version 165856 (0.0007) [2023-12-27 06:48:18,598][20381] Updated weights for policy 1, policy_version 165866 (0.0007) [2023-12-27 06:48:18,650][20381] Updated weights for policy 1, policy_version 165876 (0.0008) [2023-12-27 06:48:18,698][20347] Updated weights for policy 0, policy_version 166162 (0.0009) [2023-12-27 06:48:18,742][20347] Updated weights for policy 0, policy_version 166172 (0.0006) [2023-12-27 06:48:18,798][20347] Updated weights for policy 0, policy_version 166182 (0.0008) [2023-12-27 06:48:19,378][20381] Updated weights for policy 1, policy_version 165886 (0.0008) [2023-12-27 06:48:19,428][20381] Updated weights for policy 1, policy_version 165896 (0.0006) [2023-12-27 06:48:19,488][20381] Updated weights for policy 1, policy_version 165906 (0.0006) [2023-12-27 06:48:19,518][20347] Updated weights for policy 0, policy_version 166192 (0.0008) [2023-12-27 06:48:19,575][20347] Updated weights for policy 0, policy_version 166202 (0.0007) [2023-12-27 06:48:19,634][20347] Updated weights for policy 0, policy_version 166212 (0.0007) [2023-12-27 06:48:20,107][20381] Updated weights for policy 1, policy_version 165916 (0.0007) [2023-12-27 06:48:20,170][20381] Updated weights for policy 1, policy_version 165926 (0.0006) [2023-12-27 06:48:20,230][20381] Updated weights for policy 1, policy_version 165936 (0.0009) [2023-12-27 06:48:20,312][20347] Updated weights for policy 0, policy_version 166222 (0.0011) [2023-12-27 06:48:20,371][20347] Updated weights for policy 0, policy_version 166232 (0.0011) [2023-12-27 06:48:20,429][20347] Updated weights for policy 0, policy_version 166242 (0.0010) [2023-12-27 06:48:20,901][20381] Updated weights for policy 1, policy_version 165946 (0.0008) [2023-12-27 06:48:20,958][20381] Updated weights for policy 1, policy_version 165956 (0.0007) [2023-12-27 06:48:21,015][20381] Updated weights for policy 1, policy_version 165966 (0.0009) [2023-12-27 06:48:21,074][20381] Updated weights for policy 1, policy_version 165976 (0.0009) [2023-12-27 06:48:21,082][20347] Updated weights for policy 0, policy_version 166252 (0.0008) [2023-12-27 06:48:21,149][20347] Updated weights for policy 0, policy_version 166262 (0.0009) [2023-12-27 06:48:21,212][20347] Updated weights for policy 0, policy_version 166272 (0.0011) [2023-12-27 06:48:21,778][20381] Updated weights for policy 1, policy_version 165986 (0.0008) [2023-12-27 06:48:21,843][20381] Updated weights for policy 1, policy_version 165996 (0.0007) [2023-12-27 06:48:21,910][20381] Updated weights for policy 1, policy_version 166006 (0.0008) [2023-12-27 06:48:21,954][20347] Updated weights for policy 0, policy_version 166282 (0.0010) [2023-12-27 06:48:22,005][20347] Updated weights for policy 0, policy_version 166292 (0.0008) [2023-12-27 06:48:22,055][20347] Updated weights for policy 0, policy_version 166302 (0.0008) [2023-12-27 06:48:22,120][20347] Updated weights for policy 0, policy_version 166312 (0.0008) [2023-12-27 06:48:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 85098496. Throughput: 0: 10450.2, 1: 10373.2. Samples: 85067816. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000166008_42508288.pth... [2023-12-27 06:48:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000166312_42590208.pth... [2023-12-27 06:48:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000164792_42196992.pth [2023-12-27 06:48:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000165064_42270720.pth [2023-12-27 06:48:22,548][20381] Updated weights for policy 1, policy_version 166016 (0.0006) [2023-12-27 06:48:22,609][20381] Updated weights for policy 1, policy_version 166026 (0.0007) [2023-12-27 06:48:22,667][20381] Updated weights for policy 1, policy_version 166036 (0.0006) [2023-12-27 06:48:22,887][20347] Updated weights for policy 0, policy_version 166322 (0.0008) [2023-12-27 06:48:22,950][20347] Updated weights for policy 0, policy_version 166332 (0.0008) [2023-12-27 06:48:23,014][20347] Updated weights for policy 0, policy_version 166342 (0.0008) [2023-12-27 06:48:23,251][20381] Updated weights for policy 1, policy_version 166046 (0.0006) [2023-12-27 06:48:23,307][20381] Updated weights for policy 1, policy_version 166056 (0.0005) [2023-12-27 06:48:23,358][20381] Updated weights for policy 1, policy_version 166066 (0.0005) [2023-12-27 06:48:23,742][20347] Updated weights for policy 0, policy_version 166352 (0.0008) [2023-12-27 06:48:23,803][20347] Updated weights for policy 0, policy_version 166362 (0.0008) [2023-12-27 06:48:23,856][20347] Updated weights for policy 0, policy_version 166372 (0.0007) [2023-12-27 06:48:23,896][20381] Updated weights for policy 1, policy_version 166076 (0.0006) [2023-12-27 06:48:23,951][20381] Updated weights for policy 1, policy_version 166086 (0.0006) [2023-12-27 06:48:24,000][20381] Updated weights for policy 1, policy_version 166096 (0.0006) [2023-12-27 06:48:24,562][20347] Updated weights for policy 0, policy_version 166382 (0.0008) [2023-12-27 06:48:24,607][20381] Updated weights for policy 1, policy_version 166106 (0.0005) [2023-12-27 06:48:24,616][20347] Updated weights for policy 0, policy_version 166392 (0.0007) [2023-12-27 06:48:24,653][20381] Updated weights for policy 1, policy_version 166116 (0.0006) [2023-12-27 06:48:24,672][20347] Updated weights for policy 0, policy_version 166402 (0.0007) [2023-12-27 06:48:24,711][20381] Updated weights for policy 1, policy_version 166126 (0.0007) [2023-12-27 06:48:24,767][20381] Updated weights for policy 1, policy_version 166136 (0.0007) [2023-12-27 06:48:25,341][20347] Updated weights for policy 0, policy_version 166412 (0.0005) [2023-12-27 06:48:25,392][20347] Updated weights for policy 0, policy_version 166422 (0.0005) [2023-12-27 06:48:25,452][20347] Updated weights for policy 0, policy_version 166432 (0.0006) [2023-12-27 06:48:25,470][20381] Updated weights for policy 1, policy_version 166146 (0.0008) [2023-12-27 06:48:25,524][20381] Updated weights for policy 1, policy_version 166156 (0.0007) [2023-12-27 06:48:25,582][20381] Updated weights for policy 1, policy_version 166166 (0.0008) [2023-12-27 06:48:26,114][20347] Updated weights for policy 0, policy_version 166442 (0.0006) [2023-12-27 06:48:26,168][20347] Updated weights for policy 0, policy_version 166452 (0.0007) [2023-12-27 06:48:26,223][20347] Updated weights for policy 0, policy_version 166462 (0.0006) [2023-12-27 06:48:26,268][20381] Updated weights for policy 1, policy_version 166176 (0.0007) [2023-12-27 06:48:26,277][20347] Updated weights for policy 0, policy_version 166472 (0.0005) [2023-12-27 06:48:26,322][20381] Updated weights for policy 1, policy_version 166186 (0.0008) [2023-12-27 06:48:26,377][20381] Updated weights for policy 1, policy_version 166196 (0.0009) [2023-12-27 06:48:26,806][20347] Updated weights for policy 0, policy_version 166482 (0.0007) [2023-12-27 06:48:26,853][20347] Updated weights for policy 0, policy_version 166492 (0.0007) [2023-12-27 06:48:26,912][20347] Updated weights for policy 0, policy_version 166502 (0.0005) [2023-12-27 06:48:27,107][20381] Updated weights for policy 1, policy_version 166206 (0.0008) [2023-12-27 06:48:27,158][20381] Updated weights for policy 1, policy_version 166216 (0.0008) [2023-12-27 06:48:27,228][20381] Updated weights for policy 1, policy_version 166226 (0.0008) [2023-12-27 06:48:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 85204992. Throughput: 0: 10378.5, 1: 10492.0. Samples: 85191044. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:27,486][20347] Updated weights for policy 0, policy_version 166512 (0.0007) [2023-12-27 06:48:27,544][20347] Updated weights for policy 0, policy_version 166522 (0.0007) [2023-12-27 06:48:27,598][20347] Updated weights for policy 0, policy_version 166532 (0.0007) [2023-12-27 06:48:27,889][20381] Updated weights for policy 1, policy_version 166236 (0.0008) [2023-12-27 06:48:27,950][20381] Updated weights for policy 1, policy_version 166246 (0.0008) [2023-12-27 06:48:27,994][20381] Updated weights for policy 1, policy_version 166256 (0.0007) [2023-12-27 06:48:28,232][20347] Updated weights for policy 0, policy_version 166542 (0.0006) [2023-12-27 06:48:28,288][20347] Updated weights for policy 0, policy_version 166552 (0.0007) [2023-12-27 06:48:28,340][20347] Updated weights for policy 0, policy_version 166562 (0.0007) [2023-12-27 06:48:28,726][20381] Updated weights for policy 1, policy_version 166266 (0.0009) [2023-12-27 06:48:28,784][20381] Updated weights for policy 1, policy_version 166276 (0.0007) [2023-12-27 06:48:28,843][20381] Updated weights for policy 1, policy_version 166286 (0.0008) [2023-12-27 06:48:28,898][20381] Updated weights for policy 1, policy_version 166296 (0.0007) [2023-12-27 06:48:29,012][20347] Updated weights for policy 0, policy_version 166572 (0.0006) [2023-12-27 06:48:29,070][20347] Updated weights for policy 0, policy_version 166582 (0.0008) [2023-12-27 06:48:29,125][20347] Updated weights for policy 0, policy_version 166592 (0.0008) [2023-12-27 06:48:29,582][20381] Updated weights for policy 1, policy_version 166306 (0.0008) [2023-12-27 06:48:29,634][20381] Updated weights for policy 1, policy_version 166316 (0.0008) [2023-12-27 06:48:29,685][20381] Updated weights for policy 1, policy_version 166326 (0.0008) [2023-12-27 06:48:29,801][20347] Updated weights for policy 0, policy_version 166602 (0.0007) [2023-12-27 06:48:29,860][20347] Updated weights for policy 0, policy_version 166612 (0.0007) [2023-12-27 06:48:29,919][20347] Updated weights for policy 0, policy_version 166622 (0.0006) [2023-12-27 06:48:29,987][20347] Updated weights for policy 0, policy_version 166632 (0.0008) [2023-12-27 06:48:30,385][20381] Updated weights for policy 1, policy_version 166336 (0.0006) [2023-12-27 06:48:30,440][20381] Updated weights for policy 1, policy_version 166346 (0.0005) [2023-12-27 06:48:30,491][20381] Updated weights for policy 1, policy_version 166356 (0.0008) [2023-12-27 06:48:30,530][20347] Updated weights for policy 0, policy_version 166642 (0.0006) [2023-12-27 06:48:30,582][20347] Updated weights for policy 0, policy_version 166652 (0.0007) [2023-12-27 06:48:30,626][20347] Updated weights for policy 0, policy_version 166662 (0.0007) [2023-12-27 06:48:31,149][20381] Updated weights for policy 1, policy_version 166366 (0.0008) [2023-12-27 06:48:31,199][20381] Updated weights for policy 1, policy_version 166376 (0.0005) [2023-12-27 06:48:31,258][20381] Updated weights for policy 1, policy_version 166386 (0.0006) [2023-12-27 06:48:31,306][20347] Updated weights for policy 0, policy_version 166672 (0.0008) [2023-12-27 06:48:31,358][20347] Updated weights for policy 0, policy_version 166682 (0.0008) [2023-12-27 06:48:31,417][20347] Updated weights for policy 0, policy_version 166692 (0.0008) [2023-12-27 06:48:31,977][20381] Updated weights for policy 1, policy_version 166396 (0.0008) [2023-12-27 06:48:32,034][20381] Updated weights for policy 1, policy_version 166406 (0.0008) [2023-12-27 06:48:32,051][20347] Updated weights for policy 0, policy_version 166702 (0.0007) [2023-12-27 06:48:32,082][20381] Updated weights for policy 1, policy_version 166416 (0.0005) [2023-12-27 06:48:32,110][20347] Updated weights for policy 0, policy_version 166712 (0.0006) [2023-12-27 06:48:32,167][20347] Updated weights for policy 0, policy_version 166722 (0.0007) [2023-12-27 06:48:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 85311488. Throughput: 0: 10506.0, 1: 10377.6. Samples: 85317564. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:32,786][20381] Updated weights for policy 1, policy_version 166426 (0.0006) [2023-12-27 06:48:32,837][20347] Updated weights for policy 0, policy_version 166732 (0.0007) [2023-12-27 06:48:32,839][20381] Updated weights for policy 1, policy_version 166436 (0.0007) [2023-12-27 06:48:32,887][20381] Updated weights for policy 1, policy_version 166446 (0.0005) [2023-12-27 06:48:32,892][20347] Updated weights for policy 0, policy_version 166742 (0.0007) [2023-12-27 06:48:32,932][20381] Updated weights for policy 1, policy_version 166456 (0.0006) [2023-12-27 06:48:32,948][20347] Updated weights for policy 0, policy_version 166752 (0.0007) [2023-12-27 06:48:33,605][20381] Updated weights for policy 1, policy_version 166466 (0.0010) [2023-12-27 06:48:33,657][20347] Updated weights for policy 0, policy_version 166762 (0.0007) [2023-12-27 06:48:33,667][20381] Updated weights for policy 1, policy_version 166476 (0.0006) [2023-12-27 06:48:33,715][20347] Updated weights for policy 0, policy_version 166772 (0.0007) [2023-12-27 06:48:33,730][20381] Updated weights for policy 1, policy_version 166486 (0.0008) [2023-12-27 06:48:33,768][20347] Updated weights for policy 0, policy_version 166782 (0.0008) [2023-12-27 06:48:33,818][20347] Updated weights for policy 0, policy_version 166792 (0.0006) [2023-12-27 06:48:34,368][20381] Updated weights for policy 1, policy_version 166496 (0.0009) [2023-12-27 06:48:34,433][20381] Updated weights for policy 1, policy_version 166506 (0.0007) [2023-12-27 06:48:34,459][20347] Updated weights for policy 0, policy_version 166802 (0.0007) [2023-12-27 06:48:34,493][20381] Updated weights for policy 1, policy_version 166516 (0.0007) [2023-12-27 06:48:34,517][20347] Updated weights for policy 0, policy_version 166812 (0.0006) [2023-12-27 06:48:34,574][20347] Updated weights for policy 0, policy_version 166822 (0.0006) [2023-12-27 06:48:35,185][20381] Updated weights for policy 1, policy_version 166526 (0.0008) [2023-12-27 06:48:35,240][20381] Updated weights for policy 1, policy_version 166536 (0.0007) [2023-12-27 06:48:35,249][20347] Updated weights for policy 0, policy_version 166832 (0.0006) [2023-12-27 06:48:35,296][20381] Updated weights for policy 1, policy_version 166546 (0.0005) [2023-12-27 06:48:35,305][20347] Updated weights for policy 0, policy_version 166842 (0.0006) [2023-12-27 06:48:35,360][20347] Updated weights for policy 0, policy_version 166852 (0.0008) [2023-12-27 06:48:35,920][20381] Updated weights for policy 1, policy_version 166556 (0.0007) [2023-12-27 06:48:35,971][20381] Updated weights for policy 1, policy_version 166566 (0.0010) [2023-12-27 06:48:36,020][20381] Updated weights for policy 1, policy_version 166576 (0.0010) [2023-12-27 06:48:36,062][20347] Updated weights for policy 0, policy_version 166862 (0.0009) [2023-12-27 06:48:36,119][20347] Updated weights for policy 0, policy_version 166872 (0.0009) [2023-12-27 06:48:36,171][20347] Updated weights for policy 0, policy_version 166882 (0.0008) [2023-12-27 06:48:36,732][20381] Updated weights for policy 1, policy_version 166586 (0.0011) [2023-12-27 06:48:36,795][20381] Updated weights for policy 1, policy_version 166596 (0.0011) [2023-12-27 06:48:36,853][20381] Updated weights for policy 1, policy_version 166606 (0.0011) [2023-12-27 06:48:36,867][20347] Updated weights for policy 0, policy_version 166892 (0.0009) [2023-12-27 06:48:36,907][20381] Updated weights for policy 1, policy_version 166616 (0.0011) [2023-12-27 06:48:36,923][20347] Updated weights for policy 0, policy_version 166902 (0.0010) [2023-12-27 06:48:36,975][20347] Updated weights for policy 0, policy_version 166912 (0.0010) [2023-12-27 06:48:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 85409792. Throughput: 0: 10488.7, 1: 10331.6. Samples: 85379172. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:48:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000166616_42663936.pth... [2023-12-27 06:48:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000166920_42745856.pth... [2023-12-27 06:48:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000165704_42434560.pth [2023-12-27 06:48:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000165432_42360832.pth [2023-12-27 06:48:37,615][20381] Updated weights for policy 1, policy_version 166626 (0.0010) [2023-12-27 06:48:37,620][20347] Updated weights for policy 0, policy_version 166922 (0.0010) [2023-12-27 06:48:37,673][20381] Updated weights for policy 1, policy_version 166636 (0.0008) [2023-12-27 06:48:37,678][20347] Updated weights for policy 0, policy_version 166932 (0.0010) [2023-12-27 06:48:37,728][20381] Updated weights for policy 1, policy_version 166646 (0.0007) [2023-12-27 06:48:37,733][20347] Updated weights for policy 0, policy_version 166942 (0.0010) [2023-12-27 06:48:37,793][20347] Updated weights for policy 0, policy_version 166952 (0.0010) [2023-12-27 06:48:38,439][20381] Updated weights for policy 1, policy_version 166656 (0.0006) [2023-12-27 06:48:38,486][20347] Updated weights for policy 0, policy_version 166962 (0.0008) [2023-12-27 06:48:38,496][20381] Updated weights for policy 1, policy_version 166666 (0.0005) [2023-12-27 06:48:38,546][20381] Updated weights for policy 1, policy_version 166676 (0.0007) [2023-12-27 06:48:38,552][20347] Updated weights for policy 0, policy_version 166972 (0.0008) [2023-12-27 06:48:38,615][20347] Updated weights for policy 0, policy_version 166982 (0.0008) [2023-12-27 06:48:39,234][20381] Updated weights for policy 1, policy_version 166686 (0.0007) [2023-12-27 06:48:39,295][20381] Updated weights for policy 1, policy_version 166696 (0.0011) [2023-12-27 06:48:39,327][20347] Updated weights for policy 0, policy_version 166992 (0.0008) [2023-12-27 06:48:39,362][20381] Updated weights for policy 1, policy_version 166706 (0.0008) [2023-12-27 06:48:39,390][20347] Updated weights for policy 0, policy_version 167002 (0.0008) [2023-12-27 06:48:39,452][20347] Updated weights for policy 0, policy_version 167012 (0.0011) [2023-12-27 06:48:40,135][20381] Updated weights for policy 1, policy_version 166716 (0.0008) [2023-12-27 06:48:40,192][20381] Updated weights for policy 1, policy_version 166726 (0.0009) [2023-12-27 06:48:40,197][20347] Updated weights for policy 0, policy_version 167022 (0.0009) [2023-12-27 06:48:40,250][20381] Updated weights for policy 1, policy_version 166736 (0.0009) [2023-12-27 06:48:40,255][20347] Updated weights for policy 0, policy_version 167032 (0.0009) [2023-12-27 06:48:40,312][20347] Updated weights for policy 0, policy_version 167042 (0.0009) [2023-12-27 06:48:40,970][20381] Updated weights for policy 1, policy_version 166746 (0.0008) [2023-12-27 06:48:41,021][20347] Updated weights for policy 0, policy_version 167052 (0.0009) [2023-12-27 06:48:41,028][20381] Updated weights for policy 1, policy_version 166756 (0.0010) [2023-12-27 06:48:41,092][20347] Updated weights for policy 0, policy_version 167062 (0.0008) [2023-12-27 06:48:41,094][20381] Updated weights for policy 1, policy_version 166766 (0.0011) [2023-12-27 06:48:41,156][20381] Updated weights for policy 1, policy_version 166776 (0.0009) [2023-12-27 06:48:41,162][20347] Updated weights for policy 0, policy_version 167072 (0.0008) [2023-12-27 06:48:41,871][20292] KL-divergence is very high: 147.5087 [2023-12-27 06:48:41,883][20292] KL-divergence is very high: 180.6161 [2023-12-27 06:48:41,899][20347] Updated weights for policy 0, policy_version 167082 (0.0008) [2023-12-27 06:48:41,899][20381] Updated weights for policy 1, policy_version 166786 (0.0008) [2023-12-27 06:48:41,922][20292] KL-divergence is very high: 102.3512 [2023-12-27 06:48:41,935][20292] KL-divergence is very high: 260.0623 [2023-12-27 06:48:41,957][20347] Updated weights for policy 0, policy_version 167092 (0.0007) [2023-12-27 06:48:41,965][20381] Updated weights for policy 1, policy_version 166796 (0.0006) [2023-12-27 06:48:42,015][20347] Updated weights for policy 0, policy_version 167102 (0.0007) [2023-12-27 06:48:42,024][20381] Updated weights for policy 1, policy_version 166806 (0.0007) [2023-12-27 06:48:42,067][20347] Updated weights for policy 0, policy_version 167112 (0.0008) [2023-12-27 06:48:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 85508096. Throughput: 0: 10460.4, 1: 10233.8. Samples: 85497228. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '964.360')] [2023-12-27 06:48:42,644][20292] KL-divergence is very high: 144.9684 [2023-12-27 06:48:42,656][20292] KL-divergence is very high: 224.1122 [2023-12-27 06:48:42,666][20292] KL-divergence is very high: 211.3063 [2023-12-27 06:48:42,673][20292] KL-divergence is very high: 382.2417 [2023-12-27 06:48:42,679][20292] KL-divergence is very high: 198.3328 [2023-12-27 06:48:42,686][20292] KL-divergence is very high: 263.6439 [2023-12-27 06:48:42,691][20381] Updated weights for policy 1, policy_version 166816 (0.0008) [2023-12-27 06:48:42,692][20292] KL-divergence is very high: 312.2323 [2023-12-27 06:48:42,705][20292] KL-divergence is very high: 263.8543 [2023-12-27 06:48:42,718][20292] KL-divergence is very high: 237.5260 [2023-12-27 06:48:42,725][20292] KL-divergence is very high: 1252.8373 [2023-12-27 06:48:42,732][20292] KL-divergence is very high: 223.2627 [2023-12-27 06:48:42,734][20347] Updated weights for policy 0, policy_version 167122 (0.0010) [2023-12-27 06:48:42,739][20292] KL-divergence is very high: 291.0368 [2023-12-27 06:48:42,746][20292] KL-divergence is very high: 113.2034 [2023-12-27 06:48:42,757][20381] Updated weights for policy 1, policy_version 166826 (0.0011) [2023-12-27 06:48:42,777][20292] KL-divergence is very high: 438.8399 [2023-12-27 06:48:42,789][20292] KL-divergence is very high: 112.4584 [2023-12-27 06:48:42,793][20347] Updated weights for policy 0, policy_version 167132 (0.0010) [2023-12-27 06:48:42,795][20292] KL-divergence is very high: 193.5695 [2023-12-27 06:48:42,807][20292] KL-divergence is very high: 214.8793 [2023-12-27 06:48:42,819][20381] Updated weights for policy 1, policy_version 166836 (0.0011) [2023-12-27 06:48:42,825][20292] KL-divergence is very high: 585.7938 [2023-12-27 06:48:42,837][20292] KL-divergence is very high: 368.2201 [2023-12-27 06:48:42,855][20347] Updated weights for policy 0, policy_version 167142 (0.0010) [2023-12-27 06:48:43,469][20292] KL-divergence is very high: 147.5462 [2023-12-27 06:48:43,476][20292] KL-divergence is very high: 159.7533 [2023-12-27 06:48:43,492][20381] Updated weights for policy 1, policy_version 166846 (0.0011) [2023-12-27 06:48:43,510][20347] Updated weights for policy 0, policy_version 167152 (0.0010) [2023-12-27 06:48:43,527][20292] KL-divergence is very high: 119.6639 [2023-12-27 06:48:43,559][20381] Updated weights for policy 1, policy_version 166856 (0.0010) [2023-12-27 06:48:43,561][20347] Updated weights for policy 0, policy_version 167162 (0.0010) [2023-12-27 06:48:43,573][20292] KL-divergence is very high: 146.8951 [2023-12-27 06:48:43,580][20292] KL-divergence is very high: 139.1201 [2023-12-27 06:48:43,611][20292] KL-divergence is very high: 177.2315 [2023-12-27 06:48:43,617][20347] Updated weights for policy 0, policy_version 167172 (0.0010) [2023-12-27 06:48:43,618][20292] KL-divergence is very high: 248.8052 [2023-12-27 06:48:43,625][20292] KL-divergence is very high: 468.8993 [2023-12-27 06:48:43,626][20381] Updated weights for policy 1, policy_version 166866 (0.0011) [2023-12-27 06:48:43,631][20292] KL-divergence is very high: 324.0811 [2023-12-27 06:48:44,272][20292] KL-divergence is very high: 137.2481 [2023-12-27 06:48:44,278][20292] KL-divergence is very high: 140.0377 [2023-12-27 06:48:44,284][20292] KL-divergence is very high: 137.2572 [2023-12-27 06:48:44,288][20347] Updated weights for policy 0, policy_version 167182 (0.0010) [2023-12-27 06:48:44,290][20292] KL-divergence is very high: 173.8594 [2023-12-27 06:48:44,290][20381] Updated weights for policy 1, policy_version 166876 (0.0011) [2023-12-27 06:48:44,305][20292] KL-divergence is very high: 126.7824 [2023-12-27 06:48:44,310][20292] KL-divergence is very high: 133.2043 [2023-12-27 06:48:44,316][20292] KL-divergence is very high: 131.2194 [2023-12-27 06:48:44,321][20292] KL-divergence is very high: 291.5434 [2023-12-27 06:48:44,328][20292] KL-divergence is very high: 538.6411 [2023-12-27 06:48:44,335][20292] KL-divergence is very high: 820.0643 [2023-12-27 06:48:44,341][20292] KL-divergence is very high: 547.2538 [2023-12-27 06:48:44,345][20347] Updated weights for policy 0, policy_version 167192 (0.0010) [2023-12-27 06:48:44,346][20381] Updated weights for policy 1, policy_version 166886 (0.0011) [2023-12-27 06:48:44,363][20292] KL-divergence is very high: 127.8839 [2023-12-27 06:48:44,368][20292] KL-divergence is very high: 402.4111 [2023-12-27 06:48:44,373][20292] KL-divergence is very high: 207.6462 [2023-12-27 06:48:44,379][20292] KL-divergence is very high: 351.8582 [2023-12-27 06:48:44,384][20292] KL-divergence is very high: 123.9668 [2023-12-27 06:48:44,399][20347] Updated weights for policy 0, policy_version 167202 (0.0010) [2023-12-27 06:48:44,401][20381] Updated weights for policy 1, policy_version 166896 (0.0010) [2023-12-27 06:48:44,407][20292] KL-divergence is very high: 137.1865 [2023-12-27 06:48:44,413][20292] KL-divergence is very high: 198.8938 [2023-12-27 06:48:44,418][20292] KL-divergence is very high: 122.3349 [2023-12-27 06:48:44,424][20292] KL-divergence is very high: 123.6593 [2023-12-27 06:48:45,017][20292] KL-divergence is very high: 403.9601 [2023-12-27 06:48:45,024][20292] KL-divergence is very high: 611.5912 [2023-12-27 06:48:45,028][20381] Updated weights for policy 1, policy_version 166906 (0.0010) [2023-12-27 06:48:45,046][20347] Updated weights for policy 0, policy_version 167212 (0.0010) [2023-12-27 06:48:45,069][20292] KL-divergence is very high: 475.4568 [2023-12-27 06:48:45,076][20292] KL-divergence is very high: 615.1053 [2023-12-27 06:48:45,095][20381] Updated weights for policy 1, policy_version 166916 (0.0005) [2023-12-27 06:48:45,106][20347] Updated weights for policy 0, policy_version 167222 (0.0010) [2023-12-27 06:48:45,119][20292] KL-divergence is very high: 366.4550 [2023-12-27 06:48:45,125][20292] KL-divergence is very high: 478.6738 [2023-12-27 06:48:45,154][20381] Updated weights for policy 1, policy_version 166926 (0.0008) [2023-12-27 06:48:45,162][20347] Updated weights for policy 0, policy_version 167232 (0.0010) [2023-12-27 06:48:45,163][20292] KL-divergence is very high: 618.5568 [2023-12-27 06:48:45,169][20292] KL-divergence is very high: 800.4468 [2023-12-27 06:48:45,204][20381] Updated weights for policy 1, policy_version 166936 (0.0010) [2023-12-27 06:48:45,736][20292] KL-divergence is very high: 103.7000 [2023-12-27 06:48:45,786][20381] Updated weights for policy 1, policy_version 166946 (0.0009) [2023-12-27 06:48:45,797][20347] Updated weights for policy 0, policy_version 167242 (0.0010) [2023-12-27 06:48:45,835][20381] Updated weights for policy 1, policy_version 166956 (0.0010) [2023-12-27 06:48:45,853][20347] Updated weights for policy 0, policy_version 167252 (0.0010) [2023-12-27 06:48:45,887][20381] Updated weights for policy 1, policy_version 166966 (0.0010) [2023-12-27 06:48:45,911][20347] Updated weights for policy 0, policy_version 167262 (0.0010) [2023-12-27 06:48:45,966][20347] Updated weights for policy 0, policy_version 167272 (0.0010) [2023-12-27 06:48:46,530][20381] Updated weights for policy 1, policy_version 166976 (0.0007) [2023-12-27 06:48:46,534][20292] KL-divergence is very high: 124.7165 [2023-12-27 06:48:46,579][20347] Updated weights for policy 0, policy_version 167282 (0.0007) [2023-12-27 06:48:46,581][20381] Updated weights for policy 1, policy_version 166986 (0.0007) [2023-12-27 06:48:46,638][20381] Updated weights for policy 1, policy_version 166996 (0.0006) [2023-12-27 06:48:46,642][20347] Updated weights for policy 0, policy_version 167292 (0.0006) [2023-12-27 06:48:46,700][20347] Updated weights for policy 0, policy_version 167302 (0.0005) [2023-12-27 06:48:47,294][20381] Updated weights for policy 1, policy_version 167006 (0.0007) [2023-12-27 06:48:47,333][20347] Updated weights for policy 0, policy_version 167312 (0.0009) [2023-12-27 06:48:47,343][20381] Updated weights for policy 1, policy_version 167016 (0.0009) [2023-12-27 06:48:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 85606400. Throughput: 0: 10451.9, 1: 10259.4. Samples: 85622484. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '150.510')] [2023-12-27 06:48:47,386][20347] Updated weights for policy 0, policy_version 167322 (0.0006) [2023-12-27 06:48:47,395][20381] Updated weights for policy 1, policy_version 167026 (0.0010) [2023-12-27 06:48:47,443][20347] Updated weights for policy 0, policy_version 167332 (0.0005) [2023-12-27 06:48:47,985][20347] Updated weights for policy 0, policy_version 167342 (0.0007) [2023-12-27 06:48:48,037][20381] Updated weights for policy 1, policy_version 167036 (0.0010) [2023-12-27 06:48:48,038][20347] Updated weights for policy 0, policy_version 167352 (0.0010) [2023-12-27 06:48:48,039][20292] KL-divergence is very high: 155.9122 [2023-12-27 06:48:48,045][20292] KL-divergence is very high: 221.2605 [2023-12-27 06:48:48,057][20292] KL-divergence is very high: 286.5881 [2023-12-27 06:48:48,063][20292] KL-divergence is very high: 587.3765 [2023-12-27 06:48:48,068][20292] KL-divergence is very high: 363.5479 [2023-12-27 06:48:48,092][20347] Updated weights for policy 0, policy_version 167362 (0.0010) [2023-12-27 06:48:48,101][20381] Updated weights for policy 1, policy_version 167046 (0.0010) [2023-12-27 06:48:48,143][20292] KL-divergence is very high: 101.6549 [2023-12-27 06:48:48,148][20292] KL-divergence is very high: 117.7218 [2023-12-27 06:48:48,154][20292] KL-divergence is very high: 119.9633 [2023-12-27 06:48:48,159][20381] Updated weights for policy 1, policy_version 167056 (0.0010) [2023-12-27 06:48:48,768][20292] KL-divergence is very high: 101.1550 [2023-12-27 06:48:48,771][20347] Updated weights for policy 0, policy_version 167372 (0.0010) [2023-12-27 06:48:48,781][20381] Updated weights for policy 1, policy_version 167066 (0.0010) [2023-12-27 06:48:48,823][20347] Updated weights for policy 0, policy_version 167382 (0.0006) [2023-12-27 06:48:48,836][20381] Updated weights for policy 1, policy_version 167076 (0.0007) [2023-12-27 06:48:48,861][20292] KL-divergence is very high: 114.7517 [2023-12-27 06:48:48,880][20347] Updated weights for policy 0, policy_version 167392 (0.0007) [2023-12-27 06:48:48,896][20381] Updated weights for policy 1, policy_version 167086 (0.0006) [2023-12-27 06:48:48,954][20381] Updated weights for policy 1, policy_version 167096 (0.0006) [2023-12-27 06:48:49,468][20347] Updated weights for policy 0, policy_version 167402 (0.0007) [2023-12-27 06:48:49,530][20347] Updated weights for policy 0, policy_version 167412 (0.0010) [2023-12-27 06:48:49,588][20347] Updated weights for policy 0, policy_version 167422 (0.0010) [2023-12-27 06:48:49,632][20381] Updated weights for policy 1, policy_version 167106 (0.0006) [2023-12-27 06:48:49,647][20347] Updated weights for policy 0, policy_version 167432 (0.0010) [2023-12-27 06:48:49,661][20292] KL-divergence is very high: 135.7236 [2023-12-27 06:48:49,679][20381] Updated weights for policy 1, policy_version 167116 (0.0006) [2023-12-27 06:48:49,732][20381] Updated weights for policy 1, policy_version 167126 (0.0007) [2023-12-27 06:48:50,350][20347] Updated weights for policy 0, policy_version 167442 (0.0010) [2023-12-27 06:48:50,406][20347] Updated weights for policy 0, policy_version 167452 (0.0010) [2023-12-27 06:48:50,441][20381] Updated weights for policy 1, policy_version 167136 (0.0006) [2023-12-27 06:48:50,464][20347] Updated weights for policy 0, policy_version 167462 (0.0010) [2023-12-27 06:48:50,499][20381] Updated weights for policy 1, policy_version 167146 (0.0006) [2023-12-27 06:48:50,547][20292] KL-divergence is very high: 104.0048 [2023-12-27 06:48:50,560][20381] Updated weights for policy 1, policy_version 167156 (0.0007) [2023-12-27 06:48:51,170][20347] Updated weights for policy 0, policy_version 167472 (0.0006) [2023-12-27 06:48:51,218][20347] Updated weights for policy 0, policy_version 167482 (0.0007) [2023-12-27 06:48:51,230][20381] Updated weights for policy 1, policy_version 167166 (0.0007) [2023-12-27 06:48:51,275][20347] Updated weights for policy 0, policy_version 167492 (0.0006) [2023-12-27 06:48:51,291][20381] Updated weights for policy 1, policy_version 167176 (0.0009) [2023-12-27 06:48:51,352][20381] Updated weights for policy 1, policy_version 167186 (0.0007) [2023-12-27 06:48:52,008][20347] Updated weights for policy 0, policy_version 167502 (0.0008) [2023-12-27 06:48:52,057][20347] Updated weights for policy 0, policy_version 167512 (0.0007) [2023-12-27 06:48:52,083][20381] Updated weights for policy 1, policy_version 167196 (0.0009) [2023-12-27 06:48:52,104][20292] KL-divergence is very high: 107.2218 [2023-12-27 06:48:52,116][20347] Updated weights for policy 0, policy_version 167522 (0.0006) [2023-12-27 06:48:52,136][20381] Updated weights for policy 1, policy_version 167206 (0.0007) [2023-12-27 06:48:52,146][20292] KL-divergence is very high: 216.3537 [2023-12-27 06:48:52,151][20292] KL-divergence is very high: 155.4979 [2023-12-27 06:48:52,184][20292] KL-divergence is very high: 267.2862 [2023-12-27 06:48:52,185][20381] Updated weights for policy 1, policy_version 167216 (0.0009) [2023-12-27 06:48:52,189][20292] KL-divergence is very high: 268.4026 [2023-12-27 06:48:52,194][20292] KL-divergence is very high: 196.1481 [2023-12-27 06:48:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 85721088. Throughput: 0: 10484.3, 1: 10241.4. Samples: 85686652. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '79.360')] [2023-12-27 06:48:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000167224_42819584.pth... [2023-12-27 06:48:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000167528_42901504.pth... [2023-12-27 06:48:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000166008_42508288.pth [2023-12-27 06:48:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000166312_42590208.pth [2023-12-27 06:48:52,809][20347] Updated weights for policy 0, policy_version 167532 (0.0006) [2023-12-27 06:48:52,870][20347] Updated weights for policy 0, policy_version 167542 (0.0007) [2023-12-27 06:48:52,884][20292] KL-divergence is very high: 146.9588 [2023-12-27 06:48:52,888][20381] Updated weights for policy 1, policy_version 167226 (0.0007) [2023-12-27 06:48:52,890][20292] KL-divergence is very high: 113.2593 [2023-12-27 06:48:52,918][20292] KL-divergence is very high: 147.1538 [2023-12-27 06:48:52,923][20292] KL-divergence is very high: 129.5240 [2023-12-27 06:48:52,929][20347] Updated weights for policy 0, policy_version 167552 (0.0006) [2023-12-27 06:48:52,945][20381] Updated weights for policy 1, policy_version 167236 (0.0006) [2023-12-27 06:48:52,961][20292] KL-divergence is very high: 161.4465 [2023-12-27 06:48:52,977][20292] KL-divergence is very high: 114.6503 [2023-12-27 06:48:52,982][20292] KL-divergence is very high: 111.3062 [2023-12-27 06:48:53,001][20381] Updated weights for policy 1, policy_version 167246 (0.0005) [2023-12-27 06:48:53,002][20292] KL-divergence is very high: 123.8769 [2023-12-27 06:48:53,008][20292] KL-divergence is very high: 173.5380 [2023-12-27 06:48:53,024][20292] KL-divergence is very high: 106.3616 [2023-12-27 06:48:53,030][20292] KL-divergence is very high: 107.9270 [2023-12-27 06:48:53,052][20292] KL-divergence is very high: 143.5284 [2023-12-27 06:48:53,055][20381] Updated weights for policy 1, policy_version 167256 (0.0005) [2023-12-27 06:48:53,599][20347] Updated weights for policy 0, policy_version 167562 (0.0007) [2023-12-27 06:48:53,619][20381] Updated weights for policy 1, policy_version 167266 (0.0005) [2023-12-27 06:48:53,653][20347] Updated weights for policy 0, policy_version 167572 (0.0008) [2023-12-27 06:48:53,667][20381] Updated weights for policy 1, policy_version 167276 (0.0006) [2023-12-27 06:48:53,702][20347] Updated weights for policy 0, policy_version 167582 (0.0005) [2023-12-27 06:48:53,720][20381] Updated weights for policy 1, policy_version 167286 (0.0006) [2023-12-27 06:48:53,762][20347] Updated weights for policy 0, policy_version 167592 (0.0005) [2023-12-27 06:48:54,400][20347] Updated weights for policy 0, policy_version 167602 (0.0008) [2023-12-27 06:48:54,446][20381] Updated weights for policy 1, policy_version 167296 (0.0005) [2023-12-27 06:48:54,462][20347] Updated weights for policy 0, policy_version 167612 (0.0007) [2023-12-27 06:48:54,497][20381] Updated weights for policy 1, policy_version 167306 (0.0005) [2023-12-27 06:48:54,516][20347] Updated weights for policy 0, policy_version 167622 (0.0007) [2023-12-27 06:48:54,551][20381] Updated weights for policy 1, policy_version 167316 (0.0006) [2023-12-27 06:48:55,121][20381] Updated weights for policy 1, policy_version 167326 (0.0006) [2023-12-27 06:48:55,181][20381] Updated weights for policy 1, policy_version 167336 (0.0007) [2023-12-27 06:48:55,190][20347] Updated weights for policy 0, policy_version 167632 (0.0007) [2023-12-27 06:48:55,226][20381] Updated weights for policy 1, policy_version 167346 (0.0006) [2023-12-27 06:48:55,244][20347] Updated weights for policy 0, policy_version 167642 (0.0008) [2023-12-27 06:48:55,304][20347] Updated weights for policy 0, policy_version 167652 (0.0007) [2023-12-27 06:48:55,928][20381] Updated weights for policy 1, policy_version 167356 (0.0007) [2023-12-27 06:48:55,963][20347] Updated weights for policy 0, policy_version 167662 (0.0007) [2023-12-27 06:48:55,991][20381] Updated weights for policy 1, policy_version 167366 (0.0008) [2023-12-27 06:48:56,010][20347] Updated weights for policy 0, policy_version 167672 (0.0005) [2023-12-27 06:48:56,046][20381] Updated weights for policy 1, policy_version 167376 (0.0009) [2023-12-27 06:48:56,061][20347] Updated weights for policy 0, policy_version 167682 (0.0005) [2023-12-27 06:48:56,698][20347] Updated weights for policy 0, policy_version 167692 (0.0005) [2023-12-27 06:48:56,728][20381] Updated weights for policy 1, policy_version 167386 (0.0008) [2023-12-27 06:48:56,749][20347] Updated weights for policy 0, policy_version 167702 (0.0005) [2023-12-27 06:48:56,767][20292] KL-divergence is very high: 103.6111 [2023-12-27 06:48:56,784][20381] Updated weights for policy 1, policy_version 167396 (0.0008) [2023-12-27 06:48:56,810][20292] KL-divergence is very high: 109.3230 [2023-12-27 06:48:56,813][20347] Updated weights for policy 0, policy_version 167712 (0.0005) [2023-12-27 06:48:56,838][20381] Updated weights for policy 1, policy_version 167406 (0.0008) [2023-12-27 06:48:56,889][20381] Updated weights for policy 1, policy_version 167416 (0.0008) [2023-12-27 06:48:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 85819392. Throughput: 0: 10400.1, 1: 10283.0. Samples: 85810616. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '31.580')] [2023-12-27 06:48:57,385][20347] Updated weights for policy 0, policy_version 167722 (0.0006) [2023-12-27 06:48:57,441][20347] Updated weights for policy 0, policy_version 167732 (0.0007) [2023-12-27 06:48:57,488][20347] Updated weights for policy 0, policy_version 167742 (0.0008) [2023-12-27 06:48:57,544][20347] Updated weights for policy 0, policy_version 167752 (0.0008) [2023-12-27 06:48:57,603][20381] Updated weights for policy 1, policy_version 167426 (0.0010) [2023-12-27 06:48:57,660][20381] Updated weights for policy 1, policy_version 167436 (0.0010) [2023-12-27 06:48:57,717][20381] Updated weights for policy 1, policy_version 167446 (0.0010) [2023-12-27 06:48:58,173][20347] Updated weights for policy 0, policy_version 167762 (0.0008) [2023-12-27 06:48:58,235][20347] Updated weights for policy 0, policy_version 167772 (0.0008) [2023-12-27 06:48:58,291][20347] Updated weights for policy 0, policy_version 167782 (0.0008) [2023-12-27 06:48:58,406][20381] Updated weights for policy 1, policy_version 167456 (0.0008) [2023-12-27 06:48:58,468][20381] Updated weights for policy 1, policy_version 167466 (0.0008) [2023-12-27 06:48:58,529][20381] Updated weights for policy 1, policy_version 167476 (0.0007) [2023-12-27 06:48:59,113][20347] Updated weights for policy 0, policy_version 167792 (0.0009) [2023-12-27 06:48:59,172][20347] Updated weights for policy 0, policy_version 167802 (0.0009) [2023-12-27 06:48:59,235][20347] Updated weights for policy 0, policy_version 167812 (0.0008) [2023-12-27 06:48:59,236][20381] Updated weights for policy 1, policy_version 167486 (0.0009) [2023-12-27 06:48:59,295][20381] Updated weights for policy 1, policy_version 167496 (0.0007) [2023-12-27 06:48:59,351][20381] Updated weights for policy 1, policy_version 167506 (0.0009) [2023-12-27 06:48:59,879][20347] Updated weights for policy 0, policy_version 167822 (0.0008) [2023-12-27 06:48:59,942][20347] Updated weights for policy 0, policy_version 167832 (0.0008) [2023-12-27 06:49:00,004][20347] Updated weights for policy 0, policy_version 167842 (0.0008) [2023-12-27 06:49:00,025][20381] Updated weights for policy 1, policy_version 167516 (0.0007) [2023-12-27 06:49:00,077][20381] Updated weights for policy 1, policy_version 167526 (0.0007) [2023-12-27 06:49:00,130][20381] Updated weights for policy 1, policy_version 167536 (0.0008) [2023-12-27 06:49:00,672][20347] Updated weights for policy 0, policy_version 167852 (0.0007) [2023-12-27 06:49:00,723][20347] Updated weights for policy 0, policy_version 167862 (0.0007) [2023-12-27 06:49:00,787][20347] Updated weights for policy 0, policy_version 167872 (0.0008) [2023-12-27 06:49:00,811][20381] Updated weights for policy 1, policy_version 167546 (0.0008) [2023-12-27 06:49:00,870][20381] Updated weights for policy 1, policy_version 167556 (0.0008) [2023-12-27 06:49:00,915][20381] Updated weights for policy 1, policy_version 167566 (0.0007) [2023-12-27 06:49:00,962][20381] Updated weights for policy 1, policy_version 167576 (0.0007) [2023-12-27 06:49:01,491][20347] Updated weights for policy 0, policy_version 167882 (0.0008) [2023-12-27 06:49:01,552][20347] Updated weights for policy 0, policy_version 167892 (0.0008) [2023-12-27 06:49:01,613][20347] Updated weights for policy 0, policy_version 167902 (0.0008) [2023-12-27 06:49:01,672][20347] Updated weights for policy 0, policy_version 167912 (0.0008) [2023-12-27 06:49:01,690][20381] Updated weights for policy 1, policy_version 167586 (0.0007) [2023-12-27 06:49:01,749][20381] Updated weights for policy 1, policy_version 167596 (0.0008) [2023-12-27 06:49:01,805][20381] Updated weights for policy 1, policy_version 167606 (0.0005) [2023-12-27 06:49:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 85917696. Throughput: 0: 10316.6, 1: 10279.6. Samples: 85932532. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:49:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '33.040')] [2023-12-27 06:49:02,398][20347] Updated weights for policy 0, policy_version 167922 (0.0008) [2023-12-27 06:49:02,443][20381] Updated weights for policy 1, policy_version 167616 (0.0006) [2023-12-27 06:49:02,451][20347] Updated weights for policy 0, policy_version 167932 (0.0006) [2023-12-27 06:49:02,496][20381] Updated weights for policy 1, policy_version 167626 (0.0006) [2023-12-27 06:49:02,502][20347] Updated weights for policy 0, policy_version 167942 (0.0007) [2023-12-27 06:49:02,543][20381] Updated weights for policy 1, policy_version 167636 (0.0007) [2023-12-27 06:49:03,145][20381] Updated weights for policy 1, policy_version 167646 (0.0006) [2023-12-27 06:49:03,201][20381] Updated weights for policy 1, policy_version 167656 (0.0007) [2023-12-27 06:49:03,244][20347] Updated weights for policy 0, policy_version 167952 (0.0007) [2023-12-27 06:49:03,253][20381] Updated weights for policy 1, policy_version 167666 (0.0006) [2023-12-27 06:49:03,301][20347] Updated weights for policy 0, policy_version 167962 (0.0007) [2023-12-27 06:49:03,356][20347] Updated weights for policy 0, policy_version 167972 (0.0008) [2023-12-27 06:49:03,875][20381] Updated weights for policy 1, policy_version 167676 (0.0006) [2023-12-27 06:49:03,934][20381] Updated weights for policy 1, policy_version 167686 (0.0006) [2023-12-27 06:49:03,985][20381] Updated weights for policy 1, policy_version 167696 (0.0005) [2023-12-27 06:49:04,054][20347] Updated weights for policy 0, policy_version 167982 (0.0007) [2023-12-27 06:49:04,104][20347] Updated weights for policy 0, policy_version 167992 (0.0006) [2023-12-27 06:49:04,154][20347] Updated weights for policy 0, policy_version 168002 (0.0006) [2023-12-27 06:49:04,621][20381] Updated weights for policy 1, policy_version 167706 (0.0005) [2023-12-27 06:49:04,676][20381] Updated weights for policy 1, policy_version 167716 (0.0005) [2023-12-27 06:49:04,724][20381] Updated weights for policy 1, policy_version 167726 (0.0005) [2023-12-27 06:49:04,782][20381] Updated weights for policy 1, policy_version 167736 (0.0008) [2023-12-27 06:49:04,797][20347] Updated weights for policy 0, policy_version 168012 (0.0006) [2023-12-27 06:49:04,854][20347] Updated weights for policy 0, policy_version 168022 (0.0005) [2023-12-27 06:49:04,917][20347] Updated weights for policy 0, policy_version 168032 (0.0007) [2023-12-27 06:49:05,405][20381] Updated weights for policy 1, policy_version 167746 (0.0005) [2023-12-27 06:49:05,454][20381] Updated weights for policy 1, policy_version 167756 (0.0005) [2023-12-27 06:49:05,502][20381] Updated weights for policy 1, policy_version 167766 (0.0005) [2023-12-27 06:49:05,594][20347] Updated weights for policy 0, policy_version 168042 (0.0007) [2023-12-27 06:49:05,646][20347] Updated weights for policy 0, policy_version 168052 (0.0007) [2023-12-27 06:49:05,698][20347] Updated weights for policy 0, policy_version 168062 (0.0009) [2023-12-27 06:49:05,753][20347] Updated weights for policy 0, policy_version 168072 (0.0009) [2023-12-27 06:49:06,091][20381] Updated weights for policy 1, policy_version 167776 (0.0008) [2023-12-27 06:49:06,152][20381] Updated weights for policy 1, policy_version 167786 (0.0007) [2023-12-27 06:49:06,208][20381] Updated weights for policy 1, policy_version 167796 (0.0006) [2023-12-27 06:49:06,500][20347] Updated weights for policy 0, policy_version 168082 (0.0008) [2023-12-27 06:49:06,560][20347] Updated weights for policy 0, policy_version 168092 (0.0008) [2023-12-27 06:49:06,611][20347] Updated weights for policy 0, policy_version 168102 (0.0008) [2023-12-27 06:49:06,848][20381] Updated weights for policy 1, policy_version 167806 (0.0009) [2023-12-27 06:49:06,908][20381] Updated weights for policy 1, policy_version 167816 (0.0011) [2023-12-27 06:49:06,975][20381] Updated weights for policy 1, policy_version 167826 (0.0010) [2023-12-27 06:49:07,271][20347] Updated weights for policy 0, policy_version 168112 (0.0007) [2023-12-27 06:49:07,333][20347] Updated weights for policy 0, policy_version 168122 (0.0008) [2023-12-27 06:49:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 86024192. Throughput: 0: 10318.7, 1: 10300.5. Samples: 85995680. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:49:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '33.830')] [2023-12-27 06:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000167832_42975232.pth... [2023-12-27 06:49:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000166616_42663936.pth [2023-12-27 06:49:07,393][20347] Updated weights for policy 0, policy_version 168132 (0.0010) [2023-12-27 06:49:07,414][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000168136_43057152.pth... [2023-12-27 06:49:07,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000166920_42745856.pth [2023-12-27 06:49:07,662][20381] Updated weights for policy 1, policy_version 167836 (0.0008) [2023-12-27 06:49:07,724][20381] Updated weights for policy 1, policy_version 167846 (0.0010) [2023-12-27 06:49:07,781][20381] Updated weights for policy 1, policy_version 167856 (0.0010) [2023-12-27 06:49:08,032][20347] Updated weights for policy 0, policy_version 168142 (0.0010) [2023-12-27 06:49:08,084][20347] Updated weights for policy 0, policy_version 168152 (0.0010) [2023-12-27 06:49:08,141][20347] Updated weights for policy 0, policy_version 168162 (0.0010) [2023-12-27 06:49:08,349][20381] Updated weights for policy 1, policy_version 167866 (0.0008) [2023-12-27 06:49:08,402][20381] Updated weights for policy 1, policy_version 167876 (0.0010) [2023-12-27 06:49:08,454][20381] Updated weights for policy 1, policy_version 167886 (0.0011) [2023-12-27 06:49:08,503][20381] Updated weights for policy 1, policy_version 167896 (0.0010) [2023-12-27 06:49:08,865][20347] Updated weights for policy 0, policy_version 168172 (0.0011) [2023-12-27 06:49:08,921][20347] Updated weights for policy 0, policy_version 168182 (0.0010) [2023-12-27 06:49:08,979][20347] Updated weights for policy 0, policy_version 168192 (0.0010) [2023-12-27 06:49:09,166][20381] Updated weights for policy 1, policy_version 167906 (0.0011) [2023-12-27 06:49:09,225][20381] Updated weights for policy 1, policy_version 167916 (0.0011) [2023-12-27 06:49:09,291][20381] Updated weights for policy 1, policy_version 167926 (0.0009) [2023-12-27 06:49:09,668][20347] Updated weights for policy 0, policy_version 168202 (0.0010) [2023-12-27 06:49:09,728][20347] Updated weights for policy 0, policy_version 168212 (0.0008) [2023-12-27 06:49:09,789][20347] Updated weights for policy 0, policy_version 168222 (0.0007) [2023-12-27 06:49:09,855][20347] Updated weights for policy 0, policy_version 168232 (0.0009) [2023-12-27 06:49:10,027][20381] Updated weights for policy 1, policy_version 167936 (0.0008) [2023-12-27 06:49:10,080][20381] Updated weights for policy 1, policy_version 167946 (0.0008) [2023-12-27 06:49:10,134][20381] Updated weights for policy 1, policy_version 167956 (0.0007) [2023-12-27 06:49:10,587][20347] Updated weights for policy 0, policy_version 168242 (0.0008) [2023-12-27 06:49:10,641][20347] Updated weights for policy 0, policy_version 168252 (0.0007) [2023-12-27 06:49:10,703][20347] Updated weights for policy 0, policy_version 168262 (0.0005) [2023-12-27 06:49:10,783][20381] Updated weights for policy 1, policy_version 167966 (0.0006) [2023-12-27 06:49:10,843][20381] Updated weights for policy 1, policy_version 167976 (0.0007) [2023-12-27 06:49:10,900][20381] Updated weights for policy 1, policy_version 167986 (0.0006) [2023-12-27 06:49:11,344][20347] Updated weights for policy 0, policy_version 168272 (0.0008) [2023-12-27 06:49:11,404][20347] Updated weights for policy 0, policy_version 168282 (0.0007) [2023-12-27 06:49:11,468][20347] Updated weights for policy 0, policy_version 168292 (0.0005) [2023-12-27 06:49:11,591][20381] Updated weights for policy 1, policy_version 167996 (0.0008) [2023-12-27 06:49:11,656][20381] Updated weights for policy 1, policy_version 168006 (0.0008) [2023-12-27 06:49:11,721][20381] Updated weights for policy 1, policy_version 168016 (0.0006) [2023-12-27 06:49:12,105][20347] Updated weights for policy 0, policy_version 168302 (0.0006) [2023-12-27 06:49:12,165][20347] Updated weights for policy 0, policy_version 168312 (0.0007) [2023-12-27 06:49:12,227][20347] Updated weights for policy 0, policy_version 168322 (0.0008) [2023-12-27 06:49:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 86130688. Throughput: 0: 10347.8, 1: 10294.1. Samples: 86119924. Policy #0 lag: (min: 14.0, avg: 16.1, max: 46.0) [2023-12-27 06:49:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '38.020')] [2023-12-27 06:49:12,427][20381] Updated weights for policy 1, policy_version 168026 (0.0007) [2023-12-27 06:49:12,483][20381] Updated weights for policy 1, policy_version 168036 (0.0008) [2023-12-27 06:49:12,542][20381] Updated weights for policy 1, policy_version 168046 (0.0007) [2023-12-27 06:49:12,598][20381] Updated weights for policy 1, policy_version 168056 (0.0007) [2023-12-27 06:49:12,951][20347] Updated weights for policy 0, policy_version 168332 (0.0007) [2023-12-27 06:49:13,010][20347] Updated weights for policy 0, policy_version 168342 (0.0005) [2023-12-27 06:49:13,075][20347] Updated weights for policy 0, policy_version 168352 (0.0005) [2023-12-27 06:49:13,262][20381] Updated weights for policy 1, policy_version 168066 (0.0005) [2023-12-27 06:49:13,320][20381] Updated weights for policy 1, policy_version 168076 (0.0005) [2023-12-27 06:49:13,376][20381] Updated weights for policy 1, policy_version 168086 (0.0005) [2023-12-27 06:49:13,616][20347] Updated weights for policy 0, policy_version 168362 (0.0005) [2023-12-27 06:49:13,667][20347] Updated weights for policy 0, policy_version 168372 (0.0005) [2023-12-27 06:49:13,739][20347] Updated weights for policy 0, policy_version 168382 (0.0005) [2023-12-27 06:49:13,798][20347] Updated weights for policy 0, policy_version 168392 (0.0006) [2023-12-27 06:49:13,887][20381] Updated weights for policy 1, policy_version 168096 (0.0007) [2023-12-27 06:49:13,945][20381] Updated weights for policy 1, policy_version 168106 (0.0008) [2023-12-27 06:49:13,997][20381] Updated weights for policy 1, policy_version 168116 (0.0009) [2023-12-27 06:49:14,335][20347] Updated weights for policy 0, policy_version 168402 (0.0005) [2023-12-27 06:49:14,390][20347] Updated weights for policy 0, policy_version 168412 (0.0005) [2023-12-27 06:49:14,447][20347] Updated weights for policy 0, policy_version 168422 (0.0006) [2023-12-27 06:49:14,719][20381] Updated weights for policy 1, policy_version 168126 (0.0010) [2023-12-27 06:49:14,770][20381] Updated weights for policy 1, policy_version 168136 (0.0011) [2023-12-27 06:49:14,836][20381] Updated weights for policy 1, policy_version 168146 (0.0011) [2023-12-27 06:49:15,016][20347] Updated weights for policy 0, policy_version 168432 (0.0007) [2023-12-27 06:49:15,075][20347] Updated weights for policy 0, policy_version 168442 (0.0007) [2023-12-27 06:49:15,127][20347] Updated weights for policy 0, policy_version 168452 (0.0008) [2023-12-27 06:49:15,553][20381] Updated weights for policy 1, policy_version 168156 (0.0010) [2023-12-27 06:49:15,605][20381] Updated weights for policy 1, policy_version 168166 (0.0011) [2023-12-27 06:49:15,660][20381] Updated weights for policy 1, policy_version 168176 (0.0010) [2023-12-27 06:49:15,786][20347] Updated weights for policy 0, policy_version 168462 (0.0007) [2023-12-27 06:49:15,830][20347] Updated weights for policy 0, policy_version 168472 (0.0007) [2023-12-27 06:49:15,874][20347] Updated weights for policy 0, policy_version 168482 (0.0008) [2023-12-27 06:49:16,364][20381] Updated weights for policy 1, policy_version 168186 (0.0011) [2023-12-27 06:49:16,413][20381] Updated weights for policy 1, policy_version 168196 (0.0010) [2023-12-27 06:49:16,458][20381] Updated weights for policy 1, policy_version 168206 (0.0010) [2023-12-27 06:49:16,510][20381] Updated weights for policy 1, policy_version 168216 (0.0010) [2023-12-27 06:49:16,584][20347] Updated weights for policy 0, policy_version 168492 (0.0006) [2023-12-27 06:49:16,637][20347] Updated weights for policy 0, policy_version 168502 (0.0008) [2023-12-27 06:49:16,684][20347] Updated weights for policy 0, policy_version 168512 (0.0006) [2023-12-27 06:49:17,226][20381] Updated weights for policy 1, policy_version 168226 (0.0010) [2023-12-27 06:49:17,277][20381] Updated weights for policy 1, policy_version 168236 (0.0010) [2023-12-27 06:49:17,295][20347] Updated weights for policy 0, policy_version 168522 (0.0006) [2023-12-27 06:49:17,323][20381] Updated weights for policy 1, policy_version 168246 (0.0010) [2023-12-27 06:49:17,359][20347] Updated weights for policy 0, policy_version 168532 (0.0010) [2023-12-27 06:49:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 86237184. Throughput: 0: 10313.5, 1: 10325.4. Samples: 86246316. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '39.140')] [2023-12-27 06:49:17,417][20347] Updated weights for policy 0, policy_version 168542 (0.0010) [2023-12-27 06:49:17,477][20347] Updated weights for policy 0, policy_version 168552 (0.0010) [2023-12-27 06:49:18,019][20381] Updated weights for policy 1, policy_version 168256 (0.0010) [2023-12-27 06:49:18,077][20381] Updated weights for policy 1, policy_version 168266 (0.0011) [2023-12-27 06:49:18,132][20381] Updated weights for policy 1, policy_version 168276 (0.0010) [2023-12-27 06:49:18,139][20347] Updated weights for policy 0, policy_version 168562 (0.0009) [2023-12-27 06:49:18,199][20347] Updated weights for policy 0, policy_version 168572 (0.0005) [2023-12-27 06:49:18,262][20347] Updated weights for policy 0, policy_version 168582 (0.0006) [2023-12-27 06:49:18,766][20381] Updated weights for policy 1, policy_version 168286 (0.0008) [2023-12-27 06:49:18,830][20381] Updated weights for policy 1, policy_version 168296 (0.0009) [2023-12-27 06:49:18,888][20381] Updated weights for policy 1, policy_version 168306 (0.0007) [2023-12-27 06:49:18,904][20347] Updated weights for policy 0, policy_version 168592 (0.0007) [2023-12-27 06:49:18,963][20347] Updated weights for policy 0, policy_version 168602 (0.0009) [2023-12-27 06:49:19,027][20347] Updated weights for policy 0, policy_version 168612 (0.0010) [2023-12-27 06:49:19,505][20381] Updated weights for policy 1, policy_version 168316 (0.0008) [2023-12-27 06:49:19,560][20381] Updated weights for policy 1, policy_version 168326 (0.0006) [2023-12-27 06:49:19,619][20381] Updated weights for policy 1, policy_version 168336 (0.0007) [2023-12-27 06:49:19,702][20347] Updated weights for policy 0, policy_version 168622 (0.0010) [2023-12-27 06:49:19,758][20347] Updated weights for policy 0, policy_version 168632 (0.0010) [2023-12-27 06:49:19,818][20347] Updated weights for policy 0, policy_version 168642 (0.0010) [2023-12-27 06:49:20,321][20381] Updated weights for policy 1, policy_version 168346 (0.0007) [2023-12-27 06:49:20,381][20381] Updated weights for policy 1, policy_version 168356 (0.0005) [2023-12-27 06:49:20,442][20381] Updated weights for policy 1, policy_version 168366 (0.0007) [2023-12-27 06:49:20,504][20381] Updated weights for policy 1, policy_version 168376 (0.0007) [2023-12-27 06:49:20,505][20347] Updated weights for policy 0, policy_version 168652 (0.0010) [2023-12-27 06:49:20,564][20347] Updated weights for policy 0, policy_version 168662 (0.0010) [2023-12-27 06:49:20,629][20347] Updated weights for policy 0, policy_version 168672 (0.0010) [2023-12-27 06:49:21,210][20381] Updated weights for policy 1, policy_version 168386 (0.0008) [2023-12-27 06:49:21,274][20381] Updated weights for policy 1, policy_version 168396 (0.0007) [2023-12-27 06:49:21,329][20347] Updated weights for policy 0, policy_version 168682 (0.0010) [2023-12-27 06:49:21,342][20381] Updated weights for policy 1, policy_version 168406 (0.0007) [2023-12-27 06:49:21,395][20347] Updated weights for policy 0, policy_version 168692 (0.0008) [2023-12-27 06:49:21,455][20347] Updated weights for policy 0, policy_version 168702 (0.0008) [2023-12-27 06:49:21,517][20347] Updated weights for policy 0, policy_version 168712 (0.0008) [2023-12-27 06:49:22,027][20381] Updated weights for policy 1, policy_version 168416 (0.0009) [2023-12-27 06:49:22,081][20381] Updated weights for policy 1, policy_version 168426 (0.0009) [2023-12-27 06:49:22,130][20381] Updated weights for policy 1, policy_version 168436 (0.0008) [2023-12-27 06:49:22,229][20347] Updated weights for policy 0, policy_version 168722 (0.0008) [2023-12-27 06:49:22,286][20347] Updated weights for policy 0, policy_version 168732 (0.0007) [2023-12-27 06:49:22,344][20347] Updated weights for policy 0, policy_version 168742 (0.0007) [2023-12-27 06:49:22,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 86343680. Throughput: 0: 10315.2, 1: 10326.1. Samples: 86308028. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '36.520')] [2023-12-27 06:49:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000168440_43130880.pth... [2023-12-27 06:49:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000168744_43212800.pth... [2023-12-27 06:49:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000167224_42819584.pth [2023-12-27 06:49:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000167528_42901504.pth [2023-12-27 06:49:22,869][20381] Updated weights for policy 1, policy_version 168446 (0.0008) [2023-12-27 06:49:22,931][20381] Updated weights for policy 1, policy_version 168456 (0.0008) [2023-12-27 06:49:22,981][20347] Updated weights for policy 0, policy_version 168752 (0.0008) [2023-12-27 06:49:22,989][20381] Updated weights for policy 1, policy_version 168466 (0.0007) [2023-12-27 06:49:23,032][20347] Updated weights for policy 0, policy_version 168762 (0.0008) [2023-12-27 06:49:23,078][20347] Updated weights for policy 0, policy_version 168772 (0.0009) [2023-12-27 06:49:23,568][20381] Updated weights for policy 1, policy_version 168476 (0.0007) [2023-12-27 06:49:23,627][20381] Updated weights for policy 1, policy_version 168486 (0.0011) [2023-12-27 06:49:23,688][20381] Updated weights for policy 1, policy_version 168496 (0.0010) [2023-12-27 06:49:23,844][20347] Updated weights for policy 0, policy_version 168782 (0.0008) [2023-12-27 06:49:23,899][20347] Updated weights for policy 0, policy_version 168792 (0.0007) [2023-12-27 06:49:23,952][20347] Updated weights for policy 0, policy_version 168802 (0.0007) [2023-12-27 06:49:24,366][20381] Updated weights for policy 1, policy_version 168506 (0.0007) [2023-12-27 06:49:24,420][20381] Updated weights for policy 1, policy_version 168516 (0.0011) [2023-12-27 06:49:24,475][20381] Updated weights for policy 1, policy_version 168526 (0.0008) [2023-12-27 06:49:24,530][20381] Updated weights for policy 1, policy_version 168536 (0.0007) [2023-12-27 06:49:24,637][20347] Updated weights for policy 0, policy_version 168812 (0.0007) [2023-12-27 06:49:24,686][20347] Updated weights for policy 0, policy_version 168822 (0.0007) [2023-12-27 06:49:24,732][20347] Updated weights for policy 0, policy_version 168832 (0.0006) [2023-12-27 06:49:25,147][20381] Updated weights for policy 1, policy_version 168546 (0.0005) [2023-12-27 06:49:25,203][20381] Updated weights for policy 1, policy_version 168556 (0.0005) [2023-12-27 06:49:25,264][20381] Updated weights for policy 1, policy_version 168566 (0.0005) [2023-12-27 06:49:25,410][20347] Updated weights for policy 0, policy_version 168842 (0.0006) [2023-12-27 06:49:25,474][20347] Updated weights for policy 0, policy_version 168852 (0.0005) [2023-12-27 06:49:25,531][20347] Updated weights for policy 0, policy_version 168862 (0.0007) [2023-12-27 06:49:25,586][20347] Updated weights for policy 0, policy_version 168872 (0.0005) [2023-12-27 06:49:25,855][20381] Updated weights for policy 1, policy_version 168576 (0.0010) [2023-12-27 06:49:25,918][20381] Updated weights for policy 1, policy_version 168586 (0.0007) [2023-12-27 06:49:25,972][20381] Updated weights for policy 1, policy_version 168596 (0.0009) [2023-12-27 06:49:26,120][20347] Updated weights for policy 0, policy_version 168882 (0.0007) [2023-12-27 06:49:26,165][20347] Updated weights for policy 0, policy_version 168892 (0.0007) [2023-12-27 06:49:26,213][20347] Updated weights for policy 0, policy_version 168902 (0.0007) [2023-12-27 06:49:26,637][20381] Updated weights for policy 1, policy_version 168606 (0.0010) [2023-12-27 06:49:26,692][20381] Updated weights for policy 1, policy_version 168616 (0.0010) [2023-12-27 06:49:26,743][20381] Updated weights for policy 1, policy_version 168626 (0.0011) [2023-12-27 06:49:26,849][20347] Updated weights for policy 0, policy_version 168912 (0.0007) [2023-12-27 06:49:26,897][20347] Updated weights for policy 0, policy_version 168922 (0.0007) [2023-12-27 06:49:26,946][20347] Updated weights for policy 0, policy_version 168932 (0.0007) [2023-12-27 06:49:27,314][20381] Updated weights for policy 1, policy_version 168636 (0.0008) [2023-12-27 06:49:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 86441984. Throughput: 0: 10365.6, 1: 10408.8. Samples: 86432076. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '49.280')] [2023-12-27 06:49:27,393][20381] Updated weights for policy 1, policy_version 168646 (0.0010) [2023-12-27 06:49:27,445][20381] Updated weights for policy 1, policy_version 168656 (0.0011) [2023-12-27 06:49:27,682][20347] Updated weights for policy 0, policy_version 168942 (0.0007) [2023-12-27 06:49:27,741][20347] Updated weights for policy 0, policy_version 168952 (0.0007) [2023-12-27 06:49:27,797][20347] Updated weights for policy 0, policy_version 168962 (0.0006) [2023-12-27 06:49:28,085][20381] Updated weights for policy 1, policy_version 168666 (0.0010) [2023-12-27 06:49:28,149][20381] Updated weights for policy 1, policy_version 168676 (0.0007) [2023-12-27 06:49:28,212][20381] Updated weights for policy 1, policy_version 168686 (0.0006) [2023-12-27 06:49:28,277][20381] Updated weights for policy 1, policy_version 168696 (0.0007) [2023-12-27 06:49:28,450][20347] Updated weights for policy 0, policy_version 168972 (0.0006) [2023-12-27 06:49:28,505][20347] Updated weights for policy 0, policy_version 168982 (0.0008) [2023-12-27 06:49:28,553][20347] Updated weights for policy 0, policy_version 168992 (0.0007) [2023-12-27 06:49:28,952][20381] Updated weights for policy 1, policy_version 168706 (0.0007) [2023-12-27 06:49:29,014][20381] Updated weights for policy 1, policy_version 168716 (0.0007) [2023-12-27 06:49:29,076][20381] Updated weights for policy 1, policy_version 168726 (0.0007) [2023-12-27 06:49:29,217][20347] Updated weights for policy 0, policy_version 169002 (0.0008) [2023-12-27 06:49:29,276][20347] Updated weights for policy 0, policy_version 169012 (0.0007) [2023-12-27 06:49:29,329][20347] Updated weights for policy 0, policy_version 169022 (0.0006) [2023-12-27 06:49:29,390][20347] Updated weights for policy 0, policy_version 169032 (0.0007) [2023-12-27 06:49:29,739][20381] Updated weights for policy 1, policy_version 168736 (0.0008) [2023-12-27 06:49:29,784][20381] Updated weights for policy 1, policy_version 168746 (0.0007) [2023-12-27 06:49:29,833][20381] Updated weights for policy 1, policy_version 168756 (0.0008) [2023-12-27 06:49:30,101][20347] Updated weights for policy 0, policy_version 169042 (0.0007) [2023-12-27 06:49:30,151][20347] Updated weights for policy 0, policy_version 169052 (0.0007) [2023-12-27 06:49:30,201][20347] Updated weights for policy 0, policy_version 169062 (0.0007) [2023-12-27 06:49:30,530][20381] Updated weights for policy 1, policy_version 168766 (0.0007) [2023-12-27 06:49:30,578][20381] Updated weights for policy 1, policy_version 168776 (0.0005) [2023-12-27 06:49:30,623][20381] Updated weights for policy 1, policy_version 168786 (0.0005) [2023-12-27 06:49:30,875][20347] Updated weights for policy 0, policy_version 169072 (0.0005) [2023-12-27 06:49:30,937][20347] Updated weights for policy 0, policy_version 169082 (0.0005) [2023-12-27 06:49:30,998][20347] Updated weights for policy 0, policy_version 169092 (0.0007) [2023-12-27 06:49:31,267][20381] Updated weights for policy 1, policy_version 168796 (0.0006) [2023-12-27 06:49:31,327][20381] Updated weights for policy 1, policy_version 168806 (0.0008) [2023-12-27 06:49:31,385][20381] Updated weights for policy 1, policy_version 168816 (0.0008) [2023-12-27 06:49:31,624][20347] Updated weights for policy 0, policy_version 169102 (0.0007) [2023-12-27 06:49:31,680][20347] Updated weights for policy 0, policy_version 169112 (0.0006) [2023-12-27 06:49:31,744][20347] Updated weights for policy 0, policy_version 169122 (0.0007) [2023-12-27 06:49:32,082][20381] Updated weights for policy 1, policy_version 168826 (0.0008) [2023-12-27 06:49:32,138][20381] Updated weights for policy 1, policy_version 168836 (0.0008) [2023-12-27 06:49:32,199][20381] Updated weights for policy 1, policy_version 168846 (0.0005) [2023-12-27 06:49:32,257][20381] Updated weights for policy 1, policy_version 168856 (0.0006) [2023-12-27 06:49:32,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 86548480. Throughput: 0: 10359.7, 1: 10409.8. Samples: 86557108. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '63.290')] [2023-12-27 06:49:32,454][20347] Updated weights for policy 0, policy_version 169132 (0.0007) [2023-12-27 06:49:32,511][20347] Updated weights for policy 0, policy_version 169142 (0.0008) [2023-12-27 06:49:32,564][20347] Updated weights for policy 0, policy_version 169152 (0.0005) [2023-12-27 06:49:32,932][20381] Updated weights for policy 1, policy_version 168866 (0.0008) [2023-12-27 06:49:32,979][20381] Updated weights for policy 1, policy_version 168876 (0.0007) [2023-12-27 06:49:33,030][20381] Updated weights for policy 1, policy_version 168886 (0.0007) [2023-12-27 06:49:33,237][20347] Updated weights for policy 0, policy_version 169162 (0.0009) [2023-12-27 06:49:33,290][20347] Updated weights for policy 0, policy_version 169172 (0.0010) [2023-12-27 06:49:33,345][20347] Updated weights for policy 0, policy_version 169182 (0.0010) [2023-12-27 06:49:33,396][20347] Updated weights for policy 0, policy_version 169192 (0.0010) [2023-12-27 06:49:33,726][20381] Updated weights for policy 1, policy_version 168896 (0.0008) [2023-12-27 06:49:33,775][20381] Updated weights for policy 1, policy_version 168906 (0.0010) [2023-12-27 06:49:33,826][20381] Updated weights for policy 1, policy_version 168916 (0.0008) [2023-12-27 06:49:33,946][20347] Updated weights for policy 0, policy_version 169202 (0.0006) [2023-12-27 06:49:34,004][20347] Updated weights for policy 0, policy_version 169212 (0.0005) [2023-12-27 06:49:34,060][20347] Updated weights for policy 0, policy_version 169222 (0.0008) [2023-12-27 06:49:34,596][20381] Updated weights for policy 1, policy_version 168926 (0.0008) [2023-12-27 06:49:34,647][20381] Updated weights for policy 1, policy_version 168936 (0.0006) [2023-12-27 06:49:34,668][20347] Updated weights for policy 0, policy_version 169232 (0.0010) [2023-12-27 06:49:34,714][20381] Updated weights for policy 1, policy_version 168946 (0.0008) [2023-12-27 06:49:34,719][20347] Updated weights for policy 0, policy_version 169242 (0.0008) [2023-12-27 06:49:34,776][20347] Updated weights for policy 0, policy_version 169252 (0.0006) [2023-12-27 06:49:35,359][20381] Updated weights for policy 1, policy_version 168956 (0.0008) [2023-12-27 06:49:35,383][20347] Updated weights for policy 0, policy_version 169262 (0.0007) [2023-12-27 06:49:35,422][20381] Updated weights for policy 1, policy_version 168966 (0.0007) [2023-12-27 06:49:35,432][20347] Updated weights for policy 0, policy_version 169272 (0.0007) [2023-12-27 06:49:35,479][20381] Updated weights for policy 1, policy_version 168976 (0.0006) [2023-12-27 06:49:35,484][20347] Updated weights for policy 0, policy_version 169282 (0.0010) [2023-12-27 06:49:36,049][20381] Updated weights for policy 1, policy_version 168986 (0.0007) [2023-12-27 06:49:36,112][20347] Updated weights for policy 0, policy_version 169292 (0.0009) [2023-12-27 06:49:36,112][20381] Updated weights for policy 1, policy_version 168996 (0.0008) [2023-12-27 06:49:36,169][20347] Updated weights for policy 0, policy_version 169302 (0.0007) [2023-12-27 06:49:36,175][20381] Updated weights for policy 1, policy_version 169006 (0.0008) [2023-12-27 06:49:36,226][20347] Updated weights for policy 0, policy_version 169312 (0.0007) [2023-12-27 06:49:36,236][20381] Updated weights for policy 1, policy_version 169016 (0.0006) [2023-12-27 06:49:36,847][20381] Updated weights for policy 1, policy_version 169026 (0.0006) [2023-12-27 06:49:36,904][20381] Updated weights for policy 1, policy_version 169036 (0.0008) [2023-12-27 06:49:36,959][20381] Updated weights for policy 1, policy_version 169046 (0.0006) [2023-12-27 06:49:36,992][20347] Updated weights for policy 0, policy_version 169322 (0.0008) [2023-12-27 06:49:37,043][20347] Updated weights for policy 0, policy_version 169332 (0.0007) [2023-12-27 06:49:37,095][20347] Updated weights for policy 0, policy_version 169342 (0.0008) [2023-12-27 06:49:37,152][20347] Updated weights for policy 0, policy_version 169352 (0.0007) [2023-12-27 06:49:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 86654976. Throughput: 0: 10361.4, 1: 10378.3. Samples: 86619940. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:37,363][19170] Avg episode reward: [(0, '999.070'), (1, '117.860')] [2023-12-27 06:49:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000169048_43286528.pth... [2023-12-27 06:49:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000169352_43368448.pth... [2023-12-27 06:49:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000168136_43057152.pth [2023-12-27 06:49:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000167832_42975232.pth [2023-12-27 06:49:37,639][20381] Updated weights for policy 1, policy_version 169056 (0.0010) [2023-12-27 06:49:37,702][20381] Updated weights for policy 1, policy_version 169066 (0.0011) [2023-12-27 06:49:37,762][20381] Updated weights for policy 1, policy_version 169076 (0.0011) [2023-12-27 06:49:37,793][20291] KL-divergence is very high: 119.4183 [2023-12-27 06:49:37,805][20291] KL-divergence is very high: 1887.0015 [2023-12-27 06:49:37,816][20347] Updated weights for policy 0, policy_version 169362 (0.0006) [2023-12-27 06:49:37,837][20291] KL-divergence is very high: 209.1704 [2023-12-27 06:49:37,846][20291] KL-divergence is very high: 1752.2788 [2023-12-27 06:49:37,867][20347] Updated weights for policy 0, policy_version 169372 (0.0005) [2023-12-27 06:49:37,923][20347] Updated weights for policy 0, policy_version 169382 (0.0005) [2023-12-27 06:49:38,455][20381] Updated weights for policy 1, policy_version 169086 (0.0010) [2023-12-27 06:49:38,477][20291] KL-divergence is very high: 2214.3826 [2023-12-27 06:49:38,489][20291] KL-divergence is very high: 760.6299 [2023-12-27 06:49:38,508][20381] Updated weights for policy 1, policy_version 169096 (0.0011) [2023-12-27 06:49:38,517][20347] Updated weights for policy 0, policy_version 169392 (0.0006) [2023-12-27 06:49:38,526][20291] KL-divergence is very high: 1788.4009 [2023-12-27 06:49:38,539][20291] KL-divergence is very high: 774.4169 [2023-12-27 06:49:38,559][20381] Updated weights for policy 1, policy_version 169106 (0.0010) [2023-12-27 06:49:38,578][20291] KL-divergence is very high: 1164.9264 [2023-12-27 06:49:38,582][20347] Updated weights for policy 0, policy_version 169402 (0.0008) [2023-12-27 06:49:38,589][20291] KL-divergence is very high: 1135.8728 [2023-12-27 06:49:38,622][20291] KL-divergence is very high: 1125.4736 [2023-12-27 06:49:38,635][20291] KL-divergence is very high: 1083.7430 [2023-12-27 06:49:38,639][20347] Updated weights for policy 0, policy_version 169412 (0.0007) [2023-12-27 06:49:39,206][20381] Updated weights for policy 1, policy_version 169116 (0.0011) [2023-12-27 06:49:39,269][20381] Updated weights for policy 1, policy_version 169126 (0.0011) [2023-12-27 06:49:39,299][20347] Updated weights for policy 0, policy_version 169422 (0.0007) [2023-12-27 06:49:39,323][20291] KL-divergence is very high: 140.8766 [2023-12-27 06:49:39,329][20381] Updated weights for policy 1, policy_version 169136 (0.0006) [2023-12-27 06:49:39,336][20291] KL-divergence is very high: 3724.2600 [2023-12-27 06:49:39,343][20291] KL-divergence is very high: 2503.5542 [2023-12-27 06:49:39,350][20291] KL-divergence is very high: 472.4608 [2023-12-27 06:49:39,363][20347] Updated weights for policy 0, policy_version 169432 (0.0007) [2023-12-27 06:49:39,379][20291] KL-divergence is very high: 247.0039 [2023-12-27 06:49:39,393][20291] KL-divergence is very high: 3525.8792 [2023-12-27 06:49:39,404][20291] KL-divergence is very high: 2857.2849 [2023-12-27 06:49:39,411][20291] KL-divergence is very high: 2106.3455 [2023-12-27 06:49:39,419][20291] KL-divergence is very high: 458.6796 [2023-12-27 06:49:39,437][20347] Updated weights for policy 0, policy_version 169442 (0.0009) [2023-12-27 06:49:39,438][20291] KL-divergence is very high: 128.3467 [2023-12-27 06:49:39,453][20291] KL-divergence is very high: 1104.1387 [2023-12-27 06:49:39,459][20291] KL-divergence is very high: 1239.1444 [2023-12-27 06:49:39,466][20291] KL-divergence is very high: 664.6917 [2023-12-27 06:49:39,473][20291] KL-divergence is very high: 699.9531 [2023-12-27 06:49:39,983][20381] Updated weights for policy 1, policy_version 169146 (0.0009) [2023-12-27 06:49:40,045][20381] Updated weights for policy 1, policy_version 169156 (0.0006) [2023-12-27 06:49:40,080][20291] KL-divergence is very high: 4569.2847 [2023-12-27 06:49:40,086][20291] KL-divergence is very high: 223.3451 [2023-12-27 06:49:40,093][20291] KL-divergence is very high: 3947.6724 [2023-12-27 06:49:40,104][20291] KL-divergence is very high: 2044.6060 [2023-12-27 06:49:40,106][20347] Updated weights for policy 0, policy_version 169452 (0.0007) [2023-12-27 06:49:40,108][20381] Updated weights for policy 1, policy_version 169166 (0.0007) [2023-12-27 06:49:40,112][20291] KL-divergence is very high: 1225.6079 [2023-12-27 06:49:40,119][20291] KL-divergence is very high: 587.1237 [2023-12-27 06:49:40,131][20291] KL-divergence is very high: 2688.6973 [2023-12-27 06:49:40,138][20291] KL-divergence is very high: 356.4652 [2023-12-27 06:49:40,145][20291] KL-divergence is very high: 2436.9185 [2023-12-27 06:49:40,159][20291] KL-divergence is very high: 795.0335 [2023-12-27 06:49:40,171][20347] Updated weights for policy 0, policy_version 169462 (0.0007) [2023-12-27 06:49:40,172][20381] Updated weights for policy 1, policy_version 169176 (0.0007) [2023-12-27 06:49:40,172][20291] KL-divergence is very high: 257.7556 [2023-12-27 06:49:40,186][20291] KL-divergence is very high: 2550.4392 [2023-12-27 06:49:40,194][20291] KL-divergence is very high: 532.4760 [2023-12-27 06:49:40,204][20291] KL-divergence is very high: 4672.0601 [2023-12-27 06:49:40,217][20291] KL-divergence is very high: 296.4545 [2023-12-27 06:49:40,232][20291] KL-divergence is very high: 133.0592 [2023-12-27 06:49:40,244][20347] Updated weights for policy 0, policy_version 169472 (0.0007) [2023-12-27 06:49:40,246][20291] KL-divergence is very high: 1713.9895 [2023-12-27 06:49:40,259][20291] KL-divergence is very high: 2552.7778 [2023-12-27 06:49:40,272][20291] KL-divergence is very high: 714.8278 [2023-12-27 06:49:40,278][20291] KL-divergence is very high: 275.0301 [2023-12-27 06:49:40,863][20381] Updated weights for policy 1, policy_version 169186 (0.0007) [2023-12-27 06:49:40,908][20347] Updated weights for policy 0, policy_version 169482 (0.0007) [2023-12-27 06:49:40,923][20381] Updated weights for policy 1, policy_version 169196 (0.0008) [2023-12-27 06:49:40,968][20347] Updated weights for policy 0, policy_version 169492 (0.0006) [2023-12-27 06:49:40,980][20381] Updated weights for policy 1, policy_version 169206 (0.0008) [2023-12-27 06:49:41,031][20347] Updated weights for policy 0, policy_version 169502 (0.0006) [2023-12-27 06:49:41,091][20347] Updated weights for policy 0, policy_version 169512 (0.0006) [2023-12-27 06:49:41,690][20347] Updated weights for policy 0, policy_version 169522 (0.0005) [2023-12-27 06:49:41,754][20347] Updated weights for policy 0, policy_version 169532 (0.0008) [2023-12-27 06:49:41,767][20381] Updated weights for policy 1, policy_version 169216 (0.0007) [2023-12-27 06:49:41,802][20347] Updated weights for policy 0, policy_version 169542 (0.0006) [2023-12-27 06:49:41,831][20381] Updated weights for policy 1, policy_version 169226 (0.0008) [2023-12-27 06:49:41,897][20381] Updated weights for policy 1, policy_version 169236 (0.0009) [2023-12-27 06:49:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 86753280. Throughput: 0: 10393.4, 1: 10380.1. Samples: 86745424. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:42,363][19170] Avg episode reward: [(0, '948.250'), (1, '219.620')] [2023-12-27 06:49:42,412][20347] Updated weights for policy 0, policy_version 169552 (0.0007) [2023-12-27 06:49:42,477][20347] Updated weights for policy 0, policy_version 169562 (0.0009) [2023-12-27 06:49:42,538][20347] Updated weights for policy 0, policy_version 169572 (0.0008) [2023-12-27 06:49:42,590][20381] Updated weights for policy 1, policy_version 169246 (0.0007) [2023-12-27 06:49:42,647][20381] Updated weights for policy 1, policy_version 169256 (0.0006) [2023-12-27 06:49:42,697][20381] Updated weights for policy 1, policy_version 169266 (0.0006) [2023-12-27 06:49:43,240][20347] Updated weights for policy 0, policy_version 169582 (0.0006) [2023-12-27 06:49:43,294][20347] Updated weights for policy 0, policy_version 169592 (0.0005) [2023-12-27 06:49:43,338][20381] Updated weights for policy 1, policy_version 169276 (0.0007) [2023-12-27 06:49:43,348][20347] Updated weights for policy 0, policy_version 169602 (0.0006) [2023-12-27 06:49:43,396][20381] Updated weights for policy 1, policy_version 169286 (0.0007) [2023-12-27 06:49:43,457][20381] Updated weights for policy 1, policy_version 169296 (0.0007) [2023-12-27 06:49:43,909][20347] Updated weights for policy 0, policy_version 169612 (0.0007) [2023-12-27 06:49:43,961][20347] Updated weights for policy 0, policy_version 169622 (0.0005) [2023-12-27 06:49:44,012][20347] Updated weights for policy 0, policy_version 169632 (0.0005) [2023-12-27 06:49:44,072][20381] Updated weights for policy 1, policy_version 169306 (0.0008) [2023-12-27 06:49:44,127][20381] Updated weights for policy 1, policy_version 169316 (0.0009) [2023-12-27 06:49:44,180][20381] Updated weights for policy 1, policy_version 169326 (0.0007) [2023-12-27 06:49:44,241][20381] Updated weights for policy 1, policy_version 169336 (0.0007) [2023-12-27 06:49:44,633][20347] Updated weights for policy 0, policy_version 169642 (0.0006) [2023-12-27 06:49:44,692][20347] Updated weights for policy 0, policy_version 169652 (0.0007) [2023-12-27 06:49:44,753][20347] Updated weights for policy 0, policy_version 169662 (0.0007) [2023-12-27 06:49:44,810][20347] Updated weights for policy 0, policy_version 169672 (0.0008) [2023-12-27 06:49:44,912][20381] Updated weights for policy 1, policy_version 169346 (0.0008) [2023-12-27 06:49:44,973][20381] Updated weights for policy 1, policy_version 169356 (0.0008) [2023-12-27 06:49:45,025][20381] Updated weights for policy 1, policy_version 169366 (0.0007) [2023-12-27 06:49:45,446][20347] Updated weights for policy 0, policy_version 169682 (0.0005) [2023-12-27 06:49:45,504][20347] Updated weights for policy 0, policy_version 169693 (0.0009) [2023-12-27 06:49:45,811][20381] Updated weights for policy 1, policy_version 169376 (0.0008) [2023-12-27 06:49:45,874][20381] Updated weights for policy 1, policy_version 169386 (0.0009) [2023-12-27 06:49:45,940][20381] Updated weights for policy 1, policy_version 169396 (0.0008) [2023-12-27 06:49:46,259][20347] Updated weights for policy 0, policy_version 169705 (0.0008) [2023-12-27 06:49:46,311][20347] Updated weights for policy 0, policy_version 169715 (0.0009) [2023-12-27 06:49:46,359][20347] Updated weights for policy 0, policy_version 169725 (0.0011) [2023-12-27 06:49:46,408][20347] Updated weights for policy 0, policy_version 169735 (0.0010) [2023-12-27 06:49:46,533][20381] Updated weights for policy 1, policy_version 169406 (0.0008) [2023-12-27 06:49:46,592][20381] Updated weights for policy 1, policy_version 169416 (0.0005) [2023-12-27 06:49:46,650][20381] Updated weights for policy 1, policy_version 169426 (0.0007) [2023-12-27 06:49:47,042][20347] Updated weights for policy 0, policy_version 169745 (0.0010) [2023-12-27 06:49:47,094][20347] Updated weights for policy 0, policy_version 169755 (0.0009) [2023-12-27 06:49:47,141][20347] Updated weights for policy 0, policy_version 169765 (0.0009) [2023-12-27 06:49:47,314][20381] Updated weights for policy 1, policy_version 169436 (0.0008) [2023-12-27 06:49:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 86859776. Throughput: 0: 10464.1, 1: 10401.8. Samples: 86871504. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:47,364][20381] Updated weights for policy 1, policy_version 169446 (0.0008) [2023-12-27 06:49:47,364][19170] Avg episode reward: [(0, '948.250'), (1, '123.100')] [2023-12-27 06:49:47,417][20381] Updated weights for policy 1, policy_version 169456 (0.0007) [2023-12-27 06:49:47,854][20347] Updated weights for policy 0, policy_version 169775 (0.0010) [2023-12-27 06:49:47,908][20347] Updated weights for policy 0, policy_version 169785 (0.0010) [2023-12-27 06:49:47,964][20347] Updated weights for policy 0, policy_version 169795 (0.0011) [2023-12-27 06:49:48,005][20381] Updated weights for policy 1, policy_version 169466 (0.0007) [2023-12-27 06:49:48,066][20381] Updated weights for policy 1, policy_version 169476 (0.0007) [2023-12-27 06:49:48,127][20381] Updated weights for policy 1, policy_version 169486 (0.0008) [2023-12-27 06:49:48,189][20381] Updated weights for policy 1, policy_version 169496 (0.0009) [2023-12-27 06:49:48,612][20347] Updated weights for policy 0, policy_version 169805 (0.0011) [2023-12-27 06:49:48,673][20347] Updated weights for policy 0, policy_version 169815 (0.0006) [2023-12-27 06:49:48,729][20347] Updated weights for policy 0, policy_version 169825 (0.0008) [2023-12-27 06:49:48,810][20381] Updated weights for policy 1, policy_version 169506 (0.0011) [2023-12-27 06:49:48,865][20381] Updated weights for policy 1, policy_version 169516 (0.0008) [2023-12-27 06:49:48,926][20381] Updated weights for policy 1, policy_version 169526 (0.0007) [2023-12-27 06:49:49,324][20347] Updated weights for policy 0, policy_version 169835 (0.0007) [2023-12-27 06:49:49,386][20347] Updated weights for policy 0, policy_version 169845 (0.0010) [2023-12-27 06:49:49,435][20347] Updated weights for policy 0, policy_version 169855 (0.0010) [2023-12-27 06:49:49,622][20381] Updated weights for policy 1, policy_version 169536 (0.0011) [2023-12-27 06:49:49,680][20381] Updated weights for policy 1, policy_version 169546 (0.0011) [2023-12-27 06:49:49,739][20381] Updated weights for policy 1, policy_version 169556 (0.0011) [2023-12-27 06:49:50,116][20347] Updated weights for policy 0, policy_version 169865 (0.0009) [2023-12-27 06:49:50,178][20347] Updated weights for policy 0, policy_version 169875 (0.0010) [2023-12-27 06:49:50,231][20347] Updated weights for policy 0, policy_version 169885 (0.0009) [2023-12-27 06:49:50,286][20347] Updated weights for policy 0, policy_version 169895 (0.0009) [2023-12-27 06:49:50,412][20381] Updated weights for policy 1, policy_version 169566 (0.0006) [2023-12-27 06:49:50,462][20381] Updated weights for policy 1, policy_version 169576 (0.0005) [2023-12-27 06:49:50,515][20381] Updated weights for policy 1, policy_version 169586 (0.0006) [2023-12-27 06:49:50,961][20347] Updated weights for policy 0, policy_version 169905 (0.0010) [2023-12-27 06:49:51,016][20347] Updated weights for policy 0, policy_version 169915 (0.0010) [2023-12-27 06:49:51,085][20347] Updated weights for policy 0, policy_version 169925 (0.0011) [2023-12-27 06:49:51,127][20381] Updated weights for policy 1, policy_version 169596 (0.0007) [2023-12-27 06:49:51,191][20381] Updated weights for policy 1, policy_version 169606 (0.0007) [2023-12-27 06:49:51,258][20381] Updated weights for policy 1, policy_version 169616 (0.0006) [2023-12-27 06:49:51,788][20347] Updated weights for policy 0, policy_version 169935 (0.0008) [2023-12-27 06:49:51,850][20347] Updated weights for policy 0, policy_version 169945 (0.0009) [2023-12-27 06:49:51,906][20347] Updated weights for policy 0, policy_version 169955 (0.0011) [2023-12-27 06:49:51,925][20381] Updated weights for policy 1, policy_version 169626 (0.0006) [2023-12-27 06:49:51,981][20381] Updated weights for policy 1, policy_version 169636 (0.0007) [2023-12-27 06:49:52,037][20381] Updated weights for policy 1, policy_version 169646 (0.0007) [2023-12-27 06:49:52,093][20381] Updated weights for policy 1, policy_version 169656 (0.0008) [2023-12-27 06:49:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 86966272. Throughput: 0: 10496.2, 1: 10373.6. Samples: 86934820. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:52,363][19170] Avg episode reward: [(0, '972.130'), (1, '144.990')] [2023-12-27 06:49:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000169656_43442176.pth... [2023-12-27 06:49:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000169960_43524096.pth... [2023-12-27 06:49:52,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000168440_43130880.pth [2023-12-27 06:49:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000168744_43212800.pth [2023-12-27 06:49:52,538][20347] Updated weights for policy 0, policy_version 169965 (0.0009) [2023-12-27 06:49:52,593][20347] Updated weights for policy 0, policy_version 169975 (0.0010) [2023-12-27 06:49:52,651][20347] Updated weights for policy 0, policy_version 169985 (0.0009) [2023-12-27 06:49:52,819][20381] Updated weights for policy 1, policy_version 169666 (0.0007) [2023-12-27 06:49:52,876][20381] Updated weights for policy 1, policy_version 169676 (0.0007) [2023-12-27 06:49:52,926][20381] Updated weights for policy 1, policy_version 169686 (0.0007) [2023-12-27 06:49:53,318][20347] Updated weights for policy 0, policy_version 169995 (0.0010) [2023-12-27 06:49:53,367][20347] Updated weights for policy 0, policy_version 170005 (0.0006) [2023-12-27 06:49:53,388][20291] KL-divergence is very high: 134.2937 [2023-12-27 06:49:53,394][20291] KL-divergence is very high: 116.3803 [2023-12-27 06:49:53,423][20347] Updated weights for policy 0, policy_version 170015 (0.0007) [2023-12-27 06:49:53,440][20291] KL-divergence is very high: 143.1786 [2023-12-27 06:49:53,464][20291] KL-divergence is very high: 177.6060 [2023-12-27 06:49:53,525][20381] Updated weights for policy 1, policy_version 169696 (0.0007) [2023-12-27 06:49:53,570][20381] Updated weights for policy 1, policy_version 169706 (0.0008) [2023-12-27 06:49:53,618][20381] Updated weights for policy 1, policy_version 169716 (0.0006) [2023-12-27 06:49:53,999][20347] Updated weights for policy 0, policy_version 170025 (0.0007) [2023-12-27 06:49:54,001][20291] KL-divergence is very high: 197.9284 [2023-12-27 06:49:54,007][20291] KL-divergence is very high: 178.6240 [2023-12-27 06:49:54,014][20291] KL-divergence is very high: 198.0861 [2023-12-27 06:49:54,057][20291] KL-divergence is very high: 102.1682 [2023-12-27 06:49:54,059][20347] Updated weights for policy 0, policy_version 170035 (0.0006) [2023-12-27 06:49:54,071][20291] KL-divergence is very high: 185.1760 [2023-12-27 06:49:54,081][20291] KL-divergence is very high: 150.4722 [2023-12-27 06:49:54,086][20291] KL-divergence is very high: 108.0654 [2023-12-27 06:49:54,098][20291] KL-divergence is very high: 196.1678 [2023-12-27 06:49:54,111][20347] Updated weights for policy 0, policy_version 170045 (0.0010) [2023-12-27 06:49:54,116][20291] KL-divergence is very high: 174.1198 [2023-12-27 06:49:54,148][20291] KL-divergence is very high: 105.0208 [2023-12-27 06:49:54,155][20291] KL-divergence is very high: 105.1332 [2023-12-27 06:49:54,162][20291] KL-divergence is very high: 136.3359 [2023-12-27 06:49:54,168][20291] KL-divergence is very high: 765.5343 [2023-12-27 06:49:54,175][20347] Updated weights for policy 0, policy_version 170055 (0.0010) [2023-12-27 06:49:54,370][20381] Updated weights for policy 1, policy_version 169726 (0.0007) [2023-12-27 06:49:54,424][20381] Updated weights for policy 1, policy_version 169736 (0.0008) [2023-12-27 06:49:54,484][20381] Updated weights for policy 1, policy_version 169746 (0.0007) [2023-12-27 06:49:54,755][20291] KL-divergence is very high: 336.6597 [2023-12-27 06:49:54,774][20291] KL-divergence is very high: 549.0977 [2023-12-27 06:49:54,811][20347] Updated weights for policy 0, policy_version 170065 (0.0010) [2023-12-27 06:49:54,821][20291] KL-divergence is very high: 266.0120 [2023-12-27 06:49:54,857][20291] KL-divergence is very high: 123.7140 [2023-12-27 06:49:54,864][20291] KL-divergence is very high: 119.6460 [2023-12-27 06:49:54,870][20291] KL-divergence is very high: 171.1598 [2023-12-27 06:49:54,872][20347] Updated weights for policy 0, policy_version 170075 (0.0009) [2023-12-27 06:49:54,886][20291] KL-divergence is very high: 114.9926 [2023-12-27 06:49:54,900][20291] KL-divergence is very high: 106.6796 [2023-12-27 06:49:54,910][20291] KL-divergence is very high: 146.4983 [2023-12-27 06:49:54,922][20347] Updated weights for policy 0, policy_version 170085 (0.0009) [2023-12-27 06:49:55,132][20381] Updated weights for policy 1, policy_version 169756 (0.0007) [2023-12-27 06:49:55,199][20381] Updated weights for policy 1, policy_version 169766 (0.0005) [2023-12-27 06:49:55,263][20381] Updated weights for policy 1, policy_version 169776 (0.0005) [2023-12-27 06:49:55,633][20347] Updated weights for policy 0, policy_version 170095 (0.0009) [2023-12-27 06:49:55,691][20347] Updated weights for policy 0, policy_version 170105 (0.0010) [2023-12-27 06:49:55,746][20347] Updated weights for policy 0, policy_version 170115 (0.0011) [2023-12-27 06:49:55,873][20381] Updated weights for policy 1, policy_version 169786 (0.0007) [2023-12-27 06:49:55,933][20381] Updated weights for policy 1, policy_version 169796 (0.0006) [2023-12-27 06:49:55,990][20381] Updated weights for policy 1, policy_version 169806 (0.0006) [2023-12-27 06:49:56,046][20381] Updated weights for policy 1, policy_version 169816 (0.0005) [2023-12-27 06:49:56,396][20347] Updated weights for policy 0, policy_version 170125 (0.0010) [2023-12-27 06:49:56,454][20347] Updated weights for policy 0, policy_version 170135 (0.0010) [2023-12-27 06:49:56,512][20347] Updated weights for policy 0, policy_version 170145 (0.0010) [2023-12-27 06:49:56,605][20381] Updated weights for policy 1, policy_version 169826 (0.0010) [2023-12-27 06:49:56,668][20381] Updated weights for policy 1, policy_version 169836 (0.0010) [2023-12-27 06:49:56,722][20381] Updated weights for policy 1, policy_version 169846 (0.0009) [2023-12-27 06:49:57,151][20347] Updated weights for policy 0, policy_version 170155 (0.0010) [2023-12-27 06:49:57,198][20347] Updated weights for policy 0, policy_version 170165 (0.0010) [2023-12-27 06:49:57,253][20347] Updated weights for policy 0, policy_version 170175 (0.0010) [2023-12-27 06:49:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 87072768. Throughput: 0: 10533.4, 1: 10376.0. Samples: 87060848. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:49:57,363][19170] Avg episode reward: [(0, '381.870'), (1, '130.160')] [2023-12-27 06:49:57,373][20381] Updated weights for policy 1, policy_version 169856 (0.0005) [2023-12-27 06:49:57,419][20381] Updated weights for policy 1, policy_version 169866 (0.0007) [2023-12-27 06:49:57,467][20381] Updated weights for policy 1, policy_version 169876 (0.0010) [2023-12-27 06:49:57,935][20347] Updated weights for policy 0, policy_version 170185 (0.0010) [2023-12-27 06:49:57,989][20347] Updated weights for policy 0, policy_version 170195 (0.0010) [2023-12-27 06:49:58,043][20347] Updated weights for policy 0, policy_version 170205 (0.0010) [2023-12-27 06:49:58,090][20347] Updated weights for policy 0, policy_version 170215 (0.0010) [2023-12-27 06:49:58,145][20381] Updated weights for policy 1, policy_version 169886 (0.0010) [2023-12-27 06:49:58,204][20381] Updated weights for policy 1, policy_version 169896 (0.0010) [2023-12-27 06:49:58,256][20381] Updated weights for policy 1, policy_version 169906 (0.0010) [2023-12-27 06:49:58,852][20347] Updated weights for policy 0, policy_version 170225 (0.0009) [2023-12-27 06:49:58,915][20347] Updated weights for policy 0, policy_version 170235 (0.0008) [2023-12-27 06:49:58,979][20347] Updated weights for policy 0, policy_version 170245 (0.0007) [2023-12-27 06:49:59,009][20381] Updated weights for policy 1, policy_version 169916 (0.0009) [2023-12-27 06:49:59,072][20381] Updated weights for policy 1, policy_version 169926 (0.0007) [2023-12-27 06:49:59,130][20381] Updated weights for policy 1, policy_version 169936 (0.0008) [2023-12-27 06:49:59,642][20347] Updated weights for policy 0, policy_version 170255 (0.0009) [2023-12-27 06:49:59,693][20347] Updated weights for policy 0, policy_version 170265 (0.0008) [2023-12-27 06:49:59,737][20381] Updated weights for policy 1, policy_version 169946 (0.0006) [2023-12-27 06:49:59,755][20347] Updated weights for policy 0, policy_version 170275 (0.0007) [2023-12-27 06:49:59,796][20381] Updated weights for policy 1, policy_version 169956 (0.0005) [2023-12-27 06:49:59,857][20381] Updated weights for policy 1, policy_version 169966 (0.0006) [2023-12-27 06:49:59,931][20381] Updated weights for policy 1, policy_version 169976 (0.0007) [2023-12-27 06:50:00,479][20347] Updated weights for policy 0, policy_version 170285 (0.0007) [2023-12-27 06:50:00,522][20347] Updated weights for policy 0, policy_version 170295 (0.0006) [2023-12-27 06:50:00,552][20381] Updated weights for policy 1, policy_version 169986 (0.0006) [2023-12-27 06:50:00,575][20347] Updated weights for policy 0, policy_version 170305 (0.0005) [2023-12-27 06:50:00,604][20381] Updated weights for policy 1, policy_version 169996 (0.0008) [2023-12-27 06:50:00,653][20381] Updated weights for policy 1, policy_version 170006 (0.0008) [2023-12-27 06:50:01,154][20347] Updated weights for policy 0, policy_version 170315 (0.0005) [2023-12-27 06:50:01,215][20347] Updated weights for policy 0, policy_version 170325 (0.0005) [2023-12-27 06:50:01,280][20347] Updated weights for policy 0, policy_version 170335 (0.0007) [2023-12-27 06:50:01,381][20381] Updated weights for policy 1, policy_version 170016 (0.0008) [2023-12-27 06:50:01,433][20381] Updated weights for policy 1, policy_version 170026 (0.0008) [2023-12-27 06:50:01,483][20381] Updated weights for policy 1, policy_version 170036 (0.0008) [2023-12-27 06:50:01,978][20347] Updated weights for policy 0, policy_version 170345 (0.0007) [2023-12-27 06:50:02,040][20291] KL-divergence is very high: 127.4637 [2023-12-27 06:50:02,041][20347] Updated weights for policy 0, policy_version 170355 (0.0007) [2023-12-27 06:50:02,046][20291] KL-divergence is very high: 101.1407 [2023-12-27 06:50:02,096][20347] Updated weights for policy 0, policy_version 170365 (0.0008) [2023-12-27 06:50:02,149][20347] Updated weights for policy 0, policy_version 170375 (0.0007) [2023-12-27 06:50:02,166][20381] Updated weights for policy 1, policy_version 170046 (0.0008) [2023-12-27 06:50:02,226][20381] Updated weights for policy 1, policy_version 170056 (0.0007) [2023-12-27 06:50:02,289][20381] Updated weights for policy 1, policy_version 170066 (0.0008) [2023-12-27 06:50:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 87179264. Throughput: 0: 10462.7, 1: 10379.9. Samples: 87184232. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:50:02,363][19170] Avg episode reward: [(0, '407.430'), (1, '228.540')] [2023-12-27 06:50:02,791][20347] Updated weights for policy 0, policy_version 170385 (0.0005) [2023-12-27 06:50:02,801][20291] KL-divergence is very high: 153.6091 [2023-12-27 06:50:02,832][20291] KL-divergence is very high: 133.0091 [2023-12-27 06:50:02,848][20347] Updated weights for policy 0, policy_version 170395 (0.0005) [2023-12-27 06:50:02,904][20347] Updated weights for policy 0, policy_version 170405 (0.0006) [2023-12-27 06:50:03,003][20381] Updated weights for policy 1, policy_version 170076 (0.0007) [2023-12-27 06:50:03,052][20381] Updated weights for policy 1, policy_version 170086 (0.0005) [2023-12-27 06:50:03,106][20381] Updated weights for policy 1, policy_version 170096 (0.0005) [2023-12-27 06:50:03,482][20347] Updated weights for policy 0, policy_version 170415 (0.0007) [2023-12-27 06:50:03,540][20347] Updated weights for policy 0, policy_version 170425 (0.0005) [2023-12-27 06:50:03,601][20347] Updated weights for policy 0, policy_version 170435 (0.0005) [2023-12-27 06:50:03,756][20381] Updated weights for policy 1, policy_version 170106 (0.0006) [2023-12-27 06:50:03,805][20381] Updated weights for policy 1, policy_version 170116 (0.0007) [2023-12-27 06:50:03,859][20381] Updated weights for policy 1, policy_version 170126 (0.0007) [2023-12-27 06:50:03,906][20381] Updated weights for policy 1, policy_version 170136 (0.0008) [2023-12-27 06:50:04,239][20347] Updated weights for policy 0, policy_version 170445 (0.0008) [2023-12-27 06:50:04,295][20347] Updated weights for policy 0, policy_version 170455 (0.0010) [2023-12-27 06:50:04,305][20291] KL-divergence is very high: 148.3351 [2023-12-27 06:50:04,311][20291] KL-divergence is very high: 118.1445 [2023-12-27 06:50:04,340][20291] KL-divergence is very high: 147.1352 [2023-12-27 06:50:04,349][20347] Updated weights for policy 0, policy_version 170465 (0.0010) [2023-12-27 06:50:04,534][20381] Updated weights for policy 1, policy_version 170146 (0.0006) [2023-12-27 06:50:04,585][20381] Updated weights for policy 1, policy_version 170156 (0.0006) [2023-12-27 06:50:04,637][20381] Updated weights for policy 1, policy_version 170166 (0.0005) [2023-12-27 06:50:04,949][20347] Updated weights for policy 0, policy_version 170475 (0.0011) [2023-12-27 06:50:05,007][20347] Updated weights for policy 0, policy_version 170485 (0.0010) [2023-12-27 06:50:05,022][20291] KL-divergence is very high: 133.8528 [2023-12-27 06:50:05,027][20291] KL-divergence is very high: 116.0821 [2023-12-27 06:50:05,054][20347] Updated weights for policy 0, policy_version 170495 (0.0010) [2023-12-27 06:50:05,226][20381] Updated weights for policy 1, policy_version 170176 (0.0005) [2023-12-27 06:50:05,279][20381] Updated weights for policy 1, policy_version 170186 (0.0007) [2023-12-27 06:50:05,333][20381] Updated weights for policy 1, policy_version 170196 (0.0006) [2023-12-27 06:50:05,623][20291] KL-divergence is very high: 184.0455 [2023-12-27 06:50:05,631][20347] Updated weights for policy 0, policy_version 170505 (0.0007) [2023-12-27 06:50:05,650][20291] KL-divergence is very high: 149.3473 [2023-12-27 06:50:05,670][20291] KL-divergence is very high: 171.0016 [2023-12-27 06:50:05,690][20347] Updated weights for policy 0, policy_version 170515 (0.0009) [2023-12-27 06:50:05,715][20291] KL-divergence is very high: 122.9616 [2023-12-27 06:50:05,737][20347] Updated weights for policy 0, policy_version 170525 (0.0005) [2023-12-27 06:50:05,752][20291] KL-divergence is very high: 103.9389 [2023-12-27 06:50:05,792][20347] Updated weights for policy 0, policy_version 170535 (0.0009) [2023-12-27 06:50:05,908][20381] Updated weights for policy 1, policy_version 170206 (0.0007) [2023-12-27 06:50:05,971][20381] Updated weights for policy 1, policy_version 170216 (0.0007) [2023-12-27 06:50:06,027][20381] Updated weights for policy 1, policy_version 170226 (0.0008) [2023-12-27 06:50:06,371][20347] Updated weights for policy 0, policy_version 170545 (0.0008) [2023-12-27 06:50:06,423][20347] Updated weights for policy 0, policy_version 170555 (0.0007) [2023-12-27 06:50:06,473][20347] Updated weights for policy 0, policy_version 170565 (0.0008) [2023-12-27 06:50:06,733][20381] Updated weights for policy 1, policy_version 170236 (0.0007) [2023-12-27 06:50:06,792][20381] Updated weights for policy 1, policy_version 170246 (0.0007) [2023-12-27 06:50:06,840][20381] Updated weights for policy 1, policy_version 170256 (0.0007) [2023-12-27 06:50:07,200][20347] Updated weights for policy 0, policy_version 170575 (0.0006) [2023-12-27 06:50:07,255][20347] Updated weights for policy 0, policy_version 170585 (0.0006) [2023-12-27 06:50:07,310][20347] Updated weights for policy 0, policy_version 170595 (0.0005) [2023-12-27 06:50:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 87285760. Throughput: 0: 10515.5, 1: 10415.1. Samples: 87249904. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:50:07,363][19170] Avg episode reward: [(0, '192.240'), (1, '270.870')] [2023-12-27 06:50:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000170264_43597824.pth... [2023-12-27 06:50:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000170600_43687936.pth... [2023-12-27 06:50:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000169048_43286528.pth [2023-12-27 06:50:07,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000169352_43368448.pth [2023-12-27 06:50:07,424][20381] Updated weights for policy 1, policy_version 170266 (0.0007) [2023-12-27 06:50:07,484][20381] Updated weights for policy 1, policy_version 170276 (0.0007) [2023-12-27 06:50:07,542][20381] Updated weights for policy 1, policy_version 170286 (0.0007) [2023-12-27 06:50:07,605][20381] Updated weights for policy 1, policy_version 170296 (0.0008) [2023-12-27 06:50:07,959][20347] Updated weights for policy 0, policy_version 170605 (0.0005) [2023-12-27 06:50:08,023][20347] Updated weights for policy 0, policy_version 170615 (0.0006) [2023-12-27 06:50:08,082][20347] Updated weights for policy 0, policy_version 170625 (0.0009) [2023-12-27 06:50:08,297][20381] Updated weights for policy 1, policy_version 170306 (0.0006) [2023-12-27 06:50:08,354][20381] Updated weights for policy 1, policy_version 170316 (0.0008) [2023-12-27 06:50:08,420][20381] Updated weights for policy 1, policy_version 170326 (0.0006) [2023-12-27 06:50:08,785][20347] Updated weights for policy 0, policy_version 170635 (0.0007) [2023-12-27 06:50:08,831][20347] Updated weights for policy 0, policy_version 170645 (0.0006) [2023-12-27 06:50:08,887][20347] Updated weights for policy 0, policy_version 170655 (0.0008) [2023-12-27 06:50:08,942][20381] Updated weights for policy 1, policy_version 170336 (0.0007) [2023-12-27 06:50:08,996][20381] Updated weights for policy 1, policy_version 170346 (0.0008) [2023-12-27 06:50:09,052][20381] Updated weights for policy 1, policy_version 170356 (0.0007) [2023-12-27 06:50:09,645][20347] Updated weights for policy 0, policy_version 170665 (0.0007) [2023-12-27 06:50:09,699][20347] Updated weights for policy 0, policy_version 170675 (0.0008) [2023-12-27 06:50:09,738][20381] Updated weights for policy 1, policy_version 170366 (0.0006) [2023-12-27 06:50:09,755][20347] Updated weights for policy 0, policy_version 170685 (0.0007) [2023-12-27 06:50:09,805][20381] Updated weights for policy 1, policy_version 170376 (0.0007) [2023-12-27 06:50:09,812][20347] Updated weights for policy 0, policy_version 170695 (0.0007) [2023-12-27 06:50:09,867][20381] Updated weights for policy 1, policy_version 170386 (0.0007) [2023-12-27 06:50:10,521][20347] Updated weights for policy 0, policy_version 170705 (0.0006) [2023-12-27 06:50:10,535][20381] Updated weights for policy 1, policy_version 170396 (0.0008) [2023-12-27 06:50:10,579][20347] Updated weights for policy 0, policy_version 170715 (0.0008) [2023-12-27 06:50:10,594][20381] Updated weights for policy 1, policy_version 170406 (0.0010) [2023-12-27 06:50:10,631][20347] Updated weights for policy 0, policy_version 170725 (0.0007) [2023-12-27 06:50:10,650][20381] Updated weights for policy 1, policy_version 170416 (0.0010) [2023-12-27 06:50:11,323][20381] Updated weights for policy 1, policy_version 170426 (0.0010) [2023-12-27 06:50:11,370][20347] Updated weights for policy 0, policy_version 170735 (0.0008) [2023-12-27 06:50:11,387][20381] Updated weights for policy 1, policy_version 170436 (0.0007) [2023-12-27 06:50:11,439][20291] KL-divergence is very high: 104.1518 [2023-12-27 06:50:11,439][20347] Updated weights for policy 0, policy_version 170745 (0.0012) [2023-12-27 06:50:11,443][20381] Updated weights for policy 1, policy_version 170446 (0.0008) [2023-12-27 06:50:11,480][20291] KL-divergence is very high: 106.7030 [2023-12-27 06:50:11,486][20291] KL-divergence is very high: 137.8601 [2023-12-27 06:50:11,498][20347] Updated weights for policy 0, policy_version 170755 (0.0010) [2023-12-27 06:50:11,505][20381] Updated weights for policy 1, policy_version 170456 (0.0006) [2023-12-27 06:50:12,135][20381] Updated weights for policy 1, policy_version 170466 (0.0009) [2023-12-27 06:50:12,193][20381] Updated weights for policy 1, policy_version 170476 (0.0008) [2023-12-27 06:50:12,215][20347] Updated weights for policy 0, policy_version 170765 (0.0009) [2023-12-27 06:50:12,245][20381] Updated weights for policy 1, policy_version 170486 (0.0009) [2023-12-27 06:50:12,273][20347] Updated weights for policy 0, policy_version 170775 (0.0009) [2023-12-27 06:50:12,330][20347] Updated weights for policy 0, policy_version 170785 (0.0007) [2023-12-27 06:50:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 87384064. Throughput: 0: 10527.0, 1: 10460.3. Samples: 87376504. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:50:12,363][19170] Avg episode reward: [(0, '42.280'), (1, '246.190')] [2023-12-27 06:50:13,006][20381] Updated weights for policy 1, policy_version 170496 (0.0007) [2023-12-27 06:50:13,032][20347] Updated weights for policy 0, policy_version 170795 (0.0007) [2023-12-27 06:50:13,060][20381] Updated weights for policy 1, policy_version 170506 (0.0007) [2023-12-27 06:50:13,081][20347] Updated weights for policy 0, policy_version 170805 (0.0005) [2023-12-27 06:50:13,114][20381] Updated weights for policy 1, policy_version 170516 (0.0007) [2023-12-27 06:50:13,131][20347] Updated weights for policy 0, policy_version 170815 (0.0005) [2023-12-27 06:50:13,752][20381] Updated weights for policy 1, policy_version 170526 (0.0007) [2023-12-27 06:50:13,805][20381] Updated weights for policy 1, policy_version 170536 (0.0008) [2023-12-27 06:50:13,833][20347] Updated weights for policy 0, policy_version 170825 (0.0006) [2023-12-27 06:50:13,861][20381] Updated weights for policy 1, policy_version 170546 (0.0009) [2023-12-27 06:50:13,889][20347] Updated weights for policy 0, policy_version 170835 (0.0005) [2023-12-27 06:50:13,940][20347] Updated weights for policy 0, policy_version 170845 (0.0006) [2023-12-27 06:50:13,996][20347] Updated weights for policy 0, policy_version 170855 (0.0005) [2023-12-27 06:50:14,435][20381] Updated weights for policy 1, policy_version 170556 (0.0010) [2023-12-27 06:50:14,488][20381] Updated weights for policy 1, policy_version 170566 (0.0010) [2023-12-27 06:50:14,552][20381] Updated weights for policy 1, policy_version 170576 (0.0006) [2023-12-27 06:50:14,604][20347] Updated weights for policy 0, policy_version 170865 (0.0006) [2023-12-27 06:50:14,663][20347] Updated weights for policy 0, policy_version 170875 (0.0007) [2023-12-27 06:50:14,713][20347] Updated weights for policy 0, policy_version 170885 (0.0008) [2023-12-27 06:50:15,257][20381] Updated weights for policy 1, policy_version 170586 (0.0009) [2023-12-27 06:50:15,315][20381] Updated weights for policy 1, policy_version 170596 (0.0008) [2023-12-27 06:50:15,362][20381] Updated weights for policy 1, policy_version 170606 (0.0008) [2023-12-27 06:50:15,413][20381] Updated weights for policy 1, policy_version 170616 (0.0007) [2023-12-27 06:50:15,432][20347] Updated weights for policy 0, policy_version 170895 (0.0008) [2023-12-27 06:50:15,488][20347] Updated weights for policy 0, policy_version 170905 (0.0007) [2023-12-27 06:50:15,534][20347] Updated weights for policy 0, policy_version 170915 (0.0008) [2023-12-27 06:50:16,124][20381] Updated weights for policy 1, policy_version 170626 (0.0008) [2023-12-27 06:50:16,170][20381] Updated weights for policy 1, policy_version 170636 (0.0006) [2023-12-27 06:50:16,221][20347] Updated weights for policy 0, policy_version 170925 (0.0006) [2023-12-27 06:50:16,223][20381] Updated weights for policy 1, policy_version 170646 (0.0007) [2023-12-27 06:50:16,284][20347] Updated weights for policy 0, policy_version 170935 (0.0009) [2023-12-27 06:50:16,330][20347] Updated weights for policy 0, policy_version 170945 (0.0008) [2023-12-27 06:50:16,805][20381] Updated weights for policy 1, policy_version 170656 (0.0006) [2023-12-27 06:50:16,857][20381] Updated weights for policy 1, policy_version 170666 (0.0005) [2023-12-27 06:50:16,917][20381] Updated weights for policy 1, policy_version 170676 (0.0006) [2023-12-27 06:50:17,010][20347] Updated weights for policy 0, policy_version 170955 (0.0008) [2023-12-27 06:50:17,060][20347] Updated weights for policy 0, policy_version 170965 (0.0007) [2023-12-27 06:50:17,116][20347] Updated weights for policy 0, policy_version 170975 (0.0007) [2023-12-27 06:50:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 87490560. Throughput: 0: 10480.0, 1: 10470.6. Samples: 87499888. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-12-27 06:50:17,364][19170] Avg episode reward: [(0, '33.690'), (1, '260.360')] [2023-12-27 06:50:17,619][20381] Updated weights for policy 1, policy_version 170686 (0.0008) [2023-12-27 06:50:17,673][20381] Updated weights for policy 1, policy_version 170696 (0.0008) [2023-12-27 06:50:17,684][20347] Updated weights for policy 0, policy_version 170985 (0.0007) [2023-12-27 06:50:17,726][20381] Updated weights for policy 1, policy_version 170706 (0.0007) [2023-12-27 06:50:17,741][20347] Updated weights for policy 0, policy_version 170995 (0.0006) [2023-12-27 06:50:17,797][20347] Updated weights for policy 0, policy_version 171005 (0.0005) [2023-12-27 06:50:17,843][20347] Updated weights for policy 0, policy_version 171015 (0.0005) [2023-12-27 06:50:18,372][20347] Updated weights for policy 0, policy_version 171025 (0.0007) [2023-12-27 06:50:18,427][20347] Updated weights for policy 0, policy_version 171035 (0.0007) [2023-12-27 06:50:18,476][20347] Updated weights for policy 0, policy_version 171045 (0.0007) [2023-12-27 06:50:18,499][20381] Updated weights for policy 1, policy_version 170716 (0.0007) [2023-12-27 06:50:18,549][20381] Updated weights for policy 1, policy_version 170726 (0.0008) [2023-12-27 06:50:18,601][20381] Updated weights for policy 1, policy_version 170736 (0.0008) [2023-12-27 06:50:19,103][20347] Updated weights for policy 0, policy_version 171055 (0.0006) [2023-12-27 06:50:19,161][20347] Updated weights for policy 0, policy_version 171065 (0.0008) [2023-12-27 06:50:19,216][20347] Updated weights for policy 0, policy_version 171075 (0.0007) [2023-12-27 06:50:19,321][20381] Updated weights for policy 1, policy_version 170746 (0.0008) [2023-12-27 06:50:19,379][20381] Updated weights for policy 1, policy_version 170756 (0.0008) [2023-12-27 06:50:19,430][20381] Updated weights for policy 1, policy_version 170766 (0.0006) [2023-12-27 06:50:19,487][20381] Updated weights for policy 1, policy_version 170776 (0.0006) [2023-12-27 06:50:19,925][20347] Updated weights for policy 0, policy_version 171085 (0.0007) [2023-12-27 06:50:19,983][20347] Updated weights for policy 0, policy_version 171095 (0.0008) [2023-12-27 06:50:20,042][20347] Updated weights for policy 0, policy_version 171105 (0.0006) [2023-12-27 06:50:20,168][20381] Updated weights for policy 1, policy_version 170786 (0.0009) [2023-12-27 06:50:20,229][20381] Updated weights for policy 1, policy_version 170796 (0.0009) [2023-12-27 06:50:20,288][20381] Updated weights for policy 1, policy_version 170806 (0.0008) [2023-12-27 06:50:20,703][20347] Updated weights for policy 0, policy_version 171115 (0.0006) [2023-12-27 06:50:20,758][20347] Updated weights for policy 0, policy_version 171125 (0.0007) [2023-12-27 06:50:20,816][20347] Updated weights for policy 0, policy_version 171135 (0.0008) [2023-12-27 06:50:21,041][20381] Updated weights for policy 1, policy_version 170816 (0.0010) [2023-12-27 06:50:21,100][20381] Updated weights for policy 1, policy_version 170826 (0.0007) [2023-12-27 06:50:21,165][20381] Updated weights for policy 1, policy_version 170836 (0.0009) [2023-12-27 06:50:21,542][20347] Updated weights for policy 0, policy_version 171145 (0.0008) [2023-12-27 06:50:21,604][20347] Updated weights for policy 0, policy_version 171155 (0.0008) [2023-12-27 06:50:21,666][20347] Updated weights for policy 0, policy_version 171165 (0.0008) [2023-12-27 06:50:21,723][20347] Updated weights for policy 0, policy_version 171175 (0.0008) [2023-12-27 06:50:21,887][20381] Updated weights for policy 1, policy_version 170846 (0.0009) [2023-12-27 06:50:21,945][20381] Updated weights for policy 1, policy_version 170856 (0.0008) [2023-12-27 06:50:22,007][20381] Updated weights for policy 1, policy_version 170866 (0.0008) [2023-12-27 06:50:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 87588864. Throughput: 0: 10498.9, 1: 10449.9. Samples: 87562632. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:22,363][19170] Avg episode reward: [(0, '33.780'), (1, '220.390')] [2023-12-27 06:50:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000170872_43753472.pth... [2023-12-27 06:50:22,371][20347] Updated weights for policy 0, policy_version 171185 (0.0007) [2023-12-27 06:50:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000169656_43442176.pth [2023-12-27 06:50:22,428][20347] Updated weights for policy 0, policy_version 171195 (0.0006) [2023-12-27 06:50:22,493][20347] Updated weights for policy 0, policy_version 171205 (0.0006) [2023-12-27 06:50:22,510][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000171208_43843584.pth... [2023-12-27 06:50:22,514][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000169960_43524096.pth [2023-12-27 06:50:22,758][20381] Updated weights for policy 1, policy_version 170876 (0.0009) [2023-12-27 06:50:22,817][20381] Updated weights for policy 1, policy_version 170886 (0.0008) [2023-12-27 06:50:22,870][20381] Updated weights for policy 1, policy_version 170896 (0.0007) [2023-12-27 06:50:23,175][20347] Updated weights for policy 0, policy_version 171215 (0.0008) [2023-12-27 06:50:23,229][20347] Updated weights for policy 0, policy_version 171225 (0.0008) [2023-12-27 06:50:23,286][20347] Updated weights for policy 0, policy_version 171235 (0.0007) [2023-12-27 06:50:23,607][20381] Updated weights for policy 1, policy_version 170906 (0.0008) [2023-12-27 06:50:23,672][20381] Updated weights for policy 1, policy_version 170916 (0.0008) [2023-12-27 06:50:23,732][20381] Updated weights for policy 1, policy_version 170926 (0.0008) [2023-12-27 06:50:23,788][20381] Updated weights for policy 1, policy_version 170936 (0.0008) [2023-12-27 06:50:23,857][20347] Updated weights for policy 0, policy_version 171245 (0.0007) [2023-12-27 06:50:23,917][20347] Updated weights for policy 0, policy_version 171255 (0.0008) [2023-12-27 06:50:23,976][20347] Updated weights for policy 0, policy_version 171265 (0.0007) [2023-12-27 06:50:24,524][20381] Updated weights for policy 1, policy_version 170946 (0.0007) [2023-12-27 06:50:24,541][20347] Updated weights for policy 0, policy_version 171275 (0.0008) [2023-12-27 06:50:24,585][20381] Updated weights for policy 1, policy_version 170956 (0.0007) [2023-12-27 06:50:24,608][20347] Updated weights for policy 0, policy_version 171285 (0.0006) [2023-12-27 06:50:24,635][20381] Updated weights for policy 1, policy_version 170966 (0.0006) [2023-12-27 06:50:24,668][20347] Updated weights for policy 0, policy_version 171295 (0.0006) [2023-12-27 06:50:25,200][20347] Updated weights for policy 0, policy_version 171305 (0.0005) [2023-12-27 06:50:25,259][20347] Updated weights for policy 0, policy_version 171315 (0.0007) [2023-12-27 06:50:25,306][20381] Updated weights for policy 1, policy_version 170976 (0.0008) [2023-12-27 06:50:25,318][20347] Updated weights for policy 0, policy_version 171325 (0.0005) [2023-12-27 06:50:25,367][20381] Updated weights for policy 1, policy_version 170986 (0.0008) [2023-12-27 06:50:25,377][20347] Updated weights for policy 0, policy_version 171335 (0.0006) [2023-12-27 06:50:25,425][20381] Updated weights for policy 1, policy_version 170996 (0.0007) [2023-12-27 06:50:26,012][20347] Updated weights for policy 0, policy_version 171345 (0.0006) [2023-12-27 06:50:26,075][20347] Updated weights for policy 0, policy_version 171355 (0.0010) [2023-12-27 06:50:26,107][20381] Updated weights for policy 1, policy_version 171006 (0.0007) [2023-12-27 06:50:26,131][20347] Updated weights for policy 0, policy_version 171365 (0.0009) [2023-12-27 06:50:26,161][20381] Updated weights for policy 1, policy_version 171016 (0.0007) [2023-12-27 06:50:26,223][20381] Updated weights for policy 1, policy_version 171026 (0.0007) [2023-12-27 06:50:26,692][20347] Updated weights for policy 0, policy_version 171375 (0.0005) [2023-12-27 06:50:26,759][20347] Updated weights for policy 0, policy_version 171385 (0.0005) [2023-12-27 06:50:26,808][20347] Updated weights for policy 0, policy_version 171395 (0.0006) [2023-12-27 06:50:26,860][20381] Updated weights for policy 1, policy_version 171036 (0.0009) [2023-12-27 06:50:26,919][20381] Updated weights for policy 1, policy_version 171046 (0.0010) [2023-12-27 06:50:26,977][20381] Updated weights for policy 1, policy_version 171056 (0.0011) [2023-12-27 06:50:27,354][20347] Updated weights for policy 0, policy_version 171405 (0.0009) [2023-12-27 06:50:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 87695360. Throughput: 0: 10546.9, 1: 10341.4. Samples: 87685396. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:27,363][19170] Avg episode reward: [(0, '34.740'), (1, '126.320')] [2023-12-27 06:50:27,415][20347] Updated weights for policy 0, policy_version 171415 (0.0008) [2023-12-27 06:50:27,473][20347] Updated weights for policy 0, policy_version 171425 (0.0008) [2023-12-27 06:50:27,668][20381] Updated weights for policy 1, policy_version 171066 (0.0010) [2023-12-27 06:50:27,719][20381] Updated weights for policy 1, policy_version 171076 (0.0010) [2023-12-27 06:50:27,768][20381] Updated weights for policy 1, policy_version 171086 (0.0010) [2023-12-27 06:50:27,812][20381] Updated weights for policy 1, policy_version 171096 (0.0010) [2023-12-27 06:50:28,113][20347] Updated weights for policy 0, policy_version 171435 (0.0008) [2023-12-27 06:50:28,172][20347] Updated weights for policy 0, policy_version 171445 (0.0010) [2023-12-27 06:50:28,238][20347] Updated weights for policy 0, policy_version 171455 (0.0006) [2023-12-27 06:50:28,486][20381] Updated weights for policy 1, policy_version 171106 (0.0011) [2023-12-27 06:50:28,544][20381] Updated weights for policy 1, policy_version 171116 (0.0010) [2023-12-27 06:50:28,596][20381] Updated weights for policy 1, policy_version 171126 (0.0011) [2023-12-27 06:50:28,830][20347] Updated weights for policy 0, policy_version 171465 (0.0007) [2023-12-27 06:50:28,883][20347] Updated weights for policy 0, policy_version 171475 (0.0009) [2023-12-27 06:50:28,938][20347] Updated weights for policy 0, policy_version 171485 (0.0007) [2023-12-27 06:50:28,991][20347] Updated weights for policy 0, policy_version 171495 (0.0007) [2023-12-27 06:50:29,243][20381] Updated weights for policy 1, policy_version 171136 (0.0010) [2023-12-27 06:50:29,302][20381] Updated weights for policy 1, policy_version 171146 (0.0007) [2023-12-27 06:50:29,361][20381] Updated weights for policy 1, policy_version 171156 (0.0008) [2023-12-27 06:50:29,690][20347] Updated weights for policy 0, policy_version 171505 (0.0009) [2023-12-27 06:50:29,736][20347] Updated weights for policy 0, policy_version 171515 (0.0007) [2023-12-27 06:50:29,791][20347] Updated weights for policy 0, policy_version 171525 (0.0008) [2023-12-27 06:50:29,966][20381] Updated weights for policy 1, policy_version 171166 (0.0008) [2023-12-27 06:50:30,029][20381] Updated weights for policy 1, policy_version 171176 (0.0007) [2023-12-27 06:50:30,085][20381] Updated weights for policy 1, policy_version 171186 (0.0007) [2023-12-27 06:50:30,535][20347] Updated weights for policy 0, policy_version 171535 (0.0008) [2023-12-27 06:50:30,589][20347] Updated weights for policy 0, policy_version 171545 (0.0008) [2023-12-27 06:50:30,636][20347] Updated weights for policy 0, policy_version 171555 (0.0008) [2023-12-27 06:50:30,727][20381] Updated weights for policy 1, policy_version 171196 (0.0006) [2023-12-27 06:50:30,787][20381] Updated weights for policy 1, policy_version 171206 (0.0006) [2023-12-27 06:50:30,837][20381] Updated weights for policy 1, policy_version 171216 (0.0008) [2023-12-27 06:50:31,373][20347] Updated weights for policy 0, policy_version 171565 (0.0008) [2023-12-27 06:50:31,434][20347] Updated weights for policy 0, policy_version 171575 (0.0009) [2023-12-27 06:50:31,447][20381] Updated weights for policy 1, policy_version 171226 (0.0008) [2023-12-27 06:50:31,488][20347] Updated weights for policy 0, policy_version 171585 (0.0008) [2023-12-27 06:50:31,495][20381] Updated weights for policy 1, policy_version 171236 (0.0006) [2023-12-27 06:50:31,551][20381] Updated weights for policy 1, policy_version 171246 (0.0007) [2023-12-27 06:50:31,604][20381] Updated weights for policy 1, policy_version 171256 (0.0008) [2023-12-27 06:50:32,141][20347] Updated weights for policy 0, policy_version 171595 (0.0008) [2023-12-27 06:50:32,194][20347] Updated weights for policy 0, policy_version 171605 (0.0007) [2023-12-27 06:50:32,254][20347] Updated weights for policy 0, policy_version 171615 (0.0008) [2023-12-27 06:50:32,330][20381] Updated weights for policy 1, policy_version 171266 (0.0008) [2023-12-27 06:50:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 87801856. Throughput: 0: 10527.0, 1: 10381.3. Samples: 87812372. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:32,363][19170] Avg episode reward: [(0, '30.530'), (1, '244.040')] [2023-12-27 06:50:32,389][20381] Updated weights for policy 1, policy_version 171276 (0.0008) [2023-12-27 06:50:32,446][20381] Updated weights for policy 1, policy_version 171286 (0.0008) [2023-12-27 06:50:32,892][20347] Updated weights for policy 0, policy_version 171625 (0.0007) [2023-12-27 06:50:32,951][20347] Updated weights for policy 0, policy_version 171635 (0.0008) [2023-12-27 06:50:33,007][20347] Updated weights for policy 0, policy_version 171645 (0.0008) [2023-12-27 06:50:33,065][20347] Updated weights for policy 0, policy_version 171655 (0.0008) [2023-12-27 06:50:33,199][20381] Updated weights for policy 1, policy_version 171296 (0.0006) [2023-12-27 06:50:33,255][20381] Updated weights for policy 1, policy_version 171306 (0.0007) [2023-12-27 06:50:33,309][20381] Updated weights for policy 1, policy_version 171316 (0.0006) [2023-12-27 06:50:33,736][20347] Updated weights for policy 0, policy_version 171665 (0.0007) [2023-12-27 06:50:33,792][20347] Updated weights for policy 0, policy_version 171675 (0.0005) [2023-12-27 06:50:33,847][20347] Updated weights for policy 0, policy_version 171685 (0.0006) [2023-12-27 06:50:33,953][20381] Updated weights for policy 1, policy_version 171326 (0.0005) [2023-12-27 06:50:34,002][20381] Updated weights for policy 1, policy_version 171336 (0.0009) [2023-12-27 06:50:34,054][20381] Updated weights for policy 1, policy_version 171346 (0.0010) [2023-12-27 06:50:34,443][20347] Updated weights for policy 0, policy_version 171695 (0.0006) [2023-12-27 06:50:34,494][20347] Updated weights for policy 0, policy_version 171705 (0.0005) [2023-12-27 06:50:34,549][20347] Updated weights for policy 0, policy_version 171715 (0.0005) [2023-12-27 06:50:34,688][20381] Updated weights for policy 1, policy_version 171356 (0.0008) [2023-12-27 06:50:34,747][20381] Updated weights for policy 1, policy_version 171366 (0.0005) [2023-12-27 06:50:34,808][20381] Updated weights for policy 1, policy_version 171376 (0.0009) [2023-12-27 06:50:35,183][20347] Updated weights for policy 0, policy_version 171725 (0.0010) [2023-12-27 06:50:35,247][20347] Updated weights for policy 0, policy_version 171735 (0.0010) [2023-12-27 06:50:35,302][20347] Updated weights for policy 0, policy_version 171745 (0.0010) [2023-12-27 06:50:35,406][20381] Updated weights for policy 1, policy_version 171386 (0.0011) [2023-12-27 06:50:35,458][20381] Updated weights for policy 1, policy_version 171396 (0.0010) [2023-12-27 06:50:35,522][20381] Updated weights for policy 1, policy_version 171406 (0.0010) [2023-12-27 06:50:35,576][20381] Updated weights for policy 1, policy_version 171416 (0.0010) [2023-12-27 06:50:35,885][20347] Updated weights for policy 0, policy_version 171755 (0.0007) [2023-12-27 06:50:35,941][20347] Updated weights for policy 0, policy_version 171765 (0.0005) [2023-12-27 06:50:35,988][20347] Updated weights for policy 0, policy_version 171775 (0.0005) [2023-12-27 06:50:36,256][20381] Updated weights for policy 1, policy_version 171426 (0.0008) [2023-12-27 06:50:36,314][20381] Updated weights for policy 1, policy_version 171436 (0.0008) [2023-12-27 06:50:36,375][20381] Updated weights for policy 1, policy_version 171446 (0.0008) [2023-12-27 06:50:36,624][20347] Updated weights for policy 0, policy_version 171785 (0.0006) [2023-12-27 06:50:36,684][20347] Updated weights for policy 0, policy_version 171795 (0.0008) [2023-12-27 06:50:36,732][20347] Updated weights for policy 0, policy_version 171805 (0.0007) [2023-12-27 06:50:36,779][20347] Updated weights for policy 0, policy_version 171815 (0.0007) [2023-12-27 06:50:37,093][20381] Updated weights for policy 1, policy_version 171456 (0.0010) [2023-12-27 06:50:37,149][20381] Updated weights for policy 1, policy_version 171466 (0.0011) [2023-12-27 06:50:37,198][20381] Updated weights for policy 1, policy_version 171476 (0.0011) [2023-12-27 06:50:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 87908352. Throughput: 0: 10539.1, 1: 10364.5. Samples: 87875484. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:37,363][19170] Avg episode reward: [(0, '29.780'), (1, '358.120')] [2023-12-27 06:50:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000171480_43909120.pth... [2023-12-27 06:50:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000171816_43999232.pth... [2023-12-27 06:50:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000170264_43597824.pth [2023-12-27 06:50:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000170600_43687936.pth [2023-12-27 06:50:37,505][20347] Updated weights for policy 0, policy_version 171825 (0.0005) [2023-12-27 06:50:37,560][20347] Updated weights for policy 0, policy_version 171835 (0.0005) [2023-12-27 06:50:37,621][20347] Updated weights for policy 0, policy_version 171845 (0.0005) [2023-12-27 06:50:37,867][20381] Updated weights for policy 1, policy_version 171486 (0.0011) [2023-12-27 06:50:37,927][20381] Updated weights for policy 1, policy_version 171496 (0.0011) [2023-12-27 06:50:37,983][20381] Updated weights for policy 1, policy_version 171506 (0.0010) [2023-12-27 06:50:38,269][20347] Updated weights for policy 0, policy_version 171855 (0.0008) [2023-12-27 06:50:38,324][20347] Updated weights for policy 0, policy_version 171865 (0.0007) [2023-12-27 06:50:38,386][20347] Updated weights for policy 0, policy_version 171875 (0.0007) [2023-12-27 06:50:38,692][20381] Updated weights for policy 1, policy_version 171516 (0.0011) [2023-12-27 06:50:38,744][20381] Updated weights for policy 1, policy_version 171526 (0.0011) [2023-12-27 06:50:38,794][20381] Updated weights for policy 1, policy_version 171536 (0.0010) [2023-12-27 06:50:39,069][20347] Updated weights for policy 0, policy_version 171885 (0.0007) [2023-12-27 06:50:39,120][20347] Updated weights for policy 0, policy_version 171895 (0.0007) [2023-12-27 06:50:39,176][20347] Updated weights for policy 0, policy_version 171905 (0.0008) [2023-12-27 06:50:39,505][20381] Updated weights for policy 1, policy_version 171546 (0.0010) [2023-12-27 06:50:39,558][20381] Updated weights for policy 1, policy_version 171556 (0.0008) [2023-12-27 06:50:39,620][20381] Updated weights for policy 1, policy_version 171566 (0.0008) [2023-12-27 06:50:39,686][20381] Updated weights for policy 1, policy_version 171576 (0.0008) [2023-12-27 06:50:39,904][20347] Updated weights for policy 0, policy_version 171915 (0.0008) [2023-12-27 06:50:39,967][20347] Updated weights for policy 0, policy_version 171925 (0.0008) [2023-12-27 06:50:40,026][20347] Updated weights for policy 0, policy_version 171935 (0.0007) [2023-12-27 06:50:40,398][20381] Updated weights for policy 1, policy_version 171586 (0.0008) [2023-12-27 06:50:40,453][20381] Updated weights for policy 1, policy_version 171596 (0.0008) [2023-12-27 06:50:40,509][20381] Updated weights for policy 1, policy_version 171606 (0.0005) [2023-12-27 06:50:40,756][20347] Updated weights for policy 0, policy_version 171945 (0.0007) [2023-12-27 06:50:40,807][20347] Updated weights for policy 0, policy_version 171955 (0.0007) [2023-12-27 06:50:40,859][20347] Updated weights for policy 0, policy_version 171965 (0.0008) [2023-12-27 06:50:40,914][20347] Updated weights for policy 0, policy_version 171975 (0.0008) [2023-12-27 06:50:41,130][20381] Updated weights for policy 1, policy_version 171616 (0.0007) [2023-12-27 06:50:41,187][20381] Updated weights for policy 1, policy_version 171626 (0.0006) [2023-12-27 06:50:41,247][20381] Updated weights for policy 1, policy_version 171636 (0.0006) [2023-12-27 06:50:41,672][20347] Updated weights for policy 0, policy_version 171985 (0.0007) [2023-12-27 06:50:41,739][20347] Updated weights for policy 0, policy_version 171995 (0.0008) [2023-12-27 06:50:41,801][20347] Updated weights for policy 0, policy_version 172005 (0.0006) [2023-12-27 06:50:41,952][20381] Updated weights for policy 1, policy_version 171646 (0.0007) [2023-12-27 06:50:42,021][20381] Updated weights for policy 1, policy_version 171656 (0.0008) [2023-12-27 06:50:42,089][20381] Updated weights for policy 1, policy_version 171666 (0.0006) [2023-12-27 06:50:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 88006656. Throughput: 0: 10512.6, 1: 10301.6. Samples: 87997488. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:42,363][19170] Avg episode reward: [(0, '28.070'), (1, '756.900')] [2023-12-27 06:50:42,465][20347] Updated weights for policy 0, policy_version 172015 (0.0006) [2023-12-27 06:50:42,525][20347] Updated weights for policy 0, policy_version 172025 (0.0006) [2023-12-27 06:50:42,576][20347] Updated weights for policy 0, policy_version 172035 (0.0007) [2023-12-27 06:50:42,801][20381] Updated weights for policy 1, policy_version 171676 (0.0009) [2023-12-27 06:50:42,865][20381] Updated weights for policy 1, policy_version 171686 (0.0006) [2023-12-27 06:50:42,921][20381] Updated weights for policy 1, policy_version 171696 (0.0008) [2023-12-27 06:50:43,199][20347] Updated weights for policy 0, policy_version 172045 (0.0007) [2023-12-27 06:50:43,246][20347] Updated weights for policy 0, policy_version 172055 (0.0009) [2023-12-27 06:50:43,298][20347] Updated weights for policy 0, policy_version 172065 (0.0008) [2023-12-27 06:50:43,563][20381] Updated weights for policy 1, policy_version 171706 (0.0007) [2023-12-27 06:50:43,613][20381] Updated weights for policy 1, policy_version 171716 (0.0007) [2023-12-27 06:50:43,661][20381] Updated weights for policy 1, policy_version 171726 (0.0007) [2023-12-27 06:50:43,706][20381] Updated weights for policy 1, policy_version 171736 (0.0007) [2023-12-27 06:50:44,031][20347] Updated weights for policy 0, policy_version 172075 (0.0008) [2023-12-27 06:50:44,080][20347] Updated weights for policy 0, policy_version 172085 (0.0010) [2023-12-27 06:50:44,132][20347] Updated weights for policy 0, policy_version 172095 (0.0010) [2023-12-27 06:50:44,423][20381] Updated weights for policy 1, policy_version 171746 (0.0007) [2023-12-27 06:50:44,479][20381] Updated weights for policy 1, policy_version 171756 (0.0008) [2023-12-27 06:50:44,531][20381] Updated weights for policy 1, policy_version 171766 (0.0007) [2023-12-27 06:50:44,823][20347] Updated weights for policy 0, policy_version 172105 (0.0010) [2023-12-27 06:50:44,882][20347] Updated weights for policy 0, policy_version 172115 (0.0006) [2023-12-27 06:50:44,943][20347] Updated weights for policy 0, policy_version 172125 (0.0007) [2023-12-27 06:50:45,006][20347] Updated weights for policy 0, policy_version 172135 (0.0007) [2023-12-27 06:50:45,265][20381] Updated weights for policy 1, policy_version 171776 (0.0008) [2023-12-27 06:50:45,327][20381] Updated weights for policy 1, policy_version 171786 (0.0008) [2023-12-27 06:50:45,384][20381] Updated weights for policy 1, policy_version 171796 (0.0008) [2023-12-27 06:50:45,659][20347] Updated weights for policy 0, policy_version 172145 (0.0008) [2023-12-27 06:50:45,711][20347] Updated weights for policy 0, policy_version 172155 (0.0006) [2023-12-27 06:50:45,767][20347] Updated weights for policy 0, policy_version 172165 (0.0007) [2023-12-27 06:50:45,975][20381] Updated weights for policy 1, policy_version 171806 (0.0008) [2023-12-27 06:50:46,021][20381] Updated weights for policy 1, policy_version 171816 (0.0008) [2023-12-27 06:50:46,069][20381] Updated weights for policy 1, policy_version 171826 (0.0007) [2023-12-27 06:50:46,462][20347] Updated weights for policy 0, policy_version 172175 (0.0007) [2023-12-27 06:50:46,513][20347] Updated weights for policy 0, policy_version 172185 (0.0007) [2023-12-27 06:50:46,569][20347] Updated weights for policy 0, policy_version 172195 (0.0007) [2023-12-27 06:50:46,744][20381] Updated weights for policy 1, policy_version 171836 (0.0008) [2023-12-27 06:50:46,801][20381] Updated weights for policy 1, policy_version 171846 (0.0008) [2023-12-27 06:50:46,859][20381] Updated weights for policy 1, policy_version 171856 (0.0008) [2023-12-27 06:50:47,292][20347] Updated weights for policy 0, policy_version 172205 (0.0008) [2023-12-27 06:50:47,340][20347] Updated weights for policy 0, policy_version 172215 (0.0008) [2023-12-27 06:50:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 88104960. Throughput: 0: 10482.3, 1: 10286.4. Samples: 88118824. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:47,363][19170] Avg episode reward: [(0, '27.120'), (1, '299.730')] [2023-12-27 06:50:47,391][20347] Updated weights for policy 0, policy_version 172225 (0.0007) [2023-12-27 06:50:47,590][20381] Updated weights for policy 1, policy_version 171866 (0.0008) [2023-12-27 06:50:47,649][20381] Updated weights for policy 1, policy_version 171876 (0.0011) [2023-12-27 06:50:47,708][20381] Updated weights for policy 1, policy_version 171886 (0.0011) [2023-12-27 06:50:47,771][20381] Updated weights for policy 1, policy_version 171896 (0.0011) [2023-12-27 06:50:48,059][20347] Updated weights for policy 0, policy_version 172235 (0.0008) [2023-12-27 06:50:48,107][20347] Updated weights for policy 0, policy_version 172245 (0.0007) [2023-12-27 06:50:48,159][20347] Updated weights for policy 0, policy_version 172255 (0.0007) [2023-12-27 06:50:48,455][20381] Updated weights for policy 1, policy_version 171906 (0.0005) [2023-12-27 06:50:48,511][20381] Updated weights for policy 1, policy_version 171916 (0.0005) [2023-12-27 06:50:48,569][20381] Updated weights for policy 1, policy_version 171926 (0.0005) [2023-12-27 06:50:48,895][20347] Updated weights for policy 0, policy_version 172265 (0.0008) [2023-12-27 06:50:48,952][20347] Updated weights for policy 0, policy_version 172275 (0.0010) [2023-12-27 06:50:49,008][20347] Updated weights for policy 0, policy_version 172285 (0.0010) [2023-12-27 06:50:49,071][20347] Updated weights for policy 0, policy_version 172295 (0.0011) [2023-12-27 06:50:49,123][20381] Updated weights for policy 1, policy_version 171936 (0.0009) [2023-12-27 06:50:49,181][20381] Updated weights for policy 1, policy_version 171946 (0.0010) [2023-12-27 06:50:49,240][20381] Updated weights for policy 1, policy_version 171956 (0.0011) [2023-12-27 06:50:49,754][20347] Updated weights for policy 0, policy_version 172305 (0.0008) [2023-12-27 06:50:49,800][20347] Updated weights for policy 0, policy_version 172315 (0.0006) [2023-12-27 06:50:49,860][20347] Updated weights for policy 0, policy_version 172325 (0.0007) [2023-12-27 06:50:49,948][20381] Updated weights for policy 1, policy_version 171966 (0.0010) [2023-12-27 06:50:50,007][20381] Updated weights for policy 1, policy_version 171976 (0.0008) [2023-12-27 06:50:50,061][20381] Updated weights for policy 1, policy_version 171986 (0.0008) [2023-12-27 06:50:50,536][20347] Updated weights for policy 0, policy_version 172335 (0.0006) [2023-12-27 06:50:50,593][20347] Updated weights for policy 0, policy_version 172345 (0.0007) [2023-12-27 06:50:50,641][20347] Updated weights for policy 0, policy_version 172355 (0.0006) [2023-12-27 06:50:50,751][20381] Updated weights for policy 1, policy_version 171996 (0.0007) [2023-12-27 06:50:50,799][20381] Updated weights for policy 1, policy_version 172006 (0.0009) [2023-12-27 06:50:50,850][20381] Updated weights for policy 1, policy_version 172016 (0.0008) [2023-12-27 06:50:51,333][20347] Updated weights for policy 0, policy_version 172365 (0.0007) [2023-12-27 06:50:51,398][20347] Updated weights for policy 0, policy_version 172375 (0.0008) [2023-12-27 06:50:51,449][20347] Updated weights for policy 0, policy_version 172385 (0.0008) [2023-12-27 06:50:51,515][20381] Updated weights for policy 1, policy_version 172026 (0.0006) [2023-12-27 06:50:51,563][20381] Updated weights for policy 1, policy_version 172036 (0.0006) [2023-12-27 06:50:51,619][20381] Updated weights for policy 1, policy_version 172046 (0.0007) [2023-12-27 06:50:51,686][20381] Updated weights for policy 1, policy_version 172056 (0.0006) [2023-12-27 06:50:52,168][20347] Updated weights for policy 0, policy_version 172395 (0.0006) [2023-12-27 06:50:52,216][20347] Updated weights for policy 0, policy_version 172405 (0.0005) [2023-12-27 06:50:52,274][20347] Updated weights for policy 0, policy_version 172415 (0.0007) [2023-12-27 06:50:52,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 88211456. Throughput: 0: 10406.4, 1: 10256.6. Samples: 88179744. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:52,363][19170] Avg episode reward: [(0, '26.890'), (1, '237.660')] [2023-12-27 06:50:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000172424_44154880.pth... [2023-12-27 06:50:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000171208_43843584.pth [2023-12-27 06:50:52,436][20381] Updated weights for policy 1, policy_version 172066 (0.0008) [2023-12-27 06:50:52,496][20381] Updated weights for policy 1, policy_version 172076 (0.0008) [2023-12-27 06:50:52,550][20381] Updated weights for policy 1, policy_version 172086 (0.0007) [2023-12-27 06:50:52,559][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000172088_44064768.pth... [2023-12-27 06:50:52,562][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000170872_43753472.pth [2023-12-27 06:50:52,972][20347] Updated weights for policy 0, policy_version 172425 (0.0007) [2023-12-27 06:50:53,029][20347] Updated weights for policy 0, policy_version 172435 (0.0006) [2023-12-27 06:50:53,086][20347] Updated weights for policy 0, policy_version 172445 (0.0006) [2023-12-27 06:50:53,140][20347] Updated weights for policy 0, policy_version 172455 (0.0007) [2023-12-27 06:50:53,225][20381] Updated weights for policy 1, policy_version 172096 (0.0006) [2023-12-27 06:50:53,281][20381] Updated weights for policy 1, policy_version 172106 (0.0007) [2023-12-27 06:50:53,328][20381] Updated weights for policy 1, policy_version 172116 (0.0009) [2023-12-27 06:50:53,734][20347] Updated weights for policy 0, policy_version 172465 (0.0007) [2023-12-27 06:50:53,782][20347] Updated weights for policy 0, policy_version 172475 (0.0007) [2023-12-27 06:50:53,831][20347] Updated weights for policy 0, policy_version 172485 (0.0008) [2023-12-27 06:50:54,012][20381] Updated weights for policy 1, policy_version 172126 (0.0008) [2023-12-27 06:50:54,069][20381] Updated weights for policy 1, policy_version 172136 (0.0005) [2023-12-27 06:50:54,130][20381] Updated weights for policy 1, policy_version 172146 (0.0005) [2023-12-27 06:50:54,517][20347] Updated weights for policy 0, policy_version 172495 (0.0007) [2023-12-27 06:50:54,575][20347] Updated weights for policy 0, policy_version 172505 (0.0007) [2023-12-27 06:50:54,626][20347] Updated weights for policy 0, policy_version 172515 (0.0007) [2023-12-27 06:50:54,728][20381] Updated weights for policy 1, policy_version 172156 (0.0007) [2023-12-27 06:50:54,773][20381] Updated weights for policy 1, policy_version 172166 (0.0010) [2023-12-27 06:50:54,821][20381] Updated weights for policy 1, policy_version 172176 (0.0010) [2023-12-27 06:50:55,273][20347] Updated weights for policy 0, policy_version 172525 (0.0009) [2023-12-27 06:50:55,331][20347] Updated weights for policy 0, policy_version 172535 (0.0007) [2023-12-27 06:50:55,398][20347] Updated weights for policy 0, policy_version 172545 (0.0007) [2023-12-27 06:50:55,523][20381] Updated weights for policy 1, policy_version 172186 (0.0010) [2023-12-27 06:50:55,577][20381] Updated weights for policy 1, policy_version 172196 (0.0010) [2023-12-27 06:50:55,632][20381] Updated weights for policy 1, policy_version 172206 (0.0010) [2023-12-27 06:50:55,686][20381] Updated weights for policy 1, policy_version 172216 (0.0010) [2023-12-27 06:50:55,948][20347] Updated weights for policy 0, policy_version 172555 (0.0007) [2023-12-27 06:50:55,994][20347] Updated weights for policy 0, policy_version 172565 (0.0010) [2023-12-27 06:50:56,044][20347] Updated weights for policy 0, policy_version 172575 (0.0010) [2023-12-27 06:50:56,338][20381] Updated weights for policy 1, policy_version 172226 (0.0011) [2023-12-27 06:50:56,394][20381] Updated weights for policy 1, policy_version 172236 (0.0011) [2023-12-27 06:50:56,448][20381] Updated weights for policy 1, policy_version 172246 (0.0010) [2023-12-27 06:50:56,721][20347] Updated weights for policy 0, policy_version 172585 (0.0010) [2023-12-27 06:50:56,771][20347] Updated weights for policy 0, policy_version 172595 (0.0008) [2023-12-27 06:50:56,836][20347] Updated weights for policy 0, policy_version 172605 (0.0006) [2023-12-27 06:50:56,887][20347] Updated weights for policy 0, policy_version 172615 (0.0005) [2023-12-27 06:50:57,159][20381] Updated weights for policy 1, policy_version 172256 (0.0010) [2023-12-27 06:50:57,214][20381] Updated weights for policy 1, policy_version 172266 (0.0008) [2023-12-27 06:50:57,260][20381] Updated weights for policy 1, policy_version 172276 (0.0007) [2023-12-27 06:50:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 88317952. Throughput: 0: 10443.7, 1: 10201.7. Samples: 88305548. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:50:57,363][19170] Avg episode reward: [(0, '28.320'), (1, '641.020')] [2023-12-27 06:50:57,517][20347] Updated weights for policy 0, policy_version 172625 (0.0010) [2023-12-27 06:50:57,572][20347] Updated weights for policy 0, policy_version 172635 (0.0010) [2023-12-27 06:50:57,629][20347] Updated weights for policy 0, policy_version 172645 (0.0005) [2023-12-27 06:50:57,990][20381] Updated weights for policy 1, policy_version 172286 (0.0006) [2023-12-27 06:50:58,042][20381] Updated weights for policy 1, policy_version 172296 (0.0007) [2023-12-27 06:50:58,094][20381] Updated weights for policy 1, policy_version 172306 (0.0008) [2023-12-27 06:50:58,186][20347] Updated weights for policy 0, policy_version 172655 (0.0008) [2023-12-27 06:50:58,250][20347] Updated weights for policy 0, policy_version 172665 (0.0006) [2023-12-27 06:50:58,303][20347] Updated weights for policy 0, policy_version 172675 (0.0010) [2023-12-27 06:50:58,885][20381] Updated weights for policy 1, policy_version 172316 (0.0009) [2023-12-27 06:50:58,950][20381] Updated weights for policy 1, policy_version 172326 (0.0009) [2023-12-27 06:50:59,012][20381] Updated weights for policy 1, policy_version 172336 (0.0008) [2023-12-27 06:50:59,094][20347] Updated weights for policy 0, policy_version 172685 (0.0008) [2023-12-27 06:50:59,152][20347] Updated weights for policy 0, policy_version 172695 (0.0008) [2023-12-27 06:50:59,209][20347] Updated weights for policy 0, policy_version 172705 (0.0008) [2023-12-27 06:50:59,740][20381] Updated weights for policy 1, policy_version 172346 (0.0007) [2023-12-27 06:50:59,811][20381] Updated weights for policy 1, policy_version 172356 (0.0007) [2023-12-27 06:50:59,862][20347] Updated weights for policy 0, policy_version 172715 (0.0009) [2023-12-27 06:50:59,872][20381] Updated weights for policy 1, policy_version 172366 (0.0007) [2023-12-27 06:50:59,929][20381] Updated weights for policy 1, policy_version 172376 (0.0009) [2023-12-27 06:50:59,930][20347] Updated weights for policy 0, policy_version 172725 (0.0012) [2023-12-27 06:50:59,993][20347] Updated weights for policy 0, policy_version 172735 (0.0011) [2023-12-27 06:51:00,501][20381] Updated weights for policy 1, policy_version 172386 (0.0010) [2023-12-27 06:51:00,558][20381] Updated weights for policy 1, policy_version 172396 (0.0010) [2023-12-27 06:51:00,598][20347] Updated weights for policy 0, policy_version 172745 (0.0010) [2023-12-27 06:51:00,621][20381] Updated weights for policy 1, policy_version 172406 (0.0008) [2023-12-27 06:51:00,657][20347] Updated weights for policy 0, policy_version 172755 (0.0005) [2023-12-27 06:51:00,703][20347] Updated weights for policy 0, policy_version 172765 (0.0008) [2023-12-27 06:51:00,752][20347] Updated weights for policy 0, policy_version 172775 (0.0010) [2023-12-27 06:51:01,276][20381] Updated weights for policy 1, policy_version 172416 (0.0007) [2023-12-27 06:51:01,334][20381] Updated weights for policy 1, policy_version 172426 (0.0007) [2023-12-27 06:51:01,393][20381] Updated weights for policy 1, policy_version 172436 (0.0007) [2023-12-27 06:51:01,446][20347] Updated weights for policy 0, policy_version 172785 (0.0006) [2023-12-27 06:51:01,510][20347] Updated weights for policy 0, policy_version 172795 (0.0005) [2023-12-27 06:51:01,563][20347] Updated weights for policy 0, policy_version 172805 (0.0006) [2023-12-27 06:51:02,040][20381] Updated weights for policy 1, policy_version 172446 (0.0009) [2023-12-27 06:51:02,096][20381] Updated weights for policy 1, policy_version 172456 (0.0011) [2023-12-27 06:51:02,151][20381] Updated weights for policy 1, policy_version 172466 (0.0011) [2023-12-27 06:51:02,155][20347] Updated weights for policy 0, policy_version 172815 (0.0009) [2023-12-27 06:51:02,212][20347] Updated weights for policy 0, policy_version 172825 (0.0008) [2023-12-27 06:51:02,271][20347] Updated weights for policy 0, policy_version 172835 (0.0007) [2023-12-27 06:51:02,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 88424448. Throughput: 0: 10527.3, 1: 10132.1. Samples: 88429560. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:51:02,363][19170] Avg episode reward: [(0, '28.910'), (1, '632.440')] [2023-12-27 06:51:02,799][20381] Updated weights for policy 1, policy_version 172476 (0.0009) [2023-12-27 06:51:02,858][20381] Updated weights for policy 1, policy_version 172486 (0.0006) [2023-12-27 06:51:02,909][20381] Updated weights for policy 1, policy_version 172496 (0.0008) [2023-12-27 06:51:02,962][20347] Updated weights for policy 0, policy_version 172845 (0.0007) [2023-12-27 06:51:03,020][20347] Updated weights for policy 0, policy_version 172855 (0.0007) [2023-12-27 06:51:03,079][20347] Updated weights for policy 0, policy_version 172865 (0.0005) [2023-12-27 06:51:03,565][20381] Updated weights for policy 1, policy_version 172506 (0.0007) [2023-12-27 06:51:03,628][20381] Updated weights for policy 1, policy_version 172516 (0.0010) [2023-12-27 06:51:03,649][20347] Updated weights for policy 0, policy_version 172875 (0.0006) [2023-12-27 06:51:03,686][20381] Updated weights for policy 1, policy_version 172526 (0.0011) [2023-12-27 06:51:03,699][20347] Updated weights for policy 0, policy_version 172885 (0.0005) [2023-12-27 06:51:03,747][20381] Updated weights for policy 1, policy_version 172536 (0.0009) [2023-12-27 06:51:03,750][20347] Updated weights for policy 0, policy_version 172895 (0.0005) [2023-12-27 06:51:04,379][20381] Updated weights for policy 1, policy_version 172546 (0.0011) [2023-12-27 06:51:04,423][20347] Updated weights for policy 0, policy_version 172905 (0.0005) [2023-12-27 06:51:04,442][20381] Updated weights for policy 1, policy_version 172556 (0.0011) [2023-12-27 06:51:04,482][20347] Updated weights for policy 0, policy_version 172915 (0.0006) [2023-12-27 06:51:04,503][20381] Updated weights for policy 1, policy_version 172566 (0.0011) [2023-12-27 06:51:04,543][20347] Updated weights for policy 0, policy_version 172925 (0.0006) [2023-12-27 06:51:04,602][20347] Updated weights for policy 0, policy_version 172935 (0.0006) [2023-12-27 06:51:05,194][20381] Updated weights for policy 1, policy_version 172576 (0.0011) [2023-12-27 06:51:05,248][20347] Updated weights for policy 0, policy_version 172945 (0.0006) [2023-12-27 06:51:05,249][20381] Updated weights for policy 1, policy_version 172586 (0.0010) [2023-12-27 06:51:05,309][20347] Updated weights for policy 0, policy_version 172955 (0.0006) [2023-12-27 06:51:05,311][20381] Updated weights for policy 1, policy_version 172596 (0.0010) [2023-12-27 06:51:05,360][20347] Updated weights for policy 0, policy_version 172965 (0.0006) [2023-12-27 06:51:05,944][20381] Updated weights for policy 1, policy_version 172606 (0.0007) [2023-12-27 06:51:05,999][20347] Updated weights for policy 0, policy_version 172975 (0.0008) [2023-12-27 06:51:05,999][20381] Updated weights for policy 1, policy_version 172616 (0.0006) [2023-12-27 06:51:06,056][20347] Updated weights for policy 0, policy_version 172985 (0.0006) [2023-12-27 06:51:06,056][20381] Updated weights for policy 1, policy_version 172626 (0.0006) [2023-12-27 06:51:06,111][20347] Updated weights for policy 0, policy_version 172995 (0.0007) [2023-12-27 06:51:06,683][20381] Updated weights for policy 1, policy_version 172636 (0.0008) [2023-12-27 06:51:06,742][20381] Updated weights for policy 1, policy_version 172646 (0.0011) [2023-12-27 06:51:06,802][20381] Updated weights for policy 1, policy_version 172656 (0.0011) [2023-12-27 06:51:06,817][20347] Updated weights for policy 0, policy_version 173005 (0.0007) [2023-12-27 06:51:06,864][20347] Updated weights for policy 0, policy_version 173015 (0.0006) [2023-12-27 06:51:06,925][20347] Updated weights for policy 0, policy_version 173025 (0.0007) [2023-12-27 06:51:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 88522752. Throughput: 0: 10472.4, 1: 10193.5. Samples: 88492600. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:51:07,363][19170] Avg episode reward: [(0, '29.310'), (1, '739.460')] [2023-12-27 06:51:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000172664_44212224.pth... [2023-12-27 06:51:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000173032_44310528.pth... [2023-12-27 06:51:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000171480_43909120.pth [2023-12-27 06:51:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000171816_43999232.pth [2023-12-27 06:51:07,493][20381] Updated weights for policy 1, policy_version 172666 (0.0010) [2023-12-27 06:51:07,559][20381] Updated weights for policy 1, policy_version 172676 (0.0010) [2023-12-27 06:51:07,615][20347] Updated weights for policy 0, policy_version 173035 (0.0007) [2023-12-27 06:51:07,620][20381] Updated weights for policy 1, policy_version 172686 (0.0010) [2023-12-27 06:51:07,665][20347] Updated weights for policy 0, policy_version 173045 (0.0006) [2023-12-27 06:51:07,675][20381] Updated weights for policy 1, policy_version 172696 (0.0011) [2023-12-27 06:51:07,724][20347] Updated weights for policy 0, policy_version 173055 (0.0006) [2023-12-27 06:51:08,374][20381] Updated weights for policy 1, policy_version 172706 (0.0010) [2023-12-27 06:51:08,413][20347] Updated weights for policy 0, policy_version 173065 (0.0007) [2023-12-27 06:51:08,431][20381] Updated weights for policy 1, policy_version 172716 (0.0011) [2023-12-27 06:51:08,468][20347] Updated weights for policy 0, policy_version 173075 (0.0006) [2023-12-27 06:51:08,474][20381] Updated weights for policy 1, policy_version 172726 (0.0011) [2023-12-27 06:51:08,523][20347] Updated weights for policy 0, policy_version 173085 (0.0007) [2023-12-27 06:51:08,580][20347] Updated weights for policy 0, policy_version 173095 (0.0008) [2023-12-27 06:51:09,181][20381] Updated weights for policy 1, policy_version 172736 (0.0008) [2023-12-27 06:51:09,246][20381] Updated weights for policy 1, policy_version 172746 (0.0008) [2023-12-27 06:51:09,278][20347] Updated weights for policy 0, policy_version 173105 (0.0007) [2023-12-27 06:51:09,302][20381] Updated weights for policy 1, policy_version 172756 (0.0011) [2023-12-27 06:51:09,340][20347] Updated weights for policy 0, policy_version 173115 (0.0007) [2023-12-27 06:51:09,402][20347] Updated weights for policy 0, policy_version 173125 (0.0008) [2023-12-27 06:51:10,047][20347] Updated weights for policy 0, policy_version 173135 (0.0007) [2023-12-27 06:51:10,049][20381] Updated weights for policy 1, policy_version 172766 (0.0008) [2023-12-27 06:51:10,101][20347] Updated weights for policy 0, policy_version 173145 (0.0008) [2023-12-27 06:51:10,110][20381] Updated weights for policy 1, policy_version 172776 (0.0005) [2023-12-27 06:51:10,162][20347] Updated weights for policy 0, policy_version 173155 (0.0008) [2023-12-27 06:51:10,172][20381] Updated weights for policy 1, policy_version 172786 (0.0006) [2023-12-27 06:51:10,793][20381] Updated weights for policy 1, policy_version 172796 (0.0007) [2023-12-27 06:51:10,793][20347] Updated weights for policy 0, policy_version 173165 (0.0009) [2023-12-27 06:51:10,846][20381] Updated weights for policy 1, policy_version 172806 (0.0011) [2023-12-27 06:51:10,848][20347] Updated weights for policy 0, policy_version 173175 (0.0006) [2023-12-27 06:51:10,908][20381] Updated weights for policy 1, policy_version 172816 (0.0008) [2023-12-27 06:51:10,908][20347] Updated weights for policy 0, policy_version 173185 (0.0007) [2023-12-27 06:51:11,598][20381] Updated weights for policy 1, policy_version 172826 (0.0008) [2023-12-27 06:51:11,624][20347] Updated weights for policy 0, policy_version 173195 (0.0007) [2023-12-27 06:51:11,659][20381] Updated weights for policy 1, policy_version 172836 (0.0011) [2023-12-27 06:51:11,686][20347] Updated weights for policy 0, policy_version 173205 (0.0007) [2023-12-27 06:51:11,719][20381] Updated weights for policy 1, policy_version 172846 (0.0011) [2023-12-27 06:51:11,749][20347] Updated weights for policy 0, policy_version 173215 (0.0007) [2023-12-27 06:51:11,779][20381] Updated weights for policy 1, policy_version 172856 (0.0008) [2023-12-27 06:51:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 88621056. Throughput: 0: 10404.7, 1: 10262.6. Samples: 88615428. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:51:12,363][19170] Avg episode reward: [(0, '28.000'), (1, '802.640')] [2023-12-27 06:51:12,395][20347] Updated weights for policy 0, policy_version 173225 (0.0008) [2023-12-27 06:51:12,459][20347] Updated weights for policy 0, policy_version 173235 (0.0007) [2023-12-27 06:51:12,472][20381] Updated weights for policy 1, policy_version 172866 (0.0008) [2023-12-27 06:51:12,513][20347] Updated weights for policy 0, policy_version 173245 (0.0005) [2023-12-27 06:51:12,533][20381] Updated weights for policy 1, policy_version 172876 (0.0007) [2023-12-27 06:51:12,561][20347] Updated weights for policy 0, policy_version 173255 (0.0006) [2023-12-27 06:51:12,591][20381] Updated weights for policy 1, policy_version 172886 (0.0007) [2023-12-27 06:51:13,229][20347] Updated weights for policy 0, policy_version 173265 (0.0008) [2023-12-27 06:51:13,287][20347] Updated weights for policy 0, policy_version 173275 (0.0007) [2023-12-27 06:51:13,320][20381] Updated weights for policy 1, policy_version 172896 (0.0007) [2023-12-27 06:51:13,350][20347] Updated weights for policy 0, policy_version 173285 (0.0009) [2023-12-27 06:51:13,386][20381] Updated weights for policy 1, policy_version 172906 (0.0006) [2023-12-27 06:51:13,444][20381] Updated weights for policy 1, policy_version 172916 (0.0006) [2023-12-27 06:51:14,054][20347] Updated weights for policy 0, policy_version 173295 (0.0007) [2023-12-27 06:51:14,102][20381] Updated weights for policy 1, policy_version 172926 (0.0009) [2023-12-27 06:51:14,107][20347] Updated weights for policy 0, policy_version 173305 (0.0006) [2023-12-27 06:51:14,151][20381] Updated weights for policy 1, policy_version 172936 (0.0010) [2023-12-27 06:51:14,163][20347] Updated weights for policy 0, policy_version 173315 (0.0007) [2023-12-27 06:51:14,207][20381] Updated weights for policy 1, policy_version 172946 (0.0010) [2023-12-27 06:51:14,867][20347] Updated weights for policy 0, policy_version 173325 (0.0008) [2023-12-27 06:51:14,907][20381] Updated weights for policy 1, policy_version 172956 (0.0008) [2023-12-27 06:51:14,925][20347] Updated weights for policy 0, policy_version 173335 (0.0008) [2023-12-27 06:51:14,969][20381] Updated weights for policy 1, policy_version 172966 (0.0006) [2023-12-27 06:51:14,981][20347] Updated weights for policy 0, policy_version 173345 (0.0006) [2023-12-27 06:51:15,030][20381] Updated weights for policy 1, policy_version 172976 (0.0006) [2023-12-27 06:51:15,590][20381] Updated weights for policy 1, policy_version 172986 (0.0006) [2023-12-27 06:51:15,635][20381] Updated weights for policy 1, policy_version 172996 (0.0005) [2023-12-27 06:51:15,681][20381] Updated weights for policy 1, policy_version 173006 (0.0005) [2023-12-27 06:51:15,710][20347] Updated weights for policy 0, policy_version 173355 (0.0007) [2023-12-27 06:51:15,730][20381] Updated weights for policy 1, policy_version 173016 (0.0005) [2023-12-27 06:51:15,768][20347] Updated weights for policy 0, policy_version 173365 (0.0008) [2023-12-27 06:51:15,818][20347] Updated weights for policy 0, policy_version 173375 (0.0009) [2023-12-27 06:51:16,253][20381] Updated weights for policy 1, policy_version 173026 (0.0005) [2023-12-27 06:51:16,310][20381] Updated weights for policy 1, policy_version 173036 (0.0005) [2023-12-27 06:51:16,370][20381] Updated weights for policy 1, policy_version 173046 (0.0005) [2023-12-27 06:51:16,596][20347] Updated weights for policy 0, policy_version 173385 (0.0008) [2023-12-27 06:51:16,655][20347] Updated weights for policy 0, policy_version 173395 (0.0007) [2023-12-27 06:51:16,710][20347] Updated weights for policy 0, policy_version 173405 (0.0009) [2023-12-27 06:51:16,764][20347] Updated weights for policy 0, policy_version 173415 (0.0009) [2023-12-27 06:51:16,948][20381] Updated weights for policy 1, policy_version 173056 (0.0007) [2023-12-27 06:51:17,005][20381] Updated weights for policy 1, policy_version 173066 (0.0006) [2023-12-27 06:51:17,058][20381] Updated weights for policy 1, policy_version 173076 (0.0006) [2023-12-27 06:51:17,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 88727552. Throughput: 0: 10288.5, 1: 10291.7. Samples: 88738488. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:51:17,364][19170] Avg episode reward: [(0, '27.510'), (1, '776.630')] [2023-12-27 06:51:17,507][20347] Updated weights for policy 0, policy_version 173425 (0.0008) [2023-12-27 06:51:17,560][20347] Updated weights for policy 0, policy_version 173435 (0.0007) [2023-12-27 06:51:17,579][20381] Updated weights for policy 1, policy_version 173086 (0.0005) [2023-12-27 06:51:17,616][20347] Updated weights for policy 0, policy_version 173445 (0.0007) [2023-12-27 06:51:17,622][20381] Updated weights for policy 1, policy_version 173096 (0.0005) [2023-12-27 06:51:17,667][20381] Updated weights for policy 1, policy_version 173106 (0.0005) [2023-12-27 06:51:18,201][20381] Updated weights for policy 1, policy_version 173116 (0.0005) [2023-12-27 06:51:18,260][20381] Updated weights for policy 1, policy_version 173126 (0.0005) [2023-12-27 06:51:18,310][20381] Updated weights for policy 1, policy_version 173136 (0.0008) [2023-12-27 06:51:18,323][20347] Updated weights for policy 0, policy_version 173455 (0.0006) [2023-12-27 06:51:18,382][20347] Updated weights for policy 0, policy_version 173465 (0.0008) [2023-12-27 06:51:18,438][20347] Updated weights for policy 0, policy_version 173475 (0.0008) [2023-12-27 06:51:19,033][20381] Updated weights for policy 1, policy_version 173146 (0.0008) [2023-12-27 06:51:19,089][20381] Updated weights for policy 1, policy_version 173156 (0.0008) [2023-12-27 06:51:19,092][20347] Updated weights for policy 0, policy_version 173485 (0.0009) [2023-12-27 06:51:19,145][20381] Updated weights for policy 1, policy_version 173166 (0.0007) [2023-12-27 06:51:19,149][20347] Updated weights for policy 0, policy_version 173495 (0.0009) [2023-12-27 06:51:19,196][20381] Updated weights for policy 1, policy_version 173176 (0.0008) [2023-12-27 06:51:19,205][20347] Updated weights for policy 0, policy_version 173505 (0.0008) [2023-12-27 06:51:19,851][20347] Updated weights for policy 0, policy_version 173515 (0.0008) [2023-12-27 06:51:19,912][20347] Updated weights for policy 0, policy_version 173525 (0.0006) [2023-12-27 06:51:19,927][20381] Updated weights for policy 1, policy_version 173186 (0.0008) [2023-12-27 06:51:19,977][20347] Updated weights for policy 0, policy_version 173535 (0.0008) [2023-12-27 06:51:19,988][20381] Updated weights for policy 1, policy_version 173196 (0.0006) [2023-12-27 06:51:20,039][20381] Updated weights for policy 1, policy_version 173206 (0.0006) [2023-12-27 06:51:20,650][20347] Updated weights for policy 0, policy_version 173545 (0.0008) [2023-12-27 06:51:20,711][20347] Updated weights for policy 0, policy_version 173555 (0.0007) [2023-12-27 06:51:20,757][20381] Updated weights for policy 1, policy_version 173216 (0.0005) [2023-12-27 06:51:20,775][20347] Updated weights for policy 0, policy_version 173565 (0.0008) [2023-12-27 06:51:20,807][20381] Updated weights for policy 1, policy_version 173226 (0.0005) [2023-12-27 06:51:20,839][20347] Updated weights for policy 0, policy_version 173575 (0.0007) [2023-12-27 06:51:20,862][20381] Updated weights for policy 1, policy_version 173236 (0.0005) [2023-12-27 06:51:21,573][20381] Updated weights for policy 1, policy_version 173246 (0.0006) [2023-12-27 06:51:21,587][20347] Updated weights for policy 0, policy_version 173585 (0.0007) [2023-12-27 06:51:21,630][20381] Updated weights for policy 1, policy_version 173256 (0.0006) [2023-12-27 06:51:21,642][20347] Updated weights for policy 0, policy_version 173595 (0.0007) [2023-12-27 06:51:21,696][20381] Updated weights for policy 1, policy_version 173266 (0.0009) [2023-12-27 06:51:21,699][20347] Updated weights for policy 0, policy_version 173605 (0.0008) [2023-12-27 06:51:22,347][20381] Updated weights for policy 1, policy_version 173276 (0.0007) [2023-12-27 06:51:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 88825856. Throughput: 0: 10249.2, 1: 10321.2. Samples: 88801148. Policy #0 lag: (min: 17.0, avg: 37.9, max: 49.0) [2023-12-27 06:51:22,363][19170] Avg episode reward: [(0, '26.960'), (1, '891.230')] [2023-12-27 06:51:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000173608_44457984.pth... [2023-12-27 06:51:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000172424_44154880.pth [2023-12-27 06:51:22,410][20381] Updated weights for policy 1, policy_version 173286 (0.0006) [2023-12-27 06:51:22,446][20347] Updated weights for policy 0, policy_version 173615 (0.0007) [2023-12-27 06:51:22,467][20381] Updated weights for policy 1, policy_version 173296 (0.0006) [2023-12-27 06:51:22,504][20347] Updated weights for policy 0, policy_version 173625 (0.0006) [2023-12-27 06:51:22,516][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000173304_44376064.pth... [2023-12-27 06:51:22,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000172088_44064768.pth [2023-12-27 06:51:22,557][20347] Updated weights for policy 0, policy_version 173635 (0.0006) [2023-12-27 06:51:23,145][20381] Updated weights for policy 1, policy_version 173306 (0.0006) [2023-12-27 06:51:23,213][20381] Updated weights for policy 1, policy_version 173316 (0.0008) [2023-12-27 06:51:23,275][20381] Updated weights for policy 1, policy_version 173326 (0.0007) [2023-12-27 06:51:23,277][20347] Updated weights for policy 0, policy_version 173645 (0.0006) [2023-12-27 06:51:23,324][20381] Updated weights for policy 1, policy_version 173336 (0.0006) [2023-12-27 06:51:23,334][20347] Updated weights for policy 0, policy_version 173655 (0.0005) [2023-12-27 06:51:23,393][20347] Updated weights for policy 0, policy_version 173665 (0.0008) [2023-12-27 06:51:23,987][20381] Updated weights for policy 1, policy_version 173346 (0.0008) [2023-12-27 06:51:24,016][20347] Updated weights for policy 0, policy_version 173675 (0.0007) [2023-12-27 06:51:24,041][20381] Updated weights for policy 1, policy_version 173356 (0.0008) [2023-12-27 06:51:24,075][20347] Updated weights for policy 0, policy_version 173685 (0.0005) [2023-12-27 06:51:24,098][20381] Updated weights for policy 1, policy_version 173366 (0.0008) [2023-12-27 06:51:24,123][20347] Updated weights for policy 0, policy_version 173695 (0.0006) [2023-12-27 06:51:24,800][20381] Updated weights for policy 1, policy_version 173376 (0.0010) [2023-12-27 06:51:24,803][20347] Updated weights for policy 0, policy_version 173705 (0.0008) [2023-12-27 06:51:24,857][20381] Updated weights for policy 1, policy_version 173386 (0.0010) [2023-12-27 06:51:24,858][20347] Updated weights for policy 0, policy_version 173715 (0.0010) [2023-12-27 06:51:24,912][20381] Updated weights for policy 1, policy_version 173396 (0.0011) [2023-12-27 06:51:24,915][20347] Updated weights for policy 0, policy_version 173725 (0.0010) [2023-12-27 06:51:24,978][20347] Updated weights for policy 0, policy_version 173735 (0.0010) [2023-12-27 06:51:25,482][20381] Updated weights for policy 1, policy_version 173406 (0.0011) [2023-12-27 06:51:25,538][20381] Updated weights for policy 1, policy_version 173416 (0.0010) [2023-12-27 06:51:25,596][20381] Updated weights for policy 1, policy_version 173426 (0.0010) [2023-12-27 06:51:25,636][20347] Updated weights for policy 0, policy_version 173745 (0.0006) [2023-12-27 06:51:25,697][20347] Updated weights for policy 0, policy_version 173755 (0.0005) [2023-12-27 06:51:25,755][20347] Updated weights for policy 0, policy_version 173765 (0.0006) [2023-12-27 06:51:26,177][20381] Updated weights for policy 1, policy_version 173436 (0.0010) [2023-12-27 06:51:26,226][20381] Updated weights for policy 1, policy_version 173446 (0.0006) [2023-12-27 06:51:26,273][20381] Updated weights for policy 1, policy_version 173456 (0.0005) [2023-12-27 06:51:26,435][20347] Updated weights for policy 0, policy_version 173775 (0.0006) [2023-12-27 06:51:26,484][20347] Updated weights for policy 0, policy_version 173785 (0.0006) [2023-12-27 06:51:26,533][20347] Updated weights for policy 0, policy_version 173795 (0.0006) [2023-12-27 06:51:26,831][20381] Updated weights for policy 1, policy_version 173466 (0.0005) [2023-12-27 06:51:26,882][20381] Updated weights for policy 1, policy_version 173476 (0.0005) [2023-12-27 06:51:26,926][20381] Updated weights for policy 1, policy_version 173486 (0.0005) [2023-12-27 06:51:26,977][20381] Updated weights for policy 1, policy_version 173496 (0.0007) [2023-12-27 06:51:27,181][20347] Updated weights for policy 0, policy_version 173805 (0.0006) [2023-12-27 06:51:27,233][20347] Updated weights for policy 0, policy_version 173815 (0.0006) [2023-12-27 06:51:27,279][20347] Updated weights for policy 0, policy_version 173825 (0.0005) [2023-12-27 06:51:27,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 88940544. Throughput: 0: 10226.9, 1: 10370.8. Samples: 88924388. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:27,363][19170] Avg episode reward: [(0, '25.820'), (1, '990.300')] [2023-12-27 06:51:27,566][20381] Updated weights for policy 1, policy_version 173506 (0.0005) [2023-12-27 06:51:27,627][20381] Updated weights for policy 1, policy_version 173516 (0.0005) [2023-12-27 06:51:27,680][20381] Updated weights for policy 1, policy_version 173526 (0.0007) [2023-12-27 06:51:27,976][20347] Updated weights for policy 0, policy_version 173835 (0.0007) [2023-12-27 06:51:28,034][20347] Updated weights for policy 0, policy_version 173845 (0.0008) [2023-12-27 06:51:28,082][20347] Updated weights for policy 0, policy_version 173855 (0.0007) [2023-12-27 06:51:28,317][20381] Updated weights for policy 1, policy_version 173536 (0.0009) [2023-12-27 06:51:28,379][20381] Updated weights for policy 1, policy_version 173546 (0.0008) [2023-12-27 06:51:28,438][20381] Updated weights for policy 1, policy_version 173556 (0.0008) [2023-12-27 06:51:28,772][20347] Updated weights for policy 0, policy_version 173865 (0.0008) [2023-12-27 06:51:28,834][20347] Updated weights for policy 0, policy_version 173875 (0.0007) [2023-12-27 06:51:28,893][20347] Updated weights for policy 0, policy_version 173885 (0.0008) [2023-12-27 06:51:28,960][20347] Updated weights for policy 0, policy_version 173895 (0.0008) [2023-12-27 06:51:29,170][20381] Updated weights for policy 1, policy_version 173566 (0.0007) [2023-12-27 06:51:29,224][20381] Updated weights for policy 1, policy_version 173576 (0.0008) [2023-12-27 06:51:29,289][20381] Updated weights for policy 1, policy_version 173586 (0.0007) [2023-12-27 06:51:29,641][20347] Updated weights for policy 0, policy_version 173905 (0.0005) [2023-12-27 06:51:29,693][20347] Updated weights for policy 0, policy_version 173915 (0.0005) [2023-12-27 06:51:29,754][20347] Updated weights for policy 0, policy_version 173925 (0.0005) [2023-12-27 06:51:30,028][20381] Updated weights for policy 1, policy_version 173596 (0.0008) [2023-12-27 06:51:30,081][20381] Updated weights for policy 1, policy_version 173606 (0.0008) [2023-12-27 06:51:30,140][20381] Updated weights for policy 1, policy_version 173616 (0.0007) [2023-12-27 06:51:30,398][20347] Updated weights for policy 0, policy_version 173935 (0.0008) [2023-12-27 06:51:30,459][20347] Updated weights for policy 0, policy_version 173945 (0.0007) [2023-12-27 06:51:30,514][20347] Updated weights for policy 0, policy_version 173955 (0.0008) [2023-12-27 06:51:30,831][20381] Updated weights for policy 1, policy_version 173626 (0.0008) [2023-12-27 06:51:30,882][20381] Updated weights for policy 1, policy_version 173636 (0.0008) [2023-12-27 06:51:30,946][20381] Updated weights for policy 1, policy_version 173646 (0.0007) [2023-12-27 06:51:30,996][20381] Updated weights for policy 1, policy_version 173656 (0.0005) [2023-12-27 06:51:31,228][20347] Updated weights for policy 0, policy_version 173965 (0.0008) [2023-12-27 06:51:31,287][20347] Updated weights for policy 0, policy_version 173975 (0.0008) [2023-12-27 06:51:31,343][20347] Updated weights for policy 0, policy_version 173985 (0.0008) [2023-12-27 06:51:31,652][20381] Updated weights for policy 1, policy_version 173666 (0.0007) [2023-12-27 06:51:31,708][20381] Updated weights for policy 1, policy_version 173676 (0.0007) [2023-12-27 06:51:31,772][20381] Updated weights for policy 1, policy_version 173686 (0.0008) [2023-12-27 06:51:32,075][20347] Updated weights for policy 0, policy_version 173995 (0.0008) [2023-12-27 06:51:32,129][20347] Updated weights for policy 0, policy_version 174005 (0.0007) [2023-12-27 06:51:32,178][20347] Updated weights for policy 0, policy_version 174015 (0.0007) [2023-12-27 06:51:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 89038848. Throughput: 0: 10239.9, 1: 10415.5. Samples: 89048316. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:32,363][19170] Avg episode reward: [(0, '25.850'), (1, '1000.000')] [2023-12-27 06:51:32,450][20381] Updated weights for policy 1, policy_version 173696 (0.0007) [2023-12-27 06:51:32,507][20381] Updated weights for policy 1, policy_version 173706 (0.0008) [2023-12-27 06:51:32,558][20381] Updated weights for policy 1, policy_version 173716 (0.0007) [2023-12-27 06:51:32,886][20347] Updated weights for policy 0, policy_version 174025 (0.0008) [2023-12-27 06:51:32,936][20347] Updated weights for policy 0, policy_version 174035 (0.0007) [2023-12-27 06:51:32,986][20347] Updated weights for policy 0, policy_version 174045 (0.0005) [2023-12-27 06:51:33,033][20347] Updated weights for policy 0, policy_version 174055 (0.0006) [2023-12-27 06:51:33,264][20381] Updated weights for policy 1, policy_version 173726 (0.0007) [2023-12-27 06:51:33,316][20381] Updated weights for policy 1, policy_version 173736 (0.0007) [2023-12-27 06:51:33,364][20381] Updated weights for policy 1, policy_version 173746 (0.0007) [2023-12-27 06:51:33,694][20347] Updated weights for policy 0, policy_version 174065 (0.0008) [2023-12-27 06:51:33,754][20347] Updated weights for policy 0, policy_version 174075 (0.0007) [2023-12-27 06:51:33,815][20347] Updated weights for policy 0, policy_version 174085 (0.0006) [2023-12-27 06:51:34,000][20381] Updated weights for policy 1, policy_version 173756 (0.0007) [2023-12-27 06:51:34,051][20381] Updated weights for policy 1, policy_version 173766 (0.0005) [2023-12-27 06:51:34,102][20381] Updated weights for policy 1, policy_version 173776 (0.0006) [2023-12-27 06:51:34,377][20347] Updated weights for policy 0, policy_version 174095 (0.0007) [2023-12-27 06:51:34,439][20347] Updated weights for policy 0, policy_version 174105 (0.0007) [2023-12-27 06:51:34,501][20347] Updated weights for policy 0, policy_version 174115 (0.0007) [2023-12-27 06:51:34,799][20381] Updated weights for policy 1, policy_version 173786 (0.0007) [2023-12-27 06:51:34,861][20381] Updated weights for policy 1, policy_version 173796 (0.0006) [2023-12-27 06:51:34,923][20381] Updated weights for policy 1, policy_version 173806 (0.0008) [2023-12-27 06:51:34,979][20381] Updated weights for policy 1, policy_version 173816 (0.0008) [2023-12-27 06:51:35,149][20347] Updated weights for policy 0, policy_version 174125 (0.0006) [2023-12-27 06:51:35,203][20347] Updated weights for policy 0, policy_version 174135 (0.0005) [2023-12-27 06:51:35,255][20347] Updated weights for policy 0, policy_version 174145 (0.0005) [2023-12-27 06:51:35,573][20381] Updated weights for policy 1, policy_version 173826 (0.0008) [2023-12-27 06:51:35,619][20381] Updated weights for policy 1, policy_version 173836 (0.0010) [2023-12-27 06:51:35,667][20381] Updated weights for policy 1, policy_version 173846 (0.0007) [2023-12-27 06:51:35,871][20347] Updated weights for policy 0, policy_version 174155 (0.0006) [2023-12-27 06:51:35,920][20347] Updated weights for policy 0, policy_version 174165 (0.0008) [2023-12-27 06:51:35,984][20347] Updated weights for policy 0, policy_version 174175 (0.0008) [2023-12-27 06:51:36,287][20381] Updated weights for policy 1, policy_version 173856 (0.0007) [2023-12-27 06:51:36,340][20381] Updated weights for policy 1, policy_version 173866 (0.0007) [2023-12-27 06:51:36,397][20381] Updated weights for policy 1, policy_version 173876 (0.0009) [2023-12-27 06:51:36,741][20347] Updated weights for policy 0, policy_version 174185 (0.0007) [2023-12-27 06:51:36,797][20347] Updated weights for policy 0, policy_version 174195 (0.0007) [2023-12-27 06:51:36,856][20347] Updated weights for policy 0, policy_version 174205 (0.0008) [2023-12-27 06:51:36,914][20347] Updated weights for policy 0, policy_version 174215 (0.0008) [2023-12-27 06:51:37,049][20381] Updated weights for policy 1, policy_version 173886 (0.0007) [2023-12-27 06:51:37,105][20381] Updated weights for policy 1, policy_version 173896 (0.0008) [2023-12-27 06:51:37,165][20381] Updated weights for policy 1, policy_version 173906 (0.0007) [2023-12-27 06:51:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 89145344. Throughput: 0: 10284.0, 1: 10400.7. Samples: 89110548. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:37,363][19170] Avg episode reward: [(0, '27.590'), (1, '990.420')] [2023-12-27 06:51:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000173912_44531712.pth... [2023-12-27 06:51:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000174216_44613632.pth... [2023-12-27 06:51:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000172664_44212224.pth [2023-12-27 06:51:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000173032_44310528.pth [2023-12-27 06:51:37,604][20347] Updated weights for policy 0, policy_version 174225 (0.0009) [2023-12-27 06:51:37,661][20347] Updated weights for policy 0, policy_version 174235 (0.0008) [2023-12-27 06:51:37,722][20347] Updated weights for policy 0, policy_version 174245 (0.0008) [2023-12-27 06:51:37,829][20381] Updated weights for policy 1, policy_version 173916 (0.0009) [2023-12-27 06:51:37,890][20381] Updated weights for policy 1, policy_version 173926 (0.0011) [2023-12-27 06:51:37,948][20381] Updated weights for policy 1, policy_version 173936 (0.0010) [2023-12-27 06:51:38,438][20347] Updated weights for policy 0, policy_version 174255 (0.0008) [2023-12-27 06:51:38,499][20347] Updated weights for policy 0, policy_version 174265 (0.0007) [2023-12-27 06:51:38,560][20347] Updated weights for policy 0, policy_version 174275 (0.0007) [2023-12-27 06:51:38,636][20381] Updated weights for policy 1, policy_version 173946 (0.0009) [2023-12-27 06:51:38,697][20381] Updated weights for policy 1, policy_version 173956 (0.0007) [2023-12-27 06:51:38,760][20381] Updated weights for policy 1, policy_version 173966 (0.0007) [2023-12-27 06:51:38,816][20381] Updated weights for policy 1, policy_version 173976 (0.0008) [2023-12-27 06:51:39,275][20347] Updated weights for policy 0, policy_version 174285 (0.0007) [2023-12-27 06:51:39,328][20347] Updated weights for policy 0, policy_version 174295 (0.0007) [2023-12-27 06:51:39,393][20347] Updated weights for policy 0, policy_version 174305 (0.0008) [2023-12-27 06:51:39,473][20381] Updated weights for policy 1, policy_version 173986 (0.0007) [2023-12-27 06:51:39,526][20381] Updated weights for policy 1, policy_version 173996 (0.0008) [2023-12-27 06:51:39,580][20381] Updated weights for policy 1, policy_version 174006 (0.0008) [2023-12-27 06:51:40,127][20347] Updated weights for policy 0, policy_version 174315 (0.0008) [2023-12-27 06:51:40,173][20347] Updated weights for policy 0, policy_version 174325 (0.0010) [2023-12-27 06:51:40,236][20347] Updated weights for policy 0, policy_version 174335 (0.0008) [2023-12-27 06:51:40,257][20381] Updated weights for policy 1, policy_version 174016 (0.0006) [2023-12-27 06:51:40,316][20381] Updated weights for policy 1, policy_version 174026 (0.0006) [2023-12-27 06:51:40,380][20381] Updated weights for policy 1, policy_version 174036 (0.0005) [2023-12-27 06:51:40,904][20381] Updated weights for policy 1, policy_version 174046 (0.0007) [2023-12-27 06:51:40,956][20381] Updated weights for policy 1, policy_version 174056 (0.0007) [2023-12-27 06:51:41,001][20347] Updated weights for policy 0, policy_version 174345 (0.0007) [2023-12-27 06:51:41,007][20381] Updated weights for policy 1, policy_version 174066 (0.0007) [2023-12-27 06:51:41,061][20347] Updated weights for policy 0, policy_version 174355 (0.0007) [2023-12-27 06:51:41,119][20347] Updated weights for policy 0, policy_version 174365 (0.0006) [2023-12-27 06:51:41,178][20347] Updated weights for policy 0, policy_version 174375 (0.0007) [2023-12-27 06:51:41,826][20381] Updated weights for policy 1, policy_version 174076 (0.0008) [2023-12-27 06:51:41,889][20381] Updated weights for policy 1, policy_version 174086 (0.0009) [2023-12-27 06:51:41,919][20347] Updated weights for policy 0, policy_version 174385 (0.0008) [2023-12-27 06:51:41,952][20381] Updated weights for policy 1, policy_version 174096 (0.0007) [2023-12-27 06:51:41,973][20347] Updated weights for policy 0, policy_version 174395 (0.0006) [2023-12-27 06:51:42,034][20347] Updated weights for policy 0, policy_version 174405 (0.0008) [2023-12-27 06:51:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 89243648. Throughput: 0: 10182.0, 1: 10450.4. Samples: 89234008. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:42,363][19170] Avg episode reward: [(0, '26.940'), (1, '980.830')] [2023-12-27 06:51:42,626][20381] Updated weights for policy 1, policy_version 174106 (0.0005) [2023-12-27 06:51:42,684][20381] Updated weights for policy 1, policy_version 174116 (0.0006) [2023-12-27 06:51:42,747][20381] Updated weights for policy 1, policy_version 174126 (0.0006) [2023-12-27 06:51:42,790][20347] Updated weights for policy 0, policy_version 174415 (0.0008) [2023-12-27 06:51:42,804][20381] Updated weights for policy 1, policy_version 174136 (0.0006) [2023-12-27 06:51:42,850][20347] Updated weights for policy 0, policy_version 174425 (0.0007) [2023-12-27 06:51:42,910][20347] Updated weights for policy 0, policy_version 174435 (0.0008) [2023-12-27 06:51:43,459][20381] Updated weights for policy 1, policy_version 174146 (0.0005) [2023-12-27 06:51:43,515][20381] Updated weights for policy 1, policy_version 174156 (0.0005) [2023-12-27 06:51:43,568][20381] Updated weights for policy 1, policy_version 174166 (0.0005) [2023-12-27 06:51:43,611][20347] Updated weights for policy 0, policy_version 174445 (0.0008) [2023-12-27 06:51:43,667][20347] Updated weights for policy 0, policy_version 174455 (0.0007) [2023-12-27 06:51:43,725][20347] Updated weights for policy 0, policy_version 174465 (0.0007) [2023-12-27 06:51:44,181][20381] Updated weights for policy 1, policy_version 174176 (0.0008) [2023-12-27 06:51:44,250][20381] Updated weights for policy 1, policy_version 174186 (0.0008) [2023-12-27 06:51:44,313][20381] Updated weights for policy 1, policy_version 174196 (0.0008) [2023-12-27 06:51:44,411][20347] Updated weights for policy 0, policy_version 174475 (0.0007) [2023-12-27 06:51:44,460][20347] Updated weights for policy 0, policy_version 174485 (0.0007) [2023-12-27 06:51:44,512][20347] Updated weights for policy 0, policy_version 174495 (0.0008) [2023-12-27 06:51:44,906][20381] Updated weights for policy 1, policy_version 174206 (0.0007) [2023-12-27 06:51:44,961][20381] Updated weights for policy 1, policy_version 174216 (0.0007) [2023-12-27 06:51:45,008][20381] Updated weights for policy 1, policy_version 174226 (0.0008) [2023-12-27 06:51:45,281][20347] Updated weights for policy 0, policy_version 174505 (0.0008) [2023-12-27 06:51:45,342][20347] Updated weights for policy 0, policy_version 174515 (0.0008) [2023-12-27 06:51:45,400][20347] Updated weights for policy 0, policy_version 174525 (0.0008) [2023-12-27 06:51:45,456][20347] Updated weights for policy 0, policy_version 174535 (0.0007) [2023-12-27 06:51:45,676][20381] Updated weights for policy 1, policy_version 174236 (0.0008) [2023-12-27 06:51:45,736][20381] Updated weights for policy 1, policy_version 174246 (0.0008) [2023-12-27 06:51:45,791][20381] Updated weights for policy 1, policy_version 174256 (0.0008) [2023-12-27 06:51:46,145][20347] Updated weights for policy 0, policy_version 174545 (0.0006) [2023-12-27 06:51:46,195][20347] Updated weights for policy 0, policy_version 174555 (0.0005) [2023-12-27 06:51:46,239][20347] Updated weights for policy 0, policy_version 174565 (0.0005) [2023-12-27 06:51:46,498][20381] Updated weights for policy 1, policy_version 174266 (0.0007) [2023-12-27 06:51:46,560][20381] Updated weights for policy 1, policy_version 174276 (0.0009) [2023-12-27 06:51:46,606][20381] Updated weights for policy 1, policy_version 174286 (0.0008) [2023-12-27 06:51:46,654][20381] Updated weights for policy 1, policy_version 174296 (0.0008) [2023-12-27 06:51:46,852][20347] Updated weights for policy 0, policy_version 174575 (0.0005) [2023-12-27 06:51:46,915][20347] Updated weights for policy 0, policy_version 174585 (0.0005) [2023-12-27 06:51:46,990][20347] Updated weights for policy 0, policy_version 174595 (0.0005) [2023-12-27 06:51:47,282][20381] Updated weights for policy 1, policy_version 174306 (0.0008) [2023-12-27 06:51:47,337][20381] Updated weights for policy 1, policy_version 174316 (0.0007) [2023-12-27 06:51:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 89341952. Throughput: 0: 10084.3, 1: 10506.1. Samples: 89356124. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:47,363][19170] Avg episode reward: [(0, '27.110'), (1, '990.410')] [2023-12-27 06:51:47,382][20381] Updated weights for policy 1, policy_version 174326 (0.0008) [2023-12-27 06:51:47,571][20347] Updated weights for policy 0, policy_version 174605 (0.0007) [2023-12-27 06:51:47,624][20347] Updated weights for policy 0, policy_version 174615 (0.0008) [2023-12-27 06:51:47,673][20347] Updated weights for policy 0, policy_version 174625 (0.0008) [2023-12-27 06:51:48,081][20381] Updated weights for policy 1, policy_version 174336 (0.0008) [2023-12-27 06:51:48,134][20381] Updated weights for policy 1, policy_version 174346 (0.0006) [2023-12-27 06:51:48,192][20381] Updated weights for policy 1, policy_version 174356 (0.0005) [2023-12-27 06:51:48,345][20347] Updated weights for policy 0, policy_version 174635 (0.0008) [2023-12-27 06:51:48,409][20347] Updated weights for policy 0, policy_version 174645 (0.0008) [2023-12-27 06:51:48,471][20347] Updated weights for policy 0, policy_version 174655 (0.0008) [2023-12-27 06:51:48,781][20381] Updated weights for policy 1, policy_version 174366 (0.0006) [2023-12-27 06:51:48,834][20381] Updated weights for policy 1, policy_version 174376 (0.0007) [2023-12-27 06:51:48,889][20381] Updated weights for policy 1, policy_version 174386 (0.0008) [2023-12-27 06:51:49,127][20347] Updated weights for policy 0, policy_version 174665 (0.0009) [2023-12-27 06:51:49,178][20347] Updated weights for policy 0, policy_version 174675 (0.0010) [2023-12-27 06:51:49,244][20347] Updated weights for policy 0, policy_version 174685 (0.0011) [2023-12-27 06:51:49,304][20347] Updated weights for policy 0, policy_version 174695 (0.0009) [2023-12-27 06:51:49,579][20381] Updated weights for policy 1, policy_version 174396 (0.0009) [2023-12-27 06:51:49,633][20381] Updated weights for policy 1, policy_version 174406 (0.0007) [2023-12-27 06:51:49,688][20381] Updated weights for policy 1, policy_version 174416 (0.0006) [2023-12-27 06:51:50,005][20347] Updated weights for policy 0, policy_version 174705 (0.0006) [2023-12-27 06:51:50,061][20347] Updated weights for policy 0, policy_version 174715 (0.0008) [2023-12-27 06:51:50,123][20347] Updated weights for policy 0, policy_version 174725 (0.0008) [2023-12-27 06:51:50,375][20381] Updated weights for policy 1, policy_version 174426 (0.0007) [2023-12-27 06:51:50,430][20381] Updated weights for policy 1, policy_version 174436 (0.0008) [2023-12-27 06:51:50,486][20381] Updated weights for policy 1, policy_version 174446 (0.0008) [2023-12-27 06:51:50,535][20381] Updated weights for policy 1, policy_version 174456 (0.0007) [2023-12-27 06:51:50,796][20347] Updated weights for policy 0, policy_version 174735 (0.0007) [2023-12-27 06:51:50,859][20347] Updated weights for policy 0, policy_version 174745 (0.0008) [2023-12-27 06:51:50,918][20347] Updated weights for policy 0, policy_version 174755 (0.0009) [2023-12-27 06:51:51,302][20381] Updated weights for policy 1, policy_version 174466 (0.0006) [2023-12-27 06:51:51,366][20381] Updated weights for policy 1, policy_version 174476 (0.0009) [2023-12-27 06:51:51,435][20381] Updated weights for policy 1, policy_version 174486 (0.0008) [2023-12-27 06:51:51,583][20347] Updated weights for policy 0, policy_version 174765 (0.0010) [2023-12-27 06:51:51,644][20347] Updated weights for policy 0, policy_version 174775 (0.0010) [2023-12-27 06:51:51,715][20347] Updated weights for policy 0, policy_version 174785 (0.0009) [2023-12-27 06:51:52,044][20381] Updated weights for policy 1, policy_version 174496 (0.0010) [2023-12-27 06:51:52,103][20381] Updated weights for policy 1, policy_version 174506 (0.0011) [2023-12-27 06:51:52,161][20381] Updated weights for policy 1, policy_version 174516 (0.0008) [2023-12-27 06:51:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 89448448. Throughput: 0: 10073.9, 1: 10499.5. Samples: 89418408. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:52,363][19170] Avg episode reward: [(0, '27.910'), (1, '990.360')] [2023-12-27 06:51:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000174520_44687360.pth... [2023-12-27 06:51:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000173304_44376064.pth [2023-12-27 06:51:52,375][20347] Updated weights for policy 0, policy_version 174795 (0.0007) [2023-12-27 06:51:52,443][20347] Updated weights for policy 0, policy_version 174805 (0.0006) [2023-12-27 06:51:52,498][20347] Updated weights for policy 0, policy_version 174815 (0.0010) [2023-12-27 06:51:52,545][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000174824_44769280.pth... [2023-12-27 06:51:52,550][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000173608_44457984.pth [2023-12-27 06:51:52,853][20381] Updated weights for policy 1, policy_version 174526 (0.0007) [2023-12-27 06:51:52,911][20381] Updated weights for policy 1, policy_version 174536 (0.0007) [2023-12-27 06:51:52,975][20381] Updated weights for policy 1, policy_version 174546 (0.0008) [2023-12-27 06:51:53,024][20347] Updated weights for policy 0, policy_version 174825 (0.0006) [2023-12-27 06:51:53,083][20347] Updated weights for policy 0, policy_version 174835 (0.0005) [2023-12-27 06:51:53,134][20347] Updated weights for policy 0, policy_version 174845 (0.0005) [2023-12-27 06:51:53,189][20347] Updated weights for policy 0, policy_version 174855 (0.0006) [2023-12-27 06:51:53,568][20381] Updated weights for policy 1, policy_version 174556 (0.0009) [2023-12-27 06:51:53,630][20381] Updated weights for policy 1, policy_version 174566 (0.0010) [2023-12-27 06:51:53,682][20381] Updated weights for policy 1, policy_version 174576 (0.0010) [2023-12-27 06:51:53,813][20347] Updated weights for policy 0, policy_version 174865 (0.0006) [2023-12-27 06:51:53,877][20347] Updated weights for policy 0, policy_version 174875 (0.0007) [2023-12-27 06:51:53,941][20347] Updated weights for policy 0, policy_version 174885 (0.0006) [2023-12-27 06:51:54,363][20381] Updated weights for policy 1, policy_version 174586 (0.0010) [2023-12-27 06:51:54,423][20381] Updated weights for policy 1, policy_version 174596 (0.0006) [2023-12-27 06:51:54,509][20381] Updated weights for policy 1, policy_version 174606 (0.0007) [2023-12-27 06:51:54,567][20381] Updated weights for policy 1, policy_version 174616 (0.0010) [2023-12-27 06:51:54,609][20347] Updated weights for policy 0, policy_version 174895 (0.0006) [2023-12-27 06:51:54,674][20347] Updated weights for policy 0, policy_version 174905 (0.0005) [2023-12-27 06:51:54,731][20347] Updated weights for policy 0, policy_version 174915 (0.0007) [2023-12-27 06:51:55,197][20381] Updated weights for policy 1, policy_version 174626 (0.0011) [2023-12-27 06:51:55,257][20381] Updated weights for policy 1, policy_version 174636 (0.0010) [2023-12-27 06:51:55,315][20381] Updated weights for policy 1, policy_version 174646 (0.0010) [2023-12-27 06:51:55,398][20347] Updated weights for policy 0, policy_version 174925 (0.0007) [2023-12-27 06:51:55,441][20347] Updated weights for policy 0, policy_version 174935 (0.0007) [2023-12-27 06:51:55,496][20347] Updated weights for policy 0, policy_version 174945 (0.0006) [2023-12-27 06:51:56,006][20381] Updated weights for policy 1, policy_version 174656 (0.0011) [2023-12-27 06:51:56,063][20381] Updated weights for policy 1, policy_version 174666 (0.0011) [2023-12-27 06:51:56,121][20381] Updated weights for policy 1, policy_version 174676 (0.0011) [2023-12-27 06:51:56,175][20347] Updated weights for policy 0, policy_version 174955 (0.0007) [2023-12-27 06:51:56,223][20347] Updated weights for policy 0, policy_version 174965 (0.0010) [2023-12-27 06:51:56,271][20347] Updated weights for policy 0, policy_version 174975 (0.0006) [2023-12-27 06:51:56,792][20381] Updated weights for policy 1, policy_version 174686 (0.0011) [2023-12-27 06:51:56,843][20381] Updated weights for policy 1, policy_version 174696 (0.0010) [2023-12-27 06:51:56,856][20347] Updated weights for policy 0, policy_version 174985 (0.0005) [2023-12-27 06:51:56,892][20381] Updated weights for policy 1, policy_version 174706 (0.0010) [2023-12-27 06:51:56,904][20347] Updated weights for policy 0, policy_version 174995 (0.0005) [2023-12-27 06:51:56,950][20347] Updated weights for policy 0, policy_version 175005 (0.0006) [2023-12-27 06:51:56,997][20347] Updated weights for policy 0, policy_version 175015 (0.0007) [2023-12-27 06:51:57,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 89554944. Throughput: 0: 10100.4, 1: 10511.0. Samples: 89542944. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:51:57,363][19170] Avg episode reward: [(0, '27.680'), (1, '990.360')] [2023-12-27 06:51:57,568][20381] Updated weights for policy 1, policy_version 174716 (0.0008) [2023-12-27 06:51:57,619][20381] Updated weights for policy 1, policy_version 174726 (0.0005) [2023-12-27 06:51:57,651][20347] Updated weights for policy 0, policy_version 175025 (0.0008) [2023-12-27 06:51:57,665][20381] Updated weights for policy 1, policy_version 174736 (0.0005) [2023-12-27 06:51:57,699][20347] Updated weights for policy 0, policy_version 175035 (0.0007) [2023-12-27 06:51:57,753][20347] Updated weights for policy 0, policy_version 175045 (0.0005) [2023-12-27 06:51:58,325][20381] Updated weights for policy 1, policy_version 174746 (0.0007) [2023-12-27 06:51:58,379][20347] Updated weights for policy 0, policy_version 175055 (0.0007) [2023-12-27 06:51:58,394][20381] Updated weights for policy 1, policy_version 174756 (0.0008) [2023-12-27 06:51:58,444][20347] Updated weights for policy 0, policy_version 175065 (0.0010) [2023-12-27 06:51:58,459][20381] Updated weights for policy 1, policy_version 174766 (0.0007) [2023-12-27 06:51:58,500][20347] Updated weights for policy 0, policy_version 175075 (0.0011) [2023-12-27 06:51:58,520][20381] Updated weights for policy 1, policy_version 174776 (0.0009) [2023-12-27 06:51:59,240][20347] Updated weights for policy 0, policy_version 175085 (0.0010) [2023-12-27 06:51:59,281][20381] Updated weights for policy 1, policy_version 174786 (0.0008) [2023-12-27 06:51:59,293][20347] Updated weights for policy 0, policy_version 175095 (0.0008) [2023-12-27 06:51:59,343][20381] Updated weights for policy 1, policy_version 174796 (0.0008) [2023-12-27 06:51:59,360][20347] Updated weights for policy 0, policy_version 175105 (0.0007) [2023-12-27 06:51:59,411][20381] Updated weights for policy 1, policy_version 174806 (0.0008) [2023-12-27 06:52:00,064][20347] Updated weights for policy 0, policy_version 175115 (0.0008) [2023-12-27 06:52:00,085][20381] Updated weights for policy 1, policy_version 174816 (0.0007) [2023-12-27 06:52:00,125][20347] Updated weights for policy 0, policy_version 175125 (0.0008) [2023-12-27 06:52:00,137][20381] Updated weights for policy 1, policy_version 174826 (0.0006) [2023-12-27 06:52:00,187][20347] Updated weights for policy 0, policy_version 175135 (0.0006) [2023-12-27 06:52:00,193][20381] Updated weights for policy 1, policy_version 174836 (0.0007) [2023-12-27 06:52:00,762][20381] Updated weights for policy 1, policy_version 174846 (0.0006) [2023-12-27 06:52:00,785][20347] Updated weights for policy 0, policy_version 175145 (0.0005) [2023-12-27 06:52:00,812][20381] Updated weights for policy 1, policy_version 174856 (0.0005) [2023-12-27 06:52:00,844][20347] Updated weights for policy 0, policy_version 175155 (0.0005) [2023-12-27 06:52:00,860][20381] Updated weights for policy 1, policy_version 174866 (0.0007) [2023-12-27 06:52:00,898][20347] Updated weights for policy 0, policy_version 175165 (0.0006) [2023-12-27 06:52:00,953][20347] Updated weights for policy 0, policy_version 175175 (0.0006) [2023-12-27 06:52:01,526][20381] Updated weights for policy 1, policy_version 174876 (0.0007) [2023-12-27 06:52:01,583][20381] Updated weights for policy 1, policy_version 174886 (0.0007) [2023-12-27 06:52:01,639][20381] Updated weights for policy 1, policy_version 174896 (0.0008) [2023-12-27 06:52:01,673][20347] Updated weights for policy 0, policy_version 175185 (0.0007) [2023-12-27 06:52:01,735][20347] Updated weights for policy 0, policy_version 175195 (0.0007) [2023-12-27 06:52:01,787][20347] Updated weights for policy 0, policy_version 175205 (0.0005) [2023-12-27 06:52:02,313][20381] Updated weights for policy 1, policy_version 174906 (0.0008) [2023-12-27 06:52:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 89653248. Throughput: 0: 10199.1, 1: 10450.3. Samples: 89667704. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:02,363][19170] Avg episode reward: [(0, '28.930'), (1, '1000.000')] [2023-12-27 06:52:02,374][20381] Updated weights for policy 1, policy_version 174916 (0.0007) [2023-12-27 06:52:02,399][20347] Updated weights for policy 0, policy_version 175215 (0.0007) [2023-12-27 06:52:02,421][20381] Updated weights for policy 1, policy_version 174926 (0.0005) [2023-12-27 06:52:02,457][20347] Updated weights for policy 0, policy_version 175225 (0.0006) [2023-12-27 06:52:02,476][20381] Updated weights for policy 1, policy_version 174936 (0.0007) [2023-12-27 06:52:02,512][20347] Updated weights for policy 0, policy_version 175235 (0.0006) [2023-12-27 06:52:03,080][20347] Updated weights for policy 0, policy_version 175245 (0.0005) [2023-12-27 06:52:03,147][20347] Updated weights for policy 0, policy_version 175255 (0.0005) [2023-12-27 06:52:03,201][20347] Updated weights for policy 0, policy_version 175265 (0.0005) [2023-12-27 06:52:03,242][20381] Updated weights for policy 1, policy_version 174946 (0.0007) [2023-12-27 06:52:03,301][20381] Updated weights for policy 1, policy_version 174956 (0.0008) [2023-12-27 06:52:03,350][20381] Updated weights for policy 1, policy_version 174966 (0.0008) [2023-12-27 06:52:03,814][20347] Updated weights for policy 0, policy_version 175275 (0.0007) [2023-12-27 06:52:03,870][20347] Updated weights for policy 0, policy_version 175285 (0.0007) [2023-12-27 06:52:03,929][20347] Updated weights for policy 0, policy_version 175295 (0.0007) [2023-12-27 06:52:04,028][20381] Updated weights for policy 1, policy_version 174976 (0.0008) [2023-12-27 06:52:04,087][20381] Updated weights for policy 1, policy_version 174986 (0.0006) [2023-12-27 06:52:04,145][20381] Updated weights for policy 1, policy_version 174996 (0.0007) [2023-12-27 06:52:04,622][20347] Updated weights for policy 0, policy_version 175305 (0.0007) [2023-12-27 06:52:04,687][20347] Updated weights for policy 0, policy_version 175315 (0.0008) [2023-12-27 06:52:04,737][20347] Updated weights for policy 0, policy_version 175325 (0.0007) [2023-12-27 06:52:04,784][20347] Updated weights for policy 0, policy_version 175335 (0.0007) [2023-12-27 06:52:04,806][20381] Updated weights for policy 1, policy_version 175006 (0.0007) [2023-12-27 06:52:04,863][20381] Updated weights for policy 1, policy_version 175016 (0.0008) [2023-12-27 06:52:04,912][20381] Updated weights for policy 1, policy_version 175026 (0.0008) [2023-12-27 06:52:05,447][20347] Updated weights for policy 0, policy_version 175345 (0.0006) [2023-12-27 06:52:05,493][20347] Updated weights for policy 0, policy_version 175355 (0.0005) [2023-12-27 06:52:05,549][20347] Updated weights for policy 0, policy_version 175365 (0.0005) [2023-12-27 06:52:05,647][20381] Updated weights for policy 1, policy_version 175036 (0.0008) [2023-12-27 06:52:05,698][20381] Updated weights for policy 1, policy_version 175046 (0.0008) [2023-12-27 06:52:05,751][20381] Updated weights for policy 1, policy_version 175056 (0.0006) [2023-12-27 06:52:06,172][20347] Updated weights for policy 0, policy_version 175375 (0.0009) [2023-12-27 06:52:06,232][20347] Updated weights for policy 0, policy_version 175385 (0.0011) [2023-12-27 06:52:06,289][20347] Updated weights for policy 0, policy_version 175395 (0.0011) [2023-12-27 06:52:06,450][20381] Updated weights for policy 1, policy_version 175066 (0.0006) [2023-12-27 06:52:06,504][20381] Updated weights for policy 1, policy_version 175076 (0.0008) [2023-12-27 06:52:06,563][20381] Updated weights for policy 1, policy_version 175086 (0.0010) [2023-12-27 06:52:06,622][20381] Updated weights for policy 1, policy_version 175096 (0.0010) [2023-12-27 06:52:06,995][20347] Updated weights for policy 0, policy_version 175405 (0.0010) [2023-12-27 06:52:07,060][20347] Updated weights for policy 0, policy_version 175415 (0.0010) [2023-12-27 06:52:07,121][20347] Updated weights for policy 0, policy_version 175425 (0.0010) [2023-12-27 06:52:07,312][20381] Updated weights for policy 1, policy_version 175106 (0.0011) [2023-12-27 06:52:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 89759744. Throughput: 0: 10242.1, 1: 10396.0. Samples: 89729864. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:07,363][20381] Updated weights for policy 1, policy_version 175116 (0.0010) [2023-12-27 06:52:07,363][19170] Avg episode reward: [(0, '27.660'), (1, '990.490')] [2023-12-27 06:52:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000175432_44924928.pth... [2023-12-27 06:52:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000174216_44613632.pth [2023-12-27 06:52:07,417][20381] Updated weights for policy 1, policy_version 175126 (0.0010) [2023-12-27 06:52:07,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000175128_44843008.pth... [2023-12-27 06:52:07,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000173912_44531712.pth [2023-12-27 06:52:07,780][20347] Updated weights for policy 0, policy_version 175435 (0.0010) [2023-12-27 06:52:07,841][20347] Updated weights for policy 0, policy_version 175445 (0.0010) [2023-12-27 06:52:07,896][20347] Updated weights for policy 0, policy_version 175455 (0.0010) [2023-12-27 06:52:08,076][20381] Updated weights for policy 1, policy_version 175136 (0.0008) [2023-12-27 06:52:08,128][20381] Updated weights for policy 1, policy_version 175146 (0.0008) [2023-12-27 06:52:08,185][20381] Updated weights for policy 1, policy_version 175156 (0.0007) [2023-12-27 06:52:08,578][20347] Updated weights for policy 0, policy_version 175465 (0.0010) [2023-12-27 06:52:08,623][20347] Updated weights for policy 0, policy_version 175475 (0.0010) [2023-12-27 06:52:08,672][20347] Updated weights for policy 0, policy_version 175485 (0.0010) [2023-12-27 06:52:08,737][20347] Updated weights for policy 0, policy_version 175495 (0.0010) [2023-12-27 06:52:08,803][20381] Updated weights for policy 1, policy_version 175166 (0.0006) [2023-12-27 06:52:08,863][20381] Updated weights for policy 1, policy_version 175176 (0.0005) [2023-12-27 06:52:08,917][20381] Updated weights for policy 1, policy_version 175186 (0.0005) [2023-12-27 06:52:09,473][20347] Updated weights for policy 0, policy_version 175505 (0.0008) [2023-12-27 06:52:09,533][20347] Updated weights for policy 0, policy_version 175515 (0.0007) [2023-12-27 06:52:09,554][20381] Updated weights for policy 1, policy_version 175196 (0.0009) [2023-12-27 06:52:09,592][20347] Updated weights for policy 0, policy_version 175525 (0.0008) [2023-12-27 06:52:09,608][20381] Updated weights for policy 1, policy_version 175206 (0.0011) [2023-12-27 06:52:09,665][20381] Updated weights for policy 1, policy_version 175216 (0.0011) [2023-12-27 06:52:10,302][20347] Updated weights for policy 0, policy_version 175535 (0.0010) [2023-12-27 06:52:10,340][20381] Updated weights for policy 1, policy_version 175226 (0.0011) [2023-12-27 06:52:10,363][20347] Updated weights for policy 0, policy_version 175545 (0.0011) [2023-12-27 06:52:10,397][20381] Updated weights for policy 1, policy_version 175236 (0.0011) [2023-12-27 06:52:10,420][20347] Updated weights for policy 0, policy_version 175555 (0.0011) [2023-12-27 06:52:10,457][20381] Updated weights for policy 1, policy_version 175246 (0.0011) [2023-12-27 06:52:10,516][20381] Updated weights for policy 1, policy_version 175256 (0.0011) [2023-12-27 06:52:11,082][20347] Updated weights for policy 0, policy_version 175565 (0.0009) [2023-12-27 06:52:11,150][20347] Updated weights for policy 0, policy_version 175575 (0.0012) [2023-12-27 06:52:11,176][20381] Updated weights for policy 1, policy_version 175266 (0.0009) [2023-12-27 06:52:11,210][20347] Updated weights for policy 0, policy_version 175585 (0.0007) [2023-12-27 06:52:11,240][20381] Updated weights for policy 1, policy_version 175276 (0.0010) [2023-12-27 06:52:11,304][20381] Updated weights for policy 1, policy_version 175286 (0.0008) [2023-12-27 06:52:11,935][20347] Updated weights for policy 0, policy_version 175595 (0.0008) [2023-12-27 06:52:11,996][20347] Updated weights for policy 0, policy_version 175605 (0.0008) [2023-12-27 06:52:12,023][20381] Updated weights for policy 1, policy_version 175296 (0.0006) [2023-12-27 06:52:12,055][20347] Updated weights for policy 0, policy_version 175615 (0.0006) [2023-12-27 06:52:12,084][20381] Updated weights for policy 1, policy_version 175306 (0.0006) [2023-12-27 06:52:12,136][20381] Updated weights for policy 1, policy_version 175316 (0.0006) [2023-12-27 06:52:12,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.2, 300 sec: 20743.8). Total num frames: 89866240. Throughput: 0: 10270.0, 1: 10370.6. Samples: 89853212. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:12,363][19170] Avg episode reward: [(0, '28.650'), (1, '986.570')] [2023-12-27 06:52:12,767][20381] Updated weights for policy 1, policy_version 175326 (0.0008) [2023-12-27 06:52:12,778][20347] Updated weights for policy 0, policy_version 175625 (0.0007) [2023-12-27 06:52:12,827][20381] Updated weights for policy 1, policy_version 175336 (0.0009) [2023-12-27 06:52:12,828][20347] Updated weights for policy 0, policy_version 175635 (0.0007) [2023-12-27 06:52:12,882][20381] Updated weights for policy 1, policy_version 175346 (0.0010) [2023-12-27 06:52:12,886][20347] Updated weights for policy 0, policy_version 175645 (0.0009) [2023-12-27 06:52:12,943][20347] Updated weights for policy 0, policy_version 175655 (0.0008) [2023-12-27 06:52:13,499][20381] Updated weights for policy 1, policy_version 175356 (0.0008) [2023-12-27 06:52:13,549][20381] Updated weights for policy 1, policy_version 175366 (0.0008) [2023-12-27 06:52:13,599][20381] Updated weights for policy 1, policy_version 175376 (0.0008) [2023-12-27 06:52:13,670][20347] Updated weights for policy 0, policy_version 175665 (0.0008) [2023-12-27 06:52:13,734][20347] Updated weights for policy 0, policy_version 175675 (0.0005) [2023-12-27 06:52:13,797][20347] Updated weights for policy 0, policy_version 175685 (0.0006) [2023-12-27 06:52:14,186][20381] Updated weights for policy 1, policy_version 175386 (0.0007) [2023-12-27 06:52:14,243][20381] Updated weights for policy 1, policy_version 175396 (0.0005) [2023-12-27 06:52:14,305][20381] Updated weights for policy 1, policy_version 175406 (0.0005) [2023-12-27 06:52:14,361][20381] Updated weights for policy 1, policy_version 175416 (0.0006) [2023-12-27 06:52:14,410][20347] Updated weights for policy 0, policy_version 175695 (0.0008) [2023-12-27 06:52:14,494][20347] Updated weights for policy 0, policy_version 175705 (0.0009) [2023-12-27 06:52:14,546][20347] Updated weights for policy 0, policy_version 175715 (0.0010) [2023-12-27 06:52:15,050][20381] Updated weights for policy 1, policy_version 175426 (0.0009) [2023-12-27 06:52:15,107][20381] Updated weights for policy 1, policy_version 175436 (0.0009) [2023-12-27 06:52:15,171][20381] Updated weights for policy 1, policy_version 175446 (0.0007) [2023-12-27 06:52:15,201][20347] Updated weights for policy 0, policy_version 175725 (0.0008) [2023-12-27 06:52:15,260][20347] Updated weights for policy 0, policy_version 175735 (0.0005) [2023-12-27 06:52:15,319][20347] Updated weights for policy 0, policy_version 175745 (0.0006) [2023-12-27 06:52:15,861][20381] Updated weights for policy 1, policy_version 175456 (0.0007) [2023-12-27 06:52:15,865][20347] Updated weights for policy 0, policy_version 175755 (0.0006) [2023-12-27 06:52:15,915][20347] Updated weights for policy 0, policy_version 175765 (0.0005) [2023-12-27 06:52:15,916][20381] Updated weights for policy 1, policy_version 175466 (0.0008) [2023-12-27 06:52:15,963][20381] Updated weights for policy 1, policy_version 175476 (0.0008) [2023-12-27 06:52:15,966][20347] Updated weights for policy 0, policy_version 175775 (0.0005) [2023-12-27 06:52:16,610][20381] Updated weights for policy 1, policy_version 175486 (0.0006) [2023-12-27 06:52:16,631][20347] Updated weights for policy 0, policy_version 175785 (0.0007) [2023-12-27 06:52:16,659][20381] Updated weights for policy 1, policy_version 175496 (0.0005) [2023-12-27 06:52:16,682][20347] Updated weights for policy 0, policy_version 175795 (0.0010) [2023-12-27 06:52:16,712][20381] Updated weights for policy 1, policy_version 175506 (0.0007) [2023-12-27 06:52:16,734][20347] Updated weights for policy 0, policy_version 175805 (0.0010) [2023-12-27 06:52:16,792][20347] Updated weights for policy 0, policy_version 175815 (0.0010) [2023-12-27 06:52:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.7, 300 sec: 20743.8). Total num frames: 89964544. Throughput: 0: 10285.1, 1: 10383.6. Samples: 89978408. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:17,363][19170] Avg episode reward: [(0, '30.160'), (1, '996.080')] [2023-12-27 06:52:17,378][20381] Updated weights for policy 1, policy_version 175516 (0.0008) [2023-12-27 06:52:17,433][20381] Updated weights for policy 1, policy_version 175526 (0.0008) [2023-12-27 06:52:17,458][20347] Updated weights for policy 0, policy_version 175825 (0.0008) [2023-12-27 06:52:17,483][20381] Updated weights for policy 1, policy_version 175536 (0.0006) [2023-12-27 06:52:17,514][20347] Updated weights for policy 0, policy_version 175835 (0.0006) [2023-12-27 06:52:17,573][20347] Updated weights for policy 0, policy_version 175845 (0.0008) [2023-12-27 06:52:18,043][20381] Updated weights for policy 1, policy_version 175546 (0.0006) [2023-12-27 06:52:18,099][20381] Updated weights for policy 1, policy_version 175556 (0.0005) [2023-12-27 06:52:18,159][20381] Updated weights for policy 1, policy_version 175566 (0.0005) [2023-12-27 06:52:18,196][20347] Updated weights for policy 0, policy_version 175855 (0.0007) [2023-12-27 06:52:18,207][20381] Updated weights for policy 1, policy_version 175576 (0.0006) [2023-12-27 06:52:18,248][20347] Updated weights for policy 0, policy_version 175865 (0.0007) [2023-12-27 06:52:18,294][20347] Updated weights for policy 0, policy_version 175875 (0.0009) [2023-12-27 06:52:18,821][20381] Updated weights for policy 1, policy_version 175586 (0.0006) [2023-12-27 06:52:18,886][20381] Updated weights for policy 1, policy_version 175596 (0.0007) [2023-12-27 06:52:18,943][20381] Updated weights for policy 1, policy_version 175606 (0.0007) [2023-12-27 06:52:19,047][20347] Updated weights for policy 0, policy_version 175885 (0.0007) [2023-12-27 06:52:19,098][20347] Updated weights for policy 0, policy_version 175895 (0.0007) [2023-12-27 06:52:19,148][20347] Updated weights for policy 0, policy_version 175905 (0.0007) [2023-12-27 06:52:19,635][20381] Updated weights for policy 1, policy_version 175616 (0.0007) [2023-12-27 06:52:19,687][20381] Updated weights for policy 1, policy_version 175626 (0.0007) [2023-12-27 06:52:19,742][20381] Updated weights for policy 1, policy_version 175636 (0.0007) [2023-12-27 06:52:19,863][20347] Updated weights for policy 0, policy_version 175915 (0.0008) [2023-12-27 06:52:19,926][20347] Updated weights for policy 0, policy_version 175925 (0.0011) [2023-12-27 06:52:19,983][20347] Updated weights for policy 0, policy_version 175935 (0.0011) [2023-12-27 06:52:20,528][20381] Updated weights for policy 1, policy_version 175646 (0.0010) [2023-12-27 06:52:20,589][20381] Updated weights for policy 1, policy_version 175656 (0.0011) [2023-12-27 06:52:20,649][20381] Updated weights for policy 1, policy_version 175666 (0.0011) [2023-12-27 06:52:20,680][20347] Updated weights for policy 0, policy_version 175945 (0.0010) [2023-12-27 06:52:20,739][20347] Updated weights for policy 0, policy_version 175955 (0.0007) [2023-12-27 06:52:20,800][20347] Updated weights for policy 0, policy_version 175965 (0.0006) [2023-12-27 06:52:20,853][20347] Updated weights for policy 0, policy_version 175975 (0.0010) [2023-12-27 06:52:21,404][20381] Updated weights for policy 1, policy_version 175676 (0.0009) [2023-12-27 06:52:21,463][20381] Updated weights for policy 1, policy_version 175686 (0.0006) [2023-12-27 06:52:21,519][20381] Updated weights for policy 1, policy_version 175696 (0.0006) [2023-12-27 06:52:21,665][20347] Updated weights for policy 0, policy_version 175985 (0.0009) [2023-12-27 06:52:21,724][20347] Updated weights for policy 0, policy_version 175995 (0.0008) [2023-12-27 06:52:21,789][20347] Updated weights for policy 0, policy_version 176005 (0.0009) [2023-12-27 06:52:22,159][20381] Updated weights for policy 1, policy_version 175706 (0.0008) [2023-12-27 06:52:22,221][20381] Updated weights for policy 1, policy_version 175716 (0.0007) [2023-12-27 06:52:22,284][20381] Updated weights for policy 1, policy_version 175726 (0.0008) [2023-12-27 06:52:22,347][20381] Updated weights for policy 1, policy_version 175736 (0.0008) [2023-12-27 06:52:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 90071040. Throughput: 0: 10247.4, 1: 10414.8. Samples: 90040348. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:22,363][19170] Avg episode reward: [(0, '30.860'), (1, '1000.000')] [2023-12-27 06:52:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000175736_44998656.pth... [2023-12-27 06:52:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000176008_45072384.pth... [2023-12-27 06:52:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000174520_44687360.pth [2023-12-27 06:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000174824_44769280.pth [2023-12-27 06:52:22,473][20347] Updated weights for policy 0, policy_version 176015 (0.0008) [2023-12-27 06:52:22,533][20347] Updated weights for policy 0, policy_version 176025 (0.0007) [2023-12-27 06:52:22,593][20347] Updated weights for policy 0, policy_version 176035 (0.0007) [2023-12-27 06:52:23,082][20381] Updated weights for policy 1, policy_version 175746 (0.0006) [2023-12-27 06:52:23,138][20381] Updated weights for policy 1, policy_version 175756 (0.0007) [2023-12-27 06:52:23,190][20381] Updated weights for policy 1, policy_version 175766 (0.0007) [2023-12-27 06:52:23,192][20347] Updated weights for policy 0, policy_version 176045 (0.0006) [2023-12-27 06:52:23,247][20347] Updated weights for policy 0, policy_version 176055 (0.0007) [2023-12-27 06:52:23,299][20347] Updated weights for policy 0, policy_version 176065 (0.0008) [2023-12-27 06:52:23,880][20381] Updated weights for policy 1, policy_version 175776 (0.0008) [2023-12-27 06:52:23,936][20381] Updated weights for policy 1, policy_version 175786 (0.0008) [2023-12-27 06:52:23,980][20347] Updated weights for policy 0, policy_version 176075 (0.0008) [2023-12-27 06:52:23,996][20381] Updated weights for policy 1, policy_version 175796 (0.0007) [2023-12-27 06:52:24,044][20347] Updated weights for policy 0, policy_version 176085 (0.0006) [2023-12-27 06:52:24,104][20347] Updated weights for policy 0, policy_version 176095 (0.0008) [2023-12-27 06:52:24,668][20381] Updated weights for policy 1, policy_version 175806 (0.0009) [2023-12-27 06:52:24,712][20381] Updated weights for policy 1, policy_version 175816 (0.0007) [2023-12-27 06:52:24,749][20347] Updated weights for policy 0, policy_version 176105 (0.0008) [2023-12-27 06:52:24,761][20381] Updated weights for policy 1, policy_version 175826 (0.0007) [2023-12-27 06:52:24,797][20347] Updated weights for policy 0, policy_version 176115 (0.0010) [2023-12-27 06:52:24,860][20347] Updated weights for policy 0, policy_version 176125 (0.0010) [2023-12-27 06:52:24,918][20347] Updated weights for policy 0, policy_version 176135 (0.0010) [2023-12-27 06:52:25,363][20381] Updated weights for policy 1, policy_version 175836 (0.0008) [2023-12-27 06:52:25,416][20381] Updated weights for policy 1, policy_version 175846 (0.0006) [2023-12-27 06:52:25,463][20381] Updated weights for policy 1, policy_version 175856 (0.0005) [2023-12-27 06:52:25,589][20347] Updated weights for policy 0, policy_version 176145 (0.0006) [2023-12-27 06:52:25,650][20347] Updated weights for policy 0, policy_version 176155 (0.0006) [2023-12-27 06:52:25,711][20347] Updated weights for policy 0, policy_version 176165 (0.0010) [2023-12-27 06:52:26,132][20381] Updated weights for policy 1, policy_version 175866 (0.0006) [2023-12-27 06:52:26,186][20381] Updated weights for policy 1, policy_version 175876 (0.0008) [2023-12-27 06:52:26,246][20381] Updated weights for policy 1, policy_version 175886 (0.0008) [2023-12-27 06:52:26,280][20347] Updated weights for policy 0, policy_version 176175 (0.0007) [2023-12-27 06:52:26,296][20381] Updated weights for policy 1, policy_version 175896 (0.0007) [2023-12-27 06:52:26,341][20347] Updated weights for policy 0, policy_version 176185 (0.0007) [2023-12-27 06:52:26,399][20347] Updated weights for policy 0, policy_version 176195 (0.0007) [2023-12-27 06:52:26,916][20381] Updated weights for policy 1, policy_version 175906 (0.0005) [2023-12-27 06:52:26,966][20381] Updated weights for policy 1, policy_version 175916 (0.0008) [2023-12-27 06:52:26,981][20347] Updated weights for policy 0, policy_version 176205 (0.0006) [2023-12-27 06:52:27,013][20381] Updated weights for policy 1, policy_version 175926 (0.0010) [2023-12-27 06:52:27,038][20347] Updated weights for policy 0, policy_version 176215 (0.0005) [2023-12-27 06:52:27,098][20347] Updated weights for policy 0, policy_version 176225 (0.0007) [2023-12-27 06:52:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 90177536. Throughput: 0: 10306.1, 1: 10335.0. Samples: 90162852. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:27,363][19170] Avg episode reward: [(0, '31.100'), (1, '1000.000')] [2023-12-27 06:52:27,635][20381] Updated weights for policy 1, policy_version 175936 (0.0007) [2023-12-27 06:52:27,690][20381] Updated weights for policy 1, policy_version 175946 (0.0005) [2023-12-27 06:52:27,725][20347] Updated weights for policy 0, policy_version 176235 (0.0007) [2023-12-27 06:52:27,746][20381] Updated weights for policy 1, policy_version 175956 (0.0005) [2023-12-27 06:52:27,776][20347] Updated weights for policy 0, policy_version 176245 (0.0007) [2023-12-27 06:52:27,831][20347] Updated weights for policy 0, policy_version 176255 (0.0007) [2023-12-27 06:52:28,325][20381] Updated weights for policy 1, policy_version 175966 (0.0008) [2023-12-27 06:52:28,384][20381] Updated weights for policy 1, policy_version 175976 (0.0009) [2023-12-27 06:52:28,441][20381] Updated weights for policy 1, policy_version 175986 (0.0005) [2023-12-27 06:52:28,490][20347] Updated weights for policy 0, policy_version 176265 (0.0008) [2023-12-27 06:52:28,548][20347] Updated weights for policy 0, policy_version 176275 (0.0007) [2023-12-27 06:52:28,613][20347] Updated weights for policy 0, policy_version 176285 (0.0008) [2023-12-27 06:52:28,670][20347] Updated weights for policy 0, policy_version 176295 (0.0008) [2023-12-27 06:52:28,998][20381] Updated weights for policy 1, policy_version 175996 (0.0007) [2023-12-27 06:52:29,046][20381] Updated weights for policy 1, policy_version 176006 (0.0005) [2023-12-27 06:52:29,091][20381] Updated weights for policy 1, policy_version 176016 (0.0005) [2023-12-27 06:52:29,344][20347] Updated weights for policy 0, policy_version 176305 (0.0007) [2023-12-27 06:52:29,403][20347] Updated weights for policy 0, policy_version 176315 (0.0007) [2023-12-27 06:52:29,461][20347] Updated weights for policy 0, policy_version 176325 (0.0007) [2023-12-27 06:52:29,813][20381] Updated weights for policy 1, policy_version 176026 (0.0008) [2023-12-27 06:52:29,876][20381] Updated weights for policy 1, policy_version 176036 (0.0009) [2023-12-27 06:52:29,945][20381] Updated weights for policy 1, policy_version 176046 (0.0009) [2023-12-27 06:52:30,006][20381] Updated weights for policy 1, policy_version 176056 (0.0008) [2023-12-27 06:52:30,136][20347] Updated weights for policy 0, policy_version 176335 (0.0008) [2023-12-27 06:52:30,197][20347] Updated weights for policy 0, policy_version 176345 (0.0007) [2023-12-27 06:52:30,253][20347] Updated weights for policy 0, policy_version 176355 (0.0008) [2023-12-27 06:52:30,687][20381] Updated weights for policy 1, policy_version 176066 (0.0008) [2023-12-27 06:52:30,747][20381] Updated weights for policy 1, policy_version 176076 (0.0008) [2023-12-27 06:52:30,807][20381] Updated weights for policy 1, policy_version 176086 (0.0008) [2023-12-27 06:52:30,937][20347] Updated weights for policy 0, policy_version 176365 (0.0006) [2023-12-27 06:52:30,987][20347] Updated weights for policy 0, policy_version 176375 (0.0005) [2023-12-27 06:52:31,043][20347] Updated weights for policy 0, policy_version 176385 (0.0007) [2023-12-27 06:52:31,577][20381] Updated weights for policy 1, policy_version 176096 (0.0008) [2023-12-27 06:52:31,633][20381] Updated weights for policy 1, policy_version 176106 (0.0008) [2023-12-27 06:52:31,686][20381] Updated weights for policy 1, policy_version 176116 (0.0007) [2023-12-27 06:52:31,713][20347] Updated weights for policy 0, policy_version 176395 (0.0009) [2023-12-27 06:52:31,771][20347] Updated weights for policy 0, policy_version 176405 (0.0013) [2023-12-27 06:52:31,823][20347] Updated weights for policy 0, policy_version 176415 (0.0008) [2023-12-27 06:52:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 90275840. Throughput: 0: 10379.5, 1: 10341.5. Samples: 90288568. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 06:52:32,363][19170] Avg episode reward: [(0, '31.430'), (1, '1000.000')] [2023-12-27 06:52:32,468][20381] Updated weights for policy 1, policy_version 176126 (0.0007) [2023-12-27 06:52:32,522][20381] Updated weights for policy 1, policy_version 176136 (0.0007) [2023-12-27 06:52:32,531][20347] Updated weights for policy 0, policy_version 176425 (0.0009) [2023-12-27 06:52:32,568][20381] Updated weights for policy 1, policy_version 176146 (0.0006) [2023-12-27 06:52:32,590][20347] Updated weights for policy 0, policy_version 176435 (0.0010) [2023-12-27 06:52:32,647][20347] Updated weights for policy 0, policy_version 176445 (0.0010) [2023-12-27 06:52:32,705][20347] Updated weights for policy 0, policy_version 176455 (0.0007) [2023-12-27 06:52:33,286][20381] Updated weights for policy 1, policy_version 176156 (0.0008) [2023-12-27 06:52:33,343][20381] Updated weights for policy 1, policy_version 176166 (0.0008) [2023-12-27 06:52:33,375][20347] Updated weights for policy 0, policy_version 176465 (0.0006) [2023-12-27 06:52:33,401][20381] Updated weights for policy 1, policy_version 176176 (0.0006) [2023-12-27 06:52:33,435][20347] Updated weights for policy 0, policy_version 176475 (0.0006) [2023-12-27 06:52:33,483][20347] Updated weights for policy 0, policy_version 176485 (0.0005) [2023-12-27 06:52:34,037][20381] Updated weights for policy 1, policy_version 176186 (0.0007) [2023-12-27 06:52:34,094][20381] Updated weights for policy 1, policy_version 176196 (0.0005) [2023-12-27 06:52:34,095][20347] Updated weights for policy 0, policy_version 176495 (0.0009) [2023-12-27 06:52:34,145][20347] Updated weights for policy 0, policy_version 176505 (0.0010) [2023-12-27 06:52:34,154][20381] Updated weights for policy 1, policy_version 176206 (0.0007) [2023-12-27 06:52:34,206][20347] Updated weights for policy 0, policy_version 176515 (0.0009) [2023-12-27 06:52:34,217][20381] Updated weights for policy 1, policy_version 176216 (0.0007) [2023-12-27 06:52:34,896][20347] Updated weights for policy 0, policy_version 176525 (0.0010) [2023-12-27 06:52:34,908][20381] Updated weights for policy 1, policy_version 176226 (0.0007) [2023-12-27 06:52:34,947][20347] Updated weights for policy 0, policy_version 176535 (0.0010) [2023-12-27 06:52:34,962][20381] Updated weights for policy 1, policy_version 176236 (0.0005) [2023-12-27 06:52:34,992][20347] Updated weights for policy 0, policy_version 176545 (0.0010) [2023-12-27 06:52:35,022][20381] Updated weights for policy 1, policy_version 176246 (0.0007) [2023-12-27 06:52:35,612][20347] Updated weights for policy 0, policy_version 176555 (0.0008) [2023-12-27 06:52:35,661][20381] Updated weights for policy 1, policy_version 176256 (0.0009) [2023-12-27 06:52:35,669][20347] Updated weights for policy 0, policy_version 176565 (0.0005) [2023-12-27 06:52:35,710][20381] Updated weights for policy 1, policy_version 176266 (0.0010) [2023-12-27 06:52:35,730][20347] Updated weights for policy 0, policy_version 176575 (0.0005) [2023-12-27 06:52:35,759][20381] Updated weights for policy 1, policy_version 176276 (0.0010) [2023-12-27 06:52:36,319][20347] Updated weights for policy 0, policy_version 176585 (0.0008) [2023-12-27 06:52:36,377][20347] Updated weights for policy 0, policy_version 176595 (0.0009) [2023-12-27 06:52:36,442][20347] Updated weights for policy 0, policy_version 176605 (0.0006) [2023-12-27 06:52:36,444][20381] Updated weights for policy 1, policy_version 176286 (0.0010) [2023-12-27 06:52:36,501][20347] Updated weights for policy 0, policy_version 176615 (0.0010) [2023-12-27 06:52:36,501][20381] Updated weights for policy 1, policy_version 176296 (0.0010) [2023-12-27 06:52:36,566][20381] Updated weights for policy 1, policy_version 176306 (0.0010) [2023-12-27 06:52:37,147][20381] Updated weights for policy 1, policy_version 176316 (0.0009) [2023-12-27 06:52:37,152][20347] Updated weights for policy 0, policy_version 176625 (0.0010) [2023-12-27 06:52:37,204][20347] Updated weights for policy 0, policy_version 176635 (0.0010) [2023-12-27 06:52:37,215][20381] Updated weights for policy 1, policy_version 176326 (0.0006) [2023-12-27 06:52:37,266][20381] Updated weights for policy 1, policy_version 176336 (0.0005) [2023-12-27 06:52:37,269][20347] Updated weights for policy 0, policy_version 176645 (0.0010) [2023-12-27 06:52:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 90390528. Throughput: 0: 10383.5, 1: 10317.6. Samples: 90349956. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:52:37,363][19170] Avg episode reward: [(0, '45.200'), (1, '1000.000')] [2023-12-27 06:52:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000176648_45236224.pth... [2023-12-27 06:52:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000176344_45154304.pth... [2023-12-27 06:52:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000175128_44843008.pth [2023-12-27 06:52:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000175432_44924928.pth [2023-12-27 06:52:37,841][20381] Updated weights for policy 1, policy_version 176346 (0.0006) [2023-12-27 06:52:37,898][20381] Updated weights for policy 1, policy_version 176356 (0.0007) [2023-12-27 06:52:37,952][20381] Updated weights for policy 1, policy_version 176366 (0.0008) [2023-12-27 06:52:37,959][20347] Updated weights for policy 0, policy_version 176655 (0.0010) [2023-12-27 06:52:38,012][20381] Updated weights for policy 1, policy_version 176376 (0.0005) [2023-12-27 06:52:38,014][20347] Updated weights for policy 0, policy_version 176665 (0.0010) [2023-12-27 06:52:38,065][20347] Updated weights for policy 0, policy_version 176675 (0.0010) [2023-12-27 06:52:38,653][20381] Updated weights for policy 1, policy_version 176386 (0.0008) [2023-12-27 06:52:38,710][20381] Updated weights for policy 1, policy_version 176396 (0.0010) [2023-12-27 06:52:38,763][20347] Updated weights for policy 0, policy_version 176685 (0.0010) [2023-12-27 06:52:38,776][20381] Updated weights for policy 1, policy_version 176406 (0.0009) [2023-12-27 06:52:38,823][20347] Updated weights for policy 0, policy_version 176695 (0.0011) [2023-12-27 06:52:38,885][20347] Updated weights for policy 0, policy_version 176705 (0.0010) [2023-12-27 06:52:39,374][20381] Updated weights for policy 1, policy_version 176416 (0.0008) [2023-12-27 06:52:39,433][20381] Updated weights for policy 1, policy_version 176426 (0.0010) [2023-12-27 06:52:39,486][20381] Updated weights for policy 1, policy_version 176436 (0.0009) [2023-12-27 06:52:39,567][20347] Updated weights for policy 0, policy_version 176715 (0.0009) [2023-12-27 06:52:39,614][20347] Updated weights for policy 0, policy_version 176725 (0.0008) [2023-12-27 06:52:39,666][20347] Updated weights for policy 0, policy_version 176735 (0.0007) [2023-12-27 06:52:40,131][20381] Updated weights for policy 1, policy_version 176446 (0.0005) [2023-12-27 06:52:40,198][20381] Updated weights for policy 1, policy_version 176456 (0.0011) [2023-12-27 06:52:40,251][20381] Updated weights for policy 1, policy_version 176466 (0.0011) [2023-12-27 06:52:40,394][20347] Updated weights for policy 0, policy_version 176745 (0.0008) [2023-12-27 06:52:40,450][20347] Updated weights for policy 0, policy_version 176755 (0.0007) [2023-12-27 06:52:40,498][20347] Updated weights for policy 0, policy_version 176765 (0.0007) [2023-12-27 06:52:40,556][20347] Updated weights for policy 0, policy_version 176775 (0.0007) [2023-12-27 06:52:40,909][20381] Updated weights for policy 1, policy_version 176476 (0.0011) [2023-12-27 06:52:40,968][20381] Updated weights for policy 1, policy_version 176486 (0.0010) [2023-12-27 06:52:41,020][20381] Updated weights for policy 1, policy_version 176496 (0.0010) [2023-12-27 06:52:41,254][20347] Updated weights for policy 0, policy_version 176785 (0.0007) [2023-12-27 06:52:41,316][20347] Updated weights for policy 0, policy_version 176795 (0.0008) [2023-12-27 06:52:41,381][20347] Updated weights for policy 0, policy_version 176805 (0.0008) [2023-12-27 06:52:41,758][20381] Updated weights for policy 1, policy_version 176506 (0.0008) [2023-12-27 06:52:41,816][20381] Updated weights for policy 1, policy_version 176516 (0.0011) [2023-12-27 06:52:41,877][20381] Updated weights for policy 1, policy_version 176526 (0.0011) [2023-12-27 06:52:41,939][20381] Updated weights for policy 1, policy_version 176536 (0.0011) [2023-12-27 06:52:42,073][20347] Updated weights for policy 0, policy_version 176815 (0.0008) [2023-12-27 06:52:42,130][20347] Updated weights for policy 0, policy_version 176825 (0.0007) [2023-12-27 06:52:42,187][20347] Updated weights for policy 0, policy_version 176835 (0.0005) [2023-12-27 06:52:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 90488832. Throughput: 0: 10379.5, 1: 10395.2. Samples: 90477804. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:52:42,363][19170] Avg episode reward: [(0, '50.100'), (1, '1000.000')] [2023-12-27 06:52:42,607][20381] Updated weights for policy 1, policy_version 176546 (0.0010) [2023-12-27 06:52:42,666][20381] Updated weights for policy 1, policy_version 176556 (0.0008) [2023-12-27 06:52:42,723][20381] Updated weights for policy 1, policy_version 176566 (0.0007) [2023-12-27 06:52:42,870][20347] Updated weights for policy 0, policy_version 176845 (0.0008) [2023-12-27 06:52:42,927][20347] Updated weights for policy 0, policy_version 176855 (0.0007) [2023-12-27 06:52:42,979][20347] Updated weights for policy 0, policy_version 176865 (0.0007) [2023-12-27 06:52:43,392][20381] Updated weights for policy 1, policy_version 176576 (0.0007) [2023-12-27 06:52:43,450][20381] Updated weights for policy 1, policy_version 176586 (0.0008) [2023-12-27 06:52:43,513][20381] Updated weights for policy 1, policy_version 176596 (0.0005) [2023-12-27 06:52:43,621][20347] Updated weights for policy 0, policy_version 176875 (0.0007) [2023-12-27 06:52:43,677][20347] Updated weights for policy 0, policy_version 176885 (0.0005) [2023-12-27 06:52:43,739][20347] Updated weights for policy 0, policy_version 176895 (0.0006) [2023-12-27 06:52:44,174][20381] Updated weights for policy 1, policy_version 176606 (0.0006) [2023-12-27 06:52:44,230][20381] Updated weights for policy 1, policy_version 176616 (0.0007) [2023-12-27 06:52:44,287][20381] Updated weights for policy 1, policy_version 176626 (0.0007) [2023-12-27 06:52:44,363][20347] Updated weights for policy 0, policy_version 176905 (0.0006) [2023-12-27 06:52:44,415][20347] Updated weights for policy 0, policy_version 176915 (0.0010) [2023-12-27 06:52:44,462][20347] Updated weights for policy 0, policy_version 176925 (0.0006) [2023-12-27 06:52:44,510][20347] Updated weights for policy 0, policy_version 176935 (0.0007) [2023-12-27 06:52:44,973][20381] Updated weights for policy 1, policy_version 176636 (0.0008) [2023-12-27 06:52:45,029][20381] Updated weights for policy 1, policy_version 176646 (0.0008) [2023-12-27 06:52:45,078][20381] Updated weights for policy 1, policy_version 176656 (0.0009) [2023-12-27 06:52:45,233][20347] Updated weights for policy 0, policy_version 176945 (0.0010) [2023-12-27 06:52:45,289][20347] Updated weights for policy 0, policy_version 176955 (0.0010) [2023-12-27 06:52:45,349][20347] Updated weights for policy 0, policy_version 176965 (0.0011) [2023-12-27 06:52:45,741][20381] Updated weights for policy 1, policy_version 176666 (0.0009) [2023-12-27 06:52:45,800][20381] Updated weights for policy 1, policy_version 176676 (0.0011) [2023-12-27 06:52:45,860][20381] Updated weights for policy 1, policy_version 176686 (0.0011) [2023-12-27 06:52:45,913][20381] Updated weights for policy 1, policy_version 176696 (0.0011) [2023-12-27 06:52:46,051][20347] Updated weights for policy 0, policy_version 176975 (0.0010) [2023-12-27 06:52:46,115][20347] Updated weights for policy 0, policy_version 176985 (0.0009) [2023-12-27 06:52:46,180][20347] Updated weights for policy 0, policy_version 176995 (0.0011) [2023-12-27 06:52:46,545][20381] Updated weights for policy 1, policy_version 176706 (0.0005) [2023-12-27 06:52:46,599][20381] Updated weights for policy 1, policy_version 176716 (0.0005) [2023-12-27 06:52:46,655][20381] Updated weights for policy 1, policy_version 176726 (0.0005) [2023-12-27 06:52:46,797][20347] Updated weights for policy 0, policy_version 177005 (0.0010) [2023-12-27 06:52:46,856][20347] Updated weights for policy 0, policy_version 177015 (0.0011) [2023-12-27 06:52:46,914][20347] Updated weights for policy 0, policy_version 177025 (0.0006) [2023-12-27 06:52:47,230][20381] Updated weights for policy 1, policy_version 176736 (0.0007) [2023-12-27 06:52:47,276][20381] Updated weights for policy 1, policy_version 176746 (0.0011) [2023-12-27 06:52:47,325][20381] Updated weights for policy 1, policy_version 176756 (0.0008) [2023-12-27 06:52:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 90595328. Throughput: 0: 10353.9, 1: 10405.3. Samples: 90601872. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:52:47,363][19170] Avg episode reward: [(0, '47.220'), (1, '1000.000')] [2023-12-27 06:52:47,556][20347] Updated weights for policy 0, policy_version 177035 (0.0006) [2023-12-27 06:52:47,612][20347] Updated weights for policy 0, policy_version 177045 (0.0005) [2023-12-27 06:52:47,676][20347] Updated weights for policy 0, policy_version 177055 (0.0005) [2023-12-27 06:52:47,917][20381] Updated weights for policy 1, policy_version 176766 (0.0007) [2023-12-27 06:52:47,965][20381] Updated weights for policy 1, policy_version 176776 (0.0005) [2023-12-27 06:52:48,010][20381] Updated weights for policy 1, policy_version 176786 (0.0005) [2023-12-27 06:52:48,251][20347] Updated weights for policy 0, policy_version 177065 (0.0008) [2023-12-27 06:52:48,318][20347] Updated weights for policy 0, policy_version 177075 (0.0009) [2023-12-27 06:52:48,379][20347] Updated weights for policy 0, policy_version 177085 (0.0007) [2023-12-27 06:52:48,435][20347] Updated weights for policy 0, policy_version 177095 (0.0008) [2023-12-27 06:52:48,660][20381] Updated weights for policy 1, policy_version 176796 (0.0006) [2023-12-27 06:52:48,717][20381] Updated weights for policy 1, policy_version 176806 (0.0008) [2023-12-27 06:52:48,774][20381] Updated weights for policy 1, policy_version 176816 (0.0008) [2023-12-27 06:52:49,138][20347] Updated weights for policy 0, policy_version 177105 (0.0007) [2023-12-27 06:52:49,185][20347] Updated weights for policy 0, policy_version 177115 (0.0008) [2023-12-27 06:52:49,245][20347] Updated weights for policy 0, policy_version 177125 (0.0007) [2023-12-27 06:52:49,433][20381] Updated weights for policy 1, policy_version 176826 (0.0007) [2023-12-27 06:52:49,490][20381] Updated weights for policy 1, policy_version 176836 (0.0008) [2023-12-27 06:52:49,547][20381] Updated weights for policy 1, policy_version 176846 (0.0008) [2023-12-27 06:52:49,598][20381] Updated weights for policy 1, policy_version 176856 (0.0008) [2023-12-27 06:52:49,926][20347] Updated weights for policy 0, policy_version 177135 (0.0007) [2023-12-27 06:52:49,985][20347] Updated weights for policy 0, policy_version 177145 (0.0007) [2023-12-27 06:52:50,041][20347] Updated weights for policy 0, policy_version 177155 (0.0007) [2023-12-27 06:52:50,286][20381] Updated weights for policy 1, policy_version 176866 (0.0007) [2023-12-27 06:52:50,346][20381] Updated weights for policy 1, policy_version 176876 (0.0008) [2023-12-27 06:52:50,397][20381] Updated weights for policy 1, policy_version 176886 (0.0007) [2023-12-27 06:52:50,714][20347] Updated weights for policy 0, policy_version 177165 (0.0006) [2023-12-27 06:52:50,763][20347] Updated weights for policy 0, policy_version 177175 (0.0007) [2023-12-27 06:52:50,813][20347] Updated weights for policy 0, policy_version 177185 (0.0006) [2023-12-27 06:52:51,126][20381] Updated weights for policy 1, policy_version 176896 (0.0007) [2023-12-27 06:52:51,196][20381] Updated weights for policy 1, policy_version 176906 (0.0008) [2023-12-27 06:52:51,256][20381] Updated weights for policy 1, policy_version 176916 (0.0007) [2023-12-27 06:52:51,548][20347] Updated weights for policy 0, policy_version 177195 (0.0008) [2023-12-27 06:52:51,602][20347] Updated weights for policy 0, policy_version 177205 (0.0007) [2023-12-27 06:52:51,669][20347] Updated weights for policy 0, policy_version 177215 (0.0008) [2023-12-27 06:52:52,006][20381] Updated weights for policy 1, policy_version 176926 (0.0008) [2023-12-27 06:52:52,062][20381] Updated weights for policy 1, policy_version 176936 (0.0008) [2023-12-27 06:52:52,114][20381] Updated weights for policy 1, policy_version 176946 (0.0008) [2023-12-27 06:52:52,361][20347] Updated weights for policy 0, policy_version 177225 (0.0008) [2023-12-27 06:52:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 90693632. Throughput: 0: 10328.0, 1: 10456.5. Samples: 90665164. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:52:52,363][19170] Avg episode reward: [(0, '52.970'), (1, '1000.000')] [2023-12-27 06:52:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000176952_45309952.pth... [2023-12-27 06:52:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000175736_44998656.pth [2023-12-27 06:52:52,417][20347] Updated weights for policy 0, policy_version 177235 (0.0007) [2023-12-27 06:52:52,469][20347] Updated weights for policy 0, policy_version 177245 (0.0007) [2023-12-27 06:52:52,522][20347] Updated weights for policy 0, policy_version 177255 (0.0008) [2023-12-27 06:52:52,527][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000177256_45391872.pth... [2023-12-27 06:52:52,531][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000176008_45072384.pth [2023-12-27 06:52:52,771][20381] Updated weights for policy 1, policy_version 176956 (0.0007) [2023-12-27 06:52:52,827][20381] Updated weights for policy 1, policy_version 176966 (0.0009) [2023-12-27 06:52:52,887][20381] Updated weights for policy 1, policy_version 176976 (0.0006) [2023-12-27 06:52:53,180][20347] Updated weights for policy 0, policy_version 177265 (0.0008) [2023-12-27 06:52:53,234][20347] Updated weights for policy 0, policy_version 177275 (0.0010) [2023-12-27 06:52:53,289][20347] Updated weights for policy 0, policy_version 177285 (0.0010) [2023-12-27 06:52:53,468][20381] Updated weights for policy 1, policy_version 176986 (0.0006) [2023-12-27 06:52:53,513][20381] Updated weights for policy 1, policy_version 176996 (0.0010) [2023-12-27 06:52:53,566][20381] Updated weights for policy 1, policy_version 177006 (0.0010) [2023-12-27 06:52:53,622][20381] Updated weights for policy 1, policy_version 177016 (0.0009) [2023-12-27 06:52:53,931][20347] Updated weights for policy 0, policy_version 177295 (0.0010) [2023-12-27 06:52:53,985][20347] Updated weights for policy 0, policy_version 177305 (0.0011) [2023-12-27 06:52:54,036][20347] Updated weights for policy 0, policy_version 177315 (0.0008) [2023-12-27 06:52:54,292][20381] Updated weights for policy 1, policy_version 177026 (0.0009) [2023-12-27 06:52:54,342][20381] Updated weights for policy 1, policy_version 177036 (0.0010) [2023-12-27 06:52:54,394][20381] Updated weights for policy 1, policy_version 177046 (0.0010) [2023-12-27 06:52:54,644][20347] Updated weights for policy 0, policy_version 177325 (0.0010) [2023-12-27 06:52:54,689][20347] Updated weights for policy 0, policy_version 177335 (0.0009) [2023-12-27 06:52:54,742][20347] Updated weights for policy 0, policy_version 177345 (0.0008) [2023-12-27 06:52:55,018][20381] Updated weights for policy 1, policy_version 177056 (0.0006) [2023-12-27 06:52:55,081][20381] Updated weights for policy 1, policy_version 177066 (0.0006) [2023-12-27 06:52:55,142][20381] Updated weights for policy 1, policy_version 177076 (0.0008) [2023-12-27 06:52:55,390][20347] Updated weights for policy 0, policy_version 177355 (0.0008) [2023-12-27 06:52:55,445][20347] Updated weights for policy 0, policy_version 177365 (0.0010) [2023-12-27 06:52:55,497][20347] Updated weights for policy 0, policy_version 177375 (0.0010) [2023-12-27 06:52:55,764][20381] Updated weights for policy 1, policy_version 177086 (0.0007) [2023-12-27 06:52:55,821][20381] Updated weights for policy 1, policy_version 177096 (0.0007) [2023-12-27 06:52:55,873][20381] Updated weights for policy 1, policy_version 177106 (0.0007) [2023-12-27 06:52:56,163][20347] Updated weights for policy 0, policy_version 177385 (0.0010) [2023-12-27 06:52:56,221][20347] Updated weights for policy 0, policy_version 177395 (0.0007) [2023-12-27 06:52:56,277][20347] Updated weights for policy 0, policy_version 177405 (0.0008) [2023-12-27 06:52:56,337][20347] Updated weights for policy 0, policy_version 177415 (0.0009) [2023-12-27 06:52:56,524][20381] Updated weights for policy 1, policy_version 177116 (0.0006) [2023-12-27 06:52:56,581][20381] Updated weights for policy 1, policy_version 177126 (0.0005) [2023-12-27 06:52:56,639][20381] Updated weights for policy 1, policy_version 177136 (0.0007) [2023-12-27 06:52:57,001][20347] Updated weights for policy 0, policy_version 177425 (0.0007) [2023-12-27 06:52:57,050][20347] Updated weights for policy 0, policy_version 177435 (0.0008) [2023-12-27 06:52:57,095][20347] Updated weights for policy 0, policy_version 177445 (0.0007) [2023-12-27 06:52:57,268][20381] Updated weights for policy 1, policy_version 177146 (0.0008) [2023-12-27 06:52:57,326][20381] Updated weights for policy 1, policy_version 177156 (0.0007) [2023-12-27 06:52:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 90800128. Throughput: 0: 10369.5, 1: 10462.5. Samples: 90790660. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:52:57,364][19170] Avg episode reward: [(0, '51.120'), (1, '1000.000')] [2023-12-27 06:52:57,379][20381] Updated weights for policy 1, policy_version 177166 (0.0008) [2023-12-27 06:52:57,427][20381] Updated weights for policy 1, policy_version 177176 (0.0007) [2023-12-27 06:52:57,806][20347] Updated weights for policy 0, policy_version 177455 (0.0008) [2023-12-27 06:52:57,854][20347] Updated weights for policy 0, policy_version 177465 (0.0007) [2023-12-27 06:52:57,901][20347] Updated weights for policy 0, policy_version 177475 (0.0007) [2023-12-27 06:52:58,085][20381] Updated weights for policy 1, policy_version 177186 (0.0010) [2023-12-27 06:52:58,145][20381] Updated weights for policy 1, policy_version 177196 (0.0010) [2023-12-27 06:52:58,208][20381] Updated weights for policy 1, policy_version 177206 (0.0008) [2023-12-27 06:52:58,684][20347] Updated weights for policy 0, policy_version 177485 (0.0007) [2023-12-27 06:52:58,747][20347] Updated weights for policy 0, policy_version 177495 (0.0008) [2023-12-27 06:52:58,811][20347] Updated weights for policy 0, policy_version 177505 (0.0007) [2023-12-27 06:52:58,946][20381] Updated weights for policy 1, policy_version 177216 (0.0009) [2023-12-27 06:52:59,004][20381] Updated weights for policy 1, policy_version 177226 (0.0007) [2023-12-27 06:52:59,064][20381] Updated weights for policy 1, policy_version 177236 (0.0008) [2023-12-27 06:52:59,529][20347] Updated weights for policy 0, policy_version 177515 (0.0007) [2023-12-27 06:52:59,587][20347] Updated weights for policy 0, policy_version 177525 (0.0007) [2023-12-27 06:52:59,642][20347] Updated weights for policy 0, policy_version 177535 (0.0007) [2023-12-27 06:52:59,766][20381] Updated weights for policy 1, policy_version 177246 (0.0009) [2023-12-27 06:52:59,816][20381] Updated weights for policy 1, policy_version 177256 (0.0007) [2023-12-27 06:52:59,873][20381] Updated weights for policy 1, policy_version 177266 (0.0008) [2023-12-27 06:53:00,305][20347] Updated weights for policy 0, policy_version 177545 (0.0007) [2023-12-27 06:53:00,356][20347] Updated weights for policy 0, policy_version 177555 (0.0010) [2023-12-27 06:53:00,417][20347] Updated weights for policy 0, policy_version 177565 (0.0008) [2023-12-27 06:53:00,473][20347] Updated weights for policy 0, policy_version 177575 (0.0007) [2023-12-27 06:53:00,534][20381] Updated weights for policy 1, policy_version 177276 (0.0008) [2023-12-27 06:53:00,589][20381] Updated weights for policy 1, policy_version 177286 (0.0010) [2023-12-27 06:53:00,642][20381] Updated weights for policy 1, policy_version 177296 (0.0006) [2023-12-27 06:53:01,148][20347] Updated weights for policy 0, policy_version 177585 (0.0006) [2023-12-27 06:53:01,203][20347] Updated weights for policy 0, policy_version 177595 (0.0007) [2023-12-27 06:53:01,260][20347] Updated weights for policy 0, policy_version 177605 (0.0008) [2023-12-27 06:53:01,339][20381] Updated weights for policy 1, policy_version 177306 (0.0006) [2023-12-27 06:53:01,408][20381] Updated weights for policy 1, policy_version 177316 (0.0008) [2023-12-27 06:53:01,466][20381] Updated weights for policy 1, policy_version 177326 (0.0007) [2023-12-27 06:53:01,522][20381] Updated weights for policy 1, policy_version 177336 (0.0008) [2023-12-27 06:53:01,935][20347] Updated weights for policy 0, policy_version 177615 (0.0005) [2023-12-27 06:53:01,987][20347] Updated weights for policy 0, policy_version 177625 (0.0006) [2023-12-27 06:53:02,040][20347] Updated weights for policy 0, policy_version 177635 (0.0007) [2023-12-27 06:53:02,239][20381] Updated weights for policy 1, policy_version 177346 (0.0005) [2023-12-27 06:53:02,301][20381] Updated weights for policy 1, policy_version 177356 (0.0011) [2023-12-27 06:53:02,358][20381] Updated weights for policy 1, policy_version 177366 (0.0010) [2023-12-27 06:53:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 90898432. Throughput: 0: 10334.7, 1: 10411.1. Samples: 90911972. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:02,363][19170] Avg episode reward: [(0, '55.760'), (1, '1000.000')] [2023-12-27 06:53:02,654][20347] Updated weights for policy 0, policy_version 177645 (0.0006) [2023-12-27 06:53:02,710][20347] Updated weights for policy 0, policy_version 177655 (0.0006) [2023-12-27 06:53:02,766][20347] Updated weights for policy 0, policy_version 177665 (0.0008) [2023-12-27 06:53:03,030][20381] Updated weights for policy 1, policy_version 177376 (0.0010) [2023-12-27 06:53:03,078][20381] Updated weights for policy 1, policy_version 177386 (0.0010) [2023-12-27 06:53:03,123][20381] Updated weights for policy 1, policy_version 177396 (0.0009) [2023-12-27 06:53:03,322][20347] Updated weights for policy 0, policy_version 177675 (0.0009) [2023-12-27 06:53:03,382][20347] Updated weights for policy 0, policy_version 177685 (0.0006) [2023-12-27 06:53:03,440][20347] Updated weights for policy 0, policy_version 177695 (0.0009) [2023-12-27 06:53:03,781][20381] Updated weights for policy 1, policy_version 177406 (0.0006) [2023-12-27 06:53:03,835][20381] Updated weights for policy 1, policy_version 177416 (0.0007) [2023-12-27 06:53:03,889][20381] Updated weights for policy 1, policy_version 177426 (0.0007) [2023-12-27 06:53:03,997][20347] Updated weights for policy 0, policy_version 177705 (0.0009) [2023-12-27 06:53:04,052][20347] Updated weights for policy 0, policy_version 177715 (0.0005) [2023-12-27 06:53:04,111][20347] Updated weights for policy 0, policy_version 177725 (0.0006) [2023-12-27 06:53:04,169][20347] Updated weights for policy 0, policy_version 177735 (0.0008) [2023-12-27 06:53:04,621][20381] Updated weights for policy 1, policy_version 177436 (0.0007) [2023-12-27 06:53:04,678][20381] Updated weights for policy 1, policy_version 177446 (0.0008) [2023-12-27 06:53:04,740][20381] Updated weights for policy 1, policy_version 177456 (0.0008) [2023-12-27 06:53:04,856][20347] Updated weights for policy 0, policy_version 177745 (0.0008) [2023-12-27 06:53:04,920][20347] Updated weights for policy 0, policy_version 177755 (0.0008) [2023-12-27 06:53:04,966][20347] Updated weights for policy 0, policy_version 177765 (0.0008) [2023-12-27 06:53:05,386][20381] Updated weights for policy 1, policy_version 177466 (0.0007) [2023-12-27 06:53:05,433][20381] Updated weights for policy 1, policy_version 177476 (0.0007) [2023-12-27 06:53:05,482][20381] Updated weights for policy 1, policy_version 177486 (0.0008) [2023-12-27 06:53:05,537][20381] Updated weights for policy 1, policy_version 177496 (0.0007) [2023-12-27 06:53:05,686][20347] Updated weights for policy 0, policy_version 177775 (0.0009) [2023-12-27 06:53:05,749][20347] Updated weights for policy 0, policy_version 177785 (0.0008) [2023-12-27 06:53:05,805][20347] Updated weights for policy 0, policy_version 177795 (0.0008) [2023-12-27 06:53:06,155][20381] Updated weights for policy 1, policy_version 177506 (0.0007) [2023-12-27 06:53:06,215][20381] Updated weights for policy 1, policy_version 177516 (0.0005) [2023-12-27 06:53:06,282][20381] Updated weights for policy 1, policy_version 177526 (0.0005) [2023-12-27 06:53:06,524][20347] Updated weights for policy 0, policy_version 177805 (0.0010) [2023-12-27 06:53:06,584][20347] Updated weights for policy 0, policy_version 177815 (0.0010) [2023-12-27 06:53:06,640][20347] Updated weights for policy 0, policy_version 177825 (0.0010) [2023-12-27 06:53:06,912][20381] Updated weights for policy 1, policy_version 177536 (0.0010) [2023-12-27 06:53:06,970][20381] Updated weights for policy 1, policy_version 177546 (0.0011) [2023-12-27 06:53:07,025][20381] Updated weights for policy 1, policy_version 177556 (0.0011) [2023-12-27 06:53:07,303][20347] Updated weights for policy 0, policy_version 177835 (0.0010) [2023-12-27 06:53:07,362][20347] Updated weights for policy 0, policy_version 177845 (0.0006) [2023-12-27 06:53:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 91004928. Throughput: 0: 10400.1, 1: 10392.5. Samples: 90976020. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:07,363][19170] Avg episode reward: [(0, '59.830'), (1, '998.470')] [2023-12-27 06:53:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000177560_45465600.pth... [2023-12-27 06:53:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000176344_45154304.pth [2023-12-27 06:53:07,419][20347] Updated weights for policy 0, policy_version 177855 (0.0005) [2023-12-27 06:53:07,466][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000177864_45547520.pth... [2023-12-27 06:53:07,469][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000176648_45236224.pth [2023-12-27 06:53:07,707][20381] Updated weights for policy 1, policy_version 177566 (0.0008) [2023-12-27 06:53:07,760][20381] Updated weights for policy 1, policy_version 177576 (0.0008) [2023-12-27 06:53:07,815][20381] Updated weights for policy 1, policy_version 177586 (0.0008) [2023-12-27 06:53:07,999][20347] Updated weights for policy 0, policy_version 177865 (0.0005) [2023-12-27 06:53:08,060][20347] Updated weights for policy 0, policy_version 177875 (0.0006) [2023-12-27 06:53:08,119][20347] Updated weights for policy 0, policy_version 177885 (0.0008) [2023-12-27 06:53:08,173][20347] Updated weights for policy 0, policy_version 177895 (0.0008) [2023-12-27 06:53:08,485][20381] Updated weights for policy 1, policy_version 177596 (0.0008) [2023-12-27 06:53:08,542][20381] Updated weights for policy 1, policy_version 177606 (0.0008) [2023-12-27 06:53:08,593][20381] Updated weights for policy 1, policy_version 177616 (0.0007) [2023-12-27 06:53:08,877][20347] Updated weights for policy 0, policy_version 177905 (0.0007) [2023-12-27 06:53:08,928][20347] Updated weights for policy 0, policy_version 177915 (0.0005) [2023-12-27 06:53:08,976][20347] Updated weights for policy 0, policy_version 177925 (0.0005) [2023-12-27 06:53:09,353][20381] Updated weights for policy 1, policy_version 177626 (0.0007) [2023-12-27 06:53:09,416][20381] Updated weights for policy 1, policy_version 177636 (0.0008) [2023-12-27 06:53:09,472][20381] Updated weights for policy 1, policy_version 177646 (0.0009) [2023-12-27 06:53:09,527][20381] Updated weights for policy 1, policy_version 177656 (0.0008) [2023-12-27 06:53:09,616][20347] Updated weights for policy 0, policy_version 177935 (0.0009) [2023-12-27 06:53:09,673][20347] Updated weights for policy 0, policy_version 177945 (0.0008) [2023-12-27 06:53:09,731][20347] Updated weights for policy 0, policy_version 177955 (0.0008) [2023-12-27 06:53:10,259][20381] Updated weights for policy 1, policy_version 177666 (0.0007) [2023-12-27 06:53:10,321][20381] Updated weights for policy 1, policy_version 177676 (0.0007) [2023-12-27 06:53:10,371][20381] Updated weights for policy 1, policy_version 177686 (0.0007) [2023-12-27 06:53:10,453][20347] Updated weights for policy 0, policy_version 177965 (0.0009) [2023-12-27 06:53:10,500][20347] Updated weights for policy 0, policy_version 177975 (0.0010) [2023-12-27 06:53:10,548][20347] Updated weights for policy 0, policy_version 177985 (0.0010) [2023-12-27 06:53:11,072][20381] Updated weights for policy 1, policy_version 177696 (0.0008) [2023-12-27 06:53:11,134][20381] Updated weights for policy 1, policy_version 177706 (0.0008) [2023-12-27 06:53:11,187][20381] Updated weights for policy 1, policy_version 177716 (0.0008) [2023-12-27 06:53:11,273][20347] Updated weights for policy 0, policy_version 177995 (0.0010) [2023-12-27 06:53:11,341][20347] Updated weights for policy 0, policy_version 178005 (0.0011) [2023-12-27 06:53:11,410][20347] Updated weights for policy 0, policy_version 178015 (0.0010) [2023-12-27 06:53:11,908][20381] Updated weights for policy 1, policy_version 177726 (0.0008) [2023-12-27 06:53:11,975][20381] Updated weights for policy 1, policy_version 177736 (0.0007) [2023-12-27 06:53:12,039][20381] Updated weights for policy 1, policy_version 177746 (0.0007) [2023-12-27 06:53:12,090][20347] Updated weights for policy 0, policy_version 178025 (0.0010) [2023-12-27 06:53:12,152][20347] Updated weights for policy 0, policy_version 178035 (0.0007) [2023-12-27 06:53:12,214][20347] Updated weights for policy 0, policy_version 178045 (0.0007) [2023-12-27 06:53:12,276][20347] Updated weights for policy 0, policy_version 178055 (0.0007) [2023-12-27 06:53:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 91111424. Throughput: 0: 10398.9, 1: 10385.1. Samples: 91098136. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:12,363][19170] Avg episode reward: [(0, '70.570'), (1, '988.910')] [2023-12-27 06:53:12,637][20381] Updated weights for policy 1, policy_version 177756 (0.0007) [2023-12-27 06:53:12,699][20381] Updated weights for policy 1, policy_version 177766 (0.0008) [2023-12-27 06:53:12,759][20381] Updated weights for policy 1, policy_version 177776 (0.0008) [2023-12-27 06:53:12,993][20347] Updated weights for policy 0, policy_version 178065 (0.0010) [2023-12-27 06:53:13,041][20347] Updated weights for policy 0, policy_version 178075 (0.0010) [2023-12-27 06:53:13,093][20347] Updated weights for policy 0, policy_version 178085 (0.0010) [2023-12-27 06:53:13,426][20381] Updated weights for policy 1, policy_version 177786 (0.0008) [2023-12-27 06:53:13,475][20381] Updated weights for policy 1, policy_version 177796 (0.0007) [2023-12-27 06:53:13,521][20381] Updated weights for policy 1, policy_version 177806 (0.0007) [2023-12-27 06:53:13,553][20292] KL-divergence is very high: 933.4456 [2023-12-27 06:53:13,568][20381] Updated weights for policy 1, policy_version 177816 (0.0008) [2023-12-27 06:53:13,781][20347] Updated weights for policy 0, policy_version 178095 (0.0011) [2023-12-27 06:53:13,842][20347] Updated weights for policy 0, policy_version 178105 (0.0010) [2023-12-27 06:53:13,894][20347] Updated weights for policy 0, policy_version 178115 (0.0006) [2023-12-27 06:53:14,286][20381] Updated weights for policy 1, policy_version 177826 (0.0008) [2023-12-27 06:53:14,343][20381] Updated weights for policy 1, policy_version 177836 (0.0006) [2023-12-27 06:53:14,397][20381] Updated weights for policy 1, policy_version 177846 (0.0009) [2023-12-27 06:53:14,486][20347] Updated weights for policy 0, policy_version 178125 (0.0007) [2023-12-27 06:53:14,535][20347] Updated weights for policy 0, policy_version 178135 (0.0005) [2023-12-27 06:53:14,587][20347] Updated weights for policy 0, policy_version 178145 (0.0005) [2023-12-27 06:53:15,119][20292] KL-divergence is very high: 224.2372 [2023-12-27 06:53:15,151][20381] Updated weights for policy 1, policy_version 177856 (0.0008) [2023-12-27 06:53:15,170][20292] KL-divergence is very high: 187.9952 [2023-12-27 06:53:15,207][20381] Updated weights for policy 1, policy_version 177866 (0.0008) [2023-12-27 06:53:15,212][20292] KL-divergence is very high: 148.6752 [2023-12-27 06:53:15,229][20347] Updated weights for policy 0, policy_version 178155 (0.0005) [2023-12-27 06:53:15,256][20292] KL-divergence is very high: 195.5143 [2023-12-27 06:53:15,264][20381] Updated weights for policy 1, policy_version 177876 (0.0008) [2023-12-27 06:53:15,288][20347] Updated weights for policy 0, policy_version 178165 (0.0005) [2023-12-27 06:53:15,356][20347] Updated weights for policy 0, policy_version 178175 (0.0006) [2023-12-27 06:53:15,900][20347] Updated weights for policy 0, policy_version 178185 (0.0005) [2023-12-27 06:53:15,941][20381] Updated weights for policy 1, policy_version 177886 (0.0006) [2023-12-27 06:53:15,961][20347] Updated weights for policy 0, policy_version 178195 (0.0006) [2023-12-27 06:53:15,990][20381] Updated weights for policy 1, policy_version 177896 (0.0007) [2023-12-27 06:53:16,014][20347] Updated weights for policy 0, policy_version 178205 (0.0006) [2023-12-27 06:53:16,043][20381] Updated weights for policy 1, policy_version 177906 (0.0007) [2023-12-27 06:53:16,080][20347] Updated weights for policy 0, policy_version 178215 (0.0005) [2023-12-27 06:53:16,693][20381] Updated weights for policy 1, policy_version 177916 (0.0009) [2023-12-27 06:53:16,714][20347] Updated weights for policy 0, policy_version 178225 (0.0005) [2023-12-27 06:53:16,715][20292] KL-divergence is very high: 102.8168 [2023-12-27 06:53:16,749][20381] Updated weights for policy 1, policy_version 177926 (0.0011) [2023-12-27 06:53:16,775][20347] Updated weights for policy 0, policy_version 178235 (0.0006) [2023-12-27 06:53:16,806][20381] Updated weights for policy 1, policy_version 177936 (0.0011) [2023-12-27 06:53:16,828][20347] Updated weights for policy 0, policy_version 178245 (0.0010) [2023-12-27 06:53:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 91209728. Throughput: 0: 10407.7, 1: 10355.0. Samples: 91222888. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:17,363][19170] Avg episode reward: [(0, '41.940'), (1, '950.860')] [2023-12-27 06:53:17,457][20381] Updated weights for policy 1, policy_version 177946 (0.0006) [2023-12-27 06:53:17,498][20347] Updated weights for policy 0, policy_version 178255 (0.0008) [2023-12-27 06:53:17,516][20381] Updated weights for policy 1, policy_version 177956 (0.0011) [2023-12-27 06:53:17,560][20347] Updated weights for policy 0, policy_version 178265 (0.0005) [2023-12-27 06:53:17,575][20381] Updated weights for policy 1, policy_version 177966 (0.0010) [2023-12-27 06:53:17,617][20347] Updated weights for policy 0, policy_version 178275 (0.0007) [2023-12-27 06:53:17,630][20381] Updated weights for policy 1, policy_version 177976 (0.0005) [2023-12-27 06:53:18,214][20381] Updated weights for policy 1, policy_version 177986 (0.0011) [2023-12-27 06:53:18,247][20292] KL-divergence is very high: 197.6721 [2023-12-27 06:53:18,261][20381] Updated weights for policy 1, policy_version 177996 (0.0010) [2023-12-27 06:53:18,266][20292] KL-divergence is very high: 218.2567 [2023-12-27 06:53:18,286][20347] Updated weights for policy 0, policy_version 178285 (0.0007) [2023-12-27 06:53:18,290][20292] KL-divergence is very high: 128.0922 [2023-12-27 06:53:18,322][20381] Updated weights for policy 1, policy_version 178006 (0.0009) [2023-12-27 06:53:18,346][20347] Updated weights for policy 0, policy_version 178295 (0.0009) [2023-12-27 06:53:18,408][20347] Updated weights for policy 0, policy_version 178305 (0.0010) [2023-12-27 06:53:18,960][20381] Updated weights for policy 1, policy_version 178016 (0.0005) [2023-12-27 06:53:19,017][20381] Updated weights for policy 1, policy_version 178026 (0.0005) [2023-12-27 06:53:19,081][20381] Updated weights for policy 1, policy_version 178036 (0.0005) [2023-12-27 06:53:19,083][20347] Updated weights for policy 0, policy_version 178315 (0.0009) [2023-12-27 06:53:19,143][20347] Updated weights for policy 0, policy_version 178325 (0.0008) [2023-12-27 06:53:19,205][20347] Updated weights for policy 0, policy_version 178335 (0.0010) [2023-12-27 06:53:19,735][20381] Updated weights for policy 1, policy_version 178046 (0.0009) [2023-12-27 06:53:19,793][20381] Updated weights for policy 1, policy_version 178056 (0.0011) [2023-12-27 06:53:19,857][20381] Updated weights for policy 1, policy_version 178066 (0.0009) [2023-12-27 06:53:19,885][20347] Updated weights for policy 0, policy_version 178345 (0.0010) [2023-12-27 06:53:19,947][20347] Updated weights for policy 0, policy_version 178355 (0.0007) [2023-12-27 06:53:20,006][20347] Updated weights for policy 0, policy_version 178365 (0.0008) [2023-12-27 06:53:20,066][20347] Updated weights for policy 0, policy_version 178375 (0.0007) [2023-12-27 06:53:20,484][20381] Updated weights for policy 1, policy_version 178076 (0.0007) [2023-12-27 06:53:20,542][20381] Updated weights for policy 1, policy_version 178086 (0.0007) [2023-12-27 06:53:20,607][20381] Updated weights for policy 1, policy_version 178096 (0.0009) [2023-12-27 06:53:20,814][20347] Updated weights for policy 0, policy_version 178385 (0.0008) [2023-12-27 06:53:20,875][20347] Updated weights for policy 0, policy_version 178395 (0.0007) [2023-12-27 06:53:20,934][20347] Updated weights for policy 0, policy_version 178405 (0.0008) [2023-12-27 06:53:21,316][20381] Updated weights for policy 1, policy_version 178106 (0.0008) [2023-12-27 06:53:21,360][20292] KL-divergence is very high: 104.8076 [2023-12-27 06:53:21,380][20381] Updated weights for policy 1, policy_version 178116 (0.0008) [2023-12-27 06:53:21,442][20381] Updated weights for policy 1, policy_version 178126 (0.0007) [2023-12-27 06:53:21,497][20381] Updated weights for policy 1, policy_version 178136 (0.0008) [2023-12-27 06:53:21,654][20347] Updated weights for policy 0, policy_version 178415 (0.0008) [2023-12-27 06:53:21,712][20347] Updated weights for policy 0, policy_version 178425 (0.0008) [2023-12-27 06:53:21,778][20347] Updated weights for policy 0, policy_version 178435 (0.0008) [2023-12-27 06:53:22,176][20381] Updated weights for policy 1, policy_version 178146 (0.0006) [2023-12-27 06:53:22,229][20381] Updated weights for policy 1, policy_version 178156 (0.0007) [2023-12-27 06:53:22,291][20381] Updated weights for policy 1, policy_version 178166 (0.0008) [2023-12-27 06:53:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20716.1). Total num frames: 91316224. Throughput: 0: 10388.2, 1: 10400.6. Samples: 91285452. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:22,363][19170] Avg episode reward: [(0, '53.180'), (1, '853.770')] [2023-12-27 06:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000178168_45621248.pth... [2023-12-27 06:53:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000178440_45694976.pth... [2023-12-27 06:53:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000176952_45309952.pth [2023-12-27 06:53:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000177256_45391872.pth [2023-12-27 06:53:22,493][20347] Updated weights for policy 0, policy_version 178445 (0.0008) [2023-12-27 06:53:22,556][20347] Updated weights for policy 0, policy_version 178455 (0.0008) [2023-12-27 06:53:22,620][20347] Updated weights for policy 0, policy_version 178465 (0.0008) [2023-12-27 06:53:22,964][20381] Updated weights for policy 1, policy_version 178176 (0.0010) [2023-12-27 06:53:23,027][20381] Updated weights for policy 1, policy_version 178186 (0.0011) [2023-12-27 06:53:23,090][20381] Updated weights for policy 1, policy_version 178196 (0.0011) [2023-12-27 06:53:23,288][20347] Updated weights for policy 0, policy_version 178475 (0.0007) [2023-12-27 06:53:23,356][20347] Updated weights for policy 0, policy_version 178485 (0.0005) [2023-12-27 06:53:23,412][20347] Updated weights for policy 0, policy_version 178495 (0.0005) [2023-12-27 06:53:23,687][20381] Updated weights for policy 1, policy_version 178206 (0.0009) [2023-12-27 06:53:23,746][20381] Updated weights for policy 1, policy_version 178217 (0.0007) [2023-12-27 06:53:23,788][20381] Updated weights for policy 1, policy_version 178227 (0.0005) [2023-12-27 06:53:23,941][20347] Updated weights for policy 0, policy_version 178505 (0.0005) [2023-12-27 06:53:24,001][20347] Updated weights for policy 0, policy_version 178515 (0.0005) [2023-12-27 06:53:24,060][20347] Updated weights for policy 0, policy_version 178525 (0.0005) [2023-12-27 06:53:24,122][20347] Updated weights for policy 0, policy_version 178535 (0.0006) [2023-12-27 06:53:24,417][20381] Updated weights for policy 1, policy_version 178237 (0.0005) [2023-12-27 06:53:24,471][20381] Updated weights for policy 1, policy_version 178247 (0.0005) [2023-12-27 06:53:24,528][20381] Updated weights for policy 1, policy_version 178257 (0.0005) [2023-12-27 06:53:24,818][20347] Updated weights for policy 0, policy_version 178545 (0.0009) [2023-12-27 06:53:24,863][20347] Updated weights for policy 0, policy_version 178555 (0.0008) [2023-12-27 06:53:24,912][20347] Updated weights for policy 0, policy_version 178565 (0.0009) [2023-12-27 06:53:25,038][20381] Updated weights for policy 1, policy_version 178267 (0.0006) [2023-12-27 06:53:25,094][20381] Updated weights for policy 1, policy_version 178277 (0.0008) [2023-12-27 06:53:25,154][20381] Updated weights for policy 1, policy_version 178287 (0.0008) [2023-12-27 06:53:25,646][20347] Updated weights for policy 0, policy_version 178575 (0.0008) [2023-12-27 06:53:25,700][20347] Updated weights for policy 0, policy_version 178585 (0.0008) [2023-12-27 06:53:25,748][20347] Updated weights for policy 0, policy_version 178595 (0.0007) [2023-12-27 06:53:25,850][20381] Updated weights for policy 1, policy_version 178297 (0.0007) [2023-12-27 06:53:25,895][20381] Updated weights for policy 1, policy_version 178307 (0.0008) [2023-12-27 06:53:25,944][20381] Updated weights for policy 1, policy_version 178317 (0.0008) [2023-12-27 06:53:25,965][20292] KL-divergence is very high: 107.7746 [2023-12-27 06:53:25,994][20381] Updated weights for policy 1, policy_version 178327 (0.0008) [2023-12-27 06:53:26,450][20347] Updated weights for policy 0, policy_version 178605 (0.0007) [2023-12-27 06:53:26,494][20347] Updated weights for policy 0, policy_version 178615 (0.0007) [2023-12-27 06:53:26,541][20347] Updated weights for policy 0, policy_version 178625 (0.0007) [2023-12-27 06:53:26,691][20381] Updated weights for policy 1, policy_version 178337 (0.0008) [2023-12-27 06:53:26,735][20381] Updated weights for policy 1, policy_version 178347 (0.0007) [2023-12-27 06:53:26,784][20381] Updated weights for policy 1, policy_version 178357 (0.0008) [2023-12-27 06:53:27,267][20347] Updated weights for policy 0, policy_version 178635 (0.0008) [2023-12-27 06:53:27,320][20347] Updated weights for policy 0, policy_version 178645 (0.0007) [2023-12-27 06:53:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 91414528. Throughput: 0: 10317.7, 1: 10374.7. Samples: 91408960. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:27,363][19170] Avg episode reward: [(0, '119.360'), (1, '816.920')] [2023-12-27 06:53:27,375][20347] Updated weights for policy 0, policy_version 178655 (0.0005) [2023-12-27 06:53:27,390][20381] Updated weights for policy 1, policy_version 178367 (0.0007) [2023-12-27 06:53:27,443][20381] Updated weights for policy 1, policy_version 178377 (0.0008) [2023-12-27 06:53:27,502][20381] Updated weights for policy 1, policy_version 178387 (0.0008) [2023-12-27 06:53:27,959][20347] Updated weights for policy 0, policy_version 178665 (0.0006) [2023-12-27 06:53:28,012][20347] Updated weights for policy 0, policy_version 178675 (0.0006) [2023-12-27 06:53:28,069][20347] Updated weights for policy 0, policy_version 178685 (0.0007) [2023-12-27 06:53:28,119][20381] Updated weights for policy 1, policy_version 178397 (0.0006) [2023-12-27 06:53:28,119][20347] Updated weights for policy 0, policy_version 178695 (0.0007) [2023-12-27 06:53:28,174][20381] Updated weights for policy 1, policy_version 178407 (0.0006) [2023-12-27 06:53:28,229][20381] Updated weights for policy 1, policy_version 178417 (0.0005) [2023-12-27 06:53:28,840][20347] Updated weights for policy 0, policy_version 178705 (0.0007) [2023-12-27 06:53:28,845][20381] Updated weights for policy 1, policy_version 178427 (0.0006) [2023-12-27 06:53:28,896][20347] Updated weights for policy 0, policy_version 178715 (0.0005) [2023-12-27 06:53:28,898][20381] Updated weights for policy 1, policy_version 178437 (0.0005) [2023-12-27 06:53:28,951][20347] Updated weights for policy 0, policy_version 178725 (0.0005) [2023-12-27 06:53:28,960][20381] Updated weights for policy 1, policy_version 178447 (0.0006) [2023-12-27 06:53:29,595][20381] Updated weights for policy 1, policy_version 178457 (0.0007) [2023-12-27 06:53:29,645][20381] Updated weights for policy 1, policy_version 178467 (0.0008) [2023-12-27 06:53:29,654][20347] Updated weights for policy 0, policy_version 178735 (0.0005) [2023-12-27 06:53:29,700][20381] Updated weights for policy 1, policy_version 178477 (0.0007) [2023-12-27 06:53:29,712][20347] Updated weights for policy 0, policy_version 178745 (0.0006) [2023-12-27 06:53:29,753][20381] Updated weights for policy 1, policy_version 178487 (0.0007) [2023-12-27 06:53:29,771][20347] Updated weights for policy 0, policy_version 178755 (0.0006) [2023-12-27 06:53:30,359][20347] Updated weights for policy 0, policy_version 178765 (0.0006) [2023-12-27 06:53:30,415][20347] Updated weights for policy 0, policy_version 178775 (0.0008) [2023-12-27 06:53:30,472][20347] Updated weights for policy 0, policy_version 178785 (0.0007) [2023-12-27 06:53:30,498][20381] Updated weights for policy 1, policy_version 178497 (0.0008) [2023-12-27 06:53:30,559][20381] Updated weights for policy 1, policy_version 178507 (0.0008) [2023-12-27 06:53:30,615][20381] Updated weights for policy 1, policy_version 178517 (0.0007) [2023-12-27 06:53:31,114][20347] Updated weights for policy 0, policy_version 178795 (0.0007) [2023-12-27 06:53:31,198][20347] Updated weights for policy 0, policy_version 178805 (0.0007) [2023-12-27 06:53:31,263][20347] Updated weights for policy 0, policy_version 178815 (0.0008) [2023-12-27 06:53:31,319][20381] Updated weights for policy 1, policy_version 178527 (0.0007) [2023-12-27 06:53:31,387][20381] Updated weights for policy 1, policy_version 178537 (0.0008) [2023-12-27 06:53:31,447][20381] Updated weights for policy 1, policy_version 178547 (0.0006) [2023-12-27 06:53:31,983][20347] Updated weights for policy 0, policy_version 178825 (0.0007) [2023-12-27 06:53:32,036][20347] Updated weights for policy 0, policy_version 178835 (0.0007) [2023-12-27 06:53:32,087][20347] Updated weights for policy 0, policy_version 178845 (0.0007) [2023-12-27 06:53:32,135][20381] Updated weights for policy 1, policy_version 178557 (0.0008) [2023-12-27 06:53:32,147][20347] Updated weights for policy 0, policy_version 178855 (0.0007) [2023-12-27 06:53:32,194][20381] Updated weights for policy 1, policy_version 178567 (0.0007) [2023-12-27 06:53:32,252][20381] Updated weights for policy 1, policy_version 178577 (0.0009) [2023-12-27 06:53:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 91529216. Throughput: 0: 10317.9, 1: 10385.0. Samples: 91533500. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:32,363][19170] Avg episode reward: [(0, '148.430'), (1, '630.570')] [2023-12-27 06:53:32,839][20347] Updated weights for policy 0, policy_version 178865 (0.0007) [2023-12-27 06:53:32,895][20347] Updated weights for policy 0, policy_version 178875 (0.0008) [2023-12-27 06:53:32,935][20381] Updated weights for policy 1, policy_version 178587 (0.0007) [2023-12-27 06:53:32,952][20347] Updated weights for policy 0, policy_version 178885 (0.0007) [2023-12-27 06:53:32,992][20381] Updated weights for policy 1, policy_version 178597 (0.0006) [2023-12-27 06:53:33,043][20381] Updated weights for policy 1, policy_version 178607 (0.0007) [2023-12-27 06:53:33,611][20347] Updated weights for policy 0, policy_version 178895 (0.0005) [2023-12-27 06:53:33,657][20347] Updated weights for policy 0, policy_version 178905 (0.0005) [2023-12-27 06:53:33,716][20347] Updated weights for policy 0, policy_version 178915 (0.0005) [2023-12-27 06:53:33,762][20381] Updated weights for policy 1, policy_version 178617 (0.0008) [2023-12-27 06:53:33,816][20381] Updated weights for policy 1, policy_version 178627 (0.0008) [2023-12-27 06:53:33,867][20381] Updated weights for policy 1, policy_version 178637 (0.0007) [2023-12-27 06:53:33,922][20381] Updated weights for policy 1, policy_version 178647 (0.0007) [2023-12-27 06:53:34,283][20347] Updated weights for policy 0, policy_version 178925 (0.0007) [2023-12-27 06:53:34,340][20347] Updated weights for policy 0, policy_version 178935 (0.0007) [2023-12-27 06:53:34,393][20347] Updated weights for policy 0, policy_version 178945 (0.0005) [2023-12-27 06:53:34,663][20381] Updated weights for policy 1, policy_version 178657 (0.0008) [2023-12-27 06:53:34,717][20381] Updated weights for policy 1, policy_version 178667 (0.0008) [2023-12-27 06:53:34,774][20381] Updated weights for policy 1, policy_version 178677 (0.0011) [2023-12-27 06:53:35,071][20347] Updated weights for policy 0, policy_version 178955 (0.0007) [2023-12-27 06:53:35,125][20347] Updated weights for policy 0, policy_version 178965 (0.0007) [2023-12-27 06:53:35,175][20347] Updated weights for policy 0, policy_version 178975 (0.0007) [2023-12-27 06:53:35,472][20381] Updated weights for policy 1, policy_version 178687 (0.0007) [2023-12-27 06:53:35,532][20381] Updated weights for policy 1, policy_version 178697 (0.0005) [2023-12-27 06:53:35,587][20381] Updated weights for policy 1, policy_version 178707 (0.0005) [2023-12-27 06:53:35,853][20347] Updated weights for policy 0, policy_version 178985 (0.0007) [2023-12-27 06:53:35,899][20347] Updated weights for policy 0, policy_version 178995 (0.0005) [2023-12-27 06:53:35,952][20347] Updated weights for policy 0, policy_version 179005 (0.0005) [2023-12-27 06:53:36,003][20347] Updated weights for policy 0, policy_version 179015 (0.0005) [2023-12-27 06:53:36,136][20381] Updated weights for policy 1, policy_version 178717 (0.0007) [2023-12-27 06:53:36,197][20381] Updated weights for policy 1, policy_version 178727 (0.0008) [2023-12-27 06:53:36,254][20381] Updated weights for policy 1, policy_version 178737 (0.0007) [2023-12-27 06:53:36,694][20347] Updated weights for policy 0, policy_version 179025 (0.0007) [2023-12-27 06:53:36,747][20347] Updated weights for policy 0, policy_version 179035 (0.0008) [2023-12-27 06:53:36,794][20347] Updated weights for policy 0, policy_version 179045 (0.0008) [2023-12-27 06:53:36,834][20381] Updated weights for policy 1, policy_version 178747 (0.0006) [2023-12-27 06:53:36,890][20381] Updated weights for policy 1, policy_version 178757 (0.0006) [2023-12-27 06:53:36,938][20381] Updated weights for policy 1, policy_version 178767 (0.0005) [2023-12-27 06:53:37,363][19170] Fps is (10 sec: 21298.0, 60 sec: 20616.3, 300 sec: 20743.8). Total num frames: 91627520. Throughput: 0: 10346.2, 1: 10323.1. Samples: 91595292. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-12-27 06:53:37,363][19170] Avg episode reward: [(0, '229.760'), (1, '858.500')] [2023-12-27 06:53:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000178776_45776896.pth... [2023-12-27 06:53:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000179048_45850624.pth... [2023-12-27 06:53:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000177560_45465600.pth [2023-12-27 06:53:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000177864_45547520.pth [2023-12-27 06:53:37,530][20381] Updated weights for policy 1, policy_version 178777 (0.0006) [2023-12-27 06:53:37,564][20347] Updated weights for policy 0, policy_version 179055 (0.0007) [2023-12-27 06:53:37,596][20381] Updated weights for policy 1, policy_version 178787 (0.0005) [2023-12-27 06:53:37,632][20347] Updated weights for policy 0, policy_version 179065 (0.0007) [2023-12-27 06:53:37,653][20381] Updated weights for policy 1, policy_version 178797 (0.0006) [2023-12-27 06:53:37,694][20347] Updated weights for policy 0, policy_version 179075 (0.0006) [2023-12-27 06:53:37,707][20381] Updated weights for policy 1, policy_version 178807 (0.0005) [2023-12-27 06:53:38,304][20347] Updated weights for policy 0, policy_version 179085 (0.0006) [2023-12-27 06:53:38,334][20381] Updated weights for policy 1, policy_version 178817 (0.0007) [2023-12-27 06:53:38,371][20347] Updated weights for policy 0, policy_version 179095 (0.0007) [2023-12-27 06:53:38,391][20381] Updated weights for policy 1, policy_version 178827 (0.0007) [2023-12-27 06:53:38,423][20347] Updated weights for policy 0, policy_version 179105 (0.0005) [2023-12-27 06:53:38,452][20381] Updated weights for policy 1, policy_version 178837 (0.0007) [2023-12-27 06:53:39,001][20347] Updated weights for policy 0, policy_version 179115 (0.0005) [2023-12-27 06:53:39,062][20347] Updated weights for policy 0, policy_version 179125 (0.0005) [2023-12-27 06:53:39,114][20347] Updated weights for policy 0, policy_version 179135 (0.0007) [2023-12-27 06:53:39,186][20381] Updated weights for policy 1, policy_version 178847 (0.0008) [2023-12-27 06:53:39,242][20381] Updated weights for policy 1, policy_version 178857 (0.0007) [2023-12-27 06:53:39,295][20381] Updated weights for policy 1, policy_version 178867 (0.0008) [2023-12-27 06:53:39,803][20347] Updated weights for policy 0, policy_version 179145 (0.0007) [2023-12-27 06:53:39,870][20347] Updated weights for policy 0, policy_version 179155 (0.0007) [2023-12-27 06:53:39,934][20347] Updated weights for policy 0, policy_version 179165 (0.0006) [2023-12-27 06:53:40,000][20347] Updated weights for policy 0, policy_version 179175 (0.0006) [2023-12-27 06:53:40,053][20381] Updated weights for policy 1, policy_version 178877 (0.0008) [2023-12-27 06:53:40,108][20381] Updated weights for policy 1, policy_version 178887 (0.0008) [2023-12-27 06:53:40,163][20381] Updated weights for policy 1, policy_version 178897 (0.0008) [2023-12-27 06:53:40,617][20347] Updated weights for policy 0, policy_version 179185 (0.0007) [2023-12-27 06:53:40,671][20347] Updated weights for policy 0, policy_version 179195 (0.0005) [2023-12-27 06:53:40,724][20347] Updated weights for policy 0, policy_version 179205 (0.0007) [2023-12-27 06:53:40,878][20381] Updated weights for policy 1, policy_version 178907 (0.0009) [2023-12-27 06:53:40,936][20381] Updated weights for policy 1, policy_version 178917 (0.0010) [2023-12-27 06:53:40,992][20381] Updated weights for policy 1, policy_version 178927 (0.0011) [2023-12-27 06:53:41,453][20347] Updated weights for policy 0, policy_version 179215 (0.0008) [2023-12-27 06:53:41,511][20347] Updated weights for policy 0, policy_version 179225 (0.0008) [2023-12-27 06:53:41,570][20347] Updated weights for policy 0, policy_version 179235 (0.0008) [2023-12-27 06:53:41,716][20381] Updated weights for policy 1, policy_version 178937 (0.0010) [2023-12-27 06:53:41,773][20381] Updated weights for policy 1, policy_version 178947 (0.0011) [2023-12-27 06:53:41,829][20381] Updated weights for policy 1, policy_version 178957 (0.0008) [2023-12-27 06:53:41,883][20381] Updated weights for policy 1, policy_version 178967 (0.0007) [2023-12-27 06:53:42,224][20347] Updated weights for policy 0, policy_version 179245 (0.0008) [2023-12-27 06:53:42,289][20347] Updated weights for policy 0, policy_version 179255 (0.0010) [2023-12-27 06:53:42,351][20347] Updated weights for policy 0, policy_version 179265 (0.0008) [2023-12-27 06:53:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 91725824. Throughput: 0: 10328.6, 1: 10350.9. Samples: 91721232. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:53:42,363][19170] Avg episode reward: [(0, '300.280'), (1, '990.190')] [2023-12-27 06:53:42,563][20381] Updated weights for policy 1, policy_version 178977 (0.0006) [2023-12-27 06:53:42,616][20381] Updated weights for policy 1, policy_version 178987 (0.0005) [2023-12-27 06:53:42,676][20381] Updated weights for policy 1, policy_version 178997 (0.0006) [2023-12-27 06:53:43,013][20347] Updated weights for policy 0, policy_version 179275 (0.0008) [2023-12-27 06:53:43,075][20347] Updated weights for policy 0, policy_version 179285 (0.0005) [2023-12-27 06:53:43,133][20347] Updated weights for policy 0, policy_version 179295 (0.0005) [2023-12-27 06:53:43,257][20381] Updated weights for policy 1, policy_version 179007 (0.0005) [2023-12-27 06:53:43,311][20381] Updated weights for policy 1, policy_version 179017 (0.0007) [2023-12-27 06:53:43,363][20381] Updated weights for policy 1, policy_version 179027 (0.0007) [2023-12-27 06:53:43,691][20347] Updated weights for policy 0, policy_version 179305 (0.0006) [2023-12-27 06:53:43,751][20347] Updated weights for policy 0, policy_version 179315 (0.0005) [2023-12-27 06:53:43,806][20347] Updated weights for policy 0, policy_version 179325 (0.0005) [2023-12-27 06:53:43,861][20347] Updated weights for policy 0, policy_version 179335 (0.0007) [2023-12-27 06:53:44,084][20381] Updated weights for policy 1, policy_version 179037 (0.0008) [2023-12-27 06:53:44,133][20381] Updated weights for policy 1, policy_version 179047 (0.0007) [2023-12-27 06:53:44,188][20381] Updated weights for policy 1, policy_version 179057 (0.0008) [2023-12-27 06:53:44,442][20347] Updated weights for policy 0, policy_version 179345 (0.0007) [2023-12-27 06:53:44,491][20347] Updated weights for policy 0, policy_version 179355 (0.0005) [2023-12-27 06:53:44,538][20347] Updated weights for policy 0, policy_version 179365 (0.0007) [2023-12-27 06:53:44,902][20381] Updated weights for policy 1, policy_version 179067 (0.0008) [2023-12-27 06:53:44,963][20381] Updated weights for policy 1, policy_version 179077 (0.0009) [2023-12-27 06:53:45,024][20381] Updated weights for policy 1, policy_version 179087 (0.0007) [2023-12-27 06:53:45,228][20347] Updated weights for policy 0, policy_version 179375 (0.0008) [2023-12-27 06:53:45,296][20347] Updated weights for policy 0, policy_version 179385 (0.0010) [2023-12-27 06:53:45,358][20347] Updated weights for policy 0, policy_version 179395 (0.0010) [2023-12-27 06:53:45,595][20381] Updated weights for policy 1, policy_version 179097 (0.0006) [2023-12-27 06:53:45,658][20381] Updated weights for policy 1, policy_version 179107 (0.0008) [2023-12-27 06:53:45,715][20381] Updated weights for policy 1, policy_version 179117 (0.0007) [2023-12-27 06:53:45,772][20381] Updated weights for policy 1, policy_version 179127 (0.0007) [2023-12-27 06:53:45,964][20347] Updated weights for policy 0, policy_version 179405 (0.0008) [2023-12-27 06:53:46,013][20347] Updated weights for policy 0, policy_version 179415 (0.0006) [2023-12-27 06:53:46,062][20347] Updated weights for policy 0, policy_version 179425 (0.0008) [2023-12-27 06:53:46,456][20381] Updated weights for policy 1, policy_version 179137 (0.0007) [2023-12-27 06:53:46,511][20381] Updated weights for policy 1, policy_version 179147 (0.0007) [2023-12-27 06:53:46,566][20381] Updated weights for policy 1, policy_version 179157 (0.0007) [2023-12-27 06:53:46,748][20347] Updated weights for policy 0, policy_version 179435 (0.0008) [2023-12-27 06:53:46,797][20347] Updated weights for policy 0, policy_version 179445 (0.0008) [2023-12-27 06:53:46,848][20347] Updated weights for policy 0, policy_version 179455 (0.0007) [2023-12-27 06:53:47,259][20381] Updated weights for policy 1, policy_version 179167 (0.0008) [2023-12-27 06:53:47,304][20381] Updated weights for policy 1, policy_version 179177 (0.0008) [2023-12-27 06:53:47,359][20381] Updated weights for policy 1, policy_version 179187 (0.0007) [2023-12-27 06:53:47,362][19170] Fps is (10 sec: 20481.3, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 91832320. Throughput: 0: 10402.9, 1: 10376.8. Samples: 91847056. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:53:47,363][19170] Avg episode reward: [(0, '285.280'), (1, '1000.000')] [2023-12-27 06:53:47,519][20347] Updated weights for policy 0, policy_version 179465 (0.0005) [2023-12-27 06:53:47,575][20347] Updated weights for policy 0, policy_version 179475 (0.0007) [2023-12-27 06:53:47,634][20347] Updated weights for policy 0, policy_version 179485 (0.0008) [2023-12-27 06:53:47,698][20347] Updated weights for policy 0, policy_version 179495 (0.0008) [2023-12-27 06:53:47,993][20381] Updated weights for policy 1, policy_version 179197 (0.0008) [2023-12-27 06:53:48,054][20381] Updated weights for policy 1, policy_version 179207 (0.0008) [2023-12-27 06:53:48,114][20381] Updated weights for policy 1, policy_version 179217 (0.0008) [2023-12-27 06:53:48,434][20347] Updated weights for policy 0, policy_version 179505 (0.0008) [2023-12-27 06:53:48,501][20347] Updated weights for policy 0, policy_version 179515 (0.0008) [2023-12-27 06:53:48,569][20347] Updated weights for policy 0, policy_version 179525 (0.0007) [2023-12-27 06:53:48,827][20381] Updated weights for policy 1, policy_version 179227 (0.0008) [2023-12-27 06:53:48,887][20381] Updated weights for policy 1, policy_version 179237 (0.0008) [2023-12-27 06:53:48,944][20381] Updated weights for policy 1, policy_version 179247 (0.0008) [2023-12-27 06:53:49,185][20347] Updated weights for policy 0, policy_version 179535 (0.0008) [2023-12-27 06:53:49,247][20347] Updated weights for policy 0, policy_version 179545 (0.0007) [2023-12-27 06:53:49,304][20347] Updated weights for policy 0, policy_version 179555 (0.0007) [2023-12-27 06:53:49,663][20381] Updated weights for policy 1, policy_version 179257 (0.0008) [2023-12-27 06:53:49,719][20381] Updated weights for policy 1, policy_version 179267 (0.0008) [2023-12-27 06:53:49,768][20381] Updated weights for policy 1, policy_version 179277 (0.0008) [2023-12-27 06:53:49,828][20381] Updated weights for policy 1, policy_version 179287 (0.0008) [2023-12-27 06:53:50,048][20347] Updated weights for policy 0, policy_version 179565 (0.0009) [2023-12-27 06:53:50,105][20347] Updated weights for policy 0, policy_version 179576 (0.0009) [2023-12-27 06:53:50,165][20347] Updated weights for policy 0, policy_version 179586 (0.0008) [2023-12-27 06:53:50,510][20381] Updated weights for policy 1, policy_version 179297 (0.0007) [2023-12-27 06:53:50,562][20381] Updated weights for policy 1, policy_version 179307 (0.0008) [2023-12-27 06:53:50,625][20381] Updated weights for policy 1, policy_version 179317 (0.0008) [2023-12-27 06:53:50,863][20347] Updated weights for policy 0, policy_version 179596 (0.0007) [2023-12-27 06:53:50,917][20347] Updated weights for policy 0, policy_version 179606 (0.0007) [2023-12-27 06:53:50,972][20347] Updated weights for policy 0, policy_version 179616 (0.0008) [2023-12-27 06:53:51,352][20381] Updated weights for policy 1, policy_version 179327 (0.0009) [2023-12-27 06:53:51,414][20381] Updated weights for policy 1, policy_version 179337 (0.0007) [2023-12-27 06:53:51,471][20381] Updated weights for policy 1, policy_version 179347 (0.0008) [2023-12-27 06:53:51,686][20347] Updated weights for policy 0, policy_version 179626 (0.0007) [2023-12-27 06:53:51,752][20347] Updated weights for policy 0, policy_version 179636 (0.0007) [2023-12-27 06:53:51,809][20347] Updated weights for policy 0, policy_version 179646 (0.0008) [2023-12-27 06:53:51,867][20347] Updated weights for policy 0, policy_version 179656 (0.0008) [2023-12-27 06:53:52,191][20381] Updated weights for policy 1, policy_version 179357 (0.0009) [2023-12-27 06:53:52,247][20381] Updated weights for policy 1, policy_version 179367 (0.0006) [2023-12-27 06:53:52,312][20381] Updated weights for policy 1, policy_version 179377 (0.0008) [2023-12-27 06:53:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 91938816. Throughput: 0: 10336.5, 1: 10357.8. Samples: 91907264. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:53:52,363][19170] Avg episode reward: [(0, '454.790'), (1, '1000.000')] [2023-12-27 06:53:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000179656_46006272.pth... [2023-12-27 06:53:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000179384_45932544.pth... [2023-12-27 06:53:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000178440_45694976.pth [2023-12-27 06:53:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000178168_45621248.pth [2023-12-27 06:53:52,579][20347] Updated weights for policy 0, policy_version 179666 (0.0007) [2023-12-27 06:53:52,637][20347] Updated weights for policy 0, policy_version 179676 (0.0007) [2023-12-27 06:53:52,694][20347] Updated weights for policy 0, policy_version 179686 (0.0007) [2023-12-27 06:53:52,966][20381] Updated weights for policy 1, policy_version 179387 (0.0007) [2023-12-27 06:53:53,016][20381] Updated weights for policy 1, policy_version 179397 (0.0005) [2023-12-27 06:53:53,070][20381] Updated weights for policy 1, policy_version 179407 (0.0005) [2023-12-27 06:53:53,392][20347] Updated weights for policy 0, policy_version 179696 (0.0007) [2023-12-27 06:53:53,447][20347] Updated weights for policy 0, policy_version 179706 (0.0006) [2023-12-27 06:53:53,494][20347] Updated weights for policy 0, policy_version 179716 (0.0005) [2023-12-27 06:53:53,637][20381] Updated weights for policy 1, policy_version 179417 (0.0005) [2023-12-27 06:53:53,690][20381] Updated weights for policy 1, policy_version 179427 (0.0010) [2023-12-27 06:53:53,747][20381] Updated weights for policy 1, policy_version 179437 (0.0010) [2023-12-27 06:53:53,805][20381] Updated weights for policy 1, policy_version 179447 (0.0008) [2023-12-27 06:53:54,152][20347] Updated weights for policy 0, policy_version 179726 (0.0007) [2023-12-27 06:53:54,208][20347] Updated weights for policy 0, policy_version 179736 (0.0008) [2023-12-27 06:53:54,266][20347] Updated weights for policy 0, policy_version 179746 (0.0007) [2023-12-27 06:53:54,454][20381] Updated weights for policy 1, policy_version 179457 (0.0009) [2023-12-27 06:53:54,510][20381] Updated weights for policy 1, policy_version 179467 (0.0009) [2023-12-27 06:53:54,565][20381] Updated weights for policy 1, policy_version 179477 (0.0010) [2023-12-27 06:53:55,041][20347] Updated weights for policy 0, policy_version 179756 (0.0008) [2023-12-27 06:53:55,088][20347] Updated weights for policy 0, policy_version 179766 (0.0006) [2023-12-27 06:53:55,139][20347] Updated weights for policy 0, policy_version 179776 (0.0007) [2023-12-27 06:53:55,198][20381] Updated weights for policy 1, policy_version 179487 (0.0008) [2023-12-27 06:53:55,243][20381] Updated weights for policy 1, policy_version 179497 (0.0008) [2023-12-27 06:53:55,294][20381] Updated weights for policy 1, policy_version 179507 (0.0006) [2023-12-27 06:53:55,709][20347] Updated weights for policy 0, policy_version 179786 (0.0006) [2023-12-27 06:53:55,766][20347] Updated weights for policy 0, policy_version 179796 (0.0005) [2023-12-27 06:53:55,814][20347] Updated weights for policy 0, policy_version 179806 (0.0005) [2023-12-27 06:53:55,866][20347] Updated weights for policy 0, policy_version 179816 (0.0007) [2023-12-27 06:53:55,931][20381] Updated weights for policy 1, policy_version 179517 (0.0007) [2023-12-27 06:53:55,982][20381] Updated weights for policy 1, policy_version 179527 (0.0007) [2023-12-27 06:53:56,027][20381] Updated weights for policy 1, policy_version 179537 (0.0007) [2023-12-27 06:53:56,488][20347] Updated weights for policy 0, policy_version 179826 (0.0007) [2023-12-27 06:53:56,539][20347] Updated weights for policy 0, policy_version 179836 (0.0006) [2023-12-27 06:53:56,585][20347] Updated weights for policy 0, policy_version 179846 (0.0006) [2023-12-27 06:53:56,724][20381] Updated weights for policy 1, policy_version 179547 (0.0007) [2023-12-27 06:53:56,775][20381] Updated weights for policy 1, policy_version 179557 (0.0008) [2023-12-27 06:53:56,819][20381] Updated weights for policy 1, policy_version 179567 (0.0005) [2023-12-27 06:53:57,229][20347] Updated weights for policy 0, policy_version 179856 (0.0008) [2023-12-27 06:53:57,276][20347] Updated weights for policy 0, policy_version 179866 (0.0008) [2023-12-27 06:53:57,330][20347] Updated weights for policy 0, policy_version 179876 (0.0006) [2023-12-27 06:53:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 92045312. Throughput: 0: 10320.8, 1: 10416.5. Samples: 92031320. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:53:57,364][19170] Avg episode reward: [(0, '799.230'), (1, '1000.000')] [2023-12-27 06:53:57,379][20381] Updated weights for policy 1, policy_version 179577 (0.0005) [2023-12-27 06:53:57,435][20381] Updated weights for policy 1, policy_version 179587 (0.0005) [2023-12-27 06:53:57,485][20381] Updated weights for policy 1, policy_version 179597 (0.0005) [2023-12-27 06:53:57,545][20381] Updated weights for policy 1, policy_version 179607 (0.0010) [2023-12-27 06:53:58,029][20347] Updated weights for policy 0, policy_version 179886 (0.0005) [2023-12-27 06:53:58,075][20347] Updated weights for policy 0, policy_version 179896 (0.0007) [2023-12-27 06:53:58,130][20381] Updated weights for policy 1, policy_version 179617 (0.0006) [2023-12-27 06:53:58,133][20347] Updated weights for policy 0, policy_version 179906 (0.0008) [2023-12-27 06:53:58,192][20381] Updated weights for policy 1, policy_version 179627 (0.0007) [2023-12-27 06:53:58,251][20381] Updated weights for policy 1, policy_version 179637 (0.0008) [2023-12-27 06:53:58,841][20347] Updated weights for policy 0, policy_version 179916 (0.0009) [2023-12-27 06:53:58,900][20347] Updated weights for policy 0, policy_version 179926 (0.0010) [2023-12-27 06:53:58,959][20347] Updated weights for policy 0, policy_version 179936 (0.0010) [2023-12-27 06:53:58,960][20381] Updated weights for policy 1, policy_version 179647 (0.0006) [2023-12-27 06:53:59,011][20381] Updated weights for policy 1, policy_version 179657 (0.0010) [2023-12-27 06:53:59,061][20381] Updated weights for policy 1, policy_version 179667 (0.0010) [2023-12-27 06:53:59,635][20347] Updated weights for policy 0, policy_version 179946 (0.0010) [2023-12-27 06:53:59,699][20347] Updated weights for policy 0, policy_version 179956 (0.0010) [2023-12-27 06:53:59,711][20381] Updated weights for policy 1, policy_version 179677 (0.0008) [2023-12-27 06:53:59,756][20347] Updated weights for policy 0, policy_version 179966 (0.0010) [2023-12-27 06:53:59,770][20381] Updated weights for policy 1, policy_version 179687 (0.0005) [2023-12-27 06:53:59,819][20347] Updated weights for policy 0, policy_version 179976 (0.0011) [2023-12-27 06:53:59,833][20381] Updated weights for policy 1, policy_version 179697 (0.0008) [2023-12-27 06:54:00,415][20347] Updated weights for policy 0, policy_version 179986 (0.0008) [2023-12-27 06:54:00,464][20347] Updated weights for policy 0, policy_version 179996 (0.0008) [2023-12-27 06:54:00,473][20381] Updated weights for policy 1, policy_version 179707 (0.0009) [2023-12-27 06:54:00,525][20347] Updated weights for policy 0, policy_version 180006 (0.0005) [2023-12-27 06:54:00,526][20381] Updated weights for policy 1, policy_version 179717 (0.0010) [2023-12-27 06:54:00,574][20381] Updated weights for policy 1, policy_version 179727 (0.0010) [2023-12-27 06:54:01,091][20347] Updated weights for policy 0, policy_version 180016 (0.0008) [2023-12-27 06:54:01,150][20347] Updated weights for policy 0, policy_version 180026 (0.0010) [2023-12-27 06:54:01,198][20347] Updated weights for policy 0, policy_version 180036 (0.0008) [2023-12-27 06:54:01,299][20381] Updated weights for policy 1, policy_version 179737 (0.0010) [2023-12-27 06:54:01,356][20381] Updated weights for policy 1, policy_version 179747 (0.0008) [2023-12-27 06:54:01,414][20381] Updated weights for policy 1, policy_version 179757 (0.0006) [2023-12-27 06:54:01,475][20381] Updated weights for policy 1, policy_version 179767 (0.0007) [2023-12-27 06:54:01,896][20347] Updated weights for policy 0, policy_version 180046 (0.0007) [2023-12-27 06:54:01,959][20347] Updated weights for policy 0, policy_version 180056 (0.0008) [2023-12-27 06:54:02,016][20347] Updated weights for policy 0, policy_version 180066 (0.0008) [2023-12-27 06:54:02,107][20381] Updated weights for policy 1, policy_version 179777 (0.0006) [2023-12-27 06:54:02,161][20381] Updated weights for policy 1, policy_version 179787 (0.0005) [2023-12-27 06:54:02,217][20381] Updated weights for policy 1, policy_version 179797 (0.0005) [2023-12-27 06:54:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 92151808. Throughput: 0: 10307.9, 1: 10472.8. Samples: 92158020. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:02,363][19170] Avg episode reward: [(0, '486.250'), (1, '1000.000')] [2023-12-27 06:54:02,764][20347] Updated weights for policy 0, policy_version 180076 (0.0008) [2023-12-27 06:54:02,813][20347] Updated weights for policy 0, policy_version 180086 (0.0009) [2023-12-27 06:54:02,861][20381] Updated weights for policy 1, policy_version 179807 (0.0009) [2023-12-27 06:54:02,864][20347] Updated weights for policy 0, policy_version 180096 (0.0007) [2023-12-27 06:54:02,923][20381] Updated weights for policy 1, policy_version 179817 (0.0012) [2023-12-27 06:54:02,979][20381] Updated weights for policy 1, policy_version 179827 (0.0008) [2023-12-27 06:54:03,551][20381] Updated weights for policy 1, policy_version 179837 (0.0007) [2023-12-27 06:54:03,582][20347] Updated weights for policy 0, policy_version 180106 (0.0008) [2023-12-27 06:54:03,602][20381] Updated weights for policy 1, policy_version 179847 (0.0007) [2023-12-27 06:54:03,630][20347] Updated weights for policy 0, policy_version 180116 (0.0005) [2023-12-27 06:54:03,659][20381] Updated weights for policy 1, policy_version 179857 (0.0006) [2023-12-27 06:54:03,682][20347] Updated weights for policy 0, policy_version 180126 (0.0006) [2023-12-27 06:54:03,739][20347] Updated weights for policy 0, policy_version 180136 (0.0007) [2023-12-27 06:54:04,323][20381] Updated weights for policy 1, policy_version 179867 (0.0011) [2023-12-27 06:54:04,382][20381] Updated weights for policy 1, policy_version 179877 (0.0011) [2023-12-27 06:54:04,440][20381] Updated weights for policy 1, policy_version 179887 (0.0010) [2023-12-27 06:54:04,469][20347] Updated weights for policy 0, policy_version 180146 (0.0010) [2023-12-27 06:54:04,527][20347] Updated weights for policy 0, policy_version 180156 (0.0010) [2023-12-27 06:54:04,579][20347] Updated weights for policy 0, policy_version 180166 (0.0010) [2023-12-27 06:54:05,086][20381] Updated weights for policy 1, policy_version 179897 (0.0010) [2023-12-27 06:54:05,145][20381] Updated weights for policy 1, policy_version 179907 (0.0010) [2023-12-27 06:54:05,197][20381] Updated weights for policy 1, policy_version 179917 (0.0009) [2023-12-27 06:54:05,209][20347] Updated weights for policy 0, policy_version 180176 (0.0007) [2023-12-27 06:54:05,259][20381] Updated weights for policy 1, policy_version 179927 (0.0010) [2023-12-27 06:54:05,272][20347] Updated weights for policy 0, policy_version 180186 (0.0005) [2023-12-27 06:54:05,329][20347] Updated weights for policy 0, policy_version 180196 (0.0005) [2023-12-27 06:54:05,912][20347] Updated weights for policy 0, policy_version 180206 (0.0006) [2023-12-27 06:54:05,925][20381] Updated weights for policy 1, policy_version 179937 (0.0010) [2023-12-27 06:54:05,964][20347] Updated weights for policy 0, policy_version 180216 (0.0005) [2023-12-27 06:54:05,971][20381] Updated weights for policy 1, policy_version 179947 (0.0010) [2023-12-27 06:54:06,017][20381] Updated weights for policy 1, policy_version 179957 (0.0010) [2023-12-27 06:54:06,018][20347] Updated weights for policy 0, policy_version 180226 (0.0005) [2023-12-27 06:54:06,621][20347] Updated weights for policy 0, policy_version 180236 (0.0005) [2023-12-27 06:54:06,672][20347] Updated weights for policy 0, policy_version 180246 (0.0007) [2023-12-27 06:54:06,716][20347] Updated weights for policy 0, policy_version 180256 (0.0007) [2023-12-27 06:54:06,748][20381] Updated weights for policy 1, policy_version 179967 (0.0011) [2023-12-27 06:54:06,802][20381] Updated weights for policy 1, policy_version 179977 (0.0009) [2023-12-27 06:54:06,857][20381] Updated weights for policy 1, policy_version 179987 (0.0009) [2023-12-27 06:54:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 92250112. Throughput: 0: 10303.7, 1: 10474.3. Samples: 92220460. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:07,363][19170] Avg episode reward: [(0, '399.950'), (1, '1000.000')] [2023-12-27 06:54:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000180264_46161920.pth... [2023-12-27 06:54:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000179992_46088192.pth... [2023-12-27 06:54:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000179048_45850624.pth [2023-12-27 06:54:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000178776_45776896.pth [2023-12-27 06:54:07,409][20347] Updated weights for policy 0, policy_version 180266 (0.0006) [2023-12-27 06:54:07,461][20347] Updated weights for policy 0, policy_version 180276 (0.0005) [2023-12-27 06:54:07,515][20347] Updated weights for policy 0, policy_version 180286 (0.0007) [2023-12-27 06:54:07,524][20381] Updated weights for policy 1, policy_version 179997 (0.0008) [2023-12-27 06:54:07,566][20347] Updated weights for policy 0, policy_version 180296 (0.0010) [2023-12-27 06:54:07,579][20381] Updated weights for policy 1, policy_version 180007 (0.0007) [2023-12-27 06:54:07,635][20381] Updated weights for policy 1, policy_version 180017 (0.0010) [2023-12-27 06:54:08,233][20347] Updated weights for policy 0, policy_version 180306 (0.0010) [2023-12-27 06:54:08,287][20347] Updated weights for policy 0, policy_version 180316 (0.0010) [2023-12-27 06:54:08,308][20381] Updated weights for policy 1, policy_version 180027 (0.0009) [2023-12-27 06:54:08,342][20347] Updated weights for policy 0, policy_version 180326 (0.0011) [2023-12-27 06:54:08,374][20381] Updated weights for policy 1, policy_version 180037 (0.0006) [2023-12-27 06:54:08,423][20381] Updated weights for policy 1, policy_version 180047 (0.0007) [2023-12-27 06:54:09,011][20347] Updated weights for policy 0, policy_version 180336 (0.0008) [2023-12-27 06:54:09,072][20347] Updated weights for policy 0, policy_version 180346 (0.0006) [2023-12-27 06:54:09,074][20381] Updated weights for policy 1, policy_version 180057 (0.0008) [2023-12-27 06:54:09,130][20381] Updated weights for policy 1, policy_version 180067 (0.0005) [2023-12-27 06:54:09,139][20347] Updated weights for policy 0, policy_version 180356 (0.0007) [2023-12-27 06:54:09,186][20381] Updated weights for policy 1, policy_version 180077 (0.0005) [2023-12-27 06:54:09,250][20381] Updated weights for policy 1, policy_version 180087 (0.0008) [2023-12-27 06:54:09,727][20347] Updated weights for policy 0, policy_version 180366 (0.0008) [2023-12-27 06:54:09,777][20347] Updated weights for policy 0, policy_version 180376 (0.0008) [2023-12-27 06:54:09,840][20347] Updated weights for policy 0, policy_version 180386 (0.0008) [2023-12-27 06:54:09,913][20381] Updated weights for policy 1, policy_version 180097 (0.0008) [2023-12-27 06:54:09,975][20381] Updated weights for policy 1, policy_version 180107 (0.0009) [2023-12-27 06:54:10,027][20381] Updated weights for policy 1, policy_version 180117 (0.0008) [2023-12-27 06:54:10,534][20347] Updated weights for policy 0, policy_version 180396 (0.0008) [2023-12-27 06:54:10,593][20347] Updated weights for policy 0, policy_version 180406 (0.0008) [2023-12-27 06:54:10,644][20347] Updated weights for policy 0, policy_version 180416 (0.0007) [2023-12-27 06:54:10,756][20381] Updated weights for policy 1, policy_version 180127 (0.0007) [2023-12-27 06:54:10,816][20381] Updated weights for policy 1, policy_version 180137 (0.0008) [2023-12-27 06:54:10,869][20381] Updated weights for policy 1, policy_version 180147 (0.0008) [2023-12-27 06:54:11,303][20347] Updated weights for policy 0, policy_version 180426 (0.0007) [2023-12-27 06:54:11,362][20347] Updated weights for policy 0, policy_version 180436 (0.0007) [2023-12-27 06:54:11,427][20347] Updated weights for policy 0, policy_version 180446 (0.0007) [2023-12-27 06:54:11,485][20347] Updated weights for policy 0, policy_version 180456 (0.0008) [2023-12-27 06:54:11,581][20381] Updated weights for policy 1, policy_version 180157 (0.0009) [2023-12-27 06:54:11,643][20381] Updated weights for policy 1, policy_version 180167 (0.0008) [2023-12-27 06:54:11,708][20381] Updated weights for policy 1, policy_version 180177 (0.0008) [2023-12-27 06:54:12,191][20347] Updated weights for policy 0, policy_version 180466 (0.0005) [2023-12-27 06:54:12,256][20347] Updated weights for policy 0, policy_version 180476 (0.0006) [2023-12-27 06:54:12,324][20347] Updated weights for policy 0, policy_version 180486 (0.0008) [2023-12-27 06:54:12,353][20381] Updated weights for policy 1, policy_version 180187 (0.0008) [2023-12-27 06:54:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 92356608. Throughput: 0: 10405.2, 1: 10400.5. Samples: 92345220. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:12,363][19170] Avg episode reward: [(0, '460.340'), (1, '1000.000')] [2023-12-27 06:54:12,413][20381] Updated weights for policy 1, policy_version 180197 (0.0008) [2023-12-27 06:54:12,469][20381] Updated weights for policy 1, policy_version 180207 (0.0008) [2023-12-27 06:54:12,959][20347] Updated weights for policy 0, policy_version 180496 (0.0006) [2023-12-27 06:54:13,008][20347] Updated weights for policy 0, policy_version 180506 (0.0008) [2023-12-27 06:54:13,063][20347] Updated weights for policy 0, policy_version 180516 (0.0007) [2023-12-27 06:54:13,175][20381] Updated weights for policy 1, policy_version 180217 (0.0008) [2023-12-27 06:54:13,227][20381] Updated weights for policy 1, policy_version 180227 (0.0006) [2023-12-27 06:54:13,280][20381] Updated weights for policy 1, policy_version 180237 (0.0008) [2023-12-27 06:54:13,332][20381] Updated weights for policy 1, policy_version 180247 (0.0008) [2023-12-27 06:54:13,686][20347] Updated weights for policy 0, policy_version 180526 (0.0006) [2023-12-27 06:54:13,734][20347] Updated weights for policy 0, policy_version 180536 (0.0007) [2023-12-27 06:54:13,782][20347] Updated weights for policy 0, policy_version 180546 (0.0008) [2023-12-27 06:54:14,055][20381] Updated weights for policy 1, policy_version 180257 (0.0008) [2023-12-27 06:54:14,111][20381] Updated weights for policy 1, policy_version 180267 (0.0007) [2023-12-27 06:54:14,161][20381] Updated weights for policy 1, policy_version 180277 (0.0008) [2023-12-27 06:54:14,416][20347] Updated weights for policy 0, policy_version 180556 (0.0007) [2023-12-27 06:54:14,473][20347] Updated weights for policy 0, policy_version 180566 (0.0008) [2023-12-27 06:54:14,521][20347] Updated weights for policy 0, policy_version 180576 (0.0008) [2023-12-27 06:54:14,825][20381] Updated weights for policy 1, policy_version 180287 (0.0008) [2023-12-27 06:54:14,887][20381] Updated weights for policy 1, policy_version 180297 (0.0008) [2023-12-27 06:54:14,945][20381] Updated weights for policy 1, policy_version 180307 (0.0008) [2023-12-27 06:54:15,236][20347] Updated weights for policy 0, policy_version 180586 (0.0007) [2023-12-27 06:54:15,293][20347] Updated weights for policy 0, policy_version 180596 (0.0007) [2023-12-27 06:54:15,348][20347] Updated weights for policy 0, policy_version 180606 (0.0007) [2023-12-27 06:54:15,407][20347] Updated weights for policy 0, policy_version 180616 (0.0007) [2023-12-27 06:54:15,677][20381] Updated weights for policy 1, policy_version 180317 (0.0009) [2023-12-27 06:54:15,723][20381] Updated weights for policy 1, policy_version 180327 (0.0010) [2023-12-27 06:54:15,772][20381] Updated weights for policy 1, policy_version 180337 (0.0010) [2023-12-27 06:54:15,970][20347] Updated weights for policy 0, policy_version 180626 (0.0008) [2023-12-27 06:54:16,029][20347] Updated weights for policy 0, policy_version 180636 (0.0007) [2023-12-27 06:54:16,095][20347] Updated weights for policy 0, policy_version 180646 (0.0009) [2023-12-27 06:54:16,421][20381] Updated weights for policy 1, policy_version 180347 (0.0007) [2023-12-27 06:54:16,471][20381] Updated weights for policy 1, policy_version 180357 (0.0008) [2023-12-27 06:54:16,523][20381] Updated weights for policy 1, policy_version 180367 (0.0009) [2023-12-27 06:54:16,751][20347] Updated weights for policy 0, policy_version 180656 (0.0007) [2023-12-27 06:54:16,813][20347] Updated weights for policy 0, policy_version 180666 (0.0008) [2023-12-27 06:54:16,869][20347] Updated weights for policy 0, policy_version 180676 (0.0007) [2023-12-27 06:54:17,158][20381] Updated weights for policy 1, policy_version 180377 (0.0008) [2023-12-27 06:54:17,213][20381] Updated weights for policy 1, policy_version 180387 (0.0008) [2023-12-27 06:54:17,260][20381] Updated weights for policy 1, policy_version 180397 (0.0005) [2023-12-27 06:54:17,313][20381] Updated weights for policy 1, policy_version 180407 (0.0007) [2023-12-27 06:54:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 92463104. Throughput: 0: 10440.7, 1: 10360.6. Samples: 92469560. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:17,363][19170] Avg episode reward: [(0, '772.230'), (1, '1000.000')] [2023-12-27 06:54:17,430][20347] Updated weights for policy 0, policy_version 180686 (0.0009) [2023-12-27 06:54:17,491][20347] Updated weights for policy 0, policy_version 180696 (0.0008) [2023-12-27 06:54:17,542][20347] Updated weights for policy 0, policy_version 180706 (0.0008) [2023-12-27 06:54:17,950][20381] Updated weights for policy 1, policy_version 180417 (0.0009) [2023-12-27 06:54:18,003][20381] Updated weights for policy 1, policy_version 180427 (0.0011) [2023-12-27 06:54:18,056][20381] Updated weights for policy 1, policy_version 180437 (0.0011) [2023-12-27 06:54:18,208][20347] Updated weights for policy 0, policy_version 180716 (0.0007) [2023-12-27 06:54:18,268][20347] Updated weights for policy 0, policy_version 180726 (0.0007) [2023-12-27 06:54:18,317][20347] Updated weights for policy 0, policy_version 180736 (0.0007) [2023-12-27 06:54:18,796][20381] Updated weights for policy 1, policy_version 180447 (0.0009) [2023-12-27 06:54:18,850][20381] Updated weights for policy 1, policy_version 180457 (0.0007) [2023-12-27 06:54:18,910][20381] Updated weights for policy 1, policy_version 180467 (0.0008) [2023-12-27 06:54:19,054][20347] Updated weights for policy 0, policy_version 180746 (0.0007) [2023-12-27 06:54:19,118][20347] Updated weights for policy 0, policy_version 180756 (0.0008) [2023-12-27 06:54:19,182][20347] Updated weights for policy 0, policy_version 180766 (0.0008) [2023-12-27 06:54:19,241][20347] Updated weights for policy 0, policy_version 180776 (0.0008) [2023-12-27 06:54:19,594][20381] Updated weights for policy 1, policy_version 180477 (0.0008) [2023-12-27 06:54:19,662][20381] Updated weights for policy 1, policy_version 180487 (0.0008) [2023-12-27 06:54:19,726][20381] Updated weights for policy 1, policy_version 180497 (0.0008) [2023-12-27 06:54:19,993][20347] Updated weights for policy 0, policy_version 180786 (0.0008) [2023-12-27 06:54:20,051][20347] Updated weights for policy 0, policy_version 180796 (0.0008) [2023-12-27 06:54:20,106][20347] Updated weights for policy 0, policy_version 180806 (0.0008) [2023-12-27 06:54:20,418][20381] Updated weights for policy 1, policy_version 180507 (0.0008) [2023-12-27 06:54:20,474][20381] Updated weights for policy 1, policy_version 180517 (0.0007) [2023-12-27 06:54:20,534][20381] Updated weights for policy 1, policy_version 180527 (0.0005) [2023-12-27 06:54:20,861][20347] Updated weights for policy 0, policy_version 180816 (0.0007) [2023-12-27 06:54:20,925][20347] Updated weights for policy 0, policy_version 180826 (0.0008) [2023-12-27 06:54:20,991][20347] Updated weights for policy 0, policy_version 180836 (0.0007) [2023-12-27 06:54:21,180][20381] Updated weights for policy 1, policy_version 180537 (0.0006) [2023-12-27 06:54:21,245][20381] Updated weights for policy 1, policy_version 180547 (0.0011) [2023-12-27 06:54:21,307][20381] Updated weights for policy 1, policy_version 180557 (0.0009) [2023-12-27 06:54:21,394][20381] Updated weights for policy 1, policy_version 180567 (0.0008) [2023-12-27 06:54:21,628][20347] Updated weights for policy 0, policy_version 180846 (0.0007) [2023-12-27 06:54:21,684][20347] Updated weights for policy 0, policy_version 180856 (0.0008) [2023-12-27 06:54:21,747][20347] Updated weights for policy 0, policy_version 180866 (0.0007) [2023-12-27 06:54:22,142][20381] Updated weights for policy 1, policy_version 180577 (0.0008) [2023-12-27 06:54:22,199][20381] Updated weights for policy 1, policy_version 180587 (0.0008) [2023-12-27 06:54:22,251][20381] Updated weights for policy 1, policy_version 180597 (0.0007) [2023-12-27 06:54:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 92561408. Throughput: 0: 10389.4, 1: 10383.6. Samples: 92530068. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:22,363][19170] Avg episode reward: [(0, '776.150'), (1, '1000.000')] [2023-12-27 06:54:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000180872_46317568.pth... [2023-12-27 06:54:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000180600_46243840.pth... [2023-12-27 06:54:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000179384_45932544.pth [2023-12-27 06:54:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000179656_46006272.pth [2023-12-27 06:54:22,420][20347] Updated weights for policy 0, policy_version 180876 (0.0007) [2023-12-27 06:54:22,488][20347] Updated weights for policy 0, policy_version 180886 (0.0006) [2023-12-27 06:54:22,557][20347] Updated weights for policy 0, policy_version 180896 (0.0005) [2023-12-27 06:54:22,988][20381] Updated weights for policy 1, policy_version 180607 (0.0007) [2023-12-27 06:54:23,048][20381] Updated weights for policy 1, policy_version 180617 (0.0007) [2023-12-27 06:54:23,099][20381] Updated weights for policy 1, policy_version 180627 (0.0008) [2023-12-27 06:54:23,159][20347] Updated weights for policy 0, policy_version 180906 (0.0006) [2023-12-27 06:54:23,218][20347] Updated weights for policy 0, policy_version 180916 (0.0008) [2023-12-27 06:54:23,276][20347] Updated weights for policy 0, policy_version 180926 (0.0008) [2023-12-27 06:54:23,335][20347] Updated weights for policy 0, policy_version 180936 (0.0008) [2023-12-27 06:54:23,662][20381] Updated weights for policy 1, policy_version 180637 (0.0007) [2023-12-27 06:54:23,713][20381] Updated weights for policy 1, policy_version 180647 (0.0005) [2023-12-27 06:54:23,762][20381] Updated weights for policy 1, policy_version 180657 (0.0005) [2023-12-27 06:54:24,002][20347] Updated weights for policy 0, policy_version 180946 (0.0006) [2023-12-27 06:54:24,054][20347] Updated weights for policy 0, policy_version 180956 (0.0005) [2023-12-27 06:54:24,107][20347] Updated weights for policy 0, policy_version 180966 (0.0006) [2023-12-27 06:54:24,288][20381] Updated weights for policy 1, policy_version 180667 (0.0007) [2023-12-27 06:54:24,344][20381] Updated weights for policy 1, policy_version 180677 (0.0010) [2023-12-27 06:54:24,394][20381] Updated weights for policy 1, policy_version 180687 (0.0009) [2023-12-27 06:54:24,637][20347] Updated weights for policy 0, policy_version 180976 (0.0006) [2023-12-27 06:54:24,700][20347] Updated weights for policy 0, policy_version 180986 (0.0008) [2023-12-27 06:54:24,761][20347] Updated weights for policy 0, policy_version 180996 (0.0008) [2023-12-27 06:54:25,072][20381] Updated weights for policy 1, policy_version 180697 (0.0010) [2023-12-27 06:54:25,131][20381] Updated weights for policy 1, policy_version 180707 (0.0010) [2023-12-27 06:54:25,187][20381] Updated weights for policy 1, policy_version 180717 (0.0011) [2023-12-27 06:54:25,242][20381] Updated weights for policy 1, policy_version 180727 (0.0010) [2023-12-27 06:54:25,451][20347] Updated weights for policy 0, policy_version 181006 (0.0008) [2023-12-27 06:54:25,506][20347] Updated weights for policy 0, policy_version 181016 (0.0007) [2023-12-27 06:54:25,564][20347] Updated weights for policy 0, policy_version 181026 (0.0007) [2023-12-27 06:54:25,908][20381] Updated weights for policy 1, policy_version 180737 (0.0007) [2023-12-27 06:54:25,960][20381] Updated weights for policy 1, policy_version 180747 (0.0008) [2023-12-27 06:54:26,021][20381] Updated weights for policy 1, policy_version 180757 (0.0010) [2023-12-27 06:54:26,211][20347] Updated weights for policy 0, policy_version 181036 (0.0006) [2023-12-27 06:54:26,253][20347] Updated weights for policy 0, policy_version 181046 (0.0005) [2023-12-27 06:54:26,301][20347] Updated weights for policy 0, policy_version 181056 (0.0005) [2023-12-27 06:54:26,544][20381] Updated weights for policy 1, policy_version 180767 (0.0006) [2023-12-27 06:54:26,595][20381] Updated weights for policy 1, policy_version 180777 (0.0010) [2023-12-27 06:54:26,646][20381] Updated weights for policy 1, policy_version 180787 (0.0010) [2023-12-27 06:54:26,923][20347] Updated weights for policy 0, policy_version 181066 (0.0006) [2023-12-27 06:54:26,973][20347] Updated weights for policy 0, policy_version 181076 (0.0007) [2023-12-27 06:54:27,019][20347] Updated weights for policy 0, policy_version 181086 (0.0007) [2023-12-27 06:54:27,073][20347] Updated weights for policy 0, policy_version 181096 (0.0007) [2023-12-27 06:54:27,297][20381] Updated weights for policy 1, policy_version 180797 (0.0010) [2023-12-27 06:54:27,349][20381] Updated weights for policy 1, policy_version 180807 (0.0010) [2023-12-27 06:54:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 92667904. Throughput: 0: 10400.7, 1: 10396.7. Samples: 92657116. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:27,363][19170] Avg episode reward: [(0, '925.900'), (1, '1000.000')] [2023-12-27 06:54:27,401][20381] Updated weights for policy 1, policy_version 180817 (0.0010) [2023-12-27 06:54:27,764][20347] Updated weights for policy 0, policy_version 181106 (0.0009) [2023-12-27 06:54:27,823][20347] Updated weights for policy 0, policy_version 181116 (0.0007) [2023-12-27 06:54:27,885][20347] Updated weights for policy 0, policy_version 181126 (0.0006) [2023-12-27 06:54:28,029][20381] Updated weights for policy 1, policy_version 180827 (0.0010) [2023-12-27 06:54:28,079][20381] Updated weights for policy 1, policy_version 180837 (0.0007) [2023-12-27 06:54:28,134][20381] Updated weights for policy 1, policy_version 180847 (0.0008) [2023-12-27 06:54:28,571][20347] Updated weights for policy 0, policy_version 181136 (0.0007) [2023-12-27 06:54:28,626][20347] Updated weights for policy 0, policy_version 181146 (0.0005) [2023-12-27 06:54:28,681][20347] Updated weights for policy 0, policy_version 181156 (0.0008) [2023-12-27 06:54:28,788][20381] Updated weights for policy 1, policy_version 180857 (0.0007) [2023-12-27 06:54:28,850][20381] Updated weights for policy 1, policy_version 180867 (0.0008) [2023-12-27 06:54:28,906][20381] Updated weights for policy 1, policy_version 180877 (0.0007) [2023-12-27 06:54:28,959][20381] Updated weights for policy 1, policy_version 180887 (0.0007) [2023-12-27 06:54:29,410][20347] Updated weights for policy 0, policy_version 181166 (0.0007) [2023-12-27 06:54:29,462][20347] Updated weights for policy 0, policy_version 181176 (0.0008) [2023-12-27 06:54:29,516][20347] Updated weights for policy 0, policy_version 181186 (0.0008) [2023-12-27 06:54:29,601][20381] Updated weights for policy 1, policy_version 180897 (0.0006) [2023-12-27 06:54:29,659][20381] Updated weights for policy 1, policy_version 180907 (0.0007) [2023-12-27 06:54:29,720][20381] Updated weights for policy 1, policy_version 180917 (0.0005) [2023-12-27 06:54:30,192][20347] Updated weights for policy 0, policy_version 181196 (0.0008) [2023-12-27 06:54:30,240][20347] Updated weights for policy 0, policy_version 181206 (0.0010) [2023-12-27 06:54:30,295][20347] Updated weights for policy 0, policy_version 181216 (0.0010) [2023-12-27 06:54:30,363][20381] Updated weights for policy 1, policy_version 180927 (0.0006) [2023-12-27 06:54:30,418][20381] Updated weights for policy 1, policy_version 180937 (0.0007) [2023-12-27 06:54:30,475][20381] Updated weights for policy 1, policy_version 180947 (0.0007) [2023-12-27 06:54:30,975][20347] Updated weights for policy 0, policy_version 181226 (0.0010) [2023-12-27 06:54:31,025][20347] Updated weights for policy 0, policy_version 181236 (0.0010) [2023-12-27 06:54:31,087][20347] Updated weights for policy 0, policy_version 181246 (0.0011) [2023-12-27 06:54:31,145][20347] Updated weights for policy 0, policy_version 181256 (0.0011) [2023-12-27 06:54:31,165][20381] Updated weights for policy 1, policy_version 180957 (0.0007) [2023-12-27 06:54:31,210][20381] Updated weights for policy 1, policy_version 180967 (0.0007) [2023-12-27 06:54:31,263][20381] Updated weights for policy 1, policy_version 180977 (0.0007) [2023-12-27 06:54:31,818][20347] Updated weights for policy 0, policy_version 181266 (0.0010) [2023-12-27 06:54:31,880][20347] Updated weights for policy 0, policy_version 181276 (0.0010) [2023-12-27 06:54:31,900][20381] Updated weights for policy 1, policy_version 180987 (0.0008) [2023-12-27 06:54:31,934][20347] Updated weights for policy 0, policy_version 181286 (0.0010) [2023-12-27 06:54:31,959][20381] Updated weights for policy 1, policy_version 180997 (0.0006) [2023-12-27 06:54:32,017][20381] Updated weights for policy 1, policy_version 181007 (0.0008) [2023-12-27 06:54:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 92774400. Throughput: 0: 10354.5, 1: 10458.7. Samples: 92783656. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:54:32,619][20347] Updated weights for policy 0, policy_version 181296 (0.0010) [2023-12-27 06:54:32,665][20381] Updated weights for policy 1, policy_version 181017 (0.0007) [2023-12-27 06:54:32,675][20347] Updated weights for policy 0, policy_version 181306 (0.0010) [2023-12-27 06:54:32,725][20381] Updated weights for policy 1, policy_version 181027 (0.0005) [2023-12-27 06:54:32,735][20347] Updated weights for policy 0, policy_version 181316 (0.0008) [2023-12-27 06:54:32,787][20381] Updated weights for policy 1, policy_version 181037 (0.0005) [2023-12-27 06:54:32,852][20381] Updated weights for policy 1, policy_version 181047 (0.0007) [2023-12-27 06:54:33,418][20381] Updated weights for policy 1, policy_version 181057 (0.0006) [2023-12-27 06:54:33,420][20347] Updated weights for policy 0, policy_version 181326 (0.0008) [2023-12-27 06:54:33,468][20381] Updated weights for policy 1, policy_version 181067 (0.0005) [2023-12-27 06:54:33,474][20347] Updated weights for policy 0, policy_version 181336 (0.0007) [2023-12-27 06:54:33,518][20381] Updated weights for policy 1, policy_version 181077 (0.0005) [2023-12-27 06:54:33,533][20347] Updated weights for policy 0, policy_version 181346 (0.0008) [2023-12-27 06:54:34,058][20381] Updated weights for policy 1, policy_version 181087 (0.0005) [2023-12-27 06:54:34,115][20381] Updated weights for policy 1, policy_version 181097 (0.0006) [2023-12-27 06:54:34,182][20381] Updated weights for policy 1, policy_version 181107 (0.0009) [2023-12-27 06:54:34,185][20347] Updated weights for policy 0, policy_version 181356 (0.0010) [2023-12-27 06:54:34,250][20347] Updated weights for policy 0, policy_version 181366 (0.0011) [2023-12-27 06:54:34,310][20347] Updated weights for policy 0, policy_version 181376 (0.0010) [2023-12-27 06:54:34,894][20381] Updated weights for policy 1, policy_version 181117 (0.0007) [2023-12-27 06:54:34,956][20381] Updated weights for policy 1, policy_version 181127 (0.0007) [2023-12-27 06:54:34,994][20347] Updated weights for policy 0, policy_version 181386 (0.0009) [2023-12-27 06:54:35,014][20381] Updated weights for policy 1, policy_version 181137 (0.0005) [2023-12-27 06:54:35,052][20347] Updated weights for policy 0, policy_version 181396 (0.0006) [2023-12-27 06:54:35,110][20347] Updated weights for policy 0, policy_version 181406 (0.0010) [2023-12-27 06:54:35,175][20347] Updated weights for policy 0, policy_version 181416 (0.0010) [2023-12-27 06:54:35,555][20381] Updated weights for policy 1, policy_version 181147 (0.0006) [2023-12-27 06:54:35,610][20381] Updated weights for policy 1, policy_version 181157 (0.0008) [2023-12-27 06:54:35,666][20381] Updated weights for policy 1, policy_version 181167 (0.0008) [2023-12-27 06:54:35,802][20347] Updated weights for policy 0, policy_version 181426 (0.0007) [2023-12-27 06:54:35,860][20347] Updated weights for policy 0, policy_version 181436 (0.0007) [2023-12-27 06:54:35,918][20347] Updated weights for policy 0, policy_version 181446 (0.0008) [2023-12-27 06:54:36,398][20381] Updated weights for policy 1, policy_version 181177 (0.0008) [2023-12-27 06:54:36,455][20381] Updated weights for policy 1, policy_version 181187 (0.0008) [2023-12-27 06:54:36,501][20347] Updated weights for policy 0, policy_version 181456 (0.0006) [2023-12-27 06:54:36,503][20381] Updated weights for policy 1, policy_version 181197 (0.0006) [2023-12-27 06:54:36,554][20381] Updated weights for policy 1, policy_version 181207 (0.0008) [2023-12-27 06:54:36,562][20347] Updated weights for policy 0, policy_version 181466 (0.0005) [2023-12-27 06:54:36,620][20347] Updated weights for policy 0, policy_version 181476 (0.0006) [2023-12-27 06:54:37,231][20381] Updated weights for policy 1, policy_version 181217 (0.0008) [2023-12-27 06:54:37,261][20347] Updated weights for policy 0, policy_version 181486 (0.0009) [2023-12-27 06:54:37,282][20381] Updated weights for policy 1, policy_version 181227 (0.0006) [2023-12-27 06:54:37,318][20347] Updated weights for policy 0, policy_version 181496 (0.0011) [2023-12-27 06:54:37,338][20381] Updated weights for policy 1, policy_version 181237 (0.0005) [2023-12-27 06:54:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.8, 300 sec: 20771.6). Total num frames: 92880896. Throughput: 0: 10373.3, 1: 10521.0. Samples: 92847508. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:54:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000181240_46407680.pth... [2023-12-27 06:54:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000179992_46088192.pth [2023-12-27 06:54:37,374][20347] Updated weights for policy 0, policy_version 181506 (0.0010) [2023-12-27 06:54:37,407][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000181512_46481408.pth... [2023-12-27 06:54:37,412][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000180264_46161920.pth [2023-12-27 06:54:38,038][20381] Updated weights for policy 1, policy_version 181247 (0.0006) [2023-12-27 06:54:38,054][20347] Updated weights for policy 0, policy_version 181516 (0.0010) [2023-12-27 06:54:38,081][20381] Updated weights for policy 1, policy_version 181257 (0.0007) [2023-12-27 06:54:38,109][20347] Updated weights for policy 0, policy_version 181526 (0.0010) [2023-12-27 06:54:38,134][20381] Updated weights for policy 1, policy_version 181267 (0.0006) [2023-12-27 06:54:38,168][20347] Updated weights for policy 0, policy_version 181536 (0.0011) [2023-12-27 06:54:38,785][20381] Updated weights for policy 1, policy_version 181277 (0.0005) [2023-12-27 06:54:38,785][20347] Updated weights for policy 0, policy_version 181546 (0.0010) [2023-12-27 06:54:38,843][20347] Updated weights for policy 0, policy_version 181556 (0.0007) [2023-12-27 06:54:38,844][20381] Updated weights for policy 1, policy_version 181287 (0.0006) [2023-12-27 06:54:38,896][20347] Updated weights for policy 0, policy_version 181566 (0.0008) [2023-12-27 06:54:38,904][20381] Updated weights for policy 1, policy_version 181297 (0.0005) [2023-12-27 06:54:38,952][20347] Updated weights for policy 0, policy_version 181576 (0.0007) [2023-12-27 06:54:39,552][20381] Updated weights for policy 1, policy_version 181307 (0.0006) [2023-12-27 06:54:39,608][20381] Updated weights for policy 1, policy_version 181317 (0.0008) [2023-12-27 06:54:39,622][20347] Updated weights for policy 0, policy_version 181586 (0.0006) [2023-12-27 06:54:39,667][20381] Updated weights for policy 1, policy_version 181327 (0.0008) [2023-12-27 06:54:39,678][20347] Updated weights for policy 0, policy_version 181596 (0.0006) [2023-12-27 06:54:39,737][20347] Updated weights for policy 0, policy_version 181606 (0.0006) [2023-12-27 06:54:40,383][20381] Updated weights for policy 1, policy_version 181337 (0.0006) [2023-12-27 06:54:40,436][20347] Updated weights for policy 0, policy_version 181616 (0.0009) [2023-12-27 06:54:40,440][20381] Updated weights for policy 1, policy_version 181347 (0.0006) [2023-12-27 06:54:40,494][20347] Updated weights for policy 0, policy_version 181626 (0.0010) [2023-12-27 06:54:40,495][20381] Updated weights for policy 1, policy_version 181357 (0.0009) [2023-12-27 06:54:40,542][20347] Updated weights for policy 0, policy_version 181636 (0.0008) [2023-12-27 06:54:40,550][20381] Updated weights for policy 1, policy_version 181367 (0.0006) [2023-12-27 06:54:41,176][20347] Updated weights for policy 0, policy_version 181646 (0.0007) [2023-12-27 06:54:41,236][20381] Updated weights for policy 1, policy_version 181377 (0.0007) [2023-12-27 06:54:41,244][20347] Updated weights for policy 0, policy_version 181656 (0.0009) [2023-12-27 06:54:41,297][20381] Updated weights for policy 1, policy_version 181387 (0.0006) [2023-12-27 06:54:41,327][20347] Updated weights for policy 0, policy_version 181666 (0.0010) [2023-12-27 06:54:41,358][20381] Updated weights for policy 1, policy_version 181397 (0.0010) [2023-12-27 06:54:42,048][20347] Updated weights for policy 0, policy_version 181676 (0.0010) [2023-12-27 06:54:42,104][20347] Updated weights for policy 0, policy_version 181686 (0.0009) [2023-12-27 06:54:42,134][20381] Updated weights for policy 1, policy_version 181407 (0.0008) [2023-12-27 06:54:42,164][20347] Updated weights for policy 0, policy_version 181696 (0.0009) [2023-12-27 06:54:42,191][20381] Updated weights for policy 1, policy_version 181417 (0.0009) [2023-12-27 06:54:42,252][20381] Updated weights for policy 1, policy_version 181427 (0.0008) [2023-12-27 06:54:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 92987392. Throughput: 0: 10441.5, 1: 10499.9. Samples: 92973684. Policy #0 lag: (min: 10.0, avg: 21.2, max: 42.0) [2023-12-27 06:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:54:42,906][20347] Updated weights for policy 0, policy_version 181706 (0.0009) [2023-12-27 06:54:42,965][20381] Updated weights for policy 1, policy_version 181437 (0.0006) [2023-12-27 06:54:42,966][20347] Updated weights for policy 0, policy_version 181716 (0.0006) [2023-12-27 06:54:43,024][20381] Updated weights for policy 1, policy_version 181447 (0.0008) [2023-12-27 06:54:43,032][20347] Updated weights for policy 0, policy_version 181726 (0.0008) [2023-12-27 06:54:43,081][20381] Updated weights for policy 1, policy_version 181457 (0.0006) [2023-12-27 06:54:43,087][20347] Updated weights for policy 0, policy_version 181736 (0.0010) [2023-12-27 06:54:43,622][20347] Updated weights for policy 0, policy_version 181746 (0.0008) [2023-12-27 06:54:43,688][20347] Updated weights for policy 0, policy_version 181756 (0.0008) [2023-12-27 06:54:43,740][20381] Updated weights for policy 1, policy_version 181467 (0.0006) [2023-12-27 06:54:43,753][20347] Updated weights for policy 0, policy_version 181766 (0.0009) [2023-12-27 06:54:43,792][20292] KL-divergence is very high: 172.7112 [2023-12-27 06:54:43,793][20381] Updated weights for policy 1, policy_version 181477 (0.0006) [2023-12-27 06:54:43,798][20292] KL-divergence is very high: 108.7954 [2023-12-27 06:54:43,846][20292] KL-divergence is very high: 123.2303 [2023-12-27 06:54:43,859][20381] Updated weights for policy 1, policy_version 181487 (0.0008) [2023-12-27 06:54:44,423][20347] Updated weights for policy 0, policy_version 181776 (0.0010) [2023-12-27 06:54:44,449][20381] Updated weights for policy 1, policy_version 181497 (0.0006) [2023-12-27 06:54:44,477][20347] Updated weights for policy 0, policy_version 181786 (0.0010) [2023-12-27 06:54:44,505][20381] Updated weights for policy 1, policy_version 181507 (0.0007) [2023-12-27 06:54:44,532][20347] Updated weights for policy 0, policy_version 181796 (0.0010) [2023-12-27 06:54:44,561][20381] Updated weights for policy 1, policy_version 181517 (0.0006) [2023-12-27 06:54:44,609][20381] Updated weights for policy 1, policy_version 181527 (0.0007) [2023-12-27 06:54:45,241][20347] Updated weights for policy 0, policy_version 181806 (0.0007) [2023-12-27 06:54:45,278][20381] Updated weights for policy 1, policy_version 181537 (0.0006) [2023-12-27 06:54:45,302][20347] Updated weights for policy 0, policy_version 181816 (0.0010) [2023-12-27 06:54:45,336][20381] Updated weights for policy 1, policy_version 181547 (0.0006) [2023-12-27 06:54:45,361][20347] Updated weights for policy 0, policy_version 181826 (0.0010) [2023-12-27 06:54:45,393][20381] Updated weights for policy 1, policy_version 181557 (0.0007) [2023-12-27 06:54:45,997][20381] Updated weights for policy 1, policy_version 181567 (0.0008) [2023-12-27 06:54:46,019][20347] Updated weights for policy 0, policy_version 181836 (0.0010) [2023-12-27 06:54:46,051][20381] Updated weights for policy 1, policy_version 181577 (0.0007) [2023-12-27 06:54:46,075][20347] Updated weights for policy 0, policy_version 181846 (0.0008) [2023-12-27 06:54:46,107][20381] Updated weights for policy 1, policy_version 181587 (0.0007) [2023-12-27 06:54:46,134][20347] Updated weights for policy 0, policy_version 181856 (0.0006) [2023-12-27 06:54:46,664][20347] Updated weights for policy 0, policy_version 181866 (0.0005) [2023-12-27 06:54:46,713][20347] Updated weights for policy 0, policy_version 181876 (0.0006) [2023-12-27 06:54:46,768][20347] Updated weights for policy 0, policy_version 181886 (0.0010) [2023-12-27 06:54:46,806][20381] Updated weights for policy 1, policy_version 181597 (0.0007) [2023-12-27 06:54:46,823][20347] Updated weights for policy 0, policy_version 181896 (0.0010) [2023-12-27 06:54:46,856][20381] Updated weights for policy 1, policy_version 181607 (0.0006) [2023-12-27 06:54:46,902][20381] Updated weights for policy 1, policy_version 181617 (0.0007) [2023-12-27 06:54:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 93085696. Throughput: 0: 10458.6, 1: 10443.3. Samples: 93098604. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '998.520')] [2023-12-27 06:54:47,397][20347] Updated weights for policy 0, policy_version 181906 (0.0010) [2023-12-27 06:54:47,455][20347] Updated weights for policy 0, policy_version 181916 (0.0010) [2023-12-27 06:54:47,509][20347] Updated weights for policy 0, policy_version 181926 (0.0010) [2023-12-27 06:54:47,554][20381] Updated weights for policy 1, policy_version 181627 (0.0008) [2023-12-27 06:54:47,602][20381] Updated weights for policy 1, policy_version 181637 (0.0010) [2023-12-27 06:54:47,651][20381] Updated weights for policy 1, policy_version 181647 (0.0010) [2023-12-27 06:54:48,100][20347] Updated weights for policy 0, policy_version 181936 (0.0010) [2023-12-27 06:54:48,151][20347] Updated weights for policy 0, policy_version 181946 (0.0010) [2023-12-27 06:54:48,202][20347] Updated weights for policy 0, policy_version 181956 (0.0010) [2023-12-27 06:54:48,271][20381] Updated weights for policy 1, policy_version 181657 (0.0011) [2023-12-27 06:54:48,329][20381] Updated weights for policy 1, policy_version 181667 (0.0009) [2023-12-27 06:54:48,389][20381] Updated weights for policy 1, policy_version 181677 (0.0010) [2023-12-27 06:54:48,447][20381] Updated weights for policy 1, policy_version 181687 (0.0010) [2023-12-27 06:54:48,904][20347] Updated weights for policy 0, policy_version 181966 (0.0007) [2023-12-27 06:54:48,961][20347] Updated weights for policy 0, policy_version 181976 (0.0008) [2023-12-27 06:54:49,020][20347] Updated weights for policy 0, policy_version 181986 (0.0009) [2023-12-27 06:54:49,107][20381] Updated weights for policy 1, policy_version 181697 (0.0011) [2023-12-27 06:54:49,156][20381] Updated weights for policy 1, policy_version 181707 (0.0010) [2023-12-27 06:54:49,211][20381] Updated weights for policy 1, policy_version 181717 (0.0010) [2023-12-27 06:54:49,663][20347] Updated weights for policy 0, policy_version 181996 (0.0007) [2023-12-27 06:54:49,731][20347] Updated weights for policy 0, policy_version 182006 (0.0007) [2023-12-27 06:54:49,794][20347] Updated weights for policy 0, policy_version 182016 (0.0007) [2023-12-27 06:54:49,905][20381] Updated weights for policy 1, policy_version 181727 (0.0008) [2023-12-27 06:54:49,972][20381] Updated weights for policy 1, policy_version 181737 (0.0009) [2023-12-27 06:54:50,027][20381] Updated weights for policy 1, policy_version 181747 (0.0006) [2023-12-27 06:54:50,426][20347] Updated weights for policy 0, policy_version 182026 (0.0007) [2023-12-27 06:54:50,473][20347] Updated weights for policy 0, policy_version 182036 (0.0008) [2023-12-27 06:54:50,519][20347] Updated weights for policy 0, policy_version 182046 (0.0008) [2023-12-27 06:54:50,570][20347] Updated weights for policy 0, policy_version 182056 (0.0008) [2023-12-27 06:54:50,697][20381] Updated weights for policy 1, policy_version 181757 (0.0007) [2023-12-27 06:54:50,759][20381] Updated weights for policy 1, policy_version 181767 (0.0008) [2023-12-27 06:54:50,821][20381] Updated weights for policy 1, policy_version 181777 (0.0008) [2023-12-27 06:54:51,367][20347] Updated weights for policy 0, policy_version 182066 (0.0008) [2023-12-27 06:54:51,423][20347] Updated weights for policy 0, policy_version 182076 (0.0008) [2023-12-27 06:54:51,475][20347] Updated weights for policy 0, policy_version 182086 (0.0008) [2023-12-27 06:54:51,584][20381] Updated weights for policy 1, policy_version 181787 (0.0007) [2023-12-27 06:54:51,644][20381] Updated weights for policy 1, policy_version 181797 (0.0008) [2023-12-27 06:54:51,698][20381] Updated weights for policy 1, policy_version 181807 (0.0007) [2023-12-27 06:54:52,235][20347] Updated weights for policy 0, policy_version 182096 (0.0007) [2023-12-27 06:54:52,302][20347] Updated weights for policy 0, policy_version 182106 (0.0007) [2023-12-27 06:54:52,361][20347] Updated weights for policy 0, policy_version 182116 (0.0006) [2023-12-27 06:54:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 93184000. Throughput: 0: 10514.0, 1: 10415.3. Samples: 93162284. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:54:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '999.040')] [2023-12-27 06:54:52,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000181816_46555136.pth... [2023-12-27 06:54:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000180600_46243840.pth [2023-12-27 06:54:52,385][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000182120_46637056.pth... [2023-12-27 06:54:52,389][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000180872_46317568.pth [2023-12-27 06:54:52,444][20381] Updated weights for policy 1, policy_version 181817 (0.0008) [2023-12-27 06:54:52,506][20381] Updated weights for policy 1, policy_version 181827 (0.0011) [2023-12-27 06:54:52,561][20381] Updated weights for policy 1, policy_version 181837 (0.0011) [2023-12-27 06:54:52,626][20381] Updated weights for policy 1, policy_version 181847 (0.0011) [2023-12-27 06:54:52,989][20347] Updated weights for policy 0, policy_version 182126 (0.0007) [2023-12-27 06:54:53,046][20347] Updated weights for policy 0, policy_version 182136 (0.0007) [2023-12-27 06:54:53,100][20347] Updated weights for policy 0, policy_version 182146 (0.0007) [2023-12-27 06:54:53,241][20381] Updated weights for policy 1, policy_version 181857 (0.0006) [2023-12-27 06:54:53,292][20381] Updated weights for policy 1, policy_version 181867 (0.0005) [2023-12-27 06:54:53,339][20381] Updated weights for policy 1, policy_version 181877 (0.0007) [2023-12-27 06:54:53,780][20347] Updated weights for policy 0, policy_version 182156 (0.0009) [2023-12-27 06:54:53,830][20347] Updated weights for policy 0, policy_version 182166 (0.0007) [2023-12-27 06:54:53,890][20347] Updated weights for policy 0, policy_version 182176 (0.0007) [2023-12-27 06:54:53,961][20381] Updated weights for policy 1, policy_version 181887 (0.0010) [2023-12-27 06:54:54,010][20381] Updated weights for policy 1, policy_version 181897 (0.0010) [2023-12-27 06:54:54,061][20381] Updated weights for policy 1, policy_version 181907 (0.0010) [2023-12-27 06:54:54,540][20347] Updated weights for policy 0, policy_version 182186 (0.0007) [2023-12-27 06:54:54,599][20347] Updated weights for policy 0, policy_version 182196 (0.0006) [2023-12-27 06:54:54,664][20347] Updated weights for policy 0, policy_version 182206 (0.0005) [2023-12-27 06:54:54,717][20347] Updated weights for policy 0, policy_version 182216 (0.0005) [2023-12-27 06:54:54,781][20381] Updated weights for policy 1, policy_version 181917 (0.0010) [2023-12-27 06:54:54,835][20381] Updated weights for policy 1, policy_version 181927 (0.0010) [2023-12-27 06:54:54,886][20381] Updated weights for policy 1, policy_version 181937 (0.0010) [2023-12-27 06:54:55,356][20347] Updated weights for policy 0, policy_version 182226 (0.0008) [2023-12-27 06:54:55,423][20347] Updated weights for policy 0, policy_version 182236 (0.0006) [2023-12-27 06:54:55,484][20347] Updated weights for policy 0, policy_version 182246 (0.0008) [2023-12-27 06:54:55,553][20381] Updated weights for policy 1, policy_version 181947 (0.0010) [2023-12-27 06:54:55,612][20381] Updated weights for policy 1, policy_version 181957 (0.0008) [2023-12-27 06:54:55,672][20381] Updated weights for policy 1, policy_version 181967 (0.0006) [2023-12-27 06:54:56,095][20347] Updated weights for policy 0, policy_version 182256 (0.0010) [2023-12-27 06:54:56,154][20347] Updated weights for policy 0, policy_version 182266 (0.0010) [2023-12-27 06:54:56,211][20347] Updated weights for policy 0, policy_version 182276 (0.0010) [2023-12-27 06:54:56,308][20381] Updated weights for policy 1, policy_version 181977 (0.0007) [2023-12-27 06:54:56,363][20381] Updated weights for policy 1, policy_version 181987 (0.0005) [2023-12-27 06:54:56,412][20381] Updated weights for policy 1, policy_version 181997 (0.0005) [2023-12-27 06:54:56,464][20381] Updated weights for policy 1, policy_version 182007 (0.0005) [2023-12-27 06:54:56,791][20347] Updated weights for policy 0, policy_version 182286 (0.0009) [2023-12-27 06:54:56,863][20347] Updated weights for policy 0, policy_version 182296 (0.0007) [2023-12-27 06:54:56,916][20347] Updated weights for policy 0, policy_version 182306 (0.0006) [2023-12-27 06:54:57,089][20381] Updated weights for policy 1, policy_version 182017 (0.0007) [2023-12-27 06:54:57,143][20381] Updated weights for policy 1, policy_version 182027 (0.0007) [2023-12-27 06:54:57,200][20381] Updated weights for policy 1, policy_version 182037 (0.0007) [2023-12-27 06:54:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.7, 300 sec: 20743.8). Total num frames: 93298688. Throughput: 0: 10444.8, 1: 10434.5. Samples: 93284784. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:54:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:54:57,550][20347] Updated weights for policy 0, policy_version 182316 (0.0010) [2023-12-27 06:54:57,603][20347] Updated weights for policy 0, policy_version 182326 (0.0010) [2023-12-27 06:54:57,657][20347] Updated weights for policy 0, policy_version 182336 (0.0010) [2023-12-27 06:54:57,901][20381] Updated weights for policy 1, policy_version 182047 (0.0007) [2023-12-27 06:54:57,957][20381] Updated weights for policy 1, policy_version 182058 (0.0007) [2023-12-27 06:54:58,006][20381] Updated weights for policy 1, policy_version 182068 (0.0005) [2023-12-27 06:54:58,268][20347] Updated weights for policy 0, policy_version 182346 (0.0010) [2023-12-27 06:54:58,335][20347] Updated weights for policy 0, policy_version 182356 (0.0008) [2023-12-27 06:54:58,400][20347] Updated weights for policy 0, policy_version 182366 (0.0009) [2023-12-27 06:54:58,462][20347] Updated weights for policy 0, policy_version 182376 (0.0007) [2023-12-27 06:54:58,673][20381] Updated weights for policy 1, policy_version 182078 (0.0007) [2023-12-27 06:54:58,739][20381] Updated weights for policy 1, policy_version 182088 (0.0008) [2023-12-27 06:54:58,806][20381] Updated weights for policy 1, policy_version 182098 (0.0009) [2023-12-27 06:54:59,182][20347] Updated weights for policy 0, policy_version 182386 (0.0007) [2023-12-27 06:54:59,243][20347] Updated weights for policy 0, policy_version 182396 (0.0007) [2023-12-27 06:54:59,300][20347] Updated weights for policy 0, policy_version 182406 (0.0007) [2023-12-27 06:54:59,516][20381] Updated weights for policy 1, policy_version 182108 (0.0009) [2023-12-27 06:54:59,567][20381] Updated weights for policy 1, policy_version 182118 (0.0005) [2023-12-27 06:54:59,619][20381] Updated weights for policy 1, policy_version 182128 (0.0009) [2023-12-27 06:55:00,061][20347] Updated weights for policy 0, policy_version 182416 (0.0010) [2023-12-27 06:55:00,113][20347] Updated weights for policy 0, policy_version 182426 (0.0010) [2023-12-27 06:55:00,161][20347] Updated weights for policy 0, policy_version 182436 (0.0010) [2023-12-27 06:55:00,271][20381] Updated weights for policy 1, policy_version 182138 (0.0010) [2023-12-27 06:55:00,322][20381] Updated weights for policy 1, policy_version 182148 (0.0010) [2023-12-27 06:55:00,377][20381] Updated weights for policy 1, policy_version 182158 (0.0011) [2023-12-27 06:55:00,442][20381] Updated weights for policy 1, policy_version 182168 (0.0011) [2023-12-27 06:55:00,854][20347] Updated weights for policy 0, policy_version 182446 (0.0010) [2023-12-27 06:55:00,917][20347] Updated weights for policy 0, policy_version 182456 (0.0010) [2023-12-27 06:55:00,974][20347] Updated weights for policy 0, policy_version 182466 (0.0010) [2023-12-27 06:55:01,147][20381] Updated weights for policy 1, policy_version 182178 (0.0011) [2023-12-27 06:55:01,202][20381] Updated weights for policy 1, policy_version 182188 (0.0010) [2023-12-27 06:55:01,262][20381] Updated weights for policy 1, policy_version 182198 (0.0008) [2023-12-27 06:55:01,667][20347] Updated weights for policy 0, policy_version 182476 (0.0009) [2023-12-27 06:55:01,715][20347] Updated weights for policy 0, policy_version 182486 (0.0007) [2023-12-27 06:55:01,775][20347] Updated weights for policy 0, policy_version 182496 (0.0008) [2023-12-27 06:55:01,949][20381] Updated weights for policy 1, policy_version 182208 (0.0010) [2023-12-27 06:55:01,998][20381] Updated weights for policy 1, policy_version 182218 (0.0011) [2023-12-27 06:55:02,054][20381] Updated weights for policy 1, policy_version 182228 (0.0011) [2023-12-27 06:55:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 93396992. Throughput: 0: 10399.8, 1: 10436.7. Samples: 93407204. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:02,477][20347] Updated weights for policy 0, policy_version 182506 (0.0007) [2023-12-27 06:55:02,536][20347] Updated weights for policy 0, policy_version 182516 (0.0005) [2023-12-27 06:55:02,600][20347] Updated weights for policy 0, policy_version 182526 (0.0005) [2023-12-27 06:55:02,662][20347] Updated weights for policy 0, policy_version 182536 (0.0006) [2023-12-27 06:55:02,780][20381] Updated weights for policy 1, policy_version 182238 (0.0009) [2023-12-27 06:55:02,828][20381] Updated weights for policy 1, policy_version 182248 (0.0008) [2023-12-27 06:55:02,885][20381] Updated weights for policy 1, policy_version 182258 (0.0006) [2023-12-27 06:55:03,240][20347] Updated weights for policy 0, policy_version 182546 (0.0006) [2023-12-27 06:55:03,294][20347] Updated weights for policy 0, policy_version 182556 (0.0006) [2023-12-27 06:55:03,348][20347] Updated weights for policy 0, policy_version 182566 (0.0006) [2023-12-27 06:55:03,584][20381] Updated weights for policy 1, policy_version 182268 (0.0006) [2023-12-27 06:55:03,636][20381] Updated weights for policy 1, policy_version 182278 (0.0006) [2023-12-27 06:55:03,681][20381] Updated weights for policy 1, policy_version 182288 (0.0005) [2023-12-27 06:55:03,966][20347] Updated weights for policy 0, policy_version 182576 (0.0007) [2023-12-27 06:55:04,022][20347] Updated weights for policy 0, policy_version 182586 (0.0007) [2023-12-27 06:55:04,072][20347] Updated weights for policy 0, policy_version 182596 (0.0007) [2023-12-27 06:55:04,266][20381] Updated weights for policy 1, policy_version 182298 (0.0005) [2023-12-27 06:55:04,321][20381] Updated weights for policy 1, policy_version 182308 (0.0006) [2023-12-27 06:55:04,383][20381] Updated weights for policy 1, policy_version 182318 (0.0006) [2023-12-27 06:55:04,439][20381] Updated weights for policy 1, policy_version 182328 (0.0007) [2023-12-27 06:55:04,741][20347] Updated weights for policy 0, policy_version 182606 (0.0007) [2023-12-27 06:55:04,799][20347] Updated weights for policy 0, policy_version 182616 (0.0007) [2023-12-27 06:55:04,850][20347] Updated weights for policy 0, policy_version 182626 (0.0008) [2023-12-27 06:55:05,135][20381] Updated weights for policy 1, policy_version 182338 (0.0008) [2023-12-27 06:55:05,187][20381] Updated weights for policy 1, policy_version 182348 (0.0008) [2023-12-27 06:55:05,240][20381] Updated weights for policy 1, policy_version 182358 (0.0007) [2023-12-27 06:55:05,556][20347] Updated weights for policy 0, policy_version 182636 (0.0008) [2023-12-27 06:55:05,613][20347] Updated weights for policy 0, policy_version 182646 (0.0011) [2023-12-27 06:55:05,674][20347] Updated weights for policy 0, policy_version 182656 (0.0010) [2023-12-27 06:55:05,961][20381] Updated weights for policy 1, policy_version 182368 (0.0008) [2023-12-27 06:55:06,015][20381] Updated weights for policy 1, policy_version 182378 (0.0007) [2023-12-27 06:55:06,069][20381] Updated weights for policy 1, policy_version 182388 (0.0008) [2023-12-27 06:55:06,377][20347] Updated weights for policy 0, policy_version 182666 (0.0010) [2023-12-27 06:55:06,426][20347] Updated weights for policy 0, policy_version 182676 (0.0007) [2023-12-27 06:55:06,480][20347] Updated weights for policy 0, policy_version 182686 (0.0007) [2023-12-27 06:55:06,528][20347] Updated weights for policy 0, policy_version 182696 (0.0007) [2023-12-27 06:55:06,802][20381] Updated weights for policy 1, policy_version 182398 (0.0008) [2023-12-27 06:55:06,860][20381] Updated weights for policy 1, policy_version 182408 (0.0008) [2023-12-27 06:55:06,915][20381] Updated weights for policy 1, policy_version 182418 (0.0006) [2023-12-27 06:55:07,282][20347] Updated weights for policy 0, policy_version 182706 (0.0008) [2023-12-27 06:55:07,338][20347] Updated weights for policy 0, policy_version 182716 (0.0008) [2023-12-27 06:55:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 93495296. Throughput: 0: 10444.3, 1: 10455.4. Samples: 93470552. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000182424_46710784.pth... [2023-12-27 06:55:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000181240_46407680.pth [2023-12-27 06:55:07,396][20347] Updated weights for policy 0, policy_version 182726 (0.0007) [2023-12-27 06:55:07,408][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000182728_46792704.pth... [2023-12-27 06:55:07,413][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000181512_46481408.pth [2023-12-27 06:55:07,515][20381] Updated weights for policy 1, policy_version 182428 (0.0006) [2023-12-27 06:55:07,580][20381] Updated weights for policy 1, policy_version 182438 (0.0008) [2023-12-27 06:55:07,625][20381] Updated weights for policy 1, policy_version 182448 (0.0008) [2023-12-27 06:55:08,068][20347] Updated weights for policy 0, policy_version 182736 (0.0006) [2023-12-27 06:55:08,129][20347] Updated weights for policy 0, policy_version 182746 (0.0006) [2023-12-27 06:55:08,190][20347] Updated weights for policy 0, policy_version 182756 (0.0010) [2023-12-27 06:55:08,265][20381] Updated weights for policy 1, policy_version 182458 (0.0007) [2023-12-27 06:55:08,333][20381] Updated weights for policy 1, policy_version 182468 (0.0006) [2023-12-27 06:55:08,390][20381] Updated weights for policy 1, policy_version 182478 (0.0006) [2023-12-27 06:55:08,453][20381] Updated weights for policy 1, policy_version 182488 (0.0008) [2023-12-27 06:55:08,876][20347] Updated weights for policy 0, policy_version 182766 (0.0009) [2023-12-27 06:55:08,933][20347] Updated weights for policy 0, policy_version 182776 (0.0009) [2023-12-27 06:55:08,989][20347] Updated weights for policy 0, policy_version 182786 (0.0007) [2023-12-27 06:55:09,074][20381] Updated weights for policy 1, policy_version 182498 (0.0008) [2023-12-27 06:55:09,124][20381] Updated weights for policy 1, policy_version 182508 (0.0011) [2023-12-27 06:55:09,180][20381] Updated weights for policy 1, policy_version 182518 (0.0011) [2023-12-27 06:55:09,672][20347] Updated weights for policy 0, policy_version 182796 (0.0008) [2023-12-27 06:55:09,738][20347] Updated weights for policy 0, policy_version 182806 (0.0008) [2023-12-27 06:55:09,803][20347] Updated weights for policy 0, policy_version 182816 (0.0007) [2023-12-27 06:55:09,911][20381] Updated weights for policy 1, policy_version 182528 (0.0008) [2023-12-27 06:55:09,971][20381] Updated weights for policy 1, policy_version 182538 (0.0008) [2023-12-27 06:55:10,026][20381] Updated weights for policy 1, policy_version 182548 (0.0008) [2023-12-27 06:55:10,499][20347] Updated weights for policy 0, policy_version 182826 (0.0009) [2023-12-27 06:55:10,558][20347] Updated weights for policy 0, policy_version 182836 (0.0007) [2023-12-27 06:55:10,616][20347] Updated weights for policy 0, policy_version 182846 (0.0007) [2023-12-27 06:55:10,665][20381] Updated weights for policy 1, policy_version 182558 (0.0008) [2023-12-27 06:55:10,675][20347] Updated weights for policy 0, policy_version 182856 (0.0006) [2023-12-27 06:55:10,731][20381] Updated weights for policy 1, policy_version 182568 (0.0005) [2023-12-27 06:55:10,793][20381] Updated weights for policy 1, policy_version 182578 (0.0007) [2023-12-27 06:55:11,300][20347] Updated weights for policy 0, policy_version 182866 (0.0008) [2023-12-27 06:55:11,359][20347] Updated weights for policy 0, policy_version 182876 (0.0008) [2023-12-27 06:55:11,422][20347] Updated weights for policy 0, policy_version 182886 (0.0008) [2023-12-27 06:55:11,475][20381] Updated weights for policy 1, policy_version 182588 (0.0011) [2023-12-27 06:55:11,530][20381] Updated weights for policy 1, policy_version 182598 (0.0011) [2023-12-27 06:55:11,590][20381] Updated weights for policy 1, policy_version 182608 (0.0011) [2023-12-27 06:55:12,175][20347] Updated weights for policy 0, policy_version 182896 (0.0007) [2023-12-27 06:55:12,221][20347] Updated weights for policy 0, policy_version 182906 (0.0009) [2023-12-27 06:55:12,262][20381] Updated weights for policy 1, policy_version 182618 (0.0009) [2023-12-27 06:55:12,279][20347] Updated weights for policy 0, policy_version 182916 (0.0007) [2023-12-27 06:55:12,329][20381] Updated weights for policy 1, policy_version 182628 (0.0008) [2023-12-27 06:55:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 93601792. Throughput: 0: 10376.7, 1: 10410.4. Samples: 93592536. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:12,394][20381] Updated weights for policy 1, policy_version 182638 (0.0010) [2023-12-27 06:55:12,453][20381] Updated weights for policy 1, policy_version 182648 (0.0011) [2023-12-27 06:55:13,025][20347] Updated weights for policy 0, policy_version 182926 (0.0006) [2023-12-27 06:55:13,076][20381] Updated weights for policy 1, policy_version 182658 (0.0009) [2023-12-27 06:55:13,078][20347] Updated weights for policy 0, policy_version 182936 (0.0006) [2023-12-27 06:55:13,131][20381] Updated weights for policy 1, policy_version 182668 (0.0010) [2023-12-27 06:55:13,133][20347] Updated weights for policy 0, policy_version 182946 (0.0005) [2023-12-27 06:55:13,183][20381] Updated weights for policy 1, policy_version 182678 (0.0010) [2023-12-27 06:55:13,798][20347] Updated weights for policy 0, policy_version 182956 (0.0005) [2023-12-27 06:55:13,799][20381] Updated weights for policy 1, policy_version 182688 (0.0007) [2023-12-27 06:55:13,853][20381] Updated weights for policy 1, policy_version 182698 (0.0008) [2023-12-27 06:55:13,858][20347] Updated weights for policy 0, policy_version 182966 (0.0005) [2023-12-27 06:55:13,915][20381] Updated weights for policy 1, policy_version 182708 (0.0005) [2023-12-27 06:55:13,916][20347] Updated weights for policy 0, policy_version 182976 (0.0008) [2023-12-27 06:55:14,523][20381] Updated weights for policy 1, policy_version 182718 (0.0007) [2023-12-27 06:55:14,579][20381] Updated weights for policy 1, policy_version 182728 (0.0007) [2023-12-27 06:55:14,602][20347] Updated weights for policy 0, policy_version 182986 (0.0008) [2023-12-27 06:55:14,634][20381] Updated weights for policy 1, policy_version 182738 (0.0005) [2023-12-27 06:55:14,654][20347] Updated weights for policy 0, policy_version 182996 (0.0008) [2023-12-27 06:55:14,698][20347] Updated weights for policy 0, policy_version 183006 (0.0008) [2023-12-27 06:55:14,743][20347] Updated weights for policy 0, policy_version 183016 (0.0007) [2023-12-27 06:55:15,221][20381] Updated weights for policy 1, policy_version 182748 (0.0005) [2023-12-27 06:55:15,282][20381] Updated weights for policy 1, policy_version 182758 (0.0008) [2023-12-27 06:55:15,338][20381] Updated weights for policy 1, policy_version 182768 (0.0008) [2023-12-27 06:55:15,511][20347] Updated weights for policy 0, policy_version 183026 (0.0009) [2023-12-27 06:55:15,561][20347] Updated weights for policy 0, policy_version 183036 (0.0009) [2023-12-27 06:55:15,616][20347] Updated weights for policy 0, policy_version 183046 (0.0009) [2023-12-27 06:55:15,874][20381] Updated weights for policy 1, policy_version 182778 (0.0007) [2023-12-27 06:55:15,921][20381] Updated weights for policy 1, policy_version 182788 (0.0007) [2023-12-27 06:55:15,966][20381] Updated weights for policy 1, policy_version 182798 (0.0007) [2023-12-27 06:55:16,016][20381] Updated weights for policy 1, policy_version 182808 (0.0006) [2023-12-27 06:55:16,351][20347] Updated weights for policy 0, policy_version 183056 (0.0008) [2023-12-27 06:55:16,402][20347] Updated weights for policy 0, policy_version 183066 (0.0006) [2023-12-27 06:55:16,459][20347] Updated weights for policy 0, policy_version 183076 (0.0008) [2023-12-27 06:55:16,736][20381] Updated weights for policy 1, policy_version 182818 (0.0008) [2023-12-27 06:55:16,797][20381] Updated weights for policy 1, policy_version 182828 (0.0007) [2023-12-27 06:55:16,851][20381] Updated weights for policy 1, policy_version 182838 (0.0008) [2023-12-27 06:55:17,156][20347] Updated weights for policy 0, policy_version 183086 (0.0007) [2023-12-27 06:55:17,206][20347] Updated weights for policy 0, policy_version 183096 (0.0007) [2023-12-27 06:55:17,255][20347] Updated weights for policy 0, policy_version 183106 (0.0007) [2023-12-27 06:55:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 93708288. Throughput: 0: 10306.1, 1: 10411.0. Samples: 93715928. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '999.920')] [2023-12-27 06:55:17,533][20381] Updated weights for policy 1, policy_version 182848 (0.0009) [2023-12-27 06:55:17,590][20381] Updated weights for policy 1, policy_version 182858 (0.0011) [2023-12-27 06:55:17,648][20381] Updated weights for policy 1, policy_version 182868 (0.0011) [2023-12-27 06:55:17,957][20347] Updated weights for policy 0, policy_version 183116 (0.0007) [2023-12-27 06:55:18,023][20347] Updated weights for policy 0, policy_version 183126 (0.0008) [2023-12-27 06:55:18,070][20347] Updated weights for policy 0, policy_version 183136 (0.0007) [2023-12-27 06:55:18,316][20381] Updated weights for policy 1, policy_version 182878 (0.0011) [2023-12-27 06:55:18,375][20381] Updated weights for policy 1, policy_version 182888 (0.0011) [2023-12-27 06:55:18,433][20381] Updated weights for policy 1, policy_version 182898 (0.0011) [2023-12-27 06:55:18,744][20347] Updated weights for policy 0, policy_version 183146 (0.0007) [2023-12-27 06:55:18,792][20347] Updated weights for policy 0, policy_version 183156 (0.0007) [2023-12-27 06:55:18,839][20347] Updated weights for policy 0, policy_version 183166 (0.0010) [2023-12-27 06:55:18,885][20347] Updated weights for policy 0, policy_version 183176 (0.0010) [2023-12-27 06:55:19,103][20381] Updated weights for policy 1, policy_version 182908 (0.0008) [2023-12-27 06:55:19,150][20381] Updated weights for policy 1, policy_version 182918 (0.0008) [2023-12-27 06:55:19,203][20381] Updated weights for policy 1, policy_version 182928 (0.0007) [2023-12-27 06:55:19,561][20347] Updated weights for policy 0, policy_version 183186 (0.0006) [2023-12-27 06:55:19,611][20347] Updated weights for policy 0, policy_version 183196 (0.0008) [2023-12-27 06:55:19,670][20347] Updated weights for policy 0, policy_version 183206 (0.0010) [2023-12-27 06:55:19,875][20381] Updated weights for policy 1, policy_version 182938 (0.0007) [2023-12-27 06:55:19,940][20381] Updated weights for policy 1, policy_version 182948 (0.0007) [2023-12-27 06:55:19,999][20381] Updated weights for policy 1, policy_version 182958 (0.0007) [2023-12-27 06:55:20,058][20381] Updated weights for policy 1, policy_version 182968 (0.0009) [2023-12-27 06:55:20,360][20347] Updated weights for policy 0, policy_version 183216 (0.0007) [2023-12-27 06:55:20,419][20347] Updated weights for policy 0, policy_version 183226 (0.0007) [2023-12-27 06:55:20,471][20347] Updated weights for policy 0, policy_version 183236 (0.0010) [2023-12-27 06:55:20,726][20381] Updated weights for policy 1, policy_version 182978 (0.0009) [2023-12-27 06:55:20,779][20381] Updated weights for policy 1, policy_version 182988 (0.0011) [2023-12-27 06:55:20,837][20381] Updated weights for policy 1, policy_version 182998 (0.0011) [2023-12-27 06:55:21,157][20347] Updated weights for policy 0, policy_version 183246 (0.0009) [2023-12-27 06:55:21,214][20347] Updated weights for policy 0, policy_version 183256 (0.0010) [2023-12-27 06:55:21,280][20347] Updated weights for policy 0, policy_version 183266 (0.0010) [2023-12-27 06:55:21,588][20381] Updated weights for policy 1, policy_version 183008 (0.0009) [2023-12-27 06:55:21,658][20381] Updated weights for policy 1, policy_version 183018 (0.0009) [2023-12-27 06:55:21,714][20381] Updated weights for policy 1, policy_version 183028 (0.0010) [2023-12-27 06:55:21,993][20347] Updated weights for policy 0, policy_version 183276 (0.0011) [2023-12-27 06:55:22,051][20347] Updated weights for policy 0, policy_version 183286 (0.0009) [2023-12-27 06:55:22,116][20347] Updated weights for policy 0, policy_version 183296 (0.0007) [2023-12-27 06:55:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 93806592. Throughput: 0: 10304.8, 1: 10366.8. Samples: 93777728. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '999.920')] [2023-12-27 06:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000183032_46866432.pth... [2023-12-27 06:55:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000183304_46940160.pth... [2023-12-27 06:55:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000181816_46555136.pth [2023-12-27 06:55:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000182120_46637056.pth [2023-12-27 06:55:22,442][20381] Updated weights for policy 1, policy_version 183038 (0.0009) [2023-12-27 06:55:22,502][20381] Updated weights for policy 1, policy_version 183048 (0.0009) [2023-12-27 06:55:22,569][20381] Updated weights for policy 1, policy_version 183058 (0.0005) [2023-12-27 06:55:22,797][20347] Updated weights for policy 0, policy_version 183306 (0.0007) [2023-12-27 06:55:22,857][20347] Updated weights for policy 0, policy_version 183316 (0.0008) [2023-12-27 06:55:22,912][20347] Updated weights for policy 0, policy_version 183326 (0.0007) [2023-12-27 06:55:22,977][20347] Updated weights for policy 0, policy_version 183336 (0.0006) [2023-12-27 06:55:23,175][20381] Updated weights for policy 1, policy_version 183068 (0.0006) [2023-12-27 06:55:23,241][20381] Updated weights for policy 1, policy_version 183078 (0.0005) [2023-12-27 06:55:23,306][20381] Updated weights for policy 1, policy_version 183088 (0.0006) [2023-12-27 06:55:23,680][20347] Updated weights for policy 0, policy_version 183346 (0.0009) [2023-12-27 06:55:23,739][20347] Updated weights for policy 0, policy_version 183356 (0.0008) [2023-12-27 06:55:23,799][20347] Updated weights for policy 0, policy_version 183366 (0.0006) [2023-12-27 06:55:23,834][20381] Updated weights for policy 1, policy_version 183098 (0.0006) [2023-12-27 06:55:23,892][20381] Updated weights for policy 1, policy_version 183108 (0.0006) [2023-12-27 06:55:23,949][20381] Updated weights for policy 1, policy_version 183118 (0.0005) [2023-12-27 06:55:23,993][20381] Updated weights for policy 1, policy_version 183128 (0.0005) [2023-12-27 06:55:24,394][20347] Updated weights for policy 0, policy_version 183376 (0.0006) [2023-12-27 06:55:24,461][20347] Updated weights for policy 0, policy_version 183386 (0.0007) [2023-12-27 06:55:24,521][20347] Updated weights for policy 0, policy_version 183396 (0.0009) [2023-12-27 06:55:24,570][20381] Updated weights for policy 1, policy_version 183138 (0.0011) [2023-12-27 06:55:24,622][20381] Updated weights for policy 1, policy_version 183148 (0.0010) [2023-12-27 06:55:24,683][20381] Updated weights for policy 1, policy_version 183158 (0.0011) [2023-12-27 06:55:25,074][20347] Updated weights for policy 0, policy_version 183406 (0.0007) [2023-12-27 06:55:25,124][20347] Updated weights for policy 0, policy_version 183416 (0.0005) [2023-12-27 06:55:25,184][20347] Updated weights for policy 0, policy_version 183426 (0.0008) [2023-12-27 06:55:25,371][20381] Updated weights for policy 1, policy_version 183168 (0.0008) [2023-12-27 06:55:25,424][20381] Updated weights for policy 1, policy_version 183178 (0.0011) [2023-12-27 06:55:25,486][20381] Updated weights for policy 1, policy_version 183188 (0.0008) [2023-12-27 06:55:25,809][20347] Updated weights for policy 0, policy_version 183436 (0.0008) [2023-12-27 06:55:25,871][20347] Updated weights for policy 0, policy_version 183446 (0.0008) [2023-12-27 06:55:25,941][20347] Updated weights for policy 0, policy_version 183456 (0.0010) [2023-12-27 06:55:26,104][20381] Updated weights for policy 1, policy_version 183198 (0.0007) [2023-12-27 06:55:26,150][20381] Updated weights for policy 1, policy_version 183208 (0.0005) [2023-12-27 06:55:26,200][20381] Updated weights for policy 1, policy_version 183218 (0.0005) [2023-12-27 06:55:26,560][20347] Updated weights for policy 0, policy_version 183466 (0.0010) [2023-12-27 06:55:26,607][20347] Updated weights for policy 0, policy_version 183476 (0.0007) [2023-12-27 06:55:26,653][20347] Updated weights for policy 0, policy_version 183486 (0.0007) [2023-12-27 06:55:26,704][20347] Updated weights for policy 0, policy_version 183496 (0.0007) [2023-12-27 06:55:26,827][20381] Updated weights for policy 1, policy_version 183228 (0.0008) [2023-12-27 06:55:26,877][20381] Updated weights for policy 1, policy_version 183238 (0.0009) [2023-12-27 06:55:26,923][20381] Updated weights for policy 1, policy_version 183248 (0.0005) [2023-12-27 06:55:27,341][20347] Updated weights for policy 0, policy_version 183506 (0.0008) [2023-12-27 06:55:27,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 93913088. Throughput: 0: 10269.1, 1: 10399.6. Samples: 93903776. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:27,396][20347] Updated weights for policy 0, policy_version 183516 (0.0007) [2023-12-27 06:55:27,447][20347] Updated weights for policy 0, policy_version 183526 (0.0009) [2023-12-27 06:55:27,534][20381] Updated weights for policy 1, policy_version 183258 (0.0005) [2023-12-27 06:55:27,581][20381] Updated weights for policy 1, policy_version 183268 (0.0005) [2023-12-27 06:55:27,629][20381] Updated weights for policy 1, policy_version 183278 (0.0005) [2023-12-27 06:55:27,687][20381] Updated weights for policy 1, policy_version 183288 (0.0005) [2023-12-27 06:55:28,202][20347] Updated weights for policy 0, policy_version 183536 (0.0006) [2023-12-27 06:55:28,203][20381] Updated weights for policy 1, policy_version 183298 (0.0005) [2023-12-27 06:55:28,257][20381] Updated weights for policy 1, policy_version 183308 (0.0005) [2023-12-27 06:55:28,261][20347] Updated weights for policy 0, policy_version 183546 (0.0005) [2023-12-27 06:55:28,312][20381] Updated weights for policy 1, policy_version 183318 (0.0006) [2023-12-27 06:55:28,321][20347] Updated weights for policy 0, policy_version 183556 (0.0006) [2023-12-27 06:55:28,894][20347] Updated weights for policy 0, policy_version 183566 (0.0009) [2023-12-27 06:55:28,949][20347] Updated weights for policy 0, policy_version 183576 (0.0008) [2023-12-27 06:55:28,956][20381] Updated weights for policy 1, policy_version 183328 (0.0007) [2023-12-27 06:55:29,003][20347] Updated weights for policy 0, policy_version 183586 (0.0007) [2023-12-27 06:55:29,005][20381] Updated weights for policy 1, policy_version 183338 (0.0005) [2023-12-27 06:55:29,058][20381] Updated weights for policy 1, policy_version 183348 (0.0006) [2023-12-27 06:55:29,617][20347] Updated weights for policy 0, policy_version 183596 (0.0007) [2023-12-27 06:55:29,673][20347] Updated weights for policy 0, policy_version 183606 (0.0006) [2023-12-27 06:55:29,710][20381] Updated weights for policy 1, policy_version 183358 (0.0005) [2023-12-27 06:55:29,731][20347] Updated weights for policy 0, policy_version 183616 (0.0007) [2023-12-27 06:55:29,762][20381] Updated weights for policy 1, policy_version 183368 (0.0008) [2023-12-27 06:55:29,810][20381] Updated weights for policy 1, policy_version 183378 (0.0008) [2023-12-27 06:55:30,321][20347] Updated weights for policy 0, policy_version 183626 (0.0006) [2023-12-27 06:55:30,377][20347] Updated weights for policy 0, policy_version 183636 (0.0009) [2023-12-27 06:55:30,429][20347] Updated weights for policy 0, policy_version 183646 (0.0009) [2023-12-27 06:55:30,490][20347] Updated weights for policy 0, policy_version 183656 (0.0005) [2023-12-27 06:55:30,527][20381] Updated weights for policy 1, policy_version 183388 (0.0008) [2023-12-27 06:55:30,584][20381] Updated weights for policy 1, policy_version 183398 (0.0005) [2023-12-27 06:55:30,634][20381] Updated weights for policy 1, policy_version 183408 (0.0007) [2023-12-27 06:55:31,083][20347] Updated weights for policy 0, policy_version 183666 (0.0008) [2023-12-27 06:55:31,148][20347] Updated weights for policy 0, policy_version 183676 (0.0008) [2023-12-27 06:55:31,181][20381] Updated weights for policy 1, policy_version 183418 (0.0006) [2023-12-27 06:55:31,206][20347] Updated weights for policy 0, policy_version 183686 (0.0008) [2023-12-27 06:55:31,236][20381] Updated weights for policy 1, policy_version 183428 (0.0006) [2023-12-27 06:55:31,291][20381] Updated weights for policy 1, policy_version 183438 (0.0007) [2023-12-27 06:55:31,345][20381] Updated weights for policy 1, policy_version 183448 (0.0008) [2023-12-27 06:55:31,885][20347] Updated weights for policy 0, policy_version 183696 (0.0007) [2023-12-27 06:55:31,945][20347] Updated weights for policy 0, policy_version 183706 (0.0007) [2023-12-27 06:55:31,996][20347] Updated weights for policy 0, policy_version 183716 (0.0007) [2023-12-27 06:55:32,055][20381] Updated weights for policy 1, policy_version 183458 (0.0010) [2023-12-27 06:55:32,114][20381] Updated weights for policy 1, policy_version 183468 (0.0011) [2023-12-27 06:55:32,171][20381] Updated weights for policy 1, policy_version 183478 (0.0009) [2023-12-27 06:55:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 94027776. Throughput: 0: 10303.5, 1: 10501.7. Samples: 94034840. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:32,721][20347] Updated weights for policy 0, policy_version 183726 (0.0006) [2023-12-27 06:55:32,777][20347] Updated weights for policy 0, policy_version 183736 (0.0005) [2023-12-27 06:55:32,800][20381] Updated weights for policy 1, policy_version 183488 (0.0008) [2023-12-27 06:55:32,834][20347] Updated weights for policy 0, policy_version 183746 (0.0006) [2023-12-27 06:55:32,854][20381] Updated weights for policy 1, policy_version 183498 (0.0006) [2023-12-27 06:55:32,905][20381] Updated weights for policy 1, policy_version 183508 (0.0007) [2023-12-27 06:55:33,484][20347] Updated weights for policy 0, policy_version 183756 (0.0008) [2023-12-27 06:55:33,522][20381] Updated weights for policy 1, policy_version 183518 (0.0006) [2023-12-27 06:55:33,536][20347] Updated weights for policy 0, policy_version 183766 (0.0007) [2023-12-27 06:55:33,574][20381] Updated weights for policy 1, policy_version 183528 (0.0005) [2023-12-27 06:55:33,585][20347] Updated weights for policy 0, policy_version 183776 (0.0007) [2023-12-27 06:55:33,629][20381] Updated weights for policy 1, policy_version 183538 (0.0005) [2023-12-27 06:55:34,175][20381] Updated weights for policy 1, policy_version 183548 (0.0006) [2023-12-27 06:55:34,233][20381] Updated weights for policy 1, policy_version 183558 (0.0008) [2023-12-27 06:55:34,292][20381] Updated weights for policy 1, policy_version 183568 (0.0011) [2023-12-27 06:55:34,299][20347] Updated weights for policy 0, policy_version 183786 (0.0008) [2023-12-27 06:55:34,351][20347] Updated weights for policy 0, policy_version 183796 (0.0008) [2023-12-27 06:55:34,410][20347] Updated weights for policy 0, policy_version 183806 (0.0008) [2023-12-27 06:55:34,475][20347] Updated weights for policy 0, policy_version 183816 (0.0007) [2023-12-27 06:55:34,976][20381] Updated weights for policy 1, policy_version 183578 (0.0011) [2023-12-27 06:55:35,026][20381] Updated weights for policy 1, policy_version 183588 (0.0008) [2023-12-27 06:55:35,073][20381] Updated weights for policy 1, policy_version 183598 (0.0007) [2023-12-27 06:55:35,114][20347] Updated weights for policy 0, policy_version 183826 (0.0008) [2023-12-27 06:55:35,122][20381] Updated weights for policy 1, policy_version 183608 (0.0010) [2023-12-27 06:55:35,167][20347] Updated weights for policy 0, policy_version 183836 (0.0008) [2023-12-27 06:55:35,223][20347] Updated weights for policy 0, policy_version 183846 (0.0007) [2023-12-27 06:55:35,781][20381] Updated weights for policy 1, policy_version 183618 (0.0010) [2023-12-27 06:55:35,850][20381] Updated weights for policy 1, policy_version 183628 (0.0011) [2023-12-27 06:55:35,901][20381] Updated weights for policy 1, policy_version 183638 (0.0009) [2023-12-27 06:55:35,929][20347] Updated weights for policy 0, policy_version 183856 (0.0007) [2023-12-27 06:55:35,994][20347] Updated weights for policy 0, policy_version 183866 (0.0007) [2023-12-27 06:55:36,058][20347] Updated weights for policy 0, policy_version 183876 (0.0008) [2023-12-27 06:55:36,567][20381] Updated weights for policy 1, policy_version 183648 (0.0008) [2023-12-27 06:55:36,628][20381] Updated weights for policy 1, policy_version 183658 (0.0008) [2023-12-27 06:55:36,686][20381] Updated weights for policy 1, policy_version 183668 (0.0007) [2023-12-27 06:55:36,775][20347] Updated weights for policy 0, policy_version 183886 (0.0008) [2023-12-27 06:55:36,829][20347] Updated weights for policy 0, policy_version 183896 (0.0008) [2023-12-27 06:55:36,889][20347] Updated weights for policy 0, policy_version 183906 (0.0008) [2023-12-27 06:55:37,299][20381] Updated weights for policy 1, policy_version 183678 (0.0007) [2023-12-27 06:55:37,351][20381] Updated weights for policy 1, policy_version 183688 (0.0007) [2023-12-27 06:55:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 94126080. Throughput: 0: 10267.1, 1: 10549.5. Samples: 94099032. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000183912_47095808.pth... [2023-12-27 06:55:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000182728_46792704.pth [2023-12-27 06:55:37,403][20381] Updated weights for policy 1, policy_version 183698 (0.0008) [2023-12-27 06:55:37,432][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000183704_47038464.pth... [2023-12-27 06:55:37,435][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000182424_46710784.pth [2023-12-27 06:55:37,615][20347] Updated weights for policy 0, policy_version 183916 (0.0007) [2023-12-27 06:55:37,672][20347] Updated weights for policy 0, policy_version 183926 (0.0007) [2023-12-27 06:55:37,724][20347] Updated weights for policy 0, policy_version 183936 (0.0005) [2023-12-27 06:55:38,123][20381] Updated weights for policy 1, policy_version 183708 (0.0008) [2023-12-27 06:55:38,181][20381] Updated weights for policy 1, policy_version 183718 (0.0007) [2023-12-27 06:55:38,242][20381] Updated weights for policy 1, policy_version 183728 (0.0007) [2023-12-27 06:55:38,403][20347] Updated weights for policy 0, policy_version 183946 (0.0007) [2023-12-27 06:55:38,465][20347] Updated weights for policy 0, policy_version 183956 (0.0008) [2023-12-27 06:55:38,518][20347] Updated weights for policy 0, policy_version 183966 (0.0008) [2023-12-27 06:55:38,573][20347] Updated weights for policy 0, policy_version 183976 (0.0008) [2023-12-27 06:55:38,912][20381] Updated weights for policy 1, policy_version 183738 (0.0008) [2023-12-27 06:55:38,964][20381] Updated weights for policy 1, policy_version 183748 (0.0008) [2023-12-27 06:55:39,012][20381] Updated weights for policy 1, policy_version 183758 (0.0007) [2023-12-27 06:55:39,058][20381] Updated weights for policy 1, policy_version 183768 (0.0008) [2023-12-27 06:55:39,292][20347] Updated weights for policy 0, policy_version 183986 (0.0008) [2023-12-27 06:55:39,358][20347] Updated weights for policy 0, policy_version 183996 (0.0008) [2023-12-27 06:55:39,422][20347] Updated weights for policy 0, policy_version 184006 (0.0008) [2023-12-27 06:55:39,772][20381] Updated weights for policy 1, policy_version 183778 (0.0008) [2023-12-27 06:55:39,832][20381] Updated weights for policy 1, policy_version 183788 (0.0007) [2023-12-27 06:55:39,896][20381] Updated weights for policy 1, policy_version 183798 (0.0006) [2023-12-27 06:55:40,124][20347] Updated weights for policy 0, policy_version 184016 (0.0008) [2023-12-27 06:55:40,184][20347] Updated weights for policy 0, policy_version 184026 (0.0007) [2023-12-27 06:55:40,236][20347] Updated weights for policy 0, policy_version 184036 (0.0008) [2023-12-27 06:55:40,621][20381] Updated weights for policy 1, policy_version 183808 (0.0007) [2023-12-27 06:55:40,676][20381] Updated weights for policy 1, policy_version 183818 (0.0007) [2023-12-27 06:55:40,726][20381] Updated weights for policy 1, policy_version 183828 (0.0008) [2023-12-27 06:55:40,935][20347] Updated weights for policy 0, policy_version 184046 (0.0007) [2023-12-27 06:55:40,985][20347] Updated weights for policy 0, policy_version 184056 (0.0007) [2023-12-27 06:55:41,043][20347] Updated weights for policy 0, policy_version 184066 (0.0008) [2023-12-27 06:55:41,474][20381] Updated weights for policy 1, policy_version 183838 (0.0008) [2023-12-27 06:55:41,530][20381] Updated weights for policy 1, policy_version 183848 (0.0008) [2023-12-27 06:55:41,592][20381] Updated weights for policy 1, policy_version 183858 (0.0007) [2023-12-27 06:55:41,787][20347] Updated weights for policy 0, policy_version 184076 (0.0007) [2023-12-27 06:55:41,842][20347] Updated weights for policy 0, policy_version 184086 (0.0007) [2023-12-27 06:55:41,894][20347] Updated weights for policy 0, policy_version 184096 (0.0007) [2023-12-27 06:55:42,318][20381] Updated weights for policy 1, policy_version 183868 (0.0008) [2023-12-27 06:55:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 94224384. Throughput: 0: 10206.2, 1: 10534.2. Samples: 94218104. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:42,379][20381] Updated weights for policy 1, policy_version 183878 (0.0011) [2023-12-27 06:55:42,432][20381] Updated weights for policy 1, policy_version 183888 (0.0011) [2023-12-27 06:55:42,597][20347] Updated weights for policy 0, policy_version 184106 (0.0007) [2023-12-27 06:55:42,661][20347] Updated weights for policy 0, policy_version 184116 (0.0007) [2023-12-27 06:55:42,712][20347] Updated weights for policy 0, policy_version 184126 (0.0008) [2023-12-27 06:55:42,757][20347] Updated weights for policy 0, policy_version 184136 (0.0006) [2023-12-27 06:55:43,129][20381] Updated weights for policy 1, policy_version 183898 (0.0011) [2023-12-27 06:55:43,188][20381] Updated weights for policy 1, policy_version 183908 (0.0009) [2023-12-27 06:55:43,240][20381] Updated weights for policy 1, policy_version 183918 (0.0010) [2023-12-27 06:55:43,294][20381] Updated weights for policy 1, policy_version 183928 (0.0010) [2023-12-27 06:55:43,458][20347] Updated weights for policy 0, policy_version 184146 (0.0007) [2023-12-27 06:55:43,505][20347] Updated weights for policy 0, policy_version 184156 (0.0007) [2023-12-27 06:55:43,557][20347] Updated weights for policy 0, policy_version 184166 (0.0009) [2023-12-27 06:55:43,870][20381] Updated weights for policy 1, policy_version 183938 (0.0011) [2023-12-27 06:55:43,925][20381] Updated weights for policy 1, policy_version 183948 (0.0010) [2023-12-27 06:55:43,977][20381] Updated weights for policy 1, policy_version 183958 (0.0010) [2023-12-27 06:55:44,296][20347] Updated weights for policy 0, policy_version 184176 (0.0007) [2023-12-27 06:55:44,355][20347] Updated weights for policy 0, policy_version 184186 (0.0008) [2023-12-27 06:55:44,414][20347] Updated weights for policy 0, policy_version 184196 (0.0008) [2023-12-27 06:55:44,630][20381] Updated weights for policy 1, policy_version 183968 (0.0011) [2023-12-27 06:55:44,676][20381] Updated weights for policy 1, policy_version 183978 (0.0010) [2023-12-27 06:55:44,721][20381] Updated weights for policy 1, policy_version 183988 (0.0010) [2023-12-27 06:55:45,099][20347] Updated weights for policy 0, policy_version 184206 (0.0007) [2023-12-27 06:55:45,153][20347] Updated weights for policy 0, policy_version 184216 (0.0007) [2023-12-27 06:55:45,215][20347] Updated weights for policy 0, policy_version 184226 (0.0008) [2023-12-27 06:55:45,406][20381] Updated weights for policy 1, policy_version 183998 (0.0011) [2023-12-27 06:55:45,466][20381] Updated weights for policy 1, policy_version 184008 (0.0009) [2023-12-27 06:55:45,520][20381] Updated weights for policy 1, policy_version 184018 (0.0005) [2023-12-27 06:55:45,930][20347] Updated weights for policy 0, policy_version 184236 (0.0008) [2023-12-27 06:55:45,982][20347] Updated weights for policy 0, policy_version 184246 (0.0008) [2023-12-27 06:55:46,029][20347] Updated weights for policy 0, policy_version 184256 (0.0007) [2023-12-27 06:55:46,087][20381] Updated weights for policy 1, policy_version 184028 (0.0006) [2023-12-27 06:55:46,140][20381] Updated weights for policy 1, policy_version 184038 (0.0008) [2023-12-27 06:55:46,194][20381] Updated weights for policy 1, policy_version 184048 (0.0008) [2023-12-27 06:55:46,728][20347] Updated weights for policy 0, policy_version 184266 (0.0008) [2023-12-27 06:55:46,786][20347] Updated weights for policy 0, policy_version 184276 (0.0008) [2023-12-27 06:55:46,840][20347] Updated weights for policy 0, policy_version 184286 (0.0007) [2023-12-27 06:55:46,872][20381] Updated weights for policy 1, policy_version 184058 (0.0008) [2023-12-27 06:55:46,900][20347] Updated weights for policy 0, policy_version 184296 (0.0007) [2023-12-27 06:55:46,923][20381] Updated weights for policy 1, policy_version 184068 (0.0010) [2023-12-27 06:55:46,975][20381] Updated weights for policy 1, policy_version 184078 (0.0009) [2023-12-27 06:55:47,026][20381] Updated weights for policy 1, policy_version 184088 (0.0005) [2023-12-27 06:55:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 94330880. Throughput: 0: 10161.3, 1: 10586.8. Samples: 94340872. Policy #0 lag: (min: 22.0, avg: 25.5, max: 54.0) [2023-12-27 06:55:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:55:47,602][20381] Updated weights for policy 1, policy_version 184098 (0.0005) [2023-12-27 06:55:47,628][20347] Updated weights for policy 0, policy_version 184306 (0.0007) [2023-12-27 06:55:47,655][20381] Updated weights for policy 1, policy_version 184108 (0.0007) [2023-12-27 06:55:47,676][20347] Updated weights for policy 0, policy_version 184316 (0.0006) [2023-12-27 06:55:47,704][20381] Updated weights for policy 1, policy_version 184118 (0.0005) [2023-12-27 06:55:47,725][20347] Updated weights for policy 0, policy_version 184326 (0.0007) [2023-12-27 06:55:48,242][20381] Updated weights for policy 1, policy_version 184128 (0.0005) [2023-12-27 06:55:48,302][20381] Updated weights for policy 1, policy_version 184138 (0.0005) [2023-12-27 06:55:48,367][20381] Updated weights for policy 1, policy_version 184148 (0.0006) [2023-12-27 06:55:48,483][20347] Updated weights for policy 0, policy_version 184336 (0.0008) [2023-12-27 06:55:48,543][20347] Updated weights for policy 0, policy_version 184346 (0.0008) [2023-12-27 06:55:48,600][20347] Updated weights for policy 0, policy_version 184356 (0.0007) [2023-12-27 06:55:48,993][20381] Updated weights for policy 1, policy_version 184158 (0.0009) [2023-12-27 06:55:49,059][20381] Updated weights for policy 1, policy_version 184168 (0.0011) [2023-12-27 06:55:49,128][20381] Updated weights for policy 1, policy_version 184178 (0.0010) [2023-12-27 06:55:49,261][20347] Updated weights for policy 0, policy_version 184366 (0.0008) [2023-12-27 06:55:49,321][20347] Updated weights for policy 0, policy_version 184376 (0.0007) [2023-12-27 06:55:49,384][20347] Updated weights for policy 0, policy_version 184386 (0.0008) [2023-12-27 06:55:49,795][20381] Updated weights for policy 1, policy_version 184188 (0.0010) [2023-12-27 06:55:49,861][20381] Updated weights for policy 1, policy_version 184198 (0.0009) [2023-12-27 06:55:49,926][20381] Updated weights for policy 1, policy_version 184208 (0.0009) [2023-12-27 06:55:49,935][20292] KL-divergence is very high: 100.4093 [2023-12-27 06:55:50,106][20347] Updated weights for policy 0, policy_version 184396 (0.0006) [2023-12-27 06:55:50,164][20347] Updated weights for policy 0, policy_version 184406 (0.0008) [2023-12-27 06:55:50,226][20347] Updated weights for policy 0, policy_version 184416 (0.0007) [2023-12-27 06:55:50,566][20381] Updated weights for policy 1, policy_version 184218 (0.0010) [2023-12-27 06:55:50,626][20381] Updated weights for policy 1, policy_version 184228 (0.0011) [2023-12-27 06:55:50,673][20381] Updated weights for policy 1, policy_version 184238 (0.0010) [2023-12-27 06:55:50,726][20381] Updated weights for policy 1, policy_version 184248 (0.0010) [2023-12-27 06:55:50,922][20347] Updated weights for policy 0, policy_version 184426 (0.0006) [2023-12-27 06:55:50,987][20347] Updated weights for policy 0, policy_version 184436 (0.0008) [2023-12-27 06:55:51,053][20347] Updated weights for policy 0, policy_version 184446 (0.0009) [2023-12-27 06:55:51,117][20347] Updated weights for policy 0, policy_version 184456 (0.0008) [2023-12-27 06:55:51,403][20381] Updated weights for policy 1, policy_version 184258 (0.0007) [2023-12-27 06:55:51,461][20381] Updated weights for policy 1, policy_version 184268 (0.0008) [2023-12-27 06:55:51,515][20381] Updated weights for policy 1, policy_version 184278 (0.0005) [2023-12-27 06:55:51,868][20347] Updated weights for policy 0, policy_version 184466 (0.0006) [2023-12-27 06:55:51,921][20347] Updated weights for policy 0, policy_version 184476 (0.0007) [2023-12-27 06:55:51,980][20347] Updated weights for policy 0, policy_version 184486 (0.0006) [2023-12-27 06:55:52,196][20381] Updated weights for policy 1, policy_version 184288 (0.0006) [2023-12-27 06:55:52,252][20381] Updated weights for policy 1, policy_version 184298 (0.0006) [2023-12-27 06:55:52,312][20381] Updated weights for policy 1, policy_version 184308 (0.0007) [2023-12-27 06:55:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 94437376. Throughput: 0: 10103.8, 1: 10634.6. Samples: 94403784. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:55:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '999.880')] [2023-12-27 06:55:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000184488_47243264.pth... [2023-12-27 06:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000184312_47194112.pth... [2023-12-27 06:55:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000183304_46940160.pth [2023-12-27 06:55:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000183032_46866432.pth [2023-12-27 06:55:52,650][20347] Updated weights for policy 0, policy_version 184496 (0.0008) [2023-12-27 06:55:52,717][20347] Updated weights for policy 0, policy_version 184506 (0.0008) [2023-12-27 06:55:52,782][20347] Updated weights for policy 0, policy_version 184516 (0.0007) [2023-12-27 06:55:53,007][20381] Updated weights for policy 1, policy_version 184318 (0.0007) [2023-12-27 06:55:53,067][20381] Updated weights for policy 1, policy_version 184328 (0.0008) [2023-12-27 06:55:53,122][20381] Updated weights for policy 1, policy_version 184338 (0.0006) [2023-12-27 06:55:53,447][20347] Updated weights for policy 0, policy_version 184526 (0.0007) [2023-12-27 06:55:53,505][20347] Updated weights for policy 0, policy_version 184536 (0.0006) [2023-12-27 06:55:53,567][20347] Updated weights for policy 0, policy_version 184546 (0.0005) [2023-12-27 06:55:53,745][20381] Updated weights for policy 1, policy_version 184348 (0.0006) [2023-12-27 06:55:53,788][20381] Updated weights for policy 1, policy_version 184358 (0.0005) [2023-12-27 06:55:53,846][20381] Updated weights for policy 1, policy_version 184368 (0.0007) [2023-12-27 06:55:54,172][20347] Updated weights for policy 0, policy_version 184556 (0.0007) [2023-12-27 06:55:54,222][20347] Updated weights for policy 0, policy_version 184566 (0.0006) [2023-12-27 06:55:54,282][20347] Updated weights for policy 0, policy_version 184576 (0.0008) [2023-12-27 06:55:54,571][20381] Updated weights for policy 1, policy_version 184378 (0.0008) [2023-12-27 06:55:54,629][20381] Updated weights for policy 1, policy_version 184388 (0.0007) [2023-12-27 06:55:54,683][20381] Updated weights for policy 1, policy_version 184398 (0.0006) [2023-12-27 06:55:54,742][20381] Updated weights for policy 1, policy_version 184408 (0.0005) [2023-12-27 06:55:54,954][20347] Updated weights for policy 0, policy_version 184586 (0.0009) [2023-12-27 06:55:55,013][20347] Updated weights for policy 0, policy_version 184596 (0.0009) [2023-12-27 06:55:55,064][20347] Updated weights for policy 0, policy_version 184606 (0.0010) [2023-12-27 06:55:55,119][20347] Updated weights for policy 0, policy_version 184616 (0.0010) [2023-12-27 06:55:55,408][20381] Updated weights for policy 1, policy_version 184418 (0.0007) [2023-12-27 06:55:55,467][20381] Updated weights for policy 1, policy_version 184428 (0.0006) [2023-12-27 06:55:55,522][20381] Updated weights for policy 1, policy_version 184438 (0.0007) [2023-12-27 06:55:55,796][20347] Updated weights for policy 0, policy_version 184626 (0.0010) [2023-12-27 06:55:55,858][20347] Updated weights for policy 0, policy_version 184636 (0.0011) [2023-12-27 06:55:55,918][20347] Updated weights for policy 0, policy_version 184646 (0.0010) [2023-12-27 06:55:56,136][20381] Updated weights for policy 1, policy_version 184448 (0.0005) [2023-12-27 06:55:56,182][20381] Updated weights for policy 1, policy_version 184458 (0.0005) [2023-12-27 06:55:56,231][20381] Updated weights for policy 1, policy_version 184468 (0.0005) [2023-12-27 06:55:56,538][20347] Updated weights for policy 0, policy_version 184656 (0.0007) [2023-12-27 06:55:56,599][20347] Updated weights for policy 0, policy_version 184666 (0.0008) [2023-12-27 06:55:56,646][20347] Updated weights for policy 0, policy_version 184676 (0.0010) [2023-12-27 06:55:56,847][20381] Updated weights for policy 1, policy_version 184478 (0.0005) [2023-12-27 06:55:56,893][20381] Updated weights for policy 1, policy_version 184488 (0.0005) [2023-12-27 06:55:56,955][20381] Updated weights for policy 1, policy_version 184498 (0.0005) [2023-12-27 06:55:57,303][20347] Updated weights for policy 0, policy_version 184686 (0.0010) [2023-12-27 06:55:57,353][20347] Updated weights for policy 0, policy_version 184696 (0.0006) [2023-12-27 06:55:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 94535680. Throughput: 0: 10133.3, 1: 10639.7. Samples: 94527320. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:55:57,363][19170] Avg episode reward: [(0, '985.330'), (1, '1000.000')] [2023-12-27 06:55:57,400][20347] Updated weights for policy 0, policy_version 184706 (0.0005) [2023-12-27 06:55:57,566][20381] Updated weights for policy 1, policy_version 184508 (0.0007) [2023-12-27 06:55:57,623][20381] Updated weights for policy 1, policy_version 184518 (0.0009) [2023-12-27 06:55:57,668][20381] Updated weights for policy 1, policy_version 184528 (0.0005) [2023-12-27 06:55:57,931][20347] Updated weights for policy 0, policy_version 184716 (0.0007) [2023-12-27 06:55:57,982][20347] Updated weights for policy 0, policy_version 184726 (0.0010) [2023-12-27 06:55:58,037][20347] Updated weights for policy 0, policy_version 184736 (0.0010) [2023-12-27 06:55:58,306][20381] Updated weights for policy 1, policy_version 184538 (0.0006) [2023-12-27 06:55:58,369][20381] Updated weights for policy 1, policy_version 184548 (0.0009) [2023-12-27 06:55:58,432][20381] Updated weights for policy 1, policy_version 184558 (0.0008) [2023-12-27 06:55:58,488][20381] Updated weights for policy 1, policy_version 184568 (0.0010) [2023-12-27 06:55:58,768][20347] Updated weights for policy 0, policy_version 184746 (0.0010) [2023-12-27 06:55:58,834][20347] Updated weights for policy 0, policy_version 184756 (0.0013) [2023-12-27 06:55:58,903][20347] Updated weights for policy 0, policy_version 184766 (0.0008) [2023-12-27 06:55:58,969][20347] Updated weights for policy 0, policy_version 184776 (0.0008) [2023-12-27 06:55:59,303][20381] Updated weights for policy 1, policy_version 184578 (0.0009) [2023-12-27 06:55:59,360][20381] Updated weights for policy 1, policy_version 184588 (0.0007) [2023-12-27 06:55:59,425][20381] Updated weights for policy 1, policy_version 184598 (0.0008) [2023-12-27 06:55:59,661][20347] Updated weights for policy 0, policy_version 184786 (0.0005) [2023-12-27 06:55:59,716][20347] Updated weights for policy 0, policy_version 184796 (0.0010) [2023-12-27 06:55:59,777][20347] Updated weights for policy 0, policy_version 184806 (0.0010) [2023-12-27 06:56:00,172][20381] Updated weights for policy 1, policy_version 184608 (0.0008) [2023-12-27 06:56:00,220][20381] Updated weights for policy 1, policy_version 184618 (0.0007) [2023-12-27 06:56:00,281][20381] Updated weights for policy 1, policy_version 184628 (0.0007) [2023-12-27 06:56:00,442][20347] Updated weights for policy 0, policy_version 184816 (0.0008) [2023-12-27 06:56:00,502][20347] Updated weights for policy 0, policy_version 184826 (0.0010) [2023-12-27 06:56:00,559][20347] Updated weights for policy 0, policy_version 184836 (0.0010) [2023-12-27 06:56:00,860][20381] Updated weights for policy 1, policy_version 184638 (0.0007) [2023-12-27 06:56:00,913][20381] Updated weights for policy 1, policy_version 184648 (0.0007) [2023-12-27 06:56:00,970][20381] Updated weights for policy 1, policy_version 184658 (0.0006) [2023-12-27 06:56:01,235][20347] Updated weights for policy 0, policy_version 184846 (0.0009) [2023-12-27 06:56:01,284][20347] Updated weights for policy 0, policy_version 184856 (0.0008) [2023-12-27 06:56:01,342][20347] Updated weights for policy 0, policy_version 184866 (0.0010) [2023-12-27 06:56:01,673][20381] Updated weights for policy 1, policy_version 184668 (0.0006) [2023-12-27 06:56:01,738][20381] Updated weights for policy 1, policy_version 184678 (0.0008) [2023-12-27 06:56:01,799][20381] Updated weights for policy 1, policy_version 184688 (0.0007) [2023-12-27 06:56:01,968][20347] Updated weights for policy 0, policy_version 184876 (0.0009) [2023-12-27 06:56:02,018][20347] Updated weights for policy 0, policy_version 184886 (0.0005) [2023-12-27 06:56:02,079][20347] Updated weights for policy 0, policy_version 184896 (0.0006) [2023-12-27 06:56:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 94642176. Throughput: 0: 10240.9, 1: 10566.7. Samples: 94652268. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:02,363][19170] Avg episode reward: [(0, '984.680'), (1, '1000.000')] [2023-12-27 06:56:02,465][20381] Updated weights for policy 1, policy_version 184698 (0.0008) [2023-12-27 06:56:02,520][20381] Updated weights for policy 1, policy_version 184708 (0.0007) [2023-12-27 06:56:02,577][20381] Updated weights for policy 1, policy_version 184718 (0.0008) [2023-12-27 06:56:02,632][20381] Updated weights for policy 1, policy_version 184728 (0.0007) [2023-12-27 06:56:02,733][20347] Updated weights for policy 0, policy_version 184906 (0.0009) [2023-12-27 06:56:02,788][20347] Updated weights for policy 0, policy_version 184916 (0.0005) [2023-12-27 06:56:02,839][20347] Updated weights for policy 0, policy_version 184926 (0.0006) [2023-12-27 06:56:02,890][20347] Updated weights for policy 0, policy_version 184936 (0.0005) [2023-12-27 06:56:03,371][20381] Updated weights for policy 1, policy_version 184738 (0.0008) [2023-12-27 06:56:03,429][20381] Updated weights for policy 1, policy_version 184748 (0.0008) [2023-12-27 06:56:03,474][20347] Updated weights for policy 0, policy_version 184946 (0.0006) [2023-12-27 06:56:03,484][20381] Updated weights for policy 1, policy_version 184758 (0.0007) [2023-12-27 06:56:03,528][20347] Updated weights for policy 0, policy_version 184956 (0.0005) [2023-12-27 06:56:03,592][20347] Updated weights for policy 0, policy_version 184966 (0.0005) [2023-12-27 06:56:04,159][20347] Updated weights for policy 0, policy_version 184976 (0.0009) [2023-12-27 06:56:04,212][20347] Updated weights for policy 0, policy_version 184986 (0.0010) [2023-12-27 06:56:04,244][20381] Updated weights for policy 1, policy_version 184768 (0.0006) [2023-12-27 06:56:04,265][20347] Updated weights for policy 0, policy_version 184996 (0.0010) [2023-12-27 06:56:04,301][20381] Updated weights for policy 1, policy_version 184778 (0.0006) [2023-12-27 06:56:04,353][20381] Updated weights for policy 1, policy_version 184788 (0.0008) [2023-12-27 06:56:04,967][20347] Updated weights for policy 0, policy_version 185006 (0.0010) [2023-12-27 06:56:05,019][20347] Updated weights for policy 0, policy_version 185016 (0.0010) [2023-12-27 06:56:05,056][20381] Updated weights for policy 1, policy_version 184798 (0.0007) [2023-12-27 06:56:05,066][20347] Updated weights for policy 0, policy_version 185026 (0.0010) [2023-12-27 06:56:05,112][20381] Updated weights for policy 1, policy_version 184808 (0.0006) [2023-12-27 06:56:05,167][20381] Updated weights for policy 1, policy_version 184818 (0.0007) [2023-12-27 06:56:05,766][20347] Updated weights for policy 0, policy_version 185036 (0.0009) [2023-12-27 06:56:05,820][20347] Updated weights for policy 0, policy_version 185046 (0.0005) [2023-12-27 06:56:05,872][20381] Updated weights for policy 1, policy_version 184828 (0.0008) [2023-12-27 06:56:05,877][20347] Updated weights for policy 0, policy_version 185056 (0.0006) [2023-12-27 06:56:05,924][20381] Updated weights for policy 1, policy_version 184838 (0.0007) [2023-12-27 06:56:05,981][20381] Updated weights for policy 1, policy_version 184848 (0.0009) [2023-12-27 06:56:06,509][20347] Updated weights for policy 0, policy_version 185066 (0.0006) [2023-12-27 06:56:06,566][20347] Updated weights for policy 0, policy_version 185076 (0.0007) [2023-12-27 06:56:06,628][20347] Updated weights for policy 0, policy_version 185086 (0.0007) [2023-12-27 06:56:06,646][20381] Updated weights for policy 1, policy_version 184858 (0.0008) [2023-12-27 06:56:06,678][20347] Updated weights for policy 0, policy_version 185096 (0.0007) [2023-12-27 06:56:06,697][20381] Updated weights for policy 1, policy_version 184868 (0.0006) [2023-12-27 06:56:06,751][20381] Updated weights for policy 1, policy_version 184878 (0.0008) [2023-12-27 06:56:06,812][20381] Updated weights for policy 1, policy_version 184888 (0.0008) [2023-12-27 06:56:07,356][20347] Updated weights for policy 0, policy_version 185106 (0.0006) [2023-12-27 06:56:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 94740480. Throughput: 0: 10283.9, 1: 10522.7. Samples: 94714024. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:07,363][19170] Avg episode reward: [(0, '999.350'), (1, '1000.000')] [2023-12-27 06:56:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000184888_47341568.pth... [2023-12-27 06:56:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000183704_47038464.pth [2023-12-27 06:56:07,412][20347] Updated weights for policy 0, policy_version 185116 (0.0007) [2023-12-27 06:56:07,461][20347] Updated weights for policy 0, policy_version 185126 (0.0005) [2023-12-27 06:56:07,470][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000185128_47407104.pth... [2023-12-27 06:56:07,473][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000183912_47095808.pth [2023-12-27 06:56:07,548][20381] Updated weights for policy 1, policy_version 184898 (0.0007) [2023-12-27 06:56:07,606][20381] Updated weights for policy 1, policy_version 184908 (0.0008) [2023-12-27 06:56:07,662][20381] Updated weights for policy 1, policy_version 184918 (0.0008) [2023-12-27 06:56:08,099][20347] Updated weights for policy 0, policy_version 185136 (0.0007) [2023-12-27 06:56:08,154][20347] Updated weights for policy 0, policy_version 185146 (0.0007) [2023-12-27 06:56:08,214][20347] Updated weights for policy 0, policy_version 185156 (0.0007) [2023-12-27 06:56:08,405][20381] Updated weights for policy 1, policy_version 184928 (0.0007) [2023-12-27 06:56:08,452][20381] Updated weights for policy 1, policy_version 184938 (0.0010) [2023-12-27 06:56:08,511][20381] Updated weights for policy 1, policy_version 184948 (0.0008) [2023-12-27 06:56:08,783][20347] Updated weights for policy 0, policy_version 185166 (0.0005) [2023-12-27 06:56:08,848][20347] Updated weights for policy 0, policy_version 185176 (0.0005) [2023-12-27 06:56:08,908][20347] Updated weights for policy 0, policy_version 185186 (0.0005) [2023-12-27 06:56:09,269][20381] Updated weights for policy 1, policy_version 184958 (0.0007) [2023-12-27 06:56:09,331][20381] Updated weights for policy 1, policy_version 184968 (0.0007) [2023-12-27 06:56:09,401][20381] Updated weights for policy 1, policy_version 184978 (0.0009) [2023-12-27 06:56:09,488][20347] Updated weights for policy 0, policy_version 185196 (0.0007) [2023-12-27 06:56:09,552][20347] Updated weights for policy 0, policy_version 185206 (0.0007) [2023-12-27 06:56:09,614][20347] Updated weights for policy 0, policy_version 185216 (0.0007) [2023-12-27 06:56:10,090][20381] Updated weights for policy 1, policy_version 184988 (0.0007) [2023-12-27 06:56:10,151][20381] Updated weights for policy 1, policy_version 184998 (0.0007) [2023-12-27 06:56:10,213][20381] Updated weights for policy 1, policy_version 185008 (0.0007) [2023-12-27 06:56:10,339][20347] Updated weights for policy 0, policy_version 185226 (0.0007) [2023-12-27 06:56:10,393][20347] Updated weights for policy 0, policy_version 185236 (0.0006) [2023-12-27 06:56:10,446][20347] Updated weights for policy 0, policy_version 185246 (0.0006) [2023-12-27 06:56:10,502][20347] Updated weights for policy 0, policy_version 185256 (0.0005) [2023-12-27 06:56:10,874][20381] Updated weights for policy 1, policy_version 185018 (0.0007) [2023-12-27 06:56:10,931][20381] Updated weights for policy 1, policy_version 185028 (0.0007) [2023-12-27 06:56:10,985][20381] Updated weights for policy 1, policy_version 185038 (0.0006) [2023-12-27 06:56:11,048][20381] Updated weights for policy 1, policy_version 185048 (0.0008) [2023-12-27 06:56:11,107][20347] Updated weights for policy 0, policy_version 185266 (0.0007) [2023-12-27 06:56:11,166][20347] Updated weights for policy 0, policy_version 185276 (0.0007) [2023-12-27 06:56:11,226][20347] Updated weights for policy 0, policy_version 185286 (0.0006) [2023-12-27 06:56:11,797][20381] Updated weights for policy 1, policy_version 185058 (0.0008) [2023-12-27 06:56:11,858][20381] Updated weights for policy 1, policy_version 185068 (0.0008) [2023-12-27 06:56:11,919][20381] Updated weights for policy 1, policy_version 185078 (0.0008) [2023-12-27 06:56:11,979][20347] Updated weights for policy 0, policy_version 185296 (0.0006) [2023-12-27 06:56:12,043][20347] Updated weights for policy 0, policy_version 185306 (0.0005) [2023-12-27 06:56:12,105][20347] Updated weights for policy 0, policy_version 185316 (0.0005) [2023-12-27 06:56:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 94846976. Throughput: 0: 10327.4, 1: 10410.3. Samples: 94836972. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:12,688][20381] Updated weights for policy 1, policy_version 185088 (0.0009) [2023-12-27 06:56:12,731][20347] Updated weights for policy 0, policy_version 185326 (0.0005) [2023-12-27 06:56:12,747][20381] Updated weights for policy 1, policy_version 185098 (0.0010) [2023-12-27 06:56:12,780][20347] Updated weights for policy 0, policy_version 185336 (0.0009) [2023-12-27 06:56:12,806][20381] Updated weights for policy 1, policy_version 185108 (0.0010) [2023-12-27 06:56:12,831][20347] Updated weights for policy 0, policy_version 185346 (0.0006) [2023-12-27 06:56:13,370][20381] Updated weights for policy 1, policy_version 185118 (0.0007) [2023-12-27 06:56:13,421][20381] Updated weights for policy 1, policy_version 185128 (0.0005) [2023-12-27 06:56:13,472][20381] Updated weights for policy 1, policy_version 185138 (0.0006) [2023-12-27 06:56:13,534][20347] Updated weights for policy 0, policy_version 185356 (0.0007) [2023-12-27 06:56:13,588][20347] Updated weights for policy 0, policy_version 185366 (0.0008) [2023-12-27 06:56:13,642][20347] Updated weights for policy 0, policy_version 185376 (0.0010) [2023-12-27 06:56:14,113][20381] Updated weights for policy 1, policy_version 185148 (0.0008) [2023-12-27 06:56:14,165][20381] Updated weights for policy 1, policy_version 185158 (0.0007) [2023-12-27 06:56:14,225][20381] Updated weights for policy 1, policy_version 185168 (0.0007) [2023-12-27 06:56:14,273][20347] Updated weights for policy 0, policy_version 185386 (0.0011) [2023-12-27 06:56:14,327][20347] Updated weights for policy 0, policy_version 185396 (0.0010) [2023-12-27 06:56:14,382][20347] Updated weights for policy 0, policy_version 185406 (0.0010) [2023-12-27 06:56:14,433][20347] Updated weights for policy 0, policy_version 185416 (0.0005) [2023-12-27 06:56:14,949][20381] Updated weights for policy 1, policy_version 185178 (0.0007) [2023-12-27 06:56:15,003][20381] Updated weights for policy 1, policy_version 185188 (0.0008) [2023-12-27 06:56:15,016][20292] KL-divergence is very high: 1075.7456 [2023-12-27 06:56:15,028][20292] KL-divergence is very high: 151.1910 [2023-12-27 06:56:15,065][20381] Updated weights for policy 1, policy_version 185198 (0.0006) [2023-12-27 06:56:15,067][20347] Updated weights for policy 0, policy_version 185426 (0.0007) [2023-12-27 06:56:15,119][20381] Updated weights for policy 1, policy_version 185208 (0.0007) [2023-12-27 06:56:15,121][20347] Updated weights for policy 0, policy_version 185436 (0.0006) [2023-12-27 06:56:15,176][20347] Updated weights for policy 0, policy_version 185446 (0.0006) [2023-12-27 06:56:15,741][20347] Updated weights for policy 0, policy_version 185456 (0.0007) [2023-12-27 06:56:15,791][20347] Updated weights for policy 0, policy_version 185466 (0.0008) [2023-12-27 06:56:15,846][20347] Updated weights for policy 0, policy_version 185476 (0.0007) [2023-12-27 06:56:15,870][20381] Updated weights for policy 1, policy_version 185218 (0.0008) [2023-12-27 06:56:15,924][20381] Updated weights for policy 1, policy_version 185228 (0.0007) [2023-12-27 06:56:15,983][20381] Updated weights for policy 1, policy_version 185238 (0.0008) [2023-12-27 06:56:16,506][20347] Updated weights for policy 0, policy_version 185486 (0.0006) [2023-12-27 06:56:16,569][20347] Updated weights for policy 0, policy_version 185496 (0.0007) [2023-12-27 06:56:16,623][20347] Updated weights for policy 0, policy_version 185506 (0.0008) [2023-12-27 06:56:16,651][20381] Updated weights for policy 1, policy_version 185248 (0.0010) [2023-12-27 06:56:16,703][20381] Updated weights for policy 1, policy_version 185258 (0.0010) [2023-12-27 06:56:16,748][20381] Updated weights for policy 1, policy_version 185268 (0.0010) [2023-12-27 06:56:17,311][20347] Updated weights for policy 0, policy_version 185516 (0.0007) [2023-12-27 06:56:17,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 94945280. Throughput: 0: 10298.7, 1: 10290.2. Samples: 94961344. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.440')] [2023-12-27 06:56:17,367][20347] Updated weights for policy 0, policy_version 185526 (0.0007) [2023-12-27 06:56:17,376][20381] Updated weights for policy 1, policy_version 185278 (0.0007) [2023-12-27 06:56:17,423][20347] Updated weights for policy 0, policy_version 185536 (0.0007) [2023-12-27 06:56:17,425][20381] Updated weights for policy 1, policy_version 185288 (0.0005) [2023-12-27 06:56:17,478][20381] Updated weights for policy 1, policy_version 185298 (0.0006) [2023-12-27 06:56:18,118][20381] Updated weights for policy 1, policy_version 185308 (0.0007) [2023-12-27 06:56:18,138][20347] Updated weights for policy 0, policy_version 185546 (0.0006) [2023-12-27 06:56:18,166][20381] Updated weights for policy 1, policy_version 185318 (0.0010) [2023-12-27 06:56:18,201][20347] Updated weights for policy 0, policy_version 185556 (0.0006) [2023-12-27 06:56:18,221][20381] Updated weights for policy 1, policy_version 185328 (0.0010) [2023-12-27 06:56:18,258][20347] Updated weights for policy 0, policy_version 185566 (0.0005) [2023-12-27 06:56:18,311][20347] Updated weights for policy 0, policy_version 185576 (0.0007) [2023-12-27 06:56:18,813][20381] Updated weights for policy 1, policy_version 185338 (0.0011) [2023-12-27 06:56:18,866][20381] Updated weights for policy 1, policy_version 185348 (0.0011) [2023-12-27 06:56:18,875][20292] KL-divergence is very high: 409.0617 [2023-12-27 06:56:18,918][20381] Updated weights for policy 1, policy_version 185358 (0.0011) [2023-12-27 06:56:18,964][20292] KL-divergence is very high: 261.2436 [2023-12-27 06:56:18,974][20381] Updated weights for policy 1, policy_version 185368 (0.0010) [2023-12-27 06:56:19,019][20347] Updated weights for policy 0, policy_version 185586 (0.0007) [2023-12-27 06:56:19,069][20347] Updated weights for policy 0, policy_version 185596 (0.0008) [2023-12-27 06:56:19,130][20347] Updated weights for policy 0, policy_version 185606 (0.0008) [2023-12-27 06:56:19,588][20292] KL-divergence is very high: 658.2725 [2023-12-27 06:56:19,632][20292] KL-divergence is very high: 222.6733 [2023-12-27 06:56:19,638][20381] Updated weights for policy 1, policy_version 185378 (0.0006) [2023-12-27 06:56:19,674][20292] KL-divergence is very high: 162.9722 [2023-12-27 06:56:19,688][20381] Updated weights for policy 1, policy_version 185388 (0.0007) [2023-12-27 06:56:19,719][20292] KL-divergence is very high: 157.6492 [2023-12-27 06:56:19,750][20381] Updated weights for policy 1, policy_version 185398 (0.0011) [2023-12-27 06:56:19,760][20347] Updated weights for policy 0, policy_version 185616 (0.0007) [2023-12-27 06:56:19,819][20347] Updated weights for policy 0, policy_version 185626 (0.0008) [2023-12-27 06:56:19,873][20347] Updated weights for policy 0, policy_version 185636 (0.0007) [2023-12-27 06:56:20,480][20381] Updated weights for policy 1, policy_version 185408 (0.0011) [2023-12-27 06:56:20,488][20292] KL-divergence is very high: 108.1904 [2023-12-27 06:56:20,537][20381] Updated weights for policy 1, policy_version 185418 (0.0011) [2023-12-27 06:56:20,579][20292] KL-divergence is very high: 216.3008 [2023-12-27 06:56:20,597][20381] Updated weights for policy 1, policy_version 185428 (0.0010) [2023-12-27 06:56:20,616][20347] Updated weights for policy 0, policy_version 185646 (0.0008) [2023-12-27 06:56:20,677][20347] Updated weights for policy 0, policy_version 185656 (0.0008) [2023-12-27 06:56:20,735][20347] Updated weights for policy 0, policy_version 185666 (0.0005) [2023-12-27 06:56:21,259][20381] Updated weights for policy 1, policy_version 185438 (0.0007) [2023-12-27 06:56:21,295][20292] KL-divergence is very high: 112.0576 [2023-12-27 06:56:21,327][20381] Updated weights for policy 1, policy_version 185448 (0.0007) [2023-12-27 06:56:21,346][20347] Updated weights for policy 0, policy_version 185676 (0.0006) [2023-12-27 06:56:21,395][20381] Updated weights for policy 1, policy_version 185458 (0.0008) [2023-12-27 06:56:21,416][20347] Updated weights for policy 0, policy_version 185686 (0.0007) [2023-12-27 06:56:21,477][20347] Updated weights for policy 0, policy_version 185696 (0.0009) [2023-12-27 06:56:22,149][20381] Updated weights for policy 1, policy_version 185468 (0.0008) [2023-12-27 06:56:22,191][20347] Updated weights for policy 0, policy_version 185706 (0.0007) [2023-12-27 06:56:22,209][20381] Updated weights for policy 1, policy_version 185478 (0.0008) [2023-12-27 06:56:22,244][20347] Updated weights for policy 0, policy_version 185716 (0.0005) [2023-12-27 06:56:22,267][20381] Updated weights for policy 1, policy_version 185488 (0.0008) [2023-12-27 06:56:22,300][20347] Updated weights for policy 0, policy_version 185726 (0.0006) [2023-12-27 06:56:22,362][20347] Updated weights for policy 0, policy_version 185736 (0.0009) [2023-12-27 06:56:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 95059968. Throughput: 0: 10267.8, 1: 10268.9. Samples: 95023184. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '966.880')] [2023-12-27 06:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000185496_47497216.pth... [2023-12-27 06:56:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000185736_47562752.pth... [2023-12-27 06:56:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000184312_47194112.pth [2023-12-27 06:56:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000184488_47243264.pth [2023-12-27 06:56:23,003][20381] Updated weights for policy 1, policy_version 185498 (0.0008) [2023-12-27 06:56:23,013][20347] Updated weights for policy 0, policy_version 185746 (0.0007) [2023-12-27 06:56:23,050][20381] Updated weights for policy 1, policy_version 185508 (0.0007) [2023-12-27 06:56:23,064][20347] Updated weights for policy 0, policy_version 185756 (0.0006) [2023-12-27 06:56:23,095][20381] Updated weights for policy 1, policy_version 185518 (0.0006) [2023-12-27 06:56:23,116][20347] Updated weights for policy 0, policy_version 185766 (0.0006) [2023-12-27 06:56:23,142][20381] Updated weights for policy 1, policy_version 185528 (0.0007) [2023-12-27 06:56:23,773][20347] Updated weights for policy 0, policy_version 185776 (0.0007) [2023-12-27 06:56:23,827][20347] Updated weights for policy 0, policy_version 185786 (0.0007) [2023-12-27 06:56:23,866][20381] Updated weights for policy 1, policy_version 185538 (0.0007) [2023-12-27 06:56:23,887][20347] Updated weights for policy 0, policy_version 185796 (0.0006) [2023-12-27 06:56:23,930][20381] Updated weights for policy 1, policy_version 185549 (0.0006) [2023-12-27 06:56:23,984][20381] Updated weights for policy 1, policy_version 185559 (0.0008) [2023-12-27 06:56:24,596][20347] Updated weights for policy 0, policy_version 185806 (0.0006) [2023-12-27 06:56:24,631][20381] Updated weights for policy 1, policy_version 185569 (0.0006) [2023-12-27 06:56:24,648][20347] Updated weights for policy 0, policy_version 185816 (0.0005) [2023-12-27 06:56:24,680][20381] Updated weights for policy 1, policy_version 185579 (0.0005) [2023-12-27 06:56:24,698][20347] Updated weights for policy 0, policy_version 185826 (0.0006) [2023-12-27 06:56:24,726][20381] Updated weights for policy 1, policy_version 185589 (0.0006) [2023-12-27 06:56:25,404][20381] Updated weights for policy 1, policy_version 185599 (0.0009) [2023-12-27 06:56:25,425][20347] Updated weights for policy 0, policy_version 185836 (0.0007) [2023-12-27 06:56:25,453][20381] Updated weights for policy 1, policy_version 185609 (0.0011) [2023-12-27 06:56:25,474][20347] Updated weights for policy 0, policy_version 185846 (0.0007) [2023-12-27 06:56:25,504][20381] Updated weights for policy 1, policy_version 185619 (0.0010) [2023-12-27 06:56:25,529][20347] Updated weights for policy 0, policy_version 185856 (0.0005) [2023-12-27 06:56:26,167][20381] Updated weights for policy 1, policy_version 185629 (0.0009) [2023-12-27 06:56:26,232][20381] Updated weights for policy 1, policy_version 185639 (0.0008) [2023-12-27 06:56:26,276][20347] Updated weights for policy 0, policy_version 185866 (0.0008) [2023-12-27 06:56:26,292][20381] Updated weights for policy 1, policy_version 185649 (0.0008) [2023-12-27 06:56:26,327][20347] Updated weights for policy 0, policy_version 185876 (0.0010) [2023-12-27 06:56:26,374][20347] Updated weights for policy 0, policy_version 185886 (0.0010) [2023-12-27 06:56:26,419][20347] Updated weights for policy 0, policy_version 185896 (0.0010) [2023-12-27 06:56:26,885][20381] Updated weights for policy 1, policy_version 185659 (0.0007) [2023-12-27 06:56:26,935][20381] Updated weights for policy 1, policy_version 185669 (0.0007) [2023-12-27 06:56:26,979][20347] Updated weights for policy 0, policy_version 185906 (0.0009) [2023-12-27 06:56:26,992][20381] Updated weights for policy 1, policy_version 185679 (0.0005) [2023-12-27 06:56:27,027][20347] Updated weights for policy 0, policy_version 185916 (0.0006) [2023-12-27 06:56:27,072][20347] Updated weights for policy 0, policy_version 185926 (0.0005) [2023-12-27 06:56:27,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 95158272. Throughput: 0: 10312.0, 1: 10265.0. Samples: 95144068. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '975.160')] [2023-12-27 06:56:27,608][20347] Updated weights for policy 0, policy_version 185936 (0.0005) [2023-12-27 06:56:27,641][20381] Updated weights for policy 1, policy_version 185689 (0.0010) [2023-12-27 06:56:27,653][20347] Updated weights for policy 0, policy_version 185946 (0.0006) [2023-12-27 06:56:27,692][20381] Updated weights for policy 1, policy_version 185699 (0.0006) [2023-12-27 06:56:27,707][20347] Updated weights for policy 0, policy_version 185956 (0.0010) [2023-12-27 06:56:27,753][20381] Updated weights for policy 1, policy_version 185709 (0.0006) [2023-12-27 06:56:27,810][20381] Updated weights for policy 1, policy_version 185719 (0.0007) [2023-12-27 06:56:28,378][20347] Updated weights for policy 0, policy_version 185966 (0.0010) [2023-12-27 06:56:28,436][20347] Updated weights for policy 0, policy_version 185976 (0.0010) [2023-12-27 06:56:28,441][20381] Updated weights for policy 1, policy_version 185729 (0.0007) [2023-12-27 06:56:28,481][20347] Updated weights for policy 0, policy_version 185986 (0.0010) [2023-12-27 06:56:28,499][20381] Updated weights for policy 1, policy_version 185739 (0.0007) [2023-12-27 06:56:28,559][20381] Updated weights for policy 1, policy_version 185749 (0.0008) [2023-12-27 06:56:29,164][20347] Updated weights for policy 0, policy_version 185996 (0.0010) [2023-12-27 06:56:29,216][20347] Updated weights for policy 0, policy_version 186006 (0.0010) [2023-12-27 06:56:29,254][20381] Updated weights for policy 1, policy_version 185759 (0.0008) [2023-12-27 06:56:29,279][20347] Updated weights for policy 0, policy_version 186016 (0.0010) [2023-12-27 06:56:29,312][20381] Updated weights for policy 1, policy_version 185769 (0.0006) [2023-12-27 06:56:29,376][20381] Updated weights for policy 1, policy_version 185779 (0.0006) [2023-12-27 06:56:29,984][20347] Updated weights for policy 0, policy_version 186026 (0.0010) [2023-12-27 06:56:30,048][20347] Updated weights for policy 0, policy_version 186036 (0.0005) [2023-12-27 06:56:30,081][20381] Updated weights for policy 1, policy_version 185789 (0.0006) [2023-12-27 06:56:30,105][20347] Updated weights for policy 0, policy_version 186046 (0.0010) [2023-12-27 06:56:30,139][20381] Updated weights for policy 1, policy_version 185799 (0.0005) [2023-12-27 06:56:30,160][20347] Updated weights for policy 0, policy_version 186056 (0.0010) [2023-12-27 06:56:30,192][20381] Updated weights for policy 1, policy_version 185809 (0.0006) [2023-12-27 06:56:30,760][20347] Updated weights for policy 0, policy_version 186066 (0.0008) [2023-12-27 06:56:30,812][20347] Updated weights for policy 0, policy_version 186076 (0.0008) [2023-12-27 06:56:30,847][20381] Updated weights for policy 1, policy_version 185819 (0.0007) [2023-12-27 06:56:30,872][20347] Updated weights for policy 0, policy_version 186086 (0.0005) [2023-12-27 06:56:30,900][20381] Updated weights for policy 1, policy_version 185829 (0.0006) [2023-12-27 06:56:30,957][20381] Updated weights for policy 1, policy_version 185839 (0.0007) [2023-12-27 06:56:31,517][20347] Updated weights for policy 0, policy_version 186096 (0.0007) [2023-12-27 06:56:31,576][20347] Updated weights for policy 0, policy_version 186106 (0.0007) [2023-12-27 06:56:31,638][20347] Updated weights for policy 0, policy_version 186116 (0.0008) [2023-12-27 06:56:31,668][20381] Updated weights for policy 1, policy_version 185849 (0.0008) [2023-12-27 06:56:31,729][20381] Updated weights for policy 1, policy_version 185859 (0.0008) [2023-12-27 06:56:31,784][20381] Updated weights for policy 1, policy_version 185869 (0.0008) [2023-12-27 06:56:31,839][20381] Updated weights for policy 1, policy_version 185879 (0.0008) [2023-12-27 06:56:32,329][20347] Updated weights for policy 0, policy_version 186126 (0.0006) [2023-12-27 06:56:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 95256576. Throughput: 0: 10444.9, 1: 10232.3. Samples: 95271344. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '992.870')] [2023-12-27 06:56:32,383][20347] Updated weights for policy 0, policy_version 186136 (0.0007) [2023-12-27 06:56:32,436][20347] Updated weights for policy 0, policy_version 186146 (0.0008) [2023-12-27 06:56:32,524][20381] Updated weights for policy 1, policy_version 185889 (0.0007) [2023-12-27 06:56:32,579][20381] Updated weights for policy 1, policy_version 185899 (0.0007) [2023-12-27 06:56:32,641][20381] Updated weights for policy 1, policy_version 185909 (0.0008) [2023-12-27 06:56:33,084][20347] Updated weights for policy 0, policy_version 186156 (0.0006) [2023-12-27 06:56:33,136][20347] Updated weights for policy 0, policy_version 186166 (0.0006) [2023-12-27 06:56:33,184][20347] Updated weights for policy 0, policy_version 186176 (0.0007) [2023-12-27 06:56:33,323][20381] Updated weights for policy 1, policy_version 185919 (0.0006) [2023-12-27 06:56:33,375][20381] Updated weights for policy 1, policy_version 185929 (0.0005) [2023-12-27 06:56:33,422][20381] Updated weights for policy 1, policy_version 185939 (0.0005) [2023-12-27 06:56:33,879][20347] Updated weights for policy 0, policy_version 186186 (0.0008) [2023-12-27 06:56:33,939][20347] Updated weights for policy 0, policy_version 186196 (0.0010) [2023-12-27 06:56:33,990][20347] Updated weights for policy 0, policy_version 186206 (0.0010) [2023-12-27 06:56:34,025][20381] Updated weights for policy 1, policy_version 185949 (0.0006) [2023-12-27 06:56:34,045][20347] Updated weights for policy 0, policy_version 186216 (0.0010) [2023-12-27 06:56:34,071][20381] Updated weights for policy 1, policy_version 185959 (0.0006) [2023-12-27 06:56:34,118][20381] Updated weights for policy 1, policy_version 185969 (0.0007) [2023-12-27 06:56:34,720][20347] Updated weights for policy 0, policy_version 186226 (0.0008) [2023-12-27 06:56:34,771][20347] Updated weights for policy 0, policy_version 186236 (0.0007) [2023-12-27 06:56:34,806][20381] Updated weights for policy 1, policy_version 185979 (0.0006) [2023-12-27 06:56:34,825][20347] Updated weights for policy 0, policy_version 186246 (0.0007) [2023-12-27 06:56:34,863][20381] Updated weights for policy 1, policy_version 185989 (0.0006) [2023-12-27 06:56:34,919][20381] Updated weights for policy 1, policy_version 185999 (0.0005) [2023-12-27 06:56:35,470][20347] Updated weights for policy 0, policy_version 186256 (0.0010) [2023-12-27 06:56:35,514][20381] Updated weights for policy 1, policy_version 186009 (0.0006) [2023-12-27 06:56:35,522][20347] Updated weights for policy 0, policy_version 186266 (0.0010) [2023-12-27 06:56:35,565][20381] Updated weights for policy 1, policy_version 186019 (0.0006) [2023-12-27 06:56:35,571][20347] Updated weights for policy 0, policy_version 186276 (0.0010) [2023-12-27 06:56:35,612][20381] Updated weights for policy 1, policy_version 186029 (0.0006) [2023-12-27 06:56:35,665][20381] Updated weights for policy 1, policy_version 186039 (0.0008) [2023-12-27 06:56:36,251][20347] Updated weights for policy 0, policy_version 186286 (0.0010) [2023-12-27 06:56:36,307][20347] Updated weights for policy 0, policy_version 186296 (0.0010) [2023-12-27 06:56:36,320][20381] Updated weights for policy 1, policy_version 186049 (0.0006) [2023-12-27 06:56:36,364][20347] Updated weights for policy 0, policy_version 186306 (0.0011) [2023-12-27 06:56:36,377][20381] Updated weights for policy 1, policy_version 186059 (0.0006) [2023-12-27 06:56:36,425][20381] Updated weights for policy 1, policy_version 186069 (0.0006) [2023-12-27 06:56:37,059][20347] Updated weights for policy 0, policy_version 186316 (0.0011) [2023-12-27 06:56:37,112][20347] Updated weights for policy 0, policy_version 186326 (0.0010) [2023-12-27 06:56:37,134][20381] Updated weights for policy 1, policy_version 186079 (0.0006) [2023-12-27 06:56:37,178][20347] Updated weights for policy 0, policy_version 186336 (0.0009) [2023-12-27 06:56:37,190][20381] Updated weights for policy 1, policy_version 186089 (0.0009) [2023-12-27 06:56:37,243][20381] Updated weights for policy 1, policy_version 186099 (0.0011) [2023-12-27 06:56:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 95371264. Throughput: 0: 10473.7, 1: 10210.2. Samples: 95334560. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000186104_47652864.pth... [2023-12-27 06:56:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000186344_47718400.pth... [2023-12-27 06:56:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000184888_47341568.pth [2023-12-27 06:56:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000185128_47407104.pth [2023-12-27 06:56:37,844][20347] Updated weights for policy 0, policy_version 186346 (0.0010) [2023-12-27 06:56:37,888][20381] Updated weights for policy 1, policy_version 186109 (0.0008) [2023-12-27 06:56:37,909][20347] Updated weights for policy 0, policy_version 186356 (0.0010) [2023-12-27 06:56:37,942][20381] Updated weights for policy 1, policy_version 186119 (0.0007) [2023-12-27 06:56:37,966][20347] Updated weights for policy 0, policy_version 186366 (0.0009) [2023-12-27 06:56:37,993][20381] Updated weights for policy 1, policy_version 186129 (0.0008) [2023-12-27 06:56:38,029][20347] Updated weights for policy 0, policy_version 186376 (0.0010) [2023-12-27 06:56:38,636][20347] Updated weights for policy 0, policy_version 186386 (0.0010) [2023-12-27 06:56:38,688][20347] Updated weights for policy 0, policy_version 186396 (0.0010) [2023-12-27 06:56:38,747][20347] Updated weights for policy 0, policy_version 186406 (0.0010) [2023-12-27 06:56:38,750][20381] Updated weights for policy 1, policy_version 186139 (0.0006) [2023-12-27 06:56:38,798][20381] Updated weights for policy 1, policy_version 186149 (0.0008) [2023-12-27 06:56:38,847][20381] Updated weights for policy 1, policy_version 186159 (0.0008) [2023-12-27 06:56:39,396][20347] Updated weights for policy 0, policy_version 186416 (0.0008) [2023-12-27 06:56:39,460][20347] Updated weights for policy 0, policy_version 186426 (0.0006) [2023-12-27 06:56:39,521][20347] Updated weights for policy 0, policy_version 186436 (0.0006) [2023-12-27 06:56:39,579][20381] Updated weights for policy 1, policy_version 186169 (0.0008) [2023-12-27 06:56:39,639][20381] Updated weights for policy 1, policy_version 186179 (0.0009) [2023-12-27 06:56:39,701][20381] Updated weights for policy 1, policy_version 186189 (0.0008) [2023-12-27 06:56:39,766][20381] Updated weights for policy 1, policy_version 186199 (0.0008) [2023-12-27 06:56:40,158][20347] Updated weights for policy 0, policy_version 186446 (0.0007) [2023-12-27 06:56:40,211][20347] Updated weights for policy 0, policy_version 186456 (0.0008) [2023-12-27 06:56:40,268][20347] Updated weights for policy 0, policy_version 186466 (0.0008) [2023-12-27 06:56:40,487][20381] Updated weights for policy 1, policy_version 186209 (0.0010) [2023-12-27 06:56:40,546][20381] Updated weights for policy 1, policy_version 186219 (0.0011) [2023-12-27 06:56:40,598][20381] Updated weights for policy 1, policy_version 186229 (0.0008) [2023-12-27 06:56:40,916][20347] Updated weights for policy 0, policy_version 186476 (0.0007) [2023-12-27 06:56:40,983][20347] Updated weights for policy 0, policy_version 186486 (0.0009) [2023-12-27 06:56:41,050][20347] Updated weights for policy 0, policy_version 186496 (0.0008) [2023-12-27 06:56:41,253][20381] Updated weights for policy 1, policy_version 186239 (0.0006) [2023-12-27 06:56:41,317][20381] Updated weights for policy 1, policy_version 186249 (0.0009) [2023-12-27 06:56:41,382][20381] Updated weights for policy 1, policy_version 186259 (0.0008) [2023-12-27 06:56:41,700][20347] Updated weights for policy 0, policy_version 186506 (0.0008) [2023-12-27 06:56:41,767][20347] Updated weights for policy 0, policy_version 186516 (0.0008) [2023-12-27 06:56:41,828][20347] Updated weights for policy 0, policy_version 186526 (0.0008) [2023-12-27 06:56:41,888][20347] Updated weights for policy 0, policy_version 186536 (0.0008) [2023-12-27 06:56:42,098][20381] Updated weights for policy 1, policy_version 186269 (0.0007) [2023-12-27 06:56:42,161][20381] Updated weights for policy 1, policy_version 186279 (0.0006) [2023-12-27 06:56:42,219][20381] Updated weights for policy 1, policy_version 186289 (0.0006) [2023-12-27 06:56:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 95469568. Throughput: 0: 10519.8, 1: 10166.1. Samples: 95458188. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:42,623][20347] Updated weights for policy 0, policy_version 186546 (0.0008) [2023-12-27 06:56:42,678][20347] Updated weights for policy 0, policy_version 186556 (0.0008) [2023-12-27 06:56:42,726][20347] Updated weights for policy 0, policy_version 186566 (0.0009) [2023-12-27 06:56:42,823][20381] Updated weights for policy 1, policy_version 186299 (0.0006) [2023-12-27 06:56:42,869][20381] Updated weights for policy 1, policy_version 186309 (0.0005) [2023-12-27 06:56:42,931][20381] Updated weights for policy 1, policy_version 186319 (0.0005) [2023-12-27 06:56:43,420][20347] Updated weights for policy 0, policy_version 186576 (0.0005) [2023-12-27 06:56:43,483][20347] Updated weights for policy 0, policy_version 186586 (0.0005) [2023-12-27 06:56:43,543][20347] Updated weights for policy 0, policy_version 186596 (0.0006) [2023-12-27 06:56:43,548][20381] Updated weights for policy 1, policy_version 186329 (0.0010) [2023-12-27 06:56:43,597][20381] Updated weights for policy 1, policy_version 186339 (0.0005) [2023-12-27 06:56:43,659][20381] Updated weights for policy 1, policy_version 186349 (0.0005) [2023-12-27 06:56:43,712][20381] Updated weights for policy 1, policy_version 186359 (0.0006) [2023-12-27 06:56:44,116][20347] Updated weights for policy 0, policy_version 186606 (0.0009) [2023-12-27 06:56:44,174][20347] Updated weights for policy 0, policy_version 186616 (0.0010) [2023-12-27 06:56:44,240][20347] Updated weights for policy 0, policy_version 186626 (0.0010) [2023-12-27 06:56:44,272][20381] Updated weights for policy 1, policy_version 186369 (0.0006) [2023-12-27 06:56:44,335][20381] Updated weights for policy 1, policy_version 186379 (0.0006) [2023-12-27 06:56:44,389][20381] Updated weights for policy 1, policy_version 186389 (0.0008) [2023-12-27 06:56:44,865][20347] Updated weights for policy 0, policy_version 186636 (0.0010) [2023-12-27 06:56:44,924][20347] Updated weights for policy 0, policy_version 186646 (0.0008) [2023-12-27 06:56:44,986][20347] Updated weights for policy 0, policy_version 186656 (0.0007) [2023-12-27 06:56:45,058][20381] Updated weights for policy 1, policy_version 186399 (0.0007) [2023-12-27 06:56:45,119][20381] Updated weights for policy 1, policy_version 186409 (0.0007) [2023-12-27 06:56:45,175][20381] Updated weights for policy 1, policy_version 186419 (0.0011) [2023-12-27 06:56:45,600][20347] Updated weights for policy 0, policy_version 186666 (0.0007) [2023-12-27 06:56:45,649][20347] Updated weights for policy 0, policy_version 186676 (0.0005) [2023-12-27 06:56:45,711][20347] Updated weights for policy 0, policy_version 186686 (0.0010) [2023-12-27 06:56:45,771][20347] Updated weights for policy 0, policy_version 186696 (0.0010) [2023-12-27 06:56:45,836][20381] Updated weights for policy 1, policy_version 186429 (0.0009) [2023-12-27 06:56:45,893][20381] Updated weights for policy 1, policy_version 186439 (0.0009) [2023-12-27 06:56:45,948][20381] Updated weights for policy 1, policy_version 186449 (0.0008) [2023-12-27 06:56:46,343][20347] Updated weights for policy 0, policy_version 186706 (0.0010) [2023-12-27 06:56:46,404][20347] Updated weights for policy 0, policy_version 186716 (0.0010) [2023-12-27 06:56:46,464][20347] Updated weights for policy 0, policy_version 186726 (0.0010) [2023-12-27 06:56:46,621][20381] Updated weights for policy 1, policy_version 186459 (0.0008) [2023-12-27 06:56:46,677][20381] Updated weights for policy 1, policy_version 186469 (0.0005) [2023-12-27 06:56:46,734][20381] Updated weights for policy 1, policy_version 186479 (0.0005) [2023-12-27 06:56:47,060][20347] Updated weights for policy 0, policy_version 186736 (0.0006) [2023-12-27 06:56:47,117][20347] Updated weights for policy 0, policy_version 186746 (0.0005) [2023-12-27 06:56:47,174][20347] Updated weights for policy 0, policy_version 186756 (0.0005) [2023-12-27 06:56:47,341][20381] Updated weights for policy 1, policy_version 186489 (0.0005) [2023-12-27 06:56:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 95576064. Throughput: 0: 10543.7, 1: 10229.2. Samples: 95587044. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:47,404][20381] Updated weights for policy 1, policy_version 186499 (0.0005) [2023-12-27 06:56:47,456][20381] Updated weights for policy 1, policy_version 186509 (0.0005) [2023-12-27 06:56:47,514][20381] Updated weights for policy 1, policy_version 186519 (0.0005) [2023-12-27 06:56:47,731][20347] Updated weights for policy 0, policy_version 186766 (0.0006) [2023-12-27 06:56:47,778][20347] Updated weights for policy 0, policy_version 186776 (0.0010) [2023-12-27 06:56:47,829][20347] Updated weights for policy 0, policy_version 186786 (0.0010) [2023-12-27 06:56:48,111][20381] Updated weights for policy 1, policy_version 186529 (0.0008) [2023-12-27 06:56:48,164][20381] Updated weights for policy 1, policy_version 186539 (0.0007) [2023-12-27 06:56:48,222][20381] Updated weights for policy 1, policy_version 186549 (0.0006) [2023-12-27 06:56:48,447][20347] Updated weights for policy 0, policy_version 186796 (0.0007) [2023-12-27 06:56:48,492][20347] Updated weights for policy 0, policy_version 186806 (0.0010) [2023-12-27 06:56:48,548][20347] Updated weights for policy 0, policy_version 186816 (0.0010) [2023-12-27 06:56:48,912][20381] Updated weights for policy 1, policy_version 186559 (0.0008) [2023-12-27 06:56:48,969][20381] Updated weights for policy 1, policy_version 186569 (0.0007) [2023-12-27 06:56:49,027][20381] Updated weights for policy 1, policy_version 186579 (0.0008) [2023-12-27 06:56:49,234][20347] Updated weights for policy 0, policy_version 186826 (0.0011) [2023-12-27 06:56:49,290][20347] Updated weights for policy 0, policy_version 186836 (0.0010) [2023-12-27 06:56:49,349][20347] Updated weights for policy 0, policy_version 186846 (0.0011) [2023-12-27 06:56:49,413][20347] Updated weights for policy 0, policy_version 186856 (0.0010) [2023-12-27 06:56:49,697][20381] Updated weights for policy 1, policy_version 186589 (0.0007) [2023-12-27 06:56:49,752][20381] Updated weights for policy 1, policy_version 186599 (0.0007) [2023-12-27 06:56:49,809][20381] Updated weights for policy 1, policy_version 186609 (0.0006) [2023-12-27 06:56:50,073][20347] Updated weights for policy 0, policy_version 186866 (0.0006) [2023-12-27 06:56:50,130][20347] Updated weights for policy 0, policy_version 186876 (0.0008) [2023-12-27 06:56:50,180][20347] Updated weights for policy 0, policy_version 186886 (0.0007) [2023-12-27 06:56:50,544][20381] Updated weights for policy 1, policy_version 186619 (0.0006) [2023-12-27 06:56:50,612][20381] Updated weights for policy 1, policy_version 186629 (0.0006) [2023-12-27 06:56:50,665][20381] Updated weights for policy 1, policy_version 186639 (0.0006) [2023-12-27 06:56:50,798][20347] Updated weights for policy 0, policy_version 186896 (0.0010) [2023-12-27 06:56:50,850][20347] Updated weights for policy 0, policy_version 186906 (0.0010) [2023-12-27 06:56:50,897][20347] Updated weights for policy 0, policy_version 186916 (0.0009) [2023-12-27 06:56:51,321][20381] Updated weights for policy 1, policy_version 186649 (0.0007) [2023-12-27 06:56:51,386][20381] Updated weights for policy 1, policy_version 186659 (0.0009) [2023-12-27 06:56:51,451][20381] Updated weights for policy 1, policy_version 186669 (0.0009) [2023-12-27 06:56:51,508][20381] Updated weights for policy 1, policy_version 186679 (0.0009) [2023-12-27 06:56:51,593][20347] Updated weights for policy 0, policy_version 186926 (0.0006) [2023-12-27 06:56:51,650][20347] Updated weights for policy 0, policy_version 186936 (0.0008) [2023-12-27 06:56:51,719][20347] Updated weights for policy 0, policy_version 186946 (0.0007) [2023-12-27 06:56:52,176][20381] Updated weights for policy 1, policy_version 186689 (0.0010) [2023-12-27 06:56:52,233][20381] Updated weights for policy 1, policy_version 186699 (0.0010) [2023-12-27 06:56:52,293][20381] Updated weights for policy 1, policy_version 186709 (0.0011) [2023-12-27 06:56:52,298][20347] Updated weights for policy 0, policy_version 186956 (0.0007) [2023-12-27 06:56:52,354][20347] Updated weights for policy 0, policy_version 186966 (0.0007) [2023-12-27 06:56:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 95682560. Throughput: 0: 10540.4, 1: 10285.0. Samples: 95651168. Policy #0 lag: (min: 4.0, avg: 6.0, max: 36.0) [2023-12-27 06:56:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000186712_47808512.pth... [2023-12-27 06:56:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000185496_47497216.pth [2023-12-27 06:56:52,407][20347] Updated weights for policy 0, policy_version 186976 (0.0007) [2023-12-27 06:56:52,445][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000186984_47882240.pth... [2023-12-27 06:56:52,448][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000185736_47562752.pth [2023-12-27 06:56:52,962][20381] Updated weights for policy 1, policy_version 186719 (0.0008) [2023-12-27 06:56:53,017][20381] Updated weights for policy 1, policy_version 186729 (0.0006) [2023-12-27 06:56:53,077][20381] Updated weights for policy 1, policy_version 186739 (0.0007) [2023-12-27 06:56:53,120][20347] Updated weights for policy 0, policy_version 186986 (0.0007) [2023-12-27 06:56:53,165][20347] Updated weights for policy 0, policy_version 186996 (0.0007) [2023-12-27 06:56:53,216][20347] Updated weights for policy 0, policy_version 187006 (0.0009) [2023-12-27 06:56:53,265][20347] Updated weights for policy 0, policy_version 187016 (0.0010) [2023-12-27 06:56:53,782][20381] Updated weights for policy 1, policy_version 186749 (0.0006) [2023-12-27 06:56:53,845][20381] Updated weights for policy 1, policy_version 186759 (0.0005) [2023-12-27 06:56:53,875][20347] Updated weights for policy 0, policy_version 187026 (0.0010) [2023-12-27 06:56:53,899][20381] Updated weights for policy 1, policy_version 186769 (0.0006) [2023-12-27 06:56:53,927][20347] Updated weights for policy 0, policy_version 187036 (0.0010) [2023-12-27 06:56:53,985][20347] Updated weights for policy 0, policy_version 187046 (0.0008) [2023-12-27 06:56:54,528][20381] Updated weights for policy 1, policy_version 186779 (0.0006) [2023-12-27 06:56:54,553][20347] Updated weights for policy 0, policy_version 187056 (0.0008) [2023-12-27 06:56:54,587][20381] Updated weights for policy 1, policy_version 186789 (0.0006) [2023-12-27 06:56:54,605][20347] Updated weights for policy 0, policy_version 187066 (0.0008) [2023-12-27 06:56:54,646][20381] Updated weights for policy 1, policy_version 186799 (0.0006) [2023-12-27 06:56:54,660][20347] Updated weights for policy 0, policy_version 187076 (0.0007) [2023-12-27 06:56:55,305][20347] Updated weights for policy 0, policy_version 187086 (0.0007) [2023-12-27 06:56:55,358][20347] Updated weights for policy 0, policy_version 187096 (0.0005) [2023-12-27 06:56:55,376][20381] Updated weights for policy 1, policy_version 186809 (0.0008) [2023-12-27 06:56:55,412][20347] Updated weights for policy 0, policy_version 187106 (0.0010) [2023-12-27 06:56:55,437][20381] Updated weights for policy 1, policy_version 186819 (0.0008) [2023-12-27 06:56:55,490][20381] Updated weights for policy 1, policy_version 186829 (0.0005) [2023-12-27 06:56:55,546][20381] Updated weights for policy 1, policy_version 186839 (0.0007) [2023-12-27 06:56:56,068][20347] Updated weights for policy 0, policy_version 187116 (0.0010) [2023-12-27 06:56:56,130][20347] Updated weights for policy 0, policy_version 187126 (0.0005) [2023-12-27 06:56:56,181][20347] Updated weights for policy 0, policy_version 187136 (0.0005) [2023-12-27 06:56:56,225][20381] Updated weights for policy 1, policy_version 186849 (0.0010) [2023-12-27 06:56:56,271][20381] Updated weights for policy 1, policy_version 186859 (0.0010) [2023-12-27 06:56:56,320][20381] Updated weights for policy 1, policy_version 186869 (0.0010) [2023-12-27 06:56:56,784][20347] Updated weights for policy 0, policy_version 187146 (0.0005) [2023-12-27 06:56:56,842][20347] Updated weights for policy 0, policy_version 187156 (0.0005) [2023-12-27 06:56:56,895][20347] Updated weights for policy 0, policy_version 187166 (0.0005) [2023-12-27 06:56:56,951][20347] Updated weights for policy 0, policy_version 187176 (0.0007) [2023-12-27 06:56:57,029][20381] Updated weights for policy 1, policy_version 186879 (0.0010) [2023-12-27 06:56:57,076][20381] Updated weights for policy 1, policy_version 186889 (0.0010) [2023-12-27 06:56:57,128][20381] Updated weights for policy 1, policy_version 186899 (0.0008) [2023-12-27 06:56:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 95789056. Throughput: 0: 10551.6, 1: 10334.5. Samples: 95776844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:56:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:56:57,512][20347] Updated weights for policy 0, policy_version 187186 (0.0007) [2023-12-27 06:56:57,574][20347] Updated weights for policy 0, policy_version 187196 (0.0007) [2023-12-27 06:56:57,627][20347] Updated weights for policy 0, policy_version 187206 (0.0007) [2023-12-27 06:56:57,836][20381] Updated weights for policy 1, policy_version 186909 (0.0006) [2023-12-27 06:56:57,888][20381] Updated weights for policy 1, policy_version 186919 (0.0007) [2023-12-27 06:56:57,945][20381] Updated weights for policy 1, policy_version 186929 (0.0007) [2023-12-27 06:56:58,309][20347] Updated weights for policy 0, policy_version 187216 (0.0006) [2023-12-27 06:56:58,374][20347] Updated weights for policy 0, policy_version 187226 (0.0010) [2023-12-27 06:56:58,438][20347] Updated weights for policy 0, policy_version 187236 (0.0008) [2023-12-27 06:56:58,698][20381] Updated weights for policy 1, policy_version 186939 (0.0007) [2023-12-27 06:56:58,751][20381] Updated weights for policy 1, policy_version 186949 (0.0006) [2023-12-27 06:56:58,816][20381] Updated weights for policy 1, policy_version 186959 (0.0007) [2023-12-27 06:56:59,242][20347] Updated weights for policy 0, policy_version 187246 (0.0009) [2023-12-27 06:56:59,311][20347] Updated weights for policy 0, policy_version 187256 (0.0007) [2023-12-27 06:56:59,377][20347] Updated weights for policy 0, policy_version 187266 (0.0008) [2023-12-27 06:56:59,561][20381] Updated weights for policy 1, policy_version 186969 (0.0008) [2023-12-27 06:56:59,625][20381] Updated weights for policy 1, policy_version 186979 (0.0007) [2023-12-27 06:56:59,676][20381] Updated weights for policy 1, policy_version 186989 (0.0008) [2023-12-27 06:56:59,732][20381] Updated weights for policy 1, policy_version 186999 (0.0008) [2023-12-27 06:57:00,057][20347] Updated weights for policy 0, policy_version 187276 (0.0007) [2023-12-27 06:57:00,094][20291] KL-divergence is very high: 319.9067 [2023-12-27 06:57:00,108][20347] Updated weights for policy 0, policy_version 187286 (0.0010) [2023-12-27 06:57:00,116][20291] KL-divergence is very high: 893.4295 [2023-12-27 06:57:00,139][20291] KL-divergence is very high: 478.7262 [2023-12-27 06:57:00,162][20291] KL-divergence is very high: 371.0239 [2023-12-27 06:57:00,167][20347] Updated weights for policy 0, policy_version 187296 (0.0009) [2023-12-27 06:57:00,340][20381] Updated weights for policy 1, policy_version 187009 (0.0006) [2023-12-27 06:57:00,399][20381] Updated weights for policy 1, policy_version 187019 (0.0005) [2023-12-27 06:57:00,447][20381] Updated weights for policy 1, policy_version 187029 (0.0008) [2023-12-27 06:57:00,751][20347] Updated weights for policy 0, policy_version 187306 (0.0007) [2023-12-27 06:57:00,812][20347] Updated weights for policy 0, policy_version 187316 (0.0010) [2023-12-27 06:57:00,869][20347] Updated weights for policy 0, policy_version 187326 (0.0010) [2023-12-27 06:57:00,921][20347] Updated weights for policy 0, policy_version 187336 (0.0006) [2023-12-27 06:57:01,047][20381] Updated weights for policy 1, policy_version 187039 (0.0008) [2023-12-27 06:57:01,101][20381] Updated weights for policy 1, policy_version 187049 (0.0008) [2023-12-27 06:57:01,163][20381] Updated weights for policy 1, policy_version 187059 (0.0007) [2023-12-27 06:57:01,584][20347] Updated weights for policy 0, policy_version 187346 (0.0010) [2023-12-27 06:57:01,639][20347] Updated weights for policy 0, policy_version 187356 (0.0009) [2023-12-27 06:57:01,686][20347] Updated weights for policy 0, policy_version 187366 (0.0008) [2023-12-27 06:57:01,904][20381] Updated weights for policy 1, policy_version 187069 (0.0008) [2023-12-27 06:57:01,962][20381] Updated weights for policy 1, policy_version 187079 (0.0008) [2023-12-27 06:57:02,014][20381] Updated weights for policy 1, policy_version 187089 (0.0007) [2023-12-27 06:57:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 95887360. Throughput: 0: 10536.0, 1: 10334.9. Samples: 95900528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:02,363][19170] Avg episode reward: [(0, '995.790'), (1, '1000.000')] [2023-12-27 06:57:02,378][20347] Updated weights for policy 0, policy_version 187376 (0.0009) [2023-12-27 06:57:02,438][20347] Updated weights for policy 0, policy_version 187386 (0.0007) [2023-12-27 06:57:02,494][20347] Updated weights for policy 0, policy_version 187396 (0.0005) [2023-12-27 06:57:02,622][20381] Updated weights for policy 1, policy_version 187099 (0.0009) [2023-12-27 06:57:02,681][20381] Updated weights for policy 1, policy_version 187109 (0.0008) [2023-12-27 06:57:02,738][20381] Updated weights for policy 1, policy_version 187119 (0.0008) [2023-12-27 06:57:03,082][20347] Updated weights for policy 0, policy_version 187406 (0.0007) [2023-12-27 06:57:03,139][20347] Updated weights for policy 0, policy_version 187416 (0.0005) [2023-12-27 06:57:03,194][20347] Updated weights for policy 0, policy_version 187426 (0.0007) [2023-12-27 06:57:03,445][20381] Updated weights for policy 1, policy_version 187129 (0.0007) [2023-12-27 06:57:03,506][20381] Updated weights for policy 1, policy_version 187139 (0.0007) [2023-12-27 06:57:03,568][20381] Updated weights for policy 1, policy_version 187149 (0.0005) [2023-12-27 06:57:03,628][20381] Updated weights for policy 1, policy_version 187159 (0.0005) [2023-12-27 06:57:03,862][20347] Updated weights for policy 0, policy_version 187436 (0.0008) [2023-12-27 06:57:03,919][20347] Updated weights for policy 0, policy_version 187446 (0.0008) [2023-12-27 06:57:03,979][20347] Updated weights for policy 0, policy_version 187456 (0.0008) [2023-12-27 06:57:04,152][20381] Updated weights for policy 1, policy_version 187169 (0.0006) [2023-12-27 06:57:04,207][20381] Updated weights for policy 1, policy_version 187179 (0.0008) [2023-12-27 06:57:04,261][20381] Updated weights for policy 1, policy_version 187189 (0.0010) [2023-12-27 06:57:04,658][20347] Updated weights for policy 0, policy_version 187466 (0.0008) [2023-12-27 06:57:04,718][20347] Updated weights for policy 0, policy_version 187476 (0.0007) [2023-12-27 06:57:04,768][20347] Updated weights for policy 0, policy_version 187486 (0.0008) [2023-12-27 06:57:04,830][20347] Updated weights for policy 0, policy_version 187496 (0.0008) [2023-12-27 06:57:04,915][20381] Updated weights for policy 1, policy_version 187199 (0.0007) [2023-12-27 06:57:04,972][20381] Updated weights for policy 1, policy_version 187209 (0.0006) [2023-12-27 06:57:05,031][20381] Updated weights for policy 1, policy_version 187219 (0.0011) [2023-12-27 06:57:05,529][20347] Updated weights for policy 0, policy_version 187506 (0.0007) [2023-12-27 06:57:05,582][20347] Updated weights for policy 0, policy_version 187516 (0.0007) [2023-12-27 06:57:05,642][20347] Updated weights for policy 0, policy_version 187526 (0.0007) [2023-12-27 06:57:05,671][20381] Updated weights for policy 1, policy_version 187229 (0.0009) [2023-12-27 06:57:05,725][20381] Updated weights for policy 1, policy_version 187239 (0.0005) [2023-12-27 06:57:05,777][20381] Updated weights for policy 1, policy_version 187249 (0.0005) [2023-12-27 06:57:06,379][20347] Updated weights for policy 0, policy_version 187536 (0.0007) [2023-12-27 06:57:06,392][20381] Updated weights for policy 1, policy_version 187259 (0.0006) [2023-12-27 06:57:06,432][20347] Updated weights for policy 0, policy_version 187546 (0.0006) [2023-12-27 06:57:06,450][20381] Updated weights for policy 1, policy_version 187269 (0.0006) [2023-12-27 06:57:06,483][20347] Updated weights for policy 0, policy_version 187556 (0.0006) [2023-12-27 06:57:06,507][20381] Updated weights for policy 1, policy_version 187279 (0.0007) [2023-12-27 06:57:07,137][20347] Updated weights for policy 0, policy_version 187566 (0.0006) [2023-12-27 06:57:07,150][20381] Updated weights for policy 1, policy_version 187289 (0.0006) [2023-12-27 06:57:07,192][20347] Updated weights for policy 0, policy_version 187576 (0.0007) [2023-12-27 06:57:07,211][20381] Updated weights for policy 1, policy_version 187299 (0.0007) [2023-12-27 06:57:07,244][20347] Updated weights for policy 0, policy_version 187586 (0.0005) [2023-12-27 06:57:07,271][20381] Updated weights for policy 1, policy_version 187309 (0.0006) [2023-12-27 06:57:07,325][20381] Updated weights for policy 1, policy_version 187319 (0.0007) [2023-12-27 06:57:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 96002048. Throughput: 0: 10573.3, 1: 10345.4. Samples: 95964528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:07,363][19170] Avg episode reward: [(0, '993.950'), (1, '1000.000')] [2023-12-27 06:57:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000187592_48037888.pth... [2023-12-27 06:57:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000187320_47964160.pth... [2023-12-27 06:57:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000186344_47718400.pth [2023-12-27 06:57:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000186104_47652864.pth [2023-12-27 06:57:07,824][20347] Updated weights for policy 0, policy_version 187596 (0.0005) [2023-12-27 06:57:07,881][20347] Updated weights for policy 0, policy_version 187606 (0.0005) [2023-12-27 06:57:07,939][20347] Updated weights for policy 0, policy_version 187616 (0.0008) [2023-12-27 06:57:08,029][20381] Updated weights for policy 1, policy_version 187329 (0.0008) [2023-12-27 06:57:08,089][20381] Updated weights for policy 1, policy_version 187339 (0.0007) [2023-12-27 06:57:08,141][20381] Updated weights for policy 1, policy_version 187349 (0.0007) [2023-12-27 06:57:08,574][20347] Updated weights for policy 0, policy_version 187626 (0.0007) [2023-12-27 06:57:08,636][20347] Updated weights for policy 0, policy_version 187636 (0.0005) [2023-12-27 06:57:08,697][20347] Updated weights for policy 0, policy_version 187646 (0.0005) [2023-12-27 06:57:08,763][20347] Updated weights for policy 0, policy_version 187656 (0.0005) [2023-12-27 06:57:08,828][20381] Updated weights for policy 1, policy_version 187359 (0.0006) [2023-12-27 06:57:08,896][20381] Updated weights for policy 1, policy_version 187369 (0.0006) [2023-12-27 06:57:08,954][20381] Updated weights for policy 1, policy_version 187379 (0.0006) [2023-12-27 06:57:09,444][20347] Updated weights for policy 0, policy_version 187666 (0.0009) [2023-12-27 06:57:09,502][20347] Updated weights for policy 0, policy_version 187676 (0.0008) [2023-12-27 06:57:09,570][20347] Updated weights for policy 0, policy_version 187686 (0.0009) [2023-12-27 06:57:09,595][20381] Updated weights for policy 1, policy_version 187389 (0.0006) [2023-12-27 06:57:09,664][20381] Updated weights for policy 1, policy_version 187399 (0.0006) [2023-12-27 06:57:09,732][20381] Updated weights for policy 1, policy_version 187409 (0.0006) [2023-12-27 06:57:10,310][20347] Updated weights for policy 0, policy_version 187696 (0.0007) [2023-12-27 06:57:10,373][20347] Updated weights for policy 0, policy_version 187706 (0.0005) [2023-12-27 06:57:10,391][20381] Updated weights for policy 1, policy_version 187419 (0.0006) [2023-12-27 06:57:10,431][20347] Updated weights for policy 0, policy_version 187716 (0.0007) [2023-12-27 06:57:10,452][20381] Updated weights for policy 1, policy_version 187429 (0.0006) [2023-12-27 06:57:10,518][20381] Updated weights for policy 1, policy_version 187439 (0.0006) [2023-12-27 06:57:11,006][20347] Updated weights for policy 0, policy_version 187726 (0.0006) [2023-12-27 06:57:11,071][20347] Updated weights for policy 0, policy_version 187736 (0.0008) [2023-12-27 06:57:11,130][20347] Updated weights for policy 0, policy_version 187746 (0.0008) [2023-12-27 06:57:11,245][20381] Updated weights for policy 1, policy_version 187449 (0.0008) [2023-12-27 06:57:11,304][20381] Updated weights for policy 1, policy_version 187459 (0.0008) [2023-12-27 06:57:11,369][20381] Updated weights for policy 1, policy_version 187469 (0.0008) [2023-12-27 06:57:11,430][20381] Updated weights for policy 1, policy_version 187479 (0.0007) [2023-12-27 06:57:11,882][20347] Updated weights for policy 0, policy_version 187756 (0.0008) [2023-12-27 06:57:11,938][20347] Updated weights for policy 0, policy_version 187766 (0.0007) [2023-12-27 06:57:11,989][20347] Updated weights for policy 0, policy_version 187776 (0.0007) [2023-12-27 06:57:12,107][20381] Updated weights for policy 1, policy_version 187489 (0.0007) [2023-12-27 06:57:12,164][20381] Updated weights for policy 1, policy_version 187499 (0.0005) [2023-12-27 06:57:12,224][20381] Updated weights for policy 1, policy_version 187509 (0.0005) [2023-12-27 06:57:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 96100352. Throughput: 0: 10628.3, 1: 10368.3. Samples: 96088912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:12,363][19170] Avg episode reward: [(0, '999.530'), (1, '1000.000')] [2023-12-27 06:57:12,754][20347] Updated weights for policy 0, policy_version 187786 (0.0007) [2023-12-27 06:57:12,816][20347] Updated weights for policy 0, policy_version 187796 (0.0008) [2023-12-27 06:57:12,876][20347] Updated weights for policy 0, policy_version 187806 (0.0007) [2023-12-27 06:57:12,893][20381] Updated weights for policy 1, policy_version 187519 (0.0010) [2023-12-27 06:57:12,938][20347] Updated weights for policy 0, policy_version 187816 (0.0006) [2023-12-27 06:57:12,952][20381] Updated weights for policy 1, policy_version 187529 (0.0009) [2023-12-27 06:57:13,014][20381] Updated weights for policy 1, policy_version 187539 (0.0005) [2023-12-27 06:57:13,639][20347] Updated weights for policy 0, policy_version 187826 (0.0007) [2023-12-27 06:57:13,644][20381] Updated weights for policy 1, policy_version 187549 (0.0008) [2023-12-27 06:57:13,696][20347] Updated weights for policy 0, policy_version 187836 (0.0006) [2023-12-27 06:57:13,702][20381] Updated weights for policy 1, policy_version 187559 (0.0011) [2023-12-27 06:57:13,751][20347] Updated weights for policy 0, policy_version 187846 (0.0005) [2023-12-27 06:57:13,759][20381] Updated weights for policy 1, policy_version 187569 (0.0010) [2023-12-27 06:57:14,343][20381] Updated weights for policy 1, policy_version 187579 (0.0006) [2023-12-27 06:57:14,398][20381] Updated weights for policy 1, policy_version 187589 (0.0006) [2023-12-27 06:57:14,454][20381] Updated weights for policy 1, policy_version 187599 (0.0006) [2023-12-27 06:57:14,482][20347] Updated weights for policy 0, policy_version 187856 (0.0009) [2023-12-27 06:57:14,543][20347] Updated weights for policy 0, policy_version 187866 (0.0008) [2023-12-27 06:57:14,600][20347] Updated weights for policy 0, policy_version 187876 (0.0008) [2023-12-27 06:57:15,125][20381] Updated weights for policy 1, policy_version 187609 (0.0005) [2023-12-27 06:57:15,187][20381] Updated weights for policy 1, policy_version 187619 (0.0010) [2023-12-27 06:57:15,232][20347] Updated weights for policy 0, policy_version 187886 (0.0006) [2023-12-27 06:57:15,248][20381] Updated weights for policy 1, policy_version 187629 (0.0011) [2023-12-27 06:57:15,293][20347] Updated weights for policy 0, policy_version 187896 (0.0009) [2023-12-27 06:57:15,305][20381] Updated weights for policy 1, policy_version 187639 (0.0010) [2023-12-27 06:57:15,342][20347] Updated weights for policy 0, policy_version 187906 (0.0007) [2023-12-27 06:57:15,966][20381] Updated weights for policy 1, policy_version 187649 (0.0007) [2023-12-27 06:57:15,997][20347] Updated weights for policy 0, policy_version 187916 (0.0007) [2023-12-27 06:57:16,021][20381] Updated weights for policy 1, policy_version 187659 (0.0005) [2023-12-27 06:57:16,050][20347] Updated weights for policy 0, policy_version 187926 (0.0005) [2023-12-27 06:57:16,070][20381] Updated weights for policy 1, policy_version 187669 (0.0006) [2023-12-27 06:57:16,099][20347] Updated weights for policy 0, policy_version 187936 (0.0006) [2023-12-27 06:57:16,767][20381] Updated weights for policy 1, policy_version 187679 (0.0007) [2023-12-27 06:57:16,796][20347] Updated weights for policy 0, policy_version 187946 (0.0007) [2023-12-27 06:57:16,820][20381] Updated weights for policy 1, policy_version 187689 (0.0006) [2023-12-27 06:57:16,847][20347] Updated weights for policy 0, policy_version 187956 (0.0006) [2023-12-27 06:57:16,874][20381] Updated weights for policy 1, policy_version 187699 (0.0005) [2023-12-27 06:57:16,896][20347] Updated weights for policy 0, policy_version 187966 (0.0007) [2023-12-27 06:57:16,955][20347] Updated weights for policy 0, policy_version 187976 (0.0008) [2023-12-27 06:57:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 96198656. Throughput: 0: 10493.2, 1: 10382.1. Samples: 96210732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:17,486][20381] Updated weights for policy 1, policy_version 187709 (0.0006) [2023-12-27 06:57:17,539][20381] Updated weights for policy 1, policy_version 187719 (0.0008) [2023-12-27 06:57:17,589][20381] Updated weights for policy 1, policy_version 187729 (0.0008) [2023-12-27 06:57:17,679][20347] Updated weights for policy 0, policy_version 187986 (0.0007) [2023-12-27 06:57:17,730][20347] Updated weights for policy 0, policy_version 187996 (0.0007) [2023-12-27 06:57:17,779][20347] Updated weights for policy 0, policy_version 188006 (0.0008) [2023-12-27 06:57:18,275][20381] Updated weights for policy 1, policy_version 187739 (0.0008) [2023-12-27 06:57:18,323][20381] Updated weights for policy 1, policy_version 187749 (0.0008) [2023-12-27 06:57:18,378][20381] Updated weights for policy 1, policy_version 187759 (0.0009) [2023-12-27 06:57:18,484][20347] Updated weights for policy 0, policy_version 188016 (0.0007) [2023-12-27 06:57:18,531][20347] Updated weights for policy 0, policy_version 188026 (0.0007) [2023-12-27 06:57:18,590][20347] Updated weights for policy 0, policy_version 188036 (0.0008) [2023-12-27 06:57:19,068][20381] Updated weights for policy 1, policy_version 187769 (0.0007) [2023-12-27 06:57:19,126][20381] Updated weights for policy 1, policy_version 187779 (0.0008) [2023-12-27 06:57:19,178][20381] Updated weights for policy 1, policy_version 187789 (0.0007) [2023-12-27 06:57:19,234][20381] Updated weights for policy 1, policy_version 187799 (0.0008) [2023-12-27 06:57:19,295][20347] Updated weights for policy 0, policy_version 188046 (0.0008) [2023-12-27 06:57:19,350][20347] Updated weights for policy 0, policy_version 188056 (0.0007) [2023-12-27 06:57:19,414][20347] Updated weights for policy 0, policy_version 188066 (0.0007) [2023-12-27 06:57:19,952][20381] Updated weights for policy 1, policy_version 187809 (0.0008) [2023-12-27 06:57:20,015][20381] Updated weights for policy 1, policy_version 187819 (0.0007) [2023-12-27 06:57:20,079][20381] Updated weights for policy 1, policy_version 187829 (0.0006) [2023-12-27 06:57:20,143][20347] Updated weights for policy 0, policy_version 188076 (0.0007) [2023-12-27 06:57:20,196][20347] Updated weights for policy 0, policy_version 188086 (0.0008) [2023-12-27 06:57:20,254][20347] Updated weights for policy 0, policy_version 188096 (0.0007) [2023-12-27 06:57:20,694][20381] Updated weights for policy 1, policy_version 187839 (0.0008) [2023-12-27 06:57:20,750][20381] Updated weights for policy 1, policy_version 187849 (0.0010) [2023-12-27 06:57:20,809][20381] Updated weights for policy 1, policy_version 187859 (0.0010) [2023-12-27 06:57:20,864][20347] Updated weights for policy 0, policy_version 188106 (0.0006) [2023-12-27 06:57:20,928][20347] Updated weights for policy 0, policy_version 188116 (0.0008) [2023-12-27 06:57:20,992][20347] Updated weights for policy 0, policy_version 188126 (0.0008) [2023-12-27 06:57:21,055][20347] Updated weights for policy 0, policy_version 188136 (0.0007) [2023-12-27 06:57:21,491][20381] Updated weights for policy 1, policy_version 187869 (0.0011) [2023-12-27 06:57:21,557][20381] Updated weights for policy 1, policy_version 187879 (0.0011) [2023-12-27 06:57:21,626][20381] Updated weights for policy 1, policy_version 187889 (0.0010) [2023-12-27 06:57:21,677][20347] Updated weights for policy 0, policy_version 188146 (0.0007) [2023-12-27 06:57:21,736][20347] Updated weights for policy 0, policy_version 188156 (0.0008) [2023-12-27 06:57:21,795][20347] Updated weights for policy 0, policy_version 188166 (0.0008) [2023-12-27 06:57:22,246][20381] Updated weights for policy 1, policy_version 187899 (0.0007) [2023-12-27 06:57:22,299][20381] Updated weights for policy 1, policy_version 187909 (0.0008) [2023-12-27 06:57:22,353][20381] Updated weights for policy 1, policy_version 187919 (0.0006) [2023-12-27 06:57:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 96296960. Throughput: 0: 10468.5, 1: 10354.0. Samples: 96271572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000188168_48185344.pth... [2023-12-27 06:57:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000186984_47882240.pth [2023-12-27 06:57:22,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000187928_48119808.pth... [2023-12-27 06:57:22,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000186712_47808512.pth [2023-12-27 06:57:22,493][20347] Updated weights for policy 0, policy_version 188176 (0.0007) [2023-12-27 06:57:22,554][20347] Updated weights for policy 0, policy_version 188186 (0.0008) [2023-12-27 06:57:22,601][20347] Updated weights for policy 0, policy_version 188196 (0.0008) [2023-12-27 06:57:23,071][20381] Updated weights for policy 1, policy_version 187929 (0.0007) [2023-12-27 06:57:23,126][20381] Updated weights for policy 1, policy_version 187939 (0.0008) [2023-12-27 06:57:23,185][20381] Updated weights for policy 1, policy_version 187949 (0.0008) [2023-12-27 06:57:23,241][20381] Updated weights for policy 1, policy_version 187959 (0.0007) [2023-12-27 06:57:23,265][20347] Updated weights for policy 0, policy_version 188206 (0.0007) [2023-12-27 06:57:23,314][20347] Updated weights for policy 0, policy_version 188216 (0.0008) [2023-12-27 06:57:23,366][20347] Updated weights for policy 0, policy_version 188226 (0.0008) [2023-12-27 06:57:23,905][20381] Updated weights for policy 1, policy_version 187969 (0.0007) [2023-12-27 06:57:23,953][20381] Updated weights for policy 1, policy_version 187979 (0.0007) [2023-12-27 06:57:24,004][20347] Updated weights for policy 0, policy_version 188236 (0.0009) [2023-12-27 06:57:24,006][20381] Updated weights for policy 1, policy_version 187989 (0.0006) [2023-12-27 06:57:24,069][20347] Updated weights for policy 0, policy_version 188246 (0.0010) [2023-12-27 06:57:24,134][20347] Updated weights for policy 0, policy_version 188256 (0.0010) [2023-12-27 06:57:24,606][20381] Updated weights for policy 1, policy_version 187999 (0.0007) [2023-12-27 06:57:24,655][20381] Updated weights for policy 1, policy_version 188009 (0.0007) [2023-12-27 06:57:24,713][20381] Updated weights for policy 1, policy_version 188019 (0.0007) [2023-12-27 06:57:24,817][20347] Updated weights for policy 0, policy_version 188266 (0.0010) [2023-12-27 06:57:24,871][20347] Updated weights for policy 0, policy_version 188276 (0.0010) [2023-12-27 06:57:24,944][20347] Updated weights for policy 0, policy_version 188286 (0.0007) [2023-12-27 06:57:25,004][20347] Updated weights for policy 0, policy_version 188296 (0.0008) [2023-12-27 06:57:25,337][20381] Updated weights for policy 1, policy_version 188029 (0.0008) [2023-12-27 06:57:25,386][20381] Updated weights for policy 1, policy_version 188039 (0.0008) [2023-12-27 06:57:25,435][20381] Updated weights for policy 1, policy_version 188049 (0.0008) [2023-12-27 06:57:25,660][20347] Updated weights for policy 0, policy_version 188306 (0.0008) [2023-12-27 06:57:25,718][20347] Updated weights for policy 0, policy_version 188316 (0.0007) [2023-12-27 06:57:25,773][20347] Updated weights for policy 0, policy_version 188326 (0.0007) [2023-12-27 06:57:26,133][20381] Updated weights for policy 1, policy_version 188059 (0.0008) [2023-12-27 06:57:26,194][20381] Updated weights for policy 1, policy_version 188069 (0.0009) [2023-12-27 06:57:26,263][20381] Updated weights for policy 1, policy_version 188079 (0.0011) [2023-12-27 06:57:26,430][20347] Updated weights for policy 0, policy_version 188336 (0.0006) [2023-12-27 06:57:26,486][20347] Updated weights for policy 0, policy_version 188346 (0.0005) [2023-12-27 06:57:26,547][20347] Updated weights for policy 0, policy_version 188356 (0.0005) [2023-12-27 06:57:26,848][20381] Updated weights for policy 1, policy_version 188089 (0.0011) [2023-12-27 06:57:26,896][20381] Updated weights for policy 1, policy_version 188099 (0.0010) [2023-12-27 06:57:26,944][20381] Updated weights for policy 1, policy_version 188109 (0.0010) [2023-12-27 06:57:26,993][20381] Updated weights for policy 1, policy_version 188119 (0.0010) [2023-12-27 06:57:27,115][20347] Updated weights for policy 0, policy_version 188366 (0.0007) [2023-12-27 06:57:27,164][20347] Updated weights for policy 0, policy_version 188376 (0.0007) [2023-12-27 06:57:27,217][20347] Updated weights for policy 0, policy_version 188386 (0.0006) [2023-12-27 06:57:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 96411648. Throughput: 0: 10434.0, 1: 10406.6. Samples: 96396012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:27,675][20381] Updated weights for policy 1, policy_version 188129 (0.0006) [2023-12-27 06:57:27,739][20381] Updated weights for policy 1, policy_version 188139 (0.0009) [2023-12-27 06:57:27,790][20381] Updated weights for policy 1, policy_version 188149 (0.0009) [2023-12-27 06:57:27,862][20347] Updated weights for policy 0, policy_version 188396 (0.0007) [2023-12-27 06:57:27,921][20347] Updated weights for policy 0, policy_version 188406 (0.0007) [2023-12-27 06:57:27,979][20347] Updated weights for policy 0, policy_version 188416 (0.0008) [2023-12-27 06:57:28,421][20381] Updated weights for policy 1, policy_version 188159 (0.0010) [2023-12-27 06:57:28,480][20381] Updated weights for policy 1, policy_version 188169 (0.0011) [2023-12-27 06:57:28,537][20381] Updated weights for policy 1, policy_version 188179 (0.0011) [2023-12-27 06:57:28,663][20347] Updated weights for policy 0, policy_version 188426 (0.0006) [2023-12-27 06:57:28,725][20347] Updated weights for policy 0, policy_version 188436 (0.0008) [2023-12-27 06:57:28,787][20347] Updated weights for policy 0, policy_version 188446 (0.0008) [2023-12-27 06:57:28,848][20347] Updated weights for policy 0, policy_version 188456 (0.0007) [2023-12-27 06:57:29,199][20381] Updated weights for policy 1, policy_version 188189 (0.0009) [2023-12-27 06:57:29,256][20381] Updated weights for policy 1, policy_version 188199 (0.0010) [2023-12-27 06:57:29,312][20381] Updated weights for policy 1, policy_version 188209 (0.0009) [2023-12-27 06:57:29,564][20347] Updated weights for policy 0, policy_version 188466 (0.0008) [2023-12-27 06:57:29,622][20347] Updated weights for policy 0, policy_version 188476 (0.0008) [2023-12-27 06:57:29,678][20347] Updated weights for policy 0, policy_version 188486 (0.0008) [2023-12-27 06:57:29,944][20381] Updated weights for policy 1, policy_version 188219 (0.0008) [2023-12-27 06:57:30,005][20381] Updated weights for policy 1, policy_version 188229 (0.0008) [2023-12-27 06:57:30,066][20381] Updated weights for policy 1, policy_version 188239 (0.0008) [2023-12-27 06:57:30,412][20347] Updated weights for policy 0, policy_version 188496 (0.0008) [2023-12-27 06:57:30,472][20347] Updated weights for policy 0, policy_version 188506 (0.0008) [2023-12-27 06:57:30,532][20347] Updated weights for policy 0, policy_version 188516 (0.0008) [2023-12-27 06:57:30,662][20381] Updated weights for policy 1, policy_version 188249 (0.0008) [2023-12-27 06:57:30,716][20381] Updated weights for policy 1, policy_version 188259 (0.0006) [2023-12-27 06:57:30,773][20381] Updated weights for policy 1, policy_version 188269 (0.0008) [2023-12-27 06:57:30,829][20381] Updated weights for policy 1, policy_version 188279 (0.0007) [2023-12-27 06:57:31,255][20347] Updated weights for policy 0, policy_version 188526 (0.0008) [2023-12-27 06:57:31,308][20347] Updated weights for policy 0, policy_version 188536 (0.0009) [2023-12-27 06:57:31,360][20347] Updated weights for policy 0, policy_version 188546 (0.0008) [2023-12-27 06:57:31,455][20381] Updated weights for policy 1, policy_version 188289 (0.0006) [2023-12-27 06:57:31,517][20381] Updated weights for policy 1, policy_version 188299 (0.0005) [2023-12-27 06:57:31,573][20381] Updated weights for policy 1, policy_version 188309 (0.0006) [2023-12-27 06:57:32,082][20347] Updated weights for policy 0, policy_version 188556 (0.0008) [2023-12-27 06:57:32,128][20347] Updated weights for policy 0, policy_version 188566 (0.0008) [2023-12-27 06:57:32,173][20347] Updated weights for policy 0, policy_version 188576 (0.0008) [2023-12-27 06:57:32,229][20381] Updated weights for policy 1, policy_version 188319 (0.0008) [2023-12-27 06:57:32,290][20381] Updated weights for policy 1, policy_version 188329 (0.0008) [2023-12-27 06:57:32,353][20381] Updated weights for policy 1, policy_version 188339 (0.0007) [2023-12-27 06:57:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 96509952. Throughput: 0: 10362.6, 1: 10423.2. Samples: 96522404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:32,894][20347] Updated weights for policy 0, policy_version 188586 (0.0008) [2023-12-27 06:57:32,946][20347] Updated weights for policy 0, policy_version 188596 (0.0008) [2023-12-27 06:57:32,999][20347] Updated weights for policy 0, policy_version 188606 (0.0006) [2023-12-27 06:57:33,001][20381] Updated weights for policy 1, policy_version 188349 (0.0006) [2023-12-27 06:57:33,044][20347] Updated weights for policy 0, policy_version 188616 (0.0005) [2023-12-27 06:57:33,049][20381] Updated weights for policy 1, policy_version 188359 (0.0006) [2023-12-27 06:57:33,096][20381] Updated weights for policy 1, policy_version 188369 (0.0007) [2023-12-27 06:57:33,713][20347] Updated weights for policy 0, policy_version 188626 (0.0007) [2023-12-27 06:57:33,764][20347] Updated weights for policy 0, policy_version 188636 (0.0007) [2023-12-27 06:57:33,800][20381] Updated weights for policy 1, policy_version 188379 (0.0006) [2023-12-27 06:57:33,817][20347] Updated weights for policy 0, policy_version 188646 (0.0007) [2023-12-27 06:57:33,857][20381] Updated weights for policy 1, policy_version 188389 (0.0005) [2023-12-27 06:57:33,909][20381] Updated weights for policy 1, policy_version 188399 (0.0007) [2023-12-27 06:57:34,523][20347] Updated weights for policy 0, policy_version 188656 (0.0008) [2023-12-27 06:57:34,561][20381] Updated weights for policy 1, policy_version 188409 (0.0007) [2023-12-27 06:57:34,578][20347] Updated weights for policy 0, policy_version 188666 (0.0007) [2023-12-27 06:57:34,618][20381] Updated weights for policy 1, policy_version 188419 (0.0007) [2023-12-27 06:57:34,635][20347] Updated weights for policy 0, policy_version 188676 (0.0006) [2023-12-27 06:57:34,675][20381] Updated weights for policy 1, policy_version 188429 (0.0006) [2023-12-27 06:57:34,734][20381] Updated weights for policy 1, policy_version 188439 (0.0007) [2023-12-27 06:57:35,342][20347] Updated weights for policy 0, policy_version 188686 (0.0007) [2023-12-27 06:57:35,394][20347] Updated weights for policy 0, policy_version 188696 (0.0008) [2023-12-27 06:57:35,404][20381] Updated weights for policy 1, policy_version 188449 (0.0006) [2023-12-27 06:57:35,447][20347] Updated weights for policy 0, policy_version 188706 (0.0006) [2023-12-27 06:57:35,456][20381] Updated weights for policy 1, policy_version 188459 (0.0005) [2023-12-27 06:57:35,502][20381] Updated weights for policy 1, policy_version 188469 (0.0006) [2023-12-27 06:57:36,150][20347] Updated weights for policy 0, policy_version 188716 (0.0006) [2023-12-27 06:57:36,201][20381] Updated weights for policy 1, policy_version 188479 (0.0006) [2023-12-27 06:57:36,214][20347] Updated weights for policy 0, policy_version 188726 (0.0007) [2023-12-27 06:57:36,260][20381] Updated weights for policy 1, policy_version 188489 (0.0006) [2023-12-27 06:57:36,278][20347] Updated weights for policy 0, policy_version 188736 (0.0005) [2023-12-27 06:57:36,314][20381] Updated weights for policy 1, policy_version 188499 (0.0008) [2023-12-27 06:57:36,856][20347] Updated weights for policy 0, policy_version 188746 (0.0006) [2023-12-27 06:57:36,919][20347] Updated weights for policy 0, policy_version 188756 (0.0008) [2023-12-27 06:57:36,983][20347] Updated weights for policy 0, policy_version 188766 (0.0010) [2023-12-27 06:57:37,047][20347] Updated weights for policy 0, policy_version 188776 (0.0010) [2023-12-27 06:57:37,060][20381] Updated weights for policy 1, policy_version 188509 (0.0007) [2023-12-27 06:57:37,116][20381] Updated weights for policy 1, policy_version 188519 (0.0007) [2023-12-27 06:57:37,174][20381] Updated weights for policy 1, policy_version 188529 (0.0007) [2023-12-27 06:57:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 96616448. Throughput: 0: 10306.7, 1: 10411.7. Samples: 96583496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000188536_48275456.pth... [2023-12-27 06:57:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000188776_48340992.pth... [2023-12-27 06:57:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000187592_48037888.pth [2023-12-27 06:57:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000187320_47964160.pth [2023-12-27 06:57:37,732][20347] Updated weights for policy 0, policy_version 188786 (0.0010) [2023-12-27 06:57:37,788][20347] Updated weights for policy 0, policy_version 188796 (0.0010) [2023-12-27 06:57:37,842][20347] Updated weights for policy 0, policy_version 188806 (0.0010) [2023-12-27 06:57:37,851][20381] Updated weights for policy 1, policy_version 188539 (0.0007) [2023-12-27 06:57:37,908][20381] Updated weights for policy 1, policy_version 188549 (0.0007) [2023-12-27 06:57:37,968][20381] Updated weights for policy 1, policy_version 188559 (0.0008) [2023-12-27 06:57:38,502][20347] Updated weights for policy 0, policy_version 188816 (0.0010) [2023-12-27 06:57:38,567][20347] Updated weights for policy 0, policy_version 188826 (0.0010) [2023-12-27 06:57:38,627][20347] Updated weights for policy 0, policy_version 188836 (0.0008) [2023-12-27 06:57:38,654][20381] Updated weights for policy 1, policy_version 188569 (0.0007) [2023-12-27 06:57:38,712][20381] Updated weights for policy 1, policy_version 188579 (0.0007) [2023-12-27 06:57:38,770][20381] Updated weights for policy 1, policy_version 188589 (0.0008) [2023-12-27 06:57:38,833][20381] Updated weights for policy 1, policy_version 188599 (0.0008) [2023-12-27 06:57:39,270][20347] Updated weights for policy 0, policy_version 188846 (0.0006) [2023-12-27 06:57:39,331][20347] Updated weights for policy 0, policy_version 188856 (0.0007) [2023-12-27 06:57:39,393][20347] Updated weights for policy 0, policy_version 188866 (0.0008) [2023-12-27 06:57:39,553][20381] Updated weights for policy 1, policy_version 188609 (0.0008) [2023-12-27 06:57:39,616][20381] Updated weights for policy 1, policy_version 188619 (0.0007) [2023-12-27 06:57:39,669][20381] Updated weights for policy 1, policy_version 188629 (0.0007) [2023-12-27 06:57:40,046][20347] Updated weights for policy 0, policy_version 188876 (0.0008) [2023-12-27 06:57:40,118][20347] Updated weights for policy 0, policy_version 188886 (0.0008) [2023-12-27 06:57:40,180][20347] Updated weights for policy 0, policy_version 188896 (0.0007) [2023-12-27 06:57:40,414][20381] Updated weights for policy 1, policy_version 188639 (0.0008) [2023-12-27 06:57:40,467][20381] Updated weights for policy 1, policy_version 188649 (0.0008) [2023-12-27 06:57:40,523][20381] Updated weights for policy 1, policy_version 188659 (0.0007) [2023-12-27 06:57:40,849][20347] Updated weights for policy 0, policy_version 188906 (0.0007) [2023-12-27 06:57:40,898][20347] Updated weights for policy 0, policy_version 188916 (0.0007) [2023-12-27 06:57:40,954][20347] Updated weights for policy 0, policy_version 188926 (0.0007) [2023-12-27 06:57:41,022][20347] Updated weights for policy 0, policy_version 188936 (0.0008) [2023-12-27 06:57:41,313][20381] Updated weights for policy 1, policy_version 188669 (0.0008) [2023-12-27 06:57:41,377][20381] Updated weights for policy 1, policy_version 188679 (0.0007) [2023-12-27 06:57:41,444][20381] Updated weights for policy 1, policy_version 188689 (0.0009) [2023-12-27 06:57:41,753][20347] Updated weights for policy 0, policy_version 188946 (0.0008) [2023-12-27 06:57:41,809][20347] Updated weights for policy 0, policy_version 188956 (0.0008) [2023-12-27 06:57:41,866][20347] Updated weights for policy 0, policy_version 188966 (0.0008) [2023-12-27 06:57:42,140][20381] Updated weights for policy 1, policy_version 188699 (0.0006) [2023-12-27 06:57:42,190][20381] Updated weights for policy 1, policy_version 188709 (0.0009) [2023-12-27 06:57:42,242][20381] Updated weights for policy 1, policy_version 188719 (0.0007) [2023-12-27 06:57:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 96714752. Throughput: 0: 10251.1, 1: 10362.4. Samples: 96704452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:42,553][20347] Updated weights for policy 0, policy_version 188976 (0.0008) [2023-12-27 06:57:42,610][20347] Updated weights for policy 0, policy_version 188986 (0.0009) [2023-12-27 06:57:42,672][20347] Updated weights for policy 0, policy_version 188996 (0.0006) [2023-12-27 06:57:42,900][20381] Updated weights for policy 1, policy_version 188729 (0.0008) [2023-12-27 06:57:42,951][20381] Updated weights for policy 1, policy_version 188739 (0.0007) [2023-12-27 06:57:43,003][20381] Updated weights for policy 1, policy_version 188749 (0.0008) [2023-12-27 06:57:43,059][20381] Updated weights for policy 1, policy_version 188759 (0.0008) [2023-12-27 06:57:43,289][20347] Updated weights for policy 0, policy_version 189006 (0.0006) [2023-12-27 06:57:43,352][20347] Updated weights for policy 0, policy_version 189016 (0.0007) [2023-12-27 06:57:43,397][20347] Updated weights for policy 0, policy_version 189026 (0.0007) [2023-12-27 06:57:43,722][20381] Updated weights for policy 1, policy_version 188769 (0.0007) [2023-12-27 06:57:43,780][20381] Updated weights for policy 1, policy_version 188779 (0.0008) [2023-12-27 06:57:43,836][20381] Updated weights for policy 1, policy_version 188789 (0.0008) [2023-12-27 06:57:44,076][20347] Updated weights for policy 0, policy_version 189036 (0.0007) [2023-12-27 06:57:44,136][20347] Updated weights for policy 0, policy_version 189046 (0.0008) [2023-12-27 06:57:44,197][20347] Updated weights for policy 0, policy_version 189056 (0.0006) [2023-12-27 06:57:44,501][20381] Updated weights for policy 1, policy_version 188799 (0.0008) [2023-12-27 06:57:44,561][20381] Updated weights for policy 1, policy_version 188809 (0.0008) [2023-12-27 06:57:44,619][20381] Updated weights for policy 1, policy_version 188819 (0.0008) [2023-12-27 06:57:44,778][20347] Updated weights for policy 0, policy_version 189066 (0.0007) [2023-12-27 06:57:44,838][20347] Updated weights for policy 0, policy_version 189076 (0.0008) [2023-12-27 06:57:44,893][20347] Updated weights for policy 0, policy_version 189086 (0.0007) [2023-12-27 06:57:44,944][20347] Updated weights for policy 0, policy_version 189096 (0.0008) [2023-12-27 06:57:45,337][20381] Updated weights for policy 1, policy_version 188829 (0.0008) [2023-12-27 06:57:45,390][20381] Updated weights for policy 1, policy_version 188839 (0.0005) [2023-12-27 06:57:45,439][20381] Updated weights for policy 1, policy_version 188849 (0.0005) [2023-12-27 06:57:45,663][20347] Updated weights for policy 0, policy_version 189106 (0.0006) [2023-12-27 06:57:45,711][20347] Updated weights for policy 0, policy_version 189116 (0.0005) [2023-12-27 06:57:45,766][20347] Updated weights for policy 0, policy_version 189126 (0.0005) [2023-12-27 06:57:45,980][20381] Updated weights for policy 1, policy_version 188859 (0.0006) [2023-12-27 06:57:46,036][20381] Updated weights for policy 1, policy_version 188869 (0.0006) [2023-12-27 06:57:46,090][20381] Updated weights for policy 1, policy_version 188879 (0.0005) [2023-12-27 06:57:46,383][20347] Updated weights for policy 0, policy_version 189136 (0.0007) [2023-12-27 06:57:46,439][20347] Updated weights for policy 0, policy_version 189146 (0.0007) [2023-12-27 06:57:46,498][20347] Updated weights for policy 0, policy_version 189156 (0.0008) [2023-12-27 06:57:46,619][20381] Updated weights for policy 1, policy_version 188889 (0.0006) [2023-12-27 06:57:46,676][20381] Updated weights for policy 1, policy_version 188899 (0.0006) [2023-12-27 06:57:46,732][20381] Updated weights for policy 1, policy_version 188909 (0.0006) [2023-12-27 06:57:46,793][20381] Updated weights for policy 1, policy_version 188919 (0.0005) [2023-12-27 06:57:47,169][20347] Updated weights for policy 0, policy_version 189166 (0.0008) [2023-12-27 06:57:47,222][20347] Updated weights for policy 0, policy_version 189176 (0.0008) [2023-12-27 06:57:47,278][20347] Updated weights for policy 0, policy_version 189186 (0.0007) [2023-12-27 06:57:47,318][20381] Updated weights for policy 1, policy_version 188929 (0.0005) [2023-12-27 06:57:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 96821248. Throughput: 0: 10243.0, 1: 10467.2. Samples: 96832484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:47,368][20381] Updated weights for policy 1, policy_version 188939 (0.0005) [2023-12-27 06:57:47,425][20381] Updated weights for policy 1, policy_version 188949 (0.0007) [2023-12-27 06:57:47,962][20347] Updated weights for policy 0, policy_version 189196 (0.0007) [2023-12-27 06:57:48,016][20347] Updated weights for policy 0, policy_version 189206 (0.0005) [2023-12-27 06:57:48,049][20381] Updated weights for policy 1, policy_version 188959 (0.0008) [2023-12-27 06:57:48,069][20347] Updated weights for policy 0, policy_version 189216 (0.0005) [2023-12-27 06:57:48,121][20381] Updated weights for policy 1, policy_version 188969 (0.0008) [2023-12-27 06:57:48,182][20381] Updated weights for policy 1, policy_version 188979 (0.0006) [2023-12-27 06:57:48,698][20347] Updated weights for policy 0, policy_version 189226 (0.0006) [2023-12-27 06:57:48,755][20347] Updated weights for policy 0, policy_version 189236 (0.0007) [2023-12-27 06:57:48,759][20381] Updated weights for policy 1, policy_version 188989 (0.0005) [2023-12-27 06:57:48,818][20347] Updated weights for policy 0, policy_version 189246 (0.0007) [2023-12-27 06:57:48,821][20381] Updated weights for policy 1, policy_version 188999 (0.0005) [2023-12-27 06:57:48,877][20347] Updated weights for policy 0, policy_version 189256 (0.0007) [2023-12-27 06:57:48,878][20381] Updated weights for policy 1, policy_version 189009 (0.0005) [2023-12-27 06:57:49,507][20347] Updated weights for policy 0, policy_version 189266 (0.0005) [2023-12-27 06:57:49,563][20347] Updated weights for policy 0, policy_version 189276 (0.0008) [2023-12-27 06:57:49,585][20381] Updated weights for policy 1, policy_version 189019 (0.0007) [2023-12-27 06:57:49,621][20347] Updated weights for policy 0, policy_version 189286 (0.0006) [2023-12-27 06:57:49,639][20381] Updated weights for policy 1, policy_version 189029 (0.0006) [2023-12-27 06:57:49,690][20381] Updated weights for policy 1, policy_version 189039 (0.0007) [2023-12-27 06:57:50,281][20347] Updated weights for policy 0, policy_version 189296 (0.0007) [2023-12-27 06:57:50,337][20347] Updated weights for policy 0, policy_version 189306 (0.0007) [2023-12-27 06:57:50,396][20347] Updated weights for policy 0, policy_version 189316 (0.0007) [2023-12-27 06:57:50,406][20381] Updated weights for policy 1, policy_version 189049 (0.0008) [2023-12-27 06:57:50,453][20381] Updated weights for policy 1, policy_version 189059 (0.0007) [2023-12-27 06:57:50,514][20381] Updated weights for policy 1, policy_version 189069 (0.0007) [2023-12-27 06:57:50,581][20381] Updated weights for policy 1, policy_version 189079 (0.0006) [2023-12-27 06:57:50,973][20347] Updated weights for policy 0, policy_version 189326 (0.0008) [2023-12-27 06:57:51,030][20347] Updated weights for policy 0, policy_version 189336 (0.0008) [2023-12-27 06:57:51,089][20347] Updated weights for policy 0, policy_version 189346 (0.0006) [2023-12-27 06:57:51,301][20381] Updated weights for policy 1, policy_version 189089 (0.0008) [2023-12-27 06:57:51,371][20381] Updated weights for policy 1, policy_version 189099 (0.0010) [2023-12-27 06:57:51,429][20381] Updated weights for policy 1, policy_version 189109 (0.0007) [2023-12-27 06:57:51,688][20347] Updated weights for policy 0, policy_version 189356 (0.0007) [2023-12-27 06:57:51,743][20347] Updated weights for policy 0, policy_version 189366 (0.0007) [2023-12-27 06:57:51,798][20347] Updated weights for policy 0, policy_version 189376 (0.0007) [2023-12-27 06:57:52,130][20381] Updated weights for policy 1, policy_version 189119 (0.0006) [2023-12-27 06:57:52,184][20381] Updated weights for policy 1, policy_version 189129 (0.0005) [2023-12-27 06:57:52,242][20381] Updated weights for policy 1, policy_version 189139 (0.0005) [2023-12-27 06:57:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 96927744. Throughput: 0: 10250.6, 1: 10436.3. Samples: 96895444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000189144_48431104.pth... [2023-12-27 06:57:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000189384_48496640.pth... [2023-12-27 06:57:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000188168_48185344.pth [2023-12-27 06:57:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000187928_48119808.pth [2023-12-27 06:57:52,451][20347] Updated weights for policy 0, policy_version 189386 (0.0008) [2023-12-27 06:57:52,506][20347] Updated weights for policy 0, policy_version 189396 (0.0006) [2023-12-27 06:57:52,567][20347] Updated weights for policy 0, policy_version 189406 (0.0008) [2023-12-27 06:57:52,615][20347] Updated weights for policy 0, policy_version 189416 (0.0008) [2023-12-27 06:57:52,838][20381] Updated weights for policy 1, policy_version 189149 (0.0008) [2023-12-27 06:57:52,893][20381] Updated weights for policy 1, policy_version 189159 (0.0008) [2023-12-27 06:57:52,958][20381] Updated weights for policy 1, policy_version 189169 (0.0007) [2023-12-27 06:57:53,329][20347] Updated weights for policy 0, policy_version 189426 (0.0007) [2023-12-27 06:57:53,385][20347] Updated weights for policy 0, policy_version 189436 (0.0008) [2023-12-27 06:57:53,442][20347] Updated weights for policy 0, policy_version 189446 (0.0005) [2023-12-27 06:57:53,670][20381] Updated weights for policy 1, policy_version 189179 (0.0008) [2023-12-27 06:57:53,719][20381] Updated weights for policy 1, policy_version 189189 (0.0007) [2023-12-27 06:57:53,768][20381] Updated weights for policy 1, policy_version 189199 (0.0007) [2023-12-27 06:57:54,086][20347] Updated weights for policy 0, policy_version 189456 (0.0009) [2023-12-27 06:57:54,136][20347] Updated weights for policy 0, policy_version 189466 (0.0009) [2023-12-27 06:57:54,196][20347] Updated weights for policy 0, policy_version 189476 (0.0008) [2023-12-27 06:57:54,343][20381] Updated weights for policy 1, policy_version 189209 (0.0008) [2023-12-27 06:57:54,404][20381] Updated weights for policy 1, policy_version 189219 (0.0005) [2023-12-27 06:57:54,455][20381] Updated weights for policy 1, policy_version 189229 (0.0005) [2023-12-27 06:57:54,512][20381] Updated weights for policy 1, policy_version 189239 (0.0005) [2023-12-27 06:57:54,953][20347] Updated weights for policy 0, policy_version 189486 (0.0009) [2023-12-27 06:57:55,018][20347] Updated weights for policy 0, policy_version 189496 (0.0008) [2023-12-27 06:57:55,076][20347] Updated weights for policy 0, policy_version 189506 (0.0008) [2023-12-27 06:57:55,080][20381] Updated weights for policy 1, policy_version 189249 (0.0005) [2023-12-27 06:57:55,132][20381] Updated weights for policy 1, policy_version 189259 (0.0008) [2023-12-27 06:57:55,195][20381] Updated weights for policy 1, policy_version 189269 (0.0010) [2023-12-27 06:57:55,667][20347] Updated weights for policy 0, policy_version 189516 (0.0010) [2023-12-27 06:57:55,721][20347] Updated weights for policy 0, policy_version 189526 (0.0010) [2023-12-27 06:57:55,775][20347] Updated weights for policy 0, policy_version 189536 (0.0005) [2023-12-27 06:57:55,852][20381] Updated weights for policy 1, policy_version 189279 (0.0011) [2023-12-27 06:57:55,904][20381] Updated weights for policy 1, policy_version 189289 (0.0010) [2023-12-27 06:57:55,951][20381] Updated weights for policy 1, policy_version 189299 (0.0007) [2023-12-27 06:57:56,389][20347] Updated weights for policy 0, policy_version 189546 (0.0006) [2023-12-27 06:57:56,451][20347] Updated weights for policy 0, policy_version 189556 (0.0010) [2023-12-27 06:57:56,513][20347] Updated weights for policy 0, policy_version 189566 (0.0010) [2023-12-27 06:57:56,545][20381] Updated weights for policy 1, policy_version 189309 (0.0008) [2023-12-27 06:57:56,563][20347] Updated weights for policy 0, policy_version 189576 (0.0010) [2023-12-27 06:57:56,590][20381] Updated weights for policy 1, policy_version 189319 (0.0010) [2023-12-27 06:57:56,643][20381] Updated weights for policy 1, policy_version 189329 (0.0010) [2023-12-27 06:57:57,232][20347] Updated weights for policy 0, policy_version 189586 (0.0011) [2023-12-27 06:57:57,273][20381] Updated weights for policy 1, policy_version 189339 (0.0009) [2023-12-27 06:57:57,289][20347] Updated weights for policy 0, policy_version 189596 (0.0010) [2023-12-27 06:57:57,328][20381] Updated weights for policy 1, policy_version 189349 (0.0010) [2023-12-27 06:57:57,342][20347] Updated weights for policy 0, policy_version 189606 (0.0010) [2023-12-27 06:57:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 97034240. Throughput: 0: 10274.1, 1: 10492.3. Samples: 97023400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:57:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 06:57:57,383][20381] Updated weights for policy 1, policy_version 189359 (0.0010) [2023-12-27 06:57:57,972][20381] Updated weights for policy 1, policy_version 189369 (0.0006) [2023-12-27 06:57:57,990][20347] Updated weights for policy 0, policy_version 189616 (0.0010) [2023-12-27 06:57:58,022][20381] Updated weights for policy 1, policy_version 189379 (0.0005) [2023-12-27 06:57:58,048][20347] Updated weights for policy 0, policy_version 189626 (0.0010) [2023-12-27 06:57:58,075][20381] Updated weights for policy 1, policy_version 189389 (0.0007) [2023-12-27 06:57:58,093][20347] Updated weights for policy 0, policy_version 189636 (0.0010) [2023-12-27 06:57:58,120][20381] Updated weights for policy 1, policy_version 189399 (0.0005) [2023-12-27 06:57:58,856][20347] Updated weights for policy 0, policy_version 189646 (0.0009) [2023-12-27 06:57:58,864][20381] Updated weights for policy 1, policy_version 189409 (0.0007) [2023-12-27 06:57:58,912][20347] Updated weights for policy 0, policy_version 189656 (0.0008) [2023-12-27 06:57:58,917][20381] Updated weights for policy 1, policy_version 189419 (0.0007) [2023-12-27 06:57:58,966][20347] Updated weights for policy 0, policy_version 189666 (0.0007) [2023-12-27 06:57:58,968][20381] Updated weights for policy 1, policy_version 189429 (0.0005) [2023-12-27 06:57:59,601][20347] Updated weights for policy 0, policy_version 189676 (0.0009) [2023-12-27 06:57:59,651][20347] Updated weights for policy 0, policy_version 189686 (0.0011) [2023-12-27 06:57:59,656][20381] Updated weights for policy 1, policy_version 189439 (0.0007) [2023-12-27 06:57:59,703][20347] Updated weights for policy 0, policy_version 189696 (0.0007) [2023-12-27 06:57:59,712][20381] Updated weights for policy 1, policy_version 189449 (0.0007) [2023-12-27 06:57:59,767][20381] Updated weights for policy 1, policy_version 189459 (0.0007) [2023-12-27 06:58:00,367][20347] Updated weights for policy 0, policy_version 189706 (0.0008) [2023-12-27 06:58:00,423][20347] Updated weights for policy 0, policy_version 189716 (0.0008) [2023-12-27 06:58:00,468][20347] Updated weights for policy 0, policy_version 189726 (0.0007) [2023-12-27 06:58:00,518][20347] Updated weights for policy 0, policy_version 189736 (0.0006) [2023-12-27 06:58:00,545][20381] Updated weights for policy 1, policy_version 189469 (0.0008) [2023-12-27 06:58:00,594][20381] Updated weights for policy 1, policy_version 189479 (0.0007) [2023-12-27 06:58:00,639][20381] Updated weights for policy 1, policy_version 189489 (0.0007) [2023-12-27 06:58:01,161][20347] Updated weights for policy 0, policy_version 189746 (0.0011) [2023-12-27 06:58:01,224][20347] Updated weights for policy 0, policy_version 189756 (0.0008) [2023-12-27 06:58:01,286][20347] Updated weights for policy 0, policy_version 189766 (0.0006) [2023-12-27 06:58:01,316][20381] Updated weights for policy 1, policy_version 189499 (0.0008) [2023-12-27 06:58:01,378][20381] Updated weights for policy 1, policy_version 189509 (0.0009) [2023-12-27 06:58:01,436][20381] Updated weights for policy 1, policy_version 189519 (0.0006) [2023-12-27 06:58:01,897][20347] Updated weights for policy 0, policy_version 189776 (0.0007) [2023-12-27 06:58:01,928][20291] KL-divergence is very high: 123.1561 [2023-12-27 06:58:01,949][20347] Updated weights for policy 0, policy_version 189786 (0.0010) [2023-12-27 06:58:02,006][20347] Updated weights for policy 0, policy_version 189796 (0.0010) [2023-12-27 06:58:02,119][20381] Updated weights for policy 1, policy_version 189529 (0.0006) [2023-12-27 06:58:02,181][20381] Updated weights for policy 1, policy_version 189539 (0.0011) [2023-12-27 06:58:02,237][20381] Updated weights for policy 1, policy_version 189549 (0.0011) [2023-12-27 06:58:02,297][20381] Updated weights for policy 1, policy_version 189559 (0.0011) [2023-12-27 06:58:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 97140736. Throughput: 0: 10348.4, 1: 10467.5. Samples: 97147448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 06:58:02,363][19170] Avg episode reward: [(0, '999.870'), (1, '1000.000')] [2023-12-27 06:58:02,643][20347] Updated weights for policy 0, policy_version 189806 (0.0010) [2023-12-27 06:58:02,698][20347] Updated weights for policy 0, policy_version 189816 (0.0010) [2023-12-27 06:58:02,756][20347] Updated weights for policy 0, policy_version 189826 (0.0010) [2023-12-27 06:58:02,981][20381] Updated weights for policy 1, policy_version 189569 (0.0007) [2023-12-27 06:58:03,031][20381] Updated weights for policy 1, policy_version 189579 (0.0008) [2023-12-27 06:58:03,082][20381] Updated weights for policy 1, policy_version 189589 (0.0007) [2023-12-27 06:58:03,422][20347] Updated weights for policy 0, policy_version 189836 (0.0010) [2023-12-27 06:58:03,479][20347] Updated weights for policy 0, policy_version 189846 (0.0010) [2023-12-27 06:58:03,539][20347] Updated weights for policy 0, policy_version 189856 (0.0010) [2023-12-27 06:58:03,737][20381] Updated weights for policy 1, policy_version 189599 (0.0008) [2023-12-27 06:58:03,780][20381] Updated weights for policy 1, policy_version 189609 (0.0008) [2023-12-27 06:58:03,824][20381] Updated weights for policy 1, policy_version 189619 (0.0006) [2023-12-27 06:58:04,212][20347] Updated weights for policy 0, policy_version 189866 (0.0010) [2023-12-27 06:58:04,274][20347] Updated weights for policy 0, policy_version 189876 (0.0010) [2023-12-27 06:58:04,330][20347] Updated weights for policy 0, policy_version 189886 (0.0010) [2023-12-27 06:58:04,385][20347] Updated weights for policy 0, policy_version 189896 (0.0010) [2023-12-27 06:58:04,545][20381] Updated weights for policy 1, policy_version 189629 (0.0009) [2023-12-27 06:58:04,602][20381] Updated weights for policy 1, policy_version 189639 (0.0010) [2023-12-27 06:58:04,663][20381] Updated weights for policy 1, policy_version 189649 (0.0010) [2023-12-27 06:58:05,034][20347] Updated weights for policy 0, policy_version 189906 (0.0010) [2023-12-27 06:58:05,088][20347] Updated weights for policy 0, policy_version 189916 (0.0010) [2023-12-27 06:58:05,156][20347] Updated weights for policy 0, policy_version 189926 (0.0010) [2023-12-27 06:58:05,319][20381] Updated weights for policy 1, policy_version 189659 (0.0009) [2023-12-27 06:58:05,371][20381] Updated weights for policy 1, policy_version 189669 (0.0010) [2023-12-27 06:58:05,428][20381] Updated weights for policy 1, policy_version 189679 (0.0010) [2023-12-27 06:58:05,748][20347] Updated weights for policy 0, policy_version 189936 (0.0007) [2023-12-27 06:58:05,803][20347] Updated weights for policy 0, policy_version 189946 (0.0007) [2023-12-27 06:58:05,859][20347] Updated weights for policy 0, policy_version 189956 (0.0007) [2023-12-27 06:58:06,103][20381] Updated weights for policy 1, policy_version 189689 (0.0011) [2023-12-27 06:58:06,164][20381] Updated weights for policy 1, policy_version 189699 (0.0010) [2023-12-27 06:58:06,228][20381] Updated weights for policy 1, policy_version 189709 (0.0008) [2023-12-27 06:58:06,291][20381] Updated weights for policy 1, policy_version 189719 (0.0007) [2023-12-27 06:58:06,522][20347] Updated weights for policy 0, policy_version 189966 (0.0008) [2023-12-27 06:58:06,577][20347] Updated weights for policy 0, policy_version 189976 (0.0007) [2023-12-27 06:58:06,639][20347] Updated weights for policy 0, policy_version 189986 (0.0006) [2023-12-27 06:58:06,948][20381] Updated weights for policy 1, policy_version 189729 (0.0008) [2023-12-27 06:58:06,997][20381] Updated weights for policy 1, policy_version 189739 (0.0007) [2023-12-27 06:58:07,050][20381] Updated weights for policy 1, policy_version 189749 (0.0007) [2023-12-27 06:58:07,269][20347] Updated weights for policy 0, policy_version 189996 (0.0006) [2023-12-27 06:58:07,322][20347] Updated weights for policy 0, policy_version 190006 (0.0007) [2023-12-27 06:58:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 97239040. Throughput: 0: 10383.9, 1: 10462.1. Samples: 97209644. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:07,363][19170] Avg episode reward: [(0, '999.870'), (1, '1000.000')] [2023-12-27 06:58:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000189752_48586752.pth... [2023-12-27 06:58:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000188536_48275456.pth [2023-12-27 06:58:07,383][20347] Updated weights for policy 0, policy_version 190016 (0.0008) [2023-12-27 06:58:07,428][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000190024_48660480.pth... [2023-12-27 06:58:07,431][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000188776_48340992.pth [2023-12-27 06:58:07,754][20381] Updated weights for policy 1, policy_version 189759 (0.0008) [2023-12-27 06:58:07,799][20381] Updated weights for policy 1, policy_version 189769 (0.0007) [2023-12-27 06:58:07,843][20381] Updated weights for policy 1, policy_version 189779 (0.0007) [2023-12-27 06:58:08,098][20347] Updated weights for policy 0, policy_version 190026 (0.0007) [2023-12-27 06:58:08,150][20347] Updated weights for policy 0, policy_version 190036 (0.0007) [2023-12-27 06:58:08,208][20347] Updated weights for policy 0, policy_version 190046 (0.0006) [2023-12-27 06:58:08,258][20347] Updated weights for policy 0, policy_version 190056 (0.0006) [2023-12-27 06:58:08,563][20381] Updated weights for policy 1, policy_version 189789 (0.0008) [2023-12-27 06:58:08,621][20381] Updated weights for policy 1, policy_version 189799 (0.0008) [2023-12-27 06:58:08,682][20381] Updated weights for policy 1, policy_version 189809 (0.0008) [2023-12-27 06:58:08,867][20347] Updated weights for policy 0, policy_version 190066 (0.0008) [2023-12-27 06:58:08,926][20347] Updated weights for policy 0, policy_version 190076 (0.0007) [2023-12-27 06:58:08,988][20347] Updated weights for policy 0, policy_version 190086 (0.0007) [2023-12-27 06:58:09,338][20381] Updated weights for policy 1, policy_version 189819 (0.0007) [2023-12-27 06:58:09,405][20381] Updated weights for policy 1, policy_version 189829 (0.0009) [2023-12-27 06:58:09,466][20381] Updated weights for policy 1, policy_version 189839 (0.0009) [2023-12-27 06:58:09,714][20347] Updated weights for policy 0, policy_version 190096 (0.0007) [2023-12-27 06:58:09,766][20347] Updated weights for policy 0, policy_version 190106 (0.0007) [2023-12-27 06:58:09,818][20347] Updated weights for policy 0, policy_version 190116 (0.0007) [2023-12-27 06:58:10,147][20381] Updated weights for policy 1, policy_version 189849 (0.0009) [2023-12-27 06:58:10,207][20381] Updated weights for policy 1, policy_version 189859 (0.0008) [2023-12-27 06:58:10,258][20381] Updated weights for policy 1, policy_version 189869 (0.0008) [2023-12-27 06:58:10,304][20381] Updated weights for policy 1, policy_version 189879 (0.0008) [2023-12-27 06:58:10,528][20347] Updated weights for policy 0, policy_version 190126 (0.0007) [2023-12-27 06:58:10,593][20347] Updated weights for policy 0, policy_version 190136 (0.0008) [2023-12-27 06:58:10,659][20347] Updated weights for policy 0, policy_version 190146 (0.0007) [2023-12-27 06:58:10,990][20381] Updated weights for policy 1, policy_version 189889 (0.0006) [2023-12-27 06:58:11,048][20381] Updated weights for policy 1, policy_version 189899 (0.0007) [2023-12-27 06:58:11,112][20381] Updated weights for policy 1, policy_version 189909 (0.0006) [2023-12-27 06:58:11,321][20347] Updated weights for policy 0, policy_version 190156 (0.0008) [2023-12-27 06:58:11,393][20347] Updated weights for policy 0, policy_version 190166 (0.0008) [2023-12-27 06:58:11,461][20347] Updated weights for policy 0, policy_version 190176 (0.0008) [2023-12-27 06:58:11,798][20381] Updated weights for policy 1, policy_version 189919 (0.0008) [2023-12-27 06:58:11,857][20381] Updated weights for policy 1, policy_version 189929 (0.0008) [2023-12-27 06:58:11,919][20381] Updated weights for policy 1, policy_version 189939 (0.0008) [2023-12-27 06:58:12,216][20347] Updated weights for policy 0, policy_version 190186 (0.0008) [2023-12-27 06:58:12,275][20347] Updated weights for policy 0, policy_version 190196 (0.0007) [2023-12-27 06:58:12,338][20347] Updated weights for policy 0, policy_version 190206 (0.0008) [2023-12-27 06:58:12,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 97337344. Throughput: 0: 10404.6, 1: 10413.1. Samples: 97332808. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:12,363][19170] Avg episode reward: [(0, '998.290'), (1, '1000.000')] [2023-12-27 06:58:12,395][20347] Updated weights for policy 0, policy_version 190216 (0.0007) [2023-12-27 06:58:12,565][20381] Updated weights for policy 1, policy_version 189949 (0.0007) [2023-12-27 06:58:12,632][20381] Updated weights for policy 1, policy_version 189959 (0.0006) [2023-12-27 06:58:12,693][20381] Updated weights for policy 1, policy_version 189969 (0.0008) [2023-12-27 06:58:13,105][20347] Updated weights for policy 0, policy_version 190226 (0.0007) [2023-12-27 06:58:13,158][20347] Updated weights for policy 0, policy_version 190236 (0.0008) [2023-12-27 06:58:13,215][20347] Updated weights for policy 0, policy_version 190246 (0.0007) [2023-12-27 06:58:13,341][20381] Updated weights for policy 1, policy_version 189979 (0.0008) [2023-12-27 06:58:13,388][20381] Updated weights for policy 1, policy_version 189989 (0.0009) [2023-12-27 06:58:13,448][20381] Updated weights for policy 1, policy_version 189999 (0.0008) [2023-12-27 06:58:13,845][20347] Updated weights for policy 0, policy_version 190256 (0.0005) [2023-12-27 06:58:13,903][20347] Updated weights for policy 0, policy_version 190266 (0.0006) [2023-12-27 06:58:13,957][20347] Updated weights for policy 0, policy_version 190276 (0.0008) [2023-12-27 06:58:14,127][20381] Updated weights for policy 1, policy_version 190009 (0.0008) [2023-12-27 06:58:14,188][20381] Updated weights for policy 1, policy_version 190019 (0.0008) [2023-12-27 06:58:14,249][20381] Updated weights for policy 1, policy_version 190029 (0.0009) [2023-12-27 06:58:14,306][20381] Updated weights for policy 1, policy_version 190039 (0.0008) [2023-12-27 06:58:14,574][20347] Updated weights for policy 0, policy_version 190286 (0.0006) [2023-12-27 06:58:14,624][20347] Updated weights for policy 0, policy_version 190296 (0.0005) [2023-12-27 06:58:14,674][20347] Updated weights for policy 0, policy_version 190306 (0.0006) [2023-12-27 06:58:15,011][20381] Updated weights for policy 1, policy_version 190049 (0.0008) [2023-12-27 06:58:15,073][20381] Updated weights for policy 1, policy_version 190059 (0.0008) [2023-12-27 06:58:15,145][20381] Updated weights for policy 1, policy_version 190069 (0.0007) [2023-12-27 06:58:15,337][20347] Updated weights for policy 0, policy_version 190316 (0.0008) [2023-12-27 06:58:15,385][20347] Updated weights for policy 0, policy_version 190326 (0.0009) [2023-12-27 06:58:15,438][20347] Updated weights for policy 0, policy_version 190336 (0.0005) [2023-12-27 06:58:15,741][20381] Updated weights for policy 1, policy_version 190079 (0.0007) [2023-12-27 06:58:15,797][20381] Updated weights for policy 1, policy_version 190089 (0.0011) [2023-12-27 06:58:15,852][20381] Updated weights for policy 1, policy_version 190099 (0.0010) [2023-12-27 06:58:16,106][20347] Updated weights for policy 0, policy_version 190346 (0.0007) [2023-12-27 06:58:16,160][20347] Updated weights for policy 0, policy_version 190356 (0.0010) [2023-12-27 06:58:16,215][20347] Updated weights for policy 0, policy_version 190366 (0.0010) [2023-12-27 06:58:16,272][20347] Updated weights for policy 0, policy_version 190376 (0.0010) [2023-12-27 06:58:16,442][20381] Updated weights for policy 1, policy_version 190109 (0.0008) [2023-12-27 06:58:16,496][20381] Updated weights for policy 1, policy_version 190119 (0.0010) [2023-12-27 06:58:16,559][20381] Updated weights for policy 1, policy_version 190129 (0.0006) [2023-12-27 06:58:16,975][20347] Updated weights for policy 0, policy_version 190386 (0.0006) [2023-12-27 06:58:17,035][20347] Updated weights for policy 0, policy_version 190396 (0.0009) [2023-12-27 06:58:17,085][20347] Updated weights for policy 0, policy_version 190406 (0.0010) [2023-12-27 06:58:17,159][20381] Updated weights for policy 1, policy_version 190139 (0.0006) [2023-12-27 06:58:17,221][20381] Updated weights for policy 1, policy_version 190149 (0.0007) [2023-12-27 06:58:17,269][20381] Updated weights for policy 1, policy_version 190159 (0.0007) [2023-12-27 06:58:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 97452032. Throughput: 0: 10407.5, 1: 10378.4. Samples: 97457772. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:17,364][19170] Avg episode reward: [(0, '962.680'), (1, '1000.000')] [2023-12-27 06:58:17,730][20347] Updated weights for policy 0, policy_version 190416 (0.0008) [2023-12-27 06:58:17,784][20347] Updated weights for policy 0, policy_version 190426 (0.0006) [2023-12-27 06:58:17,848][20347] Updated weights for policy 0, policy_version 190436 (0.0005) [2023-12-27 06:58:17,935][20381] Updated weights for policy 1, policy_version 190169 (0.0007) [2023-12-27 06:58:17,980][20381] Updated weights for policy 1, policy_version 190179 (0.0006) [2023-12-27 06:58:18,027][20381] Updated weights for policy 1, policy_version 190189 (0.0005) [2023-12-27 06:58:18,074][20381] Updated weights for policy 1, policy_version 190199 (0.0006) [2023-12-27 06:58:18,499][20347] Updated weights for policy 0, policy_version 190446 (0.0007) [2023-12-27 06:58:18,551][20347] Updated weights for policy 0, policy_version 190456 (0.0007) [2023-12-27 06:58:18,611][20347] Updated weights for policy 0, policy_version 190466 (0.0007) [2023-12-27 06:58:18,758][20381] Updated weights for policy 1, policy_version 190209 (0.0008) [2023-12-27 06:58:18,810][20381] Updated weights for policy 1, policy_version 190219 (0.0008) [2023-12-27 06:58:18,869][20381] Updated weights for policy 1, policy_version 190229 (0.0008) [2023-12-27 06:58:19,274][20347] Updated weights for policy 0, policy_version 190476 (0.0006) [2023-12-27 06:58:19,338][20347] Updated weights for policy 0, policy_version 190486 (0.0007) [2023-12-27 06:58:19,389][20347] Updated weights for policy 0, policy_version 190496 (0.0007) [2023-12-27 06:58:19,547][20381] Updated weights for policy 1, policy_version 190239 (0.0006) [2023-12-27 06:58:19,607][20381] Updated weights for policy 1, policy_version 190249 (0.0009) [2023-12-27 06:58:19,663][20381] Updated weights for policy 1, policy_version 190259 (0.0011) [2023-12-27 06:58:20,079][20347] Updated weights for policy 0, policy_version 190506 (0.0006) [2023-12-27 06:58:20,139][20347] Updated weights for policy 0, policy_version 190516 (0.0008) [2023-12-27 06:58:20,202][20347] Updated weights for policy 0, policy_version 190526 (0.0008) [2023-12-27 06:58:20,261][20347] Updated weights for policy 0, policy_version 190536 (0.0008) [2023-12-27 06:58:20,347][20381] Updated weights for policy 1, policy_version 190269 (0.0011) [2023-12-27 06:58:20,405][20381] Updated weights for policy 1, policy_version 190279 (0.0010) [2023-12-27 06:58:20,454][20381] Updated weights for policy 1, policy_version 190289 (0.0010) [2023-12-27 06:58:20,966][20347] Updated weights for policy 0, policy_version 190546 (0.0011) [2023-12-27 06:58:21,026][20347] Updated weights for policy 0, policy_version 190556 (0.0010) [2023-12-27 06:58:21,094][20347] Updated weights for policy 0, policy_version 190566 (0.0009) [2023-12-27 06:58:21,142][20381] Updated weights for policy 1, policy_version 190299 (0.0010) [2023-12-27 06:58:21,199][20381] Updated weights for policy 1, policy_version 190309 (0.0008) [2023-12-27 06:58:21,261][20381] Updated weights for policy 1, policy_version 190319 (0.0008) [2023-12-27 06:58:21,826][20347] Updated weights for policy 0, policy_version 190576 (0.0009) [2023-12-27 06:58:21,887][20347] Updated weights for policy 0, policy_version 190586 (0.0007) [2023-12-27 06:58:21,950][20347] Updated weights for policy 0, policy_version 190596 (0.0008) [2023-12-27 06:58:22,047][20381] Updated weights for policy 1, policy_version 190329 (0.0008) [2023-12-27 06:58:22,095][20381] Updated weights for policy 1, policy_version 190339 (0.0007) [2023-12-27 06:58:22,151][20381] Updated weights for policy 1, policy_version 190349 (0.0008) [2023-12-27 06:58:22,203][20381] Updated weights for policy 1, policy_version 190359 (0.0007) [2023-12-27 06:58:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 97550336. Throughput: 0: 10434.3, 1: 10368.1. Samples: 97519604. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:22,363][19170] Avg episode reward: [(0, '961.570'), (1, '1000.000')] [2023-12-27 06:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000190360_48742400.pth... [2023-12-27 06:58:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000190600_48807936.pth... [2023-12-27 06:58:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000189384_48496640.pth [2023-12-27 06:58:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000189144_48431104.pth [2023-12-27 06:58:22,642][20347] Updated weights for policy 0, policy_version 190606 (0.0006) [2023-12-27 06:58:22,702][20347] Updated weights for policy 0, policy_version 190616 (0.0006) [2023-12-27 06:58:22,759][20347] Updated weights for policy 0, policy_version 190626 (0.0008) [2023-12-27 06:58:22,921][20381] Updated weights for policy 1, policy_version 190369 (0.0006) [2023-12-27 06:58:22,984][20381] Updated weights for policy 1, policy_version 190379 (0.0006) [2023-12-27 06:58:23,049][20381] Updated weights for policy 1, policy_version 190389 (0.0006) [2023-12-27 06:58:23,471][20347] Updated weights for policy 0, policy_version 190636 (0.0009) [2023-12-27 06:58:23,520][20347] Updated weights for policy 0, policy_version 190647 (0.0008) [2023-12-27 06:58:23,566][20347] Updated weights for policy 0, policy_version 190657 (0.0006) [2023-12-27 06:58:23,604][20381] Updated weights for policy 1, policy_version 190399 (0.0006) [2023-12-27 06:58:23,664][20381] Updated weights for policy 1, policy_version 190409 (0.0007) [2023-12-27 06:58:23,725][20381] Updated weights for policy 1, policy_version 190419 (0.0006) [2023-12-27 06:58:24,241][20347] Updated weights for policy 0, policy_version 190667 (0.0007) [2023-12-27 06:58:24,298][20347] Updated weights for policy 0, policy_version 190677 (0.0008) [2023-12-27 06:58:24,314][20381] Updated weights for policy 1, policy_version 190429 (0.0005) [2023-12-27 06:58:24,354][20347] Updated weights for policy 0, policy_version 190687 (0.0007) [2023-12-27 06:58:24,369][20381] Updated weights for policy 1, policy_version 190439 (0.0007) [2023-12-27 06:58:24,425][20381] Updated weights for policy 1, policy_version 190449 (0.0008) [2023-12-27 06:58:25,019][20347] Updated weights for policy 0, policy_version 190697 (0.0007) [2023-12-27 06:58:25,021][20381] Updated weights for policy 1, policy_version 190459 (0.0008) [2023-12-27 06:58:25,065][20347] Updated weights for policy 0, policy_version 190707 (0.0010) [2023-12-27 06:58:25,070][20381] Updated weights for policy 1, policy_version 190469 (0.0006) [2023-12-27 06:58:25,113][20347] Updated weights for policy 0, policy_version 190717 (0.0010) [2023-12-27 06:58:25,125][20381] Updated weights for policy 1, policy_version 190479 (0.0005) [2023-12-27 06:58:25,157][20347] Updated weights for policy 0, policy_version 190727 (0.0010) [2023-12-27 06:58:25,760][20381] Updated weights for policy 1, policy_version 190489 (0.0006) [2023-12-27 06:58:25,817][20381] Updated weights for policy 1, policy_version 190499 (0.0010) [2023-12-27 06:58:25,866][20347] Updated weights for policy 0, policy_version 190737 (0.0010) [2023-12-27 06:58:25,881][20381] Updated weights for policy 1, policy_version 190509 (0.0010) [2023-12-27 06:58:25,917][20347] Updated weights for policy 0, policy_version 190747 (0.0010) [2023-12-27 06:58:25,932][20381] Updated weights for policy 1, policy_version 190519 (0.0010) [2023-12-27 06:58:25,970][20347] Updated weights for policy 0, policy_version 190757 (0.0007) [2023-12-27 06:58:26,571][20381] Updated weights for policy 1, policy_version 190529 (0.0008) [2023-12-27 06:58:26,619][20381] Updated weights for policy 1, policy_version 190539 (0.0010) [2023-12-27 06:58:26,636][20347] Updated weights for policy 0, policy_version 190767 (0.0007) [2023-12-27 06:58:26,673][20381] Updated weights for policy 1, policy_version 190549 (0.0010) [2023-12-27 06:58:26,690][20347] Updated weights for policy 0, policy_version 190777 (0.0008) [2023-12-27 06:58:26,746][20347] Updated weights for policy 0, policy_version 190787 (0.0008) [2023-12-27 06:58:27,319][20381] Updated weights for policy 1, policy_version 190559 (0.0011) [2023-12-27 06:58:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 97648640. Throughput: 0: 10385.1, 1: 10467.4. Samples: 97642812. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:27,363][19170] Avg episode reward: [(0, '999.530'), (1, '986.920')] [2023-12-27 06:58:27,374][20381] Updated weights for policy 1, policy_version 190569 (0.0011) [2023-12-27 06:58:27,425][20381] Updated weights for policy 1, policy_version 190579 (0.0010) [2023-12-27 06:58:27,448][20347] Updated weights for policy 0, policy_version 190797 (0.0009) [2023-12-27 06:58:27,493][20347] Updated weights for policy 0, policy_version 190807 (0.0010) [2023-12-27 06:58:27,538][20347] Updated weights for policy 0, policy_version 190817 (0.0010) [2023-12-27 06:58:28,064][20381] Updated weights for policy 1, policy_version 190589 (0.0009) [2023-12-27 06:58:28,122][20381] Updated weights for policy 1, policy_version 190599 (0.0006) [2023-12-27 06:58:28,179][20347] Updated weights for policy 0, policy_version 190827 (0.0010) [2023-12-27 06:58:28,182][20381] Updated weights for policy 1, policy_version 190609 (0.0007) [2023-12-27 06:58:28,236][20347] Updated weights for policy 0, policy_version 190837 (0.0008) [2023-12-27 06:58:28,292][20347] Updated weights for policy 0, policy_version 190847 (0.0006) [2023-12-27 06:58:28,856][20381] Updated weights for policy 1, policy_version 190619 (0.0011) [2023-12-27 06:58:28,895][20347] Updated weights for policy 0, policy_version 190857 (0.0007) [2023-12-27 06:58:28,907][20381] Updated weights for policy 1, policy_version 190629 (0.0010) [2023-12-27 06:58:28,949][20347] Updated weights for policy 0, policy_version 190867 (0.0010) [2023-12-27 06:58:28,965][20381] Updated weights for policy 1, policy_version 190639 (0.0005) [2023-12-27 06:58:28,993][20347] Updated weights for policy 0, policy_version 190877 (0.0010) [2023-12-27 06:58:29,041][20347] Updated weights for policy 0, policy_version 190887 (0.0010) [2023-12-27 06:58:29,597][20381] Updated weights for policy 1, policy_version 190649 (0.0006) [2023-12-27 06:58:29,650][20381] Updated weights for policy 1, policy_version 190659 (0.0010) [2023-12-27 06:58:29,706][20381] Updated weights for policy 1, policy_version 190669 (0.0007) [2023-12-27 06:58:29,741][20347] Updated weights for policy 0, policy_version 190897 (0.0010) [2023-12-27 06:58:29,743][20292] KL-divergence is very high: 411.2319 [2023-12-27 06:58:29,758][20381] Updated weights for policy 1, policy_version 190679 (0.0005) [2023-12-27 06:58:29,795][20347] Updated weights for policy 0, policy_version 190907 (0.0010) [2023-12-27 06:58:29,863][20347] Updated weights for policy 0, policy_version 190917 (0.0010) [2023-12-27 06:58:30,496][20381] Updated weights for policy 1, policy_version 190689 (0.0007) [2023-12-27 06:58:30,540][20347] Updated weights for policy 0, policy_version 190927 (0.0010) [2023-12-27 06:58:30,542][20381] Updated weights for policy 1, policy_version 190699 (0.0006) [2023-12-27 06:58:30,590][20381] Updated weights for policy 1, policy_version 190709 (0.0008) [2023-12-27 06:58:30,594][20347] Updated weights for policy 0, policy_version 190937 (0.0010) [2023-12-27 06:58:30,651][20347] Updated weights for policy 0, policy_version 190947 (0.0010) [2023-12-27 06:58:31,270][20381] Updated weights for policy 1, policy_version 190719 (0.0009) [2023-12-27 06:58:31,330][20381] Updated weights for policy 1, policy_version 190729 (0.0007) [2023-12-27 06:58:31,348][20347] Updated weights for policy 0, policy_version 190957 (0.0010) [2023-12-27 06:58:31,395][20381] Updated weights for policy 1, policy_version 190739 (0.0007) [2023-12-27 06:58:31,409][20347] Updated weights for policy 0, policy_version 190967 (0.0009) [2023-12-27 06:58:31,469][20347] Updated weights for policy 0, policy_version 190977 (0.0007) [2023-12-27 06:58:32,079][20381] Updated weights for policy 1, policy_version 190749 (0.0008) [2023-12-27 06:58:32,134][20381] Updated weights for policy 1, policy_version 190759 (0.0010) [2023-12-27 06:58:32,155][20347] Updated weights for policy 0, policy_version 190987 (0.0007) [2023-12-27 06:58:32,187][20381] Updated weights for policy 1, policy_version 190769 (0.0011) [2023-12-27 06:58:32,202][20347] Updated weights for policy 0, policy_version 190997 (0.0005) [2023-12-27 06:58:32,203][20292] KL-divergence is very high: 162.2600 [2023-12-27 06:58:32,254][20347] Updated weights for policy 0, policy_version 191007 (0.0006) [2023-12-27 06:58:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 97763328. Throughput: 0: 10373.5, 1: 10398.4. Samples: 97767220. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:32,363][19170] Avg episode reward: [(0, '999.960'), (1, '947.450')] [2023-12-27 06:58:32,891][20292] KL-divergence is very high: 101.2746 [2023-12-27 06:58:32,906][20381] Updated weights for policy 1, policy_version 190779 (0.0009) [2023-12-27 06:58:32,931][20347] Updated weights for policy 0, policy_version 191017 (0.0008) [2023-12-27 06:58:32,933][20292] KL-divergence is very high: 181.2171 [2023-12-27 06:58:32,959][20381] Updated weights for policy 1, policy_version 190789 (0.0007) [2023-12-27 06:58:32,986][20347] Updated weights for policy 0, policy_version 191027 (0.0009) [2023-12-27 06:58:33,014][20381] Updated weights for policy 1, policy_version 190799 (0.0005) [2023-12-27 06:58:33,041][20347] Updated weights for policy 0, policy_version 191037 (0.0010) [2023-12-27 06:58:33,092][20347] Updated weights for policy 0, policy_version 191047 (0.0005) [2023-12-27 06:58:33,674][20347] Updated weights for policy 0, policy_version 191057 (0.0005) [2023-12-27 06:58:33,718][20381] Updated weights for policy 1, policy_version 190809 (0.0007) [2023-12-27 06:58:33,733][20347] Updated weights for policy 0, policy_version 191067 (0.0005) [2023-12-27 06:58:33,735][20292] KL-divergence is very high: 100.9989 [2023-12-27 06:58:33,773][20381] Updated weights for policy 1, policy_version 190819 (0.0005) [2023-12-27 06:58:33,791][20347] Updated weights for policy 0, policy_version 191077 (0.0005) [2023-12-27 06:58:33,830][20381] Updated weights for policy 1, policy_version 190829 (0.0005) [2023-12-27 06:58:33,853][20292] KL-divergence is very high: 221.8620 [2023-12-27 06:58:33,859][20292] KL-divergence is very high: 227.2899 [2023-12-27 06:58:33,871][20292] KL-divergence is very high: 204.7826 [2023-12-27 06:58:33,886][20381] Updated weights for policy 1, policy_version 190839 (0.0005) [2023-12-27 06:58:34,356][20347] Updated weights for policy 0, policy_version 191087 (0.0007) [2023-12-27 06:58:34,414][20347] Updated weights for policy 0, policy_version 191097 (0.0009) [2023-12-27 06:58:34,468][20347] Updated weights for policy 0, policy_version 191107 (0.0009) [2023-12-27 06:58:34,476][20381] Updated weights for policy 1, policy_version 190849 (0.0010) [2023-12-27 06:58:34,530][20292] KL-divergence is very high: 353.8866 [2023-12-27 06:58:34,537][20381] Updated weights for policy 1, policy_version 190859 (0.0011) [2023-12-27 06:58:34,544][20292] KL-divergence is very high: 402.7893 [2023-12-27 06:58:34,551][20292] KL-divergence is very high: 357.6499 [2023-12-27 06:58:34,565][20292] KL-divergence is very high: 230.1301 [2023-12-27 06:58:34,583][20292] KL-divergence is very high: 146.0068 [2023-12-27 06:58:34,599][20381] Updated weights for policy 1, policy_version 190869 (0.0011) [2023-12-27 06:58:34,612][20292] KL-divergence is very high: 185.4132 [2023-12-27 06:58:35,174][20347] Updated weights for policy 0, policy_version 191117 (0.0009) [2023-12-27 06:58:35,227][20347] Updated weights for policy 0, policy_version 191127 (0.0010) [2023-12-27 06:58:35,253][20292] KL-divergence is very high: 190.9443 [2023-12-27 06:58:35,264][20292] KL-divergence is very high: 1351.5931 [2023-12-27 06:58:35,270][20292] KL-divergence is very high: 664.2303 [2023-12-27 06:58:35,275][20347] Updated weights for policy 0, policy_version 191137 (0.0005) [2023-12-27 06:58:35,275][20292] KL-divergence is very high: 1030.4961 [2023-12-27 06:58:35,291][20381] Updated weights for policy 1, policy_version 190879 (0.0010) [2023-12-27 06:58:35,308][20292] KL-divergence is very high: 223.4948 [2023-12-27 06:58:35,314][20292] KL-divergence is very high: 299.9322 [2023-12-27 06:58:35,319][20292] KL-divergence is very high: 235.1531 [2023-12-27 06:58:35,324][20292] KL-divergence is very high: 563.3331 [2023-12-27 06:58:35,329][20292] KL-divergence is very high: 328.1118 [2023-12-27 06:58:35,347][20381] Updated weights for policy 1, policy_version 190889 (0.0010) [2023-12-27 06:58:35,350][20292] KL-divergence is very high: 651.7972 [2023-12-27 06:58:35,355][20292] KL-divergence is very high: 532.0842 [2023-12-27 06:58:35,360][20292] KL-divergence is very high: 782.2192 [2023-12-27 06:58:35,387][20292] KL-divergence is very high: 530.9591 [2023-12-27 06:58:35,392][20292] KL-divergence is very high: 268.1721 [2023-12-27 06:58:35,393][20381] Updated weights for policy 1, policy_version 190899 (0.0010) [2023-12-27 06:58:35,396][20292] KL-divergence is very high: 374.9955 [2023-12-27 06:58:35,401][20292] KL-divergence is very high: 337.5692 [2023-12-27 06:58:35,406][20292] KL-divergence is very high: 254.3056 [2023-12-27 06:58:35,933][20347] Updated weights for policy 0, policy_version 191147 (0.0008) [2023-12-27 06:58:35,990][20347] Updated weights for policy 0, policy_version 191157 (0.0010) [2023-12-27 06:58:36,048][20347] Updated weights for policy 0, policy_version 191167 (0.0010) [2023-12-27 06:58:36,061][20292] KL-divergence is very high: 290.5512 [2023-12-27 06:58:36,068][20292] KL-divergence is very high: 215.6791 [2023-12-27 06:58:36,081][20292] KL-divergence is very high: 157.9644 [2023-12-27 06:58:36,093][20381] Updated weights for policy 1, policy_version 190909 (0.0011) [2023-12-27 06:58:36,110][20292] KL-divergence is very high: 145.4972 [2023-12-27 06:58:36,118][20292] KL-divergence is very high: 591.8167 [2023-12-27 06:58:36,125][20292] KL-divergence is very high: 183.2918 [2023-12-27 06:58:36,138][20292] KL-divergence is very high: 194.9468 [2023-12-27 06:58:36,162][20381] Updated weights for policy 1, policy_version 190919 (0.0008) [2023-12-27 06:58:36,164][20292] KL-divergence is very high: 101.0423 [2023-12-27 06:58:36,171][20292] KL-divergence is very high: 509.1892 [2023-12-27 06:58:36,178][20292] KL-divergence is very high: 182.2610 [2023-12-27 06:58:36,193][20292] KL-divergence is very high: 115.7490 [2023-12-27 06:58:36,219][20292] KL-divergence is very high: 116.2595 [2023-12-27 06:58:36,225][20292] KL-divergence is very high: 415.0148 [2023-12-27 06:58:36,231][20292] KL-divergence is very high: 176.2056 [2023-12-27 06:58:36,232][20381] Updated weights for policy 1, policy_version 190929 (0.0007) [2023-12-27 06:58:36,244][20292] KL-divergence is very high: 125.1470 [2023-12-27 06:58:36,267][20292] KL-divergence is very high: 154.6102 [2023-12-27 06:58:36,715][20347] Updated weights for policy 0, policy_version 191177 (0.0010) [2023-12-27 06:58:36,778][20347] Updated weights for policy 0, policy_version 191187 (0.0011) [2023-12-27 06:58:36,843][20347] Updated weights for policy 0, policy_version 191197 (0.0011) [2023-12-27 06:58:36,907][20347] Updated weights for policy 0, policy_version 191207 (0.0010) [2023-12-27 06:58:36,927][20381] Updated weights for policy 1, policy_version 190939 (0.0008) [2023-12-27 06:58:36,996][20381] Updated weights for policy 1, policy_version 190949 (0.0011) [2023-12-27 06:58:37,022][20292] KL-divergence is very high: 122.9657 [2023-12-27 06:58:37,060][20381] Updated weights for policy 1, policy_version 190959 (0.0011) [2023-12-27 06:58:37,074][20292] KL-divergence is very high: 122.1031 [2023-12-27 06:58:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 97861632. Throughput: 0: 10403.7, 1: 10391.2. Samples: 97831216. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '352.330')] [2023-12-27 06:58:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000190968_48898048.pth... [2023-12-27 06:58:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000191208_48963584.pth... [2023-12-27 06:58:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000190024_48660480.pth [2023-12-27 06:58:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000189752_48586752.pth [2023-12-27 06:58:37,590][20347] Updated weights for policy 0, policy_version 191217 (0.0011) [2023-12-27 06:58:37,646][20347] Updated weights for policy 0, policy_version 191227 (0.0010) [2023-12-27 06:58:37,707][20347] Updated weights for policy 0, policy_version 191237 (0.0010) [2023-12-27 06:58:37,752][20381] Updated weights for policy 1, policy_version 190969 (0.0011) [2023-12-27 06:58:37,808][20381] Updated weights for policy 1, policy_version 190979 (0.0010) [2023-12-27 06:58:37,810][20292] KL-divergence is very high: 109.6112 [2023-12-27 06:58:37,815][20292] KL-divergence is very high: 244.9258 [2023-12-27 06:58:37,858][20292] KL-divergence is very high: 134.9713 [2023-12-27 06:58:37,863][20381] Updated weights for policy 1, policy_version 190989 (0.0009) [2023-12-27 06:58:37,919][20381] Updated weights for policy 1, policy_version 190999 (0.0006) [2023-12-27 06:58:38,387][20347] Updated weights for policy 0, policy_version 191247 (0.0011) [2023-12-27 06:58:38,443][20347] Updated weights for policy 0, policy_version 191257 (0.0010) [2023-12-27 06:58:38,498][20347] Updated weights for policy 0, policy_version 191267 (0.0010) [2023-12-27 06:58:38,559][20381] Updated weights for policy 1, policy_version 191009 (0.0011) [2023-12-27 06:58:38,613][20381] Updated weights for policy 1, policy_version 191019 (0.0011) [2023-12-27 06:58:38,664][20381] Updated weights for policy 1, policy_version 191029 (0.0007) [2023-12-27 06:58:39,191][20347] Updated weights for policy 0, policy_version 191277 (0.0010) [2023-12-27 06:58:39,249][20347] Updated weights for policy 0, policy_version 191287 (0.0010) [2023-12-27 06:58:39,313][20347] Updated weights for policy 0, policy_version 191297 (0.0010) [2023-12-27 06:58:39,366][20381] Updated weights for policy 1, policy_version 191039 (0.0007) [2023-12-27 06:58:39,433][20381] Updated weights for policy 1, policy_version 191049 (0.0008) [2023-12-27 06:58:39,496][20381] Updated weights for policy 1, policy_version 191059 (0.0008) [2023-12-27 06:58:40,006][20347] Updated weights for policy 0, policy_version 191307 (0.0008) [2023-12-27 06:58:40,067][20347] Updated weights for policy 0, policy_version 191317 (0.0008) [2023-12-27 06:58:40,132][20347] Updated weights for policy 0, policy_version 191327 (0.0008) [2023-12-27 06:58:40,195][20381] Updated weights for policy 1, policy_version 191069 (0.0007) [2023-12-27 06:58:40,262][20381] Updated weights for policy 1, policy_version 191079 (0.0008) [2023-12-27 06:58:40,327][20381] Updated weights for policy 1, policy_version 191089 (0.0008) [2023-12-27 06:58:40,806][20347] Updated weights for policy 0, policy_version 191337 (0.0006) [2023-12-27 06:58:40,856][20347] Updated weights for policy 0, policy_version 191347 (0.0006) [2023-12-27 06:58:40,911][20347] Updated weights for policy 0, policy_version 191357 (0.0007) [2023-12-27 06:58:40,969][20347] Updated weights for policy 0, policy_version 191367 (0.0006) [2023-12-27 06:58:41,005][20381] Updated weights for policy 1, policy_version 191099 (0.0007) [2023-12-27 06:58:41,073][20381] Updated weights for policy 1, policy_version 191109 (0.0010) [2023-12-27 06:58:41,137][20381] Updated weights for policy 1, policy_version 191119 (0.0008) [2023-12-27 06:58:41,685][20347] Updated weights for policy 0, policy_version 191377 (0.0008) [2023-12-27 06:58:41,754][20347] Updated weights for policy 0, policy_version 191387 (0.0008) [2023-12-27 06:58:41,818][20347] Updated weights for policy 0, policy_version 191397 (0.0007) [2023-12-27 06:58:41,860][20292] KL-divergence is very high: 103.5974 [2023-12-27 06:58:41,865][20381] Updated weights for policy 1, policy_version 191129 (0.0008) [2023-12-27 06:58:41,924][20381] Updated weights for policy 1, policy_version 191139 (0.0011) [2023-12-27 06:58:41,932][20292] KL-divergence is very high: 113.6162 [2023-12-27 06:58:41,986][20381] Updated weights for policy 1, policy_version 191149 (0.0007) [2023-12-27 06:58:42,051][20381] Updated weights for policy 1, policy_version 191159 (0.0007) [2023-12-27 06:58:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 97959936. Throughput: 0: 10330.9, 1: 10291.0. Samples: 97951388. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '380.760')] [2023-12-27 06:58:42,525][20347] Updated weights for policy 0, policy_version 191407 (0.0008) [2023-12-27 06:58:42,584][20347] Updated weights for policy 0, policy_version 191417 (0.0007) [2023-12-27 06:58:42,642][20347] Updated weights for policy 0, policy_version 191427 (0.0007) [2023-12-27 06:58:42,712][20381] Updated weights for policy 1, policy_version 191169 (0.0007) [2023-12-27 06:58:42,774][20381] Updated weights for policy 1, policy_version 191179 (0.0007) [2023-12-27 06:58:42,835][20381] Updated weights for policy 1, policy_version 191189 (0.0010) [2023-12-27 06:58:43,394][20347] Updated weights for policy 0, policy_version 191437 (0.0009) [2023-12-27 06:58:43,436][20381] Updated weights for policy 1, policy_version 191199 (0.0009) [2023-12-27 06:58:43,449][20347] Updated weights for policy 0, policy_version 191447 (0.0011) [2023-12-27 06:58:43,493][20381] Updated weights for policy 1, policy_version 191209 (0.0007) [2023-12-27 06:58:43,508][20347] Updated weights for policy 0, policy_version 191457 (0.0010) [2023-12-27 06:58:43,548][20381] Updated weights for policy 1, policy_version 191219 (0.0006) [2023-12-27 06:58:44,163][20347] Updated weights for policy 0, policy_version 191467 (0.0007) [2023-12-27 06:58:44,220][20347] Updated weights for policy 0, policy_version 191477 (0.0010) [2023-12-27 06:58:44,223][20381] Updated weights for policy 1, policy_version 191229 (0.0009) [2023-12-27 06:58:44,277][20347] Updated weights for policy 0, policy_version 191487 (0.0010) [2023-12-27 06:58:44,283][20381] Updated weights for policy 1, policy_version 191239 (0.0005) [2023-12-27 06:58:44,348][20381] Updated weights for policy 1, policy_version 191249 (0.0009) [2023-12-27 06:58:44,913][20347] Updated weights for policy 0, policy_version 191497 (0.0010) [2023-12-27 06:58:44,973][20347] Updated weights for policy 0, policy_version 191507 (0.0008) [2023-12-27 06:58:45,006][20381] Updated weights for policy 1, policy_version 191259 (0.0011) [2023-12-27 06:58:45,034][20347] Updated weights for policy 0, policy_version 191517 (0.0008) [2023-12-27 06:58:45,070][20381] Updated weights for policy 1, policy_version 191269 (0.0011) [2023-12-27 06:58:45,091][20347] Updated weights for policy 0, policy_version 191527 (0.0008) [2023-12-27 06:58:45,134][20381] Updated weights for policy 1, policy_version 191279 (0.0011) [2023-12-27 06:58:45,723][20381] Updated weights for policy 1, policy_version 191289 (0.0011) [2023-12-27 06:58:45,774][20381] Updated weights for policy 1, policy_version 191299 (0.0010) [2023-12-27 06:58:45,819][20381] Updated weights for policy 1, policy_version 191309 (0.0010) [2023-12-27 06:58:45,862][20347] Updated weights for policy 0, policy_version 191537 (0.0006) [2023-12-27 06:58:45,876][20381] Updated weights for policy 1, policy_version 191319 (0.0011) [2023-12-27 06:58:45,920][20347] Updated weights for policy 0, policy_version 191547 (0.0007) [2023-12-27 06:58:45,978][20347] Updated weights for policy 0, policy_version 191557 (0.0008) [2023-12-27 06:58:46,595][20381] Updated weights for policy 1, policy_version 191329 (0.0008) [2023-12-27 06:58:46,621][20347] Updated weights for policy 0, policy_version 191567 (0.0006) [2023-12-27 06:58:46,646][20381] Updated weights for policy 1, policy_version 191339 (0.0007) [2023-12-27 06:58:46,672][20347] Updated weights for policy 0, policy_version 191577 (0.0005) [2023-12-27 06:58:46,698][20381] Updated weights for policy 1, policy_version 191349 (0.0008) [2023-12-27 06:58:46,726][20347] Updated weights for policy 0, policy_version 191587 (0.0005) [2023-12-27 06:58:47,328][20347] Updated weights for policy 0, policy_version 191597 (0.0006) [2023-12-27 06:58:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 98058240. Throughput: 0: 10286.9, 1: 10308.6. Samples: 98074240. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '49.480')] [2023-12-27 06:58:47,387][20347] Updated weights for policy 0, policy_version 191607 (0.0007) [2023-12-27 06:58:47,407][20381] Updated weights for policy 1, policy_version 191359 (0.0007) [2023-12-27 06:58:47,446][20347] Updated weights for policy 0, policy_version 191617 (0.0007) [2023-12-27 06:58:47,470][20381] Updated weights for policy 1, policy_version 191369 (0.0005) [2023-12-27 06:58:47,537][20381] Updated weights for policy 1, policy_version 191379 (0.0007) [2023-12-27 06:58:48,118][20347] Updated weights for policy 0, policy_version 191627 (0.0007) [2023-12-27 06:58:48,174][20347] Updated weights for policy 0, policy_version 191637 (0.0007) [2023-12-27 06:58:48,207][20381] Updated weights for policy 1, policy_version 191389 (0.0006) [2023-12-27 06:58:48,226][20347] Updated weights for policy 0, policy_version 191647 (0.0007) [2023-12-27 06:58:48,267][20381] Updated weights for policy 1, policy_version 191399 (0.0007) [2023-12-27 06:58:48,319][20381] Updated weights for policy 1, policy_version 191409 (0.0008) [2023-12-27 06:58:48,954][20347] Updated weights for policy 0, policy_version 191657 (0.0007) [2023-12-27 06:58:48,999][20381] Updated weights for policy 1, policy_version 191419 (0.0007) [2023-12-27 06:58:49,002][20347] Updated weights for policy 0, policy_version 191667 (0.0007) [2023-12-27 06:58:49,050][20347] Updated weights for policy 0, policy_version 191677 (0.0005) [2023-12-27 06:58:49,055][20381] Updated weights for policy 1, policy_version 191429 (0.0007) [2023-12-27 06:58:49,107][20347] Updated weights for policy 0, policy_version 191687 (0.0007) [2023-12-27 06:58:49,108][20381] Updated weights for policy 1, policy_version 191439 (0.0006) [2023-12-27 06:58:49,796][20381] Updated weights for policy 1, policy_version 191449 (0.0007) [2023-12-27 06:58:49,851][20347] Updated weights for policy 0, policy_version 191697 (0.0008) [2023-12-27 06:58:49,854][20381] Updated weights for policy 1, policy_version 191459 (0.0008) [2023-12-27 06:58:49,899][20347] Updated weights for policy 0, policy_version 191707 (0.0008) [2023-12-27 06:58:49,903][20381] Updated weights for policy 1, policy_version 191469 (0.0007) [2023-12-27 06:58:49,964][20347] Updated weights for policy 0, policy_version 191717 (0.0010) [2023-12-27 06:58:49,966][20381] Updated weights for policy 1, policy_version 191479 (0.0007) [2023-12-27 06:58:50,671][20381] Updated weights for policy 1, policy_version 191489 (0.0007) [2023-12-27 06:58:50,698][20347] Updated weights for policy 0, policy_version 191727 (0.0010) [2023-12-27 06:58:50,733][20381] Updated weights for policy 1, policy_version 191499 (0.0006) [2023-12-27 06:58:50,754][20347] Updated weights for policy 0, policy_version 191737 (0.0011) [2023-12-27 06:58:50,792][20381] Updated weights for policy 1, policy_version 191509 (0.0005) [2023-12-27 06:58:50,812][20347] Updated weights for policy 0, policy_version 191747 (0.0010) [2023-12-27 06:58:51,485][20381] Updated weights for policy 1, policy_version 191519 (0.0007) [2023-12-27 06:58:51,514][20347] Updated weights for policy 0, policy_version 191757 (0.0010) [2023-12-27 06:58:51,542][20381] Updated weights for policy 1, policy_version 191529 (0.0005) [2023-12-27 06:58:51,580][20347] Updated weights for policy 0, policy_version 191767 (0.0010) [2023-12-27 06:58:51,604][20381] Updated weights for policy 1, policy_version 191539 (0.0006) [2023-12-27 06:58:51,642][20347] Updated weights for policy 0, policy_version 191777 (0.0009) [2023-12-27 06:58:52,262][20347] Updated weights for policy 0, policy_version 191787 (0.0008) [2023-12-27 06:58:52,325][20347] Updated weights for policy 0, policy_version 191797 (0.0008) [2023-12-27 06:58:52,336][20381] Updated weights for policy 1, policy_version 191549 (0.0007) [2023-12-27 06:58:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20716.1). Total num frames: 98156544. Throughput: 0: 10241.9, 1: 10293.5. Samples: 98133736. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '44.870')] [2023-12-27 06:58:52,376][20347] Updated weights for policy 0, policy_version 191807 (0.0008) [2023-12-27 06:58:52,393][20381] Updated weights for policy 1, policy_version 191559 (0.0007) [2023-12-27 06:58:52,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000191816_49119232.pth... [2023-12-27 06:58:52,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000190600_48807936.pth [2023-12-27 06:58:52,445][20381] Updated weights for policy 1, policy_version 191569 (0.0006) [2023-12-27 06:58:52,487][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000191576_49053696.pth... [2023-12-27 06:58:52,491][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000190360_48742400.pth [2023-12-27 06:58:53,110][20347] Updated weights for policy 0, policy_version 191817 (0.0008) [2023-12-27 06:58:53,138][20381] Updated weights for policy 1, policy_version 191579 (0.0007) [2023-12-27 06:58:53,159][20347] Updated weights for policy 0, policy_version 191827 (0.0005) [2023-12-27 06:58:53,189][20381] Updated weights for policy 1, policy_version 191589 (0.0008) [2023-12-27 06:58:53,218][20347] Updated weights for policy 0, policy_version 191837 (0.0005) [2023-12-27 06:58:53,256][20381] Updated weights for policy 1, policy_version 191599 (0.0008) [2023-12-27 06:58:53,273][20347] Updated weights for policy 0, policy_version 191847 (0.0005) [2023-12-27 06:58:53,775][20347] Updated weights for policy 0, policy_version 191857 (0.0007) [2023-12-27 06:58:53,831][20347] Updated weights for policy 0, policy_version 191867 (0.0007) [2023-12-27 06:58:53,884][20347] Updated weights for policy 0, policy_version 191877 (0.0007) [2023-12-27 06:58:53,996][20381] Updated weights for policy 1, policy_version 191609 (0.0008) [2023-12-27 06:58:54,054][20381] Updated weights for policy 1, policy_version 191619 (0.0008) [2023-12-27 06:58:54,105][20381] Updated weights for policy 1, policy_version 191629 (0.0008) [2023-12-27 06:58:54,162][20381] Updated weights for policy 1, policy_version 191639 (0.0007) [2023-12-27 06:58:54,504][20347] Updated weights for policy 0, policy_version 191887 (0.0007) [2023-12-27 06:58:54,563][20347] Updated weights for policy 0, policy_version 191897 (0.0007) [2023-12-27 06:58:54,630][20347] Updated weights for policy 0, policy_version 191907 (0.0008) [2023-12-27 06:58:54,856][20381] Updated weights for policy 1, policy_version 191649 (0.0007) [2023-12-27 06:58:54,910][20381] Updated weights for policy 1, policy_version 191659 (0.0008) [2023-12-27 06:58:54,964][20381] Updated weights for policy 1, policy_version 191669 (0.0007) [2023-12-27 06:58:55,311][20347] Updated weights for policy 0, policy_version 191917 (0.0009) [2023-12-27 06:58:55,355][20347] Updated weights for policy 0, policy_version 191927 (0.0010) [2023-12-27 06:58:55,406][20347] Updated weights for policy 0, policy_version 191937 (0.0010) [2023-12-27 06:58:55,664][20381] Updated weights for policy 1, policy_version 191679 (0.0006) [2023-12-27 06:58:55,724][20381] Updated weights for policy 1, policy_version 191689 (0.0005) [2023-12-27 06:58:55,778][20381] Updated weights for policy 1, policy_version 191699 (0.0007) [2023-12-27 06:58:56,074][20347] Updated weights for policy 0, policy_version 191947 (0.0006) [2023-12-27 06:58:56,138][20347] Updated weights for policy 0, policy_version 191957 (0.0006) [2023-12-27 06:58:56,195][20347] Updated weights for policy 0, policy_version 191967 (0.0005) [2023-12-27 06:58:56,406][20381] Updated weights for policy 1, policy_version 191709 (0.0007) [2023-12-27 06:58:56,458][20381] Updated weights for policy 1, policy_version 191719 (0.0007) [2023-12-27 06:58:56,509][20381] Updated weights for policy 1, policy_version 191729 (0.0007) [2023-12-27 06:58:56,788][20347] Updated weights for policy 0, policy_version 191977 (0.0009) [2023-12-27 06:58:56,841][20347] Updated weights for policy 0, policy_version 191987 (0.0007) [2023-12-27 06:58:56,901][20347] Updated weights for policy 0, policy_version 191997 (0.0005) [2023-12-27 06:58:56,960][20347] Updated weights for policy 0, policy_version 192007 (0.0007) [2023-12-27 06:58:57,218][20381] Updated weights for policy 1, policy_version 191739 (0.0008) [2023-12-27 06:58:57,281][20381] Updated weights for policy 1, policy_version 191749 (0.0008) [2023-12-27 06:58:57,337][20381] Updated weights for policy 1, policy_version 191759 (0.0006) [2023-12-27 06:58:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 98263040. Throughput: 0: 10260.3, 1: 10276.6. Samples: 98256972. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:58:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '95.860')] [2023-12-27 06:58:57,609][20347] Updated weights for policy 0, policy_version 192017 (0.0010) [2023-12-27 06:58:57,658][20347] Updated weights for policy 0, policy_version 192027 (0.0010) [2023-12-27 06:58:57,708][20347] Updated weights for policy 0, policy_version 192037 (0.0008) [2023-12-27 06:58:57,919][20381] Updated weights for policy 1, policy_version 191769 (0.0006) [2023-12-27 06:58:57,966][20381] Updated weights for policy 1, policy_version 191779 (0.0009) [2023-12-27 06:58:58,021][20381] Updated weights for policy 1, policy_version 191789 (0.0009) [2023-12-27 06:58:58,071][20381] Updated weights for policy 1, policy_version 191799 (0.0007) [2023-12-27 06:58:58,414][20347] Updated weights for policy 0, policy_version 192047 (0.0008) [2023-12-27 06:58:58,480][20347] Updated weights for policy 0, policy_version 192057 (0.0009) [2023-12-27 06:58:58,542][20347] Updated weights for policy 0, policy_version 192067 (0.0008) [2023-12-27 06:58:58,861][20381] Updated weights for policy 1, policy_version 191809 (0.0008) [2023-12-27 06:58:58,920][20381] Updated weights for policy 1, policy_version 191819 (0.0007) [2023-12-27 06:58:58,976][20381] Updated weights for policy 1, policy_version 191829 (0.0008) [2023-12-27 06:58:59,228][20347] Updated weights for policy 0, policy_version 192077 (0.0007) [2023-12-27 06:58:59,282][20347] Updated weights for policy 0, policy_version 192087 (0.0008) [2023-12-27 06:58:59,336][20347] Updated weights for policy 0, policy_version 192097 (0.0008) [2023-12-27 06:58:59,689][20381] Updated weights for policy 1, policy_version 191839 (0.0007) [2023-12-27 06:58:59,750][20381] Updated weights for policy 1, policy_version 191849 (0.0005) [2023-12-27 06:58:59,808][20381] Updated weights for policy 1, policy_version 191859 (0.0006) [2023-12-27 06:59:00,047][20347] Updated weights for policy 0, policy_version 192107 (0.0008) [2023-12-27 06:59:00,102][20347] Updated weights for policy 0, policy_version 192117 (0.0008) [2023-12-27 06:59:00,161][20347] Updated weights for policy 0, policy_version 192127 (0.0007) [2023-12-27 06:59:00,434][20381] Updated weights for policy 1, policy_version 191869 (0.0009) [2023-12-27 06:59:00,486][20381] Updated weights for policy 1, policy_version 191879 (0.0010) [2023-12-27 06:59:00,537][20381] Updated weights for policy 1, policy_version 191889 (0.0010) [2023-12-27 06:59:00,833][20347] Updated weights for policy 0, policy_version 192137 (0.0007) [2023-12-27 06:59:00,894][20347] Updated weights for policy 0, policy_version 192147 (0.0008) [2023-12-27 06:59:00,954][20347] Updated weights for policy 0, policy_version 192157 (0.0008) [2023-12-27 06:59:00,999][20347] Updated weights for policy 0, policy_version 192167 (0.0007) [2023-12-27 06:59:01,161][20381] Updated weights for policy 1, policy_version 191899 (0.0011) [2023-12-27 06:59:01,214][20381] Updated weights for policy 1, policy_version 191909 (0.0011) [2023-12-27 06:59:01,268][20381] Updated weights for policy 1, policy_version 191919 (0.0010) [2023-12-27 06:59:01,672][20347] Updated weights for policy 0, policy_version 192177 (0.0007) [2023-12-27 06:59:01,736][20347] Updated weights for policy 0, policy_version 192187 (0.0007) [2023-12-27 06:59:01,802][20347] Updated weights for policy 0, policy_version 192197 (0.0006) [2023-12-27 06:59:01,952][20381] Updated weights for policy 1, policy_version 191929 (0.0010) [2023-12-27 06:59:02,014][20381] Updated weights for policy 1, policy_version 191939 (0.0007) [2023-12-27 06:59:02,073][20381] Updated weights for policy 1, policy_version 191949 (0.0011) [2023-12-27 06:59:02,130][20381] Updated weights for policy 1, policy_version 191959 (0.0011) [2023-12-27 06:59:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 98369536. Throughput: 0: 10263.1, 1: 10234.0. Samples: 98380136. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:59:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '449.200')] [2023-12-27 06:59:02,475][20347] Updated weights for policy 0, policy_version 192207 (0.0007) [2023-12-27 06:59:02,533][20347] Updated weights for policy 0, policy_version 192217 (0.0007) [2023-12-27 06:59:02,589][20347] Updated weights for policy 0, policy_version 192227 (0.0005) [2023-12-27 06:59:02,835][20381] Updated weights for policy 1, policy_version 191969 (0.0010) [2023-12-27 06:59:02,900][20381] Updated weights for policy 1, policy_version 191979 (0.0011) [2023-12-27 06:59:02,960][20381] Updated weights for policy 1, policy_version 191989 (0.0011) [2023-12-27 06:59:03,222][20347] Updated weights for policy 0, policy_version 192237 (0.0009) [2023-12-27 06:59:03,280][20347] Updated weights for policy 0, policy_version 192247 (0.0010) [2023-12-27 06:59:03,337][20347] Updated weights for policy 0, policy_version 192257 (0.0010) [2023-12-27 06:59:03,600][20381] Updated weights for policy 1, policy_version 191999 (0.0007) [2023-12-27 06:59:03,647][20381] Updated weights for policy 1, policy_version 192009 (0.0005) [2023-12-27 06:59:03,709][20381] Updated weights for policy 1, policy_version 192019 (0.0008) [2023-12-27 06:59:03,992][20347] Updated weights for policy 0, policy_version 192267 (0.0011) [2023-12-27 06:59:04,043][20347] Updated weights for policy 0, policy_version 192277 (0.0010) [2023-12-27 06:59:04,089][20347] Updated weights for policy 0, policy_version 192287 (0.0010) [2023-12-27 06:59:04,361][20381] Updated weights for policy 1, policy_version 192029 (0.0009) [2023-12-27 06:59:04,426][20381] Updated weights for policy 1, policy_version 192039 (0.0008) [2023-12-27 06:59:04,487][20381] Updated weights for policy 1, policy_version 192049 (0.0006) [2023-12-27 06:59:04,794][20347] Updated weights for policy 0, policy_version 192297 (0.0010) [2023-12-27 06:59:04,847][20347] Updated weights for policy 0, policy_version 192307 (0.0007) [2023-12-27 06:59:04,907][20347] Updated weights for policy 0, policy_version 192317 (0.0007) [2023-12-27 06:59:04,961][20347] Updated weights for policy 0, policy_version 192327 (0.0008) [2023-12-27 06:59:05,119][20381] Updated weights for policy 1, policy_version 192059 (0.0008) [2023-12-27 06:59:05,164][20381] Updated weights for policy 1, policy_version 192069 (0.0011) [2023-12-27 06:59:05,218][20381] Updated weights for policy 1, policy_version 192079 (0.0010) [2023-12-27 06:59:05,576][20347] Updated weights for policy 0, policy_version 192337 (0.0005) [2023-12-27 06:59:05,639][20347] Updated weights for policy 0, policy_version 192347 (0.0007) [2023-12-27 06:59:05,695][20347] Updated weights for policy 0, policy_version 192357 (0.0008) [2023-12-27 06:59:05,899][20381] Updated weights for policy 1, policy_version 192089 (0.0010) [2023-12-27 06:59:05,959][20381] Updated weights for policy 1, policy_version 192099 (0.0005) [2023-12-27 06:59:06,020][20381] Updated weights for policy 1, policy_version 192109 (0.0005) [2023-12-27 06:59:06,084][20381] Updated weights for policy 1, policy_version 192119 (0.0005) [2023-12-27 06:59:06,383][20347] Updated weights for policy 0, policy_version 192367 (0.0009) [2023-12-27 06:59:06,441][20347] Updated weights for policy 0, policy_version 192377 (0.0010) [2023-12-27 06:59:06,507][20347] Updated weights for policy 0, policy_version 192387 (0.0010) [2023-12-27 06:59:06,623][20381] Updated weights for policy 1, policy_version 192129 (0.0008) [2023-12-27 06:59:06,690][20381] Updated weights for policy 1, policy_version 192139 (0.0008) [2023-12-27 06:59:06,763][20381] Updated weights for policy 1, policy_version 192149 (0.0009) [2023-12-27 06:59:07,199][20347] Updated weights for policy 0, policy_version 192397 (0.0009) [2023-12-27 06:59:07,260][20347] Updated weights for policy 0, policy_version 192407 (0.0006) [2023-12-27 06:59:07,318][20347] Updated weights for policy 0, policy_version 192417 (0.0005) [2023-12-27 06:59:07,345][20381] Updated weights for policy 1, policy_version 192159 (0.0009) [2023-12-27 06:59:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 98476032. Throughput: 0: 10257.0, 1: 10248.8. Samples: 98442364. Policy #0 lag: (min: 31.0, avg: 48.6, max: 63.0) [2023-12-27 06:59:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '346.820')] [2023-12-27 06:59:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000192424_49274880.pth... [2023-12-27 06:59:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000191208_48963584.pth [2023-12-27 06:59:07,406][20381] Updated weights for policy 1, policy_version 192169 (0.0011) [2023-12-27 06:59:07,464][20381] Updated weights for policy 1, policy_version 192179 (0.0011) [2023-12-27 06:59:07,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000192184_49209344.pth... [2023-12-27 06:59:07,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000190968_48898048.pth [2023-12-27 06:59:07,836][20347] Updated weights for policy 0, policy_version 192427 (0.0005) [2023-12-27 06:59:07,886][20347] Updated weights for policy 0, policy_version 192437 (0.0005) [2023-12-27 06:59:07,944][20347] Updated weights for policy 0, policy_version 192447 (0.0005) [2023-12-27 06:59:08,117][20381] Updated weights for policy 1, policy_version 192189 (0.0011) [2023-12-27 06:59:08,166][20381] Updated weights for policy 1, policy_version 192199 (0.0010) [2023-12-27 06:59:08,222][20381] Updated weights for policy 1, policy_version 192209 (0.0010) [2023-12-27 06:59:08,570][20347] Updated weights for policy 0, policy_version 192457 (0.0008) [2023-12-27 06:59:08,623][20347] Updated weights for policy 0, policy_version 192467 (0.0007) [2023-12-27 06:59:08,677][20347] Updated weights for policy 0, policy_version 192477 (0.0008) [2023-12-27 06:59:08,725][20347] Updated weights for policy 0, policy_version 192487 (0.0007) [2023-12-27 06:59:08,903][20381] Updated weights for policy 1, policy_version 192219 (0.0011) [2023-12-27 06:59:08,949][20381] Updated weights for policy 1, policy_version 192229 (0.0010) [2023-12-27 06:59:08,996][20381] Updated weights for policy 1, policy_version 192239 (0.0010) [2023-12-27 06:59:09,358][20347] Updated weights for policy 0, policy_version 192497 (0.0007) [2023-12-27 06:59:09,420][20347] Updated weights for policy 0, policy_version 192507 (0.0007) [2023-12-27 06:59:09,472][20347] Updated weights for policy 0, policy_version 192517 (0.0008) [2023-12-27 06:59:09,726][20381] Updated weights for policy 1, policy_version 192249 (0.0010) [2023-12-27 06:59:09,781][20381] Updated weights for policy 1, policy_version 192259 (0.0008) [2023-12-27 06:59:09,835][20381] Updated weights for policy 1, policy_version 192269 (0.0008) [2023-12-27 06:59:09,901][20381] Updated weights for policy 1, policy_version 192279 (0.0008) [2023-12-27 06:59:10,144][20347] Updated weights for policy 0, policy_version 192527 (0.0007) [2023-12-27 06:59:10,211][20347] Updated weights for policy 0, policy_version 192537 (0.0006) [2023-12-27 06:59:10,274][20347] Updated weights for policy 0, policy_version 192547 (0.0006) [2023-12-27 06:59:10,619][20381] Updated weights for policy 1, policy_version 192289 (0.0007) [2023-12-27 06:59:10,674][20381] Updated weights for policy 1, policy_version 192299 (0.0007) [2023-12-27 06:59:10,730][20381] Updated weights for policy 1, policy_version 192309 (0.0007) [2023-12-27 06:59:10,941][20347] Updated weights for policy 0, policy_version 192557 (0.0008) [2023-12-27 06:59:11,003][20347] Updated weights for policy 0, policy_version 192567 (0.0009) [2023-12-27 06:59:11,062][20291] KL-divergence is very high: 198.0103 [2023-12-27 06:59:11,069][20347] Updated weights for policy 0, policy_version 192577 (0.0010) [2023-12-27 06:59:11,454][20381] Updated weights for policy 1, policy_version 192319 (0.0006) [2023-12-27 06:59:11,502][20381] Updated weights for policy 1, policy_version 192329 (0.0005) [2023-12-27 06:59:11,551][20381] Updated weights for policy 1, policy_version 192339 (0.0006) [2023-12-27 06:59:11,782][20347] Updated weights for policy 0, policy_version 192587 (0.0009) [2023-12-27 06:59:11,847][20347] Updated weights for policy 0, policy_version 192597 (0.0007) [2023-12-27 06:59:11,900][20347] Updated weights for policy 0, policy_version 192607 (0.0007) [2023-12-27 06:59:12,213][20381] Updated weights for policy 1, policy_version 192349 (0.0008) [2023-12-27 06:59:12,272][20381] Updated weights for policy 1, policy_version 192359 (0.0010) [2023-12-27 06:59:12,333][20381] Updated weights for policy 1, policy_version 192369 (0.0011) [2023-12-27 06:59:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 98574336. Throughput: 0: 10360.4, 1: 10238.5. Samples: 98569764. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:12,363][19170] Avg episode reward: [(0, '995.200'), (1, '504.290')] [2023-12-27 06:59:12,556][20347] Updated weights for policy 0, policy_version 192617 (0.0007) [2023-12-27 06:59:12,616][20347] Updated weights for policy 0, policy_version 192627 (0.0008) [2023-12-27 06:59:12,678][20347] Updated weights for policy 0, policy_version 192637 (0.0008) [2023-12-27 06:59:12,732][20347] Updated weights for policy 0, policy_version 192647 (0.0007) [2023-12-27 06:59:13,035][20381] Updated weights for policy 1, policy_version 192379 (0.0007) [2023-12-27 06:59:13,085][20381] Updated weights for policy 1, policy_version 192389 (0.0005) [2023-12-27 06:59:13,150][20381] Updated weights for policy 1, policy_version 192399 (0.0005) [2023-12-27 06:59:13,408][20347] Updated weights for policy 0, policy_version 192657 (0.0008) [2023-12-27 06:59:13,464][20347] Updated weights for policy 0, policy_version 192667 (0.0007) [2023-12-27 06:59:13,519][20347] Updated weights for policy 0, policy_version 192677 (0.0010) [2023-12-27 06:59:13,788][20381] Updated weights for policy 1, policy_version 192409 (0.0007) [2023-12-27 06:59:13,857][20381] Updated weights for policy 1, policy_version 192419 (0.0008) [2023-12-27 06:59:13,923][20381] Updated weights for policy 1, policy_version 192429 (0.0008) [2023-12-27 06:59:13,984][20381] Updated weights for policy 1, policy_version 192439 (0.0007) [2023-12-27 06:59:14,142][20347] Updated weights for policy 0, policy_version 192687 (0.0008) [2023-12-27 06:59:14,196][20347] Updated weights for policy 0, policy_version 192697 (0.0008) [2023-12-27 06:59:14,261][20347] Updated weights for policy 0, policy_version 192707 (0.0006) [2023-12-27 06:59:14,608][20381] Updated weights for policy 1, policy_version 192449 (0.0007) [2023-12-27 06:59:14,658][20381] Updated weights for policy 1, policy_version 192459 (0.0008) [2023-12-27 06:59:14,717][20381] Updated weights for policy 1, policy_version 192469 (0.0008) [2023-12-27 06:59:14,968][20347] Updated weights for policy 0, policy_version 192717 (0.0007) [2023-12-27 06:59:15,025][20347] Updated weights for policy 0, policy_version 192727 (0.0006) [2023-12-27 06:59:15,087][20347] Updated weights for policy 0, policy_version 192737 (0.0008) [2023-12-27 06:59:15,307][20381] Updated weights for policy 1, policy_version 192479 (0.0006) [2023-12-27 06:59:15,388][20381] Updated weights for policy 1, policy_version 192489 (0.0008) [2023-12-27 06:59:15,434][20381] Updated weights for policy 1, policy_version 192499 (0.0006) [2023-12-27 06:59:15,786][20347] Updated weights for policy 0, policy_version 192747 (0.0008) [2023-12-27 06:59:15,850][20347] Updated weights for policy 0, policy_version 192757 (0.0008) [2023-12-27 06:59:15,907][20347] Updated weights for policy 0, policy_version 192767 (0.0008) [2023-12-27 06:59:16,073][20381] Updated weights for policy 1, policy_version 192509 (0.0008) [2023-12-27 06:59:16,126][20381] Updated weights for policy 1, policy_version 192519 (0.0009) [2023-12-27 06:59:16,175][20381] Updated weights for policy 1, policy_version 192529 (0.0008) [2023-12-27 06:59:16,492][20347] Updated weights for policy 0, policy_version 192777 (0.0007) [2023-12-27 06:59:16,548][20347] Updated weights for policy 0, policy_version 192787 (0.0005) [2023-12-27 06:59:16,611][20347] Updated weights for policy 0, policy_version 192797 (0.0005) [2023-12-27 06:59:16,667][20347] Updated weights for policy 0, policy_version 192807 (0.0005) [2023-12-27 06:59:16,924][20381] Updated weights for policy 1, policy_version 192539 (0.0009) [2023-12-27 06:59:16,988][20381] Updated weights for policy 1, policy_version 192549 (0.0006) [2023-12-27 06:59:17,049][20381] Updated weights for policy 1, policy_version 192559 (0.0009) [2023-12-27 06:59:17,197][20347] Updated weights for policy 0, policy_version 192817 (0.0007) [2023-12-27 06:59:17,261][20347] Updated weights for policy 0, policy_version 192827 (0.0009) [2023-12-27 06:59:17,321][20347] Updated weights for policy 0, policy_version 192837 (0.0008) [2023-12-27 06:59:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 98689024. Throughput: 0: 10378.0, 1: 10245.6. Samples: 98695284. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:17,363][19170] Avg episode reward: [(0, '995.690'), (1, '496.200')] [2023-12-27 06:59:17,583][20381] Updated weights for policy 1, policy_version 192569 (0.0010) [2023-12-27 06:59:17,632][20381] Updated weights for policy 1, policy_version 192579 (0.0010) [2023-12-27 06:59:17,681][20381] Updated weights for policy 1, policy_version 192589 (0.0005) [2023-12-27 06:59:17,731][20381] Updated weights for policy 1, policy_version 192599 (0.0006) [2023-12-27 06:59:18,013][20347] Updated weights for policy 0, policy_version 192847 (0.0007) [2023-12-27 06:59:18,068][20347] Updated weights for policy 0, policy_version 192857 (0.0007) [2023-12-27 06:59:18,120][20347] Updated weights for policy 0, policy_version 192867 (0.0008) [2023-12-27 06:59:18,365][20381] Updated weights for policy 1, policy_version 192609 (0.0007) [2023-12-27 06:59:18,419][20381] Updated weights for policy 1, policy_version 192619 (0.0008) [2023-12-27 06:59:18,471][20381] Updated weights for policy 1, policy_version 192629 (0.0007) [2023-12-27 06:59:18,843][20347] Updated weights for policy 0, policy_version 192877 (0.0007) [2023-12-27 06:59:18,900][20347] Updated weights for policy 0, policy_version 192887 (0.0007) [2023-12-27 06:59:18,948][20347] Updated weights for policy 0, policy_version 192897 (0.0007) [2023-12-27 06:59:19,177][20381] Updated weights for policy 1, policy_version 192639 (0.0009) [2023-12-27 06:59:19,230][20381] Updated weights for policy 1, policy_version 192649 (0.0010) [2023-12-27 06:59:19,287][20381] Updated weights for policy 1, policy_version 192659 (0.0010) [2023-12-27 06:59:19,654][20347] Updated weights for policy 0, policy_version 192907 (0.0007) [2023-12-27 06:59:19,711][20347] Updated weights for policy 0, policy_version 192917 (0.0008) [2023-12-27 06:59:19,772][20347] Updated weights for policy 0, policy_version 192927 (0.0007) [2023-12-27 06:59:19,988][20381] Updated weights for policy 1, policy_version 192669 (0.0009) [2023-12-27 06:59:20,055][20381] Updated weights for policy 1, policy_version 192679 (0.0008) [2023-12-27 06:59:20,114][20381] Updated weights for policy 1, policy_version 192689 (0.0008) [2023-12-27 06:59:20,475][20347] Updated weights for policy 0, policy_version 192937 (0.0007) [2023-12-27 06:59:20,526][20347] Updated weights for policy 0, policy_version 192947 (0.0006) [2023-12-27 06:59:20,595][20347] Updated weights for policy 0, policy_version 192957 (0.0009) [2023-12-27 06:59:20,645][20347] Updated weights for policy 0, policy_version 192967 (0.0010) [2023-12-27 06:59:20,821][20381] Updated weights for policy 1, policy_version 192699 (0.0007) [2023-12-27 06:59:20,874][20381] Updated weights for policy 1, policy_version 192709 (0.0008) [2023-12-27 06:59:20,926][20381] Updated weights for policy 1, policy_version 192719 (0.0007) [2023-12-27 06:59:21,343][20347] Updated weights for policy 0, policy_version 192977 (0.0010) [2023-12-27 06:59:21,404][20347] Updated weights for policy 0, policy_version 192987 (0.0008) [2023-12-27 06:59:21,459][20347] Updated weights for policy 0, policy_version 192997 (0.0008) [2023-12-27 06:59:21,667][20381] Updated weights for policy 1, policy_version 192729 (0.0008) [2023-12-27 06:59:21,731][20381] Updated weights for policy 1, policy_version 192739 (0.0008) [2023-12-27 06:59:21,793][20381] Updated weights for policy 1, policy_version 192749 (0.0007) [2023-12-27 06:59:21,853][20381] Updated weights for policy 1, policy_version 192759 (0.0008) [2023-12-27 06:59:22,208][20347] Updated weights for policy 0, policy_version 193007 (0.0006) [2023-12-27 06:59:22,261][20347] Updated weights for policy 0, policy_version 193017 (0.0008) [2023-12-27 06:59:22,311][20347] Updated weights for policy 0, policy_version 193027 (0.0009) [2023-12-27 06:59:22,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 98787328. Throughput: 0: 10305.9, 1: 10254.0. Samples: 98756408. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '488.900')] [2023-12-27 06:59:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000192760_49356800.pth... [2023-12-27 06:59:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000193032_49430528.pth... [2023-12-27 06:59:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000191576_49053696.pth [2023-12-27 06:59:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000191816_49119232.pth [2023-12-27 06:59:22,583][20381] Updated weights for policy 1, policy_version 192769 (0.0008) [2023-12-27 06:59:22,635][20381] Updated weights for policy 1, policy_version 192779 (0.0008) [2023-12-27 06:59:22,686][20381] Updated weights for policy 1, policy_version 192789 (0.0007) [2023-12-27 06:59:23,033][20347] Updated weights for policy 0, policy_version 193037 (0.0010) [2023-12-27 06:59:23,094][20347] Updated weights for policy 0, policy_version 193047 (0.0010) [2023-12-27 06:59:23,141][20347] Updated weights for policy 0, policy_version 193057 (0.0009) [2023-12-27 06:59:23,420][20381] Updated weights for policy 1, policy_version 192799 (0.0009) [2023-12-27 06:59:23,481][20381] Updated weights for policy 1, policy_version 192809 (0.0008) [2023-12-27 06:59:23,539][20381] Updated weights for policy 1, policy_version 192819 (0.0005) [2023-12-27 06:59:23,739][20347] Updated weights for policy 0, policy_version 193067 (0.0005) [2023-12-27 06:59:23,796][20347] Updated weights for policy 0, policy_version 193077 (0.0005) [2023-12-27 06:59:23,859][20347] Updated weights for policy 0, policy_version 193087 (0.0005) [2023-12-27 06:59:24,126][20381] Updated weights for policy 1, policy_version 192829 (0.0008) [2023-12-27 06:59:24,188][20381] Updated weights for policy 1, policy_version 192839 (0.0011) [2023-12-27 06:59:24,248][20381] Updated weights for policy 1, policy_version 192849 (0.0011) [2023-12-27 06:59:24,391][20347] Updated weights for policy 0, policy_version 193097 (0.0006) [2023-12-27 06:59:24,449][20347] Updated weights for policy 0, policy_version 193107 (0.0010) [2023-12-27 06:59:24,506][20347] Updated weights for policy 0, policy_version 193117 (0.0010) [2023-12-27 06:59:24,561][20347] Updated weights for policy 0, policy_version 193127 (0.0010) [2023-12-27 06:59:24,916][20381] Updated weights for policy 1, policy_version 192859 (0.0011) [2023-12-27 06:59:24,960][20381] Updated weights for policy 1, policy_version 192869 (0.0010) [2023-12-27 06:59:25,006][20381] Updated weights for policy 1, policy_version 192879 (0.0010) [2023-12-27 06:59:25,161][20347] Updated weights for policy 0, policy_version 193137 (0.0006) [2023-12-27 06:59:25,216][20347] Updated weights for policy 0, policy_version 193147 (0.0005) [2023-12-27 06:59:25,272][20347] Updated weights for policy 0, policy_version 193157 (0.0008) [2023-12-27 06:59:25,715][20381] Updated weights for policy 1, policy_version 192889 (0.0010) [2023-12-27 06:59:25,777][20381] Updated weights for policy 1, policy_version 192899 (0.0010) [2023-12-27 06:59:25,829][20381] Updated weights for policy 1, policy_version 192909 (0.0007) [2023-12-27 06:59:25,878][20381] Updated weights for policy 1, policy_version 192919 (0.0008) [2023-12-27 06:59:25,888][20347] Updated weights for policy 0, policy_version 193167 (0.0010) [2023-12-27 06:59:25,935][20347] Updated weights for policy 0, policy_version 193177 (0.0010) [2023-12-27 06:59:25,990][20347] Updated weights for policy 0, policy_version 193187 (0.0010) [2023-12-27 06:59:26,475][20381] Updated weights for policy 1, policy_version 192929 (0.0007) [2023-12-27 06:59:26,520][20381] Updated weights for policy 1, policy_version 192939 (0.0010) [2023-12-27 06:59:26,570][20381] Updated weights for policy 1, policy_version 192949 (0.0010) [2023-12-27 06:59:26,578][20347] Updated weights for policy 0, policy_version 193197 (0.0008) [2023-12-27 06:59:26,626][20347] Updated weights for policy 0, policy_version 193207 (0.0005) [2023-12-27 06:59:26,675][20347] Updated weights for policy 0, policy_version 193217 (0.0010) [2023-12-27 06:59:27,219][20381] Updated weights for policy 1, policy_version 192959 (0.0010) [2023-12-27 06:59:27,275][20381] Updated weights for policy 1, policy_version 192969 (0.0011) [2023-12-27 06:59:27,328][20381] Updated weights for policy 1, policy_version 192979 (0.0011) [2023-12-27 06:59:27,353][20347] Updated weights for policy 0, policy_version 193227 (0.0010) [2023-12-27 06:59:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 98893824. Throughput: 0: 10372.7, 1: 10276.0. Samples: 98880580. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '678.130')] [2023-12-27 06:59:27,407][20347] Updated weights for policy 0, policy_version 193237 (0.0010) [2023-12-27 06:59:27,458][20347] Updated weights for policy 0, policy_version 193247 (0.0010) [2023-12-27 06:59:28,033][20381] Updated weights for policy 1, policy_version 192989 (0.0011) [2023-12-27 06:59:28,085][20381] Updated weights for policy 1, policy_version 192999 (0.0011) [2023-12-27 06:59:28,138][20381] Updated weights for policy 1, policy_version 193009 (0.0009) [2023-12-27 06:59:28,140][20347] Updated weights for policy 0, policy_version 193257 (0.0010) [2023-12-27 06:59:28,197][20347] Updated weights for policy 0, policy_version 193267 (0.0010) [2023-12-27 06:59:28,258][20347] Updated weights for policy 0, policy_version 193277 (0.0009) [2023-12-27 06:59:28,316][20347] Updated weights for policy 0, policy_version 193287 (0.0007) [2023-12-27 06:59:28,758][20381] Updated weights for policy 1, policy_version 193019 (0.0006) [2023-12-27 06:59:28,814][20381] Updated weights for policy 1, policy_version 193029 (0.0008) [2023-12-27 06:59:28,873][20381] Updated weights for policy 1, policy_version 193039 (0.0008) [2023-12-27 06:59:29,016][20347] Updated weights for policy 0, policy_version 193297 (0.0007) [2023-12-27 06:59:29,064][20347] Updated weights for policy 0, policy_version 193307 (0.0005) [2023-12-27 06:59:29,127][20347] Updated weights for policy 0, policy_version 193317 (0.0005) [2023-12-27 06:59:29,571][20381] Updated weights for policy 1, policy_version 193049 (0.0007) [2023-12-27 06:59:29,638][20381] Updated weights for policy 1, policy_version 193059 (0.0007) [2023-12-27 06:59:29,693][20381] Updated weights for policy 1, policy_version 193069 (0.0006) [2023-12-27 06:59:29,752][20381] Updated weights for policy 1, policy_version 193079 (0.0006) [2023-12-27 06:59:29,758][20347] Updated weights for policy 0, policy_version 193327 (0.0007) [2023-12-27 06:59:29,817][20347] Updated weights for policy 0, policy_version 193337 (0.0008) [2023-12-27 06:59:29,880][20347] Updated weights for policy 0, policy_version 193347 (0.0007) [2023-12-27 06:59:30,353][20381] Updated weights for policy 1, policy_version 193089 (0.0008) [2023-12-27 06:59:30,402][20381] Updated weights for policy 1, policy_version 193099 (0.0008) [2023-12-27 06:59:30,461][20381] Updated weights for policy 1, policy_version 193109 (0.0006) [2023-12-27 06:59:30,523][20347] Updated weights for policy 0, policy_version 193357 (0.0006) [2023-12-27 06:59:30,574][20347] Updated weights for policy 0, policy_version 193367 (0.0008) [2023-12-27 06:59:30,625][20347] Updated weights for policy 0, policy_version 193377 (0.0008) [2023-12-27 06:59:31,033][20381] Updated weights for policy 1, policy_version 193119 (0.0007) [2023-12-27 06:59:31,090][20381] Updated weights for policy 1, policy_version 193129 (0.0007) [2023-12-27 06:59:31,151][20381] Updated weights for policy 1, policy_version 193139 (0.0008) [2023-12-27 06:59:31,352][20347] Updated weights for policy 0, policy_version 193387 (0.0009) [2023-12-27 06:59:31,412][20347] Updated weights for policy 0, policy_version 193397 (0.0010) [2023-12-27 06:59:31,467][20347] Updated weights for policy 0, policy_version 193407 (0.0010) [2023-12-27 06:59:31,854][20381] Updated weights for policy 1, policy_version 193149 (0.0007) [2023-12-27 06:59:31,900][20381] Updated weights for policy 1, policy_version 193159 (0.0007) [2023-12-27 06:59:31,958][20381] Updated weights for policy 1, policy_version 193169 (0.0005) [2023-12-27 06:59:32,169][20347] Updated weights for policy 0, policy_version 193417 (0.0010) [2023-12-27 06:59:32,222][20347] Updated weights for policy 0, policy_version 193427 (0.0008) [2023-12-27 06:59:32,284][20347] Updated weights for policy 0, policy_version 193437 (0.0010) [2023-12-27 06:59:32,346][20347] Updated weights for policy 0, policy_version 193447 (0.0008) [2023-12-27 06:59:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 99000320. Throughput: 0: 10419.9, 1: 10306.6. Samples: 99006936. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '324.130')] [2023-12-27 06:59:32,628][20381] Updated weights for policy 1, policy_version 193179 (0.0006) [2023-12-27 06:59:32,674][20381] Updated weights for policy 1, policy_version 193189 (0.0005) [2023-12-27 06:59:32,723][20381] Updated weights for policy 1, policy_version 193199 (0.0007) [2023-12-27 06:59:33,009][20347] Updated weights for policy 0, policy_version 193457 (0.0005) [2023-12-27 06:59:33,065][20347] Updated weights for policy 0, policy_version 193467 (0.0005) [2023-12-27 06:59:33,125][20347] Updated weights for policy 0, policy_version 193477 (0.0006) [2023-12-27 06:59:33,359][20381] Updated weights for policy 1, policy_version 193209 (0.0007) [2023-12-27 06:59:33,423][20381] Updated weights for policy 1, policy_version 193219 (0.0005) [2023-12-27 06:59:33,485][20381] Updated weights for policy 1, policy_version 193229 (0.0005) [2023-12-27 06:59:33,544][20381] Updated weights for policy 1, policy_version 193239 (0.0005) [2023-12-27 06:59:33,799][20347] Updated weights for policy 0, policy_version 193487 (0.0007) [2023-12-27 06:59:33,847][20347] Updated weights for policy 0, policy_version 193497 (0.0009) [2023-12-27 06:59:33,892][20347] Updated weights for policy 0, policy_version 193507 (0.0009) [2023-12-27 06:59:34,081][20381] Updated weights for policy 1, policy_version 193249 (0.0006) [2023-12-27 06:59:34,128][20381] Updated weights for policy 1, policy_version 193259 (0.0007) [2023-12-27 06:59:34,192][20381] Updated weights for policy 1, policy_version 193269 (0.0008) [2023-12-27 06:59:34,651][20347] Updated weights for policy 0, policy_version 193517 (0.0008) [2023-12-27 06:59:34,714][20347] Updated weights for policy 0, policy_version 193527 (0.0007) [2023-12-27 06:59:34,778][20347] Updated weights for policy 0, policy_version 193537 (0.0008) [2023-12-27 06:59:34,938][20381] Updated weights for policy 1, policy_version 193279 (0.0007) [2023-12-27 06:59:34,996][20381] Updated weights for policy 1, policy_version 193289 (0.0006) [2023-12-27 06:59:35,051][20381] Updated weights for policy 1, policy_version 193299 (0.0006) [2023-12-27 06:59:35,380][20347] Updated weights for policy 0, policy_version 193547 (0.0007) [2023-12-27 06:59:35,434][20347] Updated weights for policy 0, policy_version 193557 (0.0005) [2023-12-27 06:59:35,483][20347] Updated weights for policy 0, policy_version 193567 (0.0005) [2023-12-27 06:59:35,672][20381] Updated weights for policy 1, policy_version 193309 (0.0008) [2023-12-27 06:59:35,730][20381] Updated weights for policy 1, policy_version 193319 (0.0007) [2023-12-27 06:59:35,784][20381] Updated weights for policy 1, policy_version 193329 (0.0007) [2023-12-27 06:59:36,029][20347] Updated weights for policy 0, policy_version 193577 (0.0005) [2023-12-27 06:59:36,091][20347] Updated weights for policy 0, policy_version 193587 (0.0005) [2023-12-27 06:59:36,161][20347] Updated weights for policy 0, policy_version 193597 (0.0006) [2023-12-27 06:59:36,226][20347] Updated weights for policy 0, policy_version 193607 (0.0006) [2023-12-27 06:59:36,460][20381] Updated weights for policy 1, policy_version 193339 (0.0006) [2023-12-27 06:59:36,519][20381] Updated weights for policy 1, policy_version 193349 (0.0008) [2023-12-27 06:59:36,580][20381] Updated weights for policy 1, policy_version 193359 (0.0008) [2023-12-27 06:59:36,833][20347] Updated weights for policy 0, policy_version 193617 (0.0007) [2023-12-27 06:59:36,885][20347] Updated weights for policy 0, policy_version 193627 (0.0005) [2023-12-27 06:59:36,944][20347] Updated weights for policy 0, policy_version 193637 (0.0006) [2023-12-27 06:59:37,306][20381] Updated weights for policy 1, policy_version 193369 (0.0007) [2023-12-27 06:59:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 99098624. Throughput: 0: 10428.7, 1: 10350.4. Samples: 99068804. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '322.750')] [2023-12-27 06:59:37,364][20381] Updated weights for policy 1, policy_version 193379 (0.0007) [2023-12-27 06:59:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000193640_49586176.pth... [2023-12-27 06:59:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000192424_49274880.pth [2023-12-27 06:59:37,430][20381] Updated weights for policy 1, policy_version 193389 (0.0008) [2023-12-27 06:59:37,495][20381] Updated weights for policy 1, policy_version 193399 (0.0007) [2023-12-27 06:59:37,500][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000193400_49520640.pth... [2023-12-27 06:59:37,505][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000192184_49209344.pth [2023-12-27 06:59:37,550][20347] Updated weights for policy 0, policy_version 193647 (0.0007) [2023-12-27 06:59:37,604][20347] Updated weights for policy 0, policy_version 193657 (0.0008) [2023-12-27 06:59:37,659][20347] Updated weights for policy 0, policy_version 193667 (0.0007) [2023-12-27 06:59:38,134][20381] Updated weights for policy 1, policy_version 193409 (0.0007) [2023-12-27 06:59:38,185][20381] Updated weights for policy 1, policy_version 193419 (0.0005) [2023-12-27 06:59:38,236][20381] Updated weights for policy 1, policy_version 193429 (0.0005) [2023-12-27 06:59:38,370][20347] Updated weights for policy 0, policy_version 193677 (0.0007) [2023-12-27 06:59:38,435][20347] Updated weights for policy 0, policy_version 193687 (0.0008) [2023-12-27 06:59:38,498][20347] Updated weights for policy 0, policy_version 193697 (0.0008) [2023-12-27 06:59:38,911][20381] Updated weights for policy 1, policy_version 193439 (0.0008) [2023-12-27 06:59:38,972][20381] Updated weights for policy 1, policy_version 193449 (0.0008) [2023-12-27 06:59:39,031][20381] Updated weights for policy 1, policy_version 193459 (0.0008) [2023-12-27 06:59:39,117][20347] Updated weights for policy 0, policy_version 193707 (0.0008) [2023-12-27 06:59:39,177][20347] Updated weights for policy 0, policy_version 193717 (0.0005) [2023-12-27 06:59:39,230][20347] Updated weights for policy 0, policy_version 193727 (0.0006) [2023-12-27 06:59:39,741][20381] Updated weights for policy 1, policy_version 193469 (0.0008) [2023-12-27 06:59:39,799][20381] Updated weights for policy 1, policy_version 193479 (0.0007) [2023-12-27 06:59:39,819][20347] Updated weights for policy 0, policy_version 193737 (0.0005) [2023-12-27 06:59:39,865][20381] Updated weights for policy 1, policy_version 193489 (0.0007) [2023-12-27 06:59:39,878][20347] Updated weights for policy 0, policy_version 193747 (0.0007) [2023-12-27 06:59:39,930][20347] Updated weights for policy 0, policy_version 193757 (0.0007) [2023-12-27 06:59:39,997][20347] Updated weights for policy 0, policy_version 193767 (0.0008) [2023-12-27 06:59:40,574][20381] Updated weights for policy 1, policy_version 193499 (0.0007) [2023-12-27 06:59:40,633][20381] Updated weights for policy 1, policy_version 193509 (0.0010) [2023-12-27 06:59:40,672][20347] Updated weights for policy 0, policy_version 193777 (0.0006) [2023-12-27 06:59:40,686][20381] Updated weights for policy 1, policy_version 193519 (0.0010) [2023-12-27 06:59:40,731][20347] Updated weights for policy 0, policy_version 193787 (0.0005) [2023-12-27 06:59:40,792][20347] Updated weights for policy 0, policy_version 193797 (0.0005) [2023-12-27 06:59:41,400][20381] Updated weights for policy 1, policy_version 193529 (0.0010) [2023-12-27 06:59:41,405][20347] Updated weights for policy 0, policy_version 193807 (0.0007) [2023-12-27 06:59:41,458][20347] Updated weights for policy 0, policy_version 193817 (0.0006) [2023-12-27 06:59:41,459][20381] Updated weights for policy 1, policy_version 193539 (0.0008) [2023-12-27 06:59:41,508][20347] Updated weights for policy 0, policy_version 193827 (0.0008) [2023-12-27 06:59:41,510][20381] Updated weights for policy 1, policy_version 193549 (0.0008) [2023-12-27 06:59:41,574][20381] Updated weights for policy 1, policy_version 193559 (0.0008) [2023-12-27 06:59:42,177][20347] Updated weights for policy 0, policy_version 193837 (0.0008) [2023-12-27 06:59:42,227][20347] Updated weights for policy 0, policy_version 193847 (0.0006) [2023-12-27 06:59:42,240][20381] Updated weights for policy 1, policy_version 193569 (0.0007) [2023-12-27 06:59:42,286][20347] Updated weights for policy 0, policy_version 193857 (0.0007) [2023-12-27 06:59:42,300][20381] Updated weights for policy 1, policy_version 193579 (0.0007) [2023-12-27 06:59:42,360][20381] Updated weights for policy 1, policy_version 193589 (0.0007) [2023-12-27 06:59:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 99205120. Throughput: 0: 10489.6, 1: 10359.9. Samples: 99195200. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '485.560')] [2023-12-27 06:59:42,997][20347] Updated weights for policy 0, policy_version 193867 (0.0007) [2023-12-27 06:59:43,008][20381] Updated weights for policy 1, policy_version 193599 (0.0007) [2023-12-27 06:59:43,053][20347] Updated weights for policy 0, policy_version 193877 (0.0006) [2023-12-27 06:59:43,058][20381] Updated weights for policy 1, policy_version 193609 (0.0006) [2023-12-27 06:59:43,106][20347] Updated weights for policy 0, policy_version 193887 (0.0005) [2023-12-27 06:59:43,108][20381] Updated weights for policy 1, policy_version 193619 (0.0005) [2023-12-27 06:59:43,747][20347] Updated weights for policy 0, policy_version 193897 (0.0006) [2023-12-27 06:59:43,796][20381] Updated weights for policy 1, policy_version 193629 (0.0006) [2023-12-27 06:59:43,801][20347] Updated weights for policy 0, policy_version 193907 (0.0008) [2023-12-27 06:59:43,852][20347] Updated weights for policy 0, policy_version 193917 (0.0006) [2023-12-27 06:59:43,854][20381] Updated weights for policy 1, policy_version 193639 (0.0006) [2023-12-27 06:59:43,909][20381] Updated weights for policy 1, policy_version 193649 (0.0005) [2023-12-27 06:59:43,917][20347] Updated weights for policy 0, policy_version 193927 (0.0008) [2023-12-27 06:59:44,590][20347] Updated weights for policy 0, policy_version 193937 (0.0008) [2023-12-27 06:59:44,604][20381] Updated weights for policy 1, policy_version 193659 (0.0008) [2023-12-27 06:59:44,638][20347] Updated weights for policy 0, policy_version 193947 (0.0007) [2023-12-27 06:59:44,646][20381] Updated weights for policy 1, policy_version 193669 (0.0007) [2023-12-27 06:59:44,693][20347] Updated weights for policy 0, policy_version 193957 (0.0006) [2023-12-27 06:59:44,698][20381] Updated weights for policy 1, policy_version 193679 (0.0007) [2023-12-27 06:59:45,410][20347] Updated weights for policy 0, policy_version 193967 (0.0008) [2023-12-27 06:59:45,459][20381] Updated weights for policy 1, policy_version 193689 (0.0008) [2023-12-27 06:59:45,462][20347] Updated weights for policy 0, policy_version 193977 (0.0007) [2023-12-27 06:59:45,514][20347] Updated weights for policy 0, policy_version 193987 (0.0007) [2023-12-27 06:59:45,516][20381] Updated weights for policy 1, policy_version 193699 (0.0007) [2023-12-27 06:59:45,569][20381] Updated weights for policy 1, policy_version 193709 (0.0006) [2023-12-27 06:59:45,628][20381] Updated weights for policy 1, policy_version 193719 (0.0008) [2023-12-27 06:59:46,205][20347] Updated weights for policy 0, policy_version 193997 (0.0005) [2023-12-27 06:59:46,253][20347] Updated weights for policy 0, policy_version 194007 (0.0005) [2023-12-27 06:59:46,301][20381] Updated weights for policy 1, policy_version 193729 (0.0009) [2023-12-27 06:59:46,314][20347] Updated weights for policy 0, policy_version 194017 (0.0007) [2023-12-27 06:59:46,359][20381] Updated weights for policy 1, policy_version 193739 (0.0008) [2023-12-27 06:59:46,413][20381] Updated weights for policy 1, policy_version 193749 (0.0009) [2023-12-27 06:59:46,951][20347] Updated weights for policy 0, policy_version 194027 (0.0006) [2023-12-27 06:59:46,996][20347] Updated weights for policy 0, policy_version 194037 (0.0007) [2023-12-27 06:59:47,050][20347] Updated weights for policy 0, policy_version 194047 (0.0007) [2023-12-27 06:59:47,112][20381] Updated weights for policy 1, policy_version 193759 (0.0007) [2023-12-27 06:59:47,166][20381] Updated weights for policy 1, policy_version 193769 (0.0008) [2023-12-27 06:59:47,214][20381] Updated weights for policy 1, policy_version 193779 (0.0007) [2023-12-27 06:59:47,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 99311616. Throughput: 0: 10495.4, 1: 10330.6. Samples: 99317308. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:47,363][19170] Avg episode reward: [(0, '985.930'), (1, '714.700')] [2023-12-27 06:59:47,711][20347] Updated weights for policy 0, policy_version 194057 (0.0007) [2023-12-27 06:59:47,764][20347] Updated weights for policy 0, policy_version 194067 (0.0008) [2023-12-27 06:59:47,764][20291] KL-divergence is very high: 435.6508 [2023-12-27 06:59:47,801][20291] KL-divergence is very high: 223.5194 [2023-12-27 06:59:47,812][20347] Updated weights for policy 0, policy_version 194077 (0.0008) [2023-12-27 06:59:47,867][20347] Updated weights for policy 0, policy_version 194087 (0.0007) [2023-12-27 06:59:47,916][20381] Updated weights for policy 1, policy_version 193789 (0.0009) [2023-12-27 06:59:47,971][20381] Updated weights for policy 1, policy_version 193799 (0.0007) [2023-12-27 06:59:48,024][20381] Updated weights for policy 1, policy_version 193809 (0.0008) [2023-12-27 06:59:48,579][20347] Updated weights for policy 0, policy_version 194097 (0.0006) [2023-12-27 06:59:48,639][20347] Updated weights for policy 0, policy_version 194107 (0.0011) [2023-12-27 06:59:48,701][20347] Updated weights for policy 0, policy_version 194117 (0.0010) [2023-12-27 06:59:48,778][20381] Updated weights for policy 1, policy_version 193819 (0.0008) [2023-12-27 06:59:48,843][20381] Updated weights for policy 1, policy_version 193829 (0.0008) [2023-12-27 06:59:48,898][20381] Updated weights for policy 1, policy_version 193839 (0.0008) [2023-12-27 06:59:49,370][20347] Updated weights for policy 0, policy_version 194127 (0.0008) [2023-12-27 06:59:49,427][20347] Updated weights for policy 0, policy_version 194137 (0.0007) [2023-12-27 06:59:49,476][20347] Updated weights for policy 0, policy_version 194147 (0.0008) [2023-12-27 06:59:49,555][20381] Updated weights for policy 1, policy_version 193849 (0.0007) [2023-12-27 06:59:49,609][20381] Updated weights for policy 1, policy_version 193859 (0.0008) [2023-12-27 06:59:49,659][20381] Updated weights for policy 1, policy_version 193869 (0.0009) [2023-12-27 06:59:49,720][20381] Updated weights for policy 1, policy_version 193879 (0.0008) [2023-12-27 06:59:50,075][20347] Updated weights for policy 0, policy_version 194157 (0.0008) [2023-12-27 06:59:50,129][20347] Updated weights for policy 0, policy_version 194167 (0.0007) [2023-12-27 06:59:50,187][20347] Updated weights for policy 0, policy_version 194177 (0.0010) [2023-12-27 06:59:50,469][20381] Updated weights for policy 1, policy_version 193889 (0.0009) [2023-12-27 06:59:50,530][20381] Updated weights for policy 1, policy_version 193899 (0.0007) [2023-12-27 06:59:50,591][20381] Updated weights for policy 1, policy_version 193909 (0.0007) [2023-12-27 06:59:50,874][20347] Updated weights for policy 0, policy_version 194187 (0.0009) [2023-12-27 06:59:50,937][20347] Updated weights for policy 0, policy_version 194197 (0.0005) [2023-12-27 06:59:51,001][20347] Updated weights for policy 0, policy_version 194207 (0.0006) [2023-12-27 06:59:51,299][20381] Updated weights for policy 1, policy_version 193919 (0.0008) [2023-12-27 06:59:51,363][20381] Updated weights for policy 1, policy_version 193929 (0.0007) [2023-12-27 06:59:51,425][20381] Updated weights for policy 1, policy_version 193939 (0.0007) [2023-12-27 06:59:51,651][20347] Updated weights for policy 0, policy_version 194217 (0.0009) [2023-12-27 06:59:51,718][20347] Updated weights for policy 0, policy_version 194227 (0.0010) [2023-12-27 06:59:51,778][20347] Updated weights for policy 0, policy_version 194237 (0.0006) [2023-12-27 06:59:51,861][20347] Updated weights for policy 0, policy_version 194247 (0.0009) [2023-12-27 06:59:52,146][20381] Updated weights for policy 1, policy_version 193949 (0.0008) [2023-12-27 06:59:52,203][20381] Updated weights for policy 1, policy_version 193959 (0.0008) [2023-12-27 06:59:52,265][20381] Updated weights for policy 1, policy_version 193969 (0.0008) [2023-12-27 06:59:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 99409920. Throughput: 0: 10511.2, 1: 10292.7. Samples: 99378532. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:52,363][19170] Avg episode reward: [(0, '982.980'), (1, '874.670')] [2023-12-27 06:59:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000193976_49668096.pth... [2023-12-27 06:59:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000194248_49741824.pth... [2023-12-27 06:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000192760_49356800.pth [2023-12-27 06:59:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000193976_49668096.pth [2023-12-27 06:59:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000193032_49430528.pth [2023-12-27 06:59:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000194248_49741824.pth [2023-12-27 06:59:52,464][20347] Updated weights for policy 0, policy_version 194257 (0.0007) [2023-12-27 06:59:52,513][20347] Updated weights for policy 0, policy_version 194267 (0.0010) [2023-12-27 06:59:52,562][20347] Updated weights for policy 0, policy_version 194277 (0.0006) [2023-12-27 06:59:53,002][20381] Updated weights for policy 1, policy_version 193979 (0.0007) [2023-12-27 06:59:53,061][20381] Updated weights for policy 1, policy_version 193989 (0.0005) [2023-12-27 06:59:53,121][20381] Updated weights for policy 1, policy_version 193999 (0.0008) [2023-12-27 06:59:53,234][20347] Updated weights for policy 0, policy_version 194287 (0.0006) [2023-12-27 06:59:53,287][20347] Updated weights for policy 0, policy_version 194297 (0.0006) [2023-12-27 06:59:53,333][20347] Updated weights for policy 0, policy_version 194307 (0.0005) [2023-12-27 06:59:53,819][20381] Updated weights for policy 1, policy_version 194009 (0.0007) [2023-12-27 06:59:53,869][20381] Updated weights for policy 1, policy_version 194019 (0.0008) [2023-12-27 06:59:53,921][20381] Updated weights for policy 1, policy_version 194029 (0.0008) [2023-12-27 06:59:53,955][20347] Updated weights for policy 0, policy_version 194317 (0.0006) [2023-12-27 06:59:53,979][20381] Updated weights for policy 1, policy_version 194039 (0.0005) [2023-12-27 06:59:54,011][20347] Updated weights for policy 0, policy_version 194327 (0.0006) [2023-12-27 06:59:54,062][20347] Updated weights for policy 0, policy_version 194337 (0.0007) [2023-12-27 06:59:54,665][20381] Updated weights for policy 1, policy_version 194049 (0.0008) [2023-12-27 06:59:54,709][20381] Updated weights for policy 1, policy_version 194059 (0.0009) [2023-12-27 06:59:54,734][20347] Updated weights for policy 0, policy_version 194347 (0.0007) [2023-12-27 06:59:54,762][20381] Updated weights for policy 1, policy_version 194069 (0.0006) [2023-12-27 06:59:54,793][20347] Updated weights for policy 0, policy_version 194357 (0.0006) [2023-12-27 06:59:54,852][20347] Updated weights for policy 0, policy_version 194367 (0.0008) [2023-12-27 06:59:55,488][20381] Updated weights for policy 1, policy_version 194079 (0.0008) [2023-12-27 06:59:55,497][20347] Updated weights for policy 0, policy_version 194377 (0.0007) [2023-12-27 06:59:55,541][20381] Updated weights for policy 1, policy_version 194089 (0.0006) [2023-12-27 06:59:55,543][20347] Updated weights for policy 0, policy_version 194387 (0.0006) [2023-12-27 06:59:55,587][20381] Updated weights for policy 1, policy_version 194099 (0.0006) [2023-12-27 06:59:55,589][20347] Updated weights for policy 0, policy_version 194397 (0.0006) [2023-12-27 06:59:55,633][20347] Updated weights for policy 0, policy_version 194407 (0.0006) [2023-12-27 06:59:56,207][20347] Updated weights for policy 0, policy_version 194417 (0.0005) [2023-12-27 06:59:56,254][20347] Updated weights for policy 0, policy_version 194427 (0.0005) [2023-12-27 06:59:56,307][20347] Updated weights for policy 0, policy_version 194437 (0.0005) [2023-12-27 06:59:56,325][20381] Updated weights for policy 1, policy_version 194109 (0.0006) [2023-12-27 06:59:56,387][20381] Updated weights for policy 1, policy_version 194119 (0.0006) [2023-12-27 06:59:56,438][20381] Updated weights for policy 1, policy_version 194129 (0.0009) [2023-12-27 06:59:56,935][20347] Updated weights for policy 0, policy_version 194447 (0.0006) [2023-12-27 06:59:56,983][20347] Updated weights for policy 0, policy_version 194457 (0.0007) [2023-12-27 06:59:57,030][20347] Updated weights for policy 0, policy_version 194467 (0.0007) [2023-12-27 06:59:57,066][20381] Updated weights for policy 1, policy_version 194139 (0.0008) [2023-12-27 06:59:57,112][20381] Updated weights for policy 1, policy_version 194149 (0.0008) [2023-12-27 06:59:57,167][20381] Updated weights for policy 1, policy_version 194159 (0.0008) [2023-12-27 06:59:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 99516416. Throughput: 0: 10494.5, 1: 10198.8. Samples: 99500960. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 06:59:57,363][19170] Avg episode reward: [(0, '999.320'), (1, '922.790')] [2023-12-27 06:59:57,693][20347] Updated weights for policy 0, policy_version 194477 (0.0006) [2023-12-27 06:59:57,755][20347] Updated weights for policy 0, policy_version 194487 (0.0005) [2023-12-27 06:59:57,819][20347] Updated weights for policy 0, policy_version 194497 (0.0006) [2023-12-27 06:59:57,866][20381] Updated weights for policy 1, policy_version 194169 (0.0008) [2023-12-27 06:59:57,922][20381] Updated weights for policy 1, policy_version 194179 (0.0008) [2023-12-27 06:59:57,974][20381] Updated weights for policy 1, policy_version 194189 (0.0007) [2023-12-27 06:59:58,029][20381] Updated weights for policy 1, policy_version 194199 (0.0007) [2023-12-27 06:59:58,481][20347] Updated weights for policy 0, policy_version 194507 (0.0007) [2023-12-27 06:59:58,541][20347] Updated weights for policy 0, policy_version 194517 (0.0008) [2023-12-27 06:59:58,603][20347] Updated weights for policy 0, policy_version 194527 (0.0008) [2023-12-27 06:59:58,775][20381] Updated weights for policy 1, policy_version 194209 (0.0008) [2023-12-27 06:59:58,831][20381] Updated weights for policy 1, policy_version 194219 (0.0008) [2023-12-27 06:59:58,882][20381] Updated weights for policy 1, policy_version 194229 (0.0008) [2023-12-27 06:59:59,284][20347] Updated weights for policy 0, policy_version 194537 (0.0008) [2023-12-27 06:59:59,343][20347] Updated weights for policy 0, policy_version 194547 (0.0007) [2023-12-27 06:59:59,411][20347] Updated weights for policy 0, policy_version 194557 (0.0008) [2023-12-27 06:59:59,471][20347] Updated weights for policy 0, policy_version 194567 (0.0008) [2023-12-27 06:59:59,565][20381] Updated weights for policy 1, policy_version 194239 (0.0010) [2023-12-27 06:59:59,624][20381] Updated weights for policy 1, policy_version 194249 (0.0011) [2023-12-27 06:59:59,674][20381] Updated weights for policy 1, policy_version 194259 (0.0010) [2023-12-27 07:00:00,127][20347] Updated weights for policy 0, policy_version 194577 (0.0008) [2023-12-27 07:00:00,179][20347] Updated weights for policy 0, policy_version 194587 (0.0008) [2023-12-27 07:00:00,236][20347] Updated weights for policy 0, policy_version 194597 (0.0008) [2023-12-27 07:00:00,384][20381] Updated weights for policy 1, policy_version 194269 (0.0008) [2023-12-27 07:00:00,448][20381] Updated weights for policy 1, policy_version 194279 (0.0008) [2023-12-27 07:00:00,495][20381] Updated weights for policy 1, policy_version 194289 (0.0010) [2023-12-27 07:00:00,886][20347] Updated weights for policy 0, policy_version 194607 (0.0007) [2023-12-27 07:00:00,938][20347] Updated weights for policy 0, policy_version 194617 (0.0005) [2023-12-27 07:00:00,999][20347] Updated weights for policy 0, policy_version 194627 (0.0007) [2023-12-27 07:00:01,141][20381] Updated weights for policy 1, policy_version 194299 (0.0007) [2023-12-27 07:00:01,205][20381] Updated weights for policy 1, policy_version 194309 (0.0008) [2023-12-27 07:00:01,263][20381] Updated weights for policy 1, policy_version 194319 (0.0008) [2023-12-27 07:00:01,739][20347] Updated weights for policy 0, policy_version 194637 (0.0007) [2023-12-27 07:00:01,798][20347] Updated weights for policy 0, policy_version 194647 (0.0010) [2023-12-27 07:00:01,855][20347] Updated weights for policy 0, policy_version 194657 (0.0008) [2023-12-27 07:00:01,916][20381] Updated weights for policy 1, policy_version 194329 (0.0007) [2023-12-27 07:00:01,980][20381] Updated weights for policy 1, policy_version 194339 (0.0007) [2023-12-27 07:00:02,036][20381] Updated weights for policy 1, policy_version 194349 (0.0007) [2023-12-27 07:00:02,094][20381] Updated weights for policy 1, policy_version 194359 (0.0008) [2023-12-27 07:00:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 99614720. Throughput: 0: 10476.9, 1: 10161.9. Samples: 99624032. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 07:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '962.310')] [2023-12-27 07:00:02,473][20347] Updated weights for policy 0, policy_version 194667 (0.0006) [2023-12-27 07:00:02,524][20347] Updated weights for policy 0, policy_version 194677 (0.0007) [2023-12-27 07:00:02,578][20347] Updated weights for policy 0, policy_version 194687 (0.0008) [2023-12-27 07:00:02,780][20381] Updated weights for policy 1, policy_version 194369 (0.0006) [2023-12-27 07:00:02,837][20381] Updated weights for policy 1, policy_version 194379 (0.0005) [2023-12-27 07:00:02,895][20381] Updated weights for policy 1, policy_version 194389 (0.0009) [2023-12-27 07:00:03,148][20347] Updated weights for policy 0, policy_version 194697 (0.0007) [2023-12-27 07:00:03,200][20347] Updated weights for policy 0, policy_version 194707 (0.0005) [2023-12-27 07:00:03,248][20347] Updated weights for policy 0, policy_version 194717 (0.0008) [2023-12-27 07:00:03,293][20347] Updated weights for policy 0, policy_version 194727 (0.0010) [2023-12-27 07:00:03,502][20381] Updated weights for policy 1, policy_version 194399 (0.0007) [2023-12-27 07:00:03,555][20381] Updated weights for policy 1, policy_version 194409 (0.0006) [2023-12-27 07:00:03,612][20381] Updated weights for policy 1, policy_version 194419 (0.0005) [2023-12-27 07:00:03,979][20347] Updated weights for policy 0, policy_version 194737 (0.0011) [2023-12-27 07:00:04,047][20347] Updated weights for policy 0, policy_version 194747 (0.0011) [2023-12-27 07:00:04,113][20347] Updated weights for policy 0, policy_version 194757 (0.0010) [2023-12-27 07:00:04,192][20381] Updated weights for policy 1, policy_version 194429 (0.0005) [2023-12-27 07:00:04,256][20381] Updated weights for policy 1, policy_version 194439 (0.0007) [2023-12-27 07:00:04,319][20381] Updated weights for policy 1, policy_version 194449 (0.0007) [2023-12-27 07:00:04,799][20347] Updated weights for policy 0, policy_version 194767 (0.0009) [2023-12-27 07:00:04,850][20347] Updated weights for policy 0, policy_version 194777 (0.0010) [2023-12-27 07:00:04,906][20347] Updated weights for policy 0, policy_version 194787 (0.0010) [2023-12-27 07:00:04,948][20381] Updated weights for policy 1, policy_version 194459 (0.0007) [2023-12-27 07:00:05,002][20381] Updated weights for policy 1, policy_version 194469 (0.0005) [2023-12-27 07:00:05,058][20381] Updated weights for policy 1, policy_version 194479 (0.0007) [2023-12-27 07:00:05,521][20347] Updated weights for policy 0, policy_version 194797 (0.0008) [2023-12-27 07:00:05,586][20347] Updated weights for policy 0, policy_version 194807 (0.0005) [2023-12-27 07:00:05,625][20381] Updated weights for policy 1, policy_version 194489 (0.0005) [2023-12-27 07:00:05,654][20347] Updated weights for policy 0, policy_version 194817 (0.0006) [2023-12-27 07:00:05,692][20381] Updated weights for policy 1, policy_version 194499 (0.0007) [2023-12-27 07:00:05,744][20381] Updated weights for policy 1, policy_version 194509 (0.0008) [2023-12-27 07:00:05,796][20381] Updated weights for policy 1, policy_version 194519 (0.0011) [2023-12-27 07:00:06,215][20347] Updated weights for policy 0, policy_version 194827 (0.0008) [2023-12-27 07:00:06,273][20347] Updated weights for policy 0, policy_version 194837 (0.0008) [2023-12-27 07:00:06,333][20347] Updated weights for policy 0, policy_version 194847 (0.0008) [2023-12-27 07:00:06,454][20381] Updated weights for policy 1, policy_version 194529 (0.0008) [2023-12-27 07:00:06,513][20381] Updated weights for policy 1, policy_version 194539 (0.0008) [2023-12-27 07:00:06,575][20381] Updated weights for policy 1, policy_version 194549 (0.0006) [2023-12-27 07:00:07,059][20347] Updated weights for policy 0, policy_version 194857 (0.0007) [2023-12-27 07:00:07,114][20347] Updated weights for policy 0, policy_version 194867 (0.0006) [2023-12-27 07:00:07,173][20347] Updated weights for policy 0, policy_version 194877 (0.0006) [2023-12-27 07:00:07,225][20381] Updated weights for policy 1, policy_version 194559 (0.0006) [2023-12-27 07:00:07,227][20347] Updated weights for policy 0, policy_version 194887 (0.0006) [2023-12-27 07:00:07,274][20381] Updated weights for policy 1, policy_version 194569 (0.0008) [2023-12-27 07:00:07,320][20381] Updated weights for policy 1, policy_version 194579 (0.0009) [2023-12-27 07:00:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 99729408. Throughput: 0: 10533.9, 1: 10192.8. Samples: 99689108. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 07:00:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.820')] [2023-12-27 07:00:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000194584_49823744.pth... [2023-12-27 07:00:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000194888_49905664.pth... [2023-12-27 07:00:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000193400_49520640.pth [2023-12-27 07:00:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000193640_49586176.pth [2023-12-27 07:00:07,844][20347] Updated weights for policy 0, policy_version 194897 (0.0005) [2023-12-27 07:00:07,904][20347] Updated weights for policy 0, policy_version 194907 (0.0005) [2023-12-27 07:00:07,945][20381] Updated weights for policy 1, policy_version 194589 (0.0006) [2023-12-27 07:00:07,962][20347] Updated weights for policy 0, policy_version 194917 (0.0010) [2023-12-27 07:00:07,991][20381] Updated weights for policy 1, policy_version 194599 (0.0006) [2023-12-27 07:00:08,044][20381] Updated weights for policy 1, policy_version 194609 (0.0006) [2023-12-27 07:00:08,611][20347] Updated weights for policy 0, policy_version 194927 (0.0008) [2023-12-27 07:00:08,649][20381] Updated weights for policy 1, policy_version 194619 (0.0006) [2023-12-27 07:00:08,670][20347] Updated weights for policy 0, policy_version 194937 (0.0011) [2023-12-27 07:00:08,705][20381] Updated weights for policy 1, policy_version 194629 (0.0006) [2023-12-27 07:00:08,732][20347] Updated weights for policy 0, policy_version 194947 (0.0010) [2023-12-27 07:00:08,762][20381] Updated weights for policy 1, policy_version 194639 (0.0005) [2023-12-27 07:00:09,410][20347] Updated weights for policy 0, policy_version 194957 (0.0010) [2023-12-27 07:00:09,471][20347] Updated weights for policy 0, policy_version 194967 (0.0011) [2023-12-27 07:00:09,471][20381] Updated weights for policy 1, policy_version 194649 (0.0008) [2023-12-27 07:00:09,524][20347] Updated weights for policy 0, policy_version 194977 (0.0011) [2023-12-27 07:00:09,531][20381] Updated weights for policy 1, policy_version 194659 (0.0011) [2023-12-27 07:00:09,587][20381] Updated weights for policy 1, policy_version 194669 (0.0011) [2023-12-27 07:00:09,648][20381] Updated weights for policy 1, policy_version 194679 (0.0009) [2023-12-27 07:00:10,255][20347] Updated weights for policy 0, policy_version 194987 (0.0011) [2023-12-27 07:00:10,315][20347] Updated weights for policy 0, policy_version 194997 (0.0009) [2023-12-27 07:00:10,361][20381] Updated weights for policy 1, policy_version 194689 (0.0011) [2023-12-27 07:00:10,378][20347] Updated weights for policy 0, policy_version 195007 (0.0008) [2023-12-27 07:00:10,419][20381] Updated weights for policy 1, policy_version 194699 (0.0011) [2023-12-27 07:00:10,477][20381] Updated weights for policy 1, policy_version 194709 (0.0011) [2023-12-27 07:00:11,056][20347] Updated weights for policy 0, policy_version 195017 (0.0006) [2023-12-27 07:00:11,121][20347] Updated weights for policy 0, policy_version 195027 (0.0008) [2023-12-27 07:00:11,151][20381] Updated weights for policy 1, policy_version 194719 (0.0008) [2023-12-27 07:00:11,191][20347] Updated weights for policy 0, policy_version 195037 (0.0008) [2023-12-27 07:00:11,218][20381] Updated weights for policy 1, policy_version 194729 (0.0008) [2023-12-27 07:00:11,253][20347] Updated weights for policy 0, policy_version 195047 (0.0008) [2023-12-27 07:00:11,284][20381] Updated weights for policy 1, policy_version 194739 (0.0007) [2023-12-27 07:00:12,008][20381] Updated weights for policy 1, policy_version 194749 (0.0007) [2023-12-27 07:00:12,020][20347] Updated weights for policy 0, policy_version 195057 (0.0006) [2023-12-27 07:00:12,068][20381] Updated weights for policy 1, policy_version 194759 (0.0006) [2023-12-27 07:00:12,079][20347] Updated weights for policy 0, policy_version 195067 (0.0006) [2023-12-27 07:00:12,126][20381] Updated weights for policy 1, policy_version 194769 (0.0007) [2023-12-27 07:00:12,139][20347] Updated weights for policy 0, policy_version 195077 (0.0006) [2023-12-27 07:00:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 99827712. Throughput: 0: 10499.0, 1: 10256.5. Samples: 99814576. Policy #0 lag: (min: 0.0, avg: 30.2, max: 32.0) [2023-12-27 07:00:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.820')] [2023-12-27 07:00:12,819][20381] Updated weights for policy 1, policy_version 194779 (0.0009) [2023-12-27 07:00:12,824][20347] Updated weights for policy 0, policy_version 195087 (0.0007) [2023-12-27 07:00:12,874][20381] Updated weights for policy 1, policy_version 194789 (0.0011) [2023-12-27 07:00:12,876][20347] Updated weights for policy 0, policy_version 195097 (0.0005) [2023-12-27 07:00:12,926][20347] Updated weights for policy 0, policy_version 195107 (0.0005) [2023-12-27 07:00:12,928][20381] Updated weights for policy 1, policy_version 194799 (0.0010) [2023-12-27 07:00:13,541][20381] Updated weights for policy 1, policy_version 194809 (0.0010) [2023-12-27 07:00:13,570][20347] Updated weights for policy 0, policy_version 195117 (0.0007) [2023-12-27 07:00:13,593][20381] Updated weights for policy 1, policy_version 194819 (0.0006) [2023-12-27 07:00:13,616][20347] Updated weights for policy 0, policy_version 195127 (0.0009) [2023-12-27 07:00:13,639][20381] Updated weights for policy 1, policy_version 194829 (0.0008) [2023-12-27 07:00:13,673][20347] Updated weights for policy 0, policy_version 195137 (0.0007) [2023-12-27 07:00:13,687][20381] Updated weights for policy 1, policy_version 194839 (0.0010) [2023-12-27 07:00:14,328][20347] Updated weights for policy 0, policy_version 195147 (0.0008) [2023-12-27 07:00:14,365][20381] Updated weights for policy 1, policy_version 194849 (0.0011) [2023-12-27 07:00:14,387][20347] Updated weights for policy 0, policy_version 195157 (0.0011) [2023-12-27 07:00:14,422][20381] Updated weights for policy 1, policy_version 194859 (0.0011) [2023-12-27 07:00:14,446][20347] Updated weights for policy 0, policy_version 195167 (0.0011) [2023-12-27 07:00:14,487][20381] Updated weights for policy 1, policy_version 194869 (0.0011) [2023-12-27 07:00:15,161][20347] Updated weights for policy 0, policy_version 195177 (0.0010) [2023-12-27 07:00:15,195][20381] Updated weights for policy 1, policy_version 194879 (0.0011) [2023-12-27 07:00:15,224][20347] Updated weights for policy 0, policy_version 195187 (0.0011) [2023-12-27 07:00:15,261][20381] Updated weights for policy 1, policy_version 194889 (0.0011) [2023-12-27 07:00:15,279][20347] Updated weights for policy 0, policy_version 195197 (0.0010) [2023-12-27 07:00:15,322][20381] Updated weights for policy 1, policy_version 194899 (0.0006) [2023-12-27 07:00:15,331][20347] Updated weights for policy 0, policy_version 195207 (0.0011) [2023-12-27 07:00:15,985][20381] Updated weights for policy 1, policy_version 194909 (0.0007) [2023-12-27 07:00:16,045][20347] Updated weights for policy 0, policy_version 195217 (0.0011) [2023-12-27 07:00:16,047][20381] Updated weights for policy 1, policy_version 194919 (0.0007) [2023-12-27 07:00:16,101][20347] Updated weights for policy 0, policy_version 195227 (0.0010) [2023-12-27 07:00:16,103][20381] Updated weights for policy 1, policy_version 194929 (0.0007) [2023-12-27 07:00:16,159][20347] Updated weights for policy 0, policy_version 195237 (0.0010) [2023-12-27 07:00:16,792][20381] Updated weights for policy 1, policy_version 194939 (0.0008) [2023-12-27 07:00:16,827][20347] Updated weights for policy 0, policy_version 195247 (0.0008) [2023-12-27 07:00:16,846][20381] Updated weights for policy 1, policy_version 194949 (0.0010) [2023-12-27 07:00:16,880][20347] Updated weights for policy 0, policy_version 195257 (0.0008) [2023-12-27 07:00:16,902][20381] Updated weights for policy 1, policy_version 194959 (0.0008) [2023-12-27 07:00:16,931][20347] Updated weights for policy 0, policy_version 195267 (0.0006) [2023-12-27 07:00:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 99926016. Throughput: 0: 10442.2, 1: 10193.1. Samples: 99935524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.000')] [2023-12-27 07:00:17,601][20381] Updated weights for policy 1, policy_version 194969 (0.0008) [2023-12-27 07:00:17,622][20347] Updated weights for policy 0, policy_version 195277 (0.0007) [2023-12-27 07:00:17,657][20381] Updated weights for policy 1, policy_version 194979 (0.0011) [2023-12-27 07:00:17,685][20347] Updated weights for policy 0, policy_version 195287 (0.0007) [2023-12-27 07:00:17,715][20381] Updated weights for policy 1, policy_version 194989 (0.0008) [2023-12-27 07:00:17,741][20347] Updated weights for policy 0, policy_version 195297 (0.0006) [2023-12-27 07:00:17,776][20381] Updated weights for policy 1, policy_version 194999 (0.0011) [2023-12-27 07:00:18,369][20347] Updated weights for policy 0, policy_version 195307 (0.0007) [2023-12-27 07:00:18,427][20347] Updated weights for policy 0, policy_version 195317 (0.0007) [2023-12-27 07:00:18,445][20381] Updated weights for policy 1, policy_version 195009 (0.0011) [2023-12-27 07:00:18,484][20347] Updated weights for policy 0, policy_version 195327 (0.0005) [2023-12-27 07:00:18,510][20381] Updated weights for policy 1, policy_version 195019 (0.0011) [2023-12-27 07:00:18,578][20381] Updated weights for policy 1, policy_version 195029 (0.0008) [2023-12-27 07:00:19,106][20347] Updated weights for policy 0, policy_version 195337 (0.0008) [2023-12-27 07:00:19,154][20347] Updated weights for policy 0, policy_version 195347 (0.0008) [2023-12-27 07:00:19,202][20347] Updated weights for policy 0, policy_version 195357 (0.0008) [2023-12-27 07:00:19,244][20381] Updated weights for policy 1, policy_version 195039 (0.0008) [2023-12-27 07:00:19,269][20347] Updated weights for policy 0, policy_version 195367 (0.0008) [2023-12-27 07:00:19,307][20381] Updated weights for policy 1, policy_version 195049 (0.0011) [2023-12-27 07:00:19,376][20381] Updated weights for policy 1, policy_version 195059 (0.0010) [2023-12-27 07:00:20,031][20347] Updated weights for policy 0, policy_version 195377 (0.0008) [2023-12-27 07:00:20,097][20347] Updated weights for policy 0, policy_version 195387 (0.0008) [2023-12-27 07:00:20,104][20381] Updated weights for policy 1, policy_version 195069 (0.0010) [2023-12-27 07:00:20,157][20347] Updated weights for policy 0, policy_version 195397 (0.0008) [2023-12-27 07:00:20,165][20381] Updated weights for policy 1, policy_version 195079 (0.0011) [2023-12-27 07:00:20,217][20381] Updated weights for policy 1, policy_version 195089 (0.0009) [2023-12-27 07:00:20,875][20347] Updated weights for policy 0, policy_version 195407 (0.0007) [2023-12-27 07:00:20,926][20381] Updated weights for policy 1, policy_version 195099 (0.0007) [2023-12-27 07:00:20,928][20347] Updated weights for policy 0, policy_version 195417 (0.0007) [2023-12-27 07:00:20,986][20347] Updated weights for policy 0, policy_version 195427 (0.0006) [2023-12-27 07:00:20,989][20381] Updated weights for policy 1, policy_version 195109 (0.0007) [2023-12-27 07:00:21,053][20381] Updated weights for policy 1, policy_version 195119 (0.0008) [2023-12-27 07:00:21,743][20347] Updated weights for policy 0, policy_version 195437 (0.0007) [2023-12-27 07:00:21,789][20381] Updated weights for policy 1, policy_version 195129 (0.0008) [2023-12-27 07:00:21,794][20347] Updated weights for policy 0, policy_version 195447 (0.0008) [2023-12-27 07:00:21,847][20347] Updated weights for policy 0, policy_version 195457 (0.0006) [2023-12-27 07:00:21,853][20381] Updated weights for policy 1, policy_version 195139 (0.0007) [2023-12-27 07:00:21,914][20381] Updated weights for policy 1, policy_version 195149 (0.0007) [2023-12-27 07:00:21,969][20381] Updated weights for policy 1, policy_version 195159 (0.0007) [2023-12-27 07:00:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 100024320. Throughput: 0: 10457.3, 1: 10137.3. Samples: 99995556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.340')] [2023-12-27 07:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000195160_49971200.pth... [2023-12-27 07:00:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000195464_50053120.pth... [2023-12-27 07:00:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000193976_49668096.pth [2023-12-27 07:00:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000194248_49741824.pth [2023-12-27 07:00:22,557][20347] Updated weights for policy 0, policy_version 195467 (0.0007) [2023-12-27 07:00:22,617][20347] Updated weights for policy 0, policy_version 195477 (0.0007) [2023-12-27 07:00:22,683][20347] Updated weights for policy 0, policy_version 195487 (0.0008) [2023-12-27 07:00:22,709][20381] Updated weights for policy 1, policy_version 195169 (0.0007) [2023-12-27 07:00:22,765][20381] Updated weights for policy 1, policy_version 195179 (0.0006) [2023-12-27 07:00:22,788][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-27 07:00:23,398][20347] Updated weights for policy 0, policy_version 195497 (0.0007) [2023-12-27 07:00:23,449][20347] Updated weights for policy 0, policy_version 195507 (0.0007) [2023-12-27 07:00:23,460][20381] Updated weights for policy 1, policy_version 195189 (0.0007) [2023-12-27 07:00:23,493][20347] Updated weights for policy 0, policy_version 195517 (0.0006) [2023-12-27 07:00:23,506][20381] Updated weights for policy 1, policy_version 195199 (0.0006) [2023-12-27 07:00:23,537][20347] Updated weights for policy 0, policy_version 195527 (0.0007) [2023-12-27 07:00:23,554][20381] Updated weights for policy 1, policy_version 195209 (0.0007) [2023-12-27 07:00:24,222][20381] Updated weights for policy 1, policy_version 195219 (0.0008) [2023-12-27 07:00:24,278][20347] Updated weights for policy 0, policy_version 195537 (0.0006) [2023-12-27 07:00:24,282][20381] Updated weights for policy 1, policy_version 195229 (0.0008) [2023-12-27 07:00:24,333][20381] Updated weights for policy 1, policy_version 195239 (0.0008) [2023-12-27 07:00:24,336][20347] Updated weights for policy 0, policy_version 195547 (0.0007) [2023-12-27 07:00:24,390][20347] Updated weights for policy 0, policy_version 195557 (0.0007) [2023-12-27 07:00:24,948][20381] Updated weights for policy 1, policy_version 195249 (0.0007) [2023-12-27 07:00:24,982][20347] Updated weights for policy 0, policy_version 195567 (0.0005) [2023-12-27 07:00:25,005][20381] Updated weights for policy 1, policy_version 195259 (0.0011) [2023-12-27 07:00:25,036][20347] Updated weights for policy 0, policy_version 195577 (0.0006) [2023-12-27 07:00:25,060][20381] Updated weights for policy 1, policy_version 195269 (0.0010) [2023-12-27 07:00:25,080][20347] Updated weights for policy 0, policy_version 195587 (0.0005) [2023-12-27 07:00:25,110][20381] Updated weights for policy 1, policy_version 195279 (0.0010) [2023-12-27 07:00:25,779][20381] Updated weights for policy 1, policy_version 195289 (0.0007) [2023-12-27 07:00:25,784][20347] Updated weights for policy 0, policy_version 195597 (0.0007) [2023-12-27 07:00:25,833][20381] Updated weights for policy 1, policy_version 195299 (0.0005) [2023-12-27 07:00:25,841][20347] Updated weights for policy 0, policy_version 195607 (0.0007) [2023-12-27 07:00:25,891][20381] Updated weights for policy 1, policy_version 195309 (0.0010) [2023-12-27 07:00:25,897][20347] Updated weights for policy 0, policy_version 195617 (0.0006) [2023-12-27 07:00:26,482][20381] Updated weights for policy 1, policy_version 195319 (0.0007) [2023-12-27 07:00:26,547][20381] Updated weights for policy 1, policy_version 195329 (0.0009) [2023-12-27 07:00:26,560][20347] Updated weights for policy 0, policy_version 195627 (0.0007) [2023-12-27 07:00:26,608][20381] Updated weights for policy 1, policy_version 195339 (0.0006) [2023-12-27 07:00:26,618][20347] Updated weights for policy 0, policy_version 195637 (0.0006) [2023-12-27 07:00:26,679][20347] Updated weights for policy 0, policy_version 195647 (0.0007) [2023-12-27 07:00:27,248][20381] Updated weights for policy 1, policy_version 195349 (0.0008) [2023-12-27 07:00:27,313][20381] Updated weights for policy 1, policy_version 195359 (0.0010) [2023-12-27 07:00:27,359][20347] Updated weights for policy 0, policy_version 195657 (0.0008) [2023-12-27 07:00:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 100122624. Throughput: 0: 10310.1, 1: 10164.6. Samples: 100116560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.730')] [2023-12-27 07:00:27,366][20381] Updated weights for policy 1, policy_version 195369 (0.0011) [2023-12-27 07:00:27,412][20347] Updated weights for policy 0, policy_version 195667 (0.0007) [2023-12-27 07:00:27,463][20347] Updated weights for policy 0, policy_version 195677 (0.0007) [2023-12-27 07:00:27,515][20347] Updated weights for policy 0, policy_version 195687 (0.0007) [2023-12-27 07:00:28,005][20381] Updated weights for policy 1, policy_version 195379 (0.0009) [2023-12-27 07:00:28,057][20381] Updated weights for policy 1, policy_version 195389 (0.0008) [2023-12-27 07:00:28,106][20381] Updated weights for policy 1, policy_version 195399 (0.0007) [2023-12-27 07:00:28,212][20347] Updated weights for policy 0, policy_version 195697 (0.0008) [2023-12-27 07:00:28,263][20347] Updated weights for policy 0, policy_version 195707 (0.0007) [2023-12-27 07:00:28,316][20347] Updated weights for policy 0, policy_version 195717 (0.0008) [2023-12-27 07:00:28,844][20381] Updated weights for policy 1, policy_version 195409 (0.0007) [2023-12-27 07:00:28,899][20381] Updated weights for policy 1, policy_version 195419 (0.0008) [2023-12-27 07:00:28,950][20381] Updated weights for policy 1, policy_version 195429 (0.0007) [2023-12-27 07:00:28,996][20347] Updated weights for policy 0, policy_version 195727 (0.0006) [2023-12-27 07:00:29,005][20381] Updated weights for policy 1, policy_version 195439 (0.0007) [2023-12-27 07:00:29,054][20347] Updated weights for policy 0, policy_version 195737 (0.0007) [2023-12-27 07:00:29,113][20347] Updated weights for policy 0, policy_version 195747 (0.0007) [2023-12-27 07:00:29,720][20381] Updated weights for policy 1, policy_version 195449 (0.0005) [2023-12-27 07:00:29,766][20347] Updated weights for policy 0, policy_version 195757 (0.0007) [2023-12-27 07:00:29,770][20381] Updated weights for policy 1, policy_version 195459 (0.0005) [2023-12-27 07:00:29,824][20381] Updated weights for policy 1, policy_version 195469 (0.0007) [2023-12-27 07:00:29,829][20347] Updated weights for policy 0, policy_version 195767 (0.0006) [2023-12-27 07:00:29,886][20347] Updated weights for policy 0, policy_version 195777 (0.0007) [2023-12-27 07:00:30,441][20381] Updated weights for policy 1, policy_version 195479 (0.0008) [2023-12-27 07:00:30,488][20381] Updated weights for policy 1, policy_version 195489 (0.0007) [2023-12-27 07:00:30,527][20347] Updated weights for policy 0, policy_version 195787 (0.0008) [2023-12-27 07:00:30,541][20381] Updated weights for policy 1, policy_version 195499 (0.0006) [2023-12-27 07:00:30,572][20347] Updated weights for policy 0, policy_version 195797 (0.0006) [2023-12-27 07:00:30,618][20347] Updated weights for policy 0, policy_version 195807 (0.0007) [2023-12-27 07:00:31,276][20381] Updated weights for policy 1, policy_version 195509 (0.0006) [2023-12-27 07:00:31,285][20347] Updated weights for policy 0, policy_version 195817 (0.0007) [2023-12-27 07:00:31,329][20381] Updated weights for policy 1, policy_version 195519 (0.0006) [2023-12-27 07:00:31,334][20347] Updated weights for policy 0, policy_version 195827 (0.0007) [2023-12-27 07:00:31,391][20381] Updated weights for policy 1, policy_version 195529 (0.0008) [2023-12-27 07:00:31,401][20347] Updated weights for policy 0, policy_version 195837 (0.0007) [2023-12-27 07:00:31,456][20347] Updated weights for policy 0, policy_version 195847 (0.0006) [2023-12-27 07:00:32,032][20381] Updated weights for policy 1, policy_version 195539 (0.0006) [2023-12-27 07:00:32,086][20381] Updated weights for policy 1, policy_version 195549 (0.0008) [2023-12-27 07:00:32,117][20347] Updated weights for policy 0, policy_version 195857 (0.0006) [2023-12-27 07:00:32,145][20381] Updated weights for policy 1, policy_version 195559 (0.0006) [2023-12-27 07:00:32,175][20347] Updated weights for policy 0, policy_version 195867 (0.0005) [2023-12-27 07:00:32,236][20347] Updated weights for policy 0, policy_version 195877 (0.0007) [2023-12-27 07:00:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 100237312. Throughput: 0: 10325.3, 1: 10195.3. Samples: 100240736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '962.650')] [2023-12-27 07:00:32,867][20347] Updated weights for policy 0, policy_version 195887 (0.0008) [2023-12-27 07:00:32,873][20381] Updated weights for policy 1, policy_version 195569 (0.0006) [2023-12-27 07:00:32,923][20347] Updated weights for policy 0, policy_version 195897 (0.0007) [2023-12-27 07:00:32,930][20381] Updated weights for policy 1, policy_version 195579 (0.0006) [2023-12-27 07:00:32,980][20347] Updated weights for policy 0, policy_version 195907 (0.0006) [2023-12-27 07:00:32,985][20381] Updated weights for policy 1, policy_version 195589 (0.0006) [2023-12-27 07:00:33,037][20381] Updated weights for policy 1, policy_version 195599 (0.0007) [2023-12-27 07:00:33,660][20347] Updated weights for policy 0, policy_version 195917 (0.0005) [2023-12-27 07:00:33,711][20347] Updated weights for policy 0, policy_version 195927 (0.0005) [2023-12-27 07:00:33,730][20381] Updated weights for policy 1, policy_version 195609 (0.0008) [2023-12-27 07:00:33,768][20347] Updated weights for policy 0, policy_version 195937 (0.0006) [2023-12-27 07:00:33,788][20381] Updated weights for policy 1, policy_version 195619 (0.0006) [2023-12-27 07:00:33,844][20381] Updated weights for policy 1, policy_version 195629 (0.0007) [2023-12-27 07:00:34,422][20347] Updated weights for policy 0, policy_version 195947 (0.0006) [2023-12-27 07:00:34,458][20381] Updated weights for policy 1, policy_version 195639 (0.0007) [2023-12-27 07:00:34,479][20347] Updated weights for policy 0, policy_version 195957 (0.0005) [2023-12-27 07:00:34,515][20381] Updated weights for policy 1, policy_version 195649 (0.0006) [2023-12-27 07:00:34,539][20347] Updated weights for policy 0, policy_version 195967 (0.0006) [2023-12-27 07:00:34,567][20381] Updated weights for policy 1, policy_version 195659 (0.0006) [2023-12-27 07:00:35,217][20381] Updated weights for policy 1, policy_version 195669 (0.0007) [2023-12-27 07:00:35,241][20347] Updated weights for policy 0, policy_version 195977 (0.0005) [2023-12-27 07:00:35,270][20381] Updated weights for policy 1, policy_version 195679 (0.0007) [2023-12-27 07:00:35,302][20347] Updated weights for policy 0, policy_version 195987 (0.0007) [2023-12-27 07:00:35,326][20381] Updated weights for policy 1, policy_version 195689 (0.0006) [2023-12-27 07:00:35,363][20347] Updated weights for policy 0, policy_version 195997 (0.0007) [2023-12-27 07:00:35,414][20347] Updated weights for policy 0, policy_version 196007 (0.0007) [2023-12-27 07:00:36,007][20381] Updated weights for policy 1, policy_version 195699 (0.0007) [2023-12-27 07:00:36,062][20381] Updated weights for policy 1, policy_version 195709 (0.0006) [2023-12-27 07:00:36,068][20347] Updated weights for policy 0, policy_version 196017 (0.0006) [2023-12-27 07:00:36,124][20381] Updated weights for policy 1, policy_version 195719 (0.0006) [2023-12-27 07:00:36,133][20347] Updated weights for policy 0, policy_version 196027 (0.0007) [2023-12-27 07:00:36,193][20347] Updated weights for policy 0, policy_version 196037 (0.0008) [2023-12-27 07:00:36,810][20381] Updated weights for policy 1, policy_version 195729 (0.0007) [2023-12-27 07:00:36,856][20381] Updated weights for policy 1, policy_version 195739 (0.0005) [2023-12-27 07:00:36,907][20381] Updated weights for policy 1, policy_version 195749 (0.0007) [2023-12-27 07:00:36,909][20347] Updated weights for policy 0, policy_version 196047 (0.0007) [2023-12-27 07:00:36,960][20381] Updated weights for policy 1, policy_version 195759 (0.0005) [2023-12-27 07:00:36,966][20347] Updated weights for policy 0, policy_version 196057 (0.0006) [2023-12-27 07:00:37,022][20347] Updated weights for policy 0, policy_version 196067 (0.0008) [2023-12-27 07:00:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 100335616. Throughput: 0: 10311.1, 1: 10233.8. Samples: 100303052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '953.010')] [2023-12-27 07:00:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000195760_50126848.pth... [2023-12-27 07:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000196072_50208768.pth... [2023-12-27 07:00:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000194584_49823744.pth [2023-12-27 07:00:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000194888_49905664.pth [2023-12-27 07:00:37,643][20381] Updated weights for policy 1, policy_version 195769 (0.0008) [2023-12-27 07:00:37,689][20381] Updated weights for policy 1, policy_version 195779 (0.0007) [2023-12-27 07:00:37,736][20347] Updated weights for policy 0, policy_version 196077 (0.0007) [2023-12-27 07:00:37,746][20381] Updated weights for policy 1, policy_version 195789 (0.0006) [2023-12-27 07:00:37,793][20347] Updated weights for policy 0, policy_version 196087 (0.0007) [2023-12-27 07:00:37,841][20347] Updated weights for policy 0, policy_version 196097 (0.0008) [2023-12-27 07:00:38,460][20381] Updated weights for policy 1, policy_version 195799 (0.0007) [2023-12-27 07:00:38,520][20381] Updated weights for policy 1, policy_version 195809 (0.0007) [2023-12-27 07:00:38,547][20347] Updated weights for policy 0, policy_version 196107 (0.0008) [2023-12-27 07:00:38,583][20381] Updated weights for policy 1, policy_version 195819 (0.0008) [2023-12-27 07:00:38,605][20347] Updated weights for policy 0, policy_version 196117 (0.0006) [2023-12-27 07:00:38,665][20347] Updated weights for policy 0, policy_version 196127 (0.0008) [2023-12-27 07:00:39,320][20381] Updated weights for policy 1, policy_version 195829 (0.0008) [2023-12-27 07:00:39,385][20381] Updated weights for policy 1, policy_version 195839 (0.0010) [2023-12-27 07:00:39,413][20347] Updated weights for policy 0, policy_version 196137 (0.0008) [2023-12-27 07:00:39,443][20381] Updated weights for policy 1, policy_version 195849 (0.0009) [2023-12-27 07:00:39,477][20347] Updated weights for policy 0, policy_version 196147 (0.0009) [2023-12-27 07:00:39,535][20347] Updated weights for policy 0, policy_version 196157 (0.0009) [2023-12-27 07:00:39,593][20347] Updated weights for policy 0, policy_version 196167 (0.0009) [2023-12-27 07:00:40,208][20381] Updated weights for policy 1, policy_version 195859 (0.0008) [2023-12-27 07:00:40,269][20381] Updated weights for policy 1, policy_version 195869 (0.0007) [2023-12-27 07:00:40,335][20381] Updated weights for policy 1, policy_version 195879 (0.0006) [2023-12-27 07:00:40,360][20347] Updated weights for policy 0, policy_version 196177 (0.0006) [2023-12-27 07:00:40,419][20347] Updated weights for policy 0, policy_version 196187 (0.0006) [2023-12-27 07:00:40,477][20347] Updated weights for policy 0, policy_version 196197 (0.0006) [2023-12-27 07:00:41,020][20347] Updated weights for policy 0, policy_version 196207 (0.0007) [2023-12-27 07:00:41,026][20381] Updated weights for policy 1, policy_version 195889 (0.0007) [2023-12-27 07:00:41,081][20347] Updated weights for policy 0, policy_version 196217 (0.0007) [2023-12-27 07:00:41,090][20381] Updated weights for policy 1, policy_version 195899 (0.0007) [2023-12-27 07:00:41,142][20347] Updated weights for policy 0, policy_version 196227 (0.0007) [2023-12-27 07:00:41,156][20381] Updated weights for policy 1, policy_version 195909 (0.0007) [2023-12-27 07:00:41,218][20381] Updated weights for policy 1, policy_version 195919 (0.0008) [2023-12-27 07:00:41,812][20347] Updated weights for policy 0, policy_version 196237 (0.0007) [2023-12-27 07:00:41,877][20347] Updated weights for policy 0, policy_version 196247 (0.0006) [2023-12-27 07:00:41,943][20347] Updated weights for policy 0, policy_version 196257 (0.0006) [2023-12-27 07:00:41,946][20381] Updated weights for policy 1, policy_version 195929 (0.0006) [2023-12-27 07:00:42,012][20381] Updated weights for policy 1, policy_version 195939 (0.0007) [2023-12-27 07:00:42,069][20381] Updated weights for policy 1, policy_version 195949 (0.0008) [2023-12-27 07:00:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 100433920. Throughput: 0: 10215.4, 1: 10247.4. Samples: 100421792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.750')] [2023-12-27 07:00:42,585][20347] Updated weights for policy 0, policy_version 196267 (0.0007) [2023-12-27 07:00:42,647][20347] Updated weights for policy 0, policy_version 196277 (0.0006) [2023-12-27 07:00:42,702][20347] Updated weights for policy 0, policy_version 196287 (0.0006) [2023-12-27 07:00:42,748][20381] Updated weights for policy 1, policy_version 195959 (0.0008) [2023-12-27 07:00:42,814][20381] Updated weights for policy 1, policy_version 195969 (0.0006) [2023-12-27 07:00:42,877][20381] Updated weights for policy 1, policy_version 195979 (0.0007) [2023-12-27 07:00:43,367][20347] Updated weights for policy 0, policy_version 196297 (0.0007) [2023-12-27 07:00:43,423][20347] Updated weights for policy 0, policy_version 196307 (0.0008) [2023-12-27 07:00:43,472][20347] Updated weights for policy 0, policy_version 196317 (0.0007) [2023-12-27 07:00:43,531][20347] Updated weights for policy 0, policy_version 196327 (0.0008) [2023-12-27 07:00:43,559][20381] Updated weights for policy 1, policy_version 195989 (0.0008) [2023-12-27 07:00:43,615][20381] Updated weights for policy 1, policy_version 195999 (0.0008) [2023-12-27 07:00:43,662][20381] Updated weights for policy 1, policy_version 196009 (0.0007) [2023-12-27 07:00:44,204][20347] Updated weights for policy 0, policy_version 196337 (0.0005) [2023-12-27 07:00:44,252][20347] Updated weights for policy 0, policy_version 196347 (0.0005) [2023-12-27 07:00:44,308][20347] Updated weights for policy 0, policy_version 196357 (0.0005) [2023-12-27 07:00:44,353][20381] Updated weights for policy 1, policy_version 196019 (0.0007) [2023-12-27 07:00:44,412][20381] Updated weights for policy 1, policy_version 196029 (0.0007) [2023-12-27 07:00:44,479][20381] Updated weights for policy 1, policy_version 196039 (0.0008) [2023-12-27 07:00:44,912][20347] Updated weights for policy 0, policy_version 196367 (0.0007) [2023-12-27 07:00:44,963][20347] Updated weights for policy 0, policy_version 196377 (0.0007) [2023-12-27 07:00:45,023][20347] Updated weights for policy 0, policy_version 196387 (0.0007) [2023-12-27 07:00:45,209][20381] Updated weights for policy 1, policy_version 196049 (0.0008) [2023-12-27 07:00:45,275][20381] Updated weights for policy 1, policy_version 196059 (0.0008) [2023-12-27 07:00:45,340][20381] Updated weights for policy 1, policy_version 196069 (0.0007) [2023-12-27 07:00:45,402][20381] Updated weights for policy 1, policy_version 196079 (0.0007) [2023-12-27 07:00:45,659][20347] Updated weights for policy 0, policy_version 196397 (0.0008) [2023-12-27 07:00:45,717][20347] Updated weights for policy 0, policy_version 196407 (0.0008) [2023-12-27 07:00:45,776][20347] Updated weights for policy 0, policy_version 196417 (0.0007) [2023-12-27 07:00:46,053][20381] Updated weights for policy 1, policy_version 196089 (0.0008) [2023-12-27 07:00:46,101][20381] Updated weights for policy 1, policy_version 196099 (0.0008) [2023-12-27 07:00:46,146][20381] Updated weights for policy 1, policy_version 196109 (0.0008) [2023-12-27 07:00:46,367][20347] Updated weights for policy 0, policy_version 196427 (0.0007) [2023-12-27 07:00:46,422][20347] Updated weights for policy 0, policy_version 196437 (0.0006) [2023-12-27 07:00:46,469][20347] Updated weights for policy 0, policy_version 196447 (0.0005) [2023-12-27 07:00:46,863][20381] Updated weights for policy 1, policy_version 196119 (0.0006) [2023-12-27 07:00:46,911][20381] Updated weights for policy 1, policy_version 196129 (0.0009) [2023-12-27 07:00:46,958][20381] Updated weights for policy 1, policy_version 196139 (0.0011) [2023-12-27 07:00:47,053][20347] Updated weights for policy 0, policy_version 196457 (0.0006) [2023-12-27 07:00:47,110][20347] Updated weights for policy 0, policy_version 196467 (0.0009) [2023-12-27 07:00:47,172][20347] Updated weights for policy 0, policy_version 196477 (0.0007) [2023-12-27 07:00:47,234][20347] Updated weights for policy 0, policy_version 196487 (0.0010) [2023-12-27 07:00:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 100540416. Throughput: 0: 10275.6, 1: 10230.1. Samples: 100546792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:00:47,657][20381] Updated weights for policy 1, policy_version 196149 (0.0011) [2023-12-27 07:00:47,717][20381] Updated weights for policy 1, policy_version 196159 (0.0011) [2023-12-27 07:00:47,775][20381] Updated weights for policy 1, policy_version 196169 (0.0011) [2023-12-27 07:00:47,892][20347] Updated weights for policy 0, policy_version 196497 (0.0009) [2023-12-27 07:00:47,951][20347] Updated weights for policy 0, policy_version 196507 (0.0005) [2023-12-27 07:00:48,013][20347] Updated weights for policy 0, policy_version 196517 (0.0008) [2023-12-27 07:00:48,464][20381] Updated weights for policy 1, policy_version 196179 (0.0009) [2023-12-27 07:00:48,523][20381] Updated weights for policy 1, policy_version 196189 (0.0005) [2023-12-27 07:00:48,587][20381] Updated weights for policy 1, policy_version 196199 (0.0008) [2023-12-27 07:00:48,605][20347] Updated weights for policy 0, policy_version 196527 (0.0010) [2023-12-27 07:00:48,670][20347] Updated weights for policy 0, policy_version 196537 (0.0011) [2023-12-27 07:00:48,734][20347] Updated weights for policy 0, policy_version 196547 (0.0008) [2023-12-27 07:00:49,259][20381] Updated weights for policy 1, policy_version 196209 (0.0010) [2023-12-27 07:00:49,316][20381] Updated weights for policy 1, policy_version 196219 (0.0007) [2023-12-27 07:00:49,377][20381] Updated weights for policy 1, policy_version 196229 (0.0009) [2023-12-27 07:00:49,404][20347] Updated weights for policy 0, policy_version 196557 (0.0010) [2023-12-27 07:00:49,432][20381] Updated weights for policy 1, policy_version 196239 (0.0005) [2023-12-27 07:00:49,456][20347] Updated weights for policy 0, policy_version 196567 (0.0009) [2023-12-27 07:00:49,510][20347] Updated weights for policy 0, policy_version 196577 (0.0010) [2023-12-27 07:00:50,159][20381] Updated weights for policy 1, policy_version 196249 (0.0008) [2023-12-27 07:00:50,192][20347] Updated weights for policy 0, policy_version 196587 (0.0009) [2023-12-27 07:00:50,217][20381] Updated weights for policy 1, policy_version 196259 (0.0007) [2023-12-27 07:00:50,244][20347] Updated weights for policy 0, policy_version 196597 (0.0007) [2023-12-27 07:00:50,274][20381] Updated weights for policy 1, policy_version 196269 (0.0006) [2023-12-27 07:00:50,291][20347] Updated weights for policy 0, policy_version 196607 (0.0006) [2023-12-27 07:00:50,985][20381] Updated weights for policy 1, policy_version 196279 (0.0007) [2023-12-27 07:00:51,000][20347] Updated weights for policy 0, policy_version 196617 (0.0006) [2023-12-27 07:00:51,047][20381] Updated weights for policy 1, policy_version 196289 (0.0010) [2023-12-27 07:00:51,059][20347] Updated weights for policy 0, policy_version 196627 (0.0008) [2023-12-27 07:00:51,103][20381] Updated weights for policy 1, policy_version 196299 (0.0011) [2023-12-27 07:00:51,117][20347] Updated weights for policy 0, policy_version 196637 (0.0008) [2023-12-27 07:00:51,176][20347] Updated weights for policy 0, policy_version 196647 (0.0008) [2023-12-27 07:00:51,769][20381] Updated weights for policy 1, policy_version 196309 (0.0011) [2023-12-27 07:00:51,819][20381] Updated weights for policy 1, policy_version 196319 (0.0011) [2023-12-27 07:00:51,871][20381] Updated weights for policy 1, policy_version 196329 (0.0010) [2023-12-27 07:00:51,886][20347] Updated weights for policy 0, policy_version 196657 (0.0007) [2023-12-27 07:00:51,945][20347] Updated weights for policy 0, policy_version 196667 (0.0010) [2023-12-27 07:00:52,003][20347] Updated weights for policy 0, policy_version 196677 (0.0010) [2023-12-27 07:00:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 100638720. Throughput: 0: 10260.8, 1: 10160.9. Samples: 100608084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:00:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000196336_50274304.pth... [2023-12-27 07:00:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000196680_50364416.pth... [2023-12-27 07:00:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000195160_49971200.pth [2023-12-27 07:00:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000195464_50053120.pth [2023-12-27 07:00:52,557][20381] Updated weights for policy 1, policy_version 196339 (0.0010) [2023-12-27 07:00:52,615][20381] Updated weights for policy 1, policy_version 196349 (0.0006) [2023-12-27 07:00:52,674][20347] Updated weights for policy 0, policy_version 196687 (0.0011) [2023-12-27 07:00:52,682][20381] Updated weights for policy 1, policy_version 196359 (0.0005) [2023-12-27 07:00:52,728][20347] Updated weights for policy 0, policy_version 196697 (0.0010) [2023-12-27 07:00:52,786][20347] Updated weights for policy 0, policy_version 196707 (0.0010) [2023-12-27 07:00:53,294][20381] Updated weights for policy 1, policy_version 196369 (0.0006) [2023-12-27 07:00:53,361][20381] Updated weights for policy 1, policy_version 196379 (0.0008) [2023-12-27 07:00:53,423][20381] Updated weights for policy 1, policy_version 196389 (0.0008) [2023-12-27 07:00:53,455][20347] Updated weights for policy 0, policy_version 196717 (0.0008) [2023-12-27 07:00:53,480][20381] Updated weights for policy 1, policy_version 196399 (0.0007) [2023-12-27 07:00:53,501][20347] Updated weights for policy 0, policy_version 196727 (0.0005) [2023-12-27 07:00:53,548][20347] Updated weights for policy 0, policy_version 196737 (0.0008) [2023-12-27 07:00:54,125][20381] Updated weights for policy 1, policy_version 196409 (0.0008) [2023-12-27 07:00:54,185][20381] Updated weights for policy 1, policy_version 196419 (0.0008) [2023-12-27 07:00:54,242][20381] Updated weights for policy 1, policy_version 196429 (0.0008) [2023-12-27 07:00:54,245][20347] Updated weights for policy 0, policy_version 196747 (0.0009) [2023-12-27 07:00:54,299][20347] Updated weights for policy 0, policy_version 196757 (0.0007) [2023-12-27 07:00:54,350][20347] Updated weights for policy 0, policy_version 196767 (0.0007) [2023-12-27 07:00:54,841][20381] Updated weights for policy 1, policy_version 196439 (0.0005) [2023-12-27 07:00:54,897][20381] Updated weights for policy 1, policy_version 196449 (0.0006) [2023-12-27 07:00:54,955][20381] Updated weights for policy 1, policy_version 196459 (0.0006) [2023-12-27 07:00:55,052][20347] Updated weights for policy 0, policy_version 196777 (0.0008) [2023-12-27 07:00:55,113][20347] Updated weights for policy 0, policy_version 196787 (0.0008) [2023-12-27 07:00:55,170][20347] Updated weights for policy 0, policy_version 196797 (0.0008) [2023-12-27 07:00:55,219][20347] Updated weights for policy 0, policy_version 196807 (0.0009) [2023-12-27 07:00:55,500][20381] Updated weights for policy 1, policy_version 196469 (0.0005) [2023-12-27 07:00:55,551][20381] Updated weights for policy 1, policy_version 196479 (0.0005) [2023-12-27 07:00:55,597][20381] Updated weights for policy 1, policy_version 196489 (0.0006) [2023-12-27 07:00:55,974][20347] Updated weights for policy 0, policy_version 196817 (0.0008) [2023-12-27 07:00:56,036][20347] Updated weights for policy 0, policy_version 196827 (0.0007) [2023-12-27 07:00:56,095][20347] Updated weights for policy 0, policy_version 196837 (0.0008) [2023-12-27 07:00:56,208][20381] Updated weights for policy 1, policy_version 196499 (0.0006) [2023-12-27 07:00:56,257][20381] Updated weights for policy 1, policy_version 196509 (0.0007) [2023-12-27 07:00:56,319][20381] Updated weights for policy 1, policy_version 196519 (0.0008) [2023-12-27 07:00:56,740][20347] Updated weights for policy 0, policy_version 196847 (0.0007) [2023-12-27 07:00:56,787][20347] Updated weights for policy 0, policy_version 196857 (0.0007) [2023-12-27 07:00:56,831][20347] Updated weights for policy 0, policy_version 196867 (0.0007) [2023-12-27 07:00:56,972][20381] Updated weights for policy 1, policy_version 196529 (0.0006) [2023-12-27 07:00:57,031][20381] Updated weights for policy 1, policy_version 196539 (0.0010) [2023-12-27 07:00:57,078][20381] Updated weights for policy 1, policy_version 196549 (0.0010) [2023-12-27 07:00:57,125][20381] Updated weights for policy 1, policy_version 196559 (0.0010) [2023-12-27 07:00:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 100745216. Throughput: 0: 10208.6, 1: 10195.5. Samples: 100732764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:00:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:00:57,569][20347] Updated weights for policy 0, policy_version 196877 (0.0008) [2023-12-27 07:00:57,621][20347] Updated weights for policy 0, policy_version 196887 (0.0009) [2023-12-27 07:00:57,675][20347] Updated weights for policy 0, policy_version 196897 (0.0009) [2023-12-27 07:00:57,706][20381] Updated weights for policy 1, policy_version 196569 (0.0006) [2023-12-27 07:00:57,756][20381] Updated weights for policy 1, policy_version 196579 (0.0005) [2023-12-27 07:00:57,807][20381] Updated weights for policy 1, policy_version 196589 (0.0006) [2023-12-27 07:00:58,425][20381] Updated weights for policy 1, policy_version 196599 (0.0009) [2023-12-27 07:00:58,430][20347] Updated weights for policy 0, policy_version 196907 (0.0008) [2023-12-27 07:00:58,480][20381] Updated weights for policy 1, policy_version 196609 (0.0010) [2023-12-27 07:00:58,490][20347] Updated weights for policy 0, policy_version 196917 (0.0008) [2023-12-27 07:00:58,545][20381] Updated weights for policy 1, policy_version 196619 (0.0007) [2023-12-27 07:00:58,551][20347] Updated weights for policy 0, policy_version 196927 (0.0008) [2023-12-27 07:00:59,257][20381] Updated weights for policy 1, policy_version 196629 (0.0007) [2023-12-27 07:00:59,279][20347] Updated weights for policy 0, policy_version 196937 (0.0006) [2023-12-27 07:00:59,318][20381] Updated weights for policy 1, policy_version 196639 (0.0010) [2023-12-27 07:00:59,345][20347] Updated weights for policy 0, policy_version 196947 (0.0008) [2023-12-27 07:00:59,377][20381] Updated weights for policy 1, policy_version 196649 (0.0009) [2023-12-27 07:00:59,406][20347] Updated weights for policy 0, policy_version 196957 (0.0008) [2023-12-27 07:00:59,458][20347] Updated weights for policy 0, policy_version 196967 (0.0007) [2023-12-27 07:01:00,084][20381] Updated weights for policy 1, policy_version 196659 (0.0009) [2023-12-27 07:01:00,132][20381] Updated weights for policy 1, policy_version 196669 (0.0010) [2023-12-27 07:01:00,182][20347] Updated weights for policy 0, policy_version 196977 (0.0006) [2023-12-27 07:01:00,187][20381] Updated weights for policy 1, policy_version 196679 (0.0009) [2023-12-27 07:01:00,235][20347] Updated weights for policy 0, policy_version 196987 (0.0006) [2023-12-27 07:01:00,285][20347] Updated weights for policy 0, policy_version 196997 (0.0010) [2023-12-27 07:01:00,738][20381] Updated weights for policy 1, policy_version 196689 (0.0006) [2023-12-27 07:01:00,789][20381] Updated weights for policy 1, policy_version 196699 (0.0009) [2023-12-27 07:01:00,842][20381] Updated weights for policy 1, policy_version 196709 (0.0005) [2023-12-27 07:01:00,888][20381] Updated weights for policy 1, policy_version 196719 (0.0005) [2023-12-27 07:01:00,958][20347] Updated weights for policy 0, policy_version 197007 (0.0009) [2023-12-27 07:01:01,013][20347] Updated weights for policy 0, policy_version 197017 (0.0008) [2023-12-27 07:01:01,075][20347] Updated weights for policy 0, policy_version 197027 (0.0010) [2023-12-27 07:01:01,564][20381] Updated weights for policy 1, policy_version 196729 (0.0010) [2023-12-27 07:01:01,626][20381] Updated weights for policy 1, policy_version 196739 (0.0010) [2023-12-27 07:01:01,690][20381] Updated weights for policy 1, policy_version 196749 (0.0011) [2023-12-27 07:01:01,755][20347] Updated weights for policy 0, policy_version 197037 (0.0009) [2023-12-27 07:01:01,826][20347] Updated weights for policy 0, policy_version 197047 (0.0007) [2023-12-27 07:01:01,893][20347] Updated weights for policy 0, policy_version 197057 (0.0006) [2023-12-27 07:01:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 100843520. Throughput: 0: 10200.3, 1: 10247.3. Samples: 100855668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:01:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:01:02,418][20381] Updated weights for policy 1, policy_version 196759 (0.0008) [2023-12-27 07:01:02,470][20381] Updated weights for policy 1, policy_version 196769 (0.0008) [2023-12-27 07:01:02,491][20347] Updated weights for policy 0, policy_version 197067 (0.0006) [2023-12-27 07:01:02,519][20381] Updated weights for policy 1, policy_version 196779 (0.0006) [2023-12-27 07:01:02,540][20347] Updated weights for policy 0, policy_version 197077 (0.0005) [2023-12-27 07:01:02,585][20347] Updated weights for policy 0, policy_version 197087 (0.0007) [2023-12-27 07:01:03,212][20381] Updated weights for policy 1, policy_version 196789 (0.0008) [2023-12-27 07:01:03,250][20347] Updated weights for policy 0, policy_version 197097 (0.0007) [2023-12-27 07:01:03,261][20381] Updated weights for policy 1, policy_version 196799 (0.0010) [2023-12-27 07:01:03,308][20347] Updated weights for policy 0, policy_version 197107 (0.0006) [2023-12-27 07:01:03,319][20381] Updated weights for policy 1, policy_version 196809 (0.0010) [2023-12-27 07:01:03,363][20347] Updated weights for policy 0, policy_version 197117 (0.0009) [2023-12-27 07:01:03,411][20347] Updated weights for policy 0, policy_version 197127 (0.0008) [2023-12-27 07:01:03,981][20381] Updated weights for policy 1, policy_version 196819 (0.0010) [2023-12-27 07:01:04,045][20381] Updated weights for policy 1, policy_version 196829 (0.0010) [2023-12-27 07:01:04,107][20347] Updated weights for policy 0, policy_version 197137 (0.0010) [2023-12-27 07:01:04,108][20381] Updated weights for policy 1, policy_version 196839 (0.0008) [2023-12-27 07:01:04,165][20347] Updated weights for policy 0, policy_version 197147 (0.0011) [2023-12-27 07:01:04,230][20347] Updated weights for policy 0, policy_version 197157 (0.0010) [2023-12-27 07:01:04,793][20381] Updated weights for policy 1, policy_version 196849 (0.0008) [2023-12-27 07:01:04,841][20381] Updated weights for policy 1, policy_version 196859 (0.0010) [2023-12-27 07:01:04,896][20381] Updated weights for policy 1, policy_version 196869 (0.0010) [2023-12-27 07:01:04,924][20347] Updated weights for policy 0, policy_version 197167 (0.0010) [2023-12-27 07:01:04,951][20381] Updated weights for policy 1, policy_version 196879 (0.0010) [2023-12-27 07:01:04,979][20347] Updated weights for policy 0, policy_version 197177 (0.0010) [2023-12-27 07:01:05,032][20347] Updated weights for policy 0, policy_version 197187 (0.0010) [2023-12-27 07:01:05,561][20381] Updated weights for policy 1, policy_version 196889 (0.0007) [2023-12-27 07:01:05,619][20381] Updated weights for policy 1, policy_version 196899 (0.0007) [2023-12-27 07:01:05,668][20381] Updated weights for policy 1, policy_version 196909 (0.0007) [2023-12-27 07:01:05,709][20347] Updated weights for policy 0, policy_version 197197 (0.0010) [2023-12-27 07:01:05,754][20347] Updated weights for policy 0, policy_version 197207 (0.0010) [2023-12-27 07:01:05,805][20347] Updated weights for policy 0, policy_version 197217 (0.0010) [2023-12-27 07:01:06,330][20381] Updated weights for policy 1, policy_version 196919 (0.0010) [2023-12-27 07:01:06,393][20381] Updated weights for policy 1, policy_version 196929 (0.0011) [2023-12-27 07:01:06,432][20347] Updated weights for policy 0, policy_version 197227 (0.0007) [2023-12-27 07:01:06,456][20381] Updated weights for policy 1, policy_version 196939 (0.0007) [2023-12-27 07:01:06,494][20347] Updated weights for policy 0, policy_version 197237 (0.0007) [2023-12-27 07:01:06,548][20347] Updated weights for policy 0, policy_version 197247 (0.0007) [2023-12-27 07:01:07,147][20381] Updated weights for policy 1, policy_version 196949 (0.0010) [2023-12-27 07:01:07,206][20381] Updated weights for policy 1, policy_version 196959 (0.0011) [2023-12-27 07:01:07,233][20347] Updated weights for policy 0, policy_version 197257 (0.0007) [2023-12-27 07:01:07,265][20381] Updated weights for policy 1, policy_version 196969 (0.0010) [2023-12-27 07:01:07,285][20347] Updated weights for policy 0, policy_version 197267 (0.0006) [2023-12-27 07:01:07,337][20347] Updated weights for policy 0, policy_version 197277 (0.0006) [2023-12-27 07:01:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20688.3). Total num frames: 100950016. Throughput: 0: 10212.8, 1: 10274.6. Samples: 100917492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:01:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:01:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000196976_50438144.pth... [2023-12-27 07:01:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000195760_50126848.pth [2023-12-27 07:01:07,388][20347] Updated weights for policy 0, policy_version 197287 (0.0007) [2023-12-27 07:01:07,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000197288_50520064.pth... [2023-12-27 07:01:07,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000196072_50208768.pth [2023-12-27 07:01:07,851][20381] Updated weights for policy 1, policy_version 196979 (0.0009) [2023-12-27 07:01:07,905][20381] Updated weights for policy 1, policy_version 196989 (0.0008) [2023-12-27 07:01:07,958][20381] Updated weights for policy 1, policy_version 196999 (0.0005) [2023-12-27 07:01:08,113][20347] Updated weights for policy 0, policy_version 197297 (0.0007) [2023-12-27 07:01:08,163][20347] Updated weights for policy 0, policy_version 197307 (0.0007) [2023-12-27 07:01:08,215][20347] Updated weights for policy 0, policy_version 197317 (0.0008) [2023-12-27 07:01:08,643][20381] Updated weights for policy 1, policy_version 197009 (0.0007) [2023-12-27 07:01:08,694][20381] Updated weights for policy 1, policy_version 197019 (0.0009) [2023-12-27 07:01:08,750][20381] Updated weights for policy 1, policy_version 197029 (0.0010) [2023-12-27 07:01:08,812][20381] Updated weights for policy 1, policy_version 197039 (0.0010) [2023-12-27 07:01:08,856][20347] Updated weights for policy 0, policy_version 197327 (0.0008) [2023-12-27 07:01:08,914][20347] Updated weights for policy 0, policy_version 197337 (0.0007) [2023-12-27 07:01:08,972][20347] Updated weights for policy 0, policy_version 197347 (0.0008) [2023-12-27 07:01:09,526][20381] Updated weights for policy 1, policy_version 197049 (0.0006) [2023-12-27 07:01:09,586][20381] Updated weights for policy 1, policy_version 197059 (0.0005) [2023-12-27 07:01:09,648][20381] Updated weights for policy 1, policy_version 197069 (0.0007) [2023-12-27 07:01:09,750][20347] Updated weights for policy 0, policy_version 197357 (0.0007) [2023-12-27 07:01:09,812][20347] Updated weights for policy 0, policy_version 197367 (0.0009) [2023-12-27 07:01:09,873][20347] Updated weights for policy 0, policy_version 197377 (0.0008) [2023-12-27 07:01:10,351][20381] Updated weights for policy 1, policy_version 197079 (0.0008) [2023-12-27 07:01:10,414][20381] Updated weights for policy 1, policy_version 197089 (0.0008) [2023-12-27 07:01:10,469][20381] Updated weights for policy 1, policy_version 197099 (0.0008) [2023-12-27 07:01:10,532][20347] Updated weights for policy 0, policy_version 197387 (0.0007) [2023-12-27 07:01:10,600][20347] Updated weights for policy 0, policy_version 197397 (0.0006) [2023-12-27 07:01:10,655][20347] Updated weights for policy 0, policy_version 197407 (0.0005) [2023-12-27 07:01:11,151][20381] Updated weights for policy 1, policy_version 197109 (0.0008) [2023-12-27 07:01:11,208][20381] Updated weights for policy 1, policy_version 197119 (0.0006) [2023-12-27 07:01:11,272][20381] Updated weights for policy 1, policy_version 197129 (0.0007) [2023-12-27 07:01:11,288][20347] Updated weights for policy 0, policy_version 197417 (0.0006) [2023-12-27 07:01:11,348][20347] Updated weights for policy 0, policy_version 197427 (0.0008) [2023-12-27 07:01:11,410][20347] Updated weights for policy 0, policy_version 197437 (0.0009) [2023-12-27 07:01:11,471][20347] Updated weights for policy 0, policy_version 197447 (0.0008) [2023-12-27 07:01:11,979][20381] Updated weights for policy 1, policy_version 197139 (0.0007) [2023-12-27 07:01:12,033][20381] Updated weights for policy 1, policy_version 197149 (0.0007) [2023-12-27 07:01:12,086][20381] Updated weights for policy 1, policy_version 197159 (0.0007) [2023-12-27 07:01:12,129][20347] Updated weights for policy 0, policy_version 197457 (0.0007) [2023-12-27 07:01:12,184][20347] Updated weights for policy 0, policy_version 197467 (0.0009) [2023-12-27 07:01:12,227][20347] Updated weights for policy 0, policy_version 197477 (0.0009) [2023-12-27 07:01:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20716.1). Total num frames: 101056512. Throughput: 0: 10262.8, 1: 10279.5. Samples: 101040968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:01:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '919.550')] [2023-12-27 07:01:12,729][20381] Updated weights for policy 1, policy_version 197169 (0.0006) [2023-12-27 07:01:12,782][20381] Updated weights for policy 1, policy_version 197179 (0.0008) [2023-12-27 07:01:12,834][20381] Updated weights for policy 1, policy_version 197189 (0.0009) [2023-12-27 07:01:12,880][20381] Updated weights for policy 1, policy_version 197199 (0.0009) [2023-12-27 07:01:12,907][20347] Updated weights for policy 0, policy_version 197487 (0.0007) [2023-12-27 07:01:12,969][20347] Updated weights for policy 0, policy_version 197497 (0.0007) [2023-12-27 07:01:13,032][20347] Updated weights for policy 0, policy_version 197507 (0.0008) [2023-12-27 07:01:13,614][20381] Updated weights for policy 1, policy_version 197209 (0.0008) [2023-12-27 07:01:13,668][20381] Updated weights for policy 1, policy_version 197219 (0.0007) [2023-12-27 07:01:13,699][20347] Updated weights for policy 0, policy_version 197517 (0.0008) [2023-12-27 07:01:13,730][20381] Updated weights for policy 1, policy_version 197229 (0.0006) [2023-12-27 07:01:13,759][20347] Updated weights for policy 0, policy_version 197527 (0.0008) [2023-12-27 07:01:13,815][20347] Updated weights for policy 0, policy_version 197537 (0.0007) [2023-12-27 07:01:14,393][20381] Updated weights for policy 1, policy_version 197239 (0.0007) [2023-12-27 07:01:14,433][20347] Updated weights for policy 0, policy_version 197547 (0.0008) [2023-12-27 07:01:14,440][20381] Updated weights for policy 1, policy_version 197249 (0.0009) [2023-12-27 07:01:14,481][20347] Updated weights for policy 0, policy_version 197557 (0.0007) [2023-12-27 07:01:14,490][20381] Updated weights for policy 1, policy_version 197259 (0.0007) [2023-12-27 07:01:14,540][20347] Updated weights for policy 0, policy_version 197567 (0.0005) [2023-12-27 07:01:15,151][20381] Updated weights for policy 1, policy_version 197269 (0.0008) [2023-12-27 07:01:15,210][20381] Updated weights for policy 1, policy_version 197279 (0.0008) [2023-12-27 07:01:15,214][20347] Updated weights for policy 0, policy_version 197577 (0.0007) [2023-12-27 07:01:15,264][20381] Updated weights for policy 1, policy_version 197289 (0.0009) [2023-12-27 07:01:15,264][20347] Updated weights for policy 0, policy_version 197587 (0.0010) [2023-12-27 07:01:15,317][20347] Updated weights for policy 0, policy_version 197597 (0.0010) [2023-12-27 07:01:15,375][20347] Updated weights for policy 0, policy_version 197607 (0.0010) [2023-12-27 07:01:15,990][20381] Updated weights for policy 1, policy_version 197299 (0.0006) [2023-12-27 07:01:16,020][20347] Updated weights for policy 0, policy_version 197617 (0.0006) [2023-12-27 07:01:16,039][20381] Updated weights for policy 1, policy_version 197309 (0.0007) [2023-12-27 07:01:16,074][20347] Updated weights for policy 0, policy_version 197627 (0.0006) [2023-12-27 07:01:16,087][20381] Updated weights for policy 1, policy_version 197319 (0.0007) [2023-12-27 07:01:16,126][20347] Updated weights for policy 0, policy_version 197637 (0.0006) [2023-12-27 07:01:16,751][20347] Updated weights for policy 0, policy_version 197647 (0.0009) [2023-12-27 07:01:16,761][20381] Updated weights for policy 1, policy_version 197329 (0.0005) [2023-12-27 07:01:16,809][20347] Updated weights for policy 0, policy_version 197657 (0.0010) [2023-12-27 07:01:16,823][20381] Updated weights for policy 1, policy_version 197339 (0.0006) [2023-12-27 07:01:16,864][20347] Updated weights for policy 0, policy_version 197667 (0.0010) [2023-12-27 07:01:16,882][20381] Updated weights for policy 1, policy_version 197349 (0.0005) [2023-12-27 07:01:16,949][20381] Updated weights for policy 1, policy_version 197359 (0.0005) [2023-12-27 07:01:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 101154816. Throughput: 0: 10282.9, 1: 10281.2. Samples: 101166120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:01:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '717.280')] [2023-12-27 07:01:17,485][20347] Updated weights for policy 0, policy_version 197677 (0.0009) [2023-12-27 07:01:17,504][20381] Updated weights for policy 1, policy_version 197369 (0.0010) [2023-12-27 07:01:17,533][20347] Updated weights for policy 0, policy_version 197687 (0.0006) [2023-12-27 07:01:17,560][20381] Updated weights for policy 1, policy_version 197379 (0.0011) [2023-12-27 07:01:17,598][20347] Updated weights for policy 0, policy_version 197697 (0.0005) [2023-12-27 07:01:17,617][20381] Updated weights for policy 1, policy_version 197389 (0.0010) [2023-12-27 07:01:18,155][20347] Updated weights for policy 0, policy_version 197707 (0.0007) [2023-12-27 07:01:18,212][20347] Updated weights for policy 0, policy_version 197717 (0.0009) [2023-12-27 07:01:18,267][20347] Updated weights for policy 0, policy_version 197727 (0.0008) [2023-12-27 07:01:18,296][20381] Updated weights for policy 1, policy_version 197399 (0.0010) [2023-12-27 07:01:18,353][20381] Updated weights for policy 1, policy_version 197409 (0.0011) [2023-12-27 07:01:18,411][20381] Updated weights for policy 1, policy_version 197419 (0.0010) [2023-12-27 07:01:18,937][20347] Updated weights for policy 0, policy_version 197737 (0.0007) [2023-12-27 07:01:18,997][20347] Updated weights for policy 0, policy_version 197747 (0.0011) [2023-12-27 07:01:19,057][20347] Updated weights for policy 0, policy_version 197757 (0.0011) [2023-12-27 07:01:19,114][20347] Updated weights for policy 0, policy_version 197767 (0.0011) [2023-12-27 07:01:19,126][20381] Updated weights for policy 1, policy_version 197429 (0.0011) [2023-12-27 07:01:19,189][20381] Updated weights for policy 1, policy_version 197439 (0.0009) [2023-12-27 07:01:19,256][20381] Updated weights for policy 1, policy_version 197449 (0.0008) [2023-12-27 07:01:19,845][20291] KL-divergence is very high: 166.7940 [2023-12-27 07:01:19,850][20347] Updated weights for policy 0, policy_version 197777 (0.0008) [2023-12-27 07:01:19,910][20347] Updated weights for policy 0, policy_version 197787 (0.0008) [2023-12-27 07:01:19,953][20381] Updated weights for policy 1, policy_version 197459 (0.0009) [2023-12-27 07:01:19,970][20347] Updated weights for policy 0, policy_version 197797 (0.0006) [2023-12-27 07:01:20,015][20381] Updated weights for policy 1, policy_version 197469 (0.0008) [2023-12-27 07:01:20,068][20381] Updated weights for policy 1, policy_version 197479 (0.0008) [2023-12-27 07:01:20,691][20347] Updated weights for policy 0, policy_version 197807 (0.0007) [2023-12-27 07:01:20,723][20291] KL-divergence is very high: 223.5937 [2023-12-27 07:01:20,734][20291] KL-divergence is very high: 135.1017 [2023-12-27 07:01:20,753][20347] Updated weights for policy 0, policy_version 197817 (0.0008) [2023-12-27 07:01:20,769][20381] Updated weights for policy 1, policy_version 197489 (0.0009) [2023-12-27 07:01:20,815][20347] Updated weights for policy 0, policy_version 197827 (0.0007) [2023-12-27 07:01:20,825][20381] Updated weights for policy 1, policy_version 197499 (0.0005) [2023-12-27 07:01:20,885][20381] Updated weights for policy 1, policy_version 197509 (0.0005) [2023-12-27 07:01:20,945][20381] Updated weights for policy 1, policy_version 197519 (0.0005) [2023-12-27 07:01:21,499][20347] Updated weights for policy 0, policy_version 197837 (0.0008) [2023-12-27 07:01:21,554][20347] Updated weights for policy 0, policy_version 197847 (0.0006) [2023-12-27 07:01:21,597][20381] Updated weights for policy 1, policy_version 197529 (0.0008) [2023-12-27 07:01:21,612][20347] Updated weights for policy 0, policy_version 197857 (0.0006) [2023-12-27 07:01:21,664][20381] Updated weights for policy 1, policy_version 197539 (0.0008) [2023-12-27 07:01:21,717][20381] Updated weights for policy 1, policy_version 197549 (0.0007) [2023-12-27 07:01:22,224][20347] Updated weights for policy 0, policy_version 197867 (0.0008) [2023-12-27 07:01:22,282][20347] Updated weights for policy 0, policy_version 197877 (0.0007) [2023-12-27 07:01:22,342][20347] Updated weights for policy 0, policy_version 197887 (0.0008) [2023-12-27 07:01:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 101253120. Throughput: 0: 10287.0, 1: 10255.1. Samples: 101227448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:01:22,363][19170] Avg episode reward: [(0, '995.040'), (1, '727.790')] [2023-12-27 07:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000197552_50585600.pth... [2023-12-27 07:01:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000196336_50274304.pth [2023-12-27 07:01:22,397][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000197896_50675712.pth... [2023-12-27 07:01:22,400][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000196680_50364416.pth [2023-12-27 07:01:22,464][20381] Updated weights for policy 1, policy_version 197559 (0.0008) [2023-12-27 07:01:22,524][20381] Updated weights for policy 1, policy_version 197569 (0.0006) [2023-12-27 07:01:22,583][20381] Updated weights for policy 1, policy_version 197579 (0.0006) [2023-12-27 07:01:23,061][20347] Updated weights for policy 0, policy_version 197897 (0.0008) [2023-12-27 07:01:23,122][20347] Updated weights for policy 0, policy_version 197907 (0.0006) [2023-12-27 07:01:23,192][20347] Updated weights for policy 0, policy_version 197917 (0.0007) [2023-12-27 07:01:23,193][20381] Updated weights for policy 1, policy_version 197589 (0.0009) [2023-12-27 07:01:23,244][20347] Updated weights for policy 0, policy_version 197927 (0.0006) [2023-12-27 07:01:23,249][20381] Updated weights for policy 1, policy_version 197599 (0.0010) [2023-12-27 07:01:23,299][20381] Updated weights for policy 1, policy_version 197609 (0.0010) [2023-12-27 07:01:23,810][20347] Updated weights for policy 0, policy_version 197937 (0.0010) [2023-12-27 07:01:23,856][20347] Updated weights for policy 0, policy_version 197947 (0.0010) [2023-12-27 07:01:23,897][20381] Updated weights for policy 1, policy_version 197620 (0.0010) [2023-12-27 07:01:23,917][20347] Updated weights for policy 0, policy_version 197957 (0.0011) [2023-12-27 07:01:23,921][20291] KL-divergence is very high: 100.4363 [2023-12-27 07:01:23,926][20291] KL-divergence is very high: 113.3504 [2023-12-27 07:01:23,943][20381] Updated weights for policy 1, policy_version 197630 (0.0006) [2023-12-27 07:01:23,988][20381] Updated weights for policy 1, policy_version 197640 (0.0007) [2023-12-27 07:01:24,530][20347] Updated weights for policy 0, policy_version 197967 (0.0010) [2023-12-27 07:01:24,579][20347] Updated weights for policy 0, policy_version 197977 (0.0010) [2023-12-27 07:01:24,627][20347] Updated weights for policy 0, policy_version 197987 (0.0010) [2023-12-27 07:01:24,660][20381] Updated weights for policy 1, policy_version 197650 (0.0007) [2023-12-27 07:01:24,713][20381] Updated weights for policy 1, policy_version 197660 (0.0006) [2023-12-27 07:01:24,768][20381] Updated weights for policy 1, policy_version 197670 (0.0005) [2023-12-27 07:01:24,815][20381] Updated weights for policy 1, policy_version 197680 (0.0005) [2023-12-27 07:01:25,253][20347] Updated weights for policy 0, policy_version 197997 (0.0010) [2023-12-27 07:01:25,305][20347] Updated weights for policy 0, policy_version 198007 (0.0010) [2023-12-27 07:01:25,350][20381] Updated weights for policy 1, policy_version 197690 (0.0005) [2023-12-27 07:01:25,353][20347] Updated weights for policy 0, policy_version 198017 (0.0010) [2023-12-27 07:01:25,401][20381] Updated weights for policy 1, policy_version 197700 (0.0007) [2023-12-27 07:01:25,458][20381] Updated weights for policy 1, policy_version 197710 (0.0005) [2023-12-27 07:01:25,985][20347] Updated weights for policy 0, policy_version 198027 (0.0010) [2023-12-27 07:01:26,041][20381] Updated weights for policy 1, policy_version 197720 (0.0009) [2023-12-27 07:01:26,043][20347] Updated weights for policy 0, policy_version 198037 (0.0010) [2023-12-27 07:01:26,100][20347] Updated weights for policy 0, policy_version 198047 (0.0010) [2023-12-27 07:01:26,105][20381] Updated weights for policy 1, policy_version 197730 (0.0010) [2023-12-27 07:01:26,161][20381] Updated weights for policy 1, policy_version 197740 (0.0011) [2023-12-27 07:01:26,645][20291] KL-divergence is very high: 142.6436 [2023-12-27 07:01:26,649][20347] Updated weights for policy 0, policy_version 198057 (0.0007) [2023-12-27 07:01:26,692][20291] KL-divergence is very high: 134.5809 [2023-12-27 07:01:26,707][20347] Updated weights for policy 0, policy_version 198067 (0.0005) [2023-12-27 07:01:26,739][20291] KL-divergence is very high: 102.0272 [2023-12-27 07:01:26,763][20347] Updated weights for policy 0, policy_version 198077 (0.0010) [2023-12-27 07:01:26,804][20381] Updated weights for policy 1, policy_version 197750 (0.0010) [2023-12-27 07:01:26,808][20347] Updated weights for policy 0, policy_version 198087 (0.0010) [2023-12-27 07:01:26,856][20381] Updated weights for policy 1, policy_version 197760 (0.0010) [2023-12-27 07:01:26,915][20381] Updated weights for policy 1, policy_version 197770 (0.0010) [2023-12-27 07:01:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 101367808. Throughput: 0: 10387.7, 1: 10384.9. Samples: 101356556. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:27,363][19170] Avg episode reward: [(0, '852.810'), (1, '863.330')] [2023-12-27 07:01:27,444][20347] Updated weights for policy 0, policy_version 198097 (0.0006) [2023-12-27 07:01:27,493][20347] Updated weights for policy 0, policy_version 198107 (0.0010) [2023-12-27 07:01:27,540][20347] Updated weights for policy 0, policy_version 198117 (0.0010) [2023-12-27 07:01:27,581][20381] Updated weights for policy 1, policy_version 197780 (0.0010) [2023-12-27 07:01:27,632][20381] Updated weights for policy 1, policy_version 197790 (0.0010) [2023-12-27 07:01:27,680][20381] Updated weights for policy 1, policy_version 197800 (0.0007) [2023-12-27 07:01:28,116][20291] KL-divergence is very high: 223.9017 [2023-12-27 07:01:28,135][20347] Updated weights for policy 0, policy_version 198127 (0.0007) [2023-12-27 07:01:28,178][20291] KL-divergence is very high: 152.7635 [2023-12-27 07:01:28,183][20347] Updated weights for policy 0, policy_version 198137 (0.0005) [2023-12-27 07:01:28,245][20347] Updated weights for policy 0, policy_version 198147 (0.0010) [2023-12-27 07:01:28,254][20381] Updated weights for policy 1, policy_version 197810 (0.0007) [2023-12-27 07:01:28,308][20381] Updated weights for policy 1, policy_version 197820 (0.0006) [2023-12-27 07:01:28,370][20381] Updated weights for policy 1, policy_version 197830 (0.0007) [2023-12-27 07:01:28,431][20381] Updated weights for policy 1, policy_version 197840 (0.0006) [2023-12-27 07:01:28,903][20291] KL-divergence is very high: 149.2704 [2023-12-27 07:01:28,909][20291] KL-divergence is very high: 143.5070 [2023-12-27 07:01:28,927][20347] Updated weights for policy 0, policy_version 198157 (0.0010) [2023-12-27 07:01:28,930][20291] KL-divergence is very high: 180.4967 [2023-12-27 07:01:28,935][20291] KL-divergence is very high: 106.2535 [2023-12-27 07:01:28,971][20347] Updated weights for policy 0, policy_version 198167 (0.0010) [2023-12-27 07:01:28,978][20291] KL-divergence is very high: 168.5338 [2023-12-27 07:01:28,983][20291] KL-divergence is very high: 164.9537 [2023-12-27 07:01:29,003][20291] KL-divergence is very high: 238.4593 [2023-12-27 07:01:29,007][20291] KL-divergence is very high: 113.8931 [2023-12-27 07:01:29,017][20291] KL-divergence is very high: 103.3362 [2023-12-27 07:01:29,018][20347] Updated weights for policy 0, policy_version 198177 (0.0010) [2023-12-27 07:01:29,044][20291] KL-divergence is very high: 214.1136 [2023-12-27 07:01:29,050][20291] KL-divergence is very high: 194.5282 [2023-12-27 07:01:29,089][20381] Updated weights for policy 1, policy_version 197850 (0.0008) [2023-12-27 07:01:29,142][20381] Updated weights for policy 1, policy_version 197860 (0.0008) [2023-12-27 07:01:29,190][20381] Updated weights for policy 1, policy_version 197870 (0.0006) [2023-12-27 07:01:29,680][20291] KL-divergence is very high: 127.0755 [2023-12-27 07:01:29,685][20347] Updated weights for policy 0, policy_version 198187 (0.0010) [2023-12-27 07:01:29,686][20291] KL-divergence is very high: 254.2888 [2023-12-27 07:01:29,691][20291] KL-divergence is very high: 300.5419 [2023-12-27 07:01:29,697][20291] KL-divergence is very high: 277.2789 [2023-12-27 07:01:29,703][20291] KL-divergence is very high: 162.6484 [2023-12-27 07:01:29,721][20291] KL-divergence is very high: 136.7803 [2023-12-27 07:01:29,733][20291] KL-divergence is very high: 113.4785 [2023-12-27 07:01:29,743][20347] Updated weights for policy 0, policy_version 198197 (0.0010) [2023-12-27 07:01:29,764][20291] KL-divergence is very high: 322.9791 [2023-12-27 07:01:29,770][20291] KL-divergence is very high: 375.3380 [2023-12-27 07:01:29,796][20347] Updated weights for policy 0, policy_version 198207 (0.0010) [2023-12-27 07:01:29,807][20291] KL-divergence is very high: 174.9102 [2023-12-27 07:01:29,813][20291] KL-divergence is very high: 139.3073 [2023-12-27 07:01:29,916][20381] Updated weights for policy 1, policy_version 197880 (0.0007) [2023-12-27 07:01:29,978][20381] Updated weights for policy 1, policy_version 197890 (0.0009) [2023-12-27 07:01:29,991][20292] KL-divergence is very high: 154.1420 [2023-12-27 07:01:30,025][20381] Updated weights for policy 1, policy_version 197900 (0.0007) [2023-12-27 07:01:30,449][20291] KL-divergence is very high: 364.7724 [2023-12-27 07:01:30,454][20347] Updated weights for policy 0, policy_version 198217 (0.0008) [2023-12-27 07:01:30,497][20291] KL-divergence is very high: 429.4930 [2023-12-27 07:01:30,514][20347] Updated weights for policy 0, policy_version 198227 (0.0010) [2023-12-27 07:01:30,541][20291] KL-divergence is very high: 504.3865 [2023-12-27 07:01:30,574][20347] Updated weights for policy 0, policy_version 198237 (0.0010) [2023-12-27 07:01:30,590][20291] KL-divergence is very high: 289.6399 [2023-12-27 07:01:30,624][20347] Updated weights for policy 0, policy_version 198247 (0.0010) [2023-12-27 07:01:30,652][20381] Updated weights for policy 1, policy_version 197910 (0.0006) [2023-12-27 07:01:30,705][20381] Updated weights for policy 1, policy_version 197920 (0.0007) [2023-12-27 07:01:30,756][20381] Updated weights for policy 1, policy_version 197930 (0.0007) [2023-12-27 07:01:31,292][20347] Updated weights for policy 0, policy_version 198257 (0.0007) [2023-12-27 07:01:31,349][20347] Updated weights for policy 0, policy_version 198267 (0.0005) [2023-12-27 07:01:31,412][20347] Updated weights for policy 0, policy_version 198277 (0.0007) [2023-12-27 07:01:31,484][20381] Updated weights for policy 1, policy_version 197940 (0.0006) [2023-12-27 07:01:31,542][20381] Updated weights for policy 1, policy_version 197950 (0.0005) [2023-12-27 07:01:31,602][20381] Updated weights for policy 1, policy_version 197960 (0.0006) [2023-12-27 07:01:32,055][20347] Updated weights for policy 0, policy_version 198287 (0.0006) [2023-12-27 07:01:32,117][20347] Updated weights for policy 0, policy_version 198297 (0.0006) [2023-12-27 07:01:32,174][20347] Updated weights for policy 0, policy_version 198307 (0.0008) [2023-12-27 07:01:32,228][20381] Updated weights for policy 1, policy_version 197970 (0.0008) [2023-12-27 07:01:32,291][20381] Updated weights for policy 1, policy_version 197980 (0.0008) [2023-12-27 07:01:32,349][20381] Updated weights for policy 1, policy_version 197990 (0.0005) [2023-12-27 07:01:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 101474304. Throughput: 0: 10397.0, 1: 10470.0. Samples: 101485808. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:32,363][19170] Avg episode reward: [(0, '590.020'), (1, '948.620')] [2023-12-27 07:01:32,407][20381] Updated weights for policy 1, policy_version 198000 (0.0008) [2023-12-27 07:01:32,861][20347] Updated weights for policy 0, policy_version 198317 (0.0008) [2023-12-27 07:01:32,902][20347] Updated weights for policy 0, policy_version 198327 (0.0007) [2023-12-27 07:01:32,952][20347] Updated weights for policy 0, policy_version 198337 (0.0005) [2023-12-27 07:01:33,081][20381] Updated weights for policy 1, policy_version 198010 (0.0007) [2023-12-27 07:01:33,146][20381] Updated weights for policy 1, policy_version 198020 (0.0006) [2023-12-27 07:01:33,195][20381] Updated weights for policy 1, policy_version 198030 (0.0005) [2023-12-27 07:01:33,541][20347] Updated weights for policy 0, policy_version 198347 (0.0006) [2023-12-27 07:01:33,607][20347] Updated weights for policy 0, policy_version 198357 (0.0005) [2023-12-27 07:01:33,667][20347] Updated weights for policy 0, policy_version 198367 (0.0006) [2023-12-27 07:01:33,800][20381] Updated weights for policy 1, policy_version 198040 (0.0007) [2023-12-27 07:01:33,849][20381] Updated weights for policy 1, policy_version 198050 (0.0007) [2023-12-27 07:01:33,898][20381] Updated weights for policy 1, policy_version 198060 (0.0007) [2023-12-27 07:01:34,252][20347] Updated weights for policy 0, policy_version 198377 (0.0006) [2023-12-27 07:01:34,312][20347] Updated weights for policy 0, policy_version 198387 (0.0008) [2023-12-27 07:01:34,369][20347] Updated weights for policy 0, policy_version 198397 (0.0007) [2023-12-27 07:01:34,422][20347] Updated weights for policy 0, policy_version 198407 (0.0008) [2023-12-27 07:01:34,637][20381] Updated weights for policy 1, policy_version 198070 (0.0006) [2023-12-27 07:01:34,685][20381] Updated weights for policy 1, policy_version 198080 (0.0007) [2023-12-27 07:01:34,737][20381] Updated weights for policy 1, policy_version 198090 (0.0008) [2023-12-27 07:01:35,146][20347] Updated weights for policy 0, policy_version 198417 (0.0006) [2023-12-27 07:01:35,189][20347] Updated weights for policy 0, policy_version 198427 (0.0005) [2023-12-27 07:01:35,237][20347] Updated weights for policy 0, policy_version 198437 (0.0006) [2023-12-27 07:01:35,375][20381] Updated weights for policy 1, policy_version 198100 (0.0007) [2023-12-27 07:01:35,432][20381] Updated weights for policy 1, policy_version 198110 (0.0006) [2023-12-27 07:01:35,487][20381] Updated weights for policy 1, policy_version 198120 (0.0010) [2023-12-27 07:01:35,920][20347] Updated weights for policy 0, policy_version 198447 (0.0007) [2023-12-27 07:01:35,967][20347] Updated weights for policy 0, policy_version 198457 (0.0007) [2023-12-27 07:01:36,021][20347] Updated weights for policy 0, policy_version 198467 (0.0008) [2023-12-27 07:01:36,115][20381] Updated weights for policy 1, policy_version 198130 (0.0009) [2023-12-27 07:01:36,173][20381] Updated weights for policy 1, policy_version 198140 (0.0007) [2023-12-27 07:01:36,238][20381] Updated weights for policy 1, policy_version 198150 (0.0007) [2023-12-27 07:01:36,300][20381] Updated weights for policy 1, policy_version 198160 (0.0008) [2023-12-27 07:01:36,773][20347] Updated weights for policy 0, policy_version 198477 (0.0007) [2023-12-27 07:01:36,828][20347] Updated weights for policy 0, policy_version 198487 (0.0007) [2023-12-27 07:01:36,881][20347] Updated weights for policy 0, policy_version 198497 (0.0007) [2023-12-27 07:01:36,921][20381] Updated weights for policy 1, policy_version 198170 (0.0008) [2023-12-27 07:01:36,969][20381] Updated weights for policy 1, policy_version 198180 (0.0006) [2023-12-27 07:01:37,019][20381] Updated weights for policy 1, policy_version 198190 (0.0008) [2023-12-27 07:01:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 101580800. Throughput: 0: 10408.7, 1: 10504.4. Samples: 101549176. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:37,363][19170] Avg episode reward: [(0, '771.600'), (1, '974.170')] [2023-12-27 07:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000198192_50749440.pth... [2023-12-27 07:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000198504_50831360.pth... [2023-12-27 07:01:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000196976_50438144.pth [2023-12-27 07:01:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000197288_50520064.pth [2023-12-27 07:01:37,560][20347] Updated weights for policy 0, policy_version 198507 (0.0006) [2023-12-27 07:01:37,620][20347] Updated weights for policy 0, policy_version 198517 (0.0008) [2023-12-27 07:01:37,680][20347] Updated weights for policy 0, policy_version 198527 (0.0007) [2023-12-27 07:01:37,742][20381] Updated weights for policy 1, policy_version 198200 (0.0007) [2023-12-27 07:01:37,794][20381] Updated weights for policy 1, policy_version 198210 (0.0008) [2023-12-27 07:01:37,846][20381] Updated weights for policy 1, policy_version 198220 (0.0008) [2023-12-27 07:01:38,292][20347] Updated weights for policy 0, policy_version 198537 (0.0007) [2023-12-27 07:01:38,359][20347] Updated weights for policy 0, policy_version 198547 (0.0007) [2023-12-27 07:01:38,415][20347] Updated weights for policy 0, policy_version 198557 (0.0008) [2023-12-27 07:01:38,464][20347] Updated weights for policy 0, policy_version 198567 (0.0008) [2023-12-27 07:01:38,537][20381] Updated weights for policy 1, policy_version 198230 (0.0006) [2023-12-27 07:01:38,595][20381] Updated weights for policy 1, policy_version 198240 (0.0006) [2023-12-27 07:01:38,654][20381] Updated weights for policy 1, policy_version 198250 (0.0008) [2023-12-27 07:01:39,148][20347] Updated weights for policy 0, policy_version 198577 (0.0008) [2023-12-27 07:01:39,155][20291] KL-divergence is very high: 116.6927 [2023-12-27 07:01:39,161][20291] KL-divergence is very high: 111.1407 [2023-12-27 07:01:39,206][20347] Updated weights for policy 0, policy_version 198587 (0.0006) [2023-12-27 07:01:39,270][20347] Updated weights for policy 0, policy_version 198597 (0.0008) [2023-12-27 07:01:39,328][20381] Updated weights for policy 1, policy_version 198260 (0.0008) [2023-12-27 07:01:39,389][20381] Updated weights for policy 1, policy_version 198270 (0.0008) [2023-12-27 07:01:39,444][20381] Updated weights for policy 1, policy_version 198280 (0.0008) [2023-12-27 07:01:39,981][20347] Updated weights for policy 0, policy_version 198607 (0.0008) [2023-12-27 07:01:40,039][20347] Updated weights for policy 0, policy_version 198617 (0.0007) [2023-12-27 07:01:40,097][20347] Updated weights for policy 0, policy_version 198627 (0.0007) [2023-12-27 07:01:40,167][20381] Updated weights for policy 1, policy_version 198290 (0.0008) [2023-12-27 07:01:40,225][20381] Updated weights for policy 1, policy_version 198300 (0.0011) [2023-12-27 07:01:40,285][20381] Updated weights for policy 1, policy_version 198310 (0.0011) [2023-12-27 07:01:40,346][20381] Updated weights for policy 1, policy_version 198320 (0.0011) [2023-12-27 07:01:40,758][20347] Updated weights for policy 0, policy_version 198637 (0.0008) [2023-12-27 07:01:40,804][20347] Updated weights for policy 0, policy_version 198647 (0.0010) [2023-12-27 07:01:40,850][20347] Updated weights for policy 0, policy_version 198657 (0.0010) [2023-12-27 07:01:41,072][20381] Updated weights for policy 1, policy_version 198330 (0.0008) [2023-12-27 07:01:41,137][20381] Updated weights for policy 1, policy_version 198340 (0.0010) [2023-12-27 07:01:41,201][20381] Updated weights for policy 1, policy_version 198350 (0.0010) [2023-12-27 07:01:41,618][20347] Updated weights for policy 0, policy_version 198667 (0.0009) [2023-12-27 07:01:41,682][20347] Updated weights for policy 0, policy_version 198677 (0.0007) [2023-12-27 07:01:41,745][20347] Updated weights for policy 0, policy_version 198687 (0.0008) [2023-12-27 07:01:41,911][20381] Updated weights for policy 1, policy_version 198360 (0.0008) [2023-12-27 07:01:41,964][20381] Updated weights for policy 1, policy_version 198370 (0.0008) [2023-12-27 07:01:42,014][20381] Updated weights for policy 1, policy_version 198380 (0.0007) [2023-12-27 07:01:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 101679104. Throughput: 0: 10430.5, 1: 10409.2. Samples: 101670548. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:42,363][19170] Avg episode reward: [(0, '961.110'), (1, '990.660')] [2023-12-27 07:01:42,453][20347] Updated weights for policy 0, policy_version 198697 (0.0009) [2023-12-27 07:01:42,511][20347] Updated weights for policy 0, policy_version 198707 (0.0010) [2023-12-27 07:01:42,564][20347] Updated weights for policy 0, policy_version 198717 (0.0010) [2023-12-27 07:01:42,620][20347] Updated weights for policy 0, policy_version 198727 (0.0007) [2023-12-27 07:01:42,724][20381] Updated weights for policy 1, policy_version 198390 (0.0010) [2023-12-27 07:01:42,771][20381] Updated weights for policy 1, policy_version 198400 (0.0011) [2023-12-27 07:01:42,823][20381] Updated weights for policy 1, policy_version 198410 (0.0010) [2023-12-27 07:01:43,254][20347] Updated weights for policy 0, policy_version 198737 (0.0008) [2023-12-27 07:01:43,305][20347] Updated weights for policy 0, policy_version 198747 (0.0007) [2023-12-27 07:01:43,365][20347] Updated weights for policy 0, policy_version 198757 (0.0009) [2023-12-27 07:01:43,508][20381] Updated weights for policy 1, policy_version 198420 (0.0011) [2023-12-27 07:01:43,554][20381] Updated weights for policy 1, policy_version 198430 (0.0011) [2023-12-27 07:01:43,604][20381] Updated weights for policy 1, policy_version 198440 (0.0011) [2023-12-27 07:01:44,056][20347] Updated weights for policy 0, policy_version 198767 (0.0011) [2023-12-27 07:01:44,113][20347] Updated weights for policy 0, policy_version 198777 (0.0010) [2023-12-27 07:01:44,173][20347] Updated weights for policy 0, policy_version 198787 (0.0009) [2023-12-27 07:01:44,288][20381] Updated weights for policy 1, policy_version 198450 (0.0011) [2023-12-27 07:01:44,346][20381] Updated weights for policy 1, policy_version 198460 (0.0011) [2023-12-27 07:01:44,408][20381] Updated weights for policy 1, policy_version 198470 (0.0011) [2023-12-27 07:01:44,469][20381] Updated weights for policy 1, policy_version 198480 (0.0011) [2023-12-27 07:01:44,739][20347] Updated weights for policy 0, policy_version 198797 (0.0005) [2023-12-27 07:01:44,803][20347] Updated weights for policy 0, policy_version 198807 (0.0008) [2023-12-27 07:01:44,869][20347] Updated weights for policy 0, policy_version 198817 (0.0008) [2023-12-27 07:01:45,145][20381] Updated weights for policy 1, policy_version 198490 (0.0005) [2023-12-27 07:01:45,214][20381] Updated weights for policy 1, policy_version 198500 (0.0006) [2023-12-27 07:01:45,282][20381] Updated weights for policy 1, policy_version 198510 (0.0006) [2023-12-27 07:01:45,488][20347] Updated weights for policy 0, policy_version 198827 (0.0009) [2023-12-27 07:01:45,540][20347] Updated weights for policy 0, policy_version 198837 (0.0005) [2023-12-27 07:01:45,594][20347] Updated weights for policy 0, policy_version 198847 (0.0005) [2023-12-27 07:01:45,859][20381] Updated weights for policy 1, policy_version 198520 (0.0010) [2023-12-27 07:01:45,916][20381] Updated weights for policy 1, policy_version 198530 (0.0010) [2023-12-27 07:01:45,971][20381] Updated weights for policy 1, policy_version 198540 (0.0010) [2023-12-27 07:01:46,204][20347] Updated weights for policy 0, policy_version 198857 (0.0008) [2023-12-27 07:01:46,261][20347] Updated weights for policy 0, policy_version 198867 (0.0010) [2023-12-27 07:01:46,317][20347] Updated weights for policy 0, policy_version 198877 (0.0007) [2023-12-27 07:01:46,382][20347] Updated weights for policy 0, policy_version 198887 (0.0005) [2023-12-27 07:01:46,564][20381] Updated weights for policy 1, policy_version 198550 (0.0007) [2023-12-27 07:01:46,617][20381] Updated weights for policy 1, policy_version 198560 (0.0006) [2023-12-27 07:01:46,673][20381] Updated weights for policy 1, policy_version 198570 (0.0005) [2023-12-27 07:01:46,931][20347] Updated weights for policy 0, policy_version 198897 (0.0005) [2023-12-27 07:01:46,985][20347] Updated weights for policy 0, policy_version 198907 (0.0005) [2023-12-27 07:01:47,032][20347] Updated weights for policy 0, policy_version 198917 (0.0009) [2023-12-27 07:01:47,277][20381] Updated weights for policy 1, policy_version 198580 (0.0005) [2023-12-27 07:01:47,340][20381] Updated weights for policy 1, policy_version 198590 (0.0005) [2023-12-27 07:01:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 101785600. Throughput: 0: 10539.2, 1: 10408.8. Samples: 101798328. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:47,363][19170] Avg episode reward: [(0, '969.950'), (1, '1000.000')] [2023-12-27 07:01:47,396][20381] Updated weights for policy 1, policy_version 198600 (0.0005) [2023-12-27 07:01:47,543][20347] Updated weights for policy 0, policy_version 198927 (0.0006) [2023-12-27 07:01:47,594][20347] Updated weights for policy 0, policy_version 198937 (0.0005) [2023-12-27 07:01:47,648][20347] Updated weights for policy 0, policy_version 198947 (0.0005) [2023-12-27 07:01:47,931][20381] Updated weights for policy 1, policy_version 198610 (0.0006) [2023-12-27 07:01:47,992][20381] Updated weights for policy 1, policy_version 198620 (0.0010) [2023-12-27 07:01:48,056][20381] Updated weights for policy 1, policy_version 198630 (0.0011) [2023-12-27 07:01:48,104][20381] Updated weights for policy 1, policy_version 198640 (0.0008) [2023-12-27 07:01:48,203][20347] Updated weights for policy 0, policy_version 198957 (0.0005) [2023-12-27 07:01:48,251][20347] Updated weights for policy 0, policy_version 198967 (0.0007) [2023-12-27 07:01:48,304][20347] Updated weights for policy 0, policy_version 198977 (0.0010) [2023-12-27 07:01:48,773][20381] Updated weights for policy 1, policy_version 198650 (0.0009) [2023-12-27 07:01:48,838][20381] Updated weights for policy 1, policy_version 198660 (0.0011) [2023-12-27 07:01:48,901][20381] Updated weights for policy 1, policy_version 198670 (0.0011) [2023-12-27 07:01:49,011][20347] Updated weights for policy 0, policy_version 198987 (0.0008) [2023-12-27 07:01:49,065][20347] Updated weights for policy 0, policy_version 198997 (0.0007) [2023-12-27 07:01:49,124][20347] Updated weights for policy 0, policy_version 199007 (0.0008) [2023-12-27 07:01:49,572][20381] Updated weights for policy 1, policy_version 198680 (0.0008) [2023-12-27 07:01:49,632][20381] Updated weights for policy 1, policy_version 198690 (0.0007) [2023-12-27 07:01:49,700][20381] Updated weights for policy 1, policy_version 198700 (0.0008) [2023-12-27 07:01:49,840][20347] Updated weights for policy 0, policy_version 199017 (0.0008) [2023-12-27 07:01:49,909][20347] Updated weights for policy 0, policy_version 199027 (0.0008) [2023-12-27 07:01:49,973][20347] Updated weights for policy 0, policy_version 199037 (0.0008) [2023-12-27 07:01:50,029][20347] Updated weights for policy 0, policy_version 199047 (0.0008) [2023-12-27 07:01:50,350][20381] Updated weights for policy 1, policy_version 198710 (0.0007) [2023-12-27 07:01:50,404][20381] Updated weights for policy 1, policy_version 198720 (0.0007) [2023-12-27 07:01:50,461][20381] Updated weights for policy 1, policy_version 198730 (0.0007) [2023-12-27 07:01:50,717][20347] Updated weights for policy 0, policy_version 199057 (0.0008) [2023-12-27 07:01:50,776][20347] Updated weights for policy 0, policy_version 199067 (0.0007) [2023-12-27 07:01:50,834][20347] Updated weights for policy 0, policy_version 199077 (0.0007) [2023-12-27 07:01:51,123][20381] Updated weights for policy 1, policy_version 198740 (0.0007) [2023-12-27 07:01:51,182][20381] Updated weights for policy 1, policy_version 198750 (0.0007) [2023-12-27 07:01:51,232][20381] Updated weights for policy 1, policy_version 198760 (0.0006) [2023-12-27 07:01:51,570][20347] Updated weights for policy 0, policy_version 199087 (0.0006) [2023-12-27 07:01:51,637][20347] Updated weights for policy 0, policy_version 199097 (0.0008) [2023-12-27 07:01:51,695][20347] Updated weights for policy 0, policy_version 199107 (0.0007) [2023-12-27 07:01:51,896][20381] Updated weights for policy 1, policy_version 198770 (0.0006) [2023-12-27 07:01:51,952][20381] Updated weights for policy 1, policy_version 198780 (0.0005) [2023-12-27 07:01:52,011][20381] Updated weights for policy 1, policy_version 198790 (0.0007) [2023-12-27 07:01:52,066][20381] Updated weights for policy 1, policy_version 198800 (0.0006) [2023-12-27 07:01:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 101892096. Throughput: 0: 10575.4, 1: 10435.5. Samples: 101862984. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:52,363][19170] Avg episode reward: [(0, '940.830'), (1, '1000.000')] [2023-12-27 07:01:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000198800_50905088.pth... [2023-12-27 07:01:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000197552_50585600.pth [2023-12-27 07:01:52,396][20347] Updated weights for policy 0, policy_version 199117 (0.0007) [2023-12-27 07:01:52,459][20347] Updated weights for policy 0, policy_version 199127 (0.0008) [2023-12-27 07:01:52,512][20347] Updated weights for policy 0, policy_version 199137 (0.0007) [2023-12-27 07:01:52,546][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000199144_50995200.pth... [2023-12-27 07:01:52,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000197896_50675712.pth [2023-12-27 07:01:52,692][20381] Updated weights for policy 1, policy_version 198810 (0.0008) [2023-12-27 07:01:52,747][20381] Updated weights for policy 1, policy_version 198820 (0.0007) [2023-12-27 07:01:52,795][20381] Updated weights for policy 1, policy_version 198830 (0.0007) [2023-12-27 07:01:53,238][20347] Updated weights for policy 0, policy_version 199147 (0.0007) [2023-12-27 07:01:53,293][20347] Updated weights for policy 0, policy_version 199157 (0.0006) [2023-12-27 07:01:53,356][20347] Updated weights for policy 0, policy_version 199167 (0.0005) [2023-12-27 07:01:53,417][20381] Updated weights for policy 1, policy_version 198840 (0.0006) [2023-12-27 07:01:53,472][20381] Updated weights for policy 1, policy_version 198850 (0.0007) [2023-12-27 07:01:53,521][20381] Updated weights for policy 1, policy_version 198860 (0.0009) [2023-12-27 07:01:53,909][20347] Updated weights for policy 0, policy_version 199177 (0.0006) [2023-12-27 07:01:53,968][20347] Updated weights for policy 0, policy_version 199187 (0.0005) [2023-12-27 07:01:54,026][20347] Updated weights for policy 0, policy_version 199197 (0.0005) [2023-12-27 07:01:54,087][20347] Updated weights for policy 0, policy_version 199207 (0.0005) [2023-12-27 07:01:54,267][20381] Updated weights for policy 1, policy_version 198870 (0.0008) [2023-12-27 07:01:54,323][20381] Updated weights for policy 1, policy_version 198880 (0.0007) [2023-12-27 07:01:54,368][20381] Updated weights for policy 1, policy_version 198890 (0.0006) [2023-12-27 07:01:54,699][20347] Updated weights for policy 0, policy_version 199217 (0.0006) [2023-12-27 07:01:54,753][20347] Updated weights for policy 0, policy_version 199227 (0.0006) [2023-12-27 07:01:54,805][20347] Updated weights for policy 0, policy_version 199237 (0.0007) [2023-12-27 07:01:55,015][20381] Updated weights for policy 1, policy_version 198900 (0.0007) [2023-12-27 07:01:55,073][20381] Updated weights for policy 1, policy_version 198910 (0.0007) [2023-12-27 07:01:55,122][20381] Updated weights for policy 1, policy_version 198920 (0.0007) [2023-12-27 07:01:55,395][20347] Updated weights for policy 0, policy_version 199247 (0.0009) [2023-12-27 07:01:55,443][20347] Updated weights for policy 0, policy_version 199257 (0.0010) [2023-12-27 07:01:55,497][20347] Updated weights for policy 0, policy_version 199267 (0.0010) [2023-12-27 07:01:55,725][20381] Updated weights for policy 1, policy_version 198930 (0.0008) [2023-12-27 07:01:55,779][20381] Updated weights for policy 1, policy_version 198940 (0.0010) [2023-12-27 07:01:55,833][20381] Updated weights for policy 1, policy_version 198950 (0.0010) [2023-12-27 07:01:55,891][20381] Updated weights for policy 1, policy_version 198960 (0.0010) [2023-12-27 07:01:56,127][20347] Updated weights for policy 0, policy_version 199277 (0.0008) [2023-12-27 07:01:56,191][20347] Updated weights for policy 0, policy_version 199287 (0.0005) [2023-12-27 07:01:56,243][20347] Updated weights for policy 0, policy_version 199297 (0.0005) [2023-12-27 07:01:56,559][20381] Updated weights for policy 1, policy_version 198970 (0.0006) [2023-12-27 07:01:56,616][20381] Updated weights for policy 1, policy_version 198980 (0.0006) [2023-12-27 07:01:56,667][20381] Updated weights for policy 1, policy_version 198990 (0.0006) [2023-12-27 07:01:56,799][20347] Updated weights for policy 0, policy_version 199307 (0.0006) [2023-12-27 07:01:56,849][20347] Updated weights for policy 0, policy_version 199317 (0.0005) [2023-12-27 07:01:56,909][20347] Updated weights for policy 0, policy_version 199327 (0.0005) [2023-12-27 07:01:57,236][20381] Updated weights for policy 1, policy_version 199000 (0.0009) [2023-12-27 07:01:57,293][20381] Updated weights for policy 1, policy_version 199010 (0.0010) [2023-12-27 07:01:57,341][20381] Updated weights for policy 1, policy_version 199020 (0.0009) [2023-12-27 07:01:57,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 102006784. Throughput: 0: 10601.1, 1: 10479.0. Samples: 101989572. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:01:57,363][19170] Avg episode reward: [(0, '960.090'), (1, '1000.000')] [2023-12-27 07:01:57,483][20347] Updated weights for policy 0, policy_version 199337 (0.0005) [2023-12-27 07:01:57,534][20347] Updated weights for policy 0, policy_version 199347 (0.0005) [2023-12-27 07:01:57,585][20347] Updated weights for policy 0, policy_version 199357 (0.0005) [2023-12-27 07:01:57,635][20347] Updated weights for policy 0, policy_version 199367 (0.0005) [2023-12-27 07:01:57,928][20381] Updated weights for policy 1, policy_version 199030 (0.0007) [2023-12-27 07:01:57,976][20381] Updated weights for policy 1, policy_version 199040 (0.0010) [2023-12-27 07:01:58,025][20381] Updated weights for policy 1, policy_version 199050 (0.0011) [2023-12-27 07:01:58,212][20347] Updated weights for policy 0, policy_version 199377 (0.0008) [2023-12-27 07:01:58,260][20347] Updated weights for policy 0, policy_version 199387 (0.0008) [2023-12-27 07:01:58,320][20347] Updated weights for policy 0, policy_version 199397 (0.0007) [2023-12-27 07:01:58,834][20381] Updated weights for policy 1, policy_version 199060 (0.0010) [2023-12-27 07:01:58,909][20381] Updated weights for policy 1, policy_version 199070 (0.0008) [2023-12-27 07:01:58,966][20381] Updated weights for policy 1, policy_version 199080 (0.0008) [2023-12-27 07:01:59,124][20347] Updated weights for policy 0, policy_version 199407 (0.0009) [2023-12-27 07:01:59,190][20347] Updated weights for policy 0, policy_version 199417 (0.0010) [2023-12-27 07:01:59,253][20347] Updated weights for policy 0, policy_version 199427 (0.0012) [2023-12-27 07:01:59,634][20381] Updated weights for policy 1, policy_version 199090 (0.0008) [2023-12-27 07:01:59,695][20381] Updated weights for policy 1, policy_version 199100 (0.0006) [2023-12-27 07:01:59,751][20381] Updated weights for policy 1, policy_version 199110 (0.0005) [2023-12-27 07:01:59,809][20381] Updated weights for policy 1, policy_version 199120 (0.0005) [2023-12-27 07:01:59,960][20347] Updated weights for policy 0, policy_version 199437 (0.0009) [2023-12-27 07:02:00,005][20347] Updated weights for policy 0, policy_version 199447 (0.0007) [2023-12-27 07:02:00,055][20347] Updated weights for policy 0, policy_version 199457 (0.0008) [2023-12-27 07:02:00,399][20381] Updated weights for policy 1, policy_version 199130 (0.0006) [2023-12-27 07:02:00,455][20381] Updated weights for policy 1, policy_version 199140 (0.0008) [2023-12-27 07:02:00,500][20381] Updated weights for policy 1, policy_version 199150 (0.0008) [2023-12-27 07:02:00,779][20347] Updated weights for policy 0, policy_version 199467 (0.0008) [2023-12-27 07:02:00,829][20347] Updated weights for policy 0, policy_version 199477 (0.0007) [2023-12-27 07:02:00,887][20347] Updated weights for policy 0, policy_version 199487 (0.0008) [2023-12-27 07:02:01,190][20381] Updated weights for policy 1, policy_version 199160 (0.0009) [2023-12-27 07:02:01,262][20381] Updated weights for policy 1, policy_version 199170 (0.0008) [2023-12-27 07:02:01,326][20381] Updated weights for policy 1, policy_version 199180 (0.0008) [2023-12-27 07:02:01,560][20347] Updated weights for policy 0, policy_version 199497 (0.0007) [2023-12-27 07:02:01,617][20347] Updated weights for policy 0, policy_version 199507 (0.0007) [2023-12-27 07:02:01,675][20347] Updated weights for policy 0, policy_version 199517 (0.0006) [2023-12-27 07:02:01,736][20347] Updated weights for policy 0, policy_version 199527 (0.0008) [2023-12-27 07:02:01,945][20381] Updated weights for policy 1, policy_version 199190 (0.0008) [2023-12-27 07:02:02,012][20381] Updated weights for policy 1, policy_version 199200 (0.0005) [2023-12-27 07:02:02,067][20381] Updated weights for policy 1, policy_version 199210 (0.0009) [2023-12-27 07:02:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20688.3). Total num frames: 102105088. Throughput: 0: 10598.9, 1: 10522.6. Samples: 102116588. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:02,363][19170] Avg episode reward: [(0, '995.180'), (1, '1000.000')] [2023-12-27 07:02:02,394][20347] Updated weights for policy 0, policy_version 199537 (0.0008) [2023-12-27 07:02:02,441][20347] Updated weights for policy 0, policy_version 199547 (0.0008) [2023-12-27 07:02:02,489][20347] Updated weights for policy 0, policy_version 199557 (0.0007) [2023-12-27 07:02:02,690][20381] Updated weights for policy 1, policy_version 199220 (0.0009) [2023-12-27 07:02:02,748][20381] Updated weights for policy 1, policy_version 199230 (0.0008) [2023-12-27 07:02:02,799][20381] Updated weights for policy 1, policy_version 199240 (0.0008) [2023-12-27 07:02:03,226][20347] Updated weights for policy 0, policy_version 199567 (0.0007) [2023-12-27 07:02:03,275][20347] Updated weights for policy 0, policy_version 199578 (0.0007) [2023-12-27 07:02:03,337][20347] Updated weights for policy 0, policy_version 199588 (0.0007) [2023-12-27 07:02:03,360][20381] Updated weights for policy 1, policy_version 199250 (0.0007) [2023-12-27 07:02:03,415][20381] Updated weights for policy 1, policy_version 199260 (0.0005) [2023-12-27 07:02:03,472][20381] Updated weights for policy 1, policy_version 199270 (0.0005) [2023-12-27 07:02:03,530][20381] Updated weights for policy 1, policy_version 199280 (0.0005) [2023-12-27 07:02:03,997][20347] Updated weights for policy 0, policy_version 199598 (0.0008) [2023-12-27 07:02:04,060][20347] Updated weights for policy 0, policy_version 199608 (0.0008) [2023-12-27 07:02:04,119][20347] Updated weights for policy 0, policy_version 199618 (0.0008) [2023-12-27 07:02:04,151][20381] Updated weights for policy 1, policy_version 199290 (0.0007) [2023-12-27 07:02:04,213][20381] Updated weights for policy 1, policy_version 199300 (0.0007) [2023-12-27 07:02:04,272][20381] Updated weights for policy 1, policy_version 199310 (0.0007) [2023-12-27 07:02:04,839][20347] Updated weights for policy 0, policy_version 199628 (0.0008) [2023-12-27 07:02:04,900][20347] Updated weights for policy 0, policy_version 199638 (0.0005) [2023-12-27 07:02:04,944][20381] Updated weights for policy 1, policy_version 199320 (0.0007) [2023-12-27 07:02:04,958][20347] Updated weights for policy 0, policy_version 199648 (0.0009) [2023-12-27 07:02:04,996][20381] Updated weights for policy 1, policy_version 199330 (0.0006) [2023-12-27 07:02:05,047][20381] Updated weights for policy 1, policy_version 199340 (0.0007) [2023-12-27 07:02:05,602][20347] Updated weights for policy 0, policy_version 199658 (0.0010) [2023-12-27 07:02:05,660][20347] Updated weights for policy 0, policy_version 199668 (0.0010) [2023-12-27 07:02:05,717][20347] Updated weights for policy 0, policy_version 199678 (0.0010) [2023-12-27 07:02:05,728][20381] Updated weights for policy 1, policy_version 199350 (0.0008) [2023-12-27 07:02:05,775][20347] Updated weights for policy 0, policy_version 199688 (0.0010) [2023-12-27 07:02:05,781][20381] Updated weights for policy 1, policy_version 199360 (0.0006) [2023-12-27 07:02:05,828][20381] Updated weights for policy 1, policy_version 199370 (0.0008) [2023-12-27 07:02:06,424][20347] Updated weights for policy 0, policy_version 199698 (0.0005) [2023-12-27 07:02:06,479][20347] Updated weights for policy 0, policy_version 199708 (0.0005) [2023-12-27 07:02:06,536][20347] Updated weights for policy 0, policy_version 199718 (0.0006) [2023-12-27 07:02:06,577][20381] Updated weights for policy 1, policy_version 199380 (0.0008) [2023-12-27 07:02:06,634][20381] Updated weights for policy 1, policy_version 199390 (0.0008) [2023-12-27 07:02:06,693][20381] Updated weights for policy 1, policy_version 199400 (0.0008) [2023-12-27 07:02:07,184][20347] Updated weights for policy 0, policy_version 199728 (0.0008) [2023-12-27 07:02:07,242][20347] Updated weights for policy 0, policy_version 199738 (0.0008) [2023-12-27 07:02:07,297][20347] Updated weights for policy 0, policy_version 199748 (0.0007) [2023-12-27 07:02:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 102211584. Throughput: 0: 10572.7, 1: 10578.9. Samples: 102179272. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:07,363][19170] Avg episode reward: [(0, '995.180'), (1, '1000.000')] [2023-12-27 07:02:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000199752_51150848.pth... [2023-12-27 07:02:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000199408_51060736.pth... [2023-12-27 07:02:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000198192_50749440.pth [2023-12-27 07:02:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000198504_50831360.pth [2023-12-27 07:02:07,424][20381] Updated weights for policy 1, policy_version 199410 (0.0008) [2023-12-27 07:02:07,483][20381] Updated weights for policy 1, policy_version 199420 (0.0007) [2023-12-27 07:02:07,547][20381] Updated weights for policy 1, policy_version 199430 (0.0008) [2023-12-27 07:02:07,601][20381] Updated weights for policy 1, policy_version 199440 (0.0007) [2023-12-27 07:02:07,850][20347] Updated weights for policy 0, policy_version 199758 (0.0006) [2023-12-27 07:02:07,910][20347] Updated weights for policy 0, policy_version 199768 (0.0005) [2023-12-27 07:02:07,970][20347] Updated weights for policy 0, policy_version 199778 (0.0005) [2023-12-27 07:02:08,363][20381] Updated weights for policy 1, policy_version 199450 (0.0007) [2023-12-27 07:02:08,423][20381] Updated weights for policy 1, policy_version 199460 (0.0008) [2023-12-27 07:02:08,481][20381] Updated weights for policy 1, policy_version 199470 (0.0007) [2023-12-27 07:02:08,606][20347] Updated weights for policy 0, policy_version 199788 (0.0006) [2023-12-27 07:02:08,658][20347] Updated weights for policy 0, policy_version 199798 (0.0006) [2023-12-27 07:02:08,714][20347] Updated weights for policy 0, policy_version 199808 (0.0005) [2023-12-27 07:02:09,138][20381] Updated weights for policy 1, policy_version 199480 (0.0005) [2023-12-27 07:02:09,201][20381] Updated weights for policy 1, policy_version 199490 (0.0010) [2023-12-27 07:02:09,237][20347] Updated weights for policy 0, policy_version 199818 (0.0006) [2023-12-27 07:02:09,264][20381] Updated weights for policy 1, policy_version 199500 (0.0009) [2023-12-27 07:02:09,292][20347] Updated weights for policy 0, policy_version 199828 (0.0006) [2023-12-27 07:02:09,348][20347] Updated weights for policy 0, policy_version 199838 (0.0008) [2023-12-27 07:02:09,410][20347] Updated weights for policy 0, policy_version 199848 (0.0007) [2023-12-27 07:02:09,886][20381] Updated weights for policy 1, policy_version 199510 (0.0008) [2023-12-27 07:02:09,948][20381] Updated weights for policy 1, policy_version 199520 (0.0008) [2023-12-27 07:02:10,002][20381] Updated weights for policy 1, policy_version 199530 (0.0008) [2023-12-27 07:02:10,178][20347] Updated weights for policy 0, policy_version 199858 (0.0008) [2023-12-27 07:02:10,228][20347] Updated weights for policy 0, policy_version 199868 (0.0007) [2023-12-27 07:02:10,293][20347] Updated weights for policy 0, policy_version 199878 (0.0007) [2023-12-27 07:02:10,711][20381] Updated weights for policy 1, policy_version 199540 (0.0008) [2023-12-27 07:02:10,757][20381] Updated weights for policy 1, policy_version 199550 (0.0008) [2023-12-27 07:02:10,804][20381] Updated weights for policy 1, policy_version 199560 (0.0008) [2023-12-27 07:02:11,002][20347] Updated weights for policy 0, policy_version 199888 (0.0007) [2023-12-27 07:02:11,065][20347] Updated weights for policy 0, policy_version 199898 (0.0007) [2023-12-27 07:02:11,132][20347] Updated weights for policy 0, policy_version 199908 (0.0008) [2023-12-27 07:02:11,557][20381] Updated weights for policy 1, policy_version 199570 (0.0008) [2023-12-27 07:02:11,613][20381] Updated weights for policy 1, policy_version 199580 (0.0008) [2023-12-27 07:02:11,684][20381] Updated weights for policy 1, policy_version 199590 (0.0008) [2023-12-27 07:02:11,750][20381] Updated weights for policy 1, policy_version 199600 (0.0008) [2023-12-27 07:02:11,858][20347] Updated weights for policy 0, policy_version 199918 (0.0009) [2023-12-27 07:02:11,915][20347] Updated weights for policy 0, policy_version 199928 (0.0008) [2023-12-27 07:02:11,975][20347] Updated weights for policy 0, policy_version 199938 (0.0008) [2023-12-27 07:02:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 102309888. Throughput: 0: 10584.6, 1: 10454.7. Samples: 102303328. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:12,524][20381] Updated weights for policy 1, policy_version 199610 (0.0010) [2023-12-27 07:02:12,586][20381] Updated weights for policy 1, policy_version 199620 (0.0011) [2023-12-27 07:02:12,642][20381] Updated weights for policy 1, policy_version 199630 (0.0011) [2023-12-27 07:02:12,712][20347] Updated weights for policy 0, policy_version 199948 (0.0007) [2023-12-27 07:02:12,766][20347] Updated weights for policy 0, policy_version 199958 (0.0008) [2023-12-27 07:02:12,822][20347] Updated weights for policy 0, policy_version 199968 (0.0007) [2023-12-27 07:02:13,312][20381] Updated weights for policy 1, policy_version 199640 (0.0008) [2023-12-27 07:02:13,368][20381] Updated weights for policy 1, policy_version 199650 (0.0008) [2023-12-27 07:02:13,416][20381] Updated weights for policy 1, policy_version 199660 (0.0007) [2023-12-27 07:02:13,536][20347] Updated weights for policy 0, policy_version 199978 (0.0007) [2023-12-27 07:02:13,590][20347] Updated weights for policy 0, policy_version 199988 (0.0007) [2023-12-27 07:02:13,641][20347] Updated weights for policy 0, policy_version 199998 (0.0007) [2023-12-27 07:02:13,698][20347] Updated weights for policy 0, policy_version 200008 (0.0007) [2023-12-27 07:02:14,115][20381] Updated weights for policy 1, policy_version 199670 (0.0010) [2023-12-27 07:02:14,163][20381] Updated weights for policy 1, policy_version 199680 (0.0010) [2023-12-27 07:02:14,224][20381] Updated weights for policy 1, policy_version 199690 (0.0006) [2023-12-27 07:02:14,386][20347] Updated weights for policy 0, policy_version 200018 (0.0008) [2023-12-27 07:02:14,442][20347] Updated weights for policy 0, policy_version 200028 (0.0007) [2023-12-27 07:02:14,493][20347] Updated weights for policy 0, policy_version 200038 (0.0007) [2023-12-27 07:02:14,884][20381] Updated weights for policy 1, policy_version 199700 (0.0006) [2023-12-27 07:02:14,942][20381] Updated weights for policy 1, policy_version 199710 (0.0011) [2023-12-27 07:02:14,992][20381] Updated weights for policy 1, policy_version 199720 (0.0011) [2023-12-27 07:02:15,184][20347] Updated weights for policy 0, policy_version 200048 (0.0006) [2023-12-27 07:02:15,244][20347] Updated weights for policy 0, policy_version 200058 (0.0007) [2023-12-27 07:02:15,299][20347] Updated weights for policy 0, policy_version 200068 (0.0007) [2023-12-27 07:02:15,691][20381] Updated weights for policy 1, policy_version 199730 (0.0011) [2023-12-27 07:02:15,740][20381] Updated weights for policy 1, policy_version 199740 (0.0010) [2023-12-27 07:02:15,788][20381] Updated weights for policy 1, policy_version 199750 (0.0010) [2023-12-27 07:02:15,840][20381] Updated weights for policy 1, policy_version 199760 (0.0010) [2023-12-27 07:02:15,957][20347] Updated weights for policy 0, policy_version 200078 (0.0006) [2023-12-27 07:02:16,013][20347] Updated weights for policy 0, policy_version 200088 (0.0005) [2023-12-27 07:02:16,073][20347] Updated weights for policy 0, policy_version 200098 (0.0010) [2023-12-27 07:02:16,432][20381] Updated weights for policy 1, policy_version 199770 (0.0007) [2023-12-27 07:02:16,483][20381] Updated weights for policy 1, policy_version 199780 (0.0008) [2023-12-27 07:02:16,543][20381] Updated weights for policy 1, policy_version 199790 (0.0007) [2023-12-27 07:02:16,736][20347] Updated weights for policy 0, policy_version 200108 (0.0009) [2023-12-27 07:02:16,797][20347] Updated weights for policy 0, policy_version 200118 (0.0005) [2023-12-27 07:02:16,864][20347] Updated weights for policy 0, policy_version 200128 (0.0005) [2023-12-27 07:02:17,159][20381] Updated weights for policy 1, policy_version 199800 (0.0005) [2023-12-27 07:02:17,221][20381] Updated weights for policy 1, policy_version 199810 (0.0006) [2023-12-27 07:02:17,269][20381] Updated weights for policy 1, policy_version 199820 (0.0005) [2023-12-27 07:02:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 102416384. Throughput: 0: 10482.0, 1: 10412.9. Samples: 102426076. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:17,375][20347] Updated weights for policy 0, policy_version 200138 (0.0006) [2023-12-27 07:02:17,421][20347] Updated weights for policy 0, policy_version 200148 (0.0008) [2023-12-27 07:02:17,466][20347] Updated weights for policy 0, policy_version 200158 (0.0008) [2023-12-27 07:02:17,515][20347] Updated weights for policy 0, policy_version 200168 (0.0007) [2023-12-27 07:02:17,803][20381] Updated weights for policy 1, policy_version 199830 (0.0007) [2023-12-27 07:02:17,858][20381] Updated weights for policy 1, policy_version 199840 (0.0008) [2023-12-27 07:02:17,911][20381] Updated weights for policy 1, policy_version 199850 (0.0007) [2023-12-27 07:02:18,222][20347] Updated weights for policy 0, policy_version 200178 (0.0008) [2023-12-27 07:02:18,273][20347] Updated weights for policy 0, policy_version 200188 (0.0007) [2023-12-27 07:02:18,321][20347] Updated weights for policy 0, policy_version 200198 (0.0007) [2023-12-27 07:02:18,629][20381] Updated weights for policy 1, policy_version 199860 (0.0008) [2023-12-27 07:02:18,686][20381] Updated weights for policy 1, policy_version 199870 (0.0008) [2023-12-27 07:02:18,742][20381] Updated weights for policy 1, policy_version 199880 (0.0008) [2023-12-27 07:02:19,062][20347] Updated weights for policy 0, policy_version 200208 (0.0007) [2023-12-27 07:02:19,117][20347] Updated weights for policy 0, policy_version 200218 (0.0008) [2023-12-27 07:02:19,175][20347] Updated weights for policy 0, policy_version 200228 (0.0007) [2023-12-27 07:02:19,413][20381] Updated weights for policy 1, policy_version 199890 (0.0008) [2023-12-27 07:02:19,470][20381] Updated weights for policy 1, policy_version 199900 (0.0008) [2023-12-27 07:02:19,536][20381] Updated weights for policy 1, policy_version 199910 (0.0008) [2023-12-27 07:02:19,597][20381] Updated weights for policy 1, policy_version 199920 (0.0007) [2023-12-27 07:02:19,882][20347] Updated weights for policy 0, policy_version 200238 (0.0007) [2023-12-27 07:02:19,948][20347] Updated weights for policy 0, policy_version 200248 (0.0008) [2023-12-27 07:02:20,008][20347] Updated weights for policy 0, policy_version 200258 (0.0007) [2023-12-27 07:02:20,312][20381] Updated weights for policy 1, policy_version 199930 (0.0008) [2023-12-27 07:02:20,372][20381] Updated weights for policy 1, policy_version 199940 (0.0007) [2023-12-27 07:02:20,437][20381] Updated weights for policy 1, policy_version 199950 (0.0006) [2023-12-27 07:02:20,745][20347] Updated weights for policy 0, policy_version 200268 (0.0008) [2023-12-27 07:02:20,804][20347] Updated weights for policy 0, policy_version 200278 (0.0009) [2023-12-27 07:02:20,862][20347] Updated weights for policy 0, policy_version 200288 (0.0008) [2023-12-27 07:02:21,056][20381] Updated weights for policy 1, policy_version 199960 (0.0007) [2023-12-27 07:02:21,115][20381] Updated weights for policy 1, policy_version 199970 (0.0008) [2023-12-27 07:02:21,169][20381] Updated weights for policy 1, policy_version 199980 (0.0008) [2023-12-27 07:02:21,548][20347] Updated weights for policy 0, policy_version 200298 (0.0007) [2023-12-27 07:02:21,618][20347] Updated weights for policy 0, policy_version 200308 (0.0007) [2023-12-27 07:02:21,682][20347] Updated weights for policy 0, policy_version 200318 (0.0008) [2023-12-27 07:02:21,753][20347] Updated weights for policy 0, policy_version 200328 (0.0008) [2023-12-27 07:02:21,921][20381] Updated weights for policy 1, policy_version 199990 (0.0008) [2023-12-27 07:02:21,980][20381] Updated weights for policy 1, policy_version 200000 (0.0009) [2023-12-27 07:02:22,036][20381] Updated weights for policy 1, policy_version 200010 (0.0009) [2023-12-27 07:02:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 21026.1, 300 sec: 20688.3). Total num frames: 102514688. Throughput: 0: 10438.2, 1: 10410.6. Samples: 102487372. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000200016_51216384.pth... [2023-12-27 07:02:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000198800_50905088.pth [2023-12-27 07:02:22,390][20347] Updated weights for policy 0, policy_version 200338 (0.0008) [2023-12-27 07:02:22,451][20347] Updated weights for policy 0, policy_version 200348 (0.0008) [2023-12-27 07:02:22,517][20347] Updated weights for policy 0, policy_version 200358 (0.0008) [2023-12-27 07:02:22,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000200360_51306496.pth... [2023-12-27 07:02:22,528][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000199144_50995200.pth [2023-12-27 07:02:22,753][20381] Updated weights for policy 1, policy_version 200020 (0.0008) [2023-12-27 07:02:22,804][20381] Updated weights for policy 1, policy_version 200030 (0.0008) [2023-12-27 07:02:22,858][20381] Updated weights for policy 1, policy_version 200040 (0.0008) [2023-12-27 07:02:23,221][20347] Updated weights for policy 0, policy_version 200368 (0.0008) [2023-12-27 07:02:23,280][20347] Updated weights for policy 0, policy_version 200378 (0.0008) [2023-12-27 07:02:23,334][20347] Updated weights for policy 0, policy_version 200388 (0.0010) [2023-12-27 07:02:23,545][20381] Updated weights for policy 1, policy_version 200050 (0.0009) [2023-12-27 07:02:23,604][20381] Updated weights for policy 1, policy_version 200060 (0.0011) [2023-12-27 07:02:23,665][20381] Updated weights for policy 1, policy_version 200070 (0.0008) [2023-12-27 07:02:23,722][20381] Updated weights for policy 1, policy_version 200080 (0.0009) [2023-12-27 07:02:23,984][20347] Updated weights for policy 0, policy_version 200398 (0.0007) [2023-12-27 07:02:24,049][20347] Updated weights for policy 0, policy_version 200408 (0.0008) [2023-12-27 07:02:24,112][20347] Updated weights for policy 0, policy_version 200418 (0.0008) [2023-12-27 07:02:24,328][20381] Updated weights for policy 1, policy_version 200090 (0.0005) [2023-12-27 07:02:24,373][20381] Updated weights for policy 1, policy_version 200100 (0.0005) [2023-12-27 07:02:24,425][20381] Updated weights for policy 1, policy_version 200110 (0.0010) [2023-12-27 07:02:24,809][20347] Updated weights for policy 0, policy_version 200428 (0.0007) [2023-12-27 07:02:24,862][20347] Updated weights for policy 0, policy_version 200438 (0.0007) [2023-12-27 07:02:24,924][20347] Updated weights for policy 0, policy_version 200448 (0.0008) [2023-12-27 07:02:25,058][20381] Updated weights for policy 1, policy_version 200120 (0.0011) [2023-12-27 07:02:25,104][20381] Updated weights for policy 1, policy_version 200130 (0.0010) [2023-12-27 07:02:25,149][20381] Updated weights for policy 1, policy_version 200140 (0.0010) [2023-12-27 07:02:25,494][20347] Updated weights for policy 0, policy_version 200458 (0.0008) [2023-12-27 07:02:25,541][20347] Updated weights for policy 0, policy_version 200468 (0.0005) [2023-12-27 07:02:25,587][20347] Updated weights for policy 0, policy_version 200478 (0.0006) [2023-12-27 07:02:25,635][20347] Updated weights for policy 0, policy_version 200488 (0.0010) [2023-12-27 07:02:25,865][20381] Updated weights for policy 1, policy_version 200150 (0.0011) [2023-12-27 07:02:25,917][20381] Updated weights for policy 1, policy_version 200160 (0.0010) [2023-12-27 07:02:25,966][20381] Updated weights for policy 1, policy_version 200170 (0.0010) [2023-12-27 07:02:26,237][20347] Updated weights for policy 0, policy_version 200498 (0.0006) [2023-12-27 07:02:26,292][20347] Updated weights for policy 0, policy_version 200508 (0.0005) [2023-12-27 07:02:26,337][20347] Updated weights for policy 0, policy_version 200518 (0.0005) [2023-12-27 07:02:26,572][20381] Updated weights for policy 1, policy_version 200180 (0.0008) [2023-12-27 07:02:26,618][20381] Updated weights for policy 1, policy_version 200190 (0.0005) [2023-12-27 07:02:26,669][20381] Updated weights for policy 1, policy_version 200200 (0.0005) [2023-12-27 07:02:26,984][20347] Updated weights for policy 0, policy_version 200528 (0.0007) [2023-12-27 07:02:27,035][20347] Updated weights for policy 0, policy_version 200538 (0.0007) [2023-12-27 07:02:27,090][20347] Updated weights for policy 0, policy_version 200548 (0.0007) [2023-12-27 07:02:27,295][20381] Updated weights for policy 1, policy_version 200210 (0.0005) [2023-12-27 07:02:27,349][20381] Updated weights for policy 1, policy_version 200220 (0.0005) [2023-12-27 07:02:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 102621184. Throughput: 0: 10467.2, 1: 10429.8. Samples: 102610912. Policy #0 lag: (min: 21.0, avg: 32.6, max: 53.0) [2023-12-27 07:02:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:27,397][20381] Updated weights for policy 1, policy_version 200230 (0.0005) [2023-12-27 07:02:27,450][20381] Updated weights for policy 1, policy_version 200240 (0.0008) [2023-12-27 07:02:27,732][20347] Updated weights for policy 0, policy_version 200558 (0.0006) [2023-12-27 07:02:27,783][20347] Updated weights for policy 0, policy_version 200568 (0.0007) [2023-12-27 07:02:27,833][20347] Updated weights for policy 0, policy_version 200578 (0.0007) [2023-12-27 07:02:28,095][20381] Updated weights for policy 1, policy_version 200250 (0.0006) [2023-12-27 07:02:28,145][20381] Updated weights for policy 1, policy_version 200260 (0.0007) [2023-12-27 07:02:28,196][20381] Updated weights for policy 1, policy_version 200270 (0.0007) [2023-12-27 07:02:28,538][20347] Updated weights for policy 0, policy_version 200588 (0.0007) [2023-12-27 07:02:28,591][20347] Updated weights for policy 0, policy_version 200598 (0.0007) [2023-12-27 07:02:28,643][20347] Updated weights for policy 0, policy_version 200608 (0.0008) [2023-12-27 07:02:28,840][20381] Updated weights for policy 1, policy_version 200280 (0.0006) [2023-12-27 07:02:28,898][20381] Updated weights for policy 1, policy_version 200290 (0.0008) [2023-12-27 07:02:28,954][20381] Updated weights for policy 1, policy_version 200300 (0.0008) [2023-12-27 07:02:29,359][20347] Updated weights for policy 0, policy_version 200618 (0.0008) [2023-12-27 07:02:29,404][20347] Updated weights for policy 0, policy_version 200628 (0.0007) [2023-12-27 07:02:29,453][20347] Updated weights for policy 0, policy_version 200638 (0.0007) [2023-12-27 07:02:29,497][20347] Updated weights for policy 0, policy_version 200648 (0.0007) [2023-12-27 07:02:29,606][20381] Updated weights for policy 1, policy_version 200310 (0.0008) [2023-12-27 07:02:29,665][20381] Updated weights for policy 1, policy_version 200320 (0.0008) [2023-12-27 07:02:29,730][20381] Updated weights for policy 1, policy_version 200330 (0.0008) [2023-12-27 07:02:30,189][20347] Updated weights for policy 0, policy_version 200658 (0.0007) [2023-12-27 07:02:30,246][20347] Updated weights for policy 0, policy_version 200668 (0.0007) [2023-12-27 07:02:30,305][20347] Updated weights for policy 0, policy_version 200678 (0.0008) [2023-12-27 07:02:30,407][20381] Updated weights for policy 1, policy_version 200340 (0.0008) [2023-12-27 07:02:30,451][20381] Updated weights for policy 1, policy_version 200350 (0.0007) [2023-12-27 07:02:30,495][20381] Updated weights for policy 1, policy_version 200360 (0.0007) [2023-12-27 07:02:30,984][20347] Updated weights for policy 0, policy_version 200688 (0.0006) [2023-12-27 07:02:31,046][20347] Updated weights for policy 0, policy_version 200698 (0.0006) [2023-12-27 07:02:31,105][20347] Updated weights for policy 0, policy_version 200708 (0.0007) [2023-12-27 07:02:31,193][20381] Updated weights for policy 1, policy_version 200370 (0.0006) [2023-12-27 07:02:31,248][20381] Updated weights for policy 1, policy_version 200380 (0.0008) [2023-12-27 07:02:31,307][20381] Updated weights for policy 1, policy_version 200390 (0.0009) [2023-12-27 07:02:31,368][20381] Updated weights for policy 1, policy_version 200400 (0.0010) [2023-12-27 07:02:31,761][20347] Updated weights for policy 0, policy_version 200718 (0.0010) [2023-12-27 07:02:31,812][20347] Updated weights for policy 0, policy_version 200728 (0.0010) [2023-12-27 07:02:31,862][20347] Updated weights for policy 0, policy_version 200738 (0.0009) [2023-12-27 07:02:32,020][20381] Updated weights for policy 1, policy_version 200410 (0.0008) [2023-12-27 07:02:32,081][20381] Updated weights for policy 1, policy_version 200420 (0.0008) [2023-12-27 07:02:32,142][20381] Updated weights for policy 1, policy_version 200430 (0.0005) [2023-12-27 07:02:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 102727680. Throughput: 0: 10411.4, 1: 10446.5. Samples: 102736936. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:32,585][20347] Updated weights for policy 0, policy_version 200748 (0.0007) [2023-12-27 07:02:32,639][20347] Updated weights for policy 0, policy_version 200758 (0.0005) [2023-12-27 07:02:32,700][20347] Updated weights for policy 0, policy_version 200768 (0.0005) [2023-12-27 07:02:32,782][20381] Updated weights for policy 1, policy_version 200440 (0.0007) [2023-12-27 07:02:32,828][20381] Updated weights for policy 1, policy_version 200450 (0.0007) [2023-12-27 07:02:32,872][20381] Updated weights for policy 1, policy_version 200460 (0.0008) [2023-12-27 07:02:33,258][20347] Updated weights for policy 0, policy_version 200778 (0.0006) [2023-12-27 07:02:33,317][20347] Updated weights for policy 0, policy_version 200788 (0.0005) [2023-12-27 07:02:33,375][20347] Updated weights for policy 0, policy_version 200798 (0.0005) [2023-12-27 07:02:33,435][20347] Updated weights for policy 0, policy_version 200808 (0.0005) [2023-12-27 07:02:33,623][20381] Updated weights for policy 1, policy_version 200470 (0.0008) [2023-12-27 07:02:33,667][20381] Updated weights for policy 1, policy_version 200480 (0.0008) [2023-12-27 07:02:33,724][20381] Updated weights for policy 1, policy_version 200490 (0.0008) [2023-12-27 07:02:33,955][20347] Updated weights for policy 0, policy_version 200818 (0.0008) [2023-12-27 07:02:34,010][20347] Updated weights for policy 0, policy_version 200828 (0.0007) [2023-12-27 07:02:34,064][20347] Updated weights for policy 0, policy_version 200838 (0.0008) [2023-12-27 07:02:34,452][20381] Updated weights for policy 1, policy_version 200500 (0.0008) [2023-12-27 07:02:34,513][20381] Updated weights for policy 1, policy_version 200510 (0.0008) [2023-12-27 07:02:34,570][20381] Updated weights for policy 1, policy_version 200520 (0.0008) [2023-12-27 07:02:34,812][20347] Updated weights for policy 0, policy_version 200848 (0.0009) [2023-12-27 07:02:34,870][20347] Updated weights for policy 0, policy_version 200858 (0.0008) [2023-12-27 07:02:34,927][20347] Updated weights for policy 0, policy_version 200868 (0.0009) [2023-12-27 07:02:35,300][20381] Updated weights for policy 1, policy_version 200530 (0.0008) [2023-12-27 07:02:35,361][20381] Updated weights for policy 1, policy_version 200540 (0.0006) [2023-12-27 07:02:35,430][20381] Updated weights for policy 1, policy_version 200550 (0.0007) [2023-12-27 07:02:35,491][20381] Updated weights for policy 1, policy_version 200560 (0.0007) [2023-12-27 07:02:35,614][20347] Updated weights for policy 0, policy_version 200878 (0.0006) [2023-12-27 07:02:35,668][20347] Updated weights for policy 0, policy_version 200888 (0.0005) [2023-12-27 07:02:35,723][20347] Updated weights for policy 0, policy_version 200898 (0.0010) [2023-12-27 07:02:36,126][20381] Updated weights for policy 1, policy_version 200570 (0.0008) [2023-12-27 07:02:36,185][20381] Updated weights for policy 1, policy_version 200580 (0.0007) [2023-12-27 07:02:36,252][20381] Updated weights for policy 1, policy_version 200590 (0.0008) [2023-12-27 07:02:36,341][20347] Updated weights for policy 0, policy_version 200908 (0.0009) [2023-12-27 07:02:36,396][20347] Updated weights for policy 0, policy_version 200918 (0.0008) [2023-12-27 07:02:36,464][20347] Updated weights for policy 0, policy_version 200928 (0.0009) [2023-12-27 07:02:36,862][20381] Updated weights for policy 1, policy_version 200600 (0.0006) [2023-12-27 07:02:36,917][20381] Updated weights for policy 1, policy_version 200610 (0.0005) [2023-12-27 07:02:36,965][20381] Updated weights for policy 1, policy_version 200620 (0.0006) [2023-12-27 07:02:37,103][20347] Updated weights for policy 0, policy_version 200938 (0.0008) [2023-12-27 07:02:37,162][20347] Updated weights for policy 0, policy_version 200948 (0.0005) [2023-12-27 07:02:37,222][20347] Updated weights for policy 0, policy_version 200959 (0.0006) [2023-12-27 07:02:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 102834176. Throughput: 0: 10404.3, 1: 10389.9. Samples: 102798728. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000200624_51372032.pth... [2023-12-27 07:02:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000200968_51462144.pth... [2023-12-27 07:02:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000199408_51060736.pth [2023-12-27 07:02:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000199752_51150848.pth [2023-12-27 07:02:37,694][20381] Updated weights for policy 1, policy_version 200630 (0.0008) [2023-12-27 07:02:37,756][20381] Updated weights for policy 1, policy_version 200640 (0.0007) [2023-12-27 07:02:37,795][20347] Updated weights for policy 0, policy_version 200969 (0.0006) [2023-12-27 07:02:37,819][20381] Updated weights for policy 1, policy_version 200650 (0.0007) [2023-12-27 07:02:37,840][20347] Updated weights for policy 0, policy_version 200979 (0.0010) [2023-12-27 07:02:37,885][20347] Updated weights for policy 0, policy_version 200989 (0.0010) [2023-12-27 07:02:37,932][20347] Updated weights for policy 0, policy_version 200999 (0.0010) [2023-12-27 07:02:38,460][20381] Updated weights for policy 1, policy_version 200660 (0.0007) [2023-12-27 07:02:38,519][20381] Updated weights for policy 1, policy_version 200670 (0.0007) [2023-12-27 07:02:38,583][20381] Updated weights for policy 1, policy_version 200680 (0.0008) [2023-12-27 07:02:38,608][20347] Updated weights for policy 0, policy_version 201009 (0.0006) [2023-12-27 07:02:38,662][20347] Updated weights for policy 0, policy_version 201019 (0.0008) [2023-12-27 07:02:38,721][20347] Updated weights for policy 0, policy_version 201029 (0.0010) [2023-12-27 07:02:39,294][20381] Updated weights for policy 1, policy_version 200690 (0.0007) [2023-12-27 07:02:39,355][20381] Updated weights for policy 1, policy_version 200700 (0.0006) [2023-12-27 07:02:39,372][20347] Updated weights for policy 0, policy_version 201039 (0.0009) [2023-12-27 07:02:39,425][20381] Updated weights for policy 1, policy_version 200710 (0.0006) [2023-12-27 07:02:39,436][20347] Updated weights for policy 0, policy_version 201049 (0.0008) [2023-12-27 07:02:39,490][20381] Updated weights for policy 1, policy_version 200720 (0.0008) [2023-12-27 07:02:39,506][20347] Updated weights for policy 0, policy_version 201059 (0.0009) [2023-12-27 07:02:40,182][20381] Updated weights for policy 1, policy_version 200730 (0.0007) [2023-12-27 07:02:40,229][20347] Updated weights for policy 0, policy_version 201069 (0.0010) [2023-12-27 07:02:40,235][20381] Updated weights for policy 1, policy_version 200740 (0.0007) [2023-12-27 07:02:40,286][20347] Updated weights for policy 0, policy_version 201079 (0.0010) [2023-12-27 07:02:40,291][20381] Updated weights for policy 1, policy_version 200750 (0.0005) [2023-12-27 07:02:40,345][20347] Updated weights for policy 0, policy_version 201089 (0.0010) [2023-12-27 07:02:40,975][20347] Updated weights for policy 0, policy_version 201099 (0.0010) [2023-12-27 07:02:41,008][20381] Updated weights for policy 1, policy_version 200760 (0.0006) [2023-12-27 07:02:41,038][20347] Updated weights for policy 0, policy_version 201109 (0.0008) [2023-12-27 07:02:41,071][20381] Updated weights for policy 1, policy_version 200770 (0.0007) [2023-12-27 07:02:41,096][20347] Updated weights for policy 0, policy_version 201119 (0.0011) [2023-12-27 07:02:41,128][20381] Updated weights for policy 1, policy_version 200780 (0.0007) [2023-12-27 07:02:41,821][20381] Updated weights for policy 1, policy_version 200790 (0.0007) [2023-12-27 07:02:41,831][20347] Updated weights for policy 0, policy_version 201129 (0.0009) [2023-12-27 07:02:41,875][20381] Updated weights for policy 1, policy_version 200800 (0.0006) [2023-12-27 07:02:41,891][20347] Updated weights for policy 0, policy_version 201139 (0.0008) [2023-12-27 07:02:41,927][20381] Updated weights for policy 1, policy_version 200810 (0.0006) [2023-12-27 07:02:41,949][20347] Updated weights for policy 0, policy_version 201149 (0.0007) [2023-12-27 07:02:42,013][20347] Updated weights for policy 0, policy_version 201159 (0.0008) [2023-12-27 07:02:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 102932480. Throughput: 0: 10428.6, 1: 10317.7. Samples: 102923160. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:42,622][20381] Updated weights for policy 1, policy_version 200820 (0.0008) [2023-12-27 07:02:42,683][20381] Updated weights for policy 1, policy_version 200830 (0.0011) [2023-12-27 07:02:42,689][20347] Updated weights for policy 0, policy_version 201169 (0.0006) [2023-12-27 07:02:42,738][20381] Updated weights for policy 1, policy_version 200840 (0.0009) [2023-12-27 07:02:42,744][20347] Updated weights for policy 0, policy_version 201179 (0.0006) [2023-12-27 07:02:42,807][20347] Updated weights for policy 0, policy_version 201189 (0.0007) [2023-12-27 07:02:43,363][20381] Updated weights for policy 1, policy_version 200850 (0.0007) [2023-12-27 07:02:43,418][20381] Updated weights for policy 1, policy_version 200860 (0.0010) [2023-12-27 07:02:43,464][20347] Updated weights for policy 0, policy_version 201199 (0.0007) [2023-12-27 07:02:43,471][20381] Updated weights for policy 1, policy_version 200870 (0.0011) [2023-12-27 07:02:43,520][20347] Updated weights for policy 0, policy_version 201209 (0.0007) [2023-12-27 07:02:43,526][20381] Updated weights for policy 1, policy_version 200880 (0.0011) [2023-12-27 07:02:43,581][20347] Updated weights for policy 0, policy_version 201219 (0.0007) [2023-12-27 07:02:44,194][20381] Updated weights for policy 1, policy_version 200890 (0.0009) [2023-12-27 07:02:44,248][20381] Updated weights for policy 1, policy_version 200900 (0.0010) [2023-12-27 07:02:44,295][20347] Updated weights for policy 0, policy_version 201229 (0.0008) [2023-12-27 07:02:44,303][20381] Updated weights for policy 1, policy_version 200910 (0.0005) [2023-12-27 07:02:44,353][20347] Updated weights for policy 0, policy_version 201239 (0.0008) [2023-12-27 07:02:44,406][20347] Updated weights for policy 0, policy_version 201249 (0.0008) [2023-12-27 07:02:44,967][20381] Updated weights for policy 1, policy_version 200920 (0.0010) [2023-12-27 07:02:45,031][20381] Updated weights for policy 1, policy_version 200930 (0.0011) [2023-12-27 07:02:45,088][20381] Updated weights for policy 1, policy_version 200940 (0.0011) [2023-12-27 07:02:45,103][20347] Updated weights for policy 0, policy_version 201259 (0.0007) [2023-12-27 07:02:45,163][20347] Updated weights for policy 0, policy_version 201269 (0.0007) [2023-12-27 07:02:45,221][20347] Updated weights for policy 0, policy_version 201279 (0.0007) [2023-12-27 07:02:45,729][20381] Updated weights for policy 1, policy_version 200950 (0.0007) [2023-12-27 07:02:45,777][20381] Updated weights for policy 1, policy_version 200960 (0.0009) [2023-12-27 07:02:45,825][20381] Updated weights for policy 1, policy_version 200970 (0.0010) [2023-12-27 07:02:45,889][20347] Updated weights for policy 0, policy_version 201289 (0.0008) [2023-12-27 07:02:45,942][20347] Updated weights for policy 0, policy_version 201299 (0.0010) [2023-12-27 07:02:46,000][20347] Updated weights for policy 0, policy_version 201309 (0.0010) [2023-12-27 07:02:46,050][20347] Updated weights for policy 0, policy_version 201319 (0.0007) [2023-12-27 07:02:46,452][20381] Updated weights for policy 1, policy_version 200980 (0.0009) [2023-12-27 07:02:46,506][20381] Updated weights for policy 1, policy_version 200990 (0.0008) [2023-12-27 07:02:46,563][20381] Updated weights for policy 1, policy_version 201000 (0.0008) [2023-12-27 07:02:46,606][20347] Updated weights for policy 0, policy_version 201329 (0.0010) [2023-12-27 07:02:46,654][20347] Updated weights for policy 0, policy_version 201339 (0.0010) [2023-12-27 07:02:46,706][20347] Updated weights for policy 0, policy_version 201349 (0.0010) [2023-12-27 07:02:47,244][20381] Updated weights for policy 1, policy_version 201010 (0.0006) [2023-12-27 07:02:47,292][20381] Updated weights for policy 1, policy_version 201020 (0.0007) [2023-12-27 07:02:47,352][20381] Updated weights for policy 1, policy_version 201030 (0.0007) [2023-12-27 07:02:47,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 103030784. Throughput: 0: 10386.3, 1: 10296.1. Samples: 103047296. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:02:47,390][20347] Updated weights for policy 0, policy_version 201359 (0.0010) [2023-12-27 07:02:47,406][20381] Updated weights for policy 1, policy_version 201040 (0.0009) [2023-12-27 07:02:47,445][20347] Updated weights for policy 0, policy_version 201369 (0.0010) [2023-12-27 07:02:47,493][20347] Updated weights for policy 0, policy_version 201379 (0.0010) [2023-12-27 07:02:48,089][20347] Updated weights for policy 0, policy_version 201389 (0.0010) [2023-12-27 07:02:48,140][20381] Updated weights for policy 1, policy_version 201050 (0.0006) [2023-12-27 07:02:48,145][20347] Updated weights for policy 0, policy_version 201399 (0.0010) [2023-12-27 07:02:48,194][20381] Updated weights for policy 1, policy_version 201060 (0.0006) [2023-12-27 07:02:48,196][20347] Updated weights for policy 0, policy_version 201409 (0.0010) [2023-12-27 07:02:48,251][20381] Updated weights for policy 1, policy_version 201070 (0.0006) [2023-12-27 07:02:48,854][20347] Updated weights for policy 0, policy_version 201419 (0.0009) [2023-12-27 07:02:48,905][20347] Updated weights for policy 0, policy_version 201429 (0.0009) [2023-12-27 07:02:48,964][20347] Updated weights for policy 0, policy_version 201439 (0.0006) [2023-12-27 07:02:48,981][20381] Updated weights for policy 1, policy_version 201080 (0.0008) [2023-12-27 07:02:49,032][20381] Updated weights for policy 1, policy_version 201090 (0.0006) [2023-12-27 07:02:49,081][20381] Updated weights for policy 1, policy_version 201100 (0.0007) [2023-12-27 07:02:49,632][20347] Updated weights for policy 0, policy_version 201449 (0.0006) [2023-12-27 07:02:49,691][20347] Updated weights for policy 0, policy_version 201459 (0.0005) [2023-12-27 07:02:49,753][20347] Updated weights for policy 0, policy_version 201469 (0.0006) [2023-12-27 07:02:49,803][20381] Updated weights for policy 1, policy_version 201110 (0.0008) [2023-12-27 07:02:49,819][20347] Updated weights for policy 0, policy_version 201479 (0.0006) [2023-12-27 07:02:49,866][20381] Updated weights for policy 1, policy_version 201120 (0.0010) [2023-12-27 07:02:49,933][20381] Updated weights for policy 1, policy_version 201130 (0.0010) [2023-12-27 07:02:50,460][20347] Updated weights for policy 0, policy_version 201489 (0.0008) [2023-12-27 07:02:50,524][20347] Updated weights for policy 0, policy_version 201499 (0.0005) [2023-12-27 07:02:50,593][20347] Updated weights for policy 0, policy_version 201509 (0.0007) [2023-12-27 07:02:50,601][20381] Updated weights for policy 1, policy_version 201140 (0.0010) [2023-12-27 07:02:50,659][20381] Updated weights for policy 1, policy_version 201150 (0.0008) [2023-12-27 07:02:50,720][20381] Updated weights for policy 1, policy_version 201160 (0.0011) [2023-12-27 07:02:51,239][20347] Updated weights for policy 0, policy_version 201519 (0.0006) [2023-12-27 07:02:51,302][20347] Updated weights for policy 0, policy_version 201529 (0.0006) [2023-12-27 07:02:51,368][20347] Updated weights for policy 0, policy_version 201539 (0.0006) [2023-12-27 07:02:51,467][20381] Updated weights for policy 1, policy_version 201170 (0.0010) [2023-12-27 07:02:51,526][20381] Updated weights for policy 1, policy_version 201180 (0.0008) [2023-12-27 07:02:51,576][20292] KL-divergence is very high: 108.2539 [2023-12-27 07:02:51,588][20381] Updated weights for policy 1, policy_version 201190 (0.0008) [2023-12-27 07:02:51,651][20381] Updated weights for policy 1, policy_version 201200 (0.0008) [2023-12-27 07:02:52,071][20347] Updated weights for policy 0, policy_version 201549 (0.0007) [2023-12-27 07:02:52,130][20347] Updated weights for policy 0, policy_version 201559 (0.0008) [2023-12-27 07:02:52,182][20347] Updated weights for policy 0, policy_version 201569 (0.0008) [2023-12-27 07:02:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 103137280. Throughput: 0: 10426.5, 1: 10215.3. Samples: 103108152. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.320')] [2023-12-27 07:02:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000201576_51617792.pth... [2023-12-27 07:02:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000200360_51306496.pth [2023-12-27 07:02:52,376][20381] Updated weights for policy 1, policy_version 201210 (0.0008) [2023-12-27 07:02:52,433][20381] Updated weights for policy 1, policy_version 201220 (0.0008) [2023-12-27 07:02:52,498][20381] Updated weights for policy 1, policy_version 201230 (0.0009) [2023-12-27 07:02:52,507][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000201232_51527680.pth... [2023-12-27 07:02:52,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000200016_51216384.pth [2023-12-27 07:02:52,838][20347] Updated weights for policy 0, policy_version 201579 (0.0008) [2023-12-27 07:02:52,896][20347] Updated weights for policy 0, policy_version 201589 (0.0008) [2023-12-27 07:02:52,958][20347] Updated weights for policy 0, policy_version 201599 (0.0008) [2023-12-27 07:02:53,200][20381] Updated weights for policy 1, policy_version 201240 (0.0008) [2023-12-27 07:02:53,260][20381] Updated weights for policy 1, policy_version 201250 (0.0008) [2023-12-27 07:02:53,319][20381] Updated weights for policy 1, policy_version 201260 (0.0007) [2023-12-27 07:02:53,644][20347] Updated weights for policy 0, policy_version 201609 (0.0007) [2023-12-27 07:02:53,689][20347] Updated weights for policy 0, policy_version 201619 (0.0008) [2023-12-27 07:02:53,740][20347] Updated weights for policy 0, policy_version 201629 (0.0007) [2023-12-27 07:02:53,788][20347] Updated weights for policy 0, policy_version 201639 (0.0009) [2023-12-27 07:02:54,001][20381] Updated weights for policy 1, policy_version 201270 (0.0008) [2023-12-27 07:02:54,061][20381] Updated weights for policy 1, policy_version 201280 (0.0011) [2023-12-27 07:02:54,117][20381] Updated weights for policy 1, policy_version 201290 (0.0011) [2023-12-27 07:02:54,518][20347] Updated weights for policy 0, policy_version 201649 (0.0007) [2023-12-27 07:02:54,581][20347] Updated weights for policy 0, policy_version 201659 (0.0007) [2023-12-27 07:02:54,632][20347] Updated weights for policy 0, policy_version 201669 (0.0009) [2023-12-27 07:02:54,726][20381] Updated weights for policy 1, policy_version 201300 (0.0010) [2023-12-27 07:02:54,784][20381] Updated weights for policy 1, policy_version 201310 (0.0011) [2023-12-27 07:02:54,839][20381] Updated weights for policy 1, policy_version 201320 (0.0010) [2023-12-27 07:02:55,311][20347] Updated weights for policy 0, policy_version 201679 (0.0008) [2023-12-27 07:02:55,365][20347] Updated weights for policy 0, policy_version 201689 (0.0007) [2023-12-27 07:02:55,413][20347] Updated weights for policy 0, policy_version 201699 (0.0007) [2023-12-27 07:02:55,432][20381] Updated weights for policy 1, policy_version 201330 (0.0011) [2023-12-27 07:02:55,489][20381] Updated weights for policy 1, policy_version 201340 (0.0011) [2023-12-27 07:02:55,544][20381] Updated weights for policy 1, policy_version 201350 (0.0010) [2023-12-27 07:02:55,601][20381] Updated weights for policy 1, policy_version 201360 (0.0010) [2023-12-27 07:02:56,038][20347] Updated weights for policy 0, policy_version 201709 (0.0007) [2023-12-27 07:02:56,087][20347] Updated weights for policy 0, policy_version 201719 (0.0006) [2023-12-27 07:02:56,141][20347] Updated weights for policy 0, policy_version 201729 (0.0007) [2023-12-27 07:02:56,161][20381] Updated weights for policy 1, policy_version 201370 (0.0006) [2023-12-27 07:02:56,215][20381] Updated weights for policy 1, policy_version 201380 (0.0010) [2023-12-27 07:02:56,265][20381] Updated weights for policy 1, policy_version 201390 (0.0010) [2023-12-27 07:02:56,760][20347] Updated weights for policy 0, policy_version 201739 (0.0006) [2023-12-27 07:02:56,820][20347] Updated weights for policy 0, policy_version 201749 (0.0008) [2023-12-27 07:02:56,870][20347] Updated weights for policy 0, policy_version 201759 (0.0007) [2023-12-27 07:02:56,969][20381] Updated weights for policy 1, policy_version 201400 (0.0008) [2023-12-27 07:02:57,022][20381] Updated weights for policy 1, policy_version 201410 (0.0007) [2023-12-27 07:02:57,079][20381] Updated weights for policy 1, policy_version 201420 (0.0007) [2023-12-27 07:02:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 103243776. Throughput: 0: 10357.0, 1: 10285.2. Samples: 103232228. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:02:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.320')] [2023-12-27 07:02:57,511][20347] Updated weights for policy 0, policy_version 201769 (0.0007) [2023-12-27 07:02:57,578][20347] Updated weights for policy 0, policy_version 201779 (0.0009) [2023-12-27 07:02:57,640][20347] Updated weights for policy 0, policy_version 201789 (0.0007) [2023-12-27 07:02:57,660][20381] Updated weights for policy 1, policy_version 201430 (0.0006) [2023-12-27 07:02:57,698][20347] Updated weights for policy 0, policy_version 201799 (0.0009) [2023-12-27 07:02:57,720][20381] Updated weights for policy 1, policy_version 201440 (0.0006) [2023-12-27 07:02:57,775][20381] Updated weights for policy 1, policy_version 201450 (0.0006) [2023-12-27 07:02:58,254][20347] Updated weights for policy 0, policy_version 201809 (0.0008) [2023-12-27 07:02:58,320][20347] Updated weights for policy 0, policy_version 201819 (0.0009) [2023-12-27 07:02:58,372][20381] Updated weights for policy 1, policy_version 201460 (0.0008) [2023-12-27 07:02:58,385][20347] Updated weights for policy 0, policy_version 201829 (0.0009) [2023-12-27 07:02:58,434][20381] Updated weights for policy 1, policy_version 201470 (0.0007) [2023-12-27 07:02:58,493][20381] Updated weights for policy 1, policy_version 201480 (0.0008) [2023-12-27 07:02:59,120][20347] Updated weights for policy 0, policy_version 201839 (0.0011) [2023-12-27 07:02:59,180][20347] Updated weights for policy 0, policy_version 201849 (0.0010) [2023-12-27 07:02:59,220][20381] Updated weights for policy 1, policy_version 201490 (0.0008) [2023-12-27 07:02:59,246][20347] Updated weights for policy 0, policy_version 201859 (0.0010) [2023-12-27 07:02:59,283][20381] Updated weights for policy 1, policy_version 201500 (0.0008) [2023-12-27 07:02:59,344][20381] Updated weights for policy 1, policy_version 201510 (0.0008) [2023-12-27 07:02:59,404][20381] Updated weights for policy 1, policy_version 201520 (0.0009) [2023-12-27 07:02:59,935][20347] Updated weights for policy 0, policy_version 201869 (0.0007) [2023-12-27 07:02:59,986][20347] Updated weights for policy 0, policy_version 201879 (0.0009) [2023-12-27 07:03:00,037][20347] Updated weights for policy 0, policy_version 201889 (0.0007) [2023-12-27 07:03:00,062][20381] Updated weights for policy 1, policy_version 201530 (0.0006) [2023-12-27 07:03:00,113][20381] Updated weights for policy 1, policy_version 201540 (0.0007) [2023-12-27 07:03:00,166][20381] Updated weights for policy 1, policy_version 201550 (0.0007) [2023-12-27 07:03:00,745][20347] Updated weights for policy 0, policy_version 201899 (0.0007) [2023-12-27 07:03:00,800][20347] Updated weights for policy 0, policy_version 201909 (0.0007) [2023-12-27 07:03:00,829][20381] Updated weights for policy 1, policy_version 201560 (0.0006) [2023-12-27 07:03:00,856][20347] Updated weights for policy 0, policy_version 201919 (0.0005) [2023-12-27 07:03:00,884][20381] Updated weights for policy 1, policy_version 201570 (0.0006) [2023-12-27 07:03:00,943][20381] Updated weights for policy 1, policy_version 201580 (0.0005) [2023-12-27 07:03:01,519][20347] Updated weights for policy 0, policy_version 201929 (0.0008) [2023-12-27 07:03:01,583][20347] Updated weights for policy 0, policy_version 201939 (0.0006) [2023-12-27 07:03:01,608][20381] Updated weights for policy 1, policy_version 201590 (0.0006) [2023-12-27 07:03:01,644][20347] Updated weights for policy 0, policy_version 201949 (0.0008) [2023-12-27 07:03:01,673][20381] Updated weights for policy 1, policy_version 201600 (0.0007) [2023-12-27 07:03:01,705][20347] Updated weights for policy 0, policy_version 201959 (0.0006) [2023-12-27 07:03:01,740][20381] Updated weights for policy 1, policy_version 201610 (0.0007) [2023-12-27 07:03:02,292][20347] Updated weights for policy 0, policy_version 201969 (0.0007) [2023-12-27 07:03:02,347][20347] Updated weights for policy 0, policy_version 201979 (0.0008) [2023-12-27 07:03:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 103342080. Throughput: 0: 10405.2, 1: 10290.6. Samples: 103357392. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:03:02,410][20347] Updated weights for policy 0, policy_version 201989 (0.0007) [2023-12-27 07:03:02,471][20381] Updated weights for policy 1, policy_version 201620 (0.0008) [2023-12-27 07:03:02,516][20381] Updated weights for policy 1, policy_version 201630 (0.0007) [2023-12-27 07:03:02,567][20381] Updated weights for policy 1, policy_version 201640 (0.0009) [2023-12-27 07:03:03,036][20347] Updated weights for policy 0, policy_version 201999 (0.0006) [2023-12-27 07:03:03,083][20347] Updated weights for policy 0, policy_version 202009 (0.0007) [2023-12-27 07:03:03,128][20347] Updated weights for policy 0, policy_version 202019 (0.0007) [2023-12-27 07:03:03,298][20381] Updated weights for policy 1, policy_version 201650 (0.0008) [2023-12-27 07:03:03,350][20381] Updated weights for policy 1, policy_version 201660 (0.0007) [2023-12-27 07:03:03,408][20381] Updated weights for policy 1, policy_version 201670 (0.0008) [2023-12-27 07:03:03,458][20381] Updated weights for policy 1, policy_version 201680 (0.0007) [2023-12-27 07:03:03,757][20347] Updated weights for policy 0, policy_version 202029 (0.0006) [2023-12-27 07:03:03,802][20347] Updated weights for policy 0, policy_version 202039 (0.0005) [2023-12-27 07:03:03,851][20347] Updated weights for policy 0, policy_version 202049 (0.0008) [2023-12-27 07:03:04,190][20381] Updated weights for policy 1, policy_version 201690 (0.0007) [2023-12-27 07:03:04,245][20381] Updated weights for policy 1, policy_version 201700 (0.0008) [2023-12-27 07:03:04,304][20381] Updated weights for policy 1, policy_version 201710 (0.0008) [2023-12-27 07:03:04,474][20347] Updated weights for policy 0, policy_version 202059 (0.0007) [2023-12-27 07:03:04,531][20347] Updated weights for policy 0, policy_version 202069 (0.0006) [2023-12-27 07:03:04,590][20347] Updated weights for policy 0, policy_version 202079 (0.0005) [2023-12-27 07:03:05,024][20381] Updated weights for policy 1, policy_version 201720 (0.0007) [2023-12-27 07:03:05,069][20381] Updated weights for policy 1, policy_version 201730 (0.0008) [2023-12-27 07:03:05,118][20381] Updated weights for policy 1, policy_version 201740 (0.0007) [2023-12-27 07:03:05,255][20347] Updated weights for policy 0, policy_version 202089 (0.0008) [2023-12-27 07:03:05,317][20347] Updated weights for policy 0, policy_version 202099 (0.0008) [2023-12-27 07:03:05,379][20347] Updated weights for policy 0, policy_version 202109 (0.0007) [2023-12-27 07:03:05,436][20347] Updated weights for policy 0, policy_version 202119 (0.0007) [2023-12-27 07:03:05,780][20381] Updated weights for policy 1, policy_version 201750 (0.0006) [2023-12-27 07:03:05,845][20381] Updated weights for policy 1, policy_version 201760 (0.0007) [2023-12-27 07:03:05,904][20381] Updated weights for policy 1, policy_version 201770 (0.0009) [2023-12-27 07:03:06,079][20347] Updated weights for policy 0, policy_version 202129 (0.0010) [2023-12-27 07:03:06,137][20347] Updated weights for policy 0, policy_version 202139 (0.0010) [2023-12-27 07:03:06,198][20347] Updated weights for policy 0, policy_version 202149 (0.0009) [2023-12-27 07:03:06,636][20381] Updated weights for policy 1, policy_version 201780 (0.0008) [2023-12-27 07:03:06,701][20381] Updated weights for policy 1, policy_version 201790 (0.0007) [2023-12-27 07:03:06,759][20381] Updated weights for policy 1, policy_version 201800 (0.0007) [2023-12-27 07:03:06,861][20347] Updated weights for policy 0, policy_version 202159 (0.0010) [2023-12-27 07:03:06,926][20347] Updated weights for policy 0, policy_version 202169 (0.0010) [2023-12-27 07:03:06,982][20347] Updated weights for policy 0, policy_version 202179 (0.0010) [2023-12-27 07:03:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 103448576. Throughput: 0: 10468.9, 1: 10246.1. Samples: 103419544. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:03:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000201808_51675136.pth... [2023-12-27 07:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000202184_51773440.pth... [2023-12-27 07:03:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000200624_51372032.pth [2023-12-27 07:03:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000200968_51462144.pth [2023-12-27 07:03:07,459][20381] Updated weights for policy 1, policy_version 201810 (0.0008) [2023-12-27 07:03:07,514][20381] Updated weights for policy 1, policy_version 201820 (0.0008) [2023-12-27 07:03:07,565][20381] Updated weights for policy 1, policy_version 201830 (0.0007) [2023-12-27 07:03:07,619][20381] Updated weights for policy 1, policy_version 201840 (0.0007) [2023-12-27 07:03:07,650][20347] Updated weights for policy 0, policy_version 202189 (0.0010) [2023-12-27 07:03:07,718][20347] Updated weights for policy 0, policy_version 202199 (0.0010) [2023-12-27 07:03:07,777][20347] Updated weights for policy 0, policy_version 202209 (0.0010) [2023-12-27 07:03:08,320][20381] Updated weights for policy 1, policy_version 201850 (0.0007) [2023-12-27 07:03:08,385][20381] Updated weights for policy 1, policy_version 201860 (0.0008) [2023-12-27 07:03:08,437][20347] Updated weights for policy 0, policy_version 202219 (0.0009) [2023-12-27 07:03:08,445][20381] Updated weights for policy 1, policy_version 201870 (0.0007) [2023-12-27 07:03:08,491][20347] Updated weights for policy 0, policy_version 202229 (0.0007) [2023-12-27 07:03:08,542][20347] Updated weights for policy 0, policy_version 202239 (0.0007) [2023-12-27 07:03:09,052][20381] Updated weights for policy 1, policy_version 201880 (0.0008) [2023-12-27 07:03:09,110][20381] Updated weights for policy 1, policy_version 201890 (0.0008) [2023-12-27 07:03:09,174][20381] Updated weights for policy 1, policy_version 201900 (0.0006) [2023-12-27 07:03:09,187][20347] Updated weights for policy 0, policy_version 202249 (0.0008) [2023-12-27 07:03:09,248][20347] Updated weights for policy 0, policy_version 202259 (0.0010) [2023-12-27 07:03:09,312][20347] Updated weights for policy 0, policy_version 202269 (0.0009) [2023-12-27 07:03:09,384][20347] Updated weights for policy 0, policy_version 202279 (0.0007) [2023-12-27 07:03:09,881][20381] Updated weights for policy 1, policy_version 201910 (0.0007) [2023-12-27 07:03:09,945][20381] Updated weights for policy 1, policy_version 201920 (0.0008) [2023-12-27 07:03:10,009][20381] Updated weights for policy 1, policy_version 201930 (0.0006) [2023-12-27 07:03:10,021][20347] Updated weights for policy 0, policy_version 202289 (0.0008) [2023-12-27 07:03:10,082][20347] Updated weights for policy 0, policy_version 202299 (0.0006) [2023-12-27 07:03:10,135][20347] Updated weights for policy 0, policy_version 202309 (0.0005) [2023-12-27 07:03:10,711][20381] Updated weights for policy 1, policy_version 201940 (0.0008) [2023-12-27 07:03:10,769][20381] Updated weights for policy 1, policy_version 201950 (0.0009) [2023-12-27 07:03:10,819][20347] Updated weights for policy 0, policy_version 202319 (0.0006) [2023-12-27 07:03:10,820][20381] Updated weights for policy 1, policy_version 201960 (0.0007) [2023-12-27 07:03:10,877][20347] Updated weights for policy 0, policy_version 202329 (0.0007) [2023-12-27 07:03:10,932][20347] Updated weights for policy 0, policy_version 202339 (0.0006) [2023-12-27 07:03:11,576][20381] Updated weights for policy 1, policy_version 201970 (0.0005) [2023-12-27 07:03:11,641][20381] Updated weights for policy 1, policy_version 201980 (0.0007) [2023-12-27 07:03:11,673][20347] Updated weights for policy 0, policy_version 202349 (0.0007) [2023-12-27 07:03:11,710][20381] Updated weights for policy 1, policy_version 201990 (0.0007) [2023-12-27 07:03:11,743][20347] Updated weights for policy 0, policy_version 202359 (0.0008) [2023-12-27 07:03:11,773][20381] Updated weights for policy 1, policy_version 202000 (0.0008) [2023-12-27 07:03:11,805][20347] Updated weights for policy 0, policy_version 202369 (0.0007) [2023-12-27 07:03:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 103546880. Throughput: 0: 10476.4, 1: 10221.2. Samples: 103542308. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:03:12,424][20347] Updated weights for policy 0, policy_version 202379 (0.0006) [2023-12-27 07:03:12,476][20381] Updated weights for policy 1, policy_version 202010 (0.0008) [2023-12-27 07:03:12,478][20347] Updated weights for policy 0, policy_version 202389 (0.0007) [2023-12-27 07:03:12,532][20347] Updated weights for policy 0, policy_version 202399 (0.0005) [2023-12-27 07:03:12,534][20381] Updated weights for policy 1, policy_version 202020 (0.0006) [2023-12-27 07:03:12,590][20381] Updated weights for policy 1, policy_version 202030 (0.0006) [2023-12-27 07:03:13,137][20347] Updated weights for policy 0, policy_version 202409 (0.0007) [2023-12-27 07:03:13,193][20347] Updated weights for policy 0, policy_version 202419 (0.0006) [2023-12-27 07:03:13,245][20381] Updated weights for policy 1, policy_version 202040 (0.0007) [2023-12-27 07:03:13,254][20347] Updated weights for policy 0, policy_version 202429 (0.0007) [2023-12-27 07:03:13,298][20381] Updated weights for policy 1, policy_version 202050 (0.0008) [2023-12-27 07:03:13,303][20347] Updated weights for policy 0, policy_version 202439 (0.0007) [2023-12-27 07:03:13,356][20381] Updated weights for policy 1, policy_version 202060 (0.0011) [2023-12-27 07:03:13,935][20347] Updated weights for policy 0, policy_version 202449 (0.0005) [2023-12-27 07:03:13,992][20347] Updated weights for policy 0, policy_version 202459 (0.0006) [2023-12-27 07:03:14,023][20381] Updated weights for policy 1, policy_version 202070 (0.0009) [2023-12-27 07:03:14,043][20347] Updated weights for policy 0, policy_version 202469 (0.0007) [2023-12-27 07:03:14,085][20381] Updated weights for policy 1, policy_version 202080 (0.0007) [2023-12-27 07:03:14,137][20381] Updated weights for policy 1, policy_version 202090 (0.0010) [2023-12-27 07:03:14,690][20347] Updated weights for policy 0, policy_version 202479 (0.0007) [2023-12-27 07:03:14,751][20347] Updated weights for policy 0, policy_version 202489 (0.0008) [2023-12-27 07:03:14,818][20347] Updated weights for policy 0, policy_version 202499 (0.0007) [2023-12-27 07:03:14,821][20381] Updated weights for policy 1, policy_version 202100 (0.0009) [2023-12-27 07:03:14,883][20381] Updated weights for policy 1, policy_version 202110 (0.0007) [2023-12-27 07:03:14,944][20381] Updated weights for policy 1, policy_version 202120 (0.0006) [2023-12-27 07:03:15,501][20347] Updated weights for policy 0, policy_version 202509 (0.0007) [2023-12-27 07:03:15,553][20381] Updated weights for policy 1, policy_version 202130 (0.0006) [2023-12-27 07:03:15,553][20347] Updated weights for policy 0, policy_version 202519 (0.0008) [2023-12-27 07:03:15,600][20347] Updated weights for policy 0, policy_version 202529 (0.0008) [2023-12-27 07:03:15,604][20381] Updated weights for policy 1, policy_version 202140 (0.0005) [2023-12-27 07:03:15,652][20381] Updated weights for policy 1, policy_version 202150 (0.0005) [2023-12-27 07:03:15,695][20381] Updated weights for policy 1, policy_version 202160 (0.0005) [2023-12-27 07:03:16,314][20347] Updated weights for policy 0, policy_version 202539 (0.0007) [2023-12-27 07:03:16,359][20381] Updated weights for policy 1, policy_version 202170 (0.0008) [2023-12-27 07:03:16,362][20347] Updated weights for policy 0, policy_version 202549 (0.0006) [2023-12-27 07:03:16,412][20347] Updated weights for policy 0, policy_version 202559 (0.0006) [2023-12-27 07:03:16,423][20381] Updated weights for policy 1, policy_version 202180 (0.0007) [2023-12-27 07:03:16,482][20381] Updated weights for policy 1, policy_version 202190 (0.0007) [2023-12-27 07:03:17,119][20347] Updated weights for policy 0, policy_version 202569 (0.0006) [2023-12-27 07:03:17,154][20381] Updated weights for policy 1, policy_version 202200 (0.0006) [2023-12-27 07:03:17,172][20347] Updated weights for policy 0, policy_version 202579 (0.0006) [2023-12-27 07:03:17,211][20381] Updated weights for policy 1, policy_version 202210 (0.0006) [2023-12-27 07:03:17,223][20347] Updated weights for policy 0, policy_version 202589 (0.0006) [2023-12-27 07:03:17,266][20381] Updated weights for policy 1, policy_version 202220 (0.0006) [2023-12-27 07:03:17,268][20347] Updated weights for policy 0, policy_version 202599 (0.0005) [2023-12-27 07:03:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 103661568. Throughput: 0: 10483.7, 1: 10166.1. Samples: 103666172. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.740')] [2023-12-27 07:03:17,849][20381] Updated weights for policy 1, policy_version 202230 (0.0007) [2023-12-27 07:03:17,893][20381] Updated weights for policy 1, policy_version 202240 (0.0008) [2023-12-27 07:03:17,942][20381] Updated weights for policy 1, policy_version 202250 (0.0007) [2023-12-27 07:03:17,984][20347] Updated weights for policy 0, policy_version 202609 (0.0006) [2023-12-27 07:03:18,033][20347] Updated weights for policy 0, policy_version 202619 (0.0008) [2023-12-27 07:03:18,092][20347] Updated weights for policy 0, policy_version 202629 (0.0008) [2023-12-27 07:03:18,572][20381] Updated weights for policy 1, policy_version 202260 (0.0007) [2023-12-27 07:03:18,624][20381] Updated weights for policy 1, policy_version 202270 (0.0007) [2023-12-27 07:03:18,683][20381] Updated weights for policy 1, policy_version 202280 (0.0005) [2023-12-27 07:03:18,842][20347] Updated weights for policy 0, policy_version 202639 (0.0008) [2023-12-27 07:03:18,902][20347] Updated weights for policy 0, policy_version 202649 (0.0007) [2023-12-27 07:03:18,970][20347] Updated weights for policy 0, policy_version 202659 (0.0008) [2023-12-27 07:03:19,328][20381] Updated weights for policy 1, policy_version 202290 (0.0006) [2023-12-27 07:03:19,396][20381] Updated weights for policy 1, policy_version 202300 (0.0007) [2023-12-27 07:03:19,450][20381] Updated weights for policy 1, policy_version 202310 (0.0005) [2023-12-27 07:03:19,511][20381] Updated weights for policy 1, policy_version 202320 (0.0007) [2023-12-27 07:03:19,688][20347] Updated weights for policy 0, policy_version 202669 (0.0007) [2023-12-27 07:03:19,751][20347] Updated weights for policy 0, policy_version 202679 (0.0008) [2023-12-27 07:03:19,814][20347] Updated weights for policy 0, policy_version 202689 (0.0008) [2023-12-27 07:03:20,154][20381] Updated weights for policy 1, policy_version 202330 (0.0008) [2023-12-27 07:03:20,205][20381] Updated weights for policy 1, policy_version 202340 (0.0008) [2023-12-27 07:03:20,256][20381] Updated weights for policy 1, policy_version 202350 (0.0007) [2023-12-27 07:03:20,535][20347] Updated weights for policy 0, policy_version 202699 (0.0008) [2023-12-27 07:03:20,590][20291] KL-divergence is very high: 196.6131 [2023-12-27 07:03:20,596][20347] Updated weights for policy 0, policy_version 202709 (0.0007) [2023-12-27 07:03:20,597][20291] KL-divergence is very high: 205.1184 [2023-12-27 07:03:20,657][20347] Updated weights for policy 0, policy_version 202719 (0.0007) [2023-12-27 07:03:20,943][20381] Updated weights for policy 1, policy_version 202360 (0.0008) [2023-12-27 07:03:21,000][20381] Updated weights for policy 1, policy_version 202370 (0.0008) [2023-12-27 07:03:21,065][20381] Updated weights for policy 1, policy_version 202380 (0.0008) [2023-12-27 07:03:21,418][20347] Updated weights for policy 0, policy_version 202729 (0.0007) [2023-12-27 07:03:21,470][20347] Updated weights for policy 0, policy_version 202739 (0.0007) [2023-12-27 07:03:21,525][20347] Updated weights for policy 0, policy_version 202749 (0.0005) [2023-12-27 07:03:21,578][20347] Updated weights for policy 0, policy_version 202759 (0.0005) [2023-12-27 07:03:21,849][20381] Updated weights for policy 1, policy_version 202390 (0.0008) [2023-12-27 07:03:21,911][20381] Updated weights for policy 1, policy_version 202400 (0.0008) [2023-12-27 07:03:21,963][20381] Updated weights for policy 1, policy_version 202410 (0.0008) [2023-12-27 07:03:22,285][20347] Updated weights for policy 0, policy_version 202769 (0.0008) [2023-12-27 07:03:22,343][20347] Updated weights for policy 0, policy_version 202779 (0.0008) [2023-12-27 07:03:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 103751680. Throughput: 0: 10395.7, 1: 10235.3. Samples: 103727116. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:22,363][19170] Avg episode reward: [(0, '999.420'), (1, '990.740')] [2023-12-27 07:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000202416_51830784.pth... [2023-12-27 07:03:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000201232_51527680.pth [2023-12-27 07:03:22,404][20347] Updated weights for policy 0, policy_version 202789 (0.0008) [2023-12-27 07:03:22,419][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000202792_51929088.pth... [2023-12-27 07:03:22,423][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000201576_51617792.pth [2023-12-27 07:03:22,686][20381] Updated weights for policy 1, policy_version 202420 (0.0009) [2023-12-27 07:03:22,736][20381] Updated weights for policy 1, policy_version 202430 (0.0008) [2023-12-27 07:03:22,797][20381] Updated weights for policy 1, policy_version 202440 (0.0008) [2023-12-27 07:03:23,073][20347] Updated weights for policy 0, policy_version 202799 (0.0008) [2023-12-27 07:03:23,128][20347] Updated weights for policy 0, policy_version 202809 (0.0007) [2023-12-27 07:03:23,184][20347] Updated weights for policy 0, policy_version 202819 (0.0008) [2023-12-27 07:03:23,454][20381] Updated weights for policy 1, policy_version 202450 (0.0008) [2023-12-27 07:03:23,506][20381] Updated weights for policy 1, policy_version 202460 (0.0007) [2023-12-27 07:03:23,562][20381] Updated weights for policy 1, policy_version 202470 (0.0008) [2023-12-27 07:03:23,614][20381] Updated weights for policy 1, policy_version 202480 (0.0008) [2023-12-27 07:03:23,889][20347] Updated weights for policy 0, policy_version 202829 (0.0007) [2023-12-27 07:03:23,939][20347] Updated weights for policy 0, policy_version 202839 (0.0008) [2023-12-27 07:03:23,988][20347] Updated weights for policy 0, policy_version 202849 (0.0007) [2023-12-27 07:03:24,319][20381] Updated weights for policy 1, policy_version 202490 (0.0008) [2023-12-27 07:03:24,379][20381] Updated weights for policy 1, policy_version 202500 (0.0008) [2023-12-27 07:03:24,432][20381] Updated weights for policy 1, policy_version 202510 (0.0007) [2023-12-27 07:03:24,626][20347] Updated weights for policy 0, policy_version 202859 (0.0009) [2023-12-27 07:03:24,677][20347] Updated weights for policy 0, policy_version 202869 (0.0010) [2023-12-27 07:03:24,722][20347] Updated weights for policy 0, policy_version 202879 (0.0010) [2023-12-27 07:03:25,101][20381] Updated weights for policy 1, policy_version 202520 (0.0006) [2023-12-27 07:03:25,149][20381] Updated weights for policy 1, policy_version 202530 (0.0006) [2023-12-27 07:03:25,204][20381] Updated weights for policy 1, policy_version 202540 (0.0008) [2023-12-27 07:03:25,392][20347] Updated weights for policy 0, policy_version 202889 (0.0010) [2023-12-27 07:03:25,442][20347] Updated weights for policy 0, policy_version 202899 (0.0010) [2023-12-27 07:03:25,497][20347] Updated weights for policy 0, policy_version 202909 (0.0007) [2023-12-27 07:03:25,561][20347] Updated weights for policy 0, policy_version 202919 (0.0005) [2023-12-27 07:03:25,893][20381] Updated weights for policy 1, policy_version 202550 (0.0007) [2023-12-27 07:03:25,945][20381] Updated weights for policy 1, policy_version 202560 (0.0007) [2023-12-27 07:03:26,000][20381] Updated weights for policy 1, policy_version 202570 (0.0009) [2023-12-27 07:03:26,135][20347] Updated weights for policy 0, policy_version 202929 (0.0005) [2023-12-27 07:03:26,183][20347] Updated weights for policy 0, policy_version 202939 (0.0005) [2023-12-27 07:03:26,235][20347] Updated weights for policy 0, policy_version 202949 (0.0008) [2023-12-27 07:03:26,714][20381] Updated weights for policy 1, policy_version 202580 (0.0007) [2023-12-27 07:03:26,761][20381] Updated weights for policy 1, policy_version 202590 (0.0007) [2023-12-27 07:03:26,805][20381] Updated weights for policy 1, policy_version 202600 (0.0006) [2023-12-27 07:03:26,880][20347] Updated weights for policy 0, policy_version 202959 (0.0010) [2023-12-27 07:03:26,930][20347] Updated weights for policy 0, policy_version 202969 (0.0010) [2023-12-27 07:03:26,978][20347] Updated weights for policy 0, policy_version 202979 (0.0010) [2023-12-27 07:03:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 103858176. Throughput: 0: 10346.4, 1: 10218.1. Samples: 103848560. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:27,363][19170] Avg episode reward: [(0, '999.420'), (1, '990.330')] [2023-12-27 07:03:27,471][20381] Updated weights for policy 1, policy_version 202610 (0.0007) [2023-12-27 07:03:27,529][20381] Updated weights for policy 1, policy_version 202620 (0.0010) [2023-12-27 07:03:27,583][20381] Updated weights for policy 1, policy_version 202630 (0.0010) [2023-12-27 07:03:27,637][20381] Updated weights for policy 1, policy_version 202640 (0.0010) [2023-12-27 07:03:27,644][20347] Updated weights for policy 0, policy_version 202989 (0.0010) [2023-12-27 07:03:27,695][20347] Updated weights for policy 0, policy_version 202999 (0.0010) [2023-12-27 07:03:27,748][20347] Updated weights for policy 0, policy_version 203009 (0.0010) [2023-12-27 07:03:28,267][20381] Updated weights for policy 1, policy_version 202650 (0.0005) [2023-12-27 07:03:28,328][20381] Updated weights for policy 1, policy_version 202660 (0.0009) [2023-12-27 07:03:28,380][20381] Updated weights for policy 1, policy_version 202670 (0.0007) [2023-12-27 07:03:28,424][20347] Updated weights for policy 0, policy_version 203019 (0.0010) [2023-12-27 07:03:28,485][20347] Updated weights for policy 0, policy_version 203029 (0.0010) [2023-12-27 07:03:28,550][20347] Updated weights for policy 0, policy_version 203039 (0.0010) [2023-12-27 07:03:29,012][20381] Updated weights for policy 1, policy_version 202680 (0.0009) [2023-12-27 07:03:29,062][20381] Updated weights for policy 1, policy_version 202690 (0.0010) [2023-12-27 07:03:29,115][20381] Updated weights for policy 1, policy_version 202700 (0.0010) [2023-12-27 07:03:29,206][20347] Updated weights for policy 0, policy_version 203049 (0.0011) [2023-12-27 07:03:29,269][20347] Updated weights for policy 0, policy_version 203059 (0.0009) [2023-12-27 07:03:29,328][20347] Updated weights for policy 0, policy_version 203069 (0.0007) [2023-12-27 07:03:29,386][20347] Updated weights for policy 0, policy_version 203079 (0.0007) [2023-12-27 07:03:29,790][20381] Updated weights for policy 1, policy_version 202710 (0.0007) [2023-12-27 07:03:29,854][20381] Updated weights for policy 1, policy_version 202720 (0.0006) [2023-12-27 07:03:29,905][20381] Updated weights for policy 1, policy_version 202730 (0.0008) [2023-12-27 07:03:30,083][20347] Updated weights for policy 0, policy_version 203089 (0.0008) [2023-12-27 07:03:30,130][20347] Updated weights for policy 0, policy_version 203099 (0.0008) [2023-12-27 07:03:30,178][20347] Updated weights for policy 0, policy_version 203109 (0.0008) [2023-12-27 07:03:30,496][20381] Updated weights for policy 1, policy_version 202740 (0.0008) [2023-12-27 07:03:30,548][20381] Updated weights for policy 1, policy_version 202750 (0.0007) [2023-12-27 07:03:30,597][20381] Updated weights for policy 1, policy_version 202760 (0.0008) [2023-12-27 07:03:30,884][20347] Updated weights for policy 0, policy_version 203119 (0.0007) [2023-12-27 07:03:30,934][20347] Updated weights for policy 0, policy_version 203129 (0.0007) [2023-12-27 07:03:30,990][20347] Updated weights for policy 0, policy_version 203139 (0.0005) [2023-12-27 07:03:31,337][20381] Updated weights for policy 1, policy_version 202770 (0.0008) [2023-12-27 07:03:31,402][20381] Updated weights for policy 1, policy_version 202780 (0.0006) [2023-12-27 07:03:31,451][20381] Updated weights for policy 1, policy_version 202790 (0.0006) [2023-12-27 07:03:31,508][20381] Updated weights for policy 1, policy_version 202800 (0.0007) [2023-12-27 07:03:31,616][20347] Updated weights for policy 0, policy_version 203149 (0.0007) [2023-12-27 07:03:31,674][20347] Updated weights for policy 0, policy_version 203159 (0.0007) [2023-12-27 07:03:31,733][20347] Updated weights for policy 0, policy_version 203169 (0.0007) [2023-12-27 07:03:32,212][20381] Updated weights for policy 1, policy_version 202810 (0.0008) [2023-12-27 07:03:32,268][20381] Updated weights for policy 1, policy_version 202820 (0.0007) [2023-12-27 07:03:32,329][20381] Updated weights for policy 1, policy_version 202830 (0.0008) [2023-12-27 07:03:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 103964672. Throughput: 0: 10365.0, 1: 10225.6. Samples: 103973876. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:03:32,418][20347] Updated weights for policy 0, policy_version 203179 (0.0008) [2023-12-27 07:03:32,475][20347] Updated weights for policy 0, policy_version 203189 (0.0007) [2023-12-27 07:03:32,526][20347] Updated weights for policy 0, policy_version 203199 (0.0008) [2023-12-27 07:03:33,027][20381] Updated weights for policy 1, policy_version 202840 (0.0006) [2023-12-27 07:03:33,076][20347] Updated weights for policy 0, policy_version 203209 (0.0007) [2023-12-27 07:03:33,086][20381] Updated weights for policy 1, policy_version 202850 (0.0005) [2023-12-27 07:03:33,132][20347] Updated weights for policy 0, policy_version 203219 (0.0005) [2023-12-27 07:03:33,142][20381] Updated weights for policy 1, policy_version 202860 (0.0007) [2023-12-27 07:03:33,187][20347] Updated weights for policy 0, policy_version 203229 (0.0006) [2023-12-27 07:03:33,246][20347] Updated weights for policy 0, policy_version 203239 (0.0008) [2023-12-27 07:03:33,784][20381] Updated weights for policy 1, policy_version 202870 (0.0007) [2023-12-27 07:03:33,839][20381] Updated weights for policy 1, policy_version 202880 (0.0005) [2023-12-27 07:03:33,857][20347] Updated weights for policy 0, policy_version 203249 (0.0006) [2023-12-27 07:03:33,894][20381] Updated weights for policy 1, policy_version 202890 (0.0005) [2023-12-27 07:03:33,901][20347] Updated weights for policy 0, policy_version 203259 (0.0007) [2023-12-27 07:03:33,948][20347] Updated weights for policy 0, policy_version 203269 (0.0006) [2023-12-27 07:03:34,528][20381] Updated weights for policy 1, policy_version 202900 (0.0006) [2023-12-27 07:03:34,576][20381] Updated weights for policy 1, policy_version 202910 (0.0008) [2023-12-27 07:03:34,619][20381] Updated weights for policy 1, policy_version 202920 (0.0008) [2023-12-27 07:03:34,659][20347] Updated weights for policy 0, policy_version 203279 (0.0008) [2023-12-27 07:03:34,712][20347] Updated weights for policy 0, policy_version 203289 (0.0007) [2023-12-27 07:03:34,768][20347] Updated weights for policy 0, policy_version 203299 (0.0007) [2023-12-27 07:03:35,255][20381] Updated weights for policy 1, policy_version 202930 (0.0007) [2023-12-27 07:03:35,311][20381] Updated weights for policy 1, policy_version 202940 (0.0006) [2023-12-27 07:03:35,367][20381] Updated weights for policy 1, policy_version 202950 (0.0007) [2023-12-27 07:03:35,415][20381] Updated weights for policy 1, policy_version 202960 (0.0007) [2023-12-27 07:03:35,483][20347] Updated weights for policy 0, policy_version 203309 (0.0007) [2023-12-27 07:03:35,531][20347] Updated weights for policy 0, policy_version 203319 (0.0006) [2023-12-27 07:03:35,583][20347] Updated weights for policy 0, policy_version 203329 (0.0005) [2023-12-27 07:03:36,056][20381] Updated weights for policy 1, policy_version 202970 (0.0006) [2023-12-27 07:03:36,120][20381] Updated weights for policy 1, policy_version 202980 (0.0008) [2023-12-27 07:03:36,178][20381] Updated weights for policy 1, policy_version 202990 (0.0007) [2023-12-27 07:03:36,225][20347] Updated weights for policy 0, policy_version 203339 (0.0007) [2023-12-27 07:03:36,291][20347] Updated weights for policy 0, policy_version 203349 (0.0010) [2023-12-27 07:03:36,350][20347] Updated weights for policy 0, policy_version 203359 (0.0010) [2023-12-27 07:03:36,874][20381] Updated weights for policy 1, policy_version 203000 (0.0008) [2023-12-27 07:03:36,926][20381] Updated weights for policy 1, policy_version 203010 (0.0008) [2023-12-27 07:03:36,962][20347] Updated weights for policy 0, policy_version 203369 (0.0010) [2023-12-27 07:03:36,983][20381] Updated weights for policy 1, policy_version 203020 (0.0008) [2023-12-27 07:03:37,027][20347] Updated weights for policy 0, policy_version 203379 (0.0006) [2023-12-27 07:03:37,099][20347] Updated weights for policy 0, policy_version 203389 (0.0008) [2023-12-27 07:03:37,161][20347] Updated weights for policy 0, policy_version 203399 (0.0008) [2023-12-27 07:03:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 104071168. Throughput: 0: 10375.3, 1: 10292.6. Samples: 104038208. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 07:03:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:03:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000203024_51986432.pth... [2023-12-27 07:03:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000203400_52084736.pth... [2023-12-27 07:03:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000201808_51675136.pth [2023-12-27 07:03:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000202184_51773440.pth [2023-12-27 07:03:37,609][20381] Updated weights for policy 1, policy_version 203030 (0.0008) [2023-12-27 07:03:37,665][20381] Updated weights for policy 1, policy_version 203040 (0.0007) [2023-12-27 07:03:37,720][20381] Updated weights for policy 1, policy_version 203050 (0.0007) [2023-12-27 07:03:37,810][20347] Updated weights for policy 0, policy_version 203409 (0.0010) [2023-12-27 07:03:37,867][20347] Updated weights for policy 0, policy_version 203419 (0.0010) [2023-12-27 07:03:37,925][20347] Updated weights for policy 0, policy_version 203429 (0.0010) [2023-12-27 07:03:38,372][20381] Updated weights for policy 1, policy_version 203060 (0.0007) [2023-12-27 07:03:38,433][20381] Updated weights for policy 1, policy_version 203070 (0.0008) [2023-12-27 07:03:38,440][20292] KL-divergence is very high: 406.7484 [2023-12-27 07:03:38,496][20381] Updated weights for policy 1, policy_version 203080 (0.0005) [2023-12-27 07:03:38,566][20347] Updated weights for policy 0, policy_version 203439 (0.0009) [2023-12-27 07:03:38,618][20347] Updated weights for policy 0, policy_version 203449 (0.0010) [2023-12-27 07:03:38,667][20347] Updated weights for policy 0, policy_version 203459 (0.0010) [2023-12-27 07:03:39,145][20381] Updated weights for policy 1, policy_version 203090 (0.0006) [2023-12-27 07:03:39,199][20381] Updated weights for policy 1, policy_version 203100 (0.0006) [2023-12-27 07:03:39,261][20381] Updated weights for policy 1, policy_version 203110 (0.0008) [2023-12-27 07:03:39,317][20381] Updated weights for policy 1, policy_version 203120 (0.0008) [2023-12-27 07:03:39,382][20347] Updated weights for policy 0, policy_version 203469 (0.0009) [2023-12-27 07:03:39,442][20347] Updated weights for policy 0, policy_version 203479 (0.0008) [2023-12-27 07:03:39,497][20347] Updated weights for policy 0, policy_version 203489 (0.0007) [2023-12-27 07:03:40,023][20381] Updated weights for policy 1, policy_version 203130 (0.0008) [2023-12-27 07:03:40,081][20381] Updated weights for policy 1, policy_version 203140 (0.0008) [2023-12-27 07:03:40,137][20381] Updated weights for policy 1, policy_version 203150 (0.0009) [2023-12-27 07:03:40,178][20347] Updated weights for policy 0, policy_version 203499 (0.0010) [2023-12-27 07:03:40,238][20347] Updated weights for policy 0, policy_version 203509 (0.0008) [2023-12-27 07:03:40,292][20347] Updated weights for policy 0, policy_version 203519 (0.0009) [2023-12-27 07:03:40,805][20381] Updated weights for policy 1, policy_version 203160 (0.0006) [2023-12-27 07:03:40,853][20381] Updated weights for policy 1, policy_version 203170 (0.0005) [2023-12-27 07:03:40,910][20381] Updated weights for policy 1, policy_version 203180 (0.0007) [2023-12-27 07:03:40,972][20347] Updated weights for policy 0, policy_version 203529 (0.0008) [2023-12-27 07:03:41,029][20347] Updated weights for policy 0, policy_version 203539 (0.0009) [2023-12-27 07:03:41,091][20347] Updated weights for policy 0, policy_version 203549 (0.0008) [2023-12-27 07:03:41,154][20347] Updated weights for policy 0, policy_version 203559 (0.0008) [2023-12-27 07:03:41,618][20381] Updated weights for policy 1, policy_version 203190 (0.0009) [2023-12-27 07:03:41,690][20381] Updated weights for policy 1, policy_version 203200 (0.0009) [2023-12-27 07:03:41,757][20381] Updated weights for policy 1, policy_version 203210 (0.0008) [2023-12-27 07:03:41,937][20347] Updated weights for policy 0, policy_version 203569 (0.0007) [2023-12-27 07:03:41,998][20347] Updated weights for policy 0, policy_version 203579 (0.0007) [2023-12-27 07:03:42,055][20347] Updated weights for policy 0, policy_version 203589 (0.0007) [2023-12-27 07:03:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 104169472. Throughput: 0: 10386.5, 1: 10287.0. Samples: 104162536. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:03:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.880')] [2023-12-27 07:03:42,457][20381] Updated weights for policy 1, policy_version 203220 (0.0008) [2023-12-27 07:03:42,513][20381] Updated weights for policy 1, policy_version 203230 (0.0007) [2023-12-27 07:03:42,573][20381] Updated weights for policy 1, policy_version 203240 (0.0008) [2023-12-27 07:03:42,747][20347] Updated weights for policy 0, policy_version 203599 (0.0007) [2023-12-27 07:03:42,812][20347] Updated weights for policy 0, policy_version 203609 (0.0008) [2023-12-27 07:03:42,874][20347] Updated weights for policy 0, policy_version 203619 (0.0008) [2023-12-27 07:03:43,209][20381] Updated weights for policy 1, policy_version 203250 (0.0008) [2023-12-27 07:03:43,260][20381] Updated weights for policy 1, policy_version 203260 (0.0008) [2023-12-27 07:03:43,308][20381] Updated weights for policy 1, policy_version 203270 (0.0007) [2023-12-27 07:03:43,362][20381] Updated weights for policy 1, policy_version 203280 (0.0008) [2023-12-27 07:03:43,516][20347] Updated weights for policy 0, policy_version 203629 (0.0006) [2023-12-27 07:03:43,569][20347] Updated weights for policy 0, policy_version 203639 (0.0007) [2023-12-27 07:03:43,623][20347] Updated weights for policy 0, policy_version 203649 (0.0006) [2023-12-27 07:03:44,113][20381] Updated weights for policy 1, policy_version 203290 (0.0008) [2023-12-27 07:03:44,170][20381] Updated weights for policy 1, policy_version 203300 (0.0008) [2023-12-27 07:03:44,211][20347] Updated weights for policy 0, policy_version 203659 (0.0006) [2023-12-27 07:03:44,228][20381] Updated weights for policy 1, policy_version 203310 (0.0007) [2023-12-27 07:03:44,269][20347] Updated weights for policy 0, policy_version 203669 (0.0007) [2023-12-27 07:03:44,331][20347] Updated weights for policy 0, policy_version 203679 (0.0009) [2023-12-27 07:03:44,910][20381] Updated weights for policy 1, policy_version 203320 (0.0005) [2023-12-27 07:03:44,970][20381] Updated weights for policy 1, policy_version 203330 (0.0006) [2023-12-27 07:03:45,024][20381] Updated weights for policy 1, policy_version 203340 (0.0008) [2023-12-27 07:03:45,029][20347] Updated weights for policy 0, policy_version 203689 (0.0010) [2023-12-27 07:03:45,097][20347] Updated weights for policy 0, policy_version 203699 (0.0010) [2023-12-27 07:03:45,156][20347] Updated weights for policy 0, policy_version 203709 (0.0010) [2023-12-27 07:03:45,211][20347] Updated weights for policy 0, policy_version 203719 (0.0007) [2023-12-27 07:03:45,623][20381] Updated weights for policy 1, policy_version 203350 (0.0007) [2023-12-27 07:03:45,679][20381] Updated weights for policy 1, policy_version 203360 (0.0008) [2023-12-27 07:03:45,737][20381] Updated weights for policy 1, policy_version 203370 (0.0009) [2023-12-27 07:03:45,808][20347] Updated weights for policy 0, policy_version 203729 (0.0010) [2023-12-27 07:03:45,858][20347] Updated weights for policy 0, policy_version 203740 (0.0009) [2023-12-27 07:03:45,905][20347] Updated weights for policy 0, policy_version 203750 (0.0007) [2023-12-27 07:03:46,339][20381] Updated weights for policy 1, policy_version 203380 (0.0009) [2023-12-27 07:03:46,400][20381] Updated weights for policy 1, policy_version 203390 (0.0008) [2023-12-27 07:03:46,462][20381] Updated weights for policy 1, policy_version 203400 (0.0006) [2023-12-27 07:03:46,614][20347] Updated weights for policy 0, policy_version 203760 (0.0006) [2023-12-27 07:03:46,673][20347] Updated weights for policy 0, policy_version 203770 (0.0007) [2023-12-27 07:03:46,722][20347] Updated weights for policy 0, policy_version 203780 (0.0006) [2023-12-27 07:03:47,046][20381] Updated weights for policy 1, policy_version 203410 (0.0007) [2023-12-27 07:03:47,113][20381] Updated weights for policy 1, policy_version 203420 (0.0005) [2023-12-27 07:03:47,174][20381] Updated weights for policy 1, policy_version 203430 (0.0006) [2023-12-27 07:03:47,223][20381] Updated weights for policy 1, policy_version 203440 (0.0007) [2023-12-27 07:03:47,273][20347] Updated weights for policy 0, policy_version 203790 (0.0008) [2023-12-27 07:03:47,321][20347] Updated weights for policy 0, policy_version 203800 (0.0010) [2023-12-27 07:03:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 104275968. Throughput: 0: 10390.2, 1: 10299.0. Samples: 104288400. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:03:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.880')] [2023-12-27 07:03:47,376][20347] Updated weights for policy 0, policy_version 203810 (0.0010) [2023-12-27 07:03:47,856][20381] Updated weights for policy 1, policy_version 203450 (0.0010) [2023-12-27 07:03:47,908][20381] Updated weights for policy 1, policy_version 203460 (0.0005) [2023-12-27 07:03:47,958][20381] Updated weights for policy 1, policy_version 203470 (0.0005) [2023-12-27 07:03:48,014][20347] Updated weights for policy 0, policy_version 203820 (0.0010) [2023-12-27 07:03:48,071][20347] Updated weights for policy 0, policy_version 203830 (0.0010) [2023-12-27 07:03:48,127][20347] Updated weights for policy 0, policy_version 203840 (0.0010) [2023-12-27 07:03:48,603][20381] Updated weights for policy 1, policy_version 203480 (0.0005) [2023-12-27 07:03:48,657][20381] Updated weights for policy 1, policy_version 203490 (0.0010) [2023-12-27 07:03:48,713][20381] Updated weights for policy 1, policy_version 203500 (0.0005) [2023-12-27 07:03:48,844][20347] Updated weights for policy 0, policy_version 203850 (0.0010) [2023-12-27 07:03:48,908][20347] Updated weights for policy 0, policy_version 203860 (0.0007) [2023-12-27 07:03:48,962][20347] Updated weights for policy 0, policy_version 203870 (0.0008) [2023-12-27 07:03:49,010][20347] Updated weights for policy 0, policy_version 203880 (0.0007) [2023-12-27 07:03:49,311][20381] Updated weights for policy 1, policy_version 203510 (0.0008) [2023-12-27 07:03:49,367][20381] Updated weights for policy 1, policy_version 203520 (0.0010) [2023-12-27 07:03:49,428][20381] Updated weights for policy 1, policy_version 203530 (0.0010) [2023-12-27 07:03:49,710][20347] Updated weights for policy 0, policy_version 203890 (0.0006) [2023-12-27 07:03:49,774][20347] Updated weights for policy 0, policy_version 203900 (0.0007) [2023-12-27 07:03:49,838][20347] Updated weights for policy 0, policy_version 203910 (0.0007) [2023-12-27 07:03:50,080][20381] Updated weights for policy 1, policy_version 203540 (0.0010) [2023-12-27 07:03:50,135][20381] Updated weights for policy 1, policy_version 203550 (0.0010) [2023-12-27 07:03:50,187][20381] Updated weights for policy 1, policy_version 203560 (0.0010) [2023-12-27 07:03:50,515][20347] Updated weights for policy 0, policy_version 203920 (0.0007) [2023-12-27 07:03:50,573][20347] Updated weights for policy 0, policy_version 203930 (0.0006) [2023-12-27 07:03:50,633][20347] Updated weights for policy 0, policy_version 203940 (0.0006) [2023-12-27 07:03:50,827][20381] Updated weights for policy 1, policy_version 203570 (0.0009) [2023-12-27 07:03:50,888][20381] Updated weights for policy 1, policy_version 203580 (0.0010) [2023-12-27 07:03:50,951][20381] Updated weights for policy 1, policy_version 203590 (0.0009) [2023-12-27 07:03:51,008][20381] Updated weights for policy 1, policy_version 203600 (0.0010) [2023-12-27 07:03:51,247][20347] Updated weights for policy 0, policy_version 203950 (0.0007) [2023-12-27 07:03:51,307][20347] Updated weights for policy 0, policy_version 203960 (0.0006) [2023-12-27 07:03:51,368][20347] Updated weights for policy 0, policy_version 203970 (0.0009) [2023-12-27 07:03:51,637][20381] Updated weights for policy 1, policy_version 203610 (0.0008) [2023-12-27 07:03:51,700][20381] Updated weights for policy 1, policy_version 203620 (0.0009) [2023-12-27 07:03:51,758][20381] Updated weights for policy 1, policy_version 203630 (0.0007) [2023-12-27 07:03:52,117][20347] Updated weights for policy 0, policy_version 203980 (0.0007) [2023-12-27 07:03:52,162][20347] Updated weights for policy 0, policy_version 203990 (0.0008) [2023-12-27 07:03:52,217][20347] Updated weights for policy 0, policy_version 204000 (0.0008) [2023-12-27 07:03:52,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 104382464. Throughput: 0: 10344.4, 1: 10359.2. Samples: 104351208. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.480')] [2023-12-27 07:03:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000204008_52240384.pth... [2023-12-27 07:03:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000202792_51929088.pth [2023-12-27 07:03:52,415][20381] Updated weights for policy 1, policy_version 203640 (0.0007) [2023-12-27 07:03:52,476][20381] Updated weights for policy 1, policy_version 203650 (0.0008) [2023-12-27 07:03:52,540][20381] Updated weights for policy 1, policy_version 203660 (0.0008) [2023-12-27 07:03:52,561][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000203664_52150272.pth... [2023-12-27 07:03:52,564][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000202416_51830784.pth [2023-12-27 07:03:52,959][20347] Updated weights for policy 0, policy_version 204010 (0.0008) [2023-12-27 07:03:53,020][20347] Updated weights for policy 0, policy_version 204020 (0.0008) [2023-12-27 07:03:53,081][20347] Updated weights for policy 0, policy_version 204030 (0.0008) [2023-12-27 07:03:53,138][20347] Updated weights for policy 0, policy_version 204040 (0.0008) [2023-12-27 07:03:53,227][20381] Updated weights for policy 1, policy_version 203670 (0.0008) [2023-12-27 07:03:53,279][20381] Updated weights for policy 1, policy_version 203680 (0.0009) [2023-12-27 07:03:53,329][20381] Updated weights for policy 1, policy_version 203690 (0.0008) [2023-12-27 07:03:53,807][20347] Updated weights for policy 0, policy_version 204050 (0.0007) [2023-12-27 07:03:53,865][20347] Updated weights for policy 0, policy_version 204060 (0.0007) [2023-12-27 07:03:53,922][20347] Updated weights for policy 0, policy_version 204070 (0.0008) [2023-12-27 07:03:53,981][20381] Updated weights for policy 1, policy_version 203700 (0.0006) [2023-12-27 07:03:54,030][20381] Updated weights for policy 1, policy_version 203710 (0.0010) [2023-12-27 07:03:54,076][20381] Updated weights for policy 1, policy_version 203720 (0.0010) [2023-12-27 07:03:54,615][20347] Updated weights for policy 0, policy_version 204080 (0.0008) [2023-12-27 07:03:54,665][20347] Updated weights for policy 0, policy_version 204090 (0.0008) [2023-12-27 07:03:54,714][20381] Updated weights for policy 1, policy_version 203730 (0.0006) [2023-12-27 07:03:54,724][20347] Updated weights for policy 0, policy_version 204100 (0.0009) [2023-12-27 07:03:54,777][20381] Updated weights for policy 1, policy_version 203740 (0.0006) [2023-12-27 07:03:54,831][20381] Updated weights for policy 1, policy_version 203750 (0.0008) [2023-12-27 07:03:54,884][20381] Updated weights for policy 1, policy_version 203760 (0.0008) [2023-12-27 07:03:55,409][20347] Updated weights for policy 0, policy_version 204110 (0.0006) [2023-12-27 07:03:55,460][20347] Updated weights for policy 0, policy_version 204120 (0.0005) [2023-12-27 07:03:55,510][20347] Updated weights for policy 0, policy_version 204130 (0.0005) [2023-12-27 07:03:55,586][20381] Updated weights for policy 1, policy_version 203770 (0.0008) [2023-12-27 07:03:55,638][20381] Updated weights for policy 1, policy_version 203780 (0.0007) [2023-12-27 07:03:55,690][20381] Updated weights for policy 1, policy_version 203790 (0.0008) [2023-12-27 07:03:56,108][20347] Updated weights for policy 0, policy_version 204140 (0.0006) [2023-12-27 07:03:56,163][20347] Updated weights for policy 0, policy_version 204150 (0.0005) [2023-12-27 07:03:56,217][20347] Updated weights for policy 0, policy_version 204160 (0.0005) [2023-12-27 07:03:56,432][20381] Updated weights for policy 1, policy_version 203800 (0.0007) [2023-12-27 07:03:56,490][20381] Updated weights for policy 1, policy_version 203810 (0.0008) [2023-12-27 07:03:56,545][20381] Updated weights for policy 1, policy_version 203820 (0.0007) [2023-12-27 07:03:56,871][20347] Updated weights for policy 0, policy_version 204170 (0.0007) [2023-12-27 07:03:56,923][20347] Updated weights for policy 0, policy_version 204180 (0.0007) [2023-12-27 07:03:56,968][20347] Updated weights for policy 0, policy_version 204190 (0.0005) [2023-12-27 07:03:57,015][20347] Updated weights for policy 0, policy_version 204200 (0.0007) [2023-12-27 07:03:57,191][20381] Updated weights for policy 1, policy_version 203830 (0.0007) [2023-12-27 07:03:57,249][20381] Updated weights for policy 1, policy_version 203840 (0.0006) [2023-12-27 07:03:57,314][20381] Updated weights for policy 1, policy_version 203850 (0.0007) [2023-12-27 07:03:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 104488960. Throughput: 0: 10323.5, 1: 10399.1. Samples: 104474824. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:03:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '982.030')] [2023-12-27 07:03:57,744][20347] Updated weights for policy 0, policy_version 204210 (0.0008) [2023-12-27 07:03:57,793][20347] Updated weights for policy 0, policy_version 204220 (0.0007) [2023-12-27 07:03:57,834][20347] Updated weights for policy 0, policy_version 204230 (0.0007) [2023-12-27 07:03:57,860][20381] Updated weights for policy 1, policy_version 203860 (0.0008) [2023-12-27 07:03:57,908][20381] Updated weights for policy 1, policy_version 203870 (0.0005) [2023-12-27 07:03:57,962][20381] Updated weights for policy 1, policy_version 203880 (0.0005) [2023-12-27 07:03:58,554][20347] Updated weights for policy 0, policy_version 204240 (0.0008) [2023-12-27 07:03:58,614][20347] Updated weights for policy 0, policy_version 204250 (0.0008) [2023-12-27 07:03:58,674][20347] Updated weights for policy 0, policy_version 204260 (0.0008) [2023-12-27 07:03:58,679][20381] Updated weights for policy 1, policy_version 203890 (0.0007) [2023-12-27 07:03:58,746][20381] Updated weights for policy 1, policy_version 203900 (0.0008) [2023-12-27 07:03:58,812][20381] Updated weights for policy 1, policy_version 203910 (0.0008) [2023-12-27 07:03:58,881][20381] Updated weights for policy 1, policy_version 203920 (0.0008) [2023-12-27 07:03:59,406][20347] Updated weights for policy 0, policy_version 204270 (0.0006) [2023-12-27 07:03:59,460][20347] Updated weights for policy 0, policy_version 204280 (0.0005) [2023-12-27 07:03:59,523][20347] Updated weights for policy 0, policy_version 204290 (0.0006) [2023-12-27 07:03:59,568][20381] Updated weights for policy 1, policy_version 203930 (0.0007) [2023-12-27 07:03:59,634][20381] Updated weights for policy 1, policy_version 203940 (0.0006) [2023-12-27 07:03:59,696][20381] Updated weights for policy 1, policy_version 203950 (0.0006) [2023-12-27 07:04:00,212][20347] Updated weights for policy 0, policy_version 204300 (0.0007) [2023-12-27 07:04:00,261][20347] Updated weights for policy 0, policy_version 204310 (0.0010) [2023-12-27 07:04:00,290][20381] Updated weights for policy 1, policy_version 203960 (0.0006) [2023-12-27 07:04:00,306][20347] Updated weights for policy 0, policy_version 204320 (0.0010) [2023-12-27 07:04:00,349][20381] Updated weights for policy 1, policy_version 203970 (0.0005) [2023-12-27 07:04:00,398][20381] Updated weights for policy 1, policy_version 203980 (0.0006) [2023-12-27 07:04:00,946][20381] Updated weights for policy 1, policy_version 203990 (0.0006) [2023-12-27 07:04:00,995][20381] Updated weights for policy 1, policy_version 204000 (0.0007) [2023-12-27 07:04:01,005][20347] Updated weights for policy 0, policy_version 204330 (0.0010) [2023-12-27 07:04:01,053][20381] Updated weights for policy 1, policy_version 204010 (0.0007) [2023-12-27 07:04:01,065][20347] Updated weights for policy 0, policy_version 204340 (0.0009) [2023-12-27 07:04:01,119][20347] Updated weights for policy 0, policy_version 204350 (0.0010) [2023-12-27 07:04:01,178][20347] Updated weights for policy 0, policy_version 204360 (0.0011) [2023-12-27 07:04:01,773][20381] Updated weights for policy 1, policy_version 204020 (0.0006) [2023-12-27 07:04:01,824][20381] Updated weights for policy 1, policy_version 204030 (0.0006) [2023-12-27 07:04:01,829][20347] Updated weights for policy 0, policy_version 204370 (0.0008) [2023-12-27 07:04:01,880][20381] Updated weights for policy 1, policy_version 204040 (0.0007) [2023-12-27 07:04:01,887][20347] Updated weights for policy 0, policy_version 204380 (0.0008) [2023-12-27 07:04:01,944][20347] Updated weights for policy 0, policy_version 204390 (0.0008) [2023-12-27 07:04:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 104587264. Throughput: 0: 10269.7, 1: 10449.9. Samples: 104598552. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:02,363][19170] Avg episode reward: [(0, '999.380'), (1, '991.550')] [2023-12-27 07:04:02,538][20381] Updated weights for policy 1, policy_version 204050 (0.0006) [2023-12-27 07:04:02,590][20381] Updated weights for policy 1, policy_version 204060 (0.0007) [2023-12-27 07:04:02,639][20347] Updated weights for policy 0, policy_version 204400 (0.0009) [2023-12-27 07:04:02,639][20381] Updated weights for policy 1, policy_version 204070 (0.0007) [2023-12-27 07:04:02,687][20381] Updated weights for policy 1, policy_version 204080 (0.0006) [2023-12-27 07:04:02,689][20347] Updated weights for policy 0, policy_version 204410 (0.0006) [2023-12-27 07:04:02,737][20347] Updated weights for policy 0, policy_version 204420 (0.0008) [2023-12-27 07:04:03,320][20381] Updated weights for policy 1, policy_version 204090 (0.0005) [2023-12-27 07:04:03,374][20381] Updated weights for policy 1, policy_version 204100 (0.0005) [2023-12-27 07:04:03,381][20347] Updated weights for policy 0, policy_version 204430 (0.0006) [2023-12-27 07:04:03,386][20291] KL-divergence is very high: 137.1679 [2023-12-27 07:04:03,425][20381] Updated weights for policy 1, policy_version 204110 (0.0005) [2023-12-27 07:04:03,432][20347] Updated weights for policy 0, policy_version 204440 (0.0005) [2023-12-27 07:04:03,452][20291] KL-divergence is very high: 157.1736 [2023-12-27 07:04:03,479][20347] Updated weights for policy 0, policy_version 204450 (0.0007) [2023-12-27 07:04:04,002][20381] Updated weights for policy 1, policy_version 204120 (0.0005) [2023-12-27 07:04:04,043][20292] KL-divergence is very high: 103.3876 [2023-12-27 07:04:04,061][20381] Updated weights for policy 1, policy_version 204130 (0.0006) [2023-12-27 07:04:04,122][20381] Updated weights for policy 1, policy_version 204140 (0.0007) [2023-12-27 07:04:04,129][20347] Updated weights for policy 0, policy_version 204460 (0.0007) [2023-12-27 07:04:04,137][20291] KL-divergence is very high: 186.6303 [2023-12-27 07:04:04,181][20347] Updated weights for policy 0, policy_version 204470 (0.0008) [2023-12-27 07:04:04,236][20347] Updated weights for policy 0, policy_version 204480 (0.0007) [2023-12-27 07:04:04,777][20381] Updated weights for policy 1, policy_version 204150 (0.0007) [2023-12-27 07:04:04,832][20381] Updated weights for policy 1, policy_version 204160 (0.0008) [2023-12-27 07:04:04,877][20381] Updated weights for policy 1, policy_version 204170 (0.0008) [2023-12-27 07:04:04,947][20347] Updated weights for policy 0, policy_version 204490 (0.0008) [2023-12-27 07:04:04,999][20347] Updated weights for policy 0, policy_version 204500 (0.0007) [2023-12-27 07:04:05,049][20347] Updated weights for policy 0, policy_version 204510 (0.0008) [2023-12-27 07:04:05,105][20347] Updated weights for policy 0, policy_version 204520 (0.0007) [2023-12-27 07:04:05,580][20381] Updated weights for policy 1, policy_version 204180 (0.0008) [2023-12-27 07:04:05,626][20381] Updated weights for policy 1, policy_version 204190 (0.0009) [2023-12-27 07:04:05,673][20381] Updated weights for policy 1, policy_version 204200 (0.0009) [2023-12-27 07:04:05,779][20347] Updated weights for policy 0, policy_version 204530 (0.0010) [2023-12-27 07:04:05,830][20347] Updated weights for policy 0, policy_version 204540 (0.0010) [2023-12-27 07:04:05,894][20347] Updated weights for policy 0, policy_version 204550 (0.0010) [2023-12-27 07:04:06,368][20381] Updated weights for policy 1, policy_version 204210 (0.0008) [2023-12-27 07:04:06,417][20381] Updated weights for policy 1, policy_version 204220 (0.0010) [2023-12-27 07:04:06,473][20381] Updated weights for policy 1, policy_version 204230 (0.0009) [2023-12-27 07:04:06,533][20381] Updated weights for policy 1, policy_version 204240 (0.0007) [2023-12-27 07:04:06,559][20347] Updated weights for policy 0, policy_version 204560 (0.0011) [2023-12-27 07:04:06,618][20347] Updated weights for policy 0, policy_version 204570 (0.0010) [2023-12-27 07:04:06,679][20347] Updated weights for policy 0, policy_version 204580 (0.0010) [2023-12-27 07:04:07,217][20381] Updated weights for policy 1, policy_version 204250 (0.0007) [2023-12-27 07:04:07,273][20381] Updated weights for policy 1, policy_version 204260 (0.0005) [2023-12-27 07:04:07,326][20381] Updated weights for policy 1, policy_version 204270 (0.0007) [2023-12-27 07:04:07,341][20347] Updated weights for policy 0, policy_version 204590 (0.0011) [2023-12-27 07:04:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 104693760. Throughput: 0: 10333.3, 1: 10450.3. Samples: 104662380. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:07,363][19170] Avg episode reward: [(0, '976.950'), (1, '981.090')] [2023-12-27 07:04:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000204272_52305920.pth... [2023-12-27 07:04:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000203024_51986432.pth [2023-12-27 07:04:07,399][20347] Updated weights for policy 0, policy_version 204600 (0.0010) [2023-12-27 07:04:07,445][20347] Updated weights for policy 0, policy_version 204610 (0.0010) [2023-12-27 07:04:07,478][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000204616_52396032.pth... [2023-12-27 07:04:07,482][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000203400_52084736.pth [2023-12-27 07:04:07,910][20381] Updated weights for policy 1, policy_version 204280 (0.0008) [2023-12-27 07:04:07,973][20381] Updated weights for policy 1, policy_version 204290 (0.0008) [2023-12-27 07:04:08,035][20381] Updated weights for policy 1, policy_version 204300 (0.0007) [2023-12-27 07:04:08,143][20347] Updated weights for policy 0, policy_version 204620 (0.0011) [2023-12-27 07:04:08,201][20347] Updated weights for policy 0, policy_version 204630 (0.0010) [2023-12-27 07:04:08,249][20347] Updated weights for policy 0, policy_version 204640 (0.0010) [2023-12-27 07:04:08,730][20381] Updated weights for policy 1, policy_version 204310 (0.0007) [2023-12-27 07:04:08,780][20292] KL-divergence is very high: 936.5801 [2023-12-27 07:04:08,787][20381] Updated weights for policy 1, policy_version 204320 (0.0008) [2023-12-27 07:04:08,840][20381] Updated weights for policy 1, policy_version 204330 (0.0007) [2023-12-27 07:04:08,931][20347] Updated weights for policy 0, policy_version 204650 (0.0010) [2023-12-27 07:04:08,998][20347] Updated weights for policy 0, policy_version 204660 (0.0010) [2023-12-27 07:04:09,053][20347] Updated weights for policy 0, policy_version 204670 (0.0010) [2023-12-27 07:04:09,104][20347] Updated weights for policy 0, policy_version 204680 (0.0010) [2023-12-27 07:04:09,551][20381] Updated weights for policy 1, policy_version 204340 (0.0009) [2023-12-27 07:04:09,608][20381] Updated weights for policy 1, policy_version 204350 (0.0008) [2023-12-27 07:04:09,651][20292] KL-divergence is very high: 1352.9357 [2023-12-27 07:04:09,668][20381] Updated weights for policy 1, policy_version 204360 (0.0007) [2023-12-27 07:04:09,699][20292] KL-divergence is very high: 273.3406 [2023-12-27 07:04:09,852][20347] Updated weights for policy 0, policy_version 204690 (0.0010) [2023-12-27 07:04:09,912][20347] Updated weights for policy 0, policy_version 204700 (0.0010) [2023-12-27 07:04:09,979][20347] Updated weights for policy 0, policy_version 204710 (0.0011) [2023-12-27 07:04:10,399][20381] Updated weights for policy 1, policy_version 204370 (0.0007) [2023-12-27 07:04:10,454][20381] Updated weights for policy 1, policy_version 204380 (0.0008) [2023-12-27 07:04:10,509][20381] Updated weights for policy 1, policy_version 204390 (0.0007) [2023-12-27 07:04:10,570][20381] Updated weights for policy 1, policy_version 204400 (0.0007) [2023-12-27 07:04:10,630][20347] Updated weights for policy 0, policy_version 204720 (0.0007) [2023-12-27 07:04:10,684][20347] Updated weights for policy 0, policy_version 204730 (0.0008) [2023-12-27 07:04:10,737][20347] Updated weights for policy 0, policy_version 204740 (0.0010) [2023-12-27 07:04:11,277][20381] Updated weights for policy 1, policy_version 204410 (0.0008) [2023-12-27 07:04:11,336][20381] Updated weights for policy 1, policy_version 204420 (0.0006) [2023-12-27 07:04:11,401][20381] Updated weights for policy 1, policy_version 204430 (0.0008) [2023-12-27 07:04:11,420][20347] Updated weights for policy 0, policy_version 204750 (0.0007) [2023-12-27 07:04:11,483][20347] Updated weights for policy 0, policy_version 204760 (0.0008) [2023-12-27 07:04:11,548][20347] Updated weights for policy 0, policy_version 204770 (0.0011) [2023-12-27 07:04:12,128][20381] Updated weights for policy 1, policy_version 204440 (0.0007) [2023-12-27 07:04:12,167][20292] KL-divergence is very high: 149.8686 [2023-12-27 07:04:12,184][20381] Updated weights for policy 1, policy_version 204450 (0.0007) [2023-12-27 07:04:12,226][20347] Updated weights for policy 0, policy_version 204780 (0.0010) [2023-12-27 07:04:12,243][20381] Updated weights for policy 1, policy_version 204460 (0.0006) [2023-12-27 07:04:12,261][20292] KL-divergence is very high: 425.1019 [2023-12-27 07:04:12,284][20347] Updated weights for policy 0, policy_version 204790 (0.0010) [2023-12-27 07:04:12,345][20347] Updated weights for policy 0, policy_version 204800 (0.0011) [2023-12-27 07:04:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 104792064. Throughput: 0: 10314.4, 1: 10490.0. Samples: 104784760. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:12,363][19170] Avg episode reward: [(0, '974.890'), (1, '961.980')] [2023-12-27 07:04:12,974][20381] Updated weights for policy 1, policy_version 204470 (0.0007) [2023-12-27 07:04:13,022][20381] Updated weights for policy 1, policy_version 204480 (0.0007) [2023-12-27 07:04:13,063][20347] Updated weights for policy 0, policy_version 204810 (0.0011) [2023-12-27 07:04:13,069][20381] Updated weights for policy 1, policy_version 204490 (0.0006) [2023-12-27 07:04:13,118][20347] Updated weights for policy 0, policy_version 204820 (0.0010) [2023-12-27 07:04:13,166][20347] Updated weights for policy 0, policy_version 204830 (0.0010) [2023-12-27 07:04:13,223][20347] Updated weights for policy 0, policy_version 204840 (0.0009) [2023-12-27 07:04:13,785][20381] Updated weights for policy 1, policy_version 204500 (0.0006) [2023-12-27 07:04:13,835][20381] Updated weights for policy 1, policy_version 204510 (0.0007) [2023-12-27 07:04:13,884][20381] Updated weights for policy 1, policy_version 204520 (0.0007) [2023-12-27 07:04:13,891][20347] Updated weights for policy 0, policy_version 204850 (0.0010) [2023-12-27 07:04:13,944][20347] Updated weights for policy 0, policy_version 204860 (0.0010) [2023-12-27 07:04:14,001][20347] Updated weights for policy 0, policy_version 204870 (0.0010) [2023-12-27 07:04:14,597][20347] Updated weights for policy 0, policy_version 204880 (0.0008) [2023-12-27 07:04:14,622][20292] KL-divergence is very high: 162.6841 [2023-12-27 07:04:14,629][20381] Updated weights for policy 1, policy_version 204530 (0.0006) [2023-12-27 07:04:14,660][20292] KL-divergence is very high: 192.0963 [2023-12-27 07:04:14,662][20347] Updated weights for policy 0, policy_version 204890 (0.0006) [2023-12-27 07:04:14,673][20292] KL-divergence is very high: 120.6859 [2023-12-27 07:04:14,690][20381] Updated weights for policy 1, policy_version 204540 (0.0007) [2023-12-27 07:04:14,720][20347] Updated weights for policy 0, policy_version 204900 (0.0006) [2023-12-27 07:04:14,748][20381] Updated weights for policy 1, policy_version 204550 (0.0006) [2023-12-27 07:04:14,813][20381] Updated weights for policy 1, policy_version 204560 (0.0008) [2023-12-27 07:04:15,344][20347] Updated weights for policy 0, policy_version 204910 (0.0007) [2023-12-27 07:04:15,384][20291] KL-divergence is very high: 889.2397 [2023-12-27 07:04:15,400][20347] Updated weights for policy 0, policy_version 204920 (0.0007) [2023-12-27 07:04:15,449][20347] Updated weights for policy 0, policy_version 204930 (0.0007) [2023-12-27 07:04:15,466][20291] KL-divergence is very high: 104.0805 [2023-12-27 07:04:15,481][20292] KL-divergence is very high: 171.5695 [2023-12-27 07:04:15,539][20381] Updated weights for policy 1, policy_version 204570 (0.0008) [2023-12-27 07:04:15,570][20292] KL-divergence is very high: 198.2242 [2023-12-27 07:04:15,582][20292] KL-divergence is very high: 154.6470 [2023-12-27 07:04:15,599][20381] Updated weights for policy 1, policy_version 204580 (0.0008) [2023-12-27 07:04:15,628][20292] KL-divergence is very high: 120.7532 [2023-12-27 07:04:15,651][20381] Updated weights for policy 1, policy_version 204590 (0.0007) [2023-12-27 07:04:16,133][20347] Updated weights for policy 0, policy_version 204940 (0.0007) [2023-12-27 07:04:16,150][20291] KL-divergence is very high: 142.7808 [2023-12-27 07:04:16,191][20347] Updated weights for policy 0, policy_version 204950 (0.0008) [2023-12-27 07:04:16,236][20347] Updated weights for policy 0, policy_version 204960 (0.0007) [2023-12-27 07:04:16,240][20291] KL-divergence is very high: 121.2063 [2023-12-27 07:04:16,335][20381] Updated weights for policy 1, policy_version 204600 (0.0008) [2023-12-27 07:04:16,339][20292] KL-divergence is very high: 106.5753 [2023-12-27 07:04:16,384][20381] Updated weights for policy 1, policy_version 204610 (0.0007) [2023-12-27 07:04:16,439][20381] Updated weights for policy 1, policy_version 204620 (0.0008) [2023-12-27 07:04:16,912][20347] Updated weights for policy 0, policy_version 204970 (0.0007) [2023-12-27 07:04:16,957][20291] KL-divergence is very high: 172.1444 [2023-12-27 07:04:16,980][20347] Updated weights for policy 0, policy_version 204980 (0.0008) [2023-12-27 07:04:17,011][20291] KL-divergence is very high: 193.9481 [2023-12-27 07:04:17,030][20291] KL-divergence is very high: 328.8758 [2023-12-27 07:04:17,037][20291] KL-divergence is very high: 194.5209 [2023-12-27 07:04:17,041][20347] Updated weights for policy 0, policy_version 204990 (0.0009) [2023-12-27 07:04:17,073][20291] KL-divergence is very high: 200.1292 [2023-12-27 07:04:17,078][20291] KL-divergence is very high: 165.0225 [2023-12-27 07:04:17,084][20291] KL-divergence is very high: 100.4955 [2023-12-27 07:04:17,092][20347] Updated weights for policy 0, policy_version 205000 (0.0007) [2023-12-27 07:04:17,121][20381] Updated weights for policy 1, policy_version 204630 (0.0007) [2023-12-27 07:04:17,145][20292] KL-divergence is very high: 434.0433 [2023-12-27 07:04:17,151][20292] KL-divergence is very high: 328.7541 [2023-12-27 07:04:17,183][20381] Updated weights for policy 1, policy_version 204640 (0.0007) [2023-12-27 07:04:17,195][20292] KL-divergence is very high: 110.5409 [2023-12-27 07:04:17,233][20381] Updated weights for policy 1, policy_version 204650 (0.0008) [2023-12-27 07:04:17,256][20292] KL-divergence is very high: 164.9179 [2023-12-27 07:04:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 104898560. Throughput: 0: 10312.6, 1: 10383.6. Samples: 104905204. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:17,363][19170] Avg episode reward: [(0, '967.910'), (1, '344.120')] [2023-12-27 07:04:17,760][20291] KL-divergence is very high: 693.9863 [2023-12-27 07:04:17,767][20291] KL-divergence is very high: 229.7442 [2023-12-27 07:04:17,772][20291] KL-divergence is very high: 810.9249 [2023-12-27 07:04:17,777][20291] KL-divergence is very high: 179.8279 [2023-12-27 07:04:17,786][20347] Updated weights for policy 0, policy_version 205010 (0.0007) [2023-12-27 07:04:17,809][20291] KL-divergence is very high: 559.1829 [2023-12-27 07:04:17,831][20347] Updated weights for policy 0, policy_version 205020 (0.0007) [2023-12-27 07:04:17,836][20291] KL-divergence is very high: 142.1219 [2023-12-27 07:04:17,848][20291] KL-divergence is very high: 208.1406 [2023-12-27 07:04:17,882][20291] KL-divergence is very high: 256.2628 [2023-12-27 07:04:17,888][20347] Updated weights for policy 0, policy_version 205030 (0.0007) [2023-12-27 07:04:17,893][20291] KL-divergence is very high: 286.8375 [2023-12-27 07:04:17,919][20292] KL-divergence is very high: 184.5309 [2023-12-27 07:04:17,931][20292] KL-divergence is very high: 139.6053 [2023-12-27 07:04:17,944][20381] Updated weights for policy 1, policy_version 204660 (0.0009) [2023-12-27 07:04:17,947][20292] KL-divergence is very high: 101.2173 [2023-12-27 07:04:17,952][20292] KL-divergence is very high: 134.6868 [2023-12-27 07:04:17,958][20292] KL-divergence is very high: 132.8095 [2023-12-27 07:04:17,977][20292] KL-divergence is very high: 132.1067 [2023-12-27 07:04:17,996][20292] KL-divergence is very high: 114.1906 [2023-12-27 07:04:18,003][20292] KL-divergence is very high: 110.1838 [2023-12-27 07:04:18,004][20381] Updated weights for policy 1, policy_version 204670 (0.0009) [2023-12-27 07:04:18,067][20381] Updated weights for policy 1, policy_version 204680 (0.0011) [2023-12-27 07:04:18,602][20347] Updated weights for policy 0, policy_version 205040 (0.0006) [2023-12-27 07:04:18,675][20347] Updated weights for policy 0, policy_version 205050 (0.0006) [2023-12-27 07:04:18,738][20347] Updated weights for policy 0, policy_version 205060 (0.0006) [2023-12-27 07:04:18,741][20381] Updated weights for policy 1, policy_version 204690 (0.0011) [2023-12-27 07:04:18,804][20381] Updated weights for policy 1, policy_version 204700 (0.0011) [2023-12-27 07:04:18,867][20381] Updated weights for policy 1, policy_version 204710 (0.0011) [2023-12-27 07:04:18,925][20381] Updated weights for policy 1, policy_version 204720 (0.0011) [2023-12-27 07:04:19,382][20347] Updated weights for policy 0, policy_version 205070 (0.0009) [2023-12-27 07:04:19,430][20347] Updated weights for policy 0, policy_version 205080 (0.0010) [2023-12-27 07:04:19,492][20347] Updated weights for policy 0, policy_version 205090 (0.0010) [2023-12-27 07:04:19,600][20381] Updated weights for policy 1, policy_version 204730 (0.0011) [2023-12-27 07:04:19,650][20381] Updated weights for policy 1, policy_version 204740 (0.0011) [2023-12-27 07:04:19,703][20381] Updated weights for policy 1, policy_version 204750 (0.0011) [2023-12-27 07:04:20,237][20347] Updated weights for policy 0, policy_version 205100 (0.0010) [2023-12-27 07:04:20,286][20347] Updated weights for policy 0, policy_version 205110 (0.0010) [2023-12-27 07:04:20,343][20347] Updated weights for policy 0, policy_version 205120 (0.0011) [2023-12-27 07:04:20,442][20381] Updated weights for policy 1, policy_version 204760 (0.0008) [2023-12-27 07:04:20,495][20381] Updated weights for policy 1, policy_version 204770 (0.0008) [2023-12-27 07:04:20,554][20381] Updated weights for policy 1, policy_version 204780 (0.0008) [2023-12-27 07:04:21,076][20347] Updated weights for policy 0, policy_version 205130 (0.0011) [2023-12-27 07:04:21,137][20347] Updated weights for policy 0, policy_version 205140 (0.0011) [2023-12-27 07:04:21,197][20347] Updated weights for policy 0, policy_version 205150 (0.0010) [2023-12-27 07:04:21,257][20347] Updated weights for policy 0, policy_version 205160 (0.0011) [2023-12-27 07:04:21,274][20381] Updated weights for policy 1, policy_version 204790 (0.0008) [2023-12-27 07:04:21,329][20381] Updated weights for policy 1, policy_version 204800 (0.0008) [2023-12-27 07:04:21,392][20381] Updated weights for policy 1, policy_version 204810 (0.0007) [2023-12-27 07:04:21,966][20347] Updated weights for policy 0, policy_version 205170 (0.0010) [2023-12-27 07:04:22,023][20347] Updated weights for policy 0, policy_version 205180 (0.0011) [2023-12-27 07:04:22,084][20347] Updated weights for policy 0, policy_version 205190 (0.0011) [2023-12-27 07:04:22,119][20381] Updated weights for policy 1, policy_version 204820 (0.0007) [2023-12-27 07:04:22,182][20381] Updated weights for policy 1, policy_version 204830 (0.0006) [2023-12-27 07:04:22,237][20381] Updated weights for policy 1, policy_version 204840 (0.0008) [2023-12-27 07:04:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 104996864. Throughput: 0: 10254.5, 1: 10329.3. Samples: 104964476. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:22,363][19170] Avg episode reward: [(0, '955.100'), (1, '153.550')] [2023-12-27 07:04:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000204848_52453376.pth... [2023-12-27 07:04:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000205192_52543488.pth... [2023-12-27 07:04:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000203664_52150272.pth [2023-12-27 07:04:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000204008_52240384.pth [2023-12-27 07:04:22,788][20347] Updated weights for policy 0, policy_version 205200 (0.0007) [2023-12-27 07:04:22,845][20347] Updated weights for policy 0, policy_version 205210 (0.0005) [2023-12-27 07:04:22,905][20347] Updated weights for policy 0, policy_version 205220 (0.0005) [2023-12-27 07:04:22,995][20381] Updated weights for policy 1, policy_version 204850 (0.0009) [2023-12-27 07:04:23,051][20381] Updated weights for policy 1, policy_version 204860 (0.0008) [2023-12-27 07:04:23,098][20381] Updated weights for policy 1, policy_version 204870 (0.0008) [2023-12-27 07:04:23,145][20381] Updated weights for policy 1, policy_version 204880 (0.0008) [2023-12-27 07:04:23,485][20347] Updated weights for policy 0, policy_version 205230 (0.0006) [2023-12-27 07:04:23,543][20347] Updated weights for policy 0, policy_version 205240 (0.0010) [2023-12-27 07:04:23,602][20347] Updated weights for policy 0, policy_version 205250 (0.0007) [2023-12-27 07:04:23,896][20381] Updated weights for policy 1, policy_version 204890 (0.0011) [2023-12-27 07:04:23,955][20381] Updated weights for policy 1, policy_version 204900 (0.0011) [2023-12-27 07:04:24,022][20381] Updated weights for policy 1, policy_version 204910 (0.0011) [2023-12-27 07:04:24,192][20347] Updated weights for policy 0, policy_version 205260 (0.0006) [2023-12-27 07:04:24,255][20347] Updated weights for policy 0, policy_version 205270 (0.0008) [2023-12-27 07:04:24,314][20347] Updated weights for policy 0, policy_version 205280 (0.0008) [2023-12-27 07:04:24,630][20381] Updated weights for policy 1, policy_version 204920 (0.0011) [2023-12-27 07:04:24,683][20381] Updated weights for policy 1, policy_version 204930 (0.0009) [2023-12-27 07:04:24,732][20381] Updated weights for policy 1, policy_version 204940 (0.0008) [2023-12-27 07:04:24,975][20347] Updated weights for policy 0, policy_version 205290 (0.0008) [2023-12-27 07:04:25,032][20347] Updated weights for policy 0, policy_version 205300 (0.0008) [2023-12-27 07:04:25,093][20347] Updated weights for policy 0, policy_version 205310 (0.0008) [2023-12-27 07:04:25,153][20347] Updated weights for policy 0, policy_version 205320 (0.0007) [2023-12-27 07:04:25,439][20381] Updated weights for policy 1, policy_version 204950 (0.0008) [2023-12-27 07:04:25,491][20381] Updated weights for policy 1, policy_version 204960 (0.0008) [2023-12-27 07:04:25,539][20381] Updated weights for policy 1, policy_version 204970 (0.0007) [2023-12-27 07:04:25,814][20347] Updated weights for policy 0, policy_version 205330 (0.0005) [2023-12-27 07:04:25,879][20347] Updated weights for policy 0, policy_version 205340 (0.0007) [2023-12-27 07:04:25,934][20347] Updated weights for policy 0, policy_version 205350 (0.0007) [2023-12-27 07:04:26,218][20381] Updated weights for policy 1, policy_version 204980 (0.0007) [2023-12-27 07:04:26,267][20381] Updated weights for policy 1, policy_version 204990 (0.0009) [2023-12-27 07:04:26,319][20381] Updated weights for policy 1, policy_version 205000 (0.0007) [2023-12-27 07:04:26,498][20347] Updated weights for policy 0, policy_version 205360 (0.0008) [2023-12-27 07:04:26,545][20347] Updated weights for policy 0, policy_version 205370 (0.0007) [2023-12-27 07:04:26,596][20347] Updated weights for policy 0, policy_version 205380 (0.0008) [2023-12-27 07:04:27,021][20381] Updated weights for policy 1, policy_version 205010 (0.0007) [2023-12-27 07:04:27,073][20381] Updated weights for policy 1, policy_version 205020 (0.0008) [2023-12-27 07:04:27,121][20381] Updated weights for policy 1, policy_version 205030 (0.0007) [2023-12-27 07:04:27,174][20381] Updated weights for policy 1, policy_version 205040 (0.0005) [2023-12-27 07:04:27,198][20347] Updated weights for policy 0, policy_version 205390 (0.0007) [2023-12-27 07:04:27,260][20347] Updated weights for policy 0, policy_version 205400 (0.0005) [2023-12-27 07:04:27,320][20347] Updated weights for policy 0, policy_version 205410 (0.0008) [2023-12-27 07:04:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 105103360. Throughput: 0: 10249.4, 1: 10261.3. Samples: 105085520. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:27,363][19170] Avg episode reward: [(0, '979.020'), (1, '363.500')] [2023-12-27 07:04:27,864][20381] Updated weights for policy 1, policy_version 205050 (0.0007) [2023-12-27 07:04:27,869][20347] Updated weights for policy 0, policy_version 205420 (0.0005) [2023-12-27 07:04:27,925][20381] Updated weights for policy 1, policy_version 205060 (0.0005) [2023-12-27 07:04:27,926][20347] Updated weights for policy 0, policy_version 205430 (0.0007) [2023-12-27 07:04:27,980][20347] Updated weights for policy 0, policy_version 205440 (0.0005) [2023-12-27 07:04:27,982][20381] Updated weights for policy 1, policy_version 205070 (0.0006) [2023-12-27 07:04:28,497][20347] Updated weights for policy 0, policy_version 205450 (0.0006) [2023-12-27 07:04:28,551][20347] Updated weights for policy 0, policy_version 205460 (0.0008) [2023-12-27 07:04:28,604][20347] Updated weights for policy 0, policy_version 205470 (0.0008) [2023-12-27 07:04:28,618][20381] Updated weights for policy 1, policy_version 205080 (0.0006) [2023-12-27 07:04:28,662][20347] Updated weights for policy 0, policy_version 205480 (0.0007) [2023-12-27 07:04:28,674][20381] Updated weights for policy 1, policy_version 205090 (0.0006) [2023-12-27 07:04:28,720][20381] Updated weights for policy 1, policy_version 205100 (0.0008) [2023-12-27 07:04:29,364][20347] Updated weights for policy 0, policy_version 205490 (0.0008) [2023-12-27 07:04:29,422][20347] Updated weights for policy 0, policy_version 205500 (0.0005) [2023-12-27 07:04:29,423][20381] Updated weights for policy 1, policy_version 205110 (0.0008) [2023-12-27 07:04:29,481][20347] Updated weights for policy 0, policy_version 205510 (0.0005) [2023-12-27 07:04:29,483][20381] Updated weights for policy 1, policy_version 205120 (0.0007) [2023-12-27 07:04:29,542][20381] Updated weights for policy 1, policy_version 205130 (0.0005) [2023-12-27 07:04:30,140][20347] Updated weights for policy 0, policy_version 205520 (0.0007) [2023-12-27 07:04:30,142][20381] Updated weights for policy 1, policy_version 205140 (0.0007) [2023-12-27 07:04:30,190][20381] Updated weights for policy 1, policy_version 205150 (0.0010) [2023-12-27 07:04:30,199][20347] Updated weights for policy 0, policy_version 205530 (0.0005) [2023-12-27 07:04:30,242][20381] Updated weights for policy 1, policy_version 205160 (0.0010) [2023-12-27 07:04:30,254][20347] Updated weights for policy 0, policy_version 205540 (0.0005) [2023-12-27 07:04:30,835][20347] Updated weights for policy 0, policy_version 205550 (0.0007) [2023-12-27 07:04:30,878][20347] Updated weights for policy 0, policy_version 205560 (0.0007) [2023-12-27 07:04:30,928][20347] Updated weights for policy 0, policy_version 205570 (0.0007) [2023-12-27 07:04:30,934][20381] Updated weights for policy 1, policy_version 205170 (0.0010) [2023-12-27 07:04:30,982][20381] Updated weights for policy 1, policy_version 205180 (0.0010) [2023-12-27 07:04:31,031][20381] Updated weights for policy 1, policy_version 205190 (0.0010) [2023-12-27 07:04:31,087][20381] Updated weights for policy 1, policy_version 205200 (0.0011) [2023-12-27 07:04:31,578][20347] Updated weights for policy 0, policy_version 205580 (0.0006) [2023-12-27 07:04:31,648][20347] Updated weights for policy 0, policy_version 205590 (0.0007) [2023-12-27 07:04:31,707][20347] Updated weights for policy 0, policy_version 205600 (0.0008) [2023-12-27 07:04:31,750][20381] Updated weights for policy 1, policy_version 205210 (0.0008) [2023-12-27 07:04:31,807][20381] Updated weights for policy 1, policy_version 205220 (0.0005) [2023-12-27 07:04:31,862][20381] Updated weights for policy 1, policy_version 205230 (0.0005) [2023-12-27 07:04:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 105201664. Throughput: 0: 10347.1, 1: 10275.8. Samples: 105216432. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '661.910')] [2023-12-27 07:04:32,436][20347] Updated weights for policy 0, policy_version 205610 (0.0007) [2023-12-27 07:04:32,446][20381] Updated weights for policy 1, policy_version 205240 (0.0006) [2023-12-27 07:04:32,494][20347] Updated weights for policy 0, policy_version 205620 (0.0007) [2023-12-27 07:04:32,500][20381] Updated weights for policy 1, policy_version 205250 (0.0006) [2023-12-27 07:04:32,551][20347] Updated weights for policy 0, policy_version 205630 (0.0007) [2023-12-27 07:04:32,553][20381] Updated weights for policy 1, policy_version 205260 (0.0006) [2023-12-27 07:04:32,615][20347] Updated weights for policy 0, policy_version 205640 (0.0006) [2023-12-27 07:04:33,192][20381] Updated weights for policy 1, policy_version 205270 (0.0008) [2023-12-27 07:04:33,231][20347] Updated weights for policy 0, policy_version 205650 (0.0005) [2023-12-27 07:04:33,240][20381] Updated weights for policy 1, policy_version 205280 (0.0007) [2023-12-27 07:04:33,279][20347] Updated weights for policy 0, policy_version 205660 (0.0005) [2023-12-27 07:04:33,288][20381] Updated weights for policy 1, policy_version 205290 (0.0007) [2023-12-27 07:04:33,335][20347] Updated weights for policy 0, policy_version 205670 (0.0005) [2023-12-27 07:04:33,871][20381] Updated weights for policy 1, policy_version 205300 (0.0008) [2023-12-27 07:04:33,907][20347] Updated weights for policy 0, policy_version 205680 (0.0008) [2023-12-27 07:04:33,929][20381] Updated weights for policy 1, policy_version 205310 (0.0009) [2023-12-27 07:04:33,965][20347] Updated weights for policy 0, policy_version 205690 (0.0007) [2023-12-27 07:04:33,977][20381] Updated weights for policy 1, policy_version 205320 (0.0008) [2023-12-27 07:04:34,013][20347] Updated weights for policy 0, policy_version 205700 (0.0005) [2023-12-27 07:04:34,551][20381] Updated weights for policy 1, policy_version 205330 (0.0009) [2023-12-27 07:04:34,604][20381] Updated weights for policy 1, policy_version 205340 (0.0011) [2023-12-27 07:04:34,659][20381] Updated weights for policy 1, policy_version 205350 (0.0011) [2023-12-27 07:04:34,716][20347] Updated weights for policy 0, policy_version 205710 (0.0006) [2023-12-27 07:04:34,721][20381] Updated weights for policy 1, policy_version 205360 (0.0011) [2023-12-27 07:04:34,762][20347] Updated weights for policy 0, policy_version 205720 (0.0007) [2023-12-27 07:04:34,808][20347] Updated weights for policy 0, policy_version 205730 (0.0007) [2023-12-27 07:04:35,416][20381] Updated weights for policy 1, policy_version 205370 (0.0010) [2023-12-27 07:04:35,461][20381] Updated weights for policy 1, policy_version 205380 (0.0009) [2023-12-27 07:04:35,474][20347] Updated weights for policy 0, policy_version 205740 (0.0008) [2023-12-27 07:04:35,507][20381] Updated weights for policy 1, policy_version 205390 (0.0006) [2023-12-27 07:04:35,524][20347] Updated weights for policy 0, policy_version 205750 (0.0006) [2023-12-27 07:04:35,579][20347] Updated weights for policy 0, policy_version 205760 (0.0006) [2023-12-27 07:04:36,129][20381] Updated weights for policy 1, policy_version 205400 (0.0010) [2023-12-27 07:04:36,188][20381] Updated weights for policy 1, policy_version 205410 (0.0008) [2023-12-27 07:04:36,243][20381] Updated weights for policy 1, policy_version 205420 (0.0009) [2023-12-27 07:04:36,279][20347] Updated weights for policy 0, policy_version 205770 (0.0007) [2023-12-27 07:04:36,335][20347] Updated weights for policy 0, policy_version 205780 (0.0007) [2023-12-27 07:04:36,387][20347] Updated weights for policy 0, policy_version 205790 (0.0008) [2023-12-27 07:04:36,443][20347] Updated weights for policy 0, policy_version 205800 (0.0008) [2023-12-27 07:04:36,975][20381] Updated weights for policy 1, policy_version 205430 (0.0007) [2023-12-27 07:04:37,029][20381] Updated weights for policy 1, policy_version 205440 (0.0008) [2023-12-27 07:04:37,082][20381] Updated weights for policy 1, policy_version 205450 (0.0008) [2023-12-27 07:04:37,141][20347] Updated weights for policy 0, policy_version 205810 (0.0008) [2023-12-27 07:04:37,191][20347] Updated weights for policy 0, policy_version 205820 (0.0008) [2023-12-27 07:04:37,246][20347] Updated weights for policy 0, policy_version 205830 (0.0008) [2023-12-27 07:04:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 105316352. Throughput: 0: 10374.8, 1: 10303.3. Samples: 105281720. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '728.850')] [2023-12-27 07:04:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000205456_52609024.pth... [2023-12-27 07:04:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000205832_52707328.pth... [2023-12-27 07:04:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000204272_52305920.pth [2023-12-27 07:04:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000204616_52396032.pth [2023-12-27 07:04:37,778][20381] Updated weights for policy 1, policy_version 205460 (0.0009) [2023-12-27 07:04:37,822][20381] Updated weights for policy 1, policy_version 205470 (0.0007) [2023-12-27 07:04:37,870][20381] Updated weights for policy 1, policy_version 205480 (0.0005) [2023-12-27 07:04:37,974][20347] Updated weights for policy 0, policy_version 205840 (0.0008) [2023-12-27 07:04:38,037][20347] Updated weights for policy 0, policy_version 205850 (0.0008) [2023-12-27 07:04:38,098][20347] Updated weights for policy 0, policy_version 205860 (0.0008) [2023-12-27 07:04:38,558][20381] Updated weights for policy 1, policy_version 205490 (0.0007) [2023-12-27 07:04:38,615][20381] Updated weights for policy 1, policy_version 205500 (0.0008) [2023-12-27 07:04:38,674][20381] Updated weights for policy 1, policy_version 205510 (0.0008) [2023-12-27 07:04:38,731][20381] Updated weights for policy 1, policy_version 205520 (0.0008) [2023-12-27 07:04:38,794][20347] Updated weights for policy 0, policy_version 205870 (0.0008) [2023-12-27 07:04:38,850][20347] Updated weights for policy 0, policy_version 205880 (0.0007) [2023-12-27 07:04:38,909][20347] Updated weights for policy 0, policy_version 205890 (0.0007) [2023-12-27 07:04:39,386][20381] Updated weights for policy 1, policy_version 205530 (0.0008) [2023-12-27 07:04:39,448][20381] Updated weights for policy 1, policy_version 205540 (0.0008) [2023-12-27 07:04:39,505][20381] Updated weights for policy 1, policy_version 205550 (0.0008) [2023-12-27 07:04:39,605][20347] Updated weights for policy 0, policy_version 205900 (0.0007) [2023-12-27 07:04:39,663][20347] Updated weights for policy 0, policy_version 205910 (0.0007) [2023-12-27 07:04:39,722][20347] Updated weights for policy 0, policy_version 205920 (0.0006) [2023-12-27 07:04:40,235][20381] Updated weights for policy 1, policy_version 205560 (0.0009) [2023-12-27 07:04:40,295][20381] Updated weights for policy 1, policy_version 205570 (0.0008) [2023-12-27 07:04:40,362][20381] Updated weights for policy 1, policy_version 205580 (0.0008) [2023-12-27 07:04:40,383][20347] Updated weights for policy 0, policy_version 205930 (0.0007) [2023-12-27 07:04:40,436][20347] Updated weights for policy 0, policy_version 205940 (0.0008) [2023-12-27 07:04:40,486][20347] Updated weights for policy 0, policy_version 205950 (0.0010) [2023-12-27 07:04:40,534][20347] Updated weights for policy 0, policy_version 205960 (0.0007) [2023-12-27 07:04:40,996][20381] Updated weights for policy 1, policy_version 205590 (0.0007) [2023-12-27 07:04:41,065][20381] Updated weights for policy 1, policy_version 205600 (0.0008) [2023-12-27 07:04:41,129][20381] Updated weights for policy 1, policy_version 205610 (0.0008) [2023-12-27 07:04:41,242][20347] Updated weights for policy 0, policy_version 205970 (0.0007) [2023-12-27 07:04:41,307][20347] Updated weights for policy 0, policy_version 205980 (0.0005) [2023-12-27 07:04:41,374][20347] Updated weights for policy 0, policy_version 205990 (0.0008) [2023-12-27 07:04:41,859][20381] Updated weights for policy 1, policy_version 205620 (0.0007) [2023-12-27 07:04:41,918][20381] Updated weights for policy 1, policy_version 205630 (0.0006) [2023-12-27 07:04:41,974][20381] Updated weights for policy 1, policy_version 205640 (0.0005) [2023-12-27 07:04:42,015][20347] Updated weights for policy 0, policy_version 206000 (0.0008) [2023-12-27 07:04:42,074][20347] Updated weights for policy 0, policy_version 206010 (0.0007) [2023-12-27 07:04:42,131][20347] Updated weights for policy 0, policy_version 206020 (0.0008) [2023-12-27 07:04:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 105414656. Throughput: 0: 10351.8, 1: 10282.1. Samples: 105403352. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) [2023-12-27 07:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '479.110')] [2023-12-27 07:04:42,675][20381] Updated weights for policy 1, policy_version 205650 (0.0007) [2023-12-27 07:04:42,731][20381] Updated weights for policy 1, policy_version 205660 (0.0009) [2023-12-27 07:04:42,755][20347] Updated weights for policy 0, policy_version 206030 (0.0006) [2023-12-27 07:04:42,788][20381] Updated weights for policy 1, policy_version 205670 (0.0008) [2023-12-27 07:04:42,804][20347] Updated weights for policy 0, policy_version 206040 (0.0005) [2023-12-27 07:04:42,848][20381] Updated weights for policy 1, policy_version 205680 (0.0008) [2023-12-27 07:04:42,858][20347] Updated weights for policy 0, policy_version 206050 (0.0006) [2023-12-27 07:04:43,515][20347] Updated weights for policy 0, policy_version 206060 (0.0008) [2023-12-27 07:04:43,560][20381] Updated weights for policy 1, policy_version 205690 (0.0005) [2023-12-27 07:04:43,573][20347] Updated weights for policy 0, policy_version 206070 (0.0009) [2023-12-27 07:04:43,606][20381] Updated weights for policy 1, policy_version 205700 (0.0006) [2023-12-27 07:04:43,626][20347] Updated weights for policy 0, policy_version 206080 (0.0005) [2023-12-27 07:04:43,659][20381] Updated weights for policy 1, policy_version 205710 (0.0010) [2023-12-27 07:04:44,258][20347] Updated weights for policy 0, policy_version 206090 (0.0007) [2023-12-27 07:04:44,309][20347] Updated weights for policy 0, policy_version 206100 (0.0005) [2023-12-27 07:04:44,360][20347] Updated weights for policy 0, policy_version 206110 (0.0006) [2023-12-27 07:04:44,376][20381] Updated weights for policy 1, policy_version 205720 (0.0011) [2023-12-27 07:04:44,411][20347] Updated weights for policy 0, policy_version 206120 (0.0010) [2023-12-27 07:04:44,432][20381] Updated weights for policy 1, policy_version 205730 (0.0011) [2023-12-27 07:04:44,491][20381] Updated weights for policy 1, policy_version 205740 (0.0010) [2023-12-27 07:04:45,099][20347] Updated weights for policy 0, policy_version 206130 (0.0007) [2023-12-27 07:04:45,146][20381] Updated weights for policy 1, policy_version 205750 (0.0011) [2023-12-27 07:04:45,166][20347] Updated weights for policy 0, policy_version 206140 (0.0007) [2023-12-27 07:04:45,194][20381] Updated weights for policy 1, policy_version 205760 (0.0011) [2023-12-27 07:04:45,230][20347] Updated weights for policy 0, policy_version 206150 (0.0006) [2023-12-27 07:04:45,255][20381] Updated weights for policy 1, policy_version 205770 (0.0011) [2023-12-27 07:04:45,809][20347] Updated weights for policy 0, policy_version 206160 (0.0005) [2023-12-27 07:04:45,897][20347] Updated weights for policy 0, policy_version 206170 (0.0005) [2023-12-27 07:04:45,948][20381] Updated weights for policy 1, policy_version 205780 (0.0008) [2023-12-27 07:04:45,954][20347] Updated weights for policy 0, policy_version 206180 (0.0005) [2023-12-27 07:04:45,999][20381] Updated weights for policy 1, policy_version 205790 (0.0008) [2023-12-27 07:04:46,051][20381] Updated weights for policy 1, policy_version 205800 (0.0008) [2023-12-27 07:04:46,519][20347] Updated weights for policy 0, policy_version 206190 (0.0008) [2023-12-27 07:04:46,574][20347] Updated weights for policy 0, policy_version 206200 (0.0010) [2023-12-27 07:04:46,632][20347] Updated weights for policy 0, policy_version 206210 (0.0010) [2023-12-27 07:04:46,733][20381] Updated weights for policy 1, policy_version 205810 (0.0008) [2023-12-27 07:04:46,787][20381] Updated weights for policy 1, policy_version 205820 (0.0008) [2023-12-27 07:04:46,842][20381] Updated weights for policy 1, policy_version 205830 (0.0006) [2023-12-27 07:04:46,895][20381] Updated weights for policy 1, policy_version 205840 (0.0005) [2023-12-27 07:04:47,219][20347] Updated weights for policy 0, policy_version 206220 (0.0008) [2023-12-27 07:04:47,273][20347] Updated weights for policy 0, policy_version 206230 (0.0005) [2023-12-27 07:04:47,324][20347] Updated weights for policy 0, policy_version 206240 (0.0005) [2023-12-27 07:04:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 105512960. Throughput: 0: 10460.6, 1: 10218.2. Samples: 105529100. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:04:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '698.810')] [2023-12-27 07:04:47,492][20381] Updated weights for policy 1, policy_version 205850 (0.0008) [2023-12-27 07:04:47,541][20381] Updated weights for policy 1, policy_version 205860 (0.0008) [2023-12-27 07:04:47,591][20381] Updated weights for policy 1, policy_version 205870 (0.0008) [2023-12-27 07:04:47,883][20347] Updated weights for policy 0, policy_version 206250 (0.0006) [2023-12-27 07:04:47,939][20347] Updated weights for policy 0, policy_version 206260 (0.0008) [2023-12-27 07:04:47,996][20347] Updated weights for policy 0, policy_version 206270 (0.0007) [2023-12-27 07:04:48,057][20347] Updated weights for policy 0, policy_version 206280 (0.0005) [2023-12-27 07:04:48,264][20381] Updated weights for policy 1, policy_version 205880 (0.0007) [2023-12-27 07:04:48,310][20381] Updated weights for policy 1, policy_version 205890 (0.0008) [2023-12-27 07:04:48,370][20381] Updated weights for policy 1, policy_version 205900 (0.0008) [2023-12-27 07:04:48,739][20347] Updated weights for policy 0, policy_version 206290 (0.0008) [2023-12-27 07:04:48,803][20347] Updated weights for policy 0, policy_version 206300 (0.0008) [2023-12-27 07:04:48,867][20347] Updated weights for policy 0, policy_version 206310 (0.0008) [2023-12-27 07:04:49,001][20381] Updated weights for policy 1, policy_version 205910 (0.0008) [2023-12-27 07:04:49,057][20381] Updated weights for policy 1, policy_version 205920 (0.0009) [2023-12-27 07:04:49,113][20381] Updated weights for policy 1, policy_version 205930 (0.0008) [2023-12-27 07:04:49,534][20347] Updated weights for policy 0, policy_version 206320 (0.0007) [2023-12-27 07:04:49,596][20347] Updated weights for policy 0, policy_version 206330 (0.0007) [2023-12-27 07:04:49,652][20347] Updated weights for policy 0, policy_version 206340 (0.0008) [2023-12-27 07:04:49,811][20381] Updated weights for policy 1, policy_version 205940 (0.0008) [2023-12-27 07:04:49,875][20381] Updated weights for policy 1, policy_version 205950 (0.0008) [2023-12-27 07:04:49,938][20381] Updated weights for policy 1, policy_version 205960 (0.0008) [2023-12-27 07:04:50,305][20347] Updated weights for policy 0, policy_version 206350 (0.0006) [2023-12-27 07:04:50,364][20347] Updated weights for policy 0, policy_version 206360 (0.0007) [2023-12-27 07:04:50,425][20347] Updated weights for policy 0, policy_version 206370 (0.0008) [2023-12-27 07:04:50,612][20381] Updated weights for policy 1, policy_version 205970 (0.0008) [2023-12-27 07:04:50,665][20381] Updated weights for policy 1, policy_version 205980 (0.0008) [2023-12-27 07:04:50,719][20381] Updated weights for policy 1, policy_version 205990 (0.0008) [2023-12-27 07:04:50,775][20381] Updated weights for policy 1, policy_version 206000 (0.0008) [2023-12-27 07:04:51,094][20347] Updated weights for policy 0, policy_version 206380 (0.0008) [2023-12-27 07:04:51,152][20347] Updated weights for policy 0, policy_version 206390 (0.0007) [2023-12-27 07:04:51,208][20347] Updated weights for policy 0, policy_version 206400 (0.0008) [2023-12-27 07:04:51,528][20381] Updated weights for policy 1, policy_version 206010 (0.0008) [2023-12-27 07:04:51,581][20381] Updated weights for policy 1, policy_version 206020 (0.0008) [2023-12-27 07:04:51,638][20381] Updated weights for policy 1, policy_version 206030 (0.0008) [2023-12-27 07:04:51,890][20347] Updated weights for policy 0, policy_version 206410 (0.0008) [2023-12-27 07:04:51,946][20347] Updated weights for policy 0, policy_version 206420 (0.0008) [2023-12-27 07:04:51,999][20347] Updated weights for policy 0, policy_version 206430 (0.0009) [2023-12-27 07:04:52,055][20347] Updated weights for policy 0, policy_version 206440 (0.0008) [2023-12-27 07:04:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 105619456. Throughput: 0: 10474.5, 1: 10170.8. Samples: 105591416. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:04:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '763.450')] [2023-12-27 07:04:52,366][20381] Updated weights for policy 1, policy_version 206040 (0.0008) [2023-12-27 07:04:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000206440_52862976.pth... [2023-12-27 07:04:52,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000205192_52543488.pth [2023-12-27 07:04:52,422][20381] Updated weights for policy 1, policy_version 206050 (0.0007) [2023-12-27 07:04:52,483][20381] Updated weights for policy 1, policy_version 206060 (0.0007) [2023-12-27 07:04:52,506][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000206064_52764672.pth... [2023-12-27 07:04:52,509][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000204848_52453376.pth [2023-12-27 07:04:52,688][20347] Updated weights for policy 0, policy_version 206450 (0.0006) [2023-12-27 07:04:52,753][20347] Updated weights for policy 0, policy_version 206460 (0.0006) [2023-12-27 07:04:52,817][20347] Updated weights for policy 0, policy_version 206470 (0.0005) [2023-12-27 07:04:53,166][20381] Updated weights for policy 1, policy_version 206070 (0.0008) [2023-12-27 07:04:53,224][20381] Updated weights for policy 1, policy_version 206080 (0.0006) [2023-12-27 07:04:53,286][20381] Updated weights for policy 1, policy_version 206090 (0.0006) [2023-12-27 07:04:53,438][20347] Updated weights for policy 0, policy_version 206480 (0.0007) [2023-12-27 07:04:53,495][20347] Updated weights for policy 0, policy_version 206490 (0.0005) [2023-12-27 07:04:53,552][20347] Updated weights for policy 0, policy_version 206500 (0.0006) [2023-12-27 07:04:53,882][20381] Updated weights for policy 1, policy_version 206100 (0.0008) [2023-12-27 07:04:53,944][20381] Updated weights for policy 1, policy_version 206110 (0.0005) [2023-12-27 07:04:53,996][20381] Updated weights for policy 1, policy_version 206120 (0.0005) [2023-12-27 07:04:54,267][20347] Updated weights for policy 0, policy_version 206510 (0.0008) [2023-12-27 07:04:54,320][20347] Updated weights for policy 0, policy_version 206520 (0.0005) [2023-12-27 07:04:54,374][20347] Updated weights for policy 0, policy_version 206530 (0.0007) [2023-12-27 07:04:54,595][20381] Updated weights for policy 1, policy_version 206130 (0.0006) [2023-12-27 07:04:54,653][20381] Updated weights for policy 1, policy_version 206140 (0.0005) [2023-12-27 07:04:54,720][20381] Updated weights for policy 1, policy_version 206150 (0.0005) [2023-12-27 07:04:54,777][20381] Updated weights for policy 1, policy_version 206160 (0.0006) [2023-12-27 07:04:55,032][20347] Updated weights for policy 0, policy_version 206540 (0.0008) [2023-12-27 07:04:55,085][20347] Updated weights for policy 0, policy_version 206550 (0.0006) [2023-12-27 07:04:55,140][20347] Updated weights for policy 0, policy_version 206560 (0.0008) [2023-12-27 07:04:55,394][20381] Updated weights for policy 1, policy_version 206170 (0.0008) [2023-12-27 07:04:55,450][20381] Updated weights for policy 1, policy_version 206180 (0.0006) [2023-12-27 07:04:55,509][20381] Updated weights for policy 1, policy_version 206190 (0.0005) [2023-12-27 07:04:55,764][20347] Updated weights for policy 0, policy_version 206570 (0.0007) [2023-12-27 07:04:55,824][20347] Updated weights for policy 0, policy_version 206580 (0.0007) [2023-12-27 07:04:55,883][20347] Updated weights for policy 0, policy_version 206590 (0.0008) [2023-12-27 07:04:55,938][20347] Updated weights for policy 0, policy_version 206600 (0.0007) [2023-12-27 07:04:56,149][20381] Updated weights for policy 1, policy_version 206200 (0.0007) [2023-12-27 07:04:56,207][20381] Updated weights for policy 1, policy_version 206210 (0.0008) [2023-12-27 07:04:56,264][20381] Updated weights for policy 1, policy_version 206220 (0.0008) [2023-12-27 07:04:56,625][20347] Updated weights for policy 0, policy_version 206610 (0.0007) [2023-12-27 07:04:56,677][20347] Updated weights for policy 0, policy_version 206620 (0.0007) [2023-12-27 07:04:56,731][20347] Updated weights for policy 0, policy_version 206630 (0.0007) [2023-12-27 07:04:56,958][20381] Updated weights for policy 1, policy_version 206230 (0.0007) [2023-12-27 07:04:57,014][20381] Updated weights for policy 1, policy_version 206240 (0.0006) [2023-12-27 07:04:57,069][20381] Updated weights for policy 1, policy_version 206250 (0.0006) [2023-12-27 07:04:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 105725952. Throughput: 0: 10501.7, 1: 10202.2. Samples: 105716436. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:04:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '758.620')] [2023-12-27 07:04:57,433][20347] Updated weights for policy 0, policy_version 206640 (0.0007) [2023-12-27 07:04:57,495][20347] Updated weights for policy 0, policy_version 206650 (0.0006) [2023-12-27 07:04:57,548][20347] Updated weights for policy 0, policy_version 206660 (0.0007) [2023-12-27 07:04:57,690][20381] Updated weights for policy 1, policy_version 206260 (0.0006) [2023-12-27 07:04:57,736][20381] Updated weights for policy 1, policy_version 206270 (0.0007) [2023-12-27 07:04:57,782][20381] Updated weights for policy 1, policy_version 206280 (0.0005) [2023-12-27 07:04:58,280][20347] Updated weights for policy 0, policy_version 206670 (0.0008) [2023-12-27 07:04:58,340][20347] Updated weights for policy 0, policy_version 206680 (0.0008) [2023-12-27 07:04:58,386][20381] Updated weights for policy 1, policy_version 206290 (0.0005) [2023-12-27 07:04:58,402][20347] Updated weights for policy 0, policy_version 206690 (0.0007) [2023-12-27 07:04:58,449][20381] Updated weights for policy 1, policy_version 206300 (0.0006) [2023-12-27 07:04:58,507][20381] Updated weights for policy 1, policy_version 206310 (0.0007) [2023-12-27 07:04:58,571][20381] Updated weights for policy 1, policy_version 206320 (0.0007) [2023-12-27 07:04:59,107][20347] Updated weights for policy 0, policy_version 206700 (0.0006) [2023-12-27 07:04:59,157][20347] Updated weights for policy 0, policy_version 206710 (0.0006) [2023-12-27 07:04:59,211][20347] Updated weights for policy 0, policy_version 206720 (0.0007) [2023-12-27 07:04:59,284][20381] Updated weights for policy 1, policy_version 206330 (0.0007) [2023-12-27 07:04:59,348][20381] Updated weights for policy 1, policy_version 206340 (0.0006) [2023-12-27 07:04:59,404][20381] Updated weights for policy 1, policy_version 206350 (0.0006) [2023-12-27 07:04:59,967][20347] Updated weights for policy 0, policy_version 206730 (0.0007) [2023-12-27 07:05:00,019][20347] Updated weights for policy 0, policy_version 206740 (0.0007) [2023-12-27 07:05:00,025][20381] Updated weights for policy 1, policy_version 206360 (0.0006) [2023-12-27 07:05:00,078][20347] Updated weights for policy 0, policy_version 206750 (0.0006) [2023-12-27 07:05:00,080][20381] Updated weights for policy 1, policy_version 206370 (0.0006) [2023-12-27 07:05:00,135][20347] Updated weights for policy 0, policy_version 206760 (0.0006) [2023-12-27 07:05:00,140][20381] Updated weights for policy 1, policy_version 206380 (0.0007) [2023-12-27 07:05:00,772][20381] Updated weights for policy 1, policy_version 206390 (0.0008) [2023-12-27 07:05:00,826][20381] Updated weights for policy 1, policy_version 206400 (0.0007) [2023-12-27 07:05:00,876][20347] Updated weights for policy 0, policy_version 206770 (0.0007) [2023-12-27 07:05:00,878][20381] Updated weights for policy 1, policy_version 206410 (0.0006) [2023-12-27 07:05:00,933][20347] Updated weights for policy 0, policy_version 206780 (0.0007) [2023-12-27 07:05:00,983][20347] Updated weights for policy 0, policy_version 206790 (0.0007) [2023-12-27 07:05:01,543][20381] Updated weights for policy 1, policy_version 206420 (0.0006) [2023-12-27 07:05:01,610][20381] Updated weights for policy 1, policy_version 206430 (0.0006) [2023-12-27 07:05:01,670][20381] Updated weights for policy 1, policy_version 206440 (0.0011) [2023-12-27 07:05:01,716][20347] Updated weights for policy 0, policy_version 206800 (0.0008) [2023-12-27 07:05:01,774][20347] Updated weights for policy 0, policy_version 206810 (0.0007) [2023-12-27 07:05:01,830][20347] Updated weights for policy 0, policy_version 206820 (0.0008) [2023-12-27 07:05:02,358][20381] Updated weights for policy 1, policy_version 206450 (0.0010) [2023-12-27 07:05:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 105824256. Throughput: 0: 10423.4, 1: 10311.5. Samples: 105838280. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '707.200')] [2023-12-27 07:05:02,414][20381] Updated weights for policy 1, policy_version 206460 (0.0010) [2023-12-27 07:05:02,462][20381] Updated weights for policy 1, policy_version 206470 (0.0010) [2023-12-27 07:05:02,482][20347] Updated weights for policy 0, policy_version 206830 (0.0006) [2023-12-27 07:05:02,520][20381] Updated weights for policy 1, policy_version 206480 (0.0009) [2023-12-27 07:05:02,536][20347] Updated weights for policy 0, policy_version 206840 (0.0007) [2023-12-27 07:05:02,590][20347] Updated weights for policy 0, policy_version 206850 (0.0007) [2023-12-27 07:05:03,114][20381] Updated weights for policy 1, policy_version 206490 (0.0005) [2023-12-27 07:05:03,158][20381] Updated weights for policy 1, policy_version 206500 (0.0006) [2023-12-27 07:05:03,205][20381] Updated weights for policy 1, policy_version 206510 (0.0006) [2023-12-27 07:05:03,250][20347] Updated weights for policy 0, policy_version 206860 (0.0007) [2023-12-27 07:05:03,295][20347] Updated weights for policy 0, policy_version 206870 (0.0007) [2023-12-27 07:05:03,342][20347] Updated weights for policy 0, policy_version 206880 (0.0008) [2023-12-27 07:05:03,812][20381] Updated weights for policy 1, policy_version 206520 (0.0007) [2023-12-27 07:05:03,876][20381] Updated weights for policy 1, policy_version 206530 (0.0008) [2023-12-27 07:05:03,930][20381] Updated weights for policy 1, policy_version 206540 (0.0008) [2023-12-27 07:05:04,000][20347] Updated weights for policy 0, policy_version 206890 (0.0006) [2023-12-27 07:05:04,055][20347] Updated weights for policy 0, policy_version 206900 (0.0005) [2023-12-27 07:05:04,114][20347] Updated weights for policy 0, policy_version 206910 (0.0006) [2023-12-27 07:05:04,177][20347] Updated weights for policy 0, policy_version 206920 (0.0006) [2023-12-27 07:05:04,673][20381] Updated weights for policy 1, policy_version 206550 (0.0008) [2023-12-27 07:05:04,735][20381] Updated weights for policy 1, policy_version 206560 (0.0007) [2023-12-27 07:05:04,790][20381] Updated weights for policy 1, policy_version 206570 (0.0009) [2023-12-27 07:05:04,799][20347] Updated weights for policy 0, policy_version 206930 (0.0007) [2023-12-27 07:05:04,858][20347] Updated weights for policy 0, policy_version 206940 (0.0008) [2023-12-27 07:05:04,920][20347] Updated weights for policy 0, policy_version 206950 (0.0008) [2023-12-27 07:05:05,469][20381] Updated weights for policy 1, policy_version 206580 (0.0008) [2023-12-27 07:05:05,514][20381] Updated weights for policy 1, policy_version 206590 (0.0010) [2023-12-27 07:05:05,559][20381] Updated weights for policy 1, policy_version 206600 (0.0010) [2023-12-27 07:05:05,603][20347] Updated weights for policy 0, policy_version 206960 (0.0007) [2023-12-27 07:05:05,652][20347] Updated weights for policy 0, policy_version 206970 (0.0007) [2023-12-27 07:05:05,707][20347] Updated weights for policy 0, policy_version 206980 (0.0008) [2023-12-27 07:05:06,217][20381] Updated weights for policy 1, policy_version 206610 (0.0010) [2023-12-27 07:05:06,275][20381] Updated weights for policy 1, policy_version 206620 (0.0007) [2023-12-27 07:05:06,335][20381] Updated weights for policy 1, policy_version 206630 (0.0008) [2023-12-27 07:05:06,391][20381] Updated weights for policy 1, policy_version 206640 (0.0008) [2023-12-27 07:05:06,444][20347] Updated weights for policy 0, policy_version 206990 (0.0010) [2023-12-27 07:05:06,493][20347] Updated weights for policy 0, policy_version 207000 (0.0010) [2023-12-27 07:05:06,546][20347] Updated weights for policy 0, policy_version 207010 (0.0010) [2023-12-27 07:05:07,062][20381] Updated weights for policy 1, policy_version 206650 (0.0008) [2023-12-27 07:05:07,128][20381] Updated weights for policy 1, policy_version 206660 (0.0007) [2023-12-27 07:05:07,191][20381] Updated weights for policy 1, policy_version 206670 (0.0008) [2023-12-27 07:05:07,205][20347] Updated weights for policy 0, policy_version 207020 (0.0008) [2023-12-27 07:05:07,263][20347] Updated weights for policy 0, policy_version 207030 (0.0007) [2023-12-27 07:05:07,324][20347] Updated weights for policy 0, policy_version 207040 (0.0007) [2023-12-27 07:05:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 105930752. Throughput: 0: 10468.7, 1: 10357.5. Samples: 105901664. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '758.040')] [2023-12-27 07:05:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000207048_53018624.pth... [2023-12-27 07:05:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000206672_52920320.pth... [2023-12-27 07:05:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000205832_52707328.pth [2023-12-27 07:05:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000205456_52609024.pth [2023-12-27 07:05:07,815][20381] Updated weights for policy 1, policy_version 206680 (0.0008) [2023-12-27 07:05:07,872][20381] Updated weights for policy 1, policy_version 206690 (0.0008) [2023-12-27 07:05:07,918][20347] Updated weights for policy 0, policy_version 207050 (0.0007) [2023-12-27 07:05:07,926][20381] Updated weights for policy 1, policy_version 206700 (0.0007) [2023-12-27 07:05:07,977][20347] Updated weights for policy 0, policy_version 207060 (0.0005) [2023-12-27 07:05:08,037][20347] Updated weights for policy 0, policy_version 207070 (0.0005) [2023-12-27 07:05:08,087][20347] Updated weights for policy 0, policy_version 207080 (0.0010) [2023-12-27 07:05:08,534][20381] Updated weights for policy 1, policy_version 206710 (0.0006) [2023-12-27 07:05:08,593][20381] Updated weights for policy 1, policy_version 206720 (0.0006) [2023-12-27 07:05:08,651][20381] Updated weights for policy 1, policy_version 206730 (0.0008) [2023-12-27 07:05:08,690][20347] Updated weights for policy 0, policy_version 207090 (0.0010) [2023-12-27 07:05:08,752][20347] Updated weights for policy 0, policy_version 207100 (0.0010) [2023-12-27 07:05:08,800][20347] Updated weights for policy 0, policy_version 207110 (0.0010) [2023-12-27 07:05:09,268][20381] Updated weights for policy 1, policy_version 206740 (0.0008) [2023-12-27 07:05:09,327][20381] Updated weights for policy 1, policy_version 206750 (0.0010) [2023-12-27 07:05:09,392][20381] Updated weights for policy 1, policy_version 206760 (0.0010) [2023-12-27 07:05:09,513][20347] Updated weights for policy 0, policy_version 207120 (0.0009) [2023-12-27 07:05:09,585][20347] Updated weights for policy 0, policy_version 207130 (0.0006) [2023-12-27 07:05:09,655][20347] Updated weights for policy 0, policy_version 207140 (0.0007) [2023-12-27 07:05:10,063][20381] Updated weights for policy 1, policy_version 206770 (0.0007) [2023-12-27 07:05:10,127][20381] Updated weights for policy 1, policy_version 206780 (0.0007) [2023-12-27 07:05:10,183][20381] Updated weights for policy 1, policy_version 206790 (0.0008) [2023-12-27 07:05:10,245][20381] Updated weights for policy 1, policy_version 206800 (0.0008) [2023-12-27 07:05:10,306][20347] Updated weights for policy 0, policy_version 207150 (0.0007) [2023-12-27 07:05:10,371][20347] Updated weights for policy 0, policy_version 207160 (0.0007) [2023-12-27 07:05:10,422][20347] Updated weights for policy 0, policy_version 207170 (0.0006) [2023-12-27 07:05:10,961][20347] Updated weights for policy 0, policy_version 207180 (0.0006) [2023-12-27 07:05:11,006][20381] Updated weights for policy 1, policy_version 206810 (0.0007) [2023-12-27 07:05:11,017][20347] Updated weights for policy 0, policy_version 207190 (0.0007) [2023-12-27 07:05:11,075][20381] Updated weights for policy 1, policy_version 206820 (0.0008) [2023-12-27 07:05:11,077][20347] Updated weights for policy 0, policy_version 207200 (0.0008) [2023-12-27 07:05:11,141][20381] Updated weights for policy 1, policy_version 206830 (0.0008) [2023-12-27 07:05:11,810][20347] Updated weights for policy 0, policy_version 207210 (0.0006) [2023-12-27 07:05:11,841][20381] Updated weights for policy 1, policy_version 206840 (0.0006) [2023-12-27 07:05:11,871][20347] Updated weights for policy 0, policy_version 207220 (0.0008) [2023-12-27 07:05:11,900][20381] Updated weights for policy 1, policy_version 206850 (0.0007) [2023-12-27 07:05:11,924][20347] Updated weights for policy 0, policy_version 207230 (0.0008) [2023-12-27 07:05:11,952][20381] Updated weights for policy 1, policy_version 206860 (0.0006) [2023-12-27 07:05:11,972][20347] Updated weights for policy 0, policy_version 207240 (0.0006) [2023-12-27 07:05:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 106037248. Throughput: 0: 10512.5, 1: 10433.2. Samples: 106028072. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '794.390')] [2023-12-27 07:05:12,621][20347] Updated weights for policy 0, policy_version 207250 (0.0008) [2023-12-27 07:05:12,659][20381] Updated weights for policy 1, policy_version 206870 (0.0005) [2023-12-27 07:05:12,678][20347] Updated weights for policy 0, policy_version 207260 (0.0008) [2023-12-27 07:05:12,719][20381] Updated weights for policy 1, policy_version 206880 (0.0007) [2023-12-27 07:05:12,732][20347] Updated weights for policy 0, policy_version 207270 (0.0007) [2023-12-27 07:05:12,779][20381] Updated weights for policy 1, policy_version 206890 (0.0007) [2023-12-27 07:05:13,402][20347] Updated weights for policy 0, policy_version 207280 (0.0005) [2023-12-27 07:05:13,450][20347] Updated weights for policy 0, policy_version 207290 (0.0005) [2023-12-27 07:05:13,476][20381] Updated weights for policy 1, policy_version 206900 (0.0007) [2023-12-27 07:05:13,508][20347] Updated weights for policy 0, policy_version 207300 (0.0006) [2023-12-27 07:05:13,531][20381] Updated weights for policy 1, policy_version 206910 (0.0007) [2023-12-27 07:05:13,587][20381] Updated weights for policy 1, policy_version 206920 (0.0009) [2023-12-27 07:05:14,073][20347] Updated weights for policy 0, policy_version 207310 (0.0005) [2023-12-27 07:05:14,134][20347] Updated weights for policy 0, policy_version 207320 (0.0005) [2023-12-27 07:05:14,196][20347] Updated weights for policy 0, policy_version 207330 (0.0005) [2023-12-27 07:05:14,329][20381] Updated weights for policy 1, policy_version 206930 (0.0007) [2023-12-27 07:05:14,389][20381] Updated weights for policy 1, policy_version 206940 (0.0005) [2023-12-27 07:05:14,445][20381] Updated weights for policy 1, policy_version 206950 (0.0005) [2023-12-27 07:05:14,500][20381] Updated weights for policy 1, policy_version 206960 (0.0006) [2023-12-27 07:05:14,727][20347] Updated weights for policy 0, policy_version 207340 (0.0007) [2023-12-27 07:05:14,784][20347] Updated weights for policy 0, policy_version 207350 (0.0008) [2023-12-27 07:05:14,832][20347] Updated weights for policy 0, policy_version 207360 (0.0007) [2023-12-27 07:05:15,089][20381] Updated weights for policy 1, policy_version 206970 (0.0008) [2023-12-27 07:05:15,136][20381] Updated weights for policy 1, policy_version 206980 (0.0008) [2023-12-27 07:05:15,186][20381] Updated weights for policy 1, policy_version 206990 (0.0008) [2023-12-27 07:05:15,554][20347] Updated weights for policy 0, policy_version 207370 (0.0008) [2023-12-27 07:05:15,604][20347] Updated weights for policy 0, policy_version 207380 (0.0008) [2023-12-27 07:05:15,650][20347] Updated weights for policy 0, policy_version 207390 (0.0008) [2023-12-27 07:05:15,703][20347] Updated weights for policy 0, policy_version 207400 (0.0008) [2023-12-27 07:05:15,857][20381] Updated weights for policy 1, policy_version 207000 (0.0008) [2023-12-27 07:05:15,916][20381] Updated weights for policy 1, policy_version 207010 (0.0008) [2023-12-27 07:05:15,973][20381] Updated weights for policy 1, policy_version 207020 (0.0005) [2023-12-27 07:05:16,435][20347] Updated weights for policy 0, policy_version 207410 (0.0007) [2023-12-27 07:05:16,492][20347] Updated weights for policy 0, policy_version 207420 (0.0007) [2023-12-27 07:05:16,546][20347] Updated weights for policy 0, policy_version 207430 (0.0007) [2023-12-27 07:05:16,649][20381] Updated weights for policy 1, policy_version 207030 (0.0007) [2023-12-27 07:05:16,698][20381] Updated weights for policy 1, policy_version 207040 (0.0008) [2023-12-27 07:05:16,750][20381] Updated weights for policy 1, policy_version 207050 (0.0008) [2023-12-27 07:05:17,255][20347] Updated weights for policy 0, policy_version 207440 (0.0005) [2023-12-27 07:05:17,322][20347] Updated weights for policy 0, policy_version 207450 (0.0005) [2023-12-27 07:05:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 106135552. Throughput: 0: 10413.1, 1: 10373.3. Samples: 106151820. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '723.530')] [2023-12-27 07:05:17,383][20347] Updated weights for policy 0, policy_version 207460 (0.0006) [2023-12-27 07:05:17,397][20381] Updated weights for policy 1, policy_version 207060 (0.0008) [2023-12-27 07:05:17,446][20381] Updated weights for policy 1, policy_version 207070 (0.0008) [2023-12-27 07:05:17,499][20381] Updated weights for policy 1, policy_version 207080 (0.0008) [2023-12-27 07:05:17,978][20347] Updated weights for policy 0, policy_version 207470 (0.0008) [2023-12-27 07:05:18,027][20347] Updated weights for policy 0, policy_version 207480 (0.0008) [2023-12-27 07:05:18,073][20347] Updated weights for policy 0, policy_version 207490 (0.0005) [2023-12-27 07:05:18,204][20381] Updated weights for policy 1, policy_version 207090 (0.0008) [2023-12-27 07:05:18,251][20381] Updated weights for policy 1, policy_version 207100 (0.0008) [2023-12-27 07:05:18,297][20381] Updated weights for policy 1, policy_version 207110 (0.0008) [2023-12-27 07:05:18,312][20292] KL-divergence is very high: 110.9842 [2023-12-27 07:05:18,355][20381] Updated weights for policy 1, policy_version 207120 (0.0007) [2023-12-27 07:05:18,773][20347] Updated weights for policy 0, policy_version 207500 (0.0007) [2023-12-27 07:05:18,826][20347] Updated weights for policy 0, policy_version 207510 (0.0008) [2023-12-27 07:05:18,888][20347] Updated weights for policy 0, policy_version 207520 (0.0008) [2023-12-27 07:05:18,966][20381] Updated weights for policy 1, policy_version 207130 (0.0005) [2023-12-27 07:05:19,025][20381] Updated weights for policy 1, policy_version 207140 (0.0005) [2023-12-27 07:05:19,076][20381] Updated weights for policy 1, policy_version 207150 (0.0006) [2023-12-27 07:05:19,629][20347] Updated weights for policy 0, policy_version 207530 (0.0008) [2023-12-27 07:05:19,686][20347] Updated weights for policy 0, policy_version 207540 (0.0007) [2023-12-27 07:05:19,700][20381] Updated weights for policy 1, policy_version 207160 (0.0007) [2023-12-27 07:05:19,739][20347] Updated weights for policy 0, policy_version 207550 (0.0006) [2023-12-27 07:05:19,755][20381] Updated weights for policy 1, policy_version 207170 (0.0005) [2023-12-27 07:05:19,798][20347] Updated weights for policy 0, policy_version 207560 (0.0007) [2023-12-27 07:05:19,805][20381] Updated weights for policy 1, policy_version 207180 (0.0006) [2023-12-27 07:05:20,468][20347] Updated weights for policy 0, policy_version 207570 (0.0007) [2023-12-27 07:05:20,527][20347] Updated weights for policy 0, policy_version 207580 (0.0007) [2023-12-27 07:05:20,530][20381] Updated weights for policy 1, policy_version 207190 (0.0007) [2023-12-27 07:05:20,589][20347] Updated weights for policy 0, policy_version 207590 (0.0007) [2023-12-27 07:05:20,622][20381] Updated weights for policy 1, policy_version 207200 (0.0006) [2023-12-27 07:05:20,684][20381] Updated weights for policy 1, policy_version 207210 (0.0007) [2023-12-27 07:05:21,299][20347] Updated weights for policy 0, policy_version 207600 (0.0007) [2023-12-27 07:05:21,343][20381] Updated weights for policy 1, policy_version 207220 (0.0008) [2023-12-27 07:05:21,366][20347] Updated weights for policy 0, policy_version 207610 (0.0007) [2023-12-27 07:05:21,407][20381] Updated weights for policy 1, policy_version 207230 (0.0008) [2023-12-27 07:05:21,431][20347] Updated weights for policy 0, policy_version 207620 (0.0008) [2023-12-27 07:05:21,467][20381] Updated weights for policy 1, policy_version 207240 (0.0006) [2023-12-27 07:05:22,144][20347] Updated weights for policy 0, policy_version 207630 (0.0008) [2023-12-27 07:05:22,184][20381] Updated weights for policy 1, policy_version 207250 (0.0007) [2023-12-27 07:05:22,197][20347] Updated weights for policy 0, policy_version 207640 (0.0006) [2023-12-27 07:05:22,240][20381] Updated weights for policy 1, policy_version 207260 (0.0006) [2023-12-27 07:05:22,248][20347] Updated weights for policy 0, policy_version 207650 (0.0008) [2023-12-27 07:05:22,300][20381] Updated weights for policy 1, policy_version 207270 (0.0007) [2023-12-27 07:05:22,354][20381] Updated weights for policy 1, policy_version 207280 (0.0008) [2023-12-27 07:05:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 106250240. Throughput: 0: 10366.8, 1: 10342.6. Samples: 106213640. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '841.270')] [2023-12-27 07:05:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000207656_53174272.pth... [2023-12-27 07:05:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000207280_53075968.pth... [2023-12-27 07:05:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000206440_52862976.pth [2023-12-27 07:05:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000206064_52764672.pth [2023-12-27 07:05:22,994][20347] Updated weights for policy 0, policy_version 207660 (0.0009) [2023-12-27 07:05:23,041][20381] Updated weights for policy 1, policy_version 207290 (0.0006) [2023-12-27 07:05:23,051][20347] Updated weights for policy 0, policy_version 207670 (0.0006) [2023-12-27 07:05:23,098][20381] Updated weights for policy 1, policy_version 207300 (0.0007) [2023-12-27 07:05:23,104][20347] Updated weights for policy 0, policy_version 207680 (0.0006) [2023-12-27 07:05:23,155][20381] Updated weights for policy 1, policy_version 207310 (0.0006) [2023-12-27 07:05:23,785][20381] Updated weights for policy 1, policy_version 207320 (0.0008) [2023-12-27 07:05:23,795][20347] Updated weights for policy 0, policy_version 207690 (0.0006) [2023-12-27 07:05:23,832][20381] Updated weights for policy 1, policy_version 207330 (0.0007) [2023-12-27 07:05:23,852][20347] Updated weights for policy 0, policy_version 207700 (0.0010) [2023-12-27 07:05:23,879][20381] Updated weights for policy 1, policy_version 207340 (0.0006) [2023-12-27 07:05:23,905][20347] Updated weights for policy 0, policy_version 207710 (0.0005) [2023-12-27 07:05:23,960][20347] Updated weights for policy 0, policy_version 207720 (0.0005) [2023-12-27 07:05:24,504][20347] Updated weights for policy 0, policy_version 207730 (0.0010) [2023-12-27 07:05:24,558][20347] Updated weights for policy 0, policy_version 207740 (0.0010) [2023-12-27 07:05:24,609][20347] Updated weights for policy 0, policy_version 207750 (0.0010) [2023-12-27 07:05:24,633][20381] Updated weights for policy 1, policy_version 207350 (0.0007) [2023-12-27 07:05:24,686][20381] Updated weights for policy 1, policy_version 207360 (0.0009) [2023-12-27 07:05:24,740][20381] Updated weights for policy 1, policy_version 207370 (0.0007) [2023-12-27 07:05:25,198][20347] Updated weights for policy 0, policy_version 207760 (0.0006) [2023-12-27 07:05:25,241][20347] Updated weights for policy 0, policy_version 207770 (0.0005) [2023-12-27 07:05:25,286][20347] Updated weights for policy 0, policy_version 207780 (0.0005) [2023-12-27 07:05:25,501][20381] Updated weights for policy 1, policy_version 207380 (0.0007) [2023-12-27 07:05:25,556][20381] Updated weights for policy 1, policy_version 207390 (0.0006) [2023-12-27 07:05:25,602][20381] Updated weights for policy 1, policy_version 207400 (0.0005) [2023-12-27 07:05:25,902][20347] Updated weights for policy 0, policy_version 207790 (0.0007) [2023-12-27 07:05:25,959][20347] Updated weights for policy 0, policy_version 207800 (0.0008) [2023-12-27 07:05:26,016][20347] Updated weights for policy 0, policy_version 207810 (0.0008) [2023-12-27 07:05:26,248][20381] Updated weights for policy 1, policy_version 207410 (0.0007) [2023-12-27 07:05:26,296][20381] Updated weights for policy 1, policy_version 207420 (0.0006) [2023-12-27 07:05:26,344][20381] Updated weights for policy 1, policy_version 207430 (0.0005) [2023-12-27 07:05:26,397][20381] Updated weights for policy 1, policy_version 207440 (0.0005) [2023-12-27 07:05:26,718][20347] Updated weights for policy 0, policy_version 207820 (0.0009) [2023-12-27 07:05:26,770][20347] Updated weights for policy 0, policy_version 207830 (0.0006) [2023-12-27 07:05:26,822][20347] Updated weights for policy 0, policy_version 207840 (0.0005) [2023-12-27 07:05:27,049][20381] Updated weights for policy 1, policy_version 207450 (0.0010) [2023-12-27 07:05:27,097][20381] Updated weights for policy 1, policy_version 207460 (0.0010) [2023-12-27 07:05:27,148][20381] Updated weights for policy 1, policy_version 207470 (0.0010) [2023-12-27 07:05:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 106348544. Throughput: 0: 10416.6, 1: 10322.9. Samples: 106336624. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.410')] [2023-12-27 07:05:27,417][20347] Updated weights for policy 0, policy_version 207850 (0.0005) [2023-12-27 07:05:27,478][20347] Updated weights for policy 0, policy_version 207860 (0.0005) [2023-12-27 07:05:27,539][20347] Updated weights for policy 0, policy_version 207870 (0.0005) [2023-12-27 07:05:27,602][20347] Updated weights for policy 0, policy_version 207880 (0.0005) [2023-12-27 07:05:27,832][20381] Updated weights for policy 1, policy_version 207480 (0.0006) [2023-12-27 07:05:27,889][20381] Updated weights for policy 1, policy_version 207490 (0.0009) [2023-12-27 07:05:27,946][20381] Updated weights for policy 1, policy_version 207500 (0.0008) [2023-12-27 07:05:28,151][20347] Updated weights for policy 0, policy_version 207890 (0.0005) [2023-12-27 07:05:28,204][20347] Updated weights for policy 0, policy_version 207900 (0.0005) [2023-12-27 07:05:28,253][20347] Updated weights for policy 0, policy_version 207910 (0.0005) [2023-12-27 07:05:28,625][20381] Updated weights for policy 1, policy_version 207510 (0.0009) [2023-12-27 07:05:28,678][20381] Updated weights for policy 1, policy_version 207520 (0.0011) [2023-12-27 07:05:28,730][20381] Updated weights for policy 1, policy_version 207530 (0.0010) [2023-12-27 07:05:28,875][20347] Updated weights for policy 0, policy_version 207920 (0.0006) [2023-12-27 07:05:28,933][20347] Updated weights for policy 0, policy_version 207930 (0.0008) [2023-12-27 07:05:28,987][20347] Updated weights for policy 0, policy_version 207940 (0.0007) [2023-12-27 07:05:29,413][20381] Updated weights for policy 1, policy_version 207540 (0.0010) [2023-12-27 07:05:29,476][20381] Updated weights for policy 1, policy_version 207550 (0.0009) [2023-12-27 07:05:29,536][20381] Updated weights for policy 1, policy_version 207560 (0.0009) [2023-12-27 07:05:29,666][20347] Updated weights for policy 0, policy_version 207950 (0.0006) [2023-12-27 07:05:29,717][20347] Updated weights for policy 0, policy_version 207960 (0.0007) [2023-12-27 07:05:29,777][20347] Updated weights for policy 0, policy_version 207970 (0.0008) [2023-12-27 07:05:30,114][20381] Updated weights for policy 1, policy_version 207570 (0.0006) [2023-12-27 07:05:30,171][20381] Updated weights for policy 1, policy_version 207580 (0.0006) [2023-12-27 07:05:30,234][20381] Updated weights for policy 1, policy_version 207590 (0.0007) [2023-12-27 07:05:30,289][20381] Updated weights for policy 1, policy_version 207600 (0.0008) [2023-12-27 07:05:30,473][20347] Updated weights for policy 0, policy_version 207980 (0.0009) [2023-12-27 07:05:30,527][20347] Updated weights for policy 0, policy_version 207990 (0.0010) [2023-12-27 07:05:30,579][20347] Updated weights for policy 0, policy_version 208000 (0.0011) [2023-12-27 07:05:30,871][20381] Updated weights for policy 1, policy_version 207610 (0.0009) [2023-12-27 07:05:30,923][20381] Updated weights for policy 1, policy_version 207620 (0.0008) [2023-12-27 07:05:30,971][20381] Updated weights for policy 1, policy_version 207630 (0.0009) [2023-12-27 07:05:31,160][20347] Updated weights for policy 0, policy_version 208011 (0.0007) [2023-12-27 07:05:31,220][20347] Updated weights for policy 0, policy_version 208021 (0.0008) [2023-12-27 07:05:31,276][20347] Updated weights for policy 0, policy_version 208031 (0.0008) [2023-12-27 07:05:31,643][20381] Updated weights for policy 1, policy_version 207640 (0.0009) [2023-12-27 07:05:31,695][20381] Updated weights for policy 1, policy_version 207650 (0.0008) [2023-12-27 07:05:31,760][20381] Updated weights for policy 1, policy_version 207660 (0.0008) [2023-12-27 07:05:32,025][20347] Updated weights for policy 0, policy_version 208041 (0.0008) [2023-12-27 07:05:32,075][20347] Updated weights for policy 0, policy_version 208051 (0.0009) [2023-12-27 07:05:32,134][20347] Updated weights for policy 0, policy_version 208061 (0.0007) [2023-12-27 07:05:32,182][20347] Updated weights for policy 0, policy_version 208071 (0.0007) [2023-12-27 07:05:32,357][20381] Updated weights for policy 1, policy_version 207670 (0.0008) [2023-12-27 07:05:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 106455040. Throughput: 0: 10408.9, 1: 10392.7. Samples: 106465172. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.610')] [2023-12-27 07:05:32,413][20381] Updated weights for policy 1, policy_version 207680 (0.0006) [2023-12-27 07:05:32,468][20381] Updated weights for policy 1, policy_version 207690 (0.0006) [2023-12-27 07:05:32,908][20347] Updated weights for policy 0, policy_version 208081 (0.0008) [2023-12-27 07:05:32,956][20347] Updated weights for policy 0, policy_version 208091 (0.0007) [2023-12-27 07:05:33,000][20381] Updated weights for policy 1, policy_version 207700 (0.0005) [2023-12-27 07:05:33,013][20347] Updated weights for policy 0, policy_version 208101 (0.0008) [2023-12-27 07:05:33,059][20381] Updated weights for policy 1, policy_version 207710 (0.0007) [2023-12-27 07:05:33,109][20381] Updated weights for policy 1, policy_version 207720 (0.0005) [2023-12-27 07:05:33,681][20381] Updated weights for policy 1, policy_version 207730 (0.0005) [2023-12-27 07:05:33,730][20347] Updated weights for policy 0, policy_version 208111 (0.0008) [2023-12-27 07:05:33,732][20381] Updated weights for policy 1, policy_version 207740 (0.0008) [2023-12-27 07:05:33,782][20347] Updated weights for policy 0, policy_version 208121 (0.0006) [2023-12-27 07:05:33,784][20381] Updated weights for policy 1, policy_version 207750 (0.0006) [2023-12-27 07:05:33,838][20347] Updated weights for policy 0, policy_version 208131 (0.0007) [2023-12-27 07:05:33,848][20381] Updated weights for policy 1, policy_version 207760 (0.0007) [2023-12-27 07:05:34,456][20381] Updated weights for policy 1, policy_version 207770 (0.0008) [2023-12-27 07:05:34,482][20347] Updated weights for policy 0, policy_version 208141 (0.0009) [2023-12-27 07:05:34,503][20381] Updated weights for policy 1, policy_version 207780 (0.0005) [2023-12-27 07:05:34,542][20347] Updated weights for policy 0, policy_version 208151 (0.0010) [2023-12-27 07:05:34,559][20381] Updated weights for policy 1, policy_version 207790 (0.0006) [2023-12-27 07:05:34,592][20347] Updated weights for policy 0, policy_version 208161 (0.0010) [2023-12-27 07:05:35,191][20381] Updated weights for policy 1, policy_version 207800 (0.0008) [2023-12-27 07:05:35,248][20381] Updated weights for policy 1, policy_version 207810 (0.0008) [2023-12-27 07:05:35,288][20347] Updated weights for policy 0, policy_version 208171 (0.0009) [2023-12-27 07:05:35,309][20381] Updated weights for policy 1, policy_version 207820 (0.0007) [2023-12-27 07:05:35,337][20347] Updated weights for policy 0, policy_version 208181 (0.0006) [2023-12-27 07:05:35,398][20347] Updated weights for policy 0, policy_version 208191 (0.0008) [2023-12-27 07:05:35,910][20381] Updated weights for policy 1, policy_version 207830 (0.0005) [2023-12-27 07:05:35,956][20381] Updated weights for policy 1, policy_version 207840 (0.0005) [2023-12-27 07:05:36,001][20381] Updated weights for policy 1, policy_version 207850 (0.0005) [2023-12-27 07:05:36,121][20347] Updated weights for policy 0, policy_version 208201 (0.0007) [2023-12-27 07:05:36,180][20347] Updated weights for policy 0, policy_version 208211 (0.0008) [2023-12-27 07:05:36,239][20347] Updated weights for policy 0, policy_version 208221 (0.0008) [2023-12-27 07:05:36,298][20347] Updated weights for policy 0, policy_version 208231 (0.0007) [2023-12-27 07:05:36,692][20381] Updated weights for policy 1, policy_version 207860 (0.0006) [2023-12-27 07:05:36,744][20381] Updated weights for policy 1, policy_version 207870 (0.0006) [2023-12-27 07:05:36,802][20381] Updated weights for policy 1, policy_version 207880 (0.0006) [2023-12-27 07:05:36,961][20347] Updated weights for policy 0, policy_version 208241 (0.0006) [2023-12-27 07:05:37,015][20347] Updated weights for policy 0, policy_version 208251 (0.0005) [2023-12-27 07:05:37,066][20347] Updated weights for policy 0, policy_version 208261 (0.0005) [2023-12-27 07:05:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 106561536. Throughput: 0: 10368.0, 1: 10486.3. Samples: 106529868. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '980.750')] [2023-12-27 07:05:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000207888_53231616.pth... [2023-12-27 07:05:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000208264_53329920.pth... [2023-12-27 07:05:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000206672_52920320.pth [2023-12-27 07:05:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000207048_53018624.pth [2023-12-27 07:05:37,485][20381] Updated weights for policy 1, policy_version 207890 (0.0008) [2023-12-27 07:05:37,543][20381] Updated weights for policy 1, policy_version 207900 (0.0007) [2023-12-27 07:05:37,602][20381] Updated weights for policy 1, policy_version 207910 (0.0008) [2023-12-27 07:05:37,670][20381] Updated weights for policy 1, policy_version 207920 (0.0008) [2023-12-27 07:05:37,711][20347] Updated weights for policy 0, policy_version 208271 (0.0007) [2023-12-27 07:05:37,769][20347] Updated weights for policy 0, policy_version 208281 (0.0006) [2023-12-27 07:05:37,825][20347] Updated weights for policy 0, policy_version 208291 (0.0007) [2023-12-27 07:05:38,337][20381] Updated weights for policy 1, policy_version 207930 (0.0008) [2023-12-27 07:05:38,399][20381] Updated weights for policy 1, policy_version 207940 (0.0006) [2023-12-27 07:05:38,416][20347] Updated weights for policy 0, policy_version 208301 (0.0007) [2023-12-27 07:05:38,465][20381] Updated weights for policy 1, policy_version 207950 (0.0005) [2023-12-27 07:05:38,472][20347] Updated weights for policy 0, policy_version 208311 (0.0008) [2023-12-27 07:05:38,519][20347] Updated weights for policy 0, policy_version 208321 (0.0008) [2023-12-27 07:05:39,041][20381] Updated weights for policy 1, policy_version 207960 (0.0007) [2023-12-27 07:05:39,100][20381] Updated weights for policy 1, policy_version 207970 (0.0008) [2023-12-27 07:05:39,159][20381] Updated weights for policy 1, policy_version 207980 (0.0008) [2023-12-27 07:05:39,290][20347] Updated weights for policy 0, policy_version 208331 (0.0008) [2023-12-27 07:05:39,354][20347] Updated weights for policy 0, policy_version 208341 (0.0008) [2023-12-27 07:05:39,416][20347] Updated weights for policy 0, policy_version 208351 (0.0008) [2023-12-27 07:05:39,848][20381] Updated weights for policy 1, policy_version 207990 (0.0007) [2023-12-27 07:05:39,916][20381] Updated weights for policy 1, policy_version 208000 (0.0007) [2023-12-27 07:05:39,979][20381] Updated weights for policy 1, policy_version 208010 (0.0007) [2023-12-27 07:05:40,164][20347] Updated weights for policy 0, policy_version 208361 (0.0007) [2023-12-27 07:05:40,225][20347] Updated weights for policy 0, policy_version 208371 (0.0008) [2023-12-27 07:05:40,286][20347] Updated weights for policy 0, policy_version 208381 (0.0008) [2023-12-27 07:05:40,350][20347] Updated weights for policy 0, policy_version 208391 (0.0008) [2023-12-27 07:05:40,548][20381] Updated weights for policy 1, policy_version 208020 (0.0006) [2023-12-27 07:05:40,612][20381] Updated weights for policy 1, policy_version 208030 (0.0007) [2023-12-27 07:05:40,673][20381] Updated weights for policy 1, policy_version 208040 (0.0007) [2023-12-27 07:05:41,047][20347] Updated weights for policy 0, policy_version 208401 (0.0008) [2023-12-27 07:05:41,111][20347] Updated weights for policy 0, policy_version 208411 (0.0008) [2023-12-27 07:05:41,175][20347] Updated weights for policy 0, policy_version 208421 (0.0007) [2023-12-27 07:05:41,368][20381] Updated weights for policy 1, policy_version 208050 (0.0008) [2023-12-27 07:05:41,429][20381] Updated weights for policy 1, policy_version 208060 (0.0006) [2023-12-27 07:05:41,493][20381] Updated weights for policy 1, policy_version 208070 (0.0007) [2023-12-27 07:05:41,548][20381] Updated weights for policy 1, policy_version 208080 (0.0007) [2023-12-27 07:05:41,887][20347] Updated weights for policy 0, policy_version 208431 (0.0006) [2023-12-27 07:05:41,948][20347] Updated weights for policy 0, policy_version 208441 (0.0008) [2023-12-27 07:05:42,008][20347] Updated weights for policy 0, policy_version 208451 (0.0007) [2023-12-27 07:05:42,251][20381] Updated weights for policy 1, policy_version 208090 (0.0006) [2023-12-27 07:05:42,313][20381] Updated weights for policy 1, policy_version 208100 (0.0008) [2023-12-27 07:05:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 106659840. Throughput: 0: 10323.4, 1: 10500.3. Samples: 106653508. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.290')] [2023-12-27 07:05:42,382][20381] Updated weights for policy 1, policy_version 208110 (0.0008) [2023-12-27 07:05:42,649][20347] Updated weights for policy 0, policy_version 208461 (0.0008) [2023-12-27 07:05:42,700][20347] Updated weights for policy 0, policy_version 208471 (0.0008) [2023-12-27 07:05:42,751][20347] Updated weights for policy 0, policy_version 208481 (0.0007) [2023-12-27 07:05:43,015][20381] Updated weights for policy 1, policy_version 208120 (0.0009) [2023-12-27 07:05:43,061][20381] Updated weights for policy 1, policy_version 208130 (0.0008) [2023-12-27 07:05:43,106][20381] Updated weights for policy 1, policy_version 208140 (0.0009) [2023-12-27 07:05:43,348][20347] Updated weights for policy 0, policy_version 208491 (0.0006) [2023-12-27 07:05:43,398][20347] Updated weights for policy 0, policy_version 208501 (0.0005) [2023-12-27 07:05:43,453][20347] Updated weights for policy 0, policy_version 208511 (0.0005) [2023-12-27 07:05:43,892][20381] Updated weights for policy 1, policy_version 208150 (0.0008) [2023-12-27 07:05:43,946][20381] Updated weights for policy 1, policy_version 208160 (0.0007) [2023-12-27 07:05:43,982][20347] Updated weights for policy 0, policy_version 208521 (0.0005) [2023-12-27 07:05:44,003][20381] Updated weights for policy 1, policy_version 208170 (0.0007) [2023-12-27 07:05:44,039][20347] Updated weights for policy 0, policy_version 208531 (0.0006) [2023-12-27 07:05:44,087][20347] Updated weights for policy 0, policy_version 208541 (0.0007) [2023-12-27 07:05:44,138][20347] Updated weights for policy 0, policy_version 208551 (0.0007) [2023-12-27 07:05:44,672][20381] Updated weights for policy 1, policy_version 208180 (0.0007) [2023-12-27 07:05:44,725][20381] Updated weights for policy 1, policy_version 208190 (0.0008) [2023-12-27 07:05:44,780][20381] Updated weights for policy 1, policy_version 208200 (0.0007) [2023-12-27 07:05:44,803][20347] Updated weights for policy 0, policy_version 208561 (0.0007) [2023-12-27 07:05:44,859][20347] Updated weights for policy 0, policy_version 208571 (0.0008) [2023-12-27 07:05:44,919][20347] Updated weights for policy 0, policy_version 208581 (0.0008) [2023-12-27 07:05:45,505][20381] Updated weights for policy 1, policy_version 208210 (0.0007) [2023-12-27 07:05:45,558][20381] Updated weights for policy 1, policy_version 208220 (0.0008) [2023-12-27 07:05:45,604][20381] Updated weights for policy 1, policy_version 208230 (0.0008) [2023-12-27 07:05:45,663][20381] Updated weights for policy 1, policy_version 208240 (0.0008) [2023-12-27 07:05:45,665][20347] Updated weights for policy 0, policy_version 208591 (0.0007) [2023-12-27 07:05:45,714][20347] Updated weights for policy 0, policy_version 208601 (0.0008) [2023-12-27 07:05:45,763][20347] Updated weights for policy 0, policy_version 208611 (0.0007) [2023-12-27 07:05:46,381][20381] Updated weights for policy 1, policy_version 208250 (0.0010) [2023-12-27 07:05:46,428][20347] Updated weights for policy 0, policy_version 208621 (0.0006) [2023-12-27 07:05:46,444][20381] Updated weights for policy 1, policy_version 208260 (0.0010) [2023-12-27 07:05:46,479][20347] Updated weights for policy 0, policy_version 208631 (0.0006) [2023-12-27 07:05:46,492][20381] Updated weights for policy 1, policy_version 208270 (0.0010) [2023-12-27 07:05:46,534][20347] Updated weights for policy 0, policy_version 208641 (0.0006) [2023-12-27 07:05:47,135][20347] Updated weights for policy 0, policy_version 208651 (0.0006) [2023-12-27 07:05:47,161][20381] Updated weights for policy 1, policy_version 208280 (0.0010) [2023-12-27 07:05:47,190][20347] Updated weights for policy 0, policy_version 208661 (0.0005) [2023-12-27 07:05:47,215][20381] Updated weights for policy 1, policy_version 208290 (0.0010) [2023-12-27 07:05:47,238][20347] Updated weights for policy 0, policy_version 208671 (0.0005) [2023-12-27 07:05:47,259][20381] Updated weights for policy 1, policy_version 208300 (0.0010) [2023-12-27 07:05:47,363][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 106774528. Throughput: 0: 10459.3, 1: 10412.4. Samples: 106777512. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '953.080')] [2023-12-27 07:05:47,802][20347] Updated weights for policy 0, policy_version 208681 (0.0006) [2023-12-27 07:05:47,858][20347] Updated weights for policy 0, policy_version 208691 (0.0007) [2023-12-27 07:05:47,906][20347] Updated weights for policy 0, policy_version 208701 (0.0008) [2023-12-27 07:05:47,933][20381] Updated weights for policy 1, policy_version 208310 (0.0007) [2023-12-27 07:05:47,951][20347] Updated weights for policy 0, policy_version 208711 (0.0007) [2023-12-27 07:05:47,997][20381] Updated weights for policy 1, policy_version 208320 (0.0009) [2023-12-27 07:05:48,053][20381] Updated weights for policy 1, policy_version 208330 (0.0010) [2023-12-27 07:05:48,656][20347] Updated weights for policy 0, policy_version 208721 (0.0006) [2023-12-27 07:05:48,713][20347] Updated weights for policy 0, policy_version 208731 (0.0005) [2023-12-27 07:05:48,763][20381] Updated weights for policy 1, policy_version 208340 (0.0010) [2023-12-27 07:05:48,776][20347] Updated weights for policy 0, policy_version 208741 (0.0005) [2023-12-27 07:05:48,826][20381] Updated weights for policy 1, policy_version 208350 (0.0008) [2023-12-27 07:05:48,881][20381] Updated weights for policy 1, policy_version 208360 (0.0007) [2023-12-27 07:05:49,302][20347] Updated weights for policy 0, policy_version 208751 (0.0007) [2023-12-27 07:05:49,361][20347] Updated weights for policy 0, policy_version 208761 (0.0007) [2023-12-27 07:05:49,418][20347] Updated weights for policy 0, policy_version 208771 (0.0008) [2023-12-27 07:05:49,581][20381] Updated weights for policy 1, policy_version 208370 (0.0008) [2023-12-27 07:05:49,639][20381] Updated weights for policy 1, policy_version 208380 (0.0011) [2023-12-27 07:05:49,698][20381] Updated weights for policy 1, policy_version 208390 (0.0011) [2023-12-27 07:05:49,757][20381] Updated weights for policy 1, policy_version 208400 (0.0010) [2023-12-27 07:05:50,086][20347] Updated weights for policy 0, policy_version 208781 (0.0007) [2023-12-27 07:05:50,147][20347] Updated weights for policy 0, policy_version 208791 (0.0007) [2023-12-27 07:05:50,204][20347] Updated weights for policy 0, policy_version 208801 (0.0007) [2023-12-27 07:05:50,455][20381] Updated weights for policy 1, policy_version 208410 (0.0010) [2023-12-27 07:05:50,507][20381] Updated weights for policy 1, policy_version 208420 (0.0010) [2023-12-27 07:05:50,569][20381] Updated weights for policy 1, policy_version 208430 (0.0010) [2023-12-27 07:05:50,907][20347] Updated weights for policy 0, policy_version 208811 (0.0008) [2023-12-27 07:05:50,965][20347] Updated weights for policy 0, policy_version 208821 (0.0010) [2023-12-27 07:05:51,028][20347] Updated weights for policy 0, policy_version 208831 (0.0010) [2023-12-27 07:05:51,278][20381] Updated weights for policy 1, policy_version 208440 (0.0010) [2023-12-27 07:05:51,340][20381] Updated weights for policy 1, policy_version 208450 (0.0008) [2023-12-27 07:05:51,402][20381] Updated weights for policy 1, policy_version 208460 (0.0008) [2023-12-27 07:05:51,729][20347] Updated weights for policy 0, policy_version 208841 (0.0007) [2023-12-27 07:05:51,786][20347] Updated weights for policy 0, policy_version 208851 (0.0008) [2023-12-27 07:05:51,832][20347] Updated weights for policy 0, policy_version 208861 (0.0008) [2023-12-27 07:05:51,881][20347] Updated weights for policy 0, policy_version 208871 (0.0007) [2023-12-27 07:05:52,077][20381] Updated weights for policy 1, policy_version 208470 (0.0010) [2023-12-27 07:05:52,130][20381] Updated weights for policy 1, policy_version 208480 (0.0011) [2023-12-27 07:05:52,180][20381] Updated weights for policy 1, policy_version 208490 (0.0010) [2023-12-27 07:05:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 106872832. Throughput: 0: 10492.4, 1: 10382.4. Samples: 106841028. Policy #0 lag: (min: 6.0, avg: 11.1, max: 38.0) [2023-12-27 07:05:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '972.230')] [2023-12-27 07:05:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000208496_53387264.pth... [2023-12-27 07:05:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000208872_53485568.pth... [2023-12-27 07:05:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000207656_53174272.pth [2023-12-27 07:05:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000207280_53075968.pth [2023-12-27 07:05:52,576][20347] Updated weights for policy 0, policy_version 208881 (0.0008) [2023-12-27 07:05:52,636][20347] Updated weights for policy 0, policy_version 208891 (0.0008) [2023-12-27 07:05:52,688][20347] Updated weights for policy 0, policy_version 208901 (0.0007) [2023-12-27 07:05:52,867][20381] Updated weights for policy 1, policy_version 208500 (0.0011) [2023-12-27 07:05:52,922][20381] Updated weights for policy 1, policy_version 208510 (0.0010) [2023-12-27 07:05:52,976][20381] Updated weights for policy 1, policy_version 208520 (0.0010) [2023-12-27 07:05:53,343][20347] Updated weights for policy 0, policy_version 208911 (0.0007) [2023-12-27 07:05:53,395][20347] Updated weights for policy 0, policy_version 208921 (0.0009) [2023-12-27 07:05:53,448][20347] Updated weights for policy 0, policy_version 208931 (0.0007) [2023-12-27 07:05:53,606][20381] Updated weights for policy 1, policy_version 208530 (0.0010) [2023-12-27 07:05:53,666][20381] Updated weights for policy 1, policy_version 208540 (0.0005) [2023-12-27 07:05:53,726][20381] Updated weights for policy 1, policy_version 208550 (0.0005) [2023-12-27 07:05:53,788][20381] Updated weights for policy 1, policy_version 208560 (0.0005) [2023-12-27 07:05:54,149][20347] Updated weights for policy 0, policy_version 208941 (0.0008) [2023-12-27 07:05:54,208][20347] Updated weights for policy 0, policy_version 208951 (0.0008) [2023-12-27 07:05:54,265][20347] Updated weights for policy 0, policy_version 208961 (0.0007) [2023-12-27 07:05:54,332][20381] Updated weights for policy 1, policy_version 208570 (0.0005) [2023-12-27 07:05:54,388][20381] Updated weights for policy 1, policy_version 208580 (0.0005) [2023-12-27 07:05:54,439][20381] Updated weights for policy 1, policy_version 208590 (0.0005) [2023-12-27 07:05:54,937][20347] Updated weights for policy 0, policy_version 208971 (0.0008) [2023-12-27 07:05:55,000][20347] Updated weights for policy 0, policy_version 208981 (0.0007) [2023-12-27 07:05:55,045][20381] Updated weights for policy 1, policy_version 208600 (0.0010) [2023-12-27 07:05:55,050][20347] Updated weights for policy 0, policy_version 208991 (0.0007) [2023-12-27 07:05:55,097][20381] Updated weights for policy 1, policy_version 208610 (0.0010) [2023-12-27 07:05:55,156][20381] Updated weights for policy 1, policy_version 208620 (0.0011) [2023-12-27 07:05:55,748][20347] Updated weights for policy 0, policy_version 209001 (0.0006) [2023-12-27 07:05:55,799][20347] Updated weights for policy 0, policy_version 209011 (0.0007) [2023-12-27 07:05:55,854][20347] Updated weights for policy 0, policy_version 209021 (0.0007) [2023-12-27 07:05:55,856][20381] Updated weights for policy 1, policy_version 208630 (0.0011) [2023-12-27 07:05:55,910][20347] Updated weights for policy 0, policy_version 209031 (0.0005) [2023-12-27 07:05:55,912][20381] Updated weights for policy 1, policy_version 208640 (0.0011) [2023-12-27 07:05:55,972][20381] Updated weights for policy 1, policy_version 208650 (0.0011) [2023-12-27 07:05:56,602][20381] Updated weights for policy 1, policy_version 208660 (0.0010) [2023-12-27 07:05:56,618][20347] Updated weights for policy 0, policy_version 209041 (0.0005) [2023-12-27 07:05:56,665][20347] Updated weights for policy 0, policy_version 209051 (0.0005) [2023-12-27 07:05:56,671][20381] Updated weights for policy 1, policy_version 208670 (0.0007) [2023-12-27 07:05:56,717][20347] Updated weights for policy 0, policy_version 209061 (0.0005) [2023-12-27 07:05:56,730][20381] Updated weights for policy 1, policy_version 208680 (0.0010) [2023-12-27 07:05:57,284][20381] Updated weights for policy 1, policy_version 208690 (0.0007) [2023-12-27 07:05:57,333][20381] Updated weights for policy 1, policy_version 208700 (0.0011) [2023-12-27 07:05:57,350][20347] Updated weights for policy 0, policy_version 209071 (0.0005) [2023-12-27 07:05:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 106971136. Throughput: 0: 10413.5, 1: 10389.8. Samples: 106964220. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:05:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '972.060')] [2023-12-27 07:05:57,391][20381] Updated weights for policy 1, policy_version 208710 (0.0010) [2023-12-27 07:05:57,397][20347] Updated weights for policy 0, policy_version 209081 (0.0005) [2023-12-27 07:05:57,442][20381] Updated weights for policy 1, policy_version 208720 (0.0010) [2023-12-27 07:05:57,454][20347] Updated weights for policy 0, policy_version 209091 (0.0005) [2023-12-27 07:05:58,091][20381] Updated weights for policy 1, policy_version 208730 (0.0010) [2023-12-27 07:05:58,137][20381] Updated weights for policy 1, policy_version 208740 (0.0010) [2023-12-27 07:05:58,155][20347] Updated weights for policy 0, policy_version 209101 (0.0007) [2023-12-27 07:05:58,201][20381] Updated weights for policy 1, policy_version 208750 (0.0008) [2023-12-27 07:05:58,220][20347] Updated weights for policy 0, policy_version 209111 (0.0008) [2023-12-27 07:05:58,272][20347] Updated weights for policy 0, policy_version 209121 (0.0008) [2023-12-27 07:05:58,952][20381] Updated weights for policy 1, policy_version 208760 (0.0008) [2023-12-27 07:05:58,996][20347] Updated weights for policy 0, policy_version 209131 (0.0008) [2023-12-27 07:05:59,004][20381] Updated weights for policy 1, policy_version 208770 (0.0007) [2023-12-27 07:05:59,046][20347] Updated weights for policy 0, policy_version 209141 (0.0007) [2023-12-27 07:05:59,061][20381] Updated weights for policy 1, policy_version 208780 (0.0010) [2023-12-27 07:05:59,104][20347] Updated weights for policy 0, policy_version 209151 (0.0009) [2023-12-27 07:05:59,703][20381] Updated weights for policy 1, policy_version 208790 (0.0009) [2023-12-27 07:05:59,761][20381] Updated weights for policy 1, policy_version 208800 (0.0011) [2023-12-27 07:05:59,807][20347] Updated weights for policy 0, policy_version 209161 (0.0007) [2023-12-27 07:05:59,818][20381] Updated weights for policy 1, policy_version 208810 (0.0010) [2023-12-27 07:05:59,869][20347] Updated weights for policy 0, policy_version 209171 (0.0007) [2023-12-27 07:05:59,925][20347] Updated weights for policy 0, policy_version 209181 (0.0008) [2023-12-27 07:05:59,986][20347] Updated weights for policy 0, policy_version 209191 (0.0007) [2023-12-27 07:06:00,482][20381] Updated weights for policy 1, policy_version 208820 (0.0009) [2023-12-27 07:06:00,538][20381] Updated weights for policy 1, policy_version 208830 (0.0008) [2023-12-27 07:06:00,584][20381] Updated weights for policy 1, policy_version 208840 (0.0008) [2023-12-27 07:06:00,690][20347] Updated weights for policy 0, policy_version 209201 (0.0005) [2023-12-27 07:06:00,746][20347] Updated weights for policy 0, policy_version 209211 (0.0005) [2023-12-27 07:06:00,797][20347] Updated weights for policy 0, policy_version 209221 (0.0005) [2023-12-27 07:06:01,301][20381] Updated weights for policy 1, policy_version 208850 (0.0008) [2023-12-27 07:06:01,355][20381] Updated weights for policy 1, policy_version 208860 (0.0007) [2023-12-27 07:06:01,380][20347] Updated weights for policy 0, policy_version 209231 (0.0007) [2023-12-27 07:06:01,411][20381] Updated weights for policy 1, policy_version 208870 (0.0006) [2023-12-27 07:06:01,434][20347] Updated weights for policy 0, policy_version 209241 (0.0007) [2023-12-27 07:06:01,463][20381] Updated weights for policy 1, policy_version 208880 (0.0005) [2023-12-27 07:06:01,487][20347] Updated weights for policy 0, policy_version 209251 (0.0007) [2023-12-27 07:06:02,083][20381] Updated weights for policy 1, policy_version 208890 (0.0007) [2023-12-27 07:06:02,131][20381] Updated weights for policy 1, policy_version 208900 (0.0008) [2023-12-27 07:06:02,186][20381] Updated weights for policy 1, policy_version 208910 (0.0008) [2023-12-27 07:06:02,189][20347] Updated weights for policy 0, policy_version 209261 (0.0007) [2023-12-27 07:06:02,240][20347] Updated weights for policy 0, policy_version 209271 (0.0007) [2023-12-27 07:06:02,301][20347] Updated weights for policy 0, policy_version 209281 (0.0006) [2023-12-27 07:06:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 107085824. Throughput: 0: 10379.2, 1: 10456.2. Samples: 107089412. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.430')] [2023-12-27 07:06:02,824][20381] Updated weights for policy 1, policy_version 208920 (0.0006) [2023-12-27 07:06:02,884][20381] Updated weights for policy 1, policy_version 208930 (0.0005) [2023-12-27 07:06:02,926][20347] Updated weights for policy 0, policy_version 209291 (0.0005) [2023-12-27 07:06:02,938][20381] Updated weights for policy 1, policy_version 208940 (0.0007) [2023-12-27 07:06:02,980][20347] Updated weights for policy 0, policy_version 209301 (0.0005) [2023-12-27 07:06:03,032][20347] Updated weights for policy 0, policy_version 209311 (0.0005) [2023-12-27 07:06:03,548][20381] Updated weights for policy 1, policy_version 208950 (0.0007) [2023-12-27 07:06:03,590][20347] Updated weights for policy 0, policy_version 209321 (0.0006) [2023-12-27 07:06:03,596][20381] Updated weights for policy 1, policy_version 208960 (0.0006) [2023-12-27 07:06:03,647][20347] Updated weights for policy 0, policy_version 209331 (0.0005) [2023-12-27 07:06:03,649][20381] Updated weights for policy 1, policy_version 208970 (0.0006) [2023-12-27 07:06:03,705][20347] Updated weights for policy 0, policy_version 209341 (0.0007) [2023-12-27 07:06:03,762][20347] Updated weights for policy 0, policy_version 209351 (0.0007) [2023-12-27 07:06:04,368][20381] Updated weights for policy 1, policy_version 208980 (0.0006) [2023-12-27 07:06:04,424][20381] Updated weights for policy 1, policy_version 208990 (0.0007) [2023-12-27 07:06:04,453][20347] Updated weights for policy 0, policy_version 209361 (0.0006) [2023-12-27 07:06:04,476][20381] Updated weights for policy 1, policy_version 209000 (0.0007) [2023-12-27 07:06:04,507][20347] Updated weights for policy 0, policy_version 209371 (0.0006) [2023-12-27 07:06:04,565][20347] Updated weights for policy 0, policy_version 209381 (0.0006) [2023-12-27 07:06:05,119][20381] Updated weights for policy 1, policy_version 209010 (0.0008) [2023-12-27 07:06:05,179][20381] Updated weights for policy 1, policy_version 209020 (0.0008) [2023-12-27 07:06:05,222][20347] Updated weights for policy 0, policy_version 209391 (0.0006) [2023-12-27 07:06:05,235][20381] Updated weights for policy 1, policy_version 209030 (0.0006) [2023-12-27 07:06:05,274][20347] Updated weights for policy 0, policy_version 209401 (0.0006) [2023-12-27 07:06:05,294][20381] Updated weights for policy 1, policy_version 209040 (0.0007) [2023-12-27 07:06:05,332][20347] Updated weights for policy 0, policy_version 209411 (0.0007) [2023-12-27 07:06:05,884][20381] Updated weights for policy 1, policy_version 209050 (0.0007) [2023-12-27 07:06:05,943][20381] Updated weights for policy 1, policy_version 209060 (0.0008) [2023-12-27 07:06:05,994][20381] Updated weights for policy 1, policy_version 209070 (0.0008) [2023-12-27 07:06:06,023][20347] Updated weights for policy 0, policy_version 209421 (0.0007) [2023-12-27 07:06:06,074][20347] Updated weights for policy 0, policy_version 209431 (0.0005) [2023-12-27 07:06:06,138][20347] Updated weights for policy 0, policy_version 209441 (0.0006) [2023-12-27 07:06:06,708][20381] Updated weights for policy 1, policy_version 209080 (0.0010) [2023-12-27 07:06:06,762][20381] Updated weights for policy 1, policy_version 209090 (0.0011) [2023-12-27 07:06:06,811][20381] Updated weights for policy 1, policy_version 209100 (0.0010) [2023-12-27 07:06:06,825][20347] Updated weights for policy 0, policy_version 209451 (0.0008) [2023-12-27 07:06:06,884][20347] Updated weights for policy 0, policy_version 209461 (0.0007) [2023-12-27 07:06:06,941][20347] Updated weights for policy 0, policy_version 209471 (0.0007) [2023-12-27 07:06:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 107184128. Throughput: 0: 10432.3, 1: 10461.7. Samples: 107153868. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000209104_53542912.pth... [2023-12-27 07:06:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000209480_53641216.pth... [2023-12-27 07:06:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000207888_53231616.pth [2023-12-27 07:06:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000208264_53329920.pth [2023-12-27 07:06:07,444][20381] Updated weights for policy 1, policy_version 209110 (0.0010) [2023-12-27 07:06:07,502][20381] Updated weights for policy 1, policy_version 209120 (0.0010) [2023-12-27 07:06:07,560][20381] Updated weights for policy 1, policy_version 209130 (0.0010) [2023-12-27 07:06:07,600][20347] Updated weights for policy 0, policy_version 209481 (0.0007) [2023-12-27 07:06:07,647][20347] Updated weights for policy 0, policy_version 209491 (0.0008) [2023-12-27 07:06:07,698][20347] Updated weights for policy 0, policy_version 209501 (0.0008) [2023-12-27 07:06:07,754][20347] Updated weights for policy 0, policy_version 209511 (0.0008) [2023-12-27 07:06:08,173][20381] Updated weights for policy 1, policy_version 209140 (0.0010) [2023-12-27 07:06:08,227][20381] Updated weights for policy 1, policy_version 209150 (0.0007) [2023-12-27 07:06:08,287][20381] Updated weights for policy 1, policy_version 209160 (0.0005) [2023-12-27 07:06:08,507][20347] Updated weights for policy 0, policy_version 209521 (0.0007) [2023-12-27 07:06:08,564][20347] Updated weights for policy 0, policy_version 209531 (0.0008) [2023-12-27 07:06:08,612][20347] Updated weights for policy 0, policy_version 209541 (0.0009) [2023-12-27 07:06:08,929][20381] Updated weights for policy 1, policy_version 209170 (0.0009) [2023-12-27 07:06:08,987][20381] Updated weights for policy 1, policy_version 209180 (0.0008) [2023-12-27 07:06:09,038][20381] Updated weights for policy 1, policy_version 209190 (0.0008) [2023-12-27 07:06:09,093][20381] Updated weights for policy 1, policy_version 209200 (0.0008) [2023-12-27 07:06:09,331][20347] Updated weights for policy 0, policy_version 209551 (0.0007) [2023-12-27 07:06:09,395][20347] Updated weights for policy 0, policy_version 209561 (0.0007) [2023-12-27 07:06:09,458][20347] Updated weights for policy 0, policy_version 209571 (0.0006) [2023-12-27 07:06:09,810][20381] Updated weights for policy 1, policy_version 209210 (0.0010) [2023-12-27 07:06:09,873][20381] Updated weights for policy 1, policy_version 209220 (0.0008) [2023-12-27 07:06:09,939][20381] Updated weights for policy 1, policy_version 209230 (0.0008) [2023-12-27 07:06:10,084][20347] Updated weights for policy 0, policy_version 209581 (0.0007) [2023-12-27 07:06:10,135][20347] Updated weights for policy 0, policy_version 209591 (0.0008) [2023-12-27 07:06:10,196][20347] Updated weights for policy 0, policy_version 209601 (0.0007) [2023-12-27 07:06:10,633][20381] Updated weights for policy 1, policy_version 209240 (0.0007) [2023-12-27 07:06:10,693][20381] Updated weights for policy 1, policy_version 209250 (0.0007) [2023-12-27 07:06:10,749][20381] Updated weights for policy 1, policy_version 209260 (0.0007) [2023-12-27 07:06:10,894][20347] Updated weights for policy 0, policy_version 209611 (0.0008) [2023-12-27 07:06:10,953][20347] Updated weights for policy 0, policy_version 209621 (0.0008) [2023-12-27 07:06:11,007][20347] Updated weights for policy 0, policy_version 209631 (0.0008) [2023-12-27 07:06:11,486][20381] Updated weights for policy 1, policy_version 209270 (0.0007) [2023-12-27 07:06:11,539][20381] Updated weights for policy 1, policy_version 209280 (0.0008) [2023-12-27 07:06:11,603][20381] Updated weights for policy 1, policy_version 209290 (0.0008) [2023-12-27 07:06:11,741][20347] Updated weights for policy 0, policy_version 209641 (0.0007) [2023-12-27 07:06:11,801][20347] Updated weights for policy 0, policy_version 209651 (0.0010) [2023-12-27 07:06:11,862][20347] Updated weights for policy 0, policy_version 209661 (0.0011) [2023-12-27 07:06:11,912][20347] Updated weights for policy 0, policy_version 209671 (0.0010) [2023-12-27 07:06:12,289][20381] Updated weights for policy 1, policy_version 209300 (0.0008) [2023-12-27 07:06:12,354][20381] Updated weights for policy 1, policy_version 209310 (0.0007) [2023-12-27 07:06:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 107282432. Throughput: 0: 10378.7, 1: 10511.1. Samples: 107276664. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:12,404][20381] Updated weights for policy 1, policy_version 209320 (0.0006) [2023-12-27 07:06:12,597][20347] Updated weights for policy 0, policy_version 209681 (0.0011) [2023-12-27 07:06:12,665][20347] Updated weights for policy 0, policy_version 209691 (0.0010) [2023-12-27 07:06:12,722][20347] Updated weights for policy 0, policy_version 209701 (0.0010) [2023-12-27 07:06:13,055][20381] Updated weights for policy 1, policy_version 209331 (0.0007) [2023-12-27 07:06:13,106][20381] Updated weights for policy 1, policy_version 209341 (0.0005) [2023-12-27 07:06:13,154][20381] Updated weights for policy 1, policy_version 209351 (0.0005) [2023-12-27 07:06:13,268][20347] Updated weights for policy 0, policy_version 209711 (0.0010) [2023-12-27 07:06:13,323][20347] Updated weights for policy 0, policy_version 209721 (0.0010) [2023-12-27 07:06:13,380][20347] Updated weights for policy 0, policy_version 209731 (0.0010) [2023-12-27 07:06:13,806][20381] Updated weights for policy 1, policy_version 209361 (0.0005) [2023-12-27 07:06:13,858][20381] Updated weights for policy 1, policy_version 209371 (0.0005) [2023-12-27 07:06:13,912][20381] Updated weights for policy 1, policy_version 209381 (0.0006) [2023-12-27 07:06:13,967][20381] Updated weights for policy 1, policy_version 209391 (0.0008) [2023-12-27 07:06:13,996][20347] Updated weights for policy 0, policy_version 209741 (0.0007) [2023-12-27 07:06:14,050][20347] Updated weights for policy 0, policy_version 209751 (0.0005) [2023-12-27 07:06:14,099][20347] Updated weights for policy 0, policy_version 209761 (0.0005) [2023-12-27 07:06:14,646][20381] Updated weights for policy 1, policy_version 209401 (0.0010) [2023-12-27 07:06:14,700][20381] Updated weights for policy 1, policy_version 209411 (0.0010) [2023-12-27 07:06:14,704][20347] Updated weights for policy 0, policy_version 209771 (0.0006) [2023-12-27 07:06:14,755][20381] Updated weights for policy 1, policy_version 209421 (0.0009) [2023-12-27 07:06:14,778][20347] Updated weights for policy 0, policy_version 209781 (0.0009) [2023-12-27 07:06:14,835][20347] Updated weights for policy 0, policy_version 209791 (0.0007) [2023-12-27 07:06:15,363][20381] Updated weights for policy 1, policy_version 209431 (0.0006) [2023-12-27 07:06:15,427][20381] Updated weights for policy 1, policy_version 209441 (0.0006) [2023-12-27 07:06:15,484][20381] Updated weights for policy 1, policy_version 209451 (0.0008) [2023-12-27 07:06:15,569][20347] Updated weights for policy 0, policy_version 209801 (0.0007) [2023-12-27 07:06:15,635][20347] Updated weights for policy 0, policy_version 209811 (0.0008) [2023-12-27 07:06:15,692][20347] Updated weights for policy 0, policy_version 209821 (0.0007) [2023-12-27 07:06:15,751][20347] Updated weights for policy 0, policy_version 209831 (0.0007) [2023-12-27 07:06:16,126][20381] Updated weights for policy 1, policy_version 209461 (0.0009) [2023-12-27 07:06:16,172][20381] Updated weights for policy 1, policy_version 209471 (0.0008) [2023-12-27 07:06:16,228][20381] Updated weights for policy 1, policy_version 209481 (0.0008) [2023-12-27 07:06:16,458][20347] Updated weights for policy 0, policy_version 209841 (0.0008) [2023-12-27 07:06:16,514][20347] Updated weights for policy 0, policy_version 209851 (0.0008) [2023-12-27 07:06:16,568][20347] Updated weights for policy 0, policy_version 209861 (0.0007) [2023-12-27 07:06:16,919][20381] Updated weights for policy 1, policy_version 209491 (0.0007) [2023-12-27 07:06:16,987][20381] Updated weights for policy 1, policy_version 209501 (0.0005) [2023-12-27 07:06:17,045][20381] Updated weights for policy 1, policy_version 209511 (0.0005) [2023-12-27 07:06:17,233][20347] Updated weights for policy 0, policy_version 209871 (0.0006) [2023-12-27 07:06:17,287][20347] Updated weights for policy 0, policy_version 209881 (0.0007) [2023-12-27 07:06:17,334][20347] Updated weights for policy 0, policy_version 209891 (0.0007) [2023-12-27 07:06:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 107397120. Throughput: 0: 10334.0, 1: 10487.3. Samples: 107402128. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:17,683][20381] Updated weights for policy 1, policy_version 209521 (0.0010) [2023-12-27 07:06:17,744][20381] Updated weights for policy 1, policy_version 209531 (0.0011) [2023-12-27 07:06:17,805][20381] Updated weights for policy 1, policy_version 209541 (0.0010) [2023-12-27 07:06:17,870][20381] Updated weights for policy 1, policy_version 209551 (0.0006) [2023-12-27 07:06:17,900][20347] Updated weights for policy 0, policy_version 209901 (0.0006) [2023-12-27 07:06:17,951][20347] Updated weights for policy 0, policy_version 209911 (0.0007) [2023-12-27 07:06:18,002][20347] Updated weights for policy 0, policy_version 209921 (0.0007) [2023-12-27 07:06:18,528][20381] Updated weights for policy 1, policy_version 209561 (0.0010) [2023-12-27 07:06:18,590][20381] Updated weights for policy 1, policy_version 209571 (0.0010) [2023-12-27 07:06:18,648][20381] Updated weights for policy 1, policy_version 209581 (0.0010) [2023-12-27 07:06:18,704][20347] Updated weights for policy 0, policy_version 209931 (0.0006) [2023-12-27 07:06:18,761][20347] Updated weights for policy 0, policy_version 209941 (0.0006) [2023-12-27 07:06:18,820][20347] Updated weights for policy 0, policy_version 209951 (0.0006) [2023-12-27 07:06:19,314][20381] Updated weights for policy 1, policy_version 209591 (0.0008) [2023-12-27 07:06:19,379][20381] Updated weights for policy 1, policy_version 209601 (0.0008) [2023-12-27 07:06:19,433][20381] Updated weights for policy 1, policy_version 209611 (0.0005) [2023-12-27 07:06:19,491][20347] Updated weights for policy 0, policy_version 209961 (0.0007) [2023-12-27 07:06:19,554][20347] Updated weights for policy 0, policy_version 209971 (0.0011) [2023-12-27 07:06:19,609][20347] Updated weights for policy 0, policy_version 209981 (0.0010) [2023-12-27 07:06:19,673][20347] Updated weights for policy 0, policy_version 209991 (0.0010) [2023-12-27 07:06:20,131][20381] Updated weights for policy 1, policy_version 209621 (0.0008) [2023-12-27 07:06:20,191][20381] Updated weights for policy 1, policy_version 209631 (0.0010) [2023-12-27 07:06:20,247][20381] Updated weights for policy 1, policy_version 209641 (0.0010) [2023-12-27 07:06:20,366][20347] Updated weights for policy 0, policy_version 210001 (0.0011) [2023-12-27 07:06:20,423][20347] Updated weights for policy 0, policy_version 210011 (0.0010) [2023-12-27 07:06:20,485][20347] Updated weights for policy 0, policy_version 210021 (0.0010) [2023-12-27 07:06:20,977][20381] Updated weights for policy 1, policy_version 209651 (0.0010) [2023-12-27 07:06:21,034][20381] Updated weights for policy 1, policy_version 209661 (0.0008) [2023-12-27 07:06:21,094][20381] Updated weights for policy 1, policy_version 209671 (0.0008) [2023-12-27 07:06:21,228][20347] Updated weights for policy 0, policy_version 210031 (0.0010) [2023-12-27 07:06:21,288][20347] Updated weights for policy 0, policy_version 210041 (0.0009) [2023-12-27 07:06:21,348][20347] Updated weights for policy 0, policy_version 210051 (0.0007) [2023-12-27 07:06:21,897][20381] Updated weights for policy 1, policy_version 209681 (0.0008) [2023-12-27 07:06:21,946][20381] Updated weights for policy 1, policy_version 209691 (0.0011) [2023-12-27 07:06:22,006][20381] Updated weights for policy 1, policy_version 209701 (0.0011) [2023-12-27 07:06:22,059][20347] Updated weights for policy 0, policy_version 210061 (0.0010) [2023-12-27 07:06:22,063][20381] Updated weights for policy 1, policy_version 209711 (0.0011) [2023-12-27 07:06:22,113][20347] Updated weights for policy 0, policy_version 210071 (0.0008) [2023-12-27 07:06:22,158][20347] Updated weights for policy 0, policy_version 210081 (0.0008) [2023-12-27 07:06:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 107495424. Throughput: 0: 10371.5, 1: 10390.9. Samples: 107464172. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.380')] [2023-12-27 07:06:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000209712_53698560.pth... [2023-12-27 07:06:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000210088_53796864.pth... [2023-12-27 07:06:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000208872_53485568.pth [2023-12-27 07:06:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000208496_53387264.pth [2023-12-27 07:06:22,776][20381] Updated weights for policy 1, policy_version 209721 (0.0011) [2023-12-27 07:06:22,812][20347] Updated weights for policy 0, policy_version 210091 (0.0007) [2023-12-27 07:06:22,836][20381] Updated weights for policy 1, policy_version 209731 (0.0011) [2023-12-27 07:06:22,863][20347] Updated weights for policy 0, policy_version 210101 (0.0008) [2023-12-27 07:06:22,893][20381] Updated weights for policy 1, policy_version 209741 (0.0008) [2023-12-27 07:06:22,925][20347] Updated weights for policy 0, policy_version 210111 (0.0007) [2023-12-27 07:06:23,538][20292] KL-divergence is very high: 147.1426 [2023-12-27 07:06:23,552][20292] KL-divergence is very high: 112.3277 [2023-12-27 07:06:23,568][20347] Updated weights for policy 0, policy_version 210121 (0.0006) [2023-12-27 07:06:23,575][20381] Updated weights for policy 1, policy_version 209751 (0.0009) [2023-12-27 07:06:23,628][20347] Updated weights for policy 0, policy_version 210131 (0.0007) [2023-12-27 07:06:23,633][20381] Updated weights for policy 1, policy_version 209761 (0.0010) [2023-12-27 07:06:23,645][20292] KL-divergence is very high: 120.1825 [2023-12-27 07:06:23,683][20347] Updated weights for policy 0, policy_version 210141 (0.0005) [2023-12-27 07:06:23,691][20292] KL-divergence is very high: 108.4036 [2023-12-27 07:06:23,691][20381] Updated weights for policy 1, policy_version 209771 (0.0010) [2023-12-27 07:06:23,742][20347] Updated weights for policy 0, policy_version 210151 (0.0009) [2023-12-27 07:06:24,307][20347] Updated weights for policy 0, policy_version 210161 (0.0006) [2023-12-27 07:06:24,309][20381] Updated weights for policy 1, policy_version 209781 (0.0010) [2023-12-27 07:06:24,357][20347] Updated weights for policy 0, policy_version 210171 (0.0005) [2023-12-27 07:06:24,359][20381] Updated weights for policy 1, policy_version 209791 (0.0010) [2023-12-27 07:06:24,407][20347] Updated weights for policy 0, policy_version 210181 (0.0005) [2023-12-27 07:06:24,418][20381] Updated weights for policy 1, policy_version 209801 (0.0011) [2023-12-27 07:06:25,000][20347] Updated weights for policy 0, policy_version 210191 (0.0005) [2023-12-27 07:06:25,051][20347] Updated weights for policy 0, policy_version 210201 (0.0005) [2023-12-27 07:06:25,101][20381] Updated weights for policy 1, policy_version 209811 (0.0010) [2023-12-27 07:06:25,104][20347] Updated weights for policy 0, policy_version 210211 (0.0005) [2023-12-27 07:06:25,161][20381] Updated weights for policy 1, policy_version 209821 (0.0010) [2023-12-27 07:06:25,218][20381] Updated weights for policy 1, policy_version 209831 (0.0010) [2023-12-27 07:06:25,709][20347] Updated weights for policy 0, policy_version 210221 (0.0005) [2023-12-27 07:06:25,775][20347] Updated weights for policy 0, policy_version 210231 (0.0005) [2023-12-27 07:06:25,827][20347] Updated weights for policy 0, policy_version 210241 (0.0005) [2023-12-27 07:06:25,885][20381] Updated weights for policy 1, policy_version 209841 (0.0010) [2023-12-27 07:06:25,945][20381] Updated weights for policy 1, policy_version 209851 (0.0007) [2023-12-27 07:06:25,990][20381] Updated weights for policy 1, policy_version 209861 (0.0005) [2023-12-27 07:06:26,036][20381] Updated weights for policy 1, policy_version 209871 (0.0005) [2023-12-27 07:06:26,397][20347] Updated weights for policy 0, policy_version 210251 (0.0006) [2023-12-27 07:06:26,450][20347] Updated weights for policy 0, policy_version 210261 (0.0007) [2023-12-27 07:06:26,508][20347] Updated weights for policy 0, policy_version 210271 (0.0007) [2023-12-27 07:06:26,682][20381] Updated weights for policy 1, policy_version 209881 (0.0006) [2023-12-27 07:06:26,739][20381] Updated weights for policy 1, policy_version 209891 (0.0008) [2023-12-27 07:06:26,789][20381] Updated weights for policy 1, policy_version 209901 (0.0006) [2023-12-27 07:06:27,103][20347] Updated weights for policy 0, policy_version 210281 (0.0006) [2023-12-27 07:06:27,158][20347] Updated weights for policy 0, policy_version 210291 (0.0007) [2023-12-27 07:06:27,213][20347] Updated weights for policy 0, policy_version 210301 (0.0007) [2023-12-27 07:06:27,269][20347] Updated weights for policy 0, policy_version 210311 (0.0008) [2023-12-27 07:06:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 107601920. Throughput: 0: 10466.0, 1: 10319.6. Samples: 107588856. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.890')] [2023-12-27 07:06:27,383][20381] Updated weights for policy 1, policy_version 209911 (0.0007) [2023-12-27 07:06:27,447][20381] Updated weights for policy 1, policy_version 209921 (0.0006) [2023-12-27 07:06:27,508][20381] Updated weights for policy 1, policy_version 209931 (0.0005) [2023-12-27 07:06:27,976][20347] Updated weights for policy 0, policy_version 210321 (0.0005) [2023-12-27 07:06:28,032][20347] Updated weights for policy 0, policy_version 210331 (0.0006) [2023-12-27 07:06:28,046][20381] Updated weights for policy 1, policy_version 209941 (0.0006) [2023-12-27 07:06:28,082][20347] Updated weights for policy 0, policy_version 210341 (0.0005) [2023-12-27 07:06:28,092][20381] Updated weights for policy 1, policy_version 209951 (0.0005) [2023-12-27 07:06:28,152][20381] Updated weights for policy 1, policy_version 209961 (0.0005) [2023-12-27 07:06:28,698][20347] Updated weights for policy 0, policy_version 210351 (0.0007) [2023-12-27 07:06:28,750][20347] Updated weights for policy 0, policy_version 210361 (0.0007) [2023-12-27 07:06:28,753][20381] Updated weights for policy 1, policy_version 209971 (0.0007) [2023-12-27 07:06:28,805][20347] Updated weights for policy 0, policy_version 210371 (0.0005) [2023-12-27 07:06:28,815][20381] Updated weights for policy 1, policy_version 209981 (0.0010) [2023-12-27 07:06:28,874][20381] Updated weights for policy 1, policy_version 209991 (0.0011) [2023-12-27 07:06:29,408][20347] Updated weights for policy 0, policy_version 210381 (0.0006) [2023-12-27 07:06:29,464][20347] Updated weights for policy 0, policy_version 210391 (0.0007) [2023-12-27 07:06:29,516][20347] Updated weights for policy 0, policy_version 210401 (0.0008) [2023-12-27 07:06:29,583][20381] Updated weights for policy 1, policy_version 210001 (0.0006) [2023-12-27 07:06:29,643][20381] Updated weights for policy 1, policy_version 210011 (0.0008) [2023-12-27 07:06:29,703][20381] Updated weights for policy 1, policy_version 210021 (0.0008) [2023-12-27 07:06:29,761][20381] Updated weights for policy 1, policy_version 210031 (0.0008) [2023-12-27 07:06:30,227][20347] Updated weights for policy 0, policy_version 210411 (0.0008) [2023-12-27 07:06:30,275][20347] Updated weights for policy 0, policy_version 210421 (0.0010) [2023-12-27 07:06:30,323][20347] Updated weights for policy 0, policy_version 210431 (0.0010) [2023-12-27 07:06:30,421][20381] Updated weights for policy 1, policy_version 210041 (0.0008) [2023-12-27 07:06:30,480][20381] Updated weights for policy 1, policy_version 210052 (0.0008) [2023-12-27 07:06:30,535][20381] Updated weights for policy 1, policy_version 210062 (0.0006) [2023-12-27 07:06:30,953][20347] Updated weights for policy 0, policy_version 210441 (0.0010) [2023-12-27 07:06:31,001][20347] Updated weights for policy 0, policy_version 210451 (0.0010) [2023-12-27 07:06:31,063][20347] Updated weights for policy 0, policy_version 210461 (0.0009) [2023-12-27 07:06:31,117][20347] Updated weights for policy 0, policy_version 210471 (0.0006) [2023-12-27 07:06:31,156][20381] Updated weights for policy 1, policy_version 210072 (0.0008) [2023-12-27 07:06:31,216][20381] Updated weights for policy 1, policy_version 210082 (0.0008) [2023-12-27 07:06:31,276][20381] Updated weights for policy 1, policy_version 210092 (0.0008) [2023-12-27 07:06:31,812][20347] Updated weights for policy 0, policy_version 210481 (0.0009) [2023-12-27 07:06:31,878][20347] Updated weights for policy 0, policy_version 210491 (0.0009) [2023-12-27 07:06:31,939][20347] Updated weights for policy 0, policy_version 210501 (0.0009) [2023-12-27 07:06:31,961][20381] Updated weights for policy 1, policy_version 210102 (0.0009) [2023-12-27 07:06:32,020][20381] Updated weights for policy 1, policy_version 210112 (0.0008) [2023-12-27 07:06:32,077][20381] Updated weights for policy 1, policy_version 210122 (0.0007) [2023-12-27 07:06:32,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 107708416. Throughput: 0: 10447.1, 1: 10431.6. Samples: 107717052. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:32,604][20347] Updated weights for policy 0, policy_version 210511 (0.0007) [2023-12-27 07:06:32,651][20347] Updated weights for policy 0, policy_version 210521 (0.0006) [2023-12-27 07:06:32,700][20347] Updated weights for policy 0, policy_version 210531 (0.0008) [2023-12-27 07:06:32,758][20381] Updated weights for policy 1, policy_version 210132 (0.0007) [2023-12-27 07:06:32,817][20381] Updated weights for policy 1, policy_version 210142 (0.0008) [2023-12-27 07:06:32,876][20381] Updated weights for policy 1, policy_version 210152 (0.0007) [2023-12-27 07:06:33,362][20347] Updated weights for policy 0, policy_version 210541 (0.0007) [2023-12-27 07:06:33,415][20347] Updated weights for policy 0, policy_version 210551 (0.0008) [2023-12-27 07:06:33,469][20347] Updated weights for policy 0, policy_version 210561 (0.0006) [2023-12-27 07:06:33,558][20381] Updated weights for policy 1, policy_version 210162 (0.0007) [2023-12-27 07:06:33,619][20381] Updated weights for policy 1, policy_version 210172 (0.0007) [2023-12-27 07:06:33,670][20381] Updated weights for policy 1, policy_version 210182 (0.0005) [2023-12-27 07:06:33,717][20381] Updated weights for policy 1, policy_version 210192 (0.0005) [2023-12-27 07:06:34,112][20347] Updated weights for policy 0, policy_version 210571 (0.0005) [2023-12-27 07:06:34,171][20347] Updated weights for policy 0, policy_version 210581 (0.0006) [2023-12-27 07:06:34,224][20347] Updated weights for policy 0, policy_version 210591 (0.0007) [2023-12-27 07:06:34,412][20381] Updated weights for policy 1, policy_version 210202 (0.0006) [2023-12-27 07:06:34,458][20381] Updated weights for policy 1, policy_version 210212 (0.0010) [2023-12-27 07:06:34,505][20381] Updated weights for policy 1, policy_version 210222 (0.0010) [2023-12-27 07:06:34,911][20347] Updated weights for policy 0, policy_version 210601 (0.0008) [2023-12-27 07:06:34,962][20347] Updated weights for policy 0, policy_version 210611 (0.0007) [2023-12-27 07:06:35,017][20347] Updated weights for policy 0, policy_version 210621 (0.0009) [2023-12-27 07:06:35,072][20347] Updated weights for policy 0, policy_version 210631 (0.0007) [2023-12-27 07:06:35,082][20381] Updated weights for policy 1, policy_version 210232 (0.0006) [2023-12-27 07:06:35,127][20381] Updated weights for policy 1, policy_version 210242 (0.0005) [2023-12-27 07:06:35,174][20381] Updated weights for policy 1, policy_version 210252 (0.0008) [2023-12-27 07:06:35,778][20381] Updated weights for policy 1, policy_version 210262 (0.0007) [2023-12-27 07:06:35,794][20347] Updated weights for policy 0, policy_version 210641 (0.0008) [2023-12-27 07:06:35,825][20381] Updated weights for policy 1, policy_version 210272 (0.0005) [2023-12-27 07:06:35,845][20347] Updated weights for policy 0, policy_version 210651 (0.0007) [2023-12-27 07:06:35,868][20381] Updated weights for policy 1, policy_version 210282 (0.0005) [2023-12-27 07:06:35,899][20347] Updated weights for policy 0, policy_version 210661 (0.0007) [2023-12-27 07:06:36,497][20381] Updated weights for policy 1, policy_version 210292 (0.0008) [2023-12-27 07:06:36,553][20381] Updated weights for policy 1, policy_version 210302 (0.0005) [2023-12-27 07:06:36,598][20381] Updated weights for policy 1, policy_version 210312 (0.0007) [2023-12-27 07:06:36,624][20347] Updated weights for policy 0, policy_version 210671 (0.0007) [2023-12-27 07:06:36,685][20347] Updated weights for policy 0, policy_version 210681 (0.0008) [2023-12-27 07:06:36,740][20347] Updated weights for policy 0, policy_version 210691 (0.0007) [2023-12-27 07:06:37,288][20381] Updated weights for policy 1, policy_version 210322 (0.0007) [2023-12-27 07:06:37,341][20381] Updated weights for policy 1, policy_version 210332 (0.0008) [2023-12-27 07:06:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 107806720. Throughput: 0: 10401.1, 1: 10472.2. Samples: 107780332. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:37,384][20347] Updated weights for policy 0, policy_version 210701 (0.0007) [2023-12-27 07:06:37,394][20381] Updated weights for policy 1, policy_version 210342 (0.0006) [2023-12-27 07:06:37,430][20347] Updated weights for policy 0, policy_version 210711 (0.0006) [2023-12-27 07:06:37,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000210352_53862400.pth... [2023-12-27 07:06:37,443][20381] Updated weights for policy 1, policy_version 210352 (0.0005) [2023-12-27 07:06:37,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000209104_53542912.pth [2023-12-27 07:06:37,488][20347] Updated weights for policy 0, policy_version 210721 (0.0007) [2023-12-27 07:06:37,521][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000210728_53960704.pth... [2023-12-27 07:06:37,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000209480_53641216.pth [2023-12-27 07:06:38,155][20381] Updated weights for policy 1, policy_version 210362 (0.0006) [2023-12-27 07:06:38,201][20347] Updated weights for policy 0, policy_version 210731 (0.0007) [2023-12-27 07:06:38,222][20381] Updated weights for policy 1, policy_version 210372 (0.0006) [2023-12-27 07:06:38,264][20347] Updated weights for policy 0, policy_version 210741 (0.0006) [2023-12-27 07:06:38,278][20381] Updated weights for policy 1, policy_version 210382 (0.0006) [2023-12-27 07:06:38,328][20347] Updated weights for policy 0, policy_version 210751 (0.0007) [2023-12-27 07:06:38,874][20347] Updated weights for policy 0, policy_version 210761 (0.0006) [2023-12-27 07:06:38,927][20347] Updated weights for policy 0, policy_version 210771 (0.0005) [2023-12-27 07:06:38,976][20381] Updated weights for policy 1, policy_version 210392 (0.0008) [2023-12-27 07:06:38,978][20347] Updated weights for policy 0, policy_version 210781 (0.0006) [2023-12-27 07:06:39,033][20381] Updated weights for policy 1, policy_version 210402 (0.0005) [2023-12-27 07:06:39,033][20347] Updated weights for policy 0, policy_version 210791 (0.0010) [2023-12-27 07:06:39,090][20381] Updated weights for policy 1, policy_version 210412 (0.0006) [2023-12-27 07:06:39,706][20347] Updated weights for policy 0, policy_version 210801 (0.0008) [2023-12-27 07:06:39,762][20347] Updated weights for policy 0, policy_version 210811 (0.0007) [2023-12-27 07:06:39,771][20381] Updated weights for policy 1, policy_version 210422 (0.0007) [2023-12-27 07:06:39,816][20347] Updated weights for policy 0, policy_version 210821 (0.0006) [2023-12-27 07:06:39,827][20381] Updated weights for policy 1, policy_version 210432 (0.0006) [2023-12-27 07:06:39,890][20381] Updated weights for policy 1, policy_version 210442 (0.0007) [2023-12-27 07:06:40,547][20347] Updated weights for policy 0, policy_version 210831 (0.0007) [2023-12-27 07:06:40,570][20381] Updated weights for policy 1, policy_version 210452 (0.0008) [2023-12-27 07:06:40,596][20347] Updated weights for policy 0, policy_version 210841 (0.0005) [2023-12-27 07:06:40,623][20381] Updated weights for policy 1, policy_version 210462 (0.0007) [2023-12-27 07:06:40,645][20347] Updated weights for policy 0, policy_version 210851 (0.0005) [2023-12-27 07:06:40,673][20381] Updated weights for policy 1, policy_version 210472 (0.0006) [2023-12-27 07:06:41,379][20381] Updated weights for policy 1, policy_version 210482 (0.0005) [2023-12-27 07:06:41,439][20381] Updated weights for policy 1, policy_version 210492 (0.0007) [2023-12-27 07:06:41,458][20347] Updated weights for policy 0, policy_version 210861 (0.0008) [2023-12-27 07:06:41,495][20381] Updated weights for policy 1, policy_version 210502 (0.0006) [2023-12-27 07:06:41,524][20347] Updated weights for policy 0, policy_version 210871 (0.0008) [2023-12-27 07:06:41,550][20381] Updated weights for policy 1, policy_version 210512 (0.0006) [2023-12-27 07:06:41,580][20347] Updated weights for policy 0, policy_version 210881 (0.0007) [2023-12-27 07:06:42,170][20381] Updated weights for policy 1, policy_version 210522 (0.0006) [2023-12-27 07:06:42,233][20381] Updated weights for policy 1, policy_version 210532 (0.0007) [2023-12-27 07:06:42,301][20381] Updated weights for policy 1, policy_version 210542 (0.0007) [2023-12-27 07:06:42,335][20347] Updated weights for policy 0, policy_version 210891 (0.0009) [2023-12-27 07:06:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 107913216. Throughput: 0: 10416.1, 1: 10460.9. Samples: 107903684. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:42,405][20347] Updated weights for policy 0, policy_version 210901 (0.0010) [2023-12-27 07:06:42,462][20347] Updated weights for policy 0, policy_version 210911 (0.0010) [2023-12-27 07:06:42,989][20381] Updated weights for policy 1, policy_version 210552 (0.0008) [2023-12-27 07:06:43,045][20381] Updated weights for policy 1, policy_version 210562 (0.0008) [2023-12-27 07:06:43,101][20381] Updated weights for policy 1, policy_version 210572 (0.0008) [2023-12-27 07:06:43,163][20347] Updated weights for policy 0, policy_version 210921 (0.0010) [2023-12-27 07:06:43,222][20347] Updated weights for policy 0, policy_version 210931 (0.0007) [2023-12-27 07:06:43,285][20347] Updated weights for policy 0, policy_version 210941 (0.0007) [2023-12-27 07:06:43,341][20347] Updated weights for policy 0, policy_version 210951 (0.0007) [2023-12-27 07:06:43,801][20381] Updated weights for policy 1, policy_version 210582 (0.0010) [2023-12-27 07:06:43,859][20381] Updated weights for policy 1, policy_version 210592 (0.0010) [2023-12-27 07:06:43,912][20381] Updated weights for policy 1, policy_version 210602 (0.0011) [2023-12-27 07:06:44,010][20347] Updated weights for policy 0, policy_version 210961 (0.0007) [2023-12-27 07:06:44,058][20347] Updated weights for policy 0, policy_version 210971 (0.0007) [2023-12-27 07:06:44,115][20347] Updated weights for policy 0, policy_version 210981 (0.0007) [2023-12-27 07:06:44,551][20381] Updated weights for policy 1, policy_version 210612 (0.0010) [2023-12-27 07:06:44,603][20381] Updated weights for policy 1, policy_version 210622 (0.0007) [2023-12-27 07:06:44,666][20381] Updated weights for policy 1, policy_version 210632 (0.0009) [2023-12-27 07:06:44,754][20347] Updated weights for policy 0, policy_version 210991 (0.0006) [2023-12-27 07:06:44,815][20347] Updated weights for policy 0, policy_version 211001 (0.0007) [2023-12-27 07:06:44,879][20347] Updated weights for policy 0, policy_version 211011 (0.0007) [2023-12-27 07:06:45,278][20381] Updated weights for policy 1, policy_version 210642 (0.0009) [2023-12-27 07:06:45,339][20381] Updated weights for policy 1, policy_version 210652 (0.0010) [2023-12-27 07:06:45,400][20381] Updated weights for policy 1, policy_version 210662 (0.0010) [2023-12-27 07:06:45,461][20381] Updated weights for policy 1, policy_version 210672 (0.0010) [2023-12-27 07:06:45,574][20347] Updated weights for policy 0, policy_version 211021 (0.0008) [2023-12-27 07:06:45,631][20347] Updated weights for policy 0, policy_version 211031 (0.0007) [2023-12-27 07:06:45,688][20347] Updated weights for policy 0, policy_version 211041 (0.0007) [2023-12-27 07:06:46,150][20381] Updated weights for policy 1, policy_version 210682 (0.0010) [2023-12-27 07:06:46,215][20381] Updated weights for policy 1, policy_version 210692 (0.0010) [2023-12-27 07:06:46,276][20381] Updated weights for policy 1, policy_version 210702 (0.0010) [2023-12-27 07:06:46,391][20347] Updated weights for policy 0, policy_version 211051 (0.0007) [2023-12-27 07:06:46,441][20347] Updated weights for policy 0, policy_version 211061 (0.0007) [2023-12-27 07:06:46,488][20347] Updated weights for policy 0, policy_version 211071 (0.0009) [2023-12-27 07:06:46,842][20381] Updated weights for policy 1, policy_version 210712 (0.0010) [2023-12-27 07:06:46,899][20381] Updated weights for policy 1, policy_version 210722 (0.0010) [2023-12-27 07:06:46,957][20381] Updated weights for policy 1, policy_version 210732 (0.0011) [2023-12-27 07:06:47,140][20347] Updated weights for policy 0, policy_version 211081 (0.0008) [2023-12-27 07:06:47,196][20347] Updated weights for policy 0, policy_version 211091 (0.0008) [2023-12-27 07:06:47,246][20347] Updated weights for policy 0, policy_version 211101 (0.0007) [2023-12-27 07:06:47,296][20347] Updated weights for policy 0, policy_version 211111 (0.0006) [2023-12-27 07:06:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 108019712. Throughput: 0: 10386.3, 1: 10451.6. Samples: 108027116. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:47,635][20381] Updated weights for policy 1, policy_version 210742 (0.0010) [2023-12-27 07:06:47,685][20381] Updated weights for policy 1, policy_version 210752 (0.0011) [2023-12-27 07:06:47,730][20381] Updated weights for policy 1, policy_version 210762 (0.0010) [2023-12-27 07:06:47,948][20347] Updated weights for policy 0, policy_version 211121 (0.0008) [2023-12-27 07:06:48,008][20347] Updated weights for policy 0, policy_version 211131 (0.0005) [2023-12-27 07:06:48,064][20347] Updated weights for policy 0, policy_version 211141 (0.0005) [2023-12-27 07:06:48,374][20381] Updated weights for policy 1, policy_version 210772 (0.0006) [2023-12-27 07:06:48,437][20381] Updated weights for policy 1, policy_version 210782 (0.0008) [2023-12-27 07:06:48,500][20381] Updated weights for policy 1, policy_version 210792 (0.0006) [2023-12-27 07:06:48,667][20347] Updated weights for policy 0, policy_version 211151 (0.0007) [2023-12-27 07:06:48,733][20347] Updated weights for policy 0, policy_version 211161 (0.0006) [2023-12-27 07:06:48,786][20347] Updated weights for policy 0, policy_version 211171 (0.0007) [2023-12-27 07:06:49,098][20381] Updated weights for policy 1, policy_version 210802 (0.0006) [2023-12-27 07:06:49,158][20381] Updated weights for policy 1, policy_version 210812 (0.0008) [2023-12-27 07:06:49,221][20381] Updated weights for policy 1, policy_version 210822 (0.0008) [2023-12-27 07:06:49,284][20381] Updated weights for policy 1, policy_version 210832 (0.0008) [2023-12-27 07:06:49,496][20347] Updated weights for policy 0, policy_version 211181 (0.0008) [2023-12-27 07:06:49,556][20347] Updated weights for policy 0, policy_version 211191 (0.0008) [2023-12-27 07:06:49,612][20347] Updated weights for policy 0, policy_version 211201 (0.0008) [2023-12-27 07:06:49,902][20381] Updated weights for policy 1, policy_version 210842 (0.0007) [2023-12-27 07:06:49,964][20381] Updated weights for policy 1, policy_version 210852 (0.0008) [2023-12-27 07:06:50,020][20381] Updated weights for policy 1, policy_version 210862 (0.0008) [2023-12-27 07:06:50,272][20347] Updated weights for policy 0, policy_version 211211 (0.0008) [2023-12-27 07:06:50,322][20347] Updated weights for policy 0, policy_version 211221 (0.0008) [2023-12-27 07:06:50,383][20347] Updated weights for policy 0, policy_version 211231 (0.0008) [2023-12-27 07:06:50,684][20381] Updated weights for policy 1, policy_version 210872 (0.0006) [2023-12-27 07:06:50,735][20381] Updated weights for policy 1, policy_version 210882 (0.0007) [2023-12-27 07:06:50,785][20381] Updated weights for policy 1, policy_version 210892 (0.0007) [2023-12-27 07:06:51,075][20347] Updated weights for policy 0, policy_version 211241 (0.0008) [2023-12-27 07:06:51,133][20347] Updated weights for policy 0, policy_version 211251 (0.0008) [2023-12-27 07:06:51,193][20347] Updated weights for policy 0, policy_version 211261 (0.0008) [2023-12-27 07:06:51,246][20347] Updated weights for policy 0, policy_version 211271 (0.0007) [2023-12-27 07:06:51,485][20381] Updated weights for policy 1, policy_version 210902 (0.0007) [2023-12-27 07:06:51,539][20381] Updated weights for policy 1, policy_version 210912 (0.0008) [2023-12-27 07:06:51,588][20381] Updated weights for policy 1, policy_version 210922 (0.0007) [2023-12-27 07:06:51,971][20347] Updated weights for policy 0, policy_version 211281 (0.0008) [2023-12-27 07:06:52,030][20347] Updated weights for policy 0, policy_version 211291 (0.0007) [2023-12-27 07:06:52,079][20347] Updated weights for policy 0, policy_version 211301 (0.0005) [2023-12-27 07:06:52,217][20381] Updated weights for policy 1, policy_version 210932 (0.0007) [2023-12-27 07:06:52,275][20381] Updated weights for policy 1, policy_version 210942 (0.0008) [2023-12-27 07:06:52,331][20381] Updated weights for policy 1, policy_version 210952 (0.0006) [2023-12-27 07:06:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 108118016. Throughput: 0: 10364.0, 1: 10445.1. Samples: 108090276. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000211304_54108160.pth... [2023-12-27 07:06:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000210088_53796864.pth [2023-12-27 07:06:52,376][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000210960_54018048.pth... [2023-12-27 07:06:52,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000209712_53698560.pth [2023-12-27 07:06:52,762][20347] Updated weights for policy 0, policy_version 211311 (0.0005) [2023-12-27 07:06:52,823][20347] Updated weights for policy 0, policy_version 211321 (0.0006) [2023-12-27 07:06:52,878][20347] Updated weights for policy 0, policy_version 211331 (0.0007) [2023-12-27 07:06:53,000][20381] Updated weights for policy 1, policy_version 210962 (0.0007) [2023-12-27 07:06:53,048][20381] Updated weights for policy 1, policy_version 210972 (0.0006) [2023-12-27 07:06:53,105][20381] Updated weights for policy 1, policy_version 210982 (0.0008) [2023-12-27 07:06:53,157][20381] Updated weights for policy 1, policy_version 210992 (0.0008) [2023-12-27 07:06:53,488][20347] Updated weights for policy 0, policy_version 211341 (0.0005) [2023-12-27 07:06:53,545][20347] Updated weights for policy 0, policy_version 211351 (0.0007) [2023-12-27 07:06:53,592][20347] Updated weights for policy 0, policy_version 211361 (0.0007) [2023-12-27 07:06:53,766][20381] Updated weights for policy 1, policy_version 211002 (0.0005) [2023-12-27 07:06:53,812][20381] Updated weights for policy 1, policy_version 211012 (0.0005) [2023-12-27 07:06:53,858][20381] Updated weights for policy 1, policy_version 211022 (0.0005) [2023-12-27 07:06:54,291][20347] Updated weights for policy 0, policy_version 211371 (0.0007) [2023-12-27 07:06:54,350][20347] Updated weights for policy 0, policy_version 211381 (0.0007) [2023-12-27 07:06:54,410][20347] Updated weights for policy 0, policy_version 211391 (0.0006) [2023-12-27 07:06:54,504][20381] Updated weights for policy 1, policy_version 211032 (0.0009) [2023-12-27 07:06:54,552][20381] Updated weights for policy 1, policy_version 211042 (0.0007) [2023-12-27 07:06:54,609][20381] Updated weights for policy 1, policy_version 211052 (0.0007) [2023-12-27 07:06:55,033][20347] Updated weights for policy 0, policy_version 211401 (0.0006) [2023-12-27 07:06:55,086][20347] Updated weights for policy 0, policy_version 211411 (0.0010) [2023-12-27 07:06:55,137][20347] Updated weights for policy 0, policy_version 211421 (0.0010) [2023-12-27 07:06:55,186][20347] Updated weights for policy 0, policy_version 211431 (0.0010) [2023-12-27 07:06:55,275][20381] Updated weights for policy 1, policy_version 211062 (0.0005) [2023-12-27 07:06:55,340][20381] Updated weights for policy 1, policy_version 211072 (0.0005) [2023-12-27 07:06:55,406][20381] Updated weights for policy 1, policy_version 211082 (0.0005) [2023-12-27 07:06:55,896][20347] Updated weights for policy 0, policy_version 211441 (0.0006) [2023-12-27 07:06:55,957][20347] Updated weights for policy 0, policy_version 211451 (0.0009) [2023-12-27 07:06:56,016][20347] Updated weights for policy 0, policy_version 211461 (0.0010) [2023-12-27 07:06:56,018][20381] Updated weights for policy 1, policy_version 211092 (0.0011) [2023-12-27 07:06:56,071][20381] Updated weights for policy 1, policy_version 211102 (0.0011) [2023-12-27 07:06:56,126][20381] Updated weights for policy 1, policy_version 211112 (0.0010) [2023-12-27 07:06:56,610][20347] Updated weights for policy 0, policy_version 211471 (0.0007) [2023-12-27 07:06:56,666][20347] Updated weights for policy 0, policy_version 211481 (0.0009) [2023-12-27 07:06:56,722][20347] Updated weights for policy 0, policy_version 211491 (0.0006) [2023-12-27 07:06:56,815][20381] Updated weights for policy 1, policy_version 211122 (0.0010) [2023-12-27 07:06:56,867][20381] Updated weights for policy 1, policy_version 211132 (0.0010) [2023-12-27 07:06:56,912][20381] Updated weights for policy 1, policy_version 211142 (0.0010) [2023-12-27 07:06:56,956][20381] Updated weights for policy 1, policy_version 211152 (0.0010) [2023-12-27 07:06:57,260][20347] Updated weights for policy 0, policy_version 211501 (0.0008) [2023-12-27 07:06:57,309][20347] Updated weights for policy 0, policy_version 211511 (0.0006) [2023-12-27 07:06:57,358][20347] Updated weights for policy 0, policy_version 211521 (0.0008) [2023-12-27 07:06:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 108224512. Throughput: 0: 10392.7, 1: 10480.4. Samples: 108215952. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:06:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:06:57,643][20381] Updated weights for policy 1, policy_version 211162 (0.0011) [2023-12-27 07:06:57,697][20381] Updated weights for policy 1, policy_version 211172 (0.0008) [2023-12-27 07:06:57,760][20381] Updated weights for policy 1, policy_version 211182 (0.0005) [2023-12-27 07:06:58,013][20347] Updated weights for policy 0, policy_version 211531 (0.0009) [2023-12-27 07:06:58,074][20347] Updated weights for policy 0, policy_version 211541 (0.0008) [2023-12-27 07:06:58,134][20347] Updated weights for policy 0, policy_version 211551 (0.0008) [2023-12-27 07:06:58,391][20381] Updated weights for policy 1, policy_version 211192 (0.0007) [2023-12-27 07:06:58,461][20381] Updated weights for policy 1, policy_version 211202 (0.0009) [2023-12-27 07:06:58,523][20381] Updated weights for policy 1, policy_version 211212 (0.0008) [2023-12-27 07:06:58,859][20347] Updated weights for policy 0, policy_version 211561 (0.0007) [2023-12-27 07:06:58,932][20347] Updated weights for policy 0, policy_version 211571 (0.0008) [2023-12-27 07:06:58,994][20347] Updated weights for policy 0, policy_version 211581 (0.0007) [2023-12-27 07:06:59,047][20347] Updated weights for policy 0, policy_version 211591 (0.0006) [2023-12-27 07:06:59,334][20381] Updated weights for policy 1, policy_version 211222 (0.0008) [2023-12-27 07:06:59,404][20381] Updated weights for policy 1, policy_version 211232 (0.0009) [2023-12-27 07:06:59,462][20381] Updated weights for policy 1, policy_version 211242 (0.0010) [2023-12-27 07:06:59,787][20347] Updated weights for policy 0, policy_version 211601 (0.0007) [2023-12-27 07:06:59,846][20347] Updated weights for policy 0, policy_version 211611 (0.0008) [2023-12-27 07:06:59,901][20347] Updated weights for policy 0, policy_version 211621 (0.0008) [2023-12-27 07:07:00,170][20381] Updated weights for policy 1, policy_version 211252 (0.0009) [2023-12-27 07:07:00,216][20381] Updated weights for policy 1, policy_version 211262 (0.0007) [2023-12-27 07:07:00,261][20381] Updated weights for policy 1, policy_version 211272 (0.0007) [2023-12-27 07:07:00,638][20347] Updated weights for policy 0, policy_version 211631 (0.0009) [2023-12-27 07:07:00,687][20347] Updated weights for policy 0, policy_version 211641 (0.0010) [2023-12-27 07:07:00,735][20347] Updated weights for policy 0, policy_version 211651 (0.0010) [2023-12-27 07:07:00,951][20381] Updated weights for policy 1, policy_version 211282 (0.0007) [2023-12-27 07:07:01,011][20381] Updated weights for policy 1, policy_version 211292 (0.0006) [2023-12-27 07:07:01,073][20381] Updated weights for policy 1, policy_version 211302 (0.0008) [2023-12-27 07:07:01,127][20381] Updated weights for policy 1, policy_version 211312 (0.0007) [2023-12-27 07:07:01,424][20347] Updated weights for policy 0, policy_version 211661 (0.0010) [2023-12-27 07:07:01,470][20347] Updated weights for policy 0, policy_version 211671 (0.0010) [2023-12-27 07:07:01,519][20347] Updated weights for policy 0, policy_version 211681 (0.0010) [2023-12-27 07:07:01,743][20381] Updated weights for policy 1, policy_version 211322 (0.0007) [2023-12-27 07:07:01,804][20381] Updated weights for policy 1, policy_version 211332 (0.0008) [2023-12-27 07:07:01,861][20381] Updated weights for policy 1, policy_version 211342 (0.0007) [2023-12-27 07:07:02,253][20347] Updated weights for policy 0, policy_version 211691 (0.0009) [2023-12-27 07:07:02,315][20347] Updated weights for policy 0, policy_version 211701 (0.0008) [2023-12-27 07:07:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 108322816. Throughput: 0: 10374.1, 1: 10450.6. Samples: 108339244. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 07:07:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:07:02,377][20347] Updated weights for policy 0, policy_version 211711 (0.0010) [2023-12-27 07:07:02,454][20381] Updated weights for policy 1, policy_version 211352 (0.0007) [2023-12-27 07:07:02,509][20381] Updated weights for policy 1, policy_version 211362 (0.0007) [2023-12-27 07:07:02,561][20381] Updated weights for policy 1, policy_version 211372 (0.0007) [2023-12-27 07:07:03,052][20347] Updated weights for policy 0, policy_version 211721 (0.0010) [2023-12-27 07:07:03,111][20347] Updated weights for policy 0, policy_version 211731 (0.0007) [2023-12-27 07:07:03,134][20381] Updated weights for policy 1, policy_version 211382 (0.0006) [2023-12-27 07:07:03,167][20347] Updated weights for policy 0, policy_version 211741 (0.0008) [2023-12-27 07:07:03,189][20381] Updated weights for policy 1, policy_version 211392 (0.0007) [2023-12-27 07:07:03,218][20347] Updated weights for policy 0, policy_version 211751 (0.0005) [2023-12-27 07:07:03,233][20381] Updated weights for policy 1, policy_version 211402 (0.0006) [2023-12-27 07:07:03,732][20347] Updated weights for policy 0, policy_version 211761 (0.0005) [2023-12-27 07:07:03,786][20347] Updated weights for policy 0, policy_version 211771 (0.0005) [2023-12-27 07:07:03,853][20347] Updated weights for policy 0, policy_version 211781 (0.0007) [2023-12-27 07:07:03,935][20381] Updated weights for policy 1, policy_version 211412 (0.0008) [2023-12-27 07:07:03,995][20381] Updated weights for policy 1, policy_version 211422 (0.0008) [2023-12-27 07:07:04,047][20381] Updated weights for policy 1, policy_version 211432 (0.0007) [2023-12-27 07:07:04,478][20347] Updated weights for policy 0, policy_version 211791 (0.0008) [2023-12-27 07:07:04,530][20347] Updated weights for policy 0, policy_version 211801 (0.0008) [2023-12-27 07:07:04,577][20347] Updated weights for policy 0, policy_version 211811 (0.0009) [2023-12-27 07:07:04,676][20381] Updated weights for policy 1, policy_version 211442 (0.0006) [2023-12-27 07:07:04,732][20381] Updated weights for policy 1, policy_version 211452 (0.0005) [2023-12-27 07:07:04,785][20381] Updated weights for policy 1, policy_version 211462 (0.0005) [2023-12-27 07:07:04,845][20381] Updated weights for policy 1, policy_version 211472 (0.0006) [2023-12-27 07:07:05,336][20347] Updated weights for policy 0, policy_version 211821 (0.0006) [2023-12-27 07:07:05,384][20347] Updated weights for policy 0, policy_version 211831 (0.0005) [2023-12-27 07:07:05,425][20381] Updated weights for policy 1, policy_version 211482 (0.0007) [2023-12-27 07:07:05,428][20347] Updated weights for policy 0, policy_version 211841 (0.0005) [2023-12-27 07:07:05,475][20381] Updated weights for policy 1, policy_version 211492 (0.0008) [2023-12-27 07:07:05,536][20381] Updated weights for policy 1, policy_version 211502 (0.0008) [2023-12-27 07:07:06,069][20347] Updated weights for policy 0, policy_version 211851 (0.0006) [2023-12-27 07:07:06,127][20347] Updated weights for policy 0, policy_version 211861 (0.0008) [2023-12-27 07:07:06,186][20347] Updated weights for policy 0, policy_version 211871 (0.0006) [2023-12-27 07:07:06,236][20381] Updated weights for policy 1, policy_version 211512 (0.0007) [2023-12-27 07:07:06,294][20381] Updated weights for policy 1, policy_version 211522 (0.0007) [2023-12-27 07:07:06,364][20381] Updated weights for policy 1, policy_version 211532 (0.0007) [2023-12-27 07:07:06,852][20347] Updated weights for policy 0, policy_version 211881 (0.0006) [2023-12-27 07:07:06,911][20347] Updated weights for policy 0, policy_version 211891 (0.0007) [2023-12-27 07:07:06,964][20347] Updated weights for policy 0, policy_version 211901 (0.0008) [2023-12-27 07:07:07,010][20347] Updated weights for policy 0, policy_version 211911 (0.0008) [2023-12-27 07:07:07,078][20381] Updated weights for policy 1, policy_version 211542 (0.0007) [2023-12-27 07:07:07,135][20381] Updated weights for policy 1, policy_version 211552 (0.0008) [2023-12-27 07:07:07,191][20381] Updated weights for policy 1, policy_version 211562 (0.0008) [2023-12-27 07:07:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 108437504. Throughput: 0: 10382.2, 1: 10517.1. Samples: 108404640. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:07:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000211568_54173696.pth... [2023-12-27 07:07:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000211912_54263808.pth... [2023-12-27 07:07:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000210352_53862400.pth [2023-12-27 07:07:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000210728_53960704.pth [2023-12-27 07:07:07,728][20347] Updated weights for policy 0, policy_version 211921 (0.0009) [2023-12-27 07:07:07,784][20347] Updated weights for policy 0, policy_version 211931 (0.0007) [2023-12-27 07:07:07,814][20381] Updated weights for policy 1, policy_version 211572 (0.0009) [2023-12-27 07:07:07,839][20347] Updated weights for policy 0, policy_version 211941 (0.0007) [2023-12-27 07:07:07,864][20381] Updated weights for policy 1, policy_version 211582 (0.0006) [2023-12-27 07:07:07,910][20381] Updated weights for policy 1, policy_version 211592 (0.0007) [2023-12-27 07:07:08,520][20347] Updated weights for policy 0, policy_version 211951 (0.0005) [2023-12-27 07:07:08,583][20347] Updated weights for policy 0, policy_version 211961 (0.0006) [2023-12-27 07:07:08,635][20347] Updated weights for policy 0, policy_version 211971 (0.0007) [2023-12-27 07:07:08,636][20381] Updated weights for policy 1, policy_version 211602 (0.0008) [2023-12-27 07:07:08,692][20381] Updated weights for policy 1, policy_version 211612 (0.0011) [2023-12-27 07:07:08,751][20381] Updated weights for policy 1, policy_version 211622 (0.0011) [2023-12-27 07:07:08,808][20381] Updated weights for policy 1, policy_version 211632 (0.0011) [2023-12-27 07:07:09,241][20347] Updated weights for policy 0, policy_version 211981 (0.0007) [2023-12-27 07:07:09,305][20347] Updated weights for policy 0, policy_version 211991 (0.0011) [2023-12-27 07:07:09,370][20347] Updated weights for policy 0, policy_version 212001 (0.0010) [2023-12-27 07:07:09,473][20381] Updated weights for policy 1, policy_version 211642 (0.0011) [2023-12-27 07:07:09,525][20381] Updated weights for policy 1, policy_version 211652 (0.0010) [2023-12-27 07:07:09,588][20381] Updated weights for policy 1, policy_version 211662 (0.0011) [2023-12-27 07:07:09,960][20347] Updated weights for policy 0, policy_version 212011 (0.0010) [2023-12-27 07:07:10,021][20347] Updated weights for policy 0, policy_version 212021 (0.0007) [2023-12-27 07:07:10,082][20347] Updated weights for policy 0, policy_version 212031 (0.0007) [2023-12-27 07:07:10,278][20381] Updated weights for policy 1, policy_version 211672 (0.0011) [2023-12-27 07:07:10,338][20381] Updated weights for policy 1, policy_version 211682 (0.0011) [2023-12-27 07:07:10,392][20381] Updated weights for policy 1, policy_version 211692 (0.0010) [2023-12-27 07:07:10,674][20347] Updated weights for policy 0, policy_version 212041 (0.0006) [2023-12-27 07:07:10,739][20347] Updated weights for policy 0, policy_version 212051 (0.0011) [2023-12-27 07:07:10,785][20347] Updated weights for policy 0, policy_version 212061 (0.0010) [2023-12-27 07:07:10,835][20347] Updated weights for policy 0, policy_version 212071 (0.0010) [2023-12-27 07:07:11,054][20381] Updated weights for policy 1, policy_version 211702 (0.0008) [2023-12-27 07:07:11,114][20381] Updated weights for policy 1, policy_version 211712 (0.0006) [2023-12-27 07:07:11,185][20381] Updated weights for policy 1, policy_version 211722 (0.0008) [2023-12-27 07:07:11,571][20347] Updated weights for policy 0, policy_version 212081 (0.0008) [2023-12-27 07:07:11,630][20347] Updated weights for policy 0, policy_version 212091 (0.0008) [2023-12-27 07:07:11,688][20347] Updated weights for policy 0, policy_version 212101 (0.0008) [2023-12-27 07:07:11,843][20381] Updated weights for policy 1, policy_version 211732 (0.0008) [2023-12-27 07:07:11,900][20381] Updated weights for policy 1, policy_version 211742 (0.0008) [2023-12-27 07:07:11,950][20381] Updated weights for policy 1, policy_version 211752 (0.0010) [2023-12-27 07:07:12,361][20347] Updated weights for policy 0, policy_version 212111 (0.0009) [2023-12-27 07:07:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 108535808. Throughput: 0: 10354.9, 1: 10526.6. Samples: 108528524. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:07:12,421][20347] Updated weights for policy 0, policy_version 212121 (0.0008) [2023-12-27 07:07:12,504][20347] Updated weights for policy 0, policy_version 212131 (0.0008) [2023-12-27 07:07:12,631][20381] Updated weights for policy 1, policy_version 211762 (0.0010) [2023-12-27 07:07:12,688][20381] Updated weights for policy 1, policy_version 211772 (0.0008) [2023-12-27 07:07:12,747][20381] Updated weights for policy 1, policy_version 211782 (0.0011) [2023-12-27 07:07:12,810][20381] Updated weights for policy 1, policy_version 211792 (0.0010) [2023-12-27 07:07:13,156][20347] Updated weights for policy 0, policy_version 212141 (0.0011) [2023-12-27 07:07:13,216][20347] Updated weights for policy 0, policy_version 212151 (0.0011) [2023-12-27 07:07:13,271][20347] Updated weights for policy 0, policy_version 212161 (0.0010) [2023-12-27 07:07:13,445][20381] Updated weights for policy 1, policy_version 211802 (0.0005) [2023-12-27 07:07:13,503][20381] Updated weights for policy 1, policy_version 211812 (0.0005) [2023-12-27 07:07:13,559][20381] Updated weights for policy 1, policy_version 211822 (0.0005) [2023-12-27 07:07:13,960][20347] Updated weights for policy 0, policy_version 212171 (0.0009) [2023-12-27 07:07:14,021][20347] Updated weights for policy 0, policy_version 212181 (0.0007) [2023-12-27 07:07:14,078][20347] Updated weights for policy 0, policy_version 212191 (0.0007) [2023-12-27 07:07:14,119][20381] Updated weights for policy 1, policy_version 211832 (0.0009) [2023-12-27 07:07:14,173][20381] Updated weights for policy 1, policy_version 211842 (0.0010) [2023-12-27 07:07:14,225][20381] Updated weights for policy 1, policy_version 211852 (0.0010) [2023-12-27 07:07:14,774][20347] Updated weights for policy 0, policy_version 212201 (0.0008) [2023-12-27 07:07:14,838][20347] Updated weights for policy 0, policy_version 212211 (0.0009) [2023-12-27 07:07:14,897][20381] Updated weights for policy 1, policy_version 211862 (0.0009) [2023-12-27 07:07:14,903][20347] Updated weights for policy 0, policy_version 212221 (0.0007) [2023-12-27 07:07:14,947][20381] Updated weights for policy 1, policy_version 211872 (0.0011) [2023-12-27 07:07:14,966][20347] Updated weights for policy 0, policy_version 212231 (0.0007) [2023-12-27 07:07:15,006][20381] Updated weights for policy 1, policy_version 211882 (0.0011) [2023-12-27 07:07:15,615][20347] Updated weights for policy 0, policy_version 212241 (0.0008) [2023-12-27 07:07:15,619][20291] KL-divergence is very high: 457.6111 [2023-12-27 07:07:15,666][20291] KL-divergence is very high: 646.1890 [2023-12-27 07:07:15,672][20347] Updated weights for policy 0, policy_version 212251 (0.0008) [2023-12-27 07:07:15,688][20381] Updated weights for policy 1, policy_version 211892 (0.0009) [2023-12-27 07:07:15,711][20291] KL-divergence is very high: 102.6746 [2023-12-27 07:07:15,728][20347] Updated weights for policy 0, policy_version 212261 (0.0008) [2023-12-27 07:07:15,745][20381] Updated weights for policy 1, policy_version 211902 (0.0005) [2023-12-27 07:07:15,806][20381] Updated weights for policy 1, policy_version 211912 (0.0005) [2023-12-27 07:07:16,367][20381] Updated weights for policy 1, policy_version 211922 (0.0006) [2023-12-27 07:07:16,386][20347] Updated weights for policy 0, policy_version 212271 (0.0009) [2023-12-27 07:07:16,420][20381] Updated weights for policy 1, policy_version 211932 (0.0010) [2023-12-27 07:07:16,421][20291] KL-divergence is very high: 245.1710 [2023-12-27 07:07:16,445][20347] Updated weights for policy 0, policy_version 212281 (0.0010) [2023-12-27 07:07:16,465][20381] Updated weights for policy 1, policy_version 211942 (0.0010) [2023-12-27 07:07:16,499][20347] Updated weights for policy 0, policy_version 212291 (0.0010) [2023-12-27 07:07:16,517][20381] Updated weights for policy 1, policy_version 211952 (0.0010) [2023-12-27 07:07:17,140][20381] Updated weights for policy 1, policy_version 211962 (0.0007) [2023-12-27 07:07:17,151][20347] Updated weights for policy 0, policy_version 212301 (0.0007) [2023-12-27 07:07:17,151][20291] KL-divergence is very high: 241.5797 [2023-12-27 07:07:17,177][20291] KL-divergence is very high: 676.7852 [2023-12-27 07:07:17,188][20291] KL-divergence is very high: 185.3834 [2023-12-27 07:07:17,200][20291] KL-divergence is very high: 121.8361 [2023-12-27 07:07:17,207][20381] Updated weights for policy 1, policy_version 211972 (0.0006) [2023-12-27 07:07:17,210][20347] Updated weights for policy 0, policy_version 212311 (0.0008) [2023-12-27 07:07:17,222][20291] KL-divergence is very high: 265.4021 [2023-12-27 07:07:17,234][20291] KL-divergence is very high: 113.6752 [2023-12-27 07:07:17,263][20381] Updated weights for policy 1, policy_version 211982 (0.0007) [2023-12-27 07:07:17,265][20347] Updated weights for policy 0, policy_version 212321 (0.0006) [2023-12-27 07:07:17,266][20291] KL-divergence is very high: 264.9890 [2023-12-27 07:07:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 108650496. Throughput: 0: 10292.7, 1: 10534.6. Samples: 108654276. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:17,363][19170] Avg episode reward: [(0, '980.940'), (1, '1000.000')] [2023-12-27 07:07:17,822][20381] Updated weights for policy 1, policy_version 211992 (0.0008) [2023-12-27 07:07:17,872][20381] Updated weights for policy 1, policy_version 212002 (0.0008) [2023-12-27 07:07:17,920][20381] Updated weights for policy 1, policy_version 212012 (0.0008) [2023-12-27 07:07:17,930][20291] KL-divergence is very high: 392.5157 [2023-12-27 07:07:17,943][20347] Updated weights for policy 0, policy_version 212331 (0.0008) [2023-12-27 07:07:17,973][20291] KL-divergence is very high: 170.7060 [2023-12-27 07:07:17,992][20347] Updated weights for policy 0, policy_version 212341 (0.0008) [2023-12-27 07:07:18,014][20291] KL-divergence is very high: 231.5386 [2023-12-27 07:07:18,047][20347] Updated weights for policy 0, policy_version 212351 (0.0007) [2023-12-27 07:07:18,056][20291] KL-divergence is very high: 178.6829 [2023-12-27 07:07:18,079][20291] KL-divergence is very high: 163.8151 [2023-12-27 07:07:18,084][20291] KL-divergence is very high: 163.7028 [2023-12-27 07:07:18,563][20381] Updated weights for policy 1, policy_version 212022 (0.0008) [2023-12-27 07:07:18,611][20381] Updated weights for policy 1, policy_version 212032 (0.0010) [2023-12-27 07:07:18,666][20381] Updated weights for policy 1, policy_version 212042 (0.0010) [2023-12-27 07:07:18,773][20347] Updated weights for policy 0, policy_version 212361 (0.0007) [2023-12-27 07:07:18,798][20291] KL-divergence is very high: 112.7968 [2023-12-27 07:07:18,827][20347] Updated weights for policy 0, policy_version 212371 (0.0007) [2023-12-27 07:07:18,835][20291] KL-divergence is very high: 372.2045 [2023-12-27 07:07:18,850][20291] KL-divergence is very high: 135.8062 [2023-12-27 07:07:18,855][20291] KL-divergence is very high: 291.4135 [2023-12-27 07:07:18,874][20347] Updated weights for policy 0, policy_version 212381 (0.0007) [2023-12-27 07:07:18,875][20291] KL-divergence is very high: 397.4537 [2023-12-27 07:07:18,880][20291] KL-divergence is very high: 145.2008 [2023-12-27 07:07:18,886][20291] KL-divergence is very high: 118.2951 [2023-12-27 07:07:18,914][20291] KL-divergence is very high: 308.8788 [2023-12-27 07:07:18,925][20347] Updated weights for policy 0, policy_version 212391 (0.0005) [2023-12-27 07:07:19,359][20381] Updated weights for policy 1, policy_version 212052 (0.0010) [2023-12-27 07:07:19,425][20381] Updated weights for policy 1, policy_version 212062 (0.0009) [2023-12-27 07:07:19,490][20381] Updated weights for policy 1, policy_version 212072 (0.0010) [2023-12-27 07:07:19,524][20291] KL-divergence is very high: 103.1038 [2023-12-27 07:07:19,552][20291] KL-divergence is very high: 368.1281 [2023-12-27 07:07:19,563][20347] Updated weights for policy 0, policy_version 212401 (0.0006) [2023-12-27 07:07:19,570][20291] KL-divergence is very high: 161.8705 [2023-12-27 07:07:19,625][20347] Updated weights for policy 0, policy_version 212411 (0.0006) [2023-12-27 07:07:19,684][20347] Updated weights for policy 0, policy_version 212421 (0.0008) [2023-12-27 07:07:20,118][20381] Updated weights for policy 1, policy_version 212082 (0.0011) [2023-12-27 07:07:20,179][20381] Updated weights for policy 1, policy_version 212092 (0.0011) [2023-12-27 07:07:20,236][20381] Updated weights for policy 1, policy_version 212102 (0.0010) [2023-12-27 07:07:20,294][20381] Updated weights for policy 1, policy_version 212112 (0.0010) [2023-12-27 07:07:20,366][20291] KL-divergence is very high: 218.8647 [2023-12-27 07:07:20,404][20347] Updated weights for policy 0, policy_version 212431 (0.0009) [2023-12-27 07:07:20,411][20291] KL-divergence is very high: 320.4626 [2023-12-27 07:07:20,454][20291] KL-divergence is very high: 594.1928 [2023-12-27 07:07:20,459][20347] Updated weights for policy 0, policy_version 212441 (0.0010) [2023-12-27 07:07:20,494][20291] KL-divergence is very high: 105.5195 [2023-12-27 07:07:20,509][20347] Updated weights for policy 0, policy_version 212451 (0.0008) [2023-12-27 07:07:20,982][20381] Updated weights for policy 1, policy_version 212122 (0.0011) [2023-12-27 07:07:21,045][20381] Updated weights for policy 1, policy_version 212132 (0.0010) [2023-12-27 07:07:21,111][20381] Updated weights for policy 1, policy_version 212142 (0.0009) [2023-12-27 07:07:21,202][20347] Updated weights for policy 0, policy_version 212461 (0.0008) [2023-12-27 07:07:21,264][20347] Updated weights for policy 0, policy_version 212471 (0.0009) [2023-12-27 07:07:21,320][20347] Updated weights for policy 0, policy_version 212481 (0.0008) [2023-12-27 07:07:21,812][20381] Updated weights for policy 1, policy_version 212152 (0.0011) [2023-12-27 07:07:21,875][20381] Updated weights for policy 1, policy_version 212162 (0.0007) [2023-12-27 07:07:21,937][20381] Updated weights for policy 1, policy_version 212172 (0.0005) [2023-12-27 07:07:22,043][20347] Updated weights for policy 0, policy_version 212491 (0.0009) [2023-12-27 07:07:22,108][20347] Updated weights for policy 0, policy_version 212501 (0.0011) [2023-12-27 07:07:22,119][20291] KL-divergence is very high: 111.9667 [2023-12-27 07:07:22,170][20347] Updated weights for policy 0, policy_version 212511 (0.0010) [2023-12-27 07:07:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 108748800. Throughput: 0: 10276.9, 1: 10542.1. Samples: 108717184. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:22,363][19170] Avg episode reward: [(0, '803.160'), (1, '1000.000')] [2023-12-27 07:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000212520_54419456.pth... [2023-12-27 07:07:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000212176_54329344.pth... [2023-12-27 07:07:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000211304_54108160.pth [2023-12-27 07:07:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000210960_54018048.pth [2023-12-27 07:07:22,582][20381] Updated weights for policy 1, policy_version 212182 (0.0008) [2023-12-27 07:07:22,645][20381] Updated weights for policy 1, policy_version 212192 (0.0006) [2023-12-27 07:07:22,709][20381] Updated weights for policy 1, policy_version 212202 (0.0005) [2023-12-27 07:07:22,855][20291] KL-divergence is very high: 114.6382 [2023-12-27 07:07:22,863][20347] Updated weights for policy 0, policy_version 212521 (0.0010) [2023-12-27 07:07:22,903][20291] KL-divergence is very high: 301.3090 [2023-12-27 07:07:22,909][20291] KL-divergence is very high: 161.3614 [2023-12-27 07:07:22,928][20347] Updated weights for policy 0, policy_version 212531 (0.0010) [2023-12-27 07:07:22,948][20291] KL-divergence is very high: 102.9482 [2023-12-27 07:07:22,954][20291] KL-divergence is very high: 334.7299 [2023-12-27 07:07:22,961][20291] KL-divergence is very high: 183.8755 [2023-12-27 07:07:22,993][20347] Updated weights for policy 0, policy_version 212541 (0.0010) [2023-12-27 07:07:23,004][20291] KL-divergence is very high: 202.0581 [2023-12-27 07:07:23,010][20291] KL-divergence is very high: 144.2438 [2023-12-27 07:07:23,016][20291] KL-divergence is very high: 130.3979 [2023-12-27 07:07:23,058][20347] Updated weights for policy 0, policy_version 212551 (0.0008) [2023-12-27 07:07:23,059][20291] KL-divergence is very high: 118.0148 [2023-12-27 07:07:23,298][20381] Updated weights for policy 1, policy_version 212212 (0.0007) [2023-12-27 07:07:23,349][20381] Updated weights for policy 1, policy_version 212222 (0.0008) [2023-12-27 07:07:23,401][20381] Updated weights for policy 1, policy_version 212232 (0.0008) [2023-12-27 07:07:23,665][20347] Updated weights for policy 0, policy_version 212561 (0.0005) [2023-12-27 07:07:23,674][20291] KL-divergence is very high: 106.9187 [2023-12-27 07:07:23,726][20347] Updated weights for policy 0, policy_version 212571 (0.0005) [2023-12-27 07:07:23,787][20347] Updated weights for policy 0, policy_version 212581 (0.0005) [2023-12-27 07:07:24,030][20381] Updated weights for policy 1, policy_version 212242 (0.0008) [2023-12-27 07:07:24,081][20381] Updated weights for policy 1, policy_version 212252 (0.0005) [2023-12-27 07:07:24,136][20381] Updated weights for policy 1, policy_version 212262 (0.0006) [2023-12-27 07:07:24,186][20381] Updated weights for policy 1, policy_version 212272 (0.0010) [2023-12-27 07:07:24,309][20347] Updated weights for policy 0, policy_version 212591 (0.0010) [2023-12-27 07:07:24,345][20291] KL-divergence is very high: 116.0949 [2023-12-27 07:07:24,354][20347] Updated weights for policy 0, policy_version 212601 (0.0008) [2023-12-27 07:07:24,406][20347] Updated weights for policy 0, policy_version 212611 (0.0009) [2023-12-27 07:07:24,837][20381] Updated weights for policy 1, policy_version 212282 (0.0008) [2023-12-27 07:07:24,884][20381] Updated weights for policy 1, policy_version 212292 (0.0007) [2023-12-27 07:07:24,930][20381] Updated weights for policy 1, policy_version 212302 (0.0008) [2023-12-27 07:07:25,053][20347] Updated weights for policy 0, policy_version 212621 (0.0008) [2023-12-27 07:07:25,101][20347] Updated weights for policy 0, policy_version 212631 (0.0005) [2023-12-27 07:07:25,151][20347] Updated weights for policy 0, policy_version 212641 (0.0005) [2023-12-27 07:07:25,689][20381] Updated weights for policy 1, policy_version 212312 (0.0008) [2023-12-27 07:07:25,737][20347] Updated weights for policy 0, policy_version 212651 (0.0007) [2023-12-27 07:07:25,742][20381] Updated weights for policy 1, policy_version 212322 (0.0007) [2023-12-27 07:07:25,800][20347] Updated weights for policy 0, policy_version 212661 (0.0010) [2023-12-27 07:07:25,805][20381] Updated weights for policy 1, policy_version 212332 (0.0006) [2023-12-27 07:07:25,850][20347] Updated weights for policy 0, policy_version 212671 (0.0009) [2023-12-27 07:07:26,450][20347] Updated weights for policy 0, policy_version 212681 (0.0006) [2023-12-27 07:07:26,477][20381] Updated weights for policy 1, policy_version 212342 (0.0008) [2023-12-27 07:07:26,503][20347] Updated weights for policy 0, policy_version 212691 (0.0010) [2023-12-27 07:07:26,530][20381] Updated weights for policy 1, policy_version 212352 (0.0011) [2023-12-27 07:07:26,558][20347] Updated weights for policy 0, policy_version 212701 (0.0010) [2023-12-27 07:07:26,584][20381] Updated weights for policy 1, policy_version 212362 (0.0010) [2023-12-27 07:07:26,606][20347] Updated weights for policy 0, policy_version 212711 (0.0010) [2023-12-27 07:07:27,230][20347] Updated weights for policy 0, policy_version 212721 (0.0010) [2023-12-27 07:07:27,288][20347] Updated weights for policy 0, policy_version 212731 (0.0010) [2023-12-27 07:07:27,310][20381] Updated weights for policy 1, policy_version 212372 (0.0009) [2023-12-27 07:07:27,342][20347] Updated weights for policy 0, policy_version 212741 (0.0010) [2023-12-27 07:07:27,362][20381] Updated weights for policy 1, policy_version 212382 (0.0006) [2023-12-27 07:07:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 108855296. Throughput: 0: 10357.2, 1: 10525.9. Samples: 108843428. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:27,363][19170] Avg episode reward: [(0, '520.380'), (1, '1000.000')] [2023-12-27 07:07:27,423][20381] Updated weights for policy 1, policy_version 212392 (0.0009) [2023-12-27 07:07:27,991][20347] Updated weights for policy 0, policy_version 212751 (0.0007) [2023-12-27 07:07:28,051][20347] Updated weights for policy 0, policy_version 212761 (0.0007) [2023-12-27 07:07:28,055][20381] Updated weights for policy 1, policy_version 212402 (0.0009) [2023-12-27 07:07:28,102][20381] Updated weights for policy 1, policy_version 212412 (0.0007) [2023-12-27 07:07:28,117][20347] Updated weights for policy 0, policy_version 212771 (0.0007) [2023-12-27 07:07:28,155][20381] Updated weights for policy 1, policy_version 212422 (0.0006) [2023-12-27 07:07:28,207][20381] Updated weights for policy 1, policy_version 212432 (0.0008) [2023-12-27 07:07:28,704][20347] Updated weights for policy 0, policy_version 212781 (0.0006) [2023-12-27 07:07:28,762][20347] Updated weights for policy 0, policy_version 212791 (0.0007) [2023-12-27 07:07:28,821][20347] Updated weights for policy 0, policy_version 212801 (0.0007) [2023-12-27 07:07:28,932][20381] Updated weights for policy 1, policy_version 212442 (0.0007) [2023-12-27 07:07:28,981][20381] Updated weights for policy 1, policy_version 212452 (0.0007) [2023-12-27 07:07:29,031][20381] Updated weights for policy 1, policy_version 212462 (0.0007) [2023-12-27 07:07:29,527][20347] Updated weights for policy 0, policy_version 212811 (0.0009) [2023-12-27 07:07:29,586][20347] Updated weights for policy 0, policy_version 212821 (0.0010) [2023-12-27 07:07:29,641][20347] Updated weights for policy 0, policy_version 212831 (0.0010) [2023-12-27 07:07:29,778][20381] Updated weights for policy 1, policy_version 212472 (0.0007) [2023-12-27 07:07:29,849][20381] Updated weights for policy 1, policy_version 212482 (0.0008) [2023-12-27 07:07:29,903][20381] Updated weights for policy 1, policy_version 212492 (0.0007) [2023-12-27 07:07:30,319][20347] Updated weights for policy 0, policy_version 212841 (0.0010) [2023-12-27 07:07:30,371][20347] Updated weights for policy 0, policy_version 212851 (0.0007) [2023-12-27 07:07:30,424][20347] Updated weights for policy 0, policy_version 212861 (0.0005) [2023-12-27 07:07:30,473][20347] Updated weights for policy 0, policy_version 212871 (0.0005) [2023-12-27 07:07:30,546][20381] Updated weights for policy 1, policy_version 212502 (0.0006) [2023-12-27 07:07:30,592][20381] Updated weights for policy 1, policy_version 212512 (0.0005) [2023-12-27 07:07:30,643][20381] Updated weights for policy 1, policy_version 212522 (0.0005) [2023-12-27 07:07:31,066][20347] Updated weights for policy 0, policy_version 212881 (0.0008) [2023-12-27 07:07:31,126][20347] Updated weights for policy 0, policy_version 212891 (0.0006) [2023-12-27 07:07:31,185][20291] KL-divergence is very high: 110.3229 [2023-12-27 07:07:31,191][20347] Updated weights for policy 0, policy_version 212901 (0.0008) [2023-12-27 07:07:31,217][20381] Updated weights for policy 1, policy_version 212532 (0.0006) [2023-12-27 07:07:31,274][20381] Updated weights for policy 1, policy_version 212542 (0.0007) [2023-12-27 07:07:31,339][20381] Updated weights for policy 1, policy_version 212552 (0.0008) [2023-12-27 07:07:31,894][20291] KL-divergence is very high: 110.9608 [2023-12-27 07:07:31,920][20347] Updated weights for policy 0, policy_version 212911 (0.0007) [2023-12-27 07:07:31,976][20347] Updated weights for policy 0, policy_version 212921 (0.0005) [2023-12-27 07:07:32,008][20381] Updated weights for policy 1, policy_version 212562 (0.0006) [2023-12-27 07:07:32,038][20347] Updated weights for policy 0, policy_version 212931 (0.0008) [2023-12-27 07:07:32,072][20381] Updated weights for policy 1, policy_version 212572 (0.0007) [2023-12-27 07:07:32,121][20381] Updated weights for policy 1, policy_version 212582 (0.0007) [2023-12-27 07:07:32,165][20381] Updated weights for policy 1, policy_version 212592 (0.0008) [2023-12-27 07:07:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 108961792. Throughput: 0: 10441.0, 1: 10495.5. Samples: 108969256. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:32,363][19170] Avg episode reward: [(0, '303.040'), (1, '1000.000')] [2023-12-27 07:07:32,624][20347] Updated weights for policy 0, policy_version 212941 (0.0007) [2023-12-27 07:07:32,673][20347] Updated weights for policy 0, policy_version 212951 (0.0007) [2023-12-27 07:07:32,729][20347] Updated weights for policy 0, policy_version 212961 (0.0005) [2023-12-27 07:07:32,901][20381] Updated weights for policy 1, policy_version 212602 (0.0008) [2023-12-27 07:07:32,945][20381] Updated weights for policy 1, policy_version 212612 (0.0007) [2023-12-27 07:07:32,996][20381] Updated weights for policy 1, policy_version 212622 (0.0007) [2023-12-27 07:07:33,339][20347] Updated weights for policy 0, policy_version 212971 (0.0005) [2023-12-27 07:07:33,395][20347] Updated weights for policy 0, policy_version 212981 (0.0007) [2023-12-27 07:07:33,456][20347] Updated weights for policy 0, policy_version 212991 (0.0007) [2023-12-27 07:07:33,703][20381] Updated weights for policy 1, policy_version 212632 (0.0007) [2023-12-27 07:07:33,755][20381] Updated weights for policy 1, policy_version 212642 (0.0007) [2023-12-27 07:07:33,813][20381] Updated weights for policy 1, policy_version 212652 (0.0007) [2023-12-27 07:07:34,139][20347] Updated weights for policy 0, policy_version 213001 (0.0008) [2023-12-27 07:07:34,191][20347] Updated weights for policy 0, policy_version 213011 (0.0008) [2023-12-27 07:07:34,240][20347] Updated weights for policy 0, policy_version 213021 (0.0007) [2023-12-27 07:07:34,287][20347] Updated weights for policy 0, policy_version 213031 (0.0008) [2023-12-27 07:07:34,523][20381] Updated weights for policy 1, policy_version 212662 (0.0007) [2023-12-27 07:07:34,586][20381] Updated weights for policy 1, policy_version 212672 (0.0006) [2023-12-27 07:07:34,646][20381] Updated weights for policy 1, policy_version 212682 (0.0008) [2023-12-27 07:07:35,061][20347] Updated weights for policy 0, policy_version 213041 (0.0008) [2023-12-27 07:07:35,115][20347] Updated weights for policy 0, policy_version 213051 (0.0007) [2023-12-27 07:07:35,164][20347] Updated weights for policy 0, policy_version 213061 (0.0008) [2023-12-27 07:07:35,239][20381] Updated weights for policy 1, policy_version 212692 (0.0007) [2023-12-27 07:07:35,287][20381] Updated weights for policy 1, policy_version 212702 (0.0007) [2023-12-27 07:07:35,339][20381] Updated weights for policy 1, policy_version 212712 (0.0007) [2023-12-27 07:07:35,851][20347] Updated weights for policy 0, policy_version 213071 (0.0007) [2023-12-27 07:07:35,906][20347] Updated weights for policy 0, policy_version 213081 (0.0007) [2023-12-27 07:07:35,952][20347] Updated weights for policy 0, policy_version 213091 (0.0007) [2023-12-27 07:07:35,999][20381] Updated weights for policy 1, policy_version 212722 (0.0008) [2023-12-27 07:07:36,050][20381] Updated weights for policy 1, policy_version 212732 (0.0007) [2023-12-27 07:07:36,110][20381] Updated weights for policy 1, policy_version 212742 (0.0008) [2023-12-27 07:07:36,168][20381] Updated weights for policy 1, policy_version 212752 (0.0007) [2023-12-27 07:07:36,689][20347] Updated weights for policy 0, policy_version 213101 (0.0009) [2023-12-27 07:07:36,752][20347] Updated weights for policy 0, policy_version 213111 (0.0011) [2023-12-27 07:07:36,807][20381] Updated weights for policy 1, policy_version 212762 (0.0007) [2023-12-27 07:07:36,811][20347] Updated weights for policy 0, policy_version 213121 (0.0010) [2023-12-27 07:07:36,866][20381] Updated weights for policy 1, policy_version 212772 (0.0008) [2023-12-27 07:07:36,925][20381] Updated weights for policy 1, policy_version 212782 (0.0008) [2023-12-27 07:07:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 109060096. Throughput: 0: 10437.1, 1: 10479.1. Samples: 109031512. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:37,364][19170] Avg episode reward: [(0, '202.630'), (1, '1000.000')] [2023-12-27 07:07:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000212784_54484992.pth... [2023-12-27 07:07:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000213128_54575104.pth... [2023-12-27 07:07:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000211912_54263808.pth [2023-12-27 07:07:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000211568_54173696.pth [2023-12-27 07:07:37,495][20347] Updated weights for policy 0, policy_version 213131 (0.0009) [2023-12-27 07:07:37,547][20347] Updated weights for policy 0, policy_version 213141 (0.0005) [2023-12-27 07:07:37,602][20347] Updated weights for policy 0, policy_version 213151 (0.0009) [2023-12-27 07:07:37,631][20381] Updated weights for policy 1, policy_version 212792 (0.0006) [2023-12-27 07:07:37,681][20381] Updated weights for policy 1, policy_version 212802 (0.0007) [2023-12-27 07:07:37,735][20381] Updated weights for policy 1, policy_version 212812 (0.0008) [2023-12-27 07:07:38,264][20347] Updated weights for policy 0, policy_version 213161 (0.0010) [2023-12-27 07:07:38,316][20347] Updated weights for policy 0, policy_version 213171 (0.0010) [2023-12-27 07:07:38,376][20347] Updated weights for policy 0, policy_version 213181 (0.0010) [2023-12-27 07:07:38,420][20381] Updated weights for policy 1, policy_version 212822 (0.0007) [2023-12-27 07:07:38,432][20347] Updated weights for policy 0, policy_version 213191 (0.0011) [2023-12-27 07:07:38,473][20381] Updated weights for policy 1, policy_version 212832 (0.0007) [2023-12-27 07:07:38,525][20381] Updated weights for policy 1, policy_version 212842 (0.0007) [2023-12-27 07:07:39,092][20347] Updated weights for policy 0, policy_version 213201 (0.0006) [2023-12-27 07:07:39,141][20347] Updated weights for policy 0, policy_version 213211 (0.0005) [2023-12-27 07:07:39,192][20347] Updated weights for policy 0, policy_version 213221 (0.0005) [2023-12-27 07:07:39,242][20381] Updated weights for policy 1, policy_version 212852 (0.0012) [2023-12-27 07:07:39,299][20381] Updated weights for policy 1, policy_version 212862 (0.0010) [2023-12-27 07:07:39,363][20381] Updated weights for policy 1, policy_version 212872 (0.0008) [2023-12-27 07:07:39,900][20347] Updated weights for policy 0, policy_version 213231 (0.0008) [2023-12-27 07:07:39,967][20347] Updated weights for policy 0, policy_version 213241 (0.0006) [2023-12-27 07:07:40,023][20381] Updated weights for policy 1, policy_version 212882 (0.0008) [2023-12-27 07:07:40,024][20347] Updated weights for policy 0, policy_version 213251 (0.0007) [2023-12-27 07:07:40,079][20381] Updated weights for policy 1, policy_version 212892 (0.0005) [2023-12-27 07:07:40,141][20381] Updated weights for policy 1, policy_version 212902 (0.0006) [2023-12-27 07:07:40,204][20381] Updated weights for policy 1, policy_version 212912 (0.0008) [2023-12-27 07:07:40,678][20347] Updated weights for policy 0, policy_version 213261 (0.0007) [2023-12-27 07:07:40,734][20347] Updated weights for policy 0, policy_version 213271 (0.0011) [2023-12-27 07:07:40,798][20347] Updated weights for policy 0, policy_version 213281 (0.0010) [2023-12-27 07:07:40,824][20381] Updated weights for policy 1, policy_version 212922 (0.0005) [2023-12-27 07:07:40,880][20381] Updated weights for policy 1, policy_version 212932 (0.0005) [2023-12-27 07:07:40,928][20381] Updated weights for policy 1, policy_version 212942 (0.0005) [2023-12-27 07:07:41,451][20347] Updated weights for policy 0, policy_version 213291 (0.0010) [2023-12-27 07:07:41,504][20347] Updated weights for policy 0, policy_version 213301 (0.0005) [2023-12-27 07:07:41,565][20347] Updated weights for policy 0, policy_version 213311 (0.0010) [2023-12-27 07:07:41,573][20381] Updated weights for policy 1, policy_version 212952 (0.0007) [2023-12-27 07:07:41,640][20381] Updated weights for policy 1, policy_version 212962 (0.0007) [2023-12-27 07:07:41,702][20381] Updated weights for policy 1, policy_version 212972 (0.0010) [2023-12-27 07:07:42,230][20347] Updated weights for policy 0, policy_version 213321 (0.0010) [2023-12-27 07:07:42,293][20347] Updated weights for policy 0, policy_version 213331 (0.0010) [2023-12-27 07:07:42,354][20347] Updated weights for policy 0, policy_version 213341 (0.0010) [2023-12-27 07:07:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 109158400. Throughput: 0: 10429.7, 1: 10440.7. Samples: 109155120. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:42,363][19170] Avg episode reward: [(0, '256.750'), (1, '1000.000')] [2023-12-27 07:07:42,381][20381] Updated weights for policy 1, policy_version 212982 (0.0008) [2023-12-27 07:07:42,413][20347] Updated weights for policy 0, policy_version 213351 (0.0008) [2023-12-27 07:07:42,439][20381] Updated weights for policy 1, policy_version 212992 (0.0007) [2023-12-27 07:07:42,498][20381] Updated weights for policy 1, policy_version 213002 (0.0006) [2023-12-27 07:07:43,127][20381] Updated weights for policy 1, policy_version 213012 (0.0005) [2023-12-27 07:07:43,141][20347] Updated weights for policy 0, policy_version 213361 (0.0010) [2023-12-27 07:07:43,173][20381] Updated weights for policy 1, policy_version 213022 (0.0007) [2023-12-27 07:07:43,200][20347] Updated weights for policy 0, policy_version 213371 (0.0010) [2023-12-27 07:07:43,221][20381] Updated weights for policy 1, policy_version 213032 (0.0006) [2023-12-27 07:07:43,254][20347] Updated weights for policy 0, policy_version 213381 (0.0010) [2023-12-27 07:07:43,885][20381] Updated weights for policy 1, policy_version 213042 (0.0010) [2023-12-27 07:07:43,933][20347] Updated weights for policy 0, policy_version 213391 (0.0010) [2023-12-27 07:07:43,940][20381] Updated weights for policy 1, policy_version 213052 (0.0009) [2023-12-27 07:07:43,987][20381] Updated weights for policy 1, policy_version 213062 (0.0010) [2023-12-27 07:07:43,992][20347] Updated weights for policy 0, policy_version 213401 (0.0010) [2023-12-27 07:07:44,037][20381] Updated weights for policy 1, policy_version 213072 (0.0009) [2023-12-27 07:07:44,042][20347] Updated weights for policy 0, policy_version 213411 (0.0010) [2023-12-27 07:07:44,698][20381] Updated weights for policy 1, policy_version 213082 (0.0008) [2023-12-27 07:07:44,723][20347] Updated weights for policy 0, policy_version 213421 (0.0010) [2023-12-27 07:07:44,740][20381] Updated weights for policy 1, policy_version 213092 (0.0010) [2023-12-27 07:07:44,778][20347] Updated weights for policy 0, policy_version 213431 (0.0010) [2023-12-27 07:07:44,803][20381] Updated weights for policy 1, policy_version 213102 (0.0008) [2023-12-27 07:07:44,835][20347] Updated weights for policy 0, policy_version 213441 (0.0008) [2023-12-27 07:07:45,442][20381] Updated weights for policy 1, policy_version 213112 (0.0007) [2023-12-27 07:07:45,457][20347] Updated weights for policy 0, policy_version 213451 (0.0006) [2023-12-27 07:07:45,504][20381] Updated weights for policy 1, policy_version 213122 (0.0006) [2023-12-27 07:07:45,513][20347] Updated weights for policy 0, policy_version 213461 (0.0007) [2023-12-27 07:07:45,553][20381] Updated weights for policy 1, policy_version 213132 (0.0006) [2023-12-27 07:07:45,567][20347] Updated weights for policy 0, policy_version 213471 (0.0007) [2023-12-27 07:07:46,247][20381] Updated weights for policy 1, policy_version 213142 (0.0008) [2023-12-27 07:07:46,257][20347] Updated weights for policy 0, policy_version 213481 (0.0007) [2023-12-27 07:07:46,306][20381] Updated weights for policy 1, policy_version 213152 (0.0006) [2023-12-27 07:07:46,318][20347] Updated weights for policy 0, policy_version 213491 (0.0009) [2023-12-27 07:07:46,381][20347] Updated weights for policy 0, policy_version 213501 (0.0009) [2023-12-27 07:07:46,402][20381] Updated weights for policy 1, policy_version 213162 (0.0008) [2023-12-27 07:07:46,439][20347] Updated weights for policy 0, policy_version 213511 (0.0006) [2023-12-27 07:07:47,054][20381] Updated weights for policy 1, policy_version 213172 (0.0007) [2023-12-27 07:07:47,109][20381] Updated weights for policy 1, policy_version 213182 (0.0007) [2023-12-27 07:07:47,158][20381] Updated weights for policy 1, policy_version 213192 (0.0007) [2023-12-27 07:07:47,159][20347] Updated weights for policy 0, policy_version 213521 (0.0008) [2023-12-27 07:07:47,217][20347] Updated weights for policy 0, policy_version 213531 (0.0005) [2023-12-27 07:07:47,269][20347] Updated weights for policy 0, policy_version 213541 (0.0008) [2023-12-27 07:07:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 109273088. Throughput: 0: 10411.2, 1: 10443.4. Samples: 109277700. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:47,363][19170] Avg episode reward: [(0, '533.450'), (1, '1000.000')] [2023-12-27 07:07:47,869][20381] Updated weights for policy 1, policy_version 213202 (0.0006) [2023-12-27 07:07:47,874][20347] Updated weights for policy 0, policy_version 213551 (0.0008) [2023-12-27 07:07:47,917][20381] Updated weights for policy 1, policy_version 213212 (0.0005) [2023-12-27 07:07:47,932][20347] Updated weights for policy 0, policy_version 213561 (0.0007) [2023-12-27 07:07:47,963][20381] Updated weights for policy 1, policy_version 213222 (0.0006) [2023-12-27 07:07:47,990][20347] Updated weights for policy 0, policy_version 213571 (0.0006) [2023-12-27 07:07:48,011][20381] Updated weights for policy 1, policy_version 213232 (0.0005) [2023-12-27 07:07:48,634][20347] Updated weights for policy 0, policy_version 213581 (0.0006) [2023-12-27 07:07:48,692][20347] Updated weights for policy 0, policy_version 213591 (0.0008) [2023-12-27 07:07:48,722][20381] Updated weights for policy 1, policy_version 213242 (0.0006) [2023-12-27 07:07:48,750][20347] Updated weights for policy 0, policy_version 213601 (0.0008) [2023-12-27 07:07:48,781][20381] Updated weights for policy 1, policy_version 213252 (0.0007) [2023-12-27 07:07:48,832][20381] Updated weights for policy 1, policy_version 213262 (0.0007) [2023-12-27 07:07:49,459][20347] Updated weights for policy 0, policy_version 213611 (0.0006) [2023-12-27 07:07:49,527][20347] Updated weights for policy 0, policy_version 213621 (0.0005) [2023-12-27 07:07:49,537][20381] Updated weights for policy 1, policy_version 213272 (0.0010) [2023-12-27 07:07:49,593][20381] Updated weights for policy 1, policy_version 213282 (0.0011) [2023-12-27 07:07:49,594][20347] Updated weights for policy 0, policy_version 213631 (0.0007) [2023-12-27 07:07:49,642][20381] Updated weights for policy 1, policy_version 213292 (0.0010) [2023-12-27 07:07:50,265][20347] Updated weights for policy 0, policy_version 213641 (0.0007) [2023-12-27 07:07:50,318][20347] Updated weights for policy 0, policy_version 213651 (0.0006) [2023-12-27 07:07:50,324][20381] Updated weights for policy 1, policy_version 213302 (0.0010) [2023-12-27 07:07:50,376][20347] Updated weights for policy 0, policy_version 213661 (0.0006) [2023-12-27 07:07:50,378][20381] Updated weights for policy 1, policy_version 213312 (0.0011) [2023-12-27 07:07:50,433][20381] Updated weights for policy 1, policy_version 213322 (0.0011) [2023-12-27 07:07:50,433][20347] Updated weights for policy 0, policy_version 213671 (0.0008) [2023-12-27 07:07:51,118][20381] Updated weights for policy 1, policy_version 213332 (0.0009) [2023-12-27 07:07:51,124][20347] Updated weights for policy 0, policy_version 213681 (0.0007) [2023-12-27 07:07:51,175][20381] Updated weights for policy 1, policy_version 213342 (0.0007) [2023-12-27 07:07:51,185][20347] Updated weights for policy 0, policy_version 213691 (0.0007) [2023-12-27 07:07:51,233][20381] Updated weights for policy 1, policy_version 213352 (0.0006) [2023-12-27 07:07:51,240][20347] Updated weights for policy 0, policy_version 213701 (0.0008) [2023-12-27 07:07:51,922][20381] Updated weights for policy 1, policy_version 213362 (0.0007) [2023-12-27 07:07:51,970][20347] Updated weights for policy 0, policy_version 213711 (0.0007) [2023-12-27 07:07:51,976][20381] Updated weights for policy 1, policy_version 213372 (0.0006) [2023-12-27 07:07:52,023][20381] Updated weights for policy 1, policy_version 213382 (0.0005) [2023-12-27 07:07:52,025][20347] Updated weights for policy 0, policy_version 213721 (0.0006) [2023-12-27 07:07:52,073][20381] Updated weights for policy 1, policy_version 213392 (0.0006) [2023-12-27 07:07:52,074][20347] Updated weights for policy 0, policy_version 213731 (0.0006) [2023-12-27 07:07:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 109371392. Throughput: 0: 10389.5, 1: 10375.5. Samples: 109339064. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:52,363][19170] Avg episode reward: [(0, '749.110'), (1, '1000.000')] [2023-12-27 07:07:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000213392_54640640.pth... [2023-12-27 07:07:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000213736_54730752.pth... [2023-12-27 07:07:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000212176_54329344.pth [2023-12-27 07:07:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000212520_54419456.pth [2023-12-27 07:07:52,771][20381] Updated weights for policy 1, policy_version 213402 (0.0008) [2023-12-27 07:07:52,788][20347] Updated weights for policy 0, policy_version 213741 (0.0008) [2023-12-27 07:07:52,830][20381] Updated weights for policy 1, policy_version 213412 (0.0008) [2023-12-27 07:07:52,844][20347] Updated weights for policy 0, policy_version 213751 (0.0006) [2023-12-27 07:07:52,877][20381] Updated weights for policy 1, policy_version 213422 (0.0008) [2023-12-27 07:07:52,899][20347] Updated weights for policy 0, policy_version 213761 (0.0008) [2023-12-27 07:07:53,546][20381] Updated weights for policy 1, policy_version 213432 (0.0008) [2023-12-27 07:07:53,560][20347] Updated weights for policy 0, policy_version 213771 (0.0005) [2023-12-27 07:07:53,601][20381] Updated weights for policy 1, policy_version 213442 (0.0007) [2023-12-27 07:07:53,606][20347] Updated weights for policy 0, policy_version 213781 (0.0007) [2023-12-27 07:07:53,645][20381] Updated weights for policy 1, policy_version 213452 (0.0007) [2023-12-27 07:07:53,661][20347] Updated weights for policy 0, policy_version 213791 (0.0005) [2023-12-27 07:07:54,323][20347] Updated weights for policy 0, policy_version 213801 (0.0007) [2023-12-27 07:07:54,368][20381] Updated weights for policy 1, policy_version 213462 (0.0007) [2023-12-27 07:07:54,378][20347] Updated weights for policy 0, policy_version 213811 (0.0007) [2023-12-27 07:07:54,427][20381] Updated weights for policy 1, policy_version 213472 (0.0007) [2023-12-27 07:07:54,434][20347] Updated weights for policy 0, policy_version 213821 (0.0005) [2023-12-27 07:07:54,476][20381] Updated weights for policy 1, policy_version 213482 (0.0008) [2023-12-27 07:07:54,491][20347] Updated weights for policy 0, policy_version 213831 (0.0005) [2023-12-27 07:07:55,110][20347] Updated weights for policy 0, policy_version 213841 (0.0006) [2023-12-27 07:07:55,170][20347] Updated weights for policy 0, policy_version 213851 (0.0006) [2023-12-27 07:07:55,191][20381] Updated weights for policy 1, policy_version 213492 (0.0008) [2023-12-27 07:07:55,231][20347] Updated weights for policy 0, policy_version 213861 (0.0005) [2023-12-27 07:07:55,244][20381] Updated weights for policy 1, policy_version 213502 (0.0007) [2023-12-27 07:07:55,296][20381] Updated weights for policy 1, policy_version 213512 (0.0009) [2023-12-27 07:07:55,805][20347] Updated weights for policy 0, policy_version 213871 (0.0007) [2023-12-27 07:07:55,860][20347] Updated weights for policy 0, policy_version 213881 (0.0008) [2023-12-27 07:07:55,915][20347] Updated weights for policy 0, policy_version 213891 (0.0007) [2023-12-27 07:07:56,006][20381] Updated weights for policy 1, policy_version 213522 (0.0009) [2023-12-27 07:07:56,059][20381] Updated weights for policy 1, policy_version 213532 (0.0006) [2023-12-27 07:07:56,108][20381] Updated weights for policy 1, policy_version 213542 (0.0009) [2023-12-27 07:07:56,160][20381] Updated weights for policy 1, policy_version 213552 (0.0009) [2023-12-27 07:07:56,619][20347] Updated weights for policy 0, policy_version 213901 (0.0009) [2023-12-27 07:07:56,663][20347] Updated weights for policy 0, policy_version 213911 (0.0010) [2023-12-27 07:07:56,711][20347] Updated weights for policy 0, policy_version 213921 (0.0010) [2023-12-27 07:07:56,759][20381] Updated weights for policy 1, policy_version 213562 (0.0010) [2023-12-27 07:07:56,807][20381] Updated weights for policy 1, policy_version 213572 (0.0010) [2023-12-27 07:07:56,865][20381] Updated weights for policy 1, policy_version 213582 (0.0010) [2023-12-27 07:07:57,345][20347] Updated weights for policy 0, policy_version 213931 (0.0007) [2023-12-27 07:07:57,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 109469696. Throughput: 0: 10352.9, 1: 10370.9. Samples: 109461100. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:07:57,364][19170] Avg episode reward: [(0, '933.700'), (1, '1000.000')] [2023-12-27 07:07:57,392][20347] Updated weights for policy 0, policy_version 213941 (0.0006) [2023-12-27 07:07:57,434][20347] Updated weights for policy 0, policy_version 213951 (0.0008) [2023-12-27 07:07:57,524][20381] Updated weights for policy 1, policy_version 213592 (0.0006) [2023-12-27 07:07:57,578][20381] Updated weights for policy 1, policy_version 213602 (0.0006) [2023-12-27 07:07:57,626][20381] Updated weights for policy 1, policy_version 213612 (0.0010) [2023-12-27 07:07:58,065][20347] Updated weights for policy 0, policy_version 213961 (0.0008) [2023-12-27 07:07:58,110][20347] Updated weights for policy 0, policy_version 213971 (0.0010) [2023-12-27 07:07:58,163][20347] Updated weights for policy 0, policy_version 213981 (0.0011) [2023-12-27 07:07:58,225][20347] Updated weights for policy 0, policy_version 213991 (0.0010) [2023-12-27 07:07:58,271][20381] Updated weights for policy 1, policy_version 213622 (0.0008) [2023-12-27 07:07:58,332][20381] Updated weights for policy 1, policy_version 213632 (0.0007) [2023-12-27 07:07:58,392][20381] Updated weights for policy 1, policy_version 213642 (0.0008) [2023-12-27 07:07:58,973][20347] Updated weights for policy 0, policy_version 214001 (0.0008) [2023-12-27 07:07:59,031][20347] Updated weights for policy 0, policy_version 214011 (0.0008) [2023-12-27 07:07:59,081][20347] Updated weights for policy 0, policy_version 214021 (0.0007) [2023-12-27 07:07:59,107][20381] Updated weights for policy 1, policy_version 213652 (0.0008) [2023-12-27 07:07:59,168][20381] Updated weights for policy 1, policy_version 213662 (0.0010) [2023-12-27 07:07:59,229][20381] Updated weights for policy 1, policy_version 213672 (0.0010) [2023-12-27 07:07:59,806][20347] Updated weights for policy 0, policy_version 214031 (0.0009) [2023-12-27 07:07:59,874][20347] Updated weights for policy 0, policy_version 214041 (0.0008) [2023-12-27 07:07:59,932][20347] Updated weights for policy 0, policy_version 214051 (0.0010) [2023-12-27 07:07:59,937][20381] Updated weights for policy 1, policy_version 213682 (0.0008) [2023-12-27 07:08:00,003][20381] Updated weights for policy 1, policy_version 213692 (0.0009) [2023-12-27 07:08:00,054][20381] Updated weights for policy 1, policy_version 213702 (0.0008) [2023-12-27 07:08:00,109][20381] Updated weights for policy 1, policy_version 213712 (0.0008) [2023-12-27 07:08:00,545][20347] Updated weights for policy 0, policy_version 214061 (0.0005) [2023-12-27 07:08:00,607][20347] Updated weights for policy 0, policy_version 214071 (0.0010) [2023-12-27 07:08:00,661][20347] Updated weights for policy 0, policy_version 214081 (0.0010) [2023-12-27 07:08:00,843][20381] Updated weights for policy 1, policy_version 213722 (0.0008) [2023-12-27 07:08:00,888][20381] Updated weights for policy 1, policy_version 213732 (0.0008) [2023-12-27 07:08:00,936][20381] Updated weights for policy 1, policy_version 213742 (0.0008) [2023-12-27 07:08:01,279][20347] Updated weights for policy 0, policy_version 214091 (0.0009) [2023-12-27 07:08:01,334][20347] Updated weights for policy 0, policy_version 214101 (0.0006) [2023-12-27 07:08:01,399][20347] Updated weights for policy 0, policy_version 214111 (0.0008) [2023-12-27 07:08:01,665][20381] Updated weights for policy 1, policy_version 213752 (0.0007) [2023-12-27 07:08:01,733][20381] Updated weights for policy 1, policy_version 213762 (0.0008) [2023-12-27 07:08:01,786][20381] Updated weights for policy 1, policy_version 213772 (0.0008) [2023-12-27 07:08:02,127][20347] Updated weights for policy 0, policy_version 214121 (0.0008) [2023-12-27 07:08:02,186][20347] Updated weights for policy 0, policy_version 214131 (0.0007) [2023-12-27 07:08:02,243][20347] Updated weights for policy 0, policy_version 214141 (0.0005) [2023-12-27 07:08:02,303][20347] Updated weights for policy 0, policy_version 214151 (0.0008) [2023-12-27 07:08:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 109576192. Throughput: 0: 10384.4, 1: 10295.8. Samples: 109584888. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:08:02,363][19170] Avg episode reward: [(0, '957.970'), (1, '1000.000')] [2023-12-27 07:08:02,472][20381] Updated weights for policy 1, policy_version 213782 (0.0007) [2023-12-27 07:08:02,524][20381] Updated weights for policy 1, policy_version 213792 (0.0007) [2023-12-27 07:08:02,579][20381] Updated weights for policy 1, policy_version 213802 (0.0008) [2023-12-27 07:08:03,001][20347] Updated weights for policy 0, policy_version 214161 (0.0010) [2023-12-27 07:08:03,056][20347] Updated weights for policy 0, policy_version 214171 (0.0010) [2023-12-27 07:08:03,105][20347] Updated weights for policy 0, policy_version 214181 (0.0010) [2023-12-27 07:08:03,223][20381] Updated weights for policy 1, policy_version 213812 (0.0007) [2023-12-27 07:08:03,285][20381] Updated weights for policy 1, policy_version 213822 (0.0008) [2023-12-27 07:08:03,343][20381] Updated weights for policy 1, policy_version 213832 (0.0010) [2023-12-27 07:08:03,760][20347] Updated weights for policy 0, policy_version 214191 (0.0006) [2023-12-27 07:08:03,814][20347] Updated weights for policy 0, policy_version 214201 (0.0005) [2023-12-27 07:08:03,875][20347] Updated weights for policy 0, policy_version 214211 (0.0007) [2023-12-27 07:08:03,995][20381] Updated weights for policy 1, policy_version 213842 (0.0010) [2023-12-27 07:08:04,060][20381] Updated weights for policy 1, policy_version 213852 (0.0008) [2023-12-27 07:08:04,115][20381] Updated weights for policy 1, policy_version 213862 (0.0008) [2023-12-27 07:08:04,171][20381] Updated weights for policy 1, policy_version 213872 (0.0008) [2023-12-27 07:08:04,564][20347] Updated weights for policy 0, policy_version 214221 (0.0008) [2023-12-27 07:08:04,624][20347] Updated weights for policy 0, policy_version 214231 (0.0007) [2023-12-27 07:08:04,670][20347] Updated weights for policy 0, policy_version 214241 (0.0010) [2023-12-27 07:08:04,819][20381] Updated weights for policy 1, policy_version 213882 (0.0007) [2023-12-27 07:08:04,882][20381] Updated weights for policy 1, policy_version 213892 (0.0006) [2023-12-27 07:08:04,935][20381] Updated weights for policy 1, policy_version 213902 (0.0006) [2023-12-27 07:08:05,425][20347] Updated weights for policy 0, policy_version 214251 (0.0009) [2023-12-27 07:08:05,478][20381] Updated weights for policy 1, policy_version 213912 (0.0006) [2023-12-27 07:08:05,481][20347] Updated weights for policy 0, policy_version 214261 (0.0008) [2023-12-27 07:08:05,526][20381] Updated weights for policy 1, policy_version 213922 (0.0005) [2023-12-27 07:08:05,529][20347] Updated weights for policy 0, policy_version 214271 (0.0007) [2023-12-27 07:08:05,581][20381] Updated weights for policy 1, policy_version 213932 (0.0006) [2023-12-27 07:08:06,197][20347] Updated weights for policy 0, policy_version 214281 (0.0007) [2023-12-27 07:08:06,204][20381] Updated weights for policy 1, policy_version 213942 (0.0006) [2023-12-27 07:08:06,254][20381] Updated weights for policy 1, policy_version 213952 (0.0007) [2023-12-27 07:08:06,260][20347] Updated weights for policy 0, policy_version 214291 (0.0008) [2023-12-27 07:08:06,308][20381] Updated weights for policy 1, policy_version 213962 (0.0006) [2023-12-27 07:08:06,317][20347] Updated weights for policy 0, policy_version 214301 (0.0006) [2023-12-27 07:08:06,377][20347] Updated weights for policy 0, policy_version 214311 (0.0007) [2023-12-27 07:08:07,002][20381] Updated weights for policy 1, policy_version 213972 (0.0007) [2023-12-27 07:08:07,047][20381] Updated weights for policy 1, policy_version 213982 (0.0007) [2023-12-27 07:08:07,076][20347] Updated weights for policy 0, policy_version 214321 (0.0006) [2023-12-27 07:08:07,094][20381] Updated weights for policy 1, policy_version 213992 (0.0006) [2023-12-27 07:08:07,124][20347] Updated weights for policy 0, policy_version 214331 (0.0005) [2023-12-27 07:08:07,177][20347] Updated weights for policy 0, policy_version 214341 (0.0007) [2023-12-27 07:08:07,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 109682688. Throughput: 0: 10378.6, 1: 10290.2. Samples: 109647280. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 07:08:07,364][19170] Avg episode reward: [(0, '885.660'), (1, '1000.000')] [2023-12-27 07:08:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000214000_54796288.pth... [2023-12-27 07:08:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000214344_54886400.pth... [2023-12-27 07:08:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000212784_54484992.pth [2023-12-27 07:08:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000213128_54575104.pth [2023-12-27 07:08:07,781][20381] Updated weights for policy 1, policy_version 214002 (0.0006) [2023-12-27 07:08:07,829][20381] Updated weights for policy 1, policy_version 214012 (0.0007) [2023-12-27 07:08:07,880][20381] Updated weights for policy 1, policy_version 214022 (0.0007) [2023-12-27 07:08:07,900][20347] Updated weights for policy 0, policy_version 214351 (0.0008) [2023-12-27 07:08:07,932][20381] Updated weights for policy 1, policy_version 214032 (0.0006) [2023-12-27 07:08:07,960][20347] Updated weights for policy 0, policy_version 214361 (0.0010) [2023-12-27 07:08:08,017][20347] Updated weights for policy 0, policy_version 214371 (0.0010) [2023-12-27 07:08:08,612][20381] Updated weights for policy 1, policy_version 214042 (0.0007) [2023-12-27 07:08:08,670][20381] Updated weights for policy 1, policy_version 214052 (0.0006) [2023-12-27 07:08:08,700][20347] Updated weights for policy 0, policy_version 214381 (0.0010) [2023-12-27 07:08:08,727][20381] Updated weights for policy 1, policy_version 214062 (0.0005) [2023-12-27 07:08:08,755][20347] Updated weights for policy 0, policy_version 214391 (0.0010) [2023-12-27 07:08:08,807][20347] Updated weights for policy 0, policy_version 214401 (0.0010) [2023-12-27 07:08:09,378][20381] Updated weights for policy 1, policy_version 214072 (0.0007) [2023-12-27 07:08:09,446][20381] Updated weights for policy 1, policy_version 214082 (0.0007) [2023-12-27 07:08:09,510][20381] Updated weights for policy 1, policy_version 214092 (0.0009) [2023-12-27 07:08:09,521][20347] Updated weights for policy 0, policy_version 214411 (0.0010) [2023-12-27 07:08:09,581][20347] Updated weights for policy 0, policy_version 214421 (0.0010) [2023-12-27 07:08:09,627][20347] Updated weights for policy 0, policy_version 214431 (0.0010) [2023-12-27 07:08:10,230][20381] Updated weights for policy 1, policy_version 214102 (0.0007) [2023-12-27 07:08:10,291][20381] Updated weights for policy 1, policy_version 214112 (0.0008) [2023-12-27 07:08:10,303][20347] Updated weights for policy 0, policy_version 214441 (0.0011) [2023-12-27 07:08:10,347][20381] Updated weights for policy 1, policy_version 214122 (0.0006) [2023-12-27 07:08:10,361][20347] Updated weights for policy 0, policy_version 214451 (0.0010) [2023-12-27 07:08:10,417][20347] Updated weights for policy 0, policy_version 214461 (0.0010) [2023-12-27 07:08:10,476][20347] Updated weights for policy 0, policy_version 214471 (0.0010) [2023-12-27 07:08:11,057][20381] Updated weights for policy 1, policy_version 214132 (0.0007) [2023-12-27 07:08:11,112][20381] Updated weights for policy 1, policy_version 214142 (0.0007) [2023-12-27 07:08:11,142][20347] Updated weights for policy 0, policy_version 214481 (0.0008) [2023-12-27 07:08:11,174][20381] Updated weights for policy 1, policy_version 214152 (0.0008) [2023-12-27 07:08:11,212][20347] Updated weights for policy 0, policy_version 214491 (0.0008) [2023-12-27 07:08:11,278][20347] Updated weights for policy 0, policy_version 214501 (0.0007) [2023-12-27 07:08:11,919][20347] Updated weights for policy 0, policy_version 214511 (0.0008) [2023-12-27 07:08:11,949][20381] Updated weights for policy 1, policy_version 214162 (0.0007) [2023-12-27 07:08:11,980][20347] Updated weights for policy 0, policy_version 214521 (0.0006) [2023-12-27 07:08:12,014][20381] Updated weights for policy 1, policy_version 214172 (0.0009) [2023-12-27 07:08:12,043][20347] Updated weights for policy 0, policy_version 214531 (0.0007) [2023-12-27 07:08:12,072][20381] Updated weights for policy 1, policy_version 214182 (0.0006) [2023-12-27 07:08:12,136][20381] Updated weights for policy 1, policy_version 214192 (0.0008) [2023-12-27 07:08:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 109780992. Throughput: 0: 10290.1, 1: 10296.8. Samples: 109769840. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:12,363][19170] Avg episode reward: [(0, '775.160'), (1, '1000.000')] [2023-12-27 07:08:12,747][20347] Updated weights for policy 0, policy_version 214541 (0.0006) [2023-12-27 07:08:12,800][20347] Updated weights for policy 0, policy_version 214551 (0.0007) [2023-12-27 07:08:12,825][20381] Updated weights for policy 1, policy_version 214202 (0.0008) [2023-12-27 07:08:12,853][20347] Updated weights for policy 0, policy_version 214561 (0.0006) [2023-12-27 07:08:12,877][20381] Updated weights for policy 1, policy_version 214212 (0.0006) [2023-12-27 07:08:12,937][20381] Updated weights for policy 1, policy_version 214222 (0.0008) [2023-12-27 07:08:13,548][20347] Updated weights for policy 0, policy_version 214571 (0.0007) [2023-12-27 07:08:13,565][20381] Updated weights for policy 1, policy_version 214232 (0.0007) [2023-12-27 07:08:13,600][20347] Updated weights for policy 0, policy_version 214581 (0.0005) [2023-12-27 07:08:13,620][20381] Updated weights for policy 1, policy_version 214242 (0.0005) [2023-12-27 07:08:13,651][20347] Updated weights for policy 0, policy_version 214591 (0.0006) [2023-12-27 07:08:13,677][20381] Updated weights for policy 1, policy_version 214252 (0.0009) [2023-12-27 07:08:14,322][20381] Updated weights for policy 1, policy_version 214262 (0.0007) [2023-12-27 07:08:14,340][20347] Updated weights for policy 0, policy_version 214601 (0.0007) [2023-12-27 07:08:14,367][20381] Updated weights for policy 1, policy_version 214272 (0.0006) [2023-12-27 07:08:14,389][20347] Updated weights for policy 0, policy_version 214611 (0.0008) [2023-12-27 07:08:14,419][20381] Updated weights for policy 1, policy_version 214282 (0.0007) [2023-12-27 07:08:14,437][20347] Updated weights for policy 0, policy_version 214621 (0.0005) [2023-12-27 07:08:14,492][20347] Updated weights for policy 0, policy_version 214631 (0.0007) [2023-12-27 07:08:15,086][20381] Updated weights for policy 1, policy_version 214292 (0.0007) [2023-12-27 07:08:15,150][20381] Updated weights for policy 1, policy_version 214302 (0.0006) [2023-12-27 07:08:15,151][20347] Updated weights for policy 0, policy_version 214641 (0.0010) [2023-12-27 07:08:15,205][20381] Updated weights for policy 1, policy_version 214312 (0.0006) [2023-12-27 07:08:15,212][20347] Updated weights for policy 0, policy_version 214651 (0.0010) [2023-12-27 07:08:15,266][20347] Updated weights for policy 0, policy_version 214661 (0.0010) [2023-12-27 07:08:15,845][20381] Updated weights for policy 1, policy_version 214322 (0.0007) [2023-12-27 07:08:15,897][20381] Updated weights for policy 1, policy_version 214332 (0.0007) [2023-12-27 07:08:15,957][20347] Updated weights for policy 0, policy_version 214671 (0.0007) [2023-12-27 07:08:15,963][20381] Updated weights for policy 1, policy_version 214342 (0.0005) [2023-12-27 07:08:16,006][20347] Updated weights for policy 0, policy_version 214681 (0.0005) [2023-12-27 07:08:16,027][20381] Updated weights for policy 1, policy_version 214352 (0.0006) [2023-12-27 07:08:16,054][20347] Updated weights for policy 0, policy_version 214691 (0.0005) [2023-12-27 07:08:16,650][20347] Updated weights for policy 0, policy_version 214701 (0.0010) [2023-12-27 07:08:16,700][20347] Updated weights for policy 0, policy_version 214711 (0.0010) [2023-12-27 07:08:16,721][20381] Updated weights for policy 1, policy_version 214362 (0.0007) [2023-12-27 07:08:16,759][20347] Updated weights for policy 0, policy_version 214721 (0.0010) [2023-12-27 07:08:16,783][20381] Updated weights for policy 1, policy_version 214372 (0.0005) [2023-12-27 07:08:16,839][20381] Updated weights for policy 1, policy_version 214382 (0.0006) [2023-12-27 07:08:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 109879296. Throughput: 0: 10249.4, 1: 10282.0. Samples: 109893168. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:17,363][19170] Avg episode reward: [(0, '718.970'), (1, '1000.000')] [2023-12-27 07:08:17,466][20347] Updated weights for policy 0, policy_version 214731 (0.0011) [2023-12-27 07:08:17,522][20381] Updated weights for policy 1, policy_version 214392 (0.0007) [2023-12-27 07:08:17,526][20347] Updated weights for policy 0, policy_version 214741 (0.0009) [2023-12-27 07:08:17,578][20381] Updated weights for policy 1, policy_version 214402 (0.0008) [2023-12-27 07:08:17,580][20347] Updated weights for policy 0, policy_version 214751 (0.0005) [2023-12-27 07:08:17,635][20381] Updated weights for policy 1, policy_version 214412 (0.0008) [2023-12-27 07:08:18,206][20347] Updated weights for policy 0, policy_version 214761 (0.0007) [2023-12-27 07:08:18,249][20381] Updated weights for policy 1, policy_version 214422 (0.0006) [2023-12-27 07:08:18,255][20347] Updated weights for policy 0, policy_version 214771 (0.0006) [2023-12-27 07:08:18,303][20381] Updated weights for policy 1, policy_version 214432 (0.0007) [2023-12-27 07:08:18,304][20347] Updated weights for policy 0, policy_version 214781 (0.0005) [2023-12-27 07:08:18,365][20381] Updated weights for policy 1, policy_version 214442 (0.0007) [2023-12-27 07:08:18,366][20347] Updated weights for policy 0, policy_version 214791 (0.0006) [2023-12-27 07:08:18,935][20347] Updated weights for policy 0, policy_version 214801 (0.0006) [2023-12-27 07:08:18,992][20347] Updated weights for policy 0, policy_version 214811 (0.0009) [2023-12-27 07:08:19,041][20347] Updated weights for policy 0, policy_version 214821 (0.0006) [2023-12-27 07:08:19,046][20381] Updated weights for policy 1, policy_version 214452 (0.0007) [2023-12-27 07:08:19,094][20381] Updated weights for policy 1, policy_version 214462 (0.0005) [2023-12-27 07:08:19,146][20381] Updated weights for policy 1, policy_version 214472 (0.0008) [2023-12-27 07:08:19,710][20347] Updated weights for policy 0, policy_version 214831 (0.0009) [2023-12-27 07:08:19,772][20347] Updated weights for policy 0, policy_version 214841 (0.0010) [2023-12-27 07:08:19,823][20381] Updated weights for policy 1, policy_version 214482 (0.0007) [2023-12-27 07:08:19,839][20347] Updated weights for policy 0, policy_version 214851 (0.0008) [2023-12-27 07:08:19,893][20381] Updated weights for policy 1, policy_version 214492 (0.0008) [2023-12-27 07:08:19,960][20381] Updated weights for policy 1, policy_version 214502 (0.0008) [2023-12-27 07:08:20,022][20381] Updated weights for policy 1, policy_version 214512 (0.0008) [2023-12-27 07:08:20,546][20347] Updated weights for policy 0, policy_version 214861 (0.0007) [2023-12-27 07:08:20,603][20347] Updated weights for policy 0, policy_version 214871 (0.0008) [2023-12-27 07:08:20,658][20347] Updated weights for policy 0, policy_version 214881 (0.0007) [2023-12-27 07:08:20,719][20381] Updated weights for policy 1, policy_version 214522 (0.0006) [2023-12-27 07:08:20,773][20381] Updated weights for policy 1, policy_version 214532 (0.0007) [2023-12-27 07:08:20,822][20381] Updated weights for policy 1, policy_version 214542 (0.0008) [2023-12-27 07:08:21,335][20347] Updated weights for policy 0, policy_version 214891 (0.0007) [2023-12-27 07:08:21,401][20347] Updated weights for policy 0, policy_version 214901 (0.0008) [2023-12-27 07:08:21,462][20347] Updated weights for policy 0, policy_version 214911 (0.0009) [2023-12-27 07:08:21,536][20381] Updated weights for policy 1, policy_version 214552 (0.0007) [2023-12-27 07:08:21,586][20381] Updated weights for policy 1, policy_version 214562 (0.0010) [2023-12-27 07:08:21,650][20381] Updated weights for policy 1, policy_version 214572 (0.0007) [2023-12-27 07:08:22,166][20347] Updated weights for policy 0, policy_version 214921 (0.0007) [2023-12-27 07:08:22,229][20347] Updated weights for policy 0, policy_version 214931 (0.0008) [2023-12-27 07:08:22,289][20347] Updated weights for policy 0, policy_version 214941 (0.0007) [2023-12-27 07:08:22,354][20347] Updated weights for policy 0, policy_version 214951 (0.0007) [2023-12-27 07:08:22,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 109985792. Throughput: 0: 10268.7, 1: 10281.0. Samples: 109956248. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:22,363][19170] Avg episode reward: [(0, '856.310'), (1, '1000.000')] [2023-12-27 07:08:22,366][20381] Updated weights for policy 1, policy_version 214582 (0.0008) [2023-12-27 07:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000214952_55042048.pth... [2023-12-27 07:08:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000213736_54730752.pth [2023-12-27 07:08:22,429][20381] Updated weights for policy 1, policy_version 214592 (0.0008) [2023-12-27 07:08:22,491][20381] Updated weights for policy 1, policy_version 214602 (0.0008) [2023-12-27 07:08:22,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000214608_54951936.pth... [2023-12-27 07:08:22,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000213392_54640640.pth [2023-12-27 07:08:22,993][20347] Updated weights for policy 0, policy_version 214961 (0.0007) [2023-12-27 07:08:23,053][20347] Updated weights for policy 0, policy_version 214971 (0.0008) [2023-12-27 07:08:23,112][20347] Updated weights for policy 0, policy_version 214981 (0.0008) [2023-12-27 07:08:23,156][20381] Updated weights for policy 1, policy_version 214612 (0.0008) [2023-12-27 07:08:23,211][20381] Updated weights for policy 1, policy_version 214622 (0.0008) [2023-12-27 07:08:23,260][20381] Updated weights for policy 1, policy_version 214632 (0.0007) [2023-12-27 07:08:23,838][20347] Updated weights for policy 0, policy_version 214991 (0.0007) [2023-12-27 07:08:23,880][20381] Updated weights for policy 1, policy_version 214642 (0.0008) [2023-12-27 07:08:23,897][20347] Updated weights for policy 0, policy_version 215001 (0.0007) [2023-12-27 07:08:23,925][20381] Updated weights for policy 1, policy_version 214652 (0.0005) [2023-12-27 07:08:23,948][20347] Updated weights for policy 0, policy_version 215011 (0.0007) [2023-12-27 07:08:23,968][20381] Updated weights for policy 1, policy_version 214662 (0.0007) [2023-12-27 07:08:24,017][20381] Updated weights for policy 1, policy_version 214672 (0.0006) [2023-12-27 07:08:24,658][20347] Updated weights for policy 0, policy_version 215021 (0.0007) [2023-12-27 07:08:24,659][20381] Updated weights for policy 1, policy_version 214682 (0.0007) [2023-12-27 07:08:24,711][20347] Updated weights for policy 0, policy_version 215031 (0.0007) [2023-12-27 07:08:24,722][20381] Updated weights for policy 1, policy_version 214692 (0.0006) [2023-12-27 07:08:24,766][20347] Updated weights for policy 0, policy_version 215041 (0.0007) [2023-12-27 07:08:24,782][20381] Updated weights for policy 1, policy_version 214702 (0.0006) [2023-12-27 07:08:25,360][20347] Updated weights for policy 0, policy_version 215051 (0.0007) [2023-12-27 07:08:25,417][20347] Updated weights for policy 0, policy_version 215061 (0.0006) [2023-12-27 07:08:25,434][20381] Updated weights for policy 1, policy_version 214712 (0.0008) [2023-12-27 07:08:25,472][20347] Updated weights for policy 0, policy_version 215071 (0.0005) [2023-12-27 07:08:25,491][20381] Updated weights for policy 1, policy_version 214722 (0.0006) [2023-12-27 07:08:25,540][20381] Updated weights for policy 1, policy_version 214732 (0.0008) [2023-12-27 07:08:26,100][20347] Updated weights for policy 0, policy_version 215081 (0.0008) [2023-12-27 07:08:26,147][20347] Updated weights for policy 0, policy_version 215091 (0.0005) [2023-12-27 07:08:26,195][20347] Updated weights for policy 0, policy_version 215101 (0.0006) [2023-12-27 07:08:26,218][20381] Updated weights for policy 1, policy_version 214742 (0.0007) [2023-12-27 07:08:26,252][20347] Updated weights for policy 0, policy_version 215111 (0.0006) [2023-12-27 07:08:26,273][20381] Updated weights for policy 1, policy_version 214752 (0.0007) [2023-12-27 07:08:26,324][20381] Updated weights for policy 1, policy_version 214762 (0.0006) [2023-12-27 07:08:26,860][20347] Updated weights for policy 0, policy_version 215121 (0.0009) [2023-12-27 07:08:26,913][20347] Updated weights for policy 0, policy_version 215131 (0.0010) [2023-12-27 07:08:26,961][20347] Updated weights for policy 0, policy_version 215141 (0.0006) [2023-12-27 07:08:26,974][20381] Updated weights for policy 1, policy_version 214772 (0.0007) [2023-12-27 07:08:27,029][20381] Updated weights for policy 1, policy_version 214782 (0.0007) [2023-12-27 07:08:27,077][20381] Updated weights for policy 1, policy_version 214792 (0.0010) [2023-12-27 07:08:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 110092288. Throughput: 0: 10273.8, 1: 10286.6. Samples: 110080336. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:27,363][19170] Avg episode reward: [(0, '987.260'), (1, '1000.000')] [2023-12-27 07:08:27,567][20347] Updated weights for policy 0, policy_version 215151 (0.0009) [2023-12-27 07:08:27,620][20347] Updated weights for policy 0, policy_version 215161 (0.0005) [2023-12-27 07:08:27,674][20347] Updated weights for policy 0, policy_version 215171 (0.0005) [2023-12-27 07:08:27,724][20381] Updated weights for policy 1, policy_version 214802 (0.0009) [2023-12-27 07:08:27,774][20381] Updated weights for policy 1, policy_version 214812 (0.0007) [2023-12-27 07:08:27,833][20381] Updated weights for policy 1, policy_version 214822 (0.0008) [2023-12-27 07:08:27,896][20381] Updated weights for policy 1, policy_version 214832 (0.0008) [2023-12-27 07:08:28,218][20347] Updated weights for policy 0, policy_version 215181 (0.0005) [2023-12-27 07:08:28,274][20347] Updated weights for policy 0, policy_version 215191 (0.0005) [2023-12-27 07:08:28,329][20347] Updated weights for policy 0, policy_version 215201 (0.0006) [2023-12-27 07:08:28,652][20381] Updated weights for policy 1, policy_version 214842 (0.0008) [2023-12-27 07:08:28,701][20381] Updated weights for policy 1, policy_version 214852 (0.0007) [2023-12-27 07:08:28,756][20381] Updated weights for policy 1, policy_version 214862 (0.0007) [2023-12-27 07:08:28,941][20347] Updated weights for policy 0, policy_version 215211 (0.0009) [2023-12-27 07:08:28,989][20347] Updated weights for policy 0, policy_version 215221 (0.0010) [2023-12-27 07:08:29,042][20347] Updated weights for policy 0, policy_version 215231 (0.0010) [2023-12-27 07:08:29,484][20381] Updated weights for policy 1, policy_version 214872 (0.0007) [2023-12-27 07:08:29,543][20381] Updated weights for policy 1, policy_version 214882 (0.0008) [2023-12-27 07:08:29,599][20381] Updated weights for policy 1, policy_version 214892 (0.0008) [2023-12-27 07:08:29,758][20347] Updated weights for policy 0, policy_version 215241 (0.0010) [2023-12-27 07:08:29,812][20347] Updated weights for policy 0, policy_version 215251 (0.0007) [2023-12-27 07:08:29,872][20347] Updated weights for policy 0, policy_version 215261 (0.0008) [2023-12-27 07:08:29,931][20347] Updated weights for policy 0, policy_version 215271 (0.0007) [2023-12-27 07:08:30,349][20381] Updated weights for policy 1, policy_version 214902 (0.0008) [2023-12-27 07:08:30,408][20381] Updated weights for policy 1, policy_version 214912 (0.0007) [2023-12-27 07:08:30,463][20381] Updated weights for policy 1, policy_version 214922 (0.0008) [2023-12-27 07:08:30,594][20347] Updated weights for policy 0, policy_version 215281 (0.0010) [2023-12-27 07:08:30,654][20347] Updated weights for policy 0, policy_version 215291 (0.0009) [2023-12-27 07:08:30,720][20347] Updated weights for policy 0, policy_version 215301 (0.0005) [2023-12-27 07:08:31,028][20381] Updated weights for policy 1, policy_version 214932 (0.0008) [2023-12-27 07:08:31,094][20381] Updated weights for policy 1, policy_version 214942 (0.0007) [2023-12-27 07:08:31,156][20381] Updated weights for policy 1, policy_version 214952 (0.0009) [2023-12-27 07:08:31,349][20347] Updated weights for policy 0, policy_version 215311 (0.0007) [2023-12-27 07:08:31,414][20347] Updated weights for policy 0, policy_version 215321 (0.0008) [2023-12-27 07:08:31,473][20347] Updated weights for policy 0, policy_version 215331 (0.0007) [2023-12-27 07:08:31,873][20381] Updated weights for policy 1, policy_version 214962 (0.0008) [2023-12-27 07:08:31,933][20381] Updated weights for policy 1, policy_version 214972 (0.0011) [2023-12-27 07:08:31,988][20381] Updated weights for policy 1, policy_version 214982 (0.0010) [2023-12-27 07:08:32,037][20381] Updated weights for policy 1, policy_version 214992 (0.0010) [2023-12-27 07:08:32,165][20347] Updated weights for policy 0, policy_version 215341 (0.0009) [2023-12-27 07:08:32,213][20347] Updated weights for policy 0, policy_version 215351 (0.0010) [2023-12-27 07:08:32,272][20347] Updated weights for policy 0, policy_version 215361 (0.0010) [2023-12-27 07:08:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 110198784. Throughput: 0: 10365.9, 1: 10255.5. Samples: 110205664. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:32,363][19170] Avg episode reward: [(0, '991.810'), (1, '1000.000')] [2023-12-27 07:08:32,691][20381] Updated weights for policy 1, policy_version 215002 (0.0010) [2023-12-27 07:08:32,754][20381] Updated weights for policy 1, policy_version 215012 (0.0007) [2023-12-27 07:08:32,809][20381] Updated weights for policy 1, policy_version 215022 (0.0010) [2023-12-27 07:08:32,922][20347] Updated weights for policy 0, policy_version 215371 (0.0009) [2023-12-27 07:08:32,989][20347] Updated weights for policy 0, policy_version 215381 (0.0006) [2023-12-27 07:08:33,048][20347] Updated weights for policy 0, policy_version 215391 (0.0010) [2023-12-27 07:08:33,465][20381] Updated weights for policy 1, policy_version 215032 (0.0010) [2023-12-27 07:08:33,519][20381] Updated weights for policy 1, policy_version 215042 (0.0010) [2023-12-27 07:08:33,573][20381] Updated weights for policy 1, policy_version 215052 (0.0010) [2023-12-27 07:08:33,711][20347] Updated weights for policy 0, policy_version 215401 (0.0010) [2023-12-27 07:08:33,768][20347] Updated weights for policy 0, policy_version 215411 (0.0006) [2023-12-27 07:08:33,820][20347] Updated weights for policy 0, policy_version 215421 (0.0006) [2023-12-27 07:08:33,879][20347] Updated weights for policy 0, policy_version 215431 (0.0005) [2023-12-27 07:08:34,235][20381] Updated weights for policy 1, policy_version 215062 (0.0009) [2023-12-27 07:08:34,285][20381] Updated weights for policy 1, policy_version 215072 (0.0008) [2023-12-27 07:08:34,331][20381] Updated weights for policy 1, policy_version 215082 (0.0007) [2023-12-27 07:08:34,503][20347] Updated weights for policy 0, policy_version 215441 (0.0008) [2023-12-27 07:08:34,550][20347] Updated weights for policy 0, policy_version 215451 (0.0008) [2023-12-27 07:08:34,604][20347] Updated weights for policy 0, policy_version 215461 (0.0008) [2023-12-27 07:08:35,038][20381] Updated weights for policy 1, policy_version 215092 (0.0007) [2023-12-27 07:08:35,084][20381] Updated weights for policy 1, policy_version 215102 (0.0007) [2023-12-27 07:08:35,138][20381] Updated weights for policy 1, policy_version 215112 (0.0005) [2023-12-27 07:08:35,283][20347] Updated weights for policy 0, policy_version 215471 (0.0007) [2023-12-27 07:08:35,345][20347] Updated weights for policy 0, policy_version 215481 (0.0006) [2023-12-27 07:08:35,397][20347] Updated weights for policy 0, policy_version 215491 (0.0005) [2023-12-27 07:08:35,773][20381] Updated weights for policy 1, policy_version 215122 (0.0006) [2023-12-27 07:08:35,827][20381] Updated weights for policy 1, policy_version 215132 (0.0007) [2023-12-27 07:08:35,885][20381] Updated weights for policy 1, policy_version 215142 (0.0006) [2023-12-27 07:08:35,935][20381] Updated weights for policy 1, policy_version 215152 (0.0007) [2023-12-27 07:08:36,046][20347] Updated weights for policy 0, policy_version 215501 (0.0008) [2023-12-27 07:08:36,106][20347] Updated weights for policy 0, policy_version 215511 (0.0007) [2023-12-27 07:08:36,169][20347] Updated weights for policy 0, policy_version 215521 (0.0008) [2023-12-27 07:08:36,657][20381] Updated weights for policy 1, policy_version 215162 (0.0007) [2023-12-27 07:08:36,705][20381] Updated weights for policy 1, policy_version 215172 (0.0008) [2023-12-27 07:08:36,749][20381] Updated weights for policy 1, policy_version 215182 (0.0007) [2023-12-27 07:08:36,765][20347] Updated weights for policy 0, policy_version 215531 (0.0009) [2023-12-27 07:08:36,820][20347] Updated weights for policy 0, policy_version 215541 (0.0010) [2023-12-27 07:08:36,877][20347] Updated weights for policy 0, policy_version 215551 (0.0008) [2023-12-27 07:08:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 110297088. Throughput: 0: 10373.4, 1: 10274.9. Samples: 110268240. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:37,363][19170] Avg episode reward: [(0, '996.670'), (1, '1000.000')] [2023-12-27 07:08:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000215184_55099392.pth... [2023-12-27 07:08:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000215560_55197696.pth... [2023-12-27 07:08:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000214000_54796288.pth [2023-12-27 07:08:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000214344_54886400.pth [2023-12-27 07:08:37,450][20381] Updated weights for policy 1, policy_version 215192 (0.0007) [2023-12-27 07:08:37,501][20381] Updated weights for policy 1, policy_version 215202 (0.0007) [2023-12-27 07:08:37,506][20347] Updated weights for policy 0, policy_version 215561 (0.0006) [2023-12-27 07:08:37,556][20381] Updated weights for policy 1, policy_version 215212 (0.0008) [2023-12-27 07:08:37,564][20347] Updated weights for policy 0, policy_version 215571 (0.0007) [2023-12-27 07:08:37,618][20347] Updated weights for policy 0, policy_version 215581 (0.0009) [2023-12-27 07:08:37,679][20347] Updated weights for policy 0, policy_version 215591 (0.0010) [2023-12-27 07:08:38,212][20381] Updated weights for policy 1, policy_version 215222 (0.0006) [2023-12-27 07:08:38,267][20381] Updated weights for policy 1, policy_version 215232 (0.0007) [2023-12-27 07:08:38,318][20347] Updated weights for policy 0, policy_version 215601 (0.0006) [2023-12-27 07:08:38,321][20381] Updated weights for policy 1, policy_version 215242 (0.0011) [2023-12-27 07:08:38,390][20347] Updated weights for policy 0, policy_version 215611 (0.0008) [2023-12-27 07:08:38,450][20347] Updated weights for policy 0, policy_version 215621 (0.0010) [2023-12-27 07:08:38,967][20381] Updated weights for policy 1, policy_version 215252 (0.0009) [2023-12-27 07:08:39,023][20381] Updated weights for policy 1, policy_version 215262 (0.0006) [2023-12-27 07:08:39,047][20347] Updated weights for policy 0, policy_version 215631 (0.0009) [2023-12-27 07:08:39,083][20381] Updated weights for policy 1, policy_version 215272 (0.0008) [2023-12-27 07:08:39,108][20347] Updated weights for policy 0, policy_version 215641 (0.0010) [2023-12-27 07:08:39,166][20347] Updated weights for policy 0, policy_version 215651 (0.0010) [2023-12-27 07:08:39,729][20381] Updated weights for policy 1, policy_version 215282 (0.0006) [2023-12-27 07:08:39,782][20381] Updated weights for policy 1, policy_version 215292 (0.0007) [2023-12-27 07:08:39,842][20381] Updated weights for policy 1, policy_version 215302 (0.0008) [2023-12-27 07:08:39,855][20347] Updated weights for policy 0, policy_version 215661 (0.0011) [2023-12-27 07:08:39,901][20381] Updated weights for policy 1, policy_version 215312 (0.0007) [2023-12-27 07:08:39,915][20347] Updated weights for policy 0, policy_version 215671 (0.0011) [2023-12-27 07:08:39,983][20347] Updated weights for policy 0, policy_version 215681 (0.0011) [2023-12-27 07:08:40,637][20381] Updated weights for policy 1, policy_version 215322 (0.0008) [2023-12-27 07:08:40,667][20347] Updated weights for policy 0, policy_version 215691 (0.0010) [2023-12-27 07:08:40,700][20381] Updated weights for policy 1, policy_version 215332 (0.0006) [2023-12-27 07:08:40,728][20347] Updated weights for policy 0, policy_version 215701 (0.0010) [2023-12-27 07:08:40,756][20381] Updated weights for policy 1, policy_version 215342 (0.0008) [2023-12-27 07:08:40,785][20347] Updated weights for policy 0, policy_version 215711 (0.0005) [2023-12-27 07:08:41,421][20347] Updated weights for policy 0, policy_version 215721 (0.0005) [2023-12-27 07:08:41,458][20381] Updated weights for policy 1, policy_version 215352 (0.0007) [2023-12-27 07:08:41,475][20347] Updated weights for policy 0, policy_version 215731 (0.0008) [2023-12-27 07:08:41,519][20381] Updated weights for policy 1, policy_version 215362 (0.0007) [2023-12-27 07:08:41,531][20347] Updated weights for policy 0, policy_version 215741 (0.0007) [2023-12-27 07:08:41,579][20381] Updated weights for policy 1, policy_version 215372 (0.0007) [2023-12-27 07:08:41,589][20347] Updated weights for policy 0, policy_version 215751 (0.0009) [2023-12-27 07:08:42,266][20347] Updated weights for policy 0, policy_version 215761 (0.0008) [2023-12-27 07:08:42,319][20381] Updated weights for policy 1, policy_version 215382 (0.0007) [2023-12-27 07:08:42,322][20347] Updated weights for policy 0, policy_version 215771 (0.0008) [2023-12-27 07:08:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 110395392. Throughput: 0: 10429.6, 1: 10290.8. Samples: 110393512. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:08:42,379][20381] Updated weights for policy 1, policy_version 215392 (0.0007) [2023-12-27 07:08:42,389][20347] Updated weights for policy 0, policy_version 215781 (0.0008) [2023-12-27 07:08:42,432][20381] Updated weights for policy 1, policy_version 215402 (0.0007) [2023-12-27 07:08:43,076][20347] Updated weights for policy 0, policy_version 215791 (0.0008) [2023-12-27 07:08:43,123][20347] Updated weights for policy 0, policy_version 215801 (0.0007) [2023-12-27 07:08:43,146][20381] Updated weights for policy 1, policy_version 215412 (0.0006) [2023-12-27 07:08:43,172][20347] Updated weights for policy 0, policy_version 215811 (0.0005) [2023-12-27 07:08:43,199][20381] Updated weights for policy 1, policy_version 215422 (0.0007) [2023-12-27 07:08:43,258][20381] Updated weights for policy 1, policy_version 215432 (0.0007) [2023-12-27 07:08:43,767][20347] Updated weights for policy 0, policy_version 215821 (0.0007) [2023-12-27 07:08:43,811][20347] Updated weights for policy 0, policy_version 215831 (0.0007) [2023-12-27 07:08:43,868][20347] Updated weights for policy 0, policy_version 215841 (0.0007) [2023-12-27 07:08:43,928][20381] Updated weights for policy 1, policy_version 215442 (0.0007) [2023-12-27 07:08:43,989][20381] Updated weights for policy 1, policy_version 215452 (0.0008) [2023-12-27 07:08:44,041][20381] Updated weights for policy 1, policy_version 215462 (0.0009) [2023-12-27 07:08:44,090][20381] Updated weights for policy 1, policy_version 215472 (0.0008) [2023-12-27 07:08:44,549][20347] Updated weights for policy 0, policy_version 215851 (0.0008) [2023-12-27 07:08:44,593][20347] Updated weights for policy 0, policy_version 215861 (0.0007) [2023-12-27 07:08:44,637][20347] Updated weights for policy 0, policy_version 215871 (0.0007) [2023-12-27 07:08:44,784][20381] Updated weights for policy 1, policy_version 215482 (0.0007) [2023-12-27 07:08:44,841][20381] Updated weights for policy 1, policy_version 215492 (0.0008) [2023-12-27 07:08:44,902][20381] Updated weights for policy 1, policy_version 215502 (0.0008) [2023-12-27 07:08:45,354][20347] Updated weights for policy 0, policy_version 215881 (0.0007) [2023-12-27 07:08:45,412][20347] Updated weights for policy 0, policy_version 215891 (0.0008) [2023-12-27 07:08:45,470][20347] Updated weights for policy 0, policy_version 215901 (0.0007) [2023-12-27 07:08:45,532][20347] Updated weights for policy 0, policy_version 215911 (0.0008) [2023-12-27 07:08:45,596][20381] Updated weights for policy 1, policy_version 215512 (0.0008) [2023-12-27 07:08:45,654][20381] Updated weights for policy 1, policy_version 215522 (0.0008) [2023-12-27 07:08:45,708][20381] Updated weights for policy 1, policy_version 215532 (0.0008) [2023-12-27 07:08:46,194][20347] Updated weights for policy 0, policy_version 215921 (0.0008) [2023-12-27 07:08:46,242][20347] Updated weights for policy 0, policy_version 215931 (0.0007) [2023-12-27 07:08:46,291][20347] Updated weights for policy 0, policy_version 215941 (0.0005) [2023-12-27 07:08:46,427][20381] Updated weights for policy 1, policy_version 215542 (0.0010) [2023-12-27 07:08:46,486][20381] Updated weights for policy 1, policy_version 215552 (0.0010) [2023-12-27 07:08:46,535][20381] Updated weights for policy 1, policy_version 215562 (0.0008) [2023-12-27 07:08:46,863][20347] Updated weights for policy 0, policy_version 215951 (0.0007) [2023-12-27 07:08:46,915][20347] Updated weights for policy 0, policy_version 215961 (0.0006) [2023-12-27 07:08:46,974][20347] Updated weights for policy 0, policy_version 215971 (0.0006) [2023-12-27 07:08:47,252][20381] Updated weights for policy 1, policy_version 215572 (0.0008) [2023-12-27 07:08:47,309][20381] Updated weights for policy 1, policy_version 215582 (0.0007) [2023-12-27 07:08:47,361][20381] Updated weights for policy 1, policy_version 215592 (0.0007) [2023-12-27 07:08:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 110501888. Throughput: 0: 10448.3, 1: 10248.3. Samples: 110516232. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:08:47,593][20347] Updated weights for policy 0, policy_version 215981 (0.0006) [2023-12-27 07:08:47,650][20347] Updated weights for policy 0, policy_version 215991 (0.0007) [2023-12-27 07:08:47,702][20347] Updated weights for policy 0, policy_version 216001 (0.0008) [2023-12-27 07:08:47,995][20381] Updated weights for policy 1, policy_version 215602 (0.0006) [2023-12-27 07:08:48,057][20381] Updated weights for policy 1, policy_version 215612 (0.0009) [2023-12-27 07:08:48,117][20381] Updated weights for policy 1, policy_version 215622 (0.0008) [2023-12-27 07:08:48,176][20381] Updated weights for policy 1, policy_version 215632 (0.0008) [2023-12-27 07:08:48,385][20347] Updated weights for policy 0, policy_version 216011 (0.0007) [2023-12-27 07:08:48,433][20347] Updated weights for policy 0, policy_version 216021 (0.0008) [2023-12-27 07:08:48,489][20347] Updated weights for policy 0, policy_version 216031 (0.0008) [2023-12-27 07:08:48,841][20381] Updated weights for policy 1, policy_version 215642 (0.0008) [2023-12-27 07:08:48,898][20381] Updated weights for policy 1, policy_version 215652 (0.0008) [2023-12-27 07:08:48,958][20381] Updated weights for policy 1, policy_version 215662 (0.0008) [2023-12-27 07:08:49,249][20347] Updated weights for policy 0, policy_version 216041 (0.0008) [2023-12-27 07:08:49,311][20347] Updated weights for policy 0, policy_version 216051 (0.0008) [2023-12-27 07:08:49,376][20347] Updated weights for policy 0, policy_version 216061 (0.0007) [2023-12-27 07:08:49,425][20347] Updated weights for policy 0, policy_version 216071 (0.0007) [2023-12-27 07:08:49,671][20381] Updated weights for policy 1, policy_version 215672 (0.0008) [2023-12-27 07:08:49,731][20381] Updated weights for policy 1, policy_version 215682 (0.0008) [2023-12-27 07:08:49,790][20381] Updated weights for policy 1, policy_version 215692 (0.0007) [2023-12-27 07:08:50,121][20347] Updated weights for policy 0, policy_version 216081 (0.0007) [2023-12-27 07:08:50,168][20347] Updated weights for policy 0, policy_version 216091 (0.0008) [2023-12-27 07:08:50,215][20347] Updated weights for policy 0, policy_version 216101 (0.0007) [2023-12-27 07:08:50,471][20381] Updated weights for policy 1, policy_version 215702 (0.0008) [2023-12-27 07:08:50,517][20381] Updated weights for policy 1, policy_version 215712 (0.0007) [2023-12-27 07:08:50,580][20381] Updated weights for policy 1, policy_version 215722 (0.0008) [2023-12-27 07:08:50,901][20347] Updated weights for policy 0, policy_version 216111 (0.0006) [2023-12-27 07:08:50,957][20347] Updated weights for policy 0, policy_version 216121 (0.0006) [2023-12-27 07:08:51,008][20347] Updated weights for policy 0, policy_version 216131 (0.0008) [2023-12-27 07:08:51,308][20381] Updated weights for policy 1, policy_version 215732 (0.0008) [2023-12-27 07:08:51,365][20381] Updated weights for policy 1, policy_version 215742 (0.0008) [2023-12-27 07:08:51,431][20381] Updated weights for policy 1, policy_version 215752 (0.0010) [2023-12-27 07:08:51,714][20347] Updated weights for policy 0, policy_version 216141 (0.0006) [2023-12-27 07:08:51,777][20347] Updated weights for policy 0, policy_version 216151 (0.0007) [2023-12-27 07:08:51,833][20347] Updated weights for policy 0, policy_version 216161 (0.0008) [2023-12-27 07:08:52,125][20381] Updated weights for policy 1, policy_version 215762 (0.0008) [2023-12-27 07:08:52,182][20381] Updated weights for policy 1, policy_version 215772 (0.0008) [2023-12-27 07:08:52,230][20381] Updated weights for policy 1, policy_version 215782 (0.0009) [2023-12-27 07:08:52,285][20381] Updated weights for policy 1, policy_version 215792 (0.0007) [2023-12-27 07:08:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 110608384. Throughput: 0: 10451.0, 1: 10205.6. Samples: 110576824. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:08:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000215792_55255040.pth... [2023-12-27 07:08:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000216168_55353344.pth... [2023-12-27 07:08:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000214608_54951936.pth [2023-12-27 07:08:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000214952_55042048.pth [2023-12-27 07:08:52,455][20347] Updated weights for policy 0, policy_version 216171 (0.0008) [2023-12-27 07:08:52,509][20347] Updated weights for policy 0, policy_version 216181 (0.0008) [2023-12-27 07:08:52,564][20347] Updated weights for policy 0, policy_version 216191 (0.0006) [2023-12-27 07:08:52,966][20381] Updated weights for policy 1, policy_version 215802 (0.0007) [2023-12-27 07:08:53,016][20381] Updated weights for policy 1, policy_version 215812 (0.0005) [2023-12-27 07:08:53,070][20381] Updated weights for policy 1, policy_version 215822 (0.0005) [2023-12-27 07:08:53,214][20347] Updated weights for policy 0, policy_version 216201 (0.0007) [2023-12-27 07:08:53,263][20347] Updated weights for policy 0, policy_version 216211 (0.0010) [2023-12-27 07:08:53,311][20347] Updated weights for policy 0, policy_version 216221 (0.0010) [2023-12-27 07:08:53,359][20347] Updated weights for policy 0, policy_version 216231 (0.0010) [2023-12-27 07:08:53,651][20381] Updated weights for policy 1, policy_version 215832 (0.0009) [2023-12-27 07:08:53,708][20381] Updated weights for policy 1, policy_version 215842 (0.0010) [2023-12-27 07:08:53,765][20381] Updated weights for policy 1, policy_version 215852 (0.0010) [2023-12-27 07:08:54,042][20347] Updated weights for policy 0, policy_version 216241 (0.0008) [2023-12-27 07:08:54,102][20347] Updated weights for policy 0, policy_version 216251 (0.0008) [2023-12-27 07:08:54,165][20347] Updated weights for policy 0, policy_version 216261 (0.0008) [2023-12-27 07:08:54,401][20381] Updated weights for policy 1, policy_version 215862 (0.0007) [2023-12-27 07:08:54,454][20381] Updated weights for policy 1, policy_version 215872 (0.0005) [2023-12-27 07:08:54,506][20381] Updated weights for policy 1, policy_version 215882 (0.0009) [2023-12-27 07:08:54,884][20347] Updated weights for policy 0, policy_version 216271 (0.0007) [2023-12-27 07:08:54,941][20347] Updated weights for policy 0, policy_version 216281 (0.0008) [2023-12-27 07:08:54,993][20347] Updated weights for policy 0, policy_version 216291 (0.0006) [2023-12-27 07:08:55,084][20381] Updated weights for policy 1, policy_version 215892 (0.0007) [2023-12-27 07:08:55,146][20381] Updated weights for policy 1, policy_version 215902 (0.0006) [2023-12-27 07:08:55,195][20381] Updated weights for policy 1, policy_version 215912 (0.0007) [2023-12-27 07:08:55,722][20347] Updated weights for policy 0, policy_version 216301 (0.0008) [2023-12-27 07:08:55,768][20347] Updated weights for policy 0, policy_version 216311 (0.0009) [2023-12-27 07:08:55,814][20347] Updated weights for policy 0, policy_version 216321 (0.0007) [2023-12-27 07:08:55,839][20381] Updated weights for policy 1, policy_version 215922 (0.0008) [2023-12-27 07:08:55,888][20381] Updated weights for policy 1, policy_version 215932 (0.0010) [2023-12-27 07:08:55,933][20381] Updated weights for policy 1, policy_version 215942 (0.0010) [2023-12-27 07:08:55,979][20381] Updated weights for policy 1, policy_version 215952 (0.0010) [2023-12-27 07:08:56,571][20381] Updated weights for policy 1, policy_version 215962 (0.0006) [2023-12-27 07:08:56,573][20347] Updated weights for policy 0, policy_version 216331 (0.0007) [2023-12-27 07:08:56,615][20381] Updated weights for policy 1, policy_version 215972 (0.0008) [2023-12-27 07:08:56,634][20347] Updated weights for policy 0, policy_version 216341 (0.0007) [2023-12-27 07:08:56,660][20381] Updated weights for policy 1, policy_version 215982 (0.0010) [2023-12-27 07:08:56,695][20347] Updated weights for policy 0, policy_version 216351 (0.0007) [2023-12-27 07:08:57,313][20381] Updated weights for policy 1, policy_version 215992 (0.0010) [2023-12-27 07:08:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 110706688. Throughput: 0: 10454.2, 1: 10255.1. Samples: 110701752. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:08:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:08:57,373][20381] Updated weights for policy 1, policy_version 216002 (0.0010) [2023-12-27 07:08:57,374][20347] Updated weights for policy 0, policy_version 216361 (0.0006) [2023-12-27 07:08:57,427][20347] Updated weights for policy 0, policy_version 216371 (0.0006) [2023-12-27 07:08:57,433][20381] Updated weights for policy 1, policy_version 216012 (0.0008) [2023-12-27 07:08:57,480][20347] Updated weights for policy 0, policy_version 216381 (0.0007) [2023-12-27 07:08:57,528][20347] Updated weights for policy 0, policy_version 216391 (0.0007) [2023-12-27 07:08:58,026][20381] Updated weights for policy 1, policy_version 216022 (0.0006) [2023-12-27 07:08:58,074][20381] Updated weights for policy 1, policy_version 216032 (0.0005) [2023-12-27 07:08:58,133][20381] Updated weights for policy 1, policy_version 216042 (0.0005) [2023-12-27 07:08:58,216][20347] Updated weights for policy 0, policy_version 216401 (0.0008) [2023-12-27 07:08:58,266][20347] Updated weights for policy 0, policy_version 216411 (0.0008) [2023-12-27 07:08:58,323][20347] Updated weights for policy 0, policy_version 216421 (0.0008) [2023-12-27 07:08:58,921][20381] Updated weights for policy 1, policy_version 216052 (0.0006) [2023-12-27 07:08:58,981][20381] Updated weights for policy 1, policy_version 216062 (0.0007) [2023-12-27 07:08:59,039][20381] Updated weights for policy 1, policy_version 216072 (0.0008) [2023-12-27 07:08:59,154][20347] Updated weights for policy 0, policy_version 216431 (0.0007) [2023-12-27 07:08:59,216][20347] Updated weights for policy 0, policy_version 216441 (0.0007) [2023-12-27 07:08:59,283][20347] Updated weights for policy 0, policy_version 216451 (0.0007) [2023-12-27 07:08:59,731][20381] Updated weights for policy 1, policy_version 216082 (0.0008) [2023-12-27 07:08:59,791][20381] Updated weights for policy 1, policy_version 216092 (0.0005) [2023-12-27 07:08:59,852][20381] Updated weights for policy 1, policy_version 216102 (0.0005) [2023-12-27 07:08:59,905][20347] Updated weights for policy 0, policy_version 216461 (0.0007) [2023-12-27 07:08:59,912][20381] Updated weights for policy 1, policy_version 216112 (0.0007) [2023-12-27 07:08:59,970][20347] Updated weights for policy 0, policy_version 216471 (0.0007) [2023-12-27 07:09:00,032][20347] Updated weights for policy 0, policy_version 216481 (0.0005) [2023-12-27 07:09:00,563][20381] Updated weights for policy 1, policy_version 216122 (0.0007) [2023-12-27 07:09:00,622][20381] Updated weights for policy 1, policy_version 216132 (0.0008) [2023-12-27 07:09:00,623][20347] Updated weights for policy 0, policy_version 216491 (0.0005) [2023-12-27 07:09:00,678][20347] Updated weights for policy 0, policy_version 216501 (0.0005) [2023-12-27 07:09:00,679][20381] Updated weights for policy 1, policy_version 216142 (0.0006) [2023-12-27 07:09:00,733][20347] Updated weights for policy 0, policy_version 216511 (0.0006) [2023-12-27 07:09:01,359][20381] Updated weights for policy 1, policy_version 216152 (0.0009) [2023-12-27 07:09:01,394][20347] Updated weights for policy 0, policy_version 216521 (0.0008) [2023-12-27 07:09:01,423][20381] Updated weights for policy 1, policy_version 216162 (0.0007) [2023-12-27 07:09:01,449][20347] Updated weights for policy 0, policy_version 216531 (0.0006) [2023-12-27 07:09:01,478][20381] Updated weights for policy 1, policy_version 216172 (0.0006) [2023-12-27 07:09:01,499][20347] Updated weights for policy 0, policy_version 216541 (0.0006) [2023-12-27 07:09:01,550][20347] Updated weights for policy 0, policy_version 216551 (0.0008) [2023-12-27 07:09:02,223][20381] Updated weights for policy 1, policy_version 216182 (0.0007) [2023-12-27 07:09:02,248][20347] Updated weights for policy 0, policy_version 216561 (0.0008) [2023-12-27 07:09:02,287][20381] Updated weights for policy 1, policy_version 216192 (0.0008) [2023-12-27 07:09:02,311][20347] Updated weights for policy 0, policy_version 216571 (0.0007) [2023-12-27 07:09:02,342][20381] Updated weights for policy 1, policy_version 216202 (0.0006) [2023-12-27 07:09:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 110804992. Throughput: 0: 10431.2, 1: 10260.7. Samples: 110824304. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:09:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:02,376][20347] Updated weights for policy 0, policy_version 216581 (0.0007) [2023-12-27 07:09:02,968][20347] Updated weights for policy 0, policy_version 216591 (0.0007) [2023-12-27 07:09:03,028][20347] Updated weights for policy 0, policy_version 216601 (0.0008) [2023-12-27 07:09:03,062][20381] Updated weights for policy 1, policy_version 216212 (0.0009) [2023-12-27 07:09:03,089][20347] Updated weights for policy 0, policy_version 216611 (0.0005) [2023-12-27 07:09:03,120][20381] Updated weights for policy 1, policy_version 216222 (0.0010) [2023-12-27 07:09:03,179][20381] Updated weights for policy 1, policy_version 216232 (0.0010) [2023-12-27 07:09:03,743][20347] Updated weights for policy 0, policy_version 216621 (0.0008) [2023-12-27 07:09:03,792][20381] Updated weights for policy 1, policy_version 216242 (0.0006) [2023-12-27 07:09:03,801][20347] Updated weights for policy 0, policy_version 216631 (0.0010) [2023-12-27 07:09:03,851][20381] Updated weights for policy 1, policy_version 216252 (0.0009) [2023-12-27 07:09:03,860][20347] Updated weights for policy 0, policy_version 216641 (0.0010) [2023-12-27 07:09:03,913][20381] Updated weights for policy 1, policy_version 216262 (0.0007) [2023-12-27 07:09:03,981][20381] Updated weights for policy 1, policy_version 216272 (0.0008) [2023-12-27 07:09:04,549][20347] Updated weights for policy 0, policy_version 216651 (0.0010) [2023-12-27 07:09:04,591][20381] Updated weights for policy 1, policy_version 216282 (0.0008) [2023-12-27 07:09:04,615][20347] Updated weights for policy 0, policy_version 216661 (0.0009) [2023-12-27 07:09:04,657][20381] Updated weights for policy 1, policy_version 216292 (0.0007) [2023-12-27 07:09:04,679][20347] Updated weights for policy 0, policy_version 216671 (0.0011) [2023-12-27 07:09:04,719][20381] Updated weights for policy 1, policy_version 216302 (0.0010) [2023-12-27 07:09:05,295][20347] Updated weights for policy 0, policy_version 216681 (0.0011) [2023-12-27 07:09:05,328][20381] Updated weights for policy 1, policy_version 216312 (0.0010) [2023-12-27 07:09:05,347][20347] Updated weights for policy 0, policy_version 216691 (0.0010) [2023-12-27 07:09:05,374][20381] Updated weights for policy 1, policy_version 216322 (0.0010) [2023-12-27 07:09:05,392][20347] Updated weights for policy 0, policy_version 216701 (0.0010) [2023-12-27 07:09:05,419][20381] Updated weights for policy 1, policy_version 216332 (0.0010) [2023-12-27 07:09:05,440][20347] Updated weights for policy 0, policy_version 216711 (0.0010) [2023-12-27 07:09:06,068][20381] Updated weights for policy 1, policy_version 216342 (0.0007) [2023-12-27 07:09:06,137][20381] Updated weights for policy 1, policy_version 216352 (0.0008) [2023-12-27 07:09:06,166][20347] Updated weights for policy 0, policy_version 216721 (0.0007) [2023-12-27 07:09:06,197][20381] Updated weights for policy 1, policy_version 216362 (0.0011) [2023-12-27 07:09:06,221][20347] Updated weights for policy 0, policy_version 216731 (0.0005) [2023-12-27 07:09:06,275][20347] Updated weights for policy 0, policy_version 216741 (0.0007) [2023-12-27 07:09:06,854][20381] Updated weights for policy 1, policy_version 216372 (0.0010) [2023-12-27 07:09:06,907][20381] Updated weights for policy 1, policy_version 216382 (0.0010) [2023-12-27 07:09:06,962][20381] Updated weights for policy 1, policy_version 216392 (0.0011) [2023-12-27 07:09:07,011][20347] Updated weights for policy 0, policy_version 216751 (0.0006) [2023-12-27 07:09:07,064][20347] Updated weights for policy 0, policy_version 216761 (0.0007) [2023-12-27 07:09:07,123][20347] Updated weights for policy 0, policy_version 216771 (0.0007) [2023-12-27 07:09:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 110919680. Throughput: 0: 10410.7, 1: 10279.7. Samples: 110887312. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:09:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000216400_55410688.pth... [2023-12-27 07:09:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000216776_55508992.pth... [2023-12-27 07:09:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000215184_55099392.pth [2023-12-27 07:09:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000215560_55197696.pth [2023-12-27 07:09:07,635][20381] Updated weights for policy 1, policy_version 216402 (0.0010) [2023-12-27 07:09:07,690][20381] Updated weights for policy 1, policy_version 216412 (0.0011) [2023-12-27 07:09:07,724][20347] Updated weights for policy 0, policy_version 216781 (0.0007) [2023-12-27 07:09:07,750][20381] Updated weights for policy 1, policy_version 216422 (0.0010) [2023-12-27 07:09:07,769][20347] Updated weights for policy 0, policy_version 216791 (0.0005) [2023-12-27 07:09:07,804][20381] Updated weights for policy 1, policy_version 216432 (0.0005) [2023-12-27 07:09:07,829][20347] Updated weights for policy 0, policy_version 216801 (0.0007) [2023-12-27 07:09:08,348][20381] Updated weights for policy 1, policy_version 216442 (0.0008) [2023-12-27 07:09:08,406][20381] Updated weights for policy 1, policy_version 216452 (0.0007) [2023-12-27 07:09:08,462][20381] Updated weights for policy 1, policy_version 216462 (0.0005) [2023-12-27 07:09:08,586][20347] Updated weights for policy 0, policy_version 216811 (0.0009) [2023-12-27 07:09:08,641][20347] Updated weights for policy 0, policy_version 216821 (0.0010) [2023-12-27 07:09:08,686][20347] Updated weights for policy 0, policy_version 216831 (0.0009) [2023-12-27 07:09:09,131][20381] Updated weights for policy 1, policy_version 216472 (0.0007) [2023-12-27 07:09:09,193][20381] Updated weights for policy 1, policy_version 216482 (0.0007) [2023-12-27 07:09:09,258][20381] Updated weights for policy 1, policy_version 216492 (0.0007) [2023-12-27 07:09:09,324][20347] Updated weights for policy 0, policy_version 216841 (0.0005) [2023-12-27 07:09:09,387][20347] Updated weights for policy 0, policy_version 216851 (0.0008) [2023-12-27 07:09:09,446][20347] Updated weights for policy 0, policy_version 216861 (0.0006) [2023-12-27 07:09:09,507][20347] Updated weights for policy 0, policy_version 216871 (0.0008) [2023-12-27 07:09:09,951][20381] Updated weights for policy 1, policy_version 216502 (0.0007) [2023-12-27 07:09:10,013][20381] Updated weights for policy 1, policy_version 216512 (0.0008) [2023-12-27 07:09:10,081][20381] Updated weights for policy 1, policy_version 216522 (0.0008) [2023-12-27 07:09:10,249][20347] Updated weights for policy 0, policy_version 216881 (0.0008) [2023-12-27 07:09:10,306][20347] Updated weights for policy 0, policy_version 216891 (0.0008) [2023-12-27 07:09:10,363][20347] Updated weights for policy 0, policy_version 216901 (0.0008) [2023-12-27 07:09:10,758][20381] Updated weights for policy 1, policy_version 216532 (0.0008) [2023-12-27 07:09:10,813][20381] Updated weights for policy 1, policy_version 216542 (0.0007) [2023-12-27 07:09:10,864][20381] Updated weights for policy 1, policy_version 216552 (0.0007) [2023-12-27 07:09:11,089][20347] Updated weights for policy 0, policy_version 216911 (0.0007) [2023-12-27 07:09:11,150][20347] Updated weights for policy 0, policy_version 216921 (0.0007) [2023-12-27 07:09:11,211][20347] Updated weights for policy 0, policy_version 216931 (0.0007) [2023-12-27 07:09:11,558][20381] Updated weights for policy 1, policy_version 216562 (0.0005) [2023-12-27 07:09:11,619][20381] Updated weights for policy 1, policy_version 216572 (0.0006) [2023-12-27 07:09:11,683][20381] Updated weights for policy 1, policy_version 216582 (0.0007) [2023-12-27 07:09:11,751][20381] Updated weights for policy 1, policy_version 216592 (0.0008) [2023-12-27 07:09:11,891][20347] Updated weights for policy 0, policy_version 216941 (0.0009) [2023-12-27 07:09:11,953][20347] Updated weights for policy 0, policy_version 216951 (0.0008) [2023-12-27 07:09:12,009][20347] Updated weights for policy 0, policy_version 216961 (0.0007) [2023-12-27 07:09:12,315][20381] Updated weights for policy 1, policy_version 216602 (0.0007) [2023-12-27 07:09:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 111017984. Throughput: 0: 10396.3, 1: 10290.8. Samples: 111011252. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:09:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:12,379][20381] Updated weights for policy 1, policy_version 216612 (0.0007) [2023-12-27 07:09:12,443][20381] Updated weights for policy 1, policy_version 216622 (0.0005) [2023-12-27 07:09:12,764][20347] Updated weights for policy 0, policy_version 216971 (0.0008) [2023-12-27 07:09:12,820][20347] Updated weights for policy 0, policy_version 216981 (0.0008) [2023-12-27 07:09:12,874][20347] Updated weights for policy 0, policy_version 216991 (0.0007) [2023-12-27 07:09:13,053][20381] Updated weights for policy 1, policy_version 216632 (0.0008) [2023-12-27 07:09:13,110][20381] Updated weights for policy 1, policy_version 216642 (0.0007) [2023-12-27 07:09:13,160][20381] Updated weights for policy 1, policy_version 216652 (0.0008) [2023-12-27 07:09:13,506][20347] Updated weights for policy 0, policy_version 217001 (0.0007) [2023-12-27 07:09:13,569][20347] Updated weights for policy 0, policy_version 217011 (0.0005) [2023-12-27 07:09:13,630][20347] Updated weights for policy 0, policy_version 217021 (0.0005) [2023-12-27 07:09:13,692][20347] Updated weights for policy 0, policy_version 217031 (0.0006) [2023-12-27 07:09:13,894][20381] Updated weights for policy 1, policy_version 216662 (0.0007) [2023-12-27 07:09:13,939][20381] Updated weights for policy 1, policy_version 216672 (0.0008) [2023-12-27 07:09:13,985][20381] Updated weights for policy 1, policy_version 216682 (0.0007) [2023-12-27 07:09:14,237][20347] Updated weights for policy 0, policy_version 217041 (0.0006) [2023-12-27 07:09:14,293][20347] Updated weights for policy 0, policy_version 217051 (0.0005) [2023-12-27 07:09:14,346][20347] Updated weights for policy 0, policy_version 217061 (0.0005) [2023-12-27 07:09:14,758][20381] Updated weights for policy 1, policy_version 216692 (0.0006) [2023-12-27 07:09:14,824][20381] Updated weights for policy 1, policy_version 216702 (0.0005) [2023-12-27 07:09:14,890][20381] Updated weights for policy 1, policy_version 216712 (0.0006) [2023-12-27 07:09:14,898][20347] Updated weights for policy 0, policy_version 217071 (0.0009) [2023-12-27 07:09:14,947][20347] Updated weights for policy 0, policy_version 217081 (0.0010) [2023-12-27 07:09:15,006][20347] Updated weights for policy 0, policy_version 217091 (0.0006) [2023-12-27 07:09:15,565][20381] Updated weights for policy 1, policy_version 216722 (0.0006) [2023-12-27 07:09:15,625][20381] Updated weights for policy 1, policy_version 216732 (0.0006) [2023-12-27 07:09:15,663][20347] Updated weights for policy 0, policy_version 217101 (0.0010) [2023-12-27 07:09:15,680][20381] Updated weights for policy 1, policy_version 216742 (0.0006) [2023-12-27 07:09:15,706][20347] Updated weights for policy 0, policy_version 217111 (0.0010) [2023-12-27 07:09:15,724][20381] Updated weights for policy 1, policy_version 216752 (0.0008) [2023-12-27 07:09:15,751][20347] Updated weights for policy 0, policy_version 217121 (0.0010) [2023-12-27 07:09:16,411][20381] Updated weights for policy 1, policy_version 216762 (0.0005) [2023-12-27 07:09:16,412][20347] Updated weights for policy 0, policy_version 217131 (0.0010) [2023-12-27 07:09:16,470][20381] Updated weights for policy 1, policy_version 216772 (0.0006) [2023-12-27 07:09:16,475][20347] Updated weights for policy 0, policy_version 217141 (0.0010) [2023-12-27 07:09:16,529][20381] Updated weights for policy 1, policy_version 216782 (0.0005) [2023-12-27 07:09:16,530][20347] Updated weights for policy 0, policy_version 217151 (0.0010) [2023-12-27 07:09:16,570][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-27 07:09:17,108][20381] Updated weights for policy 1, policy_version 216792 (0.0006) [2023-12-27 07:09:17,171][20347] Updated weights for policy 0, policy_version 217161 (0.0010) [2023-12-27 07:09:17,172][20381] Updated weights for policy 1, policy_version 216802 (0.0007) [2023-12-27 07:09:17,232][20381] Updated weights for policy 1, policy_version 216812 (0.0008) [2023-12-27 07:09:17,236][20347] Updated weights for policy 0, policy_version 217171 (0.0006) [2023-12-27 07:09:17,293][20347] Updated weights for policy 0, policy_version 217181 (0.0005) [2023-12-27 07:09:17,347][20347] Updated weights for policy 0, policy_version 217191 (0.0008) [2023-12-27 07:09:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 111132672. Throughput: 0: 10383.0, 1: 10320.4. Samples: 111137312. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 07:09:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:17,961][20381] Updated weights for policy 1, policy_version 216822 (0.0008) [2023-12-27 07:09:17,973][20347] Updated weights for policy 0, policy_version 217201 (0.0006) [2023-12-27 07:09:18,009][20381] Updated weights for policy 1, policy_version 216832 (0.0007) [2023-12-27 07:09:18,026][20347] Updated weights for policy 0, policy_version 217211 (0.0009) [2023-12-27 07:09:18,063][20381] Updated weights for policy 1, policy_version 216842 (0.0005) [2023-12-27 07:09:18,081][20347] Updated weights for policy 0, policy_version 217221 (0.0010) [2023-12-27 07:09:18,698][20381] Updated weights for policy 1, policy_version 216852 (0.0005) [2023-12-27 07:09:18,760][20381] Updated weights for policy 1, policy_version 216862 (0.0005) [2023-12-27 07:09:18,766][20347] Updated weights for policy 0, policy_version 217231 (0.0010) [2023-12-27 07:09:18,819][20381] Updated weights for policy 1, policy_version 216872 (0.0005) [2023-12-27 07:09:18,831][20347] Updated weights for policy 0, policy_version 217241 (0.0010) [2023-12-27 07:09:18,883][20347] Updated weights for policy 0, policy_version 217251 (0.0011) [2023-12-27 07:09:19,433][20381] Updated weights for policy 1, policy_version 216882 (0.0006) [2023-12-27 07:09:19,498][20381] Updated weights for policy 1, policy_version 216892 (0.0008) [2023-12-27 07:09:19,540][20347] Updated weights for policy 0, policy_version 217261 (0.0007) [2023-12-27 07:09:19,556][20381] Updated weights for policy 1, policy_version 216902 (0.0010) [2023-12-27 07:09:19,603][20347] Updated weights for policy 0, policy_version 217271 (0.0008) [2023-12-27 07:09:19,619][20381] Updated weights for policy 1, policy_version 216912 (0.0010) [2023-12-27 07:09:19,663][20347] Updated weights for policy 0, policy_version 217281 (0.0009) [2023-12-27 07:09:20,334][20381] Updated weights for policy 1, policy_version 216922 (0.0007) [2023-12-27 07:09:20,360][20347] Updated weights for policy 0, policy_version 217291 (0.0008) [2023-12-27 07:09:20,389][20381] Updated weights for policy 1, policy_version 216932 (0.0008) [2023-12-27 07:09:20,413][20347] Updated weights for policy 0, policy_version 217301 (0.0008) [2023-12-27 07:09:20,444][20381] Updated weights for policy 1, policy_version 216942 (0.0007) [2023-12-27 07:09:20,469][20347] Updated weights for policy 0, policy_version 217311 (0.0006) [2023-12-27 07:09:21,103][20381] Updated weights for policy 1, policy_version 216952 (0.0009) [2023-12-27 07:09:21,123][20347] Updated weights for policy 0, policy_version 217321 (0.0008) [2023-12-27 07:09:21,164][20381] Updated weights for policy 1, policy_version 216962 (0.0011) [2023-12-27 07:09:21,187][20347] Updated weights for policy 0, policy_version 217331 (0.0011) [2023-12-27 07:09:21,228][20381] Updated weights for policy 1, policy_version 216972 (0.0010) [2023-12-27 07:09:21,247][20347] Updated weights for policy 0, policy_version 217341 (0.0010) [2023-12-27 07:09:21,306][20347] Updated weights for policy 0, policy_version 217351 (0.0010) [2023-12-27 07:09:21,947][20381] Updated weights for policy 1, policy_version 216982 (0.0011) [2023-12-27 07:09:22,007][20381] Updated weights for policy 1, policy_version 216992 (0.0011) [2023-12-27 07:09:22,028][20347] Updated weights for policy 0, policy_version 217361 (0.0006) [2023-12-27 07:09:22,064][20381] Updated weights for policy 1, policy_version 217002 (0.0011) [2023-12-27 07:09:22,089][20347] Updated weights for policy 0, policy_version 217371 (0.0006) [2023-12-27 07:09:22,137][20347] Updated weights for policy 0, policy_version 217381 (0.0008) [2023-12-27 07:09:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 111230976. Throughput: 0: 10362.9, 1: 10309.1. Samples: 111198476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000217008_55566336.pth... [2023-12-27 07:09:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000217384_55664640.pth... [2023-12-27 07:09:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000215792_55255040.pth [2023-12-27 07:09:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000216168_55353344.pth [2023-12-27 07:09:22,782][20381] Updated weights for policy 1, policy_version 217012 (0.0011) [2023-12-27 07:09:22,842][20381] Updated weights for policy 1, policy_version 217022 (0.0011) [2023-12-27 07:09:22,866][20347] Updated weights for policy 0, policy_version 217391 (0.0007) [2023-12-27 07:09:22,901][20381] Updated weights for policy 1, policy_version 217032 (0.0011) [2023-12-27 07:09:22,918][20347] Updated weights for policy 0, policy_version 217401 (0.0005) [2023-12-27 07:09:22,971][20347] Updated weights for policy 0, policy_version 217411 (0.0006) [2023-12-27 07:09:23,511][20381] Updated weights for policy 1, policy_version 217042 (0.0009) [2023-12-27 07:09:23,554][20347] Updated weights for policy 0, policy_version 217421 (0.0006) [2023-12-27 07:09:23,564][20381] Updated weights for policy 1, policy_version 217052 (0.0010) [2023-12-27 07:09:23,607][20347] Updated weights for policy 0, policy_version 217431 (0.0005) [2023-12-27 07:09:23,619][20381] Updated weights for policy 1, policy_version 217062 (0.0010) [2023-12-27 07:09:23,659][20347] Updated weights for policy 0, policy_version 217441 (0.0006) [2023-12-27 07:09:23,672][20381] Updated weights for policy 1, policy_version 217072 (0.0009) [2023-12-27 07:09:24,186][20347] Updated weights for policy 0, policy_version 217451 (0.0006) [2023-12-27 07:09:24,245][20347] Updated weights for policy 0, policy_version 217461 (0.0005) [2023-12-27 07:09:24,254][20381] Updated weights for policy 1, policy_version 217082 (0.0009) [2023-12-27 07:09:24,300][20381] Updated weights for policy 1, policy_version 217092 (0.0007) [2023-12-27 07:09:24,302][20347] Updated weights for policy 0, policy_version 217471 (0.0006) [2023-12-27 07:09:24,345][20381] Updated weights for policy 1, policy_version 217102 (0.0007) [2023-12-27 07:09:24,904][20347] Updated weights for policy 0, policy_version 217481 (0.0006) [2023-12-27 07:09:24,961][20347] Updated weights for policy 0, policy_version 217491 (0.0010) [2023-12-27 07:09:25,018][20347] Updated weights for policy 0, policy_version 217501 (0.0010) [2023-12-27 07:09:25,063][20381] Updated weights for policy 1, policy_version 217112 (0.0005) [2023-12-27 07:09:25,073][20347] Updated weights for policy 0, policy_version 217511 (0.0011) [2023-12-27 07:09:25,114][20381] Updated weights for policy 1, policy_version 217122 (0.0007) [2023-12-27 07:09:25,169][20381] Updated weights for policy 1, policy_version 217132 (0.0007) [2023-12-27 07:09:25,737][20347] Updated weights for policy 0, policy_version 217521 (0.0008) [2023-12-27 07:09:25,767][20381] Updated weights for policy 1, policy_version 217142 (0.0007) [2023-12-27 07:09:25,794][20347] Updated weights for policy 0, policy_version 217531 (0.0009) [2023-12-27 07:09:25,820][20381] Updated weights for policy 1, policy_version 217152 (0.0007) [2023-12-27 07:09:25,844][20347] Updated weights for policy 0, policy_version 217541 (0.0005) [2023-12-27 07:09:25,873][20381] Updated weights for policy 1, policy_version 217162 (0.0008) [2023-12-27 07:09:26,472][20347] Updated weights for policy 0, policy_version 217551 (0.0005) [2023-12-27 07:09:26,519][20381] Updated weights for policy 1, policy_version 217172 (0.0007) [2023-12-27 07:09:26,527][20347] Updated weights for policy 0, policy_version 217561 (0.0009) [2023-12-27 07:09:26,572][20381] Updated weights for policy 1, policy_version 217182 (0.0008) [2023-12-27 07:09:26,578][20347] Updated weights for policy 0, policy_version 217571 (0.0010) [2023-12-27 07:09:26,617][20381] Updated weights for policy 1, policy_version 217192 (0.0005) [2023-12-27 07:09:27,115][20347] Updated weights for policy 0, policy_version 217581 (0.0010) [2023-12-27 07:09:27,173][20347] Updated weights for policy 0, policy_version 217591 (0.0010) [2023-12-27 07:09:27,224][20381] Updated weights for policy 1, policy_version 217202 (0.0005) [2023-12-27 07:09:27,229][20347] Updated weights for policy 0, policy_version 217601 (0.0008) [2023-12-27 07:09:27,274][20381] Updated weights for policy 1, policy_version 217212 (0.0005) [2023-12-27 07:09:27,329][20381] Updated weights for policy 1, policy_version 217222 (0.0006) [2023-12-27 07:09:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 111337472. Throughput: 0: 10362.0, 1: 10347.1. Samples: 111325420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:27,376][20381] Updated weights for policy 1, policy_version 217232 (0.0008) [2023-12-27 07:09:27,806][20347] Updated weights for policy 0, policy_version 217611 (0.0007) [2023-12-27 07:09:27,862][20347] Updated weights for policy 0, policy_version 217621 (0.0010) [2023-12-27 07:09:27,925][20381] Updated weights for policy 1, policy_version 217242 (0.0008) [2023-12-27 07:09:27,929][20347] Updated weights for policy 0, policy_version 217631 (0.0009) [2023-12-27 07:09:27,980][20381] Updated weights for policy 1, policy_version 217252 (0.0007) [2023-12-27 07:09:28,036][20381] Updated weights for policy 1, policy_version 217262 (0.0007) [2023-12-27 07:09:28,597][20347] Updated weights for policy 0, policy_version 217641 (0.0009) [2023-12-27 07:09:28,663][20347] Updated weights for policy 0, policy_version 217651 (0.0007) [2023-12-27 07:09:28,667][20381] Updated weights for policy 1, policy_version 217272 (0.0006) [2023-12-27 07:09:28,729][20347] Updated weights for policy 0, policy_version 217661 (0.0007) [2023-12-27 07:09:28,733][20381] Updated weights for policy 1, policy_version 217282 (0.0006) [2023-12-27 07:09:28,792][20347] Updated weights for policy 0, policy_version 217671 (0.0007) [2023-12-27 07:09:28,798][20381] Updated weights for policy 1, policy_version 217292 (0.0006) [2023-12-27 07:09:29,347][20347] Updated weights for policy 0, policy_version 217681 (0.0006) [2023-12-27 07:09:29,384][20381] Updated weights for policy 1, policy_version 217302 (0.0007) [2023-12-27 07:09:29,405][20347] Updated weights for policy 0, policy_version 217691 (0.0007) [2023-12-27 07:09:29,437][20381] Updated weights for policy 1, policy_version 217312 (0.0006) [2023-12-27 07:09:29,460][20347] Updated weights for policy 0, policy_version 217701 (0.0007) [2023-12-27 07:09:29,494][20381] Updated weights for policy 1, policy_version 217322 (0.0007) [2023-12-27 07:09:30,062][20347] Updated weights for policy 0, policy_version 217711 (0.0007) [2023-12-27 07:09:30,116][20347] Updated weights for policy 0, policy_version 217721 (0.0007) [2023-12-27 07:09:30,176][20347] Updated weights for policy 0, policy_version 217731 (0.0007) [2023-12-27 07:09:30,195][20381] Updated weights for policy 1, policy_version 217332 (0.0008) [2023-12-27 07:09:30,252][20381] Updated weights for policy 1, policy_version 217342 (0.0008) [2023-12-27 07:09:30,308][20381] Updated weights for policy 1, policy_version 217352 (0.0007) [2023-12-27 07:09:30,814][20347] Updated weights for policy 0, policy_version 217741 (0.0006) [2023-12-27 07:09:30,871][20347] Updated weights for policy 0, policy_version 217751 (0.0005) [2023-12-27 07:09:30,917][20347] Updated weights for policy 0, policy_version 217761 (0.0007) [2023-12-27 07:09:30,944][20381] Updated weights for policy 1, policy_version 217362 (0.0007) [2023-12-27 07:09:31,000][20381] Updated weights for policy 1, policy_version 217372 (0.0006) [2023-12-27 07:09:31,058][20381] Updated weights for policy 1, policy_version 217382 (0.0008) [2023-12-27 07:09:31,128][20381] Updated weights for policy 1, policy_version 217392 (0.0008) [2023-12-27 07:09:31,550][20347] Updated weights for policy 0, policy_version 217771 (0.0007) [2023-12-27 07:09:31,606][20347] Updated weights for policy 0, policy_version 217781 (0.0007) [2023-12-27 07:09:31,669][20347] Updated weights for policy 0, policy_version 217791 (0.0008) [2023-12-27 07:09:31,822][20381] Updated weights for policy 1, policy_version 217402 (0.0006) [2023-12-27 07:09:31,869][20381] Updated weights for policy 1, policy_version 217412 (0.0006) [2023-12-27 07:09:31,913][20381] Updated weights for policy 1, policy_version 217422 (0.0007) [2023-12-27 07:09:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 111443968. Throughput: 0: 10433.0, 1: 10491.7. Samples: 111457848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:32,373][20347] Updated weights for policy 0, policy_version 217801 (0.0008) [2023-12-27 07:09:32,426][20347] Updated weights for policy 0, policy_version 217811 (0.0008) [2023-12-27 07:09:32,485][20347] Updated weights for policy 0, policy_version 217821 (0.0008) [2023-12-27 07:09:32,538][20347] Updated weights for policy 0, policy_version 217831 (0.0008) [2023-12-27 07:09:32,564][20381] Updated weights for policy 1, policy_version 217432 (0.0008) [2023-12-27 07:09:32,628][20381] Updated weights for policy 1, policy_version 217442 (0.0008) [2023-12-27 07:09:32,688][20381] Updated weights for policy 1, policy_version 217452 (0.0008) [2023-12-27 07:09:33,193][20347] Updated weights for policy 0, policy_version 217841 (0.0006) [2023-12-27 07:09:33,260][20347] Updated weights for policy 0, policy_version 217851 (0.0010) [2023-12-27 07:09:33,327][20347] Updated weights for policy 0, policy_version 217861 (0.0010) [2023-12-27 07:09:33,356][20381] Updated weights for policy 1, policy_version 217462 (0.0006) [2023-12-27 07:09:33,426][20381] Updated weights for policy 1, policy_version 217472 (0.0010) [2023-12-27 07:09:33,491][20381] Updated weights for policy 1, policy_version 217482 (0.0011) [2023-12-27 07:09:33,971][20347] Updated weights for policy 0, policy_version 217871 (0.0007) [2023-12-27 07:09:34,032][20347] Updated weights for policy 0, policy_version 217881 (0.0008) [2023-12-27 07:09:34,088][20347] Updated weights for policy 0, policy_version 217891 (0.0010) [2023-12-27 07:09:34,138][20381] Updated weights for policy 1, policy_version 217492 (0.0011) [2023-12-27 07:09:34,197][20381] Updated weights for policy 1, policy_version 217502 (0.0009) [2023-12-27 07:09:34,250][20381] Updated weights for policy 1, policy_version 217512 (0.0010) [2023-12-27 07:09:34,731][20347] Updated weights for policy 0, policy_version 217901 (0.0008) [2023-12-27 07:09:34,790][20347] Updated weights for policy 0, policy_version 217911 (0.0007) [2023-12-27 07:09:34,850][20347] Updated weights for policy 0, policy_version 217921 (0.0008) [2023-12-27 07:09:34,949][20381] Updated weights for policy 1, policy_version 217522 (0.0009) [2023-12-27 07:09:35,011][20381] Updated weights for policy 1, policy_version 217532 (0.0008) [2023-12-27 07:09:35,071][20381] Updated weights for policy 1, policy_version 217542 (0.0007) [2023-12-27 07:09:35,130][20381] Updated weights for policy 1, policy_version 217552 (0.0011) [2023-12-27 07:09:35,496][20347] Updated weights for policy 0, policy_version 217931 (0.0006) [2023-12-27 07:09:35,556][20347] Updated weights for policy 0, policy_version 217941 (0.0007) [2023-12-27 07:09:35,608][20347] Updated weights for policy 0, policy_version 217951 (0.0007) [2023-12-27 07:09:35,801][20381] Updated weights for policy 1, policy_version 217562 (0.0010) [2023-12-27 07:09:35,852][20381] Updated weights for policy 1, policy_version 217572 (0.0010) [2023-12-27 07:09:35,900][20381] Updated weights for policy 1, policy_version 217582 (0.0010) [2023-12-27 07:09:36,288][20347] Updated weights for policy 0, policy_version 217961 (0.0008) [2023-12-27 07:09:36,347][20347] Updated weights for policy 0, policy_version 217971 (0.0007) [2023-12-27 07:09:36,405][20347] Updated weights for policy 0, policy_version 217981 (0.0007) [2023-12-27 07:09:36,465][20347] Updated weights for policy 0, policy_version 217991 (0.0007) [2023-12-27 07:09:36,558][20381] Updated weights for policy 1, policy_version 217592 (0.0008) [2023-12-27 07:09:36,610][20381] Updated weights for policy 1, policy_version 217602 (0.0007) [2023-12-27 07:09:36,659][20381] Updated weights for policy 1, policy_version 217612 (0.0008) [2023-12-27 07:09:37,093][20347] Updated weights for policy 0, policy_version 218001 (0.0006) [2023-12-27 07:09:37,150][20347] Updated weights for policy 0, policy_version 218011 (0.0007) [2023-12-27 07:09:37,208][20347] Updated weights for policy 0, policy_version 218021 (0.0008) [2023-12-27 07:09:37,268][20381] Updated weights for policy 1, policy_version 217622 (0.0009) [2023-12-27 07:09:37,326][20381] Updated weights for policy 1, policy_version 217632 (0.0011) [2023-12-27 07:09:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 111550464. Throughput: 0: 10450.4, 1: 10496.1. Samples: 111519424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000218024_55828480.pth... [2023-12-27 07:09:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000216776_55508992.pth [2023-12-27 07:09:37,381][20381] Updated weights for policy 1, policy_version 217642 (0.0011) [2023-12-27 07:09:37,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000217648_55730176.pth... [2023-12-27 07:09:37,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000216400_55410688.pth [2023-12-27 07:09:37,811][20347] Updated weights for policy 0, policy_version 218031 (0.0007) [2023-12-27 07:09:37,865][20347] Updated weights for policy 0, policy_version 218041 (0.0005) [2023-12-27 07:09:37,925][20347] Updated weights for policy 0, policy_version 218051 (0.0010) [2023-12-27 07:09:38,062][20381] Updated weights for policy 1, policy_version 217652 (0.0011) [2023-12-27 07:09:38,110][20381] Updated weights for policy 1, policy_version 217662 (0.0010) [2023-12-27 07:09:38,168][20381] Updated weights for policy 1, policy_version 217672 (0.0010) [2023-12-27 07:09:38,630][20347] Updated weights for policy 0, policy_version 218061 (0.0007) [2023-12-27 07:09:38,687][20347] Updated weights for policy 0, policy_version 218071 (0.0008) [2023-12-27 07:09:38,738][20347] Updated weights for policy 0, policy_version 218081 (0.0007) [2023-12-27 07:09:38,791][20381] Updated weights for policy 1, policy_version 217682 (0.0006) [2023-12-27 07:09:38,847][20381] Updated weights for policy 1, policy_version 217692 (0.0011) [2023-12-27 07:09:38,906][20381] Updated weights for policy 1, policy_version 217702 (0.0010) [2023-12-27 07:09:38,961][20381] Updated weights for policy 1, policy_version 217712 (0.0010) [2023-12-27 07:09:39,447][20347] Updated weights for policy 0, policy_version 218091 (0.0008) [2023-12-27 07:09:39,507][20347] Updated weights for policy 0, policy_version 218101 (0.0007) [2023-12-27 07:09:39,570][20347] Updated weights for policy 0, policy_version 218111 (0.0007) [2023-12-27 07:09:39,632][20381] Updated weights for policy 1, policy_version 217722 (0.0006) [2023-12-27 07:09:39,695][20381] Updated weights for policy 1, policy_version 217732 (0.0008) [2023-12-27 07:09:39,755][20381] Updated weights for policy 1, policy_version 217742 (0.0008) [2023-12-27 07:09:40,265][20347] Updated weights for policy 0, policy_version 218121 (0.0007) [2023-12-27 07:09:40,315][20347] Updated weights for policy 0, policy_version 218131 (0.0008) [2023-12-27 07:09:40,372][20347] Updated weights for policy 0, policy_version 218141 (0.0007) [2023-12-27 07:09:40,403][20381] Updated weights for policy 1, policy_version 217752 (0.0007) [2023-12-27 07:09:40,423][20347] Updated weights for policy 0, policy_version 218151 (0.0007) [2023-12-27 07:09:40,457][20381] Updated weights for policy 1, policy_version 217762 (0.0006) [2023-12-27 07:09:40,517][20381] Updated weights for policy 1, policy_version 217772 (0.0008) [2023-12-27 07:09:41,097][20347] Updated weights for policy 0, policy_version 218161 (0.0006) [2023-12-27 07:09:41,162][20347] Updated weights for policy 0, policy_version 218171 (0.0011) [2023-12-27 07:09:41,229][20347] Updated weights for policy 0, policy_version 218181 (0.0010) [2023-12-27 07:09:41,266][20381] Updated weights for policy 1, policy_version 217782 (0.0009) [2023-12-27 07:09:41,319][20381] Updated weights for policy 1, policy_version 217792 (0.0010) [2023-12-27 07:09:41,377][20381] Updated weights for policy 1, policy_version 217802 (0.0011) [2023-12-27 07:09:41,895][20347] Updated weights for policy 0, policy_version 218191 (0.0007) [2023-12-27 07:09:41,953][20347] Updated weights for policy 0, policy_version 218201 (0.0005) [2023-12-27 07:09:42,009][20347] Updated weights for policy 0, policy_version 218211 (0.0005) [2023-12-27 07:09:42,126][20381] Updated weights for policy 1, policy_version 217812 (0.0011) [2023-12-27 07:09:42,181][20381] Updated weights for policy 1, policy_version 217822 (0.0010) [2023-12-27 07:09:42,227][20381] Updated weights for policy 1, policy_version 217832 (0.0010) [2023-12-27 07:09:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 111656960. Throughput: 0: 10484.0, 1: 10467.6. Samples: 111644572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:42,569][20347] Updated weights for policy 0, policy_version 218221 (0.0006) [2023-12-27 07:09:42,627][20347] Updated weights for policy 0, policy_version 218231 (0.0007) [2023-12-27 07:09:42,679][20347] Updated weights for policy 0, policy_version 218241 (0.0007) [2023-12-27 07:09:42,932][20381] Updated weights for policy 1, policy_version 217842 (0.0009) [2023-12-27 07:09:42,991][20381] Updated weights for policy 1, policy_version 217852 (0.0007) [2023-12-27 07:09:43,036][20381] Updated weights for policy 1, policy_version 217862 (0.0007) [2023-12-27 07:09:43,093][20381] Updated weights for policy 1, policy_version 217872 (0.0008) [2023-12-27 07:09:43,302][20347] Updated weights for policy 0, policy_version 218251 (0.0007) [2023-12-27 07:09:43,357][20347] Updated weights for policy 0, policy_version 218261 (0.0008) [2023-12-27 07:09:43,416][20347] Updated weights for policy 0, policy_version 218271 (0.0008) [2023-12-27 07:09:43,728][20381] Updated weights for policy 1, policy_version 217882 (0.0008) [2023-12-27 07:09:43,783][20381] Updated weights for policy 1, policy_version 217892 (0.0008) [2023-12-27 07:09:43,836][20381] Updated weights for policy 1, policy_version 217902 (0.0008) [2023-12-27 07:09:44,137][20347] Updated weights for policy 0, policy_version 218281 (0.0007) [2023-12-27 07:09:44,189][20347] Updated weights for policy 0, policy_version 218291 (0.0009) [2023-12-27 07:09:44,238][20347] Updated weights for policy 0, policy_version 218301 (0.0008) [2023-12-27 07:09:44,282][20347] Updated weights for policy 0, policy_version 218311 (0.0008) [2023-12-27 07:09:44,412][20381] Updated weights for policy 1, policy_version 217912 (0.0006) [2023-12-27 07:09:44,462][20381] Updated weights for policy 1, policy_version 217922 (0.0005) [2023-12-27 07:09:44,510][20381] Updated weights for policy 1, policy_version 217932 (0.0005) [2023-12-27 07:09:45,030][20347] Updated weights for policy 0, policy_version 218321 (0.0006) [2023-12-27 07:09:45,095][20347] Updated weights for policy 0, policy_version 218331 (0.0008) [2023-12-27 07:09:45,150][20347] Updated weights for policy 0, policy_version 218341 (0.0008) [2023-12-27 07:09:45,163][20381] Updated weights for policy 1, policy_version 217942 (0.0007) [2023-12-27 07:09:45,219][20381] Updated weights for policy 1, policy_version 217952 (0.0007) [2023-12-27 07:09:45,271][20381] Updated weights for policy 1, policy_version 217962 (0.0008) [2023-12-27 07:09:45,848][20347] Updated weights for policy 0, policy_version 218351 (0.0007) [2023-12-27 07:09:45,907][20347] Updated weights for policy 0, policy_version 218361 (0.0008) [2023-12-27 07:09:45,932][20381] Updated weights for policy 1, policy_version 217972 (0.0007) [2023-12-27 07:09:45,963][20347] Updated weights for policy 0, policy_version 218371 (0.0006) [2023-12-27 07:09:45,989][20381] Updated weights for policy 1, policy_version 217982 (0.0006) [2023-12-27 07:09:46,034][20381] Updated weights for policy 1, policy_version 217992 (0.0008) [2023-12-27 07:09:46,665][20347] Updated weights for policy 0, policy_version 218381 (0.0008) [2023-12-27 07:09:46,677][20381] Updated weights for policy 1, policy_version 218002 (0.0008) [2023-12-27 07:09:46,718][20347] Updated weights for policy 0, policy_version 218391 (0.0007) [2023-12-27 07:09:46,733][20381] Updated weights for policy 1, policy_version 218012 (0.0007) [2023-12-27 07:09:46,771][20347] Updated weights for policy 0, policy_version 218401 (0.0007) [2023-12-27 07:09:46,787][20381] Updated weights for policy 1, policy_version 218022 (0.0006) [2023-12-27 07:09:46,842][20381] Updated weights for policy 1, policy_version 218032 (0.0011) [2023-12-27 07:09:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 111755264. Throughput: 0: 10477.9, 1: 10509.1. Samples: 111768716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:47,454][20381] Updated weights for policy 1, policy_version 218042 (0.0009) [2023-12-27 07:09:47,487][20347] Updated weights for policy 0, policy_version 218411 (0.0008) [2023-12-27 07:09:47,514][20381] Updated weights for policy 1, policy_version 218052 (0.0007) [2023-12-27 07:09:47,546][20347] Updated weights for policy 0, policy_version 218421 (0.0007) [2023-12-27 07:09:47,581][20381] Updated weights for policy 1, policy_version 218062 (0.0006) [2023-12-27 07:09:47,606][20347] Updated weights for policy 0, policy_version 218431 (0.0008) [2023-12-27 07:09:48,246][20381] Updated weights for policy 1, policy_version 218072 (0.0008) [2023-12-27 07:09:48,293][20381] Updated weights for policy 1, policy_version 218082 (0.0010) [2023-12-27 07:09:48,313][20347] Updated weights for policy 0, policy_version 218441 (0.0009) [2023-12-27 07:09:48,350][20381] Updated weights for policy 1, policy_version 218092 (0.0009) [2023-12-27 07:09:48,374][20347] Updated weights for policy 0, policy_version 218451 (0.0007) [2023-12-27 07:09:48,433][20347] Updated weights for policy 0, policy_version 218461 (0.0007) [2023-12-27 07:09:48,494][20347] Updated weights for policy 0, policy_version 218471 (0.0007) [2023-12-27 07:09:48,979][20381] Updated weights for policy 1, policy_version 218102 (0.0007) [2023-12-27 07:09:49,022][20381] Updated weights for policy 1, policy_version 218112 (0.0008) [2023-12-27 07:09:49,073][20381] Updated weights for policy 1, policy_version 218122 (0.0009) [2023-12-27 07:09:49,191][20347] Updated weights for policy 0, policy_version 218481 (0.0008) [2023-12-27 07:09:49,254][20347] Updated weights for policy 0, policy_version 218491 (0.0008) [2023-12-27 07:09:49,315][20347] Updated weights for policy 0, policy_version 218501 (0.0008) [2023-12-27 07:09:49,719][20381] Updated weights for policy 1, policy_version 218132 (0.0008) [2023-12-27 07:09:49,776][20381] Updated weights for policy 1, policy_version 218142 (0.0006) [2023-12-27 07:09:49,833][20381] Updated weights for policy 1, policy_version 218152 (0.0006) [2023-12-27 07:09:50,091][20347] Updated weights for policy 0, policy_version 218511 (0.0008) [2023-12-27 07:09:50,160][20347] Updated weights for policy 0, policy_version 218521 (0.0007) [2023-12-27 07:09:50,227][20347] Updated weights for policy 0, policy_version 218531 (0.0007) [2023-12-27 07:09:50,440][20381] Updated weights for policy 1, policy_version 218162 (0.0006) [2023-12-27 07:09:50,489][20381] Updated weights for policy 1, policy_version 218172 (0.0007) [2023-12-27 07:09:50,549][20381] Updated weights for policy 1, policy_version 218182 (0.0007) [2023-12-27 07:09:50,603][20381] Updated weights for policy 1, policy_version 218192 (0.0008) [2023-12-27 07:09:50,908][20347] Updated weights for policy 0, policy_version 218541 (0.0007) [2023-12-27 07:09:50,958][20347] Updated weights for policy 0, policy_version 218551 (0.0008) [2023-12-27 07:09:51,003][20347] Updated weights for policy 0, policy_version 218561 (0.0008) [2023-12-27 07:09:51,325][20381] Updated weights for policy 1, policy_version 218202 (0.0008) [2023-12-27 07:09:51,394][20381] Updated weights for policy 1, policy_version 218212 (0.0008) [2023-12-27 07:09:51,444][20381] Updated weights for policy 1, policy_version 218222 (0.0008) [2023-12-27 07:09:51,750][20347] Updated weights for policy 0, policy_version 218571 (0.0008) [2023-12-27 07:09:51,814][20347] Updated weights for policy 0, policy_version 218581 (0.0008) [2023-12-27 07:09:51,876][20347] Updated weights for policy 0, policy_version 218591 (0.0008) [2023-12-27 07:09:52,149][20381] Updated weights for policy 1, policy_version 218232 (0.0008) [2023-12-27 07:09:52,211][20381] Updated weights for policy 1, policy_version 218242 (0.0008) [2023-12-27 07:09:52,272][20381] Updated weights for policy 1, policy_version 218252 (0.0009) [2023-12-27 07:09:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 111861760. Throughput: 0: 10426.5, 1: 10511.9. Samples: 111829540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:09:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000218256_55885824.pth... [2023-12-27 07:09:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000218600_55975936.pth... [2023-12-27 07:09:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000217008_55566336.pth [2023-12-27 07:09:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000217384_55664640.pth [2023-12-27 07:09:52,566][20347] Updated weights for policy 0, policy_version 218601 (0.0007) [2023-12-27 07:09:52,618][20347] Updated weights for policy 0, policy_version 218611 (0.0005) [2023-12-27 07:09:52,671][20347] Updated weights for policy 0, policy_version 218621 (0.0008) [2023-12-27 07:09:52,730][20347] Updated weights for policy 0, policy_version 218631 (0.0007) [2023-12-27 07:09:52,981][20381] Updated weights for policy 1, policy_version 218262 (0.0008) [2023-12-27 07:09:53,037][20381] Updated weights for policy 1, policy_version 218272 (0.0008) [2023-12-27 07:09:53,093][20381] Updated weights for policy 1, policy_version 218282 (0.0008) [2023-12-27 07:09:53,400][20347] Updated weights for policy 0, policy_version 218641 (0.0008) [2023-12-27 07:09:53,458][20347] Updated weights for policy 0, policy_version 218651 (0.0007) [2023-12-27 07:09:53,515][20347] Updated weights for policy 0, policy_version 218661 (0.0007) [2023-12-27 07:09:53,793][20381] Updated weights for policy 1, policy_version 218292 (0.0007) [2023-12-27 07:09:53,851][20381] Updated weights for policy 1, policy_version 218302 (0.0008) [2023-12-27 07:09:53,914][20381] Updated weights for policy 1, policy_version 218312 (0.0008) [2023-12-27 07:09:54,183][20347] Updated weights for policy 0, policy_version 218671 (0.0008) [2023-12-27 07:09:54,234][20347] Updated weights for policy 0, policy_version 218681 (0.0007) [2023-12-27 07:09:54,281][20347] Updated weights for policy 0, policy_version 218691 (0.0007) [2023-12-27 07:09:54,610][20381] Updated weights for policy 1, policy_version 218322 (0.0008) [2023-12-27 07:09:54,658][20381] Updated weights for policy 1, policy_version 218332 (0.0008) [2023-12-27 07:09:54,715][20381] Updated weights for policy 1, policy_version 218342 (0.0008) [2023-12-27 07:09:54,768][20381] Updated weights for policy 1, policy_version 218352 (0.0007) [2023-12-27 07:09:54,945][20347] Updated weights for policy 0, policy_version 218701 (0.0008) [2023-12-27 07:09:55,009][20347] Updated weights for policy 0, policy_version 218711 (0.0008) [2023-12-27 07:09:55,062][20347] Updated weights for policy 0, policy_version 218721 (0.0007) [2023-12-27 07:09:55,423][20381] Updated weights for policy 1, policy_version 218362 (0.0006) [2023-12-27 07:09:55,477][20381] Updated weights for policy 1, policy_version 218372 (0.0008) [2023-12-27 07:09:55,533][20381] Updated weights for policy 1, policy_version 218382 (0.0008) [2023-12-27 07:09:55,790][20347] Updated weights for policy 0, policy_version 218731 (0.0007) [2023-12-27 07:09:55,846][20347] Updated weights for policy 0, policy_version 218741 (0.0008) [2023-12-27 07:09:55,897][20347] Updated weights for policy 0, policy_version 218751 (0.0010) [2023-12-27 07:09:56,114][20381] Updated weights for policy 1, policy_version 218392 (0.0007) [2023-12-27 07:09:56,164][20381] Updated weights for policy 1, policy_version 218402 (0.0005) [2023-12-27 07:09:56,224][20381] Updated weights for policy 1, policy_version 218412 (0.0006) [2023-12-27 07:09:56,581][20347] Updated weights for policy 0, policy_version 218761 (0.0009) [2023-12-27 07:09:56,641][20347] Updated weights for policy 0, policy_version 218771 (0.0005) [2023-12-27 07:09:56,701][20347] Updated weights for policy 0, policy_version 218781 (0.0005) [2023-12-27 07:09:56,754][20347] Updated weights for policy 0, policy_version 218791 (0.0005) [2023-12-27 07:09:56,819][20381] Updated weights for policy 1, policy_version 218422 (0.0006) [2023-12-27 07:09:56,845][20292] KL-divergence is very high: 360.1734 [2023-12-27 07:09:56,850][20292] KL-divergence is very high: 200.6887 [2023-12-27 07:09:56,862][20381] Updated weights for policy 1, policy_version 218432 (0.0005) [2023-12-27 07:09:56,910][20381] Updated weights for policy 1, policy_version 218442 (0.0005) [2023-12-27 07:09:56,922][20292] KL-divergence is very high: 111.6819 [2023-12-27 07:09:56,928][20292] KL-divergence is very high: 108.2945 [2023-12-27 07:09:57,321][20347] Updated weights for policy 0, policy_version 218801 (0.0008) [2023-12-27 07:09:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 111960064. Throughput: 0: 10392.2, 1: 10478.1. Samples: 111950416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:09:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '574.280')] [2023-12-27 07:09:57,370][20347] Updated weights for policy 0, policy_version 218811 (0.0010) [2023-12-27 07:09:57,419][20347] Updated weights for policy 0, policy_version 218821 (0.0010) [2023-12-27 07:09:57,498][20292] KL-divergence is very high: 230.8537 [2023-12-27 07:09:57,504][20292] KL-divergence is very high: 517.7091 [2023-12-27 07:09:57,509][20292] KL-divergence is very high: 281.6384 [2023-12-27 07:09:57,514][20292] KL-divergence is very high: 252.8808 [2023-12-27 07:09:57,520][20292] KL-divergence is very high: 376.5290 [2023-12-27 07:09:57,520][20381] Updated weights for policy 1, policy_version 218452 (0.0007) [2023-12-27 07:09:57,525][20292] KL-divergence is very high: 400.4735 [2023-12-27 07:09:57,530][20292] KL-divergence is very high: 110.0641 [2023-12-27 07:09:57,536][20292] KL-divergence is very high: 236.1619 [2023-12-27 07:09:57,541][20292] KL-divergence is very high: 426.0061 [2023-12-27 07:09:57,547][20292] KL-divergence is very high: 371.7056 [2023-12-27 07:09:57,552][20292] KL-divergence is very high: 344.3356 [2023-12-27 07:09:57,558][20292] KL-divergence is very high: 288.5742 [2023-12-27 07:09:57,564][20292] KL-divergence is very high: 128.6715 [2023-12-27 07:09:57,573][20381] Updated weights for policy 1, policy_version 218462 (0.0005) [2023-12-27 07:09:57,577][20292] KL-divergence is very high: 316.0969 [2023-12-27 07:09:57,582][20292] KL-divergence is very high: 176.7565 [2023-12-27 07:09:57,587][20292] KL-divergence is very high: 711.9025 [2023-12-27 07:09:57,592][20292] KL-divergence is very high: 185.6681 [2023-12-27 07:09:57,596][20292] KL-divergence is very high: 277.7794 [2023-12-27 07:09:57,601][20292] KL-divergence is very high: 146.8912 [2023-12-27 07:09:57,606][20292] KL-divergence is very high: 101.9612 [2023-12-27 07:09:57,614][20292] KL-divergence is very high: 229.8459 [2023-12-27 07:09:57,619][20292] KL-divergence is very high: 174.9972 [2023-12-27 07:09:57,619][20381] Updated weights for policy 1, policy_version 218472 (0.0009) [2023-12-27 07:09:57,624][20292] KL-divergence is very high: 665.0969 [2023-12-27 07:09:57,628][20292] KL-divergence is very high: 336.1035 [2023-12-27 07:09:57,633][20292] KL-divergence is very high: 624.1814 [2023-12-27 07:09:57,637][20292] KL-divergence is very high: 135.8490 [2023-12-27 07:09:57,651][20292] KL-divergence is very high: 302.4009 [2023-12-27 07:09:58,073][20347] Updated weights for policy 0, policy_version 218831 (0.0009) [2023-12-27 07:09:58,122][20347] Updated weights for policy 0, policy_version 218841 (0.0005) [2023-12-27 07:09:58,183][20347] Updated weights for policy 0, policy_version 218851 (0.0008) [2023-12-27 07:09:58,287][20292] KL-divergence is very high: 173.1757 [2023-12-27 07:09:58,292][20381] Updated weights for policy 1, policy_version 218482 (0.0010) [2023-12-27 07:09:58,318][20292] KL-divergence is very high: 100.4591 [2023-12-27 07:09:58,342][20292] KL-divergence is very high: 100.4545 [2023-12-27 07:09:58,362][20381] Updated weights for policy 1, policy_version 218492 (0.0008) [2023-12-27 07:09:58,392][20292] KL-divergence is very high: 119.7349 [2023-12-27 07:09:58,425][20292] KL-divergence is very high: 110.3263 [2023-12-27 07:09:58,426][20381] Updated weights for policy 1, policy_version 218502 (0.0006) [2023-12-27 07:09:58,444][20292] KL-divergence is very high: 103.4969 [2023-12-27 07:09:58,452][20292] KL-divergence is very high: 104.0515 [2023-12-27 07:09:58,472][20292] KL-divergence is very high: 119.6463 [2023-12-27 07:09:58,480][20292] KL-divergence is very high: 172.8597 [2023-12-27 07:09:58,492][20381] Updated weights for policy 1, policy_version 218512 (0.0008) [2023-12-27 07:09:58,890][20347] Updated weights for policy 0, policy_version 218861 (0.0009) [2023-12-27 07:09:58,952][20347] Updated weights for policy 0, policy_version 218871 (0.0008) [2023-12-27 07:09:59,006][20347] Updated weights for policy 0, policy_version 218881 (0.0005) [2023-12-27 07:09:59,176][20292] KL-divergence is very high: 116.0197 [2023-12-27 07:09:59,205][20292] KL-divergence is very high: 106.9406 [2023-12-27 07:09:59,216][20381] Updated weights for policy 1, policy_version 218522 (0.0007) [2023-12-27 07:09:59,273][20381] Updated weights for policy 1, policy_version 218532 (0.0008) [2023-12-27 07:09:59,293][20292] KL-divergence is very high: 150.4337 [2023-12-27 07:09:59,332][20381] Updated weights for policy 1, policy_version 218542 (0.0010) [2023-12-27 07:09:59,643][20347] Updated weights for policy 0, policy_version 218891 (0.0005) [2023-12-27 07:09:59,704][20347] Updated weights for policy 0, policy_version 218901 (0.0005) [2023-12-27 07:09:59,764][20347] Updated weights for policy 0, policy_version 218911 (0.0006) [2023-12-27 07:09:59,988][20381] Updated weights for policy 1, policy_version 218552 (0.0007) [2023-12-27 07:10:00,039][20381] Updated weights for policy 1, policy_version 218562 (0.0008) [2023-12-27 07:10:00,086][20381] Updated weights for policy 1, policy_version 218572 (0.0007) [2023-12-27 07:10:00,325][20347] Updated weights for policy 0, policy_version 218921 (0.0009) [2023-12-27 07:10:00,381][20347] Updated weights for policy 0, policy_version 218931 (0.0006) [2023-12-27 07:10:00,442][20347] Updated weights for policy 0, policy_version 218941 (0.0005) [2023-12-27 07:10:00,503][20347] Updated weights for policy 0, policy_version 218951 (0.0005) [2023-12-27 07:10:00,820][20381] Updated weights for policy 1, policy_version 218582 (0.0010) [2023-12-27 07:10:00,877][20381] Updated weights for policy 1, policy_version 218592 (0.0010) [2023-12-27 07:10:00,934][20381] Updated weights for policy 1, policy_version 218602 (0.0010) [2023-12-27 07:10:01,083][20347] Updated weights for policy 0, policy_version 218961 (0.0010) [2023-12-27 07:10:01,145][20347] Updated weights for policy 0, policy_version 218971 (0.0011) [2023-12-27 07:10:01,203][20347] Updated weights for policy 0, policy_version 218981 (0.0010) [2023-12-27 07:10:01,616][20381] Updated weights for policy 1, policy_version 218612 (0.0009) [2023-12-27 07:10:01,672][20381] Updated weights for policy 1, policy_version 218622 (0.0007) [2023-12-27 07:10:01,733][20381] Updated weights for policy 1, policy_version 218632 (0.0010) [2023-12-27 07:10:01,930][20347] Updated weights for policy 0, policy_version 218991 (0.0008) [2023-12-27 07:10:01,982][20347] Updated weights for policy 0, policy_version 219001 (0.0010) [2023-12-27 07:10:02,037][20347] Updated weights for policy 0, policy_version 219011 (0.0009) [2023-12-27 07:10:02,314][20381] Updated weights for policy 1, policy_version 218642 (0.0010) [2023-12-27 07:10:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 112066560. Throughput: 0: 10409.4, 1: 10512.7. Samples: 112078808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:10:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '247.890')] [2023-12-27 07:10:02,375][20381] Updated weights for policy 1, policy_version 218652 (0.0008) [2023-12-27 07:10:02,426][20381] Updated weights for policy 1, policy_version 218662 (0.0008) [2023-12-27 07:10:02,471][20381] Updated weights for policy 1, policy_version 218672 (0.0007) [2023-12-27 07:10:02,729][20347] Updated weights for policy 0, policy_version 219021 (0.0008) [2023-12-27 07:10:02,784][20347] Updated weights for policy 0, policy_version 219031 (0.0008) [2023-12-27 07:10:02,833][20347] Updated weights for policy 0, policy_version 219041 (0.0007) [2023-12-27 07:10:03,137][20381] Updated weights for policy 1, policy_version 218682 (0.0008) [2023-12-27 07:10:03,187][20381] Updated weights for policy 1, policy_version 218692 (0.0007) [2023-12-27 07:10:03,232][20381] Updated weights for policy 1, policy_version 218702 (0.0007) [2023-12-27 07:10:03,508][20347] Updated weights for policy 0, policy_version 219051 (0.0007) [2023-12-27 07:10:03,563][20347] Updated weights for policy 0, policy_version 219061 (0.0005) [2023-12-27 07:10:03,608][20347] Updated weights for policy 0, policy_version 219071 (0.0005) [2023-12-27 07:10:03,922][20381] Updated weights for policy 1, policy_version 218712 (0.0006) [2023-12-27 07:10:03,987][20381] Updated weights for policy 1, policy_version 218722 (0.0007) [2023-12-27 07:10:04,048][20381] Updated weights for policy 1, policy_version 218732 (0.0008) [2023-12-27 07:10:04,301][20347] Updated weights for policy 0, policy_version 219081 (0.0005) [2023-12-27 07:10:04,359][20347] Updated weights for policy 0, policy_version 219091 (0.0008) [2023-12-27 07:10:04,426][20347] Updated weights for policy 0, policy_version 219101 (0.0008) [2023-12-27 07:10:04,487][20347] Updated weights for policy 0, policy_version 219111 (0.0007) [2023-12-27 07:10:04,681][20381] Updated weights for policy 1, policy_version 218742 (0.0007) [2023-12-27 07:10:04,698][20292] KL-divergence is very high: 112.3416 [2023-12-27 07:10:04,735][20381] Updated weights for policy 1, policy_version 218752 (0.0008) [2023-12-27 07:10:04,740][20292] KL-divergence is very high: 107.3955 [2023-12-27 07:10:04,792][20381] Updated weights for policy 1, policy_version 218762 (0.0005) [2023-12-27 07:10:05,163][20347] Updated weights for policy 0, policy_version 219121 (0.0008) [2023-12-27 07:10:05,220][20347] Updated weights for policy 0, policy_version 219131 (0.0007) [2023-12-27 07:10:05,264][20347] Updated weights for policy 0, policy_version 219141 (0.0007) [2023-12-27 07:10:05,400][20292] KL-divergence is very high: 153.9004 [2023-12-27 07:10:05,404][20381] Updated weights for policy 1, policy_version 218772 (0.0006) [2023-12-27 07:10:05,422][20292] KL-divergence is very high: 114.1710 [2023-12-27 07:10:05,463][20381] Updated weights for policy 1, policy_version 218782 (0.0005) [2023-12-27 07:10:05,493][20292] KL-divergence is very high: 109.8154 [2023-12-27 07:10:05,509][20292] KL-divergence is very high: 102.0015 [2023-12-27 07:10:05,521][20381] Updated weights for policy 1, policy_version 218792 (0.0005) [2023-12-27 07:10:06,023][20347] Updated weights for policy 0, policy_version 219151 (0.0007) [2023-12-27 07:10:06,070][20381] Updated weights for policy 1, policy_version 218802 (0.0006) [2023-12-27 07:10:06,075][20347] Updated weights for policy 0, policy_version 219161 (0.0008) [2023-12-27 07:10:06,133][20381] Updated weights for policy 1, policy_version 218812 (0.0009) [2023-12-27 07:10:06,134][20347] Updated weights for policy 0, policy_version 219171 (0.0007) [2023-12-27 07:10:06,196][20381] Updated weights for policy 1, policy_version 218822 (0.0006) [2023-12-27 07:10:06,259][20381] Updated weights for policy 1, policy_version 218832 (0.0006) [2023-12-27 07:10:06,895][20347] Updated weights for policy 0, policy_version 219181 (0.0007) [2023-12-27 07:10:06,922][20381] Updated weights for policy 1, policy_version 218842 (0.0011) [2023-12-27 07:10:06,950][20347] Updated weights for policy 0, policy_version 219191 (0.0005) [2023-12-27 07:10:06,974][20381] Updated weights for policy 1, policy_version 218852 (0.0010) [2023-12-27 07:10:07,004][20347] Updated weights for policy 0, policy_version 219201 (0.0007) [2023-12-27 07:10:07,028][20381] Updated weights for policy 1, policy_version 218862 (0.0005) [2023-12-27 07:10:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 112173056. Throughput: 0: 10405.6, 1: 10539.4. Samples: 112140996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:10:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '43.730')] [2023-12-27 07:10:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000219208_56131584.pth... [2023-12-27 07:10:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000218864_56041472.pth... [2023-12-27 07:10:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000217648_55730176.pth [2023-12-27 07:10:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000218024_55828480.pth [2023-12-27 07:10:07,598][20381] Updated weights for policy 1, policy_version 218872 (0.0009) [2023-12-27 07:10:07,598][20292] KL-divergence is very high: 164.9425 [2023-12-27 07:10:07,645][20381] Updated weights for policy 1, policy_version 218882 (0.0005) [2023-12-27 07:10:07,692][20381] Updated weights for policy 1, policy_version 218892 (0.0006) [2023-12-27 07:10:07,741][20347] Updated weights for policy 0, policy_version 219211 (0.0007) [2023-12-27 07:10:07,801][20347] Updated weights for policy 0, policy_version 219221 (0.0008) [2023-12-27 07:10:07,859][20347] Updated weights for policy 0, policy_version 219231 (0.0007) [2023-12-27 07:10:08,376][20381] Updated weights for policy 1, policy_version 218902 (0.0010) [2023-12-27 07:10:08,439][20381] Updated weights for policy 1, policy_version 218912 (0.0011) [2023-12-27 07:10:08,501][20381] Updated weights for policy 1, policy_version 218922 (0.0011) [2023-12-27 07:10:08,573][20347] Updated weights for policy 0, policy_version 219241 (0.0008) [2023-12-27 07:10:08,630][20347] Updated weights for policy 0, policy_version 219251 (0.0007) [2023-12-27 07:10:08,688][20347] Updated weights for policy 0, policy_version 219261 (0.0007) [2023-12-27 07:10:08,744][20347] Updated weights for policy 0, policy_version 219271 (0.0007) [2023-12-27 07:10:09,157][20381] Updated weights for policy 1, policy_version 218932 (0.0011) [2023-12-27 07:10:09,202][20381] Updated weights for policy 1, policy_version 218942 (0.0011) [2023-12-27 07:10:09,260][20381] Updated weights for policy 1, policy_version 218952 (0.0011) [2023-12-27 07:10:09,536][20347] Updated weights for policy 0, policy_version 219281 (0.0009) [2023-12-27 07:10:09,589][20347] Updated weights for policy 0, policy_version 219291 (0.0009) [2023-12-27 07:10:09,643][20347] Updated weights for policy 0, policy_version 219301 (0.0009) [2023-12-27 07:10:10,067][20381] Updated weights for policy 1, policy_version 218962 (0.0009) [2023-12-27 07:10:10,126][20381] Updated weights for policy 1, policy_version 218972 (0.0005) [2023-12-27 07:10:10,179][20381] Updated weights for policy 1, policy_version 218982 (0.0005) [2023-12-27 07:10:10,228][20381] Updated weights for policy 1, policy_version 218992 (0.0006) [2023-12-27 07:10:10,389][20347] Updated weights for policy 0, policy_version 219311 (0.0007) [2023-12-27 07:10:10,451][20347] Updated weights for policy 0, policy_version 219321 (0.0006) [2023-12-27 07:10:10,513][20347] Updated weights for policy 0, policy_version 219331 (0.0008) [2023-12-27 07:10:10,871][20381] Updated weights for policy 1, policy_version 219002 (0.0009) [2023-12-27 07:10:10,924][20381] Updated weights for policy 1, policy_version 219012 (0.0008) [2023-12-27 07:10:10,980][20381] Updated weights for policy 1, policy_version 219022 (0.0011) [2023-12-27 07:10:11,134][20347] Updated weights for policy 0, policy_version 219341 (0.0007) [2023-12-27 07:10:11,197][20347] Updated weights for policy 0, policy_version 219351 (0.0008) [2023-12-27 07:10:11,251][20347] Updated weights for policy 0, policy_version 219361 (0.0007) [2023-12-27 07:10:11,734][20381] Updated weights for policy 1, policy_version 219032 (0.0008) [2023-12-27 07:10:11,801][20381] Updated weights for policy 1, policy_version 219042 (0.0006) [2023-12-27 07:10:11,860][20381] Updated weights for policy 1, policy_version 219052 (0.0007) [2023-12-27 07:10:11,915][20347] Updated weights for policy 0, policy_version 219371 (0.0007) [2023-12-27 07:10:11,971][20347] Updated weights for policy 0, policy_version 219381 (0.0007) [2023-12-27 07:10:12,023][20347] Updated weights for policy 0, policy_version 219391 (0.0008) [2023-12-27 07:10:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.7, 300 sec: 20799.4). Total num frames: 112271360. Throughput: 0: 10273.9, 1: 10543.8. Samples: 112262216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:10:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '30.280')] [2023-12-27 07:10:12,467][20381] Updated weights for policy 1, policy_version 219062 (0.0007) [2023-12-27 07:10:12,516][20381] Updated weights for policy 1, policy_version 219072 (0.0008) [2023-12-27 07:10:12,574][20381] Updated weights for policy 1, policy_version 219082 (0.0010) [2023-12-27 07:10:12,673][20347] Updated weights for policy 0, policy_version 219401 (0.0008) [2023-12-27 07:10:12,723][20347] Updated weights for policy 0, policy_version 219411 (0.0008) [2023-12-27 07:10:12,773][20347] Updated weights for policy 0, policy_version 219421 (0.0008) [2023-12-27 07:10:12,828][20347] Updated weights for policy 0, policy_version 219431 (0.0007) [2023-12-27 07:10:13,268][20381] Updated weights for policy 1, policy_version 219092 (0.0008) [2023-12-27 07:10:13,324][20381] Updated weights for policy 1, policy_version 219102 (0.0006) [2023-12-27 07:10:13,372][20381] Updated weights for policy 1, policy_version 219112 (0.0005) [2023-12-27 07:10:13,493][20347] Updated weights for policy 0, policy_version 219441 (0.0007) [2023-12-27 07:10:13,538][20347] Updated weights for policy 0, policy_version 219451 (0.0007) [2023-12-27 07:10:13,585][20347] Updated weights for policy 0, policy_version 219461 (0.0008) [2023-12-27 07:10:14,061][20381] Updated weights for policy 1, policy_version 219122 (0.0006) [2023-12-27 07:10:14,126][20381] Updated weights for policy 1, policy_version 219132 (0.0007) [2023-12-27 07:10:14,188][20381] Updated weights for policy 1, policy_version 219142 (0.0006) [2023-12-27 07:10:14,249][20381] Updated weights for policy 1, policy_version 219152 (0.0006) [2023-12-27 07:10:14,324][20347] Updated weights for policy 0, policy_version 219471 (0.0008) [2023-12-27 07:10:14,375][20347] Updated weights for policy 0, policy_version 219481 (0.0007) [2023-12-27 07:10:14,430][20347] Updated weights for policy 0, policy_version 219491 (0.0008) [2023-12-27 07:10:14,827][20381] Updated weights for policy 1, policy_version 219162 (0.0007) [2023-12-27 07:10:14,898][20381] Updated weights for policy 1, policy_version 219172 (0.0008) [2023-12-27 07:10:14,958][20381] Updated weights for policy 1, policy_version 219182 (0.0011) [2023-12-27 07:10:15,154][20347] Updated weights for policy 0, policy_version 219501 (0.0007) [2023-12-27 07:10:15,213][20347] Updated weights for policy 0, policy_version 219511 (0.0008) [2023-12-27 07:10:15,274][20347] Updated weights for policy 0, policy_version 219521 (0.0007) [2023-12-27 07:10:15,612][20381] Updated weights for policy 1, policy_version 219192 (0.0010) [2023-12-27 07:10:15,658][20381] Updated weights for policy 1, policy_version 219202 (0.0010) [2023-12-27 07:10:15,709][20381] Updated weights for policy 1, policy_version 219212 (0.0010) [2023-12-27 07:10:15,871][20347] Updated weights for policy 0, policy_version 219531 (0.0006) [2023-12-27 07:10:15,923][20347] Updated weights for policy 0, policy_version 219541 (0.0008) [2023-12-27 07:10:15,976][20347] Updated weights for policy 0, policy_version 219551 (0.0008) [2023-12-27 07:10:16,320][20381] Updated weights for policy 1, policy_version 219222 (0.0007) [2023-12-27 07:10:16,371][20381] Updated weights for policy 1, policy_version 219232 (0.0005) [2023-12-27 07:10:16,424][20381] Updated weights for policy 1, policy_version 219242 (0.0005) [2023-12-27 07:10:16,760][20347] Updated weights for policy 0, policy_version 219561 (0.0009) [2023-12-27 07:10:16,811][20347] Updated weights for policy 0, policy_version 219571 (0.0010) [2023-12-27 07:10:16,856][20347] Updated weights for policy 0, policy_version 219581 (0.0010) [2023-12-27 07:10:16,905][20347] Updated weights for policy 0, policy_version 219591 (0.0009) [2023-12-27 07:10:16,980][20381] Updated weights for policy 1, policy_version 219252 (0.0007) [2023-12-27 07:10:17,043][20381] Updated weights for policy 1, policy_version 219262 (0.0011) [2023-12-27 07:10:17,104][20381] Updated weights for policy 1, policy_version 219272 (0.0011) [2023-12-27 07:10:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 112377856. Throughput: 0: 10141.7, 1: 10497.2. Samples: 112386596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:10:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.790')] [2023-12-27 07:10:17,547][20347] Updated weights for policy 0, policy_version 219601 (0.0008) [2023-12-27 07:10:17,594][20347] Updated weights for policy 0, policy_version 219611 (0.0008) [2023-12-27 07:10:17,660][20347] Updated weights for policy 0, policy_version 219621 (0.0008) [2023-12-27 07:10:17,691][20381] Updated weights for policy 1, policy_version 219282 (0.0010) [2023-12-27 07:10:17,748][20381] Updated weights for policy 1, policy_version 219292 (0.0006) [2023-12-27 07:10:17,800][20381] Updated weights for policy 1, policy_version 219302 (0.0005) [2023-12-27 07:10:17,854][20381] Updated weights for policy 1, policy_version 219312 (0.0007) [2023-12-27 07:10:18,359][20347] Updated weights for policy 0, policy_version 219631 (0.0008) [2023-12-27 07:10:18,415][20347] Updated weights for policy 0, policy_version 219641 (0.0008) [2023-12-27 07:10:18,472][20347] Updated weights for policy 0, policy_version 219651 (0.0007) [2023-12-27 07:10:18,502][20381] Updated weights for policy 1, policy_version 219322 (0.0006) [2023-12-27 07:10:18,560][20381] Updated weights for policy 1, policy_version 219332 (0.0007) [2023-12-27 07:10:18,622][20381] Updated weights for policy 1, policy_version 219342 (0.0007) [2023-12-27 07:10:19,196][20347] Updated weights for policy 0, policy_version 219661 (0.0008) [2023-12-27 07:10:19,257][20347] Updated weights for policy 0, policy_version 219671 (0.0006) [2023-12-27 07:10:19,269][20381] Updated weights for policy 1, policy_version 219352 (0.0008) [2023-12-27 07:10:19,313][20347] Updated weights for policy 0, policy_version 219681 (0.0006) [2023-12-27 07:10:19,330][20381] Updated weights for policy 1, policy_version 219362 (0.0011) [2023-12-27 07:10:19,394][20381] Updated weights for policy 1, policy_version 219372 (0.0010) [2023-12-27 07:10:20,028][20347] Updated weights for policy 0, policy_version 219691 (0.0007) [2023-12-27 07:10:20,085][20347] Updated weights for policy 0, policy_version 219701 (0.0008) [2023-12-27 07:10:20,099][20381] Updated weights for policy 1, policy_version 219382 (0.0009) [2023-12-27 07:10:20,147][20347] Updated weights for policy 0, policy_version 219711 (0.0006) [2023-12-27 07:10:20,157][20381] Updated weights for policy 1, policy_version 219392 (0.0011) [2023-12-27 07:10:20,211][20381] Updated weights for policy 1, policy_version 219402 (0.0010) [2023-12-27 07:10:20,833][20347] Updated weights for policy 0, policy_version 219721 (0.0006) [2023-12-27 07:10:20,894][20347] Updated weights for policy 0, policy_version 219731 (0.0009) [2023-12-27 07:10:20,928][20381] Updated weights for policy 1, policy_version 219412 (0.0011) [2023-12-27 07:10:20,952][20347] Updated weights for policy 0, policy_version 219741 (0.0008) [2023-12-27 07:10:20,992][20381] Updated weights for policy 1, policy_version 219422 (0.0010) [2023-12-27 07:10:21,005][20347] Updated weights for policy 0, policy_version 219751 (0.0010) [2023-12-27 07:10:21,060][20381] Updated weights for policy 1, policy_version 219432 (0.0010) [2023-12-27 07:10:21,718][20347] Updated weights for policy 0, policy_version 219761 (0.0008) [2023-12-27 07:10:21,781][20347] Updated weights for policy 0, policy_version 219771 (0.0008) [2023-12-27 07:10:21,817][20381] Updated weights for policy 1, policy_version 219442 (0.0008) [2023-12-27 07:10:21,837][20347] Updated weights for policy 0, policy_version 219781 (0.0006) [2023-12-27 07:10:21,873][20381] Updated weights for policy 1, policy_version 219452 (0.0008) [2023-12-27 07:10:21,929][20381] Updated weights for policy 1, policy_version 219462 (0.0007) [2023-12-27 07:10:21,989][20381] Updated weights for policy 1, policy_version 219472 (0.0008) [2023-12-27 07:10:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 112476160. Throughput: 0: 10118.8, 1: 10521.4. Samples: 112448228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:10:22,363][19170] Avg episode reward: [(0, '991.100'), (1, '25.310')] [2023-12-27 07:10:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000219472_56197120.pth... [2023-12-27 07:10:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000219784_56279040.pth... [2023-12-27 07:10:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000218256_55885824.pth [2023-12-27 07:10:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000218600_55975936.pth [2023-12-27 07:10:22,595][20347] Updated weights for policy 0, policy_version 219791 (0.0007) [2023-12-27 07:10:22,658][20347] Updated weights for policy 0, policy_version 219801 (0.0007) [2023-12-27 07:10:22,669][20381] Updated weights for policy 1, policy_version 219482 (0.0008) [2023-12-27 07:10:22,713][20347] Updated weights for policy 0, policy_version 219811 (0.0008) [2023-12-27 07:10:22,737][20381] Updated weights for policy 1, policy_version 219492 (0.0006) [2023-12-27 07:10:22,805][20381] Updated weights for policy 1, policy_version 219502 (0.0006) [2023-12-27 07:10:23,348][20381] Updated weights for policy 1, policy_version 219512 (0.0010) [2023-12-27 07:10:23,351][20347] Updated weights for policy 0, policy_version 219821 (0.0007) [2023-12-27 07:10:23,401][20381] Updated weights for policy 1, policy_version 219522 (0.0010) [2023-12-27 07:10:23,403][20347] Updated weights for policy 0, policy_version 219831 (0.0006) [2023-12-27 07:10:23,447][20381] Updated weights for policy 1, policy_version 219532 (0.0010) [2023-12-27 07:10:23,459][20347] Updated weights for policy 0, policy_version 219841 (0.0005) [2023-12-27 07:10:24,089][20347] Updated weights for policy 0, policy_version 219851 (0.0007) [2023-12-27 07:10:24,106][20381] Updated weights for policy 1, policy_version 219542 (0.0008) [2023-12-27 07:10:24,148][20347] Updated weights for policy 0, policy_version 219861 (0.0010) [2023-12-27 07:10:24,169][20381] Updated weights for policy 1, policy_version 219552 (0.0007) [2023-12-27 07:10:24,207][20347] Updated weights for policy 0, policy_version 219871 (0.0010) [2023-12-27 07:10:24,224][20381] Updated weights for policy 1, policy_version 219562 (0.0005) [2023-12-27 07:10:24,807][20347] Updated weights for policy 0, policy_version 219881 (0.0010) [2023-12-27 07:10:24,825][20381] Updated weights for policy 1, policy_version 219572 (0.0006) [2023-12-27 07:10:24,868][20347] Updated weights for policy 0, policy_version 219891 (0.0005) [2023-12-27 07:10:24,882][20381] Updated weights for policy 1, policy_version 219582 (0.0008) [2023-12-27 07:10:24,925][20381] Updated weights for policy 1, policy_version 219592 (0.0007) [2023-12-27 07:10:24,929][20347] Updated weights for policy 0, policy_version 219901 (0.0005) [2023-12-27 07:10:24,987][20347] Updated weights for policy 0, policy_version 219911 (0.0006) [2023-12-27 07:10:25,618][20347] Updated weights for policy 0, policy_version 219921 (0.0006) [2023-12-27 07:10:25,636][20381] Updated weights for policy 1, policy_version 219602 (0.0007) [2023-12-27 07:10:25,671][20347] Updated weights for policy 0, policy_version 219931 (0.0006) [2023-12-27 07:10:25,692][20381] Updated weights for policy 1, policy_version 219612 (0.0006) [2023-12-27 07:10:25,727][20347] Updated weights for policy 0, policy_version 219941 (0.0006) [2023-12-27 07:10:25,740][20381] Updated weights for policy 1, policy_version 219622 (0.0005) [2023-12-27 07:10:25,794][20381] Updated weights for policy 1, policy_version 219632 (0.0007) [2023-12-27 07:10:26,386][20347] Updated weights for policy 0, policy_version 219951 (0.0008) [2023-12-27 07:10:26,421][20381] Updated weights for policy 1, policy_version 219642 (0.0010) [2023-12-27 07:10:26,442][20347] Updated weights for policy 0, policy_version 219961 (0.0008) [2023-12-27 07:10:26,471][20381] Updated weights for policy 1, policy_version 219652 (0.0010) [2023-12-27 07:10:26,490][20347] Updated weights for policy 0, policy_version 219971 (0.0008) [2023-12-27 07:10:26,520][20381] Updated weights for policy 1, policy_version 219662 (0.0011) [2023-12-27 07:10:27,156][20347] Updated weights for policy 0, policy_version 219981 (0.0010) [2023-12-27 07:10:27,189][20381] Updated weights for policy 1, policy_version 219672 (0.0007) [2023-12-27 07:10:27,215][20347] Updated weights for policy 0, policy_version 219991 (0.0006) [2023-12-27 07:10:27,239][20381] Updated weights for policy 1, policy_version 219682 (0.0005) [2023-12-27 07:10:27,271][20347] Updated weights for policy 0, policy_version 220001 (0.0009) [2023-12-27 07:10:27,298][20381] Updated weights for policy 1, policy_version 219692 (0.0005) [2023-12-27 07:10:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 112590848. Throughput: 0: 10103.5, 1: 10523.8. Samples: 112572800. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:27,363][19170] Avg episode reward: [(0, '991.100'), (1, '27.190')] [2023-12-27 07:10:27,887][20347] Updated weights for policy 0, policy_version 220011 (0.0009) [2023-12-27 07:10:27,891][20381] Updated weights for policy 1, policy_version 219702 (0.0007) [2023-12-27 07:10:27,937][20347] Updated weights for policy 0, policy_version 220021 (0.0005) [2023-12-27 07:10:27,939][20381] Updated weights for policy 1, policy_version 219712 (0.0008) [2023-12-27 07:10:27,990][20347] Updated weights for policy 0, policy_version 220031 (0.0005) [2023-12-27 07:10:27,994][20381] Updated weights for policy 1, policy_version 219722 (0.0010) [2023-12-27 07:10:28,638][20347] Updated weights for policy 0, policy_version 220041 (0.0005) [2023-12-27 07:10:28,662][20381] Updated weights for policy 1, policy_version 219732 (0.0008) [2023-12-27 07:10:28,693][20347] Updated weights for policy 0, policy_version 220051 (0.0007) [2023-12-27 07:10:28,722][20381] Updated weights for policy 1, policy_version 219742 (0.0006) [2023-12-27 07:10:28,753][20347] Updated weights for policy 0, policy_version 220061 (0.0007) [2023-12-27 07:10:28,779][20381] Updated weights for policy 1, policy_version 219752 (0.0005) [2023-12-27 07:10:28,805][20347] Updated weights for policy 0, policy_version 220071 (0.0007) [2023-12-27 07:10:29,427][20381] Updated weights for policy 1, policy_version 219762 (0.0005) [2023-12-27 07:10:29,490][20381] Updated weights for policy 1, policy_version 219772 (0.0005) [2023-12-27 07:10:29,511][20347] Updated weights for policy 0, policy_version 220081 (0.0007) [2023-12-27 07:10:29,559][20381] Updated weights for policy 1, policy_version 219782 (0.0009) [2023-12-27 07:10:29,564][20347] Updated weights for policy 0, policy_version 220091 (0.0007) [2023-12-27 07:10:29,617][20347] Updated weights for policy 0, policy_version 220101 (0.0007) [2023-12-27 07:10:29,621][20381] Updated weights for policy 1, policy_version 219792 (0.0010) [2023-12-27 07:10:30,227][20347] Updated weights for policy 0, policy_version 220111 (0.0006) [2023-12-27 07:10:30,244][20381] Updated weights for policy 1, policy_version 219802 (0.0011) [2023-12-27 07:10:30,276][20347] Updated weights for policy 0, policy_version 220121 (0.0005) [2023-12-27 07:10:30,293][20381] Updated weights for policy 1, policy_version 219812 (0.0010) [2023-12-27 07:10:30,335][20347] Updated weights for policy 0, policy_version 220131 (0.0006) [2023-12-27 07:10:30,346][20381] Updated weights for policy 1, policy_version 219822 (0.0009) [2023-12-27 07:10:30,951][20347] Updated weights for policy 0, policy_version 220141 (0.0007) [2023-12-27 07:10:30,967][20381] Updated weights for policy 1, policy_version 219832 (0.0009) [2023-12-27 07:10:31,007][20347] Updated weights for policy 0, policy_version 220151 (0.0005) [2023-12-27 07:10:31,022][20381] Updated weights for policy 1, policy_version 219842 (0.0007) [2023-12-27 07:10:31,059][20347] Updated weights for policy 0, policy_version 220161 (0.0007) [2023-12-27 07:10:31,085][20381] Updated weights for policy 1, policy_version 219852 (0.0007) [2023-12-27 07:10:31,754][20381] Updated weights for policy 1, policy_version 219862 (0.0007) [2023-12-27 07:10:31,793][20347] Updated weights for policy 0, policy_version 220171 (0.0008) [2023-12-27 07:10:31,804][20381] Updated weights for policy 1, policy_version 219872 (0.0010) [2023-12-27 07:10:31,844][20347] Updated weights for policy 0, policy_version 220181 (0.0008) [2023-12-27 07:10:31,853][20381] Updated weights for policy 1, policy_version 219882 (0.0011) [2023-12-27 07:10:31,896][20347] Updated weights for policy 0, policy_version 220191 (0.0006) [2023-12-27 07:10:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 112689152. Throughput: 0: 10171.6, 1: 10544.4. Samples: 112700932. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:32,363][19170] Avg episode reward: [(0, '991.100'), (1, '25.310')] [2023-12-27 07:10:32,526][20381] Updated weights for policy 1, policy_version 219892 (0.0010) [2023-12-27 07:10:32,580][20381] Updated weights for policy 1, policy_version 219902 (0.0007) [2023-12-27 07:10:32,627][20347] Updated weights for policy 0, policy_version 220201 (0.0007) [2023-12-27 07:10:32,637][20381] Updated weights for policy 1, policy_version 219912 (0.0007) [2023-12-27 07:10:32,687][20347] Updated weights for policy 0, policy_version 220211 (0.0007) [2023-12-27 07:10:32,742][20347] Updated weights for policy 0, policy_version 220221 (0.0008) [2023-12-27 07:10:32,809][20347] Updated weights for policy 0, policy_version 220231 (0.0007) [2023-12-27 07:10:33,231][20381] Updated weights for policy 1, policy_version 219922 (0.0006) [2023-12-27 07:10:33,278][20381] Updated weights for policy 1, policy_version 219932 (0.0006) [2023-12-27 07:10:33,325][20381] Updated weights for policy 1, policy_version 219942 (0.0007) [2023-12-27 07:10:33,376][20381] Updated weights for policy 1, policy_version 219952 (0.0008) [2023-12-27 07:10:33,488][20347] Updated weights for policy 0, policy_version 220241 (0.0007) [2023-12-27 07:10:33,545][20347] Updated weights for policy 0, policy_version 220251 (0.0005) [2023-12-27 07:10:33,592][20347] Updated weights for policy 0, policy_version 220261 (0.0005) [2023-12-27 07:10:34,021][20381] Updated weights for policy 1, policy_version 219962 (0.0011) [2023-12-27 07:10:34,073][20381] Updated weights for policy 1, policy_version 219972 (0.0011) [2023-12-27 07:10:34,119][20381] Updated weights for policy 1, policy_version 219982 (0.0010) [2023-12-27 07:10:34,132][20347] Updated weights for policy 0, policy_version 220271 (0.0005) [2023-12-27 07:10:34,196][20347] Updated weights for policy 0, policy_version 220281 (0.0008) [2023-12-27 07:10:34,254][20347] Updated weights for policy 0, policy_version 220291 (0.0007) [2023-12-27 07:10:34,855][20381] Updated weights for policy 1, policy_version 219992 (0.0011) [2023-12-27 07:10:34,915][20347] Updated weights for policy 0, policy_version 220301 (0.0007) [2023-12-27 07:10:34,916][20381] Updated weights for policy 1, policy_version 220002 (0.0011) [2023-12-27 07:10:34,973][20381] Updated weights for policy 1, policy_version 220012 (0.0011) [2023-12-27 07:10:34,979][20347] Updated weights for policy 0, policy_version 220311 (0.0006) [2023-12-27 07:10:35,038][20347] Updated weights for policy 0, policy_version 220321 (0.0007) [2023-12-27 07:10:35,655][20381] Updated weights for policy 1, policy_version 220022 (0.0007) [2023-12-27 07:10:35,715][20381] Updated weights for policy 1, policy_version 220032 (0.0010) [2023-12-27 07:10:35,723][20347] Updated weights for policy 0, policy_version 220331 (0.0007) [2023-12-27 07:10:35,770][20381] Updated weights for policy 1, policy_version 220042 (0.0006) [2023-12-27 07:10:35,778][20347] Updated weights for policy 0, policy_version 220341 (0.0006) [2023-12-27 07:10:35,833][20347] Updated weights for policy 0, policy_version 220351 (0.0010) [2023-12-27 07:10:36,422][20381] Updated weights for policy 1, policy_version 220052 (0.0006) [2023-12-27 07:10:36,481][20347] Updated weights for policy 0, policy_version 220361 (0.0010) [2023-12-27 07:10:36,485][20381] Updated weights for policy 1, policy_version 220062 (0.0011) [2023-12-27 07:10:36,539][20347] Updated weights for policy 0, policy_version 220371 (0.0011) [2023-12-27 07:10:36,547][20381] Updated weights for policy 1, policy_version 220072 (0.0008) [2023-12-27 07:10:36,602][20347] Updated weights for policy 0, policy_version 220381 (0.0010) [2023-12-27 07:10:36,663][20347] Updated weights for policy 0, policy_version 220391 (0.0006) [2023-12-27 07:10:37,207][20381] Updated weights for policy 1, policy_version 220082 (0.0010) [2023-12-27 07:10:37,254][20347] Updated weights for policy 0, policy_version 220401 (0.0006) [2023-12-27 07:10:37,268][20381] Updated weights for policy 1, policy_version 220092 (0.0007) [2023-12-27 07:10:37,325][20347] Updated weights for policy 0, policy_version 220411 (0.0006) [2023-12-27 07:10:37,330][20381] Updated weights for policy 1, policy_version 220102 (0.0007) [2023-12-27 07:10:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 112787456. Throughput: 0: 10242.0, 1: 10532.5. Samples: 112764392. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '26.350')] [2023-12-27 07:10:37,384][20347] Updated weights for policy 0, policy_version 220421 (0.0008) [2023-12-27 07:10:37,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000220112_56360960.pth... [2023-12-27 07:10:37,398][20381] Updated weights for policy 1, policy_version 220112 (0.0007) [2023-12-27 07:10:37,399][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000220424_56442880.pth... [2023-12-27 07:10:37,403][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000218864_56041472.pth [2023-12-27 07:10:37,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000219208_56131584.pth [2023-12-27 07:10:37,973][20347] Updated weights for policy 0, policy_version 220431 (0.0007) [2023-12-27 07:10:38,028][20347] Updated weights for policy 0, policy_version 220441 (0.0007) [2023-12-27 07:10:38,062][20381] Updated weights for policy 1, policy_version 220122 (0.0010) [2023-12-27 07:10:38,083][20347] Updated weights for policy 0, policy_version 220451 (0.0005) [2023-12-27 07:10:38,126][20381] Updated weights for policy 1, policy_version 220132 (0.0008) [2023-12-27 07:10:38,189][20381] Updated weights for policy 1, policy_version 220142 (0.0005) [2023-12-27 07:10:38,791][20347] Updated weights for policy 0, policy_version 220461 (0.0005) [2023-12-27 07:10:38,815][20381] Updated weights for policy 1, policy_version 220152 (0.0010) [2023-12-27 07:10:38,841][20347] Updated weights for policy 0, policy_version 220471 (0.0005) [2023-12-27 07:10:38,872][20381] Updated weights for policy 1, policy_version 220162 (0.0011) [2023-12-27 07:10:38,891][20347] Updated weights for policy 0, policy_version 220481 (0.0005) [2023-12-27 07:10:38,931][20381] Updated weights for policy 1, policy_version 220172 (0.0011) [2023-12-27 07:10:39,576][20347] Updated weights for policy 0, policy_version 220491 (0.0006) [2023-12-27 07:10:39,635][20347] Updated weights for policy 0, policy_version 220501 (0.0006) [2023-12-27 07:10:39,643][20381] Updated weights for policy 1, policy_version 220182 (0.0011) [2023-12-27 07:10:39,694][20347] Updated weights for policy 0, policy_version 220511 (0.0008) [2023-12-27 07:10:39,702][20381] Updated weights for policy 1, policy_version 220192 (0.0011) [2023-12-27 07:10:39,765][20381] Updated weights for policy 1, policy_version 220202 (0.0011) [2023-12-27 07:10:40,380][20347] Updated weights for policy 0, policy_version 220521 (0.0007) [2023-12-27 07:10:40,446][20347] Updated weights for policy 0, policy_version 220531 (0.0009) [2023-12-27 07:10:40,479][20381] Updated weights for policy 1, policy_version 220212 (0.0009) [2023-12-27 07:10:40,504][20347] Updated weights for policy 0, policy_version 220541 (0.0010) [2023-12-27 07:10:40,535][20381] Updated weights for policy 1, policy_version 220222 (0.0006) [2023-12-27 07:10:40,560][20347] Updated weights for policy 0, policy_version 220551 (0.0010) [2023-12-27 07:10:40,592][20381] Updated weights for policy 1, policy_version 220232 (0.0006) [2023-12-27 07:10:41,207][20347] Updated weights for policy 0, policy_version 220561 (0.0011) [2023-12-27 07:10:41,265][20347] Updated weights for policy 0, policy_version 220571 (0.0010) [2023-12-27 07:10:41,332][20347] Updated weights for policy 0, policy_version 220581 (0.0010) [2023-12-27 07:10:41,366][20381] Updated weights for policy 1, policy_version 220242 (0.0008) [2023-12-27 07:10:41,423][20381] Updated weights for policy 1, policy_version 220252 (0.0009) [2023-12-27 07:10:41,478][20381] Updated weights for policy 1, policy_version 220262 (0.0006) [2023-12-27 07:10:41,543][20381] Updated weights for policy 1, policy_version 220272 (0.0010) [2023-12-27 07:10:42,030][20347] Updated weights for policy 0, policy_version 220591 (0.0007) [2023-12-27 07:10:42,048][20291] KL-divergence is very high: 239.6968 [2023-12-27 07:10:42,084][20347] Updated weights for policy 0, policy_version 220601 (0.0005) [2023-12-27 07:10:42,091][20291] KL-divergence is very high: 319.2493 [2023-12-27 07:10:42,098][20291] KL-divergence is very high: 321.5430 [2023-12-27 07:10:42,144][20347] Updated weights for policy 0, policy_version 220611 (0.0006) [2023-12-27 07:10:42,223][20381] Updated weights for policy 1, policy_version 220282 (0.0006) [2023-12-27 07:10:42,285][20381] Updated weights for policy 1, policy_version 220292 (0.0007) [2023-12-27 07:10:42,352][20381] Updated weights for policy 1, policy_version 220302 (0.0007) [2023-12-27 07:10:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 112902144. Throughput: 0: 10321.6, 1: 10506.8. Samples: 112887692. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:42,363][19170] Avg episode reward: [(0, '971.310'), (1, '27.330')] [2023-12-27 07:10:42,752][20347] Updated weights for policy 0, policy_version 220621 (0.0005) [2023-12-27 07:10:42,795][20291] KL-divergence is very high: 522.1441 [2023-12-27 07:10:42,814][20347] Updated weights for policy 0, policy_version 220631 (0.0006) [2023-12-27 07:10:42,866][20347] Updated weights for policy 0, policy_version 220641 (0.0010) [2023-12-27 07:10:42,955][20381] Updated weights for policy 1, policy_version 220312 (0.0008) [2023-12-27 07:10:43,011][20381] Updated weights for policy 1, policy_version 220322 (0.0008) [2023-12-27 07:10:43,076][20381] Updated weights for policy 1, policy_version 220332 (0.0005) [2023-12-27 07:10:43,438][20347] Updated weights for policy 0, policy_version 220651 (0.0009) [2023-12-27 07:10:43,486][20347] Updated weights for policy 0, policy_version 220661 (0.0010) [2023-12-27 07:10:43,540][20347] Updated weights for policy 0, policy_version 220671 (0.0010) [2023-12-27 07:10:43,634][20381] Updated weights for policy 1, policy_version 220342 (0.0005) [2023-12-27 07:10:43,679][20381] Updated weights for policy 1, policy_version 220352 (0.0005) [2023-12-27 07:10:43,731][20381] Updated weights for policy 1, policy_version 220362 (0.0005) [2023-12-27 07:10:44,199][20347] Updated weights for policy 0, policy_version 220681 (0.0010) [2023-12-27 07:10:44,262][20347] Updated weights for policy 0, policy_version 220691 (0.0010) [2023-12-27 07:10:44,285][20381] Updated weights for policy 1, policy_version 220372 (0.0007) [2023-12-27 07:10:44,315][20347] Updated weights for policy 0, policy_version 220701 (0.0008) [2023-12-27 07:10:44,342][20381] Updated weights for policy 1, policy_version 220382 (0.0009) [2023-12-27 07:10:44,367][20347] Updated weights for policy 0, policy_version 220711 (0.0008) [2023-12-27 07:10:44,392][20381] Updated weights for policy 1, policy_version 220392 (0.0010) [2023-12-27 07:10:45,024][20381] Updated weights for policy 1, policy_version 220402 (0.0010) [2023-12-27 07:10:45,042][20347] Updated weights for policy 0, policy_version 220721 (0.0008) [2023-12-27 07:10:45,086][20381] Updated weights for policy 1, policy_version 220412 (0.0010) [2023-12-27 07:10:45,111][20347] Updated weights for policy 0, policy_version 220731 (0.0010) [2023-12-27 07:10:45,147][20381] Updated weights for policy 1, policy_version 220422 (0.0011) [2023-12-27 07:10:45,179][20347] Updated weights for policy 0, policy_version 220741 (0.0010) [2023-12-27 07:10:45,206][20381] Updated weights for policy 1, policy_version 220432 (0.0009) [2023-12-27 07:10:45,784][20381] Updated weights for policy 1, policy_version 220442 (0.0011) [2023-12-27 07:10:45,804][20347] Updated weights for policy 0, policy_version 220751 (0.0007) [2023-12-27 07:10:45,838][20381] Updated weights for policy 1, policy_version 220452 (0.0009) [2023-12-27 07:10:45,854][20347] Updated weights for policy 0, policy_version 220761 (0.0005) [2023-12-27 07:10:45,903][20381] Updated weights for policy 1, policy_version 220462 (0.0006) [2023-12-27 07:10:45,906][20347] Updated weights for policy 0, policy_version 220771 (0.0005) [2023-12-27 07:10:46,429][20381] Updated weights for policy 1, policy_version 220472 (0.0006) [2023-12-27 07:10:46,491][20381] Updated weights for policy 1, policy_version 220482 (0.0007) [2023-12-27 07:10:46,540][20347] Updated weights for policy 0, policy_version 220781 (0.0006) [2023-12-27 07:10:46,550][20381] Updated weights for policy 1, policy_version 220492 (0.0005) [2023-12-27 07:10:46,596][20347] Updated weights for policy 0, policy_version 220791 (0.0010) [2023-12-27 07:10:46,650][20347] Updated weights for policy 0, policy_version 220801 (0.0010) [2023-12-27 07:10:47,107][20381] Updated weights for policy 1, policy_version 220502 (0.0008) [2023-12-27 07:10:47,158][20381] Updated weights for policy 1, policy_version 220512 (0.0010) [2023-12-27 07:10:47,217][20381] Updated weights for policy 1, policy_version 220522 (0.0010) [2023-12-27 07:10:47,345][20347] Updated weights for policy 0, policy_version 220811 (0.0008) [2023-12-27 07:10:47,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 113008640. Throughput: 0: 10288.4, 1: 10612.9. Samples: 113019364. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:47,363][19170] Avg episode reward: [(0, '942.260'), (1, '28.240')] [2023-12-27 07:10:47,404][20347] Updated weights for policy 0, policy_version 220821 (0.0006) [2023-12-27 07:10:47,460][20347] Updated weights for policy 0, policy_version 220831 (0.0007) [2023-12-27 07:10:47,931][20381] Updated weights for policy 1, policy_version 220532 (0.0009) [2023-12-27 07:10:47,996][20381] Updated weights for policy 1, policy_version 220542 (0.0008) [2023-12-27 07:10:48,052][20381] Updated weights for policy 1, policy_version 220552 (0.0008) [2023-12-27 07:10:48,091][20347] Updated weights for policy 0, policy_version 220841 (0.0007) [2023-12-27 07:10:48,141][20347] Updated weights for policy 0, policy_version 220851 (0.0006) [2023-12-27 07:10:48,189][20347] Updated weights for policy 0, policy_version 220861 (0.0007) [2023-12-27 07:10:48,235][20347] Updated weights for policy 0, policy_version 220871 (0.0007) [2023-12-27 07:10:48,806][20381] Updated weights for policy 1, policy_version 220562 (0.0008) [2023-12-27 07:10:48,860][20381] Updated weights for policy 1, policy_version 220572 (0.0008) [2023-12-27 07:10:48,884][20347] Updated weights for policy 0, policy_version 220881 (0.0007) [2023-12-27 07:10:48,917][20381] Updated weights for policy 1, policy_version 220582 (0.0006) [2023-12-27 07:10:48,941][20291] KL-divergence is very high: 110.1103 [2023-12-27 07:10:48,941][20347] Updated weights for policy 0, policy_version 220891 (0.0006) [2023-12-27 07:10:48,973][20381] Updated weights for policy 1, policy_version 220592 (0.0007) [2023-12-27 07:10:48,985][20291] KL-divergence is very high: 255.5722 [2023-12-27 07:10:48,995][20347] Updated weights for policy 0, policy_version 220901 (0.0007) [2023-12-27 07:10:49,670][20381] Updated weights for policy 1, policy_version 220602 (0.0011) [2023-12-27 07:10:49,718][20347] Updated weights for policy 0, policy_version 220911 (0.0006) [2023-12-27 07:10:49,727][20381] Updated weights for policy 1, policy_version 220612 (0.0011) [2023-12-27 07:10:49,779][20347] Updated weights for policy 0, policy_version 220921 (0.0006) [2023-12-27 07:10:49,785][20381] Updated weights for policy 1, policy_version 220622 (0.0010) [2023-12-27 07:10:49,841][20347] Updated weights for policy 0, policy_version 220931 (0.0007) [2023-12-27 07:10:50,433][20381] Updated weights for policy 1, policy_version 220632 (0.0008) [2023-12-27 07:10:50,481][20381] Updated weights for policy 1, policy_version 220642 (0.0009) [2023-12-27 07:10:50,531][20381] Updated weights for policy 1, policy_version 220652 (0.0007) [2023-12-27 07:10:50,537][20347] Updated weights for policy 0, policy_version 220941 (0.0005) [2023-12-27 07:10:50,598][20347] Updated weights for policy 0, policy_version 220951 (0.0007) [2023-12-27 07:10:50,655][20347] Updated weights for policy 0, policy_version 220961 (0.0006) [2023-12-27 07:10:51,209][20291] KL-divergence is very high: 232.6414 [2023-12-27 07:10:51,209][20381] Updated weights for policy 1, policy_version 220662 (0.0007) [2023-12-27 07:10:51,219][20347] Updated weights for policy 0, policy_version 220971 (0.0006) [2023-12-27 07:10:51,263][20291] KL-divergence is very high: 476.2012 [2023-12-27 07:10:51,274][20381] Updated weights for policy 1, policy_version 220672 (0.0008) [2023-12-27 07:10:51,287][20347] Updated weights for policy 0, policy_version 220981 (0.0006) [2023-12-27 07:10:51,311][20291] KL-divergence is very high: 274.8895 [2023-12-27 07:10:51,333][20381] Updated weights for policy 1, policy_version 220682 (0.0006) [2023-12-27 07:10:51,344][20347] Updated weights for policy 0, policy_version 220991 (0.0007) [2023-12-27 07:10:51,981][20347] Updated weights for policy 0, policy_version 221001 (0.0007) [2023-12-27 07:10:52,028][20291] KL-divergence is very high: 149.3280 [2023-12-27 07:10:52,038][20347] Updated weights for policy 0, policy_version 221011 (0.0007) [2023-12-27 07:10:52,047][20381] Updated weights for policy 1, policy_version 220692 (0.0007) [2023-12-27 07:10:52,073][20291] KL-divergence is very high: 158.5318 [2023-12-27 07:10:52,096][20347] Updated weights for policy 0, policy_version 221021 (0.0007) [2023-12-27 07:10:52,102][20381] Updated weights for policy 1, policy_version 220702 (0.0006) [2023-12-27 07:10:52,151][20347] Updated weights for policy 0, policy_version 221031 (0.0006) [2023-12-27 07:10:52,159][20381] Updated weights for policy 1, policy_version 220712 (0.0005) [2023-12-27 07:10:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 113115136. Throughput: 0: 10307.0, 1: 10558.9. Samples: 113079964. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:52,363][19170] Avg episode reward: [(0, '952.880'), (1, '30.260')] [2023-12-27 07:10:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000221032_56598528.pth... [2023-12-27 07:10:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000220720_56516608.pth... [2023-12-27 07:10:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000219472_56197120.pth [2023-12-27 07:10:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000219784_56279040.pth [2023-12-27 07:10:52,778][20381] Updated weights for policy 1, policy_version 220722 (0.0006) [2023-12-27 07:10:52,839][20381] Updated weights for policy 1, policy_version 220732 (0.0007) [2023-12-27 07:10:52,887][20347] Updated weights for policy 0, policy_version 221041 (0.0008) [2023-12-27 07:10:52,895][20381] Updated weights for policy 1, policy_version 220742 (0.0009) [2023-12-27 07:10:52,936][20347] Updated weights for policy 0, policy_version 221051 (0.0006) [2023-12-27 07:10:52,949][20381] Updated weights for policy 1, policy_version 220752 (0.0007) [2023-12-27 07:10:52,985][20347] Updated weights for policy 0, policy_version 221061 (0.0008) [2023-12-27 07:10:53,606][20347] Updated weights for policy 0, policy_version 221071 (0.0008) [2023-12-27 07:10:53,653][20347] Updated weights for policy 0, policy_version 221081 (0.0006) [2023-12-27 07:10:53,678][20381] Updated weights for policy 1, policy_version 220762 (0.0008) [2023-12-27 07:10:53,699][20347] Updated weights for policy 0, policy_version 221091 (0.0005) [2023-12-27 07:10:53,735][20381] Updated weights for policy 1, policy_version 220772 (0.0007) [2023-12-27 07:10:53,785][20381] Updated weights for policy 1, policy_version 220782 (0.0008) [2023-12-27 07:10:54,293][20347] Updated weights for policy 0, policy_version 221101 (0.0006) [2023-12-27 07:10:54,358][20347] Updated weights for policy 0, policy_version 221111 (0.0008) [2023-12-27 07:10:54,423][20347] Updated weights for policy 0, policy_version 221121 (0.0008) [2023-12-27 07:10:54,508][20381] Updated weights for policy 1, policy_version 220792 (0.0008) [2023-12-27 07:10:54,556][20381] Updated weights for policy 1, policy_version 220802 (0.0007) [2023-12-27 07:10:54,611][20381] Updated weights for policy 1, policy_version 220812 (0.0009) [2023-12-27 07:10:55,040][20347] Updated weights for policy 0, policy_version 221131 (0.0006) [2023-12-27 07:10:55,096][20347] Updated weights for policy 0, policy_version 221141 (0.0008) [2023-12-27 07:10:55,152][20347] Updated weights for policy 0, policy_version 221151 (0.0007) [2023-12-27 07:10:55,349][20381] Updated weights for policy 1, policy_version 220822 (0.0009) [2023-12-27 07:10:55,410][20381] Updated weights for policy 1, policy_version 220832 (0.0008) [2023-12-27 07:10:55,475][20381] Updated weights for policy 1, policy_version 220842 (0.0008) [2023-12-27 07:10:55,820][20347] Updated weights for policy 0, policy_version 221161 (0.0007) [2023-12-27 07:10:55,869][20347] Updated weights for policy 0, policy_version 221171 (0.0007) [2023-12-27 07:10:55,921][20347] Updated weights for policy 0, policy_version 221181 (0.0007) [2023-12-27 07:10:55,976][20347] Updated weights for policy 0, policy_version 221191 (0.0007) [2023-12-27 07:10:56,160][20381] Updated weights for policy 1, policy_version 220852 (0.0008) [2023-12-27 07:10:56,208][20381] Updated weights for policy 1, policy_version 220862 (0.0007) [2023-12-27 07:10:56,254][20381] Updated weights for policy 1, policy_version 220872 (0.0008) [2023-12-27 07:10:56,670][20347] Updated weights for policy 0, policy_version 221201 (0.0007) [2023-12-27 07:10:56,723][20347] Updated weights for policy 0, policy_version 221211 (0.0007) [2023-12-27 07:10:56,771][20347] Updated weights for policy 0, policy_version 221221 (0.0007) [2023-12-27 07:10:56,776][20291] KL-divergence is very high: 175.8170 [2023-12-27 07:10:56,953][20381] Updated weights for policy 1, policy_version 220882 (0.0008) [2023-12-27 07:10:57,010][20381] Updated weights for policy 1, policy_version 220892 (0.0007) [2023-12-27 07:10:57,065][20381] Updated weights for policy 1, policy_version 220902 (0.0007) [2023-12-27 07:10:57,114][20381] Updated weights for policy 1, policy_version 220912 (0.0009) [2023-12-27 07:10:57,360][20347] Updated weights for policy 0, policy_version 221231 (0.0008) [2023-12-27 07:10:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 113213440. Throughput: 0: 10434.2, 1: 10490.7. Samples: 113203840. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:10:57,363][19170] Avg episode reward: [(0, '953.070'), (1, '27.760')] [2023-12-27 07:10:57,421][20347] Updated weights for policy 0, policy_version 221241 (0.0010) [2023-12-27 07:10:57,479][20347] Updated weights for policy 0, policy_version 221251 (0.0010) [2023-12-27 07:10:57,825][20381] Updated weights for policy 1, policy_version 220922 (0.0008) [2023-12-27 07:10:57,870][20381] Updated weights for policy 1, policy_version 220932 (0.0007) [2023-12-27 07:10:57,923][20381] Updated weights for policy 1, policy_version 220942 (0.0007) [2023-12-27 07:10:58,124][20291] KL-divergence is very high: 105.7374 [2023-12-27 07:10:58,136][20347] Updated weights for policy 0, policy_version 221261 (0.0010) [2023-12-27 07:10:58,171][20291] KL-divergence is very high: 196.3632 [2023-12-27 07:10:58,194][20347] Updated weights for policy 0, policy_version 221271 (0.0009) [2023-12-27 07:10:58,200][20291] KL-divergence is very high: 135.3660 [2023-12-27 07:10:58,218][20291] KL-divergence is very high: 647.0838 [2023-12-27 07:10:58,253][20347] Updated weights for policy 0, policy_version 221281 (0.0008) [2023-12-27 07:10:58,692][20381] Updated weights for policy 1, policy_version 220952 (0.0008) [2023-12-27 07:10:58,758][20381] Updated weights for policy 1, policy_version 220962 (0.0007) [2023-12-27 07:10:58,825][20381] Updated weights for policy 1, policy_version 220972 (0.0007) [2023-12-27 07:10:58,995][20347] Updated weights for policy 0, policy_version 221291 (0.0010) [2023-12-27 07:10:59,051][20347] Updated weights for policy 0, policy_version 221301 (0.0008) [2023-12-27 07:10:59,109][20347] Updated weights for policy 0, policy_version 221311 (0.0007) [2023-12-27 07:10:59,126][20291] KL-divergence is very high: 546.8510 [2023-12-27 07:10:59,132][20291] KL-divergence is very high: 424.6807 [2023-12-27 07:10:59,534][20381] Updated weights for policy 1, policy_version 220982 (0.0006) [2023-12-27 07:10:59,582][20381] Updated weights for policy 1, policy_version 220992 (0.0005) [2023-12-27 07:10:59,640][20381] Updated weights for policy 1, policy_version 221002 (0.0008) [2023-12-27 07:10:59,828][20347] Updated weights for policy 0, policy_version 221321 (0.0007) [2023-12-27 07:10:59,878][20291] KL-divergence is very high: 228.7732 [2023-12-27 07:10:59,884][20347] Updated weights for policy 0, policy_version 221331 (0.0008) [2023-12-27 07:10:59,927][20291] KL-divergence is very high: 160.8066 [2023-12-27 07:10:59,943][20347] Updated weights for policy 0, policy_version 221341 (0.0007) [2023-12-27 07:10:59,988][20347] Updated weights for policy 0, policy_version 221351 (0.0007) [2023-12-27 07:11:00,337][20381] Updated weights for policy 1, policy_version 221012 (0.0009) [2023-12-27 07:11:00,391][20381] Updated weights for policy 1, policy_version 221022 (0.0010) [2023-12-27 07:11:00,450][20381] Updated weights for policy 1, policy_version 221032 (0.0010) [2023-12-27 07:11:00,730][20347] Updated weights for policy 0, policy_version 221361 (0.0007) [2023-12-27 07:11:00,780][20347] Updated weights for policy 0, policy_version 221371 (0.0008) [2023-12-27 07:11:00,828][20347] Updated weights for policy 0, policy_version 221381 (0.0008) [2023-12-27 07:11:01,031][20381] Updated weights for policy 1, policy_version 221042 (0.0010) [2023-12-27 07:11:01,081][20381] Updated weights for policy 1, policy_version 221052 (0.0007) [2023-12-27 07:11:01,140][20381] Updated weights for policy 1, policy_version 221062 (0.0008) [2023-12-27 07:11:01,193][20381] Updated weights for policy 1, policy_version 221072 (0.0007) [2023-12-27 07:11:01,591][20347] Updated weights for policy 0, policy_version 221392 (0.0008) [2023-12-27 07:11:01,664][20347] Updated weights for policy 0, policy_version 221402 (0.0007) [2023-12-27 07:11:01,726][20347] Updated weights for policy 0, policy_version 221412 (0.0008) [2023-12-27 07:11:01,883][20381] Updated weights for policy 1, policy_version 221082 (0.0007) [2023-12-27 07:11:01,936][20381] Updated weights for policy 1, policy_version 221092 (0.0007) [2023-12-27 07:11:01,988][20381] Updated weights for policy 1, policy_version 221102 (0.0008) [2023-12-27 07:11:02,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 113311744. Throughput: 0: 10430.7, 1: 10419.3. Samples: 113324852. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:02,363][19170] Avg episode reward: [(0, '942.190'), (1, '27.230')] [2023-12-27 07:11:02,421][20347] Updated weights for policy 0, policy_version 221422 (0.0006) [2023-12-27 07:11:02,470][20347] Updated weights for policy 0, policy_version 221432 (0.0005) [2023-12-27 07:11:02,525][20347] Updated weights for policy 0, policy_version 221442 (0.0006) [2023-12-27 07:11:02,665][20381] Updated weights for policy 1, policy_version 221112 (0.0006) [2023-12-27 07:11:02,723][20381] Updated weights for policy 1, policy_version 221122 (0.0007) [2023-12-27 07:11:02,791][20381] Updated weights for policy 1, policy_version 221132 (0.0007) [2023-12-27 07:11:03,123][20347] Updated weights for policy 0, policy_version 221452 (0.0007) [2023-12-27 07:11:03,188][20347] Updated weights for policy 0, policy_version 221462 (0.0007) [2023-12-27 07:11:03,245][20347] Updated weights for policy 0, policy_version 221472 (0.0008) [2023-12-27 07:11:03,379][20381] Updated weights for policy 1, policy_version 221142 (0.0008) [2023-12-27 07:11:03,435][20381] Updated weights for policy 1, policy_version 221152 (0.0009) [2023-12-27 07:11:03,491][20381] Updated weights for policy 1, policy_version 221162 (0.0005) [2023-12-27 07:11:03,944][20347] Updated weights for policy 0, policy_version 221482 (0.0007) [2023-12-27 07:11:04,003][20347] Updated weights for policy 0, policy_version 221492 (0.0008) [2023-12-27 07:11:04,067][20347] Updated weights for policy 0, policy_version 221502 (0.0007) [2023-12-27 07:11:04,129][20347] Updated weights for policy 0, policy_version 221512 (0.0009) [2023-12-27 07:11:04,147][20381] Updated weights for policy 1, policy_version 221172 (0.0008) [2023-12-27 07:11:04,209][20381] Updated weights for policy 1, policy_version 221182 (0.0010) [2023-12-27 07:11:04,262][20381] Updated weights for policy 1, policy_version 221192 (0.0007) [2023-12-27 07:11:04,832][20381] Updated weights for policy 1, policy_version 221202 (0.0008) [2023-12-27 07:11:04,842][20347] Updated weights for policy 0, policy_version 221522 (0.0008) [2023-12-27 07:11:04,888][20347] Updated weights for policy 0, policy_version 221532 (0.0006) [2023-12-27 07:11:04,890][20381] Updated weights for policy 1, policy_version 221212 (0.0009) [2023-12-27 07:11:04,940][20347] Updated weights for policy 0, policy_version 221542 (0.0008) [2023-12-27 07:11:04,952][20381] Updated weights for policy 1, policy_version 221222 (0.0008) [2023-12-27 07:11:05,014][20381] Updated weights for policy 1, policy_version 221232 (0.0008) [2023-12-27 07:11:05,618][20347] Updated weights for policy 0, policy_version 221552 (0.0008) [2023-12-27 07:11:05,664][20381] Updated weights for policy 1, policy_version 221242 (0.0010) [2023-12-27 07:11:05,674][20347] Updated weights for policy 0, policy_version 221562 (0.0006) [2023-12-27 07:11:05,732][20381] Updated weights for policy 1, policy_version 221252 (0.0008) [2023-12-27 07:11:05,741][20347] Updated weights for policy 0, policy_version 221572 (0.0006) [2023-12-27 07:11:05,792][20381] Updated weights for policy 1, policy_version 221262 (0.0010) [2023-12-27 07:11:06,386][20347] Updated weights for policy 0, policy_version 221582 (0.0008) [2023-12-27 07:11:06,446][20347] Updated weights for policy 0, policy_version 221592 (0.0007) [2023-12-27 07:11:06,480][20381] Updated weights for policy 1, policy_version 221272 (0.0010) [2023-12-27 07:11:06,496][20347] Updated weights for policy 0, policy_version 221602 (0.0007) [2023-12-27 07:11:06,544][20381] Updated weights for policy 1, policy_version 221282 (0.0011) [2023-12-27 07:11:06,607][20381] Updated weights for policy 1, policy_version 221292 (0.0010) [2023-12-27 07:11:07,085][20347] Updated weights for policy 0, policy_version 221612 (0.0006) [2023-12-27 07:11:07,146][20347] Updated weights for policy 0, policy_version 221622 (0.0007) [2023-12-27 07:11:07,204][20347] Updated weights for policy 0, policy_version 221632 (0.0005) [2023-12-27 07:11:07,261][20381] Updated weights for policy 1, policy_version 221302 (0.0008) [2023-12-27 07:11:07,314][20381] Updated weights for policy 1, policy_version 221312 (0.0008) [2023-12-27 07:11:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 113418240. Throughput: 0: 10441.8, 1: 10452.0. Samples: 113388448. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:07,363][19170] Avg episode reward: [(0, '971.070'), (1, '30.150')] [2023-12-27 07:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000221640_56754176.pth... [2023-12-27 07:11:07,371][20381] Updated weights for policy 1, policy_version 221322 (0.0007) [2023-12-27 07:11:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000220424_56442880.pth [2023-12-27 07:11:07,406][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000221328_56672256.pth... [2023-12-27 07:11:07,410][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000220112_56360960.pth [2023-12-27 07:11:07,879][20347] Updated weights for policy 0, policy_version 221642 (0.0007) [2023-12-27 07:11:07,938][20347] Updated weights for policy 0, policy_version 221652 (0.0007) [2023-12-27 07:11:07,997][20347] Updated weights for policy 0, policy_version 221662 (0.0006) [2023-12-27 07:11:08,053][20381] Updated weights for policy 1, policy_version 221332 (0.0008) [2023-12-27 07:11:08,055][20347] Updated weights for policy 0, policy_version 221672 (0.0007) [2023-12-27 07:11:08,116][20381] Updated weights for policy 1, policy_version 221342 (0.0011) [2023-12-27 07:11:08,174][20381] Updated weights for policy 1, policy_version 221352 (0.0010) [2023-12-27 07:11:08,760][20347] Updated weights for policy 0, policy_version 221682 (0.0007) [2023-12-27 07:11:08,816][20347] Updated weights for policy 0, policy_version 221692 (0.0008) [2023-12-27 07:11:08,860][20381] Updated weights for policy 1, policy_version 221362 (0.0006) [2023-12-27 07:11:08,863][20347] Updated weights for policy 0, policy_version 221702 (0.0007) [2023-12-27 07:11:08,919][20381] Updated weights for policy 1, policy_version 221372 (0.0011) [2023-12-27 07:11:08,974][20381] Updated weights for policy 1, policy_version 221382 (0.0011) [2023-12-27 07:11:09,021][20381] Updated weights for policy 1, policy_version 221392 (0.0008) [2023-12-27 07:11:09,584][20347] Updated weights for policy 0, policy_version 221712 (0.0009) [2023-12-27 07:11:09,644][20347] Updated weights for policy 0, policy_version 221722 (0.0008) [2023-12-27 07:11:09,700][20347] Updated weights for policy 0, policy_version 221732 (0.0008) [2023-12-27 07:11:09,726][20381] Updated weights for policy 1, policy_version 221402 (0.0011) [2023-12-27 07:11:09,776][20381] Updated weights for policy 1, policy_version 221412 (0.0011) [2023-12-27 07:11:09,838][20381] Updated weights for policy 1, policy_version 221422 (0.0009) [2023-12-27 07:11:10,465][20347] Updated weights for policy 0, policy_version 221742 (0.0008) [2023-12-27 07:11:10,522][20347] Updated weights for policy 0, policy_version 221752 (0.0007) [2023-12-27 07:11:10,542][20381] Updated weights for policy 1, policy_version 221432 (0.0010) [2023-12-27 07:11:10,570][20347] Updated weights for policy 0, policy_version 221762 (0.0007) [2023-12-27 07:11:10,595][20381] Updated weights for policy 1, policy_version 221442 (0.0011) [2023-12-27 07:11:10,651][20381] Updated weights for policy 1, policy_version 221452 (0.0011) [2023-12-27 07:11:11,308][20347] Updated weights for policy 0, policy_version 221772 (0.0006) [2023-12-27 07:11:11,363][20381] Updated weights for policy 1, policy_version 221462 (0.0011) [2023-12-27 07:11:11,371][20347] Updated weights for policy 0, policy_version 221782 (0.0007) [2023-12-27 07:11:11,430][20347] Updated weights for policy 0, policy_version 221792 (0.0007) [2023-12-27 07:11:11,434][20381] Updated weights for policy 1, policy_version 221472 (0.0008) [2023-12-27 07:11:11,497][20381] Updated weights for policy 1, policy_version 221482 (0.0008) [2023-12-27 07:11:12,115][20347] Updated weights for policy 0, policy_version 221802 (0.0007) [2023-12-27 07:11:12,172][20347] Updated weights for policy 0, policy_version 221812 (0.0007) [2023-12-27 07:11:12,218][20381] Updated weights for policy 1, policy_version 221492 (0.0009) [2023-12-27 07:11:12,225][20347] Updated weights for policy 0, policy_version 221822 (0.0006) [2023-12-27 07:11:12,280][20381] Updated weights for policy 1, policy_version 221502 (0.0006) [2023-12-27 07:11:12,284][20347] Updated weights for policy 0, policy_version 221832 (0.0007) [2023-12-27 07:11:12,339][20381] Updated weights for policy 1, policy_version 221512 (0.0006) [2023-12-27 07:11:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 113516544. Throughput: 0: 10418.1, 1: 10393.9. Samples: 113509344. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '29.110')] [2023-12-27 07:11:12,922][20347] Updated weights for policy 0, policy_version 221842 (0.0007) [2023-12-27 07:11:12,939][20381] Updated weights for policy 1, policy_version 221522 (0.0008) [2023-12-27 07:11:12,976][20347] Updated weights for policy 0, policy_version 221852 (0.0007) [2023-12-27 07:11:12,996][20381] Updated weights for policy 1, policy_version 221532 (0.0007) [2023-12-27 07:11:13,024][20347] Updated weights for policy 0, policy_version 221862 (0.0005) [2023-12-27 07:11:13,045][20381] Updated weights for policy 1, policy_version 221542 (0.0005) [2023-12-27 07:11:13,098][20381] Updated weights for policy 1, policy_version 221552 (0.0006) [2023-12-27 07:11:13,668][20381] Updated weights for policy 1, policy_version 221562 (0.0005) [2023-12-27 07:11:13,688][20347] Updated weights for policy 0, policy_version 221872 (0.0007) [2023-12-27 07:11:13,721][20381] Updated weights for policy 1, policy_version 221572 (0.0005) [2023-12-27 07:11:13,738][20347] Updated weights for policy 0, policy_version 221882 (0.0006) [2023-12-27 07:11:13,780][20381] Updated weights for policy 1, policy_version 221582 (0.0008) [2023-12-27 07:11:13,787][20347] Updated weights for policy 0, policy_version 221892 (0.0006) [2023-12-27 07:11:14,397][20381] Updated weights for policy 1, policy_version 221592 (0.0007) [2023-12-27 07:11:14,453][20381] Updated weights for policy 1, policy_version 221602 (0.0007) [2023-12-27 07:11:14,511][20347] Updated weights for policy 0, policy_version 221902 (0.0007) [2023-12-27 07:11:14,514][20381] Updated weights for policy 1, policy_version 221612 (0.0007) [2023-12-27 07:11:14,567][20347] Updated weights for policy 0, policy_version 221912 (0.0008) [2023-12-27 07:11:14,619][20347] Updated weights for policy 0, policy_version 221922 (0.0007) [2023-12-27 07:11:15,239][20381] Updated weights for policy 1, policy_version 221622 (0.0008) [2023-12-27 07:11:15,297][20381] Updated weights for policy 1, policy_version 221632 (0.0008) [2023-12-27 07:11:15,330][20347] Updated weights for policy 0, policy_version 221932 (0.0007) [2023-12-27 07:11:15,355][20381] Updated weights for policy 1, policy_version 221642 (0.0010) [2023-12-27 07:11:15,390][20347] Updated weights for policy 0, policy_version 221942 (0.0006) [2023-12-27 07:11:15,448][20347] Updated weights for policy 0, policy_version 221952 (0.0007) [2023-12-27 07:11:16,017][20381] Updated weights for policy 1, policy_version 221652 (0.0010) [2023-12-27 07:11:16,069][20381] Updated weights for policy 1, policy_version 221662 (0.0010) [2023-12-27 07:11:16,131][20381] Updated weights for policy 1, policy_version 221672 (0.0011) [2023-12-27 07:11:16,154][20347] Updated weights for policy 0, policy_version 221962 (0.0007) [2023-12-27 07:11:16,203][20347] Updated weights for policy 0, policy_version 221972 (0.0006) [2023-12-27 07:11:16,251][20347] Updated weights for policy 0, policy_version 221982 (0.0007) [2023-12-27 07:11:16,296][20347] Updated weights for policy 0, policy_version 221992 (0.0010) [2023-12-27 07:11:16,751][20381] Updated weights for policy 1, policy_version 221682 (0.0011) [2023-12-27 07:11:16,810][20381] Updated weights for policy 1, policy_version 221692 (0.0011) [2023-12-27 07:11:16,871][20381] Updated weights for policy 1, policy_version 221702 (0.0011) [2023-12-27 07:11:16,930][20381] Updated weights for policy 1, policy_version 221712 (0.0011) [2023-12-27 07:11:17,013][20347] Updated weights for policy 0, policy_version 222002 (0.0008) [2023-12-27 07:11:17,069][20347] Updated weights for policy 0, policy_version 222012 (0.0007) [2023-12-27 07:11:17,126][20347] Updated weights for policy 0, policy_version 222022 (0.0008) [2023-12-27 07:11:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 113623040. Throughput: 0: 10342.7, 1: 10376.0. Samples: 113633272. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '28.710')] [2023-12-27 07:11:17,512][20381] Updated weights for policy 1, policy_version 221722 (0.0011) [2023-12-27 07:11:17,569][20381] Updated weights for policy 1, policy_version 221732 (0.0010) [2023-12-27 07:11:17,628][20381] Updated weights for policy 1, policy_version 221742 (0.0010) [2023-12-27 07:11:17,776][20347] Updated weights for policy 0, policy_version 222032 (0.0007) [2023-12-27 07:11:17,831][20347] Updated weights for policy 0, policy_version 222042 (0.0007) [2023-12-27 07:11:17,887][20347] Updated weights for policy 0, policy_version 222052 (0.0006) [2023-12-27 07:11:18,327][20381] Updated weights for policy 1, policy_version 221752 (0.0011) [2023-12-27 07:11:18,390][20381] Updated weights for policy 1, policy_version 221762 (0.0009) [2023-12-27 07:11:18,448][20381] Updated weights for policy 1, policy_version 221772 (0.0008) [2023-12-27 07:11:18,543][20347] Updated weights for policy 0, policy_version 222062 (0.0008) [2023-12-27 07:11:18,605][20347] Updated weights for policy 0, policy_version 222072 (0.0010) [2023-12-27 07:11:18,667][20347] Updated weights for policy 0, policy_version 222082 (0.0011) [2023-12-27 07:11:19,145][20381] Updated weights for policy 1, policy_version 221782 (0.0008) [2023-12-27 07:11:19,199][20381] Updated weights for policy 1, policy_version 221792 (0.0008) [2023-12-27 07:11:19,256][20381] Updated weights for policy 1, policy_version 221802 (0.0007) [2023-12-27 07:11:19,327][20347] Updated weights for policy 0, policy_version 222092 (0.0010) [2023-12-27 07:11:19,389][20347] Updated weights for policy 0, policy_version 222102 (0.0010) [2023-12-27 07:11:19,437][20347] Updated weights for policy 0, policy_version 222112 (0.0010) [2023-12-27 07:11:20,002][20381] Updated weights for policy 1, policy_version 221812 (0.0007) [2023-12-27 07:11:20,050][20381] Updated weights for policy 1, policy_version 221822 (0.0008) [2023-12-27 07:11:20,094][20347] Updated weights for policy 0, policy_version 222122 (0.0010) [2023-12-27 07:11:20,103][20381] Updated weights for policy 1, policy_version 221832 (0.0008) [2023-12-27 07:11:20,142][20291] KL-divergence is very high: 123.2811 [2023-12-27 07:11:20,163][20347] Updated weights for policy 0, policy_version 222132 (0.0007) [2023-12-27 07:11:20,170][20291] KL-divergence is very high: 411.8993 [2023-12-27 07:11:20,227][20347] Updated weights for policy 0, policy_version 222142 (0.0006) [2023-12-27 07:11:20,292][20347] Updated weights for policy 0, policy_version 222152 (0.0007) [2023-12-27 07:11:20,882][20381] Updated weights for policy 1, policy_version 221842 (0.0007) [2023-12-27 07:11:20,925][20347] Updated weights for policy 0, policy_version 222162 (0.0005) [2023-12-27 07:11:20,947][20381] Updated weights for policy 1, policy_version 221852 (0.0007) [2023-12-27 07:11:20,985][20347] Updated weights for policy 0, policy_version 222172 (0.0007) [2023-12-27 07:11:21,010][20381] Updated weights for policy 1, policy_version 221862 (0.0008) [2023-12-27 07:11:21,051][20347] Updated weights for policy 0, policy_version 222182 (0.0007) [2023-12-27 07:11:21,073][20381] Updated weights for policy 1, policy_version 221872 (0.0009) [2023-12-27 07:11:21,696][20347] Updated weights for policy 0, policy_version 222192 (0.0006) [2023-12-27 07:11:21,761][20347] Updated weights for policy 0, policy_version 222202 (0.0007) [2023-12-27 07:11:21,782][20381] Updated weights for policy 1, policy_version 221882 (0.0006) [2023-12-27 07:11:21,815][20347] Updated weights for policy 0, policy_version 222212 (0.0006) [2023-12-27 07:11:21,840][20381] Updated weights for policy 1, policy_version 221892 (0.0006) [2023-12-27 07:11:21,900][20381] Updated weights for policy 1, policy_version 221902 (0.0008) [2023-12-27 07:11:22,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20752.9, 300 sec: 20743.8). Total num frames: 113721344. Throughput: 0: 10334.7, 1: 10337.9. Samples: 113694664. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:22,363][19170] Avg episode reward: [(0, '991.120'), (1, '32.710')] [2023-12-27 07:11:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000221904_56819712.pth... [2023-12-27 07:11:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000222216_56901632.pth... [2023-12-27 07:11:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000220720_56516608.pth [2023-12-27 07:11:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000221032_56598528.pth [2023-12-27 07:11:22,531][20347] Updated weights for policy 0, policy_version 222222 (0.0007) [2023-12-27 07:11:22,587][20347] Updated weights for policy 0, policy_version 222232 (0.0006) [2023-12-27 07:11:22,616][20381] Updated weights for policy 1, policy_version 221912 (0.0008) [2023-12-27 07:11:22,648][20347] Updated weights for policy 0, policy_version 222242 (0.0005) [2023-12-27 07:11:22,677][20381] Updated weights for policy 1, policy_version 221922 (0.0009) [2023-12-27 07:11:22,743][20381] Updated weights for policy 1, policy_version 221932 (0.0011) [2023-12-27 07:11:23,266][20347] Updated weights for policy 0, policy_version 222252 (0.0005) [2023-12-27 07:11:23,326][20347] Updated weights for policy 0, policy_version 222262 (0.0005) [2023-12-27 07:11:23,380][20347] Updated weights for policy 0, policy_version 222272 (0.0005) [2023-12-27 07:11:23,452][20381] Updated weights for policy 1, policy_version 221942 (0.0011) [2023-12-27 07:11:23,503][20381] Updated weights for policy 1, policy_version 221952 (0.0010) [2023-12-27 07:11:23,548][20381] Updated weights for policy 1, policy_version 221962 (0.0010) [2023-12-27 07:11:23,984][20347] Updated weights for policy 0, policy_version 222282 (0.0006) [2023-12-27 07:11:24,036][20347] Updated weights for policy 0, policy_version 222292 (0.0008) [2023-12-27 07:11:24,089][20347] Updated weights for policy 0, policy_version 222302 (0.0008) [2023-12-27 07:11:24,150][20347] Updated weights for policy 0, policy_version 222312 (0.0006) [2023-12-27 07:11:24,227][20381] Updated weights for policy 1, policy_version 221972 (0.0009) [2023-12-27 07:11:24,285][20381] Updated weights for policy 1, policy_version 221982 (0.0008) [2023-12-27 07:11:24,347][20381] Updated weights for policy 1, policy_version 221992 (0.0008) [2023-12-27 07:11:24,789][20347] Updated weights for policy 0, policy_version 222322 (0.0005) [2023-12-27 07:11:24,835][20347] Updated weights for policy 0, policy_version 222332 (0.0005) [2023-12-27 07:11:24,883][20347] Updated weights for policy 0, policy_version 222342 (0.0005) [2023-12-27 07:11:24,946][20381] Updated weights for policy 1, policy_version 222003 (0.0007) [2023-12-27 07:11:24,994][20381] Updated weights for policy 1, policy_version 222013 (0.0005) [2023-12-27 07:11:25,043][20381] Updated weights for policy 1, policy_version 222023 (0.0010) [2023-12-27 07:11:25,461][20347] Updated weights for policy 0, policy_version 222352 (0.0005) [2023-12-27 07:11:25,512][20347] Updated weights for policy 0, policy_version 222362 (0.0005) [2023-12-27 07:11:25,564][20347] Updated weights for policy 0, policy_version 222372 (0.0005) [2023-12-27 07:11:25,648][20381] Updated weights for policy 1, policy_version 222033 (0.0010) [2023-12-27 07:11:25,693][20381] Updated weights for policy 1, policy_version 222043 (0.0010) [2023-12-27 07:11:25,739][20381] Updated weights for policy 1, policy_version 222053 (0.0007) [2023-12-27 07:11:25,799][20381] Updated weights for policy 1, policy_version 222063 (0.0005) [2023-12-27 07:11:26,237][20347] Updated weights for policy 0, policy_version 222382 (0.0007) [2023-12-27 07:11:26,298][20347] Updated weights for policy 0, policy_version 222392 (0.0007) [2023-12-27 07:11:26,358][20347] Updated weights for policy 0, policy_version 222402 (0.0007) [2023-12-27 07:11:26,414][20381] Updated weights for policy 1, policy_version 222073 (0.0010) [2023-12-27 07:11:26,465][20381] Updated weights for policy 1, policy_version 222083 (0.0010) [2023-12-27 07:11:26,509][20381] Updated weights for policy 1, policy_version 222093 (0.0010) [2023-12-27 07:11:26,964][20347] Updated weights for policy 0, policy_version 222412 (0.0007) [2023-12-27 07:11:27,009][20347] Updated weights for policy 0, policy_version 222422 (0.0010) [2023-12-27 07:11:27,071][20347] Updated weights for policy 0, policy_version 222432 (0.0008) [2023-12-27 07:11:27,158][20381] Updated weights for policy 1, policy_version 222103 (0.0007) [2023-12-27 07:11:27,212][20381] Updated weights for policy 1, policy_version 222113 (0.0005) [2023-12-27 07:11:27,257][20381] Updated weights for policy 1, policy_version 222123 (0.0007) [2023-12-27 07:11:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 113836032. Throughput: 0: 10371.7, 1: 10377.5. Samples: 113821404. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:27,363][19170] Avg episode reward: [(0, '991.120'), (1, '31.920')] [2023-12-27 07:11:27,773][20347] Updated weights for policy 0, policy_version 222442 (0.0008) [2023-12-27 07:11:27,835][20347] Updated weights for policy 0, policy_version 222452 (0.0005) [2023-12-27 07:11:27,882][20347] Updated weights for policy 0, policy_version 222462 (0.0008) [2023-12-27 07:11:27,907][20381] Updated weights for policy 1, policy_version 222133 (0.0006) [2023-12-27 07:11:27,927][20347] Updated weights for policy 0, policy_version 222472 (0.0007) [2023-12-27 07:11:27,963][20381] Updated weights for policy 1, policy_version 222143 (0.0008) [2023-12-27 07:11:28,011][20381] Updated weights for policy 1, policy_version 222153 (0.0008) [2023-12-27 07:11:28,616][20347] Updated weights for policy 0, policy_version 222482 (0.0008) [2023-12-27 07:11:28,669][20347] Updated weights for policy 0, policy_version 222492 (0.0008) [2023-12-27 07:11:28,691][20381] Updated weights for policy 1, policy_version 222163 (0.0010) [2023-12-27 07:11:28,726][20347] Updated weights for policy 0, policy_version 222502 (0.0006) [2023-12-27 07:11:28,748][20381] Updated weights for policy 1, policy_version 222173 (0.0011) [2023-12-27 07:11:28,806][20381] Updated weights for policy 1, policy_version 222183 (0.0011) [2023-12-27 07:11:29,393][20347] Updated weights for policy 0, policy_version 222512 (0.0007) [2023-12-27 07:11:29,442][20347] Updated weights for policy 0, policy_version 222522 (0.0008) [2023-12-27 07:11:29,458][20381] Updated weights for policy 1, policy_version 222193 (0.0010) [2023-12-27 07:11:29,497][20347] Updated weights for policy 0, policy_version 222532 (0.0006) [2023-12-27 07:11:29,516][20381] Updated weights for policy 1, policy_version 222203 (0.0009) [2023-12-27 07:11:29,574][20381] Updated weights for policy 1, policy_version 222213 (0.0010) [2023-12-27 07:11:29,629][20381] Updated weights for policy 1, policy_version 222223 (0.0011) [2023-12-27 07:11:30,219][20347] Updated weights for policy 0, policy_version 222542 (0.0007) [2023-12-27 07:11:30,279][20347] Updated weights for policy 0, policy_version 222552 (0.0007) [2023-12-27 07:11:30,339][20347] Updated weights for policy 0, policy_version 222562 (0.0008) [2023-12-27 07:11:30,351][20381] Updated weights for policy 1, policy_version 222233 (0.0006) [2023-12-27 07:11:30,407][20381] Updated weights for policy 1, policy_version 222243 (0.0005) [2023-12-27 07:11:30,456][20381] Updated weights for policy 1, policy_version 222253 (0.0005) [2023-12-27 07:11:30,965][20381] Updated weights for policy 1, policy_version 222263 (0.0005) [2023-12-27 07:11:31,008][20381] Updated weights for policy 1, policy_version 222273 (0.0005) [2023-12-27 07:11:31,062][20381] Updated weights for policy 1, policy_version 222283 (0.0007) [2023-12-27 07:11:31,092][20347] Updated weights for policy 0, policy_version 222572 (0.0008) [2023-12-27 07:11:31,151][20347] Updated weights for policy 0, policy_version 222582 (0.0008) [2023-12-27 07:11:31,213][20347] Updated weights for policy 0, policy_version 222592 (0.0008) [2023-12-27 07:11:31,686][20381] Updated weights for policy 1, policy_version 222293 (0.0006) [2023-12-27 07:11:31,755][20381] Updated weights for policy 1, policy_version 222303 (0.0008) [2023-12-27 07:11:31,816][20381] Updated weights for policy 1, policy_version 222313 (0.0008) [2023-12-27 07:11:31,914][20347] Updated weights for policy 0, policy_version 222602 (0.0008) [2023-12-27 07:11:31,961][20347] Updated weights for policy 0, policy_version 222612 (0.0006) [2023-12-27 07:11:32,006][20347] Updated weights for policy 0, policy_version 222622 (0.0007) [2023-12-27 07:11:32,051][20347] Updated weights for policy 0, policy_version 222632 (0.0007) [2023-12-27 07:11:32,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 113934336. Throughput: 0: 10289.9, 1: 10288.7. Samples: 113945404. Policy #0 lag: (min: 1.0, avg: 17.7, max: 33.0) [2023-12-27 07:11:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '36.660')] [2023-12-27 07:11:32,520][20381] Updated weights for policy 1, policy_version 222323 (0.0008) [2023-12-27 07:11:32,581][20381] Updated weights for policy 1, policy_version 222333 (0.0007) [2023-12-27 07:11:32,638][20381] Updated weights for policy 1, policy_version 222343 (0.0007) [2023-12-27 07:11:32,786][20347] Updated weights for policy 0, policy_version 222642 (0.0007) [2023-12-27 07:11:32,835][20347] Updated weights for policy 0, policy_version 222652 (0.0008) [2023-12-27 07:11:32,885][20347] Updated weights for policy 0, policy_version 222662 (0.0007) [2023-12-27 07:11:33,314][20381] Updated weights for policy 1, policy_version 222353 (0.0010) [2023-12-27 07:11:33,370][20381] Updated weights for policy 1, policy_version 222363 (0.0011) [2023-12-27 07:11:33,422][20381] Updated weights for policy 1, policy_version 222373 (0.0010) [2023-12-27 07:11:33,467][20381] Updated weights for policy 1, policy_version 222383 (0.0010) [2023-12-27 07:11:33,612][20347] Updated weights for policy 0, policy_version 222672 (0.0008) [2023-12-27 07:11:33,666][20347] Updated weights for policy 0, policy_version 222682 (0.0008) [2023-12-27 07:11:33,714][20347] Updated weights for policy 0, policy_version 222692 (0.0007) [2023-12-27 07:11:34,115][20381] Updated weights for policy 1, policy_version 222393 (0.0007) [2023-12-27 07:11:34,175][20381] Updated weights for policy 1, policy_version 222403 (0.0010) [2023-12-27 07:11:34,230][20381] Updated weights for policy 1, policy_version 222413 (0.0010) [2023-12-27 07:11:34,436][20347] Updated weights for policy 0, policy_version 222702 (0.0007) [2023-12-27 07:11:34,482][20347] Updated weights for policy 0, policy_version 222712 (0.0008) [2023-12-27 07:11:34,540][20347] Updated weights for policy 0, policy_version 222722 (0.0009) [2023-12-27 07:11:34,904][20381] Updated weights for policy 1, policy_version 222423 (0.0010) [2023-12-27 07:11:34,962][20381] Updated weights for policy 1, policy_version 222433 (0.0010) [2023-12-27 07:11:35,011][20381] Updated weights for policy 1, policy_version 222443 (0.0011) [2023-12-27 07:11:35,218][20347] Updated weights for policy 0, policy_version 222732 (0.0007) [2023-12-27 07:11:35,263][20347] Updated weights for policy 0, policy_version 222742 (0.0008) [2023-12-27 07:11:35,326][20347] Updated weights for policy 0, policy_version 222752 (0.0008) [2023-12-27 07:11:35,705][20381] Updated weights for policy 1, policy_version 222453 (0.0010) [2023-12-27 07:11:35,753][20381] Updated weights for policy 1, policy_version 222463 (0.0010) [2023-12-27 07:11:35,815][20381] Updated weights for policy 1, policy_version 222473 (0.0010) [2023-12-27 07:11:36,032][20347] Updated weights for policy 0, policy_version 222762 (0.0007) [2023-12-27 07:11:36,083][20347] Updated weights for policy 0, policy_version 222772 (0.0007) [2023-12-27 07:11:36,145][20347] Updated weights for policy 0, policy_version 222782 (0.0007) [2023-12-27 07:11:36,201][20347] Updated weights for policy 0, policy_version 222792 (0.0007) [2023-12-27 07:11:36,562][20381] Updated weights for policy 1, policy_version 222483 (0.0010) [2023-12-27 07:11:36,623][20381] Updated weights for policy 1, policy_version 222493 (0.0008) [2023-12-27 07:11:36,684][20381] Updated weights for policy 1, policy_version 222503 (0.0008) [2023-12-27 07:11:36,926][20347] Updated weights for policy 0, policy_version 222802 (0.0008) [2023-12-27 07:11:36,989][20347] Updated weights for policy 0, policy_version 222812 (0.0007) [2023-12-27 07:11:37,046][20347] Updated weights for policy 0, policy_version 222822 (0.0008) [2023-12-27 07:11:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 114032640. Throughput: 0: 10259.6, 1: 10318.9. Samples: 114006000. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:11:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '38.860')] [2023-12-27 07:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000222512_56975360.pth... [2023-12-27 07:11:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000222824_57057280.pth... [2023-12-27 07:11:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000221328_56672256.pth [2023-12-27 07:11:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000221640_56754176.pth [2023-12-27 07:11:37,399][20381] Updated weights for policy 1, policy_version 222513 (0.0008) [2023-12-27 07:11:37,461][20381] Updated weights for policy 1, policy_version 222523 (0.0006) [2023-12-27 07:11:37,520][20381] Updated weights for policy 1, policy_version 222533 (0.0007) [2023-12-27 07:11:37,569][20381] Updated weights for policy 1, policy_version 222543 (0.0007) [2023-12-27 07:11:37,755][20347] Updated weights for policy 0, policy_version 222832 (0.0007) [2023-12-27 07:11:37,816][20347] Updated weights for policy 0, policy_version 222842 (0.0006) [2023-12-27 07:11:37,867][20347] Updated weights for policy 0, policy_version 222852 (0.0007) [2023-12-27 07:11:38,239][20381] Updated weights for policy 1, policy_version 222553 (0.0008) [2023-12-27 07:11:38,286][20381] Updated weights for policy 1, policy_version 222563 (0.0007) [2023-12-27 07:11:38,345][20381] Updated weights for policy 1, policy_version 222573 (0.0008) [2023-12-27 07:11:38,524][20347] Updated weights for policy 0, policy_version 222862 (0.0007) [2023-12-27 07:11:38,586][20347] Updated weights for policy 0, policy_version 222872 (0.0006) [2023-12-27 07:11:38,648][20347] Updated weights for policy 0, policy_version 222882 (0.0007) [2023-12-27 07:11:39,072][20381] Updated weights for policy 1, policy_version 222583 (0.0007) [2023-12-27 07:11:39,125][20381] Updated weights for policy 1, policy_version 222593 (0.0008) [2023-12-27 07:11:39,176][20381] Updated weights for policy 1, policy_version 222603 (0.0010) [2023-12-27 07:11:39,319][20347] Updated weights for policy 0, policy_version 222892 (0.0009) [2023-12-27 07:11:39,388][20347] Updated weights for policy 0, policy_version 222902 (0.0011) [2023-12-27 07:11:39,450][20347] Updated weights for policy 0, policy_version 222912 (0.0009) [2023-12-27 07:11:39,925][20381] Updated weights for policy 1, policy_version 222613 (0.0009) [2023-12-27 07:11:39,982][20381] Updated weights for policy 1, policy_version 222623 (0.0005) [2023-12-27 07:11:40,041][20381] Updated weights for policy 1, policy_version 222633 (0.0007) [2023-12-27 07:11:40,147][20347] Updated weights for policy 0, policy_version 222922 (0.0007) [2023-12-27 07:11:40,203][20347] Updated weights for policy 0, policy_version 222932 (0.0010) [2023-12-27 07:11:40,263][20347] Updated weights for policy 0, policy_version 222942 (0.0009) [2023-12-27 07:11:40,317][20347] Updated weights for policy 0, policy_version 222952 (0.0008) [2023-12-27 07:11:40,691][20381] Updated weights for policy 1, policy_version 222643 (0.0007) [2023-12-27 07:11:40,749][20381] Updated weights for policy 1, policy_version 222653 (0.0011) [2023-12-27 07:11:40,798][20381] Updated weights for policy 1, policy_version 222663 (0.0010) [2023-12-27 07:11:40,989][20347] Updated weights for policy 0, policy_version 222962 (0.0008) [2023-12-27 07:11:41,043][20347] Updated weights for policy 0, policy_version 222972 (0.0007) [2023-12-27 07:11:41,104][20347] Updated weights for policy 0, policy_version 222982 (0.0007) [2023-12-27 07:11:41,520][20381] Updated weights for policy 1, policy_version 222673 (0.0010) [2023-12-27 07:11:41,568][20381] Updated weights for policy 1, policy_version 222683 (0.0008) [2023-12-27 07:11:41,629][20381] Updated weights for policy 1, policy_version 222693 (0.0008) [2023-12-27 07:11:41,687][20381] Updated weights for policy 1, policy_version 222703 (0.0008) [2023-12-27 07:11:41,813][20347] Updated weights for policy 0, policy_version 222992 (0.0008) [2023-12-27 07:11:41,877][20347] Updated weights for policy 0, policy_version 223002 (0.0008) [2023-12-27 07:11:41,930][20347] Updated weights for policy 0, policy_version 223012 (0.0009) [2023-12-27 07:11:42,318][20381] Updated weights for policy 1, policy_version 222713 (0.0007) [2023-12-27 07:11:42,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 114130944. Throughput: 0: 10168.9, 1: 10302.3. Samples: 114125044. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:11:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '42.600')] [2023-12-27 07:11:42,386][20381] Updated weights for policy 1, policy_version 222723 (0.0008) [2023-12-27 07:11:42,443][20381] Updated weights for policy 1, policy_version 222733 (0.0008) [2023-12-27 07:11:42,648][20347] Updated weights for policy 0, policy_version 223022 (0.0009) [2023-12-27 07:11:42,698][20347] Updated weights for policy 0, policy_version 223032 (0.0010) [2023-12-27 07:11:42,753][20347] Updated weights for policy 0, policy_version 223042 (0.0010) [2023-12-27 07:11:43,131][20381] Updated weights for policy 1, policy_version 222743 (0.0006) [2023-12-27 07:11:43,195][20381] Updated weights for policy 1, policy_version 222753 (0.0007) [2023-12-27 07:11:43,252][20381] Updated weights for policy 1, policy_version 222763 (0.0007) [2023-12-27 07:11:43,461][20347] Updated weights for policy 0, policy_version 223052 (0.0010) [2023-12-27 07:11:43,519][20347] Updated weights for policy 0, policy_version 223062 (0.0010) [2023-12-27 07:11:43,579][20347] Updated weights for policy 0, policy_version 223072 (0.0010) [2023-12-27 07:11:43,926][20381] Updated weights for policy 1, policy_version 222773 (0.0008) [2023-12-27 07:11:43,977][20381] Updated weights for policy 1, policy_version 222783 (0.0008) [2023-12-27 07:11:44,023][20381] Updated weights for policy 1, policy_version 222793 (0.0007) [2023-12-27 07:11:44,181][20347] Updated weights for policy 0, policy_version 223082 (0.0010) [2023-12-27 07:11:44,236][20347] Updated weights for policy 0, policy_version 223092 (0.0007) [2023-12-27 07:11:44,292][20347] Updated weights for policy 0, policy_version 223102 (0.0008) [2023-12-27 07:11:44,342][20347] Updated weights for policy 0, policy_version 223112 (0.0007) [2023-12-27 07:11:44,722][20381] Updated weights for policy 1, policy_version 222803 (0.0008) [2023-12-27 07:11:44,787][20381] Updated weights for policy 1, policy_version 222813 (0.0008) [2023-12-27 07:11:44,854][20381] Updated weights for policy 1, policy_version 222823 (0.0008) [2023-12-27 07:11:45,076][20347] Updated weights for policy 0, policy_version 223122 (0.0008) [2023-12-27 07:11:45,134][20347] Updated weights for policy 0, policy_version 223132 (0.0008) [2023-12-27 07:11:45,191][20347] Updated weights for policy 0, policy_version 223142 (0.0008) [2023-12-27 07:11:45,529][20381] Updated weights for policy 1, policy_version 222833 (0.0008) [2023-12-27 07:11:45,585][20381] Updated weights for policy 1, policy_version 222843 (0.0010) [2023-12-27 07:11:45,646][20381] Updated weights for policy 1, policy_version 222853 (0.0010) [2023-12-27 07:11:45,701][20381] Updated weights for policy 1, policy_version 222863 (0.0006) [2023-12-27 07:11:45,915][20347] Updated weights for policy 0, policy_version 223152 (0.0007) [2023-12-27 07:11:45,973][20347] Updated weights for policy 0, policy_version 223162 (0.0007) [2023-12-27 07:11:46,028][20347] Updated weights for policy 0, policy_version 223172 (0.0009) [2023-12-27 07:11:46,320][20381] Updated weights for policy 1, policy_version 222873 (0.0007) [2023-12-27 07:11:46,367][20381] Updated weights for policy 1, policy_version 222883 (0.0008) [2023-12-27 07:11:46,414][20381] Updated weights for policy 1, policy_version 222893 (0.0008) [2023-12-27 07:11:46,663][20347] Updated weights for policy 0, policy_version 223182 (0.0008) [2023-12-27 07:11:46,718][20347] Updated weights for policy 0, policy_version 223192 (0.0006) [2023-12-27 07:11:46,775][20347] Updated weights for policy 0, policy_version 223202 (0.0007) [2023-12-27 07:11:47,063][20381] Updated weights for policy 1, policy_version 222903 (0.0007) [2023-12-27 07:11:47,120][20381] Updated weights for policy 1, policy_version 222913 (0.0005) [2023-12-27 07:11:47,186][20381] Updated weights for policy 1, policy_version 222923 (0.0005) [2023-12-27 07:11:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 114237440. Throughput: 0: 10190.8, 1: 10336.9. Samples: 114248596. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:11:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '43.210')] [2023-12-27 07:11:47,371][20347] Updated weights for policy 0, policy_version 223212 (0.0007) [2023-12-27 07:11:47,417][20347] Updated weights for policy 0, policy_version 223222 (0.0010) [2023-12-27 07:11:47,466][20347] Updated weights for policy 0, policy_version 223232 (0.0008) [2023-12-27 07:11:47,678][20381] Updated weights for policy 1, policy_version 222933 (0.0005) [2023-12-27 07:11:47,737][20381] Updated weights for policy 1, policy_version 222943 (0.0005) [2023-12-27 07:11:47,794][20381] Updated weights for policy 1, policy_version 222953 (0.0005) [2023-12-27 07:11:48,184][20347] Updated weights for policy 0, policy_version 223242 (0.0006) [2023-12-27 07:11:48,236][20347] Updated weights for policy 0, policy_version 223252 (0.0007) [2023-12-27 07:11:48,288][20347] Updated weights for policy 0, policy_version 223262 (0.0007) [2023-12-27 07:11:48,343][20347] Updated weights for policy 0, policy_version 223272 (0.0008) [2023-12-27 07:11:48,380][20381] Updated weights for policy 1, policy_version 222963 (0.0006) [2023-12-27 07:11:48,437][20381] Updated weights for policy 1, policy_version 222973 (0.0008) [2023-12-27 07:11:48,501][20381] Updated weights for policy 1, policy_version 222983 (0.0008) [2023-12-27 07:11:49,054][20347] Updated weights for policy 0, policy_version 223282 (0.0009) [2023-12-27 07:11:49,102][20347] Updated weights for policy 0, policy_version 223292 (0.0007) [2023-12-27 07:11:49,155][20381] Updated weights for policy 1, policy_version 222993 (0.0007) [2023-12-27 07:11:49,158][20347] Updated weights for policy 0, policy_version 223302 (0.0007) [2023-12-27 07:11:49,206][20381] Updated weights for policy 1, policy_version 223003 (0.0009) [2023-12-27 07:11:49,268][20381] Updated weights for policy 1, policy_version 223013 (0.0009) [2023-12-27 07:11:49,324][20381] Updated weights for policy 1, policy_version 223023 (0.0010) [2023-12-27 07:11:49,890][20347] Updated weights for policy 0, policy_version 223312 (0.0008) [2023-12-27 07:11:49,948][20347] Updated weights for policy 0, policy_version 223322 (0.0008) [2023-12-27 07:11:50,010][20347] Updated weights for policy 0, policy_version 223332 (0.0006) [2023-12-27 07:11:50,015][20381] Updated weights for policy 1, policy_version 223033 (0.0006) [2023-12-27 07:11:50,079][20381] Updated weights for policy 1, policy_version 223043 (0.0009) [2023-12-27 07:11:50,140][20381] Updated weights for policy 1, policy_version 223053 (0.0009) [2023-12-27 07:11:50,626][20347] Updated weights for policy 0, policy_version 223342 (0.0007) [2023-12-27 07:11:50,687][20347] Updated weights for policy 0, policy_version 223352 (0.0006) [2023-12-27 07:11:50,755][20347] Updated weights for policy 0, policy_version 223362 (0.0008) [2023-12-27 07:11:50,808][20381] Updated weights for policy 1, policy_version 223063 (0.0007) [2023-12-27 07:11:50,863][20381] Updated weights for policy 1, policy_version 223073 (0.0007) [2023-12-27 07:11:50,922][20381] Updated weights for policy 1, policy_version 223083 (0.0008) [2023-12-27 07:11:51,430][20347] Updated weights for policy 0, policy_version 223372 (0.0010) [2023-12-27 07:11:51,486][20347] Updated weights for policy 0, policy_version 223382 (0.0010) [2023-12-27 07:11:51,542][20347] Updated weights for policy 0, policy_version 223392 (0.0010) [2023-12-27 07:11:51,633][20381] Updated weights for policy 1, policy_version 223093 (0.0007) [2023-12-27 07:11:51,687][20381] Updated weights for policy 1, policy_version 223103 (0.0007) [2023-12-27 07:11:51,750][20381] Updated weights for policy 1, policy_version 223113 (0.0009) [2023-12-27 07:11:52,208][20347] Updated weights for policy 0, policy_version 223402 (0.0010) [2023-12-27 07:11:52,263][20347] Updated weights for policy 0, policy_version 223412 (0.0010) [2023-12-27 07:11:52,325][20347] Updated weights for policy 0, policy_version 223422 (0.0010) [2023-12-27 07:11:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20716.0). Total num frames: 114335744. Throughput: 0: 10174.0, 1: 10327.0. Samples: 114310992. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:11:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '49.600')] [2023-12-27 07:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000223120_57131008.pth... [2023-12-27 07:11:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000221904_56819712.pth [2023-12-27 07:11:52,392][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000223432_57212928.pth... [2023-12-27 07:11:52,392][20347] Updated weights for policy 0, policy_version 223432 (0.0010) [2023-12-27 07:11:52,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000222216_56901632.pth [2023-12-27 07:11:52,422][20381] Updated weights for policy 1, policy_version 223123 (0.0009) [2023-12-27 07:11:52,474][20381] Updated weights for policy 1, policy_version 223133 (0.0008) [2023-12-27 07:11:52,523][20381] Updated weights for policy 1, policy_version 223143 (0.0008) [2023-12-27 07:11:52,996][20347] Updated weights for policy 0, policy_version 223442 (0.0005) [2023-12-27 07:11:53,048][20347] Updated weights for policy 0, policy_version 223452 (0.0005) [2023-12-27 07:11:53,096][20347] Updated weights for policy 0, policy_version 223462 (0.0005) [2023-12-27 07:11:53,288][20381] Updated weights for policy 1, policy_version 223153 (0.0008) [2023-12-27 07:11:53,338][20381] Updated weights for policy 1, policy_version 223163 (0.0010) [2023-12-27 07:11:53,390][20381] Updated weights for policy 1, policy_version 223173 (0.0010) [2023-12-27 07:11:53,452][20381] Updated weights for policy 1, policy_version 223183 (0.0011) [2023-12-27 07:11:53,666][20347] Updated weights for policy 0, policy_version 223472 (0.0009) [2023-12-27 07:11:53,729][20347] Updated weights for policy 0, policy_version 223482 (0.0010) [2023-12-27 07:11:53,786][20347] Updated weights for policy 0, policy_version 223492 (0.0010) [2023-12-27 07:11:54,057][20381] Updated weights for policy 1, policy_version 223193 (0.0008) [2023-12-27 07:11:54,109][20381] Updated weights for policy 1, policy_version 223203 (0.0010) [2023-12-27 07:11:54,162][20381] Updated weights for policy 1, policy_version 223213 (0.0010) [2023-12-27 07:11:54,427][20347] Updated weights for policy 0, policy_version 223502 (0.0007) [2023-12-27 07:11:54,478][20347] Updated weights for policy 0, policy_version 223512 (0.0006) [2023-12-27 07:11:54,530][20347] Updated weights for policy 0, policy_version 223522 (0.0005) [2023-12-27 07:11:54,844][20381] Updated weights for policy 1, policy_version 223223 (0.0008) [2023-12-27 07:11:54,891][20381] Updated weights for policy 1, policy_version 223233 (0.0008) [2023-12-27 07:11:54,943][20381] Updated weights for policy 1, policy_version 223243 (0.0007) [2023-12-27 07:11:55,185][20347] Updated weights for policy 0, policy_version 223532 (0.0006) [2023-12-27 07:11:55,236][20347] Updated weights for policy 0, policy_version 223542 (0.0008) [2023-12-27 07:11:55,286][20347] Updated weights for policy 0, policy_version 223552 (0.0007) [2023-12-27 07:11:55,663][20381] Updated weights for policy 1, policy_version 223253 (0.0009) [2023-12-27 07:11:55,723][20381] Updated weights for policy 1, policy_version 223263 (0.0008) [2023-12-27 07:11:55,789][20381] Updated weights for policy 1, policy_version 223273 (0.0008) [2023-12-27 07:11:55,943][20347] Updated weights for policy 0, policy_version 223562 (0.0006) [2023-12-27 07:11:56,004][20347] Updated weights for policy 0, policy_version 223572 (0.0005) [2023-12-27 07:11:56,056][20347] Updated weights for policy 0, policy_version 223582 (0.0005) [2023-12-27 07:11:56,111][20347] Updated weights for policy 0, policy_version 223592 (0.0006) [2023-12-27 07:11:56,456][20381] Updated weights for policy 1, policy_version 223283 (0.0007) [2023-12-27 07:11:56,515][20381] Updated weights for policy 1, policy_version 223293 (0.0006) [2023-12-27 07:11:56,568][20381] Updated weights for policy 1, policy_version 223303 (0.0005) [2023-12-27 07:11:56,715][20347] Updated weights for policy 0, policy_version 223602 (0.0007) [2023-12-27 07:11:56,766][20347] Updated weights for policy 0, policy_version 223612 (0.0006) [2023-12-27 07:11:56,812][20347] Updated weights for policy 0, policy_version 223622 (0.0006) [2023-12-27 07:11:57,165][20381] Updated weights for policy 1, policy_version 223313 (0.0006) [2023-12-27 07:11:57,220][20381] Updated weights for policy 1, policy_version 223323 (0.0008) [2023-12-27 07:11:57,272][20381] Updated weights for policy 1, policy_version 223333 (0.0008) [2023-12-27 07:11:57,323][20381] Updated weights for policy 1, policy_version 223343 (0.0008) [2023-12-27 07:11:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 114450432. Throughput: 0: 10270.7, 1: 10336.7. Samples: 114436676. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:11:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '47.540')] [2023-12-27 07:11:57,516][20347] Updated weights for policy 0, policy_version 223632 (0.0007) [2023-12-27 07:11:57,573][20347] Updated weights for policy 0, policy_version 223642 (0.0008) [2023-12-27 07:11:57,634][20347] Updated weights for policy 0, policy_version 223652 (0.0007) [2023-12-27 07:11:57,950][20381] Updated weights for policy 1, policy_version 223353 (0.0005) [2023-12-27 07:11:58,006][20381] Updated weights for policy 1, policy_version 223363 (0.0005) [2023-12-27 07:11:58,061][20381] Updated weights for policy 1, policy_version 223373 (0.0005) [2023-12-27 07:11:58,341][20347] Updated weights for policy 0, policy_version 223662 (0.0007) [2023-12-27 07:11:58,402][20347] Updated weights for policy 0, policy_version 223672 (0.0008) [2023-12-27 07:11:58,453][20347] Updated weights for policy 0, policy_version 223682 (0.0008) [2023-12-27 07:11:58,713][20381] Updated weights for policy 1, policy_version 223383 (0.0007) [2023-12-27 07:11:58,775][20381] Updated weights for policy 1, policy_version 223393 (0.0008) [2023-12-27 07:11:58,834][20381] Updated weights for policy 1, policy_version 223403 (0.0008) [2023-12-27 07:11:59,136][20347] Updated weights for policy 0, policy_version 223692 (0.0006) [2023-12-27 07:11:59,206][20347] Updated weights for policy 0, policy_version 223702 (0.0006) [2023-12-27 07:11:59,273][20347] Updated weights for policy 0, policy_version 223712 (0.0010) [2023-12-27 07:11:59,584][20381] Updated weights for policy 1, policy_version 223413 (0.0008) [2023-12-27 07:11:59,637][20381] Updated weights for policy 1, policy_version 223423 (0.0007) [2023-12-27 07:11:59,695][20381] Updated weights for policy 1, policy_version 223433 (0.0008) [2023-12-27 07:11:59,912][20347] Updated weights for policy 0, policy_version 223722 (0.0010) [2023-12-27 07:11:59,971][20347] Updated weights for policy 0, policy_version 223732 (0.0010) [2023-12-27 07:12:00,030][20347] Updated weights for policy 0, policy_version 223742 (0.0010) [2023-12-27 07:12:00,085][20347] Updated weights for policy 0, policy_version 223752 (0.0010) [2023-12-27 07:12:00,408][20381] Updated weights for policy 1, policy_version 223443 (0.0007) [2023-12-27 07:12:00,464][20381] Updated weights for policy 1, policy_version 223453 (0.0008) [2023-12-27 07:12:00,518][20381] Updated weights for policy 1, policy_version 223463 (0.0008) [2023-12-27 07:12:00,671][20347] Updated weights for policy 0, policy_version 223762 (0.0010) [2023-12-27 07:12:00,725][20347] Updated weights for policy 0, policy_version 223772 (0.0010) [2023-12-27 07:12:00,776][20347] Updated weights for policy 0, policy_version 223782 (0.0010) [2023-12-27 07:12:01,232][20381] Updated weights for policy 1, policy_version 223473 (0.0008) [2023-12-27 07:12:01,291][20381] Updated weights for policy 1, policy_version 223483 (0.0007) [2023-12-27 07:12:01,345][20381] Updated weights for policy 1, policy_version 223493 (0.0008) [2023-12-27 07:12:01,405][20381] Updated weights for policy 1, policy_version 223503 (0.0007) [2023-12-27 07:12:01,453][20347] Updated weights for policy 0, policy_version 223792 (0.0008) [2023-12-27 07:12:01,510][20347] Updated weights for policy 0, policy_version 223802 (0.0007) [2023-12-27 07:12:01,565][20347] Updated weights for policy 0, policy_version 223812 (0.0008) [2023-12-27 07:12:02,099][20381] Updated weights for policy 1, policy_version 223513 (0.0007) [2023-12-27 07:12:02,155][20381] Updated weights for policy 1, policy_version 223523 (0.0005) [2023-12-27 07:12:02,208][20381] Updated weights for policy 1, policy_version 223533 (0.0005) [2023-12-27 07:12:02,258][20347] Updated weights for policy 0, policy_version 223822 (0.0009) [2023-12-27 07:12:02,320][20347] Updated weights for policy 0, policy_version 223832 (0.0007) [2023-12-27 07:12:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 114548736. Throughput: 0: 10316.2, 1: 10290.6. Samples: 114560580. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '51.540')] [2023-12-27 07:12:02,384][20347] Updated weights for policy 0, policy_version 223842 (0.0009) [2023-12-27 07:12:02,869][20381] Updated weights for policy 1, policy_version 223543 (0.0006) [2023-12-27 07:12:02,921][20381] Updated weights for policy 1, policy_version 223553 (0.0008) [2023-12-27 07:12:02,980][20381] Updated weights for policy 1, policy_version 223563 (0.0008) [2023-12-27 07:12:03,091][20347] Updated weights for policy 0, policy_version 223852 (0.0008) [2023-12-27 07:12:03,140][20347] Updated weights for policy 0, policy_version 223862 (0.0008) [2023-12-27 07:12:03,192][20347] Updated weights for policy 0, policy_version 223872 (0.0007) [2023-12-27 07:12:03,697][20381] Updated weights for policy 1, policy_version 223573 (0.0008) [2023-12-27 07:12:03,743][20381] Updated weights for policy 1, policy_version 223583 (0.0007) [2023-12-27 07:12:03,792][20381] Updated weights for policy 1, policy_version 223593 (0.0007) [2023-12-27 07:12:03,857][20347] Updated weights for policy 0, policy_version 223882 (0.0007) [2023-12-27 07:12:03,913][20347] Updated weights for policy 0, policy_version 223892 (0.0005) [2023-12-27 07:12:03,975][20347] Updated weights for policy 0, policy_version 223902 (0.0007) [2023-12-27 07:12:04,028][20347] Updated weights for policy 0, policy_version 223912 (0.0007) [2023-12-27 07:12:04,529][20381] Updated weights for policy 1, policy_version 223603 (0.0008) [2023-12-27 07:12:04,575][20381] Updated weights for policy 1, policy_version 223613 (0.0011) [2023-12-27 07:12:04,634][20381] Updated weights for policy 1, policy_version 223623 (0.0011) [2023-12-27 07:12:04,714][20347] Updated weights for policy 0, policy_version 223922 (0.0008) [2023-12-27 07:12:04,771][20347] Updated weights for policy 0, policy_version 223932 (0.0008) [2023-12-27 07:12:04,827][20347] Updated weights for policy 0, policy_version 223942 (0.0008) [2023-12-27 07:12:05,179][20381] Updated weights for policy 1, policy_version 223633 (0.0006) [2023-12-27 07:12:05,238][20381] Updated weights for policy 1, policy_version 223643 (0.0005) [2023-12-27 07:12:05,298][20381] Updated weights for policy 1, policy_version 223653 (0.0005) [2023-12-27 07:12:05,351][20381] Updated weights for policy 1, policy_version 223663 (0.0005) [2023-12-27 07:12:05,588][20347] Updated weights for policy 0, policy_version 223952 (0.0006) [2023-12-27 07:12:05,637][20347] Updated weights for policy 0, policy_version 223962 (0.0006) [2023-12-27 07:12:05,683][20347] Updated weights for policy 0, policy_version 223972 (0.0005) [2023-12-27 07:12:05,852][20381] Updated weights for policy 1, policy_version 223673 (0.0005) [2023-12-27 07:12:05,903][20381] Updated weights for policy 1, policy_version 223683 (0.0009) [2023-12-27 07:12:05,955][20381] Updated weights for policy 1, policy_version 223693 (0.0010) [2023-12-27 07:12:06,286][20347] Updated weights for policy 0, policy_version 223982 (0.0007) [2023-12-27 07:12:06,344][20347] Updated weights for policy 0, policy_version 223992 (0.0007) [2023-12-27 07:12:06,399][20347] Updated weights for policy 0, policy_version 224002 (0.0007) [2023-12-27 07:12:06,653][20381] Updated weights for policy 1, policy_version 223703 (0.0011) [2023-12-27 07:12:06,706][20381] Updated weights for policy 1, policy_version 223713 (0.0006) [2023-12-27 07:12:06,756][20381] Updated weights for policy 1, policy_version 223723 (0.0005) [2023-12-27 07:12:07,106][20347] Updated weights for policy 0, policy_version 224012 (0.0008) [2023-12-27 07:12:07,169][20347] Updated weights for policy 0, policy_version 224022 (0.0007) [2023-12-27 07:12:07,230][20347] Updated weights for policy 0, policy_version 224032 (0.0008) [2023-12-27 07:12:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 114655232. Throughput: 0: 10273.3, 1: 10331.4. Samples: 114621864. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '54.240')] [2023-12-27 07:12:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000223728_57286656.pth... [2023-12-27 07:12:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000224040_57368576.pth... [2023-12-27 07:12:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000222512_56975360.pth [2023-12-27 07:12:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000222824_57057280.pth [2023-12-27 07:12:07,421][20381] Updated weights for policy 1, policy_version 223733 (0.0009) [2023-12-27 07:12:07,478][20381] Updated weights for policy 1, policy_version 223743 (0.0011) [2023-12-27 07:12:07,539][20381] Updated weights for policy 1, policy_version 223753 (0.0011) [2023-12-27 07:12:07,951][20347] Updated weights for policy 0, policy_version 224042 (0.0008) [2023-12-27 07:12:08,010][20347] Updated weights for policy 0, policy_version 224052 (0.0008) [2023-12-27 07:12:08,070][20347] Updated weights for policy 0, policy_version 224062 (0.0008) [2023-12-27 07:12:08,128][20347] Updated weights for policy 0, policy_version 224072 (0.0008) [2023-12-27 07:12:08,227][20381] Updated weights for policy 1, policy_version 223763 (0.0011) [2023-12-27 07:12:08,273][20381] Updated weights for policy 1, policy_version 223773 (0.0010) [2023-12-27 07:12:08,319][20381] Updated weights for policy 1, policy_version 223783 (0.0010) [2023-12-27 07:12:08,791][20347] Updated weights for policy 0, policy_version 224082 (0.0008) [2023-12-27 07:12:08,847][20347] Updated weights for policy 0, policy_version 224092 (0.0007) [2023-12-27 07:12:08,901][20347] Updated weights for policy 0, policy_version 224102 (0.0008) [2023-12-27 07:12:09,011][20381] Updated weights for policy 1, policy_version 223793 (0.0008) [2023-12-27 07:12:09,063][20381] Updated weights for policy 1, policy_version 223803 (0.0006) [2023-12-27 07:12:09,131][20381] Updated weights for policy 1, policy_version 223813 (0.0006) [2023-12-27 07:12:09,189][20381] Updated weights for policy 1, policy_version 223823 (0.0008) [2023-12-27 07:12:09,604][20347] Updated weights for policy 0, policy_version 224112 (0.0007) [2023-12-27 07:12:09,651][20347] Updated weights for policy 0, policy_version 224122 (0.0007) [2023-12-27 07:12:09,704][20347] Updated weights for policy 0, policy_version 224132 (0.0008) [2023-12-27 07:12:09,899][20381] Updated weights for policy 1, policy_version 223833 (0.0008) [2023-12-27 07:12:09,959][20381] Updated weights for policy 1, policy_version 223843 (0.0008) [2023-12-27 07:12:10,023][20381] Updated weights for policy 1, policy_version 223853 (0.0008) [2023-12-27 07:12:10,464][20347] Updated weights for policy 0, policy_version 224142 (0.0008) [2023-12-27 07:12:10,528][20347] Updated weights for policy 0, policy_version 224152 (0.0008) [2023-12-27 07:12:10,585][20347] Updated weights for policy 0, policy_version 224162 (0.0007) [2023-12-27 07:12:10,735][20381] Updated weights for policy 1, policy_version 223863 (0.0008) [2023-12-27 07:12:10,791][20381] Updated weights for policy 1, policy_version 223873 (0.0008) [2023-12-27 07:12:10,845][20381] Updated weights for policy 1, policy_version 223883 (0.0007) [2023-12-27 07:12:11,321][20347] Updated weights for policy 0, policy_version 224172 (0.0008) [2023-12-27 07:12:11,382][20347] Updated weights for policy 0, policy_version 224182 (0.0008) [2023-12-27 07:12:11,446][20347] Updated weights for policy 0, policy_version 224192 (0.0008) [2023-12-27 07:12:11,559][20381] Updated weights for policy 1, policy_version 223893 (0.0008) [2023-12-27 07:12:11,617][20381] Updated weights for policy 1, policy_version 223903 (0.0007) [2023-12-27 07:12:11,686][20381] Updated weights for policy 1, policy_version 223913 (0.0008) [2023-12-27 07:12:12,154][20347] Updated weights for policy 0, policy_version 224202 (0.0008) [2023-12-27 07:12:12,210][20347] Updated weights for policy 0, policy_version 224212 (0.0007) [2023-12-27 07:12:12,269][20347] Updated weights for policy 0, policy_version 224222 (0.0007) [2023-12-27 07:12:12,320][20347] Updated weights for policy 0, policy_version 224232 (0.0008) [2023-12-27 07:12:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 114753536. Throughput: 0: 10170.1, 1: 10319.7. Samples: 114743448. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '61.940')] [2023-12-27 07:12:12,409][20381] Updated weights for policy 1, policy_version 223923 (0.0008) [2023-12-27 07:12:12,461][20381] Updated weights for policy 1, policy_version 223933 (0.0006) [2023-12-27 07:12:12,520][20381] Updated weights for policy 1, policy_version 223943 (0.0006) [2023-12-27 07:12:13,064][20347] Updated weights for policy 0, policy_version 224242 (0.0008) [2023-12-27 07:12:13,115][20347] Updated weights for policy 0, policy_version 224252 (0.0007) [2023-12-27 07:12:13,122][20381] Updated weights for policy 1, policy_version 223953 (0.0006) [2023-12-27 07:12:13,171][20347] Updated weights for policy 0, policy_version 224262 (0.0006) [2023-12-27 07:12:13,177][20381] Updated weights for policy 1, policy_version 223963 (0.0011) [2023-12-27 07:12:13,226][20381] Updated weights for policy 1, policy_version 223973 (0.0010) [2023-12-27 07:12:13,283][20381] Updated weights for policy 1, policy_version 223983 (0.0010) [2023-12-27 07:12:13,891][20381] Updated weights for policy 1, policy_version 223993 (0.0006) [2023-12-27 07:12:13,909][20347] Updated weights for policy 0, policy_version 224272 (0.0007) [2023-12-27 07:12:13,937][20381] Updated weights for policy 1, policy_version 224003 (0.0005) [2023-12-27 07:12:13,959][20347] Updated weights for policy 0, policy_version 224282 (0.0007) [2023-12-27 07:12:13,988][20381] Updated weights for policy 1, policy_version 224013 (0.0005) [2023-12-27 07:12:14,016][20347] Updated weights for policy 0, policy_version 224292 (0.0008) [2023-12-27 07:12:14,530][20381] Updated weights for policy 1, policy_version 224023 (0.0006) [2023-12-27 07:12:14,593][20381] Updated weights for policy 1, policy_version 224033 (0.0007) [2023-12-27 07:12:14,659][20381] Updated weights for policy 1, policy_version 224043 (0.0008) [2023-12-27 07:12:14,731][20347] Updated weights for policy 0, policy_version 224302 (0.0007) [2023-12-27 07:12:14,788][20347] Updated weights for policy 0, policy_version 224312 (0.0006) [2023-12-27 07:12:14,825][20291] KL-divergence is very high: 281.6253 [2023-12-27 07:12:14,858][20347] Updated weights for policy 0, policy_version 224322 (0.0006) [2023-12-27 07:12:14,878][20291] KL-divergence is very high: 1206.7850 [2023-12-27 07:12:15,306][20381] Updated weights for policy 1, policy_version 224053 (0.0007) [2023-12-27 07:12:15,351][20381] Updated weights for policy 1, policy_version 224063 (0.0007) [2023-12-27 07:12:15,401][20381] Updated weights for policy 1, policy_version 224073 (0.0007) [2023-12-27 07:12:15,502][20347] Updated weights for policy 0, policy_version 224332 (0.0005) [2023-12-27 07:12:15,552][20347] Updated weights for policy 0, policy_version 224342 (0.0010) [2023-12-27 07:12:15,605][20347] Updated weights for policy 0, policy_version 224352 (0.0006) [2023-12-27 07:12:15,977][20381] Updated weights for policy 1, policy_version 224083 (0.0008) [2023-12-27 07:12:16,030][20381] Updated weights for policy 1, policy_version 224093 (0.0009) [2023-12-27 07:12:16,083][20381] Updated weights for policy 1, policy_version 224103 (0.0010) [2023-12-27 07:12:16,234][20347] Updated weights for policy 0, policy_version 224362 (0.0007) [2023-12-27 07:12:16,297][20347] Updated weights for policy 0, policy_version 224372 (0.0007) [2023-12-27 07:12:16,360][20347] Updated weights for policy 0, policy_version 224382 (0.0008) [2023-12-27 07:12:16,422][20347] Updated weights for policy 0, policy_version 224392 (0.0009) [2023-12-27 07:12:16,661][20381] Updated weights for policy 1, policy_version 224113 (0.0010) [2023-12-27 07:12:16,717][20381] Updated weights for policy 1, policy_version 224123 (0.0005) [2023-12-27 07:12:16,770][20381] Updated weights for policy 1, policy_version 224133 (0.0006) [2023-12-27 07:12:16,821][20381] Updated weights for policy 1, policy_version 224143 (0.0005) [2023-12-27 07:12:17,073][20347] Updated weights for policy 0, policy_version 224402 (0.0007) [2023-12-27 07:12:17,122][20347] Updated weights for policy 0, policy_version 224412 (0.0006) [2023-12-27 07:12:17,180][20347] Updated weights for policy 0, policy_version 224422 (0.0006) [2023-12-27 07:12:17,346][20381] Updated weights for policy 1, policy_version 224153 (0.0006) [2023-12-27 07:12:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 114860032. Throughput: 0: 10173.0, 1: 10404.4. Samples: 114871388. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:17,364][19170] Avg episode reward: [(0, '991.070'), (1, '68.540')] [2023-12-27 07:12:17,398][20381] Updated weights for policy 1, policy_version 224163 (0.0008) [2023-12-27 07:12:17,458][20381] Updated weights for policy 1, policy_version 224173 (0.0009) [2023-12-27 07:12:17,728][20347] Updated weights for policy 0, policy_version 224432 (0.0006) [2023-12-27 07:12:17,780][20347] Updated weights for policy 0, policy_version 224442 (0.0010) [2023-12-27 07:12:17,832][20347] Updated weights for policy 0, policy_version 224452 (0.0010) [2023-12-27 07:12:18,087][20381] Updated weights for policy 1, policy_version 224183 (0.0006) [2023-12-27 07:12:18,143][20381] Updated weights for policy 1, policy_version 224193 (0.0007) [2023-12-27 07:12:18,195][20381] Updated weights for policy 1, policy_version 224203 (0.0007) [2023-12-27 07:12:18,485][20347] Updated weights for policy 0, policy_version 224462 (0.0008) [2023-12-27 07:12:18,537][20347] Updated weights for policy 0, policy_version 224472 (0.0008) [2023-12-27 07:12:18,586][20347] Updated weights for policy 0, policy_version 224482 (0.0008) [2023-12-27 07:12:18,943][20381] Updated weights for policy 1, policy_version 224213 (0.0009) [2023-12-27 07:12:18,999][20381] Updated weights for policy 1, policy_version 224223 (0.0010) [2023-12-27 07:12:19,055][20381] Updated weights for policy 1, policy_version 224233 (0.0011) [2023-12-27 07:12:19,244][20347] Updated weights for policy 0, policy_version 224492 (0.0009) [2023-12-27 07:12:19,300][20347] Updated weights for policy 0, policy_version 224502 (0.0010) [2023-12-27 07:12:19,358][20347] Updated weights for policy 0, policy_version 224512 (0.0009) [2023-12-27 07:12:19,673][20381] Updated weights for policy 1, policy_version 224243 (0.0009) [2023-12-27 07:12:19,738][20381] Updated weights for policy 1, policy_version 224253 (0.0006) [2023-12-27 07:12:19,798][20381] Updated weights for policy 1, policy_version 224263 (0.0010) [2023-12-27 07:12:20,083][20347] Updated weights for policy 0, policy_version 224522 (0.0007) [2023-12-27 07:12:20,143][20347] Updated weights for policy 0, policy_version 224532 (0.0008) [2023-12-27 07:12:20,197][20347] Updated weights for policy 0, policy_version 224542 (0.0008) [2023-12-27 07:12:20,244][20347] Updated weights for policy 0, policy_version 224552 (0.0008) [2023-12-27 07:12:20,437][20381] Updated weights for policy 1, policy_version 224273 (0.0008) [2023-12-27 07:12:20,493][20381] Updated weights for policy 1, policy_version 224283 (0.0005) [2023-12-27 07:12:20,555][20381] Updated weights for policy 1, policy_version 224293 (0.0006) [2023-12-27 07:12:20,621][20381] Updated weights for policy 1, policy_version 224303 (0.0007) [2023-12-27 07:12:20,949][20347] Updated weights for policy 0, policy_version 224562 (0.0005) [2023-12-27 07:12:21,007][20347] Updated weights for policy 0, policy_version 224572 (0.0006) [2023-12-27 07:12:21,067][20347] Updated weights for policy 0, policy_version 224582 (0.0008) [2023-12-27 07:12:21,284][20381] Updated weights for policy 1, policy_version 224313 (0.0007) [2023-12-27 07:12:21,350][20381] Updated weights for policy 1, policy_version 224323 (0.0008) [2023-12-27 07:12:21,411][20381] Updated weights for policy 1, policy_version 224333 (0.0008) [2023-12-27 07:12:21,690][20347] Updated weights for policy 0, policy_version 224592 (0.0007) [2023-12-27 07:12:21,756][20347] Updated weights for policy 0, policy_version 224602 (0.0008) [2023-12-27 07:12:21,816][20347] Updated weights for policy 0, policy_version 224612 (0.0005) [2023-12-27 07:12:22,165][20381] Updated weights for policy 1, policy_version 224343 (0.0008) [2023-12-27 07:12:22,233][20381] Updated weights for policy 1, policy_version 224353 (0.0008) [2023-12-27 07:12:22,293][20381] Updated weights for policy 1, policy_version 224363 (0.0008) [2023-12-27 07:12:22,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 114966528. Throughput: 0: 10226.6, 1: 10430.5. Samples: 114935572. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:22,363][19170] Avg episode reward: [(0, '991.070'), (1, '69.920')] [2023-12-27 07:12:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000224368_57450496.pth... [2023-12-27 07:12:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000224616_57516032.pth... [2023-12-27 07:12:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000223120_57131008.pth [2023-12-27 07:12:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000223432_57212928.pth [2023-12-27 07:12:22,472][20347] Updated weights for policy 0, policy_version 224622 (0.0008) [2023-12-27 07:12:22,533][20347] Updated weights for policy 0, policy_version 224632 (0.0008) [2023-12-27 07:12:22,584][20347] Updated weights for policy 0, policy_version 224642 (0.0009) [2023-12-27 07:12:22,989][20381] Updated weights for policy 1, policy_version 224373 (0.0008) [2023-12-27 07:12:23,051][20381] Updated weights for policy 1, policy_version 224383 (0.0007) [2023-12-27 07:12:23,107][20381] Updated weights for policy 1, policy_version 224393 (0.0007) [2023-12-27 07:12:23,274][20347] Updated weights for policy 0, policy_version 224652 (0.0007) [2023-12-27 07:12:23,339][20347] Updated weights for policy 0, policy_version 224662 (0.0007) [2023-12-27 07:12:23,397][20347] Updated weights for policy 0, policy_version 224672 (0.0007) [2023-12-27 07:12:23,808][20381] Updated weights for policy 1, policy_version 224403 (0.0009) [2023-12-27 07:12:23,865][20381] Updated weights for policy 1, policy_version 224413 (0.0006) [2023-12-27 07:12:23,921][20381] Updated weights for policy 1, policy_version 224423 (0.0005) [2023-12-27 07:12:23,975][20347] Updated weights for policy 0, policy_version 224682 (0.0007) [2023-12-27 07:12:24,024][20347] Updated weights for policy 0, policy_version 224692 (0.0008) [2023-12-27 07:12:24,074][20347] Updated weights for policy 0, policy_version 224702 (0.0008) [2023-12-27 07:12:24,119][20347] Updated weights for policy 0, policy_version 224712 (0.0006) [2023-12-27 07:12:24,519][20381] Updated weights for policy 1, policy_version 224433 (0.0010) [2023-12-27 07:12:24,577][20381] Updated weights for policy 1, policy_version 224443 (0.0010) [2023-12-27 07:12:24,622][20381] Updated weights for policy 1, policy_version 224453 (0.0010) [2023-12-27 07:12:24,680][20381] Updated weights for policy 1, policy_version 224463 (0.0010) [2023-12-27 07:12:24,850][20347] Updated weights for policy 0, policy_version 224722 (0.0008) [2023-12-27 07:12:24,901][20347] Updated weights for policy 0, policy_version 224732 (0.0008) [2023-12-27 07:12:24,959][20347] Updated weights for policy 0, policy_version 224742 (0.0008) [2023-12-27 07:12:25,378][20381] Updated weights for policy 1, policy_version 224473 (0.0010) [2023-12-27 07:12:25,429][20381] Updated weights for policy 1, policy_version 224483 (0.0010) [2023-12-27 07:12:25,481][20381] Updated weights for policy 1, policy_version 224493 (0.0010) [2023-12-27 07:12:25,533][20347] Updated weights for policy 0, policy_version 224752 (0.0008) [2023-12-27 07:12:25,583][20347] Updated weights for policy 0, policy_version 224762 (0.0007) [2023-12-27 07:12:25,640][20347] Updated weights for policy 0, policy_version 224772 (0.0007) [2023-12-27 07:12:26,163][20381] Updated weights for policy 1, policy_version 224503 (0.0011) [2023-12-27 07:12:26,210][20381] Updated weights for policy 1, policy_version 224513 (0.0010) [2023-12-27 07:12:26,230][20347] Updated weights for policy 0, policy_version 224782 (0.0007) [2023-12-27 07:12:26,269][20381] Updated weights for policy 1, policy_version 224523 (0.0008) [2023-12-27 07:12:26,279][20347] Updated weights for policy 0, policy_version 224792 (0.0007) [2023-12-27 07:12:26,326][20347] Updated weights for policy 0, policy_version 224802 (0.0008) [2023-12-27 07:12:26,806][20381] Updated weights for policy 1, policy_version 224533 (0.0008) [2023-12-27 07:12:26,863][20381] Updated weights for policy 1, policy_version 224543 (0.0010) [2023-12-27 07:12:26,917][20381] Updated weights for policy 1, policy_version 224553 (0.0010) [2023-12-27 07:12:27,051][20347] Updated weights for policy 0, policy_version 224812 (0.0007) [2023-12-27 07:12:27,104][20347] Updated weights for policy 0, policy_version 224822 (0.0007) [2023-12-27 07:12:27,168][20347] Updated weights for policy 0, policy_version 224832 (0.0005) [2023-12-27 07:12:27,363][19170] Fps is (10 sec: 21298.2, 60 sec: 20616.3, 300 sec: 20716.0). Total num frames: 115073024. Throughput: 0: 10314.0, 1: 10459.3. Samples: 115059856. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '70.790')] [2023-12-27 07:12:27,616][20381] Updated weights for policy 1, policy_version 224563 (0.0010) [2023-12-27 07:12:27,669][20381] Updated weights for policy 1, policy_version 224573 (0.0010) [2023-12-27 07:12:27,721][20381] Updated weights for policy 1, policy_version 224583 (0.0010) [2023-12-27 07:12:27,735][20347] Updated weights for policy 0, policy_version 224842 (0.0006) [2023-12-27 07:12:27,793][20347] Updated weights for policy 0, policy_version 224852 (0.0007) [2023-12-27 07:12:27,847][20347] Updated weights for policy 0, policy_version 224862 (0.0007) [2023-12-27 07:12:27,908][20347] Updated weights for policy 0, policy_version 224872 (0.0007) [2023-12-27 07:12:28,407][20381] Updated weights for policy 1, policy_version 224593 (0.0010) [2023-12-27 07:12:28,463][20381] Updated weights for policy 1, policy_version 224603 (0.0011) [2023-12-27 07:12:28,511][20381] Updated weights for policy 1, policy_version 224613 (0.0010) [2023-12-27 07:12:28,571][20381] Updated weights for policy 1, policy_version 224623 (0.0011) [2023-12-27 07:12:28,604][20347] Updated weights for policy 0, policy_version 224882 (0.0006) [2023-12-27 07:12:28,658][20347] Updated weights for policy 0, policy_version 224892 (0.0007) [2023-12-27 07:12:28,705][20347] Updated weights for policy 0, policy_version 224902 (0.0007) [2023-12-27 07:12:29,247][20381] Updated weights for policy 1, policy_version 224633 (0.0010) [2023-12-27 07:12:29,305][20381] Updated weights for policy 1, policy_version 224643 (0.0010) [2023-12-27 07:12:29,365][20381] Updated weights for policy 1, policy_version 224653 (0.0010) [2023-12-27 07:12:29,397][20347] Updated weights for policy 0, policy_version 224912 (0.0008) [2023-12-27 07:12:29,459][20347] Updated weights for policy 0, policy_version 224922 (0.0007) [2023-12-27 07:12:29,519][20347] Updated weights for policy 0, policy_version 224932 (0.0007) [2023-12-27 07:12:29,974][20381] Updated weights for policy 1, policy_version 224663 (0.0010) [2023-12-27 07:12:30,029][20381] Updated weights for policy 1, policy_version 224673 (0.0010) [2023-12-27 07:12:30,084][20381] Updated weights for policy 1, policy_version 224683 (0.0010) [2023-12-27 07:12:30,253][20347] Updated weights for policy 0, policy_version 224942 (0.0008) [2023-12-27 07:12:30,310][20347] Updated weights for policy 0, policy_version 224952 (0.0007) [2023-12-27 07:12:30,367][20347] Updated weights for policy 0, policy_version 224962 (0.0007) [2023-12-27 07:12:30,774][20381] Updated weights for policy 1, policy_version 224693 (0.0009) [2023-12-27 07:12:30,834][20381] Updated weights for policy 1, policy_version 224703 (0.0008) [2023-12-27 07:12:30,881][20381] Updated weights for policy 1, policy_version 224713 (0.0008) [2023-12-27 07:12:31,047][20347] Updated weights for policy 0, policy_version 224972 (0.0007) [2023-12-27 07:12:31,105][20347] Updated weights for policy 0, policy_version 224982 (0.0007) [2023-12-27 07:12:31,168][20347] Updated weights for policy 0, policy_version 224992 (0.0008) [2023-12-27 07:12:31,581][20381] Updated weights for policy 1, policy_version 224723 (0.0007) [2023-12-27 07:12:31,646][20381] Updated weights for policy 1, policy_version 224733 (0.0007) [2023-12-27 07:12:31,703][20381] Updated weights for policy 1, policy_version 224743 (0.0009) [2023-12-27 07:12:31,824][20347] Updated weights for policy 0, policy_version 225002 (0.0006) [2023-12-27 07:12:31,878][20347] Updated weights for policy 0, policy_version 225012 (0.0007) [2023-12-27 07:12:31,929][20347] Updated weights for policy 0, policy_version 225022 (0.0008) [2023-12-27 07:12:31,980][20347] Updated weights for policy 0, policy_version 225032 (0.0005) [2023-12-27 07:12:32,304][20381] Updated weights for policy 1, policy_version 224753 (0.0011) [2023-12-27 07:12:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 115171328. Throughput: 0: 10319.7, 1: 10489.9. Samples: 115185032. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '90.060')] [2023-12-27 07:12:32,367][20381] Updated weights for policy 1, policy_version 224763 (0.0011) [2023-12-27 07:12:32,426][20381] Updated weights for policy 1, policy_version 224773 (0.0011) [2023-12-27 07:12:32,487][20381] Updated weights for policy 1, policy_version 224783 (0.0011) [2023-12-27 07:12:32,592][20347] Updated weights for policy 0, policy_version 225042 (0.0006) [2023-12-27 07:12:32,653][20347] Updated weights for policy 0, policy_version 225052 (0.0007) [2023-12-27 07:12:32,717][20347] Updated weights for policy 0, policy_version 225062 (0.0007) [2023-12-27 07:12:33,118][20381] Updated weights for policy 1, policy_version 224793 (0.0010) [2023-12-27 07:12:33,167][20381] Updated weights for policy 1, policy_version 224803 (0.0010) [2023-12-27 07:12:33,215][20381] Updated weights for policy 1, policy_version 224813 (0.0010) [2023-12-27 07:12:33,316][20347] Updated weights for policy 0, policy_version 225072 (0.0006) [2023-12-27 07:12:33,363][20347] Updated weights for policy 0, policy_version 225082 (0.0007) [2023-12-27 07:12:33,418][20347] Updated weights for policy 0, policy_version 225092 (0.0006) [2023-12-27 07:12:33,896][20381] Updated weights for policy 1, policy_version 224823 (0.0010) [2023-12-27 07:12:33,945][20381] Updated weights for policy 1, policy_version 224833 (0.0011) [2023-12-27 07:12:33,994][20381] Updated weights for policy 1, policy_version 224843 (0.0010) [2023-12-27 07:12:34,037][20347] Updated weights for policy 0, policy_version 225102 (0.0006) [2023-12-27 07:12:34,096][20347] Updated weights for policy 0, policy_version 225112 (0.0006) [2023-12-27 07:12:34,164][20347] Updated weights for policy 0, policy_version 225122 (0.0007) [2023-12-27 07:12:34,702][20381] Updated weights for policy 1, policy_version 224853 (0.0009) [2023-12-27 07:12:34,764][20381] Updated weights for policy 1, policy_version 224863 (0.0008) [2023-12-27 07:12:34,824][20381] Updated weights for policy 1, policy_version 224873 (0.0007) [2023-12-27 07:12:34,828][20347] Updated weights for policy 0, policy_version 225132 (0.0007) [2023-12-27 07:12:34,885][20347] Updated weights for policy 0, policy_version 225142 (0.0006) [2023-12-27 07:12:34,945][20347] Updated weights for policy 0, policy_version 225152 (0.0007) [2023-12-27 07:12:35,498][20381] Updated weights for policy 1, policy_version 224883 (0.0010) [2023-12-27 07:12:35,557][20381] Updated weights for policy 1, policy_version 224893 (0.0011) [2023-12-27 07:12:35,596][20347] Updated weights for policy 0, policy_version 225162 (0.0008) [2023-12-27 07:12:35,610][20381] Updated weights for policy 1, policy_version 224903 (0.0011) [2023-12-27 07:12:35,653][20347] Updated weights for policy 0, policy_version 225172 (0.0006) [2023-12-27 07:12:35,710][20347] Updated weights for policy 0, policy_version 225182 (0.0008) [2023-12-27 07:12:35,777][20347] Updated weights for policy 0, policy_version 225192 (0.0009) [2023-12-27 07:12:36,245][20381] Updated weights for policy 1, policy_version 224913 (0.0005) [2023-12-27 07:12:36,310][20381] Updated weights for policy 1, policy_version 224923 (0.0007) [2023-12-27 07:12:36,374][20381] Updated weights for policy 1, policy_version 224933 (0.0008) [2023-12-27 07:12:36,424][20381] Updated weights for policy 1, policy_version 224943 (0.0007) [2023-12-27 07:12:36,451][20347] Updated weights for policy 0, policy_version 225202 (0.0010) [2023-12-27 07:12:36,509][20347] Updated weights for policy 0, policy_version 225212 (0.0010) [2023-12-27 07:12:36,578][20347] Updated weights for policy 0, policy_version 225222 (0.0010) [2023-12-27 07:12:37,015][20381] Updated weights for policy 1, policy_version 224953 (0.0006) [2023-12-27 07:12:37,077][20381] Updated weights for policy 1, policy_version 224963 (0.0006) [2023-12-27 07:12:37,138][20381] Updated weights for policy 1, policy_version 224973 (0.0006) [2023-12-27 07:12:37,265][20347] Updated weights for policy 0, policy_version 225232 (0.0009) [2023-12-27 07:12:37,326][20347] Updated weights for policy 0, policy_version 225242 (0.0009) [2023-12-27 07:12:37,362][19170] Fps is (10 sec: 20481.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 115277824. Throughput: 0: 10384.8, 1: 10451.8. Samples: 115248640. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '95.740')] [2023-12-27 07:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000224976_57606144.pth... [2023-12-27 07:12:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000223728_57286656.pth [2023-12-27 07:12:37,379][20347] Updated weights for policy 0, policy_version 225252 (0.0006) [2023-12-27 07:12:37,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000225256_57679872.pth... [2023-12-27 07:12:37,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000224040_57368576.pth [2023-12-27 07:12:37,767][20381] Updated weights for policy 1, policy_version 224983 (0.0006) [2023-12-27 07:12:37,826][20381] Updated weights for policy 1, policy_version 224993 (0.0006) [2023-12-27 07:12:37,885][20381] Updated weights for policy 1, policy_version 225003 (0.0007) [2023-12-27 07:12:37,982][20347] Updated weights for policy 0, policy_version 225262 (0.0006) [2023-12-27 07:12:38,040][20347] Updated weights for policy 0, policy_version 225272 (0.0007) [2023-12-27 07:12:38,095][20347] Updated weights for policy 0, policy_version 225282 (0.0007) [2023-12-27 07:12:38,523][20381] Updated weights for policy 1, policy_version 225013 (0.0009) [2023-12-27 07:12:38,575][20381] Updated weights for policy 1, policy_version 225023 (0.0008) [2023-12-27 07:12:38,634][20381] Updated weights for policy 1, policy_version 225033 (0.0007) [2023-12-27 07:12:38,756][20347] Updated weights for policy 0, policy_version 225292 (0.0009) [2023-12-27 07:12:38,812][20347] Updated weights for policy 0, policy_version 225302 (0.0010) [2023-12-27 07:12:38,869][20347] Updated weights for policy 0, policy_version 225312 (0.0010) [2023-12-27 07:12:39,316][20381] Updated weights for policy 1, policy_version 225043 (0.0009) [2023-12-27 07:12:39,379][20381] Updated weights for policy 1, policy_version 225053 (0.0011) [2023-12-27 07:12:39,438][20381] Updated weights for policy 1, policy_version 225063 (0.0010) [2023-12-27 07:12:39,578][20347] Updated weights for policy 0, policy_version 225322 (0.0009) [2023-12-27 07:12:39,638][20347] Updated weights for policy 0, policy_version 225332 (0.0005) [2023-12-27 07:12:39,693][20347] Updated weights for policy 0, policy_version 225342 (0.0005) [2023-12-27 07:12:39,747][20347] Updated weights for policy 0, policy_version 225352 (0.0006) [2023-12-27 07:12:40,146][20381] Updated weights for policy 1, policy_version 225073 (0.0007) [2023-12-27 07:12:40,224][20381] Updated weights for policy 1, policy_version 225083 (0.0011) [2023-12-27 07:12:40,280][20381] Updated weights for policy 1, policy_version 225093 (0.0008) [2023-12-27 07:12:40,343][20381] Updated weights for policy 1, policy_version 225103 (0.0010) [2023-12-27 07:12:40,395][20347] Updated weights for policy 0, policy_version 225362 (0.0007) [2023-12-27 07:12:40,455][20347] Updated weights for policy 0, policy_version 225372 (0.0007) [2023-12-27 07:12:40,513][20347] Updated weights for policy 0, policy_version 225382 (0.0005) [2023-12-27 07:12:40,976][20381] Updated weights for policy 1, policy_version 225113 (0.0010) [2023-12-27 07:12:41,040][20381] Updated weights for policy 1, policy_version 225123 (0.0010) [2023-12-27 07:12:41,103][20381] Updated weights for policy 1, policy_version 225133 (0.0007) [2023-12-27 07:12:41,180][20347] Updated weights for policy 0, policy_version 225392 (0.0007) [2023-12-27 07:12:41,239][20347] Updated weights for policy 0, policy_version 225402 (0.0008) [2023-12-27 07:12:41,305][20347] Updated weights for policy 0, policy_version 225412 (0.0008) [2023-12-27 07:12:41,832][20381] Updated weights for policy 1, policy_version 225143 (0.0009) [2023-12-27 07:12:41,889][20381] Updated weights for policy 1, policy_version 225153 (0.0010) [2023-12-27 07:12:41,956][20381] Updated weights for policy 1, policy_version 225163 (0.0008) [2023-12-27 07:12:42,031][20347] Updated weights for policy 0, policy_version 225422 (0.0009) [2023-12-27 07:12:42,084][20347] Updated weights for policy 0, policy_version 225432 (0.0010) [2023-12-27 07:12:42,141][20347] Updated weights for policy 0, policy_version 225442 (0.0009) [2023-12-27 07:12:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 115384320. Throughput: 0: 10322.0, 1: 10492.9. Samples: 115373348. Policy #0 lag: (min: 15.0, avg: 18.5, max: 47.0) [2023-12-27 07:12:42,363][19170] Avg episode reward: [(0, '991.530'), (1, '50.870')] [2023-12-27 07:12:42,663][20381] Updated weights for policy 1, policy_version 225173 (0.0008) [2023-12-27 07:12:42,719][20381] Updated weights for policy 1, policy_version 225183 (0.0007) [2023-12-27 07:12:42,778][20381] Updated weights for policy 1, policy_version 225193 (0.0007) [2023-12-27 07:12:42,866][20347] Updated weights for policy 0, policy_version 225452 (0.0009) [2023-12-27 07:12:42,913][20347] Updated weights for policy 0, policy_version 225462 (0.0008) [2023-12-27 07:12:42,959][20347] Updated weights for policy 0, policy_version 225472 (0.0010) [2023-12-27 07:12:43,382][20381] Updated weights for policy 1, policy_version 225203 (0.0007) [2023-12-27 07:12:43,443][20381] Updated weights for policy 1, policy_version 225213 (0.0007) [2023-12-27 07:12:43,501][20381] Updated weights for policy 1, policy_version 225224 (0.0007) [2023-12-27 07:12:43,634][20347] Updated weights for policy 0, policy_version 225482 (0.0008) [2023-12-27 07:12:43,683][20347] Updated weights for policy 0, policy_version 225492 (0.0008) [2023-12-27 07:12:43,735][20347] Updated weights for policy 0, policy_version 225502 (0.0010) [2023-12-27 07:12:43,787][20347] Updated weights for policy 0, policy_version 225512 (0.0010) [2023-12-27 07:12:44,129][20381] Updated weights for policy 1, policy_version 225234 (0.0005) [2023-12-27 07:12:44,182][20381] Updated weights for policy 1, policy_version 225244 (0.0005) [2023-12-27 07:12:44,242][20381] Updated weights for policy 1, policy_version 225254 (0.0007) [2023-12-27 07:12:44,288][20381] Updated weights for policy 1, policy_version 225264 (0.0007) [2023-12-27 07:12:44,424][20347] Updated weights for policy 0, policy_version 225522 (0.0005) [2023-12-27 07:12:44,438][20291] KL-divergence is very high: 132.0614 [2023-12-27 07:12:44,454][20291] KL-divergence is very high: 184.4648 [2023-12-27 07:12:44,481][20347] Updated weights for policy 0, policy_version 225532 (0.0005) [2023-12-27 07:12:44,482][20291] KL-divergence is very high: 418.5869 [2023-12-27 07:12:44,487][20291] KL-divergence is very high: 127.9681 [2023-12-27 07:12:44,527][20291] KL-divergence is very high: 514.5842 [2023-12-27 07:12:44,533][20291] KL-divergence is very high: 435.1329 [2023-12-27 07:12:44,539][20291] KL-divergence is very high: 198.6139 [2023-12-27 07:12:44,540][20347] Updated weights for policy 0, policy_version 225542 (0.0007) [2023-12-27 07:12:44,931][20381] Updated weights for policy 1, policy_version 225274 (0.0011) [2023-12-27 07:12:44,995][20381] Updated weights for policy 1, policy_version 225284 (0.0008) [2023-12-27 07:12:45,058][20381] Updated weights for policy 1, policy_version 225294 (0.0009) [2023-12-27 07:12:45,100][20291] KL-divergence is very high: 214.2611 [2023-12-27 07:12:45,106][20291] KL-divergence is very high: 699.1426 [2023-12-27 07:12:45,117][20291] KL-divergence is very high: 382.9404 [2023-12-27 07:12:45,128][20347] Updated weights for policy 0, policy_version 225552 (0.0006) [2023-12-27 07:12:45,156][20291] KL-divergence is very high: 128.9892 [2023-12-27 07:12:45,167][20291] KL-divergence is very high: 111.8713 [2023-12-27 07:12:45,193][20347] Updated weights for policy 0, policy_version 225562 (0.0007) [2023-12-27 07:12:45,252][20347] Updated weights for policy 0, policy_version 225572 (0.0006) [2023-12-27 07:12:45,798][20381] Updated weights for policy 1, policy_version 225304 (0.0005) [2023-12-27 07:12:45,801][20347] Updated weights for policy 0, policy_version 225582 (0.0007) [2023-12-27 07:12:45,852][20347] Updated weights for policy 0, policy_version 225592 (0.0008) [2023-12-27 07:12:45,854][20381] Updated weights for policy 1, policy_version 225314 (0.0005) [2023-12-27 07:12:45,906][20347] Updated weights for policy 0, policy_version 225602 (0.0005) [2023-12-27 07:12:45,915][20381] Updated weights for policy 1, policy_version 225324 (0.0006) [2023-12-27 07:12:46,511][20347] Updated weights for policy 0, policy_version 225612 (0.0007) [2023-12-27 07:12:46,566][20381] Updated weights for policy 1, policy_version 225334 (0.0007) [2023-12-27 07:12:46,575][20347] Updated weights for policy 0, policy_version 225622 (0.0010) [2023-12-27 07:12:46,618][20381] Updated weights for policy 1, policy_version 225344 (0.0005) [2023-12-27 07:12:46,631][20347] Updated weights for policy 0, policy_version 225632 (0.0007) [2023-12-27 07:12:46,674][20381] Updated weights for policy 1, policy_version 225354 (0.0007) [2023-12-27 07:12:47,222][20347] Updated weights for policy 0, policy_version 225642 (0.0006) [2023-12-27 07:12:47,278][20347] Updated weights for policy 0, policy_version 225652 (0.0007) [2023-12-27 07:12:47,334][20347] Updated weights for policy 0, policy_version 225662 (0.0008) [2023-12-27 07:12:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 115482624. Throughput: 0: 10362.8, 1: 10486.8. Samples: 115498812. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:12:47,363][19170] Avg episode reward: [(0, '987.740'), (1, '40.970')] [2023-12-27 07:12:47,387][20381] Updated weights for policy 1, policy_version 225364 (0.0007) [2023-12-27 07:12:47,389][20347] Updated weights for policy 0, policy_version 225672 (0.0007) [2023-12-27 07:12:47,435][20381] Updated weights for policy 1, policy_version 225374 (0.0007) [2023-12-27 07:12:47,485][20381] Updated weights for policy 1, policy_version 225384 (0.0008) [2023-12-27 07:12:47,980][20347] Updated weights for policy 0, policy_version 225682 (0.0007) [2023-12-27 07:12:48,036][20347] Updated weights for policy 0, policy_version 225692 (0.0009) [2023-12-27 07:12:48,085][20347] Updated weights for policy 0, policy_version 225702 (0.0009) [2023-12-27 07:12:48,128][20381] Updated weights for policy 1, policy_version 225394 (0.0007) [2023-12-27 07:12:48,186][20381] Updated weights for policy 1, policy_version 225404 (0.0008) [2023-12-27 07:12:48,249][20381] Updated weights for policy 1, policy_version 225414 (0.0011) [2023-12-27 07:12:48,306][20381] Updated weights for policy 1, policy_version 225424 (0.0010) [2023-12-27 07:12:48,809][20347] Updated weights for policy 0, policy_version 225712 (0.0008) [2023-12-27 07:12:48,867][20347] Updated weights for policy 0, policy_version 225722 (0.0007) [2023-12-27 07:12:48,915][20347] Updated weights for policy 0, policy_version 225732 (0.0007) [2023-12-27 07:12:48,973][20381] Updated weights for policy 1, policy_version 225434 (0.0010) [2023-12-27 07:12:49,027][20381] Updated weights for policy 1, policy_version 225444 (0.0007) [2023-12-27 07:12:49,090][20381] Updated weights for policy 1, policy_version 225454 (0.0007) [2023-12-27 07:12:49,634][20347] Updated weights for policy 0, policy_version 225742 (0.0007) [2023-12-27 07:12:49,697][20347] Updated weights for policy 0, policy_version 225752 (0.0006) [2023-12-27 07:12:49,761][20347] Updated weights for policy 0, policy_version 225762 (0.0009) [2023-12-27 07:12:49,790][20381] Updated weights for policy 1, policy_version 225464 (0.0007) [2023-12-27 07:12:49,854][20381] Updated weights for policy 1, policy_version 225474 (0.0007) [2023-12-27 07:12:49,910][20381] Updated weights for policy 1, policy_version 225484 (0.0010) [2023-12-27 07:12:50,431][20347] Updated weights for policy 0, policy_version 225772 (0.0007) [2023-12-27 07:12:50,484][20347] Updated weights for policy 0, policy_version 225782 (0.0005) [2023-12-27 07:12:50,533][20347] Updated weights for policy 0, policy_version 225792 (0.0007) [2023-12-27 07:12:50,588][20381] Updated weights for policy 1, policy_version 225494 (0.0008) [2023-12-27 07:12:50,641][20381] Updated weights for policy 1, policy_version 225504 (0.0008) [2023-12-27 07:12:50,700][20381] Updated weights for policy 1, policy_version 225514 (0.0008) [2023-12-27 07:12:51,263][20347] Updated weights for policy 0, policy_version 225802 (0.0007) [2023-12-27 07:12:51,319][20347] Updated weights for policy 0, policy_version 225812 (0.0008) [2023-12-27 07:12:51,323][20381] Updated weights for policy 1, policy_version 225524 (0.0007) [2023-12-27 07:12:51,383][20347] Updated weights for policy 0, policy_version 225822 (0.0008) [2023-12-27 07:12:51,388][20381] Updated weights for policy 1, policy_version 225534 (0.0009) [2023-12-27 07:12:51,442][20347] Updated weights for policy 0, policy_version 225832 (0.0006) [2023-12-27 07:12:51,447][20381] Updated weights for policy 1, policy_version 225544 (0.0006) [2023-12-27 07:12:52,119][20381] Updated weights for policy 1, policy_version 225554 (0.0008) [2023-12-27 07:12:52,133][20347] Updated weights for policy 0, policy_version 225842 (0.0010) [2023-12-27 07:12:52,181][20381] Updated weights for policy 1, policy_version 225564 (0.0006) [2023-12-27 07:12:52,192][20347] Updated weights for policy 0, policy_version 225852 (0.0010) [2023-12-27 07:12:52,233][20381] Updated weights for policy 1, policy_version 225574 (0.0005) [2023-12-27 07:12:52,255][20347] Updated weights for policy 0, policy_version 225862 (0.0010) [2023-12-27 07:12:52,289][20381] Updated weights for policy 1, policy_version 225584 (0.0008) [2023-12-27 07:12:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 115597312. Throughput: 0: 10396.8, 1: 10470.9. Samples: 115560912. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:12:52,363][19170] Avg episode reward: [(0, '989.740'), (1, '39.370')] [2023-12-27 07:12:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000225584_57761792.pth... [2023-12-27 07:12:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000225864_57835520.pth... [2023-12-27 07:12:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000224368_57450496.pth [2023-12-27 07:12:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000224616_57516032.pth [2023-12-27 07:12:52,914][20381] Updated weights for policy 1, policy_version 225594 (0.0007) [2023-12-27 07:12:52,965][20347] Updated weights for policy 0, policy_version 225872 (0.0007) [2023-12-27 07:12:52,971][20381] Updated weights for policy 1, policy_version 225604 (0.0007) [2023-12-27 07:12:53,025][20347] Updated weights for policy 0, policy_version 225882 (0.0006) [2023-12-27 07:12:53,025][20381] Updated weights for policy 1, policy_version 225614 (0.0008) [2023-12-27 07:12:53,083][20347] Updated weights for policy 0, policy_version 225892 (0.0005) [2023-12-27 07:12:53,627][20381] Updated weights for policy 1, policy_version 225624 (0.0011) [2023-12-27 07:12:53,693][20381] Updated weights for policy 1, policy_version 225634 (0.0011) [2023-12-27 07:12:53,700][20347] Updated weights for policy 0, policy_version 225902 (0.0008) [2023-12-27 07:12:53,745][20347] Updated weights for policy 0, policy_version 225912 (0.0010) [2023-12-27 07:12:53,749][20381] Updated weights for policy 1, policy_version 225644 (0.0011) [2023-12-27 07:12:53,794][20347] Updated weights for policy 0, policy_version 225922 (0.0010) [2023-12-27 07:12:54,435][20381] Updated weights for policy 1, policy_version 225654 (0.0011) [2023-12-27 07:12:54,435][20347] Updated weights for policy 0, policy_version 225932 (0.0010) [2023-12-27 07:12:54,493][20381] Updated weights for policy 1, policy_version 225664 (0.0011) [2023-12-27 07:12:54,518][20347] Updated weights for policy 0, policy_version 225942 (0.0010) [2023-12-27 07:12:54,552][20381] Updated weights for policy 1, policy_version 225674 (0.0011) [2023-12-27 07:12:54,578][20347] Updated weights for policy 0, policy_version 225952 (0.0010) [2023-12-27 07:12:55,202][20347] Updated weights for policy 0, policy_version 225962 (0.0010) [2023-12-27 07:12:55,211][20381] Updated weights for policy 1, policy_version 225684 (0.0011) [2023-12-27 07:12:55,257][20347] Updated weights for policy 0, policy_version 225972 (0.0010) [2023-12-27 07:12:55,266][20381] Updated weights for policy 1, policy_version 225694 (0.0010) [2023-12-27 07:12:55,304][20347] Updated weights for policy 0, policy_version 225982 (0.0010) [2023-12-27 07:12:55,310][20381] Updated weights for policy 1, policy_version 225704 (0.0010) [2023-12-27 07:12:55,362][20347] Updated weights for policy 0, policy_version 225992 (0.0010) [2023-12-27 07:12:55,969][20347] Updated weights for policy 0, policy_version 226002 (0.0010) [2023-12-27 07:12:55,976][20381] Updated weights for policy 1, policy_version 225714 (0.0011) [2023-12-27 07:12:56,023][20347] Updated weights for policy 0, policy_version 226012 (0.0010) [2023-12-27 07:12:56,031][20381] Updated weights for policy 1, policy_version 225724 (0.0011) [2023-12-27 07:12:56,073][20347] Updated weights for policy 0, policy_version 226022 (0.0010) [2023-12-27 07:12:56,094][20381] Updated weights for policy 1, policy_version 225734 (0.0011) [2023-12-27 07:12:56,145][20381] Updated weights for policy 1, policy_version 225744 (0.0010) [2023-12-27 07:12:56,752][20347] Updated weights for policy 0, policy_version 226032 (0.0010) [2023-12-27 07:12:56,805][20347] Updated weights for policy 0, policy_version 226042 (0.0010) [2023-12-27 07:12:56,829][20381] Updated weights for policy 1, policy_version 225754 (0.0011) [2023-12-27 07:12:56,860][20347] Updated weights for policy 0, policy_version 226052 (0.0010) [2023-12-27 07:12:56,882][20381] Updated weights for policy 1, policy_version 225764 (0.0011) [2023-12-27 07:12:56,931][20381] Updated weights for policy 1, policy_version 225774 (0.0011) [2023-12-27 07:12:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 115695616. Throughput: 0: 10447.1, 1: 10498.2. Samples: 115685988. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:12:57,363][19170] Avg episode reward: [(0, '982.370'), (1, '49.370')] [2023-12-27 07:12:57,487][20347] Updated weights for policy 0, policy_version 226062 (0.0008) [2023-12-27 07:12:57,540][20347] Updated weights for policy 0, policy_version 226072 (0.0008) [2023-12-27 07:12:57,546][20381] Updated weights for policy 1, policy_version 225784 (0.0008) [2023-12-27 07:12:57,585][20347] Updated weights for policy 0, policy_version 226082 (0.0005) [2023-12-27 07:12:57,591][20381] Updated weights for policy 1, policy_version 225794 (0.0010) [2023-12-27 07:12:57,602][20291] KL-divergence is very high: 262.2589 [2023-12-27 07:12:57,636][20381] Updated weights for policy 1, policy_version 225804 (0.0010) [2023-12-27 07:12:58,238][20347] Updated weights for policy 0, policy_version 226092 (0.0007) [2023-12-27 07:12:58,301][20347] Updated weights for policy 0, policy_version 226102 (0.0007) [2023-12-27 07:12:58,328][20381] Updated weights for policy 1, policy_version 225814 (0.0008) [2023-12-27 07:12:58,360][20347] Updated weights for policy 0, policy_version 226112 (0.0009) [2023-12-27 07:12:58,394][20381] Updated weights for policy 1, policy_version 225824 (0.0009) [2023-12-27 07:12:58,458][20381] Updated weights for policy 1, policy_version 225834 (0.0007) [2023-12-27 07:12:59,193][20347] Updated weights for policy 0, policy_version 226122 (0.0008) [2023-12-27 07:12:59,217][20381] Updated weights for policy 1, policy_version 225844 (0.0008) [2023-12-27 07:12:59,266][20347] Updated weights for policy 0, policy_version 226132 (0.0009) [2023-12-27 07:12:59,280][20381] Updated weights for policy 1, policy_version 225854 (0.0008) [2023-12-27 07:12:59,326][20381] Updated weights for policy 1, policy_version 225864 (0.0006) [2023-12-27 07:12:59,332][20347] Updated weights for policy 0, policy_version 226142 (0.0009) [2023-12-27 07:12:59,397][20347] Updated weights for policy 0, policy_version 226152 (0.0007) [2023-12-27 07:13:00,069][20381] Updated weights for policy 1, policy_version 225874 (0.0007) [2023-12-27 07:13:00,079][20347] Updated weights for policy 0, policy_version 226162 (0.0007) [2023-12-27 07:13:00,097][20291] KL-divergence is very high: 1723.8286 [2023-12-27 07:13:00,128][20381] Updated weights for policy 1, policy_version 225884 (0.0007) [2023-12-27 07:13:00,135][20347] Updated weights for policy 0, policy_version 226172 (0.0006) [2023-12-27 07:13:00,186][20381] Updated weights for policy 1, policy_version 225894 (0.0006) [2023-12-27 07:13:00,189][20347] Updated weights for policy 0, policy_version 226182 (0.0007) [2023-12-27 07:13:00,245][20381] Updated weights for policy 1, policy_version 225904 (0.0007) [2023-12-27 07:13:00,863][20347] Updated weights for policy 0, policy_version 226192 (0.0005) [2023-12-27 07:13:00,907][20381] Updated weights for policy 1, policy_version 225914 (0.0007) [2023-12-27 07:13:00,916][20347] Updated weights for policy 0, policy_version 226202 (0.0006) [2023-12-27 07:13:00,964][20381] Updated weights for policy 1, policy_version 225924 (0.0005) [2023-12-27 07:13:00,967][20347] Updated weights for policy 0, policy_version 226212 (0.0006) [2023-12-27 07:13:01,018][20381] Updated weights for policy 1, policy_version 225934 (0.0005) [2023-12-27 07:13:01,591][20347] Updated weights for policy 0, policy_version 226222 (0.0007) [2023-12-27 07:13:01,652][20347] Updated weights for policy 0, policy_version 226232 (0.0008) [2023-12-27 07:13:01,660][20381] Updated weights for policy 1, policy_version 225944 (0.0008) [2023-12-27 07:13:01,675][20291] KL-divergence is very high: 144.1376 [2023-12-27 07:13:01,712][20347] Updated weights for policy 0, policy_version 226242 (0.0006) [2023-12-27 07:13:01,717][20381] Updated weights for policy 1, policy_version 225954 (0.0008) [2023-12-27 07:13:01,777][20381] Updated weights for policy 1, policy_version 225964 (0.0008) [2023-12-27 07:13:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 115793920. Throughput: 0: 10463.4, 1: 10341.7. Samples: 115807616. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:02,363][19170] Avg episode reward: [(0, '851.640'), (1, '88.520')] [2023-12-27 07:13:02,366][20347] Updated weights for policy 0, policy_version 226252 (0.0008) [2023-12-27 07:13:02,425][20347] Updated weights for policy 0, policy_version 226262 (0.0007) [2023-12-27 07:13:02,455][20291] KL-divergence is very high: 188.7465 [2023-12-27 07:13:02,491][20347] Updated weights for policy 0, policy_version 226272 (0.0007) [2023-12-27 07:13:02,496][20291] KL-divergence is very high: 446.8150 [2023-12-27 07:13:02,502][20291] KL-divergence is very high: 298.8016 [2023-12-27 07:13:02,508][20291] KL-divergence is very high: 180.0008 [2023-12-27 07:13:02,525][20381] Updated weights for policy 1, policy_version 225974 (0.0009) [2023-12-27 07:13:02,577][20381] Updated weights for policy 1, policy_version 225984 (0.0010) [2023-12-27 07:13:02,635][20381] Updated weights for policy 1, policy_version 225994 (0.0006) [2023-12-27 07:13:03,143][20347] Updated weights for policy 0, policy_version 226282 (0.0008) [2023-12-27 07:13:03,146][20291] KL-divergence is very high: 104.1958 [2023-12-27 07:13:03,168][20291] KL-divergence is very high: 181.6217 [2023-12-27 07:13:03,187][20347] Updated weights for policy 0, policy_version 226292 (0.0007) [2023-12-27 07:13:03,206][20381] Updated weights for policy 1, policy_version 226004 (0.0005) [2023-12-27 07:13:03,207][20291] KL-divergence is very high: 101.7554 [2023-12-27 07:13:03,240][20347] Updated weights for policy 0, policy_version 226302 (0.0008) [2023-12-27 07:13:03,247][20291] KL-divergence is very high: 150.2844 [2023-12-27 07:13:03,257][20381] Updated weights for policy 1, policy_version 226014 (0.0005) [2023-12-27 07:13:03,286][20347] Updated weights for policy 0, policy_version 226312 (0.0005) [2023-12-27 07:13:03,311][20381] Updated weights for policy 1, policy_version 226024 (0.0005) [2023-12-27 07:13:03,951][20347] Updated weights for policy 0, policy_version 226322 (0.0010) [2023-12-27 07:13:03,973][20381] Updated weights for policy 1, policy_version 226034 (0.0010) [2023-12-27 07:13:04,006][20347] Updated weights for policy 0, policy_version 226332 (0.0005) [2023-12-27 07:13:04,028][20381] Updated weights for policy 1, policy_version 226044 (0.0009) [2023-12-27 07:13:04,070][20347] Updated weights for policy 0, policy_version 226342 (0.0006) [2023-12-27 07:13:04,083][20381] Updated weights for policy 1, policy_version 226054 (0.0007) [2023-12-27 07:13:04,148][20381] Updated weights for policy 1, policy_version 226064 (0.0007) [2023-12-27 07:13:04,740][20347] Updated weights for policy 0, policy_version 226352 (0.0009) [2023-12-27 07:13:04,792][20347] Updated weights for policy 0, policy_version 226362 (0.0010) [2023-12-27 07:13:04,844][20347] Updated weights for policy 0, policy_version 226372 (0.0010) [2023-12-27 07:13:04,854][20381] Updated weights for policy 1, policy_version 226074 (0.0011) [2023-12-27 07:13:04,912][20381] Updated weights for policy 1, policy_version 226084 (0.0010) [2023-12-27 07:13:04,973][20381] Updated weights for policy 1, policy_version 226094 (0.0010) [2023-12-27 07:13:05,395][20291] KL-divergence is very high: 120.4934 [2023-12-27 07:13:05,428][20347] Updated weights for policy 0, policy_version 226382 (0.0010) [2023-12-27 07:13:05,483][20347] Updated weights for policy 0, policy_version 226392 (0.0008) [2023-12-27 07:13:05,491][20291] KL-divergence is very high: 123.3521 [2023-12-27 07:13:05,537][20291] KL-divergence is very high: 157.0148 [2023-12-27 07:13:05,543][20347] Updated weights for policy 0, policy_version 226402 (0.0007) [2023-12-27 07:13:05,566][20381] Updated weights for policy 1, policy_version 226104 (0.0006) [2023-12-27 07:13:05,621][20381] Updated weights for policy 1, policy_version 226114 (0.0006) [2023-12-27 07:13:05,677][20381] Updated weights for policy 1, policy_version 226124 (0.0007) [2023-12-27 07:13:06,250][20347] Updated weights for policy 0, policy_version 226412 (0.0009) [2023-12-27 07:13:06,281][20381] Updated weights for policy 1, policy_version 226134 (0.0006) [2023-12-27 07:13:06,309][20347] Updated weights for policy 0, policy_version 226422 (0.0010) [2023-12-27 07:13:06,339][20381] Updated weights for policy 1, policy_version 226144 (0.0006) [2023-12-27 07:13:06,370][20347] Updated weights for policy 0, policy_version 226432 (0.0010) [2023-12-27 07:13:06,394][20381] Updated weights for policy 1, policy_version 226154 (0.0007) [2023-12-27 07:13:07,007][20347] Updated weights for policy 0, policy_version 226442 (0.0010) [2023-12-27 07:13:07,056][20347] Updated weights for policy 0, policy_version 226452 (0.0010) [2023-12-27 07:13:07,086][20381] Updated weights for policy 1, policy_version 226164 (0.0007) [2023-12-27 07:13:07,105][20347] Updated weights for policy 0, policy_version 226462 (0.0010) [2023-12-27 07:13:07,138][20381] Updated weights for policy 1, policy_version 226174 (0.0009) [2023-12-27 07:13:07,146][20291] KL-divergence is very high: 481.3246 [2023-12-27 07:13:07,157][20347] Updated weights for policy 0, policy_version 226472 (0.0010) [2023-12-27 07:13:07,200][20381] Updated weights for policy 1, policy_version 226184 (0.0006) [2023-12-27 07:13:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 115908608. Throughput: 0: 10449.1, 1: 10339.1. Samples: 115871040. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:07,363][19170] Avg episode reward: [(0, '507.960'), (1, '123.160')] [2023-12-27 07:13:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000226192_57917440.pth... [2023-12-27 07:13:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000226472_57991168.pth... [2023-12-27 07:13:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000224976_57606144.pth [2023-12-27 07:13:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000225256_57679872.pth [2023-12-27 07:13:07,783][20291] KL-divergence is very high: 118.0632 [2023-12-27 07:13:07,803][20347] Updated weights for policy 0, policy_version 226482 (0.0005) [2023-12-27 07:13:07,810][20381] Updated weights for policy 1, policy_version 226194 (0.0010) [2023-12-27 07:13:07,826][20291] KL-divergence is very high: 209.0018 [2023-12-27 07:13:07,860][20347] Updated weights for policy 0, policy_version 226492 (0.0009) [2023-12-27 07:13:07,867][20381] Updated weights for policy 1, policy_version 226204 (0.0010) [2023-12-27 07:13:07,872][20291] KL-divergence is very high: 114.3401 [2023-12-27 07:13:07,912][20381] Updated weights for policy 1, policy_version 226214 (0.0010) [2023-12-27 07:13:07,915][20347] Updated weights for policy 0, policy_version 226502 (0.0010) [2023-12-27 07:13:07,916][20291] KL-divergence is very high: 164.3009 [2023-12-27 07:13:07,962][20381] Updated weights for policy 1, policy_version 226224 (0.0008) [2023-12-27 07:13:08,469][20291] KL-divergence is very high: 148.9376 [2023-12-27 07:13:08,500][20291] KL-divergence is very high: 112.1145 [2023-12-27 07:13:08,521][20347] Updated weights for policy 0, policy_version 226512 (0.0010) [2023-12-27 07:13:08,526][20291] KL-divergence is very high: 115.9701 [2023-12-27 07:13:08,533][20291] KL-divergence is very high: 151.9180 [2023-12-27 07:13:08,552][20291] KL-divergence is very high: 123.3116 [2023-12-27 07:13:08,558][20291] KL-divergence is very high: 191.5069 [2023-12-27 07:13:08,578][20347] Updated weights for policy 0, policy_version 226522 (0.0009) [2023-12-27 07:13:08,595][20291] KL-divergence is very high: 117.2391 [2023-12-27 07:13:08,633][20347] Updated weights for policy 0, policy_version 226532 (0.0010) [2023-12-27 07:13:08,638][20291] KL-divergence is very high: 147.3146 [2023-12-27 07:13:08,644][20291] KL-divergence is very high: 143.8263 [2023-12-27 07:13:08,675][20381] Updated weights for policy 1, policy_version 226234 (0.0010) [2023-12-27 07:13:08,729][20381] Updated weights for policy 1, policy_version 226244 (0.0009) [2023-12-27 07:13:08,790][20381] Updated weights for policy 1, policy_version 226254 (0.0011) [2023-12-27 07:13:09,324][20291] KL-divergence is very high: 105.0173 [2023-12-27 07:13:09,330][20291] KL-divergence is very high: 149.0445 [2023-12-27 07:13:09,337][20291] KL-divergence is very high: 106.2468 [2023-12-27 07:13:09,365][20347] Updated weights for policy 0, policy_version 226542 (0.0009) [2023-12-27 07:13:09,436][20347] Updated weights for policy 0, policy_version 226552 (0.0007) [2023-12-27 07:13:09,439][20291] KL-divergence is very high: 105.6764 [2023-12-27 07:13:09,446][20291] KL-divergence is very high: 118.3793 [2023-12-27 07:13:09,481][20381] Updated weights for policy 1, policy_version 226264 (0.0009) [2023-12-27 07:13:09,505][20347] Updated weights for policy 0, policy_version 226562 (0.0007) [2023-12-27 07:13:09,532][20381] Updated weights for policy 1, policy_version 226274 (0.0006) [2023-12-27 07:13:09,588][20381] Updated weights for policy 1, policy_version 226284 (0.0007) [2023-12-27 07:13:10,208][20347] Updated weights for policy 0, policy_version 226572 (0.0007) [2023-12-27 07:13:10,269][20347] Updated weights for policy 0, policy_version 226582 (0.0006) [2023-12-27 07:13:10,326][20347] Updated weights for policy 0, policy_version 226592 (0.0006) [2023-12-27 07:13:10,342][20381] Updated weights for policy 1, policy_version 226294 (0.0010) [2023-12-27 07:13:10,398][20381] Updated weights for policy 1, policy_version 226304 (0.0011) [2023-12-27 07:13:10,449][20381] Updated weights for policy 1, policy_version 226314 (0.0011) [2023-12-27 07:13:10,913][20347] Updated weights for policy 0, policy_version 226602 (0.0006) [2023-12-27 07:13:10,961][20347] Updated weights for policy 0, policy_version 226612 (0.0007) [2023-12-27 07:13:11,009][20347] Updated weights for policy 0, policy_version 226622 (0.0006) [2023-12-27 07:13:11,074][20347] Updated weights for policy 0, policy_version 226632 (0.0008) [2023-12-27 07:13:11,150][20381] Updated weights for policy 1, policy_version 226324 (0.0009) [2023-12-27 07:13:11,212][20381] Updated weights for policy 1, policy_version 226334 (0.0008) [2023-12-27 07:13:11,279][20381] Updated weights for policy 1, policy_version 226344 (0.0008) [2023-12-27 07:13:11,848][20347] Updated weights for policy 0, policy_version 226642 (0.0011) [2023-12-27 07:13:11,907][20347] Updated weights for policy 0, policy_version 226652 (0.0010) [2023-12-27 07:13:11,965][20347] Updated weights for policy 0, policy_version 226662 (0.0010) [2023-12-27 07:13:12,005][20381] Updated weights for policy 1, policy_version 226354 (0.0007) [2023-12-27 07:13:12,054][20381] Updated weights for policy 1, policy_version 226364 (0.0007) [2023-12-27 07:13:12,103][20381] Updated weights for policy 1, policy_version 226374 (0.0007) [2023-12-27 07:13:12,151][20381] Updated weights for policy 1, policy_version 226384 (0.0007) [2023-12-27 07:13:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 116006912. Throughput: 0: 10439.8, 1: 10359.2. Samples: 115995800. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:12,363][19170] Avg episode reward: [(0, '75.500'), (1, '169.350')] [2023-12-27 07:13:12,641][20347] Updated weights for policy 0, policy_version 226672 (0.0011) [2023-12-27 07:13:12,704][20347] Updated weights for policy 0, policy_version 226682 (0.0009) [2023-12-27 07:13:12,765][20347] Updated weights for policy 0, policy_version 226692 (0.0011) [2023-12-27 07:13:12,927][20381] Updated weights for policy 1, policy_version 226394 (0.0008) [2023-12-27 07:13:12,981][20381] Updated weights for policy 1, policy_version 226404 (0.0007) [2023-12-27 07:13:13,033][20381] Updated weights for policy 1, policy_version 226414 (0.0007) [2023-12-27 07:13:13,418][20347] Updated weights for policy 0, policy_version 226702 (0.0008) [2023-12-27 07:13:13,475][20347] Updated weights for policy 0, policy_version 226712 (0.0006) [2023-12-27 07:13:13,535][20347] Updated weights for policy 0, policy_version 226722 (0.0007) [2023-12-27 07:13:13,723][20381] Updated weights for policy 1, policy_version 226424 (0.0009) [2023-12-27 07:13:13,776][20381] Updated weights for policy 1, policy_version 226434 (0.0006) [2023-12-27 07:13:13,825][20381] Updated weights for policy 1, policy_version 226444 (0.0011) [2023-12-27 07:13:14,182][20347] Updated weights for policy 0, policy_version 226732 (0.0007) [2023-12-27 07:13:14,239][20347] Updated weights for policy 0, policy_version 226742 (0.0007) [2023-12-27 07:13:14,295][20347] Updated weights for policy 0, policy_version 226752 (0.0007) [2023-12-27 07:13:14,475][20381] Updated weights for policy 1, policy_version 226454 (0.0010) [2023-12-27 07:13:14,520][20381] Updated weights for policy 1, policy_version 226464 (0.0008) [2023-12-27 07:13:14,566][20381] Updated weights for policy 1, policy_version 226474 (0.0006) [2023-12-27 07:13:15,035][20347] Updated weights for policy 0, policy_version 226762 (0.0007) [2023-12-27 07:13:15,094][20347] Updated weights for policy 0, policy_version 226772 (0.0007) [2023-12-27 07:13:15,154][20347] Updated weights for policy 0, policy_version 226782 (0.0008) [2023-12-27 07:13:15,210][20347] Updated weights for policy 0, policy_version 226792 (0.0007) [2023-12-27 07:13:15,245][20381] Updated weights for policy 1, policy_version 226484 (0.0008) [2023-12-27 07:13:15,304][20381] Updated weights for policy 1, policy_version 226494 (0.0008) [2023-12-27 07:13:15,356][20381] Updated weights for policy 1, policy_version 226504 (0.0007) [2023-12-27 07:13:15,900][20347] Updated weights for policy 0, policy_version 226802 (0.0008) [2023-12-27 07:13:15,958][20347] Updated weights for policy 0, policy_version 226812 (0.0008) [2023-12-27 07:13:16,014][20347] Updated weights for policy 0, policy_version 226822 (0.0008) [2023-12-27 07:13:16,029][20381] Updated weights for policy 1, policy_version 226514 (0.0007) [2023-12-27 07:13:16,088][20381] Updated weights for policy 1, policy_version 226524 (0.0007) [2023-12-27 07:13:16,139][20381] Updated weights for policy 1, policy_version 226534 (0.0008) [2023-12-27 07:13:16,190][20381] Updated weights for policy 1, policy_version 226544 (0.0006) [2023-12-27 07:13:16,697][20347] Updated weights for policy 0, policy_version 226832 (0.0007) [2023-12-27 07:13:16,743][20347] Updated weights for policy 0, policy_version 226842 (0.0005) [2023-12-27 07:13:16,793][20347] Updated weights for policy 0, policy_version 226852 (0.0005) [2023-12-27 07:13:16,817][20381] Updated weights for policy 1, policy_version 226554 (0.0008) [2023-12-27 07:13:16,866][20381] Updated weights for policy 1, policy_version 226564 (0.0008) [2023-12-27 07:13:16,912][20381] Updated weights for policy 1, policy_version 226574 (0.0006) [2023-12-27 07:13:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 116105216. Throughput: 0: 10407.0, 1: 10302.9. Samples: 116116976. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:17,363][19170] Avg episode reward: [(0, '660.250'), (1, '200.570')] [2023-12-27 07:13:17,481][20347] Updated weights for policy 0, policy_version 226862 (0.0007) [2023-12-27 07:13:17,528][20347] Updated weights for policy 0, policy_version 226872 (0.0008) [2023-12-27 07:13:17,582][20347] Updated weights for policy 0, policy_version 226882 (0.0008) [2023-12-27 07:13:17,611][20381] Updated weights for policy 1, policy_version 226584 (0.0005) [2023-12-27 07:13:17,660][20381] Updated weights for policy 1, policy_version 226594 (0.0005) [2023-12-27 07:13:17,721][20381] Updated weights for policy 1, policy_version 226604 (0.0006) [2023-12-27 07:13:18,267][20347] Updated weights for policy 0, policy_version 226892 (0.0008) [2023-12-27 07:13:18,327][20347] Updated weights for policy 0, policy_version 226902 (0.0010) [2023-12-27 07:13:18,377][20381] Updated weights for policy 1, policy_version 226614 (0.0008) [2023-12-27 07:13:18,386][20347] Updated weights for policy 0, policy_version 226912 (0.0010) [2023-12-27 07:13:18,438][20381] Updated weights for policy 1, policy_version 226624 (0.0009) [2023-12-27 07:13:18,490][20381] Updated weights for policy 1, policy_version 226634 (0.0008) [2023-12-27 07:13:19,014][20347] Updated weights for policy 0, policy_version 226922 (0.0010) [2023-12-27 07:13:19,070][20347] Updated weights for policy 0, policy_version 226932 (0.0009) [2023-12-27 07:13:19,120][20347] Updated weights for policy 0, policy_version 226942 (0.0005) [2023-12-27 07:13:19,158][20381] Updated weights for policy 1, policy_version 226644 (0.0008) [2023-12-27 07:13:19,173][20347] Updated weights for policy 0, policy_version 226952 (0.0007) [2023-12-27 07:13:19,209][20381] Updated weights for policy 1, policy_version 226654 (0.0008) [2023-12-27 07:13:19,270][20381] Updated weights for policy 1, policy_version 226664 (0.0008) [2023-12-27 07:13:19,817][20347] Updated weights for policy 0, policy_version 226962 (0.0006) [2023-12-27 07:13:19,883][20347] Updated weights for policy 0, policy_version 226972 (0.0007) [2023-12-27 07:13:19,941][20347] Updated weights for policy 0, policy_version 226982 (0.0008) [2023-12-27 07:13:20,002][20381] Updated weights for policy 1, policy_version 226674 (0.0007) [2023-12-27 07:13:20,058][20381] Updated weights for policy 1, policy_version 226684 (0.0008) [2023-12-27 07:13:20,118][20381] Updated weights for policy 1, policy_version 226694 (0.0008) [2023-12-27 07:13:20,177][20381] Updated weights for policy 1, policy_version 226704 (0.0008) [2023-12-27 07:13:20,554][20347] Updated weights for policy 0, policy_version 226992 (0.0010) [2023-12-27 07:13:20,616][20347] Updated weights for policy 0, policy_version 227002 (0.0011) [2023-12-27 07:13:20,673][20347] Updated weights for policy 0, policy_version 227012 (0.0010) [2023-12-27 07:13:20,928][20381] Updated weights for policy 1, policy_version 226714 (0.0011) [2023-12-27 07:13:20,984][20381] Updated weights for policy 1, policy_version 226724 (0.0011) [2023-12-27 07:13:21,044][20381] Updated weights for policy 1, policy_version 226734 (0.0011) [2023-12-27 07:13:21,333][20347] Updated weights for policy 0, policy_version 227022 (0.0010) [2023-12-27 07:13:21,398][20347] Updated weights for policy 0, policy_version 227032 (0.0010) [2023-12-27 07:13:21,452][20347] Updated weights for policy 0, policy_version 227042 (0.0007) [2023-12-27 07:13:21,773][20381] Updated weights for policy 1, policy_version 226744 (0.0008) [2023-12-27 07:13:21,834][20381] Updated weights for policy 1, policy_version 226754 (0.0007) [2023-12-27 07:13:21,897][20381] Updated weights for policy 1, policy_version 226764 (0.0007) [2023-12-27 07:13:22,173][20347] Updated weights for policy 0, policy_version 227052 (0.0009) [2023-12-27 07:13:22,232][20347] Updated weights for policy 0, policy_version 227062 (0.0009) [2023-12-27 07:13:22,292][20347] Updated weights for policy 0, policy_version 227072 (0.0011) [2023-12-27 07:13:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 116211712. Throughput: 0: 10381.6, 1: 10280.8. Samples: 116178448. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:22,363][19170] Avg episode reward: [(0, '728.450'), (1, '214.380')] [2023-12-27 07:13:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000226768_58064896.pth... [2023-12-27 07:13:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000227080_58146816.pth... [2023-12-27 07:13:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000225584_57761792.pth [2023-12-27 07:13:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000225864_57835520.pth [2023-12-27 07:13:22,547][20381] Updated weights for policy 1, policy_version 226774 (0.0008) [2023-12-27 07:13:22,601][20381] Updated weights for policy 1, policy_version 226784 (0.0007) [2023-12-27 07:13:22,657][20381] Updated weights for policy 1, policy_version 226794 (0.0008) [2023-12-27 07:13:22,972][20347] Updated weights for policy 0, policy_version 227082 (0.0010) [2023-12-27 07:13:23,040][20347] Updated weights for policy 0, policy_version 227092 (0.0006) [2023-12-27 07:13:23,105][20347] Updated weights for policy 0, policy_version 227102 (0.0008) [2023-12-27 07:13:23,157][20347] Updated weights for policy 0, policy_version 227112 (0.0007) [2023-12-27 07:13:23,381][20381] Updated weights for policy 1, policy_version 226804 (0.0007) [2023-12-27 07:13:23,433][20381] Updated weights for policy 1, policy_version 226814 (0.0008) [2023-12-27 07:13:23,488][20381] Updated weights for policy 1, policy_version 226824 (0.0007) [2023-12-27 07:13:23,754][20347] Updated weights for policy 0, policy_version 227122 (0.0010) [2023-12-27 07:13:23,813][20347] Updated weights for policy 0, policy_version 227132 (0.0007) [2023-12-27 07:13:23,870][20347] Updated weights for policy 0, policy_version 227142 (0.0008) [2023-12-27 07:13:24,104][20381] Updated weights for policy 1, policy_version 226834 (0.0006) [2023-12-27 07:13:24,153][20381] Updated weights for policy 1, policy_version 226844 (0.0007) [2023-12-27 07:13:24,199][20381] Updated weights for policy 1, policy_version 226854 (0.0007) [2023-12-27 07:13:24,259][20381] Updated weights for policy 1, policy_version 226864 (0.0008) [2023-12-27 07:13:24,568][20347] Updated weights for policy 0, policy_version 227152 (0.0008) [2023-12-27 07:13:24,619][20347] Updated weights for policy 0, policy_version 227162 (0.0008) [2023-12-27 07:13:24,677][20347] Updated weights for policy 0, policy_version 227172 (0.0007) [2023-12-27 07:13:25,001][20381] Updated weights for policy 1, policy_version 226874 (0.0008) [2023-12-27 07:13:25,054][20381] Updated weights for policy 1, policy_version 226884 (0.0008) [2023-12-27 07:13:25,112][20381] Updated weights for policy 1, policy_version 226894 (0.0008) [2023-12-27 07:13:25,371][20347] Updated weights for policy 0, policy_version 227182 (0.0007) [2023-12-27 07:13:25,422][20347] Updated weights for policy 0, policy_version 227192 (0.0007) [2023-12-27 07:13:25,468][20347] Updated weights for policy 0, policy_version 227202 (0.0007) [2023-12-27 07:13:25,744][20381] Updated weights for policy 1, policy_version 226904 (0.0007) [2023-12-27 07:13:25,796][20381] Updated weights for policy 1, policy_version 226914 (0.0008) [2023-12-27 07:13:25,855][20381] Updated weights for policy 1, policy_version 226924 (0.0008) [2023-12-27 07:13:26,140][20347] Updated weights for policy 0, policy_version 227212 (0.0008) [2023-12-27 07:13:26,208][20347] Updated weights for policy 0, policy_version 227222 (0.0007) [2023-12-27 07:13:26,269][20347] Updated weights for policy 0, policy_version 227232 (0.0010) [2023-12-27 07:13:26,517][20381] Updated weights for policy 1, policy_version 226934 (0.0008) [2023-12-27 07:13:26,563][20381] Updated weights for policy 1, policy_version 226944 (0.0010) [2023-12-27 07:13:26,609][20381] Updated weights for policy 1, policy_version 226954 (0.0007) [2023-12-27 07:13:26,895][20347] Updated weights for policy 0, policy_version 227242 (0.0010) [2023-12-27 07:13:26,954][20347] Updated weights for policy 0, policy_version 227252 (0.0010) [2023-12-27 07:13:27,014][20347] Updated weights for policy 0, policy_version 227262 (0.0010) [2023-12-27 07:13:27,074][20347] Updated weights for policy 0, policy_version 227272 (0.0010) [2023-12-27 07:13:27,319][20381] Updated weights for policy 1, policy_version 226964 (0.0007) [2023-12-27 07:13:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.7, 300 sec: 20716.0). Total num frames: 116310016. Throughput: 0: 10380.6, 1: 10252.5. Samples: 116301840. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:27,363][19170] Avg episode reward: [(0, '765.470'), (1, '206.550')] [2023-12-27 07:13:27,370][20381] Updated weights for policy 1, policy_version 226974 (0.0008) [2023-12-27 07:13:27,422][20381] Updated weights for policy 1, policy_version 226984 (0.0008) [2023-12-27 07:13:27,685][20347] Updated weights for policy 0, policy_version 227282 (0.0005) [2023-12-27 07:13:27,744][20347] Updated weights for policy 0, policy_version 227292 (0.0008) [2023-12-27 07:13:27,799][20347] Updated weights for policy 0, policy_version 227302 (0.0010) [2023-12-27 07:13:28,158][20381] Updated weights for policy 1, policy_version 226994 (0.0008) [2023-12-27 07:13:28,202][20381] Updated weights for policy 1, policy_version 227004 (0.0007) [2023-12-27 07:13:28,250][20381] Updated weights for policy 1, policy_version 227014 (0.0007) [2023-12-27 07:13:28,291][20381] Updated weights for policy 1, policy_version 227024 (0.0007) [2023-12-27 07:13:28,428][20347] Updated weights for policy 0, policy_version 227312 (0.0007) [2023-12-27 07:13:28,488][20347] Updated weights for policy 0, policy_version 227322 (0.0005) [2023-12-27 07:13:28,548][20347] Updated weights for policy 0, policy_version 227332 (0.0005) [2023-12-27 07:13:29,052][20381] Updated weights for policy 1, policy_version 227034 (0.0007) [2023-12-27 07:13:29,096][20381] Updated weights for policy 1, policy_version 227044 (0.0007) [2023-12-27 07:13:29,117][20347] Updated weights for policy 0, policy_version 227342 (0.0005) [2023-12-27 07:13:29,140][20381] Updated weights for policy 1, policy_version 227054 (0.0007) [2023-12-27 07:13:29,162][20347] Updated weights for policy 0, policy_version 227352 (0.0005) [2023-12-27 07:13:29,216][20347] Updated weights for policy 0, policy_version 227362 (0.0006) [2023-12-27 07:13:29,836][20347] Updated weights for policy 0, policy_version 227372 (0.0008) [2023-12-27 07:13:29,890][20347] Updated weights for policy 0, policy_version 227382 (0.0008) [2023-12-27 07:13:29,915][20381] Updated weights for policy 1, policy_version 227064 (0.0007) [2023-12-27 07:13:29,956][20347] Updated weights for policy 0, policy_version 227392 (0.0008) [2023-12-27 07:13:29,974][20381] Updated weights for policy 1, policy_version 227074 (0.0006) [2023-12-27 07:13:30,035][20381] Updated weights for policy 1, policy_version 227084 (0.0008) [2023-12-27 07:13:30,619][20347] Updated weights for policy 0, policy_version 227402 (0.0006) [2023-12-27 07:13:30,672][20347] Updated weights for policy 0, policy_version 227412 (0.0007) [2023-12-27 07:13:30,723][20347] Updated weights for policy 0, policy_version 227422 (0.0010) [2023-12-27 07:13:30,752][20381] Updated weights for policy 1, policy_version 227094 (0.0008) [2023-12-27 07:13:30,777][20347] Updated weights for policy 0, policy_version 227432 (0.0010) [2023-12-27 07:13:30,802][20381] Updated weights for policy 1, policy_version 227104 (0.0005) [2023-12-27 07:13:30,860][20381] Updated weights for policy 1, policy_version 227114 (0.0005) [2023-12-27 07:13:31,464][20347] Updated weights for policy 0, policy_version 227442 (0.0007) [2023-12-27 07:13:31,525][20347] Updated weights for policy 0, policy_version 227452 (0.0006) [2023-12-27 07:13:31,549][20381] Updated weights for policy 1, policy_version 227124 (0.0006) [2023-12-27 07:13:31,582][20347] Updated weights for policy 0, policy_version 227462 (0.0008) [2023-12-27 07:13:31,602][20381] Updated weights for policy 1, policy_version 227134 (0.0006) [2023-12-27 07:13:31,659][20381] Updated weights for policy 1, policy_version 227144 (0.0008) [2023-12-27 07:13:32,251][20347] Updated weights for policy 0, policy_version 227472 (0.0007) [2023-12-27 07:13:32,314][20347] Updated weights for policy 0, policy_version 227482 (0.0007) [2023-12-27 07:13:32,330][20381] Updated weights for policy 1, policy_version 227154 (0.0008) [2023-12-27 07:13:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 116408320. Throughput: 0: 10394.9, 1: 10202.4. Samples: 116425688. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:32,363][19170] Avg episode reward: [(0, '699.090'), (1, '254.630')] [2023-12-27 07:13:32,376][20347] Updated weights for policy 0, policy_version 227492 (0.0010) [2023-12-27 07:13:32,390][20381] Updated weights for policy 1, policy_version 227164 (0.0009) [2023-12-27 07:13:32,447][20381] Updated weights for policy 1, policy_version 227174 (0.0007) [2023-12-27 07:13:32,497][20381] Updated weights for policy 1, policy_version 227184 (0.0007) [2023-12-27 07:13:33,077][20347] Updated weights for policy 0, policy_version 227502 (0.0009) [2023-12-27 07:13:33,129][20347] Updated weights for policy 0, policy_version 227512 (0.0007) [2023-12-27 07:13:33,131][20381] Updated weights for policy 1, policy_version 227194 (0.0011) [2023-12-27 07:13:33,181][20347] Updated weights for policy 0, policy_version 227522 (0.0005) [2023-12-27 07:13:33,184][20381] Updated weights for policy 1, policy_version 227204 (0.0011) [2023-12-27 07:13:33,230][20381] Updated weights for policy 1, policy_version 227214 (0.0010) [2023-12-27 07:13:33,852][20347] Updated weights for policy 0, policy_version 227532 (0.0007) [2023-12-27 07:13:33,899][20381] Updated weights for policy 1, policy_version 227224 (0.0007) [2023-12-27 07:13:33,902][20347] Updated weights for policy 0, policy_version 227542 (0.0007) [2023-12-27 07:13:33,954][20347] Updated weights for policy 0, policy_version 227552 (0.0005) [2023-12-27 07:13:33,956][20381] Updated weights for policy 1, policy_version 227234 (0.0006) [2023-12-27 07:13:34,005][20381] Updated weights for policy 1, policy_version 227244 (0.0006) [2023-12-27 07:13:34,681][20347] Updated weights for policy 0, policy_version 227562 (0.0006) [2023-12-27 07:13:34,692][20381] Updated weights for policy 1, policy_version 227254 (0.0006) [2023-12-27 07:13:34,737][20347] Updated weights for policy 0, policy_version 227572 (0.0007) [2023-12-27 07:13:34,747][20381] Updated weights for policy 1, policy_version 227264 (0.0005) [2023-12-27 07:13:34,795][20347] Updated weights for policy 0, policy_version 227582 (0.0007) [2023-12-27 07:13:34,804][20381] Updated weights for policy 1, policy_version 227274 (0.0006) [2023-12-27 07:13:34,849][20347] Updated weights for policy 0, policy_version 227592 (0.0007) [2023-12-27 07:13:35,389][20381] Updated weights for policy 1, policy_version 227284 (0.0007) [2023-12-27 07:13:35,446][20381] Updated weights for policy 1, policy_version 227294 (0.0006) [2023-12-27 07:13:35,469][20347] Updated weights for policy 0, policy_version 227602 (0.0010) [2023-12-27 07:13:35,502][20381] Updated weights for policy 1, policy_version 227304 (0.0005) [2023-12-27 07:13:35,520][20347] Updated weights for policy 0, policy_version 227612 (0.0010) [2023-12-27 07:13:35,575][20347] Updated weights for policy 0, policy_version 227622 (0.0010) [2023-12-27 07:13:36,123][20381] Updated weights for policy 1, policy_version 227314 (0.0006) [2023-12-27 07:13:36,181][20381] Updated weights for policy 1, policy_version 227324 (0.0011) [2023-12-27 07:13:36,237][20381] Updated weights for policy 1, policy_version 227334 (0.0011) [2023-12-27 07:13:36,265][20347] Updated weights for policy 0, policy_version 227632 (0.0007) [2023-12-27 07:13:36,293][20381] Updated weights for policy 1, policy_version 227344 (0.0011) [2023-12-27 07:13:36,336][20347] Updated weights for policy 0, policy_version 227642 (0.0008) [2023-12-27 07:13:36,401][20347] Updated weights for policy 0, policy_version 227652 (0.0008) [2023-12-27 07:13:36,924][20381] Updated weights for policy 1, policy_version 227354 (0.0008) [2023-12-27 07:13:36,987][20381] Updated weights for policy 1, policy_version 227364 (0.0006) [2023-12-27 07:13:37,036][20347] Updated weights for policy 0, policy_version 227662 (0.0008) [2023-12-27 07:13:37,041][20381] Updated weights for policy 1, policy_version 227374 (0.0005) [2023-12-27 07:13:37,086][20347] Updated weights for policy 0, policy_version 227672 (0.0008) [2023-12-27 07:13:37,142][20347] Updated weights for policy 0, policy_version 227682 (0.0006) [2023-12-27 07:13:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 116523008. Throughput: 0: 10370.1, 1: 10228.6. Samples: 116487856. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:37,363][19170] Avg episode reward: [(0, '655.390'), (1, '239.310')] [2023-12-27 07:13:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000227376_58220544.pth... [2023-12-27 07:13:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000227688_58302464.pth... [2023-12-27 07:13:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000226192_57917440.pth [2023-12-27 07:13:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000226472_57991168.pth [2023-12-27 07:13:37,789][20347] Updated weights for policy 0, policy_version 227692 (0.0005) [2023-12-27 07:13:37,791][20381] Updated weights for policy 1, policy_version 227384 (0.0005) [2023-12-27 07:13:37,843][20347] Updated weights for policy 0, policy_version 227702 (0.0005) [2023-12-27 07:13:37,844][20381] Updated weights for policy 1, policy_version 227394 (0.0007) [2023-12-27 07:13:37,897][20381] Updated weights for policy 1, policy_version 227404 (0.0007) [2023-12-27 07:13:37,899][20347] Updated weights for policy 0, policy_version 227712 (0.0007) [2023-12-27 07:13:38,519][20381] Updated weights for policy 1, policy_version 227414 (0.0007) [2023-12-27 07:13:38,576][20381] Updated weights for policy 1, policy_version 227424 (0.0007) [2023-12-27 07:13:38,600][20347] Updated weights for policy 0, policy_version 227722 (0.0007) [2023-12-27 07:13:38,630][20381] Updated weights for policy 1, policy_version 227434 (0.0006) [2023-12-27 07:13:38,654][20347] Updated weights for policy 0, policy_version 227732 (0.0008) [2023-12-27 07:13:38,711][20347] Updated weights for policy 0, policy_version 227742 (0.0007) [2023-12-27 07:13:38,758][20347] Updated weights for policy 0, policy_version 227752 (0.0008) [2023-12-27 07:13:39,360][20381] Updated weights for policy 1, policy_version 227444 (0.0009) [2023-12-27 07:13:39,418][20347] Updated weights for policy 0, policy_version 227762 (0.0008) [2023-12-27 07:13:39,426][20381] Updated weights for policy 1, policy_version 227454 (0.0008) [2023-12-27 07:13:39,478][20347] Updated weights for policy 0, policy_version 227772 (0.0008) [2023-12-27 07:13:39,478][20381] Updated weights for policy 1, policy_version 227464 (0.0007) [2023-12-27 07:13:39,530][20347] Updated weights for policy 0, policy_version 227782 (0.0008) [2023-12-27 07:13:40,220][20381] Updated weights for policy 1, policy_version 227474 (0.0006) [2023-12-27 07:13:40,241][20347] Updated weights for policy 0, policy_version 227792 (0.0008) [2023-12-27 07:13:40,277][20381] Updated weights for policy 1, policy_version 227484 (0.0008) [2023-12-27 07:13:40,307][20347] Updated weights for policy 0, policy_version 227802 (0.0006) [2023-12-27 07:13:40,330][20381] Updated weights for policy 1, policy_version 227494 (0.0007) [2023-12-27 07:13:40,369][20347] Updated weights for policy 0, policy_version 227812 (0.0007) [2023-12-27 07:13:40,394][20381] Updated weights for policy 1, policy_version 227504 (0.0006) [2023-12-27 07:13:41,011][20347] Updated weights for policy 0, policy_version 227822 (0.0007) [2023-12-27 07:13:41,068][20381] Updated weights for policy 1, policy_version 227514 (0.0008) [2023-12-27 07:13:41,075][20347] Updated weights for policy 0, policy_version 227832 (0.0006) [2023-12-27 07:13:41,132][20381] Updated weights for policy 1, policy_version 227524 (0.0009) [2023-12-27 07:13:41,142][20347] Updated weights for policy 0, policy_version 227842 (0.0006) [2023-12-27 07:13:41,200][20381] Updated weights for policy 1, policy_version 227534 (0.0011) [2023-12-27 07:13:41,810][20347] Updated weights for policy 0, policy_version 227852 (0.0008) [2023-12-27 07:13:41,865][20347] Updated weights for policy 0, policy_version 227862 (0.0008) [2023-12-27 07:13:41,925][20347] Updated weights for policy 0, policy_version 227872 (0.0006) [2023-12-27 07:13:41,950][20381] Updated weights for policy 1, policy_version 227544 (0.0011) [2023-12-27 07:13:42,007][20381] Updated weights for policy 1, policy_version 227554 (0.0011) [2023-12-27 07:13:42,065][20381] Updated weights for policy 1, policy_version 227564 (0.0011) [2023-12-27 07:13:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 116621312. Throughput: 0: 10374.6, 1: 10193.5. Samples: 116611556. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:42,363][19170] Avg episode reward: [(0, '715.200'), (1, '293.750')] [2023-12-27 07:13:42,659][20347] Updated weights for policy 0, policy_version 227882 (0.0006) [2023-12-27 07:13:42,711][20347] Updated weights for policy 0, policy_version 227892 (0.0008) [2023-12-27 07:13:42,753][20381] Updated weights for policy 1, policy_version 227574 (0.0009) [2023-12-27 07:13:42,758][20347] Updated weights for policy 0, policy_version 227902 (0.0007) [2023-12-27 07:13:42,809][20381] Updated weights for policy 1, policy_version 227584 (0.0008) [2023-12-27 07:13:42,814][20347] Updated weights for policy 0, policy_version 227912 (0.0007) [2023-12-27 07:13:42,861][20381] Updated weights for policy 1, policy_version 227594 (0.0008) [2023-12-27 07:13:43,448][20381] Updated weights for policy 1, policy_version 227604 (0.0005) [2023-12-27 07:13:43,506][20381] Updated weights for policy 1, policy_version 227614 (0.0006) [2023-12-27 07:13:43,520][20347] Updated weights for policy 0, policy_version 227922 (0.0007) [2023-12-27 07:13:43,559][20381] Updated weights for policy 1, policy_version 227624 (0.0005) [2023-12-27 07:13:43,575][20347] Updated weights for policy 0, policy_version 227932 (0.0007) [2023-12-27 07:13:43,629][20347] Updated weights for policy 0, policy_version 227942 (0.0007) [2023-12-27 07:13:44,194][20381] Updated weights for policy 1, policy_version 227634 (0.0007) [2023-12-27 07:13:44,256][20381] Updated weights for policy 1, policy_version 227644 (0.0008) [2023-12-27 07:13:44,308][20347] Updated weights for policy 0, policy_version 227952 (0.0007) [2023-12-27 07:13:44,310][20381] Updated weights for policy 1, policy_version 227654 (0.0006) [2023-12-27 07:13:44,368][20381] Updated weights for policy 1, policy_version 227664 (0.0006) [2023-12-27 07:13:44,370][20347] Updated weights for policy 0, policy_version 227962 (0.0006) [2023-12-27 07:13:44,439][20347] Updated weights for policy 0, policy_version 227972 (0.0008) [2023-12-27 07:13:45,027][20381] Updated weights for policy 1, policy_version 227674 (0.0008) [2023-12-27 07:13:45,082][20381] Updated weights for policy 1, policy_version 227684 (0.0008) [2023-12-27 07:13:45,142][20381] Updated weights for policy 1, policy_version 227694 (0.0007) [2023-12-27 07:13:45,162][20347] Updated weights for policy 0, policy_version 227982 (0.0010) [2023-12-27 07:13:45,225][20347] Updated weights for policy 0, policy_version 227992 (0.0010) [2023-12-27 07:13:45,278][20347] Updated weights for policy 0, policy_version 228002 (0.0007) [2023-12-27 07:13:45,873][20381] Updated weights for policy 1, policy_version 227704 (0.0009) [2023-12-27 07:13:45,894][20347] Updated weights for policy 0, policy_version 228012 (0.0006) [2023-12-27 07:13:45,931][20381] Updated weights for policy 1, policy_version 227714 (0.0011) [2023-12-27 07:13:45,955][20347] Updated weights for policy 0, policy_version 228022 (0.0005) [2023-12-27 07:13:45,982][20381] Updated weights for policy 1, policy_version 227724 (0.0010) [2023-12-27 07:13:46,011][20347] Updated weights for policy 0, policy_version 228032 (0.0006) [2023-12-27 07:13:46,582][20347] Updated weights for policy 0, policy_version 228042 (0.0007) [2023-12-27 07:13:46,628][20381] Updated weights for policy 1, policy_version 227734 (0.0009) [2023-12-27 07:13:46,642][20347] Updated weights for policy 0, policy_version 228052 (0.0005) [2023-12-27 07:13:46,677][20381] Updated weights for policy 1, policy_version 227744 (0.0005) [2023-12-27 07:13:46,697][20347] Updated weights for policy 0, policy_version 228062 (0.0007) [2023-12-27 07:13:46,716][20291] KL-divergence is very high: 178.0542 [2023-12-27 07:13:46,727][20381] Updated weights for policy 1, policy_version 227754 (0.0007) [2023-12-27 07:13:46,744][20347] Updated weights for policy 0, policy_version 228072 (0.0007) [2023-12-27 07:13:47,346][20347] Updated weights for policy 0, policy_version 228082 (0.0010) [2023-12-27 07:13:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 116719616. Throughput: 0: 10399.8, 1: 10226.6. Samples: 116735804. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:47,363][19170] Avg episode reward: [(0, '660.420'), (1, '275.160')] [2023-12-27 07:13:47,376][20381] Updated weights for policy 1, policy_version 227764 (0.0005) [2023-12-27 07:13:47,398][20347] Updated weights for policy 0, policy_version 228092 (0.0010) [2023-12-27 07:13:47,436][20381] Updated weights for policy 1, policy_version 227774 (0.0006) [2023-12-27 07:13:47,452][20347] Updated weights for policy 0, policy_version 228102 (0.0006) [2023-12-27 07:13:47,490][20381] Updated weights for policy 1, policy_version 227784 (0.0008) [2023-12-27 07:13:48,105][20381] Updated weights for policy 1, policy_version 227794 (0.0007) [2023-12-27 07:13:48,126][20347] Updated weights for policy 0, policy_version 228112 (0.0006) [2023-12-27 07:13:48,155][20381] Updated weights for policy 1, policy_version 227804 (0.0005) [2023-12-27 07:13:48,179][20347] Updated weights for policy 0, policy_version 228122 (0.0006) [2023-12-27 07:13:48,207][20381] Updated weights for policy 1, policy_version 227814 (0.0006) [2023-12-27 07:13:48,229][20347] Updated weights for policy 0, policy_version 228132 (0.0006) [2023-12-27 07:13:48,267][20381] Updated weights for policy 1, policy_version 227824 (0.0005) [2023-12-27 07:13:48,835][20381] Updated weights for policy 1, policy_version 227834 (0.0005) [2023-12-27 07:13:48,888][20381] Updated weights for policy 1, policy_version 227844 (0.0006) [2023-12-27 07:13:48,939][20381] Updated weights for policy 1, policy_version 227854 (0.0006) [2023-12-27 07:13:48,943][20347] Updated weights for policy 0, policy_version 228142 (0.0009) [2023-12-27 07:13:48,991][20347] Updated weights for policy 0, policy_version 228152 (0.0009) [2023-12-27 07:13:49,042][20347] Updated weights for policy 0, policy_version 228162 (0.0009) [2023-12-27 07:13:49,611][20381] Updated weights for policy 1, policy_version 227864 (0.0010) [2023-12-27 07:13:49,668][20381] Updated weights for policy 1, policy_version 227874 (0.0008) [2023-12-27 07:13:49,722][20381] Updated weights for policy 1, policy_version 227884 (0.0008) [2023-12-27 07:13:49,724][20347] Updated weights for policy 0, policy_version 228172 (0.0008) [2023-12-27 07:13:49,787][20347] Updated weights for policy 0, policy_version 228182 (0.0010) [2023-12-27 07:13:49,844][20347] Updated weights for policy 0, policy_version 228192 (0.0009) [2023-12-27 07:13:50,363][20381] Updated weights for policy 1, policy_version 227894 (0.0007) [2023-12-27 07:13:50,415][20381] Updated weights for policy 1, policy_version 227904 (0.0005) [2023-12-27 07:13:50,464][20381] Updated weights for policy 1, policy_version 227914 (0.0005) [2023-12-27 07:13:50,512][20347] Updated weights for policy 0, policy_version 228202 (0.0006) [2023-12-27 07:13:50,567][20347] Updated weights for policy 0, policy_version 228212 (0.0010) [2023-12-27 07:13:50,630][20347] Updated weights for policy 0, policy_version 228222 (0.0007) [2023-12-27 07:13:50,685][20347] Updated weights for policy 0, policy_version 228232 (0.0007) [2023-12-27 07:13:51,145][20381] Updated weights for policy 1, policy_version 227924 (0.0006) [2023-12-27 07:13:51,212][20381] Updated weights for policy 1, policy_version 227934 (0.0006) [2023-12-27 07:13:51,273][20381] Updated weights for policy 1, policy_version 227944 (0.0007) [2023-12-27 07:13:51,365][20347] Updated weights for policy 0, policy_version 228242 (0.0008) [2023-12-27 07:13:51,417][20347] Updated weights for policy 0, policy_version 228252 (0.0008) [2023-12-27 07:13:51,477][20347] Updated weights for policy 0, policy_version 228262 (0.0008) [2023-12-27 07:13:51,910][20381] Updated weights for policy 1, policy_version 227954 (0.0006) [2023-12-27 07:13:51,962][20381] Updated weights for policy 1, policy_version 227964 (0.0008) [2023-12-27 07:13:52,016][20381] Updated weights for policy 1, policy_version 227974 (0.0008) [2023-12-27 07:13:52,070][20381] Updated weights for policy 1, policy_version 227984 (0.0007) [2023-12-27 07:13:52,151][20347] Updated weights for policy 0, policy_version 228272 (0.0009) [2023-12-27 07:13:52,206][20347] Updated weights for policy 0, policy_version 228282 (0.0010) [2023-12-27 07:13:52,262][20347] Updated weights for policy 0, policy_version 228292 (0.0010) [2023-12-27 07:13:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 116834304. Throughput: 0: 10384.6, 1: 10250.4. Samples: 116799616. Policy #0 lag: (min: 28.0, avg: 40.8, max: 60.0) [2023-12-27 07:13:52,363][19170] Avg episode reward: [(0, '722.940'), (1, '311.010')] [2023-12-27 07:13:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000227984_58376192.pth... [2023-12-27 07:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000228296_58458112.pth... [2023-12-27 07:13:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000226768_58064896.pth [2023-12-27 07:13:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000227080_58146816.pth [2023-12-27 07:13:52,743][20381] Updated weights for policy 1, policy_version 227994 (0.0008) [2023-12-27 07:13:52,797][20381] Updated weights for policy 1, policy_version 228004 (0.0008) [2023-12-27 07:13:52,848][20381] Updated weights for policy 1, policy_version 228014 (0.0007) [2023-12-27 07:13:52,977][20347] Updated weights for policy 0, policy_version 228302 (0.0009) [2023-12-27 07:13:53,021][20347] Updated weights for policy 0, policy_version 228312 (0.0008) [2023-12-27 07:13:53,067][20347] Updated weights for policy 0, policy_version 228322 (0.0007) [2023-12-27 07:13:53,389][20381] Updated weights for policy 1, policy_version 228024 (0.0005) [2023-12-27 07:13:53,439][20381] Updated weights for policy 1, policy_version 228034 (0.0005) [2023-12-27 07:13:53,487][20381] Updated weights for policy 1, policy_version 228044 (0.0005) [2023-12-27 07:13:53,778][20347] Updated weights for policy 0, policy_version 228332 (0.0009) [2023-12-27 07:13:53,823][20347] Updated weights for policy 0, policy_version 228342 (0.0008) [2023-12-27 07:13:53,871][20347] Updated weights for policy 0, policy_version 228352 (0.0007) [2023-12-27 07:13:54,118][20381] Updated weights for policy 1, policy_version 228054 (0.0007) [2023-12-27 07:13:54,174][20381] Updated weights for policy 1, policy_version 228064 (0.0008) [2023-12-27 07:13:54,226][20381] Updated weights for policy 1, policy_version 228074 (0.0008) [2023-12-27 07:13:54,588][20347] Updated weights for policy 0, policy_version 228362 (0.0007) [2023-12-27 07:13:54,645][20347] Updated weights for policy 0, policy_version 228372 (0.0008) [2023-12-27 07:13:54,705][20347] Updated weights for policy 0, policy_version 228382 (0.0009) [2023-12-27 07:13:54,774][20347] Updated weights for policy 0, policy_version 228392 (0.0008) [2023-12-27 07:13:54,882][20381] Updated weights for policy 1, policy_version 228084 (0.0008) [2023-12-27 07:13:54,940][20381] Updated weights for policy 1, policy_version 228094 (0.0007) [2023-12-27 07:13:55,010][20381] Updated weights for policy 1, policy_version 228104 (0.0008) [2023-12-27 07:13:55,455][20347] Updated weights for policy 0, policy_version 228402 (0.0010) [2023-12-27 07:13:55,509][20347] Updated weights for policy 0, policy_version 228412 (0.0008) [2023-12-27 07:13:55,559][20347] Updated weights for policy 0, policy_version 228422 (0.0005) [2023-12-27 07:13:55,722][20381] Updated weights for policy 1, policy_version 228114 (0.0008) [2023-12-27 07:13:55,774][20381] Updated weights for policy 1, policy_version 228124 (0.0007) [2023-12-27 07:13:55,834][20381] Updated weights for policy 1, policy_version 228134 (0.0008) [2023-12-27 07:13:55,884][20381] Updated weights for policy 1, policy_version 228144 (0.0008) [2023-12-27 07:13:56,201][20347] Updated weights for policy 0, policy_version 228432 (0.0009) [2023-12-27 07:13:56,248][20347] Updated weights for policy 0, policy_version 228442 (0.0010) [2023-12-27 07:13:56,308][20347] Updated weights for policy 0, policy_version 228452 (0.0010) [2023-12-27 07:13:56,578][20381] Updated weights for policy 1, policy_version 228154 (0.0009) [2023-12-27 07:13:56,641][20381] Updated weights for policy 1, policy_version 228164 (0.0005) [2023-12-27 07:13:56,695][20381] Updated weights for policy 1, policy_version 228174 (0.0005) [2023-12-27 07:13:56,959][20347] Updated weights for policy 0, policy_version 228462 (0.0009) [2023-12-27 07:13:57,003][20347] Updated weights for policy 0, policy_version 228472 (0.0007) [2023-12-27 07:13:57,051][20347] Updated weights for policy 0, policy_version 228482 (0.0007) [2023-12-27 07:13:57,309][20381] Updated weights for policy 1, policy_version 228184 (0.0007) [2023-12-27 07:13:57,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.4, 300 sec: 20771.6). Total num frames: 116932608. Throughput: 0: 10331.7, 1: 10297.6. Samples: 116924124. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:13:57,363][20381] Updated weights for policy 1, policy_version 228194 (0.0007) [2023-12-27 07:13:57,364][19170] Avg episode reward: [(0, '789.170'), (1, '380.550')] [2023-12-27 07:13:57,419][20381] Updated weights for policy 1, policy_version 228204 (0.0008) [2023-12-27 07:13:57,715][20347] Updated weights for policy 0, policy_version 228492 (0.0007) [2023-12-27 07:13:57,759][20347] Updated weights for policy 0, policy_version 228502 (0.0007) [2023-12-27 07:13:57,805][20347] Updated weights for policy 0, policy_version 228512 (0.0007) [2023-12-27 07:13:58,116][20381] Updated weights for policy 1, policy_version 228214 (0.0008) [2023-12-27 07:13:58,172][20381] Updated weights for policy 1, policy_version 228224 (0.0008) [2023-12-27 07:13:58,230][20381] Updated weights for policy 1, policy_version 228234 (0.0007) [2023-12-27 07:13:58,533][20347] Updated weights for policy 0, policy_version 228522 (0.0008) [2023-12-27 07:13:58,592][20347] Updated weights for policy 0, policy_version 228532 (0.0009) [2023-12-27 07:13:58,648][20347] Updated weights for policy 0, policy_version 228542 (0.0011) [2023-12-27 07:13:58,709][20347] Updated weights for policy 0, policy_version 228552 (0.0009) [2023-12-27 07:13:58,999][20381] Updated weights for policy 1, policy_version 228244 (0.0007) [2023-12-27 07:13:59,063][20381] Updated weights for policy 1, policy_version 228254 (0.0006) [2023-12-27 07:13:59,123][20381] Updated weights for policy 1, policy_version 228264 (0.0006) [2023-12-27 07:13:59,505][20347] Updated weights for policy 0, policy_version 228562 (0.0008) [2023-12-27 07:13:59,562][20347] Updated weights for policy 0, policy_version 228572 (0.0007) [2023-12-27 07:13:59,623][20347] Updated weights for policy 0, policy_version 228582 (0.0006) [2023-12-27 07:13:59,775][20381] Updated weights for policy 1, policy_version 228274 (0.0007) [2023-12-27 07:13:59,831][20381] Updated weights for policy 1, policy_version 228284 (0.0008) [2023-12-27 07:13:59,887][20381] Updated weights for policy 1, policy_version 228294 (0.0007) [2023-12-27 07:13:59,949][20381] Updated weights for policy 1, policy_version 228304 (0.0008) [2023-12-27 07:14:00,244][20347] Updated weights for policy 0, policy_version 228592 (0.0006) [2023-12-27 07:14:00,312][20347] Updated weights for policy 0, policy_version 228602 (0.0007) [2023-12-27 07:14:00,372][20347] Updated weights for policy 0, policy_version 228612 (0.0010) [2023-12-27 07:14:00,594][20381] Updated weights for policy 1, policy_version 228314 (0.0008) [2023-12-27 07:14:00,647][20381] Updated weights for policy 1, policy_version 228324 (0.0007) [2023-12-27 07:14:00,697][20381] Updated weights for policy 1, policy_version 228334 (0.0005) [2023-12-27 07:14:01,022][20347] Updated weights for policy 0, policy_version 228622 (0.0008) [2023-12-27 07:14:01,081][20347] Updated weights for policy 0, policy_version 228632 (0.0010) [2023-12-27 07:14:01,138][20347] Updated weights for policy 0, policy_version 228642 (0.0010) [2023-12-27 07:14:01,432][20381] Updated weights for policy 1, policy_version 228344 (0.0006) [2023-12-27 07:14:01,480][20381] Updated weights for policy 1, policy_version 228354 (0.0006) [2023-12-27 07:14:01,528][20381] Updated weights for policy 1, policy_version 228364 (0.0007) [2023-12-27 07:14:01,817][20347] Updated weights for policy 0, policy_version 228652 (0.0007) [2023-12-27 07:14:01,878][20347] Updated weights for policy 0, policy_version 228662 (0.0006) [2023-12-27 07:14:01,936][20347] Updated weights for policy 0, policy_version 228672 (0.0009) [2023-12-27 07:14:02,190][20381] Updated weights for policy 1, policy_version 228374 (0.0007) [2023-12-27 07:14:02,247][20381] Updated weights for policy 1, policy_version 228384 (0.0007) [2023-12-27 07:14:02,305][20381] Updated weights for policy 1, policy_version 228394 (0.0007) [2023-12-27 07:14:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 117039104. Throughput: 0: 10353.6, 1: 10290.7. Samples: 117045968. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:02,363][19170] Avg episode reward: [(0, '856.230'), (1, '472.670')] [2023-12-27 07:14:02,576][20347] Updated weights for policy 0, policy_version 228682 (0.0009) [2023-12-27 07:14:02,623][20347] Updated weights for policy 0, policy_version 228692 (0.0005) [2023-12-27 07:14:02,677][20347] Updated weights for policy 0, policy_version 228702 (0.0009) [2023-12-27 07:14:02,725][20347] Updated weights for policy 0, policy_version 228712 (0.0010) [2023-12-27 07:14:02,924][20381] Updated weights for policy 1, policy_version 228404 (0.0007) [2023-12-27 07:14:02,970][20381] Updated weights for policy 1, policy_version 228414 (0.0006) [2023-12-27 07:14:03,022][20381] Updated weights for policy 1, policy_version 228424 (0.0005) [2023-12-27 07:14:03,382][20347] Updated weights for policy 0, policy_version 228722 (0.0007) [2023-12-27 07:14:03,429][20347] Updated weights for policy 0, policy_version 228732 (0.0005) [2023-12-27 07:14:03,476][20347] Updated weights for policy 0, policy_version 228742 (0.0005) [2023-12-27 07:14:03,609][20381] Updated weights for policy 1, policy_version 228434 (0.0006) [2023-12-27 07:14:03,656][20381] Updated weights for policy 1, policy_version 228444 (0.0007) [2023-12-27 07:14:03,699][20381] Updated weights for policy 1, policy_version 228454 (0.0007) [2023-12-27 07:14:03,742][20381] Updated weights for policy 1, policy_version 228464 (0.0007) [2023-12-27 07:14:04,163][20347] Updated weights for policy 0, policy_version 228752 (0.0008) [2023-12-27 07:14:04,218][20347] Updated weights for policy 0, policy_version 228762 (0.0008) [2023-12-27 07:14:04,267][20347] Updated weights for policy 0, policy_version 228772 (0.0008) [2023-12-27 07:14:04,453][20381] Updated weights for policy 1, policy_version 228474 (0.0008) [2023-12-27 07:14:04,504][20381] Updated weights for policy 1, policy_version 228484 (0.0009) [2023-12-27 07:14:04,558][20381] Updated weights for policy 1, policy_version 228494 (0.0009) [2023-12-27 07:14:04,964][20347] Updated weights for policy 0, policy_version 228782 (0.0008) [2023-12-27 07:14:05,017][20347] Updated weights for policy 0, policy_version 228792 (0.0005) [2023-12-27 07:14:05,068][20347] Updated weights for policy 0, policy_version 228802 (0.0005) [2023-12-27 07:14:05,272][20381] Updated weights for policy 1, policy_version 228504 (0.0007) [2023-12-27 07:14:05,326][20381] Updated weights for policy 1, policy_version 228514 (0.0009) [2023-12-27 07:14:05,391][20381] Updated weights for policy 1, policy_version 228524 (0.0009) [2023-12-27 07:14:05,659][20347] Updated weights for policy 0, policy_version 228812 (0.0006) [2023-12-27 07:14:05,719][20347] Updated weights for policy 0, policy_version 228822 (0.0006) [2023-12-27 07:14:05,782][20347] Updated weights for policy 0, policy_version 228832 (0.0009) [2023-12-27 07:14:06,041][20381] Updated weights for policy 1, policy_version 228534 (0.0008) [2023-12-27 07:14:06,102][20381] Updated weights for policy 1, policy_version 228544 (0.0008) [2023-12-27 07:14:06,163][20381] Updated weights for policy 1, policy_version 228554 (0.0007) [2023-12-27 07:14:06,346][20347] Updated weights for policy 0, policy_version 228842 (0.0006) [2023-12-27 07:14:06,408][20347] Updated weights for policy 0, policy_version 228852 (0.0010) [2023-12-27 07:14:06,475][20347] Updated weights for policy 0, policy_version 228862 (0.0009) [2023-12-27 07:14:06,534][20347] Updated weights for policy 0, policy_version 228872 (0.0008) [2023-12-27 07:14:06,876][20381] Updated weights for policy 1, policy_version 228564 (0.0007) [2023-12-27 07:14:06,924][20381] Updated weights for policy 1, policy_version 228574 (0.0005) [2023-12-27 07:14:06,973][20381] Updated weights for policy 1, policy_version 228584 (0.0005) [2023-12-27 07:14:07,079][20291] KL-divergence is very high: 153.4674 [2023-12-27 07:14:07,132][20347] Updated weights for policy 0, policy_version 228882 (0.0009) [2023-12-27 07:14:07,183][20347] Updated weights for policy 0, policy_version 228892 (0.0008) [2023-12-27 07:14:07,246][20347] Updated weights for policy 0, policy_version 228902 (0.0008) [2023-12-27 07:14:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 117145600. Throughput: 0: 10348.4, 1: 10343.7. Samples: 117109592. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:07,363][19170] Avg episode reward: [(0, '729.680'), (1, '528.910')] [2023-12-27 07:14:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000228592_58531840.pth... [2023-12-27 07:14:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000228904_58613760.pth... [2023-12-27 07:14:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000227376_58220544.pth [2023-12-27 07:14:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000227688_58302464.pth [2023-12-27 07:14:07,669][20381] Updated weights for policy 1, policy_version 228594 (0.0007) [2023-12-27 07:14:07,731][20381] Updated weights for policy 1, policy_version 228604 (0.0008) [2023-12-27 07:14:07,785][20381] Updated weights for policy 1, policy_version 228614 (0.0009) [2023-12-27 07:14:07,846][20381] Updated weights for policy 1, policy_version 228624 (0.0008) [2023-12-27 07:14:07,856][20347] Updated weights for policy 0, policy_version 228912 (0.0006) [2023-12-27 07:14:07,907][20347] Updated weights for policy 0, policy_version 228922 (0.0010) [2023-12-27 07:14:07,952][20347] Updated weights for policy 0, policy_version 228932 (0.0010) [2023-12-27 07:14:08,546][20381] Updated weights for policy 1, policy_version 228634 (0.0007) [2023-12-27 07:14:08,604][20381] Updated weights for policy 1, policy_version 228644 (0.0007) [2023-12-27 07:14:08,620][20347] Updated weights for policy 0, policy_version 228942 (0.0009) [2023-12-27 07:14:08,659][20381] Updated weights for policy 1, policy_version 228654 (0.0006) [2023-12-27 07:14:08,679][20347] Updated weights for policy 0, policy_version 228952 (0.0010) [2023-12-27 07:14:08,743][20347] Updated weights for policy 0, policy_version 228962 (0.0010) [2023-12-27 07:14:09,375][20381] Updated weights for policy 1, policy_version 228664 (0.0009) [2023-12-27 07:14:09,419][20347] Updated weights for policy 0, policy_version 228972 (0.0010) [2023-12-27 07:14:09,440][20381] Updated weights for policy 1, policy_version 228674 (0.0007) [2023-12-27 07:14:09,482][20347] Updated weights for policy 0, policy_version 228982 (0.0008) [2023-12-27 07:14:09,508][20381] Updated weights for policy 1, policy_version 228684 (0.0006) [2023-12-27 07:14:09,539][20347] Updated weights for policy 0, policy_version 228992 (0.0010) [2023-12-27 07:14:10,202][20381] Updated weights for policy 1, policy_version 228694 (0.0006) [2023-12-27 07:14:10,250][20347] Updated weights for policy 0, policy_version 229002 (0.0010) [2023-12-27 07:14:10,260][20381] Updated weights for policy 1, policy_version 228704 (0.0007) [2023-12-27 07:14:10,310][20347] Updated weights for policy 0, policy_version 229012 (0.0007) [2023-12-27 07:14:10,320][20381] Updated weights for policy 1, policy_version 228714 (0.0006) [2023-12-27 07:14:10,371][20347] Updated weights for policy 0, policy_version 229022 (0.0007) [2023-12-27 07:14:10,420][20347] Updated weights for policy 0, policy_version 229032 (0.0007) [2023-12-27 07:14:11,028][20381] Updated weights for policy 1, policy_version 228724 (0.0007) [2023-12-27 07:14:11,094][20381] Updated weights for policy 1, policy_version 228734 (0.0008) [2023-12-27 07:14:11,117][20291] KL-divergence is very high: 115.0864 [2023-12-27 07:14:11,162][20381] Updated weights for policy 1, policy_version 228744 (0.0009) [2023-12-27 07:14:11,182][20347] Updated weights for policy 0, policy_version 229042 (0.0008) [2023-12-27 07:14:11,243][20347] Updated weights for policy 0, policy_version 229052 (0.0009) [2023-12-27 07:14:11,304][20347] Updated weights for policy 0, policy_version 229062 (0.0007) [2023-12-27 07:14:11,897][20381] Updated weights for policy 1, policy_version 228754 (0.0008) [2023-12-27 07:14:11,959][20381] Updated weights for policy 1, policy_version 228764 (0.0008) [2023-12-27 07:14:12,024][20381] Updated weights for policy 1, policy_version 228774 (0.0007) [2023-12-27 07:14:12,063][20347] Updated weights for policy 0, policy_version 229072 (0.0010) [2023-12-27 07:14:12,080][20381] Updated weights for policy 1, policy_version 228784 (0.0006) [2023-12-27 07:14:12,113][20347] Updated weights for policy 0, policy_version 229082 (0.0010) [2023-12-27 07:14:12,164][20347] Updated weights for policy 0, policy_version 229092 (0.0011) [2023-12-27 07:14:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 117243904. Throughput: 0: 10386.2, 1: 10298.1. Samples: 117232628. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:12,363][19170] Avg episode reward: [(0, '432.060'), (1, '536.140')] [2023-12-27 07:14:12,785][20381] Updated weights for policy 1, policy_version 228794 (0.0005) [2023-12-27 07:14:12,840][20381] Updated weights for policy 1, policy_version 228804 (0.0005) [2023-12-27 07:14:12,861][20347] Updated weights for policy 0, policy_version 229102 (0.0010) [2023-12-27 07:14:12,891][20381] Updated weights for policy 1, policy_version 228814 (0.0005) [2023-12-27 07:14:12,909][20347] Updated weights for policy 0, policy_version 229112 (0.0010) [2023-12-27 07:14:12,967][20347] Updated weights for policy 0, policy_version 229122 (0.0010) [2023-12-27 07:14:13,410][20381] Updated weights for policy 1, policy_version 228824 (0.0007) [2023-12-27 07:14:13,464][20381] Updated weights for policy 1, policy_version 228834 (0.0007) [2023-12-27 07:14:13,518][20381] Updated weights for policy 1, policy_version 228844 (0.0005) [2023-12-27 07:14:13,558][20347] Updated weights for policy 0, policy_version 229132 (0.0006) [2023-12-27 07:14:13,616][20347] Updated weights for policy 0, policy_version 229142 (0.0006) [2023-12-27 07:14:13,668][20347] Updated weights for policy 0, policy_version 229152 (0.0006) [2023-12-27 07:14:14,085][20381] Updated weights for policy 1, policy_version 228854 (0.0005) [2023-12-27 07:14:14,136][20381] Updated weights for policy 1, policy_version 228864 (0.0005) [2023-12-27 07:14:14,189][20381] Updated weights for policy 1, policy_version 228874 (0.0006) [2023-12-27 07:14:14,344][20347] Updated weights for policy 0, policy_version 229162 (0.0007) [2023-12-27 07:14:14,406][20347] Updated weights for policy 0, policy_version 229172 (0.0005) [2023-12-27 07:14:14,472][20347] Updated weights for policy 0, policy_version 229182 (0.0005) [2023-12-27 07:14:14,536][20347] Updated weights for policy 0, policy_version 229192 (0.0006) [2023-12-27 07:14:14,880][20381] Updated weights for policy 1, policy_version 228884 (0.0007) [2023-12-27 07:14:14,944][20381] Updated weights for policy 1, policy_version 228894 (0.0008) [2023-12-27 07:14:15,004][20381] Updated weights for policy 1, policy_version 228904 (0.0011) [2023-12-27 07:14:15,118][20347] Updated weights for policy 0, policy_version 229202 (0.0007) [2023-12-27 07:14:15,166][20347] Updated weights for policy 0, policy_version 229212 (0.0007) [2023-12-27 07:14:15,187][20291] KL-divergence is very high: 104.7940 [2023-12-27 07:14:15,217][20347] Updated weights for policy 0, policy_version 229222 (0.0007) [2023-12-27 07:14:15,686][20381] Updated weights for policy 1, policy_version 228914 (0.0011) [2023-12-27 07:14:15,741][20381] Updated weights for policy 1, policy_version 228924 (0.0010) [2023-12-27 07:14:15,797][20381] Updated weights for policy 1, policy_version 228934 (0.0010) [2023-12-27 07:14:15,853][20381] Updated weights for policy 1, policy_version 228944 (0.0010) [2023-12-27 07:14:15,903][20291] KL-divergence is very high: 124.0450 [2023-12-27 07:14:15,931][20347] Updated weights for policy 0, policy_version 229232 (0.0008) [2023-12-27 07:14:15,989][20347] Updated weights for policy 0, policy_version 229242 (0.0007) [2023-12-27 07:14:16,045][20347] Updated weights for policy 0, policy_version 229252 (0.0008) [2023-12-27 07:14:16,545][20381] Updated weights for policy 1, policy_version 228954 (0.0010) [2023-12-27 07:14:16,600][20381] Updated weights for policy 1, policy_version 228964 (0.0010) [2023-12-27 07:14:16,658][20381] Updated weights for policy 1, policy_version 228974 (0.0010) [2023-12-27 07:14:16,744][20347] Updated weights for policy 0, policy_version 229262 (0.0007) [2023-12-27 07:14:16,805][20347] Updated weights for policy 0, policy_version 229272 (0.0010) [2023-12-27 07:14:16,865][20347] Updated weights for policy 0, policy_version 229282 (0.0006) [2023-12-27 07:14:17,227][20381] Updated weights for policy 1, policy_version 228984 (0.0006) [2023-12-27 07:14:17,277][20381] Updated weights for policy 1, policy_version 228994 (0.0005) [2023-12-27 07:14:17,335][20381] Updated weights for policy 1, policy_version 229004 (0.0005) [2023-12-27 07:14:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 117350400. Throughput: 0: 10314.8, 1: 10410.1. Samples: 117358308. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:17,363][19170] Avg episode reward: [(0, '434.450'), (1, '548.780')] [2023-12-27 07:14:17,475][20347] Updated weights for policy 0, policy_version 229292 (0.0007) [2023-12-27 07:14:17,523][20347] Updated weights for policy 0, policy_version 229302 (0.0005) [2023-12-27 07:14:17,581][20347] Updated weights for policy 0, policy_version 229312 (0.0006) [2023-12-27 07:14:17,877][20381] Updated weights for policy 1, policy_version 229014 (0.0006) [2023-12-27 07:14:17,927][20381] Updated weights for policy 1, policy_version 229024 (0.0007) [2023-12-27 07:14:17,983][20381] Updated weights for policy 1, policy_version 229034 (0.0007) [2023-12-27 07:14:18,231][20347] Updated weights for policy 0, policy_version 229322 (0.0005) [2023-12-27 07:14:18,284][20347] Updated weights for policy 0, policy_version 229332 (0.0005) [2023-12-27 07:14:18,347][20347] Updated weights for policy 0, policy_version 229342 (0.0008) [2023-12-27 07:14:18,400][20347] Updated weights for policy 0, policy_version 229352 (0.0010) [2023-12-27 07:14:18,679][20381] Updated weights for policy 1, policy_version 229044 (0.0007) [2023-12-27 07:14:18,741][20381] Updated weights for policy 1, policy_version 229054 (0.0005) [2023-12-27 07:14:18,800][20381] Updated weights for policy 1, policy_version 229064 (0.0008) [2023-12-27 07:14:19,023][20347] Updated weights for policy 0, policy_version 229362 (0.0010) [2023-12-27 07:14:19,070][20347] Updated weights for policy 0, policy_version 229372 (0.0010) [2023-12-27 07:14:19,124][20347] Updated weights for policy 0, policy_version 229382 (0.0010) [2023-12-27 07:14:19,424][20381] Updated weights for policy 1, policy_version 229074 (0.0010) [2023-12-27 07:14:19,481][20381] Updated weights for policy 1, policy_version 229084 (0.0010) [2023-12-27 07:14:19,540][20381] Updated weights for policy 1, policy_version 229094 (0.0008) [2023-12-27 07:14:19,591][20381] Updated weights for policy 1, policy_version 229104 (0.0007) [2023-12-27 07:14:19,819][20347] Updated weights for policy 0, policy_version 229392 (0.0009) [2023-12-27 07:14:19,881][20347] Updated weights for policy 0, policy_version 229402 (0.0011) [2023-12-27 07:14:19,943][20347] Updated weights for policy 0, policy_version 229412 (0.0011) [2023-12-27 07:14:20,308][20381] Updated weights for policy 1, policy_version 229114 (0.0007) [2023-12-27 07:14:20,366][20381] Updated weights for policy 1, policy_version 229124 (0.0008) [2023-12-27 07:14:20,427][20381] Updated weights for policy 1, policy_version 229134 (0.0007) [2023-12-27 07:14:20,644][20347] Updated weights for policy 0, policy_version 229422 (0.0011) [2023-12-27 07:14:20,708][20347] Updated weights for policy 0, policy_version 229432 (0.0010) [2023-12-27 07:14:20,772][20347] Updated weights for policy 0, policy_version 229442 (0.0011) [2023-12-27 07:14:21,192][20381] Updated weights for policy 1, policy_version 229144 (0.0006) [2023-12-27 07:14:21,257][20381] Updated weights for policy 1, policy_version 229154 (0.0008) [2023-12-27 07:14:21,323][20381] Updated weights for policy 1, policy_version 229164 (0.0007) [2023-12-27 07:14:21,457][20347] Updated weights for policy 0, policy_version 229452 (0.0010) [2023-12-27 07:14:21,517][20347] Updated weights for policy 0, policy_version 229462 (0.0008) [2023-12-27 07:14:21,574][20347] Updated weights for policy 0, policy_version 229472 (0.0007) [2023-12-27 07:14:22,003][20381] Updated weights for policy 1, policy_version 229174 (0.0008) [2023-12-27 07:14:22,065][20381] Updated weights for policy 1, policy_version 229184 (0.0008) [2023-12-27 07:14:22,118][20381] Updated weights for policy 1, policy_version 229194 (0.0008) [2023-12-27 07:14:22,298][20347] Updated weights for policy 0, policy_version 229482 (0.0008) [2023-12-27 07:14:22,363][20347] Updated weights for policy 0, policy_version 229492 (0.0007) [2023-12-27 07:14:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 117448704. Throughput: 0: 10356.1, 1: 10409.9. Samples: 117422328. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:22,363][19170] Avg episode reward: [(0, '316.990'), (1, '706.610')] [2023-12-27 07:14:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000229200_58687488.pth... [2023-12-27 07:14:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000227984_58376192.pth [2023-12-27 07:14:22,415][20347] Updated weights for policy 0, policy_version 229502 (0.0009) [2023-12-27 07:14:22,468][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000229512_58769408.pth... [2023-12-27 07:14:22,471][20347] Updated weights for policy 0, policy_version 229512 (0.0008) [2023-12-27 07:14:22,473][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000228296_58458112.pth [2023-12-27 07:14:22,790][20381] Updated weights for policy 1, policy_version 229204 (0.0008) [2023-12-27 07:14:22,847][20381] Updated weights for policy 1, policy_version 229214 (0.0008) [2023-12-27 07:14:22,909][20381] Updated weights for policy 1, policy_version 229224 (0.0008) [2023-12-27 07:14:23,196][20347] Updated weights for policy 0, policy_version 229522 (0.0008) [2023-12-27 07:14:23,251][20347] Updated weights for policy 0, policy_version 229532 (0.0007) [2023-12-27 07:14:23,311][20347] Updated weights for policy 0, policy_version 229542 (0.0007) [2023-12-27 07:14:23,529][20381] Updated weights for policy 1, policy_version 229234 (0.0007) [2023-12-27 07:14:23,581][20381] Updated weights for policy 1, policy_version 229244 (0.0008) [2023-12-27 07:14:23,638][20381] Updated weights for policy 1, policy_version 229254 (0.0008) [2023-12-27 07:14:23,693][20381] Updated weights for policy 1, policy_version 229264 (0.0007) [2023-12-27 07:14:23,979][20347] Updated weights for policy 0, policy_version 229552 (0.0006) [2023-12-27 07:14:24,024][20347] Updated weights for policy 0, policy_version 229562 (0.0005) [2023-12-27 07:14:24,070][20347] Updated weights for policy 0, policy_version 229572 (0.0007) [2023-12-27 07:14:24,399][20381] Updated weights for policy 1, policy_version 229274 (0.0008) [2023-12-27 07:14:24,454][20381] Updated weights for policy 1, policy_version 229284 (0.0008) [2023-12-27 07:14:24,508][20381] Updated weights for policy 1, policy_version 229294 (0.0009) [2023-12-27 07:14:24,749][20347] Updated weights for policy 0, policy_version 229582 (0.0008) [2023-12-27 07:14:24,814][20347] Updated weights for policy 0, policy_version 229592 (0.0008) [2023-12-27 07:14:24,881][20347] Updated weights for policy 0, policy_version 229602 (0.0008) [2023-12-27 07:14:25,076][20381] Updated weights for policy 1, policy_version 229304 (0.0006) [2023-12-27 07:14:25,136][20381] Updated weights for policy 1, policy_version 229314 (0.0005) [2023-12-27 07:14:25,195][20381] Updated weights for policy 1, policy_version 229324 (0.0008) [2023-12-27 07:14:25,580][20347] Updated weights for policy 0, policy_version 229612 (0.0008) [2023-12-27 07:14:25,632][20347] Updated weights for policy 0, policy_version 229622 (0.0007) [2023-12-27 07:14:25,649][20291] KL-divergence is very high: 145.0139 [2023-12-27 07:14:25,655][20291] KL-divergence is very high: 142.8277 [2023-12-27 07:14:25,672][20291] KL-divergence is very high: 105.3782 [2023-12-27 07:14:25,694][20347] Updated weights for policy 0, policy_version 229633 (0.0009) [2023-12-27 07:14:25,700][20291] KL-divergence is very high: 130.7330 [2023-12-27 07:14:25,794][20381] Updated weights for policy 1, policy_version 229334 (0.0011) [2023-12-27 07:14:25,843][20381] Updated weights for policy 1, policy_version 229344 (0.0010) [2023-12-27 07:14:25,898][20381] Updated weights for policy 1, policy_version 229354 (0.0011) [2023-12-27 07:14:26,407][20347] Updated weights for policy 0, policy_version 229643 (0.0006) [2023-12-27 07:14:26,463][20347] Updated weights for policy 0, policy_version 229653 (0.0007) [2023-12-27 07:14:26,464][20381] Updated weights for policy 1, policy_version 229364 (0.0010) [2023-12-27 07:14:26,515][20347] Updated weights for policy 0, policy_version 229663 (0.0006) [2023-12-27 07:14:26,517][20381] Updated weights for policy 1, policy_version 229374 (0.0010) [2023-12-27 07:14:26,575][20381] Updated weights for policy 1, policy_version 229384 (0.0011) [2023-12-27 07:14:27,150][20381] Updated weights for policy 1, policy_version 229394 (0.0009) [2023-12-27 07:14:27,201][20381] Updated weights for policy 1, policy_version 229404 (0.0006) [2023-12-27 07:14:27,247][20381] Updated weights for policy 1, policy_version 229414 (0.0005) [2023-12-27 07:14:27,266][20347] Updated weights for policy 0, policy_version 229673 (0.0008) [2023-12-27 07:14:27,304][20381] Updated weights for policy 1, policy_version 229424 (0.0006) [2023-12-27 07:14:27,326][20347] Updated weights for policy 0, policy_version 229683 (0.0007) [2023-12-27 07:14:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 117555200. Throughput: 0: 10293.2, 1: 10445.6. Samples: 117544804. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:27,363][19170] Avg episode reward: [(0, '342.950'), (1, '902.710')] [2023-12-27 07:14:27,380][20347] Updated weights for policy 0, policy_version 229693 (0.0007) [2023-12-27 07:14:27,429][20347] Updated weights for policy 0, policy_version 229703 (0.0008) [2023-12-27 07:14:27,882][20381] Updated weights for policy 1, policy_version 229434 (0.0008) [2023-12-27 07:14:27,936][20381] Updated weights for policy 1, policy_version 229444 (0.0007) [2023-12-27 07:14:27,993][20381] Updated weights for policy 1, policy_version 229454 (0.0007) [2023-12-27 07:14:28,107][20347] Updated weights for policy 0, policy_version 229713 (0.0008) [2023-12-27 07:14:28,159][20347] Updated weights for policy 0, policy_version 229723 (0.0007) [2023-12-27 07:14:28,221][20347] Updated weights for policy 0, policy_version 229733 (0.0008) [2023-12-27 07:14:28,561][20381] Updated weights for policy 1, policy_version 229464 (0.0007) [2023-12-27 07:14:28,622][20381] Updated weights for policy 1, policy_version 229474 (0.0007) [2023-12-27 07:14:28,681][20381] Updated weights for policy 1, policy_version 229484 (0.0007) [2023-12-27 07:14:28,948][20347] Updated weights for policy 0, policy_version 229743 (0.0009) [2023-12-27 07:14:29,002][20347] Updated weights for policy 0, policy_version 229753 (0.0010) [2023-12-27 07:14:29,057][20347] Updated weights for policy 0, policy_version 229763 (0.0008) [2023-12-27 07:14:29,366][20381] Updated weights for policy 1, policy_version 229494 (0.0008) [2023-12-27 07:14:29,421][20381] Updated weights for policy 1, policy_version 229504 (0.0007) [2023-12-27 07:14:29,475][20381] Updated weights for policy 1, policy_version 229514 (0.0007) [2023-12-27 07:14:29,735][20347] Updated weights for policy 0, policy_version 229773 (0.0006) [2023-12-27 07:14:29,794][20347] Updated weights for policy 0, policy_version 229783 (0.0008) [2023-12-27 07:14:29,853][20347] Updated weights for policy 0, policy_version 229793 (0.0009) [2023-12-27 07:14:30,147][20381] Updated weights for policy 1, policy_version 229524 (0.0008) [2023-12-27 07:14:30,196][20381] Updated weights for policy 1, policy_version 229534 (0.0007) [2023-12-27 07:14:30,241][20381] Updated weights for policy 1, policy_version 229544 (0.0007) [2023-12-27 07:14:30,469][20347] Updated weights for policy 0, policy_version 229803 (0.0009) [2023-12-27 07:14:30,520][20347] Updated weights for policy 0, policy_version 229813 (0.0007) [2023-12-27 07:14:30,571][20347] Updated weights for policy 0, policy_version 229823 (0.0007) [2023-12-27 07:14:30,879][20381] Updated weights for policy 1, policy_version 229554 (0.0009) [2023-12-27 07:14:30,924][20381] Updated weights for policy 1, policy_version 229564 (0.0010) [2023-12-27 07:14:30,971][20381] Updated weights for policy 1, policy_version 229574 (0.0010) [2023-12-27 07:14:31,030][20381] Updated weights for policy 1, policy_version 229584 (0.0007) [2023-12-27 07:14:31,171][20347] Updated weights for policy 0, policy_version 229833 (0.0007) [2023-12-27 07:14:31,222][20347] Updated weights for policy 0, policy_version 229843 (0.0005) [2023-12-27 07:14:31,289][20347] Updated weights for policy 0, policy_version 229853 (0.0007) [2023-12-27 07:14:31,345][20347] Updated weights for policy 0, policy_version 229863 (0.0009) [2023-12-27 07:14:31,647][20381] Updated weights for policy 1, policy_version 229594 (0.0008) [2023-12-27 07:14:31,697][20381] Updated weights for policy 1, policy_version 229604 (0.0007) [2023-12-27 07:14:31,756][20381] Updated weights for policy 1, policy_version 229614 (0.0007) [2023-12-27 07:14:32,060][20347] Updated weights for policy 0, policy_version 229873 (0.0009) [2023-12-27 07:14:32,110][20347] Updated weights for policy 0, policy_version 229883 (0.0009) [2023-12-27 07:14:32,160][20347] Updated weights for policy 0, policy_version 229893 (0.0007) [2023-12-27 07:14:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.1). Total num frames: 117661696. Throughput: 0: 10262.6, 1: 10550.9. Samples: 117672416. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:32,363][19170] Avg episode reward: [(0, '333.680'), (1, '990.320')] [2023-12-27 07:14:32,405][20381] Updated weights for policy 1, policy_version 229624 (0.0007) [2023-12-27 07:14:32,458][20381] Updated weights for policy 1, policy_version 229634 (0.0008) [2023-12-27 07:14:32,517][20381] Updated weights for policy 1, policy_version 229644 (0.0008) [2023-12-27 07:14:32,894][20347] Updated weights for policy 0, policy_version 229903 (0.0006) [2023-12-27 07:14:32,947][20347] Updated weights for policy 0, policy_version 229913 (0.0005) [2023-12-27 07:14:33,005][20347] Updated weights for policy 0, policy_version 229923 (0.0005) [2023-12-27 07:14:33,128][20381] Updated weights for policy 1, policy_version 229654 (0.0009) [2023-12-27 07:14:33,172][20381] Updated weights for policy 1, policy_version 229664 (0.0010) [2023-12-27 07:14:33,217][20381] Updated weights for policy 1, policy_version 229674 (0.0010) [2023-12-27 07:14:33,582][20347] Updated weights for policy 0, policy_version 229933 (0.0007) [2023-12-27 07:14:33,633][20347] Updated weights for policy 0, policy_version 229943 (0.0007) [2023-12-27 07:14:33,679][20347] Updated weights for policy 0, policy_version 229953 (0.0008) [2023-12-27 07:14:33,852][20381] Updated weights for policy 1, policy_version 229684 (0.0008) [2023-12-27 07:14:33,902][20381] Updated weights for policy 1, policy_version 229694 (0.0005) [2023-12-27 07:14:33,952][20381] Updated weights for policy 1, policy_version 229704 (0.0005) [2023-12-27 07:14:34,324][20347] Updated weights for policy 0, policy_version 229964 (0.0007) [2023-12-27 07:14:34,381][20347] Updated weights for policy 0, policy_version 229974 (0.0011) [2023-12-27 07:14:34,440][20347] Updated weights for policy 0, policy_version 229984 (0.0010) [2023-12-27 07:14:34,572][20381] Updated weights for policy 1, policy_version 229714 (0.0006) [2023-12-27 07:14:34,619][20381] Updated weights for policy 1, policy_version 229724 (0.0008) [2023-12-27 07:14:34,665][20381] Updated weights for policy 1, policy_version 229734 (0.0010) [2023-12-27 07:14:34,717][20381] Updated weights for policy 1, policy_version 229744 (0.0010) [2023-12-27 07:14:35,113][20347] Updated weights for policy 0, policy_version 229994 (0.0006) [2023-12-27 07:14:35,168][20347] Updated weights for policy 0, policy_version 230004 (0.0010) [2023-12-27 07:14:35,223][20347] Updated weights for policy 0, policy_version 230014 (0.0010) [2023-12-27 07:14:35,282][20347] Updated weights for policy 0, policy_version 230024 (0.0010) [2023-12-27 07:14:35,425][20381] Updated weights for policy 1, policy_version 229754 (0.0008) [2023-12-27 07:14:35,478][20381] Updated weights for policy 1, policy_version 229764 (0.0007) [2023-12-27 07:14:35,530][20381] Updated weights for policy 1, policy_version 229774 (0.0007) [2023-12-27 07:14:35,914][20347] Updated weights for policy 0, policy_version 230034 (0.0009) [2023-12-27 07:14:35,960][20347] Updated weights for policy 0, policy_version 230044 (0.0009) [2023-12-27 07:14:36,007][20347] Updated weights for policy 0, policy_version 230054 (0.0010) [2023-12-27 07:14:36,163][20381] Updated weights for policy 1, policy_version 229784 (0.0010) [2023-12-27 07:14:36,219][20381] Updated weights for policy 1, policy_version 229794 (0.0011) [2023-12-27 07:14:36,276][20381] Updated weights for policy 1, policy_version 229804 (0.0011) [2023-12-27 07:14:36,720][20347] Updated weights for policy 0, policy_version 230064 (0.0010) [2023-12-27 07:14:36,777][20347] Updated weights for policy 0, policy_version 230074 (0.0010) [2023-12-27 07:14:36,832][20347] Updated weights for policy 0, policy_version 230084 (0.0010) [2023-12-27 07:14:36,978][20381] Updated weights for policy 1, policy_version 229814 (0.0011) [2023-12-27 07:14:37,035][20381] Updated weights for policy 1, policy_version 229824 (0.0011) [2023-12-27 07:14:37,091][20381] Updated weights for policy 1, policy_version 229834 (0.0010) [2023-12-27 07:14:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 117768192. Throughput: 0: 10282.2, 1: 10542.1. Samples: 117736708. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:37,363][19170] Avg episode reward: [(0, '289.430'), (1, '990.320')] [2023-12-27 07:14:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000229840_58851328.pth... [2023-12-27 07:14:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000230088_58916864.pth... [2023-12-27 07:14:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000228592_58531840.pth [2023-12-27 07:14:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000228904_58613760.pth [2023-12-27 07:14:37,511][20347] Updated weights for policy 0, policy_version 230094 (0.0010) [2023-12-27 07:14:37,566][20347] Updated weights for policy 0, policy_version 230104 (0.0010) [2023-12-27 07:14:37,625][20347] Updated weights for policy 0, policy_version 230114 (0.0010) [2023-12-27 07:14:37,779][20381] Updated weights for policy 1, policy_version 229844 (0.0009) [2023-12-27 07:14:37,843][20381] Updated weights for policy 1, policy_version 229854 (0.0006) [2023-12-27 07:14:37,901][20381] Updated weights for policy 1, policy_version 229864 (0.0011) [2023-12-27 07:14:38,294][20347] Updated weights for policy 0, policy_version 230124 (0.0010) [2023-12-27 07:14:38,353][20347] Updated weights for policy 0, policy_version 230134 (0.0009) [2023-12-27 07:14:38,398][20347] Updated weights for policy 0, policy_version 230144 (0.0007) [2023-12-27 07:14:38,579][20381] Updated weights for policy 1, policy_version 229874 (0.0011) [2023-12-27 07:14:38,647][20381] Updated weights for policy 1, policy_version 229884 (0.0009) [2023-12-27 07:14:38,696][20381] Updated weights for policy 1, policy_version 229894 (0.0010) [2023-12-27 07:14:38,755][20381] Updated weights for policy 1, policy_version 229904 (0.0010) [2023-12-27 07:14:39,056][20347] Updated weights for policy 0, policy_version 230154 (0.0007) [2023-12-27 07:14:39,100][20347] Updated weights for policy 0, policy_version 230164 (0.0007) [2023-12-27 07:14:39,149][20347] Updated weights for policy 0, policy_version 230174 (0.0007) [2023-12-27 07:14:39,201][20347] Updated weights for policy 0, policy_version 230184 (0.0007) [2023-12-27 07:14:39,424][20381] Updated weights for policy 1, policy_version 229914 (0.0008) [2023-12-27 07:14:39,484][20381] Updated weights for policy 1, policy_version 229924 (0.0008) [2023-12-27 07:14:39,540][20381] Updated weights for policy 1, policy_version 229934 (0.0008) [2023-12-27 07:14:39,928][20347] Updated weights for policy 0, policy_version 230194 (0.0011) [2023-12-27 07:14:39,991][20347] Updated weights for policy 0, policy_version 230204 (0.0011) [2023-12-27 07:14:40,047][20347] Updated weights for policy 0, policy_version 230214 (0.0008) [2023-12-27 07:14:40,203][20381] Updated weights for policy 1, policy_version 229944 (0.0006) [2023-12-27 07:14:40,262][20381] Updated weights for policy 1, policy_version 229954 (0.0006) [2023-12-27 07:14:40,325][20381] Updated weights for policy 1, policy_version 229964 (0.0005) [2023-12-27 07:14:40,730][20347] Updated weights for policy 0, policy_version 230224 (0.0007) [2023-12-27 07:14:40,782][20347] Updated weights for policy 0, policy_version 230234 (0.0008) [2023-12-27 07:14:40,827][20347] Updated weights for policy 0, policy_version 230244 (0.0009) [2023-12-27 07:14:40,914][20381] Updated weights for policy 1, policy_version 229974 (0.0005) [2023-12-27 07:14:40,969][20381] Updated weights for policy 1, policy_version 229984 (0.0006) [2023-12-27 07:14:41,019][20381] Updated weights for policy 1, policy_version 229994 (0.0005) [2023-12-27 07:14:41,605][20347] Updated weights for policy 0, policy_version 230254 (0.0007) [2023-12-27 07:14:41,671][20347] Updated weights for policy 0, policy_version 230264 (0.0008) [2023-12-27 07:14:41,689][20381] Updated weights for policy 1, policy_version 230004 (0.0007) [2023-12-27 07:14:41,731][20347] Updated weights for policy 0, policy_version 230274 (0.0006) [2023-12-27 07:14:41,757][20381] Updated weights for policy 1, policy_version 230014 (0.0008) [2023-12-27 07:14:41,818][20381] Updated weights for policy 1, policy_version 230024 (0.0009) [2023-12-27 07:14:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 117866496. Throughput: 0: 10285.6, 1: 10520.8. Samples: 117860404. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:42,363][19170] Avg episode reward: [(0, '266.580'), (1, '1000.000')] [2023-12-27 07:14:42,464][20347] Updated weights for policy 0, policy_version 230284 (0.0007) [2023-12-27 07:14:42,522][20347] Updated weights for policy 0, policy_version 230294 (0.0008) [2023-12-27 07:14:42,530][20381] Updated weights for policy 1, policy_version 230034 (0.0008) [2023-12-27 07:14:42,578][20347] Updated weights for policy 0, policy_version 230304 (0.0006) [2023-12-27 07:14:42,587][20381] Updated weights for policy 1, policy_version 230044 (0.0007) [2023-12-27 07:14:42,649][20381] Updated weights for policy 1, policy_version 230054 (0.0007) [2023-12-27 07:14:42,711][20381] Updated weights for policy 1, policy_version 230064 (0.0007) [2023-12-27 07:14:43,221][20347] Updated weights for policy 0, policy_version 230314 (0.0007) [2023-12-27 07:14:43,282][20347] Updated weights for policy 0, policy_version 230324 (0.0008) [2023-12-27 07:14:43,311][20291] KL-divergence is very high: 104.8878 [2023-12-27 07:14:43,323][20291] KL-divergence is very high: 143.2922 [2023-12-27 07:14:43,342][20347] Updated weights for policy 0, policy_version 230334 (0.0007) [2023-12-27 07:14:43,402][20347] Updated weights for policy 0, policy_version 230344 (0.0007) [2023-12-27 07:14:43,409][20381] Updated weights for policy 1, policy_version 230074 (0.0008) [2023-12-27 07:14:43,460][20381] Updated weights for policy 1, policy_version 230084 (0.0008) [2023-12-27 07:14:43,510][20381] Updated weights for policy 1, policy_version 230094 (0.0008) [2023-12-27 07:14:44,085][20347] Updated weights for policy 0, policy_version 230354 (0.0006) [2023-12-27 07:14:44,141][20347] Updated weights for policy 0, policy_version 230364 (0.0005) [2023-12-27 07:14:44,142][20381] Updated weights for policy 1, policy_version 230104 (0.0007) [2023-12-27 07:14:44,194][20381] Updated weights for policy 1, policy_version 230114 (0.0006) [2023-12-27 07:14:44,198][20347] Updated weights for policy 0, policy_version 230374 (0.0005) [2023-12-27 07:14:44,255][20381] Updated weights for policy 1, policy_version 230124 (0.0005) [2023-12-27 07:14:44,858][20347] Updated weights for policy 0, policy_version 230384 (0.0007) [2023-12-27 07:14:44,920][20347] Updated weights for policy 0, policy_version 230394 (0.0006) [2023-12-27 07:14:44,938][20381] Updated weights for policy 1, policy_version 230134 (0.0006) [2023-12-27 07:14:44,987][20347] Updated weights for policy 0, policy_version 230404 (0.0008) [2023-12-27 07:14:44,998][20381] Updated weights for policy 1, policy_version 230144 (0.0006) [2023-12-27 07:14:45,056][20381] Updated weights for policy 1, policy_version 230154 (0.0008) [2023-12-27 07:14:45,673][20347] Updated weights for policy 0, policy_version 230414 (0.0008) [2023-12-27 07:14:45,720][20381] Updated weights for policy 1, policy_version 230164 (0.0007) [2023-12-27 07:14:45,732][20347] Updated weights for policy 0, policy_version 230424 (0.0008) [2023-12-27 07:14:45,775][20381] Updated weights for policy 1, policy_version 230174 (0.0006) [2023-12-27 07:14:45,781][20347] Updated weights for policy 0, policy_version 230434 (0.0006) [2023-12-27 07:14:45,834][20381] Updated weights for policy 1, policy_version 230184 (0.0007) [2023-12-27 07:14:46,379][20347] Updated weights for policy 0, policy_version 230444 (0.0006) [2023-12-27 07:14:46,438][20347] Updated weights for policy 0, policy_version 230454 (0.0005) [2023-12-27 07:14:46,487][20347] Updated weights for policy 0, policy_version 230464 (0.0005) [2023-12-27 07:14:46,508][20381] Updated weights for policy 1, policy_version 230194 (0.0009) [2023-12-27 07:14:46,561][20381] Updated weights for policy 1, policy_version 230204 (0.0009) [2023-12-27 07:14:46,613][20381] Updated weights for policy 1, policy_version 230214 (0.0008) [2023-12-27 07:14:46,671][20381] Updated weights for policy 1, policy_version 230224 (0.0007) [2023-12-27 07:14:47,053][20347] Updated weights for policy 0, policy_version 230474 (0.0005) [2023-12-27 07:14:47,105][20347] Updated weights for policy 0, policy_version 230484 (0.0005) [2023-12-27 07:14:47,159][20347] Updated weights for policy 0, policy_version 230494 (0.0005) [2023-12-27 07:14:47,219][20347] Updated weights for policy 0, policy_version 230504 (0.0005) [2023-12-27 07:14:47,332][20381] Updated weights for policy 1, policy_version 230234 (0.0007) [2023-12-27 07:14:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 117972992. Throughput: 0: 10315.0, 1: 10526.5. Samples: 117983836. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:47,363][19170] Avg episode reward: [(0, '120.820'), (1, '1000.000')] [2023-12-27 07:14:47,392][20381] Updated weights for policy 1, policy_version 230244 (0.0008) [2023-12-27 07:14:47,456][20381] Updated weights for policy 1, policy_version 230254 (0.0006) [2023-12-27 07:14:47,768][20347] Updated weights for policy 0, policy_version 230514 (0.0007) [2023-12-27 07:14:47,822][20347] Updated weights for policy 0, policy_version 230524 (0.0006) [2023-12-27 07:14:47,870][20347] Updated weights for policy 0, policy_version 230534 (0.0008) [2023-12-27 07:14:48,099][20381] Updated weights for policy 1, policy_version 230264 (0.0007) [2023-12-27 07:14:48,163][20381] Updated weights for policy 1, policy_version 230274 (0.0008) [2023-12-27 07:14:48,221][20381] Updated weights for policy 1, policy_version 230284 (0.0007) [2023-12-27 07:14:48,565][20347] Updated weights for policy 0, policy_version 230544 (0.0007) [2023-12-27 07:14:48,623][20347] Updated weights for policy 0, policy_version 230554 (0.0008) [2023-12-27 07:14:48,686][20347] Updated weights for policy 0, policy_version 230564 (0.0008) [2023-12-27 07:14:48,905][20381] Updated weights for policy 1, policy_version 230294 (0.0009) [2023-12-27 07:14:48,960][20381] Updated weights for policy 1, policy_version 230304 (0.0010) [2023-12-27 07:14:49,015][20381] Updated weights for policy 1, policy_version 230314 (0.0010) [2023-12-27 07:14:49,409][20347] Updated weights for policy 0, policy_version 230574 (0.0008) [2023-12-27 07:14:49,468][20347] Updated weights for policy 0, policy_version 230584 (0.0008) [2023-12-27 07:14:49,520][20347] Updated weights for policy 0, policy_version 230594 (0.0008) [2023-12-27 07:14:49,710][20381] Updated weights for policy 1, policy_version 230324 (0.0011) [2023-12-27 07:14:49,766][20381] Updated weights for policy 1, policy_version 230334 (0.0011) [2023-12-27 07:14:49,819][20381] Updated weights for policy 1, policy_version 230344 (0.0011) [2023-12-27 07:14:50,236][20347] Updated weights for policy 0, policy_version 230604 (0.0006) [2023-12-27 07:14:50,291][20347] Updated weights for policy 0, policy_version 230614 (0.0007) [2023-12-27 07:14:50,350][20347] Updated weights for policy 0, policy_version 230624 (0.0008) [2023-12-27 07:14:50,533][20381] Updated weights for policy 1, policy_version 230354 (0.0009) [2023-12-27 07:14:50,591][20381] Updated weights for policy 1, policy_version 230364 (0.0007) [2023-12-27 07:14:50,648][20381] Updated weights for policy 1, policy_version 230374 (0.0005) [2023-12-27 07:14:50,706][20381] Updated weights for policy 1, policy_version 230384 (0.0008) [2023-12-27 07:14:51,056][20347] Updated weights for policy 0, policy_version 230634 (0.0007) [2023-12-27 07:14:51,118][20347] Updated weights for policy 0, policy_version 230644 (0.0008) [2023-12-27 07:14:51,179][20347] Updated weights for policy 0, policy_version 230654 (0.0008) [2023-12-27 07:14:51,234][20347] Updated weights for policy 0, policy_version 230664 (0.0007) [2023-12-27 07:14:51,326][20381] Updated weights for policy 1, policy_version 230394 (0.0008) [2023-12-27 07:14:51,392][20381] Updated weights for policy 1, policy_version 230404 (0.0008) [2023-12-27 07:14:51,447][20381] Updated weights for policy 1, policy_version 230414 (0.0007) [2023-12-27 07:14:51,916][20347] Updated weights for policy 0, policy_version 230674 (0.0005) [2023-12-27 07:14:51,977][20347] Updated weights for policy 0, policy_version 230684 (0.0005) [2023-12-27 07:14:52,035][20347] Updated weights for policy 0, policy_version 230694 (0.0005) [2023-12-27 07:14:52,173][20381] Updated weights for policy 1, policy_version 230424 (0.0009) [2023-12-27 07:14:52,234][20381] Updated weights for policy 1, policy_version 230434 (0.0008) [2023-12-27 07:14:52,294][20381] Updated weights for policy 1, policy_version 230444 (0.0008) [2023-12-27 07:14:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 118079488. Throughput: 0: 10312.8, 1: 10485.2. Samples: 118045504. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:52,363][19170] Avg episode reward: [(0, '305.190'), (1, '1000.000')] [2023-12-27 07:14:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000230448_59006976.pth... [2023-12-27 07:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000230696_59072512.pth... [2023-12-27 07:14:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000229200_58687488.pth [2023-12-27 07:14:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000229512_58769408.pth [2023-12-27 07:14:52,644][20347] Updated weights for policy 0, policy_version 230704 (0.0008) [2023-12-27 07:14:52,700][20347] Updated weights for policy 0, policy_version 230714 (0.0008) [2023-12-27 07:14:52,758][20347] Updated weights for policy 0, policy_version 230724 (0.0007) [2023-12-27 07:14:52,926][20381] Updated weights for policy 1, policy_version 230454 (0.0008) [2023-12-27 07:14:52,976][20381] Updated weights for policy 1, policy_version 230464 (0.0007) [2023-12-27 07:14:53,033][20381] Updated weights for policy 1, policy_version 230474 (0.0006) [2023-12-27 07:14:53,450][20347] Updated weights for policy 0, policy_version 230734 (0.0007) [2023-12-27 07:14:53,501][20347] Updated weights for policy 0, policy_version 230744 (0.0007) [2023-12-27 07:14:53,553][20347] Updated weights for policy 0, policy_version 230754 (0.0008) [2023-12-27 07:14:53,721][20381] Updated weights for policy 1, policy_version 230484 (0.0006) [2023-12-27 07:14:53,770][20381] Updated weights for policy 1, policy_version 230494 (0.0006) [2023-12-27 07:14:53,823][20381] Updated weights for policy 1, policy_version 230504 (0.0005) [2023-12-27 07:14:54,250][20347] Updated weights for policy 0, policy_version 230764 (0.0008) [2023-12-27 07:14:54,308][20347] Updated weights for policy 0, policy_version 230774 (0.0008) [2023-12-27 07:14:54,359][20347] Updated weights for policy 0, policy_version 230784 (0.0007) [2023-12-27 07:14:54,480][20381] Updated weights for policy 1, policy_version 230514 (0.0005) [2023-12-27 07:14:54,525][20381] Updated weights for policy 1, policy_version 230524 (0.0005) [2023-12-27 07:14:54,572][20381] Updated weights for policy 1, policy_version 230534 (0.0005) [2023-12-27 07:14:54,621][20381] Updated weights for policy 1, policy_version 230544 (0.0005) [2023-12-27 07:14:55,084][20347] Updated weights for policy 0, policy_version 230794 (0.0007) [2023-12-27 07:14:55,139][20347] Updated weights for policy 0, policy_version 230804 (0.0008) [2023-12-27 07:14:55,191][20347] Updated weights for policy 0, policy_version 230814 (0.0007) [2023-12-27 07:14:55,251][20347] Updated weights for policy 0, policy_version 230824 (0.0008) [2023-12-27 07:14:55,270][20381] Updated weights for policy 1, policy_version 230554 (0.0006) [2023-12-27 07:14:55,319][20381] Updated weights for policy 1, policy_version 230564 (0.0008) [2023-12-27 07:14:55,375][20381] Updated weights for policy 1, policy_version 230574 (0.0008) [2023-12-27 07:14:55,952][20347] Updated weights for policy 0, policy_version 230834 (0.0008) [2023-12-27 07:14:56,004][20347] Updated weights for policy 0, policy_version 230844 (0.0008) [2023-12-27 07:14:56,057][20381] Updated weights for policy 1, policy_version 230584 (0.0007) [2023-12-27 07:14:56,060][20347] Updated weights for policy 0, policy_version 230854 (0.0006) [2023-12-27 07:14:56,116][20381] Updated weights for policy 1, policy_version 230594 (0.0008) [2023-12-27 07:14:56,165][20381] Updated weights for policy 1, policy_version 230604 (0.0008) [2023-12-27 07:14:56,766][20347] Updated weights for policy 0, policy_version 230864 (0.0007) [2023-12-27 07:14:56,823][20347] Updated weights for policy 0, policy_version 230874 (0.0006) [2023-12-27 07:14:56,824][20381] Updated weights for policy 1, policy_version 230614 (0.0007) [2023-12-27 07:14:56,870][20381] Updated weights for policy 1, policy_version 230624 (0.0006) [2023-12-27 07:14:56,872][20347] Updated weights for policy 0, policy_version 230884 (0.0005) [2023-12-27 07:14:56,920][20381] Updated weights for policy 1, policy_version 230634 (0.0006) [2023-12-27 07:14:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 118177792. Throughput: 0: 10232.7, 1: 10552.4. Samples: 118167960. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:14:57,363][19170] Avg episode reward: [(0, '656.640'), (1, '1000.000')] [2023-12-27 07:14:57,555][20347] Updated weights for policy 0, policy_version 230894 (0.0005) [2023-12-27 07:14:57,615][20347] Updated weights for policy 0, policy_version 230904 (0.0006) [2023-12-27 07:14:57,631][20381] Updated weights for policy 1, policy_version 230644 (0.0007) [2023-12-27 07:14:57,674][20347] Updated weights for policy 0, policy_version 230914 (0.0007) [2023-12-27 07:14:57,695][20381] Updated weights for policy 1, policy_version 230654 (0.0007) [2023-12-27 07:14:57,749][20381] Updated weights for policy 1, policy_version 230664 (0.0008) [2023-12-27 07:14:58,316][20347] Updated weights for policy 0, policy_version 230924 (0.0007) [2023-12-27 07:14:58,372][20381] Updated weights for policy 1, policy_version 230674 (0.0008) [2023-12-27 07:14:58,389][20347] Updated weights for policy 0, policy_version 230934 (0.0007) [2023-12-27 07:14:58,435][20381] Updated weights for policy 1, policy_version 230684 (0.0008) [2023-12-27 07:14:58,456][20347] Updated weights for policy 0, policy_version 230944 (0.0007) [2023-12-27 07:14:58,499][20381] Updated weights for policy 1, policy_version 230694 (0.0007) [2023-12-27 07:14:58,557][20381] Updated weights for policy 1, policy_version 230704 (0.0008) [2023-12-27 07:14:59,158][20347] Updated weights for policy 0, policy_version 230954 (0.0008) [2023-12-27 07:14:59,226][20347] Updated weights for policy 0, policy_version 230964 (0.0006) [2023-12-27 07:14:59,239][20381] Updated weights for policy 1, policy_version 230714 (0.0007) [2023-12-27 07:14:59,290][20347] Updated weights for policy 0, policy_version 230974 (0.0007) [2023-12-27 07:14:59,293][20381] Updated weights for policy 1, policy_version 230724 (0.0006) [2023-12-27 07:14:59,354][20381] Updated weights for policy 1, policy_version 230734 (0.0007) [2023-12-27 07:14:59,359][20347] Updated weights for policy 0, policy_version 230984 (0.0007) [2023-12-27 07:14:59,941][20347] Updated weights for policy 0, policy_version 230994 (0.0006) [2023-12-27 07:15:00,003][20347] Updated weights for policy 0, policy_version 231004 (0.0007) [2023-12-27 07:15:00,063][20347] Updated weights for policy 0, policy_version 231014 (0.0006) [2023-12-27 07:15:00,098][20381] Updated weights for policy 1, policy_version 230744 (0.0008) [2023-12-27 07:15:00,154][20381] Updated weights for policy 1, policy_version 230754 (0.0008) [2023-12-27 07:15:00,215][20381] Updated weights for policy 1, policy_version 230764 (0.0008) [2023-12-27 07:15:00,631][20347] Updated weights for policy 0, policy_version 231024 (0.0006) [2023-12-27 07:15:00,683][20347] Updated weights for policy 0, policy_version 231034 (0.0007) [2023-12-27 07:15:00,733][20347] Updated weights for policy 0, policy_version 231044 (0.0007) [2023-12-27 07:15:00,911][20381] Updated weights for policy 1, policy_version 230774 (0.0006) [2023-12-27 07:15:00,962][20381] Updated weights for policy 1, policy_version 230784 (0.0006) [2023-12-27 07:15:01,017][20381] Updated weights for policy 1, policy_version 230794 (0.0007) [2023-12-27 07:15:01,476][20347] Updated weights for policy 0, policy_version 231054 (0.0007) [2023-12-27 07:15:01,531][20347] Updated weights for policy 0, policy_version 231064 (0.0008) [2023-12-27 07:15:01,589][20347] Updated weights for policy 0, policy_version 231074 (0.0007) [2023-12-27 07:15:01,679][20381] Updated weights for policy 1, policy_version 230804 (0.0008) [2023-12-27 07:15:01,736][20381] Updated weights for policy 1, policy_version 230814 (0.0006) [2023-12-27 07:15:01,788][20381] Updated weights for policy 1, policy_version 230824 (0.0006) [2023-12-27 07:15:02,257][20347] Updated weights for policy 0, policy_version 231084 (0.0007) [2023-12-27 07:15:02,317][20347] Updated weights for policy 0, policy_version 231094 (0.0007) [2023-12-27 07:15:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 118276096. Throughput: 0: 10256.4, 1: 10498.7. Samples: 118292292. Policy #0 lag: (min: 31.0, avg: 32.7, max: 63.0) [2023-12-27 07:15:02,363][19170] Avg episode reward: [(0, '871.430'), (1, '1000.000')] [2023-12-27 07:15:02,371][20347] Updated weights for policy 0, policy_version 231104 (0.0008) [2023-12-27 07:15:02,472][20381] Updated weights for policy 1, policy_version 230834 (0.0008) [2023-12-27 07:15:02,524][20381] Updated weights for policy 1, policy_version 230844 (0.0005) [2023-12-27 07:15:02,573][20381] Updated weights for policy 1, policy_version 230854 (0.0008) [2023-12-27 07:15:02,620][20381] Updated weights for policy 1, policy_version 230864 (0.0008) [2023-12-27 07:15:03,062][20347] Updated weights for policy 0, policy_version 231114 (0.0006) [2023-12-27 07:15:03,115][20347] Updated weights for policy 0, policy_version 231124 (0.0007) [2023-12-27 07:15:03,181][20347] Updated weights for policy 0, policy_version 231134 (0.0006) [2023-12-27 07:15:03,231][20347] Updated weights for policy 0, policy_version 231144 (0.0005) [2023-12-27 07:15:03,319][20381] Updated weights for policy 1, policy_version 230874 (0.0010) [2023-12-27 07:15:03,367][20381] Updated weights for policy 1, policy_version 230884 (0.0010) [2023-12-27 07:15:03,415][20381] Updated weights for policy 1, policy_version 230894 (0.0008) [2023-12-27 07:15:03,899][20347] Updated weights for policy 0, policy_version 231154 (0.0005) [2023-12-27 07:15:03,949][20347] Updated weights for policy 0, policy_version 231164 (0.0007) [2023-12-27 07:15:04,004][20347] Updated weights for policy 0, policy_version 231174 (0.0007) [2023-12-27 07:15:04,021][20381] Updated weights for policy 1, policy_version 230904 (0.0007) [2023-12-27 07:15:04,071][20381] Updated weights for policy 1, policy_version 230914 (0.0005) [2023-12-27 07:15:04,127][20381] Updated weights for policy 1, policy_version 230924 (0.0008) [2023-12-27 07:15:04,695][20347] Updated weights for policy 0, policy_version 231184 (0.0007) [2023-12-27 07:15:04,753][20347] Updated weights for policy 0, policy_version 231194 (0.0008) [2023-12-27 07:15:04,805][20381] Updated weights for policy 1, policy_version 230934 (0.0008) [2023-12-27 07:15:04,807][20347] Updated weights for policy 0, policy_version 231204 (0.0007) [2023-12-27 07:15:04,853][20381] Updated weights for policy 1, policy_version 230944 (0.0007) [2023-12-27 07:15:04,897][20381] Updated weights for policy 1, policy_version 230954 (0.0007) [2023-12-27 07:15:05,418][20347] Updated weights for policy 0, policy_version 231214 (0.0008) [2023-12-27 07:15:05,477][20347] Updated weights for policy 0, policy_version 231224 (0.0010) [2023-12-27 07:15:05,533][20347] Updated weights for policy 0, policy_version 231234 (0.0010) [2023-12-27 07:15:05,580][20381] Updated weights for policy 1, policy_version 230964 (0.0009) [2023-12-27 07:15:05,633][20381] Updated weights for policy 1, policy_version 230974 (0.0006) [2023-12-27 07:15:05,696][20381] Updated weights for policy 1, policy_version 230984 (0.0008) [2023-12-27 07:15:06,173][20347] Updated weights for policy 0, policy_version 231244 (0.0010) [2023-12-27 07:15:06,232][20347] Updated weights for policy 0, policy_version 231254 (0.0009) [2023-12-27 07:15:06,296][20347] Updated weights for policy 0, policy_version 231264 (0.0011) [2023-12-27 07:15:06,344][20381] Updated weights for policy 1, policy_version 230994 (0.0011) [2023-12-27 07:15:06,401][20381] Updated weights for policy 1, policy_version 231004 (0.0011) [2023-12-27 07:15:06,464][20381] Updated weights for policy 1, policy_version 231014 (0.0011) [2023-12-27 07:15:06,526][20381] Updated weights for policy 1, policy_version 231024 (0.0011) [2023-12-27 07:15:06,991][20347] Updated weights for policy 0, policy_version 231274 (0.0011) [2023-12-27 07:15:07,043][20347] Updated weights for policy 0, policy_version 231284 (0.0010) [2023-12-27 07:15:07,095][20347] Updated weights for policy 0, policy_version 231294 (0.0010) [2023-12-27 07:15:07,154][20347] Updated weights for policy 0, policy_version 231304 (0.0010) [2023-12-27 07:15:07,191][20381] Updated weights for policy 1, policy_version 231034 (0.0008) [2023-12-27 07:15:07,242][20381] Updated weights for policy 1, policy_version 231044 (0.0008) [2023-12-27 07:15:07,323][20381] Updated weights for policy 1, policy_version 231054 (0.0009) [2023-12-27 07:15:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 118390784. Throughput: 0: 10224.8, 1: 10486.5. Samples: 118354336. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:07,363][19170] Avg episode reward: [(0, '778.670'), (1, '1000.000')] [2023-12-27 07:15:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000231056_59162624.pth... [2023-12-27 07:15:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000231304_59228160.pth... [2023-12-27 07:15:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000229840_58851328.pth [2023-12-27 07:15:07,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000230088_58916864.pth [2023-12-27 07:15:07,763][20347] Updated weights for policy 0, policy_version 231314 (0.0010) [2023-12-27 07:15:07,821][20347] Updated weights for policy 0, policy_version 231324 (0.0010) [2023-12-27 07:15:07,871][20347] Updated weights for policy 0, policy_version 231334 (0.0010) [2023-12-27 07:15:07,991][20381] Updated weights for policy 1, policy_version 231064 (0.0010) [2023-12-27 07:15:08,050][20381] Updated weights for policy 1, policy_version 231074 (0.0010) [2023-12-27 07:15:08,104][20381] Updated weights for policy 1, policy_version 231084 (0.0011) [2023-12-27 07:15:08,539][20347] Updated weights for policy 0, policy_version 231344 (0.0009) [2023-12-27 07:15:08,593][20347] Updated weights for policy 0, policy_version 231354 (0.0008) [2023-12-27 07:15:08,657][20347] Updated weights for policy 0, policy_version 231364 (0.0007) [2023-12-27 07:15:08,769][20381] Updated weights for policy 1, policy_version 231094 (0.0011) [2023-12-27 07:15:08,829][20381] Updated weights for policy 1, policy_version 231104 (0.0011) [2023-12-27 07:15:08,878][20381] Updated weights for policy 1, policy_version 231114 (0.0011) [2023-12-27 07:15:09,323][20347] Updated weights for policy 0, policy_version 231374 (0.0007) [2023-12-27 07:15:09,386][20347] Updated weights for policy 0, policy_version 231384 (0.0008) [2023-12-27 07:15:09,451][20347] Updated weights for policy 0, policy_version 231394 (0.0008) [2023-12-27 07:15:09,645][20381] Updated weights for policy 1, policy_version 231124 (0.0010) [2023-12-27 07:15:09,712][20381] Updated weights for policy 1, policy_version 231134 (0.0011) [2023-12-27 07:15:09,777][20381] Updated weights for policy 1, policy_version 231144 (0.0011) [2023-12-27 07:15:10,073][20347] Updated weights for policy 0, policy_version 231404 (0.0008) [2023-12-27 07:15:10,126][20347] Updated weights for policy 0, policy_version 231414 (0.0008) [2023-12-27 07:15:10,184][20347] Updated weights for policy 0, policy_version 231424 (0.0008) [2023-12-27 07:15:10,429][20381] Updated weights for policy 1, policy_version 231154 (0.0008) [2023-12-27 07:15:10,499][20381] Updated weights for policy 1, policy_version 231164 (0.0008) [2023-12-27 07:15:10,557][20381] Updated weights for policy 1, policy_version 231174 (0.0010) [2023-12-27 07:15:10,620][20381] Updated weights for policy 1, policy_version 231184 (0.0010) [2023-12-27 07:15:10,914][20347] Updated weights for policy 0, policy_version 231434 (0.0008) [2023-12-27 07:15:10,973][20347] Updated weights for policy 0, policy_version 231444 (0.0007) [2023-12-27 07:15:11,031][20347] Updated weights for policy 0, policy_version 231454 (0.0008) [2023-12-27 07:15:11,093][20347] Updated weights for policy 0, policy_version 231464 (0.0008) [2023-12-27 07:15:11,330][20381] Updated weights for policy 1, policy_version 231194 (0.0008) [2023-12-27 07:15:11,399][20381] Updated weights for policy 1, policy_version 231204 (0.0009) [2023-12-27 07:15:11,465][20381] Updated weights for policy 1, policy_version 231214 (0.0008) [2023-12-27 07:15:11,806][20347] Updated weights for policy 0, policy_version 231474 (0.0007) [2023-12-27 07:15:11,858][20347] Updated weights for policy 0, policy_version 231484 (0.0007) [2023-12-27 07:15:11,914][20347] Updated weights for policy 0, policy_version 231494 (0.0009) [2023-12-27 07:15:12,123][20381] Updated weights for policy 1, policy_version 231224 (0.0011) [2023-12-27 07:15:12,181][20381] Updated weights for policy 1, policy_version 231234 (0.0010) [2023-12-27 07:15:12,248][20381] Updated weights for policy 1, policy_version 231244 (0.0011) [2023-12-27 07:15:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 118489088. Throughput: 0: 10292.5, 1: 10434.5. Samples: 118477516. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:12,363][19170] Avg episode reward: [(0, '669.990'), (1, '1000.000')] [2023-12-27 07:15:12,672][20347] Updated weights for policy 0, policy_version 231504 (0.0007) [2023-12-27 07:15:12,741][20347] Updated weights for policy 0, policy_version 231514 (0.0007) [2023-12-27 07:15:12,804][20347] Updated weights for policy 0, policy_version 231524 (0.0006) [2023-12-27 07:15:12,950][20381] Updated weights for policy 1, policy_version 231254 (0.0009) [2023-12-27 07:15:13,017][20381] Updated weights for policy 1, policy_version 231264 (0.0010) [2023-12-27 07:15:13,079][20381] Updated weights for policy 1, policy_version 231274 (0.0008) [2023-12-27 07:15:13,494][20347] Updated weights for policy 0, policy_version 231534 (0.0006) [2023-12-27 07:15:13,551][20347] Updated weights for policy 0, policy_version 231544 (0.0008) [2023-12-27 07:15:13,609][20347] Updated weights for policy 0, policy_version 231554 (0.0005) [2023-12-27 07:15:13,816][20381] Updated weights for policy 1, policy_version 231284 (0.0007) [2023-12-27 07:15:13,874][20381] Updated weights for policy 1, policy_version 231294 (0.0007) [2023-12-27 07:15:13,932][20381] Updated weights for policy 1, policy_version 231304 (0.0008) [2023-12-27 07:15:14,298][20347] Updated weights for policy 0, policy_version 231564 (0.0006) [2023-12-27 07:15:14,359][20347] Updated weights for policy 0, policy_version 231574 (0.0008) [2023-12-27 07:15:14,420][20347] Updated weights for policy 0, policy_version 231584 (0.0008) [2023-12-27 07:15:14,578][20381] Updated weights for policy 1, policy_version 231314 (0.0008) [2023-12-27 07:15:14,630][20381] Updated weights for policy 1, policy_version 231324 (0.0007) [2023-12-27 07:15:14,686][20381] Updated weights for policy 1, policy_version 231334 (0.0008) [2023-12-27 07:15:14,739][20381] Updated weights for policy 1, policy_version 231344 (0.0010) [2023-12-27 07:15:15,093][20347] Updated weights for policy 0, policy_version 231594 (0.0008) [2023-12-27 07:15:15,151][20347] Updated weights for policy 0, policy_version 231604 (0.0007) [2023-12-27 07:15:15,210][20347] Updated weights for policy 0, policy_version 231614 (0.0008) [2023-12-27 07:15:15,262][20347] Updated weights for policy 0, policy_version 231624 (0.0008) [2023-12-27 07:15:15,532][20381] Updated weights for policy 1, policy_version 231354 (0.0008) [2023-12-27 07:15:15,591][20381] Updated weights for policy 1, policy_version 231364 (0.0008) [2023-12-27 07:15:15,653][20381] Updated weights for policy 1, policy_version 231374 (0.0006) [2023-12-27 07:15:15,979][20347] Updated weights for policy 0, policy_version 231634 (0.0010) [2023-12-27 07:15:16,027][20347] Updated weights for policy 0, policy_version 231644 (0.0010) [2023-12-27 07:15:16,085][20347] Updated weights for policy 0, policy_version 231654 (0.0010) [2023-12-27 07:15:16,359][20381] Updated weights for policy 1, policy_version 231384 (0.0005) [2023-12-27 07:15:16,413][20381] Updated weights for policy 1, policy_version 231394 (0.0006) [2023-12-27 07:15:16,465][20381] Updated weights for policy 1, policy_version 231404 (0.0007) [2023-12-27 07:15:16,776][20347] Updated weights for policy 0, policy_version 231664 (0.0008) [2023-12-27 07:15:16,831][20347] Updated weights for policy 0, policy_version 231674 (0.0008) [2023-12-27 07:15:16,890][20347] Updated weights for policy 0, policy_version 231684 (0.0008) [2023-12-27 07:15:17,122][20381] Updated weights for policy 1, policy_version 231414 (0.0007) [2023-12-27 07:15:17,181][20381] Updated weights for policy 1, policy_version 231424 (0.0008) [2023-12-27 07:15:17,242][20381] Updated weights for policy 1, policy_version 231434 (0.0008) [2023-12-27 07:15:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 118587392. Throughput: 0: 10258.5, 1: 10262.9. Samples: 118595876. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:17,363][19170] Avg episode reward: [(0, '744.720'), (1, '1000.000')] [2023-12-27 07:15:17,562][20347] Updated weights for policy 0, policy_version 231694 (0.0006) [2023-12-27 07:15:17,629][20347] Updated weights for policy 0, policy_version 231704 (0.0007) [2023-12-27 07:15:17,683][20347] Updated weights for policy 0, policy_version 231714 (0.0008) [2023-12-27 07:15:17,943][20381] Updated weights for policy 1, policy_version 231444 (0.0008) [2023-12-27 07:15:18,003][20381] Updated weights for policy 1, policy_version 231454 (0.0008) [2023-12-27 07:15:18,055][20381] Updated weights for policy 1, policy_version 231464 (0.0007) [2023-12-27 07:15:18,267][20347] Updated weights for policy 0, policy_version 231724 (0.0006) [2023-12-27 07:15:18,332][20347] Updated weights for policy 0, policy_version 231734 (0.0005) [2023-12-27 07:15:18,395][20347] Updated weights for policy 0, policy_version 231744 (0.0007) [2023-12-27 07:15:18,804][20381] Updated weights for policy 1, policy_version 231474 (0.0007) [2023-12-27 07:15:18,871][20381] Updated weights for policy 1, policy_version 231484 (0.0008) [2023-12-27 07:15:18,937][20381] Updated weights for policy 1, policy_version 231494 (0.0009) [2023-12-27 07:15:19,002][20381] Updated weights for policy 1, policy_version 231504 (0.0008) [2023-12-27 07:15:19,040][20347] Updated weights for policy 0, policy_version 231754 (0.0008) [2023-12-27 07:15:19,102][20347] Updated weights for policy 0, policy_version 231764 (0.0008) [2023-12-27 07:15:19,163][20347] Updated weights for policy 0, policy_version 231774 (0.0008) [2023-12-27 07:15:19,226][20347] Updated weights for policy 0, policy_version 231784 (0.0008) [2023-12-27 07:15:19,727][20381] Updated weights for policy 1, policy_version 231514 (0.0008) [2023-12-27 07:15:19,793][20381] Updated weights for policy 1, policy_version 231524 (0.0008) [2023-12-27 07:15:19,859][20381] Updated weights for policy 1, policy_version 231534 (0.0007) [2023-12-27 07:15:19,936][20347] Updated weights for policy 0, policy_version 231794 (0.0007) [2023-12-27 07:15:19,999][20347] Updated weights for policy 0, policy_version 231804 (0.0007) [2023-12-27 07:15:20,057][20347] Updated weights for policy 0, policy_version 231814 (0.0008) [2023-12-27 07:15:20,563][20381] Updated weights for policy 1, policy_version 231544 (0.0009) [2023-12-27 07:15:20,628][20381] Updated weights for policy 1, policy_version 231554 (0.0009) [2023-12-27 07:15:20,688][20381] Updated weights for policy 1, policy_version 231564 (0.0008) [2023-12-27 07:15:20,749][20347] Updated weights for policy 0, policy_version 231824 (0.0007) [2023-12-27 07:15:20,810][20347] Updated weights for policy 0, policy_version 231834 (0.0006) [2023-12-27 07:15:20,871][20347] Updated weights for policy 0, policy_version 231844 (0.0005) [2023-12-27 07:15:21,404][20381] Updated weights for policy 1, policy_version 231574 (0.0007) [2023-12-27 07:15:21,468][20381] Updated weights for policy 1, policy_version 231584 (0.0006) [2023-12-27 07:15:21,529][20381] Updated weights for policy 1, policy_version 231594 (0.0006) [2023-12-27 07:15:21,550][20347] Updated weights for policy 0, policy_version 231854 (0.0006) [2023-12-27 07:15:21,609][20347] Updated weights for policy 0, policy_version 231864 (0.0007) [2023-12-27 07:15:21,674][20347] Updated weights for policy 0, policy_version 231874 (0.0008) [2023-12-27 07:15:22,126][20381] Updated weights for policy 1, policy_version 231604 (0.0006) [2023-12-27 07:15:22,182][20381] Updated weights for policy 1, policy_version 231614 (0.0008) [2023-12-27 07:15:22,243][20381] Updated weights for policy 1, policy_version 231624 (0.0008) [2023-12-27 07:15:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 118685696. Throughput: 0: 10251.4, 1: 10175.0. Samples: 118655900. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:22,363][19170] Avg episode reward: [(0, '620.510'), (1, '1000.000')] [2023-12-27 07:15:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000231632_59310080.pth... [2023-12-27 07:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000231880_59375616.pth... [2023-12-27 07:15:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000230448_59006976.pth [2023-12-27 07:15:22,402][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000230696_59072512.pth [2023-12-27 07:15:22,447][20347] Updated weights for policy 0, policy_version 231884 (0.0008) [2023-12-27 07:15:22,505][20347] Updated weights for policy 0, policy_version 231894 (0.0008) [2023-12-27 07:15:22,561][20347] Updated weights for policy 0, policy_version 231904 (0.0008) [2023-12-27 07:15:22,908][20381] Updated weights for policy 1, policy_version 231634 (0.0008) [2023-12-27 07:15:22,960][20381] Updated weights for policy 1, policy_version 231644 (0.0007) [2023-12-27 07:15:23,015][20381] Updated weights for policy 1, policy_version 231654 (0.0007) [2023-12-27 07:15:23,060][20381] Updated weights for policy 1, policy_version 231664 (0.0008) [2023-12-27 07:15:23,268][20347] Updated weights for policy 0, policy_version 231914 (0.0008) [2023-12-27 07:15:23,324][20347] Updated weights for policy 0, policy_version 231924 (0.0010) [2023-12-27 07:15:23,386][20347] Updated weights for policy 0, policy_version 231934 (0.0010) [2023-12-27 07:15:23,437][20347] Updated weights for policy 0, policy_version 231944 (0.0010) [2023-12-27 07:15:23,790][20381] Updated weights for policy 1, policy_version 231674 (0.0007) [2023-12-27 07:15:23,848][20381] Updated weights for policy 1, policy_version 231684 (0.0008) [2023-12-27 07:15:23,900][20381] Updated weights for policy 1, policy_version 231694 (0.0008) [2023-12-27 07:15:24,108][20347] Updated weights for policy 0, policy_version 231954 (0.0007) [2023-12-27 07:15:24,161][20347] Updated weights for policy 0, policy_version 231964 (0.0006) [2023-12-27 07:15:24,209][20347] Updated weights for policy 0, policy_version 231974 (0.0007) [2023-12-27 07:15:24,596][20381] Updated weights for policy 1, policy_version 231704 (0.0007) [2023-12-27 07:15:24,644][20381] Updated weights for policy 1, policy_version 231714 (0.0007) [2023-12-27 07:15:24,698][20381] Updated weights for policy 1, policy_version 231724 (0.0005) [2023-12-27 07:15:24,891][20347] Updated weights for policy 0, policy_version 231984 (0.0007) [2023-12-27 07:15:24,945][20347] Updated weights for policy 0, policy_version 231994 (0.0008) [2023-12-27 07:15:25,006][20347] Updated weights for policy 0, policy_version 232004 (0.0007) [2023-12-27 07:15:25,279][20381] Updated weights for policy 1, policy_version 231734 (0.0005) [2023-12-27 07:15:25,331][20381] Updated weights for policy 1, policy_version 231744 (0.0005) [2023-12-27 07:15:25,395][20381] Updated weights for policy 1, policy_version 231754 (0.0005) [2023-12-27 07:15:25,686][20347] Updated weights for policy 0, policy_version 232014 (0.0009) [2023-12-27 07:15:25,734][20347] Updated weights for policy 0, policy_version 232024 (0.0010) [2023-12-27 07:15:25,783][20347] Updated weights for policy 0, policy_version 232034 (0.0008) [2023-12-27 07:15:26,014][20381] Updated weights for policy 1, policy_version 231764 (0.0006) [2023-12-27 07:15:26,065][20381] Updated weights for policy 1, policy_version 231774 (0.0007) [2023-12-27 07:15:26,121][20381] Updated weights for policy 1, policy_version 231784 (0.0007) [2023-12-27 07:15:26,456][20347] Updated weights for policy 0, policy_version 232044 (0.0005) [2023-12-27 07:15:26,520][20347] Updated weights for policy 0, policy_version 232054 (0.0005) [2023-12-27 07:15:26,584][20347] Updated weights for policy 0, policy_version 232064 (0.0005) [2023-12-27 07:15:26,855][20381] Updated weights for policy 1, policy_version 231794 (0.0007) [2023-12-27 07:15:26,899][20381] Updated weights for policy 1, policy_version 231804 (0.0008) [2023-12-27 07:15:26,945][20381] Updated weights for policy 1, policy_version 231814 (0.0008) [2023-12-27 07:15:26,996][20381] Updated weights for policy 1, policy_version 231824 (0.0008) [2023-12-27 07:15:27,120][20347] Updated weights for policy 0, policy_version 232074 (0.0006) [2023-12-27 07:15:27,179][20347] Updated weights for policy 0, policy_version 232084 (0.0007) [2023-12-27 07:15:27,234][20347] Updated weights for policy 0, policy_version 232094 (0.0007) [2023-12-27 07:15:27,288][20347] Updated weights for policy 0, policy_version 232104 (0.0008) [2023-12-27 07:15:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 118792192. Throughput: 0: 10217.0, 1: 10160.4. Samples: 118777388. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:27,363][19170] Avg episode reward: [(0, '508.280'), (1, '1000.000')] [2023-12-27 07:15:27,710][20381] Updated weights for policy 1, policy_version 231834 (0.0005) [2023-12-27 07:15:27,767][20381] Updated weights for policy 1, policy_version 231844 (0.0006) [2023-12-27 07:15:27,820][20381] Updated weights for policy 1, policy_version 231854 (0.0007) [2023-12-27 07:15:27,910][20347] Updated weights for policy 0, policy_version 232114 (0.0008) [2023-12-27 07:15:27,964][20347] Updated weights for policy 0, policy_version 232124 (0.0007) [2023-12-27 07:15:28,012][20347] Updated weights for policy 0, policy_version 232134 (0.0007) [2023-12-27 07:15:28,468][20381] Updated weights for policy 1, policy_version 231864 (0.0008) [2023-12-27 07:15:28,526][20381] Updated weights for policy 1, policy_version 231874 (0.0008) [2023-12-27 07:15:28,581][20381] Updated weights for policy 1, policy_version 231884 (0.0008) [2023-12-27 07:15:28,731][20347] Updated weights for policy 0, policy_version 232144 (0.0006) [2023-12-27 07:15:28,798][20347] Updated weights for policy 0, policy_version 232154 (0.0006) [2023-12-27 07:15:28,850][20347] Updated weights for policy 0, policy_version 232164 (0.0005) [2023-12-27 07:15:29,351][20381] Updated weights for policy 1, policy_version 231894 (0.0008) [2023-12-27 07:15:29,395][20347] Updated weights for policy 0, policy_version 232174 (0.0007) [2023-12-27 07:15:29,405][20381] Updated weights for policy 1, policy_version 231904 (0.0008) [2023-12-27 07:15:29,454][20381] Updated weights for policy 1, policy_version 231914 (0.0008) [2023-12-27 07:15:29,454][20347] Updated weights for policy 0, policy_version 232184 (0.0008) [2023-12-27 07:15:29,509][20347] Updated weights for policy 0, policy_version 232194 (0.0006) [2023-12-27 07:15:30,173][20347] Updated weights for policy 0, policy_version 232204 (0.0008) [2023-12-27 07:15:30,187][20381] Updated weights for policy 1, policy_version 231924 (0.0007) [2023-12-27 07:15:30,222][20347] Updated weights for policy 0, policy_version 232214 (0.0006) [2023-12-27 07:15:30,239][20381] Updated weights for policy 1, policy_version 231934 (0.0006) [2023-12-27 07:15:30,275][20347] Updated weights for policy 0, policy_version 232224 (0.0005) [2023-12-27 07:15:30,296][20381] Updated weights for policy 1, policy_version 231944 (0.0006) [2023-12-27 07:15:30,909][20347] Updated weights for policy 0, policy_version 232234 (0.0006) [2023-12-27 07:15:30,966][20347] Updated weights for policy 0, policy_version 232244 (0.0008) [2023-12-27 07:15:30,990][20381] Updated weights for policy 1, policy_version 231954 (0.0006) [2023-12-27 07:15:31,011][20347] Updated weights for policy 0, policy_version 232254 (0.0006) [2023-12-27 07:15:31,040][20381] Updated weights for policy 1, policy_version 231964 (0.0006) [2023-12-27 07:15:31,071][20347] Updated weights for policy 0, policy_version 232264 (0.0008) [2023-12-27 07:15:31,104][20381] Updated weights for policy 1, policy_version 231974 (0.0007) [2023-12-27 07:15:31,164][20381] Updated weights for policy 1, policy_version 231984 (0.0008) [2023-12-27 07:15:31,762][20347] Updated weights for policy 0, policy_version 232274 (0.0007) [2023-12-27 07:15:31,788][20381] Updated weights for policy 1, policy_version 231994 (0.0005) [2023-12-27 07:15:31,814][20347] Updated weights for policy 0, policy_version 232284 (0.0006) [2023-12-27 07:15:31,844][20381] Updated weights for policy 1, policy_version 232004 (0.0007) [2023-12-27 07:15:31,877][20347] Updated weights for policy 0, policy_version 232294 (0.0008) [2023-12-27 07:15:31,902][20381] Updated weights for policy 1, policy_version 232014 (0.0007) [2023-12-27 07:15:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 118890496. Throughput: 0: 10275.1, 1: 10124.9. Samples: 118901836. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:32,363][19170] Avg episode reward: [(0, '516.210'), (1, '1000.000')] [2023-12-27 07:15:32,495][20347] Updated weights for policy 0, policy_version 232304 (0.0007) [2023-12-27 07:15:32,546][20347] Updated weights for policy 0, policy_version 232314 (0.0007) [2023-12-27 07:15:32,601][20347] Updated weights for policy 0, policy_version 232324 (0.0007) [2023-12-27 07:15:32,638][20381] Updated weights for policy 1, policy_version 232024 (0.0008) [2023-12-27 07:15:32,686][20381] Updated weights for policy 1, policy_version 232034 (0.0007) [2023-12-27 07:15:32,746][20381] Updated weights for policy 1, policy_version 232044 (0.0008) [2023-12-27 07:15:33,329][20347] Updated weights for policy 0, policy_version 232334 (0.0007) [2023-12-27 07:15:33,389][20347] Updated weights for policy 0, policy_version 232344 (0.0006) [2023-12-27 07:15:33,405][20381] Updated weights for policy 1, policy_version 232054 (0.0008) [2023-12-27 07:15:33,437][20347] Updated weights for policy 0, policy_version 232354 (0.0005) [2023-12-27 07:15:33,460][20381] Updated weights for policy 1, policy_version 232064 (0.0007) [2023-12-27 07:15:33,517][20381] Updated weights for policy 1, policy_version 232074 (0.0008) [2023-12-27 07:15:34,130][20347] Updated weights for policy 0, policy_version 232364 (0.0007) [2023-12-27 07:15:34,146][20381] Updated weights for policy 1, policy_version 232084 (0.0008) [2023-12-27 07:15:34,190][20347] Updated weights for policy 0, policy_version 232374 (0.0009) [2023-12-27 07:15:34,203][20381] Updated weights for policy 1, policy_version 232094 (0.0006) [2023-12-27 07:15:34,247][20347] Updated weights for policy 0, policy_version 232384 (0.0008) [2023-12-27 07:15:34,264][20381] Updated weights for policy 1, policy_version 232104 (0.0006) [2023-12-27 07:15:34,945][20347] Updated weights for policy 0, policy_version 232394 (0.0008) [2023-12-27 07:15:34,970][20381] Updated weights for policy 1, policy_version 232114 (0.0007) [2023-12-27 07:15:34,999][20347] Updated weights for policy 0, policy_version 232404 (0.0007) [2023-12-27 07:15:35,027][20381] Updated weights for policy 1, policy_version 232124 (0.0009) [2023-12-27 07:15:35,060][20347] Updated weights for policy 0, policy_version 232414 (0.0007) [2023-12-27 07:15:35,080][20381] Updated weights for policy 1, policy_version 232134 (0.0008) [2023-12-27 07:15:35,119][20347] Updated weights for policy 0, policy_version 232424 (0.0007) [2023-12-27 07:15:35,132][20381] Updated weights for policy 1, policy_version 232144 (0.0006) [2023-12-27 07:15:35,754][20381] Updated weights for policy 1, policy_version 232154 (0.0010) [2023-12-27 07:15:35,758][20347] Updated weights for policy 0, policy_version 232434 (0.0005) [2023-12-27 07:15:35,809][20347] Updated weights for policy 0, policy_version 232444 (0.0008) [2023-12-27 07:15:35,810][20381] Updated weights for policy 1, policy_version 232164 (0.0011) [2023-12-27 07:15:35,865][20347] Updated weights for policy 0, policy_version 232454 (0.0006) [2023-12-27 07:15:35,869][20381] Updated weights for policy 1, policy_version 232174 (0.0011) [2023-12-27 07:15:36,510][20347] Updated weights for policy 0, policy_version 232464 (0.0008) [2023-12-27 07:15:36,558][20347] Updated weights for policy 0, policy_version 232474 (0.0009) [2023-12-27 07:15:36,590][20381] Updated weights for policy 1, policy_version 232184 (0.0007) [2023-12-27 07:15:36,610][20347] Updated weights for policy 0, policy_version 232484 (0.0007) [2023-12-27 07:15:36,643][20381] Updated weights for policy 1, policy_version 232194 (0.0008) [2023-12-27 07:15:36,702][20381] Updated weights for policy 1, policy_version 232204 (0.0005) [2023-12-27 07:15:37,328][20347] Updated weights for policy 0, policy_version 232494 (0.0007) [2023-12-27 07:15:37,346][20381] Updated weights for policy 1, policy_version 232214 (0.0009) [2023-12-27 07:15:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 118988800. Throughput: 0: 10259.8, 1: 10133.4. Samples: 118963192. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:37,363][19170] Avg episode reward: [(0, '592.440'), (1, '1000.000')] [2023-12-27 07:15:37,384][20347] Updated weights for policy 0, policy_version 232504 (0.0009) [2023-12-27 07:15:37,405][20381] Updated weights for policy 1, policy_version 232224 (0.0010) [2023-12-27 07:15:37,440][20347] Updated weights for policy 0, policy_version 232514 (0.0007) [2023-12-27 07:15:37,461][20381] Updated weights for policy 1, policy_version 232234 (0.0011) [2023-12-27 07:15:37,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000232520_59539456.pth... [2023-12-27 07:15:37,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000231304_59228160.pth [2023-12-27 07:15:37,495][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000232240_59465728.pth... [2023-12-27 07:15:37,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000231056_59162624.pth [2023-12-27 07:15:38,125][20347] Updated weights for policy 0, policy_version 232524 (0.0006) [2023-12-27 07:15:38,128][20381] Updated weights for policy 1, policy_version 232244 (0.0011) [2023-12-27 07:15:38,173][20347] Updated weights for policy 0, policy_version 232534 (0.0007) [2023-12-27 07:15:38,176][20381] Updated weights for policy 1, policy_version 232254 (0.0010) [2023-12-27 07:15:38,220][20347] Updated weights for policy 0, policy_version 232544 (0.0008) [2023-12-27 07:15:38,229][20381] Updated weights for policy 1, policy_version 232264 (0.0010) [2023-12-27 07:15:38,921][20347] Updated weights for policy 0, policy_version 232554 (0.0006) [2023-12-27 07:15:38,933][20381] Updated weights for policy 1, policy_version 232274 (0.0010) [2023-12-27 07:15:38,976][20347] Updated weights for policy 0, policy_version 232564 (0.0006) [2023-12-27 07:15:38,988][20381] Updated weights for policy 1, policy_version 232284 (0.0011) [2023-12-27 07:15:39,035][20347] Updated weights for policy 0, policy_version 232574 (0.0007) [2023-12-27 07:15:39,054][20381] Updated weights for policy 1, policy_version 232294 (0.0007) [2023-12-27 07:15:39,091][20347] Updated weights for policy 0, policy_version 232584 (0.0008) [2023-12-27 07:15:39,110][20381] Updated weights for policy 1, policy_version 232304 (0.0008) [2023-12-27 07:15:39,791][20347] Updated weights for policy 0, policy_version 232594 (0.0007) [2023-12-27 07:15:39,794][20381] Updated weights for policy 1, policy_version 232314 (0.0011) [2023-12-27 07:15:39,853][20347] Updated weights for policy 0, policy_version 232604 (0.0007) [2023-12-27 07:15:39,856][20381] Updated weights for policy 1, policy_version 232324 (0.0010) [2023-12-27 07:15:39,914][20347] Updated weights for policy 0, policy_version 232614 (0.0007) [2023-12-27 07:15:39,916][20381] Updated weights for policy 1, policy_version 232334 (0.0010) [2023-12-27 07:15:40,575][20381] Updated weights for policy 1, policy_version 232344 (0.0006) [2023-12-27 07:15:40,632][20381] Updated weights for policy 1, policy_version 232354 (0.0007) [2023-12-27 07:15:40,644][20347] Updated weights for policy 0, policy_version 232624 (0.0006) [2023-12-27 07:15:40,691][20381] Updated weights for policy 1, policy_version 232364 (0.0010) [2023-12-27 07:15:40,698][20347] Updated weights for policy 0, policy_version 232634 (0.0006) [2023-12-27 07:15:40,755][20347] Updated weights for policy 0, policy_version 232644 (0.0007) [2023-12-27 07:15:41,371][20381] Updated weights for policy 1, policy_version 232374 (0.0009) [2023-12-27 07:15:41,418][20347] Updated weights for policy 0, policy_version 232654 (0.0008) [2023-12-27 07:15:41,434][20381] Updated weights for policy 1, policy_version 232384 (0.0007) [2023-12-27 07:15:41,478][20347] Updated weights for policy 0, policy_version 232664 (0.0008) [2023-12-27 07:15:41,497][20381] Updated weights for policy 1, policy_version 232394 (0.0006) [2023-12-27 07:15:41,537][20347] Updated weights for policy 0, policy_version 232674 (0.0007) [2023-12-27 07:15:42,213][20347] Updated weights for policy 0, policy_version 232684 (0.0008) [2023-12-27 07:15:42,214][20381] Updated weights for policy 1, policy_version 232404 (0.0007) [2023-12-27 07:15:42,261][20381] Updated weights for policy 1, policy_version 232414 (0.0008) [2023-12-27 07:15:42,275][20347] Updated weights for policy 0, policy_version 232694 (0.0010) [2023-12-27 07:15:42,315][20381] Updated weights for policy 1, policy_version 232424 (0.0007) [2023-12-27 07:15:42,331][20347] Updated weights for policy 0, policy_version 232704 (0.0010) [2023-12-27 07:15:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 119087104. Throughput: 0: 10280.1, 1: 10126.0. Samples: 119086232. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:42,363][19170] Avg episode reward: [(0, '607.370'), (1, '1000.000')] [2023-12-27 07:15:42,985][20347] Updated weights for policy 0, policy_version 232714 (0.0008) [2023-12-27 07:15:43,033][20381] Updated weights for policy 1, policy_version 232434 (0.0009) [2023-12-27 07:15:43,034][20347] Updated weights for policy 0, policy_version 232724 (0.0008) [2023-12-27 07:15:43,080][20347] Updated weights for policy 0, policy_version 232734 (0.0006) [2023-12-27 07:15:43,090][20381] Updated weights for policy 1, policy_version 232444 (0.0007) [2023-12-27 07:15:43,124][20347] Updated weights for policy 0, policy_version 232744 (0.0006) [2023-12-27 07:15:43,146][20381] Updated weights for policy 1, policy_version 232454 (0.0007) [2023-12-27 07:15:43,194][20381] Updated weights for policy 1, policy_version 232464 (0.0007) [2023-12-27 07:15:43,697][20347] Updated weights for policy 0, policy_version 232754 (0.0007) [2023-12-27 07:15:43,751][20347] Updated weights for policy 0, policy_version 232764 (0.0005) [2023-12-27 07:15:43,802][20347] Updated weights for policy 0, policy_version 232774 (0.0007) [2023-12-27 07:15:43,917][20381] Updated weights for policy 1, policy_version 232474 (0.0008) [2023-12-27 07:15:43,978][20381] Updated weights for policy 1, policy_version 232484 (0.0008) [2023-12-27 07:15:44,035][20381] Updated weights for policy 1, policy_version 232494 (0.0008) [2023-12-27 07:15:44,460][20347] Updated weights for policy 0, policy_version 232784 (0.0006) [2023-12-27 07:15:44,518][20347] Updated weights for policy 0, policy_version 232794 (0.0006) [2023-12-27 07:15:44,566][20347] Updated weights for policy 0, policy_version 232804 (0.0007) [2023-12-27 07:15:44,743][20381] Updated weights for policy 1, policy_version 232504 (0.0008) [2023-12-27 07:15:44,804][20381] Updated weights for policy 1, policy_version 232514 (0.0009) [2023-12-27 07:15:44,859][20381] Updated weights for policy 1, policy_version 232524 (0.0008) [2023-12-27 07:15:45,214][20347] Updated weights for policy 0, policy_version 232814 (0.0006) [2023-12-27 07:15:45,261][20347] Updated weights for policy 0, policy_version 232824 (0.0005) [2023-12-27 07:15:45,317][20347] Updated weights for policy 0, policy_version 232834 (0.0006) [2023-12-27 07:15:45,615][20381] Updated weights for policy 1, policy_version 232534 (0.0007) [2023-12-27 07:15:45,671][20381] Updated weights for policy 1, policy_version 232544 (0.0007) [2023-12-27 07:15:45,723][20381] Updated weights for policy 1, policy_version 232554 (0.0008) [2023-12-27 07:15:45,946][20347] Updated weights for policy 0, policy_version 232844 (0.0007) [2023-12-27 07:15:46,001][20347] Updated weights for policy 0, policy_version 232854 (0.0005) [2023-12-27 07:15:46,063][20347] Updated weights for policy 0, policy_version 232864 (0.0005) [2023-12-27 07:15:46,437][20381] Updated weights for policy 1, policy_version 232564 (0.0008) [2023-12-27 07:15:46,494][20381] Updated weights for policy 1, policy_version 232574 (0.0007) [2023-12-27 07:15:46,555][20381] Updated weights for policy 1, policy_version 232584 (0.0008) [2023-12-27 07:15:46,634][20347] Updated weights for policy 0, policy_version 232874 (0.0005) [2023-12-27 07:15:46,695][20347] Updated weights for policy 0, policy_version 232884 (0.0006) [2023-12-27 07:15:46,750][20347] Updated weights for policy 0, policy_version 232894 (0.0007) [2023-12-27 07:15:46,808][20347] Updated weights for policy 0, policy_version 232904 (0.0007) [2023-12-27 07:15:47,210][20381] Updated weights for policy 1, policy_version 232594 (0.0008) [2023-12-27 07:15:47,257][20381] Updated weights for policy 1, policy_version 232604 (0.0007) [2023-12-27 07:15:47,307][20381] Updated weights for policy 1, policy_version 232614 (0.0008) [2023-12-27 07:15:47,360][20381] Updated weights for policy 1, policy_version 232624 (0.0008) [2023-12-27 07:15:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 119201792. Throughput: 0: 10337.0, 1: 10054.7. Samples: 119209916. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:47,363][19170] Avg episode reward: [(0, '460.680'), (1, '1000.000')] [2023-12-27 07:15:47,424][20347] Updated weights for policy 0, policy_version 232914 (0.0007) [2023-12-27 07:15:47,473][20347] Updated weights for policy 0, policy_version 232924 (0.0008) [2023-12-27 07:15:47,530][20347] Updated weights for policy 0, policy_version 232934 (0.0008) [2023-12-27 07:15:48,036][20381] Updated weights for policy 1, policy_version 232634 (0.0011) [2023-12-27 07:15:48,095][20381] Updated weights for policy 1, policy_version 232644 (0.0011) [2023-12-27 07:15:48,150][20381] Updated weights for policy 1, policy_version 232654 (0.0010) [2023-12-27 07:15:48,220][20347] Updated weights for policy 0, policy_version 232944 (0.0007) [2023-12-27 07:15:48,267][20347] Updated weights for policy 0, policy_version 232954 (0.0007) [2023-12-27 07:15:48,314][20347] Updated weights for policy 0, policy_version 232964 (0.0007) [2023-12-27 07:15:48,802][20381] Updated weights for policy 1, policy_version 232664 (0.0011) [2023-12-27 07:15:48,866][20381] Updated weights for policy 1, policy_version 232674 (0.0011) [2023-12-27 07:15:48,922][20381] Updated weights for policy 1, policy_version 232684 (0.0011) [2023-12-27 07:15:49,044][20347] Updated weights for policy 0, policy_version 232974 (0.0010) [2023-12-27 07:15:49,103][20347] Updated weights for policy 0, policy_version 232984 (0.0010) [2023-12-27 07:15:49,158][20347] Updated weights for policy 0, policy_version 232994 (0.0010) [2023-12-27 07:15:49,623][20381] Updated weights for policy 1, policy_version 232694 (0.0009) [2023-12-27 07:15:49,687][20381] Updated weights for policy 1, policy_version 232704 (0.0007) [2023-12-27 07:15:49,748][20381] Updated weights for policy 1, policy_version 232714 (0.0008) [2023-12-27 07:15:49,831][20347] Updated weights for policy 0, policy_version 233004 (0.0010) [2023-12-27 07:15:49,887][20347] Updated weights for policy 0, policy_version 233014 (0.0008) [2023-12-27 07:15:49,950][20347] Updated weights for policy 0, policy_version 233024 (0.0008) [2023-12-27 07:15:50,445][20381] Updated weights for policy 1, policy_version 232724 (0.0007) [2023-12-27 07:15:50,494][20381] Updated weights for policy 1, policy_version 232734 (0.0007) [2023-12-27 07:15:50,545][20381] Updated weights for policy 1, policy_version 232744 (0.0009) [2023-12-27 07:15:50,709][20347] Updated weights for policy 0, policy_version 233034 (0.0009) [2023-12-27 07:15:50,773][20347] Updated weights for policy 0, policy_version 233044 (0.0008) [2023-12-27 07:15:50,833][20347] Updated weights for policy 0, policy_version 233054 (0.0009) [2023-12-27 07:15:50,896][20347] Updated weights for policy 0, policy_version 233064 (0.0007) [2023-12-27 07:15:51,240][20381] Updated weights for policy 1, policy_version 232754 (0.0009) [2023-12-27 07:15:51,304][20381] Updated weights for policy 1, policy_version 232764 (0.0008) [2023-12-27 07:15:51,365][20381] Updated weights for policy 1, policy_version 232774 (0.0007) [2023-12-27 07:15:51,424][20381] Updated weights for policy 1, policy_version 232784 (0.0007) [2023-12-27 07:15:51,631][20347] Updated weights for policy 0, policy_version 233074 (0.0008) [2023-12-27 07:15:51,692][20347] Updated weights for policy 0, policy_version 233084 (0.0009) [2023-12-27 07:15:51,755][20347] Updated weights for policy 0, policy_version 233094 (0.0008) [2023-12-27 07:15:52,074][20381] Updated weights for policy 1, policy_version 232794 (0.0008) [2023-12-27 07:15:52,136][20381] Updated weights for policy 1, policy_version 232804 (0.0008) [2023-12-27 07:15:52,189][20381] Updated weights for policy 1, policy_version 232814 (0.0008) [2023-12-27 07:15:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 119300096. Throughput: 0: 10322.1, 1: 10034.1. Samples: 119270368. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:52,363][19170] Avg episode reward: [(0, '389.720'), (1, '1000.000')] [2023-12-27 07:15:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000232816_59613184.pth... [2023-12-27 07:15:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000233096_59686912.pth... [2023-12-27 07:15:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000231632_59310080.pth [2023-12-27 07:15:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000231880_59375616.pth [2023-12-27 07:15:52,451][20347] Updated weights for policy 0, policy_version 233104 (0.0007) [2023-12-27 07:15:52,505][20347] Updated weights for policy 0, policy_version 233114 (0.0007) [2023-12-27 07:15:52,556][20347] Updated weights for policy 0, policy_version 233124 (0.0007) [2023-12-27 07:15:52,852][20381] Updated weights for policy 1, policy_version 232824 (0.0006) [2023-12-27 07:15:52,902][20381] Updated weights for policy 1, policy_version 232834 (0.0005) [2023-12-27 07:15:52,957][20381] Updated weights for policy 1, policy_version 232844 (0.0005) [2023-12-27 07:15:53,303][20347] Updated weights for policy 0, policy_version 233134 (0.0006) [2023-12-27 07:15:53,357][20347] Updated weights for policy 0, policy_version 233144 (0.0005) [2023-12-27 07:15:53,404][20347] Updated weights for policy 0, policy_version 233154 (0.0007) [2023-12-27 07:15:53,588][20381] Updated weights for policy 1, policy_version 232854 (0.0005) [2023-12-27 07:15:53,653][20381] Updated weights for policy 1, policy_version 232864 (0.0005) [2023-12-27 07:15:53,715][20381] Updated weights for policy 1, policy_version 232874 (0.0006) [2023-12-27 07:15:53,981][20347] Updated weights for policy 0, policy_version 233164 (0.0007) [2023-12-27 07:15:54,030][20347] Updated weights for policy 0, policy_version 233174 (0.0005) [2023-12-27 07:15:54,076][20347] Updated weights for policy 0, policy_version 233184 (0.0005) [2023-12-27 07:15:54,382][20381] Updated weights for policy 1, policy_version 232884 (0.0007) [2023-12-27 07:15:54,439][20381] Updated weights for policy 1, policy_version 232894 (0.0005) [2023-12-27 07:15:54,494][20381] Updated weights for policy 1, policy_version 232904 (0.0010) [2023-12-27 07:15:54,642][20347] Updated weights for policy 0, policy_version 233194 (0.0006) [2023-12-27 07:15:54,702][20347] Updated weights for policy 0, policy_version 233204 (0.0007) [2023-12-27 07:15:54,754][20347] Updated weights for policy 0, policy_version 233214 (0.0008) [2023-12-27 07:15:54,798][20347] Updated weights for policy 0, policy_version 233224 (0.0006) [2023-12-27 07:15:55,162][20381] Updated weights for policy 1, policy_version 232914 (0.0011) [2023-12-27 07:15:55,221][20381] Updated weights for policy 1, policy_version 232924 (0.0011) [2023-12-27 07:15:55,285][20381] Updated weights for policy 1, policy_version 232934 (0.0011) [2023-12-27 07:15:55,342][20381] Updated weights for policy 1, policy_version 232944 (0.0010) [2023-12-27 07:15:55,485][20347] Updated weights for policy 0, policy_version 233234 (0.0007) [2023-12-27 07:15:55,536][20347] Updated weights for policy 0, policy_version 233244 (0.0009) [2023-12-27 07:15:55,591][20347] Updated weights for policy 0, policy_version 233254 (0.0010) [2023-12-27 07:15:55,936][20381] Updated weights for policy 1, policy_version 232954 (0.0005) [2023-12-27 07:15:55,991][20381] Updated weights for policy 1, policy_version 232964 (0.0006) [2023-12-27 07:15:56,049][20381] Updated weights for policy 1, policy_version 232974 (0.0006) [2023-12-27 07:15:56,233][20347] Updated weights for policy 0, policy_version 233264 (0.0008) [2023-12-27 07:15:56,293][20347] Updated weights for policy 0, policy_version 233274 (0.0008) [2023-12-27 07:15:56,354][20347] Updated weights for policy 0, policy_version 233284 (0.0008) [2023-12-27 07:15:56,639][20381] Updated weights for policy 1, policy_version 232984 (0.0006) [2023-12-27 07:15:56,682][20381] Updated weights for policy 1, policy_version 232994 (0.0005) [2023-12-27 07:15:56,729][20381] Updated weights for policy 1, policy_version 233004 (0.0010) [2023-12-27 07:15:56,998][20347] Updated weights for policy 0, policy_version 233294 (0.0007) [2023-12-27 07:15:57,058][20347] Updated weights for policy 0, policy_version 233304 (0.0010) [2023-12-27 07:15:57,108][20347] Updated weights for policy 0, policy_version 233314 (0.0010) [2023-12-27 07:15:57,318][20381] Updated weights for policy 1, policy_version 233014 (0.0010) [2023-12-27 07:15:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 119406592. Throughput: 0: 10306.0, 1: 10083.6. Samples: 119395052. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:15:57,363][19170] Avg episode reward: [(0, '358.700'), (1, '1000.000')] [2023-12-27 07:15:57,382][20381] Updated weights for policy 1, policy_version 233024 (0.0010) [2023-12-27 07:15:57,436][20381] Updated weights for policy 1, policy_version 233034 (0.0010) [2023-12-27 07:15:57,673][20347] Updated weights for policy 0, policy_version 233324 (0.0008) [2023-12-27 07:15:57,726][20347] Updated weights for policy 0, policy_version 233334 (0.0007) [2023-12-27 07:15:57,775][20347] Updated weights for policy 0, policy_version 233344 (0.0006) [2023-12-27 07:15:58,044][20381] Updated weights for policy 1, policy_version 233044 (0.0009) [2023-12-27 07:15:58,096][20381] Updated weights for policy 1, policy_version 233054 (0.0007) [2023-12-27 07:15:58,148][20381] Updated weights for policy 1, policy_version 233064 (0.0008) [2023-12-27 07:15:58,518][20347] Updated weights for policy 0, policy_version 233354 (0.0008) [2023-12-27 07:15:58,578][20347] Updated weights for policy 0, policy_version 233364 (0.0009) [2023-12-27 07:15:58,641][20347] Updated weights for policy 0, policy_version 233374 (0.0011) [2023-12-27 07:15:58,705][20347] Updated weights for policy 0, policy_version 233384 (0.0011) [2023-12-27 07:15:58,954][20381] Updated weights for policy 1, policy_version 233074 (0.0007) [2023-12-27 07:15:59,002][20381] Updated weights for policy 1, policy_version 233084 (0.0007) [2023-12-27 07:15:59,061][20381] Updated weights for policy 1, policy_version 233094 (0.0008) [2023-12-27 07:15:59,121][20381] Updated weights for policy 1, policy_version 233104 (0.0008) [2023-12-27 07:15:59,491][20347] Updated weights for policy 0, policy_version 233394 (0.0010) [2023-12-27 07:15:59,558][20347] Updated weights for policy 0, policy_version 233404 (0.0010) [2023-12-27 07:15:59,620][20347] Updated weights for policy 0, policy_version 233414 (0.0010) [2023-12-27 07:15:59,829][20381] Updated weights for policy 1, policy_version 233114 (0.0007) [2023-12-27 07:15:59,886][20381] Updated weights for policy 1, policy_version 233124 (0.0007) [2023-12-27 07:15:59,947][20381] Updated weights for policy 1, policy_version 233134 (0.0007) [2023-12-27 07:16:00,269][20347] Updated weights for policy 0, policy_version 233424 (0.0010) [2023-12-27 07:16:00,327][20347] Updated weights for policy 0, policy_version 233434 (0.0010) [2023-12-27 07:16:00,387][20347] Updated weights for policy 0, policy_version 233444 (0.0010) [2023-12-27 07:16:00,573][20381] Updated weights for policy 1, policy_version 233144 (0.0007) [2023-12-27 07:16:00,617][20381] Updated weights for policy 1, policy_version 233154 (0.0007) [2023-12-27 07:16:00,670][20381] Updated weights for policy 1, policy_version 233164 (0.0007) [2023-12-27 07:16:01,061][20347] Updated weights for policy 0, policy_version 233454 (0.0011) [2023-12-27 07:16:01,109][20347] Updated weights for policy 0, policy_version 233464 (0.0006) [2023-12-27 07:16:01,167][20347] Updated weights for policy 0, policy_version 233474 (0.0010) [2023-12-27 07:16:01,401][20381] Updated weights for policy 1, policy_version 233174 (0.0007) [2023-12-27 07:16:01,458][20381] Updated weights for policy 1, policy_version 233184 (0.0008) [2023-12-27 07:16:01,509][20381] Updated weights for policy 1, policy_version 233194 (0.0008) [2023-12-27 07:16:01,830][20347] Updated weights for policy 0, policy_version 233484 (0.0008) [2023-12-27 07:16:01,891][20347] Updated weights for policy 0, policy_version 233494 (0.0006) [2023-12-27 07:16:01,949][20347] Updated weights for policy 0, policy_version 233504 (0.0010) [2023-12-27 07:16:02,242][20381] Updated weights for policy 1, policy_version 233204 (0.0009) [2023-12-27 07:16:02,300][20381] Updated weights for policy 1, policy_version 233214 (0.0007) [2023-12-27 07:16:02,358][20381] Updated weights for policy 1, policy_version 233224 (0.0008) [2023-12-27 07:16:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 119504896. Throughput: 0: 10370.1, 1: 10156.8. Samples: 119519584. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:16:02,363][19170] Avg episode reward: [(0, '345.460'), (1, '1000.000')] [2023-12-27 07:16:02,595][20347] Updated weights for policy 0, policy_version 233514 (0.0009) [2023-12-27 07:16:02,656][20347] Updated weights for policy 0, policy_version 233524 (0.0010) [2023-12-27 07:16:02,709][20347] Updated weights for policy 0, policy_version 233534 (0.0010) [2023-12-27 07:16:02,764][20347] Updated weights for policy 0, policy_version 233544 (0.0010) [2023-12-27 07:16:03,024][20381] Updated weights for policy 1, policy_version 233234 (0.0007) [2023-12-27 07:16:03,090][20381] Updated weights for policy 1, policy_version 233244 (0.0008) [2023-12-27 07:16:03,151][20381] Updated weights for policy 1, policy_version 233254 (0.0010) [2023-12-27 07:16:03,204][20381] Updated weights for policy 1, policy_version 233264 (0.0010) [2023-12-27 07:16:03,400][20347] Updated weights for policy 0, policy_version 233554 (0.0007) [2023-12-27 07:16:03,454][20347] Updated weights for policy 0, policy_version 233564 (0.0006) [2023-12-27 07:16:03,507][20347] Updated weights for policy 0, policy_version 233574 (0.0007) [2023-12-27 07:16:03,922][20381] Updated weights for policy 1, policy_version 233274 (0.0010) [2023-12-27 07:16:03,974][20381] Updated weights for policy 1, policy_version 233284 (0.0008) [2023-12-27 07:16:04,037][20381] Updated weights for policy 1, policy_version 233294 (0.0008) [2023-12-27 07:16:04,107][20347] Updated weights for policy 0, policy_version 233584 (0.0008) [2023-12-27 07:16:04,168][20347] Updated weights for policy 0, policy_version 233594 (0.0005) [2023-12-27 07:16:04,230][20347] Updated weights for policy 0, policy_version 233604 (0.0006) [2023-12-27 07:16:04,806][20381] Updated weights for policy 1, policy_version 233304 (0.0008) [2023-12-27 07:16:04,835][20347] Updated weights for policy 0, policy_version 233614 (0.0006) [2023-12-27 07:16:04,859][20381] Updated weights for policy 1, policy_version 233314 (0.0007) [2023-12-27 07:16:04,888][20347] Updated weights for policy 0, policy_version 233624 (0.0005) [2023-12-27 07:16:04,915][20381] Updated weights for policy 1, policy_version 233324 (0.0007) [2023-12-27 07:16:04,941][20347] Updated weights for policy 0, policy_version 233634 (0.0005) [2023-12-27 07:16:05,486][20347] Updated weights for policy 0, policy_version 233644 (0.0006) [2023-12-27 07:16:05,542][20347] Updated weights for policy 0, policy_version 233654 (0.0010) [2023-12-27 07:16:05,597][20347] Updated weights for policy 0, policy_version 233664 (0.0010) [2023-12-27 07:16:05,675][20381] Updated weights for policy 1, policy_version 233334 (0.0007) [2023-12-27 07:16:05,731][20381] Updated weights for policy 1, policy_version 233344 (0.0008) [2023-12-27 07:16:05,785][20381] Updated weights for policy 1, policy_version 233354 (0.0007) [2023-12-27 07:16:06,254][20347] Updated weights for policy 0, policy_version 233674 (0.0010) [2023-12-27 07:16:06,316][20347] Updated weights for policy 0, policy_version 233684 (0.0007) [2023-12-27 07:16:06,376][20347] Updated weights for policy 0, policy_version 233694 (0.0007) [2023-12-27 07:16:06,440][20347] Updated weights for policy 0, policy_version 233704 (0.0006) [2023-12-27 07:16:06,453][20381] Updated weights for policy 1, policy_version 233364 (0.0008) [2023-12-27 07:16:06,518][20381] Updated weights for policy 1, policy_version 233374 (0.0006) [2023-12-27 07:16:06,581][20381] Updated weights for policy 1, policy_version 233384 (0.0007) [2023-12-27 07:16:07,160][20347] Updated weights for policy 0, policy_version 233714 (0.0007) [2023-12-27 07:16:07,181][20381] Updated weights for policy 1, policy_version 233394 (0.0007) [2023-12-27 07:16:07,213][20347] Updated weights for policy 0, policy_version 233724 (0.0008) [2023-12-27 07:16:07,242][20381] Updated weights for policy 1, policy_version 233404 (0.0005) [2023-12-27 07:16:07,266][20347] Updated weights for policy 0, policy_version 233734 (0.0008) [2023-12-27 07:16:07,289][20381] Updated weights for policy 1, policy_version 233414 (0.0005) [2023-12-27 07:16:07,345][20381] Updated weights for policy 1, policy_version 233424 (0.0005) [2023-12-27 07:16:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 119619584. Throughput: 0: 10405.3, 1: 10158.7. Samples: 119581276. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:16:07,363][19170] Avg episode reward: [(0, '195.160'), (1, '1000.000')] [2023-12-27 07:16:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000233424_59768832.pth... [2023-12-27 07:16:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000233736_59850752.pth... [2023-12-27 07:16:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000232520_59539456.pth [2023-12-27 07:16:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000232240_59465728.pth [2023-12-27 07:16:07,943][20381] Updated weights for policy 1, policy_version 233434 (0.0005) [2023-12-27 07:16:07,950][20347] Updated weights for policy 0, policy_version 233744 (0.0008) [2023-12-27 07:16:07,991][20381] Updated weights for policy 1, policy_version 233444 (0.0005) [2023-12-27 07:16:08,007][20347] Updated weights for policy 0, policy_version 233754 (0.0006) [2023-12-27 07:16:08,042][20381] Updated weights for policy 1, policy_version 233454 (0.0005) [2023-12-27 07:16:08,051][20347] Updated weights for policy 0, policy_version 233764 (0.0005) [2023-12-27 07:16:08,620][20347] Updated weights for policy 0, policy_version 233774 (0.0005) [2023-12-27 07:16:08,681][20347] Updated weights for policy 0, policy_version 233784 (0.0005) [2023-12-27 07:16:08,683][20381] Updated weights for policy 1, policy_version 233464 (0.0007) [2023-12-27 07:16:08,742][20347] Updated weights for policy 0, policy_version 233794 (0.0006) [2023-12-27 07:16:08,743][20381] Updated weights for policy 1, policy_version 233474 (0.0007) [2023-12-27 07:16:08,799][20381] Updated weights for policy 1, policy_version 233484 (0.0006) [2023-12-27 07:16:09,400][20347] Updated weights for policy 0, policy_version 233804 (0.0007) [2023-12-27 07:16:09,464][20347] Updated weights for policy 0, policy_version 233814 (0.0008) [2023-12-27 07:16:09,493][20381] Updated weights for policy 1, policy_version 233494 (0.0006) [2023-12-27 07:16:09,523][20347] Updated weights for policy 0, policy_version 233824 (0.0006) [2023-12-27 07:16:09,552][20381] Updated weights for policy 1, policy_version 233504 (0.0008) [2023-12-27 07:16:09,604][20381] Updated weights for policy 1, policy_version 233514 (0.0009) [2023-12-27 07:16:10,241][20347] Updated weights for policy 0, policy_version 233834 (0.0009) [2023-12-27 07:16:10,301][20347] Updated weights for policy 0, policy_version 233844 (0.0006) [2023-12-27 07:16:10,322][20381] Updated weights for policy 1, policy_version 233524 (0.0008) [2023-12-27 07:16:10,359][20347] Updated weights for policy 0, policy_version 233854 (0.0007) [2023-12-27 07:16:10,385][20381] Updated weights for policy 1, policy_version 233534 (0.0009) [2023-12-27 07:16:10,417][20347] Updated weights for policy 0, policy_version 233864 (0.0007) [2023-12-27 07:16:10,442][20381] Updated weights for policy 1, policy_version 233544 (0.0009) [2023-12-27 07:16:11,098][20347] Updated weights for policy 0, policy_version 233874 (0.0007) [2023-12-27 07:16:11,157][20381] Updated weights for policy 1, policy_version 233554 (0.0009) [2023-12-27 07:16:11,170][20347] Updated weights for policy 0, policy_version 233884 (0.0007) [2023-12-27 07:16:11,211][20381] Updated weights for policy 1, policy_version 233564 (0.0006) [2023-12-27 07:16:11,225][20347] Updated weights for policy 0, policy_version 233894 (0.0006) [2023-12-27 07:16:11,267][20381] Updated weights for policy 1, policy_version 233574 (0.0006) [2023-12-27 07:16:11,323][20381] Updated weights for policy 1, policy_version 233584 (0.0008) [2023-12-27 07:16:11,990][20347] Updated weights for policy 0, policy_version 233904 (0.0011) [2023-12-27 07:16:12,041][20347] Updated weights for policy 0, policy_version 233914 (0.0007) [2023-12-27 07:16:12,065][20381] Updated weights for policy 1, policy_version 233594 (0.0008) [2023-12-27 07:16:12,098][20347] Updated weights for policy 0, policy_version 233924 (0.0006) [2023-12-27 07:16:12,124][20381] Updated weights for policy 1, policy_version 233604 (0.0006) [2023-12-27 07:16:12,179][20381] Updated weights for policy 1, policy_version 233614 (0.0006) [2023-12-27 07:16:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 119717888. Throughput: 0: 10477.0, 1: 10159.8. Samples: 119706040. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 07:16:12,363][19170] Avg episode reward: [(0, '188.260'), (1, '1000.000')] [2023-12-27 07:16:12,757][20347] Updated weights for policy 0, policy_version 233934 (0.0007) [2023-12-27 07:16:12,766][20381] Updated weights for policy 1, policy_version 233624 (0.0007) [2023-12-27 07:16:12,818][20347] Updated weights for policy 0, policy_version 233944 (0.0010) [2023-12-27 07:16:12,828][20381] Updated weights for policy 1, policy_version 233634 (0.0006) [2023-12-27 07:16:12,882][20347] Updated weights for policy 0, policy_version 233954 (0.0010) [2023-12-27 07:16:12,893][20381] Updated weights for policy 1, policy_version 233644 (0.0006) [2023-12-27 07:16:13,473][20381] Updated weights for policy 1, policy_version 233654 (0.0005) [2023-12-27 07:16:13,508][20347] Updated weights for policy 0, policy_version 233964 (0.0009) [2023-12-27 07:16:13,529][20381] Updated weights for policy 1, policy_version 233664 (0.0005) [2023-12-27 07:16:13,553][20347] Updated weights for policy 0, policy_version 233974 (0.0007) [2023-12-27 07:16:13,588][20381] Updated weights for policy 1, policy_version 233674 (0.0005) [2023-12-27 07:16:13,608][20347] Updated weights for policy 0, policy_version 233984 (0.0007) [2023-12-27 07:16:14,187][20381] Updated weights for policy 1, policy_version 233684 (0.0006) [2023-12-27 07:16:14,231][20381] Updated weights for policy 1, policy_version 233694 (0.0005) [2023-12-27 07:16:14,264][20347] Updated weights for policy 0, policy_version 233994 (0.0008) [2023-12-27 07:16:14,283][20381] Updated weights for policy 1, policy_version 233704 (0.0005) [2023-12-27 07:16:14,320][20347] Updated weights for policy 0, policy_version 234004 (0.0007) [2023-12-27 07:16:14,376][20347] Updated weights for policy 0, policy_version 234014 (0.0007) [2023-12-27 07:16:14,429][20347] Updated weights for policy 0, policy_version 234024 (0.0007) [2023-12-27 07:16:14,974][20381] Updated weights for policy 1, policy_version 233714 (0.0006) [2023-12-27 07:16:15,030][20381] Updated weights for policy 1, policy_version 233724 (0.0008) [2023-12-27 07:16:15,086][20381] Updated weights for policy 1, policy_version 233734 (0.0008) [2023-12-27 07:16:15,142][20381] Updated weights for policy 1, policy_version 233744 (0.0008) [2023-12-27 07:16:15,163][20347] Updated weights for policy 0, policy_version 234034 (0.0006) [2023-12-27 07:16:15,221][20347] Updated weights for policy 0, policy_version 234044 (0.0008) [2023-12-27 07:16:15,279][20347] Updated weights for policy 0, policy_version 234054 (0.0007) [2023-12-27 07:16:15,807][20381] Updated weights for policy 1, policy_version 233754 (0.0007) [2023-12-27 07:16:15,865][20381] Updated weights for policy 1, policy_version 233764 (0.0007) [2023-12-27 07:16:15,919][20347] Updated weights for policy 0, policy_version 234064 (0.0007) [2023-12-27 07:16:15,921][20381] Updated weights for policy 1, policy_version 233774 (0.0006) [2023-12-27 07:16:15,977][20347] Updated weights for policy 0, policy_version 234074 (0.0007) [2023-12-27 07:16:16,032][20347] Updated weights for policy 0, policy_version 234084 (0.0008) [2023-12-27 07:16:16,600][20381] Updated weights for policy 1, policy_version 233784 (0.0008) [2023-12-27 07:16:16,649][20381] Updated weights for policy 1, policy_version 233794 (0.0008) [2023-12-27 07:16:16,699][20381] Updated weights for policy 1, policy_version 233804 (0.0007) [2023-12-27 07:16:16,715][20347] Updated weights for policy 0, policy_version 234094 (0.0008) [2023-12-27 07:16:16,765][20347] Updated weights for policy 0, policy_version 234104 (0.0007) [2023-12-27 07:16:16,817][20347] Updated weights for policy 0, policy_version 234114 (0.0008) [2023-12-27 07:16:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 119816192. Throughput: 0: 10392.9, 1: 10244.2. Samples: 119830504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:17,363][19170] Avg episode reward: [(0, '263.840'), (1, '1000.000')] [2023-12-27 07:16:17,436][20381] Updated weights for policy 1, policy_version 233814 (0.0009) [2023-12-27 07:16:17,486][20381] Updated weights for policy 1, policy_version 233824 (0.0011) [2023-12-27 07:16:17,499][20347] Updated weights for policy 0, policy_version 234124 (0.0006) [2023-12-27 07:16:17,538][20381] Updated weights for policy 1, policy_version 233834 (0.0011) [2023-12-27 07:16:17,555][20347] Updated weights for policy 0, policy_version 234134 (0.0005) [2023-12-27 07:16:17,605][20347] Updated weights for policy 0, policy_version 234144 (0.0007) [2023-12-27 07:16:18,195][20381] Updated weights for policy 1, policy_version 233844 (0.0008) [2023-12-27 07:16:18,256][20381] Updated weights for policy 1, policy_version 233854 (0.0005) [2023-12-27 07:16:18,298][20347] Updated weights for policy 0, policy_version 234154 (0.0008) [2023-12-27 07:16:18,314][20381] Updated weights for policy 1, policy_version 233864 (0.0008) [2023-12-27 07:16:18,361][20347] Updated weights for policy 0, policy_version 234164 (0.0007) [2023-12-27 07:16:18,417][20347] Updated weights for policy 0, policy_version 234174 (0.0009) [2023-12-27 07:16:18,469][20347] Updated weights for policy 0, policy_version 234184 (0.0009) [2023-12-27 07:16:18,924][20381] Updated weights for policy 1, policy_version 233874 (0.0008) [2023-12-27 07:16:18,980][20381] Updated weights for policy 1, policy_version 233884 (0.0007) [2023-12-27 07:16:19,025][20381] Updated weights for policy 1, policy_version 233894 (0.0007) [2023-12-27 07:16:19,074][20381] Updated weights for policy 1, policy_version 233904 (0.0008) [2023-12-27 07:16:19,198][20347] Updated weights for policy 0, policy_version 234194 (0.0008) [2023-12-27 07:16:19,257][20347] Updated weights for policy 0, policy_version 234204 (0.0007) [2023-12-27 07:16:19,316][20347] Updated weights for policy 0, policy_version 234214 (0.0007) [2023-12-27 07:16:19,812][20381] Updated weights for policy 1, policy_version 233914 (0.0005) [2023-12-27 07:16:19,875][20381] Updated weights for policy 1, policy_version 233924 (0.0008) [2023-12-27 07:16:19,936][20381] Updated weights for policy 1, policy_version 233934 (0.0008) [2023-12-27 07:16:20,018][20347] Updated weights for policy 0, policy_version 234224 (0.0006) [2023-12-27 07:16:20,077][20347] Updated weights for policy 0, policy_version 234234 (0.0007) [2023-12-27 07:16:20,142][20347] Updated weights for policy 0, policy_version 234244 (0.0006) [2023-12-27 07:16:20,627][20381] Updated weights for policy 1, policy_version 233944 (0.0007) [2023-12-27 07:16:20,682][20381] Updated weights for policy 1, policy_version 233954 (0.0008) [2023-12-27 07:16:20,735][20381] Updated weights for policy 1, policy_version 233964 (0.0008) [2023-12-27 07:16:20,819][20347] Updated weights for policy 0, policy_version 234254 (0.0007) [2023-12-27 07:16:20,883][20347] Updated weights for policy 0, policy_version 234264 (0.0007) [2023-12-27 07:16:20,943][20347] Updated weights for policy 0, policy_version 234274 (0.0009) [2023-12-27 07:16:21,471][20381] Updated weights for policy 1, policy_version 233974 (0.0007) [2023-12-27 07:16:21,524][20381] Updated weights for policy 1, policy_version 233984 (0.0008) [2023-12-27 07:16:21,577][20381] Updated weights for policy 1, policy_version 233994 (0.0008) [2023-12-27 07:16:21,612][20347] Updated weights for policy 0, policy_version 234284 (0.0006) [2023-12-27 07:16:21,674][20347] Updated weights for policy 0, policy_version 234294 (0.0008) [2023-12-27 07:16:21,735][20347] Updated weights for policy 0, policy_version 234304 (0.0007) [2023-12-27 07:16:22,294][20381] Updated weights for policy 1, policy_version 234004 (0.0008) [2023-12-27 07:16:22,354][20381] Updated weights for policy 1, policy_version 234014 (0.0008) [2023-12-27 07:16:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 119914496. Throughput: 0: 10373.4, 1: 10243.6. Samples: 119890956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:22,363][19170] Avg episode reward: [(0, '218.950'), (1, '1000.000')] [2023-12-27 07:16:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000234312_59998208.pth... [2023-12-27 07:16:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000233096_59686912.pth [2023-12-27 07:16:22,419][20381] Updated weights for policy 1, policy_version 234024 (0.0009) [2023-12-27 07:16:22,465][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000234032_59924480.pth... [2023-12-27 07:16:22,465][20347] Updated weights for policy 0, policy_version 234314 (0.0008) [2023-12-27 07:16:22,469][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000232816_59613184.pth [2023-12-27 07:16:22,529][20347] Updated weights for policy 0, policy_version 234324 (0.0010) [2023-12-27 07:16:22,589][20347] Updated weights for policy 0, policy_version 234334 (0.0010) [2023-12-27 07:16:22,649][20347] Updated weights for policy 0, policy_version 234344 (0.0011) [2023-12-27 07:16:23,156][20381] Updated weights for policy 1, policy_version 234034 (0.0008) [2023-12-27 07:16:23,215][20381] Updated weights for policy 1, policy_version 234044 (0.0007) [2023-12-27 07:16:23,266][20381] Updated weights for policy 1, policy_version 234054 (0.0005) [2023-12-27 07:16:23,310][20381] Updated weights for policy 1, policy_version 234064 (0.0005) [2023-12-27 07:16:23,318][20347] Updated weights for policy 0, policy_version 234354 (0.0010) [2023-12-27 07:16:23,379][20347] Updated weights for policy 0, policy_version 234364 (0.0010) [2023-12-27 07:16:23,429][20347] Updated weights for policy 0, policy_version 234374 (0.0008) [2023-12-27 07:16:23,968][20381] Updated weights for policy 1, policy_version 234074 (0.0007) [2023-12-27 07:16:24,011][20347] Updated weights for policy 0, policy_version 234384 (0.0006) [2023-12-27 07:16:24,021][20381] Updated weights for policy 1, policy_version 234084 (0.0006) [2023-12-27 07:16:24,063][20347] Updated weights for policy 0, policy_version 234394 (0.0005) [2023-12-27 07:16:24,072][20381] Updated weights for policy 1, policy_version 234094 (0.0006) [2023-12-27 07:16:24,116][20347] Updated weights for policy 0, policy_version 234404 (0.0006) [2023-12-27 07:16:24,741][20381] Updated weights for policy 1, policy_version 234104 (0.0006) [2023-12-27 07:16:24,792][20381] Updated weights for policy 1, policy_version 234114 (0.0005) [2023-12-27 07:16:24,829][20347] Updated weights for policy 0, policy_version 234414 (0.0008) [2023-12-27 07:16:24,839][20381] Updated weights for policy 1, policy_version 234124 (0.0005) [2023-12-27 07:16:24,885][20347] Updated weights for policy 0, policy_version 234424 (0.0005) [2023-12-27 07:16:24,939][20347] Updated weights for policy 0, policy_version 234434 (0.0005) [2023-12-27 07:16:25,483][20381] Updated weights for policy 1, policy_version 234134 (0.0007) [2023-12-27 07:16:25,530][20381] Updated weights for policy 1, policy_version 234144 (0.0006) [2023-12-27 07:16:25,550][20347] Updated weights for policy 0, policy_version 234444 (0.0008) [2023-12-27 07:16:25,585][20381] Updated weights for policy 1, policy_version 234154 (0.0006) [2023-12-27 07:16:25,604][20347] Updated weights for policy 0, policy_version 234454 (0.0010) [2023-12-27 07:16:25,662][20347] Updated weights for policy 0, policy_version 234464 (0.0011) [2023-12-27 07:16:26,224][20381] Updated weights for policy 1, policy_version 234164 (0.0006) [2023-12-27 07:16:26,272][20381] Updated weights for policy 1, policy_version 234174 (0.0007) [2023-12-27 07:16:26,323][20381] Updated weights for policy 1, policy_version 234184 (0.0007) [2023-12-27 07:16:26,352][20347] Updated weights for policy 0, policy_version 234474 (0.0010) [2023-12-27 07:16:26,403][20347] Updated weights for policy 0, policy_version 234484 (0.0010) [2023-12-27 07:16:26,454][20347] Updated weights for policy 0, policy_version 234494 (0.0010) [2023-12-27 07:16:26,518][20347] Updated weights for policy 0, policy_version 234504 (0.0010) [2023-12-27 07:16:26,999][20381] Updated weights for policy 1, policy_version 234194 (0.0006) [2023-12-27 07:16:27,054][20381] Updated weights for policy 1, policy_version 234204 (0.0005) [2023-12-27 07:16:27,098][20347] Updated weights for policy 0, policy_version 234514 (0.0006) [2023-12-27 07:16:27,110][20381] Updated weights for policy 1, policy_version 234214 (0.0005) [2023-12-27 07:16:27,142][20347] Updated weights for policy 0, policy_version 234524 (0.0005) [2023-12-27 07:16:27,166][20381] Updated weights for policy 1, policy_version 234224 (0.0005) [2023-12-27 07:16:27,187][20347] Updated weights for policy 0, policy_version 234534 (0.0005) [2023-12-27 07:16:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 120029184. Throughput: 0: 10386.7, 1: 10234.3. Samples: 120014176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:27,363][19170] Avg episode reward: [(0, '304.140'), (1, '1000.000')] [2023-12-27 07:16:27,686][20381] Updated weights for policy 1, policy_version 234234 (0.0007) [2023-12-27 07:16:27,744][20381] Updated weights for policy 1, policy_version 234244 (0.0008) [2023-12-27 07:16:27,800][20381] Updated weights for policy 1, policy_version 234254 (0.0008) [2023-12-27 07:16:27,842][20347] Updated weights for policy 0, policy_version 234544 (0.0006) [2023-12-27 07:16:27,890][20347] Updated weights for policy 0, policy_version 234554 (0.0005) [2023-12-27 07:16:27,945][20347] Updated weights for policy 0, policy_version 234564 (0.0008) [2023-12-27 07:16:28,525][20381] Updated weights for policy 1, policy_version 234264 (0.0008) [2023-12-27 07:16:28,571][20347] Updated weights for policy 0, policy_version 234574 (0.0007) [2023-12-27 07:16:28,580][20381] Updated weights for policy 1, policy_version 234274 (0.0007) [2023-12-27 07:16:28,638][20381] Updated weights for policy 1, policy_version 234284 (0.0007) [2023-12-27 07:16:28,639][20347] Updated weights for policy 0, policy_version 234584 (0.0005) [2023-12-27 07:16:28,704][20347] Updated weights for policy 0, policy_version 234594 (0.0005) [2023-12-27 07:16:29,196][20347] Updated weights for policy 0, policy_version 234604 (0.0005) [2023-12-27 07:16:29,253][20347] Updated weights for policy 0, policy_version 234614 (0.0008) [2023-12-27 07:16:29,302][20347] Updated weights for policy 0, policy_version 234624 (0.0006) [2023-12-27 07:16:29,344][20381] Updated weights for policy 1, policy_version 234294 (0.0007) [2023-12-27 07:16:29,396][20381] Updated weights for policy 1, policy_version 234304 (0.0008) [2023-12-27 07:16:29,444][20381] Updated weights for policy 1, policy_version 234314 (0.0006) [2023-12-27 07:16:29,990][20347] Updated weights for policy 0, policy_version 234634 (0.0007) [2023-12-27 07:16:30,054][20347] Updated weights for policy 0, policy_version 234644 (0.0007) [2023-12-27 07:16:30,097][20381] Updated weights for policy 1, policy_version 234324 (0.0008) [2023-12-27 07:16:30,111][20347] Updated weights for policy 0, policy_version 234654 (0.0007) [2023-12-27 07:16:30,151][20381] Updated weights for policy 1, policy_version 234334 (0.0005) [2023-12-27 07:16:30,169][20347] Updated weights for policy 0, policy_version 234664 (0.0008) [2023-12-27 07:16:30,216][20381] Updated weights for policy 1, policy_version 234344 (0.0005) [2023-12-27 07:16:30,810][20347] Updated weights for policy 0, policy_version 234674 (0.0008) [2023-12-27 07:16:30,824][20381] Updated weights for policy 1, policy_version 234354 (0.0006) [2023-12-27 07:16:30,864][20347] Updated weights for policy 0, policy_version 234684 (0.0007) [2023-12-27 07:16:30,874][20381] Updated weights for policy 1, policy_version 234364 (0.0005) [2023-12-27 07:16:30,923][20347] Updated weights for policy 0, policy_version 234694 (0.0007) [2023-12-27 07:16:30,929][20381] Updated weights for policy 1, policy_version 234374 (0.0005) [2023-12-27 07:16:30,982][20381] Updated weights for policy 1, policy_version 234384 (0.0005) [2023-12-27 07:16:31,594][20381] Updated weights for policy 1, policy_version 234394 (0.0005) [2023-12-27 07:16:31,651][20381] Updated weights for policy 1, policy_version 234404 (0.0008) [2023-12-27 07:16:31,696][20347] Updated weights for policy 0, policy_version 234704 (0.0007) [2023-12-27 07:16:31,707][20381] Updated weights for policy 1, policy_version 234414 (0.0006) [2023-12-27 07:16:31,763][20347] Updated weights for policy 0, policy_version 234714 (0.0008) [2023-12-27 07:16:31,822][20347] Updated weights for policy 0, policy_version 234724 (0.0007) [2023-12-27 07:16:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 120127488. Throughput: 0: 10358.8, 1: 10355.6. Samples: 120142060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:32,363][19170] Avg episode reward: [(0, '208.270'), (1, '1000.000')] [2023-12-27 07:16:32,381][20381] Updated weights for policy 1, policy_version 234424 (0.0008) [2023-12-27 07:16:32,435][20381] Updated weights for policy 1, policy_version 234434 (0.0006) [2023-12-27 07:16:32,491][20381] Updated weights for policy 1, policy_version 234444 (0.0007) [2023-12-27 07:16:32,494][20347] Updated weights for policy 0, policy_version 234734 (0.0010) [2023-12-27 07:16:32,542][20347] Updated weights for policy 0, policy_version 234744 (0.0010) [2023-12-27 07:16:32,594][20347] Updated weights for policy 0, policy_version 234754 (0.0010) [2023-12-27 07:16:33,104][20381] Updated weights for policy 1, policy_version 234454 (0.0006) [2023-12-27 07:16:33,161][20381] Updated weights for policy 1, policy_version 234464 (0.0007) [2023-12-27 07:16:33,216][20381] Updated weights for policy 1, policy_version 234474 (0.0009) [2023-12-27 07:16:33,303][20347] Updated weights for policy 0, policy_version 234764 (0.0010) [2023-12-27 07:16:33,357][20347] Updated weights for policy 0, policy_version 234774 (0.0010) [2023-12-27 07:16:33,415][20347] Updated weights for policy 0, policy_version 234784 (0.0010) [2023-12-27 07:16:33,862][20381] Updated weights for policy 1, policy_version 234484 (0.0009) [2023-12-27 07:16:33,924][20381] Updated weights for policy 1, policy_version 234494 (0.0008) [2023-12-27 07:16:33,985][20381] Updated weights for policy 1, policy_version 234504 (0.0008) [2023-12-27 07:16:34,073][20347] Updated weights for policy 0, policy_version 234794 (0.0010) [2023-12-27 07:16:34,126][20347] Updated weights for policy 0, policy_version 234804 (0.0008) [2023-12-27 07:16:34,191][20347] Updated weights for policy 0, policy_version 234814 (0.0006) [2023-12-27 07:16:34,248][20347] Updated weights for policy 0, policy_version 234824 (0.0005) [2023-12-27 07:16:34,661][20381] Updated weights for policy 1, policy_version 234514 (0.0008) [2023-12-27 07:16:34,718][20381] Updated weights for policy 1, policy_version 234524 (0.0008) [2023-12-27 07:16:34,777][20381] Updated weights for policy 1, policy_version 234534 (0.0008) [2023-12-27 07:16:34,833][20381] Updated weights for policy 1, policy_version 234544 (0.0006) [2023-12-27 07:16:34,838][20347] Updated weights for policy 0, policy_version 234834 (0.0006) [2023-12-27 07:16:34,895][20347] Updated weights for policy 0, policy_version 234844 (0.0006) [2023-12-27 07:16:34,943][20347] Updated weights for policy 0, policy_version 234854 (0.0005) [2023-12-27 07:16:35,531][20381] Updated weights for policy 1, policy_version 234554 (0.0008) [2023-12-27 07:16:35,589][20381] Updated weights for policy 1, policy_version 234564 (0.0007) [2023-12-27 07:16:35,604][20347] Updated weights for policy 0, policy_version 234864 (0.0006) [2023-12-27 07:16:35,648][20381] Updated weights for policy 1, policy_version 234574 (0.0007) [2023-12-27 07:16:35,663][20347] Updated weights for policy 0, policy_version 234874 (0.0005) [2023-12-27 07:16:35,725][20347] Updated weights for policy 0, policy_version 234884 (0.0007) [2023-12-27 07:16:36,308][20381] Updated weights for policy 1, policy_version 234584 (0.0008) [2023-12-27 07:16:36,370][20381] Updated weights for policy 1, policy_version 234594 (0.0008) [2023-12-27 07:16:36,415][20347] Updated weights for policy 0, policy_version 234894 (0.0007) [2023-12-27 07:16:36,434][20381] Updated weights for policy 1, policy_version 234604 (0.0006) [2023-12-27 07:16:36,475][20347] Updated weights for policy 0, policy_version 234904 (0.0007) [2023-12-27 07:16:36,530][20347] Updated weights for policy 0, policy_version 234914 (0.0008) [2023-12-27 07:16:37,085][20381] Updated weights for policy 1, policy_version 234614 (0.0009) [2023-12-27 07:16:37,141][20381] Updated weights for policy 1, policy_version 234624 (0.0010) [2023-12-27 07:16:37,193][20381] Updated weights for policy 1, policy_version 234634 (0.0010) [2023-12-27 07:16:37,240][20347] Updated weights for policy 0, policy_version 234924 (0.0007) [2023-12-27 07:16:37,287][20347] Updated weights for policy 0, policy_version 234934 (0.0007) [2023-12-27 07:16:37,331][20347] Updated weights for policy 0, policy_version 234944 (0.0007) [2023-12-27 07:16:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 120233984. Throughput: 0: 10403.7, 1: 10368.3. Samples: 120205104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:37,363][19170] Avg episode reward: [(0, '248.460'), (1, '1000.000')] [2023-12-27 07:16:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000234952_60162048.pth... [2023-12-27 07:16:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000234640_60080128.pth... [2023-12-27 07:16:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000233736_59850752.pth [2023-12-27 07:16:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000233424_59768832.pth [2023-12-27 07:16:37,903][20381] Updated weights for policy 1, policy_version 234644 (0.0011) [2023-12-27 07:16:37,960][20381] Updated weights for policy 1, policy_version 234654 (0.0010) [2023-12-27 07:16:37,999][20347] Updated weights for policy 0, policy_version 234954 (0.0007) [2023-12-27 07:16:38,018][20381] Updated weights for policy 1, policy_version 234664 (0.0007) [2023-12-27 07:16:38,057][20347] Updated weights for policy 0, policy_version 234964 (0.0006) [2023-12-27 07:16:38,111][20347] Updated weights for policy 0, policy_version 234974 (0.0007) [2023-12-27 07:16:38,164][20347] Updated weights for policy 0, policy_version 234984 (0.0005) [2023-12-27 07:16:38,659][20381] Updated weights for policy 1, policy_version 234674 (0.0007) [2023-12-27 07:16:38,715][20381] Updated weights for policy 1, policy_version 234684 (0.0008) [2023-12-27 07:16:38,763][20381] Updated weights for policy 1, policy_version 234694 (0.0009) [2023-12-27 07:16:38,805][20347] Updated weights for policy 0, policy_version 234994 (0.0006) [2023-12-27 07:16:38,821][20381] Updated weights for policy 1, policy_version 234704 (0.0006) [2023-12-27 07:16:38,866][20347] Updated weights for policy 0, policy_version 235004 (0.0007) [2023-12-27 07:16:38,922][20347] Updated weights for policy 0, policy_version 235014 (0.0007) [2023-12-27 07:16:39,545][20381] Updated weights for policy 1, policy_version 234714 (0.0008) [2023-12-27 07:16:39,601][20381] Updated weights for policy 1, policy_version 234724 (0.0008) [2023-12-27 07:16:39,614][20347] Updated weights for policy 0, policy_version 235024 (0.0007) [2023-12-27 07:16:39,658][20381] Updated weights for policy 1, policy_version 234734 (0.0006) [2023-12-27 07:16:39,672][20347] Updated weights for policy 0, policy_version 235034 (0.0006) [2023-12-27 07:16:39,740][20347] Updated weights for policy 0, policy_version 235044 (0.0008) [2023-12-27 07:16:40,364][20381] Updated weights for policy 1, policy_version 234744 (0.0008) [2023-12-27 07:16:40,421][20381] Updated weights for policy 1, policy_version 234754 (0.0007) [2023-12-27 07:16:40,442][20347] Updated weights for policy 0, policy_version 235054 (0.0008) [2023-12-27 07:16:40,475][20381] Updated weights for policy 1, policy_version 234764 (0.0006) [2023-12-27 07:16:40,495][20347] Updated weights for policy 0, policy_version 235064 (0.0006) [2023-12-27 07:16:40,550][20347] Updated weights for policy 0, policy_version 235074 (0.0008) [2023-12-27 07:16:41,130][20381] Updated weights for policy 1, policy_version 234774 (0.0007) [2023-12-27 07:16:41,188][20381] Updated weights for policy 1, policy_version 234784 (0.0008) [2023-12-27 07:16:41,240][20381] Updated weights for policy 1, policy_version 234794 (0.0008) [2023-12-27 07:16:41,281][20347] Updated weights for policy 0, policy_version 235084 (0.0007) [2023-12-27 07:16:41,345][20347] Updated weights for policy 0, policy_version 235094 (0.0008) [2023-12-27 07:16:41,411][20347] Updated weights for policy 0, policy_version 235104 (0.0008) [2023-12-27 07:16:41,970][20381] Updated weights for policy 1, policy_version 234804 (0.0006) [2023-12-27 07:16:42,033][20381] Updated weights for policy 1, policy_version 234814 (0.0008) [2023-12-27 07:16:42,088][20381] Updated weights for policy 1, policy_version 234824 (0.0007) [2023-12-27 07:16:42,128][20347] Updated weights for policy 0, policy_version 235114 (0.0007) [2023-12-27 07:16:42,187][20347] Updated weights for policy 0, policy_version 235124 (0.0006) [2023-12-27 07:16:42,246][20347] Updated weights for policy 0, policy_version 235134 (0.0006) [2023-12-27 07:16:42,302][20347] Updated weights for policy 0, policy_version 235144 (0.0006) [2023-12-27 07:16:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 120340480. Throughput: 0: 10369.3, 1: 10322.7. Samples: 120326196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:42,363][19170] Avg episode reward: [(0, '210.900'), (1, '1000.000')] [2023-12-27 07:16:42,783][20381] Updated weights for policy 1, policy_version 234834 (0.0007) [2023-12-27 07:16:42,840][20381] Updated weights for policy 1, policy_version 234844 (0.0006) [2023-12-27 07:16:42,900][20381] Updated weights for policy 1, policy_version 234854 (0.0008) [2023-12-27 07:16:42,941][20347] Updated weights for policy 0, policy_version 235154 (0.0006) [2023-12-27 07:16:42,953][20381] Updated weights for policy 1, policy_version 234864 (0.0007) [2023-12-27 07:16:42,990][20347] Updated weights for policy 0, policy_version 235164 (0.0007) [2023-12-27 07:16:43,035][20347] Updated weights for policy 0, policy_version 235174 (0.0007) [2023-12-27 07:16:43,665][20381] Updated weights for policy 1, policy_version 234874 (0.0006) [2023-12-27 07:16:43,666][20347] Updated weights for policy 0, policy_version 235184 (0.0008) [2023-12-27 07:16:43,719][20347] Updated weights for policy 0, policy_version 235194 (0.0007) [2023-12-27 07:16:43,720][20381] Updated weights for policy 1, policy_version 234884 (0.0005) [2023-12-27 07:16:43,771][20347] Updated weights for policy 0, policy_version 235204 (0.0008) [2023-12-27 07:16:43,778][20381] Updated weights for policy 1, policy_version 234894 (0.0006) [2023-12-27 07:16:44,360][20381] Updated weights for policy 1, policy_version 234904 (0.0006) [2023-12-27 07:16:44,416][20381] Updated weights for policy 1, policy_version 234914 (0.0005) [2023-12-27 07:16:44,465][20381] Updated weights for policy 1, policy_version 234924 (0.0005) [2023-12-27 07:16:44,500][20347] Updated weights for policy 0, policy_version 235214 (0.0008) [2023-12-27 07:16:44,555][20347] Updated weights for policy 0, policy_version 235224 (0.0007) [2023-12-27 07:16:44,602][20347] Updated weights for policy 0, policy_version 235234 (0.0007) [2023-12-27 07:16:45,155][20381] Updated weights for policy 1, policy_version 234934 (0.0007) [2023-12-27 07:16:45,211][20381] Updated weights for policy 1, policy_version 234944 (0.0008) [2023-12-27 07:16:45,271][20381] Updated weights for policy 1, policy_version 234954 (0.0007) [2023-12-27 07:16:45,287][20347] Updated weights for policy 0, policy_version 235244 (0.0008) [2023-12-27 07:16:45,334][20347] Updated weights for policy 0, policy_version 235254 (0.0007) [2023-12-27 07:16:45,380][20347] Updated weights for policy 0, policy_version 235264 (0.0007) [2023-12-27 07:16:46,008][20381] Updated weights for policy 1, policy_version 234964 (0.0007) [2023-12-27 07:16:46,050][20347] Updated weights for policy 0, policy_version 235274 (0.0008) [2023-12-27 07:16:46,066][20381] Updated weights for policy 1, policy_version 234974 (0.0007) [2023-12-27 07:16:46,108][20347] Updated weights for policy 0, policy_version 235284 (0.0010) [2023-12-27 07:16:46,122][20381] Updated weights for policy 1, policy_version 234984 (0.0008) [2023-12-27 07:16:46,160][20347] Updated weights for policy 0, policy_version 235294 (0.0007) [2023-12-27 07:16:46,210][20347] Updated weights for policy 0, policy_version 235304 (0.0006) [2023-12-27 07:16:46,779][20381] Updated weights for policy 1, policy_version 234994 (0.0008) [2023-12-27 07:16:46,830][20381] Updated weights for policy 1, policy_version 235004 (0.0006) [2023-12-27 07:16:46,851][20347] Updated weights for policy 0, policy_version 235314 (0.0006) [2023-12-27 07:16:46,886][20381] Updated weights for policy 1, policy_version 235014 (0.0005) [2023-12-27 07:16:46,907][20347] Updated weights for policy 0, policy_version 235324 (0.0007) [2023-12-27 07:16:46,944][20381] Updated weights for policy 1, policy_version 235024 (0.0005) [2023-12-27 07:16:46,961][20347] Updated weights for policy 0, policy_version 235334 (0.0006) [2023-12-27 07:16:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 120438784. Throughput: 0: 10367.4, 1: 10303.8. Samples: 120449788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:47,363][19170] Avg episode reward: [(0, '308.590'), (1, '1000.000')] [2023-12-27 07:16:47,470][20381] Updated weights for policy 1, policy_version 235034 (0.0005) [2023-12-27 07:16:47,525][20381] Updated weights for policy 1, policy_version 235044 (0.0007) [2023-12-27 07:16:47,587][20381] Updated weights for policy 1, policy_version 235054 (0.0007) [2023-12-27 07:16:47,592][20347] Updated weights for policy 0, policy_version 235344 (0.0006) [2023-12-27 07:16:47,650][20347] Updated weights for policy 0, policy_version 235354 (0.0007) [2023-12-27 07:16:47,711][20347] Updated weights for policy 0, policy_version 235364 (0.0008) [2023-12-27 07:16:48,285][20381] Updated weights for policy 1, policy_version 235064 (0.0007) [2023-12-27 07:16:48,329][20347] Updated weights for policy 0, policy_version 235374 (0.0007) [2023-12-27 07:16:48,349][20381] Updated weights for policy 1, policy_version 235074 (0.0007) [2023-12-27 07:16:48,392][20347] Updated weights for policy 0, policy_version 235384 (0.0007) [2023-12-27 07:16:48,411][20381] Updated weights for policy 1, policy_version 235084 (0.0007) [2023-12-27 07:16:48,451][20347] Updated weights for policy 0, policy_version 235394 (0.0008) [2023-12-27 07:16:49,098][20381] Updated weights for policy 1, policy_version 235094 (0.0008) [2023-12-27 07:16:49,100][20347] Updated weights for policy 0, policy_version 235404 (0.0007) [2023-12-27 07:16:49,152][20381] Updated weights for policy 1, policy_version 235104 (0.0006) [2023-12-27 07:16:49,152][20347] Updated weights for policy 0, policy_version 235414 (0.0006) [2023-12-27 07:16:49,206][20347] Updated weights for policy 0, policy_version 235424 (0.0007) [2023-12-27 07:16:49,212][20381] Updated weights for policy 1, policy_version 235114 (0.0008) [2023-12-27 07:16:49,885][20381] Updated weights for policy 1, policy_version 235124 (0.0009) [2023-12-27 07:16:49,902][20347] Updated weights for policy 0, policy_version 235434 (0.0007) [2023-12-27 07:16:49,949][20381] Updated weights for policy 1, policy_version 235134 (0.0007) [2023-12-27 07:16:49,969][20347] Updated weights for policy 0, policy_version 235444 (0.0007) [2023-12-27 07:16:50,003][20381] Updated weights for policy 1, policy_version 235144 (0.0006) [2023-12-27 07:16:50,023][20347] Updated weights for policy 0, policy_version 235454 (0.0007) [2023-12-27 07:16:50,074][20347] Updated weights for policy 0, policy_version 235464 (0.0006) [2023-12-27 07:16:50,698][20381] Updated weights for policy 1, policy_version 235154 (0.0007) [2023-12-27 07:16:50,703][20347] Updated weights for policy 0, policy_version 235474 (0.0008) [2023-12-27 07:16:50,753][20347] Updated weights for policy 0, policy_version 235484 (0.0005) [2023-12-27 07:16:50,754][20381] Updated weights for policy 1, policy_version 235164 (0.0006) [2023-12-27 07:16:50,802][20347] Updated weights for policy 0, policy_version 235494 (0.0005) [2023-12-27 07:16:50,812][20381] Updated weights for policy 1, policy_version 235174 (0.0007) [2023-12-27 07:16:50,872][20381] Updated weights for policy 1, policy_version 235184 (0.0008) [2023-12-27 07:16:51,518][20347] Updated weights for policy 0, policy_version 235504 (0.0005) [2023-12-27 07:16:51,570][20347] Updated weights for policy 0, policy_version 235514 (0.0007) [2023-12-27 07:16:51,594][20381] Updated weights for policy 1, policy_version 235194 (0.0007) [2023-12-27 07:16:51,625][20347] Updated weights for policy 0, policy_version 235524 (0.0007) [2023-12-27 07:16:51,658][20381] Updated weights for policy 1, policy_version 235204 (0.0008) [2023-12-27 07:16:51,714][20381] Updated weights for policy 1, policy_version 235214 (0.0006) [2023-12-27 07:16:52,296][20347] Updated weights for policy 0, policy_version 235534 (0.0008) [2023-12-27 07:16:52,357][20347] Updated weights for policy 0, policy_version 235544 (0.0007) [2023-12-27 07:16:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 120537088. Throughput: 0: 10337.0, 1: 10342.8. Samples: 120511864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:52,363][19170] Avg episode reward: [(0, '325.110'), (1, '1000.000')] [2023-12-27 07:16:52,385][20381] Updated weights for policy 1, policy_version 235224 (0.0007) [2023-12-27 07:16:52,418][20347] Updated weights for policy 0, policy_version 235554 (0.0006) [2023-12-27 07:16:52,442][20381] Updated weights for policy 1, policy_version 235234 (0.0007) [2023-12-27 07:16:52,449][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000235560_60317696.pth... [2023-12-27 07:16:52,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000234312_59998208.pth [2023-12-27 07:16:52,498][20381] Updated weights for policy 1, policy_version 235244 (0.0008) [2023-12-27 07:16:52,521][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000235248_60235776.pth... [2023-12-27 07:16:52,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000234032_59924480.pth [2023-12-27 07:16:53,035][20347] Updated weights for policy 0, policy_version 235564 (0.0006) [2023-12-27 07:16:53,093][20347] Updated weights for policy 0, policy_version 235574 (0.0008) [2023-12-27 07:16:53,153][20347] Updated weights for policy 0, policy_version 235584 (0.0008) [2023-12-27 07:16:53,192][20381] Updated weights for policy 1, policy_version 235254 (0.0007) [2023-12-27 07:16:53,242][20381] Updated weights for policy 1, policy_version 235264 (0.0007) [2023-12-27 07:16:53,287][20381] Updated weights for policy 1, policy_version 235274 (0.0007) [2023-12-27 07:16:53,810][20347] Updated weights for policy 0, policy_version 235594 (0.0005) [2023-12-27 07:16:53,865][20347] Updated weights for policy 0, policy_version 235604 (0.0005) [2023-12-27 07:16:53,927][20347] Updated weights for policy 0, policy_version 235614 (0.0005) [2023-12-27 07:16:53,983][20347] Updated weights for policy 0, policy_version 235624 (0.0005) [2023-12-27 07:16:54,022][20381] Updated weights for policy 1, policy_version 235284 (0.0008) [2023-12-27 07:16:54,080][20381] Updated weights for policy 1, policy_version 235294 (0.0008) [2023-12-27 07:16:54,134][20381] Updated weights for policy 1, policy_version 235304 (0.0007) [2023-12-27 07:16:54,621][20347] Updated weights for policy 0, policy_version 235634 (0.0010) [2023-12-27 07:16:54,674][20347] Updated weights for policy 0, policy_version 235644 (0.0008) [2023-12-27 07:16:54,731][20347] Updated weights for policy 0, policy_version 235654 (0.0007) [2023-12-27 07:16:54,809][20381] Updated weights for policy 1, policy_version 235314 (0.0008) [2023-12-27 07:16:54,871][20381] Updated weights for policy 1, policy_version 235324 (0.0005) [2023-12-27 07:16:54,919][20381] Updated weights for policy 1, policy_version 235334 (0.0005) [2023-12-27 07:16:54,977][20381] Updated weights for policy 1, policy_version 235344 (0.0006) [2023-12-27 07:16:55,418][20347] Updated weights for policy 0, policy_version 235664 (0.0006) [2023-12-27 07:16:55,463][20347] Updated weights for policy 0, policy_version 235674 (0.0005) [2023-12-27 07:16:55,518][20347] Updated weights for policy 0, policy_version 235684 (0.0005) [2023-12-27 07:16:55,539][20381] Updated weights for policy 1, policy_version 235354 (0.0007) [2023-12-27 07:16:55,585][20381] Updated weights for policy 1, policy_version 235364 (0.0008) [2023-12-27 07:16:55,635][20381] Updated weights for policy 1, policy_version 235374 (0.0006) [2023-12-27 07:16:56,164][20347] Updated weights for policy 0, policy_version 235694 (0.0006) [2023-12-27 07:16:56,219][20347] Updated weights for policy 0, policy_version 235704 (0.0007) [2023-12-27 07:16:56,246][20381] Updated weights for policy 1, policy_version 235384 (0.0008) [2023-12-27 07:16:56,263][20347] Updated weights for policy 0, policy_version 235714 (0.0007) [2023-12-27 07:16:56,296][20381] Updated weights for policy 1, policy_version 235394 (0.0007) [2023-12-27 07:16:56,351][20381] Updated weights for policy 1, policy_version 235404 (0.0008) [2023-12-27 07:16:56,856][20347] Updated weights for policy 0, policy_version 235724 (0.0005) [2023-12-27 07:16:56,914][20347] Updated weights for policy 0, policy_version 235734 (0.0005) [2023-12-27 07:16:56,970][20347] Updated weights for policy 0, policy_version 235744 (0.0005) [2023-12-27 07:16:57,087][20381] Updated weights for policy 1, policy_version 235414 (0.0008) [2023-12-27 07:16:57,135][20381] Updated weights for policy 1, policy_version 235424 (0.0008) [2023-12-27 07:16:57,178][20381] Updated weights for policy 1, policy_version 235434 (0.0008) [2023-12-27 07:16:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 120651776. Throughput: 0: 10352.1, 1: 10353.1. Samples: 120637772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:16:57,363][19170] Avg episode reward: [(0, '386.540'), (1, '1000.000')] [2023-12-27 07:16:57,494][20347] Updated weights for policy 0, policy_version 235754 (0.0005) [2023-12-27 07:16:57,543][20347] Updated weights for policy 0, policy_version 235764 (0.0006) [2023-12-27 07:16:57,595][20347] Updated weights for policy 0, policy_version 235774 (0.0006) [2023-12-27 07:16:57,643][20347] Updated weights for policy 0, policy_version 235784 (0.0005) [2023-12-27 07:16:57,838][20381] Updated weights for policy 1, policy_version 235444 (0.0007) [2023-12-27 07:16:57,891][20381] Updated weights for policy 1, policy_version 235454 (0.0007) [2023-12-27 07:16:57,948][20381] Updated weights for policy 1, policy_version 235464 (0.0007) [2023-12-27 07:16:58,220][20347] Updated weights for policy 0, policy_version 235794 (0.0010) [2023-12-27 07:16:58,282][20347] Updated weights for policy 0, policy_version 235804 (0.0010) [2023-12-27 07:16:58,340][20347] Updated weights for policy 0, policy_version 235814 (0.0009) [2023-12-27 07:16:58,687][20381] Updated weights for policy 1, policy_version 235474 (0.0008) [2023-12-27 07:16:58,753][20381] Updated weights for policy 1, policy_version 235484 (0.0008) [2023-12-27 07:16:58,816][20381] Updated weights for policy 1, policy_version 235494 (0.0008) [2023-12-27 07:16:58,888][20381] Updated weights for policy 1, policy_version 235504 (0.0008) [2023-12-27 07:16:59,061][20347] Updated weights for policy 0, policy_version 235824 (0.0008) [2023-12-27 07:16:59,112][20347] Updated weights for policy 0, policy_version 235834 (0.0007) [2023-12-27 07:16:59,167][20347] Updated weights for policy 0, policy_version 235844 (0.0007) [2023-12-27 07:16:59,597][20381] Updated weights for policy 1, policy_version 235514 (0.0008) [2023-12-27 07:16:59,652][20381] Updated weights for policy 1, policy_version 235524 (0.0005) [2023-12-27 07:16:59,706][20381] Updated weights for policy 1, policy_version 235534 (0.0005) [2023-12-27 07:16:59,929][20347] Updated weights for policy 0, policy_version 235854 (0.0008) [2023-12-27 07:16:59,994][20347] Updated weights for policy 0, policy_version 235864 (0.0008) [2023-12-27 07:17:00,042][20347] Updated weights for policy 0, policy_version 235874 (0.0007) [2023-12-27 07:17:00,346][20381] Updated weights for policy 1, policy_version 235544 (0.0008) [2023-12-27 07:17:00,391][20381] Updated weights for policy 1, policy_version 235554 (0.0008) [2023-12-27 07:17:00,436][20381] Updated weights for policy 1, policy_version 235564 (0.0007) [2023-12-27 07:17:00,658][20347] Updated weights for policy 0, policy_version 235884 (0.0008) [2023-12-27 07:17:00,722][20347] Updated weights for policy 0, policy_version 235894 (0.0007) [2023-12-27 07:17:00,785][20347] Updated weights for policy 0, policy_version 235904 (0.0008) [2023-12-27 07:17:01,183][20381] Updated weights for policy 1, policy_version 235574 (0.0009) [2023-12-27 07:17:01,241][20381] Updated weights for policy 1, policy_version 235584 (0.0008) [2023-12-27 07:17:01,302][20381] Updated weights for policy 1, policy_version 235594 (0.0006) [2023-12-27 07:17:01,467][20347] Updated weights for policy 0, policy_version 235914 (0.0007) [2023-12-27 07:17:01,518][20347] Updated weights for policy 0, policy_version 235925 (0.0008) [2023-12-27 07:17:01,582][20347] Updated weights for policy 0, policy_version 235935 (0.0006) [2023-12-27 07:17:01,989][20381] Updated weights for policy 1, policy_version 235604 (0.0009) [2023-12-27 07:17:02,041][20381] Updated weights for policy 1, policy_version 235614 (0.0009) [2023-12-27 07:17:02,103][20381] Updated weights for policy 1, policy_version 235624 (0.0007) [2023-12-27 07:17:02,291][20347] Updated weights for policy 0, policy_version 235945 (0.0007) [2023-12-27 07:17:02,341][20347] Updated weights for policy 0, policy_version 235955 (0.0008) [2023-12-27 07:17:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 120750080. Throughput: 0: 10415.4, 1: 10270.1. Samples: 120761348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:17:02,363][19170] Avg episode reward: [(0, '466.390'), (1, '1000.000')] [2023-12-27 07:17:02,403][20347] Updated weights for policy 0, policy_version 235965 (0.0007) [2023-12-27 07:17:02,460][20347] Updated weights for policy 0, policy_version 235975 (0.0007) [2023-12-27 07:17:02,776][20381] Updated weights for policy 1, policy_version 235634 (0.0009) [2023-12-27 07:17:02,828][20381] Updated weights for policy 1, policy_version 235644 (0.0010) [2023-12-27 07:17:02,880][20381] Updated weights for policy 1, policy_version 235654 (0.0010) [2023-12-27 07:17:02,931][20381] Updated weights for policy 1, policy_version 235664 (0.0010) [2023-12-27 07:17:03,043][20347] Updated weights for policy 0, policy_version 235985 (0.0006) [2023-12-27 07:17:03,093][20347] Updated weights for policy 0, policy_version 235995 (0.0007) [2023-12-27 07:17:03,149][20347] Updated weights for policy 0, policy_version 236005 (0.0008) [2023-12-27 07:17:03,534][20381] Updated weights for policy 1, policy_version 235674 (0.0010) [2023-12-27 07:17:03,586][20381] Updated weights for policy 1, policy_version 235684 (0.0010) [2023-12-27 07:17:03,638][20381] Updated weights for policy 1, policy_version 235694 (0.0010) [2023-12-27 07:17:03,747][20347] Updated weights for policy 0, policy_version 236015 (0.0007) [2023-12-27 07:17:03,801][20347] Updated weights for policy 0, policy_version 236025 (0.0005) [2023-12-27 07:17:03,862][20347] Updated weights for policy 0, policy_version 236035 (0.0007) [2023-12-27 07:17:04,330][20381] Updated weights for policy 1, policy_version 235704 (0.0011) [2023-12-27 07:17:04,393][20381] Updated weights for policy 1, policy_version 235714 (0.0011) [2023-12-27 07:17:04,411][20347] Updated weights for policy 0, policy_version 236045 (0.0006) [2023-12-27 07:17:04,443][20381] Updated weights for policy 1, policy_version 235724 (0.0008) [2023-12-27 07:17:04,474][20347] Updated weights for policy 0, policy_version 236055 (0.0008) [2023-12-27 07:17:04,532][20347] Updated weights for policy 0, policy_version 236065 (0.0010) [2023-12-27 07:17:05,070][20381] Updated weights for policy 1, policy_version 235734 (0.0008) [2023-12-27 07:17:05,118][20381] Updated weights for policy 1, policy_version 235744 (0.0010) [2023-12-27 07:17:05,171][20381] Updated weights for policy 1, policy_version 235754 (0.0010) [2023-12-27 07:17:05,185][20347] Updated weights for policy 0, policy_version 236075 (0.0009) [2023-12-27 07:17:05,236][20347] Updated weights for policy 0, policy_version 236085 (0.0010) [2023-12-27 07:17:05,286][20347] Updated weights for policy 0, policy_version 236095 (0.0010) [2023-12-27 07:17:05,872][20381] Updated weights for policy 1, policy_version 235764 (0.0009) [2023-12-27 07:17:05,901][20347] Updated weights for policy 0, policy_version 236105 (0.0010) [2023-12-27 07:17:05,918][20381] Updated weights for policy 1, policy_version 235774 (0.0007) [2023-12-27 07:17:05,950][20347] Updated weights for policy 0, policy_version 236115 (0.0005) [2023-12-27 07:17:05,967][20381] Updated weights for policy 1, policy_version 235784 (0.0005) [2023-12-27 07:17:05,995][20347] Updated weights for policy 0, policy_version 236125 (0.0006) [2023-12-27 07:17:06,042][20347] Updated weights for policy 0, policy_version 236135 (0.0008) [2023-12-27 07:17:06,606][20381] Updated weights for policy 1, policy_version 235794 (0.0005) [2023-12-27 07:17:06,667][20381] Updated weights for policy 1, policy_version 235804 (0.0008) [2023-12-27 07:17:06,730][20381] Updated weights for policy 1, policy_version 235814 (0.0008) [2023-12-27 07:17:06,792][20381] Updated weights for policy 1, policy_version 235824 (0.0007) [2023-12-27 07:17:06,824][20347] Updated weights for policy 0, policy_version 236145 (0.0006) [2023-12-27 07:17:06,878][20347] Updated weights for policy 0, policy_version 236155 (0.0005) [2023-12-27 07:17:06,927][20347] Updated weights for policy 0, policy_version 236165 (0.0007) [2023-12-27 07:17:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20688.2). Total num frames: 120856576. Throughput: 0: 10513.8, 1: 10297.3. Samples: 120827464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:17:07,364][19170] Avg episode reward: [(0, '472.740'), (1, '1000.000')] [2023-12-27 07:17:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000236168_60473344.pth... [2023-12-27 07:17:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000234952_60162048.pth [2023-12-27 07:17:07,400][20381] Updated weights for policy 1, policy_version 235834 (0.0008) [2023-12-27 07:17:07,444][20381] Updated weights for policy 1, policy_version 235844 (0.0008) [2023-12-27 07:17:07,501][20381] Updated weights for policy 1, policy_version 235854 (0.0008) [2023-12-27 07:17:07,513][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000235856_60391424.pth... [2023-12-27 07:17:07,518][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000234640_60080128.pth [2023-12-27 07:17:07,621][20347] Updated weights for policy 0, policy_version 236175 (0.0009) [2023-12-27 07:17:07,673][20347] Updated weights for policy 0, policy_version 236185 (0.0010) [2023-12-27 07:17:07,725][20347] Updated weights for policy 0, policy_version 236195 (0.0010) [2023-12-27 07:17:08,230][20381] Updated weights for policy 1, policy_version 235864 (0.0008) [2023-12-27 07:17:08,284][20381] Updated weights for policy 1, policy_version 235874 (0.0007) [2023-12-27 07:17:08,331][20381] Updated weights for policy 1, policy_version 235884 (0.0008) [2023-12-27 07:17:08,416][20347] Updated weights for policy 0, policy_version 236205 (0.0010) [2023-12-27 07:17:08,462][20347] Updated weights for policy 0, policy_version 236215 (0.0007) [2023-12-27 07:17:08,506][20347] Updated weights for policy 0, policy_version 236225 (0.0007) [2023-12-27 07:17:09,025][20381] Updated weights for policy 1, policy_version 235894 (0.0008) [2023-12-27 07:17:09,082][20381] Updated weights for policy 1, policy_version 235904 (0.0006) [2023-12-27 07:17:09,144][20381] Updated weights for policy 1, policy_version 235914 (0.0008) [2023-12-27 07:17:09,215][20347] Updated weights for policy 0, policy_version 236235 (0.0008) [2023-12-27 07:17:09,280][20347] Updated weights for policy 0, policy_version 236245 (0.0007) [2023-12-27 07:17:09,337][20347] Updated weights for policy 0, policy_version 236255 (0.0008) [2023-12-27 07:17:09,799][20381] Updated weights for policy 1, policy_version 235924 (0.0007) [2023-12-27 07:17:09,863][20381] Updated weights for policy 1, policy_version 235934 (0.0008) [2023-12-27 07:17:09,925][20381] Updated weights for policy 1, policy_version 235944 (0.0008) [2023-12-27 07:17:10,063][20347] Updated weights for policy 0, policy_version 236265 (0.0009) [2023-12-27 07:17:10,119][20347] Updated weights for policy 0, policy_version 236275 (0.0008) [2023-12-27 07:17:10,170][20347] Updated weights for policy 0, policy_version 236285 (0.0010) [2023-12-27 07:17:10,224][20347] Updated weights for policy 0, policy_version 236295 (0.0008) [2023-12-27 07:17:10,552][20381] Updated weights for policy 1, policy_version 235954 (0.0007) [2023-12-27 07:17:10,599][20381] Updated weights for policy 1, policy_version 235964 (0.0008) [2023-12-27 07:17:10,648][20381] Updated weights for policy 1, policy_version 235974 (0.0009) [2023-12-27 07:17:10,698][20381] Updated weights for policy 1, policy_version 235984 (0.0008) [2023-12-27 07:17:10,966][20347] Updated weights for policy 0, policy_version 236305 (0.0008) [2023-12-27 07:17:11,019][20347] Updated weights for policy 0, policy_version 236315 (0.0008) [2023-12-27 07:17:11,078][20347] Updated weights for policy 0, policy_version 236325 (0.0007) [2023-12-27 07:17:11,442][20381] Updated weights for policy 1, policy_version 235994 (0.0008) [2023-12-27 07:17:11,498][20381] Updated weights for policy 1, policy_version 236004 (0.0008) [2023-12-27 07:17:11,548][20381] Updated weights for policy 1, policy_version 236014 (0.0008) [2023-12-27 07:17:11,831][20347] Updated weights for policy 0, policy_version 236335 (0.0007) [2023-12-27 07:17:11,891][20347] Updated weights for policy 0, policy_version 236345 (0.0008) [2023-12-27 07:17:11,952][20347] Updated weights for policy 0, policy_version 236355 (0.0008) [2023-12-27 07:17:12,261][20381] Updated weights for policy 1, policy_version 236024 (0.0006) [2023-12-27 07:17:12,324][20381] Updated weights for policy 1, policy_version 236034 (0.0006) [2023-12-27 07:17:12,364][19170] Fps is (10 sec: 20477.5, 60 sec: 20616.1, 300 sec: 20660.4). Total num frames: 120954880. Throughput: 0: 10468.3, 1: 10309.9. Samples: 120949220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:17:12,364][19170] Avg episode reward: [(0, '571.090'), (1, '1000.000')] [2023-12-27 07:17:12,385][20381] Updated weights for policy 1, policy_version 236044 (0.0007) [2023-12-27 07:17:12,597][20347] Updated weights for policy 0, policy_version 236365 (0.0006) [2023-12-27 07:17:12,652][20347] Updated weights for policy 0, policy_version 236375 (0.0006) [2023-12-27 07:17:12,705][20347] Updated weights for policy 0, policy_version 236385 (0.0007) [2023-12-27 07:17:13,056][20381] Updated weights for policy 1, policy_version 236054 (0.0010) [2023-12-27 07:17:13,117][20381] Updated weights for policy 1, policy_version 236064 (0.0011) [2023-12-27 07:17:13,173][20381] Updated weights for policy 1, policy_version 236074 (0.0011) [2023-12-27 07:17:13,363][20347] Updated weights for policy 0, policy_version 236395 (0.0008) [2023-12-27 07:17:13,414][20347] Updated weights for policy 0, policy_version 236405 (0.0007) [2023-12-27 07:17:13,458][20347] Updated weights for policy 0, policy_version 236415 (0.0007) [2023-12-27 07:17:13,841][20381] Updated weights for policy 1, policy_version 236084 (0.0011) [2023-12-27 07:17:13,900][20381] Updated weights for policy 1, policy_version 236094 (0.0009) [2023-12-27 07:17:13,948][20381] Updated weights for policy 1, policy_version 236104 (0.0010) [2023-12-27 07:17:14,093][20347] Updated weights for policy 0, policy_version 236425 (0.0007) [2023-12-27 07:17:14,145][20347] Updated weights for policy 0, policy_version 236435 (0.0008) [2023-12-27 07:17:14,200][20347] Updated weights for policy 0, policy_version 236445 (0.0007) [2023-12-27 07:17:14,255][20347] Updated weights for policy 0, policy_version 236455 (0.0005) [2023-12-27 07:17:14,607][20381] Updated weights for policy 1, policy_version 236114 (0.0009) [2023-12-27 07:17:14,670][20381] Updated weights for policy 1, policy_version 236124 (0.0006) [2023-12-27 07:17:14,719][20381] Updated weights for policy 1, policy_version 236134 (0.0009) [2023-12-27 07:17:14,780][20381] Updated weights for policy 1, policy_version 236144 (0.0008) [2023-12-27 07:17:14,828][20347] Updated weights for policy 0, policy_version 236465 (0.0007) [2023-12-27 07:17:14,890][20347] Updated weights for policy 0, policy_version 236475 (0.0006) [2023-12-27 07:17:14,956][20347] Updated weights for policy 0, policy_version 236485 (0.0008) [2023-12-27 07:17:15,458][20381] Updated weights for policy 1, policy_version 236154 (0.0011) [2023-12-27 07:17:15,515][20381] Updated weights for policy 1, policy_version 236164 (0.0010) [2023-12-27 07:17:15,564][20381] Updated weights for policy 1, policy_version 236174 (0.0010) [2023-12-27 07:17:15,654][20347] Updated weights for policy 0, policy_version 236495 (0.0007) [2023-12-27 07:17:15,716][20347] Updated weights for policy 0, policy_version 236505 (0.0006) [2023-12-27 07:17:15,766][20347] Updated weights for policy 0, policy_version 236515 (0.0006) [2023-12-27 07:17:16,264][20381] Updated weights for policy 1, policy_version 236184 (0.0011) [2023-12-27 07:17:16,322][20381] Updated weights for policy 1, policy_version 236194 (0.0010) [2023-12-27 07:17:16,385][20381] Updated weights for policy 1, policy_version 236204 (0.0011) [2023-12-27 07:17:16,440][20347] Updated weights for policy 0, policy_version 236525 (0.0009) [2023-12-27 07:17:16,500][20347] Updated weights for policy 0, policy_version 236535 (0.0010) [2023-12-27 07:17:16,559][20347] Updated weights for policy 0, policy_version 236545 (0.0010) [2023-12-27 07:17:17,024][20381] Updated weights for policy 1, policy_version 236214 (0.0011) [2023-12-27 07:17:17,073][20381] Updated weights for policy 1, policy_version 236224 (0.0011) [2023-12-27 07:17:17,131][20381] Updated weights for policy 1, policy_version 236234 (0.0011) [2023-12-27 07:17:17,253][20347] Updated weights for policy 0, policy_version 236555 (0.0010) [2023-12-27 07:17:17,301][20347] Updated weights for policy 0, policy_version 236565 (0.0010) [2023-12-27 07:17:17,349][20347] Updated weights for policy 0, policy_version 236575 (0.0010) [2023-12-27 07:17:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 121061376. Throughput: 0: 10434.5, 1: 10242.2. Samples: 121072512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:17:17,363][19170] Avg episode reward: [(0, '766.660'), (1, '1000.000')] [2023-12-27 07:17:17,827][20381] Updated weights for policy 1, policy_version 236244 (0.0009) [2023-12-27 07:17:17,873][20381] Updated weights for policy 1, policy_version 236254 (0.0005) [2023-12-27 07:17:17,926][20381] Updated weights for policy 1, policy_version 236264 (0.0005) [2023-12-27 07:17:18,004][20347] Updated weights for policy 0, policy_version 236585 (0.0010) [2023-12-27 07:17:18,055][20347] Updated weights for policy 0, policy_version 236595 (0.0006) [2023-12-27 07:17:18,101][20347] Updated weights for policy 0, policy_version 236605 (0.0005) [2023-12-27 07:17:18,158][20347] Updated weights for policy 0, policy_version 236615 (0.0006) [2023-12-27 07:17:18,534][20381] Updated weights for policy 1, policy_version 236274 (0.0006) [2023-12-27 07:17:18,593][20381] Updated weights for policy 1, policy_version 236284 (0.0011) [2023-12-27 07:17:18,649][20381] Updated weights for policy 1, policy_version 236294 (0.0011) [2023-12-27 07:17:18,707][20381] Updated weights for policy 1, policy_version 236304 (0.0010) [2023-12-27 07:17:18,841][20347] Updated weights for policy 0, policy_version 236625 (0.0008) [2023-12-27 07:17:18,896][20347] Updated weights for policy 0, policy_version 236635 (0.0007) [2023-12-27 07:17:18,948][20347] Updated weights for policy 0, policy_version 236645 (0.0008) [2023-12-27 07:17:19,414][20381] Updated weights for policy 1, policy_version 236314 (0.0008) [2023-12-27 07:17:19,477][20381] Updated weights for policy 1, policy_version 236324 (0.0009) [2023-12-27 07:17:19,539][20381] Updated weights for policy 1, policy_version 236334 (0.0008) [2023-12-27 07:17:19,662][20347] Updated weights for policy 0, policy_version 236655 (0.0006) [2023-12-27 07:17:19,712][20347] Updated weights for policy 0, policy_version 236665 (0.0007) [2023-12-27 07:17:19,759][20347] Updated weights for policy 0, policy_version 236675 (0.0007) [2023-12-27 07:17:20,240][20381] Updated weights for policy 1, policy_version 236344 (0.0007) [2023-12-27 07:17:20,302][20381] Updated weights for policy 1, policy_version 236354 (0.0008) [2023-12-27 07:17:20,362][20381] Updated weights for policy 1, policy_version 236364 (0.0011) [2023-12-27 07:17:20,483][20347] Updated weights for policy 0, policy_version 236685 (0.0006) [2023-12-27 07:17:20,545][20347] Updated weights for policy 0, policy_version 236695 (0.0009) [2023-12-27 07:17:20,607][20347] Updated weights for policy 0, policy_version 236705 (0.0008) [2023-12-27 07:17:21,019][20381] Updated weights for policy 1, policy_version 236374 (0.0009) [2023-12-27 07:17:21,075][20381] Updated weights for policy 1, policy_version 236384 (0.0007) [2023-12-27 07:17:21,139][20381] Updated weights for policy 1, policy_version 236394 (0.0009) [2023-12-27 07:17:21,264][20347] Updated weights for policy 0, policy_version 236715 (0.0008) [2023-12-27 07:17:21,310][20347] Updated weights for policy 0, policy_version 236725 (0.0007) [2023-12-27 07:17:21,375][20347] Updated weights for policy 0, policy_version 236735 (0.0008) [2023-12-27 07:17:21,823][20381] Updated weights for policy 1, policy_version 236404 (0.0008) [2023-12-27 07:17:21,879][20381] Updated weights for policy 1, policy_version 236414 (0.0007) [2023-12-27 07:17:21,933][20381] Updated weights for policy 1, policy_version 236424 (0.0008) [2023-12-27 07:17:22,125][20347] Updated weights for policy 0, policy_version 236745 (0.0007) [2023-12-27 07:17:22,174][20347] Updated weights for policy 0, policy_version 236755 (0.0008) [2023-12-27 07:17:22,224][20347] Updated weights for policy 0, policy_version 236765 (0.0008) [2023-12-27 07:17:22,286][20347] Updated weights for policy 0, policy_version 236775 (0.0008) [2023-12-27 07:17:22,362][19170] Fps is (10 sec: 21301.7, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 121167872. Throughput: 0: 10408.9, 1: 10241.9. Samples: 121134396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:17:22,363][19170] Avg episode reward: [(0, '934.310'), (1, '1000.000')] [2023-12-27 07:17:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000236432_60538880.pth... [2023-12-27 07:17:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000236776_60628992.pth... [2023-12-27 07:17:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000235248_60235776.pth [2023-12-27 07:17:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000235560_60317696.pth [2023-12-27 07:17:22,613][20381] Updated weights for policy 1, policy_version 236434 (0.0008) [2023-12-27 07:17:22,668][20381] Updated weights for policy 1, policy_version 236444 (0.0007) [2023-12-27 07:17:22,729][20381] Updated weights for policy 1, policy_version 236454 (0.0007) [2023-12-27 07:17:22,794][20381] Updated weights for policy 1, policy_version 236464 (0.0008) [2023-12-27 07:17:22,992][20347] Updated weights for policy 0, policy_version 236785 (0.0008) [2023-12-27 07:17:23,062][20347] Updated weights for policy 0, policy_version 236795 (0.0008) [2023-12-27 07:17:23,124][20347] Updated weights for policy 0, policy_version 236805 (0.0008) [2023-12-27 07:17:23,510][20381] Updated weights for policy 1, policy_version 236474 (0.0008) [2023-12-27 07:17:23,561][20381] Updated weights for policy 1, policy_version 236484 (0.0007) [2023-12-27 07:17:23,622][20381] Updated weights for policy 1, policy_version 236494 (0.0008) [2023-12-27 07:17:23,741][20347] Updated weights for policy 0, policy_version 236815 (0.0007) [2023-12-27 07:17:23,795][20347] Updated weights for policy 0, policy_version 236825 (0.0007) [2023-12-27 07:17:23,847][20347] Updated weights for policy 0, policy_version 236835 (0.0007) [2023-12-27 07:17:24,352][20381] Updated weights for policy 1, policy_version 236504 (0.0008) [2023-12-27 07:17:24,412][20381] Updated weights for policy 1, policy_version 236514 (0.0008) [2023-12-27 07:17:24,462][20381] Updated weights for policy 1, policy_version 236524 (0.0009) [2023-12-27 07:17:24,496][20347] Updated weights for policy 0, policy_version 236845 (0.0006) [2023-12-27 07:17:24,551][20347] Updated weights for policy 0, policy_version 236855 (0.0006) [2023-12-27 07:17:24,617][20347] Updated weights for policy 0, policy_version 236865 (0.0006) [2023-12-27 07:17:25,147][20347] Updated weights for policy 0, policy_version 236875 (0.0005) [2023-12-27 07:17:25,211][20347] Updated weights for policy 0, policy_version 236885 (0.0006) [2023-12-27 07:17:25,232][20381] Updated weights for policy 1, policy_version 236535 (0.0006) [2023-12-27 07:17:25,268][20347] Updated weights for policy 0, policy_version 236895 (0.0005) [2023-12-27 07:17:25,289][20381] Updated weights for policy 1, policy_version 236545 (0.0006) [2023-12-27 07:17:25,345][20381] Updated weights for policy 1, policy_version 236555 (0.0009) [2023-12-27 07:17:25,888][20347] Updated weights for policy 0, policy_version 236905 (0.0008) [2023-12-27 07:17:25,944][20347] Updated weights for policy 0, policy_version 236915 (0.0008) [2023-12-27 07:17:26,007][20347] Updated weights for policy 0, policy_version 236925 (0.0007) [2023-12-27 07:17:26,056][20381] Updated weights for policy 1, policy_version 236565 (0.0007) [2023-12-27 07:17:26,061][20347] Updated weights for policy 0, policy_version 236935 (0.0006) [2023-12-27 07:17:26,109][20381] Updated weights for policy 1, policy_version 236575 (0.0008) [2023-12-27 07:17:26,163][20381] Updated weights for policy 1, policy_version 236585 (0.0009) [2023-12-27 07:17:26,624][20347] Updated weights for policy 0, policy_version 236945 (0.0008) [2023-12-27 07:17:26,677][20347] Updated weights for policy 0, policy_version 236955 (0.0007) [2023-12-27 07:17:26,731][20347] Updated weights for policy 0, policy_version 236965 (0.0008) [2023-12-27 07:17:26,899][20381] Updated weights for policy 1, policy_version 236595 (0.0009) [2023-12-27 07:17:26,945][20381] Updated weights for policy 1, policy_version 236605 (0.0007) [2023-12-27 07:17:27,000][20381] Updated weights for policy 1, policy_version 236615 (0.0008) [2023-12-27 07:17:27,324][20347] Updated weights for policy 0, policy_version 236975 (0.0008) [2023-12-27 07:17:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 121266176. Throughput: 0: 10464.7, 1: 10189.3. Samples: 121255624. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:27,363][19170] Avg episode reward: [(0, '989.210'), (1, '1000.000')] [2023-12-27 07:17:27,376][20347] Updated weights for policy 0, policy_version 236985 (0.0007) [2023-12-27 07:17:27,429][20347] Updated weights for policy 0, policy_version 236995 (0.0008) [2023-12-27 07:17:27,744][20381] Updated weights for policy 1, policy_version 236625 (0.0008) [2023-12-27 07:17:27,804][20381] Updated weights for policy 1, policy_version 236635 (0.0008) [2023-12-27 07:17:27,861][20381] Updated weights for policy 1, policy_version 236645 (0.0008) [2023-12-27 07:17:27,906][20381] Updated weights for policy 1, policy_version 236655 (0.0009) [2023-12-27 07:17:28,022][20347] Updated weights for policy 0, policy_version 237005 (0.0006) [2023-12-27 07:17:28,085][20347] Updated weights for policy 0, policy_version 237015 (0.0005) [2023-12-27 07:17:28,131][20347] Updated weights for policy 0, policy_version 237025 (0.0006) [2023-12-27 07:17:28,693][20381] Updated weights for policy 1, policy_version 236665 (0.0008) [2023-12-27 07:17:28,694][20347] Updated weights for policy 0, policy_version 237035 (0.0007) [2023-12-27 07:17:28,745][20381] Updated weights for policy 1, policy_version 236675 (0.0007) [2023-12-27 07:17:28,752][20347] Updated weights for policy 0, policy_version 237045 (0.0006) [2023-12-27 07:17:28,791][20381] Updated weights for policy 1, policy_version 236685 (0.0007) [2023-12-27 07:17:28,809][20347] Updated weights for policy 0, policy_version 237055 (0.0005) [2023-12-27 07:17:29,468][20347] Updated weights for policy 0, policy_version 237065 (0.0007) [2023-12-27 07:17:29,480][20381] Updated weights for policy 1, policy_version 236695 (0.0008) [2023-12-27 07:17:29,512][20347] Updated weights for policy 0, policy_version 237075 (0.0006) [2023-12-27 07:17:29,531][20381] Updated weights for policy 1, policy_version 236705 (0.0007) [2023-12-27 07:17:29,558][20347] Updated weights for policy 0, policy_version 237085 (0.0008) [2023-12-27 07:17:29,583][20381] Updated weights for policy 1, policy_version 236715 (0.0006) [2023-12-27 07:17:29,604][20347] Updated weights for policy 0, policy_version 237095 (0.0005) [2023-12-27 07:17:30,280][20381] Updated weights for policy 1, policy_version 236725 (0.0009) [2023-12-27 07:17:30,325][20347] Updated weights for policy 0, policy_version 237105 (0.0008) [2023-12-27 07:17:30,332][20381] Updated weights for policy 1, policy_version 236735 (0.0010) [2023-12-27 07:17:30,377][20381] Updated weights for policy 1, policy_version 236745 (0.0010) [2023-12-27 07:17:30,378][20347] Updated weights for policy 0, policy_version 237115 (0.0006) [2023-12-27 07:17:30,435][20347] Updated weights for policy 0, policy_version 237125 (0.0006) [2023-12-27 07:17:31,079][20347] Updated weights for policy 0, policy_version 237135 (0.0007) [2023-12-27 07:17:31,104][20381] Updated weights for policy 1, policy_version 236755 (0.0009) [2023-12-27 07:17:31,134][20347] Updated weights for policy 0, policy_version 237145 (0.0006) [2023-12-27 07:17:31,168][20381] Updated weights for policy 1, policy_version 236765 (0.0008) [2023-12-27 07:17:31,192][20347] Updated weights for policy 0, policy_version 237155 (0.0007) [2023-12-27 07:17:31,216][20381] Updated weights for policy 1, policy_version 236775 (0.0008) [2023-12-27 07:17:31,876][20347] Updated weights for policy 0, policy_version 237165 (0.0007) [2023-12-27 07:17:31,928][20381] Updated weights for policy 1, policy_version 236785 (0.0008) [2023-12-27 07:17:31,934][20347] Updated weights for policy 0, policy_version 237175 (0.0007) [2023-12-27 07:17:31,982][20381] Updated weights for policy 1, policy_version 236795 (0.0005) [2023-12-27 07:17:31,997][20347] Updated weights for policy 0, policy_version 237185 (0.0007) [2023-12-27 07:17:32,032][20381] Updated weights for policy 1, policy_version 236805 (0.0008) [2023-12-27 07:17:32,087][20381] Updated weights for policy 1, policy_version 236815 (0.0008) [2023-12-27 07:17:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 121372672. Throughput: 0: 10555.3, 1: 10122.6. Samples: 121380292. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:32,363][19170] Avg episode reward: [(0, '998.850'), (1, '1000.000')] [2023-12-27 07:17:32,650][20347] Updated weights for policy 0, policy_version 237195 (0.0008) [2023-12-27 07:17:32,702][20347] Updated weights for policy 0, policy_version 237205 (0.0006) [2023-12-27 07:17:32,727][20381] Updated weights for policy 1, policy_version 236825 (0.0006) [2023-12-27 07:17:32,749][20347] Updated weights for policy 0, policy_version 237215 (0.0005) [2023-12-27 07:17:32,781][20381] Updated weights for policy 1, policy_version 236835 (0.0007) [2023-12-27 07:17:32,836][20381] Updated weights for policy 1, policy_version 236845 (0.0006) [2023-12-27 07:17:33,382][20347] Updated weights for policy 0, policy_version 237225 (0.0007) [2023-12-27 07:17:33,439][20347] Updated weights for policy 0, policy_version 237235 (0.0007) [2023-12-27 07:17:33,495][20347] Updated weights for policy 0, policy_version 237245 (0.0007) [2023-12-27 07:17:33,530][20381] Updated weights for policy 1, policy_version 236855 (0.0007) [2023-12-27 07:17:33,550][20347] Updated weights for policy 0, policy_version 237255 (0.0007) [2023-12-27 07:17:33,586][20381] Updated weights for policy 1, policy_version 236865 (0.0007) [2023-12-27 07:17:33,636][20381] Updated weights for policy 1, policy_version 236875 (0.0008) [2023-12-27 07:17:34,264][20347] Updated weights for policy 0, policy_version 237265 (0.0008) [2023-12-27 07:17:34,292][20381] Updated weights for policy 1, policy_version 236885 (0.0007) [2023-12-27 07:17:34,324][20347] Updated weights for policy 0, policy_version 237275 (0.0007) [2023-12-27 07:17:34,345][20381] Updated weights for policy 1, policy_version 236895 (0.0006) [2023-12-27 07:17:34,380][20347] Updated weights for policy 0, policy_version 237285 (0.0007) [2023-12-27 07:17:34,397][20381] Updated weights for policy 1, policy_version 236905 (0.0005) [2023-12-27 07:17:35,034][20347] Updated weights for policy 0, policy_version 237295 (0.0006) [2023-12-27 07:17:35,079][20347] Updated weights for policy 0, policy_version 237305 (0.0005) [2023-12-27 07:17:35,128][20347] Updated weights for policy 0, policy_version 237315 (0.0005) [2023-12-27 07:17:35,136][20381] Updated weights for policy 1, policy_version 236915 (0.0007) [2023-12-27 07:17:35,195][20381] Updated weights for policy 1, policy_version 236925 (0.0007) [2023-12-27 07:17:35,250][20381] Updated weights for policy 1, policy_version 236935 (0.0006) [2023-12-27 07:17:35,796][20347] Updated weights for policy 0, policy_version 237325 (0.0005) [2023-12-27 07:17:35,856][20347] Updated weights for policy 0, policy_version 237335 (0.0005) [2023-12-27 07:17:35,867][20381] Updated weights for policy 1, policy_version 236945 (0.0005) [2023-12-27 07:17:35,913][20347] Updated weights for policy 0, policy_version 237345 (0.0006) [2023-12-27 07:17:35,925][20381] Updated weights for policy 1, policy_version 236955 (0.0006) [2023-12-27 07:17:35,981][20381] Updated weights for policy 1, policy_version 236965 (0.0005) [2023-12-27 07:17:36,034][20381] Updated weights for policy 1, policy_version 236975 (0.0005) [2023-12-27 07:17:36,606][20347] Updated weights for policy 0, policy_version 237355 (0.0007) [2023-12-27 07:17:36,667][20347] Updated weights for policy 0, policy_version 237365 (0.0007) [2023-12-27 07:17:36,687][20381] Updated weights for policy 1, policy_version 236985 (0.0007) [2023-12-27 07:17:36,724][20347] Updated weights for policy 0, policy_version 237375 (0.0006) [2023-12-27 07:17:36,733][20381] Updated weights for policy 1, policy_version 236995 (0.0006) [2023-12-27 07:17:36,779][20381] Updated weights for policy 1, policy_version 237005 (0.0007) [2023-12-27 07:17:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 121470976. Throughput: 0: 10550.8, 1: 10137.0. Samples: 121442816. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:37,363][19170] Avg episode reward: [(0, '998.930'), (1, '1000.000')] [2023-12-27 07:17:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000237384_60784640.pth... [2023-12-27 07:17:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000237008_60686336.pth... [2023-12-27 07:17:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000236168_60473344.pth [2023-12-27 07:17:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000235856_60391424.pth [2023-12-27 07:17:37,436][20347] Updated weights for policy 0, policy_version 237385 (0.0006) [2023-12-27 07:17:37,466][20381] Updated weights for policy 1, policy_version 237015 (0.0010) [2023-12-27 07:17:37,485][20347] Updated weights for policy 0, policy_version 237395 (0.0009) [2023-12-27 07:17:37,518][20381] Updated weights for policy 1, policy_version 237025 (0.0010) [2023-12-27 07:17:37,540][20347] Updated weights for policy 0, policy_version 237405 (0.0006) [2023-12-27 07:17:37,565][20381] Updated weights for policy 1, policy_version 237035 (0.0006) [2023-12-27 07:17:37,589][20347] Updated weights for policy 0, policy_version 237415 (0.0007) [2023-12-27 07:17:38,239][20381] Updated weights for policy 1, policy_version 237045 (0.0006) [2023-12-27 07:17:38,255][20347] Updated weights for policy 0, policy_version 237425 (0.0008) [2023-12-27 07:17:38,288][20381] Updated weights for policy 1, policy_version 237055 (0.0005) [2023-12-27 07:17:38,314][20347] Updated weights for policy 0, policy_version 237435 (0.0007) [2023-12-27 07:17:38,341][20381] Updated weights for policy 1, policy_version 237065 (0.0007) [2023-12-27 07:17:38,377][20347] Updated weights for policy 0, policy_version 237445 (0.0007) [2023-12-27 07:17:39,008][20347] Updated weights for policy 0, policy_version 237455 (0.0008) [2023-12-27 07:17:39,063][20347] Updated weights for policy 0, policy_version 237465 (0.0007) [2023-12-27 07:17:39,071][20381] Updated weights for policy 1, policy_version 237075 (0.0007) [2023-12-27 07:17:39,121][20381] Updated weights for policy 1, policy_version 237085 (0.0006) [2023-12-27 07:17:39,123][20347] Updated weights for policy 0, policy_version 237475 (0.0006) [2023-12-27 07:17:39,172][20381] Updated weights for policy 1, policy_version 237095 (0.0007) [2023-12-27 07:17:39,791][20381] Updated weights for policy 1, policy_version 237105 (0.0008) [2023-12-27 07:17:39,843][20347] Updated weights for policy 0, policy_version 237485 (0.0007) [2023-12-27 07:17:39,857][20381] Updated weights for policy 1, policy_version 237115 (0.0007) [2023-12-27 07:17:39,899][20347] Updated weights for policy 0, policy_version 237495 (0.0008) [2023-12-27 07:17:39,915][20381] Updated weights for policy 1, policy_version 237125 (0.0007) [2023-12-27 07:17:39,959][20347] Updated weights for policy 0, policy_version 237505 (0.0008) [2023-12-27 07:17:39,971][20381] Updated weights for policy 1, policy_version 237135 (0.0006) [2023-12-27 07:17:40,600][20381] Updated weights for policy 1, policy_version 237145 (0.0005) [2023-12-27 07:17:40,664][20381] Updated weights for policy 1, policy_version 237155 (0.0008) [2023-12-27 07:17:40,676][20347] Updated weights for policy 0, policy_version 237515 (0.0007) [2023-12-27 07:17:40,723][20381] Updated weights for policy 1, policy_version 237165 (0.0007) [2023-12-27 07:17:40,728][20347] Updated weights for policy 0, policy_version 237525 (0.0006) [2023-12-27 07:17:40,787][20347] Updated weights for policy 0, policy_version 237535 (0.0005) [2023-12-27 07:17:41,393][20381] Updated weights for policy 1, policy_version 237175 (0.0009) [2023-12-27 07:17:41,457][20381] Updated weights for policy 1, policy_version 237185 (0.0011) [2023-12-27 07:17:41,507][20347] Updated weights for policy 0, policy_version 237545 (0.0007) [2023-12-27 07:17:41,522][20381] Updated weights for policy 1, policy_version 237195 (0.0010) [2023-12-27 07:17:41,567][20347] Updated weights for policy 0, policy_version 237555 (0.0008) [2023-12-27 07:17:41,632][20347] Updated weights for policy 0, policy_version 237565 (0.0008) [2023-12-27 07:17:41,693][20347] Updated weights for policy 0, policy_version 237575 (0.0008) [2023-12-27 07:17:42,230][20381] Updated weights for policy 1, policy_version 237205 (0.0008) [2023-12-27 07:17:42,291][20381] Updated weights for policy 1, policy_version 237215 (0.0006) [2023-12-27 07:17:42,346][20381] Updated weights for policy 1, policy_version 237225 (0.0007) [2023-12-27 07:17:42,355][20347] Updated weights for policy 0, policy_version 237585 (0.0007) [2023-12-27 07:17:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 121569280. Throughput: 0: 10494.4, 1: 10151.3. Samples: 121566828. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:42,363][19170] Avg episode reward: [(0, '996.150'), (1, '1000.000')] [2023-12-27 07:17:42,418][20347] Updated weights for policy 0, policy_version 237595 (0.0008) [2023-12-27 07:17:42,481][20347] Updated weights for policy 0, policy_version 237605 (0.0008) [2023-12-27 07:17:42,936][20381] Updated weights for policy 1, policy_version 237235 (0.0007) [2023-12-27 07:17:42,982][20381] Updated weights for policy 1, policy_version 237245 (0.0005) [2023-12-27 07:17:43,028][20381] Updated weights for policy 1, policy_version 237255 (0.0005) [2023-12-27 07:17:43,215][20347] Updated weights for policy 0, policy_version 237615 (0.0009) [2023-12-27 07:17:43,260][20347] Updated weights for policy 0, policy_version 237625 (0.0008) [2023-12-27 07:17:43,308][20347] Updated weights for policy 0, policy_version 237635 (0.0010) [2023-12-27 07:17:43,673][20381] Updated weights for policy 1, policy_version 237265 (0.0005) [2023-12-27 07:17:43,724][20381] Updated weights for policy 1, policy_version 237275 (0.0005) [2023-12-27 07:17:43,769][20381] Updated weights for policy 1, policy_version 237285 (0.0005) [2023-12-27 07:17:43,821][20381] Updated weights for policy 1, policy_version 237295 (0.0010) [2023-12-27 07:17:43,994][20347] Updated weights for policy 0, policy_version 237645 (0.0010) [2023-12-27 07:17:44,049][20347] Updated weights for policy 0, policy_version 237655 (0.0010) [2023-12-27 07:17:44,107][20347] Updated weights for policy 0, policy_version 237665 (0.0010) [2023-12-27 07:17:44,475][20381] Updated weights for policy 1, policy_version 237305 (0.0010) [2023-12-27 07:17:44,520][20381] Updated weights for policy 1, policy_version 237315 (0.0010) [2023-12-27 07:17:44,565][20381] Updated weights for policy 1, policy_version 237325 (0.0010) [2023-12-27 07:17:44,664][20347] Updated weights for policy 0, policy_version 237675 (0.0009) [2023-12-27 07:17:44,715][20347] Updated weights for policy 0, policy_version 237685 (0.0005) [2023-12-27 07:17:44,771][20347] Updated weights for policy 0, policy_version 237695 (0.0006) [2023-12-27 07:17:45,258][20381] Updated weights for policy 1, policy_version 237335 (0.0007) [2023-12-27 07:17:45,304][20381] Updated weights for policy 1, policy_version 237345 (0.0006) [2023-12-27 07:17:45,351][20347] Updated weights for policy 0, policy_version 237705 (0.0010) [2023-12-27 07:17:45,358][20381] Updated weights for policy 1, policy_version 237355 (0.0008) [2023-12-27 07:17:45,411][20347] Updated weights for policy 0, policy_version 237715 (0.0007) [2023-12-27 07:17:45,465][20347] Updated weights for policy 0, policy_version 237725 (0.0007) [2023-12-27 07:17:45,523][20347] Updated weights for policy 0, policy_version 237735 (0.0007) [2023-12-27 07:17:46,007][20381] Updated weights for policy 1, policy_version 237365 (0.0006) [2023-12-27 07:17:46,063][20381] Updated weights for policy 1, policy_version 237375 (0.0008) [2023-12-27 07:17:46,127][20381] Updated weights for policy 1, policy_version 237385 (0.0007) [2023-12-27 07:17:46,132][20347] Updated weights for policy 0, policy_version 237745 (0.0006) [2023-12-27 07:17:46,190][20347] Updated weights for policy 0, policy_version 237755 (0.0007) [2023-12-27 07:17:46,241][20347] Updated weights for policy 0, policy_version 237765 (0.0009) [2023-12-27 07:17:46,750][20381] Updated weights for policy 1, policy_version 237395 (0.0005) [2023-12-27 07:17:46,806][20381] Updated weights for policy 1, policy_version 237405 (0.0006) [2023-12-27 07:17:46,858][20381] Updated weights for policy 1, policy_version 237415 (0.0009) [2023-12-27 07:17:46,908][20347] Updated weights for policy 0, policy_version 237775 (0.0006) [2023-12-27 07:17:46,969][20347] Updated weights for policy 0, policy_version 237785 (0.0005) [2023-12-27 07:17:47,020][20347] Updated weights for policy 0, policy_version 237795 (0.0006) [2023-12-27 07:17:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 121683968. Throughput: 0: 10484.6, 1: 10234.1. Samples: 121693692. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:47,363][19170] Avg episode reward: [(0, '997.220'), (1, '1000.000')] [2023-12-27 07:17:47,546][20381] Updated weights for policy 1, policy_version 237425 (0.0007) [2023-12-27 07:17:47,604][20381] Updated weights for policy 1, policy_version 237435 (0.0008) [2023-12-27 07:17:47,638][20347] Updated weights for policy 0, policy_version 237805 (0.0006) [2023-12-27 07:17:47,657][20381] Updated weights for policy 1, policy_version 237445 (0.0008) [2023-12-27 07:17:47,695][20347] Updated weights for policy 0, policy_version 237815 (0.0005) [2023-12-27 07:17:47,717][20381] Updated weights for policy 1, policy_version 237455 (0.0008) [2023-12-27 07:17:47,747][20347] Updated weights for policy 0, policy_version 237825 (0.0006) [2023-12-27 07:17:48,423][20347] Updated weights for policy 0, policy_version 237835 (0.0007) [2023-12-27 07:17:48,447][20381] Updated weights for policy 1, policy_version 237465 (0.0008) [2023-12-27 07:17:48,468][20347] Updated weights for policy 0, policy_version 237845 (0.0006) [2023-12-27 07:17:48,504][20381] Updated weights for policy 1, policy_version 237475 (0.0006) [2023-12-27 07:17:48,514][20347] Updated weights for policy 0, policy_version 237855 (0.0007) [2023-12-27 07:17:48,560][20381] Updated weights for policy 1, policy_version 237485 (0.0006) [2023-12-27 07:17:49,194][20347] Updated weights for policy 0, policy_version 237865 (0.0008) [2023-12-27 07:17:49,244][20381] Updated weights for policy 1, policy_version 237495 (0.0006) [2023-12-27 07:17:49,261][20347] Updated weights for policy 0, policy_version 237875 (0.0008) [2023-12-27 07:17:49,302][20381] Updated weights for policy 1, policy_version 237505 (0.0006) [2023-12-27 07:17:49,316][20347] Updated weights for policy 0, policy_version 237885 (0.0006) [2023-12-27 07:17:49,367][20381] Updated weights for policy 1, policy_version 237515 (0.0008) [2023-12-27 07:17:49,376][20347] Updated weights for policy 0, policy_version 237895 (0.0006) [2023-12-27 07:17:50,067][20381] Updated weights for policy 1, policy_version 237525 (0.0007) [2023-12-27 07:17:50,080][20347] Updated weights for policy 0, policy_version 237905 (0.0007) [2023-12-27 07:17:50,128][20381] Updated weights for policy 1, policy_version 237535 (0.0007) [2023-12-27 07:17:50,136][20347] Updated weights for policy 0, policy_version 237915 (0.0006) [2023-12-27 07:17:50,188][20381] Updated weights for policy 1, policy_version 237545 (0.0008) [2023-12-27 07:17:50,197][20347] Updated weights for policy 0, policy_version 237925 (0.0005) [2023-12-27 07:17:50,845][20381] Updated weights for policy 1, policy_version 237555 (0.0007) [2023-12-27 07:17:50,876][20347] Updated weights for policy 0, policy_version 237935 (0.0006) [2023-12-27 07:17:50,902][20381] Updated weights for policy 1, policy_version 237565 (0.0005) [2023-12-27 07:17:50,926][20347] Updated weights for policy 0, policy_version 237945 (0.0007) [2023-12-27 07:17:50,961][20381] Updated weights for policy 1, policy_version 237575 (0.0005) [2023-12-27 07:17:50,986][20347] Updated weights for policy 0, policy_version 237955 (0.0008) [2023-12-27 07:17:51,669][20381] Updated weights for policy 1, policy_version 237585 (0.0006) [2023-12-27 07:17:51,726][20347] Updated weights for policy 0, policy_version 237965 (0.0008) [2023-12-27 07:17:51,731][20381] Updated weights for policy 1, policy_version 237595 (0.0007) [2023-12-27 07:17:51,779][20347] Updated weights for policy 0, policy_version 237975 (0.0007) [2023-12-27 07:17:51,789][20381] Updated weights for policy 1, policy_version 237605 (0.0006) [2023-12-27 07:17:51,825][20347] Updated weights for policy 0, policy_version 237985 (0.0007) [2023-12-27 07:17:51,849][20381] Updated weights for policy 1, policy_version 237615 (0.0005) [2023-12-27 07:17:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 121782272. Throughput: 0: 10410.1, 1: 10181.0. Samples: 121754056. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:17:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000237616_60841984.pth... [2023-12-27 07:17:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000237992_60940288.pth... [2023-12-27 07:17:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000236432_60538880.pth [2023-12-27 07:17:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000236776_60628992.pth [2023-12-27 07:17:52,477][20381] Updated weights for policy 1, policy_version 237625 (0.0009) [2023-12-27 07:17:52,537][20381] Updated weights for policy 1, policy_version 237635 (0.0008) [2023-12-27 07:17:52,592][20381] Updated weights for policy 1, policy_version 237645 (0.0006) [2023-12-27 07:17:52,604][20347] Updated weights for policy 0, policy_version 237995 (0.0008) [2023-12-27 07:17:52,665][20347] Updated weights for policy 0, policy_version 238005 (0.0010) [2023-12-27 07:17:52,724][20347] Updated weights for policy 0, policy_version 238015 (0.0010) [2023-12-27 07:17:53,316][20381] Updated weights for policy 1, policy_version 237655 (0.0006) [2023-12-27 07:17:53,320][20347] Updated weights for policy 0, policy_version 238025 (0.0010) [2023-12-27 07:17:53,365][20381] Updated weights for policy 1, policy_version 237665 (0.0007) [2023-12-27 07:17:53,381][20347] Updated weights for policy 0, policy_version 238035 (0.0006) [2023-12-27 07:17:53,408][20381] Updated weights for policy 1, policy_version 237675 (0.0007) [2023-12-27 07:17:53,437][20347] Updated weights for policy 0, policy_version 238045 (0.0006) [2023-12-27 07:17:53,491][20347] Updated weights for policy 0, policy_version 238055 (0.0010) [2023-12-27 07:17:54,051][20381] Updated weights for policy 1, policy_version 237685 (0.0006) [2023-12-27 07:17:54,083][20347] Updated weights for policy 0, policy_version 238065 (0.0007) [2023-12-27 07:17:54,107][20381] Updated weights for policy 1, policy_version 237695 (0.0006) [2023-12-27 07:17:54,145][20347] Updated weights for policy 0, policy_version 238075 (0.0008) [2023-12-27 07:17:54,159][20381] Updated weights for policy 1, policy_version 237705 (0.0007) [2023-12-27 07:17:54,194][20347] Updated weights for policy 0, policy_version 238085 (0.0006) [2023-12-27 07:17:54,791][20381] Updated weights for policy 1, policy_version 237715 (0.0008) [2023-12-27 07:17:54,837][20381] Updated weights for policy 1, policy_version 237725 (0.0007) [2023-12-27 07:17:54,852][20347] Updated weights for policy 0, policy_version 238095 (0.0007) [2023-12-27 07:17:54,894][20381] Updated weights for policy 1, policy_version 237735 (0.0008) [2023-12-27 07:17:54,906][20347] Updated weights for policy 0, policy_version 238105 (0.0006) [2023-12-27 07:17:54,960][20347] Updated weights for policy 0, policy_version 238115 (0.0006) [2023-12-27 07:17:55,523][20347] Updated weights for policy 0, policy_version 238125 (0.0005) [2023-12-27 07:17:55,524][20381] Updated weights for policy 1, policy_version 237745 (0.0008) [2023-12-27 07:17:55,572][20347] Updated weights for policy 0, policy_version 238135 (0.0005) [2023-12-27 07:17:55,575][20381] Updated weights for policy 1, policy_version 237755 (0.0007) [2023-12-27 07:17:55,630][20347] Updated weights for policy 0, policy_version 238145 (0.0005) [2023-12-27 07:17:55,631][20381] Updated weights for policy 1, policy_version 237765 (0.0007) [2023-12-27 07:17:55,687][20381] Updated weights for policy 1, policy_version 237775 (0.0008) [2023-12-27 07:17:56,223][20347] Updated weights for policy 0, policy_version 238155 (0.0009) [2023-12-27 07:17:56,274][20347] Updated weights for policy 0, policy_version 238165 (0.0010) [2023-12-27 07:17:56,328][20347] Updated weights for policy 0, policy_version 238175 (0.0010) [2023-12-27 07:17:56,333][20381] Updated weights for policy 1, policy_version 237785 (0.0005) [2023-12-27 07:17:56,391][20381] Updated weights for policy 1, policy_version 237795 (0.0007) [2023-12-27 07:17:56,450][20381] Updated weights for policy 1, policy_version 237805 (0.0008) [2023-12-27 07:17:56,968][20347] Updated weights for policy 0, policy_version 238185 (0.0009) [2023-12-27 07:17:57,034][20347] Updated weights for policy 0, policy_version 238195 (0.0010) [2023-12-27 07:17:57,094][20347] Updated weights for policy 0, policy_version 238205 (0.0008) [2023-12-27 07:17:57,118][20381] Updated weights for policy 1, policy_version 237815 (0.0006) [2023-12-27 07:17:57,154][20347] Updated weights for policy 0, policy_version 238215 (0.0010) [2023-12-27 07:17:57,171][20381] Updated weights for policy 1, policy_version 237825 (0.0007) [2023-12-27 07:17:57,228][20381] Updated weights for policy 1, policy_version 237835 (0.0007) [2023-12-27 07:17:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 121896960. Throughput: 0: 10493.7, 1: 10220.6. Samples: 121881340. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:17:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:17:57,766][20347] Updated weights for policy 0, policy_version 238225 (0.0010) [2023-12-27 07:17:57,830][20347] Updated weights for policy 0, policy_version 238235 (0.0010) [2023-12-27 07:17:57,887][20347] Updated weights for policy 0, policy_version 238245 (0.0008) [2023-12-27 07:17:57,927][20381] Updated weights for policy 1, policy_version 237845 (0.0007) [2023-12-27 07:17:57,985][20381] Updated weights for policy 1, policy_version 237855 (0.0005) [2023-12-27 07:17:58,042][20381] Updated weights for policy 1, policy_version 237865 (0.0007) [2023-12-27 07:17:58,511][20347] Updated weights for policy 0, policy_version 238255 (0.0007) [2023-12-27 07:17:58,577][20347] Updated weights for policy 0, policy_version 238265 (0.0007) [2023-12-27 07:17:58,645][20347] Updated weights for policy 0, policy_version 238275 (0.0007) [2023-12-27 07:17:58,748][20381] Updated weights for policy 1, policy_version 237875 (0.0008) [2023-12-27 07:17:58,810][20381] Updated weights for policy 1, policy_version 237885 (0.0008) [2023-12-27 07:17:58,867][20381] Updated weights for policy 1, policy_version 237895 (0.0010) [2023-12-27 07:17:59,355][20347] Updated weights for policy 0, policy_version 238285 (0.0007) [2023-12-27 07:17:59,419][20347] Updated weights for policy 0, policy_version 238295 (0.0007) [2023-12-27 07:17:59,475][20347] Updated weights for policy 0, policy_version 238305 (0.0007) [2023-12-27 07:17:59,616][20381] Updated weights for policy 1, policy_version 237905 (0.0010) [2023-12-27 07:17:59,680][20381] Updated weights for policy 1, policy_version 237915 (0.0011) [2023-12-27 07:17:59,740][20381] Updated weights for policy 1, policy_version 237925 (0.0008) [2023-12-27 07:17:59,799][20381] Updated weights for policy 1, policy_version 237935 (0.0010) [2023-12-27 07:18:00,150][20347] Updated weights for policy 0, policy_version 238316 (0.0007) [2023-12-27 07:18:00,204][20347] Updated weights for policy 0, policy_version 238326 (0.0008) [2023-12-27 07:18:00,260][20347] Updated weights for policy 0, policy_version 238336 (0.0005) [2023-12-27 07:18:00,434][20381] Updated weights for policy 1, policy_version 237945 (0.0011) [2023-12-27 07:18:00,478][20381] Updated weights for policy 1, policy_version 237955 (0.0005) [2023-12-27 07:18:00,527][20381] Updated weights for policy 1, policy_version 237965 (0.0005) [2023-12-27 07:18:00,820][20347] Updated weights for policy 0, policy_version 238346 (0.0005) [2023-12-27 07:18:00,871][20347] Updated weights for policy 0, policy_version 238356 (0.0005) [2023-12-27 07:18:00,916][20347] Updated weights for policy 0, policy_version 238366 (0.0005) [2023-12-27 07:18:00,963][20347] Updated weights for policy 0, policy_version 238376 (0.0008) [2023-12-27 07:18:01,171][20381] Updated weights for policy 1, policy_version 237975 (0.0007) [2023-12-27 07:18:01,230][20381] Updated weights for policy 1, policy_version 237985 (0.0007) [2023-12-27 07:18:01,288][20381] Updated weights for policy 1, policy_version 237995 (0.0008) [2023-12-27 07:18:01,602][20347] Updated weights for policy 0, policy_version 238386 (0.0010) [2023-12-27 07:18:01,663][20347] Updated weights for policy 0, policy_version 238396 (0.0010) [2023-12-27 07:18:01,718][20347] Updated weights for policy 0, policy_version 238406 (0.0010) [2023-12-27 07:18:01,955][20381] Updated weights for policy 1, policy_version 238005 (0.0007) [2023-12-27 07:18:02,007][20381] Updated weights for policy 1, policy_version 238015 (0.0005) [2023-12-27 07:18:02,061][20381] Updated weights for policy 1, policy_version 238025 (0.0006) [2023-12-27 07:18:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 121995264. Throughput: 0: 10517.2, 1: 10229.9. Samples: 122006132. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:02,422][20347] Updated weights for policy 0, policy_version 238416 (0.0010) [2023-12-27 07:18:02,482][20347] Updated weights for policy 0, policy_version 238426 (0.0010) [2023-12-27 07:18:02,542][20347] Updated weights for policy 0, policy_version 238436 (0.0008) [2023-12-27 07:18:02,746][20381] Updated weights for policy 1, policy_version 238035 (0.0008) [2023-12-27 07:18:02,808][20381] Updated weights for policy 1, policy_version 238045 (0.0008) [2023-12-27 07:18:02,868][20381] Updated weights for policy 1, policy_version 238055 (0.0008) [2023-12-27 07:18:03,202][20347] Updated weights for policy 0, policy_version 238446 (0.0007) [2023-12-27 07:18:03,257][20347] Updated weights for policy 0, policy_version 238456 (0.0008) [2023-12-27 07:18:03,307][20347] Updated weights for policy 0, policy_version 238466 (0.0007) [2023-12-27 07:18:03,522][20381] Updated weights for policy 1, policy_version 238065 (0.0007) [2023-12-27 07:18:03,564][20381] Updated weights for policy 1, policy_version 238075 (0.0005) [2023-12-27 07:18:03,607][20381] Updated weights for policy 1, policy_version 238085 (0.0005) [2023-12-27 07:18:03,660][20381] Updated weights for policy 1, policy_version 238095 (0.0005) [2023-12-27 07:18:04,019][20347] Updated weights for policy 0, policy_version 238476 (0.0007) [2023-12-27 07:18:04,078][20347] Updated weights for policy 0, policy_version 238486 (0.0007) [2023-12-27 07:18:04,133][20347] Updated weights for policy 0, policy_version 238496 (0.0009) [2023-12-27 07:18:04,223][20381] Updated weights for policy 1, policy_version 238105 (0.0008) [2023-12-27 07:18:04,276][20381] Updated weights for policy 1, policy_version 238115 (0.0008) [2023-12-27 07:18:04,330][20381] Updated weights for policy 1, policy_version 238125 (0.0008) [2023-12-27 07:18:04,786][20347] Updated weights for policy 0, policy_version 238506 (0.0008) [2023-12-27 07:18:04,844][20347] Updated weights for policy 0, policy_version 238516 (0.0005) [2023-12-27 07:18:04,908][20347] Updated weights for policy 0, policy_version 238526 (0.0006) [2023-12-27 07:18:04,959][20347] Updated weights for policy 0, policy_version 238536 (0.0006) [2023-12-27 07:18:05,001][20381] Updated weights for policy 1, policy_version 238135 (0.0008) [2023-12-27 07:18:05,063][20381] Updated weights for policy 1, policy_version 238145 (0.0007) [2023-12-27 07:18:05,123][20381] Updated weights for policy 1, policy_version 238155 (0.0006) [2023-12-27 07:18:05,553][20347] Updated weights for policy 0, policy_version 238546 (0.0009) [2023-12-27 07:18:05,604][20347] Updated weights for policy 0, policy_version 238556 (0.0010) [2023-12-27 07:18:05,653][20347] Updated weights for policy 0, policy_version 238566 (0.0010) [2023-12-27 07:18:05,719][20381] Updated weights for policy 1, policy_version 238165 (0.0007) [2023-12-27 07:18:05,778][20381] Updated weights for policy 1, policy_version 238175 (0.0006) [2023-12-27 07:18:05,840][20381] Updated weights for policy 1, policy_version 238185 (0.0005) [2023-12-27 07:18:06,315][20347] Updated weights for policy 0, policy_version 238576 (0.0006) [2023-12-27 07:18:06,371][20347] Updated weights for policy 0, policy_version 238586 (0.0010) [2023-12-27 07:18:06,428][20347] Updated weights for policy 0, policy_version 238596 (0.0011) [2023-12-27 07:18:06,464][20381] Updated weights for policy 1, policy_version 238195 (0.0006) [2023-12-27 07:18:06,526][20381] Updated weights for policy 1, policy_version 238205 (0.0008) [2023-12-27 07:18:06,588][20381] Updated weights for policy 1, policy_version 238215 (0.0008) [2023-12-27 07:18:07,081][20347] Updated weights for policy 0, policy_version 238606 (0.0009) [2023-12-27 07:18:07,130][20347] Updated weights for policy 0, policy_version 238616 (0.0008) [2023-12-27 07:18:07,179][20347] Updated weights for policy 0, policy_version 238626 (0.0007) [2023-12-27 07:18:07,263][20381] Updated weights for policy 1, policy_version 238225 (0.0008) [2023-12-27 07:18:07,325][20381] Updated weights for policy 1, policy_version 238235 (0.0006) [2023-12-27 07:18:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.2, 300 sec: 20660.5). Total num frames: 122101760. Throughput: 0: 10522.6, 1: 10253.8. Samples: 122069332. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000238632_61104128.pth... [2023-12-27 07:18:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000237384_60784640.pth [2023-12-27 07:18:07,373][20381] Updated weights for policy 1, policy_version 238245 (0.0006) [2023-12-27 07:18:07,418][20381] Updated weights for policy 1, policy_version 238255 (0.0007) [2023-12-27 07:18:07,421][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000238256_61005824.pth... [2023-12-27 07:18:07,424][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000237008_60686336.pth [2023-12-27 07:18:07,899][20347] Updated weights for policy 0, policy_version 238636 (0.0008) [2023-12-27 07:18:07,957][20347] Updated weights for policy 0, policy_version 238646 (0.0007) [2023-12-27 07:18:08,014][20347] Updated weights for policy 0, policy_version 238656 (0.0006) [2023-12-27 07:18:08,094][20381] Updated weights for policy 1, policy_version 238265 (0.0006) [2023-12-27 07:18:08,150][20381] Updated weights for policy 1, policy_version 238275 (0.0008) [2023-12-27 07:18:08,204][20381] Updated weights for policy 1, policy_version 238285 (0.0008) [2023-12-27 07:18:08,634][20347] Updated weights for policy 0, policy_version 238666 (0.0006) [2023-12-27 07:18:08,691][20347] Updated weights for policy 0, policy_version 238676 (0.0005) [2023-12-27 07:18:08,745][20347] Updated weights for policy 0, policy_version 238686 (0.0005) [2023-12-27 07:18:08,791][20347] Updated weights for policy 0, policy_version 238696 (0.0005) [2023-12-27 07:18:08,928][20381] Updated weights for policy 1, policy_version 238295 (0.0007) [2023-12-27 07:18:08,977][20381] Updated weights for policy 1, policy_version 238305 (0.0008) [2023-12-27 07:18:09,032][20381] Updated weights for policy 1, policy_version 238315 (0.0007) [2023-12-27 07:18:09,451][20347] Updated weights for policy 0, policy_version 238706 (0.0009) [2023-12-27 07:18:09,510][20347] Updated weights for policy 0, policy_version 238716 (0.0007) [2023-12-27 07:18:09,568][20347] Updated weights for policy 0, policy_version 238726 (0.0007) [2023-12-27 07:18:09,735][20381] Updated weights for policy 1, policy_version 238325 (0.0007) [2023-12-27 07:18:09,788][20381] Updated weights for policy 1, policy_version 238335 (0.0006) [2023-12-27 07:18:09,847][20381] Updated weights for policy 1, policy_version 238345 (0.0006) [2023-12-27 07:18:10,250][20347] Updated weights for policy 0, policy_version 238736 (0.0007) [2023-12-27 07:18:10,307][20347] Updated weights for policy 0, policy_version 238746 (0.0006) [2023-12-27 07:18:10,370][20347] Updated weights for policy 0, policy_version 238756 (0.0008) [2023-12-27 07:18:10,556][20381] Updated weights for policy 1, policy_version 238355 (0.0006) [2023-12-27 07:18:10,610][20381] Updated weights for policy 1, policy_version 238365 (0.0006) [2023-12-27 07:18:10,660][20381] Updated weights for policy 1, policy_version 238375 (0.0008) [2023-12-27 07:18:10,971][20347] Updated weights for policy 0, policy_version 238766 (0.0007) [2023-12-27 07:18:11,026][20347] Updated weights for policy 0, policy_version 238776 (0.0007) [2023-12-27 07:18:11,088][20347] Updated weights for policy 0, policy_version 238786 (0.0007) [2023-12-27 07:18:11,354][20381] Updated weights for policy 1, policy_version 238385 (0.0007) [2023-12-27 07:18:11,416][20381] Updated weights for policy 1, policy_version 238395 (0.0006) [2023-12-27 07:18:11,477][20381] Updated weights for policy 1, policy_version 238405 (0.0007) [2023-12-27 07:18:11,539][20381] Updated weights for policy 1, policy_version 238415 (0.0007) [2023-12-27 07:18:11,820][20347] Updated weights for policy 0, policy_version 238796 (0.0008) [2023-12-27 07:18:11,878][20347] Updated weights for policy 0, policy_version 238806 (0.0005) [2023-12-27 07:18:11,932][20347] Updated weights for policy 0, policy_version 238816 (0.0005) [2023-12-27 07:18:12,265][20381] Updated weights for policy 1, policy_version 238425 (0.0007) [2023-12-27 07:18:12,329][20381] Updated weights for policy 1, policy_version 238435 (0.0007) [2023-12-27 07:18:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.5, 300 sec: 20660.5). Total num frames: 122200064. Throughput: 0: 10536.8, 1: 10327.5. Samples: 122194516. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:12,389][20381] Updated weights for policy 1, policy_version 238445 (0.0007) [2023-12-27 07:18:12,599][20347] Updated weights for policy 0, policy_version 238826 (0.0006) [2023-12-27 07:18:12,660][20347] Updated weights for policy 0, policy_version 238836 (0.0007) [2023-12-27 07:18:12,722][20347] Updated weights for policy 0, policy_version 238846 (0.0007) [2023-12-27 07:18:12,775][20347] Updated weights for policy 0, policy_version 238856 (0.0007) [2023-12-27 07:18:13,119][20381] Updated weights for policy 1, policy_version 238455 (0.0007) [2023-12-27 07:18:13,185][20381] Updated weights for policy 1, policy_version 238465 (0.0008) [2023-12-27 07:18:13,243][20381] Updated weights for policy 1, policy_version 238475 (0.0008) [2023-12-27 07:18:13,391][20347] Updated weights for policy 0, policy_version 238866 (0.0007) [2023-12-27 07:18:13,440][20347] Updated weights for policy 0, policy_version 238876 (0.0006) [2023-12-27 07:18:13,486][20347] Updated weights for policy 0, policy_version 238886 (0.0007) [2023-12-27 07:18:13,937][20381] Updated weights for policy 1, policy_version 238485 (0.0008) [2023-12-27 07:18:13,998][20381] Updated weights for policy 1, policy_version 238495 (0.0007) [2023-12-27 07:18:14,052][20381] Updated weights for policy 1, policy_version 238505 (0.0008) [2023-12-27 07:18:14,166][20347] Updated weights for policy 0, policy_version 238896 (0.0008) [2023-12-27 07:18:14,215][20347] Updated weights for policy 0, policy_version 238906 (0.0008) [2023-12-27 07:18:14,267][20347] Updated weights for policy 0, policy_version 238916 (0.0008) [2023-12-27 07:18:14,683][20381] Updated weights for policy 1, policy_version 238515 (0.0007) [2023-12-27 07:18:14,736][20381] Updated weights for policy 1, policy_version 238525 (0.0008) [2023-12-27 07:18:14,799][20381] Updated weights for policy 1, policy_version 238535 (0.0008) [2023-12-27 07:18:14,893][20347] Updated weights for policy 0, policy_version 238926 (0.0007) [2023-12-27 07:18:14,957][20347] Updated weights for policy 0, policy_version 238936 (0.0006) [2023-12-27 07:18:15,012][20347] Updated weights for policy 0, policy_version 238946 (0.0008) [2023-12-27 07:18:15,497][20381] Updated weights for policy 1, policy_version 238545 (0.0010) [2023-12-27 07:18:15,545][20381] Updated weights for policy 1, policy_version 238555 (0.0006) [2023-12-27 07:18:15,575][20347] Updated weights for policy 0, policy_version 238956 (0.0007) [2023-12-27 07:18:15,599][20381] Updated weights for policy 1, policy_version 238565 (0.0005) [2023-12-27 07:18:15,638][20347] Updated weights for policy 0, policy_version 238966 (0.0008) [2023-12-27 07:18:15,654][20381] Updated weights for policy 1, policy_version 238575 (0.0005) [2023-12-27 07:18:15,690][20347] Updated weights for policy 0, policy_version 238976 (0.0009) [2023-12-27 07:18:16,204][20381] Updated weights for policy 1, policy_version 238585 (0.0005) [2023-12-27 07:18:16,264][20381] Updated weights for policy 1, policy_version 238595 (0.0006) [2023-12-27 07:18:16,312][20381] Updated weights for policy 1, policy_version 238605 (0.0005) [2023-12-27 07:18:16,381][20347] Updated weights for policy 0, policy_version 238986 (0.0007) [2023-12-27 07:18:16,446][20347] Updated weights for policy 0, policy_version 238996 (0.0006) [2023-12-27 07:18:16,495][20347] Updated weights for policy 0, policy_version 239006 (0.0008) [2023-12-27 07:18:16,540][20347] Updated weights for policy 0, policy_version 239016 (0.0010) [2023-12-27 07:18:16,820][20381] Updated weights for policy 1, policy_version 238615 (0.0005) [2023-12-27 07:18:16,879][20381] Updated weights for policy 1, policy_version 238625 (0.0005) [2023-12-27 07:18:16,932][20381] Updated weights for policy 1, policy_version 238635 (0.0005) [2023-12-27 07:18:17,184][20347] Updated weights for policy 0, policy_version 239026 (0.0010) [2023-12-27 07:18:17,236][20347] Updated weights for policy 0, policy_version 239036 (0.0009) [2023-12-27 07:18:17,295][20347] Updated weights for policy 0, policy_version 239046 (0.0010) [2023-12-27 07:18:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 122314752. Throughput: 0: 10485.2, 1: 10460.8. Samples: 122322864. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:17,479][20381] Updated weights for policy 1, policy_version 238645 (0.0006) [2023-12-27 07:18:17,539][20381] Updated weights for policy 1, policy_version 238655 (0.0005) [2023-12-27 07:18:17,595][20381] Updated weights for policy 1, policy_version 238665 (0.0005) [2023-12-27 07:18:17,897][20347] Updated weights for policy 0, policy_version 239056 (0.0006) [2023-12-27 07:18:17,949][20347] Updated weights for policy 0, policy_version 239066 (0.0005) [2023-12-27 07:18:18,008][20347] Updated weights for policy 0, policy_version 239076 (0.0005) [2023-12-27 07:18:18,223][20381] Updated weights for policy 1, policy_version 238675 (0.0006) [2023-12-27 07:18:18,283][20381] Updated weights for policy 1, policy_version 238685 (0.0008) [2023-12-27 07:18:18,347][20381] Updated weights for policy 1, policy_version 238695 (0.0007) [2023-12-27 07:18:18,637][20347] Updated weights for policy 0, policy_version 239086 (0.0008) [2023-12-27 07:18:18,699][20347] Updated weights for policy 0, policy_version 239096 (0.0010) [2023-12-27 07:18:18,764][20347] Updated weights for policy 0, policy_version 239106 (0.0010) [2023-12-27 07:18:19,070][20381] Updated weights for policy 1, policy_version 238705 (0.0007) [2023-12-27 07:18:19,119][20381] Updated weights for policy 1, policy_version 238715 (0.0007) [2023-12-27 07:18:19,173][20381] Updated weights for policy 1, policy_version 238725 (0.0007) [2023-12-27 07:18:19,246][20381] Updated weights for policy 1, policy_version 238735 (0.0007) [2023-12-27 07:18:19,445][20347] Updated weights for policy 0, policy_version 239116 (0.0010) [2023-12-27 07:18:19,511][20347] Updated weights for policy 0, policy_version 239126 (0.0010) [2023-12-27 07:18:19,571][20347] Updated weights for policy 0, policy_version 239136 (0.0008) [2023-12-27 07:18:19,874][20381] Updated weights for policy 1, policy_version 238745 (0.0007) [2023-12-27 07:18:19,938][20381] Updated weights for policy 1, policy_version 238755 (0.0007) [2023-12-27 07:18:19,998][20381] Updated weights for policy 1, policy_version 238765 (0.0008) [2023-12-27 07:18:20,284][20347] Updated weights for policy 0, policy_version 239146 (0.0008) [2023-12-27 07:18:20,347][20347] Updated weights for policy 0, policy_version 239156 (0.0008) [2023-12-27 07:18:20,408][20347] Updated weights for policy 0, policy_version 239166 (0.0009) [2023-12-27 07:18:20,471][20347] Updated weights for policy 0, policy_version 239176 (0.0008) [2023-12-27 07:18:20,689][20381] Updated weights for policy 1, policy_version 238775 (0.0007) [2023-12-27 07:18:20,751][20381] Updated weights for policy 1, policy_version 238785 (0.0006) [2023-12-27 07:18:20,814][20381] Updated weights for policy 1, policy_version 238795 (0.0005) [2023-12-27 07:18:21,219][20347] Updated weights for policy 0, policy_version 239186 (0.0008) [2023-12-27 07:18:21,281][20347] Updated weights for policy 0, policy_version 239196 (0.0008) [2023-12-27 07:18:21,339][20347] Updated weights for policy 0, policy_version 239206 (0.0008) [2023-12-27 07:18:21,474][20381] Updated weights for policy 1, policy_version 238805 (0.0007) [2023-12-27 07:18:21,527][20381] Updated weights for policy 1, policy_version 238815 (0.0008) [2023-12-27 07:18:21,586][20381] Updated weights for policy 1, policy_version 238825 (0.0008) [2023-12-27 07:18:22,065][20347] Updated weights for policy 0, policy_version 239216 (0.0007) [2023-12-27 07:18:22,119][20347] Updated weights for policy 0, policy_version 239226 (0.0007) [2023-12-27 07:18:22,177][20347] Updated weights for policy 0, policy_version 239236 (0.0008) [2023-12-27 07:18:22,252][20381] Updated weights for policy 1, policy_version 238835 (0.0007) [2023-12-27 07:18:22,316][20381] Updated weights for policy 1, policy_version 238845 (0.0008) [2023-12-27 07:18:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 122413056. Throughput: 0: 10485.4, 1: 10462.2. Samples: 122385460. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000239240_61259776.pth... [2023-12-27 07:18:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000237992_60940288.pth [2023-12-27 07:18:22,380][20381] Updated weights for policy 1, policy_version 238855 (0.0008) [2023-12-27 07:18:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000238864_61161472.pth... [2023-12-27 07:18:22,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000237616_60841984.pth [2023-12-27 07:18:22,893][20347] Updated weights for policy 0, policy_version 239246 (0.0008) [2023-12-27 07:18:22,942][20347] Updated weights for policy 0, policy_version 239256 (0.0008) [2023-12-27 07:18:22,990][20347] Updated weights for policy 0, policy_version 239266 (0.0008) [2023-12-27 07:18:23,089][20381] Updated weights for policy 1, policy_version 238865 (0.0008) [2023-12-27 07:18:23,138][20381] Updated weights for policy 1, policy_version 238875 (0.0008) [2023-12-27 07:18:23,200][20381] Updated weights for policy 1, policy_version 238885 (0.0008) [2023-12-27 07:18:23,243][20381] Updated weights for policy 1, policy_version 238895 (0.0007) [2023-12-27 07:18:23,651][20347] Updated weights for policy 0, policy_version 239276 (0.0007) [2023-12-27 07:18:23,700][20347] Updated weights for policy 0, policy_version 239286 (0.0006) [2023-12-27 07:18:23,754][20347] Updated weights for policy 0, policy_version 239296 (0.0008) [2023-12-27 07:18:23,874][20381] Updated weights for policy 1, policy_version 238905 (0.0005) [2023-12-27 07:18:23,923][20381] Updated weights for policy 1, policy_version 238915 (0.0005) [2023-12-27 07:18:23,985][20381] Updated weights for policy 1, policy_version 238925 (0.0006) [2023-12-27 07:18:24,452][20347] Updated weights for policy 0, policy_version 239306 (0.0008) [2023-12-27 07:18:24,503][20347] Updated weights for policy 0, policy_version 239316 (0.0007) [2023-12-27 07:18:24,557][20347] Updated weights for policy 0, policy_version 239326 (0.0007) [2023-12-27 07:18:24,568][20381] Updated weights for policy 1, policy_version 238935 (0.0007) [2023-12-27 07:18:24,609][20347] Updated weights for policy 0, policy_version 239336 (0.0005) [2023-12-27 07:18:24,630][20381] Updated weights for policy 1, policy_version 238945 (0.0007) [2023-12-27 07:18:24,684][20381] Updated weights for policy 1, policy_version 238955 (0.0007) [2023-12-27 07:18:25,194][20347] Updated weights for policy 0, policy_version 239346 (0.0005) [2023-12-27 07:18:25,249][20347] Updated weights for policy 0, policy_version 239356 (0.0006) [2023-12-27 07:18:25,302][20381] Updated weights for policy 1, policy_version 238965 (0.0009) [2023-12-27 07:18:25,306][20347] Updated weights for policy 0, policy_version 239366 (0.0005) [2023-12-27 07:18:25,361][20381] Updated weights for policy 1, policy_version 238975 (0.0008) [2023-12-27 07:18:25,428][20381] Updated weights for policy 1, policy_version 238985 (0.0007) [2023-12-27 07:18:25,842][20347] Updated weights for policy 0, policy_version 239376 (0.0006) [2023-12-27 07:18:25,886][20347] Updated weights for policy 0, policy_version 239386 (0.0005) [2023-12-27 07:18:25,935][20347] Updated weights for policy 0, policy_version 239396 (0.0005) [2023-12-27 07:18:26,053][20381] Updated weights for policy 1, policy_version 238995 (0.0007) [2023-12-27 07:18:26,110][20381] Updated weights for policy 1, policy_version 239005 (0.0006) [2023-12-27 07:18:26,169][20381] Updated weights for policy 1, policy_version 239015 (0.0005) [2023-12-27 07:18:26,552][20347] Updated weights for policy 0, policy_version 239406 (0.0007) [2023-12-27 07:18:26,599][20347] Updated weights for policy 0, policy_version 239416 (0.0007) [2023-12-27 07:18:26,648][20347] Updated weights for policy 0, policy_version 239426 (0.0008) [2023-12-27 07:18:26,699][20381] Updated weights for policy 1, policy_version 239025 (0.0005) [2023-12-27 07:18:26,754][20381] Updated weights for policy 1, policy_version 239035 (0.0005) [2023-12-27 07:18:26,810][20381] Updated weights for policy 1, policy_version 239045 (0.0005) [2023-12-27 07:18:26,868][20381] Updated weights for policy 1, policy_version 239055 (0.0006) [2023-12-27 07:18:27,355][20347] Updated weights for policy 0, policy_version 239436 (0.0007) [2023-12-27 07:18:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 122519552. Throughput: 0: 10515.0, 1: 10480.1. Samples: 122511608. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:27,400][20347] Updated weights for policy 0, policy_version 239446 (0.0007) [2023-12-27 07:18:27,446][20347] Updated weights for policy 0, policy_version 239456 (0.0006) [2023-12-27 07:18:27,478][20381] Updated weights for policy 1, policy_version 239065 (0.0006) [2023-12-27 07:18:27,526][20381] Updated weights for policy 1, policy_version 239075 (0.0007) [2023-12-27 07:18:27,577][20381] Updated weights for policy 1, policy_version 239085 (0.0008) [2023-12-27 07:18:28,153][20347] Updated weights for policy 0, policy_version 239466 (0.0006) [2023-12-27 07:18:28,206][20347] Updated weights for policy 0, policy_version 239476 (0.0007) [2023-12-27 07:18:28,257][20381] Updated weights for policy 1, policy_version 239095 (0.0006) [2023-12-27 07:18:28,266][20347] Updated weights for policy 0, policy_version 239486 (0.0007) [2023-12-27 07:18:28,304][20381] Updated weights for policy 1, policy_version 239105 (0.0006) [2023-12-27 07:18:28,324][20347] Updated weights for policy 0, policy_version 239496 (0.0006) [2023-12-27 07:18:28,363][20381] Updated weights for policy 1, policy_version 239115 (0.0007) [2023-12-27 07:18:28,998][20381] Updated weights for policy 1, policy_version 239125 (0.0008) [2023-12-27 07:18:29,011][20347] Updated weights for policy 0, policy_version 239506 (0.0005) [2023-12-27 07:18:29,054][20381] Updated weights for policy 1, policy_version 239135 (0.0010) [2023-12-27 07:18:29,067][20347] Updated weights for policy 0, policy_version 239516 (0.0006) [2023-12-27 07:18:29,110][20381] Updated weights for policy 1, policy_version 239145 (0.0010) [2023-12-27 07:18:29,134][20347] Updated weights for policy 0, policy_version 239526 (0.0007) [2023-12-27 07:18:29,762][20381] Updated weights for policy 1, policy_version 239155 (0.0006) [2023-12-27 07:18:29,812][20381] Updated weights for policy 1, policy_version 239165 (0.0007) [2023-12-27 07:18:29,847][20347] Updated weights for policy 0, policy_version 239536 (0.0007) [2023-12-27 07:18:29,874][20381] Updated weights for policy 1, policy_version 239175 (0.0008) [2023-12-27 07:18:29,906][20347] Updated weights for policy 0, policy_version 239546 (0.0007) [2023-12-27 07:18:29,966][20347] Updated weights for policy 0, policy_version 239556 (0.0008) [2023-12-27 07:18:30,562][20381] Updated weights for policy 1, policy_version 239185 (0.0006) [2023-12-27 07:18:30,600][20347] Updated weights for policy 0, policy_version 239566 (0.0006) [2023-12-27 07:18:30,611][20381] Updated weights for policy 1, policy_version 239195 (0.0008) [2023-12-27 07:18:30,663][20347] Updated weights for policy 0, policy_version 239576 (0.0005) [2023-12-27 07:18:30,671][20381] Updated weights for policy 1, policy_version 239205 (0.0007) [2023-12-27 07:18:30,718][20347] Updated weights for policy 0, policy_version 239586 (0.0005) [2023-12-27 07:18:30,727][20381] Updated weights for policy 1, policy_version 239215 (0.0007) [2023-12-27 07:18:31,260][20347] Updated weights for policy 0, policy_version 239596 (0.0006) [2023-12-27 07:18:31,324][20347] Updated weights for policy 0, policy_version 239606 (0.0007) [2023-12-27 07:18:31,332][20381] Updated weights for policy 1, policy_version 239225 (0.0006) [2023-12-27 07:18:31,390][20347] Updated weights for policy 0, policy_version 239616 (0.0008) [2023-12-27 07:18:31,395][20381] Updated weights for policy 1, policy_version 239235 (0.0007) [2023-12-27 07:18:31,455][20381] Updated weights for policy 1, policy_version 239245 (0.0006) [2023-12-27 07:18:32,064][20381] Updated weights for policy 1, policy_version 239255 (0.0010) [2023-12-27 07:18:32,106][20347] Updated weights for policy 0, policy_version 239626 (0.0006) [2023-12-27 07:18:32,119][20381] Updated weights for policy 1, policy_version 239265 (0.0010) [2023-12-27 07:18:32,162][20347] Updated weights for policy 0, policy_version 239636 (0.0005) [2023-12-27 07:18:32,177][20381] Updated weights for policy 1, policy_version 239275 (0.0010) [2023-12-27 07:18:32,220][20347] Updated weights for policy 0, policy_version 239646 (0.0006) [2023-12-27 07:18:32,275][20347] Updated weights for policy 0, policy_version 239656 (0.0008) [2023-12-27 07:18:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 122634240. Throughput: 0: 10482.7, 1: 10525.7. Samples: 122639072. Policy #0 lag: (min: 10.0, avg: 24.0, max: 42.0) [2023-12-27 07:18:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:32,854][20381] Updated weights for policy 1, policy_version 239285 (0.0009) [2023-12-27 07:18:32,904][20381] Updated weights for policy 1, policy_version 239295 (0.0005) [2023-12-27 07:18:32,939][20347] Updated weights for policy 0, policy_version 239666 (0.0008) [2023-12-27 07:18:32,960][20381] Updated weights for policy 1, policy_version 239305 (0.0007) [2023-12-27 07:18:32,990][20347] Updated weights for policy 0, policy_version 239676 (0.0005) [2023-12-27 07:18:33,046][20347] Updated weights for policy 0, policy_version 239686 (0.0007) [2023-12-27 07:18:33,595][20381] Updated weights for policy 1, policy_version 239315 (0.0009) [2023-12-27 07:18:33,658][20381] Updated weights for policy 1, policy_version 239325 (0.0008) [2023-12-27 07:18:33,719][20381] Updated weights for policy 1, policy_version 239335 (0.0008) [2023-12-27 07:18:33,756][20347] Updated weights for policy 0, policy_version 239696 (0.0006) [2023-12-27 07:18:33,814][20347] Updated weights for policy 0, policy_version 239706 (0.0007) [2023-12-27 07:18:33,863][20347] Updated weights for policy 0, policy_version 239716 (0.0008) [2023-12-27 07:18:34,351][20381] Updated weights for policy 1, policy_version 239345 (0.0006) [2023-12-27 07:18:34,411][20381] Updated weights for policy 1, policy_version 239355 (0.0007) [2023-12-27 07:18:34,472][20381] Updated weights for policy 1, policy_version 239365 (0.0008) [2023-12-27 07:18:34,525][20381] Updated weights for policy 1, policy_version 239375 (0.0007) [2023-12-27 07:18:34,571][20347] Updated weights for policy 0, policy_version 239726 (0.0008) [2023-12-27 07:18:34,624][20347] Updated weights for policy 0, policy_version 239736 (0.0008) [2023-12-27 07:18:34,674][20347] Updated weights for policy 0, policy_version 239746 (0.0008) [2023-12-27 07:18:35,212][20381] Updated weights for policy 1, policy_version 239385 (0.0006) [2023-12-27 07:18:35,276][20381] Updated weights for policy 1, policy_version 239395 (0.0006) [2023-12-27 07:18:35,336][20381] Updated weights for policy 1, policy_version 239405 (0.0005) [2023-12-27 07:18:35,376][20347] Updated weights for policy 0, policy_version 239756 (0.0008) [2023-12-27 07:18:35,427][20347] Updated weights for policy 0, policy_version 239766 (0.0005) [2023-12-27 07:18:35,484][20347] Updated weights for policy 0, policy_version 239776 (0.0008) [2023-12-27 07:18:35,867][20381] Updated weights for policy 1, policy_version 239415 (0.0006) [2023-12-27 07:18:35,928][20381] Updated weights for policy 1, policy_version 239425 (0.0008) [2023-12-27 07:18:35,983][20381] Updated weights for policy 1, policy_version 239435 (0.0007) [2023-12-27 07:18:36,149][20347] Updated weights for policy 0, policy_version 239786 (0.0010) [2023-12-27 07:18:36,209][20347] Updated weights for policy 0, policy_version 239796 (0.0011) [2023-12-27 07:18:36,268][20347] Updated weights for policy 0, policy_version 239806 (0.0010) [2023-12-27 07:18:36,327][20347] Updated weights for policy 0, policy_version 239816 (0.0011) [2023-12-27 07:18:36,625][20381] Updated weights for policy 1, policy_version 239445 (0.0007) [2023-12-27 07:18:36,681][20381] Updated weights for policy 1, policy_version 239455 (0.0008) [2023-12-27 07:18:36,733][20381] Updated weights for policy 1, policy_version 239465 (0.0007) [2023-12-27 07:18:36,980][20347] Updated weights for policy 0, policy_version 239826 (0.0009) [2023-12-27 07:18:37,038][20347] Updated weights for policy 0, policy_version 239836 (0.0010) [2023-12-27 07:18:37,103][20347] Updated weights for policy 0, policy_version 239846 (0.0010) [2023-12-27 07:18:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20716.0). Total num frames: 122732544. Throughput: 0: 10460.5, 1: 10576.6. Samples: 122700724. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:18:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000239848_61415424.pth... [2023-12-27 07:18:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000238632_61104128.pth [2023-12-27 07:18:37,394][20381] Updated weights for policy 1, policy_version 239475 (0.0008) [2023-12-27 07:18:37,454][20381] Updated weights for policy 1, policy_version 239485 (0.0008) [2023-12-27 07:18:37,513][20381] Updated weights for policy 1, policy_version 239495 (0.0008) [2023-12-27 07:18:37,557][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000239504_61325312.pth... [2023-12-27 07:18:37,560][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000238256_61005824.pth [2023-12-27 07:18:37,786][20347] Updated weights for policy 0, policy_version 239856 (0.0010) [2023-12-27 07:18:37,836][20347] Updated weights for policy 0, policy_version 239866 (0.0011) [2023-12-27 07:18:37,900][20347] Updated weights for policy 0, policy_version 239876 (0.0010) [2023-12-27 07:18:38,137][20381] Updated weights for policy 1, policy_version 239505 (0.0007) [2023-12-27 07:18:38,184][20381] Updated weights for policy 1, policy_version 239515 (0.0009) [2023-12-27 07:18:38,234][20381] Updated weights for policy 1, policy_version 239525 (0.0007) [2023-12-27 07:18:38,283][20381] Updated weights for policy 1, policy_version 239535 (0.0009) [2023-12-27 07:18:38,568][20347] Updated weights for policy 0, policy_version 239886 (0.0010) [2023-12-27 07:18:38,629][20347] Updated weights for policy 0, policy_version 239896 (0.0010) [2023-12-27 07:18:38,686][20347] Updated weights for policy 0, policy_version 239906 (0.0010) [2023-12-27 07:18:39,032][20381] Updated weights for policy 1, policy_version 239545 (0.0007) [2023-12-27 07:18:39,088][20381] Updated weights for policy 1, policy_version 239555 (0.0008) [2023-12-27 07:18:39,147][20381] Updated weights for policy 1, policy_version 239565 (0.0008) [2023-12-27 07:18:39,248][20347] Updated weights for policy 0, policy_version 239916 (0.0007) [2023-12-27 07:18:39,307][20347] Updated weights for policy 0, policy_version 239926 (0.0008) [2023-12-27 07:18:39,370][20347] Updated weights for policy 0, policy_version 239936 (0.0009) [2023-12-27 07:18:39,881][20381] Updated weights for policy 1, policy_version 239575 (0.0006) [2023-12-27 07:18:39,951][20381] Updated weights for policy 1, policy_version 239585 (0.0008) [2023-12-27 07:18:40,014][20381] Updated weights for policy 1, policy_version 239595 (0.0007) [2023-12-27 07:18:40,035][20347] Updated weights for policy 0, policy_version 239946 (0.0007) [2023-12-27 07:18:40,083][20347] Updated weights for policy 0, policy_version 239956 (0.0007) [2023-12-27 07:18:40,145][20347] Updated weights for policy 0, policy_version 239966 (0.0008) [2023-12-27 07:18:40,205][20347] Updated weights for policy 0, policy_version 239976 (0.0008) [2023-12-27 07:18:40,688][20381] Updated weights for policy 1, policy_version 239605 (0.0007) [2023-12-27 07:18:40,736][20381] Updated weights for policy 1, policy_version 239615 (0.0007) [2023-12-27 07:18:40,786][20381] Updated weights for policy 1, policy_version 239625 (0.0007) [2023-12-27 07:18:40,882][20347] Updated weights for policy 0, policy_version 239986 (0.0008) [2023-12-27 07:18:40,932][20347] Updated weights for policy 0, policy_version 239996 (0.0007) [2023-12-27 07:18:40,989][20347] Updated weights for policy 0, policy_version 240006 (0.0007) [2023-12-27 07:18:41,492][20381] Updated weights for policy 1, policy_version 239635 (0.0008) [2023-12-27 07:18:41,548][20381] Updated weights for policy 1, policy_version 239645 (0.0008) [2023-12-27 07:18:41,609][20381] Updated weights for policy 1, policy_version 239655 (0.0008) [2023-12-27 07:18:41,705][20347] Updated weights for policy 0, policy_version 240016 (0.0008) [2023-12-27 07:18:41,768][20347] Updated weights for policy 0, policy_version 240026 (0.0009) [2023-12-27 07:18:41,821][20347] Updated weights for policy 0, policy_version 240036 (0.0011) [2023-12-27 07:18:42,326][20381] Updated weights for policy 1, policy_version 239665 (0.0008) [2023-12-27 07:18:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 122830848. Throughput: 0: 10447.5, 1: 10535.5. Samples: 122825576. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:18:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:42,389][20381] Updated weights for policy 1, policy_version 239675 (0.0008) [2023-12-27 07:18:42,448][20381] Updated weights for policy 1, policy_version 239685 (0.0007) [2023-12-27 07:18:42,503][20381] Updated weights for policy 1, policy_version 239695 (0.0008) [2023-12-27 07:18:42,532][20347] Updated weights for policy 0, policy_version 240046 (0.0009) [2023-12-27 07:18:42,592][20347] Updated weights for policy 0, policy_version 240056 (0.0008) [2023-12-27 07:18:42,647][20347] Updated weights for policy 0, policy_version 240066 (0.0008) [2023-12-27 07:18:43,216][20381] Updated weights for policy 1, policy_version 239705 (0.0007) [2023-12-27 07:18:43,268][20381] Updated weights for policy 1, policy_version 239715 (0.0005) [2023-12-27 07:18:43,297][20347] Updated weights for policy 0, policy_version 240076 (0.0006) [2023-12-27 07:18:43,324][20381] Updated weights for policy 1, policy_version 239725 (0.0007) [2023-12-27 07:18:43,353][20347] Updated weights for policy 0, policy_version 240086 (0.0006) [2023-12-27 07:18:43,402][20347] Updated weights for policy 0, policy_version 240096 (0.0008) [2023-12-27 07:18:43,996][20381] Updated weights for policy 1, policy_version 239735 (0.0008) [2023-12-27 07:18:44,050][20381] Updated weights for policy 1, policy_version 239745 (0.0007) [2023-12-27 07:18:44,097][20381] Updated weights for policy 1, policy_version 239755 (0.0007) [2023-12-27 07:18:44,123][20347] Updated weights for policy 0, policy_version 240106 (0.0007) [2023-12-27 07:18:44,178][20347] Updated weights for policy 0, policy_version 240116 (0.0008) [2023-12-27 07:18:44,239][20347] Updated weights for policy 0, policy_version 240126 (0.0008) [2023-12-27 07:18:44,294][20347] Updated weights for policy 0, policy_version 240136 (0.0007) [2023-12-27 07:18:44,790][20381] Updated weights for policy 1, policy_version 239765 (0.0007) [2023-12-27 07:18:44,842][20381] Updated weights for policy 1, policy_version 239775 (0.0007) [2023-12-27 07:18:44,868][20347] Updated weights for policy 0, policy_version 240146 (0.0006) [2023-12-27 07:18:44,895][20381] Updated weights for policy 1, policy_version 239785 (0.0007) [2023-12-27 07:18:44,924][20347] Updated weights for policy 0, policy_version 240156 (0.0007) [2023-12-27 07:18:44,976][20347] Updated weights for policy 0, policy_version 240166 (0.0007) [2023-12-27 07:18:45,570][20381] Updated weights for policy 1, policy_version 239795 (0.0008) [2023-12-27 07:18:45,615][20381] Updated weights for policy 1, policy_version 239805 (0.0005) [2023-12-27 07:18:45,668][20381] Updated weights for policy 1, policy_version 239815 (0.0008) [2023-12-27 07:18:45,703][20347] Updated weights for policy 0, policy_version 240176 (0.0006) [2023-12-27 07:18:45,755][20347] Updated weights for policy 0, policy_version 240186 (0.0007) [2023-12-27 07:18:45,804][20347] Updated weights for policy 0, policy_version 240196 (0.0008) [2023-12-27 07:18:46,373][20381] Updated weights for policy 1, policy_version 239825 (0.0006) [2023-12-27 07:18:46,405][20347] Updated weights for policy 0, policy_version 240206 (0.0006) [2023-12-27 07:18:46,432][20381] Updated weights for policy 1, policy_version 239835 (0.0006) [2023-12-27 07:18:46,463][20347] Updated weights for policy 0, policy_version 240216 (0.0006) [2023-12-27 07:18:46,490][20381] Updated weights for policy 1, policy_version 239845 (0.0007) [2023-12-27 07:18:46,522][20347] Updated weights for policy 0, policy_version 240226 (0.0006) [2023-12-27 07:18:46,546][20381] Updated weights for policy 1, policy_version 239855 (0.0006) [2023-12-27 07:18:47,167][20381] Updated weights for policy 1, policy_version 239865 (0.0008) [2023-12-27 07:18:47,227][20381] Updated weights for policy 1, policy_version 239875 (0.0007) [2023-12-27 07:18:47,228][20347] Updated weights for policy 0, policy_version 240236 (0.0007) [2023-12-27 07:18:47,281][20381] Updated weights for policy 1, policy_version 239885 (0.0006) [2023-12-27 07:18:47,290][20347] Updated weights for policy 0, policy_version 240246 (0.0006) [2023-12-27 07:18:47,352][20347] Updated weights for policy 0, policy_version 240256 (0.0005) [2023-12-27 07:18:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 122937344. Throughput: 0: 10407.0, 1: 10520.1. Samples: 122947852. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:18:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:47,920][20347] Updated weights for policy 0, policy_version 240266 (0.0005) [2023-12-27 07:18:47,929][20381] Updated weights for policy 1, policy_version 239895 (0.0005) [2023-12-27 07:18:47,976][20347] Updated weights for policy 0, policy_version 240276 (0.0006) [2023-12-27 07:18:47,984][20381] Updated weights for policy 1, policy_version 239905 (0.0007) [2023-12-27 07:18:48,024][20347] Updated weights for policy 0, policy_version 240286 (0.0008) [2023-12-27 07:18:48,040][20381] Updated weights for policy 1, policy_version 239915 (0.0009) [2023-12-27 07:18:48,077][20347] Updated weights for policy 0, policy_version 240296 (0.0008) [2023-12-27 07:18:48,674][20381] Updated weights for policy 1, policy_version 239925 (0.0008) [2023-12-27 07:18:48,726][20381] Updated weights for policy 1, policy_version 239935 (0.0005) [2023-12-27 07:18:48,748][20347] Updated weights for policy 0, policy_version 240306 (0.0011) [2023-12-27 07:18:48,780][20381] Updated weights for policy 1, policy_version 239945 (0.0005) [2023-12-27 07:18:48,807][20347] Updated weights for policy 0, policy_version 240316 (0.0010) [2023-12-27 07:18:48,864][20347] Updated weights for policy 0, policy_version 240326 (0.0008) [2023-12-27 07:18:49,465][20381] Updated weights for policy 1, policy_version 239955 (0.0010) [2023-12-27 07:18:49,524][20381] Updated weights for policy 1, policy_version 239965 (0.0011) [2023-12-27 07:18:49,585][20381] Updated weights for policy 1, policy_version 239975 (0.0011) [2023-12-27 07:18:49,587][20347] Updated weights for policy 0, policy_version 240336 (0.0008) [2023-12-27 07:18:49,649][20347] Updated weights for policy 0, policy_version 240346 (0.0008) [2023-12-27 07:18:49,711][20347] Updated weights for policy 0, policy_version 240356 (0.0008) [2023-12-27 07:18:50,282][20381] Updated weights for policy 1, policy_version 239985 (0.0010) [2023-12-27 07:18:50,343][20381] Updated weights for policy 1, policy_version 239995 (0.0007) [2023-12-27 07:18:50,404][20381] Updated weights for policy 1, policy_version 240005 (0.0008) [2023-12-27 07:18:50,416][20347] Updated weights for policy 0, policy_version 240366 (0.0006) [2023-12-27 07:18:50,453][20381] Updated weights for policy 1, policy_version 240015 (0.0006) [2023-12-27 07:18:50,465][20347] Updated weights for policy 0, policy_version 240376 (0.0007) [2023-12-27 07:18:50,517][20347] Updated weights for policy 0, policy_version 240386 (0.0007) [2023-12-27 07:18:51,122][20381] Updated weights for policy 1, policy_version 240025 (0.0007) [2023-12-27 07:18:51,186][20381] Updated weights for policy 1, policy_version 240035 (0.0008) [2023-12-27 07:18:51,203][20347] Updated weights for policy 0, policy_version 240396 (0.0008) [2023-12-27 07:18:51,239][20381] Updated weights for policy 1, policy_version 240045 (0.0007) [2023-12-27 07:18:51,259][20347] Updated weights for policy 0, policy_version 240406 (0.0008) [2023-12-27 07:18:51,319][20347] Updated weights for policy 0, policy_version 240416 (0.0008) [2023-12-27 07:18:51,950][20381] Updated weights for policy 1, policy_version 240055 (0.0007) [2023-12-27 07:18:52,011][20381] Updated weights for policy 1, policy_version 240065 (0.0008) [2023-12-27 07:18:52,037][20347] Updated weights for policy 0, policy_version 240426 (0.0009) [2023-12-27 07:18:52,077][20381] Updated weights for policy 1, policy_version 240075 (0.0007) [2023-12-27 07:18:52,094][20347] Updated weights for policy 0, policy_version 240436 (0.0007) [2023-12-27 07:18:52,145][20347] Updated weights for policy 0, policy_version 240446 (0.0007) [2023-12-27 07:18:52,209][20347] Updated weights for policy 0, policy_version 240456 (0.0008) [2023-12-27 07:18:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20716.1). Total num frames: 123043840. Throughput: 0: 10411.5, 1: 10498.7. Samples: 123010292. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:18:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:18:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000240080_61472768.pth... [2023-12-27 07:18:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000240456_61571072.pth... [2023-12-27 07:18:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000239240_61259776.pth [2023-12-27 07:18:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000238864_61161472.pth [2023-12-27 07:18:52,824][20381] Updated weights for policy 1, policy_version 240085 (0.0007) [2023-12-27 07:18:52,864][20347] Updated weights for policy 0, policy_version 240466 (0.0006) [2023-12-27 07:18:52,871][20381] Updated weights for policy 1, policy_version 240095 (0.0006) [2023-12-27 07:18:52,917][20381] Updated weights for policy 1, policy_version 240105 (0.0006) [2023-12-27 07:18:52,918][20347] Updated weights for policy 0, policy_version 240476 (0.0006) [2023-12-27 07:18:52,976][20347] Updated weights for policy 0, policy_version 240486 (0.0007) [2023-12-27 07:18:53,610][20381] Updated weights for policy 1, policy_version 240115 (0.0007) [2023-12-27 07:18:53,665][20381] Updated weights for policy 1, policy_version 240125 (0.0007) [2023-12-27 07:18:53,674][20347] Updated weights for policy 0, policy_version 240496 (0.0007) [2023-12-27 07:18:53,724][20381] Updated weights for policy 1, policy_version 240135 (0.0008) [2023-12-27 07:18:53,737][20347] Updated weights for policy 0, policy_version 240506 (0.0008) [2023-12-27 07:18:53,793][20347] Updated weights for policy 0, policy_version 240516 (0.0010) [2023-12-27 07:18:54,402][20381] Updated weights for policy 1, policy_version 240145 (0.0007) [2023-12-27 07:18:54,459][20347] Updated weights for policy 0, policy_version 240526 (0.0007) [2023-12-27 07:18:54,461][20381] Updated weights for policy 1, policy_version 240155 (0.0006) [2023-12-27 07:18:54,514][20381] Updated weights for policy 1, policy_version 240165 (0.0006) [2023-12-27 07:18:54,520][20347] Updated weights for policy 0, policy_version 240536 (0.0007) [2023-12-27 07:18:54,571][20381] Updated weights for policy 1, policy_version 240175 (0.0006) [2023-12-27 07:18:54,576][20347] Updated weights for policy 0, policy_version 240546 (0.0007) [2023-12-27 07:18:55,195][20347] Updated weights for policy 0, policy_version 240556 (0.0006) [2023-12-27 07:18:55,253][20347] Updated weights for policy 0, policy_version 240566 (0.0006) [2023-12-27 07:18:55,274][20381] Updated weights for policy 1, policy_version 240185 (0.0007) [2023-12-27 07:18:55,283][20292] KL-divergence is very high: 183.8266 [2023-12-27 07:18:55,303][20347] Updated weights for policy 0, policy_version 240576 (0.0008) [2023-12-27 07:18:55,324][20381] Updated weights for policy 1, policy_version 240195 (0.0006) [2023-12-27 07:18:55,378][20381] Updated weights for policy 1, policy_version 240205 (0.0007) [2023-12-27 07:18:55,960][20347] Updated weights for policy 0, policy_version 240586 (0.0011) [2023-12-27 07:18:56,010][20347] Updated weights for policy 0, policy_version 240596 (0.0007) [2023-12-27 07:18:56,060][20347] Updated weights for policy 0, policy_version 240606 (0.0007) [2023-12-27 07:18:56,093][20381] Updated weights for policy 1, policy_version 240215 (0.0008) [2023-12-27 07:18:56,114][20347] Updated weights for policy 0, policy_version 240616 (0.0007) [2023-12-27 07:18:56,155][20381] Updated weights for policy 1, policy_version 240225 (0.0008) [2023-12-27 07:18:56,217][20381] Updated weights for policy 1, policy_version 240235 (0.0008) [2023-12-27 07:18:56,736][20347] Updated weights for policy 0, policy_version 240626 (0.0010) [2023-12-27 07:18:56,789][20347] Updated weights for policy 0, policy_version 240636 (0.0010) [2023-12-27 07:18:56,840][20347] Updated weights for policy 0, policy_version 240646 (0.0010) [2023-12-27 07:18:56,879][20381] Updated weights for policy 1, policy_version 240245 (0.0008) [2023-12-27 07:18:56,933][20381] Updated weights for policy 1, policy_version 240255 (0.0006) [2023-12-27 07:18:56,990][20381] Updated weights for policy 1, policy_version 240265 (0.0006) [2023-12-27 07:18:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 123142144. Throughput: 0: 10368.5, 1: 10449.6. Samples: 123131332. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:18:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.340')] [2023-12-27 07:18:57,517][20347] Updated weights for policy 0, policy_version 240656 (0.0010) [2023-12-27 07:18:57,566][20381] Updated weights for policy 1, policy_version 240275 (0.0008) [2023-12-27 07:18:57,571][20347] Updated weights for policy 0, policy_version 240666 (0.0010) [2023-12-27 07:18:57,611][20381] Updated weights for policy 1, policy_version 240285 (0.0007) [2023-12-27 07:18:57,628][20347] Updated weights for policy 0, policy_version 240676 (0.0010) [2023-12-27 07:18:57,662][20381] Updated weights for policy 1, policy_version 240295 (0.0005) [2023-12-27 07:18:58,222][20347] Updated weights for policy 0, policy_version 240686 (0.0009) [2023-12-27 07:18:58,275][20347] Updated weights for policy 0, policy_version 240696 (0.0008) [2023-12-27 07:18:58,326][20347] Updated weights for policy 0, policy_version 240706 (0.0007) [2023-12-27 07:18:58,355][20381] Updated weights for policy 1, policy_version 240305 (0.0007) [2023-12-27 07:18:58,414][20381] Updated weights for policy 1, policy_version 240315 (0.0007) [2023-12-27 07:18:58,477][20381] Updated weights for policy 1, policy_version 240325 (0.0008) [2023-12-27 07:18:58,490][20292] KL-divergence is very high: 115.0070 [2023-12-27 07:18:58,536][20381] Updated weights for policy 1, policy_version 240335 (0.0008) [2023-12-27 07:18:59,090][20347] Updated weights for policy 0, policy_version 240716 (0.0007) [2023-12-27 07:18:59,153][20347] Updated weights for policy 0, policy_version 240726 (0.0007) [2023-12-27 07:18:59,164][20381] Updated weights for policy 1, policy_version 240345 (0.0006) [2023-12-27 07:18:59,212][20381] Updated weights for policy 1, policy_version 240355 (0.0006) [2023-12-27 07:18:59,226][20347] Updated weights for policy 0, policy_version 240736 (0.0006) [2023-12-27 07:18:59,268][20381] Updated weights for policy 1, policy_version 240365 (0.0008) [2023-12-27 07:18:59,974][20347] Updated weights for policy 0, policy_version 240746 (0.0008) [2023-12-27 07:18:59,975][20381] Updated weights for policy 1, policy_version 240375 (0.0008) [2023-12-27 07:19:00,031][20381] Updated weights for policy 1, policy_version 240385 (0.0006) [2023-12-27 07:19:00,033][20347] Updated weights for policy 0, policy_version 240756 (0.0007) [2023-12-27 07:19:00,084][20381] Updated weights for policy 1, policy_version 240395 (0.0006) [2023-12-27 07:19:00,089][20347] Updated weights for policy 0, policy_version 240766 (0.0008) [2023-12-27 07:19:00,148][20347] Updated weights for policy 0, policy_version 240776 (0.0008) [2023-12-27 07:19:00,736][20347] Updated weights for policy 0, policy_version 240786 (0.0005) [2023-12-27 07:19:00,757][20381] Updated weights for policy 1, policy_version 240405 (0.0006) [2023-12-27 07:19:00,788][20347] Updated weights for policy 0, policy_version 240796 (0.0005) [2023-12-27 07:19:00,812][20381] Updated weights for policy 1, policy_version 240415 (0.0007) [2023-12-27 07:19:00,850][20347] Updated weights for policy 0, policy_version 240806 (0.0005) [2023-12-27 07:19:00,873][20381] Updated weights for policy 1, policy_version 240425 (0.0005) [2023-12-27 07:19:01,474][20381] Updated weights for policy 1, policy_version 240435 (0.0006) [2023-12-27 07:19:01,501][20347] Updated weights for policy 0, policy_version 240816 (0.0009) [2023-12-27 07:19:01,513][20292] KL-divergence is very high: 789.8381 [2023-12-27 07:19:01,524][20292] KL-divergence is very high: 1837.6107 [2023-12-27 07:19:01,525][20381] Updated weights for policy 1, policy_version 240445 (0.0007) [2023-12-27 07:19:01,558][20347] Updated weights for policy 0, policy_version 240826 (0.0010) [2023-12-27 07:19:01,584][20381] Updated weights for policy 1, policy_version 240455 (0.0010) [2023-12-27 07:19:01,595][20292] KL-divergence is very high: 144.9772 [2023-12-27 07:19:01,619][20347] Updated weights for policy 0, policy_version 240836 (0.0008) [2023-12-27 07:19:02,214][20381] Updated weights for policy 1, policy_version 240465 (0.0010) [2023-12-27 07:19:02,226][20292] KL-divergence is very high: 136.8337 [2023-12-27 07:19:02,232][20292] KL-divergence is very high: 750.6135 [2023-12-27 07:19:02,263][20292] KL-divergence is very high: 143.9484 [2023-12-27 07:19:02,273][20381] Updated weights for policy 1, policy_version 240475 (0.0006) [2023-12-27 07:19:02,275][20292] KL-divergence is very high: 272.2647 [2023-12-27 07:19:02,283][20292] KL-divergence is very high: 142.0061 [2023-12-27 07:19:02,329][20292] KL-divergence is very high: 446.2639 [2023-12-27 07:19:02,334][20381] Updated weights for policy 1, policy_version 240485 (0.0005) [2023-12-27 07:19:02,336][20347] Updated weights for policy 0, policy_version 240846 (0.0008) [2023-12-27 07:19:02,353][20292] KL-divergence is very high: 127.5534 [2023-12-27 07:19:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 123240448. Throughput: 0: 10337.5, 1: 10437.1. Samples: 123257720. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '877.060')] [2023-12-27 07:19:02,366][20292] KL-divergence is very high: 158.3941 [2023-12-27 07:19:02,373][20292] KL-divergence is very high: 126.8337 [2023-12-27 07:19:02,379][20292] KL-divergence is very high: 103.7292 [2023-12-27 07:19:02,392][20347] Updated weights for policy 0, policy_version 240856 (0.0007) [2023-12-27 07:19:02,394][20381] Updated weights for policy 1, policy_version 240495 (0.0006) [2023-12-27 07:19:02,446][20347] Updated weights for policy 0, policy_version 240866 (0.0008) [2023-12-27 07:19:02,998][20292] KL-divergence is very high: 122.4505 [2023-12-27 07:19:02,999][20381] Updated weights for policy 1, policy_version 240505 (0.0008) [2023-12-27 07:19:03,044][20292] KL-divergence is very high: 122.7849 [2023-12-27 07:19:03,056][20381] Updated weights for policy 1, policy_version 240515 (0.0007) [2023-12-27 07:19:03,114][20381] Updated weights for policy 1, policy_version 240525 (0.0007) [2023-12-27 07:19:03,121][20347] Updated weights for policy 0, policy_version 240876 (0.0008) [2023-12-27 07:19:03,180][20347] Updated weights for policy 0, policy_version 240886 (0.0006) [2023-12-27 07:19:03,238][20347] Updated weights for policy 0, policy_version 240896 (0.0008) [2023-12-27 07:19:03,719][20381] Updated weights for policy 1, policy_version 240535 (0.0006) [2023-12-27 07:19:03,779][20381] Updated weights for policy 1, policy_version 240545 (0.0006) [2023-12-27 07:19:03,823][20292] KL-divergence is very high: 195.9986 [2023-12-27 07:19:03,826][20347] Updated weights for policy 0, policy_version 240906 (0.0008) [2023-12-27 07:19:03,830][20292] KL-divergence is very high: 257.6223 [2023-12-27 07:19:03,836][20292] KL-divergence is very high: 293.8477 [2023-12-27 07:19:03,837][20381] Updated weights for policy 1, policy_version 240555 (0.0007) [2023-12-27 07:19:03,842][20292] KL-divergence is very high: 223.7338 [2023-12-27 07:19:03,848][20292] KL-divergence is very high: 195.4568 [2023-12-27 07:19:03,855][20292] KL-divergence is very high: 121.9348 [2023-12-27 07:19:03,882][20347] Updated weights for policy 0, policy_version 240916 (0.0010) [2023-12-27 07:19:03,944][20347] Updated weights for policy 0, policy_version 240926 (0.0005) [2023-12-27 07:19:04,000][20347] Updated weights for policy 0, policy_version 240936 (0.0005) [2023-12-27 07:19:04,501][20292] KL-divergence is very high: 452.7301 [2023-12-27 07:19:04,513][20381] Updated weights for policy 1, policy_version 240565 (0.0008) [2023-12-27 07:19:04,518][20292] KL-divergence is very high: 301.1071 [2023-12-27 07:19:04,546][20292] KL-divergence is very high: 200.2915 [2023-12-27 07:19:04,572][20381] Updated weights for policy 1, policy_version 240575 (0.0008) [2023-12-27 07:19:04,594][20292] KL-divergence is very high: 141.0639 [2023-12-27 07:19:04,604][20347] Updated weights for policy 0, policy_version 240946 (0.0006) [2023-12-27 07:19:04,628][20381] Updated weights for policy 1, policy_version 240585 (0.0007) [2023-12-27 07:19:04,639][20292] KL-divergence is very high: 217.6224 [2023-12-27 07:19:04,662][20292] KL-divergence is very high: 132.1662 [2023-12-27 07:19:04,664][20347] Updated weights for policy 0, policy_version 240956 (0.0007) [2023-12-27 07:19:04,727][20347] Updated weights for policy 0, policy_version 240966 (0.0008) [2023-12-27 07:19:05,341][20381] Updated weights for policy 1, policy_version 240595 (0.0007) [2023-12-27 07:19:05,349][20347] Updated weights for policy 0, policy_version 240976 (0.0006) [2023-12-27 07:19:05,394][20347] Updated weights for policy 0, policy_version 240986 (0.0005) [2023-12-27 07:19:05,396][20381] Updated weights for policy 1, policy_version 240605 (0.0007) [2023-12-27 07:19:05,445][20347] Updated weights for policy 0, policy_version 240996 (0.0005) [2023-12-27 07:19:05,457][20381] Updated weights for policy 1, policy_version 240615 (0.0007) [2023-12-27 07:19:06,048][20347] Updated weights for policy 0, policy_version 241006 (0.0008) [2023-12-27 07:19:06,111][20347] Updated weights for policy 0, policy_version 241016 (0.0010) [2023-12-27 07:19:06,171][20381] Updated weights for policy 1, policy_version 240625 (0.0008) [2023-12-27 07:19:06,174][20347] Updated weights for policy 0, policy_version 241026 (0.0008) [2023-12-27 07:19:06,229][20381] Updated weights for policy 1, policy_version 240635 (0.0008) [2023-12-27 07:19:06,279][20381] Updated weights for policy 1, policy_version 240645 (0.0008) [2023-12-27 07:19:06,331][20381] Updated weights for policy 1, policy_version 240655 (0.0008) [2023-12-27 07:19:06,871][20347] Updated weights for policy 0, policy_version 241036 (0.0008) [2023-12-27 07:19:06,924][20347] Updated weights for policy 0, policy_version 241046 (0.0007) [2023-12-27 07:19:06,977][20347] Updated weights for policy 0, policy_version 241056 (0.0008) [2023-12-27 07:19:07,033][20381] Updated weights for policy 1, policy_version 240665 (0.0008) [2023-12-27 07:19:07,094][20381] Updated weights for policy 1, policy_version 240675 (0.0005) [2023-12-27 07:19:07,150][20381] Updated weights for policy 1, policy_version 240685 (0.0005) [2023-12-27 07:19:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 123355136. Throughput: 0: 10347.7, 1: 10451.1. Samples: 123321408. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '80.950')] [2023-12-27 07:19:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000240688_61628416.pth... [2023-12-27 07:19:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000241064_61726720.pth... [2023-12-27 07:19:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000239848_61415424.pth [2023-12-27 07:19:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000239504_61325312.pth [2023-12-27 07:19:07,606][20347] Updated weights for policy 0, policy_version 241066 (0.0010) [2023-12-27 07:19:07,661][20347] Updated weights for policy 0, policy_version 241076 (0.0010) [2023-12-27 07:19:07,722][20347] Updated weights for policy 0, policy_version 241086 (0.0010) [2023-12-27 07:19:07,724][20381] Updated weights for policy 1, policy_version 240695 (0.0007) [2023-12-27 07:19:07,781][20347] Updated weights for policy 0, policy_version 241096 (0.0010) [2023-12-27 07:19:07,786][20381] Updated weights for policy 1, policy_version 240705 (0.0011) [2023-12-27 07:19:07,848][20381] Updated weights for policy 1, policy_version 240715 (0.0008) [2023-12-27 07:19:08,455][20381] Updated weights for policy 1, policy_version 240725 (0.0006) [2023-12-27 07:19:08,461][20347] Updated weights for policy 0, policy_version 241106 (0.0006) [2023-12-27 07:19:08,507][20381] Updated weights for policy 1, policy_version 240735 (0.0007) [2023-12-27 07:19:08,513][20347] Updated weights for policy 0, policy_version 241116 (0.0006) [2023-12-27 07:19:08,559][20381] Updated weights for policy 1, policy_version 240745 (0.0008) [2023-12-27 07:19:08,566][20347] Updated weights for policy 0, policy_version 241126 (0.0008) [2023-12-27 07:19:09,237][20381] Updated weights for policy 1, policy_version 240755 (0.0009) [2023-12-27 07:19:09,258][20347] Updated weights for policy 0, policy_version 241136 (0.0009) [2023-12-27 07:19:09,297][20381] Updated weights for policy 1, policy_version 240765 (0.0010) [2023-12-27 07:19:09,318][20347] Updated weights for policy 0, policy_version 241146 (0.0007) [2023-12-27 07:19:09,358][20381] Updated weights for policy 1, policy_version 240775 (0.0007) [2023-12-27 07:19:09,384][20347] Updated weights for policy 0, policy_version 241156 (0.0008) [2023-12-27 07:19:09,969][20347] Updated weights for policy 0, policy_version 241166 (0.0008) [2023-12-27 07:19:10,027][20347] Updated weights for policy 0, policy_version 241176 (0.0007) [2023-12-27 07:19:10,083][20347] Updated weights for policy 0, policy_version 241186 (0.0008) [2023-12-27 07:19:10,101][20381] Updated weights for policy 1, policy_version 240785 (0.0008) [2023-12-27 07:19:10,156][20381] Updated weights for policy 1, policy_version 240795 (0.0006) [2023-12-27 07:19:10,211][20381] Updated weights for policy 1, policy_version 240805 (0.0007) [2023-12-27 07:19:10,268][20381] Updated weights for policy 1, policy_version 240815 (0.0008) [2023-12-27 07:19:10,694][20347] Updated weights for policy 0, policy_version 241196 (0.0007) [2023-12-27 07:19:10,753][20347] Updated weights for policy 0, policy_version 241206 (0.0008) [2023-12-27 07:19:10,802][20347] Updated weights for policy 0, policy_version 241216 (0.0007) [2023-12-27 07:19:10,905][20381] Updated weights for policy 1, policy_version 240825 (0.0006) [2023-12-27 07:19:10,954][20381] Updated weights for policy 1, policy_version 240835 (0.0005) [2023-12-27 07:19:11,003][20381] Updated weights for policy 1, policy_version 240845 (0.0005) [2023-12-27 07:19:11,553][20347] Updated weights for policy 0, policy_version 241226 (0.0008) [2023-12-27 07:19:11,611][20347] Updated weights for policy 0, policy_version 241236 (0.0008) [2023-12-27 07:19:11,674][20347] Updated weights for policy 0, policy_version 241246 (0.0007) [2023-12-27 07:19:11,704][20381] Updated weights for policy 1, policy_version 240855 (0.0007) [2023-12-27 07:19:11,743][20347] Updated weights for policy 0, policy_version 241256 (0.0007) [2023-12-27 07:19:11,770][20381] Updated weights for policy 1, policy_version 240865 (0.0008) [2023-12-27 07:19:11,833][20381] Updated weights for policy 1, policy_version 240875 (0.0008) [2023-12-27 07:19:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 123453440. Throughput: 0: 10394.5, 1: 10407.4. Samples: 123447692. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '131.520')] [2023-12-27 07:19:12,428][20347] Updated weights for policy 0, policy_version 241266 (0.0009) [2023-12-27 07:19:12,481][20381] Updated weights for policy 1, policy_version 240885 (0.0006) [2023-12-27 07:19:12,484][20347] Updated weights for policy 0, policy_version 241276 (0.0010) [2023-12-27 07:19:12,529][20381] Updated weights for policy 1, policy_version 240895 (0.0006) [2023-12-27 07:19:12,543][20347] Updated weights for policy 0, policy_version 241286 (0.0010) [2023-12-27 07:19:12,579][20381] Updated weights for policy 1, policy_version 240905 (0.0006) [2023-12-27 07:19:13,253][20347] Updated weights for policy 0, policy_version 241296 (0.0009) [2023-12-27 07:19:13,296][20381] Updated weights for policy 1, policy_version 240915 (0.0007) [2023-12-27 07:19:13,301][20347] Updated weights for policy 0, policy_version 241306 (0.0010) [2023-12-27 07:19:13,347][20347] Updated weights for policy 0, policy_version 241316 (0.0010) [2023-12-27 07:19:13,352][20381] Updated weights for policy 1, policy_version 240925 (0.0006) [2023-12-27 07:19:13,412][20381] Updated weights for policy 1, policy_version 240935 (0.0005) [2023-12-27 07:19:13,969][20381] Updated weights for policy 1, policy_version 240945 (0.0006) [2023-12-27 07:19:14,022][20381] Updated weights for policy 1, policy_version 240955 (0.0005) [2023-12-27 07:19:14,041][20347] Updated weights for policy 0, policy_version 241326 (0.0010) [2023-12-27 07:19:14,078][20381] Updated weights for policy 1, policy_version 240965 (0.0005) [2023-12-27 07:19:14,097][20347] Updated weights for policy 0, policy_version 241336 (0.0010) [2023-12-27 07:19:14,140][20381] Updated weights for policy 1, policy_version 240975 (0.0005) [2023-12-27 07:19:14,152][20347] Updated weights for policy 0, policy_version 241346 (0.0009) [2023-12-27 07:19:14,783][20381] Updated weights for policy 1, policy_version 240985 (0.0007) [2023-12-27 07:19:14,787][20347] Updated weights for policy 0, policy_version 241356 (0.0008) [2023-12-27 07:19:14,837][20381] Updated weights for policy 1, policy_version 240995 (0.0006) [2023-12-27 07:19:14,848][20347] Updated weights for policy 0, policy_version 241366 (0.0011) [2023-12-27 07:19:14,896][20381] Updated weights for policy 1, policy_version 241005 (0.0006) [2023-12-27 07:19:14,908][20347] Updated weights for policy 0, policy_version 241376 (0.0011) [2023-12-27 07:19:15,530][20381] Updated weights for policy 1, policy_version 241015 (0.0007) [2023-12-27 07:19:15,593][20381] Updated weights for policy 1, policy_version 241025 (0.0007) [2023-12-27 07:19:15,599][20347] Updated weights for policy 0, policy_version 241386 (0.0011) [2023-12-27 07:19:15,648][20381] Updated weights for policy 1, policy_version 241035 (0.0005) [2023-12-27 07:19:15,650][20347] Updated weights for policy 0, policy_version 241396 (0.0010) [2023-12-27 07:19:15,699][20347] Updated weights for policy 0, policy_version 241406 (0.0010) [2023-12-27 07:19:15,746][20347] Updated weights for policy 0, policy_version 241416 (0.0008) [2023-12-27 07:19:16,353][20381] Updated weights for policy 1, policy_version 241045 (0.0007) [2023-12-27 07:19:16,406][20381] Updated weights for policy 1, policy_version 241055 (0.0008) [2023-12-27 07:19:16,457][20347] Updated weights for policy 0, policy_version 241426 (0.0006) [2023-12-27 07:19:16,459][20381] Updated weights for policy 1, policy_version 241065 (0.0007) [2023-12-27 07:19:16,504][20347] Updated weights for policy 0, policy_version 241436 (0.0006) [2023-12-27 07:19:16,547][20347] Updated weights for policy 0, policy_version 241446 (0.0007) [2023-12-27 07:19:17,181][20381] Updated weights for policy 1, policy_version 241075 (0.0006) [2023-12-27 07:19:17,201][20347] Updated weights for policy 0, policy_version 241456 (0.0007) [2023-12-27 07:19:17,230][20381] Updated weights for policy 1, policy_version 241085 (0.0005) [2023-12-27 07:19:17,258][20347] Updated weights for policy 0, policy_version 241466 (0.0007) [2023-12-27 07:19:17,282][20381] Updated weights for policy 1, policy_version 241095 (0.0005) [2023-12-27 07:19:17,313][20347] Updated weights for policy 0, policy_version 241476 (0.0005) [2023-12-27 07:19:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 123568128. Throughput: 0: 10359.3, 1: 10343.1. Samples: 123570676. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '288.790')] [2023-12-27 07:19:17,985][20347] Updated weights for policy 0, policy_version 241486 (0.0007) [2023-12-27 07:19:17,986][20381] Updated weights for policy 1, policy_version 241105 (0.0006) [2023-12-27 07:19:18,041][20347] Updated weights for policy 0, policy_version 241496 (0.0006) [2023-12-27 07:19:18,043][20381] Updated weights for policy 1, policy_version 241115 (0.0007) [2023-12-27 07:19:18,092][20381] Updated weights for policy 1, policy_version 241125 (0.0005) [2023-12-27 07:19:18,094][20347] Updated weights for policy 0, policy_version 241506 (0.0007) [2023-12-27 07:19:18,142][20381] Updated weights for policy 1, policy_version 241135 (0.0006) [2023-12-27 07:19:18,799][20347] Updated weights for policy 0, policy_version 241516 (0.0006) [2023-12-27 07:19:18,851][20381] Updated weights for policy 1, policy_version 241145 (0.0007) [2023-12-27 07:19:18,880][20347] Updated weights for policy 0, policy_version 241526 (0.0007) [2023-12-27 07:19:18,915][20381] Updated weights for policy 1, policy_version 241155 (0.0007) [2023-12-27 07:19:18,940][20347] Updated weights for policy 0, policy_version 241536 (0.0005) [2023-12-27 07:19:18,975][20381] Updated weights for policy 1, policy_version 241165 (0.0006) [2023-12-27 07:19:19,633][20381] Updated weights for policy 1, policy_version 241175 (0.0008) [2023-12-27 07:19:19,659][20347] Updated weights for policy 0, policy_version 241546 (0.0006) [2023-12-27 07:19:19,689][20381] Updated weights for policy 1, policy_version 241185 (0.0008) [2023-12-27 07:19:19,727][20347] Updated weights for policy 0, policy_version 241556 (0.0006) [2023-12-27 07:19:19,752][20381] Updated weights for policy 1, policy_version 241195 (0.0008) [2023-12-27 07:19:19,789][20347] Updated weights for policy 0, policy_version 241566 (0.0005) [2023-12-27 07:19:19,859][20347] Updated weights for policy 0, policy_version 241576 (0.0010) [2023-12-27 07:19:20,491][20381] Updated weights for policy 1, policy_version 241205 (0.0007) [2023-12-27 07:19:20,544][20381] Updated weights for policy 1, policy_version 241215 (0.0007) [2023-12-27 07:19:20,598][20381] Updated weights for policy 1, policy_version 241225 (0.0007) [2023-12-27 07:19:20,604][20347] Updated weights for policy 0, policy_version 241586 (0.0011) [2023-12-27 07:19:20,664][20347] Updated weights for policy 0, policy_version 241596 (0.0008) [2023-12-27 07:19:20,719][20347] Updated weights for policy 0, policy_version 241606 (0.0008) [2023-12-27 07:19:21,296][20381] Updated weights for policy 1, policy_version 241235 (0.0007) [2023-12-27 07:19:21,363][20381] Updated weights for policy 1, policy_version 241245 (0.0008) [2023-12-27 07:19:21,424][20381] Updated weights for policy 1, policy_version 241255 (0.0006) [2023-12-27 07:19:21,448][20347] Updated weights for policy 0, policy_version 241616 (0.0010) [2023-12-27 07:19:21,514][20347] Updated weights for policy 0, policy_version 241626 (0.0010) [2023-12-27 07:19:21,568][20347] Updated weights for policy 0, policy_version 241636 (0.0010) [2023-12-27 07:19:22,121][20381] Updated weights for policy 1, policy_version 241265 (0.0006) [2023-12-27 07:19:22,185][20381] Updated weights for policy 1, policy_version 241275 (0.0007) [2023-12-27 07:19:22,233][20381] Updated weights for policy 1, policy_version 241285 (0.0006) [2023-12-27 07:19:22,278][20347] Updated weights for policy 0, policy_version 241646 (0.0008) [2023-12-27 07:19:22,298][20381] Updated weights for policy 1, policy_version 241295 (0.0006) [2023-12-27 07:19:22,340][20347] Updated weights for policy 0, policy_version 241656 (0.0008) [2023-12-27 07:19:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 123658240. Throughput: 0: 10368.8, 1: 10294.7. Samples: 123630584. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '223.130')] [2023-12-27 07:19:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000241296_61784064.pth... [2023-12-27 07:19:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000240080_61472768.pth [2023-12-27 07:19:22,402][20347] Updated weights for policy 0, policy_version 241666 (0.0011) [2023-12-27 07:19:22,440][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000241672_61882368.pth... [2023-12-27 07:19:22,444][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000240456_61571072.pth [2023-12-27 07:19:22,982][20381] Updated weights for policy 1, policy_version 241305 (0.0010) [2023-12-27 07:19:23,036][20381] Updated weights for policy 1, policy_version 241315 (0.0011) [2023-12-27 07:19:23,089][20381] Updated weights for policy 1, policy_version 241325 (0.0010) [2023-12-27 07:19:23,098][20347] Updated weights for policy 0, policy_version 241676 (0.0010) [2023-12-27 07:19:23,160][20347] Updated weights for policy 0, policy_version 241686 (0.0010) [2023-12-27 07:19:23,216][20347] Updated weights for policy 0, policy_version 241696 (0.0010) [2023-12-27 07:19:23,777][20381] Updated weights for policy 1, policy_version 241335 (0.0010) [2023-12-27 07:19:23,834][20381] Updated weights for policy 1, policy_version 241345 (0.0010) [2023-12-27 07:19:23,884][20347] Updated weights for policy 0, policy_version 241706 (0.0010) [2023-12-27 07:19:23,889][20381] Updated weights for policy 1, policy_version 241355 (0.0010) [2023-12-27 07:19:23,938][20347] Updated weights for policy 0, policy_version 241716 (0.0006) [2023-12-27 07:19:23,990][20347] Updated weights for policy 0, policy_version 241726 (0.0007) [2023-12-27 07:19:24,034][20347] Updated weights for policy 0, policy_version 241736 (0.0006) [2023-12-27 07:19:24,480][20381] Updated weights for policy 1, policy_version 241365 (0.0009) [2023-12-27 07:19:24,542][20381] Updated weights for policy 1, policy_version 241375 (0.0008) [2023-12-27 07:19:24,597][20381] Updated weights for policy 1, policy_version 241385 (0.0007) [2023-12-27 07:19:24,695][20347] Updated weights for policy 0, policy_version 241746 (0.0007) [2023-12-27 07:19:24,758][20347] Updated weights for policy 0, policy_version 241756 (0.0008) [2023-12-27 07:19:24,810][20347] Updated weights for policy 0, policy_version 241766 (0.0008) [2023-12-27 07:19:25,311][20381] Updated weights for policy 1, policy_version 241395 (0.0008) [2023-12-27 07:19:25,365][20381] Updated weights for policy 1, policy_version 241405 (0.0008) [2023-12-27 07:19:25,412][20347] Updated weights for policy 0, policy_version 241776 (0.0006) [2023-12-27 07:19:25,417][20381] Updated weights for policy 1, policy_version 241415 (0.0008) [2023-12-27 07:19:25,460][20347] Updated weights for policy 0, policy_version 241786 (0.0006) [2023-12-27 07:19:25,503][20347] Updated weights for policy 0, policy_version 241796 (0.0007) [2023-12-27 07:19:26,078][20381] Updated weights for policy 1, policy_version 241425 (0.0007) [2023-12-27 07:19:26,135][20381] Updated weights for policy 1, policy_version 241435 (0.0007) [2023-12-27 07:19:26,184][20381] Updated weights for policy 1, policy_version 241445 (0.0007) [2023-12-27 07:19:26,190][20347] Updated weights for policy 0, policy_version 241806 (0.0009) [2023-12-27 07:19:26,232][20381] Updated weights for policy 1, policy_version 241455 (0.0006) [2023-12-27 07:19:26,241][20347] Updated weights for policy 0, policy_version 241816 (0.0010) [2023-12-27 07:19:26,287][20347] Updated weights for policy 0, policy_version 241826 (0.0010) [2023-12-27 07:19:26,865][20381] Updated weights for policy 1, policy_version 241465 (0.0007) [2023-12-27 07:19:26,907][20347] Updated weights for policy 0, policy_version 241836 (0.0010) [2023-12-27 07:19:26,928][20381] Updated weights for policy 1, policy_version 241475 (0.0008) [2023-12-27 07:19:26,973][20347] Updated weights for policy 0, policy_version 241846 (0.0007) [2023-12-27 07:19:26,988][20381] Updated weights for policy 1, policy_version 241485 (0.0011) [2023-12-27 07:19:27,033][20347] Updated weights for policy 0, policy_version 241856 (0.0007) [2023-12-27 07:19:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 123764736. Throughput: 0: 10309.9, 1: 10272.2. Samples: 123751768. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '201.850')] [2023-12-27 07:19:27,600][20347] Updated weights for policy 0, policy_version 241866 (0.0006) [2023-12-27 07:19:27,611][20381] Updated weights for policy 1, policy_version 241495 (0.0007) [2023-12-27 07:19:27,660][20381] Updated weights for policy 1, policy_version 241505 (0.0005) [2023-12-27 07:19:27,660][20347] Updated weights for policy 0, policy_version 241876 (0.0006) [2023-12-27 07:19:27,715][20381] Updated weights for policy 1, policy_version 241515 (0.0005) [2023-12-27 07:19:27,720][20347] Updated weights for policy 0, policy_version 241886 (0.0006) [2023-12-27 07:19:27,777][20347] Updated weights for policy 0, policy_version 241896 (0.0009) [2023-12-27 07:19:28,321][20347] Updated weights for policy 0, policy_version 241906 (0.0006) [2023-12-27 07:19:28,331][20381] Updated weights for policy 1, policy_version 241525 (0.0008) [2023-12-27 07:19:28,376][20347] Updated weights for policy 0, policy_version 241916 (0.0008) [2023-12-27 07:19:28,389][20381] Updated weights for policy 1, policy_version 241535 (0.0007) [2023-12-27 07:19:28,431][20347] Updated weights for policy 0, policy_version 241926 (0.0008) [2023-12-27 07:19:28,436][20381] Updated weights for policy 1, policy_version 241545 (0.0006) [2023-12-27 07:19:28,993][20381] Updated weights for policy 1, policy_version 241555 (0.0006) [2023-12-27 07:19:29,042][20381] Updated weights for policy 1, policy_version 241565 (0.0005) [2023-12-27 07:19:29,085][20347] Updated weights for policy 0, policy_version 241936 (0.0008) [2023-12-27 07:19:29,090][20381] Updated weights for policy 1, policy_version 241575 (0.0005) [2023-12-27 07:19:29,139][20347] Updated weights for policy 0, policy_version 241946 (0.0007) [2023-12-27 07:19:29,190][20347] Updated weights for policy 0, policy_version 241956 (0.0008) [2023-12-27 07:19:29,710][20381] Updated weights for policy 1, policy_version 241585 (0.0005) [2023-12-27 07:19:29,764][20381] Updated weights for policy 1, policy_version 241595 (0.0007) [2023-12-27 07:19:29,813][20381] Updated weights for policy 1, policy_version 241605 (0.0010) [2023-12-27 07:19:29,864][20347] Updated weights for policy 0, policy_version 241966 (0.0009) [2023-12-27 07:19:29,867][20381] Updated weights for policy 1, policy_version 241615 (0.0011) [2023-12-27 07:19:29,927][20347] Updated weights for policy 0, policy_version 241976 (0.0007) [2023-12-27 07:19:29,989][20347] Updated weights for policy 0, policy_version 241986 (0.0010) [2023-12-27 07:19:30,564][20381] Updated weights for policy 1, policy_version 241625 (0.0006) [2023-12-27 07:19:30,621][20381] Updated weights for policy 1, policy_version 241635 (0.0006) [2023-12-27 07:19:30,630][20347] Updated weights for policy 0, policy_version 241996 (0.0009) [2023-12-27 07:19:30,670][20381] Updated weights for policy 1, policy_version 241645 (0.0005) [2023-12-27 07:19:30,679][20347] Updated weights for policy 0, policy_version 242006 (0.0006) [2023-12-27 07:19:30,726][20347] Updated weights for policy 0, policy_version 242016 (0.0007) [2023-12-27 07:19:31,356][20381] Updated weights for policy 1, policy_version 241655 (0.0008) [2023-12-27 07:19:31,418][20381] Updated weights for policy 1, policy_version 241665 (0.0007) [2023-12-27 07:19:31,444][20347] Updated weights for policy 0, policy_version 242026 (0.0007) [2023-12-27 07:19:31,479][20381] Updated weights for policy 1, policy_version 241675 (0.0007) [2023-12-27 07:19:31,488][20347] Updated weights for policy 0, policy_version 242036 (0.0007) [2023-12-27 07:19:31,538][20347] Updated weights for policy 0, policy_version 242046 (0.0006) [2023-12-27 07:19:31,592][20347] Updated weights for policy 0, policy_version 242056 (0.0005) [2023-12-27 07:19:32,182][20381] Updated weights for policy 1, policy_version 241685 (0.0007) [2023-12-27 07:19:32,230][20381] Updated weights for policy 1, policy_version 241695 (0.0008) [2023-12-27 07:19:32,283][20347] Updated weights for policy 0, policy_version 242066 (0.0006) [2023-12-27 07:19:32,289][20381] Updated weights for policy 1, policy_version 241705 (0.0007) [2023-12-27 07:19:32,340][20347] Updated weights for policy 0, policy_version 242076 (0.0006) [2023-12-27 07:19:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 123871232. Throughput: 0: 10372.6, 1: 10371.2. Samples: 123881324. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '331.170')] [2023-12-27 07:19:32,402][20347] Updated weights for policy 0, policy_version 242086 (0.0008) [2023-12-27 07:19:33,002][20381] Updated weights for policy 1, policy_version 241715 (0.0006) [2023-12-27 07:19:33,063][20381] Updated weights for policy 1, policy_version 241725 (0.0006) [2023-12-27 07:19:33,120][20381] Updated weights for policy 1, policy_version 241735 (0.0007) [2023-12-27 07:19:33,122][20347] Updated weights for policy 0, policy_version 242096 (0.0006) [2023-12-27 07:19:33,177][20347] Updated weights for policy 0, policy_version 242106 (0.0007) [2023-12-27 07:19:33,238][20347] Updated weights for policy 0, policy_version 242116 (0.0007) [2023-12-27 07:19:33,683][20381] Updated weights for policy 1, policy_version 241745 (0.0006) [2023-12-27 07:19:33,741][20381] Updated weights for policy 1, policy_version 241755 (0.0005) [2023-12-27 07:19:33,800][20347] Updated weights for policy 0, policy_version 242126 (0.0006) [2023-12-27 07:19:33,800][20381] Updated weights for policy 1, policy_version 241765 (0.0005) [2023-12-27 07:19:33,859][20381] Updated weights for policy 1, policy_version 241775 (0.0006) [2023-12-27 07:19:33,861][20347] Updated weights for policy 0, policy_version 242136 (0.0005) [2023-12-27 07:19:33,916][20347] Updated weights for policy 0, policy_version 242146 (0.0005) [2023-12-27 07:19:34,475][20347] Updated weights for policy 0, policy_version 242156 (0.0006) [2023-12-27 07:19:34,480][20381] Updated weights for policy 1, policy_version 241785 (0.0007) [2023-12-27 07:19:34,531][20347] Updated weights for policy 0, policy_version 242166 (0.0006) [2023-12-27 07:19:34,541][20381] Updated weights for policy 1, policy_version 241795 (0.0006) [2023-12-27 07:19:34,585][20347] Updated weights for policy 0, policy_version 242176 (0.0005) [2023-12-27 07:19:34,588][20381] Updated weights for policy 1, policy_version 241805 (0.0007) [2023-12-27 07:19:35,221][20381] Updated weights for policy 1, policy_version 241815 (0.0006) [2023-12-27 07:19:35,266][20347] Updated weights for policy 0, policy_version 242186 (0.0007) [2023-12-27 07:19:35,278][20381] Updated weights for policy 1, policy_version 241825 (0.0005) [2023-12-27 07:19:35,320][20347] Updated weights for policy 0, policy_version 242196 (0.0005) [2023-12-27 07:19:35,327][20381] Updated weights for policy 1, policy_version 241835 (0.0005) [2023-12-27 07:19:35,379][20347] Updated weights for policy 0, policy_version 242206 (0.0005) [2023-12-27 07:19:35,445][20347] Updated weights for policy 0, policy_version 242216 (0.0006) [2023-12-27 07:19:35,851][20381] Updated weights for policy 1, policy_version 241845 (0.0005) [2023-12-27 07:19:35,909][20381] Updated weights for policy 1, policy_version 241855 (0.0006) [2023-12-27 07:19:35,966][20381] Updated weights for policy 1, policy_version 241865 (0.0005) [2023-12-27 07:19:36,081][20347] Updated weights for policy 0, policy_version 242226 (0.0007) [2023-12-27 07:19:36,149][20347] Updated weights for policy 0, policy_version 242236 (0.0007) [2023-12-27 07:19:36,212][20347] Updated weights for policy 0, policy_version 242246 (0.0008) [2023-12-27 07:19:36,644][20381] Updated weights for policy 1, policy_version 241875 (0.0006) [2023-12-27 07:19:36,711][20381] Updated weights for policy 1, policy_version 241885 (0.0005) [2023-12-27 07:19:36,773][20381] Updated weights for policy 1, policy_version 241895 (0.0006) [2023-12-27 07:19:36,789][20347] Updated weights for policy 0, policy_version 242256 (0.0006) [2023-12-27 07:19:36,847][20347] Updated weights for policy 0, policy_version 242266 (0.0005) [2023-12-27 07:19:36,895][20347] Updated weights for policy 0, policy_version 242276 (0.0005) [2023-12-27 07:19:37,333][20381] Updated weights for policy 1, policy_version 241905 (0.0006) [2023-12-27 07:19:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 123977728. Throughput: 0: 10406.2, 1: 10408.4. Samples: 123946948. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '172.060')] [2023-12-27 07:19:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000242280_62038016.pth... [2023-12-27 07:19:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000241064_61726720.pth [2023-12-27 07:19:37,397][20381] Updated weights for policy 1, policy_version 241915 (0.0006) [2023-12-27 07:19:37,458][20381] Updated weights for policy 1, policy_version 241925 (0.0008) [2023-12-27 07:19:37,515][20381] Updated weights for policy 1, policy_version 241935 (0.0008) [2023-12-27 07:19:37,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000241936_61947904.pth... [2023-12-27 07:19:37,522][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000240688_61628416.pth [2023-12-27 07:19:37,598][20347] Updated weights for policy 0, policy_version 242286 (0.0007) [2023-12-27 07:19:37,660][20347] Updated weights for policy 0, policy_version 242296 (0.0006) [2023-12-27 07:19:37,721][20347] Updated weights for policy 0, policy_version 242306 (0.0007) [2023-12-27 07:19:38,207][20381] Updated weights for policy 1, policy_version 241945 (0.0008) [2023-12-27 07:19:38,254][20381] Updated weights for policy 1, policy_version 241955 (0.0008) [2023-12-27 07:19:38,311][20381] Updated weights for policy 1, policy_version 241965 (0.0008) [2023-12-27 07:19:38,388][20347] Updated weights for policy 0, policy_version 242316 (0.0007) [2023-12-27 07:19:38,441][20347] Updated weights for policy 0, policy_version 242326 (0.0007) [2023-12-27 07:19:38,490][20347] Updated weights for policy 0, policy_version 242336 (0.0008) [2023-12-27 07:19:39,014][20381] Updated weights for policy 1, policy_version 241975 (0.0008) [2023-12-27 07:19:39,069][20381] Updated weights for policy 1, policy_version 241985 (0.0008) [2023-12-27 07:19:39,121][20381] Updated weights for policy 1, policy_version 241995 (0.0005) [2023-12-27 07:19:39,209][20347] Updated weights for policy 0, policy_version 242346 (0.0007) [2023-12-27 07:19:39,265][20347] Updated weights for policy 0, policy_version 242356 (0.0007) [2023-12-27 07:19:39,323][20347] Updated weights for policy 0, policy_version 242366 (0.0009) [2023-12-27 07:19:39,387][20347] Updated weights for policy 0, policy_version 242376 (0.0007) [2023-12-27 07:19:39,762][20381] Updated weights for policy 1, policy_version 242005 (0.0006) [2023-12-27 07:19:39,821][20381] Updated weights for policy 1, policy_version 242015 (0.0008) [2023-12-27 07:19:39,876][20381] Updated weights for policy 1, policy_version 242025 (0.0008) [2023-12-27 07:19:40,092][20347] Updated weights for policy 0, policy_version 242386 (0.0009) [2023-12-27 07:19:40,149][20347] Updated weights for policy 0, policy_version 242396 (0.0007) [2023-12-27 07:19:40,202][20347] Updated weights for policy 0, policy_version 242406 (0.0008) [2023-12-27 07:19:40,615][20381] Updated weights for policy 1, policy_version 242035 (0.0008) [2023-12-27 07:19:40,673][20381] Updated weights for policy 1, policy_version 242045 (0.0008) [2023-12-27 07:19:40,720][20381] Updated weights for policy 1, policy_version 242055 (0.0008) [2023-12-27 07:19:40,856][20347] Updated weights for policy 0, policy_version 242416 (0.0007) [2023-12-27 07:19:40,919][20347] Updated weights for policy 0, policy_version 242426 (0.0005) [2023-12-27 07:19:40,976][20347] Updated weights for policy 0, policy_version 242436 (0.0007) [2023-12-27 07:19:41,470][20381] Updated weights for policy 1, policy_version 242065 (0.0008) [2023-12-27 07:19:41,519][20381] Updated weights for policy 1, policy_version 242075 (0.0007) [2023-12-27 07:19:41,579][20381] Updated weights for policy 1, policy_version 242085 (0.0008) [2023-12-27 07:19:41,635][20381] Updated weights for policy 1, policy_version 242095 (0.0007) [2023-12-27 07:19:41,720][20347] Updated weights for policy 0, policy_version 242446 (0.0008) [2023-12-27 07:19:41,777][20347] Updated weights for policy 0, policy_version 242456 (0.0007) [2023-12-27 07:19:41,838][20347] Updated weights for policy 0, policy_version 242466 (0.0008) [2023-12-27 07:19:42,344][20381] Updated weights for policy 1, policy_version 242105 (0.0009) [2023-12-27 07:19:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 124076032. Throughput: 0: 10394.9, 1: 10484.4. Samples: 124070904. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 07:19:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '185.350')] [2023-12-27 07:19:42,411][20381] Updated weights for policy 1, policy_version 242115 (0.0007) [2023-12-27 07:19:42,471][20381] Updated weights for policy 1, policy_version 242125 (0.0006) [2023-12-27 07:19:42,527][20347] Updated weights for policy 0, policy_version 242476 (0.0007) [2023-12-27 07:19:42,576][20347] Updated weights for policy 0, policy_version 242486 (0.0008) [2023-12-27 07:19:42,629][20347] Updated weights for policy 0, policy_version 242496 (0.0008) [2023-12-27 07:19:43,179][20381] Updated weights for policy 1, policy_version 242135 (0.0007) [2023-12-27 07:19:43,241][20381] Updated weights for policy 1, policy_version 242145 (0.0008) [2023-12-27 07:19:43,267][20347] Updated weights for policy 0, policy_version 242506 (0.0007) [2023-12-27 07:19:43,295][20381] Updated weights for policy 1, policy_version 242155 (0.0007) [2023-12-27 07:19:43,320][20347] Updated weights for policy 0, policy_version 242516 (0.0006) [2023-12-27 07:19:43,369][20347] Updated weights for policy 0, policy_version 242526 (0.0008) [2023-12-27 07:19:43,414][20347] Updated weights for policy 0, policy_version 242536 (0.0008) [2023-12-27 07:19:43,910][20381] Updated weights for policy 1, policy_version 242165 (0.0010) [2023-12-27 07:19:43,964][20381] Updated weights for policy 1, policy_version 242175 (0.0010) [2023-12-27 07:19:44,025][20381] Updated weights for policy 1, policy_version 242185 (0.0011) [2023-12-27 07:19:44,145][20347] Updated weights for policy 0, policy_version 242546 (0.0007) [2023-12-27 07:19:44,198][20347] Updated weights for policy 0, policy_version 242556 (0.0009) [2023-12-27 07:19:44,250][20347] Updated weights for policy 0, policy_version 242566 (0.0008) [2023-12-27 07:19:44,599][20381] Updated weights for policy 1, policy_version 242195 (0.0011) [2023-12-27 07:19:44,657][20381] Updated weights for policy 1, policy_version 242205 (0.0010) [2023-12-27 07:19:44,714][20381] Updated weights for policy 1, policy_version 242215 (0.0011) [2023-12-27 07:19:45,001][20347] Updated weights for policy 0, policy_version 242576 (0.0008) [2023-12-27 07:19:45,060][20347] Updated weights for policy 0, policy_version 242586 (0.0007) [2023-12-27 07:19:45,122][20347] Updated weights for policy 0, policy_version 242596 (0.0008) [2023-12-27 07:19:45,410][20381] Updated weights for policy 1, policy_version 242225 (0.0011) [2023-12-27 07:19:45,462][20381] Updated weights for policy 1, policy_version 242235 (0.0010) [2023-12-27 07:19:45,512][20381] Updated weights for policy 1, policy_version 242246 (0.0007) [2023-12-27 07:19:45,556][20381] Updated weights for policy 1, policy_version 242256 (0.0005) [2023-12-27 07:19:45,844][20347] Updated weights for policy 0, policy_version 242606 (0.0008) [2023-12-27 07:19:45,894][20347] Updated weights for policy 0, policy_version 242616 (0.0008) [2023-12-27 07:19:45,944][20347] Updated weights for policy 0, policy_version 242626 (0.0007) [2023-12-27 07:19:46,135][20381] Updated weights for policy 1, policy_version 242266 (0.0005) [2023-12-27 07:19:46,186][20381] Updated weights for policy 1, policy_version 242276 (0.0008) [2023-12-27 07:19:46,240][20381] Updated weights for policy 1, policy_version 242286 (0.0008) [2023-12-27 07:19:46,666][20347] Updated weights for policy 0, policy_version 242636 (0.0007) [2023-12-27 07:19:46,714][20347] Updated weights for policy 0, policy_version 242646 (0.0007) [2023-12-27 07:19:46,761][20347] Updated weights for policy 0, policy_version 242656 (0.0005) [2023-12-27 07:19:46,937][20381] Updated weights for policy 1, policy_version 242296 (0.0010) [2023-12-27 07:19:46,987][20381] Updated weights for policy 1, policy_version 242306 (0.0010) [2023-12-27 07:19:47,039][20381] Updated weights for policy 1, policy_version 242316 (0.0010) [2023-12-27 07:19:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 124182528. Throughput: 0: 10350.5, 1: 10452.2. Samples: 124193844. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:19:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '106.240')] [2023-12-27 07:19:47,371][20347] Updated weights for policy 0, policy_version 242666 (0.0007) [2023-12-27 07:19:47,418][20347] Updated weights for policy 0, policy_version 242676 (0.0010) [2023-12-27 07:19:47,473][20347] Updated weights for policy 0, policy_version 242686 (0.0010) [2023-12-27 07:19:47,530][20347] Updated weights for policy 0, policy_version 242696 (0.0010) [2023-12-27 07:19:47,728][20381] Updated weights for policy 1, policy_version 242326 (0.0011) [2023-12-27 07:19:47,781][20381] Updated weights for policy 1, policy_version 242336 (0.0010) [2023-12-27 07:19:47,846][20381] Updated weights for policy 1, policy_version 242346 (0.0011) [2023-12-27 07:19:48,097][20347] Updated weights for policy 0, policy_version 242706 (0.0011) [2023-12-27 07:19:48,158][20347] Updated weights for policy 0, policy_version 242716 (0.0010) [2023-12-27 07:19:48,210][20347] Updated weights for policy 0, policy_version 242726 (0.0011) [2023-12-27 07:19:48,551][20381] Updated weights for policy 1, policy_version 242356 (0.0011) [2023-12-27 07:19:48,601][20381] Updated weights for policy 1, policy_version 242366 (0.0011) [2023-12-27 07:19:48,656][20381] Updated weights for policy 1, policy_version 242376 (0.0010) [2023-12-27 07:19:48,900][20347] Updated weights for policy 0, policy_version 242736 (0.0011) [2023-12-27 07:19:48,958][20347] Updated weights for policy 0, policy_version 242746 (0.0010) [2023-12-27 07:19:49,024][20347] Updated weights for policy 0, policy_version 242756 (0.0010) [2023-12-27 07:19:49,323][20381] Updated weights for policy 1, policy_version 242386 (0.0011) [2023-12-27 07:19:49,381][20381] Updated weights for policy 1, policy_version 242396 (0.0011) [2023-12-27 07:19:49,431][20381] Updated weights for policy 1, policy_version 242406 (0.0007) [2023-12-27 07:19:49,488][20381] Updated weights for policy 1, policy_version 242416 (0.0005) [2023-12-27 07:19:49,684][20347] Updated weights for policy 0, policy_version 242766 (0.0007) [2023-12-27 07:19:49,740][20347] Updated weights for policy 0, policy_version 242776 (0.0008) [2023-12-27 07:19:49,798][20347] Updated weights for policy 0, policy_version 242786 (0.0010) [2023-12-27 07:19:50,180][20381] Updated weights for policy 1, policy_version 242426 (0.0010) [2023-12-27 07:19:50,233][20381] Updated weights for policy 1, policy_version 242436 (0.0009) [2023-12-27 07:19:50,295][20381] Updated weights for policy 1, policy_version 242446 (0.0010) [2023-12-27 07:19:50,460][20347] Updated weights for policy 0, policy_version 242796 (0.0010) [2023-12-27 07:19:50,517][20347] Updated weights for policy 0, policy_version 242806 (0.0010) [2023-12-27 07:19:50,576][20347] Updated weights for policy 0, policy_version 242816 (0.0009) [2023-12-27 07:19:50,996][20381] Updated weights for policy 1, policy_version 242456 (0.0011) [2023-12-27 07:19:51,061][20381] Updated weights for policy 1, policy_version 242466 (0.0009) [2023-12-27 07:19:51,118][20381] Updated weights for policy 1, policy_version 242476 (0.0007) [2023-12-27 07:19:51,206][20347] Updated weights for policy 0, policy_version 242826 (0.0008) [2023-12-27 07:19:51,266][20347] Updated weights for policy 0, policy_version 242836 (0.0009) [2023-12-27 07:19:51,326][20347] Updated weights for policy 0, policy_version 242846 (0.0010) [2023-12-27 07:19:51,392][20347] Updated weights for policy 0, policy_version 242856 (0.0011) [2023-12-27 07:19:51,834][20381] Updated weights for policy 1, policy_version 242486 (0.0008) [2023-12-27 07:19:51,898][20381] Updated weights for policy 1, policy_version 242496 (0.0008) [2023-12-27 07:19:51,953][20381] Updated weights for policy 1, policy_version 242506 (0.0006) [2023-12-27 07:19:52,100][20347] Updated weights for policy 0, policy_version 242866 (0.0007) [2023-12-27 07:19:52,148][20347] Updated weights for policy 0, policy_version 242876 (0.0007) [2023-12-27 07:19:52,201][20347] Updated weights for policy 0, policy_version 242886 (0.0008) [2023-12-27 07:19:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 124289024. Throughput: 0: 10356.0, 1: 10418.4. Samples: 124256256. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:19:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '498.500')] [2023-12-27 07:19:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000242512_62095360.pth... [2023-12-27 07:19:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000242888_62193664.pth... [2023-12-27 07:19:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000241296_61784064.pth [2023-12-27 07:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000241672_61882368.pth [2023-12-27 07:19:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000242512_62095360.pth [2023-12-27 07:19:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000242888_62193664.pth [2023-12-27 07:19:52,625][20381] Updated weights for policy 1, policy_version 242516 (0.0006) [2023-12-27 07:19:52,684][20381] Updated weights for policy 1, policy_version 242526 (0.0007) [2023-12-27 07:19:52,741][20381] Updated weights for policy 1, policy_version 242536 (0.0008) [2023-12-27 07:19:52,936][20347] Updated weights for policy 0, policy_version 242896 (0.0008) [2023-12-27 07:19:52,983][20347] Updated weights for policy 0, policy_version 242906 (0.0007) [2023-12-27 07:19:53,037][20347] Updated weights for policy 0, policy_version 242916 (0.0007) [2023-12-27 07:19:53,341][20381] Updated weights for policy 1, policy_version 242546 (0.0007) [2023-12-27 07:19:53,394][20381] Updated weights for policy 1, policy_version 242556 (0.0009) [2023-12-27 07:19:53,444][20381] Updated weights for policy 1, policy_version 242566 (0.0008) [2023-12-27 07:19:53,497][20381] Updated weights for policy 1, policy_version 242576 (0.0008) [2023-12-27 07:19:53,768][20347] Updated weights for policy 0, policy_version 242926 (0.0007) [2023-12-27 07:19:53,834][20347] Updated weights for policy 0, policy_version 242936 (0.0005) [2023-12-27 07:19:53,885][20347] Updated weights for policy 0, policy_version 242946 (0.0005) [2023-12-27 07:19:54,209][20381] Updated weights for policy 1, policy_version 242586 (0.0007) [2023-12-27 07:19:54,275][20381] Updated weights for policy 1, policy_version 242596 (0.0008) [2023-12-27 07:19:54,337][20381] Updated weights for policy 1, policy_version 242606 (0.0008) [2023-12-27 07:19:54,462][20347] Updated weights for policy 0, policy_version 242956 (0.0006) [2023-12-27 07:19:54,520][20347] Updated weights for policy 0, policy_version 242966 (0.0006) [2023-12-27 07:19:54,574][20347] Updated weights for policy 0, policy_version 242976 (0.0008) [2023-12-27 07:19:55,027][20381] Updated weights for policy 1, policy_version 242616 (0.0007) [2023-12-27 07:19:55,078][20381] Updated weights for policy 1, policy_version 242626 (0.0005) [2023-12-27 07:19:55,129][20381] Updated weights for policy 1, policy_version 242636 (0.0006) [2023-12-27 07:19:55,130][20347] Updated weights for policy 0, policy_version 242986 (0.0006) [2023-12-27 07:19:55,182][20347] Updated weights for policy 0, policy_version 242996 (0.0005) [2023-12-27 07:19:55,240][20347] Updated weights for policy 0, policy_version 243006 (0.0007) [2023-12-27 07:19:55,296][20347] Updated weights for policy 0, policy_version 243016 (0.0006) [2023-12-27 07:19:55,783][20381] Updated weights for policy 1, policy_version 242646 (0.0008) [2023-12-27 07:19:55,837][20381] Updated weights for policy 1, policy_version 242656 (0.0007) [2023-12-27 07:19:55,895][20381] Updated weights for policy 1, policy_version 242666 (0.0008) [2023-12-27 07:19:55,968][20347] Updated weights for policy 0, policy_version 243026 (0.0008) [2023-12-27 07:19:56,023][20347] Updated weights for policy 0, policy_version 243036 (0.0008) [2023-12-27 07:19:56,076][20347] Updated weights for policy 0, policy_version 243046 (0.0007) [2023-12-27 07:19:56,593][20381] Updated weights for policy 1, policy_version 242676 (0.0008) [2023-12-27 07:19:56,651][20381] Updated weights for policy 1, policy_version 242686 (0.0008) [2023-12-27 07:19:56,706][20347] Updated weights for policy 0, policy_version 243056 (0.0006) [2023-12-27 07:19:56,706][20381] Updated weights for policy 1, policy_version 242696 (0.0007) [2023-12-27 07:19:56,758][20347] Updated weights for policy 0, policy_version 243066 (0.0006) [2023-12-27 07:19:56,806][20347] Updated weights for policy 0, policy_version 243076 (0.0007) [2023-12-27 07:19:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 124387328. Throughput: 0: 10316.7, 1: 10401.2. Samples: 124379996. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:19:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '554.350')] [2023-12-27 07:19:57,413][20381] Updated weights for policy 1, policy_version 242706 (0.0006) [2023-12-27 07:19:57,436][20347] Updated weights for policy 0, policy_version 243086 (0.0006) [2023-12-27 07:19:57,469][20381] Updated weights for policy 1, policy_version 242716 (0.0006) [2023-12-27 07:19:57,484][20347] Updated weights for policy 0, policy_version 243096 (0.0006) [2023-12-27 07:19:57,529][20381] Updated weights for policy 1, policy_version 242726 (0.0008) [2023-12-27 07:19:57,531][20347] Updated weights for policy 0, policy_version 243106 (0.0006) [2023-12-27 07:19:57,585][20381] Updated weights for policy 1, policy_version 242736 (0.0008) [2023-12-27 07:19:58,110][20347] Updated weights for policy 0, policy_version 243116 (0.0007) [2023-12-27 07:19:58,163][20347] Updated weights for policy 0, policy_version 243126 (0.0006) [2023-12-27 07:19:58,223][20347] Updated weights for policy 0, policy_version 243136 (0.0008) [2023-12-27 07:19:58,315][20381] Updated weights for policy 1, policy_version 242746 (0.0008) [2023-12-27 07:19:58,383][20381] Updated weights for policy 1, policy_version 242756 (0.0009) [2023-12-27 07:19:58,445][20381] Updated weights for policy 1, policy_version 242766 (0.0006) [2023-12-27 07:19:58,968][20347] Updated weights for policy 0, policy_version 243146 (0.0007) [2023-12-27 07:19:59,028][20347] Updated weights for policy 0, policy_version 243156 (0.0005) [2023-12-27 07:19:59,078][20347] Updated weights for policy 0, policy_version 243166 (0.0005) [2023-12-27 07:19:59,128][20347] Updated weights for policy 0, policy_version 243176 (0.0006) [2023-12-27 07:19:59,152][20381] Updated weights for policy 1, policy_version 242776 (0.0007) [2023-12-27 07:19:59,202][20381] Updated weights for policy 1, policy_version 242786 (0.0007) [2023-12-27 07:19:59,265][20381] Updated weights for policy 1, policy_version 242796 (0.0008) [2023-12-27 07:19:59,815][20347] Updated weights for policy 0, policy_version 243186 (0.0007) [2023-12-27 07:19:59,843][20381] Updated weights for policy 1, policy_version 242806 (0.0008) [2023-12-27 07:19:59,873][20347] Updated weights for policy 0, policy_version 243196 (0.0006) [2023-12-27 07:19:59,906][20381] Updated weights for policy 1, policy_version 242816 (0.0007) [2023-12-27 07:19:59,934][20347] Updated weights for policy 0, policy_version 243206 (0.0007) [2023-12-27 07:19:59,965][20381] Updated weights for policy 1, policy_version 242826 (0.0006) [2023-12-27 07:20:00,582][20381] Updated weights for policy 1, policy_version 242836 (0.0006) [2023-12-27 07:20:00,629][20381] Updated weights for policy 1, policy_version 242846 (0.0007) [2023-12-27 07:20:00,650][20347] Updated weights for policy 0, policy_version 243216 (0.0006) [2023-12-27 07:20:00,680][20381] Updated weights for policy 1, policy_version 242856 (0.0005) [2023-12-27 07:20:00,694][20347] Updated weights for policy 0, policy_version 243226 (0.0007) [2023-12-27 07:20:00,746][20347] Updated weights for policy 0, policy_version 243236 (0.0007) [2023-12-27 07:20:01,349][20381] Updated weights for policy 1, policy_version 242866 (0.0006) [2023-12-27 07:20:01,398][20381] Updated weights for policy 1, policy_version 242876 (0.0009) [2023-12-27 07:20:01,450][20381] Updated weights for policy 1, policy_version 242886 (0.0010) [2023-12-27 07:20:01,461][20347] Updated weights for policy 0, policy_version 243246 (0.0007) [2023-12-27 07:20:01,495][20381] Updated weights for policy 1, policy_version 242896 (0.0010) [2023-12-27 07:20:01,513][20347] Updated weights for policy 0, policy_version 243256 (0.0007) [2023-12-27 07:20:01,567][20347] Updated weights for policy 0, policy_version 243266 (0.0007) [2023-12-27 07:20:02,188][20381] Updated weights for policy 1, policy_version 242906 (0.0008) [2023-12-27 07:20:02,239][20381] Updated weights for policy 1, policy_version 242916 (0.0008) [2023-12-27 07:20:02,255][20347] Updated weights for policy 0, policy_version 243276 (0.0007) [2023-12-27 07:20:02,292][20381] Updated weights for policy 1, policy_version 242926 (0.0008) [2023-12-27 07:20:02,315][20347] Updated weights for policy 0, policy_version 243286 (0.0007) [2023-12-27 07:20:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 124493824. Throughput: 0: 10366.6, 1: 10387.0. Samples: 124504592. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '705.890')] [2023-12-27 07:20:02,375][20347] Updated weights for policy 0, policy_version 243296 (0.0008) [2023-12-27 07:20:02,986][20347] Updated weights for policy 0, policy_version 243306 (0.0008) [2023-12-27 07:20:03,036][20381] Updated weights for policy 1, policy_version 242936 (0.0007) [2023-12-27 07:20:03,043][20347] Updated weights for policy 0, policy_version 243316 (0.0006) [2023-12-27 07:20:03,085][20381] Updated weights for policy 1, policy_version 242946 (0.0010) [2023-12-27 07:20:03,103][20347] Updated weights for policy 0, policy_version 243326 (0.0005) [2023-12-27 07:20:03,140][20381] Updated weights for policy 1, policy_version 242956 (0.0010) [2023-12-27 07:20:03,161][20347] Updated weights for policy 0, policy_version 243336 (0.0010) [2023-12-27 07:20:03,789][20347] Updated weights for policy 0, policy_version 243346 (0.0007) [2023-12-27 07:20:03,814][20381] Updated weights for policy 1, policy_version 242966 (0.0010) [2023-12-27 07:20:03,852][20347] Updated weights for policy 0, policy_version 243356 (0.0007) [2023-12-27 07:20:03,903][20381] Updated weights for policy 1, policy_version 242976 (0.0007) [2023-12-27 07:20:03,905][20347] Updated weights for policy 0, policy_version 243366 (0.0007) [2023-12-27 07:20:03,971][20381] Updated weights for policy 1, policy_version 242986 (0.0006) [2023-12-27 07:20:04,533][20347] Updated weights for policy 0, policy_version 243376 (0.0006) [2023-12-27 07:20:04,586][20347] Updated weights for policy 0, policy_version 243386 (0.0005) [2023-12-27 07:20:04,637][20347] Updated weights for policy 0, policy_version 243396 (0.0005) [2023-12-27 07:20:04,665][20381] Updated weights for policy 1, policy_version 242996 (0.0008) [2023-12-27 07:20:04,726][20381] Updated weights for policy 1, policy_version 243006 (0.0009) [2023-12-27 07:20:04,785][20381] Updated weights for policy 1, policy_version 243016 (0.0008) [2023-12-27 07:20:05,249][20347] Updated weights for policy 0, policy_version 243406 (0.0006) [2023-12-27 07:20:05,294][20347] Updated weights for policy 0, policy_version 243416 (0.0008) [2023-12-27 07:20:05,352][20347] Updated weights for policy 0, policy_version 243426 (0.0008) [2023-12-27 07:20:05,411][20381] Updated weights for policy 1, policy_version 243026 (0.0008) [2023-12-27 07:20:05,455][20381] Updated weights for policy 1, policy_version 243036 (0.0005) [2023-12-27 07:20:05,505][20381] Updated weights for policy 1, policy_version 243046 (0.0005) [2023-12-27 07:20:05,553][20381] Updated weights for policy 1, policy_version 243056 (0.0005) [2023-12-27 07:20:06,040][20347] Updated weights for policy 0, policy_version 243436 (0.0009) [2023-12-27 07:20:06,098][20347] Updated weights for policy 0, policy_version 243446 (0.0006) [2023-12-27 07:20:06,139][20381] Updated weights for policy 1, policy_version 243066 (0.0008) [2023-12-27 07:20:06,159][20347] Updated weights for policy 0, policy_version 243456 (0.0008) [2023-12-27 07:20:06,195][20381] Updated weights for policy 1, policy_version 243076 (0.0006) [2023-12-27 07:20:06,252][20381] Updated weights for policy 1, policy_version 243086 (0.0008) [2023-12-27 07:20:06,854][20347] Updated weights for policy 0, policy_version 243466 (0.0006) [2023-12-27 07:20:06,917][20347] Updated weights for policy 0, policy_version 243476 (0.0007) [2023-12-27 07:20:06,949][20381] Updated weights for policy 1, policy_version 243096 (0.0006) [2023-12-27 07:20:06,973][20347] Updated weights for policy 0, policy_version 243486 (0.0007) [2023-12-27 07:20:07,013][20381] Updated weights for policy 1, policy_version 243106 (0.0008) [2023-12-27 07:20:07,022][20347] Updated weights for policy 0, policy_version 243496 (0.0005) [2023-12-27 07:20:07,077][20381] Updated weights for policy 1, policy_version 243116 (0.0008) [2023-12-27 07:20:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 124600320. Throughput: 0: 10430.3, 1: 10390.0. Samples: 124567500. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '778.340')] [2023-12-27 07:20:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000243120_62251008.pth... [2023-12-27 07:20:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000243496_62349312.pth... [2023-12-27 07:20:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000241936_61947904.pth [2023-12-27 07:20:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000242280_62038016.pth [2023-12-27 07:20:07,680][20347] Updated weights for policy 0, policy_version 243506 (0.0006) [2023-12-27 07:20:07,746][20347] Updated weights for policy 0, policy_version 243516 (0.0008) [2023-12-27 07:20:07,760][20381] Updated weights for policy 1, policy_version 243126 (0.0006) [2023-12-27 07:20:07,805][20347] Updated weights for policy 0, policy_version 243526 (0.0008) [2023-12-27 07:20:07,824][20381] Updated weights for policy 1, policy_version 243136 (0.0007) [2023-12-27 07:20:07,884][20381] Updated weights for policy 1, policy_version 243146 (0.0008) [2023-12-27 07:20:08,445][20347] Updated weights for policy 0, policy_version 243536 (0.0008) [2023-12-27 07:20:08,500][20347] Updated weights for policy 0, policy_version 243546 (0.0007) [2023-12-27 07:20:08,555][20347] Updated weights for policy 0, policy_version 243556 (0.0008) [2023-12-27 07:20:08,576][20381] Updated weights for policy 1, policy_version 243156 (0.0007) [2023-12-27 07:20:08,636][20381] Updated weights for policy 1, policy_version 243166 (0.0009) [2023-12-27 07:20:08,687][20381] Updated weights for policy 1, policy_version 243176 (0.0007) [2023-12-27 07:20:09,204][20347] Updated weights for policy 0, policy_version 243566 (0.0007) [2023-12-27 07:20:09,269][20347] Updated weights for policy 0, policy_version 243576 (0.0007) [2023-12-27 07:20:09,295][20381] Updated weights for policy 1, policy_version 243186 (0.0007) [2023-12-27 07:20:09,337][20347] Updated weights for policy 0, policy_version 243586 (0.0007) [2023-12-27 07:20:09,365][20381] Updated weights for policy 1, policy_version 243196 (0.0007) [2023-12-27 07:20:09,429][20381] Updated weights for policy 1, policy_version 243206 (0.0007) [2023-12-27 07:20:09,481][20381] Updated weights for policy 1, policy_version 243216 (0.0007) [2023-12-27 07:20:10,064][20347] Updated weights for policy 0, policy_version 243596 (0.0007) [2023-12-27 07:20:10,113][20347] Updated weights for policy 0, policy_version 243606 (0.0007) [2023-12-27 07:20:10,164][20347] Updated weights for policy 0, policy_version 243616 (0.0007) [2023-12-27 07:20:10,206][20381] Updated weights for policy 1, policy_version 243226 (0.0006) [2023-12-27 07:20:10,267][20381] Updated weights for policy 1, policy_version 243236 (0.0008) [2023-12-27 07:20:10,326][20381] Updated weights for policy 1, policy_version 243246 (0.0008) [2023-12-27 07:20:10,866][20347] Updated weights for policy 0, policy_version 243626 (0.0006) [2023-12-27 07:20:10,925][20347] Updated weights for policy 0, policy_version 243636 (0.0008) [2023-12-27 07:20:10,982][20347] Updated weights for policy 0, policy_version 243646 (0.0006) [2023-12-27 07:20:10,993][20381] Updated weights for policy 1, policy_version 243256 (0.0010) [2023-12-27 07:20:11,046][20347] Updated weights for policy 0, policy_version 243656 (0.0010) [2023-12-27 07:20:11,056][20381] Updated weights for policy 1, policy_version 243266 (0.0008) [2023-12-27 07:20:11,111][20381] Updated weights for policy 1, policy_version 243276 (0.0011) [2023-12-27 07:20:11,769][20347] Updated weights for policy 0, policy_version 243666 (0.0009) [2023-12-27 07:20:11,803][20381] Updated weights for policy 1, policy_version 243286 (0.0008) [2023-12-27 07:20:11,830][20347] Updated weights for policy 0, policy_version 243676 (0.0006) [2023-12-27 07:20:11,859][20381] Updated weights for policy 1, policy_version 243296 (0.0006) [2023-12-27 07:20:11,889][20347] Updated weights for policy 0, policy_version 243686 (0.0005) [2023-12-27 07:20:11,920][20381] Updated weights for policy 1, policy_version 243306 (0.0007) [2023-12-27 07:20:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 124698624. Throughput: 0: 10451.4, 1: 10414.3. Samples: 124690724. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '837.900')] [2023-12-27 07:20:12,481][20347] Updated weights for policy 0, policy_version 243696 (0.0007) [2023-12-27 07:20:12,540][20347] Updated weights for policy 0, policy_version 243706 (0.0008) [2023-12-27 07:20:12,597][20347] Updated weights for policy 0, policy_version 243716 (0.0008) [2023-12-27 07:20:12,623][20381] Updated weights for policy 1, policy_version 243316 (0.0008) [2023-12-27 07:20:12,675][20381] Updated weights for policy 1, policy_version 243326 (0.0007) [2023-12-27 07:20:12,728][20381] Updated weights for policy 1, policy_version 243336 (0.0008) [2023-12-27 07:20:13,248][20347] Updated weights for policy 0, policy_version 243726 (0.0007) [2023-12-27 07:20:13,305][20347] Updated weights for policy 0, policy_version 243736 (0.0006) [2023-12-27 07:20:13,362][20347] Updated weights for policy 0, policy_version 243746 (0.0006) [2023-12-27 07:20:13,449][20381] Updated weights for policy 1, policy_version 243346 (0.0009) [2023-12-27 07:20:13,504][20381] Updated weights for policy 1, policy_version 243356 (0.0008) [2023-12-27 07:20:13,554][20381] Updated weights for policy 1, policy_version 243366 (0.0007) [2023-12-27 07:20:13,602][20381] Updated weights for policy 1, policy_version 243376 (0.0007) [2023-12-27 07:20:13,945][20347] Updated weights for policy 0, policy_version 243756 (0.0005) [2023-12-27 07:20:13,991][20347] Updated weights for policy 0, policy_version 243766 (0.0006) [2023-12-27 07:20:14,046][20347] Updated weights for policy 0, policy_version 243777 (0.0008) [2023-12-27 07:20:14,316][20381] Updated weights for policy 1, policy_version 243386 (0.0008) [2023-12-27 07:20:14,368][20381] Updated weights for policy 1, policy_version 243396 (0.0007) [2023-12-27 07:20:14,419][20381] Updated weights for policy 1, policy_version 243406 (0.0005) [2023-12-27 07:20:14,703][20347] Updated weights for policy 0, policy_version 243787 (0.0006) [2023-12-27 07:20:14,755][20347] Updated weights for policy 0, policy_version 243797 (0.0005) [2023-12-27 07:20:14,822][20347] Updated weights for policy 0, policy_version 243807 (0.0006) [2023-12-27 07:20:15,088][20381] Updated weights for policy 1, policy_version 243416 (0.0007) [2023-12-27 07:20:15,149][20381] Updated weights for policy 1, policy_version 243426 (0.0006) [2023-12-27 07:20:15,208][20381] Updated weights for policy 1, policy_version 243436 (0.0006) [2023-12-27 07:20:15,497][20347] Updated weights for policy 0, policy_version 243817 (0.0006) [2023-12-27 07:20:15,556][20347] Updated weights for policy 0, policy_version 243827 (0.0008) [2023-12-27 07:20:15,619][20347] Updated weights for policy 0, policy_version 243837 (0.0008) [2023-12-27 07:20:15,678][20347] Updated weights for policy 0, policy_version 243847 (0.0007) [2023-12-27 07:20:15,776][20381] Updated weights for policy 1, policy_version 243446 (0.0007) [2023-12-27 07:20:15,837][20381] Updated weights for policy 1, policy_version 243456 (0.0005) [2023-12-27 07:20:15,893][20381] Updated weights for policy 1, policy_version 243466 (0.0008) [2023-12-27 07:20:16,307][20347] Updated weights for policy 0, policy_version 243857 (0.0005) [2023-12-27 07:20:16,353][20347] Updated weights for policy 0, policy_version 243867 (0.0005) [2023-12-27 07:20:16,411][20347] Updated weights for policy 0, policy_version 243877 (0.0006) [2023-12-27 07:20:16,511][20381] Updated weights for policy 1, policy_version 243476 (0.0007) [2023-12-27 07:20:16,571][20381] Updated weights for policy 1, policy_version 243486 (0.0010) [2023-12-27 07:20:16,632][20381] Updated weights for policy 1, policy_version 243496 (0.0011) [2023-12-27 07:20:17,023][20347] Updated weights for policy 0, policy_version 243887 (0.0007) [2023-12-27 07:20:17,066][20347] Updated weights for policy 0, policy_version 243897 (0.0005) [2023-12-27 07:20:17,112][20347] Updated weights for policy 0, policy_version 243907 (0.0005) [2023-12-27 07:20:17,279][20381] Updated weights for policy 1, policy_version 243506 (0.0011) [2023-12-27 07:20:17,324][20381] Updated weights for policy 1, policy_version 243516 (0.0006) [2023-12-27 07:20:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 124805120. Throughput: 0: 10451.1, 1: 10358.3. Samples: 124817744. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '914.180')] [2023-12-27 07:20:17,372][20381] Updated weights for policy 1, policy_version 243526 (0.0005) [2023-12-27 07:20:17,423][20381] Updated weights for policy 1, policy_version 243536 (0.0005) [2023-12-27 07:20:17,759][20347] Updated weights for policy 0, policy_version 243917 (0.0008) [2023-12-27 07:20:17,815][20347] Updated weights for policy 0, policy_version 243927 (0.0010) [2023-12-27 07:20:17,871][20347] Updated weights for policy 0, policy_version 243937 (0.0011) [2023-12-27 07:20:18,064][20381] Updated weights for policy 1, policy_version 243546 (0.0007) [2023-12-27 07:20:18,121][20381] Updated weights for policy 1, policy_version 243556 (0.0007) [2023-12-27 07:20:18,175][20381] Updated weights for policy 1, policy_version 243566 (0.0007) [2023-12-27 07:20:18,525][20347] Updated weights for policy 0, policy_version 243947 (0.0009) [2023-12-27 07:20:18,579][20347] Updated weights for policy 0, policy_version 243957 (0.0009) [2023-12-27 07:20:18,628][20347] Updated weights for policy 0, policy_version 243967 (0.0010) [2023-12-27 07:20:18,920][20381] Updated weights for policy 1, policy_version 243576 (0.0007) [2023-12-27 07:20:18,985][20381] Updated weights for policy 1, policy_version 243586 (0.0007) [2023-12-27 07:20:19,033][20381] Updated weights for policy 1, policy_version 243596 (0.0007) [2023-12-27 07:20:19,297][20347] Updated weights for policy 0, policy_version 243977 (0.0010) [2023-12-27 07:20:19,357][20347] Updated weights for policy 0, policy_version 243987 (0.0010) [2023-12-27 07:20:19,424][20347] Updated weights for policy 0, policy_version 243997 (0.0011) [2023-12-27 07:20:19,487][20347] Updated weights for policy 0, policy_version 244007 (0.0010) [2023-12-27 07:20:19,695][20381] Updated weights for policy 1, policy_version 243606 (0.0008) [2023-12-27 07:20:19,754][20381] Updated weights for policy 1, policy_version 243616 (0.0007) [2023-12-27 07:20:19,817][20381] Updated weights for policy 1, policy_version 243626 (0.0008) [2023-12-27 07:20:20,180][20347] Updated weights for policy 0, policy_version 244017 (0.0006) [2023-12-27 07:20:20,227][20347] Updated weights for policy 0, policy_version 244027 (0.0006) [2023-12-27 07:20:20,283][20347] Updated weights for policy 0, policy_version 244037 (0.0006) [2023-12-27 07:20:20,512][20381] Updated weights for policy 1, policy_version 243636 (0.0007) [2023-12-27 07:20:20,576][20381] Updated weights for policy 1, policy_version 243646 (0.0006) [2023-12-27 07:20:20,628][20381] Updated weights for policy 1, policy_version 243656 (0.0005) [2023-12-27 07:20:20,948][20347] Updated weights for policy 0, policy_version 244047 (0.0007) [2023-12-27 07:20:21,008][20347] Updated weights for policy 0, policy_version 244057 (0.0008) [2023-12-27 07:20:21,067][20347] Updated weights for policy 0, policy_version 244067 (0.0008) [2023-12-27 07:20:21,319][20381] Updated weights for policy 1, policy_version 243666 (0.0010) [2023-12-27 07:20:21,380][20381] Updated weights for policy 1, policy_version 243676 (0.0011) [2023-12-27 07:20:21,440][20381] Updated weights for policy 1, policy_version 243686 (0.0011) [2023-12-27 07:20:21,492][20381] Updated weights for policy 1, policy_version 243696 (0.0010) [2023-12-27 07:20:21,795][20347] Updated weights for policy 0, policy_version 244077 (0.0006) [2023-12-27 07:20:21,845][20347] Updated weights for policy 0, policy_version 244087 (0.0008) [2023-12-27 07:20:21,915][20347] Updated weights for policy 0, policy_version 244097 (0.0008) [2023-12-27 07:20:22,188][20381] Updated weights for policy 1, policy_version 243706 (0.0006) [2023-12-27 07:20:22,252][20381] Updated weights for policy 1, policy_version 243716 (0.0006) [2023-12-27 07:20:22,318][20381] Updated weights for policy 1, policy_version 243726 (0.0009) [2023-12-27 07:20:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 124911616. Throughput: 0: 10413.8, 1: 10299.8. Samples: 124879060. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:22,364][19170] Avg episode reward: [(0, '1000.000'), (1, '875.200')] [2023-12-27 07:20:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000243728_62406656.pth... [2023-12-27 07:20:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000244104_62504960.pth... [2023-12-27 07:20:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000242512_62095360.pth [2023-12-27 07:20:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000242888_62193664.pth [2023-12-27 07:20:22,625][20347] Updated weights for policy 0, policy_version 244107 (0.0007) [2023-12-27 07:20:22,680][20347] Updated weights for policy 0, policy_version 244117 (0.0008) [2023-12-27 07:20:22,734][20347] Updated weights for policy 0, policy_version 244127 (0.0008) [2023-12-27 07:20:23,025][20381] Updated weights for policy 1, policy_version 243736 (0.0008) [2023-12-27 07:20:23,079][20381] Updated weights for policy 1, policy_version 243746 (0.0008) [2023-12-27 07:20:23,129][20381] Updated weights for policy 1, policy_version 243756 (0.0008) [2023-12-27 07:20:23,398][20347] Updated weights for policy 0, policy_version 244137 (0.0007) [2023-12-27 07:20:23,457][20347] Updated weights for policy 0, policy_version 244147 (0.0008) [2023-12-27 07:20:23,503][20347] Updated weights for policy 0, policy_version 244157 (0.0008) [2023-12-27 07:20:23,549][20347] Updated weights for policy 0, policy_version 244167 (0.0007) [2023-12-27 07:20:23,780][20381] Updated weights for policy 1, policy_version 243766 (0.0008) [2023-12-27 07:20:23,838][20381] Updated weights for policy 1, policy_version 243776 (0.0008) [2023-12-27 07:20:23,891][20381] Updated weights for policy 1, policy_version 243786 (0.0007) [2023-12-27 07:20:24,269][20347] Updated weights for policy 0, policy_version 244177 (0.0010) [2023-12-27 07:20:24,326][20347] Updated weights for policy 0, policy_version 244187 (0.0010) [2023-12-27 07:20:24,391][20347] Updated weights for policy 0, policy_version 244197 (0.0010) [2023-12-27 07:20:24,584][20381] Updated weights for policy 1, policy_version 243796 (0.0008) [2023-12-27 07:20:24,635][20381] Updated weights for policy 1, policy_version 243806 (0.0007) [2023-12-27 07:20:24,685][20292] KL-divergence is very high: 209.9401 [2023-12-27 07:20:24,686][20381] Updated weights for policy 1, policy_version 243816 (0.0006) [2023-12-27 07:20:24,691][20292] KL-divergence is very high: 197.4349 [2023-12-27 07:20:25,055][20347] Updated weights for policy 0, policy_version 244207 (0.0009) [2023-12-27 07:20:25,107][20347] Updated weights for policy 0, policy_version 244217 (0.0010) [2023-12-27 07:20:25,153][20347] Updated weights for policy 0, policy_version 244227 (0.0010) [2023-12-27 07:20:25,286][20381] Updated weights for policy 1, policy_version 243826 (0.0006) [2023-12-27 07:20:25,344][20381] Updated weights for policy 1, policy_version 243836 (0.0007) [2023-12-27 07:20:25,393][20381] Updated weights for policy 1, policy_version 243846 (0.0008) [2023-12-27 07:20:25,443][20381] Updated weights for policy 1, policy_version 243856 (0.0006) [2023-12-27 07:20:25,776][20347] Updated weights for policy 0, policy_version 244237 (0.0010) [2023-12-27 07:20:25,817][20347] Updated weights for policy 0, policy_version 244247 (0.0005) [2023-12-27 07:20:25,872][20347] Updated weights for policy 0, policy_version 244257 (0.0005) [2023-12-27 07:20:26,047][20381] Updated weights for policy 1, policy_version 243866 (0.0010) [2023-12-27 07:20:26,101][20381] Updated weights for policy 1, policy_version 243876 (0.0010) [2023-12-27 07:20:26,149][20381] Updated weights for policy 1, policy_version 243886 (0.0010) [2023-12-27 07:20:26,525][20347] Updated weights for policy 0, policy_version 244267 (0.0008) [2023-12-27 07:20:26,579][20347] Updated weights for policy 0, policy_version 244277 (0.0010) [2023-12-27 07:20:26,633][20347] Updated weights for policy 0, policy_version 244287 (0.0010) [2023-12-27 07:20:26,786][20381] Updated weights for policy 1, policy_version 243896 (0.0010) [2023-12-27 07:20:26,836][20381] Updated weights for policy 1, policy_version 243906 (0.0010) [2023-12-27 07:20:26,887][20381] Updated weights for policy 1, policy_version 243916 (0.0010) [2023-12-27 07:20:27,270][20347] Updated weights for policy 0, policy_version 244297 (0.0010) [2023-12-27 07:20:27,334][20347] Updated weights for policy 0, policy_version 244307 (0.0010) [2023-12-27 07:20:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 125009920. Throughput: 0: 10420.8, 1: 10296.6. Samples: 125003188. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.730')] [2023-12-27 07:20:27,394][20347] Updated weights for policy 0, policy_version 244317 (0.0010) [2023-12-27 07:20:27,453][20347] Updated weights for policy 0, policy_version 244327 (0.0010) [2023-12-27 07:20:27,529][20381] Updated weights for policy 1, policy_version 243926 (0.0010) [2023-12-27 07:20:27,579][20381] Updated weights for policy 1, policy_version 243936 (0.0010) [2023-12-27 07:20:27,627][20381] Updated weights for policy 1, policy_version 243946 (0.0010) [2023-12-27 07:20:28,060][20347] Updated weights for policy 0, policy_version 244337 (0.0008) [2023-12-27 07:20:28,119][20347] Updated weights for policy 0, policy_version 244347 (0.0010) [2023-12-27 07:20:28,179][20347] Updated weights for policy 0, policy_version 244357 (0.0010) [2023-12-27 07:20:28,336][20381] Updated weights for policy 1, policy_version 243956 (0.0009) [2023-12-27 07:20:28,391][20381] Updated weights for policy 1, policy_version 243966 (0.0009) [2023-12-27 07:20:28,446][20381] Updated weights for policy 1, policy_version 243976 (0.0009) [2023-12-27 07:20:28,866][20347] Updated weights for policy 0, policy_version 244367 (0.0011) [2023-12-27 07:20:28,927][20347] Updated weights for policy 0, policy_version 244377 (0.0010) [2023-12-27 07:20:28,983][20347] Updated weights for policy 0, policy_version 244387 (0.0010) [2023-12-27 07:20:29,013][20381] Updated weights for policy 1, policy_version 243986 (0.0010) [2023-12-27 07:20:29,075][20381] Updated weights for policy 1, policy_version 243996 (0.0011) [2023-12-27 07:20:29,134][20381] Updated weights for policy 1, policy_version 244006 (0.0011) [2023-12-27 07:20:29,189][20381] Updated weights for policy 1, policy_version 244016 (0.0011) [2023-12-27 07:20:29,669][20347] Updated weights for policy 0, policy_version 244397 (0.0010) [2023-12-27 07:20:29,723][20347] Updated weights for policy 0, policy_version 244407 (0.0010) [2023-12-27 07:20:29,781][20347] Updated weights for policy 0, policy_version 244417 (0.0010) [2023-12-27 07:20:29,890][20381] Updated weights for policy 1, policy_version 244026 (0.0007) [2023-12-27 07:20:29,952][20381] Updated weights for policy 1, policy_version 244036 (0.0008) [2023-12-27 07:20:30,016][20381] Updated weights for policy 1, policy_version 244046 (0.0006) [2023-12-27 07:20:30,470][20347] Updated weights for policy 0, policy_version 244427 (0.0010) [2023-12-27 07:20:30,520][20347] Updated weights for policy 0, policy_version 244437 (0.0008) [2023-12-27 07:20:30,577][20347] Updated weights for policy 0, policy_version 244447 (0.0005) [2023-12-27 07:20:30,658][20381] Updated weights for policy 1, policy_version 244056 (0.0008) [2023-12-27 07:20:30,703][20381] Updated weights for policy 1, policy_version 244066 (0.0010) [2023-12-27 07:20:30,758][20381] Updated weights for policy 1, policy_version 244076 (0.0007) [2023-12-27 07:20:31,172][20347] Updated weights for policy 0, policy_version 244457 (0.0006) [2023-12-27 07:20:31,230][20347] Updated weights for policy 0, policy_version 244467 (0.0010) [2023-12-27 07:20:31,290][20347] Updated weights for policy 0, policy_version 244477 (0.0008) [2023-12-27 07:20:31,348][20347] Updated weights for policy 0, policy_version 244487 (0.0007) [2023-12-27 07:20:31,489][20381] Updated weights for policy 1, policy_version 244086 (0.0008) [2023-12-27 07:20:31,545][20381] Updated weights for policy 1, policy_version 244096 (0.0007) [2023-12-27 07:20:31,601][20381] Updated weights for policy 1, policy_version 244106 (0.0007) [2023-12-27 07:20:32,040][20347] Updated weights for policy 0, policy_version 244497 (0.0006) [2023-12-27 07:20:32,092][20347] Updated weights for policy 0, policy_version 244507 (0.0007) [2023-12-27 07:20:32,151][20347] Updated weights for policy 0, policy_version 244517 (0.0008) [2023-12-27 07:20:32,311][20381] Updated weights for policy 1, policy_version 244116 (0.0007) [2023-12-27 07:20:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 125116416. Throughput: 0: 10481.6, 1: 10293.6. Samples: 125128728. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '942.820')] [2023-12-27 07:20:32,379][20381] Updated weights for policy 1, policy_version 244126 (0.0009) [2023-12-27 07:20:32,436][20381] Updated weights for policy 1, policy_version 244136 (0.0008) [2023-12-27 07:20:32,708][20347] Updated weights for policy 0, policy_version 244527 (0.0007) [2023-12-27 07:20:32,758][20347] Updated weights for policy 0, policy_version 244537 (0.0006) [2023-12-27 07:20:32,810][20347] Updated weights for policy 0, policy_version 244547 (0.0008) [2023-12-27 07:20:33,028][20381] Updated weights for policy 1, policy_version 244146 (0.0008) [2023-12-27 07:20:33,090][20381] Updated weights for policy 1, policy_version 244156 (0.0008) [2023-12-27 07:20:33,157][20381] Updated weights for policy 1, policy_version 244166 (0.0007) [2023-12-27 07:20:33,218][20381] Updated weights for policy 1, policy_version 244176 (0.0008) [2023-12-27 07:20:33,441][20347] Updated weights for policy 0, policy_version 244557 (0.0007) [2023-12-27 07:20:33,495][20347] Updated weights for policy 0, policy_version 244567 (0.0005) [2023-12-27 07:20:33,547][20347] Updated weights for policy 0, policy_version 244577 (0.0005) [2023-12-27 07:20:33,758][20381] Updated weights for policy 1, policy_version 244186 (0.0008) [2023-12-27 07:20:33,820][20381] Updated weights for policy 1, policy_version 244196 (0.0008) [2023-12-27 07:20:33,870][20381] Updated weights for policy 1, policy_version 244206 (0.0008) [2023-12-27 07:20:34,186][20347] Updated weights for policy 0, policy_version 244587 (0.0007) [2023-12-27 07:20:34,253][20347] Updated weights for policy 0, policy_version 244597 (0.0008) [2023-12-27 07:20:34,315][20347] Updated weights for policy 0, policy_version 244607 (0.0007) [2023-12-27 07:20:34,581][20381] Updated weights for policy 1, policy_version 244216 (0.0006) [2023-12-27 07:20:34,634][20381] Updated weights for policy 1, policy_version 244226 (0.0008) [2023-12-27 07:20:34,687][20381] Updated weights for policy 1, policy_version 244236 (0.0007) [2023-12-27 07:20:34,953][20347] Updated weights for policy 0, policy_version 244617 (0.0008) [2023-12-27 07:20:35,019][20347] Updated weights for policy 0, policy_version 244627 (0.0008) [2023-12-27 07:20:35,081][20347] Updated weights for policy 0, policy_version 244637 (0.0007) [2023-12-27 07:20:35,140][20347] Updated weights for policy 0, policy_version 244647 (0.0007) [2023-12-27 07:20:35,329][20381] Updated weights for policy 1, policy_version 244246 (0.0007) [2023-12-27 07:20:35,382][20381] Updated weights for policy 1, policy_version 244256 (0.0005) [2023-12-27 07:20:35,433][20381] Updated weights for policy 1, policy_version 244266 (0.0005) [2023-12-27 07:20:35,835][20347] Updated weights for policy 0, policy_version 244657 (0.0008) [2023-12-27 07:20:35,889][20347] Updated weights for policy 0, policy_version 244667 (0.0007) [2023-12-27 07:20:35,946][20347] Updated weights for policy 0, policy_version 244677 (0.0007) [2023-12-27 07:20:36,078][20381] Updated weights for policy 1, policy_version 244276 (0.0005) [2023-12-27 07:20:36,135][20381] Updated weights for policy 1, policy_version 244286 (0.0007) [2023-12-27 07:20:36,187][20381] Updated weights for policy 1, policy_version 244296 (0.0008) [2023-12-27 07:20:36,621][20347] Updated weights for policy 0, policy_version 244687 (0.0007) [2023-12-27 07:20:36,675][20347] Updated weights for policy 0, policy_version 244697 (0.0007) [2023-12-27 07:20:36,735][20347] Updated weights for policy 0, policy_version 244707 (0.0007) [2023-12-27 07:20:36,909][20381] Updated weights for policy 1, policy_version 244306 (0.0008) [2023-12-27 07:20:36,965][20381] Updated weights for policy 1, policy_version 244316 (0.0008) [2023-12-27 07:20:37,014][20381] Updated weights for policy 1, policy_version 244326 (0.0005) [2023-12-27 07:20:37,070][20381] Updated weights for policy 1, policy_version 244336 (0.0008) [2023-12-27 07:20:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 125222912. Throughput: 0: 10495.7, 1: 10343.1. Samples: 125194004. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000244336_62562304.pth... [2023-12-27 07:20:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000244712_62660608.pth... [2023-12-27 07:20:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000243496_62349312.pth [2023-12-27 07:20:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000243120_62251008.pth [2023-12-27 07:20:37,414][20347] Updated weights for policy 0, policy_version 244717 (0.0007) [2023-12-27 07:20:37,477][20347] Updated weights for policy 0, policy_version 244727 (0.0007) [2023-12-27 07:20:37,537][20347] Updated weights for policy 0, policy_version 244737 (0.0008) [2023-12-27 07:20:37,758][20381] Updated weights for policy 1, policy_version 244346 (0.0008) [2023-12-27 07:20:37,819][20381] Updated weights for policy 1, policy_version 244356 (0.0007) [2023-12-27 07:20:37,877][20381] Updated weights for policy 1, policy_version 244366 (0.0010) [2023-12-27 07:20:38,230][20347] Updated weights for policy 0, policy_version 244747 (0.0007) [2023-12-27 07:20:38,288][20347] Updated weights for policy 0, policy_version 244757 (0.0006) [2023-12-27 07:20:38,353][20347] Updated weights for policy 0, policy_version 244767 (0.0008) [2023-12-27 07:20:38,583][20381] Updated weights for policy 1, policy_version 244376 (0.0010) [2023-12-27 07:20:38,642][20381] Updated weights for policy 1, policy_version 244386 (0.0009) [2023-12-27 07:20:38,703][20381] Updated weights for policy 1, policy_version 244396 (0.0010) [2023-12-27 07:20:38,985][20347] Updated weights for policy 0, policy_version 244777 (0.0007) [2023-12-27 07:20:39,033][20347] Updated weights for policy 0, policy_version 244787 (0.0005) [2023-12-27 07:20:39,083][20347] Updated weights for policy 0, policy_version 244797 (0.0005) [2023-12-27 07:20:39,146][20347] Updated weights for policy 0, policy_version 244807 (0.0005) [2023-12-27 07:20:39,325][20381] Updated weights for policy 1, policy_version 244406 (0.0010) [2023-12-27 07:20:39,388][20381] Updated weights for policy 1, policy_version 244416 (0.0009) [2023-12-27 07:20:39,449][20381] Updated weights for policy 1, policy_version 244426 (0.0008) [2023-12-27 07:20:39,752][20347] Updated weights for policy 0, policy_version 244817 (0.0005) [2023-12-27 07:20:39,803][20347] Updated weights for policy 0, policy_version 244827 (0.0005) [2023-12-27 07:20:39,867][20347] Updated weights for policy 0, policy_version 244837 (0.0007) [2023-12-27 07:20:40,171][20381] Updated weights for policy 1, policy_version 244436 (0.0010) [2023-12-27 07:20:40,229][20381] Updated weights for policy 1, policy_version 244447 (0.0008) [2023-12-27 07:20:40,282][20381] Updated weights for policy 1, policy_version 244457 (0.0009) [2023-12-27 07:20:40,537][20347] Updated weights for policy 0, policy_version 244847 (0.0009) [2023-12-27 07:20:40,591][20347] Updated weights for policy 0, policy_version 244857 (0.0008) [2023-12-27 07:20:40,635][20347] Updated weights for policy 0, policy_version 244867 (0.0009) [2023-12-27 07:20:40,913][20381] Updated weights for policy 1, policy_version 244467 (0.0006) [2023-12-27 07:20:40,975][20381] Updated weights for policy 1, policy_version 244477 (0.0007) [2023-12-27 07:20:41,033][20381] Updated weights for policy 1, policy_version 244487 (0.0006) [2023-12-27 07:20:41,372][20347] Updated weights for policy 0, policy_version 244877 (0.0009) [2023-12-27 07:20:41,433][20347] Updated weights for policy 0, policy_version 244887 (0.0010) [2023-12-27 07:20:41,493][20347] Updated weights for policy 0, policy_version 244897 (0.0006) [2023-12-27 07:20:41,711][20381] Updated weights for policy 1, policy_version 244497 (0.0008) [2023-12-27 07:20:41,772][20381] Updated weights for policy 1, policy_version 244507 (0.0008) [2023-12-27 07:20:41,831][20381] Updated weights for policy 1, policy_version 244517 (0.0007) [2023-12-27 07:20:41,889][20381] Updated weights for policy 1, policy_version 244527 (0.0007) [2023-12-27 07:20:42,202][20347] Updated weights for policy 0, policy_version 244907 (0.0011) [2023-12-27 07:20:42,271][20347] Updated weights for policy 0, policy_version 244917 (0.0010) [2023-12-27 07:20:42,333][20347] Updated weights for policy 0, policy_version 244927 (0.0008) [2023-12-27 07:20:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 125321216. Throughput: 0: 10471.6, 1: 10356.3. Samples: 125317256. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.510')] [2023-12-27 07:20:42,554][20381] Updated weights for policy 1, policy_version 244537 (0.0008) [2023-12-27 07:20:42,609][20381] Updated weights for policy 1, policy_version 244547 (0.0007) [2023-12-27 07:20:42,672][20381] Updated weights for policy 1, policy_version 244557 (0.0007) [2023-12-27 07:20:43,019][20347] Updated weights for policy 0, policy_version 244937 (0.0008) [2023-12-27 07:20:43,067][20347] Updated weights for policy 0, policy_version 244947 (0.0005) [2023-12-27 07:20:43,131][20347] Updated weights for policy 0, policy_version 244957 (0.0005) [2023-12-27 07:20:43,197][20347] Updated weights for policy 0, policy_version 244967 (0.0005) [2023-12-27 07:20:43,268][20381] Updated weights for policy 1, policy_version 244567 (0.0006) [2023-12-27 07:20:43,337][20381] Updated weights for policy 1, policy_version 244577 (0.0008) [2023-12-27 07:20:43,392][20381] Updated weights for policy 1, policy_version 244587 (0.0007) [2023-12-27 07:20:43,794][20347] Updated weights for policy 0, policy_version 244977 (0.0009) [2023-12-27 07:20:43,848][20347] Updated weights for policy 0, policy_version 244987 (0.0010) [2023-12-27 07:20:43,896][20347] Updated weights for policy 0, policy_version 244997 (0.0010) [2023-12-27 07:20:44,082][20381] Updated weights for policy 1, policy_version 244597 (0.0007) [2023-12-27 07:20:44,132][20381] Updated weights for policy 1, policy_version 244607 (0.0007) [2023-12-27 07:20:44,192][20381] Updated weights for policy 1, policy_version 244617 (0.0007) [2023-12-27 07:20:44,554][20347] Updated weights for policy 0, policy_version 245007 (0.0011) [2023-12-27 07:20:44,609][20347] Updated weights for policy 0, policy_version 245017 (0.0010) [2023-12-27 07:20:44,657][20347] Updated weights for policy 0, policy_version 245027 (0.0010) [2023-12-27 07:20:44,831][20381] Updated weights for policy 1, policy_version 244627 (0.0006) [2023-12-27 07:20:44,884][20381] Updated weights for policy 1, policy_version 244637 (0.0009) [2023-12-27 07:20:44,940][20381] Updated weights for policy 1, policy_version 244647 (0.0006) [2023-12-27 07:20:45,386][20347] Updated weights for policy 0, policy_version 245037 (0.0008) [2023-12-27 07:20:45,438][20347] Updated weights for policy 0, policy_version 245047 (0.0008) [2023-12-27 07:20:45,492][20347] Updated weights for policy 0, policy_version 245057 (0.0008) [2023-12-27 07:20:45,602][20381] Updated weights for policy 1, policy_version 244657 (0.0008) [2023-12-27 07:20:45,659][20381] Updated weights for policy 1, policy_version 244667 (0.0010) [2023-12-27 07:20:45,714][20381] Updated weights for policy 1, policy_version 244677 (0.0010) [2023-12-27 07:20:45,772][20381] Updated weights for policy 1, policy_version 244687 (0.0010) [2023-12-27 07:20:46,189][20347] Updated weights for policy 0, policy_version 245067 (0.0009) [2023-12-27 07:20:46,245][20347] Updated weights for policy 0, policy_version 245077 (0.0010) [2023-12-27 07:20:46,303][20347] Updated weights for policy 0, policy_version 245087 (0.0010) [2023-12-27 07:20:46,452][20381] Updated weights for policy 1, policy_version 244697 (0.0011) [2023-12-27 07:20:46,507][20381] Updated weights for policy 1, policy_version 244707 (0.0011) [2023-12-27 07:20:46,558][20381] Updated weights for policy 1, policy_version 244717 (0.0010) [2023-12-27 07:20:46,934][20347] Updated weights for policy 0, policy_version 245097 (0.0010) [2023-12-27 07:20:46,986][20347] Updated weights for policy 0, policy_version 245107 (0.0010) [2023-12-27 07:20:47,039][20347] Updated weights for policy 0, policy_version 245117 (0.0010) [2023-12-27 07:20:47,055][20291] KL-divergence is very high: 118.0697 [2023-12-27 07:20:47,091][20347] Updated weights for policy 0, policy_version 245127 (0.0010) [2023-12-27 07:20:47,195][20381] Updated weights for policy 1, policy_version 244727 (0.0011) [2023-12-27 07:20:47,251][20381] Updated weights for policy 1, policy_version 244737 (0.0010) [2023-12-27 07:20:47,301][20381] Updated weights for policy 1, policy_version 244747 (0.0010) [2023-12-27 07:20:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 125435904. Throughput: 0: 10440.0, 1: 10379.3. Samples: 125441460. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:47,363][19170] Avg episode reward: [(0, '990.400'), (1, '990.510')] [2023-12-27 07:20:47,771][20347] Updated weights for policy 0, policy_version 245137 (0.0006) [2023-12-27 07:20:47,817][20347] Updated weights for policy 0, policy_version 245147 (0.0005) [2023-12-27 07:20:47,866][20347] Updated weights for policy 0, policy_version 245157 (0.0007) [2023-12-27 07:20:47,895][20381] Updated weights for policy 1, policy_version 244757 (0.0009) [2023-12-27 07:20:47,956][20381] Updated weights for policy 1, policy_version 244767 (0.0008) [2023-12-27 07:20:48,019][20381] Updated weights for policy 1, policy_version 244777 (0.0009) [2023-12-27 07:20:48,525][20347] Updated weights for policy 0, policy_version 245167 (0.0010) [2023-12-27 07:20:48,590][20347] Updated weights for policy 0, policy_version 245177 (0.0010) [2023-12-27 07:20:48,643][20381] Updated weights for policy 1, policy_version 244787 (0.0009) [2023-12-27 07:20:48,643][20347] Updated weights for policy 0, policy_version 245187 (0.0008) [2023-12-27 07:20:48,703][20381] Updated weights for policy 1, policy_version 244797 (0.0006) [2023-12-27 07:20:48,764][20381] Updated weights for policy 1, policy_version 244807 (0.0010) [2023-12-27 07:20:49,315][20347] Updated weights for policy 0, policy_version 245197 (0.0008) [2023-12-27 07:20:49,340][20381] Updated weights for policy 1, policy_version 244817 (0.0011) [2023-12-27 07:20:49,376][20347] Updated weights for policy 0, policy_version 245207 (0.0009) [2023-12-27 07:20:49,408][20381] Updated weights for policy 1, policy_version 244827 (0.0011) [2023-12-27 07:20:49,435][20347] Updated weights for policy 0, policy_version 245217 (0.0010) [2023-12-27 07:20:49,463][20381] Updated weights for policy 1, policy_version 244837 (0.0010) [2023-12-27 07:20:49,515][20381] Updated weights for policy 1, policy_version 244847 (0.0010) [2023-12-27 07:20:50,077][20347] Updated weights for policy 0, policy_version 245227 (0.0010) [2023-12-27 07:20:50,141][20347] Updated weights for policy 0, policy_version 245237 (0.0010) [2023-12-27 07:20:50,197][20347] Updated weights for policy 0, policy_version 245247 (0.0011) [2023-12-27 07:20:50,203][20381] Updated weights for policy 1, policy_version 244857 (0.0010) [2023-12-27 07:20:50,264][20381] Updated weights for policy 1, policy_version 244867 (0.0005) [2023-12-27 07:20:50,320][20381] Updated weights for policy 1, policy_version 244877 (0.0007) [2023-12-27 07:20:50,826][20347] Updated weights for policy 0, policy_version 245257 (0.0010) [2023-12-27 07:20:50,883][20347] Updated weights for policy 0, policy_version 245267 (0.0009) [2023-12-27 07:20:50,940][20381] Updated weights for policy 1, policy_version 244887 (0.0007) [2023-12-27 07:20:50,943][20347] Updated weights for policy 0, policy_version 245277 (0.0008) [2023-12-27 07:20:50,994][20381] Updated weights for policy 1, policy_version 244897 (0.0005) [2023-12-27 07:20:50,996][20347] Updated weights for policy 0, policy_version 245287 (0.0007) [2023-12-27 07:20:51,060][20381] Updated weights for policy 1, policy_version 244907 (0.0007) [2023-12-27 07:20:51,637][20381] Updated weights for policy 1, policy_version 244917 (0.0007) [2023-12-27 07:20:51,701][20381] Updated weights for policy 1, policy_version 244927 (0.0006) [2023-12-27 07:20:51,737][20347] Updated weights for policy 0, policy_version 245297 (0.0008) [2023-12-27 07:20:51,762][20381] Updated weights for policy 1, policy_version 244937 (0.0007) [2023-12-27 07:20:51,789][20347] Updated weights for policy 0, policy_version 245307 (0.0008) [2023-12-27 07:20:51,834][20347] Updated weights for policy 0, policy_version 245317 (0.0008) [2023-12-27 07:20:52,314][20381] Updated weights for policy 1, policy_version 244947 (0.0006) [2023-12-27 07:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 125534208. Throughput: 0: 10406.6, 1: 10453.1. Samples: 125506184. Policy #0 lag: (min: 19.0, avg: 22.2, max: 51.0) [2023-12-27 07:20:52,363][19170] Avg episode reward: [(0, '990.400'), (1, '1000.000')] [2023-12-27 07:20:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000245320_62816256.pth... [2023-12-27 07:20:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000244104_62504960.pth [2023-12-27 07:20:52,379][20381] Updated weights for policy 1, policy_version 244957 (0.0007) [2023-12-27 07:20:52,438][20381] Updated weights for policy 1, policy_version 244967 (0.0008) [2023-12-27 07:20:52,488][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000244976_62726144.pth... [2023-12-27 07:20:52,491][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000243728_62406656.pth [2023-12-27 07:20:52,614][20347] Updated weights for policy 0, policy_version 245327 (0.0008) [2023-12-27 07:20:52,676][20347] Updated weights for policy 0, policy_version 245337 (0.0005) [2023-12-27 07:20:52,739][20347] Updated weights for policy 0, policy_version 245347 (0.0006) [2023-12-27 07:20:53,129][20381] Updated weights for policy 1, policy_version 244977 (0.0008) [2023-12-27 07:20:53,189][20381] Updated weights for policy 1, policy_version 244987 (0.0008) [2023-12-27 07:20:53,251][20381] Updated weights for policy 1, policy_version 244997 (0.0008) [2023-12-27 07:20:53,310][20381] Updated weights for policy 1, policy_version 245007 (0.0008) [2023-12-27 07:20:53,391][20347] Updated weights for policy 0, policy_version 245357 (0.0007) [2023-12-27 07:20:53,445][20347] Updated weights for policy 0, policy_version 245367 (0.0007) [2023-12-27 07:20:53,518][20347] Updated weights for policy 0, policy_version 245377 (0.0007) [2023-12-27 07:20:53,937][20381] Updated weights for policy 1, policy_version 245017 (0.0007) [2023-12-27 07:20:53,999][20381] Updated weights for policy 1, policy_version 245027 (0.0007) [2023-12-27 07:20:54,059][20381] Updated weights for policy 1, policy_version 245037 (0.0007) [2023-12-27 07:20:54,206][20347] Updated weights for policy 0, policy_version 245387 (0.0009) [2023-12-27 07:20:54,262][20347] Updated weights for policy 0, policy_version 245397 (0.0010) [2023-12-27 07:20:54,319][20347] Updated weights for policy 0, policy_version 245407 (0.0008) [2023-12-27 07:20:54,703][20381] Updated weights for policy 1, policy_version 245047 (0.0009) [2023-12-27 07:20:54,760][20381] Updated weights for policy 1, policy_version 245057 (0.0010) [2023-12-27 07:20:54,821][20381] Updated weights for policy 1, policy_version 245067 (0.0010) [2023-12-27 07:20:54,970][20347] Updated weights for policy 0, policy_version 245417 (0.0008) [2023-12-27 07:20:55,019][20347] Updated weights for policy 0, policy_version 245427 (0.0006) [2023-12-27 07:20:55,082][20347] Updated weights for policy 0, policy_version 245437 (0.0007) [2023-12-27 07:20:55,149][20347] Updated weights for policy 0, policy_version 245447 (0.0010) [2023-12-27 07:20:55,496][20381] Updated weights for policy 1, policy_version 245077 (0.0009) [2023-12-27 07:20:55,544][20381] Updated weights for policy 1, policy_version 245087 (0.0007) [2023-12-27 07:20:55,603][20381] Updated weights for policy 1, policy_version 245097 (0.0008) [2023-12-27 07:20:55,787][20347] Updated weights for policy 0, policy_version 245457 (0.0010) [2023-12-27 07:20:55,832][20347] Updated weights for policy 0, policy_version 245467 (0.0010) [2023-12-27 07:20:55,880][20347] Updated weights for policy 0, policy_version 245477 (0.0010) [2023-12-27 07:20:56,306][20381] Updated weights for policy 1, policy_version 245107 (0.0008) [2023-12-27 07:20:56,364][20381] Updated weights for policy 1, policy_version 245117 (0.0011) [2023-12-27 07:20:56,423][20381] Updated weights for policy 1, policy_version 245127 (0.0011) [2023-12-27 07:20:56,566][20347] Updated weights for policy 0, policy_version 245487 (0.0007) [2023-12-27 07:20:56,616][20347] Updated weights for policy 0, policy_version 245497 (0.0005) [2023-12-27 07:20:56,670][20347] Updated weights for policy 0, policy_version 245507 (0.0005) [2023-12-27 07:20:56,676][20291] KL-divergence is very high: 407.7417 [2023-12-27 07:20:57,048][20381] Updated weights for policy 1, policy_version 245137 (0.0011) [2023-12-27 07:20:57,102][20381] Updated weights for policy 1, policy_version 245147 (0.0010) [2023-12-27 07:20:57,154][20381] Updated weights for policy 1, policy_version 245157 (0.0010) [2023-12-27 07:20:57,202][20381] Updated weights for policy 1, policy_version 245167 (0.0010) [2023-12-27 07:20:57,307][20347] Updated weights for policy 0, policy_version 245517 (0.0006) [2023-12-27 07:20:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 125640704. Throughput: 0: 10387.3, 1: 10496.5. Samples: 125630496. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:20:57,363][19170] Avg episode reward: [(0, '990.490'), (1, '1000.000')] [2023-12-27 07:20:57,365][20347] Updated weights for policy 0, policy_version 245527 (0.0007) [2023-12-27 07:20:57,426][20347] Updated weights for policy 0, policy_version 245537 (0.0008) [2023-12-27 07:20:57,787][20381] Updated weights for policy 1, policy_version 245177 (0.0006) [2023-12-27 07:20:57,836][20381] Updated weights for policy 1, policy_version 245187 (0.0008) [2023-12-27 07:20:57,894][20381] Updated weights for policy 1, policy_version 245197 (0.0010) [2023-12-27 07:20:57,960][20347] Updated weights for policy 0, policy_version 245547 (0.0005) [2023-12-27 07:20:58,012][20347] Updated weights for policy 0, policy_version 245557 (0.0005) [2023-12-27 07:20:58,056][20347] Updated weights for policy 0, policy_version 245567 (0.0007) [2023-12-27 07:20:58,608][20381] Updated weights for policy 1, policy_version 245207 (0.0008) [2023-12-27 07:20:58,663][20381] Updated weights for policy 1, policy_version 245217 (0.0006) [2023-12-27 07:20:58,726][20381] Updated weights for policy 1, policy_version 245227 (0.0007) [2023-12-27 07:20:58,760][20347] Updated weights for policy 0, policy_version 245577 (0.0006) [2023-12-27 07:20:58,819][20347] Updated weights for policy 0, policy_version 245587 (0.0008) [2023-12-27 07:20:58,880][20347] Updated weights for policy 0, policy_version 245597 (0.0008) [2023-12-27 07:20:58,942][20347] Updated weights for policy 0, policy_version 245607 (0.0008) [2023-12-27 07:20:59,404][20381] Updated weights for policy 1, policy_version 245237 (0.0008) [2023-12-27 07:20:59,465][20381] Updated weights for policy 1, policy_version 245247 (0.0008) [2023-12-27 07:20:59,520][20381] Updated weights for policy 1, policy_version 245257 (0.0007) [2023-12-27 07:20:59,601][20347] Updated weights for policy 0, policy_version 245617 (0.0007) [2023-12-27 07:20:59,660][20347] Updated weights for policy 0, policy_version 245627 (0.0008) [2023-12-27 07:20:59,718][20347] Updated weights for policy 0, policy_version 245637 (0.0008) [2023-12-27 07:21:00,207][20381] Updated weights for policy 1, policy_version 245267 (0.0007) [2023-12-27 07:21:00,263][20381] Updated weights for policy 1, policy_version 245277 (0.0005) [2023-12-27 07:21:00,317][20381] Updated weights for policy 1, policy_version 245287 (0.0007) [2023-12-27 07:21:00,454][20347] Updated weights for policy 0, policy_version 245647 (0.0007) [2023-12-27 07:21:00,512][20347] Updated weights for policy 0, policy_version 245657 (0.0007) [2023-12-27 07:21:00,564][20347] Updated weights for policy 0, policy_version 245667 (0.0008) [2023-12-27 07:21:00,893][20381] Updated weights for policy 1, policy_version 245297 (0.0006) [2023-12-27 07:21:00,942][20381] Updated weights for policy 1, policy_version 245307 (0.0008) [2023-12-27 07:21:00,983][20381] Updated weights for policy 1, policy_version 245317 (0.0008) [2023-12-27 07:21:01,037][20381] Updated weights for policy 1, policy_version 245327 (0.0008) [2023-12-27 07:21:01,321][20347] Updated weights for policy 0, policy_version 245677 (0.0009) [2023-12-27 07:21:01,386][20347] Updated weights for policy 0, policy_version 245687 (0.0007) [2023-12-27 07:21:01,440][20347] Updated weights for policy 0, policy_version 245697 (0.0007) [2023-12-27 07:21:01,765][20381] Updated weights for policy 1, policy_version 245337 (0.0011) [2023-12-27 07:21:01,817][20381] Updated weights for policy 1, policy_version 245347 (0.0010) [2023-12-27 07:21:01,875][20381] Updated weights for policy 1, policy_version 245357 (0.0011) [2023-12-27 07:21:02,120][20347] Updated weights for policy 0, policy_version 245707 (0.0007) [2023-12-27 07:21:02,174][20347] Updated weights for policy 0, policy_version 245717 (0.0008) [2023-12-27 07:21:02,224][20347] Updated weights for policy 0, policy_version 245727 (0.0006) [2023-12-27 07:21:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 125747200. Throughput: 0: 10333.3, 1: 10517.4. Samples: 125756028. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:02,363][19170] Avg episode reward: [(0, '990.490'), (1, '1000.000')] [2023-12-27 07:21:02,561][20381] Updated weights for policy 1, policy_version 245367 (0.0009) [2023-12-27 07:21:02,628][20381] Updated weights for policy 1, policy_version 245377 (0.0009) [2023-12-27 07:21:02,686][20381] Updated weights for policy 1, policy_version 245387 (0.0010) [2023-12-27 07:21:02,902][20347] Updated weights for policy 0, policy_version 245737 (0.0008) [2023-12-27 07:21:02,959][20347] Updated weights for policy 0, policy_version 245747 (0.0007) [2023-12-27 07:21:03,017][20347] Updated weights for policy 0, policy_version 245757 (0.0006) [2023-12-27 07:21:03,079][20347] Updated weights for policy 0, policy_version 245767 (0.0006) [2023-12-27 07:21:03,352][20381] Updated weights for policy 1, policy_version 245397 (0.0010) [2023-12-27 07:21:03,404][20381] Updated weights for policy 1, policy_version 245407 (0.0010) [2023-12-27 07:21:03,467][20381] Updated weights for policy 1, policy_version 245417 (0.0010) [2023-12-27 07:21:03,759][20347] Updated weights for policy 0, policy_version 245777 (0.0008) [2023-12-27 07:21:03,806][20347] Updated weights for policy 0, policy_version 245787 (0.0007) [2023-12-27 07:21:03,866][20347] Updated weights for policy 0, policy_version 245797 (0.0007) [2023-12-27 07:21:04,107][20381] Updated weights for policy 1, policy_version 245427 (0.0010) [2023-12-27 07:21:04,163][20381] Updated weights for policy 1, policy_version 245437 (0.0011) [2023-12-27 07:21:04,217][20381] Updated weights for policy 1, policy_version 245447 (0.0007) [2023-12-27 07:21:04,575][20347] Updated weights for policy 0, policy_version 245807 (0.0007) [2023-12-27 07:21:04,631][20347] Updated weights for policy 0, policy_version 245817 (0.0005) [2023-12-27 07:21:04,690][20347] Updated weights for policy 0, policy_version 245827 (0.0008) [2023-12-27 07:21:04,900][20381] Updated weights for policy 1, policy_version 245457 (0.0007) [2023-12-27 07:21:04,949][20381] Updated weights for policy 1, policy_version 245467 (0.0005) [2023-12-27 07:21:05,006][20381] Updated weights for policy 1, policy_version 245477 (0.0006) [2023-12-27 07:21:05,063][20381] Updated weights for policy 1, policy_version 245487 (0.0008) [2023-12-27 07:21:05,368][20347] Updated weights for policy 0, policy_version 245837 (0.0007) [2023-12-27 07:21:05,427][20347] Updated weights for policy 0, policy_version 245847 (0.0007) [2023-12-27 07:21:05,490][20347] Updated weights for policy 0, policy_version 245857 (0.0008) [2023-12-27 07:21:05,631][20381] Updated weights for policy 1, policy_version 245497 (0.0007) [2023-12-27 07:21:05,681][20381] Updated weights for policy 1, policy_version 245507 (0.0010) [2023-12-27 07:21:05,734][20381] Updated weights for policy 1, policy_version 245517 (0.0008) [2023-12-27 07:21:06,132][20347] Updated weights for policy 0, policy_version 245867 (0.0007) [2023-12-27 07:21:06,193][20347] Updated weights for policy 0, policy_version 245877 (0.0008) [2023-12-27 07:21:06,246][20347] Updated weights for policy 0, policy_version 245887 (0.0010) [2023-12-27 07:21:06,419][20381] Updated weights for policy 1, policy_version 245527 (0.0008) [2023-12-27 07:21:06,474][20381] Updated weights for policy 1, policy_version 245537 (0.0007) [2023-12-27 07:21:06,526][20381] Updated weights for policy 1, policy_version 245547 (0.0007) [2023-12-27 07:21:06,938][20347] Updated weights for policy 0, policy_version 245897 (0.0010) [2023-12-27 07:21:06,999][20347] Updated weights for policy 0, policy_version 245907 (0.0009) [2023-12-27 07:21:07,057][20347] Updated weights for policy 0, policy_version 245917 (0.0008) [2023-12-27 07:21:07,116][20347] Updated weights for policy 0, policy_version 245927 (0.0007) [2023-12-27 07:21:07,218][20381] Updated weights for policy 1, policy_version 245557 (0.0008) [2023-12-27 07:21:07,281][20381] Updated weights for policy 1, policy_version 245567 (0.0007) [2023-12-27 07:21:07,341][20381] Updated weights for policy 1, policy_version 245577 (0.0008) [2023-12-27 07:21:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 125845504. Throughput: 0: 10313.0, 1: 10537.9. Samples: 125817348. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000245928_62971904.pth... [2023-12-27 07:21:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000244712_62660608.pth [2023-12-27 07:21:07,388][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000245584_62881792.pth... [2023-12-27 07:21:07,392][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000244336_62562304.pth [2023-12-27 07:21:07,725][20347] Updated weights for policy 0, policy_version 245937 (0.0007) [2023-12-27 07:21:07,770][20347] Updated weights for policy 0, policy_version 245947 (0.0010) [2023-12-27 07:21:07,818][20347] Updated weights for policy 0, policy_version 245957 (0.0010) [2023-12-27 07:21:08,048][20381] Updated weights for policy 1, policy_version 245587 (0.0007) [2023-12-27 07:21:08,105][20381] Updated weights for policy 1, policy_version 245597 (0.0006) [2023-12-27 07:21:08,164][20381] Updated weights for policy 1, policy_version 245607 (0.0008) [2023-12-27 07:21:08,528][20347] Updated weights for policy 0, policy_version 245967 (0.0007) [2023-12-27 07:21:08,591][20347] Updated weights for policy 0, policy_version 245977 (0.0005) [2023-12-27 07:21:08,658][20347] Updated weights for policy 0, policy_version 245987 (0.0007) [2023-12-27 07:21:08,859][20381] Updated weights for policy 1, policy_version 245617 (0.0009) [2023-12-27 07:21:08,911][20381] Updated weights for policy 1, policy_version 245627 (0.0007) [2023-12-27 07:21:08,968][20381] Updated weights for policy 1, policy_version 245637 (0.0008) [2023-12-27 07:21:09,017][20381] Updated weights for policy 1, policy_version 245647 (0.0007) [2023-12-27 07:21:09,272][20347] Updated weights for policy 0, policy_version 245997 (0.0008) [2023-12-27 07:21:09,337][20347] Updated weights for policy 0, policy_version 246007 (0.0007) [2023-12-27 07:21:09,400][20347] Updated weights for policy 0, policy_version 246017 (0.0008) [2023-12-27 07:21:09,747][20381] Updated weights for policy 1, policy_version 245657 (0.0007) [2023-12-27 07:21:09,809][20381] Updated weights for policy 1, policy_version 245667 (0.0006) [2023-12-27 07:21:09,877][20381] Updated weights for policy 1, policy_version 245677 (0.0007) [2023-12-27 07:21:10,014][20347] Updated weights for policy 0, policy_version 246027 (0.0006) [2023-12-27 07:21:10,064][20347] Updated weights for policy 0, policy_version 246037 (0.0006) [2023-12-27 07:21:10,116][20347] Updated weights for policy 0, policy_version 246047 (0.0007) [2023-12-27 07:21:10,584][20381] Updated weights for policy 1, policy_version 245687 (0.0007) [2023-12-27 07:21:10,649][20381] Updated weights for policy 1, policy_version 245697 (0.0007) [2023-12-27 07:21:10,704][20381] Updated weights for policy 1, policy_version 245707 (0.0005) [2023-12-27 07:21:10,756][20347] Updated weights for policy 0, policy_version 246057 (0.0008) [2023-12-27 07:21:10,807][20347] Updated weights for policy 0, policy_version 246067 (0.0005) [2023-12-27 07:21:10,865][20347] Updated weights for policy 0, policy_version 246077 (0.0006) [2023-12-27 07:21:10,919][20347] Updated weights for policy 0, policy_version 246087 (0.0007) [2023-12-27 07:21:11,379][20381] Updated weights for policy 1, policy_version 245717 (0.0008) [2023-12-27 07:21:11,440][20381] Updated weights for policy 1, policy_version 245727 (0.0007) [2023-12-27 07:21:11,499][20381] Updated weights for policy 1, policy_version 245737 (0.0007) [2023-12-27 07:21:11,624][20347] Updated weights for policy 0, policy_version 246097 (0.0008) [2023-12-27 07:21:11,684][20347] Updated weights for policy 0, policy_version 246107 (0.0007) [2023-12-27 07:21:11,746][20347] Updated weights for policy 0, policy_version 246117 (0.0008) [2023-12-27 07:21:12,167][20381] Updated weights for policy 1, policy_version 245747 (0.0007) [2023-12-27 07:21:12,220][20381] Updated weights for policy 1, policy_version 245757 (0.0008) [2023-12-27 07:21:12,286][20381] Updated weights for policy 1, policy_version 245767 (0.0009) [2023-12-27 07:21:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 125952000. Throughput: 0: 10361.8, 1: 10484.2. Samples: 125941256. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:12,412][20347] Updated weights for policy 0, policy_version 246127 (0.0009) [2023-12-27 07:21:12,469][20347] Updated weights for policy 0, policy_version 246137 (0.0011) [2023-12-27 07:21:12,525][20347] Updated weights for policy 0, policy_version 246147 (0.0010) [2023-12-27 07:21:12,974][20381] Updated weights for policy 1, policy_version 245777 (0.0008) [2023-12-27 07:21:13,027][20381] Updated weights for policy 1, policy_version 245787 (0.0005) [2023-12-27 07:21:13,076][20381] Updated weights for policy 1, policy_version 245797 (0.0005) [2023-12-27 07:21:13,125][20381] Updated weights for policy 1, policy_version 245807 (0.0005) [2023-12-27 07:21:13,197][20347] Updated weights for policy 0, policy_version 246157 (0.0011) [2023-12-27 07:21:13,252][20347] Updated weights for policy 0, policy_version 246167 (0.0010) [2023-12-27 07:21:13,309][20347] Updated weights for policy 0, policy_version 246177 (0.0010) [2023-12-27 07:21:13,693][20381] Updated weights for policy 1, policy_version 245817 (0.0007) [2023-12-27 07:21:13,750][20381] Updated weights for policy 1, policy_version 245827 (0.0007) [2023-12-27 07:21:13,801][20381] Updated weights for policy 1, policy_version 245837 (0.0007) [2023-12-27 07:21:13,988][20347] Updated weights for policy 0, policy_version 246187 (0.0010) [2023-12-27 07:21:14,053][20347] Updated weights for policy 0, policy_version 246197 (0.0007) [2023-12-27 07:21:14,110][20347] Updated weights for policy 0, policy_version 246207 (0.0007) [2023-12-27 07:21:14,520][20381] Updated weights for policy 1, policy_version 245847 (0.0007) [2023-12-27 07:21:14,584][20381] Updated weights for policy 1, policy_version 245857 (0.0007) [2023-12-27 07:21:14,637][20381] Updated weights for policy 1, policy_version 245867 (0.0007) [2023-12-27 07:21:14,776][20347] Updated weights for policy 0, policy_version 246217 (0.0006) [2023-12-27 07:21:14,839][20347] Updated weights for policy 0, policy_version 246227 (0.0008) [2023-12-27 07:21:14,895][20347] Updated weights for policy 0, policy_version 246237 (0.0006) [2023-12-27 07:21:14,953][20347] Updated weights for policy 0, policy_version 246247 (0.0006) [2023-12-27 07:21:15,326][20381] Updated weights for policy 1, policy_version 245877 (0.0008) [2023-12-27 07:21:15,380][20381] Updated weights for policy 1, policy_version 245887 (0.0008) [2023-12-27 07:21:15,437][20381] Updated weights for policy 1, policy_version 245897 (0.0007) [2023-12-27 07:21:15,604][20347] Updated weights for policy 0, policy_version 246257 (0.0010) [2023-12-27 07:21:15,662][20347] Updated weights for policy 0, policy_version 246267 (0.0008) [2023-12-27 07:21:15,713][20347] Updated weights for policy 0, policy_version 246277 (0.0007) [2023-12-27 07:21:16,106][20381] Updated weights for policy 1, policy_version 245907 (0.0008) [2023-12-27 07:21:16,169][20381] Updated weights for policy 1, policy_version 245917 (0.0007) [2023-12-27 07:21:16,216][20381] Updated weights for policy 1, policy_version 245927 (0.0007) [2023-12-27 07:21:16,402][20347] Updated weights for policy 0, policy_version 246287 (0.0009) [2023-12-27 07:21:16,463][20347] Updated weights for policy 0, policy_version 246297 (0.0010) [2023-12-27 07:21:16,517][20347] Updated weights for policy 0, policy_version 246307 (0.0010) [2023-12-27 07:21:16,900][20381] Updated weights for policy 1, policy_version 245937 (0.0008) [2023-12-27 07:21:16,960][20381] Updated weights for policy 1, policy_version 245947 (0.0007) [2023-12-27 07:21:17,026][20381] Updated weights for policy 1, policy_version 245957 (0.0007) [2023-12-27 07:21:17,079][20381] Updated weights for policy 1, policy_version 245967 (0.0007) [2023-12-27 07:21:17,196][20347] Updated weights for policy 0, policy_version 246317 (0.0010) [2023-12-27 07:21:17,244][20347] Updated weights for policy 0, policy_version 246327 (0.0010) [2023-12-27 07:21:17,306][20347] Updated weights for policy 0, policy_version 246337 (0.0010) [2023-12-27 07:21:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 126058496. Throughput: 0: 10329.9, 1: 10465.2. Samples: 126064504. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:17,750][20381] Updated weights for policy 1, policy_version 245977 (0.0008) [2023-12-27 07:21:17,812][20381] Updated weights for policy 1, policy_version 245987 (0.0007) [2023-12-27 07:21:17,866][20381] Updated weights for policy 1, policy_version 245997 (0.0007) [2023-12-27 07:21:17,993][20347] Updated weights for policy 0, policy_version 246347 (0.0010) [2023-12-27 07:21:18,047][20347] Updated weights for policy 0, policy_version 246357 (0.0010) [2023-12-27 07:21:18,096][20347] Updated weights for policy 0, policy_version 246367 (0.0010) [2023-12-27 07:21:18,533][20381] Updated weights for policy 1, policy_version 246007 (0.0006) [2023-12-27 07:21:18,591][20381] Updated weights for policy 1, policy_version 246017 (0.0006) [2023-12-27 07:21:18,649][20381] Updated weights for policy 1, policy_version 246027 (0.0007) [2023-12-27 07:21:18,785][20347] Updated weights for policy 0, policy_version 246377 (0.0008) [2023-12-27 07:21:18,835][20347] Updated weights for policy 0, policy_version 246387 (0.0011) [2023-12-27 07:21:18,893][20347] Updated weights for policy 0, policy_version 246397 (0.0010) [2023-12-27 07:21:18,951][20347] Updated weights for policy 0, policy_version 246407 (0.0010) [2023-12-27 07:21:19,306][20381] Updated weights for policy 1, policy_version 246037 (0.0008) [2023-12-27 07:21:19,374][20381] Updated weights for policy 1, policy_version 246047 (0.0008) [2023-12-27 07:21:19,423][20381] Updated weights for policy 1, policy_version 246057 (0.0006) [2023-12-27 07:21:19,659][20347] Updated weights for policy 0, policy_version 246417 (0.0007) [2023-12-27 07:21:19,724][20347] Updated weights for policy 0, policy_version 246427 (0.0007) [2023-12-27 07:21:19,789][20347] Updated weights for policy 0, policy_version 246437 (0.0010) [2023-12-27 07:21:20,156][20381] Updated weights for policy 1, policy_version 246067 (0.0008) [2023-12-27 07:21:20,223][20381] Updated weights for policy 1, policy_version 246077 (0.0008) [2023-12-27 07:21:20,286][20381] Updated weights for policy 1, policy_version 246087 (0.0007) [2023-12-27 07:21:20,390][20347] Updated weights for policy 0, policy_version 246447 (0.0009) [2023-12-27 07:21:20,453][20347] Updated weights for policy 0, policy_version 246457 (0.0008) [2023-12-27 07:21:20,524][20347] Updated weights for policy 0, policy_version 246467 (0.0008) [2023-12-27 07:21:21,001][20381] Updated weights for policy 1, policy_version 246097 (0.0008) [2023-12-27 07:21:21,069][20381] Updated weights for policy 1, policy_version 246107 (0.0009) [2023-12-27 07:21:21,131][20381] Updated weights for policy 1, policy_version 246117 (0.0008) [2023-12-27 07:21:21,155][20347] Updated weights for policy 0, policy_version 246477 (0.0008) [2023-12-27 07:21:21,193][20381] Updated weights for policy 1, policy_version 246127 (0.0008) [2023-12-27 07:21:21,216][20347] Updated weights for policy 0, policy_version 246487 (0.0009) [2023-12-27 07:21:21,282][20347] Updated weights for policy 0, policy_version 246497 (0.0010) [2023-12-27 07:21:21,924][20381] Updated weights for policy 1, policy_version 246137 (0.0008) [2023-12-27 07:21:21,973][20347] Updated weights for policy 0, policy_version 246507 (0.0009) [2023-12-27 07:21:21,987][20381] Updated weights for policy 1, policy_version 246147 (0.0007) [2023-12-27 07:21:22,027][20347] Updated weights for policy 0, policy_version 246517 (0.0006) [2023-12-27 07:21:22,049][20381] Updated weights for policy 1, policy_version 246157 (0.0008) [2023-12-27 07:21:22,085][20347] Updated weights for policy 0, policy_version 246527 (0.0007) [2023-12-27 07:21:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 126156800. Throughput: 0: 10277.2, 1: 10423.3. Samples: 126125528. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000246536_63127552.pth... [2023-12-27 07:21:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000246160_63029248.pth... [2023-12-27 07:21:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000245320_62816256.pth [2023-12-27 07:21:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000244976_62726144.pth [2023-12-27 07:21:22,691][20381] Updated weights for policy 1, policy_version 246167 (0.0008) [2023-12-27 07:21:22,706][20347] Updated weights for policy 0, policy_version 246537 (0.0008) [2023-12-27 07:21:22,756][20381] Updated weights for policy 1, policy_version 246177 (0.0006) [2023-12-27 07:21:22,758][20347] Updated weights for policy 0, policy_version 246547 (0.0006) [2023-12-27 07:21:22,816][20381] Updated weights for policy 1, policy_version 246187 (0.0007) [2023-12-27 07:21:22,818][20347] Updated weights for policy 0, policy_version 246557 (0.0006) [2023-12-27 07:21:22,876][20347] Updated weights for policy 0, policy_version 246567 (0.0007) [2023-12-27 07:21:23,499][20381] Updated weights for policy 1, policy_version 246197 (0.0007) [2023-12-27 07:21:23,555][20381] Updated weights for policy 1, policy_version 246207 (0.0006) [2023-12-27 07:21:23,586][20347] Updated weights for policy 0, policy_version 246577 (0.0010) [2023-12-27 07:21:23,606][20381] Updated weights for policy 1, policy_version 246217 (0.0005) [2023-12-27 07:21:23,644][20347] Updated weights for policy 0, policy_version 246587 (0.0009) [2023-12-27 07:21:23,708][20347] Updated weights for policy 0, policy_version 246597 (0.0010) [2023-12-27 07:21:24,269][20381] Updated weights for policy 1, policy_version 246227 (0.0007) [2023-12-27 07:21:24,329][20381] Updated weights for policy 1, policy_version 246237 (0.0008) [2023-12-27 07:21:24,385][20381] Updated weights for policy 1, policy_version 246247 (0.0007) [2023-12-27 07:21:24,432][20347] Updated weights for policy 0, policy_version 246607 (0.0007) [2023-12-27 07:21:24,487][20347] Updated weights for policy 0, policy_version 246617 (0.0008) [2023-12-27 07:21:24,544][20347] Updated weights for policy 0, policy_version 246627 (0.0009) [2023-12-27 07:21:25,064][20381] Updated weights for policy 1, policy_version 246257 (0.0006) [2023-12-27 07:21:25,122][20381] Updated weights for policy 1, policy_version 246267 (0.0009) [2023-12-27 07:21:25,177][20381] Updated weights for policy 1, policy_version 246277 (0.0009) [2023-12-27 07:21:25,235][20381] Updated weights for policy 1, policy_version 246287 (0.0006) [2023-12-27 07:21:25,246][20347] Updated weights for policy 0, policy_version 246637 (0.0009) [2023-12-27 07:21:25,312][20347] Updated weights for policy 0, policy_version 246647 (0.0010) [2023-12-27 07:21:25,375][20347] Updated weights for policy 0, policy_version 246657 (0.0010) [2023-12-27 07:21:25,792][20381] Updated weights for policy 1, policy_version 246297 (0.0006) [2023-12-27 07:21:25,849][20381] Updated weights for policy 1, policy_version 246307 (0.0006) [2023-12-27 07:21:25,894][20381] Updated weights for policy 1, policy_version 246317 (0.0006) [2023-12-27 07:21:26,046][20347] Updated weights for policy 0, policy_version 246667 (0.0009) [2023-12-27 07:21:26,105][20347] Updated weights for policy 0, policy_version 246677 (0.0010) [2023-12-27 07:21:26,161][20347] Updated weights for policy 0, policy_version 246687 (0.0009) [2023-12-27 07:21:26,480][20381] Updated weights for policy 1, policy_version 246327 (0.0009) [2023-12-27 07:21:26,534][20381] Updated weights for policy 1, policy_version 246337 (0.0010) [2023-12-27 07:21:26,580][20381] Updated weights for policy 1, policy_version 246347 (0.0009) [2023-12-27 07:21:26,709][20347] Updated weights for policy 0, policy_version 246697 (0.0007) [2023-12-27 07:21:26,761][20347] Updated weights for policy 0, policy_version 246707 (0.0005) [2023-12-27 07:21:26,810][20347] Updated weights for policy 0, policy_version 246717 (0.0005) [2023-12-27 07:21:26,866][20347] Updated weights for policy 0, policy_version 246727 (0.0005) [2023-12-27 07:21:27,184][20381] Updated weights for policy 1, policy_version 246357 (0.0008) [2023-12-27 07:21:27,244][20381] Updated weights for policy 1, policy_version 246367 (0.0010) [2023-12-27 07:21:27,298][20381] Updated weights for policy 1, policy_version 246377 (0.0010) [2023-12-27 07:21:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 126263296. Throughput: 0: 10264.4, 1: 10425.5. Samples: 126248300. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:27,461][20347] Updated weights for policy 0, policy_version 246737 (0.0005) [2023-12-27 07:21:27,509][20347] Updated weights for policy 0, policy_version 246747 (0.0005) [2023-12-27 07:21:27,567][20347] Updated weights for policy 0, policy_version 246757 (0.0008) [2023-12-27 07:21:27,979][20381] Updated weights for policy 1, policy_version 246387 (0.0009) [2023-12-27 07:21:28,029][20381] Updated weights for policy 1, policy_version 246397 (0.0007) [2023-12-27 07:21:28,076][20381] Updated weights for policy 1, policy_version 246407 (0.0005) [2023-12-27 07:21:28,185][20347] Updated weights for policy 0, policy_version 246767 (0.0010) [2023-12-27 07:21:28,236][20347] Updated weights for policy 0, policy_version 246777 (0.0010) [2023-12-27 07:21:28,286][20347] Updated weights for policy 0, policy_version 246787 (0.0010) [2023-12-27 07:21:28,660][20381] Updated weights for policy 1, policy_version 246417 (0.0006) [2023-12-27 07:21:28,715][20381] Updated weights for policy 1, policy_version 246427 (0.0008) [2023-12-27 07:21:28,779][20381] Updated weights for policy 1, policy_version 246437 (0.0007) [2023-12-27 07:21:28,835][20381] Updated weights for policy 1, policy_version 246447 (0.0008) [2023-12-27 07:21:28,881][20347] Updated weights for policy 0, policy_version 246797 (0.0008) [2023-12-27 07:21:28,931][20347] Updated weights for policy 0, policy_version 246807 (0.0005) [2023-12-27 07:21:28,985][20347] Updated weights for policy 0, policy_version 246817 (0.0008) [2023-12-27 07:21:29,534][20381] Updated weights for policy 1, policy_version 246457 (0.0008) [2023-12-27 07:21:29,585][20381] Updated weights for policy 1, policy_version 246467 (0.0008) [2023-12-27 07:21:29,605][20347] Updated weights for policy 0, policy_version 246827 (0.0007) [2023-12-27 07:21:29,636][20381] Updated weights for policy 1, policy_version 246477 (0.0007) [2023-12-27 07:21:29,659][20347] Updated weights for policy 0, policy_version 246837 (0.0008) [2023-12-27 07:21:29,720][20347] Updated weights for policy 0, policy_version 246847 (0.0006) [2023-12-27 07:21:30,314][20347] Updated weights for policy 0, policy_version 246857 (0.0009) [2023-12-27 07:21:30,335][20381] Updated weights for policy 1, policy_version 246487 (0.0006) [2023-12-27 07:21:30,364][20347] Updated weights for policy 0, policy_version 246867 (0.0007) [2023-12-27 07:21:30,392][20381] Updated weights for policy 1, policy_version 246497 (0.0010) [2023-12-27 07:21:30,412][20347] Updated weights for policy 0, policy_version 246877 (0.0005) [2023-12-27 07:21:30,443][20381] Updated weights for policy 1, policy_version 246507 (0.0010) [2023-12-27 07:21:30,460][20347] Updated weights for policy 0, policy_version 246887 (0.0006) [2023-12-27 07:21:31,110][20347] Updated weights for policy 0, policy_version 246897 (0.0007) [2023-12-27 07:21:31,125][20381] Updated weights for policy 1, policy_version 246517 (0.0010) [2023-12-27 07:21:31,169][20347] Updated weights for policy 0, policy_version 246907 (0.0008) [2023-12-27 07:21:31,194][20381] Updated weights for policy 1, policy_version 246527 (0.0011) [2023-12-27 07:21:31,222][20347] Updated weights for policy 0, policy_version 246917 (0.0005) [2023-12-27 07:21:31,249][20381] Updated weights for policy 1, policy_version 246537 (0.0011) [2023-12-27 07:21:31,881][20381] Updated weights for policy 1, policy_version 246547 (0.0009) [2023-12-27 07:21:31,934][20381] Updated weights for policy 1, policy_version 246557 (0.0006) [2023-12-27 07:21:31,937][20347] Updated weights for policy 0, policy_version 246927 (0.0006) [2023-12-27 07:21:31,987][20381] Updated weights for policy 1, policy_version 246567 (0.0008) [2023-12-27 07:21:31,989][20347] Updated weights for policy 0, policy_version 246937 (0.0006) [2023-12-27 07:21:32,049][20347] Updated weights for policy 0, policy_version 246947 (0.0007) [2023-12-27 07:21:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 126369792. Throughput: 0: 10375.7, 1: 10447.6. Samples: 126378512. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:32,607][20381] Updated weights for policy 1, policy_version 246577 (0.0007) [2023-12-27 07:21:32,663][20381] Updated weights for policy 1, policy_version 246587 (0.0007) [2023-12-27 07:21:32,721][20381] Updated weights for policy 1, policy_version 246597 (0.0007) [2023-12-27 07:21:32,747][20347] Updated weights for policy 0, policy_version 246957 (0.0007) [2023-12-27 07:21:32,774][20381] Updated weights for policy 1, policy_version 246607 (0.0006) [2023-12-27 07:21:32,808][20347] Updated weights for policy 0, policy_version 246967 (0.0006) [2023-12-27 07:21:32,869][20347] Updated weights for policy 0, policy_version 246977 (0.0008) [2023-12-27 07:21:33,424][20381] Updated weights for policy 1, policy_version 246617 (0.0006) [2023-12-27 07:21:33,482][20381] Updated weights for policy 1, policy_version 246627 (0.0007) [2023-12-27 07:21:33,541][20381] Updated weights for policy 1, policy_version 246637 (0.0008) [2023-12-27 07:21:33,543][20347] Updated weights for policy 0, policy_version 246987 (0.0006) [2023-12-27 07:21:33,597][20347] Updated weights for policy 0, policy_version 246997 (0.0008) [2023-12-27 07:21:33,653][20347] Updated weights for policy 0, policy_version 247007 (0.0008) [2023-12-27 07:21:34,098][20381] Updated weights for policy 1, policy_version 246647 (0.0005) [2023-12-27 07:21:34,159][20381] Updated weights for policy 1, policy_version 246657 (0.0006) [2023-12-27 07:21:34,214][20381] Updated weights for policy 1, policy_version 246667 (0.0008) [2023-12-27 07:21:34,308][20347] Updated weights for policy 0, policy_version 247017 (0.0008) [2023-12-27 07:21:34,373][20347] Updated weights for policy 0, policy_version 247027 (0.0006) [2023-12-27 07:21:34,444][20347] Updated weights for policy 0, policy_version 247037 (0.0006) [2023-12-27 07:21:34,511][20347] Updated weights for policy 0, policy_version 247047 (0.0005) [2023-12-27 07:21:34,839][20381] Updated weights for policy 1, policy_version 246677 (0.0009) [2023-12-27 07:21:34,894][20381] Updated weights for policy 1, policy_version 246687 (0.0010) [2023-12-27 07:21:34,949][20381] Updated weights for policy 1, policy_version 246697 (0.0011) [2023-12-27 07:21:35,139][20347] Updated weights for policy 0, policy_version 247057 (0.0007) [2023-12-27 07:21:35,189][20347] Updated weights for policy 0, policy_version 247067 (0.0007) [2023-12-27 07:21:35,244][20347] Updated weights for policy 0, policy_version 247077 (0.0006) [2023-12-27 07:21:35,609][20381] Updated weights for policy 1, policy_version 246707 (0.0009) [2023-12-27 07:21:35,661][20381] Updated weights for policy 1, policy_version 246717 (0.0010) [2023-12-27 07:21:35,715][20381] Updated weights for policy 1, policy_version 246727 (0.0010) [2023-12-27 07:21:35,834][20347] Updated weights for policy 0, policy_version 247087 (0.0007) [2023-12-27 07:21:35,888][20347] Updated weights for policy 0, policy_version 247097 (0.0006) [2023-12-27 07:21:35,948][20347] Updated weights for policy 0, policy_version 247107 (0.0006) [2023-12-27 07:21:36,416][20381] Updated weights for policy 1, policy_version 246737 (0.0011) [2023-12-27 07:21:36,475][20381] Updated weights for policy 1, policy_version 246747 (0.0011) [2023-12-27 07:21:36,538][20381] Updated weights for policy 1, policy_version 246757 (0.0011) [2023-12-27 07:21:36,584][20347] Updated weights for policy 0, policy_version 247117 (0.0006) [2023-12-27 07:21:36,601][20381] Updated weights for policy 1, policy_version 246767 (0.0011) [2023-12-27 07:21:36,649][20347] Updated weights for policy 0, policy_version 247127 (0.0007) [2023-12-27 07:21:36,712][20347] Updated weights for policy 0, policy_version 247137 (0.0007) [2023-12-27 07:21:37,221][20381] Updated weights for policy 1, policy_version 246777 (0.0006) [2023-12-27 07:21:37,280][20381] Updated weights for policy 1, policy_version 246787 (0.0005) [2023-12-27 07:21:37,334][20381] Updated weights for policy 1, policy_version 246797 (0.0006) [2023-12-27 07:21:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 126476288. Throughput: 0: 10362.5, 1: 10441.9. Samples: 126442380. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000246800_63193088.pth... [2023-12-27 07:21:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000247144_63283200.pth... [2023-12-27 07:21:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000245584_62881792.pth [2023-12-27 07:21:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000245928_62971904.pth [2023-12-27 07:21:37,436][20347] Updated weights for policy 0, policy_version 247147 (0.0007) [2023-12-27 07:21:37,487][20347] Updated weights for policy 0, policy_version 247157 (0.0007) [2023-12-27 07:21:37,545][20347] Updated weights for policy 0, policy_version 247167 (0.0008) [2023-12-27 07:21:37,987][20381] Updated weights for policy 1, policy_version 246807 (0.0006) [2023-12-27 07:21:38,055][20381] Updated weights for policy 1, policy_version 246817 (0.0010) [2023-12-27 07:21:38,122][20381] Updated weights for policy 1, policy_version 246827 (0.0010) [2023-12-27 07:21:38,154][20347] Updated weights for policy 0, policy_version 247177 (0.0005) [2023-12-27 07:21:38,200][20347] Updated weights for policy 0, policy_version 247187 (0.0005) [2023-12-27 07:21:38,246][20347] Updated weights for policy 0, policy_version 247197 (0.0005) [2023-12-27 07:21:38,298][20347] Updated weights for policy 0, policy_version 247207 (0.0009) [2023-12-27 07:21:38,760][20381] Updated weights for policy 1, policy_version 246837 (0.0009) [2023-12-27 07:21:38,820][20381] Updated weights for policy 1, policy_version 246847 (0.0008) [2023-12-27 07:21:38,882][20381] Updated weights for policy 1, policy_version 246857 (0.0007) [2023-12-27 07:21:38,986][20347] Updated weights for policy 0, policy_version 247217 (0.0011) [2023-12-27 07:21:39,051][20347] Updated weights for policy 0, policy_version 247227 (0.0011) [2023-12-27 07:21:39,106][20347] Updated weights for policy 0, policy_version 247237 (0.0010) [2023-12-27 07:21:39,567][20381] Updated weights for policy 1, policy_version 246867 (0.0007) [2023-12-27 07:21:39,625][20381] Updated weights for policy 1, policy_version 246877 (0.0007) [2023-12-27 07:21:39,686][20381] Updated weights for policy 1, policy_version 246887 (0.0007) [2023-12-27 07:21:39,871][20347] Updated weights for policy 0, policy_version 247247 (0.0009) [2023-12-27 07:21:39,933][20347] Updated weights for policy 0, policy_version 247257 (0.0008) [2023-12-27 07:21:39,994][20347] Updated weights for policy 0, policy_version 247267 (0.0009) [2023-12-27 07:21:40,325][20381] Updated weights for policy 1, policy_version 246897 (0.0007) [2023-12-27 07:21:40,381][20381] Updated weights for policy 1, policy_version 246907 (0.0008) [2023-12-27 07:21:40,425][20381] Updated weights for policy 1, policy_version 246917 (0.0007) [2023-12-27 07:21:40,478][20381] Updated weights for policy 1, policy_version 246927 (0.0005) [2023-12-27 07:21:40,743][20347] Updated weights for policy 0, policy_version 247277 (0.0008) [2023-12-27 07:21:40,802][20347] Updated weights for policy 0, policy_version 247287 (0.0007) [2023-12-27 07:21:40,851][20347] Updated weights for policy 0, policy_version 247297 (0.0008) [2023-12-27 07:21:41,163][20381] Updated weights for policy 1, policy_version 246937 (0.0007) [2023-12-27 07:21:41,222][20381] Updated weights for policy 1, policy_version 246947 (0.0006) [2023-12-27 07:21:41,287][20381] Updated weights for policy 1, policy_version 246957 (0.0006) [2023-12-27 07:21:41,569][20347] Updated weights for policy 0, policy_version 247307 (0.0006) [2023-12-27 07:21:41,631][20347] Updated weights for policy 0, policy_version 247317 (0.0008) [2023-12-27 07:21:41,691][20347] Updated weights for policy 0, policy_version 247327 (0.0008) [2023-12-27 07:21:41,960][20381] Updated weights for policy 1, policy_version 246967 (0.0007) [2023-12-27 07:21:42,025][20381] Updated weights for policy 1, policy_version 246977 (0.0008) [2023-12-27 07:21:42,088][20381] Updated weights for policy 1, policy_version 246987 (0.0008) [2023-12-27 07:21:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 126574592. Throughput: 0: 10381.2, 1: 10407.4. Samples: 126565980. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:42,383][20347] Updated weights for policy 0, policy_version 247337 (0.0008) [2023-12-27 07:21:42,438][20347] Updated weights for policy 0, policy_version 247347 (0.0008) [2023-12-27 07:21:42,498][20347] Updated weights for policy 0, policy_version 247357 (0.0008) [2023-12-27 07:21:42,561][20347] Updated weights for policy 0, policy_version 247367 (0.0008) [2023-12-27 07:21:42,762][20381] Updated weights for policy 1, policy_version 246997 (0.0007) [2023-12-27 07:21:42,818][20381] Updated weights for policy 1, policy_version 247007 (0.0007) [2023-12-27 07:21:42,871][20381] Updated weights for policy 1, policy_version 247017 (0.0008) [2023-12-27 07:21:43,257][20347] Updated weights for policy 0, policy_version 247377 (0.0008) [2023-12-27 07:21:43,309][20347] Updated weights for policy 0, policy_version 247387 (0.0008) [2023-12-27 07:21:43,360][20347] Updated weights for policy 0, policy_version 247397 (0.0007) [2023-12-27 07:21:43,568][20381] Updated weights for policy 1, policy_version 247027 (0.0008) [2023-12-27 07:21:43,615][20381] Updated weights for policy 1, policy_version 247037 (0.0010) [2023-12-27 07:21:43,667][20381] Updated weights for policy 1, policy_version 247047 (0.0009) [2023-12-27 07:21:44,025][20347] Updated weights for policy 0, policy_version 247407 (0.0008) [2023-12-27 07:21:44,079][20347] Updated weights for policy 0, policy_version 247417 (0.0008) [2023-12-27 07:21:44,128][20347] Updated weights for policy 0, policy_version 247427 (0.0008) [2023-12-27 07:21:44,298][20381] Updated weights for policy 1, policy_version 247057 (0.0006) [2023-12-27 07:21:44,356][20381] Updated weights for policy 1, policy_version 247067 (0.0008) [2023-12-27 07:21:44,401][20381] Updated weights for policy 1, policy_version 247077 (0.0007) [2023-12-27 07:21:44,456][20381] Updated weights for policy 1, policy_version 247087 (0.0008) [2023-12-27 07:21:44,773][20347] Updated weights for policy 0, policy_version 247437 (0.0007) [2023-12-27 07:21:44,827][20347] Updated weights for policy 0, policy_version 247447 (0.0007) [2023-12-27 07:21:44,878][20347] Updated weights for policy 0, policy_version 247457 (0.0008) [2023-12-27 07:21:45,177][20381] Updated weights for policy 1, policy_version 247097 (0.0008) [2023-12-27 07:21:45,236][20381] Updated weights for policy 1, policy_version 247107 (0.0008) [2023-12-27 07:21:45,295][20381] Updated weights for policy 1, policy_version 247117 (0.0008) [2023-12-27 07:21:45,560][20347] Updated weights for policy 0, policy_version 247467 (0.0009) [2023-12-27 07:21:45,622][20347] Updated weights for policy 0, policy_version 247477 (0.0005) [2023-12-27 07:21:45,685][20347] Updated weights for policy 0, policy_version 247487 (0.0005) [2023-12-27 07:21:45,988][20381] Updated weights for policy 1, policy_version 247127 (0.0010) [2023-12-27 07:21:46,036][20381] Updated weights for policy 1, policy_version 247137 (0.0010) [2023-12-27 07:21:46,097][20381] Updated weights for policy 1, policy_version 247147 (0.0011) [2023-12-27 07:21:46,253][20347] Updated weights for policy 0, policy_version 247497 (0.0006) [2023-12-27 07:21:46,316][20347] Updated weights for policy 0, policy_version 247507 (0.0005) [2023-12-27 07:21:46,363][20347] Updated weights for policy 0, policy_version 247517 (0.0005) [2023-12-27 07:21:46,411][20347] Updated weights for policy 0, policy_version 247527 (0.0005) [2023-12-27 07:21:46,716][20381] Updated weights for policy 1, policy_version 247157 (0.0008) [2023-12-27 07:21:46,766][20381] Updated weights for policy 1, policy_version 247167 (0.0007) [2023-12-27 07:21:46,821][20381] Updated weights for policy 1, policy_version 247177 (0.0006) [2023-12-27 07:21:46,946][20347] Updated weights for policy 0, policy_version 247537 (0.0005) [2023-12-27 07:21:47,009][20347] Updated weights for policy 0, policy_version 247547 (0.0006) [2023-12-27 07:21:47,059][20347] Updated weights for policy 0, policy_version 247557 (0.0005) [2023-12-27 07:21:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 126681088. Throughput: 0: 10424.4, 1: 10383.7. Samples: 126692396. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:47,441][20381] Updated weights for policy 1, policy_version 247187 (0.0007) [2023-12-27 07:21:47,498][20381] Updated weights for policy 1, policy_version 247197 (0.0010) [2023-12-27 07:21:47,556][20381] Updated weights for policy 1, policy_version 247207 (0.0010) [2023-12-27 07:21:47,653][20347] Updated weights for policy 0, policy_version 247567 (0.0009) [2023-12-27 07:21:47,719][20347] Updated weights for policy 0, policy_version 247577 (0.0007) [2023-12-27 07:21:47,780][20347] Updated weights for policy 0, policy_version 247587 (0.0009) [2023-12-27 07:21:48,191][20381] Updated weights for policy 1, policy_version 247217 (0.0010) [2023-12-27 07:21:48,253][20381] Updated weights for policy 1, policy_version 247227 (0.0007) [2023-12-27 07:21:48,313][20381] Updated weights for policy 1, policy_version 247237 (0.0006) [2023-12-27 07:21:48,356][20347] Updated weights for policy 0, policy_version 247597 (0.0009) [2023-12-27 07:21:48,377][20381] Updated weights for policy 1, policy_version 247247 (0.0007) [2023-12-27 07:21:48,417][20347] Updated weights for policy 0, policy_version 247607 (0.0011) [2023-12-27 07:21:48,476][20347] Updated weights for policy 0, policy_version 247617 (0.0010) [2023-12-27 07:21:48,985][20381] Updated weights for policy 1, policy_version 247257 (0.0007) [2023-12-27 07:21:49,040][20381] Updated weights for policy 1, policy_version 247267 (0.0007) [2023-12-27 07:21:49,088][20381] Updated weights for policy 1, policy_version 247277 (0.0007) [2023-12-27 07:21:49,156][20347] Updated weights for policy 0, policy_version 247627 (0.0010) [2023-12-27 07:21:49,201][20347] Updated weights for policy 0, policy_version 247637 (0.0010) [2023-12-27 07:21:49,266][20347] Updated weights for policy 0, policy_version 247647 (0.0010) [2023-12-27 07:21:49,857][20381] Updated weights for policy 1, policy_version 247287 (0.0008) [2023-12-27 07:21:49,917][20381] Updated weights for policy 1, policy_version 247297 (0.0006) [2023-12-27 07:21:49,962][20347] Updated weights for policy 0, policy_version 247657 (0.0010) [2023-12-27 07:21:49,980][20381] Updated weights for policy 1, policy_version 247307 (0.0007) [2023-12-27 07:21:50,019][20347] Updated weights for policy 0, policy_version 247667 (0.0008) [2023-12-27 07:21:50,073][20347] Updated weights for policy 0, policy_version 247677 (0.0006) [2023-12-27 07:21:50,127][20347] Updated weights for policy 0, policy_version 247687 (0.0006) [2023-12-27 07:21:50,698][20381] Updated weights for policy 1, policy_version 247317 (0.0007) [2023-12-27 07:21:50,749][20381] Updated weights for policy 1, policy_version 247327 (0.0006) [2023-12-27 07:21:50,763][20347] Updated weights for policy 0, policy_version 247697 (0.0007) [2023-12-27 07:21:50,802][20381] Updated weights for policy 1, policy_version 247337 (0.0008) [2023-12-27 07:21:50,817][20347] Updated weights for policy 0, policy_version 247707 (0.0005) [2023-12-27 07:21:50,868][20347] Updated weights for policy 0, policy_version 247717 (0.0005) [2023-12-27 07:21:51,479][20347] Updated weights for policy 0, policy_version 247727 (0.0007) [2023-12-27 07:21:51,536][20347] Updated weights for policy 0, policy_version 247737 (0.0008) [2023-12-27 07:21:51,549][20381] Updated weights for policy 1, policy_version 247347 (0.0007) [2023-12-27 07:21:51,589][20347] Updated weights for policy 0, policy_version 247747 (0.0006) [2023-12-27 07:21:51,599][20381] Updated weights for policy 1, policy_version 247357 (0.0005) [2023-12-27 07:21:51,661][20381] Updated weights for policy 1, policy_version 247367 (0.0008) [2023-12-27 07:21:52,325][20347] Updated weights for policy 0, policy_version 247757 (0.0007) [2023-12-27 07:21:52,335][20381] Updated weights for policy 1, policy_version 247377 (0.0008) [2023-12-27 07:21:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 126779392. Throughput: 0: 10458.2, 1: 10366.4. Samples: 126754456. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:52,387][20347] Updated weights for policy 0, policy_version 247767 (0.0008) [2023-12-27 07:21:52,399][20381] Updated weights for policy 1, policy_version 247387 (0.0008) [2023-12-27 07:21:52,442][20347] Updated weights for policy 0, policy_version 247777 (0.0006) [2023-12-27 07:21:52,459][20381] Updated weights for policy 1, policy_version 247397 (0.0011) [2023-12-27 07:21:52,472][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000247784_63447040.pth... [2023-12-27 07:21:52,475][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000246536_63127552.pth [2023-12-27 07:21:52,519][20381] Updated weights for policy 1, policy_version 247407 (0.0011) [2023-12-27 07:21:52,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000247408_63348736.pth... [2023-12-27 07:21:52,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000246160_63029248.pth [2023-12-27 07:21:53,119][20347] Updated weights for policy 0, policy_version 247787 (0.0006) [2023-12-27 07:21:53,167][20347] Updated weights for policy 0, policy_version 247797 (0.0007) [2023-12-27 07:21:53,173][20381] Updated weights for policy 1, policy_version 247417 (0.0008) [2023-12-27 07:21:53,212][20347] Updated weights for policy 0, policy_version 247807 (0.0005) [2023-12-27 07:21:53,229][20381] Updated weights for policy 1, policy_version 247427 (0.0010) [2023-12-27 07:21:53,284][20381] Updated weights for policy 1, policy_version 247437 (0.0011) [2023-12-27 07:21:53,912][20347] Updated weights for policy 0, policy_version 247817 (0.0005) [2023-12-27 07:21:53,960][20347] Updated weights for policy 0, policy_version 247827 (0.0005) [2023-12-27 07:21:54,020][20347] Updated weights for policy 0, policy_version 247837 (0.0007) [2023-12-27 07:21:54,022][20381] Updated weights for policy 1, policy_version 247447 (0.0008) [2023-12-27 07:21:54,079][20347] Updated weights for policy 0, policy_version 247847 (0.0007) [2023-12-27 07:21:54,082][20381] Updated weights for policy 1, policy_version 247457 (0.0008) [2023-12-27 07:21:54,140][20381] Updated weights for policy 1, policy_version 247467 (0.0007) [2023-12-27 07:21:54,729][20347] Updated weights for policy 0, policy_version 247857 (0.0008) [2023-12-27 07:21:54,791][20347] Updated weights for policy 0, policy_version 247867 (0.0008) [2023-12-27 07:21:54,840][20381] Updated weights for policy 1, policy_version 247477 (0.0007) [2023-12-27 07:21:54,854][20347] Updated weights for policy 0, policy_version 247877 (0.0006) [2023-12-27 07:21:54,895][20381] Updated weights for policy 1, policy_version 247487 (0.0007) [2023-12-27 07:21:54,951][20381] Updated weights for policy 1, policy_version 247497 (0.0007) [2023-12-27 07:21:55,522][20347] Updated weights for policy 0, policy_version 247887 (0.0008) [2023-12-27 07:21:55,586][20347] Updated weights for policy 0, policy_version 247897 (0.0007) [2023-12-27 07:21:55,643][20347] Updated weights for policy 0, policy_version 247907 (0.0007) [2023-12-27 07:21:55,670][20381] Updated weights for policy 1, policy_version 247507 (0.0008) [2023-12-27 07:21:55,718][20381] Updated weights for policy 1, policy_version 247517 (0.0007) [2023-12-27 07:21:55,773][20381] Updated weights for policy 1, policy_version 247527 (0.0006) [2023-12-27 07:21:56,315][20347] Updated weights for policy 0, policy_version 247917 (0.0006) [2023-12-27 07:21:56,350][20381] Updated weights for policy 1, policy_version 247537 (0.0005) [2023-12-27 07:21:56,372][20347] Updated weights for policy 0, policy_version 247927 (0.0005) [2023-12-27 07:21:56,405][20381] Updated weights for policy 1, policy_version 247547 (0.0008) [2023-12-27 07:21:56,433][20347] Updated weights for policy 0, policy_version 247937 (0.0009) [2023-12-27 07:21:56,457][20381] Updated weights for policy 1, policy_version 247557 (0.0008) [2023-12-27 07:21:56,512][20381] Updated weights for policy 1, policy_version 247567 (0.0010) [2023-12-27 07:21:57,074][20347] Updated weights for policy 0, policy_version 247947 (0.0010) [2023-12-27 07:21:57,129][20347] Updated weights for policy 0, policy_version 247957 (0.0007) [2023-12-27 07:21:57,184][20347] Updated weights for policy 0, policy_version 247967 (0.0010) [2023-12-27 07:21:57,200][20381] Updated weights for policy 1, policy_version 247578 (0.0009) [2023-12-27 07:21:57,248][20381] Updated weights for policy 1, policy_version 247588 (0.0006) [2023-12-27 07:21:57,302][20381] Updated weights for policy 1, policy_version 247598 (0.0007) [2023-12-27 07:21:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 126894080. Throughput: 0: 10416.9, 1: 10366.1. Samples: 126876492. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:21:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:21:57,853][20347] Updated weights for policy 0, policy_version 247977 (0.0010) [2023-12-27 07:21:57,908][20347] Updated weights for policy 0, policy_version 247987 (0.0010) [2023-12-27 07:21:57,962][20347] Updated weights for policy 0, policy_version 247997 (0.0010) [2023-12-27 07:21:57,967][20381] Updated weights for policy 1, policy_version 247608 (0.0010) [2023-12-27 07:21:58,017][20347] Updated weights for policy 0, policy_version 248007 (0.0010) [2023-12-27 07:21:58,019][20381] Updated weights for policy 1, policy_version 247618 (0.0010) [2023-12-27 07:21:58,069][20381] Updated weights for policy 1, policy_version 247628 (0.0010) [2023-12-27 07:21:58,722][20347] Updated weights for policy 0, policy_version 248017 (0.0008) [2023-12-27 07:21:58,782][20347] Updated weights for policy 0, policy_version 248027 (0.0007) [2023-12-27 07:21:58,807][20381] Updated weights for policy 1, policy_version 247638 (0.0009) [2023-12-27 07:21:58,842][20347] Updated weights for policy 0, policy_version 248037 (0.0007) [2023-12-27 07:21:58,871][20381] Updated weights for policy 1, policy_version 247648 (0.0007) [2023-12-27 07:21:58,931][20381] Updated weights for policy 1, policy_version 247658 (0.0006) [2023-12-27 07:21:59,555][20347] Updated weights for policy 0, policy_version 248047 (0.0006) [2023-12-27 07:21:59,589][20381] Updated weights for policy 1, policy_version 247668 (0.0006) [2023-12-27 07:21:59,608][20347] Updated weights for policy 0, policy_version 248057 (0.0007) [2023-12-27 07:21:59,648][20381] Updated weights for policy 1, policy_version 247678 (0.0010) [2023-12-27 07:21:59,666][20347] Updated weights for policy 0, policy_version 248067 (0.0006) [2023-12-27 07:21:59,710][20381] Updated weights for policy 1, policy_version 247688 (0.0011) [2023-12-27 07:22:00,340][20347] Updated weights for policy 0, policy_version 248077 (0.0007) [2023-12-27 07:22:00,358][20381] Updated weights for policy 1, policy_version 247698 (0.0009) [2023-12-27 07:22:00,391][20347] Updated weights for policy 0, policy_version 248087 (0.0007) [2023-12-27 07:22:00,408][20381] Updated weights for policy 1, policy_version 247708 (0.0008) [2023-12-27 07:22:00,446][20347] Updated weights for policy 0, policy_version 248097 (0.0007) [2023-12-27 07:22:00,460][20381] Updated weights for policy 1, policy_version 247718 (0.0007) [2023-12-27 07:22:00,503][20381] Updated weights for policy 1, policy_version 247728 (0.0007) [2023-12-27 07:22:01,040][20347] Updated weights for policy 0, policy_version 248107 (0.0007) [2023-12-27 07:22:01,093][20347] Updated weights for policy 0, policy_version 248117 (0.0008) [2023-12-27 07:22:01,156][20347] Updated weights for policy 0, policy_version 248127 (0.0008) [2023-12-27 07:22:01,211][20381] Updated weights for policy 1, policy_version 247738 (0.0011) [2023-12-27 07:22:01,263][20381] Updated weights for policy 1, policy_version 247748 (0.0010) [2023-12-27 07:22:01,324][20381] Updated weights for policy 1, policy_version 247758 (0.0008) [2023-12-27 07:22:01,892][20347] Updated weights for policy 0, policy_version 248137 (0.0006) [2023-12-27 07:22:01,930][20381] Updated weights for policy 1, policy_version 247768 (0.0005) [2023-12-27 07:22:01,947][20347] Updated weights for policy 0, policy_version 248147 (0.0008) [2023-12-27 07:22:01,986][20381] Updated weights for policy 1, policy_version 247778 (0.0010) [2023-12-27 07:22:02,002][20347] Updated weights for policy 0, policy_version 248157 (0.0007) [2023-12-27 07:22:02,042][20381] Updated weights for policy 1, policy_version 247788 (0.0008) [2023-12-27 07:22:02,067][20347] Updated weights for policy 0, policy_version 248167 (0.0008) [2023-12-27 07:22:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 126992384. Throughput: 0: 10414.4, 1: 10369.0. Samples: 126999756. Policy #0 lag: (min: 19.0, avg: 23.1, max: 51.0) [2023-12-27 07:22:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:22:02,752][20381] Updated weights for policy 1, policy_version 247798 (0.0006) [2023-12-27 07:22:02,768][20347] Updated weights for policy 0, policy_version 248177 (0.0009) [2023-12-27 07:22:02,811][20381] Updated weights for policy 1, policy_version 247808 (0.0006) [2023-12-27 07:22:02,817][20347] Updated weights for policy 0, policy_version 248187 (0.0006) [2023-12-27 07:22:02,860][20381] Updated weights for policy 1, policy_version 247818 (0.0006) [2023-12-27 07:22:02,865][20347] Updated weights for policy 0, policy_version 248197 (0.0007) [2023-12-27 07:22:03,513][20381] Updated weights for policy 1, policy_version 247828 (0.0007) [2023-12-27 07:22:03,526][20347] Updated weights for policy 0, policy_version 248207 (0.0005) [2023-12-27 07:22:03,558][20381] Updated weights for policy 1, policy_version 247838 (0.0005) [2023-12-27 07:22:03,578][20347] Updated weights for policy 0, policy_version 248217 (0.0006) [2023-12-27 07:22:03,606][20381] Updated weights for policy 1, policy_version 247848 (0.0006) [2023-12-27 07:22:03,631][20347] Updated weights for policy 0, policy_version 248227 (0.0005) [2023-12-27 07:22:04,337][20347] Updated weights for policy 0, policy_version 248237 (0.0007) [2023-12-27 07:22:04,367][20381] Updated weights for policy 1, policy_version 247858 (0.0007) [2023-12-27 07:22:04,391][20347] Updated weights for policy 0, policy_version 248247 (0.0006) [2023-12-27 07:22:04,421][20381] Updated weights for policy 1, policy_version 247868 (0.0006) [2023-12-27 07:22:04,458][20347] Updated weights for policy 0, policy_version 248257 (0.0006) [2023-12-27 07:22:04,479][20381] Updated weights for policy 1, policy_version 247878 (0.0007) [2023-12-27 07:22:04,530][20381] Updated weights for policy 1, policy_version 247888 (0.0007) [2023-12-27 07:22:05,071][20347] Updated weights for policy 0, policy_version 248267 (0.0006) [2023-12-27 07:22:05,105][20291] KL-divergence is very high: 1400.5397 [2023-12-27 07:22:05,138][20347] Updated weights for policy 0, policy_version 248277 (0.0008) [2023-12-27 07:22:05,191][20347] Updated weights for policy 0, policy_version 248287 (0.0007) [2023-12-27 07:22:05,196][20291] KL-divergence is very high: 10802.0195 [2023-12-27 07:22:05,250][20381] Updated weights for policy 1, policy_version 247898 (0.0007) [2023-12-27 07:22:05,315][20381] Updated weights for policy 1, policy_version 247908 (0.0007) [2023-12-27 07:22:05,373][20381] Updated weights for policy 1, policy_version 247918 (0.0007) [2023-12-27 07:22:05,834][20291] KL-divergence is very high: 284.8002 [2023-12-27 07:22:05,840][20347] Updated weights for policy 0, policy_version 248297 (0.0009) [2023-12-27 07:22:05,845][20291] KL-divergence is very high: 1227.1503 [2023-12-27 07:22:05,850][20291] KL-divergence is very high: 169.7885 [2023-12-27 07:22:05,874][20291] KL-divergence is very high: 368.8266 [2023-12-27 07:22:05,885][20291] KL-divergence is very high: 803.3355 [2023-12-27 07:22:05,891][20347] Updated weights for policy 0, policy_version 248307 (0.0010) [2023-12-27 07:22:05,919][20291] KL-divergence is very high: 287.1136 [2023-12-27 07:22:05,928][20291] KL-divergence is very high: 546.0933 [2023-12-27 07:22:05,942][20347] Updated weights for policy 0, policy_version 248317 (0.0010) [2023-12-27 07:22:05,957][20291] KL-divergence is very high: 319.7115 [2023-12-27 07:22:05,967][20291] KL-divergence is very high: 919.3322 [2023-12-27 07:22:05,972][20291] KL-divergence is very high: 105.8238 [2023-12-27 07:22:05,990][20347] Updated weights for policy 0, policy_version 248327 (0.0010) [2023-12-27 07:22:06,058][20381] Updated weights for policy 1, policy_version 247928 (0.0006) [2023-12-27 07:22:06,116][20381] Updated weights for policy 1, policy_version 247938 (0.0006) [2023-12-27 07:22:06,173][20381] Updated weights for policy 1, policy_version 247948 (0.0008) [2023-12-27 07:22:06,713][20347] Updated weights for policy 0, policy_version 248337 (0.0008) [2023-12-27 07:22:06,768][20347] Updated weights for policy 0, policy_version 248347 (0.0007) [2023-12-27 07:22:06,823][20347] Updated weights for policy 0, policy_version 248357 (0.0010) [2023-12-27 07:22:06,831][20381] Updated weights for policy 1, policy_version 247958 (0.0009) [2023-12-27 07:22:06,881][20381] Updated weights for policy 1, policy_version 247968 (0.0011) [2023-12-27 07:22:06,934][20381] Updated weights for policy 1, policy_version 247978 (0.0011) [2023-12-27 07:22:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 127090688. Throughput: 0: 10439.7, 1: 10358.5. Samples: 127061452. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:07,364][19170] Avg episode reward: [(0, '971.090'), (1, '1000.000')] [2023-12-27 07:22:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000247984_63496192.pth... [2023-12-27 07:22:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000246800_63193088.pth [2023-12-27 07:22:07,404][20347] Updated weights for policy 0, policy_version 248367 (0.0008) [2023-12-27 07:22:07,458][20347] Updated weights for policy 0, policy_version 248377 (0.0006) [2023-12-27 07:22:07,515][20347] Updated weights for policy 0, policy_version 248387 (0.0005) [2023-12-27 07:22:07,541][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000248392_63602688.pth... [2023-12-27 07:22:07,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000247144_63283200.pth [2023-12-27 07:22:07,632][20381] Updated weights for policy 1, policy_version 247988 (0.0009) [2023-12-27 07:22:07,694][20381] Updated weights for policy 1, policy_version 247998 (0.0006) [2023-12-27 07:22:07,742][20381] Updated weights for policy 1, policy_version 248008 (0.0010) [2023-12-27 07:22:08,155][20347] Updated weights for policy 0, policy_version 248397 (0.0007) [2023-12-27 07:22:08,208][20347] Updated weights for policy 0, policy_version 248407 (0.0007) [2023-12-27 07:22:08,256][20347] Updated weights for policy 0, policy_version 248417 (0.0007) [2023-12-27 07:22:08,354][20381] Updated weights for policy 1, policy_version 248018 (0.0010) [2023-12-27 07:22:08,416][20381] Updated weights for policy 1, policy_version 248028 (0.0011) [2023-12-27 07:22:08,469][20381] Updated weights for policy 1, policy_version 248039 (0.0011) [2023-12-27 07:22:08,874][20347] Updated weights for policy 0, policy_version 248427 (0.0007) [2023-12-27 07:22:08,927][20347] Updated weights for policy 0, policy_version 248437 (0.0007) [2023-12-27 07:22:08,972][20347] Updated weights for policy 0, policy_version 248447 (0.0009) [2023-12-27 07:22:09,224][20381] Updated weights for policy 1, policy_version 248049 (0.0011) [2023-12-27 07:22:09,276][20381] Updated weights for policy 1, policy_version 248059 (0.0008) [2023-12-27 07:22:09,330][20381] Updated weights for policy 1, policy_version 248069 (0.0008) [2023-12-27 07:22:09,393][20381] Updated weights for policy 1, policy_version 248079 (0.0009) [2023-12-27 07:22:09,714][20347] Updated weights for policy 0, policy_version 248457 (0.0007) [2023-12-27 07:22:09,771][20347] Updated weights for policy 0, policy_version 248467 (0.0007) [2023-12-27 07:22:09,826][20347] Updated weights for policy 0, policy_version 248477 (0.0007) [2023-12-27 07:22:09,891][20347] Updated weights for policy 0, policy_version 248487 (0.0007) [2023-12-27 07:22:10,139][20381] Updated weights for policy 1, policy_version 248089 (0.0008) [2023-12-27 07:22:10,147][20292] KL-divergence is very high: 269.5536 [2023-12-27 07:22:10,201][20381] Updated weights for policy 1, policy_version 248099 (0.0008) [2023-12-27 07:22:10,262][20381] Updated weights for policy 1, policy_version 248109 (0.0008) [2023-12-27 07:22:10,634][20347] Updated weights for policy 0, policy_version 248497 (0.0008) [2023-12-27 07:22:10,690][20347] Updated weights for policy 0, policy_version 248507 (0.0007) [2023-12-27 07:22:10,755][20347] Updated weights for policy 0, policy_version 248517 (0.0006) [2023-12-27 07:22:10,918][20381] Updated weights for policy 1, policy_version 248119 (0.0009) [2023-12-27 07:22:10,977][20381] Updated weights for policy 1, policy_version 248129 (0.0006) [2023-12-27 07:22:11,035][20381] Updated weights for policy 1, policy_version 248139 (0.0007) [2023-12-27 07:22:11,468][20347] Updated weights for policy 0, policy_version 248527 (0.0007) [2023-12-27 07:22:11,532][20347] Updated weights for policy 0, policy_version 248537 (0.0008) [2023-12-27 07:22:11,592][20347] Updated weights for policy 0, policy_version 248547 (0.0008) [2023-12-27 07:22:11,702][20381] Updated weights for policy 1, policy_version 248149 (0.0008) [2023-12-27 07:22:11,765][20381] Updated weights for policy 1, policy_version 248159 (0.0008) [2023-12-27 07:22:11,828][20381] Updated weights for policy 1, policy_version 248169 (0.0008) [2023-12-27 07:22:12,310][20347] Updated weights for policy 0, policy_version 248557 (0.0008) [2023-12-27 07:22:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 127188992. Throughput: 0: 10476.3, 1: 10324.1. Samples: 127184320. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:12,363][19170] Avg episode reward: [(0, '971.090'), (1, '990.370')] [2023-12-27 07:22:12,375][20347] Updated weights for policy 0, policy_version 248567 (0.0009) [2023-12-27 07:22:12,438][20347] Updated weights for policy 0, policy_version 248577 (0.0008) [2023-12-27 07:22:12,539][20381] Updated weights for policy 1, policy_version 248179 (0.0007) [2023-12-27 07:22:12,589][20381] Updated weights for policy 1, policy_version 248189 (0.0006) [2023-12-27 07:22:12,645][20381] Updated weights for policy 1, policy_version 248199 (0.0008) [2023-12-27 07:22:13,102][20347] Updated weights for policy 0, policy_version 248587 (0.0007) [2023-12-27 07:22:13,168][20347] Updated weights for policy 0, policy_version 248597 (0.0007) [2023-12-27 07:22:13,226][20347] Updated weights for policy 0, policy_version 248607 (0.0008) [2023-12-27 07:22:13,347][20381] Updated weights for policy 1, policy_version 248209 (0.0007) [2023-12-27 07:22:13,412][20381] Updated weights for policy 1, policy_version 248219 (0.0007) [2023-12-27 07:22:13,467][20381] Updated weights for policy 1, policy_version 248229 (0.0007) [2023-12-27 07:22:13,522][20381] Updated weights for policy 1, policy_version 248239 (0.0008) [2023-12-27 07:22:13,879][20347] Updated weights for policy 0, policy_version 248617 (0.0007) [2023-12-27 07:22:13,927][20347] Updated weights for policy 0, policy_version 248627 (0.0005) [2023-12-27 07:22:13,980][20347] Updated weights for policy 0, policy_version 248637 (0.0006) [2023-12-27 07:22:14,036][20347] Updated weights for policy 0, policy_version 248647 (0.0007) [2023-12-27 07:22:14,215][20381] Updated weights for policy 1, policy_version 248249 (0.0007) [2023-12-27 07:22:14,266][20381] Updated weights for policy 1, policy_version 248259 (0.0008) [2023-12-27 07:22:14,313][20381] Updated weights for policy 1, policy_version 248269 (0.0005) [2023-12-27 07:22:14,685][20347] Updated weights for policy 0, policy_version 248657 (0.0008) [2023-12-27 07:22:14,743][20347] Updated weights for policy 0, policy_version 248667 (0.0009) [2023-12-27 07:22:14,806][20347] Updated weights for policy 0, policy_version 248677 (0.0008) [2023-12-27 07:22:15,000][20381] Updated weights for policy 1, policy_version 248279 (0.0010) [2023-12-27 07:22:15,062][20381] Updated weights for policy 1, policy_version 248289 (0.0011) [2023-12-27 07:22:15,115][20381] Updated weights for policy 1, policy_version 248299 (0.0011) [2023-12-27 07:22:15,463][20347] Updated weights for policy 0, policy_version 248687 (0.0009) [2023-12-27 07:22:15,513][20347] Updated weights for policy 0, policy_version 248697 (0.0010) [2023-12-27 07:22:15,564][20347] Updated weights for policy 0, policy_version 248707 (0.0010) [2023-12-27 07:22:15,795][20381] Updated weights for policy 1, policy_version 248309 (0.0008) [2023-12-27 07:22:15,856][20381] Updated weights for policy 1, policy_version 248319 (0.0007) [2023-12-27 07:22:15,913][20381] Updated weights for policy 1, policy_version 248329 (0.0006) [2023-12-27 07:22:16,251][20347] Updated weights for policy 0, policy_version 248717 (0.0008) [2023-12-27 07:22:16,299][20347] Updated weights for policy 0, policy_version 248727 (0.0007) [2023-12-27 07:22:16,355][20347] Updated weights for policy 0, policy_version 248737 (0.0006) [2023-12-27 07:22:16,494][20381] Updated weights for policy 1, policy_version 248339 (0.0007) [2023-12-27 07:22:16,552][20381] Updated weights for policy 1, policy_version 248349 (0.0009) [2023-12-27 07:22:16,583][20292] KL-divergence is very high: 150.4281 [2023-12-27 07:22:16,613][20381] Updated weights for policy 1, policy_version 248359 (0.0005) [2023-12-27 07:22:16,629][20292] KL-divergence is very high: 106.3580 [2023-12-27 07:22:16,991][20347] Updated weights for policy 0, policy_version 248747 (0.0006) [2023-12-27 07:22:17,040][20347] Updated weights for policy 0, policy_version 248757 (0.0005) [2023-12-27 07:22:17,084][20347] Updated weights for policy 0, policy_version 248767 (0.0005) [2023-12-27 07:22:17,187][20381] Updated weights for policy 1, policy_version 248369 (0.0007) [2023-12-27 07:22:17,235][20381] Updated weights for policy 1, policy_version 248379 (0.0007) [2023-12-27 07:22:17,282][20381] Updated weights for policy 1, policy_version 248389 (0.0005) [2023-12-27 07:22:17,331][20381] Updated weights for policy 1, policy_version 248399 (0.0006) [2023-12-27 07:22:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 127303680. Throughput: 0: 10375.3, 1: 10307.3. Samples: 127309228. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.970')] [2023-12-27 07:22:17,749][20347] Updated weights for policy 0, policy_version 248777 (0.0008) [2023-12-27 07:22:17,802][20347] Updated weights for policy 0, policy_version 248787 (0.0008) [2023-12-27 07:22:17,852][20347] Updated weights for policy 0, policy_version 248797 (0.0007) [2023-12-27 07:22:17,909][20347] Updated weights for policy 0, policy_version 248807 (0.0006) [2023-12-27 07:22:18,020][20381] Updated weights for policy 1, policy_version 248409 (0.0008) [2023-12-27 07:22:18,078][20381] Updated weights for policy 1, policy_version 248419 (0.0006) [2023-12-27 07:22:18,133][20381] Updated weights for policy 1, policy_version 248429 (0.0005) [2023-12-27 07:22:18,617][20347] Updated weights for policy 0, policy_version 248817 (0.0007) [2023-12-27 07:22:18,671][20347] Updated weights for policy 0, policy_version 248827 (0.0007) [2023-12-27 07:22:18,731][20347] Updated weights for policy 0, policy_version 248837 (0.0006) [2023-12-27 07:22:18,747][20381] Updated weights for policy 1, policy_version 248439 (0.0007) [2023-12-27 07:22:18,807][20381] Updated weights for policy 1, policy_version 248449 (0.0008) [2023-12-27 07:22:18,867][20381] Updated weights for policy 1, policy_version 248459 (0.0008) [2023-12-27 07:22:19,431][20347] Updated weights for policy 0, policy_version 248847 (0.0009) [2023-12-27 07:22:19,490][20347] Updated weights for policy 0, policy_version 248857 (0.0010) [2023-12-27 07:22:19,518][20381] Updated weights for policy 1, policy_version 248469 (0.0007) [2023-12-27 07:22:19,549][20347] Updated weights for policy 0, policy_version 248867 (0.0010) [2023-12-27 07:22:19,580][20381] Updated weights for policy 1, policy_version 248479 (0.0005) [2023-12-27 07:22:19,642][20381] Updated weights for policy 1, policy_version 248489 (0.0008) [2023-12-27 07:22:20,267][20347] Updated weights for policy 0, policy_version 248877 (0.0009) [2023-12-27 07:22:20,327][20381] Updated weights for policy 1, policy_version 248499 (0.0008) [2023-12-27 07:22:20,327][20347] Updated weights for policy 0, policy_version 248887 (0.0008) [2023-12-27 07:22:20,387][20381] Updated weights for policy 1, policy_version 248509 (0.0006) [2023-12-27 07:22:20,392][20347] Updated weights for policy 0, policy_version 248897 (0.0008) [2023-12-27 07:22:20,443][20381] Updated weights for policy 1, policy_version 248519 (0.0006) [2023-12-27 07:22:21,095][20381] Updated weights for policy 1, policy_version 248529 (0.0005) [2023-12-27 07:22:21,100][20347] Updated weights for policy 0, policy_version 248907 (0.0007) [2023-12-27 07:22:21,162][20381] Updated weights for policy 1, policy_version 248539 (0.0008) [2023-12-27 07:22:21,164][20347] Updated weights for policy 0, policy_version 248917 (0.0007) [2023-12-27 07:22:21,220][20381] Updated weights for policy 1, policy_version 248549 (0.0007) [2023-12-27 07:22:21,226][20347] Updated weights for policy 0, policy_version 248927 (0.0005) [2023-12-27 07:22:21,247][20291] KL-divergence is very high: 191.5179 [2023-12-27 07:22:21,260][20291] KL-divergence is very high: 145.2384 [2023-12-27 07:22:21,281][20381] Updated weights for policy 1, policy_version 248559 (0.0008) [2023-12-27 07:22:21,860][20291] KL-divergence is very high: 116.6128 [2023-12-27 07:22:21,867][20291] KL-divergence is very high: 165.8624 [2023-12-27 07:22:21,868][20347] Updated weights for policy 0, policy_version 248937 (0.0006) [2023-12-27 07:22:21,885][20291] KL-divergence is very high: 303.0996 [2023-12-27 07:22:21,897][20291] KL-divergence is very high: 255.6578 [2023-12-27 07:22:21,905][20291] KL-divergence is very high: 318.5894 [2023-12-27 07:22:21,923][20291] KL-divergence is very high: 123.2780 [2023-12-27 07:22:21,927][20347] Updated weights for policy 0, policy_version 248947 (0.0007) [2023-12-27 07:22:21,929][20291] KL-divergence is very high: 150.8266 [2023-12-27 07:22:21,935][20291] KL-divergence is very high: 136.6448 [2023-12-27 07:22:21,955][20291] KL-divergence is very high: 165.2706 [2023-12-27 07:22:21,985][20291] KL-divergence is very high: 122.0949 [2023-12-27 07:22:21,990][20347] Updated weights for policy 0, policy_version 248957 (0.0008) [2023-12-27 07:22:22,012][20291] KL-divergence is very high: 128.5474 [2023-12-27 07:22:22,023][20381] Updated weights for policy 1, policy_version 248569 (0.0010) [2023-12-27 07:22:22,026][20291] KL-divergence is very high: 270.9778 [2023-12-27 07:22:22,032][20291] KL-divergence is very high: 281.8870 [2023-12-27 07:22:22,037][20291] KL-divergence is very high: 228.6089 [2023-12-27 07:22:22,043][20291] KL-divergence is very high: 131.3352 [2023-12-27 07:22:22,052][20347] Updated weights for policy 0, policy_version 248967 (0.0007) [2023-12-27 07:22:22,053][20291] KL-divergence is very high: 176.2853 [2023-12-27 07:22:22,080][20381] Updated weights for policy 1, policy_version 248579 (0.0010) [2023-12-27 07:22:22,139][20381] Updated weights for policy 1, policy_version 248589 (0.0010) [2023-12-27 07:22:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 127401984. Throughput: 0: 10359.1, 1: 10272.5. Samples: 127370804. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:22,363][19170] Avg episode reward: [(0, '979.090'), (1, '952.590')] [2023-12-27 07:22:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000248592_63651840.pth... [2023-12-27 07:22:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000248968_63750144.pth... [2023-12-27 07:22:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000247784_63447040.pth [2023-12-27 07:22:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000247408_63348736.pth [2023-12-27 07:22:22,678][20291] KL-divergence is very high: 181.8922 [2023-12-27 07:22:22,723][20347] Updated weights for policy 0, policy_version 248977 (0.0005) [2023-12-27 07:22:22,723][20291] KL-divergence is very high: 299.9112 [2023-12-27 07:22:22,769][20291] KL-divergence is very high: 159.0384 [2023-12-27 07:22:22,780][20347] Updated weights for policy 0, policy_version 248987 (0.0005) [2023-12-27 07:22:22,818][20291] KL-divergence is very high: 203.8573 [2023-12-27 07:22:22,842][20347] Updated weights for policy 0, policy_version 248997 (0.0008) [2023-12-27 07:22:22,875][20381] Updated weights for policy 1, policy_version 248599 (0.0008) [2023-12-27 07:22:22,932][20381] Updated weights for policy 1, policy_version 248609 (0.0007) [2023-12-27 07:22:22,995][20381] Updated weights for policy 1, policy_version 248619 (0.0007) [2023-12-27 07:22:23,475][20347] Updated weights for policy 0, policy_version 249007 (0.0007) [2023-12-27 07:22:23,498][20291] KL-divergence is very high: 120.5813 [2023-12-27 07:22:23,534][20347] Updated weights for policy 0, policy_version 249017 (0.0005) [2023-12-27 07:22:23,593][20347] Updated weights for policy 0, policy_version 249027 (0.0005) [2023-12-27 07:22:23,718][20381] Updated weights for policy 1, policy_version 248629 (0.0009) [2023-12-27 07:22:23,771][20381] Updated weights for policy 1, policy_version 248639 (0.0009) [2023-12-27 07:22:23,817][20381] Updated weights for policy 1, policy_version 248649 (0.0010) [2023-12-27 07:22:24,137][20347] Updated weights for policy 0, policy_version 249037 (0.0008) [2023-12-27 07:22:24,202][20347] Updated weights for policy 0, policy_version 249047 (0.0010) [2023-12-27 07:22:24,255][20347] Updated weights for policy 0, policy_version 249057 (0.0010) [2023-12-27 07:22:24,485][20381] Updated weights for policy 1, policy_version 248659 (0.0008) [2023-12-27 07:22:24,537][20381] Updated weights for policy 1, policy_version 248669 (0.0008) [2023-12-27 07:22:24,585][20381] Updated weights for policy 1, policy_version 248679 (0.0007) [2023-12-27 07:22:24,906][20347] Updated weights for policy 0, policy_version 249067 (0.0009) [2023-12-27 07:22:24,958][20347] Updated weights for policy 0, policy_version 249077 (0.0007) [2023-12-27 07:22:25,012][20347] Updated weights for policy 0, policy_version 249087 (0.0007) [2023-12-27 07:22:25,230][20381] Updated weights for policy 1, policy_version 248689 (0.0007) [2023-12-27 07:22:25,282][20381] Updated weights for policy 1, policy_version 248699 (0.0008) [2023-12-27 07:22:25,332][20381] Updated weights for policy 1, policy_version 248709 (0.0010) [2023-12-27 07:22:25,360][20292] KL-divergence is very high: 546.3312 [2023-12-27 07:22:25,377][20381] Updated weights for policy 1, policy_version 248719 (0.0010) [2023-12-27 07:22:25,660][20347] Updated weights for policy 0, policy_version 249097 (0.0006) [2023-12-27 07:22:25,714][20347] Updated weights for policy 0, policy_version 249107 (0.0007) [2023-12-27 07:22:25,769][20347] Updated weights for policy 0, policy_version 249117 (0.0007) [2023-12-27 07:22:25,824][20347] Updated weights for policy 0, policy_version 249127 (0.0007) [2023-12-27 07:22:26,097][20381] Updated weights for policy 1, policy_version 248729 (0.0007) [2023-12-27 07:22:26,150][20381] Updated weights for policy 1, policy_version 248739 (0.0005) [2023-12-27 07:22:26,203][20381] Updated weights for policy 1, policy_version 248749 (0.0006) [2023-12-27 07:22:26,518][20347] Updated weights for policy 0, policy_version 249137 (0.0007) [2023-12-27 07:22:26,579][20347] Updated weights for policy 0, policy_version 249147 (0.0007) [2023-12-27 07:22:26,641][20347] Updated weights for policy 0, policy_version 249157 (0.0007) [2023-12-27 07:22:26,798][20381] Updated weights for policy 1, policy_version 248759 (0.0007) [2023-12-27 07:22:26,854][20381] Updated weights for policy 1, policy_version 248769 (0.0008) [2023-12-27 07:22:26,899][20381] Updated weights for policy 1, policy_version 248779 (0.0008) [2023-12-27 07:22:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 127500288. Throughput: 0: 10399.3, 1: 10210.5. Samples: 127493420. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:27,363][19170] Avg episode reward: [(0, '975.910'), (1, '894.930')] [2023-12-27 07:22:27,369][20347] Updated weights for policy 0, policy_version 249167 (0.0008) [2023-12-27 07:22:27,430][20347] Updated weights for policy 0, policy_version 249177 (0.0007) [2023-12-27 07:22:27,466][20381] Updated weights for policy 1, policy_version 248789 (0.0006) [2023-12-27 07:22:27,484][20292] KL-divergence is very high: 277.3575 [2023-12-27 07:22:27,486][20347] Updated weights for policy 0, policy_version 249187 (0.0007) [2023-12-27 07:22:27,491][20292] KL-divergence is very high: 186.9604 [2023-12-27 07:22:27,528][20381] Updated weights for policy 1, policy_version 248799 (0.0005) [2023-12-27 07:22:27,580][20292] KL-divergence is very high: 117.9358 [2023-12-27 07:22:27,585][20381] Updated weights for policy 1, policy_version 248809 (0.0005) [2023-12-27 07:22:28,160][20347] Updated weights for policy 0, policy_version 249197 (0.0006) [2023-12-27 07:22:28,194][20381] Updated weights for policy 1, policy_version 248819 (0.0005) [2023-12-27 07:22:28,211][20347] Updated weights for policy 0, policy_version 249207 (0.0007) [2023-12-27 07:22:28,251][20381] Updated weights for policy 1, policy_version 248829 (0.0006) [2023-12-27 07:22:28,253][20347] Updated weights for policy 0, policy_version 249217 (0.0005) [2023-12-27 07:22:28,304][20381] Updated weights for policy 1, policy_version 248839 (0.0007) [2023-12-27 07:22:28,984][20381] Updated weights for policy 1, policy_version 248849 (0.0008) [2023-12-27 07:22:28,992][20347] Updated weights for policy 0, policy_version 249227 (0.0007) [2023-12-27 07:22:29,029][20291] KL-divergence is very high: 164.4324 [2023-12-27 07:22:29,030][20381] Updated weights for policy 1, policy_version 248859 (0.0007) [2023-12-27 07:22:29,043][20347] Updated weights for policy 0, policy_version 249237 (0.0007) [2023-12-27 07:22:29,083][20381] Updated weights for policy 1, policy_version 248869 (0.0007) [2023-12-27 07:22:29,091][20347] Updated weights for policy 0, policy_version 249247 (0.0005) [2023-12-27 07:22:29,138][20381] Updated weights for policy 1, policy_version 248879 (0.0007) [2023-12-27 07:22:29,800][20347] Updated weights for policy 0, policy_version 249257 (0.0005) [2023-12-27 07:22:29,807][20381] Updated weights for policy 1, policy_version 248889 (0.0007) [2023-12-27 07:22:29,859][20347] Updated weights for policy 0, policy_version 249267 (0.0009) [2023-12-27 07:22:29,869][20381] Updated weights for policy 1, policy_version 248899 (0.0007) [2023-12-27 07:22:29,915][20347] Updated weights for policy 0, policy_version 249277 (0.0006) [2023-12-27 07:22:29,928][20381] Updated weights for policy 1, policy_version 248909 (0.0006) [2023-12-27 07:22:29,978][20347] Updated weights for policy 0, policy_version 249287 (0.0008) [2023-12-27 07:22:30,535][20381] Updated weights for policy 1, policy_version 248919 (0.0009) [2023-12-27 07:22:30,591][20381] Updated weights for policy 1, policy_version 248929 (0.0008) [2023-12-27 07:22:30,609][20347] Updated weights for policy 0, policy_version 249297 (0.0006) [2023-12-27 07:22:30,651][20381] Updated weights for policy 1, policy_version 248939 (0.0007) [2023-12-27 07:22:30,665][20347] Updated weights for policy 0, policy_version 249307 (0.0005) [2023-12-27 07:22:30,718][20347] Updated weights for policy 0, policy_version 249317 (0.0007) [2023-12-27 07:22:31,300][20381] Updated weights for policy 1, policy_version 248949 (0.0009) [2023-12-27 07:22:31,361][20347] Updated weights for policy 0, policy_version 249327 (0.0007) [2023-12-27 07:22:31,364][20381] Updated weights for policy 1, policy_version 248959 (0.0010) [2023-12-27 07:22:31,422][20347] Updated weights for policy 0, policy_version 249337 (0.0006) [2023-12-27 07:22:31,424][20381] Updated weights for policy 1, policy_version 248969 (0.0008) [2023-12-27 07:22:31,478][20347] Updated weights for policy 0, policy_version 249347 (0.0007) [2023-12-27 07:22:32,054][20381] Updated weights for policy 1, policy_version 248979 (0.0008) [2023-12-27 07:22:32,112][20381] Updated weights for policy 1, policy_version 248989 (0.0009) [2023-12-27 07:22:32,175][20381] Updated weights for policy 1, policy_version 248999 (0.0008) [2023-12-27 07:22:32,196][20347] Updated weights for policy 0, policy_version 249357 (0.0006) [2023-12-27 07:22:32,246][20347] Updated weights for policy 0, policy_version 249367 (0.0008) [2023-12-27 07:22:32,303][20347] Updated weights for policy 0, policy_version 249377 (0.0008) [2023-12-27 07:22:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 127614976. Throughput: 0: 10319.4, 1: 10262.3. Samples: 127618572. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:32,363][19170] Avg episode reward: [(0, '962.160'), (1, '884.130')] [2023-12-27 07:22:32,833][20381] Updated weights for policy 1, policy_version 249009 (0.0008) [2023-12-27 07:22:32,889][20381] Updated weights for policy 1, policy_version 249019 (0.0011) [2023-12-27 07:22:32,943][20381] Updated weights for policy 1, policy_version 249029 (0.0010) [2023-12-27 07:22:32,994][20381] Updated weights for policy 1, policy_version 249039 (0.0010) [2023-12-27 07:22:33,021][20347] Updated weights for policy 0, policy_version 249388 (0.0008) [2023-12-27 07:22:33,080][20347] Updated weights for policy 0, policy_version 249398 (0.0008) [2023-12-27 07:22:33,139][20347] Updated weights for policy 0, policy_version 249408 (0.0007) [2023-12-27 07:22:33,673][20381] Updated weights for policy 1, policy_version 249049 (0.0006) [2023-12-27 07:22:33,727][20381] Updated weights for policy 1, policy_version 249059 (0.0005) [2023-12-27 07:22:33,771][20381] Updated weights for policy 1, policy_version 249069 (0.0007) [2023-12-27 07:22:33,837][20347] Updated weights for policy 0, policy_version 249418 (0.0007) [2023-12-27 07:22:33,890][20347] Updated weights for policy 0, policy_version 249428 (0.0007) [2023-12-27 07:22:33,943][20347] Updated weights for policy 0, policy_version 249438 (0.0006) [2023-12-27 07:22:33,993][20347] Updated weights for policy 0, policy_version 249448 (0.0005) [2023-12-27 07:22:34,432][20381] Updated weights for policy 1, policy_version 249079 (0.0008) [2023-12-27 07:22:34,494][20381] Updated weights for policy 1, policy_version 249089 (0.0007) [2023-12-27 07:22:34,556][20381] Updated weights for policy 1, policy_version 249099 (0.0008) [2023-12-27 07:22:34,638][20347] Updated weights for policy 0, policy_version 249458 (0.0007) [2023-12-27 07:22:34,693][20347] Updated weights for policy 0, policy_version 249468 (0.0007) [2023-12-27 07:22:34,747][20347] Updated weights for policy 0, policy_version 249478 (0.0007) [2023-12-27 07:22:35,207][20381] Updated weights for policy 1, policy_version 249109 (0.0008) [2023-12-27 07:22:35,262][20381] Updated weights for policy 1, policy_version 249119 (0.0006) [2023-12-27 07:22:35,321][20381] Updated weights for policy 1, policy_version 249129 (0.0006) [2023-12-27 07:22:35,370][20347] Updated weights for policy 0, policy_version 249488 (0.0005) [2023-12-27 07:22:35,416][20347] Updated weights for policy 0, policy_version 249498 (0.0005) [2023-12-27 07:22:35,462][20347] Updated weights for policy 0, policy_version 249508 (0.0005) [2023-12-27 07:22:35,843][20381] Updated weights for policy 1, policy_version 249139 (0.0005) [2023-12-27 07:22:35,903][20381] Updated weights for policy 1, policy_version 249149 (0.0007) [2023-12-27 07:22:35,959][20381] Updated weights for policy 1, policy_version 249159 (0.0008) [2023-12-27 07:22:36,047][20347] Updated weights for policy 0, policy_version 249518 (0.0008) [2023-12-27 07:22:36,105][20347] Updated weights for policy 0, policy_version 249528 (0.0010) [2023-12-27 07:22:36,163][20347] Updated weights for policy 0, policy_version 249538 (0.0009) [2023-12-27 07:22:36,623][20381] Updated weights for policy 1, policy_version 249169 (0.0007) [2023-12-27 07:22:36,687][20381] Updated weights for policy 1, policy_version 249179 (0.0006) [2023-12-27 07:22:36,749][20381] Updated weights for policy 1, policy_version 249189 (0.0008) [2023-12-27 07:22:36,811][20347] Updated weights for policy 0, policy_version 249548 (0.0008) [2023-12-27 07:22:36,816][20381] Updated weights for policy 1, policy_version 249199 (0.0008) [2023-12-27 07:22:36,870][20347] Updated weights for policy 0, policy_version 249558 (0.0008) [2023-12-27 07:22:36,931][20347] Updated weights for policy 0, policy_version 249568 (0.0008) [2023-12-27 07:22:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 127713280. Throughput: 0: 10290.0, 1: 10296.5. Samples: 127680848. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:37,363][19170] Avg episode reward: [(0, '311.780'), (1, '846.820')] [2023-12-27 07:22:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000249200_63807488.pth... [2023-12-27 07:22:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000249576_63905792.pth... [2023-12-27 07:22:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000247984_63496192.pth [2023-12-27 07:22:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000248392_63602688.pth [2023-12-27 07:22:37,519][20381] Updated weights for policy 1, policy_version 249209 (0.0008) [2023-12-27 07:22:37,520][20347] Updated weights for policy 0, policy_version 249578 (0.0008) [2023-12-27 07:22:37,570][20381] Updated weights for policy 1, policy_version 249219 (0.0008) [2023-12-27 07:22:37,575][20347] Updated weights for policy 0, policy_version 249588 (0.0006) [2023-12-27 07:22:37,630][20381] Updated weights for policy 1, policy_version 249229 (0.0007) [2023-12-27 07:22:37,634][20347] Updated weights for policy 0, policy_version 249598 (0.0005) [2023-12-27 07:22:37,686][20347] Updated weights for policy 0, policy_version 249608 (0.0009) [2023-12-27 07:22:38,218][20291] KL-divergence is very high: 118.5139 [2023-12-27 07:22:38,276][20347] Updated weights for policy 0, policy_version 249618 (0.0009) [2023-12-27 07:22:38,331][20347] Updated weights for policy 0, policy_version 249628 (0.0010) [2023-12-27 07:22:38,350][20381] Updated weights for policy 1, policy_version 249239 (0.0008) [2023-12-27 07:22:38,389][20347] Updated weights for policy 0, policy_version 249638 (0.0010) [2023-12-27 07:22:38,412][20381] Updated weights for policy 1, policy_version 249249 (0.0007) [2023-12-27 07:22:38,467][20381] Updated weights for policy 1, policy_version 249259 (0.0007) [2023-12-27 07:22:39,026][20347] Updated weights for policy 0, policy_version 249648 (0.0009) [2023-12-27 07:22:39,085][20347] Updated weights for policy 0, policy_version 249658 (0.0006) [2023-12-27 07:22:39,144][20347] Updated weights for policy 0, policy_version 249668 (0.0006) [2023-12-27 07:22:39,168][20381] Updated weights for policy 1, policy_version 249269 (0.0007) [2023-12-27 07:22:39,223][20381] Updated weights for policy 1, policy_version 249279 (0.0008) [2023-12-27 07:22:39,277][20381] Updated weights for policy 1, policy_version 249289 (0.0008) [2023-12-27 07:22:39,815][20347] Updated weights for policy 0, policy_version 249678 (0.0007) [2023-12-27 07:22:39,879][20347] Updated weights for policy 0, policy_version 249688 (0.0008) [2023-12-27 07:22:39,945][20347] Updated weights for policy 0, policy_version 249698 (0.0007) [2023-12-27 07:22:40,016][20381] Updated weights for policy 1, policy_version 249299 (0.0008) [2023-12-27 07:22:40,076][20381] Updated weights for policy 1, policy_version 249309 (0.0008) [2023-12-27 07:22:40,139][20381] Updated weights for policy 1, policy_version 249319 (0.0007) [2023-12-27 07:22:40,149][20292] KL-divergence is very high: 193.1508 [2023-12-27 07:22:40,155][20292] KL-divergence is very high: 113.6695 [2023-12-27 07:22:40,579][20347] Updated weights for policy 0, policy_version 249708 (0.0008) [2023-12-27 07:22:40,635][20347] Updated weights for policy 0, policy_version 249718 (0.0007) [2023-12-27 07:22:40,684][20347] Updated weights for policy 0, policy_version 249728 (0.0007) [2023-12-27 07:22:40,717][20291] KL-divergence is very high: 107.0137 [2023-12-27 07:22:40,833][20381] Updated weights for policy 1, policy_version 249329 (0.0006) [2023-12-27 07:22:40,892][20381] Updated weights for policy 1, policy_version 249339 (0.0007) [2023-12-27 07:22:40,953][20381] Updated weights for policy 1, policy_version 249349 (0.0008) [2023-12-27 07:22:41,013][20381] Updated weights for policy 1, policy_version 249359 (0.0008) [2023-12-27 07:22:41,424][20347] Updated weights for policy 0, policy_version 249738 (0.0007) [2023-12-27 07:22:41,480][20347] Updated weights for policy 0, policy_version 249748 (0.0007) [2023-12-27 07:22:41,487][20291] KL-divergence is very high: 140.0970 [2023-12-27 07:22:41,545][20347] Updated weights for policy 0, policy_version 249758 (0.0006) [2023-12-27 07:22:41,595][20347] Updated weights for policy 0, policy_version 249768 (0.0005) [2023-12-27 07:22:41,768][20381] Updated weights for policy 1, policy_version 249369 (0.0008) [2023-12-27 07:22:41,826][20381] Updated weights for policy 1, policy_version 249379 (0.0007) [2023-12-27 07:22:41,879][20381] Updated weights for policy 1, policy_version 249389 (0.0005) [2023-12-27 07:22:42,228][20347] Updated weights for policy 0, policy_version 249778 (0.0006) [2023-12-27 07:22:42,299][20347] Updated weights for policy 0, policy_version 249788 (0.0008) [2023-12-27 07:22:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 127811584. Throughput: 0: 10378.6, 1: 10289.5. Samples: 127806556. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:42,363][19170] Avg episode reward: [(0, '429.330'), (1, '836.860')] [2023-12-27 07:22:42,366][20347] Updated weights for policy 0, policy_version 249798 (0.0007) [2023-12-27 07:22:42,585][20381] Updated weights for policy 1, policy_version 249399 (0.0008) [2023-12-27 07:22:42,638][20381] Updated weights for policy 1, policy_version 249409 (0.0009) [2023-12-27 07:22:42,682][20292] KL-divergence is very high: 120.1492 [2023-12-27 07:22:42,688][20381] Updated weights for policy 1, policy_version 249419 (0.0008) [2023-12-27 07:22:42,961][20347] Updated weights for policy 0, policy_version 249808 (0.0008) [2023-12-27 07:22:43,017][20347] Updated weights for policy 0, policy_version 249818 (0.0007) [2023-12-27 07:22:43,079][20347] Updated weights for policy 0, policy_version 249828 (0.0005) [2023-12-27 07:22:43,436][20381] Updated weights for policy 1, policy_version 249429 (0.0008) [2023-12-27 07:22:43,492][20381] Updated weights for policy 1, policy_version 249439 (0.0008) [2023-12-27 07:22:43,547][20381] Updated weights for policy 1, policy_version 249449 (0.0008) [2023-12-27 07:22:43,662][20347] Updated weights for policy 0, policy_version 249838 (0.0005) [2023-12-27 07:22:43,714][20347] Updated weights for policy 0, policy_version 249848 (0.0005) [2023-12-27 07:22:43,773][20347] Updated weights for policy 0, policy_version 249858 (0.0005) [2023-12-27 07:22:44,176][20381] Updated weights for policy 1, policy_version 249459 (0.0009) [2023-12-27 07:22:44,232][20381] Updated weights for policy 1, policy_version 249469 (0.0010) [2023-12-27 07:22:44,291][20381] Updated weights for policy 1, policy_version 249479 (0.0010) [2023-12-27 07:22:44,388][20347] Updated weights for policy 0, policy_version 249868 (0.0005) [2023-12-27 07:22:44,452][20347] Updated weights for policy 0, policy_version 249878 (0.0005) [2023-12-27 07:22:44,499][20347] Updated weights for policy 0, policy_version 249888 (0.0007) [2023-12-27 07:22:44,969][20381] Updated weights for policy 1, policy_version 249489 (0.0010) [2023-12-27 07:22:45,029][20381] Updated weights for policy 1, policy_version 249499 (0.0008) [2023-12-27 07:22:45,087][20381] Updated weights for policy 1, policy_version 249509 (0.0009) [2023-12-27 07:22:45,140][20381] Updated weights for policy 1, policy_version 249519 (0.0008) [2023-12-27 07:22:45,155][20347] Updated weights for policy 0, policy_version 249898 (0.0007) [2023-12-27 07:22:45,202][20347] Updated weights for policy 0, policy_version 249908 (0.0008) [2023-12-27 07:22:45,251][20347] Updated weights for policy 0, policy_version 249918 (0.0007) [2023-12-27 07:22:45,301][20347] Updated weights for policy 0, policy_version 249928 (0.0007) [2023-12-27 07:22:45,799][20381] Updated weights for policy 1, policy_version 249529 (0.0008) [2023-12-27 07:22:45,859][20381] Updated weights for policy 1, policy_version 249539 (0.0008) [2023-12-27 07:22:45,918][20381] Updated weights for policy 1, policy_version 249549 (0.0008) [2023-12-27 07:22:46,003][20347] Updated weights for policy 0, policy_version 249938 (0.0007) [2023-12-27 07:22:46,060][20347] Updated weights for policy 0, policy_version 249948 (0.0007) [2023-12-27 07:22:46,120][20347] Updated weights for policy 0, policy_version 249958 (0.0007) [2023-12-27 07:22:46,615][20381] Updated weights for policy 1, policy_version 249559 (0.0007) [2023-12-27 07:22:46,670][20381] Updated weights for policy 1, policy_version 249569 (0.0008) [2023-12-27 07:22:46,719][20381] Updated weights for policy 1, policy_version 249579 (0.0008) [2023-12-27 07:22:46,756][20347] Updated weights for policy 0, policy_version 249968 (0.0007) [2023-12-27 07:22:46,802][20347] Updated weights for policy 0, policy_version 249978 (0.0008) [2023-12-27 07:22:46,857][20347] Updated weights for policy 0, policy_version 249988 (0.0007) [2023-12-27 07:22:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 127918080. Throughput: 0: 10437.9, 1: 10257.4. Samples: 127931044. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:47,363][19170] Avg episode reward: [(0, '314.520'), (1, '903.900')] [2023-12-27 07:22:47,378][20381] Updated weights for policy 1, policy_version 249589 (0.0008) [2023-12-27 07:22:47,436][20381] Updated weights for policy 1, policy_version 249599 (0.0009) [2023-12-27 07:22:47,488][20381] Updated weights for policy 1, policy_version 249609 (0.0008) [2023-12-27 07:22:47,553][20347] Updated weights for policy 0, policy_version 249998 (0.0008) [2023-12-27 07:22:47,612][20347] Updated weights for policy 0, policy_version 250008 (0.0008) [2023-12-27 07:22:47,668][20347] Updated weights for policy 0, policy_version 250018 (0.0007) [2023-12-27 07:22:48,224][20381] Updated weights for policy 1, policy_version 249619 (0.0007) [2023-12-27 07:22:48,264][20347] Updated weights for policy 0, policy_version 250028 (0.0006) [2023-12-27 07:22:48,270][20381] Updated weights for policy 1, policy_version 249629 (0.0006) [2023-12-27 07:22:48,308][20347] Updated weights for policy 0, policy_version 250038 (0.0005) [2023-12-27 07:22:48,317][20381] Updated weights for policy 1, policy_version 249639 (0.0006) [2023-12-27 07:22:48,371][20347] Updated weights for policy 0, policy_version 250048 (0.0008) [2023-12-27 07:22:49,028][20381] Updated weights for policy 1, policy_version 249649 (0.0006) [2023-12-27 07:22:49,080][20381] Updated weights for policy 1, policy_version 249659 (0.0008) [2023-12-27 07:22:49,090][20347] Updated weights for policy 0, policy_version 250058 (0.0007) [2023-12-27 07:22:49,132][20381] Updated weights for policy 1, policy_version 249669 (0.0008) [2023-12-27 07:22:49,145][20347] Updated weights for policy 0, policy_version 250068 (0.0006) [2023-12-27 07:22:49,178][20381] Updated weights for policy 1, policy_version 249679 (0.0006) [2023-12-27 07:22:49,194][20347] Updated weights for policy 0, policy_version 250078 (0.0006) [2023-12-27 07:22:49,263][20347] Updated weights for policy 0, policy_version 250088 (0.0007) [2023-12-27 07:22:49,849][20381] Updated weights for policy 1, policy_version 249689 (0.0008) [2023-12-27 07:22:49,899][20347] Updated weights for policy 0, policy_version 250098 (0.0007) [2023-12-27 07:22:49,909][20381] Updated weights for policy 1, policy_version 249699 (0.0006) [2023-12-27 07:22:49,965][20347] Updated weights for policy 0, policy_version 250108 (0.0007) [2023-12-27 07:22:49,967][20381] Updated weights for policy 1, policy_version 249709 (0.0006) [2023-12-27 07:22:50,029][20347] Updated weights for policy 0, policy_version 250118 (0.0008) [2023-12-27 07:22:50,650][20347] Updated weights for policy 0, policy_version 250128 (0.0009) [2023-12-27 07:22:50,661][20381] Updated weights for policy 1, policy_version 249719 (0.0008) [2023-12-27 07:22:50,704][20347] Updated weights for policy 0, policy_version 250138 (0.0009) [2023-12-27 07:22:50,718][20381] Updated weights for policy 1, policy_version 249729 (0.0006) [2023-12-27 07:22:50,755][20347] Updated weights for policy 0, policy_version 250148 (0.0006) [2023-12-27 07:22:50,769][20381] Updated weights for policy 1, policy_version 249739 (0.0006) [2023-12-27 07:22:51,408][20347] Updated weights for policy 0, policy_version 250158 (0.0007) [2023-12-27 07:22:51,460][20381] Updated weights for policy 1, policy_version 249749 (0.0006) [2023-12-27 07:22:51,466][20347] Updated weights for policy 0, policy_version 250168 (0.0006) [2023-12-27 07:22:51,521][20347] Updated weights for policy 0, policy_version 250178 (0.0005) [2023-12-27 07:22:51,527][20381] Updated weights for policy 1, policy_version 249759 (0.0006) [2023-12-27 07:22:51,587][20381] Updated weights for policy 1, policy_version 249769 (0.0007) [2023-12-27 07:22:52,209][20381] Updated weights for policy 1, policy_version 249779 (0.0008) [2023-12-27 07:22:52,262][20381] Updated weights for policy 1, policy_version 249789 (0.0007) [2023-12-27 07:22:52,287][20347] Updated weights for policy 0, policy_version 250188 (0.0007) [2023-12-27 07:22:52,316][20381] Updated weights for policy 1, policy_version 249799 (0.0006) [2023-12-27 07:22:52,345][20347] Updated weights for policy 0, policy_version 250198 (0.0006) [2023-12-27 07:22:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 128016384. Throughput: 0: 10435.3, 1: 10268.5. Samples: 127993120. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:52,363][19170] Avg episode reward: [(0, '289.950'), (1, '970.980')] [2023-12-27 07:22:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000249808_63963136.pth... [2023-12-27 07:22:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000248592_63651840.pth [2023-12-27 07:22:52,406][20347] Updated weights for policy 0, policy_version 250208 (0.0008) [2023-12-27 07:22:52,449][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000250216_64069632.pth... [2023-12-27 07:22:52,454][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000248968_63750144.pth [2023-12-27 07:22:53,008][20381] Updated weights for policy 1, policy_version 249809 (0.0008) [2023-12-27 07:22:53,059][20381] Updated weights for policy 1, policy_version 249819 (0.0008) [2023-12-27 07:22:53,104][20347] Updated weights for policy 0, policy_version 250218 (0.0007) [2023-12-27 07:22:53,109][20381] Updated weights for policy 1, policy_version 249829 (0.0008) [2023-12-27 07:22:53,149][20347] Updated weights for policy 0, policy_version 250228 (0.0006) [2023-12-27 07:22:53,164][20381] Updated weights for policy 1, policy_version 249839 (0.0007) [2023-12-27 07:22:53,203][20347] Updated weights for policy 0, policy_version 250238 (0.0009) [2023-12-27 07:22:53,259][20347] Updated weights for policy 0, policy_version 250248 (0.0007) [2023-12-27 07:22:53,726][20381] Updated weights for policy 1, policy_version 249849 (0.0005) [2023-12-27 07:22:53,787][20381] Updated weights for policy 1, policy_version 249859 (0.0008) [2023-12-27 07:22:53,843][20381] Updated weights for policy 1, policy_version 249869 (0.0008) [2023-12-27 07:22:53,937][20347] Updated weights for policy 0, policy_version 250258 (0.0006) [2023-12-27 07:22:53,991][20347] Updated weights for policy 0, policy_version 250268 (0.0008) [2023-12-27 07:22:54,048][20347] Updated weights for policy 0, policy_version 250278 (0.0007) [2023-12-27 07:22:54,486][20381] Updated weights for policy 1, policy_version 249879 (0.0008) [2023-12-27 07:22:54,535][20381] Updated weights for policy 1, policy_version 249889 (0.0008) [2023-12-27 07:22:54,588][20381] Updated weights for policy 1, policy_version 249899 (0.0008) [2023-12-27 07:22:54,695][20347] Updated weights for policy 0, policy_version 250288 (0.0008) [2023-12-27 07:22:54,751][20347] Updated weights for policy 0, policy_version 250298 (0.0008) [2023-12-27 07:22:54,806][20347] Updated weights for policy 0, policy_version 250308 (0.0008) [2023-12-27 07:22:55,259][20381] Updated weights for policy 1, policy_version 249909 (0.0008) [2023-12-27 07:22:55,320][20381] Updated weights for policy 1, policy_version 249919 (0.0008) [2023-12-27 07:22:55,384][20381] Updated weights for policy 1, policy_version 249929 (0.0008) [2023-12-27 07:22:55,509][20347] Updated weights for policy 0, policy_version 250318 (0.0008) [2023-12-27 07:22:55,550][20347] Updated weights for policy 0, policy_version 250328 (0.0006) [2023-12-27 07:22:55,595][20347] Updated weights for policy 0, policy_version 250338 (0.0005) [2023-12-27 07:22:56,012][20381] Updated weights for policy 1, policy_version 249939 (0.0006) [2023-12-27 07:22:56,069][20381] Updated weights for policy 1, policy_version 249949 (0.0006) [2023-12-27 07:22:56,127][20381] Updated weights for policy 1, policy_version 249959 (0.0008) [2023-12-27 07:22:56,303][20347] Updated weights for policy 0, policy_version 250348 (0.0006) [2023-12-27 07:22:56,351][20347] Updated weights for policy 0, policy_version 250358 (0.0007) [2023-12-27 07:22:56,405][20347] Updated weights for policy 0, policy_version 250368 (0.0007) [2023-12-27 07:22:56,727][20381] Updated weights for policy 1, policy_version 249969 (0.0008) [2023-12-27 07:22:56,784][20381] Updated weights for policy 1, policy_version 249979 (0.0008) [2023-12-27 07:22:56,844][20381] Updated weights for policy 1, policy_version 249989 (0.0005) [2023-12-27 07:22:56,905][20381] Updated weights for policy 1, policy_version 249999 (0.0005) [2023-12-27 07:22:57,168][20347] Updated weights for policy 0, policy_version 250378 (0.0008) [2023-12-27 07:22:57,227][20347] Updated weights for policy 0, policy_version 250388 (0.0007) [2023-12-27 07:22:57,284][20347] Updated weights for policy 0, policy_version 250398 (0.0009) [2023-12-27 07:22:57,344][20347] Updated weights for policy 0, policy_version 250408 (0.0008) [2023-12-27 07:22:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 128131072. Throughput: 0: 10409.4, 1: 10329.1. Samples: 128117548. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:22:57,363][19170] Avg episode reward: [(0, '313.630'), (1, '951.900')] [2023-12-27 07:22:57,458][20381] Updated weights for policy 1, policy_version 250009 (0.0005) [2023-12-27 07:22:57,520][20381] Updated weights for policy 1, policy_version 250019 (0.0007) [2023-12-27 07:22:57,583][20381] Updated weights for policy 1, policy_version 250029 (0.0005) [2023-12-27 07:22:57,940][20347] Updated weights for policy 0, policy_version 250418 (0.0005) [2023-12-27 07:22:57,997][20347] Updated weights for policy 0, policy_version 250428 (0.0005) [2023-12-27 07:22:58,047][20347] Updated weights for policy 0, policy_version 250438 (0.0005) [2023-12-27 07:22:58,099][20381] Updated weights for policy 1, policy_version 250039 (0.0006) [2023-12-27 07:22:58,139][20292] KL-divergence is very high: 123.7597 [2023-12-27 07:22:58,149][20381] Updated weights for policy 1, policy_version 250049 (0.0006) [2023-12-27 07:22:58,157][20292] KL-divergence is very high: 195.4089 [2023-12-27 07:22:58,215][20381] Updated weights for policy 1, policy_version 250059 (0.0008) [2023-12-27 07:22:58,724][20347] Updated weights for policy 0, policy_version 250448 (0.0009) [2023-12-27 07:22:58,796][20347] Updated weights for policy 0, policy_version 250458 (0.0009) [2023-12-27 07:22:58,868][20347] Updated weights for policy 0, policy_version 250468 (0.0007) [2023-12-27 07:22:58,971][20381] Updated weights for policy 1, policy_version 250069 (0.0007) [2023-12-27 07:22:59,027][20381] Updated weights for policy 1, policy_version 250079 (0.0008) [2023-12-27 07:22:59,091][20381] Updated weights for policy 1, policy_version 250089 (0.0008) [2023-12-27 07:22:59,551][20347] Updated weights for policy 0, policy_version 250478 (0.0007) [2023-12-27 07:22:59,608][20347] Updated weights for policy 0, policy_version 250488 (0.0010) [2023-12-27 07:22:59,661][20347] Updated weights for policy 0, policy_version 250498 (0.0008) [2023-12-27 07:22:59,800][20381] Updated weights for policy 1, policy_version 250099 (0.0009) [2023-12-27 07:22:59,860][20381] Updated weights for policy 1, policy_version 250109 (0.0008) [2023-12-27 07:22:59,921][20381] Updated weights for policy 1, policy_version 250119 (0.0007) [2023-12-27 07:23:00,369][20347] Updated weights for policy 0, policy_version 250508 (0.0006) [2023-12-27 07:23:00,429][20347] Updated weights for policy 0, policy_version 250518 (0.0007) [2023-12-27 07:23:00,486][20347] Updated weights for policy 0, policy_version 250528 (0.0008) [2023-12-27 07:23:00,515][20381] Updated weights for policy 1, policy_version 250129 (0.0007) [2023-12-27 07:23:00,565][20381] Updated weights for policy 1, policy_version 250139 (0.0008) [2023-12-27 07:23:00,618][20381] Updated weights for policy 1, policy_version 250149 (0.0010) [2023-12-27 07:23:00,668][20381] Updated weights for policy 1, policy_version 250159 (0.0008) [2023-12-27 07:23:01,133][20347] Updated weights for policy 0, policy_version 250538 (0.0005) [2023-12-27 07:23:01,185][20347] Updated weights for policy 0, policy_version 250548 (0.0007) [2023-12-27 07:23:01,231][20347] Updated weights for policy 0, policy_version 250558 (0.0007) [2023-12-27 07:23:01,294][20347] Updated weights for policy 0, policy_version 250568 (0.0008) [2023-12-27 07:23:01,407][20381] Updated weights for policy 1, policy_version 250169 (0.0008) [2023-12-27 07:23:01,465][20381] Updated weights for policy 1, policy_version 250179 (0.0008) [2023-12-27 07:23:01,529][20381] Updated weights for policy 1, policy_version 250189 (0.0008) [2023-12-27 07:23:02,012][20347] Updated weights for policy 0, policy_version 250578 (0.0008) [2023-12-27 07:23:02,070][20347] Updated weights for policy 0, policy_version 250588 (0.0008) [2023-12-27 07:23:02,123][20347] Updated weights for policy 0, policy_version 250598 (0.0007) [2023-12-27 07:23:02,261][20381] Updated weights for policy 1, policy_version 250199 (0.0008) [2023-12-27 07:23:02,317][20381] Updated weights for policy 1, policy_version 250209 (0.0008) [2023-12-27 07:23:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 128229376. Throughput: 0: 10376.4, 1: 10337.5. Samples: 128241356. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:23:02,363][19170] Avg episode reward: [(0, '201.630'), (1, '951.910')] [2023-12-27 07:23:02,382][20381] Updated weights for policy 1, policy_version 250219 (0.0008) [2023-12-27 07:23:02,771][20347] Updated weights for policy 0, policy_version 250608 (0.0007) [2023-12-27 07:23:02,821][20347] Updated weights for policy 0, policy_version 250618 (0.0007) [2023-12-27 07:23:02,865][20347] Updated weights for policy 0, policy_version 250628 (0.0008) [2023-12-27 07:23:02,996][20381] Updated weights for policy 1, policy_version 250229 (0.0006) [2023-12-27 07:23:03,047][20381] Updated weights for policy 1, policy_version 250239 (0.0005) [2023-12-27 07:23:03,099][20381] Updated weights for policy 1, policy_version 250249 (0.0010) [2023-12-27 07:23:03,619][20347] Updated weights for policy 0, policy_version 250638 (0.0007) [2023-12-27 07:23:03,664][20347] Updated weights for policy 0, policy_version 250648 (0.0008) [2023-12-27 07:23:03,686][20381] Updated weights for policy 1, policy_version 250259 (0.0009) [2023-12-27 07:23:03,719][20347] Updated weights for policy 0, policy_version 250658 (0.0008) [2023-12-27 07:23:03,743][20381] Updated weights for policy 1, policy_version 250269 (0.0007) [2023-12-27 07:23:03,794][20381] Updated weights for policy 1, policy_version 250279 (0.0010) [2023-12-27 07:23:04,418][20347] Updated weights for policy 0, policy_version 250668 (0.0006) [2023-12-27 07:23:04,472][20347] Updated weights for policy 0, policy_version 250678 (0.0008) [2023-12-27 07:23:04,502][20381] Updated weights for policy 1, policy_version 250289 (0.0010) [2023-12-27 07:23:04,517][20347] Updated weights for policy 0, policy_version 250688 (0.0007) [2023-12-27 07:23:04,560][20381] Updated weights for policy 1, policy_version 250299 (0.0007) [2023-12-27 07:23:04,644][20381] Updated weights for policy 1, policy_version 250309 (0.0007) [2023-12-27 07:23:04,703][20381] Updated weights for policy 1, policy_version 250319 (0.0008) [2023-12-27 07:23:05,225][20381] Updated weights for policy 1, policy_version 250329 (0.0006) [2023-12-27 07:23:05,233][20347] Updated weights for policy 0, policy_version 250698 (0.0008) [2023-12-27 07:23:05,277][20381] Updated weights for policy 1, policy_version 250339 (0.0006) [2023-12-27 07:23:05,282][20347] Updated weights for policy 0, policy_version 250708 (0.0010) [2023-12-27 07:23:05,331][20381] Updated weights for policy 1, policy_version 250349 (0.0005) [2023-12-27 07:23:05,332][20347] Updated weights for policy 0, policy_version 250718 (0.0010) [2023-12-27 07:23:05,384][20347] Updated weights for policy 0, policy_version 250728 (0.0008) [2023-12-27 07:23:05,877][20381] Updated weights for policy 1, policy_version 250359 (0.0006) [2023-12-27 07:23:05,943][20381] Updated weights for policy 1, policy_version 250369 (0.0006) [2023-12-27 07:23:06,002][20381] Updated weights for policy 1, policy_version 250379 (0.0007) [2023-12-27 07:23:06,036][20347] Updated weights for policy 0, policy_version 250738 (0.0005) [2023-12-27 07:23:06,088][20347] Updated weights for policy 0, policy_version 250748 (0.0005) [2023-12-27 07:23:06,154][20347] Updated weights for policy 0, policy_version 250758 (0.0006) [2023-12-27 07:23:06,612][20381] Updated weights for policy 1, policy_version 250389 (0.0008) [2023-12-27 07:23:06,668][20381] Updated weights for policy 1, policy_version 250399 (0.0006) [2023-12-27 07:23:06,723][20381] Updated weights for policy 1, policy_version 250409 (0.0005) [2023-12-27 07:23:06,820][20347] Updated weights for policy 0, policy_version 250768 (0.0006) [2023-12-27 07:23:06,877][20347] Updated weights for policy 0, policy_version 250778 (0.0006) [2023-12-27 07:23:06,926][20347] Updated weights for policy 0, policy_version 250788 (0.0005) [2023-12-27 07:23:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.2, 300 sec: 20799.3). Total num frames: 128335872. Throughput: 0: 10370.6, 1: 10381.4. Samples: 128304640. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:23:07,363][19170] Avg episode reward: [(0, '238.260'), (1, '962.360')] [2023-12-27 07:23:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000250792_64217088.pth... [2023-12-27 07:23:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000249576_63905792.pth [2023-12-27 07:23:07,389][20381] Updated weights for policy 1, policy_version 250419 (0.0006) [2023-12-27 07:23:07,441][20381] Updated weights for policy 1, policy_version 250429 (0.0007) [2023-12-27 07:23:07,491][20347] Updated weights for policy 0, policy_version 250798 (0.0006) [2023-12-27 07:23:07,492][20381] Updated weights for policy 1, policy_version 250439 (0.0005) [2023-12-27 07:23:07,540][20347] Updated weights for policy 0, policy_version 250808 (0.0008) [2023-12-27 07:23:07,544][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000250448_64126976.pth... [2023-12-27 07:23:07,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000249200_63807488.pth [2023-12-27 07:23:07,588][20347] Updated weights for policy 0, policy_version 250818 (0.0009) [2023-12-27 07:23:08,119][20381] Updated weights for policy 1, policy_version 250449 (0.0005) [2023-12-27 07:23:08,183][20381] Updated weights for policy 1, policy_version 250459 (0.0006) [2023-12-27 07:23:08,213][20347] Updated weights for policy 0, policy_version 250828 (0.0009) [2023-12-27 07:23:08,232][20381] Updated weights for policy 1, policy_version 250469 (0.0008) [2023-12-27 07:23:08,274][20347] Updated weights for policy 0, policy_version 250838 (0.0007) [2023-12-27 07:23:08,283][20381] Updated weights for policy 1, policy_version 250479 (0.0006) [2023-12-27 07:23:08,325][20347] Updated weights for policy 0, policy_version 250848 (0.0008) [2023-12-27 07:23:08,937][20347] Updated weights for policy 0, policy_version 250858 (0.0007) [2023-12-27 07:23:08,992][20381] Updated weights for policy 1, policy_version 250489 (0.0006) [2023-12-27 07:23:09,000][20347] Updated weights for policy 0, policy_version 250868 (0.0008) [2023-12-27 07:23:09,054][20381] Updated weights for policy 1, policy_version 250499 (0.0007) [2023-12-27 07:23:09,062][20347] Updated weights for policy 0, policy_version 250878 (0.0008) [2023-12-27 07:23:09,105][20381] Updated weights for policy 1, policy_version 250509 (0.0006) [2023-12-27 07:23:09,131][20347] Updated weights for policy 0, policy_version 250888 (0.0007) [2023-12-27 07:23:09,770][20381] Updated weights for policy 1, policy_version 250519 (0.0007) [2023-12-27 07:23:09,833][20381] Updated weights for policy 1, policy_version 250529 (0.0008) [2023-12-27 07:23:09,835][20347] Updated weights for policy 0, policy_version 250898 (0.0009) [2023-12-27 07:23:09,892][20381] Updated weights for policy 1, policy_version 250539 (0.0009) [2023-12-27 07:23:09,895][20347] Updated weights for policy 0, policy_version 250908 (0.0007) [2023-12-27 07:23:09,960][20347] Updated weights for policy 0, policy_version 250918 (0.0007) [2023-12-27 07:23:10,461][20381] Updated weights for policy 1, policy_version 250549 (0.0006) [2023-12-27 07:23:10,508][20381] Updated weights for policy 1, policy_version 250559 (0.0005) [2023-12-27 07:23:10,535][20292] KL-divergence is very high: 118.6903 [2023-12-27 07:23:10,540][20292] KL-divergence is very high: 118.2614 [2023-12-27 07:23:10,559][20381] Updated weights for policy 1, policy_version 250569 (0.0005) [2023-12-27 07:23:10,713][20347] Updated weights for policy 0, policy_version 250928 (0.0008) [2023-12-27 07:23:10,771][20347] Updated weights for policy 0, policy_version 250938 (0.0008) [2023-12-27 07:23:10,832][20347] Updated weights for policy 0, policy_version 250948 (0.0006) [2023-12-27 07:23:11,168][20381] Updated weights for policy 1, policy_version 250579 (0.0006) [2023-12-27 07:23:11,223][20381] Updated weights for policy 1, policy_version 250589 (0.0007) [2023-12-27 07:23:11,290][20381] Updated weights for policy 1, policy_version 250599 (0.0008) [2023-12-27 07:23:11,445][20347] Updated weights for policy 0, policy_version 250958 (0.0007) [2023-12-27 07:23:11,502][20347] Updated weights for policy 0, policy_version 250968 (0.0008) [2023-12-27 07:23:11,555][20347] Updated weights for policy 0, policy_version 250978 (0.0008) [2023-12-27 07:23:12,038][20381] Updated weights for policy 1, policy_version 250609 (0.0007) [2023-12-27 07:23:12,099][20381] Updated weights for policy 1, policy_version 250619 (0.0008) [2023-12-27 07:23:12,164][20381] Updated weights for policy 1, policy_version 250629 (0.0008) [2023-12-27 07:23:12,221][20381] Updated weights for policy 1, policy_version 250639 (0.0008) [2023-12-27 07:23:12,287][20347] Updated weights for policy 0, policy_version 250988 (0.0008) [2023-12-27 07:23:12,351][20347] Updated weights for policy 0, policy_version 250998 (0.0007) [2023-12-27 07:23:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 128442368. Throughput: 0: 10383.7, 1: 10518.5. Samples: 128434016. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 07:23:12,363][19170] Avg episode reward: [(0, '326.400'), (1, '896.380')] [2023-12-27 07:23:12,411][20347] Updated weights for policy 0, policy_version 251008 (0.0007) [2023-12-27 07:23:12,925][20381] Updated weights for policy 1, policy_version 250649 (0.0005) [2023-12-27 07:23:12,971][20381] Updated weights for policy 1, policy_version 250659 (0.0006) [2023-12-27 07:23:13,023][20381] Updated weights for policy 1, policy_version 250669 (0.0006) [2023-12-27 07:23:13,077][20347] Updated weights for policy 0, policy_version 251018 (0.0008) [2023-12-27 07:23:13,134][20347] Updated weights for policy 0, policy_version 251028 (0.0010) [2023-12-27 07:23:13,188][20347] Updated weights for policy 0, policy_version 251038 (0.0010) [2023-12-27 07:23:13,236][20347] Updated weights for policy 0, policy_version 251048 (0.0010) [2023-12-27 07:23:13,587][20381] Updated weights for policy 1, policy_version 250679 (0.0005) [2023-12-27 07:23:13,635][20381] Updated weights for policy 1, policy_version 250689 (0.0005) [2023-12-27 07:23:13,683][20381] Updated weights for policy 1, policy_version 250699 (0.0005) [2023-12-27 07:23:13,899][20347] Updated weights for policy 0, policy_version 251058 (0.0010) [2023-12-27 07:23:13,957][20347] Updated weights for policy 0, policy_version 251068 (0.0010) [2023-12-27 07:23:14,005][20347] Updated weights for policy 0, policy_version 251078 (0.0010) [2023-12-27 07:23:14,295][20381] Updated weights for policy 1, policy_version 250709 (0.0008) [2023-12-27 07:23:14,346][20381] Updated weights for policy 1, policy_version 250719 (0.0010) [2023-12-27 07:23:14,399][20381] Updated weights for policy 1, policy_version 250729 (0.0011) [2023-12-27 07:23:14,682][20347] Updated weights for policy 0, policy_version 251088 (0.0008) [2023-12-27 07:23:14,728][20347] Updated weights for policy 0, policy_version 251098 (0.0007) [2023-12-27 07:23:14,779][20347] Updated weights for policy 0, policy_version 251108 (0.0008) [2023-12-27 07:23:15,080][20381] Updated weights for policy 1, policy_version 250739 (0.0007) [2023-12-27 07:23:15,135][20381] Updated weights for policy 1, policy_version 250749 (0.0008) [2023-12-27 07:23:15,191][20381] Updated weights for policy 1, policy_version 250759 (0.0007) [2023-12-27 07:23:15,470][20347] Updated weights for policy 0, policy_version 251118 (0.0007) [2023-12-27 07:23:15,527][20347] Updated weights for policy 0, policy_version 251128 (0.0006) [2023-12-27 07:23:15,571][20347] Updated weights for policy 0, policy_version 251138 (0.0005) [2023-12-27 07:23:15,899][20381] Updated weights for policy 1, policy_version 250769 (0.0008) [2023-12-27 07:23:15,959][20381] Updated weights for policy 1, policy_version 250779 (0.0007) [2023-12-27 07:23:16,010][20381] Updated weights for policy 1, policy_version 250789 (0.0005) [2023-12-27 07:23:16,070][20381] Updated weights for policy 1, policy_version 250799 (0.0006) [2023-12-27 07:23:16,144][20347] Updated weights for policy 0, policy_version 251148 (0.0006) [2023-12-27 07:23:16,195][20347] Updated weights for policy 0, policy_version 251158 (0.0009) [2023-12-27 07:23:16,251][20347] Updated weights for policy 0, policy_version 251168 (0.0007) [2023-12-27 07:23:16,593][20381] Updated weights for policy 1, policy_version 250809 (0.0006) [2023-12-27 07:23:16,644][20381] Updated weights for policy 1, policy_version 250819 (0.0005) [2023-12-27 07:23:16,689][20381] Updated weights for policy 1, policy_version 250829 (0.0007) [2023-12-27 07:23:16,892][20347] Updated weights for policy 0, policy_version 251178 (0.0007) [2023-12-27 07:23:16,944][20347] Updated weights for policy 0, policy_version 251188 (0.0007) [2023-12-27 07:23:16,999][20347] Updated weights for policy 0, policy_version 251198 (0.0008) [2023-12-27 07:23:17,048][20347] Updated weights for policy 0, policy_version 251208 (0.0007) [2023-12-27 07:23:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 128548864. Throughput: 0: 10431.9, 1: 10502.3. Samples: 128560612. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:17,363][19170] Avg episode reward: [(0, '485.630'), (1, '709.650')] [2023-12-27 07:23:17,378][20381] Updated weights for policy 1, policy_version 250839 (0.0008) [2023-12-27 07:23:17,430][20381] Updated weights for policy 1, policy_version 250849 (0.0008) [2023-12-27 07:23:17,486][20381] Updated weights for policy 1, policy_version 250859 (0.0008) [2023-12-27 07:23:17,709][20347] Updated weights for policy 0, policy_version 251218 (0.0007) [2023-12-27 07:23:17,766][20347] Updated weights for policy 0, policy_version 251228 (0.0007) [2023-12-27 07:23:17,821][20347] Updated weights for policy 0, policy_version 251238 (0.0008) [2023-12-27 07:23:18,193][20381] Updated weights for policy 1, policy_version 250869 (0.0007) [2023-12-27 07:23:18,244][20381] Updated weights for policy 1, policy_version 250879 (0.0007) [2023-12-27 07:23:18,302][20381] Updated weights for policy 1, policy_version 250889 (0.0008) [2023-12-27 07:23:18,501][20347] Updated weights for policy 0, policy_version 251248 (0.0007) [2023-12-27 07:23:18,556][20347] Updated weights for policy 0, policy_version 251258 (0.0008) [2023-12-27 07:23:18,614][20347] Updated weights for policy 0, policy_version 251268 (0.0007) [2023-12-27 07:23:19,007][20381] Updated weights for policy 1, policy_version 250899 (0.0008) [2023-12-27 07:23:19,060][20381] Updated weights for policy 1, policy_version 250909 (0.0008) [2023-12-27 07:23:19,108][20292] KL-divergence is very high: 116.7515 [2023-12-27 07:23:19,124][20381] Updated weights for policy 1, policy_version 250919 (0.0008) [2023-12-27 07:23:19,328][20347] Updated weights for policy 0, policy_version 251278 (0.0008) [2023-12-27 07:23:19,386][20347] Updated weights for policy 0, policy_version 251288 (0.0006) [2023-12-27 07:23:19,439][20347] Updated weights for policy 0, policy_version 251298 (0.0008) [2023-12-27 07:23:19,851][20381] Updated weights for policy 1, policy_version 250929 (0.0008) [2023-12-27 07:23:19,902][20381] Updated weights for policy 1, policy_version 250939 (0.0008) [2023-12-27 07:23:19,966][20381] Updated weights for policy 1, policy_version 250949 (0.0007) [2023-12-27 07:23:20,027][20381] Updated weights for policy 1, policy_version 250959 (0.0008) [2023-12-27 07:23:20,200][20347] Updated weights for policy 0, policy_version 251308 (0.0007) [2023-12-27 07:23:20,261][20347] Updated weights for policy 0, policy_version 251318 (0.0008) [2023-12-27 07:23:20,314][20347] Updated weights for policy 0, policy_version 251328 (0.0008) [2023-12-27 07:23:20,711][20381] Updated weights for policy 1, policy_version 250969 (0.0009) [2023-12-27 07:23:20,764][20381] Updated weights for policy 1, policy_version 250979 (0.0006) [2023-12-27 07:23:20,816][20381] Updated weights for policy 1, policy_version 250989 (0.0005) [2023-12-27 07:23:21,085][20347] Updated weights for policy 0, policy_version 251338 (0.0007) [2023-12-27 07:23:21,145][20347] Updated weights for policy 0, policy_version 251348 (0.0008) [2023-12-27 07:23:21,208][20347] Updated weights for policy 0, policy_version 251358 (0.0006) [2023-12-27 07:23:21,267][20347] Updated weights for policy 0, policy_version 251368 (0.0007) [2023-12-27 07:23:21,480][20381] Updated weights for policy 1, policy_version 250999 (0.0007) [2023-12-27 07:23:21,538][20381] Updated weights for policy 1, policy_version 251009 (0.0006) [2023-12-27 07:23:21,602][20381] Updated weights for policy 1, policy_version 251019 (0.0006) [2023-12-27 07:23:21,959][20347] Updated weights for policy 0, policy_version 251378 (0.0009) [2023-12-27 07:23:22,023][20347] Updated weights for policy 0, policy_version 251388 (0.0011) [2023-12-27 07:23:22,090][20347] Updated weights for policy 0, policy_version 251398 (0.0011) [2023-12-27 07:23:22,288][20381] Updated weights for policy 1, policy_version 251029 (0.0011) [2023-12-27 07:23:22,342][20381] Updated weights for policy 1, policy_version 251039 (0.0011) [2023-12-27 07:23:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 128647168. Throughput: 0: 10428.1, 1: 10430.0. Samples: 128619464. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:22,363][19170] Avg episode reward: [(0, '385.710'), (1, '457.350')] [2023-12-27 07:23:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000251400_64372736.pth... [2023-12-27 07:23:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000250216_64069632.pth [2023-12-27 07:23:22,406][20381] Updated weights for policy 1, policy_version 251049 (0.0011) [2023-12-27 07:23:22,440][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000251056_64282624.pth... [2023-12-27 07:23:22,443][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000249808_63963136.pth [2023-12-27 07:23:22,777][20347] Updated weights for policy 0, policy_version 251408 (0.0011) [2023-12-27 07:23:22,839][20347] Updated weights for policy 0, policy_version 251418 (0.0010) [2023-12-27 07:23:22,900][20347] Updated weights for policy 0, policy_version 251428 (0.0010) [2023-12-27 07:23:23,009][20381] Updated weights for policy 1, policy_version 251059 (0.0010) [2023-12-27 07:23:23,056][20381] Updated weights for policy 1, policy_version 251069 (0.0010) [2023-12-27 07:23:23,103][20381] Updated weights for policy 1, policy_version 251079 (0.0008) [2023-12-27 07:23:23,547][20347] Updated weights for policy 0, policy_version 251438 (0.0010) [2023-12-27 07:23:23,596][20347] Updated weights for policy 0, policy_version 251448 (0.0010) [2023-12-27 07:23:23,647][20347] Updated weights for policy 0, policy_version 251458 (0.0010) [2023-12-27 07:23:23,761][20381] Updated weights for policy 1, policy_version 251089 (0.0007) [2023-12-27 07:23:23,814][20381] Updated weights for policy 1, policy_version 251099 (0.0005) [2023-12-27 07:23:23,862][20381] Updated weights for policy 1, policy_version 251109 (0.0006) [2023-12-27 07:23:23,913][20381] Updated weights for policy 1, policy_version 251119 (0.0006) [2023-12-27 07:23:24,288][20347] Updated weights for policy 0, policy_version 251468 (0.0008) [2023-12-27 07:23:24,336][20347] Updated weights for policy 0, policy_version 251478 (0.0006) [2023-12-27 07:23:24,402][20347] Updated weights for policy 0, policy_version 251488 (0.0010) [2023-12-27 07:23:24,595][20381] Updated weights for policy 1, policy_version 251129 (0.0007) [2023-12-27 07:23:24,651][20381] Updated weights for policy 1, policy_version 251139 (0.0006) [2023-12-27 07:23:24,708][20381] Updated weights for policy 1, policy_version 251149 (0.0007) [2023-12-27 07:23:25,067][20347] Updated weights for policy 0, policy_version 251498 (0.0010) [2023-12-27 07:23:25,129][20347] Updated weights for policy 0, policy_version 251508 (0.0008) [2023-12-27 07:23:25,188][20347] Updated weights for policy 0, policy_version 251518 (0.0010) [2023-12-27 07:23:25,249][20347] Updated weights for policy 0, policy_version 251528 (0.0010) [2023-12-27 07:23:25,350][20381] Updated weights for policy 1, policy_version 251159 (0.0006) [2023-12-27 07:23:25,407][20381] Updated weights for policy 1, policy_version 251169 (0.0006) [2023-12-27 07:23:25,464][20381] Updated weights for policy 1, policy_version 251179 (0.0005) [2023-12-27 07:23:25,914][20347] Updated weights for policy 0, policy_version 251538 (0.0010) [2023-12-27 07:23:25,969][20347] Updated weights for policy 0, policy_version 251548 (0.0010) [2023-12-27 07:23:26,027][20347] Updated weights for policy 0, policy_version 251558 (0.0010) [2023-12-27 07:23:26,049][20381] Updated weights for policy 1, policy_version 251189 (0.0005) [2023-12-27 07:23:26,107][20381] Updated weights for policy 1, policy_version 251199 (0.0005) [2023-12-27 07:23:26,161][20381] Updated weights for policy 1, policy_version 251209 (0.0005) [2023-12-27 07:23:26,662][20381] Updated weights for policy 1, policy_version 251219 (0.0006) [2023-12-27 07:23:26,694][20347] Updated weights for policy 0, policy_version 251568 (0.0010) [2023-12-27 07:23:26,710][20381] Updated weights for policy 1, policy_version 251229 (0.0005) [2023-12-27 07:23:26,739][20347] Updated weights for policy 0, policy_version 251578 (0.0010) [2023-12-27 07:23:26,755][20381] Updated weights for policy 1, policy_version 251239 (0.0005) [2023-12-27 07:23:26,787][20347] Updated weights for policy 0, policy_version 251588 (0.0010) [2023-12-27 07:23:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 128753664. Throughput: 0: 10314.9, 1: 10531.9. Samples: 128744664. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:27,363][19170] Avg episode reward: [(0, '245.450'), (1, '321.260')] [2023-12-27 07:23:27,420][20347] Updated weights for policy 0, policy_version 251598 (0.0009) [2023-12-27 07:23:27,444][20381] Updated weights for policy 1, policy_version 251249 (0.0006) [2023-12-27 07:23:27,471][20347] Updated weights for policy 0, policy_version 251608 (0.0010) [2023-12-27 07:23:27,495][20381] Updated weights for policy 1, policy_version 251259 (0.0005) [2023-12-27 07:23:27,525][20347] Updated weights for policy 0, policy_version 251618 (0.0007) [2023-12-27 07:23:27,560][20381] Updated weights for policy 1, policy_version 251269 (0.0006) [2023-12-27 07:23:27,617][20381] Updated weights for policy 1, policy_version 251279 (0.0008) [2023-12-27 07:23:28,169][20347] Updated weights for policy 0, policy_version 251628 (0.0009) [2023-12-27 07:23:28,227][20347] Updated weights for policy 0, policy_version 251638 (0.0010) [2023-12-27 07:23:28,285][20347] Updated weights for policy 0, policy_version 251648 (0.0008) [2023-12-27 07:23:28,304][20381] Updated weights for policy 1, policy_version 251289 (0.0007) [2023-12-27 07:23:28,363][20381] Updated weights for policy 1, policy_version 251299 (0.0006) [2023-12-27 07:23:28,422][20381] Updated weights for policy 1, policy_version 251309 (0.0008) [2023-12-27 07:23:28,961][20347] Updated weights for policy 0, policy_version 251658 (0.0007) [2023-12-27 07:23:29,009][20347] Updated weights for policy 0, policy_version 251668 (0.0005) [2023-12-27 07:23:29,059][20381] Updated weights for policy 1, policy_version 251319 (0.0008) [2023-12-27 07:23:29,064][20347] Updated weights for policy 0, policy_version 251678 (0.0006) [2023-12-27 07:23:29,112][20381] Updated weights for policy 1, policy_version 251329 (0.0007) [2023-12-27 07:23:29,122][20347] Updated weights for policy 0, policy_version 251688 (0.0006) [2023-12-27 07:23:29,164][20381] Updated weights for policy 1, policy_version 251339 (0.0007) [2023-12-27 07:23:29,751][20347] Updated weights for policy 0, policy_version 251698 (0.0010) [2023-12-27 07:23:29,800][20347] Updated weights for policy 0, policy_version 251708 (0.0010) [2023-12-27 07:23:29,867][20347] Updated weights for policy 0, policy_version 251718 (0.0008) [2023-12-27 07:23:29,878][20381] Updated weights for policy 1, policy_version 251349 (0.0009) [2023-12-27 07:23:29,939][20381] Updated weights for policy 1, policy_version 251359 (0.0010) [2023-12-27 07:23:29,992][20381] Updated weights for policy 1, policy_version 251369 (0.0007) [2023-12-27 07:23:30,572][20347] Updated weights for policy 0, policy_version 251728 (0.0008) [2023-12-27 07:23:30,616][20347] Updated weights for policy 0, policy_version 251738 (0.0007) [2023-12-27 07:23:30,654][20381] Updated weights for policy 1, policy_version 251379 (0.0009) [2023-12-27 07:23:30,660][20347] Updated weights for policy 0, policy_version 251748 (0.0007) [2023-12-27 07:23:30,702][20381] Updated weights for policy 1, policy_version 251389 (0.0010) [2023-12-27 07:23:30,759][20381] Updated weights for policy 1, policy_version 251399 (0.0010) [2023-12-27 07:23:31,392][20347] Updated weights for policy 0, policy_version 251758 (0.0008) [2023-12-27 07:23:31,416][20381] Updated weights for policy 1, policy_version 251409 (0.0006) [2023-12-27 07:23:31,442][20347] Updated weights for policy 0, policy_version 251768 (0.0006) [2023-12-27 07:23:31,471][20381] Updated weights for policy 1, policy_version 251419 (0.0010) [2023-12-27 07:23:31,498][20347] Updated weights for policy 0, policy_version 251778 (0.0006) [2023-12-27 07:23:31,529][20381] Updated weights for policy 1, policy_version 251429 (0.0010) [2023-12-27 07:23:31,586][20381] Updated weights for policy 1, policy_version 251439 (0.0010) [2023-12-27 07:23:32,168][20347] Updated weights for policy 0, policy_version 251788 (0.0005) [2023-12-27 07:23:32,196][20381] Updated weights for policy 1, policy_version 251449 (0.0010) [2023-12-27 07:23:32,225][20347] Updated weights for policy 0, policy_version 251798 (0.0006) [2023-12-27 07:23:32,246][20381] Updated weights for policy 1, policy_version 251459 (0.0010) [2023-12-27 07:23:32,284][20347] Updated weights for policy 0, policy_version 251808 (0.0007) [2023-12-27 07:23:32,307][20381] Updated weights for policy 1, policy_version 251469 (0.0007) [2023-12-27 07:23:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 128868352. Throughput: 0: 10285.4, 1: 10604.1. Samples: 128871068. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:32,363][19170] Avg episode reward: [(0, '403.770'), (1, '577.900')] [2023-12-27 07:23:32,916][20347] Updated weights for policy 0, policy_version 251818 (0.0007) [2023-12-27 07:23:32,969][20347] Updated weights for policy 0, policy_version 251828 (0.0005) [2023-12-27 07:23:32,984][20381] Updated weights for policy 1, policy_version 251479 (0.0010) [2023-12-27 07:23:33,019][20347] Updated weights for policy 0, policy_version 251838 (0.0006) [2023-12-27 07:23:33,035][20381] Updated weights for policy 1, policy_version 251489 (0.0011) [2023-12-27 07:23:33,074][20347] Updated weights for policy 0, policy_version 251848 (0.0005) [2023-12-27 07:23:33,081][20381] Updated weights for policy 1, policy_version 251499 (0.0010) [2023-12-27 07:23:33,744][20347] Updated weights for policy 0, policy_version 251858 (0.0007) [2023-12-27 07:23:33,747][20381] Updated weights for policy 1, policy_version 251509 (0.0008) [2023-12-27 07:23:33,798][20347] Updated weights for policy 0, policy_version 251868 (0.0005) [2023-12-27 07:23:33,800][20381] Updated weights for policy 1, policy_version 251519 (0.0007) [2023-12-27 07:23:33,847][20381] Updated weights for policy 1, policy_version 251529 (0.0007) [2023-12-27 07:23:33,849][20347] Updated weights for policy 0, policy_version 251878 (0.0007) [2023-12-27 07:23:34,479][20347] Updated weights for policy 0, policy_version 251888 (0.0007) [2023-12-27 07:23:34,537][20347] Updated weights for policy 0, policy_version 251898 (0.0005) [2023-12-27 07:23:34,573][20381] Updated weights for policy 1, policy_version 251539 (0.0007) [2023-12-27 07:23:34,594][20347] Updated weights for policy 0, policy_version 251908 (0.0005) [2023-12-27 07:23:34,632][20381] Updated weights for policy 1, policy_version 251549 (0.0009) [2023-12-27 07:23:34,682][20381] Updated weights for policy 1, policy_version 251559 (0.0008) [2023-12-27 07:23:35,133][20347] Updated weights for policy 0, policy_version 251918 (0.0006) [2023-12-27 07:23:35,189][20347] Updated weights for policy 0, policy_version 251928 (0.0007) [2023-12-27 07:23:35,241][20347] Updated weights for policy 0, policy_version 251938 (0.0009) [2023-12-27 07:23:35,355][20381] Updated weights for policy 1, policy_version 251569 (0.0008) [2023-12-27 07:23:35,412][20381] Updated weights for policy 1, policy_version 251579 (0.0006) [2023-12-27 07:23:35,462][20381] Updated weights for policy 1, policy_version 251589 (0.0006) [2023-12-27 07:23:35,508][20381] Updated weights for policy 1, policy_version 251599 (0.0008) [2023-12-27 07:23:35,965][20347] Updated weights for policy 0, policy_version 251948 (0.0008) [2023-12-27 07:23:36,009][20347] Updated weights for policy 0, policy_version 251958 (0.0007) [2023-12-27 07:23:36,056][20347] Updated weights for policy 0, policy_version 251968 (0.0007) [2023-12-27 07:23:36,172][20381] Updated weights for policy 1, policy_version 251609 (0.0007) [2023-12-27 07:23:36,225][20381] Updated weights for policy 1, policy_version 251619 (0.0008) [2023-12-27 07:23:36,287][20381] Updated weights for policy 1, policy_version 251629 (0.0008) [2023-12-27 07:23:36,733][20347] Updated weights for policy 0, policy_version 251978 (0.0008) [2023-12-27 07:23:36,796][20347] Updated weights for policy 0, policy_version 251988 (0.0011) [2023-12-27 07:23:36,861][20347] Updated weights for policy 0, policy_version 251998 (0.0010) [2023-12-27 07:23:36,927][20347] Updated weights for policy 0, policy_version 252008 (0.0010) [2023-12-27 07:23:37,016][20381] Updated weights for policy 1, policy_version 251639 (0.0008) [2023-12-27 07:23:37,071][20381] Updated weights for policy 1, policy_version 251649 (0.0007) [2023-12-27 07:23:37,129][20381] Updated weights for policy 1, policy_version 251659 (0.0008) [2023-12-27 07:23:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 128966656. Throughput: 0: 10327.4, 1: 10602.2. Samples: 128934952. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:37,363][19170] Avg episode reward: [(0, '330.510'), (1, '659.540')] [2023-12-27 07:23:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000251664_64438272.pth... [2023-12-27 07:23:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000252008_64528384.pth... [2023-12-27 07:23:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000250448_64126976.pth [2023-12-27 07:23:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000250792_64217088.pth [2023-12-27 07:23:37,596][20347] Updated weights for policy 0, policy_version 252018 (0.0010) [2023-12-27 07:23:37,641][20347] Updated weights for policy 0, policy_version 252028 (0.0010) [2023-12-27 07:23:37,693][20347] Updated weights for policy 0, policy_version 252038 (0.0006) [2023-12-27 07:23:37,854][20381] Updated weights for policy 1, policy_version 251669 (0.0007) [2023-12-27 07:23:37,898][20381] Updated weights for policy 1, policy_version 251679 (0.0007) [2023-12-27 07:23:37,946][20381] Updated weights for policy 1, policy_version 251689 (0.0007) [2023-12-27 07:23:38,342][20347] Updated weights for policy 0, policy_version 252048 (0.0009) [2023-12-27 07:23:38,403][20347] Updated weights for policy 0, policy_version 252058 (0.0010) [2023-12-27 07:23:38,460][20347] Updated weights for policy 0, policy_version 252068 (0.0010) [2023-12-27 07:23:38,681][20381] Updated weights for policy 1, policy_version 251699 (0.0007) [2023-12-27 07:23:38,727][20381] Updated weights for policy 1, policy_version 251709 (0.0009) [2023-12-27 07:23:38,780][20381] Updated weights for policy 1, policy_version 251719 (0.0008) [2023-12-27 07:23:39,056][20347] Updated weights for policy 0, policy_version 252078 (0.0010) [2023-12-27 07:23:39,105][20347] Updated weights for policy 0, policy_version 252088 (0.0010) [2023-12-27 07:23:39,162][20347] Updated weights for policy 0, policy_version 252098 (0.0010) [2023-12-27 07:23:39,565][20381] Updated weights for policy 1, policy_version 251729 (0.0008) [2023-12-27 07:23:39,625][20381] Updated weights for policy 1, policy_version 251739 (0.0008) [2023-12-27 07:23:39,682][20381] Updated weights for policy 1, policy_version 251749 (0.0007) [2023-12-27 07:23:39,741][20381] Updated weights for policy 1, policy_version 251759 (0.0007) [2023-12-27 07:23:39,949][20347] Updated weights for policy 0, policy_version 252108 (0.0011) [2023-12-27 07:23:40,008][20347] Updated weights for policy 0, policy_version 252118 (0.0011) [2023-12-27 07:23:40,070][20347] Updated weights for policy 0, policy_version 252128 (0.0010) [2023-12-27 07:23:40,434][20381] Updated weights for policy 1, policy_version 251769 (0.0011) [2023-12-27 07:23:40,492][20381] Updated weights for policy 1, policy_version 251779 (0.0007) [2023-12-27 07:23:40,552][20381] Updated weights for policy 1, policy_version 251789 (0.0005) [2023-12-27 07:23:40,726][20347] Updated weights for policy 0, policy_version 252138 (0.0010) [2023-12-27 07:23:40,773][20347] Updated weights for policy 0, policy_version 252148 (0.0010) [2023-12-27 07:23:40,836][20347] Updated weights for policy 0, policy_version 252158 (0.0010) [2023-12-27 07:23:40,896][20347] Updated weights for policy 0, policy_version 252168 (0.0011) [2023-12-27 07:23:41,139][20381] Updated weights for policy 1, policy_version 251799 (0.0009) [2023-12-27 07:23:41,199][20381] Updated weights for policy 1, policy_version 251809 (0.0010) [2023-12-27 07:23:41,262][20381] Updated weights for policy 1, policy_version 251819 (0.0012) [2023-12-27 07:23:41,625][20347] Updated weights for policy 0, policy_version 252178 (0.0008) [2023-12-27 07:23:41,679][20347] Updated weights for policy 0, policy_version 252188 (0.0009) [2023-12-27 07:23:41,742][20347] Updated weights for policy 0, policy_version 252198 (0.0009) [2023-12-27 07:23:41,979][20381] Updated weights for policy 1, policy_version 251829 (0.0008) [2023-12-27 07:23:42,039][20381] Updated weights for policy 1, policy_version 251839 (0.0007) [2023-12-27 07:23:42,099][20381] Updated weights for policy 1, policy_version 251849 (0.0008) [2023-12-27 07:23:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 129064960. Throughput: 0: 10322.7, 1: 10536.1. Samples: 129056196. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:42,363][19170] Avg episode reward: [(0, '348.720'), (1, '799.460')] [2023-12-27 07:23:42,447][20347] Updated weights for policy 0, policy_version 252208 (0.0006) [2023-12-27 07:23:42,499][20347] Updated weights for policy 0, policy_version 252218 (0.0005) [2023-12-27 07:23:42,557][20347] Updated weights for policy 0, policy_version 252228 (0.0007) [2023-12-27 07:23:42,867][20381] Updated weights for policy 1, policy_version 251859 (0.0008) [2023-12-27 07:23:42,927][20381] Updated weights for policy 1, policy_version 251869 (0.0011) [2023-12-27 07:23:42,986][20381] Updated weights for policy 1, policy_version 251879 (0.0009) [2023-12-27 07:23:43,288][20347] Updated weights for policy 0, policy_version 252238 (0.0007) [2023-12-27 07:23:43,352][20347] Updated weights for policy 0, policy_version 252248 (0.0008) [2023-12-27 07:23:43,399][20347] Updated weights for policy 0, policy_version 252258 (0.0007) [2023-12-27 07:23:43,628][20381] Updated weights for policy 1, policy_version 251889 (0.0007) [2023-12-27 07:23:43,692][20381] Updated weights for policy 1, policy_version 251899 (0.0006) [2023-12-27 07:23:43,748][20381] Updated weights for policy 1, policy_version 251909 (0.0006) [2023-12-27 07:23:43,812][20381] Updated weights for policy 1, policy_version 251919 (0.0006) [2023-12-27 07:23:44,087][20347] Updated weights for policy 0, policy_version 252268 (0.0006) [2023-12-27 07:23:44,149][20347] Updated weights for policy 0, policy_version 252278 (0.0008) [2023-12-27 07:23:44,203][20347] Updated weights for policy 0, policy_version 252288 (0.0006) [2023-12-27 07:23:44,399][20381] Updated weights for policy 1, policy_version 251929 (0.0010) [2023-12-27 07:23:44,458][20381] Updated weights for policy 1, policy_version 251939 (0.0009) [2023-12-27 07:23:44,513][20381] Updated weights for policy 1, policy_version 251949 (0.0006) [2023-12-27 07:23:44,856][20347] Updated weights for policy 0, policy_version 252298 (0.0006) [2023-12-27 07:23:44,918][20347] Updated weights for policy 0, policy_version 252308 (0.0010) [2023-12-27 07:23:44,980][20347] Updated weights for policy 0, policy_version 252318 (0.0011) [2023-12-27 07:23:45,040][20347] Updated weights for policy 0, policy_version 252328 (0.0009) [2023-12-27 07:23:45,192][20381] Updated weights for policy 1, policy_version 251959 (0.0008) [2023-12-27 07:23:45,244][20381] Updated weights for policy 1, policy_version 251969 (0.0008) [2023-12-27 07:23:45,298][20381] Updated weights for policy 1, policy_version 251979 (0.0005) [2023-12-27 07:23:45,698][20347] Updated weights for policy 0, policy_version 252338 (0.0007) [2023-12-27 07:23:45,755][20347] Updated weights for policy 0, policy_version 252348 (0.0006) [2023-12-27 07:23:45,800][20347] Updated weights for policy 0, policy_version 252358 (0.0006) [2023-12-27 07:23:45,891][20381] Updated weights for policy 1, policy_version 251989 (0.0010) [2023-12-27 07:23:45,946][20381] Updated weights for policy 1, policy_version 251999 (0.0011) [2023-12-27 07:23:46,005][20381] Updated weights for policy 1, policy_version 252009 (0.0011) [2023-12-27 07:23:46,452][20347] Updated weights for policy 0, policy_version 252368 (0.0007) [2023-12-27 07:23:46,502][20347] Updated weights for policy 0, policy_version 252378 (0.0008) [2023-12-27 07:23:46,547][20347] Updated weights for policy 0, policy_version 252388 (0.0007) [2023-12-27 07:23:46,675][20381] Updated weights for policy 1, policy_version 252019 (0.0010) [2023-12-27 07:23:46,722][20381] Updated weights for policy 1, policy_version 252029 (0.0009) [2023-12-27 07:23:46,775][20381] Updated weights for policy 1, policy_version 252039 (0.0008) [2023-12-27 07:23:47,204][20347] Updated weights for policy 0, policy_version 252398 (0.0007) [2023-12-27 07:23:47,248][20347] Updated weights for policy 0, policy_version 252408 (0.0007) [2023-12-27 07:23:47,294][20347] Updated weights for policy 0, policy_version 252418 (0.0007) [2023-12-27 07:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 129171456. Throughput: 0: 10329.4, 1: 10509.7. Samples: 129179116. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:47,363][19170] Avg episode reward: [(0, '314.850'), (1, '827.060')] [2023-12-27 07:23:47,484][20381] Updated weights for policy 1, policy_version 252049 (0.0009) [2023-12-27 07:23:47,537][20381] Updated weights for policy 1, policy_version 252059 (0.0010) [2023-12-27 07:23:47,592][20381] Updated weights for policy 1, policy_version 252069 (0.0010) [2023-12-27 07:23:47,645][20381] Updated weights for policy 1, policy_version 252079 (0.0010) [2023-12-27 07:23:48,003][20347] Updated weights for policy 0, policy_version 252428 (0.0008) [2023-12-27 07:23:48,055][20347] Updated weights for policy 0, policy_version 252438 (0.0009) [2023-12-27 07:23:48,113][20347] Updated weights for policy 0, policy_version 252448 (0.0005) [2023-12-27 07:23:48,337][20381] Updated weights for policy 1, policy_version 252089 (0.0010) [2023-12-27 07:23:48,398][20381] Updated weights for policy 1, policy_version 252099 (0.0011) [2023-12-27 07:23:48,451][20381] Updated weights for policy 1, policy_version 252109 (0.0008) [2023-12-27 07:23:48,704][20347] Updated weights for policy 0, policy_version 252458 (0.0006) [2023-12-27 07:23:48,768][20347] Updated weights for policy 0, policy_version 252468 (0.0010) [2023-12-27 07:23:48,832][20347] Updated weights for policy 0, policy_version 252478 (0.0010) [2023-12-27 07:23:48,890][20347] Updated weights for policy 0, policy_version 252488 (0.0010) [2023-12-27 07:23:49,062][20381] Updated weights for policy 1, policy_version 252119 (0.0006) [2023-12-27 07:23:49,125][20381] Updated weights for policy 1, policy_version 252129 (0.0005) [2023-12-27 07:23:49,177][20381] Updated weights for policy 1, policy_version 252139 (0.0005) [2023-12-27 07:23:49,518][20347] Updated weights for policy 0, policy_version 252498 (0.0010) [2023-12-27 07:23:49,572][20347] Updated weights for policy 0, policy_version 252508 (0.0010) [2023-12-27 07:23:49,630][20347] Updated weights for policy 0, policy_version 252518 (0.0010) [2023-12-27 07:23:49,874][20381] Updated weights for policy 1, policy_version 252149 (0.0007) [2023-12-27 07:23:49,934][20381] Updated weights for policy 1, policy_version 252159 (0.0008) [2023-12-27 07:23:49,985][20381] Updated weights for policy 1, policy_version 252169 (0.0009) [2023-12-27 07:23:50,282][20347] Updated weights for policy 0, policy_version 252528 (0.0010) [2023-12-27 07:23:50,341][20347] Updated weights for policy 0, policy_version 252538 (0.0010) [2023-12-27 07:23:50,398][20347] Updated weights for policy 0, policy_version 252548 (0.0011) [2023-12-27 07:23:50,711][20381] Updated weights for policy 1, policy_version 252179 (0.0008) [2023-12-27 07:23:50,773][20381] Updated weights for policy 1, policy_version 252189 (0.0007) [2023-12-27 07:23:50,836][20381] Updated weights for policy 1, policy_version 252199 (0.0008) [2023-12-27 07:23:51,078][20347] Updated weights for policy 0, policy_version 252558 (0.0009) [2023-12-27 07:23:51,139][20347] Updated weights for policy 0, policy_version 252568 (0.0008) [2023-12-27 07:23:51,198][20347] Updated weights for policy 0, policy_version 252578 (0.0008) [2023-12-27 07:23:51,524][20381] Updated weights for policy 1, policy_version 252209 (0.0008) [2023-12-27 07:23:51,575][20381] Updated weights for policy 1, policy_version 252219 (0.0007) [2023-12-27 07:23:51,645][20381] Updated weights for policy 1, policy_version 252229 (0.0008) [2023-12-27 07:23:51,709][20381] Updated weights for policy 1, policy_version 252239 (0.0007) [2023-12-27 07:23:51,870][20347] Updated weights for policy 0, policy_version 252588 (0.0007) [2023-12-27 07:23:51,929][20347] Updated weights for policy 0, policy_version 252598 (0.0006) [2023-12-27 07:23:51,988][20347] Updated weights for policy 0, policy_version 252608 (0.0005) [2023-12-27 07:23:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 129269760. Throughput: 0: 10374.2, 1: 10452.2. Samples: 129241832. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:52,363][19170] Avg episode reward: [(0, '125.820'), (1, '866.500')] [2023-12-27 07:23:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000252616_64684032.pth... [2023-12-27 07:23:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000251400_64372736.pth [2023-12-27 07:23:52,407][20381] Updated weights for policy 1, policy_version 252249 (0.0007) [2023-12-27 07:23:52,471][20381] Updated weights for policy 1, policy_version 252259 (0.0008) [2023-12-27 07:23:52,540][20381] Updated weights for policy 1, policy_version 252269 (0.0007) [2023-12-27 07:23:52,557][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000252272_64593920.pth... [2023-12-27 07:23:52,562][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000251056_64282624.pth [2023-12-27 07:23:52,649][20347] Updated weights for policy 0, policy_version 252618 (0.0007) [2023-12-27 07:23:52,707][20347] Updated weights for policy 0, policy_version 252628 (0.0006) [2023-12-27 07:23:52,764][20347] Updated weights for policy 0, policy_version 252638 (0.0007) [2023-12-27 07:23:52,819][20347] Updated weights for policy 0, policy_version 252648 (0.0006) [2023-12-27 07:23:53,300][20381] Updated weights for policy 1, policy_version 252279 (0.0008) [2023-12-27 07:23:53,351][20347] Updated weights for policy 0, policy_version 252658 (0.0005) [2023-12-27 07:23:53,354][20381] Updated weights for policy 1, policy_version 252289 (0.0008) [2023-12-27 07:23:53,400][20381] Updated weights for policy 1, policy_version 252299 (0.0008) [2023-12-27 07:23:53,401][20347] Updated weights for policy 0, policy_version 252668 (0.0005) [2023-12-27 07:23:53,455][20347] Updated weights for policy 0, policy_version 252678 (0.0010) [2023-12-27 07:23:54,027][20347] Updated weights for policy 0, policy_version 252688 (0.0006) [2023-12-27 07:23:54,084][20347] Updated weights for policy 0, policy_version 252698 (0.0005) [2023-12-27 07:23:54,132][20347] Updated weights for policy 0, policy_version 252708 (0.0006) [2023-12-27 07:23:54,146][20381] Updated weights for policy 1, policy_version 252309 (0.0008) [2023-12-27 07:23:54,197][20381] Updated weights for policy 1, policy_version 252319 (0.0007) [2023-12-27 07:23:54,243][20381] Updated weights for policy 1, policy_version 252329 (0.0008) [2023-12-27 07:23:54,697][20347] Updated weights for policy 0, policy_version 252718 (0.0006) [2023-12-27 07:23:54,762][20347] Updated weights for policy 0, policy_version 252728 (0.0008) [2023-12-27 07:23:54,834][20347] Updated weights for policy 0, policy_version 252738 (0.0008) [2023-12-27 07:23:54,964][20381] Updated weights for policy 1, policy_version 252339 (0.0008) [2023-12-27 07:23:55,020][20381] Updated weights for policy 1, policy_version 252349 (0.0007) [2023-12-27 07:23:55,077][20381] Updated weights for policy 1, policy_version 252359 (0.0007) [2023-12-27 07:23:55,446][20347] Updated weights for policy 0, policy_version 252748 (0.0007) [2023-12-27 07:23:55,501][20347] Updated weights for policy 0, policy_version 252758 (0.0009) [2023-12-27 07:23:55,554][20347] Updated weights for policy 0, policy_version 252768 (0.0008) [2023-12-27 07:23:55,674][20381] Updated weights for policy 1, policy_version 252369 (0.0007) [2023-12-27 07:23:55,741][20381] Updated weights for policy 1, policy_version 252379 (0.0006) [2023-12-27 07:23:55,794][20381] Updated weights for policy 1, policy_version 252389 (0.0007) [2023-12-27 07:23:55,844][20381] Updated weights for policy 1, policy_version 252399 (0.0009) [2023-12-27 07:23:56,159][20347] Updated weights for policy 0, policy_version 252778 (0.0007) [2023-12-27 07:23:56,217][20347] Updated weights for policy 0, policy_version 252788 (0.0007) [2023-12-27 07:23:56,278][20347] Updated weights for policy 0, policy_version 252798 (0.0007) [2023-12-27 07:23:56,336][20347] Updated weights for policy 0, policy_version 252808 (0.0007) [2023-12-27 07:23:56,516][20381] Updated weights for policy 1, policy_version 252409 (0.0006) [2023-12-27 07:23:56,568][20381] Updated weights for policy 1, policy_version 252419 (0.0005) [2023-12-27 07:23:56,621][20381] Updated weights for policy 1, policy_version 252429 (0.0007) [2023-12-27 07:23:56,912][20347] Updated weights for policy 0, policy_version 252818 (0.0007) [2023-12-27 07:23:56,968][20347] Updated weights for policy 0, policy_version 252828 (0.0007) [2023-12-27 07:23:57,030][20347] Updated weights for policy 0, policy_version 252838 (0.0007) [2023-12-27 07:23:57,250][20381] Updated weights for policy 1, policy_version 252439 (0.0008) [2023-12-27 07:23:57,308][20381] Updated weights for policy 1, policy_version 252449 (0.0007) [2023-12-27 07:23:57,361][20381] Updated weights for policy 1, policy_version 252459 (0.0006) [2023-12-27 07:23:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 129376256. Throughput: 0: 10436.1, 1: 10311.6. Samples: 129367660. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:23:57,363][19170] Avg episode reward: [(0, '35.890'), (1, '913.930')] [2023-12-27 07:23:57,679][20347] Updated weights for policy 0, policy_version 252848 (0.0005) [2023-12-27 07:23:57,731][20347] Updated weights for policy 0, policy_version 252858 (0.0007) [2023-12-27 07:23:57,789][20347] Updated weights for policy 0, policy_version 252868 (0.0008) [2023-12-27 07:23:57,968][20381] Updated weights for policy 1, policy_version 252469 (0.0005) [2023-12-27 07:23:58,020][20381] Updated weights for policy 1, policy_version 252479 (0.0007) [2023-12-27 07:23:58,071][20381] Updated weights for policy 1, policy_version 252489 (0.0008) [2023-12-27 07:23:58,500][20347] Updated weights for policy 0, policy_version 252878 (0.0007) [2023-12-27 07:23:58,562][20347] Updated weights for policy 0, policy_version 252888 (0.0008) [2023-12-27 07:23:58,633][20347] Updated weights for policy 0, policy_version 252898 (0.0007) [2023-12-27 07:23:58,861][20381] Updated weights for policy 1, policy_version 252499 (0.0007) [2023-12-27 07:23:58,929][20381] Updated weights for policy 1, policy_version 252509 (0.0008) [2023-12-27 07:23:58,994][20381] Updated weights for policy 1, policy_version 252519 (0.0008) [2023-12-27 07:23:59,468][20347] Updated weights for policy 0, policy_version 252908 (0.0008) [2023-12-27 07:23:59,520][20347] Updated weights for policy 0, policy_version 252918 (0.0010) [2023-12-27 07:23:59,572][20347] Updated weights for policy 0, policy_version 252928 (0.0010) [2023-12-27 07:23:59,702][20381] Updated weights for policy 1, policy_version 252529 (0.0008) [2023-12-27 07:23:59,755][20381] Updated weights for policy 1, policy_version 252539 (0.0005) [2023-12-27 07:23:59,806][20381] Updated weights for policy 1, policy_version 252549 (0.0005) [2023-12-27 07:23:59,864][20381] Updated weights for policy 1, policy_version 252559 (0.0007) [2023-12-27 07:24:00,281][20347] Updated weights for policy 0, policy_version 252938 (0.0010) [2023-12-27 07:24:00,332][20347] Updated weights for policy 0, policy_version 252948 (0.0010) [2023-12-27 07:24:00,376][20347] Updated weights for policy 0, policy_version 252958 (0.0010) [2023-12-27 07:24:00,432][20347] Updated weights for policy 0, policy_version 252968 (0.0010) [2023-12-27 07:24:00,485][20381] Updated weights for policy 1, policy_version 252569 (0.0007) [2023-12-27 07:24:00,532][20381] Updated weights for policy 1, policy_version 252579 (0.0007) [2023-12-27 07:24:00,580][20381] Updated weights for policy 1, policy_version 252589 (0.0007) [2023-12-27 07:24:01,059][20347] Updated weights for policy 0, policy_version 252978 (0.0011) [2023-12-27 07:24:01,112][20347] Updated weights for policy 0, policy_version 252988 (0.0010) [2023-12-27 07:24:01,175][20347] Updated weights for policy 0, policy_version 252998 (0.0010) [2023-12-27 07:24:01,219][20381] Updated weights for policy 1, policy_version 252599 (0.0006) [2023-12-27 07:24:01,285][20381] Updated weights for policy 1, policy_version 252609 (0.0006) [2023-12-27 07:24:01,342][20381] Updated weights for policy 1, policy_version 252619 (0.0006) [2023-12-27 07:24:01,902][20347] Updated weights for policy 0, policy_version 253008 (0.0007) [2023-12-27 07:24:01,953][20347] Updated weights for policy 0, policy_version 253018 (0.0007) [2023-12-27 07:24:01,993][20381] Updated weights for policy 1, policy_version 252629 (0.0007) [2023-12-27 07:24:02,012][20347] Updated weights for policy 0, policy_version 253028 (0.0007) [2023-12-27 07:24:02,049][20381] Updated weights for policy 1, policy_version 252639 (0.0009) [2023-12-27 07:24:02,094][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-27 07:24:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 129482752. Throughput: 0: 10381.9, 1: 10278.2. Samples: 129490316. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:24:02,363][19170] Avg episode reward: [(0, '29.700'), (1, '962.320')] [2023-12-27 07:24:02,691][20381] Updated weights for policy 1, policy_version 252649 (0.0008) [2023-12-27 07:24:02,736][20347] Updated weights for policy 0, policy_version 253038 (0.0008) [2023-12-27 07:24:02,747][20381] Updated weights for policy 1, policy_version 252659 (0.0005) [2023-12-27 07:24:02,782][20347] Updated weights for policy 0, policy_version 253048 (0.0008) [2023-12-27 07:24:02,805][20381] Updated weights for policy 1, policy_version 252669 (0.0005) [2023-12-27 07:24:02,829][20347] Updated weights for policy 0, policy_version 253058 (0.0008) [2023-12-27 07:24:02,863][20381] Updated weights for policy 1, policy_version 252679 (0.0005) [2023-12-27 07:24:03,428][20347] Updated weights for policy 0, policy_version 253068 (0.0007) [2023-12-27 07:24:03,479][20347] Updated weights for policy 0, policy_version 253078 (0.0006) [2023-12-27 07:24:03,489][20381] Updated weights for policy 1, policy_version 252689 (0.0007) [2023-12-27 07:24:03,521][20347] Updated weights for policy 0, policy_version 253088 (0.0005) [2023-12-27 07:24:03,540][20381] Updated weights for policy 1, policy_version 252699 (0.0005) [2023-12-27 07:24:03,590][20381] Updated weights for policy 1, policy_version 252709 (0.0005) [2023-12-27 07:24:04,175][20347] Updated weights for policy 0, policy_version 253098 (0.0008) [2023-12-27 07:24:04,204][20381] Updated weights for policy 1, policy_version 252719 (0.0006) [2023-12-27 07:24:04,232][20347] Updated weights for policy 0, policy_version 253108 (0.0006) [2023-12-27 07:24:04,267][20381] Updated weights for policy 1, policy_version 252729 (0.0008) [2023-12-27 07:24:04,287][20347] Updated weights for policy 0, policy_version 253118 (0.0005) [2023-12-27 07:24:04,324][20381] Updated weights for policy 1, policy_version 252739 (0.0007) [2023-12-27 07:24:04,348][20347] Updated weights for policy 0, policy_version 253128 (0.0007) [2023-12-27 07:24:04,890][20381] Updated weights for policy 1, policy_version 252749 (0.0008) [2023-12-27 07:24:04,923][20347] Updated weights for policy 0, policy_version 253138 (0.0006) [2023-12-27 07:24:04,944][20381] Updated weights for policy 1, policy_version 252759 (0.0009) [2023-12-27 07:24:04,985][20347] Updated weights for policy 0, policy_version 253148 (0.0005) [2023-12-27 07:24:05,001][20381] Updated weights for policy 1, policy_version 252769 (0.0008) [2023-12-27 07:24:05,049][20347] Updated weights for policy 0, policy_version 253158 (0.0006) [2023-12-27 07:24:05,567][20347] Updated weights for policy 0, policy_version 253168 (0.0008) [2023-12-27 07:24:05,626][20347] Updated weights for policy 0, policy_version 253178 (0.0008) [2023-12-27 07:24:05,678][20347] Updated weights for policy 0, policy_version 253188 (0.0010) [2023-12-27 07:24:05,729][20381] Updated weights for policy 1, policy_version 252779 (0.0007) [2023-12-27 07:24:05,777][20381] Updated weights for policy 1, policy_version 252789 (0.0007) [2023-12-27 07:24:05,823][20381] Updated weights for policy 1, policy_version 252799 (0.0008) [2023-12-27 07:24:06,304][20347] Updated weights for policy 0, policy_version 253198 (0.0007) [2023-12-27 07:24:06,361][20347] Updated weights for policy 0, policy_version 253208 (0.0011) [2023-12-27 07:24:06,426][20347] Updated weights for policy 0, policy_version 253218 (0.0010) [2023-12-27 07:24:06,563][20381] Updated weights for policy 1, policy_version 252809 (0.0008) [2023-12-27 07:24:06,628][20381] Updated weights for policy 1, policy_version 252819 (0.0007) [2023-12-27 07:24:06,686][20381] Updated weights for policy 1, policy_version 252829 (0.0007) [2023-12-27 07:24:06,747][20381] Updated weights for policy 1, policy_version 252839 (0.0007) [2023-12-27 07:24:07,098][20347] Updated weights for policy 0, policy_version 253228 (0.0009) [2023-12-27 07:24:07,154][20347] Updated weights for policy 0, policy_version 253238 (0.0008) [2023-12-27 07:24:07,205][20347] Updated weights for policy 0, policy_version 253248 (0.0007) [2023-12-27 07:24:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 129589248. Throughput: 0: 10450.4, 1: 10390.3. Samples: 129557296. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:24:07,363][19170] Avg episode reward: [(0, '48.230'), (1, '980.940')] [2023-12-27 07:24:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000253256_64847872.pth... [2023-12-27 07:24:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000252008_64528384.pth [2023-12-27 07:24:07,404][20381] Updated weights for policy 1, policy_version 252849 (0.0006) [2023-12-27 07:24:07,450][20381] Updated weights for policy 1, policy_version 252859 (0.0005) [2023-12-27 07:24:07,499][20381] Updated weights for policy 1, policy_version 252869 (0.0005) [2023-12-27 07:24:07,511][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000252872_64749568.pth... [2023-12-27 07:24:07,514][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000251664_64438272.pth [2023-12-27 07:24:07,832][20347] Updated weights for policy 0, policy_version 253258 (0.0006) [2023-12-27 07:24:07,880][20347] Updated weights for policy 0, policy_version 253268 (0.0009) [2023-12-27 07:24:07,935][20347] Updated weights for policy 0, policy_version 253278 (0.0009) [2023-12-27 07:24:07,996][20347] Updated weights for policy 0, policy_version 253288 (0.0007) [2023-12-27 07:24:08,002][20381] Updated weights for policy 1, policy_version 252879 (0.0006) [2023-12-27 07:24:08,052][20381] Updated weights for policy 1, policy_version 252889 (0.0005) [2023-12-27 07:24:08,106][20381] Updated weights for policy 1, policy_version 252899 (0.0006) [2023-12-27 07:24:08,720][20347] Updated weights for policy 0, policy_version 253298 (0.0006) [2023-12-27 07:24:08,779][20347] Updated weights for policy 0, policy_version 253308 (0.0006) [2023-12-27 07:24:08,811][20381] Updated weights for policy 1, policy_version 252909 (0.0008) [2023-12-27 07:24:08,836][20347] Updated weights for policy 0, policy_version 253318 (0.0005) [2023-12-27 07:24:08,875][20381] Updated weights for policy 1, policy_version 252919 (0.0008) [2023-12-27 07:24:08,932][20381] Updated weights for policy 1, policy_version 252929 (0.0007) [2023-12-27 07:24:09,477][20347] Updated weights for policy 0, policy_version 253328 (0.0007) [2023-12-27 07:24:09,540][20347] Updated weights for policy 0, policy_version 253338 (0.0009) [2023-12-27 07:24:09,597][20347] Updated weights for policy 0, policy_version 253348 (0.0008) [2023-12-27 07:24:09,633][20381] Updated weights for policy 1, policy_version 252939 (0.0008) [2023-12-27 07:24:09,695][20381] Updated weights for policy 1, policy_version 252949 (0.0008) [2023-12-27 07:24:09,756][20381] Updated weights for policy 1, policy_version 252959 (0.0008) [2023-12-27 07:24:10,272][20347] Updated weights for policy 0, policy_version 253358 (0.0006) [2023-12-27 07:24:10,332][20347] Updated weights for policy 0, policy_version 253368 (0.0007) [2023-12-27 07:24:10,389][20347] Updated weights for policy 0, policy_version 253378 (0.0007) [2023-12-27 07:24:10,461][20381] Updated weights for policy 1, policy_version 252969 (0.0007) [2023-12-27 07:24:10,513][20381] Updated weights for policy 1, policy_version 252979 (0.0006) [2023-12-27 07:24:10,561][20381] Updated weights for policy 1, policy_version 252989 (0.0005) [2023-12-27 07:24:10,612][20381] Updated weights for policy 1, policy_version 252999 (0.0005) [2023-12-27 07:24:11,122][20347] Updated weights for policy 0, policy_version 253388 (0.0007) [2023-12-27 07:24:11,190][20347] Updated weights for policy 0, policy_version 253398 (0.0008) [2023-12-27 07:24:11,194][20381] Updated weights for policy 1, policy_version 253009 (0.0008) [2023-12-27 07:24:11,252][20347] Updated weights for policy 0, policy_version 253408 (0.0007) [2023-12-27 07:24:11,257][20381] Updated weights for policy 1, policy_version 253019 (0.0009) [2023-12-27 07:24:11,322][20381] Updated weights for policy 1, policy_version 253029 (0.0008) [2023-12-27 07:24:11,970][20347] Updated weights for policy 0, policy_version 253418 (0.0007) [2023-12-27 07:24:12,003][20381] Updated weights for policy 1, policy_version 253039 (0.0007) [2023-12-27 07:24:12,024][20347] Updated weights for policy 0, policy_version 253428 (0.0006) [2023-12-27 07:24:12,063][20381] Updated weights for policy 1, policy_version 253049 (0.0007) [2023-12-27 07:24:12,077][20347] Updated weights for policy 0, policy_version 253438 (0.0005) [2023-12-27 07:24:12,121][20381] Updated weights for policy 1, policy_version 253059 (0.0006) [2023-12-27 07:24:12,132][20347] Updated weights for policy 0, policy_version 253448 (0.0005) [2023-12-27 07:24:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 129695744. Throughput: 0: 10504.8, 1: 10345.6. Samples: 129682932. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:24:12,363][19170] Avg episode reward: [(0, '52.230'), (1, '990.380')] [2023-12-27 07:24:12,767][20347] Updated weights for policy 0, policy_version 253458 (0.0008) [2023-12-27 07:24:12,810][20381] Updated weights for policy 1, policy_version 253069 (0.0010) [2023-12-27 07:24:12,824][20347] Updated weights for policy 0, policy_version 253468 (0.0007) [2023-12-27 07:24:12,867][20381] Updated weights for policy 1, policy_version 253079 (0.0010) [2023-12-27 07:24:12,880][20347] Updated weights for policy 0, policy_version 253478 (0.0006) [2023-12-27 07:24:12,929][20381] Updated weights for policy 1, policy_version 253089 (0.0006) [2023-12-27 07:24:13,518][20347] Updated weights for policy 0, policy_version 253488 (0.0006) [2023-12-27 07:24:13,571][20347] Updated weights for policy 0, policy_version 253498 (0.0005) [2023-12-27 07:24:13,613][20381] Updated weights for policy 1, policy_version 253099 (0.0011) [2023-12-27 07:24:13,630][20347] Updated weights for policy 0, policy_version 253508 (0.0005) [2023-12-27 07:24:13,672][20381] Updated weights for policy 1, policy_version 253109 (0.0010) [2023-12-27 07:24:13,725][20381] Updated weights for policy 1, policy_version 253119 (0.0010) [2023-12-27 07:24:14,213][20347] Updated weights for policy 0, policy_version 253518 (0.0006) [2023-12-27 07:24:14,274][20347] Updated weights for policy 0, policy_version 253528 (0.0007) [2023-12-27 07:24:14,335][20347] Updated weights for policy 0, policy_version 253538 (0.0007) [2023-12-27 07:24:14,422][20381] Updated weights for policy 1, policy_version 253129 (0.0010) [2023-12-27 07:24:14,471][20381] Updated weights for policy 1, policy_version 253139 (0.0008) [2023-12-27 07:24:14,518][20381] Updated weights for policy 1, policy_version 253149 (0.0008) [2023-12-27 07:24:14,566][20381] Updated weights for policy 1, policy_version 253159 (0.0007) [2023-12-27 07:24:14,999][20347] Updated weights for policy 0, policy_version 253548 (0.0007) [2023-12-27 07:24:15,051][20347] Updated weights for policy 0, policy_version 253558 (0.0007) [2023-12-27 07:24:15,101][20347] Updated weights for policy 0, policy_version 253568 (0.0007) [2023-12-27 07:24:15,312][20381] Updated weights for policy 1, policy_version 253169 (0.0010) [2023-12-27 07:24:15,368][20381] Updated weights for policy 1, policy_version 253179 (0.0011) [2023-12-27 07:24:15,420][20381] Updated weights for policy 1, policy_version 253189 (0.0010) [2023-12-27 07:24:15,754][20347] Updated weights for policy 0, policy_version 253578 (0.0007) [2023-12-27 07:24:15,801][20347] Updated weights for policy 0, policy_version 253588 (0.0006) [2023-12-27 07:24:15,859][20347] Updated weights for policy 0, policy_version 253598 (0.0005) [2023-12-27 07:24:15,908][20347] Updated weights for policy 0, policy_version 253608 (0.0007) [2023-12-27 07:24:16,040][20381] Updated weights for policy 1, policy_version 253199 (0.0005) [2023-12-27 07:24:16,101][20381] Updated weights for policy 1, policy_version 253209 (0.0005) [2023-12-27 07:24:16,166][20381] Updated weights for policy 1, policy_version 253219 (0.0005) [2023-12-27 07:24:16,624][20347] Updated weights for policy 0, policy_version 253618 (0.0007) [2023-12-27 07:24:16,669][20347] Updated weights for policy 0, policy_version 253628 (0.0007) [2023-12-27 07:24:16,723][20347] Updated weights for policy 0, policy_version 253638 (0.0008) [2023-12-27 07:24:16,744][20381] Updated weights for policy 1, policy_version 253229 (0.0006) [2023-12-27 07:24:16,797][20381] Updated weights for policy 1, policy_version 253239 (0.0007) [2023-12-27 07:24:16,841][20381] Updated weights for policy 1, policy_version 253249 (0.0008) [2023-12-27 07:24:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 129794048. Throughput: 0: 10500.2, 1: 10321.7. Samples: 129808052. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:24:17,363][19170] Avg episode reward: [(0, '38.980'), (1, '1000.000')] [2023-12-27 07:24:17,425][20381] Updated weights for policy 1, policy_version 253259 (0.0007) [2023-12-27 07:24:17,467][20347] Updated weights for policy 0, policy_version 253648 (0.0008) [2023-12-27 07:24:17,477][20381] Updated weights for policy 1, policy_version 253269 (0.0005) [2023-12-27 07:24:17,520][20347] Updated weights for policy 0, policy_version 253658 (0.0007) [2023-12-27 07:24:17,531][20381] Updated weights for policy 1, policy_version 253279 (0.0005) [2023-12-27 07:24:17,570][20347] Updated weights for policy 0, policy_version 253668 (0.0007) [2023-12-27 07:24:18,183][20381] Updated weights for policy 1, policy_version 253289 (0.0006) [2023-12-27 07:24:18,248][20381] Updated weights for policy 1, policy_version 253299 (0.0008) [2023-12-27 07:24:18,274][20347] Updated weights for policy 0, policy_version 253678 (0.0009) [2023-12-27 07:24:18,302][20381] Updated weights for policy 1, policy_version 253309 (0.0006) [2023-12-27 07:24:18,321][20347] Updated weights for policy 0, policy_version 253688 (0.0010) [2023-12-27 07:24:18,358][20381] Updated weights for policy 1, policy_version 253319 (0.0008) [2023-12-27 07:24:18,378][20347] Updated weights for policy 0, policy_version 253698 (0.0008) [2023-12-27 07:24:18,983][20381] Updated weights for policy 1, policy_version 253329 (0.0010) [2023-12-27 07:24:19,039][20381] Updated weights for policy 1, policy_version 253339 (0.0011) [2023-12-27 07:24:19,096][20347] Updated weights for policy 0, policy_version 253708 (0.0007) [2023-12-27 07:24:19,101][20381] Updated weights for policy 1, policy_version 253349 (0.0011) [2023-12-27 07:24:19,153][20347] Updated weights for policy 0, policy_version 253718 (0.0007) [2023-12-27 07:24:19,208][20347] Updated weights for policy 0, policy_version 253728 (0.0008) [2023-12-27 07:24:19,752][20381] Updated weights for policy 1, policy_version 253359 (0.0008) [2023-12-27 07:24:19,809][20381] Updated weights for policy 1, policy_version 253369 (0.0006) [2023-12-27 07:24:19,875][20381] Updated weights for policy 1, policy_version 253379 (0.0007) [2023-12-27 07:24:19,965][20347] Updated weights for policy 0, policy_version 253738 (0.0008) [2023-12-27 07:24:20,019][20347] Updated weights for policy 0, policy_version 253748 (0.0007) [2023-12-27 07:24:20,071][20347] Updated weights for policy 0, policy_version 253758 (0.0007) [2023-12-27 07:24:20,134][20347] Updated weights for policy 0, policy_version 253768 (0.0009) [2023-12-27 07:24:20,503][20381] Updated weights for policy 1, policy_version 253389 (0.0008) [2023-12-27 07:24:20,571][20381] Updated weights for policy 1, policy_version 253399 (0.0008) [2023-12-27 07:24:20,635][20381] Updated weights for policy 1, policy_version 253409 (0.0008) [2023-12-27 07:24:20,884][20347] Updated weights for policy 0, policy_version 253778 (0.0008) [2023-12-27 07:24:20,940][20347] Updated weights for policy 0, policy_version 253788 (0.0008) [2023-12-27 07:24:20,993][20347] Updated weights for policy 0, policy_version 253798 (0.0008) [2023-12-27 07:24:21,292][20381] Updated weights for policy 1, policy_version 253419 (0.0008) [2023-12-27 07:24:21,355][20381] Updated weights for policy 1, policy_version 253429 (0.0009) [2023-12-27 07:24:21,423][20381] Updated weights for policy 1, policy_version 253439 (0.0008) [2023-12-27 07:24:21,734][20347] Updated weights for policy 0, policy_version 253808 (0.0007) [2023-12-27 07:24:21,785][20347] Updated weights for policy 0, policy_version 253818 (0.0008) [2023-12-27 07:24:21,846][20347] Updated weights for policy 0, policy_version 253828 (0.0008) [2023-12-27 07:24:22,154][20381] Updated weights for policy 1, policy_version 253449 (0.0008) [2023-12-27 07:24:22,211][20381] Updated weights for policy 1, policy_version 253459 (0.0008) [2023-12-27 07:24:22,274][20381] Updated weights for policy 1, policy_version 253469 (0.0008) [2023-12-27 07:24:22,324][20381] Updated weights for policy 1, policy_version 253479 (0.0008) [2023-12-27 07:24:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 129900544. Throughput: 0: 10394.3, 1: 10368.1. Samples: 129869260. Policy #0 lag: (min: 27.0, avg: 39.4, max: 59.0) [2023-12-27 07:24:22,363][19170] Avg episode reward: [(0, '30.320'), (1, '1000.000')] [2023-12-27 07:24:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000253480_64905216.pth... [2023-12-27 07:24:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000253832_64995328.pth... [2023-12-27 07:24:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000252272_64593920.pth [2023-12-27 07:24:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000252616_64684032.pth [2023-12-27 07:24:22,570][20347] Updated weights for policy 0, policy_version 253838 (0.0008) [2023-12-27 07:24:22,635][20347] Updated weights for policy 0, policy_version 253848 (0.0008) [2023-12-27 07:24:22,692][20347] Updated weights for policy 0, policy_version 253858 (0.0008) [2023-12-27 07:24:23,004][20381] Updated weights for policy 1, policy_version 253489 (0.0005) [2023-12-27 07:24:23,067][20381] Updated weights for policy 1, policy_version 253499 (0.0006) [2023-12-27 07:24:23,130][20381] Updated weights for policy 1, policy_version 253509 (0.0005) [2023-12-27 07:24:23,409][20347] Updated weights for policy 0, policy_version 253868 (0.0007) [2023-12-27 07:24:23,461][20347] Updated weights for policy 0, policy_version 253878 (0.0007) [2023-12-27 07:24:23,511][20347] Updated weights for policy 0, policy_version 253888 (0.0008) [2023-12-27 07:24:23,760][20381] Updated weights for policy 1, policy_version 253519 (0.0008) [2023-12-27 07:24:23,819][20381] Updated weights for policy 1, policy_version 253529 (0.0009) [2023-12-27 07:24:23,875][20381] Updated weights for policy 1, policy_version 253539 (0.0007) [2023-12-27 07:24:24,125][20347] Updated weights for policy 0, policy_version 253898 (0.0007) [2023-12-27 07:24:24,184][20347] Updated weights for policy 0, policy_version 253908 (0.0009) [2023-12-27 07:24:24,232][20347] Updated weights for policy 0, policy_version 253918 (0.0010) [2023-12-27 07:24:24,278][20347] Updated weights for policy 0, policy_version 253928 (0.0010) [2023-12-27 07:24:24,560][20381] Updated weights for policy 1, policy_version 253549 (0.0009) [2023-12-27 07:24:24,615][20381] Updated weights for policy 1, policy_version 253559 (0.0011) [2023-12-27 07:24:24,666][20381] Updated weights for policy 1, policy_version 253569 (0.0008) [2023-12-27 07:24:24,942][20347] Updated weights for policy 0, policy_version 253938 (0.0007) [2023-12-27 07:24:24,995][20347] Updated weights for policy 0, policy_version 253948 (0.0008) [2023-12-27 07:24:25,046][20347] Updated weights for policy 0, policy_version 253958 (0.0007) [2023-12-27 07:24:25,308][20381] Updated weights for policy 1, policy_version 253579 (0.0006) [2023-12-27 07:24:25,363][20381] Updated weights for policy 1, policy_version 253589 (0.0009) [2023-12-27 07:24:25,415][20381] Updated weights for policy 1, policy_version 253599 (0.0008) [2023-12-27 07:24:25,742][20347] Updated weights for policy 0, policy_version 253968 (0.0009) [2023-12-27 07:24:25,794][20347] Updated weights for policy 0, policy_version 253978 (0.0010) [2023-12-27 07:24:25,842][20347] Updated weights for policy 0, policy_version 253988 (0.0010) [2023-12-27 07:24:25,989][20381] Updated weights for policy 1, policy_version 253609 (0.0008) [2023-12-27 07:24:26,040][20381] Updated weights for policy 1, policy_version 253619 (0.0010) [2023-12-27 07:24:26,096][20381] Updated weights for policy 1, policy_version 253629 (0.0011) [2023-12-27 07:24:26,155][20381] Updated weights for policy 1, policy_version 253639 (0.0007) [2023-12-27 07:24:26,544][20347] Updated weights for policy 0, policy_version 253998 (0.0010) [2023-12-27 07:24:26,594][20347] Updated weights for policy 0, policy_version 254008 (0.0010) [2023-12-27 07:24:26,638][20347] Updated weights for policy 0, policy_version 254018 (0.0010) [2023-12-27 07:24:26,757][20381] Updated weights for policy 1, policy_version 253649 (0.0005) [2023-12-27 07:24:26,812][20381] Updated weights for policy 1, policy_version 253659 (0.0006) [2023-12-27 07:24:26,865][20381] Updated weights for policy 1, policy_version 253669 (0.0010) [2023-12-27 07:24:27,306][20347] Updated weights for policy 0, policy_version 254028 (0.0010) [2023-12-27 07:24:27,358][20347] Updated weights for policy 0, policy_version 254038 (0.0010) [2023-12-27 07:24:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 129998848. Throughput: 0: 10368.4, 1: 10428.0. Samples: 129992036. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:27,363][19170] Avg episode reward: [(0, '44.780'), (1, '1000.000')] [2023-12-27 07:24:27,409][20347] Updated weights for policy 0, policy_version 254048 (0.0010) [2023-12-27 07:24:27,447][20381] Updated weights for policy 1, policy_version 253679 (0.0010) [2023-12-27 07:24:27,494][20381] Updated weights for policy 1, policy_version 253689 (0.0010) [2023-12-27 07:24:27,542][20381] Updated weights for policy 1, policy_version 253699 (0.0010) [2023-12-27 07:24:28,093][20347] Updated weights for policy 0, policy_version 254058 (0.0010) [2023-12-27 07:24:28,152][20347] Updated weights for policy 0, policy_version 254068 (0.0010) [2023-12-27 07:24:28,178][20381] Updated weights for policy 1, policy_version 253709 (0.0008) [2023-12-27 07:24:28,207][20347] Updated weights for policy 0, policy_version 254078 (0.0010) [2023-12-27 07:24:28,233][20381] Updated weights for policy 1, policy_version 253719 (0.0009) [2023-12-27 07:24:28,264][20347] Updated weights for policy 0, policy_version 254088 (0.0010) [2023-12-27 07:24:28,285][20381] Updated weights for policy 1, policy_version 253729 (0.0009) [2023-12-27 07:24:28,904][20381] Updated weights for policy 1, policy_version 253739 (0.0006) [2023-12-27 07:24:28,925][20347] Updated weights for policy 0, policy_version 254098 (0.0005) [2023-12-27 07:24:28,959][20381] Updated weights for policy 1, policy_version 253749 (0.0010) [2023-12-27 07:24:28,972][20347] Updated weights for policy 0, policy_version 254108 (0.0008) [2023-12-27 07:24:29,010][20381] Updated weights for policy 1, policy_version 253759 (0.0010) [2023-12-27 07:24:29,021][20347] Updated weights for policy 0, policy_version 254118 (0.0010) [2023-12-27 07:24:29,690][20381] Updated weights for policy 1, policy_version 253769 (0.0010) [2023-12-27 07:24:29,733][20347] Updated weights for policy 0, policy_version 254128 (0.0008) [2023-12-27 07:24:29,738][20381] Updated weights for policy 1, policy_version 253779 (0.0010) [2023-12-27 07:24:29,787][20381] Updated weights for policy 1, policy_version 253789 (0.0010) [2023-12-27 07:24:29,797][20347] Updated weights for policy 0, policy_version 254138 (0.0007) [2023-12-27 07:24:29,846][20381] Updated weights for policy 1, policy_version 253799 (0.0010) [2023-12-27 07:24:29,859][20347] Updated weights for policy 0, policy_version 254148 (0.0007) [2023-12-27 07:24:30,559][20381] Updated weights for policy 1, policy_version 253809 (0.0008) [2023-12-27 07:24:30,579][20347] Updated weights for policy 0, policy_version 254158 (0.0007) [2023-12-27 07:24:30,619][20381] Updated weights for policy 1, policy_version 253819 (0.0008) [2023-12-27 07:24:30,642][20347] Updated weights for policy 0, policy_version 254168 (0.0006) [2023-12-27 07:24:30,679][20381] Updated weights for policy 1, policy_version 253829 (0.0007) [2023-12-27 07:24:30,691][20347] Updated weights for policy 0, policy_version 254178 (0.0007) [2023-12-27 07:24:31,280][20347] Updated weights for policy 0, policy_version 254188 (0.0007) [2023-12-27 07:24:31,330][20347] Updated weights for policy 0, policy_version 254198 (0.0010) [2023-12-27 07:24:31,388][20347] Updated weights for policy 0, policy_version 254208 (0.0007) [2023-12-27 07:24:31,398][20381] Updated weights for policy 1, policy_version 253839 (0.0009) [2023-12-27 07:24:31,455][20381] Updated weights for policy 1, policy_version 253849 (0.0007) [2023-12-27 07:24:31,511][20381] Updated weights for policy 1, policy_version 253859 (0.0007) [2023-12-27 07:24:32,059][20347] Updated weights for policy 0, policy_version 254218 (0.0010) [2023-12-27 07:24:32,124][20347] Updated weights for policy 0, policy_version 254228 (0.0010) [2023-12-27 07:24:32,186][20347] Updated weights for policy 0, policy_version 254238 (0.0011) [2023-12-27 07:24:32,233][20381] Updated weights for policy 1, policy_version 253869 (0.0007) [2023-12-27 07:24:32,248][20347] Updated weights for policy 0, policy_version 254248 (0.0010) [2023-12-27 07:24:32,296][20381] Updated weights for policy 1, policy_version 253879 (0.0008) [2023-12-27 07:24:32,346][20381] Updated weights for policy 1, policy_version 253889 (0.0008) [2023-12-27 07:24:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 130105344. Throughput: 0: 10395.0, 1: 10453.3. Samples: 130117284. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:32,363][19170] Avg episode reward: [(0, '56.220'), (1, '1000.000')] [2023-12-27 07:24:32,904][20347] Updated weights for policy 0, policy_version 254258 (0.0010) [2023-12-27 07:24:32,959][20347] Updated weights for policy 0, policy_version 254268 (0.0010) [2023-12-27 07:24:33,013][20347] Updated weights for policy 0, policy_version 254278 (0.0010) [2023-12-27 07:24:33,058][20381] Updated weights for policy 1, policy_version 253899 (0.0008) [2023-12-27 07:24:33,106][20381] Updated weights for policy 1, policy_version 253909 (0.0007) [2023-12-27 07:24:33,150][20381] Updated weights for policy 1, policy_version 253919 (0.0007) [2023-12-27 07:24:33,635][20347] Updated weights for policy 0, policy_version 254288 (0.0009) [2023-12-27 07:24:33,696][20347] Updated weights for policy 0, policy_version 254298 (0.0007) [2023-12-27 07:24:33,754][20347] Updated weights for policy 0, policy_version 254308 (0.0009) [2023-12-27 07:24:33,787][20381] Updated weights for policy 1, policy_version 253929 (0.0007) [2023-12-27 07:24:33,840][20381] Updated weights for policy 1, policy_version 253939 (0.0005) [2023-12-27 07:24:33,897][20381] Updated weights for policy 1, policy_version 253949 (0.0008) [2023-12-27 07:24:33,955][20381] Updated weights for policy 1, policy_version 253959 (0.0006) [2023-12-27 07:24:34,421][20347] Updated weights for policy 0, policy_version 254318 (0.0006) [2023-12-27 07:24:34,476][20347] Updated weights for policy 0, policy_version 254328 (0.0006) [2023-12-27 07:24:34,534][20347] Updated weights for policy 0, policy_version 254338 (0.0008) [2023-12-27 07:24:34,647][20381] Updated weights for policy 1, policy_version 253969 (0.0007) [2023-12-27 07:24:34,713][20381] Updated weights for policy 1, policy_version 253979 (0.0008) [2023-12-27 07:24:34,775][20381] Updated weights for policy 1, policy_version 253989 (0.0008) [2023-12-27 07:24:35,139][20347] Updated weights for policy 0, policy_version 254348 (0.0007) [2023-12-27 07:24:35,194][20347] Updated weights for policy 0, policy_version 254358 (0.0008) [2023-12-27 07:24:35,238][20347] Updated weights for policy 0, policy_version 254368 (0.0007) [2023-12-27 07:24:35,422][20381] Updated weights for policy 1, policy_version 253999 (0.0008) [2023-12-27 07:24:35,468][20381] Updated weights for policy 1, policy_version 254009 (0.0010) [2023-12-27 07:24:35,526][20381] Updated weights for policy 1, policy_version 254019 (0.0008) [2023-12-27 07:24:35,823][20347] Updated weights for policy 0, policy_version 254378 (0.0007) [2023-12-27 07:24:35,877][20347] Updated weights for policy 0, policy_version 254388 (0.0006) [2023-12-27 07:24:35,934][20347] Updated weights for policy 0, policy_version 254398 (0.0007) [2023-12-27 07:24:35,989][20347] Updated weights for policy 0, policy_version 254408 (0.0005) [2023-12-27 07:24:36,277][20381] Updated weights for policy 1, policy_version 254029 (0.0007) [2023-12-27 07:24:36,337][20381] Updated weights for policy 1, policy_version 254039 (0.0005) [2023-12-27 07:24:36,394][20381] Updated weights for policy 1, policy_version 254049 (0.0009) [2023-12-27 07:24:36,604][20347] Updated weights for policy 0, policy_version 254418 (0.0007) [2023-12-27 07:24:36,658][20347] Updated weights for policy 0, policy_version 254428 (0.0008) [2023-12-27 07:24:36,706][20347] Updated weights for policy 0, policy_version 254438 (0.0008) [2023-12-27 07:24:37,054][20381] Updated weights for policy 1, policy_version 254059 (0.0009) [2023-12-27 07:24:37,117][20381] Updated weights for policy 1, policy_version 254069 (0.0006) [2023-12-27 07:24:37,181][20381] Updated weights for policy 1, policy_version 254079 (0.0006) [2023-12-27 07:24:37,312][20347] Updated weights for policy 0, policy_version 254448 (0.0006) [2023-12-27 07:24:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 130211840. Throughput: 0: 10394.1, 1: 10458.8. Samples: 130180208. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:37,363][19170] Avg episode reward: [(0, '53.800'), (1, '1000.000')] [2023-12-27 07:24:37,367][20347] Updated weights for policy 0, policy_version 254458 (0.0007) [2023-12-27 07:24:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000254088_65060864.pth... [2023-12-27 07:24:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000252872_64749568.pth [2023-12-27 07:24:37,418][20347] Updated weights for policy 0, policy_version 254468 (0.0007) [2023-12-27 07:24:37,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000254472_65159168.pth... [2023-12-27 07:24:37,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000253256_64847872.pth [2023-12-27 07:24:37,768][20381] Updated weights for policy 1, policy_version 254089 (0.0005) [2023-12-27 07:24:37,819][20381] Updated weights for policy 1, policy_version 254099 (0.0007) [2023-12-27 07:24:37,861][20381] Updated weights for policy 1, policy_version 254109 (0.0007) [2023-12-27 07:24:37,911][20381] Updated weights for policy 1, policy_version 254119 (0.0009) [2023-12-27 07:24:38,128][20347] Updated weights for policy 0, policy_version 254478 (0.0008) [2023-12-27 07:24:38,180][20347] Updated weights for policy 0, policy_version 254488 (0.0008) [2023-12-27 07:24:38,234][20347] Updated weights for policy 0, policy_version 254498 (0.0007) [2023-12-27 07:24:38,537][20381] Updated weights for policy 1, policy_version 254129 (0.0007) [2023-12-27 07:24:38,601][20381] Updated weights for policy 1, policy_version 254139 (0.0010) [2023-12-27 07:24:38,656][20381] Updated weights for policy 1, policy_version 254149 (0.0010) [2023-12-27 07:24:38,927][20347] Updated weights for policy 0, policy_version 254508 (0.0011) [2023-12-27 07:24:38,975][20347] Updated weights for policy 0, policy_version 254518 (0.0010) [2023-12-27 07:24:39,027][20347] Updated weights for policy 0, policy_version 254528 (0.0010) [2023-12-27 07:24:39,305][20381] Updated weights for policy 1, policy_version 254159 (0.0007) [2023-12-27 07:24:39,365][20381] Updated weights for policy 1, policy_version 254169 (0.0009) [2023-12-27 07:24:39,431][20381] Updated weights for policy 1, policy_version 254179 (0.0008) [2023-12-27 07:24:39,715][20347] Updated weights for policy 0, policy_version 254538 (0.0009) [2023-12-27 07:24:39,783][20347] Updated weights for policy 0, policy_version 254548 (0.0007) [2023-12-27 07:24:39,849][20347] Updated weights for policy 0, policy_version 254558 (0.0009) [2023-12-27 07:24:39,903][20347] Updated weights for policy 0, policy_version 254568 (0.0007) [2023-12-27 07:24:40,151][20381] Updated weights for policy 1, policy_version 254189 (0.0009) [2023-12-27 07:24:40,217][20381] Updated weights for policy 1, policy_version 254199 (0.0008) [2023-12-27 07:24:40,276][20381] Updated weights for policy 1, policy_version 254209 (0.0008) [2023-12-27 07:24:40,534][20347] Updated weights for policy 0, policy_version 254578 (0.0008) [2023-12-27 07:24:40,587][20347] Updated weights for policy 0, policy_version 254588 (0.0008) [2023-12-27 07:24:40,646][20347] Updated weights for policy 0, policy_version 254598 (0.0008) [2023-12-27 07:24:40,916][20381] Updated weights for policy 1, policy_version 254219 (0.0008) [2023-12-27 07:24:40,969][20381] Updated weights for policy 1, policy_version 254229 (0.0011) [2023-12-27 07:24:41,027][20381] Updated weights for policy 1, policy_version 254239 (0.0007) [2023-12-27 07:24:41,391][20347] Updated weights for policy 0, policy_version 254608 (0.0008) [2023-12-27 07:24:41,451][20347] Updated weights for policy 0, policy_version 254618 (0.0007) [2023-12-27 07:24:41,513][20347] Updated weights for policy 0, policy_version 254628 (0.0007) [2023-12-27 07:24:41,747][20381] Updated weights for policy 1, policy_version 254249 (0.0011) [2023-12-27 07:24:41,810][20381] Updated weights for policy 1, policy_version 254259 (0.0008) [2023-12-27 07:24:41,873][20381] Updated weights for policy 1, policy_version 254269 (0.0008) [2023-12-27 07:24:41,933][20381] Updated weights for policy 1, policy_version 254279 (0.0007) [2023-12-27 07:24:42,272][20347] Updated weights for policy 0, policy_version 254638 (0.0007) [2023-12-27 07:24:42,335][20347] Updated weights for policy 0, policy_version 254648 (0.0007) [2023-12-27 07:24:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 130310144. Throughput: 0: 10335.4, 1: 10516.1. Samples: 130305976. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:42,363][19170] Avg episode reward: [(0, '86.200'), (1, '1000.000')] [2023-12-27 07:24:42,391][20347] Updated weights for policy 0, policy_version 254658 (0.0007) [2023-12-27 07:24:42,608][20381] Updated weights for policy 1, policy_version 254289 (0.0006) [2023-12-27 07:24:42,669][20381] Updated weights for policy 1, policy_version 254299 (0.0010) [2023-12-27 07:24:42,733][20381] Updated weights for policy 1, policy_version 254309 (0.0011) [2023-12-27 07:24:43,104][20347] Updated weights for policy 0, policy_version 254668 (0.0007) [2023-12-27 07:24:43,154][20347] Updated weights for policy 0, policy_version 254678 (0.0006) [2023-12-27 07:24:43,210][20347] Updated weights for policy 0, policy_version 254688 (0.0007) [2023-12-27 07:24:43,368][20381] Updated weights for policy 1, policy_version 254319 (0.0010) [2023-12-27 07:24:43,413][20381] Updated weights for policy 1, policy_version 254329 (0.0010) [2023-12-27 07:24:43,461][20381] Updated weights for policy 1, policy_version 254339 (0.0010) [2023-12-27 07:24:43,823][20347] Updated weights for policy 0, policy_version 254698 (0.0007) [2023-12-27 07:24:43,879][20347] Updated weights for policy 0, policy_version 254708 (0.0005) [2023-12-27 07:24:43,941][20347] Updated weights for policy 0, policy_version 254718 (0.0005) [2023-12-27 07:24:44,004][20347] Updated weights for policy 0, policy_version 254728 (0.0005) [2023-12-27 07:24:44,172][20381] Updated weights for policy 1, policy_version 254349 (0.0010) [2023-12-27 07:24:44,233][20381] Updated weights for policy 1, policy_version 254359 (0.0009) [2023-12-27 07:24:44,292][20381] Updated weights for policy 1, policy_version 254369 (0.0007) [2023-12-27 07:24:44,524][20347] Updated weights for policy 0, policy_version 254738 (0.0007) [2023-12-27 07:24:44,584][20347] Updated weights for policy 0, policy_version 254748 (0.0006) [2023-12-27 07:24:44,651][20347] Updated weights for policy 0, policy_version 254758 (0.0006) [2023-12-27 07:24:44,969][20381] Updated weights for policy 1, policy_version 254379 (0.0008) [2023-12-27 07:24:45,030][20381] Updated weights for policy 1, policy_version 254389 (0.0011) [2023-12-27 07:24:45,089][20381] Updated weights for policy 1, policy_version 254399 (0.0011) [2023-12-27 07:24:45,335][20347] Updated weights for policy 0, policy_version 254768 (0.0005) [2023-12-27 07:24:45,384][20347] Updated weights for policy 0, policy_version 254778 (0.0006) [2023-12-27 07:24:45,436][20347] Updated weights for policy 0, policy_version 254788 (0.0007) [2023-12-27 07:24:45,774][20381] Updated weights for policy 1, policy_version 254409 (0.0011) [2023-12-27 07:24:45,834][20381] Updated weights for policy 1, policy_version 254419 (0.0010) [2023-12-27 07:24:45,891][20381] Updated weights for policy 1, policy_version 254429 (0.0011) [2023-12-27 07:24:45,947][20381] Updated weights for policy 1, policy_version 254439 (0.0011) [2023-12-27 07:24:46,112][20347] Updated weights for policy 0, policy_version 254798 (0.0007) [2023-12-27 07:24:46,169][20347] Updated weights for policy 0, policy_version 254808 (0.0006) [2023-12-27 07:24:46,221][20347] Updated weights for policy 0, policy_version 254818 (0.0005) [2023-12-27 07:24:46,643][20381] Updated weights for policy 1, policy_version 254449 (0.0011) [2023-12-27 07:24:46,703][20381] Updated weights for policy 1, policy_version 254459 (0.0011) [2023-12-27 07:24:46,756][20381] Updated weights for policy 1, policy_version 254469 (0.0011) [2023-12-27 07:24:46,874][20347] Updated weights for policy 0, policy_version 254828 (0.0006) [2023-12-27 07:24:46,933][20347] Updated weights for policy 0, policy_version 254838 (0.0007) [2023-12-27 07:24:46,986][20347] Updated weights for policy 0, policy_version 254848 (0.0007) [2023-12-27 07:24:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 130416640. Throughput: 0: 10392.8, 1: 10471.9. Samples: 130429228. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:47,363][19170] Avg episode reward: [(0, '120.520'), (1, '1000.000')] [2023-12-27 07:24:47,425][20381] Updated weights for policy 1, policy_version 254479 (0.0008) [2023-12-27 07:24:47,475][20381] Updated weights for policy 1, policy_version 254489 (0.0005) [2023-12-27 07:24:47,529][20381] Updated weights for policy 1, policy_version 254499 (0.0005) [2023-12-27 07:24:47,580][20347] Updated weights for policy 0, policy_version 254858 (0.0008) [2023-12-27 07:24:47,636][20347] Updated weights for policy 0, policy_version 254868 (0.0008) [2023-12-27 07:24:47,691][20347] Updated weights for policy 0, policy_version 254878 (0.0007) [2023-12-27 07:24:47,748][20347] Updated weights for policy 0, policy_version 254888 (0.0005) [2023-12-27 07:24:48,171][20381] Updated weights for policy 1, policy_version 254509 (0.0011) [2023-12-27 07:24:48,219][20381] Updated weights for policy 1, policy_version 254519 (0.0010) [2023-12-27 07:24:48,265][20381] Updated weights for policy 1, policy_version 254529 (0.0010) [2023-12-27 07:24:48,303][20347] Updated weights for policy 0, policy_version 254898 (0.0006) [2023-12-27 07:24:48,363][20347] Updated weights for policy 0, policy_version 254908 (0.0008) [2023-12-27 07:24:48,415][20347] Updated weights for policy 0, policy_version 254918 (0.0007) [2023-12-27 07:24:48,937][20381] Updated weights for policy 1, policy_version 254539 (0.0010) [2023-12-27 07:24:48,995][20381] Updated weights for policy 1, policy_version 254549 (0.0009) [2023-12-27 07:24:49,053][20381] Updated weights for policy 1, policy_version 254559 (0.0005) [2023-12-27 07:24:49,089][20347] Updated weights for policy 0, policy_version 254928 (0.0010) [2023-12-27 07:24:49,153][20347] Updated weights for policy 0, policy_version 254938 (0.0010) [2023-12-27 07:24:49,197][20347] Updated weights for policy 0, policy_version 254948 (0.0010) [2023-12-27 07:24:49,713][20381] Updated weights for policy 1, policy_version 254569 (0.0006) [2023-12-27 07:24:49,765][20381] Updated weights for policy 1, policy_version 254579 (0.0011) [2023-12-27 07:24:49,815][20381] Updated weights for policy 1, policy_version 254589 (0.0011) [2023-12-27 07:24:49,865][20347] Updated weights for policy 0, policy_version 254958 (0.0008) [2023-12-27 07:24:49,876][20381] Updated weights for policy 1, policy_version 254599 (0.0011) [2023-12-27 07:24:49,932][20347] Updated weights for policy 0, policy_version 254968 (0.0008) [2023-12-27 07:24:49,992][20347] Updated weights for policy 0, policy_version 254978 (0.0008) [2023-12-27 07:24:50,592][20381] Updated weights for policy 1, policy_version 254609 (0.0010) [2023-12-27 07:24:50,641][20381] Updated weights for policy 1, policy_version 254619 (0.0008) [2023-12-27 07:24:50,697][20381] Updated weights for policy 1, policy_version 254629 (0.0008) [2023-12-27 07:24:50,698][20347] Updated weights for policy 0, policy_version 254988 (0.0009) [2023-12-27 07:24:50,745][20347] Updated weights for policy 0, policy_version 254998 (0.0008) [2023-12-27 07:24:50,797][20347] Updated weights for policy 0, policy_version 255008 (0.0008) [2023-12-27 07:24:51,416][20381] Updated weights for policy 1, policy_version 254639 (0.0006) [2023-12-27 07:24:51,465][20381] Updated weights for policy 1, policy_version 254649 (0.0005) [2023-12-27 07:24:51,494][20347] Updated weights for policy 0, policy_version 255018 (0.0006) [2023-12-27 07:24:51,524][20381] Updated weights for policy 1, policy_version 254659 (0.0005) [2023-12-27 07:24:51,554][20347] Updated weights for policy 0, policy_version 255028 (0.0008) [2023-12-27 07:24:51,606][20347] Updated weights for policy 0, policy_version 255038 (0.0010) [2023-12-27 07:24:51,668][20347] Updated weights for policy 0, policy_version 255048 (0.0007) [2023-12-27 07:24:52,170][20381] Updated weights for policy 1, policy_version 254669 (0.0006) [2023-12-27 07:24:52,229][20381] Updated weights for policy 1, policy_version 254679 (0.0008) [2023-12-27 07:24:52,285][20381] Updated weights for policy 1, policy_version 254689 (0.0008) [2023-12-27 07:24:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 130523136. Throughput: 0: 10378.6, 1: 10425.6. Samples: 130493488. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:52,363][19170] Avg episode reward: [(0, '656.070'), (1, '1000.000')] [2023-12-27 07:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000254696_65216512.pth... [2023-12-27 07:24:52,370][20347] Updated weights for policy 0, policy_version 255058 (0.0007) [2023-12-27 07:24:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000253480_64905216.pth [2023-12-27 07:24:52,432][20347] Updated weights for policy 0, policy_version 255068 (0.0007) [2023-12-27 07:24:52,493][20347] Updated weights for policy 0, policy_version 255078 (0.0007) [2023-12-27 07:24:52,502][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000255080_65314816.pth... [2023-12-27 07:24:52,506][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000253832_64995328.pth [2023-12-27 07:24:52,926][20381] Updated weights for policy 1, policy_version 254699 (0.0008) [2023-12-27 07:24:52,990][20381] Updated weights for policy 1, policy_version 254709 (0.0005) [2023-12-27 07:24:53,051][20381] Updated weights for policy 1, policy_version 254719 (0.0005) [2023-12-27 07:24:53,126][20347] Updated weights for policy 0, policy_version 255088 (0.0007) [2023-12-27 07:24:53,184][20347] Updated weights for policy 0, policy_version 255098 (0.0006) [2023-12-27 07:24:53,236][20347] Updated weights for policy 0, policy_version 255108 (0.0007) [2023-12-27 07:24:53,656][20381] Updated weights for policy 1, policy_version 254729 (0.0005) [2023-12-27 07:24:53,713][20381] Updated weights for policy 1, policy_version 254739 (0.0008) [2023-12-27 07:24:53,776][20381] Updated weights for policy 1, policy_version 254749 (0.0008) [2023-12-27 07:24:53,821][20381] Updated weights for policy 1, policy_version 254759 (0.0008) [2023-12-27 07:24:53,919][20347] Updated weights for policy 0, policy_version 255118 (0.0007) [2023-12-27 07:24:53,976][20347] Updated weights for policy 0, policy_version 255128 (0.0007) [2023-12-27 07:24:54,034][20347] Updated weights for policy 0, policy_version 255138 (0.0008) [2023-12-27 07:24:54,460][20381] Updated weights for policy 1, policy_version 254769 (0.0006) [2023-12-27 07:24:54,510][20381] Updated weights for policy 1, policy_version 254779 (0.0008) [2023-12-27 07:24:54,554][20381] Updated weights for policy 1, policy_version 254789 (0.0007) [2023-12-27 07:24:54,735][20347] Updated weights for policy 0, policy_version 255148 (0.0007) [2023-12-27 07:24:54,780][20347] Updated weights for policy 0, policy_version 255158 (0.0006) [2023-12-27 07:24:54,835][20347] Updated weights for policy 0, policy_version 255168 (0.0005) [2023-12-27 07:24:55,166][20381] Updated weights for policy 1, policy_version 254799 (0.0007) [2023-12-27 07:24:55,227][20381] Updated weights for policy 1, policy_version 254809 (0.0007) [2023-12-27 07:24:55,284][20381] Updated weights for policy 1, policy_version 254819 (0.0007) [2023-12-27 07:24:55,516][20347] Updated weights for policy 0, policy_version 255178 (0.0006) [2023-12-27 07:24:55,579][20347] Updated weights for policy 0, policy_version 255188 (0.0010) [2023-12-27 07:24:55,637][20347] Updated weights for policy 0, policy_version 255198 (0.0010) [2023-12-27 07:24:55,694][20347] Updated weights for policy 0, policy_version 255208 (0.0009) [2023-12-27 07:24:55,923][20381] Updated weights for policy 1, policy_version 254829 (0.0008) [2023-12-27 07:24:55,981][20381] Updated weights for policy 1, policy_version 254839 (0.0010) [2023-12-27 07:24:56,028][20381] Updated weights for policy 1, policy_version 254849 (0.0010) [2023-12-27 07:24:56,256][20347] Updated weights for policy 0, policy_version 255218 (0.0007) [2023-12-27 07:24:56,312][20347] Updated weights for policy 0, policy_version 255228 (0.0007) [2023-12-27 07:24:56,368][20347] Updated weights for policy 0, policy_version 255238 (0.0007) [2023-12-27 07:24:56,687][20381] Updated weights for policy 1, policy_version 254859 (0.0010) [2023-12-27 07:24:56,741][20381] Updated weights for policy 1, policy_version 254869 (0.0010) [2023-12-27 07:24:56,789][20381] Updated weights for policy 1, policy_version 254879 (0.0010) [2023-12-27 07:24:56,997][20347] Updated weights for policy 0, policy_version 255248 (0.0008) [2023-12-27 07:24:57,044][20347] Updated weights for policy 0, policy_version 255258 (0.0008) [2023-12-27 07:24:57,094][20347] Updated weights for policy 0, policy_version 255268 (0.0006) [2023-12-27 07:24:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 130629632. Throughput: 0: 10358.2, 1: 10458.4. Samples: 130619680. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:24:57,363][19170] Avg episode reward: [(0, '845.320'), (1, '1000.000')] [2023-12-27 07:24:57,383][20381] Updated weights for policy 1, policy_version 254889 (0.0010) [2023-12-27 07:24:57,428][20381] Updated weights for policy 1, policy_version 254899 (0.0007) [2023-12-27 07:24:57,471][20381] Updated weights for policy 1, policy_version 254909 (0.0008) [2023-12-27 07:24:57,519][20381] Updated weights for policy 1, policy_version 254919 (0.0005) [2023-12-27 07:24:57,777][20347] Updated weights for policy 0, policy_version 255278 (0.0007) [2023-12-27 07:24:57,830][20347] Updated weights for policy 0, policy_version 255288 (0.0010) [2023-12-27 07:24:57,881][20347] Updated weights for policy 0, policy_version 255298 (0.0008) [2023-12-27 07:24:58,062][20381] Updated weights for policy 1, policy_version 254929 (0.0005) [2023-12-27 07:24:58,113][20381] Updated weights for policy 1, policy_version 254939 (0.0007) [2023-12-27 07:24:58,176][20381] Updated weights for policy 1, policy_version 254949 (0.0007) [2023-12-27 07:24:58,624][20347] Updated weights for policy 0, policy_version 255308 (0.0008) [2023-12-27 07:24:58,681][20347] Updated weights for policy 0, policy_version 255318 (0.0007) [2023-12-27 07:24:58,739][20347] Updated weights for policy 0, policy_version 255328 (0.0007) [2023-12-27 07:24:58,822][20381] Updated weights for policy 1, policy_version 254959 (0.0007) [2023-12-27 07:24:58,883][20381] Updated weights for policy 1, policy_version 254969 (0.0011) [2023-12-27 07:24:58,943][20381] Updated weights for policy 1, policy_version 254979 (0.0010) [2023-12-27 07:24:59,413][20347] Updated weights for policy 0, policy_version 255338 (0.0007) [2023-12-27 07:24:59,471][20347] Updated weights for policy 0, policy_version 255348 (0.0008) [2023-12-27 07:24:59,525][20347] Updated weights for policy 0, policy_version 255358 (0.0008) [2023-12-27 07:24:59,580][20347] Updated weights for policy 0, policy_version 255368 (0.0007) [2023-12-27 07:24:59,593][20381] Updated weights for policy 1, policy_version 254989 (0.0009) [2023-12-27 07:24:59,646][20381] Updated weights for policy 1, policy_version 254999 (0.0009) [2023-12-27 07:24:59,701][20381] Updated weights for policy 1, policy_version 255009 (0.0007) [2023-12-27 07:25:00,204][20347] Updated weights for policy 0, policy_version 255378 (0.0007) [2023-12-27 07:25:00,266][20347] Updated weights for policy 0, policy_version 255388 (0.0007) [2023-12-27 07:25:00,322][20347] Updated weights for policy 0, policy_version 255398 (0.0008) [2023-12-27 07:25:00,401][20381] Updated weights for policy 1, policy_version 255019 (0.0007) [2023-12-27 07:25:00,460][20381] Updated weights for policy 1, policy_version 255029 (0.0008) [2023-12-27 07:25:00,519][20381] Updated weights for policy 1, policy_version 255039 (0.0007) [2023-12-27 07:25:00,904][20347] Updated weights for policy 0, policy_version 255408 (0.0006) [2023-12-27 07:25:00,960][20347] Updated weights for policy 0, policy_version 255418 (0.0006) [2023-12-27 07:25:01,028][20347] Updated weights for policy 0, policy_version 255428 (0.0006) [2023-12-27 07:25:01,235][20381] Updated weights for policy 1, policy_version 255049 (0.0009) [2023-12-27 07:25:01,299][20381] Updated weights for policy 1, policy_version 255059 (0.0006) [2023-12-27 07:25:01,362][20381] Updated weights for policy 1, policy_version 255069 (0.0007) [2023-12-27 07:25:01,421][20381] Updated weights for policy 1, policy_version 255079 (0.0007) [2023-12-27 07:25:01,752][20347] Updated weights for policy 0, policy_version 255438 (0.0008) [2023-12-27 07:25:01,814][20347] Updated weights for policy 0, policy_version 255448 (0.0008) [2023-12-27 07:25:01,881][20347] Updated weights for policy 0, policy_version 255458 (0.0008) [2023-12-27 07:25:02,053][20381] Updated weights for policy 1, policy_version 255089 (0.0006) [2023-12-27 07:25:02,109][20381] Updated weights for policy 1, policy_version 255099 (0.0006) [2023-12-27 07:25:02,165][20381] Updated weights for policy 1, policy_version 255109 (0.0005) [2023-12-27 07:25:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 130736128. Throughput: 0: 10344.2, 1: 10498.0. Samples: 130745952. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:02,363][19170] Avg episode reward: [(0, '911.030'), (1, '991.410')] [2023-12-27 07:25:02,550][20347] Updated weights for policy 0, policy_version 255468 (0.0007) [2023-12-27 07:25:02,604][20347] Updated weights for policy 0, policy_version 255478 (0.0007) [2023-12-27 07:25:02,656][20347] Updated weights for policy 0, policy_version 255488 (0.0008) [2023-12-27 07:25:02,775][20381] Updated weights for policy 1, policy_version 255119 (0.0005) [2023-12-27 07:25:02,820][20381] Updated weights for policy 1, policy_version 255129 (0.0005) [2023-12-27 07:25:02,865][20381] Updated weights for policy 1, policy_version 255139 (0.0005) [2023-12-27 07:25:03,303][20347] Updated weights for policy 0, policy_version 255498 (0.0008) [2023-12-27 07:25:03,356][20347] Updated weights for policy 0, policy_version 255508 (0.0007) [2023-12-27 07:25:03,413][20347] Updated weights for policy 0, policy_version 255518 (0.0008) [2023-12-27 07:25:03,476][20347] Updated weights for policy 0, policy_version 255528 (0.0007) [2023-12-27 07:25:03,528][20381] Updated weights for policy 1, policy_version 255149 (0.0008) [2023-12-27 07:25:03,575][20381] Updated weights for policy 1, policy_version 255159 (0.0010) [2023-12-27 07:25:03,636][20381] Updated weights for policy 1, policy_version 255169 (0.0010) [2023-12-27 07:25:04,077][20347] Updated weights for policy 0, policy_version 255538 (0.0010) [2023-12-27 07:25:04,133][20347] Updated weights for policy 0, policy_version 255548 (0.0011) [2023-12-27 07:25:04,193][20347] Updated weights for policy 0, policy_version 255558 (0.0010) [2023-12-27 07:25:04,299][20381] Updated weights for policy 1, policy_version 255179 (0.0009) [2023-12-27 07:25:04,361][20381] Updated weights for policy 1, policy_version 255189 (0.0005) [2023-12-27 07:25:04,422][20381] Updated weights for policy 1, policy_version 255199 (0.0005) [2023-12-27 07:25:04,883][20347] Updated weights for policy 0, policy_version 255568 (0.0010) [2023-12-27 07:25:04,938][20347] Updated weights for policy 0, policy_version 255578 (0.0010) [2023-12-27 07:25:04,981][20381] Updated weights for policy 1, policy_version 255209 (0.0006) [2023-12-27 07:25:04,998][20347] Updated weights for policy 0, policy_version 255588 (0.0010) [2023-12-27 07:25:05,041][20381] Updated weights for policy 1, policy_version 255219 (0.0009) [2023-12-27 07:25:05,094][20381] Updated weights for policy 1, policy_version 255229 (0.0011) [2023-12-27 07:25:05,142][20381] Updated weights for policy 1, policy_version 255239 (0.0010) [2023-12-27 07:25:05,636][20347] Updated weights for policy 0, policy_version 255598 (0.0010) [2023-12-27 07:25:05,691][20347] Updated weights for policy 0, policy_version 255608 (0.0010) [2023-12-27 07:25:05,749][20347] Updated weights for policy 0, policy_version 255618 (0.0010) [2023-12-27 07:25:05,782][20381] Updated weights for policy 1, policy_version 255249 (0.0008) [2023-12-27 07:25:05,838][20381] Updated weights for policy 1, policy_version 255259 (0.0008) [2023-12-27 07:25:05,902][20381] Updated weights for policy 1, policy_version 255269 (0.0008) [2023-12-27 07:25:06,406][20347] Updated weights for policy 0, policy_version 255628 (0.0009) [2023-12-27 07:25:06,472][20347] Updated weights for policy 0, policy_version 255638 (0.0008) [2023-12-27 07:25:06,537][20347] Updated weights for policy 0, policy_version 255648 (0.0007) [2023-12-27 07:25:06,553][20381] Updated weights for policy 1, policy_version 255279 (0.0007) [2023-12-27 07:25:06,624][20381] Updated weights for policy 1, policy_version 255289 (0.0006) [2023-12-27 07:25:06,695][20381] Updated weights for policy 1, policy_version 255299 (0.0006) [2023-12-27 07:25:07,155][20347] Updated weights for policy 0, policy_version 255658 (0.0007) [2023-12-27 07:25:07,216][20347] Updated weights for policy 0, policy_version 255668 (0.0005) [2023-12-27 07:25:07,271][20347] Updated weights for policy 0, policy_version 255678 (0.0006) [2023-12-27 07:25:07,319][20347] Updated weights for policy 0, policy_version 255688 (0.0006) [2023-12-27 07:25:07,328][20381] Updated weights for policy 1, policy_version 255309 (0.0009) [2023-12-27 07:25:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 130842624. Throughput: 0: 10412.5, 1: 10494.2. Samples: 130810064. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:07,363][19170] Avg episode reward: [(0, '882.160'), (1, '981.920')] [2023-12-27 07:25:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000255688_65470464.pth... [2023-12-27 07:25:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000254472_65159168.pth [2023-12-27 07:25:07,382][20381] Updated weights for policy 1, policy_version 255319 (0.0010) [2023-12-27 07:25:07,431][20381] Updated weights for policy 1, policy_version 255329 (0.0010) [2023-12-27 07:25:07,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000255336_65380352.pth... [2023-12-27 07:25:07,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000254088_65060864.pth [2023-12-27 07:25:07,917][20347] Updated weights for policy 0, policy_version 255698 (0.0007) [2023-12-27 07:25:07,974][20347] Updated weights for policy 0, policy_version 255708 (0.0007) [2023-12-27 07:25:08,027][20347] Updated weights for policy 0, policy_version 255718 (0.0007) [2023-12-27 07:25:08,136][20381] Updated weights for policy 1, policy_version 255339 (0.0011) [2023-12-27 07:25:08,189][20381] Updated weights for policy 1, policy_version 255349 (0.0011) [2023-12-27 07:25:08,209][20292] KL-divergence is very high: 441.3559 [2023-12-27 07:25:08,238][20381] Updated weights for policy 1, policy_version 255359 (0.0010) [2023-12-27 07:25:08,247][20292] KL-divergence is very high: 121.8990 [2023-12-27 07:25:08,636][20347] Updated weights for policy 0, policy_version 255728 (0.0007) [2023-12-27 07:25:08,688][20347] Updated weights for policy 0, policy_version 255738 (0.0007) [2023-12-27 07:25:08,743][20347] Updated weights for policy 0, policy_version 255748 (0.0007) [2023-12-27 07:25:08,950][20381] Updated weights for policy 1, policy_version 255369 (0.0010) [2023-12-27 07:25:09,012][20381] Updated weights for policy 1, policy_version 255379 (0.0005) [2023-12-27 07:25:09,067][20381] Updated weights for policy 1, policy_version 255389 (0.0005) [2023-12-27 07:25:09,117][20381] Updated weights for policy 1, policy_version 255399 (0.0005) [2023-12-27 07:25:09,390][20347] Updated weights for policy 0, policy_version 255758 (0.0009) [2023-12-27 07:25:09,454][20347] Updated weights for policy 0, policy_version 255768 (0.0010) [2023-12-27 07:25:09,516][20347] Updated weights for policy 0, policy_version 255778 (0.0008) [2023-12-27 07:25:09,750][20381] Updated weights for policy 1, policy_version 255409 (0.0006) [2023-12-27 07:25:09,807][20381] Updated weights for policy 1, policy_version 255419 (0.0009) [2023-12-27 07:25:09,869][20381] Updated weights for policy 1, policy_version 255429 (0.0007) [2023-12-27 07:25:10,240][20347] Updated weights for policy 0, policy_version 255788 (0.0009) [2023-12-27 07:25:10,297][20347] Updated weights for policy 0, policy_version 255798 (0.0011) [2023-12-27 07:25:10,357][20347] Updated weights for policy 0, policy_version 255808 (0.0010) [2023-12-27 07:25:10,544][20381] Updated weights for policy 1, policy_version 255439 (0.0007) [2023-12-27 07:25:10,607][20381] Updated weights for policy 1, policy_version 255449 (0.0007) [2023-12-27 07:25:10,667][20381] Updated weights for policy 1, policy_version 255459 (0.0009) [2023-12-27 07:25:11,037][20347] Updated weights for policy 0, policy_version 255818 (0.0010) [2023-12-27 07:25:11,100][20347] Updated weights for policy 0, policy_version 255828 (0.0011) [2023-12-27 07:25:11,164][20347] Updated weights for policy 0, policy_version 255838 (0.0008) [2023-12-27 07:25:11,227][20347] Updated weights for policy 0, policy_version 255848 (0.0006) [2023-12-27 07:25:11,291][20381] Updated weights for policy 1, policy_version 255469 (0.0007) [2023-12-27 07:25:11,341][20381] Updated weights for policy 1, policy_version 255479 (0.0009) [2023-12-27 07:25:11,413][20381] Updated weights for policy 1, policy_version 255489 (0.0007) [2023-12-27 07:25:11,868][20347] Updated weights for policy 0, policy_version 255858 (0.0008) [2023-12-27 07:25:11,928][20347] Updated weights for policy 0, policy_version 255868 (0.0007) [2023-12-27 07:25:11,981][20347] Updated weights for policy 0, policy_version 255878 (0.0008) [2023-12-27 07:25:12,157][20381] Updated weights for policy 1, policy_version 255499 (0.0007) [2023-12-27 07:25:12,212][20381] Updated weights for policy 1, policy_version 255509 (0.0008) [2023-12-27 07:25:12,272][20381] Updated weights for policy 1, policy_version 255519 (0.0006) [2023-12-27 07:25:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 130949120. Throughput: 0: 10498.0, 1: 10490.7. Samples: 130936524. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:12,363][19170] Avg episode reward: [(0, '851.880'), (1, '980.910')] [2023-12-27 07:25:12,683][20347] Updated weights for policy 0, policy_version 255888 (0.0008) [2023-12-27 07:25:12,735][20347] Updated weights for policy 0, policy_version 255898 (0.0008) [2023-12-27 07:25:12,795][20347] Updated weights for policy 0, policy_version 255908 (0.0007) [2023-12-27 07:25:12,959][20381] Updated weights for policy 1, policy_version 255529 (0.0007) [2023-12-27 07:25:13,014][20381] Updated weights for policy 1, policy_version 255539 (0.0005) [2023-12-27 07:25:13,072][20381] Updated weights for policy 1, policy_version 255549 (0.0007) [2023-12-27 07:25:13,124][20381] Updated weights for policy 1, policy_version 255559 (0.0008) [2023-12-27 07:25:13,520][20347] Updated weights for policy 0, policy_version 255918 (0.0007) [2023-12-27 07:25:13,573][20347] Updated weights for policy 0, policy_version 255928 (0.0008) [2023-12-27 07:25:13,618][20347] Updated weights for policy 0, policy_version 255938 (0.0007) [2023-12-27 07:25:13,762][20381] Updated weights for policy 1, policy_version 255569 (0.0007) [2023-12-27 07:25:13,814][20381] Updated weights for policy 1, policy_version 255579 (0.0008) [2023-12-27 07:25:13,868][20381] Updated weights for policy 1, policy_version 255589 (0.0007) [2023-12-27 07:25:14,333][20347] Updated weights for policy 0, policy_version 255948 (0.0007) [2023-12-27 07:25:14,402][20347] Updated weights for policy 0, policy_version 255958 (0.0008) [2023-12-27 07:25:14,458][20347] Updated weights for policy 0, policy_version 255968 (0.0007) [2023-12-27 07:25:14,485][20381] Updated weights for policy 1, policy_version 255599 (0.0006) [2023-12-27 07:25:14,539][20381] Updated weights for policy 1, policy_version 255609 (0.0007) [2023-12-27 07:25:14,597][20381] Updated weights for policy 1, policy_version 255619 (0.0008) [2023-12-27 07:25:15,173][20347] Updated weights for policy 0, policy_version 255978 (0.0007) [2023-12-27 07:25:15,234][20347] Updated weights for policy 0, policy_version 255988 (0.0008) [2023-12-27 07:25:15,248][20381] Updated weights for policy 1, policy_version 255629 (0.0007) [2023-12-27 07:25:15,295][20347] Updated weights for policy 0, policy_version 255998 (0.0007) [2023-12-27 07:25:15,308][20381] Updated weights for policy 1, policy_version 255639 (0.0006) [2023-12-27 07:25:15,357][20347] Updated weights for policy 0, policy_version 256008 (0.0007) [2023-12-27 07:25:15,371][20381] Updated weights for policy 1, policy_version 255649 (0.0007) [2023-12-27 07:25:16,060][20381] Updated weights for policy 1, policy_version 255659 (0.0007) [2023-12-27 07:25:16,073][20347] Updated weights for policy 0, policy_version 256018 (0.0006) [2023-12-27 07:25:16,115][20381] Updated weights for policy 1, policy_version 255669 (0.0005) [2023-12-27 07:25:16,121][20347] Updated weights for policy 0, policy_version 256028 (0.0006) [2023-12-27 07:25:16,168][20381] Updated weights for policy 1, policy_version 255679 (0.0006) [2023-12-27 07:25:16,177][20347] Updated weights for policy 0, policy_version 256038 (0.0006) [2023-12-27 07:25:16,832][20381] Updated weights for policy 1, policy_version 255689 (0.0007) [2023-12-27 07:25:16,883][20381] Updated weights for policy 1, policy_version 255699 (0.0005) [2023-12-27 07:25:16,889][20347] Updated weights for policy 0, policy_version 256048 (0.0007) [2023-12-27 07:25:16,931][20381] Updated weights for policy 1, policy_version 255709 (0.0006) [2023-12-27 07:25:16,941][20347] Updated weights for policy 0, policy_version 256058 (0.0005) [2023-12-27 07:25:16,987][20381] Updated weights for policy 1, policy_version 255719 (0.0006) [2023-12-27 07:25:16,999][20347] Updated weights for policy 0, policy_version 256068 (0.0007) [2023-12-27 07:25:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 131047424. Throughput: 0: 10425.6, 1: 10464.7. Samples: 131057348. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:17,363][19170] Avg episode reward: [(0, '905.140'), (1, '990.400')] [2023-12-27 07:25:17,588][20347] Updated weights for policy 0, policy_version 256078 (0.0010) [2023-12-27 07:25:17,620][20381] Updated weights for policy 1, policy_version 255729 (0.0005) [2023-12-27 07:25:17,647][20347] Updated weights for policy 0, policy_version 256088 (0.0007) [2023-12-27 07:25:17,667][20381] Updated weights for policy 1, policy_version 255739 (0.0005) [2023-12-27 07:25:17,705][20347] Updated weights for policy 0, policy_version 256098 (0.0008) [2023-12-27 07:25:17,713][20381] Updated weights for policy 1, policy_version 255749 (0.0006) [2023-12-27 07:25:18,269][20381] Updated weights for policy 1, policy_version 255759 (0.0007) [2023-12-27 07:25:18,323][20381] Updated weights for policy 1, policy_version 255769 (0.0007) [2023-12-27 07:25:18,383][20381] Updated weights for policy 1, policy_version 255779 (0.0007) [2023-12-27 07:25:18,454][20347] Updated weights for policy 0, policy_version 256108 (0.0007) [2023-12-27 07:25:18,512][20347] Updated weights for policy 0, policy_version 256118 (0.0007) [2023-12-27 07:25:18,568][20347] Updated weights for policy 0, policy_version 256128 (0.0007) [2023-12-27 07:25:19,052][20381] Updated weights for policy 1, policy_version 255789 (0.0008) [2023-12-27 07:25:19,104][20381] Updated weights for policy 1, policy_version 255799 (0.0007) [2023-12-27 07:25:19,157][20381] Updated weights for policy 1, policy_version 255809 (0.0008) [2023-12-27 07:25:19,276][20347] Updated weights for policy 0, policy_version 256138 (0.0007) [2023-12-27 07:25:19,342][20347] Updated weights for policy 0, policy_version 256148 (0.0008) [2023-12-27 07:25:19,402][20347] Updated weights for policy 0, policy_version 256158 (0.0005) [2023-12-27 07:25:19,463][20347] Updated weights for policy 0, policy_version 256168 (0.0008) [2023-12-27 07:25:19,853][20381] Updated weights for policy 1, policy_version 255819 (0.0006) [2023-12-27 07:25:19,913][20381] Updated weights for policy 1, policy_version 255829 (0.0007) [2023-12-27 07:25:19,969][20381] Updated weights for policy 1, policy_version 255839 (0.0009) [2023-12-27 07:25:20,149][20347] Updated weights for policy 0, policy_version 256178 (0.0007) [2023-12-27 07:25:20,205][20347] Updated weights for policy 0, policy_version 256188 (0.0007) [2023-12-27 07:25:20,263][20347] Updated weights for policy 0, policy_version 256198 (0.0007) [2023-12-27 07:25:20,730][20381] Updated weights for policy 1, policy_version 255849 (0.0009) [2023-12-27 07:25:20,787][20381] Updated weights for policy 1, policy_version 255859 (0.0008) [2023-12-27 07:25:20,846][20381] Updated weights for policy 1, policy_version 255869 (0.0009) [2023-12-27 07:25:20,910][20381] Updated weights for policy 1, policy_version 255879 (0.0009) [2023-12-27 07:25:20,973][20347] Updated weights for policy 0, policy_version 256208 (0.0006) [2023-12-27 07:25:21,037][20347] Updated weights for policy 0, policy_version 256218 (0.0006) [2023-12-27 07:25:21,103][20347] Updated weights for policy 0, policy_version 256228 (0.0008) [2023-12-27 07:25:21,669][20381] Updated weights for policy 1, policy_version 255889 (0.0011) [2023-12-27 07:25:21,734][20381] Updated weights for policy 1, policy_version 255899 (0.0011) [2023-12-27 07:25:21,784][20347] Updated weights for policy 0, policy_version 256238 (0.0007) [2023-12-27 07:25:21,799][20381] Updated weights for policy 1, policy_version 255909 (0.0011) [2023-12-27 07:25:21,840][20347] Updated weights for policy 0, policy_version 256248 (0.0006) [2023-12-27 07:25:21,903][20347] Updated weights for policy 0, policy_version 256258 (0.0005) [2023-12-27 07:25:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 131145728. Throughput: 0: 10395.1, 1: 10482.3. Samples: 131119692. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:22,363][19170] Avg episode reward: [(0, '924.400'), (1, '1000.000')] [2023-12-27 07:25:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000255912_65527808.pth... [2023-12-27 07:25:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000256264_65617920.pth... [2023-12-27 07:25:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000254696_65216512.pth [2023-12-27 07:25:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000255080_65314816.pth [2023-12-27 07:25:22,457][20381] Updated weights for policy 1, policy_version 255919 (0.0011) [2023-12-27 07:25:22,521][20381] Updated weights for policy 1, policy_version 255929 (0.0011) [2023-12-27 07:25:22,538][20347] Updated weights for policy 0, policy_version 256268 (0.0006) [2023-12-27 07:25:22,578][20381] Updated weights for policy 1, policy_version 255939 (0.0011) [2023-12-27 07:25:22,597][20347] Updated weights for policy 0, policy_version 256278 (0.0006) [2023-12-27 07:25:22,654][20347] Updated weights for policy 0, policy_version 256288 (0.0007) [2023-12-27 07:25:23,208][20381] Updated weights for policy 1, policy_version 255949 (0.0011) [2023-12-27 07:25:23,259][20381] Updated weights for policy 1, policy_version 255959 (0.0010) [2023-12-27 07:25:23,287][20347] Updated weights for policy 0, policy_version 256298 (0.0007) [2023-12-27 07:25:23,308][20381] Updated weights for policy 1, policy_version 255969 (0.0010) [2023-12-27 07:25:23,337][20347] Updated weights for policy 0, policy_version 256308 (0.0005) [2023-12-27 07:25:23,397][20347] Updated weights for policy 0, policy_version 256318 (0.0008) [2023-12-27 07:25:23,456][20347] Updated weights for policy 0, policy_version 256328 (0.0010) [2023-12-27 07:25:23,872][20381] Updated weights for policy 1, policy_version 255979 (0.0008) [2023-12-27 07:25:23,933][20381] Updated weights for policy 1, policy_version 255989 (0.0008) [2023-12-27 07:25:23,993][20381] Updated weights for policy 1, policy_version 255999 (0.0006) [2023-12-27 07:25:24,086][20347] Updated weights for policy 0, policy_version 256338 (0.0005) [2023-12-27 07:25:24,134][20347] Updated weights for policy 0, policy_version 256348 (0.0007) [2023-12-27 07:25:24,184][20347] Updated weights for policy 0, policy_version 256358 (0.0008) [2023-12-27 07:25:24,588][20381] Updated weights for policy 1, policy_version 256009 (0.0008) [2023-12-27 07:25:24,634][20381] Updated weights for policy 1, policy_version 256019 (0.0007) [2023-12-27 07:25:24,690][20381] Updated weights for policy 1, policy_version 256029 (0.0008) [2023-12-27 07:25:24,740][20381] Updated weights for policy 1, policy_version 256039 (0.0006) [2023-12-27 07:25:24,884][20347] Updated weights for policy 0, policy_version 256368 (0.0006) [2023-12-27 07:25:24,940][20347] Updated weights for policy 0, policy_version 256378 (0.0005) [2023-12-27 07:25:24,997][20347] Updated weights for policy 0, policy_version 256388 (0.0005) [2023-12-27 07:25:25,424][20381] Updated weights for policy 1, policy_version 256049 (0.0007) [2023-12-27 07:25:25,480][20381] Updated weights for policy 1, policy_version 256059 (0.0007) [2023-12-27 07:25:25,528][20347] Updated weights for policy 0, policy_version 256398 (0.0007) [2023-12-27 07:25:25,541][20381] Updated weights for policy 1, policy_version 256069 (0.0007) [2023-12-27 07:25:25,581][20347] Updated weights for policy 0, policy_version 256408 (0.0006) [2023-12-27 07:25:25,636][20347] Updated weights for policy 0, policy_version 256418 (0.0007) [2023-12-27 07:25:26,172][20381] Updated weights for policy 1, policy_version 256079 (0.0009) [2023-12-27 07:25:26,224][20381] Updated weights for policy 1, policy_version 256089 (0.0010) [2023-12-27 07:25:26,275][20381] Updated weights for policy 1, policy_version 256099 (0.0010) [2023-12-27 07:25:26,306][20347] Updated weights for policy 0, policy_version 256428 (0.0007) [2023-12-27 07:25:26,362][20347] Updated weights for policy 0, policy_version 256438 (0.0007) [2023-12-27 07:25:26,415][20347] Updated weights for policy 0, policy_version 256448 (0.0007) [2023-12-27 07:25:26,929][20381] Updated weights for policy 1, policy_version 256109 (0.0008) [2023-12-27 07:25:26,976][20381] Updated weights for policy 1, policy_version 256119 (0.0007) [2023-12-27 07:25:27,022][20347] Updated weights for policy 0, policy_version 256458 (0.0007) [2023-12-27 07:25:27,024][20381] Updated weights for policy 1, policy_version 256129 (0.0008) [2023-12-27 07:25:27,079][20347] Updated weights for policy 0, policy_version 256468 (0.0008) [2023-12-27 07:25:27,124][20347] Updated weights for policy 0, policy_version 256478 (0.0008) [2023-12-27 07:25:27,167][20347] Updated weights for policy 0, policy_version 256488 (0.0007) [2023-12-27 07:25:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 131260416. Throughput: 0: 10388.8, 1: 10499.6. Samples: 131245956. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:27,363][19170] Avg episode reward: [(0, '744.900'), (1, '1000.000')] [2023-12-27 07:25:27,682][20381] Updated weights for policy 1, policy_version 256139 (0.0006) [2023-12-27 07:25:27,740][20381] Updated weights for policy 1, policy_version 256149 (0.0005) [2023-12-27 07:25:27,804][20381] Updated weights for policy 1, policy_version 256159 (0.0005) [2023-12-27 07:25:27,813][20347] Updated weights for policy 0, policy_version 256498 (0.0005) [2023-12-27 07:25:27,875][20347] Updated weights for policy 0, policy_version 256508 (0.0005) [2023-12-27 07:25:27,933][20347] Updated weights for policy 0, policy_version 256518 (0.0005) [2023-12-27 07:25:28,394][20381] Updated weights for policy 1, policy_version 256169 (0.0006) [2023-12-27 07:25:28,455][20381] Updated weights for policy 1, policy_version 256179 (0.0011) [2023-12-27 07:25:28,468][20347] Updated weights for policy 0, policy_version 256528 (0.0008) [2023-12-27 07:25:28,507][20381] Updated weights for policy 1, policy_version 256189 (0.0010) [2023-12-27 07:25:28,528][20347] Updated weights for policy 0, policy_version 256538 (0.0007) [2023-12-27 07:25:28,559][20381] Updated weights for policy 1, policy_version 256199 (0.0011) [2023-12-27 07:25:28,589][20347] Updated weights for policy 0, policy_version 256548 (0.0006) [2023-12-27 07:25:29,193][20347] Updated weights for policy 0, policy_version 256558 (0.0006) [2023-12-27 07:25:29,247][20381] Updated weights for policy 1, policy_version 256209 (0.0007) [2023-12-27 07:25:29,253][20347] Updated weights for policy 0, policy_version 256568 (0.0007) [2023-12-27 07:25:29,296][20381] Updated weights for policy 1, policy_version 256219 (0.0007) [2023-12-27 07:25:29,309][20347] Updated weights for policy 0, policy_version 256578 (0.0007) [2023-12-27 07:25:29,355][20381] Updated weights for policy 1, policy_version 256229 (0.0007) [2023-12-27 07:25:29,973][20347] Updated weights for policy 0, policy_version 256588 (0.0007) [2023-12-27 07:25:30,024][20347] Updated weights for policy 0, policy_version 256598 (0.0007) [2023-12-27 07:25:30,062][20381] Updated weights for policy 1, policy_version 256239 (0.0010) [2023-12-27 07:25:30,075][20347] Updated weights for policy 0, policy_version 256608 (0.0005) [2023-12-27 07:25:30,117][20381] Updated weights for policy 1, policy_version 256249 (0.0009) [2023-12-27 07:25:30,176][20381] Updated weights for policy 1, policy_version 256259 (0.0006) [2023-12-27 07:25:30,792][20347] Updated weights for policy 0, policy_version 256618 (0.0007) [2023-12-27 07:25:30,840][20381] Updated weights for policy 1, policy_version 256269 (0.0009) [2023-12-27 07:25:30,840][20347] Updated weights for policy 0, policy_version 256628 (0.0007) [2023-12-27 07:25:30,888][20381] Updated weights for policy 1, policy_version 256279 (0.0010) [2023-12-27 07:25:30,891][20347] Updated weights for policy 0, policy_version 256638 (0.0008) [2023-12-27 07:25:30,937][20381] Updated weights for policy 1, policy_version 256289 (0.0010) [2023-12-27 07:25:30,938][20347] Updated weights for policy 0, policy_version 256648 (0.0006) [2023-12-27 07:25:31,643][20347] Updated weights for policy 0, policy_version 256658 (0.0008) [2023-12-27 07:25:31,645][20381] Updated weights for policy 1, policy_version 256299 (0.0009) [2023-12-27 07:25:31,699][20381] Updated weights for policy 1, policy_version 256309 (0.0006) [2023-12-27 07:25:31,702][20347] Updated weights for policy 0, policy_version 256668 (0.0007) [2023-12-27 07:25:31,758][20381] Updated weights for policy 1, policy_version 256319 (0.0008) [2023-12-27 07:25:31,762][20347] Updated weights for policy 0, policy_version 256678 (0.0007) [2023-12-27 07:25:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 131358720. Throughput: 0: 10431.1, 1: 10539.6. Samples: 131372908. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-12-27 07:25:32,363][19170] Avg episode reward: [(0, '702.330'), (1, '1000.000')] [2023-12-27 07:25:32,434][20347] Updated weights for policy 0, policy_version 256688 (0.0008) [2023-12-27 07:25:32,452][20381] Updated weights for policy 1, policy_version 256329 (0.0006) [2023-12-27 07:25:32,493][20347] Updated weights for policy 0, policy_version 256698 (0.0007) [2023-12-27 07:25:32,515][20381] Updated weights for policy 1, policy_version 256339 (0.0006) [2023-12-27 07:25:32,556][20347] Updated weights for policy 0, policy_version 256708 (0.0006) [2023-12-27 07:25:32,570][20381] Updated weights for policy 1, policy_version 256349 (0.0007) [2023-12-27 07:25:32,630][20381] Updated weights for policy 1, policy_version 256359 (0.0007) [2023-12-27 07:25:33,161][20347] Updated weights for policy 0, policy_version 256718 (0.0006) [2023-12-27 07:25:33,225][20347] Updated weights for policy 0, policy_version 256728 (0.0007) [2023-12-27 07:25:33,288][20347] Updated weights for policy 0, policy_version 256738 (0.0008) [2023-12-27 07:25:33,320][20381] Updated weights for policy 1, policy_version 256369 (0.0005) [2023-12-27 07:25:33,366][20381] Updated weights for policy 1, policy_version 256379 (0.0008) [2023-12-27 07:25:33,412][20381] Updated weights for policy 1, policy_version 256389 (0.0008) [2023-12-27 07:25:33,879][20347] Updated weights for policy 0, policy_version 256748 (0.0006) [2023-12-27 07:25:33,930][20347] Updated weights for policy 0, policy_version 256758 (0.0005) [2023-12-27 07:25:33,982][20347] Updated weights for policy 0, policy_version 256768 (0.0005) [2023-12-27 07:25:34,054][20381] Updated weights for policy 1, policy_version 256399 (0.0010) [2023-12-27 07:25:34,119][20381] Updated weights for policy 1, policy_version 256409 (0.0010) [2023-12-27 07:25:34,182][20381] Updated weights for policy 1, policy_version 256419 (0.0009) [2023-12-27 07:25:34,561][20347] Updated weights for policy 0, policy_version 256778 (0.0006) [2023-12-27 07:25:34,623][20347] Updated weights for policy 0, policy_version 256788 (0.0010) [2023-12-27 07:25:34,684][20347] Updated weights for policy 0, policy_version 256798 (0.0007) [2023-12-27 07:25:34,746][20347] Updated weights for policy 0, policy_version 256808 (0.0008) [2023-12-27 07:25:34,883][20381] Updated weights for policy 1, policy_version 256429 (0.0006) [2023-12-27 07:25:34,928][20381] Updated weights for policy 1, policy_version 256439 (0.0005) [2023-12-27 07:25:34,978][20381] Updated weights for policy 1, policy_version 256449 (0.0010) [2023-12-27 07:25:35,362][20347] Updated weights for policy 0, policy_version 256818 (0.0007) [2023-12-27 07:25:35,421][20347] Updated weights for policy 0, policy_version 256828 (0.0006) [2023-12-27 07:25:35,476][20347] Updated weights for policy 0, policy_version 256838 (0.0006) [2023-12-27 07:25:35,606][20381] Updated weights for policy 1, policy_version 256459 (0.0010) [2023-12-27 07:25:35,659][20381] Updated weights for policy 1, policy_version 256469 (0.0010) [2023-12-27 07:25:35,707][20381] Updated weights for policy 1, policy_version 256479 (0.0010) [2023-12-27 07:25:36,116][20347] Updated weights for policy 0, policy_version 256848 (0.0007) [2023-12-27 07:25:36,165][20347] Updated weights for policy 0, policy_version 256858 (0.0008) [2023-12-27 07:25:36,225][20347] Updated weights for policy 0, policy_version 256868 (0.0008) [2023-12-27 07:25:36,432][20381] Updated weights for policy 1, policy_version 256489 (0.0010) [2023-12-27 07:25:36,499][20381] Updated weights for policy 1, policy_version 256499 (0.0011) [2023-12-27 07:25:36,567][20381] Updated weights for policy 1, policy_version 256509 (0.0011) [2023-12-27 07:25:36,631][20381] Updated weights for policy 1, policy_version 256519 (0.0011) [2023-12-27 07:25:36,938][20347] Updated weights for policy 0, policy_version 256878 (0.0007) [2023-12-27 07:25:36,995][20347] Updated weights for policy 0, policy_version 256888 (0.0009) [2023-12-27 07:25:37,047][20347] Updated weights for policy 0, policy_version 256898 (0.0008) [2023-12-27 07:25:37,247][20381] Updated weights for policy 1, policy_version 256529 (0.0006) [2023-12-27 07:25:37,304][20381] Updated weights for policy 1, policy_version 256539 (0.0010) [2023-12-27 07:25:37,352][20381] Updated weights for policy 1, policy_version 256549 (0.0008) [2023-12-27 07:25:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 131465216. Throughput: 0: 10437.7, 1: 10520.5. Samples: 131436612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:25:37,364][19170] Avg episode reward: [(0, '841.100'), (1, '1000.000')] [2023-12-27 07:25:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000256552_65691648.pth... [2023-12-27 07:25:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000256904_65781760.pth... [2023-12-27 07:25:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000255336_65380352.pth [2023-12-27 07:25:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000255688_65470464.pth [2023-12-27 07:25:37,802][20347] Updated weights for policy 0, policy_version 256908 (0.0008) [2023-12-27 07:25:37,863][20347] Updated weights for policy 0, policy_version 256918 (0.0008) [2023-12-27 07:25:37,914][20381] Updated weights for policy 1, policy_version 256559 (0.0009) [2023-12-27 07:25:37,927][20347] Updated weights for policy 0, policy_version 256928 (0.0008) [2023-12-27 07:25:37,962][20381] Updated weights for policy 1, policy_version 256569 (0.0010) [2023-12-27 07:25:38,011][20381] Updated weights for policy 1, policy_version 256579 (0.0010) [2023-12-27 07:25:38,611][20347] Updated weights for policy 0, policy_version 256938 (0.0006) [2023-12-27 07:25:38,667][20347] Updated weights for policy 0, policy_version 256948 (0.0006) [2023-12-27 07:25:38,718][20347] Updated weights for policy 0, policy_version 256958 (0.0005) [2023-12-27 07:25:38,726][20381] Updated weights for policy 1, policy_version 256589 (0.0011) [2023-12-27 07:25:38,774][20347] Updated weights for policy 0, policy_version 256968 (0.0005) [2023-12-27 07:25:38,791][20381] Updated weights for policy 1, policy_version 256599 (0.0011) [2023-12-27 07:25:38,840][20381] Updated weights for policy 1, policy_version 256609 (0.0011) [2023-12-27 07:25:39,438][20347] Updated weights for policy 0, policy_version 256978 (0.0007) [2023-12-27 07:25:39,496][20347] Updated weights for policy 0, policy_version 256988 (0.0005) [2023-12-27 07:25:39,516][20381] Updated weights for policy 1, policy_version 256619 (0.0011) [2023-12-27 07:25:39,554][20347] Updated weights for policy 0, policy_version 256998 (0.0006) [2023-12-27 07:25:39,570][20381] Updated weights for policy 1, policy_version 256629 (0.0008) [2023-12-27 07:25:39,632][20381] Updated weights for policy 1, policy_version 256639 (0.0006) [2023-12-27 07:25:40,250][20347] Updated weights for policy 0, policy_version 257008 (0.0006) [2023-12-27 07:25:40,278][20381] Updated weights for policy 1, policy_version 256649 (0.0007) [2023-12-27 07:25:40,311][20347] Updated weights for policy 0, policy_version 257018 (0.0007) [2023-12-27 07:25:40,338][20381] Updated weights for policy 1, policy_version 256659 (0.0009) [2023-12-27 07:25:40,365][20347] Updated weights for policy 0, policy_version 257028 (0.0009) [2023-12-27 07:25:40,396][20381] Updated weights for policy 1, policy_version 256669 (0.0011) [2023-12-27 07:25:40,448][20381] Updated weights for policy 1, policy_version 256679 (0.0010) [2023-12-27 07:25:41,064][20347] Updated weights for policy 0, policy_version 257038 (0.0007) [2023-12-27 07:25:41,126][20347] Updated weights for policy 0, policy_version 257048 (0.0007) [2023-12-27 07:25:41,171][20381] Updated weights for policy 1, policy_version 256689 (0.0007) [2023-12-27 07:25:41,185][20347] Updated weights for policy 0, policy_version 257058 (0.0007) [2023-12-27 07:25:41,223][20381] Updated weights for policy 1, policy_version 256699 (0.0006) [2023-12-27 07:25:41,282][20381] Updated weights for policy 1, policy_version 256709 (0.0009) [2023-12-27 07:25:41,926][20347] Updated weights for policy 0, policy_version 257068 (0.0008) [2023-12-27 07:25:41,979][20347] Updated weights for policy 0, policy_version 257078 (0.0007) [2023-12-27 07:25:42,009][20381] Updated weights for policy 1, policy_version 256719 (0.0007) [2023-12-27 07:25:42,034][20347] Updated weights for policy 0, policy_version 257088 (0.0007) [2023-12-27 07:25:42,062][20381] Updated weights for policy 1, policy_version 256729 (0.0006) [2023-12-27 07:25:42,113][20381] Updated weights for policy 1, policy_version 256739 (0.0008) [2023-12-27 07:25:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 131571712. Throughput: 0: 10408.8, 1: 10483.9. Samples: 131559852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:25:42,363][19170] Avg episode reward: [(0, '863.850'), (1, '1000.000')] [2023-12-27 07:25:42,713][20381] Updated weights for policy 1, policy_version 256749 (0.0006) [2023-12-27 07:25:42,763][20381] Updated weights for policy 1, policy_version 256759 (0.0005) [2023-12-27 07:25:42,781][20347] Updated weights for policy 0, policy_version 257098 (0.0008) [2023-12-27 07:25:42,819][20381] Updated weights for policy 1, policy_version 256769 (0.0005) [2023-12-27 07:25:42,846][20347] Updated weights for policy 0, policy_version 257108 (0.0007) [2023-12-27 07:25:42,897][20347] Updated weights for policy 0, policy_version 257118 (0.0008) [2023-12-27 07:25:42,954][20347] Updated weights for policy 0, policy_version 257128 (0.0008) [2023-12-27 07:25:43,407][20381] Updated weights for policy 1, policy_version 256779 (0.0006) [2023-12-27 07:25:43,459][20381] Updated weights for policy 1, policy_version 256789 (0.0008) [2023-12-27 07:25:43,511][20381] Updated weights for policy 1, policy_version 256799 (0.0008) [2023-12-27 07:25:43,655][20347] Updated weights for policy 0, policy_version 257138 (0.0007) [2023-12-27 07:25:43,706][20347] Updated weights for policy 0, policy_version 257148 (0.0005) [2023-12-27 07:25:43,752][20347] Updated weights for policy 0, policy_version 257158 (0.0005) [2023-12-27 07:25:44,259][20381] Updated weights for policy 1, policy_version 256809 (0.0008) [2023-12-27 07:25:44,307][20381] Updated weights for policy 1, policy_version 256819 (0.0007) [2023-12-27 07:25:44,352][20381] Updated weights for policy 1, policy_version 256829 (0.0006) [2023-12-27 07:25:44,357][20347] Updated weights for policy 0, policy_version 257168 (0.0009) [2023-12-27 07:25:44,400][20381] Updated weights for policy 1, policy_version 256839 (0.0005) [2023-12-27 07:25:44,409][20347] Updated weights for policy 0, policy_version 257178 (0.0010) [2023-12-27 07:25:44,454][20347] Updated weights for policy 0, policy_version 257188 (0.0010) [2023-12-27 07:25:45,075][20381] Updated weights for policy 1, policy_version 256849 (0.0007) [2023-12-27 07:25:45,137][20381] Updated weights for policy 1, policy_version 256859 (0.0007) [2023-12-27 07:25:45,169][20347] Updated weights for policy 0, policy_version 257198 (0.0010) [2023-12-27 07:25:45,201][20381] Updated weights for policy 1, policy_version 256869 (0.0006) [2023-12-27 07:25:45,222][20347] Updated weights for policy 0, policy_version 257208 (0.0009) [2023-12-27 07:25:45,285][20347] Updated weights for policy 0, policy_version 257218 (0.0008) [2023-12-27 07:25:45,820][20381] Updated weights for policy 1, policy_version 256879 (0.0008) [2023-12-27 07:25:45,866][20381] Updated weights for policy 1, policy_version 256889 (0.0010) [2023-12-27 07:25:45,910][20381] Updated weights for policy 1, policy_version 256899 (0.0009) [2023-12-27 07:25:45,933][20347] Updated weights for policy 0, policy_version 257228 (0.0007) [2023-12-27 07:25:45,986][20347] Updated weights for policy 0, policy_version 257238 (0.0007) [2023-12-27 07:25:46,044][20347] Updated weights for policy 0, policy_version 257248 (0.0008) [2023-12-27 07:25:46,598][20381] Updated weights for policy 1, policy_version 256909 (0.0007) [2023-12-27 07:25:46,655][20381] Updated weights for policy 1, policy_version 256919 (0.0008) [2023-12-27 07:25:46,713][20381] Updated weights for policy 1, policy_version 256929 (0.0009) [2023-12-27 07:25:46,726][20347] Updated weights for policy 0, policy_version 257258 (0.0007) [2023-12-27 07:25:46,784][20347] Updated weights for policy 0, policy_version 257268 (0.0006) [2023-12-27 07:25:46,844][20347] Updated weights for policy 0, policy_version 257278 (0.0009) [2023-12-27 07:25:46,909][20347] Updated weights for policy 0, policy_version 257288 (0.0007) [2023-12-27 07:25:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 131670016. Throughput: 0: 10378.3, 1: 10446.7. Samples: 131683076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:25:47,363][19170] Avg episode reward: [(0, '872.490'), (1, '1000.000')] [2023-12-27 07:25:47,408][20381] Updated weights for policy 1, policy_version 256939 (0.0007) [2023-12-27 07:25:47,460][20381] Updated weights for policy 1, policy_version 256949 (0.0007) [2023-12-27 07:25:47,511][20381] Updated weights for policy 1, policy_version 256959 (0.0008) [2023-12-27 07:25:47,601][20347] Updated weights for policy 0, policy_version 257298 (0.0008) [2023-12-27 07:25:47,691][20347] Updated weights for policy 0, policy_version 257308 (0.0009) [2023-12-27 07:25:47,743][20347] Updated weights for policy 0, policy_version 257318 (0.0008) [2023-12-27 07:25:48,218][20381] Updated weights for policy 1, policy_version 256969 (0.0007) [2023-12-27 07:25:48,269][20381] Updated weights for policy 1, policy_version 256979 (0.0007) [2023-12-27 07:25:48,321][20381] Updated weights for policy 1, policy_version 256989 (0.0007) [2023-12-27 07:25:48,371][20347] Updated weights for policy 0, policy_version 257328 (0.0010) [2023-12-27 07:25:48,381][20381] Updated weights for policy 1, policy_version 256999 (0.0008) [2023-12-27 07:25:48,424][20347] Updated weights for policy 0, policy_version 257338 (0.0010) [2023-12-27 07:25:48,471][20347] Updated weights for policy 0, policy_version 257348 (0.0010) [2023-12-27 07:25:49,087][20381] Updated weights for policy 1, policy_version 257009 (0.0008) [2023-12-27 07:25:49,142][20381] Updated weights for policy 1, policy_version 257019 (0.0010) [2023-12-27 07:25:49,172][20347] Updated weights for policy 0, policy_version 257358 (0.0010) [2023-12-27 07:25:49,195][20381] Updated weights for policy 1, policy_version 257029 (0.0011) [2023-12-27 07:25:49,224][20347] Updated weights for policy 0, policy_version 257368 (0.0011) [2023-12-27 07:25:49,275][20347] Updated weights for policy 0, policy_version 257378 (0.0007) [2023-12-27 07:25:49,883][20381] Updated weights for policy 1, policy_version 257039 (0.0006) [2023-12-27 07:25:49,946][20381] Updated weights for policy 1, policy_version 257049 (0.0009) [2023-12-27 07:25:49,948][20347] Updated weights for policy 0, policy_version 257388 (0.0006) [2023-12-27 07:25:50,002][20381] Updated weights for policy 1, policy_version 257059 (0.0011) [2023-12-27 07:25:50,008][20347] Updated weights for policy 0, policy_version 257398 (0.0010) [2023-12-27 07:25:50,065][20347] Updated weights for policy 0, policy_version 257408 (0.0010) [2023-12-27 07:25:50,625][20347] Updated weights for policy 0, policy_version 257418 (0.0007) [2023-12-27 07:25:50,661][20381] Updated weights for policy 1, policy_version 257069 (0.0008) [2023-12-27 07:25:50,685][20347] Updated weights for policy 0, policy_version 257428 (0.0006) [2023-12-27 07:25:50,714][20381] Updated weights for policy 1, policy_version 257079 (0.0008) [2023-12-27 07:25:50,746][20347] Updated weights for policy 0, policy_version 257438 (0.0005) [2023-12-27 07:25:50,768][20381] Updated weights for policy 1, policy_version 257089 (0.0008) [2023-12-27 07:25:50,806][20347] Updated weights for policy 0, policy_version 257448 (0.0005) [2023-12-27 07:25:51,438][20347] Updated weights for policy 0, policy_version 257458 (0.0008) [2023-12-27 07:25:51,474][20381] Updated weights for policy 1, policy_version 257099 (0.0009) [2023-12-27 07:25:51,487][20347] Updated weights for policy 0, policy_version 257468 (0.0006) [2023-12-27 07:25:51,532][20381] Updated weights for policy 1, policy_version 257109 (0.0008) [2023-12-27 07:25:51,534][20347] Updated weights for policy 0, policy_version 257478 (0.0007) [2023-12-27 07:25:51,589][20381] Updated weights for policy 1, policy_version 257119 (0.0009) [2023-12-27 07:25:52,251][20381] Updated weights for policy 1, policy_version 257129 (0.0008) [2023-12-27 07:25:52,302][20347] Updated weights for policy 0, policy_version 257488 (0.0008) [2023-12-27 07:25:52,311][20381] Updated weights for policy 1, policy_version 257139 (0.0007) [2023-12-27 07:25:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 131768320. Throughput: 0: 10358.7, 1: 10393.2. Samples: 131743904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:25:52,363][19170] Avg episode reward: [(0, '946.620'), (1, '1000.000')] [2023-12-27 07:25:52,368][20347] Updated weights for policy 0, policy_version 257498 (0.0007) [2023-12-27 07:25:52,377][20381] Updated weights for policy 1, policy_version 257149 (0.0007) [2023-12-27 07:25:52,426][20347] Updated weights for policy 0, policy_version 257508 (0.0006) [2023-12-27 07:25:52,432][20381] Updated weights for policy 1, policy_version 257159 (0.0006) [2023-12-27 07:25:52,438][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000257160_65847296.pth... [2023-12-27 07:25:52,442][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000255912_65527808.pth [2023-12-27 07:25:52,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000257512_65937408.pth... [2023-12-27 07:25:52,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000256264_65617920.pth [2023-12-27 07:25:53,108][20381] Updated weights for policy 1, policy_version 257169 (0.0005) [2023-12-27 07:25:53,111][20347] Updated weights for policy 0, policy_version 257518 (0.0009) [2023-12-27 07:25:53,153][20381] Updated weights for policy 1, policy_version 257179 (0.0005) [2023-12-27 07:25:53,160][20347] Updated weights for policy 0, policy_version 257528 (0.0010) [2023-12-27 07:25:53,205][20347] Updated weights for policy 0, policy_version 257538 (0.0010) [2023-12-27 07:25:53,207][20381] Updated weights for policy 1, policy_version 257189 (0.0005) [2023-12-27 07:25:53,871][20347] Updated weights for policy 0, policy_version 257548 (0.0008) [2023-12-27 07:25:53,888][20381] Updated weights for policy 1, policy_version 257199 (0.0006) [2023-12-27 07:25:53,934][20347] Updated weights for policy 0, policy_version 257558 (0.0005) [2023-12-27 07:25:53,941][20381] Updated weights for policy 1, policy_version 257209 (0.0007) [2023-12-27 07:25:53,993][20347] Updated weights for policy 0, policy_version 257568 (0.0008) [2023-12-27 07:25:53,995][20381] Updated weights for policy 1, policy_version 257219 (0.0005) [2023-12-27 07:25:54,581][20347] Updated weights for policy 0, policy_version 257578 (0.0010) [2023-12-27 07:25:54,598][20381] Updated weights for policy 1, policy_version 257229 (0.0008) [2023-12-27 07:25:54,636][20347] Updated weights for policy 0, policy_version 257588 (0.0010) [2023-12-27 07:25:54,658][20381] Updated weights for policy 1, policy_version 257239 (0.0011) [2023-12-27 07:25:54,689][20347] Updated weights for policy 0, policy_version 257598 (0.0010) [2023-12-27 07:25:54,705][20381] Updated weights for policy 1, policy_version 257249 (0.0011) [2023-12-27 07:25:54,739][20347] Updated weights for policy 0, policy_version 257608 (0.0010) [2023-12-27 07:25:55,358][20347] Updated weights for policy 0, policy_version 257618 (0.0010) [2023-12-27 07:25:55,376][20381] Updated weights for policy 1, policy_version 257259 (0.0005) [2023-12-27 07:25:55,413][20347] Updated weights for policy 0, policy_version 257628 (0.0010) [2023-12-27 07:25:55,430][20381] Updated weights for policy 1, policy_version 257269 (0.0006) [2023-12-27 07:25:55,468][20347] Updated weights for policy 0, policy_version 257638 (0.0010) [2023-12-27 07:25:55,485][20381] Updated weights for policy 1, policy_version 257279 (0.0005) [2023-12-27 07:25:56,109][20381] Updated weights for policy 1, policy_version 257289 (0.0008) [2023-12-27 07:25:56,135][20347] Updated weights for policy 0, policy_version 257648 (0.0011) [2023-12-27 07:25:56,165][20381] Updated weights for policy 1, policy_version 257299 (0.0006) [2023-12-27 07:25:56,188][20347] Updated weights for policy 0, policy_version 257658 (0.0010) [2023-12-27 07:25:56,225][20381] Updated weights for policy 1, policy_version 257309 (0.0006) [2023-12-27 07:25:56,240][20347] Updated weights for policy 0, policy_version 257668 (0.0010) [2023-12-27 07:25:56,281][20381] Updated weights for policy 1, policy_version 257319 (0.0007) [2023-12-27 07:25:56,877][20347] Updated weights for policy 0, policy_version 257678 (0.0010) [2023-12-27 07:25:56,911][20381] Updated weights for policy 1, policy_version 257329 (0.0011) [2023-12-27 07:25:56,929][20347] Updated weights for policy 0, policy_version 257688 (0.0010) [2023-12-27 07:25:56,965][20381] Updated weights for policy 1, policy_version 257339 (0.0009) [2023-12-27 07:25:56,977][20347] Updated weights for policy 0, policy_version 257698 (0.0010) [2023-12-27 07:25:57,020][20381] Updated weights for policy 1, policy_version 257349 (0.0007) [2023-12-27 07:25:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 131883008. Throughput: 0: 10359.1, 1: 10397.0. Samples: 131870552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:25:57,363][19170] Avg episode reward: [(0, '943.790'), (1, '1000.000')] [2023-12-27 07:25:57,611][20347] Updated weights for policy 0, policy_version 257708 (0.0010) [2023-12-27 07:25:57,662][20347] Updated weights for policy 0, policy_version 257718 (0.0010) [2023-12-27 07:25:57,688][20381] Updated weights for policy 1, policy_version 257359 (0.0007) [2023-12-27 07:25:57,719][20347] Updated weights for policy 0, policy_version 257728 (0.0009) [2023-12-27 07:25:57,740][20381] Updated weights for policy 1, policy_version 257369 (0.0009) [2023-12-27 07:25:57,802][20381] Updated weights for policy 1, policy_version 257379 (0.0011) [2023-12-27 07:25:58,291][20347] Updated weights for policy 0, policy_version 257738 (0.0006) [2023-12-27 07:25:58,356][20347] Updated weights for policy 0, policy_version 257748 (0.0008) [2023-12-27 07:25:58,421][20347] Updated weights for policy 0, policy_version 257758 (0.0007) [2023-12-27 07:25:58,451][20381] Updated weights for policy 1, policy_version 257389 (0.0009) [2023-12-27 07:25:58,479][20347] Updated weights for policy 0, policy_version 257768 (0.0008) [2023-12-27 07:25:58,514][20381] Updated weights for policy 1, policy_version 257399 (0.0008) [2023-12-27 07:25:58,576][20381] Updated weights for policy 1, policy_version 257409 (0.0008) [2023-12-27 07:25:59,193][20347] Updated weights for policy 0, policy_version 257778 (0.0008) [2023-12-27 07:25:59,257][20347] Updated weights for policy 0, policy_version 257788 (0.0008) [2023-12-27 07:25:59,262][20381] Updated weights for policy 1, policy_version 257419 (0.0009) [2023-12-27 07:25:59,318][20381] Updated weights for policy 1, policy_version 257429 (0.0007) [2023-12-27 07:25:59,320][20347] Updated weights for policy 0, policy_version 257798 (0.0006) [2023-12-27 07:25:59,380][20381] Updated weights for policy 1, policy_version 257439 (0.0007) [2023-12-27 07:26:00,009][20347] Updated weights for policy 0, policy_version 257808 (0.0006) [2023-12-27 07:26:00,063][20347] Updated weights for policy 0, policy_version 257818 (0.0007) [2023-12-27 07:26:00,082][20381] Updated weights for policy 1, policy_version 257449 (0.0008) [2023-12-27 07:26:00,120][20347] Updated weights for policy 0, policy_version 257828 (0.0005) [2023-12-27 07:26:00,140][20381] Updated weights for policy 1, policy_version 257459 (0.0007) [2023-12-27 07:26:00,193][20381] Updated weights for policy 1, policy_version 257469 (0.0007) [2023-12-27 07:26:00,246][20381] Updated weights for policy 1, policy_version 257479 (0.0008) [2023-12-27 07:26:00,770][20347] Updated weights for policy 0, policy_version 257838 (0.0008) [2023-12-27 07:26:00,821][20347] Updated weights for policy 0, policy_version 257848 (0.0006) [2023-12-27 07:26:00,876][20347] Updated weights for policy 0, policy_version 257858 (0.0007) [2023-12-27 07:26:00,899][20381] Updated weights for policy 1, policy_version 257489 (0.0007) [2023-12-27 07:26:00,950][20381] Updated weights for policy 1, policy_version 257499 (0.0007) [2023-12-27 07:26:01,005][20381] Updated weights for policy 1, policy_version 257509 (0.0007) [2023-12-27 07:26:01,503][20347] Updated weights for policy 0, policy_version 257868 (0.0006) [2023-12-27 07:26:01,562][20347] Updated weights for policy 0, policy_version 257878 (0.0008) [2023-12-27 07:26:01,624][20347] Updated weights for policy 0, policy_version 257888 (0.0007) [2023-12-27 07:26:01,659][20381] Updated weights for policy 1, policy_version 257519 (0.0007) [2023-12-27 07:26:01,724][20381] Updated weights for policy 1, policy_version 257529 (0.0007) [2023-12-27 07:26:01,775][20381] Updated weights for policy 1, policy_version 257539 (0.0009) [2023-12-27 07:26:02,278][20347] Updated weights for policy 0, policy_version 257898 (0.0007) [2023-12-27 07:26:02,332][20347] Updated weights for policy 0, policy_version 257908 (0.0006) [2023-12-27 07:26:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 131981312. Throughput: 0: 10458.5, 1: 10401.6. Samples: 131996052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:02,363][19170] Avg episode reward: [(0, '862.190'), (1, '1000.000')] [2023-12-27 07:26:02,398][20347] Updated weights for policy 0, policy_version 257918 (0.0008) [2023-12-27 07:26:02,450][20381] Updated weights for policy 1, policy_version 257549 (0.0007) [2023-12-27 07:26:02,457][20347] Updated weights for policy 0, policy_version 257928 (0.0007) [2023-12-27 07:26:02,496][20381] Updated weights for policy 1, policy_version 257559 (0.0008) [2023-12-27 07:26:02,548][20381] Updated weights for policy 1, policy_version 257569 (0.0007) [2023-12-27 07:26:03,114][20347] Updated weights for policy 0, policy_version 257938 (0.0007) [2023-12-27 07:26:03,169][20347] Updated weights for policy 0, policy_version 257948 (0.0008) [2023-12-27 07:26:03,208][20381] Updated weights for policy 1, policy_version 257579 (0.0009) [2023-12-27 07:26:03,225][20347] Updated weights for policy 0, policy_version 257958 (0.0006) [2023-12-27 07:26:03,262][20381] Updated weights for policy 1, policy_version 257589 (0.0007) [2023-12-27 07:26:03,317][20381] Updated weights for policy 1, policy_version 257599 (0.0010) [2023-12-27 07:26:03,955][20347] Updated weights for policy 0, policy_version 257968 (0.0008) [2023-12-27 07:26:03,981][20381] Updated weights for policy 1, policy_version 257609 (0.0010) [2023-12-27 07:26:04,012][20347] Updated weights for policy 0, policy_version 257978 (0.0008) [2023-12-27 07:26:04,039][20381] Updated weights for policy 1, policy_version 257619 (0.0011) [2023-12-27 07:26:04,068][20347] Updated weights for policy 0, policy_version 257988 (0.0008) [2023-12-27 07:26:04,099][20381] Updated weights for policy 1, policy_version 257629 (0.0011) [2023-12-27 07:26:04,167][20381] Updated weights for policy 1, policy_version 257639 (0.0011) [2023-12-27 07:26:04,774][20347] Updated weights for policy 0, policy_version 257998 (0.0006) [2023-12-27 07:26:04,811][20381] Updated weights for policy 1, policy_version 257649 (0.0006) [2023-12-27 07:26:04,829][20347] Updated weights for policy 0, policy_version 258008 (0.0005) [2023-12-27 07:26:04,871][20381] Updated weights for policy 1, policy_version 257659 (0.0006) [2023-12-27 07:26:04,892][20347] Updated weights for policy 0, policy_version 258018 (0.0006) [2023-12-27 07:26:04,919][20381] Updated weights for policy 1, policy_version 257669 (0.0005) [2023-12-27 07:26:05,514][20381] Updated weights for policy 1, policy_version 257679 (0.0009) [2023-12-27 07:26:05,531][20347] Updated weights for policy 0, policy_version 258028 (0.0006) [2023-12-27 07:26:05,579][20381] Updated weights for policy 1, policy_version 257689 (0.0010) [2023-12-27 07:26:05,582][20347] Updated weights for policy 0, policy_version 258038 (0.0006) [2023-12-27 07:26:05,638][20381] Updated weights for policy 1, policy_version 257699 (0.0010) [2023-12-27 07:26:05,640][20347] Updated weights for policy 0, policy_version 258048 (0.0007) [2023-12-27 07:26:06,287][20381] Updated weights for policy 1, policy_version 257709 (0.0009) [2023-12-27 07:26:06,347][20381] Updated weights for policy 1, policy_version 257719 (0.0008) [2023-12-27 07:26:06,376][20347] Updated weights for policy 0, policy_version 258058 (0.0009) [2023-12-27 07:26:06,405][20381] Updated weights for policy 1, policy_version 257729 (0.0006) [2023-12-27 07:26:06,437][20347] Updated weights for policy 0, policy_version 258068 (0.0006) [2023-12-27 07:26:06,497][20347] Updated weights for policy 0, policy_version 258078 (0.0008) [2023-12-27 07:26:06,554][20347] Updated weights for policy 0, policy_version 258088 (0.0008) [2023-12-27 07:26:07,018][20381] Updated weights for policy 1, policy_version 257739 (0.0006) [2023-12-27 07:26:07,082][20381] Updated weights for policy 1, policy_version 257749 (0.0006) [2023-12-27 07:26:07,143][20381] Updated weights for policy 1, policy_version 257759 (0.0007) [2023-12-27 07:26:07,259][20347] Updated weights for policy 0, policy_version 258098 (0.0007) [2023-12-27 07:26:07,320][20347] Updated weights for policy 0, policy_version 258108 (0.0008) [2023-12-27 07:26:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 132087808. Throughput: 0: 10452.6, 1: 10409.9. Samples: 132058508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:07,363][19170] Avg episode reward: [(0, '892.580'), (1, '1000.000')] [2023-12-27 07:26:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000257768_66002944.pth... [2023-12-27 07:26:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000256552_65691648.pth [2023-12-27 07:26:07,389][20347] Updated weights for policy 0, policy_version 258118 (0.0008) [2023-12-27 07:26:07,399][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000258120_66093056.pth... [2023-12-27 07:26:07,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000256904_65781760.pth [2023-12-27 07:26:07,798][20381] Updated weights for policy 1, policy_version 257769 (0.0008) [2023-12-27 07:26:07,858][20381] Updated weights for policy 1, policy_version 257779 (0.0008) [2023-12-27 07:26:07,915][20381] Updated weights for policy 1, policy_version 257789 (0.0008) [2023-12-27 07:26:07,964][20381] Updated weights for policy 1, policy_version 257799 (0.0008) [2023-12-27 07:26:08,033][20347] Updated weights for policy 0, policy_version 258128 (0.0007) [2023-12-27 07:26:08,096][20347] Updated weights for policy 0, policy_version 258138 (0.0007) [2023-12-27 07:26:08,158][20347] Updated weights for policy 0, policy_version 258148 (0.0009) [2023-12-27 07:26:08,684][20381] Updated weights for policy 1, policy_version 257809 (0.0007) [2023-12-27 07:26:08,740][20381] Updated weights for policy 1, policy_version 257819 (0.0008) [2023-12-27 07:26:08,796][20381] Updated weights for policy 1, policy_version 257829 (0.0008) [2023-12-27 07:26:08,854][20347] Updated weights for policy 0, policy_version 258158 (0.0008) [2023-12-27 07:26:08,909][20347] Updated weights for policy 0, policy_version 258168 (0.0007) [2023-12-27 07:26:08,965][20347] Updated weights for policy 0, policy_version 258178 (0.0009) [2023-12-27 07:26:09,556][20381] Updated weights for policy 1, policy_version 257839 (0.0008) [2023-12-27 07:26:09,617][20381] Updated weights for policy 1, policy_version 257849 (0.0007) [2023-12-27 07:26:09,650][20347] Updated weights for policy 0, policy_version 258188 (0.0010) [2023-12-27 07:26:09,678][20381] Updated weights for policy 1, policy_version 257859 (0.0006) [2023-12-27 07:26:09,715][20347] Updated weights for policy 0, policy_version 258198 (0.0008) [2023-12-27 07:26:09,774][20347] Updated weights for policy 0, policy_version 258208 (0.0007) [2023-12-27 07:26:10,435][20381] Updated weights for policy 1, policy_version 257869 (0.0007) [2023-12-27 07:26:10,450][20347] Updated weights for policy 0, policy_version 258218 (0.0008) [2023-12-27 07:26:10,484][20381] Updated weights for policy 1, policy_version 257879 (0.0008) [2023-12-27 07:26:10,508][20347] Updated weights for policy 0, policy_version 258228 (0.0005) [2023-12-27 07:26:10,534][20381] Updated weights for policy 1, policy_version 257889 (0.0007) [2023-12-27 07:26:10,573][20347] Updated weights for policy 0, policy_version 258238 (0.0006) [2023-12-27 07:26:10,631][20347] Updated weights for policy 0, policy_version 258248 (0.0008) [2023-12-27 07:26:11,287][20347] Updated weights for policy 0, policy_version 258258 (0.0007) [2023-12-27 07:26:11,299][20381] Updated weights for policy 1, policy_version 257899 (0.0006) [2023-12-27 07:26:11,350][20347] Updated weights for policy 0, policy_version 258268 (0.0008) [2023-12-27 07:26:11,367][20381] Updated weights for policy 1, policy_version 257909 (0.0008) [2023-12-27 07:26:11,402][20347] Updated weights for policy 0, policy_version 258278 (0.0006) [2023-12-27 07:26:11,422][20381] Updated weights for policy 1, policy_version 257919 (0.0008) [2023-12-27 07:26:12,121][20347] Updated weights for policy 0, policy_version 258288 (0.0007) [2023-12-27 07:26:12,131][20381] Updated weights for policy 1, policy_version 257929 (0.0008) [2023-12-27 07:26:12,171][20347] Updated weights for policy 0, policy_version 258298 (0.0006) [2023-12-27 07:26:12,191][20381] Updated weights for policy 1, policy_version 257939 (0.0006) [2023-12-27 07:26:12,222][20347] Updated weights for policy 0, policy_version 258308 (0.0005) [2023-12-27 07:26:12,246][20381] Updated weights for policy 1, policy_version 257949 (0.0007) [2023-12-27 07:26:12,306][20381] Updated weights for policy 1, policy_version 257959 (0.0008) [2023-12-27 07:26:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 132194304. Throughput: 0: 10398.4, 1: 10341.8. Samples: 132179264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:12,363][19170] Avg episode reward: [(0, '967.020'), (1, '1000.000')] [2023-12-27 07:26:12,932][20347] Updated weights for policy 0, policy_version 258318 (0.0006) [2023-12-27 07:26:12,987][20347] Updated weights for policy 0, policy_version 258328 (0.0006) [2023-12-27 07:26:12,989][20381] Updated weights for policy 1, policy_version 257969 (0.0007) [2023-12-27 07:26:13,037][20381] Updated weights for policy 1, policy_version 257979 (0.0005) [2023-12-27 07:26:13,043][20347] Updated weights for policy 0, policy_version 258338 (0.0007) [2023-12-27 07:26:13,094][20381] Updated weights for policy 1, policy_version 257989 (0.0008) [2023-12-27 07:26:13,699][20347] Updated weights for policy 0, policy_version 258348 (0.0006) [2023-12-27 07:26:13,748][20347] Updated weights for policy 0, policy_version 258358 (0.0007) [2023-12-27 07:26:13,803][20347] Updated weights for policy 0, policy_version 258368 (0.0008) [2023-12-27 07:26:13,813][20381] Updated weights for policy 1, policy_version 257999 (0.0007) [2023-12-27 07:26:13,868][20381] Updated weights for policy 1, policy_version 258009 (0.0006) [2023-12-27 07:26:13,922][20381] Updated weights for policy 1, policy_version 258019 (0.0007) [2023-12-27 07:26:14,511][20347] Updated weights for policy 0, policy_version 258378 (0.0007) [2023-12-27 07:26:14,569][20347] Updated weights for policy 0, policy_version 258388 (0.0007) [2023-12-27 07:26:14,603][20381] Updated weights for policy 1, policy_version 258029 (0.0007) [2023-12-27 07:26:14,624][20347] Updated weights for policy 0, policy_version 258398 (0.0006) [2023-12-27 07:26:14,653][20381] Updated weights for policy 1, policy_version 258039 (0.0005) [2023-12-27 07:26:14,677][20347] Updated weights for policy 0, policy_version 258408 (0.0006) [2023-12-27 07:26:14,698][20381] Updated weights for policy 1, policy_version 258049 (0.0006) [2023-12-27 07:26:15,331][20347] Updated weights for policy 0, policy_version 258418 (0.0006) [2023-12-27 07:26:15,394][20347] Updated weights for policy 0, policy_version 258428 (0.0005) [2023-12-27 07:26:15,429][20381] Updated weights for policy 1, policy_version 258059 (0.0007) [2023-12-27 07:26:15,457][20347] Updated weights for policy 0, policy_version 258438 (0.0005) [2023-12-27 07:26:15,487][20381] Updated weights for policy 1, policy_version 258069 (0.0007) [2023-12-27 07:26:15,549][20381] Updated weights for policy 1, policy_version 258079 (0.0008) [2023-12-27 07:26:15,969][20347] Updated weights for policy 0, policy_version 258448 (0.0007) [2023-12-27 07:26:16,018][20347] Updated weights for policy 0, policy_version 258458 (0.0008) [2023-12-27 07:26:16,070][20347] Updated weights for policy 0, policy_version 258468 (0.0008) [2023-12-27 07:26:16,257][20381] Updated weights for policy 1, policy_version 258089 (0.0008) [2023-12-27 07:26:16,315][20381] Updated weights for policy 1, policy_version 258099 (0.0008) [2023-12-27 07:26:16,365][20381] Updated weights for policy 1, policy_version 258109 (0.0007) [2023-12-27 07:26:16,427][20381] Updated weights for policy 1, policy_version 258119 (0.0007) [2023-12-27 07:26:16,735][20347] Updated weights for policy 0, policy_version 258478 (0.0007) [2023-12-27 07:26:16,793][20347] Updated weights for policy 0, policy_version 258488 (0.0006) [2023-12-27 07:26:16,845][20347] Updated weights for policy 0, policy_version 258498 (0.0010) [2023-12-27 07:26:17,147][20381] Updated weights for policy 1, policy_version 258129 (0.0005) [2023-12-27 07:26:17,206][20381] Updated weights for policy 1, policy_version 258139 (0.0006) [2023-12-27 07:26:17,262][20381] Updated weights for policy 1, policy_version 258149 (0.0008) [2023-12-27 07:26:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 132292608. Throughput: 0: 10357.3, 1: 10268.4. Samples: 132301068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:17,363][19170] Avg episode reward: [(0, '997.210'), (1, '1000.000')] [2023-12-27 07:26:17,481][20347] Updated weights for policy 0, policy_version 258508 (0.0010) [2023-12-27 07:26:17,527][20347] Updated weights for policy 0, policy_version 258518 (0.0010) [2023-12-27 07:26:17,572][20347] Updated weights for policy 0, policy_version 258528 (0.0010) [2023-12-27 07:26:17,924][20381] Updated weights for policy 1, policy_version 258159 (0.0008) [2023-12-27 07:26:17,972][20381] Updated weights for policy 1, policy_version 258169 (0.0007) [2023-12-27 07:26:18,022][20381] Updated weights for policy 1, policy_version 258179 (0.0007) [2023-12-27 07:26:18,280][20347] Updated weights for policy 0, policy_version 258538 (0.0010) [2023-12-27 07:26:18,342][20347] Updated weights for policy 0, policy_version 258548 (0.0010) [2023-12-27 07:26:18,397][20347] Updated weights for policy 0, policy_version 258558 (0.0007) [2023-12-27 07:26:18,456][20347] Updated weights for policy 0, policy_version 258568 (0.0008) [2023-12-27 07:26:18,747][20381] Updated weights for policy 1, policy_version 258189 (0.0008) [2023-12-27 07:26:18,806][20381] Updated weights for policy 1, policy_version 258199 (0.0007) [2023-12-27 07:26:18,861][20381] Updated weights for policy 1, policy_version 258209 (0.0007) [2023-12-27 07:26:19,154][20347] Updated weights for policy 0, policy_version 258578 (0.0010) [2023-12-27 07:26:19,214][20347] Updated weights for policy 0, policy_version 258588 (0.0010) [2023-12-27 07:26:19,278][20347] Updated weights for policy 0, policy_version 258598 (0.0011) [2023-12-27 07:26:19,596][20381] Updated weights for policy 1, policy_version 258219 (0.0007) [2023-12-27 07:26:19,658][20381] Updated weights for policy 1, policy_version 258229 (0.0007) [2023-12-27 07:26:19,721][20381] Updated weights for policy 1, policy_version 258239 (0.0007) [2023-12-27 07:26:19,999][20347] Updated weights for policy 0, policy_version 258608 (0.0007) [2023-12-27 07:26:20,059][20347] Updated weights for policy 0, policy_version 258618 (0.0006) [2023-12-27 07:26:20,110][20347] Updated weights for policy 0, policy_version 258628 (0.0006) [2023-12-27 07:26:20,443][20381] Updated weights for policy 1, policy_version 258249 (0.0008) [2023-12-27 07:26:20,498][20381] Updated weights for policy 1, policy_version 258259 (0.0008) [2023-12-27 07:26:20,552][20381] Updated weights for policy 1, policy_version 258269 (0.0007) [2023-12-27 07:26:20,612][20381] Updated weights for policy 1, policy_version 258279 (0.0008) [2023-12-27 07:26:20,736][20347] Updated weights for policy 0, policy_version 258638 (0.0008) [2023-12-27 07:26:20,793][20347] Updated weights for policy 0, policy_version 258648 (0.0007) [2023-12-27 07:26:20,856][20347] Updated weights for policy 0, policy_version 258658 (0.0007) [2023-12-27 07:26:21,393][20381] Updated weights for policy 1, policy_version 258289 (0.0008) [2023-12-27 07:26:21,448][20381] Updated weights for policy 1, policy_version 258299 (0.0009) [2023-12-27 07:26:21,501][20381] Updated weights for policy 1, policy_version 258309 (0.0009) [2023-12-27 07:26:21,567][20347] Updated weights for policy 0, policy_version 258668 (0.0008) [2023-12-27 07:26:21,627][20347] Updated weights for policy 0, policy_version 258678 (0.0010) [2023-12-27 07:26:21,694][20347] Updated weights for policy 0, policy_version 258688 (0.0010) [2023-12-27 07:26:22,292][20381] Updated weights for policy 1, policy_version 258319 (0.0008) [2023-12-27 07:26:22,345][20347] Updated weights for policy 0, policy_version 258698 (0.0011) [2023-12-27 07:26:22,356][20381] Updated weights for policy 1, policy_version 258329 (0.0008) [2023-12-27 07:26:22,362][19170] Fps is (10 sec: 18841.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 132382720. Throughput: 0: 10296.6, 1: 10237.4. Samples: 132360636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:22,363][19170] Avg episode reward: [(0, '998.770'), (1, '1000.000')] [2023-12-27 07:26:22,408][20347] Updated weights for policy 0, policy_version 258708 (0.0010) [2023-12-27 07:26:22,419][20381] Updated weights for policy 1, policy_version 258339 (0.0007) [2023-12-27 07:26:22,450][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000258344_66150400.pth... [2023-12-27 07:26:22,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000257160_65847296.pth [2023-12-27 07:26:22,470][20347] Updated weights for policy 0, policy_version 258718 (0.0008) [2023-12-27 07:26:22,530][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000258728_66248704.pth... [2023-12-27 07:26:22,532][20347] Updated weights for policy 0, policy_version 258728 (0.0008) [2023-12-27 07:26:22,533][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000257512_65937408.pth [2023-12-27 07:26:23,119][20381] Updated weights for policy 1, policy_version 258349 (0.0008) [2023-12-27 07:26:23,173][20381] Updated weights for policy 1, policy_version 258359 (0.0011) [2023-12-27 07:26:23,236][20381] Updated weights for policy 1, policy_version 258369 (0.0010) [2023-12-27 07:26:23,249][20347] Updated weights for policy 0, policy_version 258738 (0.0011) [2023-12-27 07:26:23,295][20347] Updated weights for policy 0, policy_version 258748 (0.0010) [2023-12-27 07:26:23,340][20347] Updated weights for policy 0, policy_version 258758 (0.0010) [2023-12-27 07:26:23,916][20381] Updated weights for policy 1, policy_version 258379 (0.0011) [2023-12-27 07:26:23,976][20381] Updated weights for policy 1, policy_version 258389 (0.0009) [2023-12-27 07:26:24,029][20381] Updated weights for policy 1, policy_version 258399 (0.0006) [2023-12-27 07:26:24,058][20347] Updated weights for policy 0, policy_version 258768 (0.0008) [2023-12-27 07:26:24,110][20347] Updated weights for policy 0, policy_version 258778 (0.0008) [2023-12-27 07:26:24,166][20347] Updated weights for policy 0, policy_version 258788 (0.0007) [2023-12-27 07:26:24,669][20381] Updated weights for policy 1, policy_version 258409 (0.0006) [2023-12-27 07:26:24,717][20381] Updated weights for policy 1, policy_version 258419 (0.0010) [2023-12-27 07:26:24,765][20381] Updated weights for policy 1, policy_version 258429 (0.0010) [2023-12-27 07:26:24,813][20381] Updated weights for policy 1, policy_version 258439 (0.0010) [2023-12-27 07:26:24,862][20347] Updated weights for policy 0, policy_version 258798 (0.0007) [2023-12-27 07:26:24,912][20347] Updated weights for policy 0, policy_version 258808 (0.0007) [2023-12-27 07:26:24,958][20347] Updated weights for policy 0, policy_version 258818 (0.0007) [2023-12-27 07:26:25,502][20381] Updated weights for policy 1, policy_version 258449 (0.0008) [2023-12-27 07:26:25,558][20381] Updated weights for policy 1, policy_version 258459 (0.0008) [2023-12-27 07:26:25,619][20381] Updated weights for policy 1, policy_version 258469 (0.0008) [2023-12-27 07:26:25,666][20347] Updated weights for policy 0, policy_version 258828 (0.0006) [2023-12-27 07:26:25,720][20347] Updated weights for policy 0, policy_version 258838 (0.0005) [2023-12-27 07:26:25,777][20347] Updated weights for policy 0, policy_version 258848 (0.0005) [2023-12-27 07:26:26,358][20381] Updated weights for policy 1, policy_version 258479 (0.0007) [2023-12-27 07:26:26,400][20347] Updated weights for policy 0, policy_version 258858 (0.0006) [2023-12-27 07:26:26,417][20381] Updated weights for policy 1, policy_version 258489 (0.0007) [2023-12-27 07:26:26,458][20347] Updated weights for policy 0, policy_version 258868 (0.0010) [2023-12-27 07:26:26,474][20381] Updated weights for policy 1, policy_version 258499 (0.0005) [2023-12-27 07:26:26,512][20347] Updated weights for policy 0, policy_version 258878 (0.0010) [2023-12-27 07:26:26,566][20347] Updated weights for policy 0, policy_version 258888 (0.0010) [2023-12-27 07:26:27,132][20381] Updated weights for policy 1, policy_version 258509 (0.0005) [2023-12-27 07:26:27,190][20381] Updated weights for policy 1, policy_version 258519 (0.0005) [2023-12-27 07:26:27,253][20381] Updated weights for policy 1, policy_version 258529 (0.0005) [2023-12-27 07:26:27,258][20347] Updated weights for policy 0, policy_version 258898 (0.0008) [2023-12-27 07:26:27,324][20347] Updated weights for policy 0, policy_version 258908 (0.0007) [2023-12-27 07:26:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 132489216. Throughput: 0: 10296.0, 1: 10158.9. Samples: 132480320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:27,363][19170] Avg episode reward: [(0, '994.070'), (1, '971.210')] [2023-12-27 07:26:27,378][20347] Updated weights for policy 0, policy_version 258918 (0.0006) [2023-12-27 07:26:27,840][20381] Updated weights for policy 1, policy_version 258539 (0.0005) [2023-12-27 07:26:27,885][20381] Updated weights for policy 1, policy_version 258549 (0.0005) [2023-12-27 07:26:27,941][20381] Updated weights for policy 1, policy_version 258559 (0.0006) [2023-12-27 07:26:28,022][20347] Updated weights for policy 0, policy_version 258928 (0.0006) [2023-12-27 07:26:28,081][20347] Updated weights for policy 0, policy_version 258938 (0.0008) [2023-12-27 07:26:28,140][20347] Updated weights for policy 0, policy_version 258948 (0.0006) [2023-12-27 07:26:28,503][20381] Updated weights for policy 1, policy_version 258569 (0.0007) [2023-12-27 07:26:28,565][20381] Updated weights for policy 1, policy_version 258579 (0.0006) [2023-12-27 07:26:28,624][20381] Updated weights for policy 1, policy_version 258589 (0.0006) [2023-12-27 07:26:28,683][20381] Updated weights for policy 1, policy_version 258599 (0.0005) [2023-12-27 07:26:28,806][20347] Updated weights for policy 0, policy_version 258958 (0.0005) [2023-12-27 07:26:28,865][20347] Updated weights for policy 0, policy_version 258968 (0.0005) [2023-12-27 07:26:28,922][20347] Updated weights for policy 0, policy_version 258978 (0.0007) [2023-12-27 07:26:29,297][20381] Updated weights for policy 1, policy_version 258609 (0.0006) [2023-12-27 07:26:29,337][20292] KL-divergence is very high: 430.6776 [2023-12-27 07:26:29,365][20381] Updated weights for policy 1, policy_version 258619 (0.0007) [2023-12-27 07:26:29,386][20292] KL-divergence is very high: 195.1680 [2023-12-27 07:26:29,416][20381] Updated weights for policy 1, policy_version 258629 (0.0008) [2023-12-27 07:26:29,571][20347] Updated weights for policy 0, policy_version 258988 (0.0007) [2023-12-27 07:26:29,619][20347] Updated weights for policy 0, policy_version 258998 (0.0007) [2023-12-27 07:26:29,661][20347] Updated weights for policy 0, policy_version 259008 (0.0007) [2023-12-27 07:26:30,086][20381] Updated weights for policy 1, policy_version 258639 (0.0006) [2023-12-27 07:26:30,095][20292] KL-divergence is very high: 111.9837 [2023-12-27 07:26:30,131][20381] Updated weights for policy 1, policy_version 258649 (0.0008) [2023-12-27 07:26:30,132][20292] KL-divergence is very high: 226.0688 [2023-12-27 07:26:30,177][20381] Updated weights for policy 1, policy_version 258659 (0.0007) [2023-12-27 07:26:30,335][20347] Updated weights for policy 0, policy_version 259018 (0.0007) [2023-12-27 07:26:30,399][20347] Updated weights for policy 0, policy_version 259028 (0.0006) [2023-12-27 07:26:30,456][20347] Updated weights for policy 0, policy_version 259038 (0.0005) [2023-12-27 07:26:30,507][20347] Updated weights for policy 0, policy_version 259048 (0.0008) [2023-12-27 07:26:30,849][20381] Updated weights for policy 1, policy_version 258669 (0.0006) [2023-12-27 07:26:30,867][20292] KL-divergence is very high: 245.3630 [2023-12-27 07:26:30,901][20381] Updated weights for policy 1, policy_version 258679 (0.0005) [2023-12-27 07:26:30,953][20381] Updated weights for policy 1, policy_version 258689 (0.0005) [2023-12-27 07:26:30,960][20292] KL-divergence is very high: 204.9371 [2023-12-27 07:26:31,080][20347] Updated weights for policy 0, policy_version 259058 (0.0009) [2023-12-27 07:26:31,139][20347] Updated weights for policy 0, policy_version 259068 (0.0009) [2023-12-27 07:26:31,185][20347] Updated weights for policy 0, policy_version 259078 (0.0007) [2023-12-27 07:26:31,568][20381] Updated weights for policy 1, policy_version 258699 (0.0005) [2023-12-27 07:26:31,596][20292] KL-divergence is very high: 487.2940 [2023-12-27 07:26:31,628][20381] Updated weights for policy 1, policy_version 258709 (0.0007) [2023-12-27 07:26:31,687][20381] Updated weights for policy 1, policy_version 258719 (0.0007) [2023-12-27 07:26:31,902][20347] Updated weights for policy 0, policy_version 259088 (0.0009) [2023-12-27 07:26:31,958][20347] Updated weights for policy 0, policy_version 259098 (0.0011) [2023-12-27 07:26:32,017][20347] Updated weights for policy 0, policy_version 259108 (0.0010) [2023-12-27 07:26:32,320][20381] Updated weights for policy 1, policy_version 258729 (0.0007) [2023-12-27 07:26:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 132595712. Throughput: 0: 10344.5, 1: 10221.3. Samples: 132608536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:32,363][19170] Avg episode reward: [(0, '959.330'), (1, '867.540')] [2023-12-27 07:26:32,380][20381] Updated weights for policy 1, policy_version 258739 (0.0008) [2023-12-27 07:26:32,435][20381] Updated weights for policy 1, policy_version 258749 (0.0008) [2023-12-27 07:26:32,504][20381] Updated weights for policy 1, policy_version 258759 (0.0007) [2023-12-27 07:26:32,646][20347] Updated weights for policy 0, policy_version 259118 (0.0010) [2023-12-27 07:26:32,703][20347] Updated weights for policy 0, policy_version 259128 (0.0010) [2023-12-27 07:26:32,754][20347] Updated weights for policy 0, policy_version 259138 (0.0010) [2023-12-27 07:26:33,211][20381] Updated weights for policy 1, policy_version 258769 (0.0008) [2023-12-27 07:26:33,269][20381] Updated weights for policy 1, policy_version 258779 (0.0008) [2023-12-27 07:26:33,328][20381] Updated weights for policy 1, policy_version 258789 (0.0007) [2023-12-27 07:26:33,391][20347] Updated weights for policy 0, policy_version 259148 (0.0009) [2023-12-27 07:26:33,440][20347] Updated weights for policy 0, policy_version 259158 (0.0007) [2023-12-27 07:26:33,486][20347] Updated weights for policy 0, policy_version 259168 (0.0005) [2023-12-27 07:26:34,022][20381] Updated weights for policy 1, policy_version 258799 (0.0008) [2023-12-27 07:26:34,074][20381] Updated weights for policy 1, policy_version 258809 (0.0009) [2023-12-27 07:26:34,100][20347] Updated weights for policy 0, policy_version 259178 (0.0005) [2023-12-27 07:26:34,134][20381] Updated weights for policy 1, policy_version 258819 (0.0010) [2023-12-27 07:26:34,167][20347] Updated weights for policy 0, policy_version 259188 (0.0007) [2023-12-27 07:26:34,235][20347] Updated weights for policy 0, policy_version 259198 (0.0007) [2023-12-27 07:26:34,303][20347] Updated weights for policy 0, policy_version 259208 (0.0008) [2023-12-27 07:26:34,865][20381] Updated weights for policy 1, policy_version 258829 (0.0011) [2023-12-27 07:26:34,914][20381] Updated weights for policy 1, policy_version 258839 (0.0011) [2023-12-27 07:26:34,940][20347] Updated weights for policy 0, policy_version 259218 (0.0006) [2023-12-27 07:26:34,967][20292] KL-divergence is very high: 165.6260 [2023-12-27 07:26:34,974][20381] Updated weights for policy 1, policy_version 258849 (0.0011) [2023-12-27 07:26:34,992][20347] Updated weights for policy 0, policy_version 259228 (0.0006) [2023-12-27 07:26:35,001][20292] KL-divergence is very high: 477.9733 [2023-12-27 07:26:35,048][20347] Updated weights for policy 0, policy_version 259238 (0.0007) [2023-12-27 07:26:35,630][20381] Updated weights for policy 1, policy_version 258859 (0.0009) [2023-12-27 07:26:35,664][20292] KL-divergence is very high: 196.2025 [2023-12-27 07:26:35,681][20381] Updated weights for policy 1, policy_version 258869 (0.0009) [2023-12-27 07:26:35,703][20292] KL-divergence is very high: 320.9797 [2023-12-27 07:26:35,718][20292] KL-divergence is very high: 140.2435 [2023-12-27 07:26:35,729][20381] Updated weights for policy 1, policy_version 258879 (0.0010) [2023-12-27 07:26:35,742][20347] Updated weights for policy 0, policy_version 259248 (0.0007) [2023-12-27 07:26:35,742][20292] KL-divergence is very high: 110.2212 [2023-12-27 07:26:35,752][20292] KL-divergence is very high: 109.3273 [2023-12-27 07:26:35,798][20347] Updated weights for policy 0, policy_version 259258 (0.0008) [2023-12-27 07:26:35,856][20347] Updated weights for policy 0, policy_version 259268 (0.0008) [2023-12-27 07:26:36,397][20381] Updated weights for policy 1, policy_version 258889 (0.0008) [2023-12-27 07:26:36,436][20292] KL-divergence is very high: 153.5287 [2023-12-27 07:26:36,456][20292] KL-divergence is very high: 162.5347 [2023-12-27 07:26:36,463][20381] Updated weights for policy 1, policy_version 258899 (0.0011) [2023-12-27 07:26:36,489][20292] KL-divergence is very high: 117.0692 [2023-12-27 07:26:36,507][20292] KL-divergence is very high: 104.1687 [2023-12-27 07:26:36,525][20381] Updated weights for policy 1, policy_version 258909 (0.0011) [2023-12-27 07:26:36,536][20292] KL-divergence is very high: 134.9083 [2023-12-27 07:26:36,545][20347] Updated weights for policy 0, policy_version 259278 (0.0006) [2023-12-27 07:26:36,549][20292] KL-divergence is very high: 132.3668 [2023-12-27 07:26:36,584][20381] Updated weights for policy 1, policy_version 258919 (0.0011) [2023-12-27 07:26:36,606][20347] Updated weights for policy 0, policy_version 259288 (0.0006) [2023-12-27 07:26:36,661][20347] Updated weights for policy 0, policy_version 259298 (0.0007) [2023-12-27 07:26:37,209][20292] KL-divergence is very high: 182.2551 [2023-12-27 07:26:37,248][20381] Updated weights for policy 1, policy_version 258929 (0.0010) [2023-12-27 07:26:37,260][20292] KL-divergence is very high: 169.5504 [2023-12-27 07:26:37,303][20347] Updated weights for policy 0, policy_version 259308 (0.0006) [2023-12-27 07:26:37,305][20292] KL-divergence is very high: 194.9119 [2023-12-27 07:26:37,306][20381] Updated weights for policy 1, policy_version 258939 (0.0011) [2023-12-27 07:26:37,315][20292] KL-divergence is very high: 123.2849 [2023-12-27 07:26:37,344][20292] KL-divergence is very high: 295.1006 [2023-12-27 07:26:37,355][20381] Updated weights for policy 1, policy_version 258949 (0.0011) [2023-12-27 07:26:37,355][20347] Updated weights for policy 0, policy_version 259318 (0.0006) [2023-12-27 07:26:37,355][20292] KL-divergence is very high: 148.8339 [2023-12-27 07:26:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 132694016. Throughput: 0: 10389.1, 1: 10202.2. Samples: 132670512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:37,363][19170] Avg episode reward: [(0, '869.160'), (1, '781.620')] [2023-12-27 07:26:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000258952_66306048.pth... [2023-12-27 07:26:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000257768_66002944.pth [2023-12-27 07:26:37,406][20347] Updated weights for policy 0, policy_version 259328 (0.0005) [2023-12-27 07:26:37,440][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000259336_66404352.pth... [2023-12-27 07:26:37,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000258120_66093056.pth [2023-12-27 07:26:38,035][20381] Updated weights for policy 1, policy_version 258959 (0.0007) [2023-12-27 07:26:38,086][20381] Updated weights for policy 1, policy_version 258969 (0.0005) [2023-12-27 07:26:38,103][20347] Updated weights for policy 0, policy_version 259338 (0.0007) [2023-12-27 07:26:38,140][20381] Updated weights for policy 1, policy_version 258979 (0.0005) [2023-12-27 07:26:38,148][20347] Updated weights for policy 0, policy_version 259348 (0.0008) [2023-12-27 07:26:38,195][20347] Updated weights for policy 0, policy_version 259358 (0.0007) [2023-12-27 07:26:38,247][20347] Updated weights for policy 0, policy_version 259368 (0.0009) [2023-12-27 07:26:38,719][20381] Updated weights for policy 1, policy_version 258989 (0.0007) [2023-12-27 07:26:38,769][20381] Updated weights for policy 1, policy_version 258999 (0.0008) [2023-12-27 07:26:38,821][20381] Updated weights for policy 1, policy_version 259009 (0.0007) [2023-12-27 07:26:38,976][20347] Updated weights for policy 0, policy_version 259378 (0.0008) [2023-12-27 07:26:39,027][20347] Updated weights for policy 0, policy_version 259388 (0.0007) [2023-12-27 07:26:39,079][20347] Updated weights for policy 0, policy_version 259398 (0.0008) [2023-12-27 07:26:39,514][20381] Updated weights for policy 1, policy_version 259019 (0.0008) [2023-12-27 07:26:39,575][20381] Updated weights for policy 1, policy_version 259029 (0.0009) [2023-12-27 07:26:39,627][20381] Updated weights for policy 1, policy_version 259039 (0.0007) [2023-12-27 07:26:39,710][20347] Updated weights for policy 0, policy_version 259408 (0.0006) [2023-12-27 07:26:39,778][20347] Updated weights for policy 0, policy_version 259418 (0.0006) [2023-12-27 07:26:39,846][20347] Updated weights for policy 0, policy_version 259428 (0.0008) [2023-12-27 07:26:40,329][20381] Updated weights for policy 1, policy_version 259049 (0.0006) [2023-12-27 07:26:40,383][20381] Updated weights for policy 1, policy_version 259059 (0.0009) [2023-12-27 07:26:40,433][20381] Updated weights for policy 1, policy_version 259069 (0.0007) [2023-12-27 07:26:40,485][20381] Updated weights for policy 1, policy_version 259079 (0.0007) [2023-12-27 07:26:40,499][20347] Updated weights for policy 0, policy_version 259438 (0.0007) [2023-12-27 07:26:40,555][20347] Updated weights for policy 0, policy_version 259448 (0.0008) [2023-12-27 07:26:40,614][20347] Updated weights for policy 0, policy_version 259458 (0.0007) [2023-12-27 07:26:41,220][20381] Updated weights for policy 1, policy_version 259089 (0.0007) [2023-12-27 07:26:41,239][20347] Updated weights for policy 0, policy_version 259468 (0.0007) [2023-12-27 07:26:41,280][20381] Updated weights for policy 1, policy_version 259099 (0.0008) [2023-12-27 07:26:41,292][20347] Updated weights for policy 0, policy_version 259478 (0.0006) [2023-12-27 07:26:41,341][20381] Updated weights for policy 1, policy_version 259109 (0.0009) [2023-12-27 07:26:41,353][20347] Updated weights for policy 0, policy_version 259488 (0.0007) [2023-12-27 07:26:42,035][20347] Updated weights for policy 0, policy_version 259498 (0.0008) [2023-12-27 07:26:42,070][20381] Updated weights for policy 1, policy_version 259119 (0.0007) [2023-12-27 07:26:42,083][20347] Updated weights for policy 0, policy_version 259508 (0.0007) [2023-12-27 07:26:42,128][20381] Updated weights for policy 1, policy_version 259129 (0.0009) [2023-12-27 07:26:42,135][20347] Updated weights for policy 0, policy_version 259518 (0.0008) [2023-12-27 07:26:42,150][20292] KL-divergence is very high: 110.2748 [2023-12-27 07:26:42,186][20381] Updated weights for policy 1, policy_version 259139 (0.0008) [2023-12-27 07:26:42,188][20347] Updated weights for policy 0, policy_version 259528 (0.0006) [2023-12-27 07:26:42,200][20292] KL-divergence is very high: 117.6927 [2023-12-27 07:26:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 132808704. Throughput: 0: 10353.2, 1: 10185.7. Samples: 132794800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:42,363][19170] Avg episode reward: [(0, '863.190'), (1, '798.560')] [2023-12-27 07:26:42,904][20381] Updated weights for policy 1, policy_version 259149 (0.0011) [2023-12-27 07:26:42,918][20347] Updated weights for policy 0, policy_version 259538 (0.0007) [2023-12-27 07:26:42,961][20381] Updated weights for policy 1, policy_version 259159 (0.0011) [2023-12-27 07:26:42,973][20347] Updated weights for policy 0, policy_version 259548 (0.0005) [2023-12-27 07:26:43,016][20381] Updated weights for policy 1, policy_version 259169 (0.0011) [2023-12-27 07:26:43,033][20347] Updated weights for policy 0, policy_version 259558 (0.0005) [2023-12-27 07:26:43,656][20347] Updated weights for policy 0, policy_version 259568 (0.0005) [2023-12-27 07:26:43,663][20381] Updated weights for policy 1, policy_version 259179 (0.0010) [2023-12-27 07:26:43,702][20347] Updated weights for policy 0, policy_version 259578 (0.0005) [2023-12-27 07:26:43,728][20381] Updated weights for policy 1, policy_version 259189 (0.0008) [2023-12-27 07:26:43,746][20347] Updated weights for policy 0, policy_version 259588 (0.0007) [2023-12-27 07:26:43,789][20381] Updated weights for policy 1, policy_version 259199 (0.0007) [2023-12-27 07:26:44,353][20347] Updated weights for policy 0, policy_version 259598 (0.0006) [2023-12-27 07:26:44,408][20347] Updated weights for policy 0, policy_version 259608 (0.0007) [2023-12-27 07:26:44,444][20381] Updated weights for policy 1, policy_version 259209 (0.0010) [2023-12-27 07:26:44,469][20347] Updated weights for policy 0, policy_version 259618 (0.0008) [2023-12-27 07:26:44,506][20381] Updated weights for policy 1, policy_version 259219 (0.0006) [2023-12-27 07:26:44,555][20381] Updated weights for policy 1, policy_version 259229 (0.0008) [2023-12-27 07:26:44,609][20381] Updated weights for policy 1, policy_version 259239 (0.0007) [2023-12-27 07:26:45,092][20347] Updated weights for policy 0, policy_version 259628 (0.0007) [2023-12-27 07:26:45,148][20347] Updated weights for policy 0, policy_version 259638 (0.0007) [2023-12-27 07:26:45,205][20347] Updated weights for policy 0, policy_version 259648 (0.0008) [2023-12-27 07:26:45,294][20381] Updated weights for policy 1, policy_version 259249 (0.0008) [2023-12-27 07:26:45,344][20381] Updated weights for policy 1, policy_version 259259 (0.0007) [2023-12-27 07:26:45,393][20381] Updated weights for policy 1, policy_version 259269 (0.0008) [2023-12-27 07:26:45,888][20347] Updated weights for policy 0, policy_version 259658 (0.0006) [2023-12-27 07:26:45,937][20347] Updated weights for policy 0, policy_version 259668 (0.0008) [2023-12-27 07:26:45,982][20347] Updated weights for policy 0, policy_version 259678 (0.0007) [2023-12-27 07:26:46,032][20347] Updated weights for policy 0, policy_version 259688 (0.0007) [2023-12-27 07:26:46,112][20381] Updated weights for policy 1, policy_version 259279 (0.0008) [2023-12-27 07:26:46,176][20381] Updated weights for policy 1, policy_version 259289 (0.0008) [2023-12-27 07:26:46,232][20381] Updated weights for policy 1, policy_version 259299 (0.0009) [2023-12-27 07:26:46,683][20347] Updated weights for policy 0, policy_version 259698 (0.0010) [2023-12-27 07:26:46,732][20347] Updated weights for policy 0, policy_version 259708 (0.0010) [2023-12-27 07:26:46,784][20347] Updated weights for policy 0, policy_version 259718 (0.0008) [2023-12-27 07:26:46,872][20381] Updated weights for policy 1, policy_version 259309 (0.0007) [2023-12-27 07:26:46,931][20381] Updated weights for policy 1, policy_version 259319 (0.0005) [2023-12-27 07:26:46,988][20381] Updated weights for policy 1, policy_version 259329 (0.0006) [2023-12-27 07:26:47,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 132907008. Throughput: 0: 10342.1, 1: 10160.1. Samples: 132918652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:26:47,363][19170] Avg episode reward: [(0, '923.520'), (1, '827.010')] [2023-12-27 07:26:47,498][20347] Updated weights for policy 0, policy_version 259728 (0.0006) [2023-12-27 07:26:47,553][20347] Updated weights for policy 0, policy_version 259738 (0.0006) [2023-12-27 07:26:47,562][20381] Updated weights for policy 1, policy_version 259339 (0.0005) [2023-12-27 07:26:47,613][20347] Updated weights for policy 0, policy_version 259748 (0.0007) [2023-12-27 07:26:47,617][20381] Updated weights for policy 1, policy_version 259349 (0.0005) [2023-12-27 07:26:47,675][20381] Updated weights for policy 1, policy_version 259359 (0.0005) [2023-12-27 07:26:48,249][20347] Updated weights for policy 0, policy_version 259758 (0.0009) [2023-12-27 07:26:48,300][20381] Updated weights for policy 1, policy_version 259369 (0.0006) [2023-12-27 07:26:48,303][20347] Updated weights for policy 0, policy_version 259768 (0.0010) [2023-12-27 07:26:48,361][20381] Updated weights for policy 1, policy_version 259379 (0.0010) [2023-12-27 07:26:48,363][20347] Updated weights for policy 0, policy_version 259778 (0.0008) [2023-12-27 07:26:48,416][20381] Updated weights for policy 1, policy_version 259389 (0.0011) [2023-12-27 07:26:48,467][20381] Updated weights for policy 1, policy_version 259399 (0.0007) [2023-12-27 07:26:49,040][20381] Updated weights for policy 1, policy_version 259409 (0.0006) [2023-12-27 07:26:49,059][20347] Updated weights for policy 0, policy_version 259788 (0.0007) [2023-12-27 07:26:49,100][20381] Updated weights for policy 1, policy_version 259419 (0.0008) [2023-12-27 07:26:49,121][20347] Updated weights for policy 0, policy_version 259798 (0.0005) [2023-12-27 07:26:49,157][20381] Updated weights for policy 1, policy_version 259429 (0.0011) [2023-12-27 07:26:49,182][20347] Updated weights for policy 0, policy_version 259808 (0.0005) [2023-12-27 07:26:49,767][20347] Updated weights for policy 0, policy_version 259818 (0.0007) [2023-12-27 07:26:49,821][20347] Updated weights for policy 0, policy_version 259828 (0.0006) [2023-12-27 07:26:49,880][20381] Updated weights for policy 1, policy_version 259439 (0.0008) [2023-12-27 07:26:49,883][20347] Updated weights for policy 0, policy_version 259838 (0.0008) [2023-12-27 07:26:49,942][20381] Updated weights for policy 1, policy_version 259449 (0.0008) [2023-12-27 07:26:49,943][20347] Updated weights for policy 0, policy_version 259848 (0.0007) [2023-12-27 07:26:50,010][20381] Updated weights for policy 1, policy_version 259459 (0.0008) [2023-12-27 07:26:50,596][20347] Updated weights for policy 0, policy_version 259858 (0.0008) [2023-12-27 07:26:50,658][20347] Updated weights for policy 0, policy_version 259868 (0.0007) [2023-12-27 07:26:50,716][20347] Updated weights for policy 0, policy_version 259878 (0.0007) [2023-12-27 07:26:50,722][20381] Updated weights for policy 1, policy_version 259469 (0.0007) [2023-12-27 07:26:50,782][20381] Updated weights for policy 1, policy_version 259479 (0.0008) [2023-12-27 07:26:50,845][20381] Updated weights for policy 1, policy_version 259489 (0.0008) [2023-12-27 07:26:51,376][20347] Updated weights for policy 0, policy_version 259888 (0.0007) [2023-12-27 07:26:51,430][20347] Updated weights for policy 0, policy_version 259898 (0.0008) [2023-12-27 07:26:51,482][20347] Updated weights for policy 0, policy_version 259908 (0.0007) [2023-12-27 07:26:51,528][20381] Updated weights for policy 1, policy_version 259499 (0.0007) [2023-12-27 07:26:51,591][20381] Updated weights for policy 1, policy_version 259509 (0.0008) [2023-12-27 07:26:51,655][20381] Updated weights for policy 1, policy_version 259519 (0.0008) [2023-12-27 07:26:52,208][20347] Updated weights for policy 0, policy_version 259918 (0.0010) [2023-12-27 07:26:52,273][20347] Updated weights for policy 0, policy_version 259928 (0.0010) [2023-12-27 07:26:52,333][20347] Updated weights for policy 0, policy_version 259938 (0.0011) [2023-12-27 07:26:52,359][20381] Updated weights for policy 1, policy_version 259529 (0.0008) [2023-12-27 07:26:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 133005312. Throughput: 0: 10377.3, 1: 10158.7. Samples: 132982628. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:26:52,363][19170] Avg episode reward: [(0, '969.550'), (1, '913.330')] [2023-12-27 07:26:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000259944_66560000.pth... [2023-12-27 07:26:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000258728_66248704.pth [2023-12-27 07:26:52,416][20381] Updated weights for policy 1, policy_version 259539 (0.0008) [2023-12-27 07:26:52,474][20381] Updated weights for policy 1, policy_version 259549 (0.0008) [2023-12-27 07:26:52,534][20381] Updated weights for policy 1, policy_version 259559 (0.0007) [2023-12-27 07:26:52,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000259560_66461696.pth... [2023-12-27 07:26:52,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000258344_66150400.pth [2023-12-27 07:26:52,940][20347] Updated weights for policy 0, policy_version 259948 (0.0010) [2023-12-27 07:26:52,997][20347] Updated weights for policy 0, policy_version 259958 (0.0006) [2023-12-27 07:26:53,054][20347] Updated weights for policy 0, policy_version 259968 (0.0008) [2023-12-27 07:26:53,293][20381] Updated weights for policy 1, policy_version 259569 (0.0008) [2023-12-27 07:26:53,349][20381] Updated weights for policy 1, policy_version 259579 (0.0008) [2023-12-27 07:26:53,395][20381] Updated weights for policy 1, policy_version 259589 (0.0009) [2023-12-27 07:26:53,646][20347] Updated weights for policy 0, policy_version 259978 (0.0007) [2023-12-27 07:26:53,708][20347] Updated weights for policy 0, policy_version 259988 (0.0006) [2023-12-27 07:26:53,761][20347] Updated weights for policy 0, policy_version 259998 (0.0008) [2023-12-27 07:26:53,814][20347] Updated weights for policy 0, policy_version 260008 (0.0008) [2023-12-27 07:26:54,125][20381] Updated weights for policy 1, policy_version 259599 (0.0008) [2023-12-27 07:26:54,181][20381] Updated weights for policy 1, policy_version 259609 (0.0008) [2023-12-27 07:26:54,229][20381] Updated weights for policy 1, policy_version 259619 (0.0008) [2023-12-27 07:26:54,489][20347] Updated weights for policy 0, policy_version 260018 (0.0008) [2023-12-27 07:26:54,545][20347] Updated weights for policy 0, policy_version 260028 (0.0006) [2023-12-27 07:26:54,596][20347] Updated weights for policy 0, policy_version 260038 (0.0008) [2023-12-27 07:26:54,893][20381] Updated weights for policy 1, policy_version 259629 (0.0010) [2023-12-27 07:26:54,944][20381] Updated weights for policy 1, policy_version 259639 (0.0010) [2023-12-27 07:26:55,004][20381] Updated weights for policy 1, policy_version 259649 (0.0011) [2023-12-27 07:26:55,283][20347] Updated weights for policy 0, policy_version 260048 (0.0005) [2023-12-27 07:26:55,337][20347] Updated weights for policy 0, policy_version 260058 (0.0005) [2023-12-27 07:26:55,384][20347] Updated weights for policy 0, policy_version 260068 (0.0006) [2023-12-27 07:26:55,609][20381] Updated weights for policy 1, policy_version 259659 (0.0010) [2023-12-27 07:26:55,662][20381] Updated weights for policy 1, policy_version 259669 (0.0010) [2023-12-27 07:26:55,710][20381] Updated weights for policy 1, policy_version 259679 (0.0010) [2023-12-27 07:26:56,011][20347] Updated weights for policy 0, policy_version 260078 (0.0005) [2023-12-27 07:26:56,068][20347] Updated weights for policy 0, policy_version 260088 (0.0006) [2023-12-27 07:26:56,133][20347] Updated weights for policy 0, policy_version 260098 (0.0006) [2023-12-27 07:26:56,344][20381] Updated weights for policy 1, policy_version 259689 (0.0009) [2023-12-27 07:26:56,396][20381] Updated weights for policy 1, policy_version 259699 (0.0007) [2023-12-27 07:26:56,444][20381] Updated weights for policy 1, policy_version 259709 (0.0007) [2023-12-27 07:26:56,493][20381] Updated weights for policy 1, policy_version 259719 (0.0007) [2023-12-27 07:26:56,776][20347] Updated weights for policy 0, policy_version 260108 (0.0008) [2023-12-27 07:26:56,828][20347] Updated weights for policy 0, policy_version 260118 (0.0007) [2023-12-27 07:26:56,873][20347] Updated weights for policy 0, policy_version 260128 (0.0007) [2023-12-27 07:26:57,159][20381] Updated weights for policy 1, policy_version 259729 (0.0010) [2023-12-27 07:26:57,213][20381] Updated weights for policy 1, policy_version 259739 (0.0010) [2023-12-27 07:26:57,269][20381] Updated weights for policy 1, policy_version 259749 (0.0010) [2023-12-27 07:26:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 133120000. Throughput: 0: 10430.6, 1: 10185.7. Samples: 133106996. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:26:57,363][19170] Avg episode reward: [(0, '994.400'), (1, '886.810')] [2023-12-27 07:26:57,601][20347] Updated weights for policy 0, policy_version 260138 (0.0007) [2023-12-27 07:26:57,661][20347] Updated weights for policy 0, policy_version 260148 (0.0007) [2023-12-27 07:26:57,708][20347] Updated weights for policy 0, policy_version 260158 (0.0007) [2023-12-27 07:26:57,753][20347] Updated weights for policy 0, policy_version 260168 (0.0007) [2023-12-27 07:26:57,931][20381] Updated weights for policy 1, policy_version 259759 (0.0008) [2023-12-27 07:26:57,985][20381] Updated weights for policy 1, policy_version 259769 (0.0010) [2023-12-27 07:26:58,039][20381] Updated weights for policy 1, policy_version 259779 (0.0010) [2023-12-27 07:26:58,502][20347] Updated weights for policy 0, policy_version 260178 (0.0007) [2023-12-27 07:26:58,561][20347] Updated weights for policy 0, policy_version 260188 (0.0007) [2023-12-27 07:26:58,628][20347] Updated weights for policy 0, policy_version 260198 (0.0006) [2023-12-27 07:26:58,805][20381] Updated weights for policy 1, policy_version 259789 (0.0009) [2023-12-27 07:26:58,871][20381] Updated weights for policy 1, policy_version 259799 (0.0008) [2023-12-27 07:26:58,930][20381] Updated weights for policy 1, policy_version 259809 (0.0008) [2023-12-27 07:26:59,377][20347] Updated weights for policy 0, policy_version 260208 (0.0008) [2023-12-27 07:26:59,447][20347] Updated weights for policy 0, policy_version 260218 (0.0009) [2023-12-27 07:26:59,513][20347] Updated weights for policy 0, policy_version 260228 (0.0008) [2023-12-27 07:26:59,646][20381] Updated weights for policy 1, policy_version 259819 (0.0007) [2023-12-27 07:26:59,700][20381] Updated weights for policy 1, policy_version 259829 (0.0006) [2023-12-27 07:26:59,756][20381] Updated weights for policy 1, policy_version 259839 (0.0005) [2023-12-27 07:27:00,222][20347] Updated weights for policy 0, policy_version 260238 (0.0007) [2023-12-27 07:27:00,283][20347] Updated weights for policy 0, policy_version 260248 (0.0007) [2023-12-27 07:27:00,341][20347] Updated weights for policy 0, policy_version 260258 (0.0006) [2023-12-27 07:27:00,408][20381] Updated weights for policy 1, policy_version 259849 (0.0006) [2023-12-27 07:27:00,462][20381] Updated weights for policy 1, policy_version 259859 (0.0007) [2023-12-27 07:27:00,510][20381] Updated weights for policy 1, policy_version 259869 (0.0008) [2023-12-27 07:27:00,561][20381] Updated weights for policy 1, policy_version 259879 (0.0008) [2023-12-27 07:27:00,981][20347] Updated weights for policy 0, policy_version 260268 (0.0007) [2023-12-27 07:27:01,033][20347] Updated weights for policy 0, policy_version 260278 (0.0006) [2023-12-27 07:27:01,088][20347] Updated weights for policy 0, policy_version 260288 (0.0007) [2023-12-27 07:27:01,257][20292] KL-divergence is very high: 128.2517 [2023-12-27 07:27:01,274][20381] Updated weights for policy 1, policy_version 259889 (0.0009) [2023-12-27 07:27:01,333][20381] Updated weights for policy 1, policy_version 259899 (0.0008) [2023-12-27 07:27:01,400][20381] Updated weights for policy 1, policy_version 259909 (0.0008) [2023-12-27 07:27:01,712][20347] Updated weights for policy 0, policy_version 260298 (0.0008) [2023-12-27 07:27:01,777][20347] Updated weights for policy 0, policy_version 260308 (0.0009) [2023-12-27 07:27:01,824][20347] Updated weights for policy 0, policy_version 260318 (0.0008) [2023-12-27 07:27:01,874][20347] Updated weights for policy 0, policy_version 260328 (0.0005) [2023-12-27 07:27:02,085][20381] Updated weights for policy 1, policy_version 259919 (0.0008) [2023-12-27 07:27:02,131][20292] KL-divergence is very high: 278.9578 [2023-12-27 07:27:02,145][20381] Updated weights for policy 1, policy_version 259929 (0.0008) [2023-12-27 07:27:02,202][20381] Updated weights for policy 1, policy_version 259939 (0.0007) [2023-12-27 07:27:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 133218304. Throughput: 0: 10378.5, 1: 10202.8. Samples: 133227224. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:02,363][19170] Avg episode reward: [(0, '999.250'), (1, '810.520')] [2023-12-27 07:27:02,477][20347] Updated weights for policy 0, policy_version 260338 (0.0006) [2023-12-27 07:27:02,535][20347] Updated weights for policy 0, policy_version 260348 (0.0006) [2023-12-27 07:27:02,593][20347] Updated weights for policy 0, policy_version 260358 (0.0010) [2023-12-27 07:27:02,894][20292] KL-divergence is very high: 133.7607 [2023-12-27 07:27:02,908][20381] Updated weights for policy 1, policy_version 259949 (0.0006) [2023-12-27 07:27:02,961][20381] Updated weights for policy 1, policy_version 259959 (0.0005) [2023-12-27 07:27:02,962][20292] KL-divergence is very high: 215.4364 [2023-12-27 07:27:02,978][20292] KL-divergence is very high: 348.6544 [2023-12-27 07:27:02,997][20292] KL-divergence is very high: 144.0224 [2023-12-27 07:27:03,001][20292] KL-divergence is very high: 218.6561 [2023-12-27 07:27:03,010][20381] Updated weights for policy 1, policy_version 259969 (0.0005) [2023-12-27 07:27:03,015][20292] KL-divergence is very high: 209.5881 [2023-12-27 07:27:03,033][20292] KL-divergence is very high: 111.4192 [2023-12-27 07:27:03,037][20292] KL-divergence is very high: 121.6369 [2023-12-27 07:27:03,242][20347] Updated weights for policy 0, policy_version 260368 (0.0008) [2023-12-27 07:27:03,299][20347] Updated weights for policy 0, policy_version 260378 (0.0005) [2023-12-27 07:27:03,354][20347] Updated weights for policy 0, policy_version 260388 (0.0007) [2023-12-27 07:27:03,656][20381] Updated weights for policy 1, policy_version 259979 (0.0006) [2023-12-27 07:27:03,706][20381] Updated weights for policy 1, policy_version 259989 (0.0008) [2023-12-27 07:27:03,757][20381] Updated weights for policy 1, policy_version 259999 (0.0010) [2023-12-27 07:27:03,985][20347] Updated weights for policy 0, policy_version 260398 (0.0007) [2023-12-27 07:27:04,043][20347] Updated weights for policy 0, policy_version 260408 (0.0007) [2023-12-27 07:27:04,097][20347] Updated weights for policy 0, policy_version 260418 (0.0007) [2023-12-27 07:27:04,408][20381] Updated weights for policy 1, policy_version 260009 (0.0010) [2023-12-27 07:27:04,473][20381] Updated weights for policy 1, policy_version 260019 (0.0011) [2023-12-27 07:27:04,536][20381] Updated weights for policy 1, policy_version 260029 (0.0011) [2023-12-27 07:27:04,597][20381] Updated weights for policy 1, policy_version 260039 (0.0010) [2023-12-27 07:27:04,813][20347] Updated weights for policy 0, policy_version 260428 (0.0009) [2023-12-27 07:27:04,880][20347] Updated weights for policy 0, policy_version 260438 (0.0010) [2023-12-27 07:27:04,941][20347] Updated weights for policy 0, policy_version 260448 (0.0010) [2023-12-27 07:27:05,184][20381] Updated weights for policy 1, policy_version 260049 (0.0006) [2023-12-27 07:27:05,240][20381] Updated weights for policy 1, policy_version 260059 (0.0005) [2023-12-27 07:27:05,302][20381] Updated weights for policy 1, policy_version 260069 (0.0005) [2023-12-27 07:27:05,526][20347] Updated weights for policy 0, policy_version 260458 (0.0009) [2023-12-27 07:27:05,576][20347] Updated weights for policy 0, policy_version 260468 (0.0005) [2023-12-27 07:27:05,635][20347] Updated weights for policy 0, policy_version 260478 (0.0006) [2023-12-27 07:27:05,695][20347] Updated weights for policy 0, policy_version 260488 (0.0010) [2023-12-27 07:27:05,954][20381] Updated weights for policy 1, policy_version 260079 (0.0008) [2023-12-27 07:27:06,020][20381] Updated weights for policy 1, policy_version 260089 (0.0007) [2023-12-27 07:27:06,078][20381] Updated weights for policy 1, policy_version 260099 (0.0011) [2023-12-27 07:27:06,291][20347] Updated weights for policy 0, policy_version 260498 (0.0010) [2023-12-27 07:27:06,337][20347] Updated weights for policy 0, policy_version 260508 (0.0011) [2023-12-27 07:27:06,390][20347] Updated weights for policy 0, policy_version 260518 (0.0010) [2023-12-27 07:27:06,745][20381] Updated weights for policy 1, policy_version 260109 (0.0008) [2023-12-27 07:27:06,796][20381] Updated weights for policy 1, policy_version 260119 (0.0005) [2023-12-27 07:27:06,841][20381] Updated weights for policy 1, policy_version 260129 (0.0005) [2023-12-27 07:27:07,100][20347] Updated weights for policy 0, policy_version 260528 (0.0007) [2023-12-27 07:27:07,162][20347] Updated weights for policy 0, policy_version 260538 (0.0007) [2023-12-27 07:27:07,217][20347] Updated weights for policy 0, policy_version 260548 (0.0010) [2023-12-27 07:27:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 133324800. Throughput: 0: 10402.4, 1: 10270.2. Samples: 133290908. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '818.280')] [2023-12-27 07:27:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000260552_66715648.pth... [2023-12-27 07:27:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000260136_66609152.pth... [2023-12-27 07:27:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000259336_66404352.pth [2023-12-27 07:27:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000258952_66306048.pth [2023-12-27 07:27:07,420][20381] Updated weights for policy 1, policy_version 260139 (0.0006) [2023-12-27 07:27:07,472][20381] Updated weights for policy 1, policy_version 260149 (0.0008) [2023-12-27 07:27:07,523][20381] Updated weights for policy 1, policy_version 260159 (0.0005) [2023-12-27 07:27:07,844][20347] Updated weights for policy 0, policy_version 260558 (0.0009) [2023-12-27 07:27:07,890][20347] Updated weights for policy 0, policy_version 260568 (0.0006) [2023-12-27 07:27:07,950][20347] Updated weights for policy 0, policy_version 260578 (0.0005) [2023-12-27 07:27:08,202][20381] Updated weights for policy 1, policy_version 260169 (0.0007) [2023-12-27 07:27:08,264][20381] Updated weights for policy 1, policy_version 260179 (0.0005) [2023-12-27 07:27:08,328][20381] Updated weights for policy 1, policy_version 260189 (0.0006) [2023-12-27 07:27:08,393][20381] Updated weights for policy 1, policy_version 260199 (0.0006) [2023-12-27 07:27:08,562][20347] Updated weights for policy 0, policy_version 260588 (0.0005) [2023-12-27 07:27:08,622][20347] Updated weights for policy 0, policy_version 260598 (0.0005) [2023-12-27 07:27:08,689][20347] Updated weights for policy 0, policy_version 260608 (0.0008) [2023-12-27 07:27:08,934][20381] Updated weights for policy 1, policy_version 260209 (0.0006) [2023-12-27 07:27:08,985][20381] Updated weights for policy 1, policy_version 260219 (0.0006) [2023-12-27 07:27:09,032][20381] Updated weights for policy 1, policy_version 260229 (0.0008) [2023-12-27 07:27:09,318][20347] Updated weights for policy 0, policy_version 260618 (0.0006) [2023-12-27 07:27:09,381][20347] Updated weights for policy 0, policy_version 260628 (0.0007) [2023-12-27 07:27:09,449][20347] Updated weights for policy 0, policy_version 260638 (0.0007) [2023-12-27 07:27:09,512][20347] Updated weights for policy 0, policy_version 260648 (0.0007) [2023-12-27 07:27:09,756][20381] Updated weights for policy 1, policy_version 260239 (0.0010) [2023-12-27 07:27:09,819][20381] Updated weights for policy 1, policy_version 260249 (0.0011) [2023-12-27 07:27:09,884][20381] Updated weights for policy 1, policy_version 260259 (0.0011) [2023-12-27 07:27:10,194][20347] Updated weights for policy 0, policy_version 260658 (0.0008) [2023-12-27 07:27:10,248][20347] Updated weights for policy 0, policy_version 260668 (0.0007) [2023-12-27 07:27:10,303][20347] Updated weights for policy 0, policy_version 260678 (0.0007) [2023-12-27 07:27:10,552][20381] Updated weights for policy 1, policy_version 260269 (0.0011) [2023-12-27 07:27:10,598][20381] Updated weights for policy 1, policy_version 260279 (0.0010) [2023-12-27 07:27:10,650][20381] Updated weights for policy 1, policy_version 260289 (0.0010) [2023-12-27 07:27:10,910][20347] Updated weights for policy 0, policy_version 260688 (0.0009) [2023-12-27 07:27:10,970][20347] Updated weights for policy 0, policy_version 260698 (0.0010) [2023-12-27 07:27:11,033][20347] Updated weights for policy 0, policy_version 260708 (0.0008) [2023-12-27 07:27:11,308][20381] Updated weights for policy 1, policy_version 260299 (0.0007) [2023-12-27 07:27:11,377][20381] Updated weights for policy 1, policy_version 260309 (0.0007) [2023-12-27 07:27:11,427][20292] KL-divergence is very high: 223.9820 [2023-12-27 07:27:11,439][20292] KL-divergence is very high: 227.0657 [2023-12-27 07:27:11,441][20381] Updated weights for policy 1, policy_version 260319 (0.0008) [2023-12-27 07:27:11,476][20292] KL-divergence is very high: 122.3785 [2023-12-27 07:27:11,770][20347] Updated weights for policy 0, policy_version 260718 (0.0008) [2023-12-27 07:27:11,830][20347] Updated weights for policy 0, policy_version 260728 (0.0008) [2023-12-27 07:27:11,886][20347] Updated weights for policy 0, policy_version 260738 (0.0008) [2023-12-27 07:27:12,130][20381] Updated weights for policy 1, policy_version 260329 (0.0008) [2023-12-27 07:27:12,180][20381] Updated weights for policy 1, policy_version 260339 (0.0009) [2023-12-27 07:27:12,238][20381] Updated weights for policy 1, policy_version 260349 (0.0008) [2023-12-27 07:27:12,296][20381] Updated weights for policy 1, policy_version 260359 (0.0009) [2023-12-27 07:27:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 133431296. Throughput: 0: 10495.4, 1: 10381.3. Samples: 133419772. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:12,363][19170] Avg episode reward: [(0, '999.570'), (1, '791.600')] [2023-12-27 07:27:12,548][20347] Updated weights for policy 0, policy_version 260748 (0.0007) [2023-12-27 07:27:12,596][20347] Updated weights for policy 0, policy_version 260758 (0.0008) [2023-12-27 07:27:12,653][20347] Updated weights for policy 0, policy_version 260768 (0.0008) [2023-12-27 07:27:13,000][20381] Updated weights for policy 1, policy_version 260369 (0.0008) [2023-12-27 07:27:13,056][20381] Updated weights for policy 1, policy_version 260379 (0.0007) [2023-12-27 07:27:13,114][20381] Updated weights for policy 1, policy_version 260389 (0.0007) [2023-12-27 07:27:13,287][20347] Updated weights for policy 0, policy_version 260778 (0.0006) [2023-12-27 07:27:13,339][20347] Updated weights for policy 0, policy_version 260788 (0.0010) [2023-12-27 07:27:13,396][20347] Updated weights for policy 0, policy_version 260798 (0.0008) [2023-12-27 07:27:13,451][20347] Updated weights for policy 0, policy_version 260808 (0.0007) [2023-12-27 07:27:13,751][20381] Updated weights for policy 1, policy_version 260399 (0.0006) [2023-12-27 07:27:13,812][20381] Updated weights for policy 1, policy_version 260409 (0.0011) [2023-12-27 07:27:13,863][20381] Updated weights for policy 1, policy_version 260419 (0.0008) [2023-12-27 07:27:14,034][20347] Updated weights for policy 0, policy_version 260818 (0.0006) [2023-12-27 07:27:14,093][20347] Updated weights for policy 0, policy_version 260828 (0.0005) [2023-12-27 07:27:14,152][20347] Updated weights for policy 0, policy_version 260838 (0.0005) [2023-12-27 07:27:14,502][20381] Updated weights for policy 1, policy_version 260429 (0.0007) [2023-12-27 07:27:14,556][20381] Updated weights for policy 1, policy_version 260439 (0.0005) [2023-12-27 07:27:14,603][20381] Updated weights for policy 1, policy_version 260449 (0.0006) [2023-12-27 07:27:14,709][20347] Updated weights for policy 0, policy_version 260848 (0.0006) [2023-12-27 07:27:14,761][20347] Updated weights for policy 0, policy_version 260858 (0.0009) [2023-12-27 07:27:14,823][20347] Updated weights for policy 0, policy_version 260868 (0.0006) [2023-12-27 07:27:15,289][20381] Updated weights for policy 1, policy_version 260459 (0.0010) [2023-12-27 07:27:15,345][20381] Updated weights for policy 1, policy_version 260469 (0.0008) [2023-12-27 07:27:15,398][20381] Updated weights for policy 1, policy_version 260479 (0.0008) [2023-12-27 07:27:15,470][20347] Updated weights for policy 0, policy_version 260878 (0.0007) [2023-12-27 07:27:15,538][20347] Updated weights for policy 0, policy_version 260888 (0.0008) [2023-12-27 07:27:15,604][20347] Updated weights for policy 0, policy_version 260898 (0.0007) [2023-12-27 07:27:16,069][20381] Updated weights for policy 1, policy_version 260489 (0.0007) [2023-12-27 07:27:16,123][20381] Updated weights for policy 1, policy_version 260499 (0.0010) [2023-12-27 07:27:16,181][20381] Updated weights for policy 1, policy_version 260509 (0.0010) [2023-12-27 07:27:16,235][20381] Updated weights for policy 1, policy_version 260519 (0.0010) [2023-12-27 07:27:16,243][20347] Updated weights for policy 0, policy_version 260908 (0.0007) [2023-12-27 07:27:16,286][20347] Updated weights for policy 0, policy_version 260918 (0.0007) [2023-12-27 07:27:16,329][20347] Updated weights for policy 0, policy_version 260928 (0.0007) [2023-12-27 07:27:16,910][20381] Updated weights for policy 1, policy_version 260529 (0.0006) [2023-12-27 07:27:16,962][20381] Updated weights for policy 1, policy_version 260539 (0.0011) [2023-12-27 07:27:17,011][20381] Updated weights for policy 1, policy_version 260549 (0.0009) [2023-12-27 07:27:17,062][20347] Updated weights for policy 0, policy_version 260938 (0.0007) [2023-12-27 07:27:17,113][20347] Updated weights for policy 0, policy_version 260948 (0.0007) [2023-12-27 07:27:17,171][20347] Updated weights for policy 0, policy_version 260958 (0.0008) [2023-12-27 07:27:17,223][20347] Updated weights for policy 0, policy_version 260968 (0.0008) [2023-12-27 07:27:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 133537792. Throughput: 0: 10516.9, 1: 10314.7. Samples: 133545960. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:17,363][19170] Avg episode reward: [(0, '997.980'), (1, '791.260')] [2023-12-27 07:27:17,599][20381] Updated weights for policy 1, policy_version 260559 (0.0009) [2023-12-27 07:27:17,643][20381] Updated weights for policy 1, policy_version 260569 (0.0007) [2023-12-27 07:27:17,690][20381] Updated weights for policy 1, policy_version 260579 (0.0005) [2023-12-27 07:27:17,790][20347] Updated weights for policy 0, policy_version 260978 (0.0007) [2023-12-27 07:27:17,838][20347] Updated weights for policy 0, policy_version 260988 (0.0007) [2023-12-27 07:27:17,895][20347] Updated weights for policy 0, policy_version 260998 (0.0006) [2023-12-27 07:27:18,314][20381] Updated weights for policy 1, policy_version 260589 (0.0006) [2023-12-27 07:27:18,373][20381] Updated weights for policy 1, policy_version 260599 (0.0008) [2023-12-27 07:27:18,426][20381] Updated weights for policy 1, policy_version 260609 (0.0008) [2023-12-27 07:27:18,613][20347] Updated weights for policy 0, policy_version 261008 (0.0007) [2023-12-27 07:27:18,665][20347] Updated weights for policy 0, policy_version 261018 (0.0008) [2023-12-27 07:27:18,714][20347] Updated weights for policy 0, policy_version 261028 (0.0007) [2023-12-27 07:27:19,088][20381] Updated weights for policy 1, policy_version 260619 (0.0008) [2023-12-27 07:27:19,137][20381] Updated weights for policy 1, policy_version 260629 (0.0005) [2023-12-27 07:27:19,187][20381] Updated weights for policy 1, policy_version 260639 (0.0005) [2023-12-27 07:27:19,439][20347] Updated weights for policy 0, policy_version 261038 (0.0008) [2023-12-27 07:27:19,501][20347] Updated weights for policy 0, policy_version 261048 (0.0007) [2023-12-27 07:27:19,560][20347] Updated weights for policy 0, policy_version 261058 (0.0006) [2023-12-27 07:27:19,858][20381] Updated weights for policy 1, policy_version 260649 (0.0006) [2023-12-27 07:27:19,923][20381] Updated weights for policy 1, policy_version 260659 (0.0008) [2023-12-27 07:27:19,992][20381] Updated weights for policy 1, policy_version 260669 (0.0008) [2023-12-27 07:27:20,057][20381] Updated weights for policy 1, policy_version 260679 (0.0006) [2023-12-27 07:27:20,201][20347] Updated weights for policy 0, policy_version 261068 (0.0006) [2023-12-27 07:27:20,263][20347] Updated weights for policy 0, policy_version 261078 (0.0008) [2023-12-27 07:27:20,323][20347] Updated weights for policy 0, policy_version 261088 (0.0007) [2023-12-27 07:27:20,762][20381] Updated weights for policy 1, policy_version 260689 (0.0008) [2023-12-27 07:27:20,820][20381] Updated weights for policy 1, policy_version 260699 (0.0007) [2023-12-27 07:27:20,879][20381] Updated weights for policy 1, policy_version 260709 (0.0008) [2023-12-27 07:27:21,010][20347] Updated weights for policy 0, policy_version 261098 (0.0008) [2023-12-27 07:27:21,078][20347] Updated weights for policy 0, policy_version 261108 (0.0007) [2023-12-27 07:27:21,137][20347] Updated weights for policy 0, policy_version 261118 (0.0006) [2023-12-27 07:27:21,199][20347] Updated weights for policy 0, policy_version 261128 (0.0007) [2023-12-27 07:27:21,575][20381] Updated weights for policy 1, policy_version 260719 (0.0008) [2023-12-27 07:27:21,629][20381] Updated weights for policy 1, policy_version 260729 (0.0007) [2023-12-27 07:27:21,695][20381] Updated weights for policy 1, policy_version 260739 (0.0007) [2023-12-27 07:27:21,911][20347] Updated weights for policy 0, policy_version 261138 (0.0008) [2023-12-27 07:27:21,979][20347] Updated weights for policy 0, policy_version 261148 (0.0008) [2023-12-27 07:27:22,043][20347] Updated weights for policy 0, policy_version 261158 (0.0008) [2023-12-27 07:27:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 133636096. Throughput: 0: 10474.8, 1: 10370.2. Samples: 133608532. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:22,363][19170] Avg episode reward: [(0, '998.240'), (1, '895.440')] [2023-12-27 07:27:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000260744_66764800.pth... [2023-12-27 07:27:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000261160_66871296.pth... [2023-12-27 07:27:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000259560_66461696.pth [2023-12-27 07:27:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000259944_66560000.pth [2023-12-27 07:27:22,412][20381] Updated weights for policy 1, policy_version 260749 (0.0007) [2023-12-27 07:27:22,463][20381] Updated weights for policy 1, policy_version 260759 (0.0008) [2023-12-27 07:27:22,516][20381] Updated weights for policy 1, policy_version 260769 (0.0008) [2023-12-27 07:27:22,691][20347] Updated weights for policy 0, policy_version 261168 (0.0007) [2023-12-27 07:27:22,739][20347] Updated weights for policy 0, policy_version 261178 (0.0006) [2023-12-27 07:27:22,789][20347] Updated weights for policy 0, policy_version 261188 (0.0008) [2023-12-27 07:27:23,237][20381] Updated weights for policy 1, policy_version 260779 (0.0008) [2023-12-27 07:27:23,285][20381] Updated weights for policy 1, policy_version 260789 (0.0007) [2023-12-27 07:27:23,337][20381] Updated weights for policy 1, policy_version 260799 (0.0008) [2023-12-27 07:27:23,486][20347] Updated weights for policy 0, policy_version 261198 (0.0008) [2023-12-27 07:27:23,547][20347] Updated weights for policy 0, policy_version 261208 (0.0009) [2023-12-27 07:27:23,603][20347] Updated weights for policy 0, policy_version 261218 (0.0010) [2023-12-27 07:27:24,002][20381] Updated weights for policy 1, policy_version 260809 (0.0008) [2023-12-27 07:27:24,057][20381] Updated weights for policy 1, policy_version 260819 (0.0007) [2023-12-27 07:27:24,112][20381] Updated weights for policy 1, policy_version 260829 (0.0007) [2023-12-27 07:27:24,159][20381] Updated weights for policy 1, policy_version 260839 (0.0007) [2023-12-27 07:27:24,294][20347] Updated weights for policy 0, policy_version 261228 (0.0011) [2023-12-27 07:27:24,345][20347] Updated weights for policy 0, policy_version 261238 (0.0010) [2023-12-27 07:27:24,394][20347] Updated weights for policy 0, policy_version 261248 (0.0010) [2023-12-27 07:27:24,836][20381] Updated weights for policy 1, policy_version 260849 (0.0008) [2023-12-27 07:27:24,889][20381] Updated weights for policy 1, policy_version 260859 (0.0007) [2023-12-27 07:27:24,939][20381] Updated weights for policy 1, policy_version 260869 (0.0010) [2023-12-27 07:27:25,048][20347] Updated weights for policy 0, policy_version 261258 (0.0009) [2023-12-27 07:27:25,111][20347] Updated weights for policy 0, policy_version 261268 (0.0007) [2023-12-27 07:27:25,168][20347] Updated weights for policy 0, policy_version 261278 (0.0007) [2023-12-27 07:27:25,225][20347] Updated weights for policy 0, policy_version 261288 (0.0006) [2023-12-27 07:27:25,584][20381] Updated weights for policy 1, policy_version 260879 (0.0008) [2023-12-27 07:27:25,645][20381] Updated weights for policy 1, policy_version 260889 (0.0008) [2023-12-27 07:27:25,703][20381] Updated weights for policy 1, policy_version 260899 (0.0007) [2023-12-27 07:27:25,804][20347] Updated weights for policy 0, policy_version 261298 (0.0005) [2023-12-27 07:27:25,862][20347] Updated weights for policy 0, policy_version 261308 (0.0008) [2023-12-27 07:27:25,921][20347] Updated weights for policy 0, policy_version 261318 (0.0007) [2023-12-27 07:27:26,417][20381] Updated weights for policy 1, policy_version 260909 (0.0009) [2023-12-27 07:27:26,468][20381] Updated weights for policy 1, policy_version 260919 (0.0010) [2023-12-27 07:27:26,517][20347] Updated weights for policy 0, policy_version 261328 (0.0007) [2023-12-27 07:27:26,523][20381] Updated weights for policy 1, policy_version 260929 (0.0010) [2023-12-27 07:27:26,570][20347] Updated weights for policy 0, policy_version 261338 (0.0005) [2023-12-27 07:27:26,626][20347] Updated weights for policy 0, policy_version 261348 (0.0005) [2023-12-27 07:27:27,139][20381] Updated weights for policy 1, policy_version 260939 (0.0009) [2023-12-27 07:27:27,198][20381] Updated weights for policy 1, policy_version 260949 (0.0008) [2023-12-27 07:27:27,249][20381] Updated weights for policy 1, policy_version 260959 (0.0006) [2023-12-27 07:27:27,252][20347] Updated weights for policy 0, policy_version 261358 (0.0010) [2023-12-27 07:27:27,306][20347] Updated weights for policy 0, policy_version 261368 (0.0010) [2023-12-27 07:27:27,359][20347] Updated weights for policy 0, policy_version 261378 (0.0010) [2023-12-27 07:27:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 133742592. Throughput: 0: 10467.0, 1: 10327.9. Samples: 133730568. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:27,363][19170] Avg episode reward: [(0, '999.170'), (1, '914.700')] [2023-12-27 07:27:27,895][20381] Updated weights for policy 1, policy_version 260969 (0.0010) [2023-12-27 07:27:27,951][20381] Updated weights for policy 1, policy_version 260979 (0.0010) [2023-12-27 07:27:28,006][20381] Updated weights for policy 1, policy_version 260989 (0.0010) [2023-12-27 07:27:28,023][20347] Updated weights for policy 0, policy_version 261388 (0.0010) [2023-12-27 07:27:28,054][20381] Updated weights for policy 1, policy_version 260999 (0.0010) [2023-12-27 07:27:28,081][20347] Updated weights for policy 0, policy_version 261398 (0.0010) [2023-12-27 07:27:28,130][20347] Updated weights for policy 0, policy_version 261408 (0.0010) [2023-12-27 07:27:28,671][20381] Updated weights for policy 1, policy_version 261009 (0.0010) [2023-12-27 07:27:28,731][20381] Updated weights for policy 1, policy_version 261019 (0.0010) [2023-12-27 07:27:28,789][20347] Updated weights for policy 0, policy_version 261418 (0.0010) [2023-12-27 07:27:28,789][20381] Updated weights for policy 1, policy_version 261029 (0.0011) [2023-12-27 07:27:28,837][20347] Updated weights for policy 0, policy_version 261428 (0.0010) [2023-12-27 07:27:28,887][20347] Updated weights for policy 0, policy_version 261438 (0.0008) [2023-12-27 07:27:28,938][20347] Updated weights for policy 0, policy_version 261448 (0.0005) [2023-12-27 07:27:29,502][20381] Updated weights for policy 1, policy_version 261039 (0.0011) [2023-12-27 07:27:29,557][20381] Updated weights for policy 1, policy_version 261049 (0.0010) [2023-12-27 07:27:29,574][20347] Updated weights for policy 0, policy_version 261458 (0.0010) [2023-12-27 07:27:29,612][20381] Updated weights for policy 1, policy_version 261059 (0.0010) [2023-12-27 07:27:29,632][20347] Updated weights for policy 0, policy_version 261468 (0.0010) [2023-12-27 07:27:29,692][20347] Updated weights for policy 0, policy_version 261478 (0.0010) [2023-12-27 07:27:30,335][20381] Updated weights for policy 1, policy_version 261069 (0.0008) [2023-12-27 07:27:30,372][20347] Updated weights for policy 0, policy_version 261488 (0.0010) [2023-12-27 07:27:30,395][20381] Updated weights for policy 1, policy_version 261079 (0.0006) [2023-12-27 07:27:30,423][20347] Updated weights for policy 0, policy_version 261498 (0.0008) [2023-12-27 07:27:30,447][20381] Updated weights for policy 1, policy_version 261089 (0.0005) [2023-12-27 07:27:30,475][20347] Updated weights for policy 0, policy_version 261508 (0.0006) [2023-12-27 07:27:30,976][20381] Updated weights for policy 1, policy_version 261099 (0.0006) [2023-12-27 07:27:31,030][20381] Updated weights for policy 1, policy_version 261109 (0.0007) [2023-12-27 07:27:31,049][20347] Updated weights for policy 0, policy_version 261518 (0.0007) [2023-12-27 07:27:31,089][20381] Updated weights for policy 1, policy_version 261119 (0.0007) [2023-12-27 07:27:31,109][20347] Updated weights for policy 0, policy_version 261528 (0.0010) [2023-12-27 07:27:31,174][20347] Updated weights for policy 0, policy_version 261538 (0.0011) [2023-12-27 07:27:31,806][20381] Updated weights for policy 1, policy_version 261129 (0.0007) [2023-12-27 07:27:31,855][20381] Updated weights for policy 1, policy_version 261139 (0.0007) [2023-12-27 07:27:31,858][20347] Updated weights for policy 0, policy_version 261548 (0.0010) [2023-12-27 07:27:31,908][20381] Updated weights for policy 1, policy_version 261149 (0.0006) [2023-12-27 07:27:31,910][20347] Updated weights for policy 0, policy_version 261558 (0.0010) [2023-12-27 07:27:31,964][20347] Updated weights for policy 0, policy_version 261568 (0.0010) [2023-12-27 07:27:31,966][20381] Updated weights for policy 1, policy_version 261159 (0.0005) [2023-12-27 07:27:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 133849088. Throughput: 0: 10498.7, 1: 10390.5. Samples: 133858668. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:32,363][19170] Avg episode reward: [(0, '997.920'), (1, '885.490')] [2023-12-27 07:27:32,593][20381] Updated weights for policy 1, policy_version 261169 (0.0007) [2023-12-27 07:27:32,646][20381] Updated weights for policy 1, policy_version 261179 (0.0007) [2023-12-27 07:27:32,649][20347] Updated weights for policy 0, policy_version 261578 (0.0009) [2023-12-27 07:27:32,700][20381] Updated weights for policy 1, policy_version 261189 (0.0006) [2023-12-27 07:27:32,702][20347] Updated weights for policy 0, policy_version 261588 (0.0009) [2023-12-27 07:27:32,756][20347] Updated weights for policy 0, policy_version 261598 (0.0010) [2023-12-27 07:27:32,803][20347] Updated weights for policy 0, policy_version 261608 (0.0009) [2023-12-27 07:27:33,343][20381] Updated weights for policy 1, policy_version 261199 (0.0006) [2023-12-27 07:27:33,404][20381] Updated weights for policy 1, policy_version 261209 (0.0005) [2023-12-27 07:27:33,429][20347] Updated weights for policy 0, policy_version 261618 (0.0007) [2023-12-27 07:27:33,453][20381] Updated weights for policy 1, policy_version 261219 (0.0005) [2023-12-27 07:27:33,473][20347] Updated weights for policy 0, policy_version 261628 (0.0007) [2023-12-27 07:27:33,525][20347] Updated weights for policy 0, policy_version 261638 (0.0007) [2023-12-27 07:27:34,071][20381] Updated weights for policy 1, policy_version 261229 (0.0006) [2023-12-27 07:27:34,135][20381] Updated weights for policy 1, policy_version 261239 (0.0009) [2023-12-27 07:27:34,196][20381] Updated weights for policy 1, policy_version 261249 (0.0007) [2023-12-27 07:27:34,220][20347] Updated weights for policy 0, policy_version 261648 (0.0007) [2023-12-27 07:27:34,285][20347] Updated weights for policy 0, policy_version 261658 (0.0010) [2023-12-27 07:27:34,349][20347] Updated weights for policy 0, policy_version 261668 (0.0008) [2023-12-27 07:27:34,827][20381] Updated weights for policy 1, policy_version 261259 (0.0009) [2023-12-27 07:27:34,885][20381] Updated weights for policy 1, policy_version 261269 (0.0010) [2023-12-27 07:27:34,946][20381] Updated weights for policy 1, policy_version 261279 (0.0010) [2023-12-27 07:27:34,995][20347] Updated weights for policy 0, policy_version 261678 (0.0009) [2023-12-27 07:27:35,054][20347] Updated weights for policy 0, policy_version 261688 (0.0008) [2023-12-27 07:27:35,120][20347] Updated weights for policy 0, policy_version 261698 (0.0005) [2023-12-27 07:27:35,629][20381] Updated weights for policy 1, policy_version 261289 (0.0011) [2023-12-27 07:27:35,678][20381] Updated weights for policy 1, policy_version 261299 (0.0010) [2023-12-27 07:27:35,725][20347] Updated weights for policy 0, policy_version 261708 (0.0005) [2023-12-27 07:27:35,729][20381] Updated weights for policy 1, policy_version 261309 (0.0010) [2023-12-27 07:27:35,782][20347] Updated weights for policy 0, policy_version 261718 (0.0005) [2023-12-27 07:27:35,784][20381] Updated weights for policy 1, policy_version 261319 (0.0010) [2023-12-27 07:27:35,831][20347] Updated weights for policy 0, policy_version 261728 (0.0005) [2023-12-27 07:27:36,466][20381] Updated weights for policy 1, policy_version 261329 (0.0011) [2023-12-27 07:27:36,481][20347] Updated weights for policy 0, policy_version 261738 (0.0006) [2023-12-27 07:27:36,522][20381] Updated weights for policy 1, policy_version 261339 (0.0011) [2023-12-27 07:27:36,537][20347] Updated weights for policy 0, policy_version 261748 (0.0010) [2023-12-27 07:27:36,579][20381] Updated weights for policy 1, policy_version 261349 (0.0011) [2023-12-27 07:27:36,600][20347] Updated weights for policy 0, policy_version 261758 (0.0007) [2023-12-27 07:27:36,660][20347] Updated weights for policy 0, policy_version 261768 (0.0007) [2023-12-27 07:27:37,287][20381] Updated weights for policy 1, policy_version 261359 (0.0011) [2023-12-27 07:27:37,331][20347] Updated weights for policy 0, policy_version 261778 (0.0008) [2023-12-27 07:27:37,334][20381] Updated weights for policy 1, policy_version 261369 (0.0005) [2023-12-27 07:27:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.7, 300 sec: 20799.3). Total num frames: 133947392. Throughput: 0: 10494.9, 1: 10390.5. Samples: 133922468. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:37,363][19170] Avg episode reward: [(0, '998.580'), (1, '904.490')] [2023-12-27 07:27:37,386][20381] Updated weights for policy 1, policy_version 261379 (0.0011) [2023-12-27 07:27:37,391][20347] Updated weights for policy 0, policy_version 261788 (0.0006) [2023-12-27 07:27:37,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000261384_66928640.pth... [2023-12-27 07:27:37,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000260136_66609152.pth [2023-12-27 07:27:37,453][20347] Updated weights for policy 0, policy_version 261798 (0.0006) [2023-12-27 07:27:37,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000261800_67035136.pth... [2023-12-27 07:27:37,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000260552_66715648.pth [2023-12-27 07:27:38,087][20381] Updated weights for policy 1, policy_version 261389 (0.0011) [2023-12-27 07:27:38,145][20347] Updated weights for policy 0, policy_version 261808 (0.0007) [2023-12-27 07:27:38,148][20381] Updated weights for policy 1, policy_version 261399 (0.0010) [2023-12-27 07:27:38,194][20347] Updated weights for policy 0, policy_version 261818 (0.0005) [2023-12-27 07:27:38,206][20381] Updated weights for policy 1, policy_version 261409 (0.0010) [2023-12-27 07:27:38,251][20347] Updated weights for policy 0, policy_version 261828 (0.0005) [2023-12-27 07:27:38,861][20381] Updated weights for policy 1, policy_version 261419 (0.0011) [2023-12-27 07:27:38,890][20347] Updated weights for policy 0, policy_version 261838 (0.0007) [2023-12-27 07:27:38,914][20381] Updated weights for policy 1, policy_version 261429 (0.0011) [2023-12-27 07:27:38,942][20347] Updated weights for policy 0, policy_version 261848 (0.0005) [2023-12-27 07:27:38,962][20381] Updated weights for policy 1, policy_version 261439 (0.0010) [2023-12-27 07:27:39,001][20347] Updated weights for policy 0, policy_version 261858 (0.0006) [2023-12-27 07:27:39,651][20347] Updated weights for policy 0, policy_version 261868 (0.0006) [2023-12-27 07:27:39,670][20381] Updated weights for policy 1, policy_version 261449 (0.0010) [2023-12-27 07:27:39,712][20347] Updated weights for policy 0, policy_version 261878 (0.0007) [2023-12-27 07:27:39,730][20381] Updated weights for policy 1, policy_version 261459 (0.0006) [2023-12-27 07:27:39,773][20347] Updated weights for policy 0, policy_version 261888 (0.0006) [2023-12-27 07:27:39,781][20381] Updated weights for policy 1, policy_version 261469 (0.0006) [2023-12-27 07:27:39,839][20381] Updated weights for policy 1, policy_version 261479 (0.0006) [2023-12-27 07:27:40,480][20381] Updated weights for policy 1, policy_version 261489 (0.0005) [2023-12-27 07:27:40,499][20347] Updated weights for policy 0, policy_version 261898 (0.0008) [2023-12-27 07:27:40,540][20381] Updated weights for policy 1, policy_version 261499 (0.0006) [2023-12-27 07:27:40,560][20347] Updated weights for policy 0, policy_version 261908 (0.0005) [2023-12-27 07:27:40,605][20381] Updated weights for policy 1, policy_version 261509 (0.0008) [2023-12-27 07:27:40,623][20347] Updated weights for policy 0, policy_version 261918 (0.0007) [2023-12-27 07:27:40,679][20347] Updated weights for policy 0, policy_version 261928 (0.0006) [2023-12-27 07:27:41,258][20381] Updated weights for policy 1, policy_version 261519 (0.0007) [2023-12-27 07:27:41,290][20347] Updated weights for policy 0, policy_version 261938 (0.0008) [2023-12-27 07:27:41,316][20381] Updated weights for policy 1, policy_version 261529 (0.0006) [2023-12-27 07:27:41,356][20347] Updated weights for policy 0, policy_version 261948 (0.0008) [2023-12-27 07:27:41,386][20381] Updated weights for policy 1, policy_version 261539 (0.0008) [2023-12-27 07:27:41,425][20347] Updated weights for policy 0, policy_version 261958 (0.0008) [2023-12-27 07:27:42,100][20381] Updated weights for policy 1, policy_version 261549 (0.0008) [2023-12-27 07:27:42,152][20381] Updated weights for policy 1, policy_version 261559 (0.0011) [2023-12-27 07:27:42,184][20347] Updated weights for policy 0, policy_version 261968 (0.0007) [2023-12-27 07:27:42,201][20381] Updated weights for policy 1, policy_version 261569 (0.0010) [2023-12-27 07:27:42,242][20347] Updated weights for policy 0, policy_version 261978 (0.0005) [2023-12-27 07:27:42,295][20347] Updated weights for policy 0, policy_version 261988 (0.0008) [2023-12-27 07:27:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 134062080. Throughput: 0: 10491.1, 1: 10391.4. Samples: 134046708. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:42,363][19170] Avg episode reward: [(0, '998.660'), (1, '894.260')] [2023-12-27 07:27:42,904][20381] Updated weights for policy 1, policy_version 261579 (0.0010) [2023-12-27 07:27:42,966][20381] Updated weights for policy 1, policy_version 261589 (0.0011) [2023-12-27 07:27:42,998][20347] Updated weights for policy 0, policy_version 261998 (0.0006) [2023-12-27 07:27:43,020][20381] Updated weights for policy 1, policy_version 261599 (0.0010) [2023-12-27 07:27:43,058][20347] Updated weights for policy 0, policy_version 262008 (0.0008) [2023-12-27 07:27:43,116][20347] Updated weights for policy 0, policy_version 262018 (0.0008) [2023-12-27 07:27:43,709][20381] Updated weights for policy 1, policy_version 261609 (0.0010) [2023-12-27 07:27:43,757][20381] Updated weights for policy 1, policy_version 261619 (0.0010) [2023-12-27 07:27:43,811][20347] Updated weights for policy 0, policy_version 262028 (0.0006) [2023-12-27 07:27:43,813][20381] Updated weights for policy 1, policy_version 261629 (0.0010) [2023-12-27 07:27:43,862][20381] Updated weights for policy 1, policy_version 261639 (0.0010) [2023-12-27 07:27:43,863][20347] Updated weights for policy 0, policy_version 262038 (0.0005) [2023-12-27 07:27:43,913][20347] Updated weights for policy 0, policy_version 262048 (0.0007) [2023-12-27 07:27:44,510][20347] Updated weights for policy 0, policy_version 262058 (0.0006) [2023-12-27 07:27:44,561][20381] Updated weights for policy 1, policy_version 261649 (0.0010) [2023-12-27 07:27:44,563][20347] Updated weights for policy 0, policy_version 262068 (0.0005) [2023-12-27 07:27:44,608][20347] Updated weights for policy 0, policy_version 262078 (0.0005) [2023-12-27 07:27:44,616][20381] Updated weights for policy 1, policy_version 261659 (0.0010) [2023-12-27 07:27:44,672][20347] Updated weights for policy 0, policy_version 262088 (0.0006) [2023-12-27 07:27:44,679][20381] Updated weights for policy 1, policy_version 261669 (0.0008) [2023-12-27 07:27:45,303][20347] Updated weights for policy 0, policy_version 262098 (0.0006) [2023-12-27 07:27:45,357][20347] Updated weights for policy 0, policy_version 262108 (0.0005) [2023-12-27 07:27:45,415][20347] Updated weights for policy 0, policy_version 262118 (0.0006) [2023-12-27 07:27:45,429][20381] Updated weights for policy 1, policy_version 261679 (0.0007) [2023-12-27 07:27:45,489][20381] Updated weights for policy 1, policy_version 261689 (0.0008) [2023-12-27 07:27:45,542][20381] Updated weights for policy 1, policy_version 261699 (0.0009) [2023-12-27 07:27:45,980][20347] Updated weights for policy 0, policy_version 262128 (0.0005) [2023-12-27 07:27:46,031][20347] Updated weights for policy 0, policy_version 262138 (0.0005) [2023-12-27 07:27:46,075][20347] Updated weights for policy 0, policy_version 262148 (0.0005) [2023-12-27 07:27:46,286][20381] Updated weights for policy 1, policy_version 261709 (0.0007) [2023-12-27 07:27:46,335][20381] Updated weights for policy 1, policy_version 261719 (0.0007) [2023-12-27 07:27:46,383][20381] Updated weights for policy 1, policy_version 261729 (0.0008) [2023-12-27 07:27:46,694][20347] Updated weights for policy 0, policy_version 262158 (0.0005) [2023-12-27 07:27:46,740][20347] Updated weights for policy 0, policy_version 262168 (0.0005) [2023-12-27 07:27:46,798][20347] Updated weights for policy 0, policy_version 262178 (0.0005) [2023-12-27 07:27:47,133][20381] Updated weights for policy 1, policy_version 261739 (0.0007) [2023-12-27 07:27:47,189][20381] Updated weights for policy 1, policy_version 261749 (0.0006) [2023-12-27 07:27:47,238][20381] Updated weights for policy 1, policy_version 261759 (0.0005) [2023-12-27 07:27:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 134160384. Throughput: 0: 10567.5, 1: 10365.1. Samples: 134169188. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:47,363][19170] Avg episode reward: [(0, '998.040'), (1, '923.230')] [2023-12-27 07:27:47,396][20347] Updated weights for policy 0, policy_version 262188 (0.0008) [2023-12-27 07:27:47,456][20347] Updated weights for policy 0, policy_version 262198 (0.0009) [2023-12-27 07:27:47,516][20347] Updated weights for policy 0, policy_version 262208 (0.0010) [2023-12-27 07:27:47,855][20381] Updated weights for policy 1, policy_version 261769 (0.0007) [2023-12-27 07:27:47,910][20381] Updated weights for policy 1, policy_version 261779 (0.0007) [2023-12-27 07:27:47,959][20381] Updated weights for policy 1, policy_version 261789 (0.0008) [2023-12-27 07:27:48,008][20381] Updated weights for policy 1, policy_version 261799 (0.0009) [2023-12-27 07:27:48,171][20347] Updated weights for policy 0, policy_version 262218 (0.0010) [2023-12-27 07:27:48,232][20347] Updated weights for policy 0, policy_version 262228 (0.0010) [2023-12-27 07:27:48,291][20347] Updated weights for policy 0, policy_version 262238 (0.0010) [2023-12-27 07:27:48,353][20347] Updated weights for policy 0, policy_version 262248 (0.0012) [2023-12-27 07:27:48,734][20381] Updated weights for policy 1, policy_version 261809 (0.0010) [2023-12-27 07:27:48,792][20381] Updated weights for policy 1, policy_version 261819 (0.0010) [2023-12-27 07:27:48,850][20381] Updated weights for policy 1, policy_version 261829 (0.0010) [2023-12-27 07:27:49,067][20347] Updated weights for policy 0, policy_version 262258 (0.0008) [2023-12-27 07:27:49,119][20347] Updated weights for policy 0, policy_version 262268 (0.0008) [2023-12-27 07:27:49,183][20347] Updated weights for policy 0, policy_version 262278 (0.0007) [2023-12-27 07:27:49,501][20381] Updated weights for policy 1, policy_version 261839 (0.0007) [2023-12-27 07:27:49,553][20381] Updated weights for policy 1, policy_version 261849 (0.0007) [2023-12-27 07:27:49,608][20381] Updated weights for policy 1, policy_version 261859 (0.0008) [2023-12-27 07:27:49,869][20347] Updated weights for policy 0, policy_version 262288 (0.0006) [2023-12-27 07:27:49,937][20347] Updated weights for policy 0, policy_version 262298 (0.0011) [2023-12-27 07:27:50,002][20347] Updated weights for policy 0, policy_version 262308 (0.0009) [2023-12-27 07:27:50,312][20381] Updated weights for policy 1, policy_version 261869 (0.0008) [2023-12-27 07:27:50,368][20381] Updated weights for policy 1, policy_version 261879 (0.0008) [2023-12-27 07:27:50,427][20381] Updated weights for policy 1, policy_version 261889 (0.0009) [2023-12-27 07:27:50,676][20347] Updated weights for policy 0, policy_version 262318 (0.0010) [2023-12-27 07:27:50,737][20347] Updated weights for policy 0, policy_version 262328 (0.0010) [2023-12-27 07:27:50,798][20347] Updated weights for policy 0, policy_version 262338 (0.0010) [2023-12-27 07:27:51,148][20381] Updated weights for policy 1, policy_version 261899 (0.0011) [2023-12-27 07:27:51,206][20381] Updated weights for policy 1, policy_version 261909 (0.0007) [2023-12-27 07:27:51,260][20381] Updated weights for policy 1, policy_version 261919 (0.0008) [2023-12-27 07:27:51,480][20347] Updated weights for policy 0, policy_version 262348 (0.0010) [2023-12-27 07:27:51,529][20347] Updated weights for policy 0, policy_version 262358 (0.0009) [2023-12-27 07:27:51,586][20347] Updated weights for policy 0, policy_version 262368 (0.0008) [2023-12-27 07:27:51,987][20381] Updated weights for policy 1, policy_version 261929 (0.0008) [2023-12-27 07:27:52,047][20381] Updated weights for policy 1, policy_version 261939 (0.0006) [2023-12-27 07:27:52,098][20381] Updated weights for policy 1, policy_version 261949 (0.0007) [2023-12-27 07:27:52,156][20381] Updated weights for policy 1, policy_version 261959 (0.0008) [2023-12-27 07:27:52,289][20347] Updated weights for policy 0, policy_version 262378 (0.0010) [2023-12-27 07:27:52,351][20347] Updated weights for policy 0, policy_version 262388 (0.0008) [2023-12-27 07:27:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 134258688. Throughput: 0: 10551.5, 1: 10334.7. Samples: 134230784. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:52,363][19170] Avg episode reward: [(0, '999.380'), (1, '942.630')] [2023-12-27 07:27:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000261960_67076096.pth... [2023-12-27 07:27:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000260744_66764800.pth [2023-12-27 07:27:52,415][20347] Updated weights for policy 0, policy_version 262398 (0.0010) [2023-12-27 07:27:52,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000262408_67190784.pth... [2023-12-27 07:27:52,475][20347] Updated weights for policy 0, policy_version 262408 (0.0010) [2023-12-27 07:27:52,480][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000261160_66871296.pth [2023-12-27 07:27:52,828][20381] Updated weights for policy 1, policy_version 261969 (0.0008) [2023-12-27 07:27:52,875][20381] Updated weights for policy 1, policy_version 261979 (0.0007) [2023-12-27 07:27:52,936][20381] Updated weights for policy 1, policy_version 261989 (0.0007) [2023-12-27 07:27:53,142][20347] Updated weights for policy 0, policy_version 262418 (0.0010) [2023-12-27 07:27:53,196][20347] Updated weights for policy 0, policy_version 262428 (0.0010) [2023-12-27 07:27:53,256][20347] Updated weights for policy 0, policy_version 262438 (0.0010) [2023-12-27 07:27:53,659][20381] Updated weights for policy 1, policy_version 261999 (0.0006) [2023-12-27 07:27:53,720][20381] Updated weights for policy 1, policy_version 262009 (0.0007) [2023-12-27 07:27:53,774][20381] Updated weights for policy 1, policy_version 262019 (0.0010) [2023-12-27 07:27:53,907][20347] Updated weights for policy 0, policy_version 262448 (0.0010) [2023-12-27 07:27:53,956][20347] Updated weights for policy 0, policy_version 262458 (0.0006) [2023-12-27 07:27:54,002][20347] Updated weights for policy 0, policy_version 262468 (0.0005) [2023-12-27 07:27:54,343][20381] Updated weights for policy 1, policy_version 262029 (0.0006) [2023-12-27 07:27:54,394][20381] Updated weights for policy 1, policy_version 262039 (0.0005) [2023-12-27 07:27:54,448][20381] Updated weights for policy 1, policy_version 262049 (0.0006) [2023-12-27 07:27:54,650][20347] Updated weights for policy 0, policy_version 262478 (0.0008) [2023-12-27 07:27:54,699][20347] Updated weights for policy 0, policy_version 262488 (0.0010) [2023-12-27 07:27:54,755][20347] Updated weights for policy 0, policy_version 262499 (0.0007) [2023-12-27 07:27:55,122][20381] Updated weights for policy 1, policy_version 262059 (0.0010) [2023-12-27 07:27:55,178][20381] Updated weights for policy 1, policy_version 262069 (0.0010) [2023-12-27 07:27:55,230][20381] Updated weights for policy 1, policy_version 262079 (0.0010) [2023-12-27 07:27:55,367][20347] Updated weights for policy 0, policy_version 262509 (0.0005) [2023-12-27 07:27:55,428][20347] Updated weights for policy 0, policy_version 262519 (0.0005) [2023-12-27 07:27:55,485][20347] Updated weights for policy 0, policy_version 262529 (0.0005) [2023-12-27 07:27:55,896][20381] Updated weights for policy 1, policy_version 262089 (0.0010) [2023-12-27 07:27:55,951][20381] Updated weights for policy 1, policy_version 262099 (0.0010) [2023-12-27 07:27:55,996][20381] Updated weights for policy 1, policy_version 262109 (0.0010) [2023-12-27 07:27:56,048][20381] Updated weights for policy 1, policy_version 262119 (0.0010) [2023-12-27 07:27:56,070][20347] Updated weights for policy 0, policy_version 262539 (0.0007) [2023-12-27 07:27:56,115][20347] Updated weights for policy 0, policy_version 262549 (0.0010) [2023-12-27 07:27:56,161][20347] Updated weights for policy 0, policy_version 262559 (0.0010) [2023-12-27 07:27:56,752][20381] Updated weights for policy 1, policy_version 262129 (0.0007) [2023-12-27 07:27:56,757][20347] Updated weights for policy 0, policy_version 262569 (0.0010) [2023-12-27 07:27:56,801][20381] Updated weights for policy 1, policy_version 262139 (0.0007) [2023-12-27 07:27:56,814][20347] Updated weights for policy 0, policy_version 262579 (0.0010) [2023-12-27 07:27:56,853][20381] Updated weights for policy 1, policy_version 262149 (0.0007) [2023-12-27 07:27:56,872][20347] Updated weights for policy 0, policy_version 262589 (0.0010) [2023-12-27 07:27:56,929][20347] Updated weights for policy 0, policy_version 262599 (0.0010) [2023-12-27 07:27:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 134365184. Throughput: 0: 10516.2, 1: 10272.5. Samples: 134355264. Policy #0 lag: (min: 26.0, avg: 33.8, max: 58.0) [2023-12-27 07:27:57,363][19170] Avg episode reward: [(0, '999.610'), (1, '923.810')] [2023-12-27 07:27:57,465][20381] Updated weights for policy 1, policy_version 262159 (0.0007) [2023-12-27 07:27:57,500][20347] Updated weights for policy 0, policy_version 262609 (0.0010) [2023-12-27 07:27:57,513][20381] Updated weights for policy 1, policy_version 262169 (0.0005) [2023-12-27 07:27:57,551][20347] Updated weights for policy 0, policy_version 262619 (0.0010) [2023-12-27 07:27:57,561][20381] Updated weights for policy 1, policy_version 262179 (0.0005) [2023-12-27 07:27:57,599][20347] Updated weights for policy 0, policy_version 262629 (0.0010) [2023-12-27 07:27:58,115][20381] Updated weights for policy 1, policy_version 262189 (0.0005) [2023-12-27 07:27:58,167][20381] Updated weights for policy 1, policy_version 262199 (0.0006) [2023-12-27 07:27:58,228][20381] Updated weights for policy 1, policy_version 262209 (0.0008) [2023-12-27 07:27:58,247][20347] Updated weights for policy 0, policy_version 262639 (0.0008) [2023-12-27 07:27:58,306][20347] Updated weights for policy 0, policy_version 262649 (0.0009) [2023-12-27 07:27:58,378][20347] Updated weights for policy 0, policy_version 262659 (0.0009) [2023-12-27 07:27:59,010][20381] Updated weights for policy 1, policy_version 262219 (0.0006) [2023-12-27 07:27:59,099][20381] Updated weights for policy 1, policy_version 262229 (0.0006) [2023-12-27 07:27:59,159][20347] Updated weights for policy 0, policy_version 262669 (0.0007) [2023-12-27 07:27:59,160][20381] Updated weights for policy 1, policy_version 262239 (0.0006) [2023-12-27 07:27:59,222][20347] Updated weights for policy 0, policy_version 262679 (0.0007) [2023-12-27 07:27:59,282][20347] Updated weights for policy 0, policy_version 262689 (0.0007) [2023-12-27 07:27:59,802][20381] Updated weights for policy 1, policy_version 262249 (0.0007) [2023-12-27 07:27:59,861][20381] Updated weights for policy 1, policy_version 262259 (0.0007) [2023-12-27 07:27:59,908][20381] Updated weights for policy 1, policy_version 262269 (0.0008) [2023-12-27 07:27:59,973][20381] Updated weights for policy 1, policy_version 262279 (0.0008) [2023-12-27 07:27:59,990][20347] Updated weights for policy 0, policy_version 262699 (0.0008) [2023-12-27 07:28:00,045][20347] Updated weights for policy 0, policy_version 262709 (0.0007) [2023-12-27 07:28:00,099][20347] Updated weights for policy 0, policy_version 262719 (0.0007) [2023-12-27 07:28:00,679][20381] Updated weights for policy 1, policy_version 262289 (0.0010) [2023-12-27 07:28:00,695][20347] Updated weights for policy 0, policy_version 262729 (0.0007) [2023-12-27 07:28:00,733][20381] Updated weights for policy 1, policy_version 262300 (0.0010) [2023-12-27 07:28:00,754][20347] Updated weights for policy 0, policy_version 262739 (0.0006) [2023-12-27 07:28:00,781][20381] Updated weights for policy 1, policy_version 262310 (0.0005) [2023-12-27 07:28:00,820][20347] Updated weights for policy 0, policy_version 262749 (0.0005) [2023-12-27 07:28:00,875][20347] Updated weights for policy 0, policy_version 262759 (0.0005) [2023-12-27 07:28:01,498][20347] Updated weights for policy 0, policy_version 262769 (0.0007) [2023-12-27 07:28:01,517][20381] Updated weights for policy 1, policy_version 262320 (0.0009) [2023-12-27 07:28:01,553][20347] Updated weights for policy 0, policy_version 262779 (0.0007) [2023-12-27 07:28:01,574][20381] Updated weights for policy 1, policy_version 262330 (0.0006) [2023-12-27 07:28:01,607][20347] Updated weights for policy 0, policy_version 262789 (0.0005) [2023-12-27 07:28:01,638][20381] Updated weights for policy 1, policy_version 262340 (0.0007) [2023-12-27 07:28:02,274][20347] Updated weights for policy 0, policy_version 262799 (0.0008) [2023-12-27 07:28:02,326][20347] Updated weights for policy 0, policy_version 262809 (0.0007) [2023-12-27 07:28:02,362][20381] Updated weights for policy 1, policy_version 262350 (0.0007) [2023-12-27 07:28:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 134463488. Throughput: 0: 10508.7, 1: 10239.3. Samples: 134479620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:02,363][19170] Avg episode reward: [(0, '999.610'), (1, '952.190')] [2023-12-27 07:28:02,387][20347] Updated weights for policy 0, policy_version 262819 (0.0007) [2023-12-27 07:28:02,422][20381] Updated weights for policy 1, policy_version 262360 (0.0008) [2023-12-27 07:28:02,471][20381] Updated weights for policy 1, policy_version 262370 (0.0008) [2023-12-27 07:28:03,052][20347] Updated weights for policy 0, policy_version 262829 (0.0008) [2023-12-27 07:28:03,100][20347] Updated weights for policy 0, policy_version 262839 (0.0010) [2023-12-27 07:28:03,154][20347] Updated weights for policy 0, policy_version 262849 (0.0010) [2023-12-27 07:28:03,179][20381] Updated weights for policy 1, policy_version 262380 (0.0006) [2023-12-27 07:28:03,228][20381] Updated weights for policy 1, policy_version 262390 (0.0007) [2023-12-27 07:28:03,277][20381] Updated weights for policy 1, policy_version 262400 (0.0007) [2023-12-27 07:28:03,754][20347] Updated weights for policy 0, policy_version 262859 (0.0009) [2023-12-27 07:28:03,798][20347] Updated weights for policy 0, policy_version 262869 (0.0007) [2023-12-27 07:28:03,846][20347] Updated weights for policy 0, policy_version 262879 (0.0006) [2023-12-27 07:28:04,011][20381] Updated weights for policy 1, policy_version 262410 (0.0008) [2023-12-27 07:28:04,066][20381] Updated weights for policy 1, policy_version 262420 (0.0008) [2023-12-27 07:28:04,127][20381] Updated weights for policy 1, policy_version 262430 (0.0008) [2023-12-27 07:28:04,191][20381] Updated weights for policy 1, policy_version 262440 (0.0008) [2023-12-27 07:28:04,579][20347] Updated weights for policy 0, policy_version 262889 (0.0008) [2023-12-27 07:28:04,637][20347] Updated weights for policy 0, policy_version 262899 (0.0008) [2023-12-27 07:28:04,699][20347] Updated weights for policy 0, policy_version 262909 (0.0006) [2023-12-27 07:28:04,757][20347] Updated weights for policy 0, policy_version 262919 (0.0005) [2023-12-27 07:28:04,897][20381] Updated weights for policy 1, policy_version 262450 (0.0005) [2023-12-27 07:28:04,955][20381] Updated weights for policy 1, policy_version 262460 (0.0007) [2023-12-27 07:28:05,004][20381] Updated weights for policy 1, policy_version 262470 (0.0008) [2023-12-27 07:28:05,331][20347] Updated weights for policy 0, policy_version 262929 (0.0005) [2023-12-27 07:28:05,381][20347] Updated weights for policy 0, policy_version 262939 (0.0005) [2023-12-27 07:28:05,429][20347] Updated weights for policy 0, policy_version 262949 (0.0005) [2023-12-27 07:28:05,700][20381] Updated weights for policy 1, policy_version 262480 (0.0008) [2023-12-27 07:28:05,759][20381] Updated weights for policy 1, policy_version 262490 (0.0007) [2023-12-27 07:28:05,814][20381] Updated weights for policy 1, policy_version 262500 (0.0008) [2023-12-27 07:28:06,084][20347] Updated weights for policy 0, policy_version 262959 (0.0008) [2023-12-27 07:28:06,146][20347] Updated weights for policy 0, policy_version 262969 (0.0008) [2023-12-27 07:28:06,195][20347] Updated weights for policy 0, policy_version 262979 (0.0008) [2023-12-27 07:28:06,479][20381] Updated weights for policy 1, policy_version 262510 (0.0009) [2023-12-27 07:28:06,531][20381] Updated weights for policy 1, policy_version 262520 (0.0011) [2023-12-27 07:28:06,587][20381] Updated weights for policy 1, policy_version 262530 (0.0010) [2023-12-27 07:28:06,893][20347] Updated weights for policy 0, policy_version 262989 (0.0010) [2023-12-27 07:28:06,957][20347] Updated weights for policy 0, policy_version 262999 (0.0010) [2023-12-27 07:28:07,009][20347] Updated weights for policy 0, policy_version 263009 (0.0010) [2023-12-27 07:28:07,210][20381] Updated weights for policy 1, policy_version 262540 (0.0011) [2023-12-27 07:28:07,261][20381] Updated weights for policy 1, policy_version 262550 (0.0010) [2023-12-27 07:28:07,317][20381] Updated weights for policy 1, policy_version 262560 (0.0011) [2023-12-27 07:28:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 134578176. Throughput: 0: 10531.8, 1: 10185.0. Samples: 134540788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '980.750')] [2023-12-27 07:28:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000263016_67346432.pth... [2023-12-27 07:28:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000262568_67231744.pth... [2023-12-27 07:28:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000261800_67035136.pth [2023-12-27 07:28:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000261384_66928640.pth [2023-12-27 07:28:07,632][20347] Updated weights for policy 0, policy_version 263019 (0.0006) [2023-12-27 07:28:07,692][20347] Updated weights for policy 0, policy_version 263029 (0.0008) [2023-12-27 07:28:07,747][20347] Updated weights for policy 0, policy_version 263039 (0.0008) [2023-12-27 07:28:07,959][20381] Updated weights for policy 1, policy_version 262570 (0.0010) [2023-12-27 07:28:08,022][20381] Updated weights for policy 1, policy_version 262580 (0.0011) [2023-12-27 07:28:08,069][20381] Updated weights for policy 1, policy_version 262590 (0.0010) [2023-12-27 07:28:08,116][20381] Updated weights for policy 1, policy_version 262600 (0.0010) [2023-12-27 07:28:08,471][20347] Updated weights for policy 0, policy_version 263049 (0.0008) [2023-12-27 07:28:08,532][20347] Updated weights for policy 0, policy_version 263059 (0.0005) [2023-12-27 07:28:08,591][20347] Updated weights for policy 0, policy_version 263069 (0.0006) [2023-12-27 07:28:08,652][20347] Updated weights for policy 0, policy_version 263079 (0.0005) [2023-12-27 07:28:08,848][20381] Updated weights for policy 1, policy_version 262610 (0.0009) [2023-12-27 07:28:08,912][20381] Updated weights for policy 1, policy_version 262620 (0.0007) [2023-12-27 07:28:08,968][20381] Updated weights for policy 1, policy_version 262630 (0.0010) [2023-12-27 07:28:09,319][20347] Updated weights for policy 0, policy_version 263089 (0.0007) [2023-12-27 07:28:09,387][20347] Updated weights for policy 0, policy_version 263099 (0.0008) [2023-12-27 07:28:09,454][20347] Updated weights for policy 0, policy_version 263109 (0.0008) [2023-12-27 07:28:09,658][20381] Updated weights for policy 1, policy_version 262640 (0.0011) [2023-12-27 07:28:09,719][20381] Updated weights for policy 1, policy_version 262650 (0.0010) [2023-12-27 07:28:09,781][20381] Updated weights for policy 1, policy_version 262660 (0.0009) [2023-12-27 07:28:10,191][20347] Updated weights for policy 0, policy_version 263119 (0.0008) [2023-12-27 07:28:10,248][20347] Updated weights for policy 0, policy_version 263129 (0.0007) [2023-12-27 07:28:10,305][20347] Updated weights for policy 0, policy_version 263139 (0.0008) [2023-12-27 07:28:10,392][20381] Updated weights for policy 1, policy_version 262670 (0.0008) [2023-12-27 07:28:10,439][20381] Updated weights for policy 1, policy_version 262680 (0.0006) [2023-12-27 07:28:10,487][20381] Updated weights for policy 1, policy_version 262690 (0.0005) [2023-12-27 07:28:11,037][20347] Updated weights for policy 0, policy_version 263149 (0.0008) [2023-12-27 07:28:11,097][20347] Updated weights for policy 0, policy_version 263159 (0.0008) [2023-12-27 07:28:11,104][20381] Updated weights for policy 1, policy_version 262700 (0.0006) [2023-12-27 07:28:11,161][20347] Updated weights for policy 0, policy_version 263169 (0.0008) [2023-12-27 07:28:11,166][20381] Updated weights for policy 1, policy_version 262710 (0.0006) [2023-12-27 07:28:11,225][20381] Updated weights for policy 1, policy_version 262720 (0.0007) [2023-12-27 07:28:11,898][20347] Updated weights for policy 0, policy_version 263179 (0.0008) [2023-12-27 07:28:11,964][20347] Updated weights for policy 0, policy_version 263189 (0.0010) [2023-12-27 07:28:11,981][20381] Updated weights for policy 1, policy_version 262730 (0.0008) [2023-12-27 07:28:12,019][20347] Updated weights for policy 0, policy_version 263199 (0.0009) [2023-12-27 07:28:12,030][20381] Updated weights for policy 1, policy_version 262740 (0.0010) [2023-12-27 07:28:12,079][20381] Updated weights for policy 1, policy_version 262750 (0.0005) [2023-12-27 07:28:12,131][20381] Updated weights for policy 1, policy_version 262760 (0.0007) [2023-12-27 07:28:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 134676480. Throughput: 0: 10494.8, 1: 10249.3. Samples: 134664052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:12,363][19170] Avg episode reward: [(0, '995.500'), (1, '1000.000')] [2023-12-27 07:28:12,655][20347] Updated weights for policy 0, policy_version 263209 (0.0009) [2023-12-27 07:28:12,712][20347] Updated weights for policy 0, policy_version 263219 (0.0010) [2023-12-27 07:28:12,772][20347] Updated weights for policy 0, policy_version 263229 (0.0008) [2023-12-27 07:28:12,813][20381] Updated weights for policy 1, policy_version 262770 (0.0010) [2023-12-27 07:28:12,827][20347] Updated weights for policy 0, policy_version 263239 (0.0009) [2023-12-27 07:28:12,867][20381] Updated weights for policy 1, policy_version 262780 (0.0008) [2023-12-27 07:28:12,924][20381] Updated weights for policy 1, policy_version 262790 (0.0010) [2023-12-27 07:28:13,444][20347] Updated weights for policy 0, policy_version 263249 (0.0005) [2023-12-27 07:28:13,495][20347] Updated weights for policy 0, policy_version 263259 (0.0005) [2023-12-27 07:28:13,574][20381] Updated weights for policy 1, policy_version 262800 (0.0006) [2023-12-27 07:28:13,577][20347] Updated weights for policy 0, policy_version 263269 (0.0006) [2023-12-27 07:28:13,623][20381] Updated weights for policy 1, policy_version 262810 (0.0005) [2023-12-27 07:28:13,681][20381] Updated weights for policy 1, policy_version 262820 (0.0005) [2023-12-27 07:28:14,205][20347] Updated weights for policy 0, policy_version 263279 (0.0006) [2023-12-27 07:28:14,260][20347] Updated weights for policy 0, policy_version 263289 (0.0005) [2023-12-27 07:28:14,302][20381] Updated weights for policy 1, policy_version 262830 (0.0005) [2023-12-27 07:28:14,322][20347] Updated weights for policy 0, policy_version 263299 (0.0006) [2023-12-27 07:28:14,365][20381] Updated weights for policy 1, policy_version 262840 (0.0007) [2023-12-27 07:28:14,425][20381] Updated weights for policy 1, policy_version 262850 (0.0008) [2023-12-27 07:28:14,980][20347] Updated weights for policy 0, policy_version 263309 (0.0006) [2023-12-27 07:28:15,032][20347] Updated weights for policy 0, policy_version 263319 (0.0007) [2023-12-27 07:28:15,079][20381] Updated weights for policy 1, policy_version 262860 (0.0008) [2023-12-27 07:28:15,086][20347] Updated weights for policy 0, policy_version 263329 (0.0006) [2023-12-27 07:28:15,135][20381] Updated weights for policy 1, policy_version 262870 (0.0008) [2023-12-27 07:28:15,190][20381] Updated weights for policy 1, policy_version 262880 (0.0008) [2023-12-27 07:28:15,755][20347] Updated weights for policy 0, policy_version 263339 (0.0006) [2023-12-27 07:28:15,813][20347] Updated weights for policy 0, policy_version 263349 (0.0006) [2023-12-27 07:28:15,869][20347] Updated weights for policy 0, policy_version 263359 (0.0007) [2023-12-27 07:28:15,933][20381] Updated weights for policy 1, policy_version 262890 (0.0008) [2023-12-27 07:28:15,995][20381] Updated weights for policy 1, policy_version 262900 (0.0008) [2023-12-27 07:28:16,058][20381] Updated weights for policy 1, policy_version 262910 (0.0008) [2023-12-27 07:28:16,119][20381] Updated weights for policy 1, policy_version 262920 (0.0006) [2023-12-27 07:28:16,414][20347] Updated weights for policy 0, policy_version 263369 (0.0008) [2023-12-27 07:28:16,462][20347] Updated weights for policy 0, policy_version 263379 (0.0010) [2023-12-27 07:28:16,510][20347] Updated weights for policy 0, policy_version 263389 (0.0010) [2023-12-27 07:28:16,558][20347] Updated weights for policy 0, policy_version 263399 (0.0010) [2023-12-27 07:28:16,685][20381] Updated weights for policy 1, policy_version 262930 (0.0005) [2023-12-27 07:28:16,748][20381] Updated weights for policy 1, policy_version 262940 (0.0005) [2023-12-27 07:28:16,804][20381] Updated weights for policy 1, policy_version 262950 (0.0005) [2023-12-27 07:28:17,216][20347] Updated weights for policy 0, policy_version 263409 (0.0007) [2023-12-27 07:28:17,264][20347] Updated weights for policy 0, policy_version 263419 (0.0008) [2023-12-27 07:28:17,316][20347] Updated weights for policy 0, policy_version 263429 (0.0008) [2023-12-27 07:28:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 134782976. Throughput: 0: 10492.4, 1: 10230.7. Samples: 134791204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:17,363][19170] Avg episode reward: [(0, '984.810'), (1, '1000.000')] [2023-12-27 07:28:17,392][20381] Updated weights for policy 1, policy_version 262960 (0.0007) [2023-12-27 07:28:17,437][20381] Updated weights for policy 1, policy_version 262970 (0.0008) [2023-12-27 07:28:17,486][20381] Updated weights for policy 1, policy_version 262980 (0.0008) [2023-12-27 07:28:18,007][20347] Updated weights for policy 0, policy_version 263439 (0.0007) [2023-12-27 07:28:18,059][20347] Updated weights for policy 0, policy_version 263449 (0.0007) [2023-12-27 07:28:18,107][20347] Updated weights for policy 0, policy_version 263459 (0.0009) [2023-12-27 07:28:18,158][20381] Updated weights for policy 1, policy_version 262990 (0.0006) [2023-12-27 07:28:18,212][20381] Updated weights for policy 1, policy_version 263000 (0.0005) [2023-12-27 07:28:18,261][20381] Updated weights for policy 1, policy_version 263010 (0.0006) [2023-12-27 07:28:18,832][20347] Updated weights for policy 0, policy_version 263469 (0.0009) [2023-12-27 07:28:18,887][20347] Updated weights for policy 0, policy_version 263479 (0.0009) [2023-12-27 07:28:18,919][20381] Updated weights for policy 1, policy_version 263020 (0.0007) [2023-12-27 07:28:18,939][20347] Updated weights for policy 0, policy_version 263489 (0.0006) [2023-12-27 07:28:18,975][20381] Updated weights for policy 1, policy_version 263030 (0.0006) [2023-12-27 07:28:19,030][20381] Updated weights for policy 1, policy_version 263040 (0.0008) [2023-12-27 07:28:19,607][20347] Updated weights for policy 0, policy_version 263499 (0.0005) [2023-12-27 07:28:19,657][20347] Updated weights for policy 0, policy_version 263509 (0.0005) [2023-12-27 07:28:19,711][20347] Updated weights for policy 0, policy_version 263519 (0.0005) [2023-12-27 07:28:19,763][20381] Updated weights for policy 1, policy_version 263050 (0.0007) [2023-12-27 07:28:19,810][20381] Updated weights for policy 1, policy_version 263060 (0.0008) [2023-12-27 07:28:19,876][20381] Updated weights for policy 1, policy_version 263070 (0.0008) [2023-12-27 07:28:19,937][20381] Updated weights for policy 1, policy_version 263080 (0.0007) [2023-12-27 07:28:20,395][20347] Updated weights for policy 0, policy_version 263529 (0.0008) [2023-12-27 07:28:20,456][20347] Updated weights for policy 0, policy_version 263539 (0.0007) [2023-12-27 07:28:20,518][20347] Updated weights for policy 0, policy_version 263549 (0.0007) [2023-12-27 07:28:20,578][20347] Updated weights for policy 0, policy_version 263559 (0.0007) [2023-12-27 07:28:20,659][20381] Updated weights for policy 1, policy_version 263090 (0.0007) [2023-12-27 07:28:20,717][20381] Updated weights for policy 1, policy_version 263100 (0.0009) [2023-12-27 07:28:20,778][20381] Updated weights for policy 1, policy_version 263110 (0.0008) [2023-12-27 07:28:21,256][20347] Updated weights for policy 0, policy_version 263569 (0.0008) [2023-12-27 07:28:21,320][20347] Updated weights for policy 0, policy_version 263579 (0.0007) [2023-12-27 07:28:21,383][20347] Updated weights for policy 0, policy_version 263589 (0.0007) [2023-12-27 07:28:21,470][20381] Updated weights for policy 1, policy_version 263120 (0.0008) [2023-12-27 07:28:21,524][20381] Updated weights for policy 1, policy_version 263130 (0.0008) [2023-12-27 07:28:21,584][20381] Updated weights for policy 1, policy_version 263140 (0.0007) [2023-12-27 07:28:22,085][20347] Updated weights for policy 0, policy_version 263599 (0.0006) [2023-12-27 07:28:22,144][20347] Updated weights for policy 0, policy_version 263609 (0.0005) [2023-12-27 07:28:22,200][20347] Updated weights for policy 0, policy_version 263619 (0.0006) [2023-12-27 07:28:22,312][20381] Updated weights for policy 1, policy_version 263150 (0.0007) [2023-12-27 07:28:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 134881280. Throughput: 0: 10462.7, 1: 10184.3. Samples: 134851580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:22,363][19170] Avg episode reward: [(0, '989.310'), (1, '1000.000')] [2023-12-27 07:28:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000263624_67502080.pth... [2023-12-27 07:28:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000262408_67190784.pth [2023-12-27 07:28:22,376][20381] Updated weights for policy 1, policy_version 263160 (0.0008) [2023-12-27 07:28:22,442][20381] Updated weights for policy 1, policy_version 263170 (0.0009) [2023-12-27 07:28:22,479][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000263176_67387392.pth... [2023-12-27 07:28:22,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000261960_67076096.pth [2023-12-27 07:28:22,820][20347] Updated weights for policy 0, policy_version 263629 (0.0008) [2023-12-27 07:28:22,883][20347] Updated weights for policy 0, policy_version 263639 (0.0008) [2023-12-27 07:28:22,945][20347] Updated weights for policy 0, policy_version 263649 (0.0008) [2023-12-27 07:28:23,199][20381] Updated weights for policy 1, policy_version 263180 (0.0008) [2023-12-27 07:28:23,257][20381] Updated weights for policy 1, policy_version 263190 (0.0008) [2023-12-27 07:28:23,307][20381] Updated weights for policy 1, policy_version 263200 (0.0007) [2023-12-27 07:28:23,629][20347] Updated weights for policy 0, policy_version 263659 (0.0007) [2023-12-27 07:28:23,674][20347] Updated weights for policy 0, policy_version 263669 (0.0007) [2023-12-27 07:28:23,722][20347] Updated weights for policy 0, policy_version 263679 (0.0007) [2023-12-27 07:28:23,971][20381] Updated weights for policy 1, policy_version 263210 (0.0006) [2023-12-27 07:28:24,035][20381] Updated weights for policy 1, policy_version 263220 (0.0005) [2023-12-27 07:28:24,088][20381] Updated weights for policy 1, policy_version 263230 (0.0005) [2023-12-27 07:28:24,136][20381] Updated weights for policy 1, policy_version 263240 (0.0005) [2023-12-27 07:28:24,327][20347] Updated weights for policy 0, policy_version 263689 (0.0005) [2023-12-27 07:28:24,386][20347] Updated weights for policy 0, policy_version 263699 (0.0005) [2023-12-27 07:28:24,445][20347] Updated weights for policy 0, policy_version 263709 (0.0005) [2023-12-27 07:28:24,491][20347] Updated weights for policy 0, policy_version 263719 (0.0005) [2023-12-27 07:28:24,750][20381] Updated weights for policy 1, policy_version 263250 (0.0007) [2023-12-27 07:28:24,804][20381] Updated weights for policy 1, policy_version 263260 (0.0007) [2023-12-27 07:28:24,849][20381] Updated weights for policy 1, policy_version 263270 (0.0007) [2023-12-27 07:28:25,111][20347] Updated weights for policy 0, policy_version 263729 (0.0007) [2023-12-27 07:28:25,166][20347] Updated weights for policy 0, policy_version 263739 (0.0007) [2023-12-27 07:28:25,224][20347] Updated weights for policy 0, policy_version 263749 (0.0008) [2023-12-27 07:28:25,574][20381] Updated weights for policy 1, policy_version 263280 (0.0008) [2023-12-27 07:28:25,620][20381] Updated weights for policy 1, policy_version 263290 (0.0007) [2023-12-27 07:28:25,668][20381] Updated weights for policy 1, policy_version 263300 (0.0007) [2023-12-27 07:28:25,850][20347] Updated weights for policy 0, policy_version 263759 (0.0007) [2023-12-27 07:28:25,898][20347] Updated weights for policy 0, policy_version 263769 (0.0010) [2023-12-27 07:28:25,946][20347] Updated weights for policy 0, policy_version 263779 (0.0007) [2023-12-27 07:28:26,323][20381] Updated weights for policy 1, policy_version 263310 (0.0007) [2023-12-27 07:28:26,378][20381] Updated weights for policy 1, policy_version 263320 (0.0006) [2023-12-27 07:28:26,431][20381] Updated weights for policy 1, policy_version 263330 (0.0006) [2023-12-27 07:28:26,549][20347] Updated weights for policy 0, policy_version 263789 (0.0008) [2023-12-27 07:28:26,603][20347] Updated weights for policy 0, policy_version 263799 (0.0010) [2023-12-27 07:28:26,656][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-27 07:28:27,001][20381] Updated weights for policy 1, policy_version 263340 (0.0007) [2023-12-27 07:28:27,061][20381] Updated weights for policy 1, policy_version 263350 (0.0007) [2023-12-27 07:28:27,121][20381] Updated weights for policy 1, policy_version 263360 (0.0008) [2023-12-27 07:28:27,262][20347] Updated weights for policy 0, policy_version 263809 (0.0010) [2023-12-27 07:28:27,318][20347] Updated weights for policy 0, policy_version 263819 (0.0010) [2023-12-27 07:28:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 134987776. Throughput: 0: 10467.9, 1: 10169.2. Samples: 134975376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:28:27,369][20347] Updated weights for policy 0, policy_version 263829 (0.0010) [2023-12-27 07:28:27,424][20347] Updated weights for policy 0, policy_version 263839 (0.0006) [2023-12-27 07:28:27,739][20381] Updated weights for policy 1, policy_version 263370 (0.0008) [2023-12-27 07:28:27,787][20381] Updated weights for policy 1, policy_version 263380 (0.0005) [2023-12-27 07:28:27,836][20381] Updated weights for policy 1, policy_version 263390 (0.0005) [2023-12-27 07:28:27,887][20381] Updated weights for policy 1, policy_version 263400 (0.0005) [2023-12-27 07:28:28,107][20347] Updated weights for policy 0, policy_version 263849 (0.0008) [2023-12-27 07:28:28,163][20347] Updated weights for policy 0, policy_version 263859 (0.0007) [2023-12-27 07:28:28,215][20347] Updated weights for policy 0, policy_version 263869 (0.0008) [2023-12-27 07:28:28,484][20381] Updated weights for policy 1, policy_version 263410 (0.0008) [2023-12-27 07:28:28,537][20381] Updated weights for policy 1, policy_version 263420 (0.0008) [2023-12-27 07:28:28,586][20381] Updated weights for policy 1, policy_version 263430 (0.0008) [2023-12-27 07:28:28,920][20347] Updated weights for policy 0, policy_version 263879 (0.0006) [2023-12-27 07:28:28,975][20347] Updated weights for policy 0, policy_version 263889 (0.0005) [2023-12-27 07:28:29,037][20347] Updated weights for policy 0, policy_version 263899 (0.0006) [2023-12-27 07:28:29,259][20381] Updated weights for policy 1, policy_version 263440 (0.0007) [2023-12-27 07:28:29,319][20381] Updated weights for policy 1, policy_version 263450 (0.0008) [2023-12-27 07:28:29,379][20381] Updated weights for policy 1, policy_version 263460 (0.0008) [2023-12-27 07:28:29,642][20347] Updated weights for policy 0, policy_version 263909 (0.0008) [2023-12-27 07:28:29,693][20347] Updated weights for policy 0, policy_version 263919 (0.0007) [2023-12-27 07:28:29,748][20347] Updated weights for policy 0, policy_version 263929 (0.0008) [2023-12-27 07:28:30,129][20381] Updated weights for policy 1, policy_version 263470 (0.0008) [2023-12-27 07:28:30,179][20381] Updated weights for policy 1, policy_version 263480 (0.0008) [2023-12-27 07:28:30,224][20381] Updated weights for policy 1, policy_version 263490 (0.0007) [2023-12-27 07:28:30,427][20347] Updated weights for policy 0, policy_version 263939 (0.0008) [2023-12-27 07:28:30,482][20347] Updated weights for policy 0, policy_version 263949 (0.0007) [2023-12-27 07:28:30,534][20347] Updated weights for policy 0, policy_version 263959 (0.0007) [2023-12-27 07:28:30,948][20381] Updated weights for policy 1, policy_version 263500 (0.0007) [2023-12-27 07:28:31,015][20381] Updated weights for policy 1, policy_version 263510 (0.0007) [2023-12-27 07:28:31,078][20381] Updated weights for policy 1, policy_version 263520 (0.0007) [2023-12-27 07:28:31,170][20347] Updated weights for policy 0, policy_version 263969 (0.0008) [2023-12-27 07:28:31,218][20347] Updated weights for policy 0, policy_version 263979 (0.0010) [2023-12-27 07:28:31,267][20347] Updated weights for policy 0, policy_version 263989 (0.0010) [2023-12-27 07:28:31,322][20347] Updated weights for policy 0, policy_version 263999 (0.0007) [2023-12-27 07:28:31,743][20381] Updated weights for policy 1, policy_version 263530 (0.0007) [2023-12-27 07:28:31,796][20381] Updated weights for policy 1, policy_version 263540 (0.0006) [2023-12-27 07:28:31,854][20381] Updated weights for policy 1, policy_version 263550 (0.0005) [2023-12-27 07:28:31,914][20381] Updated weights for policy 1, policy_version 263560 (0.0005) [2023-12-27 07:28:32,033][20347] Updated weights for policy 0, policy_version 264009 (0.0009) [2023-12-27 07:28:32,086][20347] Updated weights for policy 0, policy_version 264019 (0.0010) [2023-12-27 07:28:32,137][20347] Updated weights for policy 0, policy_version 264029 (0.0010) [2023-12-27 07:28:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 135094272. Throughput: 0: 10461.9, 1: 10298.7. Samples: 135103412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:28:32,566][20381] Updated weights for policy 1, policy_version 263570 (0.0010) [2023-12-27 07:28:32,622][20381] Updated weights for policy 1, policy_version 263580 (0.0010) [2023-12-27 07:28:32,676][20381] Updated weights for policy 1, policy_version 263590 (0.0008) [2023-12-27 07:28:32,800][20347] Updated weights for policy 0, policy_version 264039 (0.0007) [2023-12-27 07:28:32,855][20347] Updated weights for policy 0, policy_version 264049 (0.0007) [2023-12-27 07:28:32,910][20347] Updated weights for policy 0, policy_version 264059 (0.0007) [2023-12-27 07:28:33,322][20381] Updated weights for policy 1, policy_version 263600 (0.0006) [2023-12-27 07:28:33,383][20381] Updated weights for policy 1, policy_version 263610 (0.0008) [2023-12-27 07:28:33,431][20381] Updated weights for policy 1, policy_version 263620 (0.0007) [2023-12-27 07:28:33,546][20347] Updated weights for policy 0, policy_version 264069 (0.0007) [2023-12-27 07:28:33,590][20347] Updated weights for policy 0, policy_version 264079 (0.0006) [2023-12-27 07:28:33,638][20347] Updated weights for policy 0, policy_version 264089 (0.0005) [2023-12-27 07:28:34,037][20381] Updated weights for policy 1, policy_version 263630 (0.0007) [2023-12-27 07:28:34,102][20381] Updated weights for policy 1, policy_version 263640 (0.0008) [2023-12-27 07:28:34,160][20381] Updated weights for policy 1, policy_version 263650 (0.0011) [2023-12-27 07:28:34,341][20347] Updated weights for policy 0, policy_version 264099 (0.0006) [2023-12-27 07:28:34,391][20347] Updated weights for policy 0, policy_version 264109 (0.0007) [2023-12-27 07:28:34,444][20347] Updated weights for policy 0, policy_version 264119 (0.0007) [2023-12-27 07:28:34,824][20381] Updated weights for policy 1, policy_version 263660 (0.0009) [2023-12-27 07:28:34,887][20381] Updated weights for policy 1, policy_version 263670 (0.0006) [2023-12-27 07:28:34,948][20381] Updated weights for policy 1, policy_version 263680 (0.0008) [2023-12-27 07:28:35,140][20347] Updated weights for policy 0, policy_version 264129 (0.0006) [2023-12-27 07:28:35,193][20347] Updated weights for policy 0, policy_version 264139 (0.0007) [2023-12-27 07:28:35,247][20347] Updated weights for policy 0, policy_version 264149 (0.0005) [2023-12-27 07:28:35,304][20347] Updated weights for policy 0, policy_version 264159 (0.0006) [2023-12-27 07:28:35,563][20381] Updated weights for policy 1, policy_version 263690 (0.0006) [2023-12-27 07:28:35,605][20381] Updated weights for policy 1, policy_version 263700 (0.0009) [2023-12-27 07:28:35,656][20381] Updated weights for policy 1, policy_version 263710 (0.0005) [2023-12-27 07:28:35,717][20381] Updated weights for policy 1, policy_version 263720 (0.0005) [2023-12-27 07:28:35,978][20347] Updated weights for policy 0, policy_version 264169 (0.0007) [2023-12-27 07:28:36,039][20347] Updated weights for policy 0, policy_version 264179 (0.0005) [2023-12-27 07:28:36,098][20347] Updated weights for policy 0, policy_version 264189 (0.0007) [2023-12-27 07:28:36,423][20381] Updated weights for policy 1, policy_version 263730 (0.0011) [2023-12-27 07:28:36,486][20381] Updated weights for policy 1, policy_version 263740 (0.0011) [2023-12-27 07:28:36,535][20381] Updated weights for policy 1, policy_version 263750 (0.0010) [2023-12-27 07:28:36,757][20347] Updated weights for policy 0, policy_version 264199 (0.0008) [2023-12-27 07:28:36,821][20347] Updated weights for policy 0, policy_version 264209 (0.0007) [2023-12-27 07:28:36,880][20347] Updated weights for policy 0, policy_version 264219 (0.0008) [2023-12-27 07:28:37,156][20381] Updated weights for policy 1, policy_version 263760 (0.0010) [2023-12-27 07:28:37,204][20381] Updated weights for policy 1, policy_version 263770 (0.0007) [2023-12-27 07:28:37,248][20381] Updated weights for policy 1, policy_version 263780 (0.0005) [2023-12-27 07:28:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 135200768. Throughput: 0: 10477.2, 1: 10321.8. Samples: 135166740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:37,363][19170] Avg episode reward: [(0, '930.840'), (1, '1000.000')] [2023-12-27 07:28:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000263784_67543040.pth... [2023-12-27 07:28:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000264224_67657728.pth... [2023-12-27 07:28:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000262568_67231744.pth [2023-12-27 07:28:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000263016_67346432.pth [2023-12-27 07:28:37,602][20347] Updated weights for policy 0, policy_version 264229 (0.0008) [2023-12-27 07:28:37,660][20347] Updated weights for policy 0, policy_version 264239 (0.0006) [2023-12-27 07:28:37,718][20347] Updated weights for policy 0, policy_version 264249 (0.0007) [2023-12-27 07:28:37,895][20381] Updated weights for policy 1, policy_version 263790 (0.0007) [2023-12-27 07:28:37,945][20381] Updated weights for policy 1, policy_version 263800 (0.0009) [2023-12-27 07:28:37,995][20381] Updated weights for policy 1, policy_version 263810 (0.0007) [2023-12-27 07:28:38,318][20347] Updated weights for policy 0, policy_version 264259 (0.0007) [2023-12-27 07:28:38,375][20347] Updated weights for policy 0, policy_version 264269 (0.0007) [2023-12-27 07:28:38,430][20347] Updated weights for policy 0, policy_version 264279 (0.0007) [2023-12-27 07:28:38,677][20381] Updated weights for policy 1, policy_version 263820 (0.0007) [2023-12-27 07:28:38,740][20381] Updated weights for policy 1, policy_version 263830 (0.0010) [2023-12-27 07:28:38,792][20381] Updated weights for policy 1, policy_version 263840 (0.0010) [2023-12-27 07:28:39,139][20347] Updated weights for policy 0, policy_version 264289 (0.0006) [2023-12-27 07:28:39,196][20347] Updated weights for policy 0, policy_version 264299 (0.0008) [2023-12-27 07:28:39,261][20347] Updated weights for policy 0, policy_version 264309 (0.0008) [2023-12-27 07:28:39,320][20347] Updated weights for policy 0, policy_version 264319 (0.0008) [2023-12-27 07:28:39,492][20381] Updated weights for policy 1, policy_version 263850 (0.0006) [2023-12-27 07:28:39,551][20381] Updated weights for policy 1, policy_version 263860 (0.0008) [2023-12-27 07:28:39,605][20381] Updated weights for policy 1, policy_version 263870 (0.0008) [2023-12-27 07:28:39,665][20381] Updated weights for policy 1, policy_version 263880 (0.0008) [2023-12-27 07:28:40,069][20347] Updated weights for policy 0, policy_version 264329 (0.0006) [2023-12-27 07:28:40,125][20347] Updated weights for policy 0, policy_version 264339 (0.0007) [2023-12-27 07:28:40,181][20347] Updated weights for policy 0, policy_version 264349 (0.0007) [2023-12-27 07:28:40,186][20291] KL-divergence is very high: 127.9249 [2023-12-27 07:28:40,389][20381] Updated weights for policy 1, policy_version 263890 (0.0007) [2023-12-27 07:28:40,451][20381] Updated weights for policy 1, policy_version 263900 (0.0007) [2023-12-27 07:28:40,511][20381] Updated weights for policy 1, policy_version 263910 (0.0007) [2023-12-27 07:28:40,840][20347] Updated weights for policy 0, policy_version 264359 (0.0006) [2023-12-27 07:28:40,899][20347] Updated weights for policy 0, policy_version 264369 (0.0005) [2023-12-27 07:28:40,952][20347] Updated weights for policy 0, policy_version 264379 (0.0010) [2023-12-27 07:28:41,182][20381] Updated weights for policy 1, policy_version 263920 (0.0007) [2023-12-27 07:28:41,250][20381] Updated weights for policy 1, policy_version 263930 (0.0008) [2023-12-27 07:28:41,310][20381] Updated weights for policy 1, policy_version 263940 (0.0011) [2023-12-27 07:28:41,643][20347] Updated weights for policy 0, policy_version 264389 (0.0009) [2023-12-27 07:28:41,699][20347] Updated weights for policy 0, policy_version 264399 (0.0006) [2023-12-27 07:28:41,737][20291] KL-divergence is very high: 115.2078 [2023-12-27 07:28:41,761][20347] Updated weights for policy 0, policy_version 264409 (0.0007) [2023-12-27 07:28:42,021][20381] Updated weights for policy 1, policy_version 263950 (0.0009) [2023-12-27 07:28:42,077][20381] Updated weights for policy 1, policy_version 263960 (0.0011) [2023-12-27 07:28:42,142][20381] Updated weights for policy 1, policy_version 263970 (0.0010) [2023-12-27 07:28:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 135299072. Throughput: 0: 10421.9, 1: 10325.5. Samples: 135288900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:42,363][19170] Avg episode reward: [(0, '788.950'), (1, '1000.000')] [2023-12-27 07:28:42,409][20347] Updated weights for policy 0, policy_version 264419 (0.0008) [2023-12-27 07:28:42,464][20347] Updated weights for policy 0, policy_version 264429 (0.0007) [2023-12-27 07:28:42,520][20347] Updated weights for policy 0, policy_version 264439 (0.0007) [2023-12-27 07:28:42,794][20381] Updated weights for policy 1, policy_version 263980 (0.0010) [2023-12-27 07:28:42,852][20381] Updated weights for policy 1, policy_version 263990 (0.0011) [2023-12-27 07:28:42,912][20381] Updated weights for policy 1, policy_version 264000 (0.0007) [2023-12-27 07:28:43,223][20347] Updated weights for policy 0, policy_version 264449 (0.0005) [2023-12-27 07:28:43,270][20347] Updated weights for policy 0, policy_version 264459 (0.0007) [2023-12-27 07:28:43,317][20347] Updated weights for policy 0, policy_version 264469 (0.0007) [2023-12-27 07:28:43,362][20347] Updated weights for policy 0, policy_version 264479 (0.0009) [2023-12-27 07:28:43,474][20381] Updated weights for policy 1, policy_version 264010 (0.0010) [2023-12-27 07:28:43,538][20381] Updated weights for policy 1, policy_version 264020 (0.0011) [2023-12-27 07:28:43,596][20381] Updated weights for policy 1, policy_version 264030 (0.0010) [2023-12-27 07:28:43,659][20381] Updated weights for policy 1, policy_version 264040 (0.0011) [2023-12-27 07:28:44,065][20347] Updated weights for policy 0, policy_version 264489 (0.0007) [2023-12-27 07:28:44,122][20347] Updated weights for policy 0, policy_version 264499 (0.0007) [2023-12-27 07:28:44,170][20347] Updated weights for policy 0, policy_version 264509 (0.0007) [2023-12-27 07:28:44,252][20381] Updated weights for policy 1, policy_version 264050 (0.0008) [2023-12-27 07:28:44,304][20381] Updated weights for policy 1, policy_version 264060 (0.0008) [2023-12-27 07:28:44,359][20381] Updated weights for policy 1, policy_version 264070 (0.0007) [2023-12-27 07:28:44,835][20347] Updated weights for policy 0, policy_version 264519 (0.0007) [2023-12-27 07:28:44,899][20347] Updated weights for policy 0, policy_version 264529 (0.0008) [2023-12-27 07:28:44,955][20347] Updated weights for policy 0, policy_version 264539 (0.0007) [2023-12-27 07:28:45,091][20381] Updated weights for policy 1, policy_version 264080 (0.0010) [2023-12-27 07:28:45,144][20381] Updated weights for policy 1, policy_version 264090 (0.0011) [2023-12-27 07:28:45,200][20381] Updated weights for policy 1, policy_version 264100 (0.0011) [2023-12-27 07:28:45,635][20347] Updated weights for policy 0, policy_version 264549 (0.0006) [2023-12-27 07:28:45,694][20347] Updated weights for policy 0, policy_version 264559 (0.0005) [2023-12-27 07:28:45,745][20347] Updated weights for policy 0, policy_version 264569 (0.0006) [2023-12-27 07:28:45,872][20381] Updated weights for policy 1, policy_version 264110 (0.0008) [2023-12-27 07:28:45,930][20381] Updated weights for policy 1, policy_version 264120 (0.0007) [2023-12-27 07:28:45,983][20381] Updated weights for policy 1, policy_version 264130 (0.0008) [2023-12-27 07:28:46,399][20347] Updated weights for policy 0, policy_version 264579 (0.0007) [2023-12-27 07:28:46,459][20347] Updated weights for policy 0, policy_version 264589 (0.0008) [2023-12-27 07:28:46,521][20347] Updated weights for policy 0, policy_version 264599 (0.0008) [2023-12-27 07:28:46,678][20381] Updated weights for policy 1, policy_version 264140 (0.0008) [2023-12-27 07:28:46,727][20381] Updated weights for policy 1, policy_version 264150 (0.0008) [2023-12-27 07:28:46,777][20381] Updated weights for policy 1, policy_version 264160 (0.0010) [2023-12-27 07:28:47,073][20347] Updated weights for policy 0, policy_version 264609 (0.0008) [2023-12-27 07:28:47,129][20347] Updated weights for policy 0, policy_version 264619 (0.0008) [2023-12-27 07:28:47,178][20347] Updated weights for policy 0, policy_version 264629 (0.0009) [2023-12-27 07:28:47,229][20347] Updated weights for policy 0, policy_version 264639 (0.0007) [2023-12-27 07:28:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 135405568. Throughput: 0: 10406.2, 1: 10380.4. Samples: 135415020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:47,363][19170] Avg episode reward: [(0, '843.260'), (1, '1000.000')] [2023-12-27 07:28:47,406][20381] Updated weights for policy 1, policy_version 264170 (0.0006) [2023-12-27 07:28:47,463][20381] Updated weights for policy 1, policy_version 264180 (0.0008) [2023-12-27 07:28:47,516][20381] Updated weights for policy 1, policy_version 264190 (0.0007) [2023-12-27 07:28:47,562][20381] Updated weights for policy 1, policy_version 264200 (0.0008) [2023-12-27 07:28:47,908][20347] Updated weights for policy 0, policy_version 264649 (0.0007) [2023-12-27 07:28:47,962][20347] Updated weights for policy 0, policy_version 264659 (0.0007) [2023-12-27 07:28:48,009][20347] Updated weights for policy 0, policy_version 264669 (0.0007) [2023-12-27 07:28:48,213][20381] Updated weights for policy 1, policy_version 264210 (0.0010) [2023-12-27 07:28:48,267][20381] Updated weights for policy 1, policy_version 264220 (0.0010) [2023-12-27 07:28:48,319][20381] Updated weights for policy 1, policy_version 264230 (0.0010) [2023-12-27 07:28:48,720][20347] Updated weights for policy 0, policy_version 264679 (0.0007) [2023-12-27 07:28:48,781][20347] Updated weights for policy 0, policy_version 264689 (0.0008) [2023-12-27 07:28:48,841][20347] Updated weights for policy 0, policy_version 264699 (0.0006) [2023-12-27 07:28:48,937][20381] Updated weights for policy 1, policy_version 264240 (0.0011) [2023-12-27 07:28:48,989][20381] Updated weights for policy 1, policy_version 264250 (0.0010) [2023-12-27 07:28:49,037][20381] Updated weights for policy 1, policy_version 264260 (0.0010) [2023-12-27 07:28:49,496][20347] Updated weights for policy 0, policy_version 264709 (0.0007) [2023-12-27 07:28:49,554][20347] Updated weights for policy 0, policy_version 264719 (0.0006) [2023-12-27 07:28:49,574][20291] KL-divergence is very high: 243.8840 [2023-12-27 07:28:49,581][20291] KL-divergence is very high: 185.0232 [2023-12-27 07:28:49,615][20347] Updated weights for policy 0, policy_version 264729 (0.0005) [2023-12-27 07:28:49,623][20291] KL-divergence is very high: 126.9173 [2023-12-27 07:28:49,755][20381] Updated weights for policy 1, policy_version 264270 (0.0010) [2023-12-27 07:28:49,810][20381] Updated weights for policy 1, policy_version 264280 (0.0010) [2023-12-27 07:28:49,874][20381] Updated weights for policy 1, policy_version 264290 (0.0009) [2023-12-27 07:28:50,236][20347] Updated weights for policy 0, policy_version 264739 (0.0006) [2023-12-27 07:28:50,296][20347] Updated weights for policy 0, policy_version 264749 (0.0007) [2023-12-27 07:28:50,353][20347] Updated weights for policy 0, policy_version 264759 (0.0007) [2023-12-27 07:28:50,578][20381] Updated weights for policy 1, policy_version 264300 (0.0010) [2023-12-27 07:28:50,637][20381] Updated weights for policy 1, policy_version 264310 (0.0011) [2023-12-27 07:28:50,695][20381] Updated weights for policy 1, policy_version 264320 (0.0011) [2023-12-27 07:28:51,052][20347] Updated weights for policy 0, policy_version 264769 (0.0007) [2023-12-27 07:28:51,112][20347] Updated weights for policy 0, policy_version 264779 (0.0007) [2023-12-27 07:28:51,169][20347] Updated weights for policy 0, policy_version 264789 (0.0008) [2023-12-27 07:28:51,218][20347] Updated weights for policy 0, policy_version 264799 (0.0007) [2023-12-27 07:28:51,313][20381] Updated weights for policy 1, policy_version 264330 (0.0005) [2023-12-27 07:28:51,377][20381] Updated weights for policy 1, policy_version 264340 (0.0007) [2023-12-27 07:28:51,436][20381] Updated weights for policy 1, policy_version 264350 (0.0006) [2023-12-27 07:28:51,492][20381] Updated weights for policy 1, policy_version 264360 (0.0005) [2023-12-27 07:28:51,954][20347] Updated weights for policy 0, policy_version 264809 (0.0010) [2023-12-27 07:28:52,013][20347] Updated weights for policy 0, policy_version 264819 (0.0010) [2023-12-27 07:28:52,066][20347] Updated weights for policy 0, policy_version 264829 (0.0010) [2023-12-27 07:28:52,077][20381] Updated weights for policy 1, policy_version 264370 (0.0006) [2023-12-27 07:28:52,137][20381] Updated weights for policy 1, policy_version 264380 (0.0010) [2023-12-27 07:28:52,196][20381] Updated weights for policy 1, policy_version 264390 (0.0011) [2023-12-27 07:28:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 135512064. Throughput: 0: 10390.9, 1: 10425.0. Samples: 135477504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:52,363][19170] Avg episode reward: [(0, '961.830'), (1, '1000.000')] [2023-12-27 07:28:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000264392_67698688.pth... [2023-12-27 07:28:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000264832_67813376.pth... [2023-12-27 07:28:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000263176_67387392.pth [2023-12-27 07:28:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000263624_67502080.pth [2023-12-27 07:28:52,761][20347] Updated weights for policy 0, policy_version 264839 (0.0010) [2023-12-27 07:28:52,823][20347] Updated weights for policy 0, policy_version 264849 (0.0010) [2023-12-27 07:28:52,823][20381] Updated weights for policy 1, policy_version 264400 (0.0011) [2023-12-27 07:28:52,876][20347] Updated weights for policy 0, policy_version 264859 (0.0010) [2023-12-27 07:28:52,880][20381] Updated weights for policy 1, policy_version 264410 (0.0011) [2023-12-27 07:28:52,932][20381] Updated weights for policy 1, policy_version 264420 (0.0010) [2023-12-27 07:28:53,463][20347] Updated weights for policy 0, policy_version 264869 (0.0008) [2023-12-27 07:28:53,513][20381] Updated weights for policy 1, policy_version 264430 (0.0009) [2023-12-27 07:28:53,525][20347] Updated weights for policy 0, policy_version 264879 (0.0006) [2023-12-27 07:28:53,571][20381] Updated weights for policy 1, policy_version 264440 (0.0011) [2023-12-27 07:28:53,583][20347] Updated weights for policy 0, policy_version 264889 (0.0006) [2023-12-27 07:28:53,627][20381] Updated weights for policy 1, policy_version 264450 (0.0011) [2023-12-27 07:28:54,137][20347] Updated weights for policy 0, policy_version 264899 (0.0007) [2023-12-27 07:28:54,193][20347] Updated weights for policy 0, policy_version 264909 (0.0006) [2023-12-27 07:28:54,248][20347] Updated weights for policy 0, policy_version 264919 (0.0006) [2023-12-27 07:28:54,309][20381] Updated weights for policy 1, policy_version 264460 (0.0010) [2023-12-27 07:28:54,366][20381] Updated weights for policy 1, policy_version 264470 (0.0007) [2023-12-27 07:28:54,426][20381] Updated weights for policy 1, policy_version 264480 (0.0009) [2023-12-27 07:28:54,962][20347] Updated weights for policy 0, policy_version 264929 (0.0006) [2023-12-27 07:28:55,010][20381] Updated weights for policy 1, policy_version 264490 (0.0010) [2023-12-27 07:28:55,014][20347] Updated weights for policy 0, policy_version 264939 (0.0008) [2023-12-27 07:28:55,060][20347] Updated weights for policy 0, policy_version 264949 (0.0006) [2023-12-27 07:28:55,066][20381] Updated weights for policy 1, policy_version 264500 (0.0011) [2023-12-27 07:28:55,111][20347] Updated weights for policy 0, policy_version 264959 (0.0005) [2023-12-27 07:28:55,117][20381] Updated weights for policy 1, policy_version 264510 (0.0011) [2023-12-27 07:28:55,176][20381] Updated weights for policy 1, policy_version 264520 (0.0010) [2023-12-27 07:28:55,828][20347] Updated weights for policy 0, policy_version 264969 (0.0007) [2023-12-27 07:28:55,865][20381] Updated weights for policy 1, policy_version 264530 (0.0011) [2023-12-27 07:28:55,883][20347] Updated weights for policy 0, policy_version 264979 (0.0008) [2023-12-27 07:28:55,914][20381] Updated weights for policy 1, policy_version 264540 (0.0010) [2023-12-27 07:28:55,942][20347] Updated weights for policy 0, policy_version 264989 (0.0006) [2023-12-27 07:28:55,963][20381] Updated weights for policy 1, policy_version 264550 (0.0011) [2023-12-27 07:28:56,609][20347] Updated weights for policy 0, policy_version 264999 (0.0007) [2023-12-27 07:28:56,654][20381] Updated weights for policy 1, policy_version 264560 (0.0010) [2023-12-27 07:28:56,659][20347] Updated weights for policy 0, policy_version 265009 (0.0005) [2023-12-27 07:28:56,705][20381] Updated weights for policy 1, policy_version 264570 (0.0010) [2023-12-27 07:28:56,710][20347] Updated weights for policy 0, policy_version 265019 (0.0005) [2023-12-27 07:28:56,749][20381] Updated weights for policy 1, policy_version 264580 (0.0010) [2023-12-27 07:28:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 135610368. Throughput: 0: 10419.9, 1: 10460.8. Samples: 135603684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:28:57,363][19170] Avg episode reward: [(0, '990.450'), (1, '1000.000')] [2023-12-27 07:28:57,399][20347] Updated weights for policy 0, policy_version 265029 (0.0007) [2023-12-27 07:28:57,455][20381] Updated weights for policy 1, policy_version 264590 (0.0010) [2023-12-27 07:28:57,458][20347] Updated weights for policy 0, policy_version 265039 (0.0007) [2023-12-27 07:28:57,517][20347] Updated weights for policy 0, policy_version 265049 (0.0007) [2023-12-27 07:28:57,517][20381] Updated weights for policy 1, policy_version 264600 (0.0011) [2023-12-27 07:28:57,580][20381] Updated weights for policy 1, policy_version 264610 (0.0011) [2023-12-27 07:28:58,214][20347] Updated weights for policy 0, policy_version 265059 (0.0007) [2023-12-27 07:28:58,268][20381] Updated weights for policy 1, policy_version 264620 (0.0011) [2023-12-27 07:28:58,276][20347] Updated weights for policy 0, policy_version 265069 (0.0007) [2023-12-27 07:28:58,331][20381] Updated weights for policy 1, policy_version 264630 (0.0010) [2023-12-27 07:28:58,334][20347] Updated weights for policy 0, policy_version 265079 (0.0007) [2023-12-27 07:28:58,399][20381] Updated weights for policy 1, policy_version 264640 (0.0008) [2023-12-27 07:28:59,063][20347] Updated weights for policy 0, policy_version 265089 (0.0008) [2023-12-27 07:28:59,067][20381] Updated weights for policy 1, policy_version 264650 (0.0010) [2023-12-27 07:28:59,118][20347] Updated weights for policy 0, policy_version 265099 (0.0007) [2023-12-27 07:28:59,123][20381] Updated weights for policy 1, policy_version 264660 (0.0007) [2023-12-27 07:28:59,176][20347] Updated weights for policy 0, policy_version 265109 (0.0009) [2023-12-27 07:28:59,183][20381] Updated weights for policy 1, policy_version 264670 (0.0007) [2023-12-27 07:28:59,240][20347] Updated weights for policy 0, policy_version 265119 (0.0009) [2023-12-27 07:28:59,247][20381] Updated weights for policy 1, policy_version 264680 (0.0008) [2023-12-27 07:28:59,947][20381] Updated weights for policy 1, policy_version 264690 (0.0008) [2023-12-27 07:28:59,970][20347] Updated weights for policy 0, policy_version 265129 (0.0007) [2023-12-27 07:29:00,006][20381] Updated weights for policy 1, policy_version 264700 (0.0006) [2023-12-27 07:29:00,025][20347] Updated weights for policy 0, policy_version 265139 (0.0008) [2023-12-27 07:29:00,063][20381] Updated weights for policy 1, policy_version 264710 (0.0005) [2023-12-27 07:29:00,070][20347] Updated weights for policy 0, policy_version 265149 (0.0010) [2023-12-27 07:29:00,631][20381] Updated weights for policy 1, policy_version 264720 (0.0005) [2023-12-27 07:29:00,682][20381] Updated weights for policy 1, policy_version 264730 (0.0005) [2023-12-27 07:29:00,720][20347] Updated weights for policy 0, policy_version 265159 (0.0010) [2023-12-27 07:29:00,733][20381] Updated weights for policy 1, policy_version 264740 (0.0005) [2023-12-27 07:29:00,765][20347] Updated weights for policy 0, policy_version 265169 (0.0010) [2023-12-27 07:29:00,813][20347] Updated weights for policy 0, policy_version 265179 (0.0010) [2023-12-27 07:29:01,280][20381] Updated weights for policy 1, policy_version 264750 (0.0005) [2023-12-27 07:29:01,335][20381] Updated weights for policy 1, policy_version 264760 (0.0007) [2023-12-27 07:29:01,402][20381] Updated weights for policy 1, policy_version 264770 (0.0008) [2023-12-27 07:29:01,530][20347] Updated weights for policy 0, policy_version 265189 (0.0008) [2023-12-27 07:29:01,593][20347] Updated weights for policy 0, policy_version 265199 (0.0005) [2023-12-27 07:29:01,656][20347] Updated weights for policy 0, policy_version 265209 (0.0008) [2023-12-27 07:29:02,083][20381] Updated weights for policy 1, policy_version 264780 (0.0007) [2023-12-27 07:29:02,139][20381] Updated weights for policy 1, policy_version 264790 (0.0008) [2023-12-27 07:29:02,196][20381] Updated weights for policy 1, policy_version 264800 (0.0010) [2023-12-27 07:29:02,262][20347] Updated weights for policy 0, policy_version 265219 (0.0007) [2023-12-27 07:29:02,325][20347] Updated weights for policy 0, policy_version 265229 (0.0007) [2023-12-27 07:29:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 135716864. Throughput: 0: 10344.3, 1: 10440.1. Samples: 135726504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:29:02,363][19170] Avg episode reward: [(0, '995.560'), (1, '1000.000')] [2023-12-27 07:29:02,397][20347] Updated weights for policy 0, policy_version 265239 (0.0008) [2023-12-27 07:29:02,963][20381] Updated weights for policy 1, policy_version 264810 (0.0009) [2023-12-27 07:29:03,008][20381] Updated weights for policy 1, policy_version 264820 (0.0007) [2023-12-27 07:29:03,028][20347] Updated weights for policy 0, policy_version 265249 (0.0008) [2023-12-27 07:29:03,055][20381] Updated weights for policy 1, policy_version 264830 (0.0007) [2023-12-27 07:29:03,079][20347] Updated weights for policy 0, policy_version 265259 (0.0006) [2023-12-27 07:29:03,103][20381] Updated weights for policy 1, policy_version 264840 (0.0005) [2023-12-27 07:29:03,128][20347] Updated weights for policy 0, policy_version 265269 (0.0006) [2023-12-27 07:29:03,180][20347] Updated weights for policy 0, policy_version 265279 (0.0008) [2023-12-27 07:29:03,780][20381] Updated weights for policy 1, policy_version 264850 (0.0008) [2023-12-27 07:29:03,828][20381] Updated weights for policy 1, policy_version 264860 (0.0007) [2023-12-27 07:29:03,880][20347] Updated weights for policy 0, policy_version 265289 (0.0007) [2023-12-27 07:29:03,886][20381] Updated weights for policy 1, policy_version 264870 (0.0008) [2023-12-27 07:29:03,941][20347] Updated weights for policy 0, policy_version 265299 (0.0007) [2023-12-27 07:29:03,993][20347] Updated weights for policy 0, policy_version 265309 (0.0007) [2023-12-27 07:29:04,600][20381] Updated weights for policy 1, policy_version 264880 (0.0008) [2023-12-27 07:29:04,656][20381] Updated weights for policy 1, policy_version 264890 (0.0008) [2023-12-27 07:29:04,689][20347] Updated weights for policy 0, policy_version 265319 (0.0006) [2023-12-27 07:29:04,702][20381] Updated weights for policy 1, policy_version 264900 (0.0005) [2023-12-27 07:29:04,703][20291] KL-divergence is very high: 182.8114 [2023-12-27 07:29:04,738][20347] Updated weights for policy 0, policy_version 265329 (0.0006) [2023-12-27 07:29:04,777][20291] KL-divergence is very high: 102.6507 [2023-12-27 07:29:04,783][20347] Updated weights for policy 0, policy_version 265339 (0.0008) [2023-12-27 07:29:05,406][20381] Updated weights for policy 1, policy_version 264910 (0.0007) [2023-12-27 07:29:05,461][20381] Updated weights for policy 1, policy_version 264920 (0.0007) [2023-12-27 07:29:05,490][20347] Updated weights for policy 0, policy_version 265349 (0.0007) [2023-12-27 07:29:05,518][20381] Updated weights for policy 1, policy_version 264930 (0.0007) [2023-12-27 07:29:05,544][20347] Updated weights for policy 0, policy_version 265359 (0.0005) [2023-12-27 07:29:05,592][20347] Updated weights for policy 0, policy_version 265369 (0.0007) [2023-12-27 07:29:06,136][20381] Updated weights for policy 1, policy_version 264940 (0.0007) [2023-12-27 07:29:06,192][20381] Updated weights for policy 1, policy_version 264950 (0.0007) [2023-12-27 07:29:06,244][20381] Updated weights for policy 1, policy_version 264960 (0.0008) [2023-12-27 07:29:06,277][20347] Updated weights for policy 0, policy_version 265379 (0.0009) [2023-12-27 07:29:06,335][20347] Updated weights for policy 0, policy_version 265389 (0.0011) [2023-12-27 07:29:06,396][20347] Updated weights for policy 0, policy_version 265399 (0.0011) [2023-12-27 07:29:06,916][20381] Updated weights for policy 1, policy_version 264970 (0.0006) [2023-12-27 07:29:06,968][20381] Updated weights for policy 1, policy_version 264980 (0.0007) [2023-12-27 07:29:07,020][20381] Updated weights for policy 1, policy_version 264990 (0.0008) [2023-12-27 07:29:07,041][20347] Updated weights for policy 0, policy_version 265409 (0.0007) [2023-12-27 07:29:07,076][20381] Updated weights for policy 1, policy_version 265000 (0.0007) [2023-12-27 07:29:07,092][20347] Updated weights for policy 0, policy_version 265419 (0.0010) [2023-12-27 07:29:07,148][20347] Updated weights for policy 0, policy_version 265429 (0.0010) [2023-12-27 07:29:07,197][20347] Updated weights for policy 0, policy_version 265439 (0.0011) [2023-12-27 07:29:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 135823360. Throughput: 0: 10358.3, 1: 10449.7. Samples: 135787940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:29:07,363][19170] Avg episode reward: [(0, '977.230'), (1, '1000.000')] [2023-12-27 07:29:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000265000_67854336.pth... [2023-12-27 07:29:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000265440_67969024.pth... [2023-12-27 07:29:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000264224_67657728.pth [2023-12-27 07:29:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000263784_67543040.pth [2023-12-27 07:29:07,710][20381] Updated weights for policy 1, policy_version 265010 (0.0007) [2023-12-27 07:29:07,762][20381] Updated weights for policy 1, policy_version 265020 (0.0008) [2023-12-27 07:29:07,813][20381] Updated weights for policy 1, policy_version 265030 (0.0007) [2023-12-27 07:29:07,894][20347] Updated weights for policy 0, policy_version 265449 (0.0006) [2023-12-27 07:29:07,948][20347] Updated weights for policy 0, policy_version 265459 (0.0005) [2023-12-27 07:29:08,001][20347] Updated weights for policy 0, policy_version 265469 (0.0005) [2023-12-27 07:29:08,585][20381] Updated weights for policy 1, policy_version 265040 (0.0007) [2023-12-27 07:29:08,591][20347] Updated weights for policy 0, policy_version 265479 (0.0009) [2023-12-27 07:29:08,645][20347] Updated weights for policy 0, policy_version 265489 (0.0010) [2023-12-27 07:29:08,647][20381] Updated weights for policy 1, policy_version 265050 (0.0006) [2023-12-27 07:29:08,700][20347] Updated weights for policy 0, policy_version 265499 (0.0010) [2023-12-27 07:29:08,705][20381] Updated weights for policy 1, policy_version 265060 (0.0005) [2023-12-27 07:29:09,324][20347] Updated weights for policy 0, policy_version 265509 (0.0009) [2023-12-27 07:29:09,386][20347] Updated weights for policy 0, policy_version 265519 (0.0008) [2023-12-27 07:29:09,424][20381] Updated weights for policy 1, policy_version 265070 (0.0007) [2023-12-27 07:29:09,451][20347] Updated weights for policy 0, policy_version 265529 (0.0007) [2023-12-27 07:29:09,480][20381] Updated weights for policy 1, policy_version 265080 (0.0007) [2023-12-27 07:29:09,529][20381] Updated weights for policy 1, policy_version 265090 (0.0007) [2023-12-27 07:29:10,169][20347] Updated weights for policy 0, policy_version 265539 (0.0006) [2023-12-27 07:29:10,235][20347] Updated weights for policy 0, policy_version 265549 (0.0007) [2023-12-27 07:29:10,256][20381] Updated weights for policy 1, policy_version 265100 (0.0007) [2023-12-27 07:29:10,295][20347] Updated weights for policy 0, policy_version 265559 (0.0006) [2023-12-27 07:29:10,309][20381] Updated weights for policy 1, policy_version 265110 (0.0007) [2023-12-27 07:29:10,364][20381] Updated weights for policy 1, policy_version 265120 (0.0007) [2023-12-27 07:29:10,996][20347] Updated weights for policy 0, policy_version 265569 (0.0008) [2023-12-27 07:29:11,065][20347] Updated weights for policy 0, policy_version 265579 (0.0008) [2023-12-27 07:29:11,101][20381] Updated weights for policy 1, policy_version 265130 (0.0008) [2023-12-27 07:29:11,125][20347] Updated weights for policy 0, policy_version 265589 (0.0006) [2023-12-27 07:29:11,172][20381] Updated weights for policy 1, policy_version 265140 (0.0007) [2023-12-27 07:29:11,191][20347] Updated weights for policy 0, policy_version 265599 (0.0008) [2023-12-27 07:29:11,233][20381] Updated weights for policy 1, policy_version 265150 (0.0007) [2023-12-27 07:29:11,299][20381] Updated weights for policy 1, policy_version 265160 (0.0008) [2023-12-27 07:29:11,868][20347] Updated weights for policy 0, policy_version 265609 (0.0006) [2023-12-27 07:29:11,927][20347] Updated weights for policy 0, policy_version 265619 (0.0007) [2023-12-27 07:29:11,977][20347] Updated weights for policy 0, policy_version 265629 (0.0006) [2023-12-27 07:29:11,986][20381] Updated weights for policy 1, policy_version 265170 (0.0006) [2023-12-27 07:29:12,041][20381] Updated weights for policy 1, policy_version 265180 (0.0007) [2023-12-27 07:29:12,100][20381] Updated weights for policy 1, policy_version 265190 (0.0008) [2023-12-27 07:29:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 135921664. Throughput: 0: 10339.0, 1: 10443.4. Samples: 135910588. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:12,363][19170] Avg episode reward: [(0, '969.500'), (1, '1000.000')] [2023-12-27 07:29:12,682][20347] Updated weights for policy 0, policy_version 265639 (0.0007) [2023-12-27 07:29:12,739][20347] Updated weights for policy 0, policy_version 265649 (0.0006) [2023-12-27 07:29:12,803][20347] Updated weights for policy 0, policy_version 265659 (0.0006) [2023-12-27 07:29:12,820][20381] Updated weights for policy 1, policy_version 265200 (0.0008) [2023-12-27 07:29:12,880][20381] Updated weights for policy 1, policy_version 265210 (0.0008) [2023-12-27 07:29:12,936][20381] Updated weights for policy 1, policy_version 265220 (0.0008) [2023-12-27 07:29:13,486][20347] Updated weights for policy 0, policy_version 265669 (0.0007) [2023-12-27 07:29:13,535][20347] Updated weights for policy 0, policy_version 265679 (0.0007) [2023-12-27 07:29:13,584][20347] Updated weights for policy 0, policy_version 265689 (0.0007) [2023-12-27 07:29:13,616][20381] Updated weights for policy 1, policy_version 265230 (0.0006) [2023-12-27 07:29:13,682][20381] Updated weights for policy 1, policy_version 265240 (0.0008) [2023-12-27 07:29:13,741][20381] Updated weights for policy 1, policy_version 265250 (0.0007) [2023-12-27 07:29:14,300][20347] Updated weights for policy 0, policy_version 265699 (0.0006) [2023-12-27 07:29:14,345][20347] Updated weights for policy 0, policy_version 265709 (0.0007) [2023-12-27 07:29:14,402][20347] Updated weights for policy 0, policy_version 265719 (0.0007) [2023-12-27 07:29:14,428][20381] Updated weights for policy 1, policy_version 265260 (0.0009) [2023-12-27 07:29:14,481][20381] Updated weights for policy 1, policy_version 265270 (0.0008) [2023-12-27 07:29:14,539][20381] Updated weights for policy 1, policy_version 265280 (0.0010) [2023-12-27 07:29:15,126][20347] Updated weights for policy 0, policy_version 265729 (0.0008) [2023-12-27 07:29:15,179][20381] Updated weights for policy 1, policy_version 265290 (0.0009) [2023-12-27 07:29:15,183][20347] Updated weights for policy 0, policy_version 265739 (0.0008) [2023-12-27 07:29:15,232][20381] Updated weights for policy 1, policy_version 265300 (0.0005) [2023-12-27 07:29:15,237][20347] Updated weights for policy 0, policy_version 265749 (0.0007) [2023-12-27 07:29:15,281][20381] Updated weights for policy 1, policy_version 265310 (0.0006) [2023-12-27 07:29:15,292][20347] Updated weights for policy 0, policy_version 265759 (0.0007) [2023-12-27 07:29:15,328][20381] Updated weights for policy 1, policy_version 265320 (0.0008) [2023-12-27 07:29:15,978][20381] Updated weights for policy 1, policy_version 265330 (0.0007) [2023-12-27 07:29:16,016][20347] Updated weights for policy 0, policy_version 265769 (0.0008) [2023-12-27 07:29:16,036][20381] Updated weights for policy 1, policy_version 265340 (0.0006) [2023-12-27 07:29:16,065][20347] Updated weights for policy 0, policy_version 265779 (0.0007) [2023-12-27 07:29:16,091][20381] Updated weights for policy 1, policy_version 265350 (0.0006) [2023-12-27 07:29:16,111][20347] Updated weights for policy 0, policy_version 265789 (0.0006) [2023-12-27 07:29:16,717][20381] Updated weights for policy 1, policy_version 265360 (0.0009) [2023-12-27 07:29:16,766][20381] Updated weights for policy 1, policy_version 265370 (0.0010) [2023-12-27 07:29:16,818][20381] Updated weights for policy 1, policy_version 265380 (0.0010) [2023-12-27 07:29:16,851][20347] Updated weights for policy 0, policy_version 265799 (0.0008) [2023-12-27 07:29:16,900][20347] Updated weights for policy 0, policy_version 265809 (0.0008) [2023-12-27 07:29:16,953][20347] Updated weights for policy 0, policy_version 265819 (0.0007) [2023-12-27 07:29:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 136019968. Throughput: 0: 10240.1, 1: 10378.7. Samples: 136031264. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:17,364][19170] Avg episode reward: [(0, '984.190'), (1, '1000.000')] [2023-12-27 07:29:17,477][20381] Updated weights for policy 1, policy_version 265390 (0.0007) [2023-12-27 07:29:17,521][20381] Updated weights for policy 1, policy_version 265400 (0.0005) [2023-12-27 07:29:17,569][20381] Updated weights for policy 1, policy_version 265410 (0.0010) [2023-12-27 07:29:17,662][20347] Updated weights for policy 0, policy_version 265829 (0.0008) [2023-12-27 07:29:17,713][20347] Updated weights for policy 0, policy_version 265839 (0.0008) [2023-12-27 07:29:17,759][20347] Updated weights for policy 0, policy_version 265849 (0.0007) [2023-12-27 07:29:18,169][20381] Updated weights for policy 1, policy_version 265420 (0.0008) [2023-12-27 07:29:18,218][20381] Updated weights for policy 1, policy_version 265430 (0.0007) [2023-12-27 07:29:18,274][20381] Updated weights for policy 1, policy_version 265440 (0.0007) [2023-12-27 07:29:18,487][20347] Updated weights for policy 0, policy_version 265859 (0.0008) [2023-12-27 07:29:18,543][20347] Updated weights for policy 0, policy_version 265869 (0.0007) [2023-12-27 07:29:18,601][20347] Updated weights for policy 0, policy_version 265879 (0.0007) [2023-12-27 07:29:18,951][20381] Updated weights for policy 1, policy_version 265450 (0.0011) [2023-12-27 07:29:19,011][20381] Updated weights for policy 1, policy_version 265460 (0.0010) [2023-12-27 07:29:19,065][20381] Updated weights for policy 1, policy_version 265470 (0.0010) [2023-12-27 07:29:19,119][20381] Updated weights for policy 1, policy_version 265480 (0.0010) [2023-12-27 07:29:19,281][20347] Updated weights for policy 0, policy_version 265889 (0.0006) [2023-12-27 07:29:19,343][20347] Updated weights for policy 0, policy_version 265899 (0.0007) [2023-12-27 07:29:19,402][20347] Updated weights for policy 0, policy_version 265909 (0.0007) [2023-12-27 07:29:19,464][20347] Updated weights for policy 0, policy_version 265919 (0.0007) [2023-12-27 07:29:19,767][20381] Updated weights for policy 1, policy_version 265490 (0.0006) [2023-12-27 07:29:19,832][20381] Updated weights for policy 1, policy_version 265500 (0.0006) [2023-12-27 07:29:19,900][20381] Updated weights for policy 1, policy_version 265511 (0.0007) [2023-12-27 07:29:20,192][20347] Updated weights for policy 0, policy_version 265929 (0.0008) [2023-12-27 07:29:20,244][20347] Updated weights for policy 0, policy_version 265939 (0.0007) [2023-12-27 07:29:20,297][20347] Updated weights for policy 0, policy_version 265949 (0.0008) [2023-12-27 07:29:20,565][20381] Updated weights for policy 1, policy_version 265522 (0.0010) [2023-12-27 07:29:20,623][20381] Updated weights for policy 1, policy_version 265532 (0.0009) [2023-12-27 07:29:20,691][20381] Updated weights for policy 1, policy_version 265542 (0.0009) [2023-12-27 07:29:20,963][20347] Updated weights for policy 0, policy_version 265959 (0.0007) [2023-12-27 07:29:21,027][20347] Updated weights for policy 0, policy_version 265969 (0.0008) [2023-12-27 07:29:21,091][20347] Updated weights for policy 0, policy_version 265979 (0.0007) [2023-12-27 07:29:21,380][20381] Updated weights for policy 1, policy_version 265552 (0.0009) [2023-12-27 07:29:21,444][20381] Updated weights for policy 1, policy_version 265562 (0.0007) [2023-12-27 07:29:21,498][20381] Updated weights for policy 1, policy_version 265572 (0.0010) [2023-12-27 07:29:21,811][20347] Updated weights for policy 0, policy_version 265989 (0.0007) [2023-12-27 07:29:21,879][20347] Updated weights for policy 0, policy_version 265999 (0.0006) [2023-12-27 07:29:21,946][20347] Updated weights for policy 0, policy_version 266009 (0.0007) [2023-12-27 07:29:22,138][20381] Updated weights for policy 1, policy_version 265582 (0.0007) [2023-12-27 07:29:22,196][20381] Updated weights for policy 1, policy_version 265592 (0.0005) [2023-12-27 07:29:22,252][20381] Updated weights for policy 1, policy_version 265602 (0.0006) [2023-12-27 07:29:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 136126464. Throughput: 0: 10200.8, 1: 10378.1. Samples: 136092792. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:22,363][19170] Avg episode reward: [(0, '995.380'), (1, '1000.000')] [2023-12-27 07:29:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000265608_68009984.pth... [2023-12-27 07:29:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000266016_68116480.pth... [2023-12-27 07:29:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000264392_67698688.pth [2023-12-27 07:29:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000264832_67813376.pth [2023-12-27 07:29:22,632][20347] Updated weights for policy 0, policy_version 266019 (0.0008) [2023-12-27 07:29:22,681][20347] Updated weights for policy 0, policy_version 266029 (0.0007) [2023-12-27 07:29:22,734][20347] Updated weights for policy 0, policy_version 266039 (0.0008) [2023-12-27 07:29:22,886][20381] Updated weights for policy 1, policy_version 265612 (0.0006) [2023-12-27 07:29:22,946][20381] Updated weights for policy 1, policy_version 265622 (0.0008) [2023-12-27 07:29:23,005][20381] Updated weights for policy 1, policy_version 265632 (0.0007) [2023-12-27 07:29:23,476][20347] Updated weights for policy 0, policy_version 266049 (0.0008) [2023-12-27 07:29:23,526][20347] Updated weights for policy 0, policy_version 266059 (0.0009) [2023-12-27 07:29:23,579][20347] Updated weights for policy 0, policy_version 266069 (0.0008) [2023-12-27 07:29:23,589][20381] Updated weights for policy 1, policy_version 265642 (0.0007) [2023-12-27 07:29:23,634][20347] Updated weights for policy 0, policy_version 266079 (0.0007) [2023-12-27 07:29:23,639][20381] Updated weights for policy 1, policy_version 265652 (0.0009) [2023-12-27 07:29:23,691][20381] Updated weights for policy 1, policy_version 265662 (0.0010) [2023-12-27 07:29:23,751][20381] Updated weights for policy 1, policy_version 265672 (0.0010) [2023-12-27 07:29:24,310][20347] Updated weights for policy 0, policy_version 266089 (0.0007) [2023-12-27 07:29:24,361][20381] Updated weights for policy 1, policy_version 265682 (0.0006) [2023-12-27 07:29:24,364][20347] Updated weights for policy 0, policy_version 266099 (0.0005) [2023-12-27 07:29:24,413][20381] Updated weights for policy 1, policy_version 265692 (0.0006) [2023-12-27 07:29:24,422][20347] Updated weights for policy 0, policy_version 266109 (0.0006) [2023-12-27 07:29:24,478][20381] Updated weights for policy 1, policy_version 265702 (0.0008) [2023-12-27 07:29:25,051][20347] Updated weights for policy 0, policy_version 266119 (0.0009) [2023-12-27 07:29:25,109][20347] Updated weights for policy 0, policy_version 266129 (0.0010) [2023-12-27 07:29:25,163][20381] Updated weights for policy 1, policy_version 265712 (0.0006) [2023-12-27 07:29:25,167][20347] Updated weights for policy 0, policy_version 266139 (0.0010) [2023-12-27 07:29:25,218][20381] Updated weights for policy 1, policy_version 265722 (0.0007) [2023-12-27 07:29:25,269][20381] Updated weights for policy 1, policy_version 265732 (0.0007) [2023-12-27 07:29:25,869][20381] Updated weights for policy 1, policy_version 265742 (0.0007) [2023-12-27 07:29:25,881][20347] Updated weights for policy 0, policy_version 266149 (0.0010) [2023-12-27 07:29:25,924][20381] Updated weights for policy 1, policy_version 265752 (0.0006) [2023-12-27 07:29:25,933][20347] Updated weights for policy 0, policy_version 266159 (0.0007) [2023-12-27 07:29:25,978][20381] Updated weights for policy 1, policy_version 265762 (0.0006) [2023-12-27 07:29:25,989][20347] Updated weights for policy 0, policy_version 266169 (0.0008) [2023-12-27 07:29:26,643][20381] Updated weights for policy 1, policy_version 265772 (0.0007) [2023-12-27 07:29:26,651][20347] Updated weights for policy 0, policy_version 266179 (0.0006) [2023-12-27 07:29:26,695][20381] Updated weights for policy 1, policy_version 265782 (0.0007) [2023-12-27 07:29:26,700][20347] Updated weights for policy 0, policy_version 266189 (0.0005) [2023-12-27 07:29:26,752][20381] Updated weights for policy 1, policy_version 265792 (0.0005) [2023-12-27 07:29:26,757][20347] Updated weights for policy 0, policy_version 266199 (0.0006) [2023-12-27 07:29:27,338][20347] Updated weights for policy 0, policy_version 266209 (0.0006) [2023-12-27 07:29:27,355][20381] Updated weights for policy 1, policy_version 265802 (0.0006) [2023-12-27 07:29:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 136224768. Throughput: 0: 10184.5, 1: 10443.0. Samples: 136217136. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:27,363][19170] Avg episode reward: [(0, '999.140'), (1, '1000.000')] [2023-12-27 07:29:27,392][20347] Updated weights for policy 0, policy_version 266219 (0.0007) [2023-12-27 07:29:27,413][20381] Updated weights for policy 1, policy_version 265812 (0.0005) [2023-12-27 07:29:27,445][20347] Updated weights for policy 0, policy_version 266229 (0.0008) [2023-12-27 07:29:27,471][20381] Updated weights for policy 1, policy_version 265822 (0.0005) [2023-12-27 07:29:27,497][20347] Updated weights for policy 0, policy_version 266239 (0.0007) [2023-12-27 07:29:27,533][20381] Updated weights for policy 1, policy_version 265832 (0.0006) [2023-12-27 07:29:28,129][20381] Updated weights for policy 1, policy_version 265842 (0.0007) [2023-12-27 07:29:28,185][20381] Updated weights for policy 1, policy_version 265852 (0.0007) [2023-12-27 07:29:28,209][20347] Updated weights for policy 0, policy_version 266249 (0.0006) [2023-12-27 07:29:28,233][20381] Updated weights for policy 1, policy_version 265862 (0.0008) [2023-12-27 07:29:28,262][20347] Updated weights for policy 0, policy_version 266259 (0.0005) [2023-12-27 07:29:28,313][20347] Updated weights for policy 0, policy_version 266269 (0.0005) [2023-12-27 07:29:28,888][20381] Updated weights for policy 1, policy_version 265872 (0.0008) [2023-12-27 07:29:28,942][20381] Updated weights for policy 1, policy_version 265882 (0.0008) [2023-12-27 07:29:28,957][20347] Updated weights for policy 0, policy_version 266279 (0.0007) [2023-12-27 07:29:28,996][20381] Updated weights for policy 1, policy_version 265892 (0.0009) [2023-12-27 07:29:29,015][20347] Updated weights for policy 0, policy_version 266289 (0.0006) [2023-12-27 07:29:29,070][20347] Updated weights for policy 0, policy_version 266299 (0.0006) [2023-12-27 07:29:29,674][20381] Updated weights for policy 1, policy_version 265902 (0.0007) [2023-12-27 07:29:29,717][20347] Updated weights for policy 0, policy_version 266309 (0.0007) [2023-12-27 07:29:29,734][20381] Updated weights for policy 1, policy_version 265912 (0.0011) [2023-12-27 07:29:29,774][20347] Updated weights for policy 0, policy_version 266319 (0.0008) [2023-12-27 07:29:29,784][20381] Updated weights for policy 1, policy_version 265922 (0.0011) [2023-12-27 07:29:29,831][20347] Updated weights for policy 0, policy_version 266329 (0.0010) [2023-12-27 07:29:30,477][20381] Updated weights for policy 1, policy_version 265932 (0.0011) [2023-12-27 07:29:30,520][20347] Updated weights for policy 0, policy_version 266339 (0.0007) [2023-12-27 07:29:30,527][20381] Updated weights for policy 1, policy_version 265942 (0.0010) [2023-12-27 07:29:30,566][20347] Updated weights for policy 0, policy_version 266349 (0.0006) [2023-12-27 07:29:30,572][20381] Updated weights for policy 1, policy_version 265952 (0.0010) [2023-12-27 07:29:30,618][20347] Updated weights for policy 0, policy_version 266359 (0.0006) [2023-12-27 07:29:31,254][20347] Updated weights for policy 0, policy_version 266369 (0.0008) [2023-12-27 07:29:31,272][20381] Updated weights for policy 1, policy_version 265962 (0.0010) [2023-12-27 07:29:31,307][20347] Updated weights for policy 0, policy_version 266379 (0.0007) [2023-12-27 07:29:31,332][20381] Updated weights for policy 1, policy_version 265972 (0.0006) [2023-12-27 07:29:31,358][20347] Updated weights for policy 0, policy_version 266389 (0.0006) [2023-12-27 07:29:31,399][20381] Updated weights for policy 1, policy_version 265982 (0.0008) [2023-12-27 07:29:31,419][20347] Updated weights for policy 0, policy_version 266399 (0.0007) [2023-12-27 07:29:31,459][20381] Updated weights for policy 1, policy_version 265992 (0.0007) [2023-12-27 07:29:32,028][20347] Updated weights for policy 0, policy_version 266409 (0.0006) [2023-12-27 07:29:32,084][20347] Updated weights for policy 0, policy_version 266419 (0.0006) [2023-12-27 07:29:32,111][20381] Updated weights for policy 1, policy_version 266002 (0.0007) [2023-12-27 07:29:32,135][20347] Updated weights for policy 0, policy_version 266429 (0.0007) [2023-12-27 07:29:32,172][20381] Updated weights for policy 1, policy_version 266012 (0.0006) [2023-12-27 07:29:32,229][20381] Updated weights for policy 1, policy_version 266022 (0.0007) [2023-12-27 07:29:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 136339456. Throughput: 0: 10227.8, 1: 10434.7. Samples: 136344832. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:29:32,829][20347] Updated weights for policy 0, policy_version 266439 (0.0007) [2023-12-27 07:29:32,863][20381] Updated weights for policy 1, policy_version 266032 (0.0008) [2023-12-27 07:29:32,894][20347] Updated weights for policy 0, policy_version 266449 (0.0005) [2023-12-27 07:29:32,910][20381] Updated weights for policy 1, policy_version 266042 (0.0007) [2023-12-27 07:29:32,945][20347] Updated weights for policy 0, policy_version 266459 (0.0006) [2023-12-27 07:29:32,966][20381] Updated weights for policy 1, policy_version 266052 (0.0007) [2023-12-27 07:29:33,530][20381] Updated weights for policy 1, policy_version 266062 (0.0006) [2023-12-27 07:29:33,584][20381] Updated weights for policy 1, policy_version 266072 (0.0005) [2023-12-27 07:29:33,638][20347] Updated weights for policy 0, policy_version 266469 (0.0005) [2023-12-27 07:29:33,646][20381] Updated weights for policy 1, policy_version 266082 (0.0006) [2023-12-27 07:29:33,688][20347] Updated weights for policy 0, policy_version 266479 (0.0006) [2023-12-27 07:29:33,738][20347] Updated weights for policy 0, policy_version 266489 (0.0007) [2023-12-27 07:29:34,240][20381] Updated weights for policy 1, policy_version 266092 (0.0010) [2023-12-27 07:29:34,290][20381] Updated weights for policy 1, policy_version 266102 (0.0010) [2023-12-27 07:29:34,343][20381] Updated weights for policy 1, policy_version 266112 (0.0010) [2023-12-27 07:29:34,394][20347] Updated weights for policy 0, policy_version 266499 (0.0007) [2023-12-27 07:29:34,446][20347] Updated weights for policy 0, policy_version 266509 (0.0007) [2023-12-27 07:29:34,511][20347] Updated weights for policy 0, policy_version 266519 (0.0005) [2023-12-27 07:29:35,046][20381] Updated weights for policy 1, policy_version 266122 (0.0010) [2023-12-27 07:29:35,098][20381] Updated weights for policy 1, policy_version 266132 (0.0009) [2023-12-27 07:29:35,115][20347] Updated weights for policy 0, policy_version 266529 (0.0007) [2023-12-27 07:29:35,156][20381] Updated weights for policy 1, policy_version 266142 (0.0006) [2023-12-27 07:29:35,168][20347] Updated weights for policy 0, policy_version 266539 (0.0006) [2023-12-27 07:29:35,214][20381] Updated weights for policy 1, policy_version 266152 (0.0008) [2023-12-27 07:29:35,217][20347] Updated weights for policy 0, policy_version 266549 (0.0005) [2023-12-27 07:29:35,272][20347] Updated weights for policy 0, policy_version 266559 (0.0005) [2023-12-27 07:29:35,825][20347] Updated weights for policy 0, policy_version 266569 (0.0005) [2023-12-27 07:29:35,874][20347] Updated weights for policy 0, policy_version 266579 (0.0005) [2023-12-27 07:29:35,909][20381] Updated weights for policy 1, policy_version 266162 (0.0006) [2023-12-27 07:29:35,928][20347] Updated weights for policy 0, policy_version 266589 (0.0006) [2023-12-27 07:29:35,959][20381] Updated weights for policy 1, policy_version 266172 (0.0005) [2023-12-27 07:29:36,020][20381] Updated weights for policy 1, policy_version 266182 (0.0007) [2023-12-27 07:29:36,538][20347] Updated weights for policy 0, policy_version 266600 (0.0005) [2023-12-27 07:29:36,597][20347] Updated weights for policy 0, policy_version 266610 (0.0006) [2023-12-27 07:29:36,653][20347] Updated weights for policy 0, policy_version 266620 (0.0005) [2023-12-27 07:29:36,715][20381] Updated weights for policy 1, policy_version 266192 (0.0007) [2023-12-27 07:29:36,767][20381] Updated weights for policy 1, policy_version 266202 (0.0008) [2023-12-27 07:29:36,820][20381] Updated weights for policy 1, policy_version 266212 (0.0008) [2023-12-27 07:29:37,315][20347] Updated weights for policy 0, policy_version 266630 (0.0007) [2023-12-27 07:29:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 136437760. Throughput: 0: 10230.7, 1: 10470.0. Samples: 136409036. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:29:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000266216_68165632.pth... [2023-12-27 07:29:37,369][20347] Updated weights for policy 0, policy_version 266640 (0.0008) [2023-12-27 07:29:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000265000_67854336.pth [2023-12-27 07:29:37,419][20347] Updated weights for policy 0, policy_version 266650 (0.0008) [2023-12-27 07:29:37,445][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000266656_68280320.pth... [2023-12-27 07:29:37,448][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000265440_67969024.pth [2023-12-27 07:29:37,503][20381] Updated weights for policy 1, policy_version 266222 (0.0008) [2023-12-27 07:29:37,546][20381] Updated weights for policy 1, policy_version 266232 (0.0008) [2023-12-27 07:29:37,608][20381] Updated weights for policy 1, policy_version 266242 (0.0010) [2023-12-27 07:29:38,036][20347] Updated weights for policy 0, policy_version 266660 (0.0008) [2023-12-27 07:29:38,086][20347] Updated weights for policy 0, policy_version 266670 (0.0007) [2023-12-27 07:29:38,139][20347] Updated weights for policy 0, policy_version 266680 (0.0008) [2023-12-27 07:29:38,198][20381] Updated weights for policy 1, policy_version 266252 (0.0006) [2023-12-27 07:29:38,253][20381] Updated weights for policy 1, policy_version 266262 (0.0005) [2023-12-27 07:29:38,311][20381] Updated weights for policy 1, policy_version 266272 (0.0006) [2023-12-27 07:29:38,883][20381] Updated weights for policy 1, policy_version 266282 (0.0007) [2023-12-27 07:29:38,910][20347] Updated weights for policy 0, policy_version 266690 (0.0008) [2023-12-27 07:29:38,940][20381] Updated weights for policy 1, policy_version 266292 (0.0006) [2023-12-27 07:29:38,961][20347] Updated weights for policy 0, policy_version 266700 (0.0007) [2023-12-27 07:29:38,999][20381] Updated weights for policy 1, policy_version 266302 (0.0006) [2023-12-27 07:29:39,006][20347] Updated weights for policy 0, policy_version 266710 (0.0006) [2023-12-27 07:29:39,057][20347] Updated weights for policy 0, policy_version 266720 (0.0008) [2023-12-27 07:29:39,059][20381] Updated weights for policy 1, policy_version 266312 (0.0007) [2023-12-27 07:29:39,694][20381] Updated weights for policy 1, policy_version 266322 (0.0007) [2023-12-27 07:29:39,753][20381] Updated weights for policy 1, policy_version 266332 (0.0007) [2023-12-27 07:29:39,808][20381] Updated weights for policy 1, policy_version 266342 (0.0006) [2023-12-27 07:29:39,832][20347] Updated weights for policy 0, policy_version 266730 (0.0007) [2023-12-27 07:29:39,895][20347] Updated weights for policy 0, policy_version 266740 (0.0007) [2023-12-27 07:29:39,953][20347] Updated weights for policy 0, policy_version 266750 (0.0007) [2023-12-27 07:29:40,491][20381] Updated weights for policy 1, policy_version 266352 (0.0008) [2023-12-27 07:29:40,542][20381] Updated weights for policy 1, policy_version 266362 (0.0008) [2023-12-27 07:29:40,599][20381] Updated weights for policy 1, policy_version 266372 (0.0008) [2023-12-27 07:29:40,655][20347] Updated weights for policy 0, policy_version 266760 (0.0007) [2023-12-27 07:29:40,713][20347] Updated weights for policy 0, policy_version 266770 (0.0007) [2023-12-27 07:29:40,757][20347] Updated weights for policy 0, policy_version 266780 (0.0008) [2023-12-27 07:29:41,245][20381] Updated weights for policy 1, policy_version 266382 (0.0008) [2023-12-27 07:29:41,301][20381] Updated weights for policy 1, policy_version 266392 (0.0008) [2023-12-27 07:29:41,368][20381] Updated weights for policy 1, policy_version 266402 (0.0008) [2023-12-27 07:29:41,487][20347] Updated weights for policy 0, policy_version 266790 (0.0008) [2023-12-27 07:29:41,547][20347] Updated weights for policy 0, policy_version 266800 (0.0010) [2023-12-27 07:29:41,611][20347] Updated weights for policy 0, policy_version 266810 (0.0010) [2023-12-27 07:29:42,055][20381] Updated weights for policy 1, policy_version 266412 (0.0006) [2023-12-27 07:29:42,113][20381] Updated weights for policy 1, policy_version 266422 (0.0007) [2023-12-27 07:29:42,177][20381] Updated weights for policy 1, policy_version 266432 (0.0007) [2023-12-27 07:29:42,292][20347] Updated weights for policy 0, policy_version 266820 (0.0008) [2023-12-27 07:29:42,357][20347] Updated weights for policy 0, policy_version 266830 (0.0008) [2023-12-27 07:29:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 136544256. Throughput: 0: 10244.8, 1: 10454.5. Samples: 136535152. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:42,363][19170] Avg episode reward: [(0, '999.300'), (1, '1000.000')] [2023-12-27 07:29:42,425][20347] Updated weights for policy 0, policy_version 266840 (0.0007) [2023-12-27 07:29:42,905][20381] Updated weights for policy 1, policy_version 266442 (0.0008) [2023-12-27 07:29:42,964][20381] Updated weights for policy 1, policy_version 266452 (0.0006) [2023-12-27 07:29:43,023][20381] Updated weights for policy 1, policy_version 266462 (0.0007) [2023-12-27 07:29:43,047][20347] Updated weights for policy 0, policy_version 266850 (0.0006) [2023-12-27 07:29:43,069][20381] Updated weights for policy 1, policy_version 266472 (0.0007) [2023-12-27 07:29:43,099][20347] Updated weights for policy 0, policy_version 266860 (0.0009) [2023-12-27 07:29:43,164][20347] Updated weights for policy 0, policy_version 266870 (0.0006) [2023-12-27 07:29:43,228][20347] Updated weights for policy 0, policy_version 266880 (0.0007) [2023-12-27 07:29:43,667][20381] Updated weights for policy 1, policy_version 266482 (0.0007) [2023-12-27 07:29:43,720][20381] Updated weights for policy 1, policy_version 266492 (0.0007) [2023-12-27 07:29:43,775][20381] Updated weights for policy 1, policy_version 266502 (0.0008) [2023-12-27 07:29:43,938][20347] Updated weights for policy 0, policy_version 266890 (0.0008) [2023-12-27 07:29:43,988][20347] Updated weights for policy 0, policy_version 266900 (0.0007) [2023-12-27 07:29:44,043][20347] Updated weights for policy 0, policy_version 266910 (0.0008) [2023-12-27 07:29:44,472][20381] Updated weights for policy 1, policy_version 266512 (0.0009) [2023-12-27 07:29:44,532][20381] Updated weights for policy 1, policy_version 266522 (0.0011) [2023-12-27 07:29:44,578][20381] Updated weights for policy 1, policy_version 266532 (0.0011) [2023-12-27 07:29:44,746][20347] Updated weights for policy 0, policy_version 266920 (0.0007) [2023-12-27 07:29:44,806][20347] Updated weights for policy 0, policy_version 266930 (0.0007) [2023-12-27 07:29:44,861][20347] Updated weights for policy 0, policy_version 266940 (0.0008) [2023-12-27 07:29:45,255][20381] Updated weights for policy 1, policy_version 266542 (0.0007) [2023-12-27 07:29:45,319][20381] Updated weights for policy 1, policy_version 266552 (0.0005) [2023-12-27 07:29:45,371][20381] Updated weights for policy 1, policy_version 266562 (0.0005) [2023-12-27 07:29:45,618][20347] Updated weights for policy 0, policy_version 266950 (0.0007) [2023-12-27 07:29:45,667][20347] Updated weights for policy 0, policy_version 266960 (0.0007) [2023-12-27 07:29:45,718][20347] Updated weights for policy 0, policy_version 266970 (0.0006) [2023-12-27 07:29:45,895][20381] Updated weights for policy 1, policy_version 266572 (0.0006) [2023-12-27 07:29:45,951][20381] Updated weights for policy 1, policy_version 266582 (0.0007) [2023-12-27 07:29:46,008][20381] Updated weights for policy 1, policy_version 266592 (0.0008) [2023-12-27 07:29:46,430][20347] Updated weights for policy 0, policy_version 266980 (0.0007) [2023-12-27 07:29:46,474][20347] Updated weights for policy 0, policy_version 266990 (0.0007) [2023-12-27 07:29:46,525][20347] Updated weights for policy 0, policy_version 267000 (0.0007) [2023-12-27 07:29:46,665][20381] Updated weights for policy 1, policy_version 266602 (0.0008) [2023-12-27 07:29:46,727][20381] Updated weights for policy 1, policy_version 266612 (0.0008) [2023-12-27 07:29:46,785][20381] Updated weights for policy 1, policy_version 266622 (0.0008) [2023-12-27 07:29:46,840][20381] Updated weights for policy 1, policy_version 266632 (0.0008) [2023-12-27 07:29:47,198][20347] Updated weights for policy 0, policy_version 267010 (0.0007) [2023-12-27 07:29:47,251][20347] Updated weights for policy 0, policy_version 267020 (0.0006) [2023-12-27 07:29:47,295][20347] Updated weights for policy 0, policy_version 267030 (0.0007) [2023-12-27 07:29:47,345][20347] Updated weights for policy 0, policy_version 267040 (0.0008) [2023-12-27 07:29:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 136650752. Throughput: 0: 10214.0, 1: 10470.7. Samples: 136657312. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:47,363][19170] Avg episode reward: [(0, '998.670'), (1, '1000.000')] [2023-12-27 07:29:47,529][20381] Updated weights for policy 1, policy_version 266642 (0.0005) [2023-12-27 07:29:47,591][20381] Updated weights for policy 1, policy_version 266652 (0.0005) [2023-12-27 07:29:47,646][20381] Updated weights for policy 1, policy_version 266662 (0.0011) [2023-12-27 07:29:48,043][20347] Updated weights for policy 0, policy_version 267050 (0.0008) [2023-12-27 07:29:48,088][20347] Updated weights for policy 0, policy_version 267060 (0.0008) [2023-12-27 07:29:48,137][20347] Updated weights for policy 0, policy_version 267070 (0.0008) [2023-12-27 07:29:48,187][20381] Updated weights for policy 1, policy_version 266672 (0.0007) [2023-12-27 07:29:48,238][20381] Updated weights for policy 1, policy_version 266682 (0.0005) [2023-12-27 07:29:48,291][20381] Updated weights for policy 1, policy_version 266692 (0.0008) [2023-12-27 07:29:48,836][20347] Updated weights for policy 0, policy_version 267080 (0.0007) [2023-12-27 07:29:48,886][20347] Updated weights for policy 0, policy_version 267090 (0.0006) [2023-12-27 07:29:48,909][20381] Updated weights for policy 1, policy_version 266702 (0.0010) [2023-12-27 07:29:48,941][20347] Updated weights for policy 0, policy_version 267100 (0.0005) [2023-12-27 07:29:48,961][20381] Updated weights for policy 1, policy_version 266712 (0.0010) [2023-12-27 07:29:49,021][20381] Updated weights for policy 1, policy_version 266722 (0.0010) [2023-12-27 07:29:49,544][20347] Updated weights for policy 0, policy_version 267110 (0.0005) [2023-12-27 07:29:49,603][20347] Updated weights for policy 0, policy_version 267120 (0.0008) [2023-12-27 07:29:49,664][20347] Updated weights for policy 0, policy_version 267130 (0.0008) [2023-12-27 07:29:49,675][20381] Updated weights for policy 1, policy_version 266732 (0.0008) [2023-12-27 07:29:49,729][20381] Updated weights for policy 1, policy_version 266742 (0.0006) [2023-12-27 07:29:49,788][20381] Updated weights for policy 1, policy_version 266752 (0.0008) [2023-12-27 07:29:50,383][20381] Updated weights for policy 1, policy_version 266762 (0.0009) [2023-12-27 07:29:50,400][20347] Updated weights for policy 0, policy_version 267140 (0.0008) [2023-12-27 07:29:50,442][20381] Updated weights for policy 1, policy_version 266772 (0.0006) [2023-12-27 07:29:50,458][20347] Updated weights for policy 0, policy_version 267150 (0.0007) [2023-12-27 07:29:50,494][20381] Updated weights for policy 1, policy_version 266782 (0.0005) [2023-12-27 07:29:50,506][20347] Updated weights for policy 0, policy_version 267160 (0.0006) [2023-12-27 07:29:50,540][20381] Updated weights for policy 1, policy_version 266792 (0.0006) [2023-12-27 07:29:51,149][20381] Updated weights for policy 1, policy_version 266802 (0.0007) [2023-12-27 07:29:51,210][20381] Updated weights for policy 1, policy_version 266812 (0.0006) [2023-12-27 07:29:51,244][20347] Updated weights for policy 0, policy_version 267170 (0.0008) [2023-12-27 07:29:51,270][20381] Updated weights for policy 1, policy_version 266822 (0.0007) [2023-12-27 07:29:51,296][20347] Updated weights for policy 0, policy_version 267180 (0.0007) [2023-12-27 07:29:51,345][20347] Updated weights for policy 0, policy_version 267190 (0.0008) [2023-12-27 07:29:51,415][20347] Updated weights for policy 0, policy_version 267200 (0.0007) [2023-12-27 07:29:51,925][20381] Updated weights for policy 1, policy_version 266832 (0.0010) [2023-12-27 07:29:51,984][20381] Updated weights for policy 1, policy_version 266842 (0.0011) [2023-12-27 07:29:52,042][20381] Updated weights for policy 1, policy_version 266852 (0.0011) [2023-12-27 07:29:52,120][20347] Updated weights for policy 0, policy_version 267210 (0.0007) [2023-12-27 07:29:52,177][20347] Updated weights for policy 0, policy_version 267220 (0.0005) [2023-12-27 07:29:52,231][20347] Updated weights for policy 0, policy_version 267230 (0.0006) [2023-12-27 07:29:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 136757248. Throughput: 0: 10220.8, 1: 10538.1. Samples: 136722092. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:52,363][19170] Avg episode reward: [(0, '999.370'), (1, '1000.000')] [2023-12-27 07:29:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000266856_68329472.pth... [2023-12-27 07:29:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000267232_68427776.pth... [2023-12-27 07:29:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000265608_68009984.pth [2023-12-27 07:29:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000266016_68116480.pth [2023-12-27 07:29:52,700][20381] Updated weights for policy 1, policy_version 266862 (0.0008) [2023-12-27 07:29:52,758][20381] Updated weights for policy 1, policy_version 266872 (0.0007) [2023-12-27 07:29:52,815][20381] Updated weights for policy 1, policy_version 266882 (0.0006) [2023-12-27 07:29:52,910][20347] Updated weights for policy 0, policy_version 267240 (0.0007) [2023-12-27 07:29:52,973][20347] Updated weights for policy 0, policy_version 267250 (0.0008) [2023-12-27 07:29:52,979][20291] KL-divergence is very high: 825.9846 [2023-12-27 07:29:53,036][20347] Updated weights for policy 0, policy_version 267260 (0.0008) [2023-12-27 07:29:53,445][20381] Updated weights for policy 1, policy_version 266892 (0.0006) [2023-12-27 07:29:53,506][20381] Updated weights for policy 1, policy_version 266902 (0.0007) [2023-12-27 07:29:53,551][20381] Updated weights for policy 1, policy_version 266912 (0.0008) [2023-12-27 07:29:53,609][20347] Updated weights for policy 0, policy_version 267270 (0.0006) [2023-12-27 07:29:53,662][20347] Updated weights for policy 0, policy_version 267280 (0.0005) [2023-12-27 07:29:53,705][20347] Updated weights for policy 0, policy_version 267290 (0.0005) [2023-12-27 07:29:54,271][20381] Updated weights for policy 1, policy_version 266922 (0.0007) [2023-12-27 07:29:54,276][20347] Updated weights for policy 0, policy_version 267300 (0.0006) [2023-12-27 07:29:54,322][20381] Updated weights for policy 1, policy_version 266932 (0.0007) [2023-12-27 07:29:54,333][20347] Updated weights for policy 0, policy_version 267310 (0.0009) [2023-12-27 07:29:54,375][20381] Updated weights for policy 1, policy_version 266942 (0.0010) [2023-12-27 07:29:54,381][20347] Updated weights for policy 0, policy_version 267320 (0.0008) [2023-12-27 07:29:54,423][20381] Updated weights for policy 1, policy_version 266952 (0.0007) [2023-12-27 07:29:55,013][20347] Updated weights for policy 0, policy_version 267330 (0.0008) [2023-12-27 07:29:55,069][20347] Updated weights for policy 0, policy_version 267340 (0.0006) [2023-12-27 07:29:55,093][20381] Updated weights for policy 1, policy_version 266962 (0.0011) [2023-12-27 07:29:55,121][20347] Updated weights for policy 0, policy_version 267350 (0.0006) [2023-12-27 07:29:55,149][20381] Updated weights for policy 1, policy_version 266972 (0.0009) [2023-12-27 07:29:55,170][20347] Updated weights for policy 0, policy_version 267360 (0.0006) [2023-12-27 07:29:55,205][20381] Updated weights for policy 1, policy_version 266982 (0.0011) [2023-12-27 07:29:55,867][20347] Updated weights for policy 0, policy_version 267370 (0.0006) [2023-12-27 07:29:55,879][20381] Updated weights for policy 1, policy_version 266992 (0.0010) [2023-12-27 07:29:55,914][20347] Updated weights for policy 0, policy_version 267380 (0.0006) [2023-12-27 07:29:55,936][20381] Updated weights for policy 1, policy_version 267002 (0.0010) [2023-12-27 07:29:55,970][20347] Updated weights for policy 0, policy_version 267390 (0.0007) [2023-12-27 07:29:55,988][20381] Updated weights for policy 1, policy_version 267012 (0.0011) [2023-12-27 07:29:56,639][20347] Updated weights for policy 0, policy_version 267400 (0.0007) [2023-12-27 07:29:56,679][20381] Updated weights for policy 1, policy_version 267022 (0.0011) [2023-12-27 07:29:56,697][20347] Updated weights for policy 0, policy_version 267410 (0.0009) [2023-12-27 07:29:56,732][20381] Updated weights for policy 1, policy_version 267032 (0.0011) [2023-12-27 07:29:56,753][20347] Updated weights for policy 0, policy_version 267420 (0.0005) [2023-12-27 07:29:56,783][20381] Updated weights for policy 1, policy_version 267042 (0.0010) [2023-12-27 07:29:57,352][20381] Updated weights for policy 1, policy_version 267052 (0.0009) [2023-12-27 07:29:57,362][20347] Updated weights for policy 0, policy_version 267430 (0.0008) [2023-12-27 07:29:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 136855552. Throughput: 0: 10225.4, 1: 10606.0. Samples: 136848004. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:29:57,363][19170] Avg episode reward: [(0, '990.530'), (1, '1000.000')] [2023-12-27 07:29:57,401][20381] Updated weights for policy 1, policy_version 267062 (0.0006) [2023-12-27 07:29:57,420][20347] Updated weights for policy 0, policy_version 267440 (0.0010) [2023-12-27 07:29:57,452][20381] Updated weights for policy 1, policy_version 267072 (0.0006) [2023-12-27 07:29:57,474][20347] Updated weights for policy 0, policy_version 267450 (0.0010) [2023-12-27 07:29:58,119][20347] Updated weights for policy 0, policy_version 267460 (0.0010) [2023-12-27 07:29:58,132][20381] Updated weights for policy 1, policy_version 267082 (0.0009) [2023-12-27 07:29:58,177][20347] Updated weights for policy 0, policy_version 267470 (0.0011) [2023-12-27 07:29:58,196][20381] Updated weights for policy 1, policy_version 267092 (0.0009) [2023-12-27 07:29:58,237][20347] Updated weights for policy 0, policy_version 267480 (0.0010) [2023-12-27 07:29:58,255][20381] Updated weights for policy 1, policy_version 267103 (0.0008) [2023-12-27 07:29:58,959][20347] Updated weights for policy 0, policy_version 267490 (0.0009) [2023-12-27 07:29:59,024][20347] Updated weights for policy 0, policy_version 267500 (0.0008) [2023-12-27 07:29:59,037][20381] Updated weights for policy 1, policy_version 267113 (0.0008) [2023-12-27 07:29:59,070][20347] Updated weights for policy 0, policy_version 267510 (0.0007) [2023-12-27 07:29:59,101][20381] Updated weights for policy 1, policy_version 267123 (0.0008) [2023-12-27 07:29:59,121][20347] Updated weights for policy 0, policy_version 267520 (0.0007) [2023-12-27 07:29:59,168][20381] Updated weights for policy 1, policy_version 267133 (0.0008) [2023-12-27 07:29:59,233][20381] Updated weights for policy 1, policy_version 267143 (0.0011) [2023-12-27 07:29:59,813][20291] KL-divergence is very high: 243.1136 [2023-12-27 07:29:59,819][20347] Updated weights for policy 0, policy_version 267530 (0.0009) [2023-12-27 07:29:59,833][20381] Updated weights for policy 1, policy_version 267153 (0.0007) [2023-12-27 07:29:59,871][20347] Updated weights for policy 0, policy_version 267540 (0.0007) [2023-12-27 07:29:59,889][20381] Updated weights for policy 1, policy_version 267163 (0.0007) [2023-12-27 07:29:59,920][20347] Updated weights for policy 0, policy_version 267550 (0.0008) [2023-12-27 07:29:59,954][20381] Updated weights for policy 1, policy_version 267173 (0.0007) [2023-12-27 07:30:00,611][20347] Updated weights for policy 0, policy_version 267560 (0.0008) [2023-12-27 07:30:00,661][20347] Updated weights for policy 0, policy_version 267570 (0.0006) [2023-12-27 07:30:00,674][20381] Updated weights for policy 1, policy_version 267183 (0.0007) [2023-12-27 07:30:00,717][20347] Updated weights for policy 0, policy_version 267580 (0.0006) [2023-12-27 07:30:00,733][20381] Updated weights for policy 1, policy_version 267193 (0.0006) [2023-12-27 07:30:00,782][20381] Updated weights for policy 1, policy_version 267203 (0.0007) [2023-12-27 07:30:01,451][20381] Updated weights for policy 1, policy_version 267213 (0.0006) [2023-12-27 07:30:01,464][20347] Updated weights for policy 0, policy_version 267590 (0.0007) [2023-12-27 07:30:01,520][20381] Updated weights for policy 1, policy_version 267223 (0.0005) [2023-12-27 07:30:01,526][20347] Updated weights for policy 0, policy_version 267600 (0.0007) [2023-12-27 07:30:01,576][20381] Updated weights for policy 1, policy_version 267233 (0.0006) [2023-12-27 07:30:01,579][20347] Updated weights for policy 0, policy_version 267610 (0.0005) [2023-12-27 07:30:02,232][20381] Updated weights for policy 1, policy_version 267243 (0.0006) [2023-12-27 07:30:02,295][20381] Updated weights for policy 1, policy_version 267253 (0.0007) [2023-12-27 07:30:02,305][20347] Updated weights for policy 0, policy_version 267620 (0.0007) [2023-12-27 07:30:02,361][20381] Updated weights for policy 1, policy_version 267263 (0.0007) [2023-12-27 07:30:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 136953856. Throughput: 0: 10264.1, 1: 10620.4. Samples: 136971064. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:30:02,363][19170] Avg episode reward: [(0, '952.030'), (1, '1000.000')] [2023-12-27 07:30:02,368][20347] Updated weights for policy 0, policy_version 267630 (0.0008) [2023-12-27 07:30:02,427][20347] Updated weights for policy 0, policy_version 267640 (0.0007) [2023-12-27 07:30:02,995][20381] Updated weights for policy 1, policy_version 267273 (0.0006) [2023-12-27 07:30:03,053][20381] Updated weights for policy 1, policy_version 267283 (0.0007) [2023-12-27 07:30:03,103][20381] Updated weights for policy 1, policy_version 267293 (0.0007) [2023-12-27 07:30:03,115][20347] Updated weights for policy 0, policy_version 267650 (0.0008) [2023-12-27 07:30:03,161][20381] Updated weights for policy 1, policy_version 267303 (0.0006) [2023-12-27 07:30:03,168][20347] Updated weights for policy 0, policy_version 267660 (0.0005) [2023-12-27 07:30:03,218][20347] Updated weights for policy 0, policy_version 267670 (0.0007) [2023-12-27 07:30:03,272][20347] Updated weights for policy 0, policy_version 267680 (0.0007) [2023-12-27 07:30:03,862][20381] Updated weights for policy 1, policy_version 267313 (0.0008) [2023-12-27 07:30:03,891][20347] Updated weights for policy 0, policy_version 267690 (0.0008) [2023-12-27 07:30:03,920][20381] Updated weights for policy 1, policy_version 267323 (0.0007) [2023-12-27 07:30:03,949][20347] Updated weights for policy 0, policy_version 267700 (0.0005) [2023-12-27 07:30:03,973][20381] Updated weights for policy 1, policy_version 267333 (0.0008) [2023-12-27 07:30:03,999][20347] Updated weights for policy 0, policy_version 267710 (0.0005) [2023-12-27 07:30:04,616][20347] Updated weights for policy 0, policy_version 267720 (0.0006) [2023-12-27 07:30:04,672][20347] Updated weights for policy 0, policy_version 267730 (0.0005) [2023-12-27 07:30:04,695][20381] Updated weights for policy 1, policy_version 267343 (0.0008) [2023-12-27 07:30:04,730][20347] Updated weights for policy 0, policy_version 267740 (0.0006) [2023-12-27 07:30:04,754][20381] Updated weights for policy 1, policy_version 267353 (0.0007) [2023-12-27 07:30:04,818][20381] Updated weights for policy 1, policy_version 267363 (0.0008) [2023-12-27 07:30:05,326][20347] Updated weights for policy 0, policy_version 267750 (0.0005) [2023-12-27 07:30:05,380][20347] Updated weights for policy 0, policy_version 267760 (0.0008) [2023-12-27 07:30:05,432][20347] Updated weights for policy 0, policy_version 267770 (0.0007) [2023-12-27 07:30:05,487][20381] Updated weights for policy 1, policy_version 267373 (0.0007) [2023-12-27 07:30:05,546][20381] Updated weights for policy 1, policy_version 267383 (0.0007) [2023-12-27 07:30:05,608][20381] Updated weights for policy 1, policy_version 267393 (0.0008) [2023-12-27 07:30:06,061][20347] Updated weights for policy 0, policy_version 267780 (0.0007) [2023-12-27 07:30:06,120][20347] Updated weights for policy 0, policy_version 267790 (0.0008) [2023-12-27 07:30:06,179][20347] Updated weights for policy 0, policy_version 267800 (0.0006) [2023-12-27 07:30:06,332][20381] Updated weights for policy 1, policy_version 267403 (0.0009) [2023-12-27 07:30:06,389][20381] Updated weights for policy 1, policy_version 267413 (0.0008) [2023-12-27 07:30:06,441][20381] Updated weights for policy 1, policy_version 267423 (0.0008) [2023-12-27 07:30:06,813][20347] Updated weights for policy 0, policy_version 267810 (0.0007) [2023-12-27 07:30:06,882][20347] Updated weights for policy 0, policy_version 267820 (0.0006) [2023-12-27 07:30:06,945][20347] Updated weights for policy 0, policy_version 267830 (0.0010) [2023-12-27 07:30:07,000][20347] Updated weights for policy 0, policy_version 267840 (0.0010) [2023-12-27 07:30:07,102][20381] Updated weights for policy 1, policy_version 267433 (0.0007) [2023-12-27 07:30:07,157][20381] Updated weights for policy 1, policy_version 267443 (0.0008) [2023-12-27 07:30:07,175][20292] KL-divergence is very high: 659.0668 [2023-12-27 07:30:07,212][20381] Updated weights for policy 1, policy_version 267453 (0.0008) [2023-12-27 07:30:07,220][20292] KL-divergence is very high: 264.5764 [2023-12-27 07:30:07,274][20381] Updated weights for policy 1, policy_version 267463 (0.0007) [2023-12-27 07:30:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 137068544. Throughput: 0: 10315.0, 1: 10580.3. Samples: 137033080. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:30:07,363][19170] Avg episode reward: [(0, '961.480'), (1, '991.250')] [2023-12-27 07:30:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000267840_68583424.pth... [2023-12-27 07:30:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000267464_68485120.pth... [2023-12-27 07:30:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000266656_68280320.pth [2023-12-27 07:30:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000266216_68165632.pth [2023-12-27 07:30:07,651][20347] Updated weights for policy 0, policy_version 267850 (0.0010) [2023-12-27 07:30:07,715][20347] Updated weights for policy 0, policy_version 267860 (0.0010) [2023-12-27 07:30:07,769][20347] Updated weights for policy 0, policy_version 267870 (0.0010) [2023-12-27 07:30:07,884][20381] Updated weights for policy 1, policy_version 267473 (0.0005) [2023-12-27 07:30:07,940][20381] Updated weights for policy 1, policy_version 267483 (0.0005) [2023-12-27 07:30:07,990][20381] Updated weights for policy 1, policy_version 267493 (0.0005) [2023-12-27 07:30:08,385][20347] Updated weights for policy 0, policy_version 267880 (0.0007) [2023-12-27 07:30:08,440][20347] Updated weights for policy 0, policy_version 267890 (0.0005) [2023-12-27 07:30:08,505][20347] Updated weights for policy 0, policy_version 267900 (0.0010) [2023-12-27 07:30:08,616][20381] Updated weights for policy 1, policy_version 267503 (0.0009) [2023-12-27 07:30:08,665][20381] Updated weights for policy 1, policy_version 267513 (0.0010) [2023-12-27 07:30:08,723][20381] Updated weights for policy 1, policy_version 267523 (0.0010) [2023-12-27 07:30:09,171][20347] Updated weights for policy 0, policy_version 267910 (0.0010) [2023-12-27 07:30:09,240][20347] Updated weights for policy 0, policy_version 267920 (0.0011) [2023-12-27 07:30:09,295][20347] Updated weights for policy 0, policy_version 267930 (0.0008) [2023-12-27 07:30:09,435][20381] Updated weights for policy 1, policy_version 267533 (0.0009) [2023-12-27 07:30:09,497][20381] Updated weights for policy 1, policy_version 267543 (0.0008) [2023-12-27 07:30:09,551][20381] Updated weights for policy 1, policy_version 267553 (0.0008) [2023-12-27 07:30:09,950][20347] Updated weights for policy 0, policy_version 267940 (0.0007) [2023-12-27 07:30:10,009][20347] Updated weights for policy 0, policy_version 267950 (0.0008) [2023-12-27 07:30:10,074][20347] Updated weights for policy 0, policy_version 267960 (0.0008) [2023-12-27 07:30:10,317][20381] Updated weights for policy 1, policy_version 267563 (0.0008) [2023-12-27 07:30:10,379][20381] Updated weights for policy 1, policy_version 267573 (0.0011) [2023-12-27 07:30:10,442][20381] Updated weights for policy 1, policy_version 267583 (0.0010) [2023-12-27 07:30:10,799][20347] Updated weights for policy 0, policy_version 267970 (0.0009) [2023-12-27 07:30:10,856][20347] Updated weights for policy 0, policy_version 267980 (0.0011) [2023-12-27 07:30:10,913][20347] Updated weights for policy 0, policy_version 267990 (0.0010) [2023-12-27 07:30:10,976][20347] Updated weights for policy 0, policy_version 268000 (0.0011) [2023-12-27 07:30:11,106][20381] Updated weights for policy 1, policy_version 267593 (0.0011) [2023-12-27 07:30:11,166][20381] Updated weights for policy 1, policy_version 267603 (0.0011) [2023-12-27 07:30:11,228][20381] Updated weights for policy 1, policy_version 267613 (0.0009) [2023-12-27 07:30:11,295][20381] Updated weights for policy 1, policy_version 267623 (0.0009) [2023-12-27 07:30:11,646][20347] Updated weights for policy 0, policy_version 268010 (0.0010) [2023-12-27 07:30:11,712][20347] Updated weights for policy 0, policy_version 268020 (0.0009) [2023-12-27 07:30:11,782][20347] Updated weights for policy 0, policy_version 268030 (0.0008) [2023-12-27 07:30:11,996][20381] Updated weights for policy 1, policy_version 267633 (0.0009) [2023-12-27 07:30:12,048][20381] Updated weights for policy 1, policy_version 267643 (0.0006) [2023-12-27 07:30:12,103][20381] Updated weights for policy 1, policy_version 267653 (0.0007) [2023-12-27 07:30:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 137166848. Throughput: 0: 10378.0, 1: 10518.9. Samples: 137157500. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:30:12,363][19170] Avg episode reward: [(0, '990.440'), (1, '991.250')] [2023-12-27 07:30:12,384][20347] Updated weights for policy 0, policy_version 268040 (0.0009) [2023-12-27 07:30:12,446][20347] Updated weights for policy 0, policy_version 268050 (0.0011) [2023-12-27 07:30:12,504][20347] Updated weights for policy 0, policy_version 268060 (0.0010) [2023-12-27 07:30:12,801][20381] Updated weights for policy 1, policy_version 267663 (0.0007) [2023-12-27 07:30:12,866][20381] Updated weights for policy 1, policy_version 267673 (0.0006) [2023-12-27 07:30:12,953][20381] Updated weights for policy 1, policy_version 267683 (0.0009) [2023-12-27 07:30:13,227][20347] Updated weights for policy 0, policy_version 268070 (0.0011) [2023-12-27 07:30:13,287][20347] Updated weights for policy 0, policy_version 268080 (0.0008) [2023-12-27 07:30:13,354][20347] Updated weights for policy 0, policy_version 268090 (0.0008) [2023-12-27 07:30:13,627][20381] Updated weights for policy 1, policy_version 267693 (0.0007) [2023-12-27 07:30:13,690][20381] Updated weights for policy 1, policy_version 267703 (0.0005) [2023-12-27 07:30:13,750][20381] Updated weights for policy 1, policy_version 267713 (0.0007) [2023-12-27 07:30:14,035][20347] Updated weights for policy 0, policy_version 268100 (0.0010) [2023-12-27 07:30:14,080][20347] Updated weights for policy 0, policy_version 268110 (0.0010) [2023-12-27 07:30:14,135][20347] Updated weights for policy 0, policy_version 268120 (0.0009) [2023-12-27 07:30:14,361][20381] Updated weights for policy 1, policy_version 267723 (0.0008) [2023-12-27 07:30:14,415][20381] Updated weights for policy 1, policy_version 267733 (0.0008) [2023-12-27 07:30:14,461][20381] Updated weights for policy 1, policy_version 267743 (0.0006) [2023-12-27 07:30:14,835][20347] Updated weights for policy 0, policy_version 268130 (0.0009) [2023-12-27 07:30:14,892][20347] Updated weights for policy 0, policy_version 268140 (0.0008) [2023-12-27 07:30:14,950][20347] Updated weights for policy 0, policy_version 268150 (0.0008) [2023-12-27 07:30:15,019][20347] Updated weights for policy 0, policy_version 268160 (0.0008) [2023-12-27 07:30:15,185][20381] Updated weights for policy 1, policy_version 267753 (0.0008) [2023-12-27 07:30:15,246][20381] Updated weights for policy 1, policy_version 267763 (0.0008) [2023-12-27 07:30:15,303][20381] Updated weights for policy 1, policy_version 267773 (0.0009) [2023-12-27 07:30:15,366][20381] Updated weights for policy 1, policy_version 267783 (0.0008) [2023-12-27 07:30:15,755][20347] Updated weights for policy 0, policy_version 268170 (0.0007) [2023-12-27 07:30:15,806][20347] Updated weights for policy 0, policy_version 268180 (0.0007) [2023-12-27 07:30:15,864][20347] Updated weights for policy 0, policy_version 268190 (0.0006) [2023-12-27 07:30:16,092][20381] Updated weights for policy 1, policy_version 267793 (0.0009) [2023-12-27 07:30:16,153][20381] Updated weights for policy 1, policy_version 267803 (0.0008) [2023-12-27 07:30:16,206][20381] Updated weights for policy 1, policy_version 267813 (0.0008) [2023-12-27 07:30:16,488][20347] Updated weights for policy 0, policy_version 268200 (0.0007) [2023-12-27 07:30:16,536][20347] Updated weights for policy 0, policy_version 268210 (0.0007) [2023-12-27 07:30:16,600][20347] Updated weights for policy 0, policy_version 268220 (0.0007) [2023-12-27 07:30:16,919][20381] Updated weights for policy 1, policy_version 267823 (0.0007) [2023-12-27 07:30:16,970][20381] Updated weights for policy 1, policy_version 267833 (0.0005) [2023-12-27 07:30:17,026][20381] Updated weights for policy 1, policy_version 267843 (0.0005) [2023-12-27 07:30:17,248][20347] Updated weights for policy 0, policy_version 268230 (0.0008) [2023-12-27 07:30:17,309][20347] Updated weights for policy 0, policy_version 268240 (0.0007) [2023-12-27 07:30:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 137265152. Throughput: 0: 10309.8, 1: 10440.1. Samples: 137278576. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:30:17,363][19170] Avg episode reward: [(0, '990.440'), (1, '991.250')] [2023-12-27 07:30:17,372][20347] Updated weights for policy 0, policy_version 268250 (0.0008) [2023-12-27 07:30:17,598][20381] Updated weights for policy 1, policy_version 267853 (0.0005) [2023-12-27 07:30:17,656][20381] Updated weights for policy 1, policy_version 267863 (0.0005) [2023-12-27 07:30:17,716][20381] Updated weights for policy 1, policy_version 267873 (0.0006) [2023-12-27 07:30:18,064][20347] Updated weights for policy 0, policy_version 268260 (0.0007) [2023-12-27 07:30:18,128][20347] Updated weights for policy 0, policy_version 268270 (0.0007) [2023-12-27 07:30:18,188][20347] Updated weights for policy 0, policy_version 268280 (0.0007) [2023-12-27 07:30:18,342][20381] Updated weights for policy 1, policy_version 267883 (0.0006) [2023-12-27 07:30:18,404][20381] Updated weights for policy 1, policy_version 267893 (0.0008) [2023-12-27 07:30:18,462][20381] Updated weights for policy 1, policy_version 267903 (0.0011) [2023-12-27 07:30:18,871][20347] Updated weights for policy 0, policy_version 268290 (0.0007) [2023-12-27 07:30:18,931][20347] Updated weights for policy 0, policy_version 268300 (0.0008) [2023-12-27 07:30:18,994][20347] Updated weights for policy 0, policy_version 268310 (0.0009) [2023-12-27 07:30:19,001][20291] KL-divergence is very high: 257.6599 [2023-12-27 07:30:19,058][20347] Updated weights for policy 0, policy_version 268320 (0.0011) [2023-12-27 07:30:19,200][20381] Updated weights for policy 1, policy_version 267913 (0.0010) [2023-12-27 07:30:19,279][20381] Updated weights for policy 1, policy_version 267923 (0.0008) [2023-12-27 07:30:19,335][20381] Updated weights for policy 1, policy_version 267933 (0.0008) [2023-12-27 07:30:19,401][20381] Updated weights for policy 1, policy_version 267943 (0.0008) [2023-12-27 07:30:19,813][20347] Updated weights for policy 0, policy_version 268330 (0.0008) [2023-12-27 07:30:19,873][20347] Updated weights for policy 0, policy_version 268340 (0.0008) [2023-12-27 07:30:19,932][20347] Updated weights for policy 0, policy_version 268350 (0.0008) [2023-12-27 07:30:20,115][20381] Updated weights for policy 1, policy_version 267953 (0.0008) [2023-12-27 07:30:20,175][20381] Updated weights for policy 1, policy_version 267963 (0.0008) [2023-12-27 07:30:20,238][20381] Updated weights for policy 1, policy_version 267973 (0.0007) [2023-12-27 07:30:20,642][20347] Updated weights for policy 0, policy_version 268360 (0.0007) [2023-12-27 07:30:20,686][20291] KL-divergence is very high: 119.9746 [2023-12-27 07:30:20,706][20347] Updated weights for policy 0, policy_version 268370 (0.0009) [2023-12-27 07:30:20,762][20347] Updated weights for policy 0, policy_version 268380 (0.0009) [2023-12-27 07:30:21,008][20381] Updated weights for policy 1, policy_version 267983 (0.0008) [2023-12-27 07:30:21,071][20381] Updated weights for policy 1, policy_version 267993 (0.0008) [2023-12-27 07:30:21,135][20381] Updated weights for policy 1, policy_version 268003 (0.0007) [2023-12-27 07:30:21,444][20291] KL-divergence is very high: 114.5661 [2023-12-27 07:30:21,469][20347] Updated weights for policy 0, policy_version 268390 (0.0008) [2023-12-27 07:30:21,535][20347] Updated weights for policy 0, policy_version 268400 (0.0008) [2023-12-27 07:30:21,550][20291] KL-divergence is very high: 108.6802 [2023-12-27 07:30:21,601][20347] Updated weights for policy 0, policy_version 268410 (0.0008) [2023-12-27 07:30:21,604][20291] KL-divergence is very high: 129.3134 [2023-12-27 07:30:21,850][20381] Updated weights for policy 1, policy_version 268013 (0.0006) [2023-12-27 07:30:21,903][20381] Updated weights for policy 1, policy_version 268023 (0.0008) [2023-12-27 07:30:21,960][20381] Updated weights for policy 1, policy_version 268033 (0.0005) [2023-12-27 07:30:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 137363456. Throughput: 0: 10246.2, 1: 10388.6. Samples: 137337596. Policy #0 lag: (min: 0.0, avg: 20.8, max: 32.0) [2023-12-27 07:30:22,363][19170] Avg episode reward: [(0, '860.560'), (1, '990.450')] [2023-12-27 07:30:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000268040_68632576.pth... [2023-12-27 07:30:22,370][20347] Updated weights for policy 0, policy_version 268420 (0.0009) [2023-12-27 07:30:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000266856_68329472.pth [2023-12-27 07:30:22,424][20347] Updated weights for policy 0, policy_version 268430 (0.0008) [2023-12-27 07:30:22,482][20347] Updated weights for policy 0, policy_version 268440 (0.0007) [2023-12-27 07:30:22,534][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000268448_68739072.pth... [2023-12-27 07:30:22,538][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000267232_68427776.pth [2023-12-27 07:30:22,546][20381] Updated weights for policy 1, policy_version 268043 (0.0007) [2023-12-27 07:30:22,597][20381] Updated weights for policy 1, policy_version 268053 (0.0008) [2023-12-27 07:30:22,610][20292] KL-divergence is very high: 192.7691 [2023-12-27 07:30:22,660][20381] Updated weights for policy 1, policy_version 268063 (0.0008) [2023-12-27 07:30:23,222][20347] Updated weights for policy 0, policy_version 268450 (0.0007) [2023-12-27 07:30:23,268][20347] Updated weights for policy 0, policy_version 268460 (0.0005) [2023-12-27 07:30:23,320][20381] Updated weights for policy 1, policy_version 268073 (0.0006) [2023-12-27 07:30:23,330][20347] Updated weights for policy 0, policy_version 268470 (0.0008) [2023-12-27 07:30:23,368][20381] Updated weights for policy 1, policy_version 268083 (0.0005) [2023-12-27 07:30:23,383][20347] Updated weights for policy 0, policy_version 268480 (0.0010) [2023-12-27 07:30:23,416][20381] Updated weights for policy 1, policy_version 268093 (0.0005) [2023-12-27 07:30:23,460][20292] KL-divergence is very high: 126.8185 [2023-12-27 07:30:23,472][20381] Updated weights for policy 1, policy_version 268103 (0.0008) [2023-12-27 07:30:23,999][20347] Updated weights for policy 0, policy_version 268490 (0.0008) [2023-12-27 07:30:24,059][20347] Updated weights for policy 0, policy_version 268500 (0.0010) [2023-12-27 07:30:24,116][20347] Updated weights for policy 0, policy_version 268510 (0.0010) [2023-12-27 07:30:24,169][20292] KL-divergence is very high: 106.8058 [2023-12-27 07:30:24,198][20381] Updated weights for policy 1, policy_version 268113 (0.0006) [2023-12-27 07:30:24,218][20292] KL-divergence is very high: 203.9040 [2023-12-27 07:30:24,259][20381] Updated weights for policy 1, policy_version 268123 (0.0005) [2023-12-27 07:30:24,310][20292] KL-divergence is very high: 181.6903 [2023-12-27 07:30:24,315][20381] Updated weights for policy 1, policy_version 268133 (0.0005) [2023-12-27 07:30:24,752][20347] Updated weights for policy 0, policy_version 268520 (0.0010) [2023-12-27 07:30:24,801][20347] Updated weights for policy 0, policy_version 268530 (0.0010) [2023-12-27 07:30:24,867][20347] Updated weights for policy 0, policy_version 268540 (0.0006) [2023-12-27 07:30:24,928][20381] Updated weights for policy 1, policy_version 268143 (0.0009) [2023-12-27 07:30:24,982][20381] Updated weights for policy 1, policy_version 268153 (0.0010) [2023-12-27 07:30:25,037][20381] Updated weights for policy 1, policy_version 268163 (0.0011) [2023-12-27 07:30:25,055][20292] KL-divergence is very high: 116.5682 [2023-12-27 07:30:25,522][20347] Updated weights for policy 0, policy_version 268550 (0.0010) [2023-12-27 07:30:25,575][20347] Updated weights for policy 0, policy_version 268560 (0.0010) [2023-12-27 07:30:25,631][20347] Updated weights for policy 0, policy_version 268570 (0.0010) [2023-12-27 07:30:25,718][20381] Updated weights for policy 1, policy_version 268173 (0.0010) [2023-12-27 07:30:25,777][20381] Updated weights for policy 1, policy_version 268183 (0.0010) [2023-12-27 07:30:25,782][20292] KL-divergence is very high: 204.0363 [2023-12-27 07:30:25,835][20381] Updated weights for policy 1, policy_version 268193 (0.0010) [2023-12-27 07:30:26,313][20347] Updated weights for policy 0, policy_version 268580 (0.0010) [2023-12-27 07:30:26,363][20347] Updated weights for policy 0, policy_version 268590 (0.0010) [2023-12-27 07:30:26,410][20347] Updated weights for policy 0, policy_version 268600 (0.0010) [2023-12-27 07:30:26,502][20381] Updated weights for policy 1, policy_version 268203 (0.0010) [2023-12-27 07:30:26,562][20381] Updated weights for policy 1, policy_version 268213 (0.0010) [2023-12-27 07:30:26,612][20381] Updated weights for policy 1, policy_version 268223 (0.0010) [2023-12-27 07:30:27,094][20347] Updated weights for policy 0, policy_version 268610 (0.0010) [2023-12-27 07:30:27,143][20347] Updated weights for policy 0, policy_version 268620 (0.0010) [2023-12-27 07:30:27,199][20347] Updated weights for policy 0, policy_version 268630 (0.0010) [2023-12-27 07:30:27,257][20347] Updated weights for policy 0, policy_version 268640 (0.0009) [2023-12-27 07:30:27,265][20381] Updated weights for policy 1, policy_version 268233 (0.0010) [2023-12-27 07:30:27,331][20381] Updated weights for policy 1, policy_version 268243 (0.0010) [2023-12-27 07:30:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 137469952. Throughput: 0: 10211.3, 1: 10333.2. Samples: 137459652. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:27,363][19170] Avg episode reward: [(0, '777.970'), (1, '942.500')] [2023-12-27 07:30:27,393][20381] Updated weights for policy 1, policy_version 268253 (0.0010) [2023-12-27 07:30:27,452][20381] Updated weights for policy 1, policy_version 268263 (0.0010) [2023-12-27 07:30:27,870][20347] Updated weights for policy 0, policy_version 268650 (0.0010) [2023-12-27 07:30:27,918][20347] Updated weights for policy 0, policy_version 268660 (0.0010) [2023-12-27 07:30:27,973][20347] Updated weights for policy 0, policy_version 268670 (0.0006) [2023-12-27 07:30:28,130][20381] Updated weights for policy 1, policy_version 268273 (0.0007) [2023-12-27 07:30:28,192][20381] Updated weights for policy 1, policy_version 268283 (0.0007) [2023-12-27 07:30:28,249][20381] Updated weights for policy 1, policy_version 268293 (0.0007) [2023-12-27 07:30:28,607][20347] Updated weights for policy 0, policy_version 268680 (0.0010) [2023-12-27 07:30:28,654][20347] Updated weights for policy 0, policy_version 268690 (0.0010) [2023-12-27 07:30:28,702][20347] Updated weights for policy 0, policy_version 268700 (0.0010) [2023-12-27 07:30:28,893][20381] Updated weights for policy 1, policy_version 268303 (0.0007) [2023-12-27 07:30:28,950][20381] Updated weights for policy 1, policy_version 268313 (0.0007) [2023-12-27 07:30:29,001][20381] Updated weights for policy 1, policy_version 268323 (0.0008) [2023-12-27 07:30:29,397][20347] Updated weights for policy 0, policy_version 268710 (0.0010) [2023-12-27 07:30:29,450][20347] Updated weights for policy 0, policy_version 268720 (0.0010) [2023-12-27 07:30:29,498][20347] Updated weights for policy 0, policy_version 268730 (0.0010) [2023-12-27 07:30:29,734][20381] Updated weights for policy 1, policy_version 268333 (0.0007) [2023-12-27 07:30:29,792][20381] Updated weights for policy 1, policy_version 268343 (0.0007) [2023-12-27 07:30:29,849][20381] Updated weights for policy 1, policy_version 268353 (0.0006) [2023-12-27 07:30:30,186][20347] Updated weights for policy 0, policy_version 268740 (0.0010) [2023-12-27 07:30:30,246][20347] Updated weights for policy 0, policy_version 268750 (0.0006) [2023-12-27 07:30:30,310][20347] Updated weights for policy 0, policy_version 268760 (0.0005) [2023-12-27 07:30:30,515][20381] Updated weights for policy 1, policy_version 268363 (0.0006) [2023-12-27 07:30:30,568][20381] Updated weights for policy 1, policy_version 268373 (0.0006) [2023-12-27 07:30:30,571][20292] KL-divergence is very high: 161.6462 [2023-12-27 07:30:30,592][20292] KL-divergence is very high: 276.6261 [2023-12-27 07:30:30,614][20292] KL-divergence is very high: 149.1564 [2023-12-27 07:30:30,619][20381] Updated weights for policy 1, policy_version 268383 (0.0007) [2023-12-27 07:30:30,922][20347] Updated weights for policy 0, policy_version 268770 (0.0007) [2023-12-27 07:30:30,975][20291] KL-divergence is very high: 114.6086 [2023-12-27 07:30:30,983][20291] KL-divergence is very high: 156.9534 [2023-12-27 07:30:30,983][20347] Updated weights for policy 0, policy_version 268780 (0.0010) [2023-12-27 07:30:31,047][20347] Updated weights for policy 0, policy_version 268790 (0.0010) [2023-12-27 07:30:31,114][20347] Updated weights for policy 0, policy_version 268800 (0.0010) [2023-12-27 07:30:31,246][20381] Updated weights for policy 1, policy_version 268393 (0.0007) [2023-12-27 07:30:31,308][20381] Updated weights for policy 1, policy_version 268403 (0.0011) [2023-12-27 07:30:31,372][20381] Updated weights for policy 1, policy_version 268413 (0.0010) [2023-12-27 07:30:31,426][20381] Updated weights for policy 1, policy_version 268423 (0.0008) [2023-12-27 07:30:31,774][20347] Updated weights for policy 0, policy_version 268810 (0.0010) [2023-12-27 07:30:31,822][20347] Updated weights for policy 0, policy_version 268820 (0.0010) [2023-12-27 07:30:31,879][20347] Updated weights for policy 0, policy_version 268830 (0.0010) [2023-12-27 07:30:32,134][20381] Updated weights for policy 1, policy_version 268433 (0.0011) [2023-12-27 07:30:32,194][20381] Updated weights for policy 1, policy_version 268443 (0.0011) [2023-12-27 07:30:32,254][20381] Updated weights for policy 1, policy_version 268453 (0.0010) [2023-12-27 07:30:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 137576448. Throughput: 0: 10302.6, 1: 10300.7. Samples: 137584460. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:32,363][19170] Avg episode reward: [(0, '559.000'), (1, '923.130')] [2023-12-27 07:30:32,555][20347] Updated weights for policy 0, policy_version 268840 (0.0009) [2023-12-27 07:30:32,610][20347] Updated weights for policy 0, policy_version 268850 (0.0007) [2023-12-27 07:30:32,667][20347] Updated weights for policy 0, policy_version 268860 (0.0008) [2023-12-27 07:30:32,849][20381] Updated weights for policy 1, policy_version 268463 (0.0007) [2023-12-27 07:30:32,856][20292] KL-divergence is very high: 115.1702 [2023-12-27 07:30:32,904][20381] Updated weights for policy 1, policy_version 268473 (0.0006) [2023-12-27 07:30:32,953][20381] Updated weights for policy 1, policy_version 268483 (0.0006) [2023-12-27 07:30:33,326][20347] Updated weights for policy 0, policy_version 268870 (0.0009) [2023-12-27 07:30:33,372][20347] Updated weights for policy 0, policy_version 268880 (0.0010) [2023-12-27 07:30:33,421][20347] Updated weights for policy 0, policy_version 268890 (0.0009) [2023-12-27 07:30:33,511][20381] Updated weights for policy 1, policy_version 268493 (0.0007) [2023-12-27 07:30:33,556][20381] Updated weights for policy 1, policy_version 268503 (0.0007) [2023-12-27 07:30:33,602][20381] Updated weights for policy 1, policy_version 268513 (0.0007) [2023-12-27 07:30:34,091][20347] Updated weights for policy 0, policy_version 268900 (0.0010) [2023-12-27 07:30:34,151][20347] Updated weights for policy 0, policy_version 268910 (0.0008) [2023-12-27 07:30:34,210][20347] Updated weights for policy 0, policy_version 268920 (0.0008) [2023-12-27 07:30:34,215][20381] Updated weights for policy 1, policy_version 268523 (0.0005) [2023-12-27 07:30:34,276][20381] Updated weights for policy 1, policy_version 268533 (0.0007) [2023-12-27 07:30:34,330][20381] Updated weights for policy 1, policy_version 268543 (0.0008) [2023-12-27 07:30:34,920][20347] Updated weights for policy 0, policy_version 268930 (0.0009) [2023-12-27 07:30:34,980][20347] Updated weights for policy 0, policy_version 268940 (0.0010) [2023-12-27 07:30:34,997][20381] Updated weights for policy 1, policy_version 268553 (0.0007) [2023-12-27 07:30:35,034][20347] Updated weights for policy 0, policy_version 268950 (0.0010) [2023-12-27 07:30:35,053][20381] Updated weights for policy 1, policy_version 268563 (0.0005) [2023-12-27 07:30:35,094][20347] Updated weights for policy 0, policy_version 268960 (0.0010) [2023-12-27 07:30:35,111][20381] Updated weights for policy 1, policy_version 268573 (0.0007) [2023-12-27 07:30:35,165][20381] Updated weights for policy 1, policy_version 268583 (0.0010) [2023-12-27 07:30:35,695][20347] Updated weights for policy 0, policy_version 268970 (0.0011) [2023-12-27 07:30:35,716][20381] Updated weights for policy 1, policy_version 268593 (0.0010) [2023-12-27 07:30:35,753][20347] Updated weights for policy 0, policy_version 268980 (0.0010) [2023-12-27 07:30:35,767][20381] Updated weights for policy 1, policy_version 268603 (0.0010) [2023-12-27 07:30:35,801][20347] Updated weights for policy 0, policy_version 268990 (0.0010) [2023-12-27 07:30:35,816][20381] Updated weights for policy 1, policy_version 268613 (0.0010) [2023-12-27 07:30:36,453][20381] Updated weights for policy 1, policy_version 268623 (0.0007) [2023-12-27 07:30:36,484][20347] Updated weights for policy 0, policy_version 269000 (0.0010) [2023-12-27 07:30:36,513][20381] Updated weights for policy 1, policy_version 268633 (0.0006) [2023-12-27 07:30:36,546][20347] Updated weights for policy 0, policy_version 269010 (0.0010) [2023-12-27 07:30:36,574][20381] Updated weights for policy 1, policy_version 268643 (0.0005) [2023-12-27 07:30:36,607][20347] Updated weights for policy 0, policy_version 269020 (0.0010) [2023-12-27 07:30:37,103][20381] Updated weights for policy 1, policy_version 268653 (0.0005) [2023-12-27 07:30:37,158][20381] Updated weights for policy 1, policy_version 268663 (0.0010) [2023-12-27 07:30:37,200][20292] KL-divergence is very high: 188.6405 [2023-12-27 07:30:37,225][20381] Updated weights for policy 1, policy_version 268673 (0.0009) [2023-12-27 07:30:37,296][20347] Updated weights for policy 0, policy_version 269030 (0.0011) [2023-12-27 07:30:37,358][20347] Updated weights for policy 0, policy_version 269040 (0.0010) [2023-12-27 07:30:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 137682944. Throughput: 0: 10293.8, 1: 10306.9. Samples: 137649120. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:37,363][19170] Avg episode reward: [(0, '352.320'), (1, '916.750')] [2023-12-27 07:30:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000268680_68796416.pth... [2023-12-27 07:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000267464_68485120.pth [2023-12-27 07:30:37,411][20347] Updated weights for policy 0, policy_version 269050 (0.0007) [2023-12-27 07:30:37,443][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000269056_68894720.pth... [2023-12-27 07:30:37,447][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000267840_68583424.pth [2023-12-27 07:30:37,912][20381] Updated weights for policy 1, policy_version 268683 (0.0007) [2023-12-27 07:30:37,963][20381] Updated weights for policy 1, policy_version 268693 (0.0005) [2023-12-27 07:30:38,020][20381] Updated weights for policy 1, policy_version 268703 (0.0005) [2023-12-27 07:30:38,051][20347] Updated weights for policy 0, policy_version 269060 (0.0009) [2023-12-27 07:30:38,099][20347] Updated weights for policy 0, policy_version 269070 (0.0009) [2023-12-27 07:30:38,143][20347] Updated weights for policy 0, policy_version 269080 (0.0008) [2023-12-27 07:30:38,656][20381] Updated weights for policy 1, policy_version 268713 (0.0006) [2023-12-27 07:30:38,715][20381] Updated weights for policy 1, policy_version 268723 (0.0011) [2023-12-27 07:30:38,774][20381] Updated weights for policy 1, policy_version 268733 (0.0011) [2023-12-27 07:30:38,806][20347] Updated weights for policy 0, policy_version 269090 (0.0007) [2023-12-27 07:30:38,834][20381] Updated weights for policy 1, policy_version 268743 (0.0011) [2023-12-27 07:30:38,856][20347] Updated weights for policy 0, policy_version 269100 (0.0010) [2023-12-27 07:30:38,905][20347] Updated weights for policy 0, policy_version 269110 (0.0010) [2023-12-27 07:30:38,963][20347] Updated weights for policy 0, policy_version 269120 (0.0009) [2023-12-27 07:30:39,484][20381] Updated weights for policy 1, policy_version 268753 (0.0009) [2023-12-27 07:30:39,550][20381] Updated weights for policy 1, policy_version 268763 (0.0005) [2023-12-27 07:30:39,614][20381] Updated weights for policy 1, policy_version 268773 (0.0005) [2023-12-27 07:30:39,639][20347] Updated weights for policy 0, policy_version 269130 (0.0006) [2023-12-27 07:30:39,689][20347] Updated weights for policy 0, policy_version 269140 (0.0005) [2023-12-27 07:30:39,748][20347] Updated weights for policy 0, policy_version 269150 (0.0005) [2023-12-27 07:30:40,190][20381] Updated weights for policy 1, policy_version 268783 (0.0009) [2023-12-27 07:30:40,250][20381] Updated weights for policy 1, policy_version 268793 (0.0011) [2023-12-27 07:30:40,312][20381] Updated weights for policy 1, policy_version 268803 (0.0006) [2023-12-27 07:30:40,422][20347] Updated weights for policy 0, policy_version 269160 (0.0010) [2023-12-27 07:30:40,481][20347] Updated weights for policy 0, policy_version 269170 (0.0010) [2023-12-27 07:30:40,538][20347] Updated weights for policy 0, policy_version 269180 (0.0010) [2023-12-27 07:30:40,964][20381] Updated weights for policy 1, policy_version 268813 (0.0008) [2023-12-27 07:30:41,027][20381] Updated weights for policy 1, policy_version 268823 (0.0006) [2023-12-27 07:30:41,088][20381] Updated weights for policy 1, policy_version 268833 (0.0008) [2023-12-27 07:30:41,234][20347] Updated weights for policy 0, policy_version 269190 (0.0009) [2023-12-27 07:30:41,304][20347] Updated weights for policy 0, policy_version 269200 (0.0008) [2023-12-27 07:30:41,378][20347] Updated weights for policy 0, policy_version 269211 (0.0009) [2023-12-27 07:30:41,773][20381] Updated weights for policy 1, policy_version 268843 (0.0008) [2023-12-27 07:30:41,827][20381] Updated weights for policy 1, policy_version 268853 (0.0006) [2023-12-27 07:30:41,844][20292] KL-divergence is very high: 140.7944 [2023-12-27 07:30:41,886][20381] Updated weights for policy 1, policy_version 268863 (0.0007) [2023-12-27 07:30:42,101][20347] Updated weights for policy 0, policy_version 269221 (0.0009) [2023-12-27 07:30:42,181][20347] Updated weights for policy 0, policy_version 269231 (0.0007) [2023-12-27 07:30:42,236][20347] Updated weights for policy 0, policy_version 269241 (0.0008) [2023-12-27 07:30:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 137789440. Throughput: 0: 10291.4, 1: 10366.7. Samples: 137777616. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:42,363][19170] Avg episode reward: [(0, '413.110'), (1, '917.010')] [2023-12-27 07:30:42,520][20381] Updated weights for policy 1, policy_version 268873 (0.0008) [2023-12-27 07:30:42,573][20381] Updated weights for policy 1, policy_version 268883 (0.0008) [2023-12-27 07:30:42,623][20381] Updated weights for policy 1, policy_version 268893 (0.0008) [2023-12-27 07:30:42,684][20381] Updated weights for policy 1, policy_version 268903 (0.0007) [2023-12-27 07:30:42,901][20347] Updated weights for policy 0, policy_version 269251 (0.0007) [2023-12-27 07:30:42,952][20347] Updated weights for policy 0, policy_version 269262 (0.0007) [2023-12-27 07:30:43,016][20347] Updated weights for policy 0, policy_version 269272 (0.0006) [2023-12-27 07:30:43,391][20381] Updated weights for policy 1, policy_version 268913 (0.0006) [2023-12-27 07:30:43,451][20381] Updated weights for policy 1, policy_version 268923 (0.0011) [2023-12-27 07:30:43,501][20381] Updated weights for policy 1, policy_version 268933 (0.0008) [2023-12-27 07:30:43,657][20347] Updated weights for policy 0, policy_version 269282 (0.0006) [2023-12-27 07:30:43,719][20347] Updated weights for policy 0, policy_version 269292 (0.0007) [2023-12-27 07:30:43,784][20347] Updated weights for policy 0, policy_version 269302 (0.0007) [2023-12-27 07:30:43,846][20347] Updated weights for policy 0, policy_version 269312 (0.0006) [2023-12-27 07:30:44,161][20381] Updated weights for policy 1, policy_version 268943 (0.0009) [2023-12-27 07:30:44,201][20292] KL-divergence is very high: 157.6111 [2023-12-27 07:30:44,220][20381] Updated weights for policy 1, policy_version 268953 (0.0010) [2023-12-27 07:30:44,279][20381] Updated weights for policy 1, policy_version 268963 (0.0011) [2023-12-27 07:30:44,495][20347] Updated weights for policy 0, policy_version 269322 (0.0010) [2023-12-27 07:30:44,547][20347] Updated weights for policy 0, policy_version 269332 (0.0010) [2023-12-27 07:30:44,602][20347] Updated weights for policy 0, policy_version 269342 (0.0010) [2023-12-27 07:30:44,967][20381] Updated weights for policy 1, policy_version 268973 (0.0011) [2023-12-27 07:30:45,024][20381] Updated weights for policy 1, policy_version 268983 (0.0011) [2023-12-27 07:30:45,077][20381] Updated weights for policy 1, policy_version 268993 (0.0011) [2023-12-27 07:30:45,273][20347] Updated weights for policy 0, policy_version 269352 (0.0010) [2023-12-27 07:30:45,322][20347] Updated weights for policy 0, policy_version 269362 (0.0010) [2023-12-27 07:30:45,367][20347] Updated weights for policy 0, policy_version 269372 (0.0010) [2023-12-27 07:30:45,677][20381] Updated weights for policy 1, policy_version 269003 (0.0009) [2023-12-27 07:30:45,730][20381] Updated weights for policy 1, policy_version 269013 (0.0005) [2023-12-27 07:30:45,786][20381] Updated weights for policy 1, policy_version 269023 (0.0005) [2023-12-27 07:30:46,016][20347] Updated weights for policy 0, policy_version 269382 (0.0009) [2023-12-27 07:30:46,080][20347] Updated weights for policy 0, policy_version 269392 (0.0009) [2023-12-27 07:30:46,141][20347] Updated weights for policy 0, policy_version 269402 (0.0010) [2023-12-27 07:30:46,401][20381] Updated weights for policy 1, policy_version 269033 (0.0007) [2023-12-27 07:30:46,458][20381] Updated weights for policy 1, policy_version 269043 (0.0007) [2023-12-27 07:30:46,512][20381] Updated weights for policy 1, policy_version 269053 (0.0007) [2023-12-27 07:30:46,561][20381] Updated weights for policy 1, policy_version 269063 (0.0008) [2023-12-27 07:30:46,800][20347] Updated weights for policy 0, policy_version 269412 (0.0009) [2023-12-27 07:30:46,853][20347] Updated weights for policy 0, policy_version 269422 (0.0005) [2023-12-27 07:30:46,902][20347] Updated weights for policy 0, policy_version 269432 (0.0005) [2023-12-27 07:30:47,280][20381] Updated weights for policy 1, policy_version 269073 (0.0010) [2023-12-27 07:30:47,331][20381] Updated weights for policy 1, policy_version 269083 (0.0010) [2023-12-27 07:30:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 137887744. Throughput: 0: 10313.2, 1: 10383.3. Samples: 137902404. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:47,363][19170] Avg episode reward: [(0, '362.280'), (1, '932.990')] [2023-12-27 07:30:47,382][20381] Updated weights for policy 1, policy_version 269093 (0.0010) [2023-12-27 07:30:47,511][20347] Updated weights for policy 0, policy_version 269442 (0.0007) [2023-12-27 07:30:47,557][20347] Updated weights for policy 0, policy_version 269452 (0.0005) [2023-12-27 07:30:47,610][20347] Updated weights for policy 0, policy_version 269462 (0.0005) [2023-12-27 07:30:47,666][20347] Updated weights for policy 0, policy_version 269472 (0.0005) [2023-12-27 07:30:47,982][20381] Updated weights for policy 1, policy_version 269103 (0.0007) [2023-12-27 07:30:48,041][20381] Updated weights for policy 1, policy_version 269113 (0.0006) [2023-12-27 07:30:48,088][20381] Updated weights for policy 1, policy_version 269123 (0.0005) [2023-12-27 07:30:48,224][20347] Updated weights for policy 0, policy_version 269482 (0.0005) [2023-12-27 07:30:48,276][20347] Updated weights for policy 0, policy_version 269492 (0.0005) [2023-12-27 07:30:48,331][20347] Updated weights for policy 0, policy_version 269502 (0.0006) [2023-12-27 07:30:48,751][20381] Updated weights for policy 1, policy_version 269133 (0.0006) [2023-12-27 07:30:48,799][20381] Updated weights for policy 1, policy_version 269143 (0.0007) [2023-12-27 07:30:48,851][20381] Updated weights for policy 1, policy_version 269153 (0.0007) [2023-12-27 07:30:48,880][20292] KL-divergence is very high: 149.6359 [2023-12-27 07:30:48,982][20347] Updated weights for policy 0, policy_version 269512 (0.0010) [2023-12-27 07:30:49,028][20347] Updated weights for policy 0, policy_version 269522 (0.0010) [2023-12-27 07:30:49,087][20347] Updated weights for policy 0, policy_version 269532 (0.0010) [2023-12-27 07:30:49,543][20381] Updated weights for policy 1, policy_version 269163 (0.0007) [2023-12-27 07:30:49,600][20381] Updated weights for policy 1, policy_version 269173 (0.0009) [2023-12-27 07:30:49,658][20381] Updated weights for policy 1, policy_version 269183 (0.0009) [2023-12-27 07:30:49,792][20347] Updated weights for policy 0, policy_version 269542 (0.0007) [2023-12-27 07:30:49,847][20347] Updated weights for policy 0, policy_version 269552 (0.0007) [2023-12-27 07:30:49,908][20347] Updated weights for policy 0, policy_version 269562 (0.0007) [2023-12-27 07:30:50,348][20381] Updated weights for policy 1, policy_version 269193 (0.0008) [2023-12-27 07:30:50,403][20381] Updated weights for policy 1, policy_version 269203 (0.0008) [2023-12-27 07:30:50,467][20381] Updated weights for policy 1, policy_version 269213 (0.0008) [2023-12-27 07:30:50,530][20381] Updated weights for policy 1, policy_version 269223 (0.0008) [2023-12-27 07:30:50,567][20347] Updated weights for policy 0, policy_version 269572 (0.0007) [2023-12-27 07:30:50,623][20347] Updated weights for policy 0, policy_version 269582 (0.0006) [2023-12-27 07:30:50,681][20347] Updated weights for policy 0, policy_version 269592 (0.0007) [2023-12-27 07:30:51,271][20381] Updated weights for policy 1, policy_version 269233 (0.0008) [2023-12-27 07:30:51,326][20381] Updated weights for policy 1, policy_version 269243 (0.0007) [2023-12-27 07:30:51,334][20347] Updated weights for policy 0, policy_version 269602 (0.0007) [2023-12-27 07:30:51,385][20381] Updated weights for policy 1, policy_version 269253 (0.0008) [2023-12-27 07:30:51,400][20347] Updated weights for policy 0, policy_version 269612 (0.0007) [2023-12-27 07:30:51,462][20347] Updated weights for policy 0, policy_version 269622 (0.0007) [2023-12-27 07:30:51,522][20347] Updated weights for policy 0, policy_version 269632 (0.0008) [2023-12-27 07:30:52,069][20381] Updated weights for policy 1, policy_version 269263 (0.0006) [2023-12-27 07:30:52,123][20381] Updated weights for policy 1, policy_version 269273 (0.0005) [2023-12-27 07:30:52,190][20381] Updated weights for policy 1, policy_version 269283 (0.0005) [2023-12-27 07:30:52,283][20347] Updated weights for policy 0, policy_version 269642 (0.0008) [2023-12-27 07:30:52,340][20347] Updated weights for policy 0, policy_version 269652 (0.0008) [2023-12-27 07:30:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 137994240. Throughput: 0: 10340.8, 1: 10411.6. Samples: 137966940. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:52,363][19170] Avg episode reward: [(0, '249.670'), (1, '953.340')] [2023-12-27 07:30:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000269288_68952064.pth... [2023-12-27 07:30:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000268040_68632576.pth [2023-12-27 07:30:52,400][20347] Updated weights for policy 0, policy_version 269662 (0.0008) [2023-12-27 07:30:52,408][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000269664_69050368.pth... [2023-12-27 07:30:52,411][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000268448_68739072.pth [2023-12-27 07:30:52,740][20381] Updated weights for policy 1, policy_version 269293 (0.0007) [2023-12-27 07:30:52,796][20381] Updated weights for policy 1, policy_version 269303 (0.0008) [2023-12-27 07:30:52,852][20381] Updated weights for policy 1, policy_version 269313 (0.0008) [2023-12-27 07:30:53,115][20347] Updated weights for policy 0, policy_version 269672 (0.0008) [2023-12-27 07:30:53,175][20347] Updated weights for policy 0, policy_version 269682 (0.0008) [2023-12-27 07:30:53,230][20347] Updated weights for policy 0, policy_version 269692 (0.0008) [2023-12-27 07:30:53,527][20381] Updated weights for policy 1, policy_version 269323 (0.0006) [2023-12-27 07:30:53,576][20381] Updated weights for policy 1, policy_version 269333 (0.0005) [2023-12-27 07:30:53,629][20381] Updated weights for policy 1, policy_version 269343 (0.0005) [2023-12-27 07:30:53,948][20347] Updated weights for policy 0, policy_version 269702 (0.0009) [2023-12-27 07:30:54,003][20347] Updated weights for policy 0, policy_version 269712 (0.0010) [2023-12-27 07:30:54,061][20347] Updated weights for policy 0, policy_version 269722 (0.0010) [2023-12-27 07:30:54,183][20381] Updated weights for policy 1, policy_version 269353 (0.0005) [2023-12-27 07:30:54,251][20381] Updated weights for policy 1, policy_version 269363 (0.0005) [2023-12-27 07:30:54,308][20381] Updated weights for policy 1, policy_version 269373 (0.0005) [2023-12-27 07:30:54,368][20381] Updated weights for policy 1, policy_version 269383 (0.0005) [2023-12-27 07:30:54,671][20347] Updated weights for policy 0, policy_version 269732 (0.0009) [2023-12-27 07:30:54,730][20347] Updated weights for policy 0, policy_version 269742 (0.0008) [2023-12-27 07:30:54,797][20347] Updated weights for policy 0, policy_version 269752 (0.0007) [2023-12-27 07:30:54,851][20381] Updated weights for policy 1, policy_version 269393 (0.0005) [2023-12-27 07:30:54,907][20381] Updated weights for policy 1, policy_version 269403 (0.0007) [2023-12-27 07:30:54,952][20381] Updated weights for policy 1, policy_version 269413 (0.0006) [2023-12-27 07:30:55,470][20347] Updated weights for policy 0, policy_version 269762 (0.0007) [2023-12-27 07:30:55,517][20347] Updated weights for policy 0, policy_version 269772 (0.0008) [2023-12-27 07:30:55,558][20381] Updated weights for policy 1, policy_version 269423 (0.0008) [2023-12-27 07:30:55,565][20347] Updated weights for policy 0, policy_version 269782 (0.0006) [2023-12-27 07:30:55,610][20381] Updated weights for policy 1, policy_version 269433 (0.0007) [2023-12-27 07:30:55,620][20347] Updated weights for policy 0, policy_version 269792 (0.0006) [2023-12-27 07:30:55,663][20381] Updated weights for policy 1, policy_version 269443 (0.0007) [2023-12-27 07:30:56,316][20347] Updated weights for policy 0, policy_version 269802 (0.0007) [2023-12-27 07:30:56,364][20381] Updated weights for policy 1, policy_version 269453 (0.0007) [2023-12-27 07:30:56,370][20347] Updated weights for policy 0, policy_version 269812 (0.0007) [2023-12-27 07:30:56,419][20381] Updated weights for policy 1, policy_version 269463 (0.0007) [2023-12-27 07:30:56,425][20347] Updated weights for policy 0, policy_version 269822 (0.0005) [2023-12-27 07:30:56,469][20381] Updated weights for policy 1, policy_version 269473 (0.0007) [2023-12-27 07:30:57,015][20381] Updated weights for policy 1, policy_version 269483 (0.0007) [2023-12-27 07:30:57,066][20381] Updated weights for policy 1, policy_version 269493 (0.0007) [2023-12-27 07:30:57,112][20381] Updated weights for policy 1, policy_version 269503 (0.0007) [2023-12-27 07:30:57,142][20347] Updated weights for policy 0, policy_version 269832 (0.0006) [2023-12-27 07:30:57,195][20347] Updated weights for policy 0, policy_version 269842 (0.0007) [2023-12-27 07:30:57,243][20347] Updated weights for policy 0, policy_version 269852 (0.0007) [2023-12-27 07:30:57,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 138108928. Throughput: 0: 10290.7, 1: 10502.8. Samples: 138093204. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:30:57,363][19170] Avg episode reward: [(0, '151.180'), (1, '990.500')] [2023-12-27 07:30:57,831][20347] Updated weights for policy 0, policy_version 269862 (0.0007) [2023-12-27 07:30:57,842][20381] Updated weights for policy 1, policy_version 269513 (0.0005) [2023-12-27 07:30:57,873][20347] Updated weights for policy 0, policy_version 269872 (0.0006) [2023-12-27 07:30:57,895][20381] Updated weights for policy 1, policy_version 269523 (0.0006) [2023-12-27 07:30:57,919][20347] Updated weights for policy 0, policy_version 269882 (0.0006) [2023-12-27 07:30:57,946][20381] Updated weights for policy 1, policy_version 269533 (0.0007) [2023-12-27 07:30:57,990][20381] Updated weights for policy 1, policy_version 269543 (0.0007) [2023-12-27 07:30:58,717][20347] Updated weights for policy 0, policy_version 269892 (0.0008) [2023-12-27 07:30:58,765][20381] Updated weights for policy 1, policy_version 269553 (0.0010) [2023-12-27 07:30:58,779][20347] Updated weights for policy 0, policy_version 269902 (0.0007) [2023-12-27 07:30:58,829][20381] Updated weights for policy 1, policy_version 269563 (0.0009) [2023-12-27 07:30:58,847][20347] Updated weights for policy 0, policy_version 269912 (0.0009) [2023-12-27 07:30:58,901][20381] Updated weights for policy 1, policy_version 269574 (0.0010) [2023-12-27 07:30:59,634][20381] Updated weights for policy 1, policy_version 269584 (0.0007) [2023-12-27 07:30:59,636][20347] Updated weights for policy 0, policy_version 269922 (0.0009) [2023-12-27 07:30:59,689][20381] Updated weights for policy 1, policy_version 269594 (0.0010) [2023-12-27 07:30:59,691][20347] Updated weights for policy 0, policy_version 269932 (0.0010) [2023-12-27 07:30:59,741][20381] Updated weights for policy 1, policy_version 269604 (0.0010) [2023-12-27 07:30:59,745][20347] Updated weights for policy 0, policy_version 269942 (0.0010) [2023-12-27 07:30:59,796][20347] Updated weights for policy 0, policy_version 269952 (0.0010) [2023-12-27 07:31:00,441][20381] Updated weights for policy 1, policy_version 269614 (0.0010) [2023-12-27 07:31:00,493][20381] Updated weights for policy 1, policy_version 269624 (0.0010) [2023-12-27 07:31:00,520][20347] Updated weights for policy 0, policy_version 269962 (0.0006) [2023-12-27 07:31:00,544][20381] Updated weights for policy 1, policy_version 269634 (0.0010) [2023-12-27 07:31:00,570][20347] Updated weights for policy 0, policy_version 269972 (0.0005) [2023-12-27 07:31:00,627][20347] Updated weights for policy 0, policy_version 269982 (0.0005) [2023-12-27 07:31:01,119][20381] Updated weights for policy 1, policy_version 269644 (0.0010) [2023-12-27 07:31:01,185][20381] Updated weights for policy 1, policy_version 269654 (0.0011) [2023-12-27 07:31:01,238][20381] Updated weights for policy 1, policy_version 269664 (0.0009) [2023-12-27 07:31:01,346][20347] Updated weights for policy 0, policy_version 269992 (0.0007) [2023-12-27 07:31:01,403][20347] Updated weights for policy 0, policy_version 270002 (0.0007) [2023-12-27 07:31:01,455][20347] Updated weights for policy 0, policy_version 270012 (0.0007) [2023-12-27 07:31:01,888][20381] Updated weights for policy 1, policy_version 269674 (0.0010) [2023-12-27 07:31:01,949][20381] Updated weights for policy 1, policy_version 269684 (0.0006) [2023-12-27 07:31:02,015][20381] Updated weights for policy 1, policy_version 269694 (0.0006) [2023-12-27 07:31:02,081][20381] Updated weights for policy 1, policy_version 269704 (0.0011) [2023-12-27 07:31:02,123][20347] Updated weights for policy 0, policy_version 270022 (0.0006) [2023-12-27 07:31:02,181][20347] Updated weights for policy 0, policy_version 270032 (0.0005) [2023-12-27 07:31:02,238][20347] Updated weights for policy 0, policy_version 270042 (0.0007) [2023-12-27 07:31:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 138207232. Throughput: 0: 10248.7, 1: 10561.7. Samples: 138215044. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:02,363][19170] Avg episode reward: [(0, '606.470'), (1, '971.260')] [2023-12-27 07:31:02,678][20381] Updated weights for policy 1, policy_version 269714 (0.0008) [2023-12-27 07:31:02,733][20381] Updated weights for policy 1, policy_version 269724 (0.0008) [2023-12-27 07:31:02,788][20381] Updated weights for policy 1, policy_version 269734 (0.0008) [2023-12-27 07:31:02,885][20347] Updated weights for policy 0, policy_version 270052 (0.0007) [2023-12-27 07:31:02,938][20347] Updated weights for policy 0, policy_version 270062 (0.0006) [2023-12-27 07:31:02,988][20347] Updated weights for policy 0, policy_version 270072 (0.0007) [2023-12-27 07:31:03,437][20381] Updated weights for policy 1, policy_version 269744 (0.0006) [2023-12-27 07:31:03,503][20381] Updated weights for policy 1, policy_version 269754 (0.0005) [2023-12-27 07:31:03,557][20381] Updated weights for policy 1, policy_version 269764 (0.0005) [2023-12-27 07:31:03,685][20347] Updated weights for policy 0, policy_version 270082 (0.0006) [2023-12-27 07:31:03,757][20347] Updated weights for policy 0, policy_version 270092 (0.0005) [2023-12-27 07:31:03,822][20347] Updated weights for policy 0, policy_version 270102 (0.0005) [2023-12-27 07:31:03,884][20347] Updated weights for policy 0, policy_version 270112 (0.0008) [2023-12-27 07:31:04,198][20381] Updated weights for policy 1, policy_version 269774 (0.0008) [2023-12-27 07:31:04,229][20292] KL-divergence is very high: 267.7444 [2023-12-27 07:31:04,236][20292] KL-divergence is very high: 291.0398 [2023-12-27 07:31:04,258][20381] Updated weights for policy 1, policy_version 269784 (0.0008) [2023-12-27 07:31:04,315][20381] Updated weights for policy 1, policy_version 269794 (0.0007) [2023-12-27 07:31:04,490][20347] Updated weights for policy 0, policy_version 270122 (0.0011) [2023-12-27 07:31:04,545][20347] Updated weights for policy 0, policy_version 270132 (0.0010) [2023-12-27 07:31:04,602][20347] Updated weights for policy 0, policy_version 270142 (0.0010) [2023-12-27 07:31:05,023][20381] Updated weights for policy 1, policy_version 269804 (0.0007) [2023-12-27 07:31:05,077][20381] Updated weights for policy 1, policy_version 269814 (0.0007) [2023-12-27 07:31:05,127][20381] Updated weights for policy 1, policy_version 269824 (0.0007) [2023-12-27 07:31:05,288][20347] Updated weights for policy 0, policy_version 270152 (0.0010) [2023-12-27 07:31:05,340][20347] Updated weights for policy 0, policy_version 270162 (0.0010) [2023-12-27 07:31:05,396][20347] Updated weights for policy 0, policy_version 270172 (0.0010) [2023-12-27 07:31:05,839][20381] Updated weights for policy 1, policy_version 269834 (0.0007) [2023-12-27 07:31:05,889][20381] Updated weights for policy 1, policy_version 269844 (0.0007) [2023-12-27 07:31:05,943][20381] Updated weights for policy 1, policy_version 269854 (0.0007) [2023-12-27 07:31:06,003][20381] Updated weights for policy 1, policy_version 269864 (0.0007) [2023-12-27 07:31:06,053][20347] Updated weights for policy 0, policy_version 270182 (0.0010) [2023-12-27 07:31:06,106][20347] Updated weights for policy 0, policy_version 270192 (0.0010) [2023-12-27 07:31:06,172][20347] Updated weights for policy 0, policy_version 270202 (0.0007) [2023-12-27 07:31:06,721][20381] Updated weights for policy 1, policy_version 269874 (0.0007) [2023-12-27 07:31:06,777][20381] Updated weights for policy 1, policy_version 269884 (0.0007) [2023-12-27 07:31:06,815][20347] Updated weights for policy 0, policy_version 270212 (0.0009) [2023-12-27 07:31:06,831][20381] Updated weights for policy 1, policy_version 269894 (0.0007) [2023-12-27 07:31:06,871][20347] Updated weights for policy 0, policy_version 270222 (0.0010) [2023-12-27 07:31:06,933][20347] Updated weights for policy 0, policy_version 270232 (0.0011) [2023-12-27 07:31:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 138305536. Throughput: 0: 10308.7, 1: 10574.7. Samples: 138277352. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:07,363][19170] Avg episode reward: [(0, '674.340'), (1, '971.220')] [2023-12-27 07:31:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000269896_69107712.pth... [2023-12-27 07:31:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000270240_69197824.pth... [2023-12-27 07:31:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000268680_68796416.pth [2023-12-27 07:31:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000269056_68894720.pth [2023-12-27 07:31:07,547][20381] Updated weights for policy 1, policy_version 269904 (0.0007) [2023-12-27 07:31:07,584][20347] Updated weights for policy 0, policy_version 270242 (0.0010) [2023-12-27 07:31:07,599][20381] Updated weights for policy 1, policy_version 269914 (0.0008) [2023-12-27 07:31:07,641][20347] Updated weights for policy 0, policy_version 270252 (0.0010) [2023-12-27 07:31:07,647][20381] Updated weights for policy 1, policy_version 269924 (0.0005) [2023-12-27 07:31:07,702][20347] Updated weights for policy 0, policy_version 270262 (0.0010) [2023-12-27 07:31:07,753][20347] Updated weights for policy 0, policy_version 270272 (0.0010) [2023-12-27 07:31:08,374][20347] Updated weights for policy 0, policy_version 270282 (0.0008) [2023-12-27 07:31:08,386][20381] Updated weights for policy 1, policy_version 269934 (0.0008) [2023-12-27 07:31:08,427][20347] Updated weights for policy 0, policy_version 270292 (0.0010) [2023-12-27 07:31:08,448][20381] Updated weights for policy 1, policy_version 269944 (0.0007) [2023-12-27 07:31:08,483][20347] Updated weights for policy 0, policy_version 270302 (0.0010) [2023-12-27 07:31:08,504][20381] Updated weights for policy 1, policy_version 269954 (0.0006) [2023-12-27 07:31:09,138][20347] Updated weights for policy 0, policy_version 270312 (0.0008) [2023-12-27 07:31:09,200][20347] Updated weights for policy 0, policy_version 270322 (0.0008) [2023-12-27 07:31:09,220][20381] Updated weights for policy 1, policy_version 269964 (0.0008) [2023-12-27 07:31:09,269][20347] Updated weights for policy 0, policy_version 270332 (0.0008) [2023-12-27 07:31:09,281][20381] Updated weights for policy 1, policy_version 269974 (0.0007) [2023-12-27 07:31:09,339][20381] Updated weights for policy 1, policy_version 269984 (0.0008) [2023-12-27 07:31:09,903][20347] Updated weights for policy 0, policy_version 270342 (0.0005) [2023-12-27 07:31:09,971][20347] Updated weights for policy 0, policy_version 270352 (0.0007) [2023-12-27 07:31:10,028][20347] Updated weights for policy 0, policy_version 270362 (0.0008) [2023-12-27 07:31:10,057][20381] Updated weights for policy 1, policy_version 269994 (0.0006) [2023-12-27 07:31:10,121][20381] Updated weights for policy 1, policy_version 270004 (0.0010) [2023-12-27 07:31:10,174][20381] Updated weights for policy 1, policy_version 270014 (0.0011) [2023-12-27 07:31:10,231][20381] Updated weights for policy 1, policy_version 270024 (0.0010) [2023-12-27 07:31:10,730][20347] Updated weights for policy 0, policy_version 270372 (0.0007) [2023-12-27 07:31:10,791][20347] Updated weights for policy 0, policy_version 270382 (0.0005) [2023-12-27 07:31:10,840][20347] Updated weights for policy 0, policy_version 270392 (0.0007) [2023-12-27 07:31:10,930][20381] Updated weights for policy 1, policy_version 270034 (0.0007) [2023-12-27 07:31:10,979][20381] Updated weights for policy 1, policy_version 270044 (0.0007) [2023-12-27 07:31:11,036][20381] Updated weights for policy 1, policy_version 270054 (0.0006) [2023-12-27 07:31:11,530][20347] Updated weights for policy 0, policy_version 270402 (0.0008) [2023-12-27 07:31:11,591][20347] Updated weights for policy 0, policy_version 270412 (0.0008) [2023-12-27 07:31:11,656][20347] Updated weights for policy 0, policy_version 270422 (0.0008) [2023-12-27 07:31:11,719][20347] Updated weights for policy 0, policy_version 270432 (0.0007) [2023-12-27 07:31:11,807][20381] Updated weights for policy 1, policy_version 270064 (0.0008) [2023-12-27 07:31:11,862][20381] Updated weights for policy 1, policy_version 270074 (0.0008) [2023-12-27 07:31:11,910][20381] Updated weights for policy 1, policy_version 270084 (0.0008) [2023-12-27 07:31:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 138403840. Throughput: 0: 10365.1, 1: 10502.1. Samples: 138398672. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:12,363][19170] Avg episode reward: [(0, '633.670'), (1, '982.240')] [2023-12-27 07:31:12,398][20347] Updated weights for policy 0, policy_version 270442 (0.0008) [2023-12-27 07:31:12,463][20347] Updated weights for policy 0, policy_version 270452 (0.0008) [2023-12-27 07:31:12,522][20347] Updated weights for policy 0, policy_version 270462 (0.0007) [2023-12-27 07:31:12,668][20381] Updated weights for policy 1, policy_version 270094 (0.0007) [2023-12-27 07:31:12,726][20381] Updated weights for policy 1, policy_version 270104 (0.0008) [2023-12-27 07:31:12,785][20381] Updated weights for policy 1, policy_version 270114 (0.0007) [2023-12-27 07:31:13,159][20347] Updated weights for policy 0, policy_version 270472 (0.0007) [2023-12-27 07:31:13,215][20347] Updated weights for policy 0, policy_version 270482 (0.0005) [2023-12-27 07:31:13,271][20347] Updated weights for policy 0, policy_version 270492 (0.0005) [2023-12-27 07:31:13,500][20381] Updated weights for policy 1, policy_version 270124 (0.0008) [2023-12-27 07:31:13,562][20381] Updated weights for policy 1, policy_version 270134 (0.0008) [2023-12-27 07:31:13,614][20381] Updated weights for policy 1, policy_version 270144 (0.0008) [2023-12-27 07:31:13,881][20347] Updated weights for policy 0, policy_version 270502 (0.0005) [2023-12-27 07:31:13,935][20347] Updated weights for policy 0, policy_version 270512 (0.0007) [2023-12-27 07:31:13,987][20347] Updated weights for policy 0, policy_version 270522 (0.0008) [2023-12-27 07:31:14,280][20381] Updated weights for policy 1, policy_version 270154 (0.0007) [2023-12-27 07:31:14,335][20381] Updated weights for policy 1, policy_version 270164 (0.0009) [2023-12-27 07:31:14,383][20381] Updated weights for policy 1, policy_version 270174 (0.0008) [2023-12-27 07:31:14,452][20381] Updated weights for policy 1, policy_version 270184 (0.0008) [2023-12-27 07:31:14,612][20347] Updated weights for policy 0, policy_version 270532 (0.0008) [2023-12-27 07:31:14,668][20347] Updated weights for policy 0, policy_version 270542 (0.0008) [2023-12-27 07:31:14,723][20347] Updated weights for policy 0, policy_version 270552 (0.0008) [2023-12-27 07:31:15,164][20381] Updated weights for policy 1, policy_version 270194 (0.0007) [2023-12-27 07:31:15,227][20381] Updated weights for policy 1, policy_version 270204 (0.0008) [2023-12-27 07:31:15,294][20381] Updated weights for policy 1, policy_version 270214 (0.0008) [2023-12-27 07:31:15,455][20347] Updated weights for policy 0, policy_version 270562 (0.0008) [2023-12-27 07:31:15,512][20347] Updated weights for policy 0, policy_version 270572 (0.0008) [2023-12-27 07:31:15,570][20347] Updated weights for policy 0, policy_version 270582 (0.0008) [2023-12-27 07:31:15,627][20347] Updated weights for policy 0, policy_version 270592 (0.0008) [2023-12-27 07:31:15,973][20381] Updated weights for policy 1, policy_version 270224 (0.0007) [2023-12-27 07:31:16,027][20381] Updated weights for policy 1, policy_version 270234 (0.0007) [2023-12-27 07:31:16,083][20381] Updated weights for policy 1, policy_version 270244 (0.0007) [2023-12-27 07:31:16,325][20347] Updated weights for policy 0, policy_version 270602 (0.0010) [2023-12-27 07:31:16,371][20347] Updated weights for policy 0, policy_version 270612 (0.0010) [2023-12-27 07:31:16,427][20347] Updated weights for policy 0, policy_version 270622 (0.0007) [2023-12-27 07:31:16,723][20381] Updated weights for policy 1, policy_version 270254 (0.0007) [2023-12-27 07:31:16,778][20381] Updated weights for policy 1, policy_version 270264 (0.0007) [2023-12-27 07:31:16,826][20381] Updated weights for policy 1, policy_version 270274 (0.0008) [2023-12-27 07:31:17,017][20347] Updated weights for policy 0, policy_version 270632 (0.0008) [2023-12-27 07:31:17,079][20347] Updated weights for policy 0, policy_version 270642 (0.0006) [2023-12-27 07:31:17,135][20347] Updated weights for policy 0, policy_version 270652 (0.0005) [2023-12-27 07:31:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 138510336. Throughput: 0: 10352.3, 1: 10463.4. Samples: 138521168. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:17,363][19170] Avg episode reward: [(0, '710.990'), (1, '982.270')] [2023-12-27 07:31:17,592][20381] Updated weights for policy 1, policy_version 270284 (0.0008) [2023-12-27 07:31:17,648][20381] Updated weights for policy 1, policy_version 270294 (0.0007) [2023-12-27 07:31:17,702][20381] Updated weights for policy 1, policy_version 270304 (0.0008) [2023-12-27 07:31:17,750][20347] Updated weights for policy 0, policy_version 270662 (0.0009) [2023-12-27 07:31:17,811][20347] Updated weights for policy 0, policy_version 270672 (0.0008) [2023-12-27 07:31:17,867][20347] Updated weights for policy 0, policy_version 270682 (0.0005) [2023-12-27 07:31:18,389][20381] Updated weights for policy 1, policy_version 270314 (0.0008) [2023-12-27 07:31:18,449][20381] Updated weights for policy 1, policy_version 270324 (0.0009) [2023-12-27 07:31:18,469][20347] Updated weights for policy 0, policy_version 270692 (0.0005) [2023-12-27 07:31:18,507][20381] Updated weights for policy 1, policy_version 270334 (0.0007) [2023-12-27 07:31:18,528][20347] Updated weights for policy 0, policy_version 270702 (0.0006) [2023-12-27 07:31:18,563][20381] Updated weights for policy 1, policy_version 270344 (0.0007) [2023-12-27 07:31:18,576][20347] Updated weights for policy 0, policy_version 270712 (0.0006) [2023-12-27 07:31:19,118][20347] Updated weights for policy 0, policy_version 270722 (0.0005) [2023-12-27 07:31:19,174][20347] Updated weights for policy 0, policy_version 270732 (0.0005) [2023-12-27 07:31:19,233][20347] Updated weights for policy 0, policy_version 270742 (0.0009) [2023-12-27 07:31:19,292][20347] Updated weights for policy 0, policy_version 270752 (0.0007) [2023-12-27 07:31:19,331][20381] Updated weights for policy 1, policy_version 270354 (0.0007) [2023-12-27 07:31:19,391][20381] Updated weights for policy 1, policy_version 270364 (0.0008) [2023-12-27 07:31:19,448][20381] Updated weights for policy 1, policy_version 270374 (0.0008) [2023-12-27 07:31:19,927][20347] Updated weights for policy 0, policy_version 270762 (0.0007) [2023-12-27 07:31:19,982][20347] Updated weights for policy 0, policy_version 270772 (0.0010) [2023-12-27 07:31:20,044][20347] Updated weights for policy 0, policy_version 270782 (0.0009) [2023-12-27 07:31:20,113][20292] KL-divergence is very high: 114.6103 [2023-12-27 07:31:20,135][20381] Updated weights for policy 1, policy_version 270384 (0.0010) [2023-12-27 07:31:20,194][20381] Updated weights for policy 1, policy_version 270394 (0.0011) [2023-12-27 07:31:20,223][20292] KL-divergence is very high: 149.0764 [2023-12-27 07:31:20,244][20292] KL-divergence is very high: 132.6403 [2023-12-27 07:31:20,245][20381] Updated weights for policy 1, policy_version 270404 (0.0010) [2023-12-27 07:31:20,703][20347] Updated weights for policy 0, policy_version 270792 (0.0008) [2023-12-27 07:31:20,762][20347] Updated weights for policy 0, policy_version 270802 (0.0008) [2023-12-27 07:31:20,814][20347] Updated weights for policy 0, policy_version 270812 (0.0009) [2023-12-27 07:31:20,880][20381] Updated weights for policy 1, policy_version 270414 (0.0010) [2023-12-27 07:31:20,938][20381] Updated weights for policy 1, policy_version 270424 (0.0007) [2023-12-27 07:31:20,995][20381] Updated weights for policy 1, policy_version 270434 (0.0009) [2023-12-27 07:31:21,557][20347] Updated weights for policy 0, policy_version 270822 (0.0006) [2023-12-27 07:31:21,620][20347] Updated weights for policy 0, policy_version 270832 (0.0011) [2023-12-27 07:31:21,688][20347] Updated weights for policy 0, policy_version 270842 (0.0007) [2023-12-27 07:31:21,730][20381] Updated weights for policy 1, policy_version 270444 (0.0009) [2023-12-27 07:31:21,790][20381] Updated weights for policy 1, policy_version 270454 (0.0006) [2023-12-27 07:31:21,842][20381] Updated weights for policy 1, policy_version 270464 (0.0005) [2023-12-27 07:31:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 138608640. Throughput: 0: 10408.9, 1: 10355.2. Samples: 138583504. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:22,363][19170] Avg episode reward: [(0, '787.810'), (1, '961.580')] [2023-12-27 07:31:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000270472_69255168.pth... [2023-12-27 07:31:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000270848_69353472.pth... [2023-12-27 07:31:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000269288_68952064.pth [2023-12-27 07:31:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000269664_69050368.pth [2023-12-27 07:31:22,423][20347] Updated weights for policy 0, policy_version 270852 (0.0008) [2023-12-27 07:31:22,425][20381] Updated weights for policy 1, policy_version 270474 (0.0005) [2023-12-27 07:31:22,474][20347] Updated weights for policy 0, policy_version 270862 (0.0008) [2023-12-27 07:31:22,482][20381] Updated weights for policy 1, policy_version 270484 (0.0005) [2023-12-27 07:31:22,522][20347] Updated weights for policy 0, policy_version 270872 (0.0007) [2023-12-27 07:31:22,537][20381] Updated weights for policy 1, policy_version 270494 (0.0006) [2023-12-27 07:31:22,593][20381] Updated weights for policy 1, policy_version 270504 (0.0006) [2023-12-27 07:31:23,170][20347] Updated weights for policy 0, policy_version 270882 (0.0007) [2023-12-27 07:31:23,224][20347] Updated weights for policy 0, policy_version 270892 (0.0008) [2023-12-27 07:31:23,275][20347] Updated weights for policy 0, policy_version 270902 (0.0007) [2023-12-27 07:31:23,277][20381] Updated weights for policy 1, policy_version 270514 (0.0006) [2023-12-27 07:31:23,325][20347] Updated weights for policy 0, policy_version 270912 (0.0006) [2023-12-27 07:31:23,337][20381] Updated weights for policy 1, policy_version 270524 (0.0006) [2023-12-27 07:31:23,390][20381] Updated weights for policy 1, policy_version 270534 (0.0008) [2023-12-27 07:31:23,975][20347] Updated weights for policy 0, policy_version 270922 (0.0005) [2023-12-27 07:31:24,026][20347] Updated weights for policy 0, policy_version 270932 (0.0005) [2023-12-27 07:31:24,073][20381] Updated weights for policy 1, policy_version 270544 (0.0008) [2023-12-27 07:31:24,083][20347] Updated weights for policy 0, policy_version 270942 (0.0005) [2023-12-27 07:31:24,124][20381] Updated weights for policy 1, policy_version 270554 (0.0009) [2023-12-27 07:31:24,181][20381] Updated weights for policy 1, policy_version 270564 (0.0007) [2023-12-27 07:31:24,708][20347] Updated weights for policy 0, policy_version 270952 (0.0007) [2023-12-27 07:31:24,760][20347] Updated weights for policy 0, policy_version 270962 (0.0007) [2023-12-27 07:31:24,808][20347] Updated weights for policy 0, policy_version 270972 (0.0008) [2023-12-27 07:31:24,856][20381] Updated weights for policy 1, policy_version 270574 (0.0006) [2023-12-27 07:31:24,923][20381] Updated weights for policy 1, policy_version 270584 (0.0005) [2023-12-27 07:31:24,985][20381] Updated weights for policy 1, policy_version 270594 (0.0006) [2023-12-27 07:31:25,533][20347] Updated weights for policy 0, policy_version 270982 (0.0006) [2023-12-27 07:31:25,536][20381] Updated weights for policy 1, policy_version 270604 (0.0006) [2023-12-27 07:31:25,592][20347] Updated weights for policy 0, policy_version 270992 (0.0006) [2023-12-27 07:31:25,594][20381] Updated weights for policy 1, policy_version 270614 (0.0005) [2023-12-27 07:31:25,644][20347] Updated weights for policy 0, policy_version 271002 (0.0005) [2023-12-27 07:31:25,646][20381] Updated weights for policy 1, policy_version 270624 (0.0006) [2023-12-27 07:31:26,292][20347] Updated weights for policy 0, policy_version 271012 (0.0006) [2023-12-27 07:31:26,336][20381] Updated weights for policy 1, policy_version 270634 (0.0006) [2023-12-27 07:31:26,348][20347] Updated weights for policy 0, policy_version 271022 (0.0008) [2023-12-27 07:31:26,397][20381] Updated weights for policy 1, policy_version 270644 (0.0007) [2023-12-27 07:31:26,403][20347] Updated weights for policy 0, policy_version 271032 (0.0008) [2023-12-27 07:31:26,450][20381] Updated weights for policy 1, policy_version 270654 (0.0007) [2023-12-27 07:31:26,503][20381] Updated weights for policy 1, policy_version 270664 (0.0007) [2023-12-27 07:31:27,074][20347] Updated weights for policy 0, policy_version 271042 (0.0006) [2023-12-27 07:31:27,132][20347] Updated weights for policy 0, policy_version 271052 (0.0007) [2023-12-27 07:31:27,178][20347] Updated weights for policy 0, policy_version 271062 (0.0006) [2023-12-27 07:31:27,184][20381] Updated weights for policy 1, policy_version 270674 (0.0006) [2023-12-27 07:31:27,232][20347] Updated weights for policy 0, policy_version 271072 (0.0007) [2023-12-27 07:31:27,234][20381] Updated weights for policy 1, policy_version 270684 (0.0006) [2023-12-27 07:31:27,285][20381] Updated weights for policy 1, policy_version 270694 (0.0007) [2023-12-27 07:31:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 138723328. Throughput: 0: 10382.3, 1: 10292.1. Samples: 138707964. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:27,363][19170] Avg episode reward: [(0, '756.410'), (1, '980.670')] [2023-12-27 07:31:27,906][20347] Updated weights for policy 0, policy_version 271082 (0.0007) [2023-12-27 07:31:27,963][20347] Updated weights for policy 0, policy_version 271092 (0.0007) [2023-12-27 07:31:27,976][20381] Updated weights for policy 1, policy_version 270704 (0.0006) [2023-12-27 07:31:28,017][20347] Updated weights for policy 0, policy_version 271102 (0.0006) [2023-12-27 07:31:28,029][20381] Updated weights for policy 1, policy_version 270714 (0.0006) [2023-12-27 07:31:28,082][20381] Updated weights for policy 1, policy_version 270724 (0.0007) [2023-12-27 07:31:28,694][20381] Updated weights for policy 1, policy_version 270734 (0.0007) [2023-12-27 07:31:28,734][20347] Updated weights for policy 0, policy_version 271112 (0.0006) [2023-12-27 07:31:28,750][20381] Updated weights for policy 1, policy_version 270744 (0.0006) [2023-12-27 07:31:28,778][20347] Updated weights for policy 0, policy_version 271122 (0.0007) [2023-12-27 07:31:28,800][20381] Updated weights for policy 1, policy_version 270754 (0.0007) [2023-12-27 07:31:28,825][20347] Updated weights for policy 0, policy_version 271132 (0.0006) [2023-12-27 07:31:29,446][20381] Updated weights for policy 1, policy_version 270764 (0.0007) [2023-12-27 07:31:29,497][20381] Updated weights for policy 1, policy_version 270774 (0.0007) [2023-12-27 07:31:29,552][20381] Updated weights for policy 1, policy_version 270784 (0.0006) [2023-12-27 07:31:29,575][20347] Updated weights for policy 0, policy_version 271142 (0.0008) [2023-12-27 07:31:29,637][20347] Updated weights for policy 0, policy_version 271152 (0.0008) [2023-12-27 07:31:29,698][20347] Updated weights for policy 0, policy_version 271162 (0.0008) [2023-12-27 07:31:30,226][20381] Updated weights for policy 1, policy_version 270794 (0.0006) [2023-12-27 07:31:30,289][20381] Updated weights for policy 1, policy_version 270804 (0.0008) [2023-12-27 07:31:30,351][20381] Updated weights for policy 1, policy_version 270814 (0.0006) [2023-12-27 07:31:30,402][20381] Updated weights for policy 1, policy_version 270824 (0.0007) [2023-12-27 07:31:30,426][20347] Updated weights for policy 0, policy_version 271172 (0.0010) [2023-12-27 07:31:30,475][20347] Updated weights for policy 0, policy_version 271182 (0.0010) [2023-12-27 07:31:30,533][20347] Updated weights for policy 0, policy_version 271192 (0.0010) [2023-12-27 07:31:31,033][20381] Updated weights for policy 1, policy_version 270834 (0.0006) [2023-12-27 07:31:31,101][20381] Updated weights for policy 1, policy_version 270844 (0.0008) [2023-12-27 07:31:31,158][20381] Updated weights for policy 1, policy_version 270854 (0.0008) [2023-12-27 07:31:31,205][20347] Updated weights for policy 0, policy_version 271202 (0.0010) [2023-12-27 07:31:31,256][20347] Updated weights for policy 0, policy_version 271212 (0.0010) [2023-12-27 07:31:31,310][20347] Updated weights for policy 0, policy_version 271222 (0.0006) [2023-12-27 07:31:31,372][20347] Updated weights for policy 0, policy_version 271232 (0.0007) [2023-12-27 07:31:31,857][20381] Updated weights for policy 1, policy_version 270864 (0.0008) [2023-12-27 07:31:31,918][20381] Updated weights for policy 1, policy_version 270874 (0.0010) [2023-12-27 07:31:31,966][20381] Updated weights for policy 1, policy_version 270884 (0.0010) [2023-12-27 07:31:31,997][20347] Updated weights for policy 0, policy_version 271242 (0.0010) [2023-12-27 07:31:32,053][20347] Updated weights for policy 0, policy_version 271252 (0.0010) [2023-12-27 07:31:32,109][20347] Updated weights for policy 0, policy_version 271262 (0.0010) [2023-12-27 07:31:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 138821632. Throughput: 0: 10348.4, 1: 10279.9. Samples: 138830680. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 07:31:32,363][19170] Avg episode reward: [(0, '764.930'), (1, '1000.000')] [2023-12-27 07:31:32,673][20381] Updated weights for policy 1, policy_version 270894 (0.0010) [2023-12-27 07:31:32,731][20381] Updated weights for policy 1, policy_version 270904 (0.0010) [2023-12-27 07:31:32,781][20381] Updated weights for policy 1, policy_version 270914 (0.0010) [2023-12-27 07:31:32,819][20347] Updated weights for policy 0, policy_version 271272 (0.0007) [2023-12-27 07:31:32,877][20347] Updated weights for policy 0, policy_version 271282 (0.0007) [2023-12-27 07:31:32,928][20347] Updated weights for policy 0, policy_version 271292 (0.0007) [2023-12-27 07:31:33,410][20381] Updated weights for policy 1, policy_version 270924 (0.0010) [2023-12-27 07:31:33,462][20381] Updated weights for policy 1, policy_version 270934 (0.0008) [2023-12-27 07:31:33,493][20347] Updated weights for policy 0, policy_version 271302 (0.0007) [2023-12-27 07:31:33,517][20381] Updated weights for policy 1, policy_version 270944 (0.0005) [2023-12-27 07:31:33,550][20347] Updated weights for policy 0, policy_version 271312 (0.0007) [2023-12-27 07:31:33,593][20347] Updated weights for policy 0, policy_version 271322 (0.0007) [2023-12-27 07:31:34,167][20381] Updated weights for policy 1, policy_version 270954 (0.0006) [2023-12-27 07:31:34,225][20381] Updated weights for policy 1, policy_version 270964 (0.0007) [2023-12-27 07:31:34,266][20347] Updated weights for policy 0, policy_version 271332 (0.0007) [2023-12-27 07:31:34,287][20381] Updated weights for policy 1, policy_version 270974 (0.0007) [2023-12-27 07:31:34,321][20347] Updated weights for policy 0, policy_version 271342 (0.0009) [2023-12-27 07:31:34,344][20381] Updated weights for policy 1, policy_version 270984 (0.0006) [2023-12-27 07:31:34,387][20347] Updated weights for policy 0, policy_version 271352 (0.0009) [2023-12-27 07:31:35,028][20381] Updated weights for policy 1, policy_version 270994 (0.0008) [2023-12-27 07:31:35,071][20347] Updated weights for policy 0, policy_version 271362 (0.0008) [2023-12-27 07:31:35,080][20381] Updated weights for policy 1, policy_version 271004 (0.0007) [2023-12-27 07:31:35,122][20347] Updated weights for policy 0, policy_version 271372 (0.0010) [2023-12-27 07:31:35,142][20381] Updated weights for policy 1, policy_version 271014 (0.0005) [2023-12-27 07:31:35,180][20347] Updated weights for policy 0, policy_version 271382 (0.0010) [2023-12-27 07:31:35,239][20347] Updated weights for policy 0, policy_version 271392 (0.0010) [2023-12-27 07:31:35,861][20381] Updated weights for policy 1, policy_version 271024 (0.0006) [2023-12-27 07:31:35,911][20381] Updated weights for policy 1, policy_version 271034 (0.0006) [2023-12-27 07:31:35,920][20347] Updated weights for policy 0, policy_version 271402 (0.0007) [2023-12-27 07:31:35,960][20381] Updated weights for policy 1, policy_version 271044 (0.0005) [2023-12-27 07:31:35,965][20347] Updated weights for policy 0, policy_version 271412 (0.0007) [2023-12-27 07:31:36,014][20347] Updated weights for policy 0, policy_version 271422 (0.0007) [2023-12-27 07:31:36,689][20381] Updated weights for policy 1, policy_version 271054 (0.0006) [2023-12-27 07:31:36,731][20347] Updated weights for policy 0, policy_version 271432 (0.0009) [2023-12-27 07:31:36,747][20381] Updated weights for policy 1, policy_version 271064 (0.0007) [2023-12-27 07:31:36,786][20347] Updated weights for policy 0, policy_version 271442 (0.0010) [2023-12-27 07:31:36,800][20381] Updated weights for policy 1, policy_version 271074 (0.0005) [2023-12-27 07:31:36,835][20347] Updated weights for policy 0, policy_version 271452 (0.0010) [2023-12-27 07:31:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 138919936. Throughput: 0: 10321.0, 1: 10272.4. Samples: 138893644. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:31:37,363][19170] Avg episode reward: [(0, '707.840'), (1, '1000.000')] [2023-12-27 07:31:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000271080_69410816.pth... [2023-12-27 07:31:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000271456_69509120.pth... [2023-12-27 07:31:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000269896_69107712.pth [2023-12-27 07:31:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000270240_69197824.pth [2023-12-27 07:31:37,492][20381] Updated weights for policy 1, policy_version 271084 (0.0006) [2023-12-27 07:31:37,536][20347] Updated weights for policy 0, policy_version 271462 (0.0010) [2023-12-27 07:31:37,545][20381] Updated weights for policy 1, policy_version 271094 (0.0005) [2023-12-27 07:31:37,591][20347] Updated weights for policy 0, policy_version 271472 (0.0010) [2023-12-27 07:31:37,600][20381] Updated weights for policy 1, policy_version 271104 (0.0005) [2023-12-27 07:31:37,645][20347] Updated weights for policy 0, policy_version 271482 (0.0010) [2023-12-27 07:31:38,299][20381] Updated weights for policy 1, policy_version 271114 (0.0006) [2023-12-27 07:31:38,325][20347] Updated weights for policy 0, policy_version 271492 (0.0010) [2023-12-27 07:31:38,360][20381] Updated weights for policy 1, policy_version 271124 (0.0008) [2023-12-27 07:31:38,390][20347] Updated weights for policy 0, policy_version 271502 (0.0011) [2023-12-27 07:31:38,414][20381] Updated weights for policy 1, policy_version 271134 (0.0006) [2023-12-27 07:31:38,446][20347] Updated weights for policy 0, policy_version 271512 (0.0010) [2023-12-27 07:31:38,467][20381] Updated weights for policy 1, policy_version 271144 (0.0005) [2023-12-27 07:31:39,130][20347] Updated weights for policy 0, policy_version 271522 (0.0010) [2023-12-27 07:31:39,175][20381] Updated weights for policy 1, policy_version 271154 (0.0006) [2023-12-27 07:31:39,183][20347] Updated weights for policy 0, policy_version 271532 (0.0010) [2023-12-27 07:31:39,237][20381] Updated weights for policy 1, policy_version 271164 (0.0007) [2023-12-27 07:31:39,239][20347] Updated weights for policy 0, policy_version 271542 (0.0010) [2023-12-27 07:31:39,298][20381] Updated weights for policy 1, policy_version 271174 (0.0006) [2023-12-27 07:31:39,304][20347] Updated weights for policy 0, policy_version 271552 (0.0011) [2023-12-27 07:31:40,015][20347] Updated weights for policy 0, policy_version 271562 (0.0008) [2023-12-27 07:31:40,021][20381] Updated weights for policy 1, policy_version 271184 (0.0005) [2023-12-27 07:31:40,067][20347] Updated weights for policy 0, policy_version 271572 (0.0008) [2023-12-27 07:31:40,081][20381] Updated weights for policy 1, policy_version 271194 (0.0005) [2023-12-27 07:31:40,120][20347] Updated weights for policy 0, policy_version 271582 (0.0007) [2023-12-27 07:31:40,140][20381] Updated weights for policy 1, policy_version 271204 (0.0006) [2023-12-27 07:31:40,842][20347] Updated weights for policy 0, policy_version 271592 (0.0009) [2023-12-27 07:31:40,848][20381] Updated weights for policy 1, policy_version 271214 (0.0007) [2023-12-27 07:31:40,891][20347] Updated weights for policy 0, policy_version 271602 (0.0008) [2023-12-27 07:31:40,903][20381] Updated weights for policy 1, policy_version 271224 (0.0005) [2023-12-27 07:31:40,949][20347] Updated weights for policy 0, policy_version 271612 (0.0007) [2023-12-27 07:31:40,961][20381] Updated weights for policy 1, policy_version 271234 (0.0005) [2023-12-27 07:31:41,666][20381] Updated weights for policy 1, policy_version 271244 (0.0007) [2023-12-27 07:31:41,682][20347] Updated weights for policy 0, policy_version 271622 (0.0010) [2023-12-27 07:31:41,729][20381] Updated weights for policy 1, policy_version 271254 (0.0006) [2023-12-27 07:31:41,746][20347] Updated weights for policy 0, policy_version 271632 (0.0011) [2023-12-27 07:31:41,796][20381] Updated weights for policy 1, policy_version 271264 (0.0009) [2023-12-27 07:31:41,806][20347] Updated weights for policy 0, policy_version 271642 (0.0010) [2023-12-27 07:31:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 139018240. Throughput: 0: 10294.1, 1: 10126.7. Samples: 139012140. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:31:42,363][19170] Avg episode reward: [(0, '680.320'), (1, '1000.000')] [2023-12-27 07:31:42,459][20347] Updated weights for policy 0, policy_version 271652 (0.0005) [2023-12-27 07:31:42,493][20381] Updated weights for policy 1, policy_version 271274 (0.0007) [2023-12-27 07:31:42,516][20347] Updated weights for policy 0, policy_version 271662 (0.0008) [2023-12-27 07:31:42,551][20381] Updated weights for policy 1, policy_version 271284 (0.0007) [2023-12-27 07:31:42,564][20347] Updated weights for policy 0, policy_version 271672 (0.0005) [2023-12-27 07:31:42,606][20381] Updated weights for policy 1, policy_version 271294 (0.0008) [2023-12-27 07:31:42,666][20381] Updated weights for policy 1, policy_version 271304 (0.0009) [2023-12-27 07:31:43,210][20347] Updated weights for policy 0, policy_version 271682 (0.0006) [2023-12-27 07:31:43,264][20347] Updated weights for policy 0, policy_version 271692 (0.0007) [2023-12-27 07:31:43,321][20347] Updated weights for policy 0, policy_version 271702 (0.0006) [2023-12-27 07:31:43,348][20381] Updated weights for policy 1, policy_version 271314 (0.0006) [2023-12-27 07:31:43,369][20347] Updated weights for policy 0, policy_version 271712 (0.0006) [2023-12-27 07:31:43,413][20381] Updated weights for policy 1, policy_version 271324 (0.0007) [2023-12-27 07:31:43,466][20381] Updated weights for policy 1, policy_version 271334 (0.0009) [2023-12-27 07:31:43,984][20347] Updated weights for policy 0, policy_version 271722 (0.0007) [2023-12-27 07:31:44,033][20347] Updated weights for policy 0, policy_version 271732 (0.0007) [2023-12-27 07:31:44,082][20347] Updated weights for policy 0, policy_version 271742 (0.0008) [2023-12-27 07:31:44,146][20381] Updated weights for policy 1, policy_version 271344 (0.0008) [2023-12-27 07:31:44,202][20381] Updated weights for policy 1, policy_version 271354 (0.0008) [2023-12-27 07:31:44,258][20381] Updated weights for policy 1, policy_version 271364 (0.0008) [2023-12-27 07:31:44,762][20347] Updated weights for policy 0, policy_version 271752 (0.0007) [2023-12-27 07:31:44,825][20347] Updated weights for policy 0, policy_version 271762 (0.0007) [2023-12-27 07:31:44,881][20347] Updated weights for policy 0, policy_version 271772 (0.0007) [2023-12-27 07:31:44,994][20381] Updated weights for policy 1, policy_version 271374 (0.0006) [2023-12-27 07:31:45,047][20381] Updated weights for policy 1, policy_version 271384 (0.0010) [2023-12-27 07:31:45,105][20381] Updated weights for policy 1, policy_version 271394 (0.0008) [2023-12-27 07:31:45,579][20347] Updated weights for policy 0, policy_version 271782 (0.0008) [2023-12-27 07:31:45,634][20347] Updated weights for policy 0, policy_version 271792 (0.0008) [2023-12-27 07:31:45,684][20347] Updated weights for policy 0, policy_version 271802 (0.0007) [2023-12-27 07:31:45,787][20381] Updated weights for policy 1, policy_version 271404 (0.0011) [2023-12-27 07:31:45,837][20381] Updated weights for policy 1, policy_version 271414 (0.0011) [2023-12-27 07:31:45,890][20381] Updated weights for policy 1, policy_version 271424 (0.0011) [2023-12-27 07:31:46,362][20347] Updated weights for policy 0, policy_version 271812 (0.0006) [2023-12-27 07:31:46,419][20347] Updated weights for policy 0, policy_version 271822 (0.0007) [2023-12-27 07:31:46,476][20347] Updated weights for policy 0, policy_version 271832 (0.0008) [2023-12-27 07:31:46,602][20381] Updated weights for policy 1, policy_version 271434 (0.0010) [2023-12-27 07:31:46,655][20381] Updated weights for policy 1, policy_version 271444 (0.0005) [2023-12-27 07:31:46,709][20381] Updated weights for policy 1, policy_version 271454 (0.0005) [2023-12-27 07:31:46,772][20381] Updated weights for policy 1, policy_version 271464 (0.0007) [2023-12-27 07:31:47,131][20347] Updated weights for policy 0, policy_version 271842 (0.0008) [2023-12-27 07:31:47,179][20347] Updated weights for policy 0, policy_version 271852 (0.0008) [2023-12-27 07:31:47,230][20347] Updated weights for policy 0, policy_version 271862 (0.0007) [2023-12-27 07:31:47,276][20347] Updated weights for policy 0, policy_version 271872 (0.0008) [2023-12-27 07:31:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 139124736. Throughput: 0: 10357.5, 1: 10081.4. Samples: 139134796. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:31:47,363][19170] Avg episode reward: [(0, '786.680'), (1, '1000.000')] [2023-12-27 07:31:47,378][20381] Updated weights for policy 1, policy_version 271474 (0.0006) [2023-12-27 07:31:47,432][20381] Updated weights for policy 1, policy_version 271484 (0.0007) [2023-12-27 07:31:47,481][20381] Updated weights for policy 1, policy_version 271494 (0.0009) [2023-12-27 07:31:47,935][20347] Updated weights for policy 0, policy_version 271882 (0.0008) [2023-12-27 07:31:47,984][20347] Updated weights for policy 0, policy_version 271892 (0.0007) [2023-12-27 07:31:48,035][20347] Updated weights for policy 0, policy_version 271902 (0.0008) [2023-12-27 07:31:48,213][20381] Updated weights for policy 1, policy_version 271504 (0.0008) [2023-12-27 07:31:48,267][20381] Updated weights for policy 1, policy_version 271514 (0.0006) [2023-12-27 07:31:48,324][20381] Updated weights for policy 1, policy_version 271524 (0.0009) [2023-12-27 07:31:48,688][20347] Updated weights for policy 0, policy_version 271912 (0.0006) [2023-12-27 07:31:48,747][20347] Updated weights for policy 0, policy_version 271922 (0.0010) [2023-12-27 07:31:48,806][20347] Updated weights for policy 0, policy_version 271932 (0.0009) [2023-12-27 07:31:49,046][20381] Updated weights for policy 1, policy_version 271534 (0.0009) [2023-12-27 07:31:49,099][20381] Updated weights for policy 1, policy_version 271544 (0.0011) [2023-12-27 07:31:49,165][20381] Updated weights for policy 1, policy_version 271554 (0.0011) [2023-12-27 07:31:49,474][20347] Updated weights for policy 0, policy_version 271942 (0.0008) [2023-12-27 07:31:49,531][20347] Updated weights for policy 0, policy_version 271952 (0.0009) [2023-12-27 07:31:49,586][20347] Updated weights for policy 0, policy_version 271962 (0.0010) [2023-12-27 07:31:49,877][20381] Updated weights for policy 1, policy_version 271564 (0.0009) [2023-12-27 07:31:49,943][20381] Updated weights for policy 1, policy_version 271574 (0.0008) [2023-12-27 07:31:49,993][20381] Updated weights for policy 1, policy_version 271584 (0.0006) [2023-12-27 07:31:50,245][20347] Updated weights for policy 0, policy_version 271972 (0.0008) [2023-12-27 07:31:50,300][20347] Updated weights for policy 0, policy_version 271982 (0.0005) [2023-12-27 07:31:50,357][20347] Updated weights for policy 0, policy_version 271992 (0.0007) [2023-12-27 07:31:50,603][20381] Updated weights for policy 1, policy_version 271594 (0.0006) [2023-12-27 07:31:50,665][20381] Updated weights for policy 1, policy_version 271604 (0.0007) [2023-12-27 07:31:50,733][20381] Updated weights for policy 1, policy_version 271614 (0.0011) [2023-12-27 07:31:50,799][20381] Updated weights for policy 1, policy_version 271624 (0.0011) [2023-12-27 07:31:50,912][20347] Updated weights for policy 0, policy_version 272002 (0.0006) [2023-12-27 07:31:50,972][20347] Updated weights for policy 0, policy_version 272012 (0.0006) [2023-12-27 07:31:51,038][20347] Updated weights for policy 0, policy_version 272022 (0.0008) [2023-12-27 07:31:51,102][20347] Updated weights for policy 0, policy_version 272032 (0.0008) [2023-12-27 07:31:51,447][20381] Updated weights for policy 1, policy_version 271634 (0.0008) [2023-12-27 07:31:51,508][20381] Updated weights for policy 1, policy_version 271644 (0.0006) [2023-12-27 07:31:51,571][20381] Updated weights for policy 1, policy_version 271654 (0.0006) [2023-12-27 07:31:51,794][20347] Updated weights for policy 0, policy_version 272042 (0.0007) [2023-12-27 07:31:51,849][20347] Updated weights for policy 0, policy_version 272052 (0.0005) [2023-12-27 07:31:51,903][20347] Updated weights for policy 0, policy_version 272062 (0.0005) [2023-12-27 07:31:52,248][20381] Updated weights for policy 1, policy_version 271664 (0.0008) [2023-12-27 07:31:52,306][20381] Updated weights for policy 1, policy_version 271674 (0.0009) [2023-12-27 07:31:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20688.3). Total num frames: 139223040. Throughput: 0: 10362.9, 1: 10048.1. Samples: 139195848. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:31:52,363][19170] Avg episode reward: [(0, '893.390'), (1, '1000.000')] [2023-12-27 07:31:52,367][20381] Updated weights for policy 1, policy_version 271684 (0.0007) [2023-12-27 07:31:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000272064_69664768.pth... [2023-12-27 07:31:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000270848_69353472.pth [2023-12-27 07:31:52,392][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000271688_69566464.pth... [2023-12-27 07:31:52,396][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000270472_69255168.pth [2023-12-27 07:31:52,505][20347] Updated weights for policy 0, policy_version 272072 (0.0009) [2023-12-27 07:31:52,563][20347] Updated weights for policy 0, policy_version 272082 (0.0007) [2023-12-27 07:31:52,635][20347] Updated weights for policy 0, policy_version 272092 (0.0006) [2023-12-27 07:31:53,074][20381] Updated weights for policy 1, policy_version 271694 (0.0008) [2023-12-27 07:31:53,135][20381] Updated weights for policy 1, policy_version 271704 (0.0008) [2023-12-27 07:31:53,183][20381] Updated weights for policy 1, policy_version 271714 (0.0008) [2023-12-27 07:31:53,222][20347] Updated weights for policy 0, policy_version 272102 (0.0007) [2023-12-27 07:31:53,280][20347] Updated weights for policy 0, policy_version 272112 (0.0007) [2023-12-27 07:31:53,338][20347] Updated weights for policy 0, policy_version 272122 (0.0008) [2023-12-27 07:31:53,824][20381] Updated weights for policy 1, policy_version 271724 (0.0007) [2023-12-27 07:31:53,884][20381] Updated weights for policy 1, policy_version 271734 (0.0008) [2023-12-27 07:31:53,942][20347] Updated weights for policy 0, policy_version 272132 (0.0007) [2023-12-27 07:31:53,945][20381] Updated weights for policy 1, policy_version 271744 (0.0007) [2023-12-27 07:31:53,993][20347] Updated weights for policy 0, policy_version 272142 (0.0005) [2023-12-27 07:31:54,055][20347] Updated weights for policy 0, policy_version 272152 (0.0008) [2023-12-27 07:31:54,591][20381] Updated weights for policy 1, policy_version 271754 (0.0007) [2023-12-27 07:31:54,631][20347] Updated weights for policy 0, policy_version 272162 (0.0008) [2023-12-27 07:31:54,641][20381] Updated weights for policy 1, policy_version 271764 (0.0006) [2023-12-27 07:31:54,686][20347] Updated weights for policy 0, policy_version 272172 (0.0006) [2023-12-27 07:31:54,690][20381] Updated weights for policy 1, policy_version 271774 (0.0005) [2023-12-27 07:31:54,739][20347] Updated weights for policy 0, policy_version 272182 (0.0006) [2023-12-27 07:31:54,740][20381] Updated weights for policy 1, policy_version 271784 (0.0006) [2023-12-27 07:31:54,792][20347] Updated weights for policy 0, policy_version 272192 (0.0005) [2023-12-27 07:31:55,310][20381] Updated weights for policy 1, policy_version 271794 (0.0008) [2023-12-27 07:31:55,364][20381] Updated weights for policy 1, policy_version 271804 (0.0008) [2023-12-27 07:31:55,405][20347] Updated weights for policy 0, policy_version 272202 (0.0007) [2023-12-27 07:31:55,417][20381] Updated weights for policy 1, policy_version 271814 (0.0005) [2023-12-27 07:31:55,451][20347] Updated weights for policy 0, policy_version 272212 (0.0007) [2023-12-27 07:31:55,499][20347] Updated weights for policy 0, policy_version 272222 (0.0007) [2023-12-27 07:31:55,997][20381] Updated weights for policy 1, policy_version 271824 (0.0005) [2023-12-27 07:31:56,045][20381] Updated weights for policy 1, policy_version 271834 (0.0005) [2023-12-27 07:31:56,095][20381] Updated weights for policy 1, policy_version 271844 (0.0006) [2023-12-27 07:31:56,267][20347] Updated weights for policy 0, policy_version 272232 (0.0008) [2023-12-27 07:31:56,325][20347] Updated weights for policy 0, policy_version 272242 (0.0007) [2023-12-27 07:31:56,388][20347] Updated weights for policy 0, policy_version 272252 (0.0007) [2023-12-27 07:31:56,721][20381] Updated weights for policy 1, policy_version 271854 (0.0008) [2023-12-27 07:31:56,779][20381] Updated weights for policy 1, policy_version 271864 (0.0008) [2023-12-27 07:31:56,831][20381] Updated weights for policy 1, policy_version 271874 (0.0008) [2023-12-27 07:31:57,052][20347] Updated weights for policy 0, policy_version 272262 (0.0007) [2023-12-27 07:31:57,096][20347] Updated weights for policy 0, policy_version 272272 (0.0005) [2023-12-27 07:31:57,154][20347] Updated weights for policy 0, policy_version 272282 (0.0005) [2023-12-27 07:31:57,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 139337728. Throughput: 0: 10414.4, 1: 10192.6. Samples: 139325988. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:31:57,363][19170] Avg episode reward: [(0, '941.970'), (1, '1000.000')] [2023-12-27 07:31:57,383][20381] Updated weights for policy 1, policy_version 271884 (0.0007) [2023-12-27 07:31:57,432][20381] Updated weights for policy 1, policy_version 271894 (0.0005) [2023-12-27 07:31:57,482][20381] Updated weights for policy 1, policy_version 271904 (0.0005) [2023-12-27 07:31:57,816][20347] Updated weights for policy 0, policy_version 272292 (0.0006) [2023-12-27 07:31:57,869][20347] Updated weights for policy 0, policy_version 272302 (0.0008) [2023-12-27 07:31:57,916][20347] Updated weights for policy 0, policy_version 272312 (0.0008) [2023-12-27 07:31:58,011][20381] Updated weights for policy 1, policy_version 271914 (0.0005) [2023-12-27 07:31:58,067][20381] Updated weights for policy 1, policy_version 271924 (0.0005) [2023-12-27 07:31:58,127][20381] Updated weights for policy 1, policy_version 271934 (0.0005) [2023-12-27 07:31:58,191][20381] Updated weights for policy 1, policy_version 271944 (0.0006) [2023-12-27 07:31:58,687][20347] Updated weights for policy 0, policy_version 272322 (0.0008) [2023-12-27 07:31:58,748][20347] Updated weights for policy 0, policy_version 272332 (0.0007) [2023-12-27 07:31:58,809][20347] Updated weights for policy 0, policy_version 272342 (0.0008) [2023-12-27 07:31:58,873][20347] Updated weights for policy 0, policy_version 272352 (0.0008) [2023-12-27 07:31:58,957][20381] Updated weights for policy 1, policy_version 271954 (0.0007) [2023-12-27 07:31:59,016][20381] Updated weights for policy 1, policy_version 271964 (0.0008) [2023-12-27 07:31:59,074][20381] Updated weights for policy 1, policy_version 271974 (0.0008) [2023-12-27 07:31:59,659][20347] Updated weights for policy 0, policy_version 272362 (0.0006) [2023-12-27 07:31:59,721][20347] Updated weights for policy 0, policy_version 272372 (0.0006) [2023-12-27 07:31:59,746][20381] Updated weights for policy 1, policy_version 271984 (0.0007) [2023-12-27 07:31:59,781][20347] Updated weights for policy 0, policy_version 272382 (0.0006) [2023-12-27 07:31:59,800][20381] Updated weights for policy 1, policy_version 271994 (0.0007) [2023-12-27 07:31:59,858][20381] Updated weights for policy 1, policy_version 272004 (0.0005) [2023-12-27 07:32:00,377][20347] Updated weights for policy 0, policy_version 272392 (0.0006) [2023-12-27 07:32:00,436][20347] Updated weights for policy 0, policy_version 272402 (0.0005) [2023-12-27 07:32:00,498][20347] Updated weights for policy 0, policy_version 272412 (0.0007) [2023-12-27 07:32:00,511][20381] Updated weights for policy 1, policy_version 272014 (0.0006) [2023-12-27 07:32:00,566][20381] Updated weights for policy 1, policy_version 272024 (0.0007) [2023-12-27 07:32:00,625][20381] Updated weights for policy 1, policy_version 272034 (0.0007) [2023-12-27 07:32:01,147][20347] Updated weights for policy 0, policy_version 272422 (0.0007) [2023-12-27 07:32:01,206][20347] Updated weights for policy 0, policy_version 272432 (0.0006) [2023-12-27 07:32:01,266][20347] Updated weights for policy 0, policy_version 272442 (0.0006) [2023-12-27 07:32:01,335][20381] Updated weights for policy 1, policy_version 272044 (0.0008) [2023-12-27 07:32:01,398][20381] Updated weights for policy 1, policy_version 272054 (0.0007) [2023-12-27 07:32:01,460][20381] Updated weights for policy 1, policy_version 272064 (0.0006) [2023-12-27 07:32:01,921][20347] Updated weights for policy 0, policy_version 272452 (0.0007) [2023-12-27 07:32:01,983][20347] Updated weights for policy 0, policy_version 272462 (0.0007) [2023-12-27 07:32:02,042][20347] Updated weights for policy 0, policy_version 272472 (0.0008) [2023-12-27 07:32:02,112][20381] Updated weights for policy 1, policy_version 272074 (0.0007) [2023-12-27 07:32:02,170][20381] Updated weights for policy 1, policy_version 272084 (0.0005) [2023-12-27 07:32:02,230][20381] Updated weights for policy 1, policy_version 272094 (0.0006) [2023-12-27 07:32:02,290][20381] Updated weights for policy 1, policy_version 272104 (0.0007) [2023-12-27 07:32:02,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 139444224. Throughput: 0: 10359.7, 1: 10287.4. Samples: 139450284. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:02,363][19170] Avg episode reward: [(0, '951.700'), (1, '1000.000')] [2023-12-27 07:32:02,710][20347] Updated weights for policy 0, policy_version 272482 (0.0007) [2023-12-27 07:32:02,761][20347] Updated weights for policy 0, policy_version 272492 (0.0006) [2023-12-27 07:32:02,809][20347] Updated weights for policy 0, policy_version 272502 (0.0005) [2023-12-27 07:32:02,860][20347] Updated weights for policy 0, policy_version 272512 (0.0005) [2023-12-27 07:32:03,012][20381] Updated weights for policy 1, policy_version 272114 (0.0007) [2023-12-27 07:32:03,070][20381] Updated weights for policy 1, policy_version 272124 (0.0008) [2023-12-27 07:32:03,125][20381] Updated weights for policy 1, policy_version 272134 (0.0009) [2023-12-27 07:32:03,463][20347] Updated weights for policy 0, policy_version 272522 (0.0007) [2023-12-27 07:32:03,512][20347] Updated weights for policy 0, policy_version 272532 (0.0007) [2023-12-27 07:32:03,557][20347] Updated weights for policy 0, policy_version 272542 (0.0008) [2023-12-27 07:32:03,743][20381] Updated weights for policy 1, policy_version 272144 (0.0007) [2023-12-27 07:32:03,794][20381] Updated weights for policy 1, policy_version 272154 (0.0007) [2023-12-27 07:32:03,853][20381] Updated weights for policy 1, policy_version 272164 (0.0007) [2023-12-27 07:32:04,282][20347] Updated weights for policy 0, policy_version 272552 (0.0008) [2023-12-27 07:32:04,343][20347] Updated weights for policy 0, policy_version 272562 (0.0007) [2023-12-27 07:32:04,400][20347] Updated weights for policy 0, policy_version 272572 (0.0007) [2023-12-27 07:32:04,546][20381] Updated weights for policy 1, policy_version 272174 (0.0006) [2023-12-27 07:32:04,611][20381] Updated weights for policy 1, policy_version 272184 (0.0005) [2023-12-27 07:32:04,672][20381] Updated weights for policy 1, policy_version 272194 (0.0007) [2023-12-27 07:32:05,113][20347] Updated weights for policy 0, policy_version 272582 (0.0007) [2023-12-27 07:32:05,158][20347] Updated weights for policy 0, policy_version 272592 (0.0007) [2023-12-27 07:32:05,217][20347] Updated weights for policy 0, policy_version 272602 (0.0008) [2023-12-27 07:32:05,278][20381] Updated weights for policy 1, policy_version 272204 (0.0007) [2023-12-27 07:32:05,320][20381] Updated weights for policy 1, policy_version 272214 (0.0007) [2023-12-27 07:32:05,379][20381] Updated weights for policy 1, policy_version 272225 (0.0008) [2023-12-27 07:32:05,942][20347] Updated weights for policy 0, policy_version 272612 (0.0007) [2023-12-27 07:32:06,001][20347] Updated weights for policy 0, policy_version 272622 (0.0008) [2023-12-27 07:32:06,008][20381] Updated weights for policy 1, policy_version 272235 (0.0006) [2023-12-27 07:32:06,058][20381] Updated weights for policy 1, policy_version 272245 (0.0007) [2023-12-27 07:32:06,063][20347] Updated weights for policy 0, policy_version 272632 (0.0006) [2023-12-27 07:32:06,120][20381] Updated weights for policy 1, policy_version 272255 (0.0007) [2023-12-27 07:32:06,755][20347] Updated weights for policy 0, policy_version 272642 (0.0006) [2023-12-27 07:32:06,815][20347] Updated weights for policy 0, policy_version 272652 (0.0007) [2023-12-27 07:32:06,849][20381] Updated weights for policy 1, policy_version 272265 (0.0007) [2023-12-27 07:32:06,873][20347] Updated weights for policy 0, policy_version 272662 (0.0006) [2023-12-27 07:32:06,907][20381] Updated weights for policy 1, policy_version 272275 (0.0008) [2023-12-27 07:32:06,937][20347] Updated weights for policy 0, policy_version 272672 (0.0006) [2023-12-27 07:32:06,966][20381] Updated weights for policy 1, policy_version 272285 (0.0008) [2023-12-27 07:32:07,029][20381] Updated weights for policy 1, policy_version 272295 (0.0008) [2023-12-27 07:32:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.4, 300 sec: 20716.0). Total num frames: 139542528. Throughput: 0: 10309.4, 1: 10344.1. Samples: 139512920. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:07,364][19170] Avg episode reward: [(0, '980.720'), (1, '1000.000')] [2023-12-27 07:32:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000272672_69820416.pth... [2023-12-27 07:32:07,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000272296_69722112.pth... [2023-12-27 07:32:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000271456_69509120.pth [2023-12-27 07:32:07,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000271080_69410816.pth [2023-12-27 07:32:07,601][20347] Updated weights for policy 0, policy_version 272682 (0.0008) [2023-12-27 07:32:07,662][20347] Updated weights for policy 0, policy_version 272692 (0.0008) [2023-12-27 07:32:07,712][20347] Updated weights for policy 0, policy_version 272702 (0.0007) [2023-12-27 07:32:07,734][20381] Updated weights for policy 1, policy_version 272305 (0.0007) [2023-12-27 07:32:07,779][20381] Updated weights for policy 1, policy_version 272315 (0.0008) [2023-12-27 07:32:07,821][20381] Updated weights for policy 1, policy_version 272325 (0.0007) [2023-12-27 07:32:08,365][20347] Updated weights for policy 0, policy_version 272712 (0.0007) [2023-12-27 07:32:08,423][20347] Updated weights for policy 0, policy_version 272722 (0.0008) [2023-12-27 07:32:08,475][20347] Updated weights for policy 0, policy_version 272732 (0.0006) [2023-12-27 07:32:08,549][20381] Updated weights for policy 1, policy_version 272335 (0.0010) [2023-12-27 07:32:08,611][20381] Updated weights for policy 1, policy_version 272345 (0.0011) [2023-12-27 07:32:08,667][20381] Updated weights for policy 1, policy_version 272355 (0.0010) [2023-12-27 07:32:09,146][20347] Updated weights for policy 0, policy_version 272742 (0.0006) [2023-12-27 07:32:09,204][20347] Updated weights for policy 0, policy_version 272752 (0.0006) [2023-12-27 07:32:09,265][20347] Updated weights for policy 0, policy_version 272762 (0.0007) [2023-12-27 07:32:09,320][20381] Updated weights for policy 1, policy_version 272365 (0.0008) [2023-12-27 07:32:09,379][20381] Updated weights for policy 1, policy_version 272375 (0.0008) [2023-12-27 07:32:09,449][20381] Updated weights for policy 1, policy_version 272385 (0.0007) [2023-12-27 07:32:09,961][20347] Updated weights for policy 0, policy_version 272772 (0.0008) [2023-12-27 07:32:10,011][20347] Updated weights for policy 0, policy_version 272782 (0.0010) [2023-12-27 07:32:10,065][20347] Updated weights for policy 0, policy_version 272792 (0.0011) [2023-12-27 07:32:10,149][20381] Updated weights for policy 1, policy_version 272395 (0.0008) [2023-12-27 07:32:10,202][20381] Updated weights for policy 1, policy_version 272405 (0.0010) [2023-12-27 07:32:10,258][20381] Updated weights for policy 1, policy_version 272415 (0.0010) [2023-12-27 07:32:10,821][20347] Updated weights for policy 0, policy_version 272802 (0.0009) [2023-12-27 07:32:10,864][20347] Updated weights for policy 0, policy_version 272812 (0.0005) [2023-12-27 07:32:10,912][20347] Updated weights for policy 0, policy_version 272822 (0.0005) [2023-12-27 07:32:10,963][20347] Updated weights for policy 0, policy_version 272832 (0.0005) [2023-12-27 07:32:10,967][20381] Updated weights for policy 1, policy_version 272425 (0.0010) [2023-12-27 07:32:11,035][20381] Updated weights for policy 1, policy_version 272435 (0.0010) [2023-12-27 07:32:11,100][20381] Updated weights for policy 1, policy_version 272445 (0.0011) [2023-12-27 07:32:11,166][20381] Updated weights for policy 1, policy_version 272455 (0.0009) [2023-12-27 07:32:11,662][20347] Updated weights for policy 0, policy_version 272842 (0.0010) [2023-12-27 07:32:11,722][20347] Updated weights for policy 0, policy_version 272852 (0.0010) [2023-12-27 07:32:11,790][20347] Updated weights for policy 0, policy_version 272862 (0.0011) [2023-12-27 07:32:11,869][20381] Updated weights for policy 1, policy_version 272465 (0.0008) [2023-12-27 07:32:11,925][20381] Updated weights for policy 1, policy_version 272475 (0.0008) [2023-12-27 07:32:11,978][20381] Updated weights for policy 1, policy_version 272485 (0.0006) [2023-12-27 07:32:12,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 139640832. Throughput: 0: 10293.0, 1: 10272.1. Samples: 139633396. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:12,363][19170] Avg episode reward: [(0, '980.720'), (1, '990.450')] [2023-12-27 07:32:12,482][20347] Updated weights for policy 0, policy_version 272872 (0.0008) [2023-12-27 07:32:12,545][20347] Updated weights for policy 0, policy_version 272882 (0.0009) [2023-12-27 07:32:12,607][20347] Updated weights for policy 0, policy_version 272892 (0.0008) [2023-12-27 07:32:12,650][20381] Updated weights for policy 1, policy_version 272495 (0.0006) [2023-12-27 07:32:12,707][20381] Updated weights for policy 1, policy_version 272505 (0.0007) [2023-12-27 07:32:12,767][20381] Updated weights for policy 1, policy_version 272515 (0.0007) [2023-12-27 07:32:13,223][20347] Updated weights for policy 0, policy_version 272902 (0.0007) [2023-12-27 07:32:13,290][20347] Updated weights for policy 0, policy_version 272912 (0.0008) [2023-12-27 07:32:13,339][20381] Updated weights for policy 1, policy_version 272525 (0.0005) [2023-12-27 07:32:13,352][20347] Updated weights for policy 0, policy_version 272922 (0.0006) [2023-12-27 07:32:13,398][20381] Updated weights for policy 1, policy_version 272535 (0.0005) [2023-12-27 07:32:13,458][20381] Updated weights for policy 1, policy_version 272545 (0.0005) [2023-12-27 07:32:13,992][20347] Updated weights for policy 0, policy_version 272932 (0.0007) [2023-12-27 07:32:14,052][20347] Updated weights for policy 0, policy_version 272942 (0.0006) [2023-12-27 07:32:14,071][20381] Updated weights for policy 1, policy_version 272555 (0.0007) [2023-12-27 07:32:14,108][20347] Updated weights for policy 0, policy_version 272952 (0.0006) [2023-12-27 07:32:14,140][20381] Updated weights for policy 1, policy_version 272566 (0.0011) [2023-12-27 07:32:14,202][20381] Updated weights for policy 1, policy_version 272576 (0.0008) [2023-12-27 07:32:14,758][20347] Updated weights for policy 0, policy_version 272962 (0.0007) [2023-12-27 07:32:14,818][20347] Updated weights for policy 0, policy_version 272972 (0.0008) [2023-12-27 07:32:14,847][20381] Updated weights for policy 1, policy_version 272586 (0.0006) [2023-12-27 07:32:14,878][20347] Updated weights for policy 0, policy_version 272982 (0.0006) [2023-12-27 07:32:14,909][20292] KL-divergence is very high: 169.7382 [2023-12-27 07:32:14,910][20381] Updated weights for policy 1, policy_version 272596 (0.0006) [2023-12-27 07:32:14,940][20347] Updated weights for policy 0, policy_version 272992 (0.0009) [2023-12-27 07:32:14,968][20381] Updated weights for policy 1, policy_version 272606 (0.0009) [2023-12-27 07:32:15,030][20381] Updated weights for policy 1, policy_version 272616 (0.0009) [2023-12-27 07:32:15,595][20381] Updated weights for policy 1, policy_version 272626 (0.0011) [2023-12-27 07:32:15,647][20381] Updated weights for policy 1, policy_version 272636 (0.0010) [2023-12-27 07:32:15,652][20347] Updated weights for policy 0, policy_version 273002 (0.0005) [2023-12-27 07:32:15,696][20381] Updated weights for policy 1, policy_version 272646 (0.0011) [2023-12-27 07:32:15,705][20347] Updated weights for policy 0, policy_version 273012 (0.0006) [2023-12-27 07:32:15,764][20347] Updated weights for policy 0, policy_version 273022 (0.0007) [2023-12-27 07:32:16,287][20381] Updated weights for policy 1, policy_version 272656 (0.0006) [2023-12-27 07:32:16,342][20381] Updated weights for policy 1, policy_version 272666 (0.0011) [2023-12-27 07:32:16,394][20381] Updated weights for policy 1, policy_version 272676 (0.0010) [2023-12-27 07:32:16,467][20347] Updated weights for policy 0, policy_version 273032 (0.0007) [2023-12-27 07:32:16,514][20347] Updated weights for policy 0, policy_version 273042 (0.0008) [2023-12-27 07:32:16,565][20347] Updated weights for policy 0, policy_version 273052 (0.0007) [2023-12-27 07:32:17,017][20381] Updated weights for policy 1, policy_version 272686 (0.0007) [2023-12-27 07:32:17,066][20381] Updated weights for policy 1, policy_version 272696 (0.0011) [2023-12-27 07:32:17,125][20381] Updated weights for policy 1, policy_version 272706 (0.0011) [2023-12-27 07:32:17,311][20347] Updated weights for policy 0, policy_version 273062 (0.0007) [2023-12-27 07:32:17,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 139747328. Throughput: 0: 10293.8, 1: 10347.2. Samples: 139759524. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:17,363][19170] Avg episode reward: [(0, '990.610'), (1, '971.230')] [2023-12-27 07:32:17,374][20347] Updated weights for policy 0, policy_version 273072 (0.0008) [2023-12-27 07:32:17,433][20347] Updated weights for policy 0, policy_version 273082 (0.0008) [2023-12-27 07:32:17,767][20381] Updated weights for policy 1, policy_version 272716 (0.0011) [2023-12-27 07:32:17,822][20381] Updated weights for policy 1, policy_version 272726 (0.0010) [2023-12-27 07:32:17,883][20381] Updated weights for policy 1, policy_version 272736 (0.0006) [2023-12-27 07:32:18,058][20347] Updated weights for policy 0, policy_version 273092 (0.0007) [2023-12-27 07:32:18,104][20347] Updated weights for policy 0, policy_version 273102 (0.0008) [2023-12-27 07:32:18,149][20347] Updated weights for policy 0, policy_version 273112 (0.0005) [2023-12-27 07:32:18,555][20381] Updated weights for policy 1, policy_version 272746 (0.0006) [2023-12-27 07:32:18,612][20381] Updated weights for policy 1, policy_version 272756 (0.0006) [2023-12-27 07:32:18,663][20381] Updated weights for policy 1, policy_version 272766 (0.0007) [2023-12-27 07:32:18,718][20381] Updated weights for policy 1, policy_version 272776 (0.0011) [2023-12-27 07:32:18,779][20347] Updated weights for policy 0, policy_version 273122 (0.0006) [2023-12-27 07:32:18,841][20347] Updated weights for policy 0, policy_version 273132 (0.0006) [2023-12-27 07:32:18,904][20347] Updated weights for policy 0, policy_version 273142 (0.0005) [2023-12-27 07:32:18,963][20347] Updated weights for policy 0, policy_version 273152 (0.0008) [2023-12-27 07:32:19,382][20381] Updated weights for policy 1, policy_version 272786 (0.0011) [2023-12-27 07:32:19,439][20381] Updated weights for policy 1, policy_version 272796 (0.0011) [2023-12-27 07:32:19,507][20381] Updated weights for policy 1, policy_version 272806 (0.0009) [2023-12-27 07:32:19,591][20347] Updated weights for policy 0, policy_version 273162 (0.0007) [2023-12-27 07:32:19,647][20347] Updated weights for policy 0, policy_version 273172 (0.0007) [2023-12-27 07:32:19,705][20347] Updated weights for policy 0, policy_version 273182 (0.0007) [2023-12-27 07:32:20,267][20381] Updated weights for policy 1, policy_version 272816 (0.0008) [2023-12-27 07:32:20,334][20381] Updated weights for policy 1, policy_version 272826 (0.0006) [2023-12-27 07:32:20,385][20347] Updated weights for policy 0, policy_version 273192 (0.0006) [2023-12-27 07:32:20,395][20381] Updated weights for policy 1, policy_version 272836 (0.0010) [2023-12-27 07:32:20,439][20347] Updated weights for policy 0, policy_version 273202 (0.0006) [2023-12-27 07:32:20,491][20347] Updated weights for policy 0, policy_version 273212 (0.0007) [2023-12-27 07:32:21,060][20381] Updated weights for policy 1, policy_version 272846 (0.0010) [2023-12-27 07:32:21,120][20381] Updated weights for policy 1, policy_version 272856 (0.0011) [2023-12-27 07:32:21,171][20347] Updated weights for policy 0, policy_version 273222 (0.0007) [2023-12-27 07:32:21,184][20381] Updated weights for policy 1, policy_version 272866 (0.0010) [2023-12-27 07:32:21,227][20347] Updated weights for policy 0, policy_version 273232 (0.0005) [2023-12-27 07:32:21,287][20347] Updated weights for policy 0, policy_version 273242 (0.0006) [2023-12-27 07:32:21,822][20381] Updated weights for policy 1, policy_version 272876 (0.0008) [2023-12-27 07:32:21,883][20381] Updated weights for policy 1, policy_version 272886 (0.0007) [2023-12-27 07:32:21,904][20347] Updated weights for policy 0, policy_version 273252 (0.0006) [2023-12-27 07:32:21,946][20381] Updated weights for policy 1, policy_version 272896 (0.0006) [2023-12-27 07:32:21,964][20347] Updated weights for policy 0, policy_version 273262 (0.0008) [2023-12-27 07:32:22,018][20347] Updated weights for policy 0, policy_version 273272 (0.0007) [2023-12-27 07:32:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 139853824. Throughput: 0: 10296.2, 1: 10328.5. Samples: 139821756. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:22,363][19170] Avg episode reward: [(0, '942.200'), (1, '980.780')] [2023-12-27 07:32:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000272904_69877760.pth... [2023-12-27 07:32:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000273280_69976064.pth... [2023-12-27 07:32:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000271688_69566464.pth [2023-12-27 07:32:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000272064_69664768.pth [2023-12-27 07:32:22,548][20381] Updated weights for policy 1, policy_version 272906 (0.0007) [2023-12-27 07:32:22,607][20381] Updated weights for policy 1, policy_version 272916 (0.0010) [2023-12-27 07:32:22,668][20381] Updated weights for policy 1, policy_version 272926 (0.0011) [2023-12-27 07:32:22,727][20381] Updated weights for policy 1, policy_version 272936 (0.0011) [2023-12-27 07:32:22,798][20347] Updated weights for policy 0, policy_version 273282 (0.0008) [2023-12-27 07:32:22,862][20347] Updated weights for policy 0, policy_version 273292 (0.0008) [2023-12-27 07:32:22,931][20347] Updated weights for policy 0, policy_version 273302 (0.0007) [2023-12-27 07:32:22,986][20347] Updated weights for policy 0, policy_version 273312 (0.0008) [2023-12-27 07:32:23,339][20381] Updated weights for policy 1, policy_version 272946 (0.0010) [2023-12-27 07:32:23,387][20381] Updated weights for policy 1, policy_version 272956 (0.0010) [2023-12-27 07:32:23,440][20381] Updated weights for policy 1, policy_version 272966 (0.0009) [2023-12-27 07:32:23,663][20347] Updated weights for policy 0, policy_version 273322 (0.0009) [2023-12-27 07:32:23,708][20347] Updated weights for policy 0, policy_version 273332 (0.0008) [2023-12-27 07:32:23,753][20347] Updated weights for policy 0, policy_version 273342 (0.0009) [2023-12-27 07:32:24,063][20381] Updated weights for policy 1, policy_version 272976 (0.0010) [2023-12-27 07:32:24,118][20381] Updated weights for policy 1, policy_version 272986 (0.0011) [2023-12-27 07:32:24,179][20381] Updated weights for policy 1, policy_version 272996 (0.0011) [2023-12-27 07:32:24,455][20347] Updated weights for policy 0, policy_version 273352 (0.0007) [2023-12-27 07:32:24,503][20347] Updated weights for policy 0, policy_version 273362 (0.0007) [2023-12-27 07:32:24,566][20347] Updated weights for policy 0, policy_version 273372 (0.0007) [2023-12-27 07:32:24,860][20381] Updated weights for policy 1, policy_version 273006 (0.0009) [2023-12-27 07:32:24,910][20381] Updated weights for policy 1, policy_version 273016 (0.0010) [2023-12-27 07:32:24,969][20381] Updated weights for policy 1, policy_version 273026 (0.0009) [2023-12-27 07:32:25,164][20347] Updated weights for policy 0, policy_version 273382 (0.0007) [2023-12-27 07:32:25,216][20347] Updated weights for policy 0, policy_version 273392 (0.0007) [2023-12-27 07:32:25,270][20347] Updated weights for policy 0, policy_version 273402 (0.0005) [2023-12-27 07:32:25,631][20381] Updated weights for policy 1, policy_version 273036 (0.0008) [2023-12-27 07:32:25,691][20381] Updated weights for policy 1, policy_version 273046 (0.0006) [2023-12-27 07:32:25,749][20381] Updated weights for policy 1, policy_version 273056 (0.0006) [2023-12-27 07:32:25,928][20347] Updated weights for policy 0, policy_version 273412 (0.0006) [2023-12-27 07:32:25,985][20347] Updated weights for policy 0, policy_version 273422 (0.0007) [2023-12-27 07:32:26,042][20347] Updated weights for policy 0, policy_version 273432 (0.0005) [2023-12-27 07:32:26,372][20381] Updated weights for policy 1, policy_version 273066 (0.0007) [2023-12-27 07:32:26,427][20381] Updated weights for policy 1, policy_version 273076 (0.0009) [2023-12-27 07:32:26,475][20381] Updated weights for policy 1, policy_version 273086 (0.0009) [2023-12-27 07:32:26,520][20381] Updated weights for policy 1, policy_version 273096 (0.0008) [2023-12-27 07:32:26,565][20347] Updated weights for policy 0, policy_version 273442 (0.0005) [2023-12-27 07:32:26,630][20347] Updated weights for policy 0, policy_version 273452 (0.0005) [2023-12-27 07:32:26,689][20347] Updated weights for policy 0, policy_version 273462 (0.0006) [2023-12-27 07:32:26,741][20347] Updated weights for policy 0, policy_version 273472 (0.0008) [2023-12-27 07:32:27,213][20381] Updated weights for policy 1, policy_version 273106 (0.0008) [2023-12-27 07:32:27,261][20381] Updated weights for policy 1, policy_version 273116 (0.0008) [2023-12-27 07:32:27,305][20381] Updated weights for policy 1, policy_version 273126 (0.0007) [2023-12-27 07:32:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 139960320. Throughput: 0: 10349.4, 1: 10439.5. Samples: 139947640. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:27,363][19170] Avg episode reward: [(0, '932.320'), (1, '1000.000')] [2023-12-27 07:32:27,376][20347] Updated weights for policy 0, policy_version 273482 (0.0008) [2023-12-27 07:32:27,423][20347] Updated weights for policy 0, policy_version 273492 (0.0007) [2023-12-27 07:32:27,481][20347] Updated weights for policy 0, policy_version 273502 (0.0008) [2023-12-27 07:32:28,003][20381] Updated weights for policy 1, policy_version 273136 (0.0007) [2023-12-27 07:32:28,062][20381] Updated weights for policy 1, policy_version 273146 (0.0007) [2023-12-27 07:32:28,117][20381] Updated weights for policy 1, policy_version 273156 (0.0008) [2023-12-27 07:32:28,175][20347] Updated weights for policy 0, policy_version 273512 (0.0008) [2023-12-27 07:32:28,232][20347] Updated weights for policy 0, policy_version 273522 (0.0007) [2023-12-27 07:32:28,283][20347] Updated weights for policy 0, policy_version 273532 (0.0007) [2023-12-27 07:32:28,826][20381] Updated weights for policy 1, policy_version 273166 (0.0007) [2023-12-27 07:32:28,884][20381] Updated weights for policy 1, policy_version 273176 (0.0008) [2023-12-27 07:32:28,901][20347] Updated weights for policy 0, policy_version 273542 (0.0006) [2023-12-27 07:32:28,936][20381] Updated weights for policy 1, policy_version 273186 (0.0006) [2023-12-27 07:32:28,953][20347] Updated weights for policy 0, policy_version 273552 (0.0005) [2023-12-27 07:32:29,008][20347] Updated weights for policy 0, policy_version 273562 (0.0006) [2023-12-27 07:32:29,636][20347] Updated weights for policy 0, policy_version 273572 (0.0006) [2023-12-27 07:32:29,660][20381] Updated weights for policy 1, policy_version 273196 (0.0007) [2023-12-27 07:32:29,685][20347] Updated weights for policy 0, policy_version 273582 (0.0005) [2023-12-27 07:32:29,713][20381] Updated weights for policy 1, policy_version 273206 (0.0008) [2023-12-27 07:32:29,736][20347] Updated weights for policy 0, policy_version 273592 (0.0005) [2023-12-27 07:32:29,762][20381] Updated weights for policy 1, policy_version 273216 (0.0006) [2023-12-27 07:32:30,379][20347] Updated weights for policy 0, policy_version 273602 (0.0006) [2023-12-27 07:32:30,434][20347] Updated weights for policy 0, policy_version 273612 (0.0007) [2023-12-27 07:32:30,474][20381] Updated weights for policy 1, policy_version 273226 (0.0007) [2023-12-27 07:32:30,484][20347] Updated weights for policy 0, policy_version 273622 (0.0007) [2023-12-27 07:32:30,527][20381] Updated weights for policy 1, policy_version 273236 (0.0006) [2023-12-27 07:32:30,534][20347] Updated weights for policy 0, policy_version 273632 (0.0007) [2023-12-27 07:32:30,574][20381] Updated weights for policy 1, policy_version 273246 (0.0006) [2023-12-27 07:32:30,624][20381] Updated weights for policy 1, policy_version 273256 (0.0005) [2023-12-27 07:32:31,135][20347] Updated weights for policy 0, policy_version 273642 (0.0011) [2023-12-27 07:32:31,195][20347] Updated weights for policy 0, policy_version 273652 (0.0008) [2023-12-27 07:32:31,213][20381] Updated weights for policy 1, policy_version 273266 (0.0008) [2023-12-27 07:32:31,253][20347] Updated weights for policy 0, policy_version 273662 (0.0008) [2023-12-27 07:32:31,273][20381] Updated weights for policy 1, policy_version 273276 (0.0008) [2023-12-27 07:32:31,329][20381] Updated weights for policy 1, policy_version 273286 (0.0007) [2023-12-27 07:32:31,995][20347] Updated weights for policy 0, policy_version 273672 (0.0010) [2023-12-27 07:32:32,024][20381] Updated weights for policy 1, policy_version 273296 (0.0010) [2023-12-27 07:32:32,054][20347] Updated weights for policy 0, policy_version 273682 (0.0010) [2023-12-27 07:32:32,081][20381] Updated weights for policy 1, policy_version 273306 (0.0009) [2023-12-27 07:32:32,110][20347] Updated weights for policy 0, policy_version 273692 (0.0010) [2023-12-27 07:32:32,147][20381] Updated weights for policy 1, policy_version 273316 (0.0005) [2023-12-27 07:32:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 140066816. Throughput: 0: 10409.2, 1: 10466.8. Samples: 140074220. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:32,363][19170] Avg episode reward: [(0, '951.630'), (1, '1000.000')] [2023-12-27 07:32:32,693][20381] Updated weights for policy 1, policy_version 273326 (0.0008) [2023-12-27 07:32:32,750][20381] Updated weights for policy 1, policy_version 273336 (0.0010) [2023-12-27 07:32:32,796][20347] Updated weights for policy 0, policy_version 273702 (0.0010) [2023-12-27 07:32:32,804][20381] Updated weights for policy 1, policy_version 273346 (0.0008) [2023-12-27 07:32:32,851][20347] Updated weights for policy 0, policy_version 273712 (0.0010) [2023-12-27 07:32:32,903][20347] Updated weights for policy 0, policy_version 273722 (0.0010) [2023-12-27 07:32:33,430][20381] Updated weights for policy 1, policy_version 273356 (0.0010) [2023-12-27 07:32:33,475][20381] Updated weights for policy 1, policy_version 273366 (0.0010) [2023-12-27 07:32:33,524][20381] Updated weights for policy 1, policy_version 273376 (0.0010) [2023-12-27 07:32:33,585][20347] Updated weights for policy 0, policy_version 273732 (0.0010) [2023-12-27 07:32:33,642][20347] Updated weights for policy 0, policy_version 273742 (0.0010) [2023-12-27 07:32:33,695][20347] Updated weights for policy 0, policy_version 273752 (0.0010) [2023-12-27 07:32:34,222][20381] Updated weights for policy 1, policy_version 273386 (0.0010) [2023-12-27 07:32:34,281][20381] Updated weights for policy 1, policy_version 273396 (0.0011) [2023-12-27 07:32:34,333][20347] Updated weights for policy 0, policy_version 273762 (0.0008) [2023-12-27 07:32:34,341][20381] Updated weights for policy 1, policy_version 273406 (0.0011) [2023-12-27 07:32:34,395][20347] Updated weights for policy 0, policy_version 273772 (0.0009) [2023-12-27 07:32:34,397][20381] Updated weights for policy 1, policy_version 273416 (0.0010) [2023-12-27 07:32:34,453][20347] Updated weights for policy 0, policy_version 273782 (0.0010) [2023-12-27 07:32:34,517][20347] Updated weights for policy 0, policy_version 273792 (0.0010) [2023-12-27 07:32:35,062][20381] Updated weights for policy 1, policy_version 273426 (0.0010) [2023-12-27 07:32:35,120][20381] Updated weights for policy 1, policy_version 273436 (0.0010) [2023-12-27 07:32:35,178][20381] Updated weights for policy 1, policy_version 273446 (0.0010) [2023-12-27 07:32:35,193][20347] Updated weights for policy 0, policy_version 273802 (0.0005) [2023-12-27 07:32:35,253][20347] Updated weights for policy 0, policy_version 273812 (0.0007) [2023-12-27 07:32:35,314][20347] Updated weights for policy 0, policy_version 273822 (0.0009) [2023-12-27 07:32:35,835][20381] Updated weights for policy 1, policy_version 273456 (0.0006) [2023-12-27 07:32:35,886][20381] Updated weights for policy 1, policy_version 273466 (0.0005) [2023-12-27 07:32:35,916][20347] Updated weights for policy 0, policy_version 273832 (0.0010) [2023-12-27 07:32:35,933][20381] Updated weights for policy 1, policy_version 273476 (0.0010) [2023-12-27 07:32:35,965][20347] Updated weights for policy 0, policy_version 273842 (0.0010) [2023-12-27 07:32:36,013][20347] Updated weights for policy 0, policy_version 273852 (0.0010) [2023-12-27 07:32:36,598][20381] Updated weights for policy 1, policy_version 273486 (0.0011) [2023-12-27 07:32:36,656][20347] Updated weights for policy 0, policy_version 273862 (0.0010) [2023-12-27 07:32:36,658][20381] Updated weights for policy 1, policy_version 273496 (0.0011) [2023-12-27 07:32:36,711][20381] Updated weights for policy 1, policy_version 273506 (0.0011) [2023-12-27 07:32:36,712][20347] Updated weights for policy 0, policy_version 273872 (0.0010) [2023-12-27 07:32:36,770][20347] Updated weights for policy 0, policy_version 273882 (0.0010) [2023-12-27 07:32:37,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 140165120. Throughput: 0: 10400.6, 1: 10527.5. Samples: 140137612. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:37,363][19170] Avg episode reward: [(0, '971.120'), (1, '1000.000')] [2023-12-27 07:32:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000273512_70033408.pth... [2023-12-27 07:32:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000272296_69722112.pth [2023-12-27 07:32:37,389][20347] Updated weights for policy 0, policy_version 273892 (0.0011) [2023-12-27 07:32:37,440][20347] Updated weights for policy 0, policy_version 273902 (0.0010) [2023-12-27 07:32:37,462][20381] Updated weights for policy 1, policy_version 273516 (0.0009) [2023-12-27 07:32:37,493][20347] Updated weights for policy 0, policy_version 273912 (0.0010) [2023-12-27 07:32:37,511][20381] Updated weights for policy 1, policy_version 273526 (0.0008) [2023-12-27 07:32:37,529][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000273920_70139904.pth... [2023-12-27 07:32:37,532][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000272672_69820416.pth [2023-12-27 07:32:37,561][20381] Updated weights for policy 1, policy_version 273536 (0.0007) [2023-12-27 07:32:38,097][20347] Updated weights for policy 0, policy_version 273922 (0.0009) [2023-12-27 07:32:38,156][20347] Updated weights for policy 0, policy_version 273932 (0.0007) [2023-12-27 07:32:38,221][20347] Updated weights for policy 0, policy_version 273942 (0.0005) [2023-12-27 07:32:38,272][20347] Updated weights for policy 0, policy_version 273952 (0.0005) [2023-12-27 07:32:38,309][20381] Updated weights for policy 1, policy_version 273546 (0.0008) [2023-12-27 07:32:38,371][20381] Updated weights for policy 1, policy_version 273556 (0.0009) [2023-12-27 07:32:38,438][20381] Updated weights for policy 1, policy_version 273566 (0.0007) [2023-12-27 07:32:38,489][20381] Updated weights for policy 1, policy_version 273576 (0.0007) [2023-12-27 07:32:38,912][20347] Updated weights for policy 0, policy_version 273962 (0.0007) [2023-12-27 07:32:38,973][20347] Updated weights for policy 0, policy_version 273972 (0.0007) [2023-12-27 07:32:39,031][20347] Updated weights for policy 0, policy_version 273982 (0.0008) [2023-12-27 07:32:39,203][20381] Updated weights for policy 1, policy_version 273586 (0.0010) [2023-12-27 07:32:39,262][20381] Updated weights for policy 1, policy_version 273596 (0.0010) [2023-12-27 07:32:39,321][20381] Updated weights for policy 1, policy_version 273606 (0.0011) [2023-12-27 07:32:39,761][20347] Updated weights for policy 0, policy_version 273992 (0.0010) [2023-12-27 07:32:39,825][20347] Updated weights for policy 0, policy_version 274002 (0.0011) [2023-12-27 07:32:39,884][20347] Updated weights for policy 0, policy_version 274012 (0.0010) [2023-12-27 07:32:40,028][20381] Updated weights for policy 1, policy_version 273616 (0.0010) [2023-12-27 07:32:40,080][20381] Updated weights for policy 1, policy_version 273626 (0.0008) [2023-12-27 07:32:40,136][20381] Updated weights for policy 1, policy_version 273636 (0.0008) [2023-12-27 07:32:40,569][20347] Updated weights for policy 0, policy_version 274022 (0.0010) [2023-12-27 07:32:40,622][20347] Updated weights for policy 0, policy_version 274032 (0.0010) [2023-12-27 07:32:40,670][20347] Updated weights for policy 0, policy_version 274042 (0.0010) [2023-12-27 07:32:40,809][20381] Updated weights for policy 1, policy_version 273646 (0.0010) [2023-12-27 07:32:40,865][20381] Updated weights for policy 1, policy_version 273656 (0.0010) [2023-12-27 07:32:40,921][20381] Updated weights for policy 1, policy_version 273666 (0.0008) [2023-12-27 07:32:41,408][20347] Updated weights for policy 0, policy_version 274052 (0.0009) [2023-12-27 07:32:41,464][20347] Updated weights for policy 0, policy_version 274062 (0.0008) [2023-12-27 07:32:41,523][20347] Updated weights for policy 0, policy_version 274072 (0.0008) [2023-12-27 07:32:41,631][20381] Updated weights for policy 1, policy_version 273676 (0.0006) [2023-12-27 07:32:41,698][20381] Updated weights for policy 1, policy_version 273686 (0.0008) [2023-12-27 07:32:41,762][20381] Updated weights for policy 1, policy_version 273696 (0.0008) [2023-12-27 07:32:42,256][20347] Updated weights for policy 0, policy_version 274082 (0.0008) [2023-12-27 07:32:42,315][20347] Updated weights for policy 0, policy_version 274092 (0.0008) [2023-12-27 07:32:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 140263424. Throughput: 0: 10349.3, 1: 10418.9. Samples: 140260552. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 07:32:42,363][19170] Avg episode reward: [(0, '942.180'), (1, '1000.000')] [2023-12-27 07:32:42,381][20347] Updated weights for policy 0, policy_version 274102 (0.0007) [2023-12-27 07:32:42,415][20381] Updated weights for policy 1, policy_version 273706 (0.0007) [2023-12-27 07:32:42,442][20347] Updated weights for policy 0, policy_version 274112 (0.0007) [2023-12-27 07:32:42,471][20381] Updated weights for policy 1, policy_version 273716 (0.0007) [2023-12-27 07:32:42,525][20381] Updated weights for policy 1, policy_version 273726 (0.0008) [2023-12-27 07:32:42,578][20381] Updated weights for policy 1, policy_version 273736 (0.0008) [2023-12-27 07:32:43,064][20347] Updated weights for policy 0, policy_version 274122 (0.0006) [2023-12-27 07:32:43,122][20347] Updated weights for policy 0, policy_version 274132 (0.0006) [2023-12-27 07:32:43,165][20347] Updated weights for policy 0, policy_version 274142 (0.0008) [2023-12-27 07:32:43,218][20381] Updated weights for policy 1, policy_version 273746 (0.0005) [2023-12-27 07:32:43,278][20381] Updated weights for policy 1, policy_version 273756 (0.0008) [2023-12-27 07:32:43,319][20292] KL-divergence is very high: 259.2175 [2023-12-27 07:32:43,338][20381] Updated weights for policy 1, policy_version 273766 (0.0008) [2023-12-27 07:32:43,706][20347] Updated weights for policy 0, policy_version 274152 (0.0006) [2023-12-27 07:32:43,758][20347] Updated weights for policy 0, policy_version 274162 (0.0005) [2023-12-27 07:32:43,804][20347] Updated weights for policy 0, policy_version 274172 (0.0005) [2023-12-27 07:32:43,921][20381] Updated weights for policy 1, policy_version 273776 (0.0007) [2023-12-27 07:32:43,922][20292] KL-divergence is very high: 124.2461 [2023-12-27 07:32:43,975][20381] Updated weights for policy 1, policy_version 273786 (0.0006) [2023-12-27 07:32:44,037][20381] Updated weights for policy 1, policy_version 273796 (0.0005) [2023-12-27 07:32:44,498][20347] Updated weights for policy 0, policy_version 274182 (0.0007) [2023-12-27 07:32:44,554][20347] Updated weights for policy 0, policy_version 274192 (0.0008) [2023-12-27 07:32:44,609][20347] Updated weights for policy 0, policy_version 274202 (0.0008) [2023-12-27 07:32:44,674][20381] Updated weights for policy 1, policy_version 273806 (0.0006) [2023-12-27 07:32:44,730][20381] Updated weights for policy 1, policy_version 273816 (0.0007) [2023-12-27 07:32:44,789][20381] Updated weights for policy 1, policy_version 273826 (0.0007) [2023-12-27 07:32:45,315][20347] Updated weights for policy 0, policy_version 274212 (0.0008) [2023-12-27 07:32:45,373][20347] Updated weights for policy 0, policy_version 274222 (0.0008) [2023-12-27 07:32:45,435][20347] Updated weights for policy 0, policy_version 274232 (0.0008) [2023-12-27 07:32:45,480][20381] Updated weights for policy 1, policy_version 273836 (0.0007) [2023-12-27 07:32:45,535][20381] Updated weights for policy 1, policy_version 273846 (0.0005) [2023-12-27 07:32:45,595][20381] Updated weights for policy 1, policy_version 273856 (0.0005) [2023-12-27 07:32:46,151][20347] Updated weights for policy 0, policy_version 274242 (0.0008) [2023-12-27 07:32:46,203][20347] Updated weights for policy 0, policy_version 274252 (0.0007) [2023-12-27 07:32:46,224][20381] Updated weights for policy 1, policy_version 273866 (0.0006) [2023-12-27 07:32:46,265][20347] Updated weights for policy 0, policy_version 274262 (0.0006) [2023-12-27 07:32:46,291][20381] Updated weights for policy 1, policy_version 273876 (0.0007) [2023-12-27 07:32:46,324][20347] Updated weights for policy 0, policy_version 274272 (0.0006) [2023-12-27 07:32:46,343][20381] Updated weights for policy 1, policy_version 273886 (0.0005) [2023-12-27 07:32:46,400][20381] Updated weights for policy 1, policy_version 273896 (0.0005) [2023-12-27 07:32:46,958][20381] Updated weights for policy 1, policy_version 273906 (0.0005) [2023-12-27 07:32:47,009][20381] Updated weights for policy 1, policy_version 273916 (0.0005) [2023-12-27 07:32:47,065][20381] Updated weights for policy 1, policy_version 273926 (0.0005) [2023-12-27 07:32:47,066][20347] Updated weights for policy 0, policy_version 274282 (0.0008) [2023-12-27 07:32:47,122][20347] Updated weights for policy 0, policy_version 274292 (0.0008) [2023-12-27 07:32:47,178][20347] Updated weights for policy 0, policy_version 274302 (0.0007) [2023-12-27 07:32:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 140378112. Throughput: 0: 10357.3, 1: 10430.8. Samples: 140385752. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:32:47,363][19170] Avg episode reward: [(0, '864.580'), (1, '971.250')] [2023-12-27 07:32:47,701][20381] Updated weights for policy 1, policy_version 273936 (0.0007) [2023-12-27 07:32:47,757][20381] Updated weights for policy 1, policy_version 273946 (0.0008) [2023-12-27 07:32:47,762][20347] Updated weights for policy 0, policy_version 274312 (0.0007) [2023-12-27 07:32:47,814][20381] Updated weights for policy 1, policy_version 273956 (0.0006) [2023-12-27 07:32:47,820][20347] Updated weights for policy 0, policy_version 274322 (0.0006) [2023-12-27 07:32:47,885][20347] Updated weights for policy 0, policy_version 274332 (0.0007) [2023-12-27 07:32:48,531][20381] Updated weights for policy 1, policy_version 273966 (0.0007) [2023-12-27 07:32:48,549][20347] Updated weights for policy 0, policy_version 274342 (0.0007) [2023-12-27 07:32:48,581][20381] Updated weights for policy 1, policy_version 273976 (0.0007) [2023-12-27 07:32:48,603][20347] Updated weights for policy 0, policy_version 274352 (0.0006) [2023-12-27 07:32:48,632][20381] Updated weights for policy 1, policy_version 273986 (0.0006) [2023-12-27 07:32:48,664][20347] Updated weights for policy 0, policy_version 274362 (0.0006) [2023-12-27 07:32:49,287][20381] Updated weights for policy 1, policy_version 273996 (0.0006) [2023-12-27 07:32:49,348][20381] Updated weights for policy 1, policy_version 274006 (0.0008) [2023-12-27 07:32:49,392][20292] KL-divergence is very high: 145.2655 [2023-12-27 07:32:49,404][20381] Updated weights for policy 1, policy_version 274016 (0.0007) [2023-12-27 07:32:49,406][20347] Updated weights for policy 0, policy_version 274372 (0.0008) [2023-12-27 07:32:49,460][20347] Updated weights for policy 0, policy_version 274382 (0.0006) [2023-12-27 07:32:49,519][20347] Updated weights for policy 0, policy_version 274392 (0.0008) [2023-12-27 07:32:50,132][20381] Updated weights for policy 1, policy_version 274026 (0.0007) [2023-12-27 07:32:50,177][20347] Updated weights for policy 0, policy_version 274402 (0.0008) [2023-12-27 07:32:50,191][20381] Updated weights for policy 1, policy_version 274036 (0.0007) [2023-12-27 07:32:50,225][20347] Updated weights for policy 0, policy_version 274412 (0.0007) [2023-12-27 07:32:50,246][20381] Updated weights for policy 1, policy_version 274046 (0.0006) [2023-12-27 07:32:50,272][20347] Updated weights for policy 0, policy_version 274422 (0.0006) [2023-12-27 07:32:50,310][20381] Updated weights for policy 1, policy_version 274056 (0.0008) [2023-12-27 07:32:50,329][20347] Updated weights for policy 0, policy_version 274432 (0.0009) [2023-12-27 07:32:50,954][20381] Updated weights for policy 1, policy_version 274066 (0.0008) [2023-12-27 07:32:51,019][20381] Updated weights for policy 1, policy_version 274076 (0.0008) [2023-12-27 07:32:51,084][20381] Updated weights for policy 1, policy_version 274086 (0.0010) [2023-12-27 07:32:51,094][20347] Updated weights for policy 0, policy_version 274442 (0.0007) [2023-12-27 07:32:51,153][20347] Updated weights for policy 0, policy_version 274452 (0.0008) [2023-12-27 07:32:51,205][20347] Updated weights for policy 0, policy_version 274462 (0.0007) [2023-12-27 07:32:51,807][20381] Updated weights for policy 1, policy_version 274096 (0.0011) [2023-12-27 07:32:51,869][20381] Updated weights for policy 1, policy_version 274106 (0.0010) [2023-12-27 07:32:51,908][20347] Updated weights for policy 0, policy_version 274472 (0.0006) [2023-12-27 07:32:51,922][20381] Updated weights for policy 1, policy_version 274116 (0.0010) [2023-12-27 07:32:51,966][20347] Updated weights for policy 0, policy_version 274482 (0.0006) [2023-12-27 07:32:52,020][20347] Updated weights for policy 0, policy_version 274492 (0.0008) [2023-12-27 07:32:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 140476416. Throughput: 0: 10351.5, 1: 10397.7. Samples: 140446624. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:32:52,363][19170] Avg episode reward: [(0, '874.340'), (1, '942.220')] [2023-12-27 07:32:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000274120_70189056.pth... [2023-12-27 07:32:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000274496_70287360.pth... [2023-12-27 07:32:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000272904_69877760.pth [2023-12-27 07:32:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000273280_69976064.pth [2023-12-27 07:32:52,658][20347] Updated weights for policy 0, policy_version 274502 (0.0006) [2023-12-27 07:32:52,667][20381] Updated weights for policy 1, policy_version 274126 (0.0008) [2023-12-27 07:32:52,671][20291] KL-divergence is very high: 151.9832 [2023-12-27 07:32:52,714][20347] Updated weights for policy 0, policy_version 274512 (0.0006) [2023-12-27 07:32:52,720][20381] Updated weights for policy 1, policy_version 274136 (0.0005) [2023-12-27 07:32:52,756][20291] KL-divergence is very high: 114.3965 [2023-12-27 07:32:52,774][20347] Updated weights for policy 0, policy_version 274522 (0.0007) [2023-12-27 07:32:52,780][20381] Updated weights for policy 1, policy_version 274146 (0.0005) [2023-12-27 07:32:53,384][20347] Updated weights for policy 0, policy_version 274532 (0.0009) [2023-12-27 07:32:53,392][20381] Updated weights for policy 1, policy_version 274156 (0.0006) [2023-12-27 07:32:53,438][20347] Updated weights for policy 0, policy_version 274542 (0.0010) [2023-12-27 07:32:53,448][20381] Updated weights for policy 1, policy_version 274166 (0.0006) [2023-12-27 07:32:53,487][20347] Updated weights for policy 0, policy_version 274552 (0.0010) [2023-12-27 07:32:53,507][20381] Updated weights for policy 1, policy_version 274176 (0.0006) [2023-12-27 07:32:54,130][20381] Updated weights for policy 1, policy_version 274186 (0.0007) [2023-12-27 07:32:54,164][20347] Updated weights for policy 0, policy_version 274562 (0.0010) [2023-12-27 07:32:54,186][20381] Updated weights for policy 1, policy_version 274196 (0.0007) [2023-12-27 07:32:54,213][20347] Updated weights for policy 0, policy_version 274572 (0.0010) [2023-12-27 07:32:54,240][20381] Updated weights for policy 1, policy_version 274206 (0.0005) [2023-12-27 07:32:54,274][20347] Updated weights for policy 0, policy_version 274582 (0.0010) [2023-12-27 07:32:54,295][20381] Updated weights for policy 1, policy_version 274216 (0.0006) [2023-12-27 07:32:54,335][20347] Updated weights for policy 0, policy_version 274592 (0.0010) [2023-12-27 07:32:54,989][20381] Updated weights for policy 1, policy_version 274226 (0.0007) [2023-12-27 07:32:55,013][20347] Updated weights for policy 0, policy_version 274602 (0.0010) [2023-12-27 07:32:55,049][20381] Updated weights for policy 1, policy_version 274236 (0.0006) [2023-12-27 07:32:55,067][20347] Updated weights for policy 0, policy_version 274612 (0.0007) [2023-12-27 07:32:55,109][20381] Updated weights for policy 1, policy_version 274246 (0.0008) [2023-12-27 07:32:55,119][20347] Updated weights for policy 0, policy_version 274622 (0.0007) [2023-12-27 07:32:55,701][20347] Updated weights for policy 0, policy_version 274632 (0.0005) [2023-12-27 07:32:55,748][20347] Updated weights for policy 0, policy_version 274642 (0.0005) [2023-12-27 07:32:55,791][20347] Updated weights for policy 0, policy_version 274652 (0.0005) [2023-12-27 07:32:55,841][20381] Updated weights for policy 1, policy_version 274256 (0.0008) [2023-12-27 07:32:55,910][20381] Updated weights for policy 1, policy_version 274266 (0.0008) [2023-12-27 07:32:55,961][20381] Updated weights for policy 1, policy_version 274276 (0.0008) [2023-12-27 07:32:56,435][20347] Updated weights for policy 0, policy_version 274662 (0.0008) [2023-12-27 07:32:56,486][20347] Updated weights for policy 0, policy_version 274672 (0.0010) [2023-12-27 07:32:56,536][20347] Updated weights for policy 0, policy_version 274682 (0.0010) [2023-12-27 07:32:56,601][20381] Updated weights for policy 1, policy_version 274286 (0.0007) [2023-12-27 07:32:56,661][20381] Updated weights for policy 1, policy_version 274296 (0.0005) [2023-12-27 07:32:56,720][20381] Updated weights for policy 1, policy_version 274306 (0.0005) [2023-12-27 07:32:57,152][20347] Updated weights for policy 0, policy_version 274692 (0.0009) [2023-12-27 07:32:57,202][20347] Updated weights for policy 0, policy_version 274702 (0.0009) [2023-12-27 07:32:57,258][20347] Updated weights for policy 0, policy_version 274712 (0.0007) [2023-12-27 07:32:57,266][20381] Updated weights for policy 1, policy_version 274316 (0.0005) [2023-12-27 07:32:57,316][20381] Updated weights for policy 1, policy_version 274326 (0.0007) [2023-12-27 07:32:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 140582912. Throughput: 0: 10410.9, 1: 10418.0. Samples: 140570692. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:32:57,363][19170] Avg episode reward: [(0, '922.650'), (1, '951.600')] [2023-12-27 07:32:57,371][20381] Updated weights for policy 1, policy_version 274336 (0.0006) [2023-12-27 07:32:57,873][20347] Updated weights for policy 0, policy_version 274722 (0.0007) [2023-12-27 07:32:57,917][20347] Updated weights for policy 0, policy_version 274732 (0.0007) [2023-12-27 07:32:57,965][20347] Updated weights for policy 0, policy_version 274742 (0.0006) [2023-12-27 07:32:58,013][20381] Updated weights for policy 1, policy_version 274346 (0.0006) [2023-12-27 07:32:58,016][20347] Updated weights for policy 0, policy_version 274752 (0.0006) [2023-12-27 07:32:58,067][20381] Updated weights for policy 1, policy_version 274356 (0.0007) [2023-12-27 07:32:58,116][20381] Updated weights for policy 1, policy_version 274366 (0.0008) [2023-12-27 07:32:58,168][20381] Updated weights for policy 1, policy_version 274376 (0.0008) [2023-12-27 07:32:58,764][20347] Updated weights for policy 0, policy_version 274762 (0.0008) [2023-12-27 07:32:58,836][20347] Updated weights for policy 0, policy_version 274772 (0.0007) [2023-12-27 07:32:58,859][20381] Updated weights for policy 1, policy_version 274386 (0.0007) [2023-12-27 07:32:58,901][20347] Updated weights for policy 0, policy_version 274782 (0.0008) [2023-12-27 07:32:58,925][20381] Updated weights for policy 1, policy_version 274396 (0.0009) [2023-12-27 07:32:58,983][20381] Updated weights for policy 1, policy_version 274406 (0.0008) [2023-12-27 07:32:59,687][20347] Updated weights for policy 0, policy_version 274792 (0.0007) [2023-12-27 07:32:59,697][20381] Updated weights for policy 1, policy_version 274416 (0.0008) [2023-12-27 07:32:59,742][20347] Updated weights for policy 0, policy_version 274802 (0.0006) [2023-12-27 07:32:59,752][20381] Updated weights for policy 1, policy_version 274426 (0.0007) [2023-12-27 07:32:59,793][20347] Updated weights for policy 0, policy_version 274812 (0.0005) [2023-12-27 07:32:59,810][20381] Updated weights for policy 1, policy_version 274436 (0.0006) [2023-12-27 07:33:00,430][20381] Updated weights for policy 1, policy_version 274446 (0.0008) [2023-12-27 07:33:00,489][20381] Updated weights for policy 1, policy_version 274456 (0.0007) [2023-12-27 07:33:00,527][20347] Updated weights for policy 0, policy_version 274822 (0.0006) [2023-12-27 07:33:00,536][20381] Updated weights for policy 1, policy_version 274466 (0.0007) [2023-12-27 07:33:00,584][20347] Updated weights for policy 0, policy_version 274832 (0.0007) [2023-12-27 07:33:00,639][20347] Updated weights for policy 0, policy_version 274842 (0.0008) [2023-12-27 07:33:01,258][20381] Updated weights for policy 1, policy_version 274476 (0.0007) [2023-12-27 07:33:01,303][20347] Updated weights for policy 0, policy_version 274852 (0.0007) [2023-12-27 07:33:01,312][20381] Updated weights for policy 1, policy_version 274486 (0.0007) [2023-12-27 07:33:01,366][20347] Updated weights for policy 0, policy_version 274862 (0.0010) [2023-12-27 07:33:01,371][20381] Updated weights for policy 1, policy_version 274496 (0.0007) [2023-12-27 07:33:01,422][20347] Updated weights for policy 0, policy_version 274872 (0.0008) [2023-12-27 07:33:01,994][20381] Updated weights for policy 1, policy_version 274506 (0.0006) [2023-12-27 07:33:02,053][20381] Updated weights for policy 1, policy_version 274516 (0.0007) [2023-12-27 07:33:02,111][20381] Updated weights for policy 1, policy_version 274526 (0.0007) [2023-12-27 07:33:02,111][20347] Updated weights for policy 0, policy_version 274882 (0.0006) [2023-12-27 07:33:02,164][20347] Updated weights for policy 0, policy_version 274892 (0.0006) [2023-12-27 07:33:02,169][20381] Updated weights for policy 1, policy_version 274536 (0.0008) [2023-12-27 07:33:02,217][20347] Updated weights for policy 0, policy_version 274902 (0.0006) [2023-12-27 07:33:02,277][20347] Updated weights for policy 0, policy_version 274912 (0.0007) [2023-12-27 07:33:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 140689408. Throughput: 0: 10423.5, 1: 10361.2. Samples: 140694836. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:02,363][19170] Avg episode reward: [(0, '855.260'), (1, '980.680')] [2023-12-27 07:33:02,890][20381] Updated weights for policy 1, policy_version 274546 (0.0010) [2023-12-27 07:33:02,932][20347] Updated weights for policy 0, policy_version 274922 (0.0008) [2023-12-27 07:33:02,940][20381] Updated weights for policy 1, policy_version 274556 (0.0010) [2023-12-27 07:33:02,986][20347] Updated weights for policy 0, policy_version 274932 (0.0005) [2023-12-27 07:33:02,995][20381] Updated weights for policy 1, policy_version 274566 (0.0010) [2023-12-27 07:33:03,043][20347] Updated weights for policy 0, policy_version 274942 (0.0006) [2023-12-27 07:33:03,574][20381] Updated weights for policy 1, policy_version 274576 (0.0006) [2023-12-27 07:33:03,619][20381] Updated weights for policy 1, policy_version 274586 (0.0008) [2023-12-27 07:33:03,667][20381] Updated weights for policy 1, policy_version 274596 (0.0007) [2023-12-27 07:33:03,727][20347] Updated weights for policy 0, policy_version 274952 (0.0007) [2023-12-27 07:33:03,775][20347] Updated weights for policy 0, policy_version 274962 (0.0006) [2023-12-27 07:33:03,825][20347] Updated weights for policy 0, policy_version 274972 (0.0005) [2023-12-27 07:33:04,374][20381] Updated weights for policy 1, policy_version 274606 (0.0009) [2023-12-27 07:33:04,423][20381] Updated weights for policy 1, policy_version 274616 (0.0010) [2023-12-27 07:33:04,480][20381] Updated weights for policy 1, policy_version 274626 (0.0010) [2023-12-27 07:33:04,525][20347] Updated weights for policy 0, policy_version 274982 (0.0007) [2023-12-27 07:33:04,580][20347] Updated weights for policy 0, policy_version 274992 (0.0008) [2023-12-27 07:33:04,632][20347] Updated weights for policy 0, policy_version 275002 (0.0007) [2023-12-27 07:33:05,126][20381] Updated weights for policy 1, policy_version 274636 (0.0008) [2023-12-27 07:33:05,186][20381] Updated weights for policy 1, policy_version 274646 (0.0005) [2023-12-27 07:33:05,242][20381] Updated weights for policy 1, policy_version 274656 (0.0005) [2023-12-27 07:33:05,248][20347] Updated weights for policy 0, policy_version 275012 (0.0008) [2023-12-27 07:33:05,304][20347] Updated weights for policy 0, policy_version 275022 (0.0010) [2023-12-27 07:33:05,358][20347] Updated weights for policy 0, policy_version 275032 (0.0010) [2023-12-27 07:33:05,797][20381] Updated weights for policy 1, policy_version 274666 (0.0005) [2023-12-27 07:33:05,848][20381] Updated weights for policy 1, policy_version 274676 (0.0005) [2023-12-27 07:33:05,905][20381] Updated weights for policy 1, policy_version 274686 (0.0007) [2023-12-27 07:33:05,960][20381] Updated weights for policy 1, policy_version 274696 (0.0010) [2023-12-27 07:33:06,021][20347] Updated weights for policy 0, policy_version 275042 (0.0010) [2023-12-27 07:33:06,074][20347] Updated weights for policy 0, policy_version 275052 (0.0006) [2023-12-27 07:33:06,136][20347] Updated weights for policy 0, policy_version 275062 (0.0010) [2023-12-27 07:33:06,194][20347] Updated weights for policy 0, policy_version 275072 (0.0009) [2023-12-27 07:33:06,612][20381] Updated weights for policy 1, policy_version 274706 (0.0008) [2023-12-27 07:33:06,664][20381] Updated weights for policy 1, policy_version 274716 (0.0007) [2023-12-27 07:33:06,728][20381] Updated weights for policy 1, policy_version 274726 (0.0006) [2023-12-27 07:33:06,776][20347] Updated weights for policy 0, policy_version 275082 (0.0007) [2023-12-27 07:33:06,825][20347] Updated weights for policy 0, policy_version 275092 (0.0007) [2023-12-27 07:33:06,869][20347] Updated weights for policy 0, policy_version 275102 (0.0008) [2023-12-27 07:33:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 140787712. Throughput: 0: 10406.4, 1: 10396.6. Samples: 140757888. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:07,363][19170] Avg episode reward: [(0, '865.210'), (1, '990.400')] [2023-12-27 07:33:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000275104_70443008.pth... [2023-12-27 07:33:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000273920_70139904.pth [2023-12-27 07:33:07,397][20381] Updated weights for policy 1, policy_version 274736 (0.0011) [2023-12-27 07:33:07,455][20381] Updated weights for policy 1, policy_version 274746 (0.0010) [2023-12-27 07:33:07,508][20381] Updated weights for policy 1, policy_version 274756 (0.0007) [2023-12-27 07:33:07,528][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000274760_70352896.pth... [2023-12-27 07:33:07,531][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000273512_70033408.pth [2023-12-27 07:33:07,592][20347] Updated weights for policy 0, policy_version 275112 (0.0007) [2023-12-27 07:33:07,646][20347] Updated weights for policy 0, policy_version 275122 (0.0006) [2023-12-27 07:33:07,695][20347] Updated weights for policy 0, policy_version 275132 (0.0007) [2023-12-27 07:33:08,180][20381] Updated weights for policy 1, policy_version 274766 (0.0008) [2023-12-27 07:33:08,232][20381] Updated weights for policy 1, policy_version 274776 (0.0010) [2023-12-27 07:33:08,283][20381] Updated weights for policy 1, policy_version 274786 (0.0010) [2023-12-27 07:33:08,385][20347] Updated weights for policy 0, policy_version 275142 (0.0008) [2023-12-27 07:33:08,448][20347] Updated weights for policy 0, policy_version 275152 (0.0006) [2023-12-27 07:33:08,505][20347] Updated weights for policy 0, policy_version 275162 (0.0008) [2023-12-27 07:33:08,982][20381] Updated weights for policy 1, policy_version 274796 (0.0010) [2023-12-27 07:33:09,031][20381] Updated weights for policy 1, policy_version 274806 (0.0010) [2023-12-27 07:33:09,086][20381] Updated weights for policy 1, policy_version 274816 (0.0011) [2023-12-27 07:33:09,190][20347] Updated weights for policy 0, policy_version 275172 (0.0008) [2023-12-27 07:33:09,253][20347] Updated weights for policy 0, policy_version 275182 (0.0007) [2023-12-27 07:33:09,339][20347] Updated weights for policy 0, policy_version 275192 (0.0008) [2023-12-27 07:33:09,812][20381] Updated weights for policy 1, policy_version 274826 (0.0010) [2023-12-27 07:33:09,874][20381] Updated weights for policy 1, policy_version 274836 (0.0008) [2023-12-27 07:33:09,939][20381] Updated weights for policy 1, policy_version 274846 (0.0009) [2023-12-27 07:33:09,999][20381] Updated weights for policy 1, policy_version 274856 (0.0008) [2023-12-27 07:33:10,072][20347] Updated weights for policy 0, policy_version 275202 (0.0007) [2023-12-27 07:33:10,130][20347] Updated weights for policy 0, policy_version 275212 (0.0008) [2023-12-27 07:33:10,194][20347] Updated weights for policy 0, policy_version 275222 (0.0007) [2023-12-27 07:33:10,255][20347] Updated weights for policy 0, policy_version 275232 (0.0008) [2023-12-27 07:33:10,713][20381] Updated weights for policy 1, policy_version 274866 (0.0007) [2023-12-27 07:33:10,759][20381] Updated weights for policy 1, policy_version 274876 (0.0008) [2023-12-27 07:33:10,811][20381] Updated weights for policy 1, policy_version 274886 (0.0008) [2023-12-27 07:33:10,927][20291] KL-divergence is very high: 177.3723 [2023-12-27 07:33:10,945][20347] Updated weights for policy 0, policy_version 275242 (0.0008) [2023-12-27 07:33:10,947][20291] KL-divergence is very high: 109.0693 [2023-12-27 07:33:10,959][20291] KL-divergence is very high: 104.7431 [2023-12-27 07:33:10,976][20291] KL-divergence is very high: 125.6360 [2023-12-27 07:33:10,987][20291] KL-divergence is very high: 100.9705 [2023-12-27 07:33:11,004][20347] Updated weights for policy 0, policy_version 275252 (0.0007) [2023-12-27 07:33:11,067][20347] Updated weights for policy 0, policy_version 275262 (0.0008) [2023-12-27 07:33:11,556][20381] Updated weights for policy 1, policy_version 274896 (0.0007) [2023-12-27 07:33:11,619][20381] Updated weights for policy 1, policy_version 274906 (0.0008) [2023-12-27 07:33:11,692][20381] Updated weights for policy 1, policy_version 274916 (0.0007) [2023-12-27 07:33:11,747][20291] KL-divergence is very high: 114.1501 [2023-12-27 07:33:11,754][20347] Updated weights for policy 0, policy_version 275272 (0.0008) [2023-12-27 07:33:11,800][20291] KL-divergence is very high: 143.9382 [2023-12-27 07:33:11,804][20347] Updated weights for policy 0, policy_version 275282 (0.0009) [2023-12-27 07:33:11,821][20291] KL-divergence is very high: 108.2942 [2023-12-27 07:33:11,857][20347] Updated weights for policy 0, policy_version 275292 (0.0007) [2023-12-27 07:33:11,873][20291] KL-divergence is very high: 151.1396 [2023-12-27 07:33:12,318][20381] Updated weights for policy 1, policy_version 274926 (0.0007) [2023-12-27 07:33:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 140886016. Throughput: 0: 10386.5, 1: 10333.6. Samples: 140880048. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:12,363][19170] Avg episode reward: [(0, '904.320'), (1, '990.430')] [2023-12-27 07:33:12,397][20381] Updated weights for policy 1, policy_version 274936 (0.0008) [2023-12-27 07:33:12,468][20381] Updated weights for policy 1, policy_version 274946 (0.0007) [2023-12-27 07:33:12,544][20347] Updated weights for policy 0, policy_version 275302 (0.0007) [2023-12-27 07:33:12,598][20347] Updated weights for policy 0, policy_version 275312 (0.0008) [2023-12-27 07:33:12,652][20347] Updated weights for policy 0, policy_version 275322 (0.0006) [2023-12-27 07:33:13,108][20381] Updated weights for policy 1, policy_version 274956 (0.0007) [2023-12-27 07:33:13,164][20381] Updated weights for policy 1, policy_version 274966 (0.0006) [2023-12-27 07:33:13,224][20381] Updated weights for policy 1, policy_version 274976 (0.0008) [2023-12-27 07:33:13,315][20347] Updated weights for policy 0, policy_version 275332 (0.0006) [2023-12-27 07:33:13,372][20347] Updated weights for policy 0, policy_version 275342 (0.0007) [2023-12-27 07:33:13,421][20347] Updated weights for policy 0, policy_version 275352 (0.0007) [2023-12-27 07:33:13,866][20381] Updated weights for policy 1, policy_version 274986 (0.0008) [2023-12-27 07:33:13,919][20381] Updated weights for policy 1, policy_version 274996 (0.0008) [2023-12-27 07:33:13,981][20381] Updated weights for policy 1, policy_version 275006 (0.0007) [2023-12-27 07:33:14,026][20347] Updated weights for policy 0, policy_version 275362 (0.0007) [2023-12-27 07:33:14,035][20381] Updated weights for policy 1, policy_version 275016 (0.0010) [2023-12-27 07:33:14,074][20347] Updated weights for policy 0, policy_version 275372 (0.0007) [2023-12-27 07:33:14,129][20347] Updated weights for policy 0, policy_version 275382 (0.0007) [2023-12-27 07:33:14,190][20347] Updated weights for policy 0, policy_version 275392 (0.0007) [2023-12-27 07:33:14,671][20381] Updated weights for policy 1, policy_version 275026 (0.0006) [2023-12-27 07:33:14,683][20292] KL-divergence is very high: 175.4562 [2023-12-27 07:33:14,726][20381] Updated weights for policy 1, policy_version 275036 (0.0005) [2023-12-27 07:33:14,788][20381] Updated weights for policy 1, policy_version 275046 (0.0007) [2023-12-27 07:33:14,897][20347] Updated weights for policy 0, policy_version 275402 (0.0008) [2023-12-27 07:33:14,955][20347] Updated weights for policy 0, policy_version 275412 (0.0008) [2023-12-27 07:33:15,023][20347] Updated weights for policy 0, policy_version 275422 (0.0007) [2023-12-27 07:33:15,416][20381] Updated weights for policy 1, policy_version 275056 (0.0010) [2023-12-27 07:33:15,479][20381] Updated weights for policy 1, policy_version 275066 (0.0008) [2023-12-27 07:33:15,538][20381] Updated weights for policy 1, policy_version 275076 (0.0011) [2023-12-27 07:33:15,637][20347] Updated weights for policy 0, policy_version 275432 (0.0007) [2023-12-27 07:33:15,698][20347] Updated weights for policy 0, policy_version 275442 (0.0007) [2023-12-27 07:33:15,762][20347] Updated weights for policy 0, policy_version 275452 (0.0009) [2023-12-27 07:33:16,155][20381] Updated weights for policy 1, policy_version 275086 (0.0007) [2023-12-27 07:33:16,221][20381] Updated weights for policy 1, policy_version 275096 (0.0007) [2023-12-27 07:33:16,272][20381] Updated weights for policy 1, policy_version 275106 (0.0010) [2023-12-27 07:33:16,335][20347] Updated weights for policy 0, policy_version 275462 (0.0009) [2023-12-27 07:33:16,395][20347] Updated weights for policy 0, policy_version 275472 (0.0008) [2023-12-27 07:33:16,446][20347] Updated weights for policy 0, policy_version 275482 (0.0007) [2023-12-27 07:33:16,853][20381] Updated weights for policy 1, policy_version 275116 (0.0008) [2023-12-27 07:33:16,904][20381] Updated weights for policy 1, policy_version 275126 (0.0008) [2023-12-27 07:33:16,953][20381] Updated weights for policy 1, policy_version 275136 (0.0008) [2023-12-27 07:33:17,079][20347] Updated weights for policy 0, policy_version 275492 (0.0005) [2023-12-27 07:33:17,134][20347] Updated weights for policy 0, policy_version 275502 (0.0006) [2023-12-27 07:33:17,186][20347] Updated weights for policy 0, policy_version 275512 (0.0006) [2023-12-27 07:33:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 141000704. Throughput: 0: 10360.3, 1: 10400.2. Samples: 141008440. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:17,363][19170] Avg episode reward: [(0, '752.830'), (1, '980.810')] [2023-12-27 07:33:17,618][20381] Updated weights for policy 1, policy_version 275146 (0.0006) [2023-12-27 07:33:17,663][20381] Updated weights for policy 1, policy_version 275156 (0.0009) [2023-12-27 07:33:17,714][20381] Updated weights for policy 1, policy_version 275166 (0.0010) [2023-12-27 07:33:17,759][20381] Updated weights for policy 1, policy_version 275176 (0.0010) [2023-12-27 07:33:17,793][20347] Updated weights for policy 0, policy_version 275522 (0.0006) [2023-12-27 07:33:17,846][20347] Updated weights for policy 0, policy_version 275532 (0.0007) [2023-12-27 07:33:17,901][20347] Updated weights for policy 0, policy_version 275542 (0.0007) [2023-12-27 07:33:17,947][20347] Updated weights for policy 0, policy_version 275552 (0.0010) [2023-12-27 07:33:18,383][20381] Updated weights for policy 1, policy_version 275186 (0.0011) [2023-12-27 07:33:18,435][20381] Updated weights for policy 1, policy_version 275196 (0.0010) [2023-12-27 07:33:18,488][20381] Updated weights for policy 1, policy_version 275206 (0.0009) [2023-12-27 07:33:18,668][20347] Updated weights for policy 0, policy_version 275562 (0.0008) [2023-12-27 07:33:18,724][20347] Updated weights for policy 0, policy_version 275572 (0.0008) [2023-12-27 07:33:18,780][20347] Updated weights for policy 0, policy_version 275582 (0.0008) [2023-12-27 07:33:19,205][20381] Updated weights for policy 1, policy_version 275216 (0.0007) [2023-12-27 07:33:19,276][20381] Updated weights for policy 1, policy_version 275226 (0.0008) [2023-12-27 07:33:19,335][20381] Updated weights for policy 1, policy_version 275236 (0.0006) [2023-12-27 07:33:19,498][20347] Updated weights for policy 0, policy_version 275592 (0.0007) [2023-12-27 07:33:19,558][20347] Updated weights for policy 0, policy_version 275602 (0.0007) [2023-12-27 07:33:19,620][20347] Updated weights for policy 0, policy_version 275612 (0.0008) [2023-12-27 07:33:19,997][20381] Updated weights for policy 1, policy_version 275246 (0.0010) [2023-12-27 07:33:20,061][20381] Updated weights for policy 1, policy_version 275256 (0.0011) [2023-12-27 07:33:20,115][20381] Updated weights for policy 1, policy_version 275266 (0.0011) [2023-12-27 07:33:20,304][20347] Updated weights for policy 0, policy_version 275622 (0.0008) [2023-12-27 07:33:20,362][20347] Updated weights for policy 0, policy_version 275632 (0.0007) [2023-12-27 07:33:20,424][20347] Updated weights for policy 0, policy_version 275642 (0.0008) [2023-12-27 07:33:20,816][20381] Updated weights for policy 1, policy_version 275276 (0.0011) [2023-12-27 07:33:20,883][20381] Updated weights for policy 1, policy_version 275286 (0.0010) [2023-12-27 07:33:20,948][20381] Updated weights for policy 1, policy_version 275296 (0.0007) [2023-12-27 07:33:21,087][20347] Updated weights for policy 0, policy_version 275652 (0.0007) [2023-12-27 07:33:21,148][20347] Updated weights for policy 0, policy_version 275662 (0.0007) [2023-12-27 07:33:21,206][20347] Updated weights for policy 0, policy_version 275672 (0.0007) [2023-12-27 07:33:21,752][20381] Updated weights for policy 1, policy_version 275306 (0.0008) [2023-12-27 07:33:21,817][20381] Updated weights for policy 1, policy_version 275316 (0.0008) [2023-12-27 07:33:21,871][20381] Updated weights for policy 1, policy_version 275326 (0.0007) [2023-12-27 07:33:21,927][20347] Updated weights for policy 0, policy_version 275682 (0.0008) [2023-12-27 07:33:21,927][20381] Updated weights for policy 1, policy_version 275336 (0.0008) [2023-12-27 07:33:21,989][20347] Updated weights for policy 0, policy_version 275692 (0.0007) [2023-12-27 07:33:22,050][20347] Updated weights for policy 0, policy_version 275702 (0.0008) [2023-12-27 07:33:22,103][20347] Updated weights for policy 0, policy_version 275712 (0.0008) [2023-12-27 07:33:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 141099008. Throughput: 0: 10336.4, 1: 10367.2. Samples: 141069276. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:22,363][19170] Avg episode reward: [(0, '804.520'), (1, '990.380')] [2023-12-27 07:33:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000275336_70500352.pth... [2023-12-27 07:33:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000275712_70598656.pth... [2023-12-27 07:33:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000274496_70287360.pth [2023-12-27 07:33:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000274120_70189056.pth [2023-12-27 07:33:22,622][20381] Updated weights for policy 1, policy_version 275346 (0.0008) [2023-12-27 07:33:22,681][20381] Updated weights for policy 1, policy_version 275356 (0.0006) [2023-12-27 07:33:22,743][20381] Updated weights for policy 1, policy_version 275366 (0.0006) [2023-12-27 07:33:22,824][20347] Updated weights for policy 0, policy_version 275722 (0.0007) [2023-12-27 07:33:22,878][20347] Updated weights for policy 0, policy_version 275732 (0.0006) [2023-12-27 07:33:22,925][20347] Updated weights for policy 0, policy_version 275742 (0.0005) [2023-12-27 07:33:23,355][20381] Updated weights for policy 1, policy_version 275376 (0.0009) [2023-12-27 07:33:23,401][20381] Updated weights for policy 1, policy_version 275386 (0.0005) [2023-12-27 07:33:23,449][20381] Updated weights for policy 1, policy_version 275396 (0.0007) [2023-12-27 07:33:23,544][20347] Updated weights for policy 0, policy_version 275752 (0.0006) [2023-12-27 07:33:23,603][20347] Updated weights for policy 0, policy_version 275762 (0.0007) [2023-12-27 07:33:23,650][20347] Updated weights for policy 0, policy_version 275772 (0.0008) [2023-12-27 07:33:24,094][20381] Updated weights for policy 1, policy_version 275406 (0.0008) [2023-12-27 07:33:24,139][20381] Updated weights for policy 1, policy_version 275416 (0.0008) [2023-12-27 07:33:24,185][20381] Updated weights for policy 1, policy_version 275426 (0.0008) [2023-12-27 07:33:24,212][20347] Updated weights for policy 0, policy_version 275782 (0.0006) [2023-12-27 07:33:24,268][20347] Updated weights for policy 0, policy_version 275792 (0.0005) [2023-12-27 07:33:24,322][20347] Updated weights for policy 0, policy_version 275802 (0.0007) [2023-12-27 07:33:24,915][20381] Updated weights for policy 1, policy_version 275436 (0.0007) [2023-12-27 07:33:24,941][20347] Updated weights for policy 0, policy_version 275812 (0.0007) [2023-12-27 07:33:24,972][20381] Updated weights for policy 1, policy_version 275446 (0.0006) [2023-12-27 07:33:24,992][20347] Updated weights for policy 0, policy_version 275822 (0.0007) [2023-12-27 07:33:25,024][20381] Updated weights for policy 1, policy_version 275456 (0.0005) [2023-12-27 07:33:25,043][20347] Updated weights for policy 0, policy_version 275832 (0.0007) [2023-12-27 07:33:25,650][20381] Updated weights for policy 1, policy_version 275466 (0.0006) [2023-12-27 07:33:25,675][20347] Updated weights for policy 0, policy_version 275842 (0.0006) [2023-12-27 07:33:25,699][20381] Updated weights for policy 1, policy_version 275476 (0.0006) [2023-12-27 07:33:25,720][20347] Updated weights for policy 0, policy_version 275852 (0.0005) [2023-12-27 07:33:25,743][20381] Updated weights for policy 1, policy_version 275486 (0.0005) [2023-12-27 07:33:25,779][20347] Updated weights for policy 0, policy_version 275862 (0.0007) [2023-12-27 07:33:25,792][20381] Updated weights for policy 1, policy_version 275496 (0.0005) [2023-12-27 07:33:25,834][20347] Updated weights for policy 0, policy_version 275872 (0.0007) [2023-12-27 07:33:26,452][20381] Updated weights for policy 1, policy_version 275506 (0.0009) [2023-12-27 07:33:26,478][20347] Updated weights for policy 0, policy_version 275882 (0.0005) [2023-12-27 07:33:26,494][20381] Updated weights for policy 1, policy_version 275516 (0.0010) [2023-12-27 07:33:26,543][20347] Updated weights for policy 0, policy_version 275892 (0.0005) [2023-12-27 07:33:26,548][20381] Updated weights for policy 1, policy_version 275526 (0.0010) [2023-12-27 07:33:26,596][20347] Updated weights for policy 0, policy_version 275902 (0.0005) [2023-12-27 07:33:27,102][20347] Updated weights for policy 0, policy_version 275912 (0.0008) [2023-12-27 07:33:27,158][20347] Updated weights for policy 0, policy_version 275922 (0.0010) [2023-12-27 07:33:27,201][20381] Updated weights for policy 1, policy_version 275536 (0.0006) [2023-12-27 07:33:27,212][20347] Updated weights for policy 0, policy_version 275932 (0.0010) [2023-12-27 07:33:27,253][20381] Updated weights for policy 1, policy_version 275546 (0.0005) [2023-12-27 07:33:27,298][20381] Updated weights for policy 1, policy_version 275556 (0.0009) [2023-12-27 07:33:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 141213696. Throughput: 0: 10346.8, 1: 10408.4. Samples: 141194532. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:27,363][19170] Avg episode reward: [(0, '809.400'), (1, '990.380')] [2023-12-27 07:33:27,799][20347] Updated weights for policy 0, policy_version 275942 (0.0007) [2023-12-27 07:33:27,855][20347] Updated weights for policy 0, policy_version 275952 (0.0005) [2023-12-27 07:33:27,915][20347] Updated weights for policy 0, policy_version 275962 (0.0005) [2023-12-27 07:33:27,922][20381] Updated weights for policy 1, policy_version 275566 (0.0007) [2023-12-27 07:33:27,971][20381] Updated weights for policy 1, policy_version 275576 (0.0007) [2023-12-27 07:33:28,022][20381] Updated weights for policy 1, policy_version 275586 (0.0006) [2023-12-27 07:33:28,527][20347] Updated weights for policy 0, policy_version 275972 (0.0006) [2023-12-27 07:33:28,584][20347] Updated weights for policy 0, policy_version 275982 (0.0007) [2023-12-27 07:33:28,637][20381] Updated weights for policy 1, policy_version 275596 (0.0005) [2023-12-27 07:33:28,642][20347] Updated weights for policy 0, policy_version 275992 (0.0008) [2023-12-27 07:33:28,689][20381] Updated weights for policy 1, policy_version 275606 (0.0005) [2023-12-27 07:33:28,745][20381] Updated weights for policy 1, policy_version 275616 (0.0006) [2023-12-27 07:33:29,336][20381] Updated weights for policy 1, policy_version 275626 (0.0006) [2023-12-27 07:33:29,376][20347] Updated weights for policy 0, policy_version 276002 (0.0007) [2023-12-27 07:33:29,393][20381] Updated weights for policy 1, policy_version 275636 (0.0007) [2023-12-27 07:33:29,434][20347] Updated weights for policy 0, policy_version 276012 (0.0006) [2023-12-27 07:33:29,453][20381] Updated weights for policy 1, policy_version 275646 (0.0011) [2023-12-27 07:33:29,491][20347] Updated weights for policy 0, policy_version 276022 (0.0005) [2023-12-27 07:33:29,511][20381] Updated weights for policy 1, policy_version 275656 (0.0010) [2023-12-27 07:33:29,539][20347] Updated weights for policy 0, policy_version 276032 (0.0006) [2023-12-27 07:33:30,176][20381] Updated weights for policy 1, policy_version 275666 (0.0011) [2023-12-27 07:33:30,215][20347] Updated weights for policy 0, policy_version 276042 (0.0006) [2023-12-27 07:33:30,228][20381] Updated weights for policy 1, policy_version 275676 (0.0010) [2023-12-27 07:33:30,268][20347] Updated weights for policy 0, policy_version 276052 (0.0006) [2023-12-27 07:33:30,278][20381] Updated weights for policy 1, policy_version 275686 (0.0008) [2023-12-27 07:33:30,324][20347] Updated weights for policy 0, policy_version 276062 (0.0006) [2023-12-27 07:33:30,899][20381] Updated weights for policy 1, policy_version 275696 (0.0009) [2023-12-27 07:33:30,903][20347] Updated weights for policy 0, policy_version 276072 (0.0010) [2023-12-27 07:33:30,946][20381] Updated weights for policy 1, policy_version 275706 (0.0010) [2023-12-27 07:33:30,951][20347] Updated weights for policy 0, policy_version 276082 (0.0010) [2023-12-27 07:33:30,994][20381] Updated weights for policy 1, policy_version 275716 (0.0010) [2023-12-27 07:33:31,002][20347] Updated weights for policy 0, policy_version 276092 (0.0010) [2023-12-27 07:33:31,708][20381] Updated weights for policy 1, policy_version 275726 (0.0011) [2023-12-27 07:33:31,745][20347] Updated weights for policy 0, policy_version 276102 (0.0007) [2023-12-27 07:33:31,757][20381] Updated weights for policy 1, policy_version 275736 (0.0010) [2023-12-27 07:33:31,796][20347] Updated weights for policy 0, policy_version 276112 (0.0005) [2023-12-27 07:33:31,810][20381] Updated weights for policy 1, policy_version 275746 (0.0010) [2023-12-27 07:33:31,855][20347] Updated weights for policy 0, policy_version 276122 (0.0005) [2023-12-27 07:33:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 141312000. Throughput: 0: 10468.1, 1: 10396.4. Samples: 141324656. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:32,363][19170] Avg episode reward: [(0, '865.440'), (1, '971.100')] [2023-12-27 07:33:32,403][20347] Updated weights for policy 0, policy_version 276132 (0.0006) [2023-12-27 07:33:32,465][20347] Updated weights for policy 0, policy_version 276142 (0.0008) [2023-12-27 07:33:32,511][20381] Updated weights for policy 1, policy_version 275756 (0.0008) [2023-12-27 07:33:32,525][20347] Updated weights for policy 0, policy_version 276152 (0.0008) [2023-12-27 07:33:32,579][20381] Updated weights for policy 1, policy_version 275766 (0.0007) [2023-12-27 07:33:32,628][20381] Updated weights for policy 1, policy_version 275776 (0.0010) [2023-12-27 07:33:33,218][20347] Updated weights for policy 0, policy_version 276162 (0.0006) [2023-12-27 07:33:33,229][20381] Updated weights for policy 1, policy_version 275786 (0.0009) [2023-12-27 07:33:33,262][20347] Updated weights for policy 0, policy_version 276172 (0.0007) [2023-12-27 07:33:33,286][20381] Updated weights for policy 1, policy_version 275796 (0.0011) [2023-12-27 07:33:33,316][20347] Updated weights for policy 0, policy_version 276182 (0.0005) [2023-12-27 07:33:33,346][20381] Updated weights for policy 1, policy_version 275806 (0.0010) [2023-12-27 07:33:33,371][20347] Updated weights for policy 0, policy_version 276192 (0.0006) [2023-12-27 07:33:33,399][20381] Updated weights for policy 1, policy_version 275816 (0.0006) [2023-12-27 07:33:33,966][20381] Updated weights for policy 1, policy_version 275826 (0.0011) [2023-12-27 07:33:34,019][20381] Updated weights for policy 1, policy_version 275836 (0.0009) [2023-12-27 07:33:34,073][20381] Updated weights for policy 1, policy_version 275846 (0.0010) [2023-12-27 07:33:34,106][20347] Updated weights for policy 0, policy_version 276202 (0.0010) [2023-12-27 07:33:34,168][20347] Updated weights for policy 0, policy_version 276212 (0.0007) [2023-12-27 07:33:34,228][20347] Updated weights for policy 0, policy_version 276222 (0.0006) [2023-12-27 07:33:34,758][20381] Updated weights for policy 1, policy_version 275856 (0.0008) [2023-12-27 07:33:34,811][20381] Updated weights for policy 1, policy_version 275866 (0.0010) [2023-12-27 07:33:34,827][20347] Updated weights for policy 0, policy_version 276232 (0.0009) [2023-12-27 07:33:34,862][20291] KL-divergence is very high: 118.3117 [2023-12-27 07:33:34,865][20381] Updated weights for policy 1, policy_version 275876 (0.0010) [2023-12-27 07:33:34,881][20347] Updated weights for policy 0, policy_version 276242 (0.0010) [2023-12-27 07:33:34,910][20291] KL-divergence is very high: 175.1050 [2023-12-27 07:33:34,941][20347] Updated weights for policy 0, policy_version 276252 (0.0010) [2023-12-27 07:33:35,448][20381] Updated weights for policy 1, policy_version 275886 (0.0008) [2023-12-27 07:33:35,494][20292] KL-divergence is very high: 596.9772 [2023-12-27 07:33:35,504][20381] Updated weights for policy 1, policy_version 275896 (0.0005) [2023-12-27 07:33:35,547][20381] Updated weights for policy 1, policy_version 275906 (0.0005) [2023-12-27 07:33:35,605][20347] Updated weights for policy 0, policy_version 276262 (0.0010) [2023-12-27 07:33:35,666][20347] Updated weights for policy 0, policy_version 276272 (0.0008) [2023-12-27 07:33:35,720][20347] Updated weights for policy 0, policy_version 276282 (0.0010) [2023-12-27 07:33:36,168][20381] Updated weights for policy 1, policy_version 275916 (0.0006) [2023-12-27 07:33:36,224][20381] Updated weights for policy 1, policy_version 275926 (0.0006) [2023-12-27 07:33:36,279][20381] Updated weights for policy 1, policy_version 275936 (0.0007) [2023-12-27 07:33:36,403][20347] Updated weights for policy 0, policy_version 276292 (0.0010) [2023-12-27 07:33:36,455][20347] Updated weights for policy 0, policy_version 276302 (0.0010) [2023-12-27 07:33:36,508][20347] Updated weights for policy 0, policy_version 276312 (0.0010) [2023-12-27 07:33:36,935][20381] Updated weights for policy 1, policy_version 275946 (0.0007) [2023-12-27 07:33:36,982][20381] Updated weights for policy 1, policy_version 275956 (0.0007) [2023-12-27 07:33:37,031][20381] Updated weights for policy 1, policy_version 275966 (0.0005) [2023-12-27 07:33:37,084][20381] Updated weights for policy 1, policy_version 275976 (0.0007) [2023-12-27 07:33:37,205][20347] Updated weights for policy 0, policy_version 276322 (0.0010) [2023-12-27 07:33:37,264][20347] Updated weights for policy 0, policy_version 276332 (0.0011) [2023-12-27 07:33:37,325][20347] Updated weights for policy 0, policy_version 276342 (0.0010) [2023-12-27 07:33:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 141418496. Throughput: 0: 10474.5, 1: 10462.1. Samples: 141388776. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:37,363][19170] Avg episode reward: [(0, '922.640'), (1, '971.190')] [2023-12-27 07:33:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000275976_70664192.pth... [2023-12-27 07:33:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000274760_70352896.pth [2023-12-27 07:33:37,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000276352_70762496.pth... [2023-12-27 07:33:37,381][20347] Updated weights for policy 0, policy_version 276352 (0.0010) [2023-12-27 07:33:37,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000275104_70443008.pth [2023-12-27 07:33:37,751][20381] Updated weights for policy 1, policy_version 275986 (0.0009) [2023-12-27 07:33:37,798][20381] Updated weights for policy 1, policy_version 275996 (0.0008) [2023-12-27 07:33:37,847][20381] Updated weights for policy 1, policy_version 276006 (0.0007) [2023-12-27 07:33:37,997][20347] Updated weights for policy 0, policy_version 276362 (0.0006) [2023-12-27 07:33:38,050][20347] Updated weights for policy 0, policy_version 276372 (0.0010) [2023-12-27 07:33:38,109][20347] Updated weights for policy 0, policy_version 276382 (0.0009) [2023-12-27 07:33:38,551][20381] Updated weights for policy 1, policy_version 276016 (0.0006) [2023-12-27 07:33:38,607][20381] Updated weights for policy 1, policy_version 276026 (0.0007) [2023-12-27 07:33:38,655][20381] Updated weights for policy 1, policy_version 276036 (0.0007) [2023-12-27 07:33:38,731][20347] Updated weights for policy 0, policy_version 276392 (0.0006) [2023-12-27 07:33:38,789][20347] Updated weights for policy 0, policy_version 276402 (0.0007) [2023-12-27 07:33:38,839][20347] Updated weights for policy 0, policy_version 276412 (0.0007) [2023-12-27 07:33:39,293][20381] Updated weights for policy 1, policy_version 276046 (0.0008) [2023-12-27 07:33:39,357][20381] Updated weights for policy 1, policy_version 276056 (0.0006) [2023-12-27 07:33:39,421][20381] Updated weights for policy 1, policy_version 276066 (0.0008) [2023-12-27 07:33:39,590][20347] Updated weights for policy 0, policy_version 276422 (0.0007) [2023-12-27 07:33:39,657][20347] Updated weights for policy 0, policy_version 276432 (0.0008) [2023-12-27 07:33:39,722][20347] Updated weights for policy 0, policy_version 276442 (0.0008) [2023-12-27 07:33:40,071][20381] Updated weights for policy 1, policy_version 276076 (0.0006) [2023-12-27 07:33:40,134][20381] Updated weights for policy 1, policy_version 276086 (0.0007) [2023-12-27 07:33:40,186][20381] Updated weights for policy 1, policy_version 276096 (0.0008) [2023-12-27 07:33:40,410][20347] Updated weights for policy 0, policy_version 276452 (0.0008) [2023-12-27 07:33:40,466][20347] Updated weights for policy 0, policy_version 276462 (0.0007) [2023-12-27 07:33:40,528][20347] Updated weights for policy 0, policy_version 276472 (0.0008) [2023-12-27 07:33:40,931][20381] Updated weights for policy 1, policy_version 276106 (0.0008) [2023-12-27 07:33:40,993][20381] Updated weights for policy 1, policy_version 276116 (0.0008) [2023-12-27 07:33:41,057][20381] Updated weights for policy 1, policy_version 276126 (0.0008) [2023-12-27 07:33:41,130][20381] Updated weights for policy 1, policy_version 276136 (0.0007) [2023-12-27 07:33:41,178][20347] Updated weights for policy 0, policy_version 276482 (0.0007) [2023-12-27 07:33:41,232][20347] Updated weights for policy 0, policy_version 276492 (0.0008) [2023-12-27 07:33:41,295][20347] Updated weights for policy 0, policy_version 276502 (0.0007) [2023-12-27 07:33:41,362][20347] Updated weights for policy 0, policy_version 276512 (0.0007) [2023-12-27 07:33:41,814][20381] Updated weights for policy 1, policy_version 276146 (0.0007) [2023-12-27 07:33:41,868][20381] Updated weights for policy 1, policy_version 276156 (0.0007) [2023-12-27 07:33:41,921][20381] Updated weights for policy 1, policy_version 276166 (0.0007) [2023-12-27 07:33:42,059][20347] Updated weights for policy 0, policy_version 276522 (0.0007) [2023-12-27 07:33:42,116][20347] Updated weights for policy 0, policy_version 276532 (0.0008) [2023-12-27 07:33:42,171][20347] Updated weights for policy 0, policy_version 276542 (0.0010) [2023-12-27 07:33:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 141524992. Throughput: 0: 10443.8, 1: 10510.1. Samples: 141513616. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:42,363][19170] Avg episode reward: [(0, '932.630'), (1, '980.920')] [2023-12-27 07:33:42,646][20381] Updated weights for policy 1, policy_version 276176 (0.0007) [2023-12-27 07:33:42,705][20381] Updated weights for policy 1, policy_version 276186 (0.0008) [2023-12-27 07:33:42,757][20381] Updated weights for policy 1, policy_version 276196 (0.0007) [2023-12-27 07:33:42,841][20347] Updated weights for policy 0, policy_version 276552 (0.0008) [2023-12-27 07:33:42,896][20347] Updated weights for policy 0, policy_version 276562 (0.0008) [2023-12-27 07:33:42,953][20347] Updated weights for policy 0, policy_version 276572 (0.0007) [2023-12-27 07:33:43,484][20381] Updated weights for policy 1, policy_version 276206 (0.0009) [2023-12-27 07:33:43,530][20381] Updated weights for policy 1, policy_version 276216 (0.0008) [2023-12-27 07:33:43,577][20347] Updated weights for policy 0, policy_version 276582 (0.0006) [2023-12-27 07:33:43,580][20381] Updated weights for policy 1, policy_version 276226 (0.0007) [2023-12-27 07:33:43,604][20292] KL-divergence is very high: 190.1269 [2023-12-27 07:33:43,610][20292] KL-divergence is very high: 129.0355 [2023-12-27 07:33:43,641][20347] Updated weights for policy 0, policy_version 276592 (0.0006) [2023-12-27 07:33:43,702][20347] Updated weights for policy 0, policy_version 276602 (0.0008) [2023-12-27 07:33:44,308][20381] Updated weights for policy 1, policy_version 276236 (0.0008) [2023-12-27 07:33:44,312][20347] Updated weights for policy 0, policy_version 276612 (0.0006) [2023-12-27 07:33:44,367][20347] Updated weights for policy 0, policy_version 276622 (0.0005) [2023-12-27 07:33:44,381][20381] Updated weights for policy 1, policy_version 276246 (0.0008) [2023-12-27 07:33:44,422][20347] Updated weights for policy 0, policy_version 276632 (0.0007) [2023-12-27 07:33:44,429][20381] Updated weights for policy 1, policy_version 276256 (0.0006) [2023-12-27 07:33:45,036][20347] Updated weights for policy 0, policy_version 276642 (0.0007) [2023-12-27 07:33:45,097][20381] Updated weights for policy 1, policy_version 276266 (0.0007) [2023-12-27 07:33:45,098][20347] Updated weights for policy 0, policy_version 276652 (0.0007) [2023-12-27 07:33:45,156][20381] Updated weights for policy 1, policy_version 276276 (0.0006) [2023-12-27 07:33:45,158][20347] Updated weights for policy 0, policy_version 276662 (0.0006) [2023-12-27 07:33:45,209][20381] Updated weights for policy 1, policy_version 276286 (0.0006) [2023-12-27 07:33:45,214][20347] Updated weights for policy 0, policy_version 276672 (0.0007) [2023-12-27 07:33:45,269][20381] Updated weights for policy 1, policy_version 276296 (0.0008) [2023-12-27 07:33:45,849][20381] Updated weights for policy 1, policy_version 276306 (0.0006) [2023-12-27 07:33:45,849][20347] Updated weights for policy 0, policy_version 276682 (0.0005) [2023-12-27 07:33:45,896][20381] Updated weights for policy 1, policy_version 276316 (0.0008) [2023-12-27 07:33:45,904][20347] Updated weights for policy 0, policy_version 276692 (0.0006) [2023-12-27 07:33:45,950][20381] Updated weights for policy 1, policy_version 276326 (0.0005) [2023-12-27 07:33:45,955][20347] Updated weights for policy 0, policy_version 276702 (0.0006) [2023-12-27 07:33:46,565][20381] Updated weights for policy 1, policy_version 276336 (0.0007) [2023-12-27 07:33:46,623][20381] Updated weights for policy 1, policy_version 276346 (0.0008) [2023-12-27 07:33:46,652][20347] Updated weights for policy 0, policy_version 276712 (0.0006) [2023-12-27 07:33:46,675][20381] Updated weights for policy 1, policy_version 276356 (0.0006) [2023-12-27 07:33:46,699][20347] Updated weights for policy 0, policy_version 276722 (0.0007) [2023-12-27 07:33:46,744][20347] Updated weights for policy 0, policy_version 276732 (0.0009) [2023-12-27 07:33:47,336][20381] Updated weights for policy 1, policy_version 276366 (0.0007) [2023-12-27 07:33:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 141623296. Throughput: 0: 10475.6, 1: 10484.5. Samples: 141638044. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:47,363][19170] Avg episode reward: [(0, '951.930'), (1, '932.260')] [2023-12-27 07:33:47,392][20381] Updated weights for policy 1, policy_version 276376 (0.0008) [2023-12-27 07:33:47,428][20347] Updated weights for policy 0, policy_version 276742 (0.0007) [2023-12-27 07:33:47,439][20292] KL-divergence is very high: 341.9783 [2023-12-27 07:33:47,445][20292] KL-divergence is very high: 190.0203 [2023-12-27 07:33:47,450][20381] Updated weights for policy 1, policy_version 276386 (0.0008) [2023-12-27 07:33:47,456][20292] KL-divergence is very high: 209.0157 [2023-12-27 07:33:47,467][20292] KL-divergence is very high: 397.6255 [2023-12-27 07:33:47,472][20292] KL-divergence is very high: 261.1607 [2023-12-27 07:33:47,477][20347] Updated weights for policy 0, policy_version 276752 (0.0005) [2023-12-27 07:33:47,525][20347] Updated weights for policy 0, policy_version 276762 (0.0005) [2023-12-27 07:33:48,080][20347] Updated weights for policy 0, policy_version 276772 (0.0006) [2023-12-27 07:33:48,125][20347] Updated weights for policy 0, policy_version 276782 (0.0007) [2023-12-27 07:33:48,168][20292] KL-divergence is very high: 124.9399 [2023-12-27 07:33:48,177][20347] Updated weights for policy 0, policy_version 276792 (0.0007) [2023-12-27 07:33:48,195][20381] Updated weights for policy 1, policy_version 276396 (0.0009) [2023-12-27 07:33:48,216][20292] KL-divergence is very high: 113.6417 [2023-12-27 07:33:48,245][20381] Updated weights for policy 1, policy_version 276406 (0.0006) [2023-12-27 07:33:48,256][20292] KL-divergence is very high: 100.1829 [2023-12-27 07:33:48,291][20381] Updated weights for policy 1, policy_version 276416 (0.0007) [2023-12-27 07:33:48,866][20347] Updated weights for policy 0, policy_version 276802 (0.0008) [2023-12-27 07:33:48,927][20347] Updated weights for policy 0, policy_version 276812 (0.0006) [2023-12-27 07:33:48,990][20347] Updated weights for policy 0, policy_version 276822 (0.0007) [2023-12-27 07:33:49,022][20381] Updated weights for policy 1, policy_version 276426 (0.0008) [2023-12-27 07:33:49,043][20347] Updated weights for policy 0, policy_version 276832 (0.0006) [2023-12-27 07:33:49,062][20292] KL-divergence is very high: 112.4104 [2023-12-27 07:33:49,072][20381] Updated weights for policy 1, policy_version 276436 (0.0007) [2023-12-27 07:33:49,076][20292] KL-divergence is very high: 115.8910 [2023-12-27 07:33:49,122][20381] Updated weights for policy 1, policy_version 276446 (0.0009) [2023-12-27 07:33:49,173][20381] Updated weights for policy 1, policy_version 276456 (0.0010) [2023-12-27 07:33:49,627][20347] Updated weights for policy 0, policy_version 276842 (0.0007) [2023-12-27 07:33:49,681][20347] Updated weights for policy 0, policy_version 276852 (0.0008) [2023-12-27 07:33:49,746][20347] Updated weights for policy 0, policy_version 276862 (0.0008) [2023-12-27 07:33:49,862][20381] Updated weights for policy 1, policy_version 276466 (0.0007) [2023-12-27 07:33:49,930][20381] Updated weights for policy 1, policy_version 276476 (0.0008) [2023-12-27 07:33:49,997][20381] Updated weights for policy 1, policy_version 276486 (0.0008) [2023-12-27 07:33:50,502][20347] Updated weights for policy 0, policy_version 276872 (0.0006) [2023-12-27 07:33:50,557][20347] Updated weights for policy 0, policy_version 276882 (0.0005) [2023-12-27 07:33:50,582][20292] KL-divergence is very high: 162.1553 [2023-12-27 07:33:50,613][20381] Updated weights for policy 1, policy_version 276496 (0.0007) [2023-12-27 07:33:50,615][20347] Updated weights for policy 0, policy_version 276892 (0.0007) [2023-12-27 07:33:50,668][20381] Updated weights for policy 1, policy_version 276506 (0.0008) [2023-12-27 07:33:50,701][20292] KL-divergence is very high: 195.9395 [2023-12-27 07:33:50,707][20292] KL-divergence is very high: 142.2151 [2023-12-27 07:33:50,713][20292] KL-divergence is very high: 155.5517 [2023-12-27 07:33:50,724][20381] Updated weights for policy 1, policy_version 276516 (0.0007) [2023-12-27 07:33:51,242][20347] Updated weights for policy 0, policy_version 276902 (0.0007) [2023-12-27 07:33:51,301][20347] Updated weights for policy 0, policy_version 276912 (0.0007) [2023-12-27 07:33:51,367][20347] Updated weights for policy 0, policy_version 276922 (0.0008) [2023-12-27 07:33:51,465][20381] Updated weights for policy 1, policy_version 276526 (0.0008) [2023-12-27 07:33:51,514][20381] Updated weights for policy 1, policy_version 276536 (0.0008) [2023-12-27 07:33:51,563][20381] Updated weights for policy 1, policy_version 276546 (0.0008) [2023-12-27 07:33:52,109][20347] Updated weights for policy 0, policy_version 276932 (0.0007) [2023-12-27 07:33:52,166][20347] Updated weights for policy 0, policy_version 276942 (0.0006) [2023-12-27 07:33:52,225][20347] Updated weights for policy 0, policy_version 276952 (0.0010) [2023-12-27 07:33:52,352][20381] Updated weights for policy 1, policy_version 276556 (0.0009) [2023-12-27 07:33:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 141729792. Throughput: 0: 10516.0, 1: 10430.8. Samples: 141700496. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:52,363][19170] Avg episode reward: [(0, '971.140'), (1, '855.720')] [2023-12-27 07:33:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000276960_70918144.pth... [2023-12-27 07:33:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000275712_70598656.pth [2023-12-27 07:33:52,413][20381] Updated weights for policy 1, policy_version 276566 (0.0011) [2023-12-27 07:33:52,467][20381] Updated weights for policy 1, policy_version 276576 (0.0010) [2023-12-27 07:33:52,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000276584_70819840.pth... [2023-12-27 07:33:52,514][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000275336_70500352.pth [2023-12-27 07:33:52,956][20347] Updated weights for policy 0, policy_version 276962 (0.0010) [2023-12-27 07:33:53,016][20347] Updated weights for policy 0, policy_version 276972 (0.0010) [2023-12-27 07:33:53,074][20347] Updated weights for policy 0, policy_version 276982 (0.0009) [2023-12-27 07:33:53,122][20347] Updated weights for policy 0, policy_version 276992 (0.0010) [2023-12-27 07:33:53,137][20381] Updated weights for policy 1, policy_version 276586 (0.0007) [2023-12-27 07:33:53,198][20381] Updated weights for policy 1, policy_version 276596 (0.0005) [2023-12-27 07:33:53,257][20381] Updated weights for policy 1, policy_version 276606 (0.0005) [2023-12-27 07:33:53,316][20381] Updated weights for policy 1, policy_version 276616 (0.0008) [2023-12-27 07:33:53,784][20347] Updated weights for policy 0, policy_version 277002 (0.0007) [2023-12-27 07:33:53,796][20291] KL-divergence is very high: 364.4147 [2023-12-27 07:33:53,828][20347] Updated weights for policy 0, policy_version 277012 (0.0007) [2023-12-27 07:33:53,886][20347] Updated weights for policy 0, policy_version 277022 (0.0008) [2023-12-27 07:33:53,901][20381] Updated weights for policy 1, policy_version 276626 (0.0006) [2023-12-27 07:33:53,949][20381] Updated weights for policy 1, policy_version 276636 (0.0005) [2023-12-27 07:33:54,003][20381] Updated weights for policy 1, policy_version 276646 (0.0005) [2023-12-27 07:33:54,541][20381] Updated weights for policy 1, policy_version 276656 (0.0006) [2023-12-27 07:33:54,562][20347] Updated weights for policy 0, policy_version 277032 (0.0010) [2023-12-27 07:33:54,591][20381] Updated weights for policy 1, policy_version 276666 (0.0008) [2023-12-27 07:33:54,613][20347] Updated weights for policy 0, policy_version 277042 (0.0010) [2023-12-27 07:33:54,642][20381] Updated weights for policy 1, policy_version 276676 (0.0005) [2023-12-27 07:33:54,661][20347] Updated weights for policy 0, policy_version 277052 (0.0010) [2023-12-27 07:33:55,248][20381] Updated weights for policy 1, policy_version 276686 (0.0009) [2023-12-27 07:33:55,303][20381] Updated weights for policy 1, policy_version 276696 (0.0010) [2023-12-27 07:33:55,318][20347] Updated weights for policy 0, policy_version 277062 (0.0009) [2023-12-27 07:33:55,358][20381] Updated weights for policy 1, policy_version 276706 (0.0011) [2023-12-27 07:33:55,379][20347] Updated weights for policy 0, policy_version 277072 (0.0007) [2023-12-27 07:33:55,442][20347] Updated weights for policy 0, policy_version 277082 (0.0007) [2023-12-27 07:33:56,002][20381] Updated weights for policy 1, policy_version 276716 (0.0011) [2023-12-27 07:33:56,060][20381] Updated weights for policy 1, policy_version 276726 (0.0011) [2023-12-27 07:33:56,122][20381] Updated weights for policy 1, policy_version 276736 (0.0010) [2023-12-27 07:33:56,126][20347] Updated weights for policy 0, policy_version 277092 (0.0007) [2023-12-27 07:33:56,174][20347] Updated weights for policy 0, policy_version 277102 (0.0005) [2023-12-27 07:33:56,220][20347] Updated weights for policy 0, policy_version 277112 (0.0007) [2023-12-27 07:33:56,797][20381] Updated weights for policy 1, policy_version 276746 (0.0011) [2023-12-27 07:33:56,846][20381] Updated weights for policy 1, policy_version 276756 (0.0010) [2023-12-27 07:33:56,891][20347] Updated weights for policy 0, policy_version 277122 (0.0007) [2023-12-27 07:33:56,896][20381] Updated weights for policy 1, policy_version 276766 (0.0010) [2023-12-27 07:33:56,941][20347] Updated weights for policy 0, policy_version 277132 (0.0006) [2023-12-27 07:33:56,943][20381] Updated weights for policy 1, policy_version 276776 (0.0010) [2023-12-27 07:33:56,983][20347] Updated weights for policy 0, policy_version 277142 (0.0007) [2023-12-27 07:33:57,026][20347] Updated weights for policy 0, policy_version 277152 (0.0007) [2023-12-27 07:33:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 141836288. Throughput: 0: 10513.0, 1: 10482.5. Samples: 141824848. Policy #0 lag: (min: 23.0, avg: 25.6, max: 55.0) [2023-12-27 07:33:57,364][19170] Avg episode reward: [(0, '942.210'), (1, '865.340')] [2023-12-27 07:33:57,619][20381] Updated weights for policy 1, policy_version 276786 (0.0010) [2023-12-27 07:33:57,668][20381] Updated weights for policy 1, policy_version 276796 (0.0011) [2023-12-27 07:33:57,711][20347] Updated weights for policy 0, policy_version 277162 (0.0007) [2023-12-27 07:33:57,722][20381] Updated weights for policy 1, policy_version 276806 (0.0011) [2023-12-27 07:33:57,765][20347] Updated weights for policy 0, policy_version 277172 (0.0007) [2023-12-27 07:33:57,809][20347] Updated weights for policy 0, policy_version 277182 (0.0007) [2023-12-27 07:33:58,480][20381] Updated weights for policy 1, policy_version 276816 (0.0008) [2023-12-27 07:33:58,507][20347] Updated weights for policy 0, policy_version 277192 (0.0009) [2023-12-27 07:33:58,535][20381] Updated weights for policy 1, policy_version 276826 (0.0008) [2023-12-27 07:33:58,569][20347] Updated weights for policy 0, policy_version 277202 (0.0008) [2023-12-27 07:33:58,598][20381] Updated weights for policy 1, policy_version 276836 (0.0007) [2023-12-27 07:33:58,630][20347] Updated weights for policy 0, policy_version 277212 (0.0010) [2023-12-27 07:33:59,294][20381] Updated weights for policy 1, policy_version 276846 (0.0008) [2023-12-27 07:33:59,344][20347] Updated weights for policy 0, policy_version 277222 (0.0010) [2023-12-27 07:33:59,357][20381] Updated weights for policy 1, policy_version 276856 (0.0006) [2023-12-27 07:33:59,410][20347] Updated weights for policy 0, policy_version 277232 (0.0010) [2023-12-27 07:33:59,417][20381] Updated weights for policy 1, policy_version 276866 (0.0006) [2023-12-27 07:33:59,474][20347] Updated weights for policy 0, policy_version 277242 (0.0010) [2023-12-27 07:34:00,100][20347] Updated weights for policy 0, policy_version 277252 (0.0009) [2023-12-27 07:34:00,117][20381] Updated weights for policy 1, policy_version 276876 (0.0008) [2023-12-27 07:34:00,156][20347] Updated weights for policy 0, policy_version 277262 (0.0005) [2023-12-27 07:34:00,172][20381] Updated weights for policy 1, policy_version 276886 (0.0010) [2023-12-27 07:34:00,214][20347] Updated weights for policy 0, policy_version 277272 (0.0006) [2023-12-27 07:34:00,233][20381] Updated weights for policy 1, policy_version 276896 (0.0010) [2023-12-27 07:34:00,843][20347] Updated weights for policy 0, policy_version 277282 (0.0007) [2023-12-27 07:34:00,900][20381] Updated weights for policy 1, policy_version 276906 (0.0010) [2023-12-27 07:34:00,901][20347] Updated weights for policy 0, policy_version 277292 (0.0010) [2023-12-27 07:34:00,932][20292] KL-divergence is very high: 552.7556 [2023-12-27 07:34:00,949][20292] KL-divergence is very high: 521.2679 [2023-12-27 07:34:00,954][20292] KL-divergence is very high: 427.1074 [2023-12-27 07:34:00,954][20381] Updated weights for policy 1, policy_version 276916 (0.0010) [2023-12-27 07:34:00,955][20347] Updated weights for policy 0, policy_version 277302 (0.0010) [2023-12-27 07:34:00,964][20292] KL-divergence is very high: 117.9980 [2023-12-27 07:34:01,002][20381] Updated weights for policy 1, policy_version 276926 (0.0010) [2023-12-27 07:34:01,006][20347] Updated weights for policy 0, policy_version 277312 (0.0010) [2023-12-27 07:34:01,046][20292] KL-divergence is very high: 135.4887 [2023-12-27 07:34:01,058][20381] Updated weights for policy 1, policy_version 276936 (0.0011) [2023-12-27 07:34:01,659][20292] KL-divergence is very high: 282.0235 [2023-12-27 07:34:01,707][20292] KL-divergence is very high: 506.3564 [2023-12-27 07:34:01,711][20347] Updated weights for policy 0, policy_version 277322 (0.0010) [2023-12-27 07:34:01,714][20381] Updated weights for policy 1, policy_version 276946 (0.0011) [2023-12-27 07:34:01,756][20292] KL-divergence is very high: 216.0746 [2023-12-27 07:34:01,773][20347] Updated weights for policy 0, policy_version 277332 (0.0010) [2023-12-27 07:34:01,777][20381] Updated weights for policy 1, policy_version 276956 (0.0009) [2023-12-27 07:34:01,804][20292] KL-divergence is very high: 282.7337 [2023-12-27 07:34:01,828][20347] Updated weights for policy 0, policy_version 277342 (0.0008) [2023-12-27 07:34:01,831][20381] Updated weights for policy 1, policy_version 276966 (0.0007) [2023-12-27 07:34:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 141934592. Throughput: 0: 10465.3, 1: 10399.6. Samples: 141947364. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:02,363][19170] Avg episode reward: [(0, '942.460'), (1, '875.120')] [2023-12-27 07:34:02,432][20347] Updated weights for policy 0, policy_version 277352 (0.0008) [2023-12-27 07:34:02,482][20347] Updated weights for policy 0, policy_version 277362 (0.0007) [2023-12-27 07:34:02,506][20381] Updated weights for policy 1, policy_version 276976 (0.0007) [2023-12-27 07:34:02,530][20347] Updated weights for policy 0, policy_version 277372 (0.0005) [2023-12-27 07:34:02,558][20381] Updated weights for policy 1, policy_version 276986 (0.0006) [2023-12-27 07:34:02,616][20381] Updated weights for policy 1, policy_version 276996 (0.0009) [2023-12-27 07:34:03,118][20347] Updated weights for policy 0, policy_version 277382 (0.0007) [2023-12-27 07:34:03,166][20347] Updated weights for policy 0, policy_version 277392 (0.0009) [2023-12-27 07:34:03,213][20347] Updated weights for policy 0, policy_version 277402 (0.0008) [2023-12-27 07:34:03,301][20381] Updated weights for policy 1, policy_version 277006 (0.0009) [2023-12-27 07:34:03,346][20381] Updated weights for policy 1, policy_version 277016 (0.0009) [2023-12-27 07:34:03,397][20381] Updated weights for policy 1, policy_version 277026 (0.0008) [2023-12-27 07:34:03,785][20347] Updated weights for policy 0, policy_version 277412 (0.0008) [2023-12-27 07:34:03,845][20347] Updated weights for policy 0, policy_version 277422 (0.0007) [2023-12-27 07:34:03,905][20347] Updated weights for policy 0, policy_version 277432 (0.0007) [2023-12-27 07:34:04,125][20381] Updated weights for policy 1, policy_version 277036 (0.0010) [2023-12-27 07:34:04,180][20381] Updated weights for policy 1, policy_version 277046 (0.0011) [2023-12-27 07:34:04,236][20381] Updated weights for policy 1, policy_version 277056 (0.0011) [2023-12-27 07:34:04,566][20347] Updated weights for policy 0, policy_version 277442 (0.0007) [2023-12-27 07:34:04,624][20347] Updated weights for policy 0, policy_version 277452 (0.0006) [2023-12-27 07:34:04,676][20347] Updated weights for policy 0, policy_version 277462 (0.0010) [2023-12-27 07:34:04,724][20347] Updated weights for policy 0, policy_version 277472 (0.0010) [2023-12-27 07:34:04,936][20381] Updated weights for policy 1, policy_version 277066 (0.0010) [2023-12-27 07:34:05,001][20381] Updated weights for policy 1, policy_version 277076 (0.0007) [2023-12-27 07:34:05,063][20381] Updated weights for policy 1, policy_version 277086 (0.0008) [2023-12-27 07:34:05,123][20381] Updated weights for policy 1, policy_version 277096 (0.0009) [2023-12-27 07:34:05,378][20347] Updated weights for policy 0, policy_version 277482 (0.0010) [2023-12-27 07:34:05,423][20347] Updated weights for policy 0, policy_version 277492 (0.0010) [2023-12-27 07:34:05,471][20347] Updated weights for policy 0, policy_version 277502 (0.0010) [2023-12-27 07:34:05,754][20381] Updated weights for policy 1, policy_version 277106 (0.0010) [2023-12-27 07:34:05,756][20292] KL-divergence is very high: 158.2892 [2023-12-27 07:34:05,779][20292] KL-divergence is very high: 381.9968 [2023-12-27 07:34:05,801][20292] KL-divergence is very high: 304.2766 [2023-12-27 07:34:05,812][20381] Updated weights for policy 1, policy_version 277116 (0.0010) [2023-12-27 07:34:05,825][20292] KL-divergence is very high: 236.2962 [2023-12-27 07:34:05,872][20381] Updated weights for policy 1, policy_version 277126 (0.0011) [2023-12-27 07:34:06,141][20347] Updated weights for policy 0, policy_version 277512 (0.0008) [2023-12-27 07:34:06,199][20347] Updated weights for policy 0, policy_version 277522 (0.0010) [2023-12-27 07:34:06,265][20347] Updated weights for policy 0, policy_version 277532 (0.0010) [2023-12-27 07:34:06,533][20381] Updated weights for policy 1, policy_version 277136 (0.0010) [2023-12-27 07:34:06,584][20381] Updated weights for policy 1, policy_version 277146 (0.0011) [2023-12-27 07:34:06,631][20381] Updated weights for policy 1, policy_version 277156 (0.0011) [2023-12-27 07:34:06,931][20347] Updated weights for policy 0, policy_version 277542 (0.0010) [2023-12-27 07:34:06,979][20347] Updated weights for policy 0, policy_version 277552 (0.0010) [2023-12-27 07:34:07,027][20347] Updated weights for policy 0, policy_version 277562 (0.0010) [2023-12-27 07:34:07,321][20381] Updated weights for policy 1, policy_version 277166 (0.0008) [2023-12-27 07:34:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 142041088. Throughput: 0: 10549.0, 1: 10381.6. Samples: 142011152. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:07,363][19170] Avg episode reward: [(0, '962.080'), (1, '855.530')] [2023-12-27 07:34:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000277568_71073792.pth... [2023-12-27 07:34:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000276352_70762496.pth [2023-12-27 07:34:07,376][20381] Updated weights for policy 1, policy_version 277176 (0.0006) [2023-12-27 07:34:07,425][20381] Updated weights for policy 1, policy_version 277186 (0.0005) [2023-12-27 07:34:07,459][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000277192_70975488.pth... [2023-12-27 07:34:07,463][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000275976_70664192.pth [2023-12-27 07:34:07,717][20347] Updated weights for policy 0, policy_version 277572 (0.0010) [2023-12-27 07:34:07,763][20347] Updated weights for policy 0, policy_version 277582 (0.0009) [2023-12-27 07:34:07,813][20347] Updated weights for policy 0, policy_version 277592 (0.0010) [2023-12-27 07:34:07,985][20381] Updated weights for policy 1, policy_version 277196 (0.0011) [2023-12-27 07:34:08,041][20381] Updated weights for policy 1, policy_version 277206 (0.0011) [2023-12-27 07:34:08,093][20381] Updated weights for policy 1, policy_version 277216 (0.0011) [2023-12-27 07:34:08,498][20347] Updated weights for policy 0, policy_version 277602 (0.0009) [2023-12-27 07:34:08,553][20347] Updated weights for policy 0, policy_version 277612 (0.0007) [2023-12-27 07:34:08,608][20347] Updated weights for policy 0, policy_version 277622 (0.0008) [2023-12-27 07:34:08,663][20347] Updated weights for policy 0, policy_version 277632 (0.0007) [2023-12-27 07:34:08,785][20381] Updated weights for policy 1, policy_version 277226 (0.0011) [2023-12-27 07:34:08,850][20381] Updated weights for policy 1, policy_version 277236 (0.0011) [2023-12-27 07:34:08,912][20381] Updated weights for policy 1, policy_version 277246 (0.0011) [2023-12-27 07:34:08,971][20381] Updated weights for policy 1, policy_version 277256 (0.0011) [2023-12-27 07:34:09,384][20347] Updated weights for policy 0, policy_version 277642 (0.0008) [2023-12-27 07:34:09,441][20347] Updated weights for policy 0, policy_version 277652 (0.0007) [2023-12-27 07:34:09,501][20347] Updated weights for policy 0, policy_version 277662 (0.0008) [2023-12-27 07:34:09,628][20381] Updated weights for policy 1, policy_version 277266 (0.0008) [2023-12-27 07:34:09,683][20381] Updated weights for policy 1, policy_version 277276 (0.0008) [2023-12-27 07:34:09,733][20381] Updated weights for policy 1, policy_version 277286 (0.0010) [2023-12-27 07:34:10,217][20347] Updated weights for policy 0, policy_version 277672 (0.0007) [2023-12-27 07:34:10,266][20347] Updated weights for policy 0, policy_version 277682 (0.0007) [2023-12-27 07:34:10,318][20347] Updated weights for policy 0, policy_version 277692 (0.0008) [2023-12-27 07:34:10,402][20381] Updated weights for policy 1, policy_version 277296 (0.0010) [2023-12-27 07:34:10,463][20381] Updated weights for policy 1, policy_version 277306 (0.0007) [2023-12-27 07:34:10,516][20381] Updated weights for policy 1, policy_version 277316 (0.0006) [2023-12-27 07:34:10,988][20347] Updated weights for policy 0, policy_version 277702 (0.0009) [2023-12-27 07:34:11,052][20347] Updated weights for policy 0, policy_version 277712 (0.0010) [2023-12-27 07:34:11,113][20347] Updated weights for policy 0, policy_version 277722 (0.0011) [2023-12-27 07:34:11,134][20381] Updated weights for policy 1, policy_version 277326 (0.0007) [2023-12-27 07:34:11,198][20381] Updated weights for policy 1, policy_version 277336 (0.0007) [2023-12-27 07:34:11,261][20381] Updated weights for policy 1, policy_version 277346 (0.0008) [2023-12-27 07:34:11,817][20347] Updated weights for policy 0, policy_version 277732 (0.0009) [2023-12-27 07:34:11,875][20347] Updated weights for policy 0, policy_version 277742 (0.0005) [2023-12-27 07:34:11,939][20347] Updated weights for policy 0, policy_version 277752 (0.0005) [2023-12-27 07:34:12,001][20381] Updated weights for policy 1, policy_version 277356 (0.0007) [2023-12-27 07:34:12,073][20381] Updated weights for policy 1, policy_version 277366 (0.0008) [2023-12-27 07:34:12,134][20381] Updated weights for policy 1, policy_version 277376 (0.0008) [2023-12-27 07:34:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 142147584. Throughput: 0: 10490.6, 1: 10426.5. Samples: 142135800. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:12,363][19170] Avg episode reward: [(0, '990.360'), (1, '912.580')] [2023-12-27 07:34:12,546][20347] Updated weights for policy 0, policy_version 277762 (0.0005) [2023-12-27 07:34:12,597][20347] Updated weights for policy 0, policy_version 277772 (0.0005) [2023-12-27 07:34:12,655][20347] Updated weights for policy 0, policy_version 277782 (0.0006) [2023-12-27 07:34:12,723][20347] Updated weights for policy 0, policy_version 277792 (0.0009) [2023-12-27 07:34:12,776][20381] Updated weights for policy 1, policy_version 277386 (0.0008) [2023-12-27 07:34:12,841][20381] Updated weights for policy 1, policy_version 277396 (0.0007) [2023-12-27 07:34:12,909][20381] Updated weights for policy 1, policy_version 277406 (0.0008) [2023-12-27 07:34:12,970][20381] Updated weights for policy 1, policy_version 277416 (0.0007) [2023-12-27 07:34:13,322][20347] Updated weights for policy 0, policy_version 277802 (0.0005) [2023-12-27 07:34:13,391][20347] Updated weights for policy 0, policy_version 277812 (0.0008) [2023-12-27 07:34:13,457][20347] Updated weights for policy 0, policy_version 277822 (0.0011) [2023-12-27 07:34:13,701][20381] Updated weights for policy 1, policy_version 277426 (0.0007) [2023-12-27 07:34:13,754][20381] Updated weights for policy 1, policy_version 277436 (0.0005) [2023-12-27 07:34:13,809][20381] Updated weights for policy 1, policy_version 277446 (0.0005) [2023-12-27 07:34:14,102][20347] Updated weights for policy 0, policy_version 277832 (0.0008) [2023-12-27 07:34:14,158][20347] Updated weights for policy 0, policy_version 277842 (0.0007) [2023-12-27 07:34:14,216][20347] Updated weights for policy 0, policy_version 277852 (0.0007) [2023-12-27 07:34:14,386][20381] Updated weights for policy 1, policy_version 277456 (0.0007) [2023-12-27 07:34:14,445][20381] Updated weights for policy 1, policy_version 277466 (0.0008) [2023-12-27 07:34:14,496][20381] Updated weights for policy 1, policy_version 277476 (0.0008) [2023-12-27 07:34:14,870][20347] Updated weights for policy 0, policy_version 277862 (0.0010) [2023-12-27 07:34:14,920][20347] Updated weights for policy 0, policy_version 277872 (0.0010) [2023-12-27 07:34:14,982][20347] Updated weights for policy 0, policy_version 277882 (0.0011) [2023-12-27 07:34:15,183][20381] Updated weights for policy 1, policy_version 277486 (0.0008) [2023-12-27 07:34:15,244][20381] Updated weights for policy 1, policy_version 277496 (0.0008) [2023-12-27 07:34:15,299][20381] Updated weights for policy 1, policy_version 277506 (0.0007) [2023-12-27 07:34:15,655][20347] Updated weights for policy 0, policy_version 277892 (0.0010) [2023-12-27 07:34:15,712][20347] Updated weights for policy 0, policy_version 277902 (0.0010) [2023-12-27 07:34:15,777][20347] Updated weights for policy 0, policy_version 277912 (0.0010) [2023-12-27 07:34:15,967][20381] Updated weights for policy 1, policy_version 277516 (0.0007) [2023-12-27 07:34:16,016][20381] Updated weights for policy 1, policy_version 277526 (0.0006) [2023-12-27 07:34:16,067][20381] Updated weights for policy 1, policy_version 277536 (0.0007) [2023-12-27 07:34:16,434][20347] Updated weights for policy 0, policy_version 277922 (0.0010) [2023-12-27 07:34:16,485][20347] Updated weights for policy 0, policy_version 277932 (0.0010) [2023-12-27 07:34:16,538][20347] Updated weights for policy 0, policy_version 277942 (0.0010) [2023-12-27 07:34:16,586][20347] Updated weights for policy 0, policy_version 277952 (0.0010) [2023-12-27 07:34:16,761][20381] Updated weights for policy 1, policy_version 277546 (0.0008) [2023-12-27 07:34:16,806][20381] Updated weights for policy 1, policy_version 277556 (0.0010) [2023-12-27 07:34:16,857][20381] Updated weights for policy 1, policy_version 277566 (0.0009) [2023-12-27 07:34:16,922][20381] Updated weights for policy 1, policy_version 277576 (0.0009) [2023-12-27 07:34:17,268][20347] Updated weights for policy 0, policy_version 277962 (0.0010) [2023-12-27 07:34:17,328][20347] Updated weights for policy 0, policy_version 277972 (0.0010) [2023-12-27 07:34:17,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 142245888. Throughput: 0: 10421.9, 1: 10381.1. Samples: 142260796. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:17,364][19170] Avg episode reward: [(0, '990.360'), (1, '942.060')] [2023-12-27 07:34:17,391][20347] Updated weights for policy 0, policy_version 277982 (0.0010) [2023-12-27 07:34:17,555][20381] Updated weights for policy 1, policy_version 277586 (0.0010) [2023-12-27 07:34:17,617][20381] Updated weights for policy 1, policy_version 277596 (0.0008) [2023-12-27 07:34:17,675][20381] Updated weights for policy 1, policy_version 277606 (0.0008) [2023-12-27 07:34:18,000][20347] Updated weights for policy 0, policy_version 277992 (0.0010) [2023-12-27 07:34:18,051][20347] Updated weights for policy 0, policy_version 278002 (0.0010) [2023-12-27 07:34:18,100][20347] Updated weights for policy 0, policy_version 278012 (0.0010) [2023-12-27 07:34:18,328][20381] Updated weights for policy 1, policy_version 277616 (0.0009) [2023-12-27 07:34:18,394][20381] Updated weights for policy 1, policy_version 277626 (0.0009) [2023-12-27 07:34:18,450][20381] Updated weights for policy 1, policy_version 277636 (0.0006) [2023-12-27 07:34:18,786][20347] Updated weights for policy 0, policy_version 278022 (0.0010) [2023-12-27 07:34:18,848][20347] Updated weights for policy 0, policy_version 278032 (0.0011) [2023-12-27 07:34:18,910][20347] Updated weights for policy 0, policy_version 278042 (0.0010) [2023-12-27 07:34:19,055][20381] Updated weights for policy 1, policy_version 277646 (0.0009) [2023-12-27 07:34:19,105][20381] Updated weights for policy 1, policy_version 277656 (0.0009) [2023-12-27 07:34:19,160][20381] Updated weights for policy 1, policy_version 277666 (0.0008) [2023-12-27 07:34:19,545][20347] Updated weights for policy 0, policy_version 278052 (0.0009) [2023-12-27 07:34:19,608][20347] Updated weights for policy 0, policy_version 278062 (0.0010) [2023-12-27 07:34:19,667][20347] Updated weights for policy 0, policy_version 278072 (0.0007) [2023-12-27 07:34:19,831][20381] Updated weights for policy 1, policy_version 277676 (0.0011) [2023-12-27 07:34:19,890][20381] Updated weights for policy 1, policy_version 277686 (0.0008) [2023-12-27 07:34:19,954][20381] Updated weights for policy 1, policy_version 277696 (0.0009) [2023-12-27 07:34:20,328][20347] Updated weights for policy 0, policy_version 278082 (0.0006) [2023-12-27 07:34:20,389][20347] Updated weights for policy 0, policy_version 278092 (0.0010) [2023-12-27 07:34:20,448][20347] Updated weights for policy 0, policy_version 278102 (0.0010) [2023-12-27 07:34:20,514][20347] Updated weights for policy 0, policy_version 278112 (0.0009) [2023-12-27 07:34:20,688][20381] Updated weights for policy 1, policy_version 277706 (0.0009) [2023-12-27 07:34:20,747][20381] Updated weights for policy 1, policy_version 277716 (0.0008) [2023-12-27 07:34:20,812][20381] Updated weights for policy 1, policy_version 277726 (0.0008) [2023-12-27 07:34:20,864][20381] Updated weights for policy 1, policy_version 277736 (0.0008) [2023-12-27 07:34:21,194][20347] Updated weights for policy 0, policy_version 278122 (0.0008) [2023-12-27 07:34:21,254][20347] Updated weights for policy 0, policy_version 278132 (0.0007) [2023-12-27 07:34:21,318][20347] Updated weights for policy 0, policy_version 278142 (0.0007) [2023-12-27 07:34:21,575][20381] Updated weights for policy 1, policy_version 277746 (0.0009) [2023-12-27 07:34:21,632][20381] Updated weights for policy 1, policy_version 277756 (0.0007) [2023-12-27 07:34:21,693][20381] Updated weights for policy 1, policy_version 277766 (0.0011) [2023-12-27 07:34:22,061][20347] Updated weights for policy 0, policy_version 278152 (0.0008) [2023-12-27 07:34:22,120][20347] Updated weights for policy 0, policy_version 278162 (0.0007) [2023-12-27 07:34:22,181][20347] Updated weights for policy 0, policy_version 278172 (0.0007) [2023-12-27 07:34:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 142352384. Throughput: 0: 10430.2, 1: 10330.8. Samples: 142323020. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:22,363][19170] Avg episode reward: [(0, '990.360'), (1, '961.640')] [2023-12-27 07:34:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000278176_71229440.pth... [2023-12-27 07:34:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000276960_70918144.pth [2023-12-27 07:34:22,384][20381] Updated weights for policy 1, policy_version 277776 (0.0009) [2023-12-27 07:34:22,442][20381] Updated weights for policy 1, policy_version 277786 (0.0007) [2023-12-27 07:34:22,504][20381] Updated weights for policy 1, policy_version 277796 (0.0008) [2023-12-27 07:34:22,525][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000277800_71131136.pth... [2023-12-27 07:34:22,528][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000276584_70819840.pth [2023-12-27 07:34:22,865][20347] Updated weights for policy 0, policy_version 278182 (0.0008) [2023-12-27 07:34:22,928][20347] Updated weights for policy 0, policy_version 278192 (0.0008) [2023-12-27 07:34:22,991][20347] Updated weights for policy 0, policy_version 278202 (0.0008) [2023-12-27 07:34:23,198][20381] Updated weights for policy 1, policy_version 277806 (0.0008) [2023-12-27 07:34:23,251][20381] Updated weights for policy 1, policy_version 277816 (0.0008) [2023-12-27 07:34:23,307][20381] Updated weights for policy 1, policy_version 277826 (0.0008) [2023-12-27 07:34:23,679][20347] Updated weights for policy 0, policy_version 278212 (0.0008) [2023-12-27 07:34:23,728][20347] Updated weights for policy 0, policy_version 278222 (0.0008) [2023-12-27 07:34:23,776][20347] Updated weights for policy 0, policy_version 278232 (0.0007) [2023-12-27 07:34:23,978][20381] Updated weights for policy 1, policy_version 277836 (0.0007) [2023-12-27 07:34:24,032][20381] Updated weights for policy 1, policy_version 277846 (0.0006) [2023-12-27 07:34:24,094][20381] Updated weights for policy 1, policy_version 277856 (0.0008) [2023-12-27 07:34:24,476][20347] Updated weights for policy 0, policy_version 278242 (0.0007) [2023-12-27 07:34:24,534][20347] Updated weights for policy 0, policy_version 278252 (0.0008) [2023-12-27 07:34:24,588][20347] Updated weights for policy 0, policy_version 278262 (0.0007) [2023-12-27 07:34:24,645][20347] Updated weights for policy 0, policy_version 278272 (0.0007) [2023-12-27 07:34:24,789][20381] Updated weights for policy 1, policy_version 277866 (0.0008) [2023-12-27 07:34:24,845][20381] Updated weights for policy 1, policy_version 277876 (0.0007) [2023-12-27 07:34:24,890][20381] Updated weights for policy 1, policy_version 277886 (0.0008) [2023-12-27 07:34:24,936][20381] Updated weights for policy 1, policy_version 277896 (0.0007) [2023-12-27 07:34:25,332][20347] Updated weights for policy 0, policy_version 278282 (0.0007) [2023-12-27 07:34:25,381][20347] Updated weights for policy 0, policy_version 278292 (0.0008) [2023-12-27 07:34:25,433][20347] Updated weights for policy 0, policy_version 278302 (0.0008) [2023-12-27 07:34:25,644][20381] Updated weights for policy 1, policy_version 277906 (0.0007) [2023-12-27 07:34:25,699][20381] Updated weights for policy 1, policy_version 277916 (0.0007) [2023-12-27 07:34:25,754][20381] Updated weights for policy 1, policy_version 277926 (0.0008) [2023-12-27 07:34:26,119][20347] Updated weights for policy 0, policy_version 278312 (0.0008) [2023-12-27 07:34:26,177][20347] Updated weights for policy 0, policy_version 278322 (0.0010) [2023-12-27 07:34:26,229][20347] Updated weights for policy 0, policy_version 278332 (0.0010) [2023-12-27 07:34:26,444][20381] Updated weights for policy 1, policy_version 277936 (0.0006) [2023-12-27 07:34:26,491][20381] Updated weights for policy 1, policy_version 277946 (0.0005) [2023-12-27 07:34:26,540][20381] Updated weights for policy 1, policy_version 277956 (0.0005) [2023-12-27 07:34:26,876][20347] Updated weights for policy 0, policy_version 278342 (0.0010) [2023-12-27 07:34:26,920][20347] Updated weights for policy 0, policy_version 278352 (0.0010) [2023-12-27 07:34:26,968][20347] Updated weights for policy 0, policy_version 278362 (0.0010) [2023-12-27 07:34:27,054][20381] Updated weights for policy 1, policy_version 277966 (0.0006) [2023-12-27 07:34:27,098][20381] Updated weights for policy 1, policy_version 277976 (0.0007) [2023-12-27 07:34:27,154][20381] Updated weights for policy 1, policy_version 277986 (0.0005) [2023-12-27 07:34:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 142458880. Throughput: 0: 10382.2, 1: 10265.1. Samples: 142442744. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:27,363][19170] Avg episode reward: [(0, '981.340'), (1, '969.950')] [2023-12-27 07:34:27,658][20347] Updated weights for policy 0, policy_version 278372 (0.0010) [2023-12-27 07:34:27,709][20347] Updated weights for policy 0, policy_version 278382 (0.0008) [2023-12-27 07:34:27,766][20347] Updated weights for policy 0, policy_version 278392 (0.0006) [2023-12-27 07:34:27,810][20381] Updated weights for policy 1, policy_version 277996 (0.0007) [2023-12-27 07:34:27,861][20381] Updated weights for policy 1, policy_version 278006 (0.0008) [2023-12-27 07:34:27,914][20381] Updated weights for policy 1, policy_version 278016 (0.0008) [2023-12-27 07:34:28,334][20347] Updated weights for policy 0, policy_version 278402 (0.0007) [2023-12-27 07:34:28,389][20347] Updated weights for policy 0, policy_version 278412 (0.0007) [2023-12-27 07:34:28,449][20347] Updated weights for policy 0, policy_version 278422 (0.0010) [2023-12-27 07:34:28,509][20347] Updated weights for policy 0, policy_version 278432 (0.0010) [2023-12-27 07:34:28,633][20381] Updated weights for policy 1, policy_version 278026 (0.0007) [2023-12-27 07:34:28,683][20381] Updated weights for policy 1, policy_version 278036 (0.0010) [2023-12-27 07:34:28,727][20381] Updated weights for policy 1, policy_version 278046 (0.0010) [2023-12-27 07:34:28,786][20381] Updated weights for policy 1, policy_version 278056 (0.0010) [2023-12-27 07:34:29,162][20347] Updated weights for policy 0, policy_version 278442 (0.0010) [2023-12-27 07:34:29,220][20347] Updated weights for policy 0, policy_version 278452 (0.0010) [2023-12-27 07:34:29,284][20347] Updated weights for policy 0, policy_version 278462 (0.0011) [2023-12-27 07:34:29,422][20381] Updated weights for policy 1, policy_version 278066 (0.0005) [2023-12-27 07:34:29,475][20381] Updated weights for policy 1, policy_version 278076 (0.0005) [2023-12-27 07:34:29,536][20381] Updated weights for policy 1, policy_version 278086 (0.0006) [2023-12-27 07:34:29,940][20347] Updated weights for policy 0, policy_version 278472 (0.0009) [2023-12-27 07:34:29,991][20347] Updated weights for policy 0, policy_version 278482 (0.0008) [2023-12-27 07:34:30,038][20347] Updated weights for policy 0, policy_version 278492 (0.0008) [2023-12-27 07:34:30,120][20381] Updated weights for policy 1, policy_version 278096 (0.0010) [2023-12-27 07:34:30,182][20381] Updated weights for policy 1, policy_version 278106 (0.0010) [2023-12-27 07:34:30,241][20381] Updated weights for policy 1, policy_version 278116 (0.0011) [2023-12-27 07:34:30,822][20347] Updated weights for policy 0, policy_version 278502 (0.0007) [2023-12-27 07:34:30,856][20381] Updated weights for policy 1, policy_version 278126 (0.0010) [2023-12-27 07:34:30,865][20347] Updated weights for policy 0, policy_version 278512 (0.0007) [2023-12-27 07:34:30,912][20347] Updated weights for policy 0, policy_version 278522 (0.0005) [2023-12-27 07:34:30,913][20381] Updated weights for policy 1, policy_version 278136 (0.0005) [2023-12-27 07:34:30,964][20381] Updated weights for policy 1, policy_version 278146 (0.0008) [2023-12-27 07:34:31,586][20347] Updated weights for policy 0, policy_version 278532 (0.0006) [2023-12-27 07:34:31,646][20347] Updated weights for policy 0, policy_version 278542 (0.0006) [2023-12-27 07:34:31,705][20347] Updated weights for policy 0, policy_version 278552 (0.0007) [2023-12-27 07:34:31,715][20381] Updated weights for policy 1, policy_version 278156 (0.0008) [2023-12-27 07:34:31,769][20381] Updated weights for policy 1, policy_version 278166 (0.0008) [2023-12-27 07:34:31,828][20381] Updated weights for policy 1, policy_version 278176 (0.0008) [2023-12-27 07:34:32,290][20347] Updated weights for policy 0, policy_version 278562 (0.0007) [2023-12-27 07:34:32,348][20347] Updated weights for policy 0, policy_version 278572 (0.0008) [2023-12-27 07:34:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 142557184. Throughput: 0: 10399.1, 1: 10348.1. Samples: 142571668. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:32,363][19170] Avg episode reward: [(0, '942.390'), (1, '972.910')] [2023-12-27 07:34:32,408][20347] Updated weights for policy 0, policy_version 278582 (0.0006) [2023-12-27 07:34:32,437][20381] Updated weights for policy 1, policy_version 278186 (0.0007) [2023-12-27 07:34:32,460][20347] Updated weights for policy 0, policy_version 278592 (0.0005) [2023-12-27 07:34:32,487][20381] Updated weights for policy 1, policy_version 278196 (0.0005) [2023-12-27 07:34:32,544][20381] Updated weights for policy 1, policy_version 278206 (0.0006) [2023-12-27 07:34:32,593][20381] Updated weights for policy 1, policy_version 278216 (0.0009) [2023-12-27 07:34:33,085][20347] Updated weights for policy 0, policy_version 278602 (0.0010) [2023-12-27 07:34:33,130][20347] Updated weights for policy 0, policy_version 278612 (0.0010) [2023-12-27 07:34:33,175][20347] Updated weights for policy 0, policy_version 278622 (0.0010) [2023-12-27 07:34:33,215][20381] Updated weights for policy 1, policy_version 278226 (0.0006) [2023-12-27 07:34:33,262][20381] Updated weights for policy 1, policy_version 278236 (0.0005) [2023-12-27 07:34:33,311][20381] Updated weights for policy 1, policy_version 278246 (0.0006) [2023-12-27 07:34:33,860][20347] Updated weights for policy 0, policy_version 278632 (0.0010) [2023-12-27 07:34:33,895][20381] Updated weights for policy 1, policy_version 278256 (0.0008) [2023-12-27 07:34:33,908][20347] Updated weights for policy 0, policy_version 278642 (0.0010) [2023-12-27 07:34:33,943][20381] Updated weights for policy 1, policy_version 278266 (0.0005) [2023-12-27 07:34:33,956][20347] Updated weights for policy 0, policy_version 278652 (0.0010) [2023-12-27 07:34:33,996][20381] Updated weights for policy 1, policy_version 278276 (0.0006) [2023-12-27 07:34:34,653][20381] Updated weights for policy 1, policy_version 278286 (0.0008) [2023-12-27 07:34:34,667][20347] Updated weights for policy 0, policy_version 278662 (0.0009) [2023-12-27 07:34:34,711][20381] Updated weights for policy 1, policy_version 278296 (0.0006) [2023-12-27 07:34:34,723][20347] Updated weights for policy 0, policy_version 278672 (0.0007) [2023-12-27 07:34:34,767][20381] Updated weights for policy 1, policy_version 278306 (0.0006) [2023-12-27 07:34:34,776][20347] Updated weights for policy 0, policy_version 278682 (0.0007) [2023-12-27 07:34:35,345][20347] Updated weights for policy 0, policy_version 278692 (0.0006) [2023-12-27 07:34:35,384][20291] KL-divergence is very high: 1177.5676 [2023-12-27 07:34:35,396][20291] KL-divergence is very high: 181.0809 [2023-12-27 07:34:35,403][20347] Updated weights for policy 0, policy_version 278702 (0.0008) [2023-12-27 07:34:35,408][20291] KL-divergence is very high: 204.4223 [2023-12-27 07:34:35,419][20291] KL-divergence is very high: 134.2999 [2023-12-27 07:34:35,429][20291] KL-divergence is very high: 113.5245 [2023-12-27 07:34:35,444][20381] Updated weights for policy 1, policy_version 278316 (0.0006) [2023-12-27 07:34:35,454][20347] Updated weights for policy 0, policy_version 278712 (0.0006) [2023-12-27 07:34:35,467][20291] KL-divergence is very high: 125.1750 [2023-12-27 07:34:35,505][20381] Updated weights for policy 1, policy_version 278326 (0.0006) [2023-12-27 07:34:35,560][20381] Updated weights for policy 1, policy_version 278336 (0.0007) [2023-12-27 07:34:36,085][20347] Updated weights for policy 0, policy_version 278722 (0.0006) [2023-12-27 07:34:36,147][20347] Updated weights for policy 0, policy_version 278732 (0.0008) [2023-12-27 07:34:36,175][20381] Updated weights for policy 1, policy_version 278346 (0.0006) [2023-12-27 07:34:36,205][20347] Updated weights for policy 0, policy_version 278742 (0.0008) [2023-12-27 07:34:36,238][20381] Updated weights for policy 1, policy_version 278356 (0.0011) [2023-12-27 07:34:36,260][20347] Updated weights for policy 0, policy_version 278752 (0.0008) [2023-12-27 07:34:36,298][20381] Updated weights for policy 1, policy_version 278366 (0.0011) [2023-12-27 07:34:36,357][20381] Updated weights for policy 1, policy_version 278376 (0.0011) [2023-12-27 07:34:36,987][20347] Updated weights for policy 0, policy_version 278762 (0.0010) [2023-12-27 07:34:37,007][20381] Updated weights for policy 1, policy_version 278386 (0.0006) [2023-12-27 07:34:37,038][20347] Updated weights for policy 0, policy_version 278772 (0.0007) [2023-12-27 07:34:37,059][20381] Updated weights for policy 1, policy_version 278396 (0.0005) [2023-12-27 07:34:37,092][20347] Updated weights for policy 0, policy_version 278782 (0.0007) [2023-12-27 07:34:37,110][20381] Updated weights for policy 1, policy_version 278406 (0.0007) [2023-12-27 07:34:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 142671872. Throughput: 0: 10373.2, 1: 10426.1. Samples: 142636460. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:37,363][19170] Avg episode reward: [(0, '882.780'), (1, '985.480')] [2023-12-27 07:34:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000278408_71286784.pth... [2023-12-27 07:34:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000278784_71385088.pth... [2023-12-27 07:34:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000277192_70975488.pth [2023-12-27 07:34:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000277568_71073792.pth [2023-12-27 07:34:37,685][20347] Updated weights for policy 0, policy_version 278792 (0.0008) [2023-12-27 07:34:37,699][20381] Updated weights for policy 1, policy_version 278416 (0.0008) [2023-12-27 07:34:37,736][20347] Updated weights for policy 0, policy_version 278802 (0.0008) [2023-12-27 07:34:37,754][20381] Updated weights for policy 1, policy_version 278426 (0.0007) [2023-12-27 07:34:37,791][20347] Updated weights for policy 0, policy_version 278812 (0.0005) [2023-12-27 07:34:37,807][20381] Updated weights for policy 1, policy_version 278436 (0.0008) [2023-12-27 07:34:38,368][20347] Updated weights for policy 0, policy_version 278822 (0.0008) [2023-12-27 07:34:38,423][20347] Updated weights for policy 0, policy_version 278832 (0.0011) [2023-12-27 07:34:38,473][20347] Updated weights for policy 0, policy_version 278842 (0.0011) [2023-12-27 07:34:38,496][20381] Updated weights for policy 1, policy_version 278446 (0.0008) [2023-12-27 07:34:38,551][20381] Updated weights for policy 1, policy_version 278456 (0.0008) [2023-12-27 07:34:38,606][20381] Updated weights for policy 1, policy_version 278466 (0.0008) [2023-12-27 07:34:39,174][20347] Updated weights for policy 0, policy_version 278852 (0.0011) [2023-12-27 07:34:39,232][20347] Updated weights for policy 0, policy_version 278862 (0.0009) [2023-12-27 07:34:39,294][20347] Updated weights for policy 0, policy_version 278872 (0.0009) [2023-12-27 07:34:39,329][20381] Updated weights for policy 1, policy_version 278476 (0.0007) [2023-12-27 07:34:39,397][20381] Updated weights for policy 1, policy_version 278487 (0.0008) [2023-12-27 07:34:39,456][20381] Updated weights for policy 1, policy_version 278497 (0.0007) [2023-12-27 07:34:40,043][20347] Updated weights for policy 0, policy_version 278882 (0.0011) [2023-12-27 07:34:40,099][20347] Updated weights for policy 0, policy_version 278892 (0.0010) [2023-12-27 07:34:40,159][20347] Updated weights for policy 0, policy_version 278902 (0.0009) [2023-12-27 07:34:40,170][20381] Updated weights for policy 1, policy_version 278507 (0.0007) [2023-12-27 07:34:40,220][20347] Updated weights for policy 0, policy_version 278912 (0.0009) [2023-12-27 07:34:40,228][20381] Updated weights for policy 1, policy_version 278517 (0.0007) [2023-12-27 07:34:40,281][20381] Updated weights for policy 1, policy_version 278527 (0.0010) [2023-12-27 07:34:40,885][20347] Updated weights for policy 0, policy_version 278922 (0.0010) [2023-12-27 07:34:40,930][20381] Updated weights for policy 1, policy_version 278537 (0.0005) [2023-12-27 07:34:40,940][20347] Updated weights for policy 0, policy_version 278932 (0.0010) [2023-12-27 07:34:40,983][20381] Updated weights for policy 1, policy_version 278547 (0.0006) [2023-12-27 07:34:40,992][20347] Updated weights for policy 0, policy_version 278942 (0.0010) [2023-12-27 07:34:41,044][20381] Updated weights for policy 1, policy_version 278557 (0.0007) [2023-12-27 07:34:41,105][20381] Updated weights for policy 1, policy_version 278567 (0.0008) [2023-12-27 07:34:41,626][20347] Updated weights for policy 0, policy_version 278952 (0.0010) [2023-12-27 07:34:41,654][20291] KL-divergence is very high: 102.2706 [2023-12-27 07:34:41,686][20347] Updated weights for policy 0, policy_version 278962 (0.0008) [2023-12-27 07:34:41,756][20347] Updated weights for policy 0, policy_version 278972 (0.0010) [2023-12-27 07:34:41,756][20291] KL-divergence is very high: 106.4492 [2023-12-27 07:34:41,853][20381] Updated weights for policy 1, policy_version 278577 (0.0006) [2023-12-27 07:34:41,922][20381] Updated weights for policy 1, policy_version 278587 (0.0006) [2023-12-27 07:34:41,987][20381] Updated weights for policy 1, policy_version 278597 (0.0007) [2023-12-27 07:34:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 142770176. Throughput: 0: 10431.2, 1: 10398.3. Samples: 142762172. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:42,363][19170] Avg episode reward: [(0, '905.070'), (1, '992.220')] [2023-12-27 07:34:42,463][20347] Updated weights for policy 0, policy_version 278982 (0.0010) [2023-12-27 07:34:42,524][20347] Updated weights for policy 0, policy_version 278992 (0.0010) [2023-12-27 07:34:42,572][20347] Updated weights for policy 0, policy_version 279002 (0.0007) [2023-12-27 07:34:42,668][20381] Updated weights for policy 1, policy_version 278607 (0.0008) [2023-12-27 07:34:42,726][20381] Updated weights for policy 1, policy_version 278617 (0.0007) [2023-12-27 07:34:42,777][20381] Updated weights for policy 1, policy_version 278627 (0.0010) [2023-12-27 07:34:43,158][20347] Updated weights for policy 0, policy_version 279012 (0.0005) [2023-12-27 07:34:43,204][20347] Updated weights for policy 0, policy_version 279022 (0.0005) [2023-12-27 07:34:43,254][20347] Updated weights for policy 0, policy_version 279032 (0.0005) [2023-12-27 07:34:43,467][20381] Updated weights for policy 1, policy_version 278637 (0.0010) [2023-12-27 07:34:43,522][20381] Updated weights for policy 1, policy_version 278647 (0.0010) [2023-12-27 07:34:43,581][20381] Updated weights for policy 1, policy_version 278657 (0.0011) [2023-12-27 07:34:43,856][20347] Updated weights for policy 0, policy_version 279042 (0.0008) [2023-12-27 07:34:43,901][20347] Updated weights for policy 0, policy_version 279052 (0.0010) [2023-12-27 07:34:43,950][20347] Updated weights for policy 0, policy_version 279062 (0.0010) [2023-12-27 07:34:44,002][20347] Updated weights for policy 0, policy_version 279072 (0.0010) [2023-12-27 07:34:44,267][20381] Updated weights for policy 1, policy_version 278667 (0.0011) [2023-12-27 07:34:44,326][20381] Updated weights for policy 1, policy_version 278677 (0.0011) [2023-12-27 07:34:44,385][20381] Updated weights for policy 1, policy_version 278687 (0.0011) [2023-12-27 07:34:44,596][20347] Updated weights for policy 0, policy_version 279082 (0.0008) [2023-12-27 07:34:44,649][20347] Updated weights for policy 0, policy_version 279092 (0.0008) [2023-12-27 07:34:44,702][20347] Updated weights for policy 0, policy_version 279102 (0.0008) [2023-12-27 07:34:45,021][20381] Updated weights for policy 1, policy_version 278697 (0.0008) [2023-12-27 07:34:45,092][20381] Updated weights for policy 1, policy_version 278707 (0.0007) [2023-12-27 07:34:45,155][20381] Updated weights for policy 1, policy_version 278717 (0.0008) [2023-12-27 07:34:45,220][20381] Updated weights for policy 1, policy_version 278727 (0.0007) [2023-12-27 07:34:45,478][20347] Updated weights for policy 0, policy_version 279112 (0.0008) [2023-12-27 07:34:45,528][20347] Updated weights for policy 0, policy_version 279122 (0.0008) [2023-12-27 07:34:45,579][20347] Updated weights for policy 0, policy_version 279132 (0.0007) [2023-12-27 07:34:45,895][20381] Updated weights for policy 1, policy_version 278737 (0.0010) [2023-12-27 07:34:45,956][20381] Updated weights for policy 1, policy_version 278747 (0.0010) [2023-12-27 07:34:46,008][20381] Updated weights for policy 1, policy_version 278757 (0.0010) [2023-12-27 07:34:46,215][20347] Updated weights for policy 0, policy_version 279142 (0.0006) [2023-12-27 07:34:46,261][20347] Updated weights for policy 0, policy_version 279152 (0.0005) [2023-12-27 07:34:46,313][20347] Updated weights for policy 0, policy_version 279162 (0.0005) [2023-12-27 07:34:46,655][20381] Updated weights for policy 1, policy_version 278767 (0.0011) [2023-12-27 07:34:46,703][20381] Updated weights for policy 1, policy_version 278777 (0.0010) [2023-12-27 07:34:46,748][20381] Updated weights for policy 1, policy_version 278787 (0.0010) [2023-12-27 07:34:46,859][20347] Updated weights for policy 0, policy_version 279172 (0.0005) [2023-12-27 07:34:46,912][20347] Updated weights for policy 0, policy_version 279182 (0.0005) [2023-12-27 07:34:46,972][20347] Updated weights for policy 0, policy_version 279192 (0.0008) [2023-12-27 07:34:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 142876672. Throughput: 0: 10514.0, 1: 10411.6. Samples: 142889016. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:47,363][19170] Avg episode reward: [(0, '943.170'), (1, '1000.000')] [2023-12-27 07:34:47,455][20381] Updated weights for policy 1, policy_version 278797 (0.0009) [2023-12-27 07:34:47,500][20381] Updated weights for policy 1, policy_version 278807 (0.0010) [2023-12-27 07:34:47,559][20347] Updated weights for policy 0, policy_version 279202 (0.0008) [2023-12-27 07:34:47,564][20381] Updated weights for policy 1, policy_version 278817 (0.0011) [2023-12-27 07:34:47,610][20347] Updated weights for policy 0, policy_version 279212 (0.0010) [2023-12-27 07:34:47,655][20347] Updated weights for policy 0, policy_version 279222 (0.0010) [2023-12-27 07:34:47,701][20347] Updated weights for policy 0, policy_version 279232 (0.0010) [2023-12-27 07:34:48,186][20381] Updated weights for policy 1, policy_version 278827 (0.0011) [2023-12-27 07:34:48,247][20381] Updated weights for policy 1, policy_version 278837 (0.0010) [2023-12-27 07:34:48,308][20381] Updated weights for policy 1, policy_version 278847 (0.0011) [2023-12-27 07:34:48,376][20347] Updated weights for policy 0, policy_version 279242 (0.0009) [2023-12-27 07:34:48,437][20347] Updated weights for policy 0, policy_version 279252 (0.0008) [2023-12-27 07:34:48,487][20347] Updated weights for policy 0, policy_version 279262 (0.0008) [2023-12-27 07:34:48,997][20381] Updated weights for policy 1, policy_version 278857 (0.0009) [2023-12-27 07:34:49,045][20381] Updated weights for policy 1, policy_version 278867 (0.0011) [2023-12-27 07:34:49,090][20381] Updated weights for policy 1, policy_version 278877 (0.0010) [2023-12-27 07:34:49,138][20381] Updated weights for policy 1, policy_version 278887 (0.0010) [2023-12-27 07:34:49,207][20347] Updated weights for policy 0, policy_version 279272 (0.0007) [2023-12-27 07:34:49,242][20291] KL-divergence is very high: 100.2890 [2023-12-27 07:34:49,266][20347] Updated weights for policy 0, policy_version 279282 (0.0008) [2023-12-27 07:34:49,311][20347] Updated weights for policy 0, policy_version 279292 (0.0007) [2023-12-27 07:34:49,821][20381] Updated weights for policy 1, policy_version 278897 (0.0010) [2023-12-27 07:34:49,883][20381] Updated weights for policy 1, policy_version 278907 (0.0008) [2023-12-27 07:34:49,943][20381] Updated weights for policy 1, policy_version 278917 (0.0007) [2023-12-27 07:34:50,016][20347] Updated weights for policy 0, policy_version 279302 (0.0007) [2023-12-27 07:34:50,076][20347] Updated weights for policy 0, policy_version 279312 (0.0007) [2023-12-27 07:34:50,125][20347] Updated weights for policy 0, policy_version 279322 (0.0007) [2023-12-27 07:34:50,604][20381] Updated weights for policy 1, policy_version 278927 (0.0008) [2023-12-27 07:34:50,658][20381] Updated weights for policy 1, policy_version 278937 (0.0007) [2023-12-27 07:34:50,712][20381] Updated weights for policy 1, policy_version 278947 (0.0008) [2023-12-27 07:34:50,846][20347] Updated weights for policy 0, policy_version 279332 (0.0008) [2023-12-27 07:34:50,905][20347] Updated weights for policy 0, policy_version 279342 (0.0009) [2023-12-27 07:34:50,954][20347] Updated weights for policy 0, policy_version 279352 (0.0010) [2023-12-27 07:34:51,430][20381] Updated weights for policy 1, policy_version 278957 (0.0008) [2023-12-27 07:34:51,488][20381] Updated weights for policy 1, policy_version 278967 (0.0008) [2023-12-27 07:34:51,538][20381] Updated weights for policy 1, policy_version 278977 (0.0008) [2023-12-27 07:34:51,658][20347] Updated weights for policy 0, policy_version 279362 (0.0010) [2023-12-27 07:34:51,721][20347] Updated weights for policy 0, policy_version 279372 (0.0008) [2023-12-27 07:34:51,787][20347] Updated weights for policy 0, policy_version 279382 (0.0010) [2023-12-27 07:34:51,844][20347] Updated weights for policy 0, policy_version 279392 (0.0010) [2023-12-27 07:34:52,234][20381] Updated weights for policy 1, policy_version 278987 (0.0011) [2023-12-27 07:34:52,301][20381] Updated weights for policy 1, policy_version 278997 (0.0011) [2023-12-27 07:34:52,362][20381] Updated weights for policy 1, policy_version 279007 (0.0010) [2023-12-27 07:34:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 142974976. Throughput: 0: 10456.5, 1: 10435.6. Samples: 142951300. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:52,363][19170] Avg episode reward: [(0, '933.870'), (1, '1000.000')] [2023-12-27 07:34:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000279392_71540736.pth... [2023-12-27 07:34:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000278176_71229440.pth [2023-12-27 07:34:52,416][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000279016_71442432.pth... [2023-12-27 07:34:52,420][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000277800_71131136.pth [2023-12-27 07:34:52,518][20347] Updated weights for policy 0, policy_version 279402 (0.0005) [2023-12-27 07:34:52,582][20347] Updated weights for policy 0, policy_version 279412 (0.0009) [2023-12-27 07:34:52,641][20347] Updated weights for policy 0, policy_version 279422 (0.0010) [2023-12-27 07:34:53,059][20381] Updated weights for policy 1, policy_version 279017 (0.0009) [2023-12-27 07:34:53,108][20381] Updated weights for policy 1, policy_version 279027 (0.0007) [2023-12-27 07:34:53,156][20381] Updated weights for policy 1, policy_version 279037 (0.0008) [2023-12-27 07:34:53,220][20381] Updated weights for policy 1, policy_version 279047 (0.0008) [2023-12-27 07:34:53,290][20347] Updated weights for policy 0, policy_version 279432 (0.0009) [2023-12-27 07:34:53,338][20347] Updated weights for policy 0, policy_version 279442 (0.0010) [2023-12-27 07:34:53,389][20347] Updated weights for policy 0, policy_version 279452 (0.0010) [2023-12-27 07:34:53,929][20381] Updated weights for policy 1, policy_version 279057 (0.0008) [2023-12-27 07:34:53,980][20381] Updated weights for policy 1, policy_version 279067 (0.0008) [2023-12-27 07:34:53,982][20347] Updated weights for policy 0, policy_version 279462 (0.0010) [2023-12-27 07:34:54,028][20381] Updated weights for policy 1, policy_version 279077 (0.0008) [2023-12-27 07:34:54,040][20347] Updated weights for policy 0, policy_version 279472 (0.0010) [2023-12-27 07:34:54,094][20347] Updated weights for policy 0, policy_version 279482 (0.0010) [2023-12-27 07:34:54,705][20381] Updated weights for policy 1, policy_version 279087 (0.0009) [2023-12-27 07:34:54,769][20381] Updated weights for policy 1, policy_version 279097 (0.0008) [2023-12-27 07:34:54,772][20347] Updated weights for policy 0, policy_version 279492 (0.0009) [2023-12-27 07:34:54,831][20381] Updated weights for policy 1, policy_version 279107 (0.0007) [2023-12-27 07:34:54,838][20347] Updated weights for policy 0, policy_version 279502 (0.0007) [2023-12-27 07:34:54,885][20347] Updated weights for policy 0, policy_version 279512 (0.0007) [2023-12-27 07:34:55,490][20381] Updated weights for policy 1, policy_version 279117 (0.0011) [2023-12-27 07:34:55,510][20347] Updated weights for policy 0, policy_version 279522 (0.0008) [2023-12-27 07:34:55,541][20381] Updated weights for policy 1, policy_version 279127 (0.0010) [2023-12-27 07:34:55,558][20347] Updated weights for policy 0, policy_version 279532 (0.0005) [2023-12-27 07:34:55,586][20381] Updated weights for policy 1, policy_version 279137 (0.0010) [2023-12-27 07:34:55,603][20347] Updated weights for policy 0, policy_version 279542 (0.0009) [2023-12-27 07:34:55,648][20347] Updated weights for policy 0, policy_version 279552 (0.0008) [2023-12-27 07:34:56,257][20381] Updated weights for policy 1, policy_version 279147 (0.0009) [2023-12-27 07:34:56,306][20381] Updated weights for policy 1, policy_version 279157 (0.0010) [2023-12-27 07:34:56,331][20347] Updated weights for policy 0, policy_version 279562 (0.0005) [2023-12-27 07:34:56,350][20381] Updated weights for policy 1, policy_version 279167 (0.0010) [2023-12-27 07:34:56,390][20347] Updated weights for policy 0, policy_version 279572 (0.0005) [2023-12-27 07:34:56,450][20347] Updated weights for policy 0, policy_version 279582 (0.0006) [2023-12-27 07:34:57,017][20347] Updated weights for policy 0, policy_version 279592 (0.0007) [2023-12-27 07:34:57,042][20381] Updated weights for policy 1, policy_version 279177 (0.0010) [2023-12-27 07:34:57,072][20347] Updated weights for policy 0, policy_version 279602 (0.0009) [2023-12-27 07:34:57,096][20381] Updated weights for policy 1, policy_version 279187 (0.0010) [2023-12-27 07:34:57,119][20347] Updated weights for policy 0, policy_version 279612 (0.0007) [2023-12-27 07:34:57,154][20381] Updated weights for policy 1, policy_version 279197 (0.0011) [2023-12-27 07:34:57,211][20381] Updated weights for policy 1, policy_version 279207 (0.0010) [2023-12-27 07:34:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.7, 300 sec: 20799.3). Total num frames: 143089664. Throughput: 0: 10484.2, 1: 10369.4. Samples: 143074208. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:34:57,363][19170] Avg episode reward: [(0, '883.860'), (1, '1000.000')] [2023-12-27 07:34:57,726][20347] Updated weights for policy 0, policy_version 279622 (0.0008) [2023-12-27 07:34:57,772][20347] Updated weights for policy 0, policy_version 279632 (0.0005) [2023-12-27 07:34:57,823][20347] Updated weights for policy 0, policy_version 279642 (0.0005) [2023-12-27 07:34:57,886][20381] Updated weights for policy 1, policy_version 279217 (0.0010) [2023-12-27 07:34:57,940][20381] Updated weights for policy 1, policy_version 279227 (0.0010) [2023-12-27 07:34:57,988][20381] Updated weights for policy 1, policy_version 279237 (0.0010) [2023-12-27 07:34:58,470][20347] Updated weights for policy 0, policy_version 279652 (0.0006) [2023-12-27 07:34:58,537][20347] Updated weights for policy 0, policy_version 279662 (0.0008) [2023-12-27 07:34:58,612][20347] Updated weights for policy 0, policy_version 279672 (0.0007) [2023-12-27 07:34:58,695][20381] Updated weights for policy 1, policy_version 279247 (0.0011) [2023-12-27 07:34:58,758][20381] Updated weights for policy 1, policy_version 279257 (0.0011) [2023-12-27 07:34:58,822][20381] Updated weights for policy 1, policy_version 279267 (0.0010) [2023-12-27 07:34:59,348][20347] Updated weights for policy 0, policy_version 279682 (0.0010) [2023-12-27 07:34:59,409][20347] Updated weights for policy 0, policy_version 279692 (0.0009) [2023-12-27 07:34:59,463][20347] Updated weights for policy 0, policy_version 279702 (0.0007) [2023-12-27 07:34:59,518][20347] Updated weights for policy 0, policy_version 279712 (0.0009) [2023-12-27 07:34:59,519][20381] Updated weights for policy 1, policy_version 279277 (0.0007) [2023-12-27 07:34:59,566][20381] Updated weights for policy 1, policy_version 279287 (0.0008) [2023-12-27 07:34:59,619][20381] Updated weights for policy 1, policy_version 279297 (0.0009) [2023-12-27 07:35:00,183][20347] Updated weights for policy 0, policy_version 279722 (0.0009) [2023-12-27 07:35:00,246][20347] Updated weights for policy 0, policy_version 279732 (0.0010) [2023-12-27 07:35:00,306][20347] Updated weights for policy 0, policy_version 279742 (0.0010) [2023-12-27 07:35:00,354][20381] Updated weights for policy 1, policy_version 279307 (0.0008) [2023-12-27 07:35:00,413][20381] Updated weights for policy 1, policy_version 279317 (0.0008) [2023-12-27 07:35:00,470][20381] Updated weights for policy 1, policy_version 279327 (0.0007) [2023-12-27 07:35:00,994][20347] Updated weights for policy 0, policy_version 279752 (0.0007) [2023-12-27 07:35:01,054][20347] Updated weights for policy 0, policy_version 279762 (0.0010) [2023-12-27 07:35:01,110][20347] Updated weights for policy 0, policy_version 279772 (0.0011) [2023-12-27 07:35:01,167][20381] Updated weights for policy 1, policy_version 279337 (0.0007) [2023-12-27 07:35:01,232][20381] Updated weights for policy 1, policy_version 279347 (0.0011) [2023-12-27 07:35:01,293][20381] Updated weights for policy 1, policy_version 279357 (0.0009) [2023-12-27 07:35:01,361][20381] Updated weights for policy 1, policy_version 279367 (0.0007) [2023-12-27 07:35:01,736][20347] Updated weights for policy 0, policy_version 279782 (0.0010) [2023-12-27 07:35:01,791][20347] Updated weights for policy 0, policy_version 279792 (0.0008) [2023-12-27 07:35:01,842][20347] Updated weights for policy 0, policy_version 279802 (0.0005) [2023-12-27 07:35:02,043][20381] Updated weights for policy 1, policy_version 279377 (0.0005) [2023-12-27 07:35:02,101][20381] Updated weights for policy 1, policy_version 279387 (0.0008) [2023-12-27 07:35:02,156][20381] Updated weights for policy 1, policy_version 279397 (0.0007) [2023-12-27 07:35:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 143187968. Throughput: 0: 10502.7, 1: 10318.1. Samples: 143197728. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:35:02,363][19170] Avg episode reward: [(0, '845.080'), (1, '1000.000')] [2023-12-27 07:35:02,520][20347] Updated weights for policy 0, policy_version 279812 (0.0006) [2023-12-27 07:35:02,572][20347] Updated weights for policy 0, policy_version 279822 (0.0007) [2023-12-27 07:35:02,626][20347] Updated weights for policy 0, policy_version 279832 (0.0008) [2023-12-27 07:35:02,802][20381] Updated weights for policy 1, policy_version 279407 (0.0007) [2023-12-27 07:35:02,861][20381] Updated weights for policy 1, policy_version 279417 (0.0006) [2023-12-27 07:35:02,918][20381] Updated weights for policy 1, policy_version 279427 (0.0007) [2023-12-27 07:35:03,335][20347] Updated weights for policy 0, policy_version 279842 (0.0008) [2023-12-27 07:35:03,377][20347] Updated weights for policy 0, policy_version 279852 (0.0007) [2023-12-27 07:35:03,427][20347] Updated weights for policy 0, policy_version 279862 (0.0005) [2023-12-27 07:35:03,489][20347] Updated weights for policy 0, policy_version 279872 (0.0010) [2023-12-27 07:35:03,561][20381] Updated weights for policy 1, policy_version 279437 (0.0007) [2023-12-27 07:35:03,619][20381] Updated weights for policy 1, policy_version 279447 (0.0006) [2023-12-27 07:35:03,679][20381] Updated weights for policy 1, policy_version 279457 (0.0006) [2023-12-27 07:35:04,112][20347] Updated weights for policy 0, policy_version 279882 (0.0006) [2023-12-27 07:35:04,167][20347] Updated weights for policy 0, policy_version 279892 (0.0005) [2023-12-27 07:35:04,222][20347] Updated weights for policy 0, policy_version 279902 (0.0005) [2023-12-27 07:35:04,296][20381] Updated weights for policy 1, policy_version 279467 (0.0006) [2023-12-27 07:35:04,352][20381] Updated weights for policy 1, policy_version 279477 (0.0008) [2023-12-27 07:35:04,408][20381] Updated weights for policy 1, policy_version 279487 (0.0008) [2023-12-27 07:35:04,836][20347] Updated weights for policy 0, policy_version 279912 (0.0007) [2023-12-27 07:35:04,892][20347] Updated weights for policy 0, policy_version 279922 (0.0006) [2023-12-27 07:35:04,945][20347] Updated weights for policy 0, policy_version 279932 (0.0006) [2023-12-27 07:35:05,103][20381] Updated weights for policy 1, policy_version 279497 (0.0008) [2023-12-27 07:35:05,167][20381] Updated weights for policy 1, policy_version 279507 (0.0010) [2023-12-27 07:35:05,216][20381] Updated weights for policy 1, policy_version 279517 (0.0010) [2023-12-27 07:35:05,267][20381] Updated weights for policy 1, policy_version 279527 (0.0010) [2023-12-27 07:35:05,593][20347] Updated weights for policy 0, policy_version 279942 (0.0006) [2023-12-27 07:35:05,647][20347] Updated weights for policy 0, policy_version 279952 (0.0007) [2023-12-27 07:35:05,705][20347] Updated weights for policy 0, policy_version 279962 (0.0007) [2023-12-27 07:35:05,948][20381] Updated weights for policy 1, policy_version 279537 (0.0008) [2023-12-27 07:35:06,008][20381] Updated weights for policy 1, policy_version 279547 (0.0008) [2023-12-27 07:35:06,065][20381] Updated weights for policy 1, policy_version 279557 (0.0008) [2023-12-27 07:35:06,419][20347] Updated weights for policy 0, policy_version 279972 (0.0007) [2023-12-27 07:35:06,484][20347] Updated weights for policy 0, policy_version 279982 (0.0008) [2023-12-27 07:35:06,553][20347] Updated weights for policy 0, policy_version 279992 (0.0010) [2023-12-27 07:35:06,739][20381] Updated weights for policy 1, policy_version 279567 (0.0007) [2023-12-27 07:35:06,790][20381] Updated weights for policy 1, policy_version 279577 (0.0008) [2023-12-27 07:35:06,844][20381] Updated weights for policy 1, policy_version 279587 (0.0008) [2023-12-27 07:35:07,247][20347] Updated weights for policy 0, policy_version 280002 (0.0008) [2023-12-27 07:35:07,289][20347] Updated weights for policy 0, policy_version 280012 (0.0007) [2023-12-27 07:35:07,332][20347] Updated weights for policy 0, policy_version 280022 (0.0007) [2023-12-27 07:35:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 143286272. Throughput: 0: 10511.1, 1: 10347.2. Samples: 143261644. Policy #0 lag: (min: 18.0, avg: 31.9, max: 50.0) [2023-12-27 07:35:07,363][19170] Avg episode reward: [(0, '914.190'), (1, '1000.000')] [2023-12-27 07:35:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000279592_71589888.pth... [2023-12-27 07:35:07,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000280032_71704576.pth... [2023-12-27 07:35:07,378][20347] Updated weights for policy 0, policy_version 280032 (0.0007) [2023-12-27 07:35:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000278784_71385088.pth [2023-12-27 07:35:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000278408_71286784.pth [2023-12-27 07:35:07,543][20381] Updated weights for policy 1, policy_version 279597 (0.0008) [2023-12-27 07:35:07,593][20381] Updated weights for policy 1, policy_version 279607 (0.0007) [2023-12-27 07:35:07,638][20381] Updated weights for policy 1, policy_version 279617 (0.0008) [2023-12-27 07:35:08,123][20347] Updated weights for policy 0, policy_version 280042 (0.0009) [2023-12-27 07:35:08,181][20347] Updated weights for policy 0, policy_version 280052 (0.0008) [2023-12-27 07:35:08,239][20347] Updated weights for policy 0, policy_version 280062 (0.0007) [2023-12-27 07:35:08,278][20381] Updated weights for policy 1, policy_version 279627 (0.0007) [2023-12-27 07:35:08,340][20381] Updated weights for policy 1, policy_version 279637 (0.0008) [2023-12-27 07:35:08,401][20381] Updated weights for policy 1, policy_version 279647 (0.0010) [2023-12-27 07:35:08,957][20347] Updated weights for policy 0, policy_version 280072 (0.0008) [2023-12-27 07:35:08,999][20381] Updated weights for policy 1, policy_version 279657 (0.0009) [2023-12-27 07:35:09,009][20347] Updated weights for policy 0, policy_version 280082 (0.0007) [2023-12-27 07:35:09,056][20347] Updated weights for policy 0, policy_version 280092 (0.0007) [2023-12-27 07:35:09,063][20381] Updated weights for policy 1, policy_version 279667 (0.0010) [2023-12-27 07:35:09,112][20381] Updated weights for policy 1, policy_version 279677 (0.0005) [2023-12-27 07:35:09,167][20381] Updated weights for policy 1, policy_version 279687 (0.0005) [2023-12-27 07:35:09,764][20347] Updated weights for policy 0, policy_version 280102 (0.0007) [2023-12-27 07:35:09,804][20381] Updated weights for policy 1, policy_version 279697 (0.0010) [2023-12-27 07:35:09,824][20347] Updated weights for policy 0, policy_version 280112 (0.0007) [2023-12-27 07:35:09,866][20381] Updated weights for policy 1, policy_version 279707 (0.0011) [2023-12-27 07:35:09,887][20347] Updated weights for policy 0, policy_version 280122 (0.0007) [2023-12-27 07:35:09,926][20381] Updated weights for policy 1, policy_version 279717 (0.0011) [2023-12-27 07:35:10,600][20381] Updated weights for policy 1, policy_version 279727 (0.0007) [2023-12-27 07:35:10,612][20347] Updated weights for policy 0, policy_version 280132 (0.0006) [2023-12-27 07:35:10,651][20381] Updated weights for policy 1, policy_version 279737 (0.0010) [2023-12-27 07:35:10,671][20347] Updated weights for policy 0, policy_version 280142 (0.0006) [2023-12-27 07:35:10,704][20381] Updated weights for policy 1, policy_version 279747 (0.0011) [2023-12-27 07:35:10,726][20347] Updated weights for policy 0, policy_version 280152 (0.0007) [2023-12-27 07:35:11,312][20381] Updated weights for policy 1, policy_version 279757 (0.0010) [2023-12-27 07:35:11,381][20381] Updated weights for policy 1, policy_version 279767 (0.0009) [2023-12-27 07:35:11,450][20381] Updated weights for policy 1, policy_version 279777 (0.0008) [2023-12-27 07:35:11,488][20347] Updated weights for policy 0, policy_version 280162 (0.0009) [2023-12-27 07:35:11,538][20347] Updated weights for policy 0, policy_version 280172 (0.0007) [2023-12-27 07:35:11,588][20347] Updated weights for policy 0, policy_version 280182 (0.0007) [2023-12-27 07:35:11,657][20347] Updated weights for policy 0, policy_version 280192 (0.0008) [2023-12-27 07:35:12,077][20381] Updated weights for policy 1, policy_version 279787 (0.0007) [2023-12-27 07:35:12,144][20381] Updated weights for policy 1, policy_version 279797 (0.0006) [2023-12-27 07:35:12,208][20381] Updated weights for policy 1, policy_version 279807 (0.0008) [2023-12-27 07:35:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 143392768. Throughput: 0: 10489.8, 1: 10424.0. Samples: 143383868. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:12,363][19170] Avg episode reward: [(0, '873.980'), (1, '1000.000')] [2023-12-27 07:35:12,392][20347] Updated weights for policy 0, policy_version 280202 (0.0010) [2023-12-27 07:35:12,450][20347] Updated weights for policy 0, policy_version 280212 (0.0010) [2023-12-27 07:35:12,508][20347] Updated weights for policy 0, policy_version 280222 (0.0010) [2023-12-27 07:35:12,813][20381] Updated weights for policy 1, policy_version 279817 (0.0008) [2023-12-27 07:35:12,867][20381] Updated weights for policy 1, policy_version 279827 (0.0007) [2023-12-27 07:35:12,932][20381] Updated weights for policy 1, policy_version 279837 (0.0006) [2023-12-27 07:35:12,981][20381] Updated weights for policy 1, policy_version 279847 (0.0005) [2023-12-27 07:35:13,187][20347] Updated weights for policy 0, policy_version 280232 (0.0008) [2023-12-27 07:35:13,245][20347] Updated weights for policy 0, policy_version 280242 (0.0008) [2023-12-27 07:35:13,294][20347] Updated weights for policy 0, policy_version 280252 (0.0005) [2023-12-27 07:35:13,558][20381] Updated weights for policy 1, policy_version 279857 (0.0005) [2023-12-27 07:35:13,612][20381] Updated weights for policy 1, policy_version 279867 (0.0005) [2023-12-27 07:35:13,678][20381] Updated weights for policy 1, policy_version 279877 (0.0008) [2023-12-27 07:35:13,971][20347] Updated weights for policy 0, policy_version 280262 (0.0006) [2023-12-27 07:35:14,027][20347] Updated weights for policy 0, policy_version 280272 (0.0007) [2023-12-27 07:35:14,087][20347] Updated weights for policy 0, policy_version 280282 (0.0007) [2023-12-27 07:35:14,333][20381] Updated weights for policy 1, policy_version 279887 (0.0009) [2023-12-27 07:35:14,387][20381] Updated weights for policy 1, policy_version 279897 (0.0007) [2023-12-27 07:35:14,445][20381] Updated weights for policy 1, policy_version 279907 (0.0007) [2023-12-27 07:35:14,774][20347] Updated weights for policy 0, policy_version 280292 (0.0008) [2023-12-27 07:35:14,825][20347] Updated weights for policy 0, policy_version 280302 (0.0007) [2023-12-27 07:35:14,881][20347] Updated weights for policy 0, policy_version 280312 (0.0008) [2023-12-27 07:35:15,137][20381] Updated weights for policy 1, policy_version 279917 (0.0009) [2023-12-27 07:35:15,193][20381] Updated weights for policy 1, policy_version 279927 (0.0010) [2023-12-27 07:35:15,249][20381] Updated weights for policy 1, policy_version 279937 (0.0011) [2023-12-27 07:35:15,605][20347] Updated weights for policy 0, policy_version 280322 (0.0007) [2023-12-27 07:35:15,669][20347] Updated weights for policy 0, policy_version 280332 (0.0007) [2023-12-27 07:35:15,727][20347] Updated weights for policy 0, policy_version 280342 (0.0008) [2023-12-27 07:35:15,787][20347] Updated weights for policy 0, policy_version 280352 (0.0007) [2023-12-27 07:35:15,957][20381] Updated weights for policy 1, policy_version 279947 (0.0011) [2023-12-27 07:35:16,014][20381] Updated weights for policy 1, policy_version 279957 (0.0006) [2023-12-27 07:35:16,070][20381] Updated weights for policy 1, policy_version 279967 (0.0005) [2023-12-27 07:35:16,461][20347] Updated weights for policy 0, policy_version 280362 (0.0007) [2023-12-27 07:35:16,519][20347] Updated weights for policy 0, policy_version 280372 (0.0007) [2023-12-27 07:35:16,576][20347] Updated weights for policy 0, policy_version 280382 (0.0007) [2023-12-27 07:35:16,702][20381] Updated weights for policy 1, policy_version 279977 (0.0006) [2023-12-27 07:35:16,750][20381] Updated weights for policy 1, policy_version 279987 (0.0008) [2023-12-27 07:35:16,797][20381] Updated weights for policy 1, policy_version 279997 (0.0007) [2023-12-27 07:35:16,844][20381] Updated weights for policy 1, policy_version 280007 (0.0007) [2023-12-27 07:35:17,238][20347] Updated weights for policy 0, policy_version 280392 (0.0008) [2023-12-27 07:35:17,291][20347] Updated weights for policy 0, policy_version 280402 (0.0009) [2023-12-27 07:35:17,349][20347] Updated weights for policy 0, policy_version 280412 (0.0008) [2023-12-27 07:35:17,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 143491072. Throughput: 0: 10423.1, 1: 10376.0. Samples: 143507628. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:17,364][19170] Avg episode reward: [(0, '859.770'), (1, '990.450')] [2023-12-27 07:35:17,501][20381] Updated weights for policy 1, policy_version 280017 (0.0006) [2023-12-27 07:35:17,554][20381] Updated weights for policy 1, policy_version 280027 (0.0005) [2023-12-27 07:35:17,605][20381] Updated weights for policy 1, policy_version 280037 (0.0005) [2023-12-27 07:35:18,021][20347] Updated weights for policy 0, policy_version 280422 (0.0006) [2023-12-27 07:35:18,082][20347] Updated weights for policy 0, policy_version 280432 (0.0005) [2023-12-27 07:35:18,139][20347] Updated weights for policy 0, policy_version 280442 (0.0005) [2023-12-27 07:35:18,221][20381] Updated weights for policy 1, policy_version 280047 (0.0006) [2023-12-27 07:35:18,275][20381] Updated weights for policy 1, policy_version 280057 (0.0005) [2023-12-27 07:35:18,325][20381] Updated weights for policy 1, policy_version 280067 (0.0007) [2023-12-27 07:35:18,774][20347] Updated weights for policy 0, policy_version 280452 (0.0009) [2023-12-27 07:35:18,838][20347] Updated weights for policy 0, policy_version 280462 (0.0010) [2023-12-27 07:35:18,896][20347] Updated weights for policy 0, policy_version 280472 (0.0010) [2023-12-27 07:35:18,902][20381] Updated weights for policy 1, policy_version 280077 (0.0006) [2023-12-27 07:35:18,958][20381] Updated weights for policy 1, policy_version 280087 (0.0006) [2023-12-27 07:35:19,021][20381] Updated weights for policy 1, policy_version 280097 (0.0005) [2023-12-27 07:35:19,507][20347] Updated weights for policy 0, policy_version 280482 (0.0010) [2023-12-27 07:35:19,566][20347] Updated weights for policy 0, policy_version 280492 (0.0010) [2023-12-27 07:35:19,624][20381] Updated weights for policy 1, policy_version 280107 (0.0005) [2023-12-27 07:35:19,626][20347] Updated weights for policy 0, policy_version 280502 (0.0011) [2023-12-27 07:35:19,682][20381] Updated weights for policy 1, policy_version 280117 (0.0006) [2023-12-27 07:35:19,685][20347] Updated weights for policy 0, policy_version 280512 (0.0011) [2023-12-27 07:35:19,740][20381] Updated weights for policy 1, policy_version 280127 (0.0007) [2023-12-27 07:35:20,367][20347] Updated weights for policy 0, policy_version 280522 (0.0007) [2023-12-27 07:35:20,399][20381] Updated weights for policy 1, policy_version 280137 (0.0007) [2023-12-27 07:35:20,425][20347] Updated weights for policy 0, policy_version 280532 (0.0006) [2023-12-27 07:35:20,460][20381] Updated weights for policy 1, policy_version 280147 (0.0006) [2023-12-27 07:35:20,484][20347] Updated weights for policy 0, policy_version 280542 (0.0005) [2023-12-27 07:35:20,511][20381] Updated weights for policy 1, policy_version 280157 (0.0005) [2023-12-27 07:35:20,561][20381] Updated weights for policy 1, policy_version 280167 (0.0006) [2023-12-27 07:35:21,121][20347] Updated weights for policy 0, policy_version 280552 (0.0009) [2023-12-27 07:35:21,184][20347] Updated weights for policy 0, policy_version 280562 (0.0011) [2023-12-27 07:35:21,201][20381] Updated weights for policy 1, policy_version 280177 (0.0006) [2023-12-27 07:35:21,244][20347] Updated weights for policy 0, policy_version 280572 (0.0011) [2023-12-27 07:35:21,260][20381] Updated weights for policy 1, policy_version 280187 (0.0007) [2023-12-27 07:35:21,321][20381] Updated weights for policy 1, policy_version 280197 (0.0007) [2023-12-27 07:35:21,947][20381] Updated weights for policy 1, policy_version 280207 (0.0008) [2023-12-27 07:35:21,982][20347] Updated weights for policy 0, policy_version 280582 (0.0008) [2023-12-27 07:35:22,003][20381] Updated weights for policy 1, policy_version 280217 (0.0008) [2023-12-27 07:35:22,036][20347] Updated weights for policy 0, policy_version 280592 (0.0007) [2023-12-27 07:35:22,062][20381] Updated weights for policy 1, policy_version 280227 (0.0007) [2023-12-27 07:35:22,092][20347] Updated weights for policy 0, policy_version 280602 (0.0006) [2023-12-27 07:35:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 143605760. Throughput: 0: 10415.9, 1: 10387.6. Samples: 143572620. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:22,363][19170] Avg episode reward: [(0, '913.030'), (1, '990.450')] [2023-12-27 07:35:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000280232_71753728.pth... [2023-12-27 07:35:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000280608_71852032.pth... [2023-12-27 07:35:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000279016_71442432.pth [2023-12-27 07:35:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000279392_71540736.pth [2023-12-27 07:35:22,757][20381] Updated weights for policy 1, policy_version 280237 (0.0009) [2023-12-27 07:35:22,819][20381] Updated weights for policy 1, policy_version 280247 (0.0007) [2023-12-27 07:35:22,851][20347] Updated weights for policy 0, policy_version 280612 (0.0007) [2023-12-27 07:35:22,877][20381] Updated weights for policy 1, policy_version 280257 (0.0007) [2023-12-27 07:35:22,909][20347] Updated weights for policy 0, policy_version 280622 (0.0006) [2023-12-27 07:35:22,974][20347] Updated weights for policy 0, policy_version 280632 (0.0007) [2023-12-27 07:35:23,468][20381] Updated weights for policy 1, policy_version 280267 (0.0006) [2023-12-27 07:35:23,526][20381] Updated weights for policy 1, policy_version 280277 (0.0009) [2023-12-27 07:35:23,582][20381] Updated weights for policy 1, policy_version 280287 (0.0007) [2023-12-27 07:35:23,689][20347] Updated weights for policy 0, policy_version 280642 (0.0007) [2023-12-27 07:35:23,747][20347] Updated weights for policy 0, policy_version 280652 (0.0007) [2023-12-27 07:35:23,792][20347] Updated weights for policy 0, policy_version 280662 (0.0010) [2023-12-27 07:35:23,845][20347] Updated weights for policy 0, policy_version 280672 (0.0010) [2023-12-27 07:35:24,200][20381] Updated weights for policy 1, policy_version 280297 (0.0007) [2023-12-27 07:35:24,268][20381] Updated weights for policy 1, policy_version 280307 (0.0009) [2023-12-27 07:35:24,331][20381] Updated weights for policy 1, policy_version 280317 (0.0009) [2023-12-27 07:35:24,386][20381] Updated weights for policy 1, policy_version 280327 (0.0009) [2023-12-27 07:35:24,500][20347] Updated weights for policy 0, policy_version 280682 (0.0006) [2023-12-27 07:35:24,555][20347] Updated weights for policy 0, policy_version 280692 (0.0007) [2023-12-27 07:35:24,613][20347] Updated weights for policy 0, policy_version 280702 (0.0008) [2023-12-27 07:35:25,092][20381] Updated weights for policy 1, policy_version 280337 (0.0007) [2023-12-27 07:35:25,152][20381] Updated weights for policy 1, policy_version 280347 (0.0008) [2023-12-27 07:35:25,207][20381] Updated weights for policy 1, policy_version 280357 (0.0008) [2023-12-27 07:35:25,214][20347] Updated weights for policy 0, policy_version 280712 (0.0006) [2023-12-27 07:35:25,275][20347] Updated weights for policy 0, policy_version 280722 (0.0006) [2023-12-27 07:35:25,330][20347] Updated weights for policy 0, policy_version 280732 (0.0007) [2023-12-27 07:35:25,913][20381] Updated weights for policy 1, policy_version 280367 (0.0008) [2023-12-27 07:35:25,947][20347] Updated weights for policy 0, policy_version 280742 (0.0008) [2023-12-27 07:35:25,972][20381] Updated weights for policy 1, policy_version 280377 (0.0006) [2023-12-27 07:35:26,004][20347] Updated weights for policy 0, policy_version 280752 (0.0008) [2023-12-27 07:35:26,029][20381] Updated weights for policy 1, policy_version 280387 (0.0006) [2023-12-27 07:35:26,066][20347] Updated weights for policy 0, policy_version 280762 (0.0008) [2023-12-27 07:35:26,672][20347] Updated weights for policy 0, policy_version 280772 (0.0007) [2023-12-27 07:35:26,705][20381] Updated weights for policy 1, policy_version 280397 (0.0009) [2023-12-27 07:35:26,725][20347] Updated weights for policy 0, policy_version 280782 (0.0005) [2023-12-27 07:35:26,757][20381] Updated weights for policy 1, policy_version 280407 (0.0010) [2023-12-27 07:35:26,778][20347] Updated weights for policy 0, policy_version 280792 (0.0006) [2023-12-27 07:35:26,810][20381] Updated weights for policy 1, policy_version 280417 (0.0006) [2023-12-27 07:35:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 143704064. Throughput: 0: 10363.8, 1: 10388.7. Samples: 143696036. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:27,363][19170] Avg episode reward: [(0, '855.070'), (1, '1000.000')] [2023-12-27 07:35:27,422][20381] Updated weights for policy 1, policy_version 280427 (0.0005) [2023-12-27 07:35:27,448][20347] Updated weights for policy 0, policy_version 280802 (0.0006) [2023-12-27 07:35:27,482][20381] Updated weights for policy 1, policy_version 280437 (0.0005) [2023-12-27 07:35:27,508][20347] Updated weights for policy 0, policy_version 280812 (0.0007) [2023-12-27 07:35:27,541][20381] Updated weights for policy 1, policy_version 280447 (0.0005) [2023-12-27 07:35:27,568][20347] Updated weights for policy 0, policy_version 280822 (0.0008) [2023-12-27 07:35:27,618][20347] Updated weights for policy 0, policy_version 280832 (0.0008) [2023-12-27 07:35:28,124][20381] Updated weights for policy 1, policy_version 280457 (0.0005) [2023-12-27 07:35:28,176][20381] Updated weights for policy 1, policy_version 280467 (0.0005) [2023-12-27 07:35:28,226][20381] Updated weights for policy 1, policy_version 280477 (0.0007) [2023-12-27 07:35:28,276][20381] Updated weights for policy 1, policy_version 280488 (0.0008) [2023-12-27 07:35:28,322][20347] Updated weights for policy 0, policy_version 280842 (0.0007) [2023-12-27 07:35:28,379][20347] Updated weights for policy 0, policy_version 280852 (0.0007) [2023-12-27 07:35:28,426][20347] Updated weights for policy 0, policy_version 280862 (0.0007) [2023-12-27 07:35:28,997][20381] Updated weights for policy 1, policy_version 280498 (0.0010) [2023-12-27 07:35:29,059][20381] Updated weights for policy 1, policy_version 280508 (0.0010) [2023-12-27 07:35:29,113][20381] Updated weights for policy 1, policy_version 280518 (0.0010) [2023-12-27 07:35:29,116][20347] Updated weights for policy 0, policy_version 280872 (0.0007) [2023-12-27 07:35:29,164][20347] Updated weights for policy 0, policy_version 280882 (0.0008) [2023-12-27 07:35:29,219][20347] Updated weights for policy 0, policy_version 280892 (0.0008) [2023-12-27 07:35:29,765][20381] Updated weights for policy 1, policy_version 280528 (0.0006) [2023-12-27 07:35:29,820][20381] Updated weights for policy 1, policy_version 280538 (0.0006) [2023-12-27 07:35:29,875][20381] Updated weights for policy 1, policy_version 280548 (0.0008) [2023-12-27 07:35:29,965][20347] Updated weights for policy 0, policy_version 280902 (0.0007) [2023-12-27 07:35:30,025][20347] Updated weights for policy 0, policy_version 280912 (0.0007) [2023-12-27 07:35:30,087][20347] Updated weights for policy 0, policy_version 280922 (0.0006) [2023-12-27 07:35:30,601][20381] Updated weights for policy 1, policy_version 280558 (0.0007) [2023-12-27 07:35:30,660][20381] Updated weights for policy 1, policy_version 280568 (0.0007) [2023-12-27 07:35:30,709][20381] Updated weights for policy 1, policy_version 280578 (0.0007) [2023-12-27 07:35:30,778][20347] Updated weights for policy 0, policy_version 280932 (0.0007) [2023-12-27 07:35:30,834][20347] Updated weights for policy 0, policy_version 280942 (0.0008) [2023-12-27 07:35:30,879][20347] Updated weights for policy 0, policy_version 280952 (0.0008) [2023-12-27 07:35:31,386][20381] Updated weights for policy 1, policy_version 280588 (0.0007) [2023-12-27 07:35:31,445][20381] Updated weights for policy 1, policy_version 280598 (0.0008) [2023-12-27 07:35:31,497][20381] Updated weights for policy 1, policy_version 280608 (0.0008) [2023-12-27 07:35:31,585][20347] Updated weights for policy 0, policy_version 280962 (0.0008) [2023-12-27 07:35:31,644][20347] Updated weights for policy 0, policy_version 280972 (0.0008) [2023-12-27 07:35:31,696][20347] Updated weights for policy 0, policy_version 280982 (0.0008) [2023-12-27 07:35:31,757][20347] Updated weights for policy 0, policy_version 280992 (0.0008) [2023-12-27 07:35:32,185][20381] Updated weights for policy 1, policy_version 280618 (0.0005) [2023-12-27 07:35:32,241][20381] Updated weights for policy 1, policy_version 280628 (0.0005) [2023-12-27 07:35:32,300][20381] Updated weights for policy 1, policy_version 280638 (0.0006) [2023-12-27 07:35:32,356][20381] Updated weights for policy 1, policy_version 280648 (0.0009) [2023-12-27 07:35:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 143810560. Throughput: 0: 10254.6, 1: 10424.5. Samples: 143819576. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:32,363][19170] Avg episode reward: [(0, '883.700'), (1, '1000.000')] [2023-12-27 07:35:32,432][20347] Updated weights for policy 0, policy_version 281002 (0.0007) [2023-12-27 07:35:32,487][20347] Updated weights for policy 0, policy_version 281012 (0.0007) [2023-12-27 07:35:32,545][20347] Updated weights for policy 0, policy_version 281022 (0.0007) [2023-12-27 07:35:32,906][20381] Updated weights for policy 1, policy_version 280658 (0.0008) [2023-12-27 07:35:32,962][20381] Updated weights for policy 1, policy_version 280668 (0.0005) [2023-12-27 07:35:33,012][20381] Updated weights for policy 1, policy_version 280678 (0.0005) [2023-12-27 07:35:33,247][20347] Updated weights for policy 0, policy_version 281032 (0.0007) [2023-12-27 07:35:33,300][20347] Updated weights for policy 0, policy_version 281042 (0.0007) [2023-12-27 07:35:33,346][20347] Updated weights for policy 0, policy_version 281052 (0.0008) [2023-12-27 07:35:33,576][20381] Updated weights for policy 1, policy_version 280688 (0.0008) [2023-12-27 07:35:33,630][20381] Updated weights for policy 1, policy_version 280698 (0.0010) [2023-12-27 07:35:33,688][20381] Updated weights for policy 1, policy_version 280708 (0.0011) [2023-12-27 07:35:34,089][20347] Updated weights for policy 0, policy_version 281062 (0.0008) [2023-12-27 07:35:34,146][20347] Updated weights for policy 0, policy_version 281072 (0.0006) [2023-12-27 07:35:34,206][20347] Updated weights for policy 0, policy_version 281082 (0.0008) [2023-12-27 07:35:34,247][20381] Updated weights for policy 1, policy_version 280718 (0.0011) [2023-12-27 07:35:34,307][20381] Updated weights for policy 1, policy_version 280728 (0.0011) [2023-12-27 07:35:34,367][20381] Updated weights for policy 1, policy_version 280738 (0.0011) [2023-12-27 07:35:34,866][20347] Updated weights for policy 0, policy_version 281092 (0.0007) [2023-12-27 07:35:34,914][20347] Updated weights for policy 0, policy_version 281102 (0.0008) [2023-12-27 07:35:34,959][20347] Updated weights for policy 0, policy_version 281112 (0.0009) [2023-12-27 07:35:34,998][20381] Updated weights for policy 1, policy_version 280748 (0.0008) [2023-12-27 07:35:35,053][20381] Updated weights for policy 1, policy_version 280758 (0.0010) [2023-12-27 07:35:35,111][20381] Updated weights for policy 1, policy_version 280768 (0.0010) [2023-12-27 07:35:35,616][20347] Updated weights for policy 0, policy_version 281122 (0.0008) [2023-12-27 07:35:35,674][20347] Updated weights for policy 0, policy_version 281132 (0.0010) [2023-12-27 07:35:35,730][20347] Updated weights for policy 0, policy_version 281142 (0.0007) [2023-12-27 07:35:35,742][20381] Updated weights for policy 1, policy_version 280778 (0.0010) [2023-12-27 07:35:35,801][20381] Updated weights for policy 1, policy_version 280788 (0.0011) [2023-12-27 07:35:35,812][20347] Updated weights for policy 0, policy_version 281152 (0.0006) [2023-12-27 07:35:35,857][20381] Updated weights for policy 1, policy_version 280798 (0.0009) [2023-12-27 07:35:35,919][20381] Updated weights for policy 1, policy_version 280808 (0.0006) [2023-12-27 07:35:36,430][20347] Updated weights for policy 0, policy_version 281162 (0.0008) [2023-12-27 07:35:36,491][20347] Updated weights for policy 0, policy_version 281172 (0.0008) [2023-12-27 07:35:36,550][20347] Updated weights for policy 0, policy_version 281182 (0.0007) [2023-12-27 07:35:36,585][20381] Updated weights for policy 1, policy_version 280818 (0.0007) [2023-12-27 07:35:36,646][20381] Updated weights for policy 1, policy_version 280828 (0.0008) [2023-12-27 07:35:36,703][20381] Updated weights for policy 1, policy_version 280838 (0.0008) [2023-12-27 07:35:37,229][20347] Updated weights for policy 0, policy_version 281192 (0.0008) [2023-12-27 07:35:37,286][20347] Updated weights for policy 0, policy_version 281202 (0.0006) [2023-12-27 07:35:37,347][20347] Updated weights for policy 0, policy_version 281212 (0.0005) [2023-12-27 07:35:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 143908864. Throughput: 0: 10224.7, 1: 10503.1. Samples: 143884044. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:37,363][19170] Avg episode reward: [(0, '932.880'), (1, '1000.000')] [2023-12-27 07:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000280840_71909376.pth... [2023-12-27 07:35:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000281216_72007680.pth... [2023-12-27 07:35:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000279592_71589888.pth [2023-12-27 07:35:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000280032_71704576.pth [2023-12-27 07:35:37,418][20381] Updated weights for policy 1, policy_version 280848 (0.0010) [2023-12-27 07:35:37,467][20381] Updated weights for policy 1, policy_version 280858 (0.0010) [2023-12-27 07:35:37,523][20381] Updated weights for policy 1, policy_version 280868 (0.0005) [2023-12-27 07:35:37,952][20347] Updated weights for policy 0, policy_version 281222 (0.0008) [2023-12-27 07:35:38,006][20347] Updated weights for policy 0, policy_version 281232 (0.0010) [2023-12-27 07:35:38,065][20347] Updated weights for policy 0, policy_version 281242 (0.0010) [2023-12-27 07:35:38,191][20381] Updated weights for policy 1, policy_version 280878 (0.0009) [2023-12-27 07:35:38,247][20381] Updated weights for policy 1, policy_version 280888 (0.0010) [2023-12-27 07:35:38,307][20381] Updated weights for policy 1, policy_version 280898 (0.0010) [2023-12-27 07:35:38,747][20347] Updated weights for policy 0, policy_version 281252 (0.0010) [2023-12-27 07:35:38,806][20347] Updated weights for policy 0, policy_version 281262 (0.0009) [2023-12-27 07:35:38,863][20347] Updated weights for policy 0, policy_version 281272 (0.0007) [2023-12-27 07:35:38,962][20381] Updated weights for policy 1, policy_version 280908 (0.0009) [2023-12-27 07:35:39,013][20381] Updated weights for policy 1, policy_version 280918 (0.0008) [2023-12-27 07:35:39,066][20381] Updated weights for policy 1, policy_version 280928 (0.0009) [2023-12-27 07:35:39,510][20347] Updated weights for policy 0, policy_version 281282 (0.0008) [2023-12-27 07:35:39,565][20347] Updated weights for policy 0, policy_version 281292 (0.0010) [2023-12-27 07:35:39,618][20347] Updated weights for policy 0, policy_version 281302 (0.0011) [2023-12-27 07:35:39,674][20347] Updated weights for policy 0, policy_version 281312 (0.0010) [2023-12-27 07:35:39,818][20381] Updated weights for policy 1, policy_version 280938 (0.0009) [2023-12-27 07:35:39,878][20381] Updated weights for policy 1, policy_version 280948 (0.0006) [2023-12-27 07:35:39,934][20381] Updated weights for policy 1, policy_version 280958 (0.0006) [2023-12-27 07:35:39,996][20381] Updated weights for policy 1, policy_version 280968 (0.0007) [2023-12-27 07:35:40,391][20347] Updated weights for policy 0, policy_version 281322 (0.0008) [2023-12-27 07:35:40,450][20347] Updated weights for policy 0, policy_version 281332 (0.0007) [2023-12-27 07:35:40,508][20347] Updated weights for policy 0, policy_version 281342 (0.0006) [2023-12-27 07:35:40,647][20381] Updated weights for policy 1, policy_version 280978 (0.0005) [2023-12-27 07:35:40,707][20381] Updated weights for policy 1, policy_version 280988 (0.0006) [2023-12-27 07:35:40,758][20381] Updated weights for policy 1, policy_version 280998 (0.0005) [2023-12-27 07:35:41,263][20347] Updated weights for policy 0, policy_version 281352 (0.0007) [2023-12-27 07:35:41,329][20347] Updated weights for policy 0, policy_version 281362 (0.0006) [2023-12-27 07:35:41,399][20347] Updated weights for policy 0, policy_version 281372 (0.0007) [2023-12-27 07:35:41,453][20381] Updated weights for policy 1, policy_version 281008 (0.0008) [2023-12-27 07:35:41,506][20381] Updated weights for policy 1, policy_version 281018 (0.0008) [2023-12-27 07:35:41,563][20381] Updated weights for policy 1, policy_version 281028 (0.0009) [2023-12-27 07:35:42,010][20347] Updated weights for policy 0, policy_version 281382 (0.0007) [2023-12-27 07:35:42,070][20347] Updated weights for policy 0, policy_version 281392 (0.0009) [2023-12-27 07:35:42,138][20347] Updated weights for policy 0, policy_version 281402 (0.0009) [2023-12-27 07:35:42,226][20381] Updated weights for policy 1, policy_version 281038 (0.0007) [2023-12-27 07:35:42,289][20381] Updated weights for policy 1, policy_version 281048 (0.0007) [2023-12-27 07:35:42,346][20381] Updated weights for policy 1, policy_version 281058 (0.0007) [2023-12-27 07:35:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 144015360. Throughput: 0: 10226.1, 1: 10530.6. Samples: 144008260. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:42,363][19170] Avg episode reward: [(0, '971.630'), (1, '1000.000')] [2023-12-27 07:35:42,845][20347] Updated weights for policy 0, policy_version 281412 (0.0009) [2023-12-27 07:35:42,899][20347] Updated weights for policy 0, policy_version 281422 (0.0008) [2023-12-27 07:35:42,925][20381] Updated weights for policy 1, policy_version 281068 (0.0007) [2023-12-27 07:35:42,952][20347] Updated weights for policy 0, policy_version 281432 (0.0008) [2023-12-27 07:35:42,973][20381] Updated weights for policy 1, policy_version 281078 (0.0007) [2023-12-27 07:35:43,024][20381] Updated weights for policy 1, policy_version 281088 (0.0006) [2023-12-27 07:35:43,658][20347] Updated weights for policy 0, policy_version 281442 (0.0007) [2023-12-27 07:35:43,711][20347] Updated weights for policy 0, policy_version 281452 (0.0007) [2023-12-27 07:35:43,741][20381] Updated weights for policy 1, policy_version 281098 (0.0008) [2023-12-27 07:35:43,768][20347] Updated weights for policy 0, policy_version 281462 (0.0009) [2023-12-27 07:35:43,793][20381] Updated weights for policy 1, policy_version 281108 (0.0007) [2023-12-27 07:35:43,823][20347] Updated weights for policy 0, policy_version 281472 (0.0008) [2023-12-27 07:35:43,838][20381] Updated weights for policy 1, policy_version 281118 (0.0007) [2023-12-27 07:35:43,884][20381] Updated weights for policy 1, policy_version 281128 (0.0005) [2023-12-27 07:35:44,504][20381] Updated weights for policy 1, policy_version 281138 (0.0005) [2023-12-27 07:35:44,563][20381] Updated weights for policy 1, policy_version 281148 (0.0005) [2023-12-27 07:35:44,578][20347] Updated weights for policy 0, policy_version 281482 (0.0007) [2023-12-27 07:35:44,628][20381] Updated weights for policy 1, policy_version 281158 (0.0005) [2023-12-27 07:35:44,633][20347] Updated weights for policy 0, policy_version 281492 (0.0008) [2023-12-27 07:35:44,678][20347] Updated weights for policy 0, policy_version 281502 (0.0009) [2023-12-27 07:35:45,219][20381] Updated weights for policy 1, policy_version 281168 (0.0006) [2023-12-27 07:35:45,274][20381] Updated weights for policy 1, policy_version 281178 (0.0006) [2023-12-27 07:35:45,329][20381] Updated weights for policy 1, policy_version 281188 (0.0006) [2023-12-27 07:35:45,376][20347] Updated weights for policy 0, policy_version 281512 (0.0008) [2023-12-27 07:35:45,441][20347] Updated weights for policy 0, policy_version 281522 (0.0006) [2023-12-27 07:35:45,502][20347] Updated weights for policy 0, policy_version 281532 (0.0007) [2023-12-27 07:35:45,969][20381] Updated weights for policy 1, policy_version 281198 (0.0007) [2023-12-27 07:35:46,025][20381] Updated weights for policy 1, policy_version 281208 (0.0007) [2023-12-27 07:35:46,083][20381] Updated weights for policy 1, policy_version 281218 (0.0008) [2023-12-27 07:35:46,165][20347] Updated weights for policy 0, policy_version 281542 (0.0008) [2023-12-27 07:35:46,216][20347] Updated weights for policy 0, policy_version 281552 (0.0007) [2023-12-27 07:35:46,275][20347] Updated weights for policy 0, policy_version 281562 (0.0008) [2023-12-27 07:35:46,785][20381] Updated weights for policy 1, policy_version 281228 (0.0008) [2023-12-27 07:35:46,834][20381] Updated weights for policy 1, policy_version 281238 (0.0006) [2023-12-27 07:35:46,882][20381] Updated weights for policy 1, policy_version 281248 (0.0005) [2023-12-27 07:35:46,987][20347] Updated weights for policy 0, policy_version 281572 (0.0007) [2023-12-27 07:35:47,050][20347] Updated weights for policy 0, policy_version 281582 (0.0008) [2023-12-27 07:35:47,120][20347] Updated weights for policy 0, policy_version 281592 (0.0008) [2023-12-27 07:35:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 144121856. Throughput: 0: 10134.1, 1: 10630.0. Samples: 144132116. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.710')] [2023-12-27 07:35:47,520][20381] Updated weights for policy 1, policy_version 281258 (0.0005) [2023-12-27 07:35:47,588][20381] Updated weights for policy 1, policy_version 281268 (0.0009) [2023-12-27 07:35:47,648][20381] Updated weights for policy 1, policy_version 281278 (0.0007) [2023-12-27 07:35:47,704][20381] Updated weights for policy 1, policy_version 281288 (0.0007) [2023-12-27 07:35:47,720][20347] Updated weights for policy 0, policy_version 281602 (0.0008) [2023-12-27 07:35:47,770][20347] Updated weights for policy 0, policy_version 281612 (0.0007) [2023-12-27 07:35:47,827][20347] Updated weights for policy 0, policy_version 281622 (0.0006) [2023-12-27 07:35:47,885][20347] Updated weights for policy 0, policy_version 281632 (0.0009) [2023-12-27 07:35:48,397][20381] Updated weights for policy 1, policy_version 281298 (0.0006) [2023-12-27 07:35:48,456][20381] Updated weights for policy 1, policy_version 281308 (0.0006) [2023-12-27 07:35:48,519][20381] Updated weights for policy 1, policy_version 281318 (0.0006) [2023-12-27 07:35:48,536][20347] Updated weights for policy 0, policy_version 281642 (0.0008) [2023-12-27 07:35:48,594][20347] Updated weights for policy 0, policy_version 281652 (0.0008) [2023-12-27 07:35:48,652][20347] Updated weights for policy 0, policy_version 281662 (0.0008) [2023-12-27 07:35:49,151][20381] Updated weights for policy 1, policy_version 281328 (0.0007) [2023-12-27 07:35:49,207][20381] Updated weights for policy 1, policy_version 281338 (0.0007) [2023-12-27 07:35:49,273][20381] Updated weights for policy 1, policy_version 281348 (0.0008) [2023-12-27 07:35:49,358][20347] Updated weights for policy 0, policy_version 281672 (0.0009) [2023-12-27 07:35:49,424][20347] Updated weights for policy 0, policy_version 281682 (0.0006) [2023-12-27 07:35:49,484][20347] Updated weights for policy 0, policy_version 281692 (0.0005) [2023-12-27 07:35:49,986][20381] Updated weights for policy 1, policy_version 281358 (0.0007) [2023-12-27 07:35:50,043][20381] Updated weights for policy 1, policy_version 281368 (0.0006) [2023-12-27 07:35:50,106][20381] Updated weights for policy 1, policy_version 281378 (0.0008) [2023-12-27 07:35:50,131][20347] Updated weights for policy 0, policy_version 281702 (0.0008) [2023-12-27 07:35:50,186][20347] Updated weights for policy 0, policy_version 281712 (0.0007) [2023-12-27 07:35:50,251][20347] Updated weights for policy 0, policy_version 281722 (0.0007) [2023-12-27 07:35:50,801][20381] Updated weights for policy 1, policy_version 281388 (0.0008) [2023-12-27 07:35:50,860][20381] Updated weights for policy 1, policy_version 281398 (0.0007) [2023-12-27 07:35:50,922][20381] Updated weights for policy 1, policy_version 281408 (0.0007) [2023-12-27 07:35:50,946][20347] Updated weights for policy 0, policy_version 281732 (0.0008) [2023-12-27 07:35:51,001][20347] Updated weights for policy 0, policy_version 281742 (0.0011) [2023-12-27 07:35:51,064][20347] Updated weights for policy 0, policy_version 281752 (0.0011) [2023-12-27 07:35:51,599][20381] Updated weights for policy 1, policy_version 281418 (0.0006) [2023-12-27 07:35:51,660][20381] Updated weights for policy 1, policy_version 281428 (0.0008) [2023-12-27 07:35:51,719][20381] Updated weights for policy 1, policy_version 281438 (0.0007) [2023-12-27 07:35:51,735][20347] Updated weights for policy 0, policy_version 281762 (0.0012) [2023-12-27 07:35:51,786][20381] Updated weights for policy 1, policy_version 281448 (0.0008) [2023-12-27 07:35:51,791][20347] Updated weights for policy 0, policy_version 281772 (0.0008) [2023-12-27 07:35:51,839][20347] Updated weights for policy 0, policy_version 281782 (0.0008) [2023-12-27 07:35:51,887][20347] Updated weights for policy 0, policy_version 281792 (0.0008) [2023-12-27 07:35:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 144220160. Throughput: 0: 10128.6, 1: 10585.3. Samples: 144193772. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.710')] [2023-12-27 07:35:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000281792_72155136.pth... [2023-12-27 07:35:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000280608_71852032.pth [2023-12-27 07:35:52,431][20381] Updated weights for policy 1, policy_version 281458 (0.0006) [2023-12-27 07:35:52,493][20381] Updated weights for policy 1, policy_version 281468 (0.0008) [2023-12-27 07:35:52,554][20381] Updated weights for policy 1, policy_version 281478 (0.0008) [2023-12-27 07:35:52,562][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000281480_72073216.pth... [2023-12-27 07:35:52,566][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000280232_71753728.pth [2023-12-27 07:35:52,608][20347] Updated weights for policy 0, policy_version 281802 (0.0006) [2023-12-27 07:35:52,675][20347] Updated weights for policy 0, policy_version 281812 (0.0006) [2023-12-27 07:35:52,737][20347] Updated weights for policy 0, policy_version 281822 (0.0006) [2023-12-27 07:35:53,175][20381] Updated weights for policy 1, policy_version 281488 (0.0008) [2023-12-27 07:35:53,222][20381] Updated weights for policy 1, policy_version 281498 (0.0007) [2023-12-27 07:35:53,276][20381] Updated weights for policy 1, policy_version 281508 (0.0007) [2023-12-27 07:35:53,349][20347] Updated weights for policy 0, policy_version 281832 (0.0006) [2023-12-27 07:35:53,400][20347] Updated weights for policy 0, policy_version 281842 (0.0005) [2023-12-27 07:35:53,448][20347] Updated weights for policy 0, policy_version 281852 (0.0005) [2023-12-27 07:35:53,994][20347] Updated weights for policy 0, policy_version 281862 (0.0008) [2023-12-27 07:35:54,021][20381] Updated weights for policy 1, policy_version 281518 (0.0007) [2023-12-27 07:35:54,054][20347] Updated weights for policy 0, policy_version 281872 (0.0010) [2023-12-27 07:35:54,081][20381] Updated weights for policy 1, policy_version 281528 (0.0007) [2023-12-27 07:35:54,115][20347] Updated weights for policy 0, policy_version 281882 (0.0010) [2023-12-27 07:35:54,139][20381] Updated weights for policy 1, policy_version 281538 (0.0007) [2023-12-27 07:35:54,754][20381] Updated weights for policy 1, policy_version 281548 (0.0007) [2023-12-27 07:35:54,773][20347] Updated weights for policy 0, policy_version 281892 (0.0009) [2023-12-27 07:35:54,811][20381] Updated weights for policy 1, policy_version 281558 (0.0005) [2023-12-27 07:35:54,840][20347] Updated weights for policy 0, policy_version 281902 (0.0009) [2023-12-27 07:35:54,868][20381] Updated weights for policy 1, policy_version 281568 (0.0006) [2023-12-27 07:35:54,892][20347] Updated weights for policy 0, policy_version 281912 (0.0007) [2023-12-27 07:35:55,451][20381] Updated weights for policy 1, policy_version 281578 (0.0006) [2023-12-27 07:35:55,500][20381] Updated weights for policy 1, policy_version 281588 (0.0008) [2023-12-27 07:35:55,557][20381] Updated weights for policy 1, policy_version 281598 (0.0007) [2023-12-27 07:35:55,577][20347] Updated weights for policy 0, policy_version 281922 (0.0008) [2023-12-27 07:35:55,615][20381] Updated weights for policy 1, policy_version 281608 (0.0007) [2023-12-27 07:35:55,631][20347] Updated weights for policy 0, policy_version 281932 (0.0007) [2023-12-27 07:35:55,689][20347] Updated weights for policy 0, policy_version 281942 (0.0009) [2023-12-27 07:35:55,743][20347] Updated weights for policy 0, policy_version 281952 (0.0010) [2023-12-27 07:35:56,232][20381] Updated weights for policy 1, policy_version 281618 (0.0006) [2023-12-27 07:35:56,286][20381] Updated weights for policy 1, policy_version 281628 (0.0005) [2023-12-27 07:35:56,341][20381] Updated weights for policy 1, policy_version 281638 (0.0005) [2023-12-27 07:35:56,392][20347] Updated weights for policy 0, policy_version 281962 (0.0010) [2023-12-27 07:35:56,446][20347] Updated weights for policy 0, policy_version 281972 (0.0010) [2023-12-27 07:35:56,497][20347] Updated weights for policy 0, policy_version 281982 (0.0010) [2023-12-27 07:35:56,933][20381] Updated weights for policy 1, policy_version 281648 (0.0007) [2023-12-27 07:35:56,987][20381] Updated weights for policy 1, policy_version 281658 (0.0007) [2023-12-27 07:35:57,041][20381] Updated weights for policy 1, policy_version 281668 (0.0007) [2023-12-27 07:35:57,152][20347] Updated weights for policy 0, policy_version 281992 (0.0010) [2023-12-27 07:35:57,203][20347] Updated weights for policy 0, policy_version 282002 (0.0010) [2023-12-27 07:35:57,257][20347] Updated weights for policy 0, policy_version 282012 (0.0010) [2023-12-27 07:35:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 144334848. Throughput: 0: 10212.8, 1: 10587.3. Samples: 144319872. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:35:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '971.020')] [2023-12-27 07:35:57,623][20381] Updated weights for policy 1, policy_version 281678 (0.0006) [2023-12-27 07:35:57,683][20381] Updated weights for policy 1, policy_version 281688 (0.0006) [2023-12-27 07:35:57,712][20292] KL-divergence is very high: 415.7610 [2023-12-27 07:35:57,740][20381] Updated weights for policy 1, policy_version 281698 (0.0010) [2023-12-27 07:35:57,749][20292] KL-divergence is very high: 126.6430 [2023-12-27 07:35:57,755][20292] KL-divergence is very high: 132.6846 [2023-12-27 07:35:57,944][20347] Updated weights for policy 0, policy_version 282022 (0.0010) [2023-12-27 07:35:57,988][20347] Updated weights for policy 0, policy_version 282032 (0.0010) [2023-12-27 07:35:58,046][20347] Updated weights for policy 0, policy_version 282042 (0.0006) [2023-12-27 07:35:58,357][20381] Updated weights for policy 1, policy_version 281708 (0.0010) [2023-12-27 07:35:58,390][20292] KL-divergence is very high: 1887.1639 [2023-12-27 07:35:58,420][20381] Updated weights for policy 1, policy_version 281718 (0.0008) [2023-12-27 07:35:58,483][20381] Updated weights for policy 1, policy_version 281728 (0.0008) [2023-12-27 07:35:58,749][20347] Updated weights for policy 0, policy_version 282052 (0.0007) [2023-12-27 07:35:58,804][20347] Updated weights for policy 0, policy_version 282062 (0.0010) [2023-12-27 07:35:58,867][20347] Updated weights for policy 0, policy_version 282072 (0.0010) [2023-12-27 07:35:59,185][20381] Updated weights for policy 1, policy_version 281738 (0.0008) [2023-12-27 07:35:59,245][20381] Updated weights for policy 1, policy_version 281748 (0.0007) [2023-12-27 07:35:59,300][20381] Updated weights for policy 1, policy_version 281758 (0.0006) [2023-12-27 07:35:59,364][20381] Updated weights for policy 1, policy_version 281768 (0.0008) [2023-12-27 07:35:59,526][20347] Updated weights for policy 0, policy_version 282082 (0.0009) [2023-12-27 07:35:59,591][20347] Updated weights for policy 0, policy_version 282092 (0.0011) [2023-12-27 07:35:59,645][20347] Updated weights for policy 0, policy_version 282102 (0.0008) [2023-12-27 07:35:59,705][20347] Updated weights for policy 0, policy_version 282112 (0.0007) [2023-12-27 07:35:59,980][20381] Updated weights for policy 1, policy_version 281778 (0.0008) [2023-12-27 07:36:00,033][20381] Updated weights for policy 1, policy_version 281788 (0.0008) [2023-12-27 07:36:00,081][20381] Updated weights for policy 1, policy_version 281798 (0.0006) [2023-12-27 07:36:00,336][20347] Updated weights for policy 0, policy_version 282122 (0.0005) [2023-12-27 07:36:00,392][20347] Updated weights for policy 0, policy_version 282132 (0.0005) [2023-12-27 07:36:00,443][20347] Updated weights for policy 0, policy_version 282142 (0.0005) [2023-12-27 07:36:00,781][20381] Updated weights for policy 1, policy_version 281808 (0.0007) [2023-12-27 07:36:00,829][20381] Updated weights for policy 1, policy_version 281818 (0.0007) [2023-12-27 07:36:00,877][20381] Updated weights for policy 1, policy_version 281828 (0.0008) [2023-12-27 07:36:00,957][20347] Updated weights for policy 0, policy_version 282152 (0.0007) [2023-12-27 07:36:01,016][20347] Updated weights for policy 0, policy_version 282162 (0.0007) [2023-12-27 07:36:01,082][20347] Updated weights for policy 0, policy_version 282172 (0.0006) [2023-12-27 07:36:01,657][20381] Updated weights for policy 1, policy_version 281838 (0.0009) [2023-12-27 07:36:01,666][20347] Updated weights for policy 0, policy_version 282182 (0.0006) [2023-12-27 07:36:01,711][20381] Updated weights for policy 1, policy_version 281848 (0.0006) [2023-12-27 07:36:01,725][20347] Updated weights for policy 0, policy_version 282192 (0.0006) [2023-12-27 07:36:01,771][20381] Updated weights for policy 1, policy_version 281858 (0.0008) [2023-12-27 07:36:01,775][20347] Updated weights for policy 0, policy_version 282202 (0.0008) [2023-12-27 07:36:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 144433152. Throughput: 0: 10301.3, 1: 10582.3. Samples: 144447388. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:36:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.360')] [2023-12-27 07:36:02,386][20381] Updated weights for policy 1, policy_version 281868 (0.0008) [2023-12-27 07:36:02,429][20347] Updated weights for policy 0, policy_version 282212 (0.0008) [2023-12-27 07:36:02,446][20381] Updated weights for policy 1, policy_version 281878 (0.0007) [2023-12-27 07:36:02,478][20347] Updated weights for policy 0, policy_version 282222 (0.0005) [2023-12-27 07:36:02,504][20381] Updated weights for policy 1, policy_version 281888 (0.0008) [2023-12-27 07:36:02,532][20347] Updated weights for policy 0, policy_version 282232 (0.0006) [2023-12-27 07:36:03,145][20347] Updated weights for policy 0, policy_version 282242 (0.0007) [2023-12-27 07:36:03,187][20381] Updated weights for policy 1, policy_version 281898 (0.0006) [2023-12-27 07:36:03,206][20347] Updated weights for policy 0, policy_version 282252 (0.0009) [2023-12-27 07:36:03,241][20381] Updated weights for policy 1, policy_version 281908 (0.0005) [2023-12-27 07:36:03,260][20347] Updated weights for policy 0, policy_version 282262 (0.0010) [2023-12-27 07:36:03,279][20292] KL-divergence is very high: 171.8360 [2023-12-27 07:36:03,288][20381] Updated weights for policy 1, policy_version 281918 (0.0006) [2023-12-27 07:36:03,315][20347] Updated weights for policy 0, policy_version 282272 (0.0010) [2023-12-27 07:36:03,341][20381] Updated weights for policy 1, policy_version 281928 (0.0007) [2023-12-27 07:36:03,945][20347] Updated weights for policy 0, policy_version 282282 (0.0008) [2023-12-27 07:36:04,001][20347] Updated weights for policy 0, policy_version 282292 (0.0010) [2023-12-27 07:36:04,060][20347] Updated weights for policy 0, policy_version 282302 (0.0008) [2023-12-27 07:36:04,073][20381] Updated weights for policy 1, policy_version 281938 (0.0005) [2023-12-27 07:36:04,098][20292] KL-divergence is very high: 546.4177 [2023-12-27 07:36:04,135][20381] Updated weights for policy 1, policy_version 281948 (0.0008) [2023-12-27 07:36:04,149][20292] KL-divergence is very high: 284.7689 [2023-12-27 07:36:04,195][20381] Updated weights for policy 1, policy_version 281958 (0.0007) [2023-12-27 07:36:04,197][20292] KL-divergence is very high: 102.1872 [2023-12-27 07:36:04,773][20347] Updated weights for policy 0, policy_version 282312 (0.0007) [2023-12-27 07:36:04,831][20347] Updated weights for policy 0, policy_version 282322 (0.0005) [2023-12-27 07:36:04,859][20381] Updated weights for policy 1, policy_version 281968 (0.0007) [2023-12-27 07:36:04,876][20347] Updated weights for policy 0, policy_version 282332 (0.0005) [2023-12-27 07:36:04,906][20381] Updated weights for policy 1, policy_version 281978 (0.0007) [2023-12-27 07:36:04,962][20381] Updated weights for policy 1, policy_version 281989 (0.0008) [2023-12-27 07:36:05,548][20347] Updated weights for policy 0, policy_version 282342 (0.0007) [2023-12-27 07:36:05,565][20381] Updated weights for policy 1, policy_version 281999 (0.0005) [2023-12-27 07:36:05,606][20347] Updated weights for policy 0, policy_version 282352 (0.0006) [2023-12-27 07:36:05,618][20381] Updated weights for policy 1, policy_version 282009 (0.0009) [2023-12-27 07:36:05,665][20347] Updated weights for policy 0, policy_version 282362 (0.0007) [2023-12-27 07:36:05,670][20381] Updated weights for policy 1, policy_version 282019 (0.0005) [2023-12-27 07:36:06,234][20381] Updated weights for policy 1, policy_version 282029 (0.0007) [2023-12-27 07:36:06,247][20347] Updated weights for policy 0, policy_version 282372 (0.0007) [2023-12-27 07:36:06,287][20381] Updated weights for policy 1, policy_version 282039 (0.0006) [2023-12-27 07:36:06,304][20347] Updated weights for policy 0, policy_version 282382 (0.0006) [2023-12-27 07:36:06,351][20381] Updated weights for policy 1, policy_version 282049 (0.0006) [2023-12-27 07:36:06,364][20347] Updated weights for policy 0, policy_version 282392 (0.0006) [2023-12-27 07:36:07,021][20347] Updated weights for policy 0, policy_version 282402 (0.0009) [2023-12-27 07:36:07,041][20381] Updated weights for policy 1, policy_version 282059 (0.0006) [2023-12-27 07:36:07,078][20347] Updated weights for policy 0, policy_version 282412 (0.0009) [2023-12-27 07:36:07,091][20292] KL-divergence is very high: 366.5180 [2023-12-27 07:36:07,097][20381] Updated weights for policy 1, policy_version 282069 (0.0006) [2023-12-27 07:36:07,133][20292] KL-divergence is very high: 410.1092 [2023-12-27 07:36:07,143][20347] Updated weights for policy 0, policy_version 282422 (0.0006) [2023-12-27 07:36:07,149][20381] Updated weights for policy 1, policy_version 282079 (0.0007) [2023-12-27 07:36:07,172][20292] KL-divergence is very high: 310.1298 [2023-12-27 07:36:07,210][20347] Updated weights for policy 0, policy_version 282432 (0.0006) [2023-12-27 07:36:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 144547840. Throughput: 0: 10313.5, 1: 10510.3. Samples: 144509700. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:36:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '961.210')] [2023-12-27 07:36:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000282088_72228864.pth... [2023-12-27 07:36:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000282432_72318976.pth... [2023-12-27 07:36:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000280840_71909376.pth [2023-12-27 07:36:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000281216_72007680.pth [2023-12-27 07:36:07,778][20381] Updated weights for policy 1, policy_version 282089 (0.0007) [2023-12-27 07:36:07,803][20347] Updated weights for policy 0, policy_version 282442 (0.0007) [2023-12-27 07:36:07,839][20381] Updated weights for policy 1, policy_version 282099 (0.0009) [2023-12-27 07:36:07,859][20347] Updated weights for policy 0, policy_version 282452 (0.0006) [2023-12-27 07:36:07,905][20381] Updated weights for policy 1, policy_version 282109 (0.0008) [2023-12-27 07:36:07,918][20347] Updated weights for policy 0, policy_version 282462 (0.0007) [2023-12-27 07:36:07,966][20381] Updated weights for policy 1, policy_version 282119 (0.0008) [2023-12-27 07:36:08,567][20347] Updated weights for policy 0, policy_version 282472 (0.0008) [2023-12-27 07:36:08,620][20347] Updated weights for policy 0, policy_version 282482 (0.0008) [2023-12-27 07:36:08,655][20381] Updated weights for policy 1, policy_version 282129 (0.0008) [2023-12-27 07:36:08,669][20347] Updated weights for policy 0, policy_version 282492 (0.0006) [2023-12-27 07:36:08,680][20292] KL-divergence is very high: 264.8771 [2023-12-27 07:36:08,715][20381] Updated weights for policy 1, policy_version 282139 (0.0006) [2023-12-27 07:36:08,734][20292] KL-divergence is very high: 222.8604 [2023-12-27 07:36:08,774][20292] KL-divergence is very high: 210.3999 [2023-12-27 07:36:08,775][20381] Updated weights for policy 1, policy_version 282149 (0.0008) [2023-12-27 07:36:08,781][20292] KL-divergence is very high: 280.6904 [2023-12-27 07:36:09,288][20347] Updated weights for policy 0, policy_version 282502 (0.0007) [2023-12-27 07:36:09,343][20347] Updated weights for policy 0, policy_version 282512 (0.0007) [2023-12-27 07:36:09,409][20347] Updated weights for policy 0, policy_version 282522 (0.0009) [2023-12-27 07:36:09,542][20381] Updated weights for policy 1, policy_version 282159 (0.0008) [2023-12-27 07:36:09,604][20381] Updated weights for policy 1, policy_version 282169 (0.0008) [2023-12-27 07:36:09,665][20381] Updated weights for policy 1, policy_version 282179 (0.0007) [2023-12-27 07:36:10,101][20347] Updated weights for policy 0, policy_version 282532 (0.0008) [2023-12-27 07:36:10,151][20347] Updated weights for policy 0, policy_version 282542 (0.0007) [2023-12-27 07:36:10,203][20347] Updated weights for policy 0, policy_version 282552 (0.0007) [2023-12-27 07:36:10,441][20381] Updated weights for policy 1, policy_version 282189 (0.0007) [2023-12-27 07:36:10,504][20381] Updated weights for policy 1, policy_version 282199 (0.0006) [2023-12-27 07:36:10,582][20381] Updated weights for policy 1, policy_version 282209 (0.0008) [2023-12-27 07:36:10,941][20347] Updated weights for policy 0, policy_version 282562 (0.0007) [2023-12-27 07:36:11,000][20347] Updated weights for policy 0, policy_version 282572 (0.0008) [2023-12-27 07:36:11,064][20347] Updated weights for policy 0, policy_version 282582 (0.0009) [2023-12-27 07:36:11,122][20347] Updated weights for policy 0, policy_version 282592 (0.0008) [2023-12-27 07:36:11,168][20381] Updated weights for policy 1, policy_version 282219 (0.0008) [2023-12-27 07:36:11,231][20381] Updated weights for policy 1, policy_version 282229 (0.0006) [2023-12-27 07:36:11,299][20381] Updated weights for policy 1, policy_version 282239 (0.0008) [2023-12-27 07:36:11,830][20347] Updated weights for policy 0, policy_version 282602 (0.0006) [2023-12-27 07:36:11,887][20347] Updated weights for policy 0, policy_version 282612 (0.0008) [2023-12-27 07:36:11,947][20347] Updated weights for policy 0, policy_version 282622 (0.0010) [2023-12-27 07:36:11,980][20381] Updated weights for policy 1, policy_version 282249 (0.0007) [2023-12-27 07:36:12,034][20381] Updated weights for policy 1, policy_version 282259 (0.0007) [2023-12-27 07:36:12,089][20381] Updated weights for policy 1, policy_version 282269 (0.0007) [2023-12-27 07:36:12,144][20381] Updated weights for policy 1, policy_version 282279 (0.0009) [2023-12-27 07:36:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 144646144. Throughput: 0: 10366.1, 1: 10511.8. Samples: 144635540. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:36:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.620')] [2023-12-27 07:36:12,610][20347] Updated weights for policy 0, policy_version 282632 (0.0011) [2023-12-27 07:36:12,672][20347] Updated weights for policy 0, policy_version 282642 (0.0007) [2023-12-27 07:36:12,734][20347] Updated weights for policy 0, policy_version 282652 (0.0005) [2023-12-27 07:36:12,879][20381] Updated weights for policy 1, policy_version 282289 (0.0006) [2023-12-27 07:36:12,940][20381] Updated weights for policy 1, policy_version 282299 (0.0006) [2023-12-27 07:36:13,002][20381] Updated weights for policy 1, policy_version 282309 (0.0007) [2023-12-27 07:36:13,366][20347] Updated weights for policy 0, policy_version 282662 (0.0009) [2023-12-27 07:36:13,432][20347] Updated weights for policy 0, policy_version 282672 (0.0010) [2023-12-27 07:36:13,486][20347] Updated weights for policy 0, policy_version 282682 (0.0008) [2023-12-27 07:36:13,690][20381] Updated weights for policy 1, policy_version 282319 (0.0010) [2023-12-27 07:36:13,746][20381] Updated weights for policy 1, policy_version 282329 (0.0006) [2023-12-27 07:36:13,807][20381] Updated weights for policy 1, policy_version 282339 (0.0007) [2023-12-27 07:36:14,116][20347] Updated weights for policy 0, policy_version 282692 (0.0010) [2023-12-27 07:36:14,168][20347] Updated weights for policy 0, policy_version 282702 (0.0010) [2023-12-27 07:36:14,221][20347] Updated weights for policy 0, policy_version 282712 (0.0010) [2023-12-27 07:36:14,402][20381] Updated weights for policy 1, policy_version 282349 (0.0008) [2023-12-27 07:36:14,462][20381] Updated weights for policy 1, policy_version 282359 (0.0006) [2023-12-27 07:36:14,509][20381] Updated weights for policy 1, policy_version 282369 (0.0007) [2023-12-27 07:36:14,930][20347] Updated weights for policy 0, policy_version 282722 (0.0010) [2023-12-27 07:36:14,990][20347] Updated weights for policy 0, policy_version 282732 (0.0008) [2023-12-27 07:36:15,046][20347] Updated weights for policy 0, policy_version 282742 (0.0007) [2023-12-27 07:36:15,102][20347] Updated weights for policy 0, policy_version 282752 (0.0007) [2023-12-27 07:36:15,124][20381] Updated weights for policy 1, policy_version 282379 (0.0008) [2023-12-27 07:36:15,185][20381] Updated weights for policy 1, policy_version 282389 (0.0008) [2023-12-27 07:36:15,235][20381] Updated weights for policy 1, policy_version 282399 (0.0006) [2023-12-27 07:36:15,805][20347] Updated weights for policy 0, policy_version 282762 (0.0007) [2023-12-27 07:36:15,861][20347] Updated weights for policy 0, policy_version 282772 (0.0008) [2023-12-27 07:36:15,911][20347] Updated weights for policy 0, policy_version 282782 (0.0007) [2023-12-27 07:36:15,917][20381] Updated weights for policy 1, policy_version 282409 (0.0007) [2023-12-27 07:36:15,971][20381] Updated weights for policy 1, policy_version 282419 (0.0009) [2023-12-27 07:36:16,020][20381] Updated weights for policy 1, policy_version 282429 (0.0009) [2023-12-27 07:36:16,075][20381] Updated weights for policy 1, policy_version 282439 (0.0008) [2023-12-27 07:36:16,572][20347] Updated weights for policy 0, policy_version 282792 (0.0007) [2023-12-27 07:36:16,619][20347] Updated weights for policy 0, policy_version 282802 (0.0009) [2023-12-27 07:36:16,671][20347] Updated weights for policy 0, policy_version 282812 (0.0010) [2023-12-27 07:36:16,780][20381] Updated weights for policy 1, policy_version 282449 (0.0006) [2023-12-27 07:36:16,836][20381] Updated weights for policy 1, policy_version 282459 (0.0005) [2023-12-27 07:36:16,898][20381] Updated weights for policy 1, policy_version 282469 (0.0007) [2023-12-27 07:36:17,357][20347] Updated weights for policy 0, policy_version 282822 (0.0008) [2023-12-27 07:36:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20889.7, 300 sec: 20799.4). Total num frames: 144744448. Throughput: 0: 10384.9, 1: 10491.4. Samples: 144759008. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:36:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.440')] [2023-12-27 07:36:17,418][20347] Updated weights for policy 0, policy_version 282832 (0.0010) [2023-12-27 07:36:17,470][20347] Updated weights for policy 0, policy_version 282842 (0.0008) [2023-12-27 07:36:17,534][20381] Updated weights for policy 1, policy_version 282479 (0.0011) [2023-12-27 07:36:17,589][20381] Updated weights for policy 1, policy_version 282489 (0.0010) [2023-12-27 07:36:17,640][20381] Updated weights for policy 1, policy_version 282499 (0.0010) [2023-12-27 07:36:18,087][20347] Updated weights for policy 0, policy_version 282852 (0.0008) [2023-12-27 07:36:18,136][20347] Updated weights for policy 0, policy_version 282862 (0.0010) [2023-12-27 07:36:18,188][20347] Updated weights for policy 0, policy_version 282872 (0.0010) [2023-12-27 07:36:18,273][20381] Updated weights for policy 1, policy_version 282509 (0.0008) [2023-12-27 07:36:18,330][20381] Updated weights for policy 1, policy_version 282519 (0.0007) [2023-12-27 07:36:18,389][20381] Updated weights for policy 1, policy_version 282529 (0.0011) [2023-12-27 07:36:18,923][20347] Updated weights for policy 0, policy_version 282882 (0.0010) [2023-12-27 07:36:18,981][20347] Updated weights for policy 0, policy_version 282892 (0.0010) [2023-12-27 07:36:19,004][20381] Updated weights for policy 1, policy_version 282539 (0.0011) [2023-12-27 07:36:19,043][20347] Updated weights for policy 0, policy_version 282902 (0.0010) [2023-12-27 07:36:19,069][20381] Updated weights for policy 1, policy_version 282549 (0.0011) [2023-12-27 07:36:19,098][20347] Updated weights for policy 0, policy_version 282912 (0.0010) [2023-12-27 07:36:19,133][20381] Updated weights for policy 1, policy_version 282559 (0.0010) [2023-12-27 07:36:19,767][20381] Updated weights for policy 1, policy_version 282569 (0.0008) [2023-12-27 07:36:19,829][20381] Updated weights for policy 1, policy_version 282579 (0.0010) [2023-12-27 07:36:19,845][20347] Updated weights for policy 0, policy_version 282922 (0.0010) [2023-12-27 07:36:19,893][20381] Updated weights for policy 1, policy_version 282589 (0.0009) [2023-12-27 07:36:19,905][20347] Updated weights for policy 0, policy_version 282932 (0.0008) [2023-12-27 07:36:19,961][20381] Updated weights for policy 1, policy_version 282599 (0.0011) [2023-12-27 07:36:19,973][20347] Updated weights for policy 0, policy_version 282942 (0.0008) [2023-12-27 07:36:20,636][20381] Updated weights for policy 1, policy_version 282609 (0.0010) [2023-12-27 07:36:20,676][20347] Updated weights for policy 0, policy_version 282952 (0.0009) [2023-12-27 07:36:20,696][20381] Updated weights for policy 1, policy_version 282619 (0.0011) [2023-12-27 07:36:20,736][20347] Updated weights for policy 0, policy_version 282962 (0.0010) [2023-12-27 07:36:20,752][20381] Updated weights for policy 1, policy_version 282629 (0.0010) [2023-12-27 07:36:20,794][20347] Updated weights for policy 0, policy_version 282972 (0.0010) [2023-12-27 07:36:21,472][20347] Updated weights for policy 0, policy_version 282982 (0.0010) [2023-12-27 07:36:21,511][20381] Updated weights for policy 1, policy_version 282639 (0.0007) [2023-12-27 07:36:21,524][20347] Updated weights for policy 0, policy_version 282992 (0.0010) [2023-12-27 07:36:21,570][20381] Updated weights for policy 1, policy_version 282649 (0.0007) [2023-12-27 07:36:21,572][20347] Updated weights for policy 0, policy_version 283002 (0.0010) [2023-12-27 07:36:21,629][20381] Updated weights for policy 1, policy_version 282659 (0.0006) [2023-12-27 07:36:22,299][20347] Updated weights for policy 0, policy_version 283012 (0.0009) [2023-12-27 07:36:22,301][20381] Updated weights for policy 1, policy_version 282669 (0.0008) [2023-12-27 07:36:22,355][20347] Updated weights for policy 0, policy_version 283022 (0.0010) [2023-12-27 07:36:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 144842752. Throughput: 0: 10393.7, 1: 10423.6. Samples: 144820824. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-12-27 07:36:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.860')] [2023-12-27 07:36:22,364][20381] Updated weights for policy 1, policy_version 282679 (0.0006) [2023-12-27 07:36:22,416][20347] Updated weights for policy 0, policy_version 283032 (0.0008) [2023-12-27 07:36:22,437][20381] Updated weights for policy 1, policy_version 282689 (0.0006) [2023-12-27 07:36:22,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000283040_72474624.pth... [2023-12-27 07:36:22,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000281792_72155136.pth [2023-12-27 07:36:22,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000282696_72384512.pth... [2023-12-27 07:36:22,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000281480_72073216.pth [2023-12-27 07:36:23,025][20347] Updated weights for policy 0, policy_version 283042 (0.0007) [2023-12-27 07:36:23,092][20347] Updated weights for policy 0, policy_version 283052 (0.0010) [2023-12-27 07:36:23,139][20347] Updated weights for policy 0, policy_version 283062 (0.0009) [2023-12-27 07:36:23,139][20381] Updated weights for policy 1, policy_version 282699 (0.0007) [2023-12-27 07:36:23,186][20347] Updated weights for policy 0, policy_version 283072 (0.0007) [2023-12-27 07:36:23,203][20381] Updated weights for policy 1, policy_version 282709 (0.0006) [2023-12-27 07:36:23,263][20381] Updated weights for policy 1, policy_version 282719 (0.0005) [2023-12-27 07:36:23,792][20381] Updated weights for policy 1, policy_version 282729 (0.0005) [2023-12-27 07:36:23,847][20381] Updated weights for policy 1, policy_version 282739 (0.0009) [2023-12-27 07:36:23,860][20347] Updated weights for policy 0, policy_version 283082 (0.0005) [2023-12-27 07:36:23,902][20381] Updated weights for policy 1, policy_version 282749 (0.0007) [2023-12-27 07:36:23,906][20347] Updated weights for policy 0, policy_version 283092 (0.0005) [2023-12-27 07:36:23,957][20347] Updated weights for policy 0, policy_version 283102 (0.0005) [2023-12-27 07:36:23,959][20381] Updated weights for policy 1, policy_version 282759 (0.0006) [2023-12-27 07:36:24,545][20347] Updated weights for policy 0, policy_version 283112 (0.0006) [2023-12-27 07:36:24,560][20381] Updated weights for policy 1, policy_version 282769 (0.0010) [2023-12-27 07:36:24,601][20347] Updated weights for policy 0, policy_version 283122 (0.0006) [2023-12-27 07:36:24,620][20381] Updated weights for policy 1, policy_version 282779 (0.0011) [2023-12-27 07:36:24,665][20347] Updated weights for policy 0, policy_version 283132 (0.0006) [2023-12-27 07:36:24,677][20381] Updated weights for policy 1, policy_version 282789 (0.0010) [2023-12-27 07:36:25,266][20347] Updated weights for policy 0, policy_version 283142 (0.0007) [2023-12-27 07:36:25,323][20347] Updated weights for policy 0, policy_version 283152 (0.0008) [2023-12-27 07:36:25,374][20381] Updated weights for policy 1, policy_version 282799 (0.0011) [2023-12-27 07:36:25,380][20347] Updated weights for policy 0, policy_version 283162 (0.0005) [2023-12-27 07:36:25,435][20381] Updated weights for policy 1, policy_version 282809 (0.0011) [2023-12-27 07:36:25,485][20381] Updated weights for policy 1, policy_version 282819 (0.0011) [2023-12-27 07:36:26,008][20347] Updated weights for policy 0, policy_version 283172 (0.0006) [2023-12-27 07:36:26,065][20347] Updated weights for policy 0, policy_version 283183 (0.0008) [2023-12-27 07:36:26,119][20347] Updated weights for policy 0, policy_version 283193 (0.0008) [2023-12-27 07:36:26,153][20381] Updated weights for policy 1, policy_version 282829 (0.0008) [2023-12-27 07:36:26,212][20381] Updated weights for policy 1, policy_version 282839 (0.0006) [2023-12-27 07:36:26,280][20381] Updated weights for policy 1, policy_version 282849 (0.0005) [2023-12-27 07:36:26,742][20347] Updated weights for policy 0, policy_version 283203 (0.0008) [2023-12-27 07:36:26,790][20347] Updated weights for policy 0, policy_version 283213 (0.0008) [2023-12-27 07:36:26,806][20381] Updated weights for policy 1, policy_version 282859 (0.0005) [2023-12-27 07:36:26,836][20347] Updated weights for policy 0, policy_version 283223 (0.0008) [2023-12-27 07:36:26,859][20381] Updated weights for policy 1, policy_version 282869 (0.0005) [2023-12-27 07:36:26,914][20381] Updated weights for policy 1, policy_version 282879 (0.0005) [2023-12-27 07:36:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 144957440. Throughput: 0: 10427.4, 1: 10431.7. Samples: 144946924. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '980.780')] [2023-12-27 07:36:27,421][20381] Updated weights for policy 1, policy_version 282889 (0.0006) [2023-12-27 07:36:27,472][20381] Updated weights for policy 1, policy_version 282899 (0.0010) [2023-12-27 07:36:27,516][20381] Updated weights for policy 1, policy_version 282909 (0.0010) [2023-12-27 07:36:27,517][20292] KL-divergence is very high: 241.9916 [2023-12-27 07:36:27,554][20292] KL-divergence is very high: 127.1090 [2023-12-27 07:36:27,563][20381] Updated weights for policy 1, policy_version 282919 (0.0008) [2023-12-27 07:36:27,570][20347] Updated weights for policy 0, policy_version 283233 (0.0007) [2023-12-27 07:36:27,629][20347] Updated weights for policy 0, policy_version 283243 (0.0005) [2023-12-27 07:36:27,678][20347] Updated weights for policy 0, policy_version 283253 (0.0005) [2023-12-27 07:36:27,724][20347] Updated weights for policy 0, policy_version 283263 (0.0006) [2023-12-27 07:36:28,133][20381] Updated weights for policy 1, policy_version 282929 (0.0010) [2023-12-27 07:36:28,174][20381] Updated weights for policy 1, policy_version 282939 (0.0010) [2023-12-27 07:36:28,220][20381] Updated weights for policy 1, policy_version 282949 (0.0008) [2023-12-27 07:36:28,416][20347] Updated weights for policy 0, policy_version 283273 (0.0007) [2023-12-27 07:36:28,468][20347] Updated weights for policy 0, policy_version 283283 (0.0007) [2023-12-27 07:36:28,524][20347] Updated weights for policy 0, policy_version 283293 (0.0009) [2023-12-27 07:36:28,831][20381] Updated weights for policy 1, policy_version 282959 (0.0006) [2023-12-27 07:36:28,886][20381] Updated weights for policy 1, policy_version 282969 (0.0006) [2023-12-27 07:36:28,945][20381] Updated weights for policy 1, policy_version 282979 (0.0005) [2023-12-27 07:36:29,279][20347] Updated weights for policy 0, policy_version 283303 (0.0008) [2023-12-27 07:36:29,343][20347] Updated weights for policy 0, policy_version 283313 (0.0008) [2023-12-27 07:36:29,401][20347] Updated weights for policy 0, policy_version 283323 (0.0009) [2023-12-27 07:36:29,540][20381] Updated weights for policy 1, policy_version 282989 (0.0009) [2023-12-27 07:36:29,586][20381] Updated weights for policy 1, policy_version 282999 (0.0011) [2023-12-27 07:36:29,635][20381] Updated weights for policy 1, policy_version 283009 (0.0010) [2023-12-27 07:36:30,109][20347] Updated weights for policy 0, policy_version 283333 (0.0008) [2023-12-27 07:36:30,163][20347] Updated weights for policy 0, policy_version 283343 (0.0006) [2023-12-27 07:36:30,213][20347] Updated weights for policy 0, policy_version 283353 (0.0006) [2023-12-27 07:36:30,304][20381] Updated weights for policy 1, policy_version 283019 (0.0011) [2023-12-27 07:36:30,366][20381] Updated weights for policy 1, policy_version 283029 (0.0011) [2023-12-27 07:36:30,427][20381] Updated weights for policy 1, policy_version 283039 (0.0011) [2023-12-27 07:36:30,889][20347] Updated weights for policy 0, policy_version 283363 (0.0006) [2023-12-27 07:36:30,946][20347] Updated weights for policy 0, policy_version 283373 (0.0007) [2023-12-27 07:36:30,999][20347] Updated weights for policy 0, policy_version 283383 (0.0007) [2023-12-27 07:36:31,133][20381] Updated weights for policy 1, policy_version 283049 (0.0011) [2023-12-27 07:36:31,189][20381] Updated weights for policy 1, policy_version 283059 (0.0011) [2023-12-27 07:36:31,252][20381] Updated weights for policy 1, policy_version 283069 (0.0010) [2023-12-27 07:36:31,306][20381] Updated weights for policy 1, policy_version 283079 (0.0008) [2023-12-27 07:36:31,670][20347] Updated weights for policy 0, policy_version 283393 (0.0008) [2023-12-27 07:36:31,728][20347] Updated weights for policy 0, policy_version 283403 (0.0009) [2023-12-27 07:36:31,781][20347] Updated weights for policy 0, policy_version 283413 (0.0006) [2023-12-27 07:36:31,838][20347] Updated weights for policy 0, policy_version 283423 (0.0008) [2023-12-27 07:36:31,985][20381] Updated weights for policy 1, policy_version 283089 (0.0006) [2023-12-27 07:36:32,051][20381] Updated weights for policy 1, policy_version 283099 (0.0007) [2023-12-27 07:36:32,101][20381] Updated weights for policy 1, policy_version 283109 (0.0010) [2023-12-27 07:36:32,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 145063936. Throughput: 0: 10440.5, 1: 10493.9. Samples: 145074164. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.890')] [2023-12-27 07:36:32,470][20347] Updated weights for policy 0, policy_version 283433 (0.0007) [2023-12-27 07:36:32,527][20347] Updated weights for policy 0, policy_version 283443 (0.0007) [2023-12-27 07:36:32,588][20347] Updated weights for policy 0, policy_version 283453 (0.0008) [2023-12-27 07:36:32,727][20381] Updated weights for policy 1, policy_version 283119 (0.0007) [2023-12-27 07:36:32,788][20381] Updated weights for policy 1, policy_version 283129 (0.0005) [2023-12-27 07:36:32,840][20381] Updated weights for policy 1, policy_version 283139 (0.0006) [2023-12-27 07:36:33,246][20347] Updated weights for policy 0, policy_version 283463 (0.0010) [2023-12-27 07:36:33,300][20347] Updated weights for policy 0, policy_version 283473 (0.0007) [2023-12-27 07:36:33,353][20347] Updated weights for policy 0, policy_version 283483 (0.0008) [2023-12-27 07:36:33,365][20292] KL-divergence is very high: 246.4915 [2023-12-27 07:36:33,391][20381] Updated weights for policy 1, policy_version 283149 (0.0007) [2023-12-27 07:36:33,441][20381] Updated weights for policy 1, policy_version 283159 (0.0007) [2023-12-27 07:36:33,487][20381] Updated weights for policy 1, policy_version 283169 (0.0006) [2023-12-27 07:36:33,946][20347] Updated weights for policy 0, policy_version 283493 (0.0008) [2023-12-27 07:36:33,992][20347] Updated weights for policy 0, policy_version 283503 (0.0005) [2023-12-27 07:36:34,035][20347] Updated weights for policy 0, policy_version 283513 (0.0005) [2023-12-27 07:36:34,138][20381] Updated weights for policy 1, policy_version 283179 (0.0008) [2023-12-27 07:36:34,139][20292] KL-divergence is very high: 386.3807 [2023-12-27 07:36:34,162][20292] KL-divergence is very high: 143.6278 [2023-12-27 07:36:34,178][20292] KL-divergence is very high: 105.0066 [2023-12-27 07:36:34,184][20292] KL-divergence is very high: 276.2711 [2023-12-27 07:36:34,193][20381] Updated weights for policy 1, policy_version 283189 (0.0008) [2023-12-27 07:36:34,253][20381] Updated weights for policy 1, policy_version 283199 (0.0008) [2023-12-27 07:36:34,299][20292] KL-divergence is very high: 122.1317 [2023-12-27 07:36:34,687][20347] Updated weights for policy 0, policy_version 283523 (0.0008) [2023-12-27 07:36:34,746][20347] Updated weights for policy 0, policy_version 283533 (0.0010) [2023-12-27 07:36:34,801][20347] Updated weights for policy 0, policy_version 283543 (0.0010) [2023-12-27 07:36:34,955][20292] KL-divergence is very high: 141.3450 [2023-12-27 07:36:34,962][20381] Updated weights for policy 1, policy_version 283209 (0.0008) [2023-12-27 07:36:35,000][20292] KL-divergence is very high: 117.0789 [2023-12-27 07:36:35,019][20381] Updated weights for policy 1, policy_version 283219 (0.0009) [2023-12-27 07:36:35,046][20292] KL-divergence is very high: 118.0881 [2023-12-27 07:36:35,075][20381] Updated weights for policy 1, policy_version 283229 (0.0005) [2023-12-27 07:36:35,133][20381] Updated weights for policy 1, policy_version 283239 (0.0010) [2023-12-27 07:36:35,477][20347] Updated weights for policy 0, policy_version 283553 (0.0010) [2023-12-27 07:36:35,521][20347] Updated weights for policy 0, policy_version 283563 (0.0010) [2023-12-27 07:36:35,566][20347] Updated weights for policy 0, policy_version 283573 (0.0010) [2023-12-27 07:36:35,617][20347] Updated weights for policy 0, policy_version 283583 (0.0010) [2023-12-27 07:36:35,809][20381] Updated weights for policy 1, policy_version 283249 (0.0010) [2023-12-27 07:36:35,867][20381] Updated weights for policy 1, policy_version 283259 (0.0010) [2023-12-27 07:36:35,924][20381] Updated weights for policy 1, policy_version 283269 (0.0006) [2023-12-27 07:36:36,344][20347] Updated weights for policy 0, policy_version 283593 (0.0010) [2023-12-27 07:36:36,405][20347] Updated weights for policy 0, policy_version 283603 (0.0010) [2023-12-27 07:36:36,471][20347] Updated weights for policy 0, policy_version 283613 (0.0010) [2023-12-27 07:36:36,582][20381] Updated weights for policy 1, policy_version 283279 (0.0010) [2023-12-27 07:36:36,646][20381] Updated weights for policy 1, policy_version 283289 (0.0011) [2023-12-27 07:36:36,701][20381] Updated weights for policy 1, policy_version 283299 (0.0011) [2023-12-27 07:36:37,107][20347] Updated weights for policy 0, policy_version 283623 (0.0010) [2023-12-27 07:36:37,166][20347] Updated weights for policy 0, policy_version 283633 (0.0010) [2023-12-27 07:36:37,226][20347] Updated weights for policy 0, policy_version 283643 (0.0009) [2023-12-27 07:36:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 21025.9, 300 sec: 20854.8). Total num frames: 145170432. Throughput: 0: 10465.7, 1: 10554.4. Samples: 145139684. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '884.470')] [2023-12-27 07:36:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000283648_72630272.pth... [2023-12-27 07:36:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000282432_72318976.pth [2023-12-27 07:36:37,400][20381] Updated weights for policy 1, policy_version 283309 (0.0011) [2023-12-27 07:36:37,459][20381] Updated weights for policy 1, policy_version 283319 (0.0011) [2023-12-27 07:36:37,511][20381] Updated weights for policy 1, policy_version 283329 (0.0011) [2023-12-27 07:36:37,549][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000283336_72548352.pth... [2023-12-27 07:36:37,553][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000282088_72228864.pth [2023-12-27 07:36:37,902][20347] Updated weights for policy 0, policy_version 283653 (0.0007) [2023-12-27 07:36:37,953][20347] Updated weights for policy 0, policy_version 283663 (0.0007) [2023-12-27 07:36:38,014][20347] Updated weights for policy 0, policy_version 283673 (0.0007) [2023-12-27 07:36:38,211][20381] Updated weights for policy 1, policy_version 283339 (0.0010) [2023-12-27 07:36:38,260][20381] Updated weights for policy 1, policy_version 283349 (0.0010) [2023-12-27 07:36:38,311][20381] Updated weights for policy 1, policy_version 283359 (0.0010) [2023-12-27 07:36:38,711][20347] Updated weights for policy 0, policy_version 283683 (0.0008) [2023-12-27 07:36:38,763][20347] Updated weights for policy 0, policy_version 283693 (0.0010) [2023-12-27 07:36:38,816][20347] Updated weights for policy 0, policy_version 283703 (0.0010) [2023-12-27 07:36:39,017][20381] Updated weights for policy 1, policy_version 283369 (0.0008) [2023-12-27 07:36:39,065][20381] Updated weights for policy 1, policy_version 283379 (0.0010) [2023-12-27 07:36:39,117][20381] Updated weights for policy 1, policy_version 283389 (0.0009) [2023-12-27 07:36:39,129][20292] KL-divergence is very high: 115.4264 [2023-12-27 07:36:39,179][20381] Updated weights for policy 1, policy_version 283399 (0.0010) [2023-12-27 07:36:39,559][20347] Updated weights for policy 0, policy_version 283713 (0.0010) [2023-12-27 07:36:39,618][20347] Updated weights for policy 0, policy_version 283723 (0.0005) [2023-12-27 07:36:39,670][20347] Updated weights for policy 0, policy_version 283733 (0.0005) [2023-12-27 07:36:39,720][20347] Updated weights for policy 0, policy_version 283743 (0.0005) [2023-12-27 07:36:39,897][20381] Updated weights for policy 1, policy_version 283409 (0.0009) [2023-12-27 07:36:39,956][20381] Updated weights for policy 1, policy_version 283419 (0.0008) [2023-12-27 07:36:40,019][20381] Updated weights for policy 1, policy_version 283429 (0.0007) [2023-12-27 07:36:40,358][20347] Updated weights for policy 0, policy_version 283753 (0.0010) [2023-12-27 07:36:40,413][20347] Updated weights for policy 0, policy_version 283763 (0.0010) [2023-12-27 07:36:40,466][20347] Updated weights for policy 0, policy_version 283773 (0.0010) [2023-12-27 07:36:40,638][20381] Updated weights for policy 1, policy_version 283439 (0.0007) [2023-12-27 07:36:40,694][20381] Updated weights for policy 1, policy_version 283449 (0.0008) [2023-12-27 07:36:40,743][20381] Updated weights for policy 1, policy_version 283459 (0.0008) [2023-12-27 07:36:41,057][20347] Updated weights for policy 0, policy_version 283783 (0.0010) [2023-12-27 07:36:41,116][20347] Updated weights for policy 0, policy_version 283793 (0.0010) [2023-12-27 07:36:41,180][20347] Updated weights for policy 0, policy_version 283803 (0.0011) [2023-12-27 07:36:41,474][20381] Updated weights for policy 1, policy_version 283469 (0.0008) [2023-12-27 07:36:41,535][20381] Updated weights for policy 1, policy_version 283479 (0.0007) [2023-12-27 07:36:41,595][20381] Updated weights for policy 1, policy_version 283489 (0.0007) [2023-12-27 07:36:41,863][20347] Updated weights for policy 0, policy_version 283813 (0.0009) [2023-12-27 07:36:41,926][20347] Updated weights for policy 0, policy_version 283823 (0.0006) [2023-12-27 07:36:41,973][20347] Updated weights for policy 0, policy_version 283833 (0.0006) [2023-12-27 07:36:42,286][20381] Updated weights for policy 1, policy_version 283499 (0.0008) [2023-12-27 07:36:42,342][20381] Updated weights for policy 1, policy_version 283509 (0.0008) [2023-12-27 07:36:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 145268736. Throughput: 0: 10462.2, 1: 10496.8. Samples: 145263028. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.810')] [2023-12-27 07:36:42,404][20381] Updated weights for policy 1, policy_version 283519 (0.0008) [2023-12-27 07:36:42,562][20347] Updated weights for policy 0, policy_version 283843 (0.0006) [2023-12-27 07:36:42,621][20347] Updated weights for policy 0, policy_version 283853 (0.0008) [2023-12-27 07:36:42,683][20347] Updated weights for policy 0, policy_version 283863 (0.0006) [2023-12-27 07:36:43,034][20381] Updated weights for policy 1, policy_version 283529 (0.0006) [2023-12-27 07:36:43,095][20381] Updated weights for policy 1, policy_version 283539 (0.0008) [2023-12-27 07:36:43,153][20381] Updated weights for policy 1, policy_version 283549 (0.0008) [2023-12-27 07:36:43,212][20381] Updated weights for policy 1, policy_version 283559 (0.0009) [2023-12-27 07:36:43,277][20347] Updated weights for policy 0, policy_version 283873 (0.0005) [2023-12-27 07:36:43,330][20347] Updated weights for policy 0, policy_version 283883 (0.0005) [2023-12-27 07:36:43,391][20347] Updated weights for policy 0, policy_version 283893 (0.0007) [2023-12-27 07:36:43,451][20347] Updated weights for policy 0, policy_version 283903 (0.0008) [2023-12-27 07:36:43,880][20381] Updated weights for policy 1, policy_version 283569 (0.0011) [2023-12-27 07:36:43,923][20381] Updated weights for policy 1, policy_version 283579 (0.0010) [2023-12-27 07:36:43,970][20381] Updated weights for policy 1, policy_version 283589 (0.0010) [2023-12-27 07:36:44,127][20347] Updated weights for policy 0, policy_version 283913 (0.0007) [2023-12-27 07:36:44,187][20347] Updated weights for policy 0, policy_version 283923 (0.0007) [2023-12-27 07:36:44,234][20347] Updated weights for policy 0, policy_version 283933 (0.0007) [2023-12-27 07:36:44,688][20381] Updated weights for policy 1, policy_version 283599 (0.0010) [2023-12-27 07:36:44,732][20381] Updated weights for policy 1, policy_version 283609 (0.0010) [2023-12-27 07:36:44,794][20381] Updated weights for policy 1, policy_version 283619 (0.0010) [2023-12-27 07:36:44,888][20347] Updated weights for policy 0, policy_version 283943 (0.0006) [2023-12-27 07:36:44,936][20347] Updated weights for policy 0, policy_version 283953 (0.0005) [2023-12-27 07:36:44,988][20347] Updated weights for policy 0, policy_version 283963 (0.0006) [2023-12-27 07:36:45,427][20381] Updated weights for policy 1, policy_version 283629 (0.0007) [2023-12-27 07:36:45,482][20381] Updated weights for policy 1, policy_version 283639 (0.0005) [2023-12-27 07:36:45,531][20381] Updated weights for policy 1, policy_version 283649 (0.0006) [2023-12-27 07:36:45,689][20347] Updated weights for policy 0, policy_version 283973 (0.0007) [2023-12-27 07:36:45,732][20347] Updated weights for policy 0, policy_version 283983 (0.0007) [2023-12-27 07:36:45,782][20347] Updated weights for policy 0, policy_version 283993 (0.0008) [2023-12-27 07:36:46,159][20381] Updated weights for policy 1, policy_version 283659 (0.0010) [2023-12-27 07:36:46,216][20381] Updated weights for policy 1, policy_version 283669 (0.0008) [2023-12-27 07:36:46,268][20381] Updated weights for policy 1, policy_version 283679 (0.0007) [2023-12-27 07:36:46,522][20347] Updated weights for policy 0, policy_version 284003 (0.0007) [2023-12-27 07:36:46,586][20347] Updated weights for policy 0, policy_version 284013 (0.0008) [2023-12-27 07:36:46,643][20347] Updated weights for policy 0, policy_version 284023 (0.0008) [2023-12-27 07:36:46,920][20381] Updated weights for policy 1, policy_version 283689 (0.0008) [2023-12-27 07:36:46,978][20381] Updated weights for policy 1, policy_version 283699 (0.0007) [2023-12-27 07:36:47,041][20381] Updated weights for policy 1, policy_version 283709 (0.0007) [2023-12-27 07:36:47,102][20381] Updated weights for policy 1, policy_version 283719 (0.0008) [2023-12-27 07:36:47,317][20347] Updated weights for policy 0, policy_version 284033 (0.0009) [2023-12-27 07:36:47,362][19170] Fps is (10 sec: 20481.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 145375232. Throughput: 0: 10413.5, 1: 10467.5. Samples: 145387032. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '875.160')] [2023-12-27 07:36:47,385][20347] Updated weights for policy 0, policy_version 284043 (0.0011) [2023-12-27 07:36:47,440][20347] Updated weights for policy 0, policy_version 284053 (0.0010) [2023-12-27 07:36:47,490][20347] Updated weights for policy 0, policy_version 284063 (0.0009) [2023-12-27 07:36:47,780][20381] Updated weights for policy 1, policy_version 283729 (0.0008) [2023-12-27 07:36:47,842][20381] Updated weights for policy 1, policy_version 283739 (0.0008) [2023-12-27 07:36:47,891][20381] Updated weights for policy 1, policy_version 283749 (0.0008) [2023-12-27 07:36:48,102][20347] Updated weights for policy 0, policy_version 284073 (0.0008) [2023-12-27 07:36:48,162][20347] Updated weights for policy 0, policy_version 284083 (0.0006) [2023-12-27 07:36:48,216][20347] Updated weights for policy 0, policy_version 284093 (0.0005) [2023-12-27 07:36:48,581][20381] Updated weights for policy 1, policy_version 283759 (0.0008) [2023-12-27 07:36:48,627][20381] Updated weights for policy 1, policy_version 283769 (0.0009) [2023-12-27 07:36:48,677][20381] Updated weights for policy 1, policy_version 283779 (0.0008) [2023-12-27 07:36:48,800][20347] Updated weights for policy 0, policy_version 284103 (0.0005) [2023-12-27 07:36:48,846][20347] Updated weights for policy 0, policy_version 284113 (0.0006) [2023-12-27 07:36:48,897][20347] Updated weights for policy 0, policy_version 284123 (0.0007) [2023-12-27 07:36:49,320][20381] Updated weights for policy 1, policy_version 283789 (0.0008) [2023-12-27 07:36:49,383][20381] Updated weights for policy 1, policy_version 283799 (0.0008) [2023-12-27 07:36:49,443][20381] Updated weights for policy 1, policy_version 283809 (0.0008) [2023-12-27 07:36:49,641][20347] Updated weights for policy 0, policy_version 284133 (0.0008) [2023-12-27 07:36:49,706][20347] Updated weights for policy 0, policy_version 284143 (0.0008) [2023-12-27 07:36:49,767][20347] Updated weights for policy 0, policy_version 284153 (0.0008) [2023-12-27 07:36:50,091][20381] Updated weights for policy 1, policy_version 283819 (0.0007) [2023-12-27 07:36:50,146][20381] Updated weights for policy 1, policy_version 283829 (0.0007) [2023-12-27 07:36:50,202][20381] Updated weights for policy 1, policy_version 283839 (0.0007) [2023-12-27 07:36:50,463][20347] Updated weights for policy 0, policy_version 284163 (0.0007) [2023-12-27 07:36:50,522][20347] Updated weights for policy 0, policy_version 284173 (0.0005) [2023-12-27 07:36:50,580][20347] Updated weights for policy 0, policy_version 284183 (0.0006) [2023-12-27 07:36:50,974][20381] Updated weights for policy 1, policy_version 283849 (0.0007) [2023-12-27 07:36:51,024][20292] KL-divergence is very high: 129.0065 [2023-12-27 07:36:51,034][20381] Updated weights for policy 1, policy_version 283859 (0.0008) [2023-12-27 07:36:51,036][20292] KL-divergence is very high: 167.6636 [2023-12-27 07:36:51,043][20292] KL-divergence is very high: 102.7546 [2023-12-27 07:36:51,070][20292] KL-divergence is very high: 203.4743 [2023-12-27 07:36:51,102][20381] Updated weights for policy 1, policy_version 283869 (0.0008) [2023-12-27 07:36:51,120][20292] KL-divergence is very high: 120.1021 [2023-12-27 07:36:51,127][20292] KL-divergence is very high: 211.5520 [2023-12-27 07:36:51,140][20292] KL-divergence is very high: 143.9786 [2023-12-27 07:36:51,162][20381] Updated weights for policy 1, policy_version 283879 (0.0008) [2023-12-27 07:36:51,238][20347] Updated weights for policy 0, policy_version 284193 (0.0005) [2023-12-27 07:36:51,304][20347] Updated weights for policy 0, policy_version 284203 (0.0011) [2023-12-27 07:36:51,361][20347] Updated weights for policy 0, policy_version 284213 (0.0010) [2023-12-27 07:36:51,431][20347] Updated weights for policy 0, policy_version 284223 (0.0009) [2023-12-27 07:36:51,859][20292] KL-divergence is very high: 168.1447 [2023-12-27 07:36:51,875][20381] Updated weights for policy 1, policy_version 283889 (0.0008) [2023-12-27 07:36:51,903][20292] KL-divergence is very high: 190.0547 [2023-12-27 07:36:51,910][20292] KL-divergence is very high: 119.3802 [2023-12-27 07:36:51,935][20381] Updated weights for policy 1, policy_version 283899 (0.0008) [2023-12-27 07:36:51,946][20292] KL-divergence is very high: 232.9107 [2023-12-27 07:36:51,951][20292] KL-divergence is very high: 1035.0785 [2023-12-27 07:36:51,959][20292] KL-divergence is very high: 759.7104 [2023-12-27 07:36:51,965][20292] KL-divergence is very high: 436.2061 [2023-12-27 07:36:51,995][20381] Updated weights for policy 1, policy_version 283909 (0.0007) [2023-12-27 07:36:51,997][20292] KL-divergence is very high: 260.7079 [2023-12-27 07:36:52,002][20292] KL-divergence is very high: 239.5779 [2023-12-27 07:36:52,093][20347] Updated weights for policy 0, policy_version 284233 (0.0008) [2023-12-27 07:36:52,150][20347] Updated weights for policy 0, policy_version 284243 (0.0008) [2023-12-27 07:36:52,215][20347] Updated weights for policy 0, policy_version 284253 (0.0008) [2023-12-27 07:36:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 145481728. Throughput: 0: 10397.3, 1: 10486.5. Samples: 145449468. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '553.220')] [2023-12-27 07:36:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000284256_72785920.pth... [2023-12-27 07:36:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000283912_72695808.pth... [2023-12-27 07:36:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000282696_72384512.pth [2023-12-27 07:36:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000283040_72474624.pth [2023-12-27 07:36:52,527][20292] KL-divergence is very high: 229.6967 [2023-12-27 07:36:52,533][20292] KL-divergence is very high: 1320.4869 [2023-12-27 07:36:52,541][20292] KL-divergence is very high: 854.0582 [2023-12-27 07:36:52,547][20292] KL-divergence is very high: 765.2493 [2023-12-27 07:36:52,552][20292] KL-divergence is very high: 160.2593 [2023-12-27 07:36:52,558][20292] KL-divergence is very high: 1156.5614 [2023-12-27 07:36:52,568][20381] Updated weights for policy 1, policy_version 283919 (0.0007) [2023-12-27 07:36:52,574][20292] KL-divergence is very high: 436.7161 [2023-12-27 07:36:52,579][20292] KL-divergence is very high: 2198.5935 [2023-12-27 07:36:52,584][20292] KL-divergence is very high: 1558.6901 [2023-12-27 07:36:52,590][20292] KL-divergence is very high: 1109.9392 [2023-12-27 07:36:52,596][20292] KL-divergence is very high: 221.4259 [2023-12-27 07:36:52,601][20292] KL-divergence is very high: 621.0906 [2023-12-27 07:36:52,607][20292] KL-divergence is very high: 184.7437 [2023-12-27 07:36:52,613][20292] KL-divergence is very high: 101.4479 [2023-12-27 07:36:52,623][20381] Updated weights for policy 1, policy_version 283929 (0.0005) [2023-12-27 07:36:52,624][20292] KL-divergence is very high: 1313.2920 [2023-12-27 07:36:52,629][20292] KL-divergence is very high: 1232.7532 [2023-12-27 07:36:52,634][20292] KL-divergence is very high: 1058.6879 [2023-12-27 07:36:52,640][20292] KL-divergence is very high: 150.2642 [2023-12-27 07:36:52,645][20292] KL-divergence is very high: 797.6001 [2023-12-27 07:36:52,666][20292] KL-divergence is very high: 1230.9775 [2023-12-27 07:36:52,672][20292] KL-divergence is very high: 1143.8916 [2023-12-27 07:36:52,678][20381] Updated weights for policy 1, policy_version 283939 (0.0005) [2023-12-27 07:36:52,679][20292] KL-divergence is very high: 1160.7645 [2023-12-27 07:36:52,685][20292] KL-divergence is very high: 143.5940 [2023-12-27 07:36:52,692][20292] KL-divergence is very high: 1009.0454 [2023-12-27 07:36:52,963][20347] Updated weights for policy 0, policy_version 284263 (0.0007) [2023-12-27 07:36:53,023][20347] Updated weights for policy 0, policy_version 284273 (0.0008) [2023-12-27 07:36:53,076][20347] Updated weights for policy 0, policy_version 284283 (0.0007) [2023-12-27 07:36:53,236][20292] KL-divergence is very high: 202.8642 [2023-12-27 07:36:53,242][20292] KL-divergence is very high: 189.4492 [2023-12-27 07:36:53,248][20292] KL-divergence is very high: 308.6939 [2023-12-27 07:36:53,254][20292] KL-divergence is very high: 165.6866 [2023-12-27 07:36:53,265][20381] Updated weights for policy 1, policy_version 283949 (0.0006) [2023-12-27 07:36:53,279][20292] KL-divergence is very high: 189.4340 [2023-12-27 07:36:53,285][20292] KL-divergence is very high: 174.3604 [2023-12-27 07:36:53,290][20292] KL-divergence is very high: 262.4362 [2023-12-27 07:36:53,295][20292] KL-divergence is very high: 139.5111 [2023-12-27 07:36:53,315][20381] Updated weights for policy 1, policy_version 283959 (0.0007) [2023-12-27 07:36:53,319][20292] KL-divergence is very high: 145.3702 [2023-12-27 07:36:53,326][20292] KL-divergence is very high: 128.4080 [2023-12-27 07:36:53,333][20292] KL-divergence is very high: 207.9704 [2023-12-27 07:36:53,368][20292] KL-divergence is very high: 164.9741 [2023-12-27 07:36:53,375][20292] KL-divergence is very high: 119.9689 [2023-12-27 07:36:53,375][20381] Updated weights for policy 1, policy_version 283969 (0.0007) [2023-12-27 07:36:53,381][20292] KL-divergence is very high: 236.1559 [2023-12-27 07:36:53,738][20347] Updated weights for policy 0, policy_version 284293 (0.0009) [2023-12-27 07:36:53,795][20347] Updated weights for policy 0, policy_version 284303 (0.0010) [2023-12-27 07:36:53,856][20347] Updated weights for policy 0, policy_version 284313 (0.0010) [2023-12-27 07:36:53,997][20381] Updated weights for policy 1, policy_version 283979 (0.0008) [2023-12-27 07:36:54,044][20381] Updated weights for policy 1, policy_version 283989 (0.0006) [2023-12-27 07:36:54,092][20381] Updated weights for policy 1, policy_version 283999 (0.0006) [2023-12-27 07:36:54,447][20347] Updated weights for policy 0, policy_version 284323 (0.0010) [2023-12-27 07:36:54,507][20347] Updated weights for policy 0, policy_version 284333 (0.0007) [2023-12-27 07:36:54,568][20347] Updated weights for policy 0, policy_version 284343 (0.0010) [2023-12-27 07:36:54,830][20381] Updated weights for policy 1, policy_version 284009 (0.0008) [2023-12-27 07:36:54,888][20381] Updated weights for policy 1, policy_version 284019 (0.0008) [2023-12-27 07:36:54,948][20381] Updated weights for policy 1, policy_version 284029 (0.0008) [2023-12-27 07:36:55,008][20381] Updated weights for policy 1, policy_version 284039 (0.0008) [2023-12-27 07:36:55,143][20347] Updated weights for policy 0, policy_version 284353 (0.0010) [2023-12-27 07:36:55,194][20347] Updated weights for policy 0, policy_version 284363 (0.0005) [2023-12-27 07:36:55,240][20347] Updated weights for policy 0, policy_version 284373 (0.0010) [2023-12-27 07:36:55,285][20347] Updated weights for policy 0, policy_version 284383 (0.0010) [2023-12-27 07:36:55,724][20381] Updated weights for policy 1, policy_version 284049 (0.0007) [2023-12-27 07:36:55,782][20381] Updated weights for policy 1, policy_version 284059 (0.0008) [2023-12-27 07:36:55,841][20381] Updated weights for policy 1, policy_version 284069 (0.0007) [2023-12-27 07:36:55,965][20347] Updated weights for policy 0, policy_version 284393 (0.0011) [2023-12-27 07:36:56,024][20347] Updated weights for policy 0, policy_version 284403 (0.0010) [2023-12-27 07:36:56,084][20347] Updated weights for policy 0, policy_version 284413 (0.0010) [2023-12-27 07:36:56,558][20381] Updated weights for policy 1, policy_version 284079 (0.0008) [2023-12-27 07:36:56,624][20381] Updated weights for policy 1, policy_version 284089 (0.0008) [2023-12-27 07:36:56,681][20381] Updated weights for policy 1, policy_version 284099 (0.0008) [2023-12-27 07:36:56,726][20347] Updated weights for policy 0, policy_version 284423 (0.0007) [2023-12-27 07:36:56,776][20347] Updated weights for policy 0, policy_version 284433 (0.0005) [2023-12-27 07:36:56,829][20347] Updated weights for policy 0, policy_version 284443 (0.0006) [2023-12-27 07:36:57,303][20381] Updated weights for policy 1, policy_version 284109 (0.0007) [2023-12-27 07:36:57,356][20381] Updated weights for policy 1, policy_version 284119 (0.0008) [2023-12-27 07:36:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 145580032. Throughput: 0: 10381.2, 1: 10468.4. Samples: 145573768. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:36:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '295.430')] [2023-12-27 07:36:57,411][20381] Updated weights for policy 1, policy_version 284129 (0.0008) [2023-12-27 07:36:57,505][20347] Updated weights for policy 0, policy_version 284453 (0.0006) [2023-12-27 07:36:57,562][20347] Updated weights for policy 0, policy_version 284463 (0.0006) [2023-12-27 07:36:57,616][20347] Updated weights for policy 0, policy_version 284473 (0.0006) [2023-12-27 07:36:58,116][20381] Updated weights for policy 1, policy_version 284139 (0.0007) [2023-12-27 07:36:58,178][20381] Updated weights for policy 1, policy_version 284149 (0.0008) [2023-12-27 07:36:58,220][20347] Updated weights for policy 0, policy_version 284483 (0.0006) [2023-12-27 07:36:58,238][20381] Updated weights for policy 1, policy_version 284159 (0.0008) [2023-12-27 07:36:58,281][20347] Updated weights for policy 0, policy_version 284493 (0.0008) [2023-12-27 07:36:58,339][20347] Updated weights for policy 0, policy_version 284503 (0.0008) [2023-12-27 07:36:58,990][20381] Updated weights for policy 1, policy_version 284169 (0.0008) [2023-12-27 07:36:59,046][20381] Updated weights for policy 1, policy_version 284179 (0.0006) [2023-12-27 07:36:59,078][20347] Updated weights for policy 0, policy_version 284513 (0.0007) [2023-12-27 07:36:59,098][20381] Updated weights for policy 1, policy_version 284189 (0.0006) [2023-12-27 07:36:59,133][20347] Updated weights for policy 0, policy_version 284523 (0.0006) [2023-12-27 07:36:59,146][20381] Updated weights for policy 1, policy_version 284199 (0.0010) [2023-12-27 07:36:59,189][20347] Updated weights for policy 0, policy_version 284533 (0.0007) [2023-12-27 07:36:59,248][20347] Updated weights for policy 0, policy_version 284543 (0.0008) [2023-12-27 07:36:59,806][20381] Updated weights for policy 1, policy_version 284209 (0.0010) [2023-12-27 07:36:59,864][20381] Updated weights for policy 1, policy_version 284219 (0.0008) [2023-12-27 07:36:59,903][20347] Updated weights for policy 0, policy_version 284553 (0.0005) [2023-12-27 07:36:59,922][20381] Updated weights for policy 1, policy_version 284229 (0.0006) [2023-12-27 07:36:59,964][20347] Updated weights for policy 0, policy_version 284563 (0.0007) [2023-12-27 07:37:00,019][20347] Updated weights for policy 0, policy_version 284573 (0.0007) [2023-12-27 07:37:00,632][20381] Updated weights for policy 1, policy_version 284239 (0.0007) [2023-12-27 07:37:00,660][20347] Updated weights for policy 0, policy_version 284583 (0.0005) [2023-12-27 07:37:00,683][20381] Updated weights for policy 1, policy_version 284249 (0.0006) [2023-12-27 07:37:00,704][20347] Updated weights for policy 0, policy_version 284593 (0.0005) [2023-12-27 07:37:00,737][20381] Updated weights for policy 1, policy_version 284259 (0.0007) [2023-12-27 07:37:00,752][20347] Updated weights for policy 0, policy_version 284603 (0.0007) [2023-12-27 07:37:01,392][20381] Updated weights for policy 1, policy_version 284269 (0.0008) [2023-12-27 07:37:01,444][20381] Updated weights for policy 1, policy_version 284279 (0.0008) [2023-12-27 07:37:01,501][20347] Updated weights for policy 0, policy_version 284613 (0.0008) [2023-12-27 07:37:01,509][20381] Updated weights for policy 1, policy_version 284289 (0.0008) [2023-12-27 07:37:01,566][20347] Updated weights for policy 0, policy_version 284623 (0.0007) [2023-12-27 07:37:01,624][20347] Updated weights for policy 0, policy_version 284633 (0.0007) [2023-12-27 07:37:02,160][20381] Updated weights for policy 1, policy_version 284299 (0.0008) [2023-12-27 07:37:02,218][20381] Updated weights for policy 1, policy_version 284309 (0.0006) [2023-12-27 07:37:02,273][20381] Updated weights for policy 1, policy_version 284319 (0.0008) [2023-12-27 07:37:02,323][20347] Updated weights for policy 0, policy_version 284643 (0.0007) [2023-12-27 07:37:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 145686528. Throughput: 0: 10387.9, 1: 10440.3. Samples: 145696280. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '644.690')] [2023-12-27 07:37:02,383][20347] Updated weights for policy 0, policy_version 284653 (0.0008) [2023-12-27 07:37:02,446][20347] Updated weights for policy 0, policy_version 284663 (0.0008) [2023-12-27 07:37:02,837][20381] Updated weights for policy 1, policy_version 284329 (0.0008) [2023-12-27 07:37:02,886][20381] Updated weights for policy 1, policy_version 284339 (0.0005) [2023-12-27 07:37:02,937][20381] Updated weights for policy 1, policy_version 284349 (0.0005) [2023-12-27 07:37:02,985][20381] Updated weights for policy 1, policy_version 284359 (0.0007) [2023-12-27 07:37:03,208][20347] Updated weights for policy 0, policy_version 284673 (0.0008) [2023-12-27 07:37:03,254][20347] Updated weights for policy 0, policy_version 284683 (0.0008) [2023-12-27 07:37:03,299][20347] Updated weights for policy 0, policy_version 284693 (0.0008) [2023-12-27 07:37:03,354][20347] Updated weights for policy 0, policy_version 284703 (0.0006) [2023-12-27 07:37:03,589][20381] Updated weights for policy 1, policy_version 284369 (0.0005) [2023-12-27 07:37:03,636][20381] Updated weights for policy 1, policy_version 284379 (0.0009) [2023-12-27 07:37:03,685][20381] Updated weights for policy 1, policy_version 284389 (0.0009) [2023-12-27 07:37:03,935][20347] Updated weights for policy 0, policy_version 284713 (0.0005) [2023-12-27 07:37:03,989][20347] Updated weights for policy 0, policy_version 284723 (0.0008) [2023-12-27 07:37:04,051][20347] Updated weights for policy 0, policy_version 284733 (0.0008) [2023-12-27 07:37:04,353][20381] Updated weights for policy 1, policy_version 284399 (0.0006) [2023-12-27 07:37:04,409][20381] Updated weights for policy 1, policy_version 284409 (0.0006) [2023-12-27 07:37:04,465][20381] Updated weights for policy 1, policy_version 284419 (0.0008) [2023-12-27 07:37:04,691][20347] Updated weights for policy 0, policy_version 284743 (0.0006) [2023-12-27 07:37:04,738][20347] Updated weights for policy 0, policy_version 284753 (0.0005) [2023-12-27 07:37:04,792][20347] Updated weights for policy 0, policy_version 284763 (0.0005) [2023-12-27 07:37:05,030][20381] Updated weights for policy 1, policy_version 284429 (0.0007) [2023-12-27 07:37:05,085][20381] Updated weights for policy 1, policy_version 284439 (0.0005) [2023-12-27 07:37:05,136][20381] Updated weights for policy 1, policy_version 284449 (0.0005) [2023-12-27 07:37:05,448][20347] Updated weights for policy 0, policy_version 284773 (0.0007) [2023-12-27 07:37:05,493][20347] Updated weights for policy 0, policy_version 284783 (0.0007) [2023-12-27 07:37:05,546][20347] Updated weights for policy 0, policy_version 284793 (0.0008) [2023-12-27 07:37:05,712][20381] Updated weights for policy 1, policy_version 284459 (0.0006) [2023-12-27 07:37:05,770][20381] Updated weights for policy 1, policy_version 284469 (0.0008) [2023-12-27 07:37:05,820][20381] Updated weights for policy 1, policy_version 284479 (0.0007) [2023-12-27 07:37:06,249][20347] Updated weights for policy 0, policy_version 284803 (0.0008) [2023-12-27 07:37:06,308][20347] Updated weights for policy 0, policy_version 284813 (0.0007) [2023-12-27 07:37:06,360][20347] Updated weights for policy 0, policy_version 284823 (0.0007) [2023-12-27 07:37:06,492][20381] Updated weights for policy 1, policy_version 284489 (0.0007) [2023-12-27 07:37:06,552][20381] Updated weights for policy 1, policy_version 284499 (0.0008) [2023-12-27 07:37:06,601][20381] Updated weights for policy 1, policy_version 284509 (0.0007) [2023-12-27 07:37:06,654][20381] Updated weights for policy 1, policy_version 284519 (0.0007) [2023-12-27 07:37:06,998][20347] Updated weights for policy 0, policy_version 284833 (0.0007) [2023-12-27 07:37:07,059][20347] Updated weights for policy 0, policy_version 284843 (0.0007) [2023-12-27 07:37:07,117][20347] Updated weights for policy 0, policy_version 284853 (0.0009) [2023-12-27 07:37:07,178][20347] Updated weights for policy 0, policy_version 284863 (0.0007) [2023-12-27 07:37:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20854.9). Total num frames: 145793024. Throughput: 0: 10429.1, 1: 10505.2. Samples: 145762864. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '779.010')] [2023-12-27 07:37:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000284864_72941568.pth... [2023-12-27 07:37:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000283648_72630272.pth [2023-12-27 07:37:07,398][20381] Updated weights for policy 1, policy_version 284529 (0.0006) [2023-12-27 07:37:07,454][20381] Updated weights for policy 1, policy_version 284539 (0.0006) [2023-12-27 07:37:07,503][20381] Updated weights for policy 1, policy_version 284549 (0.0007) [2023-12-27 07:37:07,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000284552_72859648.pth... [2023-12-27 07:37:07,522][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000283336_72548352.pth [2023-12-27 07:37:07,861][20347] Updated weights for policy 0, policy_version 284873 (0.0007) [2023-12-27 07:37:07,913][20347] Updated weights for policy 0, policy_version 284883 (0.0008) [2023-12-27 07:37:07,961][20347] Updated weights for policy 0, policy_version 284893 (0.0008) [2023-12-27 07:37:08,131][20381] Updated weights for policy 1, policy_version 284559 (0.0006) [2023-12-27 07:37:08,186][20381] Updated weights for policy 1, policy_version 284569 (0.0006) [2023-12-27 07:37:08,239][20381] Updated weights for policy 1, policy_version 284579 (0.0006) [2023-12-27 07:37:08,720][20347] Updated weights for policy 0, policy_version 284903 (0.0007) [2023-12-27 07:37:08,783][20347] Updated weights for policy 0, policy_version 284913 (0.0006) [2023-12-27 07:37:08,785][20381] Updated weights for policy 1, policy_version 284589 (0.0006) [2023-12-27 07:37:08,841][20347] Updated weights for policy 0, policy_version 284923 (0.0006) [2023-12-27 07:37:08,847][20381] Updated weights for policy 1, policy_version 284599 (0.0006) [2023-12-27 07:37:08,904][20381] Updated weights for policy 1, policy_version 284609 (0.0005) [2023-12-27 07:37:09,504][20381] Updated weights for policy 1, policy_version 284619 (0.0008) [2023-12-27 07:37:09,554][20347] Updated weights for policy 0, policy_version 284933 (0.0006) [2023-12-27 07:37:09,564][20381] Updated weights for policy 1, policy_version 284629 (0.0011) [2023-12-27 07:37:09,614][20347] Updated weights for policy 0, policy_version 284943 (0.0009) [2023-12-27 07:37:09,625][20381] Updated weights for policy 1, policy_version 284639 (0.0009) [2023-12-27 07:37:09,679][20347] Updated weights for policy 0, policy_version 284953 (0.0009) [2023-12-27 07:37:10,330][20381] Updated weights for policy 1, policy_version 284649 (0.0009) [2023-12-27 07:37:10,388][20381] Updated weights for policy 1, policy_version 284659 (0.0007) [2023-12-27 07:37:10,395][20347] Updated weights for policy 0, policy_version 284963 (0.0007) [2023-12-27 07:37:10,449][20381] Updated weights for policy 1, policy_version 284669 (0.0007) [2023-12-27 07:37:10,458][20347] Updated weights for policy 0, policy_version 284973 (0.0005) [2023-12-27 07:37:10,511][20381] Updated weights for policy 1, policy_version 284679 (0.0006) [2023-12-27 07:37:10,513][20347] Updated weights for policy 0, policy_version 284983 (0.0005) [2023-12-27 07:37:11,173][20381] Updated weights for policy 1, policy_version 284689 (0.0008) [2023-12-27 07:37:11,177][20347] Updated weights for policy 0, policy_version 284993 (0.0007) [2023-12-27 07:37:11,230][20381] Updated weights for policy 1, policy_version 284699 (0.0006) [2023-12-27 07:37:11,242][20347] Updated weights for policy 0, policy_version 285003 (0.0011) [2023-12-27 07:37:11,291][20381] Updated weights for policy 1, policy_version 284709 (0.0008) [2023-12-27 07:37:11,299][20347] Updated weights for policy 0, policy_version 285013 (0.0009) [2023-12-27 07:37:11,356][20347] Updated weights for policy 0, policy_version 285023 (0.0008) [2023-12-27 07:37:11,968][20381] Updated weights for policy 1, policy_version 284719 (0.0007) [2023-12-27 07:37:12,025][20381] Updated weights for policy 1, policy_version 284729 (0.0006) [2023-12-27 07:37:12,030][20347] Updated weights for policy 0, policy_version 285033 (0.0006) [2023-12-27 07:37:12,074][20381] Updated weights for policy 1, policy_version 284739 (0.0005) [2023-12-27 07:37:12,093][20347] Updated weights for policy 0, policy_version 285043 (0.0007) [2023-12-27 07:37:12,152][20347] Updated weights for policy 0, policy_version 285053 (0.0008) [2023-12-27 07:37:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 145899520. Throughput: 0: 10336.7, 1: 10527.1. Samples: 145885792. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:12,363][19170] Avg episode reward: [(0, '999.710'), (1, '884.700')] [2023-12-27 07:37:12,803][20381] Updated weights for policy 1, policy_version 284749 (0.0008) [2023-12-27 07:37:12,814][20347] Updated weights for policy 0, policy_version 285063 (0.0007) [2023-12-27 07:37:12,821][20291] KL-divergence is very high: 131.6394 [2023-12-27 07:37:12,827][20291] KL-divergence is very high: 228.9316 [2023-12-27 07:37:12,855][20381] Updated weights for policy 1, policy_version 284759 (0.0008) [2023-12-27 07:37:12,872][20347] Updated weights for policy 0, policy_version 285073 (0.0008) [2023-12-27 07:37:12,873][20291] KL-divergence is very high: 200.6090 [2023-12-27 07:37:12,910][20381] Updated weights for policy 1, policy_version 284769 (0.0008) [2023-12-27 07:37:12,939][20347] Updated weights for policy 0, policy_version 285083 (0.0006) [2023-12-27 07:37:13,555][20381] Updated weights for policy 1, policy_version 284779 (0.0007) [2023-12-27 07:37:13,602][20381] Updated weights for policy 1, policy_version 284789 (0.0007) [2023-12-27 07:37:13,624][20347] Updated weights for policy 0, policy_version 285093 (0.0006) [2023-12-27 07:37:13,652][20381] Updated weights for policy 1, policy_version 284799 (0.0006) [2023-12-27 07:37:13,676][20347] Updated weights for policy 0, policy_version 285103 (0.0005) [2023-12-27 07:37:13,726][20347] Updated weights for policy 0, policy_version 285113 (0.0006) [2023-12-27 07:37:14,298][20381] Updated weights for policy 1, policy_version 284809 (0.0007) [2023-12-27 07:37:14,350][20381] Updated weights for policy 1, policy_version 284819 (0.0007) [2023-12-27 07:37:14,381][20347] Updated weights for policy 0, policy_version 285123 (0.0008) [2023-12-27 07:37:14,406][20381] Updated weights for policy 1, policy_version 284829 (0.0006) [2023-12-27 07:37:14,434][20347] Updated weights for policy 0, policy_version 285133 (0.0006) [2023-12-27 07:37:14,457][20381] Updated weights for policy 1, policy_version 284839 (0.0005) [2023-12-27 07:37:14,481][20347] Updated weights for policy 0, policy_version 285143 (0.0008) [2023-12-27 07:37:14,500][20291] KL-divergence is very high: 122.9868 [2023-12-27 07:37:15,120][20381] Updated weights for policy 1, policy_version 284849 (0.0007) [2023-12-27 07:37:15,149][20347] Updated weights for policy 0, policy_version 285153 (0.0009) [2023-12-27 07:37:15,174][20381] Updated weights for policy 1, policy_version 284859 (0.0007) [2023-12-27 07:37:15,205][20347] Updated weights for policy 0, policy_version 285163 (0.0008) [2023-12-27 07:37:15,229][20381] Updated weights for policy 1, policy_version 284869 (0.0006) [2023-12-27 07:37:15,258][20347] Updated weights for policy 0, policy_version 285173 (0.0006) [2023-12-27 07:37:15,317][20347] Updated weights for policy 0, policy_version 285183 (0.0008) [2023-12-27 07:37:15,803][20381] Updated weights for policy 1, policy_version 284879 (0.0006) [2023-12-27 07:37:15,855][20381] Updated weights for policy 1, policy_version 284889 (0.0005) [2023-12-27 07:37:15,915][20381] Updated weights for policy 1, policy_version 284899 (0.0005) [2023-12-27 07:37:16,070][20347] Updated weights for policy 0, policy_version 285193 (0.0008) [2023-12-27 07:37:16,128][20347] Updated weights for policy 0, policy_version 285203 (0.0008) [2023-12-27 07:37:16,186][20347] Updated weights for policy 0, policy_version 285213 (0.0007) [2023-12-27 07:37:16,533][20381] Updated weights for policy 1, policy_version 284909 (0.0007) [2023-12-27 07:37:16,581][20381] Updated weights for policy 1, policy_version 284919 (0.0008) [2023-12-27 07:37:16,655][20381] Updated weights for policy 1, policy_version 284929 (0.0007) [2023-12-27 07:37:16,855][20347] Updated weights for policy 0, policy_version 285223 (0.0009) [2023-12-27 07:37:16,906][20347] Updated weights for policy 0, policy_version 285233 (0.0010) [2023-12-27 07:37:16,959][20347] Updated weights for policy 0, policy_version 285243 (0.0010) [2023-12-27 07:37:17,316][20381] Updated weights for policy 1, policy_version 284939 (0.0007) [2023-12-27 07:37:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 145997824. Throughput: 0: 10336.9, 1: 10448.8. Samples: 146009520. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:17,363][19170] Avg episode reward: [(0, '916.850'), (1, '874.930')] [2023-12-27 07:37:17,371][20381] Updated weights for policy 1, policy_version 284949 (0.0006) [2023-12-27 07:37:17,436][20381] Updated weights for policy 1, policy_version 284959 (0.0008) [2023-12-27 07:37:17,547][20347] Updated weights for policy 0, policy_version 285253 (0.0009) [2023-12-27 07:37:17,596][20347] Updated weights for policy 0, policy_version 285263 (0.0007) [2023-12-27 07:37:17,658][20347] Updated weights for policy 0, policy_version 285273 (0.0010) [2023-12-27 07:37:18,026][20381] Updated weights for policy 1, policy_version 284969 (0.0011) [2023-12-27 07:37:18,091][20381] Updated weights for policy 1, policy_version 284979 (0.0011) [2023-12-27 07:37:18,147][20381] Updated weights for policy 1, policy_version 284989 (0.0010) [2023-12-27 07:37:18,204][20381] Updated weights for policy 1, policy_version 284999 (0.0008) [2023-12-27 07:37:18,335][20347] Updated weights for policy 0, policy_version 285283 (0.0010) [2023-12-27 07:37:18,399][20347] Updated weights for policy 0, policy_version 285293 (0.0007) [2023-12-27 07:37:18,460][20347] Updated weights for policy 0, policy_version 285303 (0.0008) [2023-12-27 07:37:18,845][20381] Updated weights for policy 1, policy_version 285009 (0.0007) [2023-12-27 07:37:18,891][20381] Updated weights for policy 1, policy_version 285019 (0.0008) [2023-12-27 07:37:18,942][20381] Updated weights for policy 1, policy_version 285029 (0.0008) [2023-12-27 07:37:19,129][20347] Updated weights for policy 0, policy_version 285313 (0.0008) [2023-12-27 07:37:19,179][20347] Updated weights for policy 0, policy_version 285323 (0.0007) [2023-12-27 07:37:19,239][20347] Updated weights for policy 0, policy_version 285333 (0.0008) [2023-12-27 07:37:19,305][20347] Updated weights for policy 0, policy_version 285343 (0.0007) [2023-12-27 07:37:19,700][20381] Updated weights for policy 1, policy_version 285039 (0.0007) [2023-12-27 07:37:19,757][20381] Updated weights for policy 1, policy_version 285049 (0.0008) [2023-12-27 07:37:19,817][20381] Updated weights for policy 1, policy_version 285059 (0.0007) [2023-12-27 07:37:20,019][20347] Updated weights for policy 0, policy_version 285353 (0.0007) [2023-12-27 07:37:20,083][20347] Updated weights for policy 0, policy_version 285363 (0.0005) [2023-12-27 07:37:20,145][20347] Updated weights for policy 0, policy_version 285373 (0.0007) [2023-12-27 07:37:20,535][20381] Updated weights for policy 1, policy_version 285069 (0.0007) [2023-12-27 07:37:20,596][20381] Updated weights for policy 1, policy_version 285079 (0.0008) [2023-12-27 07:37:20,654][20381] Updated weights for policy 1, policy_version 285089 (0.0008) [2023-12-27 07:37:20,798][20347] Updated weights for policy 0, policy_version 285383 (0.0006) [2023-12-27 07:37:20,858][20347] Updated weights for policy 0, policy_version 285393 (0.0005) [2023-12-27 07:37:20,916][20347] Updated weights for policy 0, policy_version 285403 (0.0005) [2023-12-27 07:37:21,466][20381] Updated weights for policy 1, policy_version 285099 (0.0009) [2023-12-27 07:37:21,524][20381] Updated weights for policy 1, policy_version 285109 (0.0007) [2023-12-27 07:37:21,527][20347] Updated weights for policy 0, policy_version 285413 (0.0007) [2023-12-27 07:37:21,583][20381] Updated weights for policy 1, policy_version 285119 (0.0006) [2023-12-27 07:37:21,585][20347] Updated weights for policy 0, policy_version 285423 (0.0008) [2023-12-27 07:37:21,646][20347] Updated weights for policy 0, policy_version 285433 (0.0007) [2023-12-27 07:37:22,310][20381] Updated weights for policy 1, policy_version 285129 (0.0008) [2023-12-27 07:37:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 146096128. Throughput: 0: 10304.3, 1: 10411.1. Samples: 146071864. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:22,363][19170] Avg episode reward: [(0, '751.930'), (1, '865.060')] [2023-12-27 07:37:22,367][20347] Updated weights for policy 0, policy_version 285443 (0.0008) [2023-12-27 07:37:22,372][20381] Updated weights for policy 1, policy_version 285139 (0.0008) [2023-12-27 07:37:22,424][20347] Updated weights for policy 0, policy_version 285453 (0.0007) [2023-12-27 07:37:22,430][20381] Updated weights for policy 1, policy_version 285149 (0.0008) [2023-12-27 07:37:22,476][20347] Updated weights for policy 0, policy_version 285463 (0.0006) [2023-12-27 07:37:22,489][20381] Updated weights for policy 1, policy_version 285159 (0.0006) [2023-12-27 07:37:22,496][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000285160_73015296.pth... [2023-12-27 07:37:22,500][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000283912_72695808.pth [2023-12-27 07:37:22,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000285472_73097216.pth... [2023-12-27 07:37:22,528][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000284256_72785920.pth [2023-12-27 07:37:23,160][20381] Updated weights for policy 1, policy_version 285169 (0.0008) [2023-12-27 07:37:23,197][20347] Updated weights for policy 0, policy_version 285473 (0.0008) [2023-12-27 07:37:23,212][20381] Updated weights for policy 1, policy_version 285179 (0.0007) [2023-12-27 07:37:23,259][20347] Updated weights for policy 0, policy_version 285483 (0.0007) [2023-12-27 07:37:23,265][20381] Updated weights for policy 1, policy_version 285189 (0.0007) [2023-12-27 07:37:23,314][20347] Updated weights for policy 0, policy_version 285493 (0.0005) [2023-12-27 07:37:23,374][20347] Updated weights for policy 0, policy_version 285503 (0.0005) [2023-12-27 07:37:23,908][20347] Updated weights for policy 0, policy_version 285513 (0.0010) [2023-12-27 07:37:23,953][20347] Updated weights for policy 0, policy_version 285523 (0.0010) [2023-12-27 07:37:23,972][20381] Updated weights for policy 1, policy_version 285199 (0.0010) [2023-12-27 07:37:24,001][20347] Updated weights for policy 0, policy_version 285533 (0.0010) [2023-12-27 07:37:24,021][20381] Updated weights for policy 1, policy_version 285209 (0.0010) [2023-12-27 07:37:24,073][20381] Updated weights for policy 1, policy_version 285219 (0.0006) [2023-12-27 07:37:24,697][20347] Updated weights for policy 0, policy_version 285543 (0.0010) [2023-12-27 07:37:24,724][20381] Updated weights for policy 1, policy_version 285229 (0.0005) [2023-12-27 07:37:24,751][20347] Updated weights for policy 0, policy_version 285553 (0.0010) [2023-12-27 07:37:24,782][20381] Updated weights for policy 1, policy_version 285239 (0.0007) [2023-12-27 07:37:24,806][20347] Updated weights for policy 0, policy_version 285563 (0.0010) [2023-12-27 07:37:24,836][20381] Updated weights for policy 1, policy_version 285249 (0.0005) [2023-12-27 07:37:25,421][20347] Updated weights for policy 0, policy_version 285573 (0.0010) [2023-12-27 07:37:25,430][20381] Updated weights for policy 1, policy_version 285259 (0.0007) [2023-12-27 07:37:25,482][20347] Updated weights for policy 0, policy_version 285583 (0.0010) [2023-12-27 07:37:25,488][20381] Updated weights for policy 1, policy_version 285269 (0.0006) [2023-12-27 07:37:25,536][20347] Updated weights for policy 0, policy_version 285593 (0.0010) [2023-12-27 07:37:25,544][20381] Updated weights for policy 1, policy_version 285279 (0.0005) [2023-12-27 07:37:26,154][20381] Updated weights for policy 1, policy_version 285289 (0.0005) [2023-12-27 07:37:26,190][20347] Updated weights for policy 0, policy_version 285603 (0.0010) [2023-12-27 07:37:26,208][20381] Updated weights for policy 1, policy_version 285299 (0.0005) [2023-12-27 07:37:26,245][20347] Updated weights for policy 0, policy_version 285613 (0.0010) [2023-12-27 07:37:26,254][20381] Updated weights for policy 1, policy_version 285309 (0.0005) [2023-12-27 07:37:26,292][20347] Updated weights for policy 0, policy_version 285623 (0.0010) [2023-12-27 07:37:26,298][20381] Updated weights for policy 1, policy_version 285319 (0.0005) [2023-12-27 07:37:26,861][20347] Updated weights for policy 0, policy_version 285633 (0.0008) [2023-12-27 07:37:26,895][20381] Updated weights for policy 1, policy_version 285329 (0.0007) [2023-12-27 07:37:26,920][20347] Updated weights for policy 0, policy_version 285643 (0.0006) [2023-12-27 07:37:26,953][20381] Updated weights for policy 1, policy_version 285339 (0.0006) [2023-12-27 07:37:26,984][20347] Updated weights for policy 0, policy_version 285653 (0.0007) [2023-12-27 07:37:27,019][20381] Updated weights for policy 1, policy_version 285349 (0.0010) [2023-12-27 07:37:27,041][20347] Updated weights for policy 0, policy_version 285663 (0.0008) [2023-12-27 07:37:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 146210816. Throughput: 0: 10326.2, 1: 10424.1. Samples: 146196792. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:27,364][19170] Avg episode reward: [(0, '642.400'), (1, '827.030')] [2023-12-27 07:37:27,624][20381] Updated weights for policy 1, policy_version 285359 (0.0008) [2023-12-27 07:37:27,674][20381] Updated weights for policy 1, policy_version 285369 (0.0007) [2023-12-27 07:37:27,711][20347] Updated weights for policy 0, policy_version 285673 (0.0007) [2023-12-27 07:37:27,728][20381] Updated weights for policy 1, policy_version 285379 (0.0008) [2023-12-27 07:37:27,765][20347] Updated weights for policy 0, policy_version 285683 (0.0007) [2023-12-27 07:37:27,824][20347] Updated weights for policy 0, policy_version 285693 (0.0008) [2023-12-27 07:37:28,427][20381] Updated weights for policy 1, policy_version 285389 (0.0006) [2023-12-27 07:37:28,459][20347] Updated weights for policy 0, policy_version 285703 (0.0008) [2023-12-27 07:37:28,478][20381] Updated weights for policy 1, policy_version 285399 (0.0006) [2023-12-27 07:37:28,509][20347] Updated weights for policy 0, policy_version 285713 (0.0008) [2023-12-27 07:37:28,527][20381] Updated weights for policy 1, policy_version 285409 (0.0006) [2023-12-27 07:37:28,567][20347] Updated weights for policy 0, policy_version 285723 (0.0008) [2023-12-27 07:37:29,231][20381] Updated weights for policy 1, policy_version 285419 (0.0007) [2023-12-27 07:37:29,294][20381] Updated weights for policy 1, policy_version 285429 (0.0007) [2023-12-27 07:37:29,309][20347] Updated weights for policy 0, policy_version 285733 (0.0008) [2023-12-27 07:37:29,355][20381] Updated weights for policy 1, policy_version 285439 (0.0008) [2023-12-27 07:37:29,374][20347] Updated weights for policy 0, policy_version 285743 (0.0008) [2023-12-27 07:37:29,426][20347] Updated weights for policy 0, policy_version 285753 (0.0007) [2023-12-27 07:37:30,014][20381] Updated weights for policy 1, policy_version 285449 (0.0007) [2023-12-27 07:37:30,069][20381] Updated weights for policy 1, policy_version 285459 (0.0006) [2023-12-27 07:37:30,079][20347] Updated weights for policy 0, policy_version 285763 (0.0008) [2023-12-27 07:37:30,116][20381] Updated weights for policy 1, policy_version 285469 (0.0006) [2023-12-27 07:37:30,129][20347] Updated weights for policy 0, policy_version 285773 (0.0006) [2023-12-27 07:37:30,171][20381] Updated weights for policy 1, policy_version 285479 (0.0010) [2023-12-27 07:37:30,177][20347] Updated weights for policy 0, policy_version 285783 (0.0005) [2023-12-27 07:37:30,816][20381] Updated weights for policy 1, policy_version 285489 (0.0010) [2023-12-27 07:37:30,866][20381] Updated weights for policy 1, policy_version 285499 (0.0010) [2023-12-27 07:37:30,875][20347] Updated weights for policy 0, policy_version 285793 (0.0007) [2023-12-27 07:37:30,917][20381] Updated weights for policy 1, policy_version 285509 (0.0010) [2023-12-27 07:37:30,923][20347] Updated weights for policy 0, policy_version 285803 (0.0005) [2023-12-27 07:37:30,976][20347] Updated weights for policy 0, policy_version 285813 (0.0005) [2023-12-27 07:37:31,027][20347] Updated weights for policy 0, policy_version 285823 (0.0006) [2023-12-27 07:37:31,671][20347] Updated weights for policy 0, policy_version 285833 (0.0008) [2023-12-27 07:37:31,675][20381] Updated weights for policy 1, policy_version 285519 (0.0008) [2023-12-27 07:37:31,733][20381] Updated weights for policy 1, policy_version 285529 (0.0007) [2023-12-27 07:37:31,736][20347] Updated weights for policy 0, policy_version 285843 (0.0007) [2023-12-27 07:37:31,789][20347] Updated weights for policy 0, policy_version 285853 (0.0006) [2023-12-27 07:37:31,794][20381] Updated weights for policy 1, policy_version 285539 (0.0007) [2023-12-27 07:37:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 146309120. Throughput: 0: 10341.8, 1: 10439.7. Samples: 146322204. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 07:37:32,363][19170] Avg episode reward: [(0, '664.090'), (1, '865.480')] [2023-12-27 07:37:32,443][20381] Updated weights for policy 1, policy_version 285549 (0.0006) [2023-12-27 07:37:32,496][20347] Updated weights for policy 0, policy_version 285863 (0.0007) [2023-12-27 07:37:32,498][20381] Updated weights for policy 1, policy_version 285559 (0.0005) [2023-12-27 07:37:32,552][20347] Updated weights for policy 0, policy_version 285873 (0.0005) [2023-12-27 07:37:32,554][20381] Updated weights for policy 1, policy_version 285569 (0.0006) [2023-12-27 07:37:32,609][20347] Updated weights for policy 0, policy_version 285883 (0.0006) [2023-12-27 07:37:33,161][20381] Updated weights for policy 1, policy_version 285579 (0.0006) [2023-12-27 07:37:33,216][20381] Updated weights for policy 1, policy_version 285589 (0.0007) [2023-12-27 07:37:33,259][20347] Updated weights for policy 0, policy_version 285893 (0.0007) [2023-12-27 07:37:33,273][20381] Updated weights for policy 1, policy_version 285599 (0.0006) [2023-12-27 07:37:33,313][20347] Updated weights for policy 0, policy_version 285903 (0.0006) [2023-12-27 07:37:33,370][20347] Updated weights for policy 0, policy_version 285913 (0.0007) [2023-12-27 07:37:33,894][20381] Updated weights for policy 1, policy_version 285609 (0.0007) [2023-12-27 07:37:33,947][20381] Updated weights for policy 1, policy_version 285619 (0.0007) [2023-12-27 07:37:34,007][20381] Updated weights for policy 1, policy_version 285629 (0.0007) [2023-12-27 07:37:34,029][20347] Updated weights for policy 0, policy_version 285923 (0.0008) [2023-12-27 07:37:34,060][20381] Updated weights for policy 1, policy_version 285639 (0.0006) [2023-12-27 07:37:34,077][20347] Updated weights for policy 0, policy_version 285933 (0.0010) [2023-12-27 07:37:34,130][20347] Updated weights for policy 0, policy_version 285943 (0.0010) [2023-12-27 07:37:34,747][20381] Updated weights for policy 1, policy_version 285649 (0.0007) [2023-12-27 07:37:34,799][20381] Updated weights for policy 1, policy_version 285659 (0.0007) [2023-12-27 07:37:34,846][20347] Updated weights for policy 0, policy_version 285953 (0.0010) [2023-12-27 07:37:34,851][20381] Updated weights for policy 1, policy_version 285669 (0.0006) [2023-12-27 07:37:34,891][20347] Updated weights for policy 0, policy_version 285963 (0.0010) [2023-12-27 07:37:34,943][20347] Updated weights for policy 0, policy_version 285973 (0.0008) [2023-12-27 07:37:34,989][20347] Updated weights for policy 0, policy_version 285983 (0.0005) [2023-12-27 07:37:35,572][20381] Updated weights for policy 1, policy_version 285679 (0.0009) [2023-12-27 07:37:35,596][20347] Updated weights for policy 0, policy_version 285993 (0.0006) [2023-12-27 07:37:35,627][20381] Updated weights for policy 1, policy_version 285689 (0.0010) [2023-12-27 07:37:35,653][20347] Updated weights for policy 0, policy_version 286003 (0.0009) [2023-12-27 07:37:35,675][20381] Updated weights for policy 1, policy_version 285699 (0.0010) [2023-12-27 07:37:35,721][20347] Updated weights for policy 0, policy_version 286013 (0.0007) [2023-12-27 07:37:36,350][20347] Updated weights for policy 0, policy_version 286023 (0.0007) [2023-12-27 07:37:36,357][20381] Updated weights for policy 1, policy_version 285709 (0.0010) [2023-12-27 07:37:36,410][20347] Updated weights for policy 0, policy_version 286033 (0.0006) [2023-12-27 07:37:36,410][20381] Updated weights for policy 1, policy_version 285719 (0.0011) [2023-12-27 07:37:36,463][20381] Updated weights for policy 1, policy_version 285729 (0.0010) [2023-12-27 07:37:36,464][20347] Updated weights for policy 0, policy_version 286043 (0.0008) [2023-12-27 07:37:37,158][20381] Updated weights for policy 1, policy_version 285739 (0.0011) [2023-12-27 07:37:37,160][20347] Updated weights for policy 0, policy_version 286053 (0.0006) [2023-12-27 07:37:37,207][20381] Updated weights for policy 1, policy_version 285749 (0.0010) [2023-12-27 07:37:37,209][20347] Updated weights for policy 0, policy_version 286063 (0.0005) [2023-12-27 07:37:37,254][20347] Updated weights for policy 0, policy_version 286073 (0.0005) [2023-12-27 07:37:37,266][20381] Updated weights for policy 1, policy_version 285759 (0.0011) [2023-12-27 07:37:37,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.8, 300 sec: 20882.7). Total num frames: 146423808. Throughput: 0: 10334.6, 1: 10449.3. Samples: 146384740. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:37:37,363][19170] Avg episode reward: [(0, '848.290'), (1, '915.880')] [2023-12-27 07:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000285768_73170944.pth... [2023-12-27 07:37:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000286080_73252864.pth... [2023-12-27 07:37:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000284864_72941568.pth [2023-12-27 07:37:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000284552_72859648.pth [2023-12-27 07:37:37,965][20347] Updated weights for policy 0, policy_version 286083 (0.0007) [2023-12-27 07:37:37,966][20381] Updated weights for policy 1, policy_version 285769 (0.0010) [2023-12-27 07:37:38,017][20347] Updated weights for policy 0, policy_version 286093 (0.0005) [2023-12-27 07:37:38,020][20381] Updated weights for policy 1, policy_version 285779 (0.0011) [2023-12-27 07:37:38,070][20347] Updated weights for policy 0, policy_version 286103 (0.0005) [2023-12-27 07:37:38,075][20381] Updated weights for policy 1, policy_version 285789 (0.0010) [2023-12-27 07:37:38,128][20381] Updated weights for policy 1, policy_version 285799 (0.0010) [2023-12-27 07:37:38,658][20347] Updated weights for policy 0, policy_version 286113 (0.0005) [2023-12-27 07:37:38,711][20347] Updated weights for policy 0, policy_version 286123 (0.0005) [2023-12-27 07:37:38,736][20381] Updated weights for policy 1, policy_version 285809 (0.0006) [2023-12-27 07:37:38,766][20347] Updated weights for policy 0, policy_version 286133 (0.0005) [2023-12-27 07:37:38,796][20381] Updated weights for policy 1, policy_version 285819 (0.0006) [2023-12-27 07:37:38,819][20347] Updated weights for policy 0, policy_version 286143 (0.0005) [2023-12-27 07:37:38,843][20381] Updated weights for policy 1, policy_version 285829 (0.0005) [2023-12-27 07:37:39,387][20347] Updated weights for policy 0, policy_version 286153 (0.0006) [2023-12-27 07:37:39,409][20381] Updated weights for policy 1, policy_version 285839 (0.0006) [2023-12-27 07:37:39,447][20347] Updated weights for policy 0, policy_version 286163 (0.0007) [2023-12-27 07:37:39,469][20381] Updated weights for policy 1, policy_version 285849 (0.0007) [2023-12-27 07:37:39,503][20347] Updated weights for policy 0, policy_version 286173 (0.0007) [2023-12-27 07:37:39,530][20381] Updated weights for policy 1, policy_version 285859 (0.0008) [2023-12-27 07:37:40,199][20347] Updated weights for policy 0, policy_version 286183 (0.0009) [2023-12-27 07:37:40,265][20347] Updated weights for policy 0, policy_version 286193 (0.0011) [2023-12-27 07:37:40,274][20381] Updated weights for policy 1, policy_version 285869 (0.0007) [2023-12-27 07:37:40,320][20347] Updated weights for policy 0, policy_version 286203 (0.0009) [2023-12-27 07:37:40,328][20381] Updated weights for policy 1, policy_version 285879 (0.0006) [2023-12-27 07:37:40,384][20381] Updated weights for policy 1, policy_version 285889 (0.0008) [2023-12-27 07:37:41,016][20347] Updated weights for policy 0, policy_version 286213 (0.0007) [2023-12-27 07:37:41,079][20347] Updated weights for policy 0, policy_version 286223 (0.0009) [2023-12-27 07:37:41,100][20381] Updated weights for policy 1, policy_version 285899 (0.0007) [2023-12-27 07:37:41,132][20347] Updated weights for policy 0, policy_version 286233 (0.0007) [2023-12-27 07:37:41,168][20381] Updated weights for policy 1, policy_version 285909 (0.0008) [2023-12-27 07:37:41,234][20381] Updated weights for policy 1, policy_version 285919 (0.0008) [2023-12-27 07:37:41,828][20347] Updated weights for policy 0, policy_version 286243 (0.0008) [2023-12-27 07:37:41,884][20347] Updated weights for policy 0, policy_version 286253 (0.0008) [2023-12-27 07:37:41,941][20347] Updated weights for policy 0, policy_version 286263 (0.0007) [2023-12-27 07:37:41,977][20381] Updated weights for policy 1, policy_version 285929 (0.0008) [2023-12-27 07:37:42,038][20381] Updated weights for policy 1, policy_version 285939 (0.0006) [2023-12-27 07:37:42,088][20381] Updated weights for policy 1, policy_version 285949 (0.0005) [2023-12-27 07:37:42,149][20381] Updated weights for policy 1, policy_version 285959 (0.0006) [2023-12-27 07:37:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 146522112. Throughput: 0: 10362.2, 1: 10451.9. Samples: 146510404. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:37:42,363][19170] Avg episode reward: [(0, '905.040'), (1, '925.480')] [2023-12-27 07:37:42,645][20347] Updated weights for policy 0, policy_version 286273 (0.0008) [2023-12-27 07:37:42,703][20347] Updated weights for policy 0, policy_version 286283 (0.0010) [2023-12-27 07:37:42,760][20347] Updated weights for policy 0, policy_version 286293 (0.0010) [2023-12-27 07:37:42,809][20381] Updated weights for policy 1, policy_version 285969 (0.0007) [2023-12-27 07:37:42,814][20347] Updated weights for policy 0, policy_version 286303 (0.0006) [2023-12-27 07:37:42,863][20381] Updated weights for policy 1, policy_version 285979 (0.0006) [2023-12-27 07:37:42,918][20381] Updated weights for policy 1, policy_version 285989 (0.0005) [2023-12-27 07:37:43,469][20381] Updated weights for policy 1, policy_version 285999 (0.0007) [2023-12-27 07:37:43,527][20381] Updated weights for policy 1, policy_version 286009 (0.0008) [2023-12-27 07:37:43,548][20347] Updated weights for policy 0, policy_version 286313 (0.0005) [2023-12-27 07:37:43,585][20381] Updated weights for policy 1, policy_version 286019 (0.0006) [2023-12-27 07:37:43,596][20347] Updated weights for policy 0, policy_version 286323 (0.0007) [2023-12-27 07:37:43,652][20347] Updated weights for policy 0, policy_version 286333 (0.0008) [2023-12-27 07:37:44,127][20381] Updated weights for policy 1, policy_version 286029 (0.0007) [2023-12-27 07:37:44,179][20381] Updated weights for policy 1, policy_version 286039 (0.0007) [2023-12-27 07:37:44,231][20381] Updated weights for policy 1, policy_version 286049 (0.0005) [2023-12-27 07:37:44,306][20347] Updated weights for policy 0, policy_version 286343 (0.0009) [2023-12-27 07:37:44,351][20347] Updated weights for policy 0, policy_version 286353 (0.0010) [2023-12-27 07:37:44,407][20347] Updated weights for policy 0, policy_version 286363 (0.0010) [2023-12-27 07:37:44,842][20381] Updated weights for policy 1, policy_version 286059 (0.0006) [2023-12-27 07:37:44,904][20381] Updated weights for policy 1, policy_version 286069 (0.0007) [2023-12-27 07:37:44,961][20381] Updated weights for policy 1, policy_version 286079 (0.0006) [2023-12-27 07:37:45,132][20347] Updated weights for policy 0, policy_version 286373 (0.0010) [2023-12-27 07:37:45,185][20347] Updated weights for policy 0, policy_version 286383 (0.0010) [2023-12-27 07:37:45,238][20347] Updated weights for policy 0, policy_version 286393 (0.0010) [2023-12-27 07:37:45,643][20381] Updated weights for policy 1, policy_version 286089 (0.0005) [2023-12-27 07:37:45,697][20381] Updated weights for policy 1, policy_version 286099 (0.0007) [2023-12-27 07:37:45,745][20381] Updated weights for policy 1, policy_version 286109 (0.0007) [2023-12-27 07:37:45,802][20381] Updated weights for policy 1, policy_version 286119 (0.0007) [2023-12-27 07:37:45,943][20347] Updated weights for policy 0, policy_version 286403 (0.0010) [2023-12-27 07:37:45,997][20347] Updated weights for policy 0, policy_version 286413 (0.0010) [2023-12-27 07:37:46,045][20347] Updated weights for policy 0, policy_version 286423 (0.0010) [2023-12-27 07:37:46,508][20381] Updated weights for policy 1, policy_version 286129 (0.0008) [2023-12-27 07:37:46,560][20381] Updated weights for policy 1, policy_version 286139 (0.0007) [2023-12-27 07:37:46,619][20381] Updated weights for policy 1, policy_version 286149 (0.0008) [2023-12-27 07:37:46,667][20347] Updated weights for policy 0, policy_version 286433 (0.0010) [2023-12-27 07:37:46,690][20291] KL-divergence is very high: 125.2170 [2023-12-27 07:37:46,718][20347] Updated weights for policy 0, policy_version 286443 (0.0005) [2023-12-27 07:37:46,732][20291] KL-divergence is very high: 121.3269 [2023-12-27 07:37:46,774][20347] Updated weights for policy 0, policy_version 286453 (0.0007) [2023-12-27 07:37:46,816][20291] KL-divergence is very high: 140.8983 [2023-12-27 07:37:46,828][20347] Updated weights for policy 0, policy_version 286463 (0.0010) [2023-12-27 07:37:47,270][20381] Updated weights for policy 1, policy_version 286159 (0.0007) [2023-12-27 07:37:47,328][20381] Updated weights for policy 1, policy_version 286169 (0.0008) [2023-12-27 07:37:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 146620416. Throughput: 0: 10336.4, 1: 10539.4. Samples: 146635688. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:37:47,363][19170] Avg episode reward: [(0, '837.820'), (1, '923.580')] [2023-12-27 07:37:47,379][20381] Updated weights for policy 1, policy_version 286179 (0.0008) [2023-12-27 07:37:47,456][20347] Updated weights for policy 0, policy_version 286473 (0.0008) [2023-12-27 07:37:47,513][20347] Updated weights for policy 0, policy_version 286483 (0.0007) [2023-12-27 07:37:47,562][20347] Updated weights for policy 0, policy_version 286493 (0.0007) [2023-12-27 07:37:48,123][20381] Updated weights for policy 1, policy_version 286189 (0.0007) [2023-12-27 07:37:48,171][20347] Updated weights for policy 0, policy_version 286503 (0.0007) [2023-12-27 07:37:48,180][20381] Updated weights for policy 1, policy_version 286199 (0.0006) [2023-12-27 07:37:48,230][20347] Updated weights for policy 0, policy_version 286513 (0.0006) [2023-12-27 07:37:48,236][20381] Updated weights for policy 1, policy_version 286209 (0.0005) [2023-12-27 07:37:48,279][20347] Updated weights for policy 0, policy_version 286523 (0.0005) [2023-12-27 07:37:48,931][20381] Updated weights for policy 1, policy_version 286219 (0.0007) [2023-12-27 07:37:48,983][20381] Updated weights for policy 1, policy_version 286229 (0.0010) [2023-12-27 07:37:48,991][20347] Updated weights for policy 0, policy_version 286533 (0.0006) [2023-12-27 07:37:49,032][20381] Updated weights for policy 1, policy_version 286239 (0.0010) [2023-12-27 07:37:49,039][20347] Updated weights for policy 0, policy_version 286543 (0.0005) [2023-12-27 07:37:49,090][20347] Updated weights for policy 0, policy_version 286553 (0.0005) [2023-12-27 07:37:49,741][20381] Updated weights for policy 1, policy_version 286249 (0.0010) [2023-12-27 07:37:49,755][20347] Updated weights for policy 0, policy_version 286563 (0.0007) [2023-12-27 07:37:49,794][20381] Updated weights for policy 1, policy_version 286259 (0.0005) [2023-12-27 07:37:49,807][20347] Updated weights for policy 0, policy_version 286573 (0.0010) [2023-12-27 07:37:49,852][20381] Updated weights for policy 1, policy_version 286269 (0.0006) [2023-12-27 07:37:49,869][20347] Updated weights for policy 0, policy_version 286583 (0.0010) [2023-12-27 07:37:49,907][20381] Updated weights for policy 1, policy_version 286279 (0.0006) [2023-12-27 07:37:50,567][20347] Updated weights for policy 0, policy_version 286593 (0.0009) [2023-12-27 07:37:50,574][20381] Updated weights for policy 1, policy_version 286289 (0.0008) [2023-12-27 07:37:50,629][20347] Updated weights for policy 0, policy_version 286603 (0.0007) [2023-12-27 07:37:50,633][20381] Updated weights for policy 1, policy_version 286299 (0.0007) [2023-12-27 07:37:50,685][20347] Updated weights for policy 0, policy_version 286613 (0.0006) [2023-12-27 07:37:50,686][20381] Updated weights for policy 1, policy_version 286309 (0.0007) [2023-12-27 07:37:50,749][20347] Updated weights for policy 0, policy_version 286623 (0.0006) [2023-12-27 07:37:51,338][20381] Updated weights for policy 1, policy_version 286319 (0.0007) [2023-12-27 07:37:51,404][20381] Updated weights for policy 1, policy_version 286329 (0.0008) [2023-12-27 07:37:51,462][20347] Updated weights for policy 0, policy_version 286633 (0.0007) [2023-12-27 07:37:51,464][20381] Updated weights for policy 1, policy_version 286339 (0.0006) [2023-12-27 07:37:51,517][20347] Updated weights for policy 0, policy_version 286643 (0.0007) [2023-12-27 07:37:51,572][20347] Updated weights for policy 0, policy_version 286653 (0.0007) [2023-12-27 07:37:52,119][20381] Updated weights for policy 1, policy_version 286349 (0.0006) [2023-12-27 07:37:52,169][20381] Updated weights for policy 1, policy_version 286359 (0.0005) [2023-12-27 07:37:52,198][20347] Updated weights for policy 0, policy_version 286663 (0.0007) [2023-12-27 07:37:52,225][20381] Updated weights for policy 1, policy_version 286369 (0.0005) [2023-12-27 07:37:52,257][20347] Updated weights for policy 0, policy_version 286673 (0.0008) [2023-12-27 07:37:52,321][20347] Updated weights for policy 0, policy_version 286683 (0.0007) [2023-12-27 07:37:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 146735104. Throughput: 0: 10327.9, 1: 10420.3. Samples: 146696532. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:37:52,363][19170] Avg episode reward: [(0, '857.500'), (1, '933.130')] [2023-12-27 07:37:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000286376_73326592.pth... [2023-12-27 07:37:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000286688_73408512.pth... [2023-12-27 07:37:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000285160_73015296.pth [2023-12-27 07:37:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000285472_73097216.pth [2023-12-27 07:37:52,784][20381] Updated weights for policy 1, policy_version 286379 (0.0007) [2023-12-27 07:37:52,841][20381] Updated weights for policy 1, policy_version 286389 (0.0006) [2023-12-27 07:37:52,891][20381] Updated weights for policy 1, policy_version 286399 (0.0008) [2023-12-27 07:37:53,065][20347] Updated weights for policy 0, policy_version 286693 (0.0008) [2023-12-27 07:37:53,114][20347] Updated weights for policy 0, policy_version 286703 (0.0009) [2023-12-27 07:37:53,163][20347] Updated weights for policy 0, policy_version 286713 (0.0009) [2023-12-27 07:37:53,524][20381] Updated weights for policy 1, policy_version 286409 (0.0008) [2023-12-27 07:37:53,571][20381] Updated weights for policy 1, policy_version 286419 (0.0005) [2023-12-27 07:37:53,627][20381] Updated weights for policy 1, policy_version 286429 (0.0005) [2023-12-27 07:37:53,690][20381] Updated weights for policy 1, policy_version 286439 (0.0007) [2023-12-27 07:37:53,874][20347] Updated weights for policy 0, policy_version 286723 (0.0007) [2023-12-27 07:37:53,935][20347] Updated weights for policy 0, policy_version 286733 (0.0008) [2023-12-27 07:37:53,986][20347] Updated weights for policy 0, policy_version 286743 (0.0008) [2023-12-27 07:37:54,336][20381] Updated weights for policy 1, policy_version 286449 (0.0008) [2023-12-27 07:37:54,397][20381] Updated weights for policy 1, policy_version 286459 (0.0008) [2023-12-27 07:37:54,459][20381] Updated weights for policy 1, policy_version 286469 (0.0008) [2023-12-27 07:37:54,641][20347] Updated weights for policy 0, policy_version 286753 (0.0007) [2023-12-27 07:37:54,696][20347] Updated weights for policy 0, policy_version 286763 (0.0008) [2023-12-27 07:37:54,752][20347] Updated weights for policy 0, policy_version 286773 (0.0009) [2023-12-27 07:37:54,798][20347] Updated weights for policy 0, policy_version 286783 (0.0009) [2023-12-27 07:37:55,078][20381] Updated weights for policy 1, policy_version 286479 (0.0009) [2023-12-27 07:37:55,137][20381] Updated weights for policy 1, policy_version 286489 (0.0007) [2023-12-27 07:37:55,188][20381] Updated weights for policy 1, policy_version 286499 (0.0008) [2023-12-27 07:37:55,494][20347] Updated weights for policy 0, policy_version 286793 (0.0008) [2023-12-27 07:37:55,552][20347] Updated weights for policy 0, policy_version 286803 (0.0008) [2023-12-27 07:37:55,607][20347] Updated weights for policy 0, policy_version 286813 (0.0008) [2023-12-27 07:37:55,882][20381] Updated weights for policy 1, policy_version 286509 (0.0007) [2023-12-27 07:37:55,937][20381] Updated weights for policy 1, policy_version 286519 (0.0005) [2023-12-27 07:37:55,985][20381] Updated weights for policy 1, policy_version 286529 (0.0007) [2023-12-27 07:37:56,334][20347] Updated weights for policy 0, policy_version 286823 (0.0007) [2023-12-27 07:37:56,393][20347] Updated weights for policy 0, policy_version 286833 (0.0008) [2023-12-27 07:37:56,440][20347] Updated weights for policy 0, policy_version 286843 (0.0007) [2023-12-27 07:37:56,578][20381] Updated weights for policy 1, policy_version 286539 (0.0009) [2023-12-27 07:37:56,621][20381] Updated weights for policy 1, policy_version 286549 (0.0006) [2023-12-27 07:37:56,670][20381] Updated weights for policy 1, policy_version 286559 (0.0006) [2023-12-27 07:37:57,067][20347] Updated weights for policy 0, policy_version 286853 (0.0006) [2023-12-27 07:37:57,124][20347] Updated weights for policy 0, policy_version 286863 (0.0008) [2023-12-27 07:37:57,176][20347] Updated weights for policy 0, policy_version 286873 (0.0006) [2023-12-27 07:37:57,294][20381] Updated weights for policy 1, policy_version 286569 (0.0006) [2023-12-27 07:37:57,349][20381] Updated weights for policy 1, policy_version 286579 (0.0005) [2023-12-27 07:37:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 146833408. Throughput: 0: 10339.9, 1: 10461.3. Samples: 146821848. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:37:57,363][19170] Avg episode reward: [(0, '828.560'), (1, '971.390')] [2023-12-27 07:37:57,394][20381] Updated weights for policy 1, policy_version 286589 (0.0005) [2023-12-27 07:37:57,451][20381] Updated weights for policy 1, policy_version 286599 (0.0005) [2023-12-27 07:37:57,839][20347] Updated weights for policy 0, policy_version 286883 (0.0005) [2023-12-27 07:37:57,888][20347] Updated weights for policy 0, policy_version 286893 (0.0005) [2023-12-27 07:37:57,939][20347] Updated weights for policy 0, policy_version 286903 (0.0005) [2023-12-27 07:37:58,029][20381] Updated weights for policy 1, policy_version 286609 (0.0005) [2023-12-27 07:37:58,075][20381] Updated weights for policy 1, policy_version 286619 (0.0007) [2023-12-27 07:37:58,125][20381] Updated weights for policy 1, policy_version 286629 (0.0008) [2023-12-27 07:37:58,609][20347] Updated weights for policy 0, policy_version 286913 (0.0006) [2023-12-27 07:37:58,672][20347] Updated weights for policy 0, policy_version 286923 (0.0008) [2023-12-27 07:37:58,733][20347] Updated weights for policy 0, policy_version 286933 (0.0008) [2023-12-27 07:37:58,793][20347] Updated weights for policy 0, policy_version 286943 (0.0008) [2023-12-27 07:37:58,912][20381] Updated weights for policy 1, policy_version 286639 (0.0008) [2023-12-27 07:37:58,965][20381] Updated weights for policy 1, policy_version 286649 (0.0009) [2023-12-27 07:37:59,016][20381] Updated weights for policy 1, policy_version 286659 (0.0008) [2023-12-27 07:37:59,501][20347] Updated weights for policy 0, policy_version 286953 (0.0008) [2023-12-27 07:37:59,549][20347] Updated weights for policy 0, policy_version 286963 (0.0008) [2023-12-27 07:37:59,605][20347] Updated weights for policy 0, policy_version 286973 (0.0006) [2023-12-27 07:37:59,689][20381] Updated weights for policy 1, policy_version 286669 (0.0008) [2023-12-27 07:37:59,751][20381] Updated weights for policy 1, policy_version 286679 (0.0010) [2023-12-27 07:37:59,797][20381] Updated weights for policy 1, policy_version 286689 (0.0008) [2023-12-27 07:38:00,300][20347] Updated weights for policy 0, policy_version 286983 (0.0007) [2023-12-27 07:38:00,346][20347] Updated weights for policy 0, policy_version 286993 (0.0009) [2023-12-27 07:38:00,379][20381] Updated weights for policy 1, policy_version 286699 (0.0008) [2023-12-27 07:38:00,397][20347] Updated weights for policy 0, policy_version 287003 (0.0007) [2023-12-27 07:38:00,436][20381] Updated weights for policy 1, policy_version 286709 (0.0010) [2023-12-27 07:38:00,494][20381] Updated weights for policy 1, policy_version 286719 (0.0010) [2023-12-27 07:38:01,087][20347] Updated weights for policy 0, policy_version 287013 (0.0006) [2023-12-27 07:38:01,122][20381] Updated weights for policy 1, policy_version 286729 (0.0010) [2023-12-27 07:38:01,143][20347] Updated weights for policy 0, policy_version 287023 (0.0007) [2023-12-27 07:38:01,187][20381] Updated weights for policy 1, policy_version 286739 (0.0006) [2023-12-27 07:38:01,204][20347] Updated weights for policy 0, policy_version 287033 (0.0007) [2023-12-27 07:38:01,249][20381] Updated weights for policy 1, policy_version 286749 (0.0006) [2023-12-27 07:38:01,308][20381] Updated weights for policy 1, policy_version 286759 (0.0005) [2023-12-27 07:38:01,877][20347] Updated weights for policy 0, policy_version 287043 (0.0007) [2023-12-27 07:38:01,937][20347] Updated weights for policy 0, policy_version 287053 (0.0007) [2023-12-27 07:38:01,983][20381] Updated weights for policy 1, policy_version 286769 (0.0006) [2023-12-27 07:38:01,993][20347] Updated weights for policy 0, policy_version 287063 (0.0007) [2023-12-27 07:38:02,004][20292] KL-divergence is very high: 181.5562 [2023-12-27 07:38:02,036][20381] Updated weights for policy 1, policy_version 286779 (0.0005) [2023-12-27 07:38:02,093][20381] Updated weights for policy 1, policy_version 286789 (0.0006) [2023-12-27 07:38:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 146939904. Throughput: 0: 10369.1, 1: 10489.2. Samples: 146948140. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:02,363][19170] Avg episode reward: [(0, '942.410'), (1, '963.270')] [2023-12-27 07:38:02,659][20381] Updated weights for policy 1, policy_version 286799 (0.0008) [2023-12-27 07:38:02,709][20381] Updated weights for policy 1, policy_version 286809 (0.0008) [2023-12-27 07:38:02,718][20347] Updated weights for policy 0, policy_version 287073 (0.0008) [2023-12-27 07:38:02,764][20347] Updated weights for policy 0, policy_version 287083 (0.0006) [2023-12-27 07:38:02,765][20381] Updated weights for policy 1, policy_version 286819 (0.0007) [2023-12-27 07:38:02,824][20347] Updated weights for policy 0, policy_version 287093 (0.0008) [2023-12-27 07:38:02,876][20347] Updated weights for policy 0, policy_version 287103 (0.0007) [2023-12-27 07:38:03,431][20381] Updated weights for policy 1, policy_version 286829 (0.0008) [2023-12-27 07:38:03,494][20381] Updated weights for policy 1, policy_version 286839 (0.0010) [2023-12-27 07:38:03,542][20347] Updated weights for policy 0, policy_version 287113 (0.0005) [2023-12-27 07:38:03,542][20381] Updated weights for policy 1, policy_version 286849 (0.0010) [2023-12-27 07:38:03,599][20347] Updated weights for policy 0, policy_version 287123 (0.0007) [2023-12-27 07:38:03,661][20347] Updated weights for policy 0, policy_version 287133 (0.0010) [2023-12-27 07:38:04,226][20381] Updated weights for policy 1, policy_version 286859 (0.0010) [2023-12-27 07:38:04,291][20381] Updated weights for policy 1, policy_version 286869 (0.0011) [2023-12-27 07:38:04,346][20347] Updated weights for policy 0, policy_version 287143 (0.0007) [2023-12-27 07:38:04,353][20381] Updated weights for policy 1, policy_version 286879 (0.0011) [2023-12-27 07:38:04,405][20347] Updated weights for policy 0, policy_version 287153 (0.0006) [2023-12-27 07:38:04,460][20347] Updated weights for policy 0, policy_version 287163 (0.0007) [2023-12-27 07:38:04,968][20381] Updated weights for policy 1, policy_version 286889 (0.0011) [2023-12-27 07:38:05,017][20381] Updated weights for policy 1, policy_version 286899 (0.0010) [2023-12-27 07:38:05,069][20381] Updated weights for policy 1, policy_version 286909 (0.0010) [2023-12-27 07:38:05,093][20347] Updated weights for policy 0, policy_version 287173 (0.0008) [2023-12-27 07:38:05,124][20381] Updated weights for policy 1, policy_version 286919 (0.0010) [2023-12-27 07:38:05,141][20347] Updated weights for policy 0, policy_version 287183 (0.0010) [2023-12-27 07:38:05,199][20347] Updated weights for policy 0, policy_version 287193 (0.0010) [2023-12-27 07:38:05,728][20381] Updated weights for policy 1, policy_version 286929 (0.0008) [2023-12-27 07:38:05,786][20381] Updated weights for policy 1, policy_version 286939 (0.0007) [2023-12-27 07:38:05,832][20381] Updated weights for policy 1, policy_version 286949 (0.0008) [2023-12-27 07:38:05,855][20347] Updated weights for policy 0, policy_version 287203 (0.0009) [2023-12-27 07:38:05,911][20347] Updated weights for policy 0, policy_version 287213 (0.0007) [2023-12-27 07:38:05,963][20347] Updated weights for policy 0, policy_version 287223 (0.0008) [2023-12-27 07:38:06,497][20381] Updated weights for policy 1, policy_version 286959 (0.0006) [2023-12-27 07:38:06,561][20381] Updated weights for policy 1, policy_version 286969 (0.0007) [2023-12-27 07:38:06,564][20347] Updated weights for policy 0, policy_version 287233 (0.0008) [2023-12-27 07:38:06,617][20347] Updated weights for policy 0, policy_version 287243 (0.0007) [2023-12-27 07:38:06,619][20381] Updated weights for policy 1, policy_version 286979 (0.0008) [2023-12-27 07:38:06,666][20347] Updated weights for policy 0, policy_version 287253 (0.0006) [2023-12-27 07:38:06,724][20347] Updated weights for policy 0, policy_version 287263 (0.0008) [2023-12-27 07:38:07,279][20381] Updated weights for policy 1, policy_version 286989 (0.0006) [2023-12-27 07:38:07,329][20381] Updated weights for policy 1, policy_version 286999 (0.0008) [2023-12-27 07:38:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 147038208. Throughput: 0: 10360.8, 1: 10512.6. Samples: 147011168. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:07,363][19170] Avg episode reward: [(0, '980.660'), (1, '938.520')] [2023-12-27 07:38:07,388][20381] Updated weights for policy 1, policy_version 287009 (0.0007) [2023-12-27 07:38:07,397][20347] Updated weights for policy 0, policy_version 287273 (0.0007) [2023-12-27 07:38:07,425][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000287016_73490432.pth... [2023-12-27 07:38:07,429][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000285768_73170944.pth [2023-12-27 07:38:07,450][20347] Updated weights for policy 0, policy_version 287283 (0.0006) [2023-12-27 07:38:07,502][20347] Updated weights for policy 0, policy_version 287293 (0.0007) [2023-12-27 07:38:07,514][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000287296_73564160.pth... [2023-12-27 07:38:07,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000286080_73252864.pth [2023-12-27 07:38:07,976][20381] Updated weights for policy 1, policy_version 287019 (0.0006) [2023-12-27 07:38:08,033][20381] Updated weights for policy 1, policy_version 287029 (0.0005) [2023-12-27 07:38:08,085][20381] Updated weights for policy 1, policy_version 287039 (0.0005) [2023-12-27 07:38:08,262][20347] Updated weights for policy 0, policy_version 287303 (0.0008) [2023-12-27 07:38:08,319][20347] Updated weights for policy 0, policy_version 287313 (0.0008) [2023-12-27 07:38:08,377][20347] Updated weights for policy 0, policy_version 287323 (0.0008) [2023-12-27 07:38:08,737][20381] Updated weights for policy 1, policy_version 287049 (0.0006) [2023-12-27 07:38:08,797][20381] Updated weights for policy 1, policy_version 287059 (0.0007) [2023-12-27 07:38:08,860][20381] Updated weights for policy 1, policy_version 287069 (0.0008) [2023-12-27 07:38:08,915][20381] Updated weights for policy 1, policy_version 287079 (0.0006) [2023-12-27 07:38:08,980][20347] Updated weights for policy 0, policy_version 287333 (0.0007) [2023-12-27 07:38:09,035][20347] Updated weights for policy 0, policy_version 287343 (0.0007) [2023-12-27 07:38:09,093][20347] Updated weights for policy 0, policy_version 287353 (0.0006) [2023-12-27 07:38:09,560][20381] Updated weights for policy 1, policy_version 287089 (0.0007) [2023-12-27 07:38:09,616][20381] Updated weights for policy 1, policy_version 287099 (0.0007) [2023-12-27 07:38:09,668][20381] Updated weights for policy 1, policy_version 287109 (0.0008) [2023-12-27 07:38:09,781][20347] Updated weights for policy 0, policy_version 287363 (0.0007) [2023-12-27 07:38:09,842][20347] Updated weights for policy 0, policy_version 287373 (0.0010) [2023-12-27 07:38:09,904][20347] Updated weights for policy 0, policy_version 287383 (0.0008) [2023-12-27 07:38:10,365][20381] Updated weights for policy 1, policy_version 287119 (0.0007) [2023-12-27 07:38:10,413][20381] Updated weights for policy 1, policy_version 287129 (0.0006) [2023-12-27 07:38:10,474][20381] Updated weights for policy 1, policy_version 287139 (0.0008) [2023-12-27 07:38:10,529][20347] Updated weights for policy 0, policy_version 287393 (0.0008) [2023-12-27 07:38:10,591][20347] Updated weights for policy 0, policy_version 287403 (0.0006) [2023-12-27 07:38:10,637][20347] Updated weights for policy 0, policy_version 287413 (0.0005) [2023-12-27 07:38:10,688][20347] Updated weights for policy 0, policy_version 287423 (0.0005) [2023-12-27 07:38:11,071][20381] Updated weights for policy 1, policy_version 287149 (0.0009) [2023-12-27 07:38:11,134][20381] Updated weights for policy 1, policy_version 287159 (0.0006) [2023-12-27 07:38:11,204][20381] Updated weights for policy 1, policy_version 287169 (0.0007) [2023-12-27 07:38:11,334][20347] Updated weights for policy 0, policy_version 287433 (0.0008) [2023-12-27 07:38:11,403][20347] Updated weights for policy 0, policy_version 287443 (0.0008) [2023-12-27 07:38:11,469][20347] Updated weights for policy 0, policy_version 287453 (0.0009) [2023-12-27 07:38:11,864][20381] Updated weights for policy 1, policy_version 287179 (0.0007) [2023-12-27 07:38:11,926][20381] Updated weights for policy 1, policy_version 287189 (0.0008) [2023-12-27 07:38:11,990][20381] Updated weights for policy 1, policy_version 287199 (0.0008) [2023-12-27 07:38:12,218][20347] Updated weights for policy 0, policy_version 287463 (0.0007) [2023-12-27 07:38:12,280][20347] Updated weights for policy 0, policy_version 287473 (0.0007) [2023-12-27 07:38:12,341][20347] Updated weights for policy 0, policy_version 287483 (0.0009) [2023-12-27 07:38:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 147144704. Throughput: 0: 10349.4, 1: 10578.2. Samples: 147138528. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:12,363][19170] Avg episode reward: [(0, '990.360'), (1, '936.840')] [2023-12-27 07:38:12,724][20381] Updated weights for policy 1, policy_version 287209 (0.0008) [2023-12-27 07:38:12,786][20381] Updated weights for policy 1, policy_version 287219 (0.0008) [2023-12-27 07:38:12,842][20381] Updated weights for policy 1, policy_version 287229 (0.0008) [2023-12-27 07:38:12,898][20381] Updated weights for policy 1, policy_version 287239 (0.0006) [2023-12-27 07:38:12,966][20347] Updated weights for policy 0, policy_version 287493 (0.0007) [2023-12-27 07:38:13,014][20347] Updated weights for policy 0, policy_version 287503 (0.0008) [2023-12-27 07:38:13,069][20347] Updated weights for policy 0, policy_version 287513 (0.0008) [2023-12-27 07:38:13,496][20381] Updated weights for policy 1, policy_version 287249 (0.0008) [2023-12-27 07:38:13,541][20381] Updated weights for policy 1, policy_version 287259 (0.0007) [2023-12-27 07:38:13,586][20381] Updated weights for policy 1, policy_version 287269 (0.0007) [2023-12-27 07:38:13,787][20347] Updated weights for policy 0, policy_version 287523 (0.0008) [2023-12-27 07:38:13,831][20347] Updated weights for policy 0, policy_version 287533 (0.0008) [2023-12-27 07:38:13,879][20347] Updated weights for policy 0, policy_version 287543 (0.0008) [2023-12-27 07:38:14,266][20381] Updated weights for policy 1, policy_version 287279 (0.0007) [2023-12-27 07:38:14,315][20381] Updated weights for policy 1, policy_version 287289 (0.0008) [2023-12-27 07:38:14,366][20381] Updated weights for policy 1, policy_version 287299 (0.0007) [2023-12-27 07:38:14,542][20347] Updated weights for policy 0, policy_version 287553 (0.0007) [2023-12-27 07:38:14,598][20347] Updated weights for policy 0, policy_version 287563 (0.0009) [2023-12-27 07:38:14,647][20347] Updated weights for policy 0, policy_version 287573 (0.0008) [2023-12-27 07:38:14,700][20347] Updated weights for policy 0, policy_version 287583 (0.0007) [2023-12-27 07:38:14,993][20381] Updated weights for policy 1, policy_version 287309 (0.0008) [2023-12-27 07:38:15,050][20381] Updated weights for policy 1, policy_version 287319 (0.0008) [2023-12-27 07:38:15,104][20381] Updated weights for policy 1, policy_version 287329 (0.0008) [2023-12-27 07:38:15,420][20347] Updated weights for policy 0, policy_version 287593 (0.0007) [2023-12-27 07:38:15,476][20347] Updated weights for policy 0, policy_version 287603 (0.0008) [2023-12-27 07:38:15,536][20347] Updated weights for policy 0, policy_version 287613 (0.0008) [2023-12-27 07:38:15,721][20381] Updated weights for policy 1, policy_version 287339 (0.0008) [2023-12-27 07:38:15,771][20381] Updated weights for policy 1, policy_version 287349 (0.0008) [2023-12-27 07:38:15,825][20381] Updated weights for policy 1, policy_version 287359 (0.0007) [2023-12-27 07:38:16,245][20347] Updated weights for policy 0, policy_version 287623 (0.0008) [2023-12-27 07:38:16,296][20347] Updated weights for policy 0, policy_version 287633 (0.0008) [2023-12-27 07:38:16,350][20347] Updated weights for policy 0, policy_version 287643 (0.0007) [2023-12-27 07:38:16,457][20381] Updated weights for policy 1, policy_version 287369 (0.0010) [2023-12-27 07:38:16,510][20381] Updated weights for policy 1, policy_version 287379 (0.0005) [2023-12-27 07:38:16,566][20381] Updated weights for policy 1, policy_version 287389 (0.0006) [2023-12-27 07:38:16,614][20381] Updated weights for policy 1, policy_version 287399 (0.0005) [2023-12-27 07:38:17,071][20347] Updated weights for policy 0, policy_version 287653 (0.0009) [2023-12-27 07:38:17,133][20347] Updated weights for policy 0, policy_version 287663 (0.0009) [2023-12-27 07:38:17,180][20347] Updated weights for policy 0, policy_version 287673 (0.0008) [2023-12-27 07:38:17,253][20381] Updated weights for policy 1, policy_version 287409 (0.0008) [2023-12-27 07:38:17,311][20381] Updated weights for policy 1, policy_version 287419 (0.0008) [2023-12-27 07:38:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 147251200. Throughput: 0: 10294.0, 1: 10609.8. Samples: 147262872. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '943.700')] [2023-12-27 07:38:17,375][20381] Updated weights for policy 1, policy_version 287429 (0.0007) [2023-12-27 07:38:17,869][20347] Updated weights for policy 0, policy_version 287683 (0.0007) [2023-12-27 07:38:17,929][20347] Updated weights for policy 0, policy_version 287693 (0.0009) [2023-12-27 07:38:17,982][20347] Updated weights for policy 0, policy_version 287703 (0.0008) [2023-12-27 07:38:18,042][20381] Updated weights for policy 1, policy_version 287439 (0.0007) [2023-12-27 07:38:18,098][20381] Updated weights for policy 1, policy_version 287449 (0.0006) [2023-12-27 07:38:18,150][20381] Updated weights for policy 1, policy_version 287459 (0.0006) [2023-12-27 07:38:18,659][20347] Updated weights for policy 0, policy_version 287713 (0.0008) [2023-12-27 07:38:18,720][20347] Updated weights for policy 0, policy_version 287723 (0.0010) [2023-12-27 07:38:18,792][20347] Updated weights for policy 0, policy_version 287733 (0.0009) [2023-12-27 07:38:18,815][20381] Updated weights for policy 1, policy_version 287469 (0.0007) [2023-12-27 07:38:18,847][20347] Updated weights for policy 0, policy_version 287743 (0.0008) [2023-12-27 07:38:18,872][20381] Updated weights for policy 1, policy_version 287479 (0.0011) [2023-12-27 07:38:18,934][20381] Updated weights for policy 1, policy_version 287489 (0.0011) [2023-12-27 07:38:19,519][20347] Updated weights for policy 0, policy_version 287753 (0.0007) [2023-12-27 07:38:19,575][20347] Updated weights for policy 0, policy_version 287763 (0.0007) [2023-12-27 07:38:19,618][20381] Updated weights for policy 1, policy_version 287499 (0.0011) [2023-12-27 07:38:19,628][20347] Updated weights for policy 0, policy_version 287773 (0.0007) [2023-12-27 07:38:19,671][20381] Updated weights for policy 1, policy_version 287509 (0.0010) [2023-12-27 07:38:19,730][20381] Updated weights for policy 1, policy_version 287519 (0.0010) [2023-12-27 07:38:20,348][20381] Updated weights for policy 1, policy_version 287529 (0.0010) [2023-12-27 07:38:20,374][20347] Updated weights for policy 0, policy_version 287783 (0.0007) [2023-12-27 07:38:20,396][20381] Updated weights for policy 1, policy_version 287539 (0.0009) [2023-12-27 07:38:20,426][20347] Updated weights for policy 0, policy_version 287793 (0.0008) [2023-12-27 07:38:20,449][20381] Updated weights for policy 1, policy_version 287549 (0.0011) [2023-12-27 07:38:20,480][20347] Updated weights for policy 0, policy_version 287803 (0.0008) [2023-12-27 07:38:20,508][20381] Updated weights for policy 1, policy_version 287559 (0.0011) [2023-12-27 07:38:21,136][20381] Updated weights for policy 1, policy_version 287569 (0.0011) [2023-12-27 07:38:21,205][20381] Updated weights for policy 1, policy_version 287579 (0.0011) [2023-12-27 07:38:21,239][20347] Updated weights for policy 0, policy_version 287813 (0.0007) [2023-12-27 07:38:21,270][20381] Updated weights for policy 1, policy_version 287589 (0.0008) [2023-12-27 07:38:21,296][20347] Updated weights for policy 0, policy_version 287823 (0.0006) [2023-12-27 07:38:21,353][20347] Updated weights for policy 0, policy_version 287833 (0.0006) [2023-12-27 07:38:21,992][20381] Updated weights for policy 1, policy_version 287599 (0.0010) [2023-12-27 07:38:22,016][20347] Updated weights for policy 0, policy_version 287843 (0.0008) [2023-12-27 07:38:22,053][20381] Updated weights for policy 1, policy_version 287609 (0.0007) [2023-12-27 07:38:22,078][20347] Updated weights for policy 0, policy_version 287853 (0.0006) [2023-12-27 07:38:22,111][20381] Updated weights for policy 1, policy_version 287619 (0.0008) [2023-12-27 07:38:22,141][20347] Updated weights for policy 0, policy_version 287863 (0.0006) [2023-12-27 07:38:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 147357696. Throughput: 0: 10270.2, 1: 10597.7. Samples: 147323796. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '935.590')] [2023-12-27 07:38:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000287872_73711616.pth... [2023-12-27 07:38:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000287624_73646080.pth... [2023-12-27 07:38:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000286688_73408512.pth [2023-12-27 07:38:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000286376_73326592.pth [2023-12-27 07:38:22,721][20381] Updated weights for policy 1, policy_version 287629 (0.0008) [2023-12-27 07:38:22,782][20381] Updated weights for policy 1, policy_version 287639 (0.0008) [2023-12-27 07:38:22,834][20381] Updated weights for policy 1, policy_version 287649 (0.0008) [2023-12-27 07:38:22,854][20347] Updated weights for policy 0, policy_version 287873 (0.0009) [2023-12-27 07:38:22,908][20347] Updated weights for policy 0, policy_version 287883 (0.0007) [2023-12-27 07:38:22,975][20347] Updated weights for policy 0, policy_version 287893 (0.0008) [2023-12-27 07:38:23,033][20347] Updated weights for policy 0, policy_version 287903 (0.0008) [2023-12-27 07:38:23,456][20381] Updated weights for policy 1, policy_version 287659 (0.0006) [2023-12-27 07:38:23,505][20381] Updated weights for policy 1, policy_version 287669 (0.0008) [2023-12-27 07:38:23,558][20381] Updated weights for policy 1, policy_version 287679 (0.0008) [2023-12-27 07:38:23,727][20347] Updated weights for policy 0, policy_version 287913 (0.0008) [2023-12-27 07:38:23,785][20347] Updated weights for policy 0, policy_version 287923 (0.0006) [2023-12-27 07:38:23,837][20347] Updated weights for policy 0, policy_version 287933 (0.0005) [2023-12-27 07:38:24,261][20381] Updated weights for policy 1, policy_version 287689 (0.0008) [2023-12-27 07:38:24,312][20381] Updated weights for policy 1, policy_version 287699 (0.0007) [2023-12-27 07:38:24,367][20381] Updated weights for policy 1, policy_version 287709 (0.0008) [2023-12-27 07:38:24,423][20381] Updated weights for policy 1, policy_version 287719 (0.0008) [2023-12-27 07:38:24,469][20347] Updated weights for policy 0, policy_version 287943 (0.0007) [2023-12-27 07:38:24,521][20347] Updated weights for policy 0, policy_version 287953 (0.0008) [2023-12-27 07:38:24,568][20347] Updated weights for policy 0, policy_version 287963 (0.0008) [2023-12-27 07:38:25,096][20381] Updated weights for policy 1, policy_version 287729 (0.0007) [2023-12-27 07:38:25,144][20381] Updated weights for policy 1, policy_version 287739 (0.0008) [2023-12-27 07:38:25,201][20381] Updated weights for policy 1, policy_version 287749 (0.0008) [2023-12-27 07:38:25,286][20347] Updated weights for policy 0, policy_version 287973 (0.0008) [2023-12-27 07:38:25,341][20347] Updated weights for policy 0, policy_version 287983 (0.0008) [2023-12-27 07:38:25,399][20347] Updated weights for policy 0, policy_version 287993 (0.0007) [2023-12-27 07:38:25,880][20381] Updated weights for policy 1, policy_version 287759 (0.0006) [2023-12-27 07:38:25,932][20381] Updated weights for policy 1, policy_version 287769 (0.0005) [2023-12-27 07:38:25,980][20381] Updated weights for policy 1, policy_version 287779 (0.0005) [2023-12-27 07:38:26,124][20347] Updated weights for policy 0, policy_version 288003 (0.0008) [2023-12-27 07:38:26,182][20347] Updated weights for policy 0, policy_version 288013 (0.0008) [2023-12-27 07:38:26,243][20347] Updated weights for policy 0, policy_version 288023 (0.0008) [2023-12-27 07:38:26,588][20381] Updated weights for policy 1, policy_version 287789 (0.0007) [2023-12-27 07:38:26,648][20381] Updated weights for policy 1, policy_version 287799 (0.0008) [2023-12-27 07:38:26,718][20381] Updated weights for policy 1, policy_version 287809 (0.0008) [2023-12-27 07:38:26,927][20347] Updated weights for policy 0, policy_version 288033 (0.0007) [2023-12-27 07:38:26,982][20347] Updated weights for policy 0, policy_version 288043 (0.0008) [2023-12-27 07:38:27,034][20347] Updated weights for policy 0, policy_version 288053 (0.0008) [2023-12-27 07:38:27,088][20347] Updated weights for policy 0, policy_version 288063 (0.0008) [2023-12-27 07:38:27,308][20381] Updated weights for policy 1, policy_version 287819 (0.0007) [2023-12-27 07:38:27,361][20381] Updated weights for policy 1, policy_version 287829 (0.0005) [2023-12-27 07:38:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 147456000. Throughput: 0: 10159.5, 1: 10626.7. Samples: 147445780. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:27,363][19170] Avg episode reward: [(0, '990.970'), (1, '961.760')] [2023-12-27 07:38:27,419][20381] Updated weights for policy 1, policy_version 287839 (0.0005) [2023-12-27 07:38:27,864][20347] Updated weights for policy 0, policy_version 288073 (0.0007) [2023-12-27 07:38:27,917][20347] Updated weights for policy 0, policy_version 288083 (0.0007) [2023-12-27 07:38:27,943][20381] Updated weights for policy 1, policy_version 287849 (0.0005) [2023-12-27 07:38:27,972][20347] Updated weights for policy 0, policy_version 288093 (0.0006) [2023-12-27 07:38:27,988][20381] Updated weights for policy 1, policy_version 287859 (0.0006) [2023-12-27 07:38:28,037][20381] Updated weights for policy 1, policy_version 287869 (0.0008) [2023-12-27 07:38:28,085][20381] Updated weights for policy 1, policy_version 287879 (0.0006) [2023-12-27 07:38:28,655][20347] Updated weights for policy 0, policy_version 288103 (0.0006) [2023-12-27 07:38:28,725][20347] Updated weights for policy 0, policy_version 288113 (0.0007) [2023-12-27 07:38:28,770][20381] Updated weights for policy 1, policy_version 287889 (0.0008) [2023-12-27 07:38:28,787][20347] Updated weights for policy 0, policy_version 288123 (0.0006) [2023-12-27 07:38:28,820][20381] Updated weights for policy 1, policy_version 287899 (0.0008) [2023-12-27 07:38:28,882][20381] Updated weights for policy 1, policy_version 287909 (0.0008) [2023-12-27 07:38:29,341][20347] Updated weights for policy 0, policy_version 288133 (0.0006) [2023-12-27 07:38:29,403][20347] Updated weights for policy 0, policy_version 288143 (0.0007) [2023-12-27 07:38:29,466][20347] Updated weights for policy 0, policy_version 288153 (0.0007) [2023-12-27 07:38:29,602][20381] Updated weights for policy 1, policy_version 287919 (0.0006) [2023-12-27 07:38:29,659][20381] Updated weights for policy 1, policy_version 287929 (0.0008) [2023-12-27 07:38:29,716][20381] Updated weights for policy 1, policy_version 287939 (0.0008) [2023-12-27 07:38:30,118][20347] Updated weights for policy 0, policy_version 288163 (0.0006) [2023-12-27 07:38:30,175][20347] Updated weights for policy 0, policy_version 288173 (0.0008) [2023-12-27 07:38:30,234][20347] Updated weights for policy 0, policy_version 288183 (0.0008) [2023-12-27 07:38:30,383][20381] Updated weights for policy 1, policy_version 287949 (0.0007) [2023-12-27 07:38:30,439][20381] Updated weights for policy 1, policy_version 287959 (0.0008) [2023-12-27 07:38:30,483][20381] Updated weights for policy 1, policy_version 287969 (0.0007) [2023-12-27 07:38:30,877][20347] Updated weights for policy 0, policy_version 288193 (0.0007) [2023-12-27 07:38:30,939][20347] Updated weights for policy 0, policy_version 288203 (0.0005) [2023-12-27 07:38:30,995][20347] Updated weights for policy 0, policy_version 288213 (0.0005) [2023-12-27 07:38:31,059][20347] Updated weights for policy 0, policy_version 288223 (0.0006) [2023-12-27 07:38:31,226][20381] Updated weights for policy 1, policy_version 287979 (0.0008) [2023-12-27 07:38:31,280][20381] Updated weights for policy 1, policy_version 287989 (0.0008) [2023-12-27 07:38:31,331][20381] Updated weights for policy 1, policy_version 287999 (0.0008) [2023-12-27 07:38:31,686][20347] Updated weights for policy 0, policy_version 288233 (0.0006) [2023-12-27 07:38:31,752][20347] Updated weights for policy 0, policy_version 288243 (0.0007) [2023-12-27 07:38:31,813][20347] Updated weights for policy 0, policy_version 288253 (0.0008) [2023-12-27 07:38:32,098][20381] Updated weights for policy 1, policy_version 288009 (0.0008) [2023-12-27 07:38:32,154][20381] Updated weights for policy 1, policy_version 288019 (0.0007) [2023-12-27 07:38:32,216][20381] Updated weights for policy 1, policy_version 288029 (0.0008) [2023-12-27 07:38:32,278][20381] Updated weights for policy 1, policy_version 288039 (0.0009) [2023-12-27 07:38:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 147562496. Throughput: 0: 10202.5, 1: 10575.4. Samples: 147570692. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:32,363][19170] Avg episode reward: [(0, '981.350'), (1, '990.430')] [2023-12-27 07:38:32,449][20347] Updated weights for policy 0, policy_version 288263 (0.0009) [2023-12-27 07:38:32,498][20347] Updated weights for policy 0, policy_version 288273 (0.0005) [2023-12-27 07:38:32,548][20347] Updated weights for policy 0, policy_version 288283 (0.0005) [2023-12-27 07:38:33,036][20381] Updated weights for policy 1, policy_version 288049 (0.0008) [2023-12-27 07:38:33,079][20347] Updated weights for policy 0, policy_version 288293 (0.0005) [2023-12-27 07:38:33,085][20381] Updated weights for policy 1, policy_version 288059 (0.0007) [2023-12-27 07:38:33,129][20347] Updated weights for policy 0, policy_version 288303 (0.0005) [2023-12-27 07:38:33,138][20381] Updated weights for policy 1, policy_version 288069 (0.0008) [2023-12-27 07:38:33,174][20347] Updated weights for policy 0, policy_version 288313 (0.0005) [2023-12-27 07:38:33,782][20347] Updated weights for policy 0, policy_version 288323 (0.0007) [2023-12-27 07:38:33,840][20347] Updated weights for policy 0, policy_version 288333 (0.0010) [2023-12-27 07:38:33,864][20381] Updated weights for policy 1, policy_version 288079 (0.0006) [2023-12-27 07:38:33,887][20347] Updated weights for policy 0, policy_version 288343 (0.0010) [2023-12-27 07:38:33,913][20381] Updated weights for policy 1, policy_version 288089 (0.0005) [2023-12-27 07:38:33,970][20381] Updated weights for policy 1, policy_version 288099 (0.0005) [2023-12-27 07:38:34,565][20347] Updated weights for policy 0, policy_version 288353 (0.0010) [2023-12-27 07:38:34,602][20381] Updated weights for policy 1, policy_version 288109 (0.0008) [2023-12-27 07:38:34,626][20347] Updated weights for policy 0, policy_version 288363 (0.0010) [2023-12-27 07:38:34,655][20381] Updated weights for policy 1, policy_version 288119 (0.0010) [2023-12-27 07:38:34,688][20347] Updated weights for policy 0, policy_version 288373 (0.0007) [2023-12-27 07:38:34,701][20381] Updated weights for policy 1, policy_version 288129 (0.0010) [2023-12-27 07:38:34,746][20347] Updated weights for policy 0, policy_version 288383 (0.0009) [2023-12-27 07:38:35,318][20381] Updated weights for policy 1, policy_version 288139 (0.0006) [2023-12-27 07:38:35,372][20381] Updated weights for policy 1, policy_version 288149 (0.0007) [2023-12-27 07:38:35,373][20347] Updated weights for policy 0, policy_version 288393 (0.0010) [2023-12-27 07:38:35,421][20347] Updated weights for policy 0, policy_version 288403 (0.0006) [2023-12-27 07:38:35,433][20381] Updated weights for policy 1, policy_version 288159 (0.0008) [2023-12-27 07:38:35,470][20347] Updated weights for policy 0, policy_version 288413 (0.0005) [2023-12-27 07:38:36,036][20347] Updated weights for policy 0, policy_version 288423 (0.0009) [2023-12-27 07:38:36,062][20381] Updated weights for policy 1, policy_version 288169 (0.0010) [2023-12-27 07:38:36,089][20347] Updated weights for policy 0, policy_version 288433 (0.0010) [2023-12-27 07:38:36,116][20381] Updated weights for policy 1, policy_version 288179 (0.0007) [2023-12-27 07:38:36,161][20347] Updated weights for policy 0, policy_version 288443 (0.0008) [2023-12-27 07:38:36,178][20381] Updated weights for policy 1, policy_version 288189 (0.0007) [2023-12-27 07:38:36,239][20381] Updated weights for policy 1, policy_version 288199 (0.0008) [2023-12-27 07:38:36,771][20347] Updated weights for policy 0, policy_version 288453 (0.0009) [2023-12-27 07:38:36,842][20347] Updated weights for policy 0, policy_version 288463 (0.0007) [2023-12-27 07:38:36,909][20347] Updated weights for policy 0, policy_version 288473 (0.0006) [2023-12-27 07:38:36,910][20381] Updated weights for policy 1, policy_version 288209 (0.0006) [2023-12-27 07:38:36,975][20381] Updated weights for policy 1, policy_version 288219 (0.0006) [2023-12-27 07:38:37,040][20381] Updated weights for policy 1, policy_version 288229 (0.0007) [2023-12-27 07:38:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 147668992. Throughput: 0: 10247.3, 1: 10603.1. Samples: 147634800. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:37,363][19170] Avg episode reward: [(0, '990.380'), (1, '952.360')] [2023-12-27 07:38:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000288480_73867264.pth... [2023-12-27 07:38:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000288232_73801728.pth... [2023-12-27 07:38:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000287296_73564160.pth [2023-12-27 07:38:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000287016_73490432.pth [2023-12-27 07:38:37,547][20347] Updated weights for policy 0, policy_version 288483 (0.0006) [2023-12-27 07:38:37,605][20347] Updated weights for policy 0, policy_version 288493 (0.0005) [2023-12-27 07:38:37,631][20381] Updated weights for policy 1, policy_version 288239 (0.0006) [2023-12-27 07:38:37,651][20347] Updated weights for policy 0, policy_version 288503 (0.0006) [2023-12-27 07:38:37,682][20381] Updated weights for policy 1, policy_version 288249 (0.0006) [2023-12-27 07:38:37,741][20381] Updated weights for policy 1, policy_version 288259 (0.0007) [2023-12-27 07:38:38,309][20347] Updated weights for policy 0, policy_version 288513 (0.0007) [2023-12-27 07:38:38,370][20347] Updated weights for policy 0, policy_version 288523 (0.0010) [2023-12-27 07:38:38,409][20381] Updated weights for policy 1, policy_version 288269 (0.0008) [2023-12-27 07:38:38,425][20347] Updated weights for policy 0, policy_version 288533 (0.0010) [2023-12-27 07:38:38,466][20381] Updated weights for policy 1, policy_version 288279 (0.0007) [2023-12-27 07:38:38,483][20347] Updated weights for policy 0, policy_version 288543 (0.0010) [2023-12-27 07:38:38,527][20381] Updated weights for policy 1, policy_version 288289 (0.0007) [2023-12-27 07:38:39,145][20347] Updated weights for policy 0, policy_version 288553 (0.0010) [2023-12-27 07:38:39,180][20381] Updated weights for policy 1, policy_version 288299 (0.0008) [2023-12-27 07:38:39,202][20347] Updated weights for policy 0, policy_version 288563 (0.0008) [2023-12-27 07:38:39,241][20381] Updated weights for policy 1, policy_version 288309 (0.0008) [2023-12-27 07:38:39,265][20347] Updated weights for policy 0, policy_version 288573 (0.0008) [2023-12-27 07:38:39,301][20381] Updated weights for policy 1, policy_version 288319 (0.0006) [2023-12-27 07:38:39,954][20381] Updated weights for policy 1, policy_version 288329 (0.0007) [2023-12-27 07:38:40,019][20381] Updated weights for policy 1, policy_version 288339 (0.0009) [2023-12-27 07:38:40,026][20347] Updated weights for policy 0, policy_version 288583 (0.0007) [2023-12-27 07:38:40,077][20347] Updated weights for policy 0, policy_version 288593 (0.0008) [2023-12-27 07:38:40,082][20381] Updated weights for policy 1, policy_version 288349 (0.0007) [2023-12-27 07:38:40,128][20347] Updated weights for policy 0, policy_version 288603 (0.0007) [2023-12-27 07:38:40,141][20381] Updated weights for policy 1, policy_version 288359 (0.0007) [2023-12-27 07:38:40,834][20381] Updated weights for policy 1, policy_version 288369 (0.0007) [2023-12-27 07:38:40,871][20347] Updated weights for policy 0, policy_version 288613 (0.0006) [2023-12-27 07:38:40,900][20381] Updated weights for policy 1, policy_version 288379 (0.0008) [2023-12-27 07:38:40,930][20347] Updated weights for policy 0, policy_version 288623 (0.0007) [2023-12-27 07:38:40,956][20381] Updated weights for policy 1, policy_version 288389 (0.0007) [2023-12-27 07:38:40,989][20347] Updated weights for policy 0, policy_version 288633 (0.0006) [2023-12-27 07:38:41,624][20347] Updated weights for policy 0, policy_version 288643 (0.0007) [2023-12-27 07:38:41,684][20381] Updated weights for policy 1, policy_version 288399 (0.0007) [2023-12-27 07:38:41,689][20347] Updated weights for policy 0, policy_version 288653 (0.0006) [2023-12-27 07:38:41,749][20347] Updated weights for policy 0, policy_version 288663 (0.0007) [2023-12-27 07:38:41,750][20381] Updated weights for policy 1, policy_version 288409 (0.0007) [2023-12-27 07:38:41,814][20381] Updated weights for policy 1, policy_version 288419 (0.0009) [2023-12-27 07:38:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 147767296. Throughput: 0: 10296.0, 1: 10544.0. Samples: 147759648. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '923.430')] [2023-12-27 07:38:42,405][20347] Updated weights for policy 0, policy_version 288673 (0.0006) [2023-12-27 07:38:42,455][20347] Updated weights for policy 0, policy_version 288683 (0.0008) [2023-12-27 07:38:42,460][20381] Updated weights for policy 1, policy_version 288429 (0.0007) [2023-12-27 07:38:42,519][20347] Updated weights for policy 0, policy_version 288693 (0.0006) [2023-12-27 07:38:42,521][20381] Updated weights for policy 1, policy_version 288439 (0.0008) [2023-12-27 07:38:42,579][20381] Updated weights for policy 1, policy_version 288449 (0.0007) [2023-12-27 07:38:42,585][20347] Updated weights for policy 0, policy_version 288703 (0.0006) [2023-12-27 07:38:43,269][20347] Updated weights for policy 0, policy_version 288713 (0.0008) [2023-12-27 07:38:43,282][20381] Updated weights for policy 1, policy_version 288459 (0.0007) [2023-12-27 07:38:43,327][20347] Updated weights for policy 0, policy_version 288723 (0.0005) [2023-12-27 07:38:43,340][20381] Updated weights for policy 1, policy_version 288469 (0.0007) [2023-12-27 07:38:43,385][20347] Updated weights for policy 0, policy_version 288733 (0.0006) [2023-12-27 07:38:43,399][20381] Updated weights for policy 1, policy_version 288479 (0.0006) [2023-12-27 07:38:43,930][20347] Updated weights for policy 0, policy_version 288743 (0.0006) [2023-12-27 07:38:43,987][20347] Updated weights for policy 0, policy_version 288753 (0.0005) [2023-12-27 07:38:44,040][20347] Updated weights for policy 0, policy_version 288763 (0.0005) [2023-12-27 07:38:44,138][20381] Updated weights for policy 1, policy_version 288489 (0.0008) [2023-12-27 07:38:44,190][20381] Updated weights for policy 1, policy_version 288499 (0.0009) [2023-12-27 07:38:44,240][20381] Updated weights for policy 1, policy_version 288509 (0.0008) [2023-12-27 07:38:44,290][20381] Updated weights for policy 1, policy_version 288519 (0.0007) [2023-12-27 07:38:44,589][20347] Updated weights for policy 0, policy_version 288773 (0.0006) [2023-12-27 07:38:44,644][20347] Updated weights for policy 0, policy_version 288783 (0.0008) [2023-12-27 07:38:44,696][20347] Updated weights for policy 0, policy_version 288793 (0.0008) [2023-12-27 07:38:44,986][20381] Updated weights for policy 1, policy_version 288529 (0.0007) [2023-12-27 07:38:45,039][20381] Updated weights for policy 1, policy_version 288539 (0.0006) [2023-12-27 07:38:45,089][20381] Updated weights for policy 1, policy_version 288549 (0.0005) [2023-12-27 07:38:45,408][20347] Updated weights for policy 0, policy_version 288803 (0.0008) [2023-12-27 07:38:45,456][20347] Updated weights for policy 0, policy_version 288813 (0.0008) [2023-12-27 07:38:45,513][20347] Updated weights for policy 0, policy_version 288823 (0.0008) [2023-12-27 07:38:45,661][20381] Updated weights for policy 1, policy_version 288559 (0.0009) [2023-12-27 07:38:45,716][20381] Updated weights for policy 1, policy_version 288569 (0.0011) [2023-12-27 07:38:45,773][20381] Updated weights for policy 1, policy_version 288579 (0.0008) [2023-12-27 07:38:46,178][20347] Updated weights for policy 0, policy_version 288833 (0.0007) [2023-12-27 07:38:46,228][20347] Updated weights for policy 0, policy_version 288843 (0.0005) [2023-12-27 07:38:46,280][20347] Updated weights for policy 0, policy_version 288853 (0.0006) [2023-12-27 07:38:46,335][20347] Updated weights for policy 0, policy_version 288863 (0.0007) [2023-12-27 07:38:46,433][20381] Updated weights for policy 1, policy_version 288589 (0.0007) [2023-12-27 07:38:46,481][20381] Updated weights for policy 1, policy_version 288599 (0.0008) [2023-12-27 07:38:46,530][20381] Updated weights for policy 1, policy_version 288609 (0.0006) [2023-12-27 07:38:46,910][20347] Updated weights for policy 0, policy_version 288873 (0.0005) [2023-12-27 07:38:46,955][20347] Updated weights for policy 0, policy_version 288883 (0.0005) [2023-12-27 07:38:47,004][20347] Updated weights for policy 0, policy_version 288893 (0.0007) [2023-12-27 07:38:47,159][20381] Updated weights for policy 1, policy_version 288619 (0.0006) [2023-12-27 07:38:47,216][20381] Updated weights for policy 1, policy_version 288629 (0.0007) [2023-12-27 07:38:47,269][20381] Updated weights for policy 1, policy_version 288639 (0.0008) [2023-12-27 07:38:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 147881984. Throughput: 0: 10384.3, 1: 10481.4. Samples: 147887096. Policy #0 lag: (min: 19.0, avg: 26.6, max: 51.0) [2023-12-27 07:38:47,363][19170] Avg episode reward: [(0, '990.940'), (1, '951.810')] [2023-12-27 07:38:47,651][20347] Updated weights for policy 0, policy_version 288903 (0.0006) [2023-12-27 07:38:47,715][20347] Updated weights for policy 0, policy_version 288913 (0.0005) [2023-12-27 07:38:47,778][20347] Updated weights for policy 0, policy_version 288923 (0.0005) [2023-12-27 07:38:47,930][20381] Updated weights for policy 1, policy_version 288649 (0.0007) [2023-12-27 07:38:47,983][20381] Updated weights for policy 1, policy_version 288659 (0.0009) [2023-12-27 07:38:48,036][20381] Updated weights for policy 1, policy_version 288669 (0.0009) [2023-12-27 07:38:48,090][20381] Updated weights for policy 1, policy_version 288679 (0.0008) [2023-12-27 07:38:48,382][20347] Updated weights for policy 0, policy_version 288933 (0.0005) [2023-12-27 07:38:48,383][20291] KL-divergence is very high: 100.6672 [2023-12-27 07:38:48,442][20347] Updated weights for policy 0, policy_version 288943 (0.0006) [2023-12-27 07:38:48,506][20347] Updated weights for policy 0, policy_version 288953 (0.0008) [2023-12-27 07:38:48,758][20381] Updated weights for policy 1, policy_version 288689 (0.0008) [2023-12-27 07:38:48,810][20381] Updated weights for policy 1, policy_version 288699 (0.0011) [2023-12-27 07:38:48,862][20381] Updated weights for policy 1, policy_version 288709 (0.0011) [2023-12-27 07:38:49,179][20347] Updated weights for policy 0, policy_version 288963 (0.0007) [2023-12-27 07:38:49,237][20347] Updated weights for policy 0, policy_version 288973 (0.0007) [2023-12-27 07:38:49,290][20347] Updated weights for policy 0, policy_version 288983 (0.0008) [2023-12-27 07:38:49,504][20381] Updated weights for policy 1, policy_version 288719 (0.0011) [2023-12-27 07:38:49,558][20381] Updated weights for policy 1, policy_version 288729 (0.0010) [2023-12-27 07:38:49,614][20381] Updated weights for policy 1, policy_version 288739 (0.0009) [2023-12-27 07:38:49,994][20347] Updated weights for policy 0, policy_version 288993 (0.0007) [2023-12-27 07:38:50,058][20347] Updated weights for policy 0, policy_version 289003 (0.0006) [2023-12-27 07:38:50,122][20347] Updated weights for policy 0, policy_version 289013 (0.0008) [2023-12-27 07:38:50,185][20347] Updated weights for policy 0, policy_version 289023 (0.0006) [2023-12-27 07:38:50,288][20381] Updated weights for policy 1, policy_version 288749 (0.0008) [2023-12-27 07:38:50,350][20381] Updated weights for policy 1, policy_version 288759 (0.0011) [2023-12-27 07:38:50,411][20381] Updated weights for policy 1, policy_version 288769 (0.0011) [2023-12-27 07:38:50,816][20347] Updated weights for policy 0, policy_version 289034 (0.0009) [2023-12-27 07:38:50,872][20347] Updated weights for policy 0, policy_version 289044 (0.0009) [2023-12-27 07:38:50,930][20347] Updated weights for policy 0, policy_version 289054 (0.0008) [2023-12-27 07:38:51,089][20381] Updated weights for policy 1, policy_version 288779 (0.0011) [2023-12-27 07:38:51,151][20381] Updated weights for policy 1, policy_version 288789 (0.0011) [2023-12-27 07:38:51,214][20381] Updated weights for policy 1, policy_version 288799 (0.0011) [2023-12-27 07:38:51,599][20347] Updated weights for policy 0, policy_version 289064 (0.0006) [2023-12-27 07:38:51,661][20347] Updated weights for policy 0, policy_version 289074 (0.0009) [2023-12-27 07:38:51,723][20347] Updated weights for policy 0, policy_version 289084 (0.0009) [2023-12-27 07:38:51,914][20381] Updated weights for policy 1, policy_version 288809 (0.0011) [2023-12-27 07:38:51,962][20381] Updated weights for policy 1, policy_version 288819 (0.0010) [2023-12-27 07:38:52,010][20381] Updated weights for policy 1, policy_version 288829 (0.0010) [2023-12-27 07:38:52,067][20381] Updated weights for policy 1, policy_version 288839 (0.0010) [2023-12-27 07:38:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 147980288. Throughput: 0: 10394.0, 1: 10476.0. Samples: 147950320. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:38:52,363][19170] Avg episode reward: [(0, '953.600'), (1, '971.190')] [2023-12-27 07:38:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000288840_73957376.pth... [2023-12-27 07:38:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000287624_73646080.pth [2023-12-27 07:38:52,397][20347] Updated weights for policy 0, policy_version 289094 (0.0007) [2023-12-27 07:38:52,455][20347] Updated weights for policy 0, policy_version 289104 (0.0007) [2023-12-27 07:38:52,511][20347] Updated weights for policy 0, policy_version 289114 (0.0007) [2023-12-27 07:38:52,548][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000289120_74031104.pth... [2023-12-27 07:38:52,551][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000287872_73711616.pth [2023-12-27 07:38:52,802][20381] Updated weights for policy 1, policy_version 288849 (0.0011) [2023-12-27 07:38:52,853][20381] Updated weights for policy 1, policy_version 288859 (0.0011) [2023-12-27 07:38:52,903][20381] Updated weights for policy 1, policy_version 288869 (0.0007) [2023-12-27 07:38:53,096][20347] Updated weights for policy 0, policy_version 289124 (0.0006) [2023-12-27 07:38:53,156][20347] Updated weights for policy 0, policy_version 289134 (0.0008) [2023-12-27 07:38:53,208][20347] Updated weights for policy 0, policy_version 289144 (0.0008) [2023-12-27 07:38:53,549][20381] Updated weights for policy 1, policy_version 288879 (0.0007) [2023-12-27 07:38:53,601][20381] Updated weights for policy 1, policy_version 288889 (0.0007) [2023-12-27 07:38:53,657][20381] Updated weights for policy 1, policy_version 288899 (0.0008) [2023-12-27 07:38:53,910][20347] Updated weights for policy 0, policy_version 289154 (0.0008) [2023-12-27 07:38:53,964][20347] Updated weights for policy 0, policy_version 289164 (0.0006) [2023-12-27 07:38:54,012][20347] Updated weights for policy 0, policy_version 289174 (0.0006) [2023-12-27 07:38:54,060][20347] Updated weights for policy 0, policy_version 289184 (0.0007) [2023-12-27 07:38:54,367][20381] Updated weights for policy 1, policy_version 288909 (0.0008) [2023-12-27 07:38:54,417][20381] Updated weights for policy 1, policy_version 288919 (0.0009) [2023-12-27 07:38:54,473][20381] Updated weights for policy 1, policy_version 288929 (0.0009) [2023-12-27 07:38:54,694][20347] Updated weights for policy 0, policy_version 289194 (0.0005) [2023-12-27 07:38:54,749][20347] Updated weights for policy 0, policy_version 289204 (0.0005) [2023-12-27 07:38:54,810][20347] Updated weights for policy 0, policy_version 289214 (0.0008) [2023-12-27 07:38:55,093][20381] Updated weights for policy 1, policy_version 288939 (0.0006) [2023-12-27 07:38:55,143][20381] Updated weights for policy 1, policy_version 288949 (0.0006) [2023-12-27 07:38:55,199][20381] Updated weights for policy 1, policy_version 288959 (0.0007) [2023-12-27 07:38:55,466][20347] Updated weights for policy 0, policy_version 289224 (0.0007) [2023-12-27 07:38:55,524][20347] Updated weights for policy 0, policy_version 289234 (0.0010) [2023-12-27 07:38:55,586][20347] Updated weights for policy 0, policy_version 289244 (0.0006) [2023-12-27 07:38:55,821][20381] Updated weights for policy 1, policy_version 288969 (0.0007) [2023-12-27 07:38:55,892][20381] Updated weights for policy 1, policy_version 288979 (0.0011) [2023-12-27 07:38:55,937][20381] Updated weights for policy 1, policy_version 288989 (0.0010) [2023-12-27 07:38:55,993][20381] Updated weights for policy 1, policy_version 288999 (0.0010) [2023-12-27 07:38:56,179][20347] Updated weights for policy 0, policy_version 289254 (0.0008) [2023-12-27 07:38:56,234][20347] Updated weights for policy 0, policy_version 289264 (0.0010) [2023-12-27 07:38:56,293][20347] Updated weights for policy 0, policy_version 289274 (0.0010) [2023-12-27 07:38:56,611][20381] Updated weights for policy 1, policy_version 289009 (0.0010) [2023-12-27 07:38:56,675][20381] Updated weights for policy 1, policy_version 289019 (0.0005) [2023-12-27 07:38:56,725][20381] Updated weights for policy 1, policy_version 289029 (0.0005) [2023-12-27 07:38:56,888][20347] Updated weights for policy 0, policy_version 289284 (0.0008) [2023-12-27 07:38:56,939][20347] Updated weights for policy 0, policy_version 289294 (0.0010) [2023-12-27 07:38:56,999][20347] Updated weights for policy 0, policy_version 289304 (0.0005) [2023-12-27 07:38:57,275][20381] Updated weights for policy 1, policy_version 289039 (0.0005) [2023-12-27 07:38:57,327][20381] Updated weights for policy 1, policy_version 289049 (0.0005) [2023-12-27 07:38:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 148086784. Throughput: 0: 10413.6, 1: 10422.0. Samples: 148076128. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:38:57,363][19170] Avg episode reward: [(0, '962.270'), (1, '961.580')] [2023-12-27 07:38:57,378][20381] Updated weights for policy 1, policy_version 289059 (0.0009) [2023-12-27 07:38:57,623][20347] Updated weights for policy 0, policy_version 289314 (0.0005) [2023-12-27 07:38:57,669][20347] Updated weights for policy 0, policy_version 289324 (0.0005) [2023-12-27 07:38:57,726][20347] Updated weights for policy 0, policy_version 289334 (0.0005) [2023-12-27 07:38:57,781][20347] Updated weights for policy 0, policy_version 289344 (0.0010) [2023-12-27 07:38:58,027][20381] Updated weights for policy 1, policy_version 289069 (0.0008) [2023-12-27 07:38:58,079][20381] Updated weights for policy 1, policy_version 289079 (0.0005) [2023-12-27 07:38:58,124][20381] Updated weights for policy 1, policy_version 289089 (0.0006) [2023-12-27 07:38:58,432][20347] Updated weights for policy 0, policy_version 289354 (0.0006) [2023-12-27 07:38:58,489][20347] Updated weights for policy 0, policy_version 289364 (0.0008) [2023-12-27 07:38:58,554][20347] Updated weights for policy 0, policy_version 289374 (0.0008) [2023-12-27 07:38:58,836][20381] Updated weights for policy 1, policy_version 289099 (0.0009) [2023-12-27 07:38:58,909][20381] Updated weights for policy 1, policy_version 289109 (0.0008) [2023-12-27 07:38:58,967][20381] Updated weights for policy 1, policy_version 289119 (0.0007) [2023-12-27 07:38:59,324][20347] Updated weights for policy 0, policy_version 289384 (0.0009) [2023-12-27 07:38:59,390][20347] Updated weights for policy 0, policy_version 289394 (0.0010) [2023-12-27 07:38:59,444][20347] Updated weights for policy 0, policy_version 289404 (0.0010) [2023-12-27 07:38:59,677][20381] Updated weights for policy 1, policy_version 289129 (0.0007) [2023-12-27 07:38:59,741][20381] Updated weights for policy 1, policy_version 289139 (0.0007) [2023-12-27 07:38:59,801][20381] Updated weights for policy 1, policy_version 289149 (0.0007) [2023-12-27 07:38:59,866][20381] Updated weights for policy 1, policy_version 289159 (0.0007) [2023-12-27 07:39:00,122][20347] Updated weights for policy 0, policy_version 289414 (0.0008) [2023-12-27 07:39:00,181][20347] Updated weights for policy 0, policy_version 289424 (0.0008) [2023-12-27 07:39:00,243][20347] Updated weights for policy 0, policy_version 289434 (0.0006) [2023-12-27 07:39:00,559][20381] Updated weights for policy 1, policy_version 289169 (0.0008) [2023-12-27 07:39:00,610][20381] Updated weights for policy 1, policy_version 289179 (0.0007) [2023-12-27 07:39:00,665][20381] Updated weights for policy 1, policy_version 289189 (0.0008) [2023-12-27 07:39:00,846][20347] Updated weights for policy 0, policy_version 289444 (0.0006) [2023-12-27 07:39:00,904][20347] Updated weights for policy 0, policy_version 289454 (0.0010) [2023-12-27 07:39:00,958][20347] Updated weights for policy 0, policy_version 289464 (0.0008) [2023-12-27 07:39:01,391][20381] Updated weights for policy 1, policy_version 289199 (0.0007) [2023-12-27 07:39:01,454][20381] Updated weights for policy 1, policy_version 289209 (0.0009) [2023-12-27 07:39:01,514][20381] Updated weights for policy 1, policy_version 289219 (0.0007) [2023-12-27 07:39:01,624][20347] Updated weights for policy 0, policy_version 289474 (0.0005) [2023-12-27 07:39:01,682][20347] Updated weights for policy 0, policy_version 289484 (0.0007) [2023-12-27 07:39:01,742][20347] Updated weights for policy 0, policy_version 289494 (0.0008) [2023-12-27 07:39:01,794][20347] Updated weights for policy 0, policy_version 289504 (0.0008) [2023-12-27 07:39:02,199][20381] Updated weights for policy 1, policy_version 289229 (0.0007) [2023-12-27 07:39:02,260][20381] Updated weights for policy 1, policy_version 289239 (0.0008) [2023-12-27 07:39:02,314][20381] Updated weights for policy 1, policy_version 289249 (0.0008) [2023-12-27 07:39:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 148193280. Throughput: 0: 10479.7, 1: 10365.3. Samples: 148200900. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:02,363][19170] Avg episode reward: [(0, '961.480'), (1, '942.410')] [2023-12-27 07:39:02,461][20347] Updated weights for policy 0, policy_version 289514 (0.0007) [2023-12-27 07:39:02,521][20347] Updated weights for policy 0, policy_version 289524 (0.0006) [2023-12-27 07:39:02,578][20347] Updated weights for policy 0, policy_version 289534 (0.0005) [2023-12-27 07:39:03,067][20381] Updated weights for policy 1, policy_version 289259 (0.0008) [2023-12-27 07:39:03,118][20381] Updated weights for policy 1, policy_version 289269 (0.0009) [2023-12-27 07:39:03,164][20381] Updated weights for policy 1, policy_version 289279 (0.0007) [2023-12-27 07:39:03,170][20347] Updated weights for policy 0, policy_version 289544 (0.0006) [2023-12-27 07:39:03,228][20347] Updated weights for policy 0, policy_version 289554 (0.0007) [2023-12-27 07:39:03,287][20347] Updated weights for policy 0, policy_version 289564 (0.0005) [2023-12-27 07:39:03,851][20347] Updated weights for policy 0, policy_version 289574 (0.0007) [2023-12-27 07:39:03,858][20381] Updated weights for policy 1, policy_version 289289 (0.0007) [2023-12-27 07:39:03,902][20347] Updated weights for policy 0, policy_version 289584 (0.0006) [2023-12-27 07:39:03,916][20381] Updated weights for policy 1, policy_version 289299 (0.0007) [2023-12-27 07:39:03,952][20347] Updated weights for policy 0, policy_version 289594 (0.0006) [2023-12-27 07:39:03,976][20381] Updated weights for policy 1, policy_version 289309 (0.0006) [2023-12-27 07:39:04,030][20381] Updated weights for policy 1, policy_version 289319 (0.0007) [2023-12-27 07:39:04,662][20347] Updated weights for policy 0, policy_version 289604 (0.0007) [2023-12-27 07:39:04,718][20347] Updated weights for policy 0, policy_version 289614 (0.0007) [2023-12-27 07:39:04,737][20381] Updated weights for policy 1, policy_version 289329 (0.0007) [2023-12-27 07:39:04,774][20347] Updated weights for policy 0, policy_version 289624 (0.0007) [2023-12-27 07:39:04,795][20381] Updated weights for policy 1, policy_version 289339 (0.0007) [2023-12-27 07:39:04,856][20381] Updated weights for policy 1, policy_version 289349 (0.0009) [2023-12-27 07:39:05,350][20347] Updated weights for policy 0, policy_version 289634 (0.0005) [2023-12-27 07:39:05,410][20347] Updated weights for policy 0, policy_version 289644 (0.0007) [2023-12-27 07:39:05,472][20347] Updated weights for policy 0, policy_version 289654 (0.0009) [2023-12-27 07:39:05,531][20347] Updated weights for policy 0, policy_version 289664 (0.0006) [2023-12-27 07:39:05,565][20381] Updated weights for policy 1, policy_version 289359 (0.0006) [2023-12-27 07:39:05,611][20381] Updated weights for policy 1, policy_version 289369 (0.0005) [2023-12-27 07:39:05,665][20381] Updated weights for policy 1, policy_version 289379 (0.0005) [2023-12-27 07:39:06,170][20347] Updated weights for policy 0, policy_version 289674 (0.0008) [2023-12-27 07:39:06,233][20347] Updated weights for policy 0, policy_version 289684 (0.0008) [2023-12-27 07:39:06,235][20381] Updated weights for policy 1, policy_version 289389 (0.0007) [2023-12-27 07:39:06,296][20347] Updated weights for policy 0, policy_version 289694 (0.0006) [2023-12-27 07:39:06,297][20381] Updated weights for policy 1, policy_version 289399 (0.0007) [2023-12-27 07:39:06,360][20381] Updated weights for policy 1, policy_version 289409 (0.0008) [2023-12-27 07:39:06,937][20347] Updated weights for policy 0, policy_version 289704 (0.0006) [2023-12-27 07:39:06,985][20347] Updated weights for policy 0, policy_version 289714 (0.0006) [2023-12-27 07:39:07,008][20381] Updated weights for policy 1, policy_version 289419 (0.0007) [2023-12-27 07:39:07,043][20347] Updated weights for policy 0, policy_version 289724 (0.0007) [2023-12-27 07:39:07,064][20381] Updated weights for policy 1, policy_version 289429 (0.0005) [2023-12-27 07:39:07,122][20381] Updated weights for policy 1, policy_version 289439 (0.0005) [2023-12-27 07:39:07,362][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 148299776. Throughput: 0: 10538.1, 1: 10326.2. Samples: 148262692. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:07,363][19170] Avg episode reward: [(0, '942.260'), (1, '865.140')] [2023-12-27 07:39:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000289448_74113024.pth... [2023-12-27 07:39:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000289728_74186752.pth... [2023-12-27 07:39:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000288480_73867264.pth [2023-12-27 07:39:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000288232_73801728.pth [2023-12-27 07:39:07,658][20381] Updated weights for policy 1, policy_version 289449 (0.0006) [2023-12-27 07:39:07,701][20347] Updated weights for policy 0, policy_version 289734 (0.0007) [2023-12-27 07:39:07,712][20381] Updated weights for policy 1, policy_version 289459 (0.0008) [2023-12-27 07:39:07,764][20347] Updated weights for policy 0, policy_version 289744 (0.0008) [2023-12-27 07:39:07,771][20381] Updated weights for policy 1, policy_version 289469 (0.0006) [2023-12-27 07:39:07,819][20347] Updated weights for policy 0, policy_version 289754 (0.0007) [2023-12-27 07:39:07,834][20381] Updated weights for policy 1, policy_version 289479 (0.0007) [2023-12-27 07:39:08,507][20381] Updated weights for policy 1, policy_version 289489 (0.0011) [2023-12-27 07:39:08,527][20347] Updated weights for policy 0, policy_version 289764 (0.0006) [2023-12-27 07:39:08,564][20381] Updated weights for policy 1, policy_version 289499 (0.0011) [2023-12-27 07:39:08,586][20347] Updated weights for policy 0, policy_version 289774 (0.0008) [2023-12-27 07:39:08,615][20381] Updated weights for policy 1, policy_version 289509 (0.0011) [2023-12-27 07:39:08,651][20347] Updated weights for policy 0, policy_version 289784 (0.0006) [2023-12-27 07:39:09,236][20381] Updated weights for policy 1, policy_version 289519 (0.0011) [2023-12-27 07:39:09,293][20381] Updated weights for policy 1, policy_version 289529 (0.0011) [2023-12-27 07:39:09,312][20347] Updated weights for policy 0, policy_version 289794 (0.0007) [2023-12-27 07:39:09,354][20381] Updated weights for policy 1, policy_version 289539 (0.0011) [2023-12-27 07:39:09,382][20347] Updated weights for policy 0, policy_version 289804 (0.0006) [2023-12-27 07:39:09,438][20347] Updated weights for policy 0, policy_version 289814 (0.0008) [2023-12-27 07:39:09,484][20347] Updated weights for policy 0, policy_version 289824 (0.0007) [2023-12-27 07:39:10,089][20381] Updated weights for policy 1, policy_version 289549 (0.0009) [2023-12-27 07:39:10,149][20381] Updated weights for policy 1, policy_version 289559 (0.0008) [2023-12-27 07:39:10,205][20381] Updated weights for policy 1, policy_version 289569 (0.0008) [2023-12-27 07:39:10,225][20347] Updated weights for policy 0, policy_version 289834 (0.0010) [2023-12-27 07:39:10,282][20347] Updated weights for policy 0, policy_version 289844 (0.0007) [2023-12-27 07:39:10,342][20347] Updated weights for policy 0, policy_version 289854 (0.0007) [2023-12-27 07:39:10,886][20381] Updated weights for policy 1, policy_version 289579 (0.0009) [2023-12-27 07:39:10,902][20292] KL-divergence is very high: 362.6010 [2023-12-27 07:39:10,939][20381] Updated weights for policy 1, policy_version 289589 (0.0009) [2023-12-27 07:39:10,946][20292] KL-divergence is very high: 114.6551 [2023-12-27 07:39:10,988][20292] KL-divergence is very high: 184.9952 [2023-12-27 07:39:10,992][20381] Updated weights for policy 1, policy_version 289599 (0.0010) [2023-12-27 07:39:10,999][20347] Updated weights for policy 0, policy_version 289864 (0.0006) [2023-12-27 07:39:11,029][20292] KL-divergence is very high: 132.6675 [2023-12-27 07:39:11,063][20347] Updated weights for policy 0, policy_version 289874 (0.0007) [2023-12-27 07:39:11,122][20347] Updated weights for policy 0, policy_version 289884 (0.0007) [2023-12-27 07:39:11,780][20381] Updated weights for policy 1, policy_version 289609 (0.0010) [2023-12-27 07:39:11,811][20347] Updated weights for policy 0, policy_version 289894 (0.0009) [2023-12-27 07:39:11,840][20381] Updated weights for policy 1, policy_version 289619 (0.0007) [2023-12-27 07:39:11,873][20347] Updated weights for policy 0, policy_version 289904 (0.0011) [2023-12-27 07:39:11,902][20381] Updated weights for policy 1, policy_version 289629 (0.0006) [2023-12-27 07:39:11,927][20347] Updated weights for policy 0, policy_version 289914 (0.0009) [2023-12-27 07:39:11,958][20381] Updated weights for policy 1, policy_version 289639 (0.0008) [2023-12-27 07:39:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 148398080. Throughput: 0: 10609.4, 1: 10337.6. Samples: 148388396. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:12,363][19170] Avg episode reward: [(0, '943.080'), (1, '798.790')] [2023-12-27 07:39:12,522][20347] Updated weights for policy 0, policy_version 289924 (0.0007) [2023-12-27 07:39:12,577][20291] KL-divergence is very high: 131.1483 [2023-12-27 07:39:12,583][20347] Updated weights for policy 0, policy_version 289934 (0.0010) [2023-12-27 07:39:12,644][20347] Updated weights for policy 0, policy_version 289944 (0.0010) [2023-12-27 07:39:12,666][20381] Updated weights for policy 1, policy_version 289649 (0.0006) [2023-12-27 07:39:12,724][20381] Updated weights for policy 1, policy_version 289659 (0.0008) [2023-12-27 07:39:12,784][20381] Updated weights for policy 1, policy_version 289669 (0.0010) [2023-12-27 07:39:13,307][20347] Updated weights for policy 0, policy_version 289954 (0.0009) [2023-12-27 07:39:13,356][20347] Updated weights for policy 0, policy_version 289964 (0.0006) [2023-12-27 07:39:13,402][20347] Updated weights for policy 0, policy_version 289974 (0.0010) [2023-12-27 07:39:13,415][20381] Updated weights for policy 1, policy_version 289679 (0.0007) [2023-12-27 07:39:13,454][20347] Updated weights for policy 0, policy_version 289984 (0.0008) [2023-12-27 07:39:13,473][20381] Updated weights for policy 1, policy_version 289689 (0.0005) [2023-12-27 07:39:13,528][20381] Updated weights for policy 1, policy_version 289699 (0.0006) [2023-12-27 07:39:14,045][20381] Updated weights for policy 1, policy_version 289709 (0.0005) [2023-12-27 07:39:14,071][20347] Updated weights for policy 0, policy_version 289994 (0.0005) [2023-12-27 07:39:14,104][20381] Updated weights for policy 1, policy_version 289719 (0.0005) [2023-12-27 07:39:14,126][20347] Updated weights for policy 0, policy_version 290004 (0.0006) [2023-12-27 07:39:14,163][20381] Updated weights for policy 1, policy_version 289729 (0.0005) [2023-12-27 07:39:14,181][20347] Updated weights for policy 0, policy_version 290014 (0.0010) [2023-12-27 07:39:14,787][20381] Updated weights for policy 1, policy_version 289739 (0.0006) [2023-12-27 07:39:14,804][20347] Updated weights for policy 0, policy_version 290024 (0.0010) [2023-12-27 07:39:14,844][20381] Updated weights for policy 1, policy_version 289749 (0.0009) [2023-12-27 07:39:14,859][20347] Updated weights for policy 0, policy_version 290034 (0.0010) [2023-12-27 07:39:14,901][20381] Updated weights for policy 1, policy_version 289759 (0.0011) [2023-12-27 07:39:14,918][20347] Updated weights for policy 0, policy_version 290044 (0.0010) [2023-12-27 07:39:15,556][20381] Updated weights for policy 1, policy_version 289769 (0.0010) [2023-12-27 07:39:15,601][20347] Updated weights for policy 0, policy_version 290054 (0.0008) [2023-12-27 07:39:15,610][20381] Updated weights for policy 1, policy_version 289779 (0.0007) [2023-12-27 07:39:15,652][20347] Updated weights for policy 0, policy_version 290064 (0.0007) [2023-12-27 07:39:15,656][20381] Updated weights for policy 1, policy_version 289789 (0.0006) [2023-12-27 07:39:15,701][20381] Updated weights for policy 1, policy_version 289799 (0.0006) [2023-12-27 07:39:15,702][20347] Updated weights for policy 0, policy_version 290074 (0.0008) [2023-12-27 07:39:16,335][20347] Updated weights for policy 0, policy_version 290084 (0.0006) [2023-12-27 07:39:16,335][20292] KL-divergence is very high: 118.8370 [2023-12-27 07:39:16,369][20381] Updated weights for policy 1, policy_version 289809 (0.0007) [2023-12-27 07:39:16,381][20292] KL-divergence is very high: 107.0492 [2023-12-27 07:39:16,387][20347] Updated weights for policy 0, policy_version 290094 (0.0006) [2023-12-27 07:39:16,421][20381] Updated weights for policy 1, policy_version 289819 (0.0008) [2023-12-27 07:39:16,442][20347] Updated weights for policy 0, policy_version 290104 (0.0010) [2023-12-27 07:39:16,475][20381] Updated weights for policy 1, policy_version 289829 (0.0006) [2023-12-27 07:39:16,999][20347] Updated weights for policy 0, policy_version 290114 (0.0007) [2023-12-27 07:39:17,054][20347] Updated weights for policy 0, policy_version 290124 (0.0006) [2023-12-27 07:39:17,101][20347] Updated weights for policy 0, policy_version 290134 (0.0007) [2023-12-27 07:39:17,152][20347] Updated weights for policy 0, policy_version 290144 (0.0006) [2023-12-27 07:39:17,185][20381] Updated weights for policy 1, policy_version 289839 (0.0009) [2023-12-27 07:39:17,242][20381] Updated weights for policy 1, policy_version 289849 (0.0008) [2023-12-27 07:39:17,303][20381] Updated weights for policy 1, policy_version 289859 (0.0008) [2023-12-27 07:39:17,363][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 148512768. Throughput: 0: 10668.6, 1: 10360.1. Samples: 148516988. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:17,364][19170] Avg episode reward: [(0, '933.560'), (1, '818.680')] [2023-12-27 07:39:17,701][20347] Updated weights for policy 0, policy_version 290154 (0.0007) [2023-12-27 07:39:17,765][20347] Updated weights for policy 0, policy_version 290164 (0.0005) [2023-12-27 07:39:17,819][20347] Updated weights for policy 0, policy_version 290174 (0.0008) [2023-12-27 07:39:18,011][20381] Updated weights for policy 1, policy_version 289869 (0.0007) [2023-12-27 07:39:18,065][20381] Updated weights for policy 1, policy_version 289879 (0.0008) [2023-12-27 07:39:18,114][20381] Updated weights for policy 1, policy_version 289889 (0.0006) [2023-12-27 07:39:18,468][20347] Updated weights for policy 0, policy_version 290184 (0.0008) [2023-12-27 07:39:18,537][20347] Updated weights for policy 0, policy_version 290194 (0.0008) [2023-12-27 07:39:18,599][20347] Updated weights for policy 0, policy_version 290204 (0.0008) [2023-12-27 07:39:18,744][20381] Updated weights for policy 1, policy_version 289899 (0.0007) [2023-12-27 07:39:18,804][20381] Updated weights for policy 1, policy_version 289909 (0.0007) [2023-12-27 07:39:18,827][20292] KL-divergence is very high: 219.8637 [2023-12-27 07:39:18,860][20381] Updated weights for policy 1, policy_version 289919 (0.0008) [2023-12-27 07:39:19,308][20347] Updated weights for policy 0, policy_version 290214 (0.0008) [2023-12-27 07:39:19,367][20347] Updated weights for policy 0, policy_version 290224 (0.0008) [2023-12-27 07:39:19,426][20347] Updated weights for policy 0, policy_version 290234 (0.0007) [2023-12-27 07:39:19,515][20381] Updated weights for policy 1, policy_version 289929 (0.0008) [2023-12-27 07:39:19,571][20381] Updated weights for policy 1, policy_version 289939 (0.0008) [2023-12-27 07:39:19,629][20381] Updated weights for policy 1, policy_version 289949 (0.0007) [2023-12-27 07:39:19,683][20381] Updated weights for policy 1, policy_version 289959 (0.0008) [2023-12-27 07:39:20,150][20347] Updated weights for policy 0, policy_version 290244 (0.0008) [2023-12-27 07:39:20,204][20347] Updated weights for policy 0, policy_version 290254 (0.0008) [2023-12-27 07:39:20,260][20347] Updated weights for policy 0, policy_version 290264 (0.0007) [2023-12-27 07:39:20,381][20292] KL-divergence is very high: 210.5839 [2023-12-27 07:39:20,387][20292] KL-divergence is very high: 214.6218 [2023-12-27 07:39:20,392][20292] KL-divergence is very high: 136.2841 [2023-12-27 07:39:20,394][20381] Updated weights for policy 1, policy_version 289969 (0.0008) [2023-12-27 07:39:20,426][20292] KL-divergence is very high: 134.1446 [2023-12-27 07:39:20,434][20292] KL-divergence is very high: 160.4645 [2023-12-27 07:39:20,441][20292] KL-divergence is very high: 165.8145 [2023-12-27 07:39:20,447][20292] KL-divergence is very high: 150.2264 [2023-12-27 07:39:20,450][20381] Updated weights for policy 1, policy_version 289979 (0.0006) [2023-12-27 07:39:20,505][20381] Updated weights for policy 1, policy_version 289989 (0.0008) [2023-12-27 07:39:20,904][20347] Updated weights for policy 0, policy_version 290274 (0.0007) [2023-12-27 07:39:20,964][20347] Updated weights for policy 0, policy_version 290284 (0.0007) [2023-12-27 07:39:21,018][20347] Updated weights for policy 0, policy_version 290294 (0.0006) [2023-12-27 07:39:21,085][20347] Updated weights for policy 0, policy_version 290304 (0.0008) [2023-12-27 07:39:21,246][20381] Updated weights for policy 1, policy_version 289999 (0.0007) [2023-12-27 07:39:21,306][20381] Updated weights for policy 1, policy_version 290009 (0.0007) [2023-12-27 07:39:21,367][20381] Updated weights for policy 1, policy_version 290019 (0.0007) [2023-12-27 07:39:21,774][20347] Updated weights for policy 0, policy_version 290314 (0.0007) [2023-12-27 07:39:21,833][20347] Updated weights for policy 0, policy_version 290324 (0.0007) [2023-12-27 07:39:21,898][20347] Updated weights for policy 0, policy_version 290334 (0.0007) [2023-12-27 07:39:22,020][20381] Updated weights for policy 1, policy_version 290029 (0.0008) [2023-12-27 07:39:22,072][20381] Updated weights for policy 1, policy_version 290039 (0.0007) [2023-12-27 07:39:22,146][20381] Updated weights for policy 1, policy_version 290049 (0.0008) [2023-12-27 07:39:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 148611072. Throughput: 0: 10609.5, 1: 10366.2. Samples: 148578704. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:22,363][19170] Avg episode reward: [(0, '923.360'), (1, '738.910')] [2023-12-27 07:39:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000290056_74268672.pth... [2023-12-27 07:39:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000290336_74342400.pth... [2023-12-27 07:39:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000288840_73957376.pth [2023-12-27 07:39:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000289120_74031104.pth [2023-12-27 07:39:22,587][20347] Updated weights for policy 0, policy_version 290344 (0.0005) [2023-12-27 07:39:22,648][20347] Updated weights for policy 0, policy_version 290354 (0.0005) [2023-12-27 07:39:22,709][20347] Updated weights for policy 0, policy_version 290364 (0.0009) [2023-12-27 07:39:22,853][20381] Updated weights for policy 1, policy_version 290059 (0.0007) [2023-12-27 07:39:22,906][20381] Updated weights for policy 1, policy_version 290069 (0.0005) [2023-12-27 07:39:22,966][20381] Updated weights for policy 1, policy_version 290079 (0.0007) [2023-12-27 07:39:23,350][20347] Updated weights for policy 0, policy_version 290374 (0.0011) [2023-12-27 07:39:23,396][20347] Updated weights for policy 0, policy_version 290384 (0.0008) [2023-12-27 07:39:23,452][20347] Updated weights for policy 0, policy_version 290394 (0.0005) [2023-12-27 07:39:23,629][20381] Updated weights for policy 1, policy_version 290089 (0.0007) [2023-12-27 07:39:23,683][20381] Updated weights for policy 1, policy_version 290099 (0.0007) [2023-12-27 07:39:23,741][20381] Updated weights for policy 1, policy_version 290109 (0.0007) [2023-12-27 07:39:23,799][20381] Updated weights for policy 1, policy_version 290119 (0.0007) [2023-12-27 07:39:24,099][20347] Updated weights for policy 0, policy_version 290404 (0.0006) [2023-12-27 07:39:24,164][20347] Updated weights for policy 0, policy_version 290414 (0.0008) [2023-12-27 07:39:24,213][20347] Updated weights for policy 0, policy_version 290424 (0.0010) [2023-12-27 07:39:24,444][20381] Updated weights for policy 1, policy_version 290129 (0.0007) [2023-12-27 07:39:24,504][20381] Updated weights for policy 1, policy_version 290139 (0.0006) [2023-12-27 07:39:24,567][20381] Updated weights for policy 1, policy_version 290149 (0.0005) [2023-12-27 07:39:24,833][20347] Updated weights for policy 0, policy_version 290434 (0.0005) [2023-12-27 07:39:24,891][20347] Updated weights for policy 0, policy_version 290444 (0.0005) [2023-12-27 07:39:24,955][20347] Updated weights for policy 0, policy_version 290454 (0.0008) [2023-12-27 07:39:25,014][20347] Updated weights for policy 0, policy_version 290464 (0.0008) [2023-12-27 07:39:25,167][20381] Updated weights for policy 1, policy_version 290159 (0.0006) [2023-12-27 07:39:25,215][20381] Updated weights for policy 1, policy_version 290169 (0.0007) [2023-12-27 07:39:25,270][20381] Updated weights for policy 1, policy_version 290179 (0.0008) [2023-12-27 07:39:25,653][20347] Updated weights for policy 0, policy_version 290474 (0.0008) [2023-12-27 07:39:25,715][20347] Updated weights for policy 0, policy_version 290484 (0.0008) [2023-12-27 07:39:25,777][20347] Updated weights for policy 0, policy_version 290494 (0.0009) [2023-12-27 07:39:25,898][20381] Updated weights for policy 1, policy_version 290189 (0.0006) [2023-12-27 07:39:25,952][20381] Updated weights for policy 1, policy_version 290199 (0.0005) [2023-12-27 07:39:26,013][20381] Updated weights for policy 1, policy_version 290209 (0.0005) [2023-12-27 07:39:26,365][20347] Updated weights for policy 0, policy_version 290504 (0.0008) [2023-12-27 07:39:26,418][20347] Updated weights for policy 0, policy_version 290514 (0.0007) [2023-12-27 07:39:26,472][20347] Updated weights for policy 0, policy_version 290524 (0.0007) [2023-12-27 07:39:26,587][20381] Updated weights for policy 1, policy_version 290219 (0.0006) [2023-12-27 07:39:26,637][20381] Updated weights for policy 1, policy_version 290229 (0.0008) [2023-12-27 07:39:26,694][20381] Updated weights for policy 1, policy_version 290239 (0.0007) [2023-12-27 07:39:27,111][20347] Updated weights for policy 0, policy_version 290534 (0.0007) [2023-12-27 07:39:27,164][20347] Updated weights for policy 0, policy_version 290544 (0.0007) [2023-12-27 07:39:27,221][20347] Updated weights for policy 0, policy_version 290554 (0.0007) [2023-12-27 07:39:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 148717568. Throughput: 0: 10612.1, 1: 10381.8. Samples: 148704372. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:27,363][19170] Avg episode reward: [(0, '971.350'), (1, '541.160')] [2023-12-27 07:39:27,402][20381] Updated weights for policy 1, policy_version 290249 (0.0007) [2023-12-27 07:39:27,448][20381] Updated weights for policy 1, policy_version 290259 (0.0010) [2023-12-27 07:39:27,495][20381] Updated weights for policy 1, policy_version 290269 (0.0008) [2023-12-27 07:39:27,539][20381] Updated weights for policy 1, policy_version 290279 (0.0007) [2023-12-27 07:39:27,827][20347] Updated weights for policy 0, policy_version 290564 (0.0008) [2023-12-27 07:39:27,888][20347] Updated weights for policy 0, policy_version 290574 (0.0008) [2023-12-27 07:39:27,944][20347] Updated weights for policy 0, policy_version 290584 (0.0008) [2023-12-27 07:39:28,313][20381] Updated weights for policy 1, policy_version 290289 (0.0009) [2023-12-27 07:39:28,372][20381] Updated weights for policy 1, policy_version 290299 (0.0007) [2023-12-27 07:39:28,426][20381] Updated weights for policy 1, policy_version 290309 (0.0008) [2023-12-27 07:39:28,532][20347] Updated weights for policy 0, policy_version 290594 (0.0007) [2023-12-27 07:39:28,591][20347] Updated weights for policy 0, policy_version 290604 (0.0007) [2023-12-27 07:39:28,648][20347] Updated weights for policy 0, policy_version 290614 (0.0008) [2023-12-27 07:39:28,705][20347] Updated weights for policy 0, policy_version 290624 (0.0007) [2023-12-27 07:39:29,133][20381] Updated weights for policy 1, policy_version 290319 (0.0008) [2023-12-27 07:39:29,195][20381] Updated weights for policy 1, policy_version 290329 (0.0008) [2023-12-27 07:39:29,244][20381] Updated weights for policy 1, policy_version 290339 (0.0007) [2023-12-27 07:39:29,394][20347] Updated weights for policy 0, policy_version 290634 (0.0009) [2023-12-27 07:39:29,458][20347] Updated weights for policy 0, policy_version 290644 (0.0010) [2023-12-27 07:39:29,523][20347] Updated weights for policy 0, policy_version 290654 (0.0006) [2023-12-27 07:39:29,951][20381] Updated weights for policy 1, policy_version 290349 (0.0006) [2023-12-27 07:39:30,008][20381] Updated weights for policy 1, policy_version 290359 (0.0008) [2023-12-27 07:39:30,069][20381] Updated weights for policy 1, policy_version 290369 (0.0007) [2023-12-27 07:39:30,178][20347] Updated weights for policy 0, policy_version 290664 (0.0010) [2023-12-27 07:39:30,238][20347] Updated weights for policy 0, policy_version 290674 (0.0008) [2023-12-27 07:39:30,294][20347] Updated weights for policy 0, policy_version 290684 (0.0007) [2023-12-27 07:39:30,710][20381] Updated weights for policy 1, policy_version 290379 (0.0008) [2023-12-27 07:39:30,762][20381] Updated weights for policy 1, policy_version 290389 (0.0007) [2023-12-27 07:39:30,821][20381] Updated weights for policy 1, policy_version 290399 (0.0008) [2023-12-27 07:39:30,917][20347] Updated weights for policy 0, policy_version 290694 (0.0006) [2023-12-27 07:39:30,978][20347] Updated weights for policy 0, policy_version 290704 (0.0005) [2023-12-27 07:39:31,043][20347] Updated weights for policy 0, policy_version 290714 (0.0008) [2023-12-27 07:39:31,582][20381] Updated weights for policy 1, policy_version 290409 (0.0008) [2023-12-27 07:39:31,645][20381] Updated weights for policy 1, policy_version 290419 (0.0008) [2023-12-27 07:39:31,703][20381] Updated weights for policy 1, policy_version 290429 (0.0006) [2023-12-27 07:39:31,704][20292] KL-divergence is very high: 243.4980 [2023-12-27 07:39:31,717][20292] KL-divergence is very high: 157.8354 [2023-12-27 07:39:31,735][20347] Updated weights for policy 0, policy_version 290724 (0.0007) [2023-12-27 07:39:31,756][20292] KL-divergence is very high: 206.2390 [2023-12-27 07:39:31,767][20381] Updated weights for policy 1, policy_version 290439 (0.0008) [2023-12-27 07:39:31,785][20347] Updated weights for policy 0, policy_version 290734 (0.0007) [2023-12-27 07:39:31,802][20291] KL-divergence is very high: 228.5456 [2023-12-27 07:39:31,840][20347] Updated weights for policy 0, policy_version 290744 (0.0008) [2023-12-27 07:39:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 148815872. Throughput: 0: 10575.2, 1: 10345.7. Samples: 148828536. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:32,363][19170] Avg episode reward: [(0, '981.110'), (1, '662.050')] [2023-12-27 07:39:32,408][20381] Updated weights for policy 1, policy_version 290449 (0.0007) [2023-12-27 07:39:32,470][20381] Updated weights for policy 1, policy_version 290459 (0.0008) [2023-12-27 07:39:32,493][20347] Updated weights for policy 0, policy_version 290754 (0.0007) [2023-12-27 07:39:32,530][20381] Updated weights for policy 1, policy_version 290469 (0.0007) [2023-12-27 07:39:32,550][20347] Updated weights for policy 0, policy_version 290764 (0.0008) [2023-12-27 07:39:32,603][20347] Updated weights for policy 0, policy_version 290774 (0.0008) [2023-12-27 07:39:32,655][20347] Updated weights for policy 0, policy_version 290784 (0.0008) [2023-12-27 07:39:33,186][20381] Updated weights for policy 1, policy_version 290479 (0.0006) [2023-12-27 07:39:33,248][20381] Updated weights for policy 1, policy_version 290489 (0.0005) [2023-12-27 07:39:33,304][20381] Updated weights for policy 1, policy_version 290499 (0.0008) [2023-12-27 07:39:33,375][20347] Updated weights for policy 0, policy_version 290794 (0.0010) [2023-12-27 07:39:33,420][20347] Updated weights for policy 0, policy_version 290804 (0.0010) [2023-12-27 07:39:33,465][20347] Updated weights for policy 0, policy_version 290814 (0.0010) [2023-12-27 07:39:33,915][20381] Updated weights for policy 1, policy_version 290509 (0.0007) [2023-12-27 07:39:33,961][20381] Updated weights for policy 1, policy_version 290519 (0.0008) [2023-12-27 07:39:34,010][20381] Updated weights for policy 1, policy_version 290529 (0.0007) [2023-12-27 07:39:34,177][20347] Updated weights for policy 0, policy_version 290824 (0.0009) [2023-12-27 07:39:34,233][20347] Updated weights for policy 0, policy_version 290834 (0.0008) [2023-12-27 07:39:34,290][20347] Updated weights for policy 0, policy_version 290844 (0.0008) [2023-12-27 07:39:34,770][20381] Updated weights for policy 1, policy_version 290539 (0.0007) [2023-12-27 07:39:34,824][20381] Updated weights for policy 1, policy_version 290549 (0.0008) [2023-12-27 07:39:34,869][20381] Updated weights for policy 1, policy_version 290559 (0.0007) [2023-12-27 07:39:34,918][20347] Updated weights for policy 0, policy_version 290854 (0.0007) [2023-12-27 07:39:34,975][20347] Updated weights for policy 0, policy_version 290864 (0.0007) [2023-12-27 07:39:35,034][20347] Updated weights for policy 0, policy_version 290874 (0.0008) [2023-12-27 07:39:35,557][20381] Updated weights for policy 1, policy_version 290569 (0.0008) [2023-12-27 07:39:35,615][20381] Updated weights for policy 1, policy_version 290579 (0.0007) [2023-12-27 07:39:35,677][20381] Updated weights for policy 1, policy_version 290589 (0.0008) [2023-12-27 07:39:35,698][20347] Updated weights for policy 0, policy_version 290884 (0.0007) [2023-12-27 07:39:35,733][20381] Updated weights for policy 1, policy_version 290599 (0.0006) [2023-12-27 07:39:35,755][20347] Updated weights for policy 0, policy_version 290894 (0.0006) [2023-12-27 07:39:35,817][20347] Updated weights for policy 0, policy_version 290904 (0.0008) [2023-12-27 07:39:36,438][20381] Updated weights for policy 1, policy_version 290609 (0.0008) [2023-12-27 07:39:36,495][20347] Updated weights for policy 0, policy_version 290914 (0.0007) [2023-12-27 07:39:36,497][20381] Updated weights for policy 1, policy_version 290619 (0.0007) [2023-12-27 07:39:36,553][20347] Updated weights for policy 0, policy_version 290924 (0.0007) [2023-12-27 07:39:36,559][20381] Updated weights for policy 1, policy_version 290629 (0.0006) [2023-12-27 07:39:36,609][20347] Updated weights for policy 0, policy_version 290934 (0.0007) [2023-12-27 07:39:36,654][20347] Updated weights for policy 0, policy_version 290944 (0.0007) [2023-12-27 07:39:37,240][20381] Updated weights for policy 1, policy_version 290639 (0.0006) [2023-12-27 07:39:37,304][20381] Updated weights for policy 1, policy_version 290649 (0.0011) [2023-12-27 07:39:37,317][20347] Updated weights for policy 0, policy_version 290954 (0.0006) [2023-12-27 07:39:37,359][20381] Updated weights for policy 1, policy_version 290659 (0.0007) [2023-12-27 07:39:37,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 148914176. Throughput: 0: 10567.3, 1: 10317.8. Samples: 148890156. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:37,364][19170] Avg episode reward: [(0, '990.700'), (1, '612.180')] [2023-12-27 07:39:37,369][20347] Updated weights for policy 0, policy_version 290964 (0.0007) [2023-12-27 07:39:37,387][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000290664_74424320.pth... [2023-12-27 07:39:37,391][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000289448_74113024.pth [2023-12-27 07:39:37,423][20347] Updated weights for policy 0, policy_version 290974 (0.0006) [2023-12-27 07:39:37,432][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000290976_74506240.pth... [2023-12-27 07:39:37,436][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000289728_74186752.pth [2023-12-27 07:39:38,031][20381] Updated weights for policy 1, policy_version 290669 (0.0006) [2023-12-27 07:39:38,054][20292] KL-divergence is very high: 114.0522 [2023-12-27 07:39:38,084][20381] Updated weights for policy 1, policy_version 290679 (0.0006) [2023-12-27 07:39:38,108][20347] Updated weights for policy 0, policy_version 290984 (0.0007) [2023-12-27 07:39:38,137][20381] Updated weights for policy 1, policy_version 290689 (0.0005) [2023-12-27 07:39:38,168][20347] Updated weights for policy 0, policy_version 290994 (0.0009) [2023-12-27 07:39:38,220][20347] Updated weights for policy 0, policy_version 291004 (0.0008) [2023-12-27 07:39:38,722][20381] Updated weights for policy 1, policy_version 290699 (0.0006) [2023-12-27 07:39:38,777][20381] Updated weights for policy 1, policy_version 290709 (0.0008) [2023-12-27 07:39:38,829][20381] Updated weights for policy 1, policy_version 290719 (0.0008) [2023-12-27 07:39:38,851][20347] Updated weights for policy 0, policy_version 291014 (0.0007) [2023-12-27 07:39:38,903][20347] Updated weights for policy 0, policy_version 291024 (0.0009) [2023-12-27 07:39:38,953][20347] Updated weights for policy 0, policy_version 291034 (0.0007) [2023-12-27 07:39:39,544][20381] Updated weights for policy 1, policy_version 290729 (0.0008) [2023-12-27 07:39:39,607][20381] Updated weights for policy 1, policy_version 290739 (0.0005) [2023-12-27 07:39:39,670][20381] Updated weights for policy 1, policy_version 290749 (0.0005) [2023-12-27 07:39:39,683][20347] Updated weights for policy 0, policy_version 291044 (0.0006) [2023-12-27 07:39:39,736][20347] Updated weights for policy 0, policy_version 291054 (0.0008) [2023-12-27 07:39:39,737][20381] Updated weights for policy 1, policy_version 290759 (0.0006) [2023-12-27 07:39:39,799][20347] Updated weights for policy 0, policy_version 291064 (0.0008) [2023-12-27 07:39:40,286][20381] Updated weights for policy 1, policy_version 290769 (0.0006) [2023-12-27 07:39:40,344][20381] Updated weights for policy 1, policy_version 290779 (0.0005) [2023-12-27 07:39:40,406][20381] Updated weights for policy 1, policy_version 290789 (0.0008) [2023-12-27 07:39:40,524][20347] Updated weights for policy 0, policy_version 291074 (0.0008) [2023-12-27 07:39:40,580][20347] Updated weights for policy 0, policy_version 291084 (0.0007) [2023-12-27 07:39:40,632][20347] Updated weights for policy 0, policy_version 291094 (0.0006) [2023-12-27 07:39:40,687][20347] Updated weights for policy 0, policy_version 291104 (0.0007) [2023-12-27 07:39:41,087][20381] Updated weights for policy 1, policy_version 290799 (0.0007) [2023-12-27 07:39:41,151][20381] Updated weights for policy 1, policy_version 290809 (0.0007) [2023-12-27 07:39:41,199][20381] Updated weights for policy 1, policy_version 290819 (0.0007) [2023-12-27 07:39:41,336][20347] Updated weights for policy 0, policy_version 291114 (0.0008) [2023-12-27 07:39:41,408][20347] Updated weights for policy 0, policy_version 291124 (0.0009) [2023-12-27 07:39:41,458][20347] Updated weights for policy 0, policy_version 291134 (0.0008) [2023-12-27 07:39:41,923][20381] Updated weights for policy 1, policy_version 290829 (0.0007) [2023-12-27 07:39:41,979][20381] Updated weights for policy 1, policy_version 290839 (0.0008) [2023-12-27 07:39:42,036][20381] Updated weights for policy 1, policy_version 290849 (0.0006) [2023-12-27 07:39:42,113][20347] Updated weights for policy 0, policy_version 291144 (0.0007) [2023-12-27 07:39:42,173][20347] Updated weights for policy 0, policy_version 291154 (0.0007) [2023-12-27 07:39:42,227][20347] Updated weights for policy 0, policy_version 291164 (0.0006) [2023-12-27 07:39:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 149028864. Throughput: 0: 10519.2, 1: 10327.6. Samples: 149014236. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:42,363][19170] Avg episode reward: [(0, '990.520'), (1, '737.270')] [2023-12-27 07:39:42,752][20381] Updated weights for policy 1, policy_version 290859 (0.0007) [2023-12-27 07:39:42,817][20381] Updated weights for policy 1, policy_version 290869 (0.0008) [2023-12-27 07:39:42,850][20347] Updated weights for policy 0, policy_version 291174 (0.0007) [2023-12-27 07:39:42,875][20381] Updated weights for policy 1, policy_version 290879 (0.0006) [2023-12-27 07:39:42,904][20347] Updated weights for policy 0, policy_version 291184 (0.0006) [2023-12-27 07:39:42,958][20347] Updated weights for policy 0, policy_version 291194 (0.0007) [2023-12-27 07:39:43,523][20347] Updated weights for policy 0, policy_version 291204 (0.0006) [2023-12-27 07:39:43,532][20381] Updated weights for policy 1, policy_version 290889 (0.0007) [2023-12-27 07:39:43,571][20347] Updated weights for policy 0, policy_version 291214 (0.0006) [2023-12-27 07:39:43,584][20381] Updated weights for policy 1, policy_version 290899 (0.0007) [2023-12-27 07:39:43,619][20347] Updated weights for policy 0, policy_version 291224 (0.0005) [2023-12-27 07:39:43,632][20381] Updated weights for policy 1, policy_version 290909 (0.0008) [2023-12-27 07:39:43,683][20381] Updated weights for policy 1, policy_version 290919 (0.0007) [2023-12-27 07:39:44,278][20347] Updated weights for policy 0, policy_version 291234 (0.0006) [2023-12-27 07:39:44,315][20381] Updated weights for policy 1, policy_version 290929 (0.0011) [2023-12-27 07:39:44,337][20347] Updated weights for policy 0, policy_version 291244 (0.0007) [2023-12-27 07:39:44,382][20381] Updated weights for policy 1, policy_version 290939 (0.0011) [2023-12-27 07:39:44,392][20347] Updated weights for policy 0, policy_version 291254 (0.0006) [2023-12-27 07:39:44,443][20381] Updated weights for policy 1, policy_version 290949 (0.0008) [2023-12-27 07:39:44,452][20347] Updated weights for policy 0, policy_version 291264 (0.0010) [2023-12-27 07:39:45,131][20381] Updated weights for policy 1, policy_version 290959 (0.0010) [2023-12-27 07:39:45,135][20347] Updated weights for policy 0, policy_version 291274 (0.0011) [2023-12-27 07:39:45,191][20381] Updated weights for policy 1, policy_version 290969 (0.0011) [2023-12-27 07:39:45,197][20347] Updated weights for policy 0, policy_version 291284 (0.0010) [2023-12-27 07:39:45,250][20381] Updated weights for policy 1, policy_version 290979 (0.0011) [2023-12-27 07:39:45,252][20347] Updated weights for policy 0, policy_version 291294 (0.0010) [2023-12-27 07:39:45,890][20381] Updated weights for policy 1, policy_version 290989 (0.0011) [2023-12-27 07:39:45,926][20347] Updated weights for policy 0, policy_version 291304 (0.0010) [2023-12-27 07:39:45,940][20381] Updated weights for policy 1, policy_version 290999 (0.0011) [2023-12-27 07:39:45,974][20347] Updated weights for policy 0, policy_version 291314 (0.0010) [2023-12-27 07:39:45,990][20381] Updated weights for policy 1, policy_version 291009 (0.0007) [2023-12-27 07:39:46,027][20347] Updated weights for policy 0, policy_version 291324 (0.0010) [2023-12-27 07:39:46,635][20381] Updated weights for policy 1, policy_version 291019 (0.0009) [2023-12-27 07:39:46,698][20381] Updated weights for policy 1, policy_version 291029 (0.0006) [2023-12-27 07:39:46,727][20347] Updated weights for policy 0, policy_version 291334 (0.0010) [2023-12-27 07:39:46,751][20381] Updated weights for policy 1, policy_version 291039 (0.0006) [2023-12-27 07:39:46,778][20347] Updated weights for policy 0, policy_version 291344 (0.0010) [2023-12-27 07:39:46,831][20347] Updated weights for policy 0, policy_version 291354 (0.0010) [2023-12-27 07:39:47,359][20381] Updated weights for policy 1, policy_version 291049 (0.0007) [2023-12-27 07:39:47,363][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 149127168. Throughput: 0: 10521.4, 1: 10339.1. Samples: 149139632. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:47,364][19170] Avg episode reward: [(0, '990.520'), (1, '743.770')] [2023-12-27 07:39:47,411][20381] Updated weights for policy 1, policy_version 291059 (0.0007) [2023-12-27 07:39:47,459][20381] Updated weights for policy 1, policy_version 291069 (0.0007) [2023-12-27 07:39:47,513][20381] Updated weights for policy 1, policy_version 291079 (0.0007) [2023-12-27 07:39:47,519][20347] Updated weights for policy 0, policy_version 291364 (0.0010) [2023-12-27 07:39:47,570][20347] Updated weights for policy 0, policy_version 291374 (0.0010) [2023-12-27 07:39:47,619][20347] Updated weights for policy 0, policy_version 291384 (0.0010) [2023-12-27 07:39:48,183][20381] Updated weights for policy 1, policy_version 291089 (0.0009) [2023-12-27 07:39:48,236][20381] Updated weights for policy 1, policy_version 291099 (0.0007) [2023-12-27 07:39:48,269][20347] Updated weights for policy 0, policy_version 291394 (0.0010) [2023-12-27 07:39:48,289][20381] Updated weights for policy 1, policy_version 291109 (0.0011) [2023-12-27 07:39:48,328][20347] Updated weights for policy 0, policy_version 291404 (0.0010) [2023-12-27 07:39:48,386][20347] Updated weights for policy 0, policy_version 291414 (0.0006) [2023-12-27 07:39:48,447][20347] Updated weights for policy 0, policy_version 291424 (0.0008) [2023-12-27 07:39:48,942][20381] Updated weights for policy 1, policy_version 291119 (0.0007) [2023-12-27 07:39:48,997][20381] Updated weights for policy 1, policy_version 291129 (0.0006) [2023-12-27 07:39:49,061][20381] Updated weights for policy 1, policy_version 291139 (0.0006) [2023-12-27 07:39:49,127][20347] Updated weights for policy 0, policy_version 291434 (0.0008) [2023-12-27 07:39:49,178][20347] Updated weights for policy 0, policy_version 291444 (0.0007) [2023-12-27 07:39:49,238][20347] Updated weights for policy 0, policy_version 291454 (0.0008) [2023-12-27 07:39:49,699][20381] Updated weights for policy 1, policy_version 291149 (0.0009) [2023-12-27 07:39:49,758][20381] Updated weights for policy 1, policy_version 291159 (0.0011) [2023-12-27 07:39:49,810][20292] KL-divergence is very high: 500.8912 [2023-12-27 07:39:49,816][20292] KL-divergence is very high: 440.2186 [2023-12-27 07:39:49,824][20292] KL-divergence is very high: 401.9308 [2023-12-27 07:39:49,824][20381] Updated weights for policy 1, policy_version 291169 (0.0010) [2023-12-27 07:39:49,860][20292] KL-divergence is very high: 162.3556 [2023-12-27 07:39:49,934][20347] Updated weights for policy 0, policy_version 291464 (0.0010) [2023-12-27 07:39:49,998][20347] Updated weights for policy 0, policy_version 291474 (0.0011) [2023-12-27 07:39:50,056][20347] Updated weights for policy 0, policy_version 291484 (0.0010) [2023-12-27 07:39:50,491][20381] Updated weights for policy 1, policy_version 291179 (0.0009) [2023-12-27 07:39:50,549][20381] Updated weights for policy 1, policy_version 291189 (0.0007) [2023-12-27 07:39:50,606][20381] Updated weights for policy 1, policy_version 291199 (0.0007) [2023-12-27 07:39:50,697][20347] Updated weights for policy 0, policy_version 291494 (0.0010) [2023-12-27 07:39:50,750][20347] Updated weights for policy 0, policy_version 291504 (0.0011) [2023-12-27 07:39:50,802][20347] Updated weights for policy 0, policy_version 291514 (0.0010) [2023-12-27 07:39:51,136][20381] Updated weights for policy 1, policy_version 291209 (0.0007) [2023-12-27 07:39:51,181][20381] Updated weights for policy 1, policy_version 291219 (0.0011) [2023-12-27 07:39:51,238][20381] Updated weights for policy 1, policy_version 291229 (0.0011) [2023-12-27 07:39:51,301][20381] Updated weights for policy 1, policy_version 291239 (0.0008) [2023-12-27 07:39:51,556][20347] Updated weights for policy 0, policy_version 291524 (0.0009) [2023-12-27 07:39:51,619][20347] Updated weights for policy 0, policy_version 291534 (0.0007) [2023-12-27 07:39:51,676][20347] Updated weights for policy 0, policy_version 291544 (0.0005) [2023-12-27 07:39:51,957][20381] Updated weights for policy 1, policy_version 291249 (0.0006) [2023-12-27 07:39:52,018][20381] Updated weights for policy 1, policy_version 291259 (0.0009) [2023-12-27 07:39:52,074][20381] Updated weights for policy 1, policy_version 291269 (0.0011) [2023-12-27 07:39:52,332][20347] Updated weights for policy 0, policy_version 291554 (0.0007) [2023-12-27 07:39:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 149233664. Throughput: 0: 10480.5, 1: 10386.2. Samples: 149201692. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '768.580')] [2023-12-27 07:39:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000291272_74579968.pth... [2023-12-27 07:39:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000290056_74268672.pth [2023-12-27 07:39:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000291272_74579968.pth [2023-12-27 07:39:52,392][20347] Updated weights for policy 0, policy_version 291564 (0.0008) [2023-12-27 07:39:52,454][20347] Updated weights for policy 0, policy_version 291574 (0.0008) [2023-12-27 07:39:52,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000291584_74661888.pth... [2023-12-27 07:39:52,504][20347] Updated weights for policy 0, policy_version 291584 (0.0007) [2023-12-27 07:39:52,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000290336_74342400.pth [2023-12-27 07:39:52,507][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000291584_74661888.pth [2023-12-27 07:39:52,770][20381] Updated weights for policy 1, policy_version 291279 (0.0009) [2023-12-27 07:39:52,818][20381] Updated weights for policy 1, policy_version 291289 (0.0008) [2023-12-27 07:39:52,865][20381] Updated weights for policy 1, policy_version 291299 (0.0009) [2023-12-27 07:39:53,189][20347] Updated weights for policy 0, policy_version 291594 (0.0009) [2023-12-27 07:39:53,243][20347] Updated weights for policy 0, policy_version 291604 (0.0008) [2023-12-27 07:39:53,300][20347] Updated weights for policy 0, policy_version 291614 (0.0007) [2023-12-27 07:39:53,563][20381] Updated weights for policy 1, policy_version 291309 (0.0008) [2023-12-27 07:39:53,629][20381] Updated weights for policy 1, policy_version 291319 (0.0007) [2023-12-27 07:39:53,695][20381] Updated weights for policy 1, policy_version 291329 (0.0007) [2023-12-27 07:39:54,001][20347] Updated weights for policy 0, policy_version 291624 (0.0006) [2023-12-27 07:39:54,060][20347] Updated weights for policy 0, policy_version 291634 (0.0005) [2023-12-27 07:39:54,119][20347] Updated weights for policy 0, policy_version 291644 (0.0005) [2023-12-27 07:39:54,371][20381] Updated weights for policy 1, policy_version 291339 (0.0007) [2023-12-27 07:39:54,435][20381] Updated weights for policy 1, policy_version 291349 (0.0007) [2023-12-27 07:39:54,492][20381] Updated weights for policy 1, policy_version 291359 (0.0010) [2023-12-27 07:39:54,764][20347] Updated weights for policy 0, policy_version 291654 (0.0010) [2023-12-27 07:39:54,827][20347] Updated weights for policy 0, policy_version 291664 (0.0010) [2023-12-27 07:39:54,889][20347] Updated weights for policy 0, policy_version 291674 (0.0008) [2023-12-27 07:39:55,052][20381] Updated weights for policy 1, policy_version 291369 (0.0010) [2023-12-27 07:39:55,101][20381] Updated weights for policy 1, policy_version 291379 (0.0005) [2023-12-27 07:39:55,152][20381] Updated weights for policy 1, policy_version 291389 (0.0005) [2023-12-27 07:39:55,202][20381] Updated weights for policy 1, policy_version 291399 (0.0005) [2023-12-27 07:39:55,580][20347] Updated weights for policy 0, policy_version 291684 (0.0010) [2023-12-27 07:39:55,638][20347] Updated weights for policy 0, policy_version 291694 (0.0010) [2023-12-27 07:39:55,695][20347] Updated weights for policy 0, policy_version 291704 (0.0008) [2023-12-27 07:39:55,798][20381] Updated weights for policy 1, policy_version 291409 (0.0010) [2023-12-27 07:39:55,847][20381] Updated weights for policy 1, policy_version 291419 (0.0010) [2023-12-27 07:39:55,905][20381] Updated weights for policy 1, policy_version 291429 (0.0010) [2023-12-27 07:39:56,314][20347] Updated weights for policy 0, policy_version 291714 (0.0006) [2023-12-27 07:39:56,374][20347] Updated weights for policy 0, policy_version 291724 (0.0010) [2023-12-27 07:39:56,430][20347] Updated weights for policy 0, policy_version 291734 (0.0010) [2023-12-27 07:39:56,484][20347] Updated weights for policy 0, policy_version 291744 (0.0010) [2023-12-27 07:39:56,605][20381] Updated weights for policy 1, policy_version 291439 (0.0008) [2023-12-27 07:39:56,654][20381] Updated weights for policy 1, policy_version 291449 (0.0005) [2023-12-27 07:39:56,707][20381] Updated weights for policy 1, policy_version 291459 (0.0005) [2023-12-27 07:39:57,103][20347] Updated weights for policy 0, policy_version 291754 (0.0006) [2023-12-27 07:39:57,152][20347] Updated weights for policy 0, policy_version 291764 (0.0007) [2023-12-27 07:39:57,201][20347] Updated weights for policy 0, policy_version 291774 (0.0005) [2023-12-27 07:39:57,290][20381] Updated weights for policy 1, policy_version 291469 (0.0008) [2023-12-27 07:39:57,342][20381] Updated weights for policy 1, policy_version 291479 (0.0011) [2023-12-27 07:39:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 149340160. Throughput: 0: 10465.3, 1: 10417.4. Samples: 149328120. Policy #0 lag: (min: 19.0, avg: 33.0, max: 51.0) [2023-12-27 07:39:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '866.750')] [2023-12-27 07:39:57,390][20381] Updated weights for policy 1, policy_version 291489 (0.0010) [2023-12-27 07:39:57,850][20347] Updated weights for policy 0, policy_version 291784 (0.0007) [2023-12-27 07:39:57,902][20347] Updated weights for policy 0, policy_version 291794 (0.0010) [2023-12-27 07:39:57,950][20347] Updated weights for policy 0, policy_version 291804 (0.0010) [2023-12-27 07:39:58,005][20381] Updated weights for policy 1, policy_version 291499 (0.0009) [2023-12-27 07:39:58,059][20381] Updated weights for policy 1, policy_version 291509 (0.0006) [2023-12-27 07:39:58,124][20381] Updated weights for policy 1, policy_version 291519 (0.0005) [2023-12-27 07:39:58,643][20347] Updated weights for policy 0, policy_version 291814 (0.0009) [2023-12-27 07:39:58,708][20347] Updated weights for policy 0, policy_version 291824 (0.0008) [2023-12-27 07:39:58,772][20347] Updated weights for policy 0, policy_version 291834 (0.0008) [2023-12-27 07:39:58,888][20381] Updated weights for policy 1, policy_version 291529 (0.0007) [2023-12-27 07:39:58,954][20381] Updated weights for policy 1, policy_version 291539 (0.0009) [2023-12-27 07:39:59,014][20381] Updated weights for policy 1, policy_version 291549 (0.0009) [2023-12-27 07:39:59,073][20381] Updated weights for policy 1, policy_version 291559 (0.0009) [2023-12-27 07:39:59,554][20347] Updated weights for policy 0, policy_version 291844 (0.0009) [2023-12-27 07:39:59,612][20347] Updated weights for policy 0, policy_version 291854 (0.0010) [2023-12-27 07:39:59,675][20347] Updated weights for policy 0, policy_version 291864 (0.0010) [2023-12-27 07:39:59,859][20381] Updated weights for policy 1, policy_version 291569 (0.0008) [2023-12-27 07:39:59,923][20381] Updated weights for policy 1, policy_version 291579 (0.0008) [2023-12-27 07:39:59,972][20381] Updated weights for policy 1, policy_version 291589 (0.0007) [2023-12-27 07:40:00,377][20347] Updated weights for policy 0, policy_version 291874 (0.0010) [2023-12-27 07:40:00,425][20347] Updated weights for policy 0, policy_version 291884 (0.0010) [2023-12-27 07:40:00,484][20347] Updated weights for policy 0, policy_version 291894 (0.0010) [2023-12-27 07:40:00,537][20347] Updated weights for policy 0, policy_version 291904 (0.0005) [2023-12-27 07:40:00,626][20381] Updated weights for policy 1, policy_version 291599 (0.0008) [2023-12-27 07:40:00,668][20381] Updated weights for policy 1, policy_version 291609 (0.0010) [2023-12-27 07:40:00,713][20381] Updated weights for policy 1, policy_version 291619 (0.0010) [2023-12-27 07:40:01,090][20347] Updated weights for policy 0, policy_version 291914 (0.0010) [2023-12-27 07:40:01,153][20347] Updated weights for policy 0, policy_version 291924 (0.0010) [2023-12-27 07:40:01,205][20347] Updated weights for policy 0, policy_version 291934 (0.0010) [2023-12-27 07:40:01,458][20381] Updated weights for policy 1, policy_version 291629 (0.0009) [2023-12-27 07:40:01,520][20381] Updated weights for policy 1, policy_version 291639 (0.0007) [2023-12-27 07:40:01,580][20381] Updated weights for policy 1, policy_version 291649 (0.0007) [2023-12-27 07:40:01,932][20347] Updated weights for policy 0, policy_version 291944 (0.0008) [2023-12-27 07:40:01,999][20347] Updated weights for policy 0, policy_version 291954 (0.0009) [2023-12-27 07:40:02,059][20347] Updated weights for policy 0, policy_version 291964 (0.0007) [2023-12-27 07:40:02,305][20381] Updated weights for policy 1, policy_version 291659 (0.0007) [2023-12-27 07:40:02,360][20381] Updated weights for policy 1, policy_version 291669 (0.0007) [2023-12-27 07:40:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 149438464. Throughput: 0: 10386.0, 1: 10346.5. Samples: 149449948. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '924.670')] [2023-12-27 07:40:02,426][20381] Updated weights for policy 1, policy_version 291679 (0.0008) [2023-12-27 07:40:02,755][20347] Updated weights for policy 0, policy_version 291974 (0.0009) [2023-12-27 07:40:02,816][20347] Updated weights for policy 0, policy_version 291984 (0.0010) [2023-12-27 07:40:02,874][20347] Updated weights for policy 0, policy_version 291994 (0.0010) [2023-12-27 07:40:03,115][20381] Updated weights for policy 1, policy_version 291689 (0.0008) [2023-12-27 07:40:03,167][20381] Updated weights for policy 1, policy_version 291699 (0.0010) [2023-12-27 07:40:03,227][20381] Updated weights for policy 1, policy_version 291709 (0.0010) [2023-12-27 07:40:03,285][20381] Updated weights for policy 1, policy_version 291719 (0.0010) [2023-12-27 07:40:03,486][20347] Updated weights for policy 0, policy_version 292004 (0.0007) [2023-12-27 07:40:03,544][20347] Updated weights for policy 0, policy_version 292014 (0.0006) [2023-12-27 07:40:03,594][20347] Updated weights for policy 0, policy_version 292024 (0.0005) [2023-12-27 07:40:03,902][20381] Updated weights for policy 1, policy_version 291729 (0.0009) [2023-12-27 07:40:03,952][20381] Updated weights for policy 1, policy_version 291739 (0.0011) [2023-12-27 07:40:04,000][20381] Updated weights for policy 1, policy_version 291749 (0.0007) [2023-12-27 07:40:04,247][20347] Updated weights for policy 0, policy_version 292034 (0.0008) [2023-12-27 07:40:04,309][20347] Updated weights for policy 0, policy_version 292044 (0.0010) [2023-12-27 07:40:04,370][20347] Updated weights for policy 0, policy_version 292054 (0.0010) [2023-12-27 07:40:04,429][20347] Updated weights for policy 0, policy_version 292064 (0.0010) [2023-12-27 07:40:04,652][20381] Updated weights for policy 1, policy_version 291759 (0.0005) [2023-12-27 07:40:04,708][20381] Updated weights for policy 1, policy_version 291769 (0.0005) [2023-12-27 07:40:04,770][20381] Updated weights for policy 1, policy_version 291779 (0.0006) [2023-12-27 07:40:05,067][20347] Updated weights for policy 0, policy_version 292074 (0.0005) [2023-12-27 07:40:05,112][20347] Updated weights for policy 0, policy_version 292084 (0.0009) [2023-12-27 07:40:05,158][20347] Updated weights for policy 0, policy_version 292094 (0.0010) [2023-12-27 07:40:05,343][20381] Updated weights for policy 1, policy_version 291789 (0.0008) [2023-12-27 07:40:05,397][20381] Updated weights for policy 1, policy_version 291799 (0.0007) [2023-12-27 07:40:05,452][20381] Updated weights for policy 1, policy_version 291809 (0.0006) [2023-12-27 07:40:05,798][20347] Updated weights for policy 0, policy_version 292104 (0.0010) [2023-12-27 07:40:05,848][20347] Updated weights for policy 0, policy_version 292114 (0.0009) [2023-12-27 07:40:05,899][20347] Updated weights for policy 0, policy_version 292124 (0.0005) [2023-12-27 07:40:06,053][20381] Updated weights for policy 1, policy_version 291819 (0.0006) [2023-12-27 07:40:06,110][20381] Updated weights for policy 1, policy_version 291829 (0.0009) [2023-12-27 07:40:06,175][20381] Updated weights for policy 1, policy_version 291839 (0.0009) [2023-12-27 07:40:06,687][20347] Updated weights for policy 0, policy_version 292134 (0.0005) [2023-12-27 07:40:06,748][20347] Updated weights for policy 0, policy_version 292144 (0.0005) [2023-12-27 07:40:06,813][20347] Updated weights for policy 0, policy_version 292154 (0.0005) [2023-12-27 07:40:06,999][20381] Updated weights for policy 1, policy_version 291849 (0.0009) [2023-12-27 07:40:07,058][20381] Updated weights for policy 1, policy_version 291859 (0.0008) [2023-12-27 07:40:07,111][20381] Updated weights for policy 1, policy_version 291869 (0.0008) [2023-12-27 07:40:07,174][20381] Updated weights for policy 1, policy_version 291879 (0.0009) [2023-12-27 07:40:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 149544960. Throughput: 0: 10399.6, 1: 10381.1. Samples: 149513836. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '855.510')] [2023-12-27 07:40:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000291880_74735616.pth... [2023-12-27 07:40:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000290664_74424320.pth [2023-12-27 07:40:07,393][20347] Updated weights for policy 0, policy_version 292164 (0.0007) [2023-12-27 07:40:07,449][20347] Updated weights for policy 0, policy_version 292174 (0.0007) [2023-12-27 07:40:07,509][20347] Updated weights for policy 0, policy_version 292184 (0.0007) [2023-12-27 07:40:07,558][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000292192_74817536.pth... [2023-12-27 07:40:07,561][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000290976_74506240.pth [2023-12-27 07:40:07,820][20381] Updated weights for policy 1, policy_version 291889 (0.0006) [2023-12-27 07:40:07,880][20381] Updated weights for policy 1, policy_version 291899 (0.0005) [2023-12-27 07:40:07,932][20381] Updated weights for policy 1, policy_version 291909 (0.0006) [2023-12-27 07:40:08,184][20347] Updated weights for policy 0, policy_version 292194 (0.0006) [2023-12-27 07:40:08,240][20347] Updated weights for policy 0, policy_version 292204 (0.0005) [2023-12-27 07:40:08,302][20347] Updated weights for policy 0, policy_version 292214 (0.0005) [2023-12-27 07:40:08,362][20347] Updated weights for policy 0, policy_version 292224 (0.0006) [2023-12-27 07:40:08,572][20381] Updated weights for policy 1, policy_version 291919 (0.0008) [2023-12-27 07:40:08,632][20381] Updated weights for policy 1, policy_version 291929 (0.0009) [2023-12-27 07:40:08,688][20381] Updated weights for policy 1, policy_version 291939 (0.0009) [2023-12-27 07:40:08,948][20347] Updated weights for policy 0, policy_version 292234 (0.0006) [2023-12-27 07:40:09,014][20347] Updated weights for policy 0, policy_version 292244 (0.0006) [2023-12-27 07:40:09,077][20347] Updated weights for policy 0, policy_version 292254 (0.0006) [2023-12-27 07:40:09,349][20381] Updated weights for policy 1, policy_version 291949 (0.0009) [2023-12-27 07:40:09,406][20381] Updated weights for policy 1, policy_version 291959 (0.0006) [2023-12-27 07:40:09,461][20381] Updated weights for policy 1, policy_version 291969 (0.0006) [2023-12-27 07:40:09,820][20347] Updated weights for policy 0, policy_version 292264 (0.0007) [2023-12-27 07:40:09,885][20347] Updated weights for policy 0, policy_version 292274 (0.0006) [2023-12-27 07:40:09,947][20347] Updated weights for policy 0, policy_version 292284 (0.0008) [2023-12-27 07:40:10,072][20381] Updated weights for policy 1, policy_version 291979 (0.0006) [2023-12-27 07:40:10,133][20381] Updated weights for policy 1, policy_version 291989 (0.0008) [2023-12-27 07:40:10,196][20381] Updated weights for policy 1, policy_version 291999 (0.0008) [2023-12-27 07:40:10,646][20347] Updated weights for policy 0, policy_version 292294 (0.0007) [2023-12-27 07:40:10,700][20347] Updated weights for policy 0, policy_version 292304 (0.0008) [2023-12-27 07:40:10,754][20347] Updated weights for policy 0, policy_version 292314 (0.0008) [2023-12-27 07:40:10,898][20381] Updated weights for policy 1, policy_version 292009 (0.0008) [2023-12-27 07:40:10,951][20381] Updated weights for policy 1, policy_version 292019 (0.0005) [2023-12-27 07:40:10,999][20381] Updated weights for policy 1, policy_version 292029 (0.0006) [2023-12-27 07:40:11,065][20381] Updated weights for policy 1, policy_version 292039 (0.0007) [2023-12-27 07:40:11,472][20347] Updated weights for policy 0, policy_version 292324 (0.0008) [2023-12-27 07:40:11,539][20347] Updated weights for policy 0, policy_version 292334 (0.0007) [2023-12-27 07:40:11,601][20347] Updated weights for policy 0, policy_version 292344 (0.0008) [2023-12-27 07:40:11,772][20381] Updated weights for policy 1, policy_version 292049 (0.0007) [2023-12-27 07:40:11,826][20381] Updated weights for policy 1, policy_version 292059 (0.0008) [2023-12-27 07:40:11,885][20381] Updated weights for policy 1, policy_version 292069 (0.0008) [2023-12-27 07:40:12,312][20347] Updated weights for policy 0, policy_version 292354 (0.0009) [2023-12-27 07:40:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 149643264. Throughput: 0: 10363.7, 1: 10348.6. Samples: 149636428. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '685.610')] [2023-12-27 07:40:12,375][20347] Updated weights for policy 0, policy_version 292364 (0.0007) [2023-12-27 07:40:12,424][20347] Updated weights for policy 0, policy_version 292374 (0.0008) [2023-12-27 07:40:12,474][20347] Updated weights for policy 0, policy_version 292384 (0.0008) [2023-12-27 07:40:12,624][20381] Updated weights for policy 1, policy_version 292079 (0.0007) [2023-12-27 07:40:12,680][20381] Updated weights for policy 1, policy_version 292089 (0.0007) [2023-12-27 07:40:12,733][20381] Updated weights for policy 1, policy_version 292099 (0.0007) [2023-12-27 07:40:13,144][20347] Updated weights for policy 0, policy_version 292394 (0.0005) [2023-12-27 07:40:13,196][20347] Updated weights for policy 0, policy_version 292404 (0.0005) [2023-12-27 07:40:13,251][20347] Updated weights for policy 0, policy_version 292414 (0.0007) [2023-12-27 07:40:13,436][20381] Updated weights for policy 1, policy_version 292109 (0.0007) [2023-12-27 07:40:13,492][20381] Updated weights for policy 1, policy_version 292119 (0.0007) [2023-12-27 07:40:13,544][20381] Updated weights for policy 1, policy_version 292129 (0.0009) [2023-12-27 07:40:13,873][20347] Updated weights for policy 0, policy_version 292424 (0.0008) [2023-12-27 07:40:13,924][20347] Updated weights for policy 0, policy_version 292434 (0.0006) [2023-12-27 07:40:13,980][20347] Updated weights for policy 0, policy_version 292444 (0.0005) [2023-12-27 07:40:14,174][20381] Updated weights for policy 1, policy_version 292139 (0.0005) [2023-12-27 07:40:14,233][20381] Updated weights for policy 1, policy_version 292149 (0.0007) [2023-12-27 07:40:14,293][20381] Updated weights for policy 1, policy_version 292159 (0.0008) [2023-12-27 07:40:14,630][20347] Updated weights for policy 0, policy_version 292454 (0.0006) [2023-12-27 07:40:14,692][20347] Updated weights for policy 0, policy_version 292464 (0.0008) [2023-12-27 07:40:14,747][20347] Updated weights for policy 0, policy_version 292474 (0.0007) [2023-12-27 07:40:14,864][20381] Updated weights for policy 1, policy_version 292169 (0.0008) [2023-12-27 07:40:14,917][20381] Updated weights for policy 1, policy_version 292179 (0.0005) [2023-12-27 07:40:14,980][20381] Updated weights for policy 1, policy_version 292189 (0.0007) [2023-12-27 07:40:15,041][20381] Updated weights for policy 1, policy_version 292199 (0.0008) [2023-12-27 07:40:15,369][20347] Updated weights for policy 0, policy_version 292484 (0.0007) [2023-12-27 07:40:15,414][20347] Updated weights for policy 0, policy_version 292494 (0.0010) [2023-12-27 07:40:15,469][20347] Updated weights for policy 0, policy_version 292504 (0.0009) [2023-12-27 07:40:15,698][20381] Updated weights for policy 1, policy_version 292209 (0.0009) [2023-12-27 07:40:15,749][20381] Updated weights for policy 1, policy_version 292219 (0.0008) [2023-12-27 07:40:15,804][20381] Updated weights for policy 1, policy_version 292229 (0.0008) [2023-12-27 07:40:16,054][20347] Updated weights for policy 0, policy_version 292514 (0.0008) [2023-12-27 07:40:16,111][20347] Updated weights for policy 0, policy_version 292524 (0.0010) [2023-12-27 07:40:16,172][20347] Updated weights for policy 0, policy_version 292534 (0.0010) [2023-12-27 07:40:16,229][20347] Updated weights for policy 0, policy_version 292544 (0.0010) [2023-12-27 07:40:16,515][20381] Updated weights for policy 1, policy_version 292239 (0.0007) [2023-12-27 07:40:16,572][20381] Updated weights for policy 1, policy_version 292249 (0.0007) [2023-12-27 07:40:16,616][20381] Updated weights for policy 1, policy_version 292259 (0.0005) [2023-12-27 07:40:16,907][20347] Updated weights for policy 0, policy_version 292554 (0.0005) [2023-12-27 07:40:16,952][20347] Updated weights for policy 0, policy_version 292564 (0.0005) [2023-12-27 07:40:17,008][20347] Updated weights for policy 0, policy_version 292574 (0.0005) [2023-12-27 07:40:17,231][20381] Updated weights for policy 1, policy_version 292269 (0.0006) [2023-12-27 07:40:17,282][20381] Updated weights for policy 1, policy_version 292279 (0.0008) [2023-12-27 07:40:17,330][20381] Updated weights for policy 1, policy_version 292289 (0.0008) [2023-12-27 07:40:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 149757952. Throughput: 0: 10363.6, 1: 10404.4. Samples: 149763092. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '499.710')] [2023-12-27 07:40:17,559][20347] Updated weights for policy 0, policy_version 292584 (0.0009) [2023-12-27 07:40:17,609][20347] Updated weights for policy 0, policy_version 292594 (0.0010) [2023-12-27 07:40:17,670][20347] Updated weights for policy 0, policy_version 292604 (0.0006) [2023-12-27 07:40:18,050][20381] Updated weights for policy 1, policy_version 292299 (0.0007) [2023-12-27 07:40:18,102][20381] Updated weights for policy 1, policy_version 292309 (0.0005) [2023-12-27 07:40:18,159][20381] Updated weights for policy 1, policy_version 292319 (0.0006) [2023-12-27 07:40:18,208][20347] Updated weights for policy 0, policy_version 292614 (0.0007) [2023-12-27 07:40:18,267][20347] Updated weights for policy 0, policy_version 292624 (0.0007) [2023-12-27 07:40:18,326][20347] Updated weights for policy 0, policy_version 292634 (0.0008) [2023-12-27 07:40:18,805][20381] Updated weights for policy 1, policy_version 292329 (0.0007) [2023-12-27 07:40:18,857][20381] Updated weights for policy 1, policy_version 292339 (0.0008) [2023-12-27 07:40:18,908][20381] Updated weights for policy 1, policy_version 292349 (0.0005) [2023-12-27 07:40:18,913][20347] Updated weights for policy 0, policy_version 292644 (0.0006) [2023-12-27 07:40:18,960][20381] Updated weights for policy 1, policy_version 292359 (0.0005) [2023-12-27 07:40:18,964][20347] Updated weights for policy 0, policy_version 292654 (0.0006) [2023-12-27 07:40:19,010][20347] Updated weights for policy 0, policy_version 292664 (0.0010) [2023-12-27 07:40:19,626][20381] Updated weights for policy 1, policy_version 292369 (0.0008) [2023-12-27 07:40:19,681][20381] Updated weights for policy 1, policy_version 292379 (0.0008) [2023-12-27 07:40:19,696][20347] Updated weights for policy 0, policy_version 292674 (0.0008) [2023-12-27 07:40:19,736][20381] Updated weights for policy 1, policy_version 292389 (0.0008) [2023-12-27 07:40:19,745][20347] Updated weights for policy 0, policy_version 292684 (0.0007) [2023-12-27 07:40:19,797][20347] Updated weights for policy 0, policy_version 292694 (0.0005) [2023-12-27 07:40:19,854][20347] Updated weights for policy 0, policy_version 292704 (0.0007) [2023-12-27 07:40:20,412][20381] Updated weights for policy 1, policy_version 292399 (0.0008) [2023-12-27 07:40:20,467][20381] Updated weights for policy 1, policy_version 292409 (0.0007) [2023-12-27 07:40:20,525][20381] Updated weights for policy 1, policy_version 292419 (0.0008) [2023-12-27 07:40:20,602][20347] Updated weights for policy 0, policy_version 292714 (0.0008) [2023-12-27 07:40:20,651][20347] Updated weights for policy 0, policy_version 292724 (0.0008) [2023-12-27 07:40:20,707][20347] Updated weights for policy 0, policy_version 292734 (0.0009) [2023-12-27 07:40:21,148][20381] Updated weights for policy 1, policy_version 292429 (0.0007) [2023-12-27 07:40:21,208][20381] Updated weights for policy 1, policy_version 292439 (0.0008) [2023-12-27 07:40:21,270][20381] Updated weights for policy 1, policy_version 292449 (0.0007) [2023-12-27 07:40:21,513][20347] Updated weights for policy 0, policy_version 292744 (0.0008) [2023-12-27 07:40:21,571][20347] Updated weights for policy 0, policy_version 292754 (0.0007) [2023-12-27 07:40:21,636][20347] Updated weights for policy 0, policy_version 292764 (0.0007) [2023-12-27 07:40:21,936][20381] Updated weights for policy 1, policy_version 292459 (0.0007) [2023-12-27 07:40:21,986][20381] Updated weights for policy 1, policy_version 292469 (0.0005) [2023-12-27 07:40:22,033][20381] Updated weights for policy 1, policy_version 292479 (0.0005) [2023-12-27 07:40:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 149856256. Throughput: 0: 10423.7, 1: 10408.9. Samples: 149827620. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '407.730')] [2023-12-27 07:40:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000292488_74891264.pth... [2023-12-27 07:40:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000291272_74579968.pth [2023-12-27 07:40:22,396][20347] Updated weights for policy 0, policy_version 292774 (0.0008) [2023-12-27 07:40:22,454][20347] Updated weights for policy 0, policy_version 292784 (0.0009) [2023-12-27 07:40:22,501][20347] Updated weights for policy 0, policy_version 292794 (0.0011) [2023-12-27 07:40:22,527][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000292800_74973184.pth... [2023-12-27 07:40:22,531][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000291584_74661888.pth [2023-12-27 07:40:22,675][20381] Updated weights for policy 1, policy_version 292489 (0.0007) [2023-12-27 07:40:22,728][20381] Updated weights for policy 1, policy_version 292499 (0.0007) [2023-12-27 07:40:22,780][20381] Updated weights for policy 1, policy_version 292509 (0.0007) [2023-12-27 07:40:22,840][20381] Updated weights for policy 1, policy_version 292519 (0.0006) [2023-12-27 07:40:23,178][20347] Updated weights for policy 0, policy_version 292804 (0.0010) [2023-12-27 07:40:23,239][20347] Updated weights for policy 0, policy_version 292814 (0.0009) [2023-12-27 07:40:23,298][20347] Updated weights for policy 0, policy_version 292824 (0.0010) [2023-12-27 07:40:23,543][20381] Updated weights for policy 1, policy_version 292529 (0.0007) [2023-12-27 07:40:23,588][20381] Updated weights for policy 1, policy_version 292539 (0.0006) [2023-12-27 07:40:23,639][20381] Updated weights for policy 1, policy_version 292549 (0.0005) [2023-12-27 07:40:23,883][20347] Updated weights for policy 0, policy_version 292834 (0.0006) [2023-12-27 07:40:23,944][20347] Updated weights for policy 0, policy_version 292844 (0.0010) [2023-12-27 07:40:23,998][20347] Updated weights for policy 0, policy_version 292854 (0.0010) [2023-12-27 07:40:24,049][20347] Updated weights for policy 0, policy_version 292864 (0.0010) [2023-12-27 07:40:24,302][20381] Updated weights for policy 1, policy_version 292559 (0.0007) [2023-12-27 07:40:24,370][20381] Updated weights for policy 1, policy_version 292569 (0.0005) [2023-12-27 07:40:24,432][20381] Updated weights for policy 1, policy_version 292579 (0.0005) [2023-12-27 07:40:24,680][20347] Updated weights for policy 0, policy_version 292874 (0.0010) [2023-12-27 07:40:24,736][20347] Updated weights for policy 0, policy_version 292884 (0.0008) [2023-12-27 07:40:24,798][20347] Updated weights for policy 0, policy_version 292894 (0.0007) [2023-12-27 07:40:24,950][20381] Updated weights for policy 1, policy_version 292589 (0.0006) [2023-12-27 07:40:25,012][20381] Updated weights for policy 1, policy_version 292599 (0.0009) [2023-12-27 07:40:25,068][20381] Updated weights for policy 1, policy_version 292609 (0.0010) [2023-12-27 07:40:25,458][20347] Updated weights for policy 0, policy_version 292904 (0.0010) [2023-12-27 07:40:25,510][20347] Updated weights for policy 0, policy_version 292914 (0.0010) [2023-12-27 07:40:25,558][20347] Updated weights for policy 0, policy_version 292924 (0.0010) [2023-12-27 07:40:25,685][20381] Updated weights for policy 1, policy_version 292619 (0.0010) [2023-12-27 07:40:25,743][20381] Updated weights for policy 1, policy_version 292629 (0.0008) [2023-12-27 07:40:25,794][20381] Updated weights for policy 1, policy_version 292639 (0.0011) [2023-12-27 07:40:26,213][20347] Updated weights for policy 0, policy_version 292934 (0.0010) [2023-12-27 07:40:26,268][20347] Updated weights for policy 0, policy_version 292944 (0.0010) [2023-12-27 07:40:26,328][20347] Updated weights for policy 0, policy_version 292954 (0.0006) [2023-12-27 07:40:26,451][20381] Updated weights for policy 1, policy_version 292649 (0.0011) [2023-12-27 07:40:26,513][20381] Updated weights for policy 1, policy_version 292659 (0.0007) [2023-12-27 07:40:26,568][20381] Updated weights for policy 1, policy_version 292669 (0.0005) [2023-12-27 07:40:26,628][20381] Updated weights for policy 1, policy_version 292679 (0.0008) [2023-12-27 07:40:26,920][20347] Updated weights for policy 0, policy_version 292964 (0.0006) [2023-12-27 07:40:26,962][20347] Updated weights for policy 0, policy_version 292974 (0.0007) [2023-12-27 07:40:27,009][20347] Updated weights for policy 0, policy_version 292984 (0.0006) [2023-12-27 07:40:27,267][20381] Updated weights for policy 1, policy_version 292689 (0.0007) [2023-12-27 07:40:27,320][20381] Updated weights for policy 1, policy_version 292699 (0.0006) [2023-12-27 07:40:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 149962752. Throughput: 0: 10404.3, 1: 10466.5. Samples: 149953420. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '180.540')] [2023-12-27 07:40:27,369][20381] Updated weights for policy 1, policy_version 292709 (0.0005) [2023-12-27 07:40:27,656][20347] Updated weights for policy 0, policy_version 292994 (0.0007) [2023-12-27 07:40:27,704][20347] Updated weights for policy 0, policy_version 293004 (0.0008) [2023-12-27 07:40:27,752][20347] Updated weights for policy 0, policy_version 293014 (0.0008) [2023-12-27 07:40:27,805][20347] Updated weights for policy 0, policy_version 293024 (0.0008) [2023-12-27 07:40:27,893][20381] Updated weights for policy 1, policy_version 292719 (0.0006) [2023-12-27 07:40:27,948][20381] Updated weights for policy 1, policy_version 292729 (0.0007) [2023-12-27 07:40:27,998][20381] Updated weights for policy 1, policy_version 292739 (0.0009) [2023-12-27 07:40:28,540][20347] Updated weights for policy 0, policy_version 293034 (0.0007) [2023-12-27 07:40:28,592][20347] Updated weights for policy 0, policy_version 293044 (0.0008) [2023-12-27 07:40:28,639][20347] Updated weights for policy 0, policy_version 293054 (0.0006) [2023-12-27 07:40:28,641][20381] Updated weights for policy 1, policy_version 292749 (0.0007) [2023-12-27 07:40:28,698][20381] Updated weights for policy 1, policy_version 292759 (0.0006) [2023-12-27 07:40:28,753][20381] Updated weights for policy 1, policy_version 292769 (0.0005) [2023-12-27 07:40:29,266][20347] Updated weights for policy 0, policy_version 293064 (0.0007) [2023-12-27 07:40:29,322][20347] Updated weights for policy 0, policy_version 293074 (0.0008) [2023-12-27 07:40:29,387][20347] Updated weights for policy 0, policy_version 293084 (0.0007) [2023-12-27 07:40:29,392][20381] Updated weights for policy 1, policy_version 292779 (0.0006) [2023-12-27 07:40:29,440][20381] Updated weights for policy 1, policy_version 292789 (0.0005) [2023-12-27 07:40:29,500][20381] Updated weights for policy 1, policy_version 292799 (0.0006) [2023-12-27 07:40:30,106][20381] Updated weights for policy 1, policy_version 292809 (0.0006) [2023-12-27 07:40:30,128][20347] Updated weights for policy 0, policy_version 293094 (0.0006) [2023-12-27 07:40:30,162][20381] Updated weights for policy 1, policy_version 292819 (0.0008) [2023-12-27 07:40:30,179][20347] Updated weights for policy 0, policy_version 293104 (0.0006) [2023-12-27 07:40:30,214][20381] Updated weights for policy 1, policy_version 292829 (0.0008) [2023-12-27 07:40:30,231][20347] Updated weights for policy 0, policy_version 293114 (0.0005) [2023-12-27 07:40:30,265][20381] Updated weights for policy 1, policy_version 292839 (0.0007) [2023-12-27 07:40:30,917][20381] Updated weights for policy 1, policy_version 292849 (0.0005) [2023-12-27 07:40:30,951][20347] Updated weights for policy 0, policy_version 293124 (0.0005) [2023-12-27 07:40:30,982][20381] Updated weights for policy 1, policy_version 292859 (0.0006) [2023-12-27 07:40:31,004][20347] Updated weights for policy 0, policy_version 293134 (0.0005) [2023-12-27 07:40:31,040][20381] Updated weights for policy 1, policy_version 292869 (0.0008) [2023-12-27 07:40:31,060][20347] Updated weights for policy 0, policy_version 293144 (0.0007) [2023-12-27 07:40:31,672][20381] Updated weights for policy 1, policy_version 292879 (0.0010) [2023-12-27 07:40:31,695][20347] Updated weights for policy 0, policy_version 293154 (0.0007) [2023-12-27 07:40:31,726][20381] Updated weights for policy 1, policy_version 292889 (0.0010) [2023-12-27 07:40:31,761][20347] Updated weights for policy 0, policy_version 293164 (0.0008) [2023-12-27 07:40:31,786][20381] Updated weights for policy 1, policy_version 292899 (0.0009) [2023-12-27 07:40:31,821][20347] Updated weights for policy 0, policy_version 293174 (0.0006) [2023-12-27 07:40:31,880][20347] Updated weights for policy 0, policy_version 293184 (0.0008) [2023-12-27 07:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 150069248. Throughput: 0: 10379.8, 1: 10543.7. Samples: 150081184. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '211.500')] [2023-12-27 07:40:32,496][20381] Updated weights for policy 1, policy_version 292909 (0.0009) [2023-12-27 07:40:32,552][20347] Updated weights for policy 0, policy_version 293194 (0.0007) [2023-12-27 07:40:32,554][20381] Updated weights for policy 1, policy_version 292919 (0.0007) [2023-12-27 07:40:32,616][20381] Updated weights for policy 1, policy_version 292929 (0.0006) [2023-12-27 07:40:32,622][20347] Updated weights for policy 0, policy_version 293204 (0.0008) [2023-12-27 07:40:32,676][20347] Updated weights for policy 0, policy_version 293214 (0.0007) [2023-12-27 07:40:33,318][20381] Updated weights for policy 1, policy_version 292939 (0.0006) [2023-12-27 07:40:33,363][20347] Updated weights for policy 0, policy_version 293224 (0.0006) [2023-12-27 07:40:33,368][20381] Updated weights for policy 1, policy_version 292949 (0.0006) [2023-12-27 07:40:33,423][20347] Updated weights for policy 0, policy_version 293234 (0.0006) [2023-12-27 07:40:33,428][20381] Updated weights for policy 1, policy_version 292959 (0.0007) [2023-12-27 07:40:33,483][20347] Updated weights for policy 0, policy_version 293244 (0.0007) [2023-12-27 07:40:34,142][20381] Updated weights for policy 1, policy_version 292969 (0.0007) [2023-12-27 07:40:34,175][20347] Updated weights for policy 0, policy_version 293254 (0.0010) [2023-12-27 07:40:34,204][20381] Updated weights for policy 1, policy_version 292979 (0.0006) [2023-12-27 07:40:34,228][20347] Updated weights for policy 0, policy_version 293264 (0.0008) [2023-12-27 07:40:34,260][20381] Updated weights for policy 1, policy_version 292989 (0.0006) [2023-12-27 07:40:34,282][20347] Updated weights for policy 0, policy_version 293274 (0.0007) [2023-12-27 07:40:34,315][20381] Updated weights for policy 1, policy_version 292999 (0.0006) [2023-12-27 07:40:34,927][20347] Updated weights for policy 0, policy_version 293284 (0.0006) [2023-12-27 07:40:34,992][20347] Updated weights for policy 0, policy_version 293294 (0.0006) [2023-12-27 07:40:34,999][20381] Updated weights for policy 1, policy_version 293009 (0.0008) [2023-12-27 07:40:35,050][20347] Updated weights for policy 0, policy_version 293304 (0.0010) [2023-12-27 07:40:35,056][20381] Updated weights for policy 1, policy_version 293019 (0.0010) [2023-12-27 07:40:35,117][20381] Updated weights for policy 1, policy_version 293029 (0.0010) [2023-12-27 07:40:35,659][20347] Updated weights for policy 0, policy_version 293314 (0.0010) [2023-12-27 07:40:35,724][20347] Updated weights for policy 0, policy_version 293324 (0.0010) [2023-12-27 07:40:35,779][20347] Updated weights for policy 0, policy_version 293334 (0.0010) [2023-12-27 07:40:35,788][20381] Updated weights for policy 1, policy_version 293039 (0.0007) [2023-12-27 07:40:35,834][20347] Updated weights for policy 0, policy_version 293344 (0.0010) [2023-12-27 07:40:35,836][20381] Updated weights for policy 1, policy_version 293049 (0.0005) [2023-12-27 07:40:35,884][20381] Updated weights for policy 1, policy_version 293059 (0.0008) [2023-12-27 07:40:36,498][20347] Updated weights for policy 0, policy_version 293354 (0.0005) [2023-12-27 07:40:36,556][20347] Updated weights for policy 0, policy_version 293364 (0.0007) [2023-12-27 07:40:36,603][20381] Updated weights for policy 1, policy_version 293069 (0.0009) [2023-12-27 07:40:36,614][20347] Updated weights for policy 0, policy_version 293374 (0.0006) [2023-12-27 07:40:36,660][20381] Updated weights for policy 1, policy_version 293079 (0.0011) [2023-12-27 07:40:36,709][20381] Updated weights for policy 1, policy_version 293089 (0.0011) [2023-12-27 07:40:37,248][20347] Updated weights for policy 0, policy_version 293384 (0.0006) [2023-12-27 07:40:37,300][20347] Updated weights for policy 0, policy_version 293394 (0.0005) [2023-12-27 07:40:37,340][20381] Updated weights for policy 1, policy_version 293099 (0.0011) [2023-12-27 07:40:37,347][20347] Updated weights for policy 0, policy_version 293404 (0.0007) [2023-12-27 07:40:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 150167552. Throughput: 0: 10382.8, 1: 10506.9. Samples: 150141732. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '250.230')] [2023-12-27 07:40:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000293408_75128832.pth... [2023-12-27 07:40:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000292192_74817536.pth [2023-12-27 07:40:37,396][20381] Updated weights for policy 1, policy_version 293109 (0.0010) [2023-12-27 07:40:37,448][20381] Updated weights for policy 1, policy_version 293119 (0.0010) [2023-12-27 07:40:37,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000293128_75055104.pth... [2023-12-27 07:40:37,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000291880_74735616.pth [2023-12-27 07:40:38,042][20347] Updated weights for policy 0, policy_version 293414 (0.0008) [2023-12-27 07:40:38,095][20347] Updated weights for policy 0, policy_version 293424 (0.0007) [2023-12-27 07:40:38,144][20347] Updated weights for policy 0, policy_version 293434 (0.0007) [2023-12-27 07:40:38,147][20381] Updated weights for policy 1, policy_version 293129 (0.0010) [2023-12-27 07:40:38,201][20381] Updated weights for policy 1, policy_version 293139 (0.0005) [2023-12-27 07:40:38,248][20381] Updated weights for policy 1, policy_version 293149 (0.0005) [2023-12-27 07:40:38,298][20381] Updated weights for policy 1, policy_version 293159 (0.0005) [2023-12-27 07:40:38,884][20347] Updated weights for policy 0, policy_version 293444 (0.0007) [2023-12-27 07:40:38,940][20347] Updated weights for policy 0, policy_version 293454 (0.0007) [2023-12-27 07:40:38,950][20381] Updated weights for policy 1, policy_version 293169 (0.0005) [2023-12-27 07:40:38,994][20347] Updated weights for policy 0, policy_version 293464 (0.0005) [2023-12-27 07:40:39,006][20381] Updated weights for policy 1, policy_version 293179 (0.0006) [2023-12-27 07:40:39,064][20381] Updated weights for policy 1, policy_version 293189 (0.0007) [2023-12-27 07:40:39,627][20347] Updated weights for policy 0, policy_version 293474 (0.0006) [2023-12-27 07:40:39,690][20347] Updated weights for policy 0, policy_version 293484 (0.0008) [2023-12-27 07:40:39,753][20347] Updated weights for policy 0, policy_version 293494 (0.0008) [2023-12-27 07:40:39,778][20381] Updated weights for policy 1, policy_version 293199 (0.0007) [2023-12-27 07:40:39,820][20347] Updated weights for policy 0, policy_version 293504 (0.0008) [2023-12-27 07:40:39,841][20381] Updated weights for policy 1, policy_version 293209 (0.0007) [2023-12-27 07:40:39,899][20381] Updated weights for policy 1, policy_version 293219 (0.0007) [2023-12-27 07:40:40,479][20347] Updated weights for policy 0, policy_version 293514 (0.0008) [2023-12-27 07:40:40,539][20347] Updated weights for policy 0, policy_version 293524 (0.0007) [2023-12-27 07:40:40,585][20347] Updated weights for policy 0, policy_version 293534 (0.0006) [2023-12-27 07:40:40,620][20381] Updated weights for policy 1, policy_version 293229 (0.0008) [2023-12-27 07:40:40,670][20381] Updated weights for policy 1, policy_version 293239 (0.0008) [2023-12-27 07:40:40,728][20381] Updated weights for policy 1, policy_version 293249 (0.0008) [2023-12-27 07:40:41,298][20347] Updated weights for policy 0, policy_version 293544 (0.0006) [2023-12-27 07:40:41,366][20347] Updated weights for policy 0, policy_version 293554 (0.0007) [2023-12-27 07:40:41,423][20347] Updated weights for policy 0, policy_version 293564 (0.0006) [2023-12-27 07:40:41,463][20381] Updated weights for policy 1, policy_version 293259 (0.0007) [2023-12-27 07:40:41,531][20381] Updated weights for policy 1, policy_version 293269 (0.0009) [2023-12-27 07:40:41,592][20381] Updated weights for policy 1, policy_version 293279 (0.0008) [2023-12-27 07:40:42,116][20347] Updated weights for policy 0, policy_version 293574 (0.0006) [2023-12-27 07:40:42,184][20347] Updated weights for policy 0, policy_version 293584 (0.0005) [2023-12-27 07:40:42,240][20381] Updated weights for policy 1, policy_version 293289 (0.0007) [2023-12-27 07:40:42,246][20347] Updated weights for policy 0, policy_version 293594 (0.0006) [2023-12-27 07:40:42,298][20381] Updated weights for policy 1, policy_version 293299 (0.0006) [2023-12-27 07:40:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 150274048. Throughput: 0: 10392.7, 1: 10413.1. Samples: 150264380. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '180.010')] [2023-12-27 07:40:42,366][20381] Updated weights for policy 1, policy_version 293309 (0.0007) [2023-12-27 07:40:42,420][20381] Updated weights for policy 1, policy_version 293319 (0.0007) [2023-12-27 07:40:42,923][20347] Updated weights for policy 0, policy_version 293604 (0.0009) [2023-12-27 07:40:42,988][20347] Updated weights for policy 0, policy_version 293614 (0.0007) [2023-12-27 07:40:43,046][20347] Updated weights for policy 0, policy_version 293624 (0.0008) [2023-12-27 07:40:43,098][20381] Updated weights for policy 1, policy_version 293329 (0.0009) [2023-12-27 07:40:43,152][20381] Updated weights for policy 1, policy_version 293339 (0.0010) [2023-12-27 07:40:43,210][20381] Updated weights for policy 1, policy_version 293349 (0.0005) [2023-12-27 07:40:43,646][20347] Updated weights for policy 0, policy_version 293634 (0.0009) [2023-12-27 07:40:43,692][20347] Updated weights for policy 0, policy_version 293644 (0.0008) [2023-12-27 07:40:43,740][20347] Updated weights for policy 0, policy_version 293654 (0.0010) [2023-12-27 07:40:43,784][20347] Updated weights for policy 0, policy_version 293664 (0.0010) [2023-12-27 07:40:43,878][20381] Updated weights for policy 1, policy_version 293359 (0.0009) [2023-12-27 07:40:43,920][20381] Updated weights for policy 1, policy_version 293369 (0.0009) [2023-12-27 07:40:43,968][20381] Updated weights for policy 1, policy_version 293379 (0.0007) [2023-12-27 07:40:44,435][20347] Updated weights for policy 0, policy_version 293674 (0.0010) [2023-12-27 07:40:44,480][20347] Updated weights for policy 0, policy_version 293684 (0.0010) [2023-12-27 07:40:44,531][20347] Updated weights for policy 0, policy_version 293694 (0.0010) [2023-12-27 07:40:44,605][20381] Updated weights for policy 1, policy_version 293389 (0.0009) [2023-12-27 07:40:44,657][20381] Updated weights for policy 1, policy_version 293399 (0.0011) [2023-12-27 07:40:44,715][20381] Updated weights for policy 1, policy_version 293409 (0.0011) [2023-12-27 07:40:45,191][20347] Updated weights for policy 0, policy_version 293704 (0.0011) [2023-12-27 07:40:45,252][20347] Updated weights for policy 0, policy_version 293714 (0.0008) [2023-12-27 07:40:45,311][20347] Updated weights for policy 0, policy_version 293724 (0.0010) [2023-12-27 07:40:45,411][20381] Updated weights for policy 1, policy_version 293419 (0.0011) [2023-12-27 07:40:45,459][20381] Updated weights for policy 1, policy_version 293429 (0.0009) [2023-12-27 07:40:45,507][20381] Updated weights for policy 1, policy_version 293439 (0.0010) [2023-12-27 07:40:45,867][20347] Updated weights for policy 0, policy_version 293734 (0.0007) [2023-12-27 07:40:45,914][20347] Updated weights for policy 0, policy_version 293744 (0.0005) [2023-12-27 07:40:45,964][20347] Updated weights for policy 0, policy_version 293754 (0.0005) [2023-12-27 07:40:46,217][20381] Updated weights for policy 1, policy_version 293449 (0.0010) [2023-12-27 07:40:46,274][20381] Updated weights for policy 1, policy_version 293459 (0.0011) [2023-12-27 07:40:46,331][20381] Updated weights for policy 1, policy_version 293469 (0.0011) [2023-12-27 07:40:46,394][20381] Updated weights for policy 1, policy_version 293479 (0.0011) [2023-12-27 07:40:46,529][20347] Updated weights for policy 0, policy_version 293764 (0.0007) [2023-12-27 07:40:46,578][20347] Updated weights for policy 0, policy_version 293774 (0.0010) [2023-12-27 07:40:46,630][20347] Updated weights for policy 0, policy_version 293784 (0.0010) [2023-12-27 07:40:47,086][20381] Updated weights for policy 1, policy_version 293489 (0.0011) [2023-12-27 07:40:47,132][20381] Updated weights for policy 1, policy_version 293499 (0.0010) [2023-12-27 07:40:47,181][20381] Updated weights for policy 1, policy_version 293509 (0.0011) [2023-12-27 07:40:47,275][20347] Updated weights for policy 0, policy_version 293794 (0.0009) [2023-12-27 07:40:47,336][20347] Updated weights for policy 0, policy_version 293804 (0.0006) [2023-12-27 07:40:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 150380544. Throughput: 0: 10462.5, 1: 10437.9. Samples: 150390472. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '236.140')] [2023-12-27 07:40:47,395][20347] Updated weights for policy 0, policy_version 293814 (0.0006) [2023-12-27 07:40:47,466][20347] Updated weights for policy 0, policy_version 293824 (0.0005) [2023-12-27 07:40:47,880][20381] Updated weights for policy 1, policy_version 293519 (0.0010) [2023-12-27 07:40:47,926][20381] Updated weights for policy 1, policy_version 293529 (0.0010) [2023-12-27 07:40:47,976][20381] Updated weights for policy 1, policy_version 293539 (0.0010) [2023-12-27 07:40:48,000][20347] Updated weights for policy 0, policy_version 293834 (0.0007) [2023-12-27 07:40:48,059][20347] Updated weights for policy 0, policy_version 293844 (0.0008) [2023-12-27 07:40:48,120][20347] Updated weights for policy 0, policy_version 293854 (0.0007) [2023-12-27 07:40:48,658][20381] Updated weights for policy 1, policy_version 293549 (0.0008) [2023-12-27 07:40:48,717][20381] Updated weights for policy 1, policy_version 293559 (0.0006) [2023-12-27 07:40:48,763][20347] Updated weights for policy 0, policy_version 293864 (0.0007) [2023-12-27 07:40:48,775][20381] Updated weights for policy 1, policy_version 293569 (0.0007) [2023-12-27 07:40:48,827][20347] Updated weights for policy 0, policy_version 293874 (0.0008) [2023-12-27 07:40:48,880][20347] Updated weights for policy 0, policy_version 293884 (0.0008) [2023-12-27 07:40:49,438][20381] Updated weights for policy 1, policy_version 293579 (0.0006) [2023-12-27 07:40:49,501][20381] Updated weights for policy 1, policy_version 293589 (0.0006) [2023-12-27 07:40:49,518][20347] Updated weights for policy 0, policy_version 293894 (0.0005) [2023-12-27 07:40:49,556][20381] Updated weights for policy 1, policy_version 293599 (0.0008) [2023-12-27 07:40:49,575][20347] Updated weights for policy 0, policy_version 293904 (0.0008) [2023-12-27 07:40:49,627][20347] Updated weights for policy 0, policy_version 293914 (0.0007) [2023-12-27 07:40:50,193][20381] Updated weights for policy 1, policy_version 293609 (0.0006) [2023-12-27 07:40:50,259][20381] Updated weights for policy 1, policy_version 293619 (0.0006) [2023-12-27 07:40:50,313][20381] Updated weights for policy 1, policy_version 293629 (0.0010) [2023-12-27 07:40:50,355][20347] Updated weights for policy 0, policy_version 293924 (0.0007) [2023-12-27 07:40:50,368][20381] Updated weights for policy 1, policy_version 293639 (0.0010) [2023-12-27 07:40:50,398][20347] Updated weights for policy 0, policy_version 293934 (0.0006) [2023-12-27 07:40:50,445][20347] Updated weights for policy 0, policy_version 293944 (0.0007) [2023-12-27 07:40:50,967][20381] Updated weights for policy 1, policy_version 293649 (0.0006) [2023-12-27 07:40:51,022][20381] Updated weights for policy 1, policy_version 293659 (0.0008) [2023-12-27 07:40:51,087][20381] Updated weights for policy 1, policy_version 293669 (0.0008) [2023-12-27 07:40:51,211][20347] Updated weights for policy 0, policy_version 293954 (0.0007) [2023-12-27 07:40:51,270][20347] Updated weights for policy 0, policy_version 293964 (0.0008) [2023-12-27 07:40:51,320][20347] Updated weights for policy 0, policy_version 293974 (0.0007) [2023-12-27 07:40:51,386][20347] Updated weights for policy 0, policy_version 293984 (0.0007) [2023-12-27 07:40:51,772][20381] Updated weights for policy 1, policy_version 293679 (0.0008) [2023-12-27 07:40:51,841][20381] Updated weights for policy 1, policy_version 293689 (0.0007) [2023-12-27 07:40:51,908][20381] Updated weights for policy 1, policy_version 293699 (0.0007) [2023-12-27 07:40:52,060][20347] Updated weights for policy 0, policy_version 293994 (0.0008) [2023-12-27 07:40:52,114][20347] Updated weights for policy 0, policy_version 294004 (0.0007) [2023-12-27 07:40:52,168][20347] Updated weights for policy 0, policy_version 294014 (0.0005) [2023-12-27 07:40:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 150487040. Throughput: 0: 10492.1, 1: 10430.3. Samples: 150455344. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '313.750')] [2023-12-27 07:40:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000294016_75284480.pth... [2023-12-27 07:40:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000293704_75202560.pth... [2023-12-27 07:40:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000292800_74973184.pth [2023-12-27 07:40:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000292488_74891264.pth [2023-12-27 07:40:52,654][20381] Updated weights for policy 1, policy_version 293709 (0.0009) [2023-12-27 07:40:52,712][20381] Updated weights for policy 1, policy_version 293719 (0.0008) [2023-12-27 07:40:52,767][20381] Updated weights for policy 1, policy_version 293729 (0.0006) [2023-12-27 07:40:52,773][20347] Updated weights for policy 0, policy_version 294024 (0.0007) [2023-12-27 07:40:52,844][20347] Updated weights for policy 0, policy_version 294034 (0.0009) [2023-12-27 07:40:52,910][20347] Updated weights for policy 0, policy_version 294044 (0.0008) [2023-12-27 07:40:53,405][20381] Updated weights for policy 1, policy_version 293739 (0.0006) [2023-12-27 07:40:53,459][20381] Updated weights for policy 1, policy_version 293749 (0.0007) [2023-12-27 07:40:53,509][20347] Updated weights for policy 0, policy_version 294054 (0.0009) [2023-12-27 07:40:53,511][20381] Updated weights for policy 1, policy_version 293759 (0.0007) [2023-12-27 07:40:53,563][20347] Updated weights for policy 0, policy_version 294064 (0.0010) [2023-12-27 07:40:53,617][20347] Updated weights for policy 0, policy_version 294074 (0.0010) [2023-12-27 07:40:54,110][20381] Updated weights for policy 1, policy_version 293769 (0.0005) [2023-12-27 07:40:54,171][20381] Updated weights for policy 1, policy_version 293779 (0.0008) [2023-12-27 07:40:54,229][20381] Updated weights for policy 1, policy_version 293789 (0.0008) [2023-12-27 07:40:54,251][20347] Updated weights for policy 0, policy_version 294084 (0.0010) [2023-12-27 07:40:54,282][20381] Updated weights for policy 1, policy_version 293799 (0.0010) [2023-12-27 07:40:54,303][20347] Updated weights for policy 0, policy_version 294094 (0.0009) [2023-12-27 07:40:54,352][20347] Updated weights for policy 0, policy_version 294104 (0.0007) [2023-12-27 07:40:54,997][20347] Updated weights for policy 0, policy_version 294114 (0.0008) [2023-12-27 07:40:55,027][20381] Updated weights for policy 1, policy_version 293809 (0.0007) [2023-12-27 07:40:55,047][20347] Updated weights for policy 0, policy_version 294124 (0.0008) [2023-12-27 07:40:55,082][20381] Updated weights for policy 1, policy_version 293819 (0.0007) [2023-12-27 07:40:55,101][20347] Updated weights for policy 0, policy_version 294134 (0.0007) [2023-12-27 07:40:55,146][20381] Updated weights for policy 1, policy_version 293829 (0.0011) [2023-12-27 07:40:55,158][20347] Updated weights for policy 0, policy_version 294144 (0.0005) [2023-12-27 07:40:55,719][20381] Updated weights for policy 1, policy_version 293839 (0.0007) [2023-12-27 07:40:55,727][20347] Updated weights for policy 0, policy_version 294154 (0.0008) [2023-12-27 07:40:55,772][20381] Updated weights for policy 1, policy_version 293849 (0.0005) [2023-12-27 07:40:55,776][20347] Updated weights for policy 0, policy_version 294164 (0.0008) [2023-12-27 07:40:55,829][20381] Updated weights for policy 1, policy_version 293859 (0.0005) [2023-12-27 07:40:55,829][20347] Updated weights for policy 0, policy_version 294174 (0.0007) [2023-12-27 07:40:56,453][20381] Updated weights for policy 1, policy_version 293869 (0.0008) [2023-12-27 07:40:56,499][20347] Updated weights for policy 0, policy_version 294184 (0.0006) [2023-12-27 07:40:56,511][20381] Updated weights for policy 1, policy_version 293879 (0.0011) [2023-12-27 07:40:56,547][20347] Updated weights for policy 0, policy_version 294194 (0.0005) [2023-12-27 07:40:56,568][20381] Updated weights for policy 1, policy_version 293889 (0.0010) [2023-12-27 07:40:56,592][20347] Updated weights for policy 0, policy_version 294204 (0.0005) [2023-12-27 07:40:57,233][20347] Updated weights for policy 0, policy_version 294214 (0.0006) [2023-12-27 07:40:57,249][20381] Updated weights for policy 1, policy_version 293899 (0.0010) [2023-12-27 07:40:57,283][20347] Updated weights for policy 0, policy_version 294224 (0.0006) [2023-12-27 07:40:57,301][20381] Updated weights for policy 1, policy_version 293909 (0.0009) [2023-12-27 07:40:57,344][20347] Updated weights for policy 0, policy_version 294234 (0.0006) [2023-12-27 07:40:57,361][20381] Updated weights for policy 1, policy_version 293919 (0.0009) [2023-12-27 07:40:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 150585344. Throughput: 0: 10553.5, 1: 10459.2. Samples: 150582000. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:40:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '224.940')] [2023-12-27 07:40:58,019][20347] Updated weights for policy 0, policy_version 294244 (0.0008) [2023-12-27 07:40:58,045][20381] Updated weights for policy 1, policy_version 293929 (0.0010) [2023-12-27 07:40:58,073][20347] Updated weights for policy 0, policy_version 294254 (0.0010) [2023-12-27 07:40:58,093][20381] Updated weights for policy 1, policy_version 293939 (0.0010) [2023-12-27 07:40:58,130][20347] Updated weights for policy 0, policy_version 294264 (0.0010) [2023-12-27 07:40:58,140][20381] Updated weights for policy 1, policy_version 293949 (0.0010) [2023-12-27 07:40:58,202][20381] Updated weights for policy 1, policy_version 293959 (0.0007) [2023-12-27 07:40:58,852][20347] Updated weights for policy 0, policy_version 294274 (0.0010) [2023-12-27 07:40:58,897][20381] Updated weights for policy 1, policy_version 293969 (0.0008) [2023-12-27 07:40:58,916][20347] Updated weights for policy 0, policy_version 294284 (0.0008) [2023-12-27 07:40:58,964][20381] Updated weights for policy 1, policy_version 293979 (0.0007) [2023-12-27 07:40:58,982][20347] Updated weights for policy 0, policy_version 294294 (0.0010) [2023-12-27 07:40:59,016][20381] Updated weights for policy 1, policy_version 293989 (0.0008) [2023-12-27 07:40:59,035][20347] Updated weights for policy 0, policy_version 294304 (0.0006) [2023-12-27 07:40:59,677][20381] Updated weights for policy 1, policy_version 293999 (0.0007) [2023-12-27 07:40:59,733][20381] Updated weights for policy 1, policy_version 294009 (0.0007) [2023-12-27 07:40:59,745][20347] Updated weights for policy 0, policy_version 294314 (0.0006) [2023-12-27 07:40:59,790][20381] Updated weights for policy 1, policy_version 294019 (0.0009) [2023-12-27 07:40:59,803][20347] Updated weights for policy 0, policy_version 294324 (0.0005) [2023-12-27 07:40:59,865][20347] Updated weights for policy 0, policy_version 294334 (0.0008) [2023-12-27 07:41:00,454][20381] Updated weights for policy 1, policy_version 294029 (0.0007) [2023-12-27 07:41:00,498][20381] Updated weights for policy 1, policy_version 294039 (0.0007) [2023-12-27 07:41:00,536][20347] Updated weights for policy 0, policy_version 294344 (0.0008) [2023-12-27 07:41:00,543][20381] Updated weights for policy 1, policy_version 294049 (0.0007) [2023-12-27 07:41:00,587][20347] Updated weights for policy 0, policy_version 294354 (0.0008) [2023-12-27 07:41:00,650][20347] Updated weights for policy 0, policy_version 294364 (0.0008) [2023-12-27 07:41:01,149][20381] Updated weights for policy 1, policy_version 294059 (0.0006) [2023-12-27 07:41:01,208][20381] Updated weights for policy 1, policy_version 294069 (0.0006) [2023-12-27 07:41:01,261][20381] Updated weights for policy 1, policy_version 294079 (0.0007) [2023-12-27 07:41:01,340][20347] Updated weights for policy 0, policy_version 294374 (0.0008) [2023-12-27 07:41:01,398][20347] Updated weights for policy 0, policy_version 294384 (0.0007) [2023-12-27 07:41:01,447][20347] Updated weights for policy 0, policy_version 294394 (0.0007) [2023-12-27 07:41:01,894][20381] Updated weights for policy 1, policy_version 294089 (0.0007) [2023-12-27 07:41:01,951][20381] Updated weights for policy 1, policy_version 294099 (0.0008) [2023-12-27 07:41:02,010][20381] Updated weights for policy 1, policy_version 294109 (0.0008) [2023-12-27 07:41:02,065][20381] Updated weights for policy 1, policy_version 294119 (0.0007) [2023-12-27 07:41:02,197][20347] Updated weights for policy 0, policy_version 294404 (0.0008) [2023-12-27 07:41:02,256][20347] Updated weights for policy 0, policy_version 294414 (0.0007) [2023-12-27 07:41:02,314][20347] Updated weights for policy 0, policy_version 294424 (0.0008) [2023-12-27 07:41:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 150700032. Throughput: 0: 10486.1, 1: 10439.1. Samples: 150704724. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:41:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '197.690')] [2023-12-27 07:41:02,769][20381] Updated weights for policy 1, policy_version 294129 (0.0008) [2023-12-27 07:41:02,825][20381] Updated weights for policy 1, policy_version 294139 (0.0008) [2023-12-27 07:41:02,877][20381] Updated weights for policy 1, policy_version 294149 (0.0008) [2023-12-27 07:41:02,912][20347] Updated weights for policy 0, policy_version 294434 (0.0008) [2023-12-27 07:41:02,962][20347] Updated weights for policy 0, policy_version 294444 (0.0008) [2023-12-27 07:41:03,012][20347] Updated weights for policy 0, policy_version 294454 (0.0008) [2023-12-27 07:41:03,061][20347] Updated weights for policy 0, policy_version 294464 (0.0007) [2023-12-27 07:41:03,552][20381] Updated weights for policy 1, policy_version 294159 (0.0007) [2023-12-27 07:41:03,598][20381] Updated weights for policy 1, policy_version 294169 (0.0008) [2023-12-27 07:41:03,659][20381] Updated weights for policy 1, policy_version 294179 (0.0008) [2023-12-27 07:41:03,772][20347] Updated weights for policy 0, policy_version 294474 (0.0008) [2023-12-27 07:41:03,827][20347] Updated weights for policy 0, policy_version 294484 (0.0007) [2023-12-27 07:41:03,888][20347] Updated weights for policy 0, policy_version 294494 (0.0009) [2023-12-27 07:41:04,432][20381] Updated weights for policy 1, policy_version 294189 (0.0008) [2023-12-27 07:41:04,496][20381] Updated weights for policy 1, policy_version 294199 (0.0008) [2023-12-27 07:41:04,560][20381] Updated weights for policy 1, policy_version 294209 (0.0008) [2023-12-27 07:41:04,598][20347] Updated weights for policy 0, policy_version 294504 (0.0008) [2023-12-27 07:41:04,659][20347] Updated weights for policy 0, policy_version 294514 (0.0008) [2023-12-27 07:41:04,720][20347] Updated weights for policy 0, policy_version 294524 (0.0008) [2023-12-27 07:41:05,218][20381] Updated weights for policy 1, policy_version 294219 (0.0007) [2023-12-27 07:41:05,274][20381] Updated weights for policy 1, policy_version 294229 (0.0008) [2023-12-27 07:41:05,327][20381] Updated weights for policy 1, policy_version 294239 (0.0008) [2023-12-27 07:41:05,376][20347] Updated weights for policy 0, policy_version 294534 (0.0006) [2023-12-27 07:41:05,439][20347] Updated weights for policy 0, policy_version 294544 (0.0007) [2023-12-27 07:41:05,498][20347] Updated weights for policy 0, policy_version 294554 (0.0008) [2023-12-27 07:41:05,968][20381] Updated weights for policy 1, policy_version 294249 (0.0007) [2023-12-27 07:41:06,030][20381] Updated weights for policy 1, policy_version 294259 (0.0008) [2023-12-27 07:41:06,087][20381] Updated weights for policy 1, policy_version 294269 (0.0007) [2023-12-27 07:41:06,088][20347] Updated weights for policy 0, policy_version 294564 (0.0007) [2023-12-27 07:41:06,147][20381] Updated weights for policy 1, policy_version 294279 (0.0006) [2023-12-27 07:41:06,152][20347] Updated weights for policy 0, policy_version 294574 (0.0007) [2023-12-27 07:41:06,213][20347] Updated weights for policy 0, policy_version 294584 (0.0006) [2023-12-27 07:41:06,835][20347] Updated weights for policy 0, policy_version 294594 (0.0006) [2023-12-27 07:41:06,882][20381] Updated weights for policy 1, policy_version 294289 (0.0007) [2023-12-27 07:41:06,889][20347] Updated weights for policy 0, policy_version 294604 (0.0005) [2023-12-27 07:41:06,934][20381] Updated weights for policy 1, policy_version 294299 (0.0007) [2023-12-27 07:41:06,939][20347] Updated weights for policy 0, policy_version 294614 (0.0005) [2023-12-27 07:41:06,988][20381] Updated weights for policy 1, policy_version 294309 (0.0007) [2023-12-27 07:41:06,998][20347] Updated weights for policy 0, policy_version 294624 (0.0006) [2023-12-27 07:41:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 150798336. Throughput: 0: 10412.6, 1: 10426.6. Samples: 150765384. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:41:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '305.430')] [2023-12-27 07:41:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000294312_75358208.pth... [2023-12-27 07:41:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000294624_75440128.pth... [2023-12-27 07:41:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000293408_75128832.pth [2023-12-27 07:41:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000293128_75055104.pth [2023-12-27 07:41:07,608][20347] Updated weights for policy 0, policy_version 294634 (0.0005) [2023-12-27 07:41:07,655][20347] Updated weights for policy 0, policy_version 294644 (0.0005) [2023-12-27 07:41:07,701][20347] Updated weights for policy 0, policy_version 294654 (0.0005) [2023-12-27 07:41:07,730][20381] Updated weights for policy 1, policy_version 294319 (0.0007) [2023-12-27 07:41:07,787][20381] Updated weights for policy 1, policy_version 294329 (0.0008) [2023-12-27 07:41:07,835][20381] Updated weights for policy 1, policy_version 294339 (0.0009) [2023-12-27 07:41:08,222][20347] Updated weights for policy 0, policy_version 294664 (0.0005) [2023-12-27 07:41:08,266][20347] Updated weights for policy 0, policy_version 294674 (0.0005) [2023-12-27 07:41:08,312][20347] Updated weights for policy 0, policy_version 294684 (0.0009) [2023-12-27 07:41:08,575][20381] Updated weights for policy 1, policy_version 294349 (0.0007) [2023-12-27 07:41:08,628][20381] Updated weights for policy 1, policy_version 294359 (0.0010) [2023-12-27 07:41:08,677][20381] Updated weights for policy 1, policy_version 294369 (0.0005) [2023-12-27 07:41:08,992][20347] Updated weights for policy 0, policy_version 294694 (0.0007) [2023-12-27 07:41:09,054][20347] Updated weights for policy 0, policy_version 294704 (0.0006) [2023-12-27 07:41:09,114][20347] Updated weights for policy 0, policy_version 294714 (0.0007) [2023-12-27 07:41:09,324][20381] Updated weights for policy 1, policy_version 294379 (0.0006) [2023-12-27 07:41:09,387][20381] Updated weights for policy 1, policy_version 294389 (0.0009) [2023-12-27 07:41:09,451][20381] Updated weights for policy 1, policy_version 294399 (0.0007) [2023-12-27 07:41:09,786][20347] Updated weights for policy 0, policy_version 294724 (0.0010) [2023-12-27 07:41:09,845][20347] Updated weights for policy 0, policy_version 294734 (0.0008) [2023-12-27 07:41:09,896][20347] Updated weights for policy 0, policy_version 294744 (0.0008) [2023-12-27 07:41:10,145][20381] Updated weights for policy 1, policy_version 294409 (0.0007) [2023-12-27 07:41:10,208][20381] Updated weights for policy 1, policy_version 294419 (0.0008) [2023-12-27 07:41:10,266][20381] Updated weights for policy 1, policy_version 294429 (0.0007) [2023-12-27 07:41:10,316][20381] Updated weights for policy 1, policy_version 294439 (0.0007) [2023-12-27 07:41:10,644][20347] Updated weights for policy 0, policy_version 294754 (0.0007) [2023-12-27 07:41:10,713][20347] Updated weights for policy 0, policy_version 294764 (0.0008) [2023-12-27 07:41:10,773][20347] Updated weights for policy 0, policy_version 294774 (0.0008) [2023-12-27 07:41:10,831][20347] Updated weights for policy 0, policy_version 294784 (0.0008) [2023-12-27 07:41:10,958][20381] Updated weights for policy 1, policy_version 294449 (0.0008) [2023-12-27 07:41:11,013][20381] Updated weights for policy 1, policy_version 294459 (0.0007) [2023-12-27 07:41:11,076][20381] Updated weights for policy 1, policy_version 294469 (0.0008) [2023-12-27 07:41:11,545][20347] Updated weights for policy 0, policy_version 294794 (0.0006) [2023-12-27 07:41:11,602][20347] Updated weights for policy 0, policy_version 294804 (0.0008) [2023-12-27 07:41:11,659][20347] Updated weights for policy 0, policy_version 294814 (0.0009) [2023-12-27 07:41:11,793][20381] Updated weights for policy 1, policy_version 294479 (0.0008) [2023-12-27 07:41:11,845][20381] Updated weights for policy 1, policy_version 294489 (0.0008) [2023-12-27 07:41:11,903][20381] Updated weights for policy 1, policy_version 294499 (0.0007) [2023-12-27 07:41:12,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 150896640. Throughput: 0: 10499.4, 1: 10304.2. Samples: 150889584. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-12-27 07:41:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '346.050')] [2023-12-27 07:41:12,386][20347] Updated weights for policy 0, policy_version 294824 (0.0006) [2023-12-27 07:41:12,455][20347] Updated weights for policy 0, policy_version 294834 (0.0006) [2023-12-27 07:41:12,507][20347] Updated weights for policy 0, policy_version 294844 (0.0007) [2023-12-27 07:41:12,586][20381] Updated weights for policy 1, policy_version 294509 (0.0006) [2023-12-27 07:41:12,651][20381] Updated weights for policy 1, policy_version 294519 (0.0010) [2023-12-27 07:41:12,701][20381] Updated weights for policy 1, policy_version 294529 (0.0011) [2023-12-27 07:41:13,216][20347] Updated weights for policy 0, policy_version 294854 (0.0009) [2023-12-27 07:41:13,258][20347] Updated weights for policy 0, policy_version 294864 (0.0010) [2023-12-27 07:41:13,307][20347] Updated weights for policy 0, policy_version 294874 (0.0010) [2023-12-27 07:41:13,322][20381] Updated weights for policy 1, policy_version 294539 (0.0010) [2023-12-27 07:41:13,377][20381] Updated weights for policy 1, policy_version 294549 (0.0010) [2023-12-27 07:41:13,432][20381] Updated weights for policy 1, policy_version 294559 (0.0010) [2023-12-27 07:41:13,932][20347] Updated weights for policy 0, policy_version 294884 (0.0008) [2023-12-27 07:41:13,980][20347] Updated weights for policy 0, policy_version 294894 (0.0007) [2023-12-27 07:41:14,028][20347] Updated weights for policy 0, policy_version 294904 (0.0007) [2023-12-27 07:41:14,085][20381] Updated weights for policy 1, policy_version 294569 (0.0011) [2023-12-27 07:41:14,140][20381] Updated weights for policy 1, policy_version 294579 (0.0010) [2023-12-27 07:41:14,195][20381] Updated weights for policy 1, policy_version 294589 (0.0010) [2023-12-27 07:41:14,254][20381] Updated weights for policy 1, policy_version 294599 (0.0010) [2023-12-27 07:41:14,624][20347] Updated weights for policy 0, policy_version 294914 (0.0007) [2023-12-27 07:41:14,677][20347] Updated weights for policy 0, policy_version 294924 (0.0007) [2023-12-27 07:41:14,730][20347] Updated weights for policy 0, policy_version 294934 (0.0008) [2023-12-27 07:41:14,777][20347] Updated weights for policy 0, policy_version 294944 (0.0008) [2023-12-27 07:41:14,909][20381] Updated weights for policy 1, policy_version 294609 (0.0011) [2023-12-27 07:41:14,968][20381] Updated weights for policy 1, policy_version 294619 (0.0011) [2023-12-27 07:41:15,029][20381] Updated weights for policy 1, policy_version 294629 (0.0007) [2023-12-27 07:41:15,469][20347] Updated weights for policy 0, policy_version 294954 (0.0009) [2023-12-27 07:41:15,529][20347] Updated weights for policy 0, policy_version 294964 (0.0005) [2023-12-27 07:41:15,582][20347] Updated weights for policy 0, policy_version 294974 (0.0009) [2023-12-27 07:41:15,678][20381] Updated weights for policy 1, policy_version 294639 (0.0010) [2023-12-27 07:41:15,731][20381] Updated weights for policy 1, policy_version 294649 (0.0010) [2023-12-27 07:41:15,777][20381] Updated weights for policy 1, policy_version 294659 (0.0010) [2023-12-27 07:41:16,222][20347] Updated weights for policy 0, policy_version 294984 (0.0010) [2023-12-27 07:41:16,284][20347] Updated weights for policy 0, policy_version 294994 (0.0011) [2023-12-27 07:41:16,339][20347] Updated weights for policy 0, policy_version 295004 (0.0010) [2023-12-27 07:41:16,389][20381] Updated weights for policy 1, policy_version 294669 (0.0010) [2023-12-27 07:41:16,442][20381] Updated weights for policy 1, policy_version 294679 (0.0005) [2023-12-27 07:41:16,494][20381] Updated weights for policy 1, policy_version 294689 (0.0005) [2023-12-27 07:41:16,981][20347] Updated weights for policy 0, policy_version 295014 (0.0008) [2023-12-27 07:41:17,036][20347] Updated weights for policy 0, policy_version 295024 (0.0010) [2023-12-27 07:41:17,094][20347] Updated weights for policy 0, policy_version 295034 (0.0010) [2023-12-27 07:41:17,139][20381] Updated weights for policy 1, policy_version 294699 (0.0008) [2023-12-27 07:41:17,194][20381] Updated weights for policy 1, policy_version 294709 (0.0008) [2023-12-27 07:41:17,252][20381] Updated weights for policy 1, policy_version 294719 (0.0005) [2023-12-27 07:41:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 151011328. Throughput: 0: 10513.8, 1: 10262.1. Samples: 151016100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '371.670')] [2023-12-27 07:41:17,754][20347] Updated weights for policy 0, policy_version 295044 (0.0010) [2023-12-27 07:41:17,812][20347] Updated weights for policy 0, policy_version 295054 (0.0010) [2023-12-27 07:41:17,861][20347] Updated weights for policy 0, policy_version 295064 (0.0010) [2023-12-27 07:41:17,887][20381] Updated weights for policy 1, policy_version 294729 (0.0006) [2023-12-27 07:41:17,940][20381] Updated weights for policy 1, policy_version 294739 (0.0011) [2023-12-27 07:41:17,997][20381] Updated weights for policy 1, policy_version 294749 (0.0011) [2023-12-27 07:41:18,051][20381] Updated weights for policy 1, policy_version 294759 (0.0011) [2023-12-27 07:41:18,554][20347] Updated weights for policy 0, policy_version 295074 (0.0010) [2023-12-27 07:41:18,612][20347] Updated weights for policy 0, policy_version 295084 (0.0010) [2023-12-27 07:41:18,667][20347] Updated weights for policy 0, policy_version 295094 (0.0011) [2023-12-27 07:41:18,729][20347] Updated weights for policy 0, policy_version 295104 (0.0010) [2023-12-27 07:41:18,765][20381] Updated weights for policy 1, policy_version 294769 (0.0011) [2023-12-27 07:41:18,824][20381] Updated weights for policy 1, policy_version 294779 (0.0011) [2023-12-27 07:41:18,877][20381] Updated weights for policy 1, policy_version 294789 (0.0011) [2023-12-27 07:41:19,412][20347] Updated weights for policy 0, policy_version 295114 (0.0010) [2023-12-27 07:41:19,467][20347] Updated weights for policy 0, policy_version 295124 (0.0010) [2023-12-27 07:41:19,525][20381] Updated weights for policy 1, policy_version 294799 (0.0008) [2023-12-27 07:41:19,532][20347] Updated weights for policy 0, policy_version 295134 (0.0011) [2023-12-27 07:41:19,586][20381] Updated weights for policy 1, policy_version 294809 (0.0007) [2023-12-27 07:41:19,648][20381] Updated weights for policy 1, policy_version 294819 (0.0008) [2023-12-27 07:41:20,229][20347] Updated weights for policy 0, policy_version 295144 (0.0010) [2023-12-27 07:41:20,291][20347] Updated weights for policy 0, policy_version 295154 (0.0011) [2023-12-27 07:41:20,347][20347] Updated weights for policy 0, policy_version 295164 (0.0010) [2023-12-27 07:41:20,348][20381] Updated weights for policy 1, policy_version 294829 (0.0009) [2023-12-27 07:41:20,400][20381] Updated weights for policy 1, policy_version 294839 (0.0007) [2023-12-27 07:41:20,450][20381] Updated weights for policy 1, policy_version 294849 (0.0007) [2023-12-27 07:41:21,004][20347] Updated weights for policy 0, policy_version 295174 (0.0009) [2023-12-27 07:41:21,075][20347] Updated weights for policy 0, policy_version 295184 (0.0008) [2023-12-27 07:41:21,129][20347] Updated weights for policy 0, policy_version 295194 (0.0008) [2023-12-27 07:41:21,174][20381] Updated weights for policy 1, policy_version 294859 (0.0007) [2023-12-27 07:41:21,234][20381] Updated weights for policy 1, policy_version 294869 (0.0007) [2023-12-27 07:41:21,301][20381] Updated weights for policy 1, policy_version 294879 (0.0008) [2023-12-27 07:41:21,874][20347] Updated weights for policy 0, policy_version 295204 (0.0007) [2023-12-27 07:41:21,931][20347] Updated weights for policy 0, policy_version 295214 (0.0007) [2023-12-27 07:41:21,991][20347] Updated weights for policy 0, policy_version 295224 (0.0011) [2023-12-27 07:41:22,024][20381] Updated weights for policy 1, policy_version 294889 (0.0008) [2023-12-27 07:41:22,075][20381] Updated weights for policy 1, policy_version 294899 (0.0006) [2023-12-27 07:41:22,131][20381] Updated weights for policy 1, policy_version 294909 (0.0007) [2023-12-27 07:41:22,193][20381] Updated weights for policy 1, policy_version 294919 (0.0007) [2023-12-27 07:41:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 151109632. Throughput: 0: 10498.3, 1: 10288.2. Samples: 151077124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '204.290')] [2023-12-27 07:41:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000294920_75513856.pth... [2023-12-27 07:41:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000295232_75595776.pth... [2023-12-27 07:41:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000293704_75202560.pth [2023-12-27 07:41:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000294016_75284480.pth [2023-12-27 07:41:22,705][20347] Updated weights for policy 0, policy_version 295234 (0.0010) [2023-12-27 07:41:22,764][20347] Updated weights for policy 0, policy_version 295244 (0.0010) [2023-12-27 07:41:22,824][20347] Updated weights for policy 0, policy_version 295254 (0.0010) [2023-12-27 07:41:22,873][20381] Updated weights for policy 1, policy_version 294929 (0.0010) [2023-12-27 07:41:22,883][20347] Updated weights for policy 0, policy_version 295264 (0.0010) [2023-12-27 07:41:22,925][20381] Updated weights for policy 1, policy_version 294939 (0.0008) [2023-12-27 07:41:22,973][20381] Updated weights for policy 1, policy_version 294949 (0.0010) [2023-12-27 07:41:23,542][20381] Updated weights for policy 1, policy_version 294959 (0.0010) [2023-12-27 07:41:23,555][20347] Updated weights for policy 0, policy_version 295274 (0.0010) [2023-12-27 07:41:23,599][20381] Updated weights for policy 1, policy_version 294969 (0.0010) [2023-12-27 07:41:23,603][20347] Updated weights for policy 0, policy_version 295284 (0.0010) [2023-12-27 07:41:23,646][20381] Updated weights for policy 1, policy_version 294979 (0.0007) [2023-12-27 07:41:23,652][20347] Updated weights for policy 0, policy_version 295294 (0.0010) [2023-12-27 07:41:24,220][20381] Updated weights for policy 1, policy_version 294989 (0.0008) [2023-12-27 07:41:24,220][20347] Updated weights for policy 0, policy_version 295304 (0.0006) [2023-12-27 07:41:24,274][20381] Updated weights for policy 1, policy_version 294999 (0.0009) [2023-12-27 07:41:24,276][20347] Updated weights for policy 0, policy_version 295314 (0.0006) [2023-12-27 07:41:24,327][20381] Updated weights for policy 1, policy_version 295009 (0.0010) [2023-12-27 07:41:24,328][20347] Updated weights for policy 0, policy_version 295324 (0.0005) [2023-12-27 07:41:24,958][20347] Updated weights for policy 0, policy_version 295334 (0.0009) [2023-12-27 07:41:24,985][20381] Updated weights for policy 1, policy_version 295019 (0.0010) [2023-12-27 07:41:25,022][20347] Updated weights for policy 0, policy_version 295344 (0.0006) [2023-12-27 07:41:25,031][20381] Updated weights for policy 1, policy_version 295029 (0.0010) [2023-12-27 07:41:25,080][20347] Updated weights for policy 0, policy_version 295354 (0.0006) [2023-12-27 07:41:25,087][20381] Updated weights for policy 1, policy_version 295039 (0.0011) [2023-12-27 07:41:25,697][20347] Updated weights for policy 0, policy_version 295364 (0.0008) [2023-12-27 07:41:25,750][20347] Updated weights for policy 0, policy_version 295374 (0.0010) [2023-12-27 07:41:25,790][20381] Updated weights for policy 1, policy_version 295049 (0.0010) [2023-12-27 07:41:25,813][20347] Updated weights for policy 0, policy_version 295384 (0.0010) [2023-12-27 07:41:25,843][20381] Updated weights for policy 1, policy_version 295059 (0.0011) [2023-12-27 07:41:25,898][20381] Updated weights for policy 1, policy_version 295069 (0.0010) [2023-12-27 07:41:25,955][20381] Updated weights for policy 1, policy_version 295079 (0.0010) [2023-12-27 07:41:26,432][20347] Updated weights for policy 0, policy_version 295394 (0.0010) [2023-12-27 07:41:26,483][20347] Updated weights for policy 0, policy_version 295404 (0.0007) [2023-12-27 07:41:26,539][20347] Updated weights for policy 0, policy_version 295414 (0.0010) [2023-12-27 07:41:26,593][20347] Updated weights for policy 0, policy_version 295424 (0.0010) [2023-12-27 07:41:26,602][20381] Updated weights for policy 1, policy_version 295089 (0.0006) [2023-12-27 07:41:26,646][20381] Updated weights for policy 1, policy_version 295099 (0.0007) [2023-12-27 07:41:26,692][20381] Updated weights for policy 1, policy_version 295109 (0.0006) [2023-12-27 07:41:27,252][20347] Updated weights for policy 0, policy_version 295434 (0.0007) [2023-12-27 07:41:27,303][20347] Updated weights for policy 0, policy_version 295444 (0.0009) [2023-12-27 07:41:27,355][20347] Updated weights for policy 0, policy_version 295454 (0.0010) [2023-12-27 07:41:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 151216128. Throughput: 0: 10508.7, 1: 10329.1. Samples: 151202080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '166.200')] [2023-12-27 07:41:27,370][20381] Updated weights for policy 1, policy_version 295119 (0.0009) [2023-12-27 07:41:27,420][20381] Updated weights for policy 1, policy_version 295129 (0.0010) [2023-12-27 07:41:27,473][20381] Updated weights for policy 1, policy_version 295139 (0.0010) [2023-12-27 07:41:27,950][20347] Updated weights for policy 0, policy_version 295464 (0.0006) [2023-12-27 07:41:27,993][20347] Updated weights for policy 0, policy_version 295474 (0.0005) [2023-12-27 07:41:28,052][20347] Updated weights for policy 0, policy_version 295484 (0.0007) [2023-12-27 07:41:28,148][20381] Updated weights for policy 1, policy_version 295149 (0.0010) [2023-12-27 07:41:28,212][20381] Updated weights for policy 1, policy_version 295159 (0.0010) [2023-12-27 07:41:28,270][20381] Updated weights for policy 1, policy_version 295169 (0.0010) [2023-12-27 07:41:28,672][20347] Updated weights for policy 0, policy_version 295494 (0.0006) [2023-12-27 07:41:28,721][20347] Updated weights for policy 0, policy_version 295504 (0.0005) [2023-12-27 07:41:28,772][20347] Updated weights for policy 0, policy_version 295514 (0.0007) [2023-12-27 07:41:28,964][20381] Updated weights for policy 1, policy_version 295179 (0.0010) [2023-12-27 07:41:29,018][20381] Updated weights for policy 1, policy_version 295189 (0.0010) [2023-12-27 07:41:29,070][20381] Updated weights for policy 1, policy_version 295199 (0.0010) [2023-12-27 07:41:29,392][20347] Updated weights for policy 0, policy_version 295524 (0.0007) [2023-12-27 07:41:29,452][20347] Updated weights for policy 0, policy_version 295534 (0.0007) [2023-12-27 07:41:29,501][20347] Updated weights for policy 0, policy_version 295544 (0.0006) [2023-12-27 07:41:29,786][20381] Updated weights for policy 1, policy_version 295209 (0.0010) [2023-12-27 07:41:29,847][20381] Updated weights for policy 1, policy_version 295219 (0.0008) [2023-12-27 07:41:29,906][20381] Updated weights for policy 1, policy_version 295229 (0.0008) [2023-12-27 07:41:29,960][20381] Updated weights for policy 1, policy_version 295239 (0.0007) [2023-12-27 07:41:30,150][20347] Updated weights for policy 0, policy_version 295554 (0.0006) [2023-12-27 07:41:30,202][20347] Updated weights for policy 0, policy_version 295564 (0.0007) [2023-12-27 07:41:30,253][20347] Updated weights for policy 0, policy_version 295574 (0.0008) [2023-12-27 07:41:30,307][20347] Updated weights for policy 0, policy_version 295584 (0.0007) [2023-12-27 07:41:30,669][20381] Updated weights for policy 1, policy_version 295249 (0.0007) [2023-12-27 07:41:30,725][20381] Updated weights for policy 1, policy_version 295259 (0.0007) [2023-12-27 07:41:30,772][20381] Updated weights for policy 1, policy_version 295269 (0.0008) [2023-12-27 07:41:30,958][20347] Updated weights for policy 0, policy_version 295594 (0.0005) [2023-12-27 07:41:31,025][20347] Updated weights for policy 0, policy_version 295604 (0.0007) [2023-12-27 07:41:31,084][20347] Updated weights for policy 0, policy_version 295614 (0.0010) [2023-12-27 07:41:31,444][20381] Updated weights for policy 1, policy_version 295279 (0.0005) [2023-12-27 07:41:31,499][20381] Updated weights for policy 1, policy_version 295289 (0.0010) [2023-12-27 07:41:31,560][20381] Updated weights for policy 1, policy_version 295299 (0.0011) [2023-12-27 07:41:31,716][20347] Updated weights for policy 0, policy_version 295624 (0.0011) [2023-12-27 07:41:31,774][20347] Updated weights for policy 0, policy_version 295634 (0.0009) [2023-12-27 07:41:31,831][20347] Updated weights for policy 0, policy_version 295644 (0.0007) [2023-12-27 07:41:32,325][20381] Updated weights for policy 1, policy_version 295309 (0.0010) [2023-12-27 07:41:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20827.2). Total num frames: 151314432. Throughput: 0: 10524.4, 1: 10330.0. Samples: 151328916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '204.300')] [2023-12-27 07:41:32,392][20381] Updated weights for policy 1, policy_version 295319 (0.0008) [2023-12-27 07:41:32,447][20381] Updated weights for policy 1, policy_version 295329 (0.0007) [2023-12-27 07:41:32,497][20347] Updated weights for policy 0, policy_version 295654 (0.0009) [2023-12-27 07:41:32,553][20347] Updated weights for policy 0, policy_version 295664 (0.0009) [2023-12-27 07:41:32,604][20347] Updated weights for policy 0, policy_version 295674 (0.0007) [2023-12-27 07:41:33,026][20381] Updated weights for policy 1, policy_version 295339 (0.0007) [2023-12-27 07:41:33,084][20381] Updated weights for policy 1, policy_version 295349 (0.0008) [2023-12-27 07:41:33,145][20381] Updated weights for policy 1, policy_version 295359 (0.0008) [2023-12-27 07:41:33,282][20347] Updated weights for policy 0, policy_version 295684 (0.0007) [2023-12-27 07:41:33,330][20347] Updated weights for policy 0, policy_version 295694 (0.0007) [2023-12-27 07:41:33,392][20347] Updated weights for policy 0, policy_version 295704 (0.0006) [2023-12-27 07:41:33,820][20381] Updated weights for policy 1, policy_version 295369 (0.0008) [2023-12-27 07:41:33,878][20381] Updated weights for policy 1, policy_version 295379 (0.0007) [2023-12-27 07:41:33,929][20381] Updated weights for policy 1, policy_version 295389 (0.0008) [2023-12-27 07:41:33,992][20381] Updated weights for policy 1, policy_version 295399 (0.0007) [2023-12-27 07:41:34,062][20347] Updated weights for policy 0, policy_version 295714 (0.0007) [2023-12-27 07:41:34,118][20347] Updated weights for policy 0, policy_version 295724 (0.0008) [2023-12-27 07:41:34,180][20347] Updated weights for policy 0, policy_version 295734 (0.0008) [2023-12-27 07:41:34,185][20291] KL-divergence is very high: 156.0189 [2023-12-27 07:41:34,234][20291] KL-divergence is very high: 426.1599 [2023-12-27 07:41:34,240][20347] Updated weights for policy 0, policy_version 295744 (0.0008) [2023-12-27 07:41:34,669][20381] Updated weights for policy 1, policy_version 295409 (0.0008) [2023-12-27 07:41:34,721][20381] Updated weights for policy 1, policy_version 295419 (0.0008) [2023-12-27 07:41:34,774][20381] Updated weights for policy 1, policy_version 295429 (0.0008) [2023-12-27 07:41:34,990][20347] Updated weights for policy 0, policy_version 295754 (0.0008) [2023-12-27 07:41:35,042][20347] Updated weights for policy 0, policy_version 295764 (0.0008) [2023-12-27 07:41:35,092][20347] Updated weights for policy 0, policy_version 295774 (0.0008) [2023-12-27 07:41:35,472][20381] Updated weights for policy 1, policy_version 295439 (0.0008) [2023-12-27 07:41:35,529][20381] Updated weights for policy 1, policy_version 295449 (0.0008) [2023-12-27 07:41:35,592][20381] Updated weights for policy 1, policy_version 295459 (0.0007) [2023-12-27 07:41:35,806][20347] Updated weights for policy 0, policy_version 295784 (0.0007) [2023-12-27 07:41:35,864][20347] Updated weights for policy 0, policy_version 295794 (0.0007) [2023-12-27 07:41:35,913][20347] Updated weights for policy 0, policy_version 295804 (0.0007) [2023-12-27 07:41:36,294][20381] Updated weights for policy 1, policy_version 295469 (0.0008) [2023-12-27 07:41:36,353][20381] Updated weights for policy 1, policy_version 295479 (0.0007) [2023-12-27 07:41:36,406][20381] Updated weights for policy 1, policy_version 295489 (0.0008) [2023-12-27 07:41:36,624][20347] Updated weights for policy 0, policy_version 295814 (0.0006) [2023-12-27 07:41:36,682][20347] Updated weights for policy 0, policy_version 295824 (0.0006) [2023-12-27 07:41:36,740][20347] Updated weights for policy 0, policy_version 295834 (0.0005) [2023-12-27 07:41:37,126][20381] Updated weights for policy 1, policy_version 295499 (0.0009) [2023-12-27 07:41:37,184][20381] Updated weights for policy 1, policy_version 295509 (0.0007) [2023-12-27 07:41:37,246][20381] Updated weights for policy 1, policy_version 295519 (0.0008) [2023-12-27 07:41:37,322][20347] Updated weights for policy 0, policy_version 295844 (0.0006) [2023-12-27 07:41:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 151420928. Throughput: 0: 10460.5, 1: 10316.1. Samples: 151390292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:37,363][19170] Avg episode reward: [(0, '981.460'), (1, '224.630')] [2023-12-27 07:41:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000295528_75669504.pth... [2023-12-27 07:41:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000294312_75358208.pth [2023-12-27 07:41:37,375][20291] KL-divergence is very high: 186.7308 [2023-12-27 07:41:37,385][20347] Updated weights for policy 0, policy_version 295854 (0.0006) [2023-12-27 07:41:37,419][20291] KL-divergence is very high: 308.7368 [2023-12-27 07:41:37,442][20347] Updated weights for policy 0, policy_version 295864 (0.0008) [2023-12-27 07:41:37,463][20291] KL-divergence is very high: 157.7290 [2023-12-27 07:41:37,481][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000295872_75759616.pth... [2023-12-27 07:41:37,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000294624_75440128.pth [2023-12-27 07:41:37,993][20381] Updated weights for policy 1, policy_version 295529 (0.0007) [2023-12-27 07:41:38,048][20381] Updated weights for policy 1, policy_version 295539 (0.0008) [2023-12-27 07:41:38,055][20347] Updated weights for policy 0, policy_version 295874 (0.0007) [2023-12-27 07:41:38,104][20381] Updated weights for policy 1, policy_version 295549 (0.0008) [2023-12-27 07:41:38,110][20347] Updated weights for policy 0, policy_version 295884 (0.0005) [2023-12-27 07:41:38,154][20381] Updated weights for policy 1, policy_version 295559 (0.0005) [2023-12-27 07:41:38,159][20347] Updated weights for policy 0, policy_version 295894 (0.0005) [2023-12-27 07:41:38,208][20347] Updated weights for policy 0, policy_version 295904 (0.0007) [2023-12-27 07:41:38,830][20381] Updated weights for policy 1, policy_version 295569 (0.0005) [2023-12-27 07:41:38,887][20381] Updated weights for policy 1, policy_version 295579 (0.0006) [2023-12-27 07:41:38,935][20347] Updated weights for policy 0, policy_version 295914 (0.0006) [2023-12-27 07:41:38,940][20381] Updated weights for policy 1, policy_version 295589 (0.0006) [2023-12-27 07:41:38,992][20347] Updated weights for policy 0, policy_version 295924 (0.0007) [2023-12-27 07:41:39,044][20347] Updated weights for policy 0, policy_version 295934 (0.0008) [2023-12-27 07:41:39,627][20381] Updated weights for policy 1, policy_version 295599 (0.0005) [2023-12-27 07:41:39,687][20381] Updated weights for policy 1, policy_version 295609 (0.0007) [2023-12-27 07:41:39,716][20347] Updated weights for policy 0, policy_version 295944 (0.0006) [2023-12-27 07:41:39,748][20381] Updated weights for policy 1, policy_version 295619 (0.0006) [2023-12-27 07:41:39,772][20347] Updated weights for policy 0, policy_version 295954 (0.0005) [2023-12-27 07:41:39,832][20347] Updated weights for policy 0, policy_version 295964 (0.0007) [2023-12-27 07:41:40,383][20381] Updated weights for policy 1, policy_version 295629 (0.0007) [2023-12-27 07:41:40,445][20381] Updated weights for policy 1, policy_version 295639 (0.0008) [2023-12-27 07:41:40,502][20381] Updated weights for policy 1, policy_version 295649 (0.0008) [2023-12-27 07:41:40,579][20347] Updated weights for policy 0, policy_version 295974 (0.0007) [2023-12-27 07:41:40,639][20347] Updated weights for policy 0, policy_version 295984 (0.0006) [2023-12-27 07:41:40,700][20347] Updated weights for policy 0, policy_version 295994 (0.0007) [2023-12-27 07:41:41,194][20381] Updated weights for policy 1, policy_version 295659 (0.0008) [2023-12-27 07:41:41,255][20381] Updated weights for policy 1, policy_version 295669 (0.0008) [2023-12-27 07:41:41,319][20381] Updated weights for policy 1, policy_version 295679 (0.0009) [2023-12-27 07:41:41,372][20347] Updated weights for policy 0, policy_version 296004 (0.0007) [2023-12-27 07:41:41,436][20347] Updated weights for policy 0, policy_version 296014 (0.0007) [2023-12-27 07:41:41,498][20347] Updated weights for policy 0, policy_version 296024 (0.0006) [2023-12-27 07:41:42,026][20381] Updated weights for policy 1, policy_version 295689 (0.0008) [2023-12-27 07:41:42,091][20381] Updated weights for policy 1, policy_version 295699 (0.0007) [2023-12-27 07:41:42,158][20381] Updated weights for policy 1, policy_version 295709 (0.0007) [2023-12-27 07:41:42,221][20381] Updated weights for policy 1, policy_version 295719 (0.0007) [2023-12-27 07:41:42,229][20347] Updated weights for policy 0, policy_version 296034 (0.0008) [2023-12-27 07:41:42,291][20347] Updated weights for policy 0, policy_version 296044 (0.0010) [2023-12-27 07:41:42,352][20347] Updated weights for policy 0, policy_version 296054 (0.0010) [2023-12-27 07:41:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 151519232. Throughput: 0: 10403.2, 1: 10247.5. Samples: 151511284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:42,363][19170] Avg episode reward: [(0, '981.460'), (1, '270.400')] [2023-12-27 07:41:42,408][20347] Updated weights for policy 0, policy_version 296064 (0.0009) [2023-12-27 07:41:42,912][20381] Updated weights for policy 1, policy_version 295729 (0.0006) [2023-12-27 07:41:42,965][20381] Updated weights for policy 1, policy_version 295739 (0.0007) [2023-12-27 07:41:43,024][20381] Updated weights for policy 1, policy_version 295749 (0.0010) [2023-12-27 07:41:43,040][20347] Updated weights for policy 0, policy_version 296074 (0.0006) [2023-12-27 07:41:43,089][20347] Updated weights for policy 0, policy_version 296084 (0.0007) [2023-12-27 07:41:43,136][20347] Updated weights for policy 0, policy_version 296094 (0.0010) [2023-12-27 07:41:43,604][20381] Updated weights for policy 1, policy_version 295759 (0.0007) [2023-12-27 07:41:43,650][20381] Updated weights for policy 1, policy_version 295769 (0.0008) [2023-12-27 07:41:43,695][20381] Updated weights for policy 1, policy_version 295779 (0.0007) [2023-12-27 07:41:43,839][20347] Updated weights for policy 0, policy_version 296104 (0.0007) [2023-12-27 07:41:43,843][20291] KL-divergence is very high: 176.4257 [2023-12-27 07:41:43,861][20291] KL-divergence is very high: 357.0923 [2023-12-27 07:41:43,866][20291] KL-divergence is very high: 144.7387 [2023-12-27 07:41:43,889][20291] KL-divergence is very high: 206.6760 [2023-12-27 07:41:43,894][20347] Updated weights for policy 0, policy_version 296114 (0.0007) [2023-12-27 07:41:43,907][20291] KL-divergence is very high: 182.2225 [2023-12-27 07:41:43,934][20291] KL-divergence is very high: 217.8063 [2023-12-27 07:41:43,949][20347] Updated weights for policy 0, policy_version 296124 (0.0008) [2023-12-27 07:41:43,951][20291] KL-divergence is very high: 354.0220 [2023-12-27 07:41:43,956][20291] KL-divergence is very high: 129.1685 [2023-12-27 07:41:44,343][20381] Updated weights for policy 1, policy_version 295789 (0.0007) [2023-12-27 07:41:44,403][20381] Updated weights for policy 1, policy_version 295799 (0.0007) [2023-12-27 07:41:44,455][20381] Updated weights for policy 1, policy_version 295809 (0.0007) [2023-12-27 07:41:44,642][20347] Updated weights for policy 0, policy_version 296134 (0.0007) [2023-12-27 07:41:44,684][20291] KL-divergence is very high: 146.9658 [2023-12-27 07:41:44,690][20291] KL-divergence is very high: 118.1426 [2023-12-27 07:41:44,699][20347] Updated weights for policy 0, policy_version 296144 (0.0007) [2023-12-27 07:41:44,746][20347] Updated weights for policy 0, policy_version 296154 (0.0007) [2023-12-27 07:41:45,159][20381] Updated weights for policy 1, policy_version 295819 (0.0009) [2023-12-27 07:41:45,208][20381] Updated weights for policy 1, policy_version 295829 (0.0010) [2023-12-27 07:41:45,256][20381] Updated weights for policy 1, policy_version 295839 (0.0010) [2023-12-27 07:41:45,425][20347] Updated weights for policy 0, policy_version 296164 (0.0006) [2023-12-27 07:41:45,482][20347] Updated weights for policy 0, policy_version 296174 (0.0005) [2023-12-27 07:41:45,535][20347] Updated weights for policy 0, policy_version 296184 (0.0006) [2023-12-27 07:41:45,953][20381] Updated weights for policy 1, policy_version 295849 (0.0008) [2023-12-27 07:41:46,002][20381] Updated weights for policy 1, policy_version 295859 (0.0010) [2023-12-27 07:41:46,052][20381] Updated weights for policy 1, policy_version 295869 (0.0010) [2023-12-27 07:41:46,107][20381] Updated weights for policy 1, policy_version 295879 (0.0010) [2023-12-27 07:41:46,192][20347] Updated weights for policy 0, policy_version 296194 (0.0008) [2023-12-27 07:41:46,246][20347] Updated weights for policy 0, policy_version 296204 (0.0007) [2023-12-27 07:41:46,303][20347] Updated weights for policy 0, policy_version 296214 (0.0007) [2023-12-27 07:41:46,353][20347] Updated weights for policy 0, policy_version 296224 (0.0007) [2023-12-27 07:41:46,763][20381] Updated weights for policy 1, policy_version 295889 (0.0010) [2023-12-27 07:41:46,812][20381] Updated weights for policy 1, policy_version 295899 (0.0010) [2023-12-27 07:41:46,870][20381] Updated weights for policy 1, policy_version 295909 (0.0010) [2023-12-27 07:41:47,003][20291] KL-divergence is very high: 101.0442 [2023-12-27 07:41:47,022][20347] Updated weights for policy 0, policy_version 296234 (0.0005) [2023-12-27 07:41:47,049][20291] KL-divergence is very high: 110.6272 [2023-12-27 07:41:47,076][20347] Updated weights for policy 0, policy_version 296244 (0.0005) [2023-12-27 07:41:47,131][20347] Updated weights for policy 0, policy_version 296254 (0.0006) [2023-12-27 07:41:47,137][20291] KL-divergence is very high: 145.8489 [2023-12-27 07:41:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 151625728. Throughput: 0: 10425.9, 1: 10250.9. Samples: 151635184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:47,363][19170] Avg episode reward: [(0, '803.170'), (1, '159.420')] [2023-12-27 07:41:47,570][20381] Updated weights for policy 1, policy_version 295919 (0.0010) [2023-12-27 07:41:47,619][20381] Updated weights for policy 1, policy_version 295929 (0.0010) [2023-12-27 07:41:47,667][20381] Updated weights for policy 1, policy_version 295939 (0.0010) [2023-12-27 07:41:47,761][20347] Updated weights for policy 0, policy_version 296264 (0.0007) [2023-12-27 07:41:47,821][20347] Updated weights for policy 0, policy_version 296274 (0.0007) [2023-12-27 07:41:47,871][20347] Updated weights for policy 0, policy_version 296284 (0.0007) [2023-12-27 07:41:48,339][20381] Updated weights for policy 1, policy_version 295949 (0.0011) [2023-12-27 07:41:48,399][20381] Updated weights for policy 1, policy_version 295959 (0.0007) [2023-12-27 07:41:48,456][20381] Updated weights for policy 1, policy_version 295969 (0.0007) [2023-12-27 07:41:48,605][20347] Updated weights for policy 0, policy_version 296294 (0.0007) [2023-12-27 07:41:48,665][20347] Updated weights for policy 0, policy_version 296304 (0.0009) [2023-12-27 07:41:48,729][20347] Updated weights for policy 0, policy_version 296314 (0.0008) [2023-12-27 07:41:49,158][20381] Updated weights for policy 1, policy_version 295979 (0.0008) [2023-12-27 07:41:49,212][20381] Updated weights for policy 1, policy_version 295989 (0.0006) [2023-12-27 07:41:49,276][20381] Updated weights for policy 1, policy_version 295999 (0.0008) [2023-12-27 07:41:49,451][20347] Updated weights for policy 0, policy_version 296324 (0.0007) [2023-12-27 07:41:49,510][20347] Updated weights for policy 0, policy_version 296334 (0.0008) [2023-12-27 07:41:49,515][20291] KL-divergence is very high: 694.6679 [2023-12-27 07:41:49,559][20347] Updated weights for policy 0, policy_version 296344 (0.0008) [2023-12-27 07:41:49,597][20291] KL-divergence is very high: 291.4890 [2023-12-27 07:41:49,984][20381] Updated weights for policy 1, policy_version 296009 (0.0007) [2023-12-27 07:41:50,043][20381] Updated weights for policy 1, policy_version 296019 (0.0008) [2023-12-27 07:41:50,100][20381] Updated weights for policy 1, policy_version 296029 (0.0008) [2023-12-27 07:41:50,159][20381] Updated weights for policy 1, policy_version 296039 (0.0011) [2023-12-27 07:41:50,293][20347] Updated weights for policy 0, policy_version 296354 (0.0007) [2023-12-27 07:41:50,337][20291] KL-divergence is very high: 106.2646 [2023-12-27 07:41:50,356][20347] Updated weights for policy 0, policy_version 296364 (0.0007) [2023-12-27 07:41:50,386][20291] KL-divergence is very high: 333.9645 [2023-12-27 07:41:50,398][20291] KL-divergence is very high: 369.7804 [2023-12-27 07:41:50,403][20291] KL-divergence is very high: 322.6266 [2023-12-27 07:41:50,416][20347] Updated weights for policy 0, policy_version 296374 (0.0007) [2023-12-27 07:41:50,469][20347] Updated weights for policy 0, policy_version 296384 (0.0007) [2023-12-27 07:41:50,860][20381] Updated weights for policy 1, policy_version 296049 (0.0010) [2023-12-27 07:41:50,921][20381] Updated weights for policy 1, policy_version 296059 (0.0010) [2023-12-27 07:41:50,982][20381] Updated weights for policy 1, policy_version 296069 (0.0011) [2023-12-27 07:41:51,157][20347] Updated weights for policy 0, policy_version 296394 (0.0007) [2023-12-27 07:41:51,212][20347] Updated weights for policy 0, policy_version 296404 (0.0008) [2023-12-27 07:41:51,271][20347] Updated weights for policy 0, policy_version 296414 (0.0008) [2023-12-27 07:41:51,654][20381] Updated weights for policy 1, policy_version 296079 (0.0011) [2023-12-27 07:41:51,703][20381] Updated weights for policy 1, policy_version 296089 (0.0009) [2023-12-27 07:41:51,765][20381] Updated weights for policy 1, policy_version 296099 (0.0009) [2023-12-27 07:41:51,991][20347] Updated weights for policy 0, policy_version 296424 (0.0008) [2023-12-27 07:41:52,052][20347] Updated weights for policy 0, policy_version 296434 (0.0007) [2023-12-27 07:41:52,114][20347] Updated weights for policy 0, policy_version 296444 (0.0008) [2023-12-27 07:41:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 151724032. Throughput: 0: 10401.8, 1: 10241.2. Samples: 151694316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:52,363][19170] Avg episode reward: [(0, '752.600'), (1, '247.960')] [2023-12-27 07:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000296104_75816960.pth... [2023-12-27 07:41:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000296448_75907072.pth... [2023-12-27 07:41:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000294920_75513856.pth [2023-12-27 07:41:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000295232_75595776.pth [2023-12-27 07:41:52,470][20381] Updated weights for policy 1, policy_version 296109 (0.0010) [2023-12-27 07:41:52,528][20381] Updated weights for policy 1, policy_version 296119 (0.0009) [2023-12-27 07:41:52,591][20381] Updated weights for policy 1, policy_version 296129 (0.0010) [2023-12-27 07:41:52,766][20347] Updated weights for policy 0, policy_version 296454 (0.0008) [2023-12-27 07:41:52,821][20347] Updated weights for policy 0, policy_version 296464 (0.0008) [2023-12-27 07:41:52,878][20347] Updated weights for policy 0, policy_version 296474 (0.0008) [2023-12-27 07:41:53,175][20381] Updated weights for policy 1, policy_version 296139 (0.0009) [2023-12-27 07:41:53,234][20381] Updated weights for policy 1, policy_version 296149 (0.0006) [2023-12-27 07:41:53,289][20381] Updated weights for policy 1, policy_version 296159 (0.0005) [2023-12-27 07:41:53,571][20347] Updated weights for policy 0, policy_version 296484 (0.0007) [2023-12-27 07:41:53,619][20347] Updated weights for policy 0, policy_version 296494 (0.0008) [2023-12-27 07:41:53,674][20347] Updated weights for policy 0, policy_version 296504 (0.0010) [2023-12-27 07:41:53,893][20381] Updated weights for policy 1, policy_version 296169 (0.0006) [2023-12-27 07:41:53,946][20381] Updated weights for policy 1, policy_version 296179 (0.0008) [2023-12-27 07:41:54,002][20381] Updated weights for policy 1, policy_version 296189 (0.0007) [2023-12-27 07:41:54,047][20381] Updated weights for policy 1, policy_version 296199 (0.0007) [2023-12-27 07:41:54,296][20347] Updated weights for policy 0, policy_version 296514 (0.0008) [2023-12-27 07:41:54,351][20347] Updated weights for policy 0, policy_version 296524 (0.0006) [2023-12-27 07:41:54,408][20347] Updated weights for policy 0, policy_version 296534 (0.0005) [2023-12-27 07:41:54,464][20347] Updated weights for policy 0, policy_version 296544 (0.0005) [2023-12-27 07:41:54,713][20381] Updated weights for policy 1, policy_version 296209 (0.0009) [2023-12-27 07:41:54,758][20381] Updated weights for policy 1, policy_version 296219 (0.0007) [2023-12-27 07:41:54,809][20381] Updated weights for policy 1, policy_version 296229 (0.0009) [2023-12-27 07:41:54,998][20347] Updated weights for policy 0, policy_version 296554 (0.0007) [2023-12-27 07:41:55,055][20347] Updated weights for policy 0, policy_version 296564 (0.0008) [2023-12-27 07:41:55,111][20347] Updated weights for policy 0, policy_version 296574 (0.0007) [2023-12-27 07:41:55,444][20381] Updated weights for policy 1, policy_version 296239 (0.0007) [2023-12-27 07:41:55,492][20381] Updated weights for policy 1, policy_version 296249 (0.0005) [2023-12-27 07:41:55,546][20381] Updated weights for policy 1, policy_version 296259 (0.0008) [2023-12-27 07:41:55,755][20347] Updated weights for policy 0, policy_version 296584 (0.0008) [2023-12-27 07:41:55,807][20347] Updated weights for policy 0, policy_version 296594 (0.0007) [2023-12-27 07:41:55,855][20347] Updated weights for policy 0, policy_version 296604 (0.0008) [2023-12-27 07:41:56,153][20381] Updated weights for policy 1, policy_version 296269 (0.0008) [2023-12-27 07:41:56,203][20381] Updated weights for policy 1, policy_version 296279 (0.0006) [2023-12-27 07:41:56,250][20381] Updated weights for policy 1, policy_version 296289 (0.0005) [2023-12-27 07:41:56,553][20347] Updated weights for policy 0, policy_version 296614 (0.0006) [2023-12-27 07:41:56,611][20347] Updated weights for policy 0, policy_version 296624 (0.0005) [2023-12-27 07:41:56,669][20347] Updated weights for policy 0, policy_version 296634 (0.0005) [2023-12-27 07:41:56,883][20381] Updated weights for policy 1, policy_version 296299 (0.0005) [2023-12-27 07:41:56,943][20381] Updated weights for policy 1, policy_version 296309 (0.0005) [2023-12-27 07:41:57,001][20381] Updated weights for policy 1, policy_version 296319 (0.0005) [2023-12-27 07:41:57,208][20347] Updated weights for policy 0, policy_version 296644 (0.0007) [2023-12-27 07:41:57,209][20291] KL-divergence is very high: 232.9609 [2023-12-27 07:41:57,216][20291] KL-divergence is very high: 351.4487 [2023-12-27 07:41:57,222][20291] KL-divergence is very high: 477.9954 [2023-12-27 07:41:57,227][20291] KL-divergence is very high: 447.3650 [2023-12-27 07:41:57,234][20291] KL-divergence is very high: 335.8040 [2023-12-27 07:41:57,241][20291] KL-divergence is very high: 106.1058 [2023-12-27 07:41:57,248][20291] KL-divergence is very high: 202.0296 [2023-12-27 07:41:57,254][20291] KL-divergence is very high: 191.8173 [2023-12-27 07:41:57,260][20291] KL-divergence is very high: 116.9991 [2023-12-27 07:41:57,271][20347] Updated weights for policy 0, policy_version 296654 (0.0010) [2023-12-27 07:41:57,272][20291] KL-divergence is very high: 128.8815 [2023-12-27 07:41:57,279][20291] KL-divergence is very high: 201.5105 [2023-12-27 07:41:57,285][20291] KL-divergence is very high: 217.4714 [2023-12-27 07:41:57,291][20291] KL-divergence is very high: 189.4553 [2023-12-27 07:41:57,300][20291] KL-divergence is very high: 101.1308 [2023-12-27 07:41:57,308][20291] KL-divergence is very high: 155.6813 [2023-12-27 07:41:57,315][20291] KL-divergence is very high: 305.9638 [2023-12-27 07:41:57,320][20291] KL-divergence is very high: 235.1572 [2023-12-27 07:41:57,326][20291] KL-divergence is very high: 208.6299 [2023-12-27 07:41:57,332][20291] KL-divergence is very high: 107.7464 [2023-12-27 07:41:57,336][20347] Updated weights for policy 0, policy_version 296664 (0.0010) [2023-12-27 07:41:57,337][20291] KL-divergence is very high: 130.6428 [2023-12-27 07:41:57,344][20291] KL-divergence is very high: 126.5192 [2023-12-27 07:41:57,350][20291] KL-divergence is very high: 197.6453 [2023-12-27 07:41:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 151830528. Throughput: 0: 10363.9, 1: 10342.4. Samples: 151821368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:41:57,363][19170] Avg episode reward: [(0, '756.530'), (1, '264.780')] [2023-12-27 07:41:57,370][20291] KL-divergence is very high: 140.7270 [2023-12-27 07:41:57,374][20291] KL-divergence is very high: 112.6946 [2023-12-27 07:41:57,579][20381] Updated weights for policy 1, policy_version 296329 (0.0010) [2023-12-27 07:41:57,629][20381] Updated weights for policy 1, policy_version 296339 (0.0005) [2023-12-27 07:41:57,680][20381] Updated weights for policy 1, policy_version 296349 (0.0007) [2023-12-27 07:41:57,735][20381] Updated weights for policy 1, policy_version 296359 (0.0010) [2023-12-27 07:41:57,941][20291] KL-divergence is very high: 525.4620 [2023-12-27 07:41:57,947][20347] Updated weights for policy 0, policy_version 296674 (0.0009) [2023-12-27 07:41:57,948][20291] KL-divergence is very high: 493.8061 [2023-12-27 07:41:57,953][20291] KL-divergence is very high: 234.1459 [2023-12-27 07:41:57,959][20291] KL-divergence is very high: 173.4930 [2023-12-27 07:41:57,965][20291] KL-divergence is very high: 106.7109 [2023-12-27 07:41:57,977][20291] KL-divergence is very high: 151.9257 [2023-12-27 07:41:57,988][20291] KL-divergence is very high: 227.8977 [2023-12-27 07:41:57,995][20291] KL-divergence is very high: 361.0034 [2023-12-27 07:41:58,001][20291] KL-divergence is very high: 137.7011 [2023-12-27 07:41:58,005][20347] Updated weights for policy 0, policy_version 296684 (0.0009) [2023-12-27 07:41:58,006][20291] KL-divergence is very high: 180.3417 [2023-12-27 07:41:58,011][20291] KL-divergence is very high: 106.9516 [2023-12-27 07:41:58,026][20291] KL-divergence is very high: 177.8453 [2023-12-27 07:41:58,030][20291] KL-divergence is very high: 277.8736 [2023-12-27 07:41:58,035][20291] KL-divergence is very high: 296.4265 [2023-12-27 07:41:58,040][20291] KL-divergence is very high: 170.0952 [2023-12-27 07:41:58,044][20291] KL-divergence is very high: 255.8520 [2023-12-27 07:41:58,056][20347] Updated weights for policy 0, policy_version 296694 (0.0006) [2023-12-27 07:41:58,071][20291] KL-divergence is very high: 291.5038 [2023-12-27 07:41:58,077][20291] KL-divergence is very high: 294.8711 [2023-12-27 07:41:58,082][20291] KL-divergence is very high: 143.5084 [2023-12-27 07:41:58,089][20291] KL-divergence is very high: 230.3693 [2023-12-27 07:41:58,112][20347] Updated weights for policy 0, policy_version 296704 (0.0007) [2023-12-27 07:41:58,359][20381] Updated weights for policy 1, policy_version 296369 (0.0009) [2023-12-27 07:41:58,420][20381] Updated weights for policy 1, policy_version 296379 (0.0009) [2023-12-27 07:41:58,486][20381] Updated weights for policy 1, policy_version 296389 (0.0008) [2023-12-27 07:41:58,810][20347] Updated weights for policy 0, policy_version 296714 (0.0007) [2023-12-27 07:41:58,865][20347] Updated weights for policy 0, policy_version 296724 (0.0008) [2023-12-27 07:41:58,917][20347] Updated weights for policy 0, policy_version 296734 (0.0007) [2023-12-27 07:41:59,185][20381] Updated weights for policy 1, policy_version 296399 (0.0006) [2023-12-27 07:41:59,242][20381] Updated weights for policy 1, policy_version 296409 (0.0007) [2023-12-27 07:41:59,296][20381] Updated weights for policy 1, policy_version 296419 (0.0010) [2023-12-27 07:41:59,567][20347] Updated weights for policy 0, policy_version 296744 (0.0007) [2023-12-27 07:41:59,624][20347] Updated weights for policy 0, policy_version 296754 (0.0008) [2023-12-27 07:41:59,679][20347] Updated weights for policy 0, policy_version 296764 (0.0007) [2023-12-27 07:41:59,959][20381] Updated weights for policy 1, policy_version 296429 (0.0011) [2023-12-27 07:42:00,014][20381] Updated weights for policy 1, policy_version 296439 (0.0010) [2023-12-27 07:42:00,071][20381] Updated weights for policy 1, policy_version 296449 (0.0010) [2023-12-27 07:42:00,416][20347] Updated weights for policy 0, policy_version 296774 (0.0007) [2023-12-27 07:42:00,470][20347] Updated weights for policy 0, policy_version 296784 (0.0006) [2023-12-27 07:42:00,526][20347] Updated weights for policy 0, policy_version 296794 (0.0005) [2023-12-27 07:42:00,744][20381] Updated weights for policy 1, policy_version 296459 (0.0010) [2023-12-27 07:42:00,801][20381] Updated weights for policy 1, policy_version 296469 (0.0010) [2023-12-27 07:42:00,853][20381] Updated weights for policy 1, policy_version 296479 (0.0010) [2023-12-27 07:42:01,201][20347] Updated weights for policy 0, policy_version 296804 (0.0007) [2023-12-27 07:42:01,256][20347] Updated weights for policy 0, policy_version 296814 (0.0010) [2023-12-27 07:42:01,313][20347] Updated weights for policy 0, policy_version 296824 (0.0006) [2023-12-27 07:42:01,592][20381] Updated weights for policy 1, policy_version 296489 (0.0010) [2023-12-27 07:42:01,647][20381] Updated weights for policy 1, policy_version 296499 (0.0009) [2023-12-27 07:42:01,703][20381] Updated weights for policy 1, policy_version 296509 (0.0008) [2023-12-27 07:42:01,761][20381] Updated weights for policy 1, policy_version 296519 (0.0011) [2023-12-27 07:42:01,949][20347] Updated weights for policy 0, policy_version 296834 (0.0008) [2023-12-27 07:42:01,997][20347] Updated weights for policy 0, policy_version 296844 (0.0010) [2023-12-27 07:42:02,046][20347] Updated weights for policy 0, policy_version 296854 (0.0010) [2023-12-27 07:42:02,091][20347] Updated weights for policy 0, policy_version 296864 (0.0010) [2023-12-27 07:42:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 151937024. Throughput: 0: 10376.0, 1: 10332.1. Samples: 151947964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:02,363][19170] Avg episode reward: [(0, '463.290'), (1, '223.320')] [2023-12-27 07:42:02,390][20381] Updated weights for policy 1, policy_version 296529 (0.0007) [2023-12-27 07:42:02,444][20381] Updated weights for policy 1, policy_version 296539 (0.0008) [2023-12-27 07:42:02,496][20381] Updated weights for policy 1, policy_version 296549 (0.0010) [2023-12-27 07:42:02,809][20347] Updated weights for policy 0, policy_version 296874 (0.0006) [2023-12-27 07:42:02,862][20347] Updated weights for policy 0, policy_version 296884 (0.0007) [2023-12-27 07:42:02,925][20347] Updated weights for policy 0, policy_version 296894 (0.0006) [2023-12-27 07:42:03,100][20381] Updated weights for policy 1, policy_version 296559 (0.0007) [2023-12-27 07:42:03,163][20381] Updated weights for policy 1, policy_version 296569 (0.0007) [2023-12-27 07:42:03,229][20381] Updated weights for policy 1, policy_version 296579 (0.0005) [2023-12-27 07:42:03,509][20347] Updated weights for policy 0, policy_version 296904 (0.0008) [2023-12-27 07:42:03,563][20347] Updated weights for policy 0, policy_version 296914 (0.0010) [2023-12-27 07:42:03,625][20347] Updated weights for policy 0, policy_version 296924 (0.0010) [2023-12-27 07:42:03,790][20381] Updated weights for policy 1, policy_version 296589 (0.0008) [2023-12-27 07:42:03,850][20381] Updated weights for policy 1, policy_version 296599 (0.0010) [2023-12-27 07:42:03,908][20381] Updated weights for policy 1, policy_version 296609 (0.0008) [2023-12-27 07:42:04,316][20347] Updated weights for policy 0, policy_version 296934 (0.0007) [2023-12-27 07:42:04,369][20347] Updated weights for policy 0, policy_version 296944 (0.0005) [2023-12-27 07:42:04,428][20347] Updated weights for policy 0, policy_version 296954 (0.0009) [2023-12-27 07:42:04,623][20381] Updated weights for policy 1, policy_version 296619 (0.0009) [2023-12-27 07:42:04,675][20381] Updated weights for policy 1, policy_version 296629 (0.0009) [2023-12-27 07:42:04,722][20381] Updated weights for policy 1, policy_version 296639 (0.0008) [2023-12-27 07:42:04,996][20347] Updated weights for policy 0, policy_version 296964 (0.0010) [2023-12-27 07:42:05,046][20347] Updated weights for policy 0, policy_version 296974 (0.0010) [2023-12-27 07:42:05,091][20347] Updated weights for policy 0, policy_version 296984 (0.0010) [2023-12-27 07:42:05,461][20381] Updated weights for policy 1, policy_version 296649 (0.0009) [2023-12-27 07:42:05,506][20381] Updated weights for policy 1, policy_version 296659 (0.0007) [2023-12-27 07:42:05,550][20381] Updated weights for policy 1, policy_version 296669 (0.0007) [2023-12-27 07:42:05,597][20381] Updated weights for policy 1, policy_version 296679 (0.0007) [2023-12-27 07:42:05,791][20347] Updated weights for policy 0, policy_version 296994 (0.0010) [2023-12-27 07:42:05,846][20347] Updated weights for policy 0, policy_version 297004 (0.0010) [2023-12-27 07:42:05,910][20347] Updated weights for policy 0, policy_version 297014 (0.0010) [2023-12-27 07:42:05,967][20347] Updated weights for policy 0, policy_version 297024 (0.0009) [2023-12-27 07:42:06,294][20381] Updated weights for policy 1, policy_version 296689 (0.0007) [2023-12-27 07:42:06,356][20381] Updated weights for policy 1, policy_version 296699 (0.0009) [2023-12-27 07:42:06,416][20381] Updated weights for policy 1, policy_version 296709 (0.0009) [2023-12-27 07:42:06,638][20347] Updated weights for policy 0, policy_version 297034 (0.0006) [2023-12-27 07:42:06,699][20347] Updated weights for policy 0, policy_version 297044 (0.0006) [2023-12-27 07:42:06,763][20347] Updated weights for policy 0, policy_version 297054 (0.0007) [2023-12-27 07:42:07,144][20381] Updated weights for policy 1, policy_version 296719 (0.0010) [2023-12-27 07:42:07,208][20381] Updated weights for policy 1, policy_version 296729 (0.0011) [2023-12-27 07:42:07,263][20381] Updated weights for policy 1, policy_version 296739 (0.0011) [2023-12-27 07:42:07,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 152043520. Throughput: 0: 10431.0, 1: 10353.5. Samples: 152012428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:07,364][19170] Avg episode reward: [(0, '524.590'), (1, '243.320')] [2023-12-27 07:42:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000296744_75980800.pth... [2023-12-27 07:42:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000297056_76062720.pth... [2023-12-27 07:42:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000295872_75759616.pth [2023-12-27 07:42:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000295528_75669504.pth [2023-12-27 07:42:07,457][20347] Updated weights for policy 0, policy_version 297064 (0.0006) [2023-12-27 07:42:07,505][20347] Updated weights for policy 0, policy_version 297074 (0.0005) [2023-12-27 07:42:07,559][20347] Updated weights for policy 0, policy_version 297084 (0.0005) [2023-12-27 07:42:07,877][20381] Updated weights for policy 1, policy_version 296749 (0.0008) [2023-12-27 07:42:07,928][20381] Updated weights for policy 1, policy_version 296759 (0.0005) [2023-12-27 07:42:07,988][20381] Updated weights for policy 1, policy_version 296769 (0.0010) [2023-12-27 07:42:08,261][20347] Updated weights for policy 0, policy_version 297094 (0.0007) [2023-12-27 07:42:08,320][20347] Updated weights for policy 0, policy_version 297104 (0.0007) [2023-12-27 07:42:08,378][20347] Updated weights for policy 0, policy_version 297114 (0.0008) [2023-12-27 07:42:08,655][20381] Updated weights for policy 1, policy_version 296779 (0.0011) [2023-12-27 07:42:08,718][20381] Updated weights for policy 1, policy_version 296789 (0.0011) [2023-12-27 07:42:08,779][20381] Updated weights for policy 1, policy_version 296799 (0.0011) [2023-12-27 07:42:09,092][20347] Updated weights for policy 0, policy_version 297124 (0.0007) [2023-12-27 07:42:09,150][20347] Updated weights for policy 0, policy_version 297134 (0.0007) [2023-12-27 07:42:09,205][20347] Updated weights for policy 0, policy_version 297144 (0.0008) [2023-12-27 07:42:09,442][20381] Updated weights for policy 1, policy_version 296809 (0.0011) [2023-12-27 07:42:09,511][20381] Updated weights for policy 1, policy_version 296819 (0.0011) [2023-12-27 07:42:09,573][20381] Updated weights for policy 1, policy_version 296829 (0.0011) [2023-12-27 07:42:09,635][20381] Updated weights for policy 1, policy_version 296839 (0.0010) [2023-12-27 07:42:09,922][20347] Updated weights for policy 0, policy_version 297154 (0.0008) [2023-12-27 07:42:09,980][20347] Updated weights for policy 0, policy_version 297164 (0.0008) [2023-12-27 07:42:10,045][20347] Updated weights for policy 0, policy_version 297174 (0.0006) [2023-12-27 07:42:10,106][20347] Updated weights for policy 0, policy_version 297184 (0.0007) [2023-12-27 07:42:10,317][20381] Updated weights for policy 1, policy_version 296849 (0.0008) [2023-12-27 07:42:10,373][20381] Updated weights for policy 1, policy_version 296859 (0.0008) [2023-12-27 07:42:10,430][20381] Updated weights for policy 1, policy_version 296869 (0.0008) [2023-12-27 07:42:10,794][20347] Updated weights for policy 0, policy_version 297194 (0.0005) [2023-12-27 07:42:10,848][20347] Updated weights for policy 0, policy_version 297204 (0.0005) [2023-12-27 07:42:10,909][20347] Updated weights for policy 0, policy_version 297214 (0.0005) [2023-12-27 07:42:11,101][20381] Updated weights for policy 1, policy_version 296879 (0.0008) [2023-12-27 07:42:11,166][20381] Updated weights for policy 1, policy_version 296889 (0.0008) [2023-12-27 07:42:11,227][20381] Updated weights for policy 1, policy_version 296899 (0.0006) [2023-12-27 07:42:11,577][20347] Updated weights for policy 0, policy_version 297224 (0.0007) [2023-12-27 07:42:11,641][20347] Updated weights for policy 0, policy_version 297234 (0.0008) [2023-12-27 07:42:11,699][20347] Updated weights for policy 0, policy_version 297244 (0.0006) [2023-12-27 07:42:11,879][20381] Updated weights for policy 1, policy_version 296909 (0.0007) [2023-12-27 07:42:11,943][20381] Updated weights for policy 1, policy_version 296919 (0.0006) [2023-12-27 07:42:11,996][20381] Updated weights for policy 1, policy_version 296929 (0.0007) [2023-12-27 07:42:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 152141824. Throughput: 0: 10374.4, 1: 10330.3. Samples: 152133792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:12,363][19170] Avg episode reward: [(0, '321.590'), (1, '208.750')] [2023-12-27 07:42:12,407][20347] Updated weights for policy 0, policy_version 297254 (0.0010) [2023-12-27 07:42:12,456][20347] Updated weights for policy 0, policy_version 297264 (0.0010) [2023-12-27 07:42:12,505][20347] Updated weights for policy 0, policy_version 297274 (0.0010) [2023-12-27 07:42:12,680][20381] Updated weights for policy 1, policy_version 296939 (0.0008) [2023-12-27 07:42:12,740][20381] Updated weights for policy 1, policy_version 296949 (0.0009) [2023-12-27 07:42:12,804][20381] Updated weights for policy 1, policy_version 296959 (0.0010) [2023-12-27 07:42:13,204][20347] Updated weights for policy 0, policy_version 297284 (0.0010) [2023-12-27 07:42:13,265][20347] Updated weights for policy 0, policy_version 297294 (0.0010) [2023-12-27 07:42:13,322][20347] Updated weights for policy 0, policy_version 297304 (0.0010) [2023-12-27 07:42:13,446][20381] Updated weights for policy 1, policy_version 296969 (0.0008) [2023-12-27 07:42:13,503][20381] Updated weights for policy 1, policy_version 296979 (0.0008) [2023-12-27 07:42:13,562][20381] Updated weights for policy 1, policy_version 296989 (0.0007) [2023-12-27 07:42:13,613][20381] Updated weights for policy 1, policy_version 296999 (0.0010) [2023-12-27 07:42:13,981][20347] Updated weights for policy 0, policy_version 297314 (0.0010) [2023-12-27 07:42:14,034][20347] Updated weights for policy 0, policy_version 297324 (0.0008) [2023-12-27 07:42:14,095][20347] Updated weights for policy 0, policy_version 297334 (0.0009) [2023-12-27 07:42:14,151][20347] Updated weights for policy 0, policy_version 297344 (0.0010) [2023-12-27 07:42:14,172][20381] Updated weights for policy 1, policy_version 297009 (0.0011) [2023-12-27 07:42:14,221][20381] Updated weights for policy 1, policy_version 297019 (0.0010) [2023-12-27 07:42:14,273][20381] Updated weights for policy 1, policy_version 297029 (0.0010) [2023-12-27 07:42:14,829][20347] Updated weights for policy 0, policy_version 297354 (0.0011) [2023-12-27 07:42:14,876][20347] Updated weights for policy 0, policy_version 297364 (0.0010) [2023-12-27 07:42:14,926][20347] Updated weights for policy 0, policy_version 297374 (0.0010) [2023-12-27 07:42:14,977][20381] Updated weights for policy 1, policy_version 297039 (0.0011) [2023-12-27 07:42:15,030][20381] Updated weights for policy 1, policy_version 297049 (0.0011) [2023-12-27 07:42:15,094][20381] Updated weights for policy 1, policy_version 297059 (0.0011) [2023-12-27 07:42:15,590][20347] Updated weights for policy 0, policy_version 297384 (0.0006) [2023-12-27 07:42:15,642][20347] Updated weights for policy 0, policy_version 297394 (0.0005) [2023-12-27 07:42:15,699][20347] Updated weights for policy 0, policy_version 297404 (0.0005) [2023-12-27 07:42:15,772][20381] Updated weights for policy 1, policy_version 297069 (0.0011) [2023-12-27 07:42:15,832][20381] Updated weights for policy 1, policy_version 297079 (0.0008) [2023-12-27 07:42:15,893][20381] Updated weights for policy 1, policy_version 297089 (0.0011) [2023-12-27 07:42:16,262][20347] Updated weights for policy 0, policy_version 297414 (0.0006) [2023-12-27 07:42:16,322][20347] Updated weights for policy 0, policy_version 297424 (0.0007) [2023-12-27 07:42:16,376][20347] Updated weights for policy 0, policy_version 297434 (0.0007) [2023-12-27 07:42:16,529][20381] Updated weights for policy 1, policy_version 297099 (0.0010) [2023-12-27 07:42:16,583][20381] Updated weights for policy 1, policy_version 297109 (0.0010) [2023-12-27 07:42:16,645][20381] Updated weights for policy 1, policy_version 297119 (0.0011) [2023-12-27 07:42:16,954][20347] Updated weights for policy 0, policy_version 297444 (0.0007) [2023-12-27 07:42:17,004][20347] Updated weights for policy 0, policy_version 297454 (0.0007) [2023-12-27 07:42:17,056][20347] Updated weights for policy 0, policy_version 297464 (0.0007) [2023-12-27 07:42:17,294][20381] Updated weights for policy 1, policy_version 297129 (0.0011) [2023-12-27 07:42:17,347][20381] Updated weights for policy 1, policy_version 297139 (0.0008) [2023-12-27 07:42:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 152248320. Throughput: 0: 10322.1, 1: 10369.5. Samples: 152260040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:17,364][19170] Avg episode reward: [(0, '340.230'), (1, '148.880')] [2023-12-27 07:42:17,393][20381] Updated weights for policy 1, policy_version 297149 (0.0005) [2023-12-27 07:42:17,452][20381] Updated weights for policy 1, policy_version 297159 (0.0007) [2023-12-27 07:42:17,723][20347] Updated weights for policy 0, policy_version 297474 (0.0008) [2023-12-27 07:42:17,781][20347] Updated weights for policy 0, policy_version 297484 (0.0007) [2023-12-27 07:42:17,834][20347] Updated weights for policy 0, policy_version 297494 (0.0006) [2023-12-27 07:42:17,880][20347] Updated weights for policy 0, policy_version 297504 (0.0007) [2023-12-27 07:42:18,041][20381] Updated weights for policy 1, policy_version 297169 (0.0008) [2023-12-27 07:42:18,096][20381] Updated weights for policy 1, policy_version 297179 (0.0008) [2023-12-27 07:42:18,154][20381] Updated weights for policy 1, policy_version 297189 (0.0005) [2023-12-27 07:42:18,602][20347] Updated weights for policy 0, policy_version 297514 (0.0008) [2023-12-27 07:42:18,652][20347] Updated weights for policy 0, policy_version 297524 (0.0008) [2023-12-27 07:42:18,706][20347] Updated weights for policy 0, policy_version 297534 (0.0008) [2023-12-27 07:42:18,802][20381] Updated weights for policy 1, policy_version 297199 (0.0006) [2023-12-27 07:42:18,861][20381] Updated weights for policy 1, policy_version 297209 (0.0008) [2023-12-27 07:42:18,908][20381] Updated weights for policy 1, policy_version 297219 (0.0007) [2023-12-27 07:42:19,432][20347] Updated weights for policy 0, policy_version 297544 (0.0008) [2023-12-27 07:42:19,494][20347] Updated weights for policy 0, policy_version 297554 (0.0007) [2023-12-27 07:42:19,560][20347] Updated weights for policy 0, policy_version 297564 (0.0009) [2023-12-27 07:42:19,633][20381] Updated weights for policy 1, policy_version 297229 (0.0007) [2023-12-27 07:42:19,697][20381] Updated weights for policy 1, policy_version 297239 (0.0008) [2023-12-27 07:42:19,762][20381] Updated weights for policy 1, policy_version 297249 (0.0006) [2023-12-27 07:42:20,288][20347] Updated weights for policy 0, policy_version 297574 (0.0008) [2023-12-27 07:42:20,346][20347] Updated weights for policy 0, policy_version 297584 (0.0008) [2023-12-27 07:42:20,392][20381] Updated weights for policy 1, policy_version 297259 (0.0007) [2023-12-27 07:42:20,395][20347] Updated weights for policy 0, policy_version 297594 (0.0007) [2023-12-27 07:42:20,443][20381] Updated weights for policy 1, policy_version 297269 (0.0007) [2023-12-27 07:42:20,493][20381] Updated weights for policy 1, policy_version 297279 (0.0007) [2023-12-27 07:42:21,122][20381] Updated weights for policy 1, policy_version 297289 (0.0008) [2023-12-27 07:42:21,128][20347] Updated weights for policy 0, policy_version 297604 (0.0006) [2023-12-27 07:42:21,179][20347] Updated weights for policy 0, policy_version 297614 (0.0007) [2023-12-27 07:42:21,188][20381] Updated weights for policy 1, policy_version 297299 (0.0008) [2023-12-27 07:42:21,241][20347] Updated weights for policy 0, policy_version 297624 (0.0006) [2023-12-27 07:42:21,247][20381] Updated weights for policy 1, policy_version 297309 (0.0006) [2023-12-27 07:42:21,309][20381] Updated weights for policy 1, policy_version 297319 (0.0007) [2023-12-27 07:42:21,912][20347] Updated weights for policy 0, policy_version 297634 (0.0006) [2023-12-27 07:42:21,965][20347] Updated weights for policy 0, policy_version 297644 (0.0007) [2023-12-27 07:42:22,002][20381] Updated weights for policy 1, policy_version 297329 (0.0010) [2023-12-27 07:42:22,015][20347] Updated weights for policy 0, policy_version 297654 (0.0005) [2023-12-27 07:42:22,060][20381] Updated weights for policy 1, policy_version 297339 (0.0011) [2023-12-27 07:42:22,071][20347] Updated weights for policy 0, policy_version 297664 (0.0007) [2023-12-27 07:42:22,113][20381] Updated weights for policy 1, policy_version 297349 (0.0011) [2023-12-27 07:42:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 152354816. Throughput: 0: 10311.1, 1: 10375.1. Samples: 152321172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:22,363][19170] Avg episode reward: [(0, '376.700'), (1, '51.040')] [2023-12-27 07:42:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000297352_76136448.pth... [2023-12-27 07:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000297664_76218368.pth... [2023-12-27 07:42:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000296104_75816960.pth [2023-12-27 07:42:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000296448_75907072.pth [2023-12-27 07:42:22,767][20347] Updated weights for policy 0, policy_version 297674 (0.0008) [2023-12-27 07:42:22,820][20347] Updated weights for policy 0, policy_version 297684 (0.0007) [2023-12-27 07:42:22,821][20381] Updated weights for policy 1, policy_version 297359 (0.0011) [2023-12-27 07:42:22,871][20381] Updated weights for policy 1, policy_version 297369 (0.0011) [2023-12-27 07:42:22,872][20347] Updated weights for policy 0, policy_version 297694 (0.0006) [2023-12-27 07:42:22,927][20381] Updated weights for policy 1, policy_version 297379 (0.0011) [2023-12-27 07:42:23,508][20347] Updated weights for policy 0, policy_version 297704 (0.0007) [2023-12-27 07:42:23,559][20347] Updated weights for policy 0, policy_version 297714 (0.0007) [2023-12-27 07:42:23,612][20347] Updated weights for policy 0, policy_version 297724 (0.0007) [2023-12-27 07:42:23,636][20381] Updated weights for policy 1, policy_version 297389 (0.0010) [2023-12-27 07:42:23,691][20381] Updated weights for policy 1, policy_version 297399 (0.0010) [2023-12-27 07:42:23,740][20381] Updated weights for policy 1, policy_version 297409 (0.0011) [2023-12-27 07:42:24,315][20347] Updated weights for policy 0, policy_version 297734 (0.0007) [2023-12-27 07:42:24,353][20381] Updated weights for policy 1, policy_version 297419 (0.0009) [2023-12-27 07:42:24,374][20347] Updated weights for policy 0, policy_version 297744 (0.0006) [2023-12-27 07:42:24,408][20381] Updated weights for policy 1, policy_version 297429 (0.0008) [2023-12-27 07:42:24,431][20347] Updated weights for policy 0, policy_version 297754 (0.0006) [2023-12-27 07:42:24,462][20381] Updated weights for policy 1, policy_version 297439 (0.0007) [2023-12-27 07:42:25,007][20347] Updated weights for policy 0, policy_version 297764 (0.0007) [2023-12-27 07:42:25,055][20347] Updated weights for policy 0, policy_version 297774 (0.0005) [2023-12-27 07:42:25,104][20347] Updated weights for policy 0, policy_version 297784 (0.0005) [2023-12-27 07:42:25,184][20381] Updated weights for policy 1, policy_version 297449 (0.0008) [2023-12-27 07:42:25,236][20381] Updated weights for policy 1, policy_version 297459 (0.0007) [2023-12-27 07:42:25,280][20381] Updated weights for policy 1, policy_version 297469 (0.0008) [2023-12-27 07:42:25,334][20381] Updated weights for policy 1, policy_version 297479 (0.0006) [2023-12-27 07:42:25,658][20347] Updated weights for policy 0, policy_version 297794 (0.0005) [2023-12-27 07:42:25,716][20347] Updated weights for policy 0, policy_version 297804 (0.0010) [2023-12-27 07:42:25,771][20347] Updated weights for policy 0, policy_version 297814 (0.0010) [2023-12-27 07:42:25,828][20347] Updated weights for policy 0, policy_version 297824 (0.0010) [2023-12-27 07:42:26,038][20381] Updated weights for policy 1, policy_version 297489 (0.0007) [2023-12-27 07:42:26,090][20381] Updated weights for policy 1, policy_version 297499 (0.0009) [2023-12-27 07:42:26,146][20381] Updated weights for policy 1, policy_version 297509 (0.0007) [2023-12-27 07:42:26,476][20347] Updated weights for policy 0, policy_version 297834 (0.0010) [2023-12-27 07:42:26,520][20347] Updated weights for policy 0, policy_version 297844 (0.0010) [2023-12-27 07:42:26,570][20347] Updated weights for policy 0, policy_version 297854 (0.0010) [2023-12-27 07:42:26,766][20381] Updated weights for policy 1, policy_version 297519 (0.0005) [2023-12-27 07:42:26,812][20381] Updated weights for policy 1, policy_version 297529 (0.0009) [2023-12-27 07:42:26,864][20381] Updated weights for policy 1, policy_version 297539 (0.0010) [2023-12-27 07:42:27,250][20347] Updated weights for policy 0, policy_version 297864 (0.0010) [2023-12-27 07:42:27,304][20347] Updated weights for policy 0, policy_version 297874 (0.0010) [2023-12-27 07:42:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 152453120. Throughput: 0: 10371.9, 1: 10410.6. Samples: 152446496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:42:27,363][19170] Avg episode reward: [(0, '448.350'), (1, '141.140')] [2023-12-27 07:42:27,365][20347] Updated weights for policy 0, policy_version 297884 (0.0009) [2023-12-27 07:42:27,429][20381] Updated weights for policy 1, policy_version 297549 (0.0007) [2023-12-27 07:42:27,487][20381] Updated weights for policy 1, policy_version 297559 (0.0005) [2023-12-27 07:42:27,545][20381] Updated weights for policy 1, policy_version 297569 (0.0005) [2023-12-27 07:42:28,039][20347] Updated weights for policy 0, policy_version 297894 (0.0010) [2023-12-27 07:42:28,090][20347] Updated weights for policy 0, policy_version 297904 (0.0010) [2023-12-27 07:42:28,121][20381] Updated weights for policy 1, policy_version 297579 (0.0007) [2023-12-27 07:42:28,153][20347] Updated weights for policy 0, policy_version 297914 (0.0010) [2023-12-27 07:42:28,172][20381] Updated weights for policy 1, policy_version 297589 (0.0010) [2023-12-27 07:42:28,233][20381] Updated weights for policy 1, policy_version 297599 (0.0010) [2023-12-27 07:42:28,832][20347] Updated weights for policy 0, policy_version 297924 (0.0010) [2023-12-27 07:42:28,836][20381] Updated weights for policy 1, policy_version 297609 (0.0010) [2023-12-27 07:42:28,888][20381] Updated weights for policy 1, policy_version 297619 (0.0010) [2023-12-27 07:42:28,918][20347] Updated weights for policy 0, policy_version 297934 (0.0010) [2023-12-27 07:42:28,952][20381] Updated weights for policy 1, policy_version 297629 (0.0010) [2023-12-27 07:42:28,978][20347] Updated weights for policy 0, policy_version 297944 (0.0010) [2023-12-27 07:42:29,004][20381] Updated weights for policy 1, policy_version 297639 (0.0009) [2023-12-27 07:42:29,610][20347] Updated weights for policy 0, policy_version 297954 (0.0009) [2023-12-27 07:42:29,659][20347] Updated weights for policy 0, policy_version 297964 (0.0008) [2023-12-27 07:42:29,660][20381] Updated weights for policy 1, policy_version 297649 (0.0011) [2023-12-27 07:42:29,710][20347] Updated weights for policy 0, policy_version 297974 (0.0008) [2023-12-27 07:42:29,719][20381] Updated weights for policy 1, policy_version 297659 (0.0011) [2023-12-27 07:42:29,764][20347] Updated weights for policy 0, policy_version 297984 (0.0007) [2023-12-27 07:42:29,772][20381] Updated weights for policy 1, policy_version 297669 (0.0011) [2023-12-27 07:42:30,439][20347] Updated weights for policy 0, policy_version 297994 (0.0007) [2023-12-27 07:42:30,470][20381] Updated weights for policy 1, policy_version 297679 (0.0011) [2023-12-27 07:42:30,491][20347] Updated weights for policy 0, policy_version 298004 (0.0007) [2023-12-27 07:42:30,522][20381] Updated weights for policy 1, policy_version 297689 (0.0010) [2023-12-27 07:42:30,539][20347] Updated weights for policy 0, policy_version 298014 (0.0005) [2023-12-27 07:42:30,569][20381] Updated weights for policy 1, policy_version 297699 (0.0009) [2023-12-27 07:42:31,146][20347] Updated weights for policy 0, policy_version 298024 (0.0008) [2023-12-27 07:42:31,191][20381] Updated weights for policy 1, policy_version 297709 (0.0008) [2023-12-27 07:42:31,201][20347] Updated weights for policy 0, policy_version 298034 (0.0005) [2023-12-27 07:42:31,252][20381] Updated weights for policy 1, policy_version 297719 (0.0010) [2023-12-27 07:42:31,262][20347] Updated weights for policy 0, policy_version 298044 (0.0006) [2023-12-27 07:42:31,311][20381] Updated weights for policy 1, policy_version 297729 (0.0011) [2023-12-27 07:42:31,951][20347] Updated weights for policy 0, policy_version 298054 (0.0006) [2023-12-27 07:42:32,006][20381] Updated weights for policy 1, policy_version 297739 (0.0009) [2023-12-27 07:42:32,007][20347] Updated weights for policy 0, policy_version 298064 (0.0008) [2023-12-27 07:42:32,062][20347] Updated weights for policy 0, policy_version 298074 (0.0006) [2023-12-27 07:42:32,071][20381] Updated weights for policy 1, policy_version 297749 (0.0010) [2023-12-27 07:42:32,129][20381] Updated weights for policy 1, policy_version 297759 (0.0011) [2023-12-27 07:42:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 152567808. Throughput: 0: 10383.1, 1: 10493.0. Samples: 152574612. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:32,363][19170] Avg episode reward: [(0, '378.080'), (1, '160.220')] [2023-12-27 07:42:32,683][20347] Updated weights for policy 0, policy_version 298084 (0.0005) [2023-12-27 07:42:32,735][20347] Updated weights for policy 0, policy_version 298094 (0.0007) [2023-12-27 07:42:32,758][20381] Updated weights for policy 1, policy_version 297769 (0.0010) [2023-12-27 07:42:32,788][20347] Updated weights for policy 0, policy_version 298104 (0.0006) [2023-12-27 07:42:32,818][20381] Updated weights for policy 1, policy_version 297779 (0.0006) [2023-12-27 07:42:32,869][20381] Updated weights for policy 1, policy_version 297789 (0.0006) [2023-12-27 07:42:32,914][20381] Updated weights for policy 1, policy_version 297799 (0.0008) [2023-12-27 07:42:33,319][20347] Updated weights for policy 0, policy_version 298114 (0.0005) [2023-12-27 07:42:33,370][20347] Updated weights for policy 0, policy_version 298124 (0.0005) [2023-12-27 07:42:33,415][20347] Updated weights for policy 0, policy_version 298134 (0.0005) [2023-12-27 07:42:33,466][20347] Updated weights for policy 0, policy_version 298144 (0.0005) [2023-12-27 07:42:33,495][20381] Updated weights for policy 1, policy_version 297809 (0.0005) [2023-12-27 07:42:33,540][20381] Updated weights for policy 1, policy_version 297819 (0.0005) [2023-12-27 07:42:33,583][20381] Updated weights for policy 1, policy_version 297829 (0.0005) [2023-12-27 07:42:34,044][20347] Updated weights for policy 0, policy_version 298154 (0.0005) [2023-12-27 07:42:34,100][20347] Updated weights for policy 0, policy_version 298164 (0.0005) [2023-12-27 07:42:34,151][20381] Updated weights for policy 1, policy_version 297839 (0.0006) [2023-12-27 07:42:34,162][20347] Updated weights for policy 0, policy_version 298174 (0.0006) [2023-12-27 07:42:34,211][20381] Updated weights for policy 1, policy_version 297849 (0.0008) [2023-12-27 07:42:34,258][20381] Updated weights for policy 1, policy_version 297859 (0.0008) [2023-12-27 07:42:34,757][20347] Updated weights for policy 0, policy_version 298184 (0.0007) [2023-12-27 07:42:34,819][20347] Updated weights for policy 0, policy_version 298194 (0.0008) [2023-12-27 07:42:34,878][20347] Updated weights for policy 0, policy_version 298204 (0.0007) [2023-12-27 07:42:34,905][20381] Updated weights for policy 1, policy_version 297870 (0.0007) [2023-12-27 07:42:34,962][20381] Updated weights for policy 1, policy_version 297880 (0.0005) [2023-12-27 07:42:35,018][20381] Updated weights for policy 1, policy_version 297890 (0.0006) [2023-12-27 07:42:35,567][20381] Updated weights for policy 1, policy_version 297900 (0.0006) [2023-12-27 07:42:35,601][20347] Updated weights for policy 0, policy_version 298214 (0.0007) [2023-12-27 07:42:35,615][20381] Updated weights for policy 1, policy_version 297910 (0.0005) [2023-12-27 07:42:35,657][20347] Updated weights for policy 0, policy_version 298224 (0.0007) [2023-12-27 07:42:35,663][20381] Updated weights for policy 1, policy_version 297920 (0.0005) [2023-12-27 07:42:35,714][20347] Updated weights for policy 0, policy_version 298234 (0.0008) [2023-12-27 07:42:36,334][20381] Updated weights for policy 1, policy_version 297930 (0.0006) [2023-12-27 07:42:36,341][20347] Updated weights for policy 0, policy_version 298244 (0.0006) [2023-12-27 07:42:36,391][20381] Updated weights for policy 1, policy_version 297940 (0.0006) [2023-12-27 07:42:36,405][20347] Updated weights for policy 0, policy_version 298254 (0.0006) [2023-12-27 07:42:36,449][20381] Updated weights for policy 1, policy_version 297950 (0.0006) [2023-12-27 07:42:36,466][20347] Updated weights for policy 0, policy_version 298264 (0.0007) [2023-12-27 07:42:36,502][20381] Updated weights for policy 1, policy_version 297960 (0.0006) [2023-12-27 07:42:37,169][20347] Updated weights for policy 0, policy_version 298274 (0.0006) [2023-12-27 07:42:37,218][20347] Updated weights for policy 0, policy_version 298284 (0.0006) [2023-12-27 07:42:37,237][20381] Updated weights for policy 1, policy_version 297970 (0.0007) [2023-12-27 07:42:37,268][20347] Updated weights for policy 0, policy_version 298294 (0.0008) [2023-12-27 07:42:37,296][20381] Updated weights for policy 1, policy_version 297980 (0.0008) [2023-12-27 07:42:37,326][20347] Updated weights for policy 0, policy_version 298304 (0.0006) [2023-12-27 07:42:37,360][20381] Updated weights for policy 1, policy_version 297990 (0.0007) [2023-12-27 07:42:37,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 152674304. Throughput: 0: 10505.5, 1: 10603.4. Samples: 152644216. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:37,363][19170] Avg episode reward: [(0, '324.570'), (1, '109.680')] [2023-12-27 07:42:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000298304_76382208.pth... [2023-12-27 07:42:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000297056_76062720.pth [2023-12-27 07:42:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000297992_76300288.pth... [2023-12-27 07:42:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000296744_75980800.pth [2023-12-27 07:42:37,975][20347] Updated weights for policy 0, policy_version 298314 (0.0008) [2023-12-27 07:42:38,033][20347] Updated weights for policy 0, policy_version 298324 (0.0006) [2023-12-27 07:42:38,049][20381] Updated weights for policy 1, policy_version 298000 (0.0007) [2023-12-27 07:42:38,093][20347] Updated weights for policy 0, policy_version 298334 (0.0005) [2023-12-27 07:42:38,099][20381] Updated weights for policy 1, policy_version 298010 (0.0007) [2023-12-27 07:42:38,146][20381] Updated weights for policy 1, policy_version 298020 (0.0008) [2023-12-27 07:42:38,698][20347] Updated weights for policy 0, policy_version 298344 (0.0007) [2023-12-27 07:42:38,759][20347] Updated weights for policy 0, policy_version 298354 (0.0008) [2023-12-27 07:42:38,824][20347] Updated weights for policy 0, policy_version 298364 (0.0007) [2023-12-27 07:42:38,890][20381] Updated weights for policy 1, policy_version 298030 (0.0009) [2023-12-27 07:42:38,937][20381] Updated weights for policy 1, policy_version 298040 (0.0008) [2023-12-27 07:42:38,982][20381] Updated weights for policy 1, policy_version 298050 (0.0007) [2023-12-27 07:42:39,505][20347] Updated weights for policy 0, policy_version 298374 (0.0008) [2023-12-27 07:42:39,564][20347] Updated weights for policy 0, policy_version 298384 (0.0008) [2023-12-27 07:42:39,628][20347] Updated weights for policy 0, policy_version 298394 (0.0008) [2023-12-27 07:42:39,634][20381] Updated weights for policy 1, policy_version 298060 (0.0005) [2023-12-27 07:42:39,696][20381] Updated weights for policy 1, policy_version 298070 (0.0007) [2023-12-27 07:42:39,753][20381] Updated weights for policy 1, policy_version 298080 (0.0006) [2023-12-27 07:42:40,426][20347] Updated weights for policy 0, policy_version 298404 (0.0008) [2023-12-27 07:42:40,453][20381] Updated weights for policy 1, policy_version 298090 (0.0007) [2023-12-27 07:42:40,490][20347] Updated weights for policy 0, policy_version 298414 (0.0007) [2023-12-27 07:42:40,507][20381] Updated weights for policy 1, policy_version 298100 (0.0007) [2023-12-27 07:42:40,553][20347] Updated weights for policy 0, policy_version 298424 (0.0007) [2023-12-27 07:42:40,561][20381] Updated weights for policy 1, policy_version 298110 (0.0007) [2023-12-27 07:42:40,616][20381] Updated weights for policy 1, policy_version 298120 (0.0006) [2023-12-27 07:42:41,145][20347] Updated weights for policy 0, policy_version 298434 (0.0008) [2023-12-27 07:42:41,208][20347] Updated weights for policy 0, policy_version 298444 (0.0007) [2023-12-27 07:42:41,272][20347] Updated weights for policy 0, policy_version 298454 (0.0010) [2023-12-27 07:42:41,326][20347] Updated weights for policy 0, policy_version 298464 (0.0011) [2023-12-27 07:42:41,373][20381] Updated weights for policy 1, policy_version 298130 (0.0007) [2023-12-27 07:42:41,429][20381] Updated weights for policy 1, policy_version 298140 (0.0007) [2023-12-27 07:42:41,489][20381] Updated weights for policy 1, policy_version 298150 (0.0008) [2023-12-27 07:42:42,006][20347] Updated weights for policy 0, policy_version 298474 (0.0010) [2023-12-27 07:42:42,067][20347] Updated weights for policy 0, policy_version 298484 (0.0010) [2023-12-27 07:42:42,132][20347] Updated weights for policy 0, policy_version 298494 (0.0011) [2023-12-27 07:42:42,184][20381] Updated weights for policy 1, policy_version 298160 (0.0006) [2023-12-27 07:42:42,239][20381] Updated weights for policy 1, policy_version 298170 (0.0005) [2023-12-27 07:42:42,300][20381] Updated weights for policy 1, policy_version 298180 (0.0006) [2023-12-27 07:42:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 152780800. Throughput: 0: 10468.3, 1: 10529.9. Samples: 152766292. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:42,363][19170] Avg episode reward: [(0, '344.520'), (1, '125.620')] [2023-12-27 07:42:42,833][20347] Updated weights for policy 0, policy_version 298504 (0.0011) [2023-12-27 07:42:42,888][20347] Updated weights for policy 0, policy_version 298514 (0.0010) [2023-12-27 07:42:42,940][20347] Updated weights for policy 0, policy_version 298524 (0.0010) [2023-12-27 07:42:42,957][20381] Updated weights for policy 1, policy_version 298190 (0.0006) [2023-12-27 07:42:43,011][20381] Updated weights for policy 1, policy_version 298200 (0.0005) [2023-12-27 07:42:43,063][20381] Updated weights for policy 1, policy_version 298210 (0.0009) [2023-12-27 07:42:43,622][20347] Updated weights for policy 0, policy_version 298534 (0.0010) [2023-12-27 07:42:43,680][20381] Updated weights for policy 1, policy_version 298220 (0.0006) [2023-12-27 07:42:43,683][20347] Updated weights for policy 0, policy_version 298544 (0.0010) [2023-12-27 07:42:43,738][20347] Updated weights for policy 0, policy_version 298554 (0.0010) [2023-12-27 07:42:43,740][20381] Updated weights for policy 1, policy_version 298230 (0.0005) [2023-12-27 07:42:43,796][20381] Updated weights for policy 1, policy_version 298240 (0.0006) [2023-12-27 07:42:44,404][20347] Updated weights for policy 0, policy_version 298564 (0.0010) [2023-12-27 07:42:44,460][20381] Updated weights for policy 1, policy_version 298250 (0.0008) [2023-12-27 07:42:44,462][20347] Updated weights for policy 0, policy_version 298574 (0.0010) [2023-12-27 07:42:44,513][20381] Updated weights for policy 1, policy_version 298260 (0.0006) [2023-12-27 07:42:44,519][20347] Updated weights for policy 0, policy_version 298584 (0.0010) [2023-12-27 07:42:44,562][20381] Updated weights for policy 1, policy_version 298270 (0.0007) [2023-12-27 07:42:44,619][20381] Updated weights for policy 1, policy_version 298280 (0.0007) [2023-12-27 07:42:45,199][20347] Updated weights for policy 0, policy_version 298594 (0.0010) [2023-12-27 07:42:45,261][20347] Updated weights for policy 0, policy_version 298604 (0.0011) [2023-12-27 07:42:45,322][20347] Updated weights for policy 0, policy_version 298614 (0.0011) [2023-12-27 07:42:45,334][20381] Updated weights for policy 1, policy_version 298290 (0.0011) [2023-12-27 07:42:45,380][20347] Updated weights for policy 0, policy_version 298624 (0.0010) [2023-12-27 07:42:45,390][20381] Updated weights for policy 1, policy_version 298300 (0.0010) [2023-12-27 07:42:45,449][20381] Updated weights for policy 1, policy_version 298310 (0.0010) [2023-12-27 07:42:46,044][20347] Updated weights for policy 0, policy_version 298634 (0.0010) [2023-12-27 07:42:46,073][20381] Updated weights for policy 1, policy_version 298320 (0.0008) [2023-12-27 07:42:46,104][20347] Updated weights for policy 0, policy_version 298644 (0.0006) [2023-12-27 07:42:46,128][20381] Updated weights for policy 1, policy_version 298330 (0.0006) [2023-12-27 07:42:46,159][20347] Updated weights for policy 0, policy_version 298654 (0.0005) [2023-12-27 07:42:46,174][20381] Updated weights for policy 1, policy_version 298340 (0.0006) [2023-12-27 07:42:46,789][20347] Updated weights for policy 0, policy_version 298664 (0.0009) [2023-12-27 07:42:46,838][20347] Updated weights for policy 0, policy_version 298674 (0.0010) [2023-12-27 07:42:46,840][20381] Updated weights for policy 1, policy_version 298350 (0.0007) [2023-12-27 07:42:46,888][20381] Updated weights for policy 1, policy_version 298360 (0.0006) [2023-12-27 07:42:46,893][20347] Updated weights for policy 0, policy_version 298684 (0.0010) [2023-12-27 07:42:46,939][20381] Updated weights for policy 1, policy_version 298370 (0.0007) [2023-12-27 07:42:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 152879104. Throughput: 0: 10420.3, 1: 10517.8. Samples: 152890180. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:47,363][19170] Avg episode reward: [(0, '401.210'), (1, '164.260')] [2023-12-27 07:42:47,544][20347] Updated weights for policy 0, policy_version 298694 (0.0010) [2023-12-27 07:42:47,578][20381] Updated weights for policy 1, policy_version 298380 (0.0007) [2023-12-27 07:42:47,602][20347] Updated weights for policy 0, policy_version 298704 (0.0010) [2023-12-27 07:42:47,634][20381] Updated weights for policy 1, policy_version 298390 (0.0005) [2023-12-27 07:42:47,653][20347] Updated weights for policy 0, policy_version 298714 (0.0010) [2023-12-27 07:42:47,686][20381] Updated weights for policy 1, policy_version 298400 (0.0005) [2023-12-27 07:42:48,301][20381] Updated weights for policy 1, policy_version 298410 (0.0005) [2023-12-27 07:42:48,314][20347] Updated weights for policy 0, policy_version 298724 (0.0008) [2023-12-27 07:42:48,369][20381] Updated weights for policy 1, policy_version 298420 (0.0007) [2023-12-27 07:42:48,374][20347] Updated weights for policy 0, policy_version 298734 (0.0010) [2023-12-27 07:42:48,423][20347] Updated weights for policy 0, policy_version 298744 (0.0010) [2023-12-27 07:42:48,425][20381] Updated weights for policy 1, policy_version 298430 (0.0005) [2023-12-27 07:42:48,485][20381] Updated weights for policy 1, policy_version 298440 (0.0006) [2023-12-27 07:42:49,102][20347] Updated weights for policy 0, policy_version 298754 (0.0010) [2023-12-27 07:42:49,116][20381] Updated weights for policy 1, policy_version 298450 (0.0006) [2023-12-27 07:42:49,151][20347] Updated weights for policy 0, policy_version 298764 (0.0010) [2023-12-27 07:42:49,171][20381] Updated weights for policy 1, policy_version 298460 (0.0005) [2023-12-27 07:42:49,199][20347] Updated weights for policy 0, policy_version 298774 (0.0010) [2023-12-27 07:42:49,225][20381] Updated weights for policy 1, policy_version 298470 (0.0008) [2023-12-27 07:42:49,262][20347] Updated weights for policy 0, policy_version 298784 (0.0011) [2023-12-27 07:42:49,959][20381] Updated weights for policy 1, policy_version 298480 (0.0008) [2023-12-27 07:42:49,999][20347] Updated weights for policy 0, policy_version 298794 (0.0006) [2023-12-27 07:42:50,013][20381] Updated weights for policy 1, policy_version 298490 (0.0006) [2023-12-27 07:42:50,054][20347] Updated weights for policy 0, policy_version 298804 (0.0005) [2023-12-27 07:42:50,068][20381] Updated weights for policy 1, policy_version 298500 (0.0006) [2023-12-27 07:42:50,109][20347] Updated weights for policy 0, policy_version 298814 (0.0006) [2023-12-27 07:42:50,746][20381] Updated weights for policy 1, policy_version 298510 (0.0008) [2023-12-27 07:42:50,807][20381] Updated weights for policy 1, policy_version 298520 (0.0009) [2023-12-27 07:42:50,836][20347] Updated weights for policy 0, policy_version 298824 (0.0006) [2023-12-27 07:42:50,861][20381] Updated weights for policy 1, policy_version 298530 (0.0008) [2023-12-27 07:42:50,894][20347] Updated weights for policy 0, policy_version 298834 (0.0007) [2023-12-27 07:42:50,954][20347] Updated weights for policy 0, policy_version 298844 (0.0008) [2023-12-27 07:42:51,572][20381] Updated weights for policy 1, policy_version 298540 (0.0006) [2023-12-27 07:42:51,627][20381] Updated weights for policy 1, policy_version 298550 (0.0008) [2023-12-27 07:42:51,656][20347] Updated weights for policy 0, policy_version 298854 (0.0007) [2023-12-27 07:42:51,686][20381] Updated weights for policy 1, policy_version 298560 (0.0007) [2023-12-27 07:42:51,715][20347] Updated weights for policy 0, policy_version 298864 (0.0008) [2023-12-27 07:42:51,778][20347] Updated weights for policy 0, policy_version 298874 (0.0007) [2023-12-27 07:42:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 152977408. Throughput: 0: 10384.4, 1: 10501.7. Samples: 152952300. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:52,363][19170] Avg episode reward: [(0, '382.190'), (1, '164.580')] [2023-12-27 07:42:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000298568_76447744.pth... [2023-12-27 07:42:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000298880_76529664.pth... [2023-12-27 07:42:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000297352_76136448.pth [2023-12-27 07:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000297664_76218368.pth [2023-12-27 07:42:52,403][20381] Updated weights for policy 1, policy_version 298570 (0.0008) [2023-12-27 07:42:52,430][20347] Updated weights for policy 0, policy_version 298884 (0.0007) [2023-12-27 07:42:52,453][20381] Updated weights for policy 1, policy_version 298580 (0.0007) [2023-12-27 07:42:52,493][20347] Updated weights for policy 0, policy_version 298894 (0.0005) [2023-12-27 07:42:52,504][20381] Updated weights for policy 1, policy_version 298590 (0.0008) [2023-12-27 07:42:52,549][20347] Updated weights for policy 0, policy_version 298904 (0.0005) [2023-12-27 07:42:52,562][20381] Updated weights for policy 1, policy_version 298600 (0.0007) [2023-12-27 07:42:53,176][20347] Updated weights for policy 0, policy_version 298914 (0.0007) [2023-12-27 07:42:53,234][20347] Updated weights for policy 0, policy_version 298924 (0.0007) [2023-12-27 07:42:53,283][20347] Updated weights for policy 0, policy_version 298934 (0.0008) [2023-12-27 07:42:53,300][20381] Updated weights for policy 1, policy_version 298610 (0.0007) [2023-12-27 07:42:53,335][20347] Updated weights for policy 0, policy_version 298944 (0.0006) [2023-12-27 07:42:53,345][20381] Updated weights for policy 1, policy_version 298620 (0.0006) [2023-12-27 07:42:53,388][20381] Updated weights for policy 1, policy_version 298630 (0.0008) [2023-12-27 07:42:54,005][20381] Updated weights for policy 1, policy_version 298640 (0.0007) [2023-12-27 07:42:54,051][20347] Updated weights for policy 0, policy_version 298954 (0.0007) [2023-12-27 07:42:54,056][20381] Updated weights for policy 1, policy_version 298650 (0.0005) [2023-12-27 07:42:54,107][20347] Updated weights for policy 0, policy_version 298964 (0.0006) [2023-12-27 07:42:54,112][20381] Updated weights for policy 1, policy_version 298660 (0.0006) [2023-12-27 07:42:54,161][20347] Updated weights for policy 0, policy_version 298974 (0.0007) [2023-12-27 07:42:54,814][20381] Updated weights for policy 1, policy_version 298670 (0.0007) [2023-12-27 07:42:54,851][20347] Updated weights for policy 0, policy_version 298984 (0.0008) [2023-12-27 07:42:54,872][20381] Updated weights for policy 1, policy_version 298680 (0.0007) [2023-12-27 07:42:54,907][20347] Updated weights for policy 0, policy_version 298994 (0.0006) [2023-12-27 07:42:54,934][20381] Updated weights for policy 1, policy_version 298690 (0.0007) [2023-12-27 07:42:54,963][20347] Updated weights for policy 0, policy_version 299004 (0.0006) [2023-12-27 07:42:55,630][20381] Updated weights for policy 1, policy_version 298700 (0.0006) [2023-12-27 07:42:55,643][20347] Updated weights for policy 0, policy_version 299014 (0.0006) [2023-12-27 07:42:55,682][20381] Updated weights for policy 1, policy_version 298710 (0.0006) [2023-12-27 07:42:55,698][20347] Updated weights for policy 0, policy_version 299024 (0.0008) [2023-12-27 07:42:55,732][20381] Updated weights for policy 1, policy_version 298720 (0.0009) [2023-12-27 07:42:55,751][20347] Updated weights for policy 0, policy_version 299034 (0.0008) [2023-12-27 07:42:56,391][20381] Updated weights for policy 1, policy_version 298730 (0.0008) [2023-12-27 07:42:56,443][20381] Updated weights for policy 1, policy_version 298740 (0.0008) [2023-12-27 07:42:56,470][20347] Updated weights for policy 0, policy_version 299044 (0.0006) [2023-12-27 07:42:56,501][20381] Updated weights for policy 1, policy_version 298750 (0.0006) [2023-12-27 07:42:56,521][20347] Updated weights for policy 0, policy_version 299054 (0.0005) [2023-12-27 07:42:56,559][20381] Updated weights for policy 1, policy_version 298760 (0.0006) [2023-12-27 07:42:56,572][20347] Updated weights for policy 0, policy_version 299064 (0.0006) [2023-12-27 07:42:57,220][20347] Updated weights for policy 0, policy_version 299074 (0.0007) [2023-12-27 07:42:57,260][20381] Updated weights for policy 1, policy_version 298770 (0.0006) [2023-12-27 07:42:57,269][20347] Updated weights for policy 0, policy_version 299084 (0.0006) [2023-12-27 07:42:57,314][20381] Updated weights for policy 1, policy_version 298780 (0.0007) [2023-12-27 07:42:57,322][20347] Updated weights for policy 0, policy_version 299094 (0.0005) [2023-12-27 07:42:57,360][20381] Updated weights for policy 1, policy_version 298790 (0.0007) [2023-12-27 07:42:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 153075712. Throughput: 0: 10393.4, 1: 10495.9. Samples: 153073812. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:42:57,363][19170] Avg episode reward: [(0, '487.880'), (1, '115.800')] [2023-12-27 07:42:57,382][20347] Updated weights for policy 0, policy_version 299104 (0.0005) [2023-12-27 07:42:57,989][20347] Updated weights for policy 0, policy_version 299114 (0.0008) [2023-12-27 07:42:58,046][20347] Updated weights for policy 0, policy_version 299124 (0.0007) [2023-12-27 07:42:58,094][20381] Updated weights for policy 1, policy_version 298800 (0.0006) [2023-12-27 07:42:58,101][20347] Updated weights for policy 0, policy_version 299134 (0.0007) [2023-12-27 07:42:58,159][20381] Updated weights for policy 1, policy_version 298810 (0.0007) [2023-12-27 07:42:58,221][20381] Updated weights for policy 1, policy_version 298820 (0.0008) [2023-12-27 07:42:58,916][20347] Updated weights for policy 0, policy_version 299144 (0.0008) [2023-12-27 07:42:58,941][20381] Updated weights for policy 1, policy_version 298830 (0.0007) [2023-12-27 07:42:58,984][20347] Updated weights for policy 0, policy_version 299154 (0.0007) [2023-12-27 07:42:58,999][20381] Updated weights for policy 1, policy_version 298840 (0.0009) [2023-12-27 07:42:59,036][20347] Updated weights for policy 0, policy_version 299164 (0.0005) [2023-12-27 07:42:59,058][20381] Updated weights for policy 1, policy_version 298850 (0.0008) [2023-12-27 07:42:59,716][20381] Updated weights for policy 1, policy_version 298860 (0.0007) [2023-12-27 07:42:59,773][20381] Updated weights for policy 1, policy_version 298870 (0.0006) [2023-12-27 07:42:59,774][20347] Updated weights for policy 0, policy_version 299174 (0.0006) [2023-12-27 07:42:59,827][20381] Updated weights for policy 1, policy_version 298880 (0.0006) [2023-12-27 07:42:59,828][20347] Updated weights for policy 0, policy_version 299184 (0.0008) [2023-12-27 07:42:59,890][20347] Updated weights for policy 0, policy_version 299194 (0.0007) [2023-12-27 07:43:00,384][20381] Updated weights for policy 1, policy_version 298890 (0.0008) [2023-12-27 07:43:00,436][20381] Updated weights for policy 1, policy_version 298900 (0.0007) [2023-12-27 07:43:00,493][20381] Updated weights for policy 1, policy_version 298910 (0.0007) [2023-12-27 07:43:00,555][20381] Updated weights for policy 1, policy_version 298920 (0.0005) [2023-12-27 07:43:00,593][20347] Updated weights for policy 0, policy_version 299204 (0.0008) [2023-12-27 07:43:00,637][20347] Updated weights for policy 0, policy_version 299214 (0.0010) [2023-12-27 07:43:00,694][20347] Updated weights for policy 0, policy_version 299224 (0.0010) [2023-12-27 07:43:01,201][20381] Updated weights for policy 1, policy_version 298930 (0.0008) [2023-12-27 07:43:01,267][20381] Updated weights for policy 1, policy_version 298940 (0.0007) [2023-12-27 07:43:01,332][20381] Updated weights for policy 1, policy_version 298950 (0.0009) [2023-12-27 07:43:01,392][20347] Updated weights for policy 0, policy_version 299234 (0.0010) [2023-12-27 07:43:01,450][20347] Updated weights for policy 0, policy_version 299244 (0.0008) [2023-12-27 07:43:01,499][20347] Updated weights for policy 0, policy_version 299254 (0.0007) [2023-12-27 07:43:01,556][20347] Updated weights for policy 0, policy_version 299264 (0.0007) [2023-12-27 07:43:01,939][20381] Updated weights for policy 1, policy_version 298960 (0.0006) [2023-12-27 07:43:01,998][20381] Updated weights for policy 1, policy_version 298970 (0.0005) [2023-12-27 07:43:02,052][20381] Updated weights for policy 1, policy_version 298980 (0.0007) [2023-12-27 07:43:02,242][20347] Updated weights for policy 0, policy_version 299274 (0.0005) [2023-12-27 07:43:02,297][20347] Updated weights for policy 0, policy_version 299284 (0.0008) [2023-12-27 07:43:02,352][20347] Updated weights for policy 0, policy_version 299294 (0.0008) [2023-12-27 07:43:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 153190400. Throughput: 0: 10304.1, 1: 10467.8. Samples: 153194772. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:02,363][19170] Avg episode reward: [(0, '583.840'), (1, '151.650')] [2023-12-27 07:43:02,640][20381] Updated weights for policy 1, policy_version 298990 (0.0006) [2023-12-27 07:43:02,699][20381] Updated weights for policy 1, policy_version 299000 (0.0009) [2023-12-27 07:43:02,756][20381] Updated weights for policy 1, policy_version 299010 (0.0006) [2023-12-27 07:43:03,084][20347] Updated weights for policy 0, policy_version 299304 (0.0008) [2023-12-27 07:43:03,133][20347] Updated weights for policy 0, policy_version 299314 (0.0008) [2023-12-27 07:43:03,191][20347] Updated weights for policy 0, policy_version 299324 (0.0008) [2023-12-27 07:43:03,330][20381] Updated weights for policy 1, policy_version 299020 (0.0007) [2023-12-27 07:43:03,371][20381] Updated weights for policy 1, policy_version 299030 (0.0008) [2023-12-27 07:43:03,415][20381] Updated weights for policy 1, policy_version 299040 (0.0007) [2023-12-27 07:43:03,843][20347] Updated weights for policy 0, policy_version 299334 (0.0007) [2023-12-27 07:43:03,904][20347] Updated weights for policy 0, policy_version 299344 (0.0006) [2023-12-27 07:43:03,968][20347] Updated weights for policy 0, policy_version 299354 (0.0008) [2023-12-27 07:43:04,070][20381] Updated weights for policy 1, policy_version 299050 (0.0006) [2023-12-27 07:43:04,132][20381] Updated weights for policy 1, policy_version 299060 (0.0008) [2023-12-27 07:43:04,189][20381] Updated weights for policy 1, policy_version 299070 (0.0008) [2023-12-27 07:43:04,248][20381] Updated weights for policy 1, policy_version 299080 (0.0007) [2023-12-27 07:43:04,651][20347] Updated weights for policy 0, policy_version 299364 (0.0007) [2023-12-27 07:43:04,705][20347] Updated weights for policy 0, policy_version 299374 (0.0008) [2023-12-27 07:43:04,762][20347] Updated weights for policy 0, policy_version 299384 (0.0008) [2023-12-27 07:43:04,948][20381] Updated weights for policy 1, policy_version 299090 (0.0008) [2023-12-27 07:43:05,006][20381] Updated weights for policy 1, policy_version 299100 (0.0008) [2023-12-27 07:43:05,062][20381] Updated weights for policy 1, policy_version 299110 (0.0005) [2023-12-27 07:43:05,503][20347] Updated weights for policy 0, policy_version 299394 (0.0008) [2023-12-27 07:43:05,552][20347] Updated weights for policy 0, policy_version 299404 (0.0008) [2023-12-27 07:43:05,599][20347] Updated weights for policy 0, policy_version 299414 (0.0005) [2023-12-27 07:43:05,642][20381] Updated weights for policy 1, policy_version 299120 (0.0005) [2023-12-27 07:43:05,656][20347] Updated weights for policy 0, policy_version 299424 (0.0005) [2023-12-27 07:43:05,699][20381] Updated weights for policy 1, policy_version 299130 (0.0009) [2023-12-27 07:43:05,751][20381] Updated weights for policy 1, policy_version 299140 (0.0010) [2023-12-27 07:43:06,279][20347] Updated weights for policy 0, policy_version 299434 (0.0008) [2023-12-27 07:43:06,336][20347] Updated weights for policy 0, policy_version 299444 (0.0010) [2023-12-27 07:43:06,389][20347] Updated weights for policy 0, policy_version 299454 (0.0010) [2023-12-27 07:43:06,455][20381] Updated weights for policy 1, policy_version 299150 (0.0009) [2023-12-27 07:43:06,510][20381] Updated weights for policy 1, policy_version 299160 (0.0008) [2023-12-27 07:43:06,562][20381] Updated weights for policy 1, policy_version 299170 (0.0008) [2023-12-27 07:43:07,097][20347] Updated weights for policy 0, policy_version 299464 (0.0010) [2023-12-27 07:43:07,158][20347] Updated weights for policy 0, policy_version 299474 (0.0010) [2023-12-27 07:43:07,221][20347] Updated weights for policy 0, policy_version 299484 (0.0010) [2023-12-27 07:43:07,294][20381] Updated weights for policy 1, policy_version 299180 (0.0007) [2023-12-27 07:43:07,356][20381] Updated weights for policy 1, policy_version 299190 (0.0007) [2023-12-27 07:43:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 153288704. Throughput: 0: 10319.2, 1: 10500.4. Samples: 153258052. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:07,363][19170] Avg episode reward: [(0, '645.260'), (1, '122.870')] [2023-12-27 07:43:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000299488_76685312.pth... [2023-12-27 07:43:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000298304_76382208.pth [2023-12-27 07:43:07,419][20381] Updated weights for policy 1, policy_version 299200 (0.0007) [2023-12-27 07:43:07,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000299208_76611584.pth... [2023-12-27 07:43:07,470][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000297992_76300288.pth [2023-12-27 07:43:07,876][20347] Updated weights for policy 0, policy_version 299494 (0.0007) [2023-12-27 07:43:07,928][20347] Updated weights for policy 0, policy_version 299504 (0.0005) [2023-12-27 07:43:07,979][20347] Updated weights for policy 0, policy_version 299514 (0.0007) [2023-12-27 07:43:08,110][20381] Updated weights for policy 1, policy_version 299210 (0.0008) [2023-12-27 07:43:08,167][20381] Updated weights for policy 1, policy_version 299220 (0.0007) [2023-12-27 07:43:08,222][20381] Updated weights for policy 1, policy_version 299230 (0.0006) [2023-12-27 07:43:08,275][20381] Updated weights for policy 1, policy_version 299240 (0.0007) [2023-12-27 07:43:08,621][20347] Updated weights for policy 0, policy_version 299524 (0.0009) [2023-12-27 07:43:08,686][20347] Updated weights for policy 0, policy_version 299534 (0.0008) [2023-12-27 07:43:08,756][20347] Updated weights for policy 0, policy_version 299544 (0.0008) [2023-12-27 07:43:08,961][20381] Updated weights for policy 1, policy_version 299250 (0.0007) [2023-12-27 07:43:09,020][20381] Updated weights for policy 1, policy_version 299260 (0.0008) [2023-12-27 07:43:09,073][20381] Updated weights for policy 1, policy_version 299270 (0.0008) [2023-12-27 07:43:09,481][20347] Updated weights for policy 0, policy_version 299554 (0.0009) [2023-12-27 07:43:09,544][20347] Updated weights for policy 0, policy_version 299564 (0.0008) [2023-12-27 07:43:09,610][20347] Updated weights for policy 0, policy_version 299574 (0.0007) [2023-12-27 07:43:09,679][20347] Updated weights for policy 0, policy_version 299584 (0.0008) [2023-12-27 07:43:09,746][20381] Updated weights for policy 1, policy_version 299280 (0.0007) [2023-12-27 07:43:09,805][20381] Updated weights for policy 1, policy_version 299290 (0.0008) [2023-12-27 07:43:09,867][20381] Updated weights for policy 1, policy_version 299300 (0.0007) [2023-12-27 07:43:10,329][20347] Updated weights for policy 0, policy_version 299594 (0.0008) [2023-12-27 07:43:10,390][20347] Updated weights for policy 0, policy_version 299604 (0.0008) [2023-12-27 07:43:10,449][20347] Updated weights for policy 0, policy_version 299614 (0.0007) [2023-12-27 07:43:10,562][20381] Updated weights for policy 1, policy_version 299310 (0.0010) [2023-12-27 07:43:10,618][20381] Updated weights for policy 1, policy_version 299320 (0.0011) [2023-12-27 07:43:10,679][20381] Updated weights for policy 1, policy_version 299330 (0.0010) [2023-12-27 07:43:11,085][20347] Updated weights for policy 0, policy_version 299624 (0.0007) [2023-12-27 07:43:11,151][20347] Updated weights for policy 0, policy_version 299634 (0.0008) [2023-12-27 07:43:11,213][20347] Updated weights for policy 0, policy_version 299644 (0.0007) [2023-12-27 07:43:11,423][20381] Updated weights for policy 1, policy_version 299340 (0.0011) [2023-12-27 07:43:11,484][20381] Updated weights for policy 1, policy_version 299350 (0.0011) [2023-12-27 07:43:11,547][20381] Updated weights for policy 1, policy_version 299360 (0.0011) [2023-12-27 07:43:11,914][20347] Updated weights for policy 0, policy_version 299654 (0.0007) [2023-12-27 07:43:11,975][20347] Updated weights for policy 0, policy_version 299664 (0.0007) [2023-12-27 07:43:12,029][20347] Updated weights for policy 0, policy_version 299674 (0.0007) [2023-12-27 07:43:12,259][20381] Updated weights for policy 1, policy_version 299370 (0.0011) [2023-12-27 07:43:12,320][20381] Updated weights for policy 1, policy_version 299380 (0.0011) [2023-12-27 07:43:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 153387008. Throughput: 0: 10270.7, 1: 10473.7. Samples: 153379996. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:12,363][19170] Avg episode reward: [(0, '800.750'), (1, '113.550')] [2023-12-27 07:43:12,381][20381] Updated weights for policy 1, policy_version 299390 (0.0011) [2023-12-27 07:43:12,434][20381] Updated weights for policy 1, policy_version 299400 (0.0010) [2023-12-27 07:43:12,796][20347] Updated weights for policy 0, policy_version 299684 (0.0007) [2023-12-27 07:43:12,847][20347] Updated weights for policy 0, policy_version 299694 (0.0007) [2023-12-27 07:43:12,857][20291] KL-divergence is very high: 157.6570 [2023-12-27 07:43:12,894][20291] KL-divergence is very high: 117.5687 [2023-12-27 07:43:12,894][20347] Updated weights for policy 0, policy_version 299704 (0.0007) [2023-12-27 07:43:13,131][20381] Updated weights for policy 1, policy_version 299410 (0.0011) [2023-12-27 07:43:13,190][20381] Updated weights for policy 1, policy_version 299420 (0.0008) [2023-12-27 07:43:13,242][20381] Updated weights for policy 1, policy_version 299430 (0.0005) [2023-12-27 07:43:13,577][20347] Updated weights for policy 0, policy_version 299714 (0.0006) [2023-12-27 07:43:13,632][20347] Updated weights for policy 0, policy_version 299724 (0.0007) [2023-12-27 07:43:13,689][20347] Updated weights for policy 0, policy_version 299734 (0.0005) [2023-12-27 07:43:13,748][20347] Updated weights for policy 0, policy_version 299744 (0.0005) [2023-12-27 07:43:13,826][20381] Updated weights for policy 1, policy_version 299440 (0.0009) [2023-12-27 07:43:13,885][20381] Updated weights for policy 1, policy_version 299450 (0.0007) [2023-12-27 07:43:13,949][20381] Updated weights for policy 1, policy_version 299460 (0.0008) [2023-12-27 07:43:14,317][20347] Updated weights for policy 0, policy_version 299754 (0.0007) [2023-12-27 07:43:14,366][20347] Updated weights for policy 0, policy_version 299764 (0.0010) [2023-12-27 07:43:14,424][20347] Updated weights for policy 0, policy_version 299774 (0.0010) [2023-12-27 07:43:14,554][20381] Updated weights for policy 1, policy_version 299470 (0.0007) [2023-12-27 07:43:14,607][20381] Updated weights for policy 1, policy_version 299480 (0.0006) [2023-12-27 07:43:14,665][20381] Updated weights for policy 1, policy_version 299490 (0.0006) [2023-12-27 07:43:15,112][20347] Updated weights for policy 0, policy_version 299784 (0.0008) [2023-12-27 07:43:15,166][20347] Updated weights for policy 0, policy_version 299794 (0.0008) [2023-12-27 07:43:15,220][20347] Updated weights for policy 0, policy_version 299804 (0.0007) [2023-12-27 07:43:15,308][20381] Updated weights for policy 1, policy_version 299500 (0.0007) [2023-12-27 07:43:15,367][20381] Updated weights for policy 1, policy_version 299510 (0.0008) [2023-12-27 07:43:15,424][20381] Updated weights for policy 1, policy_version 299520 (0.0007) [2023-12-27 07:43:15,860][20347] Updated weights for policy 0, policy_version 299814 (0.0006) [2023-12-27 07:43:15,916][20347] Updated weights for policy 0, policy_version 299824 (0.0005) [2023-12-27 07:43:15,962][20347] Updated weights for policy 0, policy_version 299834 (0.0005) [2023-12-27 07:43:16,169][20381] Updated weights for policy 1, policy_version 299530 (0.0008) [2023-12-27 07:43:16,218][20381] Updated weights for policy 1, policy_version 299540 (0.0007) [2023-12-27 07:43:16,264][20381] Updated weights for policy 1, policy_version 299550 (0.0006) [2023-12-27 07:43:16,318][20381] Updated weights for policy 1, policy_version 299560 (0.0007) [2023-12-27 07:43:16,513][20347] Updated weights for policy 0, policy_version 299844 (0.0005) [2023-12-27 07:43:16,573][20347] Updated weights for policy 0, policy_version 299854 (0.0005) [2023-12-27 07:43:16,630][20347] Updated weights for policy 0, policy_version 299864 (0.0007) [2023-12-27 07:43:17,048][20381] Updated weights for policy 1, policy_version 299570 (0.0008) [2023-12-27 07:43:17,105][20381] Updated weights for policy 1, policy_version 299580 (0.0008) [2023-12-27 07:43:17,164][20381] Updated weights for policy 1, policy_version 299590 (0.0007) [2023-12-27 07:43:17,230][20347] Updated weights for policy 0, policy_version 299874 (0.0007) [2023-12-27 07:43:17,288][20347] Updated weights for policy 0, policy_version 299884 (0.0008) [2023-12-27 07:43:17,336][20347] Updated weights for policy 0, policy_version 299894 (0.0007) [2023-12-27 07:43:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 153493504. Throughput: 0: 10293.0, 1: 10363.3. Samples: 153504144. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:17,363][19170] Avg episode reward: [(0, '863.030'), (1, '121.200')] [2023-12-27 07:43:17,388][20347] Updated weights for policy 0, policy_version 299904 (0.0008) [2023-12-27 07:43:17,846][20381] Updated weights for policy 1, policy_version 299600 (0.0007) [2023-12-27 07:43:17,913][20381] Updated weights for policy 1, policy_version 299610 (0.0006) [2023-12-27 07:43:17,972][20381] Updated weights for policy 1, policy_version 299620 (0.0006) [2023-12-27 07:43:18,089][20347] Updated weights for policy 0, policy_version 299914 (0.0007) [2023-12-27 07:43:18,147][20347] Updated weights for policy 0, policy_version 299924 (0.0007) [2023-12-27 07:43:18,206][20347] Updated weights for policy 0, policy_version 299934 (0.0007) [2023-12-27 07:43:18,610][20381] Updated weights for policy 1, policy_version 299630 (0.0006) [2023-12-27 07:43:18,670][20381] Updated weights for policy 1, policy_version 299640 (0.0006) [2023-12-27 07:43:18,732][20381] Updated weights for policy 1, policy_version 299650 (0.0008) [2023-12-27 07:43:18,840][20347] Updated weights for policy 0, policy_version 299944 (0.0006) [2023-12-27 07:43:18,899][20347] Updated weights for policy 0, policy_version 299954 (0.0006) [2023-12-27 07:43:18,954][20347] Updated weights for policy 0, policy_version 299964 (0.0008) [2023-12-27 07:43:19,354][20381] Updated weights for policy 1, policy_version 299660 (0.0007) [2023-12-27 07:43:19,415][20381] Updated weights for policy 1, policy_version 299670 (0.0008) [2023-12-27 07:43:19,474][20381] Updated weights for policy 1, policy_version 299680 (0.0008) [2023-12-27 07:43:19,609][20347] Updated weights for policy 0, policy_version 299974 (0.0006) [2023-12-27 07:43:19,664][20347] Updated weights for policy 0, policy_version 299984 (0.0006) [2023-12-27 07:43:19,722][20347] Updated weights for policy 0, policy_version 299994 (0.0008) [2023-12-27 07:43:20,212][20381] Updated weights for policy 1, policy_version 299690 (0.0009) [2023-12-27 07:43:20,266][20381] Updated weights for policy 1, policy_version 299700 (0.0008) [2023-12-27 07:43:20,321][20381] Updated weights for policy 1, policy_version 299710 (0.0007) [2023-12-27 07:43:20,382][20381] Updated weights for policy 1, policy_version 299720 (0.0007) [2023-12-27 07:43:20,436][20347] Updated weights for policy 0, policy_version 300004 (0.0008) [2023-12-27 07:43:20,486][20347] Updated weights for policy 0, policy_version 300014 (0.0008) [2023-12-27 07:43:20,530][20347] Updated weights for policy 0, policy_version 300024 (0.0007) [2023-12-27 07:43:21,086][20381] Updated weights for policy 1, policy_version 299730 (0.0008) [2023-12-27 07:43:21,147][20381] Updated weights for policy 1, policy_version 299740 (0.0007) [2023-12-27 07:43:21,210][20381] Updated weights for policy 1, policy_version 299750 (0.0006) [2023-12-27 07:43:21,301][20347] Updated weights for policy 0, policy_version 300034 (0.0008) [2023-12-27 07:43:21,368][20347] Updated weights for policy 0, policy_version 300044 (0.0009) [2023-12-27 07:43:21,426][20347] Updated weights for policy 0, policy_version 300054 (0.0008) [2023-12-27 07:43:21,485][20347] Updated weights for policy 0, policy_version 300064 (0.0007) [2023-12-27 07:43:21,895][20381] Updated weights for policy 1, policy_version 299760 (0.0006) [2023-12-27 07:43:21,955][20381] Updated weights for policy 1, policy_version 299770 (0.0006) [2023-12-27 07:43:22,012][20381] Updated weights for policy 1, policy_version 299780 (0.0006) [2023-12-27 07:43:22,294][20347] Updated weights for policy 0, policy_version 300074 (0.0008) [2023-12-27 07:43:22,359][20347] Updated weights for policy 0, policy_version 300084 (0.0008) [2023-12-27 07:43:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 153591808. Throughput: 0: 10200.3, 1: 10291.0. Samples: 153566328. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:22,363][19170] Avg episode reward: [(0, '815.520'), (1, '143.330')] [2023-12-27 07:43:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000299784_76759040.pth... [2023-12-27 07:43:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000298568_76447744.pth [2023-12-27 07:43:22,424][20347] Updated weights for policy 0, policy_version 300094 (0.0007) [2023-12-27 07:43:22,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000300096_76840960.pth... [2023-12-27 07:43:22,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000298880_76529664.pth [2023-12-27 07:43:22,704][20381] Updated weights for policy 1, policy_version 299790 (0.0007) [2023-12-27 07:43:22,761][20381] Updated weights for policy 1, policy_version 299800 (0.0008) [2023-12-27 07:43:22,817][20381] Updated weights for policy 1, policy_version 299810 (0.0007) [2023-12-27 07:43:23,086][20347] Updated weights for policy 0, policy_version 300104 (0.0008) [2023-12-27 07:43:23,141][20347] Updated weights for policy 0, policy_version 300114 (0.0009) [2023-12-27 07:43:23,202][20347] Updated weights for policy 0, policy_version 300124 (0.0006) [2023-12-27 07:43:23,480][20381] Updated weights for policy 1, policy_version 299820 (0.0007) [2023-12-27 07:43:23,531][20381] Updated weights for policy 1, policy_version 299830 (0.0007) [2023-12-27 07:43:23,585][20381] Updated weights for policy 1, policy_version 299840 (0.0007) [2023-12-27 07:43:23,919][20347] Updated weights for policy 0, policy_version 300134 (0.0007) [2023-12-27 07:43:23,974][20347] Updated weights for policy 0, policy_version 300144 (0.0008) [2023-12-27 07:43:24,034][20347] Updated weights for policy 0, policy_version 300154 (0.0010) [2023-12-27 07:43:24,285][20381] Updated weights for policy 1, policy_version 299850 (0.0008) [2023-12-27 07:43:24,351][20381] Updated weights for policy 1, policy_version 299860 (0.0008) [2023-12-27 07:43:24,409][20381] Updated weights for policy 1, policy_version 299870 (0.0008) [2023-12-27 07:43:24,463][20381] Updated weights for policy 1, policy_version 299880 (0.0007) [2023-12-27 07:43:24,669][20347] Updated weights for policy 0, policy_version 300164 (0.0010) [2023-12-27 07:43:24,720][20347] Updated weights for policy 0, policy_version 300174 (0.0010) [2023-12-27 07:43:24,774][20347] Updated weights for policy 0, policy_version 300184 (0.0010) [2023-12-27 07:43:25,077][20381] Updated weights for policy 1, policy_version 299890 (0.0005) [2023-12-27 07:43:25,140][20381] Updated weights for policy 1, policy_version 299900 (0.0005) [2023-12-27 07:43:25,196][20381] Updated weights for policy 1, policy_version 299910 (0.0006) [2023-12-27 07:43:25,455][20347] Updated weights for policy 0, policy_version 300194 (0.0010) [2023-12-27 07:43:25,518][20347] Updated weights for policy 0, policy_version 300204 (0.0010) [2023-12-27 07:43:25,566][20347] Updated weights for policy 0, policy_version 300214 (0.0010) [2023-12-27 07:43:25,623][20347] Updated weights for policy 0, policy_version 300224 (0.0010) [2023-12-27 07:43:25,812][20381] Updated weights for policy 1, policy_version 299920 (0.0006) [2023-12-27 07:43:25,872][20381] Updated weights for policy 1, policy_version 299930 (0.0008) [2023-12-27 07:43:25,936][20381] Updated weights for policy 1, policy_version 299940 (0.0007) [2023-12-27 07:43:26,298][20347] Updated weights for policy 0, policy_version 300234 (0.0005) [2023-12-27 07:43:26,358][20347] Updated weights for policy 0, policy_version 300244 (0.0005) [2023-12-27 07:43:26,418][20347] Updated weights for policy 0, policy_version 300254 (0.0005) [2023-12-27 07:43:26,539][20381] Updated weights for policy 1, policy_version 299950 (0.0007) [2023-12-27 07:43:26,590][20381] Updated weights for policy 1, policy_version 299960 (0.0005) [2023-12-27 07:43:26,640][20381] Updated weights for policy 1, policy_version 299970 (0.0005) [2023-12-27 07:43:27,020][20347] Updated weights for policy 0, policy_version 300264 (0.0009) [2023-12-27 07:43:27,070][20347] Updated weights for policy 0, policy_version 300274 (0.0005) [2023-12-27 07:43:27,121][20347] Updated weights for policy 0, policy_version 300284 (0.0005) [2023-12-27 07:43:27,155][20381] Updated weights for policy 1, policy_version 299980 (0.0007) [2023-12-27 07:43:27,207][20381] Updated weights for policy 1, policy_version 299990 (0.0010) [2023-12-27 07:43:27,264][20381] Updated weights for policy 1, policy_version 300000 (0.0010) [2023-12-27 07:43:27,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 153706496. Throughput: 0: 10157.5, 1: 10321.8. Samples: 153687860. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:27,363][19170] Avg episode reward: [(0, '796.820'), (1, '132.790')] [2023-12-27 07:43:27,753][20347] Updated weights for policy 0, policy_version 300294 (0.0005) [2023-12-27 07:43:27,802][20347] Updated weights for policy 0, policy_version 300304 (0.0005) [2023-12-27 07:43:27,845][20347] Updated weights for policy 0, policy_version 300314 (0.0005) [2023-12-27 07:43:27,874][20381] Updated weights for policy 1, policy_version 300010 (0.0008) [2023-12-27 07:43:27,936][20381] Updated weights for policy 1, policy_version 300020 (0.0005) [2023-12-27 07:43:27,994][20381] Updated weights for policy 1, policy_version 300030 (0.0005) [2023-12-27 07:43:28,057][20381] Updated weights for policy 1, policy_version 300040 (0.0005) [2023-12-27 07:43:28,430][20347] Updated weights for policy 0, policy_version 300324 (0.0006) [2023-12-27 07:43:28,486][20347] Updated weights for policy 0, policy_version 300334 (0.0005) [2023-12-27 07:43:28,544][20347] Updated weights for policy 0, policy_version 300344 (0.0009) [2023-12-27 07:43:28,625][20381] Updated weights for policy 1, policy_version 300050 (0.0005) [2023-12-27 07:43:28,677][20381] Updated weights for policy 1, policy_version 300060 (0.0009) [2023-12-27 07:43:28,732][20381] Updated weights for policy 1, policy_version 300070 (0.0010) [2023-12-27 07:43:29,177][20347] Updated weights for policy 0, policy_version 300354 (0.0009) [2023-12-27 07:43:29,243][20347] Updated weights for policy 0, policy_version 300364 (0.0007) [2023-12-27 07:43:29,297][20347] Updated weights for policy 0, policy_version 300374 (0.0007) [2023-12-27 07:43:29,355][20347] Updated weights for policy 0, policy_version 300384 (0.0007) [2023-12-27 07:43:29,421][20381] Updated weights for policy 1, policy_version 300080 (0.0009) [2023-12-27 07:43:29,495][20381] Updated weights for policy 1, policy_version 300090 (0.0010) [2023-12-27 07:43:29,547][20381] Updated weights for policy 1, policy_version 300100 (0.0011) [2023-12-27 07:43:29,992][20347] Updated weights for policy 0, policy_version 300394 (0.0007) [2023-12-27 07:43:30,047][20347] Updated weights for policy 0, policy_version 300404 (0.0008) [2023-12-27 07:43:30,099][20347] Updated weights for policy 0, policy_version 300414 (0.0007) [2023-12-27 07:43:30,204][20381] Updated weights for policy 1, policy_version 300110 (0.0008) [2023-12-27 07:43:30,260][20381] Updated weights for policy 1, policy_version 300120 (0.0008) [2023-12-27 07:43:30,315][20381] Updated weights for policy 1, policy_version 300130 (0.0007) [2023-12-27 07:43:30,815][20347] Updated weights for policy 0, policy_version 300424 (0.0008) [2023-12-27 07:43:30,866][20347] Updated weights for policy 0, policy_version 300434 (0.0007) [2023-12-27 07:43:30,912][20347] Updated weights for policy 0, policy_version 300444 (0.0008) [2023-12-27 07:43:30,963][20381] Updated weights for policy 1, policy_version 300140 (0.0008) [2023-12-27 07:43:31,012][20381] Updated weights for policy 1, policy_version 300150 (0.0007) [2023-12-27 07:43:31,068][20381] Updated weights for policy 1, policy_version 300160 (0.0008) [2023-12-27 07:43:31,621][20347] Updated weights for policy 0, policy_version 300454 (0.0008) [2023-12-27 07:43:31,679][20347] Updated weights for policy 0, policy_version 300464 (0.0007) [2023-12-27 07:43:31,747][20347] Updated weights for policy 0, policy_version 300474 (0.0008) [2023-12-27 07:43:31,763][20381] Updated weights for policy 1, policy_version 300170 (0.0008) [2023-12-27 07:43:31,820][20381] Updated weights for policy 1, policy_version 300180 (0.0008) [2023-12-27 07:43:31,875][20381] Updated weights for policy 1, policy_version 300190 (0.0007) [2023-12-27 07:43:31,927][20381] Updated weights for policy 1, policy_version 300200 (0.0008) [2023-12-27 07:43:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 153804800. Throughput: 0: 10218.9, 1: 10373.5. Samples: 153816836. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:32,363][19170] Avg episode reward: [(0, '848.510'), (1, '134.340')] [2023-12-27 07:43:32,459][20347] Updated weights for policy 0, policy_version 300484 (0.0006) [2023-12-27 07:43:32,511][20347] Updated weights for policy 0, policy_version 300494 (0.0007) [2023-12-27 07:43:32,568][20347] Updated weights for policy 0, policy_version 300504 (0.0006) [2023-12-27 07:43:32,576][20381] Updated weights for policy 1, policy_version 300210 (0.0008) [2023-12-27 07:43:32,633][20381] Updated weights for policy 1, policy_version 300220 (0.0006) [2023-12-27 07:43:32,692][20381] Updated weights for policy 1, policy_version 300230 (0.0008) [2023-12-27 07:43:33,281][20347] Updated weights for policy 0, policy_version 300514 (0.0006) [2023-12-27 07:43:33,324][20381] Updated weights for policy 1, policy_version 300240 (0.0006) [2023-12-27 07:43:33,332][20347] Updated weights for policy 0, policy_version 300524 (0.0008) [2023-12-27 07:43:33,368][20381] Updated weights for policy 1, policy_version 300250 (0.0005) [2023-12-27 07:43:33,380][20347] Updated weights for policy 0, policy_version 300534 (0.0008) [2023-12-27 07:43:33,423][20381] Updated weights for policy 1, policy_version 300260 (0.0005) [2023-12-27 07:43:33,425][20347] Updated weights for policy 0, policy_version 300544 (0.0007) [2023-12-27 07:43:33,951][20381] Updated weights for policy 1, policy_version 300270 (0.0005) [2023-12-27 07:43:34,001][20381] Updated weights for policy 1, policy_version 300280 (0.0008) [2023-12-27 07:43:34,047][20381] Updated weights for policy 1, policy_version 300290 (0.0006) [2023-12-27 07:43:34,210][20347] Updated weights for policy 0, policy_version 300554 (0.0008) [2023-12-27 07:43:34,267][20347] Updated weights for policy 0, policy_version 300564 (0.0008) [2023-12-27 07:43:34,321][20347] Updated weights for policy 0, policy_version 300574 (0.0007) [2023-12-27 07:43:34,713][20381] Updated weights for policy 1, policy_version 300300 (0.0007) [2023-12-27 07:43:34,766][20381] Updated weights for policy 1, policy_version 300310 (0.0010) [2023-12-27 07:43:34,818][20381] Updated weights for policy 1, policy_version 300320 (0.0010) [2023-12-27 07:43:35,043][20347] Updated weights for policy 0, policy_version 300584 (0.0008) [2023-12-27 07:43:35,098][20347] Updated weights for policy 0, policy_version 300594 (0.0008) [2023-12-27 07:43:35,154][20347] Updated weights for policy 0, policy_version 300604 (0.0008) [2023-12-27 07:43:35,424][20381] Updated weights for policy 1, policy_version 300330 (0.0010) [2023-12-27 07:43:35,472][20381] Updated weights for policy 1, policy_version 300340 (0.0010) [2023-12-27 07:43:35,516][20381] Updated weights for policy 1, policy_version 300350 (0.0010) [2023-12-27 07:43:35,572][20381] Updated weights for policy 1, policy_version 300360 (0.0005) [2023-12-27 07:43:35,935][20347] Updated weights for policy 0, policy_version 300614 (0.0008) [2023-12-27 07:43:35,988][20347] Updated weights for policy 0, policy_version 300624 (0.0007) [2023-12-27 07:43:36,032][20347] Updated weights for policy 0, policy_version 300634 (0.0007) [2023-12-27 07:43:36,167][20381] Updated weights for policy 1, policy_version 300370 (0.0011) [2023-12-27 07:43:36,225][20381] Updated weights for policy 1, policy_version 300380 (0.0010) [2023-12-27 07:43:36,283][20381] Updated weights for policy 1, policy_version 300390 (0.0010) [2023-12-27 07:43:36,748][20347] Updated weights for policy 0, policy_version 300644 (0.0007) [2023-12-27 07:43:36,803][20347] Updated weights for policy 0, policy_version 300654 (0.0007) [2023-12-27 07:43:36,852][20347] Updated weights for policy 0, policy_version 300664 (0.0006) [2023-12-27 07:43:36,963][20381] Updated weights for policy 1, policy_version 300400 (0.0010) [2023-12-27 07:43:37,018][20381] Updated weights for policy 1, policy_version 300410 (0.0010) [2023-12-27 07:43:37,064][20381] Updated weights for policy 1, policy_version 300420 (0.0010) [2023-12-27 07:43:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 153911296. Throughput: 0: 10168.4, 1: 10433.9. Samples: 153879404. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:37,363][19170] Avg episode reward: [(0, '895.310'), (1, '144.490')] [2023-12-27 07:43:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000300424_76922880.pth... [2023-12-27 07:43:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000300672_76988416.pth... [2023-12-27 07:43:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000299488_76685312.pth [2023-12-27 07:43:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000299208_76611584.pth [2023-12-27 07:43:37,554][20347] Updated weights for policy 0, policy_version 300674 (0.0006) [2023-12-27 07:43:37,602][20347] Updated weights for policy 0, policy_version 300684 (0.0007) [2023-12-27 07:43:37,648][20347] Updated weights for policy 0, policy_version 300694 (0.0007) [2023-12-27 07:43:37,673][20381] Updated weights for policy 1, policy_version 300430 (0.0010) [2023-12-27 07:43:37,693][20347] Updated weights for policy 0, policy_version 300704 (0.0005) [2023-12-27 07:43:37,728][20381] Updated weights for policy 1, policy_version 300440 (0.0010) [2023-12-27 07:43:37,773][20381] Updated weights for policy 1, policy_version 300450 (0.0006) [2023-12-27 07:43:38,367][20381] Updated weights for policy 1, policy_version 300460 (0.0008) [2023-12-27 07:43:38,423][20381] Updated weights for policy 1, policy_version 300470 (0.0009) [2023-12-27 07:43:38,429][20347] Updated weights for policy 0, policy_version 300714 (0.0007) [2023-12-27 07:43:38,479][20381] Updated weights for policy 1, policy_version 300480 (0.0010) [2023-12-27 07:43:38,489][20347] Updated weights for policy 0, policy_version 300724 (0.0005) [2023-12-27 07:43:38,548][20347] Updated weights for policy 0, policy_version 300734 (0.0006) [2023-12-27 07:43:39,202][20381] Updated weights for policy 1, policy_version 300490 (0.0010) [2023-12-27 07:43:39,240][20347] Updated weights for policy 0, policy_version 300744 (0.0008) [2023-12-27 07:43:39,272][20381] Updated weights for policy 1, policy_version 300500 (0.0009) [2023-12-27 07:43:39,303][20347] Updated weights for policy 0, policy_version 300754 (0.0010) [2023-12-27 07:43:39,334][20381] Updated weights for policy 1, policy_version 300510 (0.0008) [2023-12-27 07:43:39,375][20347] Updated weights for policy 0, policy_version 300765 (0.0008) [2023-12-27 07:43:39,409][20381] Updated weights for policy 1, policy_version 300520 (0.0008) [2023-12-27 07:43:40,106][20381] Updated weights for policy 1, policy_version 300530 (0.0007) [2023-12-27 07:43:40,163][20347] Updated weights for policy 0, policy_version 300775 (0.0007) [2023-12-27 07:43:40,167][20381] Updated weights for policy 1, policy_version 300540 (0.0009) [2023-12-27 07:43:40,229][20347] Updated weights for policy 0, policy_version 300785 (0.0006) [2023-12-27 07:43:40,229][20381] Updated weights for policy 1, policy_version 300550 (0.0008) [2023-12-27 07:43:40,289][20347] Updated weights for policy 0, policy_version 300795 (0.0007) [2023-12-27 07:43:40,861][20347] Updated weights for policy 0, policy_version 300805 (0.0006) [2023-12-27 07:43:40,912][20347] Updated weights for policy 0, policy_version 300815 (0.0008) [2023-12-27 07:43:40,958][20347] Updated weights for policy 0, policy_version 300825 (0.0006) [2023-12-27 07:43:40,969][20381] Updated weights for policy 1, policy_version 300560 (0.0007) [2023-12-27 07:43:41,020][20381] Updated weights for policy 1, policy_version 300570 (0.0007) [2023-12-27 07:43:41,078][20381] Updated weights for policy 1, policy_version 300580 (0.0008) [2023-12-27 07:43:41,672][20347] Updated weights for policy 0, policy_version 300835 (0.0007) [2023-12-27 07:43:41,739][20347] Updated weights for policy 0, policy_version 300845 (0.0008) [2023-12-27 07:43:41,807][20347] Updated weights for policy 0, policy_version 300855 (0.0008) [2023-12-27 07:43:41,865][20381] Updated weights for policy 1, policy_version 300590 (0.0009) [2023-12-27 07:43:41,927][20381] Updated weights for policy 1, policy_version 300600 (0.0010) [2023-12-27 07:43:41,986][20381] Updated weights for policy 1, policy_version 300610 (0.0011) [2023-12-27 07:43:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 154009600. Throughput: 0: 10136.7, 1: 10456.9. Samples: 154000524. Policy #0 lag: (min: 21.0, avg: 23.0, max: 53.0) [2023-12-27 07:43:42,363][19170] Avg episode reward: [(0, '845.010'), (1, '159.370')] [2023-12-27 07:43:42,502][20347] Updated weights for policy 0, policy_version 300865 (0.0008) [2023-12-27 07:43:42,557][20347] Updated weights for policy 0, policy_version 300875 (0.0008) [2023-12-27 07:43:42,622][20347] Updated weights for policy 0, policy_version 300885 (0.0010) [2023-12-27 07:43:42,669][20381] Updated weights for policy 1, policy_version 300620 (0.0011) [2023-12-27 07:43:42,678][20347] Updated weights for policy 0, policy_version 300895 (0.0010) [2023-12-27 07:43:42,725][20381] Updated weights for policy 1, policy_version 300630 (0.0011) [2023-12-27 07:43:42,784][20381] Updated weights for policy 1, policy_version 300640 (0.0010) [2023-12-27 07:43:43,337][20347] Updated weights for policy 0, policy_version 300905 (0.0010) [2023-12-27 07:43:43,386][20347] Updated weights for policy 0, policy_version 300915 (0.0010) [2023-12-27 07:43:43,435][20347] Updated weights for policy 0, policy_version 300925 (0.0010) [2023-12-27 07:43:43,476][20381] Updated weights for policy 1, policy_version 300650 (0.0010) [2023-12-27 07:43:43,533][20381] Updated weights for policy 1, policy_version 300660 (0.0010) [2023-12-27 07:43:43,595][20381] Updated weights for policy 1, policy_version 300670 (0.0011) [2023-12-27 07:43:43,650][20381] Updated weights for policy 1, policy_version 300680 (0.0011) [2023-12-27 07:43:44,080][20347] Updated weights for policy 0, policy_version 300935 (0.0010) [2023-12-27 07:43:44,140][20347] Updated weights for policy 0, policy_version 300945 (0.0010) [2023-12-27 07:43:44,201][20347] Updated weights for policy 0, policy_version 300955 (0.0010) [2023-12-27 07:43:44,325][20381] Updated weights for policy 1, policy_version 300690 (0.0006) [2023-12-27 07:43:44,385][20381] Updated weights for policy 1, policy_version 300700 (0.0011) [2023-12-27 07:43:44,447][20381] Updated weights for policy 1, policy_version 300710 (0.0011) [2023-12-27 07:43:44,833][20347] Updated weights for policy 0, policy_version 300965 (0.0007) [2023-12-27 07:43:44,890][20347] Updated weights for policy 0, policy_version 300975 (0.0007) [2023-12-27 07:43:44,949][20347] Updated weights for policy 0, policy_version 300985 (0.0007) [2023-12-27 07:43:45,061][20381] Updated weights for policy 1, policy_version 300720 (0.0010) [2023-12-27 07:43:45,124][20381] Updated weights for policy 1, policy_version 300730 (0.0010) [2023-12-27 07:43:45,180][20381] Updated weights for policy 1, policy_version 300740 (0.0008) [2023-12-27 07:43:45,562][20347] Updated weights for policy 0, policy_version 300995 (0.0008) [2023-12-27 07:43:45,617][20347] Updated weights for policy 0, policy_version 301005 (0.0010) [2023-12-27 07:43:45,668][20347] Updated weights for policy 0, policy_version 301015 (0.0010) [2023-12-27 07:43:45,771][20381] Updated weights for policy 1, policy_version 300750 (0.0006) [2023-12-27 07:43:45,835][20381] Updated weights for policy 1, policy_version 300760 (0.0008) [2023-12-27 07:43:45,890][20381] Updated weights for policy 1, policy_version 300770 (0.0008) [2023-12-27 07:43:46,339][20347] Updated weights for policy 0, policy_version 301025 (0.0007) [2023-12-27 07:43:46,395][20347] Updated weights for policy 0, policy_version 301035 (0.0009) [2023-12-27 07:43:46,450][20347] Updated weights for policy 0, policy_version 301045 (0.0010) [2023-12-27 07:43:46,494][20347] Updated weights for policy 0, policy_version 301055 (0.0009) [2023-12-27 07:43:46,551][20381] Updated weights for policy 1, policy_version 300780 (0.0007) [2023-12-27 07:43:46,606][20381] Updated weights for policy 1, policy_version 300790 (0.0007) [2023-12-27 07:43:46,662][20381] Updated weights for policy 1, policy_version 300800 (0.0009) [2023-12-27 07:43:47,112][20347] Updated weights for policy 0, policy_version 301065 (0.0006) [2023-12-27 07:43:47,163][20347] Updated weights for policy 0, policy_version 301075 (0.0006) [2023-12-27 07:43:47,216][20347] Updated weights for policy 0, policy_version 301085 (0.0008) [2023-12-27 07:43:47,361][20381] Updated weights for policy 1, policy_version 300810 (0.0009) [2023-12-27 07:43:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 154116096. Throughput: 0: 10244.0, 1: 10460.2. Samples: 154126460. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:43:47,363][19170] Avg episode reward: [(0, '922.540'), (1, '140.080')] [2023-12-27 07:43:47,413][20381] Updated weights for policy 1, policy_version 300820 (0.0008) [2023-12-27 07:43:47,468][20381] Updated weights for policy 1, policy_version 300830 (0.0008) [2023-12-27 07:43:47,515][20381] Updated weights for policy 1, policy_version 300840 (0.0008) [2023-12-27 07:43:47,877][20347] Updated weights for policy 0, policy_version 301095 (0.0008) [2023-12-27 07:43:47,932][20347] Updated weights for policy 0, policy_version 301105 (0.0007) [2023-12-27 07:43:47,992][20347] Updated weights for policy 0, policy_version 301115 (0.0007) [2023-12-27 07:43:48,236][20381] Updated weights for policy 1, policy_version 300850 (0.0011) [2023-12-27 07:43:48,297][20381] Updated weights for policy 1, policy_version 300860 (0.0011) [2023-12-27 07:43:48,359][20381] Updated weights for policy 1, policy_version 300870 (0.0010) [2023-12-27 07:43:48,654][20347] Updated weights for policy 0, policy_version 301125 (0.0007) [2023-12-27 07:43:48,712][20347] Updated weights for policy 0, policy_version 301135 (0.0007) [2023-12-27 07:43:48,770][20347] Updated weights for policy 0, policy_version 301145 (0.0007) [2023-12-27 07:43:49,062][20381] Updated weights for policy 1, policy_version 300880 (0.0011) [2023-12-27 07:43:49,114][20381] Updated weights for policy 1, policy_version 300890 (0.0010) [2023-12-27 07:43:49,170][20381] Updated weights for policy 1, policy_version 300900 (0.0010) [2023-12-27 07:43:49,465][20347] Updated weights for policy 0, policy_version 301155 (0.0008) [2023-12-27 07:43:49,519][20347] Updated weights for policy 0, policy_version 301165 (0.0007) [2023-12-27 07:43:49,577][20347] Updated weights for policy 0, policy_version 301175 (0.0008) [2023-12-27 07:43:49,598][20291] KL-divergence is very high: 133.9720 [2023-12-27 07:43:49,912][20381] Updated weights for policy 1, policy_version 300910 (0.0009) [2023-12-27 07:43:49,975][20381] Updated weights for policy 1, policy_version 300920 (0.0009) [2023-12-27 07:43:50,038][20381] Updated weights for policy 1, policy_version 300930 (0.0008) [2023-12-27 07:43:50,258][20347] Updated weights for policy 0, policy_version 301185 (0.0007) [2023-12-27 07:43:50,312][20347] Updated weights for policy 0, policy_version 301195 (0.0007) [2023-12-27 07:43:50,362][20347] Updated weights for policy 0, policy_version 301205 (0.0007) [2023-12-27 07:43:50,420][20347] Updated weights for policy 0, policy_version 301215 (0.0008) [2023-12-27 07:43:50,704][20381] Updated weights for policy 1, policy_version 300940 (0.0008) [2023-12-27 07:43:50,761][20381] Updated weights for policy 1, policy_version 300950 (0.0011) [2023-12-27 07:43:50,818][20381] Updated weights for policy 1, policy_version 300960 (0.0011) [2023-12-27 07:43:51,135][20347] Updated weights for policy 0, policy_version 301225 (0.0008) [2023-12-27 07:43:51,189][20347] Updated weights for policy 0, policy_version 301235 (0.0007) [2023-12-27 07:43:51,244][20347] Updated weights for policy 0, policy_version 301245 (0.0007) [2023-12-27 07:43:51,555][20381] Updated weights for policy 1, policy_version 300970 (0.0011) [2023-12-27 07:43:51,605][20381] Updated weights for policy 1, policy_version 300980 (0.0011) [2023-12-27 07:43:51,670][20381] Updated weights for policy 1, policy_version 300990 (0.0011) [2023-12-27 07:43:51,736][20381] Updated weights for policy 1, policy_version 301000 (0.0011) [2023-12-27 07:43:51,965][20347] Updated weights for policy 0, policy_version 301255 (0.0007) [2023-12-27 07:43:51,974][20291] KL-divergence is very high: 101.5229 [2023-12-27 07:43:52,013][20347] Updated weights for policy 0, policy_version 301265 (0.0007) [2023-12-27 07:43:52,064][20347] Updated weights for policy 0, policy_version 301275 (0.0007) [2023-12-27 07:43:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 154214400. Throughput: 0: 10250.7, 1: 10383.4. Samples: 154186588. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:43:52,363][19170] Avg episode reward: [(0, '942.080'), (1, '156.780')] [2023-12-27 07:43:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000301000_77070336.pth... [2023-12-27 07:43:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000301280_77144064.pth... [2023-12-27 07:43:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000300096_76840960.pth [2023-12-27 07:43:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000299784_76759040.pth [2023-12-27 07:43:52,454][20381] Updated weights for policy 1, policy_version 301010 (0.0008) [2023-12-27 07:43:52,500][20381] Updated weights for policy 1, policy_version 301020 (0.0008) [2023-12-27 07:43:52,553][20381] Updated weights for policy 1, policy_version 301030 (0.0006) [2023-12-27 07:43:52,792][20347] Updated weights for policy 0, policy_version 301285 (0.0007) [2023-12-27 07:43:52,849][20347] Updated weights for policy 0, policy_version 301295 (0.0007) [2023-12-27 07:43:52,907][20347] Updated weights for policy 0, policy_version 301305 (0.0006) [2023-12-27 07:43:53,254][20381] Updated weights for policy 1, policy_version 301040 (0.0008) [2023-12-27 07:43:53,310][20381] Updated weights for policy 1, policy_version 301050 (0.0009) [2023-12-27 07:43:53,369][20381] Updated weights for policy 1, policy_version 301060 (0.0008) [2023-12-27 07:43:53,465][20347] Updated weights for policy 0, policy_version 301315 (0.0007) [2023-12-27 07:43:53,530][20347] Updated weights for policy 0, policy_version 301325 (0.0005) [2023-12-27 07:43:53,591][20347] Updated weights for policy 0, policy_version 301335 (0.0008) [2023-12-27 07:43:54,103][20381] Updated weights for policy 1, policy_version 301070 (0.0007) [2023-12-27 07:43:54,158][20381] Updated weights for policy 1, policy_version 301080 (0.0006) [2023-12-27 07:43:54,217][20381] Updated weights for policy 1, policy_version 301090 (0.0008) [2023-12-27 07:43:54,250][20347] Updated weights for policy 0, policy_version 301345 (0.0008) [2023-12-27 07:43:54,304][20347] Updated weights for policy 0, policy_version 301355 (0.0008) [2023-12-27 07:43:54,365][20347] Updated weights for policy 0, policy_version 301365 (0.0009) [2023-12-27 07:43:54,419][20347] Updated weights for policy 0, policy_version 301375 (0.0007) [2023-12-27 07:43:54,844][20381] Updated weights for policy 1, policy_version 301100 (0.0006) [2023-12-27 07:43:54,900][20381] Updated weights for policy 1, policy_version 301110 (0.0005) [2023-12-27 07:43:54,955][20381] Updated weights for policy 1, policy_version 301120 (0.0005) [2023-12-27 07:43:55,151][20347] Updated weights for policy 0, policy_version 301385 (0.0007) [2023-12-27 07:43:55,201][20347] Updated weights for policy 0, policy_version 301395 (0.0007) [2023-12-27 07:43:55,262][20347] Updated weights for policy 0, policy_version 301405 (0.0007) [2023-12-27 07:43:55,508][20381] Updated weights for policy 1, policy_version 301130 (0.0005) [2023-12-27 07:43:55,562][20381] Updated weights for policy 1, policy_version 301140 (0.0005) [2023-12-27 07:43:55,617][20381] Updated weights for policy 1, policy_version 301150 (0.0005) [2023-12-27 07:43:55,666][20381] Updated weights for policy 1, policy_version 301160 (0.0007) [2023-12-27 07:43:55,966][20347] Updated weights for policy 0, policy_version 301415 (0.0007) [2023-12-27 07:43:56,014][20347] Updated weights for policy 0, policy_version 301425 (0.0007) [2023-12-27 07:43:56,064][20347] Updated weights for policy 0, policy_version 301435 (0.0007) [2023-12-27 07:43:56,316][20381] Updated weights for policy 1, policy_version 301170 (0.0008) [2023-12-27 07:43:56,364][20381] Updated weights for policy 1, policy_version 301180 (0.0007) [2023-12-27 07:43:56,419][20381] Updated weights for policy 1, policy_version 301190 (0.0008) [2023-12-27 07:43:56,751][20347] Updated weights for policy 0, policy_version 301445 (0.0007) [2023-12-27 07:43:56,806][20347] Updated weights for policy 0, policy_version 301455 (0.0005) [2023-12-27 07:43:56,863][20347] Updated weights for policy 0, policy_version 301465 (0.0005) [2023-12-27 07:43:57,045][20381] Updated weights for policy 1, policy_version 301200 (0.0007) [2023-12-27 07:43:57,088][20381] Updated weights for policy 1, policy_version 301210 (0.0006) [2023-12-27 07:43:57,136][20381] Updated weights for policy 1, policy_version 301220 (0.0007) [2023-12-27 07:43:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 154320896. Throughput: 0: 10221.7, 1: 10419.1. Samples: 154308836. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:43:57,363][19170] Avg episode reward: [(0, '961.500'), (1, '143.310')] [2023-12-27 07:43:57,425][20347] Updated weights for policy 0, policy_version 301475 (0.0006) [2023-12-27 07:43:57,472][20347] Updated weights for policy 0, policy_version 301485 (0.0008) [2023-12-27 07:43:57,523][20347] Updated weights for policy 0, policy_version 301495 (0.0007) [2023-12-27 07:43:57,841][20381] Updated weights for policy 1, policy_version 301230 (0.0009) [2023-12-27 07:43:57,897][20381] Updated weights for policy 1, policy_version 301240 (0.0010) [2023-12-27 07:43:57,957][20381] Updated weights for policy 1, policy_version 301250 (0.0010) [2023-12-27 07:43:58,176][20347] Updated weights for policy 0, policy_version 301505 (0.0007) [2023-12-27 07:43:58,232][20347] Updated weights for policy 0, policy_version 301515 (0.0008) [2023-12-27 07:43:58,295][20347] Updated weights for policy 0, policy_version 301525 (0.0008) [2023-12-27 07:43:58,361][20347] Updated weights for policy 0, policy_version 301535 (0.0007) [2023-12-27 07:43:58,656][20381] Updated weights for policy 1, policy_version 301260 (0.0010) [2023-12-27 07:43:58,722][20381] Updated weights for policy 1, policy_version 301271 (0.0009) [2023-12-27 07:43:58,784][20381] Updated weights for policy 1, policy_version 301281 (0.0007) [2023-12-27 07:43:59,069][20347] Updated weights for policy 0, policy_version 301545 (0.0007) [2023-12-27 07:43:59,132][20347] Updated weights for policy 0, policy_version 301555 (0.0008) [2023-12-27 07:43:59,195][20347] Updated weights for policy 0, policy_version 301565 (0.0008) [2023-12-27 07:43:59,552][20381] Updated weights for policy 1, policy_version 301291 (0.0008) [2023-12-27 07:43:59,604][20381] Updated weights for policy 1, policy_version 301301 (0.0007) [2023-12-27 07:43:59,662][20381] Updated weights for policy 1, policy_version 301311 (0.0008) [2023-12-27 07:43:59,915][20347] Updated weights for policy 0, policy_version 301575 (0.0008) [2023-12-27 07:43:59,972][20347] Updated weights for policy 0, policy_version 301585 (0.0007) [2023-12-27 07:44:00,033][20347] Updated weights for policy 0, policy_version 301595 (0.0007) [2023-12-27 07:44:00,368][20381] Updated weights for policy 1, policy_version 301321 (0.0007) [2023-12-27 07:44:00,425][20381] Updated weights for policy 1, policy_version 301331 (0.0007) [2023-12-27 07:44:00,485][20381] Updated weights for policy 1, policy_version 301341 (0.0007) [2023-12-27 07:44:00,535][20381] Updated weights for policy 1, policy_version 301351 (0.0007) [2023-12-27 07:44:00,692][20347] Updated weights for policy 0, policy_version 301605 (0.0009) [2023-12-27 07:44:00,743][20347] Updated weights for policy 0, policy_version 301615 (0.0010) [2023-12-27 07:44:00,792][20347] Updated weights for policy 0, policy_version 301625 (0.0008) [2023-12-27 07:44:01,249][20381] Updated weights for policy 1, policy_version 301361 (0.0008) [2023-12-27 07:44:01,306][20381] Updated weights for policy 1, policy_version 301371 (0.0007) [2023-12-27 07:44:01,368][20381] Updated weights for policy 1, policy_version 301381 (0.0007) [2023-12-27 07:44:01,432][20347] Updated weights for policy 0, policy_version 301635 (0.0007) [2023-12-27 07:44:01,497][20347] Updated weights for policy 0, policy_version 301645 (0.0011) [2023-12-27 07:44:01,560][20347] Updated weights for policy 0, policy_version 301655 (0.0011) [2023-12-27 07:44:01,985][20381] Updated weights for policy 1, policy_version 301391 (0.0008) [2023-12-27 07:44:02,033][20381] Updated weights for policy 1, policy_version 301401 (0.0010) [2023-12-27 07:44:02,087][20381] Updated weights for policy 1, policy_version 301411 (0.0010) [2023-12-27 07:44:02,242][20347] Updated weights for policy 0, policy_version 301665 (0.0010) [2023-12-27 07:44:02,303][20347] Updated weights for policy 0, policy_version 301675 (0.0007) [2023-12-27 07:44:02,357][20347] Updated weights for policy 0, policy_version 301685 (0.0007) [2023-12-27 07:44:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 154419200. Throughput: 0: 10193.7, 1: 10404.7. Samples: 154431072. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:02,363][19170] Avg episode reward: [(0, '981.330'), (1, '158.100')] [2023-12-27 07:44:02,416][20347] Updated weights for policy 0, policy_version 301695 (0.0007) [2023-12-27 07:44:02,689][20381] Updated weights for policy 1, policy_version 301421 (0.0008) [2023-12-27 07:44:02,745][20381] Updated weights for policy 1, policy_version 301431 (0.0006) [2023-12-27 07:44:02,801][20381] Updated weights for policy 1, policy_version 301441 (0.0009) [2023-12-27 07:44:03,100][20347] Updated weights for policy 0, policy_version 301705 (0.0010) [2023-12-27 07:44:03,158][20347] Updated weights for policy 0, policy_version 301715 (0.0010) [2023-12-27 07:44:03,209][20347] Updated weights for policy 0, policy_version 301725 (0.0010) [2023-12-27 07:44:03,394][20381] Updated weights for policy 1, policy_version 301451 (0.0009) [2023-12-27 07:44:03,438][20381] Updated weights for policy 1, policy_version 301461 (0.0010) [2023-12-27 07:44:03,489][20381] Updated weights for policy 1, policy_version 301471 (0.0010) [2023-12-27 07:44:03,781][20347] Updated weights for policy 0, policy_version 301735 (0.0009) [2023-12-27 07:44:03,842][20347] Updated weights for policy 0, policy_version 301745 (0.0010) [2023-12-27 07:44:03,900][20347] Updated weights for policy 0, policy_version 301755 (0.0007) [2023-12-27 07:44:04,175][20381] Updated weights for policy 1, policy_version 301481 (0.0010) [2023-12-27 07:44:04,228][20381] Updated weights for policy 1, policy_version 301491 (0.0011) [2023-12-27 07:44:04,294][20381] Updated weights for policy 1, policy_version 301501 (0.0011) [2023-12-27 07:44:04,357][20381] Updated weights for policy 1, policy_version 301511 (0.0011) [2023-12-27 07:44:04,510][20347] Updated weights for policy 0, policy_version 301765 (0.0008) [2023-12-27 07:44:04,569][20347] Updated weights for policy 0, policy_version 301775 (0.0010) [2023-12-27 07:44:04,637][20347] Updated weights for policy 0, policy_version 301785 (0.0010) [2023-12-27 07:44:05,024][20381] Updated weights for policy 1, policy_version 301521 (0.0010) [2023-12-27 07:44:05,085][20381] Updated weights for policy 1, policy_version 301531 (0.0011) [2023-12-27 07:44:05,149][20381] Updated weights for policy 1, policy_version 301541 (0.0011) [2023-12-27 07:44:05,227][20347] Updated weights for policy 0, policy_version 301795 (0.0010) [2023-12-27 07:44:05,282][20347] Updated weights for policy 0, policy_version 301805 (0.0006) [2023-12-27 07:44:05,340][20347] Updated weights for policy 0, policy_version 301815 (0.0010) [2023-12-27 07:44:05,825][20381] Updated weights for policy 1, policy_version 301551 (0.0010) [2023-12-27 07:44:05,876][20381] Updated weights for policy 1, policy_version 301561 (0.0010) [2023-12-27 07:44:05,934][20381] Updated weights for policy 1, policy_version 301571 (0.0011) [2023-12-27 07:44:05,983][20347] Updated weights for policy 0, policy_version 301825 (0.0010) [2023-12-27 07:44:06,037][20347] Updated weights for policy 0, policy_version 301835 (0.0010) [2023-12-27 07:44:06,089][20347] Updated weights for policy 0, policy_version 301845 (0.0009) [2023-12-27 07:44:06,153][20347] Updated weights for policy 0, policy_version 301855 (0.0007) [2023-12-27 07:44:06,679][20381] Updated weights for policy 1, policy_version 301581 (0.0009) [2023-12-27 07:44:06,731][20381] Updated weights for policy 1, policy_version 301591 (0.0007) [2023-12-27 07:44:06,780][20347] Updated weights for policy 0, policy_version 301865 (0.0008) [2023-12-27 07:44:06,785][20381] Updated weights for policy 1, policy_version 301601 (0.0005) [2023-12-27 07:44:06,835][20347] Updated weights for policy 0, policy_version 301875 (0.0006) [2023-12-27 07:44:06,902][20347] Updated weights for policy 0, policy_version 301885 (0.0006) [2023-12-27 07:44:07,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 154525696. Throughput: 0: 10242.9, 1: 10423.1. Samples: 154496292. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:07,363][19170] Avg episode reward: [(0, '981.330'), (1, '130.020')] [2023-12-27 07:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000301608_77225984.pth... [2023-12-27 07:44:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000301888_77299712.pth... [2023-12-27 07:44:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000300424_76922880.pth [2023-12-27 07:44:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000300672_76988416.pth [2023-12-27 07:44:07,447][20381] Updated weights for policy 1, policy_version 301611 (0.0007) [2023-12-27 07:44:07,502][20381] Updated weights for policy 1, policy_version 301621 (0.0011) [2023-12-27 07:44:07,518][20347] Updated weights for policy 0, policy_version 301895 (0.0005) [2023-12-27 07:44:07,556][20381] Updated weights for policy 1, policy_version 301631 (0.0010) [2023-12-27 07:44:07,572][20347] Updated weights for policy 0, policy_version 301905 (0.0005) [2023-12-27 07:44:07,618][20347] Updated weights for policy 0, policy_version 301915 (0.0006) [2023-12-27 07:44:08,201][20381] Updated weights for policy 1, policy_version 301641 (0.0010) [2023-12-27 07:44:08,255][20381] Updated weights for policy 1, policy_version 301651 (0.0005) [2023-12-27 07:44:08,308][20381] Updated weights for policy 1, policy_version 301661 (0.0006) [2023-12-27 07:44:08,334][20347] Updated weights for policy 0, policy_version 301925 (0.0008) [2023-12-27 07:44:08,371][20381] Updated weights for policy 1, policy_version 301671 (0.0008) [2023-12-27 07:44:08,394][20347] Updated weights for policy 0, policy_version 301935 (0.0009) [2023-12-27 07:44:08,453][20347] Updated weights for policy 0, policy_version 301945 (0.0010) [2023-12-27 07:44:08,986][20381] Updated weights for policy 1, policy_version 301681 (0.0007) [2023-12-27 07:44:09,031][20381] Updated weights for policy 1, policy_version 301691 (0.0007) [2023-12-27 07:44:09,095][20381] Updated weights for policy 1, policy_version 301701 (0.0007) [2023-12-27 07:44:09,119][20347] Updated weights for policy 0, policy_version 301955 (0.0009) [2023-12-27 07:44:09,174][20347] Updated weights for policy 0, policy_version 301965 (0.0008) [2023-12-27 07:44:09,229][20347] Updated weights for policy 0, policy_version 301975 (0.0008) [2023-12-27 07:44:09,811][20381] Updated weights for policy 1, policy_version 301711 (0.0007) [2023-12-27 07:44:09,880][20381] Updated weights for policy 1, policy_version 301721 (0.0008) [2023-12-27 07:44:09,941][20381] Updated weights for policy 1, policy_version 301731 (0.0007) [2023-12-27 07:44:09,988][20347] Updated weights for policy 0, policy_version 301985 (0.0007) [2023-12-27 07:44:10,039][20347] Updated weights for policy 0, policy_version 301995 (0.0008) [2023-12-27 07:44:10,093][20347] Updated weights for policy 0, policy_version 302005 (0.0008) [2023-12-27 07:44:10,156][20347] Updated weights for policy 0, policy_version 302015 (0.0008) [2023-12-27 07:44:10,578][20381] Updated weights for policy 1, policy_version 301741 (0.0008) [2023-12-27 07:44:10,640][20381] Updated weights for policy 1, policy_version 301751 (0.0007) [2023-12-27 07:44:10,703][20381] Updated weights for policy 1, policy_version 301761 (0.0007) [2023-12-27 07:44:10,942][20347] Updated weights for policy 0, policy_version 302025 (0.0009) [2023-12-27 07:44:10,998][20347] Updated weights for policy 0, policy_version 302035 (0.0008) [2023-12-27 07:44:11,062][20347] Updated weights for policy 0, policy_version 302045 (0.0007) [2023-12-27 07:44:11,473][20381] Updated weights for policy 1, policy_version 301771 (0.0008) [2023-12-27 07:44:11,533][20381] Updated weights for policy 1, policy_version 301781 (0.0007) [2023-12-27 07:44:11,593][20381] Updated weights for policy 1, policy_version 301791 (0.0008) [2023-12-27 07:44:11,826][20347] Updated weights for policy 0, policy_version 302055 (0.0010) [2023-12-27 07:44:11,886][20347] Updated weights for policy 0, policy_version 302065 (0.0011) [2023-12-27 07:44:11,942][20347] Updated weights for policy 0, policy_version 302075 (0.0011) [2023-12-27 07:44:12,268][20381] Updated weights for policy 1, policy_version 301801 (0.0007) [2023-12-27 07:44:12,336][20381] Updated weights for policy 1, policy_version 301811 (0.0008) [2023-12-27 07:44:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 154624000. Throughput: 0: 10279.9, 1: 10414.0. Samples: 154619084. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:12,363][19170] Avg episode reward: [(0, '980.710'), (1, '136.740')] [2023-12-27 07:44:12,400][20381] Updated weights for policy 1, policy_version 301821 (0.0006) [2023-12-27 07:44:12,457][20381] Updated weights for policy 1, policy_version 301831 (0.0007) [2023-12-27 07:44:12,620][20347] Updated weights for policy 0, policy_version 302085 (0.0011) [2023-12-27 07:44:12,675][20347] Updated weights for policy 0, policy_version 302095 (0.0010) [2023-12-27 07:44:12,734][20347] Updated weights for policy 0, policy_version 302105 (0.0010) [2023-12-27 07:44:13,118][20381] Updated weights for policy 1, policy_version 301841 (0.0008) [2023-12-27 07:44:13,178][20381] Updated weights for policy 1, policy_version 301851 (0.0008) [2023-12-27 07:44:13,225][20381] Updated weights for policy 1, policy_version 301861 (0.0010) [2023-12-27 07:44:13,413][20347] Updated weights for policy 0, policy_version 302115 (0.0010) [2023-12-27 07:44:13,474][20347] Updated weights for policy 0, policy_version 302125 (0.0010) [2023-12-27 07:44:13,533][20347] Updated weights for policy 0, policy_version 302135 (0.0010) [2023-12-27 07:44:13,934][20381] Updated weights for policy 1, policy_version 301871 (0.0008) [2023-12-27 07:44:13,980][20381] Updated weights for policy 1, policy_version 301881 (0.0007) [2023-12-27 07:44:14,035][20381] Updated weights for policy 1, policy_version 301891 (0.0007) [2023-12-27 07:44:14,204][20347] Updated weights for policy 0, policy_version 302145 (0.0010) [2023-12-27 07:44:14,261][20347] Updated weights for policy 0, policy_version 302155 (0.0008) [2023-12-27 07:44:14,319][20347] Updated weights for policy 0, policy_version 302165 (0.0008) [2023-12-27 07:44:14,372][20347] Updated weights for policy 0, policy_version 302175 (0.0008) [2023-12-27 07:44:14,734][20381] Updated weights for policy 1, policy_version 301901 (0.0007) [2023-12-27 07:44:14,803][20381] Updated weights for policy 1, policy_version 301911 (0.0008) [2023-12-27 07:44:14,858][20381] Updated weights for policy 1, policy_version 301921 (0.0008) [2023-12-27 07:44:15,091][20347] Updated weights for policy 0, policy_version 302185 (0.0007) [2023-12-27 07:44:15,155][20347] Updated weights for policy 0, policy_version 302195 (0.0007) [2023-12-27 07:44:15,216][20347] Updated weights for policy 0, policy_version 302205 (0.0008) [2023-12-27 07:44:15,545][20381] Updated weights for policy 1, policy_version 301931 (0.0007) [2023-12-27 07:44:15,597][20381] Updated weights for policy 1, policy_version 301941 (0.0008) [2023-12-27 07:44:15,649][20381] Updated weights for policy 1, policy_version 301951 (0.0007) [2023-12-27 07:44:15,922][20347] Updated weights for policy 0, policy_version 302215 (0.0007) [2023-12-27 07:44:15,973][20347] Updated weights for policy 0, policy_version 302225 (0.0008) [2023-12-27 07:44:16,020][20347] Updated weights for policy 0, policy_version 302235 (0.0008) [2023-12-27 07:44:16,386][20381] Updated weights for policy 1, policy_version 301961 (0.0008) [2023-12-27 07:44:16,446][20381] Updated weights for policy 1, policy_version 301971 (0.0008) [2023-12-27 07:44:16,496][20381] Updated weights for policy 1, policy_version 301981 (0.0008) [2023-12-27 07:44:16,540][20381] Updated weights for policy 1, policy_version 301991 (0.0007) [2023-12-27 07:44:16,660][20347] Updated weights for policy 0, policy_version 302245 (0.0007) [2023-12-27 07:44:16,717][20347] Updated weights for policy 0, policy_version 302255 (0.0008) [2023-12-27 07:44:16,768][20347] Updated weights for policy 0, policy_version 302265 (0.0008) [2023-12-27 07:44:17,186][20381] Updated weights for policy 1, policy_version 302001 (0.0006) [2023-12-27 07:44:17,237][20381] Updated weights for policy 1, policy_version 302011 (0.0008) [2023-12-27 07:44:17,290][20381] Updated weights for policy 1, policy_version 302021 (0.0009) [2023-12-27 07:44:17,359][20347] Updated weights for policy 0, policy_version 302275 (0.0009) [2023-12-27 07:44:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 154730496. Throughput: 0: 10199.9, 1: 10310.8. Samples: 154739816. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:17,363][19170] Avg episode reward: [(0, '980.710'), (1, '125.540')] [2023-12-27 07:44:17,411][20347] Updated weights for policy 0, policy_version 302285 (0.0010) [2023-12-27 07:44:17,461][20347] Updated weights for policy 0, policy_version 302295 (0.0010) [2023-12-27 07:44:18,012][20381] Updated weights for policy 1, policy_version 302031 (0.0007) [2023-12-27 07:44:18,071][20381] Updated weights for policy 1, policy_version 302041 (0.0010) [2023-12-27 07:44:18,074][20347] Updated weights for policy 0, policy_version 302305 (0.0005) [2023-12-27 07:44:18,125][20381] Updated weights for policy 1, policy_version 302051 (0.0007) [2023-12-27 07:44:18,130][20347] Updated weights for policy 0, policy_version 302315 (0.0005) [2023-12-27 07:44:18,185][20347] Updated weights for policy 0, policy_version 302325 (0.0006) [2023-12-27 07:44:18,240][20347] Updated weights for policy 0, policy_version 302335 (0.0010) [2023-12-27 07:44:18,706][20381] Updated weights for policy 1, policy_version 302061 (0.0006) [2023-12-27 07:44:18,760][20381] Updated weights for policy 1, policy_version 302071 (0.0006) [2023-12-27 07:44:18,820][20381] Updated weights for policy 1, policy_version 302081 (0.0005) [2023-12-27 07:44:18,878][20347] Updated weights for policy 0, policy_version 302345 (0.0010) [2023-12-27 07:44:18,928][20347] Updated weights for policy 0, policy_version 302355 (0.0010) [2023-12-27 07:44:18,977][20347] Updated weights for policy 0, policy_version 302365 (0.0010) [2023-12-27 07:44:19,431][20381] Updated weights for policy 1, policy_version 302091 (0.0006) [2023-12-27 07:44:19,492][20381] Updated weights for policy 1, policy_version 302101 (0.0007) [2023-12-27 07:44:19,557][20381] Updated weights for policy 1, policy_version 302111 (0.0010) [2023-12-27 07:44:19,675][20347] Updated weights for policy 0, policy_version 302375 (0.0010) [2023-12-27 07:44:19,736][20347] Updated weights for policy 0, policy_version 302385 (0.0010) [2023-12-27 07:44:19,792][20347] Updated weights for policy 0, policy_version 302395 (0.0010) [2023-12-27 07:44:20,230][20381] Updated weights for policy 1, policy_version 302121 (0.0010) [2023-12-27 07:44:20,288][20381] Updated weights for policy 1, policy_version 302131 (0.0006) [2023-12-27 07:44:20,356][20381] Updated weights for policy 1, policy_version 302141 (0.0009) [2023-12-27 07:44:20,417][20381] Updated weights for policy 1, policy_version 302151 (0.0011) [2023-12-27 07:44:20,482][20347] Updated weights for policy 0, policy_version 302405 (0.0011) [2023-12-27 07:44:20,531][20347] Updated weights for policy 0, policy_version 302415 (0.0010) [2023-12-27 07:44:20,581][20347] Updated weights for policy 0, policy_version 302425 (0.0009) [2023-12-27 07:44:21,112][20381] Updated weights for policy 1, policy_version 302161 (0.0010) [2023-12-27 07:44:21,170][20381] Updated weights for policy 1, policy_version 302171 (0.0011) [2023-12-27 07:44:21,237][20381] Updated weights for policy 1, policy_version 302181 (0.0011) [2023-12-27 07:44:21,316][20347] Updated weights for policy 0, policy_version 302435 (0.0007) [2023-12-27 07:44:21,378][20347] Updated weights for policy 0, policy_version 302445 (0.0007) [2023-12-27 07:44:21,445][20347] Updated weights for policy 0, policy_version 302455 (0.0006) [2023-12-27 07:44:21,949][20381] Updated weights for policy 1, policy_version 302191 (0.0011) [2023-12-27 07:44:21,999][20381] Updated weights for policy 1, policy_version 302201 (0.0010) [2023-12-27 07:44:22,058][20381] Updated weights for policy 1, policy_version 302211 (0.0011) [2023-12-27 07:44:22,115][20347] Updated weights for policy 0, policy_version 302465 (0.0007) [2023-12-27 07:44:22,175][20347] Updated weights for policy 0, policy_version 302475 (0.0007) [2023-12-27 07:44:22,232][20347] Updated weights for policy 0, policy_version 302485 (0.0007) [2023-12-27 07:44:22,292][20347] Updated weights for policy 0, policy_version 302495 (0.0006) [2023-12-27 07:44:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 154836992. Throughput: 0: 10286.3, 1: 10272.2. Samples: 154804532. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '152.190')] [2023-12-27 07:44:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000302216_77381632.pth... [2023-12-27 07:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000302496_77455360.pth... [2023-12-27 07:44:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000301000_77070336.pth [2023-12-27 07:44:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000301280_77144064.pth [2023-12-27 07:44:22,762][20381] Updated weights for policy 1, policy_version 302221 (0.0010) [2023-12-27 07:44:22,818][20381] Updated weights for policy 1, policy_version 302231 (0.0008) [2023-12-27 07:44:22,880][20381] Updated weights for policy 1, policy_version 302241 (0.0009) [2023-12-27 07:44:22,981][20347] Updated weights for policy 0, policy_version 302505 (0.0008) [2023-12-27 07:44:23,045][20347] Updated weights for policy 0, policy_version 302515 (0.0006) [2023-12-27 07:44:23,108][20347] Updated weights for policy 0, policy_version 302525 (0.0005) [2023-12-27 07:44:23,462][20381] Updated weights for policy 1, policy_version 302251 (0.0007) [2023-12-27 07:44:23,513][20381] Updated weights for policy 1, policy_version 302261 (0.0010) [2023-12-27 07:44:23,564][20381] Updated weights for policy 1, policy_version 302271 (0.0010) [2023-12-27 07:44:23,774][20347] Updated weights for policy 0, policy_version 302535 (0.0007) [2023-12-27 07:44:23,832][20347] Updated weights for policy 0, policy_version 302545 (0.0007) [2023-12-27 07:44:23,886][20347] Updated weights for policy 0, policy_version 302555 (0.0008) [2023-12-27 07:44:24,162][20381] Updated weights for policy 1, policy_version 302281 (0.0010) [2023-12-27 07:44:24,221][20381] Updated weights for policy 1, policy_version 302291 (0.0008) [2023-12-27 07:44:24,279][20381] Updated weights for policy 1, policy_version 302301 (0.0005) [2023-12-27 07:44:24,339][20381] Updated weights for policy 1, policy_version 302311 (0.0011) [2023-12-27 07:44:24,579][20347] Updated weights for policy 0, policy_version 302565 (0.0006) [2023-12-27 07:44:24,640][20347] Updated weights for policy 0, policy_version 302575 (0.0007) [2023-12-27 07:44:24,690][20347] Updated weights for policy 0, policy_version 302585 (0.0007) [2023-12-27 07:44:24,970][20381] Updated weights for policy 1, policy_version 302321 (0.0006) [2023-12-27 07:44:25,030][20381] Updated weights for policy 1, policy_version 302331 (0.0006) [2023-12-27 07:44:25,090][20381] Updated weights for policy 1, policy_version 302341 (0.0005) [2023-12-27 07:44:25,358][20347] Updated weights for policy 0, policy_version 302595 (0.0007) [2023-12-27 07:44:25,411][20347] Updated weights for policy 0, policy_version 302605 (0.0008) [2023-12-27 07:44:25,461][20347] Updated weights for policy 0, policy_version 302615 (0.0007) [2023-12-27 07:44:25,667][20381] Updated weights for policy 1, policy_version 302351 (0.0008) [2023-12-27 07:44:25,722][20381] Updated weights for policy 1, policy_version 302361 (0.0011) [2023-12-27 07:44:25,781][20381] Updated weights for policy 1, policy_version 302371 (0.0011) [2023-12-27 07:44:26,121][20347] Updated weights for policy 0, policy_version 302625 (0.0008) [2023-12-27 07:44:26,182][20347] Updated weights for policy 0, policy_version 302635 (0.0007) [2023-12-27 07:44:26,237][20347] Updated weights for policy 0, policy_version 302645 (0.0008) [2023-12-27 07:44:26,294][20347] Updated weights for policy 0, policy_version 302655 (0.0005) [2023-12-27 07:44:26,407][20381] Updated weights for policy 1, policy_version 302381 (0.0008) [2023-12-27 07:44:26,471][20381] Updated weights for policy 1, policy_version 302391 (0.0005) [2023-12-27 07:44:26,526][20381] Updated weights for policy 1, policy_version 302401 (0.0006) [2023-12-27 07:44:26,896][20347] Updated weights for policy 0, policy_version 302665 (0.0006) [2023-12-27 07:44:26,953][20347] Updated weights for policy 0, policy_version 302675 (0.0008) [2023-12-27 07:44:27,007][20347] Updated weights for policy 0, policy_version 302685 (0.0008) [2023-12-27 07:44:27,104][20381] Updated weights for policy 1, policy_version 302411 (0.0007) [2023-12-27 07:44:27,159][20381] Updated weights for policy 1, policy_version 302421 (0.0007) [2023-12-27 07:44:27,216][20381] Updated weights for policy 1, policy_version 302431 (0.0006) [2023-12-27 07:44:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 154943488. Throughput: 0: 10329.2, 1: 10295.4. Samples: 154928632. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '161.690')] [2023-12-27 07:44:27,597][20347] Updated weights for policy 0, policy_version 302695 (0.0006) [2023-12-27 07:44:27,642][20347] Updated weights for policy 0, policy_version 302705 (0.0006) [2023-12-27 07:44:27,686][20347] Updated weights for policy 0, policy_version 302715 (0.0007) [2023-12-27 07:44:27,835][20381] Updated weights for policy 1, policy_version 302441 (0.0008) [2023-12-27 07:44:27,895][20381] Updated weights for policy 1, policy_version 302451 (0.0005) [2023-12-27 07:44:27,950][20381] Updated weights for policy 1, policy_version 302461 (0.0005) [2023-12-27 07:44:28,009][20381] Updated weights for policy 1, policy_version 302471 (0.0006) [2023-12-27 07:44:28,249][20347] Updated weights for policy 0, policy_version 302725 (0.0006) [2023-12-27 07:44:28,305][20347] Updated weights for policy 0, policy_version 302735 (0.0007) [2023-12-27 07:44:28,387][20347] Updated weights for policy 0, policy_version 302745 (0.0009) [2023-12-27 07:44:28,612][20381] Updated weights for policy 1, policy_version 302481 (0.0006) [2023-12-27 07:44:28,673][20381] Updated weights for policy 1, policy_version 302491 (0.0008) [2023-12-27 07:44:28,738][20381] Updated weights for policy 1, policy_version 302501 (0.0008) [2023-12-27 07:44:28,966][20347] Updated weights for policy 0, policy_version 302755 (0.0007) [2023-12-27 07:44:29,014][20347] Updated weights for policy 0, policy_version 302765 (0.0010) [2023-12-27 07:44:29,059][20347] Updated weights for policy 0, policy_version 302775 (0.0010) [2023-12-27 07:44:29,423][20381] Updated weights for policy 1, policy_version 302511 (0.0006) [2023-12-27 07:44:29,483][20381] Updated weights for policy 1, policy_version 302521 (0.0006) [2023-12-27 07:44:29,540][20381] Updated weights for policy 1, policy_version 302531 (0.0007) [2023-12-27 07:44:29,745][20347] Updated weights for policy 0, policy_version 302785 (0.0009) [2023-12-27 07:44:29,807][20347] Updated weights for policy 0, policy_version 302795 (0.0010) [2023-12-27 07:44:29,867][20347] Updated weights for policy 0, policy_version 302805 (0.0010) [2023-12-27 07:44:29,921][20347] Updated weights for policy 0, policy_version 302815 (0.0006) [2023-12-27 07:44:30,146][20381] Updated weights for policy 1, policy_version 302541 (0.0006) [2023-12-27 07:44:30,197][20381] Updated weights for policy 1, policy_version 302551 (0.0005) [2023-12-27 07:44:30,246][20381] Updated weights for policy 1, policy_version 302561 (0.0005) [2023-12-27 07:44:30,590][20347] Updated weights for policy 0, policy_version 302825 (0.0010) [2023-12-27 07:44:30,644][20347] Updated weights for policy 0, policy_version 302835 (0.0010) [2023-12-27 07:44:30,691][20347] Updated weights for policy 0, policy_version 302845 (0.0010) [2023-12-27 07:44:30,860][20381] Updated weights for policy 1, policy_version 302571 (0.0005) [2023-12-27 07:44:30,905][20381] Updated weights for policy 1, policy_version 302581 (0.0005) [2023-12-27 07:44:30,956][20381] Updated weights for policy 1, policy_version 302591 (0.0005) [2023-12-27 07:44:31,392][20347] Updated weights for policy 0, policy_version 302855 (0.0008) [2023-12-27 07:44:31,447][20347] Updated weights for policy 0, policy_version 302865 (0.0005) [2023-12-27 07:44:31,501][20347] Updated weights for policy 0, policy_version 302875 (0.0005) [2023-12-27 07:44:31,665][20381] Updated weights for policy 1, policy_version 302601 (0.0005) [2023-12-27 07:44:31,718][20381] Updated weights for policy 1, policy_version 302611 (0.0008) [2023-12-27 07:44:31,774][20381] Updated weights for policy 1, policy_version 302621 (0.0008) [2023-12-27 07:44:31,833][20381] Updated weights for policy 1, policy_version 302631 (0.0008) [2023-12-27 07:44:32,131][20347] Updated weights for policy 0, policy_version 302885 (0.0006) [2023-12-27 07:44:32,178][20347] Updated weights for policy 0, policy_version 302895 (0.0008) [2023-12-27 07:44:32,222][20347] Updated weights for policy 0, policy_version 302905 (0.0007) [2023-12-27 07:44:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 155049984. Throughput: 0: 10368.8, 1: 10356.6. Samples: 155059104. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '164.360')] [2023-12-27 07:44:32,527][20381] Updated weights for policy 1, policy_version 302641 (0.0009) [2023-12-27 07:44:32,579][20381] Updated weights for policy 1, policy_version 302651 (0.0005) [2023-12-27 07:44:32,646][20381] Updated weights for policy 1, policy_version 302662 (0.0007) [2023-12-27 07:44:32,909][20347] Updated weights for policy 0, policy_version 302915 (0.0007) [2023-12-27 07:44:32,953][20347] Updated weights for policy 0, policy_version 302925 (0.0005) [2023-12-27 07:44:32,998][20347] Updated weights for policy 0, policy_version 302935 (0.0006) [2023-12-27 07:44:33,248][20381] Updated weights for policy 1, policy_version 302672 (0.0006) [2023-12-27 07:44:33,296][20381] Updated weights for policy 1, policy_version 302682 (0.0005) [2023-12-27 07:44:33,355][20381] Updated weights for policy 1, policy_version 302692 (0.0005) [2023-12-27 07:44:33,581][20347] Updated weights for policy 0, policy_version 302945 (0.0006) [2023-12-27 07:44:33,624][20347] Updated weights for policy 0, policy_version 302955 (0.0006) [2023-12-27 07:44:33,670][20347] Updated weights for policy 0, policy_version 302965 (0.0008) [2023-12-27 07:44:33,711][20347] Updated weights for policy 0, policy_version 302975 (0.0008) [2023-12-27 07:44:33,950][20381] Updated weights for policy 1, policy_version 302702 (0.0006) [2023-12-27 07:44:34,004][20381] Updated weights for policy 1, policy_version 302712 (0.0008) [2023-12-27 07:44:34,055][20381] Updated weights for policy 1, policy_version 302722 (0.0006) [2023-12-27 07:44:34,437][20347] Updated weights for policy 0, policy_version 302985 (0.0005) [2023-12-27 07:44:34,502][20347] Updated weights for policy 0, policy_version 302995 (0.0008) [2023-12-27 07:44:34,558][20347] Updated weights for policy 0, policy_version 303005 (0.0007) [2023-12-27 07:44:34,739][20381] Updated weights for policy 1, policy_version 302732 (0.0006) [2023-12-27 07:44:34,794][20381] Updated weights for policy 1, policy_version 302742 (0.0005) [2023-12-27 07:44:34,852][20381] Updated weights for policy 1, policy_version 302752 (0.0005) [2023-12-27 07:44:35,216][20347] Updated weights for policy 0, policy_version 303015 (0.0010) [2023-12-27 07:44:35,268][20347] Updated weights for policy 0, policy_version 303025 (0.0011) [2023-12-27 07:44:35,320][20347] Updated weights for policy 0, policy_version 303035 (0.0010) [2023-12-27 07:44:35,496][20381] Updated weights for policy 1, policy_version 302762 (0.0005) [2023-12-27 07:44:35,554][20381] Updated weights for policy 1, policy_version 302772 (0.0007) [2023-12-27 07:44:35,607][20381] Updated weights for policy 1, policy_version 302782 (0.0007) [2023-12-27 07:44:35,667][20381] Updated weights for policy 1, policy_version 302792 (0.0008) [2023-12-27 07:44:36,014][20347] Updated weights for policy 0, policy_version 303045 (0.0010) [2023-12-27 07:44:36,072][20347] Updated weights for policy 0, policy_version 303055 (0.0010) [2023-12-27 07:44:36,126][20347] Updated weights for policy 0, policy_version 303065 (0.0007) [2023-12-27 07:44:36,312][20381] Updated weights for policy 1, policy_version 302802 (0.0006) [2023-12-27 07:44:36,369][20381] Updated weights for policy 1, policy_version 302812 (0.0006) [2023-12-27 07:44:36,430][20381] Updated weights for policy 1, policy_version 302822 (0.0008) [2023-12-27 07:44:36,854][20347] Updated weights for policy 0, policy_version 303075 (0.0008) [2023-12-27 07:44:36,904][20347] Updated weights for policy 0, policy_version 303085 (0.0008) [2023-12-27 07:44:36,969][20347] Updated weights for policy 0, policy_version 303095 (0.0008) [2023-12-27 07:44:37,102][20381] Updated weights for policy 1, policy_version 302832 (0.0008) [2023-12-27 07:44:37,160][20381] Updated weights for policy 1, policy_version 302842 (0.0008) [2023-12-27 07:44:37,218][20381] Updated weights for policy 1, policy_version 302852 (0.0006) [2023-12-27 07:44:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 155156480. Throughput: 0: 10403.6, 1: 10435.1. Samples: 155124332. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '168.260')] [2023-12-27 07:44:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000303104_77611008.pth... [2023-12-27 07:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000302856_77545472.pth... [2023-12-27 07:44:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000301888_77299712.pth [2023-12-27 07:44:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000301608_77225984.pth [2023-12-27 07:44:37,647][20347] Updated weights for policy 0, policy_version 303105 (0.0008) [2023-12-27 07:44:37,699][20347] Updated weights for policy 0, policy_version 303115 (0.0007) [2023-12-27 07:44:37,748][20347] Updated weights for policy 0, policy_version 303125 (0.0007) [2023-12-27 07:44:37,796][20347] Updated weights for policy 0, policy_version 303135 (0.0008) [2023-12-27 07:44:37,851][20381] Updated weights for policy 1, policy_version 302862 (0.0007) [2023-12-27 07:44:37,895][20381] Updated weights for policy 1, policy_version 302872 (0.0007) [2023-12-27 07:44:37,953][20381] Updated weights for policy 1, policy_version 302882 (0.0008) [2023-12-27 07:44:38,466][20347] Updated weights for policy 0, policy_version 303145 (0.0010) [2023-12-27 07:44:38,520][20347] Updated weights for policy 0, policy_version 303155 (0.0010) [2023-12-27 07:44:38,575][20347] Updated weights for policy 0, policy_version 303165 (0.0010) [2023-12-27 07:44:38,648][20381] Updated weights for policy 1, policy_version 302892 (0.0007) [2023-12-27 07:44:38,696][20381] Updated weights for policy 1, policy_version 302902 (0.0008) [2023-12-27 07:44:38,744][20381] Updated weights for policy 1, policy_version 302912 (0.0010) [2023-12-27 07:44:39,180][20347] Updated weights for policy 0, policy_version 303175 (0.0008) [2023-12-27 07:44:39,236][20347] Updated weights for policy 0, policy_version 303185 (0.0011) [2023-12-27 07:44:39,291][20347] Updated weights for policy 0, policy_version 303195 (0.0010) [2023-12-27 07:44:39,375][20381] Updated weights for policy 1, policy_version 302922 (0.0008) [2023-12-27 07:44:39,435][20381] Updated weights for policy 1, policy_version 302932 (0.0008) [2023-12-27 07:44:39,497][20381] Updated weights for policy 1, policy_version 302942 (0.0005) [2023-12-27 07:44:39,562][20381] Updated weights for policy 1, policy_version 302952 (0.0005) [2023-12-27 07:44:39,914][20347] Updated weights for policy 0, policy_version 303205 (0.0008) [2023-12-27 07:44:39,978][20347] Updated weights for policy 0, policy_version 303215 (0.0007) [2023-12-27 07:44:40,047][20347] Updated weights for policy 0, policy_version 303225 (0.0006) [2023-12-27 07:44:40,206][20381] Updated weights for policy 1, policy_version 302962 (0.0006) [2023-12-27 07:44:40,263][20381] Updated weights for policy 1, policy_version 302972 (0.0008) [2023-12-27 07:44:40,322][20381] Updated weights for policy 1, policy_version 302982 (0.0007) [2023-12-27 07:44:40,698][20347] Updated weights for policy 0, policy_version 303235 (0.0006) [2023-12-27 07:44:40,750][20347] Updated weights for policy 0, policy_version 303245 (0.0007) [2023-12-27 07:44:40,807][20347] Updated weights for policy 0, policy_version 303255 (0.0007) [2023-12-27 07:44:41,030][20381] Updated weights for policy 1, policy_version 302992 (0.0007) [2023-12-27 07:44:41,091][20381] Updated weights for policy 1, policy_version 303002 (0.0006) [2023-12-27 07:44:41,149][20381] Updated weights for policy 1, policy_version 303012 (0.0007) [2023-12-27 07:44:41,608][20347] Updated weights for policy 0, policy_version 303265 (0.0008) [2023-12-27 07:44:41,675][20347] Updated weights for policy 0, policy_version 303275 (0.0010) [2023-12-27 07:44:41,743][20347] Updated weights for policy 0, policy_version 303285 (0.0009) [2023-12-27 07:44:41,799][20347] Updated weights for policy 0, policy_version 303295 (0.0007) [2023-12-27 07:44:41,887][20381] Updated weights for policy 1, policy_version 303022 (0.0006) [2023-12-27 07:44:41,939][20381] Updated weights for policy 1, policy_version 303032 (0.0005) [2023-12-27 07:44:41,992][20381] Updated weights for policy 1, policy_version 303042 (0.0006) [2023-12-27 07:44:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 155254784. Throughput: 0: 10441.4, 1: 10434.3. Samples: 155248240. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '168.220')] [2023-12-27 07:44:42,508][20347] Updated weights for policy 0, policy_version 303305 (0.0009) [2023-12-27 07:44:42,561][20347] Updated weights for policy 0, policy_version 303315 (0.0009) [2023-12-27 07:44:42,623][20347] Updated weights for policy 0, policy_version 303325 (0.0010) [2023-12-27 07:44:42,652][20381] Updated weights for policy 1, policy_version 303052 (0.0006) [2023-12-27 07:44:42,713][20381] Updated weights for policy 1, policy_version 303062 (0.0006) [2023-12-27 07:44:42,776][20381] Updated weights for policy 1, policy_version 303072 (0.0006) [2023-12-27 07:44:43,307][20347] Updated weights for policy 0, policy_version 303335 (0.0007) [2023-12-27 07:44:43,375][20347] Updated weights for policy 0, policy_version 303345 (0.0008) [2023-12-27 07:44:43,400][20381] Updated weights for policy 1, policy_version 303082 (0.0006) [2023-12-27 07:44:43,431][20347] Updated weights for policy 0, policy_version 303355 (0.0007) [2023-12-27 07:44:43,456][20381] Updated weights for policy 1, policy_version 303092 (0.0005) [2023-12-27 07:44:43,519][20381] Updated weights for policy 1, policy_version 303102 (0.0006) [2023-12-27 07:44:43,583][20381] Updated weights for policy 1, policy_version 303112 (0.0007) [2023-12-27 07:44:44,070][20347] Updated weights for policy 0, policy_version 303365 (0.0006) [2023-12-27 07:44:44,124][20347] Updated weights for policy 0, policy_version 303375 (0.0007) [2023-12-27 07:44:44,170][20381] Updated weights for policy 1, policy_version 303122 (0.0011) [2023-12-27 07:44:44,183][20347] Updated weights for policy 0, policy_version 303385 (0.0006) [2023-12-27 07:44:44,219][20381] Updated weights for policy 1, policy_version 303132 (0.0011) [2023-12-27 07:44:44,271][20381] Updated weights for policy 1, policy_version 303142 (0.0010) [2023-12-27 07:44:44,889][20347] Updated weights for policy 0, policy_version 303395 (0.0007) [2023-12-27 07:44:44,955][20347] Updated weights for policy 0, policy_version 303405 (0.0008) [2023-12-27 07:44:44,990][20381] Updated weights for policy 1, policy_version 303152 (0.0008) [2023-12-27 07:44:45,006][20347] Updated weights for policy 0, policy_version 303415 (0.0006) [2023-12-27 07:44:45,049][20381] Updated weights for policy 1, policy_version 303162 (0.0006) [2023-12-27 07:44:45,107][20381] Updated weights for policy 1, policy_version 303172 (0.0007) [2023-12-27 07:44:45,711][20381] Updated weights for policy 1, policy_version 303182 (0.0007) [2023-12-27 07:44:45,739][20347] Updated weights for policy 0, policy_version 303425 (0.0006) [2023-12-27 07:44:45,763][20381] Updated weights for policy 1, policy_version 303192 (0.0007) [2023-12-27 07:44:45,783][20347] Updated weights for policy 0, policy_version 303435 (0.0005) [2023-12-27 07:44:45,817][20381] Updated weights for policy 1, policy_version 303202 (0.0007) [2023-12-27 07:44:45,833][20347] Updated weights for policy 0, policy_version 303445 (0.0007) [2023-12-27 07:44:45,881][20347] Updated weights for policy 0, policy_version 303455 (0.0007) [2023-12-27 07:44:46,444][20381] Updated weights for policy 1, policy_version 303212 (0.0005) [2023-12-27 07:44:46,498][20381] Updated weights for policy 1, policy_version 303222 (0.0005) [2023-12-27 07:44:46,549][20381] Updated weights for policy 1, policy_version 303232 (0.0009) [2023-12-27 07:44:46,595][20347] Updated weights for policy 0, policy_version 303465 (0.0010) [2023-12-27 07:44:46,649][20347] Updated weights for policy 0, policy_version 303475 (0.0010) [2023-12-27 07:44:46,707][20347] Updated weights for policy 0, policy_version 303485 (0.0010) [2023-12-27 07:44:47,202][20381] Updated weights for policy 1, policy_version 303242 (0.0010) [2023-12-27 07:44:47,256][20381] Updated weights for policy 1, policy_version 303252 (0.0008) [2023-12-27 07:44:47,311][20381] Updated weights for policy 1, policy_version 303262 (0.0010) [2023-12-27 07:44:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 155353088. Throughput: 0: 10374.8, 1: 10506.3. Samples: 155370724. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '183.260')] [2023-12-27 07:44:47,367][20381] Updated weights for policy 1, policy_version 303272 (0.0010) [2023-12-27 07:44:47,386][20347] Updated weights for policy 0, policy_version 303495 (0.0009) [2023-12-27 07:44:47,434][20347] Updated weights for policy 0, policy_version 303505 (0.0010) [2023-12-27 07:44:47,488][20347] Updated weights for policy 0, policy_version 303515 (0.0010) [2023-12-27 07:44:47,998][20381] Updated weights for policy 1, policy_version 303282 (0.0005) [2023-12-27 07:44:48,051][20381] Updated weights for policy 1, policy_version 303292 (0.0005) [2023-12-27 07:44:48,102][20381] Updated weights for policy 1, policy_version 303302 (0.0005) [2023-12-27 07:44:48,187][20347] Updated weights for policy 0, policy_version 303525 (0.0010) [2023-12-27 07:44:48,242][20347] Updated weights for policy 0, policy_version 303535 (0.0010) [2023-12-27 07:44:48,297][20347] Updated weights for policy 0, policy_version 303545 (0.0010) [2023-12-27 07:44:48,739][20381] Updated weights for policy 1, policy_version 303312 (0.0006) [2023-12-27 07:44:48,787][20381] Updated weights for policy 1, policy_version 303322 (0.0006) [2023-12-27 07:44:48,847][20381] Updated weights for policy 1, policy_version 303332 (0.0006) [2023-12-27 07:44:48,967][20347] Updated weights for policy 0, policy_version 303555 (0.0010) [2023-12-27 07:44:49,026][20347] Updated weights for policy 0, policy_version 303565 (0.0010) [2023-12-27 07:44:49,081][20347] Updated weights for policy 0, policy_version 303575 (0.0010) [2023-12-27 07:44:49,504][20381] Updated weights for policy 1, policy_version 303342 (0.0007) [2023-12-27 07:44:49,561][20381] Updated weights for policy 1, policy_version 303352 (0.0007) [2023-12-27 07:44:49,624][20381] Updated weights for policy 1, policy_version 303362 (0.0007) [2023-12-27 07:44:49,754][20347] Updated weights for policy 0, policy_version 303585 (0.0010) [2023-12-27 07:44:49,813][20347] Updated weights for policy 0, policy_version 303595 (0.0010) [2023-12-27 07:44:49,875][20347] Updated weights for policy 0, policy_version 303605 (0.0010) [2023-12-27 07:44:49,934][20347] Updated weights for policy 0, policy_version 303615 (0.0010) [2023-12-27 07:44:50,313][20381] Updated weights for policy 1, policy_version 303372 (0.0008) [2023-12-27 07:44:50,373][20381] Updated weights for policy 1, policy_version 303382 (0.0006) [2023-12-27 07:44:50,435][20381] Updated weights for policy 1, policy_version 303392 (0.0011) [2023-12-27 07:44:50,565][20347] Updated weights for policy 0, policy_version 303625 (0.0010) [2023-12-27 07:44:50,625][20347] Updated weights for policy 0, policy_version 303635 (0.0009) [2023-12-27 07:44:50,682][20347] Updated weights for policy 0, policy_version 303645 (0.0010) [2023-12-27 07:44:51,135][20381] Updated weights for policy 1, policy_version 303402 (0.0009) [2023-12-27 07:44:51,190][20381] Updated weights for policy 1, policy_version 303412 (0.0005) [2023-12-27 07:44:51,251][20381] Updated weights for policy 1, policy_version 303422 (0.0007) [2023-12-27 07:44:51,292][20347] Updated weights for policy 0, policy_version 303655 (0.0010) [2023-12-27 07:44:51,305][20381] Updated weights for policy 1, policy_version 303432 (0.0006) [2023-12-27 07:44:51,356][20347] Updated weights for policy 0, policy_version 303665 (0.0012) [2023-12-27 07:44:51,418][20347] Updated weights for policy 0, policy_version 303675 (0.0007) [2023-12-27 07:44:51,912][20381] Updated weights for policy 1, policy_version 303442 (0.0010) [2023-12-27 07:44:51,963][20381] Updated weights for policy 1, policy_version 303452 (0.0007) [2023-12-27 07:44:52,024][20381] Updated weights for policy 1, policy_version 303462 (0.0007) [2023-12-27 07:44:52,027][20347] Updated weights for policy 0, policy_version 303685 (0.0009) [2023-12-27 07:44:52,076][20347] Updated weights for policy 0, policy_version 303695 (0.0006) [2023-12-27 07:44:52,136][20347] Updated weights for policy 0, policy_version 303705 (0.0006) [2023-12-27 07:44:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 155467776. Throughput: 0: 10318.5, 1: 10499.5. Samples: 155433108. Policy #0 lag: (min: 21.0, avg: 28.4, max: 53.0) [2023-12-27 07:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '170.100')] [2023-12-27 07:44:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000303464_77701120.pth... [2023-12-27 07:44:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000303712_77766656.pth... [2023-12-27 07:44:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000302216_77381632.pth [2023-12-27 07:44:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000302496_77455360.pth [2023-12-27 07:44:52,728][20347] Updated weights for policy 0, policy_version 303715 (0.0006) [2023-12-27 07:44:52,775][20347] Updated weights for policy 0, policy_version 303725 (0.0007) [2023-12-27 07:44:52,790][20381] Updated weights for policy 1, policy_version 303472 (0.0008) [2023-12-27 07:44:52,826][20347] Updated weights for policy 0, policy_version 303735 (0.0006) [2023-12-27 07:44:52,843][20381] Updated weights for policy 1, policy_version 303482 (0.0006) [2023-12-27 07:44:52,889][20381] Updated weights for policy 1, policy_version 303492 (0.0007) [2023-12-27 07:44:53,467][20347] Updated weights for policy 0, policy_version 303745 (0.0007) [2023-12-27 07:44:53,527][20347] Updated weights for policy 0, policy_version 303755 (0.0010) [2023-12-27 07:44:53,575][20347] Updated weights for policy 0, policy_version 303765 (0.0010) [2023-12-27 07:44:53,625][20347] Updated weights for policy 0, policy_version 303775 (0.0007) [2023-12-27 07:44:53,626][20381] Updated weights for policy 1, policy_version 303502 (0.0007) [2023-12-27 07:44:53,693][20381] Updated weights for policy 1, policy_version 303512 (0.0007) [2023-12-27 07:44:53,756][20381] Updated weights for policy 1, policy_version 303522 (0.0007) [2023-12-27 07:44:54,323][20347] Updated weights for policy 0, policy_version 303785 (0.0010) [2023-12-27 07:44:54,384][20347] Updated weights for policy 0, policy_version 303795 (0.0009) [2023-12-27 07:44:54,425][20381] Updated weights for policy 1, policy_version 303532 (0.0008) [2023-12-27 07:44:54,448][20347] Updated weights for policy 0, policy_version 303805 (0.0005) [2023-12-27 07:44:54,484][20381] Updated weights for policy 1, policy_version 303542 (0.0007) [2023-12-27 07:44:54,538][20381] Updated weights for policy 1, policy_version 303552 (0.0008) [2023-12-27 07:44:55,049][20347] Updated weights for policy 0, policy_version 303815 (0.0009) [2023-12-27 07:44:55,113][20347] Updated weights for policy 0, policy_version 303825 (0.0010) [2023-12-27 07:44:55,173][20381] Updated weights for policy 1, policy_version 303562 (0.0007) [2023-12-27 07:44:55,178][20347] Updated weights for policy 0, policy_version 303835 (0.0009) [2023-12-27 07:44:55,223][20381] Updated weights for policy 1, policy_version 303572 (0.0005) [2023-12-27 07:44:55,269][20381] Updated weights for policy 1, policy_version 303582 (0.0005) [2023-12-27 07:44:55,317][20381] Updated weights for policy 1, policy_version 303592 (0.0007) [2023-12-27 07:44:55,837][20347] Updated weights for policy 0, policy_version 303845 (0.0010) [2023-12-27 07:44:55,892][20347] Updated weights for policy 0, policy_version 303855 (0.0006) [2023-12-27 07:44:55,955][20347] Updated weights for policy 0, policy_version 303865 (0.0008) [2023-12-27 07:44:55,991][20381] Updated weights for policy 1, policy_version 303602 (0.0005) [2023-12-27 07:44:56,049][20381] Updated weights for policy 1, policy_version 303612 (0.0005) [2023-12-27 07:44:56,101][20381] Updated weights for policy 1, policy_version 303622 (0.0007) [2023-12-27 07:44:56,583][20347] Updated weights for policy 0, policy_version 303875 (0.0010) [2023-12-27 07:44:56,642][20347] Updated weights for policy 0, policy_version 303885 (0.0008) [2023-12-27 07:44:56,708][20347] Updated weights for policy 0, policy_version 303895 (0.0007) [2023-12-27 07:44:56,753][20381] Updated weights for policy 1, policy_version 303632 (0.0007) [2023-12-27 07:44:56,801][20381] Updated weights for policy 1, policy_version 303642 (0.0007) [2023-12-27 07:44:56,851][20381] Updated weights for policy 1, policy_version 303652 (0.0007) [2023-12-27 07:44:57,278][20347] Updated weights for policy 0, policy_version 303905 (0.0006) [2023-12-27 07:44:57,337][20347] Updated weights for policy 0, policy_version 303915 (0.0007) [2023-12-27 07:44:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 155566080. Throughput: 0: 10404.3, 1: 10478.5. Samples: 155558812. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:44:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '154.140')] [2023-12-27 07:44:57,394][20347] Updated weights for policy 0, policy_version 303925 (0.0007) [2023-12-27 07:44:57,453][20347] Updated weights for policy 0, policy_version 303935 (0.0007) [2023-12-27 07:44:57,585][20381] Updated weights for policy 1, policy_version 303662 (0.0008) [2023-12-27 07:44:57,630][20381] Updated weights for policy 1, policy_version 303672 (0.0007) [2023-12-27 07:44:57,675][20381] Updated weights for policy 1, policy_version 303682 (0.0006) [2023-12-27 07:44:58,059][20347] Updated weights for policy 0, policy_version 303945 (0.0010) [2023-12-27 07:44:58,103][20347] Updated weights for policy 0, policy_version 303955 (0.0010) [2023-12-27 07:44:58,159][20347] Updated weights for policy 0, policy_version 303965 (0.0011) [2023-12-27 07:44:58,328][20381] Updated weights for policy 1, policy_version 303692 (0.0009) [2023-12-27 07:44:58,390][20381] Updated weights for policy 1, policy_version 303702 (0.0009) [2023-12-27 07:44:58,453][20381] Updated weights for policy 1, policy_version 303712 (0.0011) [2023-12-27 07:44:58,922][20347] Updated weights for policy 0, policy_version 303975 (0.0007) [2023-12-27 07:44:58,981][20347] Updated weights for policy 0, policy_version 303985 (0.0005) [2023-12-27 07:44:59,043][20347] Updated weights for policy 0, policy_version 303995 (0.0007) [2023-12-27 07:44:59,175][20381] Updated weights for policy 1, policy_version 303722 (0.0007) [2023-12-27 07:44:59,235][20381] Updated weights for policy 1, policy_version 303732 (0.0007) [2023-12-27 07:44:59,299][20381] Updated weights for policy 1, policy_version 303742 (0.0007) [2023-12-27 07:44:59,360][20381] Updated weights for policy 1, policy_version 303752 (0.0007) [2023-12-27 07:44:59,685][20347] Updated weights for policy 0, policy_version 304005 (0.0008) [2023-12-27 07:44:59,738][20347] Updated weights for policy 0, policy_version 304015 (0.0010) [2023-12-27 07:44:59,786][20347] Updated weights for policy 0, policy_version 304025 (0.0010) [2023-12-27 07:45:00,073][20381] Updated weights for policy 1, policy_version 303762 (0.0007) [2023-12-27 07:45:00,131][20381] Updated weights for policy 1, policy_version 303772 (0.0007) [2023-12-27 07:45:00,185][20381] Updated weights for policy 1, policy_version 303782 (0.0007) [2023-12-27 07:45:00,491][20347] Updated weights for policy 0, policy_version 304035 (0.0010) [2023-12-27 07:45:00,536][20347] Updated weights for policy 0, policy_version 304045 (0.0010) [2023-12-27 07:45:00,581][20347] Updated weights for policy 0, policy_version 304055 (0.0010) [2023-12-27 07:45:00,785][20381] Updated weights for policy 1, policy_version 303792 (0.0007) [2023-12-27 07:45:00,830][20381] Updated weights for policy 1, policy_version 303802 (0.0008) [2023-12-27 07:45:00,881][20381] Updated weights for policy 1, policy_version 303812 (0.0007) [2023-12-27 07:45:01,281][20347] Updated weights for policy 0, policy_version 304065 (0.0010) [2023-12-27 07:45:01,343][20347] Updated weights for policy 0, policy_version 304075 (0.0009) [2023-12-27 07:45:01,407][20347] Updated weights for policy 0, policy_version 304085 (0.0010) [2023-12-27 07:45:01,462][20347] Updated weights for policy 0, policy_version 304095 (0.0010) [2023-12-27 07:45:01,652][20381] Updated weights for policy 1, policy_version 303822 (0.0009) [2023-12-27 07:45:01,709][20381] Updated weights for policy 1, policy_version 303832 (0.0010) [2023-12-27 07:45:01,774][20381] Updated weights for policy 1, policy_version 303842 (0.0006) [2023-12-27 07:45:02,149][20347] Updated weights for policy 0, policy_version 304105 (0.0010) [2023-12-27 07:45:02,202][20347] Updated weights for policy 0, policy_version 304115 (0.0010) [2023-12-27 07:45:02,260][20347] Updated weights for policy 0, policy_version 304125 (0.0010) [2023-12-27 07:45:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 155672576. Throughput: 0: 10449.0, 1: 10492.0. Samples: 155682160. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '145.450')] [2023-12-27 07:45:02,368][20381] Updated weights for policy 1, policy_version 303852 (0.0006) [2023-12-27 07:45:02,429][20381] Updated weights for policy 1, policy_version 303862 (0.0006) [2023-12-27 07:45:02,486][20381] Updated weights for policy 1, policy_version 303872 (0.0008) [2023-12-27 07:45:02,947][20347] Updated weights for policy 0, policy_version 304135 (0.0010) [2023-12-27 07:45:03,008][20347] Updated weights for policy 0, policy_version 304145 (0.0010) [2023-12-27 07:45:03,048][20381] Updated weights for policy 1, policy_version 303882 (0.0007) [2023-12-27 07:45:03,069][20347] Updated weights for policy 0, policy_version 304155 (0.0011) [2023-12-27 07:45:03,098][20381] Updated weights for policy 1, policy_version 303892 (0.0005) [2023-12-27 07:45:03,152][20381] Updated weights for policy 1, policy_version 303902 (0.0007) [2023-12-27 07:45:03,202][20381] Updated weights for policy 1, policy_version 303912 (0.0007) [2023-12-27 07:45:03,704][20347] Updated weights for policy 0, policy_version 304165 (0.0009) [2023-12-27 07:45:03,760][20347] Updated weights for policy 0, policy_version 304175 (0.0007) [2023-12-27 07:45:03,815][20347] Updated weights for policy 0, policy_version 304185 (0.0006) [2023-12-27 07:45:03,818][20381] Updated weights for policy 1, policy_version 303922 (0.0007) [2023-12-27 07:45:03,879][20381] Updated weights for policy 1, policy_version 303932 (0.0009) [2023-12-27 07:45:03,934][20381] Updated weights for policy 1, policy_version 303942 (0.0010) [2023-12-27 07:45:04,474][20347] Updated weights for policy 0, policy_version 304196 (0.0008) [2023-12-27 07:45:04,523][20347] Updated weights for policy 0, policy_version 304206 (0.0009) [2023-12-27 07:45:04,571][20347] Updated weights for policy 0, policy_version 304216 (0.0009) [2023-12-27 07:45:04,694][20381] Updated weights for policy 1, policy_version 303952 (0.0008) [2023-12-27 07:45:04,752][20381] Updated weights for policy 1, policy_version 303962 (0.0008) [2023-12-27 07:45:04,808][20381] Updated weights for policy 1, policy_version 303972 (0.0010) [2023-12-27 07:45:05,202][20347] Updated weights for policy 0, policy_version 304226 (0.0008) [2023-12-27 07:45:05,262][20347] Updated weights for policy 0, policy_version 304236 (0.0010) [2023-12-27 07:45:05,313][20347] Updated weights for policy 0, policy_version 304246 (0.0010) [2023-12-27 07:45:05,371][20347] Updated weights for policy 0, policy_version 304256 (0.0010) [2023-12-27 07:45:05,493][20381] Updated weights for policy 1, policy_version 303982 (0.0006) [2023-12-27 07:45:05,554][20381] Updated weights for policy 1, policy_version 303992 (0.0006) [2023-12-27 07:45:05,604][20381] Updated weights for policy 1, policy_version 304002 (0.0006) [2023-12-27 07:45:06,012][20347] Updated weights for policy 0, policy_version 304266 (0.0006) [2023-12-27 07:45:06,069][20347] Updated weights for policy 0, policy_version 304276 (0.0005) [2023-12-27 07:45:06,126][20347] Updated weights for policy 0, policy_version 304286 (0.0008) [2023-12-27 07:45:06,361][20381] Updated weights for policy 1, policy_version 304012 (0.0008) [2023-12-27 07:45:06,421][20381] Updated weights for policy 1, policy_version 304022 (0.0006) [2023-12-27 07:45:06,481][20381] Updated weights for policy 1, policy_version 304032 (0.0006) [2023-12-27 07:45:06,780][20347] Updated weights for policy 0, policy_version 304296 (0.0009) [2023-12-27 07:45:06,828][20347] Updated weights for policy 0, policy_version 304306 (0.0009) [2023-12-27 07:45:06,881][20347] Updated weights for policy 0, policy_version 304316 (0.0007) [2023-12-27 07:45:07,143][20381] Updated weights for policy 1, policy_version 304042 (0.0006) [2023-12-27 07:45:07,198][20381] Updated weights for policy 1, policy_version 304052 (0.0007) [2023-12-27 07:45:07,249][20381] Updated weights for policy 1, policy_version 304062 (0.0007) [2023-12-27 07:45:07,298][20381] Updated weights for policy 1, policy_version 304072 (0.0007) [2023-12-27 07:45:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.5, 300 sec: 20799.4). Total num frames: 155779072. Throughput: 0: 10421.4, 1: 10480.8. Samples: 155745132. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '154.670')] [2023-12-27 07:45:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000304320_77922304.pth... [2023-12-27 07:45:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000304072_77856768.pth... [2023-12-27 07:45:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000302856_77545472.pth [2023-12-27 07:45:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000303104_77611008.pth [2023-12-27 07:45:07,550][20347] Updated weights for policy 0, policy_version 304326 (0.0007) [2023-12-27 07:45:07,608][20347] Updated weights for policy 0, policy_version 304336 (0.0005) [2023-12-27 07:45:07,671][20347] Updated weights for policy 0, policy_version 304346 (0.0005) [2023-12-27 07:45:08,036][20381] Updated weights for policy 1, policy_version 304082 (0.0009) [2023-12-27 07:45:08,092][20381] Updated weights for policy 1, policy_version 304092 (0.0006) [2023-12-27 07:45:08,155][20381] Updated weights for policy 1, policy_version 304102 (0.0007) [2023-12-27 07:45:08,256][20347] Updated weights for policy 0, policy_version 304356 (0.0007) [2023-12-27 07:45:08,320][20347] Updated weights for policy 0, policy_version 304366 (0.0010) [2023-12-27 07:45:08,377][20347] Updated weights for policy 0, policy_version 304376 (0.0008) [2023-12-27 07:45:08,779][20381] Updated weights for policy 1, policy_version 304112 (0.0011) [2023-12-27 07:45:08,837][20381] Updated weights for policy 1, policy_version 304122 (0.0011) [2023-12-27 07:45:08,895][20381] Updated weights for policy 1, policy_version 304132 (0.0010) [2023-12-27 07:45:08,996][20347] Updated weights for policy 0, policy_version 304386 (0.0006) [2023-12-27 07:45:09,063][20347] Updated weights for policy 0, policy_version 304396 (0.0010) [2023-12-27 07:45:09,128][20347] Updated weights for policy 0, policy_version 304406 (0.0010) [2023-12-27 07:45:09,187][20347] Updated weights for policy 0, policy_version 304416 (0.0009) [2023-12-27 07:45:09,568][20381] Updated weights for policy 1, policy_version 304142 (0.0008) [2023-12-27 07:45:09,625][20381] Updated weights for policy 1, policy_version 304152 (0.0005) [2023-12-27 07:45:09,682][20381] Updated weights for policy 1, policy_version 304162 (0.0005) [2023-12-27 07:45:09,859][20347] Updated weights for policy 0, policy_version 304426 (0.0007) [2023-12-27 07:45:09,921][20347] Updated weights for policy 0, policy_version 304436 (0.0007) [2023-12-27 07:45:09,979][20347] Updated weights for policy 0, policy_version 304446 (0.0007) [2023-12-27 07:45:10,293][20381] Updated weights for policy 1, policy_version 304172 (0.0005) [2023-12-27 07:45:10,356][20381] Updated weights for policy 1, policy_version 304182 (0.0006) [2023-12-27 07:45:10,417][20381] Updated weights for policy 1, policy_version 304192 (0.0009) [2023-12-27 07:45:10,617][20347] Updated weights for policy 0, policy_version 304456 (0.0005) [2023-12-27 07:45:10,674][20347] Updated weights for policy 0, policy_version 304466 (0.0005) [2023-12-27 07:45:10,733][20347] Updated weights for policy 0, policy_version 304476 (0.0007) [2023-12-27 07:45:11,061][20381] Updated weights for policy 1, policy_version 304202 (0.0007) [2023-12-27 07:45:11,120][20381] Updated weights for policy 1, policy_version 304212 (0.0007) [2023-12-27 07:45:11,184][20381] Updated weights for policy 1, policy_version 304222 (0.0008) [2023-12-27 07:45:11,236][20381] Updated weights for policy 1, policy_version 304232 (0.0007) [2023-12-27 07:45:11,389][20347] Updated weights for policy 0, policy_version 304486 (0.0009) [2023-12-27 07:45:11,448][20347] Updated weights for policy 0, policy_version 304496 (0.0008) [2023-12-27 07:45:11,500][20347] Updated weights for policy 0, policy_version 304506 (0.0008) [2023-12-27 07:45:11,947][20381] Updated weights for policy 1, policy_version 304242 (0.0011) [2023-12-27 07:45:12,000][20381] Updated weights for policy 1, policy_version 304252 (0.0011) [2023-12-27 07:45:12,059][20381] Updated weights for policy 1, policy_version 304262 (0.0010) [2023-12-27 07:45:12,263][20347] Updated weights for policy 0, policy_version 304516 (0.0007) [2023-12-27 07:45:12,326][20347] Updated weights for policy 0, policy_version 304526 (0.0008) [2023-12-27 07:45:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 155877376. Throughput: 0: 10509.0, 1: 10443.0. Samples: 155871476. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '163.400')] [2023-12-27 07:45:12,389][20347] Updated weights for policy 0, policy_version 304536 (0.0008) [2023-12-27 07:45:12,680][20381] Updated weights for policy 1, policy_version 304272 (0.0007) [2023-12-27 07:45:12,738][20381] Updated weights for policy 1, policy_version 304282 (0.0006) [2023-12-27 07:45:12,804][20381] Updated weights for policy 1, policy_version 304292 (0.0009) [2023-12-27 07:45:13,104][20347] Updated weights for policy 0, policy_version 304546 (0.0008) [2023-12-27 07:45:13,163][20347] Updated weights for policy 0, policy_version 304556 (0.0008) [2023-12-27 07:45:13,220][20347] Updated weights for policy 0, policy_version 304566 (0.0009) [2023-12-27 07:45:13,276][20347] Updated weights for policy 0, policy_version 304576 (0.0007) [2023-12-27 07:45:13,395][20381] Updated weights for policy 1, policy_version 304302 (0.0005) [2023-12-27 07:45:13,457][20381] Updated weights for policy 1, policy_version 304312 (0.0006) [2023-12-27 07:45:13,508][20381] Updated weights for policy 1, policy_version 304322 (0.0007) [2023-12-27 07:45:13,994][20347] Updated weights for policy 0, policy_version 304586 (0.0006) [2023-12-27 07:45:14,041][20347] Updated weights for policy 0, policy_version 304596 (0.0006) [2023-12-27 07:45:14,091][20347] Updated weights for policy 0, policy_version 304606 (0.0005) [2023-12-27 07:45:14,141][20381] Updated weights for policy 1, policy_version 304332 (0.0010) [2023-12-27 07:45:14,210][20381] Updated weights for policy 1, policy_version 304342 (0.0011) [2023-12-27 07:45:14,263][20381] Updated weights for policy 1, policy_version 304352 (0.0011) [2023-12-27 07:45:14,770][20347] Updated weights for policy 0, policy_version 304616 (0.0008) [2023-12-27 07:45:14,835][20347] Updated weights for policy 0, policy_version 304626 (0.0008) [2023-12-27 07:45:14,893][20347] Updated weights for policy 0, policy_version 304636 (0.0008) [2023-12-27 07:45:14,948][20381] Updated weights for policy 1, policy_version 304362 (0.0011) [2023-12-27 07:45:15,016][20381] Updated weights for policy 1, policy_version 304372 (0.0011) [2023-12-27 07:45:15,077][20381] Updated weights for policy 1, policy_version 304382 (0.0011) [2023-12-27 07:45:15,139][20381] Updated weights for policy 1, policy_version 304392 (0.0011) [2023-12-27 07:45:15,620][20347] Updated weights for policy 0, policy_version 304646 (0.0007) [2023-12-27 07:45:15,682][20347] Updated weights for policy 0, policy_version 304656 (0.0007) [2023-12-27 07:45:15,745][20347] Updated weights for policy 0, policy_version 304666 (0.0007) [2023-12-27 07:45:15,820][20381] Updated weights for policy 1, policy_version 304402 (0.0009) [2023-12-27 07:45:15,869][20381] Updated weights for policy 1, policy_version 304412 (0.0010) [2023-12-27 07:45:15,919][20381] Updated weights for policy 1, policy_version 304422 (0.0009) [2023-12-27 07:45:16,415][20347] Updated weights for policy 0, policy_version 304676 (0.0007) [2023-12-27 07:45:16,477][20347] Updated weights for policy 0, policy_version 304686 (0.0006) [2023-12-27 07:45:16,536][20347] Updated weights for policy 0, policy_version 304696 (0.0006) [2023-12-27 07:45:16,627][20381] Updated weights for policy 1, policy_version 304432 (0.0009) [2023-12-27 07:45:16,679][20381] Updated weights for policy 1, policy_version 304442 (0.0010) [2023-12-27 07:45:16,731][20381] Updated weights for policy 1, policy_version 304452 (0.0010) [2023-12-27 07:45:17,061][20347] Updated weights for policy 0, policy_version 304706 (0.0005) [2023-12-27 07:45:17,117][20347] Updated weights for policy 0, policy_version 304716 (0.0006) [2023-12-27 07:45:17,176][20347] Updated weights for policy 0, policy_version 304726 (0.0011) [2023-12-27 07:45:17,236][20347] Updated weights for policy 0, policy_version 304736 (0.0010) [2023-12-27 07:45:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 155983872. Throughput: 0: 10385.0, 1: 10395.0. Samples: 155994204. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '158.510')] [2023-12-27 07:45:17,419][20381] Updated weights for policy 1, policy_version 304462 (0.0009) [2023-12-27 07:45:17,481][20381] Updated weights for policy 1, policy_version 304472 (0.0007) [2023-12-27 07:45:17,538][20381] Updated weights for policy 1, policy_version 304482 (0.0010) [2023-12-27 07:45:17,845][20347] Updated weights for policy 0, policy_version 304746 (0.0006) [2023-12-27 07:45:17,898][20347] Updated weights for policy 0, policy_version 304756 (0.0005) [2023-12-27 07:45:17,958][20347] Updated weights for policy 0, policy_version 304766 (0.0010) [2023-12-27 07:45:18,216][20381] Updated weights for policy 1, policy_version 304492 (0.0008) [2023-12-27 07:45:18,274][20381] Updated weights for policy 1, policy_version 304502 (0.0006) [2023-12-27 07:45:18,324][20381] Updated weights for policy 1, policy_version 304512 (0.0007) [2023-12-27 07:45:18,636][20347] Updated weights for policy 0, policy_version 304776 (0.0007) [2023-12-27 07:45:18,699][20347] Updated weights for policy 0, policy_version 304786 (0.0007) [2023-12-27 07:45:18,759][20347] Updated weights for policy 0, policy_version 304796 (0.0008) [2023-12-27 07:45:19,042][20381] Updated weights for policy 1, policy_version 304522 (0.0008) [2023-12-27 07:45:19,101][20381] Updated weights for policy 1, policy_version 304532 (0.0008) [2023-12-27 07:45:19,167][20381] Updated weights for policy 1, policy_version 304542 (0.0011) [2023-12-27 07:45:19,224][20381] Updated weights for policy 1, policy_version 304552 (0.0010) [2023-12-27 07:45:19,521][20347] Updated weights for policy 0, policy_version 304806 (0.0008) [2023-12-27 07:45:19,580][20347] Updated weights for policy 0, policy_version 304816 (0.0008) [2023-12-27 07:45:19,641][20347] Updated weights for policy 0, policy_version 304826 (0.0008) [2023-12-27 07:45:19,897][20381] Updated weights for policy 1, policy_version 304562 (0.0007) [2023-12-27 07:45:19,960][20381] Updated weights for policy 1, policy_version 304572 (0.0008) [2023-12-27 07:45:20,027][20381] Updated weights for policy 1, policy_version 304582 (0.0008) [2023-12-27 07:45:20,403][20347] Updated weights for policy 0, policy_version 304836 (0.0008) [2023-12-27 07:45:20,469][20347] Updated weights for policy 0, policy_version 304846 (0.0008) [2023-12-27 07:45:20,533][20347] Updated weights for policy 0, policy_version 304856 (0.0008) [2023-12-27 07:45:20,670][20381] Updated weights for policy 1, policy_version 304592 (0.0007) [2023-12-27 07:45:20,721][20381] Updated weights for policy 1, policy_version 304602 (0.0007) [2023-12-27 07:45:20,782][20381] Updated weights for policy 1, policy_version 304612 (0.0009) [2023-12-27 07:45:21,272][20347] Updated weights for policy 0, policy_version 304866 (0.0009) [2023-12-27 07:45:21,334][20347] Updated weights for policy 0, policy_version 304876 (0.0008) [2023-12-27 07:45:21,408][20347] Updated weights for policy 0, policy_version 304886 (0.0008) [2023-12-27 07:45:21,461][20347] Updated weights for policy 0, policy_version 304896 (0.0008) [2023-12-27 07:45:21,533][20381] Updated weights for policy 1, policy_version 304622 (0.0010) [2023-12-27 07:45:21,587][20381] Updated weights for policy 1, policy_version 304632 (0.0011) [2023-12-27 07:45:21,641][20381] Updated weights for policy 1, policy_version 304642 (0.0009) [2023-12-27 07:45:22,212][20347] Updated weights for policy 0, policy_version 304906 (0.0008) [2023-12-27 07:45:22,275][20347] Updated weights for policy 0, policy_version 304916 (0.0009) [2023-12-27 07:45:22,318][20381] Updated weights for policy 1, policy_version 304652 (0.0010) [2023-12-27 07:45:22,338][20347] Updated weights for policy 0, policy_version 304926 (0.0008) [2023-12-27 07:45:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 156082176. Throughput: 0: 10329.3, 1: 10326.6. Samples: 156053848. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '163.410')] [2023-12-27 07:45:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000304928_78077952.pth... [2023-12-27 07:45:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000303712_77766656.pth [2023-12-27 07:45:22,381][20381] Updated weights for policy 1, policy_version 304662 (0.0008) [2023-12-27 07:45:22,445][20381] Updated weights for policy 1, policy_version 304672 (0.0006) [2023-12-27 07:45:22,491][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000304680_78012416.pth... [2023-12-27 07:45:22,496][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000303464_77701120.pth [2023-12-27 07:45:23,059][20347] Updated weights for policy 0, policy_version 304936 (0.0007) [2023-12-27 07:45:23,119][20347] Updated weights for policy 0, policy_version 304946 (0.0008) [2023-12-27 07:45:23,154][20381] Updated weights for policy 1, policy_version 304682 (0.0008) [2023-12-27 07:45:23,176][20347] Updated weights for policy 0, policy_version 304956 (0.0006) [2023-12-27 07:45:23,215][20381] Updated weights for policy 1, policy_version 304692 (0.0009) [2023-12-27 07:45:23,267][20381] Updated weights for policy 1, policy_version 304702 (0.0007) [2023-12-27 07:45:23,328][20381] Updated weights for policy 1, policy_version 304712 (0.0009) [2023-12-27 07:45:23,884][20347] Updated weights for policy 0, policy_version 304966 (0.0008) [2023-12-27 07:45:23,941][20347] Updated weights for policy 0, policy_version 304976 (0.0008) [2023-12-27 07:45:23,999][20347] Updated weights for policy 0, policy_version 304986 (0.0006) [2023-12-27 07:45:24,004][20381] Updated weights for policy 1, policy_version 304722 (0.0011) [2023-12-27 07:45:24,061][20381] Updated weights for policy 1, policy_version 304732 (0.0010) [2023-12-27 07:45:24,120][20381] Updated weights for policy 1, policy_version 304742 (0.0011) [2023-12-27 07:45:24,705][20347] Updated weights for policy 0, policy_version 304996 (0.0006) [2023-12-27 07:45:24,760][20347] Updated weights for policy 0, policy_version 305006 (0.0007) [2023-12-27 07:45:24,802][20381] Updated weights for policy 1, policy_version 304752 (0.0006) [2023-12-27 07:45:24,815][20347] Updated weights for policy 0, policy_version 305016 (0.0006) [2023-12-27 07:45:24,851][20381] Updated weights for policy 1, policy_version 304762 (0.0005) [2023-12-27 07:45:24,913][20381] Updated weights for policy 1, policy_version 304772 (0.0007) [2023-12-27 07:45:25,519][20347] Updated weights for policy 0, policy_version 305026 (0.0007) [2023-12-27 07:45:25,560][20381] Updated weights for policy 1, policy_version 304782 (0.0007) [2023-12-27 07:45:25,570][20347] Updated weights for policy 0, policy_version 305036 (0.0005) [2023-12-27 07:45:25,613][20381] Updated weights for policy 1, policy_version 304792 (0.0005) [2023-12-27 07:45:25,625][20347] Updated weights for policy 0, policy_version 305046 (0.0006) [2023-12-27 07:45:25,665][20381] Updated weights for policy 1, policy_version 304802 (0.0006) [2023-12-27 07:45:25,675][20347] Updated weights for policy 0, policy_version 305056 (0.0006) [2023-12-27 07:45:26,307][20381] Updated weights for policy 1, policy_version 304812 (0.0006) [2023-12-27 07:45:26,353][20381] Updated weights for policy 1, policy_version 304822 (0.0007) [2023-12-27 07:45:26,397][20347] Updated weights for policy 0, policy_version 305066 (0.0007) [2023-12-27 07:45:26,407][20381] Updated weights for policy 1, policy_version 304832 (0.0008) [2023-12-27 07:45:26,453][20347] Updated weights for policy 0, policy_version 305076 (0.0006) [2023-12-27 07:45:26,513][20347] Updated weights for policy 0, policy_version 305086 (0.0008) [2023-12-27 07:45:27,095][20381] Updated weights for policy 1, policy_version 304842 (0.0006) [2023-12-27 07:45:27,141][20347] Updated weights for policy 0, policy_version 305096 (0.0008) [2023-12-27 07:45:27,143][20381] Updated weights for policy 1, policy_version 304852 (0.0007) [2023-12-27 07:45:27,195][20347] Updated weights for policy 0, policy_version 305106 (0.0006) [2023-12-27 07:45:27,198][20381] Updated weights for policy 1, policy_version 304862 (0.0006) [2023-12-27 07:45:27,244][20381] Updated weights for policy 1, policy_version 304872 (0.0005) [2023-12-27 07:45:27,249][20347] Updated weights for policy 0, policy_version 305116 (0.0007) [2023-12-27 07:45:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 156188672. Throughput: 0: 10229.9, 1: 10301.4. Samples: 156172148. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '200.340')] [2023-12-27 07:45:27,880][20381] Updated weights for policy 1, policy_version 304882 (0.0008) [2023-12-27 07:45:27,932][20381] Updated weights for policy 1, policy_version 304892 (0.0007) [2023-12-27 07:45:27,950][20347] Updated weights for policy 0, policy_version 305126 (0.0007) [2023-12-27 07:45:27,989][20381] Updated weights for policy 1, policy_version 304902 (0.0006) [2023-12-27 07:45:28,001][20347] Updated weights for policy 0, policy_version 305136 (0.0005) [2023-12-27 07:45:28,049][20347] Updated weights for policy 0, policy_version 305146 (0.0006) [2023-12-27 07:45:28,599][20381] Updated weights for policy 1, policy_version 304912 (0.0006) [2023-12-27 07:45:28,658][20381] Updated weights for policy 1, policy_version 304922 (0.0007) [2023-12-27 07:45:28,684][20347] Updated weights for policy 0, policy_version 305156 (0.0007) [2023-12-27 07:45:28,708][20381] Updated weights for policy 1, policy_version 304932 (0.0008) [2023-12-27 07:45:28,735][20347] Updated weights for policy 0, policy_version 305166 (0.0005) [2023-12-27 07:45:28,787][20347] Updated weights for policy 0, policy_version 305176 (0.0005) [2023-12-27 07:45:29,426][20381] Updated weights for policy 1, policy_version 304942 (0.0007) [2023-12-27 07:45:29,436][20347] Updated weights for policy 0, policy_version 305186 (0.0005) [2023-12-27 07:45:29,488][20381] Updated weights for policy 1, policy_version 304952 (0.0007) [2023-12-27 07:45:29,495][20347] Updated weights for policy 0, policy_version 305196 (0.0007) [2023-12-27 07:45:29,542][20381] Updated weights for policy 1, policy_version 304962 (0.0005) [2023-12-27 07:45:29,552][20347] Updated weights for policy 0, policy_version 305206 (0.0006) [2023-12-27 07:45:29,617][20347] Updated weights for policy 0, policy_version 305216 (0.0006) [2023-12-27 07:45:30,113][20381] Updated weights for policy 1, policy_version 304972 (0.0006) [2023-12-27 07:45:30,170][20381] Updated weights for policy 1, policy_version 304982 (0.0005) [2023-12-27 07:45:30,225][20381] Updated weights for policy 1, policy_version 304992 (0.0005) [2023-12-27 07:45:30,327][20347] Updated weights for policy 0, policy_version 305226 (0.0007) [2023-12-27 07:45:30,381][20347] Updated weights for policy 0, policy_version 305236 (0.0006) [2023-12-27 07:45:30,439][20347] Updated weights for policy 0, policy_version 305246 (0.0006) [2023-12-27 07:45:30,895][20381] Updated weights for policy 1, policy_version 305002 (0.0006) [2023-12-27 07:45:30,947][20381] Updated weights for policy 1, policy_version 305012 (0.0005) [2023-12-27 07:45:30,995][20381] Updated weights for policy 1, policy_version 305022 (0.0005) [2023-12-27 07:45:31,029][20347] Updated weights for policy 0, policy_version 305256 (0.0007) [2023-12-27 07:45:31,055][20381] Updated weights for policy 1, policy_version 305032 (0.0006) [2023-12-27 07:45:31,093][20347] Updated weights for policy 0, policy_version 305266 (0.0007) [2023-12-27 07:45:31,160][20347] Updated weights for policy 0, policy_version 305276 (0.0007) [2023-12-27 07:45:31,720][20381] Updated weights for policy 1, policy_version 305042 (0.0008) [2023-12-27 07:45:31,778][20381] Updated weights for policy 1, policy_version 305052 (0.0007) [2023-12-27 07:45:31,837][20381] Updated weights for policy 1, policy_version 305062 (0.0007) [2023-12-27 07:45:31,850][20347] Updated weights for policy 0, policy_version 305286 (0.0006) [2023-12-27 07:45:31,903][20347] Updated weights for policy 0, policy_version 305296 (0.0005) [2023-12-27 07:45:31,967][20347] Updated weights for policy 0, policy_version 305306 (0.0005) [2023-12-27 07:45:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 156286976. Throughput: 0: 10311.1, 1: 10316.9. Samples: 156298980. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '217.060')] [2023-12-27 07:45:32,506][20381] Updated weights for policy 1, policy_version 305072 (0.0008) [2023-12-27 07:45:32,564][20381] Updated weights for policy 1, policy_version 305082 (0.0007) [2023-12-27 07:45:32,605][20347] Updated weights for policy 0, policy_version 305316 (0.0008) [2023-12-27 07:45:32,618][20381] Updated weights for policy 1, policy_version 305092 (0.0005) [2023-12-27 07:45:32,663][20347] Updated weights for policy 0, policy_version 305326 (0.0007) [2023-12-27 07:45:32,721][20347] Updated weights for policy 0, policy_version 305336 (0.0007) [2023-12-27 07:45:33,274][20381] Updated weights for policy 1, policy_version 305102 (0.0007) [2023-12-27 07:45:33,322][20381] Updated weights for policy 1, policy_version 305112 (0.0008) [2023-12-27 07:45:33,379][20381] Updated weights for policy 1, policy_version 305122 (0.0007) [2023-12-27 07:45:33,403][20347] Updated weights for policy 0, policy_version 305346 (0.0008) [2023-12-27 07:45:33,453][20347] Updated weights for policy 0, policy_version 305356 (0.0008) [2023-12-27 07:45:33,508][20347] Updated weights for policy 0, policy_version 305366 (0.0008) [2023-12-27 07:45:33,562][20347] Updated weights for policy 0, policy_version 305376 (0.0007) [2023-12-27 07:45:34,097][20381] Updated weights for policy 1, policy_version 305132 (0.0007) [2023-12-27 07:45:34,150][20381] Updated weights for policy 1, policy_version 305142 (0.0007) [2023-12-27 07:45:34,155][20347] Updated weights for policy 0, policy_version 305386 (0.0006) [2023-12-27 07:45:34,211][20381] Updated weights for policy 1, policy_version 305152 (0.0007) [2023-12-27 07:45:34,213][20347] Updated weights for policy 0, policy_version 305396 (0.0006) [2023-12-27 07:45:34,271][20347] Updated weights for policy 0, policy_version 305406 (0.0006) [2023-12-27 07:45:34,836][20381] Updated weights for policy 1, policy_version 305162 (0.0006) [2023-12-27 07:45:34,893][20381] Updated weights for policy 1, policy_version 305172 (0.0007) [2023-12-27 07:45:34,950][20381] Updated weights for policy 1, policy_version 305182 (0.0008) [2023-12-27 07:45:35,007][20347] Updated weights for policy 0, policy_version 305416 (0.0007) [2023-12-27 07:45:35,012][20381] Updated weights for policy 1, policy_version 305192 (0.0006) [2023-12-27 07:45:35,063][20347] Updated weights for policy 0, policy_version 305426 (0.0007) [2023-12-27 07:45:35,121][20347] Updated weights for policy 0, policy_version 305436 (0.0008) [2023-12-27 07:45:35,688][20381] Updated weights for policy 1, policy_version 305202 (0.0007) [2023-12-27 07:45:35,740][20381] Updated weights for policy 1, policy_version 305212 (0.0009) [2023-12-27 07:45:35,789][20381] Updated weights for policy 1, policy_version 305222 (0.0010) [2023-12-27 07:45:35,811][20347] Updated weights for policy 0, policy_version 305446 (0.0009) [2023-12-27 07:45:35,864][20347] Updated weights for policy 0, policy_version 305456 (0.0011) [2023-12-27 07:45:35,932][20347] Updated weights for policy 0, policy_version 305468 (0.0008) [2023-12-27 07:45:36,498][20381] Updated weights for policy 1, policy_version 305232 (0.0010) [2023-12-27 07:45:36,548][20347] Updated weights for policy 0, policy_version 305478 (0.0007) [2023-12-27 07:45:36,550][20381] Updated weights for policy 1, policy_version 305242 (0.0010) [2023-12-27 07:45:36,604][20381] Updated weights for policy 1, policy_version 305252 (0.0010) [2023-12-27 07:45:36,610][20347] Updated weights for policy 0, policy_version 305488 (0.0005) [2023-12-27 07:45:36,671][20347] Updated weights for policy 0, policy_version 305498 (0.0006) [2023-12-27 07:45:37,222][20381] Updated weights for policy 1, policy_version 305262 (0.0008) [2023-12-27 07:45:37,283][20381] Updated weights for policy 1, policy_version 305272 (0.0011) [2023-12-27 07:45:37,341][20347] Updated weights for policy 0, policy_version 305508 (0.0006) [2023-12-27 07:45:37,343][20381] Updated weights for policy 1, policy_version 305282 (0.0011) [2023-12-27 07:45:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 156385280. Throughput: 0: 10325.0, 1: 10297.7. Samples: 156361128. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '216.360')] [2023-12-27 07:45:37,381][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000305288_78168064.pth... [2023-12-27 07:45:37,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000304072_77856768.pth [2023-12-27 07:45:37,398][20347] Updated weights for policy 0, policy_version 305518 (0.0006) [2023-12-27 07:45:37,455][20347] Updated weights for policy 0, policy_version 305528 (0.0007) [2023-12-27 07:45:37,498][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000305536_78233600.pth... [2023-12-27 07:45:37,502][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000304320_77922304.pth [2023-12-27 07:45:37,979][20381] Updated weights for policy 1, policy_version 305292 (0.0011) [2023-12-27 07:45:38,044][20381] Updated weights for policy 1, policy_version 305302 (0.0010) [2023-12-27 07:45:38,096][20381] Updated weights for policy 1, policy_version 305312 (0.0010) [2023-12-27 07:45:38,124][20347] Updated weights for policy 0, policy_version 305538 (0.0007) [2023-12-27 07:45:38,168][20347] Updated weights for policy 0, policy_version 305548 (0.0006) [2023-12-27 07:45:38,219][20347] Updated weights for policy 0, policy_version 305558 (0.0005) [2023-12-27 07:45:38,285][20347] Updated weights for policy 0, policy_version 305568 (0.0006) [2023-12-27 07:45:38,763][20381] Updated weights for policy 1, policy_version 305322 (0.0011) [2023-12-27 07:45:38,818][20381] Updated weights for policy 1, policy_version 305332 (0.0011) [2023-12-27 07:45:38,877][20381] Updated weights for policy 1, policy_version 305342 (0.0011) [2023-12-27 07:45:38,933][20381] Updated weights for policy 1, policy_version 305352 (0.0011) [2023-12-27 07:45:38,942][20347] Updated weights for policy 0, policy_version 305578 (0.0005) [2023-12-27 07:45:38,996][20347] Updated weights for policy 0, policy_version 305588 (0.0007) [2023-12-27 07:45:39,043][20347] Updated weights for policy 0, policy_version 305598 (0.0007) [2023-12-27 07:45:39,631][20381] Updated weights for policy 1, policy_version 305362 (0.0011) [2023-12-27 07:45:39,692][20381] Updated weights for policy 1, policy_version 305372 (0.0011) [2023-12-27 07:45:39,756][20381] Updated weights for policy 1, policy_version 305382 (0.0011) [2023-12-27 07:45:39,780][20347] Updated weights for policy 0, policy_version 305608 (0.0008) [2023-12-27 07:45:39,837][20347] Updated weights for policy 0, policy_version 305618 (0.0007) [2023-12-27 07:45:39,890][20347] Updated weights for policy 0, policy_version 305628 (0.0007) [2023-12-27 07:45:40,447][20381] Updated weights for policy 1, policy_version 305392 (0.0008) [2023-12-27 07:45:40,506][20381] Updated weights for policy 1, policy_version 305402 (0.0007) [2023-12-27 07:45:40,560][20381] Updated weights for policy 1, policy_version 305412 (0.0007) [2023-12-27 07:45:40,574][20347] Updated weights for policy 0, policy_version 305638 (0.0006) [2023-12-27 07:45:40,624][20347] Updated weights for policy 0, policy_version 305648 (0.0008) [2023-12-27 07:45:40,672][20347] Updated weights for policy 0, policy_version 305658 (0.0005) [2023-12-27 07:45:41,190][20381] Updated weights for policy 1, policy_version 305422 (0.0008) [2023-12-27 07:45:41,250][20381] Updated weights for policy 1, policy_version 305432 (0.0007) [2023-12-27 07:45:41,305][20381] Updated weights for policy 1, policy_version 305442 (0.0007) [2023-12-27 07:45:41,395][20347] Updated weights for policy 0, policy_version 305668 (0.0008) [2023-12-27 07:45:41,457][20347] Updated weights for policy 0, policy_version 305678 (0.0008) [2023-12-27 07:45:41,520][20347] Updated weights for policy 0, policy_version 305688 (0.0008) [2023-12-27 07:45:42,048][20381] Updated weights for policy 1, policy_version 305452 (0.0008) [2023-12-27 07:45:42,102][20381] Updated weights for policy 1, policy_version 305462 (0.0010) [2023-12-27 07:45:42,152][20381] Updated weights for policy 1, policy_version 305472 (0.0010) [2023-12-27 07:45:42,195][20347] Updated weights for policy 0, policy_version 305698 (0.0006) [2023-12-27 07:45:42,256][20347] Updated weights for policy 0, policy_version 305708 (0.0006) [2023-12-27 07:45:42,315][20347] Updated weights for policy 0, policy_version 305718 (0.0008) [2023-12-27 07:45:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 156491776. Throughput: 0: 10250.4, 1: 10331.4. Samples: 156484992. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:42,363][19170] Avg episode reward: [(0, '990.440'), (1, '224.740')] [2023-12-27 07:45:42,381][20347] Updated weights for policy 0, policy_version 305728 (0.0008) [2023-12-27 07:45:42,883][20381] Updated weights for policy 1, policy_version 305482 (0.0008) [2023-12-27 07:45:42,947][20381] Updated weights for policy 1, policy_version 305492 (0.0008) [2023-12-27 07:45:42,998][20347] Updated weights for policy 0, policy_version 305738 (0.0006) [2023-12-27 07:45:43,000][20381] Updated weights for policy 1, policy_version 305502 (0.0006) [2023-12-27 07:45:43,049][20347] Updated weights for policy 0, policy_version 305748 (0.0006) [2023-12-27 07:45:43,052][20381] Updated weights for policy 1, policy_version 305512 (0.0005) [2023-12-27 07:45:43,098][20347] Updated weights for policy 0, policy_version 305758 (0.0006) [2023-12-27 07:45:43,683][20381] Updated weights for policy 1, policy_version 305522 (0.0008) [2023-12-27 07:45:43,743][20381] Updated weights for policy 1, policy_version 305532 (0.0008) [2023-12-27 07:45:43,776][20347] Updated weights for policy 0, policy_version 305768 (0.0006) [2023-12-27 07:45:43,800][20381] Updated weights for policy 1, policy_version 305542 (0.0006) [2023-12-27 07:45:43,828][20347] Updated weights for policy 0, policy_version 305778 (0.0006) [2023-12-27 07:45:43,880][20347] Updated weights for policy 0, policy_version 305788 (0.0007) [2023-12-27 07:45:44,482][20381] Updated weights for policy 1, policy_version 305552 (0.0005) [2023-12-27 07:45:44,538][20381] Updated weights for policy 1, policy_version 305562 (0.0005) [2023-12-27 07:45:44,597][20381] Updated weights for policy 1, policy_version 305572 (0.0005) [2023-12-27 07:45:44,602][20347] Updated weights for policy 0, policy_version 305798 (0.0007) [2023-12-27 07:45:44,653][20347] Updated weights for policy 0, policy_version 305808 (0.0007) [2023-12-27 07:45:44,704][20347] Updated weights for policy 0, policy_version 305818 (0.0008) [2023-12-27 07:45:45,222][20381] Updated weights for policy 1, policy_version 305582 (0.0005) [2023-12-27 07:45:45,281][20381] Updated weights for policy 1, policy_version 305592 (0.0005) [2023-12-27 07:45:45,335][20381] Updated weights for policy 1, policy_version 305602 (0.0005) [2023-12-27 07:45:45,471][20347] Updated weights for policy 0, policy_version 305828 (0.0008) [2023-12-27 07:45:45,520][20347] Updated weights for policy 0, policy_version 305838 (0.0010) [2023-12-27 07:45:45,575][20347] Updated weights for policy 0, policy_version 305848 (0.0010) [2023-12-27 07:45:45,954][20381] Updated weights for policy 1, policy_version 305612 (0.0008) [2023-12-27 07:45:46,011][20381] Updated weights for policy 1, policy_version 305622 (0.0011) [2023-12-27 07:45:46,071][20381] Updated weights for policy 1, policy_version 305632 (0.0010) [2023-12-27 07:45:46,275][20347] Updated weights for policy 0, policy_version 305858 (0.0010) [2023-12-27 07:45:46,337][20347] Updated weights for policy 0, policy_version 305868 (0.0007) [2023-12-27 07:45:46,395][20347] Updated weights for policy 0, policy_version 305878 (0.0006) [2023-12-27 07:45:46,447][20347] Updated weights for policy 0, policy_version 305888 (0.0007) [2023-12-27 07:45:46,633][20381] Updated weights for policy 1, policy_version 305642 (0.0007) [2023-12-27 07:45:46,694][20381] Updated weights for policy 1, policy_version 305652 (0.0007) [2023-12-27 07:45:46,753][20381] Updated weights for policy 1, policy_version 305662 (0.0007) [2023-12-27 07:45:46,810][20381] Updated weights for policy 1, policy_version 305672 (0.0007) [2023-12-27 07:45:47,180][20347] Updated weights for policy 0, policy_version 305898 (0.0009) [2023-12-27 07:45:47,242][20347] Updated weights for policy 0, policy_version 305908 (0.0007) [2023-12-27 07:45:47,287][20347] Updated weights for policy 0, policy_version 305918 (0.0009) [2023-12-27 07:45:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 156598272. Throughput: 0: 10199.8, 1: 10376.1. Samples: 156608080. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:47,364][19170] Avg episode reward: [(0, '990.440'), (1, '245.860')] [2023-12-27 07:45:47,402][20381] Updated weights for policy 1, policy_version 305682 (0.0009) [2023-12-27 07:45:47,463][20381] Updated weights for policy 1, policy_version 305692 (0.0008) [2023-12-27 07:45:47,526][20381] Updated weights for policy 1, policy_version 305702 (0.0008) [2023-12-27 07:45:47,925][20347] Updated weights for policy 0, policy_version 305928 (0.0008) [2023-12-27 07:45:47,974][20347] Updated weights for policy 0, policy_version 305938 (0.0008) [2023-12-27 07:45:48,037][20347] Updated weights for policy 0, policy_version 305948 (0.0007) [2023-12-27 07:45:48,127][20381] Updated weights for policy 1, policy_version 305712 (0.0008) [2023-12-27 07:45:48,187][20381] Updated weights for policy 1, policy_version 305722 (0.0007) [2023-12-27 07:45:48,235][20381] Updated weights for policy 1, policy_version 305732 (0.0005) [2023-12-27 07:45:48,762][20347] Updated weights for policy 0, policy_version 305958 (0.0008) [2023-12-27 07:45:48,810][20347] Updated weights for policy 0, policy_version 305968 (0.0007) [2023-12-27 07:45:48,874][20347] Updated weights for policy 0, policy_version 305978 (0.0007) [2023-12-27 07:45:48,915][20381] Updated weights for policy 1, policy_version 305742 (0.0008) [2023-12-27 07:45:48,975][20381] Updated weights for policy 1, policy_version 305752 (0.0011) [2023-12-27 07:45:49,024][20381] Updated weights for policy 1, policy_version 305762 (0.0011) [2023-12-27 07:45:49,593][20347] Updated weights for policy 0, policy_version 305988 (0.0006) [2023-12-27 07:45:49,654][20347] Updated weights for policy 0, policy_version 305998 (0.0007) [2023-12-27 07:45:49,715][20347] Updated weights for policy 0, policy_version 306008 (0.0007) [2023-12-27 07:45:49,731][20381] Updated weights for policy 1, policy_version 305772 (0.0011) [2023-12-27 07:45:49,787][20381] Updated weights for policy 1, policy_version 305782 (0.0010) [2023-12-27 07:45:49,845][20381] Updated weights for policy 1, policy_version 305792 (0.0008) [2023-12-27 07:45:50,443][20347] Updated weights for policy 0, policy_version 306018 (0.0006) [2023-12-27 07:45:50,493][20347] Updated weights for policy 0, policy_version 306028 (0.0007) [2023-12-27 07:45:50,495][20381] Updated weights for policy 1, policy_version 305802 (0.0007) [2023-12-27 07:45:50,541][20347] Updated weights for policy 0, policy_version 306038 (0.0005) [2023-12-27 07:45:50,543][20381] Updated weights for policy 1, policy_version 305812 (0.0006) [2023-12-27 07:45:50,600][20347] Updated weights for policy 0, policy_version 306048 (0.0006) [2023-12-27 07:45:50,603][20381] Updated weights for policy 1, policy_version 305822 (0.0006) [2023-12-27 07:45:50,661][20381] Updated weights for policy 1, policy_version 305832 (0.0007) [2023-12-27 07:45:51,350][20347] Updated weights for policy 0, policy_version 306058 (0.0009) [2023-12-27 07:45:51,385][20381] Updated weights for policy 1, policy_version 305842 (0.0008) [2023-12-27 07:45:51,403][20347] Updated weights for policy 0, policy_version 306068 (0.0007) [2023-12-27 07:45:51,432][20381] Updated weights for policy 1, policy_version 305852 (0.0005) [2023-12-27 07:45:51,454][20347] Updated weights for policy 0, policy_version 306078 (0.0005) [2023-12-27 07:45:51,482][20381] Updated weights for policy 1, policy_version 305862 (0.0006) [2023-12-27 07:45:52,179][20347] Updated weights for policy 0, policy_version 306088 (0.0007) [2023-12-27 07:45:52,200][20381] Updated weights for policy 1, policy_version 305872 (0.0007) [2023-12-27 07:45:52,234][20347] Updated weights for policy 0, policy_version 306098 (0.0007) [2023-12-27 07:45:52,252][20381] Updated weights for policy 1, policy_version 305882 (0.0006) [2023-12-27 07:45:52,297][20347] Updated weights for policy 0, policy_version 306108 (0.0007) [2023-12-27 07:45:52,303][20381] Updated weights for policy 1, policy_version 305892 (0.0006) [2023-12-27 07:45:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 156704768. Throughput: 0: 10158.0, 1: 10369.6. Samples: 156668872. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '251.740')] [2023-12-27 07:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000305896_78323712.pth... [2023-12-27 07:45:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000306112_78381056.pth... [2023-12-27 07:45:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000304680_78012416.pth [2023-12-27 07:45:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000304928_78077952.pth [2023-12-27 07:45:52,943][20347] Updated weights for policy 0, policy_version 306118 (0.0009) [2023-12-27 07:45:53,010][20347] Updated weights for policy 0, policy_version 306128 (0.0011) [2023-12-27 07:45:53,053][20381] Updated weights for policy 1, policy_version 305902 (0.0005) [2023-12-27 07:45:53,075][20347] Updated weights for policy 0, policy_version 306138 (0.0010) [2023-12-27 07:45:53,105][20381] Updated weights for policy 1, policy_version 305912 (0.0005) [2023-12-27 07:45:53,171][20381] Updated weights for policy 1, policy_version 305922 (0.0006) [2023-12-27 07:45:53,640][20347] Updated weights for policy 0, policy_version 306148 (0.0008) [2023-12-27 07:45:53,700][20347] Updated weights for policy 0, policy_version 306158 (0.0009) [2023-12-27 07:45:53,726][20381] Updated weights for policy 1, policy_version 305932 (0.0007) [2023-12-27 07:45:53,763][20347] Updated weights for policy 0, policy_version 306168 (0.0010) [2023-12-27 07:45:53,771][20381] Updated weights for policy 1, policy_version 305942 (0.0007) [2023-12-27 07:45:53,824][20381] Updated weights for policy 1, policy_version 305952 (0.0007) [2023-12-27 07:45:54,416][20347] Updated weights for policy 0, policy_version 306178 (0.0010) [2023-12-27 07:45:54,450][20381] Updated weights for policy 1, policy_version 305962 (0.0006) [2023-12-27 07:45:54,477][20347] Updated weights for policy 0, policy_version 306188 (0.0011) [2023-12-27 07:45:54,506][20381] Updated weights for policy 1, policy_version 305972 (0.0010) [2023-12-27 07:45:54,532][20347] Updated weights for policy 0, policy_version 306198 (0.0010) [2023-12-27 07:45:54,562][20381] Updated weights for policy 1, policy_version 305982 (0.0011) [2023-12-27 07:45:54,584][20347] Updated weights for policy 0, policy_version 306208 (0.0010) [2023-12-27 07:45:54,615][20381] Updated weights for policy 1, policy_version 305992 (0.0006) [2023-12-27 07:45:55,210][20347] Updated weights for policy 0, policy_version 306218 (0.0011) [2023-12-27 07:45:55,248][20381] Updated weights for policy 1, policy_version 306002 (0.0006) [2023-12-27 07:45:55,264][20347] Updated weights for policy 0, policy_version 306228 (0.0010) [2023-12-27 07:45:55,304][20381] Updated weights for policy 1, policy_version 306012 (0.0006) [2023-12-27 07:45:55,322][20347] Updated weights for policy 0, policy_version 306238 (0.0010) [2023-12-27 07:45:55,361][20381] Updated weights for policy 1, policy_version 306022 (0.0009) [2023-12-27 07:45:55,941][20381] Updated weights for policy 1, policy_version 306032 (0.0006) [2023-12-27 07:45:55,976][20347] Updated weights for policy 0, policy_version 306248 (0.0011) [2023-12-27 07:45:56,001][20381] Updated weights for policy 1, policy_version 306042 (0.0009) [2023-12-27 07:45:56,031][20347] Updated weights for policy 0, policy_version 306258 (0.0010) [2023-12-27 07:45:56,050][20381] Updated weights for policy 1, policy_version 306052 (0.0010) [2023-12-27 07:45:56,083][20347] Updated weights for policy 0, policy_version 306268 (0.0010) [2023-12-27 07:45:56,641][20381] Updated weights for policy 1, policy_version 306062 (0.0007) [2023-12-27 07:45:56,685][20381] Updated weights for policy 1, policy_version 306072 (0.0005) [2023-12-27 07:45:56,732][20381] Updated weights for policy 1, policy_version 306082 (0.0006) [2023-12-27 07:45:56,734][20347] Updated weights for policy 0, policy_version 306278 (0.0008) [2023-12-27 07:45:56,781][20347] Updated weights for policy 0, policy_version 306288 (0.0005) [2023-12-27 07:45:56,832][20347] Updated weights for policy 0, policy_version 306298 (0.0007) [2023-12-27 07:45:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 156803072. Throughput: 0: 10098.8, 1: 10417.8. Samples: 156794720. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:45:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '244.110')] [2023-12-27 07:45:57,394][20381] Updated weights for policy 1, policy_version 306092 (0.0008) [2023-12-27 07:45:57,443][20381] Updated weights for policy 1, policy_version 306102 (0.0010) [2023-12-27 07:45:57,447][20347] Updated weights for policy 0, policy_version 306308 (0.0008) [2023-12-27 07:45:57,498][20381] Updated weights for policy 1, policy_version 306112 (0.0011) [2023-12-27 07:45:57,505][20347] Updated weights for policy 0, policy_version 306318 (0.0005) [2023-12-27 07:45:57,559][20347] Updated weights for policy 0, policy_version 306328 (0.0008) [2023-12-27 07:45:58,143][20347] Updated weights for policy 0, policy_version 306338 (0.0010) [2023-12-27 07:45:58,145][20381] Updated weights for policy 1, policy_version 306122 (0.0010) [2023-12-27 07:45:58,204][20347] Updated weights for policy 0, policy_version 306348 (0.0009) [2023-12-27 07:45:58,214][20381] Updated weights for policy 1, policy_version 306132 (0.0008) [2023-12-27 07:45:58,261][20347] Updated weights for policy 0, policy_version 306358 (0.0008) [2023-12-27 07:45:58,273][20381] Updated weights for policy 1, policy_version 306142 (0.0008) [2023-12-27 07:45:58,315][20347] Updated weights for policy 0, policy_version 306368 (0.0007) [2023-12-27 07:45:58,332][20381] Updated weights for policy 1, policy_version 306152 (0.0008) [2023-12-27 07:45:59,021][20381] Updated weights for policy 1, policy_version 306162 (0.0008) [2023-12-27 07:45:59,071][20381] Updated weights for policy 1, policy_version 306172 (0.0007) [2023-12-27 07:45:59,113][20347] Updated weights for policy 0, policy_version 306378 (0.0007) [2023-12-27 07:45:59,122][20381] Updated weights for policy 1, policy_version 306182 (0.0007) [2023-12-27 07:45:59,174][20347] Updated weights for policy 0, policy_version 306388 (0.0007) [2023-12-27 07:45:59,233][20347] Updated weights for policy 0, policy_version 306398 (0.0007) [2023-12-27 07:45:59,871][20381] Updated weights for policy 1, policy_version 306192 (0.0008) [2023-12-27 07:45:59,886][20347] Updated weights for policy 0, policy_version 306408 (0.0009) [2023-12-27 07:45:59,930][20381] Updated weights for policy 1, policy_version 306202 (0.0006) [2023-12-27 07:45:59,937][20347] Updated weights for policy 0, policy_version 306418 (0.0010) [2023-12-27 07:45:59,990][20381] Updated weights for policy 1, policy_version 306212 (0.0008) [2023-12-27 07:45:59,999][20347] Updated weights for policy 0, policy_version 306428 (0.0008) [2023-12-27 07:46:00,577][20347] Updated weights for policy 0, policy_version 306438 (0.0008) [2023-12-27 07:46:00,633][20347] Updated weights for policy 0, policy_version 306448 (0.0010) [2023-12-27 07:46:00,686][20347] Updated weights for policy 0, policy_version 306458 (0.0009) [2023-12-27 07:46:00,700][20381] Updated weights for policy 1, policy_version 306222 (0.0007) [2023-12-27 07:46:00,762][20381] Updated weights for policy 1, policy_version 306232 (0.0008) [2023-12-27 07:46:00,822][20381] Updated weights for policy 1, policy_version 306242 (0.0008) [2023-12-27 07:46:01,264][20347] Updated weights for policy 0, policy_version 306468 (0.0007) [2023-12-27 07:46:01,331][20347] Updated weights for policy 0, policy_version 306478 (0.0011) [2023-12-27 07:46:01,389][20347] Updated weights for policy 0, policy_version 306488 (0.0010) [2023-12-27 07:46:01,539][20381] Updated weights for policy 1, policy_version 306252 (0.0007) [2023-12-27 07:46:01,591][20381] Updated weights for policy 1, policy_version 306262 (0.0007) [2023-12-27 07:46:01,656][20381] Updated weights for policy 1, policy_version 306272 (0.0008) [2023-12-27 07:46:02,149][20347] Updated weights for policy 0, policy_version 306498 (0.0009) [2023-12-27 07:46:02,207][20347] Updated weights for policy 0, policy_version 306508 (0.0010) [2023-12-27 07:46:02,265][20347] Updated weights for policy 0, policy_version 306518 (0.0010) [2023-12-27 07:46:02,323][20347] Updated weights for policy 0, policy_version 306528 (0.0010) [2023-12-27 07:46:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 156909568. Throughput: 0: 10176.9, 1: 10367.0. Samples: 156918680. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:46:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '229.610')] [2023-12-27 07:46:02,402][20381] Updated weights for policy 1, policy_version 306282 (0.0008) [2023-12-27 07:46:02,457][20381] Updated weights for policy 1, policy_version 306292 (0.0008) [2023-12-27 07:46:02,511][20381] Updated weights for policy 1, policy_version 306302 (0.0008) [2023-12-27 07:46:02,562][20381] Updated weights for policy 1, policy_version 306312 (0.0007) [2023-12-27 07:46:02,996][20347] Updated weights for policy 0, policy_version 306538 (0.0010) [2023-12-27 07:46:03,041][20347] Updated weights for policy 0, policy_version 306548 (0.0010) [2023-12-27 07:46:03,092][20347] Updated weights for policy 0, policy_version 306558 (0.0010) [2023-12-27 07:46:03,230][20381] Updated weights for policy 1, policy_version 306322 (0.0007) [2023-12-27 07:46:03,282][20381] Updated weights for policy 1, policy_version 306332 (0.0007) [2023-12-27 07:46:03,328][20381] Updated weights for policy 1, policy_version 306342 (0.0006) [2023-12-27 07:46:03,775][20347] Updated weights for policy 0, policy_version 306568 (0.0010) [2023-12-27 07:46:03,832][20347] Updated weights for policy 0, policy_version 306578 (0.0010) [2023-12-27 07:46:03,897][20347] Updated weights for policy 0, policy_version 306588 (0.0010) [2023-12-27 07:46:03,911][20381] Updated weights for policy 1, policy_version 306352 (0.0006) [2023-12-27 07:46:03,961][20381] Updated weights for policy 1, policy_version 306362 (0.0008) [2023-12-27 07:46:04,020][20381] Updated weights for policy 1, policy_version 306372 (0.0007) [2023-12-27 07:46:04,593][20347] Updated weights for policy 0, policy_version 306598 (0.0010) [2023-12-27 07:46:04,652][20347] Updated weights for policy 0, policy_version 306608 (0.0010) [2023-12-27 07:46:04,710][20347] Updated weights for policy 0, policy_version 306618 (0.0010) [2023-12-27 07:46:04,740][20381] Updated weights for policy 1, policy_version 306382 (0.0010) [2023-12-27 07:46:04,796][20381] Updated weights for policy 1, policy_version 306392 (0.0011) [2023-12-27 07:46:04,842][20381] Updated weights for policy 1, policy_version 306402 (0.0010) [2023-12-27 07:46:05,337][20347] Updated weights for policy 0, policy_version 306628 (0.0008) [2023-12-27 07:46:05,392][20347] Updated weights for policy 0, policy_version 306638 (0.0009) [2023-12-27 07:46:05,438][20347] Updated weights for policy 0, policy_version 306648 (0.0010) [2023-12-27 07:46:05,493][20381] Updated weights for policy 1, policy_version 306412 (0.0010) [2023-12-27 07:46:05,551][20381] Updated weights for policy 1, policy_version 306422 (0.0010) [2023-12-27 07:46:05,603][20381] Updated weights for policy 1, policy_version 306432 (0.0009) [2023-12-27 07:46:06,069][20347] Updated weights for policy 0, policy_version 306658 (0.0009) [2023-12-27 07:46:06,137][20347] Updated weights for policy 0, policy_version 306668 (0.0008) [2023-12-27 07:46:06,200][20347] Updated weights for policy 0, policy_version 306678 (0.0010) [2023-12-27 07:46:06,235][20381] Updated weights for policy 1, policy_version 306442 (0.0007) [2023-12-27 07:46:06,260][20347] Updated weights for policy 0, policy_version 306688 (0.0008) [2023-12-27 07:46:06,290][20381] Updated weights for policy 1, policy_version 306452 (0.0005) [2023-12-27 07:46:06,348][20381] Updated weights for policy 1, policy_version 306462 (0.0006) [2023-12-27 07:46:06,410][20381] Updated weights for policy 1, policy_version 306472 (0.0006) [2023-12-27 07:46:06,902][20347] Updated weights for policy 0, policy_version 306698 (0.0010) [2023-12-27 07:46:06,961][20347] Updated weights for policy 0, policy_version 306708 (0.0010) [2023-12-27 07:46:07,002][20381] Updated weights for policy 1, policy_version 306482 (0.0006) [2023-12-27 07:46:07,007][20347] Updated weights for policy 0, policy_version 306718 (0.0010) [2023-12-27 07:46:07,052][20381] Updated weights for policy 1, policy_version 306492 (0.0005) [2023-12-27 07:46:07,104][20381] Updated weights for policy 1, policy_version 306502 (0.0005) [2023-12-27 07:46:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 157016064. Throughput: 0: 10193.6, 1: 10403.4. Samples: 156980716. Policy #0 lag: (min: 18.0, avg: 27.6, max: 50.0) [2023-12-27 07:46:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '249.290')] [2023-12-27 07:46:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000306504_78479360.pth... [2023-12-27 07:46:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000306720_78536704.pth... [2023-12-27 07:46:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000305536_78233600.pth [2023-12-27 07:46:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000305288_78168064.pth [2023-12-27 07:46:07,684][20347] Updated weights for policy 0, policy_version 306728 (0.0010) [2023-12-27 07:46:07,739][20347] Updated weights for policy 0, policy_version 306738 (0.0010) [2023-12-27 07:46:07,749][20381] Updated weights for policy 1, policy_version 306512 (0.0010) [2023-12-27 07:46:07,787][20347] Updated weights for policy 0, policy_version 306748 (0.0010) [2023-12-27 07:46:07,801][20381] Updated weights for policy 1, policy_version 306522 (0.0010) [2023-12-27 07:46:07,856][20381] Updated weights for policy 1, policy_version 306532 (0.0010) [2023-12-27 07:46:08,493][20381] Updated weights for policy 1, policy_version 306542 (0.0010) [2023-12-27 07:46:08,494][20347] Updated weights for policy 0, policy_version 306758 (0.0007) [2023-12-27 07:46:08,541][20381] Updated weights for policy 1, policy_version 306552 (0.0010) [2023-12-27 07:46:08,543][20347] Updated weights for policy 0, policy_version 306768 (0.0005) [2023-12-27 07:46:08,597][20347] Updated weights for policy 0, policy_version 306778 (0.0007) [2023-12-27 07:46:08,600][20381] Updated weights for policy 1, policy_version 306562 (0.0010) [2023-12-27 07:46:09,240][20347] Updated weights for policy 0, policy_version 306788 (0.0008) [2023-12-27 07:46:09,284][20381] Updated weights for policy 1, policy_version 306572 (0.0009) [2023-12-27 07:46:09,300][20347] Updated weights for policy 0, policy_version 306798 (0.0007) [2023-12-27 07:46:09,342][20381] Updated weights for policy 1, policy_version 306582 (0.0007) [2023-12-27 07:46:09,365][20347] Updated weights for policy 0, policy_version 306808 (0.0007) [2023-12-27 07:46:09,412][20381] Updated weights for policy 1, policy_version 306592 (0.0007) [2023-12-27 07:46:10,082][20347] Updated weights for policy 0, policy_version 306818 (0.0006) [2023-12-27 07:46:10,091][20381] Updated weights for policy 1, policy_version 306602 (0.0007) [2023-12-27 07:46:10,142][20347] Updated weights for policy 0, policy_version 306828 (0.0007) [2023-12-27 07:46:10,151][20381] Updated weights for policy 1, policy_version 306612 (0.0008) [2023-12-27 07:46:10,203][20347] Updated weights for policy 0, policy_version 306838 (0.0006) [2023-12-27 07:46:10,215][20381] Updated weights for policy 1, policy_version 306622 (0.0008) [2023-12-27 07:46:10,257][20347] Updated weights for policy 0, policy_version 306848 (0.0007) [2023-12-27 07:46:10,277][20381] Updated weights for policy 1, policy_version 306632 (0.0007) [2023-12-27 07:46:10,888][20347] Updated weights for policy 0, policy_version 306858 (0.0005) [2023-12-27 07:46:10,937][20347] Updated weights for policy 0, policy_version 306868 (0.0005) [2023-12-27 07:46:10,988][20347] Updated weights for policy 0, policy_version 306878 (0.0005) [2023-12-27 07:46:11,002][20381] Updated weights for policy 1, policy_version 306642 (0.0008) [2023-12-27 07:46:11,071][20381] Updated weights for policy 1, policy_version 306652 (0.0007) [2023-12-27 07:46:11,140][20381] Updated weights for policy 1, policy_version 306662 (0.0007) [2023-12-27 07:46:11,652][20347] Updated weights for policy 0, policy_version 306888 (0.0008) [2023-12-27 07:46:11,714][20347] Updated weights for policy 0, policy_version 306898 (0.0008) [2023-12-27 07:46:11,780][20347] Updated weights for policy 0, policy_version 306908 (0.0008) [2023-12-27 07:46:11,916][20381] Updated weights for policy 1, policy_version 306672 (0.0008) [2023-12-27 07:46:11,969][20381] Updated weights for policy 1, policy_version 306682 (0.0008) [2023-12-27 07:46:12,032][20381] Updated weights for policy 1, policy_version 306692 (0.0008) [2023-12-27 07:46:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 157114368. Throughput: 0: 10311.7, 1: 10447.5. Samples: 157106312. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '254.950')] [2023-12-27 07:46:12,480][20347] Updated weights for policy 0, policy_version 306918 (0.0007) [2023-12-27 07:46:12,538][20347] Updated weights for policy 0, policy_version 306928 (0.0005) [2023-12-27 07:46:12,589][20347] Updated weights for policy 0, policy_version 306938 (0.0006) [2023-12-27 07:46:12,758][20381] Updated weights for policy 1, policy_version 306702 (0.0008) [2023-12-27 07:46:12,808][20381] Updated weights for policy 1, policy_version 306712 (0.0008) [2023-12-27 07:46:12,861][20381] Updated weights for policy 1, policy_version 306722 (0.0008) [2023-12-27 07:46:13,236][20347] Updated weights for policy 0, policy_version 306948 (0.0007) [2023-12-27 07:46:13,285][20347] Updated weights for policy 0, policy_version 306958 (0.0005) [2023-12-27 07:46:13,343][20347] Updated weights for policy 0, policy_version 306968 (0.0005) [2023-12-27 07:46:13,530][20381] Updated weights for policy 1, policy_version 306732 (0.0006) [2023-12-27 07:46:13,578][20381] Updated weights for policy 1, policy_version 306742 (0.0005) [2023-12-27 07:46:13,635][20381] Updated weights for policy 1, policy_version 306752 (0.0005) [2023-12-27 07:46:13,883][20347] Updated weights for policy 0, policy_version 306978 (0.0005) [2023-12-27 07:46:13,950][20347] Updated weights for policy 0, policy_version 306988 (0.0005) [2023-12-27 07:46:14,000][20347] Updated weights for policy 0, policy_version 306998 (0.0007) [2023-12-27 07:46:14,052][20347] Updated weights for policy 0, policy_version 307008 (0.0010) [2023-12-27 07:46:14,200][20381] Updated weights for policy 1, policy_version 306762 (0.0006) [2023-12-27 07:46:14,255][20381] Updated weights for policy 1, policy_version 306772 (0.0010) [2023-12-27 07:46:14,310][20381] Updated weights for policy 1, policy_version 306782 (0.0010) [2023-12-27 07:46:14,365][20381] Updated weights for policy 1, policy_version 306792 (0.0010) [2023-12-27 07:46:14,644][20347] Updated weights for policy 0, policy_version 307018 (0.0010) [2023-12-27 07:46:14,697][20347] Updated weights for policy 0, policy_version 307028 (0.0010) [2023-12-27 07:46:14,754][20347] Updated weights for policy 0, policy_version 307038 (0.0010) [2023-12-27 07:46:15,087][20381] Updated weights for policy 1, policy_version 306802 (0.0010) [2023-12-27 07:46:15,148][20381] Updated weights for policy 1, policy_version 306812 (0.0006) [2023-12-27 07:46:15,209][20381] Updated weights for policy 1, policy_version 306822 (0.0006) [2023-12-27 07:46:15,433][20347] Updated weights for policy 0, policy_version 307048 (0.0009) [2023-12-27 07:46:15,488][20347] Updated weights for policy 0, policy_version 307058 (0.0007) [2023-12-27 07:46:15,545][20347] Updated weights for policy 0, policy_version 307068 (0.0008) [2023-12-27 07:46:15,840][20381] Updated weights for policy 1, policy_version 306832 (0.0007) [2023-12-27 07:46:15,885][20381] Updated weights for policy 1, policy_version 306842 (0.0006) [2023-12-27 07:46:15,934][20381] Updated weights for policy 1, policy_version 306852 (0.0005) [2023-12-27 07:46:16,286][20347] Updated weights for policy 0, policy_version 307078 (0.0010) [2023-12-27 07:46:16,339][20347] Updated weights for policy 0, policy_version 307088 (0.0006) [2023-12-27 07:46:16,388][20347] Updated weights for policy 0, policy_version 307098 (0.0005) [2023-12-27 07:46:16,487][20381] Updated weights for policy 1, policy_version 306862 (0.0008) [2023-12-27 07:46:16,537][20381] Updated weights for policy 1, policy_version 306872 (0.0008) [2023-12-27 07:46:16,590][20381] Updated weights for policy 1, policy_version 306882 (0.0007) [2023-12-27 07:46:17,006][20347] Updated weights for policy 0, policy_version 307108 (0.0007) [2023-12-27 07:46:17,054][20347] Updated weights for policy 0, policy_version 307118 (0.0009) [2023-12-27 07:46:17,102][20347] Updated weights for policy 0, policy_version 307128 (0.0008) [2023-12-27 07:46:17,284][20381] Updated weights for policy 1, policy_version 306892 (0.0008) [2023-12-27 07:46:17,346][20381] Updated weights for policy 1, policy_version 306902 (0.0006) [2023-12-27 07:46:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 157220864. Throughput: 0: 10330.7, 1: 10426.1. Samples: 157233040. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '317.340')] [2023-12-27 07:46:17,406][20381] Updated weights for policy 1, policy_version 306912 (0.0008) [2023-12-27 07:46:17,836][20347] Updated weights for policy 0, policy_version 307138 (0.0008) [2023-12-27 07:46:17,895][20347] Updated weights for policy 0, policy_version 307148 (0.0008) [2023-12-27 07:46:17,961][20347] Updated weights for policy 0, policy_version 307158 (0.0007) [2023-12-27 07:46:18,018][20347] Updated weights for policy 0, policy_version 307168 (0.0008) [2023-12-27 07:46:18,065][20381] Updated weights for policy 1, policy_version 306922 (0.0007) [2023-12-27 07:46:18,118][20381] Updated weights for policy 1, policy_version 306932 (0.0007) [2023-12-27 07:46:18,180][20381] Updated weights for policy 1, policy_version 306942 (0.0005) [2023-12-27 07:46:18,235][20381] Updated weights for policy 1, policy_version 306952 (0.0007) [2023-12-27 07:46:18,668][20347] Updated weights for policy 0, policy_version 307178 (0.0008) [2023-12-27 07:46:18,721][20347] Updated weights for policy 0, policy_version 307188 (0.0007) [2023-12-27 07:46:18,780][20347] Updated weights for policy 0, policy_version 307198 (0.0007) [2023-12-27 07:46:18,912][20381] Updated weights for policy 1, policy_version 306962 (0.0007) [2023-12-27 07:46:18,967][20381] Updated weights for policy 1, policy_version 306972 (0.0007) [2023-12-27 07:46:19,021][20381] Updated weights for policy 1, policy_version 306982 (0.0007) [2023-12-27 07:46:19,496][20347] Updated weights for policy 0, policy_version 307208 (0.0008) [2023-12-27 07:46:19,566][20347] Updated weights for policy 0, policy_version 307218 (0.0007) [2023-12-27 07:46:19,615][20347] Updated weights for policy 0, policy_version 307228 (0.0007) [2023-12-27 07:46:19,707][20381] Updated weights for policy 1, policy_version 306992 (0.0007) [2023-12-27 07:46:19,765][20381] Updated weights for policy 1, policy_version 307002 (0.0008) [2023-12-27 07:46:19,830][20381] Updated weights for policy 1, policy_version 307012 (0.0008) [2023-12-27 07:46:20,345][20347] Updated weights for policy 0, policy_version 307238 (0.0007) [2023-12-27 07:46:20,405][20347] Updated weights for policy 0, policy_version 307248 (0.0008) [2023-12-27 07:46:20,464][20347] Updated weights for policy 0, policy_version 307258 (0.0006) [2023-12-27 07:46:20,530][20381] Updated weights for policy 1, policy_version 307022 (0.0007) [2023-12-27 07:46:20,587][20381] Updated weights for policy 1, policy_version 307032 (0.0008) [2023-12-27 07:46:20,650][20381] Updated weights for policy 1, policy_version 307042 (0.0008) [2023-12-27 07:46:21,118][20347] Updated weights for policy 0, policy_version 307268 (0.0006) [2023-12-27 07:46:21,187][20347] Updated weights for policy 0, policy_version 307278 (0.0008) [2023-12-27 07:46:21,249][20347] Updated weights for policy 0, policy_version 307288 (0.0008) [2023-12-27 07:46:21,348][20381] Updated weights for policy 1, policy_version 307052 (0.0008) [2023-12-27 07:46:21,417][20381] Updated weights for policy 1, policy_version 307062 (0.0008) [2023-12-27 07:46:21,479][20381] Updated weights for policy 1, policy_version 307072 (0.0008) [2023-12-27 07:46:21,938][20347] Updated weights for policy 0, policy_version 307298 (0.0007) [2023-12-27 07:46:22,001][20347] Updated weights for policy 0, policy_version 307308 (0.0008) [2023-12-27 07:46:22,060][20347] Updated weights for policy 0, policy_version 307318 (0.0007) [2023-12-27 07:46:22,120][20347] Updated weights for policy 0, policy_version 307328 (0.0006) [2023-12-27 07:46:22,209][20381] Updated weights for policy 1, policy_version 307082 (0.0008) [2023-12-27 07:46:22,270][20381] Updated weights for policy 1, policy_version 307092 (0.0008) [2023-12-27 07:46:22,330][20381] Updated weights for policy 1, policy_version 307102 (0.0008) [2023-12-27 07:46:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 157319168. Throughput: 0: 10296.4, 1: 10408.9. Samples: 157292868. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '316.720')] [2023-12-27 07:46:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000307328_78692352.pth... [2023-12-27 07:46:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000306112_78381056.pth [2023-12-27 07:46:22,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000307112_78635008.pth... [2023-12-27 07:46:22,396][20381] Updated weights for policy 1, policy_version 307112 (0.0007) [2023-12-27 07:46:22,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000305896_78323712.pth [2023-12-27 07:46:22,828][20347] Updated weights for policy 0, policy_version 307338 (0.0008) [2023-12-27 07:46:22,890][20347] Updated weights for policy 0, policy_version 307348 (0.0005) [2023-12-27 07:46:22,952][20347] Updated weights for policy 0, policy_version 307358 (0.0005) [2023-12-27 07:46:23,085][20381] Updated weights for policy 1, policy_version 307122 (0.0008) [2023-12-27 07:46:23,130][20381] Updated weights for policy 1, policy_version 307132 (0.0008) [2023-12-27 07:46:23,175][20381] Updated weights for policy 1, policy_version 307142 (0.0008) [2023-12-27 07:46:23,490][20347] Updated weights for policy 0, policy_version 307368 (0.0006) [2023-12-27 07:46:23,544][20347] Updated weights for policy 0, policy_version 307378 (0.0005) [2023-12-27 07:46:23,602][20347] Updated weights for policy 0, policy_version 307388 (0.0005) [2023-12-27 07:46:23,868][20381] Updated weights for policy 1, policy_version 307152 (0.0010) [2023-12-27 07:46:23,917][20381] Updated weights for policy 1, policy_version 307162 (0.0010) [2023-12-27 07:46:23,965][20381] Updated weights for policy 1, policy_version 307172 (0.0010) [2023-12-27 07:46:24,250][20347] Updated weights for policy 0, policy_version 307398 (0.0010) [2023-12-27 07:46:24,308][20347] Updated weights for policy 0, policy_version 307408 (0.0006) [2023-12-27 07:46:24,367][20347] Updated weights for policy 0, policy_version 307418 (0.0005) [2023-12-27 07:46:24,653][20381] Updated weights for policy 1, policy_version 307182 (0.0011) [2023-12-27 07:46:24,704][20381] Updated weights for policy 1, policy_version 307192 (0.0011) [2023-12-27 07:46:24,760][20381] Updated weights for policy 1, policy_version 307202 (0.0008) [2023-12-27 07:46:24,996][20347] Updated weights for policy 0, policy_version 307428 (0.0006) [2023-12-27 07:46:25,054][20347] Updated weights for policy 0, policy_version 307438 (0.0007) [2023-12-27 07:46:25,100][20347] Updated weights for policy 0, policy_version 307448 (0.0005) [2023-12-27 07:46:25,437][20381] Updated weights for policy 1, policy_version 307212 (0.0009) [2023-12-27 07:46:25,507][20381] Updated weights for policy 1, policy_version 307222 (0.0011) [2023-12-27 07:46:25,571][20381] Updated weights for policy 1, policy_version 307232 (0.0010) [2023-12-27 07:46:25,653][20347] Updated weights for policy 0, policy_version 307458 (0.0006) [2023-12-27 07:46:25,711][20347] Updated weights for policy 0, policy_version 307468 (0.0008) [2023-12-27 07:46:25,771][20347] Updated weights for policy 0, policy_version 307478 (0.0008) [2023-12-27 07:46:25,834][20347] Updated weights for policy 0, policy_version 307488 (0.0008) [2023-12-27 07:46:26,205][20381] Updated weights for policy 1, policy_version 307242 (0.0011) [2023-12-27 07:46:26,247][20381] Updated weights for policy 1, policy_version 307252 (0.0010) [2023-12-27 07:46:26,300][20381] Updated weights for policy 1, policy_version 307262 (0.0010) [2023-12-27 07:46:26,349][20381] Updated weights for policy 1, policy_version 307272 (0.0010) [2023-12-27 07:46:26,402][20347] Updated weights for policy 0, policy_version 307498 (0.0007) [2023-12-27 07:46:26,459][20347] Updated weights for policy 0, policy_version 307508 (0.0007) [2023-12-27 07:46:26,513][20347] Updated weights for policy 0, policy_version 307518 (0.0007) [2023-12-27 07:46:27,045][20381] Updated weights for policy 1, policy_version 307282 (0.0011) [2023-12-27 07:46:27,100][20381] Updated weights for policy 1, policy_version 307292 (0.0011) [2023-12-27 07:46:27,161][20381] Updated weights for policy 1, policy_version 307302 (0.0009) [2023-12-27 07:46:27,208][20347] Updated weights for policy 0, policy_version 307528 (0.0006) [2023-12-27 07:46:27,259][20347] Updated weights for policy 0, policy_version 307538 (0.0005) [2023-12-27 07:46:27,323][20347] Updated weights for policy 0, policy_version 307548 (0.0006) [2023-12-27 07:46:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 157433856. Throughput: 0: 10356.7, 1: 10363.6. Samples: 157417400. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '305.740')] [2023-12-27 07:46:27,799][20381] Updated weights for policy 1, policy_version 307312 (0.0005) [2023-12-27 07:46:27,845][20381] Updated weights for policy 1, policy_version 307322 (0.0005) [2023-12-27 07:46:27,891][20381] Updated weights for policy 1, policy_version 307332 (0.0005) [2023-12-27 07:46:27,977][20347] Updated weights for policy 0, policy_version 307558 (0.0007) [2023-12-27 07:46:28,033][20347] Updated weights for policy 0, policy_version 307568 (0.0008) [2023-12-27 07:46:28,084][20347] Updated weights for policy 0, policy_version 307578 (0.0007) [2023-12-27 07:46:28,522][20381] Updated weights for policy 1, policy_version 307342 (0.0007) [2023-12-27 07:46:28,570][20381] Updated weights for policy 1, policy_version 307352 (0.0007) [2023-12-27 07:46:28,619][20381] Updated weights for policy 1, policy_version 307362 (0.0008) [2023-12-27 07:46:28,740][20347] Updated weights for policy 0, policy_version 307588 (0.0008) [2023-12-27 07:46:28,796][20347] Updated weights for policy 0, policy_version 307598 (0.0010) [2023-12-27 07:46:28,851][20347] Updated weights for policy 0, policy_version 307608 (0.0010) [2023-12-27 07:46:29,265][20381] Updated weights for policy 1, policy_version 307372 (0.0008) [2023-12-27 07:46:29,319][20381] Updated weights for policy 1, policy_version 307382 (0.0008) [2023-12-27 07:46:29,375][20381] Updated weights for policy 1, policy_version 307392 (0.0008) [2023-12-27 07:46:29,487][20347] Updated weights for policy 0, policy_version 307618 (0.0010) [2023-12-27 07:46:29,537][20347] Updated weights for policy 0, policy_version 307628 (0.0008) [2023-12-27 07:46:29,585][20347] Updated weights for policy 0, policy_version 307638 (0.0008) [2023-12-27 07:46:29,629][20347] Updated weights for policy 0, policy_version 307648 (0.0007) [2023-12-27 07:46:30,027][20381] Updated weights for policy 1, policy_version 307402 (0.0008) [2023-12-27 07:46:30,078][20381] Updated weights for policy 1, policy_version 307412 (0.0010) [2023-12-27 07:46:30,126][20381] Updated weights for policy 1, policy_version 307422 (0.0010) [2023-12-27 07:46:30,178][20381] Updated weights for policy 1, policy_version 307432 (0.0011) [2023-12-27 07:46:30,355][20347] Updated weights for policy 0, policy_version 307658 (0.0007) [2023-12-27 07:46:30,414][20347] Updated weights for policy 0, policy_version 307668 (0.0008) [2023-12-27 07:46:30,470][20347] Updated weights for policy 0, policy_version 307678 (0.0010) [2023-12-27 07:46:30,880][20381] Updated weights for policy 1, policy_version 307442 (0.0010) [2023-12-27 07:46:30,924][20381] Updated weights for policy 1, policy_version 307452 (0.0010) [2023-12-27 07:46:30,978][20381] Updated weights for policy 1, policy_version 307462 (0.0010) [2023-12-27 07:46:31,146][20347] Updated weights for policy 0, policy_version 307688 (0.0010) [2023-12-27 07:46:31,198][20347] Updated weights for policy 0, policy_version 307698 (0.0010) [2023-12-27 07:46:31,262][20347] Updated weights for policy 0, policy_version 307708 (0.0010) [2023-12-27 07:46:31,706][20381] Updated weights for policy 1, policy_version 307472 (0.0006) [2023-12-27 07:46:31,772][20381] Updated weights for policy 1, policy_version 307482 (0.0011) [2023-12-27 07:46:31,831][20381] Updated weights for policy 1, policy_version 307492 (0.0010) [2023-12-27 07:46:31,885][20347] Updated weights for policy 0, policy_version 307718 (0.0007) [2023-12-27 07:46:31,932][20347] Updated weights for policy 0, policy_version 307728 (0.0005) [2023-12-27 07:46:31,986][20347] Updated weights for policy 0, policy_version 307738 (0.0007) [2023-12-27 07:46:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 157532160. Throughput: 0: 10435.1, 1: 10363.8. Samples: 157544024. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '360.420')] [2023-12-27 07:46:32,452][20381] Updated weights for policy 1, policy_version 307502 (0.0008) [2023-12-27 07:46:32,517][20381] Updated weights for policy 1, policy_version 307512 (0.0006) [2023-12-27 07:46:32,574][20381] Updated weights for policy 1, policy_version 307522 (0.0007) [2023-12-27 07:46:32,645][20347] Updated weights for policy 0, policy_version 307748 (0.0008) [2023-12-27 07:46:32,706][20347] Updated weights for policy 0, policy_version 307758 (0.0010) [2023-12-27 07:46:32,768][20347] Updated weights for policy 0, policy_version 307768 (0.0010) [2023-12-27 07:46:33,245][20381] Updated weights for policy 1, policy_version 307532 (0.0009) [2023-12-27 07:46:33,289][20381] Updated weights for policy 1, policy_version 307542 (0.0007) [2023-12-27 07:46:33,333][20381] Updated weights for policy 1, policy_version 307552 (0.0007) [2023-12-27 07:46:33,426][20347] Updated weights for policy 0, policy_version 307778 (0.0010) [2023-12-27 07:46:33,485][20347] Updated weights for policy 0, policy_version 307788 (0.0010) [2023-12-27 07:46:33,535][20347] Updated weights for policy 0, policy_version 307798 (0.0010) [2023-12-27 07:46:33,587][20347] Updated weights for policy 0, policy_version 307808 (0.0010) [2023-12-27 07:46:33,953][20381] Updated weights for policy 1, policy_version 307562 (0.0008) [2023-12-27 07:46:34,001][20381] Updated weights for policy 1, policy_version 307572 (0.0010) [2023-12-27 07:46:34,045][20381] Updated weights for policy 1, policy_version 307582 (0.0010) [2023-12-27 07:46:34,089][20381] Updated weights for policy 1, policy_version 307592 (0.0005) [2023-12-27 07:46:34,255][20347] Updated weights for policy 0, policy_version 307818 (0.0010) [2023-12-27 07:46:34,310][20347] Updated weights for policy 0, policy_version 307828 (0.0010) [2023-12-27 07:46:34,371][20347] Updated weights for policy 0, policy_version 307838 (0.0008) [2023-12-27 07:46:34,709][20381] Updated weights for policy 1, policy_version 307602 (0.0007) [2023-12-27 07:46:34,762][20381] Updated weights for policy 1, policy_version 307612 (0.0011) [2023-12-27 07:46:34,820][20381] Updated weights for policy 1, policy_version 307622 (0.0010) [2023-12-27 07:46:35,099][20347] Updated weights for policy 0, policy_version 307848 (0.0009) [2023-12-27 07:46:35,151][20347] Updated weights for policy 0, policy_version 307858 (0.0010) [2023-12-27 07:46:35,200][20347] Updated weights for policy 0, policy_version 307868 (0.0010) [2023-12-27 07:46:35,466][20381] Updated weights for policy 1, policy_version 307632 (0.0006) [2023-12-27 07:46:35,515][20381] Updated weights for policy 1, policy_version 307642 (0.0005) [2023-12-27 07:46:35,560][20381] Updated weights for policy 1, policy_version 307652 (0.0006) [2023-12-27 07:46:35,840][20347] Updated weights for policy 0, policy_version 307878 (0.0008) [2023-12-27 07:46:35,890][20347] Updated weights for policy 0, policy_version 307888 (0.0005) [2023-12-27 07:46:35,946][20347] Updated weights for policy 0, policy_version 307898 (0.0006) [2023-12-27 07:46:36,102][20381] Updated weights for policy 1, policy_version 307662 (0.0008) [2023-12-27 07:46:36,166][20381] Updated weights for policy 1, policy_version 307672 (0.0008) [2023-12-27 07:46:36,232][20381] Updated weights for policy 1, policy_version 307682 (0.0007) [2023-12-27 07:46:36,596][20347] Updated weights for policy 0, policy_version 307908 (0.0008) [2023-12-27 07:46:36,655][20347] Updated weights for policy 0, policy_version 307918 (0.0008) [2023-12-27 07:46:36,709][20347] Updated weights for policy 0, policy_version 307928 (0.0008) [2023-12-27 07:46:36,893][20381] Updated weights for policy 1, policy_version 307692 (0.0008) [2023-12-27 07:46:36,942][20381] Updated weights for policy 1, policy_version 307702 (0.0008) [2023-12-27 07:46:36,991][20381] Updated weights for policy 1, policy_version 307712 (0.0008) [2023-12-27 07:46:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 157638656. Throughput: 0: 10455.1, 1: 10404.8. Samples: 157607576. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '365.750')] [2023-12-27 07:46:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000307720_78790656.pth... [2023-12-27 07:46:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000307936_78848000.pth... [2023-12-27 07:46:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000306720_78536704.pth [2023-12-27 07:46:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000306504_78479360.pth [2023-12-27 07:46:37,435][20347] Updated weights for policy 0, policy_version 307938 (0.0007) [2023-12-27 07:46:37,497][20347] Updated weights for policy 0, policy_version 307948 (0.0007) [2023-12-27 07:46:37,545][20347] Updated weights for policy 0, policy_version 307958 (0.0008) [2023-12-27 07:46:37,597][20347] Updated weights for policy 0, policy_version 307968 (0.0008) [2023-12-27 07:46:37,649][20381] Updated weights for policy 1, policy_version 307722 (0.0008) [2023-12-27 07:46:37,708][20381] Updated weights for policy 1, policy_version 307732 (0.0008) [2023-12-27 07:46:37,776][20381] Updated weights for policy 1, policy_version 307742 (0.0007) [2023-12-27 07:46:37,835][20381] Updated weights for policy 1, policy_version 307752 (0.0008) [2023-12-27 07:46:38,195][20347] Updated weights for policy 0, policy_version 307978 (0.0005) [2023-12-27 07:46:38,247][20347] Updated weights for policy 0, policy_version 307988 (0.0006) [2023-12-27 07:46:38,305][20347] Updated weights for policy 0, policy_version 307998 (0.0008) [2023-12-27 07:46:38,590][20381] Updated weights for policy 1, policy_version 307762 (0.0007) [2023-12-27 07:46:38,650][20381] Updated weights for policy 1, policy_version 307772 (0.0011) [2023-12-27 07:46:38,705][20381] Updated weights for policy 1, policy_version 307782 (0.0007) [2023-12-27 07:46:39,010][20347] Updated weights for policy 0, policy_version 308008 (0.0007) [2023-12-27 07:46:39,065][20347] Updated weights for policy 0, policy_version 308018 (0.0008) [2023-12-27 07:46:39,123][20347] Updated weights for policy 0, policy_version 308028 (0.0007) [2023-12-27 07:46:39,426][20381] Updated weights for policy 1, policy_version 307792 (0.0007) [2023-12-27 07:46:39,483][20381] Updated weights for policy 1, policy_version 307802 (0.0008) [2023-12-27 07:46:39,531][20381] Updated weights for policy 1, policy_version 307812 (0.0008) [2023-12-27 07:46:39,793][20347] Updated weights for policy 0, policy_version 308038 (0.0008) [2023-12-27 07:46:39,861][20347] Updated weights for policy 0, policy_version 308048 (0.0008) [2023-12-27 07:46:39,924][20347] Updated weights for policy 0, policy_version 308058 (0.0007) [2023-12-27 07:46:40,238][20381] Updated weights for policy 1, policy_version 307822 (0.0008) [2023-12-27 07:46:40,293][20381] Updated weights for policy 1, policy_version 307832 (0.0008) [2023-12-27 07:46:40,344][20381] Updated weights for policy 1, policy_version 307842 (0.0007) [2023-12-27 07:46:40,660][20347] Updated weights for policy 0, policy_version 308068 (0.0008) [2023-12-27 07:46:40,711][20347] Updated weights for policy 0, policy_version 308078 (0.0008) [2023-12-27 07:46:40,769][20347] Updated weights for policy 0, policy_version 308088 (0.0007) [2023-12-27 07:46:40,961][20381] Updated weights for policy 1, policy_version 307852 (0.0007) [2023-12-27 07:46:41,016][20381] Updated weights for policy 1, policy_version 307862 (0.0006) [2023-12-27 07:46:41,083][20381] Updated weights for policy 1, policy_version 307872 (0.0008) [2023-12-27 07:46:41,492][20347] Updated weights for policy 0, policy_version 308098 (0.0008) [2023-12-27 07:46:41,548][20347] Updated weights for policy 0, policy_version 308108 (0.0010) [2023-12-27 07:46:41,613][20347] Updated weights for policy 0, policy_version 308118 (0.0011) [2023-12-27 07:46:41,679][20347] Updated weights for policy 0, policy_version 308128 (0.0011) [2023-12-27 07:46:41,857][20381] Updated weights for policy 1, policy_version 307882 (0.0008) [2023-12-27 07:46:41,918][20381] Updated weights for policy 1, policy_version 307892 (0.0008) [2023-12-27 07:46:41,971][20381] Updated weights for policy 1, policy_version 307902 (0.0008) [2023-12-27 07:46:42,022][20381] Updated weights for policy 1, policy_version 307912 (0.0007) [2023-12-27 07:46:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 157736960. Throughput: 0: 10433.1, 1: 10357.9. Samples: 157730312. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '448.680')] [2023-12-27 07:46:42,376][20347] Updated weights for policy 0, policy_version 308138 (0.0008) [2023-12-27 07:46:42,432][20347] Updated weights for policy 0, policy_version 308148 (0.0008) [2023-12-27 07:46:42,495][20347] Updated weights for policy 0, policy_version 308158 (0.0008) [2023-12-27 07:46:42,730][20381] Updated weights for policy 1, policy_version 307922 (0.0008) [2023-12-27 07:46:42,789][20381] Updated weights for policy 1, policy_version 307932 (0.0008) [2023-12-27 07:46:42,848][20381] Updated weights for policy 1, policy_version 307942 (0.0007) [2023-12-27 07:46:43,191][20347] Updated weights for policy 0, policy_version 308168 (0.0008) [2023-12-27 07:46:43,241][20347] Updated weights for policy 0, policy_version 308178 (0.0008) [2023-12-27 07:46:43,284][20347] Updated weights for policy 0, policy_version 308188 (0.0007) [2023-12-27 07:46:43,515][20381] Updated weights for policy 1, policy_version 307952 (0.0008) [2023-12-27 07:46:43,561][20381] Updated weights for policy 1, policy_version 307962 (0.0009) [2023-12-27 07:46:43,610][20381] Updated weights for policy 1, policy_version 307972 (0.0008) [2023-12-27 07:46:43,888][20347] Updated weights for policy 0, policy_version 308198 (0.0009) [2023-12-27 07:46:43,936][20347] Updated weights for policy 0, policy_version 308208 (0.0010) [2023-12-27 07:46:43,983][20347] Updated weights for policy 0, policy_version 308218 (0.0007) [2023-12-27 07:46:44,292][20381] Updated weights for policy 1, policy_version 307982 (0.0007) [2023-12-27 07:46:44,350][20381] Updated weights for policy 1, policy_version 307992 (0.0006) [2023-12-27 07:46:44,405][20381] Updated weights for policy 1, policy_version 308002 (0.0005) [2023-12-27 07:46:44,625][20347] Updated weights for policy 0, policy_version 308228 (0.0007) [2023-12-27 07:46:44,674][20347] Updated weights for policy 0, policy_version 308238 (0.0010) [2023-12-27 07:46:44,726][20347] Updated weights for policy 0, policy_version 308248 (0.0010) [2023-12-27 07:46:44,980][20381] Updated weights for policy 1, policy_version 308012 (0.0007) [2023-12-27 07:46:45,038][20381] Updated weights for policy 1, policy_version 308022 (0.0007) [2023-12-27 07:46:45,091][20381] Updated weights for policy 1, policy_version 308032 (0.0006) [2023-12-27 07:46:45,463][20347] Updated weights for policy 0, policy_version 308258 (0.0009) [2023-12-27 07:46:45,524][20347] Updated weights for policy 0, policy_version 308268 (0.0008) [2023-12-27 07:46:45,586][20347] Updated weights for policy 0, policy_version 308278 (0.0007) [2023-12-27 07:46:45,651][20347] Updated weights for policy 0, policy_version 308288 (0.0009) [2023-12-27 07:46:45,670][20381] Updated weights for policy 1, policy_version 308042 (0.0006) [2023-12-27 07:46:45,724][20381] Updated weights for policy 1, policy_version 308052 (0.0007) [2023-12-27 07:46:45,777][20381] Updated weights for policy 1, policy_version 308062 (0.0007) [2023-12-27 07:46:45,827][20381] Updated weights for policy 1, policy_version 308072 (0.0007) [2023-12-27 07:46:46,257][20347] Updated weights for policy 0, policy_version 308298 (0.0007) [2023-12-27 07:46:46,325][20347] Updated weights for policy 0, policy_version 308308 (0.0009) [2023-12-27 07:46:46,384][20347] Updated weights for policy 0, policy_version 308318 (0.0007) [2023-12-27 07:46:46,553][20381] Updated weights for policy 1, policy_version 308082 (0.0008) [2023-12-27 07:46:46,601][20381] Updated weights for policy 1, policy_version 308092 (0.0008) [2023-12-27 07:46:46,652][20381] Updated weights for policy 1, policy_version 308102 (0.0007) [2023-12-27 07:46:46,962][20347] Updated weights for policy 0, policy_version 308328 (0.0006) [2023-12-27 07:46:47,023][20347] Updated weights for policy 0, policy_version 308338 (0.0006) [2023-12-27 07:46:47,079][20347] Updated weights for policy 0, policy_version 308348 (0.0005) [2023-12-27 07:46:47,344][20381] Updated weights for policy 1, policy_version 308112 (0.0006) [2023-12-27 07:46:47,363][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 157843456. Throughput: 0: 10424.3, 1: 10397.5. Samples: 157855664. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '451.170')] [2023-12-27 07:46:47,394][20381] Updated weights for policy 1, policy_version 308122 (0.0005) [2023-12-27 07:46:47,451][20381] Updated weights for policy 1, policy_version 308132 (0.0006) [2023-12-27 07:46:47,639][20347] Updated weights for policy 0, policy_version 308358 (0.0005) [2023-12-27 07:46:47,692][20347] Updated weights for policy 0, policy_version 308368 (0.0005) [2023-12-27 07:46:47,748][20347] Updated weights for policy 0, policy_version 308378 (0.0008) [2023-12-27 07:46:48,137][20381] Updated weights for policy 1, policy_version 308142 (0.0007) [2023-12-27 07:46:48,200][20381] Updated weights for policy 1, policy_version 308152 (0.0008) [2023-12-27 07:46:48,255][20381] Updated weights for policy 1, policy_version 308162 (0.0008) [2023-12-27 07:46:48,425][20347] Updated weights for policy 0, policy_version 308388 (0.0008) [2023-12-27 07:46:48,490][20347] Updated weights for policy 0, policy_version 308398 (0.0007) [2023-12-27 07:46:48,548][20347] Updated weights for policy 0, policy_version 308408 (0.0007) [2023-12-27 07:46:48,957][20381] Updated weights for policy 1, policy_version 308172 (0.0007) [2023-12-27 07:46:49,010][20381] Updated weights for policy 1, policy_version 308182 (0.0008) [2023-12-27 07:46:49,070][20381] Updated weights for policy 1, policy_version 308192 (0.0008) [2023-12-27 07:46:49,242][20347] Updated weights for policy 0, policy_version 308418 (0.0006) [2023-12-27 07:46:49,302][20347] Updated weights for policy 0, policy_version 308428 (0.0008) [2023-12-27 07:46:49,366][20347] Updated weights for policy 0, policy_version 308438 (0.0008) [2023-12-27 07:46:49,418][20347] Updated weights for policy 0, policy_version 308448 (0.0007) [2023-12-27 07:46:49,793][20381] Updated weights for policy 1, policy_version 308202 (0.0007) [2023-12-27 07:46:49,850][20381] Updated weights for policy 1, policy_version 308212 (0.0007) [2023-12-27 07:46:49,905][20381] Updated weights for policy 1, policy_version 308222 (0.0008) [2023-12-27 07:46:49,961][20381] Updated weights for policy 1, policy_version 308232 (0.0008) [2023-12-27 07:46:50,117][20347] Updated weights for policy 0, policy_version 308458 (0.0007) [2023-12-27 07:46:50,160][20347] Updated weights for policy 0, policy_version 308468 (0.0008) [2023-12-27 07:46:50,207][20347] Updated weights for policy 0, policy_version 308478 (0.0006) [2023-12-27 07:46:50,652][20381] Updated weights for policy 1, policy_version 308242 (0.0005) [2023-12-27 07:46:50,711][20381] Updated weights for policy 1, policy_version 308252 (0.0006) [2023-12-27 07:46:50,772][20381] Updated weights for policy 1, policy_version 308262 (0.0007) [2023-12-27 07:46:50,954][20347] Updated weights for policy 0, policy_version 308488 (0.0008) [2023-12-27 07:46:51,016][20347] Updated weights for policy 0, policy_version 308498 (0.0007) [2023-12-27 07:46:51,083][20347] Updated weights for policy 0, policy_version 308508 (0.0007) [2023-12-27 07:46:51,416][20381] Updated weights for policy 1, policy_version 308272 (0.0007) [2023-12-27 07:46:51,469][20381] Updated weights for policy 1, policy_version 308282 (0.0008) [2023-12-27 07:46:51,521][20381] Updated weights for policy 1, policy_version 308292 (0.0008) [2023-12-27 07:46:51,784][20347] Updated weights for policy 0, policy_version 308518 (0.0008) [2023-12-27 07:46:51,834][20347] Updated weights for policy 0, policy_version 308528 (0.0008) [2023-12-27 07:46:51,890][20347] Updated weights for policy 0, policy_version 308538 (0.0008) [2023-12-27 07:46:52,246][20381] Updated weights for policy 1, policy_version 308302 (0.0008) [2023-12-27 07:46:52,306][20381] Updated weights for policy 1, policy_version 308312 (0.0007) [2023-12-27 07:46:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 157941760. Throughput: 0: 10438.0, 1: 10356.2. Samples: 157916452. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '445.070')] [2023-12-27 07:46:52,367][20381] Updated weights for policy 1, policy_version 308322 (0.0008) [2023-12-27 07:46:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000308544_79003648.pth... [2023-12-27 07:46:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000307328_78692352.pth [2023-12-27 07:46:52,402][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000308328_78946304.pth... [2023-12-27 07:46:52,405][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000307112_78635008.pth [2023-12-27 07:46:52,597][20347] Updated weights for policy 0, policy_version 308548 (0.0007) [2023-12-27 07:46:52,657][20347] Updated weights for policy 0, policy_version 308558 (0.0006) [2023-12-27 07:46:52,715][20347] Updated weights for policy 0, policy_version 308568 (0.0005) [2023-12-27 07:46:53,119][20381] Updated weights for policy 1, policy_version 308332 (0.0009) [2023-12-27 07:46:53,167][20381] Updated weights for policy 1, policy_version 308342 (0.0009) [2023-12-27 07:46:53,231][20381] Updated weights for policy 1, policy_version 308352 (0.0010) [2023-12-27 07:46:53,303][20347] Updated weights for policy 0, policy_version 308578 (0.0006) [2023-12-27 07:46:53,374][20347] Updated weights for policy 0, policy_version 308588 (0.0008) [2023-12-27 07:46:53,431][20347] Updated weights for policy 0, policy_version 308598 (0.0009) [2023-12-27 07:46:53,494][20347] Updated weights for policy 0, policy_version 308608 (0.0009) [2023-12-27 07:46:53,794][20381] Updated weights for policy 1, policy_version 308362 (0.0009) [2023-12-27 07:46:53,854][20381] Updated weights for policy 1, policy_version 308372 (0.0005) [2023-12-27 07:46:53,907][20381] Updated weights for policy 1, policy_version 308382 (0.0005) [2023-12-27 07:46:53,964][20381] Updated weights for policy 1, policy_version 308392 (0.0005) [2023-12-27 07:46:54,249][20347] Updated weights for policy 0, policy_version 308618 (0.0008) [2023-12-27 07:46:54,313][20347] Updated weights for policy 0, policy_version 308628 (0.0005) [2023-12-27 07:46:54,382][20347] Updated weights for policy 0, policy_version 308638 (0.0007) [2023-12-27 07:46:54,489][20381] Updated weights for policy 1, policy_version 308402 (0.0005) [2023-12-27 07:46:54,538][20381] Updated weights for policy 1, policy_version 308412 (0.0008) [2023-12-27 07:46:54,589][20381] Updated weights for policy 1, policy_version 308422 (0.0005) [2023-12-27 07:46:55,024][20347] Updated weights for policy 0, policy_version 308648 (0.0007) [2023-12-27 07:46:55,073][20347] Updated weights for policy 0, policy_version 308658 (0.0007) [2023-12-27 07:46:55,128][20347] Updated weights for policy 0, policy_version 308668 (0.0008) [2023-12-27 07:46:55,257][20381] Updated weights for policy 1, policy_version 308432 (0.0010) [2023-12-27 07:46:55,301][20381] Updated weights for policy 1, policy_version 308442 (0.0010) [2023-12-27 07:46:55,350][20381] Updated weights for policy 1, policy_version 308452 (0.0010) [2023-12-27 07:46:55,780][20347] Updated weights for policy 0, policy_version 308678 (0.0007) [2023-12-27 07:46:55,834][20347] Updated weights for policy 0, policy_version 308688 (0.0007) [2023-12-27 07:46:55,887][20347] Updated weights for policy 0, policy_version 308698 (0.0007) [2023-12-27 07:46:56,020][20381] Updated weights for policy 1, policy_version 308462 (0.0007) [2023-12-27 07:46:56,070][20381] Updated weights for policy 1, policy_version 308472 (0.0010) [2023-12-27 07:46:56,116][20381] Updated weights for policy 1, policy_version 308482 (0.0010) [2023-12-27 07:46:56,637][20347] Updated weights for policy 0, policy_version 308708 (0.0007) [2023-12-27 07:46:56,672][20381] Updated weights for policy 1, policy_version 308492 (0.0007) [2023-12-27 07:46:56,692][20347] Updated weights for policy 0, policy_version 308718 (0.0008) [2023-12-27 07:46:56,717][20381] Updated weights for policy 1, policy_version 308502 (0.0005) [2023-12-27 07:46:56,747][20347] Updated weights for policy 0, policy_version 308728 (0.0007) [2023-12-27 07:46:56,763][20381] Updated weights for policy 1, policy_version 308512 (0.0005) [2023-12-27 07:46:57,341][20381] Updated weights for policy 1, policy_version 308522 (0.0006) [2023-12-27 07:46:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 158048256. Throughput: 0: 10380.0, 1: 10372.2. Samples: 158040160. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:46:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '555.120')] [2023-12-27 07:46:57,392][20381] Updated weights for policy 1, policy_version 308532 (0.0010) [2023-12-27 07:46:57,443][20381] Updated weights for policy 1, policy_version 308542 (0.0010) [2023-12-27 07:46:57,466][20347] Updated weights for policy 0, policy_version 308738 (0.0007) [2023-12-27 07:46:57,494][20381] Updated weights for policy 1, policy_version 308552 (0.0010) [2023-12-27 07:46:57,518][20347] Updated weights for policy 0, policy_version 308748 (0.0006) [2023-12-27 07:46:57,568][20347] Updated weights for policy 0, policy_version 308758 (0.0007) [2023-12-27 07:46:57,620][20347] Updated weights for policy 0, policy_version 308768 (0.0007) [2023-12-27 07:46:58,142][20381] Updated weights for policy 1, policy_version 308562 (0.0010) [2023-12-27 07:46:58,201][20381] Updated weights for policy 1, policy_version 308572 (0.0011) [2023-12-27 07:46:58,258][20381] Updated weights for policy 1, policy_version 308582 (0.0011) [2023-12-27 07:46:58,292][20347] Updated weights for policy 0, policy_version 308778 (0.0008) [2023-12-27 07:46:58,358][20347] Updated weights for policy 0, policy_version 308788 (0.0008) [2023-12-27 07:46:58,422][20347] Updated weights for policy 0, policy_version 308798 (0.0008) [2023-12-27 07:46:59,020][20381] Updated weights for policy 1, policy_version 308592 (0.0008) [2023-12-27 07:46:59,070][20381] Updated weights for policy 1, policy_version 308602 (0.0008) [2023-12-27 07:46:59,124][20381] Updated weights for policy 1, policy_version 308612 (0.0007) [2023-12-27 07:46:59,164][20347] Updated weights for policy 0, policy_version 308808 (0.0007) [2023-12-27 07:46:59,234][20347] Updated weights for policy 0, policy_version 308818 (0.0008) [2023-12-27 07:46:59,290][20347] Updated weights for policy 0, policy_version 308828 (0.0010) [2023-12-27 07:46:59,776][20381] Updated weights for policy 1, policy_version 308622 (0.0007) [2023-12-27 07:46:59,827][20381] Updated weights for policy 1, policy_version 308632 (0.0007) [2023-12-27 07:46:59,889][20381] Updated weights for policy 1, policy_version 308642 (0.0007) [2023-12-27 07:46:59,966][20347] Updated weights for policy 0, policy_version 308838 (0.0009) [2023-12-27 07:47:00,030][20347] Updated weights for policy 0, policy_version 308848 (0.0008) [2023-12-27 07:47:00,094][20347] Updated weights for policy 0, policy_version 308858 (0.0007) [2023-12-27 07:47:00,521][20381] Updated weights for policy 1, policy_version 308652 (0.0008) [2023-12-27 07:47:00,567][20381] Updated weights for policy 1, policy_version 308662 (0.0010) [2023-12-27 07:47:00,619][20381] Updated weights for policy 1, policy_version 308672 (0.0008) [2023-12-27 07:47:00,690][20347] Updated weights for policy 0, policy_version 308868 (0.0010) [2023-12-27 07:47:00,738][20347] Updated weights for policy 0, policy_version 308878 (0.0010) [2023-12-27 07:47:00,788][20347] Updated weights for policy 0, policy_version 308888 (0.0010) [2023-12-27 07:47:01,282][20381] Updated weights for policy 1, policy_version 308682 (0.0009) [2023-12-27 07:47:01,335][20381] Updated weights for policy 1, policy_version 308692 (0.0011) [2023-12-27 07:47:01,396][20381] Updated weights for policy 1, policy_version 308702 (0.0010) [2023-12-27 07:47:01,447][20381] Updated weights for policy 1, policy_version 308712 (0.0010) [2023-12-27 07:47:01,462][20347] Updated weights for policy 0, policy_version 308898 (0.0010) [2023-12-27 07:47:01,513][20347] Updated weights for policy 0, policy_version 308908 (0.0010) [2023-12-27 07:47:01,577][20347] Updated weights for policy 0, policy_version 308918 (0.0010) [2023-12-27 07:47:01,646][20347] Updated weights for policy 0, policy_version 308928 (0.0010) [2023-12-27 07:47:02,140][20381] Updated weights for policy 1, policy_version 308722 (0.0011) [2023-12-27 07:47:02,194][20381] Updated weights for policy 1, policy_version 308732 (0.0009) [2023-12-27 07:47:02,227][20347] Updated weights for policy 0, policy_version 308938 (0.0010) [2023-12-27 07:47:02,252][20381] Updated weights for policy 1, policy_version 308742 (0.0010) [2023-12-27 07:47:02,281][20347] Updated weights for policy 0, policy_version 308948 (0.0008) [2023-12-27 07:47:02,337][20347] Updated weights for policy 0, policy_version 308958 (0.0008) [2023-12-27 07:47:02,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 158162944. Throughput: 0: 10321.8, 1: 10394.0. Samples: 158165256. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:47:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '540.880')] [2023-12-27 07:47:02,929][20381] Updated weights for policy 1, policy_version 308752 (0.0010) [2023-12-27 07:47:02,932][20347] Updated weights for policy 0, policy_version 308968 (0.0006) [2023-12-27 07:47:02,981][20381] Updated weights for policy 1, policy_version 308762 (0.0010) [2023-12-27 07:47:02,988][20347] Updated weights for policy 0, policy_version 308978 (0.0005) [2023-12-27 07:47:03,033][20381] Updated weights for policy 1, policy_version 308772 (0.0010) [2023-12-27 07:47:03,040][20347] Updated weights for policy 0, policy_version 308988 (0.0007) [2023-12-27 07:47:03,689][20347] Updated weights for policy 0, policy_version 308998 (0.0010) [2023-12-27 07:47:03,705][20381] Updated weights for policy 1, policy_version 308782 (0.0008) [2023-12-27 07:47:03,741][20347] Updated weights for policy 0, policy_version 309008 (0.0010) [2023-12-27 07:47:03,765][20381] Updated weights for policy 1, policy_version 308792 (0.0006) [2023-12-27 07:47:03,795][20347] Updated weights for policy 0, policy_version 309018 (0.0010) [2023-12-27 07:47:03,822][20381] Updated weights for policy 1, policy_version 308802 (0.0006) [2023-12-27 07:47:04,486][20347] Updated weights for policy 0, policy_version 309028 (0.0010) [2023-12-27 07:47:04,531][20381] Updated weights for policy 1, policy_version 308812 (0.0007) [2023-12-27 07:47:04,544][20347] Updated weights for policy 0, policy_version 309038 (0.0010) [2023-12-27 07:47:04,595][20381] Updated weights for policy 1, policy_version 308822 (0.0006) [2023-12-27 07:47:04,598][20347] Updated weights for policy 0, policy_version 309048 (0.0010) [2023-12-27 07:47:04,652][20381] Updated weights for policy 1, policy_version 308832 (0.0010) [2023-12-27 07:47:05,274][20347] Updated weights for policy 0, policy_version 309058 (0.0010) [2023-12-27 07:47:05,279][20381] Updated weights for policy 1, policy_version 308842 (0.0010) [2023-12-27 07:47:05,324][20381] Updated weights for policy 1, policy_version 308852 (0.0010) [2023-12-27 07:47:05,328][20347] Updated weights for policy 0, policy_version 309068 (0.0010) [2023-12-27 07:47:05,366][20381] Updated weights for policy 1, policy_version 308862 (0.0010) [2023-12-27 07:47:05,382][20347] Updated weights for policy 0, policy_version 309078 (0.0010) [2023-12-27 07:47:05,414][20381] Updated weights for policy 1, policy_version 308872 (0.0010) [2023-12-27 07:47:05,437][20347] Updated weights for policy 0, policy_version 309088 (0.0010) [2023-12-27 07:47:06,082][20381] Updated weights for policy 1, policy_version 308882 (0.0010) [2023-12-27 07:47:06,102][20347] Updated weights for policy 0, policy_version 309098 (0.0009) [2023-12-27 07:47:06,145][20381] Updated weights for policy 1, policy_version 308892 (0.0010) [2023-12-27 07:47:06,157][20347] Updated weights for policy 0, policy_version 309108 (0.0010) [2023-12-27 07:47:06,202][20381] Updated weights for policy 1, policy_version 308902 (0.0011) [2023-12-27 07:47:06,215][20347] Updated weights for policy 0, policy_version 309118 (0.0010) [2023-12-27 07:47:06,867][20381] Updated weights for policy 1, policy_version 308912 (0.0007) [2023-12-27 07:47:06,925][20381] Updated weights for policy 1, policy_version 308922 (0.0009) [2023-12-27 07:47:06,928][20347] Updated weights for policy 0, policy_version 309128 (0.0010) [2023-12-27 07:47:06,985][20381] Updated weights for policy 1, policy_version 308932 (0.0011) [2023-12-27 07:47:06,988][20347] Updated weights for policy 0, policy_version 309138 (0.0011) [2023-12-27 07:47:07,043][20347] Updated weights for policy 0, policy_version 309148 (0.0011) [2023-12-27 07:47:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 158261248. Throughput: 0: 10376.8, 1: 10408.4. Samples: 158228204. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:47:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '559.050')] [2023-12-27 07:47:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000308936_79101952.pth... [2023-12-27 07:47:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000309152_79159296.pth... [2023-12-27 07:47:07,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000307720_78790656.pth [2023-12-27 07:47:07,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000307936_78848000.pth [2023-12-27 07:47:07,643][20381] Updated weights for policy 1, policy_version 308942 (0.0010) [2023-12-27 07:47:07,697][20381] Updated weights for policy 1, policy_version 308952 (0.0010) [2023-12-27 07:47:07,732][20347] Updated weights for policy 0, policy_version 309158 (0.0011) [2023-12-27 07:47:07,750][20381] Updated weights for policy 1, policy_version 308962 (0.0006) [2023-12-27 07:47:07,787][20347] Updated weights for policy 0, policy_version 309168 (0.0010) [2023-12-27 07:47:07,844][20347] Updated weights for policy 0, policy_version 309178 (0.0010) [2023-12-27 07:47:08,422][20381] Updated weights for policy 1, policy_version 308972 (0.0009) [2023-12-27 07:47:08,488][20381] Updated weights for policy 1, policy_version 308982 (0.0010) [2023-12-27 07:47:08,499][20347] Updated weights for policy 0, policy_version 309188 (0.0008) [2023-12-27 07:47:08,549][20381] Updated weights for policy 1, policy_version 308992 (0.0010) [2023-12-27 07:47:08,556][20347] Updated weights for policy 0, policy_version 309198 (0.0005) [2023-12-27 07:47:08,615][20347] Updated weights for policy 0, policy_version 309208 (0.0007) [2023-12-27 07:47:09,191][20381] Updated weights for policy 1, policy_version 309002 (0.0009) [2023-12-27 07:47:09,255][20381] Updated weights for policy 1, policy_version 309012 (0.0010) [2023-12-27 07:47:09,297][20347] Updated weights for policy 0, policy_version 309218 (0.0006) [2023-12-27 07:47:09,319][20381] Updated weights for policy 1, policy_version 309022 (0.0011) [2023-12-27 07:47:09,360][20347] Updated weights for policy 0, policy_version 309228 (0.0007) [2023-12-27 07:47:09,385][20381] Updated weights for policy 1, policy_version 309032 (0.0010) [2023-12-27 07:47:09,429][20347] Updated weights for policy 0, policy_version 309238 (0.0007) [2023-12-27 07:47:09,489][20347] Updated weights for policy 0, policy_version 309248 (0.0006) [2023-12-27 07:47:10,081][20381] Updated weights for policy 1, policy_version 309042 (0.0008) [2023-12-27 07:47:10,137][20381] Updated weights for policy 1, policy_version 309052 (0.0007) [2023-12-27 07:47:10,185][20347] Updated weights for policy 0, policy_version 309258 (0.0008) [2023-12-27 07:47:10,192][20381] Updated weights for policy 1, policy_version 309062 (0.0007) [2023-12-27 07:47:10,246][20347] Updated weights for policy 0, policy_version 309268 (0.0008) [2023-12-27 07:47:10,305][20347] Updated weights for policy 0, policy_version 309278 (0.0007) [2023-12-27 07:47:10,871][20381] Updated weights for policy 1, policy_version 309072 (0.0008) [2023-12-27 07:47:10,920][20381] Updated weights for policy 1, policy_version 309082 (0.0009) [2023-12-27 07:47:10,962][20347] Updated weights for policy 0, policy_version 309288 (0.0006) [2023-12-27 07:47:10,976][20381] Updated weights for policy 1, policy_version 309092 (0.0007) [2023-12-27 07:47:11,016][20347] Updated weights for policy 0, policy_version 309298 (0.0006) [2023-12-27 07:47:11,080][20347] Updated weights for policy 0, policy_version 309308 (0.0009) [2023-12-27 07:47:11,792][20347] Updated weights for policy 0, policy_version 309318 (0.0006) [2023-12-27 07:47:11,795][20381] Updated weights for policy 1, policy_version 309102 (0.0007) [2023-12-27 07:47:11,847][20347] Updated weights for policy 0, policy_version 309328 (0.0007) [2023-12-27 07:47:11,854][20381] Updated weights for policy 1, policy_version 309112 (0.0007) [2023-12-27 07:47:11,907][20347] Updated weights for policy 0, policy_version 309338 (0.0007) [2023-12-27 07:47:11,917][20381] Updated weights for policy 1, policy_version 309122 (0.0005) [2023-12-27 07:47:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 158359552. Throughput: 0: 10305.9, 1: 10434.9. Samples: 158350740. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:47:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '547.550')] [2023-12-27 07:47:12,553][20381] Updated weights for policy 1, policy_version 309132 (0.0007) [2023-12-27 07:47:12,597][20347] Updated weights for policy 0, policy_version 309348 (0.0007) [2023-12-27 07:47:12,607][20381] Updated weights for policy 1, policy_version 309142 (0.0007) [2023-12-27 07:47:12,652][20347] Updated weights for policy 0, policy_version 309358 (0.0008) [2023-12-27 07:47:12,665][20381] Updated weights for policy 1, policy_version 309152 (0.0006) [2023-12-27 07:47:12,701][20347] Updated weights for policy 0, policy_version 309368 (0.0005) [2023-12-27 07:47:13,343][20381] Updated weights for policy 1, policy_version 309162 (0.0006) [2023-12-27 07:47:13,355][20347] Updated weights for policy 0, policy_version 309378 (0.0007) [2023-12-27 07:47:13,398][20381] Updated weights for policy 1, policy_version 309172 (0.0010) [2023-12-27 07:47:13,408][20347] Updated weights for policy 0, policy_version 309388 (0.0005) [2023-12-27 07:47:13,444][20381] Updated weights for policy 1, policy_version 309182 (0.0010) [2023-12-27 07:47:13,467][20347] Updated weights for policy 0, policy_version 309398 (0.0006) [2023-12-27 07:47:13,502][20381] Updated weights for policy 1, policy_version 309192 (0.0010) [2023-12-27 07:47:13,519][20347] Updated weights for policy 0, policy_version 309408 (0.0006) [2023-12-27 07:47:14,125][20381] Updated weights for policy 1, policy_version 309202 (0.0007) [2023-12-27 07:47:14,181][20381] Updated weights for policy 1, policy_version 309212 (0.0005) [2023-12-27 07:47:14,184][20347] Updated weights for policy 0, policy_version 309418 (0.0007) [2023-12-27 07:47:14,225][20381] Updated weights for policy 1, policy_version 309222 (0.0005) [2023-12-27 07:47:14,242][20347] Updated weights for policy 0, policy_version 309428 (0.0007) [2023-12-27 07:47:14,303][20347] Updated weights for policy 0, policy_version 309438 (0.0008) [2023-12-27 07:47:14,835][20381] Updated weights for policy 1, policy_version 309232 (0.0007) [2023-12-27 07:47:14,890][20381] Updated weights for policy 1, policy_version 309242 (0.0006) [2023-12-27 07:47:14,951][20381] Updated weights for policy 1, policy_version 309252 (0.0006) [2023-12-27 07:47:15,022][20347] Updated weights for policy 0, policy_version 309448 (0.0006) [2023-12-27 07:47:15,087][20347] Updated weights for policy 0, policy_version 309458 (0.0006) [2023-12-27 07:47:15,143][20347] Updated weights for policy 0, policy_version 309468 (0.0010) [2023-12-27 07:47:15,609][20381] Updated weights for policy 1, policy_version 309262 (0.0009) [2023-12-27 07:47:15,657][20381] Updated weights for policy 1, policy_version 309272 (0.0007) [2023-12-27 07:47:15,712][20381] Updated weights for policy 1, policy_version 309282 (0.0007) [2023-12-27 07:47:15,760][20347] Updated weights for policy 0, policy_version 309478 (0.0008) [2023-12-27 07:47:15,821][20347] Updated weights for policy 0, policy_version 309488 (0.0008) [2023-12-27 07:47:15,878][20347] Updated weights for policy 0, policy_version 309498 (0.0007) [2023-12-27 07:47:16,421][20381] Updated weights for policy 1, policy_version 309292 (0.0008) [2023-12-27 07:47:16,471][20381] Updated weights for policy 1, policy_version 309302 (0.0008) [2023-12-27 07:47:16,519][20381] Updated weights for policy 1, policy_version 309312 (0.0007) [2023-12-27 07:47:16,525][20347] Updated weights for policy 0, policy_version 309508 (0.0006) [2023-12-27 07:47:16,579][20347] Updated weights for policy 0, policy_version 309518 (0.0006) [2023-12-27 07:47:16,629][20347] Updated weights for policy 0, policy_version 309528 (0.0008) [2023-12-27 07:47:17,225][20381] Updated weights for policy 1, policy_version 309322 (0.0006) [2023-12-27 07:47:17,276][20381] Updated weights for policy 1, policy_version 309332 (0.0007) [2023-12-27 07:47:17,330][20381] Updated weights for policy 1, policy_version 309342 (0.0007) [2023-12-27 07:47:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 158457856. Throughput: 0: 10256.0, 1: 10420.4. Samples: 158474464. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:47:17,363][20347] Updated weights for policy 0, policy_version 309538 (0.0008) [2023-12-27 07:47:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '626.740')] [2023-12-27 07:47:17,392][20381] Updated weights for policy 1, policy_version 309352 (0.0009) [2023-12-27 07:47:17,419][20347] Updated weights for policy 0, policy_version 309548 (0.0010) [2023-12-27 07:47:17,482][20347] Updated weights for policy 0, policy_version 309558 (0.0006) [2023-12-27 07:47:17,539][20347] Updated weights for policy 0, policy_version 309568 (0.0005) [2023-12-27 07:47:18,086][20381] Updated weights for policy 1, policy_version 309362 (0.0008) [2023-12-27 07:47:18,096][20347] Updated weights for policy 0, policy_version 309578 (0.0005) [2023-12-27 07:47:18,147][20381] Updated weights for policy 1, policy_version 309372 (0.0005) [2023-12-27 07:47:18,154][20347] Updated weights for policy 0, policy_version 309588 (0.0007) [2023-12-27 07:47:18,206][20381] Updated weights for policy 1, policy_version 309382 (0.0005) [2023-12-27 07:47:18,216][20347] Updated weights for policy 0, policy_version 309598 (0.0007) [2023-12-27 07:47:18,856][20381] Updated weights for policy 1, policy_version 309392 (0.0008) [2023-12-27 07:47:18,864][20347] Updated weights for policy 0, policy_version 309608 (0.0008) [2023-12-27 07:47:18,914][20381] Updated weights for policy 1, policy_version 309402 (0.0007) [2023-12-27 07:47:18,927][20347] Updated weights for policy 0, policy_version 309618 (0.0008) [2023-12-27 07:47:18,966][20381] Updated weights for policy 1, policy_version 309412 (0.0007) [2023-12-27 07:47:18,988][20347] Updated weights for policy 0, policy_version 309628 (0.0006) [2023-12-27 07:47:19,696][20347] Updated weights for policy 0, policy_version 309638 (0.0007) [2023-12-27 07:47:19,717][20381] Updated weights for policy 1, policy_version 309422 (0.0010) [2023-12-27 07:47:19,752][20347] Updated weights for policy 0, policy_version 309648 (0.0006) [2023-12-27 07:47:19,776][20381] Updated weights for policy 1, policy_version 309432 (0.0011) [2023-12-27 07:47:19,806][20347] Updated weights for policy 0, policy_version 309658 (0.0007) [2023-12-27 07:47:19,841][20381] Updated weights for policy 1, policy_version 309442 (0.0008) [2023-12-27 07:47:20,542][20381] Updated weights for policy 1, policy_version 309452 (0.0008) [2023-12-27 07:47:20,567][20347] Updated weights for policy 0, policy_version 309668 (0.0008) [2023-12-27 07:47:20,600][20381] Updated weights for policy 1, policy_version 309462 (0.0008) [2023-12-27 07:47:20,628][20347] Updated weights for policy 0, policy_version 309678 (0.0007) [2023-12-27 07:47:20,663][20381] Updated weights for policy 1, policy_version 309472 (0.0011) [2023-12-27 07:47:20,684][20347] Updated weights for policy 0, policy_version 309688 (0.0006) [2023-12-27 07:47:21,378][20347] Updated weights for policy 0, policy_version 309698 (0.0007) [2023-12-27 07:47:21,402][20381] Updated weights for policy 1, policy_version 309482 (0.0010) [2023-12-27 07:47:21,438][20347] Updated weights for policy 0, policy_version 309708 (0.0008) [2023-12-27 07:47:21,466][20381] Updated weights for policy 1, policy_version 309492 (0.0006) [2023-12-27 07:47:21,500][20347] Updated weights for policy 0, policy_version 309718 (0.0007) [2023-12-27 07:47:21,522][20381] Updated weights for policy 1, policy_version 309502 (0.0006) [2023-12-27 07:47:21,559][20347] Updated weights for policy 0, policy_version 309728 (0.0008) [2023-12-27 07:47:21,584][20381] Updated weights for policy 1, policy_version 309512 (0.0006) [2023-12-27 07:47:22,276][20381] Updated weights for policy 1, policy_version 309522 (0.0008) [2023-12-27 07:47:22,278][20347] Updated weights for policy 0, policy_version 309738 (0.0007) [2023-12-27 07:47:22,335][20347] Updated weights for policy 0, policy_version 309748 (0.0007) [2023-12-27 07:47:22,335][20381] Updated weights for policy 1, policy_version 309532 (0.0008) [2023-12-27 07:47:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 158556160. Throughput: 0: 10272.3, 1: 10341.0. Samples: 158535168. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) [2023-12-27 07:47:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '654.920')] [2023-12-27 07:47:22,393][20347] Updated weights for policy 0, policy_version 309758 (0.0007) [2023-12-27 07:47:22,396][20381] Updated weights for policy 1, policy_version 309542 (0.0008) [2023-12-27 07:47:22,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000309760_79314944.pth... [2023-12-27 07:47:22,405][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000309544_79257600.pth... [2023-12-27 07:47:22,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000308544_79003648.pth [2023-12-27 07:47:22,408][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000308328_78946304.pth [2023-12-27 07:47:23,031][20381] Updated weights for policy 1, policy_version 309552 (0.0006) [2023-12-27 07:47:23,082][20381] Updated weights for policy 1, policy_version 309562 (0.0005) [2023-12-27 07:47:23,125][20347] Updated weights for policy 0, policy_version 309768 (0.0006) [2023-12-27 07:47:23,132][20381] Updated weights for policy 1, policy_version 309572 (0.0006) [2023-12-27 07:47:23,178][20347] Updated weights for policy 0, policy_version 309778 (0.0009) [2023-12-27 07:47:23,225][20347] Updated weights for policy 0, policy_version 309788 (0.0008) [2023-12-27 07:47:23,792][20381] Updated weights for policy 1, policy_version 309582 (0.0008) [2023-12-27 07:47:23,843][20381] Updated weights for policy 1, policy_version 309592 (0.0007) [2023-12-27 07:47:23,904][20381] Updated weights for policy 1, policy_version 309602 (0.0007) [2023-12-27 07:47:23,926][20347] Updated weights for policy 0, policy_version 309798 (0.0007) [2023-12-27 07:47:23,981][20347] Updated weights for policy 0, policy_version 309808 (0.0008) [2023-12-27 07:47:24,038][20347] Updated weights for policy 0, policy_version 309818 (0.0008) [2023-12-27 07:47:24,590][20381] Updated weights for policy 1, policy_version 309612 (0.0006) [2023-12-27 07:47:24,647][20381] Updated weights for policy 1, policy_version 309622 (0.0005) [2023-12-27 07:47:24,711][20381] Updated weights for policy 1, policy_version 309632 (0.0005) [2023-12-27 07:47:24,740][20347] Updated weights for policy 0, policy_version 309828 (0.0008) [2023-12-27 07:47:24,798][20347] Updated weights for policy 0, policy_version 309838 (0.0008) [2023-12-27 07:47:24,854][20347] Updated weights for policy 0, policy_version 309848 (0.0008) [2023-12-27 07:47:25,221][20381] Updated weights for policy 1, policy_version 309642 (0.0005) [2023-12-27 07:47:25,284][20381] Updated weights for policy 1, policy_version 309652 (0.0008) [2023-12-27 07:47:25,330][20381] Updated weights for policy 1, policy_version 309662 (0.0008) [2023-12-27 07:47:25,376][20381] Updated weights for policy 1, policy_version 309672 (0.0009) [2023-12-27 07:47:25,515][20347] Updated weights for policy 0, policy_version 309858 (0.0008) [2023-12-27 07:47:25,569][20347] Updated weights for policy 0, policy_version 309868 (0.0008) [2023-12-27 07:47:25,628][20347] Updated weights for policy 0, policy_version 309878 (0.0008) [2023-12-27 07:47:25,689][20347] Updated weights for policy 0, policy_version 309888 (0.0007) [2023-12-27 07:47:25,987][20381] Updated weights for policy 1, policy_version 309682 (0.0005) [2023-12-27 07:47:26,043][20381] Updated weights for policy 1, policy_version 309692 (0.0005) [2023-12-27 07:47:26,089][20381] Updated weights for policy 1, policy_version 309702 (0.0005) [2023-12-27 07:47:26,281][20347] Updated weights for policy 0, policy_version 309898 (0.0008) [2023-12-27 07:47:26,329][20347] Updated weights for policy 0, policy_version 309908 (0.0008) [2023-12-27 07:47:26,381][20347] Updated weights for policy 0, policy_version 309918 (0.0009) [2023-12-27 07:47:26,638][20381] Updated weights for policy 1, policy_version 309712 (0.0007) [2023-12-27 07:47:26,689][20381] Updated weights for policy 1, policy_version 309722 (0.0008) [2023-12-27 07:47:26,743][20381] Updated weights for policy 1, policy_version 309732 (0.0008) [2023-12-27 07:47:27,081][20347] Updated weights for policy 0, policy_version 309928 (0.0009) [2023-12-27 07:47:27,141][20347] Updated weights for policy 0, policy_version 309938 (0.0010) [2023-12-27 07:47:27,200][20347] Updated weights for policy 0, policy_version 309948 (0.0010) [2023-12-27 07:47:27,348][20381] Updated weights for policy 1, policy_version 309742 (0.0009) [2023-12-27 07:47:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 158670848. Throughput: 0: 10256.2, 1: 10393.3. Samples: 158659540. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '713.540')] [2023-12-27 07:47:27,396][20381] Updated weights for policy 1, policy_version 309752 (0.0010) [2023-12-27 07:47:27,453][20381] Updated weights for policy 1, policy_version 309762 (0.0010) [2023-12-27 07:47:27,814][20347] Updated weights for policy 0, policy_version 309958 (0.0007) [2023-12-27 07:47:27,871][20347] Updated weights for policy 0, policy_version 309968 (0.0005) [2023-12-27 07:47:27,932][20347] Updated weights for policy 0, policy_version 309978 (0.0005) [2023-12-27 07:47:28,110][20381] Updated weights for policy 1, policy_version 309772 (0.0010) [2023-12-27 07:47:28,163][20381] Updated weights for policy 1, policy_version 309782 (0.0010) [2023-12-27 07:47:28,219][20381] Updated weights for policy 1, policy_version 309792 (0.0010) [2023-12-27 07:47:28,551][20347] Updated weights for policy 0, policy_version 309988 (0.0006) [2023-12-27 07:47:28,613][20347] Updated weights for policy 0, policy_version 309998 (0.0008) [2023-12-27 07:47:28,660][20347] Updated weights for policy 0, policy_version 310008 (0.0008) [2023-12-27 07:47:28,908][20381] Updated weights for policy 1, policy_version 309802 (0.0010) [2023-12-27 07:47:28,969][20381] Updated weights for policy 1, policy_version 309812 (0.0010) [2023-12-27 07:47:29,014][20381] Updated weights for policy 1, policy_version 309822 (0.0010) [2023-12-27 07:47:29,062][20381] Updated weights for policy 1, policy_version 309832 (0.0010) [2023-12-27 07:47:29,341][20347] Updated weights for policy 0, policy_version 310018 (0.0008) [2023-12-27 07:47:29,404][20347] Updated weights for policy 0, policy_version 310028 (0.0008) [2023-12-27 07:47:29,455][20347] Updated weights for policy 0, policy_version 310038 (0.0007) [2023-12-27 07:47:29,504][20347] Updated weights for policy 0, policy_version 310048 (0.0008) [2023-12-27 07:47:29,779][20381] Updated weights for policy 1, policy_version 309842 (0.0011) [2023-12-27 07:47:29,835][20381] Updated weights for policy 1, policy_version 309852 (0.0011) [2023-12-27 07:47:29,899][20381] Updated weights for policy 1, policy_version 309862 (0.0011) [2023-12-27 07:47:30,199][20347] Updated weights for policy 0, policy_version 310058 (0.0008) [2023-12-27 07:47:30,253][20347] Updated weights for policy 0, policy_version 310068 (0.0010) [2023-12-27 07:47:30,312][20347] Updated weights for policy 0, policy_version 310078 (0.0010) [2023-12-27 07:47:30,513][20381] Updated weights for policy 1, policy_version 309872 (0.0010) [2023-12-27 07:47:30,558][20381] Updated weights for policy 1, policy_version 309882 (0.0010) [2023-12-27 07:47:30,606][20381] Updated weights for policy 1, policy_version 309892 (0.0010) [2023-12-27 07:47:30,963][20347] Updated weights for policy 0, policy_version 310088 (0.0010) [2023-12-27 07:47:31,017][20347] Updated weights for policy 0, policy_version 310098 (0.0009) [2023-12-27 07:47:31,072][20347] Updated weights for policy 0, policy_version 310108 (0.0008) [2023-12-27 07:47:31,328][20381] Updated weights for policy 1, policy_version 309902 (0.0010) [2023-12-27 07:47:31,396][20381] Updated weights for policy 1, policy_version 309912 (0.0010) [2023-12-27 07:47:31,452][20381] Updated weights for policy 1, policy_version 309922 (0.0011) [2023-12-27 07:47:31,850][20347] Updated weights for policy 0, policy_version 310118 (0.0008) [2023-12-27 07:47:31,893][20347] Updated weights for policy 0, policy_version 310128 (0.0008) [2023-12-27 07:47:31,947][20347] Updated weights for policy 0, policy_version 310138 (0.0006) [2023-12-27 07:47:32,131][20381] Updated weights for policy 1, policy_version 309932 (0.0010) [2023-12-27 07:47:32,179][20381] Updated weights for policy 1, policy_version 309942 (0.0009) [2023-12-27 07:47:32,229][20381] Updated weights for policy 1, policy_version 309952 (0.0006) [2023-12-27 07:47:32,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 158777344. Throughput: 0: 10211.2, 1: 10423.2. Samples: 158784212. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '780.490')] [2023-12-27 07:47:32,588][20347] Updated weights for policy 0, policy_version 310148 (0.0007) [2023-12-27 07:47:32,646][20347] Updated weights for policy 0, policy_version 310158 (0.0010) [2023-12-27 07:47:32,713][20347] Updated weights for policy 0, policy_version 310168 (0.0010) [2023-12-27 07:47:32,862][20381] Updated weights for policy 1, policy_version 309962 (0.0010) [2023-12-27 07:47:32,915][20381] Updated weights for policy 1, policy_version 309972 (0.0007) [2023-12-27 07:47:32,972][20381] Updated weights for policy 1, policy_version 309982 (0.0007) [2023-12-27 07:47:33,023][20381] Updated weights for policy 1, policy_version 309992 (0.0007) [2023-12-27 07:47:33,374][20347] Updated weights for policy 0, policy_version 310178 (0.0009) [2023-12-27 07:47:33,424][20347] Updated weights for policy 0, policy_version 310188 (0.0005) [2023-12-27 07:47:33,475][20347] Updated weights for policy 0, policy_version 310198 (0.0010) [2023-12-27 07:47:33,523][20347] Updated weights for policy 0, policy_version 310208 (0.0010) [2023-12-27 07:47:33,659][20381] Updated weights for policy 1, policy_version 310002 (0.0007) [2023-12-27 07:47:33,714][20381] Updated weights for policy 1, policy_version 310012 (0.0007) [2023-12-27 07:47:33,766][20381] Updated weights for policy 1, policy_version 310022 (0.0008) [2023-12-27 07:47:34,225][20347] Updated weights for policy 0, policy_version 310218 (0.0006) [2023-12-27 07:47:34,277][20347] Updated weights for policy 0, policy_version 310228 (0.0006) [2023-12-27 07:47:34,324][20347] Updated weights for policy 0, policy_version 310238 (0.0007) [2023-12-27 07:47:34,331][20381] Updated weights for policy 1, policy_version 310032 (0.0007) [2023-12-27 07:47:34,392][20381] Updated weights for policy 1, policy_version 310042 (0.0007) [2023-12-27 07:47:34,453][20381] Updated weights for policy 1, policy_version 310052 (0.0008) [2023-12-27 07:47:35,019][20347] Updated weights for policy 0, policy_version 310248 (0.0007) [2023-12-27 07:47:35,056][20381] Updated weights for policy 1, policy_version 310062 (0.0006) [2023-12-27 07:47:35,079][20347] Updated weights for policy 0, policy_version 310258 (0.0007) [2023-12-27 07:47:35,114][20381] Updated weights for policy 1, policy_version 310072 (0.0007) [2023-12-27 07:47:35,141][20347] Updated weights for policy 0, policy_version 310268 (0.0005) [2023-12-27 07:47:35,170][20381] Updated weights for policy 1, policy_version 310082 (0.0007) [2023-12-27 07:47:35,779][20381] Updated weights for policy 1, policy_version 310092 (0.0005) [2023-12-27 07:47:35,796][20347] Updated weights for policy 0, policy_version 310278 (0.0007) [2023-12-27 07:47:35,837][20381] Updated weights for policy 1, policy_version 310102 (0.0005) [2023-12-27 07:47:35,857][20347] Updated weights for policy 0, policy_version 310288 (0.0007) [2023-12-27 07:47:35,898][20381] Updated weights for policy 1, policy_version 310112 (0.0006) [2023-12-27 07:47:35,914][20347] Updated weights for policy 0, policy_version 310298 (0.0008) [2023-12-27 07:47:36,546][20381] Updated weights for policy 1, policy_version 310122 (0.0006) [2023-12-27 07:47:36,599][20381] Updated weights for policy 1, policy_version 310132 (0.0008) [2023-12-27 07:47:36,633][20347] Updated weights for policy 0, policy_version 310308 (0.0008) [2023-12-27 07:47:36,655][20381] Updated weights for policy 1, policy_version 310142 (0.0007) [2023-12-27 07:47:36,694][20347] Updated weights for policy 0, policy_version 310318 (0.0005) [2023-12-27 07:47:36,707][20381] Updated weights for policy 1, policy_version 310152 (0.0008) [2023-12-27 07:47:36,757][20347] Updated weights for policy 0, policy_version 310328 (0.0006) [2023-12-27 07:47:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 158875648. Throughput: 0: 10210.9, 1: 10502.4. Samples: 158848552. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '777.860')] [2023-12-27 07:47:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000310152_79413248.pth... [2023-12-27 07:47:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000310336_79462400.pth... [2023-12-27 07:47:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000309152_79159296.pth [2023-12-27 07:47:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000308936_79101952.pth [2023-12-27 07:47:37,393][20347] Updated weights for policy 0, policy_version 310338 (0.0007) [2023-12-27 07:47:37,453][20347] Updated weights for policy 0, policy_version 310348 (0.0006) [2023-12-27 07:47:37,463][20381] Updated weights for policy 1, policy_version 310162 (0.0007) [2023-12-27 07:47:37,508][20347] Updated weights for policy 0, policy_version 310358 (0.0006) [2023-12-27 07:47:37,525][20381] Updated weights for policy 1, policy_version 310172 (0.0008) [2023-12-27 07:47:37,566][20347] Updated weights for policy 0, policy_version 310368 (0.0006) [2023-12-27 07:47:37,583][20381] Updated weights for policy 1, policy_version 310182 (0.0007) [2023-12-27 07:47:38,189][20347] Updated weights for policy 0, policy_version 310378 (0.0005) [2023-12-27 07:47:38,244][20347] Updated weights for policy 0, policy_version 310388 (0.0006) [2023-12-27 07:47:38,280][20381] Updated weights for policy 1, policy_version 310192 (0.0008) [2023-12-27 07:47:38,293][20347] Updated weights for policy 0, policy_version 310398 (0.0005) [2023-12-27 07:47:38,336][20381] Updated weights for policy 1, policy_version 310202 (0.0006) [2023-12-27 07:47:38,395][20381] Updated weights for policy 1, policy_version 310212 (0.0007) [2023-12-27 07:47:38,898][20347] Updated weights for policy 0, policy_version 310408 (0.0006) [2023-12-27 07:47:38,957][20347] Updated weights for policy 0, policy_version 310418 (0.0006) [2023-12-27 07:47:39,020][20347] Updated weights for policy 0, policy_version 310428 (0.0006) [2023-12-27 07:47:39,038][20381] Updated weights for policy 1, policy_version 310222 (0.0006) [2023-12-27 07:47:39,094][20381] Updated weights for policy 1, policy_version 310232 (0.0009) [2023-12-27 07:47:39,142][20381] Updated weights for policy 1, policy_version 310242 (0.0008) [2023-12-27 07:47:39,703][20347] Updated weights for policy 0, policy_version 310438 (0.0007) [2023-12-27 07:47:39,762][20347] Updated weights for policy 0, policy_version 310448 (0.0007) [2023-12-27 07:47:39,787][20381] Updated weights for policy 1, policy_version 310252 (0.0008) [2023-12-27 07:47:39,827][20347] Updated weights for policy 0, policy_version 310458 (0.0006) [2023-12-27 07:47:39,857][20381] Updated weights for policy 1, policy_version 310262 (0.0007) [2023-12-27 07:47:39,910][20381] Updated weights for policy 1, policy_version 310272 (0.0007) [2023-12-27 07:47:40,550][20381] Updated weights for policy 1, policy_version 310282 (0.0008) [2023-12-27 07:47:40,585][20347] Updated weights for policy 0, policy_version 310468 (0.0007) [2023-12-27 07:47:40,602][20381] Updated weights for policy 1, policy_version 310292 (0.0011) [2023-12-27 07:47:40,647][20347] Updated weights for policy 0, policy_version 310478 (0.0007) [2023-12-27 07:47:40,665][20381] Updated weights for policy 1, policy_version 310302 (0.0011) [2023-12-27 07:47:40,707][20347] Updated weights for policy 0, policy_version 310488 (0.0010) [2023-12-27 07:47:40,722][20381] Updated weights for policy 1, policy_version 310312 (0.0011) [2023-12-27 07:47:41,414][20347] Updated weights for policy 0, policy_version 310498 (0.0010) [2023-12-27 07:47:41,464][20347] Updated weights for policy 0, policy_version 310508 (0.0007) [2023-12-27 07:47:41,492][20381] Updated weights for policy 1, policy_version 310322 (0.0008) [2023-12-27 07:47:41,520][20347] Updated weights for policy 0, policy_version 310518 (0.0008) [2023-12-27 07:47:41,545][20381] Updated weights for policy 1, policy_version 310332 (0.0007) [2023-12-27 07:47:41,574][20347] Updated weights for policy 0, policy_version 310528 (0.0008) [2023-12-27 07:47:41,610][20381] Updated weights for policy 1, policy_version 310342 (0.0006) [2023-12-27 07:47:42,210][20347] Updated weights for policy 0, policy_version 310538 (0.0006) [2023-12-27 07:47:42,273][20347] Updated weights for policy 0, policy_version 310548 (0.0007) [2023-12-27 07:47:42,330][20347] Updated weights for policy 0, policy_version 310558 (0.0008) [2023-12-27 07:47:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 158982144. Throughput: 0: 10243.3, 1: 10467.2. Samples: 158972132. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '859.450')] [2023-12-27 07:47:42,365][20381] Updated weights for policy 1, policy_version 310352 (0.0008) [2023-12-27 07:47:42,429][20381] Updated weights for policy 1, policy_version 310362 (0.0008) [2023-12-27 07:47:42,478][20381] Updated weights for policy 1, policy_version 310372 (0.0007) [2023-12-27 07:47:42,980][20347] Updated weights for policy 0, policy_version 310568 (0.0006) [2023-12-27 07:47:43,026][20347] Updated weights for policy 0, policy_version 310578 (0.0005) [2023-12-27 07:47:43,072][20347] Updated weights for policy 0, policy_version 310588 (0.0005) [2023-12-27 07:47:43,204][20381] Updated weights for policy 1, policy_version 310382 (0.0007) [2023-12-27 07:47:43,266][20381] Updated weights for policy 1, policy_version 310392 (0.0005) [2023-12-27 07:47:43,319][20381] Updated weights for policy 1, policy_version 310402 (0.0005) [2023-12-27 07:47:43,724][20347] Updated weights for policy 0, policy_version 310598 (0.0006) [2023-12-27 07:47:43,776][20347] Updated weights for policy 0, policy_version 310608 (0.0008) [2023-12-27 07:47:43,827][20347] Updated weights for policy 0, policy_version 310618 (0.0007) [2023-12-27 07:47:43,871][20381] Updated weights for policy 1, policy_version 310412 (0.0009) [2023-12-27 07:47:43,930][20381] Updated weights for policy 1, policy_version 310422 (0.0010) [2023-12-27 07:47:43,984][20381] Updated weights for policy 1, policy_version 310432 (0.0010) [2023-12-27 07:47:44,541][20347] Updated weights for policy 0, policy_version 310628 (0.0008) [2023-12-27 07:47:44,591][20347] Updated weights for policy 0, policy_version 310638 (0.0008) [2023-12-27 07:47:44,597][20381] Updated weights for policy 1, policy_version 310442 (0.0010) [2023-12-27 07:47:44,644][20347] Updated weights for policy 0, policy_version 310648 (0.0007) [2023-12-27 07:47:44,649][20381] Updated weights for policy 1, policy_version 310452 (0.0007) [2023-12-27 07:47:44,702][20381] Updated weights for policy 1, policy_version 310462 (0.0007) [2023-12-27 07:47:44,758][20381] Updated weights for policy 1, policy_version 310472 (0.0006) [2023-12-27 07:47:45,325][20347] Updated weights for policy 0, policy_version 310658 (0.0005) [2023-12-27 07:47:45,383][20347] Updated weights for policy 0, policy_version 310668 (0.0008) [2023-12-27 07:47:45,414][20381] Updated weights for policy 1, policy_version 310482 (0.0006) [2023-12-27 07:47:45,441][20347] Updated weights for policy 0, policy_version 310678 (0.0006) [2023-12-27 07:47:45,469][20381] Updated weights for policy 1, policy_version 310492 (0.0006) [2023-12-27 07:47:45,494][20347] Updated weights for policy 0, policy_version 310688 (0.0006) [2023-12-27 07:47:45,523][20381] Updated weights for policy 1, policy_version 310502 (0.0006) [2023-12-27 07:47:46,149][20381] Updated weights for policy 1, policy_version 310512 (0.0006) [2023-12-27 07:47:46,204][20381] Updated weights for policy 1, policy_version 310522 (0.0006) [2023-12-27 07:47:46,209][20347] Updated weights for policy 0, policy_version 310698 (0.0006) [2023-12-27 07:47:46,259][20347] Updated weights for policy 0, policy_version 310708 (0.0007) [2023-12-27 07:47:46,259][20381] Updated weights for policy 1, policy_version 310532 (0.0007) [2023-12-27 07:47:46,309][20347] Updated weights for policy 0, policy_version 310718 (0.0007) [2023-12-27 07:47:46,910][20381] Updated weights for policy 1, policy_version 310542 (0.0008) [2023-12-27 07:47:46,955][20347] Updated weights for policy 0, policy_version 310728 (0.0006) [2023-12-27 07:47:46,967][20381] Updated weights for policy 1, policy_version 310552 (0.0007) [2023-12-27 07:47:47,001][20347] Updated weights for policy 0, policy_version 310738 (0.0005) [2023-12-27 07:47:47,019][20381] Updated weights for policy 1, policy_version 310562 (0.0007) [2023-12-27 07:47:47,051][20347] Updated weights for policy 0, policy_version 310748 (0.0005) [2023-12-27 07:47:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 159088640. Throughput: 0: 10266.3, 1: 10449.3. Samples: 159097456. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '778.250')] [2023-12-27 07:47:47,628][20347] Updated weights for policy 0, policy_version 310758 (0.0008) [2023-12-27 07:47:47,659][20381] Updated weights for policy 1, policy_version 310572 (0.0006) [2023-12-27 07:47:47,682][20347] Updated weights for policy 0, policy_version 310768 (0.0008) [2023-12-27 07:47:47,712][20381] Updated weights for policy 1, policy_version 310582 (0.0008) [2023-12-27 07:47:47,735][20347] Updated weights for policy 0, policy_version 310778 (0.0006) [2023-12-27 07:47:47,768][20381] Updated weights for policy 1, policy_version 310592 (0.0006) [2023-12-27 07:47:48,376][20347] Updated weights for policy 0, policy_version 310788 (0.0008) [2023-12-27 07:47:48,444][20347] Updated weights for policy 0, policy_version 310798 (0.0008) [2023-12-27 07:47:48,454][20381] Updated weights for policy 1, policy_version 310602 (0.0006) [2023-12-27 07:47:48,501][20347] Updated weights for policy 0, policy_version 310808 (0.0009) [2023-12-27 07:47:48,517][20381] Updated weights for policy 1, policy_version 310612 (0.0007) [2023-12-27 07:47:48,575][20381] Updated weights for policy 1, policy_version 310622 (0.0007) [2023-12-27 07:47:48,633][20381] Updated weights for policy 1, policy_version 310632 (0.0007) [2023-12-27 07:47:49,169][20347] Updated weights for policy 0, policy_version 310818 (0.0010) [2023-12-27 07:47:49,221][20347] Updated weights for policy 0, policy_version 310828 (0.0009) [2023-12-27 07:47:49,247][20381] Updated weights for policy 1, policy_version 310642 (0.0007) [2023-12-27 07:47:49,285][20347] Updated weights for policy 0, policy_version 310838 (0.0006) [2023-12-27 07:47:49,303][20381] Updated weights for policy 1, policy_version 310652 (0.0008) [2023-12-27 07:47:49,344][20347] Updated weights for policy 0, policy_version 310848 (0.0010) [2023-12-27 07:47:49,361][20381] Updated weights for policy 1, policy_version 310662 (0.0008) [2023-12-27 07:47:50,034][20347] Updated weights for policy 0, policy_version 310858 (0.0010) [2023-12-27 07:47:50,057][20381] Updated weights for policy 1, policy_version 310672 (0.0007) [2023-12-27 07:47:50,095][20347] Updated weights for policy 0, policy_version 310868 (0.0007) [2023-12-27 07:47:50,117][20381] Updated weights for policy 1, policy_version 310682 (0.0008) [2023-12-27 07:47:50,157][20347] Updated weights for policy 0, policy_version 310878 (0.0008) [2023-12-27 07:47:50,174][20381] Updated weights for policy 1, policy_version 310692 (0.0007) [2023-12-27 07:47:50,793][20381] Updated weights for policy 1, policy_version 310702 (0.0007) [2023-12-27 07:47:50,842][20347] Updated weights for policy 0, policy_version 310888 (0.0011) [2023-12-27 07:47:50,846][20381] Updated weights for policy 1, policy_version 310712 (0.0005) [2023-12-27 07:47:50,899][20347] Updated weights for policy 0, policy_version 310898 (0.0011) [2023-12-27 07:47:50,903][20381] Updated weights for policy 1, policy_version 310722 (0.0006) [2023-12-27 07:47:50,951][20347] Updated weights for policy 0, policy_version 310908 (0.0010) [2023-12-27 07:47:51,542][20381] Updated weights for policy 1, policy_version 310732 (0.0006) [2023-12-27 07:47:51,591][20347] Updated weights for policy 0, policy_version 310918 (0.0008) [2023-12-27 07:47:51,603][20381] Updated weights for policy 1, policy_version 310742 (0.0009) [2023-12-27 07:47:51,657][20347] Updated weights for policy 0, policy_version 310928 (0.0009) [2023-12-27 07:47:51,672][20381] Updated weights for policy 1, policy_version 310752 (0.0008) [2023-12-27 07:47:51,721][20347] Updated weights for policy 0, policy_version 310938 (0.0008) [2023-12-27 07:47:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 159186944. Throughput: 0: 10267.0, 1: 10447.4. Samples: 159160348. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '744.730')] [2023-12-27 07:47:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000310944_79618048.pth... [2023-12-27 07:47:52,369][20381] Updated weights for policy 1, policy_version 310762 (0.0008) [2023-12-27 07:47:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000309760_79314944.pth [2023-12-27 07:47:52,409][20347] Updated weights for policy 0, policy_version 310948 (0.0008) [2023-12-27 07:47:52,428][20381] Updated weights for policy 1, policy_version 310772 (0.0005) [2023-12-27 07:47:52,474][20347] Updated weights for policy 0, policy_version 310958 (0.0010) [2023-12-27 07:47:52,487][20381] Updated weights for policy 1, policy_version 310782 (0.0006) [2023-12-27 07:47:52,526][20347] Updated weights for policy 0, policy_version 310968 (0.0010) [2023-12-27 07:47:52,539][20381] Updated weights for policy 1, policy_version 310792 (0.0006) [2023-12-27 07:47:52,539][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000310792_79577088.pth... [2023-12-27 07:47:52,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000309544_79257600.pth [2023-12-27 07:47:53,134][20347] Updated weights for policy 0, policy_version 310978 (0.0009) [2023-12-27 07:47:53,159][20381] Updated weights for policy 1, policy_version 310802 (0.0005) [2023-12-27 07:47:53,192][20347] Updated weights for policy 0, policy_version 310988 (0.0006) [2023-12-27 07:47:53,209][20381] Updated weights for policy 1, policy_version 310812 (0.0005) [2023-12-27 07:47:53,241][20347] Updated weights for policy 0, policy_version 310998 (0.0005) [2023-12-27 07:47:53,266][20381] Updated weights for policy 1, policy_version 310822 (0.0006) [2023-12-27 07:47:53,297][20347] Updated weights for policy 0, policy_version 311008 (0.0005) [2023-12-27 07:47:53,868][20347] Updated weights for policy 0, policy_version 311018 (0.0007) [2023-12-27 07:47:53,895][20381] Updated weights for policy 1, policy_version 310832 (0.0008) [2023-12-27 07:47:53,916][20347] Updated weights for policy 0, policy_version 311028 (0.0005) [2023-12-27 07:47:53,949][20381] Updated weights for policy 1, policy_version 310842 (0.0006) [2023-12-27 07:47:53,966][20347] Updated weights for policy 0, policy_version 311038 (0.0009) [2023-12-27 07:47:54,009][20381] Updated weights for policy 1, policy_version 310852 (0.0006) [2023-12-27 07:47:54,625][20347] Updated weights for policy 0, policy_version 311048 (0.0008) [2023-12-27 07:47:54,642][20381] Updated weights for policy 1, policy_version 310862 (0.0006) [2023-12-27 07:47:54,671][20347] Updated weights for policy 0, policy_version 311058 (0.0008) [2023-12-27 07:47:54,698][20381] Updated weights for policy 1, policy_version 310872 (0.0005) [2023-12-27 07:47:54,724][20347] Updated weights for policy 0, policy_version 311068 (0.0007) [2023-12-27 07:47:54,753][20381] Updated weights for policy 1, policy_version 310882 (0.0005) [2023-12-27 07:47:55,381][20381] Updated weights for policy 1, policy_version 310892 (0.0005) [2023-12-27 07:47:55,391][20347] Updated weights for policy 0, policy_version 311078 (0.0006) [2023-12-27 07:47:55,438][20381] Updated weights for policy 1, policy_version 310902 (0.0006) [2023-12-27 07:47:55,449][20347] Updated weights for policy 0, policy_version 311088 (0.0006) [2023-12-27 07:47:55,490][20381] Updated weights for policy 1, policy_version 310912 (0.0008) [2023-12-27 07:47:55,505][20347] Updated weights for policy 0, policy_version 311098 (0.0005) [2023-12-27 07:47:56,075][20347] Updated weights for policy 0, policy_version 311108 (0.0006) [2023-12-27 07:47:56,126][20347] Updated weights for policy 0, policy_version 311118 (0.0008) [2023-12-27 07:47:56,151][20381] Updated weights for policy 1, policy_version 310922 (0.0007) [2023-12-27 07:47:56,176][20347] Updated weights for policy 0, policy_version 311128 (0.0006) [2023-12-27 07:47:56,204][20381] Updated weights for policy 1, policy_version 310932 (0.0006) [2023-12-27 07:47:56,248][20381] Updated weights for policy 1, policy_version 310942 (0.0007) [2023-12-27 07:47:56,298][20381] Updated weights for policy 1, policy_version 310952 (0.0008) [2023-12-27 07:47:56,869][20347] Updated weights for policy 0, policy_version 311138 (0.0006) [2023-12-27 07:47:56,924][20347] Updated weights for policy 0, policy_version 311148 (0.0007) [2023-12-27 07:47:56,984][20347] Updated weights for policy 0, policy_version 311158 (0.0008) [2023-12-27 07:47:57,004][20381] Updated weights for policy 1, policy_version 310962 (0.0006) [2023-12-27 07:47:57,037][20347] Updated weights for policy 0, policy_version 311168 (0.0006) [2023-12-27 07:47:57,054][20381] Updated weights for policy 1, policy_version 310972 (0.0006) [2023-12-27 07:47:57,112][20381] Updated weights for policy 1, policy_version 310982 (0.0007) [2023-12-27 07:47:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 159301632. Throughput: 0: 10345.8, 1: 10507.9. Samples: 159289156. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:47:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.360')] [2023-12-27 07:47:57,622][20347] Updated weights for policy 0, policy_version 311178 (0.0005) [2023-12-27 07:47:57,677][20347] Updated weights for policy 0, policy_version 311188 (0.0006) [2023-12-27 07:47:57,726][20381] Updated weights for policy 1, policy_version 310992 (0.0006) [2023-12-27 07:47:57,733][20347] Updated weights for policy 0, policy_version 311198 (0.0008) [2023-12-27 07:47:57,775][20381] Updated weights for policy 1, policy_version 311002 (0.0007) [2023-12-27 07:47:57,820][20381] Updated weights for policy 1, policy_version 311012 (0.0008) [2023-12-27 07:47:58,346][20347] Updated weights for policy 0, policy_version 311208 (0.0008) [2023-12-27 07:47:58,411][20347] Updated weights for policy 0, policy_version 311218 (0.0007) [2023-12-27 07:47:58,476][20347] Updated weights for policy 0, policy_version 311228 (0.0008) [2023-12-27 07:47:58,569][20381] Updated weights for policy 1, policy_version 311022 (0.0007) [2023-12-27 07:47:58,627][20381] Updated weights for policy 1, policy_version 311032 (0.0008) [2023-12-27 07:47:58,684][20381] Updated weights for policy 1, policy_version 311042 (0.0007) [2023-12-27 07:47:59,197][20347] Updated weights for policy 0, policy_version 311238 (0.0006) [2023-12-27 07:47:59,255][20347] Updated weights for policy 0, policy_version 311248 (0.0007) [2023-12-27 07:47:59,306][20347] Updated weights for policy 0, policy_version 311258 (0.0006) [2023-12-27 07:47:59,372][20381] Updated weights for policy 1, policy_version 311052 (0.0008) [2023-12-27 07:47:59,428][20381] Updated weights for policy 1, policy_version 311062 (0.0007) [2023-12-27 07:47:59,472][20381] Updated weights for policy 1, policy_version 311072 (0.0007) [2023-12-27 07:47:59,960][20347] Updated weights for policy 0, policy_version 311268 (0.0007) [2023-12-27 07:48:00,016][20347] Updated weights for policy 0, policy_version 311278 (0.0008) [2023-12-27 07:48:00,068][20347] Updated weights for policy 0, policy_version 311288 (0.0008) [2023-12-27 07:48:00,192][20381] Updated weights for policy 1, policy_version 311082 (0.0007) [2023-12-27 07:48:00,247][20381] Updated weights for policy 1, policy_version 311092 (0.0008) [2023-12-27 07:48:00,296][20381] Updated weights for policy 1, policy_version 311102 (0.0008) [2023-12-27 07:48:00,348][20381] Updated weights for policy 1, policy_version 311112 (0.0007) [2023-12-27 07:48:00,786][20347] Updated weights for policy 0, policy_version 311298 (0.0008) [2023-12-27 07:48:00,832][20347] Updated weights for policy 0, policy_version 311308 (0.0008) [2023-12-27 07:48:00,890][20347] Updated weights for policy 0, policy_version 311318 (0.0010) [2023-12-27 07:48:00,947][20347] Updated weights for policy 0, policy_version 311328 (0.0010) [2023-12-27 07:48:01,051][20381] Updated weights for policy 1, policy_version 311122 (0.0007) [2023-12-27 07:48:01,107][20381] Updated weights for policy 1, policy_version 311132 (0.0008) [2023-12-27 07:48:01,161][20381] Updated weights for policy 1, policy_version 311142 (0.0008) [2023-12-27 07:48:01,683][20347] Updated weights for policy 0, policy_version 311338 (0.0006) [2023-12-27 07:48:01,752][20347] Updated weights for policy 0, policy_version 311348 (0.0007) [2023-12-27 07:48:01,802][20347] Updated weights for policy 0, policy_version 311358 (0.0007) [2023-12-27 07:48:01,879][20381] Updated weights for policy 1, policy_version 311152 (0.0008) [2023-12-27 07:48:01,930][20381] Updated weights for policy 1, policy_version 311162 (0.0008) [2023-12-27 07:48:01,980][20381] Updated weights for policy 1, policy_version 311172 (0.0008) [2023-12-27 07:48:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 159399936. Throughput: 0: 10358.5, 1: 10459.5. Samples: 159411272. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '747.850')] [2023-12-27 07:48:02,470][20347] Updated weights for policy 0, policy_version 311368 (0.0008) [2023-12-27 07:48:02,529][20347] Updated weights for policy 0, policy_version 311378 (0.0007) [2023-12-27 07:48:02,581][20347] Updated weights for policy 0, policy_version 311388 (0.0007) [2023-12-27 07:48:02,708][20381] Updated weights for policy 1, policy_version 311182 (0.0007) [2023-12-27 07:48:02,763][20381] Updated weights for policy 1, policy_version 311192 (0.0008) [2023-12-27 07:48:02,830][20381] Updated weights for policy 1, policy_version 311202 (0.0008) [2023-12-27 07:48:03,221][20347] Updated weights for policy 0, policy_version 311398 (0.0006) [2023-12-27 07:48:03,270][20347] Updated weights for policy 0, policy_version 311408 (0.0005) [2023-12-27 07:48:03,321][20347] Updated weights for policy 0, policy_version 311418 (0.0005) [2023-12-27 07:48:03,548][20381] Updated weights for policy 1, policy_version 311212 (0.0007) [2023-12-27 07:48:03,592][20381] Updated weights for policy 1, policy_version 311222 (0.0008) [2023-12-27 07:48:03,645][20381] Updated weights for policy 1, policy_version 311232 (0.0007) [2023-12-27 07:48:03,944][20347] Updated weights for policy 0, policy_version 311428 (0.0006) [2023-12-27 07:48:04,002][20347] Updated weights for policy 0, policy_version 311438 (0.0007) [2023-12-27 07:48:04,062][20347] Updated weights for policy 0, policy_version 311448 (0.0005) [2023-12-27 07:48:04,349][20381] Updated weights for policy 1, policy_version 311242 (0.0008) [2023-12-27 07:48:04,405][20381] Updated weights for policy 1, policy_version 311252 (0.0009) [2023-12-27 07:48:04,464][20381] Updated weights for policy 1, policy_version 311262 (0.0007) [2023-12-27 07:48:04,518][20381] Updated weights for policy 1, policy_version 311272 (0.0007) [2023-12-27 07:48:04,677][20347] Updated weights for policy 0, policy_version 311458 (0.0006) [2023-12-27 07:48:04,726][20347] Updated weights for policy 0, policy_version 311468 (0.0010) [2023-12-27 07:48:04,779][20347] Updated weights for policy 0, policy_version 311478 (0.0008) [2023-12-27 07:48:04,829][20347] Updated weights for policy 0, policy_version 311488 (0.0005) [2023-12-27 07:48:05,219][20381] Updated weights for policy 1, policy_version 311282 (0.0009) [2023-12-27 07:48:05,267][20381] Updated weights for policy 1, policy_version 311292 (0.0008) [2023-12-27 07:48:05,315][20381] Updated weights for policy 1, policy_version 311302 (0.0008) [2023-12-27 07:48:05,379][20347] Updated weights for policy 0, policy_version 311498 (0.0005) [2023-12-27 07:48:05,434][20347] Updated weights for policy 0, policy_version 311508 (0.0005) [2023-12-27 07:48:05,490][20347] Updated weights for policy 0, policy_version 311518 (0.0005) [2023-12-27 07:48:06,051][20381] Updated weights for policy 1, policy_version 311312 (0.0007) [2023-12-27 07:48:06,060][20347] Updated weights for policy 0, policy_version 311528 (0.0009) [2023-12-27 07:48:06,113][20381] Updated weights for policy 1, policy_version 311322 (0.0008) [2023-12-27 07:48:06,119][20347] Updated weights for policy 0, policy_version 311538 (0.0007) [2023-12-27 07:48:06,174][20347] Updated weights for policy 0, policy_version 311548 (0.0007) [2023-12-27 07:48:06,178][20381] Updated weights for policy 1, policy_version 311332 (0.0007) [2023-12-27 07:48:06,804][20347] Updated weights for policy 0, policy_version 311558 (0.0005) [2023-12-27 07:48:06,861][20347] Updated weights for policy 0, policy_version 311568 (0.0005) [2023-12-27 07:48:06,875][20381] Updated weights for policy 1, policy_version 311342 (0.0007) [2023-12-27 07:48:06,916][20347] Updated weights for policy 0, policy_version 311578 (0.0006) [2023-12-27 07:48:06,930][20381] Updated weights for policy 1, policy_version 311352 (0.0008) [2023-12-27 07:48:06,984][20381] Updated weights for policy 1, policy_version 311362 (0.0007) [2023-12-27 07:48:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 159506432. Throughput: 0: 10406.8, 1: 10465.9. Samples: 159474436. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '677.910')] [2023-12-27 07:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000311368_79724544.pth... [2023-12-27 07:48:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000311584_79781888.pth... [2023-12-27 07:48:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000310152_79413248.pth [2023-12-27 07:48:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000310336_79462400.pth [2023-12-27 07:48:07,513][20347] Updated weights for policy 0, policy_version 311588 (0.0006) [2023-12-27 07:48:07,572][20347] Updated weights for policy 0, policy_version 311598 (0.0008) [2023-12-27 07:48:07,636][20347] Updated weights for policy 0, policy_version 311608 (0.0007) [2023-12-27 07:48:07,709][20381] Updated weights for policy 1, policy_version 311372 (0.0006) [2023-12-27 07:48:07,768][20381] Updated weights for policy 1, policy_version 311382 (0.0006) [2023-12-27 07:48:07,833][20381] Updated weights for policy 1, policy_version 311392 (0.0005) [2023-12-27 07:48:08,312][20347] Updated weights for policy 0, policy_version 311618 (0.0008) [2023-12-27 07:48:08,370][20347] Updated weights for policy 0, policy_version 311628 (0.0006) [2023-12-27 07:48:08,419][20381] Updated weights for policy 1, policy_version 311402 (0.0006) [2023-12-27 07:48:08,429][20347] Updated weights for policy 0, policy_version 311638 (0.0006) [2023-12-27 07:48:08,473][20381] Updated weights for policy 1, policy_version 311412 (0.0006) [2023-12-27 07:48:08,486][20347] Updated weights for policy 0, policy_version 311648 (0.0006) [2023-12-27 07:48:08,527][20381] Updated weights for policy 1, policy_version 311422 (0.0007) [2023-12-27 07:48:08,583][20381] Updated weights for policy 1, policy_version 311432 (0.0007) [2023-12-27 07:48:09,158][20347] Updated weights for policy 0, policy_version 311658 (0.0008) [2023-12-27 07:48:09,217][20347] Updated weights for policy 0, policy_version 311668 (0.0008) [2023-12-27 07:48:09,276][20347] Updated weights for policy 0, policy_version 311678 (0.0007) [2023-12-27 07:48:09,292][20381] Updated weights for policy 1, policy_version 311442 (0.0007) [2023-12-27 07:48:09,350][20381] Updated weights for policy 1, policy_version 311452 (0.0008) [2023-12-27 07:48:09,413][20381] Updated weights for policy 1, policy_version 311462 (0.0007) [2023-12-27 07:48:10,004][20347] Updated weights for policy 0, policy_version 311688 (0.0008) [2023-12-27 07:48:10,066][20347] Updated weights for policy 0, policy_version 311698 (0.0008) [2023-12-27 07:48:10,130][20347] Updated weights for policy 0, policy_version 311708 (0.0008) [2023-12-27 07:48:10,142][20381] Updated weights for policy 1, policy_version 311472 (0.0007) [2023-12-27 07:48:10,201][20381] Updated weights for policy 1, policy_version 311482 (0.0009) [2023-12-27 07:48:10,259][20381] Updated weights for policy 1, policy_version 311492 (0.0008) [2023-12-27 07:48:10,823][20347] Updated weights for policy 0, policy_version 311718 (0.0010) [2023-12-27 07:48:10,881][20347] Updated weights for policy 0, policy_version 311728 (0.0010) [2023-12-27 07:48:10,940][20347] Updated weights for policy 0, policy_version 311738 (0.0011) [2023-12-27 07:48:10,949][20381] Updated weights for policy 1, policy_version 311502 (0.0006) [2023-12-27 07:48:10,996][20381] Updated weights for policy 1, policy_version 311512 (0.0006) [2023-12-27 07:48:11,049][20381] Updated weights for policy 1, policy_version 311522 (0.0007) [2023-12-27 07:48:11,680][20347] Updated weights for policy 0, policy_version 311748 (0.0010) [2023-12-27 07:48:11,747][20347] Updated weights for policy 0, policy_version 311758 (0.0009) [2023-12-27 07:48:11,807][20347] Updated weights for policy 0, policy_version 311768 (0.0008) [2023-12-27 07:48:11,848][20381] Updated weights for policy 1, policy_version 311532 (0.0008) [2023-12-27 07:48:11,910][20381] Updated weights for policy 1, policy_version 311542 (0.0007) [2023-12-27 07:48:11,966][20381] Updated weights for policy 1, policy_version 311552 (0.0008) [2023-12-27 07:48:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 159604736. Throughput: 0: 10468.1, 1: 10365.7. Samples: 159597064. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '727.830')] [2023-12-27 07:48:12,541][20347] Updated weights for policy 0, policy_version 311778 (0.0008) [2023-12-27 07:48:12,601][20347] Updated weights for policy 0, policy_version 311788 (0.0006) [2023-12-27 07:48:12,643][20381] Updated weights for policy 1, policy_version 311562 (0.0008) [2023-12-27 07:48:12,664][20347] Updated weights for policy 0, policy_version 311798 (0.0007) [2023-12-27 07:48:12,699][20381] Updated weights for policy 1, policy_version 311572 (0.0007) [2023-12-27 07:48:12,721][20347] Updated weights for policy 0, policy_version 311808 (0.0007) [2023-12-27 07:48:12,756][20381] Updated weights for policy 1, policy_version 311582 (0.0007) [2023-12-27 07:48:12,811][20381] Updated weights for policy 1, policy_version 311592 (0.0008) [2023-12-27 07:48:13,425][20381] Updated weights for policy 1, policy_version 311602 (0.0010) [2023-12-27 07:48:13,433][20347] Updated weights for policy 0, policy_version 311818 (0.0008) [2023-12-27 07:48:13,470][20381] Updated weights for policy 1, policy_version 311612 (0.0010) [2023-12-27 07:48:13,481][20347] Updated weights for policy 0, policy_version 311828 (0.0009) [2023-12-27 07:48:13,513][20381] Updated weights for policy 1, policy_version 311622 (0.0006) [2023-12-27 07:48:13,529][20347] Updated weights for policy 0, policy_version 311838 (0.0007) [2023-12-27 07:48:14,205][20381] Updated weights for policy 1, policy_version 311632 (0.0009) [2023-12-27 07:48:14,236][20347] Updated weights for policy 0, policy_version 311848 (0.0006) [2023-12-27 07:48:14,260][20381] Updated weights for policy 1, policy_version 311642 (0.0010) [2023-12-27 07:48:14,291][20347] Updated weights for policy 0, policy_version 311858 (0.0005) [2023-12-27 07:48:14,307][20381] Updated weights for policy 1, policy_version 311652 (0.0010) [2023-12-27 07:48:14,346][20347] Updated weights for policy 0, policy_version 311868 (0.0006) [2023-12-27 07:48:14,970][20381] Updated weights for policy 1, policy_version 311662 (0.0009) [2023-12-27 07:48:15,013][20347] Updated weights for policy 0, policy_version 311878 (0.0007) [2023-12-27 07:48:15,034][20381] Updated weights for policy 1, policy_version 311672 (0.0011) [2023-12-27 07:48:15,076][20347] Updated weights for policy 0, policy_version 311888 (0.0005) [2023-12-27 07:48:15,095][20381] Updated weights for policy 1, policy_version 311682 (0.0011) [2023-12-27 07:48:15,133][20347] Updated weights for policy 0, policy_version 311898 (0.0006) [2023-12-27 07:48:15,733][20347] Updated weights for policy 0, policy_version 311908 (0.0006) [2023-12-27 07:48:15,772][20381] Updated weights for policy 1, policy_version 311692 (0.0011) [2023-12-27 07:48:15,789][20347] Updated weights for policy 0, policy_version 311918 (0.0005) [2023-12-27 07:48:15,827][20381] Updated weights for policy 1, policy_version 311702 (0.0010) [2023-12-27 07:48:15,843][20347] Updated weights for policy 0, policy_version 311928 (0.0006) [2023-12-27 07:48:15,882][20381] Updated weights for policy 1, policy_version 311712 (0.0010) [2023-12-27 07:48:16,476][20347] Updated weights for policy 0, policy_version 311938 (0.0005) [2023-12-27 07:48:16,522][20347] Updated weights for policy 0, policy_version 311948 (0.0006) [2023-12-27 07:48:16,575][20347] Updated weights for policy 0, policy_version 311958 (0.0005) [2023-12-27 07:48:16,586][20381] Updated weights for policy 1, policy_version 311722 (0.0009) [2023-12-27 07:48:16,624][20347] Updated weights for policy 0, policy_version 311968 (0.0005) [2023-12-27 07:48:16,645][20381] Updated weights for policy 1, policy_version 311732 (0.0006) [2023-12-27 07:48:16,703][20381] Updated weights for policy 1, policy_version 311742 (0.0007) [2023-12-27 07:48:16,752][20381] Updated weights for policy 1, policy_version 311752 (0.0006) [2023-12-27 07:48:17,187][20347] Updated weights for policy 0, policy_version 311978 (0.0005) [2023-12-27 07:48:17,250][20347] Updated weights for policy 0, policy_version 311988 (0.0006) [2023-12-27 07:48:17,307][20347] Updated weights for policy 0, policy_version 311998 (0.0007) [2023-12-27 07:48:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 159711232. Throughput: 0: 10483.7, 1: 10326.2. Samples: 159720660. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '757.910')] [2023-12-27 07:48:17,404][20381] Updated weights for policy 1, policy_version 311762 (0.0011) [2023-12-27 07:48:17,459][20381] Updated weights for policy 1, policy_version 311772 (0.0008) [2023-12-27 07:48:17,508][20381] Updated weights for policy 1, policy_version 311782 (0.0008) [2023-12-27 07:48:17,965][20347] Updated weights for policy 0, policy_version 312008 (0.0008) [2023-12-27 07:48:18,026][20347] Updated weights for policy 0, policy_version 312018 (0.0008) [2023-12-27 07:48:18,082][20347] Updated weights for policy 0, policy_version 312028 (0.0007) [2023-12-27 07:48:18,151][20381] Updated weights for policy 1, policy_version 311792 (0.0010) [2023-12-27 07:48:18,203][20381] Updated weights for policy 1, policy_version 311802 (0.0011) [2023-12-27 07:48:18,256][20381] Updated weights for policy 1, policy_version 311812 (0.0011) [2023-12-27 07:48:18,737][20347] Updated weights for policy 0, policy_version 312038 (0.0008) [2023-12-27 07:48:18,790][20347] Updated weights for policy 0, policy_version 312048 (0.0006) [2023-12-27 07:48:18,848][20347] Updated weights for policy 0, policy_version 312058 (0.0008) [2023-12-27 07:48:19,006][20381] Updated weights for policy 1, policy_version 311822 (0.0011) [2023-12-27 07:48:19,065][20381] Updated weights for policy 1, policy_version 311832 (0.0011) [2023-12-27 07:48:19,124][20381] Updated weights for policy 1, policy_version 311842 (0.0011) [2023-12-27 07:48:19,551][20347] Updated weights for policy 0, policy_version 312068 (0.0007) [2023-12-27 07:48:19,609][20347] Updated weights for policy 0, policy_version 312078 (0.0006) [2023-12-27 07:48:19,665][20347] Updated weights for policy 0, policy_version 312088 (0.0006) [2023-12-27 07:48:19,859][20381] Updated weights for policy 1, policy_version 311852 (0.0009) [2023-12-27 07:48:19,948][20381] Updated weights for policy 1, policy_version 311862 (0.0007) [2023-12-27 07:48:19,997][20381] Updated weights for policy 1, policy_version 311872 (0.0007) [2023-12-27 07:48:20,357][20347] Updated weights for policy 0, policy_version 312098 (0.0006) [2023-12-27 07:48:20,414][20347] Updated weights for policy 0, policy_version 312108 (0.0008) [2023-12-27 07:48:20,476][20347] Updated weights for policy 0, policy_version 312118 (0.0008) [2023-12-27 07:48:20,539][20347] Updated weights for policy 0, policy_version 312128 (0.0008) [2023-12-27 07:48:20,692][20381] Updated weights for policy 1, policy_version 311882 (0.0008) [2023-12-27 07:48:20,754][20381] Updated weights for policy 1, policy_version 311892 (0.0008) [2023-12-27 07:48:20,816][20381] Updated weights for policy 1, policy_version 311902 (0.0006) [2023-12-27 07:48:20,880][20381] Updated weights for policy 1, policy_version 311912 (0.0006) [2023-12-27 07:48:21,189][20347] Updated weights for policy 0, policy_version 312138 (0.0008) [2023-12-27 07:48:21,248][20347] Updated weights for policy 0, policy_version 312148 (0.0008) [2023-12-27 07:48:21,311][20347] Updated weights for policy 0, policy_version 312158 (0.0007) [2023-12-27 07:48:21,526][20381] Updated weights for policy 1, policy_version 311922 (0.0008) [2023-12-27 07:48:21,576][20381] Updated weights for policy 1, policy_version 311932 (0.0008) [2023-12-27 07:48:21,638][20381] Updated weights for policy 1, policy_version 311942 (0.0008) [2023-12-27 07:48:21,988][20347] Updated weights for policy 0, policy_version 312168 (0.0006) [2023-12-27 07:48:22,053][20347] Updated weights for policy 0, policy_version 312178 (0.0010) [2023-12-27 07:48:22,119][20347] Updated weights for policy 0, policy_version 312188 (0.0006) [2023-12-27 07:48:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 159809536. Throughput: 0: 10474.5, 1: 10245.2. Samples: 159780940. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '736.550')] [2023-12-27 07:48:22,366][20381] Updated weights for policy 1, policy_version 311952 (0.0007) [2023-12-27 07:48:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000312192_79937536.pth... [2023-12-27 07:48:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000310944_79618048.pth [2023-12-27 07:48:22,427][20381] Updated weights for policy 1, policy_version 311962 (0.0007) [2023-12-27 07:48:22,492][20381] Updated weights for policy 1, policy_version 311972 (0.0007) [2023-12-27 07:48:22,515][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000311976_79880192.pth... [2023-12-27 07:48:22,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000310792_79577088.pth [2023-12-27 07:48:22,758][20347] Updated weights for policy 0, policy_version 312198 (0.0009) [2023-12-27 07:48:22,817][20347] Updated weights for policy 0, policy_version 312208 (0.0010) [2023-12-27 07:48:22,870][20347] Updated weights for policy 0, policy_version 312218 (0.0011) [2023-12-27 07:48:23,107][20381] Updated weights for policy 1, policy_version 311982 (0.0007) [2023-12-27 07:48:23,169][20381] Updated weights for policy 1, policy_version 311992 (0.0006) [2023-12-27 07:48:23,238][20381] Updated weights for policy 1, policy_version 312002 (0.0006) [2023-12-27 07:48:23,560][20347] Updated weights for policy 0, policy_version 312228 (0.0009) [2023-12-27 07:48:23,608][20347] Updated weights for policy 0, policy_version 312238 (0.0008) [2023-12-27 07:48:23,665][20347] Updated weights for policy 0, policy_version 312248 (0.0007) [2023-12-27 07:48:23,800][20381] Updated weights for policy 1, policy_version 312012 (0.0006) [2023-12-27 07:48:23,857][20381] Updated weights for policy 1, policy_version 312022 (0.0006) [2023-12-27 07:48:23,915][20381] Updated weights for policy 1, policy_version 312032 (0.0006) [2023-12-27 07:48:24,387][20347] Updated weights for policy 0, policy_version 312258 (0.0009) [2023-12-27 07:48:24,440][20347] Updated weights for policy 0, policy_version 312268 (0.0009) [2023-12-27 07:48:24,492][20381] Updated weights for policy 1, policy_version 312042 (0.0005) [2023-12-27 07:48:24,494][20347] Updated weights for policy 0, policy_version 312278 (0.0007) [2023-12-27 07:48:24,541][20381] Updated weights for policy 1, policy_version 312052 (0.0006) [2023-12-27 07:48:24,543][20347] Updated weights for policy 0, policy_version 312288 (0.0006) [2023-12-27 07:48:24,594][20381] Updated weights for policy 1, policy_version 312062 (0.0008) [2023-12-27 07:48:24,645][20381] Updated weights for policy 1, policy_version 312072 (0.0007) [2023-12-27 07:48:25,145][20347] Updated weights for policy 0, policy_version 312298 (0.0008) [2023-12-27 07:48:25,195][20347] Updated weights for policy 0, policy_version 312308 (0.0007) [2023-12-27 07:48:25,250][20347] Updated weights for policy 0, policy_version 312318 (0.0007) [2023-12-27 07:48:25,374][20381] Updated weights for policy 1, policy_version 312082 (0.0009) [2023-12-27 07:48:25,422][20381] Updated weights for policy 1, policy_version 312092 (0.0009) [2023-12-27 07:48:25,471][20381] Updated weights for policy 1, policy_version 312102 (0.0008) [2023-12-27 07:48:25,893][20347] Updated weights for policy 0, policy_version 312328 (0.0006) [2023-12-27 07:48:25,954][20347] Updated weights for policy 0, policy_version 312338 (0.0006) [2023-12-27 07:48:26,020][20347] Updated weights for policy 0, policy_version 312348 (0.0006) [2023-12-27 07:48:26,132][20381] Updated weights for policy 1, policy_version 312112 (0.0007) [2023-12-27 07:48:26,182][20381] Updated weights for policy 1, policy_version 312122 (0.0006) [2023-12-27 07:48:26,235][20381] Updated weights for policy 1, policy_version 312132 (0.0009) [2023-12-27 07:48:26,552][20347] Updated weights for policy 0, policy_version 312358 (0.0008) [2023-12-27 07:48:26,600][20347] Updated weights for policy 0, policy_version 312368 (0.0010) [2023-12-27 07:48:26,650][20347] Updated weights for policy 0, policy_version 312378 (0.0010) [2023-12-27 07:48:26,815][20381] Updated weights for policy 1, policy_version 312142 (0.0009) [2023-12-27 07:48:26,860][20381] Updated weights for policy 1, policy_version 312152 (0.0007) [2023-12-27 07:48:26,908][20381] Updated weights for policy 1, policy_version 312162 (0.0005) [2023-12-27 07:48:27,312][20347] Updated weights for policy 0, policy_version 312388 (0.0010) [2023-12-27 07:48:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 159916032. Throughput: 0: 10475.1, 1: 10270.9. Samples: 159905704. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '693.460')] [2023-12-27 07:48:27,370][20347] Updated weights for policy 0, policy_version 312398 (0.0006) [2023-12-27 07:48:27,417][20347] Updated weights for policy 0, policy_version 312408 (0.0005) [2023-12-27 07:48:27,540][20381] Updated weights for policy 1, policy_version 312172 (0.0007) [2023-12-27 07:48:27,590][20381] Updated weights for policy 1, policy_version 312182 (0.0010) [2023-12-27 07:48:27,635][20381] Updated weights for policy 1, policy_version 312192 (0.0010) [2023-12-27 07:48:28,029][20347] Updated weights for policy 0, policy_version 312418 (0.0006) [2023-12-27 07:48:28,080][20347] Updated weights for policy 0, policy_version 312428 (0.0006) [2023-12-27 07:48:28,130][20347] Updated weights for policy 0, policy_version 312438 (0.0007) [2023-12-27 07:48:28,194][20347] Updated weights for policy 0, policy_version 312448 (0.0007) [2023-12-27 07:48:28,332][20381] Updated weights for policy 1, policy_version 312202 (0.0010) [2023-12-27 07:48:28,390][20381] Updated weights for policy 1, policy_version 312212 (0.0008) [2023-12-27 07:48:28,443][20381] Updated weights for policy 1, policy_version 312222 (0.0009) [2023-12-27 07:48:28,499][20381] Updated weights for policy 1, policy_version 312232 (0.0011) [2023-12-27 07:48:28,812][20347] Updated weights for policy 0, policy_version 312458 (0.0008) [2023-12-27 07:48:28,879][20347] Updated weights for policy 0, policy_version 312468 (0.0008) [2023-12-27 07:48:28,940][20347] Updated weights for policy 0, policy_version 312478 (0.0008) [2023-12-27 07:48:29,143][20381] Updated weights for policy 1, policy_version 312242 (0.0005) [2023-12-27 07:48:29,208][20381] Updated weights for policy 1, policy_version 312252 (0.0006) [2023-12-27 07:48:29,266][20381] Updated weights for policy 1, policy_version 312262 (0.0007) [2023-12-27 07:48:29,592][20347] Updated weights for policy 0, policy_version 312488 (0.0008) [2023-12-27 07:48:29,649][20347] Updated weights for policy 0, policy_version 312498 (0.0008) [2023-12-27 07:48:29,717][20347] Updated weights for policy 0, policy_version 312508 (0.0010) [2023-12-27 07:48:29,845][20381] Updated weights for policy 1, policy_version 312272 (0.0006) [2023-12-27 07:48:29,902][20381] Updated weights for policy 1, policy_version 312282 (0.0007) [2023-12-27 07:48:29,964][20381] Updated weights for policy 1, policy_version 312292 (0.0007) [2023-12-27 07:48:30,402][20347] Updated weights for policy 0, policy_version 312518 (0.0009) [2023-12-27 07:48:30,448][20347] Updated weights for policy 0, policy_version 312528 (0.0007) [2023-12-27 07:48:30,502][20347] Updated weights for policy 0, policy_version 312538 (0.0008) [2023-12-27 07:48:30,570][20381] Updated weights for policy 1, policy_version 312302 (0.0006) [2023-12-27 07:48:30,622][20381] Updated weights for policy 1, policy_version 312312 (0.0005) [2023-12-27 07:48:30,683][20381] Updated weights for policy 1, policy_version 312322 (0.0006) [2023-12-27 07:48:31,156][20347] Updated weights for policy 0, policy_version 312548 (0.0008) [2023-12-27 07:48:31,214][20347] Updated weights for policy 0, policy_version 312558 (0.0008) [2023-12-27 07:48:31,273][20347] Updated weights for policy 0, policy_version 312568 (0.0008) [2023-12-27 07:48:31,335][20381] Updated weights for policy 1, policy_version 312332 (0.0008) [2023-12-27 07:48:31,395][20381] Updated weights for policy 1, policy_version 312342 (0.0007) [2023-12-27 07:48:31,447][20381] Updated weights for policy 1, policy_version 312352 (0.0005) [2023-12-27 07:48:31,976][20347] Updated weights for policy 0, policy_version 312578 (0.0006) [2023-12-27 07:48:32,029][20381] Updated weights for policy 1, policy_version 312362 (0.0006) [2023-12-27 07:48:32,031][20347] Updated weights for policy 0, policy_version 312588 (0.0005) [2023-12-27 07:48:32,087][20347] Updated weights for policy 0, policy_version 312598 (0.0005) [2023-12-27 07:48:32,092][20381] Updated weights for policy 1, policy_version 312372 (0.0008) [2023-12-27 07:48:32,142][20381] Updated weights for policy 1, policy_version 312382 (0.0006) [2023-12-27 07:48:32,144][20347] Updated weights for policy 0, policy_version 312608 (0.0006) [2023-12-27 07:48:32,191][20381] Updated weights for policy 1, policy_version 312392 (0.0008) [2023-12-27 07:48:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 160030720. Throughput: 0: 10532.6, 1: 10327.8. Samples: 160036176. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '712.860')] [2023-12-27 07:48:32,758][20347] Updated weights for policy 0, policy_version 312618 (0.0005) [2023-12-27 07:48:32,816][20347] Updated weights for policy 0, policy_version 312628 (0.0005) [2023-12-27 07:48:32,873][20347] Updated weights for policy 0, policy_version 312638 (0.0005) [2023-12-27 07:48:32,909][20381] Updated weights for policy 1, policy_version 312402 (0.0008) [2023-12-27 07:48:32,962][20381] Updated weights for policy 1, policy_version 312412 (0.0008) [2023-12-27 07:48:33,017][20381] Updated weights for policy 1, policy_version 312422 (0.0008) [2023-12-27 07:48:33,444][20347] Updated weights for policy 0, policy_version 312648 (0.0008) [2023-12-27 07:48:33,500][20347] Updated weights for policy 0, policy_version 312658 (0.0008) [2023-12-27 07:48:33,552][20347] Updated weights for policy 0, policy_version 312668 (0.0009) [2023-12-27 07:48:33,680][20381] Updated weights for policy 1, policy_version 312432 (0.0007) [2023-12-27 07:48:33,726][20381] Updated weights for policy 1, policy_version 312442 (0.0007) [2023-12-27 07:48:33,789][20381] Updated weights for policy 1, policy_version 312452 (0.0008) [2023-12-27 07:48:34,244][20347] Updated weights for policy 0, policy_version 312678 (0.0008) [2023-12-27 07:48:34,297][20347] Updated weights for policy 0, policy_version 312688 (0.0008) [2023-12-27 07:48:34,356][20347] Updated weights for policy 0, policy_version 312698 (0.0008) [2023-12-27 07:48:34,454][20381] Updated weights for policy 1, policy_version 312462 (0.0007) [2023-12-27 07:48:34,510][20381] Updated weights for policy 1, policy_version 312472 (0.0006) [2023-12-27 07:48:34,570][20381] Updated weights for policy 1, policy_version 312482 (0.0008) [2023-12-27 07:48:35,068][20347] Updated weights for policy 0, policy_version 312708 (0.0008) [2023-12-27 07:48:35,127][20347] Updated weights for policy 0, policy_version 312718 (0.0008) [2023-12-27 07:48:35,178][20347] Updated weights for policy 0, policy_version 312728 (0.0008) [2023-12-27 07:48:35,237][20381] Updated weights for policy 1, policy_version 312492 (0.0007) [2023-12-27 07:48:35,286][20381] Updated weights for policy 1, policy_version 312502 (0.0007) [2023-12-27 07:48:35,346][20381] Updated weights for policy 1, policy_version 312512 (0.0008) [2023-12-27 07:48:35,871][20347] Updated weights for policy 0, policy_version 312738 (0.0007) [2023-12-27 07:48:35,920][20347] Updated weights for policy 0, policy_version 312748 (0.0008) [2023-12-27 07:48:35,973][20347] Updated weights for policy 0, policy_version 312758 (0.0007) [2023-12-27 07:48:36,018][20381] Updated weights for policy 1, policy_version 312522 (0.0008) [2023-12-27 07:48:36,024][20347] Updated weights for policy 0, policy_version 312768 (0.0007) [2023-12-27 07:48:36,063][20381] Updated weights for policy 1, policy_version 312532 (0.0007) [2023-12-27 07:48:36,118][20381] Updated weights for policy 1, policy_version 312542 (0.0007) [2023-12-27 07:48:36,177][20381] Updated weights for policy 1, policy_version 312552 (0.0008) [2023-12-27 07:48:36,734][20347] Updated weights for policy 0, policy_version 312778 (0.0008) [2023-12-27 07:48:36,796][20347] Updated weights for policy 0, policy_version 312788 (0.0007) [2023-12-27 07:48:36,858][20347] Updated weights for policy 0, policy_version 312798 (0.0007) [2023-12-27 07:48:36,885][20381] Updated weights for policy 1, policy_version 312562 (0.0006) [2023-12-27 07:48:36,938][20381] Updated weights for policy 1, policy_version 312572 (0.0007) [2023-12-27 07:48:36,998][20381] Updated weights for policy 1, policy_version 312582 (0.0008) [2023-12-27 07:48:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 160129024. Throughput: 0: 10526.7, 1: 10330.6. Samples: 160098928. Policy #0 lag: (min: 14.0, avg: 15.8, max: 46.0) [2023-12-27 07:48:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '789.850')] [2023-12-27 07:48:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000312800_80093184.pth... [2023-12-27 07:48:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000312584_80035840.pth... [2023-12-27 07:48:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000311584_79781888.pth [2023-12-27 07:48:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000311368_79724544.pth [2023-12-27 07:48:37,554][20347] Updated weights for policy 0, policy_version 312808 (0.0009) [2023-12-27 07:48:37,607][20347] Updated weights for policy 0, policy_version 312818 (0.0005) [2023-12-27 07:48:37,666][20347] Updated weights for policy 0, policy_version 312828 (0.0006) [2023-12-27 07:48:37,744][20381] Updated weights for policy 1, policy_version 312592 (0.0008) [2023-12-27 07:48:37,793][20381] Updated weights for policy 1, policy_version 312602 (0.0008) [2023-12-27 07:48:37,852][20381] Updated weights for policy 1, policy_version 312612 (0.0010) [2023-12-27 07:48:38,269][20347] Updated weights for policy 0, policy_version 312838 (0.0007) [2023-12-27 07:48:38,331][20347] Updated weights for policy 0, policy_version 312848 (0.0008) [2023-12-27 07:48:38,392][20347] Updated weights for policy 0, policy_version 312858 (0.0008) [2023-12-27 07:48:38,478][20381] Updated weights for policy 1, policy_version 312622 (0.0011) [2023-12-27 07:48:38,531][20381] Updated weights for policy 1, policy_version 312632 (0.0011) [2023-12-27 07:48:38,593][20381] Updated weights for policy 1, policy_version 312642 (0.0011) [2023-12-27 07:48:39,083][20347] Updated weights for policy 0, policy_version 312868 (0.0009) [2023-12-27 07:48:39,144][20347] Updated weights for policy 0, policy_version 312878 (0.0010) [2023-12-27 07:48:39,205][20347] Updated weights for policy 0, policy_version 312888 (0.0011) [2023-12-27 07:48:39,271][20381] Updated weights for policy 1, policy_version 312652 (0.0009) [2023-12-27 07:48:39,335][20381] Updated weights for policy 1, policy_version 312662 (0.0008) [2023-12-27 07:48:39,401][20381] Updated weights for policy 1, policy_version 312672 (0.0007) [2023-12-27 07:48:40,028][20347] Updated weights for policy 0, policy_version 312898 (0.0011) [2023-12-27 07:48:40,100][20347] Updated weights for policy 0, policy_version 312908 (0.0010) [2023-12-27 07:48:40,131][20381] Updated weights for policy 1, policy_version 312682 (0.0008) [2023-12-27 07:48:40,151][20347] Updated weights for policy 0, policy_version 312918 (0.0010) [2023-12-27 07:48:40,190][20381] Updated weights for policy 1, policy_version 312692 (0.0005) [2023-12-27 07:48:40,204][20347] Updated weights for policy 0, policy_version 312928 (0.0010) [2023-12-27 07:48:40,248][20381] Updated weights for policy 1, policy_version 312702 (0.0005) [2023-12-27 07:48:40,304][20381] Updated weights for policy 1, policy_version 312712 (0.0005) [2023-12-27 07:48:40,870][20347] Updated weights for policy 0, policy_version 312938 (0.0010) [2023-12-27 07:48:40,872][20381] Updated weights for policy 1, policy_version 312722 (0.0005) [2023-12-27 07:48:40,922][20381] Updated weights for policy 1, policy_version 312732 (0.0005) [2023-12-27 07:48:40,938][20347] Updated weights for policy 0, policy_version 312948 (0.0010) [2023-12-27 07:48:40,974][20381] Updated weights for policy 1, policy_version 312742 (0.0006) [2023-12-27 07:48:40,999][20347] Updated weights for policy 0, policy_version 312958 (0.0010) [2023-12-27 07:48:41,652][20381] Updated weights for policy 1, policy_version 312752 (0.0007) [2023-12-27 07:48:41,689][20347] Updated weights for policy 0, policy_version 312968 (0.0008) [2023-12-27 07:48:41,716][20381] Updated weights for policy 1, policy_version 312762 (0.0007) [2023-12-27 07:48:41,753][20347] Updated weights for policy 0, policy_version 312978 (0.0010) [2023-12-27 07:48:41,781][20381] Updated weights for policy 1, policy_version 312772 (0.0008) [2023-12-27 07:48:41,807][20347] Updated weights for policy 0, policy_version 312988 (0.0006) [2023-12-27 07:48:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 160227328. Throughput: 0: 10435.1, 1: 10282.1. Samples: 160221432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:48:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '865.210')] [2023-12-27 07:48:42,429][20381] Updated weights for policy 1, policy_version 312782 (0.0006) [2023-12-27 07:48:42,483][20381] Updated weights for policy 1, policy_version 312792 (0.0007) [2023-12-27 07:48:42,521][20347] Updated weights for policy 0, policy_version 312998 (0.0006) [2023-12-27 07:48:42,542][20381] Updated weights for policy 1, policy_version 312802 (0.0006) [2023-12-27 07:48:42,579][20347] Updated weights for policy 0, policy_version 313008 (0.0007) [2023-12-27 07:48:42,638][20347] Updated weights for policy 0, policy_version 313018 (0.0008) [2023-12-27 07:48:43,178][20381] Updated weights for policy 1, policy_version 312812 (0.0006) [2023-12-27 07:48:43,241][20381] Updated weights for policy 1, policy_version 312822 (0.0005) [2023-12-27 07:48:43,257][20347] Updated weights for policy 0, policy_version 313028 (0.0008) [2023-12-27 07:48:43,301][20381] Updated weights for policy 1, policy_version 312832 (0.0005) [2023-12-27 07:48:43,316][20347] Updated weights for policy 0, policy_version 313038 (0.0005) [2023-12-27 07:48:43,382][20347] Updated weights for policy 0, policy_version 313048 (0.0005) [2023-12-27 07:48:43,947][20347] Updated weights for policy 0, policy_version 313058 (0.0005) [2023-12-27 07:48:43,960][20381] Updated weights for policy 1, policy_version 312842 (0.0005) [2023-12-27 07:48:43,998][20347] Updated weights for policy 0, policy_version 313068 (0.0005) [2023-12-27 07:48:44,017][20381] Updated weights for policy 1, policy_version 312852 (0.0007) [2023-12-27 07:48:44,055][20347] Updated weights for policy 0, policy_version 313078 (0.0006) [2023-12-27 07:48:44,079][20381] Updated weights for policy 1, policy_version 312862 (0.0007) [2023-12-27 07:48:44,111][20347] Updated weights for policy 0, policy_version 313088 (0.0005) [2023-12-27 07:48:44,137][20381] Updated weights for policy 1, policy_version 312872 (0.0008) [2023-12-27 07:48:44,686][20347] Updated weights for policy 0, policy_version 313098 (0.0006) [2023-12-27 07:48:44,742][20347] Updated weights for policy 0, policy_version 313108 (0.0007) [2023-12-27 07:48:44,804][20347] Updated weights for policy 0, policy_version 313118 (0.0007) [2023-12-27 07:48:44,832][20381] Updated weights for policy 1, policy_version 312882 (0.0007) [2023-12-27 07:48:44,890][20381] Updated weights for policy 1, policy_version 312892 (0.0009) [2023-12-27 07:48:44,940][20381] Updated weights for policy 1, policy_version 312902 (0.0008) [2023-12-27 07:48:45,475][20347] Updated weights for policy 0, policy_version 313128 (0.0005) [2023-12-27 07:48:45,534][20347] Updated weights for policy 0, policy_version 313138 (0.0006) [2023-12-27 07:48:45,594][20347] Updated weights for policy 0, policy_version 313148 (0.0007) [2023-12-27 07:48:45,666][20381] Updated weights for policy 1, policy_version 312912 (0.0008) [2023-12-27 07:48:45,726][20381] Updated weights for policy 1, policy_version 312922 (0.0008) [2023-12-27 07:48:45,784][20381] Updated weights for policy 1, policy_version 312932 (0.0008) [2023-12-27 07:48:46,240][20347] Updated weights for policy 0, policy_version 313158 (0.0007) [2023-12-27 07:48:46,289][20347] Updated weights for policy 0, policy_version 313168 (0.0007) [2023-12-27 07:48:46,341][20347] Updated weights for policy 0, policy_version 313178 (0.0007) [2023-12-27 07:48:46,456][20381] Updated weights for policy 1, policy_version 312942 (0.0007) [2023-12-27 07:48:46,511][20381] Updated weights for policy 1, policy_version 312952 (0.0007) [2023-12-27 07:48:46,568][20381] Updated weights for policy 1, policy_version 312962 (0.0007) [2023-12-27 07:48:47,017][20347] Updated weights for policy 0, policy_version 313188 (0.0007) [2023-12-27 07:48:47,078][20347] Updated weights for policy 0, policy_version 313198 (0.0005) [2023-12-27 07:48:47,132][20347] Updated weights for policy 0, policy_version 313208 (0.0005) [2023-12-27 07:48:47,224][20381] Updated weights for policy 1, policy_version 312972 (0.0009) [2023-12-27 07:48:47,286][20381] Updated weights for policy 1, policy_version 312982 (0.0006) [2023-12-27 07:48:47,335][20381] Updated weights for policy 1, policy_version 312992 (0.0007) [2023-12-27 07:48:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 160333824. Throughput: 0: 10486.3, 1: 10306.7. Samples: 160346956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:48:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '864.220')] [2023-12-27 07:48:47,686][20347] Updated weights for policy 0, policy_version 313218 (0.0006) [2023-12-27 07:48:47,750][20347] Updated weights for policy 0, policy_version 313228 (0.0007) [2023-12-27 07:48:47,797][20347] Updated weights for policy 0, policy_version 313238 (0.0007) [2023-12-27 07:48:47,853][20347] Updated weights for policy 0, policy_version 313248 (0.0007) [2023-12-27 07:48:47,925][20381] Updated weights for policy 1, policy_version 313002 (0.0007) [2023-12-27 07:48:47,971][20381] Updated weights for policy 1, policy_version 313012 (0.0008) [2023-12-27 07:48:48,024][20381] Updated weights for policy 1, policy_version 313022 (0.0009) [2023-12-27 07:48:48,072][20381] Updated weights for policy 1, policy_version 313032 (0.0007) [2023-12-27 07:48:48,602][20347] Updated weights for policy 0, policy_version 313258 (0.0008) [2023-12-27 07:48:48,662][20347] Updated weights for policy 0, policy_version 313268 (0.0006) [2023-12-27 07:48:48,668][20381] Updated weights for policy 1, policy_version 313042 (0.0006) [2023-12-27 07:48:48,719][20347] Updated weights for policy 0, policy_version 313278 (0.0007) [2023-12-27 07:48:48,721][20381] Updated weights for policy 1, policy_version 313052 (0.0006) [2023-12-27 07:48:48,773][20381] Updated weights for policy 1, policy_version 313062 (0.0007) [2023-12-27 07:48:49,401][20347] Updated weights for policy 0, policy_version 313288 (0.0009) [2023-12-27 07:48:49,456][20347] Updated weights for policy 0, policy_version 313298 (0.0008) [2023-12-27 07:48:49,475][20381] Updated weights for policy 1, policy_version 313072 (0.0007) [2023-12-27 07:48:49,517][20347] Updated weights for policy 0, policy_version 313308 (0.0008) [2023-12-27 07:48:49,532][20381] Updated weights for policy 1, policy_version 313082 (0.0007) [2023-12-27 07:48:49,598][20381] Updated weights for policy 1, policy_version 313092 (0.0008) [2023-12-27 07:48:50,227][20347] Updated weights for policy 0, policy_version 313318 (0.0007) [2023-12-27 07:48:50,273][20347] Updated weights for policy 0, policy_version 313328 (0.0008) [2023-12-27 07:48:50,309][20381] Updated weights for policy 1, policy_version 313102 (0.0007) [2023-12-27 07:48:50,331][20347] Updated weights for policy 0, policy_version 313338 (0.0007) [2023-12-27 07:48:50,374][20381] Updated weights for policy 1, policy_version 313112 (0.0007) [2023-12-27 07:48:50,430][20381] Updated weights for policy 1, policy_version 313122 (0.0008) [2023-12-27 07:48:51,007][20347] Updated weights for policy 0, policy_version 313348 (0.0008) [2023-12-27 07:48:51,065][20347] Updated weights for policy 0, policy_version 313358 (0.0007) [2023-12-27 07:48:51,086][20381] Updated weights for policy 1, policy_version 313132 (0.0007) [2023-12-27 07:48:51,120][20347] Updated weights for policy 0, policy_version 313368 (0.0007) [2023-12-27 07:48:51,143][20381] Updated weights for policy 1, policy_version 313142 (0.0006) [2023-12-27 07:48:51,194][20381] Updated weights for policy 1, policy_version 313152 (0.0008) [2023-12-27 07:48:51,817][20347] Updated weights for policy 0, policy_version 313378 (0.0007) [2023-12-27 07:48:51,873][20347] Updated weights for policy 0, policy_version 313388 (0.0006) [2023-12-27 07:48:51,937][20347] Updated weights for policy 0, policy_version 313398 (0.0006) [2023-12-27 07:48:51,951][20381] Updated weights for policy 1, policy_version 313162 (0.0008) [2023-12-27 07:48:51,994][20347] Updated weights for policy 0, policy_version 313408 (0.0006) [2023-12-27 07:48:52,007][20381] Updated weights for policy 1, policy_version 313172 (0.0006) [2023-12-27 07:48:52,065][20381] Updated weights for policy 1, policy_version 313182 (0.0008) [2023-12-27 07:48:52,123][20381] Updated weights for policy 1, policy_version 313192 (0.0010) [2023-12-27 07:48:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 160440320. Throughput: 0: 10428.4, 1: 10345.6. Samples: 160409272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:48:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.240')] [2023-12-27 07:48:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000313192_80191488.pth... [2023-12-27 07:48:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000313408_80248832.pth... [2023-12-27 07:48:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000312192_79937536.pth [2023-12-27 07:48:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000311976_79880192.pth [2023-12-27 07:48:52,668][20347] Updated weights for policy 0, policy_version 313418 (0.0007) [2023-12-27 07:48:52,729][20347] Updated weights for policy 0, policy_version 313428 (0.0007) [2023-12-27 07:48:52,783][20347] Updated weights for policy 0, policy_version 313438 (0.0008) [2023-12-27 07:48:52,806][20381] Updated weights for policy 1, policy_version 313202 (0.0007) [2023-12-27 07:48:52,866][20381] Updated weights for policy 1, policy_version 313212 (0.0005) [2023-12-27 07:48:52,923][20381] Updated weights for policy 1, policy_version 313222 (0.0005) [2023-12-27 07:48:53,394][20347] Updated weights for policy 0, policy_version 313448 (0.0006) [2023-12-27 07:48:53,447][20347] Updated weights for policy 0, policy_version 313458 (0.0009) [2023-12-27 07:48:53,496][20347] Updated weights for policy 0, policy_version 313468 (0.0007) [2023-12-27 07:48:53,512][20381] Updated weights for policy 1, policy_version 313232 (0.0005) [2023-12-27 07:48:53,567][20381] Updated weights for policy 1, policy_version 313242 (0.0007) [2023-12-27 07:48:53,622][20381] Updated weights for policy 1, policy_version 313252 (0.0010) [2023-12-27 07:48:54,177][20347] Updated weights for policy 0, policy_version 313478 (0.0007) [2023-12-27 07:48:54,232][20347] Updated weights for policy 0, policy_version 313488 (0.0007) [2023-12-27 07:48:54,264][20381] Updated weights for policy 1, policy_version 313262 (0.0008) [2023-12-27 07:48:54,286][20347] Updated weights for policy 0, policy_version 313498 (0.0007) [2023-12-27 07:48:54,315][20381] Updated weights for policy 1, policy_version 313272 (0.0007) [2023-12-27 07:48:54,360][20381] Updated weights for policy 1, policy_version 313282 (0.0007) [2023-12-27 07:48:55,007][20347] Updated weights for policy 0, policy_version 313508 (0.0007) [2023-12-27 07:48:55,054][20381] Updated weights for policy 1, policy_version 313292 (0.0007) [2023-12-27 07:48:55,077][20347] Updated weights for policy 0, policy_version 313518 (0.0007) [2023-12-27 07:48:55,110][20381] Updated weights for policy 1, policy_version 313302 (0.0007) [2023-12-27 07:48:55,136][20347] Updated weights for policy 0, policy_version 313528 (0.0009) [2023-12-27 07:48:55,167][20381] Updated weights for policy 1, policy_version 313312 (0.0007) [2023-12-27 07:48:55,811][20347] Updated weights for policy 0, policy_version 313538 (0.0006) [2023-12-27 07:48:55,857][20381] Updated weights for policy 1, policy_version 313322 (0.0008) [2023-12-27 07:48:55,874][20347] Updated weights for policy 0, policy_version 313548 (0.0008) [2023-12-27 07:48:55,918][20381] Updated weights for policy 1, policy_version 313332 (0.0010) [2023-12-27 07:48:55,927][20347] Updated weights for policy 0, policy_version 313558 (0.0007) [2023-12-27 07:48:55,973][20381] Updated weights for policy 1, policy_version 313342 (0.0010) [2023-12-27 07:48:55,978][20347] Updated weights for policy 0, policy_version 313568 (0.0005) [2023-12-27 07:48:56,024][20381] Updated weights for policy 1, policy_version 313352 (0.0010) [2023-12-27 07:48:56,674][20347] Updated weights for policy 0, policy_version 313578 (0.0008) [2023-12-27 07:48:56,701][20381] Updated weights for policy 1, policy_version 313362 (0.0010) [2023-12-27 07:48:56,722][20347] Updated weights for policy 0, policy_version 313588 (0.0006) [2023-12-27 07:48:56,758][20381] Updated weights for policy 1, policy_version 313372 (0.0010) [2023-12-27 07:48:56,774][20347] Updated weights for policy 0, policy_version 313598 (0.0007) [2023-12-27 07:48:56,805][20381] Updated weights for policy 1, policy_version 313382 (0.0010) [2023-12-27 07:48:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 160538624. Throughput: 0: 10388.0, 1: 10391.6. Samples: 160532144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '868.800')] [2023-12-27 07:48:57,476][20381] Updated weights for policy 1, policy_version 313392 (0.0006) [2023-12-27 07:48:57,482][20347] Updated weights for policy 0, policy_version 313608 (0.0008) [2023-12-27 07:48:57,525][20381] Updated weights for policy 1, policy_version 313402 (0.0008) [2023-12-27 07:48:57,531][20347] Updated weights for policy 0, policy_version 313618 (0.0005) [2023-12-27 07:48:57,570][20381] Updated weights for policy 1, policy_version 313412 (0.0010) [2023-12-27 07:48:57,591][20347] Updated weights for policy 0, policy_version 313628 (0.0006) [2023-12-27 07:48:58,192][20381] Updated weights for policy 1, policy_version 313422 (0.0009) [2023-12-27 07:48:58,246][20381] Updated weights for policy 1, policy_version 313432 (0.0007) [2023-12-27 07:48:58,304][20381] Updated weights for policy 1, policy_version 313442 (0.0007) [2023-12-27 07:48:58,308][20347] Updated weights for policy 0, policy_version 313638 (0.0008) [2023-12-27 07:48:58,372][20347] Updated weights for policy 0, policy_version 313648 (0.0007) [2023-12-27 07:48:58,425][20347] Updated weights for policy 0, policy_version 313658 (0.0007) [2023-12-27 07:48:59,069][20381] Updated weights for policy 1, policy_version 313452 (0.0007) [2023-12-27 07:48:59,122][20381] Updated weights for policy 1, policy_version 313462 (0.0008) [2023-12-27 07:48:59,175][20347] Updated weights for policy 0, policy_version 313668 (0.0008) [2023-12-27 07:48:59,179][20381] Updated weights for policy 1, policy_version 313472 (0.0007) [2023-12-27 07:48:59,253][20347] Updated weights for policy 0, policy_version 313678 (0.0008) [2023-12-27 07:48:59,314][20347] Updated weights for policy 0, policy_version 313688 (0.0008) [2023-12-27 07:48:59,894][20381] Updated weights for policy 1, policy_version 313482 (0.0007) [2023-12-27 07:48:59,953][20381] Updated weights for policy 1, policy_version 313492 (0.0007) [2023-12-27 07:49:00,010][20347] Updated weights for policy 0, policy_version 313698 (0.0008) [2023-12-27 07:49:00,011][20381] Updated weights for policy 1, policy_version 313502 (0.0006) [2023-12-27 07:49:00,069][20381] Updated weights for policy 1, policy_version 313512 (0.0007) [2023-12-27 07:49:00,074][20347] Updated weights for policy 0, policy_version 313708 (0.0005) [2023-12-27 07:49:00,130][20347] Updated weights for policy 0, policy_version 313718 (0.0008) [2023-12-27 07:49:00,190][20347] Updated weights for policy 0, policy_version 313728 (0.0008) [2023-12-27 07:49:00,666][20381] Updated weights for policy 1, policy_version 313522 (0.0006) [2023-12-27 07:49:00,722][20381] Updated weights for policy 1, policy_version 313532 (0.0005) [2023-12-27 07:49:00,777][20381] Updated weights for policy 1, policy_version 313542 (0.0005) [2023-12-27 07:49:00,908][20347] Updated weights for policy 0, policy_version 313738 (0.0008) [2023-12-27 07:49:00,965][20347] Updated weights for policy 0, policy_version 313748 (0.0006) [2023-12-27 07:49:01,028][20347] Updated weights for policy 0, policy_version 313758 (0.0006) [2023-12-27 07:49:01,351][20381] Updated weights for policy 1, policy_version 313552 (0.0010) [2023-12-27 07:49:01,414][20381] Updated weights for policy 1, policy_version 313562 (0.0011) [2023-12-27 07:49:01,472][20381] Updated weights for policy 1, policy_version 313572 (0.0011) [2023-12-27 07:49:01,653][20347] Updated weights for policy 0, policy_version 313768 (0.0007) [2023-12-27 07:49:01,711][20347] Updated weights for policy 0, policy_version 313778 (0.0005) [2023-12-27 07:49:01,770][20347] Updated weights for policy 0, policy_version 313788 (0.0008) [2023-12-27 07:49:02,161][20381] Updated weights for policy 1, policy_version 313582 (0.0011) [2023-12-27 07:49:02,214][20381] Updated weights for policy 1, policy_version 313592 (0.0007) [2023-12-27 07:49:02,271][20381] Updated weights for policy 1, policy_version 313602 (0.0007) [2023-12-27 07:49:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 160645120. Throughput: 0: 10317.5, 1: 10423.7. Samples: 160654016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '851.900')] [2023-12-27 07:49:02,488][20347] Updated weights for policy 0, policy_version 313798 (0.0008) [2023-12-27 07:49:02,541][20347] Updated weights for policy 0, policy_version 313808 (0.0008) [2023-12-27 07:49:02,590][20347] Updated weights for policy 0, policy_version 313818 (0.0008) [2023-12-27 07:49:02,917][20381] Updated weights for policy 1, policy_version 313612 (0.0008) [2023-12-27 07:49:02,974][20381] Updated weights for policy 1, policy_version 313622 (0.0010) [2023-12-27 07:49:03,025][20381] Updated weights for policy 1, policy_version 313632 (0.0010) [2023-12-27 07:49:03,284][20347] Updated weights for policy 0, policy_version 313828 (0.0006) [2023-12-27 07:49:03,336][20347] Updated weights for policy 0, policy_version 313838 (0.0007) [2023-12-27 07:49:03,392][20347] Updated weights for policy 0, policy_version 313848 (0.0007) [2023-12-27 07:49:03,703][20381] Updated weights for policy 1, policy_version 313642 (0.0007) [2023-12-27 07:49:03,748][20381] Updated weights for policy 1, policy_version 313652 (0.0010) [2023-12-27 07:49:03,793][20381] Updated weights for policy 1, policy_version 313662 (0.0010) [2023-12-27 07:49:03,851][20381] Updated weights for policy 1, policy_version 313672 (0.0011) [2023-12-27 07:49:04,006][20347] Updated weights for policy 0, policy_version 313858 (0.0007) [2023-12-27 07:49:04,067][20347] Updated weights for policy 0, policy_version 313868 (0.0007) [2023-12-27 07:49:04,130][20347] Updated weights for policy 0, policy_version 313878 (0.0007) [2023-12-27 07:49:04,193][20347] Updated weights for policy 0, policy_version 313888 (0.0007) [2023-12-27 07:49:04,548][20381] Updated weights for policy 1, policy_version 313682 (0.0008) [2023-12-27 07:49:04,596][20381] Updated weights for policy 1, policy_version 313692 (0.0005) [2023-12-27 07:49:04,642][20381] Updated weights for policy 1, policy_version 313702 (0.0007) [2023-12-27 07:49:04,804][20347] Updated weights for policy 0, policy_version 313898 (0.0007) [2023-12-27 07:49:04,854][20347] Updated weights for policy 0, policy_version 313908 (0.0006) [2023-12-27 07:49:04,907][20347] Updated weights for policy 0, policy_version 313918 (0.0005) [2023-12-27 07:49:05,338][20381] Updated weights for policy 1, policy_version 313712 (0.0011) [2023-12-27 07:49:05,397][20381] Updated weights for policy 1, policy_version 313722 (0.0010) [2023-12-27 07:49:05,454][20381] Updated weights for policy 1, policy_version 313732 (0.0010) [2023-12-27 07:49:05,473][20347] Updated weights for policy 0, policy_version 313928 (0.0007) [2023-12-27 07:49:05,528][20347] Updated weights for policy 0, policy_version 313938 (0.0005) [2023-12-27 07:49:05,577][20347] Updated weights for policy 0, policy_version 313948 (0.0005) [2023-12-27 07:49:06,033][20381] Updated weights for policy 1, policy_version 313742 (0.0009) [2023-12-27 07:49:06,085][20381] Updated weights for policy 1, policy_version 313752 (0.0010) [2023-12-27 07:49:06,147][20381] Updated weights for policy 1, policy_version 313762 (0.0010) [2023-12-27 07:49:06,216][20347] Updated weights for policy 0, policy_version 313958 (0.0009) [2023-12-27 07:49:06,272][20347] Updated weights for policy 0, policy_version 313968 (0.0011) [2023-12-27 07:49:06,324][20347] Updated weights for policy 0, policy_version 313978 (0.0010) [2023-12-27 07:49:06,845][20381] Updated weights for policy 1, policy_version 313772 (0.0010) [2023-12-27 07:49:06,898][20381] Updated weights for policy 1, policy_version 313782 (0.0010) [2023-12-27 07:49:06,950][20381] Updated weights for policy 1, policy_version 313792 (0.0011) [2023-12-27 07:49:06,966][20347] Updated weights for policy 0, policy_version 313988 (0.0009) [2023-12-27 07:49:07,024][20347] Updated weights for policy 0, policy_version 313998 (0.0008) [2023-12-27 07:49:07,086][20347] Updated weights for policy 0, policy_version 314008 (0.0007) [2023-12-27 07:49:07,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 160751616. Throughput: 0: 10345.1, 1: 10454.0. Samples: 160716904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '910.110')] [2023-12-27 07:49:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000313800_80347136.pth... [2023-12-27 07:49:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000314016_80404480.pth... [2023-12-27 07:49:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000312584_80035840.pth [2023-12-27 07:49:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000312800_80093184.pth [2023-12-27 07:49:07,568][20381] Updated weights for policy 1, policy_version 313802 (0.0010) [2023-12-27 07:49:07,630][20381] Updated weights for policy 1, policy_version 313812 (0.0011) [2023-12-27 07:49:07,683][20381] Updated weights for policy 1, policy_version 313822 (0.0008) [2023-12-27 07:49:07,746][20381] Updated weights for policy 1, policy_version 313832 (0.0006) [2023-12-27 07:49:07,747][20347] Updated weights for policy 0, policy_version 314018 (0.0007) [2023-12-27 07:49:07,802][20347] Updated weights for policy 0, policy_version 314028 (0.0006) [2023-12-27 07:49:07,856][20347] Updated weights for policy 0, policy_version 314038 (0.0005) [2023-12-27 07:49:07,908][20347] Updated weights for policy 0, policy_version 314048 (0.0007) [2023-12-27 07:49:08,361][20381] Updated weights for policy 1, policy_version 313842 (0.0011) [2023-12-27 07:49:08,412][20381] Updated weights for policy 1, policy_version 313852 (0.0009) [2023-12-27 07:49:08,470][20381] Updated weights for policy 1, policy_version 313862 (0.0009) [2023-12-27 07:49:08,581][20347] Updated weights for policy 0, policy_version 314058 (0.0011) [2023-12-27 07:49:08,639][20347] Updated weights for policy 0, policy_version 314068 (0.0010) [2023-12-27 07:49:08,694][20347] Updated weights for policy 0, policy_version 314078 (0.0010) [2023-12-27 07:49:09,128][20381] Updated weights for policy 1, policy_version 313872 (0.0006) [2023-12-27 07:49:09,192][20381] Updated weights for policy 1, policy_version 313882 (0.0005) [2023-12-27 07:49:09,257][20381] Updated weights for policy 1, policy_version 313892 (0.0010) [2023-12-27 07:49:09,336][20347] Updated weights for policy 0, policy_version 314088 (0.0008) [2023-12-27 07:49:09,397][20347] Updated weights for policy 0, policy_version 314098 (0.0007) [2023-12-27 07:49:09,459][20347] Updated weights for policy 0, policy_version 314108 (0.0007) [2023-12-27 07:49:09,928][20381] Updated weights for policy 1, policy_version 313902 (0.0010) [2023-12-27 07:49:09,991][20381] Updated weights for policy 1, policy_version 313912 (0.0007) [2023-12-27 07:49:10,063][20381] Updated weights for policy 1, policy_version 313922 (0.0006) [2023-12-27 07:49:10,201][20347] Updated weights for policy 0, policy_version 314118 (0.0007) [2023-12-27 07:49:10,265][20347] Updated weights for policy 0, policy_version 314128 (0.0007) [2023-12-27 07:49:10,330][20347] Updated weights for policy 0, policy_version 314138 (0.0007) [2023-12-27 07:49:10,656][20381] Updated weights for policy 1, policy_version 313932 (0.0006) [2023-12-27 07:49:10,714][20381] Updated weights for policy 1, policy_version 313942 (0.0008) [2023-12-27 07:49:10,774][20381] Updated weights for policy 1, policy_version 313952 (0.0008) [2023-12-27 07:49:11,002][20347] Updated weights for policy 0, policy_version 314148 (0.0007) [2023-12-27 07:49:11,066][20347] Updated weights for policy 0, policy_version 314158 (0.0008) [2023-12-27 07:49:11,124][20347] Updated weights for policy 0, policy_version 314168 (0.0008) [2023-12-27 07:49:11,453][20381] Updated weights for policy 1, policy_version 313962 (0.0007) [2023-12-27 07:49:11,503][20381] Updated weights for policy 1, policy_version 313972 (0.0011) [2023-12-27 07:49:11,560][20381] Updated weights for policy 1, policy_version 313982 (0.0010) [2023-12-27 07:49:11,622][20381] Updated weights for policy 1, policy_version 313992 (0.0011) [2023-12-27 07:49:11,872][20347] Updated weights for policy 0, policy_version 314178 (0.0007) [2023-12-27 07:49:11,929][20347] Updated weights for policy 0, policy_version 314188 (0.0005) [2023-12-27 07:49:11,980][20347] Updated weights for policy 0, policy_version 314198 (0.0005) [2023-12-27 07:49:12,030][20347] Updated weights for policy 0, policy_version 314208 (0.0006) [2023-12-27 07:49:12,333][20381] Updated weights for policy 1, policy_version 314002 (0.0011) [2023-12-27 07:49:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 160849920. Throughput: 0: 10373.0, 1: 10478.6. Samples: 160844024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '836.240')] [2023-12-27 07:49:12,398][20381] Updated weights for policy 1, policy_version 314012 (0.0009) [2023-12-27 07:49:12,453][20381] Updated weights for policy 1, policy_version 314022 (0.0010) [2023-12-27 07:49:12,728][20347] Updated weights for policy 0, policy_version 314218 (0.0007) [2023-12-27 07:49:12,789][20347] Updated weights for policy 0, policy_version 314228 (0.0007) [2023-12-27 07:49:12,839][20347] Updated weights for policy 0, policy_version 314238 (0.0007) [2023-12-27 07:49:13,167][20381] Updated weights for policy 1, policy_version 314032 (0.0009) [2023-12-27 07:49:13,220][20381] Updated weights for policy 1, policy_version 314042 (0.0008) [2023-12-27 07:49:13,274][20381] Updated weights for policy 1, policy_version 314052 (0.0008) [2023-12-27 07:49:13,531][20347] Updated weights for policy 0, policy_version 314248 (0.0008) [2023-12-27 07:49:13,587][20347] Updated weights for policy 0, policy_version 314258 (0.0008) [2023-12-27 07:49:13,640][20347] Updated weights for policy 0, policy_version 314268 (0.0007) [2023-12-27 07:49:13,898][20381] Updated weights for policy 1, policy_version 314062 (0.0008) [2023-12-27 07:49:13,946][20381] Updated weights for policy 1, policy_version 314072 (0.0006) [2023-12-27 07:49:14,001][20381] Updated weights for policy 1, policy_version 314082 (0.0005) [2023-12-27 07:49:14,333][20347] Updated weights for policy 0, policy_version 314278 (0.0006) [2023-12-27 07:49:14,380][20347] Updated weights for policy 0, policy_version 314288 (0.0006) [2023-12-27 07:49:14,423][20347] Updated weights for policy 0, policy_version 314298 (0.0007) [2023-12-27 07:49:14,622][20381] Updated weights for policy 1, policy_version 314092 (0.0007) [2023-12-27 07:49:14,678][20381] Updated weights for policy 1, policy_version 314102 (0.0007) [2023-12-27 07:49:14,736][20381] Updated weights for policy 1, policy_version 314112 (0.0007) [2023-12-27 07:49:15,132][20347] Updated weights for policy 0, policy_version 314308 (0.0007) [2023-12-27 07:49:15,188][20347] Updated weights for policy 0, policy_version 314318 (0.0008) [2023-12-27 07:49:15,236][20347] Updated weights for policy 0, policy_version 314328 (0.0007) [2023-12-27 07:49:15,433][20381] Updated weights for policy 1, policy_version 314122 (0.0008) [2023-12-27 07:49:15,488][20381] Updated weights for policy 1, policy_version 314132 (0.0008) [2023-12-27 07:49:15,544][20381] Updated weights for policy 1, policy_version 314142 (0.0008) [2023-12-27 07:49:15,601][20381] Updated weights for policy 1, policy_version 314152 (0.0008) [2023-12-27 07:49:15,941][20347] Updated weights for policy 0, policy_version 314338 (0.0008) [2023-12-27 07:49:15,992][20347] Updated weights for policy 0, policy_version 314348 (0.0009) [2023-12-27 07:49:16,047][20347] Updated weights for policy 0, policy_version 314358 (0.0007) [2023-12-27 07:49:16,104][20347] Updated weights for policy 0, policy_version 314368 (0.0008) [2023-12-27 07:49:16,294][20381] Updated weights for policy 1, policy_version 314162 (0.0010) [2023-12-27 07:49:16,344][20381] Updated weights for policy 1, policy_version 314172 (0.0010) [2023-12-27 07:49:16,401][20381] Updated weights for policy 1, policy_version 314182 (0.0010) [2023-12-27 07:49:16,794][20347] Updated weights for policy 0, policy_version 314378 (0.0011) [2023-12-27 07:49:16,852][20347] Updated weights for policy 0, policy_version 314388 (0.0010) [2023-12-27 07:49:16,901][20347] Updated weights for policy 0, policy_version 314398 (0.0010) [2023-12-27 07:49:17,082][20381] Updated weights for policy 1, policy_version 314192 (0.0008) [2023-12-27 07:49:17,129][20381] Updated weights for policy 1, policy_version 314202 (0.0006) [2023-12-27 07:49:17,179][20381] Updated weights for policy 1, policy_version 314212 (0.0007) [2023-12-27 07:49:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 160956416. Throughput: 0: 10263.7, 1: 10369.1. Samples: 160964652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.900')] [2023-12-27 07:49:17,477][20347] Updated weights for policy 0, policy_version 314408 (0.0008) [2023-12-27 07:49:17,537][20347] Updated weights for policy 0, policy_version 314418 (0.0010) [2023-12-27 07:49:17,598][20347] Updated weights for policy 0, policy_version 314428 (0.0010) [2023-12-27 07:49:17,768][20381] Updated weights for policy 1, policy_version 314222 (0.0007) [2023-12-27 07:49:17,822][20381] Updated weights for policy 1, policy_version 314232 (0.0007) [2023-12-27 07:49:17,874][20381] Updated weights for policy 1, policy_version 314242 (0.0007) [2023-12-27 07:49:18,255][20347] Updated weights for policy 0, policy_version 314438 (0.0010) [2023-12-27 07:49:18,315][20347] Updated weights for policy 0, policy_version 314448 (0.0010) [2023-12-27 07:49:18,377][20347] Updated weights for policy 0, policy_version 314458 (0.0010) [2023-12-27 07:49:18,526][20381] Updated weights for policy 1, policy_version 314252 (0.0007) [2023-12-27 07:49:18,574][20381] Updated weights for policy 1, policy_version 314262 (0.0008) [2023-12-27 07:49:18,625][20381] Updated weights for policy 1, policy_version 314272 (0.0008) [2023-12-27 07:49:18,989][20347] Updated weights for policy 0, policy_version 314468 (0.0009) [2023-12-27 07:49:19,049][20347] Updated weights for policy 0, policy_version 314478 (0.0010) [2023-12-27 07:49:19,109][20347] Updated weights for policy 0, policy_version 314488 (0.0010) [2023-12-27 07:49:19,341][20381] Updated weights for policy 1, policy_version 314282 (0.0007) [2023-12-27 07:49:19,406][20381] Updated weights for policy 1, policy_version 314292 (0.0007) [2023-12-27 07:49:19,460][20381] Updated weights for policy 1, policy_version 314302 (0.0008) [2023-12-27 07:49:19,519][20381] Updated weights for policy 1, policy_version 314312 (0.0008) [2023-12-27 07:49:19,848][20347] Updated weights for policy 0, policy_version 314498 (0.0010) [2023-12-27 07:49:19,903][20347] Updated weights for policy 0, policy_version 314508 (0.0011) [2023-12-27 07:49:19,960][20347] Updated weights for policy 0, policy_version 314518 (0.0011) [2023-12-27 07:49:20,013][20347] Updated weights for policy 0, policy_version 314528 (0.0010) [2023-12-27 07:49:20,254][20381] Updated weights for policy 1, policy_version 314322 (0.0007) [2023-12-27 07:49:20,307][20381] Updated weights for policy 1, policy_version 314332 (0.0007) [2023-12-27 07:49:20,366][20381] Updated weights for policy 1, policy_version 314342 (0.0008) [2023-12-27 07:49:20,733][20347] Updated weights for policy 0, policy_version 314538 (0.0010) [2023-12-27 07:49:20,791][20347] Updated weights for policy 0, policy_version 314548 (0.0010) [2023-12-27 07:49:20,853][20347] Updated weights for policy 0, policy_version 314558 (0.0010) [2023-12-27 07:49:21,088][20381] Updated weights for policy 1, policy_version 314352 (0.0010) [2023-12-27 07:49:21,157][20381] Updated weights for policy 1, policy_version 314362 (0.0010) [2023-12-27 07:49:21,209][20381] Updated weights for policy 1, policy_version 314372 (0.0008) [2023-12-27 07:49:21,581][20347] Updated weights for policy 0, policy_version 314568 (0.0008) [2023-12-27 07:49:21,650][20347] Updated weights for policy 0, policy_version 314578 (0.0008) [2023-12-27 07:49:21,716][20347] Updated weights for policy 0, policy_version 314588 (0.0009) [2023-12-27 07:49:21,930][20381] Updated weights for policy 1, policy_version 314382 (0.0006) [2023-12-27 07:49:21,992][20381] Updated weights for policy 1, policy_version 314392 (0.0008) [2023-12-27 07:49:22,054][20381] Updated weights for policy 1, policy_version 314402 (0.0011) [2023-12-27 07:49:22,361][20347] Updated weights for policy 0, policy_version 314598 (0.0008) [2023-12-27 07:49:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 161054720. Throughput: 0: 10260.6, 1: 10375.5. Samples: 161027552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '904.970')] [2023-12-27 07:49:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000314408_80502784.pth... [2023-12-27 07:49:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000313192_80191488.pth [2023-12-27 07:49:22,412][20347] Updated weights for policy 0, policy_version 314608 (0.0007) [2023-12-27 07:49:22,471][20347] Updated weights for policy 0, policy_version 314618 (0.0007) [2023-12-27 07:49:22,501][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000314624_80560128.pth... [2023-12-27 07:49:22,504][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000313408_80248832.pth [2023-12-27 07:49:22,704][20381] Updated weights for policy 1, policy_version 314412 (0.0009) [2023-12-27 07:49:22,770][20381] Updated weights for policy 1, policy_version 314422 (0.0007) [2023-12-27 07:49:22,828][20381] Updated weights for policy 1, policy_version 314432 (0.0006) [2023-12-27 07:49:23,212][20347] Updated weights for policy 0, policy_version 314628 (0.0007) [2023-12-27 07:49:23,267][20347] Updated weights for policy 0, policy_version 314638 (0.0005) [2023-12-27 07:49:23,316][20347] Updated weights for policy 0, policy_version 314648 (0.0007) [2023-12-27 07:49:23,469][20381] Updated weights for policy 1, policy_version 314442 (0.0011) [2023-12-27 07:49:23,518][20381] Updated weights for policy 1, policy_version 314452 (0.0011) [2023-12-27 07:49:23,564][20381] Updated weights for policy 1, policy_version 314462 (0.0010) [2023-12-27 07:49:23,616][20381] Updated weights for policy 1, policy_version 314472 (0.0010) [2023-12-27 07:49:23,975][20347] Updated weights for policy 0, policy_version 314658 (0.0007) [2023-12-27 07:49:24,027][20347] Updated weights for policy 0, policy_version 314668 (0.0007) [2023-12-27 07:49:24,084][20347] Updated weights for policy 0, policy_version 314678 (0.0007) [2023-12-27 07:49:24,143][20347] Updated weights for policy 0, policy_version 314688 (0.0006) [2023-12-27 07:49:24,358][20381] Updated weights for policy 1, policy_version 314482 (0.0011) [2023-12-27 07:49:24,413][20381] Updated weights for policy 1, policy_version 314492 (0.0011) [2023-12-27 07:49:24,468][20381] Updated weights for policy 1, policy_version 314502 (0.0010) [2023-12-27 07:49:24,826][20347] Updated weights for policy 0, policy_version 314698 (0.0007) [2023-12-27 07:49:24,882][20347] Updated weights for policy 0, policy_version 314708 (0.0007) [2023-12-27 07:49:24,928][20347] Updated weights for policy 0, policy_version 314718 (0.0007) [2023-12-27 07:49:25,116][20381] Updated weights for policy 1, policy_version 314512 (0.0010) [2023-12-27 07:49:25,174][20381] Updated weights for policy 1, policy_version 314522 (0.0010) [2023-12-27 07:49:25,225][20381] Updated weights for policy 1, policy_version 314532 (0.0010) [2023-12-27 07:49:25,617][20347] Updated weights for policy 0, policy_version 314728 (0.0008) [2023-12-27 07:49:25,678][20347] Updated weights for policy 0, policy_version 314738 (0.0008) [2023-12-27 07:49:25,736][20347] Updated weights for policy 0, policy_version 314748 (0.0008) [2023-12-27 07:49:25,907][20381] Updated weights for policy 1, policy_version 314542 (0.0011) [2023-12-27 07:49:25,957][20381] Updated weights for policy 1, policy_version 314552 (0.0010) [2023-12-27 07:49:26,013][20381] Updated weights for policy 1, policy_version 314562 (0.0011) [2023-12-27 07:49:26,437][20347] Updated weights for policy 0, policy_version 314758 (0.0006) [2023-12-27 07:49:26,491][20347] Updated weights for policy 0, policy_version 314768 (0.0005) [2023-12-27 07:49:26,537][20347] Updated weights for policy 0, policy_version 314778 (0.0005) [2023-12-27 07:49:26,722][20381] Updated weights for policy 1, policy_version 314572 (0.0011) [2023-12-27 07:49:26,778][20381] Updated weights for policy 1, policy_version 314582 (0.0011) [2023-12-27 07:49:26,833][20381] Updated weights for policy 1, policy_version 314592 (0.0010) [2023-12-27 07:49:27,180][20347] Updated weights for policy 0, policy_version 314788 (0.0007) [2023-12-27 07:49:27,227][20347] Updated weights for policy 0, policy_version 314798 (0.0010) [2023-12-27 07:49:27,278][20347] Updated weights for policy 0, policy_version 314808 (0.0009) [2023-12-27 07:49:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 161161216. Throughput: 0: 10249.5, 1: 10334.7. Samples: 161147720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '791.790')] [2023-12-27 07:49:27,525][20381] Updated weights for policy 1, policy_version 314602 (0.0010) [2023-12-27 07:49:27,573][20381] Updated weights for policy 1, policy_version 314612 (0.0010) [2023-12-27 07:49:27,621][20381] Updated weights for policy 1, policy_version 314622 (0.0010) [2023-12-27 07:49:27,665][20381] Updated weights for policy 1, policy_version 314632 (0.0010) [2023-12-27 07:49:27,888][20347] Updated weights for policy 0, policy_version 314818 (0.0007) [2023-12-27 07:49:27,942][20347] Updated weights for policy 0, policy_version 314828 (0.0007) [2023-12-27 07:49:27,990][20347] Updated weights for policy 0, policy_version 314838 (0.0005) [2023-12-27 07:49:28,042][20347] Updated weights for policy 0, policy_version 314848 (0.0007) [2023-12-27 07:49:28,364][20381] Updated weights for policy 1, policy_version 314642 (0.0011) [2023-12-27 07:49:28,423][20381] Updated weights for policy 1, policy_version 314652 (0.0011) [2023-12-27 07:49:28,479][20381] Updated weights for policy 1, policy_version 314662 (0.0010) [2023-12-27 07:49:28,620][20347] Updated weights for policy 0, policy_version 314858 (0.0005) [2023-12-27 07:49:28,672][20347] Updated weights for policy 0, policy_version 314868 (0.0005) [2023-12-27 07:49:28,727][20347] Updated weights for policy 0, policy_version 314878 (0.0006) [2023-12-27 07:49:29,164][20381] Updated weights for policy 1, policy_version 314672 (0.0011) [2023-12-27 07:49:29,217][20381] Updated weights for policy 1, policy_version 314682 (0.0010) [2023-12-27 07:49:29,275][20381] Updated weights for policy 1, policy_version 314692 (0.0011) [2023-12-27 07:49:29,370][20347] Updated weights for policy 0, policy_version 314888 (0.0007) [2023-12-27 07:49:29,427][20347] Updated weights for policy 0, policy_version 314898 (0.0007) [2023-12-27 07:49:29,481][20347] Updated weights for policy 0, policy_version 314908 (0.0006) [2023-12-27 07:49:29,920][20381] Updated weights for policy 1, policy_version 314702 (0.0010) [2023-12-27 07:49:29,983][20381] Updated weights for policy 1, policy_version 314712 (0.0008) [2023-12-27 07:49:30,042][20381] Updated weights for policy 1, policy_version 314722 (0.0009) [2023-12-27 07:49:30,135][20347] Updated weights for policy 0, policy_version 314918 (0.0006) [2023-12-27 07:49:30,190][20347] Updated weights for policy 0, policy_version 314928 (0.0005) [2023-12-27 07:49:30,249][20347] Updated weights for policy 0, policy_version 314938 (0.0005) [2023-12-27 07:49:30,709][20381] Updated weights for policy 1, policy_version 314732 (0.0011) [2023-12-27 07:49:30,764][20381] Updated weights for policy 1, policy_version 314742 (0.0010) [2023-12-27 07:49:30,817][20381] Updated weights for policy 1, policy_version 314752 (0.0010) [2023-12-27 07:49:30,851][20347] Updated weights for policy 0, policy_version 314948 (0.0006) [2023-12-27 07:49:30,898][20347] Updated weights for policy 0, policy_version 314958 (0.0007) [2023-12-27 07:49:30,947][20347] Updated weights for policy 0, policy_version 314968 (0.0010) [2023-12-27 07:49:31,529][20381] Updated weights for policy 1, policy_version 314762 (0.0010) [2023-12-27 07:49:31,577][20381] Updated weights for policy 1, policy_version 314772 (0.0009) [2023-12-27 07:49:31,640][20381] Updated weights for policy 1, policy_version 314782 (0.0010) [2023-12-27 07:49:31,696][20381] Updated weights for policy 1, policy_version 314792 (0.0010) [2023-12-27 07:49:31,702][20347] Updated weights for policy 0, policy_version 314978 (0.0009) [2023-12-27 07:49:31,762][20347] Updated weights for policy 0, policy_version 314988 (0.0008) [2023-12-27 07:49:31,813][20347] Updated weights for policy 0, policy_version 314998 (0.0006) [2023-12-27 07:49:31,860][20347] Updated weights for policy 0, policy_version 315008 (0.0007) [2023-12-27 07:49:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 161259520. Throughput: 0: 10248.6, 1: 10336.6. Samples: 161273292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '887.350')] [2023-12-27 07:49:32,388][20381] Updated weights for policy 1, policy_version 314802 (0.0007) [2023-12-27 07:49:32,445][20381] Updated weights for policy 1, policy_version 314812 (0.0006) [2023-12-27 07:49:32,499][20381] Updated weights for policy 1, policy_version 314822 (0.0007) [2023-12-27 07:49:32,550][20347] Updated weights for policy 0, policy_version 315018 (0.0007) [2023-12-27 07:49:32,597][20347] Updated weights for policy 0, policy_version 315028 (0.0007) [2023-12-27 07:49:32,642][20347] Updated weights for policy 0, policy_version 315038 (0.0008) [2023-12-27 07:49:33,101][20381] Updated weights for policy 1, policy_version 314832 (0.0006) [2023-12-27 07:49:33,162][20381] Updated weights for policy 1, policy_version 314842 (0.0008) [2023-12-27 07:49:33,213][20381] Updated weights for policy 1, policy_version 314852 (0.0007) [2023-12-27 07:49:33,381][20347] Updated weights for policy 0, policy_version 315048 (0.0009) [2023-12-27 07:49:33,431][20347] Updated weights for policy 0, policy_version 315058 (0.0007) [2023-12-27 07:49:33,488][20347] Updated weights for policy 0, policy_version 315068 (0.0007) [2023-12-27 07:49:33,817][20381] Updated weights for policy 1, policy_version 314862 (0.0006) [2023-12-27 07:49:33,877][20381] Updated weights for policy 1, policy_version 314872 (0.0007) [2023-12-27 07:49:33,931][20381] Updated weights for policy 1, policy_version 314882 (0.0008) [2023-12-27 07:49:34,122][20347] Updated weights for policy 0, policy_version 315078 (0.0005) [2023-12-27 07:49:34,182][20347] Updated weights for policy 0, policy_version 315088 (0.0008) [2023-12-27 07:49:34,236][20347] Updated weights for policy 0, policy_version 315098 (0.0005) [2023-12-27 07:49:34,557][20381] Updated weights for policy 1, policy_version 314892 (0.0007) [2023-12-27 07:49:34,611][20381] Updated weights for policy 1, policy_version 314902 (0.0008) [2023-12-27 07:49:34,658][20381] Updated weights for policy 1, policy_version 314912 (0.0008) [2023-12-27 07:49:34,850][20347] Updated weights for policy 0, policy_version 315108 (0.0005) [2023-12-27 07:49:34,904][20347] Updated weights for policy 0, policy_version 315118 (0.0005) [2023-12-27 07:49:34,958][20347] Updated weights for policy 0, policy_version 315128 (0.0005) [2023-12-27 07:49:35,366][20381] Updated weights for policy 1, policy_version 314922 (0.0007) [2023-12-27 07:49:35,421][20381] Updated weights for policy 1, policy_version 314932 (0.0008) [2023-12-27 07:49:35,476][20381] Updated weights for policy 1, policy_version 314942 (0.0008) [2023-12-27 07:49:35,486][20347] Updated weights for policy 0, policy_version 315138 (0.0005) [2023-12-27 07:49:35,532][20347] Updated weights for policy 0, policy_version 315148 (0.0005) [2023-12-27 07:49:35,534][20381] Updated weights for policy 1, policy_version 314952 (0.0007) [2023-12-27 07:49:35,586][20347] Updated weights for policy 0, policy_version 315158 (0.0005) [2023-12-27 07:49:35,639][20347] Updated weights for policy 0, policy_version 315168 (0.0005) [2023-12-27 07:49:36,138][20381] Updated weights for policy 1, policy_version 314962 (0.0008) [2023-12-27 07:49:36,190][20381] Updated weights for policy 1, policy_version 314972 (0.0008) [2023-12-27 07:49:36,238][20381] Updated weights for policy 1, policy_version 314982 (0.0008) [2023-12-27 07:49:36,314][20347] Updated weights for policy 0, policy_version 315178 (0.0006) [2023-12-27 07:49:36,372][20347] Updated weights for policy 0, policy_version 315188 (0.0006) [2023-12-27 07:49:36,426][20347] Updated weights for policy 0, policy_version 315198 (0.0006) [2023-12-27 07:49:36,935][20381] Updated weights for policy 1, policy_version 314992 (0.0006) [2023-12-27 07:49:36,988][20381] Updated weights for policy 1, policy_version 315002 (0.0006) [2023-12-27 07:49:37,038][20347] Updated weights for policy 0, policy_version 315208 (0.0009) [2023-12-27 07:49:37,039][20381] Updated weights for policy 1, policy_version 315012 (0.0008) [2023-12-27 07:49:37,094][20347] Updated weights for policy 0, policy_version 315218 (0.0007) [2023-12-27 07:49:37,156][20347] Updated weights for policy 0, policy_version 315228 (0.0006) [2023-12-27 07:49:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 161374208. Throughput: 0: 10277.2, 1: 10360.6. Samples: 161337968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '964.750')] [2023-12-27 07:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000315016_80658432.pth... [2023-12-27 07:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000315232_80715776.pth... [2023-12-27 07:49:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000314016_80404480.pth [2023-12-27 07:49:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000313800_80347136.pth [2023-12-27 07:49:37,677][20381] Updated weights for policy 1, policy_version 315022 (0.0009) [2023-12-27 07:49:37,713][20292] KL-divergence is very high: 123.8906 [2023-12-27 07:49:37,718][20292] KL-divergence is very high: 130.1172 [2023-12-27 07:49:37,723][20292] KL-divergence is very high: 115.8059 [2023-12-27 07:49:37,732][20381] Updated weights for policy 1, policy_version 315032 (0.0008) [2023-12-27 07:49:37,737][20292] KL-divergence is very high: 123.3088 [2023-12-27 07:49:37,786][20381] Updated weights for policy 1, policy_version 315042 (0.0007) [2023-12-27 07:49:37,832][20347] Updated weights for policy 0, policy_version 315238 (0.0007) [2023-12-27 07:49:37,889][20347] Updated weights for policy 0, policy_version 315248 (0.0008) [2023-12-27 07:49:37,945][20347] Updated weights for policy 0, policy_version 315258 (0.0008) [2023-12-27 07:49:38,531][20347] Updated weights for policy 0, policy_version 315268 (0.0008) [2023-12-27 07:49:38,532][20381] Updated weights for policy 1, policy_version 315052 (0.0007) [2023-12-27 07:49:38,564][20292] KL-divergence is very high: 132.8591 [2023-12-27 07:49:38,569][20292] KL-divergence is very high: 162.3102 [2023-12-27 07:49:38,585][20381] Updated weights for policy 1, policy_version 315062 (0.0006) [2023-12-27 07:49:38,590][20347] Updated weights for policy 0, policy_version 315278 (0.0007) [2023-12-27 07:49:38,610][20292] KL-divergence is very high: 100.6714 [2023-12-27 07:49:38,641][20381] Updated weights for policy 1, policy_version 315072 (0.0007) [2023-12-27 07:49:38,650][20347] Updated weights for policy 0, policy_version 315288 (0.0008) [2023-12-27 07:49:39,337][20381] Updated weights for policy 1, policy_version 315082 (0.0006) [2023-12-27 07:49:39,338][20292] KL-divergence is very high: 106.6803 [2023-12-27 07:49:39,339][20347] Updated weights for policy 0, policy_version 315298 (0.0007) [2023-12-27 07:49:39,401][20381] Updated weights for policy 1, policy_version 315092 (0.0008) [2023-12-27 07:49:39,401][20347] Updated weights for policy 0, policy_version 315308 (0.0009) [2023-12-27 07:49:39,458][20347] Updated weights for policy 0, policy_version 315318 (0.0010) [2023-12-27 07:49:39,460][20381] Updated weights for policy 1, policy_version 315102 (0.0006) [2023-12-27 07:49:39,516][20347] Updated weights for policy 0, policy_version 315328 (0.0010) [2023-12-27 07:49:39,516][20381] Updated weights for policy 1, policy_version 315112 (0.0007) [2023-12-27 07:49:40,232][20381] Updated weights for policy 1, policy_version 315122 (0.0006) [2023-12-27 07:49:40,261][20347] Updated weights for policy 0, policy_version 315338 (0.0006) [2023-12-27 07:49:40,295][20381] Updated weights for policy 1, policy_version 315132 (0.0007) [2023-12-27 07:49:40,316][20347] Updated weights for policy 0, policy_version 315348 (0.0006) [2023-12-27 07:49:40,357][20381] Updated weights for policy 1, policy_version 315142 (0.0011) [2023-12-27 07:49:40,374][20347] Updated weights for policy 0, policy_version 315358 (0.0008) [2023-12-27 07:49:41,052][20381] Updated weights for policy 1, policy_version 315152 (0.0010) [2023-12-27 07:49:41,056][20347] Updated weights for policy 0, policy_version 315368 (0.0008) [2023-12-27 07:49:41,110][20381] Updated weights for policy 1, policy_version 315162 (0.0009) [2023-12-27 07:49:41,112][20347] Updated weights for policy 0, policy_version 315378 (0.0006) [2023-12-27 07:49:41,174][20347] Updated weights for policy 0, policy_version 315388 (0.0006) [2023-12-27 07:49:41,180][20381] Updated weights for policy 1, policy_version 315172 (0.0010) [2023-12-27 07:49:41,821][20347] Updated weights for policy 0, policy_version 315398 (0.0007) [2023-12-27 07:49:41,874][20381] Updated weights for policy 1, policy_version 315182 (0.0008) [2023-12-27 07:49:41,875][20347] Updated weights for policy 0, policy_version 315408 (0.0008) [2023-12-27 07:49:41,934][20381] Updated weights for policy 1, policy_version 315192 (0.0007) [2023-12-27 07:49:41,936][20347] Updated weights for policy 0, policy_version 315418 (0.0006) [2023-12-27 07:49:41,981][20381] Updated weights for policy 1, policy_version 315202 (0.0007) [2023-12-27 07:49:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 161472512. Throughput: 0: 10326.9, 1: 10334.2. Samples: 161461896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '754.520')] [2023-12-27 07:49:42,607][20347] Updated weights for policy 0, policy_version 315428 (0.0006) [2023-12-27 07:49:42,663][20347] Updated weights for policy 0, policy_version 315438 (0.0006) [2023-12-27 07:49:42,668][20381] Updated weights for policy 1, policy_version 315212 (0.0008) [2023-12-27 07:49:42,718][20347] Updated weights for policy 0, policy_version 315448 (0.0006) [2023-12-27 07:49:42,723][20381] Updated weights for policy 1, policy_version 315222 (0.0007) [2023-12-27 07:49:42,772][20381] Updated weights for policy 1, policy_version 315232 (0.0006) [2023-12-27 07:49:43,348][20381] Updated weights for policy 1, policy_version 315242 (0.0006) [2023-12-27 07:49:43,407][20381] Updated weights for policy 1, policy_version 315252 (0.0008) [2023-12-27 07:49:43,427][20347] Updated weights for policy 0, policy_version 315458 (0.0007) [2023-12-27 07:49:43,465][20381] Updated weights for policy 1, policy_version 315262 (0.0005) [2023-12-27 07:49:43,482][20347] Updated weights for policy 0, policy_version 315468 (0.0007) [2023-12-27 07:49:43,522][20381] Updated weights for policy 1, policy_version 315272 (0.0006) [2023-12-27 07:49:43,537][20347] Updated weights for policy 0, policy_version 315478 (0.0007) [2023-12-27 07:49:43,585][20347] Updated weights for policy 0, policy_version 315488 (0.0008) [2023-12-27 07:49:44,169][20381] Updated weights for policy 1, policy_version 315282 (0.0006) [2023-12-27 07:49:44,230][20381] Updated weights for policy 1, policy_version 315292 (0.0006) [2023-12-27 07:49:44,284][20381] Updated weights for policy 1, policy_version 315302 (0.0005) [2023-12-27 07:49:44,295][20347] Updated weights for policy 0, policy_version 315498 (0.0008) [2023-12-27 07:49:44,340][20347] Updated weights for policy 0, policy_version 315508 (0.0007) [2023-12-27 07:49:44,398][20347] Updated weights for policy 0, policy_version 315518 (0.0007) [2023-12-27 07:49:44,917][20381] Updated weights for policy 1, policy_version 315312 (0.0006) [2023-12-27 07:49:44,970][20381] Updated weights for policy 1, policy_version 315322 (0.0010) [2023-12-27 07:49:45,032][20381] Updated weights for policy 1, policy_version 315332 (0.0009) [2023-12-27 07:49:45,068][20347] Updated weights for policy 0, policy_version 315528 (0.0007) [2023-12-27 07:49:45,127][20347] Updated weights for policy 0, policy_version 315538 (0.0007) [2023-12-27 07:49:45,186][20347] Updated weights for policy 0, policy_version 315548 (0.0008) [2023-12-27 07:49:45,717][20381] Updated weights for policy 1, policy_version 315342 (0.0009) [2023-12-27 07:49:45,771][20381] Updated weights for policy 1, policy_version 315352 (0.0010) [2023-12-27 07:49:45,827][20381] Updated weights for policy 1, policy_version 315362 (0.0008) [2023-12-27 07:49:45,867][20347] Updated weights for policy 0, policy_version 315558 (0.0006) [2023-12-27 07:49:45,925][20347] Updated weights for policy 0, policy_version 315568 (0.0005) [2023-12-27 07:49:45,981][20347] Updated weights for policy 0, policy_version 315578 (0.0005) [2023-12-27 07:49:46,442][20381] Updated weights for policy 1, policy_version 315372 (0.0006) [2023-12-27 07:49:46,507][20381] Updated weights for policy 1, policy_version 315382 (0.0005) [2023-12-27 07:49:46,564][20347] Updated weights for policy 0, policy_version 315588 (0.0006) [2023-12-27 07:49:46,566][20381] Updated weights for policy 1, policy_version 315392 (0.0005) [2023-12-27 07:49:46,619][20347] Updated weights for policy 0, policy_version 315598 (0.0005) [2023-12-27 07:49:46,680][20347] Updated weights for policy 0, policy_version 315608 (0.0005) [2023-12-27 07:49:47,065][20381] Updated weights for policy 1, policy_version 315402 (0.0005) [2023-12-27 07:49:47,134][20381] Updated weights for policy 1, policy_version 315412 (0.0009) [2023-12-27 07:49:47,188][20381] Updated weights for policy 1, policy_version 315422 (0.0010) [2023-12-27 07:49:47,237][20381] Updated weights for policy 1, policy_version 315432 (0.0010) [2023-12-27 07:49:47,244][20347] Updated weights for policy 0, policy_version 315618 (0.0008) [2023-12-27 07:49:47,301][20347] Updated weights for policy 0, policy_version 315628 (0.0007) [2023-12-27 07:49:47,355][20347] Updated weights for policy 0, policy_version 315638 (0.0007) [2023-12-27 07:49:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 161579008. Throughput: 0: 10409.4, 1: 10383.0. Samples: 161589672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '816.640')] [2023-12-27 07:49:47,403][20347] Updated weights for policy 0, policy_version 315648 (0.0007) [2023-12-27 07:49:47,897][20381] Updated weights for policy 1, policy_version 315442 (0.0008) [2023-12-27 07:49:47,947][20381] Updated weights for policy 1, policy_version 315452 (0.0007) [2023-12-27 07:49:47,995][20381] Updated weights for policy 1, policy_version 315462 (0.0007) [2023-12-27 07:49:48,055][20347] Updated weights for policy 0, policy_version 315658 (0.0005) [2023-12-27 07:49:48,115][20347] Updated weights for policy 0, policy_version 315668 (0.0007) [2023-12-27 07:49:48,169][20347] Updated weights for policy 0, policy_version 315678 (0.0010) [2023-12-27 07:49:48,672][20381] Updated weights for policy 1, policy_version 315472 (0.0010) [2023-12-27 07:49:48,731][20381] Updated weights for policy 1, policy_version 315482 (0.0007) [2023-12-27 07:49:48,786][20381] Updated weights for policy 1, policy_version 315492 (0.0011) [2023-12-27 07:49:48,834][20347] Updated weights for policy 0, policy_version 315688 (0.0010) [2023-12-27 07:49:48,883][20347] Updated weights for policy 0, policy_version 315698 (0.0010) [2023-12-27 07:49:48,943][20347] Updated weights for policy 0, policy_version 315708 (0.0010) [2023-12-27 07:49:49,462][20381] Updated weights for policy 1, policy_version 315502 (0.0011) [2023-12-27 07:49:49,511][20381] Updated weights for policy 1, policy_version 315512 (0.0011) [2023-12-27 07:49:49,569][20381] Updated weights for policy 1, policy_version 315522 (0.0010) [2023-12-27 07:49:49,649][20347] Updated weights for policy 0, policy_version 315718 (0.0007) [2023-12-27 07:49:49,710][20347] Updated weights for policy 0, policy_version 315728 (0.0006) [2023-12-27 07:49:49,772][20347] Updated weights for policy 0, policy_version 315738 (0.0006) [2023-12-27 07:49:50,282][20381] Updated weights for policy 1, policy_version 315532 (0.0011) [2023-12-27 07:49:50,341][20381] Updated weights for policy 1, policy_version 315542 (0.0010) [2023-12-27 07:49:50,402][20381] Updated weights for policy 1, policy_version 315552 (0.0007) [2023-12-27 07:49:50,426][20347] Updated weights for policy 0, policy_version 315748 (0.0007) [2023-12-27 07:49:50,478][20347] Updated weights for policy 0, policy_version 315758 (0.0007) [2023-12-27 07:49:50,529][20347] Updated weights for policy 0, policy_version 315768 (0.0007) [2023-12-27 07:49:51,060][20381] Updated weights for policy 1, policy_version 315562 (0.0007) [2023-12-27 07:49:51,126][20381] Updated weights for policy 1, policy_version 315572 (0.0008) [2023-12-27 07:49:51,192][20381] Updated weights for policy 1, policy_version 315582 (0.0007) [2023-12-27 07:49:51,255][20381] Updated weights for policy 1, policy_version 315592 (0.0007) [2023-12-27 07:49:51,281][20347] Updated weights for policy 0, policy_version 315779 (0.0008) [2023-12-27 07:49:51,338][20347] Updated weights for policy 0, policy_version 315789 (0.0009) [2023-12-27 07:49:51,407][20347] Updated weights for policy 0, policy_version 315799 (0.0007) [2023-12-27 07:49:51,931][20381] Updated weights for policy 1, policy_version 315602 (0.0009) [2023-12-27 07:49:51,984][20381] Updated weights for policy 1, policy_version 315612 (0.0008) [2023-12-27 07:49:52,036][20381] Updated weights for policy 1, policy_version 315622 (0.0008) [2023-12-27 07:49:52,136][20347] Updated weights for policy 0, policy_version 315809 (0.0007) [2023-12-27 07:49:52,194][20347] Updated weights for policy 0, policy_version 315819 (0.0007) [2023-12-27 07:49:52,282][20347] Updated weights for policy 0, policy_version 315829 (0.0007) [2023-12-27 07:49:52,340][20347] Updated weights for policy 0, policy_version 315839 (0.0009) [2023-12-27 07:49:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 161685504. Throughput: 0: 10398.4, 1: 10375.6. Samples: 161651732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 07:49:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '985.820')] [2023-12-27 07:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000315624_80814080.pth... [2023-12-27 07:49:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000315840_80871424.pth... [2023-12-27 07:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000314408_80502784.pth [2023-12-27 07:49:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000314624_80560128.pth [2023-12-27 07:49:52,775][20381] Updated weights for policy 1, policy_version 315632 (0.0008) [2023-12-27 07:49:52,823][20381] Updated weights for policy 1, policy_version 315642 (0.0007) [2023-12-27 07:49:52,878][20381] Updated weights for policy 1, policy_version 315652 (0.0008) [2023-12-27 07:49:53,027][20347] Updated weights for policy 0, policy_version 315849 (0.0008) [2023-12-27 07:49:53,075][20347] Updated weights for policy 0, policy_version 315859 (0.0007) [2023-12-27 07:49:53,122][20347] Updated weights for policy 0, policy_version 315869 (0.0008) [2023-12-27 07:49:53,620][20381] Updated weights for policy 1, policy_version 315662 (0.0007) [2023-12-27 07:49:53,679][20381] Updated weights for policy 1, policy_version 315672 (0.0009) [2023-12-27 07:49:53,740][20381] Updated weights for policy 1, policy_version 315682 (0.0008) [2023-12-27 07:49:53,877][20347] Updated weights for policy 0, policy_version 315879 (0.0007) [2023-12-27 07:49:53,935][20347] Updated weights for policy 0, policy_version 315889 (0.0007) [2023-12-27 07:49:53,992][20347] Updated weights for policy 0, policy_version 315899 (0.0009) [2023-12-27 07:49:54,391][20381] Updated weights for policy 1, policy_version 315692 (0.0008) [2023-12-27 07:49:54,445][20381] Updated weights for policy 1, policy_version 315702 (0.0008) [2023-12-27 07:49:54,498][20381] Updated weights for policy 1, policy_version 315712 (0.0008) [2023-12-27 07:49:54,709][20347] Updated weights for policy 0, policy_version 315909 (0.0007) [2023-12-27 07:49:54,764][20347] Updated weights for policy 0, policy_version 315919 (0.0008) [2023-12-27 07:49:54,816][20347] Updated weights for policy 0, policy_version 315929 (0.0007) [2023-12-27 07:49:55,208][20381] Updated weights for policy 1, policy_version 315722 (0.0008) [2023-12-27 07:49:55,258][20381] Updated weights for policy 1, policy_version 315732 (0.0005) [2023-12-27 07:49:55,308][20381] Updated weights for policy 1, policy_version 315742 (0.0005) [2023-12-27 07:49:55,358][20381] Updated weights for policy 1, policy_version 315752 (0.0005) [2023-12-27 07:49:55,503][20347] Updated weights for policy 0, policy_version 315939 (0.0007) [2023-12-27 07:49:55,553][20347] Updated weights for policy 0, policy_version 315949 (0.0007) [2023-12-27 07:49:55,606][20347] Updated weights for policy 0, policy_version 315959 (0.0007) [2023-12-27 07:49:55,950][20381] Updated weights for policy 1, policy_version 315762 (0.0008) [2023-12-27 07:49:56,000][20381] Updated weights for policy 1, policy_version 315772 (0.0010) [2023-12-27 07:49:56,051][20381] Updated weights for policy 1, policy_version 315782 (0.0006) [2023-12-27 07:49:56,341][20347] Updated weights for policy 0, policy_version 315969 (0.0007) [2023-12-27 07:49:56,395][20347] Updated weights for policy 0, policy_version 315979 (0.0008) [2023-12-27 07:49:56,446][20347] Updated weights for policy 0, policy_version 315989 (0.0008) [2023-12-27 07:49:56,499][20347] Updated weights for policy 0, policy_version 315999 (0.0007) [2023-12-27 07:49:56,675][20381] Updated weights for policy 1, policy_version 315792 (0.0010) [2023-12-27 07:49:56,724][20381] Updated weights for policy 1, policy_version 315802 (0.0010) [2023-12-27 07:49:56,775][20381] Updated weights for policy 1, policy_version 315812 (0.0010) [2023-12-27 07:49:57,154][20347] Updated weights for policy 0, policy_version 316009 (0.0005) [2023-12-27 07:49:57,200][20347] Updated weights for policy 0, policy_version 316019 (0.0005) [2023-12-27 07:49:57,249][20347] Updated weights for policy 0, policy_version 316029 (0.0008) [2023-12-27 07:49:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 161783808. Throughput: 0: 10295.0, 1: 10314.6. Samples: 161771456. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:49:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:49:57,427][20381] Updated weights for policy 1, policy_version 315822 (0.0010) [2023-12-27 07:49:57,476][20381] Updated weights for policy 1, policy_version 315832 (0.0010) [2023-12-27 07:49:57,524][20381] Updated weights for policy 1, policy_version 315842 (0.0010) [2023-12-27 07:49:57,956][20347] Updated weights for policy 0, policy_version 316039 (0.0007) [2023-12-27 07:49:58,018][20347] Updated weights for policy 0, policy_version 316049 (0.0005) [2023-12-27 07:49:58,074][20347] Updated weights for policy 0, policy_version 316059 (0.0005) [2023-12-27 07:49:58,116][20381] Updated weights for policy 1, policy_version 315852 (0.0010) [2023-12-27 07:49:58,178][20381] Updated weights for policy 1, policy_version 315862 (0.0007) [2023-12-27 07:49:58,242][20381] Updated weights for policy 1, policy_version 315872 (0.0010) [2023-12-27 07:49:58,789][20347] Updated weights for policy 0, policy_version 316069 (0.0006) [2023-12-27 07:49:58,856][20347] Updated weights for policy 0, policy_version 316079 (0.0007) [2023-12-27 07:49:58,922][20347] Updated weights for policy 0, policy_version 316089 (0.0007) [2023-12-27 07:49:59,002][20381] Updated weights for policy 1, policy_version 315882 (0.0010) [2023-12-27 07:49:59,057][20381] Updated weights for policy 1, policy_version 315892 (0.0008) [2023-12-27 07:49:59,107][20381] Updated weights for policy 1, policy_version 315902 (0.0008) [2023-12-27 07:49:59,164][20381] Updated weights for policy 1, policy_version 315912 (0.0008) [2023-12-27 07:49:59,665][20347] Updated weights for policy 0, policy_version 316099 (0.0008) [2023-12-27 07:49:59,717][20347] Updated weights for policy 0, policy_version 316109 (0.0007) [2023-12-27 07:49:59,777][20347] Updated weights for policy 0, policy_version 316119 (0.0007) [2023-12-27 07:49:59,880][20381] Updated weights for policy 1, policy_version 315922 (0.0008) [2023-12-27 07:49:59,939][20381] Updated weights for policy 1, policy_version 315932 (0.0008) [2023-12-27 07:49:59,991][20381] Updated weights for policy 1, policy_version 315942 (0.0008) [2023-12-27 07:50:00,485][20347] Updated weights for policy 0, policy_version 316129 (0.0008) [2023-12-27 07:50:00,536][20347] Updated weights for policy 0, policy_version 316139 (0.0010) [2023-12-27 07:50:00,586][20347] Updated weights for policy 0, policy_version 316149 (0.0010) [2023-12-27 07:50:00,633][20347] Updated weights for policy 0, policy_version 316159 (0.0010) [2023-12-27 07:50:00,685][20381] Updated weights for policy 1, policy_version 315952 (0.0009) [2023-12-27 07:50:00,741][20381] Updated weights for policy 1, policy_version 315962 (0.0008) [2023-12-27 07:50:00,800][20381] Updated weights for policy 1, policy_version 315972 (0.0006) [2023-12-27 07:50:01,277][20347] Updated weights for policy 0, policy_version 316169 (0.0009) [2023-12-27 07:50:01,334][20347] Updated weights for policy 0, policy_version 316179 (0.0010) [2023-12-27 07:50:01,397][20347] Updated weights for policy 0, policy_version 316189 (0.0007) [2023-12-27 07:50:01,453][20381] Updated weights for policy 1, policy_version 315982 (0.0008) [2023-12-27 07:50:01,511][20381] Updated weights for policy 1, policy_version 315992 (0.0010) [2023-12-27 07:50:01,559][20381] Updated weights for policy 1, policy_version 316002 (0.0010) [2023-12-27 07:50:02,048][20347] Updated weights for policy 0, policy_version 316199 (0.0008) [2023-12-27 07:50:02,099][20347] Updated weights for policy 0, policy_version 316209 (0.0010) [2023-12-27 07:50:02,147][20347] Updated weights for policy 0, policy_version 316219 (0.0010) [2023-12-27 07:50:02,174][20381] Updated weights for policy 1, policy_version 316012 (0.0009) [2023-12-27 07:50:02,227][20381] Updated weights for policy 1, policy_version 316022 (0.0009) [2023-12-27 07:50:02,284][20381] Updated weights for policy 1, policy_version 316032 (0.0009) [2023-12-27 07:50:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 161890304. Throughput: 0: 10321.0, 1: 10350.1. Samples: 161894852. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:02,819][20347] Updated weights for policy 0, policy_version 316229 (0.0009) [2023-12-27 07:50:02,879][20347] Updated weights for policy 0, policy_version 316239 (0.0007) [2023-12-27 07:50:02,938][20347] Updated weights for policy 0, policy_version 316249 (0.0007) [2023-12-27 07:50:02,968][20381] Updated weights for policy 1, policy_version 316042 (0.0009) [2023-12-27 07:50:03,022][20381] Updated weights for policy 1, policy_version 316052 (0.0007) [2023-12-27 07:50:03,072][20381] Updated weights for policy 1, policy_version 316062 (0.0008) [2023-12-27 07:50:03,119][20381] Updated weights for policy 1, policy_version 316072 (0.0006) [2023-12-27 07:50:03,600][20347] Updated weights for policy 0, policy_version 316259 (0.0007) [2023-12-27 07:50:03,666][20347] Updated weights for policy 0, policy_version 316269 (0.0006) [2023-12-27 07:50:03,724][20347] Updated weights for policy 0, policy_version 316279 (0.0006) [2023-12-27 07:50:03,780][20381] Updated weights for policy 1, policy_version 316082 (0.0008) [2023-12-27 07:50:03,845][20381] Updated weights for policy 1, policy_version 316092 (0.0010) [2023-12-27 07:50:03,903][20381] Updated weights for policy 1, policy_version 316102 (0.0010) [2023-12-27 07:50:04,296][20347] Updated weights for policy 0, policy_version 316289 (0.0005) [2023-12-27 07:50:04,359][20347] Updated weights for policy 0, policy_version 316299 (0.0010) [2023-12-27 07:50:04,418][20347] Updated weights for policy 0, policy_version 316309 (0.0010) [2023-12-27 07:50:04,477][20347] Updated weights for policy 0, policy_version 316319 (0.0007) [2023-12-27 07:50:04,515][20381] Updated weights for policy 1, policy_version 316112 (0.0010) [2023-12-27 07:50:04,571][20381] Updated weights for policy 1, policy_version 316122 (0.0010) [2023-12-27 07:50:04,629][20381] Updated weights for policy 1, policy_version 316132 (0.0011) [2023-12-27 07:50:05,110][20347] Updated weights for policy 0, policy_version 316329 (0.0009) [2023-12-27 07:50:05,166][20347] Updated weights for policy 0, policy_version 316339 (0.0008) [2023-12-27 07:50:05,220][20347] Updated weights for policy 0, policy_version 316349 (0.0007) [2023-12-27 07:50:05,329][20381] Updated weights for policy 1, policy_version 316142 (0.0011) [2023-12-27 07:50:05,390][20381] Updated weights for policy 1, policy_version 316152 (0.0010) [2023-12-27 07:50:05,450][20381] Updated weights for policy 1, policy_version 316162 (0.0008) [2023-12-27 07:50:05,841][20347] Updated weights for policy 0, policy_version 316359 (0.0009) [2023-12-27 07:50:05,901][20347] Updated weights for policy 0, policy_version 316369 (0.0010) [2023-12-27 07:50:05,956][20347] Updated weights for policy 0, policy_version 316379 (0.0010) [2023-12-27 07:50:06,090][20381] Updated weights for policy 1, policy_version 316172 (0.0007) [2023-12-27 07:50:06,147][20381] Updated weights for policy 1, policy_version 316182 (0.0008) [2023-12-27 07:50:06,202][20381] Updated weights for policy 1, policy_version 316192 (0.0007) [2023-12-27 07:50:06,647][20347] Updated weights for policy 0, policy_version 316389 (0.0010) [2023-12-27 07:50:06,696][20347] Updated weights for policy 0, policy_version 316399 (0.0010) [2023-12-27 07:50:06,756][20347] Updated weights for policy 0, policy_version 316409 (0.0009) [2023-12-27 07:50:06,916][20381] Updated weights for policy 1, policy_version 316202 (0.0007) [2023-12-27 07:50:06,975][20381] Updated weights for policy 1, policy_version 316212 (0.0008) [2023-12-27 07:50:07,027][20381] Updated weights for policy 1, policy_version 316222 (0.0007) [2023-12-27 07:50:07,079][20381] Updated weights for policy 1, policy_version 316232 (0.0009) [2023-12-27 07:50:07,352][20347] Updated weights for policy 0, policy_version 316419 (0.0007) [2023-12-27 07:50:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 161988608. Throughput: 0: 10333.6, 1: 10356.1. Samples: 161958588. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000316232_80969728.pth... [2023-12-27 07:50:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000315016_80658432.pth [2023-12-27 07:50:07,402][20347] Updated weights for policy 0, policy_version 316429 (0.0005) [2023-12-27 07:50:07,449][20347] Updated weights for policy 0, policy_version 316439 (0.0007) [2023-12-27 07:50:07,491][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000316448_81027072.pth... [2023-12-27 07:50:07,495][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000315232_80715776.pth [2023-12-27 07:50:07,852][20381] Updated weights for policy 1, policy_version 316242 (0.0008) [2023-12-27 07:50:07,904][20381] Updated weights for policy 1, policy_version 316252 (0.0007) [2023-12-27 07:50:07,965][20381] Updated weights for policy 1, policy_version 316262 (0.0007) [2023-12-27 07:50:08,100][20347] Updated weights for policy 0, policy_version 316449 (0.0010) [2023-12-27 07:50:08,158][20347] Updated weights for policy 0, policy_version 316459 (0.0009) [2023-12-27 07:50:08,213][20347] Updated weights for policy 0, policy_version 316469 (0.0008) [2023-12-27 07:50:08,271][20347] Updated weights for policy 0, policy_version 316479 (0.0008) [2023-12-27 07:50:08,632][20381] Updated weights for policy 1, policy_version 316272 (0.0008) [2023-12-27 07:50:08,691][20381] Updated weights for policy 1, policy_version 316282 (0.0007) [2023-12-27 07:50:08,742][20381] Updated weights for policy 1, policy_version 316292 (0.0007) [2023-12-27 07:50:08,936][20347] Updated weights for policy 0, policy_version 316489 (0.0010) [2023-12-27 07:50:08,982][20347] Updated weights for policy 0, policy_version 316499 (0.0005) [2023-12-27 07:50:09,031][20347] Updated weights for policy 0, policy_version 316509 (0.0005) [2023-12-27 07:50:09,499][20381] Updated weights for policy 1, policy_version 316302 (0.0010) [2023-12-27 07:50:09,555][20381] Updated weights for policy 1, policy_version 316312 (0.0011) [2023-12-27 07:50:09,615][20347] Updated weights for policy 0, policy_version 316519 (0.0006) [2023-12-27 07:50:09,615][20381] Updated weights for policy 1, policy_version 316322 (0.0011) [2023-12-27 07:50:09,673][20347] Updated weights for policy 0, policy_version 316529 (0.0006) [2023-12-27 07:50:09,733][20347] Updated weights for policy 0, policy_version 316539 (0.0005) [2023-12-27 07:50:10,311][20347] Updated weights for policy 0, policy_version 316549 (0.0007) [2023-12-27 07:50:10,373][20381] Updated weights for policy 1, policy_version 316332 (0.0009) [2023-12-27 07:50:10,379][20347] Updated weights for policy 0, policy_version 316559 (0.0007) [2023-12-27 07:50:10,427][20381] Updated weights for policy 1, policy_version 316342 (0.0008) [2023-12-27 07:50:10,439][20347] Updated weights for policy 0, policy_version 316569 (0.0005) [2023-12-27 07:50:10,488][20381] Updated weights for policy 1, policy_version 316352 (0.0007) [2023-12-27 07:50:11,103][20347] Updated weights for policy 0, policy_version 316579 (0.0005) [2023-12-27 07:50:11,169][20347] Updated weights for policy 0, policy_version 316589 (0.0008) [2023-12-27 07:50:11,200][20381] Updated weights for policy 1, policy_version 316362 (0.0006) [2023-12-27 07:50:11,226][20347] Updated weights for policy 0, policy_version 316599 (0.0007) [2023-12-27 07:50:11,259][20381] Updated weights for policy 1, policy_version 316372 (0.0007) [2023-12-27 07:50:11,310][20381] Updated weights for policy 1, policy_version 316382 (0.0009) [2023-12-27 07:50:11,372][20381] Updated weights for policy 1, policy_version 316392 (0.0009) [2023-12-27 07:50:11,948][20347] Updated weights for policy 0, policy_version 316609 (0.0006) [2023-12-27 07:50:12,015][20347] Updated weights for policy 0, policy_version 316619 (0.0007) [2023-12-27 07:50:12,056][20381] Updated weights for policy 1, policy_version 316402 (0.0006) [2023-12-27 07:50:12,070][20347] Updated weights for policy 0, policy_version 316629 (0.0006) [2023-12-27 07:50:12,112][20381] Updated weights for policy 1, policy_version 316412 (0.0006) [2023-12-27 07:50:12,125][20347] Updated weights for policy 0, policy_version 316639 (0.0005) [2023-12-27 07:50:12,170][20381] Updated weights for policy 1, policy_version 316422 (0.0007) [2023-12-27 07:50:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 162095104. Throughput: 0: 10444.4, 1: 10325.3. Samples: 162082360. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:12,824][20347] Updated weights for policy 0, policy_version 316649 (0.0005) [2023-12-27 07:50:12,859][20381] Updated weights for policy 1, policy_version 316432 (0.0008) [2023-12-27 07:50:12,878][20347] Updated weights for policy 0, policy_version 316659 (0.0006) [2023-12-27 07:50:12,924][20381] Updated weights for policy 1, policy_version 316442 (0.0007) [2023-12-27 07:50:12,934][20347] Updated weights for policy 0, policy_version 316669 (0.0007) [2023-12-27 07:50:12,981][20381] Updated weights for policy 1, policy_version 316452 (0.0007) [2023-12-27 07:50:13,604][20347] Updated weights for policy 0, policy_version 316679 (0.0010) [2023-12-27 07:50:13,654][20347] Updated weights for policy 0, policy_version 316689 (0.0010) [2023-12-27 07:50:13,685][20381] Updated weights for policy 1, policy_version 316462 (0.0007) [2023-12-27 07:50:13,710][20347] Updated weights for policy 0, policy_version 316699 (0.0008) [2023-12-27 07:50:13,736][20381] Updated weights for policy 1, policy_version 316472 (0.0008) [2023-12-27 07:50:13,780][20381] Updated weights for policy 1, policy_version 316482 (0.0007) [2023-12-27 07:50:14,405][20347] Updated weights for policy 0, policy_version 316709 (0.0007) [2023-12-27 07:50:14,457][20381] Updated weights for policy 1, policy_version 316492 (0.0009) [2023-12-27 07:50:14,464][20347] Updated weights for policy 0, policy_version 316719 (0.0010) [2023-12-27 07:50:14,504][20381] Updated weights for policy 1, policy_version 316502 (0.0010) [2023-12-27 07:50:14,516][20347] Updated weights for policy 0, policy_version 316729 (0.0010) [2023-12-27 07:50:14,553][20381] Updated weights for policy 1, policy_version 316512 (0.0010) [2023-12-27 07:50:15,175][20347] Updated weights for policy 0, policy_version 316739 (0.0008) [2023-12-27 07:50:15,226][20347] Updated weights for policy 0, policy_version 316749 (0.0006) [2023-12-27 07:50:15,261][20381] Updated weights for policy 1, policy_version 316522 (0.0010) [2023-12-27 07:50:15,283][20347] Updated weights for policy 0, policy_version 316759 (0.0007) [2023-12-27 07:50:15,318][20381] Updated weights for policy 1, policy_version 316532 (0.0006) [2023-12-27 07:50:15,374][20381] Updated weights for policy 1, policy_version 316542 (0.0009) [2023-12-27 07:50:15,427][20381] Updated weights for policy 1, policy_version 316552 (0.0010) [2023-12-27 07:50:15,920][20347] Updated weights for policy 0, policy_version 316769 (0.0007) [2023-12-27 07:50:15,974][20347] Updated weights for policy 0, policy_version 316779 (0.0005) [2023-12-27 07:50:16,023][20347] Updated weights for policy 0, policy_version 316789 (0.0006) [2023-12-27 07:50:16,069][20347] Updated weights for policy 0, policy_version 316799 (0.0007) [2023-12-27 07:50:16,084][20381] Updated weights for policy 1, policy_version 316562 (0.0010) [2023-12-27 07:50:16,145][20381] Updated weights for policy 1, policy_version 316572 (0.0010) [2023-12-27 07:50:16,200][20381] Updated weights for policy 1, policy_version 316582 (0.0010) [2023-12-27 07:50:16,738][20347] Updated weights for policy 0, policy_version 316809 (0.0007) [2023-12-27 07:50:16,788][20347] Updated weights for policy 0, policy_version 316819 (0.0007) [2023-12-27 07:50:16,848][20347] Updated weights for policy 0, policy_version 316829 (0.0007) [2023-12-27 07:50:16,868][20381] Updated weights for policy 1, policy_version 316592 (0.0010) [2023-12-27 07:50:16,912][20381] Updated weights for policy 1, policy_version 316602 (0.0010) [2023-12-27 07:50:16,970][20381] Updated weights for policy 1, policy_version 316612 (0.0010) [2023-12-27 07:50:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 162193408. Throughput: 0: 10372.9, 1: 10312.7. Samples: 162204148. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:17,563][20381] Updated weights for policy 1, policy_version 316622 (0.0007) [2023-12-27 07:50:17,569][20347] Updated weights for policy 0, policy_version 316839 (0.0006) [2023-12-27 07:50:17,612][20381] Updated weights for policy 1, policy_version 316632 (0.0006) [2023-12-27 07:50:17,615][20347] Updated weights for policy 0, policy_version 316849 (0.0005) [2023-12-27 07:50:17,663][20347] Updated weights for policy 0, policy_version 316859 (0.0008) [2023-12-27 07:50:17,669][20381] Updated weights for policy 1, policy_version 316642 (0.0005) [2023-12-27 07:50:18,224][20381] Updated weights for policy 1, policy_version 316652 (0.0006) [2023-12-27 07:50:18,281][20381] Updated weights for policy 1, policy_version 316662 (0.0006) [2023-12-27 07:50:18,355][20381] Updated weights for policy 1, policy_version 316672 (0.0008) [2023-12-27 07:50:18,398][20347] Updated weights for policy 0, policy_version 316869 (0.0006) [2023-12-27 07:50:18,459][20347] Updated weights for policy 0, policy_version 316879 (0.0006) [2023-12-27 07:50:18,514][20347] Updated weights for policy 0, policy_version 316889 (0.0005) [2023-12-27 07:50:19,060][20381] Updated weights for policy 1, policy_version 316682 (0.0008) [2023-12-27 07:50:19,122][20381] Updated weights for policy 1, policy_version 316692 (0.0007) [2023-12-27 07:50:19,144][20347] Updated weights for policy 0, policy_version 316899 (0.0005) [2023-12-27 07:50:19,168][20381] Updated weights for policy 1, policy_version 316702 (0.0006) [2023-12-27 07:50:19,198][20347] Updated weights for policy 0, policy_version 316909 (0.0005) [2023-12-27 07:50:19,217][20381] Updated weights for policy 1, policy_version 316712 (0.0008) [2023-12-27 07:50:19,264][20347] Updated weights for policy 0, policy_version 316919 (0.0008) [2023-12-27 07:50:19,940][20381] Updated weights for policy 1, policy_version 316722 (0.0011) [2023-12-27 07:50:19,950][20347] Updated weights for policy 0, policy_version 316929 (0.0006) [2023-12-27 07:50:19,998][20381] Updated weights for policy 1, policy_version 316732 (0.0011) [2023-12-27 07:50:20,011][20347] Updated weights for policy 0, policy_version 316939 (0.0008) [2023-12-27 07:50:20,058][20381] Updated weights for policy 1, policy_version 316742 (0.0009) [2023-12-27 07:50:20,071][20347] Updated weights for policy 0, policy_version 316949 (0.0007) [2023-12-27 07:50:20,134][20347] Updated weights for policy 0, policy_version 316959 (0.0008) [2023-12-27 07:50:20,763][20381] Updated weights for policy 1, policy_version 316752 (0.0011) [2023-12-27 07:50:20,823][20381] Updated weights for policy 1, policy_version 316762 (0.0011) [2023-12-27 07:50:20,829][20347] Updated weights for policy 0, policy_version 316969 (0.0006) [2023-12-27 07:50:20,874][20347] Updated weights for policy 0, policy_version 316979 (0.0005) [2023-12-27 07:50:20,876][20381] Updated weights for policy 1, policy_version 316772 (0.0011) [2023-12-27 07:50:20,924][20347] Updated weights for policy 0, policy_version 316989 (0.0007) [2023-12-27 07:50:21,607][20381] Updated weights for policy 1, policy_version 316782 (0.0011) [2023-12-27 07:50:21,662][20347] Updated weights for policy 0, policy_version 316999 (0.0006) [2023-12-27 07:50:21,676][20381] Updated weights for policy 1, policy_version 316792 (0.0009) [2023-12-27 07:50:21,724][20347] Updated weights for policy 0, policy_version 317009 (0.0007) [2023-12-27 07:50:21,741][20381] Updated weights for policy 1, policy_version 316802 (0.0010) [2023-12-27 07:50:21,790][20347] Updated weights for policy 0, policy_version 317019 (0.0006) [2023-12-27 07:50:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 162291712. Throughput: 0: 10334.4, 1: 10286.2. Samples: 162265896. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000316808_81117184.pth... [2023-12-27 07:50:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000317024_81174528.pth... [2023-12-27 07:50:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000315840_80871424.pth [2023-12-27 07:50:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000315624_80814080.pth [2023-12-27 07:50:22,425][20347] Updated weights for policy 0, policy_version 317029 (0.0008) [2023-12-27 07:50:22,447][20381] Updated weights for policy 1, policy_version 316812 (0.0008) [2023-12-27 07:50:22,478][20347] Updated weights for policy 0, policy_version 317039 (0.0005) [2023-12-27 07:50:22,506][20381] Updated weights for policy 1, policy_version 316822 (0.0011) [2023-12-27 07:50:22,531][20347] Updated weights for policy 0, policy_version 317049 (0.0005) [2023-12-27 07:50:22,563][20381] Updated weights for policy 1, policy_version 316832 (0.0011) [2023-12-27 07:50:23,255][20347] Updated weights for policy 0, policy_version 317059 (0.0006) [2023-12-27 07:50:23,290][20381] Updated weights for policy 1, policy_version 316842 (0.0010) [2023-12-27 07:50:23,318][20347] Updated weights for policy 0, policy_version 317069 (0.0008) [2023-12-27 07:50:23,345][20381] Updated weights for policy 1, policy_version 316852 (0.0009) [2023-12-27 07:50:23,379][20347] Updated weights for policy 0, policy_version 317079 (0.0007) [2023-12-27 07:50:23,400][20381] Updated weights for policy 1, policy_version 316862 (0.0006) [2023-12-27 07:50:23,452][20381] Updated weights for policy 1, policy_version 316872 (0.0007) [2023-12-27 07:50:24,015][20347] Updated weights for policy 0, policy_version 317089 (0.0008) [2023-12-27 07:50:24,067][20347] Updated weights for policy 0, policy_version 317099 (0.0010) [2023-12-27 07:50:24,133][20347] Updated weights for policy 0, policy_version 317109 (0.0005) [2023-12-27 07:50:24,198][20347] Updated weights for policy 0, policy_version 317119 (0.0008) [2023-12-27 07:50:24,232][20381] Updated weights for policy 1, policy_version 316882 (0.0006) [2023-12-27 07:50:24,283][20381] Updated weights for policy 1, policy_version 316892 (0.0006) [2023-12-27 07:50:24,337][20381] Updated weights for policy 1, policy_version 316902 (0.0005) [2023-12-27 07:50:24,826][20347] Updated weights for policy 0, policy_version 317129 (0.0008) [2023-12-27 07:50:24,875][20347] Updated weights for policy 0, policy_version 317139 (0.0007) [2023-12-27 07:50:24,926][20347] Updated weights for policy 0, policy_version 317149 (0.0007) [2023-12-27 07:50:24,976][20381] Updated weights for policy 1, policy_version 316912 (0.0007) [2023-12-27 07:50:25,039][20381] Updated weights for policy 1, policy_version 316922 (0.0008) [2023-12-27 07:50:25,100][20381] Updated weights for policy 1, policy_version 316932 (0.0008) [2023-12-27 07:50:25,635][20347] Updated weights for policy 0, policy_version 317159 (0.0006) [2023-12-27 07:50:25,695][20347] Updated weights for policy 0, policy_version 317169 (0.0006) [2023-12-27 07:50:25,750][20347] Updated weights for policy 0, policy_version 317179 (0.0005) [2023-12-27 07:50:25,784][20381] Updated weights for policy 1, policy_version 316942 (0.0008) [2023-12-27 07:50:25,833][20381] Updated weights for policy 1, policy_version 316952 (0.0010) [2023-12-27 07:50:25,890][20381] Updated weights for policy 1, policy_version 316962 (0.0007) [2023-12-27 07:50:26,377][20347] Updated weights for policy 0, policy_version 317189 (0.0008) [2023-12-27 07:50:26,425][20347] Updated weights for policy 0, policy_version 317199 (0.0010) [2023-12-27 07:50:26,475][20347] Updated weights for policy 0, policy_version 317209 (0.0010) [2023-12-27 07:50:26,605][20381] Updated weights for policy 1, policy_version 316972 (0.0007) [2023-12-27 07:50:26,661][20381] Updated weights for policy 1, policy_version 316982 (0.0007) [2023-12-27 07:50:26,715][20381] Updated weights for policy 1, policy_version 316992 (0.0007) [2023-12-27 07:50:27,164][20347] Updated weights for policy 0, policy_version 317219 (0.0010) [2023-12-27 07:50:27,214][20347] Updated weights for policy 0, policy_version 317229 (0.0010) [2023-12-27 07:50:27,274][20347] Updated weights for policy 0, policy_version 317239 (0.0010) [2023-12-27 07:50:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 162398208. Throughput: 0: 10279.5, 1: 10259.8. Samples: 162386160. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:50:27,400][20381] Updated weights for policy 1, policy_version 317002 (0.0007) [2023-12-27 07:50:27,463][20381] Updated weights for policy 1, policy_version 317012 (0.0005) [2023-12-27 07:50:27,513][20381] Updated weights for policy 1, policy_version 317022 (0.0005) [2023-12-27 07:50:27,572][20381] Updated weights for policy 1, policy_version 317032 (0.0007) [2023-12-27 07:50:27,949][20347] Updated weights for policy 0, policy_version 317249 (0.0008) [2023-12-27 07:50:28,007][20347] Updated weights for policy 0, policy_version 317259 (0.0010) [2023-12-27 07:50:28,055][20347] Updated weights for policy 0, policy_version 317269 (0.0010) [2023-12-27 07:50:28,106][20347] Updated weights for policy 0, policy_version 317279 (0.0010) [2023-12-27 07:50:28,190][20381] Updated weights for policy 1, policy_version 317042 (0.0010) [2023-12-27 07:50:28,244][20381] Updated weights for policy 1, policy_version 317052 (0.0010) [2023-12-27 07:50:28,300][20381] Updated weights for policy 1, policy_version 317062 (0.0010) [2023-12-27 07:50:28,797][20347] Updated weights for policy 0, policy_version 317289 (0.0010) [2023-12-27 07:50:28,849][20347] Updated weights for policy 0, policy_version 317299 (0.0010) [2023-12-27 07:50:28,900][20347] Updated weights for policy 0, policy_version 317309 (0.0010) [2023-12-27 07:50:28,939][20381] Updated weights for policy 1, policy_version 317072 (0.0006) [2023-12-27 07:50:28,989][20381] Updated weights for policy 1, policy_version 317082 (0.0008) [2023-12-27 07:50:29,038][20381] Updated weights for policy 1, policy_version 317092 (0.0010) [2023-12-27 07:50:29,579][20347] Updated weights for policy 0, policy_version 317319 (0.0007) [2023-12-27 07:50:29,638][20347] Updated weights for policy 0, policy_version 317329 (0.0010) [2023-12-27 07:50:29,693][20347] Updated weights for policy 0, policy_version 317339 (0.0010) [2023-12-27 07:50:29,721][20381] Updated weights for policy 1, policy_version 317102 (0.0010) [2023-12-27 07:50:29,774][20381] Updated weights for policy 1, policy_version 317112 (0.0010) [2023-12-27 07:50:29,827][20381] Updated weights for policy 1, policy_version 317122 (0.0010) [2023-12-27 07:50:30,366][20347] Updated weights for policy 0, policy_version 317349 (0.0009) [2023-12-27 07:50:30,420][20347] Updated weights for policy 0, policy_version 317359 (0.0008) [2023-12-27 07:50:30,473][20347] Updated weights for policy 0, policy_version 317369 (0.0007) [2023-12-27 07:50:30,515][20381] Updated weights for policy 1, policy_version 317132 (0.0008) [2023-12-27 07:50:30,566][20381] Updated weights for policy 1, policy_version 317142 (0.0010) [2023-12-27 07:50:30,611][20381] Updated weights for policy 1, policy_version 317152 (0.0010) [2023-12-27 07:50:31,117][20347] Updated weights for policy 0, policy_version 317379 (0.0008) [2023-12-27 07:50:31,177][20347] Updated weights for policy 0, policy_version 317389 (0.0011) [2023-12-27 07:50:31,213][20381] Updated weights for policy 1, policy_version 317162 (0.0009) [2023-12-27 07:50:31,234][20347] Updated weights for policy 0, policy_version 317399 (0.0007) [2023-12-27 07:50:31,272][20381] Updated weights for policy 1, policy_version 317172 (0.0010) [2023-12-27 07:50:31,331][20381] Updated weights for policy 1, policy_version 317182 (0.0011) [2023-12-27 07:50:31,396][20381] Updated weights for policy 1, policy_version 317192 (0.0009) [2023-12-27 07:50:31,917][20347] Updated weights for policy 0, policy_version 317409 (0.0010) [2023-12-27 07:50:31,974][20347] Updated weights for policy 0, policy_version 317419 (0.0007) [2023-12-27 07:50:32,041][20347] Updated weights for policy 0, policy_version 317429 (0.0007) [2023-12-27 07:50:32,042][20381] Updated weights for policy 1, policy_version 317202 (0.0011) [2023-12-27 07:50:32,097][20347] Updated weights for policy 0, policy_version 317439 (0.0006) [2023-12-27 07:50:32,099][20381] Updated weights for policy 1, policy_version 317212 (0.0008) [2023-12-27 07:50:32,147][20381] Updated weights for policy 1, policy_version 317222 (0.0005) [2023-12-27 07:50:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 162504704. Throughput: 0: 10268.1, 1: 10219.6. Samples: 162511616. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '958.780')] [2023-12-27 07:50:32,728][20381] Updated weights for policy 1, policy_version 317232 (0.0009) [2023-12-27 07:50:32,782][20381] Updated weights for policy 1, policy_version 317242 (0.0006) [2023-12-27 07:50:32,809][20347] Updated weights for policy 0, policy_version 317449 (0.0007) [2023-12-27 07:50:32,830][20381] Updated weights for policy 1, policy_version 317252 (0.0005) [2023-12-27 07:50:32,860][20347] Updated weights for policy 0, policy_version 317459 (0.0007) [2023-12-27 07:50:32,915][20347] Updated weights for policy 0, policy_version 317469 (0.0005) [2023-12-27 07:50:33,376][20381] Updated weights for policy 1, policy_version 317262 (0.0007) [2023-12-27 07:50:33,430][20381] Updated weights for policy 1, policy_version 317272 (0.0009) [2023-12-27 07:50:33,490][20381] Updated weights for policy 1, policy_version 317282 (0.0007) [2023-12-27 07:50:33,536][20347] Updated weights for policy 0, policy_version 317479 (0.0007) [2023-12-27 07:50:33,580][20347] Updated weights for policy 0, policy_version 317489 (0.0007) [2023-12-27 07:50:33,625][20347] Updated weights for policy 0, policy_version 317499 (0.0007) [2023-12-27 07:50:34,182][20381] Updated weights for policy 1, policy_version 317292 (0.0009) [2023-12-27 07:50:34,241][20381] Updated weights for policy 1, policy_version 317302 (0.0011) [2023-12-27 07:50:34,268][20347] Updated weights for policy 0, policy_version 317509 (0.0008) [2023-12-27 07:50:34,291][20381] Updated weights for policy 1, policy_version 317312 (0.0008) [2023-12-27 07:50:34,329][20347] Updated weights for policy 0, policy_version 317519 (0.0008) [2023-12-27 07:50:34,391][20347] Updated weights for policy 0, policy_version 317529 (0.0007) [2023-12-27 07:50:35,002][20381] Updated weights for policy 1, policy_version 317322 (0.0009) [2023-12-27 07:50:35,062][20381] Updated weights for policy 1, policy_version 317332 (0.0011) [2023-12-27 07:50:35,118][20381] Updated weights for policy 1, policy_version 317342 (0.0011) [2023-12-27 07:50:35,127][20347] Updated weights for policy 0, policy_version 317539 (0.0008) [2023-12-27 07:50:35,174][20381] Updated weights for policy 1, policy_version 317352 (0.0011) [2023-12-27 07:50:35,177][20347] Updated weights for policy 0, policy_version 317549 (0.0005) [2023-12-27 07:50:35,238][20347] Updated weights for policy 0, policy_version 317559 (0.0008) [2023-12-27 07:50:35,780][20381] Updated weights for policy 1, policy_version 317362 (0.0011) [2023-12-27 07:50:35,825][20381] Updated weights for policy 1, policy_version 317372 (0.0010) [2023-12-27 07:50:35,883][20381] Updated weights for policy 1, policy_version 317382 (0.0011) [2023-12-27 07:50:35,952][20347] Updated weights for policy 0, policy_version 317569 (0.0006) [2023-12-27 07:50:36,007][20347] Updated weights for policy 0, policy_version 317579 (0.0007) [2023-12-27 07:50:36,061][20347] Updated weights for policy 0, policy_version 317589 (0.0008) [2023-12-27 07:50:36,120][20347] Updated weights for policy 0, policy_version 317599 (0.0007) [2023-12-27 07:50:36,567][20381] Updated weights for policy 1, policy_version 317392 (0.0010) [2023-12-27 07:50:36,629][20381] Updated weights for policy 1, policy_version 317402 (0.0011) [2023-12-27 07:50:36,687][20381] Updated weights for policy 1, policy_version 317412 (0.0010) [2023-12-27 07:50:36,862][20347] Updated weights for policy 0, policy_version 317609 (0.0008) [2023-12-27 07:50:36,921][20347] Updated weights for policy 0, policy_version 317619 (0.0007) [2023-12-27 07:50:36,979][20347] Updated weights for policy 0, policy_version 317629 (0.0007) [2023-12-27 07:50:37,318][20381] Updated weights for policy 1, policy_version 317422 (0.0010) [2023-12-27 07:50:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 162603008. Throughput: 0: 10258.2, 1: 10259.9. Samples: 162575048. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '897.370')] [2023-12-27 07:50:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000317632_81330176.pth... [2023-12-27 07:50:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000316448_81027072.pth [2023-12-27 07:50:37,374][20381] Updated weights for policy 1, policy_version 317432 (0.0006) [2023-12-27 07:50:37,429][20381] Updated weights for policy 1, policy_version 317442 (0.0005) [2023-12-27 07:50:37,461][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000317448_81281024.pth... [2023-12-27 07:50:37,464][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000316232_80969728.pth [2023-12-27 07:50:37,709][20347] Updated weights for policy 0, policy_version 317639 (0.0008) [2023-12-27 07:50:37,771][20347] Updated weights for policy 0, policy_version 317649 (0.0007) [2023-12-27 07:50:37,825][20347] Updated weights for policy 0, policy_version 317659 (0.0008) [2023-12-27 07:50:38,056][20381] Updated weights for policy 1, policy_version 317452 (0.0007) [2023-12-27 07:50:38,112][20381] Updated weights for policy 1, policy_version 317462 (0.0008) [2023-12-27 07:50:38,165][20381] Updated weights for policy 1, policy_version 317472 (0.0008) [2023-12-27 07:50:38,497][20347] Updated weights for policy 0, policy_version 317669 (0.0008) [2023-12-27 07:50:38,546][20347] Updated weights for policy 0, policy_version 317679 (0.0008) [2023-12-27 07:50:38,603][20347] Updated weights for policy 0, policy_version 317689 (0.0006) [2023-12-27 07:50:38,922][20381] Updated weights for policy 1, policy_version 317482 (0.0008) [2023-12-27 07:50:38,972][20381] Updated weights for policy 1, policy_version 317492 (0.0007) [2023-12-27 07:50:39,018][20381] Updated weights for policy 1, policy_version 317502 (0.0008) [2023-12-27 07:50:39,068][20381] Updated weights for policy 1, policy_version 317512 (0.0008) [2023-12-27 07:50:39,148][20347] Updated weights for policy 0, policy_version 317699 (0.0006) [2023-12-27 07:50:39,206][20347] Updated weights for policy 0, policy_version 317709 (0.0007) [2023-12-27 07:50:39,267][20347] Updated weights for policy 0, policy_version 317719 (0.0007) [2023-12-27 07:50:39,839][20381] Updated weights for policy 1, policy_version 317522 (0.0008) [2023-12-27 07:50:39,900][20381] Updated weights for policy 1, policy_version 317532 (0.0008) [2023-12-27 07:50:39,952][20347] Updated weights for policy 0, policy_version 317729 (0.0008) [2023-12-27 07:50:39,966][20381] Updated weights for policy 1, policy_version 317542 (0.0007) [2023-12-27 07:50:40,009][20347] Updated weights for policy 0, policy_version 317739 (0.0006) [2023-12-27 07:50:40,076][20347] Updated weights for policy 0, policy_version 317749 (0.0008) [2023-12-27 07:50:40,135][20347] Updated weights for policy 0, policy_version 317759 (0.0008) [2023-12-27 07:50:40,613][20381] Updated weights for policy 1, policy_version 317552 (0.0006) [2023-12-27 07:50:40,675][20381] Updated weights for policy 1, policy_version 317562 (0.0008) [2023-12-27 07:50:40,734][20381] Updated weights for policy 1, policy_version 317572 (0.0008) [2023-12-27 07:50:40,820][20347] Updated weights for policy 0, policy_version 317769 (0.0005) [2023-12-27 07:50:40,879][20347] Updated weights for policy 0, policy_version 317779 (0.0006) [2023-12-27 07:50:40,938][20347] Updated weights for policy 0, policy_version 317789 (0.0008) [2023-12-27 07:50:41,423][20381] Updated weights for policy 1, policy_version 317582 (0.0010) [2023-12-27 07:50:41,480][20381] Updated weights for policy 1, policy_version 317592 (0.0011) [2023-12-27 07:50:41,540][20381] Updated weights for policy 1, policy_version 317602 (0.0011) [2023-12-27 07:50:41,653][20347] Updated weights for policy 0, policy_version 317799 (0.0008) [2023-12-27 07:50:41,710][20347] Updated weights for policy 0, policy_version 317809 (0.0008) [2023-12-27 07:50:41,770][20347] Updated weights for policy 0, policy_version 317819 (0.0007) [2023-12-27 07:50:42,250][20381] Updated weights for policy 1, policy_version 317612 (0.0010) [2023-12-27 07:50:42,313][20381] Updated weights for policy 1, policy_version 317622 (0.0009) [2023-12-27 07:50:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 162701312. Throughput: 0: 10314.7, 1: 10266.8. Samples: 162697628. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '947.070')] [2023-12-27 07:50:42,377][20381] Updated weights for policy 1, policy_version 317632 (0.0009) [2023-12-27 07:50:42,470][20347] Updated weights for policy 0, policy_version 317829 (0.0007) [2023-12-27 07:50:42,537][20347] Updated weights for policy 0, policy_version 317839 (0.0008) [2023-12-27 07:50:42,598][20347] Updated weights for policy 0, policy_version 317849 (0.0008) [2023-12-27 07:50:43,047][20381] Updated weights for policy 1, policy_version 317642 (0.0007) [2023-12-27 07:50:43,110][20381] Updated weights for policy 1, policy_version 317652 (0.0007) [2023-12-27 07:50:43,165][20381] Updated weights for policy 1, policy_version 317662 (0.0008) [2023-12-27 07:50:43,217][20381] Updated weights for policy 1, policy_version 317672 (0.0007) [2023-12-27 07:50:43,230][20347] Updated weights for policy 0, policy_version 317859 (0.0007) [2023-12-27 07:50:43,293][20347] Updated weights for policy 0, policy_version 317869 (0.0007) [2023-12-27 07:50:43,365][20347] Updated weights for policy 0, policy_version 317879 (0.0007) [2023-12-27 07:50:43,819][20381] Updated weights for policy 1, policy_version 317682 (0.0005) [2023-12-27 07:50:43,876][20381] Updated weights for policy 1, policy_version 317692 (0.0005) [2023-12-27 07:50:43,936][20381] Updated weights for policy 1, policy_version 317702 (0.0005) [2023-12-27 07:50:44,043][20347] Updated weights for policy 0, policy_version 317889 (0.0010) [2023-12-27 07:50:44,094][20347] Updated weights for policy 0, policy_version 317899 (0.0010) [2023-12-27 07:50:44,142][20347] Updated weights for policy 0, policy_version 317909 (0.0010) [2023-12-27 07:50:44,195][20347] Updated weights for policy 0, policy_version 317919 (0.0010) [2023-12-27 07:50:44,544][20381] Updated weights for policy 1, policy_version 317712 (0.0008) [2023-12-27 07:50:44,604][20381] Updated weights for policy 1, policy_version 317722 (0.0007) [2023-12-27 07:50:44,661][20381] Updated weights for policy 1, policy_version 317732 (0.0006) [2023-12-27 07:50:44,855][20347] Updated weights for policy 0, policy_version 317929 (0.0010) [2023-12-27 07:50:44,910][20347] Updated weights for policy 0, policy_version 317939 (0.0008) [2023-12-27 07:50:44,978][20347] Updated weights for policy 0, policy_version 317949 (0.0005) [2023-12-27 07:50:45,321][20381] Updated weights for policy 1, policy_version 317742 (0.0008) [2023-12-27 07:50:45,374][20381] Updated weights for policy 1, policy_version 317752 (0.0007) [2023-12-27 07:50:45,421][20381] Updated weights for policy 1, policy_version 317762 (0.0007) [2023-12-27 07:50:45,652][20347] Updated weights for policy 0, policy_version 317959 (0.0009) [2023-12-27 07:50:45,706][20347] Updated weights for policy 0, policy_version 317969 (0.0010) [2023-12-27 07:50:45,764][20347] Updated weights for policy 0, policy_version 317979 (0.0010) [2023-12-27 07:50:46,121][20381] Updated weights for policy 1, policy_version 317772 (0.0007) [2023-12-27 07:50:46,181][20381] Updated weights for policy 1, policy_version 317782 (0.0009) [2023-12-27 07:50:46,230][20292] KL-divergence is very high: 184.9618 [2023-12-27 07:50:46,241][20381] Updated weights for policy 1, policy_version 317792 (0.0006) [2023-12-27 07:50:46,400][20347] Updated weights for policy 0, policy_version 317989 (0.0008) [2023-12-27 07:50:46,452][20347] Updated weights for policy 0, policy_version 317999 (0.0008) [2023-12-27 07:50:46,499][20347] Updated weights for policy 0, policy_version 318009 (0.0007) [2023-12-27 07:50:46,846][20381] Updated weights for policy 1, policy_version 317802 (0.0007) [2023-12-27 07:50:46,912][20381] Updated weights for policy 1, policy_version 317812 (0.0005) [2023-12-27 07:50:46,990][20381] Updated weights for policy 1, policy_version 317822 (0.0008) [2023-12-27 07:50:47,042][20381] Updated weights for policy 1, policy_version 317832 (0.0008) [2023-12-27 07:50:47,103][20347] Updated weights for policy 0, policy_version 318019 (0.0007) [2023-12-27 07:50:47,167][20347] Updated weights for policy 0, policy_version 318030 (0.0006) [2023-12-27 07:50:47,224][20347] Updated weights for policy 0, policy_version 318040 (0.0006) [2023-12-27 07:50:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 162816000. Throughput: 0: 10352.9, 1: 10269.9. Samples: 162822876. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '972.390')] [2023-12-27 07:50:47,632][20381] Updated weights for policy 1, policy_version 317842 (0.0005) [2023-12-27 07:50:47,684][20381] Updated weights for policy 1, policy_version 317852 (0.0005) [2023-12-27 07:50:47,731][20381] Updated weights for policy 1, policy_version 317862 (0.0005) [2023-12-27 07:50:47,835][20347] Updated weights for policy 0, policy_version 318050 (0.0007) [2023-12-27 07:50:47,888][20347] Updated weights for policy 0, policy_version 318060 (0.0009) [2023-12-27 07:50:47,938][20347] Updated weights for policy 0, policy_version 318070 (0.0008) [2023-12-27 07:50:48,002][20347] Updated weights for policy 0, policy_version 318080 (0.0007) [2023-12-27 07:50:48,311][20381] Updated weights for policy 1, policy_version 317872 (0.0007) [2023-12-27 07:50:48,368][20381] Updated weights for policy 1, policy_version 317882 (0.0006) [2023-12-27 07:50:48,422][20381] Updated weights for policy 1, policy_version 317892 (0.0008) [2023-12-27 07:50:48,632][20347] Updated weights for policy 0, policy_version 318090 (0.0006) [2023-12-27 07:50:48,691][20347] Updated weights for policy 0, policy_version 318100 (0.0008) [2023-12-27 07:50:48,745][20347] Updated weights for policy 0, policy_version 318110 (0.0007) [2023-12-27 07:50:49,122][20381] Updated weights for policy 1, policy_version 317902 (0.0008) [2023-12-27 07:50:49,174][20381] Updated weights for policy 1, policy_version 317912 (0.0008) [2023-12-27 07:50:49,233][20381] Updated weights for policy 1, policy_version 317922 (0.0008) [2023-12-27 07:50:49,437][20347] Updated weights for policy 0, policy_version 318120 (0.0008) [2023-12-27 07:50:49,503][20347] Updated weights for policy 0, policy_version 318130 (0.0008) [2023-12-27 07:50:49,550][20347] Updated weights for policy 0, policy_version 318140 (0.0010) [2023-12-27 07:50:49,918][20381] Updated weights for policy 1, policy_version 317932 (0.0007) [2023-12-27 07:50:49,977][20381] Updated weights for policy 1, policy_version 317942 (0.0008) [2023-12-27 07:50:50,032][20381] Updated weights for policy 1, policy_version 317952 (0.0008) [2023-12-27 07:50:50,243][20347] Updated weights for policy 0, policy_version 318150 (0.0007) [2023-12-27 07:50:50,300][20347] Updated weights for policy 0, policy_version 318160 (0.0008) [2023-12-27 07:50:50,359][20347] Updated weights for policy 0, policy_version 318170 (0.0007) [2023-12-27 07:50:50,739][20381] Updated weights for policy 1, policy_version 317962 (0.0007) [2023-12-27 07:50:50,798][20381] Updated weights for policy 1, policy_version 317972 (0.0008) [2023-12-27 07:50:50,857][20381] Updated weights for policy 1, policy_version 317982 (0.0008) [2023-12-27 07:50:50,922][20381] Updated weights for policy 1, policy_version 317992 (0.0007) [2023-12-27 07:50:51,046][20347] Updated weights for policy 0, policy_version 318180 (0.0008) [2023-12-27 07:50:51,102][20347] Updated weights for policy 0, policy_version 318190 (0.0008) [2023-12-27 07:50:51,158][20347] Updated weights for policy 0, policy_version 318200 (0.0008) [2023-12-27 07:50:51,600][20381] Updated weights for policy 1, policy_version 318002 (0.0008) [2023-12-27 07:50:51,664][20381] Updated weights for policy 1, policy_version 318012 (0.0008) [2023-12-27 07:50:51,724][20381] Updated weights for policy 1, policy_version 318022 (0.0007) [2023-12-27 07:50:51,872][20347] Updated weights for policy 0, policy_version 318210 (0.0007) [2023-12-27 07:50:51,929][20347] Updated weights for policy 0, policy_version 318220 (0.0005) [2023-12-27 07:50:51,978][20347] Updated weights for policy 0, policy_version 318230 (0.0005) [2023-12-27 07:50:52,028][20347] Updated weights for policy 0, policy_version 318240 (0.0005) [2023-12-27 07:50:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 162914304. Throughput: 0: 10328.3, 1: 10280.5. Samples: 162885984. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '895.180')] [2023-12-27 07:50:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000318240_81485824.pth... [2023-12-27 07:50:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000317024_81174528.pth [2023-12-27 07:50:52,415][20381] Updated weights for policy 1, policy_version 318032 (0.0009) [2023-12-27 07:50:52,474][20381] Updated weights for policy 1, policy_version 318042 (0.0007) [2023-12-27 07:50:52,526][20381] Updated weights for policy 1, policy_version 318052 (0.0008) [2023-12-27 07:50:52,548][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000318056_81436672.pth... [2023-12-27 07:50:52,551][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000316808_81117184.pth [2023-12-27 07:50:52,651][20347] Updated weights for policy 0, policy_version 318250 (0.0007) [2023-12-27 07:50:52,706][20347] Updated weights for policy 0, policy_version 318260 (0.0005) [2023-12-27 07:50:52,761][20347] Updated weights for policy 0, policy_version 318270 (0.0005) [2023-12-27 07:50:53,236][20381] Updated weights for policy 1, policy_version 318062 (0.0007) [2023-12-27 07:50:53,288][20381] Updated weights for policy 1, policy_version 318072 (0.0008) [2023-12-27 07:50:53,335][20381] Updated weights for policy 1, policy_version 318082 (0.0006) [2023-12-27 07:50:53,349][20347] Updated weights for policy 0, policy_version 318280 (0.0006) [2023-12-27 07:50:53,403][20347] Updated weights for policy 0, policy_version 318290 (0.0007) [2023-12-27 07:50:53,455][20347] Updated weights for policy 0, policy_version 318300 (0.0008) [2023-12-27 07:50:54,038][20347] Updated weights for policy 0, policy_version 318310 (0.0008) [2023-12-27 07:50:54,059][20381] Updated weights for policy 1, policy_version 318092 (0.0006) [2023-12-27 07:50:54,097][20347] Updated weights for policy 0, policy_version 318320 (0.0006) [2023-12-27 07:50:54,116][20381] Updated weights for policy 1, policy_version 318102 (0.0006) [2023-12-27 07:50:54,157][20347] Updated weights for policy 0, policy_version 318330 (0.0007) [2023-12-27 07:50:54,178][20381] Updated weights for policy 1, policy_version 318112 (0.0007) [2023-12-27 07:50:54,866][20347] Updated weights for policy 0, policy_version 318340 (0.0006) [2023-12-27 07:50:54,895][20381] Updated weights for policy 1, policy_version 318122 (0.0007) [2023-12-27 07:50:54,919][20347] Updated weights for policy 0, policy_version 318350 (0.0005) [2023-12-27 07:50:54,955][20381] Updated weights for policy 1, policy_version 318132 (0.0008) [2023-12-27 07:50:54,976][20347] Updated weights for policy 0, policy_version 318360 (0.0006) [2023-12-27 07:50:55,014][20381] Updated weights for policy 1, policy_version 318142 (0.0007) [2023-12-27 07:50:55,070][20381] Updated weights for policy 1, policy_version 318152 (0.0008) [2023-12-27 07:50:55,570][20347] Updated weights for policy 0, policy_version 318370 (0.0006) [2023-12-27 07:50:55,631][20347] Updated weights for policy 0, policy_version 318380 (0.0007) [2023-12-27 07:50:55,679][20347] Updated weights for policy 0, policy_version 318390 (0.0005) [2023-12-27 07:50:55,736][20347] Updated weights for policy 0, policy_version 318400 (0.0005) [2023-12-27 07:50:55,787][20381] Updated weights for policy 1, policy_version 318162 (0.0009) [2023-12-27 07:50:55,837][20381] Updated weights for policy 1, policy_version 318172 (0.0010) [2023-12-27 07:50:55,892][20381] Updated weights for policy 1, policy_version 318182 (0.0009) [2023-12-27 07:50:56,350][20347] Updated weights for policy 0, policy_version 318410 (0.0008) [2023-12-27 07:50:56,403][20347] Updated weights for policy 0, policy_version 318420 (0.0008) [2023-12-27 07:50:56,456][20347] Updated weights for policy 0, policy_version 318430 (0.0007) [2023-12-27 07:50:56,625][20381] Updated weights for policy 1, policy_version 318192 (0.0007) [2023-12-27 07:50:56,674][20381] Updated weights for policy 1, policy_version 318202 (0.0008) [2023-12-27 07:50:56,723][20381] Updated weights for policy 1, policy_version 318212 (0.0008) [2023-12-27 07:50:57,138][20347] Updated weights for policy 0, policy_version 318440 (0.0007) [2023-12-27 07:50:57,187][20347] Updated weights for policy 0, policy_version 318450 (0.0008) [2023-12-27 07:50:57,237][20347] Updated weights for policy 0, policy_version 318460 (0.0008) [2023-12-27 07:50:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 163020800. Throughput: 0: 10325.2, 1: 10276.8. Samples: 163009452. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:50:57,363][19170] Avg episode reward: [(0, '990.380'), (1, '893.520')] [2023-12-27 07:50:57,408][20381] Updated weights for policy 1, policy_version 318222 (0.0006) [2023-12-27 07:50:57,459][20381] Updated weights for policy 1, policy_version 318232 (0.0007) [2023-12-27 07:50:57,516][20381] Updated weights for policy 1, policy_version 318242 (0.0007) [2023-12-27 07:50:57,941][20347] Updated weights for policy 0, policy_version 318470 (0.0008) [2023-12-27 07:50:57,985][20347] Updated weights for policy 0, policy_version 318480 (0.0010) [2023-12-27 07:50:58,029][20347] Updated weights for policy 0, policy_version 318490 (0.0010) [2023-12-27 07:50:58,117][20381] Updated weights for policy 1, policy_version 318252 (0.0005) [2023-12-27 07:50:58,181][20381] Updated weights for policy 1, policy_version 318262 (0.0006) [2023-12-27 07:50:58,242][20381] Updated weights for policy 1, policy_version 318272 (0.0008) [2023-12-27 07:50:58,824][20347] Updated weights for policy 0, policy_version 318500 (0.0009) [2023-12-27 07:50:58,831][20291] KL-divergence is very high: 266.2572 [2023-12-27 07:50:58,837][20291] KL-divergence is very high: 132.3485 [2023-12-27 07:50:58,849][20291] KL-divergence is very high: 129.1512 [2023-12-27 07:50:58,855][20291] KL-divergence is very high: 107.7430 [2023-12-27 07:50:58,886][20347] Updated weights for policy 0, policy_version 318510 (0.0006) [2023-12-27 07:50:58,934][20381] Updated weights for policy 1, policy_version 318282 (0.0008) [2023-12-27 07:50:58,946][20347] Updated weights for policy 0, policy_version 318520 (0.0007) [2023-12-27 07:50:58,947][20291] KL-divergence is very high: 134.4811 [2023-12-27 07:50:58,978][20291] KL-divergence is very high: 127.1489 [2023-12-27 07:50:58,984][20291] KL-divergence is very high: 124.8747 [2023-12-27 07:50:58,991][20381] Updated weights for policy 1, policy_version 318292 (0.0007) [2023-12-27 07:50:59,046][20381] Updated weights for policy 1, policy_version 318302 (0.0007) [2023-12-27 07:50:59,107][20381] Updated weights for policy 1, policy_version 318312 (0.0007) [2023-12-27 07:50:59,596][20291] KL-divergence is very high: 103.6989 [2023-12-27 07:50:59,596][20347] Updated weights for policy 0, policy_version 318530 (0.0007) [2023-12-27 07:50:59,617][20291] KL-divergence is very high: 287.4361 [2023-12-27 07:50:59,623][20291] KL-divergence is very high: 240.5941 [2023-12-27 07:50:59,649][20347] Updated weights for policy 0, policy_version 318540 (0.0008) [2023-12-27 07:50:59,677][20291] KL-divergence is very high: 104.6553 [2023-12-27 07:50:59,698][20347] Updated weights for policy 0, policy_version 318550 (0.0008) [2023-12-27 07:50:59,699][20291] KL-divergence is very high: 156.1974 [2023-12-27 07:50:59,762][20347] Updated weights for policy 0, policy_version 318560 (0.0008) [2023-12-27 07:50:59,853][20381] Updated weights for policy 1, policy_version 318322 (0.0010) [2023-12-27 07:50:59,915][20381] Updated weights for policy 1, policy_version 318332 (0.0009) [2023-12-27 07:50:59,970][20381] Updated weights for policy 1, policy_version 318342 (0.0007) [2023-12-27 07:51:00,371][20291] KL-divergence is very high: 133.8356 [2023-12-27 07:51:00,418][20347] Updated weights for policy 0, policy_version 318570 (0.0010) [2023-12-27 07:51:00,420][20291] KL-divergence is very high: 134.3568 [2023-12-27 07:51:00,473][20347] Updated weights for policy 0, policy_version 318580 (0.0010) [2023-12-27 07:51:00,503][20291] KL-divergence is very high: 136.9173 [2023-12-27 07:51:00,522][20347] Updated weights for policy 0, policy_version 318590 (0.0010) [2023-12-27 07:51:00,681][20381] Updated weights for policy 1, policy_version 318352 (0.0006) [2023-12-27 07:51:00,732][20381] Updated weights for policy 1, policy_version 318362 (0.0005) [2023-12-27 07:51:00,782][20381] Updated weights for policy 1, policy_version 318372 (0.0006) [2023-12-27 07:51:01,237][20347] Updated weights for policy 0, policy_version 318600 (0.0009) [2023-12-27 07:51:01,244][20291] KL-divergence is very high: 105.6217 [2023-12-27 07:51:01,306][20347] Updated weights for policy 0, policy_version 318610 (0.0008) [2023-12-27 07:51:01,379][20347] Updated weights for policy 0, policy_version 318620 (0.0008) [2023-12-27 07:51:01,385][20291] KL-divergence is very high: 101.8551 [2023-12-27 07:51:01,432][20381] Updated weights for policy 1, policy_version 318382 (0.0008) [2023-12-27 07:51:01,482][20381] Updated weights for policy 1, policy_version 318392 (0.0008) [2023-12-27 07:51:01,532][20381] Updated weights for policy 1, policy_version 318402 (0.0008) [2023-12-27 07:51:02,072][20347] Updated weights for policy 0, policy_version 318631 (0.0008) [2023-12-27 07:51:02,092][20291] KL-divergence is very high: 114.4018 [2023-12-27 07:51:02,135][20291] KL-divergence is very high: 122.7745 [2023-12-27 07:51:02,136][20347] Updated weights for policy 0, policy_version 318641 (0.0007) [2023-12-27 07:51:02,161][20291] KL-divergence is very high: 106.8105 [2023-12-27 07:51:02,167][20291] KL-divergence is very high: 117.0733 [2023-12-27 07:51:02,188][20291] KL-divergence is very high: 124.9965 [2023-12-27 07:51:02,189][20381] Updated weights for policy 1, policy_version 318412 (0.0005) [2023-12-27 07:51:02,194][20291] KL-divergence is very high: 160.6629 [2023-12-27 07:51:02,195][20347] Updated weights for policy 0, policy_version 318651 (0.0007) [2023-12-27 07:51:02,206][20291] KL-divergence is very high: 156.1772 [2023-12-27 07:51:02,248][20381] Updated weights for policy 1, policy_version 318422 (0.0005) [2023-12-27 07:51:02,313][20381] Updated weights for policy 1, policy_version 318432 (0.0008) [2023-12-27 07:51:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 163127296. Throughput: 0: 10299.3, 1: 10302.9. Samples: 163131244. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:51:02,363][19170] Avg episode reward: [(0, '485.670'), (1, '922.230')] [2023-12-27 07:51:02,826][20291] KL-divergence is very high: 126.1584 [2023-12-27 07:51:02,846][20347] Updated weights for policy 0, policy_version 318661 (0.0007) [2023-12-27 07:51:02,865][20291] KL-divergence is very high: 110.9433 [2023-12-27 07:51:02,878][20291] KL-divergence is very high: 138.8971 [2023-12-27 07:51:02,904][20347] Updated weights for policy 0, policy_version 318671 (0.0007) [2023-12-27 07:51:02,918][20291] KL-divergence is very high: 151.9937 [2023-12-27 07:51:02,932][20381] Updated weights for policy 1, policy_version 318442 (0.0008) [2023-12-27 07:51:02,959][20347] Updated weights for policy 0, policy_version 318681 (0.0006) [2023-12-27 07:51:02,965][20291] KL-divergence is very high: 121.0161 [2023-12-27 07:51:02,987][20381] Updated weights for policy 1, policy_version 318452 (0.0005) [2023-12-27 07:51:03,035][20381] Updated weights for policy 1, policy_version 318462 (0.0008) [2023-12-27 07:51:03,076][20381] Updated weights for policy 1, policy_version 318472 (0.0007) [2023-12-27 07:51:03,534][20347] Updated weights for policy 0, policy_version 318691 (0.0006) [2023-12-27 07:51:03,587][20347] Updated weights for policy 0, policy_version 318701 (0.0008) [2023-12-27 07:51:03,636][20347] Updated weights for policy 0, policy_version 318711 (0.0008) [2023-12-27 07:51:03,826][20381] Updated weights for policy 1, policy_version 318482 (0.0008) [2023-12-27 07:51:03,889][20381] Updated weights for policy 1, policy_version 318492 (0.0007) [2023-12-27 07:51:03,945][20381] Updated weights for policy 1, policy_version 318502 (0.0007) [2023-12-27 07:51:04,329][20347] Updated weights for policy 0, policy_version 318721 (0.0006) [2023-12-27 07:51:04,389][20347] Updated weights for policy 0, policy_version 318731 (0.0008) [2023-12-27 07:51:04,452][20347] Updated weights for policy 0, policy_version 318741 (0.0007) [2023-12-27 07:51:04,517][20347] Updated weights for policy 0, policy_version 318751 (0.0007) [2023-12-27 07:51:04,628][20381] Updated weights for policy 1, policy_version 318512 (0.0006) [2023-12-27 07:51:04,688][20381] Updated weights for policy 1, policy_version 318522 (0.0007) [2023-12-27 07:51:04,746][20381] Updated weights for policy 1, policy_version 318532 (0.0008) [2023-12-27 07:51:05,169][20347] Updated weights for policy 0, policy_version 318761 (0.0008) [2023-12-27 07:51:05,223][20347] Updated weights for policy 0, policy_version 318771 (0.0007) [2023-12-27 07:51:05,290][20347] Updated weights for policy 0, policy_version 318781 (0.0006) [2023-12-27 07:51:05,412][20381] Updated weights for policy 1, policy_version 318542 (0.0009) [2023-12-27 07:51:05,458][20381] Updated weights for policy 1, policy_version 318552 (0.0010) [2023-12-27 07:51:05,513][20381] Updated weights for policy 1, policy_version 318562 (0.0010) [2023-12-27 07:51:05,939][20347] Updated weights for policy 0, policy_version 318791 (0.0007) [2023-12-27 07:51:05,993][20347] Updated weights for policy 0, policy_version 318801 (0.0007) [2023-12-27 07:51:06,043][20347] Updated weights for policy 0, policy_version 318811 (0.0008) [2023-12-27 07:51:06,239][20381] Updated weights for policy 1, policy_version 318572 (0.0011) [2023-12-27 07:51:06,303][20381] Updated weights for policy 1, policy_version 318582 (0.0011) [2023-12-27 07:51:06,366][20381] Updated weights for policy 1, policy_version 318592 (0.0011) [2023-12-27 07:51:06,796][20347] Updated weights for policy 0, policy_version 318821 (0.0008) [2023-12-27 07:51:06,841][20347] Updated weights for policy 0, policy_version 318831 (0.0008) [2023-12-27 07:51:06,891][20347] Updated weights for policy 0, policy_version 318842 (0.0009) [2023-12-27 07:51:06,962][20381] Updated weights for policy 1, policy_version 318602 (0.0011) [2023-12-27 07:51:07,024][20381] Updated weights for policy 1, policy_version 318612 (0.0006) [2023-12-27 07:51:07,085][20381] Updated weights for policy 1, policy_version 318622 (0.0006) [2023-12-27 07:51:07,146][20381] Updated weights for policy 1, policy_version 318632 (0.0006) [2023-12-27 07:51:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 163225600. Throughput: 0: 10338.1, 1: 10280.1. Samples: 163193712. Policy #0 lag: (min: 5.0, avg: 9.3, max: 37.0) [2023-12-27 07:51:07,363][19170] Avg episode reward: [(0, '581.150'), (1, '912.690')] [2023-12-27 07:51:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000318632_81584128.pth... [2023-12-27 07:51:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000318848_81641472.pth... [2023-12-27 07:51:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000317448_81281024.pth [2023-12-27 07:51:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000317632_81330176.pth [2023-12-27 07:51:07,604][20347] Updated weights for policy 0, policy_version 318852 (0.0006) [2023-12-27 07:51:07,660][20347] Updated weights for policy 0, policy_version 318862 (0.0005) [2023-12-27 07:51:07,712][20381] Updated weights for policy 1, policy_version 318642 (0.0007) [2023-12-27 07:51:07,718][20347] Updated weights for policy 0, policy_version 318872 (0.0005) [2023-12-27 07:51:07,773][20381] Updated weights for policy 1, policy_version 318652 (0.0008) [2023-12-27 07:51:07,828][20381] Updated weights for policy 1, policy_version 318662 (0.0010) [2023-12-27 07:51:08,313][20347] Updated weights for policy 0, policy_version 318882 (0.0006) [2023-12-27 07:51:08,380][20347] Updated weights for policy 0, policy_version 318892 (0.0008) [2023-12-27 07:51:08,417][20381] Updated weights for policy 1, policy_version 318672 (0.0011) [2023-12-27 07:51:08,438][20347] Updated weights for policy 0, policy_version 318902 (0.0005) [2023-12-27 07:51:08,470][20381] Updated weights for policy 1, policy_version 318682 (0.0011) [2023-12-27 07:51:08,494][20347] Updated weights for policy 0, policy_version 318912 (0.0005) [2023-12-27 07:51:08,519][20381] Updated weights for policy 1, policy_version 318692 (0.0011) [2023-12-27 07:51:09,147][20347] Updated weights for policy 0, policy_version 318922 (0.0007) [2023-12-27 07:51:09,204][20347] Updated weights for policy 0, policy_version 318932 (0.0007) [2023-12-27 07:51:09,226][20381] Updated weights for policy 1, policy_version 318702 (0.0010) [2023-12-27 07:51:09,265][20347] Updated weights for policy 0, policy_version 318942 (0.0006) [2023-12-27 07:51:09,283][20381] Updated weights for policy 1, policy_version 318712 (0.0010) [2023-12-27 07:51:09,349][20381] Updated weights for policy 1, policy_version 318722 (0.0011) [2023-12-27 07:51:09,937][20347] Updated weights for policy 0, policy_version 318952 (0.0007) [2023-12-27 07:51:10,006][20347] Updated weights for policy 0, policy_version 318962 (0.0006) [2023-12-27 07:51:10,012][20381] Updated weights for policy 1, policy_version 318732 (0.0010) [2023-12-27 07:51:10,066][20347] Updated weights for policy 0, policy_version 318972 (0.0006) [2023-12-27 07:51:10,071][20381] Updated weights for policy 1, policy_version 318742 (0.0011) [2023-12-27 07:51:10,125][20381] Updated weights for policy 1, policy_version 318752 (0.0011) [2023-12-27 07:51:10,654][20347] Updated weights for policy 0, policy_version 318982 (0.0005) [2023-12-27 07:51:10,708][20347] Updated weights for policy 0, policy_version 318992 (0.0005) [2023-12-27 07:51:10,767][20347] Updated weights for policy 0, policy_version 319002 (0.0005) [2023-12-27 07:51:10,826][20381] Updated weights for policy 1, policy_version 318762 (0.0011) [2023-12-27 07:51:10,881][20381] Updated weights for policy 1, policy_version 318772 (0.0011) [2023-12-27 07:51:10,940][20381] Updated weights for policy 1, policy_version 318782 (0.0011) [2023-12-27 07:51:10,991][20381] Updated weights for policy 1, policy_version 318792 (0.0010) [2023-12-27 07:51:11,351][20347] Updated weights for policy 0, policy_version 319012 (0.0007) [2023-12-27 07:51:11,421][20347] Updated weights for policy 0, policy_version 319022 (0.0007) [2023-12-27 07:51:11,490][20347] Updated weights for policy 0, policy_version 319032 (0.0008) [2023-12-27 07:51:11,647][20381] Updated weights for policy 1, policy_version 318802 (0.0007) [2023-12-27 07:51:11,703][20381] Updated weights for policy 1, policy_version 318812 (0.0006) [2023-12-27 07:51:11,768][20381] Updated weights for policy 1, policy_version 318822 (0.0008) [2023-12-27 07:51:12,209][20347] Updated weights for policy 0, policy_version 319042 (0.0007) [2023-12-27 07:51:12,271][20347] Updated weights for policy 0, policy_version 319052 (0.0007) [2023-12-27 07:51:12,336][20347] Updated weights for policy 0, policy_version 319062 (0.0006) [2023-12-27 07:51:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 163323904. Throughput: 0: 10375.7, 1: 10361.0. Samples: 163319312. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:12,363][19170] Avg episode reward: [(0, '841.230'), (1, '896.060')] [2023-12-27 07:51:12,398][20347] Updated weights for policy 0, policy_version 319072 (0.0008) [2023-12-27 07:51:12,400][20381] Updated weights for policy 1, policy_version 318832 (0.0006) [2023-12-27 07:51:12,459][20381] Updated weights for policy 1, policy_version 318842 (0.0006) [2023-12-27 07:51:12,511][20381] Updated weights for policy 1, policy_version 318852 (0.0006) [2023-12-27 07:51:13,067][20347] Updated weights for policy 0, policy_version 319082 (0.0010) [2023-12-27 07:51:13,127][20347] Updated weights for policy 0, policy_version 319092 (0.0009) [2023-12-27 07:51:13,175][20347] Updated weights for policy 0, policy_version 319102 (0.0009) [2023-12-27 07:51:13,190][20381] Updated weights for policy 1, policy_version 318862 (0.0006) [2023-12-27 07:51:13,248][20381] Updated weights for policy 1, policy_version 318872 (0.0006) [2023-12-27 07:51:13,297][20381] Updated weights for policy 1, policy_version 318882 (0.0005) [2023-12-27 07:51:13,853][20347] Updated weights for policy 0, policy_version 319112 (0.0007) [2023-12-27 07:51:13,884][20381] Updated weights for policy 1, policy_version 318892 (0.0005) [2023-12-27 07:51:13,903][20347] Updated weights for policy 0, policy_version 319122 (0.0007) [2023-12-27 07:51:13,941][20381] Updated weights for policy 1, policy_version 318902 (0.0008) [2023-12-27 07:51:13,952][20347] Updated weights for policy 0, policy_version 319132 (0.0009) [2023-12-27 07:51:13,988][20381] Updated weights for policy 1, policy_version 318912 (0.0010) [2023-12-27 07:51:14,616][20347] Updated weights for policy 0, policy_version 319142 (0.0008) [2023-12-27 07:51:14,630][20381] Updated weights for policy 1, policy_version 318922 (0.0008) [2023-12-27 07:51:14,672][20347] Updated weights for policy 0, policy_version 319152 (0.0005) [2023-12-27 07:51:14,682][20381] Updated weights for policy 1, policy_version 318932 (0.0010) [2023-12-27 07:51:14,721][20347] Updated weights for policy 0, policy_version 319162 (0.0005) [2023-12-27 07:51:14,731][20381] Updated weights for policy 1, policy_version 318942 (0.0010) [2023-12-27 07:51:14,789][20381] Updated weights for policy 1, policy_version 318952 (0.0010) [2023-12-27 07:51:15,440][20381] Updated weights for policy 1, policy_version 318962 (0.0010) [2023-12-27 07:51:15,449][20347] Updated weights for policy 0, policy_version 319172 (0.0005) [2023-12-27 07:51:15,494][20347] Updated weights for policy 0, policy_version 319182 (0.0005) [2023-12-27 07:51:15,495][20381] Updated weights for policy 1, policy_version 318972 (0.0010) [2023-12-27 07:51:15,539][20347] Updated weights for policy 0, policy_version 319192 (0.0007) [2023-12-27 07:51:15,549][20381] Updated weights for policy 1, policy_version 318982 (0.0010) [2023-12-27 07:51:16,198][20381] Updated weights for policy 1, policy_version 318992 (0.0011) [2023-12-27 07:51:16,246][20381] Updated weights for policy 1, policy_version 319002 (0.0010) [2023-12-27 07:51:16,267][20347] Updated weights for policy 0, policy_version 319202 (0.0006) [2023-12-27 07:51:16,297][20381] Updated weights for policy 1, policy_version 319012 (0.0010) [2023-12-27 07:51:16,324][20347] Updated weights for policy 0, policy_version 319212 (0.0006) [2023-12-27 07:51:16,381][20347] Updated weights for policy 0, policy_version 319222 (0.0006) [2023-12-27 07:51:16,436][20347] Updated weights for policy 0, policy_version 319232 (0.0008) [2023-12-27 07:51:16,924][20381] Updated weights for policy 1, policy_version 319022 (0.0010) [2023-12-27 07:51:16,982][20381] Updated weights for policy 1, policy_version 319032 (0.0010) [2023-12-27 07:51:17,041][20381] Updated weights for policy 1, policy_version 319042 (0.0010) [2023-12-27 07:51:17,043][20347] Updated weights for policy 0, policy_version 319242 (0.0006) [2023-12-27 07:51:17,092][20347] Updated weights for policy 0, policy_version 319252 (0.0010) [2023-12-27 07:51:17,137][20347] Updated weights for policy 0, policy_version 319262 (0.0010) [2023-12-27 07:51:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 163438592. Throughput: 0: 10368.3, 1: 10395.3. Samples: 163445980. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:17,363][19170] Avg episode reward: [(0, '942.600'), (1, '818.870')] [2023-12-27 07:51:17,702][20381] Updated weights for policy 1, policy_version 319052 (0.0008) [2023-12-27 07:51:17,756][20381] Updated weights for policy 1, policy_version 319062 (0.0006) [2023-12-27 07:51:17,787][20347] Updated weights for policy 0, policy_version 319272 (0.0006) [2023-12-27 07:51:17,815][20381] Updated weights for policy 1, policy_version 319072 (0.0010) [2023-12-27 07:51:17,840][20347] Updated weights for policy 0, policy_version 319282 (0.0005) [2023-12-27 07:51:17,897][20347] Updated weights for policy 0, policy_version 319292 (0.0006) [2023-12-27 07:51:18,507][20381] Updated weights for policy 1, policy_version 319082 (0.0010) [2023-12-27 07:51:18,564][20347] Updated weights for policy 0, policy_version 319302 (0.0007) [2023-12-27 07:51:18,564][20381] Updated weights for policy 1, policy_version 319092 (0.0008) [2023-12-27 07:51:18,623][20347] Updated weights for policy 0, policy_version 319312 (0.0005) [2023-12-27 07:51:18,625][20381] Updated weights for policy 1, policy_version 319102 (0.0010) [2023-12-27 07:51:18,675][20347] Updated weights for policy 0, policy_version 319322 (0.0006) [2023-12-27 07:51:18,681][20381] Updated weights for policy 1, policy_version 319112 (0.0010) [2023-12-27 07:51:19,356][20381] Updated weights for policy 1, policy_version 319122 (0.0010) [2023-12-27 07:51:19,360][20347] Updated weights for policy 0, policy_version 319332 (0.0007) [2023-12-27 07:51:19,412][20381] Updated weights for policy 1, policy_version 319132 (0.0010) [2023-12-27 07:51:19,420][20347] Updated weights for policy 0, policy_version 319342 (0.0007) [2023-12-27 07:51:19,477][20381] Updated weights for policy 1, policy_version 319142 (0.0008) [2023-12-27 07:51:19,483][20347] Updated weights for policy 0, policy_version 319352 (0.0006) [2023-12-27 07:51:20,148][20347] Updated weights for policy 0, policy_version 319362 (0.0007) [2023-12-27 07:51:20,187][20381] Updated weights for policy 1, policy_version 319152 (0.0010) [2023-12-27 07:51:20,211][20347] Updated weights for policy 0, policy_version 319372 (0.0006) [2023-12-27 07:51:20,251][20381] Updated weights for policy 1, policy_version 319162 (0.0011) [2023-12-27 07:51:20,275][20347] Updated weights for policy 0, policy_version 319382 (0.0006) [2023-12-27 07:51:20,285][20291] KL-divergence is very high: 627.6025 [2023-12-27 07:51:20,300][20291] KL-divergence is very high: 539.6060 [2023-12-27 07:51:20,314][20381] Updated weights for policy 1, policy_version 319172 (0.0011) [2023-12-27 07:51:20,336][20291] KL-divergence is very high: 182.0374 [2023-12-27 07:51:20,341][20347] Updated weights for policy 0, policy_version 319392 (0.0006) [2023-12-27 07:51:20,986][20381] Updated weights for policy 1, policy_version 319182 (0.0011) [2023-12-27 07:51:21,001][20291] KL-divergence is very high: 126.2680 [2023-12-27 07:51:21,045][20381] Updated weights for policy 1, policy_version 319192 (0.0011) [2023-12-27 07:51:21,049][20291] KL-divergence is very high: 1296.4806 [2023-12-27 07:51:21,056][20291] KL-divergence is very high: 2226.6917 [2023-12-27 07:51:21,062][20347] Updated weights for policy 0, policy_version 319402 (0.0006) [2023-12-27 07:51:21,064][20291] KL-divergence is very high: 1588.5609 [2023-12-27 07:51:21,102][20291] KL-divergence is very high: 343.1703 [2023-12-27 07:51:21,109][20291] KL-divergence is very high: 1067.9540 [2023-12-27 07:51:21,112][20381] Updated weights for policy 1, policy_version 319202 (0.0009) [2023-12-27 07:51:21,129][20347] Updated weights for policy 0, policy_version 319412 (0.0007) [2023-12-27 07:51:21,163][20291] KL-divergence is very high: 174.7406 [2023-12-27 07:51:21,170][20291] KL-divergence is very high: 138.6015 [2023-12-27 07:51:21,194][20347] Updated weights for policy 0, policy_version 319422 (0.0006) [2023-12-27 07:51:21,784][20381] Updated weights for policy 1, policy_version 319212 (0.0007) [2023-12-27 07:51:21,834][20381] Updated weights for policy 1, policy_version 319222 (0.0008) [2023-12-27 07:51:21,888][20381] Updated weights for policy 1, policy_version 319232 (0.0008) [2023-12-27 07:51:21,898][20347] Updated weights for policy 0, policy_version 319432 (0.0010) [2023-12-27 07:51:21,948][20347] Updated weights for policy 0, policy_version 319442 (0.0010) [2023-12-27 07:51:22,002][20347] Updated weights for policy 0, policy_version 319452 (0.0010) [2023-12-27 07:51:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 163536896. Throughput: 0: 10371.0, 1: 10345.5. Samples: 163507292. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:22,363][19170] Avg episode reward: [(0, '846.140'), (1, '934.650')] [2023-12-27 07:51:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000319240_81739776.pth... [2023-12-27 07:51:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000319456_81797120.pth... [2023-12-27 07:51:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000318240_81485824.pth [2023-12-27 07:51:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000318056_81436672.pth [2023-12-27 07:51:22,604][20381] Updated weights for policy 1, policy_version 319242 (0.0006) [2023-12-27 07:51:22,660][20381] Updated weights for policy 1, policy_version 319252 (0.0007) [2023-12-27 07:51:22,695][20347] Updated weights for policy 0, policy_version 319462 (0.0010) [2023-12-27 07:51:22,719][20381] Updated weights for policy 1, policy_version 319262 (0.0006) [2023-12-27 07:51:22,752][20347] Updated weights for policy 0, policy_version 319472 (0.0010) [2023-12-27 07:51:22,772][20381] Updated weights for policy 1, policy_version 319272 (0.0006) [2023-12-27 07:51:22,813][20347] Updated weights for policy 0, policy_version 319482 (0.0009) [2023-12-27 07:51:23,470][20347] Updated weights for policy 0, policy_version 319492 (0.0009) [2023-12-27 07:51:23,505][20381] Updated weights for policy 1, policy_version 319282 (0.0008) [2023-12-27 07:51:23,522][20347] Updated weights for policy 0, policy_version 319502 (0.0006) [2023-12-27 07:51:23,547][20381] Updated weights for policy 1, policy_version 319292 (0.0005) [2023-12-27 07:51:23,574][20347] Updated weights for policy 0, policy_version 319512 (0.0005) [2023-12-27 07:51:23,593][20381] Updated weights for policy 1, policy_version 319302 (0.0006) [2023-12-27 07:51:24,233][20347] Updated weights for policy 0, policy_version 319522 (0.0006) [2023-12-27 07:51:24,243][20381] Updated weights for policy 1, policy_version 319312 (0.0010) [2023-12-27 07:51:24,291][20347] Updated weights for policy 0, policy_version 319532 (0.0006) [2023-12-27 07:51:24,296][20381] Updated weights for policy 1, policy_version 319322 (0.0011) [2023-12-27 07:51:24,347][20347] Updated weights for policy 0, policy_version 319542 (0.0006) [2023-12-27 07:51:24,353][20381] Updated weights for policy 1, policy_version 319332 (0.0011) [2023-12-27 07:51:24,392][20347] Updated weights for policy 0, policy_version 319552 (0.0005) [2023-12-27 07:51:24,957][20347] Updated weights for policy 0, policy_version 319562 (0.0008) [2023-12-27 07:51:25,014][20347] Updated weights for policy 0, policy_version 319572 (0.0006) [2023-12-27 07:51:25,059][20381] Updated weights for policy 1, policy_version 319342 (0.0008) [2023-12-27 07:51:25,066][20347] Updated weights for policy 0, policy_version 319582 (0.0005) [2023-12-27 07:51:25,116][20381] Updated weights for policy 1, policy_version 319352 (0.0009) [2023-12-27 07:51:25,174][20381] Updated weights for policy 1, policy_version 319362 (0.0008) [2023-12-27 07:51:25,655][20347] Updated weights for policy 0, policy_version 319592 (0.0007) [2023-12-27 07:51:25,712][20347] Updated weights for policy 0, policy_version 319602 (0.0007) [2023-12-27 07:51:25,769][20347] Updated weights for policy 0, policy_version 319612 (0.0005) [2023-12-27 07:51:25,842][20381] Updated weights for policy 1, policy_version 319372 (0.0009) [2023-12-27 07:51:25,904][20381] Updated weights for policy 1, policy_version 319382 (0.0011) [2023-12-27 07:51:25,915][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-27 07:51:26,299][20347] Updated weights for policy 0, policy_version 319622 (0.0005) [2023-12-27 07:51:26,342][20347] Updated weights for policy 0, policy_version 319632 (0.0005) [2023-12-27 07:51:26,387][20347] Updated weights for policy 0, policy_version 319642 (0.0005) [2023-12-27 07:51:26,559][20381] Updated weights for policy 1, policy_version 319392 (0.0010) [2023-12-27 07:51:26,612][20381] Updated weights for policy 1, policy_version 319402 (0.0008) [2023-12-27 07:51:26,658][20381] Updated weights for policy 1, policy_version 319412 (0.0005) [2023-12-27 07:51:26,928][20347] Updated weights for policy 0, policy_version 319652 (0.0005) [2023-12-27 07:51:26,992][20347] Updated weights for policy 0, policy_version 319662 (0.0005) [2023-12-27 07:51:27,052][20347] Updated weights for policy 0, policy_version 319672 (0.0007) [2023-12-27 07:51:27,254][20381] Updated weights for policy 1, policy_version 319422 (0.0007) [2023-12-27 07:51:27,303][20381] Updated weights for policy 1, policy_version 319432 (0.0008) [2023-12-27 07:51:27,353][20381] Updated weights for policy 1, policy_version 319442 (0.0006) [2023-12-27 07:51:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 163643392. Throughput: 0: 10424.9, 1: 10329.1. Samples: 163631556. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:27,363][19170] Avg episode reward: [(0, '904.720'), (1, '922.000')] [2023-12-27 07:51:27,602][20347] Updated weights for policy 0, policy_version 319682 (0.0007) [2023-12-27 07:51:27,652][20347] Updated weights for policy 0, policy_version 319692 (0.0008) [2023-12-27 07:51:27,696][20347] Updated weights for policy 0, policy_version 319702 (0.0007) [2023-12-27 07:51:27,744][20347] Updated weights for policy 0, policy_version 319712 (0.0007) [2023-12-27 07:51:27,948][20381] Updated weights for policy 1, policy_version 319452 (0.0007) [2023-12-27 07:51:28,005][20381] Updated weights for policy 1, policy_version 319462 (0.0010) [2023-12-27 07:51:28,063][20381] Updated weights for policy 1, policy_version 319472 (0.0006) [2023-12-27 07:51:28,405][20347] Updated weights for policy 0, policy_version 319722 (0.0007) [2023-12-27 07:51:28,461][20347] Updated weights for policy 0, policy_version 319732 (0.0007) [2023-12-27 07:51:28,525][20347] Updated weights for policy 0, policy_version 319742 (0.0007) [2023-12-27 07:51:28,690][20381] Updated weights for policy 1, policy_version 319482 (0.0006) [2023-12-27 07:51:28,744][20381] Updated weights for policy 1, policy_version 319492 (0.0007) [2023-12-27 07:51:28,793][20381] Updated weights for policy 1, policy_version 319502 (0.0008) [2023-12-27 07:51:28,851][20381] Updated weights for policy 1, policy_version 319512 (0.0008) [2023-12-27 07:51:29,127][20347] Updated weights for policy 0, policy_version 319752 (0.0006) [2023-12-27 07:51:29,180][20347] Updated weights for policy 0, policy_version 319762 (0.0008) [2023-12-27 07:51:29,243][20347] Updated weights for policy 0, policy_version 319772 (0.0008) [2023-12-27 07:51:29,552][20381] Updated weights for policy 1, policy_version 319522 (0.0009) [2023-12-27 07:51:29,603][20381] Updated weights for policy 1, policy_version 319532 (0.0010) [2023-12-27 07:51:29,658][20381] Updated weights for policy 1, policy_version 319542 (0.0010) [2023-12-27 07:51:29,908][20347] Updated weights for policy 0, policy_version 319782 (0.0006) [2023-12-27 07:51:29,966][20347] Updated weights for policy 0, policy_version 319792 (0.0006) [2023-12-27 07:51:30,024][20347] Updated weights for policy 0, policy_version 319802 (0.0006) [2023-12-27 07:51:30,376][20381] Updated weights for policy 1, policy_version 319552 (0.0006) [2023-12-27 07:51:30,432][20381] Updated weights for policy 1, policy_version 319562 (0.0005) [2023-12-27 07:51:30,485][20381] Updated weights for policy 1, policy_version 319572 (0.0005) [2023-12-27 07:51:30,692][20347] Updated weights for policy 0, policy_version 319812 (0.0006) [2023-12-27 07:51:30,757][20347] Updated weights for policy 0, policy_version 319822 (0.0006) [2023-12-27 07:51:30,811][20347] Updated weights for policy 0, policy_version 319832 (0.0008) [2023-12-27 07:51:31,001][20381] Updated weights for policy 1, policy_version 319582 (0.0006) [2023-12-27 07:51:31,061][20381] Updated weights for policy 1, policy_version 319592 (0.0008) [2023-12-27 07:51:31,112][20381] Updated weights for policy 1, policy_version 319602 (0.0009) [2023-12-27 07:51:31,507][20347] Updated weights for policy 0, policy_version 319842 (0.0007) [2023-12-27 07:51:31,552][20347] Updated weights for policy 0, policy_version 319852 (0.0007) [2023-12-27 07:51:31,605][20347] Updated weights for policy 0, policy_version 319862 (0.0006) [2023-12-27 07:51:31,626][20291] KL-divergence is very high: 172.5358 [2023-12-27 07:51:31,640][20291] KL-divergence is very high: 146.5673 [2023-12-27 07:51:31,667][20347] Updated weights for policy 0, policy_version 319872 (0.0007) [2023-12-27 07:51:31,782][20381] Updated weights for policy 1, policy_version 319612 (0.0007) [2023-12-27 07:51:31,837][20381] Updated weights for policy 1, policy_version 319622 (0.0007) [2023-12-27 07:51:31,887][20381] Updated weights for policy 1, policy_version 319632 (0.0006) [2023-12-27 07:51:32,296][20291] KL-divergence is very high: 109.0237 [2023-12-27 07:51:32,309][20291] KL-divergence is very high: 242.2613 [2023-12-27 07:51:32,327][20291] KL-divergence is very high: 183.4386 [2023-12-27 07:51:32,346][20291] KL-divergence is very high: 115.5162 [2023-12-27 07:51:32,352][20347] Updated weights for policy 0, policy_version 319882 (0.0008) [2023-12-27 07:51:32,359][20291] KL-divergence is very high: 136.3935 [2023-12-27 07:51:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20716.1). Total num frames: 163749888. Throughput: 0: 10508.1, 1: 10405.7. Samples: 163763996. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:32,363][19170] Avg episode reward: [(0, '875.680'), (1, '921.260')] [2023-12-27 07:51:32,399][20291] KL-divergence is very high: 316.1671 [2023-12-27 07:51:32,420][20347] Updated weights for policy 0, policy_version 319892 (0.0008) [2023-12-27 07:51:32,445][20291] KL-divergence is very high: 322.0091 [2023-12-27 07:51:32,469][20347] Updated weights for policy 0, policy_version 319902 (0.0007) [2023-12-27 07:51:32,542][20381] Updated weights for policy 1, policy_version 319642 (0.0005) [2023-12-27 07:51:32,589][20381] Updated weights for policy 1, policy_version 319652 (0.0006) [2023-12-27 07:51:32,639][20381] Updated weights for policy 1, policy_version 319662 (0.0008) [2023-12-27 07:51:32,692][20381] Updated weights for policy 1, policy_version 319672 (0.0008) [2023-12-27 07:51:33,220][20347] Updated weights for policy 0, policy_version 319912 (0.0007) [2023-12-27 07:51:33,224][20291] KL-divergence is very high: 110.6602 [2023-12-27 07:51:33,272][20347] Updated weights for policy 0, policy_version 319922 (0.0006) [2023-12-27 07:51:33,292][20381] Updated weights for policy 1, policy_version 319682 (0.0007) [2023-12-27 07:51:33,312][20291] KL-divergence is very high: 141.3159 [2023-12-27 07:51:33,330][20347] Updated weights for policy 0, policy_version 319932 (0.0006) [2023-12-27 07:51:33,348][20381] Updated weights for policy 1, policy_version 319692 (0.0007) [2023-12-27 07:51:33,405][20381] Updated weights for policy 1, policy_version 319702 (0.0005) [2023-12-27 07:51:33,972][20381] Updated weights for policy 1, policy_version 319712 (0.0006) [2023-12-27 07:51:34,017][20381] Updated weights for policy 1, policy_version 319722 (0.0005) [2023-12-27 07:51:34,061][20347] Updated weights for policy 0, policy_version 319942 (0.0008) [2023-12-27 07:51:34,080][20381] Updated weights for policy 1, policy_version 319732 (0.0005) [2023-12-27 07:51:34,116][20347] Updated weights for policy 0, policy_version 319952 (0.0007) [2023-12-27 07:51:34,181][20347] Updated weights for policy 0, policy_version 319962 (0.0008) [2023-12-27 07:51:34,666][20381] Updated weights for policy 1, policy_version 319742 (0.0007) [2023-12-27 07:51:34,716][20381] Updated weights for policy 1, policy_version 319752 (0.0008) [2023-12-27 07:51:34,773][20381] Updated weights for policy 1, policy_version 319762 (0.0008) [2023-12-27 07:51:34,897][20347] Updated weights for policy 0, policy_version 319972 (0.0007) [2023-12-27 07:51:34,955][20347] Updated weights for policy 0, policy_version 319982 (0.0007) [2023-12-27 07:51:35,011][20347] Updated weights for policy 0, policy_version 319992 (0.0008) [2023-12-27 07:51:35,409][20381] Updated weights for policy 1, policy_version 319772 (0.0008) [2023-12-27 07:51:35,455][20381] Updated weights for policy 1, policy_version 319782 (0.0008) [2023-12-27 07:51:35,511][20381] Updated weights for policy 1, policy_version 319792 (0.0008) [2023-12-27 07:51:35,739][20347] Updated weights for policy 0, policy_version 320002 (0.0008) [2023-12-27 07:51:35,797][20347] Updated weights for policy 0, policy_version 320012 (0.0008) [2023-12-27 07:51:35,854][20347] Updated weights for policy 0, policy_version 320022 (0.0007) [2023-12-27 07:51:35,902][20347] Updated weights for policy 0, policy_version 320032 (0.0007) [2023-12-27 07:51:36,212][20381] Updated weights for policy 1, policy_version 319802 (0.0008) [2023-12-27 07:51:36,265][20381] Updated weights for policy 1, policy_version 319812 (0.0007) [2023-12-27 07:51:36,334][20381] Updated weights for policy 1, policy_version 319822 (0.0006) [2023-12-27 07:51:36,402][20381] Updated weights for policy 1, policy_version 319832 (0.0006) [2023-12-27 07:51:36,577][20347] Updated weights for policy 0, policy_version 320042 (0.0008) [2023-12-27 07:51:36,639][20347] Updated weights for policy 0, policy_version 320052 (0.0007) [2023-12-27 07:51:36,703][20347] Updated weights for policy 0, policy_version 320062 (0.0008) [2023-12-27 07:51:37,024][20381] Updated weights for policy 1, policy_version 319842 (0.0007) [2023-12-27 07:51:37,076][20381] Updated weights for policy 1, policy_version 319852 (0.0007) [2023-12-27 07:51:37,143][20381] Updated weights for policy 1, policy_version 319862 (0.0006) [2023-12-27 07:51:37,333][20347] Updated weights for policy 0, policy_version 320072 (0.0010) [2023-12-27 07:51:37,361][20291] KL-divergence is very high: 130.0953 [2023-12-27 07:51:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 163856384. Throughput: 0: 10451.5, 1: 10453.0. Samples: 163826684. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:37,363][19170] Avg episode reward: [(0, '626.640'), (1, '955.530')] [2023-12-27 07:51:37,367][20291] KL-divergence is very high: 138.0705 [2023-12-27 07:51:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000319864_81903616.pth... [2023-12-27 07:51:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000318632_81584128.pth [2023-12-27 07:51:37,390][20347] Updated weights for policy 0, policy_version 320082 (0.0011) [2023-12-27 07:51:37,454][20347] Updated weights for policy 0, policy_version 320092 (0.0007) [2023-12-27 07:51:37,478][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000320096_81960960.pth... [2023-12-27 07:51:37,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000318848_81641472.pth [2023-12-27 07:51:37,755][20381] Updated weights for policy 1, policy_version 319872 (0.0008) [2023-12-27 07:51:37,813][20381] Updated weights for policy 1, policy_version 319882 (0.0005) [2023-12-27 07:51:37,862][20381] Updated weights for policy 1, policy_version 319892 (0.0005) [2023-12-27 07:51:38,041][20291] KL-divergence is very high: 149.9333 [2023-12-27 07:51:38,047][20347] Updated weights for policy 0, policy_version 320102 (0.0005) [2023-12-27 07:51:38,108][20347] Updated weights for policy 0, policy_version 320112 (0.0006) [2023-12-27 07:51:38,135][20291] KL-divergence is very high: 250.6746 [2023-12-27 07:51:38,157][20347] Updated weights for policy 0, policy_version 320122 (0.0008) [2023-12-27 07:51:38,172][20291] KL-divergence is very high: 141.8768 [2023-12-27 07:51:38,425][20381] Updated weights for policy 1, policy_version 319902 (0.0006) [2023-12-27 07:51:38,474][20381] Updated weights for policy 1, policy_version 319912 (0.0009) [2023-12-27 07:51:38,520][20381] Updated weights for policy 1, policy_version 319922 (0.0007) [2023-12-27 07:51:38,825][20347] Updated weights for policy 0, policy_version 320132 (0.0008) [2023-12-27 07:51:38,855][20291] KL-divergence is very high: 845.1163 [2023-12-27 07:51:38,886][20347] Updated weights for policy 0, policy_version 320142 (0.0007) [2023-12-27 07:51:38,888][20291] KL-divergence is very high: 183.3237 [2023-12-27 07:51:38,908][20291] KL-divergence is very high: 899.0346 [2023-12-27 07:51:38,948][20347] Updated weights for policy 0, policy_version 320152 (0.0007) [2023-12-27 07:51:38,954][20291] KL-divergence is very high: 437.5825 [2023-12-27 07:51:39,225][20381] Updated weights for policy 1, policy_version 319932 (0.0006) [2023-12-27 07:51:39,290][20381] Updated weights for policy 1, policy_version 319942 (0.0007) [2023-12-27 07:51:39,352][20381] Updated weights for policy 1, policy_version 319952 (0.0010) [2023-12-27 07:51:39,655][20347] Updated weights for policy 0, policy_version 320162 (0.0008) [2023-12-27 07:51:39,711][20347] Updated weights for policy 0, policy_version 320172 (0.0007) [2023-12-27 07:51:39,767][20347] Updated weights for policy 0, policy_version 320182 (0.0007) [2023-12-27 07:51:39,828][20347] Updated weights for policy 0, policy_version 320192 (0.0008) [2023-12-27 07:51:40,039][20381] Updated weights for policy 1, policy_version 319962 (0.0010) [2023-12-27 07:51:40,085][20381] Updated weights for policy 1, policy_version 319972 (0.0005) [2023-12-27 07:51:40,141][20381] Updated weights for policy 1, policy_version 319982 (0.0007) [2023-12-27 07:51:40,195][20381] Updated weights for policy 1, policy_version 319992 (0.0009) [2023-12-27 07:51:40,539][20347] Updated weights for policy 0, policy_version 320202 (0.0008) [2023-12-27 07:51:40,601][20347] Updated weights for policy 0, policy_version 320212 (0.0008) [2023-12-27 07:51:40,653][20347] Updated weights for policy 0, policy_version 320222 (0.0007) [2023-12-27 07:51:40,834][20381] Updated weights for policy 1, policy_version 320002 (0.0008) [2023-12-27 07:51:40,899][20381] Updated weights for policy 1, policy_version 320012 (0.0008) [2023-12-27 07:51:40,952][20381] Updated weights for policy 1, policy_version 320022 (0.0008) [2023-12-27 07:51:41,402][20347] Updated weights for policy 0, policy_version 320232 (0.0008) [2023-12-27 07:51:41,461][20347] Updated weights for policy 0, policy_version 320242 (0.0008) [2023-12-27 07:51:41,512][20347] Updated weights for policy 0, policy_version 320252 (0.0008) [2023-12-27 07:51:41,649][20381] Updated weights for policy 1, policy_version 320032 (0.0008) [2023-12-27 07:51:41,714][20381] Updated weights for policy 1, policy_version 320042 (0.0008) [2023-12-27 07:51:41,780][20381] Updated weights for policy 1, policy_version 320052 (0.0008) [2023-12-27 07:51:42,256][20347] Updated weights for policy 0, policy_version 320262 (0.0008) [2023-12-27 07:51:42,312][20347] Updated weights for policy 0, policy_version 320272 (0.0008) [2023-12-27 07:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 163954688. Throughput: 0: 10373.5, 1: 10556.4. Samples: 163951300. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:42,363][19170] Avg episode reward: [(0, '249.520'), (1, '908.820')] [2023-12-27 07:51:42,373][20347] Updated weights for policy 0, policy_version 320282 (0.0008) [2023-12-27 07:51:42,417][20381] Updated weights for policy 1, policy_version 320062 (0.0009) [2023-12-27 07:51:42,467][20381] Updated weights for policy 1, policy_version 320072 (0.0009) [2023-12-27 07:51:42,513][20381] Updated weights for policy 1, policy_version 320082 (0.0010) [2023-12-27 07:51:42,997][20347] Updated weights for policy 0, policy_version 320292 (0.0007) [2023-12-27 07:51:43,053][20347] Updated weights for policy 0, policy_version 320302 (0.0008) [2023-12-27 07:51:43,099][20347] Updated weights for policy 0, policy_version 320312 (0.0007) [2023-12-27 07:51:43,275][20381] Updated weights for policy 1, policy_version 320092 (0.0009) [2023-12-27 07:51:43,317][20381] Updated weights for policy 1, policy_version 320102 (0.0007) [2023-12-27 07:51:43,362][20381] Updated weights for policy 1, policy_version 320112 (0.0008) [2023-12-27 07:51:43,756][20347] Updated weights for policy 0, policy_version 320322 (0.0007) [2023-12-27 07:51:43,807][20347] Updated weights for policy 0, policy_version 320332 (0.0005) [2023-12-27 07:51:43,860][20347] Updated weights for policy 0, policy_version 320342 (0.0005) [2023-12-27 07:51:43,915][20347] Updated weights for policy 0, policy_version 320352 (0.0005) [2023-12-27 07:51:44,115][20381] Updated weights for policy 1, policy_version 320122 (0.0008) [2023-12-27 07:51:44,163][20381] Updated weights for policy 1, policy_version 320132 (0.0008) [2023-12-27 07:51:44,209][20381] Updated weights for policy 1, policy_version 320142 (0.0008) [2023-12-27 07:51:44,252][20381] Updated weights for policy 1, policy_version 320152 (0.0007) [2023-12-27 07:51:44,442][20291] KL-divergence is very high: 130.9534 [2023-12-27 07:51:44,452][20291] KL-divergence is very high: 132.2996 [2023-12-27 07:51:44,459][20291] KL-divergence is very high: 100.5888 [2023-12-27 07:51:44,475][20291] KL-divergence is very high: 108.1531 [2023-12-27 07:51:44,498][20291] KL-divergence is very high: 155.8559 [2023-12-27 07:51:44,504][20291] KL-divergence is very high: 142.1762 [2023-12-27 07:51:44,508][20347] Updated weights for policy 0, policy_version 320362 (0.0005) [2023-12-27 07:51:44,540][20291] KL-divergence is very high: 105.5892 [2023-12-27 07:51:44,559][20347] Updated weights for policy 0, policy_version 320372 (0.0005) [2023-12-27 07:51:44,620][20347] Updated weights for policy 0, policy_version 320382 (0.0005) [2023-12-27 07:51:45,023][20381] Updated weights for policy 1, policy_version 320162 (0.0009) [2023-12-27 07:51:45,084][20381] Updated weights for policy 1, policy_version 320172 (0.0008) [2023-12-27 07:51:45,139][20347] Updated weights for policy 0, policy_version 320392 (0.0005) [2023-12-27 07:51:45,146][20381] Updated weights for policy 1, policy_version 320182 (0.0007) [2023-12-27 07:51:45,192][20347] Updated weights for policy 0, policy_version 320402 (0.0007) [2023-12-27 07:51:45,255][20347] Updated weights for policy 0, policy_version 320412 (0.0007) [2023-12-27 07:51:45,883][20381] Updated weights for policy 1, policy_version 320192 (0.0007) [2023-12-27 07:51:45,895][20347] Updated weights for policy 0, policy_version 320422 (0.0006) [2023-12-27 07:51:45,940][20381] Updated weights for policy 1, policy_version 320202 (0.0008) [2023-12-27 07:51:45,944][20347] Updated weights for policy 0, policy_version 320432 (0.0005) [2023-12-27 07:51:45,990][20347] Updated weights for policy 0, policy_version 320442 (0.0005) [2023-12-27 07:51:45,994][20381] Updated weights for policy 1, policy_version 320212 (0.0007) [2023-12-27 07:51:46,553][20347] Updated weights for policy 0, policy_version 320452 (0.0006) [2023-12-27 07:51:46,610][20347] Updated weights for policy 0, policy_version 320462 (0.0007) [2023-12-27 07:51:46,627][20381] Updated weights for policy 1, policy_version 320222 (0.0006) [2023-12-27 07:51:46,662][20347] Updated weights for policy 0, policy_version 320472 (0.0007) [2023-12-27 07:51:46,678][20381] Updated weights for policy 1, policy_version 320232 (0.0006) [2023-12-27 07:51:46,733][20381] Updated weights for policy 1, policy_version 320242 (0.0010) [2023-12-27 07:51:47,308][20347] Updated weights for policy 0, policy_version 320482 (0.0008) [2023-12-27 07:51:47,319][20381] Updated weights for policy 1, policy_version 320252 (0.0010) [2023-12-27 07:51:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 164061184. Throughput: 0: 10501.8, 1: 10517.7. Samples: 164077124. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:47,363][19170] Avg episode reward: [(0, '288.490'), (1, '885.110')] [2023-12-27 07:51:47,366][20347] Updated weights for policy 0, policy_version 320492 (0.0008) [2023-12-27 07:51:47,380][20381] Updated weights for policy 1, policy_version 320262 (0.0010) [2023-12-27 07:51:47,416][20347] Updated weights for policy 0, policy_version 320502 (0.0006) [2023-12-27 07:51:47,441][20381] Updated weights for policy 1, policy_version 320272 (0.0010) [2023-12-27 07:51:47,468][20347] Updated weights for policy 0, policy_version 320512 (0.0006) [2023-12-27 07:51:48,046][20381] Updated weights for policy 1, policy_version 320282 (0.0010) [2023-12-27 07:51:48,100][20381] Updated weights for policy 1, policy_version 320292 (0.0008) [2023-12-27 07:51:48,106][20347] Updated weights for policy 0, policy_version 320522 (0.0005) [2023-12-27 07:51:48,165][20347] Updated weights for policy 0, policy_version 320532 (0.0006) [2023-12-27 07:51:48,166][20381] Updated weights for policy 1, policy_version 320302 (0.0006) [2023-12-27 07:51:48,221][20381] Updated weights for policy 1, policy_version 320312 (0.0008) [2023-12-27 07:51:48,223][20347] Updated weights for policy 0, policy_version 320542 (0.0006) [2023-12-27 07:51:48,910][20347] Updated weights for policy 0, policy_version 320552 (0.0007) [2023-12-27 07:51:48,931][20381] Updated weights for policy 1, policy_version 320322 (0.0006) [2023-12-27 07:51:48,964][20347] Updated weights for policy 0, policy_version 320562 (0.0005) [2023-12-27 07:51:48,985][20381] Updated weights for policy 1, policy_version 320332 (0.0006) [2023-12-27 07:51:49,023][20347] Updated weights for policy 0, policy_version 320572 (0.0008) [2023-12-27 07:51:49,048][20381] Updated weights for policy 1, policy_version 320342 (0.0005) [2023-12-27 07:51:49,672][20347] Updated weights for policy 0, policy_version 320582 (0.0008) [2023-12-27 07:51:49,734][20381] Updated weights for policy 1, policy_version 320352 (0.0006) [2023-12-27 07:51:49,737][20347] Updated weights for policy 0, policy_version 320592 (0.0008) [2023-12-27 07:51:49,789][20381] Updated weights for policy 1, policy_version 320362 (0.0007) [2023-12-27 07:51:49,795][20347] Updated weights for policy 0, policy_version 320603 (0.0006) [2023-12-27 07:51:49,845][20381] Updated weights for policy 1, policy_version 320372 (0.0007) [2023-12-27 07:51:50,499][20347] Updated weights for policy 0, policy_version 320613 (0.0007) [2023-12-27 07:51:50,543][20381] Updated weights for policy 1, policy_version 320382 (0.0006) [2023-12-27 07:51:50,545][20347] Updated weights for policy 0, policy_version 320623 (0.0006) [2023-12-27 07:51:50,600][20347] Updated weights for policy 0, policy_version 320633 (0.0008) [2023-12-27 07:51:50,603][20381] Updated weights for policy 1, policy_version 320392 (0.0008) [2023-12-27 07:51:50,660][20381] Updated weights for policy 1, policy_version 320402 (0.0007) [2023-12-27 07:51:51,324][20347] Updated weights for policy 0, policy_version 320643 (0.0008) [2023-12-27 07:51:51,393][20347] Updated weights for policy 0, policy_version 320653 (0.0008) [2023-12-27 07:51:51,407][20381] Updated weights for policy 1, policy_version 320412 (0.0007) [2023-12-27 07:51:51,455][20347] Updated weights for policy 0, policy_version 320663 (0.0007) [2023-12-27 07:51:51,459][20381] Updated weights for policy 1, policy_version 320422 (0.0005) [2023-12-27 07:51:51,513][20381] Updated weights for policy 1, policy_version 320432 (0.0005) [2023-12-27 07:51:52,120][20381] Updated weights for policy 1, policy_version 320442 (0.0006) [2023-12-27 07:51:52,179][20381] Updated weights for policy 1, policy_version 320452 (0.0008) [2023-12-27 07:51:52,185][20347] Updated weights for policy 0, policy_version 320673 (0.0008) [2023-12-27 07:51:52,232][20381] Updated weights for policy 1, policy_version 320462 (0.0006) [2023-12-27 07:51:52,240][20347] Updated weights for policy 0, policy_version 320683 (0.0006) [2023-12-27 07:51:52,296][20381] Updated weights for policy 1, policy_version 320472 (0.0006) [2023-12-27 07:51:52,298][20347] Updated weights for policy 0, policy_version 320693 (0.0006) [2023-12-27 07:51:52,362][20347] Updated weights for policy 0, policy_version 320703 (0.0008) [2023-12-27 07:51:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 164167680. Throughput: 0: 10484.7, 1: 10525.0. Samples: 164139152. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:52,363][19170] Avg episode reward: [(0, '422.790'), (1, '802.680')] [2023-12-27 07:51:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000320472_82059264.pth... [2023-12-27 07:51:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000320704_82116608.pth... [2023-12-27 07:51:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000319240_81739776.pth [2023-12-27 07:51:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000319456_81797120.pth [2023-12-27 07:51:52,985][20381] Updated weights for policy 1, policy_version 320482 (0.0006) [2023-12-27 07:51:53,045][20381] Updated weights for policy 1, policy_version 320492 (0.0006) [2023-12-27 07:51:53,054][20347] Updated weights for policy 0, policy_version 320713 (0.0005) [2023-12-27 07:51:53,096][20381] Updated weights for policy 1, policy_version 320502 (0.0005) [2023-12-27 07:51:53,100][20347] Updated weights for policy 0, policy_version 320723 (0.0005) [2023-12-27 07:51:53,144][20347] Updated weights for policy 0, policy_version 320733 (0.0005) [2023-12-27 07:51:53,708][20347] Updated weights for policy 0, policy_version 320743 (0.0006) [2023-12-27 07:51:53,714][20381] Updated weights for policy 1, policy_version 320512 (0.0005) [2023-12-27 07:51:53,765][20347] Updated weights for policy 0, policy_version 320753 (0.0005) [2023-12-27 07:51:53,770][20381] Updated weights for policy 1, policy_version 320522 (0.0007) [2023-12-27 07:51:53,822][20347] Updated weights for policy 0, policy_version 320763 (0.0006) [2023-12-27 07:51:53,824][20381] Updated weights for policy 1, policy_version 320532 (0.0005) [2023-12-27 07:51:54,519][20381] Updated weights for policy 1, policy_version 320542 (0.0007) [2023-12-27 07:51:54,543][20347] Updated weights for policy 0, policy_version 320773 (0.0007) [2023-12-27 07:51:54,579][20381] Updated weights for policy 1, policy_version 320552 (0.0008) [2023-12-27 07:51:54,600][20347] Updated weights for policy 0, policy_version 320783 (0.0007) [2023-12-27 07:51:54,630][20381] Updated weights for policy 1, policy_version 320562 (0.0008) [2023-12-27 07:51:54,653][20347] Updated weights for policy 0, policy_version 320793 (0.0007) [2023-12-27 07:51:55,272][20381] Updated weights for policy 1, policy_version 320572 (0.0007) [2023-12-27 07:51:55,331][20347] Updated weights for policy 0, policy_version 320803 (0.0007) [2023-12-27 07:51:55,331][20381] Updated weights for policy 1, policy_version 320582 (0.0006) [2023-12-27 07:51:55,382][20381] Updated weights for policy 1, policy_version 320592 (0.0008) [2023-12-27 07:51:55,384][20347] Updated weights for policy 0, policy_version 320813 (0.0005) [2023-12-27 07:51:55,438][20347] Updated weights for policy 0, policy_version 320823 (0.0005) [2023-12-27 07:51:55,983][20347] Updated weights for policy 0, policy_version 320833 (0.0005) [2023-12-27 07:51:55,997][20381] Updated weights for policy 1, policy_version 320602 (0.0007) [2023-12-27 07:51:56,045][20381] Updated weights for policy 1, policy_version 320612 (0.0005) [2023-12-27 07:51:56,047][20347] Updated weights for policy 0, policy_version 320843 (0.0006) [2023-12-27 07:51:56,098][20381] Updated weights for policy 1, policy_version 320622 (0.0006) [2023-12-27 07:51:56,110][20347] Updated weights for policy 0, policy_version 320853 (0.0006) [2023-12-27 07:51:56,158][20381] Updated weights for policy 1, policy_version 320632 (0.0005) [2023-12-27 07:51:56,172][20347] Updated weights for policy 0, policy_version 320863 (0.0005) [2023-12-27 07:51:56,728][20347] Updated weights for policy 0, policy_version 320873 (0.0008) [2023-12-27 07:51:56,778][20347] Updated weights for policy 0, policy_version 320883 (0.0006) [2023-12-27 07:51:56,790][20381] Updated weights for policy 1, policy_version 320642 (0.0006) [2023-12-27 07:51:56,833][20347] Updated weights for policy 0, policy_version 320893 (0.0006) [2023-12-27 07:51:56,838][20381] Updated weights for policy 1, policy_version 320652 (0.0010) [2023-12-27 07:51:56,887][20381] Updated weights for policy 1, policy_version 320662 (0.0006) [2023-12-27 07:51:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 164274176. Throughput: 0: 10476.5, 1: 10527.1. Samples: 164264476. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:51:57,363][19170] Avg episode reward: [(0, '702.220'), (1, '864.090')] [2023-12-27 07:51:57,467][20381] Updated weights for policy 1, policy_version 320672 (0.0007) [2023-12-27 07:51:57,479][20347] Updated weights for policy 0, policy_version 320903 (0.0007) [2023-12-27 07:51:57,518][20381] Updated weights for policy 1, policy_version 320682 (0.0005) [2023-12-27 07:51:57,532][20347] Updated weights for policy 0, policy_version 320913 (0.0007) [2023-12-27 07:51:57,569][20381] Updated weights for policy 1, policy_version 320692 (0.0005) [2023-12-27 07:51:57,586][20347] Updated weights for policy 0, policy_version 320923 (0.0006) [2023-12-27 07:51:58,109][20381] Updated weights for policy 1, policy_version 320702 (0.0005) [2023-12-27 07:51:58,126][20347] Updated weights for policy 0, policy_version 320933 (0.0005) [2023-12-27 07:51:58,171][20381] Updated weights for policy 1, policy_version 320712 (0.0007) [2023-12-27 07:51:58,184][20347] Updated weights for policy 0, policy_version 320943 (0.0007) [2023-12-27 07:51:58,232][20381] Updated weights for policy 1, policy_version 320722 (0.0008) [2023-12-27 07:51:58,239][20347] Updated weights for policy 0, policy_version 320953 (0.0008) [2023-12-27 07:51:58,961][20381] Updated weights for policy 1, policy_version 320732 (0.0008) [2023-12-27 07:51:59,012][20347] Updated weights for policy 0, policy_version 320963 (0.0008) [2023-12-27 07:51:59,023][20381] Updated weights for policy 1, policy_version 320742 (0.0007) [2023-12-27 07:51:59,072][20347] Updated weights for policy 0, policy_version 320973 (0.0007) [2023-12-27 07:51:59,087][20381] Updated weights for policy 1, policy_version 320752 (0.0006) [2023-12-27 07:51:59,123][20347] Updated weights for policy 0, policy_version 320983 (0.0008) [2023-12-27 07:51:59,742][20381] Updated weights for policy 1, policy_version 320762 (0.0007) [2023-12-27 07:51:59,791][20381] Updated weights for policy 1, policy_version 320772 (0.0007) [2023-12-27 07:51:59,805][20347] Updated weights for policy 0, policy_version 320993 (0.0007) [2023-12-27 07:51:59,853][20381] Updated weights for policy 1, policy_version 320782 (0.0008) [2023-12-27 07:51:59,863][20347] Updated weights for policy 0, policy_version 321003 (0.0007) [2023-12-27 07:51:59,917][20381] Updated weights for policy 1, policy_version 320792 (0.0007) [2023-12-27 07:51:59,919][20347] Updated weights for policy 0, policy_version 321013 (0.0007) [2023-12-27 07:51:59,977][20347] Updated weights for policy 0, policy_version 321023 (0.0008) [2023-12-27 07:52:00,495][20381] Updated weights for policy 1, policy_version 320802 (0.0006) [2023-12-27 07:52:00,544][20381] Updated weights for policy 1, policy_version 320812 (0.0006) [2023-12-27 07:52:00,592][20381] Updated weights for policy 1, policy_version 320822 (0.0005) [2023-12-27 07:52:00,680][20347] Updated weights for policy 0, policy_version 321033 (0.0007) [2023-12-27 07:52:00,731][20347] Updated weights for policy 0, policy_version 321043 (0.0005) [2023-12-27 07:52:00,777][20347] Updated weights for policy 0, policy_version 321053 (0.0007) [2023-12-27 07:52:01,205][20381] Updated weights for policy 1, policy_version 320832 (0.0007) [2023-12-27 07:52:01,266][20381] Updated weights for policy 1, policy_version 320842 (0.0008) [2023-12-27 07:52:01,329][20381] Updated weights for policy 1, policy_version 320852 (0.0006) [2023-12-27 07:52:01,450][20347] Updated weights for policy 0, policy_version 321063 (0.0008) [2023-12-27 07:52:01,506][20347] Updated weights for policy 0, policy_version 321073 (0.0007) [2023-12-27 07:52:01,556][20347] Updated weights for policy 0, policy_version 321083 (0.0008) [2023-12-27 07:52:02,036][20381] Updated weights for policy 1, policy_version 320862 (0.0009) [2023-12-27 07:52:02,089][20381] Updated weights for policy 1, policy_version 320872 (0.0011) [2023-12-27 07:52:02,147][20381] Updated weights for policy 1, policy_version 320882 (0.0011) [2023-12-27 07:52:02,208][20347] Updated weights for policy 0, policy_version 321093 (0.0006) [2023-12-27 07:52:02,261][20347] Updated weights for policy 0, policy_version 321103 (0.0007) [2023-12-27 07:52:02,320][20347] Updated weights for policy 0, policy_version 321113 (0.0008) [2023-12-27 07:52:02,362][19170] Fps is (10 sec: 22118.8, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 164388864. Throughput: 0: 10518.0, 1: 10529.4. Samples: 164393108. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:52:02,363][19170] Avg episode reward: [(0, '809.540'), (1, '906.240')] [2023-12-27 07:52:02,803][20381] Updated weights for policy 1, policy_version 320892 (0.0008) [2023-12-27 07:52:02,856][20381] Updated weights for policy 1, policy_version 320902 (0.0005) [2023-12-27 07:52:02,913][20381] Updated weights for policy 1, policy_version 320912 (0.0007) [2023-12-27 07:52:03,024][20347] Updated weights for policy 0, policy_version 321123 (0.0007) [2023-12-27 07:52:03,072][20347] Updated weights for policy 0, policy_version 321133 (0.0008) [2023-12-27 07:52:03,123][20347] Updated weights for policy 0, policy_version 321143 (0.0008) [2023-12-27 07:52:03,430][20381] Updated weights for policy 1, policy_version 320922 (0.0008) [2023-12-27 07:52:03,490][20381] Updated weights for policy 1, policy_version 320932 (0.0009) [2023-12-27 07:52:03,541][20381] Updated weights for policy 1, policy_version 320942 (0.0010) [2023-12-27 07:52:03,586][20381] Updated weights for policy 1, policy_version 320952 (0.0010) [2023-12-27 07:52:03,715][20347] Updated weights for policy 0, policy_version 321153 (0.0005) [2023-12-27 07:52:03,768][20347] Updated weights for policy 0, policy_version 321163 (0.0005) [2023-12-27 07:52:03,828][20347] Updated weights for policy 0, policy_version 321173 (0.0006) [2023-12-27 07:52:03,891][20347] Updated weights for policy 0, policy_version 321183 (0.0008) [2023-12-27 07:52:04,188][20381] Updated weights for policy 1, policy_version 320962 (0.0008) [2023-12-27 07:52:04,250][20381] Updated weights for policy 1, policy_version 320972 (0.0009) [2023-12-27 07:52:04,318][20381] Updated weights for policy 1, policy_version 320982 (0.0008) [2023-12-27 07:52:04,578][20347] Updated weights for policy 0, policy_version 321193 (0.0008) [2023-12-27 07:52:04,625][20347] Updated weights for policy 0, policy_version 321203 (0.0007) [2023-12-27 07:52:04,678][20347] Updated weights for policy 0, policy_version 321213 (0.0007) [2023-12-27 07:52:05,006][20381] Updated weights for policy 1, policy_version 320992 (0.0009) [2023-12-27 07:52:05,053][20381] Updated weights for policy 1, policy_version 321002 (0.0011) [2023-12-27 07:52:05,106][20381] Updated weights for policy 1, policy_version 321012 (0.0011) [2023-12-27 07:52:05,358][20347] Updated weights for policy 0, policy_version 321223 (0.0007) [2023-12-27 07:52:05,404][20347] Updated weights for policy 0, policy_version 321233 (0.0007) [2023-12-27 07:52:05,451][20347] Updated weights for policy 0, policy_version 321243 (0.0007) [2023-12-27 07:52:05,810][20381] Updated weights for policy 1, policy_version 321022 (0.0011) [2023-12-27 07:52:05,865][20381] Updated weights for policy 1, policy_version 321032 (0.0010) [2023-12-27 07:52:05,924][20381] Updated weights for policy 1, policy_version 321042 (0.0008) [2023-12-27 07:52:06,170][20347] Updated weights for policy 0, policy_version 321253 (0.0007) [2023-12-27 07:52:06,234][20347] Updated weights for policy 0, policy_version 321263 (0.0007) [2023-12-27 07:52:06,298][20347] Updated weights for policy 0, policy_version 321273 (0.0006) [2023-12-27 07:52:06,502][20381] Updated weights for policy 1, policy_version 321052 (0.0007) [2023-12-27 07:52:06,560][20381] Updated weights for policy 1, policy_version 321062 (0.0006) [2023-12-27 07:52:06,616][20381] Updated weights for policy 1, policy_version 321072 (0.0007) [2023-12-27 07:52:06,932][20347] Updated weights for policy 0, policy_version 321283 (0.0005) [2023-12-27 07:52:06,993][20347] Updated weights for policy 0, policy_version 321293 (0.0005) [2023-12-27 07:52:07,050][20347] Updated weights for policy 0, policy_version 321303 (0.0005) [2023-12-27 07:52:07,312][20381] Updated weights for policy 1, policy_version 321082 (0.0007) [2023-12-27 07:52:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 164487168. Throughput: 0: 10526.2, 1: 10591.2. Samples: 164457576. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:52:07,363][19170] Avg episode reward: [(0, '686.840'), (1, '912.490')] [2023-12-27 07:52:07,369][20381] Updated weights for policy 1, policy_version 321092 (0.0008) [2023-12-27 07:52:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000321312_82272256.pth... [2023-12-27 07:52:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000320096_81960960.pth [2023-12-27 07:52:07,427][20381] Updated weights for policy 1, policy_version 321102 (0.0007) [2023-12-27 07:52:07,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000321112_82223104.pth... [2023-12-27 07:52:07,484][20381] Updated weights for policy 1, policy_version 321112 (0.0007) [2023-12-27 07:52:07,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000319864_81903616.pth [2023-12-27 07:52:07,651][20347] Updated weights for policy 0, policy_version 321313 (0.0006) [2023-12-27 07:52:07,705][20347] Updated weights for policy 0, policy_version 321323 (0.0010) [2023-12-27 07:52:07,755][20347] Updated weights for policy 0, policy_version 321333 (0.0010) [2023-12-27 07:52:07,805][20347] Updated weights for policy 0, policy_version 321343 (0.0007) [2023-12-27 07:52:08,190][20381] Updated weights for policy 1, policy_version 321122 (0.0007) [2023-12-27 07:52:08,251][20381] Updated weights for policy 1, policy_version 321132 (0.0007) [2023-12-27 07:52:08,315][20381] Updated weights for policy 1, policy_version 321142 (0.0006) [2023-12-27 07:52:08,475][20347] Updated weights for policy 0, policy_version 321353 (0.0010) [2023-12-27 07:52:08,530][20347] Updated weights for policy 0, policy_version 321363 (0.0010) [2023-12-27 07:52:08,586][20347] Updated weights for policy 0, policy_version 321373 (0.0011) [2023-12-27 07:52:08,955][20381] Updated weights for policy 1, policy_version 321152 (0.0007) [2023-12-27 07:52:09,004][20381] Updated weights for policy 1, policy_version 321162 (0.0007) [2023-12-27 07:52:09,055][20381] Updated weights for policy 1, policy_version 321172 (0.0007) [2023-12-27 07:52:09,278][20347] Updated weights for policy 0, policy_version 321383 (0.0011) [2023-12-27 07:52:09,337][20347] Updated weights for policy 0, policy_version 321393 (0.0010) [2023-12-27 07:52:09,397][20347] Updated weights for policy 0, policy_version 321403 (0.0008) [2023-12-27 07:52:09,782][20381] Updated weights for policy 1, policy_version 321182 (0.0007) [2023-12-27 07:52:09,835][20381] Updated weights for policy 1, policy_version 321192 (0.0007) [2023-12-27 07:52:09,891][20381] Updated weights for policy 1, policy_version 321202 (0.0008) [2023-12-27 07:52:10,028][20347] Updated weights for policy 0, policy_version 321413 (0.0009) [2023-12-27 07:52:10,080][20347] Updated weights for policy 0, policy_version 321423 (0.0010) [2023-12-27 07:52:10,130][20347] Updated weights for policy 0, policy_version 321433 (0.0009) [2023-12-27 07:52:10,568][20381] Updated weights for policy 1, policy_version 321212 (0.0008) [2023-12-27 07:52:10,623][20381] Updated weights for policy 1, policy_version 321222 (0.0007) [2023-12-27 07:52:10,671][20381] Updated weights for policy 1, policy_version 321232 (0.0007) [2023-12-27 07:52:10,848][20347] Updated weights for policy 0, policy_version 321443 (0.0008) [2023-12-27 07:52:10,909][20347] Updated weights for policy 0, policy_version 321453 (0.0010) [2023-12-27 07:52:10,970][20347] Updated weights for policy 0, policy_version 321463 (0.0010) [2023-12-27 07:52:11,356][20381] Updated weights for policy 1, policy_version 321242 (0.0007) [2023-12-27 07:52:11,423][20381] Updated weights for policy 1, policy_version 321252 (0.0007) [2023-12-27 07:52:11,473][20381] Updated weights for policy 1, policy_version 321262 (0.0005) [2023-12-27 07:52:11,535][20381] Updated weights for policy 1, policy_version 321272 (0.0007) [2023-12-27 07:52:11,688][20347] Updated weights for policy 0, policy_version 321473 (0.0010) [2023-12-27 07:52:11,756][20347] Updated weights for policy 0, policy_version 321483 (0.0011) [2023-12-27 07:52:11,816][20347] Updated weights for policy 0, policy_version 321493 (0.0011) [2023-12-27 07:52:11,866][20347] Updated weights for policy 0, policy_version 321503 (0.0010) [2023-12-27 07:52:12,216][20381] Updated weights for policy 1, policy_version 321282 (0.0007) [2023-12-27 07:52:12,265][20381] Updated weights for policy 1, policy_version 321292 (0.0007) [2023-12-27 07:52:12,317][20381] Updated weights for policy 1, policy_version 321302 (0.0007) [2023-12-27 07:52:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21162.7, 300 sec: 20799.4). Total num frames: 164593664. Throughput: 0: 10508.4, 1: 10606.6. Samples: 164581732. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:52:12,363][19170] Avg episode reward: [(0, '673.500'), (1, '894.000')] [2023-12-27 07:52:12,573][20347] Updated weights for policy 0, policy_version 321513 (0.0007) [2023-12-27 07:52:12,625][20347] Updated weights for policy 0, policy_version 321523 (0.0005) [2023-12-27 07:52:12,632][20291] KL-divergence is very high: 144.4024 [2023-12-27 07:52:12,646][20291] KL-divergence is very high: 157.5021 [2023-12-27 07:52:12,693][20347] Updated weights for policy 0, policy_version 321533 (0.0008) [2023-12-27 07:52:12,998][20381] Updated weights for policy 1, policy_version 321312 (0.0007) [2023-12-27 07:52:13,050][20381] Updated weights for policy 1, policy_version 321322 (0.0007) [2023-12-27 07:52:13,097][20381] Updated weights for policy 1, policy_version 321332 (0.0007) [2023-12-27 07:52:13,372][20347] Updated weights for policy 0, policy_version 321543 (0.0009) [2023-12-27 07:52:13,425][20347] Updated weights for policy 0, policy_version 321553 (0.0010) [2023-12-27 07:52:13,479][20347] Updated weights for policy 0, policy_version 321563 (0.0010) [2023-12-27 07:52:13,760][20381] Updated weights for policy 1, policy_version 321342 (0.0006) [2023-12-27 07:52:13,818][20381] Updated weights for policy 1, policy_version 321352 (0.0005) [2023-12-27 07:52:13,877][20381] Updated weights for policy 1, policy_version 321362 (0.0007) [2023-12-27 07:52:14,170][20347] Updated weights for policy 0, policy_version 321573 (0.0009) [2023-12-27 07:52:14,232][20347] Updated weights for policy 0, policy_version 321583 (0.0009) [2023-12-27 07:52:14,286][20347] Updated weights for policy 0, policy_version 321593 (0.0008) [2023-12-27 07:52:14,449][20381] Updated weights for policy 1, policy_version 321372 (0.0007) [2023-12-27 07:52:14,504][20381] Updated weights for policy 1, policy_version 321382 (0.0007) [2023-12-27 07:52:14,567][20381] Updated weights for policy 1, policy_version 321392 (0.0005) [2023-12-27 07:52:14,922][20347] Updated weights for policy 0, policy_version 321603 (0.0009) [2023-12-27 07:52:14,988][20347] Updated weights for policy 0, policy_version 321613 (0.0008) [2023-12-27 07:52:15,047][20347] Updated weights for policy 0, policy_version 321623 (0.0009) [2023-12-27 07:52:15,116][20381] Updated weights for policy 1, policy_version 321402 (0.0006) [2023-12-27 07:52:15,173][20381] Updated weights for policy 1, policy_version 321412 (0.0007) [2023-12-27 07:52:15,228][20381] Updated weights for policy 1, policy_version 321422 (0.0008) [2023-12-27 07:52:15,284][20381] Updated weights for policy 1, policy_version 321432 (0.0008) [2023-12-27 07:52:15,743][20347] Updated weights for policy 0, policy_version 321633 (0.0008) [2023-12-27 07:52:15,797][20347] Updated weights for policy 0, policy_version 321643 (0.0005) [2023-12-27 07:52:15,852][20347] Updated weights for policy 0, policy_version 321653 (0.0007) [2023-12-27 07:52:15,920][20347] Updated weights for policy 0, policy_version 321663 (0.0008) [2023-12-27 07:52:15,992][20381] Updated weights for policy 1, policy_version 321442 (0.0005) [2023-12-27 07:52:16,052][20381] Updated weights for policy 1, policy_version 321452 (0.0007) [2023-12-27 07:52:16,102][20381] Updated weights for policy 1, policy_version 321462 (0.0007) [2023-12-27 07:52:16,547][20347] Updated weights for policy 0, policy_version 321673 (0.0006) [2023-12-27 07:52:16,610][20347] Updated weights for policy 0, policy_version 321683 (0.0006) [2023-12-27 07:52:16,670][20347] Updated weights for policy 0, policy_version 321693 (0.0006) [2023-12-27 07:52:16,761][20381] Updated weights for policy 1, policy_version 321472 (0.0010) [2023-12-27 07:52:16,813][20381] Updated weights for policy 1, policy_version 321482 (0.0011) [2023-12-27 07:52:16,871][20381] Updated weights for policy 1, policy_version 321492 (0.0008) [2023-12-27 07:52:17,330][20347] Updated weights for policy 0, policy_version 321703 (0.0008) [2023-12-27 07:52:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 164691968. Throughput: 0: 10398.0, 1: 10556.6. Samples: 164706952. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:52:17,363][19170] Avg episode reward: [(0, '788.280'), (1, '872.590')] [2023-12-27 07:52:17,381][20347] Updated weights for policy 0, policy_version 321713 (0.0007) [2023-12-27 07:52:17,437][20347] Updated weights for policy 0, policy_version 321723 (0.0008) [2023-12-27 07:52:17,575][20381] Updated weights for policy 1, policy_version 321502 (0.0008) [2023-12-27 07:52:17,621][20381] Updated weights for policy 1, policy_version 321512 (0.0008) [2023-12-27 07:52:17,673][20381] Updated weights for policy 1, policy_version 321522 (0.0008) [2023-12-27 07:52:18,100][20347] Updated weights for policy 0, policy_version 321733 (0.0007) [2023-12-27 07:52:18,157][20347] Updated weights for policy 0, policy_version 321743 (0.0005) [2023-12-27 07:52:18,210][20347] Updated weights for policy 0, policy_version 321753 (0.0006) [2023-12-27 07:52:18,279][20381] Updated weights for policy 1, policy_version 321532 (0.0007) [2023-12-27 07:52:18,332][20381] Updated weights for policy 1, policy_version 321542 (0.0007) [2023-12-27 07:52:18,391][20381] Updated weights for policy 1, policy_version 321552 (0.0008) [2023-12-27 07:52:18,856][20347] Updated weights for policy 0, policy_version 321763 (0.0007) [2023-12-27 07:52:18,906][20347] Updated weights for policy 0, policy_version 321773 (0.0007) [2023-12-27 07:52:18,967][20347] Updated weights for policy 0, policy_version 321783 (0.0007) [2023-12-27 07:52:19,047][20381] Updated weights for policy 1, policy_version 321562 (0.0007) [2023-12-27 07:52:19,100][20381] Updated weights for policy 1, policy_version 321572 (0.0010) [2023-12-27 07:52:19,159][20381] Updated weights for policy 1, policy_version 321582 (0.0011) [2023-12-27 07:52:19,215][20381] Updated weights for policy 1, policy_version 321592 (0.0010) [2023-12-27 07:52:19,708][20347] Updated weights for policy 0, policy_version 321793 (0.0008) [2023-12-27 07:52:19,773][20347] Updated weights for policy 0, policy_version 321803 (0.0007) [2023-12-27 07:52:19,829][20381] Updated weights for policy 1, policy_version 321602 (0.0009) [2023-12-27 07:52:19,835][20347] Updated weights for policy 0, policy_version 321813 (0.0007) [2023-12-27 07:52:19,859][20291] KL-divergence is very high: 103.6182 [2023-12-27 07:52:19,886][20381] Updated weights for policy 1, policy_version 321612 (0.0007) [2023-12-27 07:52:19,899][20347] Updated weights for policy 0, policy_version 321823 (0.0007) [2023-12-27 07:52:19,948][20381] Updated weights for policy 1, policy_version 321622 (0.0008) [2023-12-27 07:52:20,500][20347] Updated weights for policy 0, policy_version 321833 (0.0006) [2023-12-27 07:52:20,560][20347] Updated weights for policy 0, policy_version 321843 (0.0006) [2023-12-27 07:52:20,612][20347] Updated weights for policy 0, policy_version 321853 (0.0006) [2023-12-27 07:52:20,614][20381] Updated weights for policy 1, policy_version 321632 (0.0007) [2023-12-27 07:52:20,670][20381] Updated weights for policy 1, policy_version 321642 (0.0008) [2023-12-27 07:52:20,730][20381] Updated weights for policy 1, policy_version 321652 (0.0009) [2023-12-27 07:52:21,273][20347] Updated weights for policy 0, policy_version 321863 (0.0007) [2023-12-27 07:52:21,328][20347] Updated weights for policy 0, policy_version 321873 (0.0006) [2023-12-27 07:52:21,394][20347] Updated weights for policy 0, policy_version 321883 (0.0007) [2023-12-27 07:52:21,394][20381] Updated weights for policy 1, policy_version 321662 (0.0007) [2023-12-27 07:52:21,448][20381] Updated weights for policy 1, policy_version 321672 (0.0007) [2023-12-27 07:52:21,511][20381] Updated weights for policy 1, policy_version 321682 (0.0005) [2023-12-27 07:52:22,088][20347] Updated weights for policy 0, policy_version 321893 (0.0008) [2023-12-27 07:52:22,140][20347] Updated weights for policy 0, policy_version 321903 (0.0007) [2023-12-27 07:52:22,194][20381] Updated weights for policy 1, policy_version 321692 (0.0006) [2023-12-27 07:52:22,195][20347] Updated weights for policy 0, policy_version 321913 (0.0008) [2023-12-27 07:52:22,241][20381] Updated weights for policy 1, policy_version 321702 (0.0005) [2023-12-27 07:52:22,305][20381] Updated weights for policy 1, policy_version 321712 (0.0006) [2023-12-27 07:52:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.7, 300 sec: 20799.3). Total num frames: 164806656. Throughput: 0: 10435.4, 1: 10516.5. Samples: 164769520. Policy #0 lag: (min: 14.0, avg: 21.0, max: 46.0) [2023-12-27 07:52:22,363][19170] Avg episode reward: [(0, '877.350'), (1, '904.200')] [2023-12-27 07:52:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000321920_82427904.pth... [2023-12-27 07:52:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000321720_82378752.pth... [2023-12-27 07:52:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000320472_82059264.pth [2023-12-27 07:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000320704_82116608.pth [2023-12-27 07:52:22,950][20347] Updated weights for policy 0, policy_version 321923 (0.0007) [2023-12-27 07:52:22,954][20381] Updated weights for policy 1, policy_version 321722 (0.0007) [2023-12-27 07:52:23,004][20381] Updated weights for policy 1, policy_version 321732 (0.0006) [2023-12-27 07:52:23,008][20347] Updated weights for policy 0, policy_version 321933 (0.0008) [2023-12-27 07:52:23,064][20381] Updated weights for policy 1, policy_version 321742 (0.0008) [2023-12-27 07:52:23,066][20347] Updated weights for policy 0, policy_version 321943 (0.0006) [2023-12-27 07:52:23,114][20381] Updated weights for policy 1, policy_version 321752 (0.0006) [2023-12-27 07:52:23,710][20347] Updated weights for policy 0, policy_version 321953 (0.0007) [2023-12-27 07:52:23,758][20347] Updated weights for policy 0, policy_version 321963 (0.0010) [2023-12-27 07:52:23,805][20347] Updated weights for policy 0, policy_version 321973 (0.0010) [2023-12-27 07:52:23,840][20381] Updated weights for policy 1, policy_version 321762 (0.0007) [2023-12-27 07:52:23,860][20347] Updated weights for policy 0, policy_version 321983 (0.0009) [2023-12-27 07:52:23,890][20381] Updated weights for policy 1, policy_version 321772 (0.0008) [2023-12-27 07:52:23,951][20381] Updated weights for policy 1, policy_version 321782 (0.0009) [2023-12-27 07:52:24,497][20347] Updated weights for policy 0, policy_version 321993 (0.0008) [2023-12-27 07:52:24,552][20347] Updated weights for policy 0, policy_version 322003 (0.0007) [2023-12-27 07:52:24,597][20347] Updated weights for policy 0, policy_version 322013 (0.0006) [2023-12-27 07:52:24,641][20381] Updated weights for policy 1, policy_version 321792 (0.0008) [2023-12-27 07:52:24,693][20381] Updated weights for policy 1, policy_version 321802 (0.0009) [2023-12-27 07:52:24,738][20381] Updated weights for policy 1, policy_version 321812 (0.0005) [2023-12-27 07:52:25,158][20347] Updated weights for policy 0, policy_version 322023 (0.0007) [2023-12-27 07:52:25,220][20347] Updated weights for policy 0, policy_version 322033 (0.0008) [2023-12-27 07:52:25,276][20347] Updated weights for policy 0, policy_version 322043 (0.0008) [2023-12-27 07:52:25,438][20381] Updated weights for policy 1, policy_version 321822 (0.0007) [2023-12-27 07:52:25,490][20381] Updated weights for policy 1, policy_version 321832 (0.0008) [2023-12-27 07:52:25,546][20381] Updated weights for policy 1, policy_version 321842 (0.0007) [2023-12-27 07:52:25,890][20347] Updated weights for policy 0, policy_version 322053 (0.0009) [2023-12-27 07:52:25,934][20347] Updated weights for policy 0, policy_version 322063 (0.0008) [2023-12-27 07:52:25,989][20347] Updated weights for policy 0, policy_version 322073 (0.0008) [2023-12-27 07:52:26,145][20381] Updated weights for policy 1, policy_version 321852 (0.0005) [2023-12-27 07:52:26,198][20381] Updated weights for policy 1, policy_version 321862 (0.0006) [2023-12-27 07:52:26,247][20381] Updated weights for policy 1, policy_version 321872 (0.0005) [2023-12-27 07:52:26,734][20347] Updated weights for policy 0, policy_version 322083 (0.0009) [2023-12-27 07:52:26,785][20347] Updated weights for policy 0, policy_version 322093 (0.0010) [2023-12-27 07:52:26,847][20347] Updated weights for policy 0, policy_version 322103 (0.0010) [2023-12-27 07:52:26,902][20381] Updated weights for policy 1, policy_version 321882 (0.0006) [2023-12-27 07:52:26,954][20381] Updated weights for policy 1, policy_version 321892 (0.0008) [2023-12-27 07:52:27,013][20381] Updated weights for policy 1, policy_version 321902 (0.0007) [2023-12-27 07:52:27,061][20381] Updated weights for policy 1, policy_version 321912 (0.0007) [2023-12-27 07:52:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 164904960. Throughput: 0: 10482.9, 1: 10488.4. Samples: 164895012. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:27,363][19170] Avg episode reward: [(0, '885.640'), (1, '971.260')] [2023-12-27 07:52:27,506][20347] Updated weights for policy 0, policy_version 322113 (0.0010) [2023-12-27 07:52:27,559][20347] Updated weights for policy 0, policy_version 322123 (0.0005) [2023-12-27 07:52:27,606][20347] Updated weights for policy 0, policy_version 322133 (0.0005) [2023-12-27 07:52:27,665][20347] Updated weights for policy 0, policy_version 322143 (0.0009) [2023-12-27 07:52:27,731][20381] Updated weights for policy 1, policy_version 321922 (0.0007) [2023-12-27 07:52:27,774][20381] Updated weights for policy 1, policy_version 321932 (0.0005) [2023-12-27 07:52:27,822][20381] Updated weights for policy 1, policy_version 321942 (0.0006) [2023-12-27 07:52:28,213][20347] Updated weights for policy 0, policy_version 322153 (0.0010) [2023-12-27 07:52:28,264][20347] Updated weights for policy 0, policy_version 322163 (0.0010) [2023-12-27 07:52:28,313][20347] Updated weights for policy 0, policy_version 322173 (0.0010) [2023-12-27 07:52:28,521][20381] Updated weights for policy 1, policy_version 321952 (0.0007) [2023-12-27 07:52:28,581][20381] Updated weights for policy 1, policy_version 321962 (0.0005) [2023-12-27 07:52:28,645][20381] Updated weights for policy 1, policy_version 321972 (0.0005) [2023-12-27 07:52:28,970][20347] Updated weights for policy 0, policy_version 322183 (0.0007) [2023-12-27 07:52:29,033][20347] Updated weights for policy 0, policy_version 322193 (0.0008) [2023-12-27 07:52:29,087][20347] Updated weights for policy 0, policy_version 322203 (0.0010) [2023-12-27 07:52:29,311][20381] Updated weights for policy 1, policy_version 321982 (0.0006) [2023-12-27 07:52:29,375][20381] Updated weights for policy 1, policy_version 321992 (0.0007) [2023-12-27 07:52:29,425][20381] Updated weights for policy 1, policy_version 322002 (0.0007) [2023-12-27 07:52:29,767][20347] Updated weights for policy 0, policy_version 322213 (0.0009) [2023-12-27 07:52:29,826][20347] Updated weights for policy 0, policy_version 322223 (0.0010) [2023-12-27 07:52:29,887][20347] Updated weights for policy 0, policy_version 322233 (0.0009) [2023-12-27 07:52:30,061][20381] Updated weights for policy 1, policy_version 322012 (0.0005) [2023-12-27 07:52:30,119][20381] Updated weights for policy 1, policy_version 322022 (0.0006) [2023-12-27 07:52:30,180][20381] Updated weights for policy 1, policy_version 322032 (0.0007) [2023-12-27 07:52:30,514][20347] Updated weights for policy 0, policy_version 322243 (0.0010) [2023-12-27 07:52:30,575][20347] Updated weights for policy 0, policy_version 322253 (0.0007) [2023-12-27 07:52:30,620][20291] KL-divergence is very high: 349.3557 [2023-12-27 07:52:30,633][20347] Updated weights for policy 0, policy_version 322263 (0.0006) [2023-12-27 07:52:30,667][20291] KL-divergence is very high: 364.8730 [2023-12-27 07:52:30,859][20381] Updated weights for policy 1, policy_version 322042 (0.0008) [2023-12-27 07:52:30,905][20381] Updated weights for policy 1, policy_version 322052 (0.0008) [2023-12-27 07:52:30,958][20381] Updated weights for policy 1, policy_version 322062 (0.0008) [2023-12-27 07:52:31,004][20381] Updated weights for policy 1, policy_version 322072 (0.0007) [2023-12-27 07:52:31,258][20347] Updated weights for policy 0, policy_version 322273 (0.0006) [2023-12-27 07:52:31,320][20347] Updated weights for policy 0, policy_version 322283 (0.0008) [2023-12-27 07:52:31,385][20347] Updated weights for policy 0, policy_version 322293 (0.0008) [2023-12-27 07:52:31,444][20347] Updated weights for policy 0, policy_version 322303 (0.0008) [2023-12-27 07:52:31,735][20381] Updated weights for policy 1, policy_version 322082 (0.0007) [2023-12-27 07:52:31,797][20381] Updated weights for policy 1, policy_version 322092 (0.0008) [2023-12-27 07:52:31,854][20381] Updated weights for policy 1, policy_version 322102 (0.0008) [2023-12-27 07:52:32,134][20347] Updated weights for policy 0, policy_version 322313 (0.0008) [2023-12-27 07:52:32,191][20347] Updated weights for policy 0, policy_version 322323 (0.0006) [2023-12-27 07:52:32,239][20347] Updated weights for policy 0, policy_version 322333 (0.0005) [2023-12-27 07:52:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 165011456. Throughput: 0: 10438.5, 1: 10515.8. Samples: 165020064. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:32,363][19170] Avg episode reward: [(0, '913.130'), (1, '990.330')] [2023-12-27 07:52:32,536][20381] Updated weights for policy 1, policy_version 322112 (0.0010) [2023-12-27 07:52:32,591][20381] Updated weights for policy 1, policy_version 322122 (0.0010) [2023-12-27 07:52:32,640][20381] Updated weights for policy 1, policy_version 322132 (0.0010) [2023-12-27 07:52:32,879][20347] Updated weights for policy 0, policy_version 322343 (0.0006) [2023-12-27 07:52:32,941][20347] Updated weights for policy 0, policy_version 322353 (0.0005) [2023-12-27 07:52:32,990][20347] Updated weights for policy 0, policy_version 322363 (0.0005) [2023-12-27 07:52:33,288][20381] Updated weights for policy 1, policy_version 322142 (0.0007) [2023-12-27 07:52:33,332][20381] Updated weights for policy 1, policy_version 322152 (0.0005) [2023-12-27 07:52:33,389][20381] Updated weights for policy 1, policy_version 322162 (0.0005) [2023-12-27 07:52:33,514][20347] Updated weights for policy 0, policy_version 322373 (0.0005) [2023-12-27 07:52:33,570][20347] Updated weights for policy 0, policy_version 322383 (0.0005) [2023-12-27 07:52:33,623][20347] Updated weights for policy 0, policy_version 322393 (0.0005) [2023-12-27 07:52:33,998][20381] Updated weights for policy 1, policy_version 322172 (0.0006) [2023-12-27 07:52:34,050][20381] Updated weights for policy 1, policy_version 322182 (0.0007) [2023-12-27 07:52:34,091][20381] Updated weights for policy 1, policy_version 322192 (0.0007) [2023-12-27 07:52:34,189][20347] Updated weights for policy 0, policy_version 322403 (0.0006) [2023-12-27 07:52:34,245][20347] Updated weights for policy 0, policy_version 322413 (0.0008) [2023-12-27 07:52:34,302][20347] Updated weights for policy 0, policy_version 322423 (0.0007) [2023-12-27 07:52:34,844][20381] Updated weights for policy 1, policy_version 322202 (0.0008) [2023-12-27 07:52:34,896][20381] Updated weights for policy 1, policy_version 322212 (0.0007) [2023-12-27 07:52:34,915][20347] Updated weights for policy 0, policy_version 322433 (0.0007) [2023-12-27 07:52:34,956][20381] Updated weights for policy 1, policy_version 322222 (0.0007) [2023-12-27 07:52:34,974][20347] Updated weights for policy 0, policy_version 322443 (0.0006) [2023-12-27 07:52:35,015][20381] Updated weights for policy 1, policy_version 322232 (0.0007) [2023-12-27 07:52:35,032][20347] Updated weights for policy 0, policy_version 322453 (0.0005) [2023-12-27 07:52:35,088][20347] Updated weights for policy 0, policy_version 322463 (0.0008) [2023-12-27 07:52:35,704][20347] Updated weights for policy 0, policy_version 322473 (0.0006) [2023-12-27 07:52:35,751][20381] Updated weights for policy 1, policy_version 322242 (0.0005) [2023-12-27 07:52:35,753][20347] Updated weights for policy 0, policy_version 322483 (0.0006) [2023-12-27 07:52:35,800][20381] Updated weights for policy 1, policy_version 322252 (0.0006) [2023-12-27 07:52:35,802][20347] Updated weights for policy 0, policy_version 322493 (0.0005) [2023-12-27 07:52:35,855][20381] Updated weights for policy 1, policy_version 322262 (0.0007) [2023-12-27 07:52:36,390][20347] Updated weights for policy 0, policy_version 322503 (0.0005) [2023-12-27 07:52:36,455][20347] Updated weights for policy 0, policy_version 322513 (0.0005) [2023-12-27 07:52:36,516][20347] Updated weights for policy 0, policy_version 322523 (0.0005) [2023-12-27 07:52:36,637][20381] Updated weights for policy 1, policy_version 322272 (0.0007) [2023-12-27 07:52:36,694][20381] Updated weights for policy 1, policy_version 322282 (0.0007) [2023-12-27 07:52:36,754][20381] Updated weights for policy 1, policy_version 322292 (0.0007) [2023-12-27 07:52:37,145][20347] Updated weights for policy 0, policy_version 322533 (0.0005) [2023-12-27 07:52:37,203][20347] Updated weights for policy 0, policy_version 322543 (0.0006) [2023-12-27 07:52:37,263][20347] Updated weights for policy 0, policy_version 322553 (0.0008) [2023-12-27 07:52:37,313][20381] Updated weights for policy 1, policy_version 322302 (0.0006) [2023-12-27 07:52:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 165117952. Throughput: 0: 10512.2, 1: 10517.5. Samples: 165085488. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:37,363][19170] Avg episode reward: [(0, '912.660'), (1, '932.290')] [2023-12-27 07:52:37,365][20381] Updated weights for policy 1, policy_version 322312 (0.0007) [2023-12-27 07:52:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000322560_82591744.pth... [2023-12-27 07:52:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000321312_82272256.pth [2023-12-27 07:52:37,424][20381] Updated weights for policy 1, policy_version 322322 (0.0007) [2023-12-27 07:52:37,452][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000322328_82534400.pth... [2023-12-27 07:52:37,455][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000321112_82223104.pth [2023-12-27 07:52:37,916][20347] Updated weights for policy 0, policy_version 322563 (0.0008) [2023-12-27 07:52:37,968][20347] Updated weights for policy 0, policy_version 322573 (0.0007) [2023-12-27 07:52:38,032][20347] Updated weights for policy 0, policy_version 322583 (0.0007) [2023-12-27 07:52:38,064][20381] Updated weights for policy 1, policy_version 322332 (0.0007) [2023-12-27 07:52:38,129][20381] Updated weights for policy 1, policy_version 322342 (0.0008) [2023-12-27 07:52:38,183][20381] Updated weights for policy 1, policy_version 322352 (0.0008) [2023-12-27 07:52:38,672][20347] Updated weights for policy 0, policy_version 322593 (0.0006) [2023-12-27 07:52:38,736][20347] Updated weights for policy 0, policy_version 322603 (0.0007) [2023-12-27 07:52:38,790][20347] Updated weights for policy 0, policy_version 322613 (0.0008) [2023-12-27 07:52:38,847][20347] Updated weights for policy 0, policy_version 322623 (0.0008) [2023-12-27 07:52:38,891][20381] Updated weights for policy 1, policy_version 322362 (0.0008) [2023-12-27 07:52:38,952][20381] Updated weights for policy 1, policy_version 322372 (0.0008) [2023-12-27 07:52:39,009][20381] Updated weights for policy 1, policy_version 322382 (0.0008) [2023-12-27 07:52:39,068][20381] Updated weights for policy 1, policy_version 322392 (0.0008) [2023-12-27 07:52:39,544][20347] Updated weights for policy 0, policy_version 322633 (0.0007) [2023-12-27 07:52:39,601][20347] Updated weights for policy 0, policy_version 322643 (0.0008) [2023-12-27 07:52:39,653][20347] Updated weights for policy 0, policy_version 322653 (0.0007) [2023-12-27 07:52:39,789][20381] Updated weights for policy 1, policy_version 322402 (0.0007) [2023-12-27 07:52:39,846][20381] Updated weights for policy 1, policy_version 322412 (0.0008) [2023-12-27 07:52:39,900][20381] Updated weights for policy 1, policy_version 322422 (0.0008) [2023-12-27 07:52:40,413][20347] Updated weights for policy 0, policy_version 322663 (0.0007) [2023-12-27 07:52:40,460][20347] Updated weights for policy 0, policy_version 322673 (0.0007) [2023-12-27 07:52:40,526][20347] Updated weights for policy 0, policy_version 322683 (0.0005) [2023-12-27 07:52:40,557][20381] Updated weights for policy 1, policy_version 322432 (0.0006) [2023-12-27 07:52:40,612][20381] Updated weights for policy 1, policy_version 322442 (0.0005) [2023-12-27 07:52:40,661][20381] Updated weights for policy 1, policy_version 322452 (0.0005) [2023-12-27 07:52:41,254][20347] Updated weights for policy 0, policy_version 322693 (0.0005) [2023-12-27 07:52:41,292][20381] Updated weights for policy 1, policy_version 322462 (0.0008) [2023-12-27 07:52:41,309][20347] Updated weights for policy 0, policy_version 322703 (0.0006) [2023-12-27 07:52:41,361][20381] Updated weights for policy 1, policy_version 322472 (0.0007) [2023-12-27 07:52:41,374][20347] Updated weights for policy 0, policy_version 322713 (0.0007) [2023-12-27 07:52:41,419][20381] Updated weights for policy 1, policy_version 322482 (0.0008) [2023-12-27 07:52:42,021][20347] Updated weights for policy 0, policy_version 322723 (0.0007) [2023-12-27 07:52:42,081][20347] Updated weights for policy 0, policy_version 322733 (0.0008) [2023-12-27 07:52:42,133][20347] Updated weights for policy 0, policy_version 322743 (0.0007) [2023-12-27 07:52:42,138][20381] Updated weights for policy 1, policy_version 322492 (0.0006) [2023-12-27 07:52:42,192][20381] Updated weights for policy 1, policy_version 322502 (0.0006) [2023-12-27 07:52:42,239][20381] Updated weights for policy 1, policy_version 322512 (0.0008) [2023-12-27 07:52:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21162.7, 300 sec: 20799.4). Total num frames: 165224448. Throughput: 0: 10521.3, 1: 10497.8. Samples: 165210332. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:42,363][19170] Avg episode reward: [(0, '824.760'), (1, '876.610')] [2023-12-27 07:52:42,851][20347] Updated weights for policy 0, policy_version 322753 (0.0005) [2023-12-27 07:52:42,900][20347] Updated weights for policy 0, policy_version 322763 (0.0008) [2023-12-27 07:52:42,956][20347] Updated weights for policy 0, policy_version 322773 (0.0007) [2023-12-27 07:52:42,975][20381] Updated weights for policy 1, policy_version 322522 (0.0006) [2023-12-27 07:52:43,016][20347] Updated weights for policy 0, policy_version 322783 (0.0008) [2023-12-27 07:52:43,030][20381] Updated weights for policy 1, policy_version 322532 (0.0007) [2023-12-27 07:52:43,080][20381] Updated weights for policy 1, policy_version 322542 (0.0008) [2023-12-27 07:52:43,126][20381] Updated weights for policy 1, policy_version 322552 (0.0008) [2023-12-27 07:52:43,715][20347] Updated weights for policy 0, policy_version 322793 (0.0007) [2023-12-27 07:52:43,715][20291] KL-divergence is very high: 150.6423 [2023-12-27 07:52:43,771][20347] Updated weights for policy 0, policy_version 322803 (0.0007) [2023-12-27 07:52:43,821][20347] Updated weights for policy 0, policy_version 322813 (0.0006) [2023-12-27 07:52:43,838][20381] Updated weights for policy 1, policy_version 322562 (0.0006) [2023-12-27 07:52:43,884][20381] Updated weights for policy 1, policy_version 322572 (0.0007) [2023-12-27 07:52:43,933][20381] Updated weights for policy 1, policy_version 322582 (0.0008) [2023-12-27 07:52:44,445][20347] Updated weights for policy 0, policy_version 322823 (0.0007) [2023-12-27 07:52:44,504][20347] Updated weights for policy 0, policy_version 322833 (0.0007) [2023-12-27 07:52:44,561][20347] Updated weights for policy 0, policy_version 322843 (0.0008) [2023-12-27 07:52:44,674][20381] Updated weights for policy 1, policy_version 322592 (0.0007) [2023-12-27 07:52:44,723][20381] Updated weights for policy 1, policy_version 322602 (0.0008) [2023-12-27 07:52:44,779][20381] Updated weights for policy 1, policy_version 322612 (0.0007) [2023-12-27 07:52:45,244][20347] Updated weights for policy 0, policy_version 322853 (0.0007) [2023-12-27 07:52:45,308][20347] Updated weights for policy 0, policy_version 322863 (0.0008) [2023-12-27 07:52:45,368][20347] Updated weights for policy 0, policy_version 322873 (0.0008) [2023-12-27 07:52:45,435][20381] Updated weights for policy 1, policy_version 322622 (0.0007) [2023-12-27 07:52:45,482][20381] Updated weights for policy 1, policy_version 322632 (0.0008) [2023-12-27 07:52:45,535][20381] Updated weights for policy 1, policy_version 322642 (0.0007) [2023-12-27 07:52:46,019][20347] Updated weights for policy 0, policy_version 322883 (0.0008) [2023-12-27 07:52:46,083][20347] Updated weights for policy 0, policy_version 322893 (0.0005) [2023-12-27 07:52:46,147][20347] Updated weights for policy 0, policy_version 322903 (0.0005) [2023-12-27 07:52:46,240][20381] Updated weights for policy 1, policy_version 322652 (0.0008) [2023-12-27 07:52:46,294][20381] Updated weights for policy 1, policy_version 322662 (0.0008) [2023-12-27 07:52:46,356][20381] Updated weights for policy 1, policy_version 322672 (0.0011) [2023-12-27 07:52:46,772][20347] Updated weights for policy 0, policy_version 322913 (0.0007) [2023-12-27 07:52:46,821][20347] Updated weights for policy 0, policy_version 322923 (0.0005) [2023-12-27 07:52:46,875][20347] Updated weights for policy 0, policy_version 322933 (0.0005) [2023-12-27 07:52:46,926][20347] Updated weights for policy 0, policy_version 322943 (0.0008) [2023-12-27 07:52:46,955][20381] Updated weights for policy 1, policy_version 322682 (0.0009) [2023-12-27 07:52:46,999][20381] Updated weights for policy 1, policy_version 322692 (0.0005) [2023-12-27 07:52:47,048][20381] Updated weights for policy 1, policy_version 322702 (0.0010) [2023-12-27 07:52:47,099][20381] Updated weights for policy 1, policy_version 322712 (0.0009) [2023-12-27 07:52:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 21026.2, 300 sec: 20799.4). Total num frames: 165322752. Throughput: 0: 10486.7, 1: 10395.4. Samples: 165332800. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:47,363][19170] Avg episode reward: [(0, '785.860'), (1, '818.090')] [2023-12-27 07:52:47,596][20347] Updated weights for policy 0, policy_version 322953 (0.0007) [2023-12-27 07:52:47,661][20347] Updated weights for policy 0, policy_version 322963 (0.0008) [2023-12-27 07:52:47,720][20347] Updated weights for policy 0, policy_version 322973 (0.0010) [2023-12-27 07:52:47,725][20381] Updated weights for policy 1, policy_version 322722 (0.0011) [2023-12-27 07:52:47,780][20381] Updated weights for policy 1, policy_version 322732 (0.0010) [2023-12-27 07:52:47,832][20381] Updated weights for policy 1, policy_version 322742 (0.0010) [2023-12-27 07:52:48,325][20347] Updated weights for policy 0, policy_version 322983 (0.0007) [2023-12-27 07:52:48,345][20291] KL-divergence is very high: 111.3691 [2023-12-27 07:52:48,351][20291] KL-divergence is very high: 111.6970 [2023-12-27 07:52:48,383][20347] Updated weights for policy 0, policy_version 322993 (0.0007) [2023-12-27 07:52:48,437][20347] Updated weights for policy 0, policy_version 323003 (0.0007) [2023-12-27 07:52:48,534][20381] Updated weights for policy 1, policy_version 322752 (0.0008) [2023-12-27 07:52:48,594][20381] Updated weights for policy 1, policy_version 322762 (0.0006) [2023-12-27 07:52:48,656][20381] Updated weights for policy 1, policy_version 322772 (0.0005) [2023-12-27 07:52:49,054][20291] KL-divergence is very high: 123.2385 [2023-12-27 07:52:49,077][20291] KL-divergence is very high: 170.5009 [2023-12-27 07:52:49,083][20347] Updated weights for policy 0, policy_version 323013 (0.0008) [2023-12-27 07:52:49,122][20291] KL-divergence is very high: 142.9985 [2023-12-27 07:52:49,140][20347] Updated weights for policy 0, policy_version 323023 (0.0006) [2023-12-27 07:52:49,172][20291] KL-divergence is very high: 166.1539 [2023-12-27 07:52:49,200][20347] Updated weights for policy 0, policy_version 323033 (0.0006) [2023-12-27 07:52:49,219][20291] KL-divergence is very high: 146.0411 [2023-12-27 07:52:49,305][20381] Updated weights for policy 1, policy_version 322782 (0.0007) [2023-12-27 07:52:49,367][20381] Updated weights for policy 1, policy_version 322792 (0.0007) [2023-12-27 07:52:49,433][20381] Updated weights for policy 1, policy_version 322802 (0.0007) [2023-12-27 07:52:49,818][20347] Updated weights for policy 0, policy_version 323043 (0.0007) [2023-12-27 07:52:49,882][20347] Updated weights for policy 0, policy_version 323053 (0.0007) [2023-12-27 07:52:49,947][20347] Updated weights for policy 0, policy_version 323063 (0.0009) [2023-12-27 07:52:50,167][20381] Updated weights for policy 1, policy_version 322812 (0.0007) [2023-12-27 07:52:50,229][20381] Updated weights for policy 1, policy_version 322822 (0.0009) [2023-12-27 07:52:50,287][20381] Updated weights for policy 1, policy_version 322832 (0.0009) [2023-12-27 07:52:50,552][20347] Updated weights for policy 0, policy_version 323073 (0.0008) [2023-12-27 07:52:50,612][20347] Updated weights for policy 0, policy_version 323083 (0.0007) [2023-12-27 07:52:50,671][20347] Updated weights for policy 0, policy_version 323093 (0.0007) [2023-12-27 07:52:50,731][20347] Updated weights for policy 0, policy_version 323103 (0.0007) [2023-12-27 07:52:50,960][20381] Updated weights for policy 1, policy_version 322842 (0.0007) [2023-12-27 07:52:51,016][20381] Updated weights for policy 1, policy_version 322852 (0.0008) [2023-12-27 07:52:51,080][20381] Updated weights for policy 1, policy_version 322862 (0.0008) [2023-12-27 07:52:51,137][20381] Updated weights for policy 1, policy_version 322872 (0.0008) [2023-12-27 07:52:51,420][20347] Updated weights for policy 0, policy_version 323113 (0.0008) [2023-12-27 07:52:51,480][20347] Updated weights for policy 0, policy_version 323123 (0.0008) [2023-12-27 07:52:51,539][20347] Updated weights for policy 0, policy_version 323133 (0.0008) [2023-12-27 07:52:51,845][20381] Updated weights for policy 1, policy_version 322882 (0.0006) [2023-12-27 07:52:51,905][20381] Updated weights for policy 1, policy_version 322892 (0.0007) [2023-12-27 07:52:51,961][20381] Updated weights for policy 1, policy_version 322902 (0.0007) [2023-12-27 07:52:52,248][20347] Updated weights for policy 0, policy_version 323143 (0.0008) [2023-12-27 07:52:52,307][20347] Updated weights for policy 0, policy_version 323153 (0.0008) [2023-12-27 07:52:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 165421056. Throughput: 0: 10500.5, 1: 10337.6. Samples: 165395288. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:52,363][19170] Avg episode reward: [(0, '768.910'), (1, '872.600')] [2023-12-27 07:52:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000322904_82681856.pth... [2023-12-27 07:52:52,371][20347] Updated weights for policy 0, policy_version 323163 (0.0008) [2023-12-27 07:52:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000321720_82378752.pth [2023-12-27 07:52:52,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000323168_82747392.pth... [2023-12-27 07:52:52,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000321920_82427904.pth [2023-12-27 07:52:52,622][20381] Updated weights for policy 1, policy_version 322912 (0.0006) [2023-12-27 07:52:52,682][20381] Updated weights for policy 1, policy_version 322922 (0.0006) [2023-12-27 07:52:52,737][20381] Updated weights for policy 1, policy_version 322932 (0.0006) [2023-12-27 07:52:53,019][20347] Updated weights for policy 0, policy_version 323173 (0.0007) [2023-12-27 07:52:53,089][20347] Updated weights for policy 0, policy_version 323183 (0.0006) [2023-12-27 07:52:53,148][20347] Updated weights for policy 0, policy_version 323193 (0.0009) [2023-12-27 07:52:53,267][20381] Updated weights for policy 1, policy_version 322942 (0.0006) [2023-12-27 07:52:53,315][20381] Updated weights for policy 1, policy_version 322952 (0.0007) [2023-12-27 07:52:53,362][20381] Updated weights for policy 1, policy_version 322962 (0.0008) [2023-12-27 07:52:53,802][20347] Updated weights for policy 0, policy_version 323203 (0.0008) [2023-12-27 07:52:53,856][20347] Updated weights for policy 0, policy_version 323213 (0.0007) [2023-12-27 07:52:53,904][20347] Updated weights for policy 0, policy_version 323223 (0.0009) [2023-12-27 07:52:53,956][20381] Updated weights for policy 1, policy_version 322972 (0.0007) [2023-12-27 07:52:54,018][20381] Updated weights for policy 1, policy_version 322982 (0.0007) [2023-12-27 07:52:54,075][20381] Updated weights for policy 1, policy_version 322992 (0.0007) [2023-12-27 07:52:54,590][20347] Updated weights for policy 0, policy_version 323233 (0.0008) [2023-12-27 07:52:54,649][20347] Updated weights for policy 0, policy_version 323243 (0.0010) [2023-12-27 07:52:54,704][20347] Updated weights for policy 0, policy_version 323253 (0.0010) [2023-12-27 07:52:54,705][20381] Updated weights for policy 1, policy_version 323002 (0.0008) [2023-12-27 07:52:54,753][20347] Updated weights for policy 0, policy_version 323263 (0.0010) [2023-12-27 07:52:54,765][20381] Updated weights for policy 1, policy_version 323012 (0.0008) [2023-12-27 07:52:54,820][20381] Updated weights for policy 1, policy_version 323022 (0.0007) [2023-12-27 07:52:54,872][20381] Updated weights for policy 1, policy_version 323032 (0.0007) [2023-12-27 07:52:55,450][20347] Updated weights for policy 0, policy_version 323273 (0.0009) [2023-12-27 07:52:55,509][20347] Updated weights for policy 0, policy_version 323283 (0.0008) [2023-12-27 07:52:55,567][20347] Updated weights for policy 0, policy_version 323293 (0.0008) [2023-12-27 07:52:55,572][20381] Updated weights for policy 1, policy_version 323042 (0.0005) [2023-12-27 07:52:55,628][20381] Updated weights for policy 1, policy_version 323052 (0.0005) [2023-12-27 07:52:55,680][20381] Updated weights for policy 1, policy_version 323062 (0.0006) [2023-12-27 07:52:56,265][20347] Updated weights for policy 0, policy_version 323303 (0.0009) [2023-12-27 07:52:56,321][20347] Updated weights for policy 0, policy_version 323313 (0.0010) [2023-12-27 07:52:56,341][20381] Updated weights for policy 1, policy_version 323072 (0.0009) [2023-12-27 07:52:56,367][20347] Updated weights for policy 0, policy_version 323323 (0.0010) [2023-12-27 07:52:56,390][20381] Updated weights for policy 1, policy_version 323082 (0.0006) [2023-12-27 07:52:56,438][20381] Updated weights for policy 1, policy_version 323092 (0.0007) [2023-12-27 07:52:57,033][20347] Updated weights for policy 0, policy_version 323333 (0.0010) [2023-12-27 07:52:57,085][20347] Updated weights for policy 0, policy_version 323343 (0.0005) [2023-12-27 07:52:57,132][20381] Updated weights for policy 1, policy_version 323102 (0.0006) [2023-12-27 07:52:57,148][20347] Updated weights for policy 0, policy_version 323353 (0.0009) [2023-12-27 07:52:57,186][20381] Updated weights for policy 1, policy_version 323112 (0.0006) [2023-12-27 07:52:57,233][20381] Updated weights for policy 1, policy_version 323122 (0.0008) [2023-12-27 07:52:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 165535744. Throughput: 0: 10470.8, 1: 10401.5. Samples: 165520988. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:52:57,363][19170] Avg episode reward: [(0, '701.890'), (1, '921.790')] [2023-12-27 07:52:57,778][20347] Updated weights for policy 0, policy_version 323363 (0.0010) [2023-12-27 07:52:57,838][20347] Updated weights for policy 0, policy_version 323373 (0.0011) [2023-12-27 07:52:57,858][20381] Updated weights for policy 1, policy_version 323132 (0.0008) [2023-12-27 07:52:57,887][20347] Updated weights for policy 0, policy_version 323383 (0.0010) [2023-12-27 07:52:57,911][20381] Updated weights for policy 1, policy_version 323142 (0.0005) [2023-12-27 07:52:57,965][20381] Updated weights for policy 1, policy_version 323152 (0.0007) [2023-12-27 07:52:58,602][20347] Updated weights for policy 0, policy_version 323393 (0.0010) [2023-12-27 07:52:58,664][20347] Updated weights for policy 0, policy_version 323403 (0.0010) [2023-12-27 07:52:58,696][20381] Updated weights for policy 1, policy_version 323162 (0.0008) [2023-12-27 07:52:58,727][20347] Updated weights for policy 0, policy_version 323413 (0.0009) [2023-12-27 07:52:58,764][20381] Updated weights for policy 1, policy_version 323172 (0.0008) [2023-12-27 07:52:58,794][20347] Updated weights for policy 0, policy_version 323423 (0.0008) [2023-12-27 07:52:58,830][20381] Updated weights for policy 1, policy_version 323182 (0.0008) [2023-12-27 07:52:58,897][20381] Updated weights for policy 1, policy_version 323192 (0.0008) [2023-12-27 07:52:59,476][20347] Updated weights for policy 0, policy_version 323433 (0.0008) [2023-12-27 07:52:59,531][20347] Updated weights for policy 0, policy_version 323443 (0.0007) [2023-12-27 07:52:59,585][20347] Updated weights for policy 0, policy_version 323453 (0.0007) [2023-12-27 07:52:59,673][20381] Updated weights for policy 1, policy_version 323202 (0.0008) [2023-12-27 07:52:59,726][20381] Updated weights for policy 1, policy_version 323212 (0.0008) [2023-12-27 07:52:59,786][20381] Updated weights for policy 1, policy_version 323222 (0.0007) [2023-12-27 07:53:00,237][20347] Updated weights for policy 0, policy_version 323463 (0.0007) [2023-12-27 07:53:00,292][20347] Updated weights for policy 0, policy_version 323473 (0.0008) [2023-12-27 07:53:00,349][20347] Updated weights for policy 0, policy_version 323483 (0.0007) [2023-12-27 07:53:00,509][20381] Updated weights for policy 1, policy_version 323232 (0.0008) [2023-12-27 07:53:00,558][20381] Updated weights for policy 1, policy_version 323242 (0.0005) [2023-12-27 07:53:00,609][20381] Updated weights for policy 1, policy_version 323252 (0.0005) [2023-12-27 07:53:01,065][20347] Updated weights for policy 0, policy_version 323493 (0.0007) [2023-12-27 07:53:01,123][20347] Updated weights for policy 0, policy_version 323503 (0.0007) [2023-12-27 07:53:01,184][20347] Updated weights for policy 0, policy_version 323513 (0.0007) [2023-12-27 07:53:01,201][20381] Updated weights for policy 1, policy_version 323262 (0.0008) [2023-12-27 07:53:01,253][20381] Updated weights for policy 1, policy_version 323272 (0.0010) [2023-12-27 07:53:01,317][20381] Updated weights for policy 1, policy_version 323282 (0.0005) [2023-12-27 07:53:01,887][20347] Updated weights for policy 0, policy_version 323523 (0.0007) [2023-12-27 07:53:01,940][20347] Updated weights for policy 0, policy_version 323533 (0.0007) [2023-12-27 07:53:01,964][20381] Updated weights for policy 1, policy_version 323292 (0.0007) [2023-12-27 07:53:01,988][20347] Updated weights for policy 0, policy_version 323543 (0.0006) [2023-12-27 07:53:02,020][20381] Updated weights for policy 1, policy_version 323302 (0.0006) [2023-12-27 07:53:02,080][20381] Updated weights for policy 1, policy_version 323312 (0.0007) [2023-12-27 07:53:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 165634048. Throughput: 0: 10449.2, 1: 10329.3. Samples: 165641984. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:02,363][19170] Avg episode reward: [(0, '777.660'), (1, '927.450')] [2023-12-27 07:53:02,603][20347] Updated weights for policy 0, policy_version 323553 (0.0006) [2023-12-27 07:53:02,650][20347] Updated weights for policy 0, policy_version 323563 (0.0010) [2023-12-27 07:53:02,699][20347] Updated weights for policy 0, policy_version 323573 (0.0006) [2023-12-27 07:53:02,750][20347] Updated weights for policy 0, policy_version 323583 (0.0005) [2023-12-27 07:53:02,765][20381] Updated weights for policy 1, policy_version 323322 (0.0007) [2023-12-27 07:53:02,810][20381] Updated weights for policy 1, policy_version 323332 (0.0009) [2023-12-27 07:53:02,862][20381] Updated weights for policy 1, policy_version 323342 (0.0008) [2023-12-27 07:53:02,920][20381] Updated weights for policy 1, policy_version 323352 (0.0006) [2023-12-27 07:53:03,433][20347] Updated weights for policy 0, policy_version 323593 (0.0007) [2023-12-27 07:53:03,477][20347] Updated weights for policy 0, policy_version 323603 (0.0008) [2023-12-27 07:53:03,520][20347] Updated weights for policy 0, policy_version 323613 (0.0007) [2023-12-27 07:53:03,580][20381] Updated weights for policy 1, policy_version 323362 (0.0007) [2023-12-27 07:53:03,632][20381] Updated weights for policy 1, policy_version 323372 (0.0007) [2023-12-27 07:53:03,683][20381] Updated weights for policy 1, policy_version 323382 (0.0005) [2023-12-27 07:53:04,157][20347] Updated weights for policy 0, policy_version 323623 (0.0007) [2023-12-27 07:53:04,213][20347] Updated weights for policy 0, policy_version 323633 (0.0007) [2023-12-27 07:53:04,270][20347] Updated weights for policy 0, policy_version 323643 (0.0007) [2023-12-27 07:53:04,383][20381] Updated weights for policy 1, policy_version 323392 (0.0005) [2023-12-27 07:53:04,438][20381] Updated weights for policy 1, policy_version 323402 (0.0006) [2023-12-27 07:53:04,484][20381] Updated weights for policy 1, policy_version 323412 (0.0008) [2023-12-27 07:53:04,993][20347] Updated weights for policy 0, policy_version 323653 (0.0007) [2023-12-27 07:53:05,043][20347] Updated weights for policy 0, policy_version 323663 (0.0008) [2023-12-27 07:53:05,090][20347] Updated weights for policy 0, policy_version 323673 (0.0007) [2023-12-27 07:53:05,138][20381] Updated weights for policy 1, policy_version 323422 (0.0007) [2023-12-27 07:53:05,187][20381] Updated weights for policy 1, policy_version 323432 (0.0008) [2023-12-27 07:53:05,234][20381] Updated weights for policy 1, policy_version 323443 (0.0009) [2023-12-27 07:53:05,746][20347] Updated weights for policy 0, policy_version 323683 (0.0007) [2023-12-27 07:53:05,807][20347] Updated weights for policy 0, policy_version 323693 (0.0008) [2023-12-27 07:53:05,862][20347] Updated weights for policy 0, policy_version 323703 (0.0008) [2023-12-27 07:53:05,970][20381] Updated weights for policy 1, policy_version 323453 (0.0008) [2023-12-27 07:53:06,023][20381] Updated weights for policy 1, policy_version 323463 (0.0008) [2023-12-27 07:53:06,081][20381] Updated weights for policy 1, policy_version 323473 (0.0009) [2023-12-27 07:53:06,509][20347] Updated weights for policy 0, policy_version 323713 (0.0007) [2023-12-27 07:53:06,558][20347] Updated weights for policy 0, policy_version 323723 (0.0008) [2023-12-27 07:53:06,613][20347] Updated weights for policy 0, policy_version 323733 (0.0008) [2023-12-27 07:53:06,670][20347] Updated weights for policy 0, policy_version 323743 (0.0008) [2023-12-27 07:53:06,829][20381] Updated weights for policy 1, policy_version 323483 (0.0008) [2023-12-27 07:53:06,882][20381] Updated weights for policy 1, policy_version 323493 (0.0008) [2023-12-27 07:53:06,938][20381] Updated weights for policy 1, policy_version 323503 (0.0008) [2023-12-27 07:53:07,288][20347] Updated weights for policy 0, policy_version 323753 (0.0009) [2023-12-27 07:53:07,337][20347] Updated weights for policy 0, policy_version 323763 (0.0010) [2023-12-27 07:53:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 165732352. Throughput: 0: 10491.1, 1: 10305.3. Samples: 165705360. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:07,363][19170] Avg episode reward: [(0, '806.910'), (1, '957.220')] [2023-12-27 07:53:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000323512_82837504.pth... [2023-12-27 07:53:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000322328_82534400.pth [2023-12-27 07:53:07,393][20347] Updated weights for policy 0, policy_version 323773 (0.0010) [2023-12-27 07:53:07,407][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000323776_82903040.pth... [2023-12-27 07:53:07,412][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000322560_82591744.pth [2023-12-27 07:53:07,657][20381] Updated weights for policy 1, policy_version 323513 (0.0008) [2023-12-27 07:53:07,715][20381] Updated weights for policy 1, policy_version 323523 (0.0007) [2023-12-27 07:53:07,762][20381] Updated weights for policy 1, policy_version 323533 (0.0009) [2023-12-27 07:53:07,808][20381] Updated weights for policy 1, policy_version 323543 (0.0010) [2023-12-27 07:53:08,021][20347] Updated weights for policy 0, policy_version 323783 (0.0007) [2023-12-27 07:53:08,067][20347] Updated weights for policy 0, policy_version 323793 (0.0005) [2023-12-27 07:53:08,122][20347] Updated weights for policy 0, policy_version 323803 (0.0007) [2023-12-27 07:53:08,529][20381] Updated weights for policy 1, policy_version 323553 (0.0008) [2023-12-27 07:53:08,590][20381] Updated weights for policy 1, policy_version 323563 (0.0008) [2023-12-27 07:53:08,645][20381] Updated weights for policy 1, policy_version 323573 (0.0008) [2023-12-27 07:53:08,760][20347] Updated weights for policy 0, policy_version 323813 (0.0010) [2023-12-27 07:53:08,762][20291] KL-divergence is very high: 121.6052 [2023-12-27 07:53:08,777][20291] KL-divergence is very high: 250.6593 [2023-12-27 07:53:08,783][20291] KL-divergence is very high: 325.9981 [2023-12-27 07:53:08,788][20291] KL-divergence is very high: 245.8604 [2023-12-27 07:53:08,793][20291] KL-divergence is very high: 129.6353 [2023-12-27 07:53:08,824][20347] Updated weights for policy 0, policy_version 323823 (0.0007) [2023-12-27 07:53:08,876][20347] Updated weights for policy 0, policy_version 323833 (0.0007) [2023-12-27 07:53:09,407][20381] Updated weights for policy 1, policy_version 323583 (0.0007) [2023-12-27 07:53:09,471][20381] Updated weights for policy 1, policy_version 323593 (0.0006) [2023-12-27 07:53:09,501][20347] Updated weights for policy 0, policy_version 323843 (0.0007) [2023-12-27 07:53:09,538][20381] Updated weights for policy 1, policy_version 323603 (0.0006) [2023-12-27 07:53:09,562][20347] Updated weights for policy 0, policy_version 323853 (0.0007) [2023-12-27 07:53:09,628][20347] Updated weights for policy 0, policy_version 323863 (0.0009) [2023-12-27 07:53:10,269][20381] Updated weights for policy 1, policy_version 323613 (0.0007) [2023-12-27 07:53:10,280][20347] Updated weights for policy 0, policy_version 323873 (0.0009) [2023-12-27 07:53:10,325][20381] Updated weights for policy 1, policy_version 323623 (0.0007) [2023-12-27 07:53:10,343][20347] Updated weights for policy 0, policy_version 323883 (0.0005) [2023-12-27 07:53:10,382][20381] Updated weights for policy 1, policy_version 323633 (0.0009) [2023-12-27 07:53:10,403][20347] Updated weights for policy 0, policy_version 323893 (0.0007) [2023-12-27 07:53:10,455][20347] Updated weights for policy 0, policy_version 323903 (0.0007) [2023-12-27 07:53:11,067][20381] Updated weights for policy 1, policy_version 323643 (0.0007) [2023-12-27 07:53:11,069][20347] Updated weights for policy 0, policy_version 323913 (0.0008) [2023-12-27 07:53:11,122][20381] Updated weights for policy 1, policy_version 323653 (0.0006) [2023-12-27 07:53:11,124][20347] Updated weights for policy 0, policy_version 323923 (0.0007) [2023-12-27 07:53:11,190][20347] Updated weights for policy 0, policy_version 323933 (0.0008) [2023-12-27 07:53:11,190][20381] Updated weights for policy 1, policy_version 323663 (0.0009) [2023-12-27 07:53:11,888][20381] Updated weights for policy 1, policy_version 323673 (0.0008) [2023-12-27 07:53:11,920][20347] Updated weights for policy 0, policy_version 323943 (0.0006) [2023-12-27 07:53:11,947][20381] Updated weights for policy 1, policy_version 323683 (0.0007) [2023-12-27 07:53:11,972][20347] Updated weights for policy 0, policy_version 323953 (0.0005) [2023-12-27 07:53:12,001][20381] Updated weights for policy 1, policy_version 323693 (0.0006) [2023-12-27 07:53:12,028][20347] Updated weights for policy 0, policy_version 323963 (0.0008) [2023-12-27 07:53:12,055][20381] Updated weights for policy 1, policy_version 323703 (0.0005) [2023-12-27 07:53:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 165838848. Throughput: 0: 10532.9, 1: 10216.5. Samples: 165828732. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:12,363][19170] Avg episode reward: [(0, '849.650'), (1, '990.330')] [2023-12-27 07:53:12,670][20381] Updated weights for policy 1, policy_version 323713 (0.0007) [2023-12-27 07:53:12,707][20347] Updated weights for policy 0, policy_version 323973 (0.0007) [2023-12-27 07:53:12,729][20381] Updated weights for policy 1, policy_version 323723 (0.0007) [2023-12-27 07:53:12,766][20347] Updated weights for policy 0, policy_version 323983 (0.0006) [2023-12-27 07:53:12,787][20381] Updated weights for policy 1, policy_version 323733 (0.0007) [2023-12-27 07:53:12,822][20347] Updated weights for policy 0, policy_version 323993 (0.0010) [2023-12-27 07:53:13,405][20347] Updated weights for policy 0, policy_version 324003 (0.0010) [2023-12-27 07:53:13,414][20381] Updated weights for policy 1, policy_version 323743 (0.0006) [2023-12-27 07:53:13,458][20347] Updated weights for policy 0, policy_version 324013 (0.0011) [2023-12-27 07:53:13,468][20381] Updated weights for policy 1, policy_version 323753 (0.0005) [2023-12-27 07:53:13,524][20347] Updated weights for policy 0, policy_version 324023 (0.0010) [2023-12-27 07:53:13,529][20381] Updated weights for policy 1, policy_version 323763 (0.0005) [2023-12-27 07:53:14,132][20347] Updated weights for policy 0, policy_version 324033 (0.0010) [2023-12-27 07:53:14,193][20347] Updated weights for policy 0, policy_version 324043 (0.0006) [2023-12-27 07:53:14,208][20381] Updated weights for policy 1, policy_version 323773 (0.0006) [2023-12-27 07:53:14,251][20347] Updated weights for policy 0, policy_version 324053 (0.0005) [2023-12-27 07:53:14,290][20381] Updated weights for policy 1, policy_version 323783 (0.0007) [2023-12-27 07:53:14,310][20347] Updated weights for policy 0, policy_version 324063 (0.0005) [2023-12-27 07:53:14,347][20381] Updated weights for policy 1, policy_version 323793 (0.0009) [2023-12-27 07:53:14,851][20347] Updated weights for policy 0, policy_version 324073 (0.0007) [2023-12-27 07:53:14,904][20347] Updated weights for policy 0, policy_version 324083 (0.0008) [2023-12-27 07:53:14,959][20347] Updated weights for policy 0, policy_version 324093 (0.0008) [2023-12-27 07:53:15,053][20381] Updated weights for policy 1, policy_version 323803 (0.0009) [2023-12-27 07:53:15,111][20381] Updated weights for policy 1, policy_version 323813 (0.0010) [2023-12-27 07:53:15,169][20381] Updated weights for policy 1, policy_version 323823 (0.0010) [2023-12-27 07:53:15,588][20347] Updated weights for policy 0, policy_version 324103 (0.0005) [2023-12-27 07:53:15,636][20347] Updated weights for policy 0, policy_version 324113 (0.0005) [2023-12-27 07:53:15,682][20347] Updated weights for policy 0, policy_version 324123 (0.0005) [2023-12-27 07:53:15,879][20381] Updated weights for policy 1, policy_version 323833 (0.0010) [2023-12-27 07:53:15,941][20381] Updated weights for policy 1, policy_version 323843 (0.0007) [2023-12-27 07:53:15,988][20381] Updated weights for policy 1, policy_version 323853 (0.0011) [2023-12-27 07:53:16,044][20381] Updated weights for policy 1, policy_version 323863 (0.0009) [2023-12-27 07:53:16,233][20347] Updated weights for policy 0, policy_version 324133 (0.0006) [2023-12-27 07:53:16,294][20347] Updated weights for policy 0, policy_version 324143 (0.0008) [2023-12-27 07:53:16,344][20347] Updated weights for policy 0, policy_version 324153 (0.0007) [2023-12-27 07:53:16,717][20381] Updated weights for policy 1, policy_version 323873 (0.0010) [2023-12-27 07:53:16,777][20381] Updated weights for policy 1, policy_version 323883 (0.0010) [2023-12-27 07:53:16,833][20381] Updated weights for policy 1, policy_version 323893 (0.0008) [2023-12-27 07:53:17,035][20347] Updated weights for policy 0, policy_version 324163 (0.0007) [2023-12-27 07:53:17,089][20347] Updated weights for policy 0, policy_version 324173 (0.0010) [2023-12-27 07:53:17,144][20347] Updated weights for policy 0, policy_version 324183 (0.0010) [2023-12-27 07:53:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 165945344. Throughput: 0: 10567.3, 1: 10222.6. Samples: 165955612. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:17,363][19170] Avg episode reward: [(0, '771.810'), (1, '990.330')] [2023-12-27 07:53:17,471][20381] Updated weights for policy 1, policy_version 323903 (0.0006) [2023-12-27 07:53:17,527][20381] Updated weights for policy 1, policy_version 323913 (0.0010) [2023-12-27 07:53:17,589][20381] Updated weights for policy 1, policy_version 323923 (0.0011) [2023-12-27 07:53:17,784][20347] Updated weights for policy 0, policy_version 324193 (0.0006) [2023-12-27 07:53:17,833][20347] Updated weights for policy 0, policy_version 324203 (0.0007) [2023-12-27 07:53:17,880][20347] Updated weights for policy 0, policy_version 324213 (0.0009) [2023-12-27 07:53:17,931][20347] Updated weights for policy 0, policy_version 324223 (0.0005) [2023-12-27 07:53:18,154][20381] Updated weights for policy 1, policy_version 323933 (0.0009) [2023-12-27 07:53:18,205][20381] Updated weights for policy 1, policy_version 323943 (0.0010) [2023-12-27 07:53:18,262][20381] Updated weights for policy 1, policy_version 323953 (0.0010) [2023-12-27 07:53:18,631][20347] Updated weights for policy 0, policy_version 324233 (0.0010) [2023-12-27 07:53:18,688][20347] Updated weights for policy 0, policy_version 324243 (0.0011) [2023-12-27 07:53:18,748][20347] Updated weights for policy 0, policy_version 324253 (0.0010) [2023-12-27 07:53:18,891][20381] Updated weights for policy 1, policy_version 323963 (0.0009) [2023-12-27 07:53:18,944][20381] Updated weights for policy 1, policy_version 323973 (0.0006) [2023-12-27 07:53:18,996][20381] Updated weights for policy 1, policy_version 323983 (0.0005) [2023-12-27 07:53:19,379][20347] Updated weights for policy 0, policy_version 324263 (0.0007) [2023-12-27 07:53:19,440][20347] Updated weights for policy 0, policy_version 324273 (0.0007) [2023-12-27 07:53:19,502][20347] Updated weights for policy 0, policy_version 324283 (0.0007) [2023-12-27 07:53:19,673][20381] Updated weights for policy 1, policy_version 323993 (0.0009) [2023-12-27 07:53:19,735][20381] Updated weights for policy 1, policy_version 324003 (0.0007) [2023-12-27 07:53:19,794][20381] Updated weights for policy 1, policy_version 324013 (0.0008) [2023-12-27 07:53:19,856][20381] Updated weights for policy 1, policy_version 324023 (0.0009) [2023-12-27 07:53:20,221][20347] Updated weights for policy 0, policy_version 324293 (0.0008) [2023-12-27 07:53:20,288][20347] Updated weights for policy 0, policy_version 324303 (0.0011) [2023-12-27 07:53:20,352][20347] Updated weights for policy 0, policy_version 324313 (0.0011) [2023-12-27 07:53:20,543][20381] Updated weights for policy 1, policy_version 324033 (0.0011) [2023-12-27 07:53:20,605][20381] Updated weights for policy 1, policy_version 324043 (0.0011) [2023-12-27 07:53:20,665][20381] Updated weights for policy 1, policy_version 324053 (0.0011) [2023-12-27 07:53:21,031][20347] Updated weights for policy 0, policy_version 324323 (0.0010) [2023-12-27 07:53:21,090][20347] Updated weights for policy 0, policy_version 324333 (0.0008) [2023-12-27 07:53:21,149][20347] Updated weights for policy 0, policy_version 324343 (0.0007) [2023-12-27 07:53:21,394][20381] Updated weights for policy 1, policy_version 324063 (0.0010) [2023-12-27 07:53:21,454][20381] Updated weights for policy 1, policy_version 324073 (0.0006) [2023-12-27 07:53:21,512][20381] Updated weights for policy 1, policy_version 324083 (0.0008) [2023-12-27 07:53:21,889][20347] Updated weights for policy 0, policy_version 324353 (0.0007) [2023-12-27 07:53:21,946][20347] Updated weights for policy 0, policy_version 324363 (0.0007) [2023-12-27 07:53:22,009][20347] Updated weights for policy 0, policy_version 324373 (0.0007) [2023-12-27 07:53:22,069][20347] Updated weights for policy 0, policy_version 324383 (0.0007) [2023-12-27 07:53:22,207][20381] Updated weights for policy 1, policy_version 324093 (0.0010) [2023-12-27 07:53:22,270][20381] Updated weights for policy 1, policy_version 324103 (0.0010) [2023-12-27 07:53:22,335][20381] Updated weights for policy 1, policy_version 324113 (0.0009) [2023-12-27 07:53:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 166043648. Throughput: 0: 10490.8, 1: 10243.6. Samples: 166018532. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:22,363][19170] Avg episode reward: [(0, '836.580'), (1, '1000.000')] [2023-12-27 07:53:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000324384_83058688.pth... [2023-12-27 07:53:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000323168_82747392.pth [2023-12-27 07:53:22,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000324120_82993152.pth... [2023-12-27 07:53:22,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000322904_82681856.pth [2023-12-27 07:53:22,736][20347] Updated weights for policy 0, policy_version 324393 (0.0008) [2023-12-27 07:53:22,793][20347] Updated weights for policy 0, policy_version 324403 (0.0007) [2023-12-27 07:53:22,855][20347] Updated weights for policy 0, policy_version 324413 (0.0007) [2023-12-27 07:53:23,001][20381] Updated weights for policy 1, policy_version 324123 (0.0007) [2023-12-27 07:53:23,061][20381] Updated weights for policy 1, policy_version 324133 (0.0011) [2023-12-27 07:53:23,120][20381] Updated weights for policy 1, policy_version 324143 (0.0011) [2023-12-27 07:53:23,521][20347] Updated weights for policy 0, policy_version 324423 (0.0007) [2023-12-27 07:53:23,582][20347] Updated weights for policy 0, policy_version 324433 (0.0008) [2023-12-27 07:53:23,649][20347] Updated weights for policy 0, policy_version 324443 (0.0008) [2023-12-27 07:53:23,752][20381] Updated weights for policy 1, policy_version 324153 (0.0010) [2023-12-27 07:53:23,812][20381] Updated weights for policy 1, policy_version 324163 (0.0005) [2023-12-27 07:53:23,871][20381] Updated weights for policy 1, policy_version 324173 (0.0008) [2023-12-27 07:53:23,931][20381] Updated weights for policy 1, policy_version 324183 (0.0008) [2023-12-27 07:53:24,318][20347] Updated weights for policy 0, policy_version 324453 (0.0007) [2023-12-27 07:53:24,379][20347] Updated weights for policy 0, policy_version 324463 (0.0007) [2023-12-27 07:53:24,434][20347] Updated weights for policy 0, policy_version 324473 (0.0007) [2023-12-27 07:53:24,511][20381] Updated weights for policy 1, policy_version 324193 (0.0009) [2023-12-27 07:53:24,576][20381] Updated weights for policy 1, policy_version 324203 (0.0007) [2023-12-27 07:53:24,637][20381] Updated weights for policy 1, policy_version 324213 (0.0006) [2023-12-27 07:53:25,089][20347] Updated weights for policy 0, policy_version 324483 (0.0008) [2023-12-27 07:53:25,151][20347] Updated weights for policy 0, policy_version 324493 (0.0008) [2023-12-27 07:53:25,206][20347] Updated weights for policy 0, policy_version 324503 (0.0008) [2023-12-27 07:53:25,298][20381] Updated weights for policy 1, policy_version 324223 (0.0007) [2023-12-27 07:53:25,353][20381] Updated weights for policy 1, policy_version 324233 (0.0008) [2023-12-27 07:53:25,409][20381] Updated weights for policy 1, policy_version 324243 (0.0008) [2023-12-27 07:53:25,851][20347] Updated weights for policy 0, policy_version 324513 (0.0008) [2023-12-27 07:53:25,909][20347] Updated weights for policy 0, policy_version 324523 (0.0007) [2023-12-27 07:53:25,962][20347] Updated weights for policy 0, policy_version 324533 (0.0007) [2023-12-27 07:53:26,023][20347] Updated weights for policy 0, policy_version 324543 (0.0007) [2023-12-27 07:53:26,124][20381] Updated weights for policy 1, policy_version 324253 (0.0007) [2023-12-27 07:53:26,186][20381] Updated weights for policy 1, policy_version 324263 (0.0005) [2023-12-27 07:53:26,245][20381] Updated weights for policy 1, policy_version 324273 (0.0006) [2023-12-27 07:53:26,719][20347] Updated weights for policy 0, policy_version 324553 (0.0008) [2023-12-27 07:53:26,777][20347] Updated weights for policy 0, policy_version 324563 (0.0007) [2023-12-27 07:53:26,832][20347] Updated weights for policy 0, policy_version 324573 (0.0007) [2023-12-27 07:53:26,871][20381] Updated weights for policy 1, policy_version 324283 (0.0005) [2023-12-27 07:53:26,928][20381] Updated weights for policy 1, policy_version 324293 (0.0006) [2023-12-27 07:53:26,987][20381] Updated weights for policy 1, policy_version 324303 (0.0008) [2023-12-27 07:53:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 166150144. Throughput: 0: 10443.2, 1: 10241.7. Samples: 166141152. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:27,363][19170] Avg episode reward: [(0, '856.200'), (1, '1000.000')] [2023-12-27 07:53:27,507][20347] Updated weights for policy 0, policy_version 324583 (0.0006) [2023-12-27 07:53:27,569][20347] Updated weights for policy 0, policy_version 324593 (0.0005) [2023-12-27 07:53:27,628][20347] Updated weights for policy 0, policy_version 324603 (0.0007) [2023-12-27 07:53:27,662][20381] Updated weights for policy 1, policy_version 324313 (0.0008) [2023-12-27 07:53:27,711][20381] Updated weights for policy 1, policy_version 324323 (0.0008) [2023-12-27 07:53:27,756][20381] Updated weights for policy 1, policy_version 324333 (0.0007) [2023-12-27 07:53:27,800][20381] Updated weights for policy 1, policy_version 324343 (0.0007) [2023-12-27 07:53:28,269][20347] Updated weights for policy 0, policy_version 324613 (0.0007) [2023-12-27 07:53:28,324][20347] Updated weights for policy 0, policy_version 324623 (0.0008) [2023-12-27 07:53:28,390][20347] Updated weights for policy 0, policy_version 324633 (0.0007) [2023-12-27 07:53:28,506][20381] Updated weights for policy 1, policy_version 324353 (0.0008) [2023-12-27 07:53:28,566][20381] Updated weights for policy 1, policy_version 324363 (0.0008) [2023-12-27 07:53:28,627][20381] Updated weights for policy 1, policy_version 324373 (0.0007) [2023-12-27 07:53:29,040][20347] Updated weights for policy 0, policy_version 324643 (0.0008) [2023-12-27 07:53:29,095][20347] Updated weights for policy 0, policy_version 324653 (0.0008) [2023-12-27 07:53:29,148][20347] Updated weights for policy 0, policy_version 324663 (0.0007) [2023-12-27 07:53:29,338][20381] Updated weights for policy 1, policy_version 324383 (0.0008) [2023-12-27 07:53:29,403][20381] Updated weights for policy 1, policy_version 324393 (0.0007) [2023-12-27 07:53:29,467][20381] Updated weights for policy 1, policy_version 324403 (0.0005) [2023-12-27 07:53:29,850][20347] Updated weights for policy 0, policy_version 324673 (0.0007) [2023-12-27 07:53:29,905][20347] Updated weights for policy 0, policy_version 324683 (0.0008) [2023-12-27 07:53:29,967][20347] Updated weights for policy 0, policy_version 324693 (0.0008) [2023-12-27 07:53:30,033][20347] Updated weights for policy 0, policy_version 324703 (0.0008) [2023-12-27 07:53:30,150][20381] Updated weights for policy 1, policy_version 324413 (0.0008) [2023-12-27 07:53:30,210][20381] Updated weights for policy 1, policy_version 324423 (0.0007) [2023-12-27 07:53:30,265][20381] Updated weights for policy 1, policy_version 324433 (0.0008) [2023-12-27 07:53:30,701][20347] Updated weights for policy 0, policy_version 324713 (0.0006) [2023-12-27 07:53:30,747][20347] Updated weights for policy 0, policy_version 324723 (0.0005) [2023-12-27 07:53:30,792][20347] Updated weights for policy 0, policy_version 324733 (0.0005) [2023-12-27 07:53:31,006][20381] Updated weights for policy 1, policy_version 324443 (0.0007) [2023-12-27 07:53:31,061][20381] Updated weights for policy 1, policy_version 324453 (0.0007) [2023-12-27 07:53:31,113][20381] Updated weights for policy 1, policy_version 324463 (0.0007) [2023-12-27 07:53:31,404][20347] Updated weights for policy 0, policy_version 324743 (0.0007) [2023-12-27 07:53:31,460][20347] Updated weights for policy 0, policy_version 324753 (0.0008) [2023-12-27 07:53:31,515][20347] Updated weights for policy 0, policy_version 324763 (0.0007) [2023-12-27 07:53:31,845][20381] Updated weights for policy 1, policy_version 324473 (0.0008) [2023-12-27 07:53:31,897][20381] Updated weights for policy 1, policy_version 324483 (0.0008) [2023-12-27 07:53:31,943][20381] Updated weights for policy 1, policy_version 324493 (0.0007) [2023-12-27 07:53:31,993][20381] Updated weights for policy 1, policy_version 324503 (0.0008) [2023-12-27 07:53:32,183][20347] Updated weights for policy 0, policy_version 324773 (0.0007) [2023-12-27 07:53:32,243][20347] Updated weights for policy 0, policy_version 324783 (0.0007) [2023-12-27 07:53:32,299][20347] Updated weights for policy 0, policy_version 324793 (0.0008) [2023-12-27 07:53:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 166256640. Throughput: 0: 10450.4, 1: 10221.8. Samples: 166263048. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:32,363][19170] Avg episode reward: [(0, '914.980'), (1, '1000.000')] [2023-12-27 07:53:32,691][20381] Updated weights for policy 1, policy_version 324513 (0.0006) [2023-12-27 07:53:32,752][20381] Updated weights for policy 1, policy_version 324523 (0.0005) [2023-12-27 07:53:32,800][20381] Updated weights for policy 1, policy_version 324533 (0.0005) [2023-12-27 07:53:32,865][20347] Updated weights for policy 0, policy_version 324803 (0.0007) [2023-12-27 07:53:32,916][20347] Updated weights for policy 0, policy_version 324813 (0.0008) [2023-12-27 07:53:32,961][20347] Updated weights for policy 0, policy_version 324823 (0.0009) [2023-12-27 07:53:33,304][20381] Updated weights for policy 1, policy_version 324543 (0.0005) [2023-12-27 07:53:33,364][20381] Updated weights for policy 1, policy_version 324553 (0.0005) [2023-12-27 07:53:33,423][20381] Updated weights for policy 1, policy_version 324563 (0.0005) [2023-12-27 07:53:33,697][20347] Updated weights for policy 0, policy_version 324833 (0.0007) [2023-12-27 07:53:33,756][20347] Updated weights for policy 0, policy_version 324843 (0.0008) [2023-12-27 07:53:33,809][20347] Updated weights for policy 0, policy_version 324853 (0.0007) [2023-12-27 07:53:33,863][20347] Updated weights for policy 0, policy_version 324863 (0.0007) [2023-12-27 07:53:33,986][20381] Updated weights for policy 1, policy_version 324573 (0.0007) [2023-12-27 07:53:34,032][20381] Updated weights for policy 1, policy_version 324583 (0.0006) [2023-12-27 07:53:34,082][20381] Updated weights for policy 1, policy_version 324593 (0.0005) [2023-12-27 07:53:34,572][20347] Updated weights for policy 0, policy_version 324873 (0.0007) [2023-12-27 07:53:34,622][20347] Updated weights for policy 0, policy_version 324883 (0.0007) [2023-12-27 07:53:34,680][20381] Updated weights for policy 1, policy_version 324603 (0.0006) [2023-12-27 07:53:34,682][20347] Updated weights for policy 0, policy_version 324893 (0.0007) [2023-12-27 07:53:34,742][20381] Updated weights for policy 1, policy_version 324613 (0.0008) [2023-12-27 07:53:34,793][20381] Updated weights for policy 1, policy_version 324623 (0.0007) [2023-12-27 07:53:35,383][20347] Updated weights for policy 0, policy_version 324903 (0.0007) [2023-12-27 07:53:35,430][20347] Updated weights for policy 0, policy_version 324913 (0.0010) [2023-12-27 07:53:35,475][20347] Updated weights for policy 0, policy_version 324923 (0.0010) [2023-12-27 07:53:35,480][20381] Updated weights for policy 1, policy_version 324633 (0.0007) [2023-12-27 07:53:35,540][20381] Updated weights for policy 1, policy_version 324643 (0.0009) [2023-12-27 07:53:35,588][20381] Updated weights for policy 1, policy_version 324653 (0.0007) [2023-12-27 07:53:35,644][20381] Updated weights for policy 1, policy_version 324663 (0.0007) [2023-12-27 07:53:36,082][20347] Updated weights for policy 0, policy_version 324933 (0.0010) [2023-12-27 07:53:36,142][20347] Updated weights for policy 0, policy_version 324943 (0.0011) [2023-12-27 07:53:36,204][20347] Updated weights for policy 0, policy_version 324953 (0.0010) [2023-12-27 07:53:36,327][20381] Updated weights for policy 1, policy_version 324673 (0.0008) [2023-12-27 07:53:36,390][20381] Updated weights for policy 1, policy_version 324683 (0.0007) [2023-12-27 07:53:36,448][20381] Updated weights for policy 1, policy_version 324693 (0.0006) [2023-12-27 07:53:36,891][20347] Updated weights for policy 0, policy_version 324963 (0.0010) [2023-12-27 07:53:36,947][20347] Updated weights for policy 0, policy_version 324973 (0.0007) [2023-12-27 07:53:36,997][20347] Updated weights for policy 0, policy_version 324983 (0.0006) [2023-12-27 07:53:36,999][20381] Updated weights for policy 1, policy_version 324703 (0.0006) [2023-12-27 07:53:37,052][20381] Updated weights for policy 1, policy_version 324713 (0.0006) [2023-12-27 07:53:37,107][20381] Updated weights for policy 1, policy_version 324723 (0.0007) [2023-12-27 07:53:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 166363136. Throughput: 0: 10428.0, 1: 10323.5. Samples: 166329104. Policy #0 lag: (min: 25.0, avg: 51.9, max: 57.0) [2023-12-27 07:53:37,363][19170] Avg episode reward: [(0, '904.590'), (1, '1000.000')] [2023-12-27 07:53:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000324992_83214336.pth... [2023-12-27 07:53:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000324728_83148800.pth... [2023-12-27 07:53:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000323776_82903040.pth [2023-12-27 07:53:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000323512_82837504.pth [2023-12-27 07:53:37,679][20347] Updated weights for policy 0, policy_version 324993 (0.0005) [2023-12-27 07:53:37,743][20347] Updated weights for policy 0, policy_version 325003 (0.0006) [2023-12-27 07:53:37,770][20381] Updated weights for policy 1, policy_version 324733 (0.0008) [2023-12-27 07:53:37,804][20347] Updated weights for policy 0, policy_version 325013 (0.0006) [2023-12-27 07:53:37,827][20381] Updated weights for policy 1, policy_version 324743 (0.0007) [2023-12-27 07:53:37,864][20347] Updated weights for policy 0, policy_version 325023 (0.0005) [2023-12-27 07:53:37,877][20381] Updated weights for policy 1, policy_version 324753 (0.0008) [2023-12-27 07:53:38,423][20347] Updated weights for policy 0, policy_version 325033 (0.0010) [2023-12-27 07:53:38,485][20347] Updated weights for policy 0, policy_version 325043 (0.0010) [2023-12-27 07:53:38,543][20347] Updated weights for policy 0, policy_version 325053 (0.0010) [2023-12-27 07:53:38,561][20381] Updated weights for policy 1, policy_version 324763 (0.0008) [2023-12-27 07:53:38,621][20381] Updated weights for policy 1, policy_version 324773 (0.0011) [2023-12-27 07:53:38,673][20381] Updated weights for policy 1, policy_version 324783 (0.0010) [2023-12-27 07:53:39,254][20347] Updated weights for policy 0, policy_version 325063 (0.0008) [2023-12-27 07:53:39,306][20347] Updated weights for policy 0, policy_version 325073 (0.0011) [2023-12-27 07:53:39,339][20381] Updated weights for policy 1, policy_version 324793 (0.0008) [2023-12-27 07:53:39,371][20347] Updated weights for policy 0, policy_version 325083 (0.0011) [2023-12-27 07:53:39,405][20381] Updated weights for policy 1, policy_version 324803 (0.0010) [2023-12-27 07:53:39,476][20381] Updated weights for policy 1, policy_version 324813 (0.0010) [2023-12-27 07:53:39,532][20381] Updated weights for policy 1, policy_version 324823 (0.0011) [2023-12-27 07:53:40,107][20347] Updated weights for policy 0, policy_version 325093 (0.0008) [2023-12-27 07:53:40,167][20347] Updated weights for policy 0, policy_version 325103 (0.0007) [2023-12-27 07:53:40,222][20381] Updated weights for policy 1, policy_version 324833 (0.0006) [2023-12-27 07:53:40,228][20347] Updated weights for policy 0, policy_version 325113 (0.0006) [2023-12-27 07:53:40,281][20381] Updated weights for policy 1, policy_version 324843 (0.0008) [2023-12-27 07:53:40,339][20381] Updated weights for policy 1, policy_version 324853 (0.0008) [2023-12-27 07:53:40,844][20347] Updated weights for policy 0, policy_version 325123 (0.0007) [2023-12-27 07:53:40,897][20347] Updated weights for policy 0, policy_version 325133 (0.0005) [2023-12-27 07:53:40,959][20347] Updated weights for policy 0, policy_version 325143 (0.0005) [2023-12-27 07:53:41,088][20381] Updated weights for policy 1, policy_version 324863 (0.0009) [2023-12-27 07:53:41,145][20381] Updated weights for policy 1, policy_version 324873 (0.0008) [2023-12-27 07:53:41,209][20381] Updated weights for policy 1, policy_version 324883 (0.0006) [2023-12-27 07:53:41,689][20347] Updated weights for policy 0, policy_version 325153 (0.0007) [2023-12-27 07:53:41,752][20347] Updated weights for policy 0, policy_version 325163 (0.0008) [2023-12-27 07:53:41,803][20347] Updated weights for policy 0, policy_version 325173 (0.0008) [2023-12-27 07:53:41,857][20381] Updated weights for policy 1, policy_version 324893 (0.0008) [2023-12-27 07:53:41,858][20347] Updated weights for policy 0, policy_version 325183 (0.0010) [2023-12-27 07:53:41,908][20292] KL-divergence is very high: 122.4417 [2023-12-27 07:53:41,910][20381] Updated weights for policy 1, policy_version 324903 (0.0009) [2023-12-27 07:53:41,965][20381] Updated weights for policy 1, policy_version 324913 (0.0008) [2023-12-27 07:53:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 166461440. Throughput: 0: 10459.9, 1: 10251.3. Samples: 166452992. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:53:42,363][19170] Avg episode reward: [(0, '849.070'), (1, '980.760')] [2023-12-27 07:53:42,637][20347] Updated weights for policy 0, policy_version 325193 (0.0010) [2023-12-27 07:53:42,662][20381] Updated weights for policy 1, policy_version 324923 (0.0009) [2023-12-27 07:53:42,689][20347] Updated weights for policy 0, policy_version 325203 (0.0010) [2023-12-27 07:53:42,721][20381] Updated weights for policy 1, policy_version 324933 (0.0010) [2023-12-27 07:53:42,742][20347] Updated weights for policy 0, policy_version 325213 (0.0010) [2023-12-27 07:53:42,777][20381] Updated weights for policy 1, policy_version 324943 (0.0007) [2023-12-27 07:53:43,333][20381] Updated weights for policy 1, policy_version 324953 (0.0005) [2023-12-27 07:53:43,371][20347] Updated weights for policy 0, policy_version 325223 (0.0007) [2023-12-27 07:53:43,394][20381] Updated weights for policy 1, policy_version 324963 (0.0005) [2023-12-27 07:53:43,428][20347] Updated weights for policy 0, policy_version 325233 (0.0005) [2023-12-27 07:53:43,451][20381] Updated weights for policy 1, policy_version 324973 (0.0008) [2023-12-27 07:53:43,476][20347] Updated weights for policy 0, policy_version 325243 (0.0005) [2023-12-27 07:53:43,503][20381] Updated weights for policy 1, policy_version 324983 (0.0010) [2023-12-27 07:53:44,005][20347] Updated weights for policy 0, policy_version 325253 (0.0007) [2023-12-27 07:53:44,046][20381] Updated weights for policy 1, policy_version 324993 (0.0006) [2023-12-27 07:53:44,056][20347] Updated weights for policy 0, policy_version 325263 (0.0007) [2023-12-27 07:53:44,105][20347] Updated weights for policy 0, policy_version 325273 (0.0005) [2023-12-27 07:53:44,105][20381] Updated weights for policy 1, policy_version 325003 (0.0008) [2023-12-27 07:53:44,154][20381] Updated weights for policy 1, policy_version 325013 (0.0010) [2023-12-27 07:53:44,731][20347] Updated weights for policy 0, policy_version 325283 (0.0007) [2023-12-27 07:53:44,752][20381] Updated weights for policy 1, policy_version 325023 (0.0010) [2023-12-27 07:53:44,783][20347] Updated weights for policy 0, policy_version 325293 (0.0010) [2023-12-27 07:53:44,812][20381] Updated weights for policy 1, policy_version 325033 (0.0010) [2023-12-27 07:53:44,836][20347] Updated weights for policy 0, policy_version 325303 (0.0007) [2023-12-27 07:53:44,870][20381] Updated weights for policy 1, policy_version 325043 (0.0006) [2023-12-27 07:53:45,541][20381] Updated weights for policy 1, policy_version 325053 (0.0010) [2023-12-27 07:53:45,563][20347] Updated weights for policy 0, policy_version 325313 (0.0008) [2023-12-27 07:53:45,598][20381] Updated weights for policy 1, policy_version 325063 (0.0011) [2023-12-27 07:53:45,614][20347] Updated weights for policy 0, policy_version 325323 (0.0010) [2023-12-27 07:53:45,647][20381] Updated weights for policy 1, policy_version 325073 (0.0010) [2023-12-27 07:53:45,668][20347] Updated weights for policy 0, policy_version 325333 (0.0010) [2023-12-27 07:53:45,716][20347] Updated weights for policy 0, policy_version 325343 (0.0010) [2023-12-27 07:53:46,342][20381] Updated weights for policy 1, policy_version 325083 (0.0011) [2023-12-27 07:53:46,397][20381] Updated weights for policy 1, policy_version 325093 (0.0010) [2023-12-27 07:53:46,400][20347] Updated weights for policy 0, policy_version 325353 (0.0010) [2023-12-27 07:53:46,454][20347] Updated weights for policy 0, policy_version 325363 (0.0010) [2023-12-27 07:53:46,455][20381] Updated weights for policy 1, policy_version 325103 (0.0009) [2023-12-27 07:53:46,510][20347] Updated weights for policy 0, policy_version 325373 (0.0010) [2023-12-27 07:53:46,991][20381] Updated weights for policy 1, policy_version 325113 (0.0006) [2023-12-27 07:53:47,054][20381] Updated weights for policy 1, policy_version 325123 (0.0011) [2023-12-27 07:53:47,117][20381] Updated weights for policy 1, policy_version 325133 (0.0011) [2023-12-27 07:53:47,172][20381] Updated weights for policy 1, policy_version 325143 (0.0011) [2023-12-27 07:53:47,175][20347] Updated weights for policy 0, policy_version 325383 (0.0010) [2023-12-27 07:53:47,233][20347] Updated weights for policy 0, policy_version 325393 (0.0010) [2023-12-27 07:53:47,296][20347] Updated weights for policy 0, policy_version 325403 (0.0010) [2023-12-27 07:53:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 166576128. Throughput: 0: 10502.0, 1: 10371.2. Samples: 166581280. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:53:47,364][19170] Avg episode reward: [(0, '841.420'), (1, '971.110')] [2023-12-27 07:53:47,855][20381] Updated weights for policy 1, policy_version 325153 (0.0011) [2023-12-27 07:53:47,885][20347] Updated weights for policy 0, policy_version 325413 (0.0006) [2023-12-27 07:53:47,913][20381] Updated weights for policy 1, policy_version 325163 (0.0010) [2023-12-27 07:53:47,940][20347] Updated weights for policy 0, policy_version 325423 (0.0006) [2023-12-27 07:53:47,966][20381] Updated weights for policy 1, policy_version 325173 (0.0010) [2023-12-27 07:53:48,000][20347] Updated weights for policy 0, policy_version 325433 (0.0006) [2023-12-27 07:53:48,582][20347] Updated weights for policy 0, policy_version 325443 (0.0007) [2023-12-27 07:53:48,631][20347] Updated weights for policy 0, policy_version 325453 (0.0011) [2023-12-27 07:53:48,681][20347] Updated weights for policy 0, policy_version 325463 (0.0010) [2023-12-27 07:53:48,681][20381] Updated weights for policy 1, policy_version 325183 (0.0011) [2023-12-27 07:53:48,733][20381] Updated weights for policy 1, policy_version 325193 (0.0011) [2023-12-27 07:53:48,786][20381] Updated weights for policy 1, policy_version 325203 (0.0011) [2023-12-27 07:53:49,379][20347] Updated weights for policy 0, policy_version 325473 (0.0010) [2023-12-27 07:53:49,437][20347] Updated weights for policy 0, policy_version 325483 (0.0007) [2023-12-27 07:53:49,481][20381] Updated weights for policy 1, policy_version 325213 (0.0010) [2023-12-27 07:53:49,494][20347] Updated weights for policy 0, policy_version 325493 (0.0006) [2023-12-27 07:53:49,537][20381] Updated weights for policy 1, policy_version 325223 (0.0010) [2023-12-27 07:53:49,547][20347] Updated weights for policy 0, policy_version 325503 (0.0005) [2023-12-27 07:53:49,586][20381] Updated weights for policy 1, policy_version 325233 (0.0011) [2023-12-27 07:53:50,258][20347] Updated weights for policy 0, policy_version 325513 (0.0008) [2023-12-27 07:53:50,290][20381] Updated weights for policy 1, policy_version 325243 (0.0011) [2023-12-27 07:53:50,311][20347] Updated weights for policy 0, policy_version 325523 (0.0006) [2023-12-27 07:53:50,347][20381] Updated weights for policy 1, policy_version 325253 (0.0011) [2023-12-27 07:53:50,371][20347] Updated weights for policy 0, policy_version 325533 (0.0006) [2023-12-27 07:53:50,403][20381] Updated weights for policy 1, policy_version 325263 (0.0011) [2023-12-27 07:53:51,072][20347] Updated weights for policy 0, policy_version 325543 (0.0007) [2023-12-27 07:53:51,084][20381] Updated weights for policy 1, policy_version 325273 (0.0010) [2023-12-27 07:53:51,138][20347] Updated weights for policy 0, policy_version 325553 (0.0007) [2023-12-27 07:53:51,141][20381] Updated weights for policy 1, policy_version 325283 (0.0007) [2023-12-27 07:53:51,191][20381] Updated weights for policy 1, policy_version 325293 (0.0008) [2023-12-27 07:53:51,204][20347] Updated weights for policy 0, policy_version 325563 (0.0006) [2023-12-27 07:53:51,238][20381] Updated weights for policy 1, policy_version 325303 (0.0011) [2023-12-27 07:53:51,866][20347] Updated weights for policy 0, policy_version 325573 (0.0007) [2023-12-27 07:53:51,920][20347] Updated weights for policy 0, policy_version 325583 (0.0006) [2023-12-27 07:53:51,967][20347] Updated weights for policy 0, policy_version 325593 (0.0006) [2023-12-27 07:53:51,990][20381] Updated weights for policy 1, policy_version 325313 (0.0006) [2023-12-27 07:53:52,050][20381] Updated weights for policy 1, policy_version 325323 (0.0007) [2023-12-27 07:53:52,112][20381] Updated weights for policy 1, policy_version 325333 (0.0007) [2023-12-27 07:53:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 166674432. Throughput: 0: 10504.9, 1: 10345.9. Samples: 166643644. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:53:52,363][19170] Avg episode reward: [(0, '944.670'), (1, '990.350')] [2023-12-27 07:53:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000325336_83304448.pth... [2023-12-27 07:53:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000325600_83369984.pth... [2023-12-27 07:53:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000324384_83058688.pth [2023-12-27 07:53:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000324120_82993152.pth [2023-12-27 07:53:52,636][20347] Updated weights for policy 0, policy_version 325603 (0.0009) [2023-12-27 07:53:52,700][20347] Updated weights for policy 0, policy_version 325613 (0.0010) [2023-12-27 07:53:52,704][20381] Updated weights for policy 1, policy_version 325343 (0.0006) [2023-12-27 07:53:52,754][20381] Updated weights for policy 1, policy_version 325353 (0.0005) [2023-12-27 07:53:52,759][20347] Updated weights for policy 0, policy_version 325623 (0.0010) [2023-12-27 07:53:52,802][20381] Updated weights for policy 1, policy_version 325363 (0.0006) [2023-12-27 07:53:53,382][20381] Updated weights for policy 1, policy_version 325373 (0.0005) [2023-12-27 07:53:53,427][20381] Updated weights for policy 1, policy_version 325383 (0.0006) [2023-12-27 07:53:53,428][20347] Updated weights for policy 0, policy_version 325633 (0.0010) [2023-12-27 07:53:53,480][20381] Updated weights for policy 1, policy_version 325393 (0.0007) [2023-12-27 07:53:53,491][20347] Updated weights for policy 0, policy_version 325643 (0.0007) [2023-12-27 07:53:53,544][20347] Updated weights for policy 0, policy_version 325653 (0.0007) [2023-12-27 07:53:53,592][20347] Updated weights for policy 0, policy_version 325663 (0.0007) [2023-12-27 07:53:54,167][20381] Updated weights for policy 1, policy_version 325403 (0.0008) [2023-12-27 07:53:54,223][20381] Updated weights for policy 1, policy_version 325413 (0.0007) [2023-12-27 07:53:54,260][20347] Updated weights for policy 0, policy_version 325673 (0.0006) [2023-12-27 07:53:54,282][20381] Updated weights for policy 1, policy_version 325423 (0.0008) [2023-12-27 07:53:54,306][20347] Updated weights for policy 0, policy_version 325683 (0.0005) [2023-12-27 07:53:54,354][20347] Updated weights for policy 0, policy_version 325693 (0.0005) [2023-12-27 07:53:54,924][20347] Updated weights for policy 0, policy_version 325703 (0.0009) [2023-12-27 07:53:54,966][20381] Updated weights for policy 1, policy_version 325433 (0.0008) [2023-12-27 07:53:54,983][20347] Updated weights for policy 0, policy_version 325713 (0.0010) [2023-12-27 07:53:55,028][20381] Updated weights for policy 1, policy_version 325443 (0.0008) [2023-12-27 07:53:55,047][20347] Updated weights for policy 0, policy_version 325723 (0.0008) [2023-12-27 07:53:55,078][20381] Updated weights for policy 1, policy_version 325453 (0.0007) [2023-12-27 07:53:55,131][20381] Updated weights for policy 1, policy_version 325463 (0.0008) [2023-12-27 07:53:55,723][20347] Updated weights for policy 0, policy_version 325733 (0.0007) [2023-12-27 07:53:55,777][20347] Updated weights for policy 0, policy_version 325743 (0.0006) [2023-12-27 07:53:55,827][20347] Updated weights for policy 0, policy_version 325753 (0.0007) [2023-12-27 07:53:55,833][20381] Updated weights for policy 1, policy_version 325473 (0.0007) [2023-12-27 07:53:55,893][20381] Updated weights for policy 1, policy_version 325483 (0.0007) [2023-12-27 07:53:55,949][20381] Updated weights for policy 1, policy_version 325493 (0.0008) [2023-12-27 07:53:56,528][20347] Updated weights for policy 0, policy_version 325763 (0.0006) [2023-12-27 07:53:56,571][20381] Updated weights for policy 1, policy_version 325503 (0.0006) [2023-12-27 07:53:56,578][20347] Updated weights for policy 0, policy_version 325773 (0.0006) [2023-12-27 07:53:56,627][20347] Updated weights for policy 0, policy_version 325783 (0.0005) [2023-12-27 07:53:56,629][20381] Updated weights for policy 1, policy_version 325513 (0.0006) [2023-12-27 07:53:56,681][20381] Updated weights for policy 1, policy_version 325523 (0.0006) [2023-12-27 07:53:57,293][20347] Updated weights for policy 0, policy_version 325793 (0.0006) [2023-12-27 07:53:57,302][20381] Updated weights for policy 1, policy_version 325533 (0.0007) [2023-12-27 07:53:57,351][20347] Updated weights for policy 0, policy_version 325803 (0.0006) [2023-12-27 07:53:57,352][20381] Updated weights for policy 1, policy_version 325543 (0.0007) [2023-12-27 07:53:57,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 166772736. Throughput: 0: 10437.8, 1: 10445.9. Samples: 166768496. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:53:57,363][19170] Avg episode reward: [(0, '981.550'), (1, '1000.000')] [2023-12-27 07:53:57,398][20347] Updated weights for policy 0, policy_version 325813 (0.0006) [2023-12-27 07:53:57,399][20381] Updated weights for policy 1, policy_version 325553 (0.0007) [2023-12-27 07:53:57,442][20347] Updated weights for policy 0, policy_version 325823 (0.0005) [2023-12-27 07:53:58,060][20347] Updated weights for policy 0, policy_version 325833 (0.0006) [2023-12-27 07:53:58,082][20381] Updated weights for policy 1, policy_version 325563 (0.0007) [2023-12-27 07:53:58,120][20347] Updated weights for policy 0, policy_version 325843 (0.0007) [2023-12-27 07:53:58,144][20381] Updated weights for policy 1, policy_version 325573 (0.0005) [2023-12-27 07:53:58,182][20347] Updated weights for policy 0, policy_version 325853 (0.0007) [2023-12-27 07:53:58,208][20381] Updated weights for policy 1, policy_version 325583 (0.0007) [2023-12-27 07:53:58,907][20347] Updated weights for policy 0, policy_version 325863 (0.0008) [2023-12-27 07:53:58,913][20381] Updated weights for policy 1, policy_version 325593 (0.0007) [2023-12-27 07:53:58,966][20347] Updated weights for policy 0, policy_version 325873 (0.0007) [2023-12-27 07:53:58,972][20381] Updated weights for policy 1, policy_version 325603 (0.0005) [2023-12-27 07:53:59,021][20347] Updated weights for policy 0, policy_version 325883 (0.0007) [2023-12-27 07:53:59,033][20381] Updated weights for policy 1, policy_version 325613 (0.0009) [2023-12-27 07:53:59,091][20381] Updated weights for policy 1, policy_version 325623 (0.0008) [2023-12-27 07:53:59,701][20347] Updated weights for policy 0, policy_version 325893 (0.0009) [2023-12-27 07:53:59,763][20347] Updated weights for policy 0, policy_version 325903 (0.0009) [2023-12-27 07:53:59,818][20381] Updated weights for policy 1, policy_version 325633 (0.0006) [2023-12-27 07:53:59,825][20347] Updated weights for policy 0, policy_version 325913 (0.0010) [2023-12-27 07:53:59,880][20381] Updated weights for policy 1, policy_version 325643 (0.0007) [2023-12-27 07:53:59,931][20381] Updated weights for policy 1, policy_version 325653 (0.0007) [2023-12-27 07:54:00,522][20347] Updated weights for policy 0, policy_version 325923 (0.0008) [2023-12-27 07:54:00,572][20347] Updated weights for policy 0, policy_version 325933 (0.0007) [2023-12-27 07:54:00,596][20381] Updated weights for policy 1, policy_version 325663 (0.0006) [2023-12-27 07:54:00,620][20347] Updated weights for policy 0, policy_version 325943 (0.0006) [2023-12-27 07:54:00,663][20381] Updated weights for policy 1, policy_version 325673 (0.0006) [2023-12-27 07:54:00,714][20381] Updated weights for policy 1, policy_version 325683 (0.0007) [2023-12-27 07:54:01,255][20347] Updated weights for policy 0, policy_version 325953 (0.0006) [2023-12-27 07:54:01,324][20347] Updated weights for policy 0, policy_version 325963 (0.0008) [2023-12-27 07:54:01,389][20347] Updated weights for policy 0, policy_version 325973 (0.0009) [2023-12-27 07:54:01,433][20381] Updated weights for policy 1, policy_version 325693 (0.0008) [2023-12-27 07:54:01,447][20347] Updated weights for policy 0, policy_version 325983 (0.0009) [2023-12-27 07:54:01,492][20381] Updated weights for policy 1, policy_version 325703 (0.0007) [2023-12-27 07:54:01,551][20381] Updated weights for policy 1, policy_version 325713 (0.0007) [2023-12-27 07:54:02,136][20347] Updated weights for policy 0, policy_version 325993 (0.0010) [2023-12-27 07:54:02,194][20347] Updated weights for policy 0, policy_version 326003 (0.0010) [2023-12-27 07:54:02,250][20347] Updated weights for policy 0, policy_version 326013 (0.0010) [2023-12-27 07:54:02,252][20381] Updated weights for policy 1, policy_version 325723 (0.0007) [2023-12-27 07:54:02,300][20381] Updated weights for policy 1, policy_version 325733 (0.0007) [2023-12-27 07:54:02,355][20381] Updated weights for policy 1, policy_version 325743 (0.0007) [2023-12-27 07:54:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 166879232. Throughput: 0: 10368.9, 1: 10417.9. Samples: 166891016. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:02,363][19170] Avg episode reward: [(0, '990.950'), (1, '951.900')] [2023-12-27 07:54:02,946][20347] Updated weights for policy 0, policy_version 326023 (0.0009) [2023-12-27 07:54:03,001][20347] Updated weights for policy 0, policy_version 326033 (0.0008) [2023-12-27 07:54:03,062][20347] Updated weights for policy 0, policy_version 326043 (0.0006) [2023-12-27 07:54:03,079][20381] Updated weights for policy 1, policy_version 325753 (0.0007) [2023-12-27 07:54:03,138][20381] Updated weights for policy 1, policy_version 325763 (0.0007) [2023-12-27 07:54:03,182][20381] Updated weights for policy 1, policy_version 325773 (0.0007) [2023-12-27 07:54:03,229][20381] Updated weights for policy 1, policy_version 325783 (0.0005) [2023-12-27 07:54:03,621][20347] Updated weights for policy 0, policy_version 326053 (0.0005) [2023-12-27 07:54:03,684][20347] Updated weights for policy 0, policy_version 326063 (0.0005) [2023-12-27 07:54:03,747][20347] Updated weights for policy 0, policy_version 326073 (0.0006) [2023-12-27 07:54:03,777][20381] Updated weights for policy 1, policy_version 325793 (0.0005) [2023-12-27 07:54:03,822][20381] Updated weights for policy 1, policy_version 325803 (0.0005) [2023-12-27 07:54:03,868][20381] Updated weights for policy 1, policy_version 325813 (0.0006) [2023-12-27 07:54:04,401][20347] Updated weights for policy 0, policy_version 326083 (0.0009) [2023-12-27 07:54:04,455][20347] Updated weights for policy 0, policy_version 326093 (0.0007) [2023-12-27 07:54:04,470][20381] Updated weights for policy 1, policy_version 325823 (0.0007) [2023-12-27 07:54:04,504][20347] Updated weights for policy 0, policy_version 326103 (0.0007) [2023-12-27 07:54:04,522][20381] Updated weights for policy 1, policy_version 325833 (0.0006) [2023-12-27 07:54:04,571][20381] Updated weights for policy 1, policy_version 325843 (0.0005) [2023-12-27 07:54:05,209][20347] Updated weights for policy 0, policy_version 326113 (0.0008) [2023-12-27 07:54:05,228][20381] Updated weights for policy 1, policy_version 325853 (0.0009) [2023-12-27 07:54:05,267][20347] Updated weights for policy 0, policy_version 326123 (0.0010) [2023-12-27 07:54:05,286][20381] Updated weights for policy 1, policy_version 325863 (0.0010) [2023-12-27 07:54:05,326][20347] Updated weights for policy 0, policy_version 326133 (0.0010) [2023-12-27 07:54:05,344][20381] Updated weights for policy 1, policy_version 325873 (0.0010) [2023-12-27 07:54:05,381][20347] Updated weights for policy 0, policy_version 326143 (0.0008) [2023-12-27 07:54:05,902][20381] Updated weights for policy 1, policy_version 325883 (0.0009) [2023-12-27 07:54:05,939][20347] Updated weights for policy 0, policy_version 326153 (0.0008) [2023-12-27 07:54:05,966][20381] Updated weights for policy 1, policy_version 325893 (0.0008) [2023-12-27 07:54:05,989][20347] Updated weights for policy 0, policy_version 326163 (0.0009) [2023-12-27 07:54:06,021][20381] Updated weights for policy 1, policy_version 325903 (0.0011) [2023-12-27 07:54:06,045][20347] Updated weights for policy 0, policy_version 326173 (0.0007) [2023-12-27 07:54:06,681][20381] Updated weights for policy 1, policy_version 325913 (0.0010) [2023-12-27 07:54:06,719][20347] Updated weights for policy 0, policy_version 326183 (0.0010) [2023-12-27 07:54:06,741][20381] Updated weights for policy 1, policy_version 325923 (0.0006) [2023-12-27 07:54:06,782][20347] Updated weights for policy 0, policy_version 326193 (0.0010) [2023-12-27 07:54:06,798][20381] Updated weights for policy 1, policy_version 325933 (0.0007) [2023-12-27 07:54:06,836][20347] Updated weights for policy 0, policy_version 326203 (0.0007) [2023-12-27 07:54:06,853][20381] Updated weights for policy 1, policy_version 325943 (0.0010) [2023-12-27 07:54:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 166985728. Throughput: 0: 10377.7, 1: 10452.2. Samples: 166955876. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:07,363][19170] Avg episode reward: [(0, '981.510'), (1, '961.380')] [2023-12-27 07:54:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000325944_83460096.pth... [2023-12-27 07:54:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000326208_83525632.pth... [2023-12-27 07:54:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000324728_83148800.pth [2023-12-27 07:54:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000324992_83214336.pth [2023-12-27 07:54:07,451][20381] Updated weights for policy 1, policy_version 325953 (0.0010) [2023-12-27 07:54:07,457][20347] Updated weights for policy 0, policy_version 326213 (0.0008) [2023-12-27 07:54:07,505][20347] Updated weights for policy 0, policy_version 326223 (0.0010) [2023-12-27 07:54:07,510][20381] Updated weights for policy 1, policy_version 325963 (0.0009) [2023-12-27 07:54:07,556][20347] Updated weights for policy 0, policy_version 326233 (0.0010) [2023-12-27 07:54:07,568][20381] Updated weights for policy 1, policy_version 325973 (0.0008) [2023-12-27 07:54:08,167][20381] Updated weights for policy 1, policy_version 325983 (0.0007) [2023-12-27 07:54:08,229][20381] Updated weights for policy 1, policy_version 325993 (0.0005) [2023-12-27 07:54:08,245][20347] Updated weights for policy 0, policy_version 326243 (0.0010) [2023-12-27 07:54:08,278][20381] Updated weights for policy 1, policy_version 326003 (0.0007) [2023-12-27 07:54:08,306][20347] Updated weights for policy 0, policy_version 326253 (0.0007) [2023-12-27 07:54:08,368][20347] Updated weights for policy 0, policy_version 326263 (0.0007) [2023-12-27 07:54:08,964][20381] Updated weights for policy 1, policy_version 326013 (0.0008) [2023-12-27 07:54:08,989][20347] Updated weights for policy 0, policy_version 326273 (0.0006) [2023-12-27 07:54:09,018][20381] Updated weights for policy 1, policy_version 326023 (0.0006) [2023-12-27 07:54:09,040][20347] Updated weights for policy 0, policy_version 326283 (0.0005) [2023-12-27 07:54:09,073][20381] Updated weights for policy 1, policy_version 326033 (0.0006) [2023-12-27 07:54:09,087][20347] Updated weights for policy 0, policy_version 326293 (0.0005) [2023-12-27 07:54:09,146][20347] Updated weights for policy 0, policy_version 326303 (0.0008) [2023-12-27 07:54:09,808][20381] Updated weights for policy 1, policy_version 326043 (0.0007) [2023-12-27 07:54:09,856][20347] Updated weights for policy 0, policy_version 326313 (0.0008) [2023-12-27 07:54:09,869][20381] Updated weights for policy 1, policy_version 326053 (0.0007) [2023-12-27 07:54:09,922][20347] Updated weights for policy 0, policy_version 326323 (0.0008) [2023-12-27 07:54:09,929][20381] Updated weights for policy 1, policy_version 326063 (0.0007) [2023-12-27 07:54:09,987][20347] Updated weights for policy 0, policy_version 326333 (0.0008) [2023-12-27 07:54:10,596][20347] Updated weights for policy 0, policy_version 326343 (0.0007) [2023-12-27 07:54:10,652][20347] Updated weights for policy 0, policy_version 326353 (0.0007) [2023-12-27 07:54:10,685][20381] Updated weights for policy 1, policy_version 326073 (0.0007) [2023-12-27 07:54:10,710][20347] Updated weights for policy 0, policy_version 326363 (0.0006) [2023-12-27 07:54:10,739][20381] Updated weights for policy 1, policy_version 326083 (0.0008) [2023-12-27 07:54:10,795][20381] Updated weights for policy 1, policy_version 326093 (0.0008) [2023-12-27 07:54:10,850][20381] Updated weights for policy 1, policy_version 326103 (0.0007) [2023-12-27 07:54:11,354][20347] Updated weights for policy 0, policy_version 326373 (0.0007) [2023-12-27 07:54:11,421][20347] Updated weights for policy 0, policy_version 326383 (0.0008) [2023-12-27 07:54:11,485][20347] Updated weights for policy 0, policy_version 326393 (0.0008) [2023-12-27 07:54:11,603][20381] Updated weights for policy 1, policy_version 326113 (0.0008) [2023-12-27 07:54:11,665][20381] Updated weights for policy 1, policy_version 326123 (0.0008) [2023-12-27 07:54:11,718][20381] Updated weights for policy 1, policy_version 326133 (0.0009) [2023-12-27 07:54:12,285][20347] Updated weights for policy 0, policy_version 326403 (0.0008) [2023-12-27 07:54:12,340][20347] Updated weights for policy 0, policy_version 326413 (0.0008) [2023-12-27 07:54:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 167084032. Throughput: 0: 10458.3, 1: 10461.2. Samples: 167082528. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:12,363][19170] Avg episode reward: [(0, '990.560'), (1, '1000.000')] [2023-12-27 07:54:12,403][20347] Updated weights for policy 0, policy_version 326423 (0.0008) [2023-12-27 07:54:12,436][20381] Updated weights for policy 1, policy_version 326143 (0.0006) [2023-12-27 07:54:12,490][20381] Updated weights for policy 1, policy_version 326153 (0.0007) [2023-12-27 07:54:12,539][20381] Updated weights for policy 1, policy_version 326163 (0.0008) [2023-12-27 07:54:12,978][20347] Updated weights for policy 0, policy_version 326433 (0.0011) [2023-12-27 07:54:13,040][20347] Updated weights for policy 0, policy_version 326443 (0.0007) [2023-12-27 07:54:13,103][20347] Updated weights for policy 0, policy_version 326453 (0.0009) [2023-12-27 07:54:13,165][20347] Updated weights for policy 0, policy_version 326463 (0.0010) [2023-12-27 07:54:13,273][20381] Updated weights for policy 1, policy_version 326173 (0.0009) [2023-12-27 07:54:13,332][20381] Updated weights for policy 1, policy_version 326183 (0.0008) [2023-12-27 07:54:13,391][20381] Updated weights for policy 1, policy_version 326193 (0.0008) [2023-12-27 07:54:13,738][20347] Updated weights for policy 0, policy_version 326473 (0.0010) [2023-12-27 07:54:13,802][20347] Updated weights for policy 0, policy_version 326483 (0.0010) [2023-12-27 07:54:13,865][20347] Updated weights for policy 0, policy_version 326493 (0.0010) [2023-12-27 07:54:14,051][20381] Updated weights for policy 1, policy_version 326203 (0.0008) [2023-12-27 07:54:14,106][20381] Updated weights for policy 1, policy_version 326213 (0.0008) [2023-12-27 07:54:14,176][20381] Updated weights for policy 1, policy_version 326223 (0.0009) [2023-12-27 07:54:14,482][20347] Updated weights for policy 0, policy_version 326503 (0.0007) [2023-12-27 07:54:14,540][20347] Updated weights for policy 0, policy_version 326513 (0.0007) [2023-12-27 07:54:14,592][20347] Updated weights for policy 0, policy_version 326523 (0.0010) [2023-12-27 07:54:14,807][20381] Updated weights for policy 1, policy_version 326233 (0.0007) [2023-12-27 07:54:14,870][20381] Updated weights for policy 1, policy_version 326243 (0.0006) [2023-12-27 07:54:14,933][20381] Updated weights for policy 1, policy_version 326253 (0.0006) [2023-12-27 07:54:14,996][20381] Updated weights for policy 1, policy_version 326263 (0.0005) [2023-12-27 07:54:15,241][20347] Updated weights for policy 0, policy_version 326533 (0.0010) [2023-12-27 07:54:15,296][20347] Updated weights for policy 0, policy_version 326543 (0.0010) [2023-12-27 07:54:15,355][20347] Updated weights for policy 0, policy_version 326553 (0.0010) [2023-12-27 07:54:15,636][20381] Updated weights for policy 1, policy_version 326273 (0.0008) [2023-12-27 07:54:15,687][20381] Updated weights for policy 1, policy_version 326283 (0.0007) [2023-12-27 07:54:15,742][20381] Updated weights for policy 1, policy_version 326293 (0.0008) [2023-12-27 07:54:16,025][20347] Updated weights for policy 0, policy_version 326563 (0.0010) [2023-12-27 07:54:16,083][20347] Updated weights for policy 0, policy_version 326573 (0.0010) [2023-12-27 07:54:16,144][20347] Updated weights for policy 0, policy_version 326583 (0.0010) [2023-12-27 07:54:16,361][20381] Updated weights for policy 1, policy_version 326303 (0.0007) [2023-12-27 07:54:16,415][20381] Updated weights for policy 1, policy_version 326313 (0.0008) [2023-12-27 07:54:16,472][20381] Updated weights for policy 1, policy_version 326323 (0.0009) [2023-12-27 07:54:16,747][20347] Updated weights for policy 0, policy_version 326593 (0.0010) [2023-12-27 07:54:16,797][20347] Updated weights for policy 0, policy_version 326603 (0.0006) [2023-12-27 07:54:16,847][20347] Updated weights for policy 0, policy_version 326613 (0.0008) [2023-12-27 07:54:16,893][20347] Updated weights for policy 0, policy_version 326623 (0.0008) [2023-12-27 07:54:17,145][20381] Updated weights for policy 1, policy_version 326333 (0.0007) [2023-12-27 07:54:17,193][20381] Updated weights for policy 1, policy_version 326343 (0.0005) [2023-12-27 07:54:17,247][20381] Updated weights for policy 1, policy_version 326353 (0.0009) [2023-12-27 07:54:17,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 167198720. Throughput: 0: 10482.7, 1: 10502.4. Samples: 167207384. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:17,364][19170] Avg episode reward: [(0, '990.340'), (1, '990.330')] [2023-12-27 07:54:17,531][20347] Updated weights for policy 0, policy_version 326633 (0.0006) [2023-12-27 07:54:17,587][20347] Updated weights for policy 0, policy_version 326643 (0.0007) [2023-12-27 07:54:17,639][20347] Updated weights for policy 0, policy_version 326653 (0.0009) [2023-12-27 07:54:17,896][20381] Updated weights for policy 1, policy_version 326363 (0.0010) [2023-12-27 07:54:17,946][20381] Updated weights for policy 1, policy_version 326373 (0.0009) [2023-12-27 07:54:18,000][20381] Updated weights for policy 1, policy_version 326383 (0.0009) [2023-12-27 07:54:18,247][20347] Updated weights for policy 0, policy_version 326663 (0.0008) [2023-12-27 07:54:18,303][20347] Updated weights for policy 0, policy_version 326673 (0.0006) [2023-12-27 07:54:18,373][20347] Updated weights for policy 0, policy_version 326683 (0.0008) [2023-12-27 07:54:18,618][20381] Updated weights for policy 1, policy_version 326393 (0.0008) [2023-12-27 07:54:18,669][20381] Updated weights for policy 1, policy_version 326403 (0.0010) [2023-12-27 07:54:18,715][20381] Updated weights for policy 1, policy_version 326413 (0.0009) [2023-12-27 07:54:18,773][20381] Updated weights for policy 1, policy_version 326423 (0.0006) [2023-12-27 07:54:18,997][20347] Updated weights for policy 0, policy_version 326693 (0.0008) [2023-12-27 07:54:19,044][20347] Updated weights for policy 0, policy_version 326703 (0.0008) [2023-12-27 07:54:19,096][20347] Updated weights for policy 0, policy_version 326713 (0.0009) [2023-12-27 07:54:19,393][20381] Updated weights for policy 1, policy_version 326433 (0.0007) [2023-12-27 07:54:19,457][20381] Updated weights for policy 1, policy_version 326443 (0.0007) [2023-12-27 07:54:19,518][20381] Updated weights for policy 1, policy_version 326453 (0.0007) [2023-12-27 07:54:19,790][20347] Updated weights for policy 0, policy_version 326723 (0.0007) [2023-12-27 07:54:19,846][20347] Updated weights for policy 0, policy_version 326733 (0.0007) [2023-12-27 07:54:19,909][20347] Updated weights for policy 0, policy_version 326743 (0.0006) [2023-12-27 07:54:20,153][20381] Updated weights for policy 1, policy_version 326463 (0.0009) [2023-12-27 07:54:20,210][20381] Updated weights for policy 1, policy_version 326473 (0.0010) [2023-12-27 07:54:20,274][20381] Updated weights for policy 1, policy_version 326483 (0.0010) [2023-12-27 07:54:20,625][20347] Updated weights for policy 0, policy_version 326753 (0.0008) [2023-12-27 07:54:20,684][20347] Updated weights for policy 0, policy_version 326763 (0.0008) [2023-12-27 07:54:20,748][20347] Updated weights for policy 0, policy_version 326773 (0.0007) [2023-12-27 07:54:20,807][20347] Updated weights for policy 0, policy_version 326783 (0.0007) [2023-12-27 07:54:20,888][20381] Updated weights for policy 1, policy_version 326493 (0.0011) [2023-12-27 07:54:20,947][20381] Updated weights for policy 1, policy_version 326503 (0.0011) [2023-12-27 07:54:21,006][20381] Updated weights for policy 1, policy_version 326513 (0.0011) [2023-12-27 07:54:21,467][20347] Updated weights for policy 0, policy_version 326793 (0.0007) [2023-12-27 07:54:21,529][20347] Updated weights for policy 0, policy_version 326803 (0.0008) [2023-12-27 07:54:21,580][20347] Updated weights for policy 0, policy_version 326813 (0.0007) [2023-12-27 07:54:21,716][20381] Updated weights for policy 1, policy_version 326523 (0.0010) [2023-12-27 07:54:21,784][20381] Updated weights for policy 1, policy_version 326533 (0.0008) [2023-12-27 07:54:21,846][20381] Updated weights for policy 1, policy_version 326543 (0.0010) [2023-12-27 07:54:22,342][20347] Updated weights for policy 0, policy_version 326823 (0.0007) [2023-12-27 07:54:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 167297024. Throughput: 0: 10508.1, 1: 10446.2. Samples: 167272048. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.330')] [2023-12-27 07:54:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000326552_83615744.pth... [2023-12-27 07:54:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000325336_83304448.pth [2023-12-27 07:54:22,405][20347] Updated weights for policy 0, policy_version 326833 (0.0008) [2023-12-27 07:54:22,463][20347] Updated weights for policy 0, policy_version 326843 (0.0008) [2023-12-27 07:54:22,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000326848_83689472.pth... [2023-12-27 07:54:22,494][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000325600_83369984.pth [2023-12-27 07:54:22,554][20381] Updated weights for policy 1, policy_version 326553 (0.0011) [2023-12-27 07:54:22,604][20381] Updated weights for policy 1, policy_version 326563 (0.0008) [2023-12-27 07:54:22,655][20381] Updated weights for policy 1, policy_version 326573 (0.0008) [2023-12-27 07:54:22,706][20381] Updated weights for policy 1, policy_version 326583 (0.0007) [2023-12-27 07:54:23,204][20347] Updated weights for policy 0, policy_version 326853 (0.0007) [2023-12-27 07:54:23,257][20347] Updated weights for policy 0, policy_version 326863 (0.0007) [2023-12-27 07:54:23,311][20347] Updated weights for policy 0, policy_version 326873 (0.0007) [2023-12-27 07:54:23,370][20381] Updated weights for policy 1, policy_version 326593 (0.0008) [2023-12-27 07:54:23,418][20381] Updated weights for policy 1, policy_version 326603 (0.0008) [2023-12-27 07:54:23,463][20381] Updated weights for policy 1, policy_version 326613 (0.0007) [2023-12-27 07:54:24,022][20347] Updated weights for policy 0, policy_version 326883 (0.0008) [2023-12-27 07:54:24,076][20347] Updated weights for policy 0, policy_version 326893 (0.0006) [2023-12-27 07:54:24,105][20381] Updated weights for policy 1, policy_version 326623 (0.0009) [2023-12-27 07:54:24,129][20347] Updated weights for policy 0, policy_version 326903 (0.0006) [2023-12-27 07:54:24,157][20381] Updated weights for policy 1, policy_version 326633 (0.0010) [2023-12-27 07:54:24,211][20381] Updated weights for policy 1, policy_version 326643 (0.0010) [2023-12-27 07:54:24,789][20347] Updated weights for policy 0, policy_version 326913 (0.0010) [2023-12-27 07:54:24,847][20347] Updated weights for policy 0, policy_version 326923 (0.0010) [2023-12-27 07:54:24,869][20381] Updated weights for policy 1, policy_version 326653 (0.0010) [2023-12-27 07:54:24,893][20347] Updated weights for policy 0, policy_version 326933 (0.0010) [2023-12-27 07:54:24,914][20381] Updated weights for policy 1, policy_version 326663 (0.0010) [2023-12-27 07:54:24,938][20347] Updated weights for policy 0, policy_version 326943 (0.0010) [2023-12-27 07:54:24,962][20381] Updated weights for policy 1, policy_version 326673 (0.0010) [2023-12-27 07:54:25,561][20347] Updated weights for policy 0, policy_version 326953 (0.0010) [2023-12-27 07:54:25,608][20381] Updated weights for policy 1, policy_version 326683 (0.0010) [2023-12-27 07:54:25,629][20347] Updated weights for policy 0, policy_version 326963 (0.0011) [2023-12-27 07:54:25,663][20381] Updated weights for policy 1, policy_version 326693 (0.0011) [2023-12-27 07:54:25,691][20347] Updated weights for policy 0, policy_version 326973 (0.0008) [2023-12-27 07:54:25,725][20381] Updated weights for policy 1, policy_version 326703 (0.0011) [2023-12-27 07:54:26,315][20347] Updated weights for policy 0, policy_version 326983 (0.0008) [2023-12-27 07:54:26,363][20347] Updated weights for policy 0, policy_version 326993 (0.0010) [2023-12-27 07:54:26,404][20347] Updated weights for policy 0, policy_version 327003 (0.0007) [2023-12-27 07:54:26,410][20381] Updated weights for policy 1, policy_version 326713 (0.0010) [2023-12-27 07:54:26,467][20381] Updated weights for policy 1, policy_version 326723 (0.0010) [2023-12-27 07:54:26,507][20381] Updated weights for policy 1, policy_version 326733 (0.0010) [2023-12-27 07:54:26,561][20381] Updated weights for policy 1, policy_version 326743 (0.0010) [2023-12-27 07:54:27,070][20347] Updated weights for policy 0, policy_version 327013 (0.0006) [2023-12-27 07:54:27,116][20347] Updated weights for policy 0, policy_version 327023 (0.0005) [2023-12-27 07:54:27,162][20347] Updated weights for policy 0, policy_version 327033 (0.0005) [2023-12-27 07:54:27,228][20381] Updated weights for policy 1, policy_version 326753 (0.0010) [2023-12-27 07:54:27,300][20381] Updated weights for policy 1, policy_version 326763 (0.0010) [2023-12-27 07:54:27,355][20381] Updated weights for policy 1, policy_version 326773 (0.0010) [2023-12-27 07:54:27,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 167403520. Throughput: 0: 10452.2, 1: 10477.0. Samples: 167394804. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 07:54:27,793][20347] Updated weights for policy 0, policy_version 327043 (0.0006) [2023-12-27 07:54:27,836][20347] Updated weights for policy 0, policy_version 327053 (0.0007) [2023-12-27 07:54:27,882][20347] Updated weights for policy 0, policy_version 327063 (0.0007) [2023-12-27 07:54:28,013][20381] Updated weights for policy 1, policy_version 326783 (0.0010) [2023-12-27 07:54:28,061][20381] Updated weights for policy 1, policy_version 326793 (0.0010) [2023-12-27 07:54:28,121][20381] Updated weights for policy 1, policy_version 326803 (0.0010) [2023-12-27 07:54:28,584][20347] Updated weights for policy 0, policy_version 327073 (0.0009) [2023-12-27 07:54:28,639][20347] Updated weights for policy 0, policy_version 327083 (0.0010) [2023-12-27 07:54:28,687][20347] Updated weights for policy 0, policy_version 327093 (0.0010) [2023-12-27 07:54:28,735][20347] Updated weights for policy 0, policy_version 327103 (0.0010) [2023-12-27 07:54:28,750][20381] Updated weights for policy 1, policy_version 326813 (0.0006) [2023-12-27 07:54:28,806][20381] Updated weights for policy 1, policy_version 326823 (0.0010) [2023-12-27 07:54:28,865][20381] Updated weights for policy 1, policy_version 326833 (0.0011) [2023-12-27 07:54:29,454][20347] Updated weights for policy 0, policy_version 327113 (0.0007) [2023-12-27 07:54:29,460][20291] KL-divergence is very high: 102.7758 [2023-12-27 07:54:29,470][20381] Updated weights for policy 1, policy_version 326843 (0.0007) [2023-12-27 07:54:29,510][20347] Updated weights for policy 0, policy_version 327123 (0.0007) [2023-12-27 07:54:29,523][20381] Updated weights for policy 1, policy_version 326853 (0.0011) [2023-12-27 07:54:29,564][20347] Updated weights for policy 0, policy_version 327133 (0.0007) [2023-12-27 07:54:29,576][20381] Updated weights for policy 1, policy_version 326863 (0.0011) [2023-12-27 07:54:30,183][20347] Updated weights for policy 0, policy_version 327143 (0.0005) [2023-12-27 07:54:30,235][20347] Updated weights for policy 0, policy_version 327153 (0.0007) [2023-12-27 07:54:30,272][20381] Updated weights for policy 1, policy_version 326873 (0.0011) [2023-12-27 07:54:30,286][20347] Updated weights for policy 0, policy_version 327163 (0.0007) [2023-12-27 07:54:30,333][20381] Updated weights for policy 1, policy_version 326883 (0.0011) [2023-12-27 07:54:30,390][20381] Updated weights for policy 1, policy_version 326893 (0.0011) [2023-12-27 07:54:30,443][20381] Updated weights for policy 1, policy_version 326903 (0.0011) [2023-12-27 07:54:30,979][20347] Updated weights for policy 0, policy_version 327173 (0.0007) [2023-12-27 07:54:31,034][20347] Updated weights for policy 0, policy_version 327183 (0.0007) [2023-12-27 07:54:31,101][20347] Updated weights for policy 0, policy_version 327193 (0.0006) [2023-12-27 07:54:31,126][20381] Updated weights for policy 1, policy_version 326913 (0.0007) [2023-12-27 07:54:31,191][20381] Updated weights for policy 1, policy_version 326923 (0.0009) [2023-12-27 07:54:31,253][20381] Updated weights for policy 1, policy_version 326933 (0.0009) [2023-12-27 07:54:31,705][20347] Updated weights for policy 0, policy_version 327203 (0.0007) [2023-12-27 07:54:31,771][20347] Updated weights for policy 0, policy_version 327213 (0.0008) [2023-12-27 07:54:31,832][20347] Updated weights for policy 0, policy_version 327223 (0.0008) [2023-12-27 07:54:31,946][20381] Updated weights for policy 1, policy_version 326943 (0.0008) [2023-12-27 07:54:32,008][20381] Updated weights for policy 1, policy_version 326953 (0.0008) [2023-12-27 07:54:32,071][20381] Updated weights for policy 1, policy_version 326963 (0.0008) [2023-12-27 07:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 167510016. Throughput: 0: 10478.9, 1: 10395.4. Samples: 167520620. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:32,363][19170] Avg episode reward: [(0, '933.970'), (1, '1000.000')] [2023-12-27 07:54:32,542][20347] Updated weights for policy 0, policy_version 327233 (0.0009) [2023-12-27 07:54:32,604][20347] Updated weights for policy 0, policy_version 327243 (0.0009) [2023-12-27 07:54:32,634][20381] Updated weights for policy 1, policy_version 326973 (0.0006) [2023-12-27 07:54:32,661][20347] Updated weights for policy 0, policy_version 327253 (0.0008) [2023-12-27 07:54:32,666][20291] KL-divergence is very high: 122.8877 [2023-12-27 07:54:32,671][20291] KL-divergence is very high: 127.2369 [2023-12-27 07:54:32,693][20381] Updated weights for policy 1, policy_version 326983 (0.0008) [2023-12-27 07:54:32,719][20347] Updated weights for policy 0, policy_version 327263 (0.0007) [2023-12-27 07:54:32,752][20381] Updated weights for policy 1, policy_version 326993 (0.0008) [2023-12-27 07:54:33,384][20291] KL-divergence is very high: 145.7109 [2023-12-27 07:54:33,414][20347] Updated weights for policy 0, policy_version 327273 (0.0007) [2023-12-27 07:54:33,431][20381] Updated weights for policy 1, policy_version 327003 (0.0008) [2023-12-27 07:54:33,434][20291] KL-divergence is very high: 241.1913 [2023-12-27 07:54:33,467][20347] Updated weights for policy 0, policy_version 327283 (0.0007) [2023-12-27 07:54:33,474][20291] KL-divergence is very high: 102.6532 [2023-12-27 07:54:33,489][20381] Updated weights for policy 1, policy_version 327013 (0.0006) [2023-12-27 07:54:33,506][20291] KL-divergence is very high: 130.8158 [2023-12-27 07:54:33,517][20291] KL-divergence is very high: 455.3491 [2023-12-27 07:54:33,523][20347] Updated weights for policy 0, policy_version 327293 (0.0006) [2023-12-27 07:54:33,543][20381] Updated weights for policy 1, policy_version 327023 (0.0005) [2023-12-27 07:54:34,103][20381] Updated weights for policy 1, policy_version 327033 (0.0005) [2023-12-27 07:54:34,163][20381] Updated weights for policy 1, policy_version 327043 (0.0008) [2023-12-27 07:54:34,178][20347] Updated weights for policy 0, policy_version 327303 (0.0007) [2023-12-27 07:54:34,222][20381] Updated weights for policy 1, policy_version 327053 (0.0011) [2023-12-27 07:54:34,243][20347] Updated weights for policy 0, policy_version 327313 (0.0006) [2023-12-27 07:54:34,277][20381] Updated weights for policy 1, policy_version 327063 (0.0011) [2023-12-27 07:54:34,307][20347] Updated weights for policy 0, policy_version 327323 (0.0008) [2023-12-27 07:54:34,906][20381] Updated weights for policy 1, policy_version 327073 (0.0006) [2023-12-27 07:54:34,912][20291] KL-divergence is very high: 121.5942 [2023-12-27 07:54:34,929][20347] Updated weights for policy 0, policy_version 327333 (0.0007) [2023-12-27 07:54:34,958][20291] KL-divergence is very high: 177.7242 [2023-12-27 07:54:34,962][20381] Updated weights for policy 1, policy_version 327083 (0.0005) [2023-12-27 07:54:34,983][20347] Updated weights for policy 0, policy_version 327343 (0.0005) [2023-12-27 07:54:35,021][20381] Updated weights for policy 1, policy_version 327093 (0.0006) [2023-12-27 07:54:35,039][20347] Updated weights for policy 0, policy_version 327353 (0.0006) [2023-12-27 07:54:35,583][20347] Updated weights for policy 0, policy_version 327363 (0.0006) [2023-12-27 07:54:35,641][20347] Updated weights for policy 0, policy_version 327373 (0.0006) [2023-12-27 07:54:35,698][20347] Updated weights for policy 0, policy_version 327383 (0.0006) [2023-12-27 07:54:35,700][20381] Updated weights for policy 1, policy_version 327103 (0.0007) [2023-12-27 07:54:35,750][20381] Updated weights for policy 1, policy_version 327113 (0.0007) [2023-12-27 07:54:35,802][20381] Updated weights for policy 1, policy_version 327123 (0.0008) [2023-12-27 07:54:36,266][20347] Updated weights for policy 0, policy_version 327393 (0.0006) [2023-12-27 07:54:36,332][20347] Updated weights for policy 0, policy_version 327403 (0.0008) [2023-12-27 07:54:36,414][20347] Updated weights for policy 0, policy_version 327413 (0.0008) [2023-12-27 07:54:36,476][20347] Updated weights for policy 0, policy_version 327423 (0.0008) [2023-12-27 07:54:36,501][20381] Updated weights for policy 1, policy_version 327133 (0.0007) [2023-12-27 07:54:36,558][20381] Updated weights for policy 1, policy_version 327143 (0.0006) [2023-12-27 07:54:36,617][20381] Updated weights for policy 1, policy_version 327153 (0.0006) [2023-12-27 07:54:37,092][20347] Updated weights for policy 0, policy_version 327433 (0.0008) [2023-12-27 07:54:37,143][20347] Updated weights for policy 0, policy_version 327443 (0.0007) [2023-12-27 07:54:37,200][20347] Updated weights for policy 0, policy_version 327453 (0.0007) [2023-12-27 07:54:37,235][20381] Updated weights for policy 1, policy_version 327163 (0.0006) [2023-12-27 07:54:37,284][20381] Updated weights for policy 1, policy_version 327173 (0.0006) [2023-12-27 07:54:37,342][20381] Updated weights for policy 1, policy_version 327183 (0.0007) [2023-12-27 07:54:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 167616512. Throughput: 0: 10450.0, 1: 10462.6. Samples: 167584716. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:37,364][19170] Avg episode reward: [(0, '659.450'), (1, '1000.000')] [2023-12-27 07:54:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000327456_83845120.pth... [2023-12-27 07:54:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000326208_83525632.pth [2023-12-27 07:54:37,392][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000327192_83779584.pth... [2023-12-27 07:54:37,396][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000325944_83460096.pth [2023-12-27 07:54:37,903][20347] Updated weights for policy 0, policy_version 327463 (0.0008) [2023-12-27 07:54:37,963][20347] Updated weights for policy 0, policy_version 327473 (0.0007) [2023-12-27 07:54:38,023][20347] Updated weights for policy 0, policy_version 327483 (0.0008) [2023-12-27 07:54:38,025][20381] Updated weights for policy 1, policy_version 327193 (0.0008) [2023-12-27 07:54:38,073][20381] Updated weights for policy 1, policy_version 327203 (0.0006) [2023-12-27 07:54:38,122][20381] Updated weights for policy 1, policy_version 327213 (0.0007) [2023-12-27 07:54:38,171][20381] Updated weights for policy 1, policy_version 327223 (0.0008) [2023-12-27 07:54:38,710][20347] Updated weights for policy 0, policy_version 327493 (0.0007) [2023-12-27 07:54:38,764][20347] Updated weights for policy 0, policy_version 327503 (0.0007) [2023-12-27 07:54:38,819][20347] Updated weights for policy 0, policy_version 327513 (0.0008) [2023-12-27 07:54:38,871][20381] Updated weights for policy 1, policy_version 327233 (0.0007) [2023-12-27 07:54:38,918][20381] Updated weights for policy 1, policy_version 327243 (0.0007) [2023-12-27 07:54:38,961][20381] Updated weights for policy 1, policy_version 327253 (0.0008) [2023-12-27 07:54:39,529][20347] Updated weights for policy 0, policy_version 327523 (0.0006) [2023-12-27 07:54:39,587][20347] Updated weights for policy 0, policy_version 327533 (0.0007) [2023-12-27 07:54:39,650][20347] Updated weights for policy 0, policy_version 327543 (0.0008) [2023-12-27 07:54:39,706][20381] Updated weights for policy 1, policy_version 327263 (0.0006) [2023-12-27 07:54:39,756][20381] Updated weights for policy 1, policy_version 327273 (0.0008) [2023-12-27 07:54:39,818][20381] Updated weights for policy 1, policy_version 327283 (0.0007) [2023-12-27 07:54:40,365][20347] Updated weights for policy 0, policy_version 327553 (0.0006) [2023-12-27 07:54:40,423][20347] Updated weights for policy 0, policy_version 327563 (0.0006) [2023-12-27 07:54:40,476][20381] Updated weights for policy 1, policy_version 327293 (0.0007) [2023-12-27 07:54:40,486][20347] Updated weights for policy 0, policy_version 327573 (0.0007) [2023-12-27 07:54:40,539][20381] Updated weights for policy 1, policy_version 327303 (0.0006) [2023-12-27 07:54:40,544][20347] Updated weights for policy 0, policy_version 327583 (0.0006) [2023-12-27 07:54:40,597][20381] Updated weights for policy 1, policy_version 327313 (0.0007) [2023-12-27 07:54:41,263][20347] Updated weights for policy 0, policy_version 327593 (0.0007) [2023-12-27 07:54:41,302][20381] Updated weights for policy 1, policy_version 327323 (0.0008) [2023-12-27 07:54:41,319][20347] Updated weights for policy 0, policy_version 327603 (0.0008) [2023-12-27 07:54:41,364][20381] Updated weights for policy 1, policy_version 327333 (0.0007) [2023-12-27 07:54:41,388][20347] Updated weights for policy 0, policy_version 327613 (0.0009) [2023-12-27 07:54:41,426][20381] Updated weights for policy 1, policy_version 327343 (0.0008) [2023-12-27 07:54:42,121][20347] Updated weights for policy 0, policy_version 327623 (0.0007) [2023-12-27 07:54:42,178][20347] Updated weights for policy 0, policy_version 327633 (0.0005) [2023-12-27 07:54:42,179][20381] Updated weights for policy 1, policy_version 327353 (0.0008) [2023-12-27 07:54:42,230][20381] Updated weights for policy 1, policy_version 327363 (0.0007) [2023-12-27 07:54:42,239][20347] Updated weights for policy 0, policy_version 327643 (0.0005) [2023-12-27 07:54:42,293][20381] Updated weights for policy 1, policy_version 327373 (0.0007) [2023-12-27 07:54:42,357][20381] Updated weights for policy 1, policy_version 327383 (0.0008) [2023-12-27 07:54:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 167714816. Throughput: 0: 10452.0, 1: 10431.4. Samples: 167708252. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:42,363][19170] Avg episode reward: [(0, '750.240'), (1, '1000.000')] [2023-12-27 07:54:42,947][20347] Updated weights for policy 0, policy_version 327653 (0.0008) [2023-12-27 07:54:43,003][20347] Updated weights for policy 0, policy_version 327663 (0.0006) [2023-12-27 07:54:43,020][20381] Updated weights for policy 1, policy_version 327393 (0.0006) [2023-12-27 07:54:43,060][20347] Updated weights for policy 0, policy_version 327673 (0.0008) [2023-12-27 07:54:43,073][20381] Updated weights for policy 1, policy_version 327403 (0.0006) [2023-12-27 07:54:43,121][20381] Updated weights for policy 1, policy_version 327413 (0.0007) [2023-12-27 07:54:43,727][20381] Updated weights for policy 1, policy_version 327423 (0.0006) [2023-12-27 07:54:43,776][20381] Updated weights for policy 1, policy_version 327433 (0.0007) [2023-12-27 07:54:43,784][20347] Updated weights for policy 0, policy_version 327683 (0.0007) [2023-12-27 07:54:43,831][20381] Updated weights for policy 1, policy_version 327443 (0.0006) [2023-12-27 07:54:43,840][20347] Updated weights for policy 0, policy_version 327693 (0.0008) [2023-12-27 07:54:43,898][20347] Updated weights for policy 0, policy_version 327703 (0.0009) [2023-12-27 07:54:44,379][20381] Updated weights for policy 1, policy_version 327453 (0.0006) [2023-12-27 07:54:44,435][20381] Updated weights for policy 1, policy_version 327463 (0.0007) [2023-12-27 07:54:44,487][20381] Updated weights for policy 1, policy_version 327473 (0.0008) [2023-12-27 07:54:44,594][20347] Updated weights for policy 0, policy_version 327713 (0.0007) [2023-12-27 07:54:44,649][20347] Updated weights for policy 0, policy_version 327723 (0.0008) [2023-12-27 07:54:44,707][20347] Updated weights for policy 0, policy_version 327733 (0.0008) [2023-12-27 07:54:44,769][20347] Updated weights for policy 0, policy_version 327743 (0.0007) [2023-12-27 07:54:45,163][20381] Updated weights for policy 1, policy_version 327483 (0.0008) [2023-12-27 07:54:45,223][20381] Updated weights for policy 1, policy_version 327493 (0.0008) [2023-12-27 07:54:45,281][20381] Updated weights for policy 1, policy_version 327503 (0.0007) [2023-12-27 07:54:45,452][20347] Updated weights for policy 0, policy_version 327753 (0.0008) [2023-12-27 07:54:45,514][20347] Updated weights for policy 0, policy_version 327763 (0.0008) [2023-12-27 07:54:45,576][20347] Updated weights for policy 0, policy_version 327773 (0.0008) [2023-12-27 07:54:45,976][20381] Updated weights for policy 1, policy_version 327513 (0.0008) [2023-12-27 07:54:46,031][20381] Updated weights for policy 1, policy_version 327523 (0.0007) [2023-12-27 07:54:46,090][20381] Updated weights for policy 1, policy_version 327533 (0.0008) [2023-12-27 07:54:46,148][20381] Updated weights for policy 1, policy_version 327543 (0.0006) [2023-12-27 07:54:46,269][20347] Updated weights for policy 0, policy_version 327783 (0.0007) [2023-12-27 07:54:46,314][20347] Updated weights for policy 0, policy_version 327793 (0.0010) [2023-12-27 07:54:46,360][20347] Updated weights for policy 0, policy_version 327803 (0.0010) [2023-12-27 07:54:46,722][20381] Updated weights for policy 1, policy_version 327553 (0.0006) [2023-12-27 07:54:46,772][20381] Updated weights for policy 1, policy_version 327563 (0.0005) [2023-12-27 07:54:46,832][20381] Updated weights for policy 1, policy_version 327573 (0.0005) [2023-12-27 07:54:47,007][20347] Updated weights for policy 0, policy_version 327813 (0.0009) [2023-12-27 07:54:47,052][20347] Updated weights for policy 0, policy_version 327823 (0.0006) [2023-12-27 07:54:47,098][20347] Updated weights for policy 0, policy_version 327833 (0.0005) [2023-12-27 07:54:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 167821312. Throughput: 0: 10396.8, 1: 10505.0. Samples: 167831600. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:47,363][19170] Avg episode reward: [(0, '731.670'), (1, '1000.000')] [2023-12-27 07:54:47,504][20381] Updated weights for policy 1, policy_version 327583 (0.0005) [2023-12-27 07:54:47,560][20381] Updated weights for policy 1, policy_version 327593 (0.0009) [2023-12-27 07:54:47,609][20381] Updated weights for policy 1, policy_version 327603 (0.0009) [2023-12-27 07:54:47,652][20347] Updated weights for policy 0, policy_version 327843 (0.0005) [2023-12-27 07:54:47,697][20347] Updated weights for policy 0, policy_version 327853 (0.0005) [2023-12-27 07:54:47,750][20347] Updated weights for policy 0, policy_version 327863 (0.0005) [2023-12-27 07:54:48,325][20381] Updated weights for policy 1, policy_version 327613 (0.0008) [2023-12-27 07:54:48,379][20381] Updated weights for policy 1, policy_version 327623 (0.0008) [2023-12-27 07:54:48,406][20347] Updated weights for policy 0, policy_version 327873 (0.0005) [2023-12-27 07:54:48,433][20381] Updated weights for policy 1, policy_version 327633 (0.0007) [2023-12-27 07:54:48,462][20347] Updated weights for policy 0, policy_version 327883 (0.0006) [2023-12-27 07:54:48,522][20347] Updated weights for policy 0, policy_version 327893 (0.0007) [2023-12-27 07:54:48,578][20347] Updated weights for policy 0, policy_version 327903 (0.0007) [2023-12-27 07:54:49,167][20381] Updated weights for policy 1, policy_version 327643 (0.0009) [2023-12-27 07:54:49,228][20381] Updated weights for policy 1, policy_version 327653 (0.0007) [2023-12-27 07:54:49,280][20381] Updated weights for policy 1, policy_version 327663 (0.0006) [2023-12-27 07:54:49,286][20347] Updated weights for policy 0, policy_version 327913 (0.0008) [2023-12-27 07:54:49,342][20347] Updated weights for policy 0, policy_version 327923 (0.0006) [2023-12-27 07:54:49,406][20347] Updated weights for policy 0, policy_version 327933 (0.0006) [2023-12-27 07:54:49,934][20381] Updated weights for policy 1, policy_version 327673 (0.0007) [2023-12-27 07:54:49,981][20381] Updated weights for policy 1, policy_version 327683 (0.0008) [2023-12-27 07:54:50,025][20381] Updated weights for policy 1, policy_version 327693 (0.0007) [2023-12-27 07:54:50,071][20381] Updated weights for policy 1, policy_version 327703 (0.0006) [2023-12-27 07:54:50,141][20347] Updated weights for policy 0, policy_version 327943 (0.0007) [2023-12-27 07:54:50,200][20347] Updated weights for policy 0, policy_version 327953 (0.0007) [2023-12-27 07:54:50,261][20347] Updated weights for policy 0, policy_version 327963 (0.0008) [2023-12-27 07:54:50,679][20381] Updated weights for policy 1, policy_version 327713 (0.0005) [2023-12-27 07:54:50,729][20381] Updated weights for policy 1, policy_version 327723 (0.0005) [2023-12-27 07:54:50,785][20381] Updated weights for policy 1, policy_version 327733 (0.0008) [2023-12-27 07:54:51,017][20347] Updated weights for policy 0, policy_version 327973 (0.0008) [2023-12-27 07:54:51,076][20347] Updated weights for policy 0, policy_version 327983 (0.0008) [2023-12-27 07:54:51,133][20347] Updated weights for policy 0, policy_version 327993 (0.0007) [2023-12-27 07:54:51,430][20381] Updated weights for policy 1, policy_version 327743 (0.0007) [2023-12-27 07:54:51,488][20381] Updated weights for policy 1, policy_version 327753 (0.0008) [2023-12-27 07:54:51,540][20381] Updated weights for policy 1, policy_version 327763 (0.0009) [2023-12-27 07:54:51,872][20347] Updated weights for policy 0, policy_version 328003 (0.0008) [2023-12-27 07:54:51,925][20347] Updated weights for policy 0, policy_version 328013 (0.0007) [2023-12-27 07:54:51,983][20347] Updated weights for policy 0, policy_version 328023 (0.0008) [2023-12-27 07:54:52,210][20381] Updated weights for policy 1, policy_version 327773 (0.0007) [2023-12-27 07:54:52,271][20381] Updated weights for policy 1, policy_version 327783 (0.0008) [2023-12-27 07:54:52,331][20381] Updated weights for policy 1, policy_version 327793 (0.0008) [2023-12-27 07:54:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 167919616. Throughput: 0: 10404.0, 1: 10428.1. Samples: 167893320. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 07:54:52,363][19170] Avg episode reward: [(0, '742.320'), (1, '1000.000')] [2023-12-27 07:54:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000328032_83992576.pth... [2023-12-27 07:54:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000326848_83689472.pth [2023-12-27 07:54:52,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000327800_83935232.pth... [2023-12-27 07:54:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000326552_83615744.pth [2023-12-27 07:54:52,708][20347] Updated weights for policy 0, policy_version 328033 (0.0008) [2023-12-27 07:54:52,762][20347] Updated weights for policy 0, policy_version 328043 (0.0008) [2023-12-27 07:54:52,810][20347] Updated weights for policy 0, policy_version 328053 (0.0007) [2023-12-27 07:54:52,864][20347] Updated weights for policy 0, policy_version 328063 (0.0007) [2023-12-27 07:54:53,008][20381] Updated weights for policy 1, policy_version 327803 (0.0007) [2023-12-27 07:54:53,065][20381] Updated weights for policy 1, policy_version 327813 (0.0008) [2023-12-27 07:54:53,123][20381] Updated weights for policy 1, policy_version 327823 (0.0007) [2023-12-27 07:54:53,528][20347] Updated weights for policy 0, policy_version 328073 (0.0006) [2023-12-27 07:54:53,577][20347] Updated weights for policy 0, policy_version 328083 (0.0007) [2023-12-27 07:54:53,643][20347] Updated weights for policy 0, policy_version 328093 (0.0008) [2023-12-27 07:54:53,807][20381] Updated weights for policy 1, policy_version 327833 (0.0006) [2023-12-27 07:54:53,867][20381] Updated weights for policy 1, policy_version 327843 (0.0005) [2023-12-27 07:54:53,931][20381] Updated weights for policy 1, policy_version 327853 (0.0005) [2023-12-27 07:54:53,991][20381] Updated weights for policy 1, policy_version 327863 (0.0005) [2023-12-27 07:54:54,350][20347] Updated weights for policy 0, policy_version 328103 (0.0007) [2023-12-27 07:54:54,356][20291] KL-divergence is very high: 162.9145 [2023-12-27 07:54:54,413][20347] Updated weights for policy 0, policy_version 328113 (0.0007) [2023-12-27 07:54:54,462][20347] Updated weights for policy 0, policy_version 328123 (0.0008) [2023-12-27 07:54:54,583][20381] Updated weights for policy 1, policy_version 327873 (0.0007) [2023-12-27 07:54:54,639][20381] Updated weights for policy 1, policy_version 327883 (0.0007) [2023-12-27 07:54:54,709][20381] Updated weights for policy 1, policy_version 327893 (0.0009) [2023-12-27 07:54:55,121][20347] Updated weights for policy 0, policy_version 328133 (0.0009) [2023-12-27 07:54:55,175][20347] Updated weights for policy 0, policy_version 328143 (0.0010) [2023-12-27 07:54:55,231][20347] Updated weights for policy 0, policy_version 328153 (0.0010) [2023-12-27 07:54:55,360][20381] Updated weights for policy 1, policy_version 327903 (0.0011) [2023-12-27 07:54:55,419][20381] Updated weights for policy 1, policy_version 327913 (0.0008) [2023-12-27 07:54:55,477][20381] Updated weights for policy 1, policy_version 327923 (0.0007) [2023-12-27 07:54:55,911][20347] Updated weights for policy 0, policy_version 328163 (0.0010) [2023-12-27 07:54:55,973][20347] Updated weights for policy 0, policy_version 328173 (0.0010) [2023-12-27 07:54:56,032][20347] Updated weights for policy 0, policy_version 328183 (0.0010) [2023-12-27 07:54:56,047][20381] Updated weights for policy 1, policy_version 327933 (0.0008) [2023-12-27 07:54:56,110][20381] Updated weights for policy 1, policy_version 327943 (0.0005) [2023-12-27 07:54:56,170][20381] Updated weights for policy 1, policy_version 327953 (0.0007) [2023-12-27 07:54:56,701][20381] Updated weights for policy 1, policy_version 327963 (0.0006) [2023-12-27 07:54:56,702][20347] Updated weights for policy 0, policy_version 328193 (0.0010) [2023-12-27 07:54:56,754][20381] Updated weights for policy 1, policy_version 327973 (0.0010) [2023-12-27 07:54:56,759][20347] Updated weights for policy 0, policy_version 328203 (0.0010) [2023-12-27 07:54:56,808][20381] Updated weights for policy 1, policy_version 327983 (0.0010) [2023-12-27 07:54:56,810][20347] Updated weights for policy 0, policy_version 328213 (0.0010) [2023-12-27 07:54:56,867][20347] Updated weights for policy 0, policy_version 328223 (0.0010) [2023-12-27 07:54:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 168026112. Throughput: 0: 10290.2, 1: 10464.6. Samples: 168016488. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:54:57,363][19170] Avg episode reward: [(0, '582.940'), (1, '990.350')] [2023-12-27 07:54:57,425][20381] Updated weights for policy 1, policy_version 327993 (0.0010) [2023-12-27 07:54:57,480][20381] Updated weights for policy 1, policy_version 328003 (0.0010) [2023-12-27 07:54:57,530][20347] Updated weights for policy 0, policy_version 328233 (0.0010) [2023-12-27 07:54:57,535][20381] Updated weights for policy 1, policy_version 328013 (0.0010) [2023-12-27 07:54:57,577][20347] Updated weights for policy 0, policy_version 328243 (0.0010) [2023-12-27 07:54:57,583][20381] Updated weights for policy 1, policy_version 328023 (0.0010) [2023-12-27 07:54:57,631][20347] Updated weights for policy 0, policy_version 328253 (0.0010) [2023-12-27 07:54:58,174][20381] Updated weights for policy 1, policy_version 328033 (0.0008) [2023-12-27 07:54:58,230][20381] Updated weights for policy 1, policy_version 328043 (0.0011) [2023-12-27 07:54:58,288][20381] Updated weights for policy 1, policy_version 328053 (0.0011) [2023-12-27 07:54:58,306][20347] Updated weights for policy 0, policy_version 328263 (0.0010) [2023-12-27 07:54:58,379][20347] Updated weights for policy 0, policy_version 328274 (0.0009) [2023-12-27 07:54:58,449][20347] Updated weights for policy 0, policy_version 328284 (0.0007) [2023-12-27 07:54:59,004][20381] Updated weights for policy 1, policy_version 328063 (0.0009) [2023-12-27 07:54:59,066][20381] Updated weights for policy 1, policy_version 328073 (0.0007) [2023-12-27 07:54:59,131][20381] Updated weights for policy 1, policy_version 328083 (0.0008) [2023-12-27 07:54:59,192][20347] Updated weights for policy 0, policy_version 328294 (0.0008) [2023-12-27 07:54:59,270][20347] Updated weights for policy 0, policy_version 328305 (0.0007) [2023-12-27 07:54:59,336][20347] Updated weights for policy 0, policy_version 328315 (0.0007) [2023-12-27 07:54:59,757][20381] Updated weights for policy 1, policy_version 328093 (0.0008) [2023-12-27 07:54:59,812][20381] Updated weights for policy 1, policy_version 328103 (0.0007) [2023-12-27 07:54:59,870][20381] Updated weights for policy 1, policy_version 328113 (0.0008) [2023-12-27 07:54:59,964][20347] Updated weights for policy 0, policy_version 328325 (0.0006) [2023-12-27 07:55:00,033][20347] Updated weights for policy 0, policy_version 328335 (0.0011) [2023-12-27 07:55:00,094][20347] Updated weights for policy 0, policy_version 328345 (0.0009) [2023-12-27 07:55:00,481][20381] Updated weights for policy 1, policy_version 328123 (0.0009) [2023-12-27 07:55:00,542][20381] Updated weights for policy 1, policy_version 328133 (0.0008) [2023-12-27 07:55:00,599][20381] Updated weights for policy 1, policy_version 328143 (0.0007) [2023-12-27 07:55:00,700][20347] Updated weights for policy 0, policy_version 328355 (0.0009) [2023-12-27 07:55:00,744][20347] Updated weights for policy 0, policy_version 328365 (0.0010) [2023-12-27 07:55:00,801][20347] Updated weights for policy 0, policy_version 328375 (0.0010) [2023-12-27 07:55:01,322][20381] Updated weights for policy 1, policy_version 328153 (0.0007) [2023-12-27 07:55:01,392][20381] Updated weights for policy 1, policy_version 328163 (0.0007) [2023-12-27 07:55:01,456][20381] Updated weights for policy 1, policy_version 328173 (0.0008) [2023-12-27 07:55:01,508][20347] Updated weights for policy 0, policy_version 328386 (0.0011) [2023-12-27 07:55:01,518][20381] Updated weights for policy 1, policy_version 328183 (0.0007) [2023-12-27 07:55:01,566][20347] Updated weights for policy 0, policy_version 328396 (0.0006) [2023-12-27 07:55:01,629][20347] Updated weights for policy 0, policy_version 328406 (0.0007) [2023-12-27 07:55:01,682][20347] Updated weights for policy 0, policy_version 328416 (0.0008) [2023-12-27 07:55:02,156][20381] Updated weights for policy 1, policy_version 328193 (0.0007) [2023-12-27 07:55:02,203][20381] Updated weights for policy 1, policy_version 328203 (0.0008) [2023-12-27 07:55:02,260][20381] Updated weights for policy 1, policy_version 328213 (0.0007) [2023-12-27 07:55:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 168132608. Throughput: 0: 10237.9, 1: 10534.2. Samples: 168142120. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:02,363][19170] Avg episode reward: [(0, '675.320'), (1, '921.560')] [2023-12-27 07:55:02,382][20347] Updated weights for policy 0, policy_version 328426 (0.0006) [2023-12-27 07:55:02,444][20347] Updated weights for policy 0, policy_version 328436 (0.0008) [2023-12-27 07:55:02,502][20347] Updated weights for policy 0, policy_version 328446 (0.0007) [2023-12-27 07:55:02,979][20381] Updated weights for policy 1, policy_version 328223 (0.0009) [2023-12-27 07:55:03,034][20381] Updated weights for policy 1, policy_version 328233 (0.0008) [2023-12-27 07:55:03,091][20381] Updated weights for policy 1, policy_version 328243 (0.0008) [2023-12-27 07:55:03,148][20347] Updated weights for policy 0, policy_version 328456 (0.0007) [2023-12-27 07:55:03,196][20347] Updated weights for policy 0, policy_version 328466 (0.0007) [2023-12-27 07:55:03,244][20347] Updated weights for policy 0, policy_version 328476 (0.0009) [2023-12-27 07:55:03,717][20381] Updated weights for policy 1, policy_version 328253 (0.0007) [2023-12-27 07:55:03,768][20381] Updated weights for policy 1, policy_version 328263 (0.0005) [2023-12-27 07:55:03,812][20381] Updated weights for policy 1, policy_version 328273 (0.0005) [2023-12-27 07:55:03,962][20347] Updated weights for policy 0, policy_version 328486 (0.0008) [2023-12-27 07:55:04,015][20347] Updated weights for policy 0, policy_version 328496 (0.0007) [2023-12-27 07:55:04,076][20347] Updated weights for policy 0, policy_version 328506 (0.0008) [2023-12-27 07:55:04,474][20381] Updated weights for policy 1, policy_version 328283 (0.0008) [2023-12-27 07:55:04,539][20381] Updated weights for policy 1, policy_version 328293 (0.0012) [2023-12-27 07:55:04,594][20381] Updated weights for policy 1, policy_version 328303 (0.0010) [2023-12-27 07:55:04,772][20347] Updated weights for policy 0, policy_version 328516 (0.0007) [2023-12-27 07:55:04,830][20347] Updated weights for policy 0, policy_version 328526 (0.0007) [2023-12-27 07:55:04,893][20347] Updated weights for policy 0, policy_version 328536 (0.0006) [2023-12-27 07:55:05,269][20381] Updated weights for policy 1, policy_version 328313 (0.0011) [2023-12-27 07:55:05,320][20381] Updated weights for policy 1, policy_version 328323 (0.0010) [2023-12-27 07:55:05,371][20381] Updated weights for policy 1, policy_version 328333 (0.0006) [2023-12-27 07:55:05,415][20381] Updated weights for policy 1, policy_version 328343 (0.0010) [2023-12-27 07:55:05,558][20347] Updated weights for policy 0, policy_version 328546 (0.0007) [2023-12-27 07:55:05,608][20347] Updated weights for policy 0, policy_version 328556 (0.0008) [2023-12-27 07:55:05,655][20347] Updated weights for policy 0, policy_version 328566 (0.0007) [2023-12-27 07:55:05,712][20347] Updated weights for policy 0, policy_version 328576 (0.0007) [2023-12-27 07:55:06,074][20381] Updated weights for policy 1, policy_version 328353 (0.0006) [2023-12-27 07:55:06,134][20381] Updated weights for policy 1, policy_version 328363 (0.0008) [2023-12-27 07:55:06,198][20381] Updated weights for policy 1, policy_version 328373 (0.0008) [2023-12-27 07:55:06,432][20347] Updated weights for policy 0, policy_version 328586 (0.0007) [2023-12-27 07:55:06,489][20347] Updated weights for policy 0, policy_version 328596 (0.0008) [2023-12-27 07:55:06,546][20347] Updated weights for policy 0, policy_version 328606 (0.0007) [2023-12-27 07:55:06,904][20381] Updated weights for policy 1, policy_version 328383 (0.0009) [2023-12-27 07:55:06,963][20381] Updated weights for policy 1, policy_version 328393 (0.0007) [2023-12-27 07:55:07,017][20381] Updated weights for policy 1, policy_version 328403 (0.0008) [2023-12-27 07:55:07,164][20347] Updated weights for policy 0, policy_version 328616 (0.0006) [2023-12-27 07:55:07,213][20347] Updated weights for policy 0, policy_version 328626 (0.0007) [2023-12-27 07:55:07,266][20347] Updated weights for policy 0, policy_version 328636 (0.0008) [2023-12-27 07:55:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 168239104. Throughput: 0: 10201.7, 1: 10515.6. Samples: 168204324. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:07,363][19170] Avg episode reward: [(0, '685.310'), (1, '908.430')] [2023-12-27 07:55:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000328408_84090880.pth... [2023-12-27 07:55:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000328640_84148224.pth... [2023-12-27 07:55:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000327192_83779584.pth [2023-12-27 07:55:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000327456_83845120.pth [2023-12-27 07:55:07,747][20381] Updated weights for policy 1, policy_version 328413 (0.0007) [2023-12-27 07:55:07,797][20381] Updated weights for policy 1, policy_version 328423 (0.0007) [2023-12-27 07:55:07,850][20381] Updated weights for policy 1, policy_version 328433 (0.0008) [2023-12-27 07:55:07,958][20347] Updated weights for policy 0, policy_version 328646 (0.0008) [2023-12-27 07:55:08,010][20347] Updated weights for policy 0, policy_version 328656 (0.0008) [2023-12-27 07:55:08,062][20347] Updated weights for policy 0, policy_version 328666 (0.0009) [2023-12-27 07:55:08,495][20381] Updated weights for policy 1, policy_version 328443 (0.0007) [2023-12-27 07:55:08,551][20381] Updated weights for policy 1, policy_version 328453 (0.0007) [2023-12-27 07:55:08,610][20381] Updated weights for policy 1, policy_version 328463 (0.0007) [2023-12-27 07:55:08,783][20347] Updated weights for policy 0, policy_version 328676 (0.0007) [2023-12-27 07:55:08,841][20347] Updated weights for policy 0, policy_version 328686 (0.0005) [2023-12-27 07:55:08,905][20347] Updated weights for policy 0, policy_version 328696 (0.0005) [2023-12-27 07:55:09,315][20381] Updated weights for policy 1, policy_version 328473 (0.0007) [2023-12-27 07:55:09,374][20381] Updated weights for policy 1, policy_version 328483 (0.0008) [2023-12-27 07:55:09,435][20381] Updated weights for policy 1, policy_version 328493 (0.0006) [2023-12-27 07:55:09,492][20381] Updated weights for policy 1, policy_version 328503 (0.0008) [2023-12-27 07:55:09,572][20347] Updated weights for policy 0, policy_version 328706 (0.0006) [2023-12-27 07:55:09,628][20347] Updated weights for policy 0, policy_version 328716 (0.0008) [2023-12-27 07:55:09,683][20347] Updated weights for policy 0, policy_version 328726 (0.0008) [2023-12-27 07:55:09,744][20347] Updated weights for policy 0, policy_version 328736 (0.0007) [2023-12-27 07:55:10,175][20381] Updated weights for policy 1, policy_version 328513 (0.0006) [2023-12-27 07:55:10,239][20381] Updated weights for policy 1, policy_version 328523 (0.0008) [2023-12-27 07:55:10,291][20381] Updated weights for policy 1, policy_version 328533 (0.0008) [2023-12-27 07:55:10,465][20347] Updated weights for policy 0, policy_version 328746 (0.0008) [2023-12-27 07:55:10,521][20347] Updated weights for policy 0, policy_version 328756 (0.0008) [2023-12-27 07:55:10,582][20347] Updated weights for policy 0, policy_version 328766 (0.0007) [2023-12-27 07:55:10,958][20381] Updated weights for policy 1, policy_version 328543 (0.0008) [2023-12-27 07:55:11,015][20381] Updated weights for policy 1, policy_version 328553 (0.0006) [2023-12-27 07:55:11,079][20381] Updated weights for policy 1, policy_version 328563 (0.0007) [2023-12-27 07:55:11,353][20347] Updated weights for policy 0, policy_version 328776 (0.0008) [2023-12-27 07:55:11,414][20347] Updated weights for policy 0, policy_version 328786 (0.0008) [2023-12-27 07:55:11,467][20347] Updated weights for policy 0, policy_version 328796 (0.0008) [2023-12-27 07:55:11,795][20381] Updated weights for policy 1, policy_version 328573 (0.0008) [2023-12-27 07:55:11,852][20381] Updated weights for policy 1, policy_version 328583 (0.0008) [2023-12-27 07:55:11,902][20381] Updated weights for policy 1, policy_version 328593 (0.0008) [2023-12-27 07:55:12,200][20347] Updated weights for policy 0, policy_version 328806 (0.0008) [2023-12-27 07:55:12,252][20347] Updated weights for policy 0, policy_version 328816 (0.0007) [2023-12-27 07:55:12,313][20347] Updated weights for policy 0, policy_version 328826 (0.0007) [2023-12-27 07:55:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 168337408. Throughput: 0: 10210.5, 1: 10463.2. Samples: 168325120. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:12,365][19170] Avg episode reward: [(0, '621.580'), (1, '948.110')] [2023-12-27 07:55:12,591][20381] Updated weights for policy 1, policy_version 328603 (0.0007) [2023-12-27 07:55:12,644][20381] Updated weights for policy 1, policy_version 328613 (0.0008) [2023-12-27 07:55:12,700][20381] Updated weights for policy 1, policy_version 328623 (0.0007) [2023-12-27 07:55:13,001][20347] Updated weights for policy 0, policy_version 328836 (0.0008) [2023-12-27 07:55:13,058][20347] Updated weights for policy 0, policy_version 328846 (0.0007) [2023-12-27 07:55:13,112][20347] Updated weights for policy 0, policy_version 328856 (0.0007) [2023-12-27 07:55:13,346][20381] Updated weights for policy 1, policy_version 328633 (0.0008) [2023-12-27 07:55:13,402][20381] Updated weights for policy 1, policy_version 328643 (0.0011) [2023-12-27 07:55:13,461][20381] Updated weights for policy 1, policy_version 328653 (0.0011) [2023-12-27 07:55:13,520][20381] Updated weights for policy 1, policy_version 328663 (0.0011) [2023-12-27 07:55:13,771][20347] Updated weights for policy 0, policy_version 328866 (0.0008) [2023-12-27 07:55:13,825][20347] Updated weights for policy 0, policy_version 328876 (0.0007) [2023-12-27 07:55:13,885][20347] Updated weights for policy 0, policy_version 328886 (0.0007) [2023-12-27 07:55:13,938][20347] Updated weights for policy 0, policy_version 328896 (0.0007) [2023-12-27 07:55:14,182][20381] Updated weights for policy 1, policy_version 328673 (0.0008) [2023-12-27 07:55:14,238][20381] Updated weights for policy 1, policy_version 328683 (0.0006) [2023-12-27 07:55:14,290][20381] Updated weights for policy 1, policy_version 328693 (0.0005) [2023-12-27 07:55:14,634][20347] Updated weights for policy 0, policy_version 328906 (0.0006) [2023-12-27 07:55:14,695][20347] Updated weights for policy 0, policy_version 328916 (0.0006) [2023-12-27 07:55:14,742][20347] Updated weights for policy 0, policy_version 328926 (0.0007) [2023-12-27 07:55:14,947][20381] Updated weights for policy 1, policy_version 328703 (0.0010) [2023-12-27 07:55:15,001][20381] Updated weights for policy 1, policy_version 328713 (0.0011) [2023-12-27 07:55:15,062][20381] Updated weights for policy 1, policy_version 328723 (0.0011) [2023-12-27 07:55:15,431][20347] Updated weights for policy 0, policy_version 328936 (0.0007) [2023-12-27 07:55:15,481][20347] Updated weights for policy 0, policy_version 328946 (0.0007) [2023-12-27 07:55:15,538][20347] Updated weights for policy 0, policy_version 328956 (0.0005) [2023-12-27 07:55:15,709][20381] Updated weights for policy 1, policy_version 328733 (0.0008) [2023-12-27 07:55:15,757][20381] Updated weights for policy 1, policy_version 328743 (0.0005) [2023-12-27 07:55:15,818][20381] Updated weights for policy 1, policy_version 328753 (0.0008) [2023-12-27 07:55:16,149][20347] Updated weights for policy 0, policy_version 328966 (0.0005) [2023-12-27 07:55:16,211][20347] Updated weights for policy 0, policy_version 328976 (0.0006) [2023-12-27 07:55:16,272][20347] Updated weights for policy 0, policy_version 328986 (0.0005) [2023-12-27 07:55:16,406][20381] Updated weights for policy 1, policy_version 328763 (0.0008) [2023-12-27 07:55:16,457][20381] Updated weights for policy 1, policy_version 328773 (0.0006) [2023-12-27 07:55:16,510][20381] Updated weights for policy 1, policy_version 328783 (0.0005) [2023-12-27 07:55:16,848][20347] Updated weights for policy 0, policy_version 328996 (0.0006) [2023-12-27 07:55:16,910][20347] Updated weights for policy 0, policy_version 329006 (0.0005) [2023-12-27 07:55:16,972][20347] Updated weights for policy 0, policy_version 329016 (0.0005) [2023-12-27 07:55:17,133][20381] Updated weights for policy 1, policy_version 328793 (0.0005) [2023-12-27 07:55:17,193][20381] Updated weights for policy 1, policy_version 328803 (0.0006) [2023-12-27 07:55:17,252][20381] Updated weights for policy 1, policy_version 328813 (0.0005) [2023-12-27 07:55:17,308][20381] Updated weights for policy 1, policy_version 328823 (0.0005) [2023-12-27 07:55:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20854.9). Total num frames: 168443904. Throughput: 0: 10180.5, 1: 10514.4. Samples: 168451892. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:17,363][19170] Avg episode reward: [(0, '694.510'), (1, '951.830')] [2023-12-27 07:55:17,485][20347] Updated weights for policy 0, policy_version 329026 (0.0005) [2023-12-27 07:55:17,539][20347] Updated weights for policy 0, policy_version 329036 (0.0005) [2023-12-27 07:55:17,585][20347] Updated weights for policy 0, policy_version 329046 (0.0005) [2023-12-27 07:55:17,629][20347] Updated weights for policy 0, policy_version 329056 (0.0005) [2023-12-27 07:55:17,827][20381] Updated weights for policy 1, policy_version 328833 (0.0005) [2023-12-27 07:55:17,885][20381] Updated weights for policy 1, policy_version 328843 (0.0005) [2023-12-27 07:55:17,950][20381] Updated weights for policy 1, policy_version 328853 (0.0005) [2023-12-27 07:55:18,168][20347] Updated weights for policy 0, policy_version 329066 (0.0010) [2023-12-27 07:55:18,222][20347] Updated weights for policy 0, policy_version 329076 (0.0010) [2023-12-27 07:55:18,280][20347] Updated weights for policy 0, policy_version 329086 (0.0010) [2023-12-27 07:55:18,530][20381] Updated weights for policy 1, policy_version 328863 (0.0007) [2023-12-27 07:55:18,583][20381] Updated weights for policy 1, policy_version 328873 (0.0006) [2023-12-27 07:55:18,645][20381] Updated weights for policy 1, policy_version 328883 (0.0006) [2023-12-27 07:55:18,968][20347] Updated weights for policy 0, policy_version 329096 (0.0008) [2023-12-27 07:55:19,014][20347] Updated weights for policy 0, policy_version 329106 (0.0007) [2023-12-27 07:55:19,066][20347] Updated weights for policy 0, policy_version 329116 (0.0006) [2023-12-27 07:55:19,256][20381] Updated weights for policy 1, policy_version 328893 (0.0007) [2023-12-27 07:55:19,306][20381] Updated weights for policy 1, policy_version 328903 (0.0008) [2023-12-27 07:55:19,370][20381] Updated weights for policy 1, policy_version 328913 (0.0008) [2023-12-27 07:55:19,819][20347] Updated weights for policy 0, policy_version 329126 (0.0007) [2023-12-27 07:55:19,878][20347] Updated weights for policy 0, policy_version 329136 (0.0008) [2023-12-27 07:55:19,934][20347] Updated weights for policy 0, policy_version 329146 (0.0008) [2023-12-27 07:55:19,994][20381] Updated weights for policy 1, policy_version 328923 (0.0006) [2023-12-27 07:55:20,046][20381] Updated weights for policy 1, policy_version 328933 (0.0006) [2023-12-27 07:55:20,097][20381] Updated weights for policy 1, policy_version 328943 (0.0006) [2023-12-27 07:55:20,698][20347] Updated weights for policy 0, policy_version 329156 (0.0008) [2023-12-27 07:55:20,759][20347] Updated weights for policy 0, policy_version 329166 (0.0008) [2023-12-27 07:55:20,773][20381] Updated weights for policy 1, policy_version 328953 (0.0008) [2023-12-27 07:55:20,820][20347] Updated weights for policy 0, policy_version 329176 (0.0007) [2023-12-27 07:55:20,829][20381] Updated weights for policy 1, policy_version 328963 (0.0006) [2023-12-27 07:55:20,888][20381] Updated weights for policy 1, policy_version 328973 (0.0007) [2023-12-27 07:55:20,951][20381] Updated weights for policy 1, policy_version 328983 (0.0009) [2023-12-27 07:55:21,489][20347] Updated weights for policy 0, policy_version 329186 (0.0006) [2023-12-27 07:55:21,539][20347] Updated weights for policy 0, policy_version 329196 (0.0008) [2023-12-27 07:55:21,595][20347] Updated weights for policy 0, policy_version 329206 (0.0006) [2023-12-27 07:55:21,661][20347] Updated weights for policy 0, policy_version 329216 (0.0007) [2023-12-27 07:55:21,684][20381] Updated weights for policy 1, policy_version 328993 (0.0007) [2023-12-27 07:55:21,749][20381] Updated weights for policy 1, policy_version 329003 (0.0008) [2023-12-27 07:55:21,816][20381] Updated weights for policy 1, policy_version 329013 (0.0008) [2023-12-27 07:55:22,309][20347] Updated weights for policy 0, policy_version 329226 (0.0006) [2023-12-27 07:55:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 168542208. Throughput: 0: 10207.7, 1: 10527.2. Samples: 168517780. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:22,363][19170] Avg episode reward: [(0, '799.120'), (1, '961.710')] [2023-12-27 07:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000329016_84246528.pth... [2023-12-27 07:55:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000327800_83935232.pth [2023-12-27 07:55:22,373][20347] Updated weights for policy 0, policy_version 329236 (0.0007) [2023-12-27 07:55:22,425][20347] Updated weights for policy 0, policy_version 329246 (0.0008) [2023-12-27 07:55:22,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000329248_84303872.pth... [2023-12-27 07:55:22,436][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000328032_83992576.pth [2023-12-27 07:55:22,516][20381] Updated weights for policy 1, policy_version 329023 (0.0007) [2023-12-27 07:55:22,572][20381] Updated weights for policy 1, policy_version 329033 (0.0007) [2023-12-27 07:55:22,635][20381] Updated weights for policy 1, policy_version 329043 (0.0010) [2023-12-27 07:55:23,064][20347] Updated weights for policy 0, policy_version 329256 (0.0009) [2023-12-27 07:55:23,120][20347] Updated weights for policy 0, policy_version 329266 (0.0008) [2023-12-27 07:55:23,179][20347] Updated weights for policy 0, policy_version 329276 (0.0006) [2023-12-27 07:55:23,291][20381] Updated weights for policy 1, policy_version 329053 (0.0006) [2023-12-27 07:55:23,342][20381] Updated weights for policy 1, policy_version 329063 (0.0006) [2023-12-27 07:55:23,394][20381] Updated weights for policy 1, policy_version 329073 (0.0005) [2023-12-27 07:55:23,884][20347] Updated weights for policy 0, policy_version 329286 (0.0008) [2023-12-27 07:55:23,920][20381] Updated weights for policy 1, policy_version 329083 (0.0005) [2023-12-27 07:55:23,933][20347] Updated weights for policy 0, policy_version 329296 (0.0008) [2023-12-27 07:55:23,975][20381] Updated weights for policy 1, policy_version 329093 (0.0006) [2023-12-27 07:55:23,983][20347] Updated weights for policy 0, policy_version 329306 (0.0006) [2023-12-27 07:55:24,032][20381] Updated weights for policy 1, policy_version 329103 (0.0005) [2023-12-27 07:55:24,595][20381] Updated weights for policy 1, policy_version 329113 (0.0005) [2023-12-27 07:55:24,646][20292] KL-divergence is very high: 149.8801 [2023-12-27 07:55:24,650][20381] Updated weights for policy 1, policy_version 329123 (0.0008) [2023-12-27 07:55:24,707][20347] Updated weights for policy 0, policy_version 329316 (0.0007) [2023-12-27 07:55:24,714][20381] Updated weights for policy 1, policy_version 329133 (0.0006) [2023-12-27 07:55:24,766][20347] Updated weights for policy 0, policy_version 329326 (0.0007) [2023-12-27 07:55:24,766][20381] Updated weights for policy 1, policy_version 329143 (0.0005) [2023-12-27 07:55:24,815][20347] Updated weights for policy 0, policy_version 329336 (0.0009) [2023-12-27 07:55:25,346][20381] Updated weights for policy 1, policy_version 329153 (0.0005) [2023-12-27 07:55:25,404][20381] Updated weights for policy 1, policy_version 329163 (0.0007) [2023-12-27 07:55:25,458][20381] Updated weights for policy 1, policy_version 329173 (0.0007) [2023-12-27 07:55:25,472][20347] Updated weights for policy 0, policy_version 329346 (0.0007) [2023-12-27 07:55:25,524][20347] Updated weights for policy 0, policy_version 329356 (0.0005) [2023-12-27 07:55:25,579][20347] Updated weights for policy 0, policy_version 329366 (0.0005) [2023-12-27 07:55:25,604][20291] KL-divergence is very high: 134.7929 [2023-12-27 07:55:25,610][20291] KL-divergence is very high: 101.8376 [2023-12-27 07:55:25,620][20291] KL-divergence is very high: 285.2065 [2023-12-27 07:55:25,625][20291] KL-divergence is very high: 108.8096 [2023-12-27 07:55:25,629][20347] Updated weights for policy 0, policy_version 329376 (0.0006) [2023-12-27 07:55:26,166][20291] KL-divergence is very high: 335.3084 [2023-12-27 07:55:26,177][20381] Updated weights for policy 1, policy_version 329183 (0.0006) [2023-12-27 07:55:26,186][20291] KL-divergence is very high: 141.6534 [2023-12-27 07:55:26,197][20347] Updated weights for policy 0, policy_version 329386 (0.0006) [2023-12-27 07:55:26,218][20291] KL-divergence is very high: 209.9335 [2023-12-27 07:55:26,227][20381] Updated weights for policy 1, policy_version 329193 (0.0005) [2023-12-27 07:55:26,257][20347] Updated weights for policy 0, policy_version 329396 (0.0007) [2023-12-27 07:55:26,273][20381] Updated weights for policy 1, policy_version 329203 (0.0005) [2023-12-27 07:55:26,308][20291] KL-divergence is very high: 203.1792 [2023-12-27 07:55:26,314][20347] Updated weights for policy 0, policy_version 329406 (0.0007) [2023-12-27 07:55:26,929][20381] Updated weights for policy 1, policy_version 329213 (0.0006) [2023-12-27 07:55:26,981][20381] Updated weights for policy 1, policy_version 329223 (0.0008) [2023-12-27 07:55:26,992][20347] Updated weights for policy 0, policy_version 329416 (0.0010) [2023-12-27 07:55:27,034][20381] Updated weights for policy 1, policy_version 329233 (0.0006) [2023-12-27 07:55:27,037][20347] Updated weights for policy 0, policy_version 329426 (0.0010) [2023-12-27 07:55:27,092][20347] Updated weights for policy 0, policy_version 329436 (0.0010) [2023-12-27 07:55:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 168656896. Throughput: 0: 10200.6, 1: 10591.0. Samples: 168643872. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:27,363][19170] Avg episode reward: [(0, '635.600'), (1, '932.970')] [2023-12-27 07:55:27,721][20381] Updated weights for policy 1, policy_version 329243 (0.0005) [2023-12-27 07:55:27,767][20347] Updated weights for policy 0, policy_version 329446 (0.0008) [2023-12-27 07:55:27,768][20381] Updated weights for policy 1, policy_version 329253 (0.0005) [2023-12-27 07:55:27,811][20381] Updated weights for policy 1, policy_version 329263 (0.0006) [2023-12-27 07:55:27,818][20347] Updated weights for policy 0, policy_version 329456 (0.0007) [2023-12-27 07:55:27,864][20347] Updated weights for policy 0, policy_version 329466 (0.0007) [2023-12-27 07:55:28,449][20381] Updated weights for policy 1, policy_version 329273 (0.0008) [2023-12-27 07:55:28,507][20381] Updated weights for policy 1, policy_version 329283 (0.0007) [2023-12-27 07:55:28,534][20347] Updated weights for policy 0, policy_version 329476 (0.0007) [2023-12-27 07:55:28,564][20381] Updated weights for policy 1, policy_version 329293 (0.0005) [2023-12-27 07:55:28,586][20347] Updated weights for policy 0, policy_version 329486 (0.0010) [2023-12-27 07:55:28,622][20381] Updated weights for policy 1, policy_version 329303 (0.0005) [2023-12-27 07:55:28,638][20347] Updated weights for policy 0, policy_version 329496 (0.0010) [2023-12-27 07:55:29,177][20381] Updated weights for policy 1, policy_version 329313 (0.0006) [2023-12-27 07:55:29,238][20381] Updated weights for policy 1, policy_version 329323 (0.0007) [2023-12-27 07:55:29,297][20381] Updated weights for policy 1, policy_version 329333 (0.0005) [2023-12-27 07:55:29,333][20347] Updated weights for policy 0, policy_version 329506 (0.0010) [2023-12-27 07:55:29,389][20347] Updated weights for policy 0, policy_version 329516 (0.0008) [2023-12-27 07:55:29,442][20347] Updated weights for policy 0, policy_version 329526 (0.0008) [2023-12-27 07:55:29,493][20347] Updated weights for policy 0, policy_version 329536 (0.0005) [2023-12-27 07:55:29,959][20381] Updated weights for policy 1, policy_version 329343 (0.0008) [2023-12-27 07:55:30,009][20381] Updated weights for policy 1, policy_version 329353 (0.0007) [2023-12-27 07:55:30,068][20381] Updated weights for policy 1, policy_version 329363 (0.0006) [2023-12-27 07:55:30,161][20347] Updated weights for policy 0, policy_version 329546 (0.0010) [2023-12-27 07:55:30,207][20347] Updated weights for policy 0, policy_version 329556 (0.0010) [2023-12-27 07:55:30,252][20347] Updated weights for policy 0, policy_version 329566 (0.0010) [2023-12-27 07:55:30,697][20381] Updated weights for policy 1, policy_version 329373 (0.0006) [2023-12-27 07:55:30,757][20381] Updated weights for policy 1, policy_version 329383 (0.0008) [2023-12-27 07:55:30,818][20381] Updated weights for policy 1, policy_version 329393 (0.0007) [2023-12-27 07:55:30,919][20347] Updated weights for policy 0, policy_version 329576 (0.0010) [2023-12-27 07:55:30,966][20347] Updated weights for policy 0, policy_version 329586 (0.0010) [2023-12-27 07:55:31,018][20347] Updated weights for policy 0, policy_version 329596 (0.0010) [2023-12-27 07:55:31,531][20381] Updated weights for policy 1, policy_version 329403 (0.0008) [2023-12-27 07:55:31,588][20381] Updated weights for policy 1, policy_version 329413 (0.0008) [2023-12-27 07:55:31,647][20381] Updated weights for policy 1, policy_version 329423 (0.0008) [2023-12-27 07:55:31,724][20347] Updated weights for policy 0, policy_version 329606 (0.0007) [2023-12-27 07:55:31,776][20347] Updated weights for policy 0, policy_version 329616 (0.0007) [2023-12-27 07:55:31,832][20347] Updated weights for policy 0, policy_version 329626 (0.0008) [2023-12-27 07:55:32,305][20381] Updated weights for policy 1, policy_version 329433 (0.0008) [2023-12-27 07:55:32,358][20381] Updated weights for policy 1, policy_version 329443 (0.0011) [2023-12-27 07:55:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 168755200. Throughput: 0: 10260.5, 1: 10582.3. Samples: 168769528. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:32,363][19170] Avg episode reward: [(0, '748.060'), (1, '922.580')] [2023-12-27 07:55:32,419][20381] Updated weights for policy 1, policy_version 329453 (0.0011) [2023-12-27 07:55:32,473][20381] Updated weights for policy 1, policy_version 329463 (0.0007) [2023-12-27 07:55:32,559][20347] Updated weights for policy 0, policy_version 329636 (0.0007) [2023-12-27 07:55:32,618][20347] Updated weights for policy 0, policy_version 329646 (0.0007) [2023-12-27 07:55:32,671][20347] Updated weights for policy 0, policy_version 329656 (0.0007) [2023-12-27 07:55:33,165][20381] Updated weights for policy 1, policy_version 329473 (0.0010) [2023-12-27 07:55:33,216][20381] Updated weights for policy 1, policy_version 329483 (0.0010) [2023-12-27 07:55:33,267][20381] Updated weights for policy 1, policy_version 329493 (0.0009) [2023-12-27 07:55:33,376][20347] Updated weights for policy 0, policy_version 329666 (0.0008) [2023-12-27 07:55:33,432][20347] Updated weights for policy 0, policy_version 329676 (0.0007) [2023-12-27 07:55:33,483][20347] Updated weights for policy 0, policy_version 329686 (0.0006) [2023-12-27 07:55:33,531][20347] Updated weights for policy 0, policy_version 329696 (0.0007) [2023-12-27 07:55:33,922][20381] Updated weights for policy 1, policy_version 329503 (0.0009) [2023-12-27 07:55:33,966][20381] Updated weights for policy 1, policy_version 329513 (0.0010) [2023-12-27 07:55:34,016][20381] Updated weights for policy 1, policy_version 329523 (0.0010) [2023-12-27 07:55:34,210][20347] Updated weights for policy 0, policy_version 329706 (0.0008) [2023-12-27 07:55:34,268][20347] Updated weights for policy 0, policy_version 329716 (0.0008) [2023-12-27 07:55:34,333][20347] Updated weights for policy 0, policy_version 329726 (0.0008) [2023-12-27 07:55:34,744][20381] Updated weights for policy 1, policy_version 329533 (0.0010) [2023-12-27 07:55:34,796][20381] Updated weights for policy 1, policy_version 329543 (0.0010) [2023-12-27 07:55:34,862][20381] Updated weights for policy 1, policy_version 329553 (0.0010) [2023-12-27 07:55:35,024][20347] Updated weights for policy 0, policy_version 329736 (0.0007) [2023-12-27 07:55:35,074][20347] Updated weights for policy 0, policy_version 329746 (0.0008) [2023-12-27 07:55:35,127][20347] Updated weights for policy 0, policy_version 329756 (0.0009) [2023-12-27 07:55:35,441][20381] Updated weights for policy 1, policy_version 329563 (0.0009) [2023-12-27 07:55:35,493][20381] Updated weights for policy 1, policy_version 329573 (0.0005) [2023-12-27 07:55:35,545][20381] Updated weights for policy 1, policy_version 329583 (0.0005) [2023-12-27 07:55:35,799][20347] Updated weights for policy 0, policy_version 329766 (0.0008) [2023-12-27 07:55:35,852][20347] Updated weights for policy 0, policy_version 329776 (0.0008) [2023-12-27 07:55:35,904][20347] Updated weights for policy 0, policy_version 329786 (0.0007) [2023-12-27 07:55:36,092][20381] Updated weights for policy 1, policy_version 329593 (0.0006) [2023-12-27 07:55:36,155][20381] Updated weights for policy 1, policy_version 329603 (0.0011) [2023-12-27 07:55:36,214][20381] Updated weights for policy 1, policy_version 329613 (0.0011) [2023-12-27 07:55:36,273][20381] Updated weights for policy 1, policy_version 329623 (0.0010) [2023-12-27 07:55:36,608][20347] Updated weights for policy 0, policy_version 329796 (0.0006) [2023-12-27 07:55:36,666][20347] Updated weights for policy 0, policy_version 329806 (0.0007) [2023-12-27 07:55:36,722][20347] Updated weights for policy 0, policy_version 329816 (0.0008) [2023-12-27 07:55:36,953][20381] Updated weights for policy 1, policy_version 329633 (0.0011) [2023-12-27 07:55:37,009][20381] Updated weights for policy 1, policy_version 329643 (0.0009) [2023-12-27 07:55:37,060][20381] Updated weights for policy 1, policy_version 329653 (0.0010) [2023-12-27 07:55:37,358][20347] Updated weights for policy 0, policy_version 329826 (0.0007) [2023-12-27 07:55:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 168861696. Throughput: 0: 10221.4, 1: 10607.4. Samples: 168830616. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:37,363][19170] Avg episode reward: [(0, '757.370'), (1, '912.970')] [2023-12-27 07:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000329656_84410368.pth... [2023-12-27 07:55:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000328408_84090880.pth [2023-12-27 07:55:37,413][20347] Updated weights for policy 0, policy_version 329836 (0.0007) [2023-12-27 07:55:37,472][20347] Updated weights for policy 0, policy_version 329846 (0.0005) [2023-12-27 07:55:37,521][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000329856_84459520.pth... [2023-12-27 07:55:37,523][20347] Updated weights for policy 0, policy_version 329856 (0.0005) [2023-12-27 07:55:37,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000328640_84148224.pth [2023-12-27 07:55:37,653][20381] Updated weights for policy 1, policy_version 329663 (0.0008) [2023-12-27 07:55:37,714][20381] Updated weights for policy 1, policy_version 329673 (0.0007) [2023-12-27 07:55:37,762][20381] Updated weights for policy 1, policy_version 329683 (0.0009) [2023-12-27 07:55:38,042][20347] Updated weights for policy 0, policy_version 329866 (0.0005) [2023-12-27 07:55:38,096][20347] Updated weights for policy 0, policy_version 329876 (0.0005) [2023-12-27 07:55:38,155][20347] Updated weights for policy 0, policy_version 329886 (0.0005) [2023-12-27 07:55:38,426][20381] Updated weights for policy 1, policy_version 329693 (0.0007) [2023-12-27 07:55:38,490][20381] Updated weights for policy 1, policy_version 329703 (0.0010) [2023-12-27 07:55:38,512][20292] KL-divergence is very high: 103.1190 [2023-12-27 07:55:38,550][20381] Updated weights for policy 1, policy_version 329713 (0.0011) [2023-12-27 07:55:38,735][20347] Updated weights for policy 0, policy_version 329896 (0.0006) [2023-12-27 07:55:38,795][20347] Updated weights for policy 0, policy_version 329906 (0.0005) [2023-12-27 07:55:38,842][20347] Updated weights for policy 0, policy_version 329916 (0.0007) [2023-12-27 07:55:39,187][20292] KL-divergence is very high: 116.3633 [2023-12-27 07:55:39,192][20381] Updated weights for policy 1, policy_version 329723 (0.0009) [2023-12-27 07:55:39,253][20381] Updated weights for policy 1, policy_version 329733 (0.0009) [2023-12-27 07:55:39,316][20381] Updated weights for policy 1, policy_version 329743 (0.0011) [2023-12-27 07:55:39,437][20347] Updated weights for policy 0, policy_version 329926 (0.0007) [2023-12-27 07:55:39,503][20347] Updated weights for policy 0, policy_version 329936 (0.0008) [2023-12-27 07:55:39,563][20347] Updated weights for policy 0, policy_version 329946 (0.0008) [2023-12-27 07:55:40,017][20381] Updated weights for policy 1, policy_version 329753 (0.0011) [2023-12-27 07:55:40,074][20381] Updated weights for policy 1, policy_version 329763 (0.0011) [2023-12-27 07:55:40,133][20381] Updated weights for policy 1, policy_version 329773 (0.0011) [2023-12-27 07:55:40,192][20381] Updated weights for policy 1, policy_version 329783 (0.0011) [2023-12-27 07:55:40,233][20347] Updated weights for policy 0, policy_version 329956 (0.0008) [2023-12-27 07:55:40,274][20291] KL-divergence is very high: 102.1597 [2023-12-27 07:55:40,285][20347] Updated weights for policy 0, policy_version 329966 (0.0008) [2023-12-27 07:55:40,294][20291] KL-divergence is very high: 125.9856 [2023-12-27 07:55:40,339][20347] Updated weights for policy 0, policy_version 329976 (0.0007) [2023-12-27 07:55:40,835][20381] Updated weights for policy 1, policy_version 329793 (0.0007) [2023-12-27 07:55:40,890][20381] Updated weights for policy 1, policy_version 329803 (0.0008) [2023-12-27 07:55:40,952][20381] Updated weights for policy 1, policy_version 329813 (0.0010) [2023-12-27 07:55:41,037][20347] Updated weights for policy 0, policy_version 329986 (0.0008) [2023-12-27 07:55:41,102][20347] Updated weights for policy 0, policy_version 329996 (0.0008) [2023-12-27 07:55:41,166][20347] Updated weights for policy 0, policy_version 330006 (0.0008) [2023-12-27 07:55:41,226][20347] Updated weights for policy 0, policy_version 330016 (0.0007) [2023-12-27 07:55:41,644][20381] Updated weights for policy 1, policy_version 329823 (0.0008) [2023-12-27 07:55:41,708][20381] Updated weights for policy 1, policy_version 329833 (0.0010) [2023-12-27 07:55:41,768][20381] Updated weights for policy 1, policy_version 329843 (0.0006) [2023-12-27 07:55:41,790][20292] KL-divergence is very high: 744.3557 [2023-12-27 07:55:42,011][20347] Updated weights for policy 0, policy_version 330026 (0.0008) [2023-12-27 07:55:42,067][20347] Updated weights for policy 0, policy_version 330036 (0.0008) [2023-12-27 07:55:42,122][20347] Updated weights for policy 0, policy_version 330046 (0.0009) [2023-12-27 07:55:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 168968192. Throughput: 0: 10349.3, 1: 10613.2. Samples: 168959804. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:42,363][19170] Avg episode reward: [(0, '747.660'), (1, '835.730')] [2023-12-27 07:55:42,403][20381] Updated weights for policy 1, policy_version 329853 (0.0008) [2023-12-27 07:55:42,460][20381] Updated weights for policy 1, policy_version 329863 (0.0008) [2023-12-27 07:55:42,521][20381] Updated weights for policy 1, policy_version 329873 (0.0008) [2023-12-27 07:55:42,833][20347] Updated weights for policy 0, policy_version 330056 (0.0006) [2023-12-27 07:55:42,891][20347] Updated weights for policy 0, policy_version 330066 (0.0006) [2023-12-27 07:55:42,946][20347] Updated weights for policy 0, policy_version 330076 (0.0005) [2023-12-27 07:55:43,287][20381] Updated weights for policy 1, policy_version 329883 (0.0008) [2023-12-27 07:55:43,342][20381] Updated weights for policy 1, policy_version 329893 (0.0008) [2023-12-27 07:55:43,391][20381] Updated weights for policy 1, policy_version 329903 (0.0008) [2023-12-27 07:55:43,480][20347] Updated weights for policy 0, policy_version 330086 (0.0007) [2023-12-27 07:55:43,527][20347] Updated weights for policy 0, policy_version 330096 (0.0007) [2023-12-27 07:55:43,576][20347] Updated weights for policy 0, policy_version 330106 (0.0007) [2023-12-27 07:55:43,988][20381] Updated weights for policy 1, policy_version 329913 (0.0008) [2023-12-27 07:55:44,052][20381] Updated weights for policy 1, policy_version 329923 (0.0009) [2023-12-27 07:55:44,121][20381] Updated weights for policy 1, policy_version 329933 (0.0009) [2023-12-27 07:55:44,179][20381] Updated weights for policy 1, policy_version 329943 (0.0009) [2023-12-27 07:55:44,326][20347] Updated weights for policy 0, policy_version 330116 (0.0009) [2023-12-27 07:55:44,385][20347] Updated weights for policy 0, policy_version 330126 (0.0011) [2023-12-27 07:55:44,445][20347] Updated weights for policy 0, policy_version 330136 (0.0011) [2023-12-27 07:55:44,836][20381] Updated weights for policy 1, policy_version 329953 (0.0008) [2023-12-27 07:55:44,887][20381] Updated weights for policy 1, policy_version 329963 (0.0010) [2023-12-27 07:55:44,938][20381] Updated weights for policy 1, policy_version 329973 (0.0011) [2023-12-27 07:55:45,156][20347] Updated weights for policy 0, policy_version 330146 (0.0010) [2023-12-27 07:55:45,209][20347] Updated weights for policy 0, policy_version 330156 (0.0007) [2023-12-27 07:55:45,266][20347] Updated weights for policy 0, policy_version 330166 (0.0007) [2023-12-27 07:55:45,325][20347] Updated weights for policy 0, policy_version 330176 (0.0006) [2023-12-27 07:55:45,667][20381] Updated weights for policy 1, policy_version 329983 (0.0011) [2023-12-27 07:55:45,722][20381] Updated weights for policy 1, policy_version 329993 (0.0011) [2023-12-27 07:55:45,770][20381] Updated weights for policy 1, policy_version 330003 (0.0010) [2023-12-27 07:55:45,931][20347] Updated weights for policy 0, policy_version 330186 (0.0005) [2023-12-27 07:55:45,994][20347] Updated weights for policy 0, policy_version 330196 (0.0007) [2023-12-27 07:55:46,054][20347] Updated weights for policy 0, policy_version 330206 (0.0008) [2023-12-27 07:55:46,448][20381] Updated weights for policy 1, policy_version 330013 (0.0010) [2023-12-27 07:55:46,496][20381] Updated weights for policy 1, policy_version 330023 (0.0010) [2023-12-27 07:55:46,542][20381] Updated weights for policy 1, policy_version 330033 (0.0011) [2023-12-27 07:55:46,585][20347] Updated weights for policy 0, policy_version 330216 (0.0006) [2023-12-27 07:55:46,643][20347] Updated weights for policy 0, policy_version 330226 (0.0006) [2023-12-27 07:55:46,696][20347] Updated weights for policy 0, policy_version 330236 (0.0007) [2023-12-27 07:55:47,239][20381] Updated weights for policy 1, policy_version 330043 (0.0011) [2023-12-27 07:55:47,294][20381] Updated weights for policy 1, policy_version 330053 (0.0010) [2023-12-27 07:55:47,322][20347] Updated weights for policy 0, policy_version 330246 (0.0007) [2023-12-27 07:55:47,345][20381] Updated weights for policy 1, policy_version 330063 (0.0010) [2023-12-27 07:55:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 169066496. Throughput: 0: 10396.8, 1: 10540.0. Samples: 169084276. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:47,363][19170] Avg episode reward: [(0, '757.720'), (1, '903.650')] [2023-12-27 07:55:47,373][20347] Updated weights for policy 0, policy_version 330256 (0.0006) [2023-12-27 07:55:47,422][20347] Updated weights for policy 0, policy_version 330266 (0.0006) [2023-12-27 07:55:47,961][20381] Updated weights for policy 1, policy_version 330073 (0.0010) [2023-12-27 07:55:48,019][20347] Updated weights for policy 0, policy_version 330276 (0.0006) [2023-12-27 07:55:48,022][20381] Updated weights for policy 1, policy_version 330083 (0.0006) [2023-12-27 07:55:48,075][20347] Updated weights for policy 0, policy_version 330286 (0.0006) [2023-12-27 07:55:48,081][20381] Updated weights for policy 1, policy_version 330093 (0.0005) [2023-12-27 07:55:48,131][20347] Updated weights for policy 0, policy_version 330296 (0.0008) [2023-12-27 07:55:48,141][20381] Updated weights for policy 1, policy_version 330103 (0.0005) [2023-12-27 07:55:48,706][20381] Updated weights for policy 1, policy_version 330113 (0.0008) [2023-12-27 07:55:48,766][20381] Updated weights for policy 1, policy_version 330123 (0.0007) [2023-12-27 07:55:48,826][20381] Updated weights for policy 1, policy_version 330133 (0.0006) [2023-12-27 07:55:48,833][20347] Updated weights for policy 0, policy_version 330306 (0.0007) [2023-12-27 07:55:48,895][20347] Updated weights for policy 0, policy_version 330316 (0.0006) [2023-12-27 07:55:48,956][20347] Updated weights for policy 0, policy_version 330326 (0.0007) [2023-12-27 07:55:49,015][20347] Updated weights for policy 0, policy_version 330336 (0.0007) [2023-12-27 07:55:49,517][20381] Updated weights for policy 1, policy_version 330143 (0.0007) [2023-12-27 07:55:49,573][20381] Updated weights for policy 1, policy_version 330153 (0.0008) [2023-12-27 07:55:49,633][20381] Updated weights for policy 1, policy_version 330163 (0.0007) [2023-12-27 07:55:49,711][20347] Updated weights for policy 0, policy_version 330346 (0.0010) [2023-12-27 07:55:49,769][20347] Updated weights for policy 0, policy_version 330356 (0.0010) [2023-12-27 07:55:49,832][20347] Updated weights for policy 0, policy_version 330366 (0.0008) [2023-12-27 07:55:50,311][20381] Updated weights for policy 1, policy_version 330173 (0.0008) [2023-12-27 07:55:50,376][20381] Updated weights for policy 1, policy_version 330183 (0.0007) [2023-12-27 07:55:50,433][20381] Updated weights for policy 1, policy_version 330193 (0.0007) [2023-12-27 07:55:50,496][20347] Updated weights for policy 0, policy_version 330376 (0.0010) [2023-12-27 07:55:50,542][20347] Updated weights for policy 0, policy_version 330386 (0.0010) [2023-12-27 07:55:50,598][20347] Updated weights for policy 0, policy_version 330396 (0.0009) [2023-12-27 07:55:51,150][20381] Updated weights for policy 1, policy_version 330203 (0.0007) [2023-12-27 07:55:51,211][20381] Updated weights for policy 1, policy_version 330213 (0.0008) [2023-12-27 07:55:51,270][20381] Updated weights for policy 1, policy_version 330223 (0.0008) [2023-12-27 07:55:51,315][20347] Updated weights for policy 0, policy_version 330406 (0.0007) [2023-12-27 07:55:51,382][20347] Updated weights for policy 0, policy_version 330416 (0.0008) [2023-12-27 07:55:51,441][20347] Updated weights for policy 0, policy_version 330426 (0.0007) [2023-12-27 07:55:51,966][20381] Updated weights for policy 1, policy_version 330233 (0.0006) [2023-12-27 07:55:52,019][20381] Updated weights for policy 1, policy_version 330243 (0.0010) [2023-12-27 07:55:52,073][20381] Updated weights for policy 1, policy_version 330253 (0.0007) [2023-12-27 07:55:52,081][20347] Updated weights for policy 0, policy_version 330436 (0.0007) [2023-12-27 07:55:52,125][20381] Updated weights for policy 1, policy_version 330263 (0.0006) [2023-12-27 07:55:52,135][20347] Updated weights for policy 0, policy_version 330446 (0.0006) [2023-12-27 07:55:52,180][20347] Updated weights for policy 0, policy_version 330456 (0.0007) [2023-12-27 07:55:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 169181184. Throughput: 0: 10400.7, 1: 10554.7. Samples: 169147316. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:52,363][19170] Avg episode reward: [(0, '738.110'), (1, '980.900')] [2023-12-27 07:55:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000330264_84566016.pth... [2023-12-27 07:55:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000330464_84615168.pth... [2023-12-27 07:55:52,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000329016_84246528.pth [2023-12-27 07:55:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000329248_84303872.pth [2023-12-27 07:55:52,826][20347] Updated weights for policy 0, policy_version 330466 (0.0007) [2023-12-27 07:55:52,863][20381] Updated weights for policy 1, policy_version 330273 (0.0007) [2023-12-27 07:55:52,875][20347] Updated weights for policy 0, policy_version 330476 (0.0005) [2023-12-27 07:55:52,915][20381] Updated weights for policy 1, policy_version 330283 (0.0008) [2023-12-27 07:55:52,927][20347] Updated weights for policy 0, policy_version 330486 (0.0005) [2023-12-27 07:55:52,977][20347] Updated weights for policy 0, policy_version 330496 (0.0005) [2023-12-27 07:55:52,978][20381] Updated weights for policy 1, policy_version 330293 (0.0008) [2023-12-27 07:55:53,617][20347] Updated weights for policy 0, policy_version 330506 (0.0007) [2023-12-27 07:55:53,649][20381] Updated weights for policy 1, policy_version 330303 (0.0006) [2023-12-27 07:55:53,674][20347] Updated weights for policy 0, policy_version 330516 (0.0005) [2023-12-27 07:55:53,707][20381] Updated weights for policy 1, policy_version 330313 (0.0008) [2023-12-27 07:55:53,723][20347] Updated weights for policy 0, policy_version 330526 (0.0008) [2023-12-27 07:55:53,766][20381] Updated weights for policy 1, policy_version 330323 (0.0007) [2023-12-27 07:55:54,295][20347] Updated weights for policy 0, policy_version 330536 (0.0006) [2023-12-27 07:55:54,357][20347] Updated weights for policy 0, policy_version 330546 (0.0008) [2023-12-27 07:55:54,412][20347] Updated weights for policy 0, policy_version 330556 (0.0008) [2023-12-27 07:55:54,493][20381] Updated weights for policy 1, policy_version 330333 (0.0008) [2023-12-27 07:55:54,551][20381] Updated weights for policy 1, policy_version 330343 (0.0007) [2023-12-27 07:55:54,609][20381] Updated weights for policy 1, policy_version 330353 (0.0008) [2023-12-27 07:55:55,051][20347] Updated weights for policy 0, policy_version 330566 (0.0006) [2023-12-27 07:55:55,113][20347] Updated weights for policy 0, policy_version 330576 (0.0007) [2023-12-27 07:55:55,165][20347] Updated weights for policy 0, policy_version 330586 (0.0009) [2023-12-27 07:55:55,265][20381] Updated weights for policy 1, policy_version 330363 (0.0008) [2023-12-27 07:55:55,331][20381] Updated weights for policy 1, policy_version 330373 (0.0008) [2023-12-27 07:55:55,394][20381] Updated weights for policy 1, policy_version 330383 (0.0007) [2023-12-27 07:55:55,833][20347] Updated weights for policy 0, policy_version 330596 (0.0007) [2023-12-27 07:55:55,888][20347] Updated weights for policy 0, policy_version 330606 (0.0007) [2023-12-27 07:55:55,943][20347] Updated weights for policy 0, policy_version 330616 (0.0008) [2023-12-27 07:55:56,016][20381] Updated weights for policy 1, policy_version 330393 (0.0008) [2023-12-27 07:55:56,077][20381] Updated weights for policy 1, policy_version 330403 (0.0007) [2023-12-27 07:55:56,134][20381] Updated weights for policy 1, policy_version 330413 (0.0008) [2023-12-27 07:55:56,193][20381] Updated weights for policy 1, policy_version 330423 (0.0009) [2023-12-27 07:55:56,600][20347] Updated weights for policy 0, policy_version 330626 (0.0007) [2023-12-27 07:55:56,657][20347] Updated weights for policy 0, policy_version 330636 (0.0007) [2023-12-27 07:55:56,721][20347] Updated weights for policy 0, policy_version 330646 (0.0008) [2023-12-27 07:55:56,779][20347] Updated weights for policy 0, policy_version 330656 (0.0007) [2023-12-27 07:55:56,846][20381] Updated weights for policy 1, policy_version 330433 (0.0007) [2023-12-27 07:55:56,897][20381] Updated weights for policy 1, policy_version 330443 (0.0009) [2023-12-27 07:55:56,961][20381] Updated weights for policy 1, policy_version 330453 (0.0010) [2023-12-27 07:55:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 169279488. Throughput: 0: 10469.1, 1: 10551.1. Samples: 169271028. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:55:57,363][19170] Avg episode reward: [(0, '796.300'), (1, '990.390')] [2023-12-27 07:55:57,472][20347] Updated weights for policy 0, policy_version 330666 (0.0005) [2023-12-27 07:55:57,523][20347] Updated weights for policy 0, policy_version 330676 (0.0006) [2023-12-27 07:55:57,576][20347] Updated weights for policy 0, policy_version 330686 (0.0007) [2023-12-27 07:55:57,611][20381] Updated weights for policy 1, policy_version 330463 (0.0010) [2023-12-27 07:55:57,659][20381] Updated weights for policy 1, policy_version 330473 (0.0010) [2023-12-27 07:55:57,707][20381] Updated weights for policy 1, policy_version 330483 (0.0010) [2023-12-27 07:55:58,219][20347] Updated weights for policy 0, policy_version 330696 (0.0009) [2023-12-27 07:55:58,280][20347] Updated weights for policy 0, policy_version 330706 (0.0010) [2023-12-27 07:55:58,284][20381] Updated weights for policy 1, policy_version 330493 (0.0007) [2023-12-27 07:55:58,348][20347] Updated weights for policy 0, policy_version 330716 (0.0009) [2023-12-27 07:55:58,351][20381] Updated weights for policy 1, policy_version 330503 (0.0009) [2023-12-27 07:55:58,411][20381] Updated weights for policy 1, policy_version 330513 (0.0009) [2023-12-27 07:55:59,103][20347] Updated weights for policy 0, policy_version 330726 (0.0007) [2023-12-27 07:55:59,139][20381] Updated weights for policy 1, policy_version 330523 (0.0011) [2023-12-27 07:55:59,155][20347] Updated weights for policy 0, policy_version 330736 (0.0005) [2023-12-27 07:55:59,195][20381] Updated weights for policy 1, policy_version 330533 (0.0010) [2023-12-27 07:55:59,207][20347] Updated weights for policy 0, policy_version 330746 (0.0005) [2023-12-27 07:55:59,254][20381] Updated weights for policy 1, policy_version 330543 (0.0009) [2023-12-27 07:55:59,768][20347] Updated weights for policy 0, policy_version 330756 (0.0007) [2023-12-27 07:55:59,822][20347] Updated weights for policy 0, policy_version 330766 (0.0007) [2023-12-27 07:55:59,881][20347] Updated weights for policy 0, policy_version 330776 (0.0008) [2023-12-27 07:55:59,939][20381] Updated weights for policy 1, policy_version 330553 (0.0008) [2023-12-27 07:55:59,997][20381] Updated weights for policy 1, policy_version 330563 (0.0009) [2023-12-27 07:56:00,053][20381] Updated weights for policy 1, policy_version 330573 (0.0011) [2023-12-27 07:56:00,108][20381] Updated weights for policy 1, policy_version 330583 (0.0007) [2023-12-27 07:56:00,514][20347] Updated weights for policy 0, policy_version 330786 (0.0008) [2023-12-27 07:56:00,561][20347] Updated weights for policy 0, policy_version 330796 (0.0007) [2023-12-27 07:56:00,617][20347] Updated weights for policy 0, policy_version 330806 (0.0008) [2023-12-27 07:56:00,669][20347] Updated weights for policy 0, policy_version 330816 (0.0007) [2023-12-27 07:56:00,708][20381] Updated weights for policy 1, policy_version 330593 (0.0008) [2023-12-27 07:56:00,755][20381] Updated weights for policy 1, policy_version 330603 (0.0007) [2023-12-27 07:56:00,801][20381] Updated weights for policy 1, policy_version 330613 (0.0006) [2023-12-27 07:56:01,407][20347] Updated weights for policy 0, policy_version 330826 (0.0007) [2023-12-27 07:56:01,462][20347] Updated weights for policy 0, policy_version 330836 (0.0005) [2023-12-27 07:56:01,468][20381] Updated weights for policy 1, policy_version 330623 (0.0005) [2023-12-27 07:56:01,521][20347] Updated weights for policy 0, policy_version 330846 (0.0005) [2023-12-27 07:56:01,527][20381] Updated weights for policy 1, policy_version 330633 (0.0006) [2023-12-27 07:56:01,585][20381] Updated weights for policy 1, policy_version 330643 (0.0009) [2023-12-27 07:56:02,157][20347] Updated weights for policy 0, policy_version 330856 (0.0007) [2023-12-27 07:56:02,214][20347] Updated weights for policy 0, policy_version 330866 (0.0008) [2023-12-27 07:56:02,238][20381] Updated weights for policy 1, policy_version 330653 (0.0009) [2023-12-27 07:56:02,276][20347] Updated weights for policy 0, policy_version 330876 (0.0006) [2023-12-27 07:56:02,293][20381] Updated weights for policy 1, policy_version 330663 (0.0007) [2023-12-27 07:56:02,351][20381] Updated weights for policy 1, policy_version 330673 (0.0008) [2023-12-27 07:56:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 169385984. Throughput: 0: 10480.1, 1: 10525.3. Samples: 169397136. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:56:02,363][19170] Avg episode reward: [(0, '834.900'), (1, '990.390')] [2023-12-27 07:56:02,952][20381] Updated weights for policy 1, policy_version 330683 (0.0007) [2023-12-27 07:56:03,004][20381] Updated weights for policy 1, policy_version 330693 (0.0009) [2023-12-27 07:56:03,032][20347] Updated weights for policy 0, policy_version 330886 (0.0007) [2023-12-27 07:56:03,052][20381] Updated weights for policy 1, policy_version 330703 (0.0010) [2023-12-27 07:56:03,081][20347] Updated weights for policy 0, policy_version 330896 (0.0005) [2023-12-27 07:56:03,136][20347] Updated weights for policy 0, policy_version 330906 (0.0007) [2023-12-27 07:56:03,757][20381] Updated weights for policy 1, policy_version 330714 (0.0010) [2023-12-27 07:56:03,816][20381] Updated weights for policy 1, policy_version 330724 (0.0011) [2023-12-27 07:56:03,850][20347] Updated weights for policy 0, policy_version 330916 (0.0007) [2023-12-27 07:56:03,881][20381] Updated weights for policy 1, policy_version 330734 (0.0008) [2023-12-27 07:56:03,906][20347] Updated weights for policy 0, policy_version 330926 (0.0009) [2023-12-27 07:56:03,939][20381] Updated weights for policy 1, policy_version 330744 (0.0008) [2023-12-27 07:56:03,966][20347] Updated weights for policy 0, policy_version 330936 (0.0007) [2023-12-27 07:56:04,620][20347] Updated weights for policy 0, policy_version 330946 (0.0010) [2023-12-27 07:56:04,626][20381] Updated weights for policy 1, policy_version 330754 (0.0009) [2023-12-27 07:56:04,670][20347] Updated weights for policy 0, policy_version 330956 (0.0006) [2023-12-27 07:56:04,692][20381] Updated weights for policy 1, policy_version 330764 (0.0011) [2023-12-27 07:56:04,722][20347] Updated weights for policy 0, policy_version 330966 (0.0006) [2023-12-27 07:56:04,755][20381] Updated weights for policy 1, policy_version 330774 (0.0011) [2023-12-27 07:56:04,774][20347] Updated weights for policy 0, policy_version 330976 (0.0006) [2023-12-27 07:56:05,422][20347] Updated weights for policy 0, policy_version 330986 (0.0010) [2023-12-27 07:56:05,428][20381] Updated weights for policy 1, policy_version 330784 (0.0010) [2023-12-27 07:56:05,474][20347] Updated weights for policy 0, policy_version 330996 (0.0010) [2023-12-27 07:56:05,478][20381] Updated weights for policy 1, policy_version 330794 (0.0010) [2023-12-27 07:56:05,532][20347] Updated weights for policy 0, policy_version 331006 (0.0010) [2023-12-27 07:56:05,537][20381] Updated weights for policy 1, policy_version 330804 (0.0010) [2023-12-27 07:56:06,205][20347] Updated weights for policy 0, policy_version 331016 (0.0009) [2023-12-27 07:56:06,224][20381] Updated weights for policy 1, policy_version 330814 (0.0007) [2023-12-27 07:56:06,265][20347] Updated weights for policy 0, policy_version 331026 (0.0008) [2023-12-27 07:56:06,282][20381] Updated weights for policy 1, policy_version 330824 (0.0006) [2023-12-27 07:56:06,323][20347] Updated weights for policy 0, policy_version 331036 (0.0008) [2023-12-27 07:56:06,338][20381] Updated weights for policy 1, policy_version 330834 (0.0006) [2023-12-27 07:56:06,979][20347] Updated weights for policy 0, policy_version 331046 (0.0007) [2023-12-27 07:56:07,002][20381] Updated weights for policy 1, policy_version 330844 (0.0007) [2023-12-27 07:56:07,036][20347] Updated weights for policy 0, policy_version 331056 (0.0008) [2023-12-27 07:56:07,049][20381] Updated weights for policy 1, policy_version 330854 (0.0011) [2023-12-27 07:56:07,091][20347] Updated weights for policy 0, policy_version 331066 (0.0007) [2023-12-27 07:56:07,103][20381] Updated weights for policy 1, policy_version 330864 (0.0011) [2023-12-27 07:56:07,363][19170] Fps is (10 sec: 21298.3, 60 sec: 20889.5, 300 sec: 20910.4). Total num frames: 169492480. Throughput: 0: 10436.4, 1: 10466.3. Samples: 169458408. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:56:07,364][19170] Avg episode reward: [(0, '922.420'), (1, '990.640')] [2023-12-27 07:56:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000330872_84721664.pth... [2023-12-27 07:56:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000331072_84770816.pth... [2023-12-27 07:56:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000329656_84410368.pth [2023-12-27 07:56:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000329856_84459520.pth [2023-12-27 07:56:07,688][20347] Updated weights for policy 0, policy_version 331076 (0.0007) [2023-12-27 07:56:07,747][20347] Updated weights for policy 0, policy_version 331086 (0.0007) [2023-12-27 07:56:07,775][20381] Updated weights for policy 1, policy_version 330874 (0.0011) [2023-12-27 07:56:07,803][20347] Updated weights for policy 0, policy_version 331096 (0.0007) [2023-12-27 07:56:07,824][20381] Updated weights for policy 1, policy_version 330884 (0.0010) [2023-12-27 07:56:07,873][20381] Updated weights for policy 1, policy_version 330894 (0.0011) [2023-12-27 07:56:07,919][20381] Updated weights for policy 1, policy_version 330904 (0.0010) [2023-12-27 07:56:08,373][20347] Updated weights for policy 0, policy_version 331106 (0.0006) [2023-12-27 07:56:08,433][20347] Updated weights for policy 0, policy_version 331116 (0.0005) [2023-12-27 07:56:08,493][20347] Updated weights for policy 0, policy_version 331126 (0.0007) [2023-12-27 07:56:08,550][20347] Updated weights for policy 0, policy_version 331136 (0.0007) [2023-12-27 07:56:08,619][20381] Updated weights for policy 1, policy_version 330914 (0.0010) [2023-12-27 07:56:08,665][20381] Updated weights for policy 1, policy_version 330924 (0.0010) [2023-12-27 07:56:08,714][20381] Updated weights for policy 1, policy_version 330934 (0.0010) [2023-12-27 07:56:09,200][20347] Updated weights for policy 0, policy_version 331146 (0.0006) [2023-12-27 07:56:09,266][20347] Updated weights for policy 0, policy_version 331156 (0.0009) [2023-12-27 07:56:09,326][20347] Updated weights for policy 0, policy_version 331166 (0.0006) [2023-12-27 07:56:09,459][20381] Updated weights for policy 1, policy_version 330944 (0.0008) [2023-12-27 07:56:09,520][20381] Updated weights for policy 1, policy_version 330954 (0.0008) [2023-12-27 07:56:09,583][20381] Updated weights for policy 1, policy_version 330964 (0.0008) [2023-12-27 07:56:09,956][20347] Updated weights for policy 0, policy_version 331176 (0.0008) [2023-12-27 07:56:10,012][20347] Updated weights for policy 0, policy_version 331186 (0.0008) [2023-12-27 07:56:10,076][20347] Updated weights for policy 0, policy_version 331196 (0.0006) [2023-12-27 07:56:10,318][20381] Updated weights for policy 1, policy_version 330974 (0.0006) [2023-12-27 07:56:10,379][20381] Updated weights for policy 1, policy_version 330984 (0.0007) [2023-12-27 07:56:10,443][20381] Updated weights for policy 1, policy_version 330994 (0.0008) [2023-12-27 07:56:10,797][20347] Updated weights for policy 0, policy_version 331206 (0.0006) [2023-12-27 07:56:10,859][20347] Updated weights for policy 0, policy_version 331216 (0.0007) [2023-12-27 07:56:10,917][20347] Updated weights for policy 0, policy_version 331226 (0.0006) [2023-12-27 07:56:11,133][20381] Updated weights for policy 1, policy_version 331004 (0.0008) [2023-12-27 07:56:11,199][20381] Updated weights for policy 1, policy_version 331014 (0.0007) [2023-12-27 07:56:11,265][20381] Updated weights for policy 1, policy_version 331024 (0.0006) [2023-12-27 07:56:11,587][20347] Updated weights for policy 0, policy_version 331236 (0.0006) [2023-12-27 07:56:11,647][20347] Updated weights for policy 0, policy_version 331246 (0.0009) [2023-12-27 07:56:11,701][20347] Updated weights for policy 0, policy_version 331256 (0.0007) [2023-12-27 07:56:12,020][20381] Updated weights for policy 1, policy_version 331034 (0.0008) [2023-12-27 07:56:12,077][20381] Updated weights for policy 1, policy_version 331044 (0.0008) [2023-12-27 07:56:12,140][20381] Updated weights for policy 1, policy_version 331054 (0.0008) [2023-12-27 07:56:12,199][20381] Updated weights for policy 1, policy_version 331064 (0.0007) [2023-12-27 07:56:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 169590784. Throughput: 0: 10470.1, 1: 10380.1. Samples: 169582128. Policy #0 lag: (min: 22.0, avg: 27.0, max: 54.0) [2023-12-27 07:56:12,363][19170] Avg episode reward: [(0, '941.890'), (1, '990.640')] [2023-12-27 07:56:12,367][20347] Updated weights for policy 0, policy_version 331266 (0.0007) [2023-12-27 07:56:12,421][20347] Updated weights for policy 0, policy_version 331276 (0.0008) [2023-12-27 07:56:12,476][20347] Updated weights for policy 0, policy_version 331286 (0.0005) [2023-12-27 07:56:12,526][20347] Updated weights for policy 0, policy_version 331296 (0.0008) [2023-12-27 07:56:12,913][20381] Updated weights for policy 1, policy_version 331074 (0.0006) [2023-12-27 07:56:12,968][20381] Updated weights for policy 1, policy_version 331084 (0.0008) [2023-12-27 07:56:13,016][20381] Updated weights for policy 1, policy_version 331094 (0.0007) [2023-12-27 07:56:13,179][20347] Updated weights for policy 0, policy_version 331306 (0.0006) [2023-12-27 07:56:13,232][20347] Updated weights for policy 0, policy_version 331316 (0.0007) [2023-12-27 07:56:13,281][20347] Updated weights for policy 0, policy_version 331326 (0.0008) [2023-12-27 07:56:13,733][20381] Updated weights for policy 1, policy_version 331104 (0.0009) [2023-12-27 07:56:13,778][20381] Updated weights for policy 1, policy_version 331114 (0.0010) [2023-12-27 07:56:13,823][20381] Updated weights for policy 1, policy_version 331124 (0.0008) [2023-12-27 07:56:13,828][20347] Updated weights for policy 0, policy_version 331336 (0.0006) [2023-12-27 07:56:13,877][20347] Updated weights for policy 0, policy_version 331346 (0.0005) [2023-12-27 07:56:13,929][20347] Updated weights for policy 0, policy_version 331356 (0.0005) [2023-12-27 07:56:14,463][20381] Updated weights for policy 1, policy_version 331134 (0.0006) [2023-12-27 07:56:14,534][20381] Updated weights for policy 1, policy_version 331144 (0.0006) [2023-12-27 07:56:14,566][20347] Updated weights for policy 0, policy_version 331366 (0.0006) [2023-12-27 07:56:14,597][20381] Updated weights for policy 1, policy_version 331154 (0.0006) [2023-12-27 07:56:14,626][20347] Updated weights for policy 0, policy_version 331376 (0.0007) [2023-12-27 07:56:14,689][20347] Updated weights for policy 0, policy_version 331386 (0.0008) [2023-12-27 07:56:15,154][20381] Updated weights for policy 1, policy_version 331164 (0.0006) [2023-12-27 07:56:15,214][20381] Updated weights for policy 1, policy_version 331174 (0.0009) [2023-12-27 07:56:15,272][20381] Updated weights for policy 1, policy_version 331184 (0.0008) [2023-12-27 07:56:15,306][20347] Updated weights for policy 0, policy_version 331396 (0.0008) [2023-12-27 07:56:15,366][20347] Updated weights for policy 0, policy_version 331406 (0.0006) [2023-12-27 07:56:15,423][20347] Updated weights for policy 0, policy_version 331416 (0.0007) [2023-12-27 07:56:15,961][20381] Updated weights for policy 1, policy_version 331194 (0.0007) [2023-12-27 07:56:16,011][20381] Updated weights for policy 1, policy_version 331204 (0.0005) [2023-12-27 07:56:16,068][20381] Updated weights for policy 1, policy_version 331214 (0.0005) [2023-12-27 07:56:16,119][20347] Updated weights for policy 0, policy_version 331426 (0.0007) [2023-12-27 07:56:16,128][20381] Updated weights for policy 1, policy_version 331224 (0.0005) [2023-12-27 07:56:16,175][20347] Updated weights for policy 0, policy_version 331436 (0.0006) [2023-12-27 07:56:16,228][20347] Updated weights for policy 0, policy_version 331446 (0.0005) [2023-12-27 07:56:16,285][20347] Updated weights for policy 0, policy_version 331456 (0.0005) [2023-12-27 07:56:16,688][20381] Updated weights for policy 1, policy_version 331234 (0.0009) [2023-12-27 07:56:16,743][20381] Updated weights for policy 1, policy_version 331244 (0.0006) [2023-12-27 07:56:16,800][20381] Updated weights for policy 1, policy_version 331254 (0.0006) [2023-12-27 07:56:16,880][20347] Updated weights for policy 0, policy_version 331466 (0.0009) [2023-12-27 07:56:16,927][20347] Updated weights for policy 0, policy_version 331476 (0.0008) [2023-12-27 07:56:16,987][20347] Updated weights for policy 0, policy_version 331486 (0.0007) [2023-12-27 07:56:17,362][19170] Fps is (10 sec: 20480.9, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 169697280. Throughput: 0: 10523.5, 1: 10376.5. Samples: 169710028. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:17,363][19170] Avg episode reward: [(0, '961.630'), (1, '1000.000')] [2023-12-27 07:56:17,396][20381] Updated weights for policy 1, policy_version 331264 (0.0007) [2023-12-27 07:56:17,450][20381] Updated weights for policy 1, policy_version 331274 (0.0006) [2023-12-27 07:56:17,505][20381] Updated weights for policy 1, policy_version 331284 (0.0005) [2023-12-27 07:56:17,708][20347] Updated weights for policy 0, policy_version 331496 (0.0009) [2023-12-27 07:56:17,762][20347] Updated weights for policy 0, policy_version 331506 (0.0007) [2023-12-27 07:56:17,810][20347] Updated weights for policy 0, policy_version 331516 (0.0009) [2023-12-27 07:56:18,074][20381] Updated weights for policy 1, policy_version 331294 (0.0007) [2023-12-27 07:56:18,125][20381] Updated weights for policy 1, policy_version 331304 (0.0006) [2023-12-27 07:56:18,186][20381] Updated weights for policy 1, policy_version 331314 (0.0008) [2023-12-27 07:56:18,504][20347] Updated weights for policy 0, policy_version 331526 (0.0007) [2023-12-27 07:56:18,561][20347] Updated weights for policy 0, policy_version 331536 (0.0008) [2023-12-27 07:56:18,623][20347] Updated weights for policy 0, policy_version 331546 (0.0008) [2023-12-27 07:56:18,917][20381] Updated weights for policy 1, policy_version 331324 (0.0007) [2023-12-27 07:56:18,966][20381] Updated weights for policy 1, policy_version 331334 (0.0008) [2023-12-27 07:56:19,015][20381] Updated weights for policy 1, policy_version 331344 (0.0008) [2023-12-27 07:56:19,311][20347] Updated weights for policy 0, policy_version 331556 (0.0008) [2023-12-27 07:56:19,375][20347] Updated weights for policy 0, policy_version 331566 (0.0008) [2023-12-27 07:56:19,424][20347] Updated weights for policy 0, policy_version 331576 (0.0007) [2023-12-27 07:56:19,723][20381] Updated weights for policy 1, policy_version 331354 (0.0008) [2023-12-27 07:56:19,787][20381] Updated weights for policy 1, policy_version 331364 (0.0007) [2023-12-27 07:56:19,849][20381] Updated weights for policy 1, policy_version 331374 (0.0009) [2023-12-27 07:56:19,917][20381] Updated weights for policy 1, policy_version 331384 (0.0008) [2023-12-27 07:56:20,171][20347] Updated weights for policy 0, policy_version 331586 (0.0007) [2023-12-27 07:56:20,231][20347] Updated weights for policy 0, policy_version 331596 (0.0007) [2023-12-27 07:56:20,289][20347] Updated weights for policy 0, policy_version 331606 (0.0007) [2023-12-27 07:56:20,349][20347] Updated weights for policy 0, policy_version 331616 (0.0007) [2023-12-27 07:56:20,547][20381] Updated weights for policy 1, policy_version 331394 (0.0008) [2023-12-27 07:56:20,612][20381] Updated weights for policy 1, policy_version 331404 (0.0008) [2023-12-27 07:56:20,666][20381] Updated weights for policy 1, policy_version 331414 (0.0008) [2023-12-27 07:56:21,011][20347] Updated weights for policy 0, policy_version 331626 (0.0005) [2023-12-27 07:56:21,073][20347] Updated weights for policy 0, policy_version 331636 (0.0007) [2023-12-27 07:56:21,129][20347] Updated weights for policy 0, policy_version 331646 (0.0005) [2023-12-27 07:56:21,388][20381] Updated weights for policy 1, policy_version 331424 (0.0008) [2023-12-27 07:56:21,441][20381] Updated weights for policy 1, policy_version 331434 (0.0008) [2023-12-27 07:56:21,502][20381] Updated weights for policy 1, policy_version 331444 (0.0008) [2023-12-27 07:56:21,746][20347] Updated weights for policy 0, policy_version 331656 (0.0008) [2023-12-27 07:56:21,801][20347] Updated weights for policy 0, policy_version 331666 (0.0008) [2023-12-27 07:56:21,847][20347] Updated weights for policy 0, policy_version 331676 (0.0007) [2023-12-27 07:56:22,177][20381] Updated weights for policy 1, policy_version 331454 (0.0008) [2023-12-27 07:56:22,235][20381] Updated weights for policy 1, policy_version 331464 (0.0007) [2023-12-27 07:56:22,303][20381] Updated weights for policy 1, policy_version 331474 (0.0008) [2023-12-27 07:56:22,362][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20882.6). Total num frames: 169803776. Throughput: 0: 10513.5, 1: 10387.0. Samples: 169771140. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:22,363][19170] Avg episode reward: [(0, '951.970'), (1, '1000.000')] [2023-12-27 07:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000331480_84877312.pth... [2023-12-27 07:56:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000331680_84926464.pth... [2023-12-27 07:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000330264_84566016.pth [2023-12-27 07:56:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000330464_84615168.pth [2023-12-27 07:56:22,569][20347] Updated weights for policy 0, policy_version 331686 (0.0007) [2023-12-27 07:56:22,622][20347] Updated weights for policy 0, policy_version 331696 (0.0008) [2023-12-27 07:56:22,674][20347] Updated weights for policy 0, policy_version 331706 (0.0008) [2023-12-27 07:56:22,923][20381] Updated weights for policy 1, policy_version 331484 (0.0008) [2023-12-27 07:56:22,985][20381] Updated weights for policy 1, policy_version 331494 (0.0007) [2023-12-27 07:56:23,043][20381] Updated weights for policy 1, policy_version 331504 (0.0008) [2023-12-27 07:56:23,378][20347] Updated weights for policy 0, policy_version 331716 (0.0007) [2023-12-27 07:56:23,435][20347] Updated weights for policy 0, policy_version 331726 (0.0005) [2023-12-27 07:56:23,490][20347] Updated weights for policy 0, policy_version 331736 (0.0005) [2023-12-27 07:56:23,793][20381] Updated weights for policy 1, policy_version 331514 (0.0008) [2023-12-27 07:56:23,851][20381] Updated weights for policy 1, policy_version 331524 (0.0008) [2023-12-27 07:56:23,903][20381] Updated weights for policy 1, policy_version 331534 (0.0008) [2023-12-27 07:56:23,956][20381] Updated weights for policy 1, policy_version 331544 (0.0008) [2023-12-27 07:56:24,039][20347] Updated weights for policy 0, policy_version 331746 (0.0006) [2023-12-27 07:56:24,087][20347] Updated weights for policy 0, policy_version 331756 (0.0008) [2023-12-27 07:56:24,138][20347] Updated weights for policy 0, policy_version 331766 (0.0007) [2023-12-27 07:56:24,190][20347] Updated weights for policy 0, policy_version 331776 (0.0008) [2023-12-27 07:56:24,668][20381] Updated weights for policy 1, policy_version 331554 (0.0008) [2023-12-27 07:56:24,717][20381] Updated weights for policy 1, policy_version 331564 (0.0009) [2023-12-27 07:56:24,763][20381] Updated weights for policy 1, policy_version 331574 (0.0005) [2023-12-27 07:56:24,871][20347] Updated weights for policy 0, policy_version 331786 (0.0005) [2023-12-27 07:56:24,932][20347] Updated weights for policy 0, policy_version 331796 (0.0005) [2023-12-27 07:56:24,956][20291] KL-divergence is very high: 974.6180 [2023-12-27 07:56:24,979][20291] KL-divergence is very high: 2695.2402 [2023-12-27 07:56:24,984][20291] KL-divergence is very high: 176.2290 [2023-12-27 07:56:24,990][20347] Updated weights for policy 0, policy_version 331806 (0.0006) [2023-12-27 07:56:24,997][20291] KL-divergence is very high: 157.6159 [2023-12-27 07:56:25,396][20381] Updated weights for policy 1, policy_version 331584 (0.0006) [2023-12-27 07:56:25,453][20381] Updated weights for policy 1, policy_version 331594 (0.0005) [2023-12-27 07:56:25,501][20381] Updated weights for policy 1, policy_version 331604 (0.0005) [2023-12-27 07:56:25,606][20347] Updated weights for policy 0, policy_version 331816 (0.0007) [2023-12-27 07:56:25,669][20347] Updated weights for policy 0, policy_version 331826 (0.0005) [2023-12-27 07:56:25,687][20291] KL-divergence is very high: 160.2910 [2023-12-27 07:56:25,693][20291] KL-divergence is very high: 134.4890 [2023-12-27 07:56:25,725][20347] Updated weights for policy 0, policy_version 331836 (0.0005) [2023-12-27 07:56:25,732][20291] KL-divergence is very high: 499.6551 [2023-12-27 07:56:25,738][20291] KL-divergence is very high: 376.5341 [2023-12-27 07:56:25,743][20291] KL-divergence is very high: 101.8992 [2023-12-27 07:56:26,147][20381] Updated weights for policy 1, policy_version 331614 (0.0005) [2023-12-27 07:56:26,208][20381] Updated weights for policy 1, policy_version 331624 (0.0006) [2023-12-27 07:56:26,245][20291] KL-divergence is very high: 1058.0332 [2023-12-27 07:56:26,251][20291] KL-divergence is very high: 1392.3571 [2023-12-27 07:56:26,257][20291] KL-divergence is very high: 241.6471 [2023-12-27 07:56:26,263][20291] KL-divergence is very high: 975.2179 [2023-12-27 07:56:26,273][20381] Updated weights for policy 1, policy_version 331634 (0.0007) [2023-12-27 07:56:26,277][20347] Updated weights for policy 0, policy_version 331846 (0.0006) [2023-12-27 07:56:26,295][20291] KL-divergence is very high: 386.8564 [2023-12-27 07:56:26,302][20291] KL-divergence is very high: 275.3543 [2023-12-27 07:56:26,308][20291] KL-divergence is very high: 172.5811 [2023-12-27 07:56:26,316][20291] KL-divergence is very high: 144.9159 [2023-12-27 07:56:26,330][20291] KL-divergence is very high: 146.1999 [2023-12-27 07:56:26,341][20347] Updated weights for policy 0, policy_version 331856 (0.0009) [2023-12-27 07:56:26,342][20291] KL-divergence is very high: 116.1524 [2023-12-27 07:56:26,347][20291] KL-divergence is very high: 447.1340 [2023-12-27 07:56:26,353][20291] KL-divergence is very high: 293.9388 [2023-12-27 07:56:26,364][20291] KL-divergence is very high: 279.8177 [2023-12-27 07:56:26,391][20291] KL-divergence is very high: 587.2327 [2023-12-27 07:56:26,396][20347] Updated weights for policy 0, policy_version 331866 (0.0010) [2023-12-27 07:56:26,396][20291] KL-divergence is very high: 685.8024 [2023-12-27 07:56:26,407][20291] KL-divergence is very high: 343.5733 [2023-12-27 07:56:26,909][20381] Updated weights for policy 1, policy_version 331644 (0.0008) [2023-12-27 07:56:26,957][20381] Updated weights for policy 1, policy_version 331654 (0.0007) [2023-12-27 07:56:27,006][20381] Updated weights for policy 1, policy_version 331664 (0.0008) [2023-12-27 07:56:27,036][20291] KL-divergence is very high: 218.4146 [2023-12-27 07:56:27,047][20347] Updated weights for policy 0, policy_version 331876 (0.0009) [2023-12-27 07:56:27,079][20291] KL-divergence is very high: 218.1315 [2023-12-27 07:56:27,101][20347] Updated weights for policy 0, policy_version 331886 (0.0005) [2023-12-27 07:56:27,124][20291] KL-divergence is very high: 170.5728 [2023-12-27 07:56:27,158][20347] Updated weights for policy 0, policy_version 331896 (0.0005) [2023-12-27 07:56:27,171][20291] KL-divergence is very high: 155.4000 [2023-12-27 07:56:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 169910272. Throughput: 0: 10501.9, 1: 10323.1. Samples: 169896928. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:27,363][19170] Avg episode reward: [(0, '740.600'), (1, '990.340')] [2023-12-27 07:56:27,673][20381] Updated weights for policy 1, policy_version 331674 (0.0007) [2023-12-27 07:56:27,731][20381] Updated weights for policy 1, policy_version 331684 (0.0006) [2023-12-27 07:56:27,759][20347] Updated weights for policy 0, policy_version 331906 (0.0006) [2023-12-27 07:56:27,779][20381] Updated weights for policy 1, policy_version 331694 (0.0005) [2023-12-27 07:56:27,809][20347] Updated weights for policy 0, policy_version 331916 (0.0010) [2023-12-27 07:56:27,830][20381] Updated weights for policy 1, policy_version 331704 (0.0005) [2023-12-27 07:56:27,860][20347] Updated weights for policy 0, policy_version 331926 (0.0010) [2023-12-27 07:56:27,911][20347] Updated weights for policy 0, policy_version 331936 (0.0010) [2023-12-27 07:56:28,376][20381] Updated weights for policy 1, policy_version 331714 (0.0011) [2023-12-27 07:56:28,441][20381] Updated weights for policy 1, policy_version 331724 (0.0010) [2023-12-27 07:56:28,502][20381] Updated weights for policy 1, policy_version 331734 (0.0010) [2023-12-27 07:56:28,555][20347] Updated weights for policy 0, policy_version 331946 (0.0006) [2023-12-27 07:56:28,613][20347] Updated weights for policy 0, policy_version 331956 (0.0005) [2023-12-27 07:56:28,664][20347] Updated weights for policy 0, policy_version 331966 (0.0005) [2023-12-27 07:56:29,200][20381] Updated weights for policy 1, policy_version 331744 (0.0008) [2023-12-27 07:56:29,259][20381] Updated weights for policy 1, policy_version 331754 (0.0008) [2023-12-27 07:56:29,259][20347] Updated weights for policy 0, policy_version 331976 (0.0007) [2023-12-27 07:56:29,267][20291] KL-divergence is very high: 104.8342 [2023-12-27 07:56:29,314][20381] Updated weights for policy 1, policy_version 331764 (0.0008) [2023-12-27 07:56:29,316][20347] Updated weights for policy 0, policy_version 331986 (0.0005) [2023-12-27 07:56:29,377][20347] Updated weights for policy 0, policy_version 331996 (0.0010) [2023-12-27 07:56:29,928][20381] Updated weights for policy 1, policy_version 331774 (0.0010) [2023-12-27 07:56:29,982][20381] Updated weights for policy 1, policy_version 331784 (0.0011) [2023-12-27 07:56:30,037][20347] Updated weights for policy 0, policy_version 332006 (0.0008) [2023-12-27 07:56:30,038][20381] Updated weights for policy 1, policy_version 331794 (0.0011) [2023-12-27 07:56:30,086][20347] Updated weights for policy 0, policy_version 332016 (0.0006) [2023-12-27 07:56:30,143][20347] Updated weights for policy 0, policy_version 332026 (0.0007) [2023-12-27 07:56:30,731][20381] Updated weights for policy 1, policy_version 331804 (0.0009) [2023-12-27 07:56:30,784][20381] Updated weights for policy 1, policy_version 331814 (0.0007) [2023-12-27 07:56:30,837][20347] Updated weights for policy 0, policy_version 332036 (0.0008) [2023-12-27 07:56:30,839][20381] Updated weights for policy 1, policy_version 331824 (0.0006) [2023-12-27 07:56:30,887][20347] Updated weights for policy 0, policy_version 332046 (0.0010) [2023-12-27 07:56:30,935][20347] Updated weights for policy 0, policy_version 332056 (0.0010) [2023-12-27 07:56:31,532][20381] Updated weights for policy 1, policy_version 331834 (0.0006) [2023-12-27 07:56:31,586][20381] Updated weights for policy 1, policy_version 331844 (0.0008) [2023-12-27 07:56:31,637][20347] Updated weights for policy 0, policy_version 332066 (0.0010) [2023-12-27 07:56:31,640][20381] Updated weights for policy 1, policy_version 331854 (0.0007) [2023-12-27 07:56:31,692][20347] Updated weights for policy 0, policy_version 332076 (0.0010) [2023-12-27 07:56:31,698][20381] Updated weights for policy 1, policy_version 331864 (0.0008) [2023-12-27 07:56:31,749][20347] Updated weights for policy 0, policy_version 332086 (0.0010) [2023-12-27 07:56:31,810][20347] Updated weights for policy 0, policy_version 332096 (0.0010) [2023-12-27 07:56:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 170008576. Throughput: 0: 10502.9, 1: 10358.7. Samples: 170023044. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:32,363][19170] Avg episode reward: [(0, '371.270'), (1, '990.340')] [2023-12-27 07:56:32,407][20381] Updated weights for policy 1, policy_version 331874 (0.0008) [2023-12-27 07:56:32,466][20381] Updated weights for policy 1, policy_version 331884 (0.0008) [2023-12-27 07:56:32,482][20347] Updated weights for policy 0, policy_version 332106 (0.0006) [2023-12-27 07:56:32,531][20381] Updated weights for policy 1, policy_version 331894 (0.0008) [2023-12-27 07:56:32,544][20347] Updated weights for policy 0, policy_version 332116 (0.0006) [2023-12-27 07:56:32,603][20347] Updated weights for policy 0, policy_version 332126 (0.0008) [2023-12-27 07:56:33,170][20381] Updated weights for policy 1, policy_version 331904 (0.0006) [2023-12-27 07:56:33,215][20347] Updated weights for policy 0, policy_version 332136 (0.0005) [2023-12-27 07:56:33,218][20381] Updated weights for policy 1, policy_version 331914 (0.0005) [2023-12-27 07:56:33,265][20347] Updated weights for policy 0, policy_version 332146 (0.0006) [2023-12-27 07:56:33,269][20381] Updated weights for policy 1, policy_version 331924 (0.0005) [2023-12-27 07:56:33,318][20347] Updated weights for policy 0, policy_version 332156 (0.0008) [2023-12-27 07:56:33,827][20381] Updated weights for policy 1, policy_version 331934 (0.0007) [2023-12-27 07:56:33,869][20347] Updated weights for policy 0, policy_version 332166 (0.0006) [2023-12-27 07:56:33,885][20381] Updated weights for policy 1, policy_version 331944 (0.0007) [2023-12-27 07:56:33,923][20347] Updated weights for policy 0, policy_version 332176 (0.0010) [2023-12-27 07:56:33,942][20381] Updated weights for policy 1, policy_version 331954 (0.0005) [2023-12-27 07:56:33,974][20347] Updated weights for policy 0, policy_version 332186 (0.0010) [2023-12-27 07:56:34,614][20381] Updated weights for policy 1, policy_version 331964 (0.0006) [2023-12-27 07:56:34,672][20347] Updated weights for policy 0, policy_version 332196 (0.0008) [2023-12-27 07:56:34,678][20381] Updated weights for policy 1, policy_version 331974 (0.0011) [2023-12-27 07:56:34,729][20347] Updated weights for policy 0, policy_version 332206 (0.0007) [2023-12-27 07:56:34,735][20381] Updated weights for policy 1, policy_version 331984 (0.0010) [2023-12-27 07:56:34,786][20347] Updated weights for policy 0, policy_version 332216 (0.0011) [2023-12-27 07:56:35,339][20381] Updated weights for policy 1, policy_version 331994 (0.0009) [2023-12-27 07:56:35,400][20381] Updated weights for policy 1, policy_version 332004 (0.0011) [2023-12-27 07:56:35,439][20347] Updated weights for policy 0, policy_version 332226 (0.0010) [2023-12-27 07:56:35,462][20381] Updated weights for policy 1, policy_version 332014 (0.0010) [2023-12-27 07:56:35,501][20347] Updated weights for policy 0, policy_version 332236 (0.0010) [2023-12-27 07:56:35,520][20381] Updated weights for policy 1, policy_version 332024 (0.0011) [2023-12-27 07:56:35,551][20347] Updated weights for policy 0, policy_version 332246 (0.0010) [2023-12-27 07:56:35,602][20347] Updated weights for policy 0, policy_version 332256 (0.0010) [2023-12-27 07:56:36,180][20347] Updated weights for policy 0, policy_version 332266 (0.0011) [2023-12-27 07:56:36,184][20381] Updated weights for policy 1, policy_version 332034 (0.0010) [2023-12-27 07:56:36,235][20381] Updated weights for policy 1, policy_version 332044 (0.0011) [2023-12-27 07:56:36,239][20347] Updated weights for policy 0, policy_version 332276 (0.0010) [2023-12-27 07:56:36,291][20381] Updated weights for policy 1, policy_version 332054 (0.0011) [2023-12-27 07:56:36,301][20347] Updated weights for policy 0, policy_version 332286 (0.0010) [2023-12-27 07:56:36,970][20381] Updated weights for policy 1, policy_version 332064 (0.0011) [2023-12-27 07:56:36,983][20347] Updated weights for policy 0, policy_version 332296 (0.0010) [2023-12-27 07:56:37,029][20381] Updated weights for policy 1, policy_version 332074 (0.0011) [2023-12-27 07:56:37,041][20347] Updated weights for policy 0, policy_version 332306 (0.0010) [2023-12-27 07:56:37,084][20381] Updated weights for policy 1, policy_version 332084 (0.0010) [2023-12-27 07:56:37,093][20347] Updated weights for policy 0, policy_version 332316 (0.0010) [2023-12-27 07:56:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 21026.0, 300 sec: 20910.4). Total num frames: 170123264. Throughput: 0: 10539.1, 1: 10372.4. Samples: 170088340. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:37,364][19170] Avg episode reward: [(0, '208.990'), (1, '1000.000')] [2023-12-27 07:56:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000332088_85032960.pth... [2023-12-27 07:56:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000332320_85090304.pth... [2023-12-27 07:56:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000331072_84770816.pth [2023-12-27 07:56:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000330872_84721664.pth [2023-12-27 07:56:37,674][20381] Updated weights for policy 1, policy_version 332094 (0.0010) [2023-12-27 07:56:37,727][20381] Updated weights for policy 1, policy_version 332104 (0.0010) [2023-12-27 07:56:37,736][20347] Updated weights for policy 0, policy_version 332326 (0.0008) [2023-12-27 07:56:37,776][20381] Updated weights for policy 1, policy_version 332114 (0.0011) [2023-12-27 07:56:37,796][20347] Updated weights for policy 0, policy_version 332336 (0.0006) [2023-12-27 07:56:37,844][20347] Updated weights for policy 0, policy_version 332346 (0.0007) [2023-12-27 07:56:38,488][20381] Updated weights for policy 1, policy_version 332124 (0.0011) [2023-12-27 07:56:38,540][20381] Updated weights for policy 1, policy_version 332134 (0.0010) [2023-12-27 07:56:38,545][20347] Updated weights for policy 0, policy_version 332356 (0.0007) [2023-12-27 07:56:38,592][20381] Updated weights for policy 1, policy_version 332144 (0.0010) [2023-12-27 07:56:38,601][20347] Updated weights for policy 0, policy_version 332366 (0.0005) [2023-12-27 07:56:38,655][20347] Updated weights for policy 0, policy_version 332376 (0.0007) [2023-12-27 07:56:39,298][20381] Updated weights for policy 1, policy_version 332154 (0.0010) [2023-12-27 07:56:39,360][20347] Updated weights for policy 0, policy_version 332386 (0.0008) [2023-12-27 07:56:39,367][20381] Updated weights for policy 1, policy_version 332164 (0.0011) [2023-12-27 07:56:39,424][20347] Updated weights for policy 0, policy_version 332396 (0.0007) [2023-12-27 07:56:39,432][20381] Updated weights for policy 1, policy_version 332174 (0.0010) [2023-12-27 07:56:39,485][20347] Updated weights for policy 0, policy_version 332406 (0.0006) [2023-12-27 07:56:39,493][20381] Updated weights for policy 1, policy_version 332184 (0.0011) [2023-12-27 07:56:39,548][20347] Updated weights for policy 0, policy_version 332416 (0.0006) [2023-12-27 07:56:40,206][20381] Updated weights for policy 1, policy_version 332194 (0.0011) [2023-12-27 07:56:40,212][20347] Updated weights for policy 0, policy_version 332426 (0.0008) [2023-12-27 07:56:40,259][20381] Updated weights for policy 1, policy_version 332204 (0.0010) [2023-12-27 07:56:40,275][20347] Updated weights for policy 0, policy_version 332436 (0.0008) [2023-12-27 07:56:40,319][20381] Updated weights for policy 1, policy_version 332214 (0.0011) [2023-12-27 07:56:40,341][20347] Updated weights for policy 0, policy_version 332446 (0.0008) [2023-12-27 07:56:40,942][20347] Updated weights for policy 0, policy_version 332456 (0.0007) [2023-12-27 07:56:40,966][20381] Updated weights for policy 1, policy_version 332224 (0.0011) [2023-12-27 07:56:40,993][20347] Updated weights for policy 0, policy_version 332466 (0.0006) [2023-12-27 07:56:41,017][20381] Updated weights for policy 1, policy_version 332234 (0.0008) [2023-12-27 07:56:41,056][20347] Updated weights for policy 0, policy_version 332476 (0.0007) [2023-12-27 07:56:41,085][20381] Updated weights for policy 1, policy_version 332244 (0.0011) [2023-12-27 07:56:41,771][20347] Updated weights for policy 0, policy_version 332486 (0.0006) [2023-12-27 07:56:41,783][20381] Updated weights for policy 1, policy_version 332254 (0.0009) [2023-12-27 07:56:41,820][20347] Updated weights for policy 0, policy_version 332496 (0.0006) [2023-12-27 07:56:41,842][20381] Updated weights for policy 1, policy_version 332264 (0.0008) [2023-12-27 07:56:41,874][20347] Updated weights for policy 0, policy_version 332506 (0.0006) [2023-12-27 07:56:41,897][20381] Updated weights for policy 1, policy_version 332274 (0.0008) [2023-12-27 07:56:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 170221568. Throughput: 0: 10529.9, 1: 10400.3. Samples: 170212888. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:42,363][19170] Avg episode reward: [(0, '122.040'), (1, '1000.000')] [2023-12-27 07:56:42,560][20347] Updated weights for policy 0, policy_version 332516 (0.0007) [2023-12-27 07:56:42,614][20347] Updated weights for policy 0, policy_version 332526 (0.0007) [2023-12-27 07:56:42,619][20381] Updated weights for policy 1, policy_version 332284 (0.0007) [2023-12-27 07:56:42,663][20347] Updated weights for policy 0, policy_version 332536 (0.0006) [2023-12-27 07:56:42,675][20381] Updated weights for policy 1, policy_version 332294 (0.0007) [2023-12-27 07:56:42,732][20381] Updated weights for policy 1, policy_version 332304 (0.0008) [2023-12-27 07:56:43,341][20347] Updated weights for policy 0, policy_version 332546 (0.0006) [2023-12-27 07:56:43,398][20347] Updated weights for policy 0, policy_version 332556 (0.0010) [2023-12-27 07:56:43,416][20381] Updated weights for policy 1, policy_version 332314 (0.0009) [2023-12-27 07:56:43,454][20347] Updated weights for policy 0, policy_version 332566 (0.0010) [2023-12-27 07:56:43,477][20381] Updated weights for policy 1, policy_version 332324 (0.0005) [2023-12-27 07:56:43,512][20347] Updated weights for policy 0, policy_version 332576 (0.0010) [2023-12-27 07:56:43,540][20381] Updated weights for policy 1, policy_version 332334 (0.0005) [2023-12-27 07:56:43,589][20381] Updated weights for policy 1, policy_version 332344 (0.0005) [2023-12-27 07:56:44,182][20347] Updated weights for policy 0, policy_version 332586 (0.0011) [2023-12-27 07:56:44,208][20381] Updated weights for policy 1, policy_version 332354 (0.0008) [2023-12-27 07:56:44,229][20347] Updated weights for policy 0, policy_version 332596 (0.0010) [2023-12-27 07:56:44,265][20381] Updated weights for policy 1, policy_version 332364 (0.0005) [2023-12-27 07:56:44,282][20347] Updated weights for policy 0, policy_version 332606 (0.0010) [2023-12-27 07:56:44,318][20381] Updated weights for policy 1, policy_version 332374 (0.0007) [2023-12-27 07:56:44,996][20347] Updated weights for policy 0, policy_version 332616 (0.0010) [2023-12-27 07:56:45,043][20381] Updated weights for policy 1, policy_version 332384 (0.0007) [2023-12-27 07:56:45,053][20347] Updated weights for policy 0, policy_version 332626 (0.0011) [2023-12-27 07:56:45,104][20381] Updated weights for policy 1, policy_version 332394 (0.0007) [2023-12-27 07:56:45,106][20347] Updated weights for policy 0, policy_version 332636 (0.0010) [2023-12-27 07:56:45,169][20381] Updated weights for policy 1, policy_version 332404 (0.0007) [2023-12-27 07:56:45,803][20347] Updated weights for policy 0, policy_version 332646 (0.0010) [2023-12-27 07:56:45,859][20347] Updated weights for policy 0, policy_version 332656 (0.0010) [2023-12-27 07:56:45,865][20381] Updated weights for policy 1, policy_version 332414 (0.0006) [2023-12-27 07:56:45,909][20347] Updated weights for policy 0, policy_version 332666 (0.0010) [2023-12-27 07:56:45,914][20381] Updated weights for policy 1, policy_version 332424 (0.0005) [2023-12-27 07:56:45,971][20381] Updated weights for policy 1, policy_version 332434 (0.0006) [2023-12-27 07:56:46,611][20347] Updated weights for policy 0, policy_version 332676 (0.0009) [2023-12-27 07:56:46,663][20347] Updated weights for policy 0, policy_version 332686 (0.0005) [2023-12-27 07:56:46,680][20381] Updated weights for policy 1, policy_version 332444 (0.0007) [2023-12-27 07:56:46,718][20347] Updated weights for policy 0, policy_version 332696 (0.0005) [2023-12-27 07:56:46,739][20381] Updated weights for policy 1, policy_version 332454 (0.0008) [2023-12-27 07:56:46,801][20381] Updated weights for policy 1, policy_version 332464 (0.0007) [2023-12-27 07:56:47,307][20347] Updated weights for policy 0, policy_version 332706 (0.0005) [2023-12-27 07:56:47,346][20381] Updated weights for policy 1, policy_version 332474 (0.0006) [2023-12-27 07:56:47,356][20347] Updated weights for policy 0, policy_version 332716 (0.0006) [2023-12-27 07:56:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 170319872. Throughput: 0: 10510.0, 1: 10352.8. Samples: 170335964. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:47,363][19170] Avg episode reward: [(0, '156.790'), (1, '1000.000')] [2023-12-27 07:56:47,405][20381] Updated weights for policy 1, policy_version 332484 (0.0008) [2023-12-27 07:56:47,412][20347] Updated weights for policy 0, policy_version 332726 (0.0005) [2023-12-27 07:56:47,465][20347] Updated weights for policy 0, policy_version 332736 (0.0005) [2023-12-27 07:56:47,467][20381] Updated weights for policy 1, policy_version 332494 (0.0007) [2023-12-27 07:56:47,535][20381] Updated weights for policy 1, policy_version 332504 (0.0008) [2023-12-27 07:56:48,040][20347] Updated weights for policy 0, policy_version 332746 (0.0008) [2023-12-27 07:56:48,089][20347] Updated weights for policy 0, policy_version 332756 (0.0008) [2023-12-27 07:56:48,136][20347] Updated weights for policy 0, policy_version 332766 (0.0009) [2023-12-27 07:56:48,227][20381] Updated weights for policy 1, policy_version 332514 (0.0007) [2023-12-27 07:56:48,279][20381] Updated weights for policy 1, policy_version 332524 (0.0007) [2023-12-27 07:56:48,335][20381] Updated weights for policy 1, policy_version 332534 (0.0008) [2023-12-27 07:56:48,840][20347] Updated weights for policy 0, policy_version 332776 (0.0008) [2023-12-27 07:56:48,901][20347] Updated weights for policy 0, policy_version 332786 (0.0007) [2023-12-27 07:56:48,953][20347] Updated weights for policy 0, policy_version 332796 (0.0008) [2023-12-27 07:56:49,041][20381] Updated weights for policy 1, policy_version 332544 (0.0008) [2023-12-27 07:56:49,093][20381] Updated weights for policy 1, policy_version 332554 (0.0008) [2023-12-27 07:56:49,144][20381] Updated weights for policy 1, policy_version 332564 (0.0008) [2023-12-27 07:56:49,640][20347] Updated weights for policy 0, policy_version 332806 (0.0007) [2023-12-27 07:56:49,698][20347] Updated weights for policy 0, policy_version 332816 (0.0006) [2023-12-27 07:56:49,753][20347] Updated weights for policy 0, policy_version 332826 (0.0005) [2023-12-27 07:56:49,856][20381] Updated weights for policy 1, policy_version 332574 (0.0008) [2023-12-27 07:56:49,922][20381] Updated weights for policy 1, policy_version 332584 (0.0008) [2023-12-27 07:56:49,989][20381] Updated weights for policy 1, policy_version 332594 (0.0008) [2023-12-27 07:56:50,420][20347] Updated weights for policy 0, policy_version 332836 (0.0006) [2023-12-27 07:56:50,474][20347] Updated weights for policy 0, policy_version 332846 (0.0005) [2023-12-27 07:56:50,523][20347] Updated weights for policy 0, policy_version 332856 (0.0005) [2023-12-27 07:56:50,676][20381] Updated weights for policy 1, policy_version 332604 (0.0008) [2023-12-27 07:56:50,731][20381] Updated weights for policy 1, policy_version 332614 (0.0008) [2023-12-27 07:56:50,790][20381] Updated weights for policy 1, policy_version 332624 (0.0005) [2023-12-27 07:56:51,188][20347] Updated weights for policy 0, policy_version 332866 (0.0008) [2023-12-27 07:56:51,253][20347] Updated weights for policy 0, policy_version 332876 (0.0008) [2023-12-27 07:56:51,308][20347] Updated weights for policy 0, policy_version 332886 (0.0008) [2023-12-27 07:56:51,367][20347] Updated weights for policy 0, policy_version 332896 (0.0006) [2023-12-27 07:56:51,472][20381] Updated weights for policy 1, policy_version 332634 (0.0006) [2023-12-27 07:56:51,524][20381] Updated weights for policy 1, policy_version 332644 (0.0008) [2023-12-27 07:56:51,576][20381] Updated weights for policy 1, policy_version 332654 (0.0007) [2023-12-27 07:56:51,634][20381] Updated weights for policy 1, policy_version 332664 (0.0008) [2023-12-27 07:56:51,978][20347] Updated weights for policy 0, policy_version 332906 (0.0005) [2023-12-27 07:56:52,036][20347] Updated weights for policy 0, policy_version 332916 (0.0006) [2023-12-27 07:56:52,096][20347] Updated weights for policy 0, policy_version 332926 (0.0006) [2023-12-27 07:56:52,317][20381] Updated weights for policy 1, policy_version 332674 (0.0007) [2023-12-27 07:56:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 170426368. Throughput: 0: 10543.7, 1: 10313.1. Samples: 170396952. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:52,363][19170] Avg episode reward: [(0, '177.210'), (1, '1000.000')] [2023-12-27 07:56:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000332928_85245952.pth... [2023-12-27 07:56:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000331680_84926464.pth [2023-12-27 07:56:52,379][20381] Updated weights for policy 1, policy_version 332684 (0.0006) [2023-12-27 07:56:52,440][20381] Updated weights for policy 1, policy_version 332694 (0.0008) [2023-12-27 07:56:52,447][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000332696_85188608.pth... [2023-12-27 07:56:52,451][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000331480_84877312.pth [2023-12-27 07:56:52,751][20347] Updated weights for policy 0, policy_version 332936 (0.0006) [2023-12-27 07:56:52,810][20347] Updated weights for policy 0, policy_version 332946 (0.0007) [2023-12-27 07:56:52,872][20347] Updated weights for policy 0, policy_version 332956 (0.0005) [2023-12-27 07:56:53,107][20381] Updated weights for policy 1, policy_version 332704 (0.0008) [2023-12-27 07:56:53,161][20381] Updated weights for policy 1, policy_version 332714 (0.0007) [2023-12-27 07:56:53,214][20381] Updated weights for policy 1, policy_version 332724 (0.0007) [2023-12-27 07:56:53,426][20347] Updated weights for policy 0, policy_version 332966 (0.0006) [2023-12-27 07:56:53,472][20347] Updated weights for policy 0, policy_version 332976 (0.0008) [2023-12-27 07:56:53,530][20347] Updated weights for policy 0, policy_version 332986 (0.0006) [2023-12-27 07:56:53,959][20381] Updated weights for policy 1, policy_version 332734 (0.0008) [2023-12-27 07:56:54,011][20381] Updated weights for policy 1, policy_version 332744 (0.0008) [2023-12-27 07:56:54,059][20381] Updated weights for policy 1, policy_version 332754 (0.0008) [2023-12-27 07:56:54,183][20347] Updated weights for policy 0, policy_version 332996 (0.0010) [2023-12-27 07:56:54,235][20347] Updated weights for policy 0, policy_version 333006 (0.0007) [2023-12-27 07:56:54,296][20347] Updated weights for policy 0, policy_version 333016 (0.0008) [2023-12-27 07:56:54,738][20381] Updated weights for policy 1, policy_version 332764 (0.0009) [2023-12-27 07:56:54,786][20381] Updated weights for policy 1, policy_version 332774 (0.0010) [2023-12-27 07:56:54,831][20381] Updated weights for policy 1, policy_version 332784 (0.0010) [2023-12-27 07:56:54,990][20347] Updated weights for policy 0, policy_version 333026 (0.0008) [2023-12-27 07:56:55,048][20347] Updated weights for policy 0, policy_version 333036 (0.0007) [2023-12-27 07:56:55,102][20347] Updated weights for policy 0, policy_version 333046 (0.0008) [2023-12-27 07:56:55,161][20347] Updated weights for policy 0, policy_version 333056 (0.0008) [2023-12-27 07:56:55,488][20381] Updated weights for policy 1, policy_version 332794 (0.0009) [2023-12-27 07:56:55,540][20381] Updated weights for policy 1, policy_version 332804 (0.0007) [2023-12-27 07:56:55,586][20381] Updated weights for policy 1, policy_version 332814 (0.0007) [2023-12-27 07:56:55,639][20381] Updated weights for policy 1, policy_version 332824 (0.0008) [2023-12-27 07:56:55,843][20347] Updated weights for policy 0, policy_version 333066 (0.0008) [2023-12-27 07:56:55,891][20347] Updated weights for policy 0, policy_version 333076 (0.0010) [2023-12-27 07:56:55,943][20347] Updated weights for policy 0, policy_version 333086 (0.0010) [2023-12-27 07:56:56,350][20381] Updated weights for policy 1, policy_version 332834 (0.0010) [2023-12-27 07:56:56,406][20381] Updated weights for policy 1, policy_version 332844 (0.0010) [2023-12-27 07:56:56,461][20381] Updated weights for policy 1, policy_version 332854 (0.0010) [2023-12-27 07:56:56,670][20347] Updated weights for policy 0, policy_version 333096 (0.0007) [2023-12-27 07:56:56,722][20347] Updated weights for policy 0, policy_version 333106 (0.0008) [2023-12-27 07:56:56,778][20347] Updated weights for policy 0, policy_version 333116 (0.0010) [2023-12-27 07:56:57,101][20381] Updated weights for policy 1, policy_version 332864 (0.0008) [2023-12-27 07:56:57,160][20381] Updated weights for policy 1, policy_version 332874 (0.0007) [2023-12-27 07:56:57,221][20381] Updated weights for policy 1, policy_version 332884 (0.0008) [2023-12-27 07:56:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 170532864. Throughput: 0: 10528.4, 1: 10342.1. Samples: 170521304. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:56:57,363][19170] Avg episode reward: [(0, '183.470'), (1, '1000.000')] [2023-12-27 07:56:57,434][20347] Updated weights for policy 0, policy_version 333126 (0.0008) [2023-12-27 07:56:57,497][20347] Updated weights for policy 0, policy_version 333136 (0.0007) [2023-12-27 07:56:57,557][20347] Updated weights for policy 0, policy_version 333146 (0.0008) [2023-12-27 07:56:57,859][20381] Updated weights for policy 1, policy_version 332894 (0.0006) [2023-12-27 07:56:57,911][20381] Updated weights for policy 1, policy_version 332904 (0.0005) [2023-12-27 07:56:57,964][20381] Updated weights for policy 1, policy_version 332914 (0.0005) [2023-12-27 07:56:58,223][20347] Updated weights for policy 0, policy_version 333156 (0.0007) [2023-12-27 07:56:58,286][20347] Updated weights for policy 0, policy_version 333166 (0.0007) [2023-12-27 07:56:58,345][20347] Updated weights for policy 0, policy_version 333176 (0.0007) [2023-12-27 07:56:58,610][20381] Updated weights for policy 1, policy_version 332924 (0.0007) [2023-12-27 07:56:58,673][20381] Updated weights for policy 1, policy_version 332934 (0.0008) [2023-12-27 07:56:58,733][20381] Updated weights for policy 1, policy_version 332944 (0.0007) [2023-12-27 07:56:59,123][20347] Updated weights for policy 0, policy_version 333186 (0.0007) [2023-12-27 07:56:59,180][20347] Updated weights for policy 0, policy_version 333196 (0.0008) [2023-12-27 07:56:59,242][20347] Updated weights for policy 0, policy_version 333206 (0.0007) [2023-12-27 07:56:59,306][20347] Updated weights for policy 0, policy_version 333216 (0.0009) [2023-12-27 07:56:59,459][20381] Updated weights for policy 1, policy_version 332954 (0.0007) [2023-12-27 07:56:59,520][20381] Updated weights for policy 1, policy_version 332964 (0.0007) [2023-12-27 07:56:59,565][20381] Updated weights for policy 1, policy_version 332974 (0.0007) [2023-12-27 07:56:59,613][20381] Updated weights for policy 1, policy_version 332984 (0.0008) [2023-12-27 07:56:59,985][20347] Updated weights for policy 0, policy_version 333226 (0.0007) [2023-12-27 07:57:00,035][20347] Updated weights for policy 0, policy_version 333236 (0.0009) [2023-12-27 07:57:00,083][20347] Updated weights for policy 0, policy_version 333246 (0.0010) [2023-12-27 07:57:00,312][20381] Updated weights for policy 1, policy_version 332994 (0.0005) [2023-12-27 07:57:00,371][20381] Updated weights for policy 1, policy_version 333004 (0.0005) [2023-12-27 07:57:00,427][20381] Updated weights for policy 1, policy_version 333014 (0.0005) [2023-12-27 07:57:00,705][20347] Updated weights for policy 0, policy_version 333256 (0.0006) [2023-12-27 07:57:00,767][20347] Updated weights for policy 0, policy_version 333266 (0.0005) [2023-12-27 07:57:00,822][20347] Updated weights for policy 0, policy_version 333276 (0.0009) [2023-12-27 07:57:00,971][20381] Updated weights for policy 1, policy_version 333024 (0.0005) [2023-12-27 07:57:01,020][20381] Updated weights for policy 1, policy_version 333034 (0.0005) [2023-12-27 07:57:01,076][20381] Updated weights for policy 1, policy_version 333044 (0.0009) [2023-12-27 07:57:01,472][20347] Updated weights for policy 0, policy_version 333286 (0.0008) [2023-12-27 07:57:01,526][20347] Updated weights for policy 0, policy_version 333296 (0.0005) [2023-12-27 07:57:01,579][20347] Updated weights for policy 0, policy_version 333306 (0.0006) [2023-12-27 07:57:01,783][20381] Updated weights for policy 1, policy_version 333054 (0.0007) [2023-12-27 07:57:01,832][20381] Updated weights for policy 1, policy_version 333064 (0.0007) [2023-12-27 07:57:01,888][20381] Updated weights for policy 1, policy_version 333074 (0.0008) [2023-12-27 07:57:02,198][20347] Updated weights for policy 0, policy_version 333316 (0.0007) [2023-12-27 07:57:02,252][20347] Updated weights for policy 0, policy_version 333326 (0.0008) [2023-12-27 07:57:02,317][20347] Updated weights for policy 0, policy_version 333336 (0.0007) [2023-12-27 07:57:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 170631168. Throughput: 0: 10464.6, 1: 10334.6. Samples: 170645992. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:02,363][19170] Avg episode reward: [(0, '181.360'), (1, '1000.000')] [2023-12-27 07:57:02,595][20381] Updated weights for policy 1, policy_version 333084 (0.0007) [2023-12-27 07:57:02,653][20381] Updated weights for policy 1, policy_version 333094 (0.0008) [2023-12-27 07:57:02,709][20381] Updated weights for policy 1, policy_version 333104 (0.0009) [2023-12-27 07:57:02,911][20347] Updated weights for policy 0, policy_version 333346 (0.0007) [2023-12-27 07:57:02,964][20347] Updated weights for policy 0, policy_version 333356 (0.0005) [2023-12-27 07:57:03,014][20347] Updated weights for policy 0, policy_version 333366 (0.0005) [2023-12-27 07:57:03,060][20347] Updated weights for policy 0, policy_version 333376 (0.0005) [2023-12-27 07:57:03,412][20381] Updated weights for policy 1, policy_version 333114 (0.0009) [2023-12-27 07:57:03,460][20381] Updated weights for policy 1, policy_version 333124 (0.0005) [2023-12-27 07:57:03,507][20381] Updated weights for policy 1, policy_version 333134 (0.0007) [2023-12-27 07:57:03,561][20381] Updated weights for policy 1, policy_version 333144 (0.0005) [2023-12-27 07:57:03,631][20347] Updated weights for policy 0, policy_version 333386 (0.0006) [2023-12-27 07:57:03,688][20347] Updated weights for policy 0, policy_version 333396 (0.0010) [2023-12-27 07:57:03,751][20347] Updated weights for policy 0, policy_version 333406 (0.0010) [2023-12-27 07:57:04,171][20381] Updated weights for policy 1, policy_version 333154 (0.0010) [2023-12-27 07:57:04,231][20381] Updated weights for policy 1, policy_version 333164 (0.0010) [2023-12-27 07:57:04,293][20381] Updated weights for policy 1, policy_version 333174 (0.0009) [2023-12-27 07:57:04,392][20347] Updated weights for policy 0, policy_version 333416 (0.0008) [2023-12-27 07:57:04,447][20347] Updated weights for policy 0, policy_version 333426 (0.0009) [2023-12-27 07:57:04,492][20347] Updated weights for policy 0, policy_version 333436 (0.0010) [2023-12-27 07:57:04,860][20381] Updated weights for policy 1, policy_version 333184 (0.0006) [2023-12-27 07:57:04,925][20381] Updated weights for policy 1, policy_version 333194 (0.0006) [2023-12-27 07:57:04,977][20381] Updated weights for policy 1, policy_version 333204 (0.0007) [2023-12-27 07:57:05,222][20347] Updated weights for policy 0, policy_version 333446 (0.0010) [2023-12-27 07:57:05,277][20347] Updated weights for policy 0, policy_version 333456 (0.0010) [2023-12-27 07:57:05,345][20347] Updated weights for policy 0, policy_version 333466 (0.0010) [2023-12-27 07:57:05,628][20381] Updated weights for policy 1, policy_version 333214 (0.0008) [2023-12-27 07:57:05,685][20381] Updated weights for policy 1, policy_version 333224 (0.0009) [2023-12-27 07:57:05,735][20381] Updated weights for policy 1, policy_version 333234 (0.0009) [2023-12-27 07:57:05,917][20347] Updated weights for policy 0, policy_version 333476 (0.0010) [2023-12-27 07:57:05,973][20347] Updated weights for policy 0, policy_version 333486 (0.0006) [2023-12-27 07:57:06,032][20347] Updated weights for policy 0, policy_version 333496 (0.0007) [2023-12-27 07:57:06,467][20381] Updated weights for policy 1, policy_version 333244 (0.0009) [2023-12-27 07:57:06,528][20381] Updated weights for policy 1, policy_version 333254 (0.0008) [2023-12-27 07:57:06,587][20381] Updated weights for policy 1, policy_version 333264 (0.0008) [2023-12-27 07:57:06,679][20347] Updated weights for policy 0, policy_version 333506 (0.0007) [2023-12-27 07:57:06,739][20347] Updated weights for policy 0, policy_version 333516 (0.0007) [2023-12-27 07:57:06,794][20347] Updated weights for policy 0, policy_version 333526 (0.0008) [2023-12-27 07:57:06,847][20347] Updated weights for policy 0, policy_version 333536 (0.0008) [2023-12-27 07:57:07,304][20381] Updated weights for policy 1, policy_version 333274 (0.0008) [2023-12-27 07:57:07,361][20381] Updated weights for policy 1, policy_version 333284 (0.0008) [2023-12-27 07:57:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 170737664. Throughput: 0: 10543.3, 1: 10354.7. Samples: 170711548. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:07,363][19170] Avg episode reward: [(0, '193.770'), (1, '1000.000')] [2023-12-27 07:57:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000333536_85401600.pth... [2023-12-27 07:57:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000332320_85090304.pth [2023-12-27 07:57:07,426][20381] Updated weights for policy 1, policy_version 333294 (0.0008) [2023-12-27 07:57:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000333304_85344256.pth... [2023-12-27 07:57:07,483][20381] Updated weights for policy 1, policy_version 333304 (0.0008) [2023-12-27 07:57:07,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000332088_85032960.pth [2023-12-27 07:57:07,565][20347] Updated weights for policy 0, policy_version 333546 (0.0008) [2023-12-27 07:57:07,620][20347] Updated weights for policy 0, policy_version 333556 (0.0007) [2023-12-27 07:57:07,669][20347] Updated weights for policy 0, policy_version 333566 (0.0008) [2023-12-27 07:57:08,171][20381] Updated weights for policy 1, policy_version 333314 (0.0008) [2023-12-27 07:57:08,219][20381] Updated weights for policy 1, policy_version 333324 (0.0006) [2023-12-27 07:57:08,270][20381] Updated weights for policy 1, policy_version 333334 (0.0007) [2023-12-27 07:57:08,331][20347] Updated weights for policy 0, policy_version 333576 (0.0007) [2023-12-27 07:57:08,390][20347] Updated weights for policy 0, policy_version 333586 (0.0007) [2023-12-27 07:57:08,451][20347] Updated weights for policy 0, policy_version 333596 (0.0006) [2023-12-27 07:57:08,992][20381] Updated weights for policy 1, policy_version 333344 (0.0007) [2023-12-27 07:57:09,046][20381] Updated weights for policy 1, policy_version 333354 (0.0008) [2023-12-27 07:57:09,103][20381] Updated weights for policy 1, policy_version 333364 (0.0007) [2023-12-27 07:57:09,118][20347] Updated weights for policy 0, policy_version 333606 (0.0007) [2023-12-27 07:57:09,173][20347] Updated weights for policy 0, policy_version 333616 (0.0007) [2023-12-27 07:57:09,228][20347] Updated weights for policy 0, policy_version 333626 (0.0008) [2023-12-27 07:57:09,794][20381] Updated weights for policy 1, policy_version 333374 (0.0007) [2023-12-27 07:57:09,853][20381] Updated weights for policy 1, policy_version 333384 (0.0007) [2023-12-27 07:57:09,906][20381] Updated weights for policy 1, policy_version 333394 (0.0008) [2023-12-27 07:57:09,991][20347] Updated weights for policy 0, policy_version 333636 (0.0008) [2023-12-27 07:57:10,047][20347] Updated weights for policy 0, policy_version 333646 (0.0011) [2023-12-27 07:57:10,106][20347] Updated weights for policy 0, policy_version 333656 (0.0010) [2023-12-27 07:57:10,634][20381] Updated weights for policy 1, policy_version 333404 (0.0008) [2023-12-27 07:57:10,683][20381] Updated weights for policy 1, policy_version 333414 (0.0008) [2023-12-27 07:57:10,737][20381] Updated weights for policy 1, policy_version 333424 (0.0007) [2023-12-27 07:57:10,768][20347] Updated weights for policy 0, policy_version 333666 (0.0008) [2023-12-27 07:57:10,822][20347] Updated weights for policy 0, policy_version 333676 (0.0006) [2023-12-27 07:57:10,884][20347] Updated weights for policy 0, policy_version 333686 (0.0005) [2023-12-27 07:57:10,941][20347] Updated weights for policy 0, policy_version 333696 (0.0007) [2023-12-27 07:57:11,524][20381] Updated weights for policy 1, policy_version 333434 (0.0009) [2023-12-27 07:57:11,585][20381] Updated weights for policy 1, policy_version 333444 (0.0008) [2023-12-27 07:57:11,654][20381] Updated weights for policy 1, policy_version 333454 (0.0009) [2023-12-27 07:57:11,691][20347] Updated weights for policy 0, policy_version 333706 (0.0007) [2023-12-27 07:57:11,712][20381] Updated weights for policy 1, policy_version 333464 (0.0010) [2023-12-27 07:57:11,743][20347] Updated weights for policy 0, policy_version 333716 (0.0007) [2023-12-27 07:57:11,798][20347] Updated weights for policy 0, policy_version 333726 (0.0008) [2023-12-27 07:57:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 170835968. Throughput: 0: 10478.5, 1: 10300.0. Samples: 170831960. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:12,363][19170] Avg episode reward: [(0, '240.440'), (1, '1000.000')] [2023-12-27 07:57:12,435][20381] Updated weights for policy 1, policy_version 333474 (0.0011) [2023-12-27 07:57:12,491][20381] Updated weights for policy 1, policy_version 333484 (0.0011) [2023-12-27 07:57:12,509][20347] Updated weights for policy 0, policy_version 333736 (0.0006) [2023-12-27 07:57:12,538][20381] Updated weights for policy 1, policy_version 333494 (0.0011) [2023-12-27 07:57:12,569][20347] Updated weights for policy 0, policy_version 333746 (0.0007) [2023-12-27 07:57:12,628][20347] Updated weights for policy 0, policy_version 333756 (0.0008) [2023-12-27 07:57:13,181][20381] Updated weights for policy 1, policy_version 333504 (0.0009) [2023-12-27 07:57:13,232][20381] Updated weights for policy 1, policy_version 333514 (0.0007) [2023-12-27 07:57:13,286][20381] Updated weights for policy 1, policy_version 333524 (0.0007) [2023-12-27 07:57:13,303][20347] Updated weights for policy 0, policy_version 333766 (0.0006) [2023-12-27 07:57:13,357][20347] Updated weights for policy 0, policy_version 333776 (0.0005) [2023-12-27 07:57:13,413][20347] Updated weights for policy 0, policy_version 333786 (0.0005) [2023-12-27 07:57:13,967][20347] Updated weights for policy 0, policy_version 333796 (0.0005) [2023-12-27 07:57:14,011][20381] Updated weights for policy 1, policy_version 333534 (0.0007) [2023-12-27 07:57:14,028][20347] Updated weights for policy 0, policy_version 333806 (0.0006) [2023-12-27 07:57:14,058][20381] Updated weights for policy 1, policy_version 333544 (0.0007) [2023-12-27 07:57:14,078][20347] Updated weights for policy 0, policy_version 333816 (0.0008) [2023-12-27 07:57:14,122][20381] Updated weights for policy 1, policy_version 333554 (0.0006) [2023-12-27 07:57:14,727][20347] Updated weights for policy 0, policy_version 333826 (0.0007) [2023-12-27 07:57:14,772][20381] Updated weights for policy 1, policy_version 333564 (0.0008) [2023-12-27 07:57:14,792][20347] Updated weights for policy 0, policy_version 333836 (0.0007) [2023-12-27 07:57:14,828][20381] Updated weights for policy 1, policy_version 333574 (0.0010) [2023-12-27 07:57:14,849][20347] Updated weights for policy 0, policy_version 333846 (0.0006) [2023-12-27 07:57:14,890][20381] Updated weights for policy 1, policy_version 333584 (0.0006) [2023-12-27 07:57:14,907][20347] Updated weights for policy 0, policy_version 333856 (0.0007) [2023-12-27 07:57:15,539][20381] Updated weights for policy 1, policy_version 333594 (0.0008) [2023-12-27 07:57:15,595][20381] Updated weights for policy 1, policy_version 333604 (0.0009) [2023-12-27 07:57:15,611][20347] Updated weights for policy 0, policy_version 333866 (0.0010) [2023-12-27 07:57:15,650][20381] Updated weights for policy 1, policy_version 333614 (0.0007) [2023-12-27 07:57:15,671][20347] Updated weights for policy 0, policy_version 333876 (0.0007) [2023-12-27 07:57:15,699][20381] Updated weights for policy 1, policy_version 333624 (0.0010) [2023-12-27 07:57:15,733][20347] Updated weights for policy 0, policy_version 333886 (0.0007) [2023-12-27 07:57:16,315][20381] Updated weights for policy 1, policy_version 333634 (0.0010) [2023-12-27 07:57:16,379][20381] Updated weights for policy 1, policy_version 333644 (0.0010) [2023-12-27 07:57:16,385][20347] Updated weights for policy 0, policy_version 333896 (0.0005) [2023-12-27 07:57:16,435][20381] Updated weights for policy 1, policy_version 333654 (0.0011) [2023-12-27 07:57:16,444][20347] Updated weights for policy 0, policy_version 333906 (0.0005) [2023-12-27 07:57:16,504][20347] Updated weights for policy 0, policy_version 333916 (0.0007) [2023-12-27 07:57:17,120][20381] Updated weights for policy 1, policy_version 333664 (0.0011) [2023-12-27 07:57:17,176][20381] Updated weights for policy 1, policy_version 333674 (0.0011) [2023-12-27 07:57:17,182][20347] Updated weights for policy 0, policy_version 333926 (0.0006) [2023-12-27 07:57:17,223][20381] Updated weights for policy 1, policy_version 333684 (0.0010) [2023-12-27 07:57:17,230][20347] Updated weights for policy 0, policy_version 333936 (0.0006) [2023-12-27 07:57:17,280][20347] Updated weights for policy 0, policy_version 333946 (0.0007) [2023-12-27 07:57:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 170950656. Throughput: 0: 10439.2, 1: 10264.6. Samples: 170954716. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:17,363][19170] Avg episode reward: [(0, '205.840'), (1, '1000.000')] [2023-12-27 07:57:17,923][20381] Updated weights for policy 1, policy_version 333694 (0.0011) [2023-12-27 07:57:17,972][20381] Updated weights for policy 1, policy_version 333704 (0.0010) [2023-12-27 07:57:17,989][20347] Updated weights for policy 0, policy_version 333956 (0.0007) [2023-12-27 07:57:18,025][20381] Updated weights for policy 1, policy_version 333714 (0.0011) [2023-12-27 07:57:18,035][20347] Updated weights for policy 0, policy_version 333966 (0.0005) [2023-12-27 07:57:18,088][20347] Updated weights for policy 0, policy_version 333976 (0.0007) [2023-12-27 07:57:18,727][20381] Updated weights for policy 1, policy_version 333724 (0.0011) [2023-12-27 07:57:18,777][20347] Updated weights for policy 0, policy_version 333986 (0.0007) [2023-12-27 07:57:18,787][20381] Updated weights for policy 1, policy_version 333734 (0.0011) [2023-12-27 07:57:18,830][20347] Updated weights for policy 0, policy_version 333996 (0.0005) [2023-12-27 07:57:18,840][20381] Updated weights for policy 1, policy_version 333744 (0.0011) [2023-12-27 07:57:18,890][20347] Updated weights for policy 0, policy_version 334006 (0.0006) [2023-12-27 07:57:18,946][20347] Updated weights for policy 0, policy_version 334016 (0.0008) [2023-12-27 07:57:19,509][20381] Updated weights for policy 1, policy_version 333754 (0.0011) [2023-12-27 07:57:19,555][20381] Updated weights for policy 1, policy_version 333764 (0.0010) [2023-12-27 07:57:19,604][20381] Updated weights for policy 1, policy_version 333774 (0.0010) [2023-12-27 07:57:19,654][20381] Updated weights for policy 1, policy_version 333784 (0.0011) [2023-12-27 07:57:19,683][20347] Updated weights for policy 0, policy_version 334026 (0.0007) [2023-12-27 07:57:19,742][20347] Updated weights for policy 0, policy_version 334036 (0.0005) [2023-12-27 07:57:19,804][20347] Updated weights for policy 0, policy_version 334046 (0.0005) [2023-12-27 07:57:20,333][20381] Updated weights for policy 1, policy_version 333794 (0.0006) [2023-12-27 07:57:20,392][20381] Updated weights for policy 1, policy_version 333804 (0.0008) [2023-12-27 07:57:20,424][20347] Updated weights for policy 0, policy_version 334056 (0.0006) [2023-12-27 07:57:20,448][20381] Updated weights for policy 1, policy_version 333814 (0.0006) [2023-12-27 07:57:20,472][20347] Updated weights for policy 0, policy_version 334066 (0.0006) [2023-12-27 07:57:20,517][20347] Updated weights for policy 0, policy_version 334076 (0.0007) [2023-12-27 07:57:21,159][20381] Updated weights for policy 1, policy_version 333824 (0.0008) [2023-12-27 07:57:21,209][20381] Updated weights for policy 1, policy_version 333834 (0.0007) [2023-12-27 07:57:21,237][20347] Updated weights for policy 0, policy_version 334086 (0.0007) [2023-12-27 07:57:21,262][20381] Updated weights for policy 1, policy_version 333844 (0.0006) [2023-12-27 07:57:21,301][20347] Updated weights for policy 0, policy_version 334096 (0.0007) [2023-12-27 07:57:21,358][20347] Updated weights for policy 0, policy_version 334106 (0.0008) [2023-12-27 07:57:22,028][20381] Updated weights for policy 1, policy_version 333854 (0.0008) [2023-12-27 07:57:22,056][20347] Updated weights for policy 0, policy_version 334116 (0.0008) [2023-12-27 07:57:22,085][20381] Updated weights for policy 1, policy_version 333864 (0.0006) [2023-12-27 07:57:22,113][20347] Updated weights for policy 0, policy_version 334126 (0.0006) [2023-12-27 07:57:22,143][20381] Updated weights for policy 1, policy_version 333874 (0.0006) [2023-12-27 07:57:22,175][20347] Updated weights for policy 0, policy_version 334136 (0.0006) [2023-12-27 07:57:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 171048960. Throughput: 0: 10387.2, 1: 10217.1. Samples: 171015528. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:22,363][19170] Avg episode reward: [(0, '276.610'), (1, '1000.000')] [2023-12-27 07:57:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000333880_85491712.pth... [2023-12-27 07:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000334144_85557248.pth... [2023-12-27 07:57:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000332696_85188608.pth [2023-12-27 07:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000332928_85245952.pth [2023-12-27 07:57:22,822][20381] Updated weights for policy 1, policy_version 333884 (0.0007) [2023-12-27 07:57:22,845][20347] Updated weights for policy 0, policy_version 334146 (0.0008) [2023-12-27 07:57:22,872][20381] Updated weights for policy 1, policy_version 333894 (0.0008) [2023-12-27 07:57:22,903][20347] Updated weights for policy 0, policy_version 334156 (0.0005) [2023-12-27 07:57:22,922][20381] Updated weights for policy 1, policy_version 333904 (0.0008) [2023-12-27 07:57:22,955][20347] Updated weights for policy 0, policy_version 334166 (0.0005) [2023-12-27 07:57:23,011][20347] Updated weights for policy 0, policy_version 334176 (0.0005) [2023-12-27 07:57:23,627][20347] Updated weights for policy 0, policy_version 334186 (0.0010) [2023-12-27 07:57:23,645][20381] Updated weights for policy 1, policy_version 333914 (0.0007) [2023-12-27 07:57:23,684][20347] Updated weights for policy 0, policy_version 334196 (0.0010) [2023-12-27 07:57:23,693][20381] Updated weights for policy 1, policy_version 333924 (0.0007) [2023-12-27 07:57:23,736][20347] Updated weights for policy 0, policy_version 334206 (0.0010) [2023-12-27 07:57:23,738][20381] Updated weights for policy 1, policy_version 333934 (0.0005) [2023-12-27 07:57:23,784][20381] Updated weights for policy 1, policy_version 333944 (0.0007) [2023-12-27 07:57:24,387][20347] Updated weights for policy 0, policy_version 334216 (0.0010) [2023-12-27 07:57:24,445][20347] Updated weights for policy 0, policy_version 334226 (0.0010) [2023-12-27 07:57:24,497][20347] Updated weights for policy 0, policy_version 334236 (0.0010) [2023-12-27 07:57:24,516][20381] Updated weights for policy 1, policy_version 333954 (0.0008) [2023-12-27 07:57:24,573][20381] Updated weights for policy 1, policy_version 333964 (0.0007) [2023-12-27 07:57:24,630][20381] Updated weights for policy 1, policy_version 333974 (0.0007) [2023-12-27 07:57:25,148][20347] Updated weights for policy 0, policy_version 334246 (0.0008) [2023-12-27 07:57:25,197][20347] Updated weights for policy 0, policy_version 334256 (0.0010) [2023-12-27 07:57:25,246][20347] Updated weights for policy 0, policy_version 334266 (0.0007) [2023-12-27 07:57:25,299][20381] Updated weights for policy 1, policy_version 333984 (0.0005) [2023-12-27 07:57:25,355][20381] Updated weights for policy 1, policy_version 333994 (0.0008) [2023-12-27 07:57:25,399][20381] Updated weights for policy 1, policy_version 334004 (0.0008) [2023-12-27 07:57:25,862][20347] Updated weights for policy 0, policy_version 334276 (0.0005) [2023-12-27 07:57:25,909][20347] Updated weights for policy 0, policy_version 334286 (0.0007) [2023-12-27 07:57:25,965][20347] Updated weights for policy 0, policy_version 334296 (0.0008) [2023-12-27 07:57:26,017][20381] Updated weights for policy 1, policy_version 334014 (0.0007) [2023-12-27 07:57:26,076][20381] Updated weights for policy 1, policy_version 334024 (0.0007) [2023-12-27 07:57:26,123][20381] Updated weights for policy 1, policy_version 334034 (0.0008) [2023-12-27 07:57:26,657][20347] Updated weights for policy 0, policy_version 334306 (0.0008) [2023-12-27 07:57:26,712][20347] Updated weights for policy 0, policy_version 334316 (0.0006) [2023-12-27 07:57:26,754][20381] Updated weights for policy 1, policy_version 334044 (0.0008) [2023-12-27 07:57:26,770][20347] Updated weights for policy 0, policy_version 334326 (0.0006) [2023-12-27 07:57:26,813][20381] Updated weights for policy 1, policy_version 334054 (0.0011) [2023-12-27 07:57:26,820][20347] Updated weights for policy 0, policy_version 334336 (0.0005) [2023-12-27 07:57:26,866][20381] Updated weights for policy 1, policy_version 334064 (0.0010) [2023-12-27 07:57:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 171147264. Throughput: 0: 10388.8, 1: 10186.0. Samples: 171138752. Policy #0 lag: (min: 13.0, avg: 20.6, max: 45.0) [2023-12-27 07:57:27,363][19170] Avg episode reward: [(0, '259.230'), (1, '1000.000')] [2023-12-27 07:57:27,498][20347] Updated weights for policy 0, policy_version 334346 (0.0005) [2023-12-27 07:57:27,534][20381] Updated weights for policy 1, policy_version 334074 (0.0009) [2023-12-27 07:57:27,557][20347] Updated weights for policy 0, policy_version 334356 (0.0005) [2023-12-27 07:57:27,587][20381] Updated weights for policy 1, policy_version 334084 (0.0006) [2023-12-27 07:57:27,618][20347] Updated weights for policy 0, policy_version 334366 (0.0005) [2023-12-27 07:57:27,641][20381] Updated weights for policy 1, policy_version 334094 (0.0006) [2023-12-27 07:57:27,685][20381] Updated weights for policy 1, policy_version 334104 (0.0005) [2023-12-27 07:57:28,243][20347] Updated weights for policy 0, policy_version 334376 (0.0007) [2023-12-27 07:57:28,295][20347] Updated weights for policy 0, policy_version 334386 (0.0006) [2023-12-27 07:57:28,308][20381] Updated weights for policy 1, policy_version 334114 (0.0006) [2023-12-27 07:57:28,353][20347] Updated weights for policy 0, policy_version 334396 (0.0006) [2023-12-27 07:57:28,368][20381] Updated weights for policy 1, policy_version 334124 (0.0006) [2023-12-27 07:57:28,422][20381] Updated weights for policy 1, policy_version 334134 (0.0005) [2023-12-27 07:57:28,948][20347] Updated weights for policy 0, policy_version 334406 (0.0008) [2023-12-27 07:57:29,003][20347] Updated weights for policy 0, policy_version 334416 (0.0010) [2023-12-27 07:57:29,029][20381] Updated weights for policy 1, policy_version 334144 (0.0007) [2023-12-27 07:57:29,062][20347] Updated weights for policy 0, policy_version 334426 (0.0007) [2023-12-27 07:57:29,095][20381] Updated weights for policy 1, policy_version 334154 (0.0008) [2023-12-27 07:57:29,146][20381] Updated weights for policy 1, policy_version 334164 (0.0010) [2023-12-27 07:57:29,708][20347] Updated weights for policy 0, policy_version 334436 (0.0006) [2023-12-27 07:57:29,769][20347] Updated weights for policy 0, policy_version 334446 (0.0005) [2023-12-27 07:57:29,822][20347] Updated weights for policy 0, policy_version 334456 (0.0007) [2023-12-27 07:57:29,855][20381] Updated weights for policy 1, policy_version 334174 (0.0010) [2023-12-27 07:57:29,911][20381] Updated weights for policy 1, policy_version 334184 (0.0007) [2023-12-27 07:57:29,969][20381] Updated weights for policy 1, policy_version 334194 (0.0007) [2023-12-27 07:57:30,489][20347] Updated weights for policy 0, policy_version 334466 (0.0010) [2023-12-27 07:57:30,555][20347] Updated weights for policy 0, policy_version 334476 (0.0010) [2023-12-27 07:57:30,615][20347] Updated weights for policy 0, policy_version 334486 (0.0010) [2023-12-27 07:57:30,628][20381] Updated weights for policy 1, policy_version 334204 (0.0005) [2023-12-27 07:57:30,670][20347] Updated weights for policy 0, policy_version 334496 (0.0010) [2023-12-27 07:57:30,674][20381] Updated weights for policy 1, policy_version 334214 (0.0005) [2023-12-27 07:57:30,732][20381] Updated weights for policy 1, policy_version 334224 (0.0007) [2023-12-27 07:57:31,323][20347] Updated weights for policy 0, policy_version 334506 (0.0008) [2023-12-27 07:57:31,383][20347] Updated weights for policy 0, policy_version 334516 (0.0008) [2023-12-27 07:57:31,420][20381] Updated weights for policy 1, policy_version 334234 (0.0008) [2023-12-27 07:57:31,445][20347] Updated weights for policy 0, policy_version 334526 (0.0008) [2023-12-27 07:57:31,474][20381] Updated weights for policy 1, policy_version 334244 (0.0011) [2023-12-27 07:57:31,529][20381] Updated weights for policy 1, policy_version 334254 (0.0010) [2023-12-27 07:57:31,573][20381] Updated weights for policy 1, policy_version 334264 (0.0010) [2023-12-27 07:57:32,163][20347] Updated weights for policy 0, policy_version 334536 (0.0010) [2023-12-27 07:57:32,224][20347] Updated weights for policy 0, policy_version 334546 (0.0010) [2023-12-27 07:57:32,262][20381] Updated weights for policy 1, policy_version 334274 (0.0010) [2023-12-27 07:57:32,289][20347] Updated weights for policy 0, policy_version 334556 (0.0011) [2023-12-27 07:57:32,322][20381] Updated weights for policy 1, policy_version 334284 (0.0009) [2023-12-27 07:57:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 171253760. Throughput: 0: 10405.2, 1: 10242.4. Samples: 171265104. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:32,363][19170] Avg episode reward: [(0, '287.150'), (1, '1000.000')] [2023-12-27 07:57:32,391][20381] Updated weights for policy 1, policy_version 334294 (0.0008) [2023-12-27 07:57:32,959][20347] Updated weights for policy 0, policy_version 334566 (0.0010) [2023-12-27 07:57:32,966][20381] Updated weights for policy 1, policy_version 334304 (0.0006) [2023-12-27 07:57:33,016][20347] Updated weights for policy 0, policy_version 334576 (0.0010) [2023-12-27 07:57:33,022][20381] Updated weights for policy 1, policy_version 334314 (0.0010) [2023-12-27 07:57:33,073][20381] Updated weights for policy 1, policy_version 334324 (0.0010) [2023-12-27 07:57:33,074][20347] Updated weights for policy 0, policy_version 334586 (0.0010) [2023-12-27 07:57:33,655][20381] Updated weights for policy 1, policy_version 334334 (0.0007) [2023-12-27 07:57:33,705][20381] Updated weights for policy 1, policy_version 334344 (0.0005) [2023-12-27 07:57:33,737][20347] Updated weights for policy 0, policy_version 334596 (0.0010) [2023-12-27 07:57:33,756][20381] Updated weights for policy 1, policy_version 334354 (0.0009) [2023-12-27 07:57:33,784][20347] Updated weights for policy 0, policy_version 334606 (0.0010) [2023-12-27 07:57:33,829][20347] Updated weights for policy 0, policy_version 334616 (0.0010) [2023-12-27 07:57:34,340][20381] Updated weights for policy 1, policy_version 334364 (0.0009) [2023-12-27 07:57:34,402][20381] Updated weights for policy 1, policy_version 334374 (0.0008) [2023-12-27 07:57:34,467][20381] Updated weights for policy 1, policy_version 334384 (0.0007) [2023-12-27 07:57:34,510][20347] Updated weights for policy 0, policy_version 334626 (0.0010) [2023-12-27 07:57:34,565][20347] Updated weights for policy 0, policy_version 334636 (0.0010) [2023-12-27 07:57:34,614][20347] Updated weights for policy 0, policy_version 334646 (0.0009) [2023-12-27 07:57:34,661][20347] Updated weights for policy 0, policy_version 334656 (0.0010) [2023-12-27 07:57:35,094][20381] Updated weights for policy 1, policy_version 334394 (0.0006) [2023-12-27 07:57:35,157][20381] Updated weights for policy 1, policy_version 334404 (0.0005) [2023-12-27 07:57:35,163][20292] KL-divergence is very high: 221.5975 [2023-12-27 07:57:35,192][20292] KL-divergence is very high: 215.4895 [2023-12-27 07:57:35,212][20381] Updated weights for policy 1, policy_version 334414 (0.0007) [2023-12-27 07:57:35,265][20381] Updated weights for policy 1, policy_version 334424 (0.0006) [2023-12-27 07:57:35,370][20347] Updated weights for policy 0, policy_version 334666 (0.0010) [2023-12-27 07:57:35,425][20347] Updated weights for policy 0, policy_version 334676 (0.0010) [2023-12-27 07:57:35,479][20347] Updated weights for policy 0, policy_version 334686 (0.0010) [2023-12-27 07:57:35,917][20381] Updated weights for policy 1, policy_version 334434 (0.0008) [2023-12-27 07:57:35,967][20381] Updated weights for policy 1, policy_version 334444 (0.0008) [2023-12-27 07:57:36,029][20381] Updated weights for policy 1, policy_version 334454 (0.0007) [2023-12-27 07:57:36,103][20347] Updated weights for policy 0, policy_version 334696 (0.0010) [2023-12-27 07:57:36,170][20347] Updated weights for policy 0, policy_version 334706 (0.0010) [2023-12-27 07:57:36,230][20347] Updated weights for policy 0, policy_version 334716 (0.0011) [2023-12-27 07:57:36,721][20381] Updated weights for policy 1, policy_version 334464 (0.0010) [2023-12-27 07:57:36,779][20381] Updated weights for policy 1, policy_version 334474 (0.0011) [2023-12-27 07:57:36,838][20381] Updated weights for policy 1, policy_version 334484 (0.0011) [2023-12-27 07:57:36,929][20347] Updated weights for policy 0, policy_version 334726 (0.0009) [2023-12-27 07:57:36,987][20347] Updated weights for policy 0, policy_version 334736 (0.0010) [2023-12-27 07:57:37,049][20347] Updated weights for policy 0, policy_version 334746 (0.0010) [2023-12-27 07:57:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 171360256. Throughput: 0: 10384.9, 1: 10347.0. Samples: 171329888. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:37,363][19170] Avg episode reward: [(0, '304.520'), (1, '971.210')] [2023-12-27 07:57:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000334488_85647360.pth... [2023-12-27 07:57:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000334752_85712896.pth... [2023-12-27 07:57:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000333304_85344256.pth [2023-12-27 07:57:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000333536_85401600.pth [2023-12-27 07:57:37,521][20381] Updated weights for policy 1, policy_version 334494 (0.0011) [2023-12-27 07:57:37,587][20381] Updated weights for policy 1, policy_version 334504 (0.0011) [2023-12-27 07:57:37,650][20381] Updated weights for policy 1, policy_version 334514 (0.0011) [2023-12-27 07:57:37,732][20347] Updated weights for policy 0, policy_version 334756 (0.0009) [2023-12-27 07:57:37,787][20347] Updated weights for policy 0, policy_version 334766 (0.0007) [2023-12-27 07:57:37,835][20347] Updated weights for policy 0, policy_version 334776 (0.0006) [2023-12-27 07:57:38,317][20381] Updated weights for policy 1, policy_version 334524 (0.0009) [2023-12-27 07:57:38,373][20381] Updated weights for policy 1, policy_version 334534 (0.0007) [2023-12-27 07:57:38,421][20381] Updated weights for policy 1, policy_version 334544 (0.0005) [2023-12-27 07:57:38,486][20347] Updated weights for policy 0, policy_version 334786 (0.0006) [2023-12-27 07:57:38,538][20347] Updated weights for policy 0, policy_version 334796 (0.0008) [2023-12-27 07:57:38,599][20347] Updated weights for policy 0, policy_version 334806 (0.0008) [2023-12-27 07:57:38,664][20347] Updated weights for policy 0, policy_version 334816 (0.0009) [2023-12-27 07:57:39,030][20381] Updated weights for policy 1, policy_version 334554 (0.0005) [2023-12-27 07:57:39,090][20381] Updated weights for policy 1, policy_version 334564 (0.0008) [2023-12-27 07:57:39,154][20381] Updated weights for policy 1, policy_version 334574 (0.0010) [2023-12-27 07:57:39,212][20381] Updated weights for policy 1, policy_version 334584 (0.0010) [2023-12-27 07:57:39,330][20347] Updated weights for policy 0, policy_version 334826 (0.0010) [2023-12-27 07:57:39,395][20347] Updated weights for policy 0, policy_version 334836 (0.0011) [2023-12-27 07:57:39,458][20347] Updated weights for policy 0, policy_version 334846 (0.0007) [2023-12-27 07:57:39,907][20381] Updated weights for policy 1, policy_version 334594 (0.0010) [2023-12-27 07:57:39,970][20381] Updated weights for policy 1, policy_version 334604 (0.0009) [2023-12-27 07:57:40,035][20381] Updated weights for policy 1, policy_version 334614 (0.0006) [2023-12-27 07:57:40,152][20347] Updated weights for policy 0, policy_version 334856 (0.0007) [2023-12-27 07:57:40,212][20347] Updated weights for policy 0, policy_version 334866 (0.0006) [2023-12-27 07:57:40,273][20347] Updated weights for policy 0, policy_version 334876 (0.0007) [2023-12-27 07:57:40,714][20381] Updated weights for policy 1, policy_version 334624 (0.0010) [2023-12-27 07:57:40,769][20381] Updated weights for policy 1, policy_version 334634 (0.0010) [2023-12-27 07:57:40,824][20381] Updated weights for policy 1, policy_version 334644 (0.0011) [2023-12-27 07:57:40,942][20347] Updated weights for policy 0, policy_version 334886 (0.0007) [2023-12-27 07:57:40,993][20347] Updated weights for policy 0, policy_version 334896 (0.0007) [2023-12-27 07:57:41,056][20347] Updated weights for policy 0, policy_version 334906 (0.0008) [2023-12-27 07:57:41,544][20381] Updated weights for policy 1, policy_version 334654 (0.0009) [2023-12-27 07:57:41,601][20381] Updated weights for policy 1, policy_version 334664 (0.0010) [2023-12-27 07:57:41,668][20381] Updated weights for policy 1, policy_version 334674 (0.0011) [2023-12-27 07:57:41,744][20347] Updated weights for policy 0, policy_version 334916 (0.0009) [2023-12-27 07:57:41,807][20347] Updated weights for policy 0, policy_version 334926 (0.0010) [2023-12-27 07:57:41,870][20347] Updated weights for policy 0, policy_version 334936 (0.0010) [2023-12-27 07:57:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 171458560. Throughput: 0: 10361.8, 1: 10342.3. Samples: 171452984. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:42,363][19170] Avg episode reward: [(0, '341.940'), (1, '961.530')] [2023-12-27 07:57:42,368][20381] Updated weights for policy 1, policy_version 334684 (0.0009) [2023-12-27 07:57:42,429][20381] Updated weights for policy 1, policy_version 334694 (0.0009) [2023-12-27 07:57:42,481][20381] Updated weights for policy 1, policy_version 334704 (0.0007) [2023-12-27 07:57:42,544][20347] Updated weights for policy 0, policy_version 334946 (0.0008) [2023-12-27 07:57:42,606][20347] Updated weights for policy 0, policy_version 334956 (0.0007) [2023-12-27 07:57:42,667][20347] Updated weights for policy 0, policy_version 334966 (0.0006) [2023-12-27 07:57:42,728][20347] Updated weights for policy 0, policy_version 334976 (0.0005) [2023-12-27 07:57:43,112][20381] Updated weights for policy 1, policy_version 334714 (0.0006) [2023-12-27 07:57:43,163][20381] Updated weights for policy 1, policy_version 334724 (0.0005) [2023-12-27 07:57:43,215][20381] Updated weights for policy 1, policy_version 334734 (0.0005) [2023-12-27 07:57:43,259][20381] Updated weights for policy 1, policy_version 334744 (0.0005) [2023-12-27 07:57:43,282][20347] Updated weights for policy 0, policy_version 334986 (0.0010) [2023-12-27 07:57:43,341][20347] Updated weights for policy 0, policy_version 334996 (0.0010) [2023-12-27 07:57:43,391][20347] Updated weights for policy 0, policy_version 335006 (0.0007) [2023-12-27 07:57:43,805][20381] Updated weights for policy 1, policy_version 334754 (0.0005) [2023-12-27 07:57:43,861][20381] Updated weights for policy 1, policy_version 334764 (0.0005) [2023-12-27 07:57:43,915][20381] Updated weights for policy 1, policy_version 334774 (0.0005) [2023-12-27 07:57:44,066][20347] Updated weights for policy 0, policy_version 335016 (0.0006) [2023-12-27 07:57:44,113][20347] Updated weights for policy 0, policy_version 335026 (0.0006) [2023-12-27 07:57:44,158][20347] Updated weights for policy 0, policy_version 335036 (0.0008) [2023-12-27 07:57:44,536][20381] Updated weights for policy 1, policy_version 334784 (0.0008) [2023-12-27 07:57:44,581][20381] Updated weights for policy 1, policy_version 334794 (0.0009) [2023-12-27 07:57:44,630][20381] Updated weights for policy 1, policy_version 334804 (0.0009) [2023-12-27 07:57:44,772][20347] Updated weights for policy 0, policy_version 335046 (0.0007) [2023-12-27 07:57:44,831][20347] Updated weights for policy 0, policy_version 335056 (0.0008) [2023-12-27 07:57:44,886][20347] Updated weights for policy 0, policy_version 335066 (0.0007) [2023-12-27 07:57:45,268][20381] Updated weights for policy 1, policy_version 334814 (0.0010) [2023-12-27 07:57:45,327][20381] Updated weights for policy 1, policy_version 334824 (0.0007) [2023-12-27 07:57:45,384][20381] Updated weights for policy 1, policy_version 334834 (0.0008) [2023-12-27 07:57:45,547][20347] Updated weights for policy 0, policy_version 335076 (0.0008) [2023-12-27 07:57:45,606][20347] Updated weights for policy 0, policy_version 335086 (0.0008) [2023-12-27 07:57:45,665][20347] Updated weights for policy 0, policy_version 335096 (0.0007) [2023-12-27 07:57:46,035][20381] Updated weights for policy 1, policy_version 334844 (0.0007) [2023-12-27 07:57:46,086][20381] Updated weights for policy 1, policy_version 334854 (0.0005) [2023-12-27 07:57:46,135][20381] Updated weights for policy 1, policy_version 334864 (0.0006) [2023-12-27 07:57:46,265][20347] Updated weights for policy 0, policy_version 335106 (0.0007) [2023-12-27 07:57:46,314][20347] Updated weights for policy 0, policy_version 335116 (0.0008) [2023-12-27 07:57:46,382][20347] Updated weights for policy 0, policy_version 335126 (0.0008) [2023-12-27 07:57:46,444][20347] Updated weights for policy 0, policy_version 335136 (0.0006) [2023-12-27 07:57:46,740][20381] Updated weights for policy 1, policy_version 334874 (0.0010) [2023-12-27 07:57:46,784][20381] Updated weights for policy 1, policy_version 334884 (0.0008) [2023-12-27 07:57:46,831][20381] Updated weights for policy 1, policy_version 334894 (0.0009) [2023-12-27 07:57:46,886][20381] Updated weights for policy 1, policy_version 334904 (0.0010) [2023-12-27 07:57:47,098][20347] Updated weights for policy 0, policy_version 335146 (0.0006) [2023-12-27 07:57:47,147][20347] Updated weights for policy 0, policy_version 335156 (0.0006) [2023-12-27 07:57:47,189][20347] Updated weights for policy 0, policy_version 335166 (0.0005) [2023-12-27 07:57:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 171573248. Throughput: 0: 10411.3, 1: 10380.1. Samples: 171581612. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:47,364][19170] Avg episode reward: [(0, '386.340'), (1, '961.410')] [2023-12-27 07:57:47,581][20381] Updated weights for policy 1, policy_version 334914 (0.0006) [2023-12-27 07:57:47,639][20381] Updated weights for policy 1, policy_version 334924 (0.0006) [2023-12-27 07:57:47,695][20381] Updated weights for policy 1, policy_version 334934 (0.0008) [2023-12-27 07:57:47,903][20347] Updated weights for policy 0, policy_version 335176 (0.0007) [2023-12-27 07:57:47,946][20347] Updated weights for policy 0, policy_version 335186 (0.0008) [2023-12-27 07:57:47,996][20347] Updated weights for policy 0, policy_version 335196 (0.0008) [2023-12-27 07:57:48,320][20381] Updated weights for policy 1, policy_version 334944 (0.0006) [2023-12-27 07:57:48,378][20381] Updated weights for policy 1, policy_version 334954 (0.0008) [2023-12-27 07:57:48,440][20381] Updated weights for policy 1, policy_version 334964 (0.0007) [2023-12-27 07:57:48,729][20347] Updated weights for policy 0, policy_version 335206 (0.0009) [2023-12-27 07:57:48,793][20347] Updated weights for policy 0, policy_version 335216 (0.0009) [2023-12-27 07:57:48,852][20347] Updated weights for policy 0, policy_version 335226 (0.0007) [2023-12-27 07:57:49,061][20381] Updated weights for policy 1, policy_version 334974 (0.0009) [2023-12-27 07:57:49,119][20381] Updated weights for policy 1, policy_version 334984 (0.0010) [2023-12-27 07:57:49,179][20381] Updated weights for policy 1, policy_version 334994 (0.0010) [2023-12-27 07:57:49,523][20347] Updated weights for policy 0, policy_version 335236 (0.0007) [2023-12-27 07:57:49,583][20347] Updated weights for policy 0, policy_version 335246 (0.0007) [2023-12-27 07:57:49,645][20347] Updated weights for policy 0, policy_version 335256 (0.0007) [2023-12-27 07:57:49,845][20381] Updated weights for policy 1, policy_version 335004 (0.0010) [2023-12-27 07:57:49,906][20381] Updated weights for policy 1, policy_version 335014 (0.0009) [2023-12-27 07:57:49,972][20381] Updated weights for policy 1, policy_version 335024 (0.0009) [2023-12-27 07:57:50,361][20347] Updated weights for policy 0, policy_version 335266 (0.0007) [2023-12-27 07:57:50,411][20347] Updated weights for policy 0, policy_version 335276 (0.0008) [2023-12-27 07:57:50,465][20347] Updated weights for policy 0, policy_version 335286 (0.0006) [2023-12-27 07:57:50,516][20347] Updated weights for policy 0, policy_version 335296 (0.0006) [2023-12-27 07:57:50,622][20381] Updated weights for policy 1, policy_version 335034 (0.0008) [2023-12-27 07:57:50,684][20381] Updated weights for policy 1, policy_version 335044 (0.0009) [2023-12-27 07:57:50,748][20381] Updated weights for policy 1, policy_version 335054 (0.0007) [2023-12-27 07:57:50,808][20381] Updated weights for policy 1, policy_version 335064 (0.0006) [2023-12-27 07:57:51,128][20347] Updated weights for policy 0, policy_version 335306 (0.0008) [2023-12-27 07:57:51,177][20347] Updated weights for policy 0, policy_version 335316 (0.0005) [2023-12-27 07:57:51,223][20347] Updated weights for policy 0, policy_version 335326 (0.0005) [2023-12-27 07:57:51,502][20381] Updated weights for policy 1, policy_version 335074 (0.0008) [2023-12-27 07:57:51,549][20381] Updated weights for policy 1, policy_version 335084 (0.0008) [2023-12-27 07:57:51,603][20381] Updated weights for policy 1, policy_version 335094 (0.0008) [2023-12-27 07:57:51,899][20347] Updated weights for policy 0, policy_version 335336 (0.0007) [2023-12-27 07:57:51,952][20347] Updated weights for policy 0, policy_version 335346 (0.0008) [2023-12-27 07:57:52,006][20347] Updated weights for policy 0, policy_version 335356 (0.0007) [2023-12-27 07:57:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 171671552. Throughput: 0: 10332.4, 1: 10370.9. Samples: 171643200. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:52,363][19170] Avg episode reward: [(0, '318.530'), (1, '980.690')] [2023-12-27 07:57:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000335360_85868544.pth... [2023-12-27 07:57:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000334144_85557248.pth [2023-12-27 07:57:52,383][20381] Updated weights for policy 1, policy_version 335104 (0.0008) [2023-12-27 07:57:52,442][20381] Updated weights for policy 1, policy_version 335114 (0.0008) [2023-12-27 07:57:52,495][20381] Updated weights for policy 1, policy_version 335124 (0.0008) [2023-12-27 07:57:52,513][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000335128_85811200.pth... [2023-12-27 07:57:52,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000333880_85491712.pth [2023-12-27 07:57:52,651][20347] Updated weights for policy 0, policy_version 335366 (0.0006) [2023-12-27 07:57:52,706][20347] Updated weights for policy 0, policy_version 335376 (0.0005) [2023-12-27 07:57:52,764][20347] Updated weights for policy 0, policy_version 335386 (0.0007) [2023-12-27 07:57:53,225][20381] Updated weights for policy 1, policy_version 335134 (0.0008) [2023-12-27 07:57:53,280][20381] Updated weights for policy 1, policy_version 335144 (0.0007) [2023-12-27 07:57:53,337][20381] Updated weights for policy 1, policy_version 335154 (0.0008) [2023-12-27 07:57:53,399][20347] Updated weights for policy 0, policy_version 335396 (0.0009) [2023-12-27 07:57:53,451][20347] Updated weights for policy 0, policy_version 335406 (0.0010) [2023-12-27 07:57:53,509][20347] Updated weights for policy 0, policy_version 335416 (0.0010) [2023-12-27 07:57:53,970][20381] Updated weights for policy 1, policy_version 335164 (0.0007) [2023-12-27 07:57:54,017][20381] Updated weights for policy 1, policy_version 335174 (0.0008) [2023-12-27 07:57:54,070][20381] Updated weights for policy 1, policy_version 335184 (0.0008) [2023-12-27 07:57:54,133][20347] Updated weights for policy 0, policy_version 335426 (0.0009) [2023-12-27 07:57:54,218][20347] Updated weights for policy 0, policy_version 335436 (0.0007) [2023-12-27 07:57:54,279][20347] Updated weights for policy 0, policy_version 335446 (0.0005) [2023-12-27 07:57:54,336][20347] Updated weights for policy 0, policy_version 335456 (0.0005) [2023-12-27 07:57:54,838][20347] Updated weights for policy 0, policy_version 335466 (0.0008) [2023-12-27 07:57:54,840][20381] Updated weights for policy 1, policy_version 335194 (0.0008) [2023-12-27 07:57:54,889][20347] Updated weights for policy 0, policy_version 335476 (0.0007) [2023-12-27 07:57:54,898][20381] Updated weights for policy 1, policy_version 335204 (0.0007) [2023-12-27 07:57:54,914][20292] KL-divergence is very high: 250.8574 [2023-12-27 07:57:54,944][20347] Updated weights for policy 0, policy_version 335486 (0.0005) [2023-12-27 07:57:54,954][20381] Updated weights for policy 1, policy_version 335214 (0.0005) [2023-12-27 07:57:54,961][20292] KL-divergence is very high: 130.3594 [2023-12-27 07:57:55,617][20347] Updated weights for policy 0, policy_version 335496 (0.0007) [2023-12-27 07:57:55,671][20381] Updated weights for policy 1, policy_version 335225 (0.0008) [2023-12-27 07:57:55,676][20347] Updated weights for policy 0, policy_version 335506 (0.0007) [2023-12-27 07:57:55,729][20347] Updated weights for policy 0, policy_version 335516 (0.0007) [2023-12-27 07:57:55,733][20381] Updated weights for policy 1, policy_version 335235 (0.0007) [2023-12-27 07:57:55,791][20381] Updated weights for policy 1, policy_version 335245 (0.0007) [2023-12-27 07:57:55,846][20381] Updated weights for policy 1, policy_version 335255 (0.0009) [2023-12-27 07:57:56,359][20347] Updated weights for policy 0, policy_version 335526 (0.0007) [2023-12-27 07:57:56,406][20347] Updated weights for policy 0, policy_version 335536 (0.0007) [2023-12-27 07:57:56,460][20347] Updated weights for policy 0, policy_version 335546 (0.0007) [2023-12-27 07:57:56,543][20381] Updated weights for policy 1, policy_version 335265 (0.0010) [2023-12-27 07:57:56,587][20381] Updated weights for policy 1, policy_version 335275 (0.0010) [2023-12-27 07:57:56,637][20381] Updated weights for policy 1, policy_version 335285 (0.0008) [2023-12-27 07:57:57,215][20347] Updated weights for policy 0, policy_version 335556 (0.0007) [2023-12-27 07:57:57,266][20347] Updated weights for policy 0, policy_version 335566 (0.0007) [2023-12-27 07:57:57,298][20381] Updated weights for policy 1, policy_version 335295 (0.0005) [2023-12-27 07:57:57,325][20347] Updated weights for policy 0, policy_version 335576 (0.0007) [2023-12-27 07:57:57,355][20381] Updated weights for policy 1, policy_version 335305 (0.0007) [2023-12-27 07:57:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 171769856. Throughput: 0: 10420.3, 1: 10357.6. Samples: 171766964. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:57:57,363][19170] Avg episode reward: [(0, '299.350'), (1, '990.380')] [2023-12-27 07:57:57,415][20381] Updated weights for policy 1, policy_version 335315 (0.0008) [2023-12-27 07:57:58,016][20381] Updated weights for policy 1, policy_version 335325 (0.0007) [2023-12-27 07:57:58,055][20347] Updated weights for policy 0, policy_version 335586 (0.0005) [2023-12-27 07:57:58,063][20381] Updated weights for policy 1, policy_version 335335 (0.0007) [2023-12-27 07:57:58,111][20347] Updated weights for policy 0, policy_version 335596 (0.0007) [2023-12-27 07:57:58,121][20381] Updated weights for policy 1, policy_version 335345 (0.0008) [2023-12-27 07:57:58,168][20347] Updated weights for policy 0, policy_version 335606 (0.0007) [2023-12-27 07:57:58,232][20347] Updated weights for policy 0, policy_version 335616 (0.0006) [2023-12-27 07:57:58,962][20381] Updated weights for policy 1, policy_version 335355 (0.0007) [2023-12-27 07:57:59,030][20347] Updated weights for policy 0, policy_version 335626 (0.0008) [2023-12-27 07:57:59,035][20381] Updated weights for policy 1, policy_version 335365 (0.0009) [2023-12-27 07:57:59,091][20347] Updated weights for policy 0, policy_version 335636 (0.0007) [2023-12-27 07:57:59,101][20381] Updated weights for policy 1, policy_version 335375 (0.0008) [2023-12-27 07:57:59,158][20347] Updated weights for policy 0, policy_version 335646 (0.0007) [2023-12-27 07:57:59,774][20381] Updated weights for policy 1, policy_version 335385 (0.0007) [2023-12-27 07:57:59,830][20347] Updated weights for policy 0, policy_version 335656 (0.0006) [2023-12-27 07:57:59,833][20381] Updated weights for policy 1, policy_version 335395 (0.0011) [2023-12-27 07:57:59,859][20292] KL-divergence is very high: 205.4981 [2023-12-27 07:57:59,891][20347] Updated weights for policy 0, policy_version 335666 (0.0006) [2023-12-27 07:57:59,893][20381] Updated weights for policy 1, policy_version 335405 (0.0007) [2023-12-27 07:57:59,950][20381] Updated weights for policy 1, policy_version 335415 (0.0010) [2023-12-27 07:57:59,955][20347] Updated weights for policy 0, policy_version 335676 (0.0006) [2023-12-27 07:58:00,556][20381] Updated weights for policy 1, policy_version 335425 (0.0010) [2023-12-27 07:58:00,603][20381] Updated weights for policy 1, policy_version 335435 (0.0008) [2023-12-27 07:58:00,631][20347] Updated weights for policy 0, policy_version 335686 (0.0007) [2023-12-27 07:58:00,647][20381] Updated weights for policy 1, policy_version 335445 (0.0010) [2023-12-27 07:58:00,678][20347] Updated weights for policy 0, policy_version 335696 (0.0008) [2023-12-27 07:58:00,735][20347] Updated weights for policy 0, policy_version 335706 (0.0008) [2023-12-27 07:58:01,315][20381] Updated weights for policy 1, policy_version 335455 (0.0005) [2023-12-27 07:58:01,376][20381] Updated weights for policy 1, policy_version 335465 (0.0010) [2023-12-27 07:58:01,439][20381] Updated weights for policy 1, policy_version 335475 (0.0010) [2023-12-27 07:58:01,467][20347] Updated weights for policy 0, policy_version 335716 (0.0007) [2023-12-27 07:58:01,531][20347] Updated weights for policy 0, policy_version 335726 (0.0007) [2023-12-27 07:58:01,579][20347] Updated weights for policy 0, policy_version 335736 (0.0008) [2023-12-27 07:58:02,105][20381] Updated weights for policy 1, policy_version 335485 (0.0006) [2023-12-27 07:58:02,166][20381] Updated weights for policy 1, policy_version 335495 (0.0006) [2023-12-27 07:58:02,218][20381] Updated weights for policy 1, policy_version 335505 (0.0005) [2023-12-27 07:58:02,325][20347] Updated weights for policy 0, policy_version 335746 (0.0008) [2023-12-27 07:58:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 171876352. Throughput: 0: 10330.1, 1: 10373.0. Samples: 171886352. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:02,363][19170] Avg episode reward: [(0, '293.830'), (1, '951.930')] [2023-12-27 07:58:02,387][20347] Updated weights for policy 0, policy_version 335756 (0.0009) [2023-12-27 07:58:02,443][20347] Updated weights for policy 0, policy_version 335766 (0.0009) [2023-12-27 07:58:02,499][20347] Updated weights for policy 0, policy_version 335776 (0.0007) [2023-12-27 07:58:02,754][20381] Updated weights for policy 1, policy_version 335515 (0.0006) [2023-12-27 07:58:02,813][20381] Updated weights for policy 1, policy_version 335525 (0.0006) [2023-12-27 07:58:02,872][20381] Updated weights for policy 1, policy_version 335535 (0.0006) [2023-12-27 07:58:03,168][20347] Updated weights for policy 0, policy_version 335786 (0.0005) [2023-12-27 07:58:03,222][20347] Updated weights for policy 0, policy_version 335796 (0.0006) [2023-12-27 07:58:03,275][20347] Updated weights for policy 0, policy_version 335806 (0.0010) [2023-12-27 07:58:03,462][20381] Updated weights for policy 1, policy_version 335545 (0.0005) [2023-12-27 07:58:03,517][20381] Updated weights for policy 1, policy_version 335555 (0.0009) [2023-12-27 07:58:03,565][20381] Updated weights for policy 1, policy_version 335565 (0.0010) [2023-12-27 07:58:03,614][20381] Updated weights for policy 1, policy_version 335575 (0.0008) [2023-12-27 07:58:03,897][20347] Updated weights for policy 0, policy_version 335816 (0.0008) [2023-12-27 07:58:03,954][20347] Updated weights for policy 0, policy_version 335826 (0.0008) [2023-12-27 07:58:04,012][20347] Updated weights for policy 0, policy_version 335836 (0.0011) [2023-12-27 07:58:04,253][20381] Updated weights for policy 1, policy_version 335585 (0.0010) [2023-12-27 07:58:04,314][20381] Updated weights for policy 1, policy_version 335595 (0.0009) [2023-12-27 07:58:04,377][20381] Updated weights for policy 1, policy_version 335605 (0.0008) [2023-12-27 07:58:04,704][20347] Updated weights for policy 0, policy_version 335846 (0.0010) [2023-12-27 07:58:04,749][20347] Updated weights for policy 0, policy_version 335856 (0.0010) [2023-12-27 07:58:04,805][20347] Updated weights for policy 0, policy_version 335866 (0.0010) [2023-12-27 07:58:05,069][20381] Updated weights for policy 1, policy_version 335615 (0.0007) [2023-12-27 07:58:05,132][20381] Updated weights for policy 1, policy_version 335625 (0.0008) [2023-12-27 07:58:05,187][20381] Updated weights for policy 1, policy_version 335635 (0.0008) [2023-12-27 07:58:05,509][20347] Updated weights for policy 0, policy_version 335876 (0.0010) [2023-12-27 07:58:05,566][20347] Updated weights for policy 0, policy_version 335886 (0.0010) [2023-12-27 07:58:05,611][20347] Updated weights for policy 0, policy_version 335896 (0.0010) [2023-12-27 07:58:05,849][20381] Updated weights for policy 1, policy_version 335645 (0.0006) [2023-12-27 07:58:05,906][20381] Updated weights for policy 1, policy_version 335655 (0.0005) [2023-12-27 07:58:05,967][20381] Updated weights for policy 1, policy_version 335665 (0.0011) [2023-12-27 07:58:06,263][20347] Updated weights for policy 0, policy_version 335906 (0.0010) [2023-12-27 07:58:06,308][20347] Updated weights for policy 0, policy_version 335916 (0.0010) [2023-12-27 07:58:06,360][20347] Updated weights for policy 0, policy_version 335926 (0.0010) [2023-12-27 07:58:06,421][20347] Updated weights for policy 0, policy_version 335936 (0.0010) [2023-12-27 07:58:06,623][20381] Updated weights for policy 1, policy_version 335675 (0.0011) [2023-12-27 07:58:06,674][20381] Updated weights for policy 1, policy_version 335685 (0.0011) [2023-12-27 07:58:06,674][20292] KL-divergence is very high: 174.2323 [2023-12-27 07:58:06,731][20381] Updated weights for policy 1, policy_version 335695 (0.0011) [2023-12-27 07:58:07,113][20347] Updated weights for policy 0, policy_version 335946 (0.0008) [2023-12-27 07:58:07,176][20347] Updated weights for policy 0, policy_version 335956 (0.0010) [2023-12-27 07:58:07,234][20347] Updated weights for policy 0, policy_version 335966 (0.0006) [2023-12-27 07:58:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 171982848. Throughput: 0: 10348.5, 1: 10416.6. Samples: 171949952. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:07,363][19170] Avg episode reward: [(0, '286.330'), (1, '951.890')] [2023-12-27 07:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000335968_86024192.pth... [2023-12-27 07:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000335704_85958656.pth... [2023-12-27 07:58:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000334488_85647360.pth [2023-12-27 07:58:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000334752_85712896.pth [2023-12-27 07:58:07,469][20381] Updated weights for policy 1, policy_version 335705 (0.0011) [2023-12-27 07:58:07,520][20381] Updated weights for policy 1, policy_version 335715 (0.0010) [2023-12-27 07:58:07,571][20381] Updated weights for policy 1, policy_version 335725 (0.0010) [2023-12-27 07:58:07,626][20381] Updated weights for policy 1, policy_version 335735 (0.0010) [2023-12-27 07:58:07,866][20347] Updated weights for policy 0, policy_version 335976 (0.0009) [2023-12-27 07:58:07,924][20347] Updated weights for policy 0, policy_version 335986 (0.0010) [2023-12-27 07:58:07,985][20347] Updated weights for policy 0, policy_version 335996 (0.0010) [2023-12-27 07:58:08,258][20381] Updated weights for policy 1, policy_version 335745 (0.0010) [2023-12-27 07:58:08,306][20381] Updated weights for policy 1, policy_version 335755 (0.0011) [2023-12-27 07:58:08,359][20381] Updated weights for policy 1, policy_version 335765 (0.0011) [2023-12-27 07:58:08,638][20347] Updated weights for policy 0, policy_version 336006 (0.0010) [2023-12-27 07:58:08,696][20347] Updated weights for policy 0, policy_version 336016 (0.0010) [2023-12-27 07:58:08,747][20347] Updated weights for policy 0, policy_version 336026 (0.0007) [2023-12-27 07:58:09,084][20381] Updated weights for policy 1, policy_version 335775 (0.0008) [2023-12-27 07:58:09,106][20292] KL-divergence is very high: 346.6097 [2023-12-27 07:58:09,134][20292] KL-divergence is very high: 126.7036 [2023-12-27 07:58:09,141][20381] Updated weights for policy 1, policy_version 335785 (0.0009) [2023-12-27 07:58:09,154][20292] KL-divergence is very high: 149.7288 [2023-12-27 07:58:09,196][20381] Updated weights for policy 1, policy_version 335795 (0.0006) [2023-12-27 07:58:09,434][20347] Updated weights for policy 0, policy_version 336036 (0.0007) [2023-12-27 07:58:09,496][20347] Updated weights for policy 0, policy_version 336046 (0.0010) [2023-12-27 07:58:09,560][20347] Updated weights for policy 0, policy_version 336056 (0.0010) [2023-12-27 07:58:09,864][20381] Updated weights for policy 1, policy_version 335805 (0.0009) [2023-12-27 07:58:09,902][20292] KL-divergence is very high: 119.8111 [2023-12-27 07:58:09,909][20292] KL-divergence is very high: 113.7216 [2023-12-27 07:58:09,927][20381] Updated weights for policy 1, policy_version 335815 (0.0011) [2023-12-27 07:58:09,979][20292] KL-divergence is very high: 114.7062 [2023-12-27 07:58:09,990][20381] Updated weights for policy 1, policy_version 335825 (0.0011) [2023-12-27 07:58:10,253][20347] Updated weights for policy 0, policy_version 336066 (0.0010) [2023-12-27 07:58:10,310][20347] Updated weights for policy 0, policy_version 336076 (0.0005) [2023-12-27 07:58:10,366][20347] Updated weights for policy 0, policy_version 336086 (0.0007) [2023-12-27 07:58:10,421][20347] Updated weights for policy 0, policy_version 336096 (0.0010) [2023-12-27 07:58:10,687][20381] Updated weights for policy 1, policy_version 335835 (0.0009) [2023-12-27 07:58:10,744][20381] Updated weights for policy 1, policy_version 335845 (0.0005) [2023-12-27 07:58:10,799][20381] Updated weights for policy 1, policy_version 335855 (0.0005) [2023-12-27 07:58:11,087][20347] Updated weights for policy 0, policy_version 336106 (0.0008) [2023-12-27 07:58:11,160][20347] Updated weights for policy 0, policy_version 336116 (0.0008) [2023-12-27 07:58:11,213][20347] Updated weights for policy 0, policy_version 336126 (0.0008) [2023-12-27 07:58:11,460][20381] Updated weights for policy 1, policy_version 335865 (0.0009) [2023-12-27 07:58:11,519][20381] Updated weights for policy 1, policy_version 335875 (0.0007) [2023-12-27 07:58:11,583][20381] Updated weights for policy 1, policy_version 335885 (0.0008) [2023-12-27 07:58:11,647][20381] Updated weights for policy 1, policy_version 335895 (0.0008) [2023-12-27 07:58:11,922][20347] Updated weights for policy 0, policy_version 336136 (0.0009) [2023-12-27 07:58:11,975][20347] Updated weights for policy 0, policy_version 336146 (0.0010) [2023-12-27 07:58:12,026][20347] Updated weights for policy 0, policy_version 336156 (0.0008) [2023-12-27 07:58:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 172081152. Throughput: 0: 10323.7, 1: 10446.9. Samples: 172073432. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:12,363][19170] Avg episode reward: [(0, '175.770'), (1, '805.890')] [2023-12-27 07:58:12,373][20381] Updated weights for policy 1, policy_version 335905 (0.0008) [2023-12-27 07:58:12,436][20381] Updated weights for policy 1, policy_version 335915 (0.0005) [2023-12-27 07:58:12,499][20381] Updated weights for policy 1, policy_version 335925 (0.0006) [2023-12-27 07:58:12,682][20347] Updated weights for policy 0, policy_version 336166 (0.0007) [2023-12-27 07:58:12,744][20347] Updated weights for policy 0, policy_version 336176 (0.0010) [2023-12-27 07:58:12,802][20347] Updated weights for policy 0, policy_version 336186 (0.0010) [2023-12-27 07:58:13,125][20381] Updated weights for policy 1, policy_version 335935 (0.0008) [2023-12-27 07:58:13,173][20381] Updated weights for policy 1, policy_version 335945 (0.0008) [2023-12-27 07:58:13,215][20381] Updated weights for policy 1, policy_version 335955 (0.0007) [2023-12-27 07:58:13,397][20347] Updated weights for policy 0, policy_version 336196 (0.0009) [2023-12-27 07:58:13,449][20347] Updated weights for policy 0, policy_version 336206 (0.0009) [2023-12-27 07:58:13,508][20347] Updated weights for policy 0, policy_version 336216 (0.0010) [2023-12-27 07:58:13,862][20381] Updated weights for policy 1, policy_version 335965 (0.0007) [2023-12-27 07:58:13,878][20292] KL-divergence is very high: 114.3140 [2023-12-27 07:58:13,919][20381] Updated weights for policy 1, policy_version 335975 (0.0008) [2023-12-27 07:58:13,974][20381] Updated weights for policy 1, policy_version 335985 (0.0007) [2023-12-27 07:58:14,157][20347] Updated weights for policy 0, policy_version 336226 (0.0010) [2023-12-27 07:58:14,215][20347] Updated weights for policy 0, policy_version 336236 (0.0010) [2023-12-27 07:58:14,275][20347] Updated weights for policy 0, policy_version 336246 (0.0010) [2023-12-27 07:58:14,330][20347] Updated weights for policy 0, policy_version 336256 (0.0010) [2023-12-27 07:58:14,657][20381] Updated weights for policy 1, policy_version 335995 (0.0006) [2023-12-27 07:58:14,706][20381] Updated weights for policy 1, policy_version 336005 (0.0008) [2023-12-27 07:58:14,761][20381] Updated weights for policy 1, policy_version 336015 (0.0007) [2023-12-27 07:58:14,966][20347] Updated weights for policy 0, policy_version 336266 (0.0011) [2023-12-27 07:58:15,020][20347] Updated weights for policy 0, policy_version 336276 (0.0010) [2023-12-27 07:58:15,076][20347] Updated weights for policy 0, policy_version 336286 (0.0010) [2023-12-27 07:58:15,481][20381] Updated weights for policy 1, policy_version 336025 (0.0008) [2023-12-27 07:58:15,532][20381] Updated weights for policy 1, policy_version 336035 (0.0006) [2023-12-27 07:58:15,591][20381] Updated weights for policy 1, policy_version 336045 (0.0007) [2023-12-27 07:58:15,652][20381] Updated weights for policy 1, policy_version 336055 (0.0007) [2023-12-27 07:58:15,768][20347] Updated weights for policy 0, policy_version 336296 (0.0006) [2023-12-27 07:58:15,817][20347] Updated weights for policy 0, policy_version 336306 (0.0006) [2023-12-27 07:58:15,874][20347] Updated weights for policy 0, policy_version 336316 (0.0010) [2023-12-27 07:58:16,317][20381] Updated weights for policy 1, policy_version 336065 (0.0008) [2023-12-27 07:58:16,369][20381] Updated weights for policy 1, policy_version 336075 (0.0007) [2023-12-27 07:58:16,425][20381] Updated weights for policy 1, policy_version 336085 (0.0008) [2023-12-27 07:58:16,504][20347] Updated weights for policy 0, policy_version 336326 (0.0007) [2023-12-27 07:58:16,565][20347] Updated weights for policy 0, policy_version 336336 (0.0005) [2023-12-27 07:58:16,626][20347] Updated weights for policy 0, policy_version 336346 (0.0005) [2023-12-27 07:58:17,122][20381] Updated weights for policy 1, policy_version 336095 (0.0010) [2023-12-27 07:58:17,167][20381] Updated weights for policy 1, policy_version 336105 (0.0010) [2023-12-27 07:58:17,192][20347] Updated weights for policy 0, policy_version 336356 (0.0005) [2023-12-27 07:58:17,217][20381] Updated weights for policy 1, policy_version 336115 (0.0010) [2023-12-27 07:58:17,254][20347] Updated weights for policy 0, policy_version 336366 (0.0006) [2023-12-27 07:58:17,304][20347] Updated weights for policy 0, policy_version 336376 (0.0009) [2023-12-27 07:58:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 172195840. Throughput: 0: 10360.5, 1: 10384.7. Samples: 172198640. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:17,363][19170] Avg episode reward: [(0, '253.430'), (1, '747.430')] [2023-12-27 07:58:17,919][20347] Updated weights for policy 0, policy_version 336386 (0.0009) [2023-12-27 07:58:17,927][20381] Updated weights for policy 1, policy_version 336125 (0.0010) [2023-12-27 07:58:17,976][20347] Updated weights for policy 0, policy_version 336396 (0.0006) [2023-12-27 07:58:17,983][20381] Updated weights for policy 1, policy_version 336135 (0.0011) [2023-12-27 07:58:18,027][20347] Updated weights for policy 0, policy_version 336406 (0.0006) [2023-12-27 07:58:18,037][20381] Updated weights for policy 1, policy_version 336145 (0.0011) [2023-12-27 07:58:18,083][20347] Updated weights for policy 0, policy_version 336416 (0.0005) [2023-12-27 07:58:18,645][20347] Updated weights for policy 0, policy_version 336426 (0.0007) [2023-12-27 07:58:18,707][20347] Updated weights for policy 0, policy_version 336436 (0.0008) [2023-12-27 07:58:18,756][20381] Updated weights for policy 1, policy_version 336155 (0.0011) [2023-12-27 07:58:18,768][20347] Updated weights for policy 0, policy_version 336446 (0.0009) [2023-12-27 07:58:18,813][20381] Updated weights for policy 1, policy_version 336165 (0.0011) [2023-12-27 07:58:18,853][20292] KL-divergence is very high: 119.4747 [2023-12-27 07:58:18,868][20381] Updated weights for policy 1, policy_version 336175 (0.0011) [2023-12-27 07:58:18,868][20292] KL-divergence is very high: 100.7377 [2023-12-27 07:58:19,417][20347] Updated weights for policy 0, policy_version 336456 (0.0009) [2023-12-27 07:58:19,466][20347] Updated weights for policy 0, policy_version 336466 (0.0007) [2023-12-27 07:58:19,530][20347] Updated weights for policy 0, policy_version 336476 (0.0006) [2023-12-27 07:58:19,555][20381] Updated weights for policy 1, policy_version 336185 (0.0011) [2023-12-27 07:58:19,611][20381] Updated weights for policy 1, policy_version 336195 (0.0011) [2023-12-27 07:58:19,669][20381] Updated weights for policy 1, policy_version 336205 (0.0011) [2023-12-27 07:58:19,725][20381] Updated weights for policy 1, policy_version 336215 (0.0011) [2023-12-27 07:58:20,269][20347] Updated weights for policy 0, policy_version 336486 (0.0007) [2023-12-27 07:58:20,328][20347] Updated weights for policy 0, policy_version 336496 (0.0007) [2023-12-27 07:58:20,390][20347] Updated weights for policy 0, policy_version 336506 (0.0006) [2023-12-27 07:58:20,468][20381] Updated weights for policy 1, policy_version 336225 (0.0010) [2023-12-27 07:58:20,532][20381] Updated weights for policy 1, policy_version 336235 (0.0011) [2023-12-27 07:58:20,593][20381] Updated weights for policy 1, policy_version 336245 (0.0010) [2023-12-27 07:58:21,043][20347] Updated weights for policy 0, policy_version 336516 (0.0007) [2023-12-27 07:58:21,106][20347] Updated weights for policy 0, policy_version 336526 (0.0007) [2023-12-27 07:58:21,167][20347] Updated weights for policy 0, policy_version 336536 (0.0008) [2023-12-27 07:58:21,216][20381] Updated weights for policy 1, policy_version 336255 (0.0010) [2023-12-27 07:58:21,251][20292] KL-divergence is very high: 115.8791 [2023-12-27 07:58:21,276][20381] Updated weights for policy 1, policy_version 336265 (0.0011) [2023-12-27 07:58:21,337][20381] Updated weights for policy 1, policy_version 336275 (0.0011) [2023-12-27 07:58:21,919][20347] Updated weights for policy 0, policy_version 336546 (0.0006) [2023-12-27 07:58:21,973][20347] Updated weights for policy 0, policy_version 336556 (0.0008) [2023-12-27 07:58:22,017][20292] KL-divergence is very high: 261.4428 [2023-12-27 07:58:22,029][20381] Updated weights for policy 1, policy_version 336285 (0.0008) [2023-12-27 07:58:22,034][20347] Updated weights for policy 0, policy_version 336566 (0.0009) [2023-12-27 07:58:22,069][20292] KL-divergence is very high: 219.7760 [2023-12-27 07:58:22,076][20292] KL-divergence is very high: 117.2353 [2023-12-27 07:58:22,082][20292] KL-divergence is very high: 134.8104 [2023-12-27 07:58:22,087][20381] Updated weights for policy 1, policy_version 336295 (0.0010) [2023-12-27 07:58:22,089][20292] KL-divergence is very high: 178.5872 [2023-12-27 07:58:22,094][20347] Updated weights for policy 0, policy_version 336576 (0.0008) [2023-12-27 07:58:22,113][20292] KL-divergence is very high: 380.0989 [2023-12-27 07:58:22,125][20292] KL-divergence is very high: 122.0153 [2023-12-27 07:58:22,131][20292] KL-divergence is very high: 246.5445 [2023-12-27 07:58:22,138][20292] KL-divergence is very high: 331.3961 [2023-12-27 07:58:22,144][20292] KL-divergence is very high: 362.0785 [2023-12-27 07:58:22,151][20381] Updated weights for policy 1, policy_version 336305 (0.0008) [2023-12-27 07:58:22,157][20292] KL-divergence is very high: 263.0467 [2023-12-27 07:58:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 172294144. Throughput: 0: 10397.0, 1: 10285.8. Samples: 172260612. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:22,363][19170] Avg episode reward: [(0, '101.660'), (1, '219.680')] [2023-12-27 07:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000336312_86114304.pth... [2023-12-27 07:58:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000336576_86179840.pth... [2023-12-27 07:58:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000335128_85811200.pth [2023-12-27 07:58:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000335360_85868544.pth [2023-12-27 07:58:22,782][20347] Updated weights for policy 0, policy_version 336586 (0.0008) [2023-12-27 07:58:22,836][20292] KL-divergence is very high: 262.9281 [2023-12-27 07:58:22,842][20347] Updated weights for policy 0, policy_version 336596 (0.0008) [2023-12-27 07:58:22,843][20292] KL-divergence is very high: 384.3650 [2023-12-27 07:58:22,849][20292] KL-divergence is very high: 427.4149 [2023-12-27 07:58:22,853][20381] Updated weights for policy 1, policy_version 336315 (0.0007) [2023-12-27 07:58:22,855][20292] KL-divergence is very high: 175.0338 [2023-12-27 07:58:22,861][20292] KL-divergence is very high: 177.1109 [2023-12-27 07:58:22,868][20292] KL-divergence is very high: 216.9717 [2023-12-27 07:58:22,887][20292] KL-divergence is very high: 277.7903 [2023-12-27 07:58:22,894][20292] KL-divergence is very high: 342.4001 [2023-12-27 07:58:22,901][20292] KL-divergence is very high: 386.2087 [2023-12-27 07:58:22,907][20347] Updated weights for policy 0, policy_version 336606 (0.0006) [2023-12-27 07:58:22,908][20292] KL-divergence is very high: 202.6270 [2023-12-27 07:58:22,917][20292] KL-divergence is very high: 167.7516 [2023-12-27 07:58:22,923][20381] Updated weights for policy 1, policy_version 336325 (0.0006) [2023-12-27 07:58:22,923][20292] KL-divergence is very high: 220.5135 [2023-12-27 07:58:22,942][20292] KL-divergence is very high: 356.7880 [2023-12-27 07:58:22,947][20292] KL-divergence is very high: 465.3517 [2023-12-27 07:58:22,953][20292] KL-divergence is very high: 573.0526 [2023-12-27 07:58:22,959][20292] KL-divergence is very high: 219.9364 [2023-12-27 07:58:22,965][20292] KL-divergence is very high: 205.2369 [2023-12-27 07:58:22,971][20292] KL-divergence is very high: 139.8273 [2023-12-27 07:58:22,981][20381] Updated weights for policy 1, policy_version 336335 (0.0005) [2023-12-27 07:58:22,986][20292] KL-divergence is very high: 470.1569 [2023-12-27 07:58:22,991][20292] KL-divergence is very high: 337.3982 [2023-12-27 07:58:22,999][20292] KL-divergence is very high: 503.1764 [2023-12-27 07:58:23,004][20292] KL-divergence is very high: 120.7791 [2023-12-27 07:58:23,010][20292] KL-divergence is very high: 184.0179 [2023-12-27 07:58:23,016][20292] KL-divergence is very high: 194.1633 [2023-12-27 07:58:23,497][20381] Updated weights for policy 1, policy_version 336345 (0.0006) [2023-12-27 07:58:23,558][20381] Updated weights for policy 1, policy_version 336355 (0.0007) [2023-12-27 07:58:23,613][20381] Updated weights for policy 1, policy_version 336365 (0.0005) [2023-12-27 07:58:23,614][20347] Updated weights for policy 0, policy_version 336616 (0.0009) [2023-12-27 07:58:23,671][20381] Updated weights for policy 1, policy_version 336375 (0.0007) [2023-12-27 07:58:23,672][20347] Updated weights for policy 0, policy_version 336626 (0.0009) [2023-12-27 07:58:23,725][20347] Updated weights for policy 0, policy_version 336636 (0.0005) [2023-12-27 07:58:24,233][20381] Updated weights for policy 1, policy_version 336385 (0.0010) [2023-12-27 07:58:24,278][20381] Updated weights for policy 1, policy_version 336395 (0.0010) [2023-12-27 07:58:24,327][20381] Updated weights for policy 1, policy_version 336405 (0.0009) [2023-12-27 07:58:24,360][20347] Updated weights for policy 0, policy_version 336646 (0.0009) [2023-12-27 07:58:24,420][20347] Updated weights for policy 0, policy_version 336656 (0.0010) [2023-12-27 07:58:24,475][20347] Updated weights for policy 0, policy_version 336666 (0.0010) [2023-12-27 07:58:24,971][20381] Updated weights for policy 1, policy_version 336415 (0.0006) [2023-12-27 07:58:25,020][20381] Updated weights for policy 1, policy_version 336425 (0.0005) [2023-12-27 07:58:25,072][20381] Updated weights for policy 1, policy_version 336435 (0.0006) [2023-12-27 07:58:25,142][20347] Updated weights for policy 0, policy_version 336676 (0.0008) [2023-12-27 07:58:25,195][20347] Updated weights for policy 0, policy_version 336686 (0.0007) [2023-12-27 07:58:25,250][20347] Updated weights for policy 0, policy_version 336696 (0.0010) [2023-12-27 07:58:25,683][20381] Updated weights for policy 1, policy_version 336445 (0.0006) [2023-12-27 07:58:25,730][20381] Updated weights for policy 1, policy_version 336455 (0.0007) [2023-12-27 07:58:25,777][20381] Updated weights for policy 1, policy_version 336465 (0.0007) [2023-12-27 07:58:25,914][20347] Updated weights for policy 0, policy_version 336706 (0.0010) [2023-12-27 07:58:25,968][20347] Updated weights for policy 0, policy_version 336716 (0.0010) [2023-12-27 07:58:26,015][20347] Updated weights for policy 0, policy_version 336726 (0.0010) [2023-12-27 07:58:26,065][20347] Updated weights for policy 0, policy_version 336736 (0.0010) [2023-12-27 07:58:26,452][20381] Updated weights for policy 1, policy_version 336475 (0.0007) [2023-12-27 07:58:26,506][20381] Updated weights for policy 1, policy_version 336485 (0.0007) [2023-12-27 07:58:26,565][20381] Updated weights for policy 1, policy_version 336495 (0.0006) [2023-12-27 07:58:26,769][20347] Updated weights for policy 0, policy_version 336746 (0.0010) [2023-12-27 07:58:26,835][20347] Updated weights for policy 0, policy_version 336756 (0.0009) [2023-12-27 07:58:26,888][20347] Updated weights for policy 0, policy_version 336766 (0.0005) [2023-12-27 07:58:27,216][20381] Updated weights for policy 1, policy_version 336505 (0.0007) [2023-12-27 07:58:27,256][20381] Updated weights for policy 1, policy_version 336515 (0.0007) [2023-12-27 07:58:27,306][20381] Updated weights for policy 1, policy_version 336525 (0.0007) [2023-12-27 07:58:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 172392448. Throughput: 0: 10377.8, 1: 10368.7. Samples: 172386576. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:27,363][19170] Avg episode reward: [(0, '212.110'), (1, '38.440')] [2023-12-27 07:58:27,372][20381] Updated weights for policy 1, policy_version 336535 (0.0007) [2023-12-27 07:58:27,540][20347] Updated weights for policy 0, policy_version 336776 (0.0009) [2023-12-27 07:58:27,594][20347] Updated weights for policy 0, policy_version 336786 (0.0010) [2023-12-27 07:58:27,652][20347] Updated weights for policy 0, policy_version 336796 (0.0010) [2023-12-27 07:58:28,100][20381] Updated weights for policy 1, policy_version 336545 (0.0008) [2023-12-27 07:58:28,161][20381] Updated weights for policy 1, policy_version 336555 (0.0008) [2023-12-27 07:58:28,218][20381] Updated weights for policy 1, policy_version 336565 (0.0007) [2023-12-27 07:58:28,335][20347] Updated weights for policy 0, policy_version 336806 (0.0011) [2023-12-27 07:58:28,392][20347] Updated weights for policy 0, policy_version 336816 (0.0010) [2023-12-27 07:58:28,444][20347] Updated weights for policy 0, policy_version 336826 (0.0010) [2023-12-27 07:58:28,844][20381] Updated weights for policy 1, policy_version 336576 (0.0008) [2023-12-27 07:58:28,901][20381] Updated weights for policy 1, policy_version 336586 (0.0007) [2023-12-27 07:58:28,959][20381] Updated weights for policy 1, policy_version 336596 (0.0007) [2023-12-27 07:58:29,106][20347] Updated weights for policy 0, policy_version 336836 (0.0010) [2023-12-27 07:58:29,160][20347] Updated weights for policy 0, policy_version 336846 (0.0010) [2023-12-27 07:58:29,221][20347] Updated weights for policy 0, policy_version 336856 (0.0010) [2023-12-27 07:58:29,680][20381] Updated weights for policy 1, policy_version 336606 (0.0008) [2023-12-27 07:58:29,728][20381] Updated weights for policy 1, policy_version 336616 (0.0008) [2023-12-27 07:58:29,776][20381] Updated weights for policy 1, policy_version 336626 (0.0007) [2023-12-27 07:58:29,906][20347] Updated weights for policy 0, policy_version 336866 (0.0010) [2023-12-27 07:58:29,973][20347] Updated weights for policy 0, policy_version 336876 (0.0010) [2023-12-27 07:58:30,026][20347] Updated weights for policy 0, policy_version 336886 (0.0010) [2023-12-27 07:58:30,081][20347] Updated weights for policy 0, policy_version 336896 (0.0010) [2023-12-27 07:58:30,515][20381] Updated weights for policy 1, policy_version 336636 (0.0007) [2023-12-27 07:58:30,564][20381] Updated weights for policy 1, policy_version 336646 (0.0007) [2023-12-27 07:58:30,611][20381] Updated weights for policy 1, policy_version 336656 (0.0007) [2023-12-27 07:58:30,761][20347] Updated weights for policy 0, policy_version 336906 (0.0005) [2023-12-27 07:58:30,811][20347] Updated weights for policy 0, policy_version 336916 (0.0005) [2023-12-27 07:58:30,860][20347] Updated weights for policy 0, policy_version 336926 (0.0006) [2023-12-27 07:58:31,338][20381] Updated weights for policy 1, policy_version 336666 (0.0008) [2023-12-27 07:58:31,401][20381] Updated weights for policy 1, policy_version 336676 (0.0012) [2023-12-27 07:58:31,458][20381] Updated weights for policy 1, policy_version 336686 (0.0011) [2023-12-27 07:58:31,473][20347] Updated weights for policy 0, policy_version 336936 (0.0009) [2023-12-27 07:58:31,510][20381] Updated weights for policy 1, policy_version 336696 (0.0010) [2023-12-27 07:58:31,524][20347] Updated weights for policy 0, policy_version 336946 (0.0010) [2023-12-27 07:58:31,572][20347] Updated weights for policy 0, policy_version 336956 (0.0010) [2023-12-27 07:58:32,172][20381] Updated weights for policy 1, policy_version 336706 (0.0009) [2023-12-27 07:58:32,216][20381] Updated weights for policy 1, policy_version 336716 (0.0008) [2023-12-27 07:58:32,274][20381] Updated weights for policy 1, policy_version 336726 (0.0009) [2023-12-27 07:58:32,279][20347] Updated weights for policy 0, policy_version 336966 (0.0008) [2023-12-27 07:58:32,336][20347] Updated weights for policy 0, policy_version 336976 (0.0006) [2023-12-27 07:58:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 172498944. Throughput: 0: 10334.9, 1: 10280.2. Samples: 172509288. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:32,363][19170] Avg episode reward: [(0, '290.440'), (1, '39.980')] [2023-12-27 07:58:32,395][20347] Updated weights for policy 0, policy_version 336986 (0.0006) [2023-12-27 07:58:32,849][20381] Updated weights for policy 1, policy_version 336736 (0.0009) [2023-12-27 07:58:32,913][20381] Updated weights for policy 1, policy_version 336746 (0.0010) [2023-12-27 07:58:32,975][20381] Updated weights for policy 1, policy_version 336756 (0.0010) [2023-12-27 07:58:33,022][20347] Updated weights for policy 0, policy_version 336996 (0.0007) [2023-12-27 07:58:33,081][20347] Updated weights for policy 0, policy_version 337006 (0.0008) [2023-12-27 07:58:33,149][20347] Updated weights for policy 0, policy_version 337016 (0.0006) [2023-12-27 07:58:33,601][20381] Updated weights for policy 1, policy_version 336766 (0.0010) [2023-12-27 07:58:33,649][20381] Updated weights for policy 1, policy_version 336776 (0.0010) [2023-12-27 07:58:33,693][20381] Updated weights for policy 1, policy_version 336786 (0.0010) [2023-12-27 07:58:33,722][20347] Updated weights for policy 0, policy_version 337026 (0.0008) [2023-12-27 07:58:33,775][20347] Updated weights for policy 0, policy_version 337036 (0.0007) [2023-12-27 07:58:33,821][20347] Updated weights for policy 0, policy_version 337046 (0.0005) [2023-12-27 07:58:33,873][20347] Updated weights for policy 0, policy_version 337056 (0.0010) [2023-12-27 07:58:34,365][20381] Updated weights for policy 1, policy_version 336796 (0.0010) [2023-12-27 07:58:34,422][20381] Updated weights for policy 1, policy_version 336806 (0.0008) [2023-12-27 07:58:34,485][20381] Updated weights for policy 1, policy_version 336816 (0.0011) [2023-12-27 07:58:34,521][20347] Updated weights for policy 0, policy_version 337066 (0.0011) [2023-12-27 07:58:34,583][20347] Updated weights for policy 0, policy_version 337076 (0.0011) [2023-12-27 07:58:34,642][20347] Updated weights for policy 0, policy_version 337086 (0.0010) [2023-12-27 07:58:35,132][20381] Updated weights for policy 1, policy_version 336826 (0.0010) [2023-12-27 07:58:35,185][20381] Updated weights for policy 1, policy_version 336836 (0.0005) [2023-12-27 07:58:35,243][20381] Updated weights for policy 1, policy_version 336846 (0.0006) [2023-12-27 07:58:35,299][20381] Updated weights for policy 1, policy_version 336856 (0.0005) [2023-12-27 07:58:35,322][20347] Updated weights for policy 0, policy_version 337096 (0.0010) [2023-12-27 07:58:35,373][20347] Updated weights for policy 0, policy_version 337106 (0.0010) [2023-12-27 07:58:35,427][20347] Updated weights for policy 0, policy_version 337116 (0.0010) [2023-12-27 07:58:35,919][20381] Updated weights for policy 1, policy_version 336866 (0.0011) [2023-12-27 07:58:35,978][20381] Updated weights for policy 1, policy_version 336876 (0.0011) [2023-12-27 07:58:36,037][20381] Updated weights for policy 1, policy_version 336886 (0.0011) [2023-12-27 07:58:36,083][20347] Updated weights for policy 0, policy_version 337126 (0.0008) [2023-12-27 07:58:36,147][20347] Updated weights for policy 0, policy_version 337136 (0.0007) [2023-12-27 07:58:36,207][20347] Updated weights for policy 0, policy_version 337146 (0.0007) [2023-12-27 07:58:36,740][20381] Updated weights for policy 1, policy_version 336896 (0.0011) [2023-12-27 07:58:36,804][20381] Updated weights for policy 1, policy_version 336906 (0.0011) [2023-12-27 07:58:36,863][20381] Updated weights for policy 1, policy_version 336916 (0.0011) [2023-12-27 07:58:36,864][20347] Updated weights for policy 0, policy_version 337156 (0.0007) [2023-12-27 07:58:36,927][20347] Updated weights for policy 0, policy_version 337166 (0.0007) [2023-12-27 07:58:36,980][20347] Updated weights for policy 0, policy_version 337176 (0.0007) [2023-12-27 07:58:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 172605440. Throughput: 0: 10396.3, 1: 10294.1. Samples: 172574264. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:37,363][19170] Avg episode reward: [(0, '218.160'), (1, '81.120')] [2023-12-27 07:58:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000336920_86269952.pth... [2023-12-27 07:58:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000337184_86335488.pth... [2023-12-27 07:58:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000335704_85958656.pth [2023-12-27 07:58:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000335968_86024192.pth [2023-12-27 07:58:37,525][20381] Updated weights for policy 1, policy_version 336926 (0.0009) [2023-12-27 07:58:37,582][20381] Updated weights for policy 1, policy_version 336936 (0.0011) [2023-12-27 07:58:37,639][20381] Updated weights for policy 1, policy_version 336946 (0.0011) [2023-12-27 07:58:37,660][20347] Updated weights for policy 0, policy_version 337186 (0.0007) [2023-12-27 07:58:37,723][20347] Updated weights for policy 0, policy_version 337196 (0.0007) [2023-12-27 07:58:37,785][20347] Updated weights for policy 0, policy_version 337206 (0.0007) [2023-12-27 07:58:37,846][20347] Updated weights for policy 0, policy_version 337216 (0.0006) [2023-12-27 07:58:38,313][20381] Updated weights for policy 1, policy_version 336956 (0.0010) [2023-12-27 07:58:38,378][20381] Updated weights for policy 1, policy_version 336966 (0.0011) [2023-12-27 07:58:38,427][20381] Updated weights for policy 1, policy_version 336976 (0.0010) [2023-12-27 07:58:38,539][20347] Updated weights for policy 0, policy_version 337226 (0.0007) [2023-12-27 07:58:38,599][20347] Updated weights for policy 0, policy_version 337236 (0.0008) [2023-12-27 07:58:38,650][20347] Updated weights for policy 0, policy_version 337246 (0.0007) [2023-12-27 07:58:39,097][20381] Updated weights for policy 1, policy_version 336986 (0.0010) [2023-12-27 07:58:39,146][20381] Updated weights for policy 1, policy_version 336996 (0.0005) [2023-12-27 07:58:39,201][20381] Updated weights for policy 1, policy_version 337006 (0.0006) [2023-12-27 07:58:39,270][20381] Updated weights for policy 1, policy_version 337016 (0.0010) [2023-12-27 07:58:39,324][20347] Updated weights for policy 0, policy_version 337256 (0.0008) [2023-12-27 07:58:39,388][20347] Updated weights for policy 0, policy_version 337266 (0.0007) [2023-12-27 07:58:39,448][20347] Updated weights for policy 0, policy_version 337276 (0.0007) [2023-12-27 07:58:39,974][20381] Updated weights for policy 1, policy_version 337026 (0.0008) [2023-12-27 07:58:40,031][20381] Updated weights for policy 1, policy_version 337036 (0.0008) [2023-12-27 07:58:40,065][20347] Updated weights for policy 0, policy_version 337286 (0.0006) [2023-12-27 07:58:40,089][20381] Updated weights for policy 1, policy_version 337046 (0.0006) [2023-12-27 07:58:40,126][20347] Updated weights for policy 0, policy_version 337296 (0.0006) [2023-12-27 07:58:40,184][20347] Updated weights for policy 0, policy_version 337306 (0.0008) [2023-12-27 07:58:40,761][20381] Updated weights for policy 1, policy_version 337056 (0.0009) [2023-12-27 07:58:40,782][20347] Updated weights for policy 0, policy_version 337316 (0.0006) [2023-12-27 07:58:40,816][20381] Updated weights for policy 1, policy_version 337066 (0.0010) [2023-12-27 07:58:40,833][20347] Updated weights for policy 0, policy_version 337326 (0.0006) [2023-12-27 07:58:40,870][20381] Updated weights for policy 1, policy_version 337076 (0.0010) [2023-12-27 07:58:40,889][20347] Updated weights for policy 0, policy_version 337336 (0.0006) [2023-12-27 07:58:41,578][20381] Updated weights for policy 1, policy_version 337086 (0.0009) [2023-12-27 07:58:41,580][20347] Updated weights for policy 0, policy_version 337346 (0.0005) [2023-12-27 07:58:41,640][20381] Updated weights for policy 1, policy_version 337096 (0.0010) [2023-12-27 07:58:41,646][20347] Updated weights for policy 0, policy_version 337356 (0.0007) [2023-12-27 07:58:41,704][20347] Updated weights for policy 0, policy_version 337366 (0.0008) [2023-12-27 07:58:41,705][20381] Updated weights for policy 1, policy_version 337106 (0.0010) [2023-12-27 07:58:41,769][20347] Updated weights for policy 0, policy_version 337376 (0.0007) [2023-12-27 07:58:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 172703744. Throughput: 0: 10334.1, 1: 10353.3. Samples: 172697900. Policy #0 lag: (min: 26.0, avg: 42.6, max: 58.0) [2023-12-27 07:58:42,363][19170] Avg episode reward: [(0, '293.180'), (1, '282.690')] [2023-12-27 07:58:42,404][20347] Updated weights for policy 0, policy_version 337386 (0.0008) [2023-12-27 07:58:42,420][20381] Updated weights for policy 1, policy_version 337116 (0.0008) [2023-12-27 07:58:42,460][20347] Updated weights for policy 0, policy_version 337396 (0.0008) [2023-12-27 07:58:42,481][20381] Updated weights for policy 1, policy_version 337126 (0.0010) [2023-12-27 07:58:42,517][20347] Updated weights for policy 0, policy_version 337406 (0.0006) [2023-12-27 07:58:42,537][20381] Updated weights for policy 1, policy_version 337136 (0.0010) [2023-12-27 07:58:43,174][20347] Updated weights for policy 0, policy_version 337416 (0.0009) [2023-12-27 07:58:43,219][20381] Updated weights for policy 1, policy_version 337146 (0.0008) [2023-12-27 07:58:43,226][20347] Updated weights for policy 0, policy_version 337426 (0.0010) [2023-12-27 07:58:43,274][20381] Updated weights for policy 1, policy_version 337156 (0.0010) [2023-12-27 07:58:43,278][20347] Updated weights for policy 0, policy_version 337436 (0.0010) [2023-12-27 07:58:43,329][20381] Updated weights for policy 1, policy_version 337166 (0.0010) [2023-12-27 07:58:43,377][20381] Updated weights for policy 1, policy_version 337176 (0.0010) [2023-12-27 07:58:43,961][20347] Updated weights for policy 0, policy_version 337446 (0.0009) [2023-12-27 07:58:44,015][20347] Updated weights for policy 0, policy_version 337456 (0.0009) [2023-12-27 07:58:44,056][20381] Updated weights for policy 1, policy_version 337186 (0.0010) [2023-12-27 07:58:44,069][20347] Updated weights for policy 0, policy_version 337466 (0.0007) [2023-12-27 07:58:44,115][20381] Updated weights for policy 1, policy_version 337196 (0.0008) [2023-12-27 07:58:44,180][20381] Updated weights for policy 1, policy_version 337206 (0.0005) [2023-12-27 07:58:44,772][20347] Updated weights for policy 0, policy_version 337476 (0.0008) [2023-12-27 07:58:44,829][20347] Updated weights for policy 0, policy_version 337486 (0.0006) [2023-12-27 07:58:44,865][20381] Updated weights for policy 1, policy_version 337216 (0.0007) [2023-12-27 07:58:44,891][20347] Updated weights for policy 0, policy_version 337496 (0.0006) [2023-12-27 07:58:44,917][20381] Updated weights for policy 1, policy_version 337226 (0.0006) [2023-12-27 07:58:44,970][20381] Updated weights for policy 1, policy_version 337236 (0.0008) [2023-12-27 07:58:45,536][20347] Updated weights for policy 0, policy_version 337506 (0.0006) [2023-12-27 07:58:45,598][20347] Updated weights for policy 0, policy_version 337516 (0.0005) [2023-12-27 07:58:45,652][20381] Updated weights for policy 1, policy_version 337246 (0.0010) [2023-12-27 07:58:45,653][20347] Updated weights for policy 0, policy_version 337526 (0.0006) [2023-12-27 07:58:45,708][20347] Updated weights for policy 0, policy_version 337536 (0.0005) [2023-12-27 07:58:45,710][20381] Updated weights for policy 1, policy_version 337256 (0.0010) [2023-12-27 07:58:45,765][20381] Updated weights for policy 1, policy_version 337266 (0.0010) [2023-12-27 07:58:46,328][20347] Updated weights for policy 0, policy_version 337546 (0.0005) [2023-12-27 07:58:46,389][20347] Updated weights for policy 0, policy_version 337556 (0.0007) [2023-12-27 07:58:46,402][20381] Updated weights for policy 1, policy_version 337276 (0.0008) [2023-12-27 07:58:46,445][20347] Updated weights for policy 0, policy_version 337566 (0.0007) [2023-12-27 07:58:46,459][20381] Updated weights for policy 1, policy_version 337286 (0.0006) [2023-12-27 07:58:46,525][20381] Updated weights for policy 1, policy_version 337296 (0.0005) [2023-12-27 07:58:47,088][20347] Updated weights for policy 0, policy_version 337576 (0.0007) [2023-12-27 07:58:47,129][20381] Updated weights for policy 1, policy_version 337306 (0.0007) [2023-12-27 07:58:47,140][20347] Updated weights for policy 0, policy_version 337586 (0.0010) [2023-12-27 07:58:47,178][20381] Updated weights for policy 1, policy_version 337316 (0.0010) [2023-12-27 07:58:47,203][20347] Updated weights for policy 0, policy_version 337596 (0.0010) [2023-12-27 07:58:47,225][20381] Updated weights for policy 1, policy_version 337326 (0.0007) [2023-12-27 07:58:47,276][20381] Updated weights for policy 1, policy_version 337336 (0.0005) [2023-12-27 07:58:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 172818432. Throughput: 0: 10450.5, 1: 10345.4. Samples: 172822168. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:58:47,363][19170] Avg episode reward: [(0, '346.120'), (1, '187.300')] [2023-12-27 07:58:47,776][20347] Updated weights for policy 0, policy_version 337606 (0.0007) [2023-12-27 07:58:47,830][20347] Updated weights for policy 0, policy_version 337616 (0.0005) [2023-12-27 07:58:47,886][20347] Updated weights for policy 0, policy_version 337626 (0.0005) [2023-12-27 07:58:47,952][20381] Updated weights for policy 1, policy_version 337346 (0.0008) [2023-12-27 07:58:48,004][20381] Updated weights for policy 1, policy_version 337356 (0.0010) [2023-12-27 07:58:48,053][20381] Updated weights for policy 1, policy_version 337366 (0.0010) [2023-12-27 07:58:48,456][20347] Updated weights for policy 0, policy_version 337636 (0.0008) [2023-12-27 07:58:48,515][20347] Updated weights for policy 0, policy_version 337646 (0.0010) [2023-12-27 07:58:48,567][20347] Updated weights for policy 0, policy_version 337656 (0.0010) [2023-12-27 07:58:48,752][20381] Updated weights for policy 1, policy_version 337376 (0.0010) [2023-12-27 07:58:48,804][20381] Updated weights for policy 1, policy_version 337386 (0.0010) [2023-12-27 07:58:48,860][20381] Updated weights for policy 1, policy_version 337396 (0.0010) [2023-12-27 07:58:49,255][20347] Updated weights for policy 0, policy_version 337666 (0.0010) [2023-12-27 07:58:49,321][20347] Updated weights for policy 0, policy_version 337676 (0.0010) [2023-12-27 07:58:49,397][20347] Updated weights for policy 0, policy_version 337686 (0.0010) [2023-12-27 07:58:49,458][20347] Updated weights for policy 0, policy_version 337696 (0.0009) [2023-12-27 07:58:49,585][20381] Updated weights for policy 1, policy_version 337406 (0.0011) [2023-12-27 07:58:49,642][20381] Updated weights for policy 1, policy_version 337416 (0.0011) [2023-12-27 07:58:49,697][20381] Updated weights for policy 1, policy_version 337426 (0.0011) [2023-12-27 07:58:50,143][20347] Updated weights for policy 0, policy_version 337706 (0.0007) [2023-12-27 07:58:50,192][20347] Updated weights for policy 0, policy_version 337716 (0.0007) [2023-12-27 07:58:50,241][20347] Updated weights for policy 0, policy_version 337726 (0.0008) [2023-12-27 07:58:50,390][20381] Updated weights for policy 1, policy_version 337436 (0.0008) [2023-12-27 07:58:50,448][20381] Updated weights for policy 1, policy_version 337446 (0.0011) [2023-12-27 07:58:50,505][20381] Updated weights for policy 1, policy_version 337456 (0.0007) [2023-12-27 07:58:50,975][20347] Updated weights for policy 0, policy_version 337736 (0.0007) [2023-12-27 07:58:51,038][20347] Updated weights for policy 0, policy_version 337746 (0.0008) [2023-12-27 07:58:51,093][20347] Updated weights for policy 0, policy_version 337756 (0.0007) [2023-12-27 07:58:51,205][20381] Updated weights for policy 1, policy_version 337466 (0.0009) [2023-12-27 07:58:51,259][20381] Updated weights for policy 1, policy_version 337476 (0.0008) [2023-12-27 07:58:51,309][20381] Updated weights for policy 1, policy_version 337486 (0.0008) [2023-12-27 07:58:51,367][20381] Updated weights for policy 1, policy_version 337496 (0.0008) [2023-12-27 07:58:51,846][20347] Updated weights for policy 0, policy_version 337766 (0.0008) [2023-12-27 07:58:51,902][20347] Updated weights for policy 0, policy_version 337776 (0.0008) [2023-12-27 07:58:51,957][20347] Updated weights for policy 0, policy_version 337786 (0.0008) [2023-12-27 07:58:52,075][20381] Updated weights for policy 1, policy_version 337506 (0.0010) [2023-12-27 07:58:52,132][20381] Updated weights for policy 1, policy_version 337516 (0.0010) [2023-12-27 07:58:52,188][20381] Updated weights for policy 1, policy_version 337526 (0.0010) [2023-12-27 07:58:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 172916736. Throughput: 0: 10474.2, 1: 10303.1. Samples: 172884932. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:58:52,363][19170] Avg episode reward: [(0, '391.820'), (1, '281.820')] [2023-12-27 07:58:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000337528_86425600.pth... [2023-12-27 07:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000337792_86491136.pth... [2023-12-27 07:58:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000336312_86114304.pth [2023-12-27 07:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000336576_86179840.pth [2023-12-27 07:58:52,743][20347] Updated weights for policy 0, policy_version 337796 (0.0008) [2023-12-27 07:58:52,797][20347] Updated weights for policy 0, policy_version 337806 (0.0008) [2023-12-27 07:58:52,814][20381] Updated weights for policy 1, policy_version 337536 (0.0007) [2023-12-27 07:58:52,849][20347] Updated weights for policy 0, policy_version 337816 (0.0008) [2023-12-27 07:58:52,865][20381] Updated weights for policy 1, policy_version 337546 (0.0006) [2023-12-27 07:58:52,920][20381] Updated weights for policy 1, policy_version 337556 (0.0007) [2023-12-27 07:58:53,563][20347] Updated weights for policy 0, policy_version 337826 (0.0008) [2023-12-27 07:58:53,599][20381] Updated weights for policy 1, policy_version 337566 (0.0007) [2023-12-27 07:58:53,623][20347] Updated weights for policy 0, policy_version 337836 (0.0007) [2023-12-27 07:58:53,655][20381] Updated weights for policy 1, policy_version 337576 (0.0006) [2023-12-27 07:58:53,676][20347] Updated weights for policy 0, policy_version 337846 (0.0006) [2023-12-27 07:58:53,714][20381] Updated weights for policy 1, policy_version 337586 (0.0006) [2023-12-27 07:58:53,721][20347] Updated weights for policy 0, policy_version 337856 (0.0005) [2023-12-27 07:58:54,394][20381] Updated weights for policy 1, policy_version 337596 (0.0007) [2023-12-27 07:58:54,443][20347] Updated weights for policy 0, policy_version 337866 (0.0006) [2023-12-27 07:58:54,450][20381] Updated weights for policy 1, policy_version 337606 (0.0006) [2023-12-27 07:58:54,503][20347] Updated weights for policy 0, policy_version 337876 (0.0008) [2023-12-27 07:58:54,512][20381] Updated weights for policy 1, policy_version 337616 (0.0006) [2023-12-27 07:58:54,559][20347] Updated weights for policy 0, policy_version 337886 (0.0007) [2023-12-27 07:58:55,146][20347] Updated weights for policy 0, policy_version 337896 (0.0008) [2023-12-27 07:58:55,192][20347] Updated weights for policy 0, policy_version 337906 (0.0007) [2023-12-27 07:58:55,199][20381] Updated weights for policy 1, policy_version 337626 (0.0006) [2023-12-27 07:58:55,240][20347] Updated weights for policy 0, policy_version 337916 (0.0005) [2023-12-27 07:58:55,256][20381] Updated weights for policy 1, policy_version 337636 (0.0010) [2023-12-27 07:58:55,307][20381] Updated weights for policy 1, policy_version 337646 (0.0010) [2023-12-27 07:58:55,361][20381] Updated weights for policy 1, policy_version 337656 (0.0007) [2023-12-27 07:58:55,879][20347] Updated weights for policy 0, policy_version 337926 (0.0006) [2023-12-27 07:58:55,937][20347] Updated weights for policy 0, policy_version 337936 (0.0007) [2023-12-27 07:58:55,990][20381] Updated weights for policy 1, policy_version 337666 (0.0006) [2023-12-27 07:58:55,995][20347] Updated weights for policy 0, policy_version 337946 (0.0007) [2023-12-27 07:58:56,041][20381] Updated weights for policy 1, policy_version 337676 (0.0006) [2023-12-27 07:58:56,088][20381] Updated weights for policy 1, policy_version 337686 (0.0006) [2023-12-27 07:58:56,608][20347] Updated weights for policy 0, policy_version 337956 (0.0007) [2023-12-27 07:58:56,658][20347] Updated weights for policy 0, policy_version 337966 (0.0005) [2023-12-27 07:58:56,665][20381] Updated weights for policy 1, policy_version 337696 (0.0006) [2023-12-27 07:58:56,709][20347] Updated weights for policy 0, policy_version 337976 (0.0005) [2023-12-27 07:58:56,719][20381] Updated weights for policy 1, policy_version 337706 (0.0007) [2023-12-27 07:58:56,775][20381] Updated weights for policy 1, policy_version 337716 (0.0005) [2023-12-27 07:58:57,280][20347] Updated weights for policy 0, policy_version 337986 (0.0005) [2023-12-27 07:58:57,336][20381] Updated weights for policy 1, policy_version 337726 (0.0005) [2023-12-27 07:58:57,337][20347] Updated weights for policy 0, policy_version 337996 (0.0006) [2023-12-27 07:58:57,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 173015040. Throughput: 0: 10429.1, 1: 10322.2. Samples: 173007244. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:58:57,364][19170] Avg episode reward: [(0, '432.710'), (1, '437.310')] [2023-12-27 07:58:57,393][20347] Updated weights for policy 0, policy_version 338006 (0.0005) [2023-12-27 07:58:57,394][20381] Updated weights for policy 1, policy_version 337736 (0.0007) [2023-12-27 07:58:57,440][20381] Updated weights for policy 1, policy_version 337746 (0.0006) [2023-12-27 07:58:57,445][20347] Updated weights for policy 0, policy_version 338016 (0.0006) [2023-12-27 07:58:58,072][20347] Updated weights for policy 0, policy_version 338026 (0.0008) [2023-12-27 07:58:58,088][20381] Updated weights for policy 1, policy_version 337756 (0.0006) [2023-12-27 07:58:58,125][20347] Updated weights for policy 0, policy_version 338036 (0.0006) [2023-12-27 07:58:58,146][20381] Updated weights for policy 1, policy_version 337766 (0.0010) [2023-12-27 07:58:58,187][20347] Updated weights for policy 0, policy_version 338046 (0.0006) [2023-12-27 07:58:58,208][20381] Updated weights for policy 1, policy_version 337776 (0.0007) [2023-12-27 07:58:58,884][20347] Updated weights for policy 0, policy_version 338056 (0.0007) [2023-12-27 07:58:58,956][20347] Updated weights for policy 0, policy_version 338066 (0.0009) [2023-12-27 07:58:58,961][20381] Updated weights for policy 1, policy_version 337786 (0.0008) [2023-12-27 07:58:59,017][20347] Updated weights for policy 0, policy_version 338076 (0.0006) [2023-12-27 07:58:59,019][20381] Updated weights for policy 1, policy_version 337796 (0.0009) [2023-12-27 07:58:59,080][20381] Updated weights for policy 1, policy_version 337806 (0.0009) [2023-12-27 07:58:59,141][20381] Updated weights for policy 1, policy_version 337816 (0.0007) [2023-12-27 07:58:59,720][20347] Updated weights for policy 0, policy_version 338086 (0.0009) [2023-12-27 07:58:59,779][20347] Updated weights for policy 0, policy_version 338096 (0.0010) [2023-12-27 07:58:59,837][20347] Updated weights for policy 0, policy_version 338106 (0.0009) [2023-12-27 07:58:59,838][20381] Updated weights for policy 1, policy_version 337826 (0.0009) [2023-12-27 07:58:59,900][20381] Updated weights for policy 1, policy_version 337836 (0.0010) [2023-12-27 07:58:59,957][20381] Updated weights for policy 1, policy_version 337846 (0.0009) [2023-12-27 07:59:00,466][20347] Updated weights for policy 0, policy_version 338116 (0.0007) [2023-12-27 07:59:00,524][20347] Updated weights for policy 0, policy_version 338126 (0.0009) [2023-12-27 07:59:00,582][20347] Updated weights for policy 0, policy_version 338136 (0.0010) [2023-12-27 07:59:00,630][20381] Updated weights for policy 1, policy_version 337856 (0.0010) [2023-12-27 07:59:00,676][20381] Updated weights for policy 1, policy_version 337866 (0.0005) [2023-12-27 07:59:00,740][20381] Updated weights for policy 1, policy_version 337876 (0.0005) [2023-12-27 07:59:01,214][20347] Updated weights for policy 0, policy_version 338146 (0.0007) [2023-12-27 07:59:01,283][20347] Updated weights for policy 0, policy_version 338156 (0.0006) [2023-12-27 07:59:01,341][20347] Updated weights for policy 0, policy_version 338166 (0.0006) [2023-12-27 07:59:01,349][20381] Updated weights for policy 1, policy_version 337886 (0.0009) [2023-12-27 07:59:01,401][20347] Updated weights for policy 0, policy_version 338176 (0.0008) [2023-12-27 07:59:01,415][20381] Updated weights for policy 1, policy_version 337896 (0.0007) [2023-12-27 07:59:01,477][20381] Updated weights for policy 1, policy_version 337906 (0.0005) [2023-12-27 07:59:02,016][20347] Updated weights for policy 0, policy_version 338186 (0.0008) [2023-12-27 07:59:02,075][20347] Updated weights for policy 0, policy_version 338196 (0.0006) [2023-12-27 07:59:02,134][20347] Updated weights for policy 0, policy_version 338206 (0.0006) [2023-12-27 07:59:02,158][20381] Updated weights for policy 1, policy_version 337916 (0.0005) [2023-12-27 07:59:02,225][20381] Updated weights for policy 1, policy_version 337926 (0.0006) [2023-12-27 07:59:02,289][20381] Updated weights for policy 1, policy_version 337936 (0.0008) [2023-12-27 07:59:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 173129728. Throughput: 0: 10442.5, 1: 10370.9. Samples: 173135244. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:02,363][19170] Avg episode reward: [(0, '416.100'), (1, '465.000')] [2023-12-27 07:59:02,802][20347] Updated weights for policy 0, policy_version 338216 (0.0008) [2023-12-27 07:59:02,865][20347] Updated weights for policy 0, policy_version 338226 (0.0008) [2023-12-27 07:59:02,890][20381] Updated weights for policy 1, policy_version 337946 (0.0006) [2023-12-27 07:59:02,917][20347] Updated weights for policy 0, policy_version 338236 (0.0008) [2023-12-27 07:59:02,944][20381] Updated weights for policy 1, policy_version 337956 (0.0006) [2023-12-27 07:59:03,002][20381] Updated weights for policy 1, policy_version 337966 (0.0006) [2023-12-27 07:59:03,067][20381] Updated weights for policy 1, policy_version 337976 (0.0006) [2023-12-27 07:59:03,574][20381] Updated weights for policy 1, policy_version 337986 (0.0005) [2023-12-27 07:59:03,603][20347] Updated weights for policy 0, policy_version 338246 (0.0006) [2023-12-27 07:59:03,629][20381] Updated weights for policy 1, policy_version 337996 (0.0005) [2023-12-27 07:59:03,657][20347] Updated weights for policy 0, policy_version 338256 (0.0007) [2023-12-27 07:59:03,684][20381] Updated weights for policy 1, policy_version 338006 (0.0006) [2023-12-27 07:59:03,709][20347] Updated weights for policy 0, policy_version 338266 (0.0007) [2023-12-27 07:59:04,292][20347] Updated weights for policy 0, policy_version 338276 (0.0007) [2023-12-27 07:59:04,348][20347] Updated weights for policy 0, policy_version 338286 (0.0006) [2023-12-27 07:59:04,369][20381] Updated weights for policy 1, policy_version 338016 (0.0006) [2023-12-27 07:59:04,400][20347] Updated weights for policy 0, policy_version 338296 (0.0006) [2023-12-27 07:59:04,429][20381] Updated weights for policy 1, policy_version 338026 (0.0007) [2023-12-27 07:59:04,480][20381] Updated weights for policy 1, policy_version 338036 (0.0007) [2023-12-27 07:59:05,030][20381] Updated weights for policy 1, policy_version 338046 (0.0006) [2023-12-27 07:59:05,062][20347] Updated weights for policy 0, policy_version 338306 (0.0007) [2023-12-27 07:59:05,083][20381] Updated weights for policy 1, policy_version 338056 (0.0005) [2023-12-27 07:59:05,110][20347] Updated weights for policy 0, policy_version 338316 (0.0007) [2023-12-27 07:59:05,132][20381] Updated weights for policy 1, policy_version 338066 (0.0005) [2023-12-27 07:59:05,154][20347] Updated weights for policy 0, policy_version 338326 (0.0006) [2023-12-27 07:59:05,202][20347] Updated weights for policy 0, policy_version 338336 (0.0008) [2023-12-27 07:59:05,690][20381] Updated weights for policy 1, policy_version 338076 (0.0007) [2023-12-27 07:59:05,738][20381] Updated weights for policy 1, policy_version 338086 (0.0007) [2023-12-27 07:59:05,789][20381] Updated weights for policy 1, policy_version 338096 (0.0007) [2023-12-27 07:59:05,831][20347] Updated weights for policy 0, policy_version 338346 (0.0005) [2023-12-27 07:59:05,875][20347] Updated weights for policy 0, policy_version 338356 (0.0007) [2023-12-27 07:59:05,923][20347] Updated weights for policy 0, policy_version 338366 (0.0009) [2023-12-27 07:59:06,460][20381] Updated weights for policy 1, policy_version 338106 (0.0006) [2023-12-27 07:59:06,526][20381] Updated weights for policy 1, policy_version 338116 (0.0007) [2023-12-27 07:59:06,584][20381] Updated weights for policy 1, policy_version 338126 (0.0008) [2023-12-27 07:59:06,643][20381] Updated weights for policy 1, policy_version 338136 (0.0009) [2023-12-27 07:59:06,666][20347] Updated weights for policy 0, policy_version 338376 (0.0007) [2023-12-27 07:59:06,730][20347] Updated weights for policy 0, policy_version 338386 (0.0006) [2023-12-27 07:59:06,788][20347] Updated weights for policy 0, policy_version 338396 (0.0005) [2023-12-27 07:59:07,327][20381] Updated weights for policy 1, policy_version 338146 (0.0005) [2023-12-27 07:59:07,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 173228032. Throughput: 0: 10425.7, 1: 10476.9. Samples: 173201228. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:07,363][19170] Avg episode reward: [(0, '341.220'), (1, '522.490')] [2023-12-27 07:59:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000338400_86646784.pth... [2023-12-27 07:59:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000337184_86335488.pth [2023-12-27 07:59:07,384][20381] Updated weights for policy 1, policy_version 338156 (0.0005) [2023-12-27 07:59:07,414][20347] Updated weights for policy 0, policy_version 338406 (0.0007) [2023-12-27 07:59:07,441][20381] Updated weights for policy 1, policy_version 338166 (0.0005) [2023-12-27 07:59:07,452][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000338168_86589440.pth... [2023-12-27 07:59:07,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000336920_86269952.pth [2023-12-27 07:59:07,468][20347] Updated weights for policy 0, policy_version 338416 (0.0007) [2023-12-27 07:59:07,520][20347] Updated weights for policy 0, policy_version 338426 (0.0007) [2023-12-27 07:59:08,056][20381] Updated weights for policy 1, policy_version 338176 (0.0006) [2023-12-27 07:59:08,121][20381] Updated weights for policy 1, policy_version 338186 (0.0005) [2023-12-27 07:59:08,182][20381] Updated weights for policy 1, policy_version 338196 (0.0005) [2023-12-27 07:59:08,263][20347] Updated weights for policy 0, policy_version 338436 (0.0008) [2023-12-27 07:59:08,331][20347] Updated weights for policy 0, policy_version 338446 (0.0008) [2023-12-27 07:59:08,395][20347] Updated weights for policy 0, policy_version 338456 (0.0007) [2023-12-27 07:59:08,804][20381] Updated weights for policy 1, policy_version 338206 (0.0007) [2023-12-27 07:59:08,862][20381] Updated weights for policy 1, policy_version 338216 (0.0007) [2023-12-27 07:59:08,922][20381] Updated weights for policy 1, policy_version 338226 (0.0007) [2023-12-27 07:59:09,112][20347] Updated weights for policy 0, policy_version 338466 (0.0008) [2023-12-27 07:59:09,167][20347] Updated weights for policy 0, policy_version 338476 (0.0007) [2023-12-27 07:59:09,223][20347] Updated weights for policy 0, policy_version 338486 (0.0010) [2023-12-27 07:59:09,287][20347] Updated weights for policy 0, policy_version 338496 (0.0008) [2023-12-27 07:59:09,595][20381] Updated weights for policy 1, policy_version 338236 (0.0007) [2023-12-27 07:59:09,656][20381] Updated weights for policy 1, policy_version 338246 (0.0006) [2023-12-27 07:59:09,719][20381] Updated weights for policy 1, policy_version 338256 (0.0008) [2023-12-27 07:59:09,964][20347] Updated weights for policy 0, policy_version 338506 (0.0010) [2023-12-27 07:59:10,016][20347] Updated weights for policy 0, policy_version 338516 (0.0010) [2023-12-27 07:59:10,076][20347] Updated weights for policy 0, policy_version 338526 (0.0010) [2023-12-27 07:59:10,386][20381] Updated weights for policy 1, policy_version 338266 (0.0008) [2023-12-27 07:59:10,445][20381] Updated weights for policy 1, policy_version 338276 (0.0006) [2023-12-27 07:59:10,495][20381] Updated weights for policy 1, policy_version 338286 (0.0011) [2023-12-27 07:59:10,542][20381] Updated weights for policy 1, policy_version 338296 (0.0011) [2023-12-27 07:59:10,740][20347] Updated weights for policy 0, policy_version 338536 (0.0009) [2023-12-27 07:59:10,795][20347] Updated weights for policy 0, policy_version 338546 (0.0010) [2023-12-27 07:59:10,852][20347] Updated weights for policy 0, policy_version 338556 (0.0010) [2023-12-27 07:59:11,259][20381] Updated weights for policy 1, policy_version 338306 (0.0009) [2023-12-27 07:59:11,321][20381] Updated weights for policy 1, policy_version 338316 (0.0011) [2023-12-27 07:59:11,387][20381] Updated weights for policy 1, policy_version 338326 (0.0009) [2023-12-27 07:59:11,546][20347] Updated weights for policy 0, policy_version 338566 (0.0008) [2023-12-27 07:59:11,615][20347] Updated weights for policy 0, policy_version 338576 (0.0008) [2023-12-27 07:59:11,676][20347] Updated weights for policy 0, policy_version 338586 (0.0009) [2023-12-27 07:59:12,067][20381] Updated weights for policy 1, policy_version 338336 (0.0006) [2023-12-27 07:59:12,125][20381] Updated weights for policy 1, policy_version 338346 (0.0008) [2023-12-27 07:59:12,185][20381] Updated weights for policy 1, policy_version 338356 (0.0010) [2023-12-27 07:59:12,306][20347] Updated weights for policy 0, policy_version 338596 (0.0008) [2023-12-27 07:59:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 173334528. Throughput: 0: 10423.8, 1: 10436.7. Samples: 173325300. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:12,363][19170] Avg episode reward: [(0, '343.840'), (1, '668.480')] [2023-12-27 07:59:12,367][20347] Updated weights for policy 0, policy_version 338606 (0.0007) [2023-12-27 07:59:12,423][20347] Updated weights for policy 0, policy_version 338616 (0.0010) [2023-12-27 07:59:12,869][20381] Updated weights for policy 1, policy_version 338366 (0.0009) [2023-12-27 07:59:12,934][20381] Updated weights for policy 1, policy_version 338376 (0.0011) [2023-12-27 07:59:12,994][20381] Updated weights for policy 1, policy_version 338386 (0.0011) [2023-12-27 07:59:13,073][20347] Updated weights for policy 0, policy_version 338626 (0.0007) [2023-12-27 07:59:13,117][20347] Updated weights for policy 0, policy_version 338636 (0.0008) [2023-12-27 07:59:13,170][20347] Updated weights for policy 0, policy_version 338646 (0.0008) [2023-12-27 07:59:13,230][20347] Updated weights for policy 0, policy_version 338656 (0.0010) [2023-12-27 07:59:13,637][20381] Updated weights for policy 1, policy_version 338396 (0.0011) [2023-12-27 07:59:13,700][20381] Updated weights for policy 1, policy_version 338406 (0.0010) [2023-12-27 07:59:13,752][20381] Updated weights for policy 1, policy_version 338416 (0.0010) [2023-12-27 07:59:13,866][20347] Updated weights for policy 0, policy_version 338666 (0.0007) [2023-12-27 07:59:13,918][20347] Updated weights for policy 0, policy_version 338676 (0.0007) [2023-12-27 07:59:13,974][20347] Updated weights for policy 0, policy_version 338686 (0.0006) [2023-12-27 07:59:14,376][20381] Updated weights for policy 1, policy_version 338426 (0.0010) [2023-12-27 07:59:14,428][20381] Updated weights for policy 1, policy_version 338436 (0.0011) [2023-12-27 07:59:14,480][20381] Updated weights for policy 1, policy_version 338446 (0.0010) [2023-12-27 07:59:14,531][20381] Updated weights for policy 1, policy_version 338456 (0.0010) [2023-12-27 07:59:14,644][20347] Updated weights for policy 0, policy_version 338696 (0.0007) [2023-12-27 07:59:14,696][20347] Updated weights for policy 0, policy_version 338706 (0.0007) [2023-12-27 07:59:14,746][20347] Updated weights for policy 0, policy_version 338716 (0.0007) [2023-12-27 07:59:15,252][20381] Updated weights for policy 1, policy_version 338466 (0.0010) [2023-12-27 07:59:15,319][20381] Updated weights for policy 1, policy_version 338476 (0.0011) [2023-12-27 07:59:15,374][20381] Updated weights for policy 1, policy_version 338486 (0.0011) [2023-12-27 07:59:15,515][20347] Updated weights for policy 0, policy_version 338726 (0.0009) [2023-12-27 07:59:15,578][20347] Updated weights for policy 0, policy_version 338736 (0.0010) [2023-12-27 07:59:15,627][20347] Updated weights for policy 0, policy_version 338746 (0.0007) [2023-12-27 07:59:16,051][20381] Updated weights for policy 1, policy_version 338496 (0.0007) [2023-12-27 07:59:16,109][20381] Updated weights for policy 1, policy_version 338506 (0.0006) [2023-12-27 07:59:16,172][20381] Updated weights for policy 1, policy_version 338516 (0.0006) [2023-12-27 07:59:16,204][20347] Updated weights for policy 0, policy_version 338756 (0.0005) [2023-12-27 07:59:16,248][20347] Updated weights for policy 0, policy_version 338766 (0.0007) [2023-12-27 07:59:16,294][20347] Updated weights for policy 0, policy_version 338776 (0.0005) [2023-12-27 07:59:16,807][20381] Updated weights for policy 1, policy_version 338526 (0.0009) [2023-12-27 07:59:16,839][20347] Updated weights for policy 0, policy_version 338786 (0.0006) [2023-12-27 07:59:16,859][20381] Updated weights for policy 1, policy_version 338536 (0.0010) [2023-12-27 07:59:16,901][20347] Updated weights for policy 0, policy_version 338796 (0.0010) [2023-12-27 07:59:16,917][20381] Updated weights for policy 1, policy_version 338546 (0.0010) [2023-12-27 07:59:16,965][20347] Updated weights for policy 0, policy_version 338806 (0.0008) [2023-12-27 07:59:17,018][20347] Updated weights for policy 0, policy_version 338816 (0.0007) [2023-12-27 07:59:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 173441024. Throughput: 0: 10461.1, 1: 10448.3. Samples: 173450216. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:17,363][19170] Avg episode reward: [(0, '383.830'), (1, '533.690')] [2023-12-27 07:59:17,601][20381] Updated weights for policy 1, policy_version 338556 (0.0011) [2023-12-27 07:59:17,616][20347] Updated weights for policy 0, policy_version 338826 (0.0009) [2023-12-27 07:59:17,650][20381] Updated weights for policy 1, policy_version 338566 (0.0011) [2023-12-27 07:59:17,667][20347] Updated weights for policy 0, policy_version 338836 (0.0010) [2023-12-27 07:59:17,701][20381] Updated weights for policy 1, policy_version 338576 (0.0010) [2023-12-27 07:59:17,726][20347] Updated weights for policy 0, policy_version 338846 (0.0010) [2023-12-27 07:59:18,342][20347] Updated weights for policy 0, policy_version 338856 (0.0009) [2023-12-27 07:59:18,403][20347] Updated weights for policy 0, policy_version 338866 (0.0010) [2023-12-27 07:59:18,404][20381] Updated weights for policy 1, policy_version 338586 (0.0011) [2023-12-27 07:59:18,458][20347] Updated weights for policy 0, policy_version 338876 (0.0010) [2023-12-27 07:59:18,462][20381] Updated weights for policy 1, policy_version 338596 (0.0011) [2023-12-27 07:59:18,517][20381] Updated weights for policy 1, policy_version 338606 (0.0010) [2023-12-27 07:59:18,572][20381] Updated weights for policy 1, policy_version 338616 (0.0011) [2023-12-27 07:59:19,134][20347] Updated weights for policy 0, policy_version 338886 (0.0008) [2023-12-27 07:59:19,177][20347] Updated weights for policy 0, policy_version 338896 (0.0007) [2023-12-27 07:59:19,178][20381] Updated weights for policy 1, policy_version 338626 (0.0005) [2023-12-27 07:59:19,229][20347] Updated weights for policy 0, policy_version 338906 (0.0007) [2023-12-27 07:59:19,235][20381] Updated weights for policy 1, policy_version 338636 (0.0010) [2023-12-27 07:59:19,287][20381] Updated weights for policy 1, policy_version 338646 (0.0007) [2023-12-27 07:59:19,942][20381] Updated weights for policy 1, policy_version 338656 (0.0008) [2023-12-27 07:59:19,954][20347] Updated weights for policy 0, policy_version 338916 (0.0007) [2023-12-27 07:59:20,004][20381] Updated weights for policy 1, policy_version 338666 (0.0008) [2023-12-27 07:59:20,014][20347] Updated weights for policy 0, policy_version 338926 (0.0008) [2023-12-27 07:59:20,056][20381] Updated weights for policy 1, policy_version 338676 (0.0007) [2023-12-27 07:59:20,073][20347] Updated weights for policy 0, policy_version 338936 (0.0008) [2023-12-27 07:59:20,799][20381] Updated weights for policy 1, policy_version 338686 (0.0007) [2023-12-27 07:59:20,803][20347] Updated weights for policy 0, policy_version 338946 (0.0007) [2023-12-27 07:59:20,857][20347] Updated weights for policy 0, policy_version 338956 (0.0008) [2023-12-27 07:59:20,859][20381] Updated weights for policy 1, policy_version 338696 (0.0007) [2023-12-27 07:59:20,908][20347] Updated weights for policy 0, policy_version 338966 (0.0011) [2023-12-27 07:59:20,913][20381] Updated weights for policy 1, policy_version 338706 (0.0006) [2023-12-27 07:59:20,962][20347] Updated weights for policy 0, policy_version 338976 (0.0010) [2023-12-27 07:59:21,670][20347] Updated weights for policy 0, policy_version 338986 (0.0008) [2023-12-27 07:59:21,687][20381] Updated weights for policy 1, policy_version 338716 (0.0005) [2023-12-27 07:59:21,732][20347] Updated weights for policy 0, policy_version 338996 (0.0008) [2023-12-27 07:59:21,752][20381] Updated weights for policy 1, policy_version 338726 (0.0009) [2023-12-27 07:59:21,793][20347] Updated weights for policy 0, policy_version 339006 (0.0006) [2023-12-27 07:59:21,812][20381] Updated weights for policy 1, policy_version 338736 (0.0011) [2023-12-27 07:59:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 173539328. Throughput: 0: 10452.8, 1: 10415.5. Samples: 173513340. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:22,363][19170] Avg episode reward: [(0, '350.020'), (1, '601.010')] [2023-12-27 07:59:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000338744_86736896.pth... [2023-12-27 07:59:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000339008_86802432.pth... [2023-12-27 07:59:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000337528_86425600.pth [2023-12-27 07:59:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000337792_86491136.pth [2023-12-27 07:59:22,458][20347] Updated weights for policy 0, policy_version 339016 (0.0010) [2023-12-27 07:59:22,499][20381] Updated weights for policy 1, policy_version 338746 (0.0010) [2023-12-27 07:59:22,514][20347] Updated weights for policy 0, policy_version 339026 (0.0011) [2023-12-27 07:59:22,554][20381] Updated weights for policy 1, policy_version 338756 (0.0009) [2023-12-27 07:59:22,573][20347] Updated weights for policy 0, policy_version 339036 (0.0007) [2023-12-27 07:59:22,614][20381] Updated weights for policy 1, policy_version 338766 (0.0006) [2023-12-27 07:59:22,675][20381] Updated weights for policy 1, policy_version 338776 (0.0007) [2023-12-27 07:59:23,235][20347] Updated weights for policy 0, policy_version 339046 (0.0008) [2023-12-27 07:59:23,248][20381] Updated weights for policy 1, policy_version 338786 (0.0005) [2023-12-27 07:59:23,284][20347] Updated weights for policy 0, policy_version 339056 (0.0010) [2023-12-27 07:59:23,294][20381] Updated weights for policy 1, policy_version 338796 (0.0005) [2023-12-27 07:59:23,338][20347] Updated weights for policy 0, policy_version 339066 (0.0010) [2023-12-27 07:59:23,340][20381] Updated weights for policy 1, policy_version 338806 (0.0007) [2023-12-27 07:59:23,953][20381] Updated weights for policy 1, policy_version 338816 (0.0009) [2023-12-27 07:59:23,983][20347] Updated weights for policy 0, policy_version 339076 (0.0010) [2023-12-27 07:59:24,009][20381] Updated weights for policy 1, policy_version 338826 (0.0006) [2023-12-27 07:59:24,036][20347] Updated weights for policy 0, policy_version 339086 (0.0010) [2023-12-27 07:59:24,068][20381] Updated weights for policy 1, policy_version 338836 (0.0009) [2023-12-27 07:59:24,085][20347] Updated weights for policy 0, policy_version 339096 (0.0010) [2023-12-27 07:59:24,700][20381] Updated weights for policy 1, policy_version 338846 (0.0007) [2023-12-27 07:59:24,748][20347] Updated weights for policy 0, policy_version 339106 (0.0010) [2023-12-27 07:59:24,759][20381] Updated weights for policy 1, policy_version 338856 (0.0007) [2023-12-27 07:59:24,803][20347] Updated weights for policy 0, policy_version 339116 (0.0007) [2023-12-27 07:59:24,815][20381] Updated weights for policy 1, policy_version 338866 (0.0005) [2023-12-27 07:59:24,858][20347] Updated weights for policy 0, policy_version 339126 (0.0007) [2023-12-27 07:59:24,918][20347] Updated weights for policy 0, policy_version 339136 (0.0007) [2023-12-27 07:59:25,358][20381] Updated weights for policy 1, policy_version 338876 (0.0005) [2023-12-27 07:59:25,412][20381] Updated weights for policy 1, policy_version 338886 (0.0005) [2023-12-27 07:59:25,460][20381] Updated weights for policy 1, policy_version 338896 (0.0005) [2023-12-27 07:59:25,574][20347] Updated weights for policy 0, policy_version 339146 (0.0007) [2023-12-27 07:59:25,627][20347] Updated weights for policy 0, policy_version 339156 (0.0007) [2023-12-27 07:59:25,677][20347] Updated weights for policy 0, policy_version 339166 (0.0007) [2023-12-27 07:59:26,082][20381] Updated weights for policy 1, policy_version 338906 (0.0005) [2023-12-27 07:59:26,133][20381] Updated weights for policy 1, policy_version 338916 (0.0005) [2023-12-27 07:59:26,185][20381] Updated weights for policy 1, policy_version 338926 (0.0011) [2023-12-27 07:59:26,234][20381] Updated weights for policy 1, policy_version 338936 (0.0010) [2023-12-27 07:59:26,360][20347] Updated weights for policy 0, policy_version 339176 (0.0007) [2023-12-27 07:59:26,411][20347] Updated weights for policy 0, policy_version 339186 (0.0007) [2023-12-27 07:59:26,454][20347] Updated weights for policy 0, policy_version 339196 (0.0007) [2023-12-27 07:59:26,869][20381] Updated weights for policy 1, policy_version 338946 (0.0010) [2023-12-27 07:59:26,917][20381] Updated weights for policy 1, policy_version 338956 (0.0010) [2023-12-27 07:59:26,965][20381] Updated weights for policy 1, policy_version 338966 (0.0010) [2023-12-27 07:59:27,129][20347] Updated weights for policy 0, policy_version 339206 (0.0007) [2023-12-27 07:59:27,175][20347] Updated weights for policy 0, policy_version 339216 (0.0006) [2023-12-27 07:59:27,223][20347] Updated weights for policy 0, policy_version 339226 (0.0005) [2023-12-27 07:59:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 173654016. Throughput: 0: 10442.2, 1: 10491.3. Samples: 173639908. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:27,364][19170] Avg episode reward: [(0, '293.230'), (1, '620.180')] [2023-12-27 07:59:27,558][20381] Updated weights for policy 1, policy_version 338976 (0.0010) [2023-12-27 07:59:27,612][20381] Updated weights for policy 1, policy_version 338986 (0.0006) [2023-12-27 07:59:27,660][20381] Updated weights for policy 1, policy_version 338996 (0.0005) [2023-12-27 07:59:27,822][20347] Updated weights for policy 0, policy_version 339236 (0.0006) [2023-12-27 07:59:27,883][20347] Updated weights for policy 0, policy_version 339246 (0.0007) [2023-12-27 07:59:27,938][20347] Updated weights for policy 0, policy_version 339256 (0.0007) [2023-12-27 07:59:28,294][20381] Updated weights for policy 1, policy_version 339006 (0.0006) [2023-12-27 07:59:28,352][20381] Updated weights for policy 1, policy_version 339016 (0.0009) [2023-12-27 07:59:28,412][20381] Updated weights for policy 1, policy_version 339026 (0.0007) [2023-12-27 07:59:28,626][20347] Updated weights for policy 0, policy_version 339266 (0.0008) [2023-12-27 07:59:28,682][20347] Updated weights for policy 0, policy_version 339276 (0.0008) [2023-12-27 07:59:28,731][20347] Updated weights for policy 0, policy_version 339286 (0.0007) [2023-12-27 07:59:28,784][20347] Updated weights for policy 0, policy_version 339296 (0.0007) [2023-12-27 07:59:29,120][20381] Updated weights for policy 1, policy_version 339036 (0.0008) [2023-12-27 07:59:29,176][20381] Updated weights for policy 1, policy_version 339046 (0.0005) [2023-12-27 07:59:29,234][20381] Updated weights for policy 1, policy_version 339056 (0.0008) [2023-12-27 07:59:29,464][20347] Updated weights for policy 0, policy_version 339306 (0.0007) [2023-12-27 07:59:29,525][20347] Updated weights for policy 0, policy_version 339316 (0.0007) [2023-12-27 07:59:29,585][20347] Updated weights for policy 0, policy_version 339326 (0.0007) [2023-12-27 07:59:29,885][20381] Updated weights for policy 1, policy_version 339066 (0.0008) [2023-12-27 07:59:29,944][20381] Updated weights for policy 1, policy_version 339076 (0.0008) [2023-12-27 07:59:30,002][20381] Updated weights for policy 1, policy_version 339086 (0.0008) [2023-12-27 07:59:30,060][20381] Updated weights for policy 1, policy_version 339096 (0.0008) [2023-12-27 07:59:30,278][20347] Updated weights for policy 0, policy_version 339336 (0.0010) [2023-12-27 07:59:30,339][20347] Updated weights for policy 0, policy_version 339346 (0.0010) [2023-12-27 07:59:30,399][20347] Updated weights for policy 0, policy_version 339356 (0.0010) [2023-12-27 07:59:30,648][20381] Updated weights for policy 1, policy_version 339106 (0.0005) [2023-12-27 07:59:30,705][20381] Updated weights for policy 1, policy_version 339116 (0.0005) [2023-12-27 07:59:30,754][20381] Updated weights for policy 1, policy_version 339126 (0.0010) [2023-12-27 07:59:31,018][20347] Updated weights for policy 0, policy_version 339366 (0.0008) [2023-12-27 07:59:31,075][20347] Updated weights for policy 0, policy_version 339376 (0.0008) [2023-12-27 07:59:31,130][20347] Updated weights for policy 0, policy_version 339386 (0.0008) [2023-12-27 07:59:31,425][20381] Updated weights for policy 1, policy_version 339136 (0.0008) [2023-12-27 07:59:31,486][20381] Updated weights for policy 1, policy_version 339146 (0.0010) [2023-12-27 07:59:31,544][20381] Updated weights for policy 1, policy_version 339156 (0.0011) [2023-12-27 07:59:31,785][20347] Updated weights for policy 0, policy_version 339396 (0.0007) [2023-12-27 07:59:31,837][20347] Updated weights for policy 0, policy_version 339406 (0.0008) [2023-12-27 07:59:31,892][20347] Updated weights for policy 0, policy_version 339416 (0.0006) [2023-12-27 07:59:32,242][20381] Updated weights for policy 1, policy_version 339166 (0.0011) [2023-12-27 07:59:32,300][20381] Updated weights for policy 1, policy_version 339176 (0.0008) [2023-12-27 07:59:32,356][20381] Updated weights for policy 1, policy_version 339186 (0.0007) [2023-12-27 07:59:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 173752320. Throughput: 0: 10447.3, 1: 10557.2. Samples: 173767368. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:32,363][19170] Avg episode reward: [(0, '368.260'), (1, '698.370')] [2023-12-27 07:59:32,526][20347] Updated weights for policy 0, policy_version 339426 (0.0006) [2023-12-27 07:59:32,578][20347] Updated weights for policy 0, policy_version 339436 (0.0009) [2023-12-27 07:59:32,630][20347] Updated weights for policy 0, policy_version 339446 (0.0009) [2023-12-27 07:59:32,682][20347] Updated weights for policy 0, policy_version 339456 (0.0008) [2023-12-27 07:59:32,933][20381] Updated weights for policy 1, policy_version 339196 (0.0008) [2023-12-27 07:59:32,981][20381] Updated weights for policy 1, policy_version 339206 (0.0010) [2023-12-27 07:59:33,030][20381] Updated weights for policy 1, policy_version 339216 (0.0010) [2023-12-27 07:59:33,367][20347] Updated weights for policy 0, policy_version 339466 (0.0005) [2023-12-27 07:59:33,430][20347] Updated weights for policy 0, policy_version 339476 (0.0005) [2023-12-27 07:59:33,492][20347] Updated weights for policy 0, policy_version 339486 (0.0006) [2023-12-27 07:59:33,712][20381] Updated weights for policy 1, policy_version 339226 (0.0010) [2023-12-27 07:59:33,759][20381] Updated weights for policy 1, policy_version 339236 (0.0010) [2023-12-27 07:59:33,811][20381] Updated weights for policy 1, policy_version 339246 (0.0010) [2023-12-27 07:59:33,862][20381] Updated weights for policy 1, policy_version 339256 (0.0010) [2023-12-27 07:59:34,041][20347] Updated weights for policy 0, policy_version 339496 (0.0006) [2023-12-27 07:59:34,099][20347] Updated weights for policy 0, policy_version 339506 (0.0005) [2023-12-27 07:59:34,159][20347] Updated weights for policy 0, policy_version 339516 (0.0008) [2023-12-27 07:59:34,550][20381] Updated weights for policy 1, policy_version 339266 (0.0010) [2023-12-27 07:59:34,605][20381] Updated weights for policy 1, policy_version 339276 (0.0010) [2023-12-27 07:59:34,660][20381] Updated weights for policy 1, policy_version 339286 (0.0010) [2023-12-27 07:59:34,806][20347] Updated weights for policy 0, policy_version 339526 (0.0009) [2023-12-27 07:59:34,869][20347] Updated weights for policy 0, policy_version 339536 (0.0011) [2023-12-27 07:59:34,928][20347] Updated weights for policy 0, policy_version 339546 (0.0010) [2023-12-27 07:59:35,292][20381] Updated weights for policy 1, policy_version 339296 (0.0007) [2023-12-27 07:59:35,347][20381] Updated weights for policy 1, policy_version 339306 (0.0011) [2023-12-27 07:59:35,396][20381] Updated weights for policy 1, policy_version 339316 (0.0011) [2023-12-27 07:59:35,534][20347] Updated weights for policy 0, policy_version 339556 (0.0011) [2023-12-27 07:59:35,590][20347] Updated weights for policy 0, policy_version 339566 (0.0007) [2023-12-27 07:59:35,638][20347] Updated weights for policy 0, policy_version 339576 (0.0007) [2023-12-27 07:59:36,013][20381] Updated weights for policy 1, policy_version 339326 (0.0009) [2023-12-27 07:59:36,076][20381] Updated weights for policy 1, policy_version 339336 (0.0005) [2023-12-27 07:59:36,142][20381] Updated weights for policy 1, policy_version 339346 (0.0010) [2023-12-27 07:59:36,309][20347] Updated weights for policy 0, policy_version 339586 (0.0010) [2023-12-27 07:59:36,371][20347] Updated weights for policy 0, policy_version 339596 (0.0010) [2023-12-27 07:59:36,419][20347] Updated weights for policy 0, policy_version 339606 (0.0010) [2023-12-27 07:59:36,469][20347] Updated weights for policy 0, policy_version 339616 (0.0010) [2023-12-27 07:59:36,807][20381] Updated weights for policy 1, policy_version 339356 (0.0010) [2023-12-27 07:59:36,856][20381] Updated weights for policy 1, policy_version 339366 (0.0011) [2023-12-27 07:59:36,906][20381] Updated weights for policy 1, policy_version 339376 (0.0006) [2023-12-27 07:59:37,127][20347] Updated weights for policy 0, policy_version 339626 (0.0009) [2023-12-27 07:59:37,179][20347] Updated weights for policy 0, policy_version 339636 (0.0010) [2023-12-27 07:59:37,234][20347] Updated weights for policy 0, policy_version 339646 (0.0009) [2023-12-27 07:59:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 173867008. Throughput: 0: 10459.5, 1: 10589.8. Samples: 173832148. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:37,363][19170] Avg episode reward: [(0, '463.110'), (1, '766.870')] [2023-12-27 07:59:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000339648_86966272.pth... [2023-12-27 07:59:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000339384_86900736.pth... [2023-12-27 07:59:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000338168_86589440.pth [2023-12-27 07:59:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000338400_86646784.pth [2023-12-27 07:59:37,569][20381] Updated weights for policy 1, policy_version 339386 (0.0006) [2023-12-27 07:59:37,622][20381] Updated weights for policy 1, policy_version 339396 (0.0008) [2023-12-27 07:59:37,669][20381] Updated weights for policy 1, policy_version 339406 (0.0005) [2023-12-27 07:59:37,729][20381] Updated weights for policy 1, policy_version 339416 (0.0008) [2023-12-27 07:59:37,852][20347] Updated weights for policy 0, policy_version 339656 (0.0006) [2023-12-27 07:59:37,906][20347] Updated weights for policy 0, policy_version 339666 (0.0007) [2023-12-27 07:59:37,967][20347] Updated weights for policy 0, policy_version 339676 (0.0007) [2023-12-27 07:59:38,405][20381] Updated weights for policy 1, policy_version 339426 (0.0010) [2023-12-27 07:59:38,466][20381] Updated weights for policy 1, policy_version 339436 (0.0010) [2023-12-27 07:59:38,532][20381] Updated weights for policy 1, policy_version 339446 (0.0010) [2023-12-27 07:59:38,635][20347] Updated weights for policy 0, policy_version 339686 (0.0008) [2023-12-27 07:59:38,689][20347] Updated weights for policy 0, policy_version 339696 (0.0010) [2023-12-27 07:59:38,744][20347] Updated weights for policy 0, policy_version 339706 (0.0010) [2023-12-27 07:59:39,211][20381] Updated weights for policy 1, policy_version 339456 (0.0008) [2023-12-27 07:59:39,271][20381] Updated weights for policy 1, policy_version 339466 (0.0009) [2023-12-27 07:59:39,336][20381] Updated weights for policy 1, policy_version 339476 (0.0008) [2023-12-27 07:59:39,444][20347] Updated weights for policy 0, policy_version 339716 (0.0011) [2023-12-27 07:59:39,506][20347] Updated weights for policy 0, policy_version 339726 (0.0011) [2023-12-27 07:59:39,565][20347] Updated weights for policy 0, policy_version 339736 (0.0011) [2023-12-27 07:59:40,092][20381] Updated weights for policy 1, policy_version 339486 (0.0007) [2023-12-27 07:59:40,148][20381] Updated weights for policy 1, policy_version 339496 (0.0008) [2023-12-27 07:59:40,196][20381] Updated weights for policy 1, policy_version 339506 (0.0007) [2023-12-27 07:59:40,273][20347] Updated weights for policy 0, policy_version 339746 (0.0011) [2023-12-27 07:59:40,329][20347] Updated weights for policy 0, policy_version 339756 (0.0008) [2023-12-27 07:59:40,382][20347] Updated weights for policy 0, policy_version 339766 (0.0010) [2023-12-27 07:59:40,433][20347] Updated weights for policy 0, policy_version 339776 (0.0010) [2023-12-27 07:59:40,916][20381] Updated weights for policy 1, policy_version 339516 (0.0008) [2023-12-27 07:59:40,982][20381] Updated weights for policy 1, policy_version 339526 (0.0008) [2023-12-27 07:59:41,050][20381] Updated weights for policy 1, policy_version 339536 (0.0008) [2023-12-27 07:59:41,127][20347] Updated weights for policy 0, policy_version 339786 (0.0007) [2023-12-27 07:59:41,191][20347] Updated weights for policy 0, policy_version 339796 (0.0011) [2023-12-27 07:59:41,252][20347] Updated weights for policy 0, policy_version 339806 (0.0009) [2023-12-27 07:59:41,770][20381] Updated weights for policy 1, policy_version 339546 (0.0010) [2023-12-27 07:59:41,818][20381] Updated weights for policy 1, policy_version 339556 (0.0008) [2023-12-27 07:59:41,863][20381] Updated weights for policy 1, policy_version 339566 (0.0008) [2023-12-27 07:59:41,910][20381] Updated weights for policy 1, policy_version 339576 (0.0007) [2023-12-27 07:59:41,957][20347] Updated weights for policy 0, policy_version 339816 (0.0008) [2023-12-27 07:59:42,010][20347] Updated weights for policy 0, policy_version 339826 (0.0008) [2023-12-27 07:59:42,063][20347] Updated weights for policy 0, policy_version 339836 (0.0007) [2023-12-27 07:59:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 173965312. Throughput: 0: 10511.7, 1: 10547.4. Samples: 173954900. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:42,363][19170] Avg episode reward: [(0, '348.950'), (1, '805.420')] [2023-12-27 07:59:42,528][20381] Updated weights for policy 1, policy_version 339586 (0.0005) [2023-12-27 07:59:42,592][20381] Updated weights for policy 1, policy_version 339596 (0.0005) [2023-12-27 07:59:42,655][20381] Updated weights for policy 1, policy_version 339606 (0.0009) [2023-12-27 07:59:42,763][20347] Updated weights for policy 0, policy_version 339846 (0.0007) [2023-12-27 07:59:42,822][20347] Updated weights for policy 0, policy_version 339856 (0.0007) [2023-12-27 07:59:42,879][20347] Updated weights for policy 0, policy_version 339866 (0.0008) [2023-12-27 07:59:43,223][20381] Updated weights for policy 1, policy_version 339616 (0.0007) [2023-12-27 07:59:43,272][20381] Updated weights for policy 1, policy_version 339626 (0.0008) [2023-12-27 07:59:43,321][20381] Updated weights for policy 1, policy_version 339636 (0.0008) [2023-12-27 07:59:43,616][20347] Updated weights for policy 0, policy_version 339876 (0.0006) [2023-12-27 07:59:43,673][20347] Updated weights for policy 0, policy_version 339886 (0.0007) [2023-12-27 07:59:43,734][20347] Updated weights for policy 0, policy_version 339896 (0.0007) [2023-12-27 07:59:43,962][20381] Updated weights for policy 1, policy_version 339646 (0.0009) [2023-12-27 07:59:44,017][20381] Updated weights for policy 1, policy_version 339656 (0.0010) [2023-12-27 07:59:44,075][20381] Updated weights for policy 1, policy_version 339666 (0.0010) [2023-12-27 07:59:44,430][20347] Updated weights for policy 0, policy_version 339906 (0.0008) [2023-12-27 07:59:44,477][20347] Updated weights for policy 0, policy_version 339916 (0.0007) [2023-12-27 07:59:44,521][20347] Updated weights for policy 0, policy_version 339926 (0.0008) [2023-12-27 07:59:44,579][20347] Updated weights for policy 0, policy_version 339936 (0.0007) [2023-12-27 07:59:44,771][20381] Updated weights for policy 1, policy_version 339676 (0.0009) [2023-12-27 07:59:44,828][20381] Updated weights for policy 1, policy_version 339686 (0.0007) [2023-12-27 07:59:44,890][20381] Updated weights for policy 1, policy_version 339696 (0.0008) [2023-12-27 07:59:45,309][20347] Updated weights for policy 0, policy_version 339946 (0.0007) [2023-12-27 07:59:45,371][20347] Updated weights for policy 0, policy_version 339956 (0.0008) [2023-12-27 07:59:45,432][20347] Updated weights for policy 0, policy_version 339966 (0.0007) [2023-12-27 07:59:45,579][20381] Updated weights for policy 1, policy_version 339706 (0.0008) [2023-12-27 07:59:45,637][20381] Updated weights for policy 1, policy_version 339716 (0.0011) [2023-12-27 07:59:45,708][20381] Updated weights for policy 1, policy_version 339726 (0.0011) [2023-12-27 07:59:45,772][20381] Updated weights for policy 1, policy_version 339736 (0.0011) [2023-12-27 07:59:46,044][20347] Updated weights for policy 0, policy_version 339976 (0.0006) [2023-12-27 07:59:46,102][20347] Updated weights for policy 0, policy_version 339986 (0.0005) [2023-12-27 07:59:46,152][20347] Updated weights for policy 0, policy_version 339996 (0.0005) [2023-12-27 07:59:46,439][20381] Updated weights for policy 1, policy_version 339746 (0.0011) [2023-12-27 07:59:46,498][20381] Updated weights for policy 1, policy_version 339756 (0.0011) [2023-12-27 07:59:46,553][20381] Updated weights for policy 1, policy_version 339766 (0.0011) [2023-12-27 07:59:46,795][20347] Updated weights for policy 0, policy_version 340006 (0.0007) [2023-12-27 07:59:46,854][20347] Updated weights for policy 0, policy_version 340016 (0.0008) [2023-12-27 07:59:46,898][20347] Updated weights for policy 0, policy_version 340026 (0.0007) [2023-12-27 07:59:47,143][20381] Updated weights for policy 1, policy_version 339776 (0.0006) [2023-12-27 07:59:47,197][20381] Updated weights for policy 1, policy_version 339786 (0.0005) [2023-12-27 07:59:47,260][20381] Updated weights for policy 1, policy_version 339796 (0.0010) [2023-12-27 07:59:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 174071808. Throughput: 0: 10400.6, 1: 10561.1. Samples: 174078520. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:47,363][19170] Avg episode reward: [(0, '394.180'), (1, '805.590')] [2023-12-27 07:59:47,603][20347] Updated weights for policy 0, policy_version 340036 (0.0008) [2023-12-27 07:59:47,666][20347] Updated weights for policy 0, policy_version 340046 (0.0007) [2023-12-27 07:59:47,721][20347] Updated weights for policy 0, policy_version 340056 (0.0007) [2023-12-27 07:59:47,891][20381] Updated weights for policy 1, policy_version 339806 (0.0008) [2023-12-27 07:59:47,941][20381] Updated weights for policy 1, policy_version 339816 (0.0005) [2023-12-27 07:59:47,997][20381] Updated weights for policy 1, policy_version 339826 (0.0005) [2023-12-27 07:59:48,400][20347] Updated weights for policy 0, policy_version 340066 (0.0007) [2023-12-27 07:59:48,463][20347] Updated weights for policy 0, policy_version 340076 (0.0007) [2023-12-27 07:59:48,511][20347] Updated weights for policy 0, policy_version 340086 (0.0010) [2023-12-27 07:59:48,563][20347] Updated weights for policy 0, policy_version 340096 (0.0010) [2023-12-27 07:59:48,616][20381] Updated weights for policy 1, policy_version 339836 (0.0006) [2023-12-27 07:59:48,664][20381] Updated weights for policy 1, policy_version 339846 (0.0005) [2023-12-27 07:59:48,715][20381] Updated weights for policy 1, policy_version 339856 (0.0005) [2023-12-27 07:59:49,239][20347] Updated weights for policy 0, policy_version 340106 (0.0010) [2023-12-27 07:59:49,302][20347] Updated weights for policy 0, policy_version 340116 (0.0011) [2023-12-27 07:59:49,310][20381] Updated weights for policy 1, policy_version 339866 (0.0007) [2023-12-27 07:59:49,363][20347] Updated weights for policy 0, policy_version 340126 (0.0010) [2023-12-27 07:59:49,378][20381] Updated weights for policy 1, policy_version 339876 (0.0011) [2023-12-27 07:59:49,440][20381] Updated weights for policy 1, policy_version 339886 (0.0011) [2023-12-27 07:59:49,502][20381] Updated weights for policy 1, policy_version 339896 (0.0011) [2023-12-27 07:59:50,044][20347] Updated weights for policy 0, policy_version 340136 (0.0008) [2023-12-27 07:59:50,107][20347] Updated weights for policy 0, policy_version 340146 (0.0008) [2023-12-27 07:59:50,175][20347] Updated weights for policy 0, policy_version 340156 (0.0007) [2023-12-27 07:59:50,187][20381] Updated weights for policy 1, policy_version 339906 (0.0007) [2023-12-27 07:59:50,244][20381] Updated weights for policy 1, policy_version 339916 (0.0007) [2023-12-27 07:59:50,296][20381] Updated weights for policy 1, policy_version 339926 (0.0007) [2023-12-27 07:59:50,840][20347] Updated weights for policy 0, policy_version 340166 (0.0007) [2023-12-27 07:59:50,902][20347] Updated weights for policy 0, policy_version 340176 (0.0008) [2023-12-27 07:59:50,913][20381] Updated weights for policy 1, policy_version 339936 (0.0006) [2023-12-27 07:59:50,968][20347] Updated weights for policy 0, policy_version 340186 (0.0011) [2023-12-27 07:59:50,972][20381] Updated weights for policy 1, policy_version 339946 (0.0005) [2023-12-27 07:59:51,035][20381] Updated weights for policy 1, policy_version 339956 (0.0006) [2023-12-27 07:59:51,669][20381] Updated weights for policy 1, policy_version 339966 (0.0007) [2023-12-27 07:59:51,686][20347] Updated weights for policy 0, policy_version 340196 (0.0010) [2023-12-27 07:59:51,730][20381] Updated weights for policy 1, policy_version 339976 (0.0007) [2023-12-27 07:59:51,751][20347] Updated weights for policy 0, policy_version 340206 (0.0009) [2023-12-27 07:59:51,792][20381] Updated weights for policy 1, policy_version 339986 (0.0007) [2023-12-27 07:59:51,810][20347] Updated weights for policy 0, policy_version 340216 (0.0007) [2023-12-27 07:59:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 174170112. Throughput: 0: 10371.9, 1: 10519.7. Samples: 174141352. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:52,363][19170] Avg episode reward: [(0, '507.020'), (1, '854.360')] [2023-12-27 07:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000339992_87056384.pth... [2023-12-27 07:59:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000340224_87113728.pth... [2023-12-27 07:59:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000338744_86736896.pth [2023-12-27 07:59:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000339008_86802432.pth [2023-12-27 07:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000339992_87056384.pth [2023-12-27 07:59:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000340224_87113728.pth [2023-12-27 07:59:52,406][20347] Updated weights for policy 0, policy_version 340226 (0.0009) [2023-12-27 07:59:52,465][20347] Updated weights for policy 0, policy_version 340236 (0.0010) [2023-12-27 07:59:52,469][20381] Updated weights for policy 1, policy_version 339996 (0.0008) [2023-12-27 07:59:52,520][20347] Updated weights for policy 0, policy_version 340246 (0.0010) [2023-12-27 07:59:52,531][20381] Updated weights for policy 1, policy_version 340006 (0.0009) [2023-12-27 07:59:52,582][20347] Updated weights for policy 0, policy_version 340256 (0.0006) [2023-12-27 07:59:52,587][20381] Updated weights for policy 1, policy_version 340016 (0.0007) [2023-12-27 07:59:53,184][20347] Updated weights for policy 0, policy_version 340266 (0.0008) [2023-12-27 07:59:53,241][20347] Updated weights for policy 0, policy_version 340276 (0.0007) [2023-12-27 07:59:53,268][20381] Updated weights for policy 1, policy_version 340026 (0.0006) [2023-12-27 07:59:53,305][20347] Updated weights for policy 0, policy_version 340286 (0.0008) [2023-12-27 07:59:53,319][20381] Updated weights for policy 1, policy_version 340036 (0.0005) [2023-12-27 07:59:53,378][20381] Updated weights for policy 1, policy_version 340046 (0.0008) [2023-12-27 07:59:53,435][20381] Updated weights for policy 1, policy_version 340056 (0.0010) [2023-12-27 07:59:53,981][20347] Updated weights for policy 0, policy_version 340296 (0.0008) [2023-12-27 07:59:54,033][20347] Updated weights for policy 0, policy_version 340306 (0.0007) [2023-12-27 07:59:54,065][20381] Updated weights for policy 1, policy_version 340066 (0.0010) [2023-12-27 07:59:54,089][20347] Updated weights for policy 0, policy_version 340316 (0.0005) [2023-12-27 07:59:54,124][20381] Updated weights for policy 1, policy_version 340076 (0.0010) [2023-12-27 07:59:54,171][20381] Updated weights for policy 1, policy_version 340086 (0.0010) [2023-12-27 07:59:54,791][20347] Updated weights for policy 0, policy_version 340326 (0.0007) [2023-12-27 07:59:54,848][20347] Updated weights for policy 0, policy_version 340336 (0.0008) [2023-12-27 07:59:54,863][20381] Updated weights for policy 1, policy_version 340096 (0.0011) [2023-12-27 07:59:54,896][20347] Updated weights for policy 0, policy_version 340346 (0.0008) [2023-12-27 07:59:54,921][20381] Updated weights for policy 1, policy_version 340106 (0.0010) [2023-12-27 07:59:54,970][20381] Updated weights for policy 1, policy_version 340116 (0.0010) [2023-12-27 07:59:55,566][20347] Updated weights for policy 0, policy_version 340356 (0.0006) [2023-12-27 07:59:55,623][20347] Updated weights for policy 0, policy_version 340366 (0.0010) [2023-12-27 07:59:55,632][20381] Updated weights for policy 1, policy_version 340126 (0.0011) [2023-12-27 07:59:55,675][20347] Updated weights for policy 0, policy_version 340376 (0.0010) [2023-12-27 07:59:55,684][20381] Updated weights for policy 1, policy_version 340136 (0.0010) [2023-12-27 07:59:55,739][20381] Updated weights for policy 1, policy_version 340146 (0.0010) [2023-12-27 07:59:56,320][20347] Updated weights for policy 0, policy_version 340386 (0.0009) [2023-12-27 07:59:56,373][20347] Updated weights for policy 0, policy_version 340396 (0.0007) [2023-12-27 07:59:56,423][20381] Updated weights for policy 1, policy_version 340156 (0.0010) [2023-12-27 07:59:56,433][20347] Updated weights for policy 0, policy_version 340406 (0.0006) [2023-12-27 07:59:56,481][20381] Updated weights for policy 1, policy_version 340166 (0.0010) [2023-12-27 07:59:56,487][20347] Updated weights for policy 0, policy_version 340416 (0.0010) [2023-12-27 07:59:56,535][20381] Updated weights for policy 1, policy_version 340176 (0.0010) [2023-12-27 07:59:57,083][20347] Updated weights for policy 0, policy_version 340426 (0.0010) [2023-12-27 07:59:57,130][20347] Updated weights for policy 0, policy_version 340436 (0.0007) [2023-12-27 07:59:57,136][20381] Updated weights for policy 1, policy_version 340186 (0.0010) [2023-12-27 07:59:57,181][20347] Updated weights for policy 0, policy_version 340446 (0.0010) [2023-12-27 07:59:57,187][20381] Updated weights for policy 1, policy_version 340196 (0.0005) [2023-12-27 07:59:57,244][20381] Updated weights for policy 1, policy_version 340206 (0.0005) [2023-12-27 07:59:57,296][20381] Updated weights for policy 1, policy_version 340216 (0.0005) [2023-12-27 07:59:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21162.7, 300 sec: 20854.9). Total num frames: 174284800. Throughput: 0: 10396.1, 1: 10509.8. Samples: 174266064. Policy #0 lag: (min: 22.0, avg: 23.9, max: 54.0) [2023-12-27 07:59:57,363][19170] Avg episode reward: [(0, '465.880'), (1, '844.160')] [2023-12-27 07:59:57,878][20347] Updated weights for policy 0, policy_version 340456 (0.0010) [2023-12-27 07:59:57,890][20381] Updated weights for policy 1, policy_version 340226 (0.0005) [2023-12-27 07:59:57,939][20347] Updated weights for policy 0, policy_version 340466 (0.0010) [2023-12-27 07:59:57,946][20381] Updated weights for policy 1, policy_version 340236 (0.0005) [2023-12-27 07:59:57,997][20347] Updated weights for policy 0, policy_version 340476 (0.0010) [2023-12-27 07:59:58,001][20381] Updated weights for policy 1, policy_version 340246 (0.0006) [2023-12-27 07:59:58,688][20347] Updated weights for policy 0, policy_version 340486 (0.0009) [2023-12-27 07:59:58,716][20381] Updated weights for policy 1, policy_version 340256 (0.0008) [2023-12-27 07:59:58,746][20347] Updated weights for policy 0, policy_version 340496 (0.0008) [2023-12-27 07:59:58,774][20381] Updated weights for policy 1, policy_version 340266 (0.0007) [2023-12-27 07:59:58,811][20347] Updated weights for policy 0, policy_version 340506 (0.0009) [2023-12-27 07:59:58,836][20381] Updated weights for policy 1, policy_version 340276 (0.0008) [2023-12-27 07:59:59,505][20381] Updated weights for policy 1, policy_version 340286 (0.0008) [2023-12-27 07:59:59,559][20381] Updated weights for policy 1, policy_version 340296 (0.0007) [2023-12-27 07:59:59,612][20347] Updated weights for policy 0, policy_version 340516 (0.0007) [2023-12-27 07:59:59,619][20381] Updated weights for policy 1, policy_version 340306 (0.0008) [2023-12-27 07:59:59,670][20347] Updated weights for policy 0, policy_version 340526 (0.0008) [2023-12-27 07:59:59,740][20347] Updated weights for policy 0, policy_version 340536 (0.0006) [2023-12-27 08:00:00,287][20381] Updated weights for policy 1, policy_version 340316 (0.0009) [2023-12-27 08:00:00,345][20381] Updated weights for policy 1, policy_version 340326 (0.0008) [2023-12-27 08:00:00,385][20347] Updated weights for policy 0, policy_version 340546 (0.0009) [2023-12-27 08:00:00,401][20381] Updated weights for policy 1, policy_version 340336 (0.0006) [2023-12-27 08:00:00,445][20347] Updated weights for policy 0, policy_version 340556 (0.0006) [2023-12-27 08:00:00,505][20347] Updated weights for policy 0, policy_version 340566 (0.0009) [2023-12-27 08:00:00,555][20347] Updated weights for policy 0, policy_version 340576 (0.0010) [2023-12-27 08:00:00,971][20381] Updated weights for policy 1, policy_version 340346 (0.0007) [2023-12-27 08:00:01,019][20381] Updated weights for policy 1, policy_version 340356 (0.0010) [2023-12-27 08:00:01,082][20381] Updated weights for policy 1, policy_version 340366 (0.0011) [2023-12-27 08:00:01,144][20381] Updated weights for policy 1, policy_version 340376 (0.0011) [2023-12-27 08:00:01,218][20347] Updated weights for policy 0, policy_version 340586 (0.0005) [2023-12-27 08:00:01,281][20347] Updated weights for policy 0, policy_version 340596 (0.0009) [2023-12-27 08:00:01,349][20347] Updated weights for policy 0, policy_version 340606 (0.0010) [2023-12-27 08:00:01,854][20381] Updated weights for policy 1, policy_version 340386 (0.0008) [2023-12-27 08:00:01,913][20381] Updated weights for policy 1, policy_version 340396 (0.0007) [2023-12-27 08:00:01,961][20381] Updated weights for policy 1, policy_version 340406 (0.0008) [2023-12-27 08:00:02,036][20347] Updated weights for policy 0, policy_version 340616 (0.0011) [2023-12-27 08:00:02,101][20347] Updated weights for policy 0, policy_version 340626 (0.0010) [2023-12-27 08:00:02,152][20347] Updated weights for policy 0, policy_version 340636 (0.0010) [2023-12-27 08:00:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 174383104. Throughput: 0: 10352.6, 1: 10551.8. Samples: 174390916. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:02,363][19170] Avg episode reward: [(0, '311.720'), (1, '902.410')] [2023-12-27 08:00:02,672][20381] Updated weights for policy 1, policy_version 340416 (0.0008) [2023-12-27 08:00:02,731][20381] Updated weights for policy 1, policy_version 340426 (0.0008) [2023-12-27 08:00:02,785][20381] Updated weights for policy 1, policy_version 340436 (0.0008) [2023-12-27 08:00:02,830][20347] Updated weights for policy 0, policy_version 340646 (0.0009) [2023-12-27 08:00:02,877][20347] Updated weights for policy 0, policy_version 340656 (0.0007) [2023-12-27 08:00:02,919][20347] Updated weights for policy 0, policy_version 340666 (0.0010) [2023-12-27 08:00:03,457][20381] Updated weights for policy 1, policy_version 340446 (0.0007) [2023-12-27 08:00:03,515][20381] Updated weights for policy 1, policy_version 340456 (0.0008) [2023-12-27 08:00:03,566][20381] Updated weights for policy 1, policy_version 340466 (0.0007) [2023-12-27 08:00:03,611][20347] Updated weights for policy 0, policy_version 340676 (0.0009) [2023-12-27 08:00:03,667][20347] Updated weights for policy 0, policy_version 340686 (0.0010) [2023-12-27 08:00:03,715][20347] Updated weights for policy 0, policy_version 340696 (0.0010) [2023-12-27 08:00:04,275][20381] Updated weights for policy 1, policy_version 340476 (0.0008) [2023-12-27 08:00:04,323][20381] Updated weights for policy 1, policy_version 340486 (0.0007) [2023-12-27 08:00:04,374][20381] Updated weights for policy 1, policy_version 340496 (0.0007) [2023-12-27 08:00:04,386][20347] Updated weights for policy 0, policy_version 340706 (0.0010) [2023-12-27 08:00:04,445][20347] Updated weights for policy 0, policy_version 340716 (0.0007) [2023-12-27 08:00:04,506][20347] Updated weights for policy 0, policy_version 340726 (0.0008) [2023-12-27 08:00:04,561][20347] Updated weights for policy 0, policy_version 340736 (0.0010) [2023-12-27 08:00:05,073][20381] Updated weights for policy 1, policy_version 340506 (0.0006) [2023-12-27 08:00:05,125][20381] Updated weights for policy 1, policy_version 340516 (0.0008) [2023-12-27 08:00:05,182][20381] Updated weights for policy 1, policy_version 340526 (0.0007) [2023-12-27 08:00:05,236][20381] Updated weights for policy 1, policy_version 340536 (0.0006) [2023-12-27 08:00:05,263][20347] Updated weights for policy 0, policy_version 340746 (0.0008) [2023-12-27 08:00:05,327][20347] Updated weights for policy 0, policy_version 340756 (0.0008) [2023-12-27 08:00:05,385][20347] Updated weights for policy 0, policy_version 340766 (0.0005) [2023-12-27 08:00:05,949][20347] Updated weights for policy 0, policy_version 340776 (0.0007) [2023-12-27 08:00:05,962][20381] Updated weights for policy 1, policy_version 340546 (0.0007) [2023-12-27 08:00:05,997][20347] Updated weights for policy 0, policy_version 340786 (0.0007) [2023-12-27 08:00:06,013][20381] Updated weights for policy 1, policy_version 340556 (0.0006) [2023-12-27 08:00:06,050][20347] Updated weights for policy 0, policy_version 340796 (0.0006) [2023-12-27 08:00:06,066][20381] Updated weights for policy 1, policy_version 340566 (0.0006) [2023-12-27 08:00:06,766][20347] Updated weights for policy 0, policy_version 340806 (0.0008) [2023-12-27 08:00:06,801][20381] Updated weights for policy 1, policy_version 340576 (0.0010) [2023-12-27 08:00:06,820][20347] Updated weights for policy 0, policy_version 340816 (0.0006) [2023-12-27 08:00:06,857][20381] Updated weights for policy 1, policy_version 340586 (0.0010) [2023-12-27 08:00:06,872][20347] Updated weights for policy 0, policy_version 340826 (0.0009) [2023-12-27 08:00:06,914][20381] Updated weights for policy 1, policy_version 340596 (0.0011) [2023-12-27 08:00:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 174481408. Throughput: 0: 10324.5, 1: 10525.7. Samples: 174451600. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:07,364][19170] Avg episode reward: [(0, '334.750'), (1, '873.280')] [2023-12-27 08:00:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000340600_87212032.pth... [2023-12-27 08:00:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000340832_87269376.pth... [2023-12-27 08:00:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000339384_86900736.pth [2023-12-27 08:00:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000339648_86966272.pth [2023-12-27 08:00:07,530][20347] Updated weights for policy 0, policy_version 340836 (0.0009) [2023-12-27 08:00:07,551][20381] Updated weights for policy 1, policy_version 340606 (0.0011) [2023-12-27 08:00:07,586][20347] Updated weights for policy 0, policy_version 340846 (0.0005) [2023-12-27 08:00:07,602][20381] Updated weights for policy 1, policy_version 340616 (0.0010) [2023-12-27 08:00:07,640][20347] Updated weights for policy 0, policy_version 340856 (0.0006) [2023-12-27 08:00:07,658][20381] Updated weights for policy 1, policy_version 340626 (0.0010) [2023-12-27 08:00:08,246][20347] Updated weights for policy 0, policy_version 340866 (0.0008) [2023-12-27 08:00:08,307][20381] Updated weights for policy 1, policy_version 340636 (0.0011) [2023-12-27 08:00:08,308][20347] Updated weights for policy 0, policy_version 340876 (0.0006) [2023-12-27 08:00:08,367][20381] Updated weights for policy 1, policy_version 340646 (0.0009) [2023-12-27 08:00:08,372][20347] Updated weights for policy 0, policy_version 340886 (0.0006) [2023-12-27 08:00:08,423][20381] Updated weights for policy 1, policy_version 340656 (0.0007) [2023-12-27 08:00:08,432][20347] Updated weights for policy 0, policy_version 340896 (0.0006) [2023-12-27 08:00:09,013][20381] Updated weights for policy 1, policy_version 340666 (0.0006) [2023-12-27 08:00:09,075][20381] Updated weights for policy 1, policy_version 340676 (0.0008) [2023-12-27 08:00:09,137][20347] Updated weights for policy 0, policy_version 340906 (0.0009) [2023-12-27 08:00:09,137][20381] Updated weights for policy 1, policy_version 340686 (0.0008) [2023-12-27 08:00:09,191][20347] Updated weights for policy 0, policy_version 340916 (0.0011) [2023-12-27 08:00:09,199][20381] Updated weights for policy 1, policy_version 340696 (0.0008) [2023-12-27 08:00:09,251][20347] Updated weights for policy 0, policy_version 340926 (0.0011) [2023-12-27 08:00:09,940][20381] Updated weights for policy 1, policy_version 340706 (0.0008) [2023-12-27 08:00:09,991][20347] Updated weights for policy 0, policy_version 340936 (0.0007) [2023-12-27 08:00:10,001][20381] Updated weights for policy 1, policy_version 340716 (0.0007) [2023-12-27 08:00:10,052][20347] Updated weights for policy 0, policy_version 340946 (0.0007) [2023-12-27 08:00:10,061][20381] Updated weights for policy 1, policy_version 340726 (0.0009) [2023-12-27 08:00:10,111][20347] Updated weights for policy 0, policy_version 340956 (0.0007) [2023-12-27 08:00:10,785][20347] Updated weights for policy 0, policy_version 340966 (0.0007) [2023-12-27 08:00:10,832][20381] Updated weights for policy 1, policy_version 340736 (0.0006) [2023-12-27 08:00:10,838][20347] Updated weights for policy 0, policy_version 340976 (0.0006) [2023-12-27 08:00:10,890][20381] Updated weights for policy 1, policy_version 340746 (0.0007) [2023-12-27 08:00:10,895][20347] Updated weights for policy 0, policy_version 340986 (0.0008) [2023-12-27 08:00:10,948][20381] Updated weights for policy 1, policy_version 340756 (0.0006) [2023-12-27 08:00:11,664][20381] Updated weights for policy 1, policy_version 340766 (0.0008) [2023-12-27 08:00:11,678][20347] Updated weights for policy 0, policy_version 340996 (0.0007) [2023-12-27 08:00:11,728][20381] Updated weights for policy 1, policy_version 340776 (0.0009) [2023-12-27 08:00:11,735][20347] Updated weights for policy 0, policy_version 341006 (0.0007) [2023-12-27 08:00:11,793][20381] Updated weights for policy 1, policy_version 340786 (0.0008) [2023-12-27 08:00:11,799][20347] Updated weights for policy 0, policy_version 341016 (0.0007) [2023-12-27 08:00:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 174579712. Throughput: 0: 10314.4, 1: 10436.6. Samples: 174573700. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:12,363][19170] Avg episode reward: [(0, '481.090'), (1, '814.540')] [2023-12-27 08:00:12,492][20381] Updated weights for policy 1, policy_version 340796 (0.0007) [2023-12-27 08:00:12,531][20347] Updated weights for policy 0, policy_version 341026 (0.0008) [2023-12-27 08:00:12,552][20381] Updated weights for policy 1, policy_version 340806 (0.0007) [2023-12-27 08:00:12,596][20347] Updated weights for policy 0, policy_version 341036 (0.0007) [2023-12-27 08:00:12,610][20381] Updated weights for policy 1, policy_version 340816 (0.0006) [2023-12-27 08:00:12,656][20347] Updated weights for policy 0, policy_version 341046 (0.0006) [2023-12-27 08:00:12,719][20347] Updated weights for policy 0, policy_version 341056 (0.0007) [2023-12-27 08:00:13,237][20381] Updated weights for policy 1, policy_version 340826 (0.0007) [2023-12-27 08:00:13,297][20381] Updated weights for policy 1, policy_version 340836 (0.0007) [2023-12-27 08:00:13,355][20381] Updated weights for policy 1, policy_version 340846 (0.0008) [2023-12-27 08:00:13,418][20347] Updated weights for policy 0, policy_version 341066 (0.0007) [2023-12-27 08:00:13,420][20381] Updated weights for policy 1, policy_version 340856 (0.0007) [2023-12-27 08:00:13,467][20347] Updated weights for policy 0, policy_version 341076 (0.0007) [2023-12-27 08:00:13,520][20347] Updated weights for policy 0, policy_version 341086 (0.0008) [2023-12-27 08:00:14,073][20381] Updated weights for policy 1, policy_version 340866 (0.0007) [2023-12-27 08:00:14,126][20381] Updated weights for policy 1, policy_version 340876 (0.0008) [2023-12-27 08:00:14,181][20381] Updated weights for policy 1, policy_version 340886 (0.0008) [2023-12-27 08:00:14,216][20347] Updated weights for policy 0, policy_version 341096 (0.0008) [2023-12-27 08:00:14,288][20347] Updated weights for policy 0, policy_version 341106 (0.0006) [2023-12-27 08:00:14,345][20347] Updated weights for policy 0, policy_version 341116 (0.0007) [2023-12-27 08:00:14,892][20381] Updated weights for policy 1, policy_version 340896 (0.0006) [2023-12-27 08:00:14,946][20381] Updated weights for policy 1, policy_version 340906 (0.0006) [2023-12-27 08:00:15,003][20381] Updated weights for policy 1, policy_version 340916 (0.0006) [2023-12-27 08:00:15,006][20347] Updated weights for policy 0, policy_version 341126 (0.0008) [2023-12-27 08:00:15,064][20347] Updated weights for policy 0, policy_version 341136 (0.0007) [2023-12-27 08:00:15,129][20347] Updated weights for policy 0, policy_version 341146 (0.0010) [2023-12-27 08:00:15,696][20381] Updated weights for policy 1, policy_version 340926 (0.0007) [2023-12-27 08:00:15,753][20381] Updated weights for policy 1, policy_version 340936 (0.0008) [2023-12-27 08:00:15,802][20347] Updated weights for policy 0, policy_version 341156 (0.0007) [2023-12-27 08:00:15,808][20381] Updated weights for policy 1, policy_version 340946 (0.0007) [2023-12-27 08:00:15,851][20347] Updated weights for policy 0, policy_version 341166 (0.0007) [2023-12-27 08:00:15,904][20347] Updated weights for policy 0, policy_version 341176 (0.0005) [2023-12-27 08:00:16,454][20381] Updated weights for policy 1, policy_version 340956 (0.0006) [2023-12-27 08:00:16,511][20381] Updated weights for policy 1, policy_version 340966 (0.0007) [2023-12-27 08:00:16,519][20347] Updated weights for policy 0, policy_version 341186 (0.0006) [2023-12-27 08:00:16,570][20381] Updated weights for policy 1, policy_version 340976 (0.0005) [2023-12-27 08:00:16,580][20347] Updated weights for policy 0, policy_version 341196 (0.0007) [2023-12-27 08:00:16,629][20347] Updated weights for policy 0, policy_version 341206 (0.0010) [2023-12-27 08:00:16,685][20347] Updated weights for policy 0, policy_version 341216 (0.0010) [2023-12-27 08:00:17,236][20381] Updated weights for policy 1, policy_version 340986 (0.0006) [2023-12-27 08:00:17,295][20381] Updated weights for policy 1, policy_version 340996 (0.0008) [2023-12-27 08:00:17,332][20347] Updated weights for policy 0, policy_version 341226 (0.0010) [2023-12-27 08:00:17,349][20381] Updated weights for policy 1, policy_version 341006 (0.0006) [2023-12-27 08:00:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 174678016. Throughput: 0: 10253.3, 1: 10355.4. Samples: 174694760. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:17,363][19170] Avg episode reward: [(0, '358.210'), (1, '848.970')] [2023-12-27 08:00:17,380][20347] Updated weights for policy 0, policy_version 341236 (0.0008) [2023-12-27 08:00:17,404][20381] Updated weights for policy 1, policy_version 341016 (0.0006) [2023-12-27 08:00:17,439][20347] Updated weights for policy 0, policy_version 341246 (0.0010) [2023-12-27 08:00:18,123][20381] Updated weights for policy 1, policy_version 341026 (0.0007) [2023-12-27 08:00:18,123][20347] Updated weights for policy 0, policy_version 341256 (0.0011) [2023-12-27 08:00:18,173][20347] Updated weights for policy 0, policy_version 341266 (0.0010) [2023-12-27 08:00:18,175][20381] Updated weights for policy 1, policy_version 341036 (0.0008) [2023-12-27 08:00:18,217][20347] Updated weights for policy 0, policy_version 341276 (0.0010) [2023-12-27 08:00:18,229][20381] Updated weights for policy 1, policy_version 341046 (0.0005) [2023-12-27 08:00:18,955][20347] Updated weights for policy 0, policy_version 341286 (0.0009) [2023-12-27 08:00:18,964][20381] Updated weights for policy 1, policy_version 341056 (0.0007) [2023-12-27 08:00:19,012][20347] Updated weights for policy 0, policy_version 341296 (0.0009) [2023-12-27 08:00:19,021][20381] Updated weights for policy 1, policy_version 341066 (0.0007) [2023-12-27 08:00:19,067][20347] Updated weights for policy 0, policy_version 341306 (0.0008) [2023-12-27 08:00:19,080][20381] Updated weights for policy 1, policy_version 341076 (0.0008) [2023-12-27 08:00:19,800][20381] Updated weights for policy 1, policy_version 341086 (0.0007) [2023-12-27 08:00:19,859][20381] Updated weights for policy 1, policy_version 341096 (0.0007) [2023-12-27 08:00:19,924][20381] Updated weights for policy 1, policy_version 341106 (0.0007) [2023-12-27 08:00:19,926][20347] Updated weights for policy 0, policy_version 341316 (0.0008) [2023-12-27 08:00:19,991][20347] Updated weights for policy 0, policy_version 341326 (0.0008) [2023-12-27 08:00:20,056][20347] Updated weights for policy 0, policy_version 341336 (0.0008) [2023-12-27 08:00:20,622][20381] Updated weights for policy 1, policy_version 341116 (0.0007) [2023-12-27 08:00:20,685][20381] Updated weights for policy 1, policy_version 341126 (0.0006) [2023-12-27 08:00:20,740][20381] Updated weights for policy 1, policy_version 341136 (0.0008) [2023-12-27 08:00:20,848][20347] Updated weights for policy 0, policy_version 341346 (0.0009) [2023-12-27 08:00:20,909][20347] Updated weights for policy 0, policy_version 341356 (0.0008) [2023-12-27 08:00:20,968][20347] Updated weights for policy 0, policy_version 341366 (0.0008) [2023-12-27 08:00:21,030][20347] Updated weights for policy 0, policy_version 341376 (0.0008) [2023-12-27 08:00:21,517][20381] Updated weights for policy 1, policy_version 341146 (0.0007) [2023-12-27 08:00:21,579][20381] Updated weights for policy 1, policy_version 341156 (0.0011) [2023-12-27 08:00:21,642][20381] Updated weights for policy 1, policy_version 341166 (0.0009) [2023-12-27 08:00:21,706][20381] Updated weights for policy 1, policy_version 341176 (0.0007) [2023-12-27 08:00:21,735][20347] Updated weights for policy 0, policy_version 341386 (0.0010) [2023-12-27 08:00:21,830][20347] Updated weights for policy 0, policy_version 341396 (0.0009) [2023-12-27 08:00:21,895][20347] Updated weights for policy 0, policy_version 341406 (0.0010) [2023-12-27 08:00:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 174776320. Throughput: 0: 10173.9, 1: 10290.7. Samples: 174753056. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:22,363][19170] Avg episode reward: [(0, '105.450'), (1, '862.880')] [2023-12-27 08:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000341176_87359488.pth... [2023-12-27 08:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000341408_87416832.pth... [2023-12-27 08:00:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000340224_87113728.pth [2023-12-27 08:00:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000339992_87056384.pth [2023-12-27 08:00:22,463][20381] Updated weights for policy 1, policy_version 341186 (0.0008) [2023-12-27 08:00:22,529][20381] Updated weights for policy 1, policy_version 341196 (0.0007) [2023-12-27 08:00:22,593][20381] Updated weights for policy 1, policy_version 341206 (0.0008) [2023-12-27 08:00:22,600][20347] Updated weights for policy 0, policy_version 341416 (0.0009) [2023-12-27 08:00:22,664][20347] Updated weights for policy 0, policy_version 341426 (0.0008) [2023-12-27 08:00:22,724][20347] Updated weights for policy 0, policy_version 341436 (0.0008) [2023-12-27 08:00:23,294][20381] Updated weights for policy 1, policy_version 341216 (0.0006) [2023-12-27 08:00:23,351][20381] Updated weights for policy 1, policy_version 341226 (0.0006) [2023-12-27 08:00:23,402][20381] Updated weights for policy 1, policy_version 341236 (0.0008) [2023-12-27 08:00:23,438][20347] Updated weights for policy 0, policy_version 341446 (0.0009) [2023-12-27 08:00:23,487][20347] Updated weights for policy 0, policy_version 341456 (0.0010) [2023-12-27 08:00:23,550][20347] Updated weights for policy 0, policy_version 341466 (0.0010) [2023-12-27 08:00:24,029][20381] Updated weights for policy 1, policy_version 341246 (0.0005) [2023-12-27 08:00:24,082][20381] Updated weights for policy 1, policy_version 341256 (0.0005) [2023-12-27 08:00:24,136][20381] Updated weights for policy 1, policy_version 341266 (0.0007) [2023-12-27 08:00:24,178][20347] Updated weights for policy 0, policy_version 341476 (0.0010) [2023-12-27 08:00:24,238][20347] Updated weights for policy 0, policy_version 341486 (0.0011) [2023-12-27 08:00:24,294][20347] Updated weights for policy 0, policy_version 341496 (0.0010) [2023-12-27 08:00:24,751][20381] Updated weights for policy 1, policy_version 341276 (0.0007) [2023-12-27 08:00:24,807][20381] Updated weights for policy 1, policy_version 341286 (0.0008) [2023-12-27 08:00:24,859][20381] Updated weights for policy 1, policy_version 341296 (0.0007) [2023-12-27 08:00:24,935][20347] Updated weights for policy 0, policy_version 341506 (0.0010) [2023-12-27 08:00:24,983][20347] Updated weights for policy 0, policy_version 341516 (0.0010) [2023-12-27 08:00:25,032][20347] Updated weights for policy 0, policy_version 341526 (0.0007) [2023-12-27 08:00:25,077][20347] Updated weights for policy 0, policy_version 341536 (0.0005) [2023-12-27 08:00:25,569][20381] Updated weights for policy 1, policy_version 341306 (0.0008) [2023-12-27 08:00:25,618][20381] Updated weights for policy 1, policy_version 341316 (0.0007) [2023-12-27 08:00:25,670][20381] Updated weights for policy 1, policy_version 341326 (0.0007) [2023-12-27 08:00:25,726][20381] Updated weights for policy 1, policy_version 341336 (0.0008) [2023-12-27 08:00:25,741][20347] Updated weights for policy 0, policy_version 341546 (0.0005) [2023-12-27 08:00:25,802][20347] Updated weights for policy 0, policy_version 341556 (0.0007) [2023-12-27 08:00:25,859][20347] Updated weights for policy 0, policy_version 341566 (0.0010) [2023-12-27 08:00:26,406][20381] Updated weights for policy 1, policy_version 341346 (0.0008) [2023-12-27 08:00:26,456][20381] Updated weights for policy 1, policy_version 341356 (0.0008) [2023-12-27 08:00:26,474][20347] Updated weights for policy 0, policy_version 341576 (0.0010) [2023-12-27 08:00:26,512][20381] Updated weights for policy 1, policy_version 341366 (0.0005) [2023-12-27 08:00:26,521][20347] Updated weights for policy 0, policy_version 341586 (0.0010) [2023-12-27 08:00:26,568][20347] Updated weights for policy 0, policy_version 341596 (0.0010) [2023-12-27 08:00:27,200][20347] Updated weights for policy 0, policy_version 341606 (0.0010) [2023-12-27 08:00:27,201][20381] Updated weights for policy 1, policy_version 341376 (0.0007) [2023-12-27 08:00:27,252][20381] Updated weights for policy 1, policy_version 341386 (0.0005) [2023-12-27 08:00:27,258][20347] Updated weights for policy 0, policy_version 341616 (0.0010) [2023-12-27 08:00:27,308][20347] Updated weights for policy 0, policy_version 341626 (0.0009) [2023-12-27 08:00:27,314][20381] Updated weights for policy 1, policy_version 341396 (0.0007) [2023-12-27 08:00:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 174891008. Throughput: 0: 10123.6, 1: 10281.5. Samples: 174873132. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:27,363][19170] Avg episode reward: [(0, '466.890'), (1, '863.930')] [2023-12-27 08:00:27,901][20381] Updated weights for policy 1, policy_version 341406 (0.0007) [2023-12-27 08:00:27,958][20381] Updated weights for policy 1, policy_version 341416 (0.0007) [2023-12-27 08:00:27,966][20347] Updated weights for policy 0, policy_version 341636 (0.0009) [2023-12-27 08:00:28,016][20381] Updated weights for policy 1, policy_version 341426 (0.0006) [2023-12-27 08:00:28,018][20347] Updated weights for policy 0, policy_version 341646 (0.0010) [2023-12-27 08:00:28,069][20347] Updated weights for policy 0, policy_version 341656 (0.0010) [2023-12-27 08:00:28,664][20381] Updated weights for policy 1, policy_version 341436 (0.0010) [2023-12-27 08:00:28,710][20347] Updated weights for policy 0, policy_version 341666 (0.0010) [2023-12-27 08:00:28,722][20381] Updated weights for policy 1, policy_version 341446 (0.0010) [2023-12-27 08:00:28,768][20347] Updated weights for policy 0, policy_version 341676 (0.0010) [2023-12-27 08:00:28,774][20381] Updated weights for policy 1, policy_version 341456 (0.0007) [2023-12-27 08:00:28,830][20347] Updated weights for policy 0, policy_version 341686 (0.0010) [2023-12-27 08:00:28,887][20347] Updated weights for policy 0, policy_version 341696 (0.0010) [2023-12-27 08:00:29,455][20381] Updated weights for policy 1, policy_version 341466 (0.0008) [2023-12-27 08:00:29,509][20381] Updated weights for policy 1, policy_version 341476 (0.0010) [2023-12-27 08:00:29,552][20347] Updated weights for policy 0, policy_version 341706 (0.0010) [2023-12-27 08:00:29,564][20381] Updated weights for policy 1, policy_version 341486 (0.0010) [2023-12-27 08:00:29,610][20347] Updated weights for policy 0, policy_version 341716 (0.0010) [2023-12-27 08:00:29,620][20381] Updated weights for policy 1, policy_version 341496 (0.0010) [2023-12-27 08:00:29,665][20347] Updated weights for policy 0, policy_version 341726 (0.0010) [2023-12-27 08:00:30,298][20381] Updated weights for policy 1, policy_version 341506 (0.0006) [2023-12-27 08:00:30,354][20347] Updated weights for policy 0, policy_version 341736 (0.0006) [2023-12-27 08:00:30,362][20381] Updated weights for policy 1, policy_version 341516 (0.0006) [2023-12-27 08:00:30,410][20347] Updated weights for policy 0, policy_version 341746 (0.0005) [2023-12-27 08:00:30,421][20381] Updated weights for policy 1, policy_version 341526 (0.0007) [2023-12-27 08:00:30,471][20347] Updated weights for policy 0, policy_version 341756 (0.0007) [2023-12-27 08:00:31,018][20381] Updated weights for policy 1, policy_version 341536 (0.0010) [2023-12-27 08:00:31,074][20347] Updated weights for policy 0, policy_version 341766 (0.0006) [2023-12-27 08:00:31,083][20381] Updated weights for policy 1, policy_version 341546 (0.0011) [2023-12-27 08:00:31,141][20347] Updated weights for policy 0, policy_version 341776 (0.0007) [2023-12-27 08:00:31,152][20381] Updated weights for policy 1, policy_version 341556 (0.0011) [2023-12-27 08:00:31,195][20347] Updated weights for policy 0, policy_version 341786 (0.0008) [2023-12-27 08:00:31,882][20347] Updated weights for policy 0, policy_version 341796 (0.0007) [2023-12-27 08:00:31,906][20381] Updated weights for policy 1, policy_version 341566 (0.0011) [2023-12-27 08:00:31,937][20347] Updated weights for policy 0, policy_version 341806 (0.0005) [2023-12-27 08:00:31,954][20381] Updated weights for policy 1, policy_version 341576 (0.0011) [2023-12-27 08:00:31,989][20347] Updated weights for policy 0, policy_version 341816 (0.0006) [2023-12-27 08:00:32,003][20381] Updated weights for policy 1, policy_version 341586 (0.0011) [2023-12-27 08:00:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 174989312. Throughput: 0: 10196.7, 1: 10242.9. Samples: 174998300. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:32,363][19170] Avg episode reward: [(0, '480.700'), (1, '864.060')] [2023-12-27 08:00:32,692][20347] Updated weights for policy 0, policy_version 341826 (0.0006) [2023-12-27 08:00:32,707][20381] Updated weights for policy 1, policy_version 341596 (0.0011) [2023-12-27 08:00:32,746][20347] Updated weights for policy 0, policy_version 341836 (0.0008) [2023-12-27 08:00:32,766][20381] Updated weights for policy 1, policy_version 341606 (0.0011) [2023-12-27 08:00:32,801][20347] Updated weights for policy 0, policy_version 341846 (0.0005) [2023-12-27 08:00:32,818][20381] Updated weights for policy 1, policy_version 341616 (0.0011) [2023-12-27 08:00:32,860][20347] Updated weights for policy 0, policy_version 341856 (0.0006) [2023-12-27 08:00:33,513][20381] Updated weights for policy 1, policy_version 341626 (0.0011) [2023-12-27 08:00:33,541][20347] Updated weights for policy 0, policy_version 341866 (0.0010) [2023-12-27 08:00:33,558][20381] Updated weights for policy 1, policy_version 341636 (0.0010) [2023-12-27 08:00:33,592][20347] Updated weights for policy 0, policy_version 341876 (0.0009) [2023-12-27 08:00:33,609][20381] Updated weights for policy 1, policy_version 341646 (0.0010) [2023-12-27 08:00:33,652][20347] Updated weights for policy 0, policy_version 341886 (0.0007) [2023-12-27 08:00:33,670][20381] Updated weights for policy 1, policy_version 341656 (0.0010) [2023-12-27 08:00:34,376][20347] Updated weights for policy 0, policy_version 341896 (0.0008) [2023-12-27 08:00:34,396][20381] Updated weights for policy 1, policy_version 341666 (0.0011) [2023-12-27 08:00:34,446][20347] Updated weights for policy 0, policy_version 341906 (0.0008) [2023-12-27 08:00:34,458][20381] Updated weights for policy 1, policy_version 341676 (0.0011) [2023-12-27 08:00:34,514][20347] Updated weights for policy 0, policy_version 341916 (0.0010) [2023-12-27 08:00:34,518][20381] Updated weights for policy 1, policy_version 341686 (0.0011) [2023-12-27 08:00:35,138][20347] Updated weights for policy 0, policy_version 341926 (0.0008) [2023-12-27 08:00:35,165][20381] Updated weights for policy 1, policy_version 341696 (0.0011) [2023-12-27 08:00:35,193][20347] Updated weights for policy 0, policy_version 341936 (0.0005) [2023-12-27 08:00:35,214][20381] Updated weights for policy 1, policy_version 341706 (0.0010) [2023-12-27 08:00:35,246][20347] Updated weights for policy 0, policy_version 341946 (0.0007) [2023-12-27 08:00:35,259][20381] Updated weights for policy 1, policy_version 341716 (0.0010) [2023-12-27 08:00:35,833][20381] Updated weights for policy 1, policy_version 341726 (0.0007) [2023-12-27 08:00:35,889][20381] Updated weights for policy 1, policy_version 341736 (0.0005) [2023-12-27 08:00:35,950][20381] Updated weights for policy 1, policy_version 341746 (0.0005) [2023-12-27 08:00:35,995][20347] Updated weights for policy 0, policy_version 341956 (0.0007) [2023-12-27 08:00:36,046][20347] Updated weights for policy 0, policy_version 341966 (0.0008) [2023-12-27 08:00:36,109][20347] Updated weights for policy 0, policy_version 341976 (0.0008) [2023-12-27 08:00:36,573][20381] Updated weights for policy 1, policy_version 341756 (0.0006) [2023-12-27 08:00:36,633][20381] Updated weights for policy 1, policy_version 341766 (0.0008) [2023-12-27 08:00:36,689][20381] Updated weights for policy 1, policy_version 341776 (0.0007) [2023-12-27 08:00:36,826][20347] Updated weights for policy 0, policy_version 341986 (0.0008) [2023-12-27 08:00:36,875][20347] Updated weights for policy 0, policy_version 341996 (0.0007) [2023-12-27 08:00:36,936][20347] Updated weights for policy 0, policy_version 342006 (0.0006) [2023-12-27 08:00:36,998][20347] Updated weights for policy 0, policy_version 342016 (0.0006) [2023-12-27 08:00:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20743.8). Total num frames: 175087616. Throughput: 0: 10201.4, 1: 10199.7. Samples: 175059400. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:37,363][19170] Avg episode reward: [(0, '711.090'), (1, '863.730')] [2023-12-27 08:00:37,367][20381] Updated weights for policy 1, policy_version 341786 (0.0007) [2023-12-27 08:00:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000342016_87572480.pth... [2023-12-27 08:00:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000340832_87269376.pth [2023-12-27 08:00:37,431][20381] Updated weights for policy 1, policy_version 341796 (0.0007) [2023-12-27 08:00:37,496][20381] Updated weights for policy 1, policy_version 341806 (0.0008) [2023-12-27 08:00:37,548][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000341816_87523328.pth... [2023-12-27 08:00:37,549][20381] Updated weights for policy 1, policy_version 341816 (0.0007) [2023-12-27 08:00:37,552][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000340600_87212032.pth [2023-12-27 08:00:37,579][20347] Updated weights for policy 0, policy_version 342026 (0.0010) [2023-12-27 08:00:37,641][20347] Updated weights for policy 0, policy_version 342036 (0.0010) [2023-12-27 08:00:37,697][20347] Updated weights for policy 0, policy_version 342046 (0.0011) [2023-12-27 08:00:38,270][20381] Updated weights for policy 1, policy_version 341826 (0.0009) [2023-12-27 08:00:38,288][20347] Updated weights for policy 0, policy_version 342056 (0.0006) [2023-12-27 08:00:38,328][20381] Updated weights for policy 1, policy_version 341836 (0.0008) [2023-12-27 08:00:38,349][20347] Updated weights for policy 0, policy_version 342066 (0.0007) [2023-12-27 08:00:38,380][20381] Updated weights for policy 1, policy_version 341846 (0.0006) [2023-12-27 08:00:38,399][20347] Updated weights for policy 0, policy_version 342076 (0.0006) [2023-12-27 08:00:38,994][20347] Updated weights for policy 0, policy_version 342086 (0.0006) [2023-12-27 08:00:39,041][20381] Updated weights for policy 1, policy_version 341856 (0.0006) [2023-12-27 08:00:39,043][20347] Updated weights for policy 0, policy_version 342096 (0.0010) [2023-12-27 08:00:39,102][20347] Updated weights for policy 0, policy_version 342106 (0.0010) [2023-12-27 08:00:39,103][20381] Updated weights for policy 1, policy_version 341866 (0.0005) [2023-12-27 08:00:39,163][20381] Updated weights for policy 1, policy_version 341876 (0.0007) [2023-12-27 08:00:39,797][20347] Updated weights for policy 0, policy_version 342116 (0.0010) [2023-12-27 08:00:39,830][20381] Updated weights for policy 1, policy_version 341886 (0.0007) [2023-12-27 08:00:39,861][20347] Updated weights for policy 0, policy_version 342126 (0.0011) [2023-12-27 08:00:39,891][20381] Updated weights for policy 1, policy_version 341896 (0.0007) [2023-12-27 08:00:39,923][20347] Updated weights for policy 0, policy_version 342136 (0.0011) [2023-12-27 08:00:39,957][20381] Updated weights for policy 1, policy_version 341906 (0.0006) [2023-12-27 08:00:40,592][20381] Updated weights for policy 1, policy_version 341916 (0.0007) [2023-12-27 08:00:40,628][20347] Updated weights for policy 0, policy_version 342146 (0.0011) [2023-12-27 08:00:40,642][20381] Updated weights for policy 1, policy_version 341926 (0.0006) [2023-12-27 08:00:40,684][20347] Updated weights for policy 0, policy_version 342156 (0.0010) [2023-12-27 08:00:40,687][20381] Updated weights for policy 1, policy_version 341936 (0.0007) [2023-12-27 08:00:40,743][20347] Updated weights for policy 0, policy_version 342166 (0.0010) [2023-12-27 08:00:40,795][20347] Updated weights for policy 0, policy_version 342176 (0.0009) [2023-12-27 08:00:41,452][20347] Updated weights for policy 0, policy_version 342186 (0.0006) [2023-12-27 08:00:41,498][20381] Updated weights for policy 1, policy_version 341946 (0.0007) [2023-12-27 08:00:41,512][20347] Updated weights for policy 0, policy_version 342196 (0.0006) [2023-12-27 08:00:41,551][20381] Updated weights for policy 1, policy_version 341956 (0.0006) [2023-12-27 08:00:41,565][20347] Updated weights for policy 0, policy_version 342206 (0.0006) [2023-12-27 08:00:41,604][20381] Updated weights for policy 1, policy_version 341966 (0.0006) [2023-12-27 08:00:41,672][20381] Updated weights for policy 1, policy_version 341976 (0.0009) [2023-12-27 08:00:42,324][20347] Updated weights for policy 0, policy_version 342216 (0.0007) [2023-12-27 08:00:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20743.8). Total num frames: 175185920. Throughput: 0: 10217.7, 1: 10192.4. Samples: 175184516. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:42,363][19170] Avg episode reward: [(0, '865.130'), (1, '892.290')] [2023-12-27 08:00:42,388][20347] Updated weights for policy 0, policy_version 342226 (0.0006) [2023-12-27 08:00:42,412][20381] Updated weights for policy 1, policy_version 341986 (0.0006) [2023-12-27 08:00:42,440][20347] Updated weights for policy 0, policy_version 342236 (0.0007) [2023-12-27 08:00:42,471][20381] Updated weights for policy 1, policy_version 341996 (0.0007) [2023-12-27 08:00:42,523][20381] Updated weights for policy 1, policy_version 342006 (0.0008) [2023-12-27 08:00:43,141][20347] Updated weights for policy 0, policy_version 342246 (0.0008) [2023-12-27 08:00:43,202][20347] Updated weights for policy 0, policy_version 342256 (0.0008) [2023-12-27 08:00:43,230][20381] Updated weights for policy 1, policy_version 342016 (0.0008) [2023-12-27 08:00:43,256][20347] Updated weights for policy 0, policy_version 342266 (0.0008) [2023-12-27 08:00:43,287][20381] Updated weights for policy 1, policy_version 342026 (0.0007) [2023-12-27 08:00:43,338][20381] Updated weights for policy 1, policy_version 342036 (0.0007) [2023-12-27 08:00:43,894][20347] Updated weights for policy 0, policy_version 342276 (0.0006) [2023-12-27 08:00:43,955][20347] Updated weights for policy 0, policy_version 342286 (0.0007) [2023-12-27 08:00:43,971][20381] Updated weights for policy 1, policy_version 342046 (0.0006) [2023-12-27 08:00:44,011][20347] Updated weights for policy 0, policy_version 342296 (0.0008) [2023-12-27 08:00:44,033][20381] Updated weights for policy 1, policy_version 342056 (0.0007) [2023-12-27 08:00:44,090][20381] Updated weights for policy 1, policy_version 342066 (0.0009) [2023-12-27 08:00:44,610][20347] Updated weights for policy 0, policy_version 342306 (0.0008) [2023-12-27 08:00:44,668][20347] Updated weights for policy 0, policy_version 342316 (0.0007) [2023-12-27 08:00:44,716][20381] Updated weights for policy 1, policy_version 342076 (0.0008) [2023-12-27 08:00:44,729][20347] Updated weights for policy 0, policy_version 342326 (0.0006) [2023-12-27 08:00:44,769][20381] Updated weights for policy 1, policy_version 342086 (0.0006) [2023-12-27 08:00:44,790][20347] Updated weights for policy 0, policy_version 342336 (0.0007) [2023-12-27 08:00:44,825][20381] Updated weights for policy 1, policy_version 342096 (0.0008) [2023-12-27 08:00:45,392][20347] Updated weights for policy 0, policy_version 342346 (0.0005) [2023-12-27 08:00:45,441][20347] Updated weights for policy 0, policy_version 342356 (0.0005) [2023-12-27 08:00:45,472][20381] Updated weights for policy 1, policy_version 342106 (0.0007) [2023-12-27 08:00:45,493][20347] Updated weights for policy 0, policy_version 342366 (0.0007) [2023-12-27 08:00:45,527][20381] Updated weights for policy 1, policy_version 342116 (0.0008) [2023-12-27 08:00:45,586][20381] Updated weights for policy 1, policy_version 342126 (0.0008) [2023-12-27 08:00:45,645][20381] Updated weights for policy 1, policy_version 342136 (0.0007) [2023-12-27 08:00:46,146][20347] Updated weights for policy 0, policy_version 342376 (0.0007) [2023-12-27 08:00:46,211][20347] Updated weights for policy 0, policy_version 342386 (0.0007) [2023-12-27 08:00:46,231][20381] Updated weights for policy 1, policy_version 342146 (0.0005) [2023-12-27 08:00:46,266][20347] Updated weights for policy 0, policy_version 342396 (0.0007) [2023-12-27 08:00:46,290][20381] Updated weights for policy 1, policy_version 342156 (0.0005) [2023-12-27 08:00:46,339][20381] Updated weights for policy 1, policy_version 342166 (0.0005) [2023-12-27 08:00:46,959][20381] Updated weights for policy 1, policy_version 342176 (0.0007) [2023-12-27 08:00:46,975][20347] Updated weights for policy 0, policy_version 342406 (0.0008) [2023-12-27 08:00:47,014][20381] Updated weights for policy 1, policy_version 342186 (0.0006) [2023-12-27 08:00:47,031][20347] Updated weights for policy 0, policy_version 342416 (0.0007) [2023-12-27 08:00:47,045][20291] KL-divergence is very high: 168.9145 [2023-12-27 08:00:47,060][20381] Updated weights for policy 1, policy_version 342196 (0.0006) [2023-12-27 08:00:47,081][20347] Updated weights for policy 0, policy_version 342426 (0.0006) [2023-12-27 08:00:47,087][20291] KL-divergence is very high: 172.7991 [2023-12-27 08:00:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 175300608. Throughput: 0: 10227.3, 1: 10181.7. Samples: 175309328. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:47,364][19170] Avg episode reward: [(0, '951.850'), (1, '941.130')] [2023-12-27 08:00:47,757][20381] Updated weights for policy 1, policy_version 342206 (0.0005) [2023-12-27 08:00:47,768][20347] Updated weights for policy 0, policy_version 342436 (0.0007) [2023-12-27 08:00:47,809][20381] Updated weights for policy 1, policy_version 342216 (0.0006) [2023-12-27 08:00:47,818][20347] Updated weights for policy 0, policy_version 342446 (0.0005) [2023-12-27 08:00:47,867][20381] Updated weights for policy 1, policy_version 342226 (0.0007) [2023-12-27 08:00:47,873][20347] Updated weights for policy 0, policy_version 342456 (0.0005) [2023-12-27 08:00:48,545][20381] Updated weights for policy 1, policy_version 342236 (0.0007) [2023-12-27 08:00:48,578][20347] Updated weights for policy 0, policy_version 342466 (0.0006) [2023-12-27 08:00:48,599][20381] Updated weights for policy 1, policy_version 342246 (0.0006) [2023-12-27 08:00:48,639][20347] Updated weights for policy 0, policy_version 342476 (0.0006) [2023-12-27 08:00:48,656][20381] Updated weights for policy 1, policy_version 342256 (0.0006) [2023-12-27 08:00:48,699][20347] Updated weights for policy 0, policy_version 342486 (0.0007) [2023-12-27 08:00:48,754][20347] Updated weights for policy 0, policy_version 342496 (0.0008) [2023-12-27 08:00:49,302][20381] Updated weights for policy 1, policy_version 342266 (0.0006) [2023-12-27 08:00:49,366][20381] Updated weights for policy 1, policy_version 342276 (0.0008) [2023-12-27 08:00:49,422][20381] Updated weights for policy 1, policy_version 342286 (0.0009) [2023-12-27 08:00:49,476][20381] Updated weights for policy 1, policy_version 342296 (0.0006) [2023-12-27 08:00:49,477][20347] Updated weights for policy 0, policy_version 342506 (0.0007) [2023-12-27 08:00:49,540][20347] Updated weights for policy 0, policy_version 342516 (0.0007) [2023-12-27 08:00:49,595][20347] Updated weights for policy 0, policy_version 342526 (0.0007) [2023-12-27 08:00:50,093][20381] Updated weights for policy 1, policy_version 342306 (0.0007) [2023-12-27 08:00:50,155][20381] Updated weights for policy 1, policy_version 342316 (0.0007) [2023-12-27 08:00:50,216][20381] Updated weights for policy 1, policy_version 342326 (0.0008) [2023-12-27 08:00:50,284][20347] Updated weights for policy 0, policy_version 342536 (0.0008) [2023-12-27 08:00:50,332][20347] Updated weights for policy 0, policy_version 342546 (0.0008) [2023-12-27 08:00:50,391][20347] Updated weights for policy 0, policy_version 342556 (0.0008) [2023-12-27 08:00:50,853][20381] Updated weights for policy 1, policy_version 342336 (0.0006) [2023-12-27 08:00:50,912][20381] Updated weights for policy 1, policy_version 342346 (0.0005) [2023-12-27 08:00:50,952][20292] KL-divergence is very high: 100.1306 [2023-12-27 08:00:50,971][20381] Updated weights for policy 1, policy_version 342356 (0.0006) [2023-12-27 08:00:51,167][20347] Updated weights for policy 0, policy_version 342566 (0.0007) [2023-12-27 08:00:51,233][20347] Updated weights for policy 0, policy_version 342576 (0.0008) [2023-12-27 08:00:51,292][20347] Updated weights for policy 0, policy_version 342586 (0.0008) [2023-12-27 08:00:51,581][20381] Updated weights for policy 1, policy_version 342366 (0.0008) [2023-12-27 08:00:51,644][20381] Updated weights for policy 1, policy_version 342376 (0.0007) [2023-12-27 08:00:51,712][20381] Updated weights for policy 1, policy_version 342386 (0.0007) [2023-12-27 08:00:51,996][20347] Updated weights for policy 0, policy_version 342596 (0.0008) [2023-12-27 08:00:52,055][20347] Updated weights for policy 0, policy_version 342606 (0.0008) [2023-12-27 08:00:52,110][20347] Updated weights for policy 0, policy_version 342616 (0.0007) [2023-12-27 08:00:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20743.8). Total num frames: 175398912. Throughput: 0: 10206.5, 1: 10210.4. Samples: 175370360. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:52,363][19170] Avg episode reward: [(0, '990.350'), (1, '912.720')] [2023-12-27 08:00:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000342624_87728128.pth... [2023-12-27 08:00:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000342392_87670784.pth... [2023-12-27 08:00:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000341408_87416832.pth [2023-12-27 08:00:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000341176_87359488.pth [2023-12-27 08:00:52,412][20381] Updated weights for policy 1, policy_version 342396 (0.0008) [2023-12-27 08:00:52,480][20381] Updated weights for policy 1, policy_version 342406 (0.0007) [2023-12-27 08:00:52,481][20292] KL-divergence is very high: 117.8576 [2023-12-27 08:00:52,531][20381] Updated weights for policy 1, policy_version 342416 (0.0008) [2023-12-27 08:00:52,780][20347] Updated weights for policy 0, policy_version 342626 (0.0008) [2023-12-27 08:00:52,836][20347] Updated weights for policy 0, policy_version 342636 (0.0008) [2023-12-27 08:00:52,891][20347] Updated weights for policy 0, policy_version 342646 (0.0008) [2023-12-27 08:00:52,946][20347] Updated weights for policy 0, policy_version 342656 (0.0007) [2023-12-27 08:00:53,222][20381] Updated weights for policy 1, policy_version 342426 (0.0009) [2023-12-27 08:00:53,271][20381] Updated weights for policy 1, policy_version 342436 (0.0008) [2023-12-27 08:00:53,317][20381] Updated weights for policy 1, policy_version 342446 (0.0008) [2023-12-27 08:00:53,366][20381] Updated weights for policy 1, policy_version 342456 (0.0006) [2023-12-27 08:00:53,669][20347] Updated weights for policy 0, policy_version 342666 (0.0009) [2023-12-27 08:00:53,717][20347] Updated weights for policy 0, policy_version 342676 (0.0007) [2023-12-27 08:00:53,770][20347] Updated weights for policy 0, policy_version 342686 (0.0008) [2023-12-27 08:00:54,033][20381] Updated weights for policy 1, policy_version 342466 (0.0011) [2023-12-27 08:00:54,089][20381] Updated weights for policy 1, policy_version 342476 (0.0010) [2023-12-27 08:00:54,145][20381] Updated weights for policy 1, policy_version 342486 (0.0010) [2023-12-27 08:00:54,444][20347] Updated weights for policy 0, policy_version 342696 (0.0006) [2023-12-27 08:00:54,498][20347] Updated weights for policy 0, policy_version 342706 (0.0006) [2023-12-27 08:00:54,549][20347] Updated weights for policy 0, policy_version 342716 (0.0007) [2023-12-27 08:00:54,827][20381] Updated weights for policy 1, policy_version 342496 (0.0010) [2023-12-27 08:00:54,875][20381] Updated weights for policy 1, policy_version 342506 (0.0010) [2023-12-27 08:00:54,932][20381] Updated weights for policy 1, policy_version 342516 (0.0010) [2023-12-27 08:00:55,174][20347] Updated weights for policy 0, policy_version 342726 (0.0008) [2023-12-27 08:00:55,224][20347] Updated weights for policy 0, policy_version 342736 (0.0009) [2023-12-27 08:00:55,273][20347] Updated weights for policy 0, policy_version 342746 (0.0009) [2023-12-27 08:00:55,533][20381] Updated weights for policy 1, policy_version 342526 (0.0010) [2023-12-27 08:00:55,586][20381] Updated weights for policy 1, policy_version 342536 (0.0010) [2023-12-27 08:00:55,634][20381] Updated weights for policy 1, policy_version 342546 (0.0010) [2023-12-27 08:00:55,976][20347] Updated weights for policy 0, policy_version 342756 (0.0008) [2023-12-27 08:00:56,024][20347] Updated weights for policy 0, policy_version 342766 (0.0007) [2023-12-27 08:00:56,084][20347] Updated weights for policy 0, policy_version 342776 (0.0006) [2023-12-27 08:00:56,256][20381] Updated weights for policy 1, policy_version 342556 (0.0010) [2023-12-27 08:00:56,307][20381] Updated weights for policy 1, policy_version 342566 (0.0010) [2023-12-27 08:00:56,357][20381] Updated weights for policy 1, policy_version 342576 (0.0010) [2023-12-27 08:00:56,752][20347] Updated weights for policy 0, policy_version 342786 (0.0007) [2023-12-27 08:00:56,815][20347] Updated weights for policy 0, policy_version 342796 (0.0008) [2023-12-27 08:00:56,881][20347] Updated weights for policy 0, policy_version 342806 (0.0007) [2023-12-27 08:00:56,933][20347] Updated weights for policy 0, policy_version 342816 (0.0008) [2023-12-27 08:00:56,982][20381] Updated weights for policy 1, policy_version 342586 (0.0009) [2023-12-27 08:00:57,040][20381] Updated weights for policy 1, policy_version 342596 (0.0007) [2023-12-27 08:00:57,060][20292] KL-divergence is very high: 113.2693 [2023-12-27 08:00:57,091][20381] Updated weights for policy 1, policy_version 342606 (0.0007) [2023-12-27 08:00:57,144][20381] Updated weights for policy 1, policy_version 342616 (0.0008) [2023-12-27 08:00:57,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20743.8). Total num frames: 175505408. Throughput: 0: 10173.6, 1: 10285.4. Samples: 175494356. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:00:57,364][19170] Avg episode reward: [(0, '990.350'), (1, '912.750')] [2023-12-27 08:00:57,640][20347] Updated weights for policy 0, policy_version 342826 (0.0008) [2023-12-27 08:00:57,689][20347] Updated weights for policy 0, policy_version 342836 (0.0008) [2023-12-27 08:00:57,702][20381] Updated weights for policy 1, policy_version 342626 (0.0005) [2023-12-27 08:00:57,744][20347] Updated weights for policy 0, policy_version 342846 (0.0007) [2023-12-27 08:00:57,746][20381] Updated weights for policy 1, policy_version 342636 (0.0005) [2023-12-27 08:00:57,797][20381] Updated weights for policy 1, policy_version 342646 (0.0007) [2023-12-27 08:00:58,490][20347] Updated weights for policy 0, policy_version 342856 (0.0009) [2023-12-27 08:00:58,509][20381] Updated weights for policy 1, policy_version 342656 (0.0011) [2023-12-27 08:00:58,552][20347] Updated weights for policy 0, policy_version 342866 (0.0008) [2023-12-27 08:00:58,570][20381] Updated weights for policy 1, policy_version 342666 (0.0008) [2023-12-27 08:00:58,612][20347] Updated weights for policy 0, policy_version 342876 (0.0008) [2023-12-27 08:00:58,632][20381] Updated weights for policy 1, policy_version 342676 (0.0013) [2023-12-27 08:00:59,414][20347] Updated weights for policy 0, policy_version 342886 (0.0008) [2023-12-27 08:00:59,435][20381] Updated weights for policy 1, policy_version 342686 (0.0007) [2023-12-27 08:00:59,463][20347] Updated weights for policy 0, policy_version 342896 (0.0008) [2023-12-27 08:00:59,501][20381] Updated weights for policy 1, policy_version 342696 (0.0006) [2023-12-27 08:00:59,522][20347] Updated weights for policy 0, policy_version 342906 (0.0007) [2023-12-27 08:00:59,568][20381] Updated weights for policy 1, policy_version 342706 (0.0007) [2023-12-27 08:01:00,190][20381] Updated weights for policy 1, policy_version 342716 (0.0008) [2023-12-27 08:01:00,244][20347] Updated weights for policy 0, policy_version 342916 (0.0008) [2023-12-27 08:01:00,249][20381] Updated weights for policy 1, policy_version 342726 (0.0010) [2023-12-27 08:01:00,299][20381] Updated weights for policy 1, policy_version 342736 (0.0010) [2023-12-27 08:01:00,300][20347] Updated weights for policy 0, policy_version 342926 (0.0006) [2023-12-27 08:01:00,351][20347] Updated weights for policy 0, policy_version 342936 (0.0008) [2023-12-27 08:01:00,968][20347] Updated weights for policy 0, policy_version 342946 (0.0007) [2023-12-27 08:01:00,991][20381] Updated weights for policy 1, policy_version 342746 (0.0011) [2023-12-27 08:01:01,026][20347] Updated weights for policy 0, policy_version 342956 (0.0006) [2023-12-27 08:01:01,050][20381] Updated weights for policy 1, policy_version 342756 (0.0011) [2023-12-27 08:01:01,080][20347] Updated weights for policy 0, policy_version 342966 (0.0006) [2023-12-27 08:01:01,101][20381] Updated weights for policy 1, policy_version 342766 (0.0011) [2023-12-27 08:01:01,139][20347] Updated weights for policy 0, policy_version 342976 (0.0006) [2023-12-27 08:01:01,159][20381] Updated weights for policy 1, policy_version 342776 (0.0010) [2023-12-27 08:01:01,791][20347] Updated weights for policy 0, policy_version 342986 (0.0007) [2023-12-27 08:01:01,827][20381] Updated weights for policy 1, policy_version 342786 (0.0007) [2023-12-27 08:01:01,839][20347] Updated weights for policy 0, policy_version 342996 (0.0006) [2023-12-27 08:01:01,873][20381] Updated weights for policy 1, policy_version 342796 (0.0006) [2023-12-27 08:01:01,890][20347] Updated weights for policy 0, policy_version 343006 (0.0006) [2023-12-27 08:01:01,931][20381] Updated weights for policy 1, policy_version 342806 (0.0006) [2023-12-27 08:01:02,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.5, 300 sec: 20716.1). Total num frames: 175603712. Throughput: 0: 10149.4, 1: 10318.7. Samples: 175615824. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:01:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.540')] [2023-12-27 08:01:02,509][20347] Updated weights for policy 0, policy_version 343016 (0.0008) [2023-12-27 08:01:02,558][20347] Updated weights for policy 0, policy_version 343026 (0.0007) [2023-12-27 08:01:02,565][20381] Updated weights for policy 1, policy_version 342816 (0.0009) [2023-12-27 08:01:02,605][20347] Updated weights for policy 0, policy_version 343036 (0.0006) [2023-12-27 08:01:02,621][20381] Updated weights for policy 1, policy_version 342826 (0.0006) [2023-12-27 08:01:02,682][20381] Updated weights for policy 1, policy_version 342836 (0.0007) [2023-12-27 08:01:03,303][20381] Updated weights for policy 1, policy_version 342846 (0.0009) [2023-12-27 08:01:03,310][20347] Updated weights for policy 0, policy_version 343046 (0.0006) [2023-12-27 08:01:03,362][20381] Updated weights for policy 1, policy_version 342856 (0.0006) [2023-12-27 08:01:03,367][20347] Updated weights for policy 0, policy_version 343056 (0.0005) [2023-12-27 08:01:03,414][20381] Updated weights for policy 1, policy_version 342866 (0.0005) [2023-12-27 08:01:03,419][20347] Updated weights for policy 0, policy_version 343066 (0.0008) [2023-12-27 08:01:04,102][20381] Updated weights for policy 1, policy_version 342876 (0.0007) [2023-12-27 08:01:04,142][20347] Updated weights for policy 0, policy_version 343076 (0.0008) [2023-12-27 08:01:04,164][20381] Updated weights for policy 1, policy_version 342886 (0.0008) [2023-12-27 08:01:04,208][20347] Updated weights for policy 0, policy_version 343086 (0.0008) [2023-12-27 08:01:04,226][20381] Updated weights for policy 1, policy_version 342896 (0.0006) [2023-12-27 08:01:04,266][20347] Updated weights for policy 0, policy_version 343096 (0.0006) [2023-12-27 08:01:04,934][20347] Updated weights for policy 0, policy_version 343106 (0.0007) [2023-12-27 08:01:04,940][20381] Updated weights for policy 1, policy_version 342906 (0.0006) [2023-12-27 08:01:04,992][20381] Updated weights for policy 1, policy_version 342916 (0.0005) [2023-12-27 08:01:04,994][20347] Updated weights for policy 0, policy_version 343116 (0.0006) [2023-12-27 08:01:05,047][20347] Updated weights for policy 0, policy_version 343126 (0.0007) [2023-12-27 08:01:05,048][20381] Updated weights for policy 1, policy_version 342926 (0.0005) [2023-12-27 08:01:05,100][20381] Updated weights for policy 1, policy_version 342936 (0.0005) [2023-12-27 08:01:05,104][20347] Updated weights for policy 0, policy_version 343136 (0.0008) [2023-12-27 08:01:05,709][20381] Updated weights for policy 1, policy_version 342946 (0.0005) [2023-12-27 08:01:05,760][20381] Updated weights for policy 1, policy_version 342956 (0.0008) [2023-12-27 08:01:05,817][20381] Updated weights for policy 1, policy_version 342966 (0.0006) [2023-12-27 08:01:05,830][20347] Updated weights for policy 0, policy_version 343146 (0.0008) [2023-12-27 08:01:05,882][20347] Updated weights for policy 0, policy_version 343156 (0.0008) [2023-12-27 08:01:05,931][20347] Updated weights for policy 0, policy_version 343166 (0.0007) [2023-12-27 08:01:06,429][20381] Updated weights for policy 1, policy_version 342976 (0.0008) [2023-12-27 08:01:06,494][20381] Updated weights for policy 1, policy_version 342986 (0.0008) [2023-12-27 08:01:06,553][20381] Updated weights for policy 1, policy_version 342996 (0.0007) [2023-12-27 08:01:06,617][20347] Updated weights for policy 0, policy_version 343176 (0.0008) [2023-12-27 08:01:06,675][20347] Updated weights for policy 0, policy_version 343186 (0.0008) [2023-12-27 08:01:06,721][20347] Updated weights for policy 0, policy_version 343196 (0.0008) [2023-12-27 08:01:07,144][20381] Updated weights for policy 1, policy_version 343006 (0.0007) [2023-12-27 08:01:07,200][20381] Updated weights for policy 1, policy_version 343016 (0.0009) [2023-12-27 08:01:07,259][20381] Updated weights for policy 1, policy_version 343026 (0.0008) [2023-12-27 08:01:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20743.8). Total num frames: 175710208. Throughput: 0: 10199.0, 1: 10357.5. Samples: 175678108. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:01:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:01:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000343032_87834624.pth... [2023-12-27 08:01:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000343200_87875584.pth... [2023-12-27 08:01:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000341816_87523328.pth [2023-12-27 08:01:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000342016_87572480.pth [2023-12-27 08:01:07,450][20347] Updated weights for policy 0, policy_version 343206 (0.0009) [2023-12-27 08:01:07,502][20347] Updated weights for policy 0, policy_version 343216 (0.0008) [2023-12-27 08:01:07,555][20347] Updated weights for policy 0, policy_version 343226 (0.0007) [2023-12-27 08:01:07,949][20381] Updated weights for policy 1, policy_version 343036 (0.0008) [2023-12-27 08:01:07,999][20381] Updated weights for policy 1, policy_version 343046 (0.0009) [2023-12-27 08:01:08,058][20381] Updated weights for policy 1, policy_version 343056 (0.0008) [2023-12-27 08:01:08,143][20347] Updated weights for policy 0, policy_version 343236 (0.0006) [2023-12-27 08:01:08,202][20347] Updated weights for policy 0, policy_version 343246 (0.0006) [2023-12-27 08:01:08,259][20347] Updated weights for policy 0, policy_version 343256 (0.0006) [2023-12-27 08:01:08,636][20381] Updated weights for policy 1, policy_version 343066 (0.0007) [2023-12-27 08:01:08,688][20381] Updated weights for policy 1, policy_version 343076 (0.0007) [2023-12-27 08:01:08,748][20381] Updated weights for policy 1, policy_version 343086 (0.0007) [2023-12-27 08:01:08,806][20381] Updated weights for policy 1, policy_version 343096 (0.0008) [2023-12-27 08:01:08,881][20347] Updated weights for policy 0, policy_version 343266 (0.0006) [2023-12-27 08:01:08,939][20347] Updated weights for policy 0, policy_version 343276 (0.0010) [2023-12-27 08:01:08,985][20347] Updated weights for policy 0, policy_version 343286 (0.0010) [2023-12-27 08:01:09,051][20347] Updated weights for policy 0, policy_version 343296 (0.0008) [2023-12-27 08:01:09,590][20381] Updated weights for policy 1, policy_version 343106 (0.0006) [2023-12-27 08:01:09,658][20381] Updated weights for policy 1, policy_version 343116 (0.0007) [2023-12-27 08:01:09,717][20381] Updated weights for policy 1, policy_version 343126 (0.0007) [2023-12-27 08:01:09,730][20347] Updated weights for policy 0, policy_version 343306 (0.0007) [2023-12-27 08:01:09,793][20347] Updated weights for policy 0, policy_version 343316 (0.0009) [2023-12-27 08:01:09,854][20347] Updated weights for policy 0, policy_version 343326 (0.0008) [2023-12-27 08:01:10,400][20381] Updated weights for policy 1, policy_version 343136 (0.0008) [2023-12-27 08:01:10,466][20381] Updated weights for policy 1, policy_version 343146 (0.0007) [2023-12-27 08:01:10,520][20347] Updated weights for policy 0, policy_version 343336 (0.0011) [2023-12-27 08:01:10,525][20381] Updated weights for policy 1, policy_version 343156 (0.0006) [2023-12-27 08:01:10,575][20347] Updated weights for policy 0, policy_version 343346 (0.0010) [2023-12-27 08:01:10,634][20347] Updated weights for policy 0, policy_version 343356 (0.0010) [2023-12-27 08:01:11,217][20381] Updated weights for policy 1, policy_version 343166 (0.0005) [2023-12-27 08:01:11,287][20381] Updated weights for policy 1, policy_version 343176 (0.0006) [2023-12-27 08:01:11,355][20381] Updated weights for policy 1, policy_version 343186 (0.0006) [2023-12-27 08:01:11,401][20347] Updated weights for policy 0, policy_version 343366 (0.0010) [2023-12-27 08:01:11,459][20347] Updated weights for policy 0, policy_version 343376 (0.0010) [2023-12-27 08:01:11,513][20347] Updated weights for policy 0, policy_version 343386 (0.0010) [2023-12-27 08:01:12,022][20381] Updated weights for policy 1, policy_version 343196 (0.0007) [2023-12-27 08:01:12,081][20381] Updated weights for policy 1, policy_version 343206 (0.0008) [2023-12-27 08:01:12,140][20381] Updated weights for policy 1, policy_version 343216 (0.0007) [2023-12-27 08:01:12,255][20347] Updated weights for policy 0, policy_version 343396 (0.0010) [2023-12-27 08:01:12,312][20347] Updated weights for policy 0, policy_version 343406 (0.0007) [2023-12-27 08:01:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 175808512. Throughput: 0: 10234.2, 1: 10426.1. Samples: 175802840. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:01:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.040')] [2023-12-27 08:01:12,372][20347] Updated weights for policy 0, policy_version 343416 (0.0007) [2023-12-27 08:01:12,881][20381] Updated weights for policy 1, policy_version 343226 (0.0007) [2023-12-27 08:01:12,937][20381] Updated weights for policy 1, policy_version 343236 (0.0009) [2023-12-27 08:01:12,999][20381] Updated weights for policy 1, policy_version 343246 (0.0008) [2023-12-27 08:01:13,049][20347] Updated weights for policy 0, policy_version 343426 (0.0008) [2023-12-27 08:01:13,062][20381] Updated weights for policy 1, policy_version 343256 (0.0007) [2023-12-27 08:01:13,108][20347] Updated weights for policy 0, policy_version 343436 (0.0007) [2023-12-27 08:01:13,168][20347] Updated weights for policy 0, policy_version 343446 (0.0008) [2023-12-27 08:01:13,224][20347] Updated weights for policy 0, policy_version 343456 (0.0007) [2023-12-27 08:01:13,751][20381] Updated weights for policy 1, policy_version 343266 (0.0007) [2023-12-27 08:01:13,793][20381] Updated weights for policy 1, policy_version 343276 (0.0008) [2023-12-27 08:01:13,856][20381] Updated weights for policy 1, policy_version 343286 (0.0007) [2023-12-27 08:01:13,901][20347] Updated weights for policy 0, policy_version 343466 (0.0007) [2023-12-27 08:01:13,960][20347] Updated weights for policy 0, policy_version 343476 (0.0007) [2023-12-27 08:01:14,014][20347] Updated weights for policy 0, policy_version 343486 (0.0007) [2023-12-27 08:01:14,531][20381] Updated weights for policy 1, policy_version 343296 (0.0006) [2023-12-27 08:01:14,589][20381] Updated weights for policy 1, policy_version 343306 (0.0005) [2023-12-27 08:01:14,650][20381] Updated weights for policy 1, policy_version 343316 (0.0006) [2023-12-27 08:01:14,768][20347] Updated weights for policy 0, policy_version 343496 (0.0008) [2023-12-27 08:01:14,822][20347] Updated weights for policy 0, policy_version 343506 (0.0007) [2023-12-27 08:01:14,879][20347] Updated weights for policy 0, policy_version 343516 (0.0008) [2023-12-27 08:01:15,296][20381] Updated weights for policy 1, policy_version 343326 (0.0010) [2023-12-27 08:01:15,362][20381] Updated weights for policy 1, policy_version 343336 (0.0007) [2023-12-27 08:01:15,412][20381] Updated weights for policy 1, policy_version 343346 (0.0005) [2023-12-27 08:01:15,564][20347] Updated weights for policy 0, policy_version 343526 (0.0007) [2023-12-27 08:01:15,610][20347] Updated weights for policy 0, policy_version 343536 (0.0010) [2023-12-27 08:01:15,655][20347] Updated weights for policy 0, policy_version 343546 (0.0009) [2023-12-27 08:01:15,998][20381] Updated weights for policy 1, policy_version 343356 (0.0006) [2023-12-27 08:01:16,047][20381] Updated weights for policy 1, policy_version 343366 (0.0007) [2023-12-27 08:01:16,096][20381] Updated weights for policy 1, policy_version 343376 (0.0007) [2023-12-27 08:01:16,361][20347] Updated weights for policy 0, policy_version 343556 (0.0007) [2023-12-27 08:01:16,408][20347] Updated weights for policy 0, policy_version 343566 (0.0006) [2023-12-27 08:01:16,470][20347] Updated weights for policy 0, policy_version 343576 (0.0008) [2023-12-27 08:01:16,733][20381] Updated weights for policy 1, policy_version 343386 (0.0007) [2023-12-27 08:01:16,785][20381] Updated weights for policy 1, policy_version 343396 (0.0005) [2023-12-27 08:01:16,833][20381] Updated weights for policy 1, policy_version 343406 (0.0005) [2023-12-27 08:01:16,884][20381] Updated weights for policy 1, policy_version 343416 (0.0005) [2023-12-27 08:01:17,193][20347] Updated weights for policy 0, policy_version 343586 (0.0007) [2023-12-27 08:01:17,248][20347] Updated weights for policy 0, policy_version 343596 (0.0007) [2023-12-27 08:01:17,302][20347] Updated weights for policy 0, policy_version 343606 (0.0007) [2023-12-27 08:01:17,348][20347] Updated weights for policy 0, policy_version 343616 (0.0006) [2023-12-27 08:01:17,362][19170] Fps is (10 sec: 20481.1, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 175915008. Throughput: 0: 10135.6, 1: 10440.1. Samples: 175924204. Policy #0 lag: (min: 14.0, avg: 18.4, max: 46.0) [2023-12-27 08:01:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.960')] [2023-12-27 08:01:17,485][20381] Updated weights for policy 1, policy_version 343426 (0.0008) [2023-12-27 08:01:17,539][20381] Updated weights for policy 1, policy_version 343436 (0.0007) [2023-12-27 08:01:17,595][20381] Updated weights for policy 1, policy_version 343446 (0.0008) [2023-12-27 08:01:18,055][20347] Updated weights for policy 0, policy_version 343626 (0.0007) [2023-12-27 08:01:18,100][20347] Updated weights for policy 0, policy_version 343636 (0.0007) [2023-12-27 08:01:18,152][20347] Updated weights for policy 0, policy_version 343646 (0.0007) [2023-12-27 08:01:18,303][20381] Updated weights for policy 1, policy_version 343456 (0.0009) [2023-12-27 08:01:18,364][20381] Updated weights for policy 1, policy_version 343466 (0.0009) [2023-12-27 08:01:18,416][20381] Updated weights for policy 1, policy_version 343476 (0.0008) [2023-12-27 08:01:18,819][20347] Updated weights for policy 0, policy_version 343656 (0.0007) [2023-12-27 08:01:18,871][20347] Updated weights for policy 0, policy_version 343666 (0.0007) [2023-12-27 08:01:18,928][20347] Updated weights for policy 0, policy_version 343676 (0.0007) [2023-12-27 08:01:19,126][20381] Updated weights for policy 1, policy_version 343486 (0.0007) [2023-12-27 08:01:19,189][20381] Updated weights for policy 1, policy_version 343496 (0.0006) [2023-12-27 08:01:19,258][20381] Updated weights for policy 1, policy_version 343506 (0.0009) [2023-12-27 08:01:19,561][20347] Updated weights for policy 0, policy_version 343686 (0.0007) [2023-12-27 08:01:19,617][20347] Updated weights for policy 0, policy_version 343696 (0.0008) [2023-12-27 08:01:19,671][20347] Updated weights for policy 0, policy_version 343706 (0.0007) [2023-12-27 08:01:19,963][20381] Updated weights for policy 1, policy_version 343516 (0.0007) [2023-12-27 08:01:20,018][20381] Updated weights for policy 1, policy_version 343526 (0.0008) [2023-12-27 08:01:20,069][20381] Updated weights for policy 1, policy_version 343536 (0.0008) [2023-12-27 08:01:20,434][20347] Updated weights for policy 0, policy_version 343716 (0.0008) [2023-12-27 08:01:20,486][20347] Updated weights for policy 0, policy_version 343726 (0.0008) [2023-12-27 08:01:20,535][20347] Updated weights for policy 0, policy_version 343736 (0.0006) [2023-12-27 08:01:20,796][20381] Updated weights for policy 1, policy_version 343546 (0.0008) [2023-12-27 08:01:20,864][20381] Updated weights for policy 1, policy_version 343556 (0.0008) [2023-12-27 08:01:20,925][20381] Updated weights for policy 1, policy_version 343566 (0.0008) [2023-12-27 08:01:20,984][20381] Updated weights for policy 1, policy_version 343576 (0.0008) [2023-12-27 08:01:21,220][20347] Updated weights for policy 0, policy_version 343746 (0.0007) [2023-12-27 08:01:21,288][20347] Updated weights for policy 0, policy_version 343756 (0.0008) [2023-12-27 08:01:21,356][20347] Updated weights for policy 0, policy_version 343766 (0.0007) [2023-12-27 08:01:21,414][20347] Updated weights for policy 0, policy_version 343776 (0.0008) [2023-12-27 08:01:21,701][20381] Updated weights for policy 1, policy_version 343586 (0.0006) [2023-12-27 08:01:21,768][20381] Updated weights for policy 1, policy_version 343596 (0.0007) [2023-12-27 08:01:21,824][20381] Updated weights for policy 1, policy_version 343606 (0.0005) [2023-12-27 08:01:22,192][20347] Updated weights for policy 0, policy_version 343786 (0.0008) [2023-12-27 08:01:22,249][20347] Updated weights for policy 0, policy_version 343796 (0.0009) [2023-12-27 08:01:22,310][20347] Updated weights for policy 0, policy_version 343806 (0.0008) [2023-12-27 08:01:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 176013312. Throughput: 0: 10136.2, 1: 10413.5. Samples: 175984136. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.130')] [2023-12-27 08:01:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000343808_88031232.pth... [2023-12-27 08:01:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000342624_87728128.pth [2023-12-27 08:01:22,390][20381] Updated weights for policy 1, policy_version 343616 (0.0007) [2023-12-27 08:01:22,443][20381] Updated weights for policy 1, policy_version 343626 (0.0005) [2023-12-27 08:01:22,501][20381] Updated weights for policy 1, policy_version 343636 (0.0006) [2023-12-27 08:01:22,520][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000343640_87990272.pth... [2023-12-27 08:01:22,523][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000342392_87670784.pth [2023-12-27 08:01:23,055][20347] Updated weights for policy 0, policy_version 343816 (0.0008) [2023-12-27 08:01:23,115][20347] Updated weights for policy 0, policy_version 343826 (0.0007) [2023-12-27 08:01:23,154][20381] Updated weights for policy 1, policy_version 343646 (0.0006) [2023-12-27 08:01:23,172][20347] Updated weights for policy 0, policy_version 343836 (0.0011) [2023-12-27 08:01:23,216][20381] Updated weights for policy 1, policy_version 343656 (0.0006) [2023-12-27 08:01:23,272][20381] Updated weights for policy 1, policy_version 343666 (0.0007) [2023-12-27 08:01:23,761][20347] Updated weights for policy 0, policy_version 343846 (0.0007) [2023-12-27 08:01:23,812][20347] Updated weights for policy 0, policy_version 343856 (0.0006) [2023-12-27 08:01:23,858][20347] Updated weights for policy 0, policy_version 343866 (0.0005) [2023-12-27 08:01:23,999][20381] Updated weights for policy 1, policy_version 343676 (0.0008) [2023-12-27 08:01:24,066][20381] Updated weights for policy 1, policy_version 343686 (0.0007) [2023-12-27 08:01:24,126][20381] Updated weights for policy 1, policy_version 343696 (0.0009) [2023-12-27 08:01:24,407][20347] Updated weights for policy 0, policy_version 343876 (0.0007) [2023-12-27 08:01:24,453][20347] Updated weights for policy 0, policy_version 343886 (0.0010) [2023-12-27 08:01:24,507][20347] Updated weights for policy 0, policy_version 343896 (0.0007) [2023-12-27 08:01:24,790][20381] Updated weights for policy 1, policy_version 343706 (0.0008) [2023-12-27 08:01:24,848][20381] Updated weights for policy 1, policy_version 343716 (0.0007) [2023-12-27 08:01:24,911][20381] Updated weights for policy 1, policy_version 343726 (0.0005) [2023-12-27 08:01:24,962][20381] Updated weights for policy 1, policy_version 343736 (0.0006) [2023-12-27 08:01:25,135][20347] Updated weights for policy 0, policy_version 343906 (0.0009) [2023-12-27 08:01:25,190][20347] Updated weights for policy 0, policy_version 343916 (0.0009) [2023-12-27 08:01:25,243][20347] Updated weights for policy 0, policy_version 343926 (0.0010) [2023-12-27 08:01:25,292][20347] Updated weights for policy 0, policy_version 343936 (0.0010) [2023-12-27 08:01:25,524][20381] Updated weights for policy 1, policy_version 343746 (0.0005) [2023-12-27 08:01:25,582][20381] Updated weights for policy 1, policy_version 343756 (0.0006) [2023-12-27 08:01:25,641][20381] Updated weights for policy 1, policy_version 343766 (0.0006) [2023-12-27 08:01:25,936][20347] Updated weights for policy 0, policy_version 343946 (0.0009) [2023-12-27 08:01:25,988][20347] Updated weights for policy 0, policy_version 343956 (0.0010) [2023-12-27 08:01:26,040][20347] Updated weights for policy 0, policy_version 343966 (0.0010) [2023-12-27 08:01:26,303][20381] Updated weights for policy 1, policy_version 343776 (0.0007) [2023-12-27 08:01:26,350][20381] Updated weights for policy 1, policy_version 343786 (0.0007) [2023-12-27 08:01:26,397][20381] Updated weights for policy 1, policy_version 343796 (0.0007) [2023-12-27 08:01:26,697][20347] Updated weights for policy 0, policy_version 343976 (0.0006) [2023-12-27 08:01:26,755][20347] Updated weights for policy 0, policy_version 343986 (0.0005) [2023-12-27 08:01:26,809][20347] Updated weights for policy 0, policy_version 343996 (0.0005) [2023-12-27 08:01:27,140][20381] Updated weights for policy 1, policy_version 343806 (0.0007) [2023-12-27 08:01:27,193][20381] Updated weights for policy 1, policy_version 343816 (0.0007) [2023-12-27 08:01:27,240][20381] Updated weights for policy 1, policy_version 343826 (0.0005) [2023-12-27 08:01:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 176119808. Throughput: 0: 10126.8, 1: 10424.8. Samples: 176109340. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.380')] [2023-12-27 08:01:27,400][20347] Updated weights for policy 0, policy_version 344006 (0.0008) [2023-12-27 08:01:27,444][20347] Updated weights for policy 0, policy_version 344016 (0.0010) [2023-12-27 08:01:27,495][20347] Updated weights for policy 0, policy_version 344026 (0.0010) [2023-12-27 08:01:27,861][20381] Updated weights for policy 1, policy_version 343836 (0.0008) [2023-12-27 08:01:27,914][20381] Updated weights for policy 1, policy_version 343846 (0.0007) [2023-12-27 08:01:27,961][20381] Updated weights for policy 1, policy_version 343856 (0.0007) [2023-12-27 08:01:28,193][20347] Updated weights for policy 0, policy_version 344036 (0.0010) [2023-12-27 08:01:28,238][20347] Updated weights for policy 0, policy_version 344046 (0.0010) [2023-12-27 08:01:28,283][20347] Updated weights for policy 0, policy_version 344056 (0.0010) [2023-12-27 08:01:28,555][20381] Updated weights for policy 1, policy_version 343866 (0.0007) [2023-12-27 08:01:28,606][20381] Updated weights for policy 1, policy_version 343876 (0.0007) [2023-12-27 08:01:28,653][20381] Updated weights for policy 1, policy_version 343886 (0.0007) [2023-12-27 08:01:28,708][20381] Updated weights for policy 1, policy_version 343896 (0.0006) [2023-12-27 08:01:28,968][20347] Updated weights for policy 0, policy_version 344066 (0.0010) [2023-12-27 08:01:29,028][20347] Updated weights for policy 0, policy_version 344076 (0.0006) [2023-12-27 08:01:29,083][20347] Updated weights for policy 0, policy_version 344086 (0.0006) [2023-12-27 08:01:29,138][20347] Updated weights for policy 0, policy_version 344096 (0.0005) [2023-12-27 08:01:29,396][20381] Updated weights for policy 1, policy_version 343906 (0.0006) [2023-12-27 08:01:29,442][20381] Updated weights for policy 1, policy_version 343916 (0.0005) [2023-12-27 08:01:29,494][20381] Updated weights for policy 1, policy_version 343926 (0.0006) [2023-12-27 08:01:29,796][20347] Updated weights for policy 0, policy_version 344106 (0.0010) [2023-12-27 08:01:29,853][20347] Updated weights for policy 0, policy_version 344116 (0.0010) [2023-12-27 08:01:29,907][20347] Updated weights for policy 0, policy_version 344126 (0.0010) [2023-12-27 08:01:30,177][20381] Updated weights for policy 1, policy_version 343936 (0.0006) [2023-12-27 08:01:30,234][20381] Updated weights for policy 1, policy_version 343946 (0.0006) [2023-12-27 08:01:30,290][20381] Updated weights for policy 1, policy_version 343956 (0.0007) [2023-12-27 08:01:30,634][20347] Updated weights for policy 0, policy_version 344136 (0.0008) [2023-12-27 08:01:30,682][20347] Updated weights for policy 0, policy_version 344146 (0.0007) [2023-12-27 08:01:30,742][20347] Updated weights for policy 0, policy_version 344156 (0.0009) [2023-12-27 08:01:30,975][20381] Updated weights for policy 1, policy_version 343966 (0.0006) [2023-12-27 08:01:31,029][20381] Updated weights for policy 1, policy_version 343976 (0.0007) [2023-12-27 08:01:31,086][20381] Updated weights for policy 1, policy_version 343986 (0.0008) [2023-12-27 08:01:31,406][20347] Updated weights for policy 0, policy_version 344166 (0.0009) [2023-12-27 08:01:31,461][20347] Updated weights for policy 0, policy_version 344176 (0.0010) [2023-12-27 08:01:31,518][20347] Updated weights for policy 0, policy_version 344186 (0.0009) [2023-12-27 08:01:31,790][20381] Updated weights for policy 1, policy_version 343996 (0.0008) [2023-12-27 08:01:31,839][20381] Updated weights for policy 1, policy_version 344006 (0.0009) [2023-12-27 08:01:31,895][20381] Updated weights for policy 1, policy_version 344016 (0.0009) [2023-12-27 08:01:32,124][20347] Updated weights for policy 0, policy_version 344196 (0.0008) [2023-12-27 08:01:32,172][20347] Updated weights for policy 0, policy_version 344206 (0.0010) [2023-12-27 08:01:32,226][20347] Updated weights for policy 0, policy_version 344216 (0.0010) [2023-12-27 08:01:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 176226304. Throughput: 0: 10159.3, 1: 10397.5. Samples: 176234380. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.890')] [2023-12-27 08:01:32,616][20381] Updated weights for policy 1, policy_version 344026 (0.0008) [2023-12-27 08:01:32,664][20381] Updated weights for policy 1, policy_version 344036 (0.0008) [2023-12-27 08:01:32,711][20381] Updated weights for policy 1, policy_version 344046 (0.0007) [2023-12-27 08:01:32,766][20381] Updated weights for policy 1, policy_version 344056 (0.0007) [2023-12-27 08:01:32,920][20347] Updated weights for policy 0, policy_version 344226 (0.0010) [2023-12-27 08:01:32,974][20347] Updated weights for policy 0, policy_version 344236 (0.0007) [2023-12-27 08:01:33,017][20347] Updated weights for policy 0, policy_version 344246 (0.0005) [2023-12-27 08:01:33,070][20347] Updated weights for policy 0, policy_version 344256 (0.0005) [2023-12-27 08:01:33,481][20381] Updated weights for policy 1, policy_version 344066 (0.0006) [2023-12-27 08:01:33,533][20381] Updated weights for policy 1, policy_version 344076 (0.0005) [2023-12-27 08:01:33,590][20381] Updated weights for policy 1, policy_version 344086 (0.0005) [2023-12-27 08:01:33,663][20347] Updated weights for policy 0, policy_version 344266 (0.0010) [2023-12-27 08:01:33,714][20347] Updated weights for policy 0, policy_version 344276 (0.0010) [2023-12-27 08:01:33,765][20347] Updated weights for policy 0, policy_version 344286 (0.0008) [2023-12-27 08:01:34,246][20381] Updated weights for policy 1, policy_version 344096 (0.0007) [2023-12-27 08:01:34,293][20381] Updated weights for policy 1, policy_version 344106 (0.0010) [2023-12-27 08:01:34,340][20381] Updated weights for policy 1, policy_version 344116 (0.0010) [2023-12-27 08:01:34,350][20347] Updated weights for policy 0, policy_version 344296 (0.0009) [2023-12-27 08:01:34,413][20347] Updated weights for policy 0, policy_version 344306 (0.0011) [2023-12-27 08:01:34,469][20347] Updated weights for policy 0, policy_version 344316 (0.0010) [2023-12-27 08:01:35,060][20381] Updated weights for policy 1, policy_version 344126 (0.0011) [2023-12-27 08:01:35,115][20381] Updated weights for policy 1, policy_version 344136 (0.0011) [2023-12-27 08:01:35,135][20347] Updated weights for policy 0, policy_version 344326 (0.0010) [2023-12-27 08:01:35,167][20381] Updated weights for policy 1, policy_version 344146 (0.0011) [2023-12-27 08:01:35,190][20347] Updated weights for policy 0, policy_version 344336 (0.0010) [2023-12-27 08:01:35,239][20347] Updated weights for policy 0, policy_version 344346 (0.0010) [2023-12-27 08:01:35,783][20381] Updated weights for policy 1, policy_version 344156 (0.0011) [2023-12-27 08:01:35,843][20381] Updated weights for policy 1, policy_version 344166 (0.0011) [2023-12-27 08:01:35,903][20381] Updated weights for policy 1, policy_version 344176 (0.0011) [2023-12-27 08:01:35,926][20347] Updated weights for policy 0, policy_version 344356 (0.0010) [2023-12-27 08:01:35,975][20347] Updated weights for policy 0, policy_version 344366 (0.0010) [2023-12-27 08:01:36,024][20347] Updated weights for policy 0, policy_version 344376 (0.0010) [2023-12-27 08:01:36,600][20381] Updated weights for policy 1, policy_version 344186 (0.0010) [2023-12-27 08:01:36,663][20381] Updated weights for policy 1, policy_version 344196 (0.0011) [2023-12-27 08:01:36,680][20347] Updated weights for policy 0, policy_version 344386 (0.0010) [2023-12-27 08:01:36,722][20381] Updated weights for policy 1, policy_version 344206 (0.0010) [2023-12-27 08:01:36,732][20347] Updated weights for policy 0, policy_version 344396 (0.0010) [2023-12-27 08:01:36,778][20381] Updated weights for policy 1, policy_version 344216 (0.0011) [2023-12-27 08:01:36,781][20347] Updated weights for policy 0, policy_version 344406 (0.0010) [2023-12-27 08:01:36,834][20347] Updated weights for policy 0, policy_version 344416 (0.0010) [2023-12-27 08:01:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 176324608. Throughput: 0: 10221.8, 1: 10375.5. Samples: 176297236. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '942.340')] [2023-12-27 08:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000344416_88186880.pth... [2023-12-27 08:01:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000343200_87875584.pth [2023-12-27 08:01:37,399][20381] Updated weights for policy 1, policy_version 344226 (0.0007) [2023-12-27 08:01:37,456][20381] Updated weights for policy 1, policy_version 344236 (0.0011) [2023-12-27 08:01:37,481][20347] Updated weights for policy 0, policy_version 344426 (0.0010) [2023-12-27 08:01:37,507][20381] Updated weights for policy 1, policy_version 344246 (0.0011) [2023-12-27 08:01:37,514][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000344248_88145920.pth... [2023-12-27 08:01:37,517][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000343032_87834624.pth [2023-12-27 08:01:37,537][20347] Updated weights for policy 0, policy_version 344436 (0.0010) [2023-12-27 08:01:37,595][20347] Updated weights for policy 0, policy_version 344446 (0.0010) [2023-12-27 08:01:38,058][20381] Updated weights for policy 1, policy_version 344256 (0.0006) [2023-12-27 08:01:38,116][20381] Updated weights for policy 1, policy_version 344266 (0.0011) [2023-12-27 08:01:38,170][20381] Updated weights for policy 1, policy_version 344276 (0.0010) [2023-12-27 08:01:38,281][20347] Updated weights for policy 0, policy_version 344456 (0.0007) [2023-12-27 08:01:38,338][20347] Updated weights for policy 0, policy_version 344466 (0.0007) [2023-12-27 08:01:38,393][20347] Updated weights for policy 0, policy_version 344476 (0.0010) [2023-12-27 08:01:38,835][20381] Updated weights for policy 1, policy_version 344286 (0.0008) [2023-12-27 08:01:38,876][20292] KL-divergence is very high: 263.5797 [2023-12-27 08:01:38,893][20381] Updated weights for policy 1, policy_version 344296 (0.0011) [2023-12-27 08:01:38,942][20381] Updated weights for policy 1, policy_version 344306 (0.0010) [2023-12-27 08:01:39,078][20347] Updated weights for policy 0, policy_version 344486 (0.0009) [2023-12-27 08:01:39,130][20347] Updated weights for policy 0, policy_version 344496 (0.0007) [2023-12-27 08:01:39,190][20347] Updated weights for policy 0, policy_version 344506 (0.0005) [2023-12-27 08:01:39,639][20381] Updated weights for policy 1, policy_version 344316 (0.0009) [2023-12-27 08:01:39,703][20381] Updated weights for policy 1, policy_version 344326 (0.0008) [2023-12-27 08:01:39,762][20381] Updated weights for policy 1, policy_version 344336 (0.0011) [2023-12-27 08:01:39,843][20347] Updated weights for policy 0, policy_version 344516 (0.0008) [2023-12-27 08:01:39,899][20347] Updated weights for policy 0, policy_version 344526 (0.0007) [2023-12-27 08:01:39,959][20347] Updated weights for policy 0, policy_version 344536 (0.0008) [2023-12-27 08:01:40,463][20381] Updated weights for policy 1, policy_version 344346 (0.0009) [2023-12-27 08:01:40,520][20381] Updated weights for policy 1, policy_version 344356 (0.0007) [2023-12-27 08:01:40,590][20381] Updated weights for policy 1, policy_version 344366 (0.0006) [2023-12-27 08:01:40,633][20381] Updated weights for policy 1, policy_version 344376 (0.0008) [2023-12-27 08:01:40,652][20347] Updated weights for policy 0, policy_version 344546 (0.0008) [2023-12-27 08:01:40,708][20347] Updated weights for policy 0, policy_version 344556 (0.0010) [2023-12-27 08:01:40,766][20347] Updated weights for policy 0, policy_version 344566 (0.0010) [2023-12-27 08:01:40,827][20347] Updated weights for policy 0, policy_version 344576 (0.0010) [2023-12-27 08:01:41,288][20381] Updated weights for policy 1, policy_version 344386 (0.0008) [2023-12-27 08:01:41,340][20381] Updated weights for policy 1, policy_version 344396 (0.0007) [2023-12-27 08:01:41,409][20381] Updated weights for policy 1, policy_version 344406 (0.0007) [2023-12-27 08:01:41,529][20347] Updated weights for policy 0, policy_version 344586 (0.0009) [2023-12-27 08:01:41,578][20347] Updated weights for policy 0, policy_version 344596 (0.0010) [2023-12-27 08:01:41,636][20347] Updated weights for policy 0, policy_version 344606 (0.0011) [2023-12-27 08:01:42,108][20381] Updated weights for policy 1, policy_version 344416 (0.0006) [2023-12-27 08:01:42,173][20381] Updated weights for policy 1, policy_version 344426 (0.0006) [2023-12-27 08:01:42,232][20381] Updated weights for policy 1, policy_version 344436 (0.0007) [2023-12-27 08:01:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 176431104. Throughput: 0: 10251.2, 1: 10383.9. Samples: 176422928. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.490')] [2023-12-27 08:01:42,375][20347] Updated weights for policy 0, policy_version 344616 (0.0011) [2023-12-27 08:01:42,437][20347] Updated weights for policy 0, policy_version 344626 (0.0010) [2023-12-27 08:01:42,503][20347] Updated weights for policy 0, policy_version 344636 (0.0011) [2023-12-27 08:01:42,896][20381] Updated weights for policy 1, policy_version 344446 (0.0008) [2023-12-27 08:01:42,952][20381] Updated weights for policy 1, policy_version 344456 (0.0007) [2023-12-27 08:01:43,004][20381] Updated weights for policy 1, policy_version 344466 (0.0008) [2023-12-27 08:01:43,127][20347] Updated weights for policy 0, policy_version 344646 (0.0010) [2023-12-27 08:01:43,185][20347] Updated weights for policy 0, policy_version 344656 (0.0010) [2023-12-27 08:01:43,239][20347] Updated weights for policy 0, policy_version 344666 (0.0010) [2023-12-27 08:01:43,654][20381] Updated weights for policy 1, policy_version 344476 (0.0008) [2023-12-27 08:01:43,713][20381] Updated weights for policy 1, policy_version 344486 (0.0011) [2023-12-27 08:01:43,779][20381] Updated weights for policy 1, policy_version 344496 (0.0010) [2023-12-27 08:01:43,919][20347] Updated weights for policy 0, policy_version 344676 (0.0010) [2023-12-27 08:01:43,964][20347] Updated weights for policy 0, policy_version 344686 (0.0010) [2023-12-27 08:01:44,018][20347] Updated weights for policy 0, policy_version 344696 (0.0010) [2023-12-27 08:01:44,478][20381] Updated weights for policy 1, policy_version 344506 (0.0011) [2023-12-27 08:01:44,537][20381] Updated weights for policy 1, policy_version 344516 (0.0010) [2023-12-27 08:01:44,592][20381] Updated weights for policy 1, policy_version 344526 (0.0010) [2023-12-27 08:01:44,646][20381] Updated weights for policy 1, policy_version 344536 (0.0010) [2023-12-27 08:01:44,734][20347] Updated weights for policy 0, policy_version 344706 (0.0010) [2023-12-27 08:01:44,798][20347] Updated weights for policy 0, policy_version 344716 (0.0011) [2023-12-27 08:01:44,863][20347] Updated weights for policy 0, policy_version 344726 (0.0011) [2023-12-27 08:01:44,921][20347] Updated weights for policy 0, policy_version 344736 (0.0010) [2023-12-27 08:01:45,342][20381] Updated weights for policy 1, policy_version 344546 (0.0008) [2023-12-27 08:01:45,400][20381] Updated weights for policy 1, policy_version 344556 (0.0007) [2023-12-27 08:01:45,457][20381] Updated weights for policy 1, policy_version 344566 (0.0007) [2023-12-27 08:01:45,605][20347] Updated weights for policy 0, policy_version 344746 (0.0010) [2023-12-27 08:01:45,662][20347] Updated weights for policy 0, policy_version 344756 (0.0010) [2023-12-27 08:01:45,724][20347] Updated weights for policy 0, policy_version 344766 (0.0011) [2023-12-27 08:01:46,125][20381] Updated weights for policy 1, policy_version 344576 (0.0010) [2023-12-27 08:01:46,174][20381] Updated weights for policy 1, policy_version 344586 (0.0010) [2023-12-27 08:01:46,234][20381] Updated weights for policy 1, policy_version 344596 (0.0011) [2023-12-27 08:01:46,367][20347] Updated weights for policy 0, policy_version 344776 (0.0007) [2023-12-27 08:01:46,418][20347] Updated weights for policy 0, policy_version 344786 (0.0008) [2023-12-27 08:01:46,473][20347] Updated weights for policy 0, policy_version 344796 (0.0008) [2023-12-27 08:01:46,818][20381] Updated weights for policy 1, policy_version 344606 (0.0008) [2023-12-27 08:01:46,867][20381] Updated weights for policy 1, policy_version 344616 (0.0005) [2023-12-27 08:01:46,918][20381] Updated weights for policy 1, policy_version 344626 (0.0006) [2023-12-27 08:01:47,142][20347] Updated weights for policy 0, policy_version 344806 (0.0007) [2023-12-27 08:01:47,197][20347] Updated weights for policy 0, policy_version 344816 (0.0008) [2023-12-27 08:01:47,259][20347] Updated weights for policy 0, policy_version 344826 (0.0006) [2023-12-27 08:01:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 176537600. Throughput: 0: 10287.0, 1: 10376.1. Samples: 176545668. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.370')] [2023-12-27 08:01:47,505][20381] Updated weights for policy 1, policy_version 344636 (0.0007) [2023-12-27 08:01:47,565][20381] Updated weights for policy 1, policy_version 344646 (0.0005) [2023-12-27 08:01:47,616][20381] Updated weights for policy 1, policy_version 344656 (0.0005) [2023-12-27 08:01:47,978][20347] Updated weights for policy 0, policy_version 344836 (0.0008) [2023-12-27 08:01:48,038][20347] Updated weights for policy 0, policy_version 344846 (0.0008) [2023-12-27 08:01:48,093][20347] Updated weights for policy 0, policy_version 344856 (0.0008) [2023-12-27 08:01:48,222][20381] Updated weights for policy 1, policy_version 344666 (0.0006) [2023-12-27 08:01:48,275][20381] Updated weights for policy 1, policy_version 344676 (0.0007) [2023-12-27 08:01:48,331][20381] Updated weights for policy 1, policy_version 344686 (0.0008) [2023-12-27 08:01:48,392][20381] Updated weights for policy 1, policy_version 344696 (0.0008) [2023-12-27 08:01:48,808][20347] Updated weights for policy 0, policy_version 344866 (0.0008) [2023-12-27 08:01:48,868][20347] Updated weights for policy 0, policy_version 344876 (0.0008) [2023-12-27 08:01:48,923][20347] Updated weights for policy 0, policy_version 344886 (0.0008) [2023-12-27 08:01:48,958][20291] KL-divergence is very high: 142.2792 [2023-12-27 08:01:48,976][20291] KL-divergence is very high: 111.4258 [2023-12-27 08:01:48,981][20347] Updated weights for policy 0, policy_version 344896 (0.0008) [2023-12-27 08:01:49,036][20381] Updated weights for policy 1, policy_version 344706 (0.0008) [2023-12-27 08:01:49,088][20381] Updated weights for policy 1, policy_version 344716 (0.0008) [2023-12-27 08:01:49,139][20381] Updated weights for policy 1, policy_version 344726 (0.0007) [2023-12-27 08:01:49,593][20347] Updated weights for policy 0, policy_version 344906 (0.0006) [2023-12-27 08:01:49,627][20291] KL-divergence is very high: 268.2253 [2023-12-27 08:01:49,641][20347] Updated weights for policy 0, policy_version 344916 (0.0005) [2023-12-27 08:01:49,664][20291] KL-divergence is very high: 268.5595 [2023-12-27 08:01:49,687][20347] Updated weights for policy 0, policy_version 344926 (0.0005) [2023-12-27 08:01:49,796][20381] Updated weights for policy 1, policy_version 344736 (0.0006) [2023-12-27 08:01:49,857][20381] Updated weights for policy 1, policy_version 344746 (0.0006) [2023-12-27 08:01:49,908][20381] Updated weights for policy 1, policy_version 344756 (0.0008) [2023-12-27 08:01:50,306][20347] Updated weights for policy 0, policy_version 344936 (0.0007) [2023-12-27 08:01:50,361][20347] Updated weights for policy 0, policy_version 344946 (0.0008) [2023-12-27 08:01:50,424][20347] Updated weights for policy 0, policy_version 344956 (0.0008) [2023-12-27 08:01:50,612][20381] Updated weights for policy 1, policy_version 344766 (0.0009) [2023-12-27 08:01:50,671][20381] Updated weights for policy 1, policy_version 344776 (0.0008) [2023-12-27 08:01:50,720][20381] Updated weights for policy 1, policy_version 344786 (0.0007) [2023-12-27 08:01:51,116][20347] Updated weights for policy 0, policy_version 344966 (0.0008) [2023-12-27 08:01:51,179][20347] Updated weights for policy 0, policy_version 344976 (0.0008) [2023-12-27 08:01:51,231][20347] Updated weights for policy 0, policy_version 344986 (0.0008) [2023-12-27 08:01:51,441][20381] Updated weights for policy 1, policy_version 344796 (0.0007) [2023-12-27 08:01:51,500][20381] Updated weights for policy 1, policy_version 344806 (0.0005) [2023-12-27 08:01:51,560][20381] Updated weights for policy 1, policy_version 344816 (0.0008) [2023-12-27 08:01:51,971][20347] Updated weights for policy 0, policy_version 344996 (0.0008) [2023-12-27 08:01:52,022][20347] Updated weights for policy 0, policy_version 345006 (0.0007) [2023-12-27 08:01:52,075][20347] Updated weights for policy 0, policy_version 345016 (0.0007) [2023-12-27 08:01:52,190][20381] Updated weights for policy 1, policy_version 344826 (0.0008) [2023-12-27 08:01:52,243][20381] Updated weights for policy 1, policy_version 344836 (0.0006) [2023-12-27 08:01:52,295][20381] Updated weights for policy 1, policy_version 344846 (0.0008) [2023-12-27 08:01:52,354][20381] Updated weights for policy 1, policy_version 344856 (0.0007) [2023-12-27 08:01:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 176644096. Throughput: 0: 10281.5, 1: 10403.3. Samples: 176608916. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:52,363][19170] Avg episode reward: [(0, '980.660'), (1, '1000.000')] [2023-12-27 08:01:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000344856_88301568.pth... [2023-12-27 08:01:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000345024_88342528.pth... [2023-12-27 08:01:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000343640_87990272.pth [2023-12-27 08:01:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000343808_88031232.pth [2023-12-27 08:01:52,737][20347] Updated weights for policy 0, policy_version 345026 (0.0008) [2023-12-27 08:01:52,795][20347] Updated weights for policy 0, policy_version 345036 (0.0007) [2023-12-27 08:01:52,853][20347] Updated weights for policy 0, policy_version 345046 (0.0008) [2023-12-27 08:01:52,905][20347] Updated weights for policy 0, policy_version 345056 (0.0007) [2023-12-27 08:01:53,070][20381] Updated weights for policy 1, policy_version 344866 (0.0008) [2023-12-27 08:01:53,136][20381] Updated weights for policy 1, policy_version 344876 (0.0007) [2023-12-27 08:01:53,196][20381] Updated weights for policy 1, policy_version 344886 (0.0008) [2023-12-27 08:01:53,539][20347] Updated weights for policy 0, policy_version 345066 (0.0007) [2023-12-27 08:01:53,584][20347] Updated weights for policy 0, policy_version 345076 (0.0008) [2023-12-27 08:01:53,631][20347] Updated weights for policy 0, policy_version 345086 (0.0005) [2023-12-27 08:01:53,889][20381] Updated weights for policy 1, policy_version 344896 (0.0008) [2023-12-27 08:01:53,940][20381] Updated weights for policy 1, policy_version 344906 (0.0008) [2023-12-27 08:01:53,995][20381] Updated weights for policy 1, policy_version 344916 (0.0007) [2023-12-27 08:01:54,285][20347] Updated weights for policy 0, policy_version 345096 (0.0008) [2023-12-27 08:01:54,330][20347] Updated weights for policy 0, policy_version 345106 (0.0007) [2023-12-27 08:01:54,391][20347] Updated weights for policy 0, policy_version 345116 (0.0005) [2023-12-27 08:01:54,725][20381] Updated weights for policy 1, policy_version 344926 (0.0008) [2023-12-27 08:01:54,780][20381] Updated weights for policy 1, policy_version 344936 (0.0006) [2023-12-27 08:01:54,831][20381] Updated weights for policy 1, policy_version 344946 (0.0005) [2023-12-27 08:01:55,078][20347] Updated weights for policy 0, policy_version 345126 (0.0007) [2023-12-27 08:01:55,132][20347] Updated weights for policy 0, policy_version 345136 (0.0007) [2023-12-27 08:01:55,186][20347] Updated weights for policy 0, policy_version 345146 (0.0008) [2023-12-27 08:01:55,452][20381] Updated weights for policy 1, policy_version 344956 (0.0007) [2023-12-27 08:01:55,511][20381] Updated weights for policy 1, policy_version 344966 (0.0008) [2023-12-27 08:01:55,567][20381] Updated weights for policy 1, policy_version 344976 (0.0008) [2023-12-27 08:01:55,897][20347] Updated weights for policy 0, policy_version 345156 (0.0008) [2023-12-27 08:01:55,957][20347] Updated weights for policy 0, policy_version 345166 (0.0008) [2023-12-27 08:01:56,012][20347] Updated weights for policy 0, policy_version 345176 (0.0008) [2023-12-27 08:01:56,266][20381] Updated weights for policy 1, policy_version 344986 (0.0007) [2023-12-27 08:01:56,324][20381] Updated weights for policy 1, policy_version 344996 (0.0008) [2023-12-27 08:01:56,382][20381] Updated weights for policy 1, policy_version 345006 (0.0007) [2023-12-27 08:01:56,439][20381] Updated weights for policy 1, policy_version 345016 (0.0007) [2023-12-27 08:01:56,688][20347] Updated weights for policy 0, policy_version 345186 (0.0008) [2023-12-27 08:01:56,738][20347] Updated weights for policy 0, policy_version 345196 (0.0007) [2023-12-27 08:01:56,794][20347] Updated weights for policy 0, policy_version 345206 (0.0007) [2023-12-27 08:01:56,852][20347] Updated weights for policy 0, policy_version 345216 (0.0007) [2023-12-27 08:01:57,061][20381] Updated weights for policy 1, policy_version 345026 (0.0009) [2023-12-27 08:01:57,114][20381] Updated weights for policy 1, policy_version 345036 (0.0008) [2023-12-27 08:01:57,179][20381] Updated weights for policy 1, policy_version 345046 (0.0005) [2023-12-27 08:01:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 176742400. Throughput: 0: 10266.8, 1: 10350.8. Samples: 176730628. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:01:57,363][19170] Avg episode reward: [(0, '980.660'), (1, '990.340')] [2023-12-27 08:01:57,490][20347] Updated weights for policy 0, policy_version 345226 (0.0008) [2023-12-27 08:01:57,538][20347] Updated weights for policy 0, policy_version 345236 (0.0008) [2023-12-27 08:01:57,586][20347] Updated weights for policy 0, policy_version 345246 (0.0006) [2023-12-27 08:01:57,743][20381] Updated weights for policy 1, policy_version 345056 (0.0005) [2023-12-27 08:01:57,787][20381] Updated weights for policy 1, policy_version 345066 (0.0006) [2023-12-27 08:01:57,837][20381] Updated weights for policy 1, policy_version 345076 (0.0007) [2023-12-27 08:01:58,261][20347] Updated weights for policy 0, policy_version 345256 (0.0007) [2023-12-27 08:01:58,316][20347] Updated weights for policy 0, policy_version 345266 (0.0007) [2023-12-27 08:01:58,376][20347] Updated weights for policy 0, policy_version 345276 (0.0007) [2023-12-27 08:01:58,530][20381] Updated weights for policy 1, policy_version 345086 (0.0009) [2023-12-27 08:01:58,587][20381] Updated weights for policy 1, policy_version 345096 (0.0008) [2023-12-27 08:01:58,647][20381] Updated weights for policy 1, policy_version 345106 (0.0008) [2023-12-27 08:01:59,143][20347] Updated weights for policy 0, policy_version 345286 (0.0008) [2023-12-27 08:01:59,202][20347] Updated weights for policy 0, policy_version 345296 (0.0009) [2023-12-27 08:01:59,264][20347] Updated weights for policy 0, policy_version 345306 (0.0008) [2023-12-27 08:01:59,387][20381] Updated weights for policy 1, policy_version 345116 (0.0007) [2023-12-27 08:01:59,435][20381] Updated weights for policy 1, policy_version 345126 (0.0007) [2023-12-27 08:01:59,483][20381] Updated weights for policy 1, policy_version 345136 (0.0009) [2023-12-27 08:01:59,933][20347] Updated weights for policy 0, policy_version 345316 (0.0007) [2023-12-27 08:01:59,998][20347] Updated weights for policy 0, policy_version 345326 (0.0006) [2023-12-27 08:02:00,057][20347] Updated weights for policy 0, policy_version 345336 (0.0007) [2023-12-27 08:02:00,228][20381] Updated weights for policy 1, policy_version 345146 (0.0008) [2023-12-27 08:02:00,279][20381] Updated weights for policy 1, policy_version 345156 (0.0011) [2023-12-27 08:02:00,330][20381] Updated weights for policy 1, policy_version 345166 (0.0008) [2023-12-27 08:02:00,378][20381] Updated weights for policy 1, policy_version 345176 (0.0008) [2023-12-27 08:02:00,690][20347] Updated weights for policy 0, policy_version 345346 (0.0007) [2023-12-27 08:02:00,750][20347] Updated weights for policy 0, policy_version 345356 (0.0005) [2023-12-27 08:02:00,809][20347] Updated weights for policy 0, policy_version 345366 (0.0005) [2023-12-27 08:02:00,862][20347] Updated weights for policy 0, policy_version 345376 (0.0005) [2023-12-27 08:02:00,984][20381] Updated weights for policy 1, policy_version 345186 (0.0005) [2023-12-27 08:02:01,043][20381] Updated weights for policy 1, policy_version 345196 (0.0007) [2023-12-27 08:02:01,099][20381] Updated weights for policy 1, policy_version 345206 (0.0006) [2023-12-27 08:02:01,457][20347] Updated weights for policy 0, policy_version 345386 (0.0008) [2023-12-27 08:02:01,511][20347] Updated weights for policy 0, policy_version 345396 (0.0008) [2023-12-27 08:02:01,568][20347] Updated weights for policy 0, policy_version 345406 (0.0007) [2023-12-27 08:02:01,689][20381] Updated weights for policy 1, policy_version 345216 (0.0009) [2023-12-27 08:02:01,753][20381] Updated weights for policy 1, policy_version 345226 (0.0011) [2023-12-27 08:02:01,815][20381] Updated weights for policy 1, policy_version 345236 (0.0007) [2023-12-27 08:02:02,321][20347] Updated weights for policy 0, policy_version 345416 (0.0008) [2023-12-27 08:02:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 176840704. Throughput: 0: 10337.5, 1: 10372.5. Samples: 176856156. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.930')] [2023-12-27 08:02:02,383][20347] Updated weights for policy 0, policy_version 345426 (0.0008) [2023-12-27 08:02:02,442][20347] Updated weights for policy 0, policy_version 345436 (0.0008) [2023-12-27 08:02:02,456][20381] Updated weights for policy 1, policy_version 345246 (0.0008) [2023-12-27 08:02:02,508][20381] Updated weights for policy 1, policy_version 345256 (0.0007) [2023-12-27 08:02:02,552][20381] Updated weights for policy 1, policy_version 345266 (0.0006) [2023-12-27 08:02:03,103][20381] Updated weights for policy 1, policy_version 345276 (0.0005) [2023-12-27 08:02:03,155][20381] Updated weights for policy 1, policy_version 345286 (0.0006) [2023-12-27 08:02:03,191][20347] Updated weights for policy 0, policy_version 345446 (0.0007) [2023-12-27 08:02:03,207][20381] Updated weights for policy 1, policy_version 345296 (0.0010) [2023-12-27 08:02:03,248][20347] Updated weights for policy 0, policy_version 345456 (0.0005) [2023-12-27 08:02:03,302][20347] Updated weights for policy 0, policy_version 345466 (0.0007) [2023-12-27 08:02:03,902][20381] Updated weights for policy 1, policy_version 345306 (0.0010) [2023-12-27 08:02:03,964][20381] Updated weights for policy 1, policy_version 345316 (0.0010) [2023-12-27 08:02:03,986][20347] Updated weights for policy 0, policy_version 345476 (0.0007) [2023-12-27 08:02:04,023][20381] Updated weights for policy 1, policy_version 345326 (0.0010) [2023-12-27 08:02:04,039][20347] Updated weights for policy 0, policy_version 345486 (0.0005) [2023-12-27 08:02:04,088][20381] Updated weights for policy 1, policy_version 345336 (0.0010) [2023-12-27 08:02:04,094][20347] Updated weights for policy 0, policy_version 345496 (0.0005) [2023-12-27 08:02:04,745][20347] Updated weights for policy 0, policy_version 345506 (0.0005) [2023-12-27 08:02:04,765][20381] Updated weights for policy 1, policy_version 345346 (0.0008) [2023-12-27 08:02:04,797][20347] Updated weights for policy 0, policy_version 345516 (0.0006) [2023-12-27 08:02:04,824][20381] Updated weights for policy 1, policy_version 345356 (0.0005) [2023-12-27 08:02:04,842][20347] Updated weights for policy 0, policy_version 345526 (0.0008) [2023-12-27 08:02:04,885][20381] Updated weights for policy 1, policy_version 345366 (0.0006) [2023-12-27 08:02:04,895][20347] Updated weights for policy 0, policy_version 345536 (0.0006) [2023-12-27 08:02:05,447][20381] Updated weights for policy 1, policy_version 345376 (0.0005) [2023-12-27 08:02:05,500][20381] Updated weights for policy 1, policy_version 345386 (0.0006) [2023-12-27 08:02:05,501][20347] Updated weights for policy 0, policy_version 345546 (0.0008) [2023-12-27 08:02:05,551][20381] Updated weights for policy 1, policy_version 345396 (0.0005) [2023-12-27 08:02:05,556][20347] Updated weights for policy 0, policy_version 345556 (0.0007) [2023-12-27 08:02:05,614][20347] Updated weights for policy 0, policy_version 345566 (0.0007) [2023-12-27 08:02:06,170][20381] Updated weights for policy 1, policy_version 345406 (0.0007) [2023-12-27 08:02:06,237][20381] Updated weights for policy 1, policy_version 345416 (0.0006) [2023-12-27 08:02:06,303][20381] Updated weights for policy 1, policy_version 345426 (0.0009) [2023-12-27 08:02:06,315][20347] Updated weights for policy 0, policy_version 345576 (0.0008) [2023-12-27 08:02:06,376][20347] Updated weights for policy 0, policy_version 345586 (0.0006) [2023-12-27 08:02:06,438][20347] Updated weights for policy 0, policy_version 345596 (0.0006) [2023-12-27 08:02:06,964][20381] Updated weights for policy 1, policy_version 345436 (0.0009) [2023-12-27 08:02:07,013][20381] Updated weights for policy 1, policy_version 345446 (0.0011) [2023-12-27 08:02:07,062][20381] Updated weights for policy 1, policy_version 345456 (0.0011) [2023-12-27 08:02:07,115][20347] Updated weights for policy 0, policy_version 345606 (0.0007) [2023-12-27 08:02:07,176][20347] Updated weights for policy 0, policy_version 345616 (0.0008) [2023-12-27 08:02:07,224][20347] Updated weights for policy 0, policy_version 345626 (0.0007) [2023-12-27 08:02:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20743.8). Total num frames: 176955392. Throughput: 0: 10315.6, 1: 10444.1. Samples: 176918324. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.920')] [2023-12-27 08:02:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000345464_88457216.pth... [2023-12-27 08:02:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000345632_88498176.pth... [2023-12-27 08:02:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000344248_88145920.pth [2023-12-27 08:02:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000344416_88186880.pth [2023-12-27 08:02:07,652][20381] Updated weights for policy 1, policy_version 345466 (0.0009) [2023-12-27 08:02:07,722][20381] Updated weights for policy 1, policy_version 345476 (0.0006) [2023-12-27 08:02:07,770][20381] Updated weights for policy 1, policy_version 345486 (0.0010) [2023-12-27 08:02:07,821][20381] Updated weights for policy 1, policy_version 345496 (0.0010) [2023-12-27 08:02:07,968][20347] Updated weights for policy 0, policy_version 345636 (0.0007) [2023-12-27 08:02:08,015][20347] Updated weights for policy 0, policy_version 345646 (0.0007) [2023-12-27 08:02:08,077][20347] Updated weights for policy 0, policy_version 345656 (0.0008) [2023-12-27 08:02:08,481][20381] Updated weights for policy 1, policy_version 345506 (0.0008) [2023-12-27 08:02:08,524][20381] Updated weights for policy 1, policy_version 345516 (0.0008) [2023-12-27 08:02:08,577][20381] Updated weights for policy 1, policy_version 345526 (0.0008) [2023-12-27 08:02:08,719][20347] Updated weights for policy 0, policy_version 345666 (0.0007) [2023-12-27 08:02:08,769][20347] Updated weights for policy 0, policy_version 345676 (0.0008) [2023-12-27 08:02:08,821][20347] Updated weights for policy 0, policy_version 345686 (0.0008) [2023-12-27 08:02:08,880][20347] Updated weights for policy 0, policy_version 345696 (0.0007) [2023-12-27 08:02:09,281][20381] Updated weights for policy 1, policy_version 345536 (0.0009) [2023-12-27 08:02:09,333][20381] Updated weights for policy 1, policy_version 345546 (0.0007) [2023-12-27 08:02:09,401][20381] Updated weights for policy 1, policy_version 345556 (0.0007) [2023-12-27 08:02:09,621][20347] Updated weights for policy 0, policy_version 345706 (0.0009) [2023-12-27 08:02:09,671][20347] Updated weights for policy 0, policy_version 345716 (0.0008) [2023-12-27 08:02:09,720][20347] Updated weights for policy 0, policy_version 345726 (0.0007) [2023-12-27 08:02:10,114][20381] Updated weights for policy 1, policy_version 345566 (0.0009) [2023-12-27 08:02:10,165][20381] Updated weights for policy 1, policy_version 345576 (0.0011) [2023-12-27 08:02:10,215][20381] Updated weights for policy 1, policy_version 345586 (0.0010) [2023-12-27 08:02:10,446][20347] Updated weights for policy 0, policy_version 345736 (0.0008) [2023-12-27 08:02:10,497][20347] Updated weights for policy 0, policy_version 345746 (0.0008) [2023-12-27 08:02:10,554][20347] Updated weights for policy 0, policy_version 345756 (0.0006) [2023-12-27 08:02:10,935][20381] Updated weights for policy 1, policy_version 345596 (0.0010) [2023-12-27 08:02:10,997][20381] Updated weights for policy 1, policy_version 345606 (0.0010) [2023-12-27 08:02:11,065][20381] Updated weights for policy 1, policy_version 345617 (0.0009) [2023-12-27 08:02:11,250][20347] Updated weights for policy 0, policy_version 345766 (0.0007) [2023-12-27 08:02:11,313][20347] Updated weights for policy 0, policy_version 345776 (0.0007) [2023-12-27 08:02:11,381][20347] Updated weights for policy 0, policy_version 345786 (0.0008) [2023-12-27 08:02:11,863][20381] Updated weights for policy 1, policy_version 345627 (0.0009) [2023-12-27 08:02:11,929][20381] Updated weights for policy 1, policy_version 345637 (0.0008) [2023-12-27 08:02:11,983][20381] Updated weights for policy 1, policy_version 345647 (0.0007) [2023-12-27 08:02:12,069][20347] Updated weights for policy 0, policy_version 345796 (0.0007) [2023-12-27 08:02:12,131][20347] Updated weights for policy 0, policy_version 345806 (0.0007) [2023-12-27 08:02:12,183][20347] Updated weights for policy 0, policy_version 345816 (0.0007) [2023-12-27 08:02:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 177053696. Throughput: 0: 10278.3, 1: 10429.6. Samples: 177041196. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.330')] [2023-12-27 08:02:12,616][20381] Updated weights for policy 1, policy_version 345657 (0.0007) [2023-12-27 08:02:12,677][20381] Updated weights for policy 1, policy_version 345667 (0.0006) [2023-12-27 08:02:12,739][20381] Updated weights for policy 1, policy_version 345677 (0.0006) [2023-12-27 08:02:12,796][20381] Updated weights for policy 1, policy_version 345687 (0.0006) [2023-12-27 08:02:12,909][20347] Updated weights for policy 0, policy_version 345826 (0.0007) [2023-12-27 08:02:12,967][20347] Updated weights for policy 0, policy_version 345836 (0.0007) [2023-12-27 08:02:13,024][20347] Updated weights for policy 0, policy_version 345846 (0.0008) [2023-12-27 08:02:13,080][20347] Updated weights for policy 0, policy_version 345856 (0.0007) [2023-12-27 08:02:13,433][20381] Updated weights for policy 1, policy_version 345697 (0.0008) [2023-12-27 08:02:13,477][20381] Updated weights for policy 1, policy_version 345707 (0.0007) [2023-12-27 08:02:13,528][20381] Updated weights for policy 1, policy_version 345717 (0.0007) [2023-12-27 08:02:13,750][20347] Updated weights for policy 0, policy_version 345866 (0.0005) [2023-12-27 08:02:13,801][20347] Updated weights for policy 0, policy_version 345876 (0.0005) [2023-12-27 08:02:13,849][20347] Updated weights for policy 0, policy_version 345886 (0.0005) [2023-12-27 08:02:14,204][20381] Updated weights for policy 1, policy_version 345727 (0.0008) [2023-12-27 08:02:14,262][20381] Updated weights for policy 1, policy_version 345738 (0.0008) [2023-12-27 08:02:14,318][20381] Updated weights for policy 1, policy_version 345748 (0.0008) [2023-12-27 08:02:14,420][20347] Updated weights for policy 0, policy_version 345896 (0.0009) [2023-12-27 08:02:14,490][20347] Updated weights for policy 0, policy_version 345906 (0.0010) [2023-12-27 08:02:14,550][20347] Updated weights for policy 0, policy_version 345916 (0.0009) [2023-12-27 08:02:15,017][20381] Updated weights for policy 1, policy_version 345758 (0.0008) [2023-12-27 08:02:15,075][20381] Updated weights for policy 1, policy_version 345768 (0.0007) [2023-12-27 08:02:15,134][20381] Updated weights for policy 1, policy_version 345778 (0.0007) [2023-12-27 08:02:15,229][20347] Updated weights for policy 0, policy_version 345926 (0.0008) [2023-12-27 08:02:15,256][20291] KL-divergence is very high: 140.4675 [2023-12-27 08:02:15,285][20347] Updated weights for policy 0, policy_version 345936 (0.0008) [2023-12-27 08:02:15,334][20347] Updated weights for policy 0, policy_version 345946 (0.0009) [2023-12-27 08:02:15,795][20381] Updated weights for policy 1, policy_version 345788 (0.0008) [2023-12-27 08:02:15,844][20381] Updated weights for policy 1, policy_version 345798 (0.0008) [2023-12-27 08:02:15,892][20381] Updated weights for policy 1, policy_version 345808 (0.0007) [2023-12-27 08:02:16,054][20347] Updated weights for policy 0, policy_version 345956 (0.0007) [2023-12-27 08:02:16,111][20347] Updated weights for policy 0, policy_version 345966 (0.0007) [2023-12-27 08:02:16,170][20347] Updated weights for policy 0, policy_version 345976 (0.0007) [2023-12-27 08:02:16,576][20381] Updated weights for policy 1, policy_version 345818 (0.0008) [2023-12-27 08:02:16,634][20381] Updated weights for policy 1, policy_version 345828 (0.0007) [2023-12-27 08:02:16,689][20381] Updated weights for policy 1, policy_version 345838 (0.0007) [2023-12-27 08:02:16,748][20381] Updated weights for policy 1, policy_version 345848 (0.0007) [2023-12-27 08:02:16,824][20347] Updated weights for policy 0, policy_version 345986 (0.0007) [2023-12-27 08:02:16,884][20347] Updated weights for policy 0, policy_version 345996 (0.0008) [2023-12-27 08:02:16,942][20347] Updated weights for policy 0, policy_version 346006 (0.0008) [2023-12-27 08:02:17,001][20347] Updated weights for policy 0, policy_version 346016 (0.0007) [2023-12-27 08:02:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 177152000. Throughput: 0: 10227.4, 1: 10441.8. Samples: 177164492. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:17,363][19170] Avg episode reward: [(0, '998.890'), (1, '1000.000')] [2023-12-27 08:02:17,367][20381] Updated weights for policy 1, policy_version 345858 (0.0007) [2023-12-27 08:02:17,427][20381] Updated weights for policy 1, policy_version 345868 (0.0007) [2023-12-27 08:02:17,472][20381] Updated weights for policy 1, policy_version 345878 (0.0010) [2023-12-27 08:02:17,683][20347] Updated weights for policy 0, policy_version 346026 (0.0007) [2023-12-27 08:02:17,737][20347] Updated weights for policy 0, policy_version 346036 (0.0008) [2023-12-27 08:02:17,788][20347] Updated weights for policy 0, policy_version 346046 (0.0007) [2023-12-27 08:02:18,121][20381] Updated weights for policy 1, policy_version 345888 (0.0008) [2023-12-27 08:02:18,170][20381] Updated weights for policy 1, policy_version 345898 (0.0007) [2023-12-27 08:02:18,221][20381] Updated weights for policy 1, policy_version 345908 (0.0007) [2023-12-27 08:02:18,463][20347] Updated weights for policy 0, policy_version 346056 (0.0007) [2023-12-27 08:02:18,512][20347] Updated weights for policy 0, policy_version 346066 (0.0007) [2023-12-27 08:02:18,574][20347] Updated weights for policy 0, policy_version 346076 (0.0007) [2023-12-27 08:02:18,947][20381] Updated weights for policy 1, policy_version 345918 (0.0008) [2023-12-27 08:02:19,002][20381] Updated weights for policy 1, policy_version 345928 (0.0008) [2023-12-27 08:02:19,060][20381] Updated weights for policy 1, policy_version 345938 (0.0008) [2023-12-27 08:02:19,282][20347] Updated weights for policy 0, policy_version 346086 (0.0008) [2023-12-27 08:02:19,347][20347] Updated weights for policy 0, policy_version 346096 (0.0008) [2023-12-27 08:02:19,408][20347] Updated weights for policy 0, policy_version 346106 (0.0007) [2023-12-27 08:02:19,771][20381] Updated weights for policy 1, policy_version 345948 (0.0008) [2023-12-27 08:02:19,826][20381] Updated weights for policy 1, policy_version 345958 (0.0007) [2023-12-27 08:02:19,886][20381] Updated weights for policy 1, policy_version 345968 (0.0008) [2023-12-27 08:02:20,101][20347] Updated weights for policy 0, policy_version 346116 (0.0007) [2023-12-27 08:02:20,160][20347] Updated weights for policy 0, policy_version 346126 (0.0007) [2023-12-27 08:02:20,212][20347] Updated weights for policy 0, policy_version 346136 (0.0007) [2023-12-27 08:02:20,630][20381] Updated weights for policy 1, policy_version 345978 (0.0008) [2023-12-27 08:02:20,691][20381] Updated weights for policy 1, policy_version 345988 (0.0008) [2023-12-27 08:02:20,755][20381] Updated weights for policy 1, policy_version 345998 (0.0007) [2023-12-27 08:02:20,819][20381] Updated weights for policy 1, policy_version 346008 (0.0008) [2023-12-27 08:02:20,859][20347] Updated weights for policy 0, policy_version 346146 (0.0007) [2023-12-27 08:02:20,910][20347] Updated weights for policy 0, policy_version 346156 (0.0007) [2023-12-27 08:02:20,969][20347] Updated weights for policy 0, policy_version 346166 (0.0006) [2023-12-27 08:02:21,032][20347] Updated weights for policy 0, policy_version 346176 (0.0006) [2023-12-27 08:02:21,463][20381] Updated weights for policy 1, policy_version 346018 (0.0007) [2023-12-27 08:02:21,480][20292] KL-divergence is very high: 162.2641 [2023-12-27 08:02:21,524][20381] Updated weights for policy 1, policy_version 346028 (0.0006) [2023-12-27 08:02:21,585][20381] Updated weights for policy 1, policy_version 346038 (0.0006) [2023-12-27 08:02:21,759][20347] Updated weights for policy 0, policy_version 346187 (0.0008) [2023-12-27 08:02:21,818][20347] Updated weights for policy 0, policy_version 346197 (0.0007) [2023-12-27 08:02:21,874][20347] Updated weights for policy 0, policy_version 346207 (0.0007) [2023-12-27 08:02:22,298][20381] Updated weights for policy 1, policy_version 346048 (0.0009) [2023-12-27 08:02:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 177250304. Throughput: 0: 10183.5, 1: 10434.7. Samples: 177225052. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:22,363][19170] Avg episode reward: [(0, '998.730'), (1, '990.430')] [2023-12-27 08:02:22,367][20381] Updated weights for policy 1, policy_version 346058 (0.0007) [2023-12-27 08:02:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000346208_88645632.pth... [2023-12-27 08:02:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000345024_88342528.pth [2023-12-27 08:02:22,434][20381] Updated weights for policy 1, policy_version 346068 (0.0008) [2023-12-27 08:02:22,453][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000346072_88612864.pth... [2023-12-27 08:02:22,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000344856_88301568.pth [2023-12-27 08:02:22,511][20347] Updated weights for policy 0, policy_version 346217 (0.0008) [2023-12-27 08:02:22,560][20347] Updated weights for policy 0, policy_version 346227 (0.0007) [2023-12-27 08:02:22,609][20347] Updated weights for policy 0, policy_version 346237 (0.0007) [2023-12-27 08:02:23,121][20381] Updated weights for policy 1, policy_version 346078 (0.0010) [2023-12-27 08:02:23,175][20381] Updated weights for policy 1, policy_version 346088 (0.0010) [2023-12-27 08:02:23,230][20381] Updated weights for policy 1, policy_version 346098 (0.0010) [2023-12-27 08:02:23,331][20347] Updated weights for policy 0, policy_version 346247 (0.0008) [2023-12-27 08:02:23,392][20347] Updated weights for policy 0, policy_version 346257 (0.0007) [2023-12-27 08:02:23,443][20347] Updated weights for policy 0, policy_version 346267 (0.0007) [2023-12-27 08:02:23,922][20381] Updated weights for policy 1, policy_version 346108 (0.0008) [2023-12-27 08:02:23,971][20381] Updated weights for policy 1, policy_version 346118 (0.0005) [2023-12-27 08:02:24,024][20381] Updated weights for policy 1, policy_version 346128 (0.0005) [2023-12-27 08:02:24,132][20347] Updated weights for policy 0, policy_version 346277 (0.0007) [2023-12-27 08:02:24,177][20347] Updated weights for policy 0, policy_version 346287 (0.0009) [2023-12-27 08:02:24,226][20347] Updated weights for policy 0, policy_version 346297 (0.0009) [2023-12-27 08:02:24,587][20381] Updated weights for policy 1, policy_version 346138 (0.0006) [2023-12-27 08:02:24,645][20381] Updated weights for policy 1, policy_version 346148 (0.0008) [2023-12-27 08:02:24,697][20381] Updated weights for policy 1, policy_version 346158 (0.0007) [2023-12-27 08:02:24,739][20381] Updated weights for policy 1, policy_version 346168 (0.0008) [2023-12-27 08:02:24,973][20347] Updated weights for policy 0, policy_version 346307 (0.0008) [2023-12-27 08:02:25,032][20347] Updated weights for policy 0, policy_version 346317 (0.0007) [2023-12-27 08:02:25,093][20347] Updated weights for policy 0, policy_version 346327 (0.0005) [2023-12-27 08:02:25,428][20381] Updated weights for policy 1, policy_version 346178 (0.0007) [2023-12-27 08:02:25,486][20381] Updated weights for policy 1, policy_version 346188 (0.0007) [2023-12-27 08:02:25,547][20381] Updated weights for policy 1, policy_version 346198 (0.0008) [2023-12-27 08:02:25,692][20347] Updated weights for policy 0, policy_version 346337 (0.0006) [2023-12-27 08:02:25,750][20347] Updated weights for policy 0, policy_version 346347 (0.0008) [2023-12-27 08:02:25,802][20347] Updated weights for policy 0, policy_version 346357 (0.0008) [2023-12-27 08:02:25,860][20347] Updated weights for policy 0, policy_version 346367 (0.0007) [2023-12-27 08:02:26,215][20381] Updated weights for policy 1, policy_version 346208 (0.0006) [2023-12-27 08:02:26,275][20381] Updated weights for policy 1, policy_version 346218 (0.0009) [2023-12-27 08:02:26,338][20381] Updated weights for policy 1, policy_version 346228 (0.0006) [2023-12-27 08:02:26,579][20347] Updated weights for policy 0, policy_version 346377 (0.0008) [2023-12-27 08:02:26,624][20347] Updated weights for policy 0, policy_version 346387 (0.0008) [2023-12-27 08:02:26,669][20347] Updated weights for policy 0, policy_version 346397 (0.0009) [2023-12-27 08:02:26,896][20381] Updated weights for policy 1, policy_version 346238 (0.0008) [2023-12-27 08:02:26,944][20381] Updated weights for policy 1, policy_version 346248 (0.0010) [2023-12-27 08:02:26,995][20381] Updated weights for policy 1, policy_version 346258 (0.0010) [2023-12-27 08:02:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 177356800. Throughput: 0: 10165.9, 1: 10383.4. Samples: 177347644. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:27,363][19170] Avg episode reward: [(0, '999.690'), (1, '981.000')] [2023-12-27 08:02:27,432][20347] Updated weights for policy 0, policy_version 346407 (0.0007) [2023-12-27 08:02:27,485][20347] Updated weights for policy 0, policy_version 346417 (0.0007) [2023-12-27 08:02:27,542][20347] Updated weights for policy 0, policy_version 346427 (0.0008) [2023-12-27 08:02:27,593][20381] Updated weights for policy 1, policy_version 346268 (0.0010) [2023-12-27 08:02:27,644][20381] Updated weights for policy 1, policy_version 346278 (0.0010) [2023-12-27 08:02:27,698][20381] Updated weights for policy 1, policy_version 346288 (0.0010) [2023-12-27 08:02:28,100][20347] Updated weights for policy 0, policy_version 346437 (0.0007) [2023-12-27 08:02:28,156][20347] Updated weights for policy 0, policy_version 346447 (0.0007) [2023-12-27 08:02:28,211][20347] Updated weights for policy 0, policy_version 346457 (0.0007) [2023-12-27 08:02:28,400][20381] Updated weights for policy 1, policy_version 346298 (0.0010) [2023-12-27 08:02:28,460][20381] Updated weights for policy 1, policy_version 346308 (0.0008) [2023-12-27 08:02:28,516][20381] Updated weights for policy 1, policy_version 346318 (0.0011) [2023-12-27 08:02:28,575][20381] Updated weights for policy 1, policy_version 346328 (0.0011) [2023-12-27 08:02:28,819][20347] Updated weights for policy 0, policy_version 346467 (0.0006) [2023-12-27 08:02:28,875][20347] Updated weights for policy 0, policy_version 346477 (0.0005) [2023-12-27 08:02:28,939][20347] Updated weights for policy 0, policy_version 346487 (0.0005) [2023-12-27 08:02:29,228][20381] Updated weights for policy 1, policy_version 346338 (0.0010) [2023-12-27 08:02:29,281][20381] Updated weights for policy 1, policy_version 346348 (0.0010) [2023-12-27 08:02:29,339][20381] Updated weights for policy 1, policy_version 346358 (0.0008) [2023-12-27 08:02:29,589][20347] Updated weights for policy 0, policy_version 346497 (0.0006) [2023-12-27 08:02:29,639][20347] Updated weights for policy 0, policy_version 346507 (0.0005) [2023-12-27 08:02:29,691][20347] Updated weights for policy 0, policy_version 346517 (0.0005) [2023-12-27 08:02:29,746][20347] Updated weights for policy 0, policy_version 346527 (0.0005) [2023-12-27 08:02:30,005][20381] Updated weights for policy 1, policy_version 346368 (0.0007) [2023-12-27 08:02:30,058][20381] Updated weights for policy 1, policy_version 346378 (0.0007) [2023-12-27 08:02:30,113][20381] Updated weights for policy 1, policy_version 346388 (0.0007) [2023-12-27 08:02:30,394][20347] Updated weights for policy 0, policy_version 346537 (0.0010) [2023-12-27 08:02:30,448][20347] Updated weights for policy 0, policy_version 346547 (0.0010) [2023-12-27 08:02:30,505][20347] Updated weights for policy 0, policy_version 346557 (0.0010) [2023-12-27 08:02:30,793][20381] Updated weights for policy 1, policy_version 346398 (0.0008) [2023-12-27 08:02:30,840][20381] Updated weights for policy 1, policy_version 346408 (0.0005) [2023-12-27 08:02:30,888][20381] Updated weights for policy 1, policy_version 346418 (0.0005) [2023-12-27 08:02:31,138][20347] Updated weights for policy 0, policy_version 346567 (0.0009) [2023-12-27 08:02:31,199][20347] Updated weights for policy 0, policy_version 346577 (0.0007) [2023-12-27 08:02:31,260][20347] Updated weights for policy 0, policy_version 346587 (0.0007) [2023-12-27 08:02:31,507][20381] Updated weights for policy 1, policy_version 346428 (0.0007) [2023-12-27 08:02:31,568][20381] Updated weights for policy 1, policy_version 346438 (0.0011) [2023-12-27 08:02:31,627][20381] Updated weights for policy 1, policy_version 346448 (0.0009) [2023-12-27 08:02:31,975][20347] Updated weights for policy 0, policy_version 346597 (0.0007) [2023-12-27 08:02:32,032][20347] Updated weights for policy 0, policy_version 346607 (0.0007) [2023-12-27 08:02:32,084][20347] Updated weights for policy 0, policy_version 346617 (0.0008) [2023-12-27 08:02:32,279][20381] Updated weights for policy 1, policy_version 346458 (0.0009) [2023-12-27 08:02:32,335][20381] Updated weights for policy 1, policy_version 346468 (0.0008) [2023-12-27 08:02:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 177463296. Throughput: 0: 10223.4, 1: 10435.4. Samples: 177475312. Policy #0 lag: (min: 45.0, avg: 48.0, max: 48.0) [2023-12-27 08:02:32,363][19170] Avg episode reward: [(0, '999.690'), (1, '990.450')] [2023-12-27 08:02:32,393][20381] Updated weights for policy 1, policy_version 346478 (0.0008) [2023-12-27 08:02:32,454][20381] Updated weights for policy 1, policy_version 346488 (0.0007) [2023-12-27 08:02:32,787][20347] Updated weights for policy 0, policy_version 346627 (0.0006) [2023-12-27 08:02:32,845][20347] Updated weights for policy 0, policy_version 346637 (0.0007) [2023-12-27 08:02:32,903][20347] Updated weights for policy 0, policy_version 346647 (0.0008) [2023-12-27 08:02:33,098][20381] Updated weights for policy 1, policy_version 346498 (0.0008) [2023-12-27 08:02:33,145][20381] Updated weights for policy 1, policy_version 346508 (0.0010) [2023-12-27 08:02:33,201][20381] Updated weights for policy 1, policy_version 346518 (0.0007) [2023-12-27 08:02:33,528][20347] Updated weights for policy 0, policy_version 346657 (0.0007) [2023-12-27 08:02:33,581][20347] Updated weights for policy 0, policy_version 346667 (0.0005) [2023-12-27 08:02:33,635][20347] Updated weights for policy 0, policy_version 346677 (0.0005) [2023-12-27 08:02:33,686][20347] Updated weights for policy 0, policy_version 346687 (0.0005) [2023-12-27 08:02:33,870][20381] Updated weights for policy 1, policy_version 346528 (0.0009) [2023-12-27 08:02:33,921][20381] Updated weights for policy 1, policy_version 346538 (0.0007) [2023-12-27 08:02:33,967][20381] Updated weights for policy 1, policy_version 346548 (0.0005) [2023-12-27 08:02:34,248][20347] Updated weights for policy 0, policy_version 346697 (0.0007) [2023-12-27 08:02:34,294][20347] Updated weights for policy 0, policy_version 346707 (0.0007) [2023-12-27 08:02:34,340][20347] Updated weights for policy 0, policy_version 346717 (0.0007) [2023-12-27 08:02:34,599][20381] Updated weights for policy 1, policy_version 346558 (0.0005) [2023-12-27 08:02:34,663][20381] Updated weights for policy 1, policy_version 346568 (0.0007) [2023-12-27 08:02:34,726][20381] Updated weights for policy 1, policy_version 346578 (0.0008) [2023-12-27 08:02:35,021][20347] Updated weights for policy 0, policy_version 346727 (0.0007) [2023-12-27 08:02:35,072][20347] Updated weights for policy 0, policy_version 346737 (0.0008) [2023-12-27 08:02:35,118][20347] Updated weights for policy 0, policy_version 346747 (0.0006) [2023-12-27 08:02:35,287][20381] Updated weights for policy 1, policy_version 346588 (0.0007) [2023-12-27 08:02:35,342][20381] Updated weights for policy 1, policy_version 346598 (0.0007) [2023-12-27 08:02:35,401][20381] Updated weights for policy 1, policy_version 346608 (0.0008) [2023-12-27 08:02:35,718][20347] Updated weights for policy 0, policy_version 346757 (0.0005) [2023-12-27 08:02:35,766][20347] Updated weights for policy 0, policy_version 346767 (0.0005) [2023-12-27 08:02:35,815][20347] Updated weights for policy 0, policy_version 346777 (0.0007) [2023-12-27 08:02:36,063][20381] Updated weights for policy 1, policy_version 346618 (0.0007) [2023-12-27 08:02:36,123][20381] Updated weights for policy 1, policy_version 346628 (0.0007) [2023-12-27 08:02:36,184][20381] Updated weights for policy 1, policy_version 346638 (0.0008) [2023-12-27 08:02:36,243][20381] Updated weights for policy 1, policy_version 346648 (0.0008) [2023-12-27 08:02:36,487][20347] Updated weights for policy 0, policy_version 346787 (0.0008) [2023-12-27 08:02:36,546][20347] Updated weights for policy 0, policy_version 346797 (0.0008) [2023-12-27 08:02:36,604][20347] Updated weights for policy 0, policy_version 346807 (0.0007) [2023-12-27 08:02:36,863][20381] Updated weights for policy 1, policy_version 346658 (0.0008) [2023-12-27 08:02:36,920][20381] Updated weights for policy 1, policy_version 346668 (0.0008) [2023-12-27 08:02:36,980][20381] Updated weights for policy 1, policy_version 346678 (0.0008) [2023-12-27 08:02:37,329][20347] Updated weights for policy 0, policy_version 346817 (0.0009) [2023-12-27 08:02:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 177569792. Throughput: 0: 10266.8, 1: 10442.5. Samples: 177540836. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:02:37,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 08:02:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000346680_88768512.pth... [2023-12-27 08:02:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000345464_88457216.pth [2023-12-27 08:02:37,383][20347] Updated weights for policy 0, policy_version 346827 (0.0008) [2023-12-27 08:02:37,441][20347] Updated weights for policy 0, policy_version 346837 (0.0007) [2023-12-27 08:02:37,501][20347] Updated weights for policy 0, policy_version 346847 (0.0005) [2023-12-27 08:02:37,504][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000346848_88809472.pth... [2023-12-27 08:02:37,508][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000345632_88498176.pth [2023-12-27 08:02:37,685][20381] Updated weights for policy 1, policy_version 346688 (0.0007) [2023-12-27 08:02:37,740][20381] Updated weights for policy 1, policy_version 346698 (0.0008) [2023-12-27 08:02:37,808][20381] Updated weights for policy 1, policy_version 346708 (0.0009) [2023-12-27 08:02:38,150][20347] Updated weights for policy 0, policy_version 346857 (0.0010) [2023-12-27 08:02:38,199][20347] Updated weights for policy 0, policy_version 346867 (0.0010) [2023-12-27 08:02:38,249][20347] Updated weights for policy 0, policy_version 346877 (0.0010) [2023-12-27 08:02:38,467][20381] Updated weights for policy 1, policy_version 346718 (0.0007) [2023-12-27 08:02:38,525][20381] Updated weights for policy 1, policy_version 346728 (0.0008) [2023-12-27 08:02:38,581][20381] Updated weights for policy 1, policy_version 346738 (0.0007) [2023-12-27 08:02:38,923][20347] Updated weights for policy 0, policy_version 346887 (0.0007) [2023-12-27 08:02:38,981][20347] Updated weights for policy 0, policy_version 346897 (0.0006) [2023-12-27 08:02:39,027][20347] Updated weights for policy 0, policy_version 346907 (0.0008) [2023-12-27 08:02:39,291][20381] Updated weights for policy 1, policy_version 346748 (0.0008) [2023-12-27 08:02:39,353][20381] Updated weights for policy 1, policy_version 346758 (0.0008) [2023-12-27 08:02:39,416][20381] Updated weights for policy 1, policy_version 346768 (0.0008) [2023-12-27 08:02:39,688][20347] Updated weights for policy 0, policy_version 346917 (0.0007) [2023-12-27 08:02:39,753][20347] Updated weights for policy 0, policy_version 346927 (0.0007) [2023-12-27 08:02:39,813][20347] Updated weights for policy 0, policy_version 346937 (0.0007) [2023-12-27 08:02:40,137][20381] Updated weights for policy 1, policy_version 346778 (0.0007) [2023-12-27 08:02:40,195][20381] Updated weights for policy 1, policy_version 346788 (0.0007) [2023-12-27 08:02:40,260][20381] Updated weights for policy 1, policy_version 346798 (0.0008) [2023-12-27 08:02:40,322][20381] Updated weights for policy 1, policy_version 346808 (0.0009) [2023-12-27 08:02:40,425][20347] Updated weights for policy 0, policy_version 346947 (0.0007) [2023-12-27 08:02:40,480][20347] Updated weights for policy 0, policy_version 346957 (0.0008) [2023-12-27 08:02:40,528][20347] Updated weights for policy 0, policy_version 346967 (0.0007) [2023-12-27 08:02:40,962][20381] Updated weights for policy 1, policy_version 346818 (0.0005) [2023-12-27 08:02:41,014][20381] Updated weights for policy 1, policy_version 346828 (0.0006) [2023-12-27 08:02:41,080][20381] Updated weights for policy 1, policy_version 346838 (0.0007) [2023-12-27 08:02:41,300][20347] Updated weights for policy 0, policy_version 346977 (0.0008) [2023-12-27 08:02:41,364][20347] Updated weights for policy 0, policy_version 346987 (0.0009) [2023-12-27 08:02:41,428][20347] Updated weights for policy 0, policy_version 346997 (0.0007) [2023-12-27 08:02:41,489][20347] Updated weights for policy 0, policy_version 347007 (0.0005) [2023-12-27 08:02:41,759][20381] Updated weights for policy 1, policy_version 346848 (0.0008) [2023-12-27 08:02:41,817][20381] Updated weights for policy 1, policy_version 346858 (0.0008) [2023-12-27 08:02:41,879][20381] Updated weights for policy 1, policy_version 346868 (0.0006) [2023-12-27 08:02:42,177][20347] Updated weights for policy 0, policy_version 347017 (0.0010) [2023-12-27 08:02:42,229][20347] Updated weights for policy 0, policy_version 347027 (0.0007) [2023-12-27 08:02:42,288][20347] Updated weights for policy 0, policy_version 347037 (0.0009) [2023-12-27 08:02:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 177676288. Throughput: 0: 10294.8, 1: 10454.4. Samples: 177664344. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:02:42,363][19170] Avg episode reward: [(0, '980.680'), (1, '985.690')] [2023-12-27 08:02:42,520][20381] Updated weights for policy 1, policy_version 346878 (0.0005) [2023-12-27 08:02:42,579][20381] Updated weights for policy 1, policy_version 346888 (0.0005) [2023-12-27 08:02:42,639][20381] Updated weights for policy 1, policy_version 346898 (0.0007) [2023-12-27 08:02:42,990][20347] Updated weights for policy 0, policy_version 347047 (0.0007) [2023-12-27 08:02:43,047][20347] Updated weights for policy 0, policy_version 347057 (0.0007) [2023-12-27 08:02:43,114][20347] Updated weights for policy 0, policy_version 347067 (0.0007) [2023-12-27 08:02:43,279][20381] Updated weights for policy 1, policy_version 346908 (0.0007) [2023-12-27 08:02:43,335][20381] Updated weights for policy 1, policy_version 346918 (0.0008) [2023-12-27 08:02:43,382][20381] Updated weights for policy 1, policy_version 346928 (0.0007) [2023-12-27 08:02:43,754][20347] Updated weights for policy 0, policy_version 347077 (0.0007) [2023-12-27 08:02:43,812][20347] Updated weights for policy 0, policy_version 347087 (0.0005) [2023-12-27 08:02:43,867][20347] Updated weights for policy 0, policy_version 347097 (0.0006) [2023-12-27 08:02:44,068][20381] Updated weights for policy 1, policy_version 346938 (0.0009) [2023-12-27 08:02:44,124][20381] Updated weights for policy 1, policy_version 346948 (0.0006) [2023-12-27 08:02:44,183][20381] Updated weights for policy 1, policy_version 346958 (0.0006) [2023-12-27 08:02:44,241][20381] Updated weights for policy 1, policy_version 346968 (0.0007) [2023-12-27 08:02:44,510][20347] Updated weights for policy 0, policy_version 347107 (0.0008) [2023-12-27 08:02:44,564][20347] Updated weights for policy 0, policy_version 347117 (0.0007) [2023-12-27 08:02:44,625][20347] Updated weights for policy 0, policy_version 347127 (0.0008) [2023-12-27 08:02:44,889][20381] Updated weights for policy 1, policy_version 346978 (0.0005) [2023-12-27 08:02:44,955][20381] Updated weights for policy 1, policy_version 346988 (0.0005) [2023-12-27 08:02:45,017][20381] Updated weights for policy 1, policy_version 346998 (0.0008) [2023-12-27 08:02:45,298][20347] Updated weights for policy 0, policy_version 347137 (0.0008) [2023-12-27 08:02:45,351][20347] Updated weights for policy 0, policy_version 347147 (0.0007) [2023-12-27 08:02:45,403][20347] Updated weights for policy 0, policy_version 347157 (0.0008) [2023-12-27 08:02:45,454][20347] Updated weights for policy 0, policy_version 347167 (0.0007) [2023-12-27 08:02:45,614][20381] Updated weights for policy 1, policy_version 347008 (0.0007) [2023-12-27 08:02:45,680][20381] Updated weights for policy 1, policy_version 347018 (0.0006) [2023-12-27 08:02:45,730][20381] Updated weights for policy 1, policy_version 347028 (0.0009) [2023-12-27 08:02:46,171][20347] Updated weights for policy 0, policy_version 347177 (0.0006) [2023-12-27 08:02:46,234][20347] Updated weights for policy 0, policy_version 347187 (0.0005) [2023-12-27 08:02:46,295][20347] Updated weights for policy 0, policy_version 347197 (0.0005) [2023-12-27 08:02:46,363][20381] Updated weights for policy 1, policy_version 347038 (0.0006) [2023-12-27 08:02:46,421][20381] Updated weights for policy 1, policy_version 347048 (0.0005) [2023-12-27 08:02:46,476][20381] Updated weights for policy 1, policy_version 347058 (0.0005) [2023-12-27 08:02:46,933][20347] Updated weights for policy 0, policy_version 347207 (0.0007) [2023-12-27 08:02:46,994][20347] Updated weights for policy 0, policy_version 347217 (0.0008) [2023-12-27 08:02:46,999][20381] Updated weights for policy 1, policy_version 347068 (0.0005) [2023-12-27 08:02:47,044][20347] Updated weights for policy 0, policy_version 347227 (0.0007) [2023-12-27 08:02:47,068][20381] Updated weights for policy 1, policy_version 347078 (0.0005) [2023-12-27 08:02:47,131][20381] Updated weights for policy 1, policy_version 347088 (0.0005) [2023-12-27 08:02:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 177782784. Throughput: 0: 10277.5, 1: 10483.7. Samples: 177790412. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:02:47,363][19170] Avg episode reward: [(0, '971.170'), (1, '966.110')] [2023-12-27 08:02:47,709][20381] Updated weights for policy 1, policy_version 347098 (0.0006) [2023-12-27 08:02:47,763][20381] Updated weights for policy 1, policy_version 347108 (0.0006) [2023-12-27 08:02:47,775][20347] Updated weights for policy 0, policy_version 347237 (0.0008) [2023-12-27 08:02:47,820][20381] Updated weights for policy 1, policy_version 347118 (0.0007) [2023-12-27 08:02:47,833][20347] Updated weights for policy 0, policy_version 347247 (0.0006) [2023-12-27 08:02:47,844][20291] KL-divergence is very high: 256.5223 [2023-12-27 08:02:47,869][20381] Updated weights for policy 1, policy_version 347128 (0.0007) [2023-12-27 08:02:47,884][20347] Updated weights for policy 0, policy_version 347257 (0.0008) [2023-12-27 08:02:48,564][20381] Updated weights for policy 1, policy_version 347138 (0.0011) [2023-12-27 08:02:48,568][20347] Updated weights for policy 0, policy_version 347267 (0.0008) [2023-12-27 08:02:48,618][20347] Updated weights for policy 0, policy_version 347277 (0.0006) [2023-12-27 08:02:48,619][20381] Updated weights for policy 1, policy_version 347148 (0.0010) [2023-12-27 08:02:48,669][20381] Updated weights for policy 1, policy_version 347158 (0.0010) [2023-12-27 08:02:48,671][20347] Updated weights for policy 0, policy_version 347287 (0.0005) [2023-12-27 08:02:49,285][20381] Updated weights for policy 1, policy_version 347168 (0.0011) [2023-12-27 08:02:49,345][20381] Updated weights for policy 1, policy_version 347178 (0.0009) [2023-12-27 08:02:49,359][20347] Updated weights for policy 0, policy_version 347297 (0.0007) [2023-12-27 08:02:49,403][20381] Updated weights for policy 1, policy_version 347188 (0.0008) [2023-12-27 08:02:49,419][20347] Updated weights for policy 0, policy_version 347307 (0.0008) [2023-12-27 08:02:49,474][20347] Updated weights for policy 0, policy_version 347317 (0.0007) [2023-12-27 08:02:49,529][20347] Updated weights for policy 0, policy_version 347327 (0.0008) [2023-12-27 08:02:50,042][20381] Updated weights for policy 1, policy_version 347198 (0.0006) [2023-12-27 08:02:50,098][20381] Updated weights for policy 1, policy_version 347208 (0.0006) [2023-12-27 08:02:50,156][20381] Updated weights for policy 1, policy_version 347218 (0.0005) [2023-12-27 08:02:50,240][20347] Updated weights for policy 0, policy_version 347337 (0.0007) [2023-12-27 08:02:50,290][20347] Updated weights for policy 0, policy_version 347347 (0.0007) [2023-12-27 08:02:50,374][20347] Updated weights for policy 0, policy_version 347357 (0.0007) [2023-12-27 08:02:50,853][20381] Updated weights for policy 1, policy_version 347228 (0.0009) [2023-12-27 08:02:50,907][20381] Updated weights for policy 1, policy_version 347238 (0.0006) [2023-12-27 08:02:50,964][20381] Updated weights for policy 1, policy_version 347248 (0.0008) [2023-12-27 08:02:51,050][20347] Updated weights for policy 0, policy_version 347367 (0.0008) [2023-12-27 08:02:51,109][20347] Updated weights for policy 0, policy_version 347377 (0.0008) [2023-12-27 08:02:51,166][20347] Updated weights for policy 0, policy_version 347387 (0.0008) [2023-12-27 08:02:51,632][20381] Updated weights for policy 1, policy_version 347258 (0.0010) [2023-12-27 08:02:51,694][20381] Updated weights for policy 1, policy_version 347268 (0.0008) [2023-12-27 08:02:51,755][20381] Updated weights for policy 1, policy_version 347278 (0.0008) [2023-12-27 08:02:51,808][20381] Updated weights for policy 1, policy_version 347288 (0.0007) [2023-12-27 08:02:51,864][20347] Updated weights for policy 0, policy_version 347397 (0.0008) [2023-12-27 08:02:51,923][20347] Updated weights for policy 0, policy_version 347407 (0.0008) [2023-12-27 08:02:51,978][20347] Updated weights for policy 0, policy_version 347417 (0.0008) [2023-12-27 08:02:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 177881088. Throughput: 0: 10283.9, 1: 10474.6. Samples: 177852456. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:02:52,363][19170] Avg episode reward: [(0, '971.180'), (1, '956.140')] [2023-12-27 08:02:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000347288_88924160.pth... [2023-12-27 08:02:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000347424_88956928.pth... [2023-12-27 08:02:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000346072_88612864.pth [2023-12-27 08:02:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000346208_88645632.pth [2023-12-27 08:02:52,455][20381] Updated weights for policy 1, policy_version 347298 (0.0007) [2023-12-27 08:02:52,514][20381] Updated weights for policy 1, policy_version 347308 (0.0006) [2023-12-27 08:02:52,576][20381] Updated weights for policy 1, policy_version 347318 (0.0007) [2023-12-27 08:02:52,722][20347] Updated weights for policy 0, policy_version 347427 (0.0008) [2023-12-27 08:02:52,772][20347] Updated weights for policy 0, policy_version 347437 (0.0007) [2023-12-27 08:02:52,821][20347] Updated weights for policy 0, policy_version 347447 (0.0006) [2023-12-27 08:02:53,229][20381] Updated weights for policy 1, policy_version 347328 (0.0008) [2023-12-27 08:02:53,281][20381] Updated weights for policy 1, policy_version 347338 (0.0007) [2023-12-27 08:02:53,325][20381] Updated weights for policy 1, policy_version 347348 (0.0008) [2023-12-27 08:02:53,452][20347] Updated weights for policy 0, policy_version 347457 (0.0008) [2023-12-27 08:02:53,508][20347] Updated weights for policy 0, policy_version 347467 (0.0005) [2023-12-27 08:02:53,562][20347] Updated weights for policy 0, policy_version 347477 (0.0005) [2023-12-27 08:02:53,623][20347] Updated weights for policy 0, policy_version 347487 (0.0005) [2023-12-27 08:02:53,980][20381] Updated weights for policy 1, policy_version 347358 (0.0005) [2023-12-27 08:02:54,027][20381] Updated weights for policy 1, policy_version 347368 (0.0006) [2023-12-27 08:02:54,077][20381] Updated weights for policy 1, policy_version 347378 (0.0006) [2023-12-27 08:02:54,213][20347] Updated weights for policy 0, policy_version 347497 (0.0010) [2023-12-27 08:02:54,270][20347] Updated weights for policy 0, policy_version 347507 (0.0010) [2023-12-27 08:02:54,333][20347] Updated weights for policy 0, policy_version 347517 (0.0009) [2023-12-27 08:02:54,637][20381] Updated weights for policy 1, policy_version 347388 (0.0005) [2023-12-27 08:02:54,686][20381] Updated weights for policy 1, policy_version 347398 (0.0007) [2023-12-27 08:02:54,731][20381] Updated weights for policy 1, policy_version 347408 (0.0010) [2023-12-27 08:02:54,922][20347] Updated weights for policy 0, policy_version 347527 (0.0010) [2023-12-27 08:02:54,980][20347] Updated weights for policy 0, policy_version 347537 (0.0007) [2023-12-27 08:02:55,036][20347] Updated weights for policy 0, policy_version 347547 (0.0006) [2023-12-27 08:02:55,412][20381] Updated weights for policy 1, policy_version 347418 (0.0010) [2023-12-27 08:02:55,461][20381] Updated weights for policy 1, policy_version 347428 (0.0010) [2023-12-27 08:02:55,515][20381] Updated weights for policy 1, policy_version 347438 (0.0010) [2023-12-27 08:02:55,570][20381] Updated weights for policy 1, policy_version 347448 (0.0010) [2023-12-27 08:02:55,672][20347] Updated weights for policy 0, policy_version 347557 (0.0007) [2023-12-27 08:02:55,722][20347] Updated weights for policy 0, policy_version 347567 (0.0007) [2023-12-27 08:02:55,774][20347] Updated weights for policy 0, policy_version 347577 (0.0008) [2023-12-27 08:02:56,256][20381] Updated weights for policy 1, policy_version 347458 (0.0011) [2023-12-27 08:02:56,299][20381] Updated weights for policy 1, policy_version 347468 (0.0007) [2023-12-27 08:02:56,344][20381] Updated weights for policy 1, policy_version 347478 (0.0008) [2023-12-27 08:02:56,457][20347] Updated weights for policy 0, policy_version 347587 (0.0007) [2023-12-27 08:02:56,510][20347] Updated weights for policy 0, policy_version 347597 (0.0007) [2023-12-27 08:02:56,550][20347] Updated weights for policy 0, policy_version 347607 (0.0007) [2023-12-27 08:02:57,034][20381] Updated weights for policy 1, policy_version 347488 (0.0010) [2023-12-27 08:02:57,089][20381] Updated weights for policy 1, policy_version 347498 (0.0011) [2023-12-27 08:02:57,146][20381] Updated weights for policy 1, policy_version 347508 (0.0011) [2023-12-27 08:02:57,264][20347] Updated weights for policy 0, policy_version 347617 (0.0008) [2023-12-27 08:02:57,322][20347] Updated weights for policy 0, policy_version 347627 (0.0006) [2023-12-27 08:02:57,322][20291] KL-divergence is very high: 167.6541 [2023-12-27 08:02:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 177987584. Throughput: 0: 10332.9, 1: 10510.5. Samples: 177979148. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:02:57,363][19170] Avg episode reward: [(0, '980.720'), (1, '972.810')] [2023-12-27 08:02:57,380][20347] Updated weights for policy 0, policy_version 347637 (0.0007) [2023-12-27 08:02:57,431][20347] Updated weights for policy 0, policy_version 347647 (0.0007) [2023-12-27 08:02:57,787][20381] Updated weights for policy 1, policy_version 347518 (0.0010) [2023-12-27 08:02:57,844][20381] Updated weights for policy 1, policy_version 347528 (0.0010) [2023-12-27 08:02:57,891][20381] Updated weights for policy 1, policy_version 347538 (0.0010) [2023-12-27 08:02:58,128][20347] Updated weights for policy 0, policy_version 347657 (0.0007) [2023-12-27 08:02:58,191][20347] Updated weights for policy 0, policy_version 347667 (0.0008) [2023-12-27 08:02:58,247][20347] Updated weights for policy 0, policy_version 347677 (0.0008) [2023-12-27 08:02:58,537][20381] Updated weights for policy 1, policy_version 347548 (0.0009) [2023-12-27 08:02:58,597][20381] Updated weights for policy 1, policy_version 347558 (0.0007) [2023-12-27 08:02:58,655][20381] Updated weights for policy 1, policy_version 347568 (0.0006) [2023-12-27 08:02:59,054][20347] Updated weights for policy 0, policy_version 347687 (0.0008) [2023-12-27 08:02:59,107][20347] Updated weights for policy 0, policy_version 347697 (0.0007) [2023-12-27 08:02:59,168][20347] Updated weights for policy 0, policy_version 347707 (0.0006) [2023-12-27 08:02:59,367][20381] Updated weights for policy 1, policy_version 347578 (0.0007) [2023-12-27 08:02:59,419][20381] Updated weights for policy 1, policy_version 347588 (0.0007) [2023-12-27 08:02:59,471][20381] Updated weights for policy 1, policy_version 347598 (0.0008) [2023-12-27 08:02:59,516][20381] Updated weights for policy 1, policy_version 347608 (0.0009) [2023-12-27 08:02:59,797][20347] Updated weights for policy 0, policy_version 347717 (0.0007) [2023-12-27 08:02:59,858][20347] Updated weights for policy 0, policy_version 347727 (0.0009) [2023-12-27 08:02:59,922][20347] Updated weights for policy 0, policy_version 347737 (0.0008) [2023-12-27 08:03:00,254][20381] Updated weights for policy 1, policy_version 347618 (0.0007) [2023-12-27 08:03:00,339][20381] Updated weights for policy 1, policy_version 347628 (0.0009) [2023-12-27 08:03:00,396][20381] Updated weights for policy 1, policy_version 347638 (0.0008) [2023-12-27 08:03:00,588][20347] Updated weights for policy 0, policy_version 347747 (0.0007) [2023-12-27 08:03:00,646][20347] Updated weights for policy 0, policy_version 347757 (0.0006) [2023-12-27 08:03:00,697][20347] Updated weights for policy 0, policy_version 347767 (0.0005) [2023-12-27 08:03:01,103][20381] Updated weights for policy 1, policy_version 347648 (0.0008) [2023-12-27 08:03:01,168][20381] Updated weights for policy 1, policy_version 347658 (0.0008) [2023-12-27 08:03:01,229][20381] Updated weights for policy 1, policy_version 347668 (0.0006) [2023-12-27 08:03:01,267][20347] Updated weights for policy 0, policy_version 347777 (0.0005) [2023-12-27 08:03:01,333][20347] Updated weights for policy 0, policy_version 347787 (0.0005) [2023-12-27 08:03:01,400][20347] Updated weights for policy 0, policy_version 347797 (0.0008) [2023-12-27 08:03:01,460][20347] Updated weights for policy 0, policy_version 347807 (0.0006) [2023-12-27 08:03:01,846][20381] Updated weights for policy 1, policy_version 347678 (0.0007) [2023-12-27 08:03:01,905][20381] Updated weights for policy 1, policy_version 347688 (0.0007) [2023-12-27 08:03:01,964][20381] Updated weights for policy 1, policy_version 347698 (0.0008) [2023-12-27 08:03:02,133][20347] Updated weights for policy 0, policy_version 347817 (0.0007) [2023-12-27 08:03:02,190][20347] Updated weights for policy 0, policy_version 347827 (0.0007) [2023-12-27 08:03:02,243][20347] Updated weights for policy 0, policy_version 347837 (0.0005) [2023-12-27 08:03:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 178094080. Throughput: 0: 10327.4, 1: 10490.4. Samples: 178101292. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:02,363][19170] Avg episode reward: [(0, '990.380'), (1, '1000.000')] [2023-12-27 08:03:02,626][20381] Updated weights for policy 1, policy_version 347708 (0.0007) [2023-12-27 08:03:02,678][20381] Updated weights for policy 1, policy_version 347718 (0.0008) [2023-12-27 08:03:02,740][20381] Updated weights for policy 1, policy_version 347728 (0.0008) [2023-12-27 08:03:02,956][20347] Updated weights for policy 0, policy_version 347847 (0.0007) [2023-12-27 08:03:03,015][20347] Updated weights for policy 0, policy_version 347857 (0.0007) [2023-12-27 08:03:03,071][20347] Updated weights for policy 0, policy_version 347867 (0.0008) [2023-12-27 08:03:03,305][20381] Updated weights for policy 1, policy_version 347738 (0.0006) [2023-12-27 08:03:03,353][20381] Updated weights for policy 1, policy_version 347748 (0.0007) [2023-12-27 08:03:03,405][20381] Updated weights for policy 1, policy_version 347758 (0.0007) [2023-12-27 08:03:03,464][20381] Updated weights for policy 1, policy_version 347768 (0.0008) [2023-12-27 08:03:03,720][20347] Updated weights for policy 0, policy_version 347877 (0.0008) [2023-12-27 08:03:03,776][20347] Updated weights for policy 0, policy_version 347887 (0.0008) [2023-12-27 08:03:03,827][20347] Updated weights for policy 0, policy_version 347897 (0.0007) [2023-12-27 08:03:04,213][20381] Updated weights for policy 1, policy_version 347778 (0.0009) [2023-12-27 08:03:04,273][20381] Updated weights for policy 1, policy_version 347788 (0.0008) [2023-12-27 08:03:04,336][20381] Updated weights for policy 1, policy_version 347798 (0.0007) [2023-12-27 08:03:04,481][20347] Updated weights for policy 0, policy_version 347907 (0.0007) [2023-12-27 08:03:04,536][20347] Updated weights for policy 0, policy_version 347917 (0.0007) [2023-12-27 08:03:04,588][20347] Updated weights for policy 0, policy_version 347927 (0.0007) [2023-12-27 08:03:05,015][20381] Updated weights for policy 1, policy_version 347808 (0.0007) [2023-12-27 08:03:05,073][20381] Updated weights for policy 1, policy_version 347818 (0.0006) [2023-12-27 08:03:05,130][20381] Updated weights for policy 1, policy_version 347828 (0.0007) [2023-12-27 08:03:05,315][20347] Updated weights for policy 0, policy_version 347937 (0.0008) [2023-12-27 08:03:05,364][20347] Updated weights for policy 0, policy_version 347947 (0.0007) [2023-12-27 08:03:05,417][20347] Updated weights for policy 0, policy_version 347957 (0.0007) [2023-12-27 08:03:05,470][20347] Updated weights for policy 0, policy_version 347967 (0.0008) [2023-12-27 08:03:05,793][20381] Updated weights for policy 1, policy_version 347838 (0.0007) [2023-12-27 08:03:05,846][20381] Updated weights for policy 1, policy_version 347848 (0.0006) [2023-12-27 08:03:05,905][20381] Updated weights for policy 1, policy_version 347858 (0.0005) [2023-12-27 08:03:06,129][20347] Updated weights for policy 0, policy_version 347977 (0.0009) [2023-12-27 08:03:06,180][20347] Updated weights for policy 0, policy_version 347987 (0.0010) [2023-12-27 08:03:06,231][20347] Updated weights for policy 0, policy_version 347997 (0.0008) [2023-12-27 08:03:06,538][20381] Updated weights for policy 1, policy_version 347868 (0.0006) [2023-12-27 08:03:06,595][20381] Updated weights for policy 1, policy_version 347878 (0.0007) [2023-12-27 08:03:06,653][20381] Updated weights for policy 1, policy_version 347888 (0.0007) [2023-12-27 08:03:06,896][20347] Updated weights for policy 0, policy_version 348007 (0.0005) [2023-12-27 08:03:06,964][20347] Updated weights for policy 0, policy_version 348017 (0.0009) [2023-12-27 08:03:07,025][20347] Updated weights for policy 0, policy_version 348027 (0.0010) [2023-12-27 08:03:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 178192384. Throughput: 0: 10323.3, 1: 10515.8. Samples: 178162812. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:03:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000348032_89112576.pth... [2023-12-27 08:03:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000346848_88809472.pth [2023-12-27 08:03:07,384][20381] Updated weights for policy 1, policy_version 347898 (0.0007) [2023-12-27 08:03:07,431][20381] Updated weights for policy 1, policy_version 347908 (0.0007) [2023-12-27 08:03:07,488][20381] Updated weights for policy 1, policy_version 347918 (0.0007) [2023-12-27 08:03:07,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000347928_89088000.pth... [2023-12-27 08:03:07,539][20381] Updated weights for policy 1, policy_version 347928 (0.0007) [2023-12-27 08:03:07,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000346680_88768512.pth [2023-12-27 08:03:07,677][20347] Updated weights for policy 0, policy_version 348037 (0.0010) [2023-12-27 08:03:07,724][20347] Updated weights for policy 0, policy_version 348047 (0.0009) [2023-12-27 08:03:07,784][20347] Updated weights for policy 0, policy_version 348057 (0.0005) [2023-12-27 08:03:08,214][20381] Updated weights for policy 1, policy_version 347938 (0.0007) [2023-12-27 08:03:08,278][20381] Updated weights for policy 1, policy_version 347948 (0.0008) [2023-12-27 08:03:08,330][20381] Updated weights for policy 1, policy_version 347958 (0.0009) [2023-12-27 08:03:08,380][20347] Updated weights for policy 0, policy_version 348067 (0.0006) [2023-12-27 08:03:08,443][20347] Updated weights for policy 0, policy_version 348077 (0.0006) [2023-12-27 08:03:08,509][20347] Updated weights for policy 0, policy_version 348087 (0.0005) [2023-12-27 08:03:09,028][20381] Updated weights for policy 1, policy_version 347968 (0.0010) [2023-12-27 08:03:09,096][20381] Updated weights for policy 1, policy_version 347978 (0.0006) [2023-12-27 08:03:09,124][20347] Updated weights for policy 0, policy_version 348097 (0.0005) [2023-12-27 08:03:09,150][20381] Updated weights for policy 1, policy_version 347988 (0.0011) [2023-12-27 08:03:09,184][20347] Updated weights for policy 0, policy_version 348107 (0.0005) [2023-12-27 08:03:09,249][20347] Updated weights for policy 0, policy_version 348117 (0.0006) [2023-12-27 08:03:09,309][20347] Updated weights for policy 0, policy_version 348127 (0.0008) [2023-12-27 08:03:09,864][20381] Updated weights for policy 1, policy_version 347998 (0.0011) [2023-12-27 08:03:09,927][20381] Updated weights for policy 1, policy_version 348008 (0.0011) [2023-12-27 08:03:09,985][20381] Updated weights for policy 1, policy_version 348018 (0.0006) [2023-12-27 08:03:10,028][20347] Updated weights for policy 0, policy_version 348137 (0.0007) [2023-12-27 08:03:10,089][20347] Updated weights for policy 0, policy_version 348147 (0.0007) [2023-12-27 08:03:10,143][20291] KL-divergence is very high: 713.7609 [2023-12-27 08:03:10,149][20291] KL-divergence is very high: 247.2500 [2023-12-27 08:03:10,151][20347] Updated weights for policy 0, policy_version 348157 (0.0007) [2023-12-27 08:03:10,604][20381] Updated weights for policy 1, policy_version 348028 (0.0010) [2023-12-27 08:03:10,660][20381] Updated weights for policy 1, policy_version 348038 (0.0011) [2023-12-27 08:03:10,719][20381] Updated weights for policy 1, policy_version 348048 (0.0010) [2023-12-27 08:03:10,856][20347] Updated weights for policy 0, policy_version 348167 (0.0006) [2023-12-27 08:03:10,911][20347] Updated weights for policy 0, policy_version 348177 (0.0006) [2023-12-27 08:03:10,961][20347] Updated weights for policy 0, policy_version 348187 (0.0007) [2023-12-27 08:03:11,419][20381] Updated weights for policy 1, policy_version 348058 (0.0012) [2023-12-27 08:03:11,478][20381] Updated weights for policy 1, policy_version 348068 (0.0008) [2023-12-27 08:03:11,527][20381] Updated weights for policy 1, policy_version 348078 (0.0009) [2023-12-27 08:03:11,573][20381] Updated weights for policy 1, policy_version 348088 (0.0009) [2023-12-27 08:03:11,661][20347] Updated weights for policy 0, policy_version 348197 (0.0007) [2023-12-27 08:03:11,724][20347] Updated weights for policy 0, policy_version 348207 (0.0007) [2023-12-27 08:03:11,787][20347] Updated weights for policy 0, policy_version 348217 (0.0007) [2023-12-27 08:03:12,291][20381] Updated weights for policy 1, policy_version 348098 (0.0006) [2023-12-27 08:03:12,361][20381] Updated weights for policy 1, policy_version 348108 (0.0007) [2023-12-27 08:03:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 178290688. Throughput: 0: 10359.2, 1: 10506.5. Samples: 178286604. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:12,363][19170] Avg episode reward: [(0, '980.730'), (1, '1000.000')] [2023-12-27 08:03:12,420][20347] Updated weights for policy 0, policy_version 348227 (0.0006) [2023-12-27 08:03:12,420][20381] Updated weights for policy 1, policy_version 348118 (0.0006) [2023-12-27 08:03:12,484][20347] Updated weights for policy 0, policy_version 348237 (0.0008) [2023-12-27 08:03:12,545][20347] Updated weights for policy 0, policy_version 348247 (0.0008) [2023-12-27 08:03:13,008][20381] Updated weights for policy 1, policy_version 348128 (0.0008) [2023-12-27 08:03:13,069][20381] Updated weights for policy 1, policy_version 348138 (0.0008) [2023-12-27 08:03:13,131][20381] Updated weights for policy 1, policy_version 348148 (0.0007) [2023-12-27 08:03:13,197][20347] Updated weights for policy 0, policy_version 348257 (0.0007) [2023-12-27 08:03:13,252][20347] Updated weights for policy 0, policy_version 348267 (0.0008) [2023-12-27 08:03:13,307][20347] Updated weights for policy 0, policy_version 348277 (0.0006) [2023-12-27 08:03:13,365][20347] Updated weights for policy 0, policy_version 348287 (0.0005) [2023-12-27 08:03:13,796][20381] Updated weights for policy 1, policy_version 348158 (0.0006) [2023-12-27 08:03:13,853][20381] Updated weights for policy 1, policy_version 348168 (0.0007) [2023-12-27 08:03:13,895][20347] Updated weights for policy 0, policy_version 348297 (0.0005) [2023-12-27 08:03:13,902][20381] Updated weights for policy 1, policy_version 348178 (0.0008) [2023-12-27 08:03:13,943][20347] Updated weights for policy 0, policy_version 348307 (0.0005) [2023-12-27 08:03:13,989][20347] Updated weights for policy 0, policy_version 348317 (0.0005) [2023-12-27 08:03:14,580][20381] Updated weights for policy 1, policy_version 348188 (0.0007) [2023-12-27 08:03:14,624][20347] Updated weights for policy 0, policy_version 348327 (0.0008) [2023-12-27 08:03:14,638][20381] Updated weights for policy 1, policy_version 348198 (0.0007) [2023-12-27 08:03:14,684][20347] Updated weights for policy 0, policy_version 348337 (0.0008) [2023-12-27 08:03:14,693][20381] Updated weights for policy 1, policy_version 348208 (0.0005) [2023-12-27 08:03:14,743][20347] Updated weights for policy 0, policy_version 348347 (0.0008) [2023-12-27 08:03:15,337][20381] Updated weights for policy 1, policy_version 348218 (0.0006) [2023-12-27 08:03:15,394][20381] Updated weights for policy 1, policy_version 348228 (0.0007) [2023-12-27 08:03:15,451][20381] Updated weights for policy 1, policy_version 348238 (0.0006) [2023-12-27 08:03:15,453][20347] Updated weights for policy 0, policy_version 348357 (0.0010) [2023-12-27 08:03:15,506][20381] Updated weights for policy 1, policy_version 348248 (0.0005) [2023-12-27 08:03:15,511][20347] Updated weights for policy 0, policy_version 348367 (0.0010) [2023-12-27 08:03:15,560][20347] Updated weights for policy 0, policy_version 348377 (0.0008) [2023-12-27 08:03:16,204][20381] Updated weights for policy 1, policy_version 348258 (0.0005) [2023-12-27 08:03:16,214][20347] Updated weights for policy 0, policy_version 348387 (0.0007) [2023-12-27 08:03:16,262][20381] Updated weights for policy 1, policy_version 348268 (0.0006) [2023-12-27 08:03:16,272][20347] Updated weights for policy 0, policy_version 348397 (0.0009) [2023-12-27 08:03:16,318][20381] Updated weights for policy 1, policy_version 348278 (0.0005) [2023-12-27 08:03:16,331][20347] Updated weights for policy 0, policy_version 348407 (0.0007) [2023-12-27 08:03:16,973][20347] Updated weights for policy 0, policy_version 348417 (0.0008) [2023-12-27 08:03:17,017][20381] Updated weights for policy 1, policy_version 348288 (0.0006) [2023-12-27 08:03:17,027][20347] Updated weights for policy 0, policy_version 348427 (0.0005) [2023-12-27 08:03:17,060][20381] Updated weights for policy 1, policy_version 348298 (0.0005) [2023-12-27 08:03:17,073][20347] Updated weights for policy 0, policy_version 348437 (0.0005) [2023-12-27 08:03:17,115][20381] Updated weights for policy 1, policy_version 348308 (0.0005) [2023-12-27 08:03:17,124][20347] Updated weights for policy 0, policy_version 348447 (0.0005) [2023-12-27 08:03:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 178405376. Throughput: 0: 10371.9, 1: 10444.7. Samples: 178412068. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:17,364][19170] Avg episode reward: [(0, '990.310'), (1, '1000.000')] [2023-12-27 08:03:17,686][20347] Updated weights for policy 0, policy_version 348457 (0.0006) [2023-12-27 08:03:17,710][20381] Updated weights for policy 1, policy_version 348318 (0.0007) [2023-12-27 08:03:17,738][20347] Updated weights for policy 0, policy_version 348467 (0.0006) [2023-12-27 08:03:17,768][20381] Updated weights for policy 1, policy_version 348328 (0.0007) [2023-12-27 08:03:17,787][20347] Updated weights for policy 0, policy_version 348477 (0.0005) [2023-12-27 08:03:17,831][20381] Updated weights for policy 1, policy_version 348338 (0.0008) [2023-12-27 08:03:18,395][20347] Updated weights for policy 0, policy_version 348487 (0.0007) [2023-12-27 08:03:18,454][20347] Updated weights for policy 0, policy_version 348497 (0.0008) [2023-12-27 08:03:18,511][20347] Updated weights for policy 0, policy_version 348507 (0.0007) [2023-12-27 08:03:18,529][20381] Updated weights for policy 1, policy_version 348348 (0.0008) [2023-12-27 08:03:18,588][20381] Updated weights for policy 1, policy_version 348359 (0.0010) [2023-12-27 08:03:18,637][20381] Updated weights for policy 1, policy_version 348369 (0.0007) [2023-12-27 08:03:19,171][20347] Updated weights for policy 0, policy_version 348517 (0.0006) [2023-12-27 08:03:19,222][20347] Updated weights for policy 0, policy_version 348527 (0.0007) [2023-12-27 08:03:19,283][20347] Updated weights for policy 0, policy_version 348537 (0.0007) [2023-12-27 08:03:19,345][20381] Updated weights for policy 1, policy_version 348379 (0.0006) [2023-12-27 08:03:19,402][20381] Updated weights for policy 1, policy_version 348389 (0.0008) [2023-12-27 08:03:19,454][20381] Updated weights for policy 1, policy_version 348399 (0.0007) [2023-12-27 08:03:20,042][20347] Updated weights for policy 0, policy_version 348547 (0.0007) [2023-12-27 08:03:20,102][20347] Updated weights for policy 0, policy_version 348557 (0.0007) [2023-12-27 08:03:20,158][20347] Updated weights for policy 0, policy_version 348567 (0.0008) [2023-12-27 08:03:20,167][20381] Updated weights for policy 1, policy_version 348409 (0.0009) [2023-12-27 08:03:20,221][20381] Updated weights for policy 1, policy_version 348419 (0.0010) [2023-12-27 08:03:20,277][20381] Updated weights for policy 1, policy_version 348429 (0.0011) [2023-12-27 08:03:20,330][20381] Updated weights for policy 1, policy_version 348439 (0.0011) [2023-12-27 08:03:20,838][20347] Updated weights for policy 0, policy_version 348577 (0.0006) [2023-12-27 08:03:20,897][20347] Updated weights for policy 0, policy_version 348587 (0.0010) [2023-12-27 08:03:20,957][20347] Updated weights for policy 0, policy_version 348597 (0.0011) [2023-12-27 08:03:20,978][20381] Updated weights for policy 1, policy_version 348449 (0.0010) [2023-12-27 08:03:21,014][20347] Updated weights for policy 0, policy_version 348607 (0.0011) [2023-12-27 08:03:21,034][20381] Updated weights for policy 1, policy_version 348459 (0.0011) [2023-12-27 08:03:21,096][20381] Updated weights for policy 1, policy_version 348469 (0.0011) [2023-12-27 08:03:21,759][20381] Updated weights for policy 1, policy_version 348479 (0.0009) [2023-12-27 08:03:21,774][20347] Updated weights for policy 0, policy_version 348617 (0.0008) [2023-12-27 08:03:21,818][20381] Updated weights for policy 1, policy_version 348489 (0.0007) [2023-12-27 08:03:21,829][20347] Updated weights for policy 0, policy_version 348627 (0.0008) [2023-12-27 08:03:21,879][20381] Updated weights for policy 1, policy_version 348499 (0.0007) [2023-12-27 08:03:21,885][20347] Updated weights for policy 0, policy_version 348637 (0.0008) [2023-12-27 08:03:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 178503680. Throughput: 0: 10355.2, 1: 10396.3. Samples: 178474652. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.560')] [2023-12-27 08:03:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000348640_89268224.pth... [2023-12-27 08:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000348504_89235456.pth... [2023-12-27 08:03:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000347288_88924160.pth [2023-12-27 08:03:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000347424_88956928.pth [2023-12-27 08:03:22,586][20347] Updated weights for policy 0, policy_version 348647 (0.0007) [2023-12-27 08:03:22,599][20381] Updated weights for policy 1, policy_version 348509 (0.0006) [2023-12-27 08:03:22,644][20347] Updated weights for policy 0, policy_version 348657 (0.0006) [2023-12-27 08:03:22,657][20381] Updated weights for policy 1, policy_version 348519 (0.0006) [2023-12-27 08:03:22,697][20347] Updated weights for policy 0, policy_version 348667 (0.0006) [2023-12-27 08:03:22,714][20381] Updated weights for policy 1, policy_version 348529 (0.0006) [2023-12-27 08:03:23,399][20347] Updated weights for policy 0, policy_version 348677 (0.0007) [2023-12-27 08:03:23,411][20381] Updated weights for policy 1, policy_version 348539 (0.0008) [2023-12-27 08:03:23,450][20347] Updated weights for policy 0, policy_version 348687 (0.0006) [2023-12-27 08:03:23,457][20381] Updated weights for policy 1, policy_version 348549 (0.0005) [2023-12-27 08:03:23,502][20381] Updated weights for policy 1, policy_version 348559 (0.0005) [2023-12-27 08:03:23,503][20347] Updated weights for policy 0, policy_version 348697 (0.0006) [2023-12-27 08:03:24,144][20347] Updated weights for policy 0, policy_version 348707 (0.0006) [2023-12-27 08:03:24,195][20347] Updated weights for policy 0, policy_version 348717 (0.0008) [2023-12-27 08:03:24,237][20381] Updated weights for policy 1, policy_version 348569 (0.0006) [2023-12-27 08:03:24,240][20347] Updated weights for policy 0, policy_version 348727 (0.0008) [2023-12-27 08:03:24,286][20381] Updated weights for policy 1, policy_version 348579 (0.0005) [2023-12-27 08:03:24,344][20381] Updated weights for policy 1, policy_version 348589 (0.0008) [2023-12-27 08:03:24,407][20381] Updated weights for policy 1, policy_version 348599 (0.0007) [2023-12-27 08:03:24,925][20347] Updated weights for policy 0, policy_version 348737 (0.0007) [2023-12-27 08:03:24,980][20347] Updated weights for policy 0, policy_version 348747 (0.0007) [2023-12-27 08:03:25,031][20347] Updated weights for policy 0, policy_version 348757 (0.0006) [2023-12-27 08:03:25,080][20347] Updated weights for policy 0, policy_version 348767 (0.0006) [2023-12-27 08:03:25,112][20381] Updated weights for policy 1, policy_version 348609 (0.0007) [2023-12-27 08:03:25,175][20381] Updated weights for policy 1, policy_version 348619 (0.0007) [2023-12-27 08:03:25,234][20381] Updated weights for policy 1, policy_version 348629 (0.0007) [2023-12-27 08:03:25,730][20347] Updated weights for policy 0, policy_version 348777 (0.0006) [2023-12-27 08:03:25,794][20347] Updated weights for policy 0, policy_version 348787 (0.0005) [2023-12-27 08:03:25,854][20347] Updated weights for policy 0, policy_version 348797 (0.0005) [2023-12-27 08:03:25,929][20381] Updated weights for policy 1, policy_version 348639 (0.0007) [2023-12-27 08:03:25,988][20381] Updated weights for policy 1, policy_version 348649 (0.0006) [2023-12-27 08:03:26,038][20381] Updated weights for policy 1, policy_version 348659 (0.0005) [2023-12-27 08:03:26,483][20347] Updated weights for policy 0, policy_version 348807 (0.0009) [2023-12-27 08:03:26,545][20347] Updated weights for policy 0, policy_version 348817 (0.0010) [2023-12-27 08:03:26,612][20347] Updated weights for policy 0, policy_version 348827 (0.0010) [2023-12-27 08:03:26,658][20381] Updated weights for policy 1, policy_version 348669 (0.0005) [2023-12-27 08:03:26,708][20381] Updated weights for policy 1, policy_version 348679 (0.0005) [2023-12-27 08:03:26,756][20381] Updated weights for policy 1, policy_version 348689 (0.0005) [2023-12-27 08:03:27,221][20347] Updated weights for policy 0, policy_version 348837 (0.0008) [2023-12-27 08:03:27,276][20347] Updated weights for policy 0, policy_version 348847 (0.0010) [2023-12-27 08:03:27,331][20347] Updated weights for policy 0, policy_version 348857 (0.0010) [2023-12-27 08:03:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 178601984. Throughput: 0: 10333.5, 1: 10382.2. Samples: 178596552. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.560')] [2023-12-27 08:03:27,409][20381] Updated weights for policy 1, policy_version 348699 (0.0005) [2023-12-27 08:03:27,466][20381] Updated weights for policy 1, policy_version 348709 (0.0005) [2023-12-27 08:03:27,521][20381] Updated weights for policy 1, policy_version 348719 (0.0005) [2023-12-27 08:03:28,013][20347] Updated weights for policy 0, policy_version 348867 (0.0009) [2023-12-27 08:03:28,067][20381] Updated weights for policy 1, policy_version 348729 (0.0006) [2023-12-27 08:03:28,069][20347] Updated weights for policy 0, policy_version 348877 (0.0007) [2023-12-27 08:03:28,119][20381] Updated weights for policy 1, policy_version 348739 (0.0011) [2023-12-27 08:03:28,128][20347] Updated weights for policy 0, policy_version 348887 (0.0006) [2023-12-27 08:03:28,173][20381] Updated weights for policy 1, policy_version 348749 (0.0006) [2023-12-27 08:03:28,231][20381] Updated weights for policy 1, policy_version 348759 (0.0008) [2023-12-27 08:03:28,751][20347] Updated weights for policy 0, policy_version 348897 (0.0006) [2023-12-27 08:03:28,799][20347] Updated weights for policy 0, policy_version 348907 (0.0007) [2023-12-27 08:03:28,854][20347] Updated weights for policy 0, policy_version 348917 (0.0008) [2023-12-27 08:03:28,881][20381] Updated weights for policy 1, policy_version 348769 (0.0011) [2023-12-27 08:03:28,912][20347] Updated weights for policy 0, policy_version 348927 (0.0006) [2023-12-27 08:03:28,926][20381] Updated weights for policy 1, policy_version 348779 (0.0010) [2023-12-27 08:03:28,977][20381] Updated weights for policy 1, policy_version 348789 (0.0010) [2023-12-27 08:03:29,612][20347] Updated weights for policy 0, policy_version 348937 (0.0010) [2023-12-27 08:03:29,664][20347] Updated weights for policy 0, policy_version 348947 (0.0010) [2023-12-27 08:03:29,671][20381] Updated weights for policy 1, policy_version 348799 (0.0007) [2023-12-27 08:03:29,718][20347] Updated weights for policy 0, policy_version 348957 (0.0010) [2023-12-27 08:03:29,728][20381] Updated weights for policy 1, policy_version 348809 (0.0008) [2023-12-27 08:03:29,784][20381] Updated weights for policy 1, policy_version 348819 (0.0011) [2023-12-27 08:03:30,402][20347] Updated weights for policy 0, policy_version 348967 (0.0010) [2023-12-27 08:03:30,404][20381] Updated weights for policy 1, policy_version 348829 (0.0011) [2023-12-27 08:03:30,459][20381] Updated weights for policy 1, policy_version 348839 (0.0011) [2023-12-27 08:03:30,460][20347] Updated weights for policy 0, policy_version 348977 (0.0010) [2023-12-27 08:03:30,514][20347] Updated weights for policy 0, policy_version 348987 (0.0010) [2023-12-27 08:03:30,517][20381] Updated weights for policy 1, policy_version 348849 (0.0011) [2023-12-27 08:03:31,121][20381] Updated weights for policy 1, policy_version 348859 (0.0008) [2023-12-27 08:03:31,183][20381] Updated weights for policy 1, policy_version 348869 (0.0008) [2023-12-27 08:03:31,186][20347] Updated weights for policy 0, policy_version 348997 (0.0008) [2023-12-27 08:03:31,234][20381] Updated weights for policy 1, policy_version 348879 (0.0008) [2023-12-27 08:03:31,240][20347] Updated weights for policy 0, policy_version 349007 (0.0006) [2023-12-27 08:03:31,292][20347] Updated weights for policy 0, policy_version 349017 (0.0008) [2023-12-27 08:03:31,965][20347] Updated weights for policy 0, policy_version 349027 (0.0008) [2023-12-27 08:03:32,007][20381] Updated weights for policy 1, policy_version 348889 (0.0008) [2023-12-27 08:03:32,026][20347] Updated weights for policy 0, policy_version 349037 (0.0010) [2023-12-27 08:03:32,064][20381] Updated weights for policy 1, policy_version 348899 (0.0011) [2023-12-27 08:03:32,083][20347] Updated weights for policy 0, policy_version 349047 (0.0010) [2023-12-27 08:03:32,119][20381] Updated weights for policy 1, policy_version 348909 (0.0011) [2023-12-27 08:03:32,170][20381] Updated weights for policy 1, policy_version 348919 (0.0010) [2023-12-27 08:03:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 178716672. Throughput: 0: 10360.5, 1: 10351.3. Samples: 178722444. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:03:32,703][20347] Updated weights for policy 0, policy_version 349057 (0.0010) [2023-12-27 08:03:32,761][20347] Updated weights for policy 0, policy_version 349067 (0.0007) [2023-12-27 08:03:32,816][20347] Updated weights for policy 0, policy_version 349077 (0.0008) [2023-12-27 08:03:32,865][20381] Updated weights for policy 1, policy_version 348929 (0.0011) [2023-12-27 08:03:32,871][20347] Updated weights for policy 0, policy_version 349087 (0.0005) [2023-12-27 08:03:32,919][20381] Updated weights for policy 1, policy_version 348939 (0.0011) [2023-12-27 08:03:32,973][20381] Updated weights for policy 1, policy_version 348949 (0.0010) [2023-12-27 08:03:33,490][20347] Updated weights for policy 0, policy_version 349097 (0.0006) [2023-12-27 08:03:33,546][20347] Updated weights for policy 0, policy_version 349107 (0.0005) [2023-12-27 08:03:33,601][20347] Updated weights for policy 0, policy_version 349117 (0.0006) [2023-12-27 08:03:33,689][20381] Updated weights for policy 1, policy_version 348959 (0.0009) [2023-12-27 08:03:33,734][20381] Updated weights for policy 1, policy_version 348969 (0.0007) [2023-12-27 08:03:33,786][20381] Updated weights for policy 1, policy_version 348979 (0.0009) [2023-12-27 08:03:34,175][20347] Updated weights for policy 0, policy_version 349127 (0.0006) [2023-12-27 08:03:34,231][20347] Updated weights for policy 0, policy_version 349137 (0.0007) [2023-12-27 08:03:34,289][20347] Updated weights for policy 0, policy_version 349147 (0.0006) [2023-12-27 08:03:34,536][20381] Updated weights for policy 1, policy_version 348989 (0.0008) [2023-12-27 08:03:34,592][20381] Updated weights for policy 1, policy_version 348999 (0.0008) [2023-12-27 08:03:34,642][20381] Updated weights for policy 1, policy_version 349009 (0.0007) [2023-12-27 08:03:34,911][20347] Updated weights for policy 0, policy_version 349157 (0.0008) [2023-12-27 08:03:34,963][20347] Updated weights for policy 0, policy_version 349167 (0.0010) [2023-12-27 08:03:35,024][20347] Updated weights for policy 0, policy_version 349177 (0.0010) [2023-12-27 08:03:35,253][20381] Updated weights for policy 1, policy_version 349019 (0.0006) [2023-12-27 08:03:35,309][20381] Updated weights for policy 1, policy_version 349029 (0.0005) [2023-12-27 08:03:35,357][20381] Updated weights for policy 1, policy_version 349039 (0.0005) [2023-12-27 08:03:35,643][20347] Updated weights for policy 0, policy_version 349187 (0.0009) [2023-12-27 08:03:35,696][20347] Updated weights for policy 0, policy_version 349197 (0.0008) [2023-12-27 08:03:35,742][20347] Updated weights for policy 0, policy_version 349207 (0.0008) [2023-12-27 08:03:35,887][20381] Updated weights for policy 1, policy_version 349049 (0.0005) [2023-12-27 08:03:35,933][20381] Updated weights for policy 1, policy_version 349059 (0.0005) [2023-12-27 08:03:35,991][20381] Updated weights for policy 1, policy_version 349069 (0.0005) [2023-12-27 08:03:36,040][20381] Updated weights for policy 1, policy_version 349079 (0.0005) [2023-12-27 08:03:36,516][20347] Updated weights for policy 0, policy_version 349217 (0.0008) [2023-12-27 08:03:36,576][20347] Updated weights for policy 0, policy_version 349227 (0.0008) [2023-12-27 08:03:36,635][20347] Updated weights for policy 0, policy_version 349237 (0.0007) [2023-12-27 08:03:36,671][20381] Updated weights for policy 1, policy_version 349089 (0.0010) [2023-12-27 08:03:36,692][20347] Updated weights for policy 0, policy_version 349247 (0.0006) [2023-12-27 08:03:36,724][20381] Updated weights for policy 1, policy_version 349099 (0.0011) [2023-12-27 08:03:36,780][20381] Updated weights for policy 1, policy_version 349109 (0.0011) [2023-12-27 08:03:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 178814976. Throughput: 0: 10434.1, 1: 10320.9. Samples: 178786428. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.460')] [2023-12-27 08:03:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000349112_89391104.pth... [2023-12-27 08:03:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000347928_89088000.pth [2023-12-27 08:03:37,407][20347] Updated weights for policy 0, policy_version 349257 (0.0007) [2023-12-27 08:03:37,466][20347] Updated weights for policy 0, policy_version 349267 (0.0007) [2023-12-27 08:03:37,498][20381] Updated weights for policy 1, policy_version 349119 (0.0011) [2023-12-27 08:03:37,522][20347] Updated weights for policy 0, policy_version 349277 (0.0006) [2023-12-27 08:03:37,537][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000349280_89432064.pth... [2023-12-27 08:03:37,541][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000348032_89112576.pth [2023-12-27 08:03:37,551][20381] Updated weights for policy 1, policy_version 349129 (0.0011) [2023-12-27 08:03:37,604][20381] Updated weights for policy 1, policy_version 349139 (0.0010) [2023-12-27 08:03:38,145][20347] Updated weights for policy 0, policy_version 349287 (0.0009) [2023-12-27 08:03:38,197][20347] Updated weights for policy 0, policy_version 349297 (0.0010) [2023-12-27 08:03:38,240][20381] Updated weights for policy 1, policy_version 349149 (0.0011) [2023-12-27 08:03:38,248][20347] Updated weights for policy 0, policy_version 349307 (0.0009) [2023-12-27 08:03:38,295][20381] Updated weights for policy 1, policy_version 349159 (0.0010) [2023-12-27 08:03:38,356][20381] Updated weights for policy 1, policy_version 349169 (0.0008) [2023-12-27 08:03:38,862][20347] Updated weights for policy 0, policy_version 349317 (0.0009) [2023-12-27 08:03:38,919][20347] Updated weights for policy 0, policy_version 349327 (0.0007) [2023-12-27 08:03:38,976][20347] Updated weights for policy 0, policy_version 349337 (0.0006) [2023-12-27 08:03:39,058][20381] Updated weights for policy 1, policy_version 349179 (0.0008) [2023-12-27 08:03:39,111][20292] KL-divergence is very high: 102.1054 [2023-12-27 08:03:39,111][20381] Updated weights for policy 1, policy_version 349189 (0.0008) [2023-12-27 08:03:39,162][20381] Updated weights for policy 1, policy_version 349199 (0.0009) [2023-12-27 08:03:39,698][20347] Updated weights for policy 0, policy_version 349347 (0.0008) [2023-12-27 08:03:39,764][20347] Updated weights for policy 0, policy_version 349357 (0.0011) [2023-12-27 08:03:39,830][20347] Updated weights for policy 0, policy_version 349367 (0.0011) [2023-12-27 08:03:39,930][20381] Updated weights for policy 1, policy_version 349209 (0.0008) [2023-12-27 08:03:39,992][20381] Updated weights for policy 1, policy_version 349219 (0.0011) [2023-12-27 08:03:40,053][20381] Updated weights for policy 1, policy_version 349229 (0.0011) [2023-12-27 08:03:40,105][20381] Updated weights for policy 1, policy_version 349239 (0.0009) [2023-12-27 08:03:40,519][20347] Updated weights for policy 0, policy_version 349377 (0.0008) [2023-12-27 08:03:40,571][20347] Updated weights for policy 0, policy_version 349387 (0.0010) [2023-12-27 08:03:40,624][20347] Updated weights for policy 0, policy_version 349397 (0.0010) [2023-12-27 08:03:40,683][20347] Updated weights for policy 0, policy_version 349407 (0.0010) [2023-12-27 08:03:40,730][20381] Updated weights for policy 1, policy_version 349249 (0.0008) [2023-12-27 08:03:40,790][20381] Updated weights for policy 1, policy_version 349259 (0.0007) [2023-12-27 08:03:40,844][20381] Updated weights for policy 1, policy_version 349269 (0.0010) [2023-12-27 08:03:41,384][20347] Updated weights for policy 0, policy_version 349417 (0.0009) [2023-12-27 08:03:41,444][20347] Updated weights for policy 0, policy_version 349427 (0.0007) [2023-12-27 08:03:41,505][20347] Updated weights for policy 0, policy_version 349437 (0.0007) [2023-12-27 08:03:41,545][20381] Updated weights for policy 1, policy_version 349279 (0.0009) [2023-12-27 08:03:41,609][20381] Updated weights for policy 1, policy_version 349289 (0.0008) [2023-12-27 08:03:41,668][20381] Updated weights for policy 1, policy_version 349299 (0.0008) [2023-12-27 08:03:42,154][20347] Updated weights for policy 0, policy_version 349447 (0.0009) [2023-12-27 08:03:42,203][20347] Updated weights for policy 0, policy_version 349457 (0.0007) [2023-12-27 08:03:42,266][20347] Updated weights for policy 0, policy_version 349467 (0.0006) [2023-12-27 08:03:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 178921472. Throughput: 0: 10399.1, 1: 10309.6. Samples: 178911040. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.780')] [2023-12-27 08:03:42,427][20381] Updated weights for policy 1, policy_version 349309 (0.0008) [2023-12-27 08:03:42,479][20381] Updated weights for policy 1, policy_version 349319 (0.0008) [2023-12-27 08:03:42,530][20381] Updated weights for policy 1, policy_version 349329 (0.0008) [2023-12-27 08:03:42,912][20347] Updated weights for policy 0, policy_version 349477 (0.0007) [2023-12-27 08:03:42,956][20347] Updated weights for policy 0, policy_version 349487 (0.0006) [2023-12-27 08:03:43,015][20347] Updated weights for policy 0, policy_version 349497 (0.0007) [2023-12-27 08:03:43,275][20381] Updated weights for policy 1, policy_version 349339 (0.0007) [2023-12-27 08:03:43,339][20381] Updated weights for policy 1, policy_version 349349 (0.0007) [2023-12-27 08:03:43,384][20381] Updated weights for policy 1, policy_version 349359 (0.0008) [2023-12-27 08:03:43,663][20347] Updated weights for policy 0, policy_version 349507 (0.0006) [2023-12-27 08:03:43,710][20347] Updated weights for policy 0, policy_version 349517 (0.0006) [2023-12-27 08:03:43,758][20347] Updated weights for policy 0, policy_version 349527 (0.0007) [2023-12-27 08:03:44,096][20381] Updated weights for policy 1, policy_version 349369 (0.0007) [2023-12-27 08:03:44,155][20381] Updated weights for policy 1, policy_version 349379 (0.0007) [2023-12-27 08:03:44,212][20381] Updated weights for policy 1, policy_version 349389 (0.0006) [2023-12-27 08:03:44,259][20381] Updated weights for policy 1, policy_version 349399 (0.0008) [2023-12-27 08:03:44,412][20347] Updated weights for policy 0, policy_version 349537 (0.0007) [2023-12-27 08:03:44,460][20347] Updated weights for policy 0, policy_version 349547 (0.0007) [2023-12-27 08:03:44,518][20347] Updated weights for policy 0, policy_version 349557 (0.0006) [2023-12-27 08:03:44,576][20347] Updated weights for policy 0, policy_version 349567 (0.0007) [2023-12-27 08:03:44,966][20381] Updated weights for policy 1, policy_version 349409 (0.0009) [2023-12-27 08:03:45,033][20381] Updated weights for policy 1, policy_version 349419 (0.0008) [2023-12-27 08:03:45,093][20381] Updated weights for policy 1, policy_version 349429 (0.0007) [2023-12-27 08:03:45,145][20347] Updated weights for policy 0, policy_version 349577 (0.0007) [2023-12-27 08:03:45,211][20347] Updated weights for policy 0, policy_version 349587 (0.0008) [2023-12-27 08:03:45,267][20347] Updated weights for policy 0, policy_version 349597 (0.0008) [2023-12-27 08:03:45,780][20381] Updated weights for policy 1, policy_version 349439 (0.0005) [2023-12-27 08:03:45,830][20381] Updated weights for policy 1, policy_version 349449 (0.0005) [2023-12-27 08:03:45,885][20381] Updated weights for policy 1, policy_version 349459 (0.0005) [2023-12-27 08:03:45,898][20347] Updated weights for policy 0, policy_version 349607 (0.0007) [2023-12-27 08:03:45,947][20347] Updated weights for policy 0, policy_version 349617 (0.0010) [2023-12-27 08:03:46,000][20347] Updated weights for policy 0, policy_version 349627 (0.0009) [2023-12-27 08:03:46,454][20381] Updated weights for policy 1, policy_version 349469 (0.0006) [2023-12-27 08:03:46,505][20381] Updated weights for policy 1, policy_version 349479 (0.0007) [2023-12-27 08:03:46,564][20381] Updated weights for policy 1, policy_version 349489 (0.0008) [2023-12-27 08:03:46,710][20347] Updated weights for policy 0, policy_version 349637 (0.0006) [2023-12-27 08:03:46,759][20347] Updated weights for policy 0, policy_version 349647 (0.0005) [2023-12-27 08:03:46,815][20347] Updated weights for policy 0, policy_version 349657 (0.0008) [2023-12-27 08:03:47,239][20381] Updated weights for policy 1, policy_version 349499 (0.0008) [2023-12-27 08:03:47,288][20381] Updated weights for policy 1, policy_version 349509 (0.0008) [2023-12-27 08:03:47,336][20381] Updated weights for policy 1, policy_version 349519 (0.0008) [2023-12-27 08:03:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 179019776. Throughput: 0: 10459.8, 1: 10282.3. Samples: 179034684. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.320')] [2023-12-27 08:03:47,468][20347] Updated weights for policy 0, policy_version 349667 (0.0009) [2023-12-27 08:03:47,521][20347] Updated weights for policy 0, policy_version 349677 (0.0008) [2023-12-27 08:03:47,579][20347] Updated weights for policy 0, policy_version 349687 (0.0008) [2023-12-27 08:03:48,053][20381] Updated weights for policy 1, policy_version 349529 (0.0008) [2023-12-27 08:03:48,112][20381] Updated weights for policy 1, policy_version 349539 (0.0008) [2023-12-27 08:03:48,166][20381] Updated weights for policy 1, policy_version 349549 (0.0008) [2023-12-27 08:03:48,225][20381] Updated weights for policy 1, policy_version 349559 (0.0008) [2023-12-27 08:03:48,286][20347] Updated weights for policy 0, policy_version 349697 (0.0008) [2023-12-27 08:03:48,359][20347] Updated weights for policy 0, policy_version 349707 (0.0006) [2023-12-27 08:03:48,419][20347] Updated weights for policy 0, policy_version 349717 (0.0006) [2023-12-27 08:03:48,469][20347] Updated weights for policy 0, policy_version 349727 (0.0008) [2023-12-27 08:03:48,883][20381] Updated weights for policy 1, policy_version 349569 (0.0007) [2023-12-27 08:03:48,936][20381] Updated weights for policy 1, policy_version 349579 (0.0008) [2023-12-27 08:03:48,981][20381] Updated weights for policy 1, policy_version 349589 (0.0007) [2023-12-27 08:03:49,081][20347] Updated weights for policy 0, policy_version 349737 (0.0007) [2023-12-27 08:03:49,129][20347] Updated weights for policy 0, policy_version 349747 (0.0007) [2023-12-27 08:03:49,178][20347] Updated weights for policy 0, policy_version 349757 (0.0008) [2023-12-27 08:03:49,719][20381] Updated weights for policy 1, policy_version 349599 (0.0008) [2023-12-27 08:03:49,781][20381] Updated weights for policy 1, policy_version 349609 (0.0008) [2023-12-27 08:03:49,843][20381] Updated weights for policy 1, policy_version 349619 (0.0007) [2023-12-27 08:03:49,903][20347] Updated weights for policy 0, policy_version 349767 (0.0007) [2023-12-27 08:03:49,973][20347] Updated weights for policy 0, policy_version 349777 (0.0008) [2023-12-27 08:03:50,032][20347] Updated weights for policy 0, policy_version 349787 (0.0007) [2023-12-27 08:03:50,542][20381] Updated weights for policy 1, policy_version 349629 (0.0007) [2023-12-27 08:03:50,607][20381] Updated weights for policy 1, policy_version 349639 (0.0007) [2023-12-27 08:03:50,662][20381] Updated weights for policy 1, policy_version 349649 (0.0007) [2023-12-27 08:03:50,694][20347] Updated weights for policy 0, policy_version 349797 (0.0006) [2023-12-27 08:03:50,749][20347] Updated weights for policy 0, policy_version 349807 (0.0007) [2023-12-27 08:03:50,804][20347] Updated weights for policy 0, policy_version 349817 (0.0007) [2023-12-27 08:03:51,340][20381] Updated weights for policy 1, policy_version 349659 (0.0008) [2023-12-27 08:03:51,411][20381] Updated weights for policy 1, policy_version 349669 (0.0009) [2023-12-27 08:03:51,467][20347] Updated weights for policy 0, policy_version 349827 (0.0007) [2023-12-27 08:03:51,472][20381] Updated weights for policy 1, policy_version 349679 (0.0010) [2023-12-27 08:03:51,512][20347] Updated weights for policy 0, policy_version 349837 (0.0007) [2023-12-27 08:03:51,562][20347] Updated weights for policy 0, policy_version 349847 (0.0007) [2023-12-27 08:03:52,131][20381] Updated weights for policy 1, policy_version 349689 (0.0010) [2023-12-27 08:03:52,189][20381] Updated weights for policy 1, policy_version 349699 (0.0006) [2023-12-27 08:03:52,247][20381] Updated weights for policy 1, policy_version 349709 (0.0007) [2023-12-27 08:03:52,305][20381] Updated weights for policy 1, policy_version 349719 (0.0010) [2023-12-27 08:03:52,318][20347] Updated weights for policy 0, policy_version 349857 (0.0008) [2023-12-27 08:03:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 179126272. Throughput: 0: 10462.2, 1: 10247.0. Samples: 179094728. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 08:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:03:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000349720_89546752.pth... [2023-12-27 08:03:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000348504_89235456.pth [2023-12-27 08:03:52,384][20347] Updated weights for policy 0, policy_version 349867 (0.0006) [2023-12-27 08:03:52,443][20347] Updated weights for policy 0, policy_version 349877 (0.0005) [2023-12-27 08:03:52,499][20347] Updated weights for policy 0, policy_version 349887 (0.0005) [2023-12-27 08:03:52,507][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000349888_89587712.pth... [2023-12-27 08:03:52,511][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000348640_89268224.pth [2023-12-27 08:03:52,987][20381] Updated weights for policy 1, policy_version 349729 (0.0008) [2023-12-27 08:03:53,036][20381] Updated weights for policy 1, policy_version 349739 (0.0007) [2023-12-27 08:03:53,093][20381] Updated weights for policy 1, policy_version 349749 (0.0007) [2023-12-27 08:03:53,162][20347] Updated weights for policy 0, policy_version 349897 (0.0008) [2023-12-27 08:03:53,211][20347] Updated weights for policy 0, policy_version 349907 (0.0006) [2023-12-27 08:03:53,257][20347] Updated weights for policy 0, policy_version 349917 (0.0007) [2023-12-27 08:03:53,764][20381] Updated weights for policy 1, policy_version 349759 (0.0006) [2023-12-27 08:03:53,830][20381] Updated weights for policy 1, policy_version 349769 (0.0005) [2023-12-27 08:03:53,888][20381] Updated weights for policy 1, policy_version 349779 (0.0008) [2023-12-27 08:03:54,004][20347] Updated weights for policy 0, policy_version 349927 (0.0008) [2023-12-27 08:03:54,058][20347] Updated weights for policy 0, policy_version 349937 (0.0008) [2023-12-27 08:03:54,120][20347] Updated weights for policy 0, policy_version 349947 (0.0008) [2023-12-27 08:03:54,528][20381] Updated weights for policy 1, policy_version 349789 (0.0008) [2023-12-27 08:03:54,583][20381] Updated weights for policy 1, policy_version 349799 (0.0006) [2023-12-27 08:03:54,641][20381] Updated weights for policy 1, policy_version 349809 (0.0007) [2023-12-27 08:03:54,779][20347] Updated weights for policy 0, policy_version 349957 (0.0007) [2023-12-27 08:03:54,832][20347] Updated weights for policy 0, policy_version 349967 (0.0008) [2023-12-27 08:03:54,885][20347] Updated weights for policy 0, policy_version 349977 (0.0007) [2023-12-27 08:03:55,309][20381] Updated weights for policy 1, policy_version 349819 (0.0007) [2023-12-27 08:03:55,364][20381] Updated weights for policy 1, policy_version 349829 (0.0007) [2023-12-27 08:03:55,422][20381] Updated weights for policy 1, policy_version 349839 (0.0008) [2023-12-27 08:03:55,555][20347] Updated weights for policy 0, policy_version 349987 (0.0008) [2023-12-27 08:03:55,605][20347] Updated weights for policy 0, policy_version 349997 (0.0008) [2023-12-27 08:03:55,655][20347] Updated weights for policy 0, policy_version 350007 (0.0007) [2023-12-27 08:03:56,096][20381] Updated weights for policy 1, policy_version 349849 (0.0007) [2023-12-27 08:03:56,144][20381] Updated weights for policy 1, policy_version 349859 (0.0007) [2023-12-27 08:03:56,202][20381] Updated weights for policy 1, policy_version 349869 (0.0006) [2023-12-27 08:03:56,260][20381] Updated weights for policy 1, policy_version 349879 (0.0006) [2023-12-27 08:03:56,367][20347] Updated weights for policy 0, policy_version 350017 (0.0007) [2023-12-27 08:03:56,427][20347] Updated weights for policy 0, policy_version 350027 (0.0007) [2023-12-27 08:03:56,484][20347] Updated weights for policy 0, policy_version 350037 (0.0007) [2023-12-27 08:03:56,543][20347] Updated weights for policy 0, policy_version 350047 (0.0008) [2023-12-27 08:03:56,934][20381] Updated weights for policy 1, policy_version 349889 (0.0007) [2023-12-27 08:03:56,986][20381] Updated weights for policy 1, policy_version 349899 (0.0005) [2023-12-27 08:03:57,044][20381] Updated weights for policy 1, policy_version 349909 (0.0005) [2023-12-27 08:03:57,193][20347] Updated weights for policy 0, policy_version 350057 (0.0005) [2023-12-27 08:03:57,245][20347] Updated weights for policy 0, policy_version 350067 (0.0007) [2023-12-27 08:03:57,302][20347] Updated weights for policy 0, policy_version 350077 (0.0006) [2023-12-27 08:03:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 179232768. Throughput: 0: 10418.8, 1: 10266.3. Samples: 179217432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:03:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:03:57,567][20381] Updated weights for policy 1, policy_version 349919 (0.0005) [2023-12-27 08:03:57,612][20381] Updated weights for policy 1, policy_version 349929 (0.0006) [2023-12-27 08:03:57,668][20381] Updated weights for policy 1, policy_version 349939 (0.0007) [2023-12-27 08:03:57,926][20347] Updated weights for policy 0, policy_version 350087 (0.0007) [2023-12-27 08:03:57,970][20347] Updated weights for policy 0, policy_version 350097 (0.0007) [2023-12-27 08:03:58,025][20347] Updated weights for policy 0, policy_version 350107 (0.0007) [2023-12-27 08:03:58,335][20381] Updated weights for policy 1, policy_version 349949 (0.0008) [2023-12-27 08:03:58,401][20381] Updated weights for policy 1, policy_version 349959 (0.0009) [2023-12-27 08:03:58,466][20381] Updated weights for policy 1, policy_version 349969 (0.0008) [2023-12-27 08:03:58,816][20347] Updated weights for policy 0, policy_version 350117 (0.0008) [2023-12-27 08:03:58,878][20347] Updated weights for policy 0, policy_version 350127 (0.0008) [2023-12-27 08:03:58,942][20347] Updated weights for policy 0, policy_version 350137 (0.0008) [2023-12-27 08:03:59,224][20381] Updated weights for policy 1, policy_version 349979 (0.0008) [2023-12-27 08:03:59,282][20381] Updated weights for policy 1, policy_version 349989 (0.0006) [2023-12-27 08:03:59,344][20381] Updated weights for policy 1, policy_version 349999 (0.0006) [2023-12-27 08:03:59,565][20347] Updated weights for policy 0, policy_version 350147 (0.0006) [2023-12-27 08:03:59,625][20347] Updated weights for policy 0, policy_version 350157 (0.0006) [2023-12-27 08:03:59,678][20347] Updated weights for policy 0, policy_version 350167 (0.0006) [2023-12-27 08:03:59,919][20381] Updated weights for policy 1, policy_version 350009 (0.0006) [2023-12-27 08:03:59,973][20381] Updated weights for policy 1, policy_version 350019 (0.0007) [2023-12-27 08:04:00,020][20381] Updated weights for policy 1, policy_version 350029 (0.0007) [2023-12-27 08:04:00,081][20381] Updated weights for policy 1, policy_version 350039 (0.0008) [2023-12-27 08:04:00,371][20347] Updated weights for policy 0, policy_version 350177 (0.0009) [2023-12-27 08:04:00,434][20347] Updated weights for policy 0, policy_version 350187 (0.0008) [2023-12-27 08:04:00,487][20347] Updated weights for policy 0, policy_version 350197 (0.0008) [2023-12-27 08:04:00,542][20347] Updated weights for policy 0, policy_version 350207 (0.0008) [2023-12-27 08:04:00,756][20381] Updated weights for policy 1, policy_version 350049 (0.0006) [2023-12-27 08:04:00,814][20381] Updated weights for policy 1, policy_version 350059 (0.0007) [2023-12-27 08:04:00,875][20381] Updated weights for policy 1, policy_version 350069 (0.0007) [2023-12-27 08:04:01,221][20347] Updated weights for policy 0, policy_version 350217 (0.0008) [2023-12-27 08:04:01,279][20347] Updated weights for policy 0, policy_version 350227 (0.0007) [2023-12-27 08:04:01,335][20347] Updated weights for policy 0, policy_version 350237 (0.0007) [2023-12-27 08:04:01,530][20381] Updated weights for policy 1, policy_version 350079 (0.0006) [2023-12-27 08:04:01,583][20381] Updated weights for policy 1, policy_version 350089 (0.0007) [2023-12-27 08:04:01,641][20381] Updated weights for policy 1, policy_version 350099 (0.0007) [2023-12-27 08:04:02,046][20347] Updated weights for policy 0, policy_version 350247 (0.0006) [2023-12-27 08:04:02,091][20347] Updated weights for policy 0, policy_version 350257 (0.0008) [2023-12-27 08:04:02,156][20347] Updated weights for policy 0, policy_version 350267 (0.0008) [2023-12-27 08:04:02,242][20381] Updated weights for policy 1, policy_version 350109 (0.0007) [2023-12-27 08:04:02,298][20381] Updated weights for policy 1, policy_version 350119 (0.0008) [2023-12-27 08:04:02,361][20381] Updated weights for policy 1, policy_version 350129 (0.0007) [2023-12-27 08:04:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 179331072. Throughput: 0: 10373.6, 1: 10323.2. Samples: 179343420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:02,785][20347] Updated weights for policy 0, policy_version 350277 (0.0009) [2023-12-27 08:04:02,834][20347] Updated weights for policy 0, policy_version 350287 (0.0008) [2023-12-27 08:04:02,892][20347] Updated weights for policy 0, policy_version 350297 (0.0006) [2023-12-27 08:04:02,975][20381] Updated weights for policy 1, policy_version 350139 (0.0007) [2023-12-27 08:04:03,031][20381] Updated weights for policy 1, policy_version 350149 (0.0005) [2023-12-27 08:04:03,082][20381] Updated weights for policy 1, policy_version 350159 (0.0005) [2023-12-27 08:04:03,445][20347] Updated weights for policy 0, policy_version 350307 (0.0005) [2023-12-27 08:04:03,499][20347] Updated weights for policy 0, policy_version 350317 (0.0005) [2023-12-27 08:04:03,549][20347] Updated weights for policy 0, policy_version 350327 (0.0005) [2023-12-27 08:04:03,597][20381] Updated weights for policy 1, policy_version 350169 (0.0005) [2023-12-27 08:04:03,643][20381] Updated weights for policy 1, policy_version 350179 (0.0009) [2023-12-27 08:04:03,687][20381] Updated weights for policy 1, policy_version 350189 (0.0009) [2023-12-27 08:04:03,736][20381] Updated weights for policy 1, policy_version 350199 (0.0010) [2023-12-27 08:04:04,110][20347] Updated weights for policy 0, policy_version 350337 (0.0005) [2023-12-27 08:04:04,173][20347] Updated weights for policy 0, policy_version 350347 (0.0005) [2023-12-27 08:04:04,221][20347] Updated weights for policy 0, policy_version 350357 (0.0005) [2023-12-27 08:04:04,271][20347] Updated weights for policy 0, policy_version 350367 (0.0006) [2023-12-27 08:04:04,469][20381] Updated weights for policy 1, policy_version 350209 (0.0011) [2023-12-27 08:04:04,530][20381] Updated weights for policy 1, policy_version 350219 (0.0011) [2023-12-27 08:04:04,590][20381] Updated weights for policy 1, policy_version 350229 (0.0011) [2023-12-27 08:04:04,880][20347] Updated weights for policy 0, policy_version 350377 (0.0007) [2023-12-27 08:04:04,939][20347] Updated weights for policy 0, policy_version 350387 (0.0009) [2023-12-27 08:04:04,987][20347] Updated weights for policy 0, policy_version 350397 (0.0010) [2023-12-27 08:04:05,211][20381] Updated weights for policy 1, policy_version 350239 (0.0007) [2023-12-27 08:04:05,268][20381] Updated weights for policy 1, policy_version 350249 (0.0005) [2023-12-27 08:04:05,317][20381] Updated weights for policy 1, policy_version 350259 (0.0005) [2023-12-27 08:04:05,635][20347] Updated weights for policy 0, policy_version 350407 (0.0008) [2023-12-27 08:04:05,695][20347] Updated weights for policy 0, policy_version 350417 (0.0006) [2023-12-27 08:04:05,756][20347] Updated weights for policy 0, policy_version 350427 (0.0005) [2023-12-27 08:04:05,922][20381] Updated weights for policy 1, policy_version 350269 (0.0008) [2023-12-27 08:04:05,980][20381] Updated weights for policy 1, policy_version 350279 (0.0011) [2023-12-27 08:04:06,028][20381] Updated weights for policy 1, policy_version 350289 (0.0010) [2023-12-27 08:04:06,410][20347] Updated weights for policy 0, policy_version 350437 (0.0009) [2023-12-27 08:04:06,470][20347] Updated weights for policy 0, policy_version 350447 (0.0007) [2023-12-27 08:04:06,524][20347] Updated weights for policy 0, policy_version 350457 (0.0009) [2023-12-27 08:04:06,643][20381] Updated weights for policy 1, policy_version 350299 (0.0009) [2023-12-27 08:04:06,700][20381] Updated weights for policy 1, policy_version 350309 (0.0005) [2023-12-27 08:04:06,757][20381] Updated weights for policy 1, policy_version 350319 (0.0006) [2023-12-27 08:04:07,279][20347] Updated weights for policy 0, policy_version 350467 (0.0009) [2023-12-27 08:04:07,320][20381] Updated weights for policy 1, policy_version 350329 (0.0006) [2023-12-27 08:04:07,342][20347] Updated weights for policy 0, policy_version 350477 (0.0007) [2023-12-27 08:04:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 179437568. Throughput: 0: 10423.6, 1: 10389.7. Samples: 179411256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:07,383][20381] Updated weights for policy 1, policy_version 350339 (0.0007) [2023-12-27 08:04:07,401][20347] Updated weights for policy 0, policy_version 350487 (0.0007) [2023-12-27 08:04:07,440][20381] Updated weights for policy 1, policy_version 350349 (0.0008) [2023-12-27 08:04:07,452][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000350496_89743360.pth... [2023-12-27 08:04:07,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000349280_89432064.pth [2023-12-27 08:04:07,499][20381] Updated weights for policy 1, policy_version 350359 (0.0007) [2023-12-27 08:04:07,501][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000350360_89710592.pth... [2023-12-27 08:04:07,505][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000349112_89391104.pth [2023-12-27 08:04:07,969][20347] Updated weights for policy 0, policy_version 350497 (0.0005) [2023-12-27 08:04:08,027][20347] Updated weights for policy 0, policy_version 350507 (0.0006) [2023-12-27 08:04:08,076][20347] Updated weights for policy 0, policy_version 350517 (0.0006) [2023-12-27 08:04:08,097][20381] Updated weights for policy 1, policy_version 350369 (0.0005) [2023-12-27 08:04:08,128][20347] Updated weights for policy 0, policy_version 350527 (0.0006) [2023-12-27 08:04:08,152][20381] Updated weights for policy 1, policy_version 350379 (0.0007) [2023-12-27 08:04:08,210][20381] Updated weights for policy 1, policy_version 350389 (0.0008) [2023-12-27 08:04:08,793][20347] Updated weights for policy 0, policy_version 350537 (0.0008) [2023-12-27 08:04:08,849][20381] Updated weights for policy 1, policy_version 350399 (0.0006) [2023-12-27 08:04:08,851][20347] Updated weights for policy 0, policy_version 350547 (0.0006) [2023-12-27 08:04:08,900][20381] Updated weights for policy 1, policy_version 350409 (0.0008) [2023-12-27 08:04:08,902][20347] Updated weights for policy 0, policy_version 350557 (0.0006) [2023-12-27 08:04:08,962][20381] Updated weights for policy 1, policy_version 350419 (0.0007) [2023-12-27 08:04:09,586][20347] Updated weights for policy 0, policy_version 350567 (0.0008) [2023-12-27 08:04:09,644][20347] Updated weights for policy 0, policy_version 350577 (0.0007) [2023-12-27 08:04:09,654][20381] Updated weights for policy 1, policy_version 350429 (0.0007) [2023-12-27 08:04:09,703][20347] Updated weights for policy 0, policy_version 350587 (0.0007) [2023-12-27 08:04:09,715][20381] Updated weights for policy 1, policy_version 350439 (0.0009) [2023-12-27 08:04:09,767][20381] Updated weights for policy 1, policy_version 350449 (0.0006) [2023-12-27 08:04:10,411][20347] Updated weights for policy 0, policy_version 350597 (0.0006) [2023-12-27 08:04:10,456][20381] Updated weights for policy 1, policy_version 350459 (0.0008) [2023-12-27 08:04:10,471][20347] Updated weights for policy 0, policy_version 350607 (0.0007) [2023-12-27 08:04:10,506][20381] Updated weights for policy 1, policy_version 350469 (0.0007) [2023-12-27 08:04:10,529][20347] Updated weights for policy 0, policy_version 350617 (0.0006) [2023-12-27 08:04:10,557][20381] Updated weights for policy 1, policy_version 350479 (0.0008) [2023-12-27 08:04:11,220][20347] Updated weights for policy 0, policy_version 350627 (0.0006) [2023-12-27 08:04:11,267][20381] Updated weights for policy 1, policy_version 350489 (0.0006) [2023-12-27 08:04:11,291][20347] Updated weights for policy 0, policy_version 350637 (0.0007) [2023-12-27 08:04:11,332][20381] Updated weights for policy 1, policy_version 350499 (0.0006) [2023-12-27 08:04:11,356][20347] Updated weights for policy 0, policy_version 350647 (0.0006) [2023-12-27 08:04:11,402][20381] Updated weights for policy 1, policy_version 350509 (0.0008) [2023-12-27 08:04:11,465][20381] Updated weights for policy 1, policy_version 350519 (0.0006) [2023-12-27 08:04:12,032][20347] Updated weights for policy 0, policy_version 350657 (0.0007) [2023-12-27 08:04:12,093][20347] Updated weights for policy 0, policy_version 350667 (0.0006) [2023-12-27 08:04:12,152][20347] Updated weights for policy 0, policy_version 350677 (0.0008) [2023-12-27 08:04:12,162][20381] Updated weights for policy 1, policy_version 350529 (0.0008) [2023-12-27 08:04:12,206][20347] Updated weights for policy 0, policy_version 350687 (0.0007) [2023-12-27 08:04:12,222][20381] Updated weights for policy 1, policy_version 350539 (0.0005) [2023-12-27 08:04:12,285][20381] Updated weights for policy 1, policy_version 350549 (0.0007) [2023-12-27 08:04:12,362][19170] Fps is (10 sec: 22118.4, 60 sec: 21026.2, 300 sec: 20716.0). Total num frames: 179552256. Throughput: 0: 10432.5, 1: 10474.2. Samples: 179537356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:12,870][20347] Updated weights for policy 0, policy_version 350697 (0.0007) [2023-12-27 08:04:12,926][20347] Updated weights for policy 0, policy_version 350707 (0.0006) [2023-12-27 08:04:12,929][20381] Updated weights for policy 1, policy_version 350559 (0.0005) [2023-12-27 08:04:12,984][20347] Updated weights for policy 0, policy_version 350717 (0.0008) [2023-12-27 08:04:12,986][20381] Updated weights for policy 1, policy_version 350569 (0.0007) [2023-12-27 08:04:13,044][20381] Updated weights for policy 1, policy_version 350579 (0.0008) [2023-12-27 08:04:13,658][20347] Updated weights for policy 0, policy_version 350727 (0.0008) [2023-12-27 08:04:13,706][20347] Updated weights for policy 0, policy_version 350737 (0.0008) [2023-12-27 08:04:13,732][20381] Updated weights for policy 1, policy_version 350589 (0.0007) [2023-12-27 08:04:13,751][20347] Updated weights for policy 0, policy_version 350747 (0.0008) [2023-12-27 08:04:13,785][20381] Updated weights for policy 1, policy_version 350599 (0.0005) [2023-12-27 08:04:13,848][20381] Updated weights for policy 1, policy_version 350609 (0.0005) [2023-12-27 08:04:14,443][20381] Updated weights for policy 1, policy_version 350619 (0.0007) [2023-12-27 08:04:14,483][20347] Updated weights for policy 0, policy_version 350757 (0.0007) [2023-12-27 08:04:14,492][20381] Updated weights for policy 1, policy_version 350629 (0.0005) [2023-12-27 08:04:14,537][20347] Updated weights for policy 0, policy_version 350767 (0.0008) [2023-12-27 08:04:14,541][20381] Updated weights for policy 1, policy_version 350639 (0.0008) [2023-12-27 08:04:14,593][20347] Updated weights for policy 0, policy_version 350777 (0.0006) [2023-12-27 08:04:15,117][20381] Updated weights for policy 1, policy_version 350649 (0.0007) [2023-12-27 08:04:15,176][20381] Updated weights for policy 1, policy_version 350659 (0.0006) [2023-12-27 08:04:15,229][20381] Updated weights for policy 1, policy_version 350669 (0.0008) [2023-12-27 08:04:15,275][20381] Updated weights for policy 1, policy_version 350679 (0.0008) [2023-12-27 08:04:15,333][20347] Updated weights for policy 0, policy_version 350787 (0.0008) [2023-12-27 08:04:15,391][20347] Updated weights for policy 0, policy_version 350797 (0.0007) [2023-12-27 08:04:15,453][20347] Updated weights for policy 0, policy_version 350807 (0.0008) [2023-12-27 08:04:15,958][20381] Updated weights for policy 1, policy_version 350689 (0.0005) [2023-12-27 08:04:16,011][20381] Updated weights for policy 1, policy_version 350699 (0.0006) [2023-12-27 08:04:16,072][20381] Updated weights for policy 1, policy_version 350709 (0.0006) [2023-12-27 08:04:16,158][20347] Updated weights for policy 0, policy_version 350817 (0.0007) [2023-12-27 08:04:16,208][20347] Updated weights for policy 0, policy_version 350827 (0.0007) [2023-12-27 08:04:16,252][20347] Updated weights for policy 0, policy_version 350837 (0.0009) [2023-12-27 08:04:16,310][20347] Updated weights for policy 0, policy_version 350847 (0.0008) [2023-12-27 08:04:16,690][20381] Updated weights for policy 1, policy_version 350719 (0.0006) [2023-12-27 08:04:16,754][20381] Updated weights for policy 1, policy_version 350729 (0.0007) [2023-12-27 08:04:16,811][20381] Updated weights for policy 1, policy_version 350739 (0.0008) [2023-12-27 08:04:17,042][20347] Updated weights for policy 0, policy_version 350857 (0.0007) [2023-12-27 08:04:17,098][20347] Updated weights for policy 0, policy_version 350867 (0.0007) [2023-12-27 08:04:17,154][20347] Updated weights for policy 0, policy_version 350877 (0.0008) [2023-12-27 08:04:17,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 179650560. Throughput: 0: 10364.3, 1: 10465.6. Samples: 179659796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:17,503][20381] Updated weights for policy 1, policy_version 350749 (0.0008) [2023-12-27 08:04:17,554][20381] Updated weights for policy 1, policy_version 350759 (0.0006) [2023-12-27 08:04:17,608][20381] Updated weights for policy 1, policy_version 350769 (0.0005) [2023-12-27 08:04:17,872][20347] Updated weights for policy 0, policy_version 350887 (0.0007) [2023-12-27 08:04:17,917][20347] Updated weights for policy 0, policy_version 350897 (0.0009) [2023-12-27 08:04:17,976][20347] Updated weights for policy 0, policy_version 350907 (0.0008) [2023-12-27 08:04:18,158][20381] Updated weights for policy 1, policy_version 350779 (0.0006) [2023-12-27 08:04:18,213][20381] Updated weights for policy 1, policy_version 350789 (0.0006) [2023-12-27 08:04:18,262][20381] Updated weights for policy 1, policy_version 350799 (0.0005) [2023-12-27 08:04:18,677][20347] Updated weights for policy 0, policy_version 350917 (0.0007) [2023-12-27 08:04:18,742][20347] Updated weights for policy 0, policy_version 350927 (0.0008) [2023-12-27 08:04:18,804][20347] Updated weights for policy 0, policy_version 350937 (0.0008) [2023-12-27 08:04:18,895][20381] Updated weights for policy 1, policy_version 350809 (0.0007) [2023-12-27 08:04:18,954][20381] Updated weights for policy 1, policy_version 350819 (0.0007) [2023-12-27 08:04:19,013][20381] Updated weights for policy 1, policy_version 350829 (0.0007) [2023-12-27 08:04:19,070][20381] Updated weights for policy 1, policy_version 350839 (0.0006) [2023-12-27 08:04:19,503][20347] Updated weights for policy 0, policy_version 350947 (0.0008) [2023-12-27 08:04:19,558][20347] Updated weights for policy 0, policy_version 350957 (0.0009) [2023-12-27 08:04:19,617][20347] Updated weights for policy 0, policy_version 350967 (0.0007) [2023-12-27 08:04:19,800][20381] Updated weights for policy 1, policy_version 350849 (0.0010) [2023-12-27 08:04:19,871][20381] Updated weights for policy 1, policy_version 350859 (0.0011) [2023-12-27 08:04:19,935][20381] Updated weights for policy 1, policy_version 350869 (0.0010) [2023-12-27 08:04:20,292][20347] Updated weights for policy 0, policy_version 350977 (0.0006) [2023-12-27 08:04:20,351][20347] Updated weights for policy 0, policy_version 350987 (0.0007) [2023-12-27 08:04:20,407][20347] Updated weights for policy 0, policy_version 350997 (0.0008) [2023-12-27 08:04:20,462][20347] Updated weights for policy 0, policy_version 351007 (0.0007) [2023-12-27 08:04:20,637][20381] Updated weights for policy 1, policy_version 350879 (0.0011) [2023-12-27 08:04:20,700][20381] Updated weights for policy 1, policy_version 350889 (0.0011) [2023-12-27 08:04:20,763][20381] Updated weights for policy 1, policy_version 350899 (0.0010) [2023-12-27 08:04:21,161][20347] Updated weights for policy 0, policy_version 351017 (0.0007) [2023-12-27 08:04:21,217][20347] Updated weights for policy 0, policy_version 351027 (0.0008) [2023-12-27 08:04:21,279][20347] Updated weights for policy 0, policy_version 351037 (0.0008) [2023-12-27 08:04:21,484][20381] Updated weights for policy 1, policy_version 350909 (0.0009) [2023-12-27 08:04:21,540][20381] Updated weights for policy 1, policy_version 350919 (0.0010) [2023-12-27 08:04:21,600][20381] Updated weights for policy 1, policy_version 350929 (0.0010) [2023-12-27 08:04:21,994][20347] Updated weights for policy 0, policy_version 351047 (0.0009) [2023-12-27 08:04:22,054][20347] Updated weights for policy 0, policy_version 351057 (0.0006) [2023-12-27 08:04:22,116][20347] Updated weights for policy 0, policy_version 351067 (0.0006) [2023-12-27 08:04:22,254][20381] Updated weights for policy 1, policy_version 350939 (0.0009) [2023-12-27 08:04:22,314][20381] Updated weights for policy 1, policy_version 350949 (0.0007) [2023-12-27 08:04:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 179748864. Throughput: 0: 10284.8, 1: 10486.0. Samples: 179721112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000351072_89890816.pth... [2023-12-27 08:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000349888_89587712.pth [2023-12-27 08:04:22,379][20381] Updated weights for policy 1, policy_version 350959 (0.0008) [2023-12-27 08:04:22,433][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000350968_89866240.pth... [2023-12-27 08:04:22,437][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000349720_89546752.pth [2023-12-27 08:04:22,755][20347] Updated weights for policy 0, policy_version 351077 (0.0007) [2023-12-27 08:04:22,806][20347] Updated weights for policy 0, policy_version 351087 (0.0007) [2023-12-27 08:04:22,868][20347] Updated weights for policy 0, policy_version 351097 (0.0008) [2023-12-27 08:04:23,067][20381] Updated weights for policy 1, policy_version 350969 (0.0008) [2023-12-27 08:04:23,112][20381] Updated weights for policy 1, policy_version 350979 (0.0005) [2023-12-27 08:04:23,158][20381] Updated weights for policy 1, policy_version 350989 (0.0006) [2023-12-27 08:04:23,219][20381] Updated weights for policy 1, policy_version 350999 (0.0010) [2023-12-27 08:04:23,532][20347] Updated weights for policy 0, policy_version 351107 (0.0007) [2023-12-27 08:04:23,588][20347] Updated weights for policy 0, policy_version 351117 (0.0005) [2023-12-27 08:04:23,641][20347] Updated weights for policy 0, policy_version 351127 (0.0006) [2023-12-27 08:04:23,877][20381] Updated weights for policy 1, policy_version 351009 (0.0010) [2023-12-27 08:04:23,936][20381] Updated weights for policy 1, policy_version 351019 (0.0010) [2023-12-27 08:04:23,993][20381] Updated weights for policy 1, policy_version 351029 (0.0010) [2023-12-27 08:04:24,284][20347] Updated weights for policy 0, policy_version 351137 (0.0007) [2023-12-27 08:04:24,332][20347] Updated weights for policy 0, policy_version 351147 (0.0007) [2023-12-27 08:04:24,388][20347] Updated weights for policy 0, policy_version 351157 (0.0007) [2023-12-27 08:04:24,442][20347] Updated weights for policy 0, policy_version 351167 (0.0007) [2023-12-27 08:04:24,669][20381] Updated weights for policy 1, policy_version 351039 (0.0011) [2023-12-27 08:04:24,723][20381] Updated weights for policy 1, policy_version 351049 (0.0010) [2023-12-27 08:04:24,770][20381] Updated weights for policy 1, policy_version 351059 (0.0010) [2023-12-27 08:04:25,158][20347] Updated weights for policy 0, policy_version 351177 (0.0006) [2023-12-27 08:04:25,219][20347] Updated weights for policy 0, policy_version 351187 (0.0005) [2023-12-27 08:04:25,270][20347] Updated weights for policy 0, policy_version 351197 (0.0005) [2023-12-27 08:04:25,453][20381] Updated weights for policy 1, policy_version 351069 (0.0009) [2023-12-27 08:04:25,510][20381] Updated weights for policy 1, policy_version 351079 (0.0007) [2023-12-27 08:04:25,575][20381] Updated weights for policy 1, policy_version 351089 (0.0007) [2023-12-27 08:04:25,840][20347] Updated weights for policy 0, policy_version 351207 (0.0007) [2023-12-27 08:04:25,889][20347] Updated weights for policy 0, policy_version 351217 (0.0010) [2023-12-27 08:04:25,947][20347] Updated weights for policy 0, policy_version 351227 (0.0010) [2023-12-27 08:04:26,304][20381] Updated weights for policy 1, policy_version 351099 (0.0007) [2023-12-27 08:04:26,359][20381] Updated weights for policy 1, policy_version 351109 (0.0005) [2023-12-27 08:04:26,411][20381] Updated weights for policy 1, policy_version 351119 (0.0005) [2023-12-27 08:04:26,565][20347] Updated weights for policy 0, policy_version 351237 (0.0011) [2023-12-27 08:04:26,618][20347] Updated weights for policy 0, policy_version 351247 (0.0010) [2023-12-27 08:04:26,674][20347] Updated weights for policy 0, policy_version 351257 (0.0010) [2023-12-27 08:04:27,039][20381] Updated weights for policy 1, policy_version 351129 (0.0007) [2023-12-27 08:04:27,098][20381] Updated weights for policy 1, policy_version 351139 (0.0006) [2023-12-27 08:04:27,148][20381] Updated weights for policy 1, policy_version 351149 (0.0007) [2023-12-27 08:04:27,195][20381] Updated weights for policy 1, policy_version 351159 (0.0007) [2023-12-27 08:04:27,349][20347] Updated weights for policy 0, policy_version 351267 (0.0010) [2023-12-27 08:04:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 179855360. Throughput: 0: 10302.4, 1: 10420.1. Samples: 179843552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:27,403][20347] Updated weights for policy 0, policy_version 351277 (0.0010) [2023-12-27 08:04:27,447][20347] Updated weights for policy 0, policy_version 351287 (0.0010) [2023-12-27 08:04:27,883][20381] Updated weights for policy 1, policy_version 351169 (0.0008) [2023-12-27 08:04:27,936][20381] Updated weights for policy 1, policy_version 351179 (0.0007) [2023-12-27 08:04:27,990][20381] Updated weights for policy 1, policy_version 351189 (0.0007) [2023-12-27 08:04:28,116][20347] Updated weights for policy 0, policy_version 351297 (0.0009) [2023-12-27 08:04:28,164][20347] Updated weights for policy 0, policy_version 351307 (0.0010) [2023-12-27 08:04:28,208][20347] Updated weights for policy 0, policy_version 351317 (0.0010) [2023-12-27 08:04:28,264][20347] Updated weights for policy 0, policy_version 351327 (0.0010) [2023-12-27 08:04:28,710][20381] Updated weights for policy 1, policy_version 351199 (0.0007) [2023-12-27 08:04:28,755][20381] Updated weights for policy 1, policy_version 351209 (0.0010) [2023-12-27 08:04:28,802][20381] Updated weights for policy 1, policy_version 351219 (0.0008) [2023-12-27 08:04:28,882][20347] Updated weights for policy 0, policy_version 351337 (0.0010) [2023-12-27 08:04:28,928][20347] Updated weights for policy 0, policy_version 351347 (0.0010) [2023-12-27 08:04:28,972][20347] Updated weights for policy 0, policy_version 351357 (0.0010) [2023-12-27 08:04:29,559][20381] Updated weights for policy 1, policy_version 351229 (0.0008) [2023-12-27 08:04:29,615][20381] Updated weights for policy 1, policy_version 351239 (0.0008) [2023-12-27 08:04:29,666][20381] Updated weights for policy 1, policy_version 351249 (0.0007) [2023-12-27 08:04:29,700][20347] Updated weights for policy 0, policy_version 351367 (0.0008) [2023-12-27 08:04:29,762][20347] Updated weights for policy 0, policy_version 351377 (0.0008) [2023-12-27 08:04:29,817][20347] Updated weights for policy 0, policy_version 351387 (0.0008) [2023-12-27 08:04:30,380][20381] Updated weights for policy 1, policy_version 351259 (0.0007) [2023-12-27 08:04:30,427][20381] Updated weights for policy 1, policy_version 351269 (0.0008) [2023-12-27 08:04:30,481][20381] Updated weights for policy 1, policy_version 351279 (0.0008) [2023-12-27 08:04:30,541][20347] Updated weights for policy 0, policy_version 351397 (0.0007) [2023-12-27 08:04:30,599][20347] Updated weights for policy 0, policy_version 351407 (0.0008) [2023-12-27 08:04:30,656][20347] Updated weights for policy 0, policy_version 351417 (0.0007) [2023-12-27 08:04:31,147][20381] Updated weights for policy 1, policy_version 351289 (0.0007) [2023-12-27 08:04:31,198][20381] Updated weights for policy 1, policy_version 351299 (0.0008) [2023-12-27 08:04:31,252][20381] Updated weights for policy 1, policy_version 351309 (0.0008) [2023-12-27 08:04:31,309][20347] Updated weights for policy 0, policy_version 351427 (0.0007) [2023-12-27 08:04:31,310][20381] Updated weights for policy 1, policy_version 351319 (0.0008) [2023-12-27 08:04:31,369][20347] Updated weights for policy 0, policy_version 351437 (0.0007) [2023-12-27 08:04:31,429][20347] Updated weights for policy 0, policy_version 351447 (0.0007) [2023-12-27 08:04:31,970][20381] Updated weights for policy 1, policy_version 351329 (0.0008) [2023-12-27 08:04:32,016][20381] Updated weights for policy 1, policy_version 351339 (0.0008) [2023-12-27 08:04:32,058][20381] Updated weights for policy 1, policy_version 351349 (0.0007) [2023-12-27 08:04:32,164][20347] Updated weights for policy 0, policy_version 351457 (0.0008) [2023-12-27 08:04:32,211][20347] Updated weights for policy 0, policy_version 351467 (0.0007) [2023-12-27 08:04:32,264][20347] Updated weights for policy 0, policy_version 351477 (0.0007) [2023-12-27 08:04:32,317][20347] Updated weights for policy 0, policy_version 351487 (0.0008) [2023-12-27 08:04:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 179961856. Throughput: 0: 10263.7, 1: 10437.5. Samples: 179966240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:32,800][20381] Updated weights for policy 1, policy_version 351359 (0.0009) [2023-12-27 08:04:32,853][20381] Updated weights for policy 1, policy_version 351369 (0.0009) [2023-12-27 08:04:32,905][20381] Updated weights for policy 1, policy_version 351379 (0.0007) [2023-12-27 08:04:32,968][20347] Updated weights for policy 0, policy_version 351497 (0.0007) [2023-12-27 08:04:33,023][20347] Updated weights for policy 0, policy_version 351507 (0.0008) [2023-12-27 08:04:33,069][20347] Updated weights for policy 0, policy_version 351517 (0.0007) [2023-12-27 08:04:33,581][20381] Updated weights for policy 1, policy_version 351389 (0.0007) [2023-12-27 08:04:33,635][20381] Updated weights for policy 1, policy_version 351399 (0.0007) [2023-12-27 08:04:33,692][20381] Updated weights for policy 1, policy_version 351409 (0.0007) [2023-12-27 08:04:33,774][20347] Updated weights for policy 0, policy_version 351527 (0.0007) [2023-12-27 08:04:33,832][20347] Updated weights for policy 0, policy_version 351537 (0.0007) [2023-12-27 08:04:33,884][20347] Updated weights for policy 0, policy_version 351547 (0.0006) [2023-12-27 08:04:34,329][20381] Updated weights for policy 1, policy_version 351419 (0.0008) [2023-12-27 08:04:34,392][20381] Updated weights for policy 1, policy_version 351429 (0.0007) [2023-12-27 08:04:34,452][20381] Updated weights for policy 1, policy_version 351439 (0.0007) [2023-12-27 08:04:34,485][20347] Updated weights for policy 0, policy_version 351557 (0.0008) [2023-12-27 08:04:34,540][20347] Updated weights for policy 0, policy_version 351567 (0.0010) [2023-12-27 08:04:34,602][20347] Updated weights for policy 0, policy_version 351577 (0.0010) [2023-12-27 08:04:35,129][20381] Updated weights for policy 1, policy_version 351449 (0.0006) [2023-12-27 08:04:35,183][20381] Updated weights for policy 1, policy_version 351459 (0.0006) [2023-12-27 08:04:35,243][20381] Updated weights for policy 1, policy_version 351469 (0.0006) [2023-12-27 08:04:35,253][20347] Updated weights for policy 0, policy_version 351587 (0.0009) [2023-12-27 08:04:35,298][20381] Updated weights for policy 1, policy_version 351479 (0.0007) [2023-12-27 08:04:35,306][20347] Updated weights for policy 0, policy_version 351597 (0.0009) [2023-12-27 08:04:35,357][20347] Updated weights for policy 0, policy_version 351607 (0.0010) [2023-12-27 08:04:35,946][20381] Updated weights for policy 1, policy_version 351489 (0.0005) [2023-12-27 08:04:35,992][20347] Updated weights for policy 0, policy_version 351617 (0.0006) [2023-12-27 08:04:36,005][20381] Updated weights for policy 1, policy_version 351499 (0.0006) [2023-12-27 08:04:36,050][20347] Updated weights for policy 0, policy_version 351627 (0.0006) [2023-12-27 08:04:36,061][20381] Updated weights for policy 1, policy_version 351509 (0.0006) [2023-12-27 08:04:36,108][20347] Updated weights for policy 0, policy_version 351637 (0.0008) [2023-12-27 08:04:36,169][20347] Updated weights for policy 0, policy_version 351647 (0.0008) [2023-12-27 08:04:36,722][20381] Updated weights for policy 1, policy_version 351519 (0.0007) [2023-12-27 08:04:36,783][20381] Updated weights for policy 1, policy_version 351529 (0.0007) [2023-12-27 08:04:36,793][20347] Updated weights for policy 0, policy_version 351657 (0.0006) [2023-12-27 08:04:36,836][20381] Updated weights for policy 1, policy_version 351539 (0.0007) [2023-12-27 08:04:36,850][20347] Updated weights for policy 0, policy_version 351667 (0.0006) [2023-12-27 08:04:36,905][20347] Updated weights for policy 0, policy_version 351677 (0.0006) [2023-12-27 08:04:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 180060160. Throughput: 0: 10287.4, 1: 10471.9. Samples: 180028900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000351544_90013696.pth... [2023-12-27 08:04:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000351680_90046464.pth... [2023-12-27 08:04:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000350360_89710592.pth [2023-12-27 08:04:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000350496_89743360.pth [2023-12-27 08:04:37,543][20347] Updated weights for policy 0, policy_version 351687 (0.0006) [2023-12-27 08:04:37,555][20381] Updated weights for policy 1, policy_version 351549 (0.0008) [2023-12-27 08:04:37,593][20347] Updated weights for policy 0, policy_version 351697 (0.0007) [2023-12-27 08:04:37,602][20381] Updated weights for policy 1, policy_version 351559 (0.0007) [2023-12-27 08:04:37,655][20347] Updated weights for policy 0, policy_version 351707 (0.0006) [2023-12-27 08:04:37,659][20381] Updated weights for policy 1, policy_version 351569 (0.0005) [2023-12-27 08:04:38,229][20381] Updated weights for policy 1, policy_version 351579 (0.0005) [2023-12-27 08:04:38,279][20381] Updated weights for policy 1, policy_version 351589 (0.0006) [2023-12-27 08:04:38,328][20381] Updated weights for policy 1, policy_version 351599 (0.0007) [2023-12-27 08:04:38,395][20347] Updated weights for policy 0, policy_version 351717 (0.0007) [2023-12-27 08:04:38,447][20347] Updated weights for policy 0, policy_version 351727 (0.0008) [2023-12-27 08:04:38,493][20347] Updated weights for policy 0, policy_version 351737 (0.0008) [2023-12-27 08:04:38,995][20381] Updated weights for policy 1, policy_version 351609 (0.0007) [2023-12-27 08:04:39,051][20381] Updated weights for policy 1, policy_version 351619 (0.0007) [2023-12-27 08:04:39,113][20381] Updated weights for policy 1, policy_version 351629 (0.0008) [2023-12-27 08:04:39,170][20381] Updated weights for policy 1, policy_version 351639 (0.0008) [2023-12-27 08:04:39,258][20347] Updated weights for policy 0, policy_version 351747 (0.0008) [2023-12-27 08:04:39,316][20347] Updated weights for policy 0, policy_version 351757 (0.0007) [2023-12-27 08:04:39,381][20347] Updated weights for policy 0, policy_version 351767 (0.0008) [2023-12-27 08:04:39,929][20381] Updated weights for policy 1, policy_version 351649 (0.0008) [2023-12-27 08:04:39,991][20381] Updated weights for policy 1, policy_version 351659 (0.0007) [2023-12-27 08:04:40,048][20381] Updated weights for policy 1, policy_version 351669 (0.0008) [2023-12-27 08:04:40,090][20347] Updated weights for policy 0, policy_version 351777 (0.0008) [2023-12-27 08:04:40,158][20347] Updated weights for policy 0, policy_version 351787 (0.0006) [2023-12-27 08:04:40,222][20347] Updated weights for policy 0, policy_version 351797 (0.0007) [2023-12-27 08:04:40,287][20347] Updated weights for policy 0, policy_version 351807 (0.0008) [2023-12-27 08:04:40,794][20381] Updated weights for policy 1, policy_version 351679 (0.0009) [2023-12-27 08:04:40,847][20381] Updated weights for policy 1, policy_version 351689 (0.0011) [2023-12-27 08:04:40,888][20347] Updated weights for policy 0, policy_version 351817 (0.0006) [2023-12-27 08:04:40,909][20381] Updated weights for policy 1, policy_version 351699 (0.0011) [2023-12-27 08:04:40,937][20347] Updated weights for policy 0, policy_version 351827 (0.0005) [2023-12-27 08:04:40,987][20347] Updated weights for policy 0, policy_version 351837 (0.0007) [2023-12-27 08:04:41,643][20381] Updated weights for policy 1, policy_version 351709 (0.0009) [2023-12-27 08:04:41,706][20381] Updated weights for policy 1, policy_version 351719 (0.0010) [2023-12-27 08:04:41,712][20347] Updated weights for policy 0, policy_version 351847 (0.0007) [2023-12-27 08:04:41,765][20381] Updated weights for policy 1, policy_version 351729 (0.0010) [2023-12-27 08:04:41,774][20347] Updated weights for policy 0, policy_version 351857 (0.0006) [2023-12-27 08:04:41,837][20347] Updated weights for policy 0, policy_version 351867 (0.0006) [2023-12-27 08:04:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 180158464. Throughput: 0: 10319.6, 1: 10446.1. Samples: 180151892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:42,497][20381] Updated weights for policy 1, policy_version 351739 (0.0009) [2023-12-27 08:04:42,543][20381] Updated weights for policy 1, policy_version 351749 (0.0007) [2023-12-27 08:04:42,574][20347] Updated weights for policy 0, policy_version 351877 (0.0006) [2023-12-27 08:04:42,602][20381] Updated weights for policy 1, policy_version 351759 (0.0007) [2023-12-27 08:04:42,641][20347] Updated weights for policy 0, policy_version 351887 (0.0008) [2023-12-27 08:04:42,709][20347] Updated weights for policy 0, policy_version 351897 (0.0009) [2023-12-27 08:04:43,292][20381] Updated weights for policy 1, policy_version 351769 (0.0006) [2023-12-27 08:04:43,352][20381] Updated weights for policy 1, policy_version 351779 (0.0008) [2023-12-27 08:04:43,373][20347] Updated weights for policy 0, policy_version 351907 (0.0008) [2023-12-27 08:04:43,411][20381] Updated weights for policy 1, policy_version 351789 (0.0007) [2023-12-27 08:04:43,423][20347] Updated weights for policy 0, policy_version 351917 (0.0005) [2023-12-27 08:04:43,468][20381] Updated weights for policy 1, policy_version 351799 (0.0006) [2023-12-27 08:04:43,470][20347] Updated weights for policy 0, policy_version 351927 (0.0006) [2023-12-27 08:04:44,035][20347] Updated weights for policy 0, policy_version 351937 (0.0005) [2023-12-27 08:04:44,063][20381] Updated weights for policy 1, policy_version 351809 (0.0010) [2023-12-27 08:04:44,102][20347] Updated weights for policy 0, policy_version 351947 (0.0005) [2023-12-27 08:04:44,123][20381] Updated weights for policy 1, policy_version 351819 (0.0009) [2023-12-27 08:04:44,155][20347] Updated weights for policy 0, policy_version 351957 (0.0005) [2023-12-27 08:04:44,175][20381] Updated weights for policy 1, policy_version 351829 (0.0010) [2023-12-27 08:04:44,211][20347] Updated weights for policy 0, policy_version 351967 (0.0006) [2023-12-27 08:04:44,740][20347] Updated weights for policy 0, policy_version 351977 (0.0007) [2023-12-27 08:04:44,802][20347] Updated weights for policy 0, policy_version 351987 (0.0008) [2023-12-27 08:04:44,863][20347] Updated weights for policy 0, policy_version 351997 (0.0006) [2023-12-27 08:04:44,869][20381] Updated weights for policy 1, policy_version 351839 (0.0011) [2023-12-27 08:04:44,930][20381] Updated weights for policy 1, policy_version 351849 (0.0011) [2023-12-27 08:04:44,989][20381] Updated weights for policy 1, policy_version 351859 (0.0011) [2023-12-27 08:04:45,538][20347] Updated weights for policy 0, policy_version 352007 (0.0006) [2023-12-27 08:04:45,594][20347] Updated weights for policy 0, policy_version 352017 (0.0007) [2023-12-27 08:04:45,656][20347] Updated weights for policy 0, policy_version 352027 (0.0007) [2023-12-27 08:04:45,683][20381] Updated weights for policy 1, policy_version 351869 (0.0011) [2023-12-27 08:04:45,741][20381] Updated weights for policy 1, policy_version 351879 (0.0010) [2023-12-27 08:04:45,790][20381] Updated weights for policy 1, policy_version 351889 (0.0008) [2023-12-27 08:04:46,352][20347] Updated weights for policy 0, policy_version 352037 (0.0008) [2023-12-27 08:04:46,407][20347] Updated weights for policy 0, policy_version 352047 (0.0007) [2023-12-27 08:04:46,456][20347] Updated weights for policy 0, policy_version 352057 (0.0007) [2023-12-27 08:04:46,501][20381] Updated weights for policy 1, policy_version 351899 (0.0008) [2023-12-27 08:04:46,553][20381] Updated weights for policy 1, policy_version 351909 (0.0007) [2023-12-27 08:04:46,615][20381] Updated weights for policy 1, policy_version 351919 (0.0005) [2023-12-27 08:04:47,097][20347] Updated weights for policy 0, policy_version 352067 (0.0007) [2023-12-27 08:04:47,149][20347] Updated weights for policy 0, policy_version 352077 (0.0005) [2023-12-27 08:04:47,188][20381] Updated weights for policy 1, policy_version 351929 (0.0005) [2023-12-27 08:04:47,204][20347] Updated weights for policy 0, policy_version 352087 (0.0006) [2023-12-27 08:04:47,237][20381] Updated weights for policy 1, policy_version 351939 (0.0005) [2023-12-27 08:04:47,287][20381] Updated weights for policy 1, policy_version 351949 (0.0005) [2023-12-27 08:04:47,338][20381] Updated weights for policy 1, policy_version 351959 (0.0005) [2023-12-27 08:04:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 180273152. Throughput: 0: 10356.9, 1: 10387.0. Samples: 180276896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:47,858][20347] Updated weights for policy 0, policy_version 352097 (0.0007) [2023-12-27 08:04:47,908][20347] Updated weights for policy 0, policy_version 352107 (0.0007) [2023-12-27 08:04:47,914][20381] Updated weights for policy 1, policy_version 351969 (0.0005) [2023-12-27 08:04:47,963][20347] Updated weights for policy 0, policy_version 352117 (0.0007) [2023-12-27 08:04:47,969][20381] Updated weights for policy 1, policy_version 351979 (0.0005) [2023-12-27 08:04:48,020][20347] Updated weights for policy 0, policy_version 352127 (0.0007) [2023-12-27 08:04:48,030][20381] Updated weights for policy 1, policy_version 351989 (0.0007) [2023-12-27 08:04:48,696][20347] Updated weights for policy 0, policy_version 352137 (0.0007) [2023-12-27 08:04:48,724][20381] Updated weights for policy 1, policy_version 351999 (0.0006) [2023-12-27 08:04:48,749][20347] Updated weights for policy 0, policy_version 352147 (0.0006) [2023-12-27 08:04:48,777][20381] Updated weights for policy 1, policy_version 352009 (0.0005) [2023-12-27 08:04:48,796][20347] Updated weights for policy 0, policy_version 352157 (0.0006) [2023-12-27 08:04:48,836][20381] Updated weights for policy 1, policy_version 352019 (0.0007) [2023-12-27 08:04:49,455][20347] Updated weights for policy 0, policy_version 352167 (0.0006) [2023-12-27 08:04:49,507][20347] Updated weights for policy 0, policy_version 352177 (0.0007) [2023-12-27 08:04:49,530][20381] Updated weights for policy 1, policy_version 352029 (0.0007) [2023-12-27 08:04:49,564][20347] Updated weights for policy 0, policy_version 352187 (0.0007) [2023-12-27 08:04:49,588][20381] Updated weights for policy 1, policy_version 352039 (0.0007) [2023-12-27 08:04:49,644][20381] Updated weights for policy 1, policy_version 352049 (0.0007) [2023-12-27 08:04:50,261][20347] Updated weights for policy 0, policy_version 352197 (0.0007) [2023-12-27 08:04:50,316][20347] Updated weights for policy 0, policy_version 352207 (0.0007) [2023-12-27 08:04:50,361][20381] Updated weights for policy 1, policy_version 352059 (0.0008) [2023-12-27 08:04:50,374][20347] Updated weights for policy 0, policy_version 352217 (0.0007) [2023-12-27 08:04:50,418][20381] Updated weights for policy 1, policy_version 352069 (0.0006) [2023-12-27 08:04:50,481][20381] Updated weights for policy 1, policy_version 352079 (0.0008) [2023-12-27 08:04:51,081][20347] Updated weights for policy 0, policy_version 352227 (0.0006) [2023-12-27 08:04:51,113][20381] Updated weights for policy 1, policy_version 352089 (0.0009) [2023-12-27 08:04:51,137][20347] Updated weights for policy 0, policy_version 352237 (0.0007) [2023-12-27 08:04:51,172][20381] Updated weights for policy 1, policy_version 352099 (0.0006) [2023-12-27 08:04:51,189][20347] Updated weights for policy 0, policy_version 352247 (0.0005) [2023-12-27 08:04:51,226][20381] Updated weights for policy 1, policy_version 352109 (0.0006) [2023-12-27 08:04:51,290][20381] Updated weights for policy 1, policy_version 352119 (0.0008) [2023-12-27 08:04:51,937][20347] Updated weights for policy 0, policy_version 352257 (0.0005) [2023-12-27 08:04:51,982][20381] Updated weights for policy 1, policy_version 352129 (0.0008) [2023-12-27 08:04:51,998][20347] Updated weights for policy 0, policy_version 352267 (0.0008) [2023-12-27 08:04:52,050][20381] Updated weights for policy 1, policy_version 352139 (0.0008) [2023-12-27 08:04:52,061][20347] Updated weights for policy 0, policy_version 352277 (0.0007) [2023-12-27 08:04:52,116][20381] Updated weights for policy 1, policy_version 352149 (0.0006) [2023-12-27 08:04:52,125][20347] Updated weights for policy 0, policy_version 352287 (0.0005) [2023-12-27 08:04:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 180371456. Throughput: 0: 10291.0, 1: 10331.3. Samples: 180339256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000352288_90202112.pth... [2023-12-27 08:04:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000352152_90169344.pth... [2023-12-27 08:04:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000350968_89866240.pth [2023-12-27 08:04:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000351072_89890816.pth [2023-12-27 08:04:52,706][20381] Updated weights for policy 1, policy_version 352159 (0.0006) [2023-12-27 08:04:52,768][20381] Updated weights for policy 1, policy_version 352169 (0.0007) [2023-12-27 08:04:52,783][20347] Updated weights for policy 0, policy_version 352297 (0.0007) [2023-12-27 08:04:52,823][20381] Updated weights for policy 1, policy_version 352179 (0.0006) [2023-12-27 08:04:52,839][20347] Updated weights for policy 0, policy_version 352307 (0.0007) [2023-12-27 08:04:52,902][20347] Updated weights for policy 0, policy_version 352317 (0.0006) [2023-12-27 08:04:53,392][20381] Updated weights for policy 1, policy_version 352189 (0.0007) [2023-12-27 08:04:53,455][20381] Updated weights for policy 1, policy_version 352199 (0.0008) [2023-12-27 08:04:53,509][20381] Updated weights for policy 1, policy_version 352209 (0.0008) [2023-12-27 08:04:53,520][20347] Updated weights for policy 0, policy_version 352327 (0.0005) [2023-12-27 08:04:53,569][20347] Updated weights for policy 0, policy_version 352337 (0.0005) [2023-12-27 08:04:53,618][20347] Updated weights for policy 0, policy_version 352347 (0.0005) [2023-12-27 08:04:54,165][20347] Updated weights for policy 0, policy_version 352357 (0.0005) [2023-12-27 08:04:54,171][20381] Updated weights for policy 1, policy_version 352219 (0.0007) [2023-12-27 08:04:54,217][20381] Updated weights for policy 1, policy_version 352229 (0.0007) [2023-12-27 08:04:54,223][20347] Updated weights for policy 0, policy_version 352367 (0.0007) [2023-12-27 08:04:54,268][20381] Updated weights for policy 1, policy_version 352239 (0.0007) [2023-12-27 08:04:54,280][20347] Updated weights for policy 0, policy_version 352377 (0.0005) [2023-12-27 08:04:54,875][20347] Updated weights for policy 0, policy_version 352387 (0.0007) [2023-12-27 08:04:54,926][20347] Updated weights for policy 0, policy_version 352397 (0.0008) [2023-12-27 08:04:54,976][20347] Updated weights for policy 0, policy_version 352407 (0.0007) [2023-12-27 08:04:54,986][20381] Updated weights for policy 1, policy_version 352249 (0.0007) [2023-12-27 08:04:55,047][20381] Updated weights for policy 1, policy_version 352259 (0.0007) [2023-12-27 08:04:55,105][20381] Updated weights for policy 1, policy_version 352269 (0.0007) [2023-12-27 08:04:55,161][20381] Updated weights for policy 1, policy_version 352279 (0.0007) [2023-12-27 08:04:55,558][20347] Updated weights for policy 0, policy_version 352417 (0.0006) [2023-12-27 08:04:55,616][20347] Updated weights for policy 0, policy_version 352427 (0.0005) [2023-12-27 08:04:55,678][20347] Updated weights for policy 0, policy_version 352437 (0.0006) [2023-12-27 08:04:55,725][20347] Updated weights for policy 0, policy_version 352447 (0.0007) [2023-12-27 08:04:55,891][20381] Updated weights for policy 1, policy_version 352289 (0.0008) [2023-12-27 08:04:55,947][20381] Updated weights for policy 1, policy_version 352299 (0.0007) [2023-12-27 08:04:55,998][20381] Updated weights for policy 1, policy_version 352309 (0.0008) [2023-12-27 08:04:56,383][20347] Updated weights for policy 0, policy_version 352457 (0.0007) [2023-12-27 08:04:56,427][20347] Updated weights for policy 0, policy_version 352467 (0.0007) [2023-12-27 08:04:56,486][20347] Updated weights for policy 0, policy_version 352477 (0.0007) [2023-12-27 08:04:56,695][20381] Updated weights for policy 1, policy_version 352319 (0.0007) [2023-12-27 08:04:56,758][20381] Updated weights for policy 1, policy_version 352329 (0.0007) [2023-12-27 08:04:56,818][20381] Updated weights for policy 1, policy_version 352339 (0.0007) [2023-12-27 08:04:57,172][20347] Updated weights for policy 0, policy_version 352487 (0.0007) [2023-12-27 08:04:57,230][20347] Updated weights for policy 0, policy_version 352497 (0.0008) [2023-12-27 08:04:57,285][20347] Updated weights for policy 0, policy_version 352507 (0.0007) [2023-12-27 08:04:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 180477952. Throughput: 0: 10345.6, 1: 10285.8. Samples: 180465768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:04:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:04:57,478][20381] Updated weights for policy 1, policy_version 352349 (0.0007) [2023-12-27 08:04:57,535][20381] Updated weights for policy 1, policy_version 352359 (0.0006) [2023-12-27 08:04:57,589][20381] Updated weights for policy 1, policy_version 352369 (0.0009) [2023-12-27 08:04:57,879][20347] Updated weights for policy 0, policy_version 352517 (0.0006) [2023-12-27 08:04:57,934][20347] Updated weights for policy 0, policy_version 352527 (0.0005) [2023-12-27 08:04:57,989][20347] Updated weights for policy 0, policy_version 352537 (0.0005) [2023-12-27 08:04:58,297][20381] Updated weights for policy 1, policy_version 352379 (0.0006) [2023-12-27 08:04:58,367][20381] Updated weights for policy 1, policy_version 352389 (0.0008) [2023-12-27 08:04:58,428][20381] Updated weights for policy 1, policy_version 352399 (0.0008) [2023-12-27 08:04:58,682][20347] Updated weights for policy 0, policy_version 352547 (0.0006) [2023-12-27 08:04:58,743][20347] Updated weights for policy 0, policy_version 352557 (0.0008) [2023-12-27 08:04:58,799][20347] Updated weights for policy 0, policy_version 352567 (0.0007) [2023-12-27 08:04:59,190][20381] Updated weights for policy 1, policy_version 352409 (0.0008) [2023-12-27 08:04:59,251][20381] Updated weights for policy 1, policy_version 352419 (0.0008) [2023-12-27 08:04:59,317][20381] Updated weights for policy 1, policy_version 352429 (0.0008) [2023-12-27 08:04:59,377][20381] Updated weights for policy 1, policy_version 352439 (0.0007) [2023-12-27 08:04:59,469][20347] Updated weights for policy 0, policy_version 352577 (0.0008) [2023-12-27 08:04:59,532][20347] Updated weights for policy 0, policy_version 352587 (0.0007) [2023-12-27 08:04:59,591][20347] Updated weights for policy 0, policy_version 352597 (0.0007) [2023-12-27 08:04:59,651][20347] Updated weights for policy 0, policy_version 352607 (0.0005) [2023-12-27 08:04:59,998][20381] Updated weights for policy 1, policy_version 352449 (0.0007) [2023-12-27 08:05:00,057][20381] Updated weights for policy 1, policy_version 352459 (0.0008) [2023-12-27 08:05:00,111][20381] Updated weights for policy 1, policy_version 352469 (0.0007) [2023-12-27 08:05:00,287][20347] Updated weights for policy 0, policy_version 352617 (0.0010) [2023-12-27 08:05:00,344][20347] Updated weights for policy 0, policy_version 352627 (0.0007) [2023-12-27 08:05:00,401][20347] Updated weights for policy 0, policy_version 352637 (0.0007) [2023-12-27 08:05:00,775][20381] Updated weights for policy 1, policy_version 352479 (0.0006) [2023-12-27 08:05:00,838][20381] Updated weights for policy 1, policy_version 352489 (0.0008) [2023-12-27 08:05:00,901][20381] Updated weights for policy 1, policy_version 352499 (0.0008) [2023-12-27 08:05:01,036][20347] Updated weights for policy 0, policy_version 352647 (0.0008) [2023-12-27 08:05:01,099][20347] Updated weights for policy 0, policy_version 352657 (0.0008) [2023-12-27 08:05:01,165][20347] Updated weights for policy 0, policy_version 352667 (0.0008) [2023-12-27 08:05:01,636][20381] Updated weights for policy 1, policy_version 352509 (0.0009) [2023-12-27 08:05:01,691][20381] Updated weights for policy 1, policy_version 352519 (0.0008) [2023-12-27 08:05:01,753][20381] Updated weights for policy 1, policy_version 352529 (0.0008) [2023-12-27 08:05:01,851][20347] Updated weights for policy 0, policy_version 352677 (0.0008) [2023-12-27 08:05:01,894][20347] Updated weights for policy 0, policy_version 352687 (0.0007) [2023-12-27 08:05:01,943][20347] Updated weights for policy 0, policy_version 352697 (0.0005) [2023-12-27 08:05:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 180576256. Throughput: 0: 10418.6, 1: 10219.1. Samples: 180588488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:02,400][20381] Updated weights for policy 1, policy_version 352539 (0.0008) [2023-12-27 08:05:02,445][20381] Updated weights for policy 1, policy_version 352549 (0.0007) [2023-12-27 08:05:02,497][20381] Updated weights for policy 1, policy_version 352559 (0.0007) [2023-12-27 08:05:02,579][20347] Updated weights for policy 0, policy_version 352707 (0.0006) [2023-12-27 08:05:02,639][20347] Updated weights for policy 0, policy_version 352717 (0.0005) [2023-12-27 08:05:02,705][20347] Updated weights for policy 0, policy_version 352727 (0.0006) [2023-12-27 08:05:03,258][20381] Updated weights for policy 1, policy_version 352569 (0.0007) [2023-12-27 08:05:03,280][20347] Updated weights for policy 0, policy_version 352737 (0.0007) [2023-12-27 08:05:03,311][20381] Updated weights for policy 1, policy_version 352579 (0.0006) [2023-12-27 08:05:03,333][20347] Updated weights for policy 0, policy_version 352747 (0.0006) [2023-12-27 08:05:03,356][20381] Updated weights for policy 1, policy_version 352589 (0.0008) [2023-12-27 08:05:03,386][20347] Updated weights for policy 0, policy_version 352757 (0.0007) [2023-12-27 08:05:03,403][20381] Updated weights for policy 1, policy_version 352599 (0.0005) [2023-12-27 08:05:03,439][20347] Updated weights for policy 0, policy_version 352767 (0.0007) [2023-12-27 08:05:04,060][20381] Updated weights for policy 1, policy_version 352609 (0.0007) [2023-12-27 08:05:04,102][20347] Updated weights for policy 0, policy_version 352777 (0.0006) [2023-12-27 08:05:04,113][20381] Updated weights for policy 1, policy_version 352619 (0.0008) [2023-12-27 08:05:04,135][20292] KL-divergence is very high: 188.9720 [2023-12-27 08:05:04,164][20347] Updated weights for policy 0, policy_version 352787 (0.0006) [2023-12-27 08:05:04,170][20381] Updated weights for policy 1, policy_version 352629 (0.0008) [2023-12-27 08:05:04,217][20347] Updated weights for policy 0, policy_version 352797 (0.0006) [2023-12-27 08:05:04,807][20347] Updated weights for policy 0, policy_version 352807 (0.0006) [2023-12-27 08:05:04,860][20347] Updated weights for policy 0, policy_version 352817 (0.0005) [2023-12-27 08:05:04,920][20347] Updated weights for policy 0, policy_version 352827 (0.0005) [2023-12-27 08:05:04,932][20381] Updated weights for policy 1, policy_version 352639 (0.0007) [2023-12-27 08:05:04,984][20381] Updated weights for policy 1, policy_version 352649 (0.0007) [2023-12-27 08:05:05,040][20381] Updated weights for policy 1, policy_version 352659 (0.0008) [2023-12-27 08:05:05,483][20347] Updated weights for policy 0, policy_version 352837 (0.0005) [2023-12-27 08:05:05,531][20347] Updated weights for policy 0, policy_version 352847 (0.0005) [2023-12-27 08:05:05,586][20347] Updated weights for policy 0, policy_version 352857 (0.0005) [2023-12-27 08:05:05,643][20381] Updated weights for policy 1, policy_version 352669 (0.0007) [2023-12-27 08:05:05,693][20381] Updated weights for policy 1, policy_version 352679 (0.0007) [2023-12-27 08:05:05,744][20381] Updated weights for policy 1, policy_version 352689 (0.0007) [2023-12-27 08:05:06,188][20347] Updated weights for policy 0, policy_version 352867 (0.0006) [2023-12-27 08:05:06,249][20347] Updated weights for policy 0, policy_version 352877 (0.0007) [2023-12-27 08:05:06,309][20347] Updated weights for policy 0, policy_version 352887 (0.0005) [2023-12-27 08:05:06,385][20381] Updated weights for policy 1, policy_version 352699 (0.0008) [2023-12-27 08:05:06,455][20381] Updated weights for policy 1, policy_version 352709 (0.0008) [2023-12-27 08:05:06,517][20381] Updated weights for policy 1, policy_version 352719 (0.0008) [2023-12-27 08:05:06,933][20347] Updated weights for policy 0, policy_version 352897 (0.0006) [2023-12-27 08:05:06,988][20347] Updated weights for policy 0, policy_version 352907 (0.0006) [2023-12-27 08:05:07,036][20347] Updated weights for policy 0, policy_version 352917 (0.0007) [2023-12-27 08:05:07,087][20347] Updated weights for policy 0, policy_version 352927 (0.0009) [2023-12-27 08:05:07,133][20381] Updated weights for policy 1, policy_version 352729 (0.0006) [2023-12-27 08:05:07,191][20381] Updated weights for policy 1, policy_version 352739 (0.0005) [2023-12-27 08:05:07,253][20381] Updated weights for policy 1, policy_version 352749 (0.0007) [2023-12-27 08:05:07,306][20381] Updated weights for policy 1, policy_version 352759 (0.0007) [2023-12-27 08:05:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 180690944. Throughput: 0: 10510.8, 1: 10171.7. Samples: 180651828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.310')] [2023-12-27 08:05:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000352760_90324992.pth... [2023-12-27 08:05:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000352928_90365952.pth... [2023-12-27 08:05:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000351544_90013696.pth [2023-12-27 08:05:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000351680_90046464.pth [2023-12-27 08:05:07,731][20347] Updated weights for policy 0, policy_version 352937 (0.0007) [2023-12-27 08:05:07,785][20347] Updated weights for policy 0, policy_version 352947 (0.0005) [2023-12-27 08:05:07,840][20347] Updated weights for policy 0, policy_version 352957 (0.0005) [2023-12-27 08:05:07,890][20381] Updated weights for policy 1, policy_version 352769 (0.0008) [2023-12-27 08:05:07,937][20381] Updated weights for policy 1, policy_version 352779 (0.0006) [2023-12-27 08:05:07,995][20381] Updated weights for policy 1, policy_version 352789 (0.0005) [2023-12-27 08:05:08,461][20347] Updated weights for policy 0, policy_version 352967 (0.0007) [2023-12-27 08:05:08,510][20347] Updated weights for policy 0, policy_version 352977 (0.0008) [2023-12-27 08:05:08,569][20347] Updated weights for policy 0, policy_version 352987 (0.0008) [2023-12-27 08:05:08,615][20381] Updated weights for policy 1, policy_version 352799 (0.0007) [2023-12-27 08:05:08,674][20381] Updated weights for policy 1, policy_version 352809 (0.0008) [2023-12-27 08:05:08,738][20381] Updated weights for policy 1, policy_version 352819 (0.0007) [2023-12-27 08:05:09,273][20347] Updated weights for policy 0, policy_version 352997 (0.0007) [2023-12-27 08:05:09,330][20347] Updated weights for policy 0, policy_version 353007 (0.0007) [2023-12-27 08:05:09,391][20347] Updated weights for policy 0, policy_version 353017 (0.0007) [2023-12-27 08:05:09,442][20381] Updated weights for policy 1, policy_version 352829 (0.0009) [2023-12-27 08:05:09,495][20381] Updated weights for policy 1, policy_version 352839 (0.0011) [2023-12-27 08:05:09,553][20381] Updated weights for policy 1, policy_version 352849 (0.0010) [2023-12-27 08:05:10,103][20347] Updated weights for policy 0, policy_version 353027 (0.0007) [2023-12-27 08:05:10,159][20347] Updated weights for policy 0, policy_version 353037 (0.0007) [2023-12-27 08:05:10,213][20347] Updated weights for policy 0, policy_version 353047 (0.0007) [2023-12-27 08:05:10,291][20381] Updated weights for policy 1, policy_version 352859 (0.0011) [2023-12-27 08:05:10,353][20381] Updated weights for policy 1, policy_version 352869 (0.0011) [2023-12-27 08:05:10,410][20381] Updated weights for policy 1, policy_version 352879 (0.0011) [2023-12-27 08:05:10,922][20347] Updated weights for policy 0, policy_version 353057 (0.0007) [2023-12-27 08:05:10,977][20347] Updated weights for policy 0, policy_version 353067 (0.0007) [2023-12-27 08:05:11,043][20347] Updated weights for policy 0, policy_version 353077 (0.0008) [2023-12-27 08:05:11,112][20347] Updated weights for policy 0, policy_version 353087 (0.0008) [2023-12-27 08:05:11,115][20381] Updated weights for policy 1, policy_version 352889 (0.0010) [2023-12-27 08:05:11,176][20381] Updated weights for policy 1, policy_version 352899 (0.0009) [2023-12-27 08:05:11,236][20381] Updated weights for policy 1, policy_version 352909 (0.0008) [2023-12-27 08:05:11,301][20381] Updated weights for policy 1, policy_version 352919 (0.0008) [2023-12-27 08:05:11,833][20347] Updated weights for policy 0, policy_version 353097 (0.0008) [2023-12-27 08:05:11,893][20347] Updated weights for policy 0, policy_version 353107 (0.0006) [2023-12-27 08:05:11,945][20347] Updated weights for policy 0, policy_version 353117 (0.0006) [2023-12-27 08:05:11,971][20381] Updated weights for policy 1, policy_version 352929 (0.0007) [2023-12-27 08:05:12,035][20381] Updated weights for policy 1, policy_version 352939 (0.0008) [2023-12-27 08:05:12,100][20381] Updated weights for policy 1, policy_version 352949 (0.0008) [2023-12-27 08:05:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 180789248. Throughput: 0: 10529.8, 1: 10233.4. Samples: 180777896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.640')] [2023-12-27 08:05:12,655][20347] Updated weights for policy 0, policy_version 353127 (0.0008) [2023-12-27 08:05:12,708][20347] Updated weights for policy 0, policy_version 353137 (0.0007) [2023-12-27 08:05:12,761][20347] Updated weights for policy 0, policy_version 353147 (0.0008) [2023-12-27 08:05:12,820][20381] Updated weights for policy 1, policy_version 352959 (0.0006) [2023-12-27 08:05:12,873][20381] Updated weights for policy 1, policy_version 352969 (0.0005) [2023-12-27 08:05:12,930][20381] Updated weights for policy 1, policy_version 352979 (0.0006) [2023-12-27 08:05:13,445][20347] Updated weights for policy 0, policy_version 353157 (0.0008) [2023-12-27 08:05:13,495][20347] Updated weights for policy 0, policy_version 353167 (0.0010) [2023-12-27 08:05:13,551][20347] Updated weights for policy 0, policy_version 353177 (0.0007) [2023-12-27 08:05:13,555][20381] Updated weights for policy 1, policy_version 352989 (0.0006) [2023-12-27 08:05:13,618][20381] Updated weights for policy 1, policy_version 352999 (0.0005) [2023-12-27 08:05:13,680][20381] Updated weights for policy 1, policy_version 353009 (0.0008) [2023-12-27 08:05:14,252][20347] Updated weights for policy 0, policy_version 353187 (0.0006) [2023-12-27 08:05:14,302][20347] Updated weights for policy 0, policy_version 353197 (0.0007) [2023-12-27 08:05:14,350][20347] Updated weights for policy 0, policy_version 353207 (0.0008) [2023-12-27 08:05:14,360][20381] Updated weights for policy 1, policy_version 353019 (0.0008) [2023-12-27 08:05:14,413][20381] Updated weights for policy 1, policy_version 353029 (0.0008) [2023-12-27 08:05:14,461][20381] Updated weights for policy 1, policy_version 353039 (0.0007) [2023-12-27 08:05:15,016][20347] Updated weights for policy 0, policy_version 353217 (0.0007) [2023-12-27 08:05:15,076][20347] Updated weights for policy 0, policy_version 353227 (0.0007) [2023-12-27 08:05:15,141][20347] Updated weights for policy 0, policy_version 353237 (0.0008) [2023-12-27 08:05:15,184][20381] Updated weights for policy 1, policy_version 353049 (0.0007) [2023-12-27 08:05:15,202][20347] Updated weights for policy 0, policy_version 353247 (0.0007) [2023-12-27 08:05:15,242][20381] Updated weights for policy 1, policy_version 353059 (0.0008) [2023-12-27 08:05:15,299][20381] Updated weights for policy 1, policy_version 353069 (0.0008) [2023-12-27 08:05:15,360][20381] Updated weights for policy 1, policy_version 353079 (0.0007) [2023-12-27 08:05:15,899][20347] Updated weights for policy 0, policy_version 353257 (0.0008) [2023-12-27 08:05:15,958][20347] Updated weights for policy 0, policy_version 353267 (0.0007) [2023-12-27 08:05:16,000][20381] Updated weights for policy 1, policy_version 353089 (0.0008) [2023-12-27 08:05:16,012][20347] Updated weights for policy 0, policy_version 353277 (0.0007) [2023-12-27 08:05:16,048][20381] Updated weights for policy 1, policy_version 353099 (0.0008) [2023-12-27 08:05:16,097][20381] Updated weights for policy 1, policy_version 353109 (0.0008) [2023-12-27 08:05:16,699][20347] Updated weights for policy 0, policy_version 353287 (0.0005) [2023-12-27 08:05:16,758][20347] Updated weights for policy 0, policy_version 353297 (0.0005) [2023-12-27 08:05:16,813][20347] Updated weights for policy 0, policy_version 353307 (0.0006) [2023-12-27 08:05:16,813][20381] Updated weights for policy 1, policy_version 353119 (0.0008) [2023-12-27 08:05:16,863][20381] Updated weights for policy 1, policy_version 353129 (0.0009) [2023-12-27 08:05:16,920][20381] Updated weights for policy 1, policy_version 353139 (0.0008) [2023-12-27 08:05:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 180887552. Throughput: 0: 10503.8, 1: 10221.6. Samples: 180898884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.330')] [2023-12-27 08:05:17,392][20347] Updated weights for policy 0, policy_version 353317 (0.0006) [2023-12-27 08:05:17,455][20347] Updated weights for policy 0, policy_version 353327 (0.0005) [2023-12-27 08:05:17,514][20347] Updated weights for policy 0, policy_version 353337 (0.0007) [2023-12-27 08:05:17,663][20381] Updated weights for policy 1, policy_version 353149 (0.0008) [2023-12-27 08:05:17,719][20381] Updated weights for policy 1, policy_version 353159 (0.0007) [2023-12-27 08:05:17,768][20381] Updated weights for policy 1, policy_version 353169 (0.0007) [2023-12-27 08:05:18,072][20347] Updated weights for policy 0, policy_version 353347 (0.0007) [2023-12-27 08:05:18,116][20347] Updated weights for policy 0, policy_version 353357 (0.0007) [2023-12-27 08:05:18,161][20347] Updated weights for policy 0, policy_version 353367 (0.0007) [2023-12-27 08:05:18,543][20381] Updated weights for policy 1, policy_version 353179 (0.0008) [2023-12-27 08:05:18,601][20381] Updated weights for policy 1, policy_version 353189 (0.0007) [2023-12-27 08:05:18,659][20381] Updated weights for policy 1, policy_version 353199 (0.0008) [2023-12-27 08:05:18,782][20347] Updated weights for policy 0, policy_version 353377 (0.0007) [2023-12-27 08:05:18,844][20347] Updated weights for policy 0, policy_version 353387 (0.0005) [2023-12-27 08:05:18,896][20347] Updated weights for policy 0, policy_version 353397 (0.0005) [2023-12-27 08:05:18,958][20347] Updated weights for policy 0, policy_version 353407 (0.0008) [2023-12-27 08:05:19,348][20381] Updated weights for policy 1, policy_version 353209 (0.0009) [2023-12-27 08:05:19,409][20381] Updated weights for policy 1, policy_version 353219 (0.0006) [2023-12-27 08:05:19,463][20381] Updated weights for policy 1, policy_version 353229 (0.0009) [2023-12-27 08:05:19,525][20381] Updated weights for policy 1, policy_version 353239 (0.0009) [2023-12-27 08:05:19,614][20347] Updated weights for policy 0, policy_version 353417 (0.0008) [2023-12-27 08:05:19,666][20347] Updated weights for policy 0, policy_version 353427 (0.0007) [2023-12-27 08:05:19,713][20347] Updated weights for policy 0, policy_version 353437 (0.0007) [2023-12-27 08:05:20,196][20381] Updated weights for policy 1, policy_version 353249 (0.0010) [2023-12-27 08:05:20,253][20381] Updated weights for policy 1, policy_version 353259 (0.0007) [2023-12-27 08:05:20,306][20381] Updated weights for policy 1, policy_version 353269 (0.0007) [2023-12-27 08:05:20,515][20347] Updated weights for policy 0, policy_version 353447 (0.0008) [2023-12-27 08:05:20,577][20347] Updated weights for policy 0, policy_version 353457 (0.0009) [2023-12-27 08:05:20,632][20347] Updated weights for policy 0, policy_version 353467 (0.0009) [2023-12-27 08:05:20,990][20381] Updated weights for policy 1, policy_version 353279 (0.0010) [2023-12-27 08:05:21,055][20381] Updated weights for policy 1, policy_version 353289 (0.0011) [2023-12-27 08:05:21,112][20381] Updated weights for policy 1, policy_version 353299 (0.0009) [2023-12-27 08:05:21,286][20347] Updated weights for policy 0, policy_version 353477 (0.0008) [2023-12-27 08:05:21,353][20347] Updated weights for policy 0, policy_version 353487 (0.0008) [2023-12-27 08:05:21,421][20347] Updated weights for policy 0, policy_version 353497 (0.0008) [2023-12-27 08:05:21,836][20381] Updated weights for policy 1, policy_version 353309 (0.0008) [2023-12-27 08:05:21,897][20381] Updated weights for policy 1, policy_version 353319 (0.0008) [2023-12-27 08:05:21,955][20381] Updated weights for policy 1, policy_version 353329 (0.0006) [2023-12-27 08:05:22,152][20347] Updated weights for policy 0, policy_version 353507 (0.0007) [2023-12-27 08:05:22,200][20347] Updated weights for policy 0, policy_version 353517 (0.0006) [2023-12-27 08:05:22,256][20347] Updated weights for policy 0, policy_version 353527 (0.0010) [2023-12-27 08:05:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 180994048. Throughput: 0: 10535.9, 1: 10187.8. Samples: 180961464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000353336_90472448.pth... [2023-12-27 08:05:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000353536_90521600.pth... [2023-12-27 08:05:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000352288_90202112.pth [2023-12-27 08:05:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000352152_90169344.pth [2023-12-27 08:05:22,688][20381] Updated weights for policy 1, policy_version 353339 (0.0007) [2023-12-27 08:05:22,739][20381] Updated weights for policy 1, policy_version 353349 (0.0011) [2023-12-27 08:05:22,787][20381] Updated weights for policy 1, policy_version 353359 (0.0010) [2023-12-27 08:05:22,905][20347] Updated weights for policy 0, policy_version 353537 (0.0010) [2023-12-27 08:05:22,970][20347] Updated weights for policy 0, policy_version 353547 (0.0011) [2023-12-27 08:05:23,030][20347] Updated weights for policy 0, policy_version 353557 (0.0010) [2023-12-27 08:05:23,092][20347] Updated weights for policy 0, policy_version 353567 (0.0009) [2023-12-27 08:05:23,467][20381] Updated weights for policy 1, policy_version 353369 (0.0007) [2023-12-27 08:05:23,518][20381] Updated weights for policy 1, policy_version 353379 (0.0011) [2023-12-27 08:05:23,572][20381] Updated weights for policy 1, policy_version 353389 (0.0010) [2023-12-27 08:05:23,630][20381] Updated weights for policy 1, policy_version 353399 (0.0011) [2023-12-27 08:05:23,704][20347] Updated weights for policy 0, policy_version 353577 (0.0009) [2023-12-27 08:05:23,757][20347] Updated weights for policy 0, policy_version 353587 (0.0008) [2023-12-27 08:05:23,805][20347] Updated weights for policy 0, policy_version 353597 (0.0010) [2023-12-27 08:05:24,302][20381] Updated weights for policy 1, policy_version 353409 (0.0010) [2023-12-27 08:05:24,351][20381] Updated weights for policy 1, policy_version 353419 (0.0010) [2023-12-27 08:05:24,398][20381] Updated weights for policy 1, policy_version 353429 (0.0010) [2023-12-27 08:05:24,458][20347] Updated weights for policy 0, policy_version 353607 (0.0010) [2023-12-27 08:05:24,512][20347] Updated weights for policy 0, policy_version 353617 (0.0010) [2023-12-27 08:05:24,564][20347] Updated weights for policy 0, policy_version 353627 (0.0010) [2023-12-27 08:05:25,066][20381] Updated weights for policy 1, policy_version 353439 (0.0008) [2023-12-27 08:05:25,133][20381] Updated weights for policy 1, policy_version 353449 (0.0010) [2023-12-27 08:05:25,164][20347] Updated weights for policy 0, policy_version 353637 (0.0010) [2023-12-27 08:05:25,198][20381] Updated weights for policy 1, policy_version 353459 (0.0009) [2023-12-27 08:05:25,214][20347] Updated weights for policy 0, policy_version 353647 (0.0010) [2023-12-27 08:05:25,259][20347] Updated weights for policy 0, policy_version 353657 (0.0010) [2023-12-27 08:05:25,801][20381] Updated weights for policy 1, policy_version 353469 (0.0008) [2023-12-27 08:05:25,854][20381] Updated weights for policy 1, policy_version 353479 (0.0005) [2023-12-27 08:05:25,910][20381] Updated weights for policy 1, policy_version 353489 (0.0008) [2023-12-27 08:05:25,952][20347] Updated weights for policy 0, policy_version 353667 (0.0009) [2023-12-27 08:05:26,008][20347] Updated weights for policy 0, policy_version 353677 (0.0008) [2023-12-27 08:05:26,065][20347] Updated weights for policy 0, policy_version 353687 (0.0007) [2023-12-27 08:05:26,540][20381] Updated weights for policy 1, policy_version 353499 (0.0007) [2023-12-27 08:05:26,593][20381] Updated weights for policy 1, policy_version 353509 (0.0005) [2023-12-27 08:05:26,638][20381] Updated weights for policy 1, policy_version 353519 (0.0005) [2023-12-27 08:05:26,670][20347] Updated weights for policy 0, policy_version 353697 (0.0006) [2023-12-27 08:05:26,726][20347] Updated weights for policy 0, policy_version 353707 (0.0006) [2023-12-27 08:05:26,774][20347] Updated weights for policy 0, policy_version 353717 (0.0005) [2023-12-27 08:05:26,832][20347] Updated weights for policy 0, policy_version 353727 (0.0005) [2023-12-27 08:05:27,268][20381] Updated weights for policy 1, policy_version 353529 (0.0005) [2023-12-27 08:05:27,330][20381] Updated weights for policy 1, policy_version 353539 (0.0005) [2023-12-27 08:05:27,358][20347] Updated weights for policy 0, policy_version 353737 (0.0007) [2023-12-27 08:05:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 181092352. Throughput: 0: 10545.6, 1: 10220.1. Samples: 181086348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:27,386][20381] Updated weights for policy 1, policy_version 353549 (0.0005) [2023-12-27 08:05:27,412][20347] Updated weights for policy 0, policy_version 353747 (0.0007) [2023-12-27 08:05:27,443][20381] Updated weights for policy 1, policy_version 353559 (0.0005) [2023-12-27 08:05:27,460][20347] Updated weights for policy 0, policy_version 353757 (0.0007) [2023-12-27 08:05:28,037][20381] Updated weights for policy 1, policy_version 353569 (0.0006) [2023-12-27 08:05:28,094][20381] Updated weights for policy 1, policy_version 353579 (0.0005) [2023-12-27 08:05:28,148][20347] Updated weights for policy 0, policy_version 353767 (0.0007) [2023-12-27 08:05:28,184][20381] Updated weights for policy 1, policy_version 353589 (0.0005) [2023-12-27 08:05:28,205][20347] Updated weights for policy 0, policy_version 353777 (0.0007) [2023-12-27 08:05:28,262][20347] Updated weights for policy 0, policy_version 353787 (0.0008) [2023-12-27 08:05:28,832][20381] Updated weights for policy 1, policy_version 353599 (0.0006) [2023-12-27 08:05:28,855][20347] Updated weights for policy 0, policy_version 353797 (0.0007) [2023-12-27 08:05:28,886][20381] Updated weights for policy 1, policy_version 353609 (0.0006) [2023-12-27 08:05:28,906][20347] Updated weights for policy 0, policy_version 353807 (0.0007) [2023-12-27 08:05:28,937][20381] Updated weights for policy 1, policy_version 353619 (0.0005) [2023-12-27 08:05:28,960][20347] Updated weights for policy 0, policy_version 353817 (0.0007) [2023-12-27 08:05:29,573][20381] Updated weights for policy 1, policy_version 353629 (0.0006) [2023-12-27 08:05:29,623][20381] Updated weights for policy 1, policy_version 353639 (0.0007) [2023-12-27 08:05:29,645][20347] Updated weights for policy 0, policy_version 353827 (0.0007) [2023-12-27 08:05:29,678][20381] Updated weights for policy 1, policy_version 353649 (0.0006) [2023-12-27 08:05:29,693][20347] Updated weights for policy 0, policy_version 353837 (0.0008) [2023-12-27 08:05:29,753][20347] Updated weights for policy 0, policy_version 353847 (0.0006) [2023-12-27 08:05:30,356][20381] Updated weights for policy 1, policy_version 353659 (0.0006) [2023-12-27 08:05:30,412][20381] Updated weights for policy 1, policy_version 353669 (0.0007) [2023-12-27 08:05:30,451][20347] Updated weights for policy 0, policy_version 353857 (0.0006) [2023-12-27 08:05:30,465][20381] Updated weights for policy 1, policy_version 353679 (0.0007) [2023-12-27 08:05:30,508][20347] Updated weights for policy 0, policy_version 353867 (0.0007) [2023-12-27 08:05:30,570][20347] Updated weights for policy 0, policy_version 353877 (0.0007) [2023-12-27 08:05:30,621][20347] Updated weights for policy 0, policy_version 353887 (0.0008) [2023-12-27 08:05:31,017][20381] Updated weights for policy 1, policy_version 353689 (0.0006) [2023-12-27 08:05:31,077][20381] Updated weights for policy 1, policy_version 353699 (0.0007) [2023-12-27 08:05:31,140][20381] Updated weights for policy 1, policy_version 353709 (0.0008) [2023-12-27 08:05:31,193][20381] Updated weights for policy 1, policy_version 353719 (0.0008) [2023-12-27 08:05:31,339][20347] Updated weights for policy 0, policy_version 353897 (0.0007) [2023-12-27 08:05:31,394][20347] Updated weights for policy 0, policy_version 353907 (0.0007) [2023-12-27 08:05:31,445][20347] Updated weights for policy 0, policy_version 353917 (0.0005) [2023-12-27 08:05:31,862][20381] Updated weights for policy 1, policy_version 353729 (0.0007) [2023-12-27 08:05:31,913][20381] Updated weights for policy 1, policy_version 353739 (0.0009) [2023-12-27 08:05:31,973][20381] Updated weights for policy 1, policy_version 353749 (0.0007) [2023-12-27 08:05:32,042][20347] Updated weights for policy 0, policy_version 353927 (0.0007) [2023-12-27 08:05:32,101][20347] Updated weights for policy 0, policy_version 353937 (0.0008) [2023-12-27 08:05:32,161][20347] Updated weights for policy 0, policy_version 353947 (0.0008) [2023-12-27 08:05:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 181207040. Throughput: 0: 10582.1, 1: 10283.7. Samples: 181215860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:32,669][20381] Updated weights for policy 1, policy_version 353759 (0.0008) [2023-12-27 08:05:32,716][20381] Updated weights for policy 1, policy_version 353769 (0.0008) [2023-12-27 08:05:32,765][20381] Updated weights for policy 1, policy_version 353779 (0.0008) [2023-12-27 08:05:32,816][20347] Updated weights for policy 0, policy_version 353957 (0.0006) [2023-12-27 08:05:32,872][20347] Updated weights for policy 0, policy_version 353967 (0.0005) [2023-12-27 08:05:32,926][20347] Updated weights for policy 0, policy_version 353977 (0.0006) [2023-12-27 08:05:33,372][20381] Updated weights for policy 1, policy_version 353789 (0.0008) [2023-12-27 08:05:33,431][20381] Updated weights for policy 1, policy_version 353799 (0.0007) [2023-12-27 08:05:33,484][20381] Updated weights for policy 1, policy_version 353809 (0.0007) [2023-12-27 08:05:33,526][20347] Updated weights for policy 0, policy_version 353987 (0.0008) [2023-12-27 08:05:33,572][20347] Updated weights for policy 0, policy_version 353997 (0.0007) [2023-12-27 08:05:33,628][20347] Updated weights for policy 0, policy_version 354007 (0.0007) [2023-12-27 08:05:34,091][20381] Updated weights for policy 1, policy_version 353819 (0.0006) [2023-12-27 08:05:34,157][20381] Updated weights for policy 1, policy_version 353829 (0.0007) [2023-12-27 08:05:34,222][20381] Updated weights for policy 1, policy_version 353839 (0.0008) [2023-12-27 08:05:34,253][20347] Updated weights for policy 0, policy_version 354017 (0.0007) [2023-12-27 08:05:34,308][20347] Updated weights for policy 0, policy_version 354027 (0.0008) [2023-12-27 08:05:34,366][20347] Updated weights for policy 0, policy_version 354037 (0.0008) [2023-12-27 08:05:34,424][20347] Updated weights for policy 0, policy_version 354047 (0.0008) [2023-12-27 08:05:34,919][20381] Updated weights for policy 1, policy_version 353849 (0.0009) [2023-12-27 08:05:34,978][20381] Updated weights for policy 1, policy_version 353859 (0.0007) [2023-12-27 08:05:35,035][20381] Updated weights for policy 1, policy_version 353869 (0.0008) [2023-12-27 08:05:35,057][20347] Updated weights for policy 0, policy_version 354057 (0.0010) [2023-12-27 08:05:35,091][20381] Updated weights for policy 1, policy_version 353879 (0.0007) [2023-12-27 08:05:35,116][20347] Updated weights for policy 0, policy_version 354067 (0.0009) [2023-12-27 08:05:35,179][20347] Updated weights for policy 0, policy_version 354077 (0.0007) [2023-12-27 08:05:35,795][20381] Updated weights for policy 1, policy_version 353889 (0.0005) [2023-12-27 08:05:35,816][20347] Updated weights for policy 0, policy_version 354087 (0.0009) [2023-12-27 08:05:35,843][20381] Updated weights for policy 1, policy_version 353899 (0.0005) [2023-12-27 08:05:35,873][20347] Updated weights for policy 0, policy_version 354097 (0.0005) [2023-12-27 08:05:35,895][20381] Updated weights for policy 1, policy_version 353909 (0.0007) [2023-12-27 08:05:35,930][20347] Updated weights for policy 0, policy_version 354107 (0.0005) [2023-12-27 08:05:36,517][20381] Updated weights for policy 1, policy_version 353919 (0.0007) [2023-12-27 08:05:36,520][20347] Updated weights for policy 0, policy_version 354117 (0.0006) [2023-12-27 08:05:36,571][20381] Updated weights for policy 1, policy_version 353929 (0.0008) [2023-12-27 08:05:36,586][20347] Updated weights for policy 0, policy_version 354127 (0.0006) [2023-12-27 08:05:36,625][20381] Updated weights for policy 1, policy_version 353939 (0.0008) [2023-12-27 08:05:36,647][20347] Updated weights for policy 0, policy_version 354137 (0.0007) [2023-12-27 08:05:37,263][20347] Updated weights for policy 0, policy_version 354147 (0.0010) [2023-12-27 08:05:37,276][20381] Updated weights for policy 1, policy_version 353949 (0.0006) [2023-12-27 08:05:37,313][20347] Updated weights for policy 0, policy_version 354157 (0.0011) [2023-12-27 08:05:37,332][20381] Updated weights for policy 1, policy_version 353959 (0.0010) [2023-12-27 08:05:37,359][20347] Updated weights for policy 0, policy_version 354167 (0.0010) [2023-12-27 08:05:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 181305344. Throughput: 0: 10607.8, 1: 10303.1. Samples: 181280248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:37,386][20381] Updated weights for policy 1, policy_version 353969 (0.0008) [2023-12-27 08:05:37,404][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000354176_90685440.pth... [2023-12-27 08:05:37,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000352928_90365952.pth [2023-12-27 08:05:37,418][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000353976_90636288.pth... [2023-12-27 08:05:37,422][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000352760_90324992.pth [2023-12-27 08:05:37,961][20347] Updated weights for policy 0, policy_version 354177 (0.0010) [2023-12-27 08:05:38,024][20347] Updated weights for policy 0, policy_version 354187 (0.0006) [2023-12-27 08:05:38,088][20347] Updated weights for policy 0, policy_version 354197 (0.0005) [2023-12-27 08:05:38,090][20381] Updated weights for policy 1, policy_version 353979 (0.0006) [2023-12-27 08:05:38,147][20347] Updated weights for policy 0, policy_version 354207 (0.0007) [2023-12-27 08:05:38,149][20381] Updated weights for policy 1, policy_version 353989 (0.0005) [2023-12-27 08:05:38,201][20381] Updated weights for policy 1, policy_version 353999 (0.0008) [2023-12-27 08:05:38,730][20347] Updated weights for policy 0, policy_version 354217 (0.0008) [2023-12-27 08:05:38,791][20347] Updated weights for policy 0, policy_version 354227 (0.0007) [2023-12-27 08:05:38,847][20347] Updated weights for policy 0, policy_version 354237 (0.0010) [2023-12-27 08:05:38,920][20381] Updated weights for policy 1, policy_version 354009 (0.0008) [2023-12-27 08:05:38,977][20381] Updated weights for policy 1, policy_version 354019 (0.0007) [2023-12-27 08:05:39,033][20381] Updated weights for policy 1, policy_version 354029 (0.0005) [2023-12-27 08:05:39,091][20381] Updated weights for policy 1, policy_version 354039 (0.0006) [2023-12-27 08:05:39,489][20347] Updated weights for policy 0, policy_version 354247 (0.0010) [2023-12-27 08:05:39,538][20347] Updated weights for policy 0, policy_version 354257 (0.0010) [2023-12-27 08:05:39,597][20347] Updated weights for policy 0, policy_version 354267 (0.0010) [2023-12-27 08:05:39,758][20381] Updated weights for policy 1, policy_version 354049 (0.0006) [2023-12-27 08:05:39,815][20381] Updated weights for policy 1, policy_version 354059 (0.0005) [2023-12-27 08:05:39,877][20381] Updated weights for policy 1, policy_version 354069 (0.0009) [2023-12-27 08:05:40,273][20347] Updated weights for policy 0, policy_version 354277 (0.0009) [2023-12-27 08:05:40,329][20347] Updated weights for policy 0, policy_version 354287 (0.0011) [2023-12-27 08:05:40,394][20347] Updated weights for policy 0, policy_version 354297 (0.0008) [2023-12-27 08:05:40,497][20381] Updated weights for policy 1, policy_version 354079 (0.0008) [2023-12-27 08:05:40,551][20381] Updated weights for policy 1, policy_version 354089 (0.0010) [2023-12-27 08:05:40,612][20381] Updated weights for policy 1, policy_version 354099 (0.0009) [2023-12-27 08:05:41,044][20347] Updated weights for policy 0, policy_version 354307 (0.0006) [2023-12-27 08:05:41,104][20347] Updated weights for policy 0, policy_version 354317 (0.0007) [2023-12-27 08:05:41,164][20347] Updated weights for policy 0, policy_version 354327 (0.0009) [2023-12-27 08:05:41,286][20381] Updated weights for policy 1, policy_version 354109 (0.0007) [2023-12-27 08:05:41,345][20381] Updated weights for policy 1, policy_version 354119 (0.0006) [2023-12-27 08:05:41,406][20381] Updated weights for policy 1, policy_version 354129 (0.0008) [2023-12-27 08:05:41,887][20347] Updated weights for policy 0, policy_version 354337 (0.0010) [2023-12-27 08:05:41,950][20347] Updated weights for policy 0, policy_version 354347 (0.0005) [2023-12-27 08:05:42,013][20347] Updated weights for policy 0, policy_version 354357 (0.0006) [2023-12-27 08:05:42,073][20347] Updated weights for policy 0, policy_version 354367 (0.0007) [2023-12-27 08:05:42,117][20381] Updated weights for policy 1, policy_version 354139 (0.0007) [2023-12-27 08:05:42,181][20381] Updated weights for policy 1, policy_version 354149 (0.0007) [2023-12-27 08:05:42,239][20381] Updated weights for policy 1, policy_version 354159 (0.0008) [2023-12-27 08:05:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 181420032. Throughput: 0: 10630.0, 1: 10298.2. Samples: 181407536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:42,783][20347] Updated weights for policy 0, policy_version 354377 (0.0008) [2023-12-27 08:05:42,842][20347] Updated weights for policy 0, policy_version 354387 (0.0007) [2023-12-27 08:05:42,890][20347] Updated weights for policy 0, policy_version 354397 (0.0005) [2023-12-27 08:05:42,954][20381] Updated weights for policy 1, policy_version 354169 (0.0008) [2023-12-27 08:05:43,013][20381] Updated weights for policy 1, policy_version 354179 (0.0008) [2023-12-27 08:05:43,073][20381] Updated weights for policy 1, policy_version 354189 (0.0008) [2023-12-27 08:05:43,129][20381] Updated weights for policy 1, policy_version 354199 (0.0007) [2023-12-27 08:05:43,482][20347] Updated weights for policy 0, policy_version 354407 (0.0009) [2023-12-27 08:05:43,542][20347] Updated weights for policy 0, policy_version 354417 (0.0011) [2023-12-27 08:05:43,594][20347] Updated weights for policy 0, policy_version 354427 (0.0010) [2023-12-27 08:05:43,871][20381] Updated weights for policy 1, policy_version 354209 (0.0008) [2023-12-27 08:05:43,932][20381] Updated weights for policy 1, policy_version 354219 (0.0005) [2023-12-27 08:05:43,990][20381] Updated weights for policy 1, policy_version 354229 (0.0006) [2023-12-27 08:05:44,272][20347] Updated weights for policy 0, policy_version 354437 (0.0009) [2023-12-27 08:05:44,330][20347] Updated weights for policy 0, policy_version 354447 (0.0007) [2023-12-27 08:05:44,380][20347] Updated weights for policy 0, policy_version 354457 (0.0007) [2023-12-27 08:05:44,667][20381] Updated weights for policy 1, policy_version 354239 (0.0008) [2023-12-27 08:05:44,729][20381] Updated weights for policy 1, policy_version 354249 (0.0009) [2023-12-27 08:05:44,793][20381] Updated weights for policy 1, policy_version 354259 (0.0008) [2023-12-27 08:05:44,950][20347] Updated weights for policy 0, policy_version 354467 (0.0005) [2023-12-27 08:05:45,024][20347] Updated weights for policy 0, policy_version 354477 (0.0009) [2023-12-27 08:05:45,098][20347] Updated weights for policy 0, policy_version 354487 (0.0009) [2023-12-27 08:05:45,398][20381] Updated weights for policy 1, policy_version 354269 (0.0007) [2023-12-27 08:05:45,460][20381] Updated weights for policy 1, policy_version 354279 (0.0006) [2023-12-27 08:05:45,506][20381] Updated weights for policy 1, policy_version 354289 (0.0005) [2023-12-27 08:05:45,785][20347] Updated weights for policy 0, policy_version 354497 (0.0008) [2023-12-27 08:05:45,833][20347] Updated weights for policy 0, policy_version 354507 (0.0007) [2023-12-27 08:05:45,879][20347] Updated weights for policy 0, policy_version 354517 (0.0008) [2023-12-27 08:05:45,924][20347] Updated weights for policy 0, policy_version 354527 (0.0007) [2023-12-27 08:05:46,143][20381] Updated weights for policy 1, policy_version 354299 (0.0006) [2023-12-27 08:05:46,206][20381] Updated weights for policy 1, policy_version 354309 (0.0008) [2023-12-27 08:05:46,257][20381] Updated weights for policy 1, policy_version 354319 (0.0008) [2023-12-27 08:05:46,625][20347] Updated weights for policy 0, policy_version 354537 (0.0009) [2023-12-27 08:05:46,678][20347] Updated weights for policy 0, policy_version 354547 (0.0007) [2023-12-27 08:05:46,734][20347] Updated weights for policy 0, policy_version 354557 (0.0009) [2023-12-27 08:05:46,903][20381] Updated weights for policy 1, policy_version 354329 (0.0008) [2023-12-27 08:05:46,960][20381] Updated weights for policy 1, policy_version 354339 (0.0011) [2023-12-27 08:05:47,021][20381] Updated weights for policy 1, policy_version 354349 (0.0010) [2023-12-27 08:05:47,076][20381] Updated weights for policy 1, policy_version 354359 (0.0011) [2023-12-27 08:05:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 181518336. Throughput: 0: 10610.7, 1: 10302.6. Samples: 181529592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:47,470][20347] Updated weights for policy 0, policy_version 354567 (0.0006) [2023-12-27 08:05:47,526][20347] Updated weights for policy 0, policy_version 354577 (0.0005) [2023-12-27 08:05:47,582][20347] Updated weights for policy 0, policy_version 354587 (0.0005) [2023-12-27 08:05:47,672][20381] Updated weights for policy 1, policy_version 354369 (0.0006) [2023-12-27 08:05:47,721][20381] Updated weights for policy 1, policy_version 354379 (0.0006) [2023-12-27 08:05:47,767][20381] Updated weights for policy 1, policy_version 354389 (0.0005) [2023-12-27 08:05:48,233][20347] Updated weights for policy 0, policy_version 354597 (0.0007) [2023-12-27 08:05:48,292][20347] Updated weights for policy 0, policy_version 354607 (0.0007) [2023-12-27 08:05:48,352][20347] Updated weights for policy 0, policy_version 354617 (0.0007) [2023-12-27 08:05:48,354][20381] Updated weights for policy 1, policy_version 354399 (0.0008) [2023-12-27 08:05:48,420][20381] Updated weights for policy 1, policy_version 354409 (0.0011) [2023-12-27 08:05:48,484][20381] Updated weights for policy 1, policy_version 354419 (0.0010) [2023-12-27 08:05:48,988][20347] Updated weights for policy 0, policy_version 354627 (0.0006) [2023-12-27 08:05:49,051][20347] Updated weights for policy 0, policy_version 354637 (0.0007) [2023-12-27 08:05:49,101][20381] Updated weights for policy 1, policy_version 354429 (0.0008) [2023-12-27 08:05:49,110][20347] Updated weights for policy 0, policy_version 354647 (0.0006) [2023-12-27 08:05:49,166][20381] Updated weights for policy 1, policy_version 354439 (0.0010) [2023-12-27 08:05:49,239][20381] Updated weights for policy 1, policy_version 354449 (0.0010) [2023-12-27 08:05:49,723][20347] Updated weights for policy 0, policy_version 354657 (0.0008) [2023-12-27 08:05:49,768][20347] Updated weights for policy 0, policy_version 354667 (0.0007) [2023-12-27 08:05:49,818][20347] Updated weights for policy 0, policy_version 354677 (0.0007) [2023-12-27 08:05:49,880][20347] Updated weights for policy 0, policy_version 354687 (0.0008) [2023-12-27 08:05:49,928][20381] Updated weights for policy 1, policy_version 354459 (0.0008) [2023-12-27 08:05:49,993][20381] Updated weights for policy 1, policy_version 354469 (0.0011) [2023-12-27 08:05:50,052][20381] Updated weights for policy 1, policy_version 354479 (0.0011) [2023-12-27 08:05:50,659][20347] Updated weights for policy 0, policy_version 354697 (0.0009) [2023-12-27 08:05:50,729][20381] Updated weights for policy 1, policy_version 354489 (0.0010) [2023-12-27 08:05:50,733][20347] Updated weights for policy 0, policy_version 354707 (0.0007) [2023-12-27 08:05:50,785][20381] Updated weights for policy 1, policy_version 354499 (0.0005) [2023-12-27 08:05:50,800][20347] Updated weights for policy 0, policy_version 354717 (0.0005) [2023-12-27 08:05:50,843][20381] Updated weights for policy 1, policy_version 354509 (0.0006) [2023-12-27 08:05:50,907][20381] Updated weights for policy 1, policy_version 354519 (0.0008) [2023-12-27 08:05:51,454][20347] Updated weights for policy 0, policy_version 354727 (0.0007) [2023-12-27 08:05:51,518][20347] Updated weights for policy 0, policy_version 354737 (0.0011) [2023-12-27 08:05:51,568][20347] Updated weights for policy 0, policy_version 354747 (0.0007) [2023-12-27 08:05:51,624][20381] Updated weights for policy 1, policy_version 354529 (0.0008) [2023-12-27 08:05:51,688][20381] Updated weights for policy 1, policy_version 354539 (0.0008) [2023-12-27 08:05:51,752][20381] Updated weights for policy 1, policy_version 354549 (0.0008) [2023-12-27 08:05:52,294][20347] Updated weights for policy 0, policy_version 354757 (0.0009) [2023-12-27 08:05:52,343][20347] Updated weights for policy 0, policy_version 354767 (0.0009) [2023-12-27 08:05:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 181616640. Throughput: 0: 10547.8, 1: 10375.7. Samples: 181593388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000354552_90783744.pth... [2023-12-27 08:05:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000353336_90472448.pth [2023-12-27 08:05:52,408][20347] Updated weights for policy 0, policy_version 354777 (0.0006) [2023-12-27 08:05:52,445][20381] Updated weights for policy 1, policy_version 354559 (0.0008) [2023-12-27 08:05:52,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000354784_90841088.pth... [2023-12-27 08:05:52,452][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000353536_90521600.pth [2023-12-27 08:05:52,505][20381] Updated weights for policy 1, policy_version 354569 (0.0011) [2023-12-27 08:05:52,566][20381] Updated weights for policy 1, policy_version 354579 (0.0010) [2023-12-27 08:05:53,096][20347] Updated weights for policy 0, policy_version 354787 (0.0009) [2023-12-27 08:05:53,148][20347] Updated weights for policy 0, policy_version 354797 (0.0008) [2023-12-27 08:05:53,196][20347] Updated weights for policy 0, policy_version 354807 (0.0008) [2023-12-27 08:05:53,228][20381] Updated weights for policy 1, policy_version 354589 (0.0008) [2023-12-27 08:05:53,284][20381] Updated weights for policy 1, policy_version 354599 (0.0006) [2023-12-27 08:05:53,340][20381] Updated weights for policy 1, policy_version 354609 (0.0006) [2023-12-27 08:05:53,850][20347] Updated weights for policy 0, policy_version 354817 (0.0007) [2023-12-27 08:05:53,915][20347] Updated weights for policy 0, policy_version 354827 (0.0008) [2023-12-27 08:05:53,935][20381] Updated weights for policy 1, policy_version 354619 (0.0007) [2023-12-27 08:05:53,969][20347] Updated weights for policy 0, policy_version 354837 (0.0006) [2023-12-27 08:05:53,991][20381] Updated weights for policy 1, policy_version 354629 (0.0005) [2023-12-27 08:05:54,016][20347] Updated weights for policy 0, policy_version 354847 (0.0006) [2023-12-27 08:05:54,052][20381] Updated weights for policy 1, policy_version 354639 (0.0005) [2023-12-27 08:05:54,598][20381] Updated weights for policy 1, policy_version 354649 (0.0008) [2023-12-27 08:05:54,637][20347] Updated weights for policy 0, policy_version 354857 (0.0005) [2023-12-27 08:05:54,653][20381] Updated weights for policy 1, policy_version 354659 (0.0008) [2023-12-27 08:05:54,692][20347] Updated weights for policy 0, policy_version 354867 (0.0006) [2023-12-27 08:05:54,705][20381] Updated weights for policy 1, policy_version 354669 (0.0008) [2023-12-27 08:05:54,744][20347] Updated weights for policy 0, policy_version 354877 (0.0008) [2023-12-27 08:05:54,760][20381] Updated weights for policy 1, policy_version 354679 (0.0005) [2023-12-27 08:05:55,425][20347] Updated weights for policy 0, policy_version 354887 (0.0006) [2023-12-27 08:05:55,449][20381] Updated weights for policy 1, policy_version 354689 (0.0005) [2023-12-27 08:05:55,476][20347] Updated weights for policy 0, policy_version 354897 (0.0006) [2023-12-27 08:05:55,497][20381] Updated weights for policy 1, policy_version 354699 (0.0005) [2023-12-27 08:05:55,528][20347] Updated weights for policy 0, policy_version 354907 (0.0007) [2023-12-27 08:05:55,549][20381] Updated weights for policy 1, policy_version 354709 (0.0006) [2023-12-27 08:05:56,201][20381] Updated weights for policy 1, policy_version 354719 (0.0006) [2023-12-27 08:05:56,206][20347] Updated weights for policy 0, policy_version 354917 (0.0007) [2023-12-27 08:05:56,263][20381] Updated weights for policy 1, policy_version 354729 (0.0006) [2023-12-27 08:05:56,264][20347] Updated weights for policy 0, policy_version 354927 (0.0007) [2023-12-27 08:05:56,317][20381] Updated weights for policy 1, policy_version 354739 (0.0005) [2023-12-27 08:05:56,317][20347] Updated weights for policy 0, policy_version 354937 (0.0008) [2023-12-27 08:05:56,914][20381] Updated weights for policy 1, policy_version 354749 (0.0005) [2023-12-27 08:05:56,969][20381] Updated weights for policy 1, policy_version 354759 (0.0008) [2023-12-27 08:05:56,987][20347] Updated weights for policy 0, policy_version 354947 (0.0007) [2023-12-27 08:05:57,013][20381] Updated weights for policy 1, policy_version 354769 (0.0010) [2023-12-27 08:05:57,031][20347] Updated weights for policy 0, policy_version 354957 (0.0005) [2023-12-27 08:05:57,077][20347] Updated weights for policy 0, policy_version 354967 (0.0006) [2023-12-27 08:05:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 181731328. Throughput: 0: 10518.6, 1: 10372.4. Samples: 181717992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:05:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:05:57,639][20381] Updated weights for policy 1, policy_version 354779 (0.0009) [2023-12-27 08:05:57,685][20381] Updated weights for policy 1, policy_version 354789 (0.0005) [2023-12-27 08:05:57,686][20347] Updated weights for policy 0, policy_version 354977 (0.0007) [2023-12-27 08:05:57,737][20347] Updated weights for policy 0, policy_version 354987 (0.0007) [2023-12-27 08:05:57,741][20381] Updated weights for policy 1, policy_version 354799 (0.0007) [2023-12-27 08:05:57,785][20347] Updated weights for policy 0, policy_version 354997 (0.0005) [2023-12-27 08:05:57,842][20347] Updated weights for policy 0, policy_version 355007 (0.0007) [2023-12-27 08:05:58,313][20381] Updated weights for policy 1, policy_version 354809 (0.0010) [2023-12-27 08:05:58,378][20381] Updated weights for policy 1, policy_version 354819 (0.0009) [2023-12-27 08:05:58,446][20381] Updated weights for policy 1, policy_version 354829 (0.0008) [2023-12-27 08:05:58,513][20381] Updated weights for policy 1, policy_version 354839 (0.0008) [2023-12-27 08:05:58,529][20347] Updated weights for policy 0, policy_version 355017 (0.0007) [2023-12-27 08:05:58,587][20347] Updated weights for policy 0, policy_version 355027 (0.0007) [2023-12-27 08:05:58,651][20347] Updated weights for policy 0, policy_version 355037 (0.0008) [2023-12-27 08:05:59,315][20381] Updated weights for policy 1, policy_version 354849 (0.0007) [2023-12-27 08:05:59,380][20381] Updated weights for policy 1, policy_version 354859 (0.0006) [2023-12-27 08:05:59,402][20347] Updated weights for policy 0, policy_version 355047 (0.0006) [2023-12-27 08:05:59,437][20381] Updated weights for policy 1, policy_version 354869 (0.0007) [2023-12-27 08:05:59,455][20347] Updated weights for policy 0, policy_version 355057 (0.0006) [2023-12-27 08:05:59,511][20347] Updated weights for policy 0, policy_version 355067 (0.0005) [2023-12-27 08:06:00,076][20381] Updated weights for policy 1, policy_version 354879 (0.0008) [2023-12-27 08:06:00,133][20381] Updated weights for policy 1, policy_version 354889 (0.0008) [2023-12-27 08:06:00,182][20347] Updated weights for policy 0, policy_version 355077 (0.0005) [2023-12-27 08:06:00,188][20381] Updated weights for policy 1, policy_version 354899 (0.0008) [2023-12-27 08:06:00,237][20347] Updated weights for policy 0, policy_version 355087 (0.0005) [2023-12-27 08:06:00,285][20347] Updated weights for policy 0, policy_version 355097 (0.0006) [2023-12-27 08:06:00,784][20381] Updated weights for policy 1, policy_version 354909 (0.0006) [2023-12-27 08:06:00,834][20381] Updated weights for policy 1, policy_version 354919 (0.0006) [2023-12-27 08:06:00,872][20347] Updated weights for policy 0, policy_version 355107 (0.0006) [2023-12-27 08:06:00,892][20381] Updated weights for policy 1, policy_version 354929 (0.0005) [2023-12-27 08:06:00,936][20347] Updated weights for policy 0, policy_version 355117 (0.0008) [2023-12-27 08:06:00,996][20347] Updated weights for policy 0, policy_version 355127 (0.0008) [2023-12-27 08:06:01,520][20381] Updated weights for policy 1, policy_version 354939 (0.0007) [2023-12-27 08:06:01,568][20381] Updated weights for policy 1, policy_version 354949 (0.0009) [2023-12-27 08:06:01,623][20381] Updated weights for policy 1, policy_version 354959 (0.0007) [2023-12-27 08:06:01,654][20347] Updated weights for policy 0, policy_version 355137 (0.0008) [2023-12-27 08:06:01,707][20347] Updated weights for policy 0, policy_version 355147 (0.0005) [2023-12-27 08:06:01,767][20347] Updated weights for policy 0, policy_version 355157 (0.0007) [2023-12-27 08:06:01,812][20347] Updated weights for policy 0, policy_version 355167 (0.0007) [2023-12-27 08:06:02,307][20381] Updated weights for policy 1, policy_version 354969 (0.0007) [2023-12-27 08:06:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 181829632. Throughput: 0: 10554.9, 1: 10462.5. Samples: 181844668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:02,371][20381] Updated weights for policy 1, policy_version 354979 (0.0007) [2023-12-27 08:06:02,428][20381] Updated weights for policy 1, policy_version 354989 (0.0008) [2023-12-27 08:06:02,481][20381] Updated weights for policy 1, policy_version 354999 (0.0007) [2023-12-27 08:06:02,506][20347] Updated weights for policy 0, policy_version 355177 (0.0006) [2023-12-27 08:06:02,560][20347] Updated weights for policy 0, policy_version 355187 (0.0007) [2023-12-27 08:06:02,619][20347] Updated weights for policy 0, policy_version 355197 (0.0008) [2023-12-27 08:06:03,212][20381] Updated weights for policy 1, policy_version 355009 (0.0007) [2023-12-27 08:06:03,257][20347] Updated weights for policy 0, policy_version 355207 (0.0007) [2023-12-27 08:06:03,266][20381] Updated weights for policy 1, policy_version 355019 (0.0007) [2023-12-27 08:06:03,310][20347] Updated weights for policy 0, policy_version 355217 (0.0005) [2023-12-27 08:06:03,323][20381] Updated weights for policy 1, policy_version 355029 (0.0008) [2023-12-27 08:06:03,360][20347] Updated weights for policy 0, policy_version 355227 (0.0006) [2023-12-27 08:06:03,966][20381] Updated weights for policy 1, policy_version 355039 (0.0008) [2023-12-27 08:06:04,028][20381] Updated weights for policy 1, policy_version 355049 (0.0007) [2023-12-27 08:06:04,083][20381] Updated weights for policy 1, policy_version 355059 (0.0008) [2023-12-27 08:06:04,101][20347] Updated weights for policy 0, policy_version 355237 (0.0006) [2023-12-27 08:06:04,161][20347] Updated weights for policy 0, policy_version 355247 (0.0007) [2023-12-27 08:06:04,223][20347] Updated weights for policy 0, policy_version 355257 (0.0008) [2023-12-27 08:06:04,726][20381] Updated weights for policy 1, policy_version 355069 (0.0007) [2023-12-27 08:06:04,786][20381] Updated weights for policy 1, policy_version 355079 (0.0008) [2023-12-27 08:06:04,847][20381] Updated weights for policy 1, policy_version 355089 (0.0007) [2023-12-27 08:06:04,914][20347] Updated weights for policy 0, policy_version 355267 (0.0007) [2023-12-27 08:06:04,962][20347] Updated weights for policy 0, policy_version 355277 (0.0007) [2023-12-27 08:06:05,006][20347] Updated weights for policy 0, policy_version 355287 (0.0007) [2023-12-27 08:06:05,520][20381] Updated weights for policy 1, policy_version 355099 (0.0007) [2023-12-27 08:06:05,578][20381] Updated weights for policy 1, policy_version 355109 (0.0008) [2023-12-27 08:06:05,623][20381] Updated weights for policy 1, policy_version 355119 (0.0007) [2023-12-27 08:06:05,722][20347] Updated weights for policy 0, policy_version 355297 (0.0007) [2023-12-27 08:06:05,776][20347] Updated weights for policy 0, policy_version 355307 (0.0007) [2023-12-27 08:06:05,824][20347] Updated weights for policy 0, policy_version 355317 (0.0008) [2023-12-27 08:06:05,875][20347] Updated weights for policy 0, policy_version 355327 (0.0007) [2023-12-27 08:06:06,284][20381] Updated weights for policy 1, policy_version 355129 (0.0007) [2023-12-27 08:06:06,339][20381] Updated weights for policy 1, policy_version 355139 (0.0007) [2023-12-27 08:06:06,394][20381] Updated weights for policy 1, policy_version 355149 (0.0007) [2023-12-27 08:06:06,444][20381] Updated weights for policy 1, policy_version 355159 (0.0007) [2023-12-27 08:06:06,587][20347] Updated weights for policy 0, policy_version 355337 (0.0010) [2023-12-27 08:06:06,647][20347] Updated weights for policy 0, policy_version 355347 (0.0011) [2023-12-27 08:06:06,712][20347] Updated weights for policy 0, policy_version 355357 (0.0011) [2023-12-27 08:06:07,142][20381] Updated weights for policy 1, policy_version 355169 (0.0006) [2023-12-27 08:06:07,191][20381] Updated weights for policy 1, policy_version 355179 (0.0007) [2023-12-27 08:06:07,237][20381] Updated weights for policy 1, policy_version 355189 (0.0006) [2023-12-27 08:06:07,293][20347] Updated weights for policy 0, policy_version 355367 (0.0008) [2023-12-27 08:06:07,356][20347] Updated weights for policy 0, policy_version 355377 (0.0005) [2023-12-27 08:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 181936128. Throughput: 0: 10490.6, 1: 10487.5. Samples: 181905476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000355192_90947584.pth... [2023-12-27 08:06:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000353976_90636288.pth [2023-12-27 08:06:07,424][20347] Updated weights for policy 0, policy_version 355387 (0.0005) [2023-12-27 08:06:07,450][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000355392_90996736.pth... [2023-12-27 08:06:07,453][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000354176_90685440.pth [2023-12-27 08:06:07,847][20381] Updated weights for policy 1, policy_version 355199 (0.0008) [2023-12-27 08:06:07,904][20381] Updated weights for policy 1, policy_version 355209 (0.0008) [2023-12-27 08:06:07,957][20381] Updated weights for policy 1, policy_version 355219 (0.0008) [2023-12-27 08:06:07,977][20347] Updated weights for policy 0, policy_version 355397 (0.0008) [2023-12-27 08:06:08,035][20347] Updated weights for policy 0, policy_version 355407 (0.0007) [2023-12-27 08:06:08,080][20347] Updated weights for policy 0, policy_version 355417 (0.0008) [2023-12-27 08:06:08,626][20381] Updated weights for policy 1, policy_version 355229 (0.0006) [2023-12-27 08:06:08,673][20381] Updated weights for policy 1, policy_version 355239 (0.0007) [2023-12-27 08:06:08,729][20381] Updated weights for policy 1, policy_version 355249 (0.0008) [2023-12-27 08:06:08,781][20347] Updated weights for policy 0, policy_version 355427 (0.0007) [2023-12-27 08:06:08,834][20347] Updated weights for policy 0, policy_version 355437 (0.0007) [2023-12-27 08:06:08,887][20347] Updated weights for policy 0, policy_version 355447 (0.0008) [2023-12-27 08:06:09,453][20381] Updated weights for policy 1, policy_version 355259 (0.0007) [2023-12-27 08:06:09,505][20381] Updated weights for policy 1, policy_version 355269 (0.0008) [2023-12-27 08:06:09,554][20381] Updated weights for policy 1, policy_version 355279 (0.0008) [2023-12-27 08:06:09,587][20347] Updated weights for policy 0, policy_version 355457 (0.0008) [2023-12-27 08:06:09,648][20347] Updated weights for policy 0, policy_version 355467 (0.0007) [2023-12-27 08:06:09,713][20347] Updated weights for policy 0, policy_version 355477 (0.0007) [2023-12-27 08:06:09,771][20347] Updated weights for policy 0, policy_version 355487 (0.0008) [2023-12-27 08:06:10,303][20381] Updated weights for policy 1, policy_version 355289 (0.0009) [2023-12-27 08:06:10,362][20381] Updated weights for policy 1, policy_version 355299 (0.0011) [2023-12-27 08:06:10,418][20381] Updated weights for policy 1, policy_version 355309 (0.0010) [2023-12-27 08:06:10,444][20347] Updated weights for policy 0, policy_version 355497 (0.0007) [2023-12-27 08:06:10,477][20381] Updated weights for policy 1, policy_version 355319 (0.0011) [2023-12-27 08:06:10,492][20347] Updated weights for policy 0, policy_version 355507 (0.0007) [2023-12-27 08:06:10,548][20347] Updated weights for policy 0, policy_version 355517 (0.0008) [2023-12-27 08:06:11,172][20381] Updated weights for policy 1, policy_version 355329 (0.0007) [2023-12-27 08:06:11,234][20381] Updated weights for policy 1, policy_version 355339 (0.0007) [2023-12-27 08:06:11,273][20347] Updated weights for policy 0, policy_version 355527 (0.0007) [2023-12-27 08:06:11,295][20381] Updated weights for policy 1, policy_version 355349 (0.0007) [2023-12-27 08:06:11,336][20347] Updated weights for policy 0, policy_version 355537 (0.0007) [2023-12-27 08:06:11,396][20347] Updated weights for policy 0, policy_version 355547 (0.0007) [2023-12-27 08:06:11,955][20381] Updated weights for policy 1, policy_version 355359 (0.0009) [2023-12-27 08:06:12,017][20381] Updated weights for policy 1, policy_version 355369 (0.0011) [2023-12-27 08:06:12,033][20347] Updated weights for policy 0, policy_version 355557 (0.0008) [2023-12-27 08:06:12,077][20381] Updated weights for policy 1, policy_version 355379 (0.0011) [2023-12-27 08:06:12,078][20347] Updated weights for policy 0, policy_version 355567 (0.0007) [2023-12-27 08:06:12,126][20347] Updated weights for policy 0, policy_version 355577 (0.0007) [2023-12-27 08:06:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 182042624. Throughput: 0: 10495.3, 1: 10465.4. Samples: 182029576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:12,750][20381] Updated weights for policy 1, policy_version 355389 (0.0011) [2023-12-27 08:06:12,805][20381] Updated weights for policy 1, policy_version 355399 (0.0010) [2023-12-27 08:06:12,847][20347] Updated weights for policy 0, policy_version 355587 (0.0007) [2023-12-27 08:06:12,859][20381] Updated weights for policy 1, policy_version 355409 (0.0010) [2023-12-27 08:06:12,902][20347] Updated weights for policy 0, policy_version 355597 (0.0006) [2023-12-27 08:06:12,963][20347] Updated weights for policy 0, policy_version 355607 (0.0008) [2023-12-27 08:06:13,541][20381] Updated weights for policy 1, policy_version 355419 (0.0010) [2023-12-27 08:06:13,595][20381] Updated weights for policy 1, policy_version 355429 (0.0010) [2023-12-27 08:06:13,642][20347] Updated weights for policy 0, policy_version 355617 (0.0007) [2023-12-27 08:06:13,654][20381] Updated weights for policy 1, policy_version 355439 (0.0011) [2023-12-27 08:06:13,703][20347] Updated weights for policy 0, policy_version 355627 (0.0005) [2023-12-27 08:06:13,759][20347] Updated weights for policy 0, policy_version 355637 (0.0007) [2023-12-27 08:06:13,819][20347] Updated weights for policy 0, policy_version 355647 (0.0005) [2023-12-27 08:06:14,343][20381] Updated weights for policy 1, policy_version 355449 (0.0009) [2023-12-27 08:06:14,407][20381] Updated weights for policy 1, policy_version 355459 (0.0006) [2023-12-27 08:06:14,421][20347] Updated weights for policy 0, policy_version 355657 (0.0010) [2023-12-27 08:06:14,463][20381] Updated weights for policy 1, policy_version 355469 (0.0007) [2023-12-27 08:06:14,505][20347] Updated weights for policy 0, policy_version 355667 (0.0010) [2023-12-27 08:06:14,521][20381] Updated weights for policy 1, policy_version 355479 (0.0007) [2023-12-27 08:06:14,553][20347] Updated weights for policy 0, policy_version 355677 (0.0010) [2023-12-27 08:06:15,187][20381] Updated weights for policy 1, policy_version 355489 (0.0010) [2023-12-27 08:06:15,209][20347] Updated weights for policy 0, policy_version 355687 (0.0008) [2023-12-27 08:06:15,245][20381] Updated weights for policy 1, policy_version 355499 (0.0011) [2023-12-27 08:06:15,272][20347] Updated weights for policy 0, policy_version 355697 (0.0010) [2023-12-27 08:06:15,306][20381] Updated weights for policy 1, policy_version 355509 (0.0011) [2023-12-27 08:06:15,324][20347] Updated weights for policy 0, policy_version 355707 (0.0010) [2023-12-27 08:06:15,945][20381] Updated weights for policy 1, policy_version 355519 (0.0011) [2023-12-27 08:06:15,954][20347] Updated weights for policy 0, policy_version 355717 (0.0009) [2023-12-27 08:06:16,005][20381] Updated weights for policy 1, policy_version 355529 (0.0008) [2023-12-27 08:06:16,010][20347] Updated weights for policy 0, policy_version 355727 (0.0005) [2023-12-27 08:06:16,055][20381] Updated weights for policy 1, policy_version 355539 (0.0011) [2023-12-27 08:06:16,067][20347] Updated weights for policy 0, policy_version 355737 (0.0007) [2023-12-27 08:06:16,686][20381] Updated weights for policy 1, policy_version 355549 (0.0011) [2023-12-27 08:06:16,728][20347] Updated weights for policy 0, policy_version 355747 (0.0010) [2023-12-27 08:06:16,744][20381] Updated weights for policy 1, policy_version 355559 (0.0007) [2023-12-27 08:06:16,784][20347] Updated weights for policy 0, policy_version 355757 (0.0009) [2023-12-27 08:06:16,797][20381] Updated weights for policy 1, policy_version 355569 (0.0010) [2023-12-27 08:06:16,841][20347] Updated weights for policy 0, policy_version 355767 (0.0006) [2023-12-27 08:06:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 182140928. Throughput: 0: 10430.0, 1: 10417.3. Samples: 182153988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:17,450][20381] Updated weights for policy 1, policy_version 355579 (0.0006) [2023-12-27 08:06:17,488][20347] Updated weights for policy 0, policy_version 355777 (0.0006) [2023-12-27 08:06:17,503][20381] Updated weights for policy 1, policy_version 355589 (0.0005) [2023-12-27 08:06:17,535][20347] Updated weights for policy 0, policy_version 355787 (0.0007) [2023-12-27 08:06:17,561][20381] Updated weights for policy 1, policy_version 355599 (0.0005) [2023-12-27 08:06:17,582][20347] Updated weights for policy 0, policy_version 355797 (0.0008) [2023-12-27 08:06:17,634][20347] Updated weights for policy 0, policy_version 355807 (0.0008) [2023-12-27 08:06:18,228][20381] Updated weights for policy 1, policy_version 355609 (0.0005) [2023-12-27 08:06:18,253][20347] Updated weights for policy 0, policy_version 355817 (0.0006) [2023-12-27 08:06:18,287][20381] Updated weights for policy 1, policy_version 355619 (0.0007) [2023-12-27 08:06:18,301][20347] Updated weights for policy 0, policy_version 355827 (0.0005) [2023-12-27 08:06:18,342][20381] Updated weights for policy 1, policy_version 355629 (0.0006) [2023-12-27 08:06:18,361][20347] Updated weights for policy 0, policy_version 355837 (0.0007) [2023-12-27 08:06:18,395][20381] Updated weights for policy 1, policy_version 355639 (0.0008) [2023-12-27 08:06:18,934][20347] Updated weights for policy 0, policy_version 355847 (0.0005) [2023-12-27 08:06:18,989][20347] Updated weights for policy 0, policy_version 355857 (0.0006) [2023-12-27 08:06:19,019][20381] Updated weights for policy 1, policy_version 355649 (0.0009) [2023-12-27 08:06:19,040][20347] Updated weights for policy 0, policy_version 355867 (0.0007) [2023-12-27 08:06:19,065][20381] Updated weights for policy 1, policy_version 355659 (0.0006) [2023-12-27 08:06:19,114][20381] Updated weights for policy 1, policy_version 355669 (0.0008) [2023-12-27 08:06:19,687][20347] Updated weights for policy 0, policy_version 355877 (0.0006) [2023-12-27 08:06:19,746][20347] Updated weights for policy 0, policy_version 355887 (0.0008) [2023-12-27 08:06:19,799][20347] Updated weights for policy 0, policy_version 355897 (0.0008) [2023-12-27 08:06:19,862][20381] Updated weights for policy 1, policy_version 355679 (0.0008) [2023-12-27 08:06:19,927][20381] Updated weights for policy 1, policy_version 355689 (0.0008) [2023-12-27 08:06:19,990][20381] Updated weights for policy 1, policy_version 355699 (0.0008) [2023-12-27 08:06:20,512][20347] Updated weights for policy 0, policy_version 355907 (0.0007) [2023-12-27 08:06:20,573][20347] Updated weights for policy 0, policy_version 355917 (0.0007) [2023-12-27 08:06:20,635][20347] Updated weights for policy 0, policy_version 355927 (0.0008) [2023-12-27 08:06:20,692][20381] Updated weights for policy 1, policy_version 355709 (0.0009) [2023-12-27 08:06:20,756][20381] Updated weights for policy 1, policy_version 355719 (0.0008) [2023-12-27 08:06:20,818][20381] Updated weights for policy 1, policy_version 355729 (0.0007) [2023-12-27 08:06:21,334][20347] Updated weights for policy 0, policy_version 355937 (0.0008) [2023-12-27 08:06:21,407][20347] Updated weights for policy 0, policy_version 355947 (0.0008) [2023-12-27 08:06:21,460][20347] Updated weights for policy 0, policy_version 355957 (0.0009) [2023-12-27 08:06:21,462][20381] Updated weights for policy 1, policy_version 355739 (0.0008) [2023-12-27 08:06:21,517][20347] Updated weights for policy 0, policy_version 355967 (0.0009) [2023-12-27 08:06:21,523][20381] Updated weights for policy 1, policy_version 355749 (0.0009) [2023-12-27 08:06:21,580][20381] Updated weights for policy 1, policy_version 355759 (0.0009) [2023-12-27 08:06:22,278][20381] Updated weights for policy 1, policy_version 355769 (0.0009) [2023-12-27 08:06:22,296][20347] Updated weights for policy 0, policy_version 355977 (0.0008) [2023-12-27 08:06:22,338][20381] Updated weights for policy 1, policy_version 355779 (0.0011) [2023-12-27 08:06:22,352][20347] Updated weights for policy 0, policy_version 355987 (0.0007) [2023-12-27 08:06:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 182239232. Throughput: 0: 10453.7, 1: 10388.3. Samples: 182218136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:22,401][20381] Updated weights for policy 1, policy_version 355789 (0.0009) [2023-12-27 08:06:22,417][20347] Updated weights for policy 0, policy_version 355997 (0.0008) [2023-12-27 08:06:22,434][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000356000_91152384.pth... [2023-12-27 08:06:22,437][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000354784_90841088.pth [2023-12-27 08:06:22,453][20381] Updated weights for policy 1, policy_version 355799 (0.0011) [2023-12-27 08:06:22,456][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000355800_91103232.pth... [2023-12-27 08:06:22,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000354552_90783744.pth [2023-12-27 08:06:23,088][20381] Updated weights for policy 1, policy_version 355809 (0.0006) [2023-12-27 08:06:23,135][20381] Updated weights for policy 1, policy_version 355819 (0.0007) [2023-12-27 08:06:23,173][20347] Updated weights for policy 0, policy_version 356007 (0.0007) [2023-12-27 08:06:23,189][20381] Updated weights for policy 1, policy_version 355829 (0.0008) [2023-12-27 08:06:23,232][20347] Updated weights for policy 0, policy_version 356017 (0.0007) [2023-12-27 08:06:23,290][20347] Updated weights for policy 0, policy_version 356027 (0.0008) [2023-12-27 08:06:23,806][20381] Updated weights for policy 1, policy_version 355839 (0.0005) [2023-12-27 08:06:23,856][20381] Updated weights for policy 1, policy_version 355849 (0.0005) [2023-12-27 08:06:23,904][20381] Updated weights for policy 1, policy_version 355859 (0.0005) [2023-12-27 08:06:24,016][20347] Updated weights for policy 0, policy_version 356037 (0.0007) [2023-12-27 08:06:24,068][20347] Updated weights for policy 0, policy_version 356047 (0.0009) [2023-12-27 08:06:24,119][20347] Updated weights for policy 0, policy_version 356057 (0.0006) [2023-12-27 08:06:24,451][20381] Updated weights for policy 1, policy_version 355869 (0.0007) [2023-12-27 08:06:24,508][20381] Updated weights for policy 1, policy_version 355879 (0.0008) [2023-12-27 08:06:24,576][20381] Updated weights for policy 1, policy_version 355889 (0.0007) [2023-12-27 08:06:24,824][20347] Updated weights for policy 0, policy_version 356067 (0.0008) [2023-12-27 08:06:24,874][20347] Updated weights for policy 0, policy_version 356077 (0.0008) [2023-12-27 08:06:24,926][20347] Updated weights for policy 0, policy_version 356087 (0.0009) [2023-12-27 08:06:25,202][20381] Updated weights for policy 1, policy_version 355899 (0.0009) [2023-12-27 08:06:25,254][20381] Updated weights for policy 1, policy_version 355909 (0.0009) [2023-12-27 08:06:25,304][20381] Updated weights for policy 1, policy_version 355919 (0.0008) [2023-12-27 08:06:25,581][20347] Updated weights for policy 0, policy_version 356097 (0.0008) [2023-12-27 08:06:25,640][20347] Updated weights for policy 0, policy_version 356107 (0.0008) [2023-12-27 08:06:25,692][20347] Updated weights for policy 0, policy_version 356117 (0.0008) [2023-12-27 08:06:25,751][20347] Updated weights for policy 0, policy_version 356127 (0.0007) [2023-12-27 08:06:25,995][20381] Updated weights for policy 1, policy_version 355929 (0.0008) [2023-12-27 08:06:26,061][20381] Updated weights for policy 1, policy_version 355939 (0.0008) [2023-12-27 08:06:26,124][20381] Updated weights for policy 1, policy_version 355949 (0.0007) [2023-12-27 08:06:26,182][20381] Updated weights for policy 1, policy_version 355959 (0.0007) [2023-12-27 08:06:26,409][20347] Updated weights for policy 0, policy_version 356137 (0.0008) [2023-12-27 08:06:26,464][20347] Updated weights for policy 0, policy_version 356147 (0.0007) [2023-12-27 08:06:26,517][20347] Updated weights for policy 0, policy_version 356157 (0.0007) [2023-12-27 08:06:26,789][20381] Updated weights for policy 1, policy_version 355969 (0.0007) [2023-12-27 08:06:26,837][20381] Updated weights for policy 1, policy_version 355979 (0.0007) [2023-12-27 08:06:26,892][20381] Updated weights for policy 1, policy_version 355989 (0.0008) [2023-12-27 08:06:27,213][20347] Updated weights for policy 0, policy_version 356167 (0.0008) [2023-12-27 08:06:27,270][20347] Updated weights for policy 0, policy_version 356177 (0.0007) [2023-12-27 08:06:27,329][20347] Updated weights for policy 0, policy_version 356187 (0.0008) [2023-12-27 08:06:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 182353920. Throughput: 0: 10305.1, 1: 10430.1. Samples: 182340620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:06:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:27,550][20381] Updated weights for policy 1, policy_version 355999 (0.0008) [2023-12-27 08:06:27,601][20381] Updated weights for policy 1, policy_version 356009 (0.0008) [2023-12-27 08:06:27,652][20381] Updated weights for policy 1, policy_version 356019 (0.0008) [2023-12-27 08:06:28,027][20347] Updated weights for policy 0, policy_version 356197 (0.0007) [2023-12-27 08:06:28,082][20347] Updated weights for policy 0, policy_version 356207 (0.0008) [2023-12-27 08:06:28,130][20347] Updated weights for policy 0, policy_version 356217 (0.0008) [2023-12-27 08:06:28,339][20381] Updated weights for policy 1, policy_version 356029 (0.0007) [2023-12-27 08:06:28,393][20381] Updated weights for policy 1, policy_version 356039 (0.0006) [2023-12-27 08:06:28,456][20381] Updated weights for policy 1, policy_version 356049 (0.0005) [2023-12-27 08:06:28,865][20347] Updated weights for policy 0, policy_version 356227 (0.0007) [2023-12-27 08:06:28,924][20347] Updated weights for policy 0, policy_version 356237 (0.0007) [2023-12-27 08:06:28,981][20347] Updated weights for policy 0, policy_version 356247 (0.0008) [2023-12-27 08:06:29,008][20381] Updated weights for policy 1, policy_version 356059 (0.0006) [2023-12-27 08:06:29,054][20381] Updated weights for policy 1, policy_version 356069 (0.0008) [2023-12-27 08:06:29,108][20381] Updated weights for policy 1, policy_version 356079 (0.0007) [2023-12-27 08:06:29,702][20347] Updated weights for policy 0, policy_version 356257 (0.0008) [2023-12-27 08:06:29,757][20347] Updated weights for policy 0, policy_version 356267 (0.0008) [2023-12-27 08:06:29,783][20381] Updated weights for policy 1, policy_version 356089 (0.0007) [2023-12-27 08:06:29,817][20347] Updated weights for policy 0, policy_version 356277 (0.0007) [2023-12-27 08:06:29,847][20381] Updated weights for policy 1, policy_version 356099 (0.0007) [2023-12-27 08:06:29,878][20347] Updated weights for policy 0, policy_version 356287 (0.0006) [2023-12-27 08:06:29,905][20381] Updated weights for policy 1, policy_version 356109 (0.0006) [2023-12-27 08:06:29,975][20381] Updated weights for policy 1, policy_version 356119 (0.0007) [2023-12-27 08:06:30,476][20347] Updated weights for policy 0, policy_version 356297 (0.0005) [2023-12-27 08:06:30,526][20347] Updated weights for policy 0, policy_version 356307 (0.0005) [2023-12-27 08:06:30,575][20347] Updated weights for policy 0, policy_version 356317 (0.0006) [2023-12-27 08:06:30,664][20381] Updated weights for policy 1, policy_version 356129 (0.0007) [2023-12-27 08:06:30,708][20381] Updated weights for policy 1, policy_version 356139 (0.0008) [2023-12-27 08:06:30,754][20381] Updated weights for policy 1, policy_version 356149 (0.0007) [2023-12-27 08:06:31,206][20347] Updated weights for policy 0, policy_version 356327 (0.0009) [2023-12-27 08:06:31,269][20347] Updated weights for policy 0, policy_version 356337 (0.0011) [2023-12-27 08:06:31,318][20347] Updated weights for policy 0, policy_version 356347 (0.0010) [2023-12-27 08:06:31,511][20381] Updated weights for policy 1, policy_version 356159 (0.0008) [2023-12-27 08:06:31,570][20381] Updated weights for policy 1, policy_version 356169 (0.0008) [2023-12-27 08:06:31,621][20381] Updated weights for policy 1, policy_version 356179 (0.0007) [2023-12-27 08:06:32,000][20347] Updated weights for policy 0, policy_version 356357 (0.0008) [2023-12-27 08:06:32,060][20347] Updated weights for policy 0, policy_version 356367 (0.0009) [2023-12-27 08:06:32,119][20347] Updated weights for policy 0, policy_version 356377 (0.0010) [2023-12-27 08:06:32,321][20381] Updated weights for policy 1, policy_version 356189 (0.0009) [2023-12-27 08:06:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 182452224. Throughput: 0: 10305.5, 1: 10482.6. Samples: 182465052. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:32,382][20381] Updated weights for policy 1, policy_version 356199 (0.0007) [2023-12-27 08:06:32,431][20381] Updated weights for policy 1, policy_version 356209 (0.0008) [2023-12-27 08:06:32,698][20347] Updated weights for policy 0, policy_version 356387 (0.0009) [2023-12-27 08:06:32,758][20347] Updated weights for policy 0, policy_version 356397 (0.0005) [2023-12-27 08:06:32,817][20347] Updated weights for policy 0, policy_version 356407 (0.0006) [2023-12-27 08:06:33,174][20381] Updated weights for policy 1, policy_version 356219 (0.0008) [2023-12-27 08:06:33,235][20381] Updated weights for policy 1, policy_version 356229 (0.0007) [2023-12-27 08:06:33,295][20381] Updated weights for policy 1, policy_version 356239 (0.0007) [2023-12-27 08:06:33,396][20347] Updated weights for policy 0, policy_version 356417 (0.0006) [2023-12-27 08:06:33,443][20347] Updated weights for policy 0, policy_version 356427 (0.0010) [2023-12-27 08:06:33,487][20347] Updated weights for policy 0, policy_version 356437 (0.0010) [2023-12-27 08:06:33,535][20347] Updated weights for policy 0, policy_version 356447 (0.0010) [2023-12-27 08:06:33,981][20381] Updated weights for policy 1, policy_version 356249 (0.0008) [2023-12-27 08:06:34,029][20381] Updated weights for policy 1, policy_version 356259 (0.0007) [2023-12-27 08:06:34,081][20381] Updated weights for policy 1, policy_version 356269 (0.0007) [2023-12-27 08:06:34,137][20381] Updated weights for policy 1, policy_version 356279 (0.0008) [2023-12-27 08:06:34,235][20347] Updated weights for policy 0, policy_version 356457 (0.0008) [2023-12-27 08:06:34,287][20347] Updated weights for policy 0, policy_version 356467 (0.0009) [2023-12-27 08:06:34,336][20347] Updated weights for policy 0, policy_version 356477 (0.0010) [2023-12-27 08:06:34,868][20381] Updated weights for policy 1, policy_version 356289 (0.0008) [2023-12-27 08:06:34,913][20381] Updated weights for policy 1, policy_version 356299 (0.0007) [2023-12-27 08:06:34,961][20381] Updated weights for policy 1, policy_version 356309 (0.0007) [2023-12-27 08:06:34,992][20347] Updated weights for policy 0, policy_version 356487 (0.0006) [2023-12-27 08:06:35,040][20347] Updated weights for policy 0, policy_version 356497 (0.0006) [2023-12-27 08:06:35,091][20347] Updated weights for policy 0, policy_version 356507 (0.0008) [2023-12-27 08:06:35,698][20381] Updated weights for policy 1, policy_version 356319 (0.0006) [2023-12-27 08:06:35,699][20347] Updated weights for policy 0, policy_version 356517 (0.0008) [2023-12-27 08:06:35,746][20347] Updated weights for policy 0, policy_version 356527 (0.0005) [2023-12-27 08:06:35,756][20381] Updated weights for policy 1, policy_version 356329 (0.0005) [2023-12-27 08:06:35,797][20347] Updated weights for policy 0, policy_version 356537 (0.0007) [2023-12-27 08:06:35,808][20381] Updated weights for policy 1, policy_version 356339 (0.0005) [2023-12-27 08:06:36,427][20381] Updated weights for policy 1, policy_version 356349 (0.0007) [2023-12-27 08:06:36,465][20347] Updated weights for policy 0, policy_version 356547 (0.0007) [2023-12-27 08:06:36,488][20381] Updated weights for policy 1, policy_version 356359 (0.0008) [2023-12-27 08:06:36,527][20347] Updated weights for policy 0, policy_version 356557 (0.0006) [2023-12-27 08:06:36,549][20381] Updated weights for policy 1, policy_version 356369 (0.0008) [2023-12-27 08:06:36,579][20347] Updated weights for policy 0, policy_version 356567 (0.0006) [2023-12-27 08:06:37,254][20347] Updated weights for policy 0, policy_version 356577 (0.0007) [2023-12-27 08:06:37,260][20381] Updated weights for policy 1, policy_version 356379 (0.0006) [2023-12-27 08:06:37,310][20347] Updated weights for policy 0, policy_version 356587 (0.0007) [2023-12-27 08:06:37,319][20381] Updated weights for policy 1, policy_version 356389 (0.0005) [2023-12-27 08:06:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 182550528. Throughput: 0: 10348.7, 1: 10401.6. Samples: 182527152. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:37,370][20347] Updated weights for policy 0, policy_version 356597 (0.0010) [2023-12-27 08:06:37,379][20381] Updated weights for policy 1, policy_version 356399 (0.0005) [2023-12-27 08:06:37,422][20347] Updated weights for policy 0, policy_version 356607 (0.0010) [2023-12-27 08:06:37,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000356608_91308032.pth... [2023-12-27 08:06:37,430][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000355392_90996736.pth [2023-12-27 08:06:37,436][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000356408_91258880.pth... [2023-12-27 08:06:37,441][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000355192_90947584.pth [2023-12-27 08:06:38,010][20347] Updated weights for policy 0, policy_version 356617 (0.0010) [2023-12-27 08:06:38,059][20347] Updated weights for policy 0, policy_version 356627 (0.0010) [2023-12-27 08:06:38,105][20381] Updated weights for policy 1, policy_version 356409 (0.0007) [2023-12-27 08:06:38,114][20347] Updated weights for policy 0, policy_version 356637 (0.0010) [2023-12-27 08:06:38,155][20381] Updated weights for policy 1, policy_version 356419 (0.0007) [2023-12-27 08:06:38,203][20381] Updated weights for policy 1, policy_version 356429 (0.0009) [2023-12-27 08:06:38,260][20381] Updated weights for policy 1, policy_version 356439 (0.0009) [2023-12-27 08:06:38,756][20347] Updated weights for policy 0, policy_version 356647 (0.0007) [2023-12-27 08:06:38,806][20347] Updated weights for policy 0, policy_version 356657 (0.0005) [2023-12-27 08:06:38,871][20347] Updated weights for policy 0, policy_version 356667 (0.0006) [2023-12-27 08:06:39,010][20381] Updated weights for policy 1, policy_version 356449 (0.0008) [2023-12-27 08:06:39,072][20381] Updated weights for policy 1, policy_version 356459 (0.0008) [2023-12-27 08:06:39,126][20381] Updated weights for policy 1, policy_version 356469 (0.0008) [2023-12-27 08:06:39,546][20347] Updated weights for policy 0, policy_version 356677 (0.0007) [2023-12-27 08:06:39,611][20347] Updated weights for policy 0, policy_version 356687 (0.0008) [2023-12-27 08:06:39,678][20347] Updated weights for policy 0, policy_version 356697 (0.0007) [2023-12-27 08:06:39,839][20381] Updated weights for policy 1, policy_version 356479 (0.0008) [2023-12-27 08:06:39,891][20381] Updated weights for policy 1, policy_version 356489 (0.0007) [2023-12-27 08:06:39,951][20381] Updated weights for policy 1, policy_version 356499 (0.0007) [2023-12-27 08:06:40,349][20347] Updated weights for policy 0, policy_version 356707 (0.0009) [2023-12-27 08:06:40,402][20347] Updated weights for policy 0, policy_version 356717 (0.0010) [2023-12-27 08:06:40,451][20347] Updated weights for policy 0, policy_version 356727 (0.0010) [2023-12-27 08:06:40,676][20381] Updated weights for policy 1, policy_version 356509 (0.0009) [2023-12-27 08:06:40,733][20381] Updated weights for policy 1, policy_version 356519 (0.0011) [2023-12-27 08:06:40,788][20381] Updated weights for policy 1, policy_version 356529 (0.0011) [2023-12-27 08:06:41,130][20347] Updated weights for policy 0, policy_version 356737 (0.0010) [2023-12-27 08:06:41,188][20347] Updated weights for policy 0, policy_version 356747 (0.0011) [2023-12-27 08:06:41,244][20347] Updated weights for policy 0, policy_version 356757 (0.0010) [2023-12-27 08:06:41,300][20347] Updated weights for policy 0, policy_version 356767 (0.0011) [2023-12-27 08:06:41,475][20381] Updated weights for policy 1, policy_version 356539 (0.0009) [2023-12-27 08:06:41,542][20381] Updated weights for policy 1, policy_version 356549 (0.0006) [2023-12-27 08:06:41,611][20381] Updated weights for policy 1, policy_version 356559 (0.0006) [2023-12-27 08:06:42,022][20347] Updated weights for policy 0, policy_version 356777 (0.0010) [2023-12-27 08:06:42,086][20347] Updated weights for policy 0, policy_version 356787 (0.0010) [2023-12-27 08:06:42,142][20347] Updated weights for policy 0, policy_version 356797 (0.0010) [2023-12-27 08:06:42,187][20381] Updated weights for policy 1, policy_version 356569 (0.0007) [2023-12-27 08:06:42,243][20381] Updated weights for policy 1, policy_version 356579 (0.0005) [2023-12-27 08:06:42,310][20381] Updated weights for policy 1, policy_version 356589 (0.0006) [2023-12-27 08:06:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 182657024. Throughput: 0: 10395.9, 1: 10315.1. Samples: 182649988. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:42,383][20381] Updated weights for policy 1, policy_version 356599 (0.0007) [2023-12-27 08:06:42,839][20347] Updated weights for policy 0, policy_version 356807 (0.0011) [2023-12-27 08:06:42,910][20347] Updated weights for policy 0, policy_version 356817 (0.0010) [2023-12-27 08:06:42,963][20381] Updated weights for policy 1, policy_version 356609 (0.0007) [2023-12-27 08:06:42,975][20347] Updated weights for policy 0, policy_version 356827 (0.0006) [2023-12-27 08:06:43,010][20381] Updated weights for policy 1, policy_version 356619 (0.0007) [2023-12-27 08:06:43,056][20381] Updated weights for policy 1, policy_version 356629 (0.0008) [2023-12-27 08:06:43,613][20347] Updated weights for policy 0, policy_version 356837 (0.0008) [2023-12-27 08:06:43,677][20347] Updated weights for policy 0, policy_version 356847 (0.0010) [2023-12-27 08:06:43,738][20347] Updated weights for policy 0, policy_version 356857 (0.0009) [2023-12-27 08:06:43,777][20381] Updated weights for policy 1, policy_version 356639 (0.0006) [2023-12-27 08:06:43,821][20381] Updated weights for policy 1, policy_version 356649 (0.0008) [2023-12-27 08:06:43,873][20381] Updated weights for policy 1, policy_version 356659 (0.0008) [2023-12-27 08:06:44,401][20347] Updated weights for policy 0, policy_version 356867 (0.0008) [2023-12-27 08:06:44,452][20347] Updated weights for policy 0, policy_version 356877 (0.0010) [2023-12-27 08:06:44,510][20347] Updated weights for policy 0, policy_version 356887 (0.0010) [2023-12-27 08:06:44,582][20381] Updated weights for policy 1, policy_version 356669 (0.0007) [2023-12-27 08:06:44,629][20381] Updated weights for policy 1, policy_version 356679 (0.0007) [2023-12-27 08:06:44,676][20381] Updated weights for policy 1, policy_version 356689 (0.0007) [2023-12-27 08:06:45,205][20347] Updated weights for policy 0, policy_version 356897 (0.0010) [2023-12-27 08:06:45,267][20347] Updated weights for policy 0, policy_version 356907 (0.0008) [2023-12-27 08:06:45,330][20347] Updated weights for policy 0, policy_version 356917 (0.0009) [2023-12-27 08:06:45,390][20347] Updated weights for policy 0, policy_version 356927 (0.0011) [2023-12-27 08:06:45,417][20381] Updated weights for policy 1, policy_version 356699 (0.0006) [2023-12-27 08:06:45,468][20381] Updated weights for policy 1, policy_version 356709 (0.0007) [2023-12-27 08:06:45,533][20381] Updated weights for policy 1, policy_version 356719 (0.0007) [2023-12-27 08:06:46,083][20347] Updated weights for policy 0, policy_version 356937 (0.0008) [2023-12-27 08:06:46,145][20347] Updated weights for policy 0, policy_version 356947 (0.0005) [2023-12-27 08:06:46,208][20347] Updated weights for policy 0, policy_version 356957 (0.0005) [2023-12-27 08:06:46,224][20381] Updated weights for policy 1, policy_version 356729 (0.0007) [2023-12-27 08:06:46,278][20381] Updated weights for policy 1, policy_version 356739 (0.0008) [2023-12-27 08:06:46,326][20381] Updated weights for policy 1, policy_version 356749 (0.0008) [2023-12-27 08:06:46,369][20381] Updated weights for policy 1, policy_version 356759 (0.0007) [2023-12-27 08:06:46,833][20347] Updated weights for policy 0, policy_version 356967 (0.0008) [2023-12-27 08:06:46,896][20347] Updated weights for policy 0, policy_version 356977 (0.0005) [2023-12-27 08:06:46,959][20347] Updated weights for policy 0, policy_version 356987 (0.0011) [2023-12-27 08:06:47,076][20381] Updated weights for policy 1, policy_version 356769 (0.0006) [2023-12-27 08:06:47,146][20381] Updated weights for policy 1, policy_version 356779 (0.0008) [2023-12-27 08:06:47,213][20381] Updated weights for policy 1, policy_version 356789 (0.0008) [2023-12-27 08:06:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 182763520. Throughput: 0: 10348.2, 1: 10259.6. Samples: 182772016. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:47,607][20347] Updated weights for policy 0, policy_version 356997 (0.0008) [2023-12-27 08:06:47,667][20347] Updated weights for policy 0, policy_version 357007 (0.0005) [2023-12-27 08:06:47,728][20347] Updated weights for policy 0, policy_version 357017 (0.0005) [2023-12-27 08:06:47,821][20381] Updated weights for policy 1, policy_version 356799 (0.0006) [2023-12-27 08:06:47,876][20381] Updated weights for policy 1, policy_version 356809 (0.0006) [2023-12-27 08:06:47,938][20381] Updated weights for policy 1, policy_version 356819 (0.0006) [2023-12-27 08:06:48,286][20347] Updated weights for policy 0, policy_version 357027 (0.0005) [2023-12-27 08:06:48,347][20347] Updated weights for policy 0, policy_version 357037 (0.0007) [2023-12-27 08:06:48,405][20347] Updated weights for policy 0, policy_version 357047 (0.0007) [2023-12-27 08:06:48,555][20381] Updated weights for policy 1, policy_version 356829 (0.0006) [2023-12-27 08:06:48,615][20381] Updated weights for policy 1, policy_version 356839 (0.0007) [2023-12-27 08:06:48,669][20381] Updated weights for policy 1, policy_version 356849 (0.0007) [2023-12-27 08:06:49,067][20347] Updated weights for policy 0, policy_version 357057 (0.0008) [2023-12-27 08:06:49,122][20347] Updated weights for policy 0, policy_version 357067 (0.0010) [2023-12-27 08:06:49,181][20347] Updated weights for policy 0, policy_version 357077 (0.0010) [2023-12-27 08:06:49,242][20347] Updated weights for policy 0, policy_version 357087 (0.0010) [2023-12-27 08:06:49,346][20381] Updated weights for policy 1, policy_version 356859 (0.0007) [2023-12-27 08:06:49,404][20381] Updated weights for policy 1, policy_version 356869 (0.0008) [2023-12-27 08:06:49,460][20381] Updated weights for policy 1, policy_version 356879 (0.0007) [2023-12-27 08:06:49,938][20347] Updated weights for policy 0, policy_version 357097 (0.0008) [2023-12-27 08:06:50,005][20347] Updated weights for policy 0, policy_version 357107 (0.0009) [2023-12-27 08:06:50,058][20347] Updated weights for policy 0, policy_version 357117 (0.0010) [2023-12-27 08:06:50,131][20381] Updated weights for policy 1, policy_version 356889 (0.0008) [2023-12-27 08:06:50,192][20381] Updated weights for policy 1, policy_version 356899 (0.0007) [2023-12-27 08:06:50,241][20381] Updated weights for policy 1, policy_version 356909 (0.0007) [2023-12-27 08:06:50,291][20381] Updated weights for policy 1, policy_version 356919 (0.0007) [2023-12-27 08:06:50,724][20347] Updated weights for policy 0, policy_version 357127 (0.0010) [2023-12-27 08:06:50,780][20347] Updated weights for policy 0, policy_version 357137 (0.0010) [2023-12-27 08:06:50,829][20347] Updated weights for policy 0, policy_version 357147 (0.0010) [2023-12-27 08:06:51,006][20381] Updated weights for policy 1, policy_version 356929 (0.0008) [2023-12-27 08:06:51,068][20381] Updated weights for policy 1, policy_version 356939 (0.0008) [2023-12-27 08:06:51,130][20381] Updated weights for policy 1, policy_version 356949 (0.0008) [2023-12-27 08:06:51,564][20347] Updated weights for policy 0, policy_version 357157 (0.0008) [2023-12-27 08:06:51,633][20347] Updated weights for policy 0, policy_version 357167 (0.0006) [2023-12-27 08:06:51,695][20347] Updated weights for policy 0, policy_version 357177 (0.0006) [2023-12-27 08:06:51,846][20381] Updated weights for policy 1, policy_version 356959 (0.0007) [2023-12-27 08:06:51,905][20381] Updated weights for policy 1, policy_version 356969 (0.0008) [2023-12-27 08:06:51,969][20381] Updated weights for policy 1, policy_version 356979 (0.0008) [2023-12-27 08:06:52,343][20347] Updated weights for policy 0, policy_version 357187 (0.0007) [2023-12-27 08:06:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 182861824. Throughput: 0: 10381.4, 1: 10282.0. Samples: 182835328. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000356984_91406336.pth... [2023-12-27 08:06:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000355800_91103232.pth [2023-12-27 08:06:52,410][20347] Updated weights for policy 0, policy_version 357197 (0.0008) [2023-12-27 08:06:52,477][20347] Updated weights for policy 0, policy_version 357207 (0.0008) [2023-12-27 08:06:52,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000357216_91463680.pth... [2023-12-27 08:06:52,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000356000_91152384.pth [2023-12-27 08:06:52,654][20381] Updated weights for policy 1, policy_version 356989 (0.0008) [2023-12-27 08:06:52,710][20381] Updated weights for policy 1, policy_version 356999 (0.0008) [2023-12-27 08:06:52,765][20381] Updated weights for policy 1, policy_version 357009 (0.0008) [2023-12-27 08:06:53,148][20347] Updated weights for policy 0, policy_version 357217 (0.0007) [2023-12-27 08:06:53,204][20347] Updated weights for policy 0, policy_version 357227 (0.0008) [2023-12-27 08:06:53,263][20347] Updated weights for policy 0, policy_version 357237 (0.0007) [2023-12-27 08:06:53,317][20347] Updated weights for policy 0, policy_version 357247 (0.0007) [2023-12-27 08:06:53,467][20381] Updated weights for policy 1, policy_version 357019 (0.0008) [2023-12-27 08:06:53,526][20381] Updated weights for policy 1, policy_version 357029 (0.0008) [2023-12-27 08:06:53,585][20381] Updated weights for policy 1, policy_version 357039 (0.0008) [2023-12-27 08:06:53,946][20347] Updated weights for policy 0, policy_version 357257 (0.0006) [2023-12-27 08:06:54,000][20347] Updated weights for policy 0, policy_version 357267 (0.0005) [2023-12-27 08:06:54,058][20347] Updated weights for policy 0, policy_version 357277 (0.0005) [2023-12-27 08:06:54,324][20381] Updated weights for policy 1, policy_version 357049 (0.0008) [2023-12-27 08:06:54,370][20381] Updated weights for policy 1, policy_version 357059 (0.0010) [2023-12-27 08:06:54,429][20381] Updated weights for policy 1, policy_version 357069 (0.0008) [2023-12-27 08:06:54,481][20381] Updated weights for policy 1, policy_version 357079 (0.0008) [2023-12-27 08:06:54,648][20347] Updated weights for policy 0, policy_version 357287 (0.0008) [2023-12-27 08:06:54,700][20347] Updated weights for policy 0, policy_version 357297 (0.0009) [2023-12-27 08:06:54,760][20347] Updated weights for policy 0, policy_version 357307 (0.0009) [2023-12-27 08:06:55,221][20381] Updated weights for policy 1, policy_version 357089 (0.0011) [2023-12-27 08:06:55,267][20381] Updated weights for policy 1, policy_version 357099 (0.0011) [2023-12-27 08:06:55,332][20381] Updated weights for policy 1, policy_version 357109 (0.0011) [2023-12-27 08:06:55,405][20347] Updated weights for policy 0, policy_version 357317 (0.0009) [2023-12-27 08:06:55,450][20347] Updated weights for policy 0, policy_version 357327 (0.0010) [2023-12-27 08:06:55,495][20347] Updated weights for policy 0, policy_version 357337 (0.0010) [2023-12-27 08:06:56,028][20381] Updated weights for policy 1, policy_version 357119 (0.0011) [2023-12-27 08:06:56,087][20381] Updated weights for policy 1, policy_version 357129 (0.0011) [2023-12-27 08:06:56,139][20381] Updated weights for policy 1, policy_version 357139 (0.0011) [2023-12-27 08:06:56,199][20347] Updated weights for policy 0, policy_version 357347 (0.0010) [2023-12-27 08:06:56,245][20347] Updated weights for policy 0, policy_version 357357 (0.0010) [2023-12-27 08:06:56,293][20347] Updated weights for policy 0, policy_version 357367 (0.0006) [2023-12-27 08:06:56,819][20381] Updated weights for policy 1, policy_version 357149 (0.0011) [2023-12-27 08:06:56,868][20381] Updated weights for policy 1, policy_version 357159 (0.0011) [2023-12-27 08:06:56,917][20381] Updated weights for policy 1, policy_version 357169 (0.0010) [2023-12-27 08:06:56,922][20347] Updated weights for policy 0, policy_version 357377 (0.0005) [2023-12-27 08:06:56,976][20347] Updated weights for policy 0, policy_version 357387 (0.0009) [2023-12-27 08:06:57,027][20347] Updated weights for policy 0, policy_version 357397 (0.0010) [2023-12-27 08:06:57,078][20347] Updated weights for policy 0, policy_version 357407 (0.0010) [2023-12-27 08:06:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 182968320. Throughput: 0: 10383.4, 1: 10219.2. Samples: 182956696. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:06:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:06:57,577][20381] Updated weights for policy 1, policy_version 357179 (0.0009) [2023-12-27 08:06:57,635][20381] Updated weights for policy 1, policy_version 357189 (0.0005) [2023-12-27 08:06:57,662][20347] Updated weights for policy 0, policy_version 357417 (0.0008) [2023-12-27 08:06:57,691][20381] Updated weights for policy 1, policy_version 357199 (0.0006) [2023-12-27 08:06:57,714][20347] Updated weights for policy 0, policy_version 357427 (0.0009) [2023-12-27 08:06:57,762][20347] Updated weights for policy 0, policy_version 357437 (0.0009) [2023-12-27 08:06:58,277][20381] Updated weights for policy 1, policy_version 357209 (0.0010) [2023-12-27 08:06:58,334][20381] Updated weights for policy 1, policy_version 357219 (0.0008) [2023-12-27 08:06:58,403][20381] Updated weights for policy 1, policy_version 357229 (0.0007) [2023-12-27 08:06:58,404][20347] Updated weights for policy 0, policy_version 357447 (0.0008) [2023-12-27 08:06:58,464][20381] Updated weights for policy 1, policy_version 357239 (0.0009) [2023-12-27 08:06:58,464][20347] Updated weights for policy 0, policy_version 357457 (0.0008) [2023-12-27 08:06:58,527][20347] Updated weights for policy 0, policy_version 357467 (0.0008) [2023-12-27 08:06:59,185][20381] Updated weights for policy 1, policy_version 357249 (0.0010) [2023-12-27 08:06:59,247][20381] Updated weights for policy 1, policy_version 357259 (0.0010) [2023-12-27 08:06:59,305][20381] Updated weights for policy 1, policy_version 357269 (0.0010) [2023-12-27 08:06:59,329][20347] Updated weights for policy 0, policy_version 357477 (0.0008) [2023-12-27 08:06:59,391][20347] Updated weights for policy 0, policy_version 357487 (0.0010) [2023-12-27 08:06:59,439][20347] Updated weights for policy 0, policy_version 357497 (0.0010) [2023-12-27 08:06:59,979][20381] Updated weights for policy 1, policy_version 357279 (0.0009) [2023-12-27 08:07:00,029][20381] Updated weights for policy 1, policy_version 357289 (0.0011) [2023-12-27 08:07:00,043][20347] Updated weights for policy 0, policy_version 357507 (0.0007) [2023-12-27 08:07:00,080][20381] Updated weights for policy 1, policy_version 357299 (0.0006) [2023-12-27 08:07:00,102][20347] Updated weights for policy 0, policy_version 357517 (0.0007) [2023-12-27 08:07:00,166][20347] Updated weights for policy 0, policy_version 357527 (0.0008) [2023-12-27 08:07:00,741][20381] Updated weights for policy 1, policy_version 357309 (0.0007) [2023-12-27 08:07:00,789][20347] Updated weights for policy 0, policy_version 357537 (0.0009) [2023-12-27 08:07:00,801][20381] Updated weights for policy 1, policy_version 357319 (0.0007) [2023-12-27 08:07:00,835][20347] Updated weights for policy 0, policy_version 357547 (0.0006) [2023-12-27 08:07:00,849][20381] Updated weights for policy 1, policy_version 357329 (0.0007) [2023-12-27 08:07:00,884][20347] Updated weights for policy 0, policy_version 357557 (0.0005) [2023-12-27 08:07:00,945][20347] Updated weights for policy 0, policy_version 357567 (0.0005) [2023-12-27 08:07:01,528][20381] Updated weights for policy 1, policy_version 357339 (0.0009) [2023-12-27 08:07:01,573][20381] Updated weights for policy 1, policy_version 357349 (0.0010) [2023-12-27 08:07:01,627][20381] Updated weights for policy 1, policy_version 357359 (0.0010) [2023-12-27 08:07:01,674][20347] Updated weights for policy 0, policy_version 357577 (0.0010) [2023-12-27 08:07:01,729][20347] Updated weights for policy 0, policy_version 357587 (0.0010) [2023-12-27 08:07:01,794][20347] Updated weights for policy 0, policy_version 357597 (0.0006) [2023-12-27 08:07:02,245][20381] Updated weights for policy 1, policy_version 357369 (0.0007) [2023-12-27 08:07:02,308][20381] Updated weights for policy 1, policy_version 357379 (0.0007) [2023-12-27 08:07:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 183066624. Throughput: 0: 10414.1, 1: 10234.8. Samples: 183083188. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:07:02,368][20381] Updated weights for policy 1, policy_version 357389 (0.0010) [2023-12-27 08:07:02,419][20381] Updated weights for policy 1, policy_version 357399 (0.0010) [2023-12-27 08:07:02,451][20347] Updated weights for policy 0, policy_version 357607 (0.0010) [2023-12-27 08:07:02,512][20347] Updated weights for policy 0, policy_version 357617 (0.0010) [2023-12-27 08:07:02,570][20347] Updated weights for policy 0, policy_version 357627 (0.0010) [2023-12-27 08:07:03,101][20381] Updated weights for policy 1, policy_version 357409 (0.0008) [2023-12-27 08:07:03,154][20381] Updated weights for policy 1, policy_version 357419 (0.0007) [2023-12-27 08:07:03,202][20347] Updated weights for policy 0, policy_version 357637 (0.0010) [2023-12-27 08:07:03,207][20381] Updated weights for policy 1, policy_version 357429 (0.0008) [2023-12-27 08:07:03,235][20291] KL-divergence is very high: 411.9830 [2023-12-27 08:07:03,239][20291] KL-divergence is very high: 167.6321 [2023-12-27 08:07:03,256][20347] Updated weights for policy 0, policy_version 357647 (0.0010) [2023-12-27 08:07:03,280][20291] KL-divergence is very high: 2594.1143 [2023-12-27 08:07:03,285][20291] KL-divergence is very high: 1283.1776 [2023-12-27 08:07:03,299][20291] KL-divergence is very high: 143.9435 [2023-12-27 08:07:03,304][20291] KL-divergence is very high: 145.4971 [2023-12-27 08:07:03,310][20347] Updated weights for policy 0, policy_version 357657 (0.0010) [2023-12-27 08:07:03,313][20291] KL-divergence is very high: 116.0456 [2023-12-27 08:07:03,317][20291] KL-divergence is very high: 1666.0239 [2023-12-27 08:07:03,322][20291] KL-divergence is very high: 1193.6586 [2023-12-27 08:07:03,336][20291] KL-divergence is very high: 108.3618 [2023-12-27 08:07:03,800][20381] Updated weights for policy 1, policy_version 357439 (0.0006) [2023-12-27 08:07:03,855][20381] Updated weights for policy 1, policy_version 357449 (0.0007) [2023-12-27 08:07:03,909][20381] Updated weights for policy 1, policy_version 357459 (0.0008) [2023-12-27 08:07:03,989][20347] Updated weights for policy 0, policy_version 357667 (0.0010) [2023-12-27 08:07:04,007][20291] KL-divergence is very high: 505.3961 [2023-12-27 08:07:04,050][20347] Updated weights for policy 0, policy_version 357677 (0.0010) [2023-12-27 08:07:04,102][20347] Updated weights for policy 0, policy_version 357687 (0.0010) [2023-12-27 08:07:04,143][20291] KL-divergence is very high: 518.2391 [2023-12-27 08:07:04,579][20381] Updated weights for policy 1, policy_version 357469 (0.0007) [2023-12-27 08:07:04,645][20381] Updated weights for policy 1, policy_version 357479 (0.0007) [2023-12-27 08:07:04,710][20381] Updated weights for policy 1, policy_version 357489 (0.0006) [2023-12-27 08:07:04,792][20347] Updated weights for policy 0, policy_version 357697 (0.0010) [2023-12-27 08:07:04,792][20291] KL-divergence is very high: 640.8056 [2023-12-27 08:07:04,806][20291] KL-divergence is very high: 156.8137 [2023-12-27 08:07:04,825][20291] KL-divergence is very high: 750.3361 [2023-12-27 08:07:04,845][20291] KL-divergence is very high: 292.1105 [2023-12-27 08:07:04,855][20347] Updated weights for policy 0, policy_version 357707 (0.0006) [2023-12-27 08:07:04,892][20291] KL-divergence is very high: 762.3818 [2023-12-27 08:07:04,915][20347] Updated weights for policy 0, policy_version 357717 (0.0005) [2023-12-27 08:07:04,923][20291] KL-divergence is very high: 290.9014 [2023-12-27 08:07:04,945][20291] KL-divergence is very high: 230.2624 [2023-12-27 08:07:04,959][20291] KL-divergence is very high: 130.2062 [2023-12-27 08:07:04,985][20347] Updated weights for policy 0, policy_version 357727 (0.0006) [2023-12-27 08:07:05,308][20381] Updated weights for policy 1, policy_version 357499 (0.0007) [2023-12-27 08:07:05,360][20381] Updated weights for policy 1, policy_version 357509 (0.0010) [2023-12-27 08:07:05,424][20381] Updated weights for policy 1, policy_version 357519 (0.0011) [2023-12-27 08:07:05,445][20291] KL-divergence is very high: 305.7021 [2023-12-27 08:07:05,490][20291] KL-divergence is very high: 414.7352 [2023-12-27 08:07:05,495][20347] Updated weights for policy 0, policy_version 357737 (0.0010) [2023-12-27 08:07:05,530][20291] KL-divergence is very high: 408.6007 [2023-12-27 08:07:05,549][20347] Updated weights for policy 0, policy_version 357747 (0.0010) [2023-12-27 08:07:05,575][20291] KL-divergence is very high: 584.4998 [2023-12-27 08:07:05,597][20347] Updated weights for policy 0, policy_version 357757 (0.0010) [2023-12-27 08:07:06,023][20381] Updated weights for policy 1, policy_version 357529 (0.0010) [2023-12-27 08:07:06,076][20381] Updated weights for policy 1, policy_version 357539 (0.0010) [2023-12-27 08:07:06,132][20381] Updated weights for policy 1, policy_version 357549 (0.0009) [2023-12-27 08:07:06,195][20381] Updated weights for policy 1, policy_version 357559 (0.0008) [2023-12-27 08:07:06,296][20347] Updated weights for policy 0, policy_version 357767 (0.0008) [2023-12-27 08:07:06,297][20291] KL-divergence is very high: 155.3620 [2023-12-27 08:07:06,347][20291] KL-divergence is very high: 153.6324 [2023-12-27 08:07:06,357][20347] Updated weights for policy 0, policy_version 357777 (0.0006) [2023-12-27 08:07:06,393][20291] KL-divergence is very high: 127.4480 [2023-12-27 08:07:06,417][20347] Updated weights for policy 0, policy_version 357787 (0.0005) [2023-12-27 08:07:06,878][20381] Updated weights for policy 1, policy_version 357569 (0.0008) [2023-12-27 08:07:06,932][20381] Updated weights for policy 1, policy_version 357579 (0.0008) [2023-12-27 08:07:06,991][20381] Updated weights for policy 1, policy_version 357589 (0.0008) [2023-12-27 08:07:07,041][20347] Updated weights for policy 0, policy_version 357797 (0.0007) [2023-12-27 08:07:07,077][20291] KL-divergence is very high: 195.8755 [2023-12-27 08:07:07,088][20347] Updated weights for policy 0, policy_version 357807 (0.0008) [2023-12-27 08:07:07,132][20347] Updated weights for policy 0, policy_version 357817 (0.0007) [2023-12-27 08:07:07,149][20291] KL-divergence is very high: 198.2762 [2023-12-27 08:07:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 183181312. Throughput: 0: 10361.6, 1: 10274.7. Samples: 183146768. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:07,363][19170] Avg episode reward: [(0, '847.020'), (1, '1000.000')] [2023-12-27 08:07:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000357824_91619328.pth... [2023-12-27 08:07:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000357592_91561984.pth... [2023-12-27 08:07:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000356608_91308032.pth [2023-12-27 08:07:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000356408_91258880.pth [2023-12-27 08:07:07,605][20381] Updated weights for policy 1, policy_version 357599 (0.0007) [2023-12-27 08:07:07,654][20381] Updated weights for policy 1, policy_version 357609 (0.0009) [2023-12-27 08:07:07,720][20381] Updated weights for policy 1, policy_version 357619 (0.0007) [2023-12-27 08:07:07,729][20347] Updated weights for policy 0, policy_version 357827 (0.0008) [2023-12-27 08:07:07,789][20347] Updated weights for policy 0, policy_version 357837 (0.0005) [2023-12-27 08:07:07,850][20347] Updated weights for policy 0, policy_version 357847 (0.0008) [2023-12-27 08:07:08,385][20381] Updated weights for policy 1, policy_version 357629 (0.0008) [2023-12-27 08:07:08,415][20347] Updated weights for policy 0, policy_version 357857 (0.0007) [2023-12-27 08:07:08,439][20291] KL-divergence is very high: 221.5539 [2023-12-27 08:07:08,440][20381] Updated weights for policy 1, policy_version 357639 (0.0007) [2023-12-27 08:07:08,450][20291] KL-divergence is very high: 297.1202 [2023-12-27 08:07:08,456][20291] KL-divergence is very high: 229.3180 [2023-12-27 08:07:08,462][20291] KL-divergence is very high: 149.4527 [2023-12-27 08:07:08,473][20347] Updated weights for policy 0, policy_version 357867 (0.0010) [2023-12-27 08:07:08,498][20381] Updated weights for policy 1, policy_version 357649 (0.0007) [2023-12-27 08:07:08,529][20347] Updated weights for policy 0, policy_version 357877 (0.0009) [2023-12-27 08:07:08,565][20291] KL-divergence is very high: 242.4608 [2023-12-27 08:07:08,574][20347] Updated weights for policy 0, policy_version 357887 (0.0008) [2023-12-27 08:07:08,575][20291] KL-divergence is very high: 299.0513 [2023-12-27 08:07:09,199][20381] Updated weights for policy 1, policy_version 357659 (0.0007) [2023-12-27 08:07:09,225][20291] KL-divergence is very high: 258.8287 [2023-12-27 08:07:09,239][20291] KL-divergence is very high: 129.3515 [2023-12-27 08:07:09,246][20291] KL-divergence is very high: 158.8917 [2023-12-27 08:07:09,262][20381] Updated weights for policy 1, policy_version 357669 (0.0008) [2023-12-27 08:07:09,270][20291] KL-divergence is very high: 108.2996 [2023-12-27 08:07:09,279][20347] Updated weights for policy 0, policy_version 357897 (0.0007) [2023-12-27 08:07:09,285][20291] KL-divergence is very high: 146.7260 [2023-12-27 08:07:09,289][20291] KL-divergence is very high: 102.3404 [2023-12-27 08:07:09,294][20291] KL-divergence is very high: 151.0025 [2023-12-27 08:07:09,311][20291] KL-divergence is very high: 124.3704 [2023-12-27 08:07:09,318][20381] Updated weights for policy 1, policy_version 357679 (0.0006) [2023-12-27 08:07:09,336][20347] Updated weights for policy 0, policy_version 357907 (0.0008) [2023-12-27 08:07:09,384][20291] KL-divergence is very high: 126.2769 [2023-12-27 08:07:09,396][20347] Updated weights for policy 0, policy_version 357917 (0.0008) [2023-12-27 08:07:09,991][20381] Updated weights for policy 1, policy_version 357690 (0.0009) [2023-12-27 08:07:10,047][20381] Updated weights for policy 1, policy_version 357700 (0.0008) [2023-12-27 08:07:10,081][20347] Updated weights for policy 0, policy_version 357927 (0.0007) [2023-12-27 08:07:10,102][20381] Updated weights for policy 1, policy_version 357710 (0.0006) [2023-12-27 08:07:10,114][20291] KL-divergence is very high: 1031.1688 [2023-12-27 08:07:10,140][20347] Updated weights for policy 0, policy_version 357937 (0.0008) [2023-12-27 08:07:10,163][20381] Updated weights for policy 1, policy_version 357720 (0.0006) [2023-12-27 08:07:10,165][20291] KL-divergence is very high: 158.6148 [2023-12-27 08:07:10,199][20347] Updated weights for policy 0, policy_version 357947 (0.0007) [2023-12-27 08:07:10,736][20381] Updated weights for policy 1, policy_version 357730 (0.0010) [2023-12-27 08:07:10,781][20381] Updated weights for policy 1, policy_version 357740 (0.0010) [2023-12-27 08:07:10,833][20381] Updated weights for policy 1, policy_version 357750 (0.0010) [2023-12-27 08:07:10,908][20347] Updated weights for policy 0, policy_version 357957 (0.0009) [2023-12-27 08:07:10,957][20347] Updated weights for policy 0, policy_version 357967 (0.0009) [2023-12-27 08:07:10,957][20291] KL-divergence is very high: 2336.2856 [2023-12-27 08:07:11,006][20291] KL-divergence is very high: 914.2646 [2023-12-27 08:07:11,018][20347] Updated weights for policy 0, policy_version 357977 (0.0007) [2023-12-27 08:07:11,504][20381] Updated weights for policy 1, policy_version 357760 (0.0006) [2023-12-27 08:07:11,557][20381] Updated weights for policy 1, policy_version 357770 (0.0005) [2023-12-27 08:07:11,614][20381] Updated weights for policy 1, policy_version 357780 (0.0006) [2023-12-27 08:07:11,728][20291] KL-divergence is very high: 596.2642 [2023-12-27 08:07:11,742][20347] Updated weights for policy 0, policy_version 357987 (0.0008) [2023-12-27 08:07:11,776][20291] KL-divergence is very high: 817.6599 [2023-12-27 08:07:11,804][20347] Updated weights for policy 0, policy_version 357997 (0.0007) [2023-12-27 08:07:11,830][20291] KL-divergence is very high: 627.6572 [2023-12-27 08:07:11,864][20347] Updated weights for policy 0, policy_version 358007 (0.0006) [2023-12-27 08:07:11,876][20291] KL-divergence is very high: 402.1244 [2023-12-27 08:07:12,245][20381] Updated weights for policy 1, policy_version 357790 (0.0008) [2023-12-27 08:07:12,303][20381] Updated weights for policy 1, policy_version 357800 (0.0011) [2023-12-27 08:07:12,357][20381] Updated weights for policy 1, policy_version 357810 (0.0009) [2023-12-27 08:07:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 183279616. Throughput: 0: 10476.3, 1: 10260.6. Samples: 183273784. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:12,363][19170] Avg episode reward: [(0, '740.280'), (1, '1000.000')] [2023-12-27 08:07:12,587][20347] Updated weights for policy 0, policy_version 358017 (0.0007) [2023-12-27 08:07:12,638][20347] Updated weights for policy 0, policy_version 358027 (0.0008) [2023-12-27 08:07:12,694][20347] Updated weights for policy 0, policy_version 358037 (0.0007) [2023-12-27 08:07:12,742][20347] Updated weights for policy 0, policy_version 358047 (0.0007) [2023-12-27 08:07:13,064][20381] Updated weights for policy 1, policy_version 357820 (0.0009) [2023-12-27 08:07:13,126][20381] Updated weights for policy 1, policy_version 357830 (0.0007) [2023-12-27 08:07:13,185][20381] Updated weights for policy 1, policy_version 357840 (0.0008) [2023-12-27 08:07:13,451][20347] Updated weights for policy 0, policy_version 358057 (0.0008) [2023-12-27 08:07:13,502][20347] Updated weights for policy 0, policy_version 358067 (0.0007) [2023-12-27 08:07:13,561][20347] Updated weights for policy 0, policy_version 358077 (0.0007) [2023-12-27 08:07:13,822][20381] Updated weights for policy 1, policy_version 357850 (0.0007) [2023-12-27 08:07:13,878][20381] Updated weights for policy 1, policy_version 357860 (0.0006) [2023-12-27 08:07:13,937][20381] Updated weights for policy 1, policy_version 357870 (0.0007) [2023-12-27 08:07:13,994][20381] Updated weights for policy 1, policy_version 357880 (0.0005) [2023-12-27 08:07:14,235][20347] Updated weights for policy 0, policy_version 358087 (0.0007) [2023-12-27 08:07:14,294][20347] Updated weights for policy 0, policy_version 358097 (0.0008) [2023-12-27 08:07:14,345][20347] Updated weights for policy 0, policy_version 358107 (0.0008) [2023-12-27 08:07:14,557][20381] Updated weights for policy 1, policy_version 357890 (0.0008) [2023-12-27 08:07:14,605][20381] Updated weights for policy 1, policy_version 357900 (0.0006) [2023-12-27 08:07:14,663][20381] Updated weights for policy 1, policy_version 357910 (0.0005) [2023-12-27 08:07:15,049][20347] Updated weights for policy 0, policy_version 358117 (0.0007) [2023-12-27 08:07:15,102][20347] Updated weights for policy 0, policy_version 358127 (0.0008) [2023-12-27 08:07:15,160][20347] Updated weights for policy 0, policy_version 358137 (0.0007) [2023-12-27 08:07:15,181][20291] KL-divergence is very high: 180.4989 [2023-12-27 08:07:15,195][20291] KL-divergence is very high: 237.6817 [2023-12-27 08:07:15,201][20291] KL-divergence is very high: 224.4563 [2023-12-27 08:07:15,279][20381] Updated weights for policy 1, policy_version 357920 (0.0008) [2023-12-27 08:07:15,337][20381] Updated weights for policy 1, policy_version 357930 (0.0007) [2023-12-27 08:07:15,402][20381] Updated weights for policy 1, policy_version 357940 (0.0009) [2023-12-27 08:07:15,793][20291] KL-divergence is very high: 238.8140 [2023-12-27 08:07:15,805][20347] Updated weights for policy 0, policy_version 358147 (0.0008) [2023-12-27 08:07:15,805][20291] KL-divergence is very high: 127.8850 [2023-12-27 08:07:15,828][20291] KL-divergence is very high: 124.3090 [2023-12-27 08:07:15,845][20291] KL-divergence is very high: 123.6589 [2023-12-27 08:07:15,875][20347] Updated weights for policy 0, policy_version 358157 (0.0010) [2023-12-27 08:07:15,893][20291] KL-divergence is very high: 130.1803 [2023-12-27 08:07:15,903][20291] KL-divergence is very high: 130.4937 [2023-12-27 08:07:15,920][20347] Updated weights for policy 0, policy_version 358167 (0.0007) [2023-12-27 08:07:15,931][20291] KL-divergence is very high: 134.7873 [2023-12-27 08:07:15,940][20291] KL-divergence is very high: 122.9938 [2023-12-27 08:07:16,038][20381] Updated weights for policy 1, policy_version 357950 (0.0007) [2023-12-27 08:07:16,091][20381] Updated weights for policy 1, policy_version 357960 (0.0006) [2023-12-27 08:07:16,153][20381] Updated weights for policy 1, policy_version 357970 (0.0011) [2023-12-27 08:07:16,575][20347] Updated weights for policy 0, policy_version 358177 (0.0007) [2023-12-27 08:07:16,639][20347] Updated weights for policy 0, policy_version 358187 (0.0006) [2023-12-27 08:07:16,701][20347] Updated weights for policy 0, policy_version 358197 (0.0006) [2023-12-27 08:07:16,761][20347] Updated weights for policy 0, policy_version 358207 (0.0010) [2023-12-27 08:07:16,811][20381] Updated weights for policy 1, policy_version 357980 (0.0011) [2023-12-27 08:07:16,870][20381] Updated weights for policy 1, policy_version 357990 (0.0011) [2023-12-27 08:07:16,920][20381] Updated weights for policy 1, policy_version 358000 (0.0011) [2023-12-27 08:07:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 183386112. Throughput: 0: 10468.9, 1: 10284.5. Samples: 183398956. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:17,363][19170] Avg episode reward: [(0, '777.310'), (1, '1000.000')] [2023-12-27 08:07:17,433][20347] Updated weights for policy 0, policy_version 358217 (0.0008) [2023-12-27 08:07:17,487][20347] Updated weights for policy 0, policy_version 358227 (0.0008) [2023-12-27 08:07:17,541][20347] Updated weights for policy 0, policy_version 358237 (0.0007) [2023-12-27 08:07:17,637][20381] Updated weights for policy 1, policy_version 358010 (0.0012) [2023-12-27 08:07:17,682][20381] Updated weights for policy 1, policy_version 358020 (0.0010) [2023-12-27 08:07:17,731][20381] Updated weights for policy 1, policy_version 358030 (0.0010) [2023-12-27 08:07:17,780][20381] Updated weights for policy 1, policy_version 358040 (0.0005) [2023-12-27 08:07:18,118][20347] Updated weights for policy 0, policy_version 358247 (0.0006) [2023-12-27 08:07:18,151][20291] KL-divergence is very high: 129.1141 [2023-12-27 08:07:18,167][20347] Updated weights for policy 0, policy_version 358257 (0.0005) [2023-12-27 08:07:18,193][20291] KL-divergence is very high: 140.2736 [2023-12-27 08:07:18,218][20347] Updated weights for policy 0, policy_version 358267 (0.0005) [2023-12-27 08:07:18,428][20381] Updated weights for policy 1, policy_version 358050 (0.0011) [2023-12-27 08:07:18,480][20381] Updated weights for policy 1, policy_version 358060 (0.0011) [2023-12-27 08:07:18,532][20381] Updated weights for policy 1, policy_version 358070 (0.0010) [2023-12-27 08:07:18,846][20347] Updated weights for policy 0, policy_version 358277 (0.0007) [2023-12-27 08:07:18,905][20347] Updated weights for policy 0, policy_version 358287 (0.0008) [2023-12-27 08:07:18,969][20347] Updated weights for policy 0, policy_version 358297 (0.0007) [2023-12-27 08:07:19,216][20381] Updated weights for policy 1, policy_version 358080 (0.0007) [2023-12-27 08:07:19,273][20381] Updated weights for policy 1, policy_version 358090 (0.0011) [2023-12-27 08:07:19,329][20381] Updated weights for policy 1, policy_version 358100 (0.0011) [2023-12-27 08:07:19,610][20347] Updated weights for policy 0, policy_version 358307 (0.0007) [2023-12-27 08:07:19,668][20347] Updated weights for policy 0, policy_version 358317 (0.0007) [2023-12-27 08:07:19,729][20347] Updated weights for policy 0, policy_version 358327 (0.0007) [2023-12-27 08:07:20,035][20381] Updated weights for policy 1, policy_version 358110 (0.0012) [2023-12-27 08:07:20,085][20381] Updated weights for policy 1, policy_version 358120 (0.0008) [2023-12-27 08:07:20,137][20381] Updated weights for policy 1, policy_version 358130 (0.0008) [2023-12-27 08:07:20,427][20347] Updated weights for policy 0, policy_version 358337 (0.0007) [2023-12-27 08:07:20,472][20347] Updated weights for policy 0, policy_version 358347 (0.0008) [2023-12-27 08:07:20,519][20347] Updated weights for policy 0, policy_version 358357 (0.0007) [2023-12-27 08:07:20,573][20347] Updated weights for policy 0, policy_version 358367 (0.0006) [2023-12-27 08:07:20,900][20381] Updated weights for policy 1, policy_version 358140 (0.0008) [2023-12-27 08:07:20,957][20381] Updated weights for policy 1, policy_version 358150 (0.0008) [2023-12-27 08:07:21,021][20381] Updated weights for policy 1, policy_version 358160 (0.0008) [2023-12-27 08:07:21,234][20347] Updated weights for policy 0, policy_version 358377 (0.0010) [2023-12-27 08:07:21,299][20347] Updated weights for policy 0, policy_version 358387 (0.0009) [2023-12-27 08:07:21,364][20347] Updated weights for policy 0, policy_version 358397 (0.0008) [2023-12-27 08:07:21,766][20381] Updated weights for policy 1, policy_version 358170 (0.0007) [2023-12-27 08:07:21,828][20381] Updated weights for policy 1, policy_version 358180 (0.0008) [2023-12-27 08:07:21,898][20381] Updated weights for policy 1, policy_version 358190 (0.0008) [2023-12-27 08:07:21,960][20381] Updated weights for policy 1, policy_version 358200 (0.0008) [2023-12-27 08:07:22,048][20347] Updated weights for policy 0, policy_version 358407 (0.0007) [2023-12-27 08:07:22,103][20347] Updated weights for policy 0, policy_version 358417 (0.0006) [2023-12-27 08:07:22,156][20347] Updated weights for policy 0, policy_version 358427 (0.0005) [2023-12-27 08:07:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 183492608. Throughput: 0: 10452.0, 1: 10313.2. Samples: 183461588. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:22,363][19170] Avg episode reward: [(0, '854.800'), (1, '1000.000')] [2023-12-27 08:07:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000358200_91717632.pth... [2023-12-27 08:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000358432_91774976.pth... [2023-12-27 08:07:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000357216_91463680.pth [2023-12-27 08:07:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000356984_91406336.pth [2023-12-27 08:07:22,700][20381] Updated weights for policy 1, policy_version 358210 (0.0007) [2023-12-27 08:07:22,765][20381] Updated weights for policy 1, policy_version 358220 (0.0008) [2023-12-27 08:07:22,813][20347] Updated weights for policy 0, policy_version 358437 (0.0006) [2023-12-27 08:07:22,821][20381] Updated weights for policy 1, policy_version 358230 (0.0008) [2023-12-27 08:07:22,874][20347] Updated weights for policy 0, policy_version 358447 (0.0005) [2023-12-27 08:07:22,928][20347] Updated weights for policy 0, policy_version 358457 (0.0005) [2023-12-27 08:07:23,447][20347] Updated weights for policy 0, policy_version 358467 (0.0006) [2023-12-27 08:07:23,500][20347] Updated weights for policy 0, policy_version 358477 (0.0006) [2023-12-27 08:07:23,548][20347] Updated weights for policy 0, policy_version 358487 (0.0005) [2023-12-27 08:07:23,588][20381] Updated weights for policy 1, policy_version 358240 (0.0008) [2023-12-27 08:07:23,643][20381] Updated weights for policy 1, policy_version 358250 (0.0009) [2023-12-27 08:07:23,695][20381] Updated weights for policy 1, policy_version 358260 (0.0008) [2023-12-27 08:07:24,188][20347] Updated weights for policy 0, policy_version 358497 (0.0007) [2023-12-27 08:07:24,248][20347] Updated weights for policy 0, policy_version 358507 (0.0007) [2023-12-27 08:07:24,309][20347] Updated weights for policy 0, policy_version 358517 (0.0006) [2023-12-27 08:07:24,375][20347] Updated weights for policy 0, policy_version 358527 (0.0007) [2023-12-27 08:07:24,430][20381] Updated weights for policy 1, policy_version 358270 (0.0008) [2023-12-27 08:07:24,477][20381] Updated weights for policy 1, policy_version 358280 (0.0009) [2023-12-27 08:07:24,524][20381] Updated weights for policy 1, policy_version 358290 (0.0009) [2023-12-27 08:07:24,936][20347] Updated weights for policy 0, policy_version 358537 (0.0006) [2023-12-27 08:07:24,996][20347] Updated weights for policy 0, policy_version 358547 (0.0006) [2023-12-27 08:07:25,049][20347] Updated weights for policy 0, policy_version 358557 (0.0008) [2023-12-27 08:07:25,301][20381] Updated weights for policy 1, policy_version 358300 (0.0009) [2023-12-27 08:07:25,356][20381] Updated weights for policy 1, policy_version 358310 (0.0007) [2023-12-27 08:07:25,409][20381] Updated weights for policy 1, policy_version 358320 (0.0008) [2023-12-27 08:07:25,649][20347] Updated weights for policy 0, policy_version 358567 (0.0007) [2023-12-27 08:07:25,704][20347] Updated weights for policy 0, policy_version 358577 (0.0007) [2023-12-27 08:07:25,767][20347] Updated weights for policy 0, policy_version 358587 (0.0008) [2023-12-27 08:07:26,164][20381] Updated weights for policy 1, policy_version 358330 (0.0008) [2023-12-27 08:07:26,219][20381] Updated weights for policy 1, policy_version 358340 (0.0009) [2023-12-27 08:07:26,268][20381] Updated weights for policy 1, policy_version 358350 (0.0008) [2023-12-27 08:07:26,321][20381] Updated weights for policy 1, policy_version 358360 (0.0008) [2023-12-27 08:07:26,351][20347] Updated weights for policy 0, policy_version 358597 (0.0008) [2023-12-27 08:07:26,405][20347] Updated weights for policy 0, policy_version 358607 (0.0007) [2023-12-27 08:07:26,450][20347] Updated weights for policy 0, policy_version 358617 (0.0008) [2023-12-27 08:07:27,017][20381] Updated weights for policy 1, policy_version 358370 (0.0009) [2023-12-27 08:07:27,072][20381] Updated weights for policy 1, policy_version 358380 (0.0008) [2023-12-27 08:07:27,125][20381] Updated weights for policy 1, policy_version 358390 (0.0009) [2023-12-27 08:07:27,128][20347] Updated weights for policy 0, policy_version 358627 (0.0007) [2023-12-27 08:07:27,178][20347] Updated weights for policy 0, policy_version 358637 (0.0005) [2023-12-27 08:07:27,221][20347] Updated weights for policy 0, policy_version 358647 (0.0005) [2023-12-27 08:07:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 183599104. Throughput: 0: 10502.0, 1: 10248.2. Samples: 183583752. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:27,364][19170] Avg episode reward: [(0, '942.340'), (1, '1000.000')] [2023-12-27 08:07:27,740][20381] Updated weights for policy 1, policy_version 358400 (0.0007) [2023-12-27 08:07:27,754][20347] Updated weights for policy 0, policy_version 358657 (0.0005) [2023-12-27 08:07:27,787][20381] Updated weights for policy 1, policy_version 358410 (0.0006) [2023-12-27 08:07:27,800][20347] Updated weights for policy 0, policy_version 358667 (0.0006) [2023-12-27 08:07:27,839][20381] Updated weights for policy 1, policy_version 358420 (0.0006) [2023-12-27 08:07:27,852][20347] Updated weights for policy 0, policy_version 358677 (0.0005) [2023-12-27 08:07:27,900][20347] Updated weights for policy 0, policy_version 358687 (0.0005) [2023-12-27 08:07:28,515][20381] Updated weights for policy 1, policy_version 358430 (0.0007) [2023-12-27 08:07:28,537][20347] Updated weights for policy 0, policy_version 358697 (0.0006) [2023-12-27 08:07:28,568][20381] Updated weights for policy 1, policy_version 358440 (0.0007) [2023-12-27 08:07:28,591][20347] Updated weights for policy 0, policy_version 358707 (0.0005) [2023-12-27 08:07:28,625][20381] Updated weights for policy 1, policy_version 358450 (0.0008) [2023-12-27 08:07:28,653][20347] Updated weights for policy 0, policy_version 358717 (0.0006) [2023-12-27 08:07:29,257][20347] Updated weights for policy 0, policy_version 358727 (0.0007) [2023-12-27 08:07:29,309][20381] Updated weights for policy 1, policy_version 358460 (0.0007) [2023-12-27 08:07:29,320][20347] Updated weights for policy 0, policy_version 358737 (0.0005) [2023-12-27 08:07:29,371][20381] Updated weights for policy 1, policy_version 358470 (0.0007) [2023-12-27 08:07:29,380][20347] Updated weights for policy 0, policy_version 358747 (0.0008) [2023-12-27 08:07:29,432][20381] Updated weights for policy 1, policy_version 358480 (0.0006) [2023-12-27 08:07:30,075][20381] Updated weights for policy 1, policy_version 358490 (0.0008) [2023-12-27 08:07:30,084][20347] Updated weights for policy 0, policy_version 358757 (0.0010) [2023-12-27 08:07:30,130][20381] Updated weights for policy 1, policy_version 358500 (0.0005) [2023-12-27 08:07:30,138][20347] Updated weights for policy 0, policy_version 358767 (0.0009) [2023-12-27 08:07:30,184][20381] Updated weights for policy 1, policy_version 358510 (0.0006) [2023-12-27 08:07:30,187][20347] Updated weights for policy 0, policy_version 358777 (0.0009) [2023-12-27 08:07:30,237][20381] Updated weights for policy 1, policy_version 358520 (0.0006) [2023-12-27 08:07:30,840][20347] Updated weights for policy 0, policy_version 358787 (0.0009) [2023-12-27 08:07:30,858][20381] Updated weights for policy 1, policy_version 358530 (0.0007) [2023-12-27 08:07:30,891][20347] Updated weights for policy 0, policy_version 358797 (0.0006) [2023-12-27 08:07:30,912][20381] Updated weights for policy 1, policy_version 358540 (0.0006) [2023-12-27 08:07:30,941][20347] Updated weights for policy 0, policy_version 358807 (0.0008) [2023-12-27 08:07:30,966][20381] Updated weights for policy 1, policy_version 358550 (0.0006) [2023-12-27 08:07:31,531][20347] Updated weights for policy 0, policy_version 358817 (0.0005) [2023-12-27 08:07:31,583][20347] Updated weights for policy 0, policy_version 358827 (0.0007) [2023-12-27 08:07:31,641][20347] Updated weights for policy 0, policy_version 358837 (0.0007) [2023-12-27 08:07:31,644][20381] Updated weights for policy 1, policy_version 358560 (0.0008) [2023-12-27 08:07:31,701][20347] Updated weights for policy 0, policy_version 358847 (0.0007) [2023-12-27 08:07:31,702][20381] Updated weights for policy 1, policy_version 358570 (0.0006) [2023-12-27 08:07:31,765][20381] Updated weights for policy 1, policy_version 358580 (0.0008) [2023-12-27 08:07:32,329][20347] Updated weights for policy 0, policy_version 358857 (0.0007) [2023-12-27 08:07:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 183697408. Throughput: 0: 10630.3, 1: 10269.7. Samples: 183712516. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:32,363][19170] Avg episode reward: [(0, '858.260'), (1, '1000.000')] [2023-12-27 08:07:32,387][20347] Updated weights for policy 0, policy_version 358867 (0.0007) [2023-12-27 08:07:32,435][20347] Updated weights for policy 0, policy_version 358877 (0.0006) [2023-12-27 08:07:32,437][20381] Updated weights for policy 1, policy_version 358590 (0.0008) [2023-12-27 08:07:32,483][20381] Updated weights for policy 1, policy_version 358600 (0.0007) [2023-12-27 08:07:32,541][20381] Updated weights for policy 1, policy_version 358610 (0.0007) [2023-12-27 08:07:32,998][20347] Updated weights for policy 0, policy_version 358887 (0.0005) [2023-12-27 08:07:33,046][20347] Updated weights for policy 0, policy_version 358897 (0.0005) [2023-12-27 08:07:33,112][20347] Updated weights for policy 0, policy_version 358907 (0.0005) [2023-12-27 08:07:33,234][20381] Updated weights for policy 1, policy_version 358620 (0.0008) [2023-12-27 08:07:33,287][20381] Updated weights for policy 1, policy_version 358630 (0.0006) [2023-12-27 08:07:33,346][20381] Updated weights for policy 1, policy_version 358640 (0.0006) [2023-12-27 08:07:33,723][20347] Updated weights for policy 0, policy_version 358917 (0.0007) [2023-12-27 08:07:33,772][20347] Updated weights for policy 0, policy_version 358927 (0.0008) [2023-12-27 08:07:33,788][20291] KL-divergence is very high: 196.4755 [2023-12-27 08:07:33,793][20291] KL-divergence is very high: 127.2044 [2023-12-27 08:07:33,798][20291] KL-divergence is very high: 166.5736 [2023-12-27 08:07:33,821][20347] Updated weights for policy 0, policy_version 358937 (0.0007) [2023-12-27 08:07:33,833][20291] KL-divergence is very high: 127.3916 [2023-12-27 08:07:33,839][20291] KL-divergence is very high: 135.3694 [2023-12-27 08:07:33,845][20291] KL-divergence is very high: 109.2081 [2023-12-27 08:07:33,995][20381] Updated weights for policy 1, policy_version 358650 (0.0007) [2023-12-27 08:07:34,046][20381] Updated weights for policy 1, policy_version 358660 (0.0007) [2023-12-27 08:07:34,094][20381] Updated weights for policy 1, policy_version 358670 (0.0005) [2023-12-27 08:07:34,144][20381] Updated weights for policy 1, policy_version 358680 (0.0006) [2023-12-27 08:07:34,498][20291] KL-divergence is very high: 109.5851 [2023-12-27 08:07:34,508][20347] Updated weights for policy 0, policy_version 358947 (0.0008) [2023-12-27 08:07:34,538][20291] KL-divergence is very high: 248.8972 [2023-12-27 08:07:34,558][20347] Updated weights for policy 0, policy_version 358957 (0.0008) [2023-12-27 08:07:34,576][20291] KL-divergence is very high: 139.3953 [2023-12-27 08:07:34,606][20347] Updated weights for policy 0, policy_version 358967 (0.0008) [2023-12-27 08:07:34,829][20381] Updated weights for policy 1, policy_version 358690 (0.0007) [2023-12-27 08:07:34,894][20381] Updated weights for policy 1, policy_version 358700 (0.0007) [2023-12-27 08:07:34,954][20381] Updated weights for policy 1, policy_version 358710 (0.0008) [2023-12-27 08:07:35,229][20347] Updated weights for policy 0, policy_version 358977 (0.0009) [2023-12-27 08:07:35,277][20347] Updated weights for policy 0, policy_version 358987 (0.0007) [2023-12-27 08:07:35,328][20347] Updated weights for policy 0, policy_version 358997 (0.0007) [2023-12-27 08:07:35,383][20347] Updated weights for policy 0, policy_version 359007 (0.0008) [2023-12-27 08:07:35,658][20381] Updated weights for policy 1, policy_version 358720 (0.0008) [2023-12-27 08:07:35,718][20381] Updated weights for policy 1, policy_version 358730 (0.0008) [2023-12-27 08:07:35,773][20381] Updated weights for policy 1, policy_version 358740 (0.0008) [2023-12-27 08:07:36,044][20347] Updated weights for policy 0, policy_version 359017 (0.0009) [2023-12-27 08:07:36,101][20347] Updated weights for policy 0, policy_version 359027 (0.0007) [2023-12-27 08:07:36,171][20347] Updated weights for policy 0, policy_version 359037 (0.0007) [2023-12-27 08:07:36,421][20381] Updated weights for policy 1, policy_version 358750 (0.0006) [2023-12-27 08:07:36,485][20381] Updated weights for policy 1, policy_version 358760 (0.0006) [2023-12-27 08:07:36,549][20381] Updated weights for policy 1, policy_version 358770 (0.0007) [2023-12-27 08:07:36,908][20347] Updated weights for policy 0, policy_version 359047 (0.0008) [2023-12-27 08:07:36,968][20347] Updated weights for policy 0, policy_version 359057 (0.0007) [2023-12-27 08:07:37,035][20347] Updated weights for policy 0, policy_version 359067 (0.0008) [2023-12-27 08:07:37,079][20381] Updated weights for policy 1, policy_version 358780 (0.0006) [2023-12-27 08:07:37,135][20381] Updated weights for policy 1, policy_version 358790 (0.0005) [2023-12-27 08:07:37,193][20381] Updated weights for policy 1, policy_version 358800 (0.0005) [2023-12-27 08:07:37,363][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.0, 300 sec: 20799.3). Total num frames: 183812096. Throughput: 0: 10670.8, 1: 10255.5. Samples: 183777016. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:37,364][19170] Avg episode reward: [(0, '693.950'), (1, '1000.000')] [2023-12-27 08:07:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000358808_91873280.pth... [2023-12-27 08:07:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000359072_91938816.pth... [2023-12-27 08:07:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000357592_91561984.pth [2023-12-27 08:07:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000357824_91619328.pth [2023-12-27 08:07:37,709][20381] Updated weights for policy 1, policy_version 358810 (0.0006) [2023-12-27 08:07:37,761][20381] Updated weights for policy 1, policy_version 358820 (0.0005) [2023-12-27 08:07:37,764][20347] Updated weights for policy 0, policy_version 359077 (0.0007) [2023-12-27 08:07:37,817][20381] Updated weights for policy 1, policy_version 358830 (0.0005) [2023-12-27 08:07:37,819][20347] Updated weights for policy 0, policy_version 359087 (0.0006) [2023-12-27 08:07:37,873][20381] Updated weights for policy 1, policy_version 358840 (0.0005) [2023-12-27 08:07:37,876][20347] Updated weights for policy 0, policy_version 359097 (0.0005) [2023-12-27 08:07:38,460][20381] Updated weights for policy 1, policy_version 358850 (0.0010) [2023-12-27 08:07:38,490][20347] Updated weights for policy 0, policy_version 359107 (0.0007) [2023-12-27 08:07:38,513][20381] Updated weights for policy 1, policy_version 358860 (0.0011) [2023-12-27 08:07:38,541][20347] Updated weights for policy 0, policy_version 359117 (0.0007) [2023-12-27 08:07:38,562][20381] Updated weights for policy 1, policy_version 358870 (0.0010) [2023-12-27 08:07:38,594][20347] Updated weights for policy 0, policy_version 359127 (0.0006) [2023-12-27 08:07:39,216][20347] Updated weights for policy 0, policy_version 359137 (0.0007) [2023-12-27 08:07:39,279][20347] Updated weights for policy 0, policy_version 359147 (0.0008) [2023-12-27 08:07:39,323][20381] Updated weights for policy 1, policy_version 358880 (0.0007) [2023-12-27 08:07:39,337][20347] Updated weights for policy 0, policy_version 359157 (0.0006) [2023-12-27 08:07:39,393][20381] Updated weights for policy 1, policy_version 358890 (0.0008) [2023-12-27 08:07:39,399][20291] KL-divergence is very high: 100.9723 [2023-12-27 08:07:39,403][20347] Updated weights for policy 0, policy_version 359167 (0.0007) [2023-12-27 08:07:39,445][20381] Updated weights for policy 1, policy_version 358900 (0.0008) [2023-12-27 08:07:40,075][20347] Updated weights for policy 0, policy_version 359177 (0.0008) [2023-12-27 08:07:40,086][20291] KL-divergence is very high: 156.5062 [2023-12-27 08:07:40,101][20291] KL-divergence is very high: 165.7152 [2023-12-27 08:07:40,124][20347] Updated weights for policy 0, policy_version 359187 (0.0007) [2023-12-27 08:07:40,182][20347] Updated weights for policy 0, policy_version 359197 (0.0007) [2023-12-27 08:07:40,187][20381] Updated weights for policy 1, policy_version 358910 (0.0007) [2023-12-27 08:07:40,247][20381] Updated weights for policy 1, policy_version 358920 (0.0007) [2023-12-27 08:07:40,304][20381] Updated weights for policy 1, policy_version 358930 (0.0008) [2023-12-27 08:07:40,798][20347] Updated weights for policy 0, policy_version 359207 (0.0007) [2023-12-27 08:07:40,857][20347] Updated weights for policy 0, policy_version 359217 (0.0005) [2023-12-27 08:07:40,910][20347] Updated weights for policy 0, policy_version 359227 (0.0005) [2023-12-27 08:07:41,052][20381] Updated weights for policy 1, policy_version 358940 (0.0008) [2023-12-27 08:07:41,115][20381] Updated weights for policy 1, policy_version 358950 (0.0008) [2023-12-27 08:07:41,176][20381] Updated weights for policy 1, policy_version 358960 (0.0007) [2023-12-27 08:07:41,629][20347] Updated weights for policy 0, policy_version 359237 (0.0010) [2023-12-27 08:07:41,691][20347] Updated weights for policy 0, policy_version 359247 (0.0011) [2023-12-27 08:07:41,756][20347] Updated weights for policy 0, policy_version 359257 (0.0009) [2023-12-27 08:07:41,883][20381] Updated weights for policy 1, policy_version 358970 (0.0010) [2023-12-27 08:07:41,945][20381] Updated weights for policy 1, policy_version 358980 (0.0008) [2023-12-27 08:07:42,007][20381] Updated weights for policy 1, policy_version 358990 (0.0009) [2023-12-27 08:07:42,072][20381] Updated weights for policy 1, policy_version 359000 (0.0009) [2023-12-27 08:07:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 183910400. Throughput: 0: 10665.9, 1: 10353.4. Samples: 183902564. Policy #0 lag: (min: 19.0, avg: 29.0, max: 51.0) [2023-12-27 08:07:42,363][19170] Avg episode reward: [(0, '482.100'), (1, '1000.000')] [2023-12-27 08:07:42,396][20347] Updated weights for policy 0, policy_version 359267 (0.0007) [2023-12-27 08:07:42,451][20347] Updated weights for policy 0, policy_version 359277 (0.0007) [2023-12-27 08:07:42,510][20347] Updated weights for policy 0, policy_version 359287 (0.0007) [2023-12-27 08:07:42,738][20381] Updated weights for policy 1, policy_version 359010 (0.0009) [2023-12-27 08:07:42,784][20381] Updated weights for policy 1, policy_version 359020 (0.0007) [2023-12-27 08:07:42,844][20381] Updated weights for policy 1, policy_version 359030 (0.0007) [2023-12-27 08:07:43,134][20347] Updated weights for policy 0, policy_version 359297 (0.0006) [2023-12-27 08:07:43,200][20347] Updated weights for policy 0, policy_version 359307 (0.0007) [2023-12-27 08:07:43,257][20347] Updated weights for policy 0, policy_version 359317 (0.0007) [2023-12-27 08:07:43,263][20291] KL-divergence is very high: 230.3168 [2023-12-27 08:07:43,310][20291] KL-divergence is very high: 167.6932 [2023-12-27 08:07:43,316][20347] Updated weights for policy 0, policy_version 359327 (0.0008) [2023-12-27 08:07:43,553][20381] Updated weights for policy 1, policy_version 359040 (0.0006) [2023-12-27 08:07:43,618][20381] Updated weights for policy 1, policy_version 359050 (0.0005) [2023-12-27 08:07:43,678][20381] Updated weights for policy 1, policy_version 359060 (0.0007) [2023-12-27 08:07:43,954][20347] Updated weights for policy 0, policy_version 359337 (0.0007) [2023-12-27 08:07:44,012][20347] Updated weights for policy 0, policy_version 359347 (0.0007) [2023-12-27 08:07:44,070][20347] Updated weights for policy 0, policy_version 359357 (0.0007) [2023-12-27 08:07:44,251][20381] Updated weights for policy 1, policy_version 359070 (0.0007) [2023-12-27 08:07:44,295][20381] Updated weights for policy 1, policy_version 359080 (0.0005) [2023-12-27 08:07:44,346][20381] Updated weights for policy 1, policy_version 359090 (0.0007) [2023-12-27 08:07:44,725][20347] Updated weights for policy 0, policy_version 359367 (0.0009) [2023-12-27 08:07:44,787][20347] Updated weights for policy 0, policy_version 359377 (0.0009) [2023-12-27 08:07:44,845][20347] Updated weights for policy 0, policy_version 359387 (0.0006) [2023-12-27 08:07:44,999][20381] Updated weights for policy 1, policy_version 359100 (0.0007) [2023-12-27 08:07:45,064][20381] Updated weights for policy 1, policy_version 359110 (0.0007) [2023-12-27 08:07:45,107][20292] KL-divergence is very high: 281.7555 [2023-12-27 08:07:45,133][20381] Updated weights for policy 1, policy_version 359120 (0.0006) [2023-12-27 08:07:45,456][20347] Updated weights for policy 0, policy_version 359397 (0.0008) [2023-12-27 08:07:45,508][20347] Updated weights for policy 0, policy_version 359407 (0.0010) [2023-12-27 08:07:45,566][20347] Updated weights for policy 0, policy_version 359417 (0.0010) [2023-12-27 08:07:45,765][20381] Updated weights for policy 1, policy_version 359130 (0.0007) [2023-12-27 08:07:45,820][20381] Updated weights for policy 1, policy_version 359140 (0.0007) [2023-12-27 08:07:45,871][20381] Updated weights for policy 1, policy_version 359150 (0.0006) [2023-12-27 08:07:45,916][20381] Updated weights for policy 1, policy_version 359160 (0.0005) [2023-12-27 08:07:46,246][20347] Updated weights for policy 0, policy_version 359427 (0.0010) [2023-12-27 08:07:46,294][20347] Updated weights for policy 0, policy_version 359437 (0.0010) [2023-12-27 08:07:46,342][20347] Updated weights for policy 0, policy_version 359447 (0.0010) [2023-12-27 08:07:46,565][20381] Updated weights for policy 1, policy_version 359170 (0.0005) [2023-12-27 08:07:46,578][20292] KL-divergence is very high: 207.3925 [2023-12-27 08:07:46,611][20381] Updated weights for policy 1, policy_version 359180 (0.0005) [2023-12-27 08:07:46,666][20381] Updated weights for policy 1, policy_version 359190 (0.0008) [2023-12-27 08:07:46,972][20347] Updated weights for policy 0, policy_version 359457 (0.0010) [2023-12-27 08:07:47,025][20347] Updated weights for policy 0, policy_version 359467 (0.0010) [2023-12-27 08:07:47,085][20347] Updated weights for policy 0, policy_version 359477 (0.0006) [2023-12-27 08:07:47,140][20347] Updated weights for policy 0, policy_version 359487 (0.0010) [2023-12-27 08:07:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 184016896. Throughput: 0: 10656.3, 1: 10356.2. Samples: 184028756. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:07:47,363][19170] Avg episode reward: [(0, '680.010'), (1, '980.780')] [2023-12-27 08:07:47,377][20381] Updated weights for policy 1, policy_version 359200 (0.0007) [2023-12-27 08:07:47,439][20381] Updated weights for policy 1, policy_version 359210 (0.0008) [2023-12-27 08:07:47,503][20381] Updated weights for policy 1, policy_version 359220 (0.0008) [2023-12-27 08:07:47,810][20347] Updated weights for policy 0, policy_version 359497 (0.0010) [2023-12-27 08:07:47,855][20347] Updated weights for policy 0, policy_version 359507 (0.0010) [2023-12-27 08:07:47,909][20347] Updated weights for policy 0, policy_version 359517 (0.0010) [2023-12-27 08:07:48,180][20381] Updated weights for policy 1, policy_version 359230 (0.0009) [2023-12-27 08:07:48,236][20381] Updated weights for policy 1, policy_version 359240 (0.0010) [2023-12-27 08:07:48,291][20381] Updated weights for policy 1, policy_version 359250 (0.0010) [2023-12-27 08:07:48,600][20347] Updated weights for policy 0, policy_version 359527 (0.0007) [2023-12-27 08:07:48,657][20347] Updated weights for policy 0, policy_version 359537 (0.0009) [2023-12-27 08:07:48,716][20347] Updated weights for policy 0, policy_version 359547 (0.0005) [2023-12-27 08:07:49,008][20381] Updated weights for policy 1, policy_version 359260 (0.0008) [2023-12-27 08:07:49,059][20381] Updated weights for policy 1, policy_version 359270 (0.0010) [2023-12-27 08:07:49,110][20381] Updated weights for policy 1, policy_version 359280 (0.0010) [2023-12-27 08:07:49,320][20347] Updated weights for policy 0, policy_version 359557 (0.0010) [2023-12-27 08:07:49,382][20347] Updated weights for policy 0, policy_version 359567 (0.0011) [2023-12-27 08:07:49,433][20347] Updated weights for policy 0, policy_version 359577 (0.0010) [2023-12-27 08:07:49,760][20381] Updated weights for policy 1, policy_version 359290 (0.0009) [2023-12-27 08:07:49,817][20381] Updated weights for policy 1, policy_version 359300 (0.0011) [2023-12-27 08:07:49,879][20381] Updated weights for policy 1, policy_version 359310 (0.0008) [2023-12-27 08:07:49,945][20381] Updated weights for policy 1, policy_version 359320 (0.0008) [2023-12-27 08:07:50,107][20347] Updated weights for policy 0, policy_version 359587 (0.0010) [2023-12-27 08:07:50,170][20347] Updated weights for policy 0, policy_version 359597 (0.0010) [2023-12-27 08:07:50,229][20347] Updated weights for policy 0, policy_version 359607 (0.0010) [2023-12-27 08:07:50,618][20381] Updated weights for policy 1, policy_version 359330 (0.0009) [2023-12-27 08:07:50,677][20381] Updated weights for policy 1, policy_version 359340 (0.0005) [2023-12-27 08:07:50,741][20381] Updated weights for policy 1, policy_version 359350 (0.0005) [2023-12-27 08:07:50,882][20347] Updated weights for policy 0, policy_version 359617 (0.0010) [2023-12-27 08:07:50,938][20347] Updated weights for policy 0, policy_version 359627 (0.0010) [2023-12-27 08:07:50,990][20347] Updated weights for policy 0, policy_version 359637 (0.0010) [2023-12-27 08:07:51,049][20347] Updated weights for policy 0, policy_version 359647 (0.0011) [2023-12-27 08:07:51,312][20381] Updated weights for policy 1, policy_version 359360 (0.0007) [2023-12-27 08:07:51,375][20381] Updated weights for policy 1, policy_version 359370 (0.0008) [2023-12-27 08:07:51,431][20381] Updated weights for policy 1, policy_version 359380 (0.0006) [2023-12-27 08:07:51,779][20347] Updated weights for policy 0, policy_version 359657 (0.0011) [2023-12-27 08:07:51,837][20347] Updated weights for policy 0, policy_version 359667 (0.0010) [2023-12-27 08:07:51,895][20347] Updated weights for policy 0, policy_version 359677 (0.0010) [2023-12-27 08:07:52,054][20381] Updated weights for policy 1, policy_version 359390 (0.0007) [2023-12-27 08:07:52,108][20381] Updated weights for policy 1, policy_version 359400 (0.0007) [2023-12-27 08:07:52,160][20381] Updated weights for policy 1, policy_version 359410 (0.0008) [2023-12-27 08:07:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 184123392. Throughput: 0: 10654.9, 1: 10306.3. Samples: 184090024. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:07:52,363][19170] Avg episode reward: [(0, '664.000'), (1, '980.780')] [2023-12-27 08:07:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000359416_92028928.pth... [2023-12-27 08:07:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000359680_92094464.pth... [2023-12-27 08:07:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000358200_91717632.pth [2023-12-27 08:07:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000358432_91774976.pth [2023-12-27 08:07:52,581][20347] Updated weights for policy 0, policy_version 359687 (0.0010) [2023-12-27 08:07:52,640][20347] Updated weights for policy 0, policy_version 359697 (0.0010) [2023-12-27 08:07:52,697][20347] Updated weights for policy 0, policy_version 359707 (0.0010) [2023-12-27 08:07:52,698][20291] KL-divergence is very high: 103.9306 [2023-12-27 08:07:52,896][20381] Updated weights for policy 1, policy_version 359420 (0.0009) [2023-12-27 08:07:52,953][20381] Updated weights for policy 1, policy_version 359430 (0.0007) [2023-12-27 08:07:53,015][20381] Updated weights for policy 1, policy_version 359440 (0.0007) [2023-12-27 08:07:53,289][20347] Updated weights for policy 0, policy_version 359717 (0.0007) [2023-12-27 08:07:53,348][20347] Updated weights for policy 0, policy_version 359727 (0.0006) [2023-12-27 08:07:53,413][20347] Updated weights for policy 0, policy_version 359737 (0.0006) [2023-12-27 08:07:53,669][20381] Updated weights for policy 1, policy_version 359450 (0.0007) [2023-12-27 08:07:53,733][20381] Updated weights for policy 1, policy_version 359460 (0.0006) [2023-12-27 08:07:53,798][20381] Updated weights for policy 1, policy_version 359470 (0.0006) [2023-12-27 08:07:53,860][20381] Updated weights for policy 1, policy_version 359480 (0.0007) [2023-12-27 08:07:54,023][20347] Updated weights for policy 0, policy_version 359747 (0.0005) [2023-12-27 08:07:54,086][20347] Updated weights for policy 0, policy_version 359757 (0.0006) [2023-12-27 08:07:54,142][20347] Updated weights for policy 0, policy_version 359767 (0.0008) [2023-12-27 08:07:54,510][20381] Updated weights for policy 1, policy_version 359490 (0.0011) [2023-12-27 08:07:54,560][20381] Updated weights for policy 1, policy_version 359500 (0.0010) [2023-12-27 08:07:54,620][20381] Updated weights for policy 1, policy_version 359510 (0.0008) [2023-12-27 08:07:54,744][20347] Updated weights for policy 0, policy_version 359777 (0.0007) [2023-12-27 08:07:54,807][20347] Updated weights for policy 0, policy_version 359787 (0.0007) [2023-12-27 08:07:54,871][20347] Updated weights for policy 0, policy_version 359797 (0.0007) [2023-12-27 08:07:54,916][20347] Updated weights for policy 0, policy_version 359807 (0.0005) [2023-12-27 08:07:55,330][20381] Updated weights for policy 1, policy_version 359520 (0.0007) [2023-12-27 08:07:55,396][20381] Updated weights for policy 1, policy_version 359530 (0.0008) [2023-12-27 08:07:55,460][20381] Updated weights for policy 1, policy_version 359540 (0.0008) [2023-12-27 08:07:55,549][20347] Updated weights for policy 0, policy_version 359817 (0.0007) [2023-12-27 08:07:55,603][20347] Updated weights for policy 0, policy_version 359827 (0.0007) [2023-12-27 08:07:55,657][20347] Updated weights for policy 0, policy_version 359837 (0.0008) [2023-12-27 08:07:56,129][20381] Updated weights for policy 1, policy_version 359550 (0.0009) [2023-12-27 08:07:56,183][20381] Updated weights for policy 1, policy_version 359560 (0.0010) [2023-12-27 08:07:56,225][20381] Updated weights for policy 1, policy_version 359570 (0.0010) [2023-12-27 08:07:56,299][20347] Updated weights for policy 0, policy_version 359847 (0.0008) [2023-12-27 08:07:56,356][20347] Updated weights for policy 0, policy_version 359857 (0.0007) [2023-12-27 08:07:56,410][20347] Updated weights for policy 0, policy_version 359867 (0.0006) [2023-12-27 08:07:56,901][20381] Updated weights for policy 1, policy_version 359580 (0.0010) [2023-12-27 08:07:56,945][20381] Updated weights for policy 1, policy_version 359590 (0.0010) [2023-12-27 08:07:56,998][20381] Updated weights for policy 1, policy_version 359600 (0.0011) [2023-12-27 08:07:57,006][20347] Updated weights for policy 0, policy_version 359877 (0.0007) [2023-12-27 08:07:57,059][20347] Updated weights for policy 0, policy_version 359887 (0.0006) [2023-12-27 08:07:57,102][20291] KL-divergence is very high: 111.3539 [2023-12-27 08:07:57,117][20347] Updated weights for policy 0, policy_version 359897 (0.0007) [2023-12-27 08:07:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20799.4). Total num frames: 184229888. Throughput: 0: 10652.0, 1: 10282.9. Samples: 184215856. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:07:57,363][19170] Avg episode reward: [(0, '654.850'), (1, '1000.000')] [2023-12-27 08:07:57,681][20381] Updated weights for policy 1, policy_version 359610 (0.0011) [2023-12-27 08:07:57,732][20381] Updated weights for policy 1, policy_version 359620 (0.0010) [2023-12-27 08:07:57,790][20381] Updated weights for policy 1, policy_version 359630 (0.0011) [2023-12-27 08:07:57,799][20347] Updated weights for policy 0, policy_version 359907 (0.0007) [2023-12-27 08:07:57,842][20381] Updated weights for policy 1, policy_version 359640 (0.0010) [2023-12-27 08:07:57,849][20347] Updated weights for policy 0, policy_version 359917 (0.0010) [2023-12-27 08:07:57,896][20347] Updated weights for policy 0, policy_version 359927 (0.0010) [2023-12-27 08:07:58,522][20381] Updated weights for policy 1, policy_version 359650 (0.0007) [2023-12-27 08:07:58,582][20381] Updated weights for policy 1, policy_version 359660 (0.0007) [2023-12-27 08:07:58,622][20347] Updated weights for policy 0, policy_version 359937 (0.0010) [2023-12-27 08:07:58,642][20381] Updated weights for policy 1, policy_version 359670 (0.0006) [2023-12-27 08:07:58,688][20347] Updated weights for policy 0, policy_version 359947 (0.0011) [2023-12-27 08:07:58,748][20347] Updated weights for policy 0, policy_version 359957 (0.0009) [2023-12-27 08:07:58,815][20347] Updated weights for policy 0, policy_version 359967 (0.0007) [2023-12-27 08:07:59,365][20381] Updated weights for policy 1, policy_version 359680 (0.0006) [2023-12-27 08:07:59,419][20381] Updated weights for policy 1, policy_version 359690 (0.0006) [2023-12-27 08:07:59,482][20381] Updated weights for policy 1, policy_version 359700 (0.0006) [2023-12-27 08:07:59,537][20347] Updated weights for policy 0, policy_version 359977 (0.0005) [2023-12-27 08:07:59,602][20347] Updated weights for policy 0, policy_version 359987 (0.0007) [2023-12-27 08:07:59,665][20347] Updated weights for policy 0, policy_version 359997 (0.0010) [2023-12-27 08:08:00,112][20381] Updated weights for policy 1, policy_version 359710 (0.0007) [2023-12-27 08:08:00,170][20381] Updated weights for policy 1, policy_version 359720 (0.0007) [2023-12-27 08:08:00,226][20381] Updated weights for policy 1, policy_version 359730 (0.0007) [2023-12-27 08:08:00,268][20347] Updated weights for policy 0, policy_version 360007 (0.0007) [2023-12-27 08:08:00,333][20347] Updated weights for policy 0, policy_version 360017 (0.0008) [2023-12-27 08:08:00,384][20347] Updated weights for policy 0, policy_version 360027 (0.0008) [2023-12-27 08:08:00,850][20381] Updated weights for policy 1, policy_version 359740 (0.0005) [2023-12-27 08:08:00,916][20381] Updated weights for policy 1, policy_version 359750 (0.0005) [2023-12-27 08:08:00,974][20381] Updated weights for policy 1, policy_version 359760 (0.0005) [2023-12-27 08:08:00,985][20347] Updated weights for policy 0, policy_version 360037 (0.0007) [2023-12-27 08:08:01,042][20347] Updated weights for policy 0, policy_version 360047 (0.0009) [2023-12-27 08:08:01,101][20347] Updated weights for policy 0, policy_version 360057 (0.0010) [2023-12-27 08:08:01,631][20381] Updated weights for policy 1, policy_version 359770 (0.0006) [2023-12-27 08:08:01,694][20381] Updated weights for policy 1, policy_version 359780 (0.0008) [2023-12-27 08:08:01,714][20347] Updated weights for policy 0, policy_version 360067 (0.0009) [2023-12-27 08:08:01,755][20381] Updated weights for policy 1, policy_version 359790 (0.0007) [2023-12-27 08:08:01,768][20347] Updated weights for policy 0, policy_version 360077 (0.0007) [2023-12-27 08:08:01,804][20381] Updated weights for policy 1, policy_version 359800 (0.0007) [2023-12-27 08:08:01,817][20347] Updated weights for policy 0, policy_version 360087 (0.0005) [2023-12-27 08:08:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 184328192. Throughput: 0: 10712.9, 1: 10229.0. Samples: 184341344. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:02,363][19170] Avg episode reward: [(0, '721.970'), (1, '1000.000')] [2023-12-27 08:08:02,410][20347] Updated weights for policy 0, policy_version 360097 (0.0007) [2023-12-27 08:08:02,467][20347] Updated weights for policy 0, policy_version 360107 (0.0006) [2023-12-27 08:08:02,531][20347] Updated weights for policy 0, policy_version 360117 (0.0006) [2023-12-27 08:08:02,556][20381] Updated weights for policy 1, policy_version 359810 (0.0007) [2023-12-27 08:08:02,588][20347] Updated weights for policy 0, policy_version 360127 (0.0005) [2023-12-27 08:08:02,615][20381] Updated weights for policy 1, policy_version 359820 (0.0008) [2023-12-27 08:08:02,677][20381] Updated weights for policy 1, policy_version 359830 (0.0008) [2023-12-27 08:08:03,147][20347] Updated weights for policy 0, policy_version 360137 (0.0005) [2023-12-27 08:08:03,203][20347] Updated weights for policy 0, policy_version 360147 (0.0005) [2023-12-27 08:08:03,259][20347] Updated weights for policy 0, policy_version 360157 (0.0005) [2023-12-27 08:08:03,371][20381] Updated weights for policy 1, policy_version 359840 (0.0006) [2023-12-27 08:08:03,418][20381] Updated weights for policy 1, policy_version 359850 (0.0010) [2023-12-27 08:08:03,466][20381] Updated weights for policy 1, policy_version 359860 (0.0010) [2023-12-27 08:08:03,801][20347] Updated weights for policy 0, policy_version 360167 (0.0005) [2023-12-27 08:08:03,856][20347] Updated weights for policy 0, policy_version 360177 (0.0007) [2023-12-27 08:08:03,903][20347] Updated weights for policy 0, policy_version 360187 (0.0006) [2023-12-27 08:08:04,160][20381] Updated weights for policy 1, policy_version 359870 (0.0010) [2023-12-27 08:08:04,223][20381] Updated weights for policy 1, policy_version 359880 (0.0010) [2023-12-27 08:08:04,284][20381] Updated weights for policy 1, policy_version 359890 (0.0011) [2023-12-27 08:08:04,594][20347] Updated weights for policy 0, policy_version 360197 (0.0007) [2023-12-27 08:08:04,646][20347] Updated weights for policy 0, policy_version 360207 (0.0006) [2023-12-27 08:08:04,699][20347] Updated weights for policy 0, policy_version 360217 (0.0005) [2023-12-27 08:08:04,724][20291] KL-divergence is very high: 101.2684 [2023-12-27 08:08:04,892][20381] Updated weights for policy 1, policy_version 359900 (0.0009) [2023-12-27 08:08:04,950][20381] Updated weights for policy 1, policy_version 359910 (0.0005) [2023-12-27 08:08:05,000][20381] Updated weights for policy 1, policy_version 359920 (0.0005) [2023-12-27 08:08:05,390][20347] Updated weights for policy 0, policy_version 360227 (0.0006) [2023-12-27 08:08:05,446][20347] Updated weights for policy 0, policy_version 360237 (0.0007) [2023-12-27 08:08:05,498][20347] Updated weights for policy 0, policy_version 360247 (0.0008) [2023-12-27 08:08:05,544][20381] Updated weights for policy 1, policy_version 359930 (0.0006) [2023-12-27 08:08:05,599][20381] Updated weights for policy 1, policy_version 359940 (0.0007) [2023-12-27 08:08:05,652][20381] Updated weights for policy 1, policy_version 359950 (0.0005) [2023-12-27 08:08:05,701][20381] Updated weights for policy 1, policy_version 359960 (0.0008) [2023-12-27 08:08:06,219][20347] Updated weights for policy 0, policy_version 360257 (0.0007) [2023-12-27 08:08:06,271][20347] Updated weights for policy 0, policy_version 360267 (0.0008) [2023-12-27 08:08:06,308][20381] Updated weights for policy 1, policy_version 359970 (0.0007) [2023-12-27 08:08:06,321][20347] Updated weights for policy 0, policy_version 360277 (0.0006) [2023-12-27 08:08:06,357][20381] Updated weights for policy 1, policy_version 359980 (0.0005) [2023-12-27 08:08:06,374][20347] Updated weights for policy 0, policy_version 360287 (0.0006) [2023-12-27 08:08:06,412][20381] Updated weights for policy 1, policy_version 359990 (0.0007) [2023-12-27 08:08:07,089][20347] Updated weights for policy 0, policy_version 360297 (0.0009) [2023-12-27 08:08:07,114][20381] Updated weights for policy 1, policy_version 360000 (0.0006) [2023-12-27 08:08:07,145][20347] Updated weights for policy 0, policy_version 360307 (0.0010) [2023-12-27 08:08:07,181][20381] Updated weights for policy 1, policy_version 360010 (0.0008) [2023-12-27 08:08:07,203][20347] Updated weights for policy 0, policy_version 360317 (0.0009) [2023-12-27 08:08:07,246][20381] Updated weights for policy 1, policy_version 360020 (0.0006) [2023-12-27 08:08:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 184442880. Throughput: 0: 10757.3, 1: 10255.9. Samples: 184407176. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:07,363][19170] Avg episode reward: [(0, '581.820'), (1, '1000.000')] [2023-12-27 08:08:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000360024_92184576.pth... [2023-12-27 08:08:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000360320_92258304.pth... [2023-12-27 08:08:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000358808_91873280.pth [2023-12-27 08:08:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000359072_91938816.pth [2023-12-27 08:08:07,860][20347] Updated weights for policy 0, policy_version 360327 (0.0010) [2023-12-27 08:08:07,862][20381] Updated weights for policy 1, policy_version 360030 (0.0006) [2023-12-27 08:08:07,916][20347] Updated weights for policy 0, policy_version 360337 (0.0010) [2023-12-27 08:08:07,917][20381] Updated weights for policy 1, policy_version 360040 (0.0005) [2023-12-27 08:08:07,973][20347] Updated weights for policy 0, policy_version 360347 (0.0010) [2023-12-27 08:08:07,975][20381] Updated weights for policy 1, policy_version 360050 (0.0006) [2023-12-27 08:08:08,619][20381] Updated weights for policy 1, policy_version 360060 (0.0007) [2023-12-27 08:08:08,652][20347] Updated weights for policy 0, policy_version 360357 (0.0008) [2023-12-27 08:08:08,680][20381] Updated weights for policy 1, policy_version 360070 (0.0010) [2023-12-27 08:08:08,713][20347] Updated weights for policy 0, policy_version 360367 (0.0006) [2023-12-27 08:08:08,735][20381] Updated weights for policy 1, policy_version 360080 (0.0010) [2023-12-27 08:08:08,778][20347] Updated weights for policy 0, policy_version 360377 (0.0008) [2023-12-27 08:08:09,391][20381] Updated weights for policy 1, policy_version 360090 (0.0010) [2023-12-27 08:08:09,408][20347] Updated weights for policy 0, policy_version 360387 (0.0006) [2023-12-27 08:08:09,456][20381] Updated weights for policy 1, policy_version 360100 (0.0007) [2023-12-27 08:08:09,465][20347] Updated weights for policy 0, policy_version 360397 (0.0006) [2023-12-27 08:08:09,513][20381] Updated weights for policy 1, policy_version 360110 (0.0006) [2023-12-27 08:08:09,515][20347] Updated weights for policy 0, policy_version 360407 (0.0005) [2023-12-27 08:08:09,568][20381] Updated weights for policy 1, policy_version 360120 (0.0007) [2023-12-27 08:08:10,216][20347] Updated weights for policy 0, policy_version 360417 (0.0006) [2023-12-27 08:08:10,268][20347] Updated weights for policy 0, policy_version 360427 (0.0010) [2023-12-27 08:08:10,276][20381] Updated weights for policy 1, policy_version 360130 (0.0006) [2023-12-27 08:08:10,325][20347] Updated weights for policy 0, policy_version 360437 (0.0010) [2023-12-27 08:08:10,336][20381] Updated weights for policy 1, policy_version 360140 (0.0006) [2023-12-27 08:08:10,389][20347] Updated weights for policy 0, policy_version 360447 (0.0007) [2023-12-27 08:08:10,396][20381] Updated weights for policy 1, policy_version 360150 (0.0008) [2023-12-27 08:08:10,991][20347] Updated weights for policy 0, policy_version 360457 (0.0010) [2023-12-27 08:08:11,052][20347] Updated weights for policy 0, policy_version 360467 (0.0010) [2023-12-27 08:08:11,078][20381] Updated weights for policy 1, policy_version 360160 (0.0007) [2023-12-27 08:08:11,113][20347] Updated weights for policy 0, policy_version 360477 (0.0007) [2023-12-27 08:08:11,136][20381] Updated weights for policy 1, policy_version 360170 (0.0007) [2023-12-27 08:08:11,195][20381] Updated weights for policy 1, policy_version 360180 (0.0008) [2023-12-27 08:08:11,845][20347] Updated weights for policy 0, policy_version 360487 (0.0007) [2023-12-27 08:08:11,899][20347] Updated weights for policy 0, policy_version 360497 (0.0007) [2023-12-27 08:08:11,961][20381] Updated weights for policy 1, policy_version 360190 (0.0008) [2023-12-27 08:08:11,962][20347] Updated weights for policy 0, policy_version 360507 (0.0008) [2023-12-27 08:08:12,017][20381] Updated weights for policy 1, policy_version 360200 (0.0008) [2023-12-27 08:08:12,073][20381] Updated weights for policy 1, policy_version 360210 (0.0008) [2023-12-27 08:08:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 184541184. Throughput: 0: 10651.2, 1: 10403.1. Samples: 184531188. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:12,363][19170] Avg episode reward: [(0, '308.720'), (1, '1000.000')] [2023-12-27 08:08:12,617][20347] Updated weights for policy 0, policy_version 360517 (0.0006) [2023-12-27 08:08:12,677][20347] Updated weights for policy 0, policy_version 360527 (0.0008) [2023-12-27 08:08:12,738][20347] Updated weights for policy 0, policy_version 360537 (0.0007) [2023-12-27 08:08:12,792][20381] Updated weights for policy 1, policy_version 360220 (0.0008) [2023-12-27 08:08:12,835][20381] Updated weights for policy 1, policy_version 360230 (0.0008) [2023-12-27 08:08:12,880][20381] Updated weights for policy 1, policy_version 360240 (0.0007) [2023-12-27 08:08:13,361][20347] Updated weights for policy 0, policy_version 360547 (0.0007) [2023-12-27 08:08:13,419][20347] Updated weights for policy 0, policy_version 360557 (0.0007) [2023-12-27 08:08:13,471][20347] Updated weights for policy 0, policy_version 360567 (0.0008) [2023-12-27 08:08:13,584][20381] Updated weights for policy 1, policy_version 360250 (0.0007) [2023-12-27 08:08:13,646][20381] Updated weights for policy 1, policy_version 360260 (0.0008) [2023-12-27 08:08:13,698][20381] Updated weights for policy 1, policy_version 360270 (0.0007) [2023-12-27 08:08:13,753][20381] Updated weights for policy 1, policy_version 360280 (0.0008) [2023-12-27 08:08:14,181][20347] Updated weights for policy 0, policy_version 360577 (0.0009) [2023-12-27 08:08:14,227][20347] Updated weights for policy 0, policy_version 360587 (0.0008) [2023-12-27 08:08:14,277][20347] Updated weights for policy 0, policy_version 360597 (0.0008) [2023-12-27 08:08:14,333][20347] Updated weights for policy 0, policy_version 360607 (0.0006) [2023-12-27 08:08:14,445][20381] Updated weights for policy 1, policy_version 360290 (0.0008) [2023-12-27 08:08:14,499][20381] Updated weights for policy 1, policy_version 360300 (0.0006) [2023-12-27 08:08:14,562][20381] Updated weights for policy 1, policy_version 360310 (0.0006) [2023-12-27 08:08:15,023][20347] Updated weights for policy 0, policy_version 360617 (0.0010) [2023-12-27 08:08:15,083][20347] Updated weights for policy 0, policy_version 360627 (0.0010) [2023-12-27 08:08:15,131][20347] Updated weights for policy 0, policy_version 360637 (0.0010) [2023-12-27 08:08:15,196][20381] Updated weights for policy 1, policy_version 360320 (0.0007) [2023-12-27 08:08:15,251][20381] Updated weights for policy 1, policy_version 360330 (0.0008) [2023-12-27 08:08:15,297][20381] Updated weights for policy 1, policy_version 360340 (0.0008) [2023-12-27 08:08:15,773][20347] Updated weights for policy 0, policy_version 360647 (0.0007) [2023-12-27 08:08:15,835][20347] Updated weights for policy 0, policy_version 360657 (0.0006) [2023-12-27 08:08:15,886][20347] Updated weights for policy 0, policy_version 360667 (0.0008) [2023-12-27 08:08:15,981][20381] Updated weights for policy 1, policy_version 360350 (0.0006) [2023-12-27 08:08:16,044][20381] Updated weights for policy 1, policy_version 360360 (0.0005) [2023-12-27 08:08:16,103][20381] Updated weights for policy 1, policy_version 360370 (0.0008) [2023-12-27 08:08:16,439][20347] Updated weights for policy 0, policy_version 360677 (0.0005) [2023-12-27 08:08:16,489][20347] Updated weights for policy 0, policy_version 360687 (0.0005) [2023-12-27 08:08:16,540][20347] Updated weights for policy 0, policy_version 360697 (0.0005) [2023-12-27 08:08:16,799][20381] Updated weights for policy 1, policy_version 360380 (0.0009) [2023-12-27 08:08:16,848][20381] Updated weights for policy 1, policy_version 360390 (0.0007) [2023-12-27 08:08:16,907][20381] Updated weights for policy 1, policy_version 360400 (0.0007) [2023-12-27 08:08:17,151][20347] Updated weights for policy 0, policy_version 360707 (0.0006) [2023-12-27 08:08:17,211][20347] Updated weights for policy 0, policy_version 360717 (0.0005) [2023-12-27 08:08:17,267][20347] Updated weights for policy 0, policy_version 360727 (0.0005) [2023-12-27 08:08:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 184647680. Throughput: 0: 10594.3, 1: 10352.5. Samples: 184655124. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:17,363][19170] Avg episode reward: [(0, '298.820'), (1, '1000.000')] [2023-12-27 08:08:17,564][20381] Updated weights for policy 1, policy_version 360410 (0.0008) [2023-12-27 08:08:17,620][20381] Updated weights for policy 1, policy_version 360420 (0.0008) [2023-12-27 08:08:17,680][20381] Updated weights for policy 1, policy_version 360430 (0.0008) [2023-12-27 08:08:17,742][20381] Updated weights for policy 1, policy_version 360440 (0.0009) [2023-12-27 08:08:17,886][20347] Updated weights for policy 0, policy_version 360737 (0.0007) [2023-12-27 08:08:17,939][20347] Updated weights for policy 0, policy_version 360747 (0.0006) [2023-12-27 08:08:17,994][20347] Updated weights for policy 0, policy_version 360757 (0.0007) [2023-12-27 08:08:18,050][20347] Updated weights for policy 0, policy_version 360767 (0.0008) [2023-12-27 08:08:18,438][20381] Updated weights for policy 1, policy_version 360450 (0.0008) [2023-12-27 08:08:18,503][20381] Updated weights for policy 1, policy_version 360460 (0.0007) [2023-12-27 08:08:18,564][20381] Updated weights for policy 1, policy_version 360470 (0.0007) [2023-12-27 08:08:18,748][20347] Updated weights for policy 0, policy_version 360777 (0.0008) [2023-12-27 08:08:18,804][20347] Updated weights for policy 0, policy_version 360787 (0.0007) [2023-12-27 08:08:18,857][20347] Updated weights for policy 0, policy_version 360797 (0.0007) [2023-12-27 08:08:19,256][20381] Updated weights for policy 1, policy_version 360480 (0.0007) [2023-12-27 08:08:19,317][20381] Updated weights for policy 1, policy_version 360490 (0.0009) [2023-12-27 08:08:19,380][20381] Updated weights for policy 1, policy_version 360500 (0.0007) [2023-12-27 08:08:19,620][20347] Updated weights for policy 0, policy_version 360807 (0.0008) [2023-12-27 08:08:19,673][20347] Updated weights for policy 0, policy_version 360817 (0.0007) [2023-12-27 08:08:19,727][20347] Updated weights for policy 0, policy_version 360827 (0.0008) [2023-12-27 08:08:20,084][20381] Updated weights for policy 1, policy_version 360510 (0.0009) [2023-12-27 08:08:20,152][20381] Updated weights for policy 1, policy_version 360520 (0.0007) [2023-12-27 08:08:20,223][20381] Updated weights for policy 1, policy_version 360530 (0.0011) [2023-12-27 08:08:20,423][20347] Updated weights for policy 0, policy_version 360837 (0.0007) [2023-12-27 08:08:20,482][20347] Updated weights for policy 0, policy_version 360847 (0.0008) [2023-12-27 08:08:20,541][20347] Updated weights for policy 0, policy_version 360857 (0.0008) [2023-12-27 08:08:20,828][20381] Updated weights for policy 1, policy_version 360540 (0.0010) [2023-12-27 08:08:20,888][20381] Updated weights for policy 1, policy_version 360550 (0.0007) [2023-12-27 08:08:20,952][20381] Updated weights for policy 1, policy_version 360560 (0.0009) [2023-12-27 08:08:21,208][20347] Updated weights for policy 0, policy_version 360867 (0.0006) [2023-12-27 08:08:21,266][20347] Updated weights for policy 0, policy_version 360877 (0.0008) [2023-12-27 08:08:21,324][20347] Updated weights for policy 0, policy_version 360887 (0.0008) [2023-12-27 08:08:21,727][20381] Updated weights for policy 1, policy_version 360570 (0.0009) [2023-12-27 08:08:21,783][20381] Updated weights for policy 1, policy_version 360580 (0.0010) [2023-12-27 08:08:21,848][20381] Updated weights for policy 1, policy_version 360590 (0.0011) [2023-12-27 08:08:21,911][20381] Updated weights for policy 1, policy_version 360600 (0.0010) [2023-12-27 08:08:22,010][20347] Updated weights for policy 0, policy_version 360897 (0.0008) [2023-12-27 08:08:22,075][20347] Updated weights for policy 0, policy_version 360907 (0.0006) [2023-12-27 08:08:22,132][20347] Updated weights for policy 0, policy_version 360917 (0.0007) [2023-12-27 08:08:22,191][20347] Updated weights for policy 0, policy_version 360927 (0.0007) [2023-12-27 08:08:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 184745984. Throughput: 0: 10517.7, 1: 10322.0. Samples: 184714800. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:22,363][19170] Avg episode reward: [(0, '551.610'), (1, '1000.000')] [2023-12-27 08:08:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000360600_92332032.pth... [2023-12-27 08:08:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000360928_92413952.pth... [2023-12-27 08:08:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000359416_92028928.pth [2023-12-27 08:08:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000359680_92094464.pth [2023-12-27 08:08:22,605][20381] Updated weights for policy 1, policy_version 360610 (0.0009) [2023-12-27 08:08:22,668][20381] Updated weights for policy 1, policy_version 360620 (0.0009) [2023-12-27 08:08:22,735][20381] Updated weights for policy 1, policy_version 360630 (0.0006) [2023-12-27 08:08:22,819][20347] Updated weights for policy 0, policy_version 360937 (0.0008) [2023-12-27 08:08:22,880][20347] Updated weights for policy 0, policy_version 360947 (0.0008) [2023-12-27 08:08:22,939][20347] Updated weights for policy 0, policy_version 360957 (0.0007) [2023-12-27 08:08:23,376][20381] Updated weights for policy 1, policy_version 360640 (0.0006) [2023-12-27 08:08:23,422][20381] Updated weights for policy 1, policy_version 360650 (0.0007) [2023-12-27 08:08:23,475][20381] Updated weights for policy 1, policy_version 360660 (0.0005) [2023-12-27 08:08:23,555][20347] Updated weights for policy 0, policy_version 360967 (0.0006) [2023-12-27 08:08:23,602][20347] Updated weights for policy 0, policy_version 360977 (0.0005) [2023-12-27 08:08:23,651][20347] Updated weights for policy 0, policy_version 360987 (0.0005) [2023-12-27 08:08:24,067][20381] Updated weights for policy 1, policy_version 360670 (0.0009) [2023-12-27 08:08:24,129][20381] Updated weights for policy 1, policy_version 360680 (0.0011) [2023-12-27 08:08:24,192][20381] Updated weights for policy 1, policy_version 360690 (0.0010) [2023-12-27 08:08:24,241][20347] Updated weights for policy 0, policy_version 360997 (0.0008) [2023-12-27 08:08:24,299][20347] Updated weights for policy 0, policy_version 361007 (0.0010) [2023-12-27 08:08:24,355][20347] Updated weights for policy 0, policy_version 361017 (0.0010) [2023-12-27 08:08:24,890][20381] Updated weights for policy 1, policy_version 360700 (0.0011) [2023-12-27 08:08:24,947][20381] Updated weights for policy 1, policy_version 360710 (0.0010) [2023-12-27 08:08:25,003][20347] Updated weights for policy 0, policy_version 361027 (0.0010) [2023-12-27 08:08:25,012][20381] Updated weights for policy 1, policy_version 360720 (0.0011) [2023-12-27 08:08:25,052][20347] Updated weights for policy 0, policy_version 361037 (0.0010) [2023-12-27 08:08:25,096][20347] Updated weights for policy 0, policy_version 361047 (0.0010) [2023-12-27 08:08:25,655][20347] Updated weights for policy 0, policy_version 361057 (0.0007) [2023-12-27 08:08:25,685][20381] Updated weights for policy 1, policy_version 360730 (0.0011) [2023-12-27 08:08:25,714][20347] Updated weights for policy 0, policy_version 361067 (0.0007) [2023-12-27 08:08:25,740][20381] Updated weights for policy 1, policy_version 360740 (0.0010) [2023-12-27 08:08:25,764][20347] Updated weights for policy 0, policy_version 361077 (0.0005) [2023-12-27 08:08:25,799][20381] Updated weights for policy 1, policy_version 360750 (0.0010) [2023-12-27 08:08:25,817][20347] Updated weights for policy 0, policy_version 361087 (0.0005) [2023-12-27 08:08:25,852][20381] Updated weights for policy 1, policy_version 360760 (0.0007) [2023-12-27 08:08:26,379][20347] Updated weights for policy 0, policy_version 361097 (0.0005) [2023-12-27 08:08:26,435][20347] Updated weights for policy 0, policy_version 361107 (0.0005) [2023-12-27 08:08:26,474][20381] Updated weights for policy 1, policy_version 360770 (0.0010) [2023-12-27 08:08:26,483][20347] Updated weights for policy 0, policy_version 361117 (0.0005) [2023-12-27 08:08:26,526][20381] Updated weights for policy 1, policy_version 360780 (0.0010) [2023-12-27 08:08:26,571][20381] Updated weights for policy 1, policy_version 360790 (0.0010) [2023-12-27 08:08:27,130][20347] Updated weights for policy 0, policy_version 361127 (0.0010) [2023-12-27 08:08:27,182][20347] Updated weights for policy 0, policy_version 361137 (0.0010) [2023-12-27 08:08:27,234][20347] Updated weights for policy 0, policy_version 361147 (0.0010) [2023-12-27 08:08:27,240][20381] Updated weights for policy 1, policy_version 360800 (0.0009) [2023-12-27 08:08:27,287][20381] Updated weights for policy 1, policy_version 360810 (0.0005) [2023-12-27 08:08:27,343][20381] Updated weights for policy 1, policy_version 360820 (0.0007) [2023-12-27 08:08:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 184860672. Throughput: 0: 10583.1, 1: 10290.9. Samples: 184841892. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:27,363][19170] Avg episode reward: [(0, '608.140'), (1, '1000.000')] [2023-12-27 08:08:27,913][20347] Updated weights for policy 0, policy_version 361157 (0.0010) [2023-12-27 08:08:27,928][20381] Updated weights for policy 1, policy_version 360830 (0.0005) [2023-12-27 08:08:27,959][20347] Updated weights for policy 0, policy_version 361167 (0.0010) [2023-12-27 08:08:27,980][20381] Updated weights for policy 1, policy_version 360840 (0.0005) [2023-12-27 08:08:28,008][20347] Updated weights for policy 0, policy_version 361177 (0.0010) [2023-12-27 08:08:28,039][20381] Updated weights for policy 1, policy_version 360850 (0.0005) [2023-12-27 08:08:28,677][20381] Updated weights for policy 1, policy_version 360860 (0.0007) [2023-12-27 08:08:28,690][20347] Updated weights for policy 0, policy_version 361187 (0.0010) [2023-12-27 08:08:28,727][20381] Updated weights for policy 1, policy_version 360870 (0.0010) [2023-12-27 08:08:28,748][20347] Updated weights for policy 0, policy_version 361197 (0.0010) [2023-12-27 08:08:28,781][20381] Updated weights for policy 1, policy_version 360880 (0.0010) [2023-12-27 08:08:28,807][20347] Updated weights for policy 0, policy_version 361207 (0.0010) [2023-12-27 08:08:29,389][20381] Updated weights for policy 1, policy_version 360890 (0.0010) [2023-12-27 08:08:29,444][20381] Updated weights for policy 1, policy_version 360900 (0.0010) [2023-12-27 08:08:29,469][20347] Updated weights for policy 0, policy_version 361217 (0.0010) [2023-12-27 08:08:29,501][20381] Updated weights for policy 1, policy_version 360910 (0.0007) [2023-12-27 08:08:29,520][20347] Updated weights for policy 0, policy_version 361227 (0.0010) [2023-12-27 08:08:29,557][20381] Updated weights for policy 1, policy_version 360920 (0.0011) [2023-12-27 08:08:29,575][20347] Updated weights for policy 0, policy_version 361237 (0.0010) [2023-12-27 08:08:29,623][20347] Updated weights for policy 0, policy_version 361247 (0.0010) [2023-12-27 08:08:30,196][20381] Updated weights for policy 1, policy_version 360930 (0.0005) [2023-12-27 08:08:30,241][20381] Updated weights for policy 1, policy_version 360940 (0.0005) [2023-12-27 08:08:30,292][20381] Updated weights for policy 1, policy_version 360950 (0.0005) [2023-12-27 08:08:30,339][20347] Updated weights for policy 0, policy_version 361257 (0.0010) [2023-12-27 08:08:30,400][20347] Updated weights for policy 0, policy_version 361267 (0.0010) [2023-12-27 08:08:30,462][20347] Updated weights for policy 0, policy_version 361277 (0.0010) [2023-12-27 08:08:30,901][20381] Updated weights for policy 1, policy_version 360960 (0.0009) [2023-12-27 08:08:30,946][20381] Updated weights for policy 1, policy_version 360970 (0.0010) [2023-12-27 08:08:30,991][20381] Updated weights for policy 1, policy_version 360980 (0.0010) [2023-12-27 08:08:31,024][20347] Updated weights for policy 0, policy_version 361287 (0.0007) [2023-12-27 08:08:31,084][20347] Updated weights for policy 0, policy_version 361297 (0.0007) [2023-12-27 08:08:31,147][20347] Updated weights for policy 0, policy_version 361307 (0.0007) [2023-12-27 08:08:31,682][20381] Updated weights for policy 1, policy_version 360990 (0.0009) [2023-12-27 08:08:31,741][20381] Updated weights for policy 1, policy_version 361000 (0.0011) [2023-12-27 08:08:31,798][20381] Updated weights for policy 1, policy_version 361010 (0.0009) [2023-12-27 08:08:31,866][20347] Updated weights for policy 0, policy_version 361317 (0.0007) [2023-12-27 08:08:31,930][20347] Updated weights for policy 0, policy_version 361327 (0.0008) [2023-12-27 08:08:31,991][20347] Updated weights for policy 0, policy_version 361337 (0.0008) [2023-12-27 08:08:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 184958976. Throughput: 0: 10574.1, 1: 10364.7. Samples: 184971004. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:32,363][19170] Avg episode reward: [(0, '329.180'), (1, '1000.000')] [2023-12-27 08:08:32,393][20381] Updated weights for policy 1, policy_version 361020 (0.0008) [2023-12-27 08:08:32,447][20381] Updated weights for policy 1, policy_version 361030 (0.0009) [2023-12-27 08:08:32,501][20381] Updated weights for policy 1, policy_version 361040 (0.0005) [2023-12-27 08:08:32,676][20347] Updated weights for policy 0, policy_version 361347 (0.0008) [2023-12-27 08:08:32,732][20347] Updated weights for policy 0, policy_version 361357 (0.0008) [2023-12-27 08:08:32,789][20347] Updated weights for policy 0, policy_version 361367 (0.0008) [2023-12-27 08:08:33,128][20381] Updated weights for policy 1, policy_version 361050 (0.0006) [2023-12-27 08:08:33,186][20381] Updated weights for policy 1, policy_version 361060 (0.0005) [2023-12-27 08:08:33,239][20381] Updated weights for policy 1, policy_version 361070 (0.0005) [2023-12-27 08:08:33,297][20381] Updated weights for policy 1, policy_version 361080 (0.0005) [2023-12-27 08:08:33,431][20347] Updated weights for policy 0, policy_version 361377 (0.0006) [2023-12-27 08:08:33,483][20347] Updated weights for policy 0, policy_version 361387 (0.0007) [2023-12-27 08:08:33,530][20347] Updated weights for policy 0, policy_version 361397 (0.0007) [2023-12-27 08:08:33,573][20347] Updated weights for policy 0, policy_version 361407 (0.0007) [2023-12-27 08:08:33,848][20381] Updated weights for policy 1, policy_version 361090 (0.0007) [2023-12-27 08:08:33,901][20381] Updated weights for policy 1, policy_version 361100 (0.0007) [2023-12-27 08:08:33,956][20381] Updated weights for policy 1, policy_version 361110 (0.0006) [2023-12-27 08:08:34,286][20347] Updated weights for policy 0, policy_version 361417 (0.0010) [2023-12-27 08:08:34,341][20347] Updated weights for policy 0, policy_version 361427 (0.0009) [2023-12-27 08:08:34,406][20347] Updated weights for policy 0, policy_version 361437 (0.0006) [2023-12-27 08:08:34,580][20381] Updated weights for policy 1, policy_version 361120 (0.0006) [2023-12-27 08:08:34,637][20381] Updated weights for policy 1, policy_version 361130 (0.0005) [2023-12-27 08:08:34,696][20381] Updated weights for policy 1, policy_version 361140 (0.0006) [2023-12-27 08:08:35,119][20347] Updated weights for policy 0, policy_version 361447 (0.0009) [2023-12-27 08:08:35,168][20347] Updated weights for policy 0, policy_version 361457 (0.0005) [2023-12-27 08:08:35,219][20347] Updated weights for policy 0, policy_version 361467 (0.0005) [2023-12-27 08:08:35,266][20381] Updated weights for policy 1, policy_version 361150 (0.0006) [2023-12-27 08:08:35,324][20381] Updated weights for policy 1, policy_version 361160 (0.0007) [2023-12-27 08:08:35,378][20381] Updated weights for policy 1, policy_version 361170 (0.0007) [2023-12-27 08:08:35,843][20347] Updated weights for policy 0, policy_version 361477 (0.0006) [2023-12-27 08:08:35,895][20347] Updated weights for policy 0, policy_version 361487 (0.0010) [2023-12-27 08:08:35,945][20347] Updated weights for policy 0, policy_version 361497 (0.0010) [2023-12-27 08:08:35,975][20381] Updated weights for policy 1, policy_version 361180 (0.0008) [2023-12-27 08:08:36,026][20381] Updated weights for policy 1, policy_version 361190 (0.0010) [2023-12-27 08:08:36,075][20381] Updated weights for policy 1, policy_version 361200 (0.0010) [2023-12-27 08:08:36,607][20347] Updated weights for policy 0, policy_version 361507 (0.0009) [2023-12-27 08:08:36,669][20347] Updated weights for policy 0, policy_version 361517 (0.0010) [2023-12-27 08:08:36,728][20347] Updated weights for policy 0, policy_version 361527 (0.0010) [2023-12-27 08:08:36,800][20381] Updated weights for policy 1, policy_version 361210 (0.0010) [2023-12-27 08:08:36,847][20381] Updated weights for policy 1, policy_version 361220 (0.0007) [2023-12-27 08:08:36,899][20381] Updated weights for policy 1, policy_version 361230 (0.0007) [2023-12-27 08:08:36,947][20381] Updated weights for policy 1, policy_version 361240 (0.0007) [2023-12-27 08:08:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185065472. Throughput: 0: 10569.1, 1: 10453.6. Samples: 185036048. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:37,364][19170] Avg episode reward: [(0, '162.760'), (1, '1000.000')] [2023-12-27 08:08:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000361240_92495872.pth... [2023-12-27 08:08:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000361536_92569600.pth... [2023-12-27 08:08:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000360320_92258304.pth [2023-12-27 08:08:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000360024_92184576.pth [2023-12-27 08:08:37,394][20347] Updated weights for policy 0, policy_version 361537 (0.0010) [2023-12-27 08:08:37,446][20347] Updated weights for policy 0, policy_version 361547 (0.0010) [2023-12-27 08:08:37,504][20347] Updated weights for policy 0, policy_version 361557 (0.0010) [2023-12-27 08:08:37,567][20347] Updated weights for policy 0, policy_version 361567 (0.0011) [2023-12-27 08:08:37,671][20381] Updated weights for policy 1, policy_version 361250 (0.0007) [2023-12-27 08:08:37,726][20381] Updated weights for policy 1, policy_version 361260 (0.0006) [2023-12-27 08:08:37,776][20381] Updated weights for policy 1, policy_version 361270 (0.0008) [2023-12-27 08:08:38,267][20347] Updated weights for policy 0, policy_version 361577 (0.0010) [2023-12-27 08:08:38,325][20347] Updated weights for policy 0, policy_version 361587 (0.0010) [2023-12-27 08:08:38,388][20347] Updated weights for policy 0, policy_version 361597 (0.0007) [2023-12-27 08:08:38,451][20381] Updated weights for policy 1, policy_version 361280 (0.0008) [2023-12-27 08:08:38,506][20381] Updated weights for policy 1, policy_version 361290 (0.0007) [2023-12-27 08:08:38,568][20381] Updated weights for policy 1, policy_version 361301 (0.0008) [2023-12-27 08:08:39,027][20347] Updated weights for policy 0, policy_version 361607 (0.0008) [2023-12-27 08:08:39,087][20347] Updated weights for policy 0, policy_version 361617 (0.0009) [2023-12-27 08:08:39,177][20347] Updated weights for policy 0, policy_version 361627 (0.0007) [2023-12-27 08:08:39,267][20381] Updated weights for policy 1, policy_version 361311 (0.0006) [2023-12-27 08:08:39,325][20381] Updated weights for policy 1, policy_version 361321 (0.0006) [2023-12-27 08:08:39,391][20381] Updated weights for policy 1, policy_version 361331 (0.0008) [2023-12-27 08:08:39,874][20347] Updated weights for policy 0, policy_version 361637 (0.0008) [2023-12-27 08:08:39,927][20347] Updated weights for policy 0, policy_version 361647 (0.0006) [2023-12-27 08:08:39,981][20347] Updated weights for policy 0, policy_version 361657 (0.0007) [2023-12-27 08:08:40,053][20381] Updated weights for policy 1, policy_version 361341 (0.0008) [2023-12-27 08:08:40,108][20381] Updated weights for policy 1, policy_version 361351 (0.0008) [2023-12-27 08:08:40,167][20381] Updated weights for policy 1, policy_version 361361 (0.0008) [2023-12-27 08:08:40,646][20347] Updated weights for policy 0, policy_version 361667 (0.0007) [2023-12-27 08:08:40,702][20347] Updated weights for policy 0, policy_version 361677 (0.0008) [2023-12-27 08:08:40,759][20347] Updated weights for policy 0, policy_version 361687 (0.0008) [2023-12-27 08:08:40,897][20381] Updated weights for policy 1, policy_version 361371 (0.0008) [2023-12-27 08:08:40,946][20381] Updated weights for policy 1, policy_version 361381 (0.0007) [2023-12-27 08:08:40,991][20381] Updated weights for policy 1, policy_version 361391 (0.0007) [2023-12-27 08:08:41,471][20347] Updated weights for policy 0, policy_version 361697 (0.0008) [2023-12-27 08:08:41,529][20347] Updated weights for policy 0, policy_version 361707 (0.0008) [2023-12-27 08:08:41,590][20347] Updated weights for policy 0, policy_version 361717 (0.0008) [2023-12-27 08:08:41,647][20347] Updated weights for policy 0, policy_version 361727 (0.0008) [2023-12-27 08:08:41,760][20381] Updated weights for policy 1, policy_version 361401 (0.0008) [2023-12-27 08:08:41,819][20381] Updated weights for policy 1, policy_version 361411 (0.0008) [2023-12-27 08:08:41,878][20381] Updated weights for policy 1, policy_version 361421 (0.0008) [2023-12-27 08:08:41,937][20381] Updated weights for policy 1, policy_version 361431 (0.0006) [2023-12-27 08:08:42,270][20347] Updated weights for policy 0, policy_version 361737 (0.0006) [2023-12-27 08:08:42,336][20347] Updated weights for policy 0, policy_version 361747 (0.0007) [2023-12-27 08:08:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185163776. Throughput: 0: 10527.4, 1: 10416.5. Samples: 185158332. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:42,363][19170] Avg episode reward: [(0, '298.230'), (1, '1000.000')] [2023-12-27 08:08:42,394][20347] Updated weights for policy 0, policy_version 361757 (0.0008) [2023-12-27 08:08:42,617][20381] Updated weights for policy 1, policy_version 361441 (0.0011) [2023-12-27 08:08:42,677][20381] Updated weights for policy 1, policy_version 361451 (0.0011) [2023-12-27 08:08:42,741][20381] Updated weights for policy 1, policy_version 361461 (0.0011) [2023-12-27 08:08:43,019][20347] Updated weights for policy 0, policy_version 361767 (0.0008) [2023-12-27 08:08:43,082][20347] Updated weights for policy 0, policy_version 361777 (0.0008) [2023-12-27 08:08:43,148][20347] Updated weights for policy 0, policy_version 361787 (0.0006) [2023-12-27 08:08:43,426][20381] Updated weights for policy 1, policy_version 361471 (0.0007) [2023-12-27 08:08:43,477][20381] Updated weights for policy 1, policy_version 361481 (0.0005) [2023-12-27 08:08:43,541][20381] Updated weights for policy 1, policy_version 361491 (0.0007) [2023-12-27 08:08:43,855][20347] Updated weights for policy 0, policy_version 361797 (0.0006) [2023-12-27 08:08:43,910][20347] Updated weights for policy 0, policy_version 361807 (0.0007) [2023-12-27 08:08:43,959][20347] Updated weights for policy 0, policy_version 361817 (0.0008) [2023-12-27 08:08:44,069][20381] Updated weights for policy 1, policy_version 361501 (0.0006) [2023-12-27 08:08:44,121][20381] Updated weights for policy 1, policy_version 361511 (0.0006) [2023-12-27 08:08:44,173][20381] Updated weights for policy 1, policy_version 361521 (0.0006) [2023-12-27 08:08:44,684][20347] Updated weights for policy 0, policy_version 361827 (0.0009) [2023-12-27 08:08:44,733][20347] Updated weights for policy 0, policy_version 361837 (0.0010) [2023-12-27 08:08:44,783][20381] Updated weights for policy 1, policy_version 361531 (0.0006) [2023-12-27 08:08:44,792][20347] Updated weights for policy 0, policy_version 361847 (0.0010) [2023-12-27 08:08:44,842][20381] Updated weights for policy 1, policy_version 361541 (0.0008) [2023-12-27 08:08:44,899][20381] Updated weights for policy 1, policy_version 361551 (0.0008) [2023-12-27 08:08:45,417][20347] Updated weights for policy 0, policy_version 361857 (0.0005) [2023-12-27 08:08:45,475][20347] Updated weights for policy 0, policy_version 361867 (0.0010) [2023-12-27 08:08:45,524][20347] Updated weights for policy 0, policy_version 361877 (0.0010) [2023-12-27 08:08:45,575][20347] Updated weights for policy 0, policy_version 361887 (0.0010) [2023-12-27 08:08:45,594][20381] Updated weights for policy 1, policy_version 361561 (0.0007) [2023-12-27 08:08:45,644][20381] Updated weights for policy 1, policy_version 361571 (0.0005) [2023-12-27 08:08:45,696][20381] Updated weights for policy 1, policy_version 361581 (0.0007) [2023-12-27 08:08:45,742][20381] Updated weights for policy 1, policy_version 361591 (0.0007) [2023-12-27 08:08:46,233][20347] Updated weights for policy 0, policy_version 361897 (0.0010) [2023-12-27 08:08:46,283][20347] Updated weights for policy 0, policy_version 361907 (0.0010) [2023-12-27 08:08:46,338][20347] Updated weights for policy 0, policy_version 361917 (0.0010) [2023-12-27 08:08:46,386][20381] Updated weights for policy 1, policy_version 361601 (0.0006) [2023-12-27 08:08:46,446][20381] Updated weights for policy 1, policy_version 361611 (0.0005) [2023-12-27 08:08:46,498][20381] Updated weights for policy 1, policy_version 361621 (0.0007) [2023-12-27 08:08:46,954][20347] Updated weights for policy 0, policy_version 361927 (0.0010) [2023-12-27 08:08:47,004][20347] Updated weights for policy 0, policy_version 361937 (0.0006) [2023-12-27 08:08:47,059][20347] Updated weights for policy 0, policy_version 361947 (0.0005) [2023-12-27 08:08:47,139][20381] Updated weights for policy 1, policy_version 361631 (0.0011) [2023-12-27 08:08:47,200][20381] Updated weights for policy 1, policy_version 361641 (0.0010) [2023-12-27 08:08:47,261][20381] Updated weights for policy 1, policy_version 361651 (0.0008) [2023-12-27 08:08:47,363][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 185278464. Throughput: 0: 10509.8, 1: 10459.4. Samples: 185284964. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:47,364][19170] Avg episode reward: [(0, '301.030'), (1, '1000.000')] [2023-12-27 08:08:47,593][20347] Updated weights for policy 0, policy_version 361957 (0.0005) [2023-12-27 08:08:47,641][20347] Updated weights for policy 0, policy_version 361967 (0.0005) [2023-12-27 08:08:47,700][20347] Updated weights for policy 0, policy_version 361977 (0.0007) [2023-12-27 08:08:47,908][20381] Updated weights for policy 1, policy_version 361661 (0.0007) [2023-12-27 08:08:47,961][20381] Updated weights for policy 1, policy_version 361671 (0.0008) [2023-12-27 08:08:48,011][20381] Updated weights for policy 1, policy_version 361681 (0.0009) [2023-12-27 08:08:48,346][20347] Updated weights for policy 0, policy_version 361987 (0.0008) [2023-12-27 08:08:48,399][20347] Updated weights for policy 0, policy_version 361997 (0.0007) [2023-12-27 08:08:48,454][20347] Updated weights for policy 0, policy_version 362007 (0.0008) [2023-12-27 08:08:48,773][20381] Updated weights for policy 1, policy_version 361691 (0.0008) [2023-12-27 08:08:48,830][20381] Updated weights for policy 1, policy_version 361701 (0.0007) [2023-12-27 08:08:48,893][20381] Updated weights for policy 1, policy_version 361711 (0.0008) [2023-12-27 08:08:49,140][20347] Updated weights for policy 0, policy_version 362017 (0.0007) [2023-12-27 08:08:49,201][20347] Updated weights for policy 0, policy_version 362027 (0.0005) [2023-12-27 08:08:49,265][20347] Updated weights for policy 0, policy_version 362037 (0.0008) [2023-12-27 08:08:49,319][20347] Updated weights for policy 0, policy_version 362047 (0.0008) [2023-12-27 08:08:49,609][20381] Updated weights for policy 1, policy_version 361721 (0.0007) [2023-12-27 08:08:49,661][20381] Updated weights for policy 1, policy_version 361731 (0.0009) [2023-12-27 08:08:49,709][20381] Updated weights for policy 1, policy_version 361741 (0.0011) [2023-12-27 08:08:49,765][20381] Updated weights for policy 1, policy_version 361751 (0.0011) [2023-12-27 08:08:49,979][20347] Updated weights for policy 0, policy_version 362057 (0.0007) [2023-12-27 08:08:50,034][20347] Updated weights for policy 0, policy_version 362067 (0.0007) [2023-12-27 08:08:50,093][20347] Updated weights for policy 0, policy_version 362077 (0.0008) [2023-12-27 08:08:50,466][20381] Updated weights for policy 1, policy_version 361761 (0.0008) [2023-12-27 08:08:50,515][20381] Updated weights for policy 1, policy_version 361771 (0.0008) [2023-12-27 08:08:50,576][20381] Updated weights for policy 1, policy_version 361781 (0.0008) [2023-12-27 08:08:50,712][20347] Updated weights for policy 0, policy_version 362087 (0.0007) [2023-12-27 08:08:50,770][20347] Updated weights for policy 0, policy_version 362097 (0.0006) [2023-12-27 08:08:50,835][20347] Updated weights for policy 0, policy_version 362107 (0.0006) [2023-12-27 08:08:51,306][20381] Updated weights for policy 1, policy_version 361791 (0.0010) [2023-12-27 08:08:51,367][20381] Updated weights for policy 1, policy_version 361801 (0.0010) [2023-12-27 08:08:51,429][20381] Updated weights for policy 1, policy_version 361811 (0.0009) [2023-12-27 08:08:51,524][20347] Updated weights for policy 0, policy_version 362117 (0.0007) [2023-12-27 08:08:51,595][20347] Updated weights for policy 0, policy_version 362127 (0.0007) [2023-12-27 08:08:51,660][20347] Updated weights for policy 0, policy_version 362137 (0.0006) [2023-12-27 08:08:52,136][20381] Updated weights for policy 1, policy_version 361821 (0.0008) [2023-12-27 08:08:52,185][20381] Updated weights for policy 1, policy_version 361831 (0.0010) [2023-12-27 08:08:52,233][20381] Updated weights for policy 1, policy_version 361841 (0.0010) [2023-12-27 08:08:52,315][20347] Updated weights for policy 0, policy_version 362147 (0.0008) [2023-12-27 08:08:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185376768. Throughput: 0: 10473.7, 1: 10403.1. Samples: 185346632. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:52,363][19170] Avg episode reward: [(0, '367.450'), (1, '1000.000')] [2023-12-27 08:08:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000361848_92651520.pth... [2023-12-27 08:08:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000360600_92332032.pth [2023-12-27 08:08:52,382][20347] Updated weights for policy 0, policy_version 362157 (0.0008) [2023-12-27 08:08:52,434][20347] Updated weights for policy 0, policy_version 362167 (0.0007) [2023-12-27 08:08:52,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000362176_92733440.pth... [2023-12-27 08:08:52,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000360928_92413952.pth [2023-12-27 08:08:52,894][20381] Updated weights for policy 1, policy_version 361851 (0.0009) [2023-12-27 08:08:52,962][20381] Updated weights for policy 1, policy_version 361861 (0.0005) [2023-12-27 08:08:53,021][20381] Updated weights for policy 1, policy_version 361871 (0.0005) [2023-12-27 08:08:53,105][20347] Updated weights for policy 0, policy_version 362177 (0.0007) [2023-12-27 08:08:53,165][20347] Updated weights for policy 0, policy_version 362187 (0.0007) [2023-12-27 08:08:53,223][20347] Updated weights for policy 0, policy_version 362197 (0.0007) [2023-12-27 08:08:53,275][20347] Updated weights for policy 0, policy_version 362207 (0.0005) [2023-12-27 08:08:53,551][20381] Updated weights for policy 1, policy_version 361881 (0.0006) [2023-12-27 08:08:53,611][20381] Updated weights for policy 1, policy_version 361891 (0.0010) [2023-12-27 08:08:53,663][20381] Updated weights for policy 1, policy_version 361901 (0.0010) [2023-12-27 08:08:53,713][20381] Updated weights for policy 1, policy_version 361911 (0.0010) [2023-12-27 08:08:53,866][20347] Updated weights for policy 0, policy_version 362217 (0.0005) [2023-12-27 08:08:53,920][20347] Updated weights for policy 0, policy_version 362227 (0.0005) [2023-12-27 08:08:53,968][20347] Updated weights for policy 0, policy_version 362237 (0.0007) [2023-12-27 08:08:54,337][20381] Updated weights for policy 1, policy_version 361921 (0.0006) [2023-12-27 08:08:54,393][20381] Updated weights for policy 1, policy_version 361931 (0.0006) [2023-12-27 08:08:54,449][20381] Updated weights for policy 1, policy_version 361941 (0.0006) [2023-12-27 08:08:54,631][20347] Updated weights for policy 0, policy_version 362247 (0.0007) [2023-12-27 08:08:54,679][20347] Updated weights for policy 0, policy_version 362257 (0.0008) [2023-12-27 08:08:54,729][20347] Updated weights for policy 0, policy_version 362267 (0.0007) [2023-12-27 08:08:55,038][20381] Updated weights for policy 1, policy_version 361951 (0.0008) [2023-12-27 08:08:55,100][20381] Updated weights for policy 1, policy_version 361961 (0.0008) [2023-12-27 08:08:55,159][20381] Updated weights for policy 1, policy_version 361971 (0.0008) [2023-12-27 08:08:55,404][20347] Updated weights for policy 0, policy_version 362277 (0.0007) [2023-12-27 08:08:55,460][20347] Updated weights for policy 0, policy_version 362287 (0.0007) [2023-12-27 08:08:55,518][20347] Updated weights for policy 0, policy_version 362297 (0.0007) [2023-12-27 08:08:55,860][20381] Updated weights for policy 1, policy_version 361981 (0.0007) [2023-12-27 08:08:55,917][20381] Updated weights for policy 1, policy_version 361991 (0.0007) [2023-12-27 08:08:55,962][20381] Updated weights for policy 1, policy_version 362001 (0.0009) [2023-12-27 08:08:56,125][20347] Updated weights for policy 0, policy_version 362307 (0.0007) [2023-12-27 08:08:56,180][20347] Updated weights for policy 0, policy_version 362317 (0.0005) [2023-12-27 08:08:56,230][20347] Updated weights for policy 0, policy_version 362327 (0.0005) [2023-12-27 08:08:56,589][20381] Updated weights for policy 1, policy_version 362011 (0.0008) [2023-12-27 08:08:56,643][20381] Updated weights for policy 1, policy_version 362021 (0.0007) [2023-12-27 08:08:56,707][20381] Updated weights for policy 1, policy_version 362031 (0.0008) [2023-12-27 08:08:56,880][20347] Updated weights for policy 0, policy_version 362337 (0.0007) [2023-12-27 08:08:56,936][20347] Updated weights for policy 0, policy_version 362347 (0.0007) [2023-12-27 08:08:56,996][20347] Updated weights for policy 0, policy_version 362357 (0.0007) [2023-12-27 08:08:57,046][20347] Updated weights for policy 0, policy_version 362367 (0.0008) [2023-12-27 08:08:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 185483264. Throughput: 0: 10527.8, 1: 10424.3. Samples: 185474028. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:08:57,363][19170] Avg episode reward: [(0, '396.340'), (1, '1000.000')] [2023-12-27 08:08:57,370][20381] Updated weights for policy 1, policy_version 362041 (0.0008) [2023-12-27 08:08:57,433][20381] Updated weights for policy 1, policy_version 362051 (0.0007) [2023-12-27 08:08:57,490][20381] Updated weights for policy 1, policy_version 362061 (0.0008) [2023-12-27 08:08:57,542][20381] Updated weights for policy 1, policy_version 362071 (0.0008) [2023-12-27 08:08:57,702][20347] Updated weights for policy 0, policy_version 362377 (0.0006) [2023-12-27 08:08:57,752][20347] Updated weights for policy 0, policy_version 362387 (0.0007) [2023-12-27 08:08:57,799][20347] Updated weights for policy 0, policy_version 362397 (0.0007) [2023-12-27 08:08:58,232][20381] Updated weights for policy 1, policy_version 362081 (0.0008) [2023-12-27 08:08:58,290][20381] Updated weights for policy 1, policy_version 362091 (0.0009) [2023-12-27 08:08:58,353][20381] Updated weights for policy 1, policy_version 362101 (0.0008) [2023-12-27 08:08:58,519][20347] Updated weights for policy 0, policy_version 362407 (0.0007) [2023-12-27 08:08:58,579][20347] Updated weights for policy 0, policy_version 362417 (0.0009) [2023-12-27 08:08:58,643][20347] Updated weights for policy 0, policy_version 362427 (0.0008) [2023-12-27 08:08:59,073][20381] Updated weights for policy 1, policy_version 362111 (0.0007) [2023-12-27 08:08:59,132][20381] Updated weights for policy 1, policy_version 362121 (0.0008) [2023-12-27 08:08:59,184][20381] Updated weights for policy 1, policy_version 362131 (0.0008) [2023-12-27 08:08:59,357][20347] Updated weights for policy 0, policy_version 362437 (0.0008) [2023-12-27 08:08:59,417][20347] Updated weights for policy 0, policy_version 362447 (0.0008) [2023-12-27 08:08:59,465][20347] Updated weights for policy 0, policy_version 362457 (0.0007) [2023-12-27 08:08:59,894][20381] Updated weights for policy 1, policy_version 362141 (0.0007) [2023-12-27 08:08:59,950][20381] Updated weights for policy 1, policy_version 362151 (0.0008) [2023-12-27 08:09:00,006][20381] Updated weights for policy 1, policy_version 362161 (0.0008) [2023-12-27 08:09:00,099][20347] Updated weights for policy 0, policy_version 362467 (0.0007) [2023-12-27 08:09:00,157][20347] Updated weights for policy 0, policy_version 362477 (0.0008) [2023-12-27 08:09:00,216][20347] Updated weights for policy 0, policy_version 362487 (0.0007) [2023-12-27 08:09:00,655][20381] Updated weights for policy 1, policy_version 362171 (0.0008) [2023-12-27 08:09:00,699][20381] Updated weights for policy 1, policy_version 362181 (0.0009) [2023-12-27 08:09:00,743][20381] Updated weights for policy 1, policy_version 362191 (0.0008) [2023-12-27 08:09:00,825][20347] Updated weights for policy 0, policy_version 362497 (0.0007) [2023-12-27 08:09:00,884][20347] Updated weights for policy 0, policy_version 362507 (0.0005) [2023-12-27 08:09:00,952][20347] Updated weights for policy 0, policy_version 362517 (0.0005) [2023-12-27 08:09:01,013][20347] Updated weights for policy 0, policy_version 362527 (0.0005) [2023-12-27 08:09:01,494][20381] Updated weights for policy 1, policy_version 362201 (0.0008) [2023-12-27 08:09:01,553][20381] Updated weights for policy 1, policy_version 362211 (0.0007) [2023-12-27 08:09:01,557][20347] Updated weights for policy 0, policy_version 362537 (0.0006) [2023-12-27 08:09:01,610][20381] Updated weights for policy 1, policy_version 362221 (0.0006) [2023-12-27 08:09:01,621][20347] Updated weights for policy 0, policy_version 362547 (0.0008) [2023-12-27 08:09:01,669][20381] Updated weights for policy 1, policy_version 362231 (0.0007) [2023-12-27 08:09:01,687][20347] Updated weights for policy 0, policy_version 362557 (0.0006) [2023-12-27 08:09:02,329][20381] Updated weights for policy 1, policy_version 362241 (0.0007) [2023-12-27 08:09:02,341][20347] Updated weights for policy 0, policy_version 362567 (0.0006) [2023-12-27 08:09:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185581568. Throughput: 0: 10532.7, 1: 10432.5. Samples: 185598556. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 08:09:02,363][19170] Avg episode reward: [(0, '528.360'), (1, '1000.000')] [2023-12-27 08:09:02,397][20381] Updated weights for policy 1, policy_version 362251 (0.0008) [2023-12-27 08:09:02,408][20347] Updated weights for policy 0, policy_version 362577 (0.0008) [2023-12-27 08:09:02,451][20381] Updated weights for policy 1, policy_version 362261 (0.0005) [2023-12-27 08:09:02,465][20347] Updated weights for policy 0, policy_version 362587 (0.0007) [2023-12-27 08:09:03,095][20347] Updated weights for policy 0, policy_version 362597 (0.0008) [2023-12-27 08:09:03,117][20381] Updated weights for policy 1, policy_version 362271 (0.0007) [2023-12-27 08:09:03,141][20347] Updated weights for policy 0, policy_version 362607 (0.0006) [2023-12-27 08:09:03,175][20381] Updated weights for policy 1, policy_version 362281 (0.0007) [2023-12-27 08:09:03,193][20347] Updated weights for policy 0, policy_version 362617 (0.0005) [2023-12-27 08:09:03,233][20381] Updated weights for policy 1, policy_version 362291 (0.0008) [2023-12-27 08:09:03,860][20347] Updated weights for policy 0, policy_version 362627 (0.0007) [2023-12-27 08:09:03,914][20381] Updated weights for policy 1, policy_version 362301 (0.0007) [2023-12-27 08:09:03,916][20347] Updated weights for policy 0, policy_version 362637 (0.0006) [2023-12-27 08:09:03,971][20381] Updated weights for policy 1, policy_version 362311 (0.0007) [2023-12-27 08:09:03,972][20347] Updated weights for policy 0, policy_version 362647 (0.0006) [2023-12-27 08:09:04,023][20381] Updated weights for policy 1, policy_version 362321 (0.0006) [2023-12-27 08:09:04,628][20347] Updated weights for policy 0, policy_version 362657 (0.0007) [2023-12-27 08:09:04,678][20347] Updated weights for policy 0, policy_version 362667 (0.0007) [2023-12-27 08:09:04,728][20347] Updated weights for policy 0, policy_version 362677 (0.0007) [2023-12-27 08:09:04,750][20381] Updated weights for policy 1, policy_version 362331 (0.0007) [2023-12-27 08:09:04,785][20347] Updated weights for policy 0, policy_version 362687 (0.0006) [2023-12-27 08:09:04,802][20381] Updated weights for policy 1, policy_version 362341 (0.0006) [2023-12-27 08:09:04,852][20381] Updated weights for policy 1, policy_version 362351 (0.0007) [2023-12-27 08:09:05,489][20347] Updated weights for policy 0, policy_version 362697 (0.0007) [2023-12-27 08:09:05,543][20347] Updated weights for policy 0, policy_version 362707 (0.0007) [2023-12-27 08:09:05,551][20381] Updated weights for policy 1, policy_version 362361 (0.0008) [2023-12-27 08:09:05,593][20347] Updated weights for policy 0, policy_version 362717 (0.0006) [2023-12-27 08:09:05,608][20381] Updated weights for policy 1, policy_version 362371 (0.0007) [2023-12-27 08:09:05,664][20381] Updated weights for policy 1, policy_version 362381 (0.0009) [2023-12-27 08:09:05,715][20381] Updated weights for policy 1, policy_version 362391 (0.0008) [2023-12-27 08:09:06,264][20347] Updated weights for policy 0, policy_version 362727 (0.0007) [2023-12-27 08:09:06,324][20347] Updated weights for policy 0, policy_version 362737 (0.0008) [2023-12-27 08:09:06,382][20347] Updated weights for policy 0, policy_version 362747 (0.0007) [2023-12-27 08:09:06,399][20381] Updated weights for policy 1, policy_version 362401 (0.0008) [2023-12-27 08:09:06,454][20381] Updated weights for policy 1, policy_version 362411 (0.0007) [2023-12-27 08:09:06,506][20381] Updated weights for policy 1, policy_version 362421 (0.0008) [2023-12-27 08:09:07,104][20347] Updated weights for policy 0, policy_version 362757 (0.0007) [2023-12-27 08:09:07,159][20347] Updated weights for policy 0, policy_version 362767 (0.0007) [2023-12-27 08:09:07,188][20381] Updated weights for policy 1, policy_version 362431 (0.0010) [2023-12-27 08:09:07,209][20347] Updated weights for policy 0, policy_version 362777 (0.0005) [2023-12-27 08:09:07,240][20381] Updated weights for policy 1, policy_version 362441 (0.0010) [2023-12-27 08:09:07,291][20381] Updated weights for policy 1, policy_version 362451 (0.0010) [2023-12-27 08:09:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185696256. Throughput: 0: 10564.8, 1: 10439.5. Samples: 185659992. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:07,363][19170] Avg episode reward: [(0, '728.270'), (1, '1000.000')] [2023-12-27 08:09:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000362784_92889088.pth... [2023-12-27 08:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000362456_92807168.pth... [2023-12-27 08:09:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000361536_92569600.pth [2023-12-27 08:09:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000361240_92495872.pth [2023-12-27 08:09:07,864][20347] Updated weights for policy 0, policy_version 362787 (0.0007) [2023-12-27 08:09:07,927][20347] Updated weights for policy 0, policy_version 362797 (0.0006) [2023-12-27 08:09:07,968][20381] Updated weights for policy 1, policy_version 362461 (0.0008) [2023-12-27 08:09:07,991][20347] Updated weights for policy 0, policy_version 362807 (0.0007) [2023-12-27 08:09:08,025][20381] Updated weights for policy 1, policy_version 362471 (0.0007) [2023-12-27 08:09:08,083][20381] Updated weights for policy 1, policy_version 362481 (0.0008) [2023-12-27 08:09:08,696][20381] Updated weights for policy 1, policy_version 362491 (0.0010) [2023-12-27 08:09:08,727][20347] Updated weights for policy 0, policy_version 362817 (0.0006) [2023-12-27 08:09:08,759][20381] Updated weights for policy 1, policy_version 362501 (0.0011) [2023-12-27 08:09:08,791][20347] Updated weights for policy 0, policy_version 362827 (0.0007) [2023-12-27 08:09:08,822][20381] Updated weights for policy 1, policy_version 362511 (0.0010) [2023-12-27 08:09:08,850][20347] Updated weights for policy 0, policy_version 362837 (0.0005) [2023-12-27 08:09:08,900][20347] Updated weights for policy 0, policy_version 362847 (0.0006) [2023-12-27 08:09:09,521][20381] Updated weights for policy 1, policy_version 362521 (0.0010) [2023-12-27 08:09:09,581][20381] Updated weights for policy 1, policy_version 362531 (0.0011) [2023-12-27 08:09:09,605][20347] Updated weights for policy 0, policy_version 362857 (0.0005) [2023-12-27 08:09:09,640][20381] Updated weights for policy 1, policy_version 362541 (0.0011) [2023-12-27 08:09:09,658][20347] Updated weights for policy 0, policy_version 362867 (0.0008) [2023-12-27 08:09:09,699][20381] Updated weights for policy 1, policy_version 362551 (0.0010) [2023-12-27 08:09:09,712][20347] Updated weights for policy 0, policy_version 362877 (0.0007) [2023-12-27 08:09:10,392][20347] Updated weights for policy 0, policy_version 362887 (0.0006) [2023-12-27 08:09:10,409][20381] Updated weights for policy 1, policy_version 362561 (0.0006) [2023-12-27 08:09:10,448][20347] Updated weights for policy 0, policy_version 362897 (0.0006) [2023-12-27 08:09:10,465][20381] Updated weights for policy 1, policy_version 362571 (0.0006) [2023-12-27 08:09:10,500][20347] Updated weights for policy 0, policy_version 362907 (0.0006) [2023-12-27 08:09:10,517][20381] Updated weights for policy 1, policy_version 362581 (0.0005) [2023-12-27 08:09:11,110][20347] Updated weights for policy 0, policy_version 362917 (0.0008) [2023-12-27 08:09:11,180][20347] Updated weights for policy 0, policy_version 362927 (0.0010) [2023-12-27 08:09:11,193][20381] Updated weights for policy 1, policy_version 362591 (0.0010) [2023-12-27 08:09:11,240][20347] Updated weights for policy 0, policy_version 362937 (0.0010) [2023-12-27 08:09:11,253][20381] Updated weights for policy 1, policy_version 362601 (0.0011) [2023-12-27 08:09:11,310][20381] Updated weights for policy 1, policy_version 362611 (0.0011) [2023-12-27 08:09:11,891][20347] Updated weights for policy 0, policy_version 362947 (0.0009) [2023-12-27 08:09:11,947][20347] Updated weights for policy 0, policy_version 362957 (0.0009) [2023-12-27 08:09:12,011][20347] Updated weights for policy 0, policy_version 362967 (0.0007) [2023-12-27 08:09:12,020][20381] Updated weights for policy 1, policy_version 362621 (0.0008) [2023-12-27 08:09:12,078][20381] Updated weights for policy 1, policy_version 362631 (0.0006) [2023-12-27 08:09:12,130][20381] Updated weights for policy 1, policy_version 362641 (0.0009) [2023-12-27 08:09:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185794560. Throughput: 0: 10461.1, 1: 10433.6. Samples: 185782156. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:12,363][19170] Avg episode reward: [(0, '836.760'), (1, '1000.000')] [2023-12-27 08:09:12,676][20347] Updated weights for policy 0, policy_version 362977 (0.0010) [2023-12-27 08:09:12,735][20347] Updated weights for policy 0, policy_version 362987 (0.0011) [2023-12-27 08:09:12,762][20381] Updated weights for policy 1, policy_version 362651 (0.0007) [2023-12-27 08:09:12,794][20347] Updated weights for policy 0, policy_version 362997 (0.0008) [2023-12-27 08:09:12,821][20381] Updated weights for policy 1, policy_version 362661 (0.0007) [2023-12-27 08:09:12,853][20347] Updated weights for policy 0, policy_version 363007 (0.0008) [2023-12-27 08:09:12,881][20381] Updated weights for policy 1, policy_version 362671 (0.0006) [2023-12-27 08:09:13,494][20347] Updated weights for policy 0, policy_version 363017 (0.0010) [2023-12-27 08:09:13,502][20381] Updated weights for policy 1, policy_version 362681 (0.0008) [2023-12-27 08:09:13,548][20347] Updated weights for policy 0, policy_version 363027 (0.0010) [2023-12-27 08:09:13,557][20381] Updated weights for policy 1, policy_version 362691 (0.0005) [2023-12-27 08:09:13,607][20381] Updated weights for policy 1, policy_version 362701 (0.0006) [2023-12-27 08:09:13,608][20347] Updated weights for policy 0, policy_version 363037 (0.0010) [2023-12-27 08:09:13,653][20381] Updated weights for policy 1, policy_version 362711 (0.0007) [2023-12-27 08:09:14,278][20347] Updated weights for policy 0, policy_version 363047 (0.0010) [2023-12-27 08:09:14,316][20381] Updated weights for policy 1, policy_version 362721 (0.0006) [2023-12-27 08:09:14,330][20347] Updated weights for policy 0, policy_version 363057 (0.0010) [2023-12-27 08:09:14,369][20381] Updated weights for policy 1, policy_version 362731 (0.0006) [2023-12-27 08:09:14,382][20347] Updated weights for policy 0, policy_version 363067 (0.0008) [2023-12-27 08:09:14,422][20381] Updated weights for policy 1, policy_version 362741 (0.0008) [2023-12-27 08:09:15,011][20381] Updated weights for policy 1, policy_version 362751 (0.0007) [2023-12-27 08:09:15,067][20381] Updated weights for policy 1, policy_version 362761 (0.0005) [2023-12-27 08:09:15,070][20347] Updated weights for policy 0, policy_version 363077 (0.0010) [2023-12-27 08:09:15,114][20381] Updated weights for policy 1, policy_version 362771 (0.0005) [2023-12-27 08:09:15,129][20347] Updated weights for policy 0, policy_version 363087 (0.0010) [2023-12-27 08:09:15,184][20347] Updated weights for policy 0, policy_version 363097 (0.0010) [2023-12-27 08:09:15,752][20381] Updated weights for policy 1, policy_version 362781 (0.0007) [2023-12-27 08:09:15,804][20381] Updated weights for policy 1, policy_version 362791 (0.0005) [2023-12-27 08:09:15,855][20381] Updated weights for policy 1, policy_version 362801 (0.0005) [2023-12-27 08:09:15,875][20347] Updated weights for policy 0, policy_version 363107 (0.0010) [2023-12-27 08:09:15,930][20347] Updated weights for policy 0, policy_version 363117 (0.0007) [2023-12-27 08:09:15,988][20347] Updated weights for policy 0, policy_version 363127 (0.0007) [2023-12-27 08:09:16,397][20381] Updated weights for policy 1, policy_version 362811 (0.0006) [2023-12-27 08:09:16,445][20381] Updated weights for policy 1, policy_version 362821 (0.0006) [2023-12-27 08:09:16,503][20381] Updated weights for policy 1, policy_version 362831 (0.0008) [2023-12-27 08:09:16,673][20347] Updated weights for policy 0, policy_version 363137 (0.0006) [2023-12-27 08:09:16,726][20347] Updated weights for policy 0, policy_version 363147 (0.0005) [2023-12-27 08:09:16,774][20347] Updated weights for policy 0, policy_version 363157 (0.0007) [2023-12-27 08:09:16,823][20347] Updated weights for policy 0, policy_version 363167 (0.0007) [2023-12-27 08:09:17,106][20381] Updated weights for policy 1, policy_version 362841 (0.0010) [2023-12-27 08:09:17,157][20381] Updated weights for policy 1, policy_version 362851 (0.0010) [2023-12-27 08:09:17,202][20381] Updated weights for policy 1, policy_version 362861 (0.0010) [2023-12-27 08:09:17,249][20381] Updated weights for policy 1, policy_version 362871 (0.0008) [2023-12-27 08:09:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 185901056. Throughput: 0: 10442.8, 1: 10425.3. Samples: 185910064. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:17,363][19170] Avg episode reward: [(0, '845.870'), (1, '1000.000')] [2023-12-27 08:09:17,460][20347] Updated weights for policy 0, policy_version 363177 (0.0008) [2023-12-27 08:09:17,507][20347] Updated weights for policy 0, policy_version 363187 (0.0007) [2023-12-27 08:09:17,556][20347] Updated weights for policy 0, policy_version 363197 (0.0006) [2023-12-27 08:09:17,855][20381] Updated weights for policy 1, policy_version 362881 (0.0008) [2023-12-27 08:09:17,910][20381] Updated weights for policy 1, policy_version 362891 (0.0007) [2023-12-27 08:09:17,972][20381] Updated weights for policy 1, policy_version 362901 (0.0009) [2023-12-27 08:09:18,263][20347] Updated weights for policy 0, policy_version 363207 (0.0005) [2023-12-27 08:09:18,323][20347] Updated weights for policy 0, policy_version 363217 (0.0006) [2023-12-27 08:09:18,384][20347] Updated weights for policy 0, policy_version 363227 (0.0006) [2023-12-27 08:09:18,551][20381] Updated weights for policy 1, policy_version 362911 (0.0010) [2023-12-27 08:09:18,604][20381] Updated weights for policy 1, policy_version 362921 (0.0011) [2023-12-27 08:09:18,653][20381] Updated weights for policy 1, policy_version 362931 (0.0011) [2023-12-27 08:09:19,007][20347] Updated weights for policy 0, policy_version 363237 (0.0006) [2023-12-27 08:09:19,051][20347] Updated weights for policy 0, policy_version 363247 (0.0007) [2023-12-27 08:09:19,098][20347] Updated weights for policy 0, policy_version 363257 (0.0007) [2023-12-27 08:09:19,307][20381] Updated weights for policy 1, policy_version 362941 (0.0011) [2023-12-27 08:09:19,368][20381] Updated weights for policy 1, policy_version 362951 (0.0010) [2023-12-27 08:09:19,424][20381] Updated weights for policy 1, policy_version 362961 (0.0011) [2023-12-27 08:09:19,783][20347] Updated weights for policy 0, policy_version 363267 (0.0007) [2023-12-27 08:09:19,840][20347] Updated weights for policy 0, policy_version 363277 (0.0007) [2023-12-27 08:09:19,898][20347] Updated weights for policy 0, policy_version 363287 (0.0009) [2023-12-27 08:09:20,174][20381] Updated weights for policy 1, policy_version 362971 (0.0010) [2023-12-27 08:09:20,225][20381] Updated weights for policy 1, policy_version 362981 (0.0009) [2023-12-27 08:09:20,278][20381] Updated weights for policy 1, policy_version 362991 (0.0009) [2023-12-27 08:09:20,729][20347] Updated weights for policy 0, policy_version 363297 (0.0009) [2023-12-27 08:09:20,787][20347] Updated weights for policy 0, policy_version 363307 (0.0008) [2023-12-27 08:09:20,860][20347] Updated weights for policy 0, policy_version 363317 (0.0008) [2023-12-27 08:09:20,924][20347] Updated weights for policy 0, policy_version 363327 (0.0007) [2023-12-27 08:09:21,036][20381] Updated weights for policy 1, policy_version 363001 (0.0009) [2023-12-27 08:09:21,093][20381] Updated weights for policy 1, policy_version 363011 (0.0008) [2023-12-27 08:09:21,145][20381] Updated weights for policy 1, policy_version 363021 (0.0008) [2023-12-27 08:09:21,204][20381] Updated weights for policy 1, policy_version 363031 (0.0008) [2023-12-27 08:09:21,595][20347] Updated weights for policy 0, policy_version 363337 (0.0007) [2023-12-27 08:09:21,652][20347] Updated weights for policy 0, policy_version 363347 (0.0007) [2023-12-27 08:09:21,709][20347] Updated weights for policy 0, policy_version 363357 (0.0007) [2023-12-27 08:09:21,958][20381] Updated weights for policy 1, policy_version 363041 (0.0008) [2023-12-27 08:09:22,023][20381] Updated weights for policy 1, policy_version 363051 (0.0009) [2023-12-27 08:09:22,088][20381] Updated weights for policy 1, policy_version 363061 (0.0008) [2023-12-27 08:09:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 185999360. Throughput: 0: 10438.2, 1: 10382.8. Samples: 185972988. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:22,363][19170] Avg episode reward: [(0, '875.140'), (1, '1000.000')] [2023-12-27 08:09:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000363064_92962816.pth... [2023-12-27 08:09:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000361848_92651520.pth [2023-12-27 08:09:22,396][20347] Updated weights for policy 0, policy_version 363367 (0.0007) [2023-12-27 08:09:22,454][20347] Updated weights for policy 0, policy_version 363377 (0.0008) [2023-12-27 08:09:22,511][20347] Updated weights for policy 0, policy_version 363387 (0.0007) [2023-12-27 08:09:22,540][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000363392_93044736.pth... [2023-12-27 08:09:22,544][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000362176_92733440.pth [2023-12-27 08:09:22,804][20381] Updated weights for policy 1, policy_version 363071 (0.0009) [2023-12-27 08:09:22,858][20381] Updated weights for policy 1, policy_version 363081 (0.0008) [2023-12-27 08:09:22,904][20381] Updated weights for policy 1, policy_version 363091 (0.0008) [2023-12-27 08:09:23,201][20347] Updated weights for policy 0, policy_version 363397 (0.0007) [2023-12-27 08:09:23,252][20347] Updated weights for policy 0, policy_version 363407 (0.0006) [2023-12-27 08:09:23,308][20347] Updated weights for policy 0, policy_version 363417 (0.0005) [2023-12-27 08:09:23,577][20381] Updated weights for policy 1, policy_version 363101 (0.0007) [2023-12-27 08:09:23,627][20381] Updated weights for policy 1, policy_version 363111 (0.0005) [2023-12-27 08:09:23,681][20381] Updated weights for policy 1, policy_version 363121 (0.0005) [2023-12-27 08:09:23,918][20347] Updated weights for policy 0, policy_version 363427 (0.0005) [2023-12-27 08:09:23,967][20347] Updated weights for policy 0, policy_version 363437 (0.0005) [2023-12-27 08:09:24,027][20347] Updated weights for policy 0, policy_version 363447 (0.0006) [2023-12-27 08:09:24,291][20381] Updated weights for policy 1, policy_version 363131 (0.0006) [2023-12-27 08:09:24,340][20381] Updated weights for policy 1, policy_version 363141 (0.0007) [2023-12-27 08:09:24,390][20381] Updated weights for policy 1, policy_version 363151 (0.0008) [2023-12-27 08:09:24,636][20347] Updated weights for policy 0, policy_version 363457 (0.0007) [2023-12-27 08:09:24,682][20347] Updated weights for policy 0, policy_version 363467 (0.0005) [2023-12-27 08:09:24,727][20347] Updated weights for policy 0, policy_version 363477 (0.0005) [2023-12-27 08:09:24,780][20347] Updated weights for policy 0, policy_version 363487 (0.0005) [2023-12-27 08:09:25,147][20381] Updated weights for policy 1, policy_version 363161 (0.0008) [2023-12-27 08:09:25,215][20381] Updated weights for policy 1, policy_version 363171 (0.0009) [2023-12-27 08:09:25,281][20381] Updated weights for policy 1, policy_version 363181 (0.0008) [2023-12-27 08:09:25,346][20381] Updated weights for policy 1, policy_version 363191 (0.0008) [2023-12-27 08:09:25,374][20347] Updated weights for policy 0, policy_version 363497 (0.0006) [2023-12-27 08:09:25,430][20347] Updated weights for policy 0, policy_version 363507 (0.0005) [2023-12-27 08:09:25,492][20347] Updated weights for policy 0, policy_version 363517 (0.0006) [2023-12-27 08:09:26,049][20381] Updated weights for policy 1, policy_version 363201 (0.0007) [2023-12-27 08:09:26,109][20381] Updated weights for policy 1, policy_version 363211 (0.0008) [2023-12-27 08:09:26,122][20347] Updated weights for policy 0, policy_version 363527 (0.0006) [2023-12-27 08:09:26,167][20381] Updated weights for policy 1, policy_version 363221 (0.0008) [2023-12-27 08:09:26,178][20347] Updated weights for policy 0, policy_version 363537 (0.0005) [2023-12-27 08:09:26,224][20347] Updated weights for policy 0, policy_version 363547 (0.0007) [2023-12-27 08:09:26,786][20381] Updated weights for policy 1, policy_version 363231 (0.0006) [2023-12-27 08:09:26,849][20381] Updated weights for policy 1, policy_version 363241 (0.0005) [2023-12-27 08:09:26,876][20347] Updated weights for policy 0, policy_version 363557 (0.0006) [2023-12-27 08:09:26,912][20381] Updated weights for policy 1, policy_version 363251 (0.0008) [2023-12-27 08:09:26,930][20347] Updated weights for policy 0, policy_version 363567 (0.0005) [2023-12-27 08:09:26,985][20347] Updated weights for policy 0, policy_version 363577 (0.0006) [2023-12-27 08:09:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 186105856. Throughput: 0: 10467.7, 1: 10351.8. Samples: 186095212. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:27,363][19170] Avg episode reward: [(0, '923.410'), (1, '1000.000')] [2023-12-27 08:09:27,485][20381] Updated weights for policy 1, policy_version 363261 (0.0006) [2023-12-27 08:09:27,552][20381] Updated weights for policy 1, policy_version 363271 (0.0007) [2023-12-27 08:09:27,583][20347] Updated weights for policy 0, policy_version 363587 (0.0006) [2023-12-27 08:09:27,615][20381] Updated weights for policy 1, policy_version 363281 (0.0010) [2023-12-27 08:09:27,640][20347] Updated weights for policy 0, policy_version 363597 (0.0007) [2023-12-27 08:09:27,688][20347] Updated weights for policy 0, policy_version 363607 (0.0008) [2023-12-27 08:09:28,218][20381] Updated weights for policy 1, policy_version 363291 (0.0009) [2023-12-27 08:09:28,267][20381] Updated weights for policy 1, policy_version 363301 (0.0005) [2023-12-27 08:09:28,325][20381] Updated weights for policy 1, policy_version 363311 (0.0005) [2023-12-27 08:09:28,388][20347] Updated weights for policy 0, policy_version 363617 (0.0007) [2023-12-27 08:09:28,443][20347] Updated weights for policy 0, policy_version 363627 (0.0010) [2023-12-27 08:09:28,497][20347] Updated weights for policy 0, policy_version 363637 (0.0006) [2023-12-27 08:09:28,560][20347] Updated weights for policy 0, policy_version 363647 (0.0005) [2023-12-27 08:09:29,002][20381] Updated weights for policy 1, policy_version 363321 (0.0010) [2023-12-27 08:09:29,052][20381] Updated weights for policy 1, policy_version 363331 (0.0007) [2023-12-27 08:09:29,096][20381] Updated weights for policy 1, policy_version 363341 (0.0007) [2023-12-27 08:09:29,140][20381] Updated weights for policy 1, policy_version 363351 (0.0007) [2023-12-27 08:09:29,199][20347] Updated weights for policy 0, policy_version 363657 (0.0010) [2023-12-27 08:09:29,261][20347] Updated weights for policy 0, policy_version 363667 (0.0010) [2023-12-27 08:09:29,310][20347] Updated weights for policy 0, policy_version 363677 (0.0010) [2023-12-27 08:09:29,890][20381] Updated weights for policy 1, policy_version 363361 (0.0010) [2023-12-27 08:09:29,952][20381] Updated weights for policy 1, policy_version 363371 (0.0011) [2023-12-27 08:09:30,006][20347] Updated weights for policy 0, policy_version 363687 (0.0008) [2023-12-27 08:09:30,019][20381] Updated weights for policy 1, policy_version 363381 (0.0008) [2023-12-27 08:09:30,063][20347] Updated weights for policy 0, policy_version 363697 (0.0006) [2023-12-27 08:09:30,121][20347] Updated weights for policy 0, policy_version 363707 (0.0010) [2023-12-27 08:09:30,637][20381] Updated weights for policy 1, policy_version 363391 (0.0007) [2023-12-27 08:09:30,699][20381] Updated weights for policy 1, policy_version 363401 (0.0006) [2023-12-27 08:09:30,759][20381] Updated weights for policy 1, policy_version 363411 (0.0006) [2023-12-27 08:09:30,781][20347] Updated weights for policy 0, policy_version 363717 (0.0009) [2023-12-27 08:09:30,826][20347] Updated weights for policy 0, policy_version 363727 (0.0005) [2023-12-27 08:09:30,882][20347] Updated weights for policy 0, policy_version 363737 (0.0005) [2023-12-27 08:09:31,354][20381] Updated weights for policy 1, policy_version 363421 (0.0006) [2023-12-27 08:09:31,414][20381] Updated weights for policy 1, policy_version 363431 (0.0010) [2023-12-27 08:09:31,459][20381] Updated weights for policy 1, policy_version 363441 (0.0007) [2023-12-27 08:09:31,510][20347] Updated weights for policy 0, policy_version 363747 (0.0007) [2023-12-27 08:09:31,567][20347] Updated weights for policy 0, policy_version 363757 (0.0005) [2023-12-27 08:09:31,627][20347] Updated weights for policy 0, policy_version 363767 (0.0006) [2023-12-27 08:09:32,109][20381] Updated weights for policy 1, policy_version 363451 (0.0007) [2023-12-27 08:09:32,163][20347] Updated weights for policy 0, policy_version 363777 (0.0006) [2023-12-27 08:09:32,166][20381] Updated weights for policy 1, policy_version 363461 (0.0005) [2023-12-27 08:09:32,223][20347] Updated weights for policy 0, policy_version 363787 (0.0005) [2023-12-27 08:09:32,229][20381] Updated weights for policy 1, policy_version 363471 (0.0005) [2023-12-27 08:09:32,286][20347] Updated weights for policy 0, policy_version 363797 (0.0006) [2023-12-27 08:09:32,340][20347] Updated weights for policy 0, policy_version 363807 (0.0008) [2023-12-27 08:09:32,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 186220544. Throughput: 0: 10490.6, 1: 10383.8. Samples: 186224308. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:32,363][19170] Avg episode reward: [(0, '923.110'), (1, '1000.000')] [2023-12-27 08:09:32,832][20381] Updated weights for policy 1, policy_version 363481 (0.0010) [2023-12-27 08:09:32,898][20381] Updated weights for policy 1, policy_version 363491 (0.0007) [2023-12-27 08:09:32,957][20381] Updated weights for policy 1, policy_version 363501 (0.0010) [2023-12-27 08:09:33,007][20381] Updated weights for policy 1, policy_version 363511 (0.0010) [2023-12-27 08:09:33,020][20347] Updated weights for policy 0, policy_version 363817 (0.0006) [2023-12-27 08:09:33,066][20347] Updated weights for policy 0, policy_version 363827 (0.0005) [2023-12-27 08:09:33,121][20347] Updated weights for policy 0, policy_version 363837 (0.0007) [2023-12-27 08:09:33,678][20381] Updated weights for policy 1, policy_version 363521 (0.0008) [2023-12-27 08:09:33,734][20381] Updated weights for policy 1, policy_version 363531 (0.0009) [2023-12-27 08:09:33,782][20347] Updated weights for policy 0, policy_version 363847 (0.0007) [2023-12-27 08:09:33,782][20381] Updated weights for policy 1, policy_version 363541 (0.0008) [2023-12-27 08:09:33,842][20347] Updated weights for policy 0, policy_version 363857 (0.0009) [2023-12-27 08:09:33,898][20347] Updated weights for policy 0, policy_version 363867 (0.0005) [2023-12-27 08:09:34,449][20381] Updated weights for policy 1, policy_version 363551 (0.0006) [2023-12-27 08:09:34,473][20347] Updated weights for policy 0, policy_version 363877 (0.0008) [2023-12-27 08:09:34,511][20381] Updated weights for policy 1, policy_version 363561 (0.0007) [2023-12-27 08:09:34,532][20347] Updated weights for policy 0, policy_version 363887 (0.0010) [2023-12-27 08:09:34,569][20381] Updated weights for policy 1, policy_version 363571 (0.0007) [2023-12-27 08:09:34,594][20347] Updated weights for policy 0, policy_version 363897 (0.0011) [2023-12-27 08:09:35,175][20381] Updated weights for policy 1, policy_version 363581 (0.0007) [2023-12-27 08:09:35,234][20381] Updated weights for policy 1, policy_version 363591 (0.0007) [2023-12-27 08:09:35,239][20347] Updated weights for policy 0, policy_version 363907 (0.0011) [2023-12-27 08:09:35,295][20381] Updated weights for policy 1, policy_version 363601 (0.0005) [2023-12-27 08:09:35,296][20347] Updated weights for policy 0, policy_version 363917 (0.0009) [2023-12-27 08:09:35,346][20347] Updated weights for policy 0, policy_version 363927 (0.0005) [2023-12-27 08:09:35,960][20381] Updated weights for policy 1, policy_version 363611 (0.0006) [2023-12-27 08:09:36,015][20381] Updated weights for policy 1, policy_version 363621 (0.0007) [2023-12-27 08:09:36,019][20347] Updated weights for policy 0, policy_version 363937 (0.0008) [2023-12-27 08:09:36,069][20381] Updated weights for policy 1, policy_version 363631 (0.0005) [2023-12-27 08:09:36,078][20347] Updated weights for policy 0, policy_version 363947 (0.0011) [2023-12-27 08:09:36,142][20347] Updated weights for policy 0, policy_version 363957 (0.0011) [2023-12-27 08:09:36,210][20347] Updated weights for policy 0, policy_version 363967 (0.0011) [2023-12-27 08:09:36,653][20381] Updated weights for policy 1, policy_version 363641 (0.0007) [2023-12-27 08:09:36,705][20381] Updated weights for policy 1, policy_version 363651 (0.0008) [2023-12-27 08:09:36,754][20381] Updated weights for policy 1, policy_version 363661 (0.0007) [2023-12-27 08:09:36,803][20381] Updated weights for policy 1, policy_version 363671 (0.0007) [2023-12-27 08:09:36,886][20347] Updated weights for policy 0, policy_version 363977 (0.0006) [2023-12-27 08:09:36,936][20347] Updated weights for policy 0, policy_version 363987 (0.0009) [2023-12-27 08:09:36,985][20347] Updated weights for policy 0, policy_version 363997 (0.0009) [2023-12-27 08:09:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 186318848. Throughput: 0: 10488.5, 1: 10443.5. Samples: 186288576. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:37,364][19170] Avg episode reward: [(0, '942.900'), (1, '1000.000')] [2023-12-27 08:09:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000363672_93118464.pth... [2023-12-27 08:09:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000364000_93200384.pth... [2023-12-27 08:09:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000362456_92807168.pth [2023-12-27 08:09:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000362784_92889088.pth [2023-12-27 08:09:37,527][20381] Updated weights for policy 1, policy_version 363681 (0.0007) [2023-12-27 08:09:37,576][20381] Updated weights for policy 1, policy_version 363691 (0.0008) [2023-12-27 08:09:37,621][20347] Updated weights for policy 0, policy_version 364007 (0.0006) [2023-12-27 08:09:37,633][20381] Updated weights for policy 1, policy_version 363701 (0.0007) [2023-12-27 08:09:37,678][20347] Updated weights for policy 0, policy_version 364017 (0.0007) [2023-12-27 08:09:37,737][20347] Updated weights for policy 0, policy_version 364027 (0.0007) [2023-12-27 08:09:38,369][20347] Updated weights for policy 0, policy_version 364037 (0.0007) [2023-12-27 08:09:38,371][20381] Updated weights for policy 1, policy_version 363711 (0.0006) [2023-12-27 08:09:38,422][20347] Updated weights for policy 0, policy_version 364047 (0.0008) [2023-12-27 08:09:38,425][20381] Updated weights for policy 1, policy_version 363721 (0.0006) [2023-12-27 08:09:38,476][20381] Updated weights for policy 1, policy_version 363731 (0.0006) [2023-12-27 08:09:38,478][20347] Updated weights for policy 0, policy_version 364057 (0.0008) [2023-12-27 08:09:39,191][20347] Updated weights for policy 0, policy_version 364067 (0.0007) [2023-12-27 08:09:39,193][20381] Updated weights for policy 1, policy_version 363741 (0.0007) [2023-12-27 08:09:39,253][20381] Updated weights for policy 1, policy_version 363751 (0.0007) [2023-12-27 08:09:39,254][20347] Updated weights for policy 0, policy_version 364077 (0.0006) [2023-12-27 08:09:39,310][20347] Updated weights for policy 0, policy_version 364087 (0.0007) [2023-12-27 08:09:39,319][20381] Updated weights for policy 1, policy_version 363761 (0.0007) [2023-12-27 08:09:39,963][20381] Updated weights for policy 1, policy_version 363771 (0.0008) [2023-12-27 08:09:40,020][20381] Updated weights for policy 1, policy_version 363781 (0.0006) [2023-12-27 08:09:40,080][20381] Updated weights for policy 1, policy_version 363791 (0.0006) [2023-12-27 08:09:40,096][20347] Updated weights for policy 0, policy_version 364097 (0.0008) [2023-12-27 08:09:40,152][20347] Updated weights for policy 0, policy_version 364107 (0.0007) [2023-12-27 08:09:40,208][20347] Updated weights for policy 0, policy_version 364117 (0.0008) [2023-12-27 08:09:40,267][20347] Updated weights for policy 0, policy_version 364127 (0.0007) [2023-12-27 08:09:40,723][20381] Updated weights for policy 1, policy_version 363801 (0.0007) [2023-12-27 08:09:40,785][20381] Updated weights for policy 1, policy_version 363811 (0.0007) [2023-12-27 08:09:40,841][20381] Updated weights for policy 1, policy_version 363821 (0.0011) [2023-12-27 08:09:40,899][20381] Updated weights for policy 1, policy_version 363831 (0.0010) [2023-12-27 08:09:40,905][20347] Updated weights for policy 0, policy_version 364137 (0.0006) [2023-12-27 08:09:40,962][20347] Updated weights for policy 0, policy_version 364147 (0.0010) [2023-12-27 08:09:41,024][20347] Updated weights for policy 0, policy_version 364157 (0.0008) [2023-12-27 08:09:41,631][20381] Updated weights for policy 1, policy_version 363841 (0.0009) [2023-12-27 08:09:41,692][20381] Updated weights for policy 1, policy_version 363851 (0.0009) [2023-12-27 08:09:41,698][20347] Updated weights for policy 0, policy_version 364167 (0.0010) [2023-12-27 08:09:41,756][20347] Updated weights for policy 0, policy_version 364177 (0.0011) [2023-12-27 08:09:41,757][20381] Updated weights for policy 1, policy_version 363861 (0.0009) [2023-12-27 08:09:41,818][20347] Updated weights for policy 0, policy_version 364187 (0.0011) [2023-12-27 08:09:42,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 186417152. Throughput: 0: 10434.7, 1: 10394.5. Samples: 186411340. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:42,363][19170] Avg episode reward: [(0, '961.730'), (1, '990.360')] [2023-12-27 08:09:42,479][20381] Updated weights for policy 1, policy_version 363871 (0.0010) [2023-12-27 08:09:42,503][20347] Updated weights for policy 0, policy_version 364197 (0.0010) [2023-12-27 08:09:42,505][20292] KL-divergence is very high: 731.2038 [2023-12-27 08:09:42,539][20381] Updated weights for policy 1, policy_version 363881 (0.0007) [2023-12-27 08:09:42,564][20347] Updated weights for policy 0, policy_version 364207 (0.0007) [2023-12-27 08:09:42,594][20381] Updated weights for policy 1, policy_version 363891 (0.0006) [2023-12-27 08:09:42,618][20347] Updated weights for policy 0, policy_version 364217 (0.0007) [2023-12-27 08:09:43,220][20381] Updated weights for policy 1, policy_version 363901 (0.0009) [2023-12-27 08:09:43,272][20381] Updated weights for policy 1, policy_version 363911 (0.0010) [2023-12-27 08:09:43,294][20347] Updated weights for policy 0, policy_version 364227 (0.0008) [2023-12-27 08:09:43,327][20381] Updated weights for policy 1, policy_version 363921 (0.0011) [2023-12-27 08:09:43,353][20347] Updated weights for policy 0, policy_version 364237 (0.0010) [2023-12-27 08:09:43,403][20347] Updated weights for policy 0, policy_version 364247 (0.0010) [2023-12-27 08:09:43,960][20381] Updated weights for policy 1, policy_version 363931 (0.0010) [2023-12-27 08:09:44,015][20381] Updated weights for policy 1, policy_version 363941 (0.0010) [2023-12-27 08:09:44,042][20347] Updated weights for policy 0, policy_version 364257 (0.0006) [2023-12-27 08:09:44,072][20381] Updated weights for policy 1, policy_version 363951 (0.0010) [2023-12-27 08:09:44,092][20347] Updated weights for policy 0, policy_version 364267 (0.0010) [2023-12-27 08:09:44,143][20347] Updated weights for policy 0, policy_version 364277 (0.0010) [2023-12-27 08:09:44,192][20347] Updated weights for policy 0, policy_version 364287 (0.0010) [2023-12-27 08:09:44,757][20381] Updated weights for policy 1, policy_version 363961 (0.0010) [2023-12-27 08:09:44,819][20381] Updated weights for policy 1, policy_version 363971 (0.0009) [2023-12-27 08:09:44,822][20347] Updated weights for policy 0, policy_version 364297 (0.0010) [2023-12-27 08:09:44,876][20381] Updated weights for policy 1, policy_version 363981 (0.0006) [2023-12-27 08:09:44,885][20347] Updated weights for policy 0, policy_version 364307 (0.0010) [2023-12-27 08:09:44,936][20381] Updated weights for policy 1, policy_version 363991 (0.0007) [2023-12-27 08:09:44,944][20347] Updated weights for policy 0, policy_version 364317 (0.0009) [2023-12-27 08:09:45,542][20347] Updated weights for policy 0, policy_version 364327 (0.0006) [2023-12-27 08:09:45,554][20381] Updated weights for policy 1, policy_version 364001 (0.0006) [2023-12-27 08:09:45,603][20347] Updated weights for policy 0, policy_version 364337 (0.0007) [2023-12-27 08:09:45,608][20381] Updated weights for policy 1, policy_version 364011 (0.0005) [2023-12-27 08:09:45,663][20381] Updated weights for policy 1, policy_version 364021 (0.0005) [2023-12-27 08:09:45,665][20347] Updated weights for policy 0, policy_version 364347 (0.0007) [2023-12-27 08:09:46,273][20381] Updated weights for policy 1, policy_version 364031 (0.0007) [2023-12-27 08:09:46,300][20347] Updated weights for policy 0, policy_version 364357 (0.0008) [2023-12-27 08:09:46,329][20381] Updated weights for policy 1, policy_version 364041 (0.0005) [2023-12-27 08:09:46,355][20347] Updated weights for policy 0, policy_version 364367 (0.0006) [2023-12-27 08:09:46,387][20381] Updated weights for policy 1, policy_version 364051 (0.0008) [2023-12-27 08:09:46,412][20347] Updated weights for policy 0, policy_version 364377 (0.0006) [2023-12-27 08:09:46,957][20347] Updated weights for policy 0, policy_version 364387 (0.0008) [2023-12-27 08:09:47,001][20347] Updated weights for policy 0, policy_version 364397 (0.0008) [2023-12-27 08:09:47,050][20347] Updated weights for policy 0, policy_version 364407 (0.0006) [2023-12-27 08:09:47,082][20381] Updated weights for policy 1, policy_version 364061 (0.0006) [2023-12-27 08:09:47,136][20381] Updated weights for policy 1, policy_version 364071 (0.0008) [2023-12-27 08:09:47,183][20381] Updated weights for policy 1, policy_version 364081 (0.0007) [2023-12-27 08:09:47,363][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 186531840. Throughput: 0: 10446.6, 1: 10423.7. Samples: 186537724. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:47,364][19170] Avg episode reward: [(0, '980.750'), (1, '990.360')] [2023-12-27 08:09:47,738][20347] Updated weights for policy 0, policy_version 364417 (0.0007) [2023-12-27 08:09:47,792][20347] Updated weights for policy 0, policy_version 364427 (0.0008) [2023-12-27 08:09:47,845][20347] Updated weights for policy 0, policy_version 364437 (0.0007) [2023-12-27 08:09:47,856][20381] Updated weights for policy 1, policy_version 364091 (0.0006) [2023-12-27 08:09:47,898][20347] Updated weights for policy 0, policy_version 364447 (0.0007) [2023-12-27 08:09:47,908][20381] Updated weights for policy 1, policy_version 364101 (0.0006) [2023-12-27 08:09:47,960][20381] Updated weights for policy 1, policy_version 364111 (0.0005) [2023-12-27 08:09:48,582][20381] Updated weights for policy 1, policy_version 364121 (0.0006) [2023-12-27 08:09:48,636][20381] Updated weights for policy 1, policy_version 364131 (0.0006) [2023-12-27 08:09:48,646][20347] Updated weights for policy 0, policy_version 364457 (0.0006) [2023-12-27 08:09:48,698][20347] Updated weights for policy 0, policy_version 364467 (0.0005) [2023-12-27 08:09:48,700][20381] Updated weights for policy 1, policy_version 364141 (0.0007) [2023-12-27 08:09:48,750][20347] Updated weights for policy 0, policy_version 364477 (0.0006) [2023-12-27 08:09:48,766][20381] Updated weights for policy 1, policy_version 364151 (0.0008) [2023-12-27 08:09:49,382][20347] Updated weights for policy 0, policy_version 364487 (0.0007) [2023-12-27 08:09:49,423][20381] Updated weights for policy 1, policy_version 364161 (0.0006) [2023-12-27 08:09:49,428][20347] Updated weights for policy 0, policy_version 364497 (0.0005) [2023-12-27 08:09:49,479][20347] Updated weights for policy 0, policy_version 364507 (0.0006) [2023-12-27 08:09:49,486][20381] Updated weights for policy 1, policy_version 364171 (0.0007) [2023-12-27 08:09:49,546][20381] Updated weights for policy 1, policy_version 364181 (0.0007) [2023-12-27 08:09:50,200][20381] Updated weights for policy 1, policy_version 364191 (0.0007) [2023-12-27 08:09:50,220][20347] Updated weights for policy 0, policy_version 364517 (0.0008) [2023-12-27 08:09:50,255][20381] Updated weights for policy 1, policy_version 364201 (0.0007) [2023-12-27 08:09:50,279][20347] Updated weights for policy 0, policy_version 364527 (0.0007) [2023-12-27 08:09:50,308][20381] Updated weights for policy 1, policy_version 364211 (0.0006) [2023-12-27 08:09:50,330][20347] Updated weights for policy 0, policy_version 364537 (0.0006) [2023-12-27 08:09:50,937][20381] Updated weights for policy 1, policy_version 364221 (0.0007) [2023-12-27 08:09:50,991][20381] Updated weights for policy 1, policy_version 364231 (0.0008) [2023-12-27 08:09:51,061][20381] Updated weights for policy 1, policy_version 364241 (0.0009) [2023-12-27 08:09:51,063][20347] Updated weights for policy 0, policy_version 364547 (0.0007) [2023-12-27 08:09:51,128][20347] Updated weights for policy 0, policy_version 364557 (0.0008) [2023-12-27 08:09:51,187][20347] Updated weights for policy 0, policy_version 364567 (0.0007) [2023-12-27 08:09:51,713][20381] Updated weights for policy 1, policy_version 364251 (0.0007) [2023-12-27 08:09:51,782][20381] Updated weights for policy 1, policy_version 364261 (0.0008) [2023-12-27 08:09:51,832][20381] Updated weights for policy 1, policy_version 364271 (0.0008) [2023-12-27 08:09:51,921][20347] Updated weights for policy 0, policy_version 364577 (0.0008) [2023-12-27 08:09:51,979][20347] Updated weights for policy 0, policy_version 364587 (0.0007) [2023-12-27 08:09:52,034][20347] Updated weights for policy 0, policy_version 364597 (0.0007) [2023-12-27 08:09:52,091][20347] Updated weights for policy 0, policy_version 364607 (0.0005) [2023-12-27 08:09:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 186630144. Throughput: 0: 10429.2, 1: 10462.9. Samples: 186600140. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:52,363][19170] Avg episode reward: [(0, '990.410'), (1, '1000.000')] [2023-12-27 08:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000364280_93274112.pth... [2023-12-27 08:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000364608_93356032.pth... [2023-12-27 08:09:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000363064_92962816.pth [2023-12-27 08:09:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000363392_93044736.pth [2023-12-27 08:09:52,542][20381] Updated weights for policy 1, policy_version 364281 (0.0008) [2023-12-27 08:09:52,593][20381] Updated weights for policy 1, policy_version 364291 (0.0007) [2023-12-27 08:09:52,644][20381] Updated weights for policy 1, policy_version 364301 (0.0007) [2023-12-27 08:09:52,691][20347] Updated weights for policy 0, policy_version 364617 (0.0008) [2023-12-27 08:09:52,693][20381] Updated weights for policy 1, policy_version 364311 (0.0007) [2023-12-27 08:09:52,738][20347] Updated weights for policy 0, policy_version 364627 (0.0008) [2023-12-27 08:09:52,801][20347] Updated weights for policy 0, policy_version 364637 (0.0006) [2023-12-27 08:09:53,379][20381] Updated weights for policy 1, policy_version 364321 (0.0007) [2023-12-27 08:09:53,429][20381] Updated weights for policy 1, policy_version 364331 (0.0007) [2023-12-27 08:09:53,446][20347] Updated weights for policy 0, policy_version 364647 (0.0007) [2023-12-27 08:09:53,477][20381] Updated weights for policy 1, policy_version 364341 (0.0007) [2023-12-27 08:09:53,504][20347] Updated weights for policy 0, policy_version 364657 (0.0005) [2023-12-27 08:09:53,567][20347] Updated weights for policy 0, policy_version 364667 (0.0008) [2023-12-27 08:09:54,206][20381] Updated weights for policy 1, policy_version 364351 (0.0007) [2023-12-27 08:09:54,222][20347] Updated weights for policy 0, policy_version 364677 (0.0008) [2023-12-27 08:09:54,262][20381] Updated weights for policy 1, policy_version 364361 (0.0007) [2023-12-27 08:09:54,279][20347] Updated weights for policy 0, policy_version 364687 (0.0006) [2023-12-27 08:09:54,314][20381] Updated weights for policy 1, policy_version 364371 (0.0007) [2023-12-27 08:09:54,337][20347] Updated weights for policy 0, policy_version 364697 (0.0008) [2023-12-27 08:09:54,971][20381] Updated weights for policy 1, policy_version 364381 (0.0006) [2023-12-27 08:09:55,028][20381] Updated weights for policy 1, policy_version 364391 (0.0006) [2023-12-27 08:09:55,034][20347] Updated weights for policy 0, policy_version 364707 (0.0008) [2023-12-27 08:09:55,037][20292] KL-divergence is very high: 110.1974 [2023-12-27 08:09:55,074][20381] Updated weights for policy 1, policy_version 364401 (0.0005) [2023-12-27 08:09:55,080][20347] Updated weights for policy 0, policy_version 364717 (0.0006) [2023-12-27 08:09:55,131][20347] Updated weights for policy 0, policy_version 364727 (0.0005) [2023-12-27 08:09:55,690][20381] Updated weights for policy 1, policy_version 364411 (0.0006) [2023-12-27 08:09:55,738][20347] Updated weights for policy 0, policy_version 364737 (0.0006) [2023-12-27 08:09:55,758][20381] Updated weights for policy 1, policy_version 364421 (0.0005) [2023-12-27 08:09:55,791][20347] Updated weights for policy 0, policy_version 364747 (0.0007) [2023-12-27 08:09:55,826][20381] Updated weights for policy 1, policy_version 364431 (0.0006) [2023-12-27 08:09:55,843][20347] Updated weights for policy 0, policy_version 364757 (0.0007) [2023-12-27 08:09:55,892][20347] Updated weights for policy 0, policy_version 364767 (0.0009) [2023-12-27 08:09:56,388][20381] Updated weights for policy 1, policy_version 364441 (0.0005) [2023-12-27 08:09:56,440][20381] Updated weights for policy 1, policy_version 364451 (0.0005) [2023-12-27 08:09:56,497][20381] Updated weights for policy 1, policy_version 364461 (0.0005) [2023-12-27 08:09:56,556][20381] Updated weights for policy 1, policy_version 364471 (0.0005) [2023-12-27 08:09:56,574][20347] Updated weights for policy 0, policy_version 364777 (0.0007) [2023-12-27 08:09:56,627][20347] Updated weights for policy 0, policy_version 364787 (0.0008) [2023-12-27 08:09:56,680][20347] Updated weights for policy 0, policy_version 364797 (0.0007) [2023-12-27 08:09:57,115][20381] Updated weights for policy 1, policy_version 364481 (0.0009) [2023-12-27 08:09:57,130][20292] KL-divergence is very high: 394.0838 [2023-12-27 08:09:57,164][20381] Updated weights for policy 1, policy_version 364491 (0.0007) [2023-12-27 08:09:57,219][20381] Updated weights for policy 1, policy_version 364501 (0.0005) [2023-12-27 08:09:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 186736640. Throughput: 0: 10448.7, 1: 10497.3. Samples: 186724724. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:09:57,363][19170] Avg episode reward: [(0, '981.280'), (1, '971.090')] [2023-12-27 08:09:57,377][20347] Updated weights for policy 0, policy_version 364807 (0.0010) [2023-12-27 08:09:57,432][20347] Updated weights for policy 0, policy_version 364817 (0.0008) [2023-12-27 08:09:57,486][20347] Updated weights for policy 0, policy_version 364827 (0.0006) [2023-12-27 08:09:57,771][20381] Updated weights for policy 1, policy_version 364511 (0.0009) [2023-12-27 08:09:57,816][20381] Updated weights for policy 1, policy_version 364521 (0.0010) [2023-12-27 08:09:57,861][20381] Updated weights for policy 1, policy_version 364531 (0.0008) [2023-12-27 08:09:58,131][20347] Updated weights for policy 0, policy_version 364837 (0.0010) [2023-12-27 08:09:58,190][20347] Updated weights for policy 0, policy_version 364847 (0.0010) [2023-12-27 08:09:58,248][20347] Updated weights for policy 0, policy_version 364857 (0.0010) [2023-12-27 08:09:58,590][20381] Updated weights for policy 1, policy_version 364541 (0.0006) [2023-12-27 08:09:58,651][20381] Updated weights for policy 1, policy_version 364551 (0.0008) [2023-12-27 08:09:58,709][20381] Updated weights for policy 1, policy_version 364561 (0.0009) [2023-12-27 08:09:59,094][20347] Updated weights for policy 0, policy_version 364867 (0.0010) [2023-12-27 08:09:59,148][20347] Updated weights for policy 0, policy_version 364877 (0.0009) [2023-12-27 08:09:59,205][20347] Updated weights for policy 0, policy_version 364887 (0.0009) [2023-12-27 08:09:59,625][20381] Updated weights for policy 1, policy_version 364571 (0.0009) [2023-12-27 08:09:59,674][20381] Updated weights for policy 1, policy_version 364581 (0.0009) [2023-12-27 08:09:59,694][20292] KL-divergence is very high: 119.0630 [2023-12-27 08:09:59,703][20292] KL-divergence is very high: 280.2842 [2023-12-27 08:09:59,722][20381] Updated weights for policy 1, policy_version 364591 (0.0007) [2023-12-27 08:09:59,981][20347] Updated weights for policy 0, policy_version 364897 (0.0009) [2023-12-27 08:10:00,040][20347] Updated weights for policy 0, policy_version 364907 (0.0006) [2023-12-27 08:10:00,096][20347] Updated weights for policy 0, policy_version 364917 (0.0008) [2023-12-27 08:10:00,148][20347] Updated weights for policy 0, policy_version 364927 (0.0007) [2023-12-27 08:10:00,437][20381] Updated weights for policy 1, policy_version 364601 (0.0006) [2023-12-27 08:10:00,480][20292] KL-divergence is very high: 100.4906 [2023-12-27 08:10:00,498][20381] Updated weights for policy 1, policy_version 364611 (0.0009) [2023-12-27 08:10:00,522][20292] KL-divergence is very high: 594.7133 [2023-12-27 08:10:00,529][20292] KL-divergence is very high: 478.9635 [2023-12-27 08:10:00,555][20381] Updated weights for policy 1, policy_version 364621 (0.0008) [2023-12-27 08:10:00,574][20292] KL-divergence is very high: 118.5633 [2023-12-27 08:10:00,616][20381] Updated weights for policy 1, policy_version 364631 (0.0008) [2023-12-27 08:10:00,816][20347] Updated weights for policy 0, policy_version 364937 (0.0008) [2023-12-27 08:10:00,864][20347] Updated weights for policy 0, policy_version 364947 (0.0008) [2023-12-27 08:10:00,918][20347] Updated weights for policy 0, policy_version 364957 (0.0007) [2023-12-27 08:10:01,292][20292] KL-divergence is very high: 211.7141 [2023-12-27 08:10:01,335][20381] Updated weights for policy 1, policy_version 364641 (0.0006) [2023-12-27 08:10:01,336][20292] KL-divergence is very high: 122.1439 [2023-12-27 08:10:01,391][20292] KL-divergence is very high: 120.9835 [2023-12-27 08:10:01,405][20381] Updated weights for policy 1, policy_version 364651 (0.0008) [2023-12-27 08:10:01,416][20292] KL-divergence is very high: 331.7855 [2023-12-27 08:10:01,429][20292] KL-divergence is very high: 273.7303 [2023-12-27 08:10:01,441][20292] KL-divergence is very high: 184.2785 [2023-12-27 08:10:01,465][20381] Updated weights for policy 1, policy_version 364661 (0.0008) [2023-12-27 08:10:01,603][20347] Updated weights for policy 0, policy_version 364967 (0.0007) [2023-12-27 08:10:01,665][20347] Updated weights for policy 0, policy_version 364977 (0.0007) [2023-12-27 08:10:01,721][20347] Updated weights for policy 0, policy_version 364987 (0.0008) [2023-12-27 08:10:02,145][20292] KL-divergence is very high: 348.2766 [2023-12-27 08:10:02,170][20381] Updated weights for policy 1, policy_version 364671 (0.0008) [2023-12-27 08:10:02,192][20292] KL-divergence is very high: 214.4662 [2023-12-27 08:10:02,197][20292] KL-divergence is very high: 126.6371 [2023-12-27 08:10:02,223][20381] Updated weights for policy 1, policy_version 364681 (0.0008) [2023-12-27 08:10:02,234][20292] KL-divergence is very high: 520.9948 [2023-12-27 08:10:02,251][20292] KL-divergence is very high: 145.3348 [2023-12-27 08:10:02,280][20381] Updated weights for policy 1, policy_version 364691 (0.0008) [2023-12-27 08:10:02,281][20292] KL-divergence is very high: 178.2219 [2023-12-27 08:10:02,299][20292] KL-divergence is very high: 114.8700 [2023-12-27 08:10:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 186834944. Throughput: 0: 10390.9, 1: 10366.3. Samples: 186844140. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:10:02,363][19170] Avg episode reward: [(0, '981.270'), (1, '890.630')] [2023-12-27 08:10:02,396][20347] Updated weights for policy 0, policy_version 364997 (0.0007) [2023-12-27 08:10:02,447][20347] Updated weights for policy 0, policy_version 365007 (0.0007) [2023-12-27 08:10:02,506][20347] Updated weights for policy 0, policy_version 365017 (0.0007) [2023-12-27 08:10:03,010][20381] Updated weights for policy 1, policy_version 364701 (0.0008) [2023-12-27 08:10:03,066][20381] Updated weights for policy 1, policy_version 364711 (0.0008) [2023-12-27 08:10:03,119][20347] Updated weights for policy 0, policy_version 365027 (0.0007) [2023-12-27 08:10:03,120][20381] Updated weights for policy 1, policy_version 364721 (0.0008) [2023-12-27 08:10:03,164][20347] Updated weights for policy 0, policy_version 365037 (0.0006) [2023-12-27 08:10:03,216][20347] Updated weights for policy 0, policy_version 365047 (0.0007) [2023-12-27 08:10:03,762][20381] Updated weights for policy 1, policy_version 364731 (0.0006) [2023-12-27 08:10:03,825][20381] Updated weights for policy 1, policy_version 364741 (0.0007) [2023-12-27 08:10:03,888][20381] Updated weights for policy 1, policy_version 364751 (0.0007) [2023-12-27 08:10:03,922][20347] Updated weights for policy 0, policy_version 365057 (0.0007) [2023-12-27 08:10:03,978][20347] Updated weights for policy 0, policy_version 365067 (0.0008) [2023-12-27 08:10:04,034][20347] Updated weights for policy 0, policy_version 365077 (0.0007) [2023-12-27 08:10:04,098][20347] Updated weights for policy 0, policy_version 365087 (0.0009) [2023-12-27 08:10:04,546][20381] Updated weights for policy 1, policy_version 364761 (0.0008) [2023-12-27 08:10:04,598][20381] Updated weights for policy 1, policy_version 364771 (0.0008) [2023-12-27 08:10:04,651][20381] Updated weights for policy 1, policy_version 364781 (0.0008) [2023-12-27 08:10:04,710][20381] Updated weights for policy 1, policy_version 364791 (0.0008) [2023-12-27 08:10:04,775][20347] Updated weights for policy 0, policy_version 365097 (0.0008) [2023-12-27 08:10:04,834][20347] Updated weights for policy 0, policy_version 365107 (0.0007) [2023-12-27 08:10:04,890][20347] Updated weights for policy 0, policy_version 365117 (0.0009) [2023-12-27 08:10:05,367][20381] Updated weights for policy 1, policy_version 364801 (0.0007) [2023-12-27 08:10:05,415][20381] Updated weights for policy 1, policy_version 364811 (0.0008) [2023-12-27 08:10:05,467][20381] Updated weights for policy 1, policy_version 364821 (0.0008) [2023-12-27 08:10:05,587][20347] Updated weights for policy 0, policy_version 365127 (0.0010) [2023-12-27 08:10:05,646][20347] Updated weights for policy 0, policy_version 365137 (0.0010) [2023-12-27 08:10:05,700][20347] Updated weights for policy 0, policy_version 365147 (0.0010) [2023-12-27 08:10:06,190][20381] Updated weights for policy 1, policy_version 364831 (0.0007) [2023-12-27 08:10:06,246][20381] Updated weights for policy 1, policy_version 364841 (0.0007) [2023-12-27 08:10:06,305][20381] Updated weights for policy 1, policy_version 364851 (0.0007) [2023-12-27 08:10:06,382][20347] Updated weights for policy 0, policy_version 365157 (0.0010) [2023-12-27 08:10:06,441][20347] Updated weights for policy 0, policy_version 365167 (0.0010) [2023-12-27 08:10:06,500][20347] Updated weights for policy 0, policy_version 365177 (0.0011) [2023-12-27 08:10:07,019][20381] Updated weights for policy 1, policy_version 364861 (0.0008) [2023-12-27 08:10:07,080][20381] Updated weights for policy 1, policy_version 364871 (0.0008) [2023-12-27 08:10:07,136][20381] Updated weights for policy 1, policy_version 364881 (0.0007) [2023-12-27 08:10:07,183][20347] Updated weights for policy 0, policy_version 365187 (0.0009) [2023-12-27 08:10:07,248][20347] Updated weights for policy 0, policy_version 365197 (0.0008) [2023-12-27 08:10:07,313][20347] Updated weights for policy 0, policy_version 365207 (0.0008) [2023-12-27 08:10:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 186933248. Throughput: 0: 10394.1, 1: 10339.0. Samples: 186905980. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:10:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.380')] [2023-12-27 08:10:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000364888_93429760.pth... [2023-12-27 08:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000365216_93511680.pth... [2023-12-27 08:10:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000363672_93118464.pth [2023-12-27 08:10:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000364000_93200384.pth [2023-12-27 08:10:07,764][20381] Updated weights for policy 1, policy_version 364891 (0.0009) [2023-12-27 08:10:07,815][20381] Updated weights for policy 1, policy_version 364901 (0.0007) [2023-12-27 08:10:07,872][20381] Updated weights for policy 1, policy_version 364911 (0.0008) [2023-12-27 08:10:07,970][20347] Updated weights for policy 0, policy_version 365217 (0.0009) [2023-12-27 08:10:08,026][20347] Updated weights for policy 0, policy_version 365227 (0.0008) [2023-12-27 08:10:08,076][20347] Updated weights for policy 0, policy_version 365238 (0.0009) [2023-12-27 08:10:08,125][20347] Updated weights for policy 0, policy_version 365248 (0.0008) [2023-12-27 08:10:08,558][20381] Updated weights for policy 1, policy_version 364921 (0.0007) [2023-12-27 08:10:08,616][20381] Updated weights for policy 1, policy_version 364931 (0.0008) [2023-12-27 08:10:08,675][20381] Updated weights for policy 1, policy_version 364941 (0.0008) [2023-12-27 08:10:08,733][20381] Updated weights for policy 1, policy_version 364951 (0.0007) [2023-12-27 08:10:08,825][20347] Updated weights for policy 0, policy_version 365258 (0.0006) [2023-12-27 08:10:08,881][20347] Updated weights for policy 0, policy_version 365268 (0.0005) [2023-12-27 08:10:08,930][20347] Updated weights for policy 0, policy_version 365278 (0.0007) [2023-12-27 08:10:09,486][20381] Updated weights for policy 1, policy_version 364961 (0.0008) [2023-12-27 08:10:09,546][20381] Updated weights for policy 1, policy_version 364971 (0.0008) [2023-12-27 08:10:09,603][20381] Updated weights for policy 1, policy_version 364981 (0.0006) [2023-12-27 08:10:09,612][20347] Updated weights for policy 0, policy_version 365288 (0.0007) [2023-12-27 08:10:09,659][20347] Updated weights for policy 0, policy_version 365298 (0.0008) [2023-12-27 08:10:09,708][20347] Updated weights for policy 0, policy_version 365308 (0.0007) [2023-12-27 08:10:10,270][20381] Updated weights for policy 1, policy_version 364991 (0.0007) [2023-12-27 08:10:10,321][20381] Updated weights for policy 1, policy_version 365001 (0.0010) [2023-12-27 08:10:10,378][20381] Updated weights for policy 1, policy_version 365011 (0.0010) [2023-12-27 08:10:10,449][20347] Updated weights for policy 0, policy_version 365318 (0.0007) [2023-12-27 08:10:10,502][20347] Updated weights for policy 0, policy_version 365328 (0.0007) [2023-12-27 08:10:10,560][20347] Updated weights for policy 0, policy_version 365338 (0.0007) [2023-12-27 08:10:11,087][20381] Updated weights for policy 1, policy_version 365021 (0.0011) [2023-12-27 08:10:11,149][20381] Updated weights for policy 1, policy_version 365031 (0.0011) [2023-12-27 08:10:11,184][20347] Updated weights for policy 0, policy_version 365348 (0.0008) [2023-12-27 08:10:11,210][20381] Updated weights for policy 1, policy_version 365041 (0.0006) [2023-12-27 08:10:11,244][20347] Updated weights for policy 0, policy_version 365358 (0.0007) [2023-12-27 08:10:11,301][20347] Updated weights for policy 0, policy_version 365368 (0.0010) [2023-12-27 08:10:11,933][20381] Updated weights for policy 1, policy_version 365051 (0.0010) [2023-12-27 08:10:11,985][20381] Updated weights for policy 1, policy_version 365061 (0.0007) [2023-12-27 08:10:11,998][20347] Updated weights for policy 0, policy_version 365378 (0.0010) [2023-12-27 08:10:12,053][20381] Updated weights for policy 1, policy_version 365071 (0.0005) [2023-12-27 08:10:12,055][20347] Updated weights for policy 0, policy_version 365388 (0.0006) [2023-12-27 08:10:12,061][20292] KL-divergence is very high: 290.7627 [2023-12-27 08:10:12,111][20347] Updated weights for policy 0, policy_version 365398 (0.0008) [2023-12-27 08:10:12,160][20347] Updated weights for policy 0, policy_version 365408 (0.0010) [2023-12-27 08:10:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 187039744. Throughput: 0: 10357.1, 1: 10347.3. Samples: 187026912. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:10:12,363][19170] Avg episode reward: [(0, '980.820'), (1, '904.300')] [2023-12-27 08:10:12,746][20381] Updated weights for policy 1, policy_version 365081 (0.0008) [2023-12-27 08:10:12,810][20381] Updated weights for policy 1, policy_version 365091 (0.0007) [2023-12-27 08:10:12,817][20347] Updated weights for policy 0, policy_version 365418 (0.0007) [2023-12-27 08:10:12,862][20381] Updated weights for policy 1, policy_version 365101 (0.0008) [2023-12-27 08:10:12,879][20347] Updated weights for policy 0, policy_version 365428 (0.0005) [2023-12-27 08:10:12,915][20381] Updated weights for policy 1, policy_version 365111 (0.0007) [2023-12-27 08:10:12,937][20347] Updated weights for policy 0, policy_version 365438 (0.0005) [2023-12-27 08:10:13,448][20347] Updated weights for policy 0, policy_version 365448 (0.0005) [2023-12-27 08:10:13,503][20347] Updated weights for policy 0, policy_version 365458 (0.0006) [2023-12-27 08:10:13,557][20347] Updated weights for policy 0, policy_version 365468 (0.0010) [2023-12-27 08:10:13,657][20381] Updated weights for policy 1, policy_version 365121 (0.0005) [2023-12-27 08:10:13,676][20292] KL-divergence is very high: 119.5117 [2023-12-27 08:10:13,685][20292] KL-divergence is very high: 118.1278 [2023-12-27 08:10:13,695][20292] KL-divergence is very high: 115.9609 [2023-12-27 08:10:13,703][20381] Updated weights for policy 1, policy_version 365131 (0.0005) [2023-12-27 08:10:13,759][20381] Updated weights for policy 1, policy_version 365141 (0.0006) [2023-12-27 08:10:14,197][20347] Updated weights for policy 0, policy_version 365478 (0.0010) [2023-12-27 08:10:14,249][20347] Updated weights for policy 0, policy_version 365488 (0.0010) [2023-12-27 08:10:14,304][20347] Updated weights for policy 0, policy_version 365498 (0.0010) [2023-12-27 08:10:14,363][20292] KL-divergence is very high: 120.6616 [2023-12-27 08:10:14,368][20381] Updated weights for policy 1, policy_version 365151 (0.0009) [2023-12-27 08:10:14,375][20292] KL-divergence is very high: 151.9856 [2023-12-27 08:10:14,402][20292] KL-divergence is very high: 107.4588 [2023-12-27 08:10:14,408][20292] KL-divergence is very high: 114.1857 [2023-12-27 08:10:14,420][20292] KL-divergence is very high: 156.0546 [2023-12-27 08:10:14,426][20381] Updated weights for policy 1, policy_version 365161 (0.0011) [2023-12-27 08:10:14,431][20292] KL-divergence is very high: 117.0166 [2023-12-27 08:10:14,482][20381] Updated weights for policy 1, policy_version 365171 (0.0010) [2023-12-27 08:10:14,982][20347] Updated weights for policy 0, policy_version 365508 (0.0008) [2023-12-27 08:10:15,039][20347] Updated weights for policy 0, policy_version 365518 (0.0007) [2023-12-27 08:10:15,089][20347] Updated weights for policy 0, policy_version 365528 (0.0006) [2023-12-27 08:10:15,116][20292] KL-divergence is very high: 132.3435 [2023-12-27 08:10:15,144][20381] Updated weights for policy 1, policy_version 365181 (0.0007) [2023-12-27 08:10:15,205][20381] Updated weights for policy 1, policy_version 365191 (0.0006) [2023-12-27 08:10:15,265][20292] KL-divergence is very high: 135.8953 [2023-12-27 08:10:15,272][20381] Updated weights for policy 1, policy_version 365201 (0.0008) [2023-12-27 08:10:15,721][20347] Updated weights for policy 0, policy_version 365538 (0.0007) [2023-12-27 08:10:15,777][20347] Updated weights for policy 0, policy_version 365548 (0.0007) [2023-12-27 08:10:15,835][20347] Updated weights for policy 0, policy_version 365558 (0.0006) [2023-12-27 08:10:15,901][20347] Updated weights for policy 0, policy_version 365568 (0.0005) [2023-12-27 08:10:15,901][20381] Updated weights for policy 1, policy_version 365211 (0.0007) [2023-12-27 08:10:15,954][20381] Updated weights for policy 1, policy_version 365221 (0.0006) [2023-12-27 08:10:16,004][20381] Updated weights for policy 1, policy_version 365231 (0.0006) [2023-12-27 08:10:16,544][20347] Updated weights for policy 0, policy_version 365578 (0.0010) [2023-12-27 08:10:16,590][20347] Updated weights for policy 0, policy_version 365588 (0.0010) [2023-12-27 08:10:16,602][20381] Updated weights for policy 1, policy_version 365241 (0.0008) [2023-12-27 08:10:16,648][20347] Updated weights for policy 0, policy_version 365598 (0.0010) [2023-12-27 08:10:16,661][20381] Updated weights for policy 1, policy_version 365251 (0.0008) [2023-12-27 08:10:16,713][20381] Updated weights for policy 1, policy_version 365261 (0.0010) [2023-12-27 08:10:16,764][20381] Updated weights for policy 1, policy_version 365271 (0.0009) [2023-12-27 08:10:17,284][20347] Updated weights for policy 0, policy_version 365608 (0.0006) [2023-12-27 08:10:17,340][20347] Updated weights for policy 0, policy_version 365618 (0.0007) [2023-12-27 08:10:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 187138048. Throughput: 0: 10350.6, 1: 10302.0. Samples: 187153672. Policy #0 lag: (min: 2.0, avg: 10.2, max: 34.0) [2023-12-27 08:10:17,363][19170] Avg episode reward: [(0, '980.820'), (1, '685.530')] [2023-12-27 08:10:17,381][20381] Updated weights for policy 1, policy_version 365281 (0.0010) [2023-12-27 08:10:17,395][20347] Updated weights for policy 0, policy_version 365628 (0.0010) [2023-12-27 08:10:17,429][20381] Updated weights for policy 1, policy_version 365291 (0.0010) [2023-12-27 08:10:17,483][20381] Updated weights for policy 1, policy_version 365301 (0.0006) [2023-12-27 08:10:17,971][20347] Updated weights for policy 0, policy_version 365638 (0.0007) [2023-12-27 08:10:18,022][20347] Updated weights for policy 0, policy_version 365648 (0.0010) [2023-12-27 08:10:18,076][20347] Updated weights for policy 0, policy_version 365658 (0.0010) [2023-12-27 08:10:18,146][20381] Updated weights for policy 1, policy_version 365311 (0.0006) [2023-12-27 08:10:18,194][20381] Updated weights for policy 1, policy_version 365321 (0.0007) [2023-12-27 08:10:18,242][20381] Updated weights for policy 1, policy_version 365331 (0.0008) [2023-12-27 08:10:18,784][20347] Updated weights for policy 0, policy_version 365668 (0.0009) [2023-12-27 08:10:18,844][20347] Updated weights for policy 0, policy_version 365678 (0.0007) [2023-12-27 08:10:18,900][20347] Updated weights for policy 0, policy_version 365688 (0.0011) [2023-12-27 08:10:18,934][20381] Updated weights for policy 1, policy_version 365341 (0.0009) [2023-12-27 08:10:18,986][20381] Updated weights for policy 1, policy_version 365351 (0.0006) [2023-12-27 08:10:19,032][20381] Updated weights for policy 1, policy_version 365361 (0.0009) [2023-12-27 08:10:19,523][20347] Updated weights for policy 0, policy_version 365698 (0.0008) [2023-12-27 08:10:19,582][20347] Updated weights for policy 0, policy_version 365708 (0.0010) [2023-12-27 08:10:19,638][20347] Updated weights for policy 0, policy_version 365718 (0.0009) [2023-12-27 08:10:19,693][20347] Updated weights for policy 0, policy_version 365728 (0.0006) [2023-12-27 08:10:19,710][20381] Updated weights for policy 1, policy_version 365371 (0.0007) [2023-12-27 08:10:19,748][20292] KL-divergence is very high: 142.9647 [2023-12-27 08:10:19,759][20292] KL-divergence is very high: 149.5973 [2023-12-27 08:10:19,763][20381] Updated weights for policy 1, policy_version 365381 (0.0005) [2023-12-27 08:10:19,802][20292] KL-divergence is very high: 202.8875 [2023-12-27 08:10:19,821][20381] Updated weights for policy 1, policy_version 365391 (0.0006) [2023-12-27 08:10:19,836][20292] KL-divergence is very high: 277.5395 [2023-12-27 08:10:20,406][20347] Updated weights for policy 0, policy_version 365738 (0.0006) [2023-12-27 08:10:20,465][20347] Updated weights for policy 0, policy_version 365748 (0.0007) [2023-12-27 08:10:20,468][20381] Updated weights for policy 1, policy_version 365401 (0.0007) [2023-12-27 08:10:20,477][20292] KL-divergence is very high: 288.9425 [2023-12-27 08:10:20,522][20347] Updated weights for policy 0, policy_version 365758 (0.0007) [2023-12-27 08:10:20,529][20292] KL-divergence is very high: 189.4066 [2023-12-27 08:10:20,535][20381] Updated weights for policy 1, policy_version 365411 (0.0006) [2023-12-27 08:10:20,544][20292] KL-divergence is very high: 111.7875 [2023-12-27 08:10:20,566][20292] KL-divergence is very high: 130.2924 [2023-12-27 08:10:20,587][20292] KL-divergence is very high: 109.5796 [2023-12-27 08:10:20,605][20381] Updated weights for policy 1, policy_version 365421 (0.0008) [2023-12-27 08:10:20,634][20292] KL-divergence is very high: 197.1519 [2023-12-27 08:10:20,665][20381] Updated weights for policy 1, policy_version 365431 (0.0007) [2023-12-27 08:10:20,665][20292] KL-divergence is very high: 107.1040 [2023-12-27 08:10:21,219][20347] Updated weights for policy 0, policy_version 365768 (0.0008) [2023-12-27 08:10:21,283][20347] Updated weights for policy 0, policy_version 365778 (0.0008) [2023-12-27 08:10:21,336][20381] Updated weights for policy 1, policy_version 365441 (0.0006) [2023-12-27 08:10:21,338][20347] Updated weights for policy 0, policy_version 365788 (0.0006) [2023-12-27 08:10:21,387][20292] KL-divergence is very high: 106.7792 [2023-12-27 08:10:21,406][20381] Updated weights for policy 1, policy_version 365451 (0.0006) [2023-12-27 08:10:21,448][20292] KL-divergence is very high: 115.2128 [2023-12-27 08:10:21,454][20292] KL-divergence is very high: 131.5803 [2023-12-27 08:10:21,461][20292] KL-divergence is very high: 120.0414 [2023-12-27 08:10:21,466][20381] Updated weights for policy 1, policy_version 365461 (0.0008) [2023-12-27 08:10:22,067][20347] Updated weights for policy 0, policy_version 365798 (0.0007) [2023-12-27 08:10:22,128][20347] Updated weights for policy 0, policy_version 365808 (0.0008) [2023-12-27 08:10:22,192][20347] Updated weights for policy 0, policy_version 365818 (0.0007) [2023-12-27 08:10:22,195][20381] Updated weights for policy 1, policy_version 365471 (0.0008) [2023-12-27 08:10:22,203][20292] KL-divergence is very high: 104.4753 [2023-12-27 08:10:22,216][20292] KL-divergence is very high: 132.7107 [2023-12-27 08:10:22,260][20381] Updated weights for policy 1, policy_version 365481 (0.0007) [2023-12-27 08:10:22,318][20292] KL-divergence is very high: 106.8699 [2023-12-27 08:10:22,323][20381] Updated weights for policy 1, policy_version 365491 (0.0008) [2023-12-27 08:10:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 187252736. Throughput: 0: 10350.3, 1: 10289.3. Samples: 187217352. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '487.960')] [2023-12-27 08:10:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000365496_93585408.pth... [2023-12-27 08:10:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000365824_93667328.pth... [2023-12-27 08:10:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000364280_93274112.pth [2023-12-27 08:10:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000364608_93356032.pth [2023-12-27 08:10:22,805][20347] Updated weights for policy 0, policy_version 365828 (0.0008) [2023-12-27 08:10:22,873][20347] Updated weights for policy 0, policy_version 365838 (0.0008) [2023-12-27 08:10:22,941][20347] Updated weights for policy 0, policy_version 365848 (0.0008) [2023-12-27 08:10:23,007][20381] Updated weights for policy 1, policy_version 365501 (0.0007) [2023-12-27 08:10:23,054][20381] Updated weights for policy 1, policy_version 365511 (0.0005) [2023-12-27 08:10:23,108][20381] Updated weights for policy 1, policy_version 365521 (0.0006) [2023-12-27 08:10:23,534][20347] Updated weights for policy 0, policy_version 365858 (0.0008) [2023-12-27 08:10:23,592][20347] Updated weights for policy 0, policy_version 365868 (0.0007) [2023-12-27 08:10:23,646][20347] Updated weights for policy 0, policy_version 365878 (0.0007) [2023-12-27 08:10:23,702][20347] Updated weights for policy 0, policy_version 365888 (0.0007) [2023-12-27 08:10:23,782][20381] Updated weights for policy 1, policy_version 365531 (0.0009) [2023-12-27 08:10:23,832][20381] Updated weights for policy 1, policy_version 365541 (0.0010) [2023-12-27 08:10:23,883][20381] Updated weights for policy 1, policy_version 365551 (0.0010) [2023-12-27 08:10:24,420][20347] Updated weights for policy 0, policy_version 365898 (0.0010) [2023-12-27 08:10:24,486][20347] Updated weights for policy 0, policy_version 365908 (0.0010) [2023-12-27 08:10:24,506][20381] Updated weights for policy 1, policy_version 365561 (0.0010) [2023-12-27 08:10:24,535][20347] Updated weights for policy 0, policy_version 365918 (0.0010) [2023-12-27 08:10:24,570][20381] Updated weights for policy 1, policy_version 365571 (0.0005) [2023-12-27 08:10:24,628][20381] Updated weights for policy 1, policy_version 365581 (0.0005) [2023-12-27 08:10:24,691][20381] Updated weights for policy 1, policy_version 365591 (0.0008) [2023-12-27 08:10:25,077][20347] Updated weights for policy 0, policy_version 365928 (0.0006) [2023-12-27 08:10:25,138][20347] Updated weights for policy 0, policy_version 365938 (0.0005) [2023-12-27 08:10:25,200][20347] Updated weights for policy 0, policy_version 365948 (0.0005) [2023-12-27 08:10:25,265][20381] Updated weights for policy 1, policy_version 365601 (0.0006) [2023-12-27 08:10:25,317][20381] Updated weights for policy 1, policy_version 365611 (0.0005) [2023-12-27 08:10:25,365][20381] Updated weights for policy 1, policy_version 365621 (0.0005) [2023-12-27 08:10:25,777][20347] Updated weights for policy 0, policy_version 365958 (0.0005) [2023-12-27 08:10:25,831][20347] Updated weights for policy 0, policy_version 365968 (0.0008) [2023-12-27 08:10:25,886][20347] Updated weights for policy 0, policy_version 365978 (0.0006) [2023-12-27 08:10:25,995][20381] Updated weights for policy 1, policy_version 365631 (0.0006) [2023-12-27 08:10:26,052][20381] Updated weights for policy 1, policy_version 365641 (0.0007) [2023-12-27 08:10:26,097][20381] Updated weights for policy 1, policy_version 365651 (0.0007) [2023-12-27 08:10:26,520][20347] Updated weights for policy 0, policy_version 365988 (0.0007) [2023-12-27 08:10:26,564][20347] Updated weights for policy 0, policy_version 365998 (0.0008) [2023-12-27 08:10:26,608][20347] Updated weights for policy 0, policy_version 366008 (0.0007) [2023-12-27 08:10:26,727][20381] Updated weights for policy 1, policy_version 365661 (0.0006) [2023-12-27 08:10:26,770][20381] Updated weights for policy 1, policy_version 365671 (0.0006) [2023-12-27 08:10:26,815][20381] Updated weights for policy 1, policy_version 365681 (0.0007) [2023-12-27 08:10:27,324][20347] Updated weights for policy 0, policy_version 366018 (0.0007) [2023-12-27 08:10:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 187351040. Throughput: 0: 10403.7, 1: 10310.5. Samples: 187343476. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '263.590')] [2023-12-27 08:10:27,378][20347] Updated weights for policy 0, policy_version 366028 (0.0005) [2023-12-27 08:10:27,378][20381] Updated weights for policy 1, policy_version 365691 (0.0009) [2023-12-27 08:10:27,426][20347] Updated weights for policy 0, policy_version 366038 (0.0005) [2023-12-27 08:10:27,435][20381] Updated weights for policy 1, policy_version 365701 (0.0010) [2023-12-27 08:10:27,486][20347] Updated weights for policy 0, policy_version 366048 (0.0005) [2023-12-27 08:10:27,493][20381] Updated weights for policy 1, policy_version 365711 (0.0010) [2023-12-27 08:10:28,073][20347] Updated weights for policy 0, policy_version 366058 (0.0005) [2023-12-27 08:10:28,125][20347] Updated weights for policy 0, policy_version 366068 (0.0005) [2023-12-27 08:10:28,174][20347] Updated weights for policy 0, policy_version 366078 (0.0005) [2023-12-27 08:10:28,184][20381] Updated weights for policy 1, policy_version 365721 (0.0010) [2023-12-27 08:10:28,237][20381] Updated weights for policy 1, policy_version 365731 (0.0011) [2023-12-27 08:10:28,290][20381] Updated weights for policy 1, policy_version 365741 (0.0011) [2023-12-27 08:10:28,352][20381] Updated weights for policy 1, policy_version 365751 (0.0011) [2023-12-27 08:10:28,791][20347] Updated weights for policy 0, policy_version 366088 (0.0008) [2023-12-27 08:10:28,850][20347] Updated weights for policy 0, policy_version 366098 (0.0006) [2023-12-27 08:10:28,915][20347] Updated weights for policy 0, policy_version 366108 (0.0005) [2023-12-27 08:10:28,980][20381] Updated weights for policy 1, policy_version 365761 (0.0006) [2023-12-27 08:10:29,036][20381] Updated weights for policy 1, policy_version 365771 (0.0010) [2023-12-27 08:10:29,094][20381] Updated weights for policy 1, policy_version 365781 (0.0011) [2023-12-27 08:10:29,580][20347] Updated weights for policy 0, policy_version 366118 (0.0007) [2023-12-27 08:10:29,639][20347] Updated weights for policy 0, policy_version 366128 (0.0007) [2023-12-27 08:10:29,686][20347] Updated weights for policy 0, policy_version 366138 (0.0007) [2023-12-27 08:10:29,745][20381] Updated weights for policy 1, policy_version 365791 (0.0007) [2023-12-27 08:10:29,800][20381] Updated weights for policy 1, policy_version 365801 (0.0007) [2023-12-27 08:10:29,864][20381] Updated weights for policy 1, policy_version 365811 (0.0008) [2023-12-27 08:10:30,367][20347] Updated weights for policy 0, policy_version 366148 (0.0008) [2023-12-27 08:10:30,423][20347] Updated weights for policy 0, policy_version 366158 (0.0007) [2023-12-27 08:10:30,475][20347] Updated weights for policy 0, policy_version 366168 (0.0007) [2023-12-27 08:10:30,543][20381] Updated weights for policy 1, policy_version 365821 (0.0006) [2023-12-27 08:10:30,591][20381] Updated weights for policy 1, policy_version 365831 (0.0005) [2023-12-27 08:10:30,644][20381] Updated weights for policy 1, policy_version 365841 (0.0006) [2023-12-27 08:10:31,196][20347] Updated weights for policy 0, policy_version 366178 (0.0009) [2023-12-27 08:10:31,245][20381] Updated weights for policy 1, policy_version 365851 (0.0006) [2023-12-27 08:10:31,254][20347] Updated weights for policy 0, policy_version 366188 (0.0008) [2023-12-27 08:10:31,301][20381] Updated weights for policy 1, policy_version 365861 (0.0005) [2023-12-27 08:10:31,310][20347] Updated weights for policy 0, policy_version 366198 (0.0008) [2023-12-27 08:10:31,364][20381] Updated weights for policy 1, policy_version 365871 (0.0008) [2023-12-27 08:10:31,375][20347] Updated weights for policy 0, policy_version 366208 (0.0008) [2023-12-27 08:10:32,042][20381] Updated weights for policy 1, policy_version 365881 (0.0008) [2023-12-27 08:10:32,099][20381] Updated weights for policy 1, policy_version 365891 (0.0008) [2023-12-27 08:10:32,111][20347] Updated weights for policy 0, policy_version 366218 (0.0009) [2023-12-27 08:10:32,155][20381] Updated weights for policy 1, policy_version 365901 (0.0008) [2023-12-27 08:10:32,168][20347] Updated weights for policy 0, policy_version 366228 (0.0010) [2023-12-27 08:10:32,211][20381] Updated weights for policy 1, policy_version 365911 (0.0006) [2023-12-27 08:10:32,222][20347] Updated weights for policy 0, policy_version 366238 (0.0010) [2023-12-27 08:10:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 187465728. Throughput: 0: 10359.3, 1: 10383.3. Samples: 187471140. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '454.980')] [2023-12-27 08:10:32,795][20381] Updated weights for policy 1, policy_version 365921 (0.0007) [2023-12-27 08:10:32,828][20347] Updated weights for policy 0, policy_version 366248 (0.0006) [2023-12-27 08:10:32,847][20381] Updated weights for policy 1, policy_version 365931 (0.0007) [2023-12-27 08:10:32,886][20347] Updated weights for policy 0, policy_version 366258 (0.0006) [2023-12-27 08:10:32,895][20381] Updated weights for policy 1, policy_version 365941 (0.0007) [2023-12-27 08:10:32,938][20347] Updated weights for policy 0, policy_version 366268 (0.0005) [2023-12-27 08:10:33,472][20347] Updated weights for policy 0, policy_version 366278 (0.0006) [2023-12-27 08:10:33,520][20347] Updated weights for policy 0, policy_version 366288 (0.0008) [2023-12-27 08:10:33,581][20381] Updated weights for policy 1, policy_version 365951 (0.0006) [2023-12-27 08:10:33,583][20347] Updated weights for policy 0, policy_version 366298 (0.0005) [2023-12-27 08:10:33,630][20381] Updated weights for policy 1, policy_version 365961 (0.0005) [2023-12-27 08:10:33,696][20381] Updated weights for policy 1, policy_version 365971 (0.0008) [2023-12-27 08:10:34,211][20347] Updated weights for policy 0, policy_version 366308 (0.0005) [2023-12-27 08:10:34,267][20347] Updated weights for policy 0, policy_version 366318 (0.0005) [2023-12-27 08:10:34,329][20347] Updated weights for policy 0, policy_version 366328 (0.0010) [2023-12-27 08:10:34,379][20381] Updated weights for policy 1, policy_version 365981 (0.0007) [2023-12-27 08:10:34,441][20381] Updated weights for policy 1, policy_version 365991 (0.0007) [2023-12-27 08:10:34,502][20381] Updated weights for policy 1, policy_version 366001 (0.0007) [2023-12-27 08:10:34,959][20347] Updated weights for policy 0, policy_version 366338 (0.0007) [2023-12-27 08:10:35,015][20347] Updated weights for policy 0, policy_version 366348 (0.0008) [2023-12-27 08:10:35,076][20347] Updated weights for policy 0, policy_version 366358 (0.0008) [2023-12-27 08:10:35,129][20347] Updated weights for policy 0, policy_version 366368 (0.0006) [2023-12-27 08:10:35,186][20381] Updated weights for policy 1, policy_version 366011 (0.0008) [2023-12-27 08:10:35,246][20381] Updated weights for policy 1, policy_version 366021 (0.0008) [2023-12-27 08:10:35,299][20381] Updated weights for policy 1, policy_version 366031 (0.0006) [2023-12-27 08:10:35,738][20347] Updated weights for policy 0, policy_version 366378 (0.0007) [2023-12-27 08:10:35,793][20347] Updated weights for policy 0, policy_version 366388 (0.0010) [2023-12-27 08:10:35,858][20347] Updated weights for policy 0, policy_version 366398 (0.0010) [2023-12-27 08:10:35,921][20381] Updated weights for policy 1, policy_version 366041 (0.0005) [2023-12-27 08:10:35,982][20381] Updated weights for policy 1, policy_version 366051 (0.0005) [2023-12-27 08:10:36,037][20381] Updated weights for policy 1, policy_version 366061 (0.0005) [2023-12-27 08:10:36,093][20381] Updated weights for policy 1, policy_version 366071 (0.0005) [2023-12-27 08:10:36,523][20347] Updated weights for policy 0, policy_version 366408 (0.0009) [2023-12-27 08:10:36,585][20347] Updated weights for policy 0, policy_version 366418 (0.0007) [2023-12-27 08:10:36,649][20347] Updated weights for policy 0, policy_version 366428 (0.0006) [2023-12-27 08:10:36,674][20381] Updated weights for policy 1, policy_version 366081 (0.0007) [2023-12-27 08:10:36,735][20381] Updated weights for policy 1, policy_version 366091 (0.0008) [2023-12-27 08:10:36,788][20381] Updated weights for policy 1, policy_version 366101 (0.0007) [2023-12-27 08:10:37,219][20347] Updated weights for policy 0, policy_version 366438 (0.0006) [2023-12-27 08:10:37,263][20347] Updated weights for policy 0, policy_version 366448 (0.0005) [2023-12-27 08:10:37,310][20347] Updated weights for policy 0, policy_version 366458 (0.0005) [2023-12-27 08:10:37,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 187572224. Throughput: 0: 10427.9, 1: 10372.9. Samples: 187536176. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:37,363][19170] Avg episode reward: [(0, '981.040'), (1, '185.820')] [2023-12-27 08:10:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000366464_93831168.pth... [2023-12-27 08:10:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000365216_93511680.pth [2023-12-27 08:10:37,401][20381] Updated weights for policy 1, policy_version 366111 (0.0006) [2023-12-27 08:10:37,464][20381] Updated weights for policy 1, policy_version 366121 (0.0009) [2023-12-27 08:10:37,529][20381] Updated weights for policy 1, policy_version 366131 (0.0011) [2023-12-27 08:10:37,554][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000366136_93749248.pth... [2023-12-27 08:10:37,558][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000364888_93429760.pth [2023-12-27 08:10:37,874][20291] KL-divergence is very high: 366.3165 [2023-12-27 08:10:37,899][20347] Updated weights for policy 0, policy_version 366468 (0.0005) [2023-12-27 08:10:37,960][20347] Updated weights for policy 0, policy_version 366478 (0.0008) [2023-12-27 08:10:38,016][20347] Updated weights for policy 0, policy_version 366488 (0.0007) [2023-12-27 08:10:38,141][20381] Updated weights for policy 1, policy_version 366141 (0.0011) [2023-12-27 08:10:38,186][20381] Updated weights for policy 1, policy_version 366151 (0.0010) [2023-12-27 08:10:38,233][20381] Updated weights for policy 1, policy_version 366161 (0.0010) [2023-12-27 08:10:38,687][20347] Updated weights for policy 0, policy_version 366498 (0.0007) [2023-12-27 08:10:38,745][20347] Updated weights for policy 0, policy_version 366508 (0.0007) [2023-12-27 08:10:38,803][20347] Updated weights for policy 0, policy_version 366518 (0.0007) [2023-12-27 08:10:38,857][20347] Updated weights for policy 0, policy_version 366528 (0.0007) [2023-12-27 08:10:38,976][20381] Updated weights for policy 1, policy_version 366171 (0.0011) [2023-12-27 08:10:39,026][20381] Updated weights for policy 1, policy_version 366181 (0.0011) [2023-12-27 08:10:39,088][20381] Updated weights for policy 1, policy_version 366191 (0.0011) [2023-12-27 08:10:39,586][20347] Updated weights for policy 0, policy_version 366538 (0.0008) [2023-12-27 08:10:39,642][20347] Updated weights for policy 0, policy_version 366548 (0.0008) [2023-12-27 08:10:39,700][20347] Updated weights for policy 0, policy_version 366558 (0.0008) [2023-12-27 08:10:39,797][20381] Updated weights for policy 1, policy_version 366201 (0.0011) [2023-12-27 08:10:39,859][20381] Updated weights for policy 1, policy_version 366211 (0.0011) [2023-12-27 08:10:39,913][20381] Updated weights for policy 1, policy_version 366221 (0.0011) [2023-12-27 08:10:39,967][20381] Updated weights for policy 1, policy_version 366231 (0.0011) [2023-12-27 08:10:40,420][20347] Updated weights for policy 0, policy_version 366568 (0.0006) [2023-12-27 08:10:40,470][20347] Updated weights for policy 0, policy_version 366578 (0.0010) [2023-12-27 08:10:40,519][20347] Updated weights for policy 0, policy_version 366588 (0.0010) [2023-12-27 08:10:40,698][20381] Updated weights for policy 1, policy_version 366241 (0.0011) [2023-12-27 08:10:40,754][20381] Updated weights for policy 1, policy_version 366251 (0.0010) [2023-12-27 08:10:40,812][20381] Updated weights for policy 1, policy_version 366261 (0.0011) [2023-12-27 08:10:41,229][20347] Updated weights for policy 0, policy_version 366598 (0.0008) [2023-12-27 08:10:41,291][20347] Updated weights for policy 0, policy_version 366608 (0.0007) [2023-12-27 08:10:41,355][20347] Updated weights for policy 0, policy_version 366618 (0.0008) [2023-12-27 08:10:41,383][20291] KL-divergence is very high: 587.0447 [2023-12-27 08:10:41,561][20381] Updated weights for policy 1, policy_version 366271 (0.0011) [2023-12-27 08:10:41,630][20381] Updated weights for policy 1, policy_version 366281 (0.0010) [2023-12-27 08:10:41,694][20381] Updated weights for policy 1, policy_version 366291 (0.0010) [2023-12-27 08:10:42,054][20347] Updated weights for policy 0, policy_version 366628 (0.0007) [2023-12-27 08:10:42,115][20347] Updated weights for policy 0, policy_version 366638 (0.0007) [2023-12-27 08:10:42,116][20291] KL-divergence is very high: 280.7590 [2023-12-27 08:10:42,122][20291] KL-divergence is very high: 278.9100 [2023-12-27 08:10:42,144][20291] KL-divergence is very high: 152.2036 [2023-12-27 08:10:42,162][20291] KL-divergence is very high: 161.6426 [2023-12-27 08:10:42,168][20291] KL-divergence is very high: 142.1370 [2023-12-27 08:10:42,172][20347] Updated weights for policy 0, policy_version 366648 (0.0007) [2023-12-27 08:10:42,212][20291] KL-divergence is very high: 133.0284 [2023-12-27 08:10:42,218][20291] KL-divergence is very high: 161.3312 [2023-12-27 08:10:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 187670528. Throughput: 0: 10457.5, 1: 10365.4. Samples: 187661756. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:42,363][19170] Avg episode reward: [(0, '875.250'), (1, '278.740')] [2023-12-27 08:10:42,376][20381] Updated weights for policy 1, policy_version 366301 (0.0007) [2023-12-27 08:10:42,440][20381] Updated weights for policy 1, policy_version 366311 (0.0007) [2023-12-27 08:10:42,499][20381] Updated weights for policy 1, policy_version 366321 (0.0006) [2023-12-27 08:10:42,754][20347] Updated weights for policy 0, policy_version 366658 (0.0005) [2023-12-27 08:10:42,767][20291] KL-divergence is very high: 160.8727 [2023-12-27 08:10:42,776][20291] KL-divergence is very high: 134.4745 [2023-12-27 08:10:42,781][20291] KL-divergence is very high: 117.5450 [2023-12-27 08:10:42,807][20347] Updated weights for policy 0, policy_version 366668 (0.0007) [2023-12-27 08:10:42,808][20291] KL-divergence is very high: 341.6787 [2023-12-27 08:10:42,847][20291] KL-divergence is very high: 127.9330 [2023-12-27 08:10:42,852][20291] KL-divergence is very high: 307.2858 [2023-12-27 08:10:42,863][20347] Updated weights for policy 0, policy_version 366678 (0.0008) [2023-12-27 08:10:42,898][20291] KL-divergence is very high: 192.6638 [2023-12-27 08:10:42,921][20347] Updated weights for policy 0, policy_version 366688 (0.0008) [2023-12-27 08:10:43,172][20381] Updated weights for policy 1, policy_version 366331 (0.0007) [2023-12-27 08:10:43,228][20381] Updated weights for policy 1, policy_version 366341 (0.0006) [2023-12-27 08:10:43,287][20381] Updated weights for policy 1, policy_version 366351 (0.0007) [2023-12-27 08:10:43,536][20291] KL-divergence is very high: 137.3616 [2023-12-27 08:10:43,553][20347] Updated weights for policy 0, policy_version 366698 (0.0005) [2023-12-27 08:10:43,609][20347] Updated weights for policy 0, policy_version 366708 (0.0005) [2023-12-27 08:10:43,626][20291] KL-divergence is very high: 224.2337 [2023-12-27 08:10:43,661][20347] Updated weights for policy 0, policy_version 366718 (0.0008) [2023-12-27 08:10:43,821][20381] Updated weights for policy 1, policy_version 366361 (0.0007) [2023-12-27 08:10:43,867][20381] Updated weights for policy 1, policy_version 366371 (0.0005) [2023-12-27 08:10:43,910][20381] Updated weights for policy 1, policy_version 366381 (0.0005) [2023-12-27 08:10:43,975][20381] Updated weights for policy 1, policy_version 366391 (0.0005) [2023-12-27 08:10:44,277][20347] Updated weights for policy 0, policy_version 366728 (0.0005) [2023-12-27 08:10:44,339][20347] Updated weights for policy 0, policy_version 366738 (0.0005) [2023-12-27 08:10:44,405][20347] Updated weights for policy 0, policy_version 366748 (0.0005) [2023-12-27 08:10:44,520][20381] Updated weights for policy 1, policy_version 366401 (0.0010) [2023-12-27 08:10:44,576][20381] Updated weights for policy 1, policy_version 366411 (0.0010) [2023-12-27 08:10:44,636][20381] Updated weights for policy 1, policy_version 366421 (0.0010) [2023-12-27 08:10:44,987][20347] Updated weights for policy 0, policy_version 366758 (0.0005) [2023-12-27 08:10:45,058][20347] Updated weights for policy 0, policy_version 366768 (0.0006) [2023-12-27 08:10:45,124][20347] Updated weights for policy 0, policy_version 366778 (0.0006) [2023-12-27 08:10:45,285][20381] Updated weights for policy 1, policy_version 366431 (0.0008) [2023-12-27 08:10:45,338][20381] Updated weights for policy 1, policy_version 366441 (0.0008) [2023-12-27 08:10:45,398][20381] Updated weights for policy 1, policy_version 366451 (0.0008) [2023-12-27 08:10:45,716][20347] Updated weights for policy 0, policy_version 366788 (0.0007) [2023-12-27 08:10:45,762][20347] Updated weights for policy 0, policy_version 366798 (0.0007) [2023-12-27 08:10:45,815][20347] Updated weights for policy 0, policy_version 366808 (0.0008) [2023-12-27 08:10:46,000][20381] Updated weights for policy 1, policy_version 366461 (0.0006) [2023-12-27 08:10:46,051][20381] Updated weights for policy 1, policy_version 366471 (0.0006) [2023-12-27 08:10:46,103][20381] Updated weights for policy 1, policy_version 366481 (0.0006) [2023-12-27 08:10:46,466][20347] Updated weights for policy 0, policy_version 366818 (0.0007) [2023-12-27 08:10:46,513][20347] Updated weights for policy 0, policy_version 366828 (0.0007) [2023-12-27 08:10:46,558][20347] Updated weights for policy 0, policy_version 366838 (0.0007) [2023-12-27 08:10:46,608][20347] Updated weights for policy 0, policy_version 366848 (0.0008) [2023-12-27 08:10:46,661][20381] Updated weights for policy 1, policy_version 366491 (0.0005) [2023-12-27 08:10:46,710][20381] Updated weights for policy 1, policy_version 366501 (0.0005) [2023-12-27 08:10:46,770][20381] Updated weights for policy 1, policy_version 366511 (0.0007) [2023-12-27 08:10:47,326][20347] Updated weights for policy 0, policy_version 366858 (0.0005) [2023-12-27 08:10:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 187777024. Throughput: 0: 10567.2, 1: 10507.1. Samples: 187792484. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:47,364][19170] Avg episode reward: [(0, '729.440'), (1, '411.650')] [2023-12-27 08:10:47,374][20347] Updated weights for policy 0, policy_version 366868 (0.0005) [2023-12-27 08:10:47,384][20381] Updated weights for policy 1, policy_version 366521 (0.0006) [2023-12-27 08:10:47,426][20347] Updated weights for policy 0, policy_version 366878 (0.0005) [2023-12-27 08:10:47,443][20381] Updated weights for policy 1, policy_version 366531 (0.0005) [2023-12-27 08:10:47,492][20381] Updated weights for policy 1, policy_version 366541 (0.0006) [2023-12-27 08:10:47,543][20381] Updated weights for policy 1, policy_version 366551 (0.0005) [2023-12-27 08:10:48,096][20347] Updated weights for policy 0, policy_version 366888 (0.0009) [2023-12-27 08:10:48,097][20381] Updated weights for policy 1, policy_version 366561 (0.0006) [2023-12-27 08:10:48,106][20291] KL-divergence is very high: 253.1435 [2023-12-27 08:10:48,112][20291] KL-divergence is very high: 254.0335 [2023-12-27 08:10:48,118][20291] KL-divergence is very high: 209.5381 [2023-12-27 08:10:48,123][20291] KL-divergence is very high: 153.2638 [2023-12-27 08:10:48,155][20381] Updated weights for policy 1, policy_version 366571 (0.0005) [2023-12-27 08:10:48,158][20347] Updated weights for policy 0, policy_version 366898 (0.0010) [2023-12-27 08:10:48,212][20347] Updated weights for policy 0, policy_version 366908 (0.0009) [2023-12-27 08:10:48,215][20381] Updated weights for policy 1, policy_version 366581 (0.0005) [2023-12-27 08:10:48,869][20381] Updated weights for policy 1, policy_version 366591 (0.0006) [2023-12-27 08:10:48,871][20347] Updated weights for policy 0, policy_version 366918 (0.0008) [2023-12-27 08:10:48,925][20381] Updated weights for policy 1, policy_version 366601 (0.0006) [2023-12-27 08:10:48,927][20347] Updated weights for policy 0, policy_version 366928 (0.0010) [2023-12-27 08:10:48,977][20347] Updated weights for policy 0, policy_version 366938 (0.0007) [2023-12-27 08:10:48,983][20381] Updated weights for policy 1, policy_version 366611 (0.0006) [2023-12-27 08:10:49,612][20347] Updated weights for policy 0, policy_version 366948 (0.0009) [2023-12-27 08:10:49,665][20381] Updated weights for policy 1, policy_version 366621 (0.0009) [2023-12-27 08:10:49,671][20347] Updated weights for policy 0, policy_version 366958 (0.0011) [2023-12-27 08:10:49,730][20381] Updated weights for policy 1, policy_version 366631 (0.0009) [2023-12-27 08:10:49,734][20347] Updated weights for policy 0, policy_version 366968 (0.0006) [2023-12-27 08:10:49,790][20381] Updated weights for policy 1, policy_version 366641 (0.0011) [2023-12-27 08:10:50,389][20347] Updated weights for policy 0, policy_version 366978 (0.0010) [2023-12-27 08:10:50,429][20381] Updated weights for policy 1, policy_version 366651 (0.0010) [2023-12-27 08:10:50,441][20347] Updated weights for policy 0, policy_version 366988 (0.0010) [2023-12-27 08:10:50,486][20347] Updated weights for policy 0, policy_version 366998 (0.0010) [2023-12-27 08:10:50,489][20381] Updated weights for policy 1, policy_version 366661 (0.0007) [2023-12-27 08:10:50,523][20291] KL-divergence is very high: 231.6832 [2023-12-27 08:10:50,534][20291] KL-divergence is very high: 181.3652 [2023-12-27 08:10:50,542][20347] Updated weights for policy 0, policy_version 367008 (0.0010) [2023-12-27 08:10:50,545][20381] Updated weights for policy 1, policy_version 366671 (0.0006) [2023-12-27 08:10:51,111][20381] Updated weights for policy 1, policy_version 366681 (0.0007) [2023-12-27 08:10:51,182][20381] Updated weights for policy 1, policy_version 366691 (0.0007) [2023-12-27 08:10:51,186][20291] KL-divergence is very high: 177.6580 [2023-12-27 08:10:51,193][20291] KL-divergence is very high: 124.7083 [2023-12-27 08:10:51,234][20291] KL-divergence is very high: 112.6072 [2023-12-27 08:10:51,243][20347] Updated weights for policy 0, policy_version 367018 (0.0007) [2023-12-27 08:10:51,250][20381] Updated weights for policy 1, policy_version 366701 (0.0007) [2023-12-27 08:10:51,268][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-27 08:10:51,282][20291] KL-divergence is very high: 116.4857 [2023-12-27 08:10:51,308][20347] Updated weights for policy 0, policy_version 367028 (0.0008) [2023-12-27 08:10:51,339][20291] KL-divergence is very high: 114.5353 [2023-12-27 08:10:51,379][20347] Updated weights for policy 0, policy_version 367038 (0.0009) [2023-12-27 08:10:52,001][20381] Updated weights for policy 1, policy_version 366711 (0.0006) [2023-12-27 08:10:52,059][20381] Updated weights for policy 1, policy_version 366721 (0.0006) [2023-12-27 08:10:52,066][20347] Updated weights for policy 0, policy_version 367048 (0.0009) [2023-12-27 08:10:52,115][20381] Updated weights for policy 1, policy_version 366731 (0.0007) [2023-12-27 08:10:52,127][20347] Updated weights for policy 0, policy_version 367058 (0.0009) [2023-12-27 08:10:52,183][20347] Updated weights for policy 0, policy_version 367068 (0.0008) [2023-12-27 08:10:52,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 187891712. Throughput: 0: 10573.3, 1: 10540.5. Samples: 187856100. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:52,363][19170] Avg episode reward: [(0, '635.640'), (1, '550.830')] [2023-12-27 08:10:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000366736_93904896.pth... [2023-12-27 08:10:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000367072_93986816.pth... [2023-12-27 08:10:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000365496_93585408.pth [2023-12-27 08:10:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000365824_93667328.pth [2023-12-27 08:10:52,812][20381] Updated weights for policy 1, policy_version 366741 (0.0008) [2023-12-27 08:10:52,819][20347] Updated weights for policy 0, policy_version 367078 (0.0007) [2023-12-27 08:10:52,869][20381] Updated weights for policy 1, policy_version 366751 (0.0011) [2023-12-27 08:10:52,879][20347] Updated weights for policy 0, policy_version 367088 (0.0007) [2023-12-27 08:10:52,929][20381] Updated weights for policy 1, policy_version 366761 (0.0011) [2023-12-27 08:10:52,939][20347] Updated weights for policy 0, policy_version 367098 (0.0006) [2023-12-27 08:10:53,478][20347] Updated weights for policy 0, policy_version 367108 (0.0006) [2023-12-27 08:10:53,527][20347] Updated weights for policy 0, policy_version 367118 (0.0007) [2023-12-27 08:10:53,549][20381] Updated weights for policy 1, policy_version 366771 (0.0010) [2023-12-27 08:10:53,576][20347] Updated weights for policy 0, policy_version 367128 (0.0006) [2023-12-27 08:10:53,601][20381] Updated weights for policy 1, policy_version 366781 (0.0010) [2023-12-27 08:10:53,657][20381] Updated weights for policy 1, policy_version 366791 (0.0010) [2023-12-27 08:10:54,272][20381] Updated weights for policy 1, policy_version 366801 (0.0010) [2023-12-27 08:10:54,309][20347] Updated weights for policy 0, policy_version 367138 (0.0005) [2023-12-27 08:10:54,329][20381] Updated weights for policy 1, policy_version 366811 (0.0010) [2023-12-27 08:10:54,361][20347] Updated weights for policy 0, policy_version 367148 (0.0005) [2023-12-27 08:10:54,381][20381] Updated weights for policy 1, policy_version 366821 (0.0010) [2023-12-27 08:10:54,406][20347] Updated weights for policy 0, policy_version 367158 (0.0005) [2023-12-27 08:10:54,437][20381] Updated weights for policy 1, policy_version 366831 (0.0010) [2023-12-27 08:10:54,453][20347] Updated weights for policy 0, policy_version 367168 (0.0007) [2023-12-27 08:10:55,105][20381] Updated weights for policy 1, policy_version 366841 (0.0010) [2023-12-27 08:10:55,167][20381] Updated weights for policy 1, policy_version 366851 (0.0011) [2023-12-27 08:10:55,169][20347] Updated weights for policy 0, policy_version 367178 (0.0006) [2023-12-27 08:10:55,216][20381] Updated weights for policy 1, policy_version 366861 (0.0010) [2023-12-27 08:10:55,217][20347] Updated weights for policy 0, policy_version 367188 (0.0006) [2023-12-27 08:10:55,263][20347] Updated weights for policy 0, policy_version 367198 (0.0006) [2023-12-27 08:10:55,904][20381] Updated weights for policy 1, policy_version 366871 (0.0007) [2023-12-27 08:10:55,949][20347] Updated weights for policy 0, policy_version 367208 (0.0008) [2023-12-27 08:10:55,954][20381] Updated weights for policy 1, policy_version 366881 (0.0007) [2023-12-27 08:10:55,973][20291] KL-divergence is very high: 190.6629 [2023-12-27 08:10:55,997][20347] Updated weights for policy 0, policy_version 367218 (0.0006) [2023-12-27 08:10:56,006][20381] Updated weights for policy 1, policy_version 366891 (0.0010) [2023-12-27 08:10:56,010][20291] KL-divergence is very high: 120.5750 [2023-12-27 08:10:56,044][20347] Updated weights for policy 0, policy_version 367228 (0.0007) [2023-12-27 08:10:56,627][20381] Updated weights for policy 1, policy_version 366901 (0.0010) [2023-12-27 08:10:56,654][20347] Updated weights for policy 0, policy_version 367238 (0.0007) [2023-12-27 08:10:56,672][20381] Updated weights for policy 1, policy_version 366911 (0.0010) [2023-12-27 08:10:56,701][20347] Updated weights for policy 0, policy_version 367248 (0.0007) [2023-12-27 08:10:56,717][20381] Updated weights for policy 1, policy_version 366921 (0.0010) [2023-12-27 08:10:56,746][20347] Updated weights for policy 0, policy_version 367258 (0.0007) [2023-12-27 08:10:57,318][20381] Updated weights for policy 1, policy_version 366931 (0.0011) [2023-12-27 08:10:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 187990016. Throughput: 0: 10593.8, 1: 10631.7. Samples: 187982060. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:10:57,363][19170] Avg episode reward: [(0, '561.580'), (1, '495.700')] [2023-12-27 08:10:57,382][20381] Updated weights for policy 1, policy_version 366941 (0.0009) [2023-12-27 08:10:57,443][20381] Updated weights for policy 1, policy_version 366951 (0.0007) [2023-12-27 08:10:57,458][20347] Updated weights for policy 0, policy_version 367268 (0.0007) [2023-12-27 08:10:57,506][20347] Updated weights for policy 0, policy_version 367278 (0.0006) [2023-12-27 08:10:57,551][20347] Updated weights for policy 0, policy_version 367288 (0.0008) [2023-12-27 08:10:58,060][20381] Updated weights for policy 1, policy_version 366961 (0.0007) [2023-12-27 08:10:58,123][20381] Updated weights for policy 1, policy_version 366971 (0.0005) [2023-12-27 08:10:58,185][20381] Updated weights for policy 1, policy_version 366981 (0.0007) [2023-12-27 08:10:58,210][20347] Updated weights for policy 0, policy_version 367298 (0.0008) [2023-12-27 08:10:58,250][20381] Updated weights for policy 1, policy_version 366991 (0.0008) [2023-12-27 08:10:58,273][20347] Updated weights for policy 0, policy_version 367308 (0.0007) [2023-12-27 08:10:58,333][20347] Updated weights for policy 0, policy_version 367318 (0.0009) [2023-12-27 08:10:58,400][20347] Updated weights for policy 0, policy_version 367328 (0.0007) [2023-12-27 08:10:58,933][20381] Updated weights for policy 1, policy_version 367001 (0.0007) [2023-12-27 08:10:58,989][20381] Updated weights for policy 1, policy_version 367011 (0.0006) [2023-12-27 08:10:59,051][20381] Updated weights for policy 1, policy_version 367021 (0.0006) [2023-12-27 08:10:59,261][20291] KL-divergence is very high: 241.3921 [2023-12-27 08:10:59,268][20291] KL-divergence is very high: 269.3852 [2023-12-27 08:10:59,274][20347] Updated weights for policy 0, policy_version 367338 (0.0008) [2023-12-27 08:10:59,279][20291] KL-divergence is very high: 186.1956 [2023-12-27 08:10:59,329][20347] Updated weights for policy 0, policy_version 367348 (0.0010) [2023-12-27 08:10:59,390][20347] Updated weights for policy 0, policy_version 367358 (0.0009) [2023-12-27 08:10:59,728][20381] Updated weights for policy 1, policy_version 367031 (0.0008) [2023-12-27 08:10:59,777][20381] Updated weights for policy 1, policy_version 367041 (0.0007) [2023-12-27 08:10:59,830][20381] Updated weights for policy 1, policy_version 367051 (0.0007) [2023-12-27 08:11:00,050][20347] Updated weights for policy 0, policy_version 367368 (0.0011) [2023-12-27 08:11:00,103][20347] Updated weights for policy 0, policy_version 367378 (0.0010) [2023-12-27 08:11:00,163][20347] Updated weights for policy 0, policy_version 367388 (0.0010) [2023-12-27 08:11:00,533][20381] Updated weights for policy 1, policy_version 367061 (0.0007) [2023-12-27 08:11:00,579][20381] Updated weights for policy 1, policy_version 367071 (0.0006) [2023-12-27 08:11:00,630][20381] Updated weights for policy 1, policy_version 367081 (0.0007) [2023-12-27 08:11:00,816][20347] Updated weights for policy 0, policy_version 367398 (0.0010) [2023-12-27 08:11:00,868][20347] Updated weights for policy 0, policy_version 367408 (0.0010) [2023-12-27 08:11:00,923][20347] Updated weights for policy 0, policy_version 367418 (0.0011) [2023-12-27 08:11:01,286][20381] Updated weights for policy 1, policy_version 367091 (0.0007) [2023-12-27 08:11:01,344][20381] Updated weights for policy 1, policy_version 367101 (0.0006) [2023-12-27 08:11:01,401][20381] Updated weights for policy 1, policy_version 367111 (0.0007) [2023-12-27 08:11:01,693][20347] Updated weights for policy 0, policy_version 367428 (0.0011) [2023-12-27 08:11:01,753][20347] Updated weights for policy 0, policy_version 367438 (0.0010) [2023-12-27 08:11:01,807][20347] Updated weights for policy 0, policy_version 367448 (0.0009) [2023-12-27 08:11:02,015][20381] Updated weights for policy 1, policy_version 367121 (0.0005) [2023-12-27 08:11:02,066][20381] Updated weights for policy 1, policy_version 367131 (0.0005) [2023-12-27 08:11:02,119][20381] Updated weights for policy 1, policy_version 367141 (0.0007) [2023-12-27 08:11:02,176][20381] Updated weights for policy 1, policy_version 367151 (0.0006) [2023-12-27 08:11:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 188096512. Throughput: 0: 10513.2, 1: 10662.8. Samples: 188106588. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:02,363][19170] Avg episode reward: [(0, '604.610'), (1, '506.140')] [2023-12-27 08:11:02,518][20347] Updated weights for policy 0, policy_version 367458 (0.0010) [2023-12-27 08:11:02,571][20347] Updated weights for policy 0, policy_version 367468 (0.0010) [2023-12-27 08:11:02,622][20347] Updated weights for policy 0, policy_version 367478 (0.0010) [2023-12-27 08:11:02,681][20347] Updated weights for policy 0, policy_version 367488 (0.0009) [2023-12-27 08:11:02,777][20381] Updated weights for policy 1, policy_version 367161 (0.0010) [2023-12-27 08:11:02,838][20381] Updated weights for policy 1, policy_version 367171 (0.0010) [2023-12-27 08:11:02,884][20381] Updated weights for policy 1, policy_version 367181 (0.0010) [2023-12-27 08:11:03,295][20347] Updated weights for policy 0, policy_version 367498 (0.0010) [2023-12-27 08:11:03,333][20291] KL-divergence is very high: 146.6371 [2023-12-27 08:11:03,343][20347] Updated weights for policy 0, policy_version 367508 (0.0010) [2023-12-27 08:11:03,392][20347] Updated weights for policy 0, policy_version 367518 (0.0010) [2023-12-27 08:11:03,534][20381] Updated weights for policy 1, policy_version 367191 (0.0008) [2023-12-27 08:11:03,582][20381] Updated weights for policy 1, policy_version 367201 (0.0008) [2023-12-27 08:11:03,632][20381] Updated weights for policy 1, policy_version 367211 (0.0007) [2023-12-27 08:11:04,046][20347] Updated weights for policy 0, policy_version 367528 (0.0010) [2023-12-27 08:11:04,105][20347] Updated weights for policy 0, policy_version 367538 (0.0011) [2023-12-27 08:11:04,162][20347] Updated weights for policy 0, policy_version 367548 (0.0009) [2023-12-27 08:11:04,298][20381] Updated weights for policy 1, policy_version 367221 (0.0007) [2023-12-27 08:11:04,353][20381] Updated weights for policy 1, policy_version 367231 (0.0008) [2023-12-27 08:11:04,412][20381] Updated weights for policy 1, policy_version 367241 (0.0008) [2023-12-27 08:11:04,800][20347] Updated weights for policy 0, policy_version 367558 (0.0010) [2023-12-27 08:11:04,855][20347] Updated weights for policy 0, policy_version 367568 (0.0010) [2023-12-27 08:11:04,906][20347] Updated weights for policy 0, policy_version 367578 (0.0010) [2023-12-27 08:11:05,088][20381] Updated weights for policy 1, policy_version 367251 (0.0008) [2023-12-27 08:11:05,154][20381] Updated weights for policy 1, policy_version 367261 (0.0006) [2023-12-27 08:11:05,206][20381] Updated weights for policy 1, policy_version 367271 (0.0009) [2023-12-27 08:11:05,530][20347] Updated weights for policy 0, policy_version 367588 (0.0010) [2023-12-27 08:11:05,586][20347] Updated weights for policy 0, policy_version 367598 (0.0010) [2023-12-27 08:11:05,644][20347] Updated weights for policy 0, policy_version 367608 (0.0009) [2023-12-27 08:11:05,888][20381] Updated weights for policy 1, policy_version 367281 (0.0009) [2023-12-27 08:11:05,944][20381] Updated weights for policy 1, policy_version 367291 (0.0008) [2023-12-27 08:11:05,999][20381] Updated weights for policy 1, policy_version 367301 (0.0008) [2023-12-27 08:11:06,047][20381] Updated weights for policy 1, policy_version 367311 (0.0008) [2023-12-27 08:11:06,330][20347] Updated weights for policy 0, policy_version 367618 (0.0008) [2023-12-27 08:11:06,386][20347] Updated weights for policy 0, policy_version 367628 (0.0007) [2023-12-27 08:11:06,444][20347] Updated weights for policy 0, policy_version 367638 (0.0008) [2023-12-27 08:11:06,502][20347] Updated weights for policy 0, policy_version 367648 (0.0008) [2023-12-27 08:11:06,748][20381] Updated weights for policy 1, policy_version 367321 (0.0007) [2023-12-27 08:11:06,811][20381] Updated weights for policy 1, policy_version 367331 (0.0007) [2023-12-27 08:11:06,866][20381] Updated weights for policy 1, policy_version 367341 (0.0008) [2023-12-27 08:11:07,189][20347] Updated weights for policy 0, policy_version 367658 (0.0010) [2023-12-27 08:11:07,251][20347] Updated weights for policy 0, policy_version 367668 (0.0007) [2023-12-27 08:11:07,316][20347] Updated weights for policy 0, policy_version 367678 (0.0010) [2023-12-27 08:11:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21162.7, 300 sec: 20882.7). Total num frames: 188203008. Throughput: 0: 10508.7, 1: 10676.3. Samples: 188170676. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:07,363][19170] Avg episode reward: [(0, '706.300'), (1, '593.570')] [2023-12-27 08:11:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000367344_94060544.pth... [2023-12-27 08:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000367680_94142464.pth... [2023-12-27 08:11:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000366464_93831168.pth [2023-12-27 08:11:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000366136_93749248.pth [2023-12-27 08:11:07,506][20381] Updated weights for policy 1, policy_version 367351 (0.0008) [2023-12-27 08:11:07,559][20381] Updated weights for policy 1, policy_version 367361 (0.0007) [2023-12-27 08:11:07,618][20381] Updated weights for policy 1, policy_version 367371 (0.0007) [2023-12-27 08:11:07,951][20347] Updated weights for policy 0, policy_version 367688 (0.0006) [2023-12-27 08:11:07,997][20347] Updated weights for policy 0, policy_version 367698 (0.0005) [2023-12-27 08:11:08,049][20347] Updated weights for policy 0, policy_version 367708 (0.0007) [2023-12-27 08:11:08,350][20381] Updated weights for policy 1, policy_version 367381 (0.0008) [2023-12-27 08:11:08,407][20381] Updated weights for policy 1, policy_version 367391 (0.0008) [2023-12-27 08:11:08,471][20381] Updated weights for policy 1, policy_version 367401 (0.0008) [2023-12-27 08:11:08,600][20347] Updated weights for policy 0, policy_version 367718 (0.0006) [2023-12-27 08:11:08,649][20347] Updated weights for policy 0, policy_version 367728 (0.0008) [2023-12-27 08:11:08,694][20347] Updated weights for policy 0, policy_version 367738 (0.0007) [2023-12-27 08:11:09,178][20381] Updated weights for policy 1, policy_version 367411 (0.0007) [2023-12-27 08:11:09,237][20381] Updated weights for policy 1, policy_version 367421 (0.0008) [2023-12-27 08:11:09,292][20381] Updated weights for policy 1, policy_version 367431 (0.0008) [2023-12-27 08:11:09,430][20347] Updated weights for policy 0, policy_version 367748 (0.0008) [2023-12-27 08:11:09,492][20347] Updated weights for policy 0, policy_version 367758 (0.0008) [2023-12-27 08:11:09,557][20347] Updated weights for policy 0, policy_version 367768 (0.0007) [2023-12-27 08:11:09,967][20381] Updated weights for policy 1, policy_version 367441 (0.0007) [2023-12-27 08:11:10,025][20381] Updated weights for policy 1, policy_version 367451 (0.0008) [2023-12-27 08:11:10,076][20381] Updated weights for policy 1, policy_version 367461 (0.0007) [2023-12-27 08:11:10,130][20381] Updated weights for policy 1, policy_version 367471 (0.0008) [2023-12-27 08:11:10,282][20347] Updated weights for policy 0, policy_version 367778 (0.0006) [2023-12-27 08:11:10,348][20347] Updated weights for policy 0, policy_version 367788 (0.0008) [2023-12-27 08:11:10,409][20347] Updated weights for policy 0, policy_version 367798 (0.0008) [2023-12-27 08:11:10,466][20347] Updated weights for policy 0, policy_version 367808 (0.0008) [2023-12-27 08:11:10,805][20381] Updated weights for policy 1, policy_version 367481 (0.0007) [2023-12-27 08:11:10,860][20381] Updated weights for policy 1, policy_version 367491 (0.0007) [2023-12-27 08:11:10,915][20381] Updated weights for policy 1, policy_version 367501 (0.0006) [2023-12-27 08:11:11,187][20347] Updated weights for policy 0, policy_version 367818 (0.0006) [2023-12-27 08:11:11,248][20347] Updated weights for policy 0, policy_version 367828 (0.0009) [2023-12-27 08:11:11,300][20347] Updated weights for policy 0, policy_version 367838 (0.0008) [2023-12-27 08:11:11,586][20381] Updated weights for policy 1, policy_version 367511 (0.0007) [2023-12-27 08:11:11,652][20381] Updated weights for policy 1, policy_version 367521 (0.0008) [2023-12-27 08:11:11,717][20381] Updated weights for policy 1, policy_version 367531 (0.0006) [2023-12-27 08:11:12,021][20347] Updated weights for policy 0, policy_version 367848 (0.0007) [2023-12-27 08:11:12,080][20347] Updated weights for policy 0, policy_version 367858 (0.0008) [2023-12-27 08:11:12,137][20347] Updated weights for policy 0, policy_version 367868 (0.0007) [2023-12-27 08:11:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 188301312. Throughput: 0: 10469.2, 1: 10633.0. Samples: 188293076. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:12,363][19170] Avg episode reward: [(0, '798.530'), (1, '630.420')] [2023-12-27 08:11:12,393][20381] Updated weights for policy 1, policy_version 367541 (0.0007) [2023-12-27 08:11:12,456][20381] Updated weights for policy 1, policy_version 367551 (0.0006) [2023-12-27 08:11:12,522][20381] Updated weights for policy 1, policy_version 367561 (0.0006) [2023-12-27 08:11:12,871][20347] Updated weights for policy 0, policy_version 367878 (0.0008) [2023-12-27 08:11:12,933][20347] Updated weights for policy 0, policy_version 367888 (0.0008) [2023-12-27 08:11:12,985][20347] Updated weights for policy 0, policy_version 367898 (0.0008) [2023-12-27 08:11:13,129][20381] Updated weights for policy 1, policy_version 367571 (0.0006) [2023-12-27 08:11:13,185][20381] Updated weights for policy 1, policy_version 367581 (0.0006) [2023-12-27 08:11:13,236][20381] Updated weights for policy 1, policy_version 367591 (0.0008) [2023-12-27 08:11:13,687][20347] Updated weights for policy 0, policy_version 367908 (0.0008) [2023-12-27 08:11:13,741][20347] Updated weights for policy 0, policy_version 367918 (0.0008) [2023-12-27 08:11:13,797][20347] Updated weights for policy 0, policy_version 367928 (0.0008) [2023-12-27 08:11:13,884][20381] Updated weights for policy 1, policy_version 367601 (0.0007) [2023-12-27 08:11:13,929][20381] Updated weights for policy 1, policy_version 367611 (0.0005) [2023-12-27 08:11:13,979][20381] Updated weights for policy 1, policy_version 367621 (0.0005) [2023-12-27 08:11:14,026][20381] Updated weights for policy 1, policy_version 367631 (0.0006) [2023-12-27 08:11:14,500][20347] Updated weights for policy 0, policy_version 367938 (0.0007) [2023-12-27 08:11:14,557][20347] Updated weights for policy 0, policy_version 367948 (0.0005) [2023-12-27 08:11:14,611][20347] Updated weights for policy 0, policy_version 367958 (0.0008) [2023-12-27 08:11:14,654][20381] Updated weights for policy 1, policy_version 367641 (0.0006) [2023-12-27 08:11:14,663][20347] Updated weights for policy 0, policy_version 367968 (0.0007) [2023-12-27 08:11:14,712][20381] Updated weights for policy 1, policy_version 367651 (0.0007) [2023-12-27 08:11:14,772][20381] Updated weights for policy 1, policy_version 367661 (0.0008) [2023-12-27 08:11:15,321][20347] Updated weights for policy 0, policy_version 367978 (0.0008) [2023-12-27 08:11:15,383][20347] Updated weights for policy 0, policy_version 367988 (0.0008) [2023-12-27 08:11:15,436][20347] Updated weights for policy 0, policy_version 367998 (0.0007) [2023-12-27 08:11:15,482][20381] Updated weights for policy 1, policy_version 367671 (0.0008) [2023-12-27 08:11:15,535][20381] Updated weights for policy 1, policy_version 367681 (0.0008) [2023-12-27 08:11:15,598][20381] Updated weights for policy 1, policy_version 367691 (0.0009) [2023-12-27 08:11:16,155][20347] Updated weights for policy 0, policy_version 368008 (0.0007) [2023-12-27 08:11:16,218][20347] Updated weights for policy 0, policy_version 368018 (0.0006) [2023-12-27 08:11:16,224][20381] Updated weights for policy 1, policy_version 367701 (0.0006) [2023-12-27 08:11:16,273][20347] Updated weights for policy 0, policy_version 368028 (0.0006) [2023-12-27 08:11:16,279][20381] Updated weights for policy 1, policy_version 367711 (0.0005) [2023-12-27 08:11:16,327][20381] Updated weights for policy 1, policy_version 367721 (0.0005) [2023-12-27 08:11:16,912][20381] Updated weights for policy 1, policy_version 367731 (0.0005) [2023-12-27 08:11:16,954][20347] Updated weights for policy 0, policy_version 368038 (0.0007) [2023-12-27 08:11:16,958][20381] Updated weights for policy 1, policy_version 367741 (0.0005) [2023-12-27 08:11:17,000][20347] Updated weights for policy 0, policy_version 368048 (0.0007) [2023-12-27 08:11:17,006][20381] Updated weights for policy 1, policy_version 367751 (0.0005) [2023-12-27 08:11:17,047][20347] Updated weights for policy 0, policy_version 368058 (0.0008) [2023-12-27 08:11:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21162.7, 300 sec: 20910.4). Total num frames: 188407808. Throughput: 0: 10394.7, 1: 10627.3. Samples: 188417128. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:17,363][19170] Avg episode reward: [(0, '922.670'), (1, '725.300')] [2023-12-27 08:11:17,514][20381] Updated weights for policy 1, policy_version 367761 (0.0005) [2023-12-27 08:11:17,573][20381] Updated weights for policy 1, policy_version 367771 (0.0005) [2023-12-27 08:11:17,632][20381] Updated weights for policy 1, policy_version 367781 (0.0005) [2023-12-27 08:11:17,695][20381] Updated weights for policy 1, policy_version 367791 (0.0005) [2023-12-27 08:11:17,821][20347] Updated weights for policy 0, policy_version 368068 (0.0007) [2023-12-27 08:11:17,874][20347] Updated weights for policy 0, policy_version 368078 (0.0007) [2023-12-27 08:11:17,931][20347] Updated weights for policy 0, policy_version 368088 (0.0007) [2023-12-27 08:11:18,216][20381] Updated weights for policy 1, policy_version 367801 (0.0009) [2023-12-27 08:11:18,276][20381] Updated weights for policy 1, policy_version 367811 (0.0010) [2023-12-27 08:11:18,336][20381] Updated weights for policy 1, policy_version 367821 (0.0011) [2023-12-27 08:11:18,650][20347] Updated weights for policy 0, policy_version 368098 (0.0007) [2023-12-27 08:11:18,707][20347] Updated weights for policy 0, policy_version 368108 (0.0007) [2023-12-27 08:11:18,765][20347] Updated weights for policy 0, policy_version 368118 (0.0007) [2023-12-27 08:11:18,813][20347] Updated weights for policy 0, policy_version 368128 (0.0007) [2023-12-27 08:11:19,014][20381] Updated weights for policy 1, policy_version 367831 (0.0008) [2023-12-27 08:11:19,072][20381] Updated weights for policy 1, policy_version 367841 (0.0006) [2023-12-27 08:11:19,133][20381] Updated weights for policy 1, policy_version 367851 (0.0005) [2023-12-27 08:11:19,547][20347] Updated weights for policy 0, policy_version 368138 (0.0008) [2023-12-27 08:11:19,602][20347] Updated weights for policy 0, policy_version 368148 (0.0007) [2023-12-27 08:11:19,663][20347] Updated weights for policy 0, policy_version 368158 (0.0007) [2023-12-27 08:11:19,787][20381] Updated weights for policy 1, policy_version 367861 (0.0006) [2023-12-27 08:11:19,849][20381] Updated weights for policy 1, policy_version 367871 (0.0007) [2023-12-27 08:11:19,913][20381] Updated weights for policy 1, policy_version 367881 (0.0007) [2023-12-27 08:11:20,391][20347] Updated weights for policy 0, policy_version 368168 (0.0008) [2023-12-27 08:11:20,450][20347] Updated weights for policy 0, policy_version 368178 (0.0008) [2023-12-27 08:11:20,509][20347] Updated weights for policy 0, policy_version 368188 (0.0007) [2023-12-27 08:11:20,598][20381] Updated weights for policy 1, policy_version 367891 (0.0007) [2023-12-27 08:11:20,658][20381] Updated weights for policy 1, policy_version 367901 (0.0007) [2023-12-27 08:11:20,722][20381] Updated weights for policy 1, policy_version 367911 (0.0006) [2023-12-27 08:11:21,229][20347] Updated weights for policy 0, policy_version 368198 (0.0008) [2023-12-27 08:11:21,290][20347] Updated weights for policy 0, policy_version 368208 (0.0007) [2023-12-27 08:11:21,349][20347] Updated weights for policy 0, policy_version 368218 (0.0009) [2023-12-27 08:11:21,400][20381] Updated weights for policy 1, policy_version 367921 (0.0006) [2023-12-27 08:11:21,470][20381] Updated weights for policy 1, policy_version 367931 (0.0008) [2023-12-27 08:11:21,531][20381] Updated weights for policy 1, policy_version 367941 (0.0007) [2023-12-27 08:11:21,591][20381] Updated weights for policy 1, policy_version 367951 (0.0008) [2023-12-27 08:11:22,067][20347] Updated weights for policy 0, policy_version 368228 (0.0008) [2023-12-27 08:11:22,127][20347] Updated weights for policy 0, policy_version 368238 (0.0007) [2023-12-27 08:11:22,186][20347] Updated weights for policy 0, policy_version 368248 (0.0006) [2023-12-27 08:11:22,316][20381] Updated weights for policy 1, policy_version 367961 (0.0008) [2023-12-27 08:11:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 188506112. Throughput: 0: 10284.7, 1: 10661.4. Samples: 188478752. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:22,363][19170] Avg episode reward: [(0, '971.060'), (1, '699.370')] [2023-12-27 08:11:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000368256_94289920.pth... [2023-12-27 08:11:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000367072_93986816.pth [2023-12-27 08:11:22,381][20381] Updated weights for policy 1, policy_version 367971 (0.0008) [2023-12-27 08:11:22,441][20381] Updated weights for policy 1, policy_version 367981 (0.0010) [2023-12-27 08:11:22,455][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000367984_94224384.pth... [2023-12-27 08:11:22,458][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000366736_93904896.pth [2023-12-27 08:11:22,894][20347] Updated weights for policy 0, policy_version 368258 (0.0007) [2023-12-27 08:11:22,950][20347] Updated weights for policy 0, policy_version 368268 (0.0008) [2023-12-27 08:11:22,998][20347] Updated weights for policy 0, policy_version 368278 (0.0009) [2023-12-27 08:11:23,053][20347] Updated weights for policy 0, policy_version 368288 (0.0008) [2023-12-27 08:11:23,076][20381] Updated weights for policy 1, policy_version 367991 (0.0007) [2023-12-27 08:11:23,130][20381] Updated weights for policy 1, policy_version 368001 (0.0007) [2023-12-27 08:11:23,188][20381] Updated weights for policy 1, policy_version 368011 (0.0006) [2023-12-27 08:11:23,709][20347] Updated weights for policy 0, policy_version 368298 (0.0005) [2023-12-27 08:11:23,763][20347] Updated weights for policy 0, policy_version 368308 (0.0005) [2023-12-27 08:11:23,792][20381] Updated weights for policy 1, policy_version 368021 (0.0006) [2023-12-27 08:11:23,810][20347] Updated weights for policy 0, policy_version 368318 (0.0007) [2023-12-27 08:11:23,853][20381] Updated weights for policy 1, policy_version 368031 (0.0005) [2023-12-27 08:11:23,903][20381] Updated weights for policy 1, policy_version 368041 (0.0005) [2023-12-27 08:11:24,419][20347] Updated weights for policy 0, policy_version 368328 (0.0006) [2023-12-27 08:11:24,471][20381] Updated weights for policy 1, policy_version 368051 (0.0005) [2023-12-27 08:11:24,473][20347] Updated weights for policy 0, policy_version 368338 (0.0006) [2023-12-27 08:11:24,524][20381] Updated weights for policy 1, policy_version 368061 (0.0007) [2023-12-27 08:11:24,532][20347] Updated weights for policy 0, policy_version 368348 (0.0008) [2023-12-27 08:11:24,602][20381] Updated weights for policy 1, policy_version 368071 (0.0008) [2023-12-27 08:11:25,206][20347] Updated weights for policy 0, policy_version 368358 (0.0007) [2023-12-27 08:11:25,258][20381] Updated weights for policy 1, policy_version 368081 (0.0008) [2023-12-27 08:11:25,271][20347] Updated weights for policy 0, policy_version 368368 (0.0005) [2023-12-27 08:11:25,313][20381] Updated weights for policy 1, policy_version 368091 (0.0007) [2023-12-27 08:11:25,332][20347] Updated weights for policy 0, policy_version 368378 (0.0005) [2023-12-27 08:11:25,372][20381] Updated weights for policy 1, policy_version 368101 (0.0005) [2023-12-27 08:11:25,429][20381] Updated weights for policy 1, policy_version 368111 (0.0005) [2023-12-27 08:11:25,956][20381] Updated weights for policy 1, policy_version 368121 (0.0005) [2023-12-27 08:11:25,964][20347] Updated weights for policy 0, policy_version 368388 (0.0006) [2023-12-27 08:11:26,014][20381] Updated weights for policy 1, policy_version 368131 (0.0005) [2023-12-27 08:11:26,018][20347] Updated weights for policy 0, policy_version 368398 (0.0006) [2023-12-27 08:11:26,064][20347] Updated weights for policy 0, policy_version 368408 (0.0007) [2023-12-27 08:11:26,068][20381] Updated weights for policy 1, policy_version 368141 (0.0005) [2023-12-27 08:11:26,645][20381] Updated weights for policy 1, policy_version 368151 (0.0006) [2023-12-27 08:11:26,700][20381] Updated weights for policy 1, policy_version 368161 (0.0008) [2023-12-27 08:11:26,748][20347] Updated weights for policy 0, policy_version 368418 (0.0005) [2023-12-27 08:11:26,754][20381] Updated weights for policy 1, policy_version 368171 (0.0007) [2023-12-27 08:11:26,797][20347] Updated weights for policy 0, policy_version 368428 (0.0006) [2023-12-27 08:11:26,855][20347] Updated weights for policy 0, policy_version 368438 (0.0008) [2023-12-27 08:11:26,909][20347] Updated weights for policy 0, policy_version 368448 (0.0008) [2023-12-27 08:11:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 188612608. Throughput: 0: 10246.7, 1: 10701.6. Samples: 188604428. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:27,363][19170] Avg episode reward: [(0, '971.700'), (1, '767.210')] [2023-12-27 08:11:27,369][20381] Updated weights for policy 1, policy_version 368181 (0.0005) [2023-12-27 08:11:27,422][20381] Updated weights for policy 1, policy_version 368191 (0.0005) [2023-12-27 08:11:27,467][20381] Updated weights for policy 1, policy_version 368201 (0.0005) [2023-12-27 08:11:27,635][20347] Updated weights for policy 0, policy_version 368458 (0.0007) [2023-12-27 08:11:27,680][20347] Updated weights for policy 0, policy_version 368468 (0.0009) [2023-12-27 08:11:27,729][20347] Updated weights for policy 0, policy_version 368478 (0.0008) [2023-12-27 08:11:28,041][20381] Updated weights for policy 1, policy_version 368211 (0.0006) [2023-12-27 08:11:28,098][20381] Updated weights for policy 1, policy_version 368221 (0.0008) [2023-12-27 08:11:28,144][20381] Updated weights for policy 1, policy_version 368231 (0.0008) [2023-12-27 08:11:28,444][20347] Updated weights for policy 0, policy_version 368488 (0.0006) [2023-12-27 08:11:28,505][20347] Updated weights for policy 0, policy_version 368498 (0.0006) [2023-12-27 08:11:28,560][20347] Updated weights for policy 0, policy_version 368508 (0.0008) [2023-12-27 08:11:28,747][20381] Updated weights for policy 1, policy_version 368241 (0.0007) [2023-12-27 08:11:28,807][20381] Updated weights for policy 1, policy_version 368251 (0.0011) [2023-12-27 08:11:28,869][20381] Updated weights for policy 1, policy_version 368261 (0.0011) [2023-12-27 08:11:28,928][20381] Updated weights for policy 1, policy_version 368271 (0.0011) [2023-12-27 08:11:29,265][20347] Updated weights for policy 0, policy_version 368518 (0.0008) [2023-12-27 08:11:29,323][20347] Updated weights for policy 0, policy_version 368528 (0.0008) [2023-12-27 08:11:29,378][20347] Updated weights for policy 0, policy_version 368538 (0.0008) [2023-12-27 08:11:29,648][20381] Updated weights for policy 1, policy_version 368281 (0.0011) [2023-12-27 08:11:29,701][20381] Updated weights for policy 1, policy_version 368291 (0.0010) [2023-12-27 08:11:29,750][20381] Updated weights for policy 1, policy_version 368301 (0.0011) [2023-12-27 08:11:30,047][20347] Updated weights for policy 0, policy_version 368548 (0.0008) [2023-12-27 08:11:30,103][20347] Updated weights for policy 0, policy_version 368558 (0.0007) [2023-12-27 08:11:30,158][20347] Updated weights for policy 0, policy_version 368568 (0.0007) [2023-12-27 08:11:30,468][20381] Updated weights for policy 1, policy_version 368311 (0.0011) [2023-12-27 08:11:30,525][20381] Updated weights for policy 1, policy_version 368321 (0.0010) [2023-12-27 08:11:30,582][20381] Updated weights for policy 1, policy_version 368331 (0.0010) [2023-12-27 08:11:30,834][20347] Updated weights for policy 0, policy_version 368578 (0.0008) [2023-12-27 08:11:30,891][20347] Updated weights for policy 0, policy_version 368588 (0.0010) [2023-12-27 08:11:30,941][20347] Updated weights for policy 0, policy_version 368598 (0.0009) [2023-12-27 08:11:30,999][20347] Updated weights for policy 0, policy_version 368608 (0.0010) [2023-12-27 08:11:31,286][20381] Updated weights for policy 1, policy_version 368341 (0.0010) [2023-12-27 08:11:31,343][20381] Updated weights for policy 1, policy_version 368351 (0.0007) [2023-12-27 08:11:31,410][20381] Updated weights for policy 1, policy_version 368361 (0.0007) [2023-12-27 08:11:31,694][20347] Updated weights for policy 0, policy_version 368618 (0.0010) [2023-12-27 08:11:31,757][20347] Updated weights for policy 0, policy_version 368628 (0.0009) [2023-12-27 08:11:31,814][20347] Updated weights for policy 0, policy_version 368638 (0.0009) [2023-12-27 08:11:32,141][20381] Updated weights for policy 1, policy_version 368371 (0.0007) [2023-12-27 08:11:32,195][20381] Updated weights for policy 1, policy_version 368381 (0.0007) [2023-12-27 08:11:32,244][20381] Updated weights for policy 1, policy_version 368391 (0.0008) [2023-12-27 08:11:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 188719104. Throughput: 0: 10155.6, 1: 10627.2. Samples: 188727708. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:32,363][19170] Avg episode reward: [(0, '981.300'), (1, '816.620')] [2023-12-27 08:11:32,510][20347] Updated weights for policy 0, policy_version 368648 (0.0010) [2023-12-27 08:11:32,568][20347] Updated weights for policy 0, policy_version 368658 (0.0010) [2023-12-27 08:11:32,623][20347] Updated weights for policy 0, policy_version 368668 (0.0010) [2023-12-27 08:11:32,918][20381] Updated weights for policy 1, policy_version 368401 (0.0008) [2023-12-27 08:11:32,973][20381] Updated weights for policy 1, policy_version 368411 (0.0005) [2023-12-27 08:11:33,017][20381] Updated weights for policy 1, policy_version 368421 (0.0005) [2023-12-27 08:11:33,072][20381] Updated weights for policy 1, policy_version 368431 (0.0007) [2023-12-27 08:11:33,289][20347] Updated weights for policy 0, policy_version 368678 (0.0009) [2023-12-27 08:11:33,340][20347] Updated weights for policy 0, policy_version 368688 (0.0010) [2023-12-27 08:11:33,399][20347] Updated weights for policy 0, policy_version 368698 (0.0008) [2023-12-27 08:11:33,647][20381] Updated weights for policy 1, policy_version 368441 (0.0006) [2023-12-27 08:11:33,705][20381] Updated weights for policy 1, policy_version 368451 (0.0005) [2023-12-27 08:11:33,759][20381] Updated weights for policy 1, policy_version 368461 (0.0005) [2023-12-27 08:11:34,146][20347] Updated weights for policy 0, policy_version 368708 (0.0009) [2023-12-27 08:11:34,201][20347] Updated weights for policy 0, policy_version 368718 (0.0010) [2023-12-27 08:11:34,260][20347] Updated weights for policy 0, policy_version 368728 (0.0010) [2023-12-27 08:11:34,342][20381] Updated weights for policy 1, policy_version 368471 (0.0009) [2023-12-27 08:11:34,410][20381] Updated weights for policy 1, policy_version 368481 (0.0010) [2023-12-27 08:11:34,472][20381] Updated weights for policy 1, policy_version 368491 (0.0011) [2023-12-27 08:11:34,945][20347] Updated weights for policy 0, policy_version 368738 (0.0009) [2023-12-27 08:11:35,003][20347] Updated weights for policy 0, policy_version 368748 (0.0006) [2023-12-27 08:11:35,059][20347] Updated weights for policy 0, policy_version 368758 (0.0008) [2023-12-27 08:11:35,116][20347] Updated weights for policy 0, policy_version 368768 (0.0005) [2023-12-27 08:11:35,148][20381] Updated weights for policy 1, policy_version 368501 (0.0008) [2023-12-27 08:11:35,200][20381] Updated weights for policy 1, policy_version 368511 (0.0006) [2023-12-27 08:11:35,255][20381] Updated weights for policy 1, policy_version 368521 (0.0010) [2023-12-27 08:11:35,748][20347] Updated weights for policy 0, policy_version 368778 (0.0010) [2023-12-27 08:11:35,802][20347] Updated weights for policy 0, policy_version 368788 (0.0010) [2023-12-27 08:11:35,851][20347] Updated weights for policy 0, policy_version 368798 (0.0010) [2023-12-27 08:11:35,868][20381] Updated weights for policy 1, policy_version 368531 (0.0010) [2023-12-27 08:11:35,925][20381] Updated weights for policy 1, policy_version 368541 (0.0010) [2023-12-27 08:11:35,980][20381] Updated weights for policy 1, policy_version 368551 (0.0010) [2023-12-27 08:11:36,547][20347] Updated weights for policy 0, policy_version 368808 (0.0010) [2023-12-27 08:11:36,602][20347] Updated weights for policy 0, policy_version 368818 (0.0010) [2023-12-27 08:11:36,663][20347] Updated weights for policy 0, policy_version 368828 (0.0010) [2023-12-27 08:11:36,676][20381] Updated weights for policy 1, policy_version 368561 (0.0010) [2023-12-27 08:11:36,734][20381] Updated weights for policy 1, policy_version 368571 (0.0010) [2023-12-27 08:11:36,795][20381] Updated weights for policy 1, policy_version 368581 (0.0010) [2023-12-27 08:11:36,852][20381] Updated weights for policy 1, policy_version 368591 (0.0010) [2023-12-27 08:11:37,335][20347] Updated weights for policy 0, policy_version 368838 (0.0010) [2023-12-27 08:11:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 188817408. Throughput: 0: 10133.6, 1: 10641.4. Samples: 188790976. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 08:11:37,363][19170] Avg episode reward: [(0, '988.240'), (1, '932.990')] [2023-12-27 08:11:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000368592_94380032.pth... [2023-12-27 08:11:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000367344_94060544.pth [2023-12-27 08:11:37,393][20347] Updated weights for policy 0, policy_version 368848 (0.0006) [2023-12-27 08:11:37,458][20347] Updated weights for policy 0, policy_version 368858 (0.0007) [2023-12-27 08:11:37,492][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000368864_94445568.pth... [2023-12-27 08:11:37,496][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000367680_94142464.pth [2023-12-27 08:11:37,499][20381] Updated weights for policy 1, policy_version 368601 (0.0010) [2023-12-27 08:11:37,557][20381] Updated weights for policy 1, policy_version 368611 (0.0005) [2023-12-27 08:11:37,617][20381] Updated weights for policy 1, policy_version 368621 (0.0009) [2023-12-27 08:11:38,164][20347] Updated weights for policy 0, policy_version 368868 (0.0007) [2023-12-27 08:11:38,216][20347] Updated weights for policy 0, policy_version 368878 (0.0006) [2023-12-27 08:11:38,231][20381] Updated weights for policy 1, policy_version 368631 (0.0010) [2023-12-27 08:11:38,279][20347] Updated weights for policy 0, policy_version 368888 (0.0005) [2023-12-27 08:11:38,288][20381] Updated weights for policy 1, policy_version 368641 (0.0008) [2023-12-27 08:11:38,346][20381] Updated weights for policy 1, policy_version 368651 (0.0007) [2023-12-27 08:11:38,882][20347] Updated weights for policy 0, policy_version 368898 (0.0007) [2023-12-27 08:11:38,947][20347] Updated weights for policy 0, policy_version 368908 (0.0011) [2023-12-27 08:11:39,008][20347] Updated weights for policy 0, policy_version 368918 (0.0010) [2023-12-27 08:11:39,039][20381] Updated weights for policy 1, policy_version 368661 (0.0007) [2023-12-27 08:11:39,069][20347] Updated weights for policy 0, policy_version 368928 (0.0010) [2023-12-27 08:11:39,095][20381] Updated weights for policy 1, policy_version 368671 (0.0005) [2023-12-27 08:11:39,156][20381] Updated weights for policy 1, policy_version 368681 (0.0006) [2023-12-27 08:11:39,702][20347] Updated weights for policy 0, policy_version 368938 (0.0010) [2023-12-27 08:11:39,764][20347] Updated weights for policy 0, policy_version 368948 (0.0010) [2023-12-27 08:11:39,784][20381] Updated weights for policy 1, policy_version 368691 (0.0007) [2023-12-27 08:11:39,827][20347] Updated weights for policy 0, policy_version 368958 (0.0010) [2023-12-27 08:11:39,839][20381] Updated weights for policy 1, policy_version 368701 (0.0010) [2023-12-27 08:11:39,898][20381] Updated weights for policy 1, policy_version 368711 (0.0008) [2023-12-27 08:11:40,487][20347] Updated weights for policy 0, policy_version 368968 (0.0007) [2023-12-27 08:11:40,546][20347] Updated weights for policy 0, policy_version 368978 (0.0005) [2023-12-27 08:11:40,567][20381] Updated weights for policy 1, policy_version 368721 (0.0007) [2023-12-27 08:11:40,594][20347] Updated weights for policy 0, policy_version 368988 (0.0007) [2023-12-27 08:11:40,631][20381] Updated weights for policy 1, policy_version 368731 (0.0005) [2023-12-27 08:11:40,691][20381] Updated weights for policy 1, policy_version 368741 (0.0006) [2023-12-27 08:11:40,754][20381] Updated weights for policy 1, policy_version 368751 (0.0005) [2023-12-27 08:11:41,290][20381] Updated weights for policy 1, policy_version 368761 (0.0006) [2023-12-27 08:11:41,302][20347] Updated weights for policy 0, policy_version 368998 (0.0008) [2023-12-27 08:11:41,358][20381] Updated weights for policy 1, policy_version 368771 (0.0006) [2023-12-27 08:11:41,370][20347] Updated weights for policy 0, policy_version 369008 (0.0007) [2023-12-27 08:11:41,423][20381] Updated weights for policy 1, policy_version 368781 (0.0006) [2023-12-27 08:11:41,436][20347] Updated weights for policy 0, policy_version 369018 (0.0008) [2023-12-27 08:11:42,027][20347] Updated weights for policy 0, policy_version 369028 (0.0006) [2023-12-27 08:11:42,090][20347] Updated weights for policy 0, policy_version 369038 (0.0007) [2023-12-27 08:11:42,095][20381] Updated weights for policy 1, policy_version 368791 (0.0006) [2023-12-27 08:11:42,146][20347] Updated weights for policy 0, policy_version 369048 (0.0006) [2023-12-27 08:11:42,153][20381] Updated weights for policy 1, policy_version 368801 (0.0006) [2023-12-27 08:11:42,207][20381] Updated weights for policy 1, policy_version 368811 (0.0009) [2023-12-27 08:11:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 188932096. Throughput: 0: 10128.3, 1: 10656.9. Samples: 188917392. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:11:42,363][19170] Avg episode reward: [(0, '989.880'), (1, '941.850')] [2023-12-27 08:11:42,870][20347] Updated weights for policy 0, policy_version 369058 (0.0009) [2023-12-27 08:11:42,899][20381] Updated weights for policy 1, policy_version 368821 (0.0007) [2023-12-27 08:11:42,925][20347] Updated weights for policy 0, policy_version 369068 (0.0006) [2023-12-27 08:11:42,954][20381] Updated weights for policy 1, policy_version 368831 (0.0008) [2023-12-27 08:11:42,981][20347] Updated weights for policy 0, policy_version 369078 (0.0007) [2023-12-27 08:11:43,012][20381] Updated weights for policy 1, policy_version 368841 (0.0005) [2023-12-27 08:11:43,041][20347] Updated weights for policy 0, policy_version 369088 (0.0006) [2023-12-27 08:11:43,586][20347] Updated weights for policy 0, policy_version 369098 (0.0005) [2023-12-27 08:11:43,642][20347] Updated weights for policy 0, policy_version 369108 (0.0005) [2023-12-27 08:11:43,672][20381] Updated weights for policy 1, policy_version 368851 (0.0008) [2023-12-27 08:11:43,703][20347] Updated weights for policy 0, policy_version 369118 (0.0005) [2023-12-27 08:11:43,731][20381] Updated weights for policy 1, policy_version 368861 (0.0010) [2023-12-27 08:11:43,784][20381] Updated weights for policy 1, policy_version 368871 (0.0010) [2023-12-27 08:11:44,276][20347] Updated weights for policy 0, policy_version 369128 (0.0005) [2023-12-27 08:11:44,335][20347] Updated weights for policy 0, policy_version 369138 (0.0007) [2023-12-27 08:11:44,376][20381] Updated weights for policy 1, policy_version 368881 (0.0009) [2023-12-27 08:11:44,384][20347] Updated weights for policy 0, policy_version 369148 (0.0007) [2023-12-27 08:11:44,432][20381] Updated weights for policy 1, policy_version 368891 (0.0010) [2023-12-27 08:11:44,488][20381] Updated weights for policy 1, policy_version 368901 (0.0010) [2023-12-27 08:11:44,549][20381] Updated weights for policy 1, policy_version 368911 (0.0005) [2023-12-27 08:11:45,060][20347] Updated weights for policy 0, policy_version 369158 (0.0007) [2023-12-27 08:11:45,122][20347] Updated weights for policy 0, policy_version 369168 (0.0008) [2023-12-27 08:11:45,182][20347] Updated weights for policy 0, policy_version 369178 (0.0007) [2023-12-27 08:11:45,216][20381] Updated weights for policy 1, policy_version 368921 (0.0010) [2023-12-27 08:11:45,273][20381] Updated weights for policy 1, policy_version 368931 (0.0011) [2023-12-27 08:11:45,326][20381] Updated weights for policy 1, policy_version 368941 (0.0010) [2023-12-27 08:11:45,813][20347] Updated weights for policy 0, policy_version 369188 (0.0006) [2023-12-27 08:11:45,867][20347] Updated weights for policy 0, policy_version 369198 (0.0008) [2023-12-27 08:11:45,920][20347] Updated weights for policy 0, policy_version 369208 (0.0007) [2023-12-27 08:11:46,010][20381] Updated weights for policy 1, policy_version 368951 (0.0010) [2023-12-27 08:11:46,071][20381] Updated weights for policy 1, policy_version 368961 (0.0010) [2023-12-27 08:11:46,132][20381] Updated weights for policy 1, policy_version 368971 (0.0010) [2023-12-27 08:11:46,566][20347] Updated weights for policy 0, policy_version 369218 (0.0007) [2023-12-27 08:11:46,621][20347] Updated weights for policy 0, policy_version 369228 (0.0006) [2023-12-27 08:11:46,668][20347] Updated weights for policy 0, policy_version 369238 (0.0007) [2023-12-27 08:11:46,715][20347] Updated weights for policy 0, policy_version 369248 (0.0007) [2023-12-27 08:11:46,757][20381] Updated weights for policy 1, policy_version 368981 (0.0010) [2023-12-27 08:11:46,819][20381] Updated weights for policy 1, policy_version 368991 (0.0010) [2023-12-27 08:11:46,874][20381] Updated weights for policy 1, policy_version 369001 (0.0010) [2023-12-27 08:11:47,356][20347] Updated weights for policy 0, policy_version 369258 (0.0008) [2023-12-27 08:11:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20910.4). Total num frames: 189030400. Throughput: 0: 10234.9, 1: 10623.0. Samples: 189045192. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:11:47,363][19170] Avg episode reward: [(0, '952.260'), (1, '855.000')] [2023-12-27 08:11:47,404][20347] Updated weights for policy 0, policy_version 369268 (0.0008) [2023-12-27 08:11:47,447][20381] Updated weights for policy 1, policy_version 369011 (0.0009) [2023-12-27 08:11:47,461][20347] Updated weights for policy 0, policy_version 369278 (0.0008) [2023-12-27 08:11:47,503][20381] Updated weights for policy 1, policy_version 369021 (0.0005) [2023-12-27 08:11:47,564][20381] Updated weights for policy 1, policy_version 369031 (0.0007) [2023-12-27 08:11:48,120][20381] Updated weights for policy 1, policy_version 369041 (0.0006) [2023-12-27 08:11:48,176][20381] Updated weights for policy 1, policy_version 369051 (0.0005) [2023-12-27 08:11:48,205][20347] Updated weights for policy 0, policy_version 369288 (0.0006) [2023-12-27 08:11:48,232][20381] Updated weights for policy 1, policy_version 369061 (0.0005) [2023-12-27 08:11:48,267][20347] Updated weights for policy 0, policy_version 369298 (0.0006) [2023-12-27 08:11:48,281][20291] KL-divergence is very high: 116.5678 [2023-12-27 08:11:48,282][20381] Updated weights for policy 1, policy_version 369071 (0.0006) [2023-12-27 08:11:48,330][20347] Updated weights for policy 0, policy_version 369308 (0.0006) [2023-12-27 08:11:48,883][20292] KL-divergence is very high: 124.8873 [2023-12-27 08:11:48,888][20292] KL-divergence is very high: 100.9583 [2023-12-27 08:11:48,911][20381] Updated weights for policy 1, policy_version 369081 (0.0010) [2023-12-27 08:11:48,924][20292] KL-divergence is very high: 282.2511 [2023-12-27 08:11:48,932][20292] KL-divergence is very high: 189.5471 [2023-12-27 08:11:48,972][20381] Updated weights for policy 1, policy_version 369091 (0.0010) [2023-12-27 08:11:48,980][20347] Updated weights for policy 0, policy_version 369318 (0.0005) [2023-12-27 08:11:49,027][20381] Updated weights for policy 1, policy_version 369101 (0.0010) [2023-12-27 08:11:49,039][20347] Updated weights for policy 0, policy_version 369328 (0.0005) [2023-12-27 08:11:49,087][20347] Updated weights for policy 0, policy_version 369338 (0.0007) [2023-12-27 08:11:49,669][20381] Updated weights for policy 1, policy_version 369111 (0.0007) [2023-12-27 08:11:49,728][20381] Updated weights for policy 1, policy_version 369121 (0.0006) [2023-12-27 08:11:49,795][20381] Updated weights for policy 1, policy_version 369131 (0.0007) [2023-12-27 08:11:49,852][20347] Updated weights for policy 0, policy_version 369348 (0.0008) [2023-12-27 08:11:49,913][20347] Updated weights for policy 0, policy_version 369358 (0.0010) [2023-12-27 08:11:49,973][20347] Updated weights for policy 0, policy_version 369368 (0.0010) [2023-12-27 08:11:50,434][20381] Updated weights for policy 1, policy_version 369141 (0.0008) [2023-12-27 08:11:50,488][20381] Updated weights for policy 1, policy_version 369151 (0.0007) [2023-12-27 08:11:50,541][20381] Updated weights for policy 1, policy_version 369161 (0.0007) [2023-12-27 08:11:50,659][20347] Updated weights for policy 0, policy_version 369378 (0.0010) [2023-12-27 08:11:50,717][20347] Updated weights for policy 0, policy_version 369388 (0.0008) [2023-12-27 08:11:50,779][20347] Updated weights for policy 0, policy_version 369398 (0.0007) [2023-12-27 08:11:50,841][20347] Updated weights for policy 0, policy_version 369408 (0.0010) [2023-12-27 08:11:51,174][20381] Updated weights for policy 1, policy_version 369171 (0.0007) [2023-12-27 08:11:51,233][20381] Updated weights for policy 1, policy_version 369181 (0.0007) [2023-12-27 08:11:51,292][20381] Updated weights for policy 1, policy_version 369191 (0.0008) [2023-12-27 08:11:51,523][20347] Updated weights for policy 0, policy_version 369418 (0.0008) [2023-12-27 08:11:51,588][20347] Updated weights for policy 0, policy_version 369428 (0.0009) [2023-12-27 08:11:51,653][20347] Updated weights for policy 0, policy_version 369438 (0.0010) [2023-12-27 08:11:51,959][20381] Updated weights for policy 1, policy_version 369201 (0.0007) [2023-12-27 08:11:52,025][20381] Updated weights for policy 1, policy_version 369211 (0.0006) [2023-12-27 08:11:52,072][20381] Updated weights for policy 1, policy_version 369221 (0.0006) [2023-12-27 08:11:52,126][20381] Updated weights for policy 1, policy_version 369231 (0.0011) [2023-12-27 08:11:52,309][20347] Updated weights for policy 0, policy_version 369448 (0.0007) [2023-12-27 08:11:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20910.4). Total num frames: 189136896. Throughput: 0: 10173.8, 1: 10660.0. Samples: 189108204. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:11:52,363][19170] Avg episode reward: [(0, '923.210'), (1, '864.650')] [2023-12-27 08:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000369232_94543872.pth... [2023-12-27 08:11:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000367984_94224384.pth [2023-12-27 08:11:52,375][20347] Updated weights for policy 0, policy_version 369458 (0.0007) [2023-12-27 08:11:52,430][20347] Updated weights for policy 0, policy_version 369468 (0.0007) [2023-12-27 08:11:52,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000369472_94601216.pth... [2023-12-27 08:11:52,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000368256_94289920.pth [2023-12-27 08:11:52,846][20381] Updated weights for policy 1, policy_version 369241 (0.0010) [2023-12-27 08:11:52,911][20381] Updated weights for policy 1, policy_version 369251 (0.0011) [2023-12-27 08:11:52,971][20381] Updated weights for policy 1, policy_version 369261 (0.0011) [2023-12-27 08:11:53,065][20347] Updated weights for policy 0, policy_version 369478 (0.0008) [2023-12-27 08:11:53,120][20347] Updated weights for policy 0, policy_version 369488 (0.0007) [2023-12-27 08:11:53,176][20347] Updated weights for policy 0, policy_version 369498 (0.0008) [2023-12-27 08:11:53,636][20381] Updated weights for policy 1, policy_version 369271 (0.0010) [2023-12-27 08:11:53,684][20381] Updated weights for policy 1, policy_version 369281 (0.0010) [2023-12-27 08:11:53,738][20381] Updated weights for policy 1, policy_version 369291 (0.0010) [2023-12-27 08:11:53,882][20347] Updated weights for policy 0, policy_version 369508 (0.0008) [2023-12-27 08:11:53,939][20347] Updated weights for policy 0, policy_version 369518 (0.0010) [2023-12-27 08:11:53,993][20347] Updated weights for policy 0, policy_version 369528 (0.0010) [2023-12-27 08:11:54,448][20381] Updated weights for policy 1, policy_version 369301 (0.0009) [2023-12-27 08:11:54,500][20381] Updated weights for policy 1, policy_version 369311 (0.0007) [2023-12-27 08:11:54,549][20381] Updated weights for policy 1, policy_version 369321 (0.0006) [2023-12-27 08:11:54,680][20347] Updated weights for policy 0, policy_version 369538 (0.0010) [2023-12-27 08:11:54,732][20347] Updated weights for policy 0, policy_version 369548 (0.0010) [2023-12-27 08:11:54,782][20347] Updated weights for policy 0, policy_version 369558 (0.0010) [2023-12-27 08:11:54,827][20347] Updated weights for policy 0, policy_version 369568 (0.0010) [2023-12-27 08:11:55,158][20381] Updated weights for policy 1, policy_version 369331 (0.0005) [2023-12-27 08:11:55,209][20381] Updated weights for policy 1, policy_version 369341 (0.0005) [2023-12-27 08:11:55,254][20381] Updated weights for policy 1, policy_version 369351 (0.0005) [2023-12-27 08:11:55,536][20347] Updated weights for policy 0, policy_version 369578 (0.0005) [2023-12-27 08:11:55,592][20347] Updated weights for policy 0, policy_version 369588 (0.0005) [2023-12-27 08:11:55,641][20347] Updated weights for policy 0, policy_version 369598 (0.0007) [2023-12-27 08:11:55,897][20381] Updated weights for policy 1, policy_version 369361 (0.0005) [2023-12-27 08:11:55,962][20381] Updated weights for policy 1, policy_version 369371 (0.0005) [2023-12-27 08:11:56,019][20381] Updated weights for policy 1, policy_version 369381 (0.0006) [2023-12-27 08:11:56,080][20381] Updated weights for policy 1, policy_version 369391 (0.0005) [2023-12-27 08:11:56,243][20347] Updated weights for policy 0, policy_version 369608 (0.0007) [2023-12-27 08:11:56,297][20347] Updated weights for policy 0, policy_version 369618 (0.0007) [2023-12-27 08:11:56,355][20347] Updated weights for policy 0, policy_version 369628 (0.0007) [2023-12-27 08:11:56,629][20381] Updated weights for policy 1, policy_version 369401 (0.0005) [2023-12-27 08:11:56,672][20381] Updated weights for policy 1, policy_version 369411 (0.0005) [2023-12-27 08:11:56,724][20381] Updated weights for policy 1, policy_version 369421 (0.0005) [2023-12-27 08:11:57,079][20347] Updated weights for policy 0, policy_version 369638 (0.0008) [2023-12-27 08:11:57,129][20347] Updated weights for policy 0, policy_version 369648 (0.0005) [2023-12-27 08:11:57,185][20347] Updated weights for policy 0, policy_version 369658 (0.0007) [2023-12-27 08:11:57,295][20381] Updated weights for policy 1, policy_version 369431 (0.0009) [2023-12-27 08:11:57,344][20381] Updated weights for policy 1, policy_version 369441 (0.0008) [2023-12-27 08:11:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 189243392. Throughput: 0: 10172.2, 1: 10719.7. Samples: 189233212. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:11:57,363][19170] Avg episode reward: [(0, '845.350'), (1, '912.980')] [2023-12-27 08:11:57,389][20381] Updated weights for policy 1, policy_version 369451 (0.0006) [2023-12-27 08:11:57,760][20347] Updated weights for policy 0, policy_version 369668 (0.0007) [2023-12-27 08:11:57,811][20347] Updated weights for policy 0, policy_version 369678 (0.0008) [2023-12-27 08:11:57,862][20347] Updated weights for policy 0, policy_version 369688 (0.0005) [2023-12-27 08:11:58,054][20381] Updated weights for policy 1, policy_version 369461 (0.0008) [2023-12-27 08:11:58,115][20381] Updated weights for policy 1, policy_version 369471 (0.0010) [2023-12-27 08:11:58,180][20381] Updated weights for policy 1, policy_version 369481 (0.0010) [2023-12-27 08:11:58,489][20347] Updated weights for policy 0, policy_version 369698 (0.0006) [2023-12-27 08:11:58,553][20347] Updated weights for policy 0, policy_version 369708 (0.0006) [2023-12-27 08:11:58,612][20347] Updated weights for policy 0, policy_version 369718 (0.0008) [2023-12-27 08:11:58,671][20347] Updated weights for policy 0, policy_version 369728 (0.0010) [2023-12-27 08:11:58,925][20381] Updated weights for policy 1, policy_version 369491 (0.0007) [2023-12-27 08:11:58,980][20381] Updated weights for policy 1, policy_version 369501 (0.0006) [2023-12-27 08:11:59,046][20381] Updated weights for policy 1, policy_version 369511 (0.0008) [2023-12-27 08:11:59,396][20347] Updated weights for policy 0, policy_version 369738 (0.0007) [2023-12-27 08:11:59,444][20347] Updated weights for policy 0, policy_version 369748 (0.0007) [2023-12-27 08:11:59,494][20347] Updated weights for policy 0, policy_version 369758 (0.0007) [2023-12-27 08:11:59,727][20381] Updated weights for policy 1, policy_version 369521 (0.0011) [2023-12-27 08:11:59,793][20381] Updated weights for policy 1, policy_version 369531 (0.0011) [2023-12-27 08:11:59,856][20381] Updated weights for policy 1, policy_version 369541 (0.0011) [2023-12-27 08:11:59,920][20381] Updated weights for policy 1, policy_version 369551 (0.0008) [2023-12-27 08:12:00,210][20291] KL-divergence is very high: 182.9799 [2023-12-27 08:12:00,232][20347] Updated weights for policy 0, policy_version 369768 (0.0007) [2023-12-27 08:12:00,291][20347] Updated weights for policy 0, policy_version 369778 (0.0007) [2023-12-27 08:12:00,350][20347] Updated weights for policy 0, policy_version 369788 (0.0007) [2023-12-27 08:12:00,593][20381] Updated weights for policy 1, policy_version 369561 (0.0007) [2023-12-27 08:12:00,663][20381] Updated weights for policy 1, policy_version 369571 (0.0006) [2023-12-27 08:12:00,722][20381] Updated weights for policy 1, policy_version 369581 (0.0005) [2023-12-27 08:12:00,964][20347] Updated weights for policy 0, policy_version 369798 (0.0007) [2023-12-27 08:12:01,016][20347] Updated weights for policy 0, policy_version 369808 (0.0007) [2023-12-27 08:12:01,080][20347] Updated weights for policy 0, policy_version 369818 (0.0007) [2023-12-27 08:12:01,350][20381] Updated weights for policy 1, policy_version 369591 (0.0008) [2023-12-27 08:12:01,419][20381] Updated weights for policy 1, policy_version 369601 (0.0007) [2023-12-27 08:12:01,483][20381] Updated weights for policy 1, policy_version 369611 (0.0008) [2023-12-27 08:12:01,786][20347] Updated weights for policy 0, policy_version 369828 (0.0006) [2023-12-27 08:12:01,832][20347] Updated weights for policy 0, policy_version 369838 (0.0007) [2023-12-27 08:12:01,879][20347] Updated weights for policy 0, policy_version 369848 (0.0008) [2023-12-27 08:12:02,122][20381] Updated weights for policy 1, policy_version 369621 (0.0006) [2023-12-27 08:12:02,191][20381] Updated weights for policy 1, policy_version 369631 (0.0009) [2023-12-27 08:12:02,261][20381] Updated weights for policy 1, policy_version 369641 (0.0008) [2023-12-27 08:12:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 189349888. Throughput: 0: 10238.5, 1: 10672.0. Samples: 189358100. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:02,363][19170] Avg episode reward: [(0, '797.660'), (1, '951.500')] [2023-12-27 08:12:02,564][20347] Updated weights for policy 0, policy_version 369858 (0.0008) [2023-12-27 08:12:02,617][20347] Updated weights for policy 0, policy_version 369868 (0.0007) [2023-12-27 08:12:02,668][20347] Updated weights for policy 0, policy_version 369878 (0.0005) [2023-12-27 08:12:02,717][20347] Updated weights for policy 0, policy_version 369888 (0.0006) [2023-12-27 08:12:02,910][20381] Updated weights for policy 1, policy_version 369651 (0.0007) [2023-12-27 08:12:02,974][20381] Updated weights for policy 1, policy_version 369661 (0.0007) [2023-12-27 08:12:03,022][20381] Updated weights for policy 1, policy_version 369671 (0.0007) [2023-12-27 08:12:03,407][20347] Updated weights for policy 0, policy_version 369898 (0.0007) [2023-12-27 08:12:03,473][20347] Updated weights for policy 0, policy_version 369908 (0.0008) [2023-12-27 08:12:03,534][20347] Updated weights for policy 0, policy_version 369918 (0.0007) [2023-12-27 08:12:03,707][20381] Updated weights for policy 1, policy_version 369681 (0.0007) [2023-12-27 08:12:03,759][20381] Updated weights for policy 1, policy_version 369691 (0.0007) [2023-12-27 08:12:03,806][20381] Updated weights for policy 1, policy_version 369701 (0.0007) [2023-12-27 08:12:03,867][20381] Updated weights for policy 1, policy_version 369711 (0.0007) [2023-12-27 08:12:04,199][20347] Updated weights for policy 0, policy_version 369928 (0.0008) [2023-12-27 08:12:04,255][20347] Updated weights for policy 0, policy_version 369938 (0.0008) [2023-12-27 08:12:04,313][20347] Updated weights for policy 0, policy_version 369948 (0.0008) [2023-12-27 08:12:04,592][20381] Updated weights for policy 1, policy_version 369721 (0.0011) [2023-12-27 08:12:04,651][20381] Updated weights for policy 1, policy_version 369731 (0.0008) [2023-12-27 08:12:04,713][20381] Updated weights for policy 1, policy_version 369741 (0.0011) [2023-12-27 08:12:04,918][20347] Updated weights for policy 0, policy_version 369958 (0.0009) [2023-12-27 08:12:04,967][20347] Updated weights for policy 0, policy_version 369968 (0.0006) [2023-12-27 08:12:05,022][20347] Updated weights for policy 0, policy_version 369978 (0.0006) [2023-12-27 08:12:05,334][20381] Updated weights for policy 1, policy_version 369751 (0.0007) [2023-12-27 08:12:05,380][20381] Updated weights for policy 1, policy_version 369761 (0.0005) [2023-12-27 08:12:05,427][20381] Updated weights for policy 1, policy_version 369771 (0.0005) [2023-12-27 08:12:05,445][20292] KL-divergence is very high: 481.4209 [2023-12-27 08:12:05,746][20347] Updated weights for policy 0, policy_version 369988 (0.0007) [2023-12-27 08:12:05,802][20347] Updated weights for policy 0, policy_version 369998 (0.0005) [2023-12-27 08:12:05,866][20347] Updated weights for policy 0, policy_version 370008 (0.0005) [2023-12-27 08:12:06,012][20381] Updated weights for policy 1, policy_version 369781 (0.0008) [2023-12-27 08:12:06,054][20292] KL-divergence is very high: 115.8149 [2023-12-27 08:12:06,075][20381] Updated weights for policy 1, policy_version 369791 (0.0009) [2023-12-27 08:12:06,139][20381] Updated weights for policy 1, policy_version 369801 (0.0009) [2023-12-27 08:12:06,420][20347] Updated weights for policy 0, policy_version 370018 (0.0006) [2023-12-27 08:12:06,485][20347] Updated weights for policy 0, policy_version 370028 (0.0007) [2023-12-27 08:12:06,549][20347] Updated weights for policy 0, policy_version 370038 (0.0008) [2023-12-27 08:12:06,607][20347] Updated weights for policy 0, policy_version 370048 (0.0007) [2023-12-27 08:12:06,779][20381] Updated weights for policy 1, policy_version 369811 (0.0008) [2023-12-27 08:12:06,827][20381] Updated weights for policy 1, policy_version 369821 (0.0007) [2023-12-27 08:12:06,880][20381] Updated weights for policy 1, policy_version 369831 (0.0007) [2023-12-27 08:12:07,316][20347] Updated weights for policy 0, policy_version 370058 (0.0007) [2023-12-27 08:12:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 189448192. Throughput: 0: 10297.2, 1: 10623.5. Samples: 189420180. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:07,363][19170] Avg episode reward: [(0, '835.920'), (1, '961.290')] [2023-12-27 08:12:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000369840_94699520.pth... [2023-12-27 08:12:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000368592_94380032.pth [2023-12-27 08:12:07,378][20347] Updated weights for policy 0, policy_version 370068 (0.0007) [2023-12-27 08:12:07,432][20347] Updated weights for policy 0, policy_version 370078 (0.0005) [2023-12-27 08:12:07,442][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000370080_94756864.pth... [2023-12-27 08:12:07,445][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000368864_94445568.pth [2023-12-27 08:12:07,469][20381] Updated weights for policy 1, policy_version 369841 (0.0005) [2023-12-27 08:12:07,522][20381] Updated weights for policy 1, policy_version 369851 (0.0005) [2023-12-27 08:12:07,569][20381] Updated weights for policy 1, policy_version 369861 (0.0006) [2023-12-27 08:12:07,622][20381] Updated weights for policy 1, policy_version 369871 (0.0005) [2023-12-27 08:12:08,082][20347] Updated weights for policy 0, policy_version 370088 (0.0005) [2023-12-27 08:12:08,143][20347] Updated weights for policy 0, policy_version 370098 (0.0007) [2023-12-27 08:12:08,154][20381] Updated weights for policy 1, policy_version 369881 (0.0007) [2023-12-27 08:12:08,203][20347] Updated weights for policy 0, policy_version 370108 (0.0007) [2023-12-27 08:12:08,210][20381] Updated weights for policy 1, policy_version 369891 (0.0005) [2023-12-27 08:12:08,256][20381] Updated weights for policy 1, policy_version 369901 (0.0005) [2023-12-27 08:12:08,910][20347] Updated weights for policy 0, policy_version 370118 (0.0007) [2023-12-27 08:12:08,935][20381] Updated weights for policy 1, policy_version 369911 (0.0010) [2023-12-27 08:12:08,972][20347] Updated weights for policy 0, policy_version 370128 (0.0006) [2023-12-27 08:12:08,981][20381] Updated weights for policy 1, policy_version 369921 (0.0010) [2023-12-27 08:12:09,025][20347] Updated weights for policy 0, policy_version 370138 (0.0007) [2023-12-27 08:12:09,040][20381] Updated weights for policy 1, policy_version 369931 (0.0005) [2023-12-27 08:12:09,665][20381] Updated weights for policy 1, policy_version 369941 (0.0005) [2023-12-27 08:12:09,724][20381] Updated weights for policy 1, policy_version 369951 (0.0006) [2023-12-27 08:12:09,752][20347] Updated weights for policy 0, policy_version 370148 (0.0007) [2023-12-27 08:12:09,780][20381] Updated weights for policy 1, policy_version 369961 (0.0006) [2023-12-27 08:12:09,808][20347] Updated weights for policy 0, policy_version 370158 (0.0006) [2023-12-27 08:12:09,873][20347] Updated weights for policy 0, policy_version 370168 (0.0007) [2023-12-27 08:12:10,476][20381] Updated weights for policy 1, policy_version 369971 (0.0008) [2023-12-27 08:12:10,504][20347] Updated weights for policy 0, policy_version 370178 (0.0007) [2023-12-27 08:12:10,529][20381] Updated weights for policy 1, policy_version 369981 (0.0011) [2023-12-27 08:12:10,560][20347] Updated weights for policy 0, policy_version 370188 (0.0007) [2023-12-27 08:12:10,578][20381] Updated weights for policy 1, policy_version 369991 (0.0010) [2023-12-27 08:12:10,616][20347] Updated weights for policy 0, policy_version 370198 (0.0006) [2023-12-27 08:12:10,675][20347] Updated weights for policy 0, policy_version 370208 (0.0007) [2023-12-27 08:12:11,198][20381] Updated weights for policy 1, policy_version 370001 (0.0010) [2023-12-27 08:12:11,260][20381] Updated weights for policy 1, policy_version 370011 (0.0007) [2023-12-27 08:12:11,321][20381] Updated weights for policy 1, policy_version 370021 (0.0007) [2023-12-27 08:12:11,390][20381] Updated weights for policy 1, policy_version 370031 (0.0009) [2023-12-27 08:12:11,446][20347] Updated weights for policy 0, policy_version 370218 (0.0008) [2023-12-27 08:12:11,503][20347] Updated weights for policy 0, policy_version 370228 (0.0008) [2023-12-27 08:12:11,558][20347] Updated weights for policy 0, policy_version 370238 (0.0008) [2023-12-27 08:12:12,055][20381] Updated weights for policy 1, policy_version 370041 (0.0005) [2023-12-27 08:12:12,118][20381] Updated weights for policy 1, policy_version 370051 (0.0005) [2023-12-27 08:12:12,179][20381] Updated weights for policy 1, policy_version 370061 (0.0007) [2023-12-27 08:12:12,257][20347] Updated weights for policy 0, policy_version 370248 (0.0008) [2023-12-27 08:12:12,309][20347] Updated weights for policy 0, policy_version 370258 (0.0008) [2023-12-27 08:12:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 189554688. Throughput: 0: 10285.1, 1: 10674.7. Samples: 189547620. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:12,363][19170] Avg episode reward: [(0, '874.690'), (1, '932.500')] [2023-12-27 08:12:12,366][20347] Updated weights for policy 0, policy_version 370268 (0.0006) [2023-12-27 08:12:12,368][20291] KL-divergence is very high: 134.5526 [2023-12-27 08:12:12,807][20381] Updated weights for policy 1, policy_version 370071 (0.0008) [2023-12-27 08:12:12,858][20381] Updated weights for policy 1, policy_version 370081 (0.0007) [2023-12-27 08:12:12,918][20381] Updated weights for policy 1, policy_version 370091 (0.0006) [2023-12-27 08:12:12,990][20347] Updated weights for policy 0, policy_version 370278 (0.0008) [2023-12-27 08:12:13,046][20347] Updated weights for policy 0, policy_version 370288 (0.0006) [2023-12-27 08:12:13,105][20347] Updated weights for policy 0, policy_version 370298 (0.0008) [2023-12-27 08:12:13,557][20381] Updated weights for policy 1, policy_version 370101 (0.0007) [2023-12-27 08:12:13,609][20381] Updated weights for policy 1, policy_version 370111 (0.0009) [2023-12-27 08:12:13,664][20381] Updated weights for policy 1, policy_version 370121 (0.0008) [2023-12-27 08:12:13,744][20347] Updated weights for policy 0, policy_version 370308 (0.0007) [2023-12-27 08:12:13,802][20347] Updated weights for policy 0, policy_version 370318 (0.0005) [2023-12-27 08:12:13,858][20347] Updated weights for policy 0, policy_version 370328 (0.0005) [2023-12-27 08:12:13,862][20291] KL-divergence is very high: 199.2483 [2023-12-27 08:12:14,363][20381] Updated weights for policy 1, policy_version 370131 (0.0008) [2023-12-27 08:12:14,418][20381] Updated weights for policy 1, policy_version 370141 (0.0007) [2023-12-27 08:12:14,425][20347] Updated weights for policy 0, policy_version 370338 (0.0005) [2023-12-27 08:12:14,470][20381] Updated weights for policy 1, policy_version 370151 (0.0007) [2023-12-27 08:12:14,485][20347] Updated weights for policy 0, policy_version 370348 (0.0006) [2023-12-27 08:12:14,549][20347] Updated weights for policy 0, policy_version 370358 (0.0008) [2023-12-27 08:12:14,605][20347] Updated weights for policy 0, policy_version 370368 (0.0009) [2023-12-27 08:12:15,165][20381] Updated weights for policy 1, policy_version 370161 (0.0007) [2023-12-27 08:12:15,219][20381] Updated weights for policy 1, policy_version 370171 (0.0007) [2023-12-27 08:12:15,245][20347] Updated weights for policy 0, policy_version 370378 (0.0010) [2023-12-27 08:12:15,268][20381] Updated weights for policy 1, policy_version 370181 (0.0008) [2023-12-27 08:12:15,303][20347] Updated weights for policy 0, policy_version 370388 (0.0010) [2023-12-27 08:12:15,320][20381] Updated weights for policy 1, policy_version 370191 (0.0005) [2023-12-27 08:12:15,357][20347] Updated weights for policy 0, policy_version 370398 (0.0010) [2023-12-27 08:12:15,916][20381] Updated weights for policy 1, policy_version 370201 (0.0008) [2023-12-27 08:12:15,970][20381] Updated weights for policy 1, policy_version 370211 (0.0010) [2023-12-27 08:12:16,022][20381] Updated weights for policy 1, policy_version 370221 (0.0010) [2023-12-27 08:12:16,023][20347] Updated weights for policy 0, policy_version 370408 (0.0010) [2023-12-27 08:12:16,081][20347] Updated weights for policy 0, policy_version 370418 (0.0010) [2023-12-27 08:12:16,139][20347] Updated weights for policy 0, policy_version 370428 (0.0010) [2023-12-27 08:12:16,561][20381] Updated weights for policy 1, policy_version 370231 (0.0007) [2023-12-27 08:12:16,614][20381] Updated weights for policy 1, policy_version 370241 (0.0011) [2023-12-27 08:12:16,659][20381] Updated weights for policy 1, policy_version 370251 (0.0010) [2023-12-27 08:12:16,737][20347] Updated weights for policy 0, policy_version 370438 (0.0008) [2023-12-27 08:12:16,794][20347] Updated weights for policy 0, policy_version 370448 (0.0010) [2023-12-27 08:12:16,854][20347] Updated weights for policy 0, policy_version 370458 (0.0009) [2023-12-27 08:12:17,320][20381] Updated weights for policy 1, policy_version 370261 (0.0010) [2023-12-27 08:12:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 189661184. Throughput: 0: 10378.4, 1: 10698.7. Samples: 189676176. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:17,363][19170] Avg episode reward: [(0, '894.960'), (1, '971.250')] [2023-12-27 08:12:17,374][20381] Updated weights for policy 1, policy_version 370271 (0.0010) [2023-12-27 08:12:17,430][20381] Updated weights for policy 1, policy_version 370281 (0.0010) [2023-12-27 08:12:17,473][20347] Updated weights for policy 0, policy_version 370468 (0.0007) [2023-12-27 08:12:17,524][20347] Updated weights for policy 0, policy_version 370478 (0.0010) [2023-12-27 08:12:17,568][20347] Updated weights for policy 0, policy_version 370488 (0.0010) [2023-12-27 08:12:18,113][20381] Updated weights for policy 1, policy_version 370291 (0.0010) [2023-12-27 08:12:18,171][20381] Updated weights for policy 1, policy_version 370301 (0.0011) [2023-12-27 08:12:18,219][20381] Updated weights for policy 1, policy_version 370311 (0.0010) [2023-12-27 08:12:18,230][20347] Updated weights for policy 0, policy_version 370498 (0.0009) [2023-12-27 08:12:18,283][20347] Updated weights for policy 0, policy_version 370508 (0.0006) [2023-12-27 08:12:18,291][20291] KL-divergence is very high: 129.5156 [2023-12-27 08:12:18,334][20347] Updated weights for policy 0, policy_version 370518 (0.0008) [2023-12-27 08:12:18,394][20347] Updated weights for policy 0, policy_version 370528 (0.0007) [2023-12-27 08:12:18,826][20381] Updated weights for policy 1, policy_version 370321 (0.0010) [2023-12-27 08:12:18,891][20381] Updated weights for policy 1, policy_version 370331 (0.0010) [2023-12-27 08:12:18,952][20381] Updated weights for policy 1, policy_version 370341 (0.0011) [2023-12-27 08:12:19,012][20381] Updated weights for policy 1, policy_version 370351 (0.0010) [2023-12-27 08:12:19,138][20347] Updated weights for policy 0, policy_version 370538 (0.0008) [2023-12-27 08:12:19,200][20347] Updated weights for policy 0, policy_version 370548 (0.0008) [2023-12-27 08:12:19,260][20347] Updated weights for policy 0, policy_version 370558 (0.0008) [2023-12-27 08:12:19,704][20381] Updated weights for policy 1, policy_version 370361 (0.0008) [2023-12-27 08:12:19,751][20381] Updated weights for policy 1, policy_version 370371 (0.0008) [2023-12-27 08:12:19,807][20381] Updated weights for policy 1, policy_version 370381 (0.0008) [2023-12-27 08:12:19,985][20347] Updated weights for policy 0, policy_version 370568 (0.0008) [2023-12-27 08:12:20,049][20347] Updated weights for policy 0, policy_version 370578 (0.0008) [2023-12-27 08:12:20,114][20347] Updated weights for policy 0, policy_version 370588 (0.0008) [2023-12-27 08:12:20,537][20381] Updated weights for policy 1, policy_version 370391 (0.0007) [2023-12-27 08:12:20,590][20381] Updated weights for policy 1, policy_version 370401 (0.0008) [2023-12-27 08:12:20,650][20381] Updated weights for policy 1, policy_version 370411 (0.0007) [2023-12-27 08:12:20,763][20347] Updated weights for policy 0, policy_version 370598 (0.0007) [2023-12-27 08:12:20,824][20347] Updated weights for policy 0, policy_version 370608 (0.0007) [2023-12-27 08:12:20,880][20347] Updated weights for policy 0, policy_version 370618 (0.0008) [2023-12-27 08:12:21,320][20381] Updated weights for policy 1, policy_version 370421 (0.0007) [2023-12-27 08:12:21,387][20381] Updated weights for policy 1, policy_version 370431 (0.0008) [2023-12-27 08:12:21,442][20381] Updated weights for policy 1, policy_version 370441 (0.0008) [2023-12-27 08:12:21,518][20347] Updated weights for policy 0, policy_version 370628 (0.0008) [2023-12-27 08:12:21,574][20347] Updated weights for policy 0, policy_version 370638 (0.0006) [2023-12-27 08:12:21,632][20347] Updated weights for policy 0, policy_version 370648 (0.0007) [2023-12-27 08:12:22,121][20381] Updated weights for policy 1, policy_version 370451 (0.0008) [2023-12-27 08:12:22,181][20381] Updated weights for policy 1, policy_version 370461 (0.0008) [2023-12-27 08:12:22,226][20381] Updated weights for policy 1, policy_version 370471 (0.0007) [2023-12-27 08:12:22,311][20347] Updated weights for policy 0, policy_version 370658 (0.0008) [2023-12-27 08:12:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 189767680. Throughput: 0: 10372.4, 1: 10645.7. Samples: 189736792. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:22,363][19170] Avg episode reward: [(0, '887.520'), (1, '990.470')] [2023-12-27 08:12:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000370480_94863360.pth... [2023-12-27 08:12:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000369232_94543872.pth [2023-12-27 08:12:22,380][20347] Updated weights for policy 0, policy_version 370668 (0.0008) [2023-12-27 08:12:22,436][20347] Updated weights for policy 0, policy_version 370678 (0.0008) [2023-12-27 08:12:22,488][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000370688_94912512.pth... [2023-12-27 08:12:22,491][20347] Updated weights for policy 0, policy_version 370688 (0.0007) [2023-12-27 08:12:22,491][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000369472_94601216.pth [2023-12-27 08:12:22,936][20381] Updated weights for policy 1, policy_version 370481 (0.0008) [2023-12-27 08:12:22,988][20381] Updated weights for policy 1, policy_version 370491 (0.0008) [2023-12-27 08:12:23,039][20381] Updated weights for policy 1, policy_version 370501 (0.0007) [2023-12-27 08:12:23,095][20381] Updated weights for policy 1, policy_version 370511 (0.0007) [2023-12-27 08:12:23,202][20347] Updated weights for policy 0, policy_version 370698 (0.0007) [2023-12-27 08:12:23,260][20347] Updated weights for policy 0, policy_version 370708 (0.0008) [2023-12-27 08:12:23,314][20347] Updated weights for policy 0, policy_version 370718 (0.0007) [2023-12-27 08:12:23,776][20381] Updated weights for policy 1, policy_version 370521 (0.0007) [2023-12-27 08:12:23,833][20381] Updated weights for policy 1, policy_version 370531 (0.0006) [2023-12-27 08:12:23,878][20381] Updated weights for policy 1, policy_version 370541 (0.0005) [2023-12-27 08:12:24,017][20347] Updated weights for policy 0, policy_version 370728 (0.0007) [2023-12-27 08:12:24,073][20347] Updated weights for policy 0, policy_version 370738 (0.0007) [2023-12-27 08:12:24,133][20347] Updated weights for policy 0, policy_version 370748 (0.0008) [2023-12-27 08:12:24,536][20381] Updated weights for policy 1, policy_version 370551 (0.0005) [2023-12-27 08:12:24,602][20381] Updated weights for policy 1, policy_version 370561 (0.0006) [2023-12-27 08:12:24,664][20381] Updated weights for policy 1, policy_version 370571 (0.0006) [2023-12-27 08:12:24,800][20347] Updated weights for policy 0, policy_version 370758 (0.0007) [2023-12-27 08:12:24,845][20347] Updated weights for policy 0, policy_version 370768 (0.0007) [2023-12-27 08:12:24,899][20347] Updated weights for policy 0, policy_version 370778 (0.0008) [2023-12-27 08:12:25,221][20381] Updated weights for policy 1, policy_version 370581 (0.0006) [2023-12-27 08:12:25,291][20381] Updated weights for policy 1, policy_version 370591 (0.0006) [2023-12-27 08:12:25,345][20381] Updated weights for policy 1, policy_version 370601 (0.0007) [2023-12-27 08:12:25,666][20347] Updated weights for policy 0, policy_version 370788 (0.0008) [2023-12-27 08:12:25,721][20347] Updated weights for policy 0, policy_version 370798 (0.0005) [2023-12-27 08:12:25,780][20347] Updated weights for policy 0, policy_version 370808 (0.0005) [2023-12-27 08:12:25,943][20381] Updated weights for policy 1, policy_version 370611 (0.0008) [2023-12-27 08:12:26,004][20381] Updated weights for policy 1, policy_version 370621 (0.0007) [2023-12-27 08:12:26,059][20381] Updated weights for policy 1, policy_version 370631 (0.0007) [2023-12-27 08:12:26,389][20347] Updated weights for policy 0, policy_version 370818 (0.0007) [2023-12-27 08:12:26,448][20347] Updated weights for policy 0, policy_version 370828 (0.0005) [2023-12-27 08:12:26,501][20347] Updated weights for policy 0, policy_version 370838 (0.0005) [2023-12-27 08:12:26,517][20291] KL-divergence is very high: 196.6483 [2023-12-27 08:12:26,550][20347] Updated weights for policy 0, policy_version 370848 (0.0005) [2023-12-27 08:12:26,813][20381] Updated weights for policy 1, policy_version 370641 (0.0008) [2023-12-27 08:12:26,864][20381] Updated weights for policy 1, policy_version 370651 (0.0008) [2023-12-27 08:12:26,928][20381] Updated weights for policy 1, policy_version 370661 (0.0008) [2023-12-27 08:12:26,993][20381] Updated weights for policy 1, policy_version 370671 (0.0006) [2023-12-27 08:12:27,126][20347] Updated weights for policy 0, policy_version 370858 (0.0008) [2023-12-27 08:12:27,188][20347] Updated weights for policy 0, policy_version 370868 (0.0008) [2023-12-27 08:12:27,248][20347] Updated weights for policy 0, policy_version 370878 (0.0007) [2023-12-27 08:12:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 189874176. Throughput: 0: 10339.8, 1: 10606.7. Samples: 189859984. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:27,363][19170] Avg episode reward: [(0, '896.710'), (1, '1000.000')] [2023-12-27 08:12:27,605][20381] Updated weights for policy 1, policy_version 370681 (0.0006) [2023-12-27 08:12:27,653][20381] Updated weights for policy 1, policy_version 370691 (0.0005) [2023-12-27 08:12:27,698][20381] Updated weights for policy 1, policy_version 370701 (0.0005) [2023-12-27 08:12:27,786][20347] Updated weights for policy 0, policy_version 370888 (0.0006) [2023-12-27 08:12:27,831][20347] Updated weights for policy 0, policy_version 370898 (0.0005) [2023-12-27 08:12:27,877][20347] Updated weights for policy 0, policy_version 370908 (0.0005) [2023-12-27 08:12:28,226][20381] Updated weights for policy 1, policy_version 370711 (0.0005) [2023-12-27 08:12:28,273][20381] Updated weights for policy 1, policy_version 370721 (0.0005) [2023-12-27 08:12:28,320][20381] Updated weights for policy 1, policy_version 370731 (0.0005) [2023-12-27 08:12:28,445][20347] Updated weights for policy 0, policy_version 370918 (0.0008) [2023-12-27 08:12:28,490][20347] Updated weights for policy 0, policy_version 370928 (0.0010) [2023-12-27 08:12:28,535][20347] Updated weights for policy 0, policy_version 370938 (0.0010) [2023-12-27 08:12:28,894][20381] Updated weights for policy 1, policy_version 370741 (0.0008) [2023-12-27 08:12:28,941][20381] Updated weights for policy 1, policy_version 370751 (0.0010) [2023-12-27 08:12:28,988][20381] Updated weights for policy 1, policy_version 370761 (0.0008) [2023-12-27 08:12:29,213][20347] Updated weights for policy 0, policy_version 370948 (0.0009) [2023-12-27 08:12:29,275][20347] Updated weights for policy 0, policy_version 370958 (0.0009) [2023-12-27 08:12:29,341][20347] Updated weights for policy 0, policy_version 370968 (0.0008) [2023-12-27 08:12:29,648][20381] Updated weights for policy 1, policy_version 370771 (0.0010) [2023-12-27 08:12:29,703][20381] Updated weights for policy 1, policy_version 370781 (0.0009) [2023-12-27 08:12:29,755][20381] Updated weights for policy 1, policy_version 370791 (0.0010) [2023-12-27 08:12:29,981][20347] Updated weights for policy 0, policy_version 370978 (0.0009) [2023-12-27 08:12:30,036][20347] Updated weights for policy 0, policy_version 370988 (0.0007) [2023-12-27 08:12:30,099][20347] Updated weights for policy 0, policy_version 370998 (0.0006) [2023-12-27 08:12:30,157][20347] Updated weights for policy 0, policy_version 371008 (0.0008) [2023-12-27 08:12:30,456][20381] Updated weights for policy 1, policy_version 370801 (0.0010) [2023-12-27 08:12:30,510][20381] Updated weights for policy 1, policy_version 370811 (0.0008) [2023-12-27 08:12:30,561][20381] Updated weights for policy 1, policy_version 370821 (0.0006) [2023-12-27 08:12:30,609][20381] Updated weights for policy 1, policy_version 370831 (0.0008) [2023-12-27 08:12:30,735][20347] Updated weights for policy 0, policy_version 371018 (0.0008) [2023-12-27 08:12:30,792][20347] Updated weights for policy 0, policy_version 371028 (0.0008) [2023-12-27 08:12:30,848][20347] Updated weights for policy 0, policy_version 371038 (0.0008) [2023-12-27 08:12:31,258][20381] Updated weights for policy 1, policy_version 370841 (0.0008) [2023-12-27 08:12:31,322][20381] Updated weights for policy 1, policy_version 370851 (0.0006) [2023-12-27 08:12:31,395][20381] Updated weights for policy 1, policy_version 370861 (0.0008) [2023-12-27 08:12:31,587][20347] Updated weights for policy 0, policy_version 371048 (0.0008) [2023-12-27 08:12:31,650][20347] Updated weights for policy 0, policy_version 371058 (0.0007) [2023-12-27 08:12:31,703][20347] Updated weights for policy 0, policy_version 371068 (0.0008) [2023-12-27 08:12:32,067][20381] Updated weights for policy 1, policy_version 370871 (0.0009) [2023-12-27 08:12:32,126][20381] Updated weights for policy 1, policy_version 370881 (0.0009) [2023-12-27 08:12:32,174][20381] Updated weights for policy 1, policy_version 370891 (0.0006) [2023-12-27 08:12:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 189980672. Throughput: 0: 10363.6, 1: 10652.6. Samples: 189990924. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:32,363][19170] Avg episode reward: [(0, '866.920'), (1, '1000.000')] [2023-12-27 08:12:32,421][20347] Updated weights for policy 0, policy_version 371078 (0.0007) [2023-12-27 08:12:32,422][20291] KL-divergence is very high: 363.3896 [2023-12-27 08:12:32,478][20347] Updated weights for policy 0, policy_version 371088 (0.0006) [2023-12-27 08:12:32,538][20347] Updated weights for policy 0, policy_version 371098 (0.0006) [2023-12-27 08:12:32,759][20381] Updated weights for policy 1, policy_version 370901 (0.0006) [2023-12-27 08:12:32,819][20381] Updated weights for policy 1, policy_version 370911 (0.0006) [2023-12-27 08:12:32,876][20381] Updated weights for policy 1, policy_version 370921 (0.0006) [2023-12-27 08:12:33,121][20347] Updated weights for policy 0, policy_version 371108 (0.0006) [2023-12-27 08:12:33,173][20347] Updated weights for policy 0, policy_version 371118 (0.0008) [2023-12-27 08:12:33,221][20347] Updated weights for policy 0, policy_version 371128 (0.0009) [2023-12-27 08:12:33,465][20381] Updated weights for policy 1, policy_version 370931 (0.0005) [2023-12-27 08:12:33,513][20381] Updated weights for policy 1, policy_version 370941 (0.0005) [2023-12-27 08:12:33,566][20381] Updated weights for policy 1, policy_version 370951 (0.0006) [2023-12-27 08:12:33,868][20347] Updated weights for policy 0, policy_version 371138 (0.0007) [2023-12-27 08:12:33,928][20347] Updated weights for policy 0, policy_version 371148 (0.0006) [2023-12-27 08:12:33,986][20347] Updated weights for policy 0, policy_version 371158 (0.0009) [2023-12-27 08:12:34,039][20347] Updated weights for policy 0, policy_version 371168 (0.0008) [2023-12-27 08:12:34,081][20381] Updated weights for policy 1, policy_version 370961 (0.0005) [2023-12-27 08:12:34,136][20381] Updated weights for policy 1, policy_version 370971 (0.0005) [2023-12-27 08:12:34,197][20381] Updated weights for policy 1, policy_version 370981 (0.0006) [2023-12-27 08:12:34,249][20381] Updated weights for policy 1, policy_version 370991 (0.0006) [2023-12-27 08:12:34,735][20347] Updated weights for policy 0, policy_version 371178 (0.0006) [2023-12-27 08:12:34,794][20347] Updated weights for policy 0, policy_version 371188 (0.0007) [2023-12-27 08:12:34,851][20347] Updated weights for policy 0, policy_version 371198 (0.0006) [2023-12-27 08:12:34,873][20381] Updated weights for policy 1, policy_version 371001 (0.0008) [2023-12-27 08:12:34,936][20381] Updated weights for policy 1, policy_version 371011 (0.0008) [2023-12-27 08:12:34,994][20381] Updated weights for policy 1, policy_version 371021 (0.0007) [2023-12-27 08:12:35,520][20347] Updated weights for policy 0, policy_version 371208 (0.0007) [2023-12-27 08:12:35,572][20347] Updated weights for policy 0, policy_version 371218 (0.0007) [2023-12-27 08:12:35,625][20347] Updated weights for policy 0, policy_version 371228 (0.0008) [2023-12-27 08:12:35,654][20381] Updated weights for policy 1, policy_version 371031 (0.0006) [2023-12-27 08:12:35,707][20381] Updated weights for policy 1, policy_version 371041 (0.0005) [2023-12-27 08:12:35,765][20381] Updated weights for policy 1, policy_version 371051 (0.0005) [2023-12-27 08:12:36,319][20381] Updated weights for policy 1, policy_version 371061 (0.0006) [2023-12-27 08:12:36,348][20347] Updated weights for policy 0, policy_version 371238 (0.0006) [2023-12-27 08:12:36,379][20381] Updated weights for policy 1, policy_version 371071 (0.0007) [2023-12-27 08:12:36,401][20347] Updated weights for policy 0, policy_version 371248 (0.0008) [2023-12-27 08:12:36,432][20381] Updated weights for policy 1, policy_version 371081 (0.0007) [2023-12-27 08:12:36,458][20347] Updated weights for policy 0, policy_version 371258 (0.0009) [2023-12-27 08:12:37,023][20381] Updated weights for policy 1, policy_version 371091 (0.0006) [2023-12-27 08:12:37,086][20381] Updated weights for policy 1, policy_version 371101 (0.0006) [2023-12-27 08:12:37,148][20381] Updated weights for policy 1, policy_version 371111 (0.0006) [2023-12-27 08:12:37,232][20347] Updated weights for policy 0, policy_version 371268 (0.0007) [2023-12-27 08:12:37,298][20347] Updated weights for policy 0, policy_version 371278 (0.0006) [2023-12-27 08:12:37,352][20347] Updated weights for policy 0, policy_version 371288 (0.0007) [2023-12-27 08:12:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 21162.6, 300 sec: 20938.2). Total num frames: 190087168. Throughput: 0: 10406.9, 1: 10684.5. Samples: 190057316. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:37,363][19170] Avg episode reward: [(0, '904.580'), (1, '1000.000')] [2023-12-27 08:12:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000371120_95027200.pth... [2023-12-27 08:12:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000369840_94699520.pth [2023-12-27 08:12:37,399][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000371296_95068160.pth... [2023-12-27 08:12:37,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000370080_94756864.pth [2023-12-27 08:12:37,748][20381] Updated weights for policy 1, policy_version 371121 (0.0005) [2023-12-27 08:12:37,807][20381] Updated weights for policy 1, policy_version 371131 (0.0006) [2023-12-27 08:12:37,866][20381] Updated weights for policy 1, policy_version 371141 (0.0007) [2023-12-27 08:12:37,919][20381] Updated weights for policy 1, policy_version 371151 (0.0007) [2023-12-27 08:12:38,004][20347] Updated weights for policy 0, policy_version 371298 (0.0007) [2023-12-27 08:12:38,069][20347] Updated weights for policy 0, policy_version 371308 (0.0009) [2023-12-27 08:12:38,133][20347] Updated weights for policy 0, policy_version 371318 (0.0010) [2023-12-27 08:12:38,190][20347] Updated weights for policy 0, policy_version 371328 (0.0010) [2023-12-27 08:12:38,514][20381] Updated weights for policy 1, policy_version 371161 (0.0007) [2023-12-27 08:12:38,568][20381] Updated weights for policy 1, policy_version 371171 (0.0007) [2023-12-27 08:12:38,617][20381] Updated weights for policy 1, policy_version 371181 (0.0007) [2023-12-27 08:12:38,837][20347] Updated weights for policy 0, policy_version 371338 (0.0011) [2023-12-27 08:12:38,892][20347] Updated weights for policy 0, policy_version 371348 (0.0010) [2023-12-27 08:12:38,944][20347] Updated weights for policy 0, policy_version 371358 (0.0008) [2023-12-27 08:12:39,287][20381] Updated weights for policy 1, policy_version 371191 (0.0007) [2023-12-27 08:12:39,349][20381] Updated weights for policy 1, policy_version 371201 (0.0008) [2023-12-27 08:12:39,416][20381] Updated weights for policy 1, policy_version 371211 (0.0008) [2023-12-27 08:12:39,655][20347] Updated weights for policy 0, policy_version 371368 (0.0009) [2023-12-27 08:12:39,722][20347] Updated weights for policy 0, policy_version 371378 (0.0010) [2023-12-27 08:12:39,778][20347] Updated weights for policy 0, policy_version 371388 (0.0011) [2023-12-27 08:12:40,141][20381] Updated weights for policy 1, policy_version 371221 (0.0008) [2023-12-27 08:12:40,197][20381] Updated weights for policy 1, policy_version 371231 (0.0007) [2023-12-27 08:12:40,253][20381] Updated weights for policy 1, policy_version 371241 (0.0007) [2023-12-27 08:12:40,459][20347] Updated weights for policy 0, policy_version 371398 (0.0010) [2023-12-27 08:12:40,512][20347] Updated weights for policy 0, policy_version 371408 (0.0010) [2023-12-27 08:12:40,565][20347] Updated weights for policy 0, policy_version 371418 (0.0010) [2023-12-27 08:12:40,923][20381] Updated weights for policy 1, policy_version 371251 (0.0009) [2023-12-27 08:12:40,980][20381] Updated weights for policy 1, policy_version 371261 (0.0009) [2023-12-27 08:12:41,040][20381] Updated weights for policy 1, policy_version 371271 (0.0008) [2023-12-27 08:12:41,203][20347] Updated weights for policy 0, policy_version 371428 (0.0010) [2023-12-27 08:12:41,254][20347] Updated weights for policy 0, policy_version 371438 (0.0010) [2023-12-27 08:12:41,315][20347] Updated weights for policy 0, policy_version 371448 (0.0007) [2023-12-27 08:12:41,756][20381] Updated weights for policy 1, policy_version 371281 (0.0008) [2023-12-27 08:12:41,817][20381] Updated weights for policy 1, policy_version 371291 (0.0011) [2023-12-27 08:12:41,881][20381] Updated weights for policy 1, policy_version 371301 (0.0011) [2023-12-27 08:12:41,935][20381] Updated weights for policy 1, policy_version 371311 (0.0011) [2023-12-27 08:12:42,038][20347] Updated weights for policy 0, policy_version 371458 (0.0010) [2023-12-27 08:12:42,093][20347] Updated weights for policy 0, policy_version 371468 (0.0007) [2023-12-27 08:12:42,152][20347] Updated weights for policy 0, policy_version 371478 (0.0007) [2023-12-27 08:12:42,211][20347] Updated weights for policy 0, policy_version 371488 (0.0007) [2023-12-27 08:12:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 190193664. Throughput: 0: 10371.9, 1: 10709.3. Samples: 190181868. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:42,363][19170] Avg episode reward: [(0, '942.230'), (1, '1000.000')] [2023-12-27 08:12:42,635][20381] Updated weights for policy 1, policy_version 371321 (0.0011) [2023-12-27 08:12:42,680][20381] Updated weights for policy 1, policy_version 371331 (0.0010) [2023-12-27 08:12:42,736][20381] Updated weights for policy 1, policy_version 371341 (0.0010) [2023-12-27 08:12:42,930][20347] Updated weights for policy 0, policy_version 371498 (0.0007) [2023-12-27 08:12:42,985][20347] Updated weights for policy 0, policy_version 371508 (0.0007) [2023-12-27 08:12:43,049][20347] Updated weights for policy 0, policy_version 371518 (0.0008) [2023-12-27 08:12:43,365][20381] Updated weights for policy 1, policy_version 371351 (0.0007) [2023-12-27 08:12:43,425][20381] Updated weights for policy 1, policy_version 371361 (0.0005) [2023-12-27 08:12:43,477][20381] Updated weights for policy 1, policy_version 371371 (0.0005) [2023-12-27 08:12:43,704][20347] Updated weights for policy 0, policy_version 371528 (0.0006) [2023-12-27 08:12:43,756][20347] Updated weights for policy 0, policy_version 371538 (0.0008) [2023-12-27 08:12:43,801][20347] Updated weights for policy 0, policy_version 371548 (0.0009) [2023-12-27 08:12:44,014][20381] Updated weights for policy 1, policy_version 371381 (0.0009) [2023-12-27 08:12:44,062][20381] Updated weights for policy 1, policy_version 371391 (0.0010) [2023-12-27 08:12:44,110][20381] Updated weights for policy 1, policy_version 371401 (0.0010) [2023-12-27 08:12:44,520][20347] Updated weights for policy 0, policy_version 371558 (0.0008) [2023-12-27 08:12:44,579][20347] Updated weights for policy 0, policy_version 371568 (0.0008) [2023-12-27 08:12:44,631][20347] Updated weights for policy 0, policy_version 371578 (0.0007) [2023-12-27 08:12:44,720][20381] Updated weights for policy 1, policy_version 371411 (0.0010) [2023-12-27 08:12:44,776][20381] Updated weights for policy 1, policy_version 371421 (0.0008) [2023-12-27 08:12:44,843][20381] Updated weights for policy 1, policy_version 371431 (0.0008) [2023-12-27 08:12:45,300][20347] Updated weights for policy 0, policy_version 371588 (0.0008) [2023-12-27 08:12:45,350][20347] Updated weights for policy 0, policy_version 371598 (0.0008) [2023-12-27 08:12:45,409][20347] Updated weights for policy 0, policy_version 371608 (0.0008) [2023-12-27 08:12:45,498][20381] Updated weights for policy 1, policy_version 371441 (0.0007) [2023-12-27 08:12:45,566][20381] Updated weights for policy 1, policy_version 371451 (0.0008) [2023-12-27 08:12:45,635][20381] Updated weights for policy 1, policy_version 371461 (0.0008) [2023-12-27 08:12:45,700][20381] Updated weights for policy 1, policy_version 371471 (0.0007) [2023-12-27 08:12:46,002][20347] Updated weights for policy 0, policy_version 371618 (0.0007) [2023-12-27 08:12:46,048][20347] Updated weights for policy 0, policy_version 371628 (0.0007) [2023-12-27 08:12:46,052][20291] KL-divergence is very high: 177.1597 [2023-12-27 08:12:46,100][20347] Updated weights for policy 0, policy_version 371638 (0.0008) [2023-12-27 08:12:46,152][20347] Updated weights for policy 0, policy_version 371648 (0.0008) [2023-12-27 08:12:46,401][20381] Updated weights for policy 1, policy_version 371481 (0.0008) [2023-12-27 08:12:46,451][20381] Updated weights for policy 1, policy_version 371491 (0.0010) [2023-12-27 08:12:46,504][20381] Updated weights for policy 1, policy_version 371501 (0.0008) [2023-12-27 08:12:46,780][20291] KL-divergence is very high: 221.3380 [2023-12-27 08:12:46,822][20291] KL-divergence is very high: 266.6824 [2023-12-27 08:12:46,827][20347] Updated weights for policy 0, policy_version 371658 (0.0006) [2023-12-27 08:12:46,862][20291] KL-divergence is very high: 139.6555 [2023-12-27 08:12:46,878][20347] Updated weights for policy 0, policy_version 371668 (0.0007) [2023-12-27 08:12:46,906][20291] KL-divergence is very high: 308.5765 [2023-12-27 08:12:46,930][20347] Updated weights for policy 0, policy_version 371678 (0.0008) [2023-12-27 08:12:47,193][20381] Updated weights for policy 1, policy_version 371511 (0.0006) [2023-12-27 08:12:47,250][20381] Updated weights for policy 1, policy_version 371521 (0.0005) [2023-12-27 08:12:47,314][20381] Updated weights for policy 1, policy_version 371531 (0.0005) [2023-12-27 08:12:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21162.7, 300 sec: 20938.2). Total num frames: 190300160. Throughput: 0: 10357.9, 1: 10736.7. Samples: 190307356. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:47,363][19170] Avg episode reward: [(0, '962.090'), (1, '1000.000')] [2023-12-27 08:12:47,595][20347] Updated weights for policy 0, policy_version 371688 (0.0006) [2023-12-27 08:12:47,658][20347] Updated weights for policy 0, policy_version 371698 (0.0005) [2023-12-27 08:12:47,713][20347] Updated weights for policy 0, policy_version 371708 (0.0007) [2023-12-27 08:12:47,867][20381] Updated weights for policy 1, policy_version 371541 (0.0005) [2023-12-27 08:12:47,917][20381] Updated weights for policy 1, policy_version 371551 (0.0005) [2023-12-27 08:12:47,969][20381] Updated weights for policy 1, policy_version 371561 (0.0005) [2023-12-27 08:12:48,365][20347] Updated weights for policy 0, policy_version 371718 (0.0008) [2023-12-27 08:12:48,410][20347] Updated weights for policy 0, policy_version 371728 (0.0009) [2023-12-27 08:12:48,458][20347] Updated weights for policy 0, policy_version 371738 (0.0010) [2023-12-27 08:12:48,587][20381] Updated weights for policy 1, policy_version 371571 (0.0005) [2023-12-27 08:12:48,640][20381] Updated weights for policy 1, policy_version 371581 (0.0011) [2023-12-27 08:12:48,693][20381] Updated weights for policy 1, policy_version 371591 (0.0011) [2023-12-27 08:12:49,153][20347] Updated weights for policy 0, policy_version 371748 (0.0008) [2023-12-27 08:12:49,202][20347] Updated weights for policy 0, policy_version 371758 (0.0005) [2023-12-27 08:12:49,275][20347] Updated weights for policy 0, policy_version 371768 (0.0010) [2023-12-27 08:12:49,385][20381] Updated weights for policy 1, policy_version 371601 (0.0010) [2023-12-27 08:12:49,438][20381] Updated weights for policy 1, policy_version 371611 (0.0007) [2023-12-27 08:12:49,502][20381] Updated weights for policy 1, policy_version 371621 (0.0008) [2023-12-27 08:12:49,563][20381] Updated weights for policy 1, policy_version 371631 (0.0011) [2023-12-27 08:12:49,893][20347] Updated weights for policy 0, policy_version 371778 (0.0009) [2023-12-27 08:12:49,945][20291] KL-divergence is very high: 255.6920 [2023-12-27 08:12:49,950][20347] Updated weights for policy 0, policy_version 371788 (0.0007) [2023-12-27 08:12:49,957][20291] KL-divergence is very high: 144.2956 [2023-12-27 08:12:50,009][20347] Updated weights for policy 0, policy_version 371798 (0.0007) [2023-12-27 08:12:50,061][20347] Updated weights for policy 0, policy_version 371808 (0.0007) [2023-12-27 08:12:50,204][20381] Updated weights for policy 1, policy_version 371641 (0.0011) [2023-12-27 08:12:50,261][20381] Updated weights for policy 1, policy_version 371651 (0.0010) [2023-12-27 08:12:50,320][20381] Updated weights for policy 1, policy_version 371661 (0.0011) [2023-12-27 08:12:50,800][20347] Updated weights for policy 0, policy_version 371818 (0.0007) [2023-12-27 08:12:50,801][20291] KL-divergence is very high: 159.2062 [2023-12-27 08:12:50,854][20347] Updated weights for policy 0, policy_version 371828 (0.0006) [2023-12-27 08:12:50,901][20347] Updated weights for policy 0, policy_version 371838 (0.0005) [2023-12-27 08:12:50,976][20381] Updated weights for policy 1, policy_version 371671 (0.0010) [2023-12-27 08:12:51,039][20381] Updated weights for policy 1, policy_version 371681 (0.0010) [2023-12-27 08:12:51,104][20381] Updated weights for policy 1, policy_version 371691 (0.0007) [2023-12-27 08:12:51,598][20347] Updated weights for policy 0, policy_version 371848 (0.0005) [2023-12-27 08:12:51,655][20347] Updated weights for policy 0, policy_version 371858 (0.0007) [2023-12-27 08:12:51,711][20347] Updated weights for policy 0, policy_version 371868 (0.0008) [2023-12-27 08:12:51,808][20381] Updated weights for policy 1, policy_version 371701 (0.0008) [2023-12-27 08:12:51,865][20381] Updated weights for policy 1, policy_version 371711 (0.0006) [2023-12-27 08:12:51,920][20381] Updated weights for policy 1, policy_version 371721 (0.0005) [2023-12-27 08:12:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 21026.2, 300 sec: 20910.4). Total num frames: 190398464. Throughput: 0: 10358.2, 1: 10777.8. Samples: 190371300. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:52,363][19170] Avg episode reward: [(0, '934.940'), (1, '989.430')] [2023-12-27 08:12:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000371728_95182848.pth... [2023-12-27 08:12:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000371872_95215616.pth... [2023-12-27 08:12:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000370480_94863360.pth [2023-12-27 08:12:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000370688_94912512.pth [2023-12-27 08:12:52,468][20347] Updated weights for policy 0, policy_version 371878 (0.0007) [2023-12-27 08:12:52,514][20347] Updated weights for policy 0, policy_version 371888 (0.0007) [2023-12-27 08:12:52,569][20347] Updated weights for policy 0, policy_version 371898 (0.0007) [2023-12-27 08:12:52,590][20381] Updated weights for policy 1, policy_version 371731 (0.0007) [2023-12-27 08:12:52,650][20381] Updated weights for policy 1, policy_version 371741 (0.0007) [2023-12-27 08:12:52,706][20381] Updated weights for policy 1, policy_version 371751 (0.0007) [2023-12-27 08:12:53,245][20347] Updated weights for policy 0, policy_version 371908 (0.0007) [2023-12-27 08:12:53,305][20347] Updated weights for policy 0, policy_version 371918 (0.0005) [2023-12-27 08:12:53,314][20381] Updated weights for policy 1, policy_version 371761 (0.0008) [2023-12-27 08:12:53,356][20347] Updated weights for policy 0, policy_version 371928 (0.0005) [2023-12-27 08:12:53,359][20381] Updated weights for policy 1, policy_version 371771 (0.0011) [2023-12-27 08:12:53,404][20381] Updated weights for policy 1, policy_version 371781 (0.0010) [2023-12-27 08:12:53,449][20381] Updated weights for policy 1, policy_version 371791 (0.0010) [2023-12-27 08:12:53,871][20347] Updated weights for policy 0, policy_version 371938 (0.0005) [2023-12-27 08:12:53,923][20347] Updated weights for policy 0, policy_version 371948 (0.0005) [2023-12-27 08:12:53,976][20347] Updated weights for policy 0, policy_version 371958 (0.0010) [2023-12-27 08:12:54,027][20347] Updated weights for policy 0, policy_version 371968 (0.0010) [2023-12-27 08:12:54,165][20381] Updated weights for policy 1, policy_version 371801 (0.0011) [2023-12-27 08:12:54,213][20381] Updated weights for policy 1, policy_version 371811 (0.0010) [2023-12-27 08:12:54,265][20381] Updated weights for policy 1, policy_version 371821 (0.0008) [2023-12-27 08:12:54,696][20347] Updated weights for policy 0, policy_version 371978 (0.0009) [2023-12-27 08:12:54,747][20347] Updated weights for policy 0, policy_version 371988 (0.0010) [2023-12-27 08:12:54,801][20347] Updated weights for policy 0, policy_version 371998 (0.0010) [2023-12-27 08:12:54,956][20381] Updated weights for policy 1, policy_version 371831 (0.0010) [2023-12-27 08:12:55,017][20381] Updated weights for policy 1, policy_version 371841 (0.0011) [2023-12-27 08:12:55,076][20381] Updated weights for policy 1, policy_version 371851 (0.0011) [2023-12-27 08:12:55,474][20347] Updated weights for policy 0, policy_version 372008 (0.0010) [2023-12-27 08:12:55,533][20347] Updated weights for policy 0, policy_version 372018 (0.0010) [2023-12-27 08:12:55,591][20347] Updated weights for policy 0, policy_version 372028 (0.0010) [2023-12-27 08:12:55,754][20381] Updated weights for policy 1, policy_version 371861 (0.0011) [2023-12-27 08:12:55,798][20381] Updated weights for policy 1, policy_version 371871 (0.0010) [2023-12-27 08:12:55,845][20381] Updated weights for policy 1, policy_version 371881 (0.0005) [2023-12-27 08:12:56,190][20347] Updated weights for policy 0, policy_version 372038 (0.0011) [2023-12-27 08:12:56,251][20347] Updated weights for policy 0, policy_version 372048 (0.0007) [2023-12-27 08:12:56,307][20347] Updated weights for policy 0, policy_version 372058 (0.0007) [2023-12-27 08:12:56,544][20381] Updated weights for policy 1, policy_version 371891 (0.0009) [2023-12-27 08:12:56,603][20381] Updated weights for policy 1, policy_version 371901 (0.0007) [2023-12-27 08:12:56,658][20381] Updated weights for policy 1, policy_version 371911 (0.0010) [2023-12-27 08:12:56,986][20347] Updated weights for policy 0, policy_version 372068 (0.0007) [2023-12-27 08:12:57,033][20347] Updated weights for policy 0, policy_version 372078 (0.0007) [2023-12-27 08:12:57,076][20347] Updated weights for policy 0, policy_version 372088 (0.0007) [2023-12-27 08:12:57,355][20381] Updated weights for policy 1, policy_version 371921 (0.0010) [2023-12-27 08:12:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 190504960. Throughput: 0: 10415.2, 1: 10668.7. Samples: 190496396. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 08:12:57,363][19170] Avg episode reward: [(0, '953.680'), (1, '941.550')] [2023-12-27 08:12:57,410][20381] Updated weights for policy 1, policy_version 371931 (0.0007) [2023-12-27 08:12:57,465][20381] Updated weights for policy 1, policy_version 371941 (0.0007) [2023-12-27 08:12:57,520][20381] Updated weights for policy 1, policy_version 371951 (0.0008) [2023-12-27 08:12:57,710][20347] Updated weights for policy 0, policy_version 372098 (0.0007) [2023-12-27 08:12:57,771][20347] Updated weights for policy 0, policy_version 372108 (0.0005) [2023-12-27 08:12:57,829][20347] Updated weights for policy 0, policy_version 372118 (0.0005) [2023-12-27 08:12:57,887][20347] Updated weights for policy 0, policy_version 372128 (0.0005) [2023-12-27 08:12:58,225][20381] Updated weights for policy 1, policy_version 371961 (0.0008) [2023-12-27 08:12:58,283][20381] Updated weights for policy 1, policy_version 371971 (0.0008) [2023-12-27 08:12:58,345][20381] Updated weights for policy 1, policy_version 371981 (0.0007) [2023-12-27 08:12:58,506][20347] Updated weights for policy 0, policy_version 372138 (0.0008) [2023-12-27 08:12:58,569][20347] Updated weights for policy 0, policy_version 372148 (0.0009) [2023-12-27 08:12:58,632][20347] Updated weights for policy 0, policy_version 372158 (0.0008) [2023-12-27 08:12:59,070][20381] Updated weights for policy 1, policy_version 371991 (0.0009) [2023-12-27 08:12:59,120][20381] Updated weights for policy 1, policy_version 372001 (0.0008) [2023-12-27 08:12:59,177][20381] Updated weights for policy 1, policy_version 372011 (0.0008) [2023-12-27 08:12:59,356][20347] Updated weights for policy 0, policy_version 372168 (0.0008) [2023-12-27 08:12:59,415][20347] Updated weights for policy 0, policy_version 372178 (0.0008) [2023-12-27 08:12:59,465][20347] Updated weights for policy 0, policy_version 372188 (0.0007) [2023-12-27 08:12:59,936][20381] Updated weights for policy 1, policy_version 372021 (0.0008) [2023-12-27 08:12:59,985][20381] Updated weights for policy 1, policy_version 372031 (0.0005) [2023-12-27 08:13:00,040][20381] Updated weights for policy 1, policy_version 372041 (0.0006) [2023-12-27 08:13:00,189][20347] Updated weights for policy 0, policy_version 372198 (0.0008) [2023-12-27 08:13:00,237][20347] Updated weights for policy 0, policy_version 372208 (0.0008) [2023-12-27 08:13:00,290][20347] Updated weights for policy 0, policy_version 372218 (0.0009) [2023-12-27 08:13:00,608][20381] Updated weights for policy 1, policy_version 372051 (0.0006) [2023-12-27 08:13:00,663][20381] Updated weights for policy 1, policy_version 372061 (0.0008) [2023-12-27 08:13:00,721][20381] Updated weights for policy 1, policy_version 372071 (0.0008) [2023-12-27 08:13:00,951][20347] Updated weights for policy 0, policy_version 372228 (0.0007) [2023-12-27 08:13:01,001][20347] Updated weights for policy 0, policy_version 372238 (0.0005) [2023-12-27 08:13:01,059][20347] Updated weights for policy 0, policy_version 372248 (0.0005) [2023-12-27 08:13:01,475][20381] Updated weights for policy 1, policy_version 372081 (0.0008) [2023-12-27 08:13:01,535][20381] Updated weights for policy 1, policy_version 372091 (0.0008) [2023-12-27 08:13:01,594][20381] Updated weights for policy 1, policy_version 372101 (0.0008) [2023-12-27 08:13:01,657][20381] Updated weights for policy 1, policy_version 372111 (0.0007) [2023-12-27 08:13:01,744][20347] Updated weights for policy 0, policy_version 372258 (0.0006) [2023-12-27 08:13:01,798][20347] Updated weights for policy 0, policy_version 372268 (0.0007) [2023-12-27 08:13:01,852][20347] Updated weights for policy 0, policy_version 372278 (0.0008) [2023-12-27 08:13:01,906][20347] Updated weights for policy 0, policy_version 372288 (0.0008) [2023-12-27 08:13:02,285][20381] Updated weights for policy 1, policy_version 372121 (0.0007) [2023-12-27 08:13:02,338][20381] Updated weights for policy 1, policy_version 372131 (0.0006) [2023-12-27 08:13:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 190603264. Throughput: 0: 10354.8, 1: 10582.5. Samples: 190618356. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:02,363][19170] Avg episode reward: [(0, '981.100'), (1, '952.170')] [2023-12-27 08:13:02,399][20381] Updated weights for policy 1, policy_version 372141 (0.0007) [2023-12-27 08:13:02,573][20347] Updated weights for policy 0, policy_version 372298 (0.0006) [2023-12-27 08:13:02,638][20347] Updated weights for policy 0, policy_version 372308 (0.0007) [2023-12-27 08:13:02,702][20347] Updated weights for policy 0, policy_version 372318 (0.0006) [2023-12-27 08:13:03,123][20381] Updated weights for policy 1, policy_version 372151 (0.0008) [2023-12-27 08:13:03,176][20381] Updated weights for policy 1, policy_version 372161 (0.0007) [2023-12-27 08:13:03,230][20381] Updated weights for policy 1, policy_version 372171 (0.0006) [2023-12-27 08:13:03,301][20347] Updated weights for policy 0, policy_version 372328 (0.0007) [2023-12-27 08:13:03,351][20347] Updated weights for policy 0, policy_version 372338 (0.0006) [2023-12-27 08:13:03,401][20347] Updated weights for policy 0, policy_version 372348 (0.0005) [2023-12-27 08:13:03,948][20381] Updated weights for policy 1, policy_version 372181 (0.0009) [2023-12-27 08:13:04,007][20381] Updated weights for policy 1, policy_version 372191 (0.0007) [2023-12-27 08:13:04,015][20347] Updated weights for policy 0, policy_version 372358 (0.0007) [2023-12-27 08:13:04,071][20381] Updated weights for policy 1, policy_version 372201 (0.0009) [2023-12-27 08:13:04,073][20347] Updated weights for policy 0, policy_version 372368 (0.0007) [2023-12-27 08:13:04,133][20347] Updated weights for policy 0, policy_version 372378 (0.0007) [2023-12-27 08:13:04,785][20381] Updated weights for policy 1, policy_version 372211 (0.0008) [2023-12-27 08:13:04,840][20347] Updated weights for policy 0, policy_version 372388 (0.0009) [2023-12-27 08:13:04,846][20381] Updated weights for policy 1, policy_version 372221 (0.0007) [2023-12-27 08:13:04,893][20347] Updated weights for policy 0, policy_version 372398 (0.0005) [2023-12-27 08:13:04,898][20381] Updated weights for policy 1, policy_version 372231 (0.0006) [2023-12-27 08:13:04,963][20347] Updated weights for policy 0, policy_version 372408 (0.0008) [2023-12-27 08:13:05,500][20381] Updated weights for policy 1, policy_version 372241 (0.0006) [2023-12-27 08:13:05,537][20292] KL-divergence is very high: 240.8220 [2023-12-27 08:13:05,546][20292] KL-divergence is very high: 174.3047 [2023-12-27 08:13:05,556][20381] Updated weights for policy 1, policy_version 372251 (0.0008) [2023-12-27 08:13:05,600][20347] Updated weights for policy 0, policy_version 372418 (0.0008) [2023-12-27 08:13:05,606][20381] Updated weights for policy 1, policy_version 372261 (0.0006) [2023-12-27 08:13:05,649][20347] Updated weights for policy 0, policy_version 372428 (0.0006) [2023-12-27 08:13:05,661][20381] Updated weights for policy 1, policy_version 372271 (0.0007) [2023-12-27 08:13:05,706][20347] Updated weights for policy 0, policy_version 372438 (0.0007) [2023-12-27 08:13:05,757][20347] Updated weights for policy 0, policy_version 372448 (0.0008) [2023-12-27 08:13:06,324][20381] Updated weights for policy 1, policy_version 372281 (0.0008) [2023-12-27 08:13:06,384][20381] Updated weights for policy 1, policy_version 372291 (0.0008) [2023-12-27 08:13:06,441][20381] Updated weights for policy 1, policy_version 372301 (0.0008) [2023-12-27 08:13:06,447][20347] Updated weights for policy 0, policy_version 372458 (0.0011) [2023-12-27 08:13:06,494][20347] Updated weights for policy 0, policy_version 372468 (0.0010) [2023-12-27 08:13:06,550][20347] Updated weights for policy 0, policy_version 372478 (0.0011) [2023-12-27 08:13:07,083][20292] KL-divergence is very high: 200.5739 [2023-12-27 08:13:07,114][20381] Updated weights for policy 1, policy_version 372311 (0.0009) [2023-12-27 08:13:07,129][20292] KL-divergence is very high: 155.5356 [2023-12-27 08:13:07,171][20381] Updated weights for policy 1, policy_version 372321 (0.0008) [2023-12-27 08:13:07,215][20347] Updated weights for policy 0, policy_version 372488 (0.0010) [2023-12-27 08:13:07,224][20381] Updated weights for policy 1, policy_version 372331 (0.0006) [2023-12-27 08:13:07,271][20347] Updated weights for policy 0, policy_version 372498 (0.0010) [2023-12-27 08:13:07,329][20347] Updated weights for policy 0, policy_version 372508 (0.0010) [2023-12-27 08:13:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.6, 300 sec: 20938.2). Total num frames: 190717952. Throughput: 0: 10388.8, 1: 10555.4. Samples: 190679284. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:07,363][19170] Avg episode reward: [(0, '979.670'), (1, '941.920')] [2023-12-27 08:13:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000372336_95338496.pth... [2023-12-27 08:13:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000372512_95379456.pth... [2023-12-27 08:13:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000371120_95027200.pth [2023-12-27 08:13:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000371296_95068160.pth [2023-12-27 08:13:07,758][20381] Updated weights for policy 1, policy_version 372341 (0.0005) [2023-12-27 08:13:07,811][20381] Updated weights for policy 1, policy_version 372351 (0.0005) [2023-12-27 08:13:07,866][20381] Updated weights for policy 1, policy_version 372361 (0.0005) [2023-12-27 08:13:07,992][20347] Updated weights for policy 0, policy_version 372518 (0.0011) [2023-12-27 08:13:08,044][20347] Updated weights for policy 0, policy_version 372528 (0.0010) [2023-12-27 08:13:08,088][20347] Updated weights for policy 0, policy_version 372538 (0.0007) [2023-12-27 08:13:08,511][20381] Updated weights for policy 1, policy_version 372371 (0.0009) [2023-12-27 08:13:08,567][20381] Updated weights for policy 1, policy_version 372381 (0.0007) [2023-12-27 08:13:08,617][20381] Updated weights for policy 1, policy_version 372391 (0.0008) [2023-12-27 08:13:08,757][20347] Updated weights for policy 0, policy_version 372548 (0.0008) [2023-12-27 08:13:08,824][20347] Updated weights for policy 0, policy_version 372558 (0.0008) [2023-12-27 08:13:08,897][20347] Updated weights for policy 0, policy_version 372568 (0.0008) [2023-12-27 08:13:09,317][20381] Updated weights for policy 1, policy_version 372401 (0.0008) [2023-12-27 08:13:09,381][20381] Updated weights for policy 1, policy_version 372411 (0.0010) [2023-12-27 08:13:09,443][20381] Updated weights for policy 1, policy_version 372421 (0.0011) [2023-12-27 08:13:09,505][20381] Updated weights for policy 1, policy_version 372431 (0.0011) [2023-12-27 08:13:09,567][20347] Updated weights for policy 0, policy_version 372578 (0.0007) [2023-12-27 08:13:09,616][20347] Updated weights for policy 0, policy_version 372588 (0.0009) [2023-12-27 08:13:09,664][20347] Updated weights for policy 0, policy_version 372598 (0.0007) [2023-12-27 08:13:09,712][20347] Updated weights for policy 0, policy_version 372608 (0.0007) [2023-12-27 08:13:10,153][20381] Updated weights for policy 1, policy_version 372441 (0.0006) [2023-12-27 08:13:10,217][20381] Updated weights for policy 1, policy_version 372451 (0.0006) [2023-12-27 08:13:10,277][20381] Updated weights for policy 1, policy_version 372461 (0.0005) [2023-12-27 08:13:10,486][20347] Updated weights for policy 0, policy_version 372618 (0.0008) [2023-12-27 08:13:10,543][20347] Updated weights for policy 0, policy_version 372628 (0.0007) [2023-12-27 08:13:10,600][20347] Updated weights for policy 0, policy_version 372638 (0.0008) [2023-12-27 08:13:10,841][20381] Updated weights for policy 1, policy_version 372471 (0.0006) [2023-12-27 08:13:10,895][20381] Updated weights for policy 1, policy_version 372481 (0.0007) [2023-12-27 08:13:10,943][20381] Updated weights for policy 1, policy_version 372491 (0.0009) [2023-12-27 08:13:11,310][20347] Updated weights for policy 0, policy_version 372648 (0.0006) [2023-12-27 08:13:11,377][20347] Updated weights for policy 0, policy_version 372658 (0.0008) [2023-12-27 08:13:11,429][20347] Updated weights for policy 0, policy_version 372668 (0.0005) [2023-12-27 08:13:11,719][20381] Updated weights for policy 1, policy_version 372501 (0.0009) [2023-12-27 08:13:11,787][20381] Updated weights for policy 1, policy_version 372511 (0.0009) [2023-12-27 08:13:11,843][20381] Updated weights for policy 1, policy_version 372521 (0.0008) [2023-12-27 08:13:12,157][20347] Updated weights for policy 0, policy_version 372678 (0.0007) [2023-12-27 08:13:12,211][20347] Updated weights for policy 0, policy_version 372688 (0.0007) [2023-12-27 08:13:12,271][20347] Updated weights for policy 0, policy_version 372698 (0.0007) [2023-12-27 08:13:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 190816256. Throughput: 0: 10408.8, 1: 10596.4. Samples: 190805220. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:12,363][19170] Avg episode reward: [(0, '946.660'), (1, '941.870')] [2023-12-27 08:13:12,571][20381] Updated weights for policy 1, policy_version 372531 (0.0008) [2023-12-27 08:13:12,646][20381] Updated weights for policy 1, policy_version 372541 (0.0008) [2023-12-27 08:13:12,707][20381] Updated weights for policy 1, policy_version 372551 (0.0007) [2023-12-27 08:13:12,942][20347] Updated weights for policy 0, policy_version 372708 (0.0009) [2023-12-27 08:13:12,991][20347] Updated weights for policy 0, policy_version 372718 (0.0008) [2023-12-27 08:13:13,043][20347] Updated weights for policy 0, policy_version 372728 (0.0007) [2023-12-27 08:13:13,355][20381] Updated weights for policy 1, policy_version 372561 (0.0007) [2023-12-27 08:13:13,414][20381] Updated weights for policy 1, policy_version 372571 (0.0007) [2023-12-27 08:13:13,474][20381] Updated weights for policy 1, policy_version 372581 (0.0008) [2023-12-27 08:13:13,532][20381] Updated weights for policy 1, policy_version 372591 (0.0008) [2023-12-27 08:13:13,759][20347] Updated weights for policy 0, policy_version 372738 (0.0008) [2023-12-27 08:13:13,807][20347] Updated weights for policy 0, policy_version 372748 (0.0008) [2023-12-27 08:13:13,862][20347] Updated weights for policy 0, policy_version 372758 (0.0007) [2023-12-27 08:13:13,871][20291] KL-divergence is very high: 115.2975 [2023-12-27 08:13:13,877][20291] KL-divergence is very high: 123.3756 [2023-12-27 08:13:13,911][20347] Updated weights for policy 0, policy_version 372768 (0.0007) [2023-12-27 08:13:14,167][20381] Updated weights for policy 1, policy_version 372601 (0.0006) [2023-12-27 08:13:14,220][20381] Updated weights for policy 1, policy_version 372611 (0.0008) [2023-12-27 08:13:14,272][20381] Updated weights for policy 1, policy_version 372621 (0.0008) [2023-12-27 08:13:14,647][20347] Updated weights for policy 0, policy_version 372778 (0.0008) [2023-12-27 08:13:14,703][20347] Updated weights for policy 0, policy_version 372788 (0.0008) [2023-12-27 08:13:14,761][20347] Updated weights for policy 0, policy_version 372798 (0.0008) [2023-12-27 08:13:14,861][20381] Updated weights for policy 1, policy_version 372631 (0.0008) [2023-12-27 08:13:14,926][20381] Updated weights for policy 1, policy_version 372641 (0.0006) [2023-12-27 08:13:14,994][20381] Updated weights for policy 1, policy_version 372651 (0.0006) [2023-12-27 08:13:15,486][20347] Updated weights for policy 0, policy_version 372808 (0.0007) [2023-12-27 08:13:15,546][20347] Updated weights for policy 0, policy_version 372818 (0.0007) [2023-12-27 08:13:15,559][20381] Updated weights for policy 1, policy_version 372661 (0.0008) [2023-12-27 08:13:15,598][20347] Updated weights for policy 0, policy_version 372828 (0.0006) [2023-12-27 08:13:15,618][20381] Updated weights for policy 1, policy_version 372671 (0.0011) [2023-12-27 08:13:15,681][20381] Updated weights for policy 1, policy_version 372681 (0.0011) [2023-12-27 08:13:16,305][20347] Updated weights for policy 0, policy_version 372838 (0.0006) [2023-12-27 08:13:16,352][20381] Updated weights for policy 1, policy_version 372691 (0.0011) [2023-12-27 08:13:16,356][20347] Updated weights for policy 0, policy_version 372848 (0.0008) [2023-12-27 08:13:16,401][20381] Updated weights for policy 1, policy_version 372701 (0.0010) [2023-12-27 08:13:16,406][20347] Updated weights for policy 0, policy_version 372858 (0.0006) [2023-12-27 08:13:16,459][20381] Updated weights for policy 1, policy_version 372711 (0.0011) [2023-12-27 08:13:17,083][20347] Updated weights for policy 0, policy_version 372868 (0.0005) [2023-12-27 08:13:17,140][20347] Updated weights for policy 0, policy_version 372878 (0.0006) [2023-12-27 08:13:17,141][20381] Updated weights for policy 1, policy_version 372721 (0.0010) [2023-12-27 08:13:17,194][20381] Updated weights for policy 1, policy_version 372731 (0.0006) [2023-12-27 08:13:17,208][20347] Updated weights for policy 0, policy_version 372888 (0.0009) [2023-12-27 08:13:17,247][20381] Updated weights for policy 1, policy_version 372741 (0.0007) [2023-12-27 08:13:17,294][20381] Updated weights for policy 1, policy_version 372751 (0.0010) [2023-12-27 08:13:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 190922752. Throughput: 0: 10257.1, 1: 10551.0. Samples: 190927284. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:17,363][19170] Avg episode reward: [(0, '883.600'), (1, '980.680')] [2023-12-27 08:13:17,752][20347] Updated weights for policy 0, policy_version 372898 (0.0009) [2023-12-27 08:13:17,810][20347] Updated weights for policy 0, policy_version 372908 (0.0008) [2023-12-27 08:13:17,862][20347] Updated weights for policy 0, policy_version 372918 (0.0008) [2023-12-27 08:13:17,906][20381] Updated weights for policy 1, policy_version 372761 (0.0011) [2023-12-27 08:13:17,914][20347] Updated weights for policy 0, policy_version 372928 (0.0005) [2023-12-27 08:13:17,963][20381] Updated weights for policy 1, policy_version 372771 (0.0011) [2023-12-27 08:13:18,015][20381] Updated weights for policy 1, policy_version 372781 (0.0010) [2023-12-27 08:13:18,566][20347] Updated weights for policy 0, policy_version 372938 (0.0005) [2023-12-27 08:13:18,629][20347] Updated weights for policy 0, policy_version 372948 (0.0006) [2023-12-27 08:13:18,684][20347] Updated weights for policy 0, policy_version 372958 (0.0010) [2023-12-27 08:13:18,722][20381] Updated weights for policy 1, policy_version 372791 (0.0010) [2023-12-27 08:13:18,777][20381] Updated weights for policy 1, policy_version 372801 (0.0011) [2023-12-27 08:13:18,831][20381] Updated weights for policy 1, policy_version 372811 (0.0010) [2023-12-27 08:13:19,330][20347] Updated weights for policy 0, policy_version 372968 (0.0008) [2023-12-27 08:13:19,390][20347] Updated weights for policy 0, policy_version 372978 (0.0007) [2023-12-27 08:13:19,444][20347] Updated weights for policy 0, policy_version 372988 (0.0008) [2023-12-27 08:13:19,497][20381] Updated weights for policy 1, policy_version 372821 (0.0009) [2023-12-27 08:13:19,558][20381] Updated weights for policy 1, policy_version 372831 (0.0006) [2023-12-27 08:13:19,613][20381] Updated weights for policy 1, policy_version 372841 (0.0009) [2023-12-27 08:13:20,181][20347] Updated weights for policy 0, policy_version 372998 (0.0007) [2023-12-27 08:13:20,244][20347] Updated weights for policy 0, policy_version 373008 (0.0009) [2023-12-27 08:13:20,296][20347] Updated weights for policy 0, policy_version 373018 (0.0006) [2023-12-27 08:13:20,330][20381] Updated weights for policy 1, policy_version 372851 (0.0007) [2023-12-27 08:13:20,384][20381] Updated weights for policy 1, policy_version 372861 (0.0008) [2023-12-27 08:13:20,434][20381] Updated weights for policy 1, policy_version 372871 (0.0008) [2023-12-27 08:13:21,021][20347] Updated weights for policy 0, policy_version 373028 (0.0007) [2023-12-27 08:13:21,078][20347] Updated weights for policy 0, policy_version 373038 (0.0009) [2023-12-27 08:13:21,083][20381] Updated weights for policy 1, policy_version 372881 (0.0008) [2023-12-27 08:13:21,136][20347] Updated weights for policy 0, policy_version 373048 (0.0007) [2023-12-27 08:13:21,142][20381] Updated weights for policy 1, policy_version 372891 (0.0008) [2023-12-27 08:13:21,188][20381] Updated weights for policy 1, policy_version 372901 (0.0008) [2023-12-27 08:13:21,203][20292] KL-divergence is very high: 578.3821 [2023-12-27 08:13:21,238][20381] Updated weights for policy 1, policy_version 372911 (0.0008) [2023-12-27 08:13:21,815][20347] Updated weights for policy 0, policy_version 373058 (0.0006) [2023-12-27 08:13:21,869][20347] Updated weights for policy 0, policy_version 373068 (0.0008) [2023-12-27 08:13:21,928][20347] Updated weights for policy 0, policy_version 373078 (0.0007) [2023-12-27 08:13:21,977][20381] Updated weights for policy 1, policy_version 372921 (0.0006) [2023-12-27 08:13:21,989][20347] Updated weights for policy 0, policy_version 373088 (0.0009) [2023-12-27 08:13:22,030][20381] Updated weights for policy 1, policy_version 372931 (0.0007) [2023-12-27 08:13:22,086][20381] Updated weights for policy 1, policy_version 372941 (0.0005) [2023-12-27 08:13:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 191021056. Throughput: 0: 10270.9, 1: 10458.6. Samples: 190990140. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:22,363][19170] Avg episode reward: [(0, '901.590'), (1, '961.240')] [2023-12-27 08:13:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000372944_95494144.pth... [2023-12-27 08:13:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000373088_95526912.pth... [2023-12-27 08:13:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000371728_95182848.pth [2023-12-27 08:13:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000371872_95215616.pth [2023-12-27 08:13:22,712][20347] Updated weights for policy 0, policy_version 373098 (0.0011) [2023-12-27 08:13:22,720][20381] Updated weights for policy 1, policy_version 372951 (0.0009) [2023-12-27 08:13:22,770][20347] Updated weights for policy 0, policy_version 373108 (0.0011) [2023-12-27 08:13:22,781][20381] Updated weights for policy 1, policy_version 372961 (0.0006) [2023-12-27 08:13:22,830][20347] Updated weights for policy 0, policy_version 373118 (0.0011) [2023-12-27 08:13:22,845][20381] Updated weights for policy 1, policy_version 372971 (0.0007) [2023-12-27 08:13:23,462][20381] Updated weights for policy 1, policy_version 372981 (0.0011) [2023-12-27 08:13:23,511][20381] Updated weights for policy 1, policy_version 372991 (0.0010) [2023-12-27 08:13:23,516][20347] Updated weights for policy 0, policy_version 373128 (0.0010) [2023-12-27 08:13:23,559][20381] Updated weights for policy 1, policy_version 373001 (0.0010) [2023-12-27 08:13:23,573][20347] Updated weights for policy 0, policy_version 373138 (0.0010) [2023-12-27 08:13:23,624][20347] Updated weights for policy 0, policy_version 373148 (0.0010) [2023-12-27 08:13:24,133][20381] Updated weights for policy 1, policy_version 373011 (0.0009) [2023-12-27 08:13:24,191][20381] Updated weights for policy 1, policy_version 373021 (0.0007) [2023-12-27 08:13:24,235][20381] Updated weights for policy 1, policy_version 373031 (0.0010) [2023-12-27 08:13:24,294][20347] Updated weights for policy 0, policy_version 373158 (0.0010) [2023-12-27 08:13:24,351][20347] Updated weights for policy 0, policy_version 373168 (0.0010) [2023-12-27 08:13:24,408][20347] Updated weights for policy 0, policy_version 373178 (0.0009) [2023-12-27 08:13:24,787][20381] Updated weights for policy 1, policy_version 373041 (0.0006) [2023-12-27 08:13:24,848][20381] Updated weights for policy 1, policy_version 373051 (0.0010) [2023-12-27 08:13:24,905][20381] Updated weights for policy 1, policy_version 373062 (0.0011) [2023-12-27 08:13:24,961][20347] Updated weights for policy 0, policy_version 373188 (0.0007) [2023-12-27 08:13:24,963][20381] Updated weights for policy 1, policy_version 373072 (0.0005) [2023-12-27 08:13:25,019][20347] Updated weights for policy 0, policy_version 373198 (0.0010) [2023-12-27 08:13:25,064][20347] Updated weights for policy 0, policy_version 373208 (0.0009) [2023-12-27 08:13:25,554][20381] Updated weights for policy 1, policy_version 373082 (0.0005) [2023-12-27 08:13:25,619][20381] Updated weights for policy 1, policy_version 373092 (0.0006) [2023-12-27 08:13:25,673][20381] Updated weights for policy 1, policy_version 373102 (0.0011) [2023-12-27 08:13:25,740][20347] Updated weights for policy 0, policy_version 373218 (0.0009) [2023-12-27 08:13:25,804][20347] Updated weights for policy 0, policy_version 373228 (0.0010) [2023-12-27 08:13:25,867][20347] Updated weights for policy 0, policy_version 373238 (0.0010) [2023-12-27 08:13:25,925][20347] Updated weights for policy 0, policy_version 373248 (0.0010) [2023-12-27 08:13:26,328][20381] Updated weights for policy 1, policy_version 373112 (0.0011) [2023-12-27 08:13:26,382][20381] Updated weights for policy 1, policy_version 373122 (0.0010) [2023-12-27 08:13:26,438][20381] Updated weights for policy 1, policy_version 373132 (0.0010) [2023-12-27 08:13:26,589][20347] Updated weights for policy 0, policy_version 373258 (0.0006) [2023-12-27 08:13:26,634][20347] Updated weights for policy 0, policy_version 373268 (0.0006) [2023-12-27 08:13:26,678][20347] Updated weights for policy 0, policy_version 373278 (0.0005) [2023-12-27 08:13:26,987][20381] Updated weights for policy 1, policy_version 373142 (0.0006) [2023-12-27 08:13:27,043][20381] Updated weights for policy 1, policy_version 373152 (0.0005) [2023-12-27 08:13:27,097][20381] Updated weights for policy 1, policy_version 373162 (0.0005) [2023-12-27 08:13:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 191127552. Throughput: 0: 10294.8, 1: 10501.0. Samples: 191117684. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:27,363][19170] Avg episode reward: [(0, '904.040'), (1, '980.670')] [2023-12-27 08:13:27,382][20347] Updated weights for policy 0, policy_version 373288 (0.0006) [2023-12-27 08:13:27,443][20347] Updated weights for policy 0, policy_version 373298 (0.0006) [2023-12-27 08:13:27,492][20347] Updated weights for policy 0, policy_version 373308 (0.0007) [2023-12-27 08:13:27,628][20381] Updated weights for policy 1, policy_version 373172 (0.0005) [2023-12-27 08:13:27,683][20381] Updated weights for policy 1, policy_version 373182 (0.0006) [2023-12-27 08:13:27,736][20381] Updated weights for policy 1, policy_version 373192 (0.0005) [2023-12-27 08:13:28,137][20347] Updated weights for policy 0, policy_version 373318 (0.0007) [2023-12-27 08:13:28,191][20347] Updated weights for policy 0, policy_version 373328 (0.0007) [2023-12-27 08:13:28,234][20347] Updated weights for policy 0, policy_version 373338 (0.0007) [2023-12-27 08:13:28,378][20381] Updated weights for policy 1, policy_version 373202 (0.0006) [2023-12-27 08:13:28,436][20381] Updated weights for policy 1, policy_version 373212 (0.0008) [2023-12-27 08:13:28,493][20381] Updated weights for policy 1, policy_version 373222 (0.0008) [2023-12-27 08:13:28,555][20381] Updated weights for policy 1, policy_version 373232 (0.0010) [2023-12-27 08:13:28,938][20347] Updated weights for policy 0, policy_version 373348 (0.0007) [2023-12-27 08:13:28,992][20347] Updated weights for policy 0, policy_version 373358 (0.0007) [2023-12-27 08:13:29,044][20347] Updated weights for policy 0, policy_version 373368 (0.0007) [2023-12-27 08:13:29,244][20381] Updated weights for policy 1, policy_version 373242 (0.0008) [2023-12-27 08:13:29,306][20381] Updated weights for policy 1, policy_version 373252 (0.0006) [2023-12-27 08:13:29,366][20381] Updated weights for policy 1, policy_version 373262 (0.0008) [2023-12-27 08:13:29,791][20347] Updated weights for policy 0, policy_version 373378 (0.0007) [2023-12-27 08:13:29,847][20347] Updated weights for policy 0, policy_version 373388 (0.0008) [2023-12-27 08:13:29,894][20347] Updated weights for policy 0, policy_version 373398 (0.0008) [2023-12-27 08:13:29,939][20381] Updated weights for policy 1, policy_version 373272 (0.0008) [2023-12-27 08:13:29,958][20347] Updated weights for policy 0, policy_version 373408 (0.0008) [2023-12-27 08:13:29,997][20381] Updated weights for policy 1, policy_version 373282 (0.0007) [2023-12-27 08:13:30,053][20381] Updated weights for policy 1, policy_version 373292 (0.0007) [2023-12-27 08:13:30,679][20347] Updated weights for policy 0, policy_version 373418 (0.0010) [2023-12-27 08:13:30,693][20381] Updated weights for policy 1, policy_version 373302 (0.0009) [2023-12-27 08:13:30,730][20347] Updated weights for policy 0, policy_version 373428 (0.0010) [2023-12-27 08:13:30,746][20381] Updated weights for policy 1, policy_version 373312 (0.0007) [2023-12-27 08:13:30,774][20347] Updated weights for policy 0, policy_version 373438 (0.0010) [2023-12-27 08:13:30,803][20381] Updated weights for policy 1, policy_version 373322 (0.0005) [2023-12-27 08:13:31,420][20347] Updated weights for policy 0, policy_version 373448 (0.0010) [2023-12-27 08:13:31,450][20381] Updated weights for policy 1, policy_version 373332 (0.0009) [2023-12-27 08:13:31,475][20347] Updated weights for policy 0, policy_version 373458 (0.0010) [2023-12-27 08:13:31,495][20381] Updated weights for policy 1, policy_version 373342 (0.0005) [2023-12-27 08:13:31,526][20347] Updated weights for policy 0, policy_version 373468 (0.0010) [2023-12-27 08:13:31,539][20381] Updated weights for policy 1, policy_version 373352 (0.0006) [2023-12-27 08:13:32,147][20347] Updated weights for policy 0, policy_version 373478 (0.0010) [2023-12-27 08:13:32,203][20347] Updated weights for policy 0, policy_version 373488 (0.0010) [2023-12-27 08:13:32,204][20381] Updated weights for policy 1, policy_version 373362 (0.0006) [2023-12-27 08:13:32,258][20347] Updated weights for policy 0, policy_version 373498 (0.0007) [2023-12-27 08:13:32,258][20381] Updated weights for policy 1, policy_version 373372 (0.0006) [2023-12-27 08:13:32,315][20381] Updated weights for policy 1, policy_version 373382 (0.0007) [2023-12-27 08:13:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 191234048. Throughput: 0: 10293.2, 1: 10541.8. Samples: 191244932. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:32,363][19170] Avg episode reward: [(0, '894.500'), (1, '1000.000')] [2023-12-27 08:13:32,372][20381] Updated weights for policy 1, policy_version 373392 (0.0007) [2023-12-27 08:13:32,902][20347] Updated weights for policy 0, policy_version 373508 (0.0007) [2023-12-27 08:13:32,959][20347] Updated weights for policy 0, policy_version 373518 (0.0006) [2023-12-27 08:13:33,012][20347] Updated weights for policy 0, policy_version 373528 (0.0005) [2023-12-27 08:13:33,056][20381] Updated weights for policy 1, policy_version 373402 (0.0007) [2023-12-27 08:13:33,106][20381] Updated weights for policy 1, policy_version 373412 (0.0008) [2023-12-27 08:13:33,157][20381] Updated weights for policy 1, policy_version 373422 (0.0007) [2023-12-27 08:13:33,593][20347] Updated weights for policy 0, policy_version 373538 (0.0005) [2023-12-27 08:13:33,680][20347] Updated weights for policy 0, policy_version 373548 (0.0007) [2023-12-27 08:13:33,748][20347] Updated weights for policy 0, policy_version 373558 (0.0008) [2023-12-27 08:13:33,805][20381] Updated weights for policy 1, policy_version 373432 (0.0006) [2023-12-27 08:13:33,810][20347] Updated weights for policy 0, policy_version 373568 (0.0008) [2023-12-27 08:13:33,858][20381] Updated weights for policy 1, policy_version 373442 (0.0006) [2023-12-27 08:13:33,908][20381] Updated weights for policy 1, policy_version 373452 (0.0007) [2023-12-27 08:13:34,465][20347] Updated weights for policy 0, policy_version 373578 (0.0011) [2023-12-27 08:13:34,523][20347] Updated weights for policy 0, policy_version 373588 (0.0010) [2023-12-27 08:13:34,526][20381] Updated weights for policy 1, policy_version 373462 (0.0010) [2023-12-27 08:13:34,583][20347] Updated weights for policy 0, policy_version 373598 (0.0011) [2023-12-27 08:13:34,584][20381] Updated weights for policy 1, policy_version 373472 (0.0008) [2023-12-27 08:13:34,641][20381] Updated weights for policy 1, policy_version 373482 (0.0009) [2023-12-27 08:13:35,241][20347] Updated weights for policy 0, policy_version 373608 (0.0010) [2023-12-27 08:13:35,274][20381] Updated weights for policy 1, policy_version 373492 (0.0009) [2023-12-27 08:13:35,298][20347] Updated weights for policy 0, policy_version 373618 (0.0010) [2023-12-27 08:13:35,332][20381] Updated weights for policy 1, policy_version 373502 (0.0006) [2023-12-27 08:13:35,352][20347] Updated weights for policy 0, policy_version 373628 (0.0010) [2023-12-27 08:13:35,386][20381] Updated weights for policy 1, policy_version 373512 (0.0006) [2023-12-27 08:13:36,069][20347] Updated weights for policy 0, policy_version 373638 (0.0010) [2023-12-27 08:13:36,116][20381] Updated weights for policy 1, policy_version 373522 (0.0008) [2023-12-27 08:13:36,128][20347] Updated weights for policy 0, policy_version 373648 (0.0010) [2023-12-27 08:13:36,183][20381] Updated weights for policy 1, policy_version 373532 (0.0006) [2023-12-27 08:13:36,189][20347] Updated weights for policy 0, policy_version 373658 (0.0010) [2023-12-27 08:13:36,240][20381] Updated weights for policy 1, policy_version 373542 (0.0007) [2023-12-27 08:13:36,288][20381] Updated weights for policy 1, policy_version 373552 (0.0007) [2023-12-27 08:13:36,884][20347] Updated weights for policy 0, policy_version 373668 (0.0011) [2023-12-27 08:13:36,927][20381] Updated weights for policy 1, policy_version 373562 (0.0006) [2023-12-27 08:13:36,946][20347] Updated weights for policy 0, policy_version 373678 (0.0010) [2023-12-27 08:13:36,981][20381] Updated weights for policy 1, policy_version 373572 (0.0006) [2023-12-27 08:13:37,001][20347] Updated weights for policy 0, policy_version 373688 (0.0010) [2023-12-27 08:13:37,036][20381] Updated weights for policy 1, policy_version 373582 (0.0006) [2023-12-27 08:13:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.7, 300 sec: 20938.2). Total num frames: 191340544. Throughput: 0: 10310.8, 1: 10529.8. Samples: 191309124. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:37,363][19170] Avg episode reward: [(0, '579.790'), (1, '1000.000')] [2023-12-27 08:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000373584_95657984.pth... [2023-12-27 08:13:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000373696_95682560.pth... [2023-12-27 08:13:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000372336_95338496.pth [2023-12-27 08:13:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000372512_95379456.pth [2023-12-27 08:13:37,606][20347] Updated weights for policy 0, policy_version 373698 (0.0011) [2023-12-27 08:13:37,638][20381] Updated weights for policy 1, policy_version 373592 (0.0005) [2023-12-27 08:13:37,668][20347] Updated weights for policy 0, policy_version 373708 (0.0011) [2023-12-27 08:13:37,701][20381] Updated weights for policy 1, policy_version 373602 (0.0006) [2023-12-27 08:13:37,728][20347] Updated weights for policy 0, policy_version 373718 (0.0011) [2023-12-27 08:13:37,758][20381] Updated weights for policy 1, policy_version 373612 (0.0007) [2023-12-27 08:13:37,782][20347] Updated weights for policy 0, policy_version 373728 (0.0006) [2023-12-27 08:13:38,346][20347] Updated weights for policy 0, policy_version 373738 (0.0009) [2023-12-27 08:13:38,406][20381] Updated weights for policy 1, policy_version 373622 (0.0007) [2023-12-27 08:13:38,408][20347] Updated weights for policy 0, policy_version 373748 (0.0006) [2023-12-27 08:13:38,458][20381] Updated weights for policy 1, policy_version 373632 (0.0006) [2023-12-27 08:13:38,471][20347] Updated weights for policy 0, policy_version 373758 (0.0011) [2023-12-27 08:13:38,512][20381] Updated weights for policy 1, policy_version 373642 (0.0006) [2023-12-27 08:13:39,073][20347] Updated weights for policy 0, policy_version 373768 (0.0006) [2023-12-27 08:13:39,133][20347] Updated weights for policy 0, policy_version 373778 (0.0006) [2023-12-27 08:13:39,188][20381] Updated weights for policy 1, policy_version 373652 (0.0007) [2023-12-27 08:13:39,191][20347] Updated weights for policy 0, policy_version 373788 (0.0010) [2023-12-27 08:13:39,251][20381] Updated weights for policy 1, policy_version 373662 (0.0007) [2023-12-27 08:13:39,314][20381] Updated weights for policy 1, policy_version 373672 (0.0005) [2023-12-27 08:13:39,911][20347] Updated weights for policy 0, policy_version 373798 (0.0008) [2023-12-27 08:13:39,977][20347] Updated weights for policy 0, policy_version 373808 (0.0006) [2023-12-27 08:13:39,980][20381] Updated weights for policy 1, policy_version 373682 (0.0008) [2023-12-27 08:13:40,040][20381] Updated weights for policy 1, policy_version 373692 (0.0008) [2023-12-27 08:13:40,041][20347] Updated weights for policy 0, policy_version 373818 (0.0008) [2023-12-27 08:13:40,097][20381] Updated weights for policy 1, policy_version 373702 (0.0006) [2023-12-27 08:13:40,154][20381] Updated weights for policy 1, policy_version 373712 (0.0008) [2023-12-27 08:13:40,714][20347] Updated weights for policy 0, policy_version 373828 (0.0007) [2023-12-27 08:13:40,768][20347] Updated weights for policy 0, policy_version 373838 (0.0008) [2023-12-27 08:13:40,818][20347] Updated weights for policy 0, policy_version 373848 (0.0007) [2023-12-27 08:13:40,833][20381] Updated weights for policy 1, policy_version 373722 (0.0005) [2023-12-27 08:13:40,888][20381] Updated weights for policy 1, policy_version 373732 (0.0007) [2023-12-27 08:13:40,947][20381] Updated weights for policy 1, policy_version 373742 (0.0007) [2023-12-27 08:13:41,543][20347] Updated weights for policy 0, policy_version 373858 (0.0006) [2023-12-27 08:13:41,596][20347] Updated weights for policy 0, policy_version 373868 (0.0006) [2023-12-27 08:13:41,658][20347] Updated weights for policy 0, policy_version 373878 (0.0009) [2023-12-27 08:13:41,716][20347] Updated weights for policy 0, policy_version 373888 (0.0008) [2023-12-27 08:13:41,740][20381] Updated weights for policy 1, policy_version 373752 (0.0009) [2023-12-27 08:13:41,802][20381] Updated weights for policy 1, policy_version 373762 (0.0008) [2023-12-27 08:13:41,867][20381] Updated weights for policy 1, policy_version 373772 (0.0008) [2023-12-27 08:13:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20882.7). Total num frames: 191438848. Throughput: 0: 10296.7, 1: 10539.4. Samples: 191434020. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:42,363][19170] Avg episode reward: [(0, '589.020'), (1, '990.350')] [2023-12-27 08:13:42,402][20347] Updated weights for policy 0, policy_version 373898 (0.0007) [2023-12-27 08:13:42,446][20347] Updated weights for policy 0, policy_version 373908 (0.0008) [2023-12-27 08:13:42,506][20347] Updated weights for policy 0, policy_version 373918 (0.0007) [2023-12-27 08:13:42,594][20381] Updated weights for policy 1, policy_version 373782 (0.0008) [2023-12-27 08:13:42,654][20381] Updated weights for policy 1, policy_version 373792 (0.0007) [2023-12-27 08:13:42,716][20381] Updated weights for policy 1, policy_version 373802 (0.0007) [2023-12-27 08:13:43,187][20347] Updated weights for policy 0, policy_version 373928 (0.0006) [2023-12-27 08:13:43,243][20347] Updated weights for policy 0, policy_version 373938 (0.0009) [2023-12-27 08:13:43,298][20347] Updated weights for policy 0, policy_version 373948 (0.0009) [2023-12-27 08:13:43,368][20381] Updated weights for policy 1, policy_version 373812 (0.0009) [2023-12-27 08:13:43,419][20381] Updated weights for policy 1, policy_version 373822 (0.0010) [2023-12-27 08:13:43,479][20381] Updated weights for policy 1, policy_version 373832 (0.0010) [2023-12-27 08:13:43,976][20347] Updated weights for policy 0, policy_version 373958 (0.0010) [2023-12-27 08:13:44,031][20347] Updated weights for policy 0, policy_version 373968 (0.0010) [2023-12-27 08:13:44,076][20347] Updated weights for policy 0, policy_version 373978 (0.0010) [2023-12-27 08:13:44,160][20381] Updated weights for policy 1, policy_version 373842 (0.0010) [2023-12-27 08:13:44,217][20381] Updated weights for policy 1, policy_version 373852 (0.0010) [2023-12-27 08:13:44,276][20381] Updated weights for policy 1, policy_version 373862 (0.0005) [2023-12-27 08:13:44,334][20381] Updated weights for policy 1, policy_version 373872 (0.0010) [2023-12-27 08:13:44,777][20347] Updated weights for policy 0, policy_version 373988 (0.0010) [2023-12-27 08:13:44,842][20347] Updated weights for policy 0, policy_version 373998 (0.0010) [2023-12-27 08:13:44,905][20347] Updated weights for policy 0, policy_version 374008 (0.0010) [2023-12-27 08:13:45,017][20381] Updated weights for policy 1, policy_version 373882 (0.0011) [2023-12-27 08:13:45,078][20381] Updated weights for policy 1, policy_version 373892 (0.0010) [2023-12-27 08:13:45,130][20381] Updated weights for policy 1, policy_version 373902 (0.0010) [2023-12-27 08:13:45,530][20347] Updated weights for policy 0, policy_version 374018 (0.0006) [2023-12-27 08:13:45,590][20347] Updated weights for policy 0, policy_version 374028 (0.0007) [2023-12-27 08:13:45,641][20347] Updated weights for policy 0, policy_version 374038 (0.0005) [2023-12-27 08:13:45,700][20347] Updated weights for policy 0, policy_version 374048 (0.0005) [2023-12-27 08:13:45,821][20381] Updated weights for policy 1, policy_version 373912 (0.0011) [2023-12-27 08:13:45,885][20381] Updated weights for policy 1, policy_version 373922 (0.0011) [2023-12-27 08:13:45,946][20381] Updated weights for policy 1, policy_version 373932 (0.0010) [2023-12-27 08:13:46,225][20347] Updated weights for policy 0, policy_version 374058 (0.0007) [2023-12-27 08:13:46,282][20347] Updated weights for policy 0, policy_version 374068 (0.0007) [2023-12-27 08:13:46,337][20347] Updated weights for policy 0, policy_version 374078 (0.0007) [2023-12-27 08:13:46,586][20381] Updated weights for policy 1, policy_version 373942 (0.0011) [2023-12-27 08:13:46,633][20292] KL-divergence is very high: 336.8248 [2023-12-27 08:13:46,638][20381] Updated weights for policy 1, policy_version 373952 (0.0010) [2023-12-27 08:13:46,681][20292] KL-divergence is very high: 108.0113 [2023-12-27 08:13:46,696][20381] Updated weights for policy 1, policy_version 373962 (0.0010) [2023-12-27 08:13:47,015][20347] Updated weights for policy 0, policy_version 374088 (0.0007) [2023-12-27 08:13:47,063][20347] Updated weights for policy 0, policy_version 374098 (0.0008) [2023-12-27 08:13:47,107][20347] Updated weights for policy 0, policy_version 374108 (0.0007) [2023-12-27 08:13:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20910.4). Total num frames: 191545344. Throughput: 0: 10321.1, 1: 10528.1. Samples: 191556568. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:47,364][19170] Avg episode reward: [(0, '778.440'), (1, '932.930')] [2023-12-27 08:13:47,395][20381] Updated weights for policy 1, policy_version 373972 (0.0010) [2023-12-27 08:13:47,458][20381] Updated weights for policy 1, policy_version 373982 (0.0011) [2023-12-27 08:13:47,513][20381] Updated weights for policy 1, policy_version 373992 (0.0011) [2023-12-27 08:13:47,826][20347] Updated weights for policy 0, policy_version 374118 (0.0007) [2023-12-27 08:13:47,880][20347] Updated weights for policy 0, policy_version 374128 (0.0008) [2023-12-27 08:13:47,931][20347] Updated weights for policy 0, policy_version 374138 (0.0007) [2023-12-27 08:13:48,200][20381] Updated weights for policy 1, policy_version 374002 (0.0010) [2023-12-27 08:13:48,256][20381] Updated weights for policy 1, policy_version 374012 (0.0010) [2023-12-27 08:13:48,311][20381] Updated weights for policy 1, policy_version 374022 (0.0010) [2023-12-27 08:13:48,401][20381] Updated weights for policy 1, policy_version 374032 (0.0011) [2023-12-27 08:13:48,630][20347] Updated weights for policy 0, policy_version 374148 (0.0006) [2023-12-27 08:13:48,686][20347] Updated weights for policy 0, policy_version 374158 (0.0005) [2023-12-27 08:13:48,744][20347] Updated weights for policy 0, policy_version 374168 (0.0005) [2023-12-27 08:13:49,097][20381] Updated weights for policy 1, policy_version 374042 (0.0007) [2023-12-27 08:13:49,146][20381] Updated weights for policy 1, policy_version 374052 (0.0008) [2023-12-27 08:13:49,169][20292] KL-divergence is very high: 188.2452 [2023-12-27 08:13:49,185][20292] KL-divergence is very high: 151.1041 [2023-12-27 08:13:49,194][20381] Updated weights for policy 1, policy_version 374062 (0.0008) [2023-12-27 08:13:49,370][20347] Updated weights for policy 0, policy_version 374178 (0.0006) [2023-12-27 08:13:49,428][20347] Updated weights for policy 0, policy_version 374188 (0.0010) [2023-12-27 08:13:49,483][20347] Updated weights for policy 0, policy_version 374198 (0.0010) [2023-12-27 08:13:49,534][20347] Updated weights for policy 0, policy_version 374208 (0.0010) [2023-12-27 08:13:49,916][20381] Updated weights for policy 1, policy_version 374072 (0.0010) [2023-12-27 08:13:49,973][20381] Updated weights for policy 1, policy_version 374082 (0.0007) [2023-12-27 08:13:50,029][20381] Updated weights for policy 1, policy_version 374092 (0.0007) [2023-12-27 08:13:50,152][20347] Updated weights for policy 0, policy_version 374218 (0.0009) [2023-12-27 08:13:50,206][20347] Updated weights for policy 0, policy_version 374228 (0.0010) [2023-12-27 08:13:50,268][20347] Updated weights for policy 0, policy_version 374238 (0.0006) [2023-12-27 08:13:50,774][20381] Updated weights for policy 1, policy_version 374102 (0.0010) [2023-12-27 08:13:50,825][20381] Updated weights for policy 1, policy_version 374112 (0.0010) [2023-12-27 08:13:50,880][20381] Updated weights for policy 1, policy_version 374122 (0.0010) [2023-12-27 08:13:50,901][20347] Updated weights for policy 0, policy_version 374248 (0.0007) [2023-12-27 08:13:50,949][20347] Updated weights for policy 0, policy_version 374258 (0.0007) [2023-12-27 08:13:50,999][20347] Updated weights for policy 0, policy_version 374268 (0.0007) [2023-12-27 08:13:51,577][20381] Updated weights for policy 1, policy_version 374132 (0.0011) [2023-12-27 08:13:51,641][20381] Updated weights for policy 1, policy_version 374142 (0.0009) [2023-12-27 08:13:51,705][20381] Updated weights for policy 1, policy_version 374152 (0.0006) [2023-12-27 08:13:51,737][20347] Updated weights for policy 0, policy_version 374278 (0.0007) [2023-12-27 08:13:51,794][20347] Updated weights for policy 0, policy_version 374288 (0.0007) [2023-12-27 08:13:51,849][20347] Updated weights for policy 0, policy_version 374298 (0.0007) [2023-12-27 08:13:52,285][20381] Updated weights for policy 1, policy_version 374162 (0.0007) [2023-12-27 08:13:52,352][20381] Updated weights for policy 1, policy_version 374172 (0.0010) [2023-12-27 08:13:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20882.6). Total num frames: 191643648. Throughput: 0: 10317.9, 1: 10521.0. Samples: 191617036. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:52,363][19170] Avg episode reward: [(0, '768.520'), (1, '923.050')] [2023-12-27 08:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000374304_95838208.pth... [2023-12-27 08:13:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000373088_95526912.pth [2023-12-27 08:13:52,412][20381] Updated weights for policy 1, policy_version 374182 (0.0007) [2023-12-27 08:13:52,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000374192_95813632.pth... [2023-12-27 08:13:52,468][20381] Updated weights for policy 1, policy_version 374192 (0.0007) [2023-12-27 08:13:52,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000372944_95494144.pth [2023-12-27 08:13:52,553][20347] Updated weights for policy 0, policy_version 374308 (0.0007) [2023-12-27 08:13:52,608][20347] Updated weights for policy 0, policy_version 374318 (0.0005) [2023-12-27 08:13:52,659][20347] Updated weights for policy 0, policy_version 374328 (0.0005) [2023-12-27 08:13:53,134][20381] Updated weights for policy 1, policy_version 374202 (0.0008) [2023-12-27 08:13:53,190][20381] Updated weights for policy 1, policy_version 374212 (0.0007) [2023-12-27 08:13:53,243][20381] Updated weights for policy 1, policy_version 374222 (0.0008) [2023-12-27 08:13:53,277][20347] Updated weights for policy 0, policy_version 374338 (0.0007) [2023-12-27 08:13:53,329][20347] Updated weights for policy 0, policy_version 374348 (0.0006) [2023-12-27 08:13:53,377][20347] Updated weights for policy 0, policy_version 374358 (0.0005) [2023-12-27 08:13:53,433][20347] Updated weights for policy 0, policy_version 374368 (0.0006) [2023-12-27 08:13:53,909][20381] Updated weights for policy 1, policy_version 374232 (0.0006) [2023-12-27 08:13:53,968][20381] Updated weights for policy 1, policy_version 374242 (0.0005) [2023-12-27 08:13:54,021][20381] Updated weights for policy 1, policy_version 374252 (0.0005) [2023-12-27 08:13:54,157][20347] Updated weights for policy 0, policy_version 374378 (0.0008) [2023-12-27 08:13:54,216][20347] Updated weights for policy 0, policy_version 374388 (0.0007) [2023-12-27 08:13:54,269][20347] Updated weights for policy 0, policy_version 374398 (0.0008) [2023-12-27 08:13:54,571][20381] Updated weights for policy 1, policy_version 374262 (0.0007) [2023-12-27 08:13:54,627][20381] Updated weights for policy 1, policy_version 374272 (0.0008) [2023-12-27 08:13:54,676][20381] Updated weights for policy 1, policy_version 374282 (0.0007) [2023-12-27 08:13:54,935][20347] Updated weights for policy 0, policy_version 374408 (0.0008) [2023-12-27 08:13:54,991][20347] Updated weights for policy 0, policy_version 374418 (0.0005) [2023-12-27 08:13:55,045][20347] Updated weights for policy 0, policy_version 374428 (0.0005) [2023-12-27 08:13:55,340][20381] Updated weights for policy 1, policy_version 374292 (0.0007) [2023-12-27 08:13:55,388][20381] Updated weights for policy 1, policy_version 374302 (0.0007) [2023-12-27 08:13:55,435][20381] Updated weights for policy 1, policy_version 374312 (0.0007) [2023-12-27 08:13:55,656][20347] Updated weights for policy 0, policy_version 374438 (0.0005) [2023-12-27 08:13:55,722][20347] Updated weights for policy 0, policy_version 374448 (0.0005) [2023-12-27 08:13:55,783][20347] Updated weights for policy 0, policy_version 374458 (0.0010) [2023-12-27 08:13:56,016][20381] Updated weights for policy 1, policy_version 374322 (0.0008) [2023-12-27 08:13:56,066][20381] Updated weights for policy 1, policy_version 374332 (0.0008) [2023-12-27 08:13:56,114][20381] Updated weights for policy 1, policy_version 374342 (0.0007) [2023-12-27 08:13:56,168][20381] Updated weights for policy 1, policy_version 374352 (0.0005) [2023-12-27 08:13:56,412][20347] Updated weights for policy 0, policy_version 374468 (0.0010) [2023-12-27 08:13:56,464][20347] Updated weights for policy 0, policy_version 374478 (0.0010) [2023-12-27 08:13:56,511][20347] Updated weights for policy 0, policy_version 374488 (0.0010) [2023-12-27 08:13:56,715][20381] Updated weights for policy 1, policy_version 374362 (0.0005) [2023-12-27 08:13:56,770][20381] Updated weights for policy 1, policy_version 374372 (0.0010) [2023-12-27 08:13:56,827][20381] Updated weights for policy 1, policy_version 374382 (0.0010) [2023-12-27 08:13:57,150][20347] Updated weights for policy 0, policy_version 374498 (0.0010) [2023-12-27 08:13:57,200][20347] Updated weights for policy 0, policy_version 374508 (0.0010) [2023-12-27 08:13:57,257][20347] Updated weights for policy 0, policy_version 374518 (0.0010) [2023-12-27 08:13:57,315][20347] Updated weights for policy 0, policy_version 374528 (0.0010) [2023-12-27 08:13:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 191758336. Throughput: 0: 10336.9, 1: 10526.1. Samples: 191744052. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:13:57,363][19170] Avg episode reward: [(0, '777.890'), (1, '903.440')] [2023-12-27 08:13:57,470][20381] Updated weights for policy 1, policy_version 374392 (0.0006) [2023-12-27 08:13:57,521][20381] Updated weights for policy 1, policy_version 374402 (0.0005) [2023-12-27 08:13:57,563][20381] Updated weights for policy 1, policy_version 374412 (0.0005) [2023-12-27 08:13:57,953][20347] Updated weights for policy 0, policy_version 374538 (0.0007) [2023-12-27 08:13:57,998][20347] Updated weights for policy 0, policy_version 374548 (0.0009) [2023-12-27 08:13:58,043][20347] Updated weights for policy 0, policy_version 374558 (0.0007) [2023-12-27 08:13:58,190][20381] Updated weights for policy 1, policy_version 374422 (0.0009) [2023-12-27 08:13:58,240][20381] Updated weights for policy 1, policy_version 374432 (0.0010) [2023-12-27 08:13:58,302][20381] Updated weights for policy 1, policy_version 374442 (0.0011) [2023-12-27 08:13:58,753][20347] Updated weights for policy 0, policy_version 374568 (0.0009) [2023-12-27 08:13:58,820][20347] Updated weights for policy 0, policy_version 374578 (0.0010) [2023-12-27 08:13:58,883][20347] Updated weights for policy 0, policy_version 374588 (0.0010) [2023-12-27 08:13:59,035][20381] Updated weights for policy 1, policy_version 374452 (0.0008) [2023-12-27 08:13:59,095][20381] Updated weights for policy 1, policy_version 374462 (0.0007) [2023-12-27 08:13:59,150][20381] Updated weights for policy 1, policy_version 374472 (0.0006) [2023-12-27 08:13:59,538][20347] Updated weights for policy 0, policy_version 374598 (0.0011) [2023-12-27 08:13:59,599][20347] Updated weights for policy 0, policy_version 374608 (0.0007) [2023-12-27 08:13:59,658][20347] Updated weights for policy 0, policy_version 374618 (0.0007) [2023-12-27 08:13:59,806][20381] Updated weights for policy 1, policy_version 374482 (0.0006) [2023-12-27 08:13:59,872][20381] Updated weights for policy 1, policy_version 374492 (0.0010) [2023-12-27 08:13:59,934][20381] Updated weights for policy 1, policy_version 374502 (0.0011) [2023-12-27 08:13:59,989][20381] Updated weights for policy 1, policy_version 374512 (0.0008) [2023-12-27 08:14:00,350][20347] Updated weights for policy 0, policy_version 374628 (0.0008) [2023-12-27 08:14:00,407][20347] Updated weights for policy 0, policy_version 374638 (0.0010) [2023-12-27 08:14:00,459][20347] Updated weights for policy 0, policy_version 374648 (0.0008) [2023-12-27 08:14:00,577][20292] KL-divergence is very high: 136.1551 [2023-12-27 08:14:00,626][20381] Updated weights for policy 1, policy_version 374522 (0.0005) [2023-12-27 08:14:00,675][20381] Updated weights for policy 1, policy_version 374532 (0.0005) [2023-12-27 08:14:00,724][20381] Updated weights for policy 1, policy_version 374542 (0.0005) [2023-12-27 08:14:01,086][20347] Updated weights for policy 0, policy_version 374658 (0.0007) [2023-12-27 08:14:01,139][20347] Updated weights for policy 0, policy_version 374668 (0.0007) [2023-12-27 08:14:01,206][20347] Updated weights for policy 0, policy_version 374678 (0.0008) [2023-12-27 08:14:01,262][20347] Updated weights for policy 0, policy_version 374688 (0.0007) [2023-12-27 08:14:01,377][20381] Updated weights for policy 1, policy_version 374552 (0.0009) [2023-12-27 08:14:01,435][20381] Updated weights for policy 1, policy_version 374562 (0.0008) [2023-12-27 08:14:01,494][20381] Updated weights for policy 1, policy_version 374572 (0.0008) [2023-12-27 08:14:01,948][20347] Updated weights for policy 0, policy_version 374698 (0.0007) [2023-12-27 08:14:01,996][20347] Updated weights for policy 0, policy_version 374708 (0.0005) [2023-12-27 08:14:02,049][20347] Updated weights for policy 0, policy_version 374718 (0.0008) [2023-12-27 08:14:02,136][20381] Updated weights for policy 1, policy_version 374582 (0.0007) [2023-12-27 08:14:02,191][20381] Updated weights for policy 1, policy_version 374592 (0.0008) [2023-12-27 08:14:02,248][20381] Updated weights for policy 1, policy_version 374602 (0.0008) [2023-12-27 08:14:02,362][19170] Fps is (10 sec: 22118.7, 60 sec: 21026.2, 300 sec: 20910.4). Total num frames: 191864832. Throughput: 0: 10394.1, 1: 10554.8. Samples: 191869988. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:14:02,363][19170] Avg episode reward: [(0, '768.360'), (1, '909.010')] [2023-12-27 08:14:02,674][20347] Updated weights for policy 0, policy_version 374728 (0.0006) [2023-12-27 08:14:02,733][20347] Updated weights for policy 0, policy_version 374738 (0.0005) [2023-12-27 08:14:02,793][20347] Updated weights for policy 0, policy_version 374748 (0.0005) [2023-12-27 08:14:02,968][20381] Updated weights for policy 1, policy_version 374612 (0.0006) [2023-12-27 08:14:03,021][20381] Updated weights for policy 1, policy_version 374622 (0.0009) [2023-12-27 08:14:03,069][20381] Updated weights for policy 1, policy_version 374632 (0.0010) [2023-12-27 08:14:03,301][20347] Updated weights for policy 0, policy_version 374758 (0.0005) [2023-12-27 08:14:03,352][20347] Updated weights for policy 0, policy_version 374768 (0.0005) [2023-12-27 08:14:03,405][20347] Updated weights for policy 0, policy_version 374778 (0.0005) [2023-12-27 08:14:03,737][20381] Updated weights for policy 1, policy_version 374642 (0.0011) [2023-12-27 08:14:03,791][20381] Updated weights for policy 1, policy_version 374652 (0.0010) [2023-12-27 08:14:03,844][20381] Updated weights for policy 1, policy_version 374662 (0.0009) [2023-12-27 08:14:03,903][20381] Updated weights for policy 1, policy_version 374672 (0.0008) [2023-12-27 08:14:04,000][20347] Updated weights for policy 0, policy_version 374788 (0.0005) [2023-12-27 08:14:04,065][20347] Updated weights for policy 0, policy_version 374798 (0.0005) [2023-12-27 08:14:04,122][20347] Updated weights for policy 0, policy_version 374808 (0.0005) [2023-12-27 08:14:04,643][20381] Updated weights for policy 1, policy_version 374682 (0.0008) [2023-12-27 08:14:04,692][20381] Updated weights for policy 1, policy_version 374692 (0.0007) [2023-12-27 08:14:04,736][20347] Updated weights for policy 0, policy_version 374818 (0.0006) [2023-12-27 08:14:04,750][20381] Updated weights for policy 1, policy_version 374702 (0.0008) [2023-12-27 08:14:04,791][20347] Updated weights for policy 0, policy_version 374828 (0.0008) [2023-12-27 08:14:04,849][20347] Updated weights for policy 0, policy_version 374838 (0.0007) [2023-12-27 08:14:04,901][20347] Updated weights for policy 0, policy_version 374848 (0.0008) [2023-12-27 08:14:05,396][20381] Updated weights for policy 1, policy_version 374712 (0.0009) [2023-12-27 08:14:05,460][20381] Updated weights for policy 1, policy_version 374722 (0.0008) [2023-12-27 08:14:05,517][20381] Updated weights for policy 1, policy_version 374732 (0.0009) [2023-12-27 08:14:05,595][20347] Updated weights for policy 0, policy_version 374858 (0.0006) [2023-12-27 08:14:05,652][20347] Updated weights for policy 0, policy_version 374868 (0.0006) [2023-12-27 08:14:05,708][20347] Updated weights for policy 0, policy_version 374878 (0.0007) [2023-12-27 08:14:06,293][20381] Updated weights for policy 1, policy_version 374742 (0.0008) [2023-12-27 08:14:06,308][20347] Updated weights for policy 0, policy_version 374888 (0.0008) [2023-12-27 08:14:06,360][20381] Updated weights for policy 1, policy_version 374752 (0.0008) [2023-12-27 08:14:06,365][20347] Updated weights for policy 0, policy_version 374898 (0.0006) [2023-12-27 08:14:06,418][20381] Updated weights for policy 1, policy_version 374762 (0.0008) [2023-12-27 08:14:06,425][20347] Updated weights for policy 0, policy_version 374908 (0.0011) [2023-12-27 08:14:07,056][20347] Updated weights for policy 0, policy_version 374918 (0.0009) [2023-12-27 08:14:07,110][20381] Updated weights for policy 1, policy_version 374772 (0.0009) [2023-12-27 08:14:07,120][20347] Updated weights for policy 0, policy_version 374928 (0.0008) [2023-12-27 08:14:07,170][20381] Updated weights for policy 1, policy_version 374782 (0.0008) [2023-12-27 08:14:07,181][20347] Updated weights for policy 0, policy_version 374938 (0.0006) [2023-12-27 08:14:07,227][20381] Updated weights for policy 1, policy_version 374792 (0.0007) [2023-12-27 08:14:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20938.2). Total num frames: 191971328. Throughput: 0: 10449.9, 1: 10552.3. Samples: 191935236. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:14:07,363][19170] Avg episode reward: [(0, '835.910'), (1, '910.690')] [2023-12-27 08:14:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000374800_95969280.pth... [2023-12-27 08:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000374944_96002048.pth... [2023-12-27 08:14:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000373584_95657984.pth [2023-12-27 08:14:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000373696_95682560.pth [2023-12-27 08:14:07,849][20347] Updated weights for policy 0, policy_version 374948 (0.0006) [2023-12-27 08:14:07,886][20381] Updated weights for policy 1, policy_version 374802 (0.0008) [2023-12-27 08:14:07,910][20347] Updated weights for policy 0, policy_version 374958 (0.0007) [2023-12-27 08:14:07,946][20381] Updated weights for policy 1, policy_version 374812 (0.0006) [2023-12-27 08:14:07,953][20292] KL-divergence is very high: 144.1446 [2023-12-27 08:14:07,962][20347] Updated weights for policy 0, policy_version 374968 (0.0005) [2023-12-27 08:14:08,001][20381] Updated weights for policy 1, policy_version 374822 (0.0005) [2023-12-27 08:14:08,053][20381] Updated weights for policy 1, policy_version 374832 (0.0007) [2023-12-27 08:14:08,681][20347] Updated weights for policy 0, policy_version 374978 (0.0007) [2023-12-27 08:14:08,695][20381] Updated weights for policy 1, policy_version 374842 (0.0007) [2023-12-27 08:14:08,740][20347] Updated weights for policy 0, policy_version 374988 (0.0007) [2023-12-27 08:14:08,750][20381] Updated weights for policy 1, policy_version 374852 (0.0006) [2023-12-27 08:14:08,801][20347] Updated weights for policy 0, policy_version 374998 (0.0007) [2023-12-27 08:14:08,806][20381] Updated weights for policy 1, policy_version 374862 (0.0008) [2023-12-27 08:14:08,867][20347] Updated weights for policy 0, policy_version 375008 (0.0006) [2023-12-27 08:14:09,466][20381] Updated weights for policy 1, policy_version 374872 (0.0007) [2023-12-27 08:14:09,519][20381] Updated weights for policy 1, policy_version 374882 (0.0006) [2023-12-27 08:14:09,539][20291] KL-divergence is very high: 126.2545 [2023-12-27 08:14:09,544][20347] Updated weights for policy 0, policy_version 375018 (0.0005) [2023-12-27 08:14:09,572][20381] Updated weights for policy 1, policy_version 374892 (0.0005) [2023-12-27 08:14:09,606][20347] Updated weights for policy 0, policy_version 375028 (0.0007) [2023-12-27 08:14:09,668][20347] Updated weights for policy 0, policy_version 375038 (0.0008) [2023-12-27 08:14:09,674][20291] KL-divergence is very high: 126.8639 [2023-12-27 08:14:10,243][20381] Updated weights for policy 1, policy_version 374902 (0.0007) [2023-12-27 08:14:10,301][20381] Updated weights for policy 1, policy_version 374912 (0.0008) [2023-12-27 08:14:10,359][20381] Updated weights for policy 1, policy_version 374922 (0.0007) [2023-12-27 08:14:10,368][20347] Updated weights for policy 0, policy_version 375048 (0.0006) [2023-12-27 08:14:10,423][20347] Updated weights for policy 0, policy_version 375058 (0.0006) [2023-12-27 08:14:10,484][20347] Updated weights for policy 0, policy_version 375068 (0.0007) [2023-12-27 08:14:11,057][20381] Updated weights for policy 1, policy_version 374932 (0.0008) [2023-12-27 08:14:11,120][20381] Updated weights for policy 1, policy_version 374942 (0.0010) [2023-12-27 08:14:11,181][20381] Updated weights for policy 1, policy_version 374952 (0.0008) [2023-12-27 08:14:11,206][20347] Updated weights for policy 0, policy_version 375078 (0.0007) [2023-12-27 08:14:11,266][20347] Updated weights for policy 0, policy_version 375088 (0.0007) [2023-12-27 08:14:11,321][20347] Updated weights for policy 0, policy_version 375098 (0.0006) [2023-12-27 08:14:11,958][20381] Updated weights for policy 1, policy_version 374962 (0.0008) [2023-12-27 08:14:12,004][20381] Updated weights for policy 1, policy_version 374972 (0.0008) [2023-12-27 08:14:12,045][20347] Updated weights for policy 0, policy_version 375108 (0.0009) [2023-12-27 08:14:12,059][20381] Updated weights for policy 1, policy_version 374982 (0.0007) [2023-12-27 08:14:12,103][20347] Updated weights for policy 0, policy_version 375118 (0.0006) [2023-12-27 08:14:12,108][20381] Updated weights for policy 1, policy_version 374992 (0.0005) [2023-12-27 08:14:12,158][20347] Updated weights for policy 0, policy_version 375128 (0.0007) [2023-12-27 08:14:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 192069632. Throughput: 0: 10458.7, 1: 10420.2. Samples: 192057232. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:14:12,363][19170] Avg episode reward: [(0, '748.850'), (1, '930.100')] [2023-12-27 08:14:12,810][20381] Updated weights for policy 1, policy_version 375002 (0.0007) [2023-12-27 08:14:12,868][20381] Updated weights for policy 1, policy_version 375012 (0.0008) [2023-12-27 08:14:12,892][20347] Updated weights for policy 0, policy_version 375138 (0.0006) [2023-12-27 08:14:12,923][20381] Updated weights for policy 1, policy_version 375022 (0.0006) [2023-12-27 08:14:12,949][20347] Updated weights for policy 0, policy_version 375148 (0.0007) [2023-12-27 08:14:13,009][20347] Updated weights for policy 0, policy_version 375158 (0.0008) [2023-12-27 08:14:13,071][20347] Updated weights for policy 0, policy_version 375168 (0.0008) [2023-12-27 08:14:13,504][20381] Updated weights for policy 1, policy_version 375032 (0.0006) [2023-12-27 08:14:13,567][20381] Updated weights for policy 1, policy_version 375042 (0.0007) [2023-12-27 08:14:13,632][20381] Updated weights for policy 1, policy_version 375052 (0.0006) [2023-12-27 08:14:13,733][20347] Updated weights for policy 0, policy_version 375178 (0.0007) [2023-12-27 08:14:13,787][20347] Updated weights for policy 0, policy_version 375188 (0.0007) [2023-12-27 08:14:13,839][20347] Updated weights for policy 0, policy_version 375198 (0.0007) [2023-12-27 08:14:14,318][20381] Updated weights for policy 1, policy_version 375062 (0.0007) [2023-12-27 08:14:14,343][20292] KL-divergence is very high: 149.1383 [2023-12-27 08:14:14,379][20381] Updated weights for policy 1, policy_version 375072 (0.0007) [2023-12-27 08:14:14,409][20347] Updated weights for policy 0, policy_version 375208 (0.0006) [2023-12-27 08:14:14,439][20381] Updated weights for policy 1, policy_version 375082 (0.0007) [2023-12-27 08:14:14,458][20347] Updated weights for policy 0, policy_version 375218 (0.0005) [2023-12-27 08:14:14,511][20347] Updated weights for policy 0, policy_version 375228 (0.0006) [2023-12-27 08:14:15,020][20381] Updated weights for policy 1, policy_version 375092 (0.0008) [2023-12-27 08:14:15,071][20381] Updated weights for policy 1, policy_version 375102 (0.0010) [2023-12-27 08:14:15,134][20381] Updated weights for policy 1, policy_version 375112 (0.0010) [2023-12-27 08:14:15,216][20347] Updated weights for policy 0, policy_version 375238 (0.0008) [2023-12-27 08:14:15,276][20347] Updated weights for policy 0, policy_version 375248 (0.0007) [2023-12-27 08:14:15,337][20347] Updated weights for policy 0, policy_version 375258 (0.0007) [2023-12-27 08:14:15,812][20381] Updated weights for policy 1, policy_version 375122 (0.0010) [2023-12-27 08:14:15,867][20381] Updated weights for policy 1, policy_version 375132 (0.0010) [2023-12-27 08:14:15,923][20381] Updated weights for policy 1, policy_version 375142 (0.0010) [2023-12-27 08:14:15,969][20381] Updated weights for policy 1, policy_version 375152 (0.0010) [2023-12-27 08:14:16,008][20347] Updated weights for policy 0, policy_version 375268 (0.0007) [2023-12-27 08:14:16,061][20347] Updated weights for policy 0, policy_version 375278 (0.0007) [2023-12-27 08:14:16,108][20347] Updated weights for policy 0, policy_version 375288 (0.0007) [2023-12-27 08:14:16,644][20381] Updated weights for policy 1, policy_version 375162 (0.0005) [2023-12-27 08:14:16,691][20381] Updated weights for policy 1, policy_version 375172 (0.0010) [2023-12-27 08:14:16,746][20381] Updated weights for policy 1, policy_version 375182 (0.0010) [2023-12-27 08:14:16,816][20347] Updated weights for policy 0, policy_version 375298 (0.0007) [2023-12-27 08:14:16,863][20347] Updated weights for policy 0, policy_version 375308 (0.0007) [2023-12-27 08:14:16,910][20347] Updated weights for policy 0, policy_version 375318 (0.0006) [2023-12-27 08:14:16,961][20347] Updated weights for policy 0, policy_version 375328 (0.0006) [2023-12-27 08:14:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 192167936. Throughput: 0: 10470.7, 1: 10339.7. Samples: 192181400. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-12-27 08:14:17,363][19170] Avg episode reward: [(0, '717.970'), (1, '915.560')] [2023-12-27 08:14:17,424][20381] Updated weights for policy 1, policy_version 375192 (0.0010) [2023-12-27 08:14:17,479][20381] Updated weights for policy 1, policy_version 375202 (0.0010) [2023-12-27 08:14:17,533][20381] Updated weights for policy 1, policy_version 375212 (0.0008) [2023-12-27 08:14:17,595][20347] Updated weights for policy 0, policy_version 375338 (0.0011) [2023-12-27 08:14:17,604][20291] KL-divergence is very high: 166.6832 [2023-12-27 08:14:17,626][20291] KL-divergence is very high: 137.1143 [2023-12-27 08:14:17,648][20291] KL-divergence is very high: 102.0887 [2023-12-27 08:14:17,650][20347] Updated weights for policy 0, policy_version 375348 (0.0010) [2023-12-27 08:14:17,699][20347] Updated weights for policy 0, policy_version 375358 (0.0010) [2023-12-27 08:14:18,230][20381] Updated weights for policy 1, policy_version 375222 (0.0010) [2023-12-27 08:14:18,291][20381] Updated weights for policy 1, policy_version 375232 (0.0011) [2023-12-27 08:14:18,305][20292] KL-divergence is very high: 187.2417 [2023-12-27 08:14:18,357][20381] Updated weights for policy 1, policy_version 375242 (0.0010) [2023-12-27 08:14:18,358][20292] KL-divergence is very high: 230.8906 [2023-12-27 08:14:18,364][20292] KL-divergence is very high: 231.9587 [2023-12-27 08:14:18,371][20292] KL-divergence is very high: 155.4671 [2023-12-27 08:14:18,385][20347] Updated weights for policy 0, policy_version 375368 (0.0010) [2023-12-27 08:14:18,435][20347] Updated weights for policy 0, policy_version 375378 (0.0009) [2023-12-27 08:14:18,488][20347] Updated weights for policy 0, policy_version 375388 (0.0009) [2023-12-27 08:14:19,135][20381] Updated weights for policy 1, policy_version 375252 (0.0010) [2023-12-27 08:14:19,190][20381] Updated weights for policy 1, policy_version 375262 (0.0011) [2023-12-27 08:14:19,249][20381] Updated weights for policy 1, policy_version 375272 (0.0009) [2023-12-27 08:14:19,280][20347] Updated weights for policy 0, policy_version 375398 (0.0010) [2023-12-27 08:14:19,352][20347] Updated weights for policy 0, policy_version 375408 (0.0010) [2023-12-27 08:14:19,411][20347] Updated weights for policy 0, policy_version 375418 (0.0006) [2023-12-27 08:14:20,001][20347] Updated weights for policy 0, policy_version 375428 (0.0007) [2023-12-27 08:14:20,006][20381] Updated weights for policy 1, policy_version 375282 (0.0007) [2023-12-27 08:14:20,061][20347] Updated weights for policy 0, policy_version 375438 (0.0010) [2023-12-27 08:14:20,072][20381] Updated weights for policy 1, policy_version 375292 (0.0006) [2023-12-27 08:14:20,123][20347] Updated weights for policy 0, policy_version 375448 (0.0010) [2023-12-27 08:14:20,128][20381] Updated weights for policy 1, policy_version 375302 (0.0008) [2023-12-27 08:14:20,186][20381] Updated weights for policy 1, policy_version 375312 (0.0007) [2023-12-27 08:14:20,808][20347] Updated weights for policy 0, policy_version 375458 (0.0007) [2023-12-27 08:14:20,859][20381] Updated weights for policy 1, policy_version 375322 (0.0008) [2023-12-27 08:14:20,874][20347] Updated weights for policy 0, policy_version 375468 (0.0006) [2023-12-27 08:14:20,916][20381] Updated weights for policy 1, policy_version 375332 (0.0008) [2023-12-27 08:14:20,940][20347] Updated weights for policy 0, policy_version 375478 (0.0006) [2023-12-27 08:14:20,976][20381] Updated weights for policy 1, policy_version 375342 (0.0008) [2023-12-27 08:14:21,004][20347] Updated weights for policy 0, policy_version 375488 (0.0007) [2023-12-27 08:14:21,671][20347] Updated weights for policy 0, policy_version 375498 (0.0011) [2023-12-27 08:14:21,733][20347] Updated weights for policy 0, policy_version 375508 (0.0008) [2023-12-27 08:14:21,791][20381] Updated weights for policy 1, policy_version 375352 (0.0007) [2023-12-27 08:14:21,795][20347] Updated weights for policy 0, policy_version 375518 (0.0006) [2023-12-27 08:14:21,853][20381] Updated weights for policy 1, policy_version 375362 (0.0007) [2023-12-27 08:14:21,918][20381] Updated weights for policy 1, policy_version 375372 (0.0008) [2023-12-27 08:14:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 20882.6). Total num frames: 192266240. Throughput: 0: 10436.5, 1: 10258.9. Samples: 192240420. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:22,363][19170] Avg episode reward: [(0, '883.490'), (1, '734.390')] [2023-12-27 08:14:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000375376_96116736.pth... [2023-12-27 08:14:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000375520_96149504.pth... [2023-12-27 08:14:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000374192_95813632.pth [2023-12-27 08:14:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000374304_95838208.pth [2023-12-27 08:14:22,444][20347] Updated weights for policy 0, policy_version 375528 (0.0006) [2023-12-27 08:14:22,511][20347] Updated weights for policy 0, policy_version 375538 (0.0005) [2023-12-27 08:14:22,567][20347] Updated weights for policy 0, policy_version 375548 (0.0007) [2023-12-27 08:14:22,628][20381] Updated weights for policy 1, policy_version 375382 (0.0008) [2023-12-27 08:14:22,682][20381] Updated weights for policy 1, policy_version 375392 (0.0007) [2023-12-27 08:14:22,737][20381] Updated weights for policy 1, policy_version 375402 (0.0008) [2023-12-27 08:14:23,155][20347] Updated weights for policy 0, policy_version 375558 (0.0005) [2023-12-27 08:14:23,200][20347] Updated weights for policy 0, policy_version 375568 (0.0005) [2023-12-27 08:14:23,252][20347] Updated weights for policy 0, policy_version 375578 (0.0005) [2023-12-27 08:14:23,485][20381] Updated weights for policy 1, policy_version 375412 (0.0008) [2023-12-27 08:14:23,546][20381] Updated weights for policy 1, policy_version 375422 (0.0008) [2023-12-27 08:14:23,599][20381] Updated weights for policy 1, policy_version 375432 (0.0008) [2023-12-27 08:14:23,869][20347] Updated weights for policy 0, policy_version 375588 (0.0006) [2023-12-27 08:14:23,918][20347] Updated weights for policy 0, policy_version 375598 (0.0007) [2023-12-27 08:14:23,969][20347] Updated weights for policy 0, policy_version 375608 (0.0008) [2023-12-27 08:14:24,228][20381] Updated weights for policy 1, policy_version 375442 (0.0007) [2023-12-27 08:14:24,290][20381] Updated weights for policy 1, policy_version 375452 (0.0007) [2023-12-27 08:14:24,349][20381] Updated weights for policy 1, policy_version 375462 (0.0008) [2023-12-27 08:14:24,397][20381] Updated weights for policy 1, policy_version 375472 (0.0006) [2023-12-27 08:14:24,698][20347] Updated weights for policy 0, policy_version 375618 (0.0007) [2023-12-27 08:14:24,745][20347] Updated weights for policy 0, policy_version 375628 (0.0007) [2023-12-27 08:14:24,799][20347] Updated weights for policy 0, policy_version 375638 (0.0007) [2023-12-27 08:14:24,861][20347] Updated weights for policy 0, policy_version 375648 (0.0007) [2023-12-27 08:14:25,017][20381] Updated weights for policy 1, policy_version 375483 (0.0007) [2023-12-27 08:14:25,070][20381] Updated weights for policy 1, policy_version 375493 (0.0005) [2023-12-27 08:14:25,116][20381] Updated weights for policy 1, policy_version 375503 (0.0006) [2023-12-27 08:14:25,558][20347] Updated weights for policy 0, policy_version 375658 (0.0008) [2023-12-27 08:14:25,627][20347] Updated weights for policy 0, policy_version 375668 (0.0008) [2023-12-27 08:14:25,671][20347] Updated weights for policy 0, policy_version 375678 (0.0007) [2023-12-27 08:14:25,720][20381] Updated weights for policy 1, policy_version 375513 (0.0008) [2023-12-27 08:14:25,777][20381] Updated weights for policy 1, policy_version 375523 (0.0008) [2023-12-27 08:14:25,834][20381] Updated weights for policy 1, policy_version 375533 (0.0006) [2023-12-27 08:14:26,350][20347] Updated weights for policy 0, policy_version 375688 (0.0008) [2023-12-27 08:14:26,399][20347] Updated weights for policy 0, policy_version 375698 (0.0007) [2023-12-27 08:14:26,441][20347] Updated weights for policy 0, policy_version 375708 (0.0007) [2023-12-27 08:14:26,469][20381] Updated weights for policy 1, policy_version 375543 (0.0006) [2023-12-27 08:14:26,523][20381] Updated weights for policy 1, policy_version 375553 (0.0007) [2023-12-27 08:14:26,573][20381] Updated weights for policy 1, policy_version 375563 (0.0005) [2023-12-27 08:14:27,123][20381] Updated weights for policy 1, policy_version 375573 (0.0005) [2023-12-27 08:14:27,152][20347] Updated weights for policy 0, policy_version 375718 (0.0006) [2023-12-27 08:14:27,175][20381] Updated weights for policy 1, policy_version 375583 (0.0005) [2023-12-27 08:14:27,199][20347] Updated weights for policy 0, policy_version 375728 (0.0007) [2023-12-27 08:14:27,220][20381] Updated weights for policy 1, policy_version 375593 (0.0005) [2023-12-27 08:14:27,244][20347] Updated weights for policy 0, policy_version 375738 (0.0006) [2023-12-27 08:14:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 192380928. Throughput: 0: 10421.7, 1: 10239.9. Samples: 192363792. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:27,363][19170] Avg episode reward: [(0, '951.860'), (1, '520.910')] [2023-12-27 08:14:27,872][20381] Updated weights for policy 1, policy_version 375603 (0.0005) [2023-12-27 08:14:27,921][20347] Updated weights for policy 0, policy_version 375748 (0.0007) [2023-12-27 08:14:27,926][20381] Updated weights for policy 1, policy_version 375613 (0.0006) [2023-12-27 08:14:27,977][20381] Updated weights for policy 1, policy_version 375623 (0.0005) [2023-12-27 08:14:27,979][20347] Updated weights for policy 0, policy_version 375758 (0.0008) [2023-12-27 08:14:28,036][20347] Updated weights for policy 0, policy_version 375768 (0.0007) [2023-12-27 08:14:28,550][20381] Updated weights for policy 1, policy_version 375633 (0.0006) [2023-12-27 08:14:28,609][20381] Updated weights for policy 1, policy_version 375643 (0.0006) [2023-12-27 08:14:28,660][20381] Updated weights for policy 1, policy_version 375653 (0.0005) [2023-12-27 08:14:28,706][20381] Updated weights for policy 1, policy_version 375663 (0.0005) [2023-12-27 08:14:28,756][20347] Updated weights for policy 0, policy_version 375778 (0.0008) [2023-12-27 08:14:28,812][20347] Updated weights for policy 0, policy_version 375788 (0.0006) [2023-12-27 08:14:28,866][20347] Updated weights for policy 0, policy_version 375798 (0.0007) [2023-12-27 08:14:28,913][20347] Updated weights for policy 0, policy_version 375808 (0.0007) [2023-12-27 08:14:29,274][20381] Updated weights for policy 1, policy_version 375673 (0.0008) [2023-12-27 08:14:29,331][20381] Updated weights for policy 1, policy_version 375683 (0.0008) [2023-12-27 08:14:29,386][20381] Updated weights for policy 1, policy_version 375694 (0.0010) [2023-12-27 08:14:29,632][20347] Updated weights for policy 0, policy_version 375818 (0.0011) [2023-12-27 08:14:29,687][20347] Updated weights for policy 0, policy_version 375828 (0.0010) [2023-12-27 08:14:29,742][20347] Updated weights for policy 0, policy_version 375838 (0.0010) [2023-12-27 08:14:30,123][20381] Updated weights for policy 1, policy_version 375704 (0.0011) [2023-12-27 08:14:30,186][20381] Updated weights for policy 1, policy_version 375714 (0.0011) [2023-12-27 08:14:30,253][20381] Updated weights for policy 1, policy_version 375724 (0.0010) [2023-12-27 08:14:30,422][20347] Updated weights for policy 0, policy_version 375848 (0.0006) [2023-12-27 08:14:30,462][20291] KL-divergence is very high: 106.9960 [2023-12-27 08:14:30,476][20347] Updated weights for policy 0, policy_version 375858 (0.0006) [2023-12-27 08:14:30,532][20347] Updated weights for policy 0, policy_version 375868 (0.0007) [2023-12-27 08:14:30,877][20381] Updated weights for policy 1, policy_version 375734 (0.0010) [2023-12-27 08:14:30,926][20381] Updated weights for policy 1, policy_version 375744 (0.0008) [2023-12-27 08:14:30,979][20381] Updated weights for policy 1, policy_version 375754 (0.0005) [2023-12-27 08:14:31,148][20347] Updated weights for policy 0, policy_version 375878 (0.0008) [2023-12-27 08:14:31,211][20347] Updated weights for policy 0, policy_version 375888 (0.0007) [2023-12-27 08:14:31,268][20347] Updated weights for policy 0, policy_version 375898 (0.0007) [2023-12-27 08:14:31,626][20381] Updated weights for policy 1, policy_version 375764 (0.0009) [2023-12-27 08:14:31,688][20381] Updated weights for policy 1, policy_version 375774 (0.0011) [2023-12-27 08:14:31,749][20381] Updated weights for policy 1, policy_version 375784 (0.0006) [2023-12-27 08:14:31,920][20347] Updated weights for policy 0, policy_version 375908 (0.0007) [2023-12-27 08:14:31,980][20347] Updated weights for policy 0, policy_version 375918 (0.0005) [2023-12-27 08:14:32,037][20347] Updated weights for policy 0, policy_version 375928 (0.0005) [2023-12-27 08:14:32,038][20291] KL-divergence is very high: 171.2978 [2023-12-27 08:14:32,043][20291] KL-divergence is very high: 233.1379 [2023-12-27 08:14:32,050][20291] KL-divergence is very high: 182.2728 [2023-12-27 08:14:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 192479232. Throughput: 0: 10411.0, 1: 10387.7. Samples: 192492504. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:32,363][19170] Avg episode reward: [(0, '903.950'), (1, '768.560')] [2023-12-27 08:14:32,381][20381] Updated weights for policy 1, policy_version 375794 (0.0006) [2023-12-27 08:14:32,445][20381] Updated weights for policy 1, policy_version 375804 (0.0007) [2023-12-27 08:14:32,498][20381] Updated weights for policy 1, policy_version 375814 (0.0007) [2023-12-27 08:14:32,558][20381] Updated weights for policy 1, policy_version 375824 (0.0006) [2023-12-27 08:14:32,676][20291] KL-divergence is very high: 171.8670 [2023-12-27 08:14:32,687][20347] Updated weights for policy 0, policy_version 375938 (0.0006) [2023-12-27 08:14:32,734][20347] Updated weights for policy 0, policy_version 375948 (0.0007) [2023-12-27 08:14:32,789][20347] Updated weights for policy 0, policy_version 375958 (0.0005) [2023-12-27 08:14:32,843][20347] Updated weights for policy 0, policy_version 375968 (0.0005) [2023-12-27 08:14:33,111][20381] Updated weights for policy 1, policy_version 375834 (0.0008) [2023-12-27 08:14:33,152][20381] Updated weights for policy 1, policy_version 375844 (0.0005) [2023-12-27 08:14:33,196][20381] Updated weights for policy 1, policy_version 375854 (0.0005) [2023-12-27 08:14:33,433][20347] Updated weights for policy 0, policy_version 375978 (0.0005) [2023-12-27 08:14:33,479][20347] Updated weights for policy 0, policy_version 375988 (0.0005) [2023-12-27 08:14:33,535][20347] Updated weights for policy 0, policy_version 375998 (0.0008) [2023-12-27 08:14:33,783][20381] Updated weights for policy 1, policy_version 375864 (0.0009) [2023-12-27 08:14:33,837][20381] Updated weights for policy 1, policy_version 375874 (0.0005) [2023-12-27 08:14:33,890][20381] Updated weights for policy 1, policy_version 375884 (0.0008) [2023-12-27 08:14:34,160][20347] Updated weights for policy 0, policy_version 376008 (0.0009) [2023-12-27 08:14:34,210][20347] Updated weights for policy 0, policy_version 376018 (0.0008) [2023-12-27 08:14:34,270][20347] Updated weights for policy 0, policy_version 376028 (0.0010) [2023-12-27 08:14:34,536][20381] Updated weights for policy 1, policy_version 375894 (0.0010) [2023-12-27 08:14:34,592][20381] Updated weights for policy 1, policy_version 375904 (0.0011) [2023-12-27 08:14:34,644][20381] Updated weights for policy 1, policy_version 375914 (0.0011) [2023-12-27 08:14:34,981][20347] Updated weights for policy 0, policy_version 376038 (0.0010) [2023-12-27 08:14:35,038][20347] Updated weights for policy 0, policy_version 376048 (0.0010) [2023-12-27 08:14:35,089][20347] Updated weights for policy 0, policy_version 376058 (0.0010) [2023-12-27 08:14:35,275][20381] Updated weights for policy 1, policy_version 375924 (0.0008) [2023-12-27 08:14:35,328][20381] Updated weights for policy 1, policy_version 375934 (0.0005) [2023-12-27 08:14:35,382][20381] Updated weights for policy 1, policy_version 375944 (0.0005) [2023-12-27 08:14:35,640][20347] Updated weights for policy 0, policy_version 376068 (0.0008) [2023-12-27 08:14:35,697][20347] Updated weights for policy 0, policy_version 376078 (0.0008) [2023-12-27 08:14:35,749][20347] Updated weights for policy 0, policy_version 376088 (0.0005) [2023-12-27 08:14:35,994][20381] Updated weights for policy 1, policy_version 375954 (0.0010) [2023-12-27 08:14:36,062][20381] Updated weights for policy 1, policy_version 375964 (0.0011) [2023-12-27 08:14:36,127][20381] Updated weights for policy 1, policy_version 375974 (0.0010) [2023-12-27 08:14:36,186][20381] Updated weights for policy 1, policy_version 375984 (0.0009) [2023-12-27 08:14:36,350][20347] Updated weights for policy 0, policy_version 376098 (0.0006) [2023-12-27 08:14:36,417][20347] Updated weights for policy 0, policy_version 376108 (0.0011) [2023-12-27 08:14:36,482][20347] Updated weights for policy 0, policy_version 376118 (0.0011) [2023-12-27 08:14:36,542][20347] Updated weights for policy 0, policy_version 376128 (0.0011) [2023-12-27 08:14:36,867][20381] Updated weights for policy 1, policy_version 375994 (0.0008) [2023-12-27 08:14:36,925][20381] Updated weights for policy 1, policy_version 376004 (0.0007) [2023-12-27 08:14:36,982][20381] Updated weights for policy 1, policy_version 376014 (0.0007) [2023-12-27 08:14:37,269][20347] Updated weights for policy 0, policy_version 376138 (0.0011) [2023-12-27 08:14:37,325][20347] Updated weights for policy 0, policy_version 376148 (0.0011) [2023-12-27 08:14:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20910.4). Total num frames: 192585728. Throughput: 0: 10428.5, 1: 10488.9. Samples: 192558320. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:37,363][19170] Avg episode reward: [(0, '932.820'), (1, '942.090')] [2023-12-27 08:14:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000376016_96280576.pth... [2023-12-27 08:14:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000374800_95969280.pth [2023-12-27 08:14:37,392][20347] Updated weights for policy 0, policy_version 376158 (0.0011) [2023-12-27 08:14:37,401][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000376160_96313344.pth... [2023-12-27 08:14:37,405][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000374944_96002048.pth [2023-12-27 08:14:37,573][20381] Updated weights for policy 1, policy_version 376024 (0.0006) [2023-12-27 08:14:37,631][20381] Updated weights for policy 1, policy_version 376034 (0.0005) [2023-12-27 08:14:37,694][20381] Updated weights for policy 1, policy_version 376044 (0.0006) [2023-12-27 08:14:38,086][20347] Updated weights for policy 0, policy_version 376168 (0.0009) [2023-12-27 08:14:38,141][20347] Updated weights for policy 0, policy_version 376178 (0.0008) [2023-12-27 08:14:38,198][20347] Updated weights for policy 0, policy_version 376188 (0.0006) [2023-12-27 08:14:38,282][20381] Updated weights for policy 1, policy_version 376054 (0.0006) [2023-12-27 08:14:38,343][20381] Updated weights for policy 1, policy_version 376064 (0.0007) [2023-12-27 08:14:38,401][20381] Updated weights for policy 1, policy_version 376074 (0.0007) [2023-12-27 08:14:38,887][20347] Updated weights for policy 0, policy_version 376198 (0.0006) [2023-12-27 08:14:38,933][20347] Updated weights for policy 0, policy_version 376208 (0.0008) [2023-12-27 08:14:38,978][20347] Updated weights for policy 0, policy_version 376218 (0.0007) [2023-12-27 08:14:39,077][20381] Updated weights for policy 1, policy_version 376084 (0.0008) [2023-12-27 08:14:39,131][20381] Updated weights for policy 1, policy_version 376094 (0.0005) [2023-12-27 08:14:39,184][20381] Updated weights for policy 1, policy_version 376104 (0.0006) [2023-12-27 08:14:39,714][20347] Updated weights for policy 0, policy_version 376228 (0.0007) [2023-12-27 08:14:39,768][20347] Updated weights for policy 0, policy_version 376238 (0.0008) [2023-12-27 08:14:39,834][20347] Updated weights for policy 0, policy_version 376248 (0.0007) [2023-12-27 08:14:39,905][20381] Updated weights for policy 1, policy_version 376114 (0.0007) [2023-12-27 08:14:39,967][20381] Updated weights for policy 1, policy_version 376124 (0.0006) [2023-12-27 08:14:40,030][20381] Updated weights for policy 1, policy_version 376134 (0.0006) [2023-12-27 08:14:40,091][20381] Updated weights for policy 1, policy_version 376144 (0.0008) [2023-12-27 08:14:40,514][20347] Updated weights for policy 0, policy_version 376258 (0.0007) [2023-12-27 08:14:40,567][20347] Updated weights for policy 0, policy_version 376268 (0.0007) [2023-12-27 08:14:40,622][20347] Updated weights for policy 0, policy_version 376278 (0.0008) [2023-12-27 08:14:40,670][20347] Updated weights for policy 0, policy_version 376288 (0.0006) [2023-12-27 08:14:40,699][20381] Updated weights for policy 1, policy_version 376154 (0.0007) [2023-12-27 08:14:40,757][20381] Updated weights for policy 1, policy_version 376164 (0.0008) [2023-12-27 08:14:40,812][20381] Updated weights for policy 1, policy_version 376174 (0.0007) [2023-12-27 08:14:41,427][20347] Updated weights for policy 0, policy_version 376298 (0.0007) [2023-12-27 08:14:41,451][20381] Updated weights for policy 1, policy_version 376184 (0.0008) [2023-12-27 08:14:41,492][20347] Updated weights for policy 0, policy_version 376308 (0.0005) [2023-12-27 08:14:41,500][20381] Updated weights for policy 1, policy_version 376194 (0.0008) [2023-12-27 08:14:41,556][20381] Updated weights for policy 1, policy_version 376204 (0.0007) [2023-12-27 08:14:41,560][20347] Updated weights for policy 0, policy_version 376318 (0.0006) [2023-12-27 08:14:42,225][20381] Updated weights for policy 1, policy_version 376214 (0.0006) [2023-12-27 08:14:42,237][20347] Updated weights for policy 0, policy_version 376328 (0.0008) [2023-12-27 08:14:42,286][20381] Updated weights for policy 1, policy_version 376224 (0.0008) [2023-12-27 08:14:42,300][20347] Updated weights for policy 0, policy_version 376338 (0.0008) [2023-12-27 08:14:42,343][20381] Updated weights for policy 1, policy_version 376234 (0.0008) [2023-12-27 08:14:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 192684032. Throughput: 0: 10418.0, 1: 10465.6. Samples: 192683816. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.090')] [2023-12-27 08:14:42,364][20347] Updated weights for policy 0, policy_version 376348 (0.0007) [2023-12-27 08:14:42,971][20347] Updated weights for policy 0, policy_version 376358 (0.0007) [2023-12-27 08:14:42,991][20381] Updated weights for policy 1, policy_version 376244 (0.0007) [2023-12-27 08:14:43,028][20347] Updated weights for policy 0, policy_version 376368 (0.0006) [2023-12-27 08:14:43,045][20381] Updated weights for policy 1, policy_version 376254 (0.0006) [2023-12-27 08:14:43,085][20347] Updated weights for policy 0, policy_version 376378 (0.0006) [2023-12-27 08:14:43,101][20381] Updated weights for policy 1, policy_version 376264 (0.0007) [2023-12-27 08:14:43,715][20347] Updated weights for policy 0, policy_version 376388 (0.0007) [2023-12-27 08:14:43,767][20347] Updated weights for policy 0, policy_version 376398 (0.0008) [2023-12-27 08:14:43,817][20381] Updated weights for policy 1, policy_version 376274 (0.0007) [2023-12-27 08:14:43,820][20347] Updated weights for policy 0, policy_version 376408 (0.0007) [2023-12-27 08:14:43,865][20381] Updated weights for policy 1, policy_version 376284 (0.0006) [2023-12-27 08:14:43,909][20381] Updated weights for policy 1, policy_version 376294 (0.0006) [2023-12-27 08:14:43,964][20381] Updated weights for policy 1, policy_version 376304 (0.0008) [2023-12-27 08:14:44,480][20347] Updated weights for policy 0, policy_version 376418 (0.0006) [2023-12-27 08:14:44,540][20347] Updated weights for policy 0, policy_version 376428 (0.0007) [2023-12-27 08:14:44,590][20347] Updated weights for policy 0, policy_version 376438 (0.0008) [2023-12-27 08:14:44,615][20381] Updated weights for policy 1, policy_version 376314 (0.0006) [2023-12-27 08:14:44,636][20347] Updated weights for policy 0, policy_version 376448 (0.0005) [2023-12-27 08:14:44,665][20381] Updated weights for policy 1, policy_version 376324 (0.0006) [2023-12-27 08:14:44,713][20381] Updated weights for policy 1, policy_version 376334 (0.0008) [2023-12-27 08:14:45,346][20347] Updated weights for policy 0, policy_version 376458 (0.0008) [2023-12-27 08:14:45,397][20347] Updated weights for policy 0, policy_version 376468 (0.0007) [2023-12-27 08:14:45,432][20381] Updated weights for policy 1, policy_version 376344 (0.0008) [2023-12-27 08:14:45,453][20347] Updated weights for policy 0, policy_version 376478 (0.0006) [2023-12-27 08:14:45,488][20381] Updated weights for policy 1, policy_version 376354 (0.0007) [2023-12-27 08:14:45,542][20381] Updated weights for policy 1, policy_version 376364 (0.0008) [2023-12-27 08:14:46,059][20347] Updated weights for policy 0, policy_version 376488 (0.0005) [2023-12-27 08:14:46,111][20347] Updated weights for policy 0, policy_version 376498 (0.0005) [2023-12-27 08:14:46,168][20347] Updated weights for policy 0, policy_version 376508 (0.0005) [2023-12-27 08:14:46,273][20381] Updated weights for policy 1, policy_version 376374 (0.0008) [2023-12-27 08:14:46,330][20381] Updated weights for policy 1, policy_version 376384 (0.0008) [2023-12-27 08:14:46,392][20381] Updated weights for policy 1, policy_version 376394 (0.0007) [2023-12-27 08:14:46,820][20347] Updated weights for policy 0, policy_version 376518 (0.0008) [2023-12-27 08:14:46,871][20347] Updated weights for policy 0, policy_version 376528 (0.0007) [2023-12-27 08:14:46,927][20347] Updated weights for policy 0, policy_version 376538 (0.0007) [2023-12-27 08:14:47,068][20381] Updated weights for policy 1, policy_version 376404 (0.0007) [2023-12-27 08:14:47,119][20381] Updated weights for policy 1, policy_version 376414 (0.0008) [2023-12-27 08:14:47,171][20381] Updated weights for policy 1, policy_version 376424 (0.0008) [2023-12-27 08:14:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.7, 300 sec: 20910.4). Total num frames: 192798720. Throughput: 0: 10432.2, 1: 10397.0. Samples: 192807300. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.090')] [2023-12-27 08:14:47,568][20347] Updated weights for policy 0, policy_version 376548 (0.0008) [2023-12-27 08:14:47,623][20347] Updated weights for policy 0, policy_version 376558 (0.0007) [2023-12-27 08:14:47,680][20347] Updated weights for policy 0, policy_version 376568 (0.0007) [2023-12-27 08:14:47,885][20381] Updated weights for policy 1, policy_version 376434 (0.0008) [2023-12-27 08:14:47,934][20381] Updated weights for policy 1, policy_version 376444 (0.0008) [2023-12-27 08:14:47,988][20381] Updated weights for policy 1, policy_version 376454 (0.0008) [2023-12-27 08:14:48,036][20381] Updated weights for policy 1, policy_version 376464 (0.0008) [2023-12-27 08:14:48,324][20347] Updated weights for policy 0, policy_version 376578 (0.0007) [2023-12-27 08:14:48,388][20347] Updated weights for policy 0, policy_version 376588 (0.0007) [2023-12-27 08:14:48,439][20347] Updated weights for policy 0, policy_version 376598 (0.0007) [2023-12-27 08:14:48,491][20347] Updated weights for policy 0, policy_version 376608 (0.0008) [2023-12-27 08:14:48,789][20381] Updated weights for policy 1, policy_version 376474 (0.0008) [2023-12-27 08:14:48,845][20381] Updated weights for policy 1, policy_version 376484 (0.0008) [2023-12-27 08:14:48,893][20381] Updated weights for policy 1, policy_version 376494 (0.0007) [2023-12-27 08:14:49,172][20347] Updated weights for policy 0, policy_version 376618 (0.0005) [2023-12-27 08:14:49,227][20347] Updated weights for policy 0, policy_version 376628 (0.0006) [2023-12-27 08:14:49,290][20347] Updated weights for policy 0, policy_version 376638 (0.0008) [2023-12-27 08:14:49,563][20381] Updated weights for policy 1, policy_version 376504 (0.0006) [2023-12-27 08:14:49,613][20381] Updated weights for policy 1, policy_version 376514 (0.0006) [2023-12-27 08:14:49,669][20381] Updated weights for policy 1, policy_version 376524 (0.0011) [2023-12-27 08:14:49,983][20347] Updated weights for policy 0, policy_version 376648 (0.0007) [2023-12-27 08:14:50,036][20347] Updated weights for policy 0, policy_version 376658 (0.0007) [2023-12-27 08:14:50,087][20347] Updated weights for policy 0, policy_version 376668 (0.0008) [2023-12-27 08:14:50,324][20381] Updated weights for policy 1, policy_version 376534 (0.0010) [2023-12-27 08:14:50,380][20381] Updated weights for policy 1, policy_version 376544 (0.0011) [2023-12-27 08:14:50,435][20381] Updated weights for policy 1, policy_version 376554 (0.0010) [2023-12-27 08:14:50,777][20347] Updated weights for policy 0, policy_version 376678 (0.0007) [2023-12-27 08:14:50,842][20347] Updated weights for policy 0, policy_version 376688 (0.0007) [2023-12-27 08:14:50,907][20347] Updated weights for policy 0, policy_version 376698 (0.0005) [2023-12-27 08:14:51,160][20381] Updated weights for policy 1, policy_version 376564 (0.0009) [2023-12-27 08:14:51,223][20381] Updated weights for policy 1, policy_version 376574 (0.0007) [2023-12-27 08:14:51,283][20381] Updated weights for policy 1, policy_version 376584 (0.0008) [2023-12-27 08:14:51,540][20347] Updated weights for policy 0, policy_version 376708 (0.0007) [2023-12-27 08:14:51,610][20347] Updated weights for policy 0, policy_version 376718 (0.0009) [2023-12-27 08:14:51,671][20347] Updated weights for policy 0, policy_version 376728 (0.0008) [2023-12-27 08:14:51,905][20381] Updated weights for policy 1, policy_version 376594 (0.0006) [2023-12-27 08:14:51,965][20381] Updated weights for policy 1, policy_version 376604 (0.0008) [2023-12-27 08:14:52,023][20381] Updated weights for policy 1, policy_version 376614 (0.0007) [2023-12-27 08:14:52,078][20381] Updated weights for policy 1, policy_version 376624 (0.0007) [2023-12-27 08:14:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 192897024. Throughput: 0: 10354.4, 1: 10392.1. Samples: 192868828. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:52,363][19170] Avg episode reward: [(0, '981.820'), (1, '961.450')] [2023-12-27 08:14:52,367][20347] Updated weights for policy 0, policy_version 376738 (0.0008) [2023-12-27 08:14:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000376624_96436224.pth... [2023-12-27 08:14:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000375376_96116736.pth [2023-12-27 08:14:52,427][20347] Updated weights for policy 0, policy_version 376748 (0.0011) [2023-12-27 08:14:52,485][20347] Updated weights for policy 0, policy_version 376758 (0.0010) [2023-12-27 08:14:52,541][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000376768_96468992.pth... [2023-12-27 08:14:52,543][20347] Updated weights for policy 0, policy_version 376768 (0.0010) [2023-12-27 08:14:52,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000375520_96149504.pth [2023-12-27 08:14:52,792][20381] Updated weights for policy 1, policy_version 376634 (0.0007) [2023-12-27 08:14:52,850][20381] Updated weights for policy 1, policy_version 376644 (0.0009) [2023-12-27 08:14:52,911][20381] Updated weights for policy 1, policy_version 376654 (0.0008) [2023-12-27 08:14:53,107][20347] Updated weights for policy 0, policy_version 376778 (0.0006) [2023-12-27 08:14:53,159][20347] Updated weights for policy 0, policy_version 376788 (0.0007) [2023-12-27 08:14:53,207][20347] Updated weights for policy 0, policy_version 376798 (0.0008) [2023-12-27 08:14:53,600][20381] Updated weights for policy 1, policy_version 376664 (0.0006) [2023-12-27 08:14:53,647][20381] Updated weights for policy 1, policy_version 376674 (0.0007) [2023-12-27 08:14:53,693][20381] Updated weights for policy 1, policy_version 376684 (0.0005) [2023-12-27 08:14:53,852][20347] Updated weights for policy 0, policy_version 376808 (0.0006) [2023-12-27 08:14:53,905][20347] Updated weights for policy 0, policy_version 376818 (0.0007) [2023-12-27 08:14:53,960][20347] Updated weights for policy 0, policy_version 376828 (0.0006) [2023-12-27 08:14:54,348][20381] Updated weights for policy 1, policy_version 376694 (0.0007) [2023-12-27 08:14:54,396][20381] Updated weights for policy 1, policy_version 376704 (0.0008) [2023-12-27 08:14:54,438][20381] Updated weights for policy 1, policy_version 376714 (0.0008) [2023-12-27 08:14:54,627][20347] Updated weights for policy 0, policy_version 376838 (0.0006) [2023-12-27 08:14:54,691][20347] Updated weights for policy 0, policy_version 376848 (0.0009) [2023-12-27 08:14:54,751][20347] Updated weights for policy 0, policy_version 376858 (0.0010) [2023-12-27 08:14:55,112][20381] Updated weights for policy 1, policy_version 376724 (0.0008) [2023-12-27 08:14:55,179][20381] Updated weights for policy 1, policy_version 376734 (0.0005) [2023-12-27 08:14:55,235][20381] Updated weights for policy 1, policy_version 376744 (0.0005) [2023-12-27 08:14:55,355][20347] Updated weights for policy 0, policy_version 376868 (0.0010) [2023-12-27 08:14:55,407][20347] Updated weights for policy 0, policy_version 376878 (0.0009) [2023-12-27 08:14:55,464][20347] Updated weights for policy 0, policy_version 376888 (0.0009) [2023-12-27 08:14:55,805][20381] Updated weights for policy 1, policy_version 376754 (0.0005) [2023-12-27 08:14:55,862][20381] Updated weights for policy 1, policy_version 376764 (0.0007) [2023-12-27 08:14:55,911][20381] Updated weights for policy 1, policy_version 376774 (0.0006) [2023-12-27 08:14:55,966][20381] Updated weights for policy 1, policy_version 376784 (0.0006) [2023-12-27 08:14:56,091][20347] Updated weights for policy 0, policy_version 376898 (0.0010) [2023-12-27 08:14:56,139][20347] Updated weights for policy 0, policy_version 376908 (0.0010) [2023-12-27 08:14:56,193][20347] Updated weights for policy 0, policy_version 376918 (0.0010) [2023-12-27 08:14:56,247][20347] Updated weights for policy 0, policy_version 376928 (0.0010) [2023-12-27 08:14:56,658][20381] Updated weights for policy 1, policy_version 376794 (0.0011) [2023-12-27 08:14:56,701][20381] Updated weights for policy 1, policy_version 376804 (0.0008) [2023-12-27 08:14:56,746][20381] Updated weights for policy 1, policy_version 376814 (0.0005) [2023-12-27 08:14:56,915][20347] Updated weights for policy 0, policy_version 376938 (0.0006) [2023-12-27 08:14:56,967][20347] Updated weights for policy 0, policy_version 376948 (0.0006) [2023-12-27 08:14:57,015][20347] Updated weights for policy 0, policy_version 376958 (0.0010) [2023-12-27 08:14:57,339][20381] Updated weights for policy 1, policy_version 376824 (0.0009) [2023-12-27 08:14:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20910.4). Total num frames: 193003520. Throughput: 0: 10403.9, 1: 10439.3. Samples: 192995176. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:14:57,363][19170] Avg episode reward: [(0, '981.820'), (1, '971.070')] [2023-12-27 08:14:57,384][20381] Updated weights for policy 1, policy_version 376834 (0.0010) [2023-12-27 08:14:57,434][20381] Updated weights for policy 1, policy_version 376844 (0.0007) [2023-12-27 08:14:57,680][20347] Updated weights for policy 0, policy_version 376968 (0.0010) [2023-12-27 08:14:57,737][20347] Updated weights for policy 0, policy_version 376978 (0.0010) [2023-12-27 08:14:57,788][20347] Updated weights for policy 0, policy_version 376988 (0.0010) [2023-12-27 08:14:58,044][20381] Updated weights for policy 1, policy_version 376854 (0.0007) [2023-12-27 08:14:58,102][20381] Updated weights for policy 1, policy_version 376864 (0.0006) [2023-12-27 08:14:58,149][20381] Updated weights for policy 1, policy_version 376874 (0.0007) [2023-12-27 08:14:58,541][20347] Updated weights for policy 0, policy_version 376998 (0.0011) [2023-12-27 08:14:58,606][20347] Updated weights for policy 0, policy_version 377008 (0.0009) [2023-12-27 08:14:58,671][20347] Updated weights for policy 0, policy_version 377018 (0.0007) [2023-12-27 08:14:58,823][20381] Updated weights for policy 1, policy_version 376884 (0.0009) [2023-12-27 08:14:58,883][20381] Updated weights for policy 1, policy_version 376894 (0.0007) [2023-12-27 08:14:58,942][20381] Updated weights for policy 1, policy_version 376904 (0.0008) [2023-12-27 08:14:59,472][20347] Updated weights for policy 0, policy_version 377028 (0.0008) [2023-12-27 08:14:59,528][20347] Updated weights for policy 0, policy_version 377038 (0.0005) [2023-12-27 08:14:59,587][20347] Updated weights for policy 0, policy_version 377048 (0.0005) [2023-12-27 08:14:59,714][20381] Updated weights for policy 1, policy_version 376914 (0.0007) [2023-12-27 08:14:59,776][20381] Updated weights for policy 1, policy_version 376924 (0.0006) [2023-12-27 08:14:59,837][20381] Updated weights for policy 1, policy_version 376934 (0.0006) [2023-12-27 08:14:59,900][20381] Updated weights for policy 1, policy_version 376944 (0.0006) [2023-12-27 08:15:00,186][20347] Updated weights for policy 0, policy_version 377058 (0.0006) [2023-12-27 08:15:00,243][20347] Updated weights for policy 0, policy_version 377068 (0.0008) [2023-12-27 08:15:00,293][20347] Updated weights for policy 0, policy_version 377078 (0.0008) [2023-12-27 08:15:00,349][20347] Updated weights for policy 0, policy_version 377088 (0.0007) [2023-12-27 08:15:00,548][20381] Updated weights for policy 1, policy_version 376954 (0.0008) [2023-12-27 08:15:00,608][20381] Updated weights for policy 1, policy_version 376964 (0.0008) [2023-12-27 08:15:00,661][20381] Updated weights for policy 1, policy_version 376974 (0.0007) [2023-12-27 08:15:00,996][20347] Updated weights for policy 0, policy_version 377098 (0.0005) [2023-12-27 08:15:01,060][20347] Updated weights for policy 0, policy_version 377108 (0.0007) [2023-12-27 08:15:01,121][20347] Updated weights for policy 0, policy_version 377118 (0.0006) [2023-12-27 08:15:01,398][20381] Updated weights for policy 1, policy_version 376984 (0.0007) [2023-12-27 08:15:01,456][20381] Updated weights for policy 1, policy_version 376994 (0.0005) [2023-12-27 08:15:01,504][20381] Updated weights for policy 1, policy_version 377004 (0.0005) [2023-12-27 08:15:01,764][20347] Updated weights for policy 0, policy_version 377128 (0.0008) [2023-12-27 08:15:01,819][20347] Updated weights for policy 0, policy_version 377138 (0.0008) [2023-12-27 08:15:01,871][20347] Updated weights for policy 0, policy_version 377148 (0.0007) [2023-12-27 08:15:02,181][20381] Updated weights for policy 1, policy_version 377014 (0.0007) [2023-12-27 08:15:02,229][20381] Updated weights for policy 1, policy_version 377024 (0.0007) [2023-12-27 08:15:02,282][20381] Updated weights for policy 1, policy_version 377034 (0.0008) [2023-12-27 08:15:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20938.2). Total num frames: 193110016. Throughput: 0: 10391.8, 1: 10430.1. Samples: 193118388. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.030')] [2023-12-27 08:15:02,554][20347] Updated weights for policy 0, policy_version 377158 (0.0007) [2023-12-27 08:15:02,601][20347] Updated weights for policy 0, policy_version 377168 (0.0007) [2023-12-27 08:15:02,649][20347] Updated weights for policy 0, policy_version 377178 (0.0007) [2023-12-27 08:15:02,985][20381] Updated weights for policy 1, policy_version 377044 (0.0007) [2023-12-27 08:15:03,040][20381] Updated weights for policy 1, policy_version 377054 (0.0007) [2023-12-27 08:15:03,101][20381] Updated weights for policy 1, policy_version 377064 (0.0007) [2023-12-27 08:15:03,263][20347] Updated weights for policy 0, policy_version 377188 (0.0007) [2023-12-27 08:15:03,316][20347] Updated weights for policy 0, policy_version 377198 (0.0005) [2023-12-27 08:15:03,363][20347] Updated weights for policy 0, policy_version 377208 (0.0005) [2023-12-27 08:15:03,852][20381] Updated weights for policy 1, policy_version 377074 (0.0008) [2023-12-27 08:15:03,903][20347] Updated weights for policy 0, policy_version 377218 (0.0006) [2023-12-27 08:15:03,915][20381] Updated weights for policy 1, policy_version 377084 (0.0007) [2023-12-27 08:15:03,963][20381] Updated weights for policy 1, policy_version 377094 (0.0007) [2023-12-27 08:15:03,965][20347] Updated weights for policy 0, policy_version 377228 (0.0007) [2023-12-27 08:15:04,016][20381] Updated weights for policy 1, policy_version 377104 (0.0005) [2023-12-27 08:15:04,017][20347] Updated weights for policy 0, policy_version 377238 (0.0006) [2023-12-27 08:15:04,067][20347] Updated weights for policy 0, policy_version 377248 (0.0005) [2023-12-27 08:15:04,647][20381] Updated weights for policy 1, policy_version 377114 (0.0007) [2023-12-27 08:15:04,701][20381] Updated weights for policy 1, policy_version 377124 (0.0009) [2023-12-27 08:15:04,719][20347] Updated weights for policy 0, policy_version 377258 (0.0005) [2023-12-27 08:15:04,747][20381] Updated weights for policy 1, policy_version 377134 (0.0010) [2023-12-27 08:15:04,777][20347] Updated weights for policy 0, policy_version 377268 (0.0005) [2023-12-27 08:15:04,831][20347] Updated weights for policy 0, policy_version 377278 (0.0005) [2023-12-27 08:15:05,364][20381] Updated weights for policy 1, policy_version 377144 (0.0006) [2023-12-27 08:15:05,415][20381] Updated weights for policy 1, policy_version 377154 (0.0007) [2023-12-27 08:15:05,420][20347] Updated weights for policy 0, policy_version 377288 (0.0007) [2023-12-27 08:15:05,468][20381] Updated weights for policy 1, policy_version 377164 (0.0006) [2023-12-27 08:15:05,473][20347] Updated weights for policy 0, policy_version 377298 (0.0007) [2023-12-27 08:15:05,518][20347] Updated weights for policy 0, policy_version 377308 (0.0008) [2023-12-27 08:15:06,060][20381] Updated weights for policy 1, policy_version 377174 (0.0007) [2023-12-27 08:15:06,126][20381] Updated weights for policy 1, policy_version 377184 (0.0006) [2023-12-27 08:15:06,175][20347] Updated weights for policy 0, policy_version 377318 (0.0009) [2023-12-27 08:15:06,187][20381] Updated weights for policy 1, policy_version 377194 (0.0011) [2023-12-27 08:15:06,229][20347] Updated weights for policy 0, policy_version 377328 (0.0011) [2023-12-27 08:15:06,282][20347] Updated weights for policy 0, policy_version 377338 (0.0010) [2023-12-27 08:15:06,858][20381] Updated weights for policy 1, policy_version 377204 (0.0011) [2023-12-27 08:15:06,907][20381] Updated weights for policy 1, policy_version 377214 (0.0011) [2023-12-27 08:15:06,962][20381] Updated weights for policy 1, policy_version 377224 (0.0009) [2023-12-27 08:15:06,981][20347] Updated weights for policy 0, policy_version 377348 (0.0010) [2023-12-27 08:15:07,043][20347] Updated weights for policy 0, policy_version 377358 (0.0009) [2023-12-27 08:15:07,103][20347] Updated weights for policy 0, policy_version 377368 (0.0008) [2023-12-27 08:15:07,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20752.9, 300 sec: 20938.2). Total num frames: 193216512. Throughput: 0: 10467.0, 1: 10492.2. Samples: 193183592. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '970.980')] [2023-12-27 08:15:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000377232_96591872.pth... [2023-12-27 08:15:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000377376_96624640.pth... [2023-12-27 08:15:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000376016_96280576.pth [2023-12-27 08:15:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000376160_96313344.pth [2023-12-27 08:15:07,668][20381] Updated weights for policy 1, policy_version 377234 (0.0011) [2023-12-27 08:15:07,715][20347] Updated weights for policy 0, policy_version 377378 (0.0010) [2023-12-27 08:15:07,720][20381] Updated weights for policy 1, policy_version 377244 (0.0010) [2023-12-27 08:15:07,773][20347] Updated weights for policy 0, policy_version 377388 (0.0009) [2023-12-27 08:15:07,778][20381] Updated weights for policy 1, policy_version 377254 (0.0005) [2023-12-27 08:15:07,827][20347] Updated weights for policy 0, policy_version 377398 (0.0010) [2023-12-27 08:15:07,834][20381] Updated weights for policy 1, policy_version 377264 (0.0006) [2023-12-27 08:15:07,883][20347] Updated weights for policy 0, policy_version 377408 (0.0009) [2023-12-27 08:15:08,516][20381] Updated weights for policy 1, policy_version 377274 (0.0007) [2023-12-27 08:15:08,569][20381] Updated weights for policy 1, policy_version 377284 (0.0008) [2023-12-27 08:15:08,589][20347] Updated weights for policy 0, policy_version 377418 (0.0009) [2023-12-27 08:15:08,632][20381] Updated weights for policy 1, policy_version 377294 (0.0007) [2023-12-27 08:15:08,644][20347] Updated weights for policy 0, policy_version 377428 (0.0007) [2023-12-27 08:15:08,702][20347] Updated weights for policy 0, policy_version 377438 (0.0009) [2023-12-27 08:15:09,348][20347] Updated weights for policy 0, policy_version 377448 (0.0010) [2023-12-27 08:15:09,363][20381] Updated weights for policy 1, policy_version 377304 (0.0007) [2023-12-27 08:15:09,418][20347] Updated weights for policy 0, policy_version 377458 (0.0009) [2023-12-27 08:15:09,429][20381] Updated weights for policy 1, policy_version 377314 (0.0006) [2023-12-27 08:15:09,469][20347] Updated weights for policy 0, policy_version 377468 (0.0008) [2023-12-27 08:15:09,488][20381] Updated weights for policy 1, policy_version 377324 (0.0007) [2023-12-27 08:15:10,150][20347] Updated weights for policy 0, policy_version 377478 (0.0007) [2023-12-27 08:15:10,155][20381] Updated weights for policy 1, policy_version 377334 (0.0007) [2023-12-27 08:15:10,207][20347] Updated weights for policy 0, policy_version 377488 (0.0007) [2023-12-27 08:15:10,221][20381] Updated weights for policy 1, policy_version 377344 (0.0008) [2023-12-27 08:15:10,268][20347] Updated weights for policy 0, policy_version 377498 (0.0007) [2023-12-27 08:15:10,285][20381] Updated weights for policy 1, policy_version 377354 (0.0009) [2023-12-27 08:15:10,964][20347] Updated weights for policy 0, policy_version 377508 (0.0006) [2023-12-27 08:15:10,967][20381] Updated weights for policy 1, policy_version 377364 (0.0008) [2023-12-27 08:15:11,008][20347] Updated weights for policy 0, policy_version 377518 (0.0005) [2023-12-27 08:15:11,033][20381] Updated weights for policy 1, policy_version 377374 (0.0008) [2023-12-27 08:15:11,068][20347] Updated weights for policy 0, policy_version 377528 (0.0006) [2023-12-27 08:15:11,099][20381] Updated weights for policy 1, policy_version 377384 (0.0008) [2023-12-27 08:15:11,778][20381] Updated weights for policy 1, policy_version 377394 (0.0008) [2023-12-27 08:15:11,798][20347] Updated weights for policy 0, policy_version 377538 (0.0007) [2023-12-27 08:15:11,841][20381] Updated weights for policy 1, policy_version 377404 (0.0008) [2023-12-27 08:15:11,852][20347] Updated weights for policy 0, policy_version 377548 (0.0007) [2023-12-27 08:15:11,900][20381] Updated weights for policy 1, policy_version 377414 (0.0008) [2023-12-27 08:15:11,904][20347] Updated weights for policy 0, policy_version 377558 (0.0006) [2023-12-27 08:15:11,955][20381] Updated weights for policy 1, policy_version 377424 (0.0008) [2023-12-27 08:15:11,961][20347] Updated weights for policy 0, policy_version 377568 (0.0005) [2023-12-27 08:15:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20938.2). Total num frames: 193314816. Throughput: 0: 10460.6, 1: 10488.4. Samples: 193306500. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.280')] [2023-12-27 08:15:12,599][20381] Updated weights for policy 1, policy_version 377434 (0.0009) [2023-12-27 08:15:12,614][20347] Updated weights for policy 0, policy_version 377578 (0.0005) [2023-12-27 08:15:12,665][20381] Updated weights for policy 1, policy_version 377444 (0.0008) [2023-12-27 08:15:12,666][20347] Updated weights for policy 0, policy_version 377588 (0.0005) [2023-12-27 08:15:12,723][20381] Updated weights for policy 1, policy_version 377454 (0.0007) [2023-12-27 08:15:12,723][20347] Updated weights for policy 0, policy_version 377598 (0.0005) [2023-12-27 08:15:13,296][20381] Updated weights for policy 1, policy_version 377464 (0.0007) [2023-12-27 08:15:13,326][20347] Updated weights for policy 0, policy_version 377608 (0.0006) [2023-12-27 08:15:13,355][20381] Updated weights for policy 1, policy_version 377474 (0.0008) [2023-12-27 08:15:13,387][20347] Updated weights for policy 0, policy_version 377618 (0.0006) [2023-12-27 08:15:13,417][20381] Updated weights for policy 1, policy_version 377484 (0.0008) [2023-12-27 08:15:13,454][20347] Updated weights for policy 0, policy_version 377628 (0.0006) [2023-12-27 08:15:14,055][20347] Updated weights for policy 0, policy_version 377638 (0.0009) [2023-12-27 08:15:14,082][20381] Updated weights for policy 1, policy_version 377494 (0.0009) [2023-12-27 08:15:14,111][20347] Updated weights for policy 0, policy_version 377648 (0.0010) [2023-12-27 08:15:14,132][20381] Updated weights for policy 1, policy_version 377504 (0.0005) [2023-12-27 08:15:14,172][20347] Updated weights for policy 0, policy_version 377658 (0.0010) [2023-12-27 08:15:14,194][20381] Updated weights for policy 1, policy_version 377514 (0.0006) [2023-12-27 08:15:14,887][20347] Updated weights for policy 0, policy_version 377668 (0.0011) [2023-12-27 08:15:14,895][20381] Updated weights for policy 1, policy_version 377524 (0.0008) [2023-12-27 08:15:14,947][20347] Updated weights for policy 0, policy_version 377678 (0.0011) [2023-12-27 08:15:14,952][20381] Updated weights for policy 1, policy_version 377534 (0.0011) [2023-12-27 08:15:15,004][20347] Updated weights for policy 0, policy_version 377688 (0.0010) [2023-12-27 08:15:15,015][20381] Updated weights for policy 1, policy_version 377544 (0.0010) [2023-12-27 08:15:15,683][20347] Updated weights for policy 0, policy_version 377698 (0.0005) [2023-12-27 08:15:15,713][20381] Updated weights for policy 1, policy_version 377554 (0.0011) [2023-12-27 08:15:15,748][20347] Updated weights for policy 0, policy_version 377708 (0.0006) [2023-12-27 08:15:15,776][20381] Updated weights for policy 1, policy_version 377564 (0.0011) [2023-12-27 08:15:15,806][20347] Updated weights for policy 0, policy_version 377718 (0.0006) [2023-12-27 08:15:15,842][20381] Updated weights for policy 1, policy_version 377574 (0.0011) [2023-12-27 08:15:15,865][20347] Updated weights for policy 0, policy_version 377728 (0.0006) [2023-12-27 08:15:15,903][20381] Updated weights for policy 1, policy_version 377584 (0.0010) [2023-12-27 08:15:16,432][20347] Updated weights for policy 0, policy_version 377738 (0.0005) [2023-12-27 08:15:16,495][20347] Updated weights for policy 0, policy_version 377748 (0.0006) [2023-12-27 08:15:16,563][20347] Updated weights for policy 0, policy_version 377758 (0.0009) [2023-12-27 08:15:16,585][20381] Updated weights for policy 1, policy_version 377594 (0.0010) [2023-12-27 08:15:16,645][20381] Updated weights for policy 1, policy_version 377604 (0.0011) [2023-12-27 08:15:16,701][20381] Updated weights for policy 1, policy_version 377614 (0.0011) [2023-12-27 08:15:17,156][20347] Updated weights for policy 0, policy_version 377768 (0.0006) [2023-12-27 08:15:17,208][20347] Updated weights for policy 0, policy_version 377778 (0.0005) [2023-12-27 08:15:17,262][20347] Updated weights for policy 0, policy_version 377788 (0.0005) [2023-12-27 08:15:17,362][19170] Fps is (10 sec: 20481.1, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 193421312. Throughput: 0: 10502.7, 1: 10372.3. Samples: 193431876. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:17,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 08:15:17,410][20381] Updated weights for policy 1, policy_version 377624 (0.0008) [2023-12-27 08:15:17,467][20381] Updated weights for policy 1, policy_version 377634 (0.0007) [2023-12-27 08:15:17,524][20381] Updated weights for policy 1, policy_version 377644 (0.0008) [2023-12-27 08:15:17,891][20347] Updated weights for policy 0, policy_version 377798 (0.0009) [2023-12-27 08:15:17,943][20347] Updated weights for policy 0, policy_version 377808 (0.0009) [2023-12-27 08:15:17,994][20347] Updated weights for policy 0, policy_version 377818 (0.0006) [2023-12-27 08:15:18,194][20381] Updated weights for policy 1, policy_version 377654 (0.0008) [2023-12-27 08:15:18,250][20381] Updated weights for policy 1, policy_version 377664 (0.0008) [2023-12-27 08:15:18,310][20381] Updated weights for policy 1, policy_version 377674 (0.0008) [2023-12-27 08:15:18,701][20347] Updated weights for policy 0, policy_version 377828 (0.0006) [2023-12-27 08:15:18,756][20347] Updated weights for policy 0, policy_version 377838 (0.0007) [2023-12-27 08:15:18,815][20347] Updated weights for policy 0, policy_version 377848 (0.0008) [2023-12-27 08:15:19,036][20381] Updated weights for policy 1, policy_version 377684 (0.0008) [2023-12-27 08:15:19,092][20381] Updated weights for policy 1, policy_version 377694 (0.0008) [2023-12-27 08:15:19,153][20381] Updated weights for policy 1, policy_version 377704 (0.0008) [2023-12-27 08:15:19,489][20347] Updated weights for policy 0, policy_version 377858 (0.0007) [2023-12-27 08:15:19,547][20347] Updated weights for policy 0, policy_version 377868 (0.0007) [2023-12-27 08:15:19,604][20347] Updated weights for policy 0, policy_version 377878 (0.0007) [2023-12-27 08:15:19,662][20347] Updated weights for policy 0, policy_version 377888 (0.0007) [2023-12-27 08:15:19,896][20381] Updated weights for policy 1, policy_version 377714 (0.0007) [2023-12-27 08:15:19,964][20381] Updated weights for policy 1, policy_version 377724 (0.0009) [2023-12-27 08:15:20,022][20381] Updated weights for policy 1, policy_version 377734 (0.0008) [2023-12-27 08:15:20,086][20381] Updated weights for policy 1, policy_version 377744 (0.0008) [2023-12-27 08:15:20,331][20347] Updated weights for policy 0, policy_version 377898 (0.0006) [2023-12-27 08:15:20,396][20347] Updated weights for policy 0, policy_version 377908 (0.0008) [2023-12-27 08:15:20,458][20347] Updated weights for policy 0, policy_version 377918 (0.0008) [2023-12-27 08:15:20,809][20381] Updated weights for policy 1, policy_version 377754 (0.0008) [2023-12-27 08:15:20,875][20381] Updated weights for policy 1, policy_version 377764 (0.0008) [2023-12-27 08:15:20,936][20381] Updated weights for policy 1, policy_version 377774 (0.0007) [2023-12-27 08:15:21,221][20347] Updated weights for policy 0, policy_version 377928 (0.0008) [2023-12-27 08:15:21,286][20347] Updated weights for policy 0, policy_version 377938 (0.0008) [2023-12-27 08:15:21,343][20347] Updated weights for policy 0, policy_version 377948 (0.0008) [2023-12-27 08:15:21,723][20381] Updated weights for policy 1, policy_version 377784 (0.0008) [2023-12-27 08:15:21,787][20381] Updated weights for policy 1, policy_version 377794 (0.0008) [2023-12-27 08:15:21,849][20381] Updated weights for policy 1, policy_version 377804 (0.0008) [2023-12-27 08:15:22,075][20347] Updated weights for policy 0, policy_version 377958 (0.0007) [2023-12-27 08:15:22,130][20347] Updated weights for policy 0, policy_version 377968 (0.0006) [2023-12-27 08:15:22,187][20347] Updated weights for policy 0, policy_version 377978 (0.0007) [2023-12-27 08:15:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20910.4). Total num frames: 193519616. Throughput: 0: 10475.0, 1: 10257.7. Samples: 193491292. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:22,363][19170] Avg episode reward: [(0, '990.350'), (1, '980.830')] [2023-12-27 08:15:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000377808_96739328.pth... [2023-12-27 08:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000377984_96780288.pth... [2023-12-27 08:15:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000376624_96436224.pth [2023-12-27 08:15:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000376768_96468992.pth [2023-12-27 08:15:22,590][20381] Updated weights for policy 1, policy_version 377814 (0.0007) [2023-12-27 08:15:22,647][20381] Updated weights for policy 1, policy_version 377824 (0.0007) [2023-12-27 08:15:22,705][20381] Updated weights for policy 1, policy_version 377834 (0.0009) [2023-12-27 08:15:22,885][20347] Updated weights for policy 0, policy_version 377988 (0.0008) [2023-12-27 08:15:22,945][20347] Updated weights for policy 0, policy_version 377998 (0.0010) [2023-12-27 08:15:23,004][20347] Updated weights for policy 0, policy_version 378008 (0.0010) [2023-12-27 08:15:23,440][20381] Updated weights for policy 1, policy_version 377844 (0.0007) [2023-12-27 08:15:23,498][20381] Updated weights for policy 1, policy_version 377854 (0.0008) [2023-12-27 08:15:23,544][20381] Updated weights for policy 1, policy_version 377864 (0.0010) [2023-12-27 08:15:23,673][20347] Updated weights for policy 0, policy_version 378018 (0.0010) [2023-12-27 08:15:23,734][20347] Updated weights for policy 0, policy_version 378028 (0.0010) [2023-12-27 08:15:23,794][20347] Updated weights for policy 0, policy_version 378038 (0.0007) [2023-12-27 08:15:23,850][20347] Updated weights for policy 0, policy_version 378048 (0.0008) [2023-12-27 08:15:24,221][20381] Updated weights for policy 1, policy_version 377874 (0.0008) [2023-12-27 08:15:24,270][20381] Updated weights for policy 1, policy_version 377884 (0.0008) [2023-12-27 08:15:24,325][20381] Updated weights for policy 1, policy_version 377894 (0.0007) [2023-12-27 08:15:24,381][20381] Updated weights for policy 1, policy_version 377904 (0.0007) [2023-12-27 08:15:24,516][20347] Updated weights for policy 0, policy_version 378058 (0.0005) [2023-12-27 08:15:24,567][20347] Updated weights for policy 0, policy_version 378068 (0.0005) [2023-12-27 08:15:24,626][20347] Updated weights for policy 0, policy_version 378078 (0.0008) [2023-12-27 08:15:25,014][20381] Updated weights for policy 1, policy_version 377914 (0.0006) [2023-12-27 08:15:25,067][20381] Updated weights for policy 1, policy_version 377924 (0.0008) [2023-12-27 08:15:25,112][20381] Updated weights for policy 1, policy_version 377934 (0.0008) [2023-12-27 08:15:25,258][20347] Updated weights for policy 0, policy_version 378088 (0.0006) [2023-12-27 08:15:25,319][20347] Updated weights for policy 0, policy_version 378098 (0.0005) [2023-12-27 08:15:25,375][20347] Updated weights for policy 0, policy_version 378108 (0.0006) [2023-12-27 08:15:25,764][20381] Updated weights for policy 1, policy_version 377944 (0.0008) [2023-12-27 08:15:25,822][20381] Updated weights for policy 1, policy_version 377954 (0.0007) [2023-12-27 08:15:25,869][20381] Updated weights for policy 1, policy_version 377964 (0.0007) [2023-12-27 08:15:26,021][20347] Updated weights for policy 0, policy_version 378118 (0.0007) [2023-12-27 08:15:26,074][20347] Updated weights for policy 0, policy_version 378128 (0.0008) [2023-12-27 08:15:26,119][20347] Updated weights for policy 0, policy_version 378138 (0.0008) [2023-12-27 08:15:26,543][20381] Updated weights for policy 1, policy_version 377974 (0.0008) [2023-12-27 08:15:26,590][20381] Updated weights for policy 1, policy_version 377984 (0.0008) [2023-12-27 08:15:26,636][20381] Updated weights for policy 1, policy_version 377994 (0.0008) [2023-12-27 08:15:26,729][20347] Updated weights for policy 0, policy_version 378148 (0.0006) [2023-12-27 08:15:26,791][20347] Updated weights for policy 0, policy_version 378158 (0.0006) [2023-12-27 08:15:26,841][20347] Updated weights for policy 0, policy_version 378168 (0.0009) [2023-12-27 08:15:27,260][20381] Updated weights for policy 1, policy_version 378004 (0.0008) [2023-12-27 08:15:27,312][20381] Updated weights for policy 1, policy_version 378014 (0.0007) [2023-12-27 08:15:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 193617920. Throughput: 0: 10454.9, 1: 10174.8. Samples: 193612156. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:27,363][19170] Avg episode reward: [(0, '971.070'), (1, '980.830')] [2023-12-27 08:15:27,367][20381] Updated weights for policy 1, policy_version 378024 (0.0007) [2023-12-27 08:15:27,473][20347] Updated weights for policy 0, policy_version 378178 (0.0010) [2023-12-27 08:15:27,523][20347] Updated weights for policy 0, policy_version 378188 (0.0008) [2023-12-27 08:15:27,578][20347] Updated weights for policy 0, policy_version 378198 (0.0006) [2023-12-27 08:15:27,639][20347] Updated weights for policy 0, policy_version 378208 (0.0006) [2023-12-27 08:15:27,923][20381] Updated weights for policy 1, policy_version 378034 (0.0007) [2023-12-27 08:15:27,971][20381] Updated weights for policy 1, policy_version 378044 (0.0006) [2023-12-27 08:15:28,025][20381] Updated weights for policy 1, policy_version 378054 (0.0007) [2023-12-27 08:15:28,069][20381] Updated weights for policy 1, policy_version 378064 (0.0007) [2023-12-27 08:15:28,207][20347] Updated weights for policy 0, policy_version 378218 (0.0010) [2023-12-27 08:15:28,251][20347] Updated weights for policy 0, policy_version 378228 (0.0010) [2023-12-27 08:15:28,303][20347] Updated weights for policy 0, policy_version 378238 (0.0010) [2023-12-27 08:15:28,719][20381] Updated weights for policy 1, policy_version 378074 (0.0005) [2023-12-27 08:15:28,775][20381] Updated weights for policy 1, policy_version 378084 (0.0006) [2023-12-27 08:15:28,826][20381] Updated weights for policy 1, policy_version 378094 (0.0007) [2023-12-27 08:15:29,006][20347] Updated weights for policy 0, policy_version 378248 (0.0009) [2023-12-27 08:15:29,070][20347] Updated weights for policy 0, policy_version 378258 (0.0010) [2023-12-27 08:15:29,128][20347] Updated weights for policy 0, policy_version 378268 (0.0010) [2023-12-27 08:15:29,493][20381] Updated weights for policy 1, policy_version 378104 (0.0008) [2023-12-27 08:15:29,540][20381] Updated weights for policy 1, policy_version 378114 (0.0005) [2023-12-27 08:15:29,597][20381] Updated weights for policy 1, policy_version 378124 (0.0005) [2023-12-27 08:15:29,832][20347] Updated weights for policy 0, policy_version 378278 (0.0009) [2023-12-27 08:15:29,892][20347] Updated weights for policy 0, policy_version 378288 (0.0008) [2023-12-27 08:15:29,951][20347] Updated weights for policy 0, policy_version 378298 (0.0007) [2023-12-27 08:15:30,273][20381] Updated weights for policy 1, policy_version 378134 (0.0007) [2023-12-27 08:15:30,325][20381] Updated weights for policy 1, policy_version 378144 (0.0008) [2023-12-27 08:15:30,387][20381] Updated weights for policy 1, policy_version 378154 (0.0007) [2023-12-27 08:15:30,608][20347] Updated weights for policy 0, policy_version 378308 (0.0007) [2023-12-27 08:15:30,665][20347] Updated weights for policy 0, policy_version 378318 (0.0007) [2023-12-27 08:15:30,723][20347] Updated weights for policy 0, policy_version 378328 (0.0008) [2023-12-27 08:15:30,963][20381] Updated weights for policy 1, policy_version 378164 (0.0008) [2023-12-27 08:15:31,018][20381] Updated weights for policy 1, policy_version 378174 (0.0011) [2023-12-27 08:15:31,081][20381] Updated weights for policy 1, policy_version 378184 (0.0011) [2023-12-27 08:15:31,402][20347] Updated weights for policy 0, policy_version 378338 (0.0008) [2023-12-27 08:15:31,468][20347] Updated weights for policy 0, policy_version 378348 (0.0009) [2023-12-27 08:15:31,530][20347] Updated weights for policy 0, policy_version 378358 (0.0008) [2023-12-27 08:15:31,584][20347] Updated weights for policy 0, policy_version 378368 (0.0007) [2023-12-27 08:15:31,810][20381] Updated weights for policy 1, policy_version 378194 (0.0010) [2023-12-27 08:15:31,865][20381] Updated weights for policy 1, policy_version 378204 (0.0005) [2023-12-27 08:15:31,920][20381] Updated weights for policy 1, policy_version 378214 (0.0005) [2023-12-27 08:15:31,977][20381] Updated weights for policy 1, policy_version 378224 (0.0006) [2023-12-27 08:15:32,333][20347] Updated weights for policy 0, policy_version 378378 (0.0008) [2023-12-27 08:15:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 193724416. Throughput: 0: 10453.2, 1: 10268.4. Samples: 193739768. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:32,363][19170] Avg episode reward: [(0, '961.570'), (1, '990.460')] [2023-12-27 08:15:32,390][20347] Updated weights for policy 0, policy_version 378388 (0.0007) [2023-12-27 08:15:32,447][20347] Updated weights for policy 0, policy_version 378398 (0.0008) [2023-12-27 08:15:32,606][20381] Updated weights for policy 1, policy_version 378234 (0.0008) [2023-12-27 08:15:32,659][20381] Updated weights for policy 1, policy_version 378244 (0.0008) [2023-12-27 08:15:32,714][20381] Updated weights for policy 1, policy_version 378254 (0.0010) [2023-12-27 08:15:33,160][20347] Updated weights for policy 0, policy_version 378408 (0.0007) [2023-12-27 08:15:33,216][20347] Updated weights for policy 0, policy_version 378418 (0.0006) [2023-12-27 08:15:33,272][20347] Updated weights for policy 0, policy_version 378428 (0.0005) [2023-12-27 08:15:33,396][20381] Updated weights for policy 1, policy_version 378264 (0.0007) [2023-12-27 08:15:33,451][20381] Updated weights for policy 1, policy_version 378274 (0.0009) [2023-12-27 08:15:33,500][20381] Updated weights for policy 1, policy_version 378284 (0.0010) [2023-12-27 08:15:33,831][20347] Updated weights for policy 0, policy_version 378438 (0.0007) [2023-12-27 08:15:33,885][20347] Updated weights for policy 0, policy_version 378448 (0.0008) [2023-12-27 08:15:33,932][20347] Updated weights for policy 0, policy_version 378458 (0.0009) [2023-12-27 08:15:34,062][20381] Updated weights for policy 1, policy_version 378294 (0.0010) [2023-12-27 08:15:34,116][20381] Updated weights for policy 1, policy_version 378304 (0.0010) [2023-12-27 08:15:34,178][20381] Updated weights for policy 1, policy_version 378314 (0.0010) [2023-12-27 08:15:34,589][20347] Updated weights for policy 0, policy_version 378468 (0.0008) [2023-12-27 08:15:34,645][20347] Updated weights for policy 0, policy_version 378478 (0.0008) [2023-12-27 08:15:34,698][20347] Updated weights for policy 0, policy_version 378488 (0.0009) [2023-12-27 08:15:34,806][20381] Updated weights for policy 1, policy_version 378324 (0.0009) [2023-12-27 08:15:34,850][20381] Updated weights for policy 1, policy_version 378334 (0.0005) [2023-12-27 08:15:34,894][20381] Updated weights for policy 1, policy_version 378344 (0.0005) [2023-12-27 08:15:35,360][20347] Updated weights for policy 0, policy_version 378498 (0.0007) [2023-12-27 08:15:35,414][20347] Updated weights for policy 0, policy_version 378508 (0.0005) [2023-12-27 08:15:35,476][20347] Updated weights for policy 0, policy_version 378518 (0.0008) [2023-12-27 08:15:35,534][20347] Updated weights for policy 0, policy_version 378528 (0.0008) [2023-12-27 08:15:35,561][20381] Updated weights for policy 1, policy_version 378354 (0.0006) [2023-12-27 08:15:35,616][20381] Updated weights for policy 1, policy_version 378364 (0.0008) [2023-12-27 08:15:35,662][20381] Updated weights for policy 1, policy_version 378374 (0.0007) [2023-12-27 08:15:35,707][20381] Updated weights for policy 1, policy_version 378384 (0.0006) [2023-12-27 08:15:36,188][20347] Updated weights for policy 0, policy_version 378538 (0.0008) [2023-12-27 08:15:36,252][20347] Updated weights for policy 0, policy_version 378548 (0.0007) [2023-12-27 08:15:36,308][20347] Updated weights for policy 0, policy_version 378558 (0.0007) [2023-12-27 08:15:36,363][20381] Updated weights for policy 1, policy_version 378394 (0.0009) [2023-12-27 08:15:36,423][20381] Updated weights for policy 1, policy_version 378404 (0.0008) [2023-12-27 08:15:36,480][20381] Updated weights for policy 1, policy_version 378414 (0.0007) [2023-12-27 08:15:36,963][20347] Updated weights for policy 0, policy_version 378568 (0.0008) [2023-12-27 08:15:37,008][20347] Updated weights for policy 0, policy_version 378578 (0.0008) [2023-12-27 08:15:37,067][20347] Updated weights for policy 0, policy_version 378588 (0.0008) [2023-12-27 08:15:37,128][20381] Updated weights for policy 1, policy_version 378424 (0.0007) [2023-12-27 08:15:37,174][20381] Updated weights for policy 1, policy_version 378434 (0.0008) [2023-12-27 08:15:37,226][20381] Updated weights for policy 1, policy_version 378444 (0.0008) [2023-12-27 08:15:37,363][19170] Fps is (10 sec: 22118.0, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 193839104. Throughput: 0: 10459.0, 1: 10313.8. Samples: 193803604. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) [2023-12-27 08:15:37,363][19170] Avg episode reward: [(0, '980.820'), (1, '990.460')] [2023-12-27 08:15:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000378592_96935936.pth... [2023-12-27 08:15:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000378448_96903168.pth... [2023-12-27 08:15:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000377376_96624640.pth [2023-12-27 08:15:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000377232_96591872.pth [2023-12-27 08:15:37,802][20347] Updated weights for policy 0, policy_version 378598 (0.0009) [2023-12-27 08:15:37,859][20347] Updated weights for policy 0, policy_version 378608 (0.0007) [2023-12-27 08:15:37,879][20381] Updated weights for policy 1, policy_version 378454 (0.0007) [2023-12-27 08:15:37,919][20347] Updated weights for policy 0, policy_version 378618 (0.0007) [2023-12-27 08:15:37,937][20381] Updated weights for policy 1, policy_version 378464 (0.0007) [2023-12-27 08:15:37,988][20381] Updated weights for policy 1, policy_version 378474 (0.0006) [2023-12-27 08:15:38,597][20381] Updated weights for policy 1, policy_version 378484 (0.0006) [2023-12-27 08:15:38,612][20347] Updated weights for policy 0, policy_version 378628 (0.0006) [2023-12-27 08:15:38,651][20381] Updated weights for policy 1, policy_version 378494 (0.0008) [2023-12-27 08:15:38,672][20347] Updated weights for policy 0, policy_version 378638 (0.0005) [2023-12-27 08:15:38,698][20381] Updated weights for policy 1, policy_version 378504 (0.0008) [2023-12-27 08:15:38,730][20347] Updated weights for policy 0, policy_version 378648 (0.0007) [2023-12-27 08:15:39,354][20381] Updated weights for policy 1, policy_version 378514 (0.0009) [2023-12-27 08:15:39,355][20347] Updated weights for policy 0, policy_version 378658 (0.0007) [2023-12-27 08:15:39,420][20381] Updated weights for policy 1, policy_version 378524 (0.0008) [2023-12-27 08:15:39,429][20347] Updated weights for policy 0, policy_version 378668 (0.0008) [2023-12-27 08:15:39,478][20381] Updated weights for policy 1, policy_version 378534 (0.0008) [2023-12-27 08:15:39,492][20347] Updated weights for policy 0, policy_version 378678 (0.0007) [2023-12-27 08:15:39,539][20381] Updated weights for policy 1, policy_version 378544 (0.0007) [2023-12-27 08:15:39,559][20347] Updated weights for policy 0, policy_version 378688 (0.0006) [2023-12-27 08:15:40,266][20381] Updated weights for policy 1, policy_version 378554 (0.0006) [2023-12-27 08:15:40,298][20347] Updated weights for policy 0, policy_version 378698 (0.0008) [2023-12-27 08:15:40,328][20381] Updated weights for policy 1, policy_version 378564 (0.0010) [2023-12-27 08:15:40,356][20347] Updated weights for policy 0, policy_version 378708 (0.0007) [2023-12-27 08:15:40,385][20381] Updated weights for policy 1, policy_version 378574 (0.0011) [2023-12-27 08:15:40,418][20347] Updated weights for policy 0, policy_version 378718 (0.0006) [2023-12-27 08:15:41,023][20381] Updated weights for policy 1, policy_version 378584 (0.0011) [2023-12-27 08:15:41,089][20381] Updated weights for policy 1, policy_version 378594 (0.0010) [2023-12-27 08:15:41,148][20347] Updated weights for policy 0, policy_version 378728 (0.0007) [2023-12-27 08:15:41,159][20381] Updated weights for policy 1, policy_version 378604 (0.0009) [2023-12-27 08:15:41,215][20347] Updated weights for policy 0, policy_version 378738 (0.0008) [2023-12-27 08:15:41,277][20347] Updated weights for policy 0, policy_version 378748 (0.0008) [2023-12-27 08:15:41,877][20381] Updated weights for policy 1, policy_version 378614 (0.0009) [2023-12-27 08:15:41,937][20381] Updated weights for policy 1, policy_version 378624 (0.0011) [2023-12-27 08:15:41,972][20347] Updated weights for policy 0, policy_version 378758 (0.0009) [2023-12-27 08:15:41,994][20381] Updated weights for policy 1, policy_version 378634 (0.0007) [2023-12-27 08:15:42,034][20347] Updated weights for policy 0, policy_version 378768 (0.0009) [2023-12-27 08:15:42,098][20347] Updated weights for policy 0, policy_version 378778 (0.0009) [2023-12-27 08:15:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 193937408. Throughput: 0: 10379.4, 1: 10314.8. Samples: 193926416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:15:42,363][19170] Avg episode reward: [(0, '922.530'), (1, '1000.000')] [2023-12-27 08:15:42,564][20381] Updated weights for policy 1, policy_version 378644 (0.0006) [2023-12-27 08:15:42,628][20381] Updated weights for policy 1, policy_version 378654 (0.0007) [2023-12-27 08:15:42,685][20381] Updated weights for policy 1, policy_version 378664 (0.0007) [2023-12-27 08:15:42,808][20347] Updated weights for policy 0, policy_version 378788 (0.0008) [2023-12-27 08:15:42,861][20347] Updated weights for policy 0, policy_version 378798 (0.0008) [2023-12-27 08:15:42,915][20347] Updated weights for policy 0, policy_version 378808 (0.0009) [2023-12-27 08:15:43,298][20381] Updated weights for policy 1, policy_version 378674 (0.0005) [2023-12-27 08:15:43,350][20381] Updated weights for policy 1, policy_version 378684 (0.0005) [2023-12-27 08:15:43,406][20381] Updated weights for policy 1, policy_version 378694 (0.0005) [2023-12-27 08:15:43,464][20381] Updated weights for policy 1, policy_version 378704 (0.0005) [2023-12-27 08:15:43,501][20347] Updated weights for policy 0, policy_version 378818 (0.0008) [2023-12-27 08:15:43,548][20347] Updated weights for policy 0, policy_version 378828 (0.0010) [2023-12-27 08:15:43,594][20347] Updated weights for policy 0, policy_version 378838 (0.0010) [2023-12-27 08:15:43,640][20347] Updated weights for policy 0, policy_version 378848 (0.0010) [2023-12-27 08:15:43,995][20381] Updated weights for policy 1, policy_version 378714 (0.0007) [2023-12-27 08:15:44,050][20381] Updated weights for policy 1, policy_version 378724 (0.0008) [2023-12-27 08:15:44,099][20381] Updated weights for policy 1, policy_version 378734 (0.0008) [2023-12-27 08:15:44,309][20347] Updated weights for policy 0, policy_version 378858 (0.0008) [2023-12-27 08:15:44,372][20347] Updated weights for policy 0, policy_version 378868 (0.0008) [2023-12-27 08:15:44,441][20347] Updated weights for policy 0, policy_version 378878 (0.0008) [2023-12-27 08:15:44,750][20381] Updated weights for policy 1, policy_version 378744 (0.0006) [2023-12-27 08:15:44,810][20381] Updated weights for policy 1, policy_version 378754 (0.0008) [2023-12-27 08:15:44,874][20381] Updated weights for policy 1, policy_version 378764 (0.0007) [2023-12-27 08:15:45,116][20347] Updated weights for policy 0, policy_version 378888 (0.0006) [2023-12-27 08:15:45,185][20347] Updated weights for policy 0, policy_version 378898 (0.0006) [2023-12-27 08:15:45,253][20347] Updated weights for policy 0, policy_version 378908 (0.0006) [2023-12-27 08:15:45,540][20381] Updated weights for policy 1, policy_version 378774 (0.0007) [2023-12-27 08:15:45,589][20381] Updated weights for policy 1, policy_version 378784 (0.0008) [2023-12-27 08:15:45,645][20381] Updated weights for policy 1, policy_version 378794 (0.0006) [2023-12-27 08:15:45,884][20347] Updated weights for policy 0, policy_version 378918 (0.0005) [2023-12-27 08:15:45,935][20347] Updated weights for policy 0, policy_version 378928 (0.0005) [2023-12-27 08:15:45,982][20347] Updated weights for policy 0, policy_version 378938 (0.0005) [2023-12-27 08:15:46,259][20381] Updated weights for policy 1, policy_version 378804 (0.0007) [2023-12-27 08:15:46,307][20381] Updated weights for policy 1, policy_version 378814 (0.0010) [2023-12-27 08:15:46,352][20381] Updated weights for policy 1, policy_version 378824 (0.0010) [2023-12-27 08:15:46,601][20347] Updated weights for policy 0, policy_version 378948 (0.0006) [2023-12-27 08:15:46,658][20347] Updated weights for policy 0, policy_version 378958 (0.0008) [2023-12-27 08:15:46,709][20347] Updated weights for policy 0, policy_version 378968 (0.0008) [2023-12-27 08:15:47,004][20381] Updated weights for policy 1, policy_version 378834 (0.0010) [2023-12-27 08:15:47,062][20381] Updated weights for policy 1, policy_version 378844 (0.0008) [2023-12-27 08:15:47,113][20381] Updated weights for policy 1, policy_version 378854 (0.0008) [2023-12-27 08:15:47,162][20381] Updated weights for policy 1, policy_version 378864 (0.0008) [2023-12-27 08:15:47,332][20347] Updated weights for policy 0, policy_version 378978 (0.0010) [2023-12-27 08:15:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 194043904. Throughput: 0: 10402.6, 1: 10399.3. Samples: 194054472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:15:47,363][19170] Avg episode reward: [(0, '822.280'), (1, '1000.000')] [2023-12-27 08:15:47,383][20347] Updated weights for policy 0, policy_version 378988 (0.0007) [2023-12-27 08:15:47,432][20347] Updated weights for policy 0, policy_version 378998 (0.0009) [2023-12-27 08:15:47,477][20347] Updated weights for policy 0, policy_version 379008 (0.0010) [2023-12-27 08:15:47,822][20381] Updated weights for policy 1, policy_version 378874 (0.0007) [2023-12-27 08:15:47,885][20381] Updated weights for policy 1, policy_version 378884 (0.0008) [2023-12-27 08:15:47,935][20381] Updated weights for policy 1, policy_version 378894 (0.0008) [2023-12-27 08:15:48,171][20347] Updated weights for policy 0, policy_version 379018 (0.0006) [2023-12-27 08:15:48,225][20347] Updated weights for policy 0, policy_version 379028 (0.0009) [2023-12-27 08:15:48,270][20347] Updated weights for policy 0, policy_version 379038 (0.0010) [2023-12-27 08:15:48,617][20381] Updated weights for policy 1, policy_version 378904 (0.0006) [2023-12-27 08:15:48,675][20381] Updated weights for policy 1, policy_version 378914 (0.0005) [2023-12-27 08:15:48,729][20381] Updated weights for policy 1, policy_version 378924 (0.0006) [2023-12-27 08:15:48,919][20347] Updated weights for policy 0, policy_version 379048 (0.0011) [2023-12-27 08:15:48,977][20347] Updated weights for policy 0, policy_version 379058 (0.0010) [2023-12-27 08:15:49,021][20347] Updated weights for policy 0, policy_version 379068 (0.0008) [2023-12-27 08:15:49,331][20381] Updated weights for policy 1, policy_version 378934 (0.0006) [2023-12-27 08:15:49,395][20381] Updated weights for policy 1, policy_version 378944 (0.0007) [2023-12-27 08:15:49,453][20381] Updated weights for policy 1, policy_version 378954 (0.0008) [2023-12-27 08:15:49,696][20347] Updated weights for policy 0, policy_version 379078 (0.0007) [2023-12-27 08:15:49,757][20347] Updated weights for policy 0, policy_version 379088 (0.0008) [2023-12-27 08:15:49,810][20347] Updated weights for policy 0, policy_version 379098 (0.0008) [2023-12-27 08:15:50,136][20381] Updated weights for policy 1, policy_version 378964 (0.0009) [2023-12-27 08:15:50,192][20381] Updated weights for policy 1, policy_version 378974 (0.0007) [2023-12-27 08:15:50,253][20381] Updated weights for policy 1, policy_version 378984 (0.0007) [2023-12-27 08:15:50,510][20347] Updated weights for policy 0, policy_version 379108 (0.0008) [2023-12-27 08:15:50,567][20347] Updated weights for policy 0, policy_version 379118 (0.0005) [2023-12-27 08:15:50,632][20347] Updated weights for policy 0, policy_version 379128 (0.0010) [2023-12-27 08:15:50,922][20381] Updated weights for policy 1, policy_version 378994 (0.0008) [2023-12-27 08:15:50,982][20381] Updated weights for policy 1, policy_version 379004 (0.0005) [2023-12-27 08:15:51,039][20381] Updated weights for policy 1, policy_version 379014 (0.0007) [2023-12-27 08:15:51,096][20381] Updated weights for policy 1, policy_version 379024 (0.0007) [2023-12-27 08:15:51,402][20347] Updated weights for policy 0, policy_version 379138 (0.0010) [2023-12-27 08:15:51,453][20347] Updated weights for policy 0, policy_version 379148 (0.0008) [2023-12-27 08:15:51,514][20347] Updated weights for policy 0, policy_version 379158 (0.0008) [2023-12-27 08:15:51,562][20347] Updated weights for policy 0, policy_version 379168 (0.0008) [2023-12-27 08:15:51,774][20381] Updated weights for policy 1, policy_version 379034 (0.0008) [2023-12-27 08:15:51,837][20381] Updated weights for policy 1, policy_version 379044 (0.0008) [2023-12-27 08:15:51,896][20381] Updated weights for policy 1, policy_version 379054 (0.0008) [2023-12-27 08:15:52,286][20347] Updated weights for policy 0, policy_version 379178 (0.0007) [2023-12-27 08:15:52,350][20347] Updated weights for policy 0, policy_version 379188 (0.0008) [2023-12-27 08:15:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 194142208. Throughput: 0: 10346.4, 1: 10413.8. Samples: 194117796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:15:52,363][19170] Avg episode reward: [(0, '728.990'), (1, '1000.000')] [2023-12-27 08:15:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000379056_97058816.pth... [2023-12-27 08:15:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000377808_96739328.pth [2023-12-27 08:15:52,415][20347] Updated weights for policy 0, policy_version 379198 (0.0006) [2023-12-27 08:15:52,425][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000379200_97091584.pth... [2023-12-27 08:15:52,430][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000377984_96780288.pth [2023-12-27 08:15:52,568][20381] Updated weights for policy 1, policy_version 379064 (0.0008) [2023-12-27 08:15:52,617][20381] Updated weights for policy 1, policy_version 379074 (0.0007) [2023-12-27 08:15:52,667][20381] Updated weights for policy 1, policy_version 379084 (0.0008) [2023-12-27 08:15:53,075][20347] Updated weights for policy 0, policy_version 379208 (0.0006) [2023-12-27 08:15:53,131][20347] Updated weights for policy 0, policy_version 379218 (0.0005) [2023-12-27 08:15:53,187][20347] Updated weights for policy 0, policy_version 379228 (0.0005) [2023-12-27 08:15:53,410][20381] Updated weights for policy 1, policy_version 379094 (0.0008) [2023-12-27 08:15:53,466][20381] Updated weights for policy 1, policy_version 379104 (0.0007) [2023-12-27 08:15:53,525][20381] Updated weights for policy 1, policy_version 379114 (0.0005) [2023-12-27 08:15:53,725][20347] Updated weights for policy 0, policy_version 379238 (0.0006) [2023-12-27 08:15:53,783][20347] Updated weights for policy 0, policy_version 379248 (0.0005) [2023-12-27 08:15:53,839][20347] Updated weights for policy 0, policy_version 379258 (0.0007) [2023-12-27 08:15:54,174][20381] Updated weights for policy 1, policy_version 379124 (0.0006) [2023-12-27 08:15:54,238][20381] Updated weights for policy 1, policy_version 379134 (0.0007) [2023-12-27 08:15:54,291][20381] Updated weights for policy 1, policy_version 379144 (0.0008) [2023-12-27 08:15:54,444][20347] Updated weights for policy 0, policy_version 379268 (0.0006) [2023-12-27 08:15:54,502][20347] Updated weights for policy 0, policy_version 379278 (0.0007) [2023-12-27 08:15:54,530][20291] KL-divergence is very high: 124.8502 [2023-12-27 08:15:54,536][20291] KL-divergence is very high: 115.7199 [2023-12-27 08:15:54,559][20347] Updated weights for policy 0, policy_version 379288 (0.0007) [2023-12-27 08:15:54,975][20381] Updated weights for policy 1, policy_version 379154 (0.0008) [2023-12-27 08:15:55,033][20381] Updated weights for policy 1, policy_version 379164 (0.0006) [2023-12-27 08:15:55,081][20381] Updated weights for policy 1, policy_version 379174 (0.0007) [2023-12-27 08:15:55,134][20381] Updated weights for policy 1, policy_version 379184 (0.0008) [2023-12-27 08:15:55,240][20347] Updated weights for policy 0, policy_version 379298 (0.0008) [2023-12-27 08:15:55,293][20347] Updated weights for policy 0, policy_version 379308 (0.0007) [2023-12-27 08:15:55,337][20347] Updated weights for policy 0, policy_version 379318 (0.0008) [2023-12-27 08:15:55,383][20347] Updated weights for policy 0, policy_version 379328 (0.0006) [2023-12-27 08:15:55,786][20381] Updated weights for policy 1, policy_version 379194 (0.0008) [2023-12-27 08:15:55,841][20381] Updated weights for policy 1, policy_version 379204 (0.0008) [2023-12-27 08:15:55,907][20381] Updated weights for policy 1, policy_version 379214 (0.0008) [2023-12-27 08:15:56,083][20347] Updated weights for policy 0, policy_version 379338 (0.0008) [2023-12-27 08:15:56,129][20347] Updated weights for policy 0, policy_version 379348 (0.0008) [2023-12-27 08:15:56,177][20347] Updated weights for policy 0, policy_version 379358 (0.0008) [2023-12-27 08:15:56,598][20381] Updated weights for policy 1, policy_version 379224 (0.0008) [2023-12-27 08:15:56,649][20381] Updated weights for policy 1, policy_version 379234 (0.0007) [2023-12-27 08:15:56,708][20381] Updated weights for policy 1, policy_version 379244 (0.0007) [2023-12-27 08:15:56,882][20347] Updated weights for policy 0, policy_version 379368 (0.0010) [2023-12-27 08:15:56,939][20347] Updated weights for policy 0, policy_version 379378 (0.0006) [2023-12-27 08:15:56,997][20347] Updated weights for policy 0, policy_version 379388 (0.0005) [2023-12-27 08:15:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 194248704. Throughput: 0: 10367.6, 1: 10406.4. Samples: 194241328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:15:57,363][19170] Avg episode reward: [(0, '630.430'), (1, '1000.000')] [2023-12-27 08:15:57,415][20381] Updated weights for policy 1, policy_version 379254 (0.0009) [2023-12-27 08:15:57,463][20381] Updated weights for policy 1, policy_version 379264 (0.0008) [2023-12-27 08:15:57,516][20381] Updated weights for policy 1, policy_version 379274 (0.0008) [2023-12-27 08:15:57,537][20291] KL-divergence is very high: 120.6022 [2023-12-27 08:15:57,558][20347] Updated weights for policy 0, policy_version 379398 (0.0005) [2023-12-27 08:15:57,568][20291] KL-divergence is very high: 367.5597 [2023-12-27 08:15:57,572][20291] KL-divergence is very high: 266.6092 [2023-12-27 08:15:57,606][20347] Updated weights for policy 0, policy_version 379408 (0.0005) [2023-12-27 08:15:57,606][20291] KL-divergence is very high: 290.4024 [2023-12-27 08:15:57,612][20291] KL-divergence is very high: 227.1885 [2023-12-27 08:15:57,651][20291] KL-divergence is very high: 198.4145 [2023-12-27 08:15:57,656][20291] KL-divergence is very high: 255.0664 [2023-12-27 08:15:57,661][20347] Updated weights for policy 0, policy_version 379418 (0.0005) [2023-12-27 08:15:58,249][20347] Updated weights for policy 0, policy_version 379428 (0.0007) [2023-12-27 08:15:58,277][20381] Updated weights for policy 1, policy_version 379285 (0.0008) [2023-12-27 08:15:58,304][20347] Updated weights for policy 0, policy_version 379438 (0.0010) [2023-12-27 08:15:58,333][20381] Updated weights for policy 1, policy_version 379295 (0.0007) [2023-12-27 08:15:58,367][20347] Updated weights for policy 0, policy_version 379448 (0.0009) [2023-12-27 08:15:58,395][20381] Updated weights for policy 1, policy_version 379305 (0.0008) [2023-12-27 08:15:59,074][20347] Updated weights for policy 0, policy_version 379458 (0.0010) [2023-12-27 08:15:59,132][20347] Updated weights for policy 0, policy_version 379468 (0.0008) [2023-12-27 08:15:59,192][20347] Updated weights for policy 0, policy_version 379478 (0.0008) [2023-12-27 08:15:59,229][20381] Updated weights for policy 1, policy_version 379315 (0.0008) [2023-12-27 08:15:59,259][20347] Updated weights for policy 0, policy_version 379488 (0.0009) [2023-12-27 08:15:59,293][20381] Updated weights for policy 1, policy_version 379325 (0.0007) [2023-12-27 08:15:59,345][20381] Updated weights for policy 1, policy_version 379335 (0.0008) [2023-12-27 08:15:59,871][20291] KL-divergence is very high: 128.0500 [2023-12-27 08:15:59,893][20347] Updated weights for policy 0, policy_version 379498 (0.0008) [2023-12-27 08:15:59,915][20291] KL-divergence is very high: 110.4615 [2023-12-27 08:15:59,951][20347] Updated weights for policy 0, policy_version 379508 (0.0008) [2023-12-27 08:15:59,977][20381] Updated weights for policy 1, policy_version 379345 (0.0007) [2023-12-27 08:15:59,990][20291] KL-divergence is very high: 113.9104 [2023-12-27 08:16:00,037][20381] Updated weights for policy 1, policy_version 379355 (0.0006) [2023-12-27 08:16:00,038][20347] Updated weights for policy 0, policy_version 379518 (0.0005) [2023-12-27 08:16:00,039][20291] KL-divergence is very high: 122.4539 [2023-12-27 08:16:00,084][20381] Updated weights for policy 1, policy_version 379365 (0.0009) [2023-12-27 08:16:00,131][20381] Updated weights for policy 1, policy_version 379375 (0.0010) [2023-12-27 08:16:00,591][20347] Updated weights for policy 0, policy_version 379528 (0.0005) [2023-12-27 08:16:00,650][20347] Updated weights for policy 0, policy_version 379538 (0.0007) [2023-12-27 08:16:00,698][20347] Updated weights for policy 0, policy_version 379548 (0.0008) [2023-12-27 08:16:00,868][20381] Updated weights for policy 1, policy_version 379385 (0.0007) [2023-12-27 08:16:00,930][20381] Updated weights for policy 1, policy_version 379395 (0.0008) [2023-12-27 08:16:00,983][20381] Updated weights for policy 1, policy_version 379405 (0.0007) [2023-12-27 08:16:01,378][20347] Updated weights for policy 0, policy_version 379558 (0.0008) [2023-12-27 08:16:01,435][20347] Updated weights for policy 0, policy_version 379568 (0.0008) [2023-12-27 08:16:01,491][20347] Updated weights for policy 0, policy_version 379578 (0.0007) [2023-12-27 08:16:01,647][20381] Updated weights for policy 1, policy_version 379415 (0.0008) [2023-12-27 08:16:01,699][20381] Updated weights for policy 1, policy_version 379425 (0.0007) [2023-12-27 08:16:01,757][20381] Updated weights for policy 1, policy_version 379435 (0.0008) [2023-12-27 08:16:02,130][20347] Updated weights for policy 0, policy_version 379588 (0.0008) [2023-12-27 08:16:02,182][20347] Updated weights for policy 0, policy_version 379598 (0.0007) [2023-12-27 08:16:02,228][20347] Updated weights for policy 0, policy_version 379608 (0.0007) [2023-12-27 08:16:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 194355200. Throughput: 0: 10375.8, 1: 10363.0. Samples: 194365124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:02,363][19170] Avg episode reward: [(0, '386.580'), (1, '1000.000')] [2023-12-27 08:16:02,431][20381] Updated weights for policy 1, policy_version 379445 (0.0007) [2023-12-27 08:16:02,487][20381] Updated weights for policy 1, policy_version 379455 (0.0008) [2023-12-27 08:16:02,543][20381] Updated weights for policy 1, policy_version 379465 (0.0006) [2023-12-27 08:16:02,978][20347] Updated weights for policy 0, policy_version 379618 (0.0008) [2023-12-27 08:16:03,035][20347] Updated weights for policy 0, policy_version 379628 (0.0007) [2023-12-27 08:16:03,096][20347] Updated weights for policy 0, policy_version 379638 (0.0007) [2023-12-27 08:16:03,155][20347] Updated weights for policy 0, policy_version 379648 (0.0007) [2023-12-27 08:16:03,183][20381] Updated weights for policy 1, policy_version 379475 (0.0008) [2023-12-27 08:16:03,237][20381] Updated weights for policy 1, policy_version 379485 (0.0007) [2023-12-27 08:16:03,290][20381] Updated weights for policy 1, policy_version 379495 (0.0006) [2023-12-27 08:16:03,840][20381] Updated weights for policy 1, policy_version 379505 (0.0006) [2023-12-27 08:16:03,855][20347] Updated weights for policy 0, policy_version 379658 (0.0008) [2023-12-27 08:16:03,900][20381] Updated weights for policy 1, policy_version 379515 (0.0009) [2023-12-27 08:16:03,916][20347] Updated weights for policy 0, policy_version 379668 (0.0006) [2023-12-27 08:16:03,963][20381] Updated weights for policy 1, policy_version 379525 (0.0009) [2023-12-27 08:16:03,976][20347] Updated weights for policy 0, policy_version 379678 (0.0006) [2023-12-27 08:16:04,021][20381] Updated weights for policy 1, policy_version 379535 (0.0008) [2023-12-27 08:16:04,621][20347] Updated weights for policy 0, policy_version 379688 (0.0008) [2023-12-27 08:16:04,677][20347] Updated weights for policy 0, policy_version 379698 (0.0008) [2023-12-27 08:16:04,727][20381] Updated weights for policy 1, policy_version 379545 (0.0007) [2023-12-27 08:16:04,736][20347] Updated weights for policy 0, policy_version 379708 (0.0007) [2023-12-27 08:16:04,791][20381] Updated weights for policy 1, policy_version 379555 (0.0008) [2023-12-27 08:16:04,853][20381] Updated weights for policy 1, policy_version 379565 (0.0006) [2023-12-27 08:16:05,411][20347] Updated weights for policy 0, policy_version 379718 (0.0007) [2023-12-27 08:16:05,430][20381] Updated weights for policy 1, policy_version 379575 (0.0006) [2023-12-27 08:16:05,463][20347] Updated weights for policy 0, policy_version 379728 (0.0007) [2023-12-27 08:16:05,476][20381] Updated weights for policy 1, policy_version 379585 (0.0005) [2023-12-27 08:16:05,514][20347] Updated weights for policy 0, policy_version 379738 (0.0007) [2023-12-27 08:16:05,526][20381] Updated weights for policy 1, policy_version 379595 (0.0005) [2023-12-27 08:16:06,100][20381] Updated weights for policy 1, policy_version 379605 (0.0007) [2023-12-27 08:16:06,167][20381] Updated weights for policy 1, policy_version 379615 (0.0007) [2023-12-27 08:16:06,178][20347] Updated weights for policy 0, policy_version 379748 (0.0009) [2023-12-27 08:16:06,226][20381] Updated weights for policy 1, policy_version 379625 (0.0006) [2023-12-27 08:16:06,238][20347] Updated weights for policy 0, policy_version 379758 (0.0010) [2023-12-27 08:16:06,296][20347] Updated weights for policy 0, policy_version 379768 (0.0009) [2023-12-27 08:16:06,834][20381] Updated weights for policy 1, policy_version 379635 (0.0008) [2023-12-27 08:16:06,895][20381] Updated weights for policy 1, policy_version 379645 (0.0008) [2023-12-27 08:16:06,952][20381] Updated weights for policy 1, policy_version 379655 (0.0008) [2023-12-27 08:16:06,959][20347] Updated weights for policy 0, policy_version 379778 (0.0007) [2023-12-27 08:16:07,014][20347] Updated weights for policy 0, policy_version 379788 (0.0006) [2023-12-27 08:16:07,061][20347] Updated weights for policy 0, policy_version 379798 (0.0006) [2023-12-27 08:16:07,110][20347] Updated weights for policy 0, policy_version 379808 (0.0005) [2023-12-27 08:16:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.2, 300 sec: 20882.7). Total num frames: 194461696. Throughput: 0: 10348.3, 1: 10453.4. Samples: 194427372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:07,363][19170] Avg episode reward: [(0, '193.110'), (1, '1000.000')] [2023-12-27 08:16:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000379664_97214464.pth... [2023-12-27 08:16:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000379808_97247232.pth... [2023-12-27 08:16:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000378448_96903168.pth [2023-12-27 08:16:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000378592_96935936.pth [2023-12-27 08:16:07,622][20381] Updated weights for policy 1, policy_version 379665 (0.0008) [2023-12-27 08:16:07,679][20381] Updated weights for policy 1, policy_version 379675 (0.0006) [2023-12-27 08:16:07,735][20381] Updated weights for policy 1, policy_version 379685 (0.0006) [2023-12-27 08:16:07,748][20347] Updated weights for policy 0, policy_version 379818 (0.0007) [2023-12-27 08:16:07,791][20381] Updated weights for policy 1, policy_version 379695 (0.0007) [2023-12-27 08:16:07,798][20347] Updated weights for policy 0, policy_version 379828 (0.0005) [2023-12-27 08:16:07,860][20347] Updated weights for policy 0, policy_version 379838 (0.0007) [2023-12-27 08:16:08,409][20381] Updated weights for policy 1, policy_version 379705 (0.0007) [2023-12-27 08:16:08,451][20347] Updated weights for policy 0, policy_version 379848 (0.0009) [2023-12-27 08:16:08,464][20381] Updated weights for policy 1, policy_version 379715 (0.0007) [2023-12-27 08:16:08,510][20347] Updated weights for policy 0, policy_version 379858 (0.0005) [2023-12-27 08:16:08,524][20381] Updated weights for policy 1, policy_version 379725 (0.0008) [2023-12-27 08:16:08,569][20347] Updated weights for policy 0, policy_version 379868 (0.0005) [2023-12-27 08:16:09,155][20381] Updated weights for policy 1, policy_version 379735 (0.0007) [2023-12-27 08:16:09,225][20381] Updated weights for policy 1, policy_version 379745 (0.0008) [2023-12-27 08:16:09,258][20347] Updated weights for policy 0, policy_version 379878 (0.0007) [2023-12-27 08:16:09,286][20381] Updated weights for policy 1, policy_version 379755 (0.0007) [2023-12-27 08:16:09,318][20347] Updated weights for policy 0, policy_version 379888 (0.0006) [2023-12-27 08:16:09,390][20347] Updated weights for policy 0, policy_version 379898 (0.0007) [2023-12-27 08:16:10,023][20381] Updated weights for policy 1, policy_version 379765 (0.0009) [2023-12-27 08:16:10,054][20347] Updated weights for policy 0, policy_version 379908 (0.0007) [2023-12-27 08:16:10,076][20381] Updated weights for policy 1, policy_version 379775 (0.0008) [2023-12-27 08:16:10,115][20347] Updated weights for policy 0, policy_version 379918 (0.0006) [2023-12-27 08:16:10,129][20381] Updated weights for policy 1, policy_version 379785 (0.0010) [2023-12-27 08:16:10,177][20347] Updated weights for policy 0, policy_version 379928 (0.0006) [2023-12-27 08:16:10,719][20347] Updated weights for policy 0, policy_version 379938 (0.0006) [2023-12-27 08:16:10,778][20347] Updated weights for policy 0, policy_version 379948 (0.0007) [2023-12-27 08:16:10,798][20381] Updated weights for policy 1, policy_version 379795 (0.0010) [2023-12-27 08:16:10,837][20347] Updated weights for policy 0, policy_version 379958 (0.0006) [2023-12-27 08:16:10,847][20381] Updated weights for policy 1, policy_version 379805 (0.0011) [2023-12-27 08:16:10,896][20347] Updated weights for policy 0, policy_version 379968 (0.0008) [2023-12-27 08:16:10,901][20381] Updated weights for policy 1, policy_version 379815 (0.0011) [2023-12-27 08:16:11,603][20347] Updated weights for policy 0, policy_version 379978 (0.0008) [2023-12-27 08:16:11,644][20381] Updated weights for policy 1, policy_version 379825 (0.0010) [2023-12-27 08:16:11,671][20347] Updated weights for policy 0, policy_version 379988 (0.0007) [2023-12-27 08:16:11,709][20381] Updated weights for policy 1, policy_version 379835 (0.0008) [2023-12-27 08:16:11,736][20347] Updated weights for policy 0, policy_version 379998 (0.0008) [2023-12-27 08:16:11,774][20381] Updated weights for policy 1, policy_version 379845 (0.0009) [2023-12-27 08:16:11,834][20381] Updated weights for policy 1, policy_version 379855 (0.0005) [2023-12-27 08:16:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 194560000. Throughput: 0: 10417.9, 1: 10523.4. Samples: 194554516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:12,363][19170] Avg episode reward: [(0, '378.730'), (1, '1000.000')] [2023-12-27 08:16:12,440][20347] Updated weights for policy 0, policy_version 380008 (0.0005) [2023-12-27 08:16:12,496][20381] Updated weights for policy 1, policy_version 379865 (0.0008) [2023-12-27 08:16:12,509][20347] Updated weights for policy 0, policy_version 380018 (0.0006) [2023-12-27 08:16:12,555][20381] Updated weights for policy 1, policy_version 379875 (0.0008) [2023-12-27 08:16:12,577][20347] Updated weights for policy 0, policy_version 380028 (0.0006) [2023-12-27 08:16:12,617][20381] Updated weights for policy 1, policy_version 379885 (0.0008) [2023-12-27 08:16:13,252][20347] Updated weights for policy 0, policy_version 380038 (0.0007) [2023-12-27 08:16:13,272][20381] Updated weights for policy 1, policy_version 379895 (0.0007) [2023-12-27 08:16:13,311][20347] Updated weights for policy 0, policy_version 380048 (0.0007) [2023-12-27 08:16:13,324][20381] Updated weights for policy 1, policy_version 379905 (0.0006) [2023-12-27 08:16:13,367][20347] Updated weights for policy 0, policy_version 380058 (0.0006) [2023-12-27 08:16:13,369][20381] Updated weights for policy 1, policy_version 379915 (0.0005) [2023-12-27 08:16:14,046][20381] Updated weights for policy 1, policy_version 379925 (0.0006) [2023-12-27 08:16:14,066][20347] Updated weights for policy 0, policy_version 380068 (0.0008) [2023-12-27 08:16:14,091][20381] Updated weights for policy 1, policy_version 379935 (0.0008) [2023-12-27 08:16:14,124][20347] Updated weights for policy 0, policy_version 380078 (0.0007) [2023-12-27 08:16:14,148][20381] Updated weights for policy 1, policy_version 379945 (0.0006) [2023-12-27 08:16:14,175][20347] Updated weights for policy 0, policy_version 380088 (0.0006) [2023-12-27 08:16:14,846][20381] Updated weights for policy 1, policy_version 379955 (0.0006) [2023-12-27 08:16:14,884][20347] Updated weights for policy 0, policy_version 380098 (0.0007) [2023-12-27 08:16:14,913][20381] Updated weights for policy 1, policy_version 379965 (0.0007) [2023-12-27 08:16:14,938][20347] Updated weights for policy 0, policy_version 380108 (0.0006) [2023-12-27 08:16:14,969][20381] Updated weights for policy 1, policy_version 379975 (0.0006) [2023-12-27 08:16:14,997][20347] Updated weights for policy 0, policy_version 380118 (0.0009) [2023-12-27 08:16:15,050][20347] Updated weights for policy 0, policy_version 380128 (0.0008) [2023-12-27 08:16:15,646][20381] Updated weights for policy 1, policy_version 379985 (0.0006) [2023-12-27 08:16:15,721][20381] Updated weights for policy 1, policy_version 379995 (0.0007) [2023-12-27 08:16:15,756][20347] Updated weights for policy 0, policy_version 380138 (0.0006) [2023-12-27 08:16:15,775][20381] Updated weights for policy 1, policy_version 380005 (0.0005) [2023-12-27 08:16:15,808][20347] Updated weights for policy 0, policy_version 380148 (0.0007) [2023-12-27 08:16:15,835][20381] Updated weights for policy 1, policy_version 380015 (0.0006) [2023-12-27 08:16:15,864][20347] Updated weights for policy 0, policy_version 380158 (0.0007) [2023-12-27 08:16:16,500][20381] Updated weights for policy 1, policy_version 380025 (0.0007) [2023-12-27 08:16:16,545][20381] Updated weights for policy 1, policy_version 380035 (0.0007) [2023-12-27 08:16:16,565][20347] Updated weights for policy 0, policy_version 380168 (0.0007) [2023-12-27 08:16:16,593][20381] Updated weights for policy 1, policy_version 380045 (0.0008) [2023-12-27 08:16:16,628][20347] Updated weights for policy 0, policy_version 380178 (0.0008) [2023-12-27 08:16:16,691][20347] Updated weights for policy 0, policy_version 380188 (0.0008) [2023-12-27 08:16:17,263][20347] Updated weights for policy 0, policy_version 380198 (0.0006) [2023-12-27 08:16:17,319][20347] Updated weights for policy 0, policy_version 380208 (0.0007) [2023-12-27 08:16:17,350][20381] Updated weights for policy 1, policy_version 380055 (0.0008) [2023-12-27 08:16:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 194658304. Throughput: 0: 10353.9, 1: 10418.0. Samples: 194674508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:17,363][19170] Avg episode reward: [(0, '445.650'), (1, '1000.000')] [2023-12-27 08:16:17,371][20347] Updated weights for policy 0, policy_version 380218 (0.0006) [2023-12-27 08:16:17,406][20381] Updated weights for policy 1, policy_version 380065 (0.0005) [2023-12-27 08:16:17,466][20381] Updated weights for policy 1, policy_version 380075 (0.0008) [2023-12-27 08:16:18,018][20347] Updated weights for policy 0, policy_version 380228 (0.0006) [2023-12-27 08:16:18,077][20347] Updated weights for policy 0, policy_version 380238 (0.0005) [2023-12-27 08:16:18,139][20347] Updated weights for policy 0, policy_version 380248 (0.0010) [2023-12-27 08:16:18,176][20381] Updated weights for policy 1, policy_version 380085 (0.0008) [2023-12-27 08:16:18,239][20381] Updated weights for policy 1, policy_version 380095 (0.0007) [2023-12-27 08:16:18,303][20381] Updated weights for policy 1, policy_version 380105 (0.0007) [2023-12-27 08:16:18,734][20347] Updated weights for policy 0, policy_version 380258 (0.0010) [2023-12-27 08:16:18,792][20347] Updated weights for policy 0, policy_version 380268 (0.0009) [2023-12-27 08:16:18,843][20347] Updated weights for policy 0, policy_version 380278 (0.0010) [2023-12-27 08:16:18,901][20347] Updated weights for policy 0, policy_version 380288 (0.0010) [2023-12-27 08:16:19,017][20381] Updated weights for policy 1, policy_version 380115 (0.0009) [2023-12-27 08:16:19,075][20381] Updated weights for policy 1, policy_version 380125 (0.0008) [2023-12-27 08:16:19,138][20381] Updated weights for policy 1, policy_version 380135 (0.0007) [2023-12-27 08:16:19,527][20347] Updated weights for policy 0, policy_version 380298 (0.0008) [2023-12-27 08:16:19,588][20347] Updated weights for policy 0, policy_version 380308 (0.0007) [2023-12-27 08:16:19,648][20347] Updated weights for policy 0, policy_version 380318 (0.0007) [2023-12-27 08:16:19,921][20381] Updated weights for policy 1, policy_version 380145 (0.0008) [2023-12-27 08:16:19,984][20381] Updated weights for policy 1, policy_version 380155 (0.0008) [2023-12-27 08:16:20,051][20381] Updated weights for policy 1, policy_version 380165 (0.0008) [2023-12-27 08:16:20,122][20381] Updated weights for policy 1, policy_version 380175 (0.0008) [2023-12-27 08:16:20,371][20347] Updated weights for policy 0, policy_version 380328 (0.0008) [2023-12-27 08:16:20,434][20347] Updated weights for policy 0, policy_version 380338 (0.0008) [2023-12-27 08:16:20,492][20347] Updated weights for policy 0, policy_version 380348 (0.0009) [2023-12-27 08:16:20,774][20381] Updated weights for policy 1, policy_version 380185 (0.0008) [2023-12-27 08:16:20,839][20381] Updated weights for policy 1, policy_version 380195 (0.0008) [2023-12-27 08:16:20,904][20381] Updated weights for policy 1, policy_version 380205 (0.0008) [2023-12-27 08:16:21,168][20347] Updated weights for policy 0, policy_version 380358 (0.0009) [2023-12-27 08:16:21,230][20347] Updated weights for policy 0, policy_version 380368 (0.0007) [2023-12-27 08:16:21,297][20347] Updated weights for policy 0, policy_version 380378 (0.0008) [2023-12-27 08:16:21,620][20381] Updated weights for policy 1, policy_version 380215 (0.0009) [2023-12-27 08:16:21,686][20381] Updated weights for policy 1, policy_version 380225 (0.0008) [2023-12-27 08:16:21,754][20381] Updated weights for policy 1, policy_version 380235 (0.0008) [2023-12-27 08:16:22,041][20347] Updated weights for policy 0, policy_version 380388 (0.0008) [2023-12-27 08:16:22,094][20347] Updated weights for policy 0, policy_version 380398 (0.0008) [2023-12-27 08:16:22,141][20347] Updated weights for policy 0, policy_version 380408 (0.0008) [2023-12-27 08:16:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 194764800. Throughput: 0: 10379.0, 1: 10334.4. Samples: 194735708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:22,363][19170] Avg episode reward: [(0, '546.540'), (1, '1000.000')] [2023-12-27 08:16:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000380240_97361920.pth... [2023-12-27 08:16:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000380416_97402880.pth... [2023-12-27 08:16:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000379056_97058816.pth [2023-12-27 08:16:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000379200_97091584.pth [2023-12-27 08:16:22,453][20381] Updated weights for policy 1, policy_version 380245 (0.0007) [2023-12-27 08:16:22,513][20381] Updated weights for policy 1, policy_version 380255 (0.0008) [2023-12-27 08:16:22,573][20381] Updated weights for policy 1, policy_version 380265 (0.0008) [2023-12-27 08:16:22,890][20347] Updated weights for policy 0, policy_version 380418 (0.0008) [2023-12-27 08:16:22,951][20347] Updated weights for policy 0, policy_version 380428 (0.0007) [2023-12-27 08:16:23,010][20347] Updated weights for policy 0, policy_version 380438 (0.0008) [2023-12-27 08:16:23,069][20347] Updated weights for policy 0, policy_version 380448 (0.0008) [2023-12-27 08:16:23,148][20381] Updated weights for policy 1, policy_version 380275 (0.0007) [2023-12-27 08:16:23,205][20381] Updated weights for policy 1, policy_version 380285 (0.0005) [2023-12-27 08:16:23,259][20381] Updated weights for policy 1, policy_version 380295 (0.0006) [2023-12-27 08:16:23,778][20347] Updated weights for policy 0, policy_version 380458 (0.0008) [2023-12-27 08:16:23,832][20347] Updated weights for policy 0, policy_version 380468 (0.0008) [2023-12-27 08:16:23,866][20381] Updated weights for policy 1, policy_version 380305 (0.0006) [2023-12-27 08:16:23,886][20347] Updated weights for policy 0, policy_version 380478 (0.0007) [2023-12-27 08:16:23,922][20381] Updated weights for policy 1, policy_version 380315 (0.0005) [2023-12-27 08:16:23,972][20381] Updated weights for policy 1, policy_version 380325 (0.0007) [2023-12-27 08:16:24,025][20381] Updated weights for policy 1, policy_version 380335 (0.0008) [2023-12-27 08:16:24,539][20347] Updated weights for policy 0, policy_version 380488 (0.0007) [2023-12-27 08:16:24,582][20347] Updated weights for policy 0, policy_version 380498 (0.0008) [2023-12-27 08:16:24,632][20347] Updated weights for policy 0, policy_version 380508 (0.0007) [2023-12-27 08:16:24,648][20381] Updated weights for policy 1, policy_version 380345 (0.0006) [2023-12-27 08:16:24,694][20381] Updated weights for policy 1, policy_version 380355 (0.0005) [2023-12-27 08:16:24,752][20381] Updated weights for policy 1, policy_version 380365 (0.0005) [2023-12-27 08:16:25,292][20347] Updated weights for policy 0, policy_version 380518 (0.0006) [2023-12-27 08:16:25,319][20381] Updated weights for policy 1, policy_version 380375 (0.0005) [2023-12-27 08:16:25,351][20347] Updated weights for policy 0, policy_version 380528 (0.0007) [2023-12-27 08:16:25,372][20381] Updated weights for policy 1, policy_version 380385 (0.0005) [2023-12-27 08:16:25,409][20347] Updated weights for policy 0, policy_version 380538 (0.0008) [2023-12-27 08:16:25,428][20381] Updated weights for policy 1, policy_version 380395 (0.0005) [2023-12-27 08:16:25,976][20381] Updated weights for policy 1, policy_version 380405 (0.0005) [2023-12-27 08:16:26,031][20381] Updated weights for policy 1, policy_version 380415 (0.0006) [2023-12-27 08:16:26,084][20381] Updated weights for policy 1, policy_version 380425 (0.0006) [2023-12-27 08:16:26,120][20347] Updated weights for policy 0, policy_version 380548 (0.0008) [2023-12-27 08:16:26,171][20347] Updated weights for policy 0, policy_version 380558 (0.0007) [2023-12-27 08:16:26,222][20347] Updated weights for policy 0, policy_version 380568 (0.0007) [2023-12-27 08:16:26,696][20381] Updated weights for policy 1, policy_version 380435 (0.0006) [2023-12-27 08:16:26,743][20381] Updated weights for policy 1, policy_version 380445 (0.0007) [2023-12-27 08:16:26,784][20381] Updated weights for policy 1, policy_version 380455 (0.0007) [2023-12-27 08:16:26,915][20347] Updated weights for policy 0, policy_version 380578 (0.0006) [2023-12-27 08:16:26,962][20347] Updated weights for policy 0, policy_version 380588 (0.0007) [2023-12-27 08:16:27,007][20347] Updated weights for policy 0, policy_version 380598 (0.0007) [2023-12-27 08:16:27,052][20347] Updated weights for policy 0, policy_version 380608 (0.0008) [2023-12-27 08:16:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 194871296. Throughput: 0: 10372.1, 1: 10405.4. Samples: 194861404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:27,363][19170] Avg episode reward: [(0, '637.220'), (1, '1000.000')] [2023-12-27 08:16:27,406][20381] Updated weights for policy 1, policy_version 380465 (0.0007) [2023-12-27 08:16:27,460][20381] Updated weights for policy 1, policy_version 380475 (0.0008) [2023-12-27 08:16:27,517][20381] Updated weights for policy 1, policy_version 380485 (0.0007) [2023-12-27 08:16:27,573][20381] Updated weights for policy 1, policy_version 380495 (0.0006) [2023-12-27 08:16:27,780][20347] Updated weights for policy 0, policy_version 380618 (0.0007) [2023-12-27 08:16:27,838][20347] Updated weights for policy 0, policy_version 380628 (0.0008) [2023-12-27 08:16:27,893][20347] Updated weights for policy 0, policy_version 380638 (0.0007) [2023-12-27 08:16:28,198][20381] Updated weights for policy 1, policy_version 380505 (0.0008) [2023-12-27 08:16:28,242][20381] Updated weights for policy 1, policy_version 380515 (0.0008) [2023-12-27 08:16:28,287][20381] Updated weights for policy 1, policy_version 380525 (0.0005) [2023-12-27 08:16:28,559][20347] Updated weights for policy 0, policy_version 380648 (0.0008) [2023-12-27 08:16:28,614][20347] Updated weights for policy 0, policy_version 380658 (0.0007) [2023-12-27 08:16:28,670][20347] Updated weights for policy 0, policy_version 380668 (0.0008) [2023-12-27 08:16:28,934][20381] Updated weights for policy 1, policy_version 380535 (0.0007) [2023-12-27 08:16:28,982][20381] Updated weights for policy 1, policy_version 380545 (0.0007) [2023-12-27 08:16:29,037][20381] Updated weights for policy 1, policy_version 380555 (0.0008) [2023-12-27 08:16:29,352][20347] Updated weights for policy 0, policy_version 380678 (0.0008) [2023-12-27 08:16:29,408][20347] Updated weights for policy 0, policy_version 380688 (0.0007) [2023-12-27 08:16:29,463][20347] Updated weights for policy 0, policy_version 380698 (0.0007) [2023-12-27 08:16:29,756][20381] Updated weights for policy 1, policy_version 380565 (0.0008) [2023-12-27 08:16:29,812][20381] Updated weights for policy 1, policy_version 380575 (0.0009) [2023-12-27 08:16:29,873][20381] Updated weights for policy 1, policy_version 380585 (0.0009) [2023-12-27 08:16:30,127][20347] Updated weights for policy 0, policy_version 380708 (0.0007) [2023-12-27 08:16:30,183][20347] Updated weights for policy 0, policy_version 380718 (0.0007) [2023-12-27 08:16:30,236][20347] Updated weights for policy 0, policy_version 380728 (0.0008) [2023-12-27 08:16:30,568][20381] Updated weights for policy 1, policy_version 380595 (0.0007) [2023-12-27 08:16:30,622][20381] Updated weights for policy 1, policy_version 380605 (0.0008) [2023-12-27 08:16:30,676][20381] Updated weights for policy 1, policy_version 380615 (0.0007) [2023-12-27 08:16:30,901][20347] Updated weights for policy 0, policy_version 380738 (0.0007) [2023-12-27 08:16:30,955][20347] Updated weights for policy 0, policy_version 380748 (0.0008) [2023-12-27 08:16:31,010][20347] Updated weights for policy 0, policy_version 380758 (0.0008) [2023-12-27 08:16:31,070][20347] Updated weights for policy 0, policy_version 380768 (0.0007) [2023-12-27 08:16:31,377][20381] Updated weights for policy 1, policy_version 380625 (0.0007) [2023-12-27 08:16:31,436][20381] Updated weights for policy 1, policy_version 380635 (0.0008) [2023-12-27 08:16:31,496][20381] Updated weights for policy 1, policy_version 380645 (0.0008) [2023-12-27 08:16:31,561][20381] Updated weights for policy 1, policy_version 380655 (0.0007) [2023-12-27 08:16:31,762][20347] Updated weights for policy 0, policy_version 380778 (0.0006) [2023-12-27 08:16:31,812][20347] Updated weights for policy 0, policy_version 380788 (0.0005) [2023-12-27 08:16:31,869][20347] Updated weights for policy 0, policy_version 380798 (0.0007) [2023-12-27 08:16:32,190][20381] Updated weights for policy 1, policy_version 380665 (0.0008) [2023-12-27 08:16:32,247][20381] Updated weights for policy 1, policy_version 380675 (0.0007) [2023-12-27 08:16:32,308][20381] Updated weights for policy 1, policy_version 380685 (0.0008) [2023-12-27 08:16:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 194977792. Throughput: 0: 10352.0, 1: 10346.8. Samples: 194985916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:32,363][19170] Avg episode reward: [(0, '662.980'), (1, '1000.000')] [2023-12-27 08:16:32,472][20347] Updated weights for policy 0, policy_version 380808 (0.0008) [2023-12-27 08:16:32,527][20347] Updated weights for policy 0, policy_version 380818 (0.0006) [2023-12-27 08:16:32,574][20347] Updated weights for policy 0, policy_version 380828 (0.0005) [2023-12-27 08:16:33,011][20381] Updated weights for policy 1, policy_version 380695 (0.0005) [2023-12-27 08:16:33,065][20381] Updated weights for policy 1, policy_version 380705 (0.0007) [2023-12-27 08:16:33,112][20381] Updated weights for policy 1, policy_version 380715 (0.0008) [2023-12-27 08:16:33,174][20347] Updated weights for policy 0, policy_version 380838 (0.0006) [2023-12-27 08:16:33,230][20347] Updated weights for policy 0, policy_version 380848 (0.0005) [2023-12-27 08:16:33,283][20347] Updated weights for policy 0, policy_version 380858 (0.0005) [2023-12-27 08:16:33,721][20381] Updated weights for policy 1, policy_version 380725 (0.0006) [2023-12-27 08:16:33,775][20381] Updated weights for policy 1, policy_version 380735 (0.0007) [2023-12-27 08:16:33,827][20381] Updated weights for policy 1, policy_version 380745 (0.0008) [2023-12-27 08:16:33,908][20347] Updated weights for policy 0, policy_version 380868 (0.0006) [2023-12-27 08:16:33,964][20347] Updated weights for policy 0, policy_version 380878 (0.0007) [2023-12-27 08:16:34,012][20347] Updated weights for policy 0, policy_version 380888 (0.0007) [2023-12-27 08:16:34,473][20381] Updated weights for policy 1, policy_version 380755 (0.0008) [2023-12-27 08:16:34,529][20381] Updated weights for policy 1, policy_version 380765 (0.0008) [2023-12-27 08:16:34,583][20381] Updated weights for policy 1, policy_version 380775 (0.0009) [2023-12-27 08:16:34,781][20347] Updated weights for policy 0, policy_version 380898 (0.0007) [2023-12-27 08:16:34,833][20347] Updated weights for policy 0, policy_version 380908 (0.0007) [2023-12-27 08:16:34,877][20347] Updated weights for policy 0, policy_version 380918 (0.0008) [2023-12-27 08:16:34,925][20347] Updated weights for policy 0, policy_version 380928 (0.0007) [2023-12-27 08:16:35,230][20381] Updated weights for policy 1, policy_version 380785 (0.0008) [2023-12-27 08:16:35,295][20381] Updated weights for policy 1, policy_version 380795 (0.0005) [2023-12-27 08:16:35,341][20381] Updated weights for policy 1, policy_version 380805 (0.0005) [2023-12-27 08:16:35,392][20381] Updated weights for policy 1, policy_version 380815 (0.0005) [2023-12-27 08:16:35,599][20347] Updated weights for policy 0, policy_version 380938 (0.0005) [2023-12-27 08:16:35,648][20347] Updated weights for policy 0, policy_version 380948 (0.0005) [2023-12-27 08:16:35,704][20347] Updated weights for policy 0, policy_version 380958 (0.0005) [2023-12-27 08:16:35,963][20381] Updated weights for policy 1, policy_version 380825 (0.0008) [2023-12-27 08:16:36,012][20381] Updated weights for policy 1, policy_version 380835 (0.0007) [2023-12-27 08:16:36,069][20381] Updated weights for policy 1, policy_version 380845 (0.0008) [2023-12-27 08:16:36,323][20347] Updated weights for policy 0, policy_version 380968 (0.0005) [2023-12-27 08:16:36,388][20347] Updated weights for policy 0, policy_version 380978 (0.0006) [2023-12-27 08:16:36,449][20347] Updated weights for policy 0, policy_version 380988 (0.0010) [2023-12-27 08:16:36,811][20381] Updated weights for policy 1, policy_version 380855 (0.0008) [2023-12-27 08:16:36,867][20381] Updated weights for policy 1, policy_version 380865 (0.0008) [2023-12-27 08:16:36,924][20381] Updated weights for policy 1, policy_version 380875 (0.0008) [2023-12-27 08:16:37,078][20347] Updated weights for policy 0, policy_version 380998 (0.0010) [2023-12-27 08:16:37,133][20347] Updated weights for policy 0, policy_version 381008 (0.0010) [2023-12-27 08:16:37,195][20347] Updated weights for policy 0, policy_version 381018 (0.0011) [2023-12-27 08:16:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 195084288. Throughput: 0: 10354.0, 1: 10352.9. Samples: 195049608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:37,364][19170] Avg episode reward: [(0, '673.550'), (1, '1000.000')] [2023-12-27 08:16:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000380880_97525760.pth... [2023-12-27 08:16:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000381024_97558528.pth... [2023-12-27 08:16:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000379808_97247232.pth [2023-12-27 08:16:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000379664_97214464.pth [2023-12-27 08:16:37,637][20381] Updated weights for policy 1, policy_version 380885 (0.0009) [2023-12-27 08:16:37,699][20381] Updated weights for policy 1, policy_version 380895 (0.0011) [2023-12-27 08:16:37,755][20381] Updated weights for policy 1, policy_version 380905 (0.0011) [2023-12-27 08:16:37,881][20347] Updated weights for policy 0, policy_version 381028 (0.0010) [2023-12-27 08:16:37,932][20347] Updated weights for policy 0, policy_version 381038 (0.0008) [2023-12-27 08:16:37,985][20347] Updated weights for policy 0, policy_version 381048 (0.0010) [2023-12-27 08:16:38,432][20381] Updated weights for policy 1, policy_version 380915 (0.0009) [2023-12-27 08:16:38,484][20381] Updated weights for policy 1, policy_version 380925 (0.0010) [2023-12-27 08:16:38,531][20381] Updated weights for policy 1, policy_version 380935 (0.0010) [2023-12-27 08:16:38,657][20347] Updated weights for policy 0, policy_version 381058 (0.0010) [2023-12-27 08:16:38,709][20347] Updated weights for policy 0, policy_version 381068 (0.0010) [2023-12-27 08:16:38,763][20347] Updated weights for policy 0, policy_version 381078 (0.0010) [2023-12-27 08:16:38,820][20347] Updated weights for policy 0, policy_version 381088 (0.0010) [2023-12-27 08:16:39,198][20381] Updated weights for policy 1, policy_version 380945 (0.0010) [2023-12-27 08:16:39,266][20381] Updated weights for policy 1, policy_version 380955 (0.0008) [2023-12-27 08:16:39,321][20381] Updated weights for policy 1, policy_version 380965 (0.0007) [2023-12-27 08:16:39,387][20381] Updated weights for policy 1, policy_version 380975 (0.0008) [2023-12-27 08:16:39,486][20347] Updated weights for policy 0, policy_version 381098 (0.0008) [2023-12-27 08:16:39,544][20347] Updated weights for policy 0, policy_version 381108 (0.0007) [2023-12-27 08:16:39,602][20347] Updated weights for policy 0, policy_version 381118 (0.0007) [2023-12-27 08:16:40,128][20381] Updated weights for policy 1, policy_version 380985 (0.0007) [2023-12-27 08:16:40,185][20381] Updated weights for policy 1, policy_version 380995 (0.0007) [2023-12-27 08:16:40,242][20381] Updated weights for policy 1, policy_version 381005 (0.0006) [2023-12-27 08:16:40,244][20347] Updated weights for policy 0, policy_version 381128 (0.0010) [2023-12-27 08:16:40,299][20347] Updated weights for policy 0, policy_version 381138 (0.0006) [2023-12-27 08:16:40,361][20347] Updated weights for policy 0, policy_version 381148 (0.0007) [2023-12-27 08:16:40,978][20381] Updated weights for policy 1, policy_version 381015 (0.0008) [2023-12-27 08:16:41,004][20347] Updated weights for policy 0, policy_version 381158 (0.0009) [2023-12-27 08:16:41,025][20381] Updated weights for policy 1, policy_version 381025 (0.0007) [2023-12-27 08:16:41,060][20347] Updated weights for policy 0, policy_version 381168 (0.0007) [2023-12-27 08:16:41,083][20381] Updated weights for policy 1, policy_version 381035 (0.0008) [2023-12-27 08:16:41,126][20347] Updated weights for policy 0, policy_version 381178 (0.0008) [2023-12-27 08:16:41,833][20381] Updated weights for policy 1, policy_version 381045 (0.0006) [2023-12-27 08:16:41,835][20347] Updated weights for policy 0, policy_version 381188 (0.0008) [2023-12-27 08:16:41,894][20381] Updated weights for policy 1, policy_version 381055 (0.0006) [2023-12-27 08:16:41,897][20347] Updated weights for policy 0, policy_version 381198 (0.0006) [2023-12-27 08:16:41,952][20381] Updated weights for policy 1, policy_version 381065 (0.0006) [2023-12-27 08:16:41,952][20347] Updated weights for policy 0, policy_version 381208 (0.0006) [2023-12-27 08:16:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 195182592. Throughput: 0: 10380.1, 1: 10325.5. Samples: 195173080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:42,363][19170] Avg episode reward: [(0, '714.070'), (1, '1000.000')] [2023-12-27 08:16:42,588][20347] Updated weights for policy 0, policy_version 381218 (0.0008) [2023-12-27 08:16:42,625][20381] Updated weights for policy 1, policy_version 381075 (0.0009) [2023-12-27 08:16:42,643][20347] Updated weights for policy 0, policy_version 381228 (0.0006) [2023-12-27 08:16:42,683][20381] Updated weights for policy 1, policy_version 381085 (0.0007) [2023-12-27 08:16:42,700][20347] Updated weights for policy 0, policy_version 381238 (0.0005) [2023-12-27 08:16:42,736][20381] Updated weights for policy 1, policy_version 381095 (0.0008) [2023-12-27 08:16:42,750][20347] Updated weights for policy 0, policy_version 381248 (0.0005) [2023-12-27 08:16:43,379][20347] Updated weights for policy 0, policy_version 381258 (0.0009) [2023-12-27 08:16:43,427][20381] Updated weights for policy 1, policy_version 381105 (0.0008) [2023-12-27 08:16:43,431][20347] Updated weights for policy 0, policy_version 381268 (0.0007) [2023-12-27 08:16:43,477][20347] Updated weights for policy 0, policy_version 381278 (0.0007) [2023-12-27 08:16:43,479][20381] Updated weights for policy 1, policy_version 381115 (0.0006) [2023-12-27 08:16:43,529][20381] Updated weights for policy 1, policy_version 381125 (0.0006) [2023-12-27 08:16:43,581][20381] Updated weights for policy 1, policy_version 381135 (0.0007) [2023-12-27 08:16:44,158][20381] Updated weights for policy 1, policy_version 381145 (0.0006) [2023-12-27 08:16:44,186][20347] Updated weights for policy 0, policy_version 381288 (0.0006) [2023-12-27 08:16:44,206][20381] Updated weights for policy 1, policy_version 381155 (0.0006) [2023-12-27 08:16:44,248][20347] Updated weights for policy 0, policy_version 381298 (0.0005) [2023-12-27 08:16:44,254][20381] Updated weights for policy 1, policy_version 381165 (0.0007) [2023-12-27 08:16:44,300][20347] Updated weights for policy 0, policy_version 381308 (0.0005) [2023-12-27 08:16:44,840][20347] Updated weights for policy 0, policy_version 381318 (0.0007) [2023-12-27 08:16:44,896][20347] Updated weights for policy 0, policy_version 381328 (0.0007) [2023-12-27 08:16:44,921][20381] Updated weights for policy 1, policy_version 381175 (0.0006) [2023-12-27 08:16:44,956][20347] Updated weights for policy 0, policy_version 381338 (0.0007) [2023-12-27 08:16:44,979][20381] Updated weights for policy 1, policy_version 381185 (0.0007) [2023-12-27 08:16:45,036][20381] Updated weights for policy 1, policy_version 381195 (0.0008) [2023-12-27 08:16:45,608][20347] Updated weights for policy 0, policy_version 381348 (0.0006) [2023-12-27 08:16:45,656][20347] Updated weights for policy 0, policy_version 381358 (0.0008) [2023-12-27 08:16:45,683][20381] Updated weights for policy 1, policy_version 381205 (0.0006) [2023-12-27 08:16:45,714][20347] Updated weights for policy 0, policy_version 381368 (0.0008) [2023-12-27 08:16:45,739][20381] Updated weights for policy 1, policy_version 381215 (0.0005) [2023-12-27 08:16:45,793][20381] Updated weights for policy 1, policy_version 381225 (0.0006) [2023-12-27 08:16:46,401][20347] Updated weights for policy 0, policy_version 381378 (0.0007) [2023-12-27 08:16:46,405][20381] Updated weights for policy 1, policy_version 381235 (0.0007) [2023-12-27 08:16:46,451][20347] Updated weights for policy 0, policy_version 381388 (0.0005) [2023-12-27 08:16:46,457][20381] Updated weights for policy 1, policy_version 381245 (0.0010) [2023-12-27 08:16:46,499][20347] Updated weights for policy 0, policy_version 381398 (0.0005) [2023-12-27 08:16:46,515][20381] Updated weights for policy 1, policy_version 381255 (0.0010) [2023-12-27 08:16:46,547][20347] Updated weights for policy 0, policy_version 381408 (0.0006) [2023-12-27 08:16:47,121][20381] Updated weights for policy 1, policy_version 381265 (0.0010) [2023-12-27 08:16:47,178][20381] Updated weights for policy 1, policy_version 381275 (0.0005) [2023-12-27 08:16:47,238][20381] Updated weights for policy 1, policy_version 381285 (0.0007) [2023-12-27 08:16:47,239][20347] Updated weights for policy 0, policy_version 381418 (0.0008) [2023-12-27 08:16:47,288][20347] Updated weights for policy 0, policy_version 381428 (0.0006) [2023-12-27 08:16:47,297][20381] Updated weights for policy 1, policy_version 381295 (0.0010) [2023-12-27 08:16:47,341][20347] Updated weights for policy 0, policy_version 381438 (0.0006) [2023-12-27 08:16:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 195297280. Throughput: 0: 10356.1, 1: 10432.2. Samples: 195300596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:47,363][19170] Avg episode reward: [(0, '809.750'), (1, '1000.000')] [2023-12-27 08:16:47,851][20381] Updated weights for policy 1, policy_version 381305 (0.0006) [2023-12-27 08:16:47,913][20381] Updated weights for policy 1, policy_version 381315 (0.0005) [2023-12-27 08:16:47,969][20381] Updated weights for policy 1, policy_version 381325 (0.0008) [2023-12-27 08:16:47,997][20347] Updated weights for policy 0, policy_version 381448 (0.0006) [2023-12-27 08:16:48,049][20347] Updated weights for policy 0, policy_version 381458 (0.0006) [2023-12-27 08:16:48,104][20347] Updated weights for policy 0, policy_version 381468 (0.0006) [2023-12-27 08:16:48,641][20381] Updated weights for policy 1, policy_version 381335 (0.0011) [2023-12-27 08:16:48,696][20381] Updated weights for policy 1, policy_version 381345 (0.0007) [2023-12-27 08:16:48,703][20347] Updated weights for policy 0, policy_version 381478 (0.0006) [2023-12-27 08:16:48,755][20347] Updated weights for policy 0, policy_version 381488 (0.0007) [2023-12-27 08:16:48,756][20381] Updated weights for policy 1, policy_version 381355 (0.0011) [2023-12-27 08:16:48,808][20347] Updated weights for policy 0, policy_version 381498 (0.0007) [2023-12-27 08:16:49,435][20381] Updated weights for policy 1, policy_version 381365 (0.0011) [2023-12-27 08:16:49,467][20347] Updated weights for policy 0, policy_version 381508 (0.0007) [2023-12-27 08:16:49,494][20381] Updated weights for policy 1, policy_version 381375 (0.0011) [2023-12-27 08:16:49,526][20347] Updated weights for policy 0, policy_version 381518 (0.0005) [2023-12-27 08:16:49,548][20381] Updated weights for policy 1, policy_version 381385 (0.0009) [2023-12-27 08:16:49,585][20347] Updated weights for policy 0, policy_version 381528 (0.0007) [2023-12-27 08:16:50,212][20381] Updated weights for policy 1, policy_version 381395 (0.0007) [2023-12-27 08:16:50,272][20381] Updated weights for policy 1, policy_version 381405 (0.0011) [2023-12-27 08:16:50,278][20347] Updated weights for policy 0, policy_version 381538 (0.0007) [2023-12-27 08:16:50,329][20381] Updated weights for policy 1, policy_version 381415 (0.0011) [2023-12-27 08:16:50,335][20347] Updated weights for policy 0, policy_version 381548 (0.0005) [2023-12-27 08:16:50,395][20347] Updated weights for policy 0, policy_version 381558 (0.0007) [2023-12-27 08:16:50,452][20347] Updated weights for policy 0, policy_version 381568 (0.0007) [2023-12-27 08:16:51,034][20381] Updated weights for policy 1, policy_version 381425 (0.0011) [2023-12-27 08:16:51,100][20381] Updated weights for policy 1, policy_version 381435 (0.0010) [2023-12-27 08:16:51,158][20347] Updated weights for policy 0, policy_version 381578 (0.0007) [2023-12-27 08:16:51,164][20381] Updated weights for policy 1, policy_version 381445 (0.0011) [2023-12-27 08:16:51,217][20347] Updated weights for policy 0, policy_version 381588 (0.0006) [2023-12-27 08:16:51,220][20381] Updated weights for policy 1, policy_version 381455 (0.0011) [2023-12-27 08:16:51,280][20347] Updated weights for policy 0, policy_version 381598 (0.0007) [2023-12-27 08:16:51,914][20381] Updated weights for policy 1, policy_version 381465 (0.0006) [2023-12-27 08:16:51,962][20381] Updated weights for policy 1, policy_version 381475 (0.0010) [2023-12-27 08:16:52,008][20381] Updated weights for policy 1, policy_version 381485 (0.0011) [2023-12-27 08:16:52,042][20347] Updated weights for policy 0, policy_version 381608 (0.0008) [2023-12-27 08:16:52,100][20347] Updated weights for policy 0, policy_version 381618 (0.0008) [2023-12-27 08:16:52,149][20347] Updated weights for policy 0, policy_version 381628 (0.0008) [2023-12-27 08:16:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 195395584. Throughput: 0: 10405.7, 1: 10407.3. Samples: 195363956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:16:52,363][19170] Avg episode reward: [(0, '798.350'), (1, '990.480')] [2023-12-27 08:16:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000381488_97681408.pth... [2023-12-27 08:16:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000381632_97714176.pth... [2023-12-27 08:16:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000380240_97361920.pth [2023-12-27 08:16:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000380416_97402880.pth [2023-12-27 08:16:52,632][20381] Updated weights for policy 1, policy_version 381495 (0.0011) [2023-12-27 08:16:52,643][20292] KL-divergence is very high: 275.0363 [2023-12-27 08:16:52,688][20381] Updated weights for policy 1, policy_version 381505 (0.0011) [2023-12-27 08:16:52,741][20381] Updated weights for policy 1, policy_version 381515 (0.0011) [2023-12-27 08:16:52,871][20347] Updated weights for policy 0, policy_version 381638 (0.0008) [2023-12-27 08:16:52,922][20347] Updated weights for policy 0, policy_version 381648 (0.0007) [2023-12-27 08:16:52,975][20347] Updated weights for policy 0, policy_version 381658 (0.0007) [2023-12-27 08:16:53,370][20381] Updated weights for policy 1, policy_version 381525 (0.0011) [2023-12-27 08:16:53,425][20381] Updated weights for policy 1, policy_version 381535 (0.0010) [2023-12-27 08:16:53,477][20381] Updated weights for policy 1, policy_version 381545 (0.0010) [2023-12-27 08:16:53,602][20347] Updated weights for policy 0, policy_version 381668 (0.0008) [2023-12-27 08:16:53,651][20347] Updated weights for policy 0, policy_version 381678 (0.0008) [2023-12-27 08:16:53,698][20347] Updated weights for policy 0, policy_version 381688 (0.0006) [2023-12-27 08:16:54,186][20381] Updated weights for policy 1, policy_version 381555 (0.0010) [2023-12-27 08:16:54,229][20381] Updated weights for policy 1, policy_version 381565 (0.0010) [2023-12-27 08:16:54,274][20381] Updated weights for policy 1, policy_version 381575 (0.0010) [2023-12-27 08:16:54,377][20347] Updated weights for policy 0, policy_version 381698 (0.0006) [2023-12-27 08:16:54,424][20347] Updated weights for policy 0, policy_version 381708 (0.0007) [2023-12-27 08:16:54,469][20347] Updated weights for policy 0, policy_version 381718 (0.0007) [2023-12-27 08:16:54,512][20347] Updated weights for policy 0, policy_version 381728 (0.0007) [2023-12-27 08:16:54,975][20381] Updated weights for policy 1, policy_version 381585 (0.0007) [2023-12-27 08:16:55,026][20381] Updated weights for policy 1, policy_version 381595 (0.0005) [2023-12-27 08:16:55,081][20381] Updated weights for policy 1, policy_version 381605 (0.0009) [2023-12-27 08:16:55,133][20381] Updated weights for policy 1, policy_version 381615 (0.0005) [2023-12-27 08:16:55,248][20347] Updated weights for policy 0, policy_version 381738 (0.0007) [2023-12-27 08:16:55,301][20347] Updated weights for policy 0, policy_version 381748 (0.0007) [2023-12-27 08:16:55,354][20347] Updated weights for policy 0, policy_version 381758 (0.0007) [2023-12-27 08:16:55,692][20381] Updated weights for policy 1, policy_version 381625 (0.0009) [2023-12-27 08:16:55,743][20381] Updated weights for policy 1, policy_version 381635 (0.0005) [2023-12-27 08:16:55,794][20381] Updated weights for policy 1, policy_version 381645 (0.0009) [2023-12-27 08:16:56,089][20347] Updated weights for policy 0, policy_version 381768 (0.0006) [2023-12-27 08:16:56,152][20347] Updated weights for policy 0, policy_version 381778 (0.0005) [2023-12-27 08:16:56,210][20347] Updated weights for policy 0, policy_version 381788 (0.0005) [2023-12-27 08:16:56,469][20381] Updated weights for policy 1, policy_version 381655 (0.0010) [2023-12-27 08:16:56,517][20381] Updated weights for policy 1, policy_version 381665 (0.0010) [2023-12-27 08:16:56,561][20381] Updated weights for policy 1, policy_version 381675 (0.0010) [2023-12-27 08:16:56,779][20347] Updated weights for policy 0, policy_version 381798 (0.0007) [2023-12-27 08:16:56,839][20347] Updated weights for policy 0, policy_version 381808 (0.0008) [2023-12-27 08:16:56,889][20347] Updated weights for policy 0, policy_version 381818 (0.0007) [2023-12-27 08:16:57,238][20381] Updated weights for policy 1, policy_version 381685 (0.0010) [2023-12-27 08:16:57,289][20381] Updated weights for policy 1, policy_version 381695 (0.0010) [2023-12-27 08:16:57,343][20381] Updated weights for policy 1, policy_version 381705 (0.0010) [2023-12-27 08:16:57,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 195493888. Throughput: 0: 10309.5, 1: 10400.5. Samples: 195486468. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:16:57,363][19170] Avg episode reward: [(0, '769.410'), (1, '990.480')] [2023-12-27 08:16:57,491][20347] Updated weights for policy 0, policy_version 381828 (0.0006) [2023-12-27 08:16:57,548][20347] Updated weights for policy 0, policy_version 381838 (0.0005) [2023-12-27 08:16:57,598][20347] Updated weights for policy 0, policy_version 381848 (0.0008) [2023-12-27 08:16:58,047][20381] Updated weights for policy 1, policy_version 381715 (0.0010) [2023-12-27 08:16:58,096][20381] Updated weights for policy 1, policy_version 381725 (0.0006) [2023-12-27 08:16:58,144][20381] Updated weights for policy 1, policy_version 381735 (0.0005) [2023-12-27 08:16:58,186][20347] Updated weights for policy 0, policy_version 381858 (0.0008) [2023-12-27 08:16:58,238][20347] Updated weights for policy 0, policy_version 381868 (0.0008) [2023-12-27 08:16:58,293][20347] Updated weights for policy 0, policy_version 381878 (0.0007) [2023-12-27 08:16:58,355][20347] Updated weights for policy 0, policy_version 381888 (0.0008) [2023-12-27 08:16:59,002][20381] Updated weights for policy 1, policy_version 381745 (0.0008) [2023-12-27 08:16:59,062][20381] Updated weights for policy 1, policy_version 381755 (0.0009) [2023-12-27 08:16:59,122][20381] Updated weights for policy 1, policy_version 381765 (0.0009) [2023-12-27 08:16:59,140][20347] Updated weights for policy 0, policy_version 381898 (0.0008) [2023-12-27 08:16:59,179][20381] Updated weights for policy 1, policy_version 381775 (0.0005) [2023-12-27 08:16:59,204][20347] Updated weights for policy 0, policy_version 381908 (0.0007) [2023-12-27 08:16:59,268][20347] Updated weights for policy 0, policy_version 381918 (0.0007) [2023-12-27 08:16:59,934][20347] Updated weights for policy 0, policy_version 381928 (0.0009) [2023-12-27 08:16:59,950][20381] Updated weights for policy 1, policy_version 381785 (0.0006) [2023-12-27 08:16:59,968][20292] KL-divergence is very high: 349.7550 [2023-12-27 08:16:59,992][20347] Updated weights for policy 0, policy_version 381938 (0.0010) [2023-12-27 08:17:00,008][20381] Updated weights for policy 1, policy_version 381795 (0.0005) [2023-12-27 08:17:00,049][20347] Updated weights for policy 0, policy_version 381948 (0.0010) [2023-12-27 08:17:00,057][20381] Updated weights for policy 1, policy_version 381805 (0.0008) [2023-12-27 08:17:00,663][20381] Updated weights for policy 1, policy_version 381815 (0.0007) [2023-12-27 08:17:00,688][20347] Updated weights for policy 0, policy_version 381958 (0.0010) [2023-12-27 08:17:00,714][20381] Updated weights for policy 1, policy_version 381825 (0.0006) [2023-12-27 08:17:00,743][20347] Updated weights for policy 0, policy_version 381968 (0.0010) [2023-12-27 08:17:00,760][20381] Updated weights for policy 1, policy_version 381835 (0.0005) [2023-12-27 08:17:00,800][20347] Updated weights for policy 0, policy_version 381978 (0.0010) [2023-12-27 08:17:01,433][20347] Updated weights for policy 0, policy_version 381988 (0.0009) [2023-12-27 08:17:01,446][20381] Updated weights for policy 1, policy_version 381845 (0.0007) [2023-12-27 08:17:01,482][20347] Updated weights for policy 0, policy_version 381998 (0.0010) [2023-12-27 08:17:01,507][20381] Updated weights for policy 1, policy_version 381855 (0.0008) [2023-12-27 08:17:01,537][20347] Updated weights for policy 0, policy_version 382008 (0.0010) [2023-12-27 08:17:01,563][20381] Updated weights for policy 1, policy_version 381865 (0.0010) [2023-12-27 08:17:02,202][20347] Updated weights for policy 0, policy_version 382018 (0.0010) [2023-12-27 08:17:02,252][20381] Updated weights for policy 1, policy_version 381875 (0.0007) [2023-12-27 08:17:02,254][20347] Updated weights for policy 0, policy_version 382028 (0.0008) [2023-12-27 08:17:02,272][20291] KL-divergence is very high: 168.9918 [2023-12-27 08:17:02,310][20347] Updated weights for policy 0, policy_version 382038 (0.0010) [2023-12-27 08:17:02,311][20381] Updated weights for policy 1, policy_version 381885 (0.0007) [2023-12-27 08:17:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 195592192. Throughput: 0: 10393.7, 1: 10395.6. Samples: 195610028. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:02,363][19170] Avg episode reward: [(0, '826.600'), (1, '990.360')] [2023-12-27 08:17:02,370][20381] Updated weights for policy 1, policy_version 381895 (0.0007) [2023-12-27 08:17:02,372][20347] Updated weights for policy 0, policy_version 382048 (0.0009) [2023-12-27 08:17:02,976][20381] Updated weights for policy 1, policy_version 381905 (0.0007) [2023-12-27 08:17:03,036][20381] Updated weights for policy 1, policy_version 381915 (0.0006) [2023-12-27 08:17:03,044][20347] Updated weights for policy 0, policy_version 382058 (0.0011) [2023-12-27 08:17:03,091][20381] Updated weights for policy 1, policy_version 381925 (0.0006) [2023-12-27 08:17:03,099][20347] Updated weights for policy 0, policy_version 382068 (0.0010) [2023-12-27 08:17:03,153][20381] Updated weights for policy 1, policy_version 381935 (0.0005) [2023-12-27 08:17:03,154][20347] Updated weights for policy 0, policy_version 382078 (0.0010) [2023-12-27 08:17:03,810][20381] Updated weights for policy 1, policy_version 381945 (0.0006) [2023-12-27 08:17:03,815][20347] Updated weights for policy 0, policy_version 382088 (0.0008) [2023-12-27 08:17:03,872][20381] Updated weights for policy 1, policy_version 381955 (0.0010) [2023-12-27 08:17:03,874][20347] Updated weights for policy 0, policy_version 382098 (0.0011) [2023-12-27 08:17:03,927][20381] Updated weights for policy 1, policy_version 381965 (0.0010) [2023-12-27 08:17:03,930][20347] Updated weights for policy 0, policy_version 382108 (0.0011) [2023-12-27 08:17:04,609][20381] Updated weights for policy 1, policy_version 381975 (0.0008) [2023-12-27 08:17:04,615][20347] Updated weights for policy 0, policy_version 382118 (0.0011) [2023-12-27 08:17:04,669][20347] Updated weights for policy 0, policy_version 382128 (0.0008) [2023-12-27 08:17:04,669][20381] Updated weights for policy 1, policy_version 381985 (0.0006) [2023-12-27 08:17:04,720][20381] Updated weights for policy 1, policy_version 381995 (0.0008) [2023-12-27 08:17:04,733][20347] Updated weights for policy 0, policy_version 382138 (0.0006) [2023-12-27 08:17:05,324][20347] Updated weights for policy 0, policy_version 382148 (0.0006) [2023-12-27 08:17:05,375][20347] Updated weights for policy 0, policy_version 382158 (0.0007) [2023-12-27 08:17:05,412][20381] Updated weights for policy 1, policy_version 382005 (0.0007) [2023-12-27 08:17:05,439][20347] Updated weights for policy 0, policy_version 382168 (0.0006) [2023-12-27 08:17:05,464][20381] Updated weights for policy 1, policy_version 382015 (0.0005) [2023-12-27 08:17:05,516][20381] Updated weights for policy 1, policy_version 382025 (0.0008) [2023-12-27 08:17:06,086][20347] Updated weights for policy 0, policy_version 382178 (0.0007) [2023-12-27 08:17:06,151][20347] Updated weights for policy 0, policy_version 382188 (0.0010) [2023-12-27 08:17:06,205][20347] Updated weights for policy 0, policy_version 382198 (0.0011) [2023-12-27 08:17:06,205][20381] Updated weights for policy 1, policy_version 382035 (0.0009) [2023-12-27 08:17:06,262][20347] Updated weights for policy 0, policy_version 382208 (0.0011) [2023-12-27 08:17:06,263][20381] Updated weights for policy 1, policy_version 382045 (0.0009) [2023-12-27 08:17:06,318][20381] Updated weights for policy 1, policy_version 382055 (0.0007) [2023-12-27 08:17:06,984][20347] Updated weights for policy 0, policy_version 382218 (0.0007) [2023-12-27 08:17:07,020][20381] Updated weights for policy 1, policy_version 382065 (0.0008) [2023-12-27 08:17:07,037][20347] Updated weights for policy 0, policy_version 382228 (0.0006) [2023-12-27 08:17:07,078][20381] Updated weights for policy 1, policy_version 382075 (0.0007) [2023-12-27 08:17:07,091][20347] Updated weights for policy 0, policy_version 382238 (0.0008) [2023-12-27 08:17:07,136][20381] Updated weights for policy 1, policy_version 382085 (0.0006) [2023-12-27 08:17:07,194][20381] Updated weights for policy 1, policy_version 382095 (0.0009) [2023-12-27 08:17:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 195706880. Throughput: 0: 10370.2, 1: 10448.6. Samples: 195672552. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:07,363][19170] Avg episode reward: [(0, '798.790'), (1, '961.270')] [2023-12-27 08:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000382096_97837056.pth... [2023-12-27 08:17:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000382240_97869824.pth... [2023-12-27 08:17:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000380880_97525760.pth [2023-12-27 08:17:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000381024_97558528.pth [2023-12-27 08:17:07,706][20347] Updated weights for policy 0, policy_version 382248 (0.0006) [2023-12-27 08:17:07,754][20347] Updated weights for policy 0, policy_version 382258 (0.0007) [2023-12-27 08:17:07,812][20347] Updated weights for policy 0, policy_version 382268 (0.0007) [2023-12-27 08:17:07,903][20381] Updated weights for policy 1, policy_version 382105 (0.0007) [2023-12-27 08:17:07,956][20381] Updated weights for policy 1, policy_version 382115 (0.0008) [2023-12-27 08:17:08,012][20381] Updated weights for policy 1, policy_version 382125 (0.0008) [2023-12-27 08:17:08,473][20347] Updated weights for policy 0, policy_version 382278 (0.0008) [2023-12-27 08:17:08,530][20347] Updated weights for policy 0, policy_version 382288 (0.0006) [2023-12-27 08:17:08,592][20347] Updated weights for policy 0, policy_version 382298 (0.0007) [2023-12-27 08:17:08,735][20381] Updated weights for policy 1, policy_version 382135 (0.0009) [2023-12-27 08:17:08,793][20381] Updated weights for policy 1, policy_version 382145 (0.0011) [2023-12-27 08:17:08,849][20381] Updated weights for policy 1, policy_version 382155 (0.0009) [2023-12-27 08:17:09,320][20347] Updated weights for policy 0, policy_version 382308 (0.0008) [2023-12-27 08:17:09,383][20347] Updated weights for policy 0, policy_version 382318 (0.0009) [2023-12-27 08:17:09,443][20381] Updated weights for policy 1, policy_version 382165 (0.0005) [2023-12-27 08:17:09,453][20347] Updated weights for policy 0, policy_version 382328 (0.0007) [2023-12-27 08:17:09,500][20381] Updated weights for policy 1, policy_version 382175 (0.0006) [2023-12-27 08:17:09,564][20381] Updated weights for policy 1, policy_version 382185 (0.0007) [2023-12-27 08:17:10,175][20347] Updated weights for policy 0, policy_version 382338 (0.0008) [2023-12-27 08:17:10,235][20347] Updated weights for policy 0, policy_version 382348 (0.0008) [2023-12-27 08:17:10,244][20381] Updated weights for policy 1, policy_version 382195 (0.0007) [2023-12-27 08:17:10,292][20347] Updated weights for policy 0, policy_version 382358 (0.0006) [2023-12-27 08:17:10,298][20381] Updated weights for policy 1, policy_version 382205 (0.0006) [2023-12-27 08:17:10,349][20347] Updated weights for policy 0, policy_version 382368 (0.0006) [2023-12-27 08:17:10,355][20381] Updated weights for policy 1, policy_version 382215 (0.0006) [2023-12-27 08:17:11,014][20381] Updated weights for policy 1, policy_version 382225 (0.0008) [2023-12-27 08:17:11,072][20381] Updated weights for policy 1, policy_version 382235 (0.0008) [2023-12-27 08:17:11,089][20347] Updated weights for policy 0, policy_version 382378 (0.0007) [2023-12-27 08:17:11,139][20381] Updated weights for policy 1, policy_version 382245 (0.0010) [2023-12-27 08:17:11,153][20347] Updated weights for policy 0, policy_version 382388 (0.0007) [2023-12-27 08:17:11,202][20381] Updated weights for policy 1, policy_version 382255 (0.0009) [2023-12-27 08:17:11,204][20347] Updated weights for policy 0, policy_version 382398 (0.0007) [2023-12-27 08:17:11,930][20381] Updated weights for policy 1, policy_version 382265 (0.0008) [2023-12-27 08:17:11,947][20347] Updated weights for policy 0, policy_version 382408 (0.0008) [2023-12-27 08:17:11,994][20381] Updated weights for policy 1, policy_version 382275 (0.0007) [2023-12-27 08:17:12,007][20347] Updated weights for policy 0, policy_version 382418 (0.0007) [2023-12-27 08:17:12,048][20381] Updated weights for policy 1, policy_version 382285 (0.0006) [2023-12-27 08:17:12,067][20347] Updated weights for policy 0, policy_version 382428 (0.0007) [2023-12-27 08:17:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 195805184. Throughput: 0: 10391.7, 1: 10338.0. Samples: 195794240. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:12,363][19170] Avg episode reward: [(0, '779.110'), (1, '942.160')] [2023-12-27 08:17:12,750][20381] Updated weights for policy 1, policy_version 382295 (0.0007) [2023-12-27 08:17:12,757][20347] Updated weights for policy 0, policy_version 382438 (0.0007) [2023-12-27 08:17:12,814][20381] Updated weights for policy 1, policy_version 382305 (0.0008) [2023-12-27 08:17:12,815][20347] Updated weights for policy 0, policy_version 382448 (0.0006) [2023-12-27 08:17:12,864][20347] Updated weights for policy 0, policy_version 382458 (0.0005) [2023-12-27 08:17:12,866][20381] Updated weights for policy 1, policy_version 382315 (0.0006) [2023-12-27 08:17:13,561][20347] Updated weights for policy 0, policy_version 382468 (0.0006) [2023-12-27 08:17:13,574][20381] Updated weights for policy 1, policy_version 382325 (0.0006) [2023-12-27 08:17:13,612][20347] Updated weights for policy 0, policy_version 382478 (0.0006) [2023-12-27 08:17:13,626][20381] Updated weights for policy 1, policy_version 382335 (0.0006) [2023-12-27 08:17:13,667][20347] Updated weights for policy 0, policy_version 382488 (0.0008) [2023-12-27 08:17:13,676][20381] Updated weights for policy 1, policy_version 382345 (0.0006) [2023-12-27 08:17:14,353][20347] Updated weights for policy 0, policy_version 382498 (0.0006) [2023-12-27 08:17:14,366][20381] Updated weights for policy 1, policy_version 382355 (0.0005) [2023-12-27 08:17:14,409][20347] Updated weights for policy 0, policy_version 382508 (0.0006) [2023-12-27 08:17:14,426][20381] Updated weights for policy 1, policy_version 382365 (0.0006) [2023-12-27 08:17:14,468][20347] Updated weights for policy 0, policy_version 382518 (0.0008) [2023-12-27 08:17:14,485][20381] Updated weights for policy 1, policy_version 382375 (0.0007) [2023-12-27 08:17:14,523][20347] Updated weights for policy 0, policy_version 382528 (0.0007) [2023-12-27 08:17:15,239][20347] Updated weights for policy 0, policy_version 382538 (0.0006) [2023-12-27 08:17:15,263][20381] Updated weights for policy 1, policy_version 382385 (0.0006) [2023-12-27 08:17:15,300][20347] Updated weights for policy 0, policy_version 382548 (0.0008) [2023-12-27 08:17:15,326][20381] Updated weights for policy 1, policy_version 382395 (0.0006) [2023-12-27 08:17:15,360][20347] Updated weights for policy 0, policy_version 382558 (0.0006) [2023-12-27 08:17:15,385][20381] Updated weights for policy 1, policy_version 382405 (0.0007) [2023-12-27 08:17:15,448][20381] Updated weights for policy 1, policy_version 382415 (0.0008) [2023-12-27 08:17:16,036][20347] Updated weights for policy 0, policy_version 382568 (0.0007) [2023-12-27 08:17:16,089][20347] Updated weights for policy 0, policy_version 382578 (0.0007) [2023-12-27 08:17:16,138][20381] Updated weights for policy 1, policy_version 382425 (0.0007) [2023-12-27 08:17:16,141][20347] Updated weights for policy 0, policy_version 382588 (0.0006) [2023-12-27 08:17:16,196][20381] Updated weights for policy 1, policy_version 382435 (0.0007) [2023-12-27 08:17:16,251][20381] Updated weights for policy 1, policy_version 382445 (0.0008) [2023-12-27 08:17:16,827][20347] Updated weights for policy 0, policy_version 382598 (0.0007) [2023-12-27 08:17:16,883][20347] Updated weights for policy 0, policy_version 382608 (0.0008) [2023-12-27 08:17:16,898][20291] KL-divergence is very high: 133.0820 [2023-12-27 08:17:16,928][20347] Updated weights for policy 0, policy_version 382618 (0.0006) [2023-12-27 08:17:16,937][20381] Updated weights for policy 1, policy_version 382455 (0.0007) [2023-12-27 08:17:16,992][20381] Updated weights for policy 1, policy_version 382465 (0.0006) [2023-12-27 08:17:17,051][20381] Updated weights for policy 1, policy_version 382475 (0.0007) [2023-12-27 08:17:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 195903488. Throughput: 0: 10347.5, 1: 10241.8. Samples: 195912432. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:17,363][19170] Avg episode reward: [(0, '788.430'), (1, '961.370')] [2023-12-27 08:17:17,648][20347] Updated weights for policy 0, policy_version 382628 (0.0006) [2023-12-27 08:17:17,667][20381] Updated weights for policy 1, policy_version 382485 (0.0007) [2023-12-27 08:17:17,694][20347] Updated weights for policy 0, policy_version 382638 (0.0008) [2023-12-27 08:17:17,720][20381] Updated weights for policy 1, policy_version 382495 (0.0005) [2023-12-27 08:17:17,742][20347] Updated weights for policy 0, policy_version 382648 (0.0007) [2023-12-27 08:17:17,777][20381] Updated weights for policy 1, policy_version 382505 (0.0006) [2023-12-27 08:17:18,429][20381] Updated weights for policy 1, policy_version 382515 (0.0007) [2023-12-27 08:17:18,460][20347] Updated weights for policy 0, policy_version 382658 (0.0006) [2023-12-27 08:17:18,489][20381] Updated weights for policy 1, policy_version 382525 (0.0007) [2023-12-27 08:17:18,521][20347] Updated weights for policy 0, policy_version 382668 (0.0007) [2023-12-27 08:17:18,538][20381] Updated weights for policy 1, policy_version 382535 (0.0006) [2023-12-27 08:17:18,581][20347] Updated weights for policy 0, policy_version 382678 (0.0007) [2023-12-27 08:17:18,634][20347] Updated weights for policy 0, policy_version 382688 (0.0008) [2023-12-27 08:17:19,158][20381] Updated weights for policy 1, policy_version 382545 (0.0005) [2023-12-27 08:17:19,220][20381] Updated weights for policy 1, policy_version 382555 (0.0008) [2023-12-27 08:17:19,276][20381] Updated weights for policy 1, policy_version 382565 (0.0010) [2023-12-27 08:17:19,340][20381] Updated weights for policy 1, policy_version 382575 (0.0011) [2023-12-27 08:17:19,355][20347] Updated weights for policy 0, policy_version 382698 (0.0008) [2023-12-27 08:17:19,417][20347] Updated weights for policy 0, policy_version 382708 (0.0006) [2023-12-27 08:17:19,474][20347] Updated weights for policy 0, policy_version 382718 (0.0006) [2023-12-27 08:17:20,067][20381] Updated weights for policy 1, policy_version 382585 (0.0009) [2023-12-27 08:17:20,134][20381] Updated weights for policy 1, policy_version 382595 (0.0006) [2023-12-27 08:17:20,143][20347] Updated weights for policy 0, policy_version 382728 (0.0007) [2023-12-27 08:17:20,196][20347] Updated weights for policy 0, policy_version 382738 (0.0008) [2023-12-27 08:17:20,198][20381] Updated weights for policy 1, policy_version 382605 (0.0006) [2023-12-27 08:17:20,256][20347] Updated weights for policy 0, policy_version 382748 (0.0008) [2023-12-27 08:17:20,835][20381] Updated weights for policy 1, policy_version 382615 (0.0009) [2023-12-27 08:17:20,891][20381] Updated weights for policy 1, policy_version 382625 (0.0011) [2023-12-27 08:17:20,950][20381] Updated weights for policy 1, policy_version 382635 (0.0011) [2023-12-27 08:17:20,961][20347] Updated weights for policy 0, policy_version 382758 (0.0010) [2023-12-27 08:17:21,011][20347] Updated weights for policy 0, policy_version 382768 (0.0008) [2023-12-27 08:17:21,072][20347] Updated weights for policy 0, policy_version 382778 (0.0010) [2023-12-27 08:17:21,659][20381] Updated weights for policy 1, policy_version 382645 (0.0009) [2023-12-27 08:17:21,726][20381] Updated weights for policy 1, policy_version 382655 (0.0007) [2023-12-27 08:17:21,788][20381] Updated weights for policy 1, policy_version 382665 (0.0007) [2023-12-27 08:17:21,831][20347] Updated weights for policy 0, policy_version 382788 (0.0009) [2023-12-27 08:17:21,880][20347] Updated weights for policy 0, policy_version 382798 (0.0008) [2023-12-27 08:17:21,935][20347] Updated weights for policy 0, policy_version 382808 (0.0008) [2023-12-27 08:17:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 196001792. Throughput: 0: 10306.0, 1: 10249.2. Samples: 195974588. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:22,363][19170] Avg episode reward: [(0, '806.830'), (1, '932.470')] [2023-12-27 08:17:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000382672_97984512.pth... [2023-12-27 08:17:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000382816_98017280.pth... [2023-12-27 08:17:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000381488_97681408.pth [2023-12-27 08:17:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000381632_97714176.pth [2023-12-27 08:17:22,410][20381] Updated weights for policy 1, policy_version 382675 (0.0005) [2023-12-27 08:17:22,464][20381] Updated weights for policy 1, policy_version 382685 (0.0005) [2023-12-27 08:17:22,515][20381] Updated weights for policy 1, policy_version 382695 (0.0006) [2023-12-27 08:17:22,704][20347] Updated weights for policy 0, policy_version 382818 (0.0009) [2023-12-27 08:17:22,774][20347] Updated weights for policy 0, policy_version 382828 (0.0010) [2023-12-27 08:17:22,839][20347] Updated weights for policy 0, policy_version 382838 (0.0010) [2023-12-27 08:17:22,907][20347] Updated weights for policy 0, policy_version 382848 (0.0010) [2023-12-27 08:17:23,150][20381] Updated weights for policy 1, policy_version 382705 (0.0009) [2023-12-27 08:17:23,212][20381] Updated weights for policy 1, policy_version 382715 (0.0007) [2023-12-27 08:17:23,261][20381] Updated weights for policy 1, policy_version 382725 (0.0007) [2023-12-27 08:17:23,309][20381] Updated weights for policy 1, policy_version 382735 (0.0008) [2023-12-27 08:17:23,518][20347] Updated weights for policy 0, policy_version 382858 (0.0009) [2023-12-27 08:17:23,583][20347] Updated weights for policy 0, policy_version 382868 (0.0010) [2023-12-27 08:17:23,649][20347] Updated weights for policy 0, policy_version 382878 (0.0010) [2023-12-27 08:17:24,025][20381] Updated weights for policy 1, policy_version 382745 (0.0005) [2023-12-27 08:17:24,076][20381] Updated weights for policy 1, policy_version 382755 (0.0006) [2023-12-27 08:17:24,126][20381] Updated weights for policy 1, policy_version 382765 (0.0007) [2023-12-27 08:17:24,260][20347] Updated weights for policy 0, policy_version 382888 (0.0006) [2023-12-27 08:17:24,323][20347] Updated weights for policy 0, policy_version 382898 (0.0006) [2023-12-27 08:17:24,386][20347] Updated weights for policy 0, policy_version 382908 (0.0008) [2023-12-27 08:17:24,796][20381] Updated weights for policy 1, policy_version 382775 (0.0006) [2023-12-27 08:17:24,851][20381] Updated weights for policy 1, policy_version 382785 (0.0010) [2023-12-27 08:17:24,900][20381] Updated weights for policy 1, policy_version 382795 (0.0006) [2023-12-27 08:17:24,933][20347] Updated weights for policy 0, policy_version 382918 (0.0009) [2023-12-27 08:17:24,977][20347] Updated weights for policy 0, policy_version 382928 (0.0010) [2023-12-27 08:17:25,028][20347] Updated weights for policy 0, policy_version 382938 (0.0010) [2023-12-27 08:17:25,430][20381] Updated weights for policy 1, policy_version 382805 (0.0008) [2023-12-27 08:17:25,479][20381] Updated weights for policy 1, policy_version 382815 (0.0010) [2023-12-27 08:17:25,527][20381] Updated weights for policy 1, policy_version 382825 (0.0010) [2023-12-27 08:17:25,703][20347] Updated weights for policy 0, policy_version 382948 (0.0010) [2023-12-27 08:17:25,756][20347] Updated weights for policy 0, policy_version 382958 (0.0009) [2023-12-27 08:17:25,810][20347] Updated weights for policy 0, policy_version 382968 (0.0008) [2023-12-27 08:17:26,193][20381] Updated weights for policy 1, policy_version 382835 (0.0010) [2023-12-27 08:17:26,241][20381] Updated weights for policy 1, policy_version 382845 (0.0010) [2023-12-27 08:17:26,292][20381] Updated weights for policy 1, policy_version 382855 (0.0010) [2023-12-27 08:17:26,432][20347] Updated weights for policy 0, policy_version 382978 (0.0008) [2023-12-27 08:17:26,480][20347] Updated weights for policy 0, policy_version 382988 (0.0005) [2023-12-27 08:17:26,523][20347] Updated weights for policy 0, policy_version 382998 (0.0005) [2023-12-27 08:17:26,572][20347] Updated weights for policy 0, policy_version 383008 (0.0005) [2023-12-27 08:17:26,979][20381] Updated weights for policy 1, policy_version 382865 (0.0010) [2023-12-27 08:17:27,039][20381] Updated weights for policy 1, policy_version 382875 (0.0010) [2023-12-27 08:17:27,092][20381] Updated weights for policy 1, policy_version 382885 (0.0008) [2023-12-27 08:17:27,153][20381] Updated weights for policy 1, policy_version 382895 (0.0005) [2023-12-27 08:17:27,223][20347] Updated weights for policy 0, policy_version 383018 (0.0009) [2023-12-27 08:17:27,286][20347] Updated weights for policy 0, policy_version 383028 (0.0005) [2023-12-27 08:17:27,342][20347] Updated weights for policy 0, policy_version 383038 (0.0010) [2023-12-27 08:17:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 196116480. Throughput: 0: 10259.7, 1: 10334.8. Samples: 196099828. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:27,363][19170] Avg episode reward: [(0, '903.650'), (1, '942.280')] [2023-12-27 08:17:27,681][20381] Updated weights for policy 1, policy_version 382905 (0.0007) [2023-12-27 08:17:27,744][20381] Updated weights for policy 1, policy_version 382915 (0.0008) [2023-12-27 08:17:27,804][20381] Updated weights for policy 1, policy_version 382925 (0.0008) [2023-12-27 08:17:27,912][20347] Updated weights for policy 0, policy_version 383048 (0.0010) [2023-12-27 08:17:27,966][20347] Updated weights for policy 0, policy_version 383058 (0.0010) [2023-12-27 08:17:28,023][20347] Updated weights for policy 0, policy_version 383068 (0.0010) [2023-12-27 08:17:28,374][20381] Updated weights for policy 1, policy_version 382935 (0.0009) [2023-12-27 08:17:28,429][20381] Updated weights for policy 1, policy_version 382945 (0.0010) [2023-12-27 08:17:28,487][20381] Updated weights for policy 1, policy_version 382955 (0.0010) [2023-12-27 08:17:28,667][20347] Updated weights for policy 0, policy_version 383078 (0.0008) [2023-12-27 08:17:28,727][20347] Updated weights for policy 0, policy_version 383088 (0.0008) [2023-12-27 08:17:28,789][20347] Updated weights for policy 0, policy_version 383098 (0.0008) [2023-12-27 08:17:29,167][20381] Updated weights for policy 1, policy_version 382965 (0.0010) [2023-12-27 08:17:29,220][20381] Updated weights for policy 1, policy_version 382975 (0.0011) [2023-12-27 08:17:29,281][20381] Updated weights for policy 1, policy_version 382985 (0.0011) [2023-12-27 08:17:29,377][20347] Updated weights for policy 0, policy_version 383108 (0.0009) [2023-12-27 08:17:29,427][20347] Updated weights for policy 0, policy_version 383118 (0.0008) [2023-12-27 08:17:29,474][20347] Updated weights for policy 0, policy_version 383128 (0.0008) [2023-12-27 08:17:29,965][20381] Updated weights for policy 1, policy_version 382995 (0.0011) [2023-12-27 08:17:30,023][20381] Updated weights for policy 1, policy_version 383005 (0.0010) [2023-12-27 08:17:30,080][20381] Updated weights for policy 1, policy_version 383015 (0.0010) [2023-12-27 08:17:30,175][20347] Updated weights for policy 0, policy_version 383138 (0.0009) [2023-12-27 08:17:30,236][20347] Updated weights for policy 0, policy_version 383148 (0.0007) [2023-12-27 08:17:30,294][20347] Updated weights for policy 0, policy_version 383158 (0.0007) [2023-12-27 08:17:30,352][20347] Updated weights for policy 0, policy_version 383168 (0.0007) [2023-12-27 08:17:30,749][20381] Updated weights for policy 1, policy_version 383025 (0.0009) [2023-12-27 08:17:30,799][20381] Updated weights for policy 1, policy_version 383035 (0.0005) [2023-12-27 08:17:30,847][20381] Updated weights for policy 1, policy_version 383045 (0.0005) [2023-12-27 08:17:30,897][20381] Updated weights for policy 1, policy_version 383055 (0.0005) [2023-12-27 08:17:31,016][20347] Updated weights for policy 0, policy_version 383178 (0.0009) [2023-12-27 08:17:31,069][20291] KL-divergence is very high: 110.9156 [2023-12-27 08:17:31,075][20347] Updated weights for policy 0, policy_version 383188 (0.0010) [2023-12-27 08:17:31,135][20347] Updated weights for policy 0, policy_version 383198 (0.0009) [2023-12-27 08:17:31,537][20381] Updated weights for policy 1, policy_version 383065 (0.0009) [2023-12-27 08:17:31,597][20381] Updated weights for policy 1, policy_version 383075 (0.0008) [2023-12-27 08:17:31,660][20381] Updated weights for policy 1, policy_version 383085 (0.0008) [2023-12-27 08:17:31,818][20347] Updated weights for policy 0, policy_version 383208 (0.0008) [2023-12-27 08:17:31,880][20347] Updated weights for policy 0, policy_version 383218 (0.0005) [2023-12-27 08:17:31,936][20347] Updated weights for policy 0, policy_version 383228 (0.0006) [2023-12-27 08:17:32,339][20381] Updated weights for policy 1, policy_version 383095 (0.0008) [2023-12-27 08:17:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 196214784. Throughput: 0: 10290.1, 1: 10327.6. Samples: 196228396. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:32,363][19170] Avg episode reward: [(0, '961.270'), (1, '990.370')] [2023-12-27 08:17:32,395][20381] Updated weights for policy 1, policy_version 383105 (0.0007) [2023-12-27 08:17:32,443][20381] Updated weights for policy 1, policy_version 383115 (0.0006) [2023-12-27 08:17:32,510][20347] Updated weights for policy 0, policy_version 383238 (0.0005) [2023-12-27 08:17:32,560][20347] Updated weights for policy 0, policy_version 383248 (0.0005) [2023-12-27 08:17:32,620][20347] Updated weights for policy 0, policy_version 383258 (0.0005) [2023-12-27 08:17:33,155][20347] Updated weights for policy 0, policy_version 383268 (0.0006) [2023-12-27 08:17:33,189][20381] Updated weights for policy 1, policy_version 383125 (0.0006) [2023-12-27 08:17:33,209][20347] Updated weights for policy 0, policy_version 383278 (0.0006) [2023-12-27 08:17:33,241][20381] Updated weights for policy 1, policy_version 383135 (0.0005) [2023-12-27 08:17:33,264][20347] Updated weights for policy 0, policy_version 383288 (0.0006) [2023-12-27 08:17:33,295][20381] Updated weights for policy 1, policy_version 383145 (0.0005) [2023-12-27 08:17:33,860][20381] Updated weights for policy 1, policy_version 383155 (0.0008) [2023-12-27 08:17:33,879][20347] Updated weights for policy 0, policy_version 383298 (0.0007) [2023-12-27 08:17:33,914][20381] Updated weights for policy 1, policy_version 383165 (0.0007) [2023-12-27 08:17:33,924][20347] Updated weights for policy 0, policy_version 383308 (0.0010) [2023-12-27 08:17:33,967][20381] Updated weights for policy 1, policy_version 383175 (0.0005) [2023-12-27 08:17:33,975][20347] Updated weights for policy 0, policy_version 383318 (0.0010) [2023-12-27 08:17:34,024][20347] Updated weights for policy 0, policy_version 383328 (0.0009) [2023-12-27 08:17:34,550][20381] Updated weights for policy 1, policy_version 383185 (0.0006) [2023-12-27 08:17:34,615][20381] Updated weights for policy 1, policy_version 383195 (0.0009) [2023-12-27 08:17:34,641][20292] KL-divergence is very high: 196.3329 [2023-12-27 08:17:34,665][20347] Updated weights for policy 0, policy_version 383338 (0.0006) [2023-12-27 08:17:34,675][20381] Updated weights for policy 1, policy_version 383205 (0.0007) [2023-12-27 08:17:34,715][20347] Updated weights for policy 0, policy_version 383348 (0.0007) [2023-12-27 08:17:34,731][20292] KL-divergence is very high: 687.3777 [2023-12-27 08:17:34,731][20381] Updated weights for policy 1, policy_version 383215 (0.0007) [2023-12-27 08:17:34,767][20347] Updated weights for policy 0, policy_version 383358 (0.0005) [2023-12-27 08:17:35,315][20347] Updated weights for policy 0, policy_version 383368 (0.0006) [2023-12-27 08:17:35,371][20347] Updated weights for policy 0, policy_version 383378 (0.0007) [2023-12-27 08:17:35,419][20347] Updated weights for policy 0, policy_version 383388 (0.0005) [2023-12-27 08:17:35,426][20381] Updated weights for policy 1, policy_version 383225 (0.0006) [2023-12-27 08:17:35,472][20381] Updated weights for policy 1, policy_version 383235 (0.0011) [2023-12-27 08:17:35,517][20381] Updated weights for policy 1, policy_version 383245 (0.0010) [2023-12-27 08:17:36,102][20347] Updated weights for policy 0, policy_version 383398 (0.0007) [2023-12-27 08:17:36,169][20381] Updated weights for policy 1, policy_version 383255 (0.0011) [2023-12-27 08:17:36,169][20347] Updated weights for policy 0, policy_version 383408 (0.0008) [2023-12-27 08:17:36,228][20347] Updated weights for policy 0, policy_version 383418 (0.0007) [2023-12-27 08:17:36,229][20381] Updated weights for policy 1, policy_version 383265 (0.0011) [2023-12-27 08:17:36,293][20381] Updated weights for policy 1, policy_version 383275 (0.0011) [2023-12-27 08:17:36,919][20347] Updated weights for policy 0, policy_version 383428 (0.0006) [2023-12-27 08:17:36,970][20291] KL-divergence is very high: 2346.7700 [2023-12-27 08:17:36,982][20347] Updated weights for policy 0, policy_version 383438 (0.0008) [2023-12-27 08:17:36,991][20381] Updated weights for policy 1, policy_version 383285 (0.0011) [2023-12-27 08:17:37,023][20291] KL-divergence is very high: 275.6584 [2023-12-27 08:17:37,035][20291] KL-divergence is very high: 123.0141 [2023-12-27 08:17:37,045][20381] Updated weights for policy 1, policy_version 383295 (0.0011) [2023-12-27 08:17:37,046][20347] Updated weights for policy 0, policy_version 383448 (0.0006) [2023-12-27 08:17:37,101][20381] Updated weights for policy 1, policy_version 383305 (0.0011) [2023-12-27 08:17:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20799.3). Total num frames: 196329472. Throughput: 0: 10362.8, 1: 10338.9. Samples: 196295528. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:37,363][19170] Avg episode reward: [(0, '865.190'), (1, '933.920')] [2023-12-27 08:17:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000383456_98181120.pth... [2023-12-27 08:17:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000383312_98148352.pth... [2023-12-27 08:17:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000382240_97869824.pth [2023-12-27 08:17:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000382096_97837056.pth [2023-12-27 08:17:37,734][20291] KL-divergence is very high: 338.5599 [2023-12-27 08:17:37,740][20347] Updated weights for policy 0, policy_version 383458 (0.0008) [2023-12-27 08:17:37,772][20291] KL-divergence is very high: 338.0428 [2023-12-27 08:17:37,786][20347] Updated weights for policy 0, policy_version 383468 (0.0005) [2023-12-27 08:17:37,809][20291] KL-divergence is very high: 153.1109 [2023-12-27 08:17:37,830][20381] Updated weights for policy 1, policy_version 383315 (0.0010) [2023-12-27 08:17:37,833][20347] Updated weights for policy 0, policy_version 383478 (0.0005) [2023-12-27 08:17:37,848][20291] KL-divergence is very high: 136.7335 [2023-12-27 08:17:37,879][20381] Updated weights for policy 1, policy_version 383325 (0.0010) [2023-12-27 08:17:37,887][20347] Updated weights for policy 0, policy_version 383488 (0.0005) [2023-12-27 08:17:37,925][20381] Updated weights for policy 1, policy_version 383335 (0.0010) [2023-12-27 08:17:38,535][20347] Updated weights for policy 0, policy_version 383498 (0.0007) [2023-12-27 08:17:38,592][20347] Updated weights for policy 0, policy_version 383508 (0.0008) [2023-12-27 08:17:38,603][20381] Updated weights for policy 1, policy_version 383345 (0.0010) [2023-12-27 08:17:38,645][20347] Updated weights for policy 0, policy_version 383518 (0.0007) [2023-12-27 08:17:38,653][20381] Updated weights for policy 1, policy_version 383355 (0.0005) [2023-12-27 08:17:38,711][20381] Updated weights for policy 1, policy_version 383365 (0.0007) [2023-12-27 08:17:38,767][20381] Updated weights for policy 1, policy_version 383375 (0.0011) [2023-12-27 08:17:39,336][20347] Updated weights for policy 0, policy_version 383528 (0.0008) [2023-12-27 08:17:39,393][20347] Updated weights for policy 0, policy_version 383538 (0.0008) [2023-12-27 08:17:39,448][20347] Updated weights for policy 0, policy_version 383548 (0.0006) [2023-12-27 08:17:39,472][20381] Updated weights for policy 1, policy_version 383385 (0.0008) [2023-12-27 08:17:39,538][20381] Updated weights for policy 1, policy_version 383395 (0.0006) [2023-12-27 08:17:39,597][20381] Updated weights for policy 1, policy_version 383405 (0.0007) [2023-12-27 08:17:40,172][20347] Updated weights for policy 0, policy_version 383558 (0.0007) [2023-12-27 08:17:40,226][20381] Updated weights for policy 1, policy_version 383415 (0.0008) [2023-12-27 08:17:40,230][20347] Updated weights for policy 0, policy_version 383568 (0.0009) [2023-12-27 08:17:40,280][20381] Updated weights for policy 1, policy_version 383425 (0.0007) [2023-12-27 08:17:40,289][20347] Updated weights for policy 0, policy_version 383578 (0.0006) [2023-12-27 08:17:40,333][20381] Updated weights for policy 1, policy_version 383435 (0.0007) [2023-12-27 08:17:40,988][20381] Updated weights for policy 1, policy_version 383445 (0.0008) [2023-12-27 08:17:41,012][20347] Updated weights for policy 0, policy_version 383588 (0.0007) [2023-12-27 08:17:41,047][20381] Updated weights for policy 1, policy_version 383455 (0.0006) [2023-12-27 08:17:41,071][20347] Updated weights for policy 0, policy_version 383598 (0.0006) [2023-12-27 08:17:41,103][20381] Updated weights for policy 1, policy_version 383465 (0.0006) [2023-12-27 08:17:41,122][20347] Updated weights for policy 0, policy_version 383608 (0.0007) [2023-12-27 08:17:41,837][20381] Updated weights for policy 1, policy_version 383475 (0.0006) [2023-12-27 08:17:41,891][20381] Updated weights for policy 1, policy_version 383485 (0.0005) [2023-12-27 08:17:41,906][20292] KL-divergence is very high: 130.0479 [2023-12-27 08:17:41,913][20347] Updated weights for policy 0, policy_version 383618 (0.0009) [2023-12-27 08:17:41,950][20381] Updated weights for policy 1, policy_version 383495 (0.0005) [2023-12-27 08:17:41,964][20347] Updated weights for policy 0, policy_version 383628 (0.0008) [2023-12-27 08:17:42,013][20347] Updated weights for policy 0, policy_version 383638 (0.0006) [2023-12-27 08:17:42,061][20347] Updated weights for policy 0, policy_version 383648 (0.0008) [2023-12-27 08:17:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 196427776. Throughput: 0: 10374.9, 1: 10299.7. Samples: 196416820. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:42,363][19170] Avg episode reward: [(0, '884.640'), (1, '809.360')] [2023-12-27 08:17:42,604][20381] Updated weights for policy 1, policy_version 383505 (0.0007) [2023-12-27 08:17:42,665][20381] Updated weights for policy 1, policy_version 383515 (0.0008) [2023-12-27 08:17:42,722][20381] Updated weights for policy 1, policy_version 383525 (0.0008) [2023-12-27 08:17:42,782][20381] Updated weights for policy 1, policy_version 383535 (0.0007) [2023-12-27 08:17:42,822][20347] Updated weights for policy 0, policy_version 383658 (0.0008) [2023-12-27 08:17:42,878][20347] Updated weights for policy 0, policy_version 383668 (0.0007) [2023-12-27 08:17:42,927][20347] Updated weights for policy 0, policy_version 383678 (0.0006) [2023-12-27 08:17:43,467][20381] Updated weights for policy 1, policy_version 383545 (0.0005) [2023-12-27 08:17:43,521][20381] Updated weights for policy 1, policy_version 383555 (0.0006) [2023-12-27 08:17:43,568][20381] Updated weights for policy 1, policy_version 383565 (0.0007) [2023-12-27 08:17:43,622][20347] Updated weights for policy 0, policy_version 383688 (0.0007) [2023-12-27 08:17:43,683][20347] Updated weights for policy 0, policy_version 383698 (0.0008) [2023-12-27 08:17:43,736][20347] Updated weights for policy 0, policy_version 383708 (0.0007) [2023-12-27 08:17:44,166][20381] Updated weights for policy 1, policy_version 383575 (0.0006) [2023-12-27 08:17:44,219][20381] Updated weights for policy 1, policy_version 383585 (0.0007) [2023-12-27 08:17:44,276][20381] Updated weights for policy 1, policy_version 383595 (0.0008) [2023-12-27 08:17:44,451][20347] Updated weights for policy 0, policy_version 383718 (0.0008) [2023-12-27 08:17:44,510][20347] Updated weights for policy 0, policy_version 383728 (0.0008) [2023-12-27 08:17:44,564][20347] Updated weights for policy 0, policy_version 383738 (0.0007) [2023-12-27 08:17:44,934][20381] Updated weights for policy 1, policy_version 383605 (0.0007) [2023-12-27 08:17:44,998][20381] Updated weights for policy 1, policy_version 383615 (0.0008) [2023-12-27 08:17:45,060][20381] Updated weights for policy 1, policy_version 383625 (0.0008) [2023-12-27 08:17:45,247][20347] Updated weights for policy 0, policy_version 383748 (0.0008) [2023-12-27 08:17:45,299][20347] Updated weights for policy 0, policy_version 383758 (0.0007) [2023-12-27 08:17:45,361][20347] Updated weights for policy 0, policy_version 383768 (0.0007) [2023-12-27 08:17:45,763][20381] Updated weights for policy 1, policy_version 383635 (0.0008) [2023-12-27 08:17:45,820][20381] Updated weights for policy 1, policy_version 383645 (0.0007) [2023-12-27 08:17:45,872][20381] Updated weights for policy 1, policy_version 383655 (0.0009) [2023-12-27 08:17:46,019][20347] Updated weights for policy 0, policy_version 383778 (0.0007) [2023-12-27 08:17:46,062][20347] Updated weights for policy 0, policy_version 383788 (0.0008) [2023-12-27 08:17:46,112][20347] Updated weights for policy 0, policy_version 383798 (0.0007) [2023-12-27 08:17:46,181][20347] Updated weights for policy 0, policy_version 383808 (0.0008) [2023-12-27 08:17:46,515][20381] Updated weights for policy 1, policy_version 383665 (0.0008) [2023-12-27 08:17:46,572][20381] Updated weights for policy 1, policy_version 383675 (0.0007) [2023-12-27 08:17:46,628][20381] Updated weights for policy 1, policy_version 383685 (0.0007) [2023-12-27 08:17:46,686][20381] Updated weights for policy 1, policy_version 383695 (0.0008) [2023-12-27 08:17:46,890][20347] Updated weights for policy 0, policy_version 383818 (0.0006) [2023-12-27 08:17:46,948][20347] Updated weights for policy 0, policy_version 383828 (0.0006) [2023-12-27 08:17:47,014][20347] Updated weights for policy 0, policy_version 383838 (0.0005) [2023-12-27 08:17:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 196526080. Throughput: 0: 10280.0, 1: 10345.2. Samples: 196538160. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:47,363][19170] Avg episode reward: [(0, '961.420'), (1, '657.490')] [2023-12-27 08:17:47,375][20381] Updated weights for policy 1, policy_version 383705 (0.0007) [2023-12-27 08:17:47,407][20292] KL-divergence is very high: 120.4208 [2023-12-27 08:17:47,433][20381] Updated weights for policy 1, policy_version 383715 (0.0008) [2023-12-27 08:17:47,454][20292] KL-divergence is very high: 175.4868 [2023-12-27 08:17:47,460][20292] KL-divergence is very high: 136.3677 [2023-12-27 08:17:47,466][20292] KL-divergence is very high: 112.0800 [2023-12-27 08:17:47,488][20381] Updated weights for policy 1, policy_version 383725 (0.0008) [2023-12-27 08:17:47,639][20347] Updated weights for policy 0, policy_version 383848 (0.0006) [2023-12-27 08:17:47,696][20347] Updated weights for policy 0, policy_version 383858 (0.0008) [2023-12-27 08:17:47,751][20347] Updated weights for policy 0, policy_version 383868 (0.0007) [2023-12-27 08:17:48,142][20292] KL-divergence is very high: 105.7450 [2023-12-27 08:17:48,167][20292] KL-divergence is very high: 171.5264 [2023-12-27 08:17:48,171][20381] Updated weights for policy 1, policy_version 383735 (0.0008) [2023-12-27 08:17:48,209][20292] KL-divergence is very high: 184.3141 [2023-12-27 08:17:48,226][20381] Updated weights for policy 1, policy_version 383745 (0.0008) [2023-12-27 08:17:48,252][20292] KL-divergence is very high: 185.0758 [2023-12-27 08:17:48,275][20381] Updated weights for policy 1, policy_version 383755 (0.0007) [2023-12-27 08:17:48,291][20292] KL-divergence is very high: 172.9399 [2023-12-27 08:17:48,364][20347] Updated weights for policy 0, policy_version 383878 (0.0007) [2023-12-27 08:17:48,409][20347] Updated weights for policy 0, policy_version 383888 (0.0008) [2023-12-27 08:17:48,467][20347] Updated weights for policy 0, policy_version 383898 (0.0008) [2023-12-27 08:17:48,978][20381] Updated weights for policy 1, policy_version 383765 (0.0009) [2023-12-27 08:17:49,024][20381] Updated weights for policy 1, policy_version 383775 (0.0010) [2023-12-27 08:17:49,070][20381] Updated weights for policy 1, policy_version 383785 (0.0009) [2023-12-27 08:17:49,094][20347] Updated weights for policy 0, policy_version 383908 (0.0008) [2023-12-27 08:17:49,150][20347] Updated weights for policy 0, policy_version 383918 (0.0010) [2023-12-27 08:17:49,204][20347] Updated weights for policy 0, policy_version 383928 (0.0009) [2023-12-27 08:17:49,817][20347] Updated weights for policy 0, policy_version 383938 (0.0007) [2023-12-27 08:17:49,828][20381] Updated weights for policy 1, policy_version 383795 (0.0009) [2023-12-27 08:17:49,877][20347] Updated weights for policy 0, policy_version 383948 (0.0009) [2023-12-27 08:17:49,888][20381] Updated weights for policy 1, policy_version 383805 (0.0009) [2023-12-27 08:17:49,939][20347] Updated weights for policy 0, policy_version 383958 (0.0007) [2023-12-27 08:17:49,951][20381] Updated weights for policy 1, policy_version 383815 (0.0010) [2023-12-27 08:17:50,529][20381] Updated weights for policy 1, policy_version 383825 (0.0010) [2023-12-27 08:17:50,584][20347] Updated weights for policy 0, policy_version 383969 (0.0008) [2023-12-27 08:17:50,591][20381] Updated weights for policy 1, policy_version 383835 (0.0008) [2023-12-27 08:17:50,643][20347] Updated weights for policy 0, policy_version 383979 (0.0010) [2023-12-27 08:17:50,643][20381] Updated weights for policy 1, policy_version 383845 (0.0011) [2023-12-27 08:17:50,694][20381] Updated weights for policy 1, policy_version 383855 (0.0011) [2023-12-27 08:17:50,699][20347] Updated weights for policy 0, policy_version 383989 (0.0008) [2023-12-27 08:17:50,759][20347] Updated weights for policy 0, policy_version 383999 (0.0006) [2023-12-27 08:17:51,383][20381] Updated weights for policy 1, policy_version 383865 (0.0009) [2023-12-27 08:17:51,442][20347] Updated weights for policy 0, policy_version 384009 (0.0010) [2023-12-27 08:17:51,445][20381] Updated weights for policy 1, policy_version 383875 (0.0006) [2023-12-27 08:17:51,501][20381] Updated weights for policy 1, policy_version 383885 (0.0006) [2023-12-27 08:17:51,502][20347] Updated weights for policy 0, policy_version 384019 (0.0008) [2023-12-27 08:17:51,558][20347] Updated weights for policy 0, policy_version 384029 (0.0008) [2023-12-27 08:17:52,222][20381] Updated weights for policy 1, policy_version 383895 (0.0006) [2023-12-27 08:17:52,224][20347] Updated weights for policy 0, policy_version 384039 (0.0009) [2023-12-27 08:17:52,279][20347] Updated weights for policy 0, policy_version 384049 (0.0007) [2023-12-27 08:17:52,281][20381] Updated weights for policy 1, policy_version 383905 (0.0007) [2023-12-27 08:17:52,333][20347] Updated weights for policy 0, policy_version 384059 (0.0007) [2023-12-27 08:17:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 196624384. Throughput: 0: 10315.4, 1: 10317.9. Samples: 196601056. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:52,363][19170] Avg episode reward: [(0, '884.210'), (1, '561.940')] [2023-12-27 08:17:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000384064_98336768.pth... [2023-12-27 08:17:52,368][20381] Updated weights for policy 1, policy_version 383915 (0.0009) [2023-12-27 08:17:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000382816_98017280.pth [2023-12-27 08:17:52,394][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000383920_98304000.pth... [2023-12-27 08:17:52,398][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000382672_97984512.pth [2023-12-27 08:17:53,015][20347] Updated weights for policy 0, policy_version 384069 (0.0008) [2023-12-27 08:17:53,050][20381] Updated weights for policy 1, policy_version 383926 (0.0008) [2023-12-27 08:17:53,066][20347] Updated weights for policy 0, policy_version 384079 (0.0007) [2023-12-27 08:17:53,101][20381] Updated weights for policy 1, policy_version 383936 (0.0008) [2023-12-27 08:17:53,128][20347] Updated weights for policy 0, policy_version 384089 (0.0008) [2023-12-27 08:17:53,153][20381] Updated weights for policy 1, policy_version 383946 (0.0007) [2023-12-27 08:17:53,711][20381] Updated weights for policy 1, policy_version 383956 (0.0006) [2023-12-27 08:17:53,761][20381] Updated weights for policy 1, policy_version 383966 (0.0006) [2023-12-27 08:17:53,796][20347] Updated weights for policy 0, policy_version 384099 (0.0008) [2023-12-27 08:17:53,814][20381] Updated weights for policy 1, policy_version 383976 (0.0008) [2023-12-27 08:17:53,848][20347] Updated weights for policy 0, policy_version 384109 (0.0010) [2023-12-27 08:17:53,897][20347] Updated weights for policy 0, policy_version 384119 (0.0010) [2023-12-27 08:17:54,442][20381] Updated weights for policy 1, policy_version 383986 (0.0009) [2023-12-27 08:17:54,488][20381] Updated weights for policy 1, policy_version 383996 (0.0010) [2023-12-27 08:17:54,547][20381] Updated weights for policy 1, policy_version 384006 (0.0011) [2023-12-27 08:17:54,581][20347] Updated weights for policy 0, policy_version 384129 (0.0010) [2023-12-27 08:17:54,607][20381] Updated weights for policy 1, policy_version 384016 (0.0011) [2023-12-27 08:17:54,633][20347] Updated weights for policy 0, policy_version 384139 (0.0010) [2023-12-27 08:17:54,687][20347] Updated weights for policy 0, policy_version 384149 (0.0010) [2023-12-27 08:17:54,741][20347] Updated weights for policy 0, policy_version 384159 (0.0009) [2023-12-27 08:17:55,299][20381] Updated weights for policy 1, policy_version 384026 (0.0005) [2023-12-27 08:17:55,355][20381] Updated weights for policy 1, policy_version 384036 (0.0005) [2023-12-27 08:17:55,357][20347] Updated weights for policy 0, policy_version 384169 (0.0010) [2023-12-27 08:17:55,403][20381] Updated weights for policy 1, policy_version 384046 (0.0005) [2023-12-27 08:17:55,416][20347] Updated weights for policy 0, policy_version 384179 (0.0009) [2023-12-27 08:17:55,466][20347] Updated weights for policy 0, policy_version 384189 (0.0005) [2023-12-27 08:17:55,940][20381] Updated weights for policy 1, policy_version 384056 (0.0009) [2023-12-27 08:17:55,995][20381] Updated weights for policy 1, policy_version 384066 (0.0010) [2023-12-27 08:17:56,047][20381] Updated weights for policy 1, policy_version 384076 (0.0010) [2023-12-27 08:17:56,121][20347] Updated weights for policy 0, policy_version 384199 (0.0010) [2023-12-27 08:17:56,172][20347] Updated weights for policy 0, policy_version 384209 (0.0010) [2023-12-27 08:17:56,226][20347] Updated weights for policy 0, policy_version 384219 (0.0010) [2023-12-27 08:17:56,627][20381] Updated weights for policy 1, policy_version 384086 (0.0007) [2023-12-27 08:17:56,683][20381] Updated weights for policy 1, policy_version 384096 (0.0009) [2023-12-27 08:17:56,733][20381] Updated weights for policy 1, policy_version 384106 (0.0007) [2023-12-27 08:17:56,900][20347] Updated weights for policy 0, policy_version 384229 (0.0010) [2023-12-27 08:17:56,957][20347] Updated weights for policy 0, policy_version 384239 (0.0010) [2023-12-27 08:17:57,009][20347] Updated weights for policy 0, policy_version 384249 (0.0010) [2023-12-27 08:17:57,312][20381] Updated weights for policy 1, policy_version 384116 (0.0008) [2023-12-27 08:17:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 196739072. Throughput: 0: 10355.4, 1: 10408.4. Samples: 196728616. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:17:57,363][19170] Avg episode reward: [(0, '893.770'), (1, '500.310')] [2023-12-27 08:17:57,374][20381] Updated weights for policy 1, policy_version 384126 (0.0005) [2023-12-27 08:17:57,419][20381] Updated weights for policy 1, policy_version 384136 (0.0005) [2023-12-27 08:17:57,688][20347] Updated weights for policy 0, policy_version 384259 (0.0010) [2023-12-27 08:17:57,736][20347] Updated weights for policy 0, policy_version 384269 (0.0010) [2023-12-27 08:17:57,784][20347] Updated weights for policy 0, policy_version 384279 (0.0006) [2023-12-27 08:17:57,994][20381] Updated weights for policy 1, policy_version 384146 (0.0006) [2023-12-27 08:17:58,052][20381] Updated weights for policy 1, policy_version 384156 (0.0011) [2023-12-27 08:17:58,108][20381] Updated weights for policy 1, policy_version 384166 (0.0010) [2023-12-27 08:17:58,162][20381] Updated weights for policy 1, policy_version 384176 (0.0008) [2023-12-27 08:17:58,420][20347] Updated weights for policy 0, policy_version 384289 (0.0005) [2023-12-27 08:17:58,473][20347] Updated weights for policy 0, policy_version 384299 (0.0007) [2023-12-27 08:17:58,526][20347] Updated weights for policy 0, policy_version 384309 (0.0007) [2023-12-27 08:17:58,579][20347] Updated weights for policy 0, policy_version 384319 (0.0009) [2023-12-27 08:17:58,889][20381] Updated weights for policy 1, policy_version 384186 (0.0008) [2023-12-27 08:17:58,952][20381] Updated weights for policy 1, policy_version 384196 (0.0008) [2023-12-27 08:17:59,012][20381] Updated weights for policy 1, policy_version 384206 (0.0008) [2023-12-27 08:17:59,333][20347] Updated weights for policy 0, policy_version 384329 (0.0007) [2023-12-27 08:17:59,392][20347] Updated weights for policy 0, policy_version 384339 (0.0010) [2023-12-27 08:17:59,446][20347] Updated weights for policy 0, policy_version 384349 (0.0008) [2023-12-27 08:17:59,758][20381] Updated weights for policy 1, policy_version 384216 (0.0006) [2023-12-27 08:17:59,805][20381] Updated weights for policy 1, policy_version 384226 (0.0008) [2023-12-27 08:17:59,868][20381] Updated weights for policy 1, policy_version 384236 (0.0008) [2023-12-27 08:18:00,068][20347] Updated weights for policy 0, policy_version 384359 (0.0006) [2023-12-27 08:18:00,118][20347] Updated weights for policy 0, policy_version 384369 (0.0007) [2023-12-27 08:18:00,175][20347] Updated weights for policy 0, policy_version 384379 (0.0008) [2023-12-27 08:18:00,482][20381] Updated weights for policy 1, policy_version 384246 (0.0008) [2023-12-27 08:18:00,544][20381] Updated weights for policy 1, policy_version 384256 (0.0007) [2023-12-27 08:18:00,608][20381] Updated weights for policy 1, policy_version 384266 (0.0006) [2023-12-27 08:18:00,842][20347] Updated weights for policy 0, policy_version 384389 (0.0010) [2023-12-27 08:18:00,898][20347] Updated weights for policy 0, policy_version 384399 (0.0010) [2023-12-27 08:18:00,944][20347] Updated weights for policy 0, policy_version 384409 (0.0010) [2023-12-27 08:18:01,226][20381] Updated weights for policy 1, policy_version 384276 (0.0005) [2023-12-27 08:18:01,285][20381] Updated weights for policy 1, policy_version 384286 (0.0008) [2023-12-27 08:18:01,348][20381] Updated weights for policy 1, policy_version 384296 (0.0008) [2023-12-27 08:18:01,683][20347] Updated weights for policy 0, policy_version 384419 (0.0010) [2023-12-27 08:18:01,743][20347] Updated weights for policy 0, policy_version 384429 (0.0008) [2023-12-27 08:18:01,806][20347] Updated weights for policy 0, policy_version 384439 (0.0008) [2023-12-27 08:18:02,035][20381] Updated weights for policy 1, policy_version 384306 (0.0007) [2023-12-27 08:18:02,091][20381] Updated weights for policy 1, policy_version 384316 (0.0008) [2023-12-27 08:18:02,144][20381] Updated weights for policy 1, policy_version 384326 (0.0008) [2023-12-27 08:18:02,199][20381] Updated weights for policy 1, policy_version 384336 (0.0006) [2023-12-27 08:18:02,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 196845568. Throughput: 0: 10408.4, 1: 10519.1. Samples: 196854172. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:18:02,363][19170] Avg episode reward: [(0, '942.070'), (1, '599.640')] [2023-12-27 08:18:02,397][20347] Updated weights for policy 0, policy_version 384449 (0.0007) [2023-12-27 08:18:02,451][20347] Updated weights for policy 0, policy_version 384459 (0.0008) [2023-12-27 08:18:02,512][20347] Updated weights for policy 0, policy_version 384469 (0.0006) [2023-12-27 08:18:02,571][20347] Updated weights for policy 0, policy_version 384479 (0.0007) [2023-12-27 08:18:02,857][20381] Updated weights for policy 1, policy_version 384346 (0.0007) [2023-12-27 08:18:02,913][20381] Updated weights for policy 1, policy_version 384356 (0.0008) [2023-12-27 08:18:02,973][20381] Updated weights for policy 1, policy_version 384366 (0.0009) [2023-12-27 08:18:03,142][20347] Updated weights for policy 0, policy_version 384489 (0.0007) [2023-12-27 08:18:03,196][20347] Updated weights for policy 0, policy_version 384499 (0.0007) [2023-12-27 08:18:03,240][20347] Updated weights for policy 0, policy_version 384509 (0.0007) [2023-12-27 08:18:03,646][20381] Updated weights for policy 1, policy_version 384376 (0.0006) [2023-12-27 08:18:03,706][20381] Updated weights for policy 1, policy_version 384386 (0.0007) [2023-12-27 08:18:03,761][20381] Updated weights for policy 1, policy_version 384396 (0.0008) [2023-12-27 08:18:03,930][20347] Updated weights for policy 0, policy_version 384519 (0.0006) [2023-12-27 08:18:03,996][20347] Updated weights for policy 0, policy_version 384529 (0.0006) [2023-12-27 08:18:04,056][20347] Updated weights for policy 0, policy_version 384539 (0.0006) [2023-12-27 08:18:04,423][20381] Updated weights for policy 1, policy_version 384406 (0.0008) [2023-12-27 08:18:04,487][20381] Updated weights for policy 1, policy_version 384416 (0.0009) [2023-12-27 08:18:04,547][20381] Updated weights for policy 1, policy_version 384426 (0.0008) [2023-12-27 08:18:04,583][20347] Updated weights for policy 0, policy_version 384549 (0.0006) [2023-12-27 08:18:04,637][20347] Updated weights for policy 0, policy_version 384559 (0.0007) [2023-12-27 08:18:04,698][20347] Updated weights for policy 0, policy_version 384569 (0.0007) [2023-12-27 08:18:05,198][20381] Updated weights for policy 1, policy_version 384436 (0.0007) [2023-12-27 08:18:05,252][20381] Updated weights for policy 1, policy_version 384446 (0.0005) [2023-12-27 08:18:05,305][20381] Updated weights for policy 1, policy_version 384456 (0.0005) [2023-12-27 08:18:05,412][20347] Updated weights for policy 0, policy_version 384579 (0.0008) [2023-12-27 08:18:05,475][20347] Updated weights for policy 0, policy_version 384589 (0.0007) [2023-12-27 08:18:05,532][20347] Updated weights for policy 0, policy_version 384599 (0.0008) [2023-12-27 08:18:05,831][20381] Updated weights for policy 1, policy_version 384466 (0.0005) [2023-12-27 08:18:05,886][20381] Updated weights for policy 1, policy_version 384476 (0.0005) [2023-12-27 08:18:05,937][20381] Updated weights for policy 1, policy_version 384486 (0.0005) [2023-12-27 08:18:05,983][20381] Updated weights for policy 1, policy_version 384496 (0.0005) [2023-12-27 08:18:06,211][20347] Updated weights for policy 0, policy_version 384609 (0.0008) [2023-12-27 08:18:06,274][20347] Updated weights for policy 0, policy_version 384619 (0.0008) [2023-12-27 08:18:06,333][20347] Updated weights for policy 0, policy_version 384629 (0.0008) [2023-12-27 08:18:06,395][20347] Updated weights for policy 0, policy_version 384639 (0.0008) [2023-12-27 08:18:06,620][20381] Updated weights for policy 1, policy_version 384506 (0.0008) [2023-12-27 08:18:06,676][20381] Updated weights for policy 1, policy_version 384516 (0.0008) [2023-12-27 08:18:06,731][20381] Updated weights for policy 1, policy_version 384526 (0.0007) [2023-12-27 08:18:07,060][20347] Updated weights for policy 0, policy_version 384649 (0.0005) [2023-12-27 08:18:07,110][20347] Updated weights for policy 0, policy_version 384659 (0.0005) [2023-12-27 08:18:07,158][20347] Updated weights for policy 0, policy_version 384669 (0.0007) [2023-12-27 08:18:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 196952064. Throughput: 0: 10494.7, 1: 10504.5. Samples: 196919556. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:18:07,364][19170] Avg episode reward: [(0, '932.640'), (1, '734.670')] [2023-12-27 08:18:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000384528_98459648.pth... [2023-12-27 08:18:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000384672_98492416.pth... [2023-12-27 08:18:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000383312_98148352.pth [2023-12-27 08:18:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000383456_98181120.pth [2023-12-27 08:18:07,460][20381] Updated weights for policy 1, policy_version 384536 (0.0008) [2023-12-27 08:18:07,514][20381] Updated weights for policy 1, policy_version 384546 (0.0008) [2023-12-27 08:18:07,571][20381] Updated weights for policy 1, policy_version 384556 (0.0007) [2023-12-27 08:18:07,878][20347] Updated weights for policy 0, policy_version 384679 (0.0007) [2023-12-27 08:18:07,943][20347] Updated weights for policy 0, policy_version 384689 (0.0008) [2023-12-27 08:18:08,006][20347] Updated weights for policy 0, policy_version 384699 (0.0007) [2023-12-27 08:18:08,293][20381] Updated weights for policy 1, policy_version 384566 (0.0007) [2023-12-27 08:18:08,357][20381] Updated weights for policy 1, policy_version 384576 (0.0008) [2023-12-27 08:18:08,416][20381] Updated weights for policy 1, policy_version 384586 (0.0008) [2023-12-27 08:18:08,679][20347] Updated weights for policy 0, policy_version 384709 (0.0007) [2023-12-27 08:18:08,744][20347] Updated weights for policy 0, policy_version 384719 (0.0008) [2023-12-27 08:18:08,799][20347] Updated weights for policy 0, policy_version 384729 (0.0007) [2023-12-27 08:18:09,101][20381] Updated weights for policy 1, policy_version 384596 (0.0008) [2023-12-27 08:18:09,152][20381] Updated weights for policy 1, policy_version 384606 (0.0007) [2023-12-27 08:18:09,197][20381] Updated weights for policy 1, policy_version 384616 (0.0008) [2023-12-27 08:18:09,521][20347] Updated weights for policy 0, policy_version 384739 (0.0007) [2023-12-27 08:18:09,578][20347] Updated weights for policy 0, policy_version 384749 (0.0007) [2023-12-27 08:18:09,643][20347] Updated weights for policy 0, policy_version 384759 (0.0006) [2023-12-27 08:18:09,968][20381] Updated weights for policy 1, policy_version 384626 (0.0008) [2023-12-27 08:18:10,023][20381] Updated weights for policy 1, policy_version 384636 (0.0008) [2023-12-27 08:18:10,085][20381] Updated weights for policy 1, policy_version 384646 (0.0008) [2023-12-27 08:18:10,156][20381] Updated weights for policy 1, policy_version 384656 (0.0008) [2023-12-27 08:18:10,271][20347] Updated weights for policy 0, policy_version 384769 (0.0008) [2023-12-27 08:18:10,338][20347] Updated weights for policy 0, policy_version 384779 (0.0008) [2023-12-27 08:18:10,405][20347] Updated weights for policy 0, policy_version 384789 (0.0007) [2023-12-27 08:18:10,463][20347] Updated weights for policy 0, policy_version 384799 (0.0007) [2023-12-27 08:18:10,801][20381] Updated weights for policy 1, policy_version 384666 (0.0005) [2023-12-27 08:18:10,853][20381] Updated weights for policy 1, policy_version 384676 (0.0005) [2023-12-27 08:18:10,906][20381] Updated weights for policy 1, policy_version 384686 (0.0005) [2023-12-27 08:18:11,194][20347] Updated weights for policy 0, policy_version 384809 (0.0008) [2023-12-27 08:18:11,247][20347] Updated weights for policy 0, policy_version 384819 (0.0007) [2023-12-27 08:18:11,311][20347] Updated weights for policy 0, policy_version 384829 (0.0007) [2023-12-27 08:18:11,541][20381] Updated weights for policy 1, policy_version 384696 (0.0008) [2023-12-27 08:18:11,605][20381] Updated weights for policy 1, policy_version 384706 (0.0007) [2023-12-27 08:18:11,665][20381] Updated weights for policy 1, policy_version 384716 (0.0008) [2023-12-27 08:18:11,994][20347] Updated weights for policy 0, policy_version 384839 (0.0007) [2023-12-27 08:18:12,054][20347] Updated weights for policy 0, policy_version 384849 (0.0006) [2023-12-27 08:18:12,110][20347] Updated weights for policy 0, policy_version 384859 (0.0009) [2023-12-27 08:18:12,346][20381] Updated weights for policy 1, policy_version 384726 (0.0009) [2023-12-27 08:18:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 197050368. Throughput: 0: 10448.8, 1: 10466.1. Samples: 197041000. Policy #0 lag: (min: 15.0, avg: 20.5, max: 47.0) [2023-12-27 08:18:12,363][19170] Avg episode reward: [(0, '857.060'), (1, '808.310')] [2023-12-27 08:18:12,407][20381] Updated weights for policy 1, policy_version 384736 (0.0009) [2023-12-27 08:18:12,469][20381] Updated weights for policy 1, policy_version 384746 (0.0007) [2023-12-27 08:18:12,792][20347] Updated weights for policy 0, policy_version 384869 (0.0007) [2023-12-27 08:18:12,855][20347] Updated weights for policy 0, policy_version 384879 (0.0006) [2023-12-27 08:18:12,909][20347] Updated weights for policy 0, policy_version 384889 (0.0007) [2023-12-27 08:18:13,163][20381] Updated weights for policy 1, policy_version 384756 (0.0009) [2023-12-27 08:18:13,224][20381] Updated weights for policy 1, policy_version 384766 (0.0011) [2023-12-27 08:18:13,279][20381] Updated weights for policy 1, policy_version 384776 (0.0010) [2023-12-27 08:18:13,552][20347] Updated weights for policy 0, policy_version 384899 (0.0006) [2023-12-27 08:18:13,603][20347] Updated weights for policy 0, policy_version 384909 (0.0007) [2023-12-27 08:18:13,649][20347] Updated weights for policy 0, policy_version 384919 (0.0006) [2023-12-27 08:18:13,922][20381] Updated weights for policy 1, policy_version 384786 (0.0007) [2023-12-27 08:18:13,981][20381] Updated weights for policy 1, policy_version 384796 (0.0011) [2023-12-27 08:18:14,036][20381] Updated weights for policy 1, policy_version 384806 (0.0008) [2023-12-27 08:18:14,087][20381] Updated weights for policy 1, policy_version 384816 (0.0010) [2023-12-27 08:18:14,309][20347] Updated weights for policy 0, policy_version 384929 (0.0006) [2023-12-27 08:18:14,367][20347] Updated weights for policy 0, policy_version 384939 (0.0008) [2023-12-27 08:18:14,417][20347] Updated weights for policy 0, policy_version 384949 (0.0008) [2023-12-27 08:18:14,472][20347] Updated weights for policy 0, policy_version 384959 (0.0008) [2023-12-27 08:18:14,779][20381] Updated weights for policy 1, policy_version 384826 (0.0008) [2023-12-27 08:18:14,840][20381] Updated weights for policy 1, policy_version 384836 (0.0009) [2023-12-27 08:18:14,906][20381] Updated weights for policy 1, policy_version 384846 (0.0008) [2023-12-27 08:18:15,160][20347] Updated weights for policy 0, policy_version 384969 (0.0007) [2023-12-27 08:18:15,221][20347] Updated weights for policy 0, policy_version 384979 (0.0008) [2023-12-27 08:18:15,279][20347] Updated weights for policy 0, policy_version 384989 (0.0008) [2023-12-27 08:18:15,602][20381] Updated weights for policy 1, policy_version 384856 (0.0008) [2023-12-27 08:18:15,660][20381] Updated weights for policy 1, policy_version 384866 (0.0008) [2023-12-27 08:18:15,709][20381] Updated weights for policy 1, policy_version 384876 (0.0008) [2023-12-27 08:18:15,902][20347] Updated weights for policy 0, policy_version 384999 (0.0006) [2023-12-27 08:18:15,966][20347] Updated weights for policy 0, policy_version 385009 (0.0007) [2023-12-27 08:18:16,015][20347] Updated weights for policy 0, policy_version 385019 (0.0007) [2023-12-27 08:18:16,431][20381] Updated weights for policy 1, policy_version 384886 (0.0008) [2023-12-27 08:18:16,483][20381] Updated weights for policy 1, policy_version 384896 (0.0007) [2023-12-27 08:18:16,528][20381] Updated weights for policy 1, policy_version 384906 (0.0008) [2023-12-27 08:18:16,623][20347] Updated weights for policy 0, policy_version 385029 (0.0007) [2023-12-27 08:18:16,669][20347] Updated weights for policy 0, policy_version 385039 (0.0007) [2023-12-27 08:18:16,718][20347] Updated weights for policy 0, policy_version 385049 (0.0006) [2023-12-27 08:18:17,178][20381] Updated weights for policy 1, policy_version 384916 (0.0006) [2023-12-27 08:18:17,233][20381] Updated weights for policy 1, policy_version 384926 (0.0005) [2023-12-27 08:18:17,288][20381] Updated weights for policy 1, policy_version 384936 (0.0005) [2023-12-27 08:18:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 197156864. Throughput: 0: 10412.6, 1: 10415.7. Samples: 197165672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:17,363][19170] Avg episode reward: [(0, '924.300'), (1, '788.810')] [2023-12-27 08:18:17,464][20347] Updated weights for policy 0, policy_version 385059 (0.0007) [2023-12-27 08:18:17,525][20347] Updated weights for policy 0, policy_version 385069 (0.0008) [2023-12-27 08:18:17,580][20347] Updated weights for policy 0, policy_version 385079 (0.0007) [2023-12-27 08:18:17,896][20381] Updated weights for policy 1, policy_version 384946 (0.0006) [2023-12-27 08:18:17,949][20381] Updated weights for policy 1, policy_version 384956 (0.0008) [2023-12-27 08:18:18,002][20381] Updated weights for policy 1, policy_version 384966 (0.0009) [2023-12-27 08:18:18,051][20381] Updated weights for policy 1, policy_version 384976 (0.0010) [2023-12-27 08:18:18,205][20347] Updated weights for policy 0, policy_version 385089 (0.0008) [2023-12-27 08:18:18,265][20347] Updated weights for policy 0, policy_version 385099 (0.0005) [2023-12-27 08:18:18,323][20347] Updated weights for policy 0, policy_version 385109 (0.0006) [2023-12-27 08:18:18,386][20347] Updated weights for policy 0, policy_version 385119 (0.0007) [2023-12-27 08:18:18,760][20381] Updated weights for policy 1, policy_version 384986 (0.0008) [2023-12-27 08:18:18,820][20381] Updated weights for policy 1, policy_version 384996 (0.0008) [2023-12-27 08:18:18,883][20381] Updated weights for policy 1, policy_version 385006 (0.0007) [2023-12-27 08:18:19,027][20347] Updated weights for policy 0, policy_version 385129 (0.0008) [2023-12-27 08:18:19,081][20347] Updated weights for policy 0, policy_version 385139 (0.0008) [2023-12-27 08:18:19,128][20347] Updated weights for policy 0, policy_version 385149 (0.0006) [2023-12-27 08:18:19,582][20381] Updated weights for policy 1, policy_version 385016 (0.0010) [2023-12-27 08:18:19,641][20381] Updated weights for policy 1, policy_version 385026 (0.0011) [2023-12-27 08:18:19,690][20381] Updated weights for policy 1, policy_version 385036 (0.0010) [2023-12-27 08:18:19,829][20347] Updated weights for policy 0, policy_version 385159 (0.0009) [2023-12-27 08:18:19,894][20347] Updated weights for policy 0, policy_version 385169 (0.0011) [2023-12-27 08:18:19,957][20347] Updated weights for policy 0, policy_version 385179 (0.0011) [2023-12-27 08:18:20,386][20381] Updated weights for policy 1, policy_version 385046 (0.0011) [2023-12-27 08:18:20,448][20381] Updated weights for policy 1, policy_version 385056 (0.0011) [2023-12-27 08:18:20,503][20381] Updated weights for policy 1, policy_version 385066 (0.0011) [2023-12-27 08:18:20,647][20347] Updated weights for policy 0, policy_version 385189 (0.0010) [2023-12-27 08:18:20,702][20347] Updated weights for policy 0, policy_version 385199 (0.0009) [2023-12-27 08:18:20,756][20347] Updated weights for policy 0, policy_version 385209 (0.0009) [2023-12-27 08:18:21,339][20381] Updated weights for policy 1, policy_version 385076 (0.0010) [2023-12-27 08:18:21,401][20381] Updated weights for policy 1, policy_version 385086 (0.0009) [2023-12-27 08:18:21,454][20381] Updated weights for policy 1, policy_version 385096 (0.0009) [2023-12-27 08:18:21,652][20347] Updated weights for policy 0, policy_version 385219 (0.0009) [2023-12-27 08:18:21,712][20347] Updated weights for policy 0, policy_version 385229 (0.0009) [2023-12-27 08:18:21,774][20347] Updated weights for policy 0, policy_version 385239 (0.0009) [2023-12-27 08:18:22,131][20381] Updated weights for policy 1, policy_version 385106 (0.0009) [2023-12-27 08:18:22,182][20381] Updated weights for policy 1, policy_version 385116 (0.0008) [2023-12-27 08:18:22,229][20381] Updated weights for policy 1, policy_version 385126 (0.0005) [2023-12-27 08:18:22,284][20381] Updated weights for policy 1, policy_version 385136 (0.0007) [2023-12-27 08:18:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 197255168. Throughput: 0: 10312.3, 1: 10378.9. Samples: 197226636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:22,363][19170] Avg episode reward: [(0, '971.250'), (1, '779.590')] [2023-12-27 08:18:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000385248_98639872.pth... [2023-12-27 08:18:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000385136_98615296.pth... [2023-12-27 08:18:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000384064_98336768.pth [2023-12-27 08:18:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000383920_98304000.pth [2023-12-27 08:18:22,530][20347] Updated weights for policy 0, policy_version 385249 (0.0008) [2023-12-27 08:18:22,590][20347] Updated weights for policy 0, policy_version 385259 (0.0008) [2023-12-27 08:18:22,650][20347] Updated weights for policy 0, policy_version 385269 (0.0006) [2023-12-27 08:18:22,710][20347] Updated weights for policy 0, policy_version 385279 (0.0008) [2023-12-27 08:18:22,979][20381] Updated weights for policy 1, policy_version 385146 (0.0007) [2023-12-27 08:18:23,029][20381] Updated weights for policy 1, policy_version 385156 (0.0007) [2023-12-27 08:18:23,089][20381] Updated weights for policy 1, policy_version 385166 (0.0009) [2023-12-27 08:18:23,397][20347] Updated weights for policy 0, policy_version 385289 (0.0008) [2023-12-27 08:18:23,455][20347] Updated weights for policy 0, policy_version 385299 (0.0008) [2023-12-27 08:18:23,515][20347] Updated weights for policy 0, policy_version 385309 (0.0008) [2023-12-27 08:18:23,759][20381] Updated weights for policy 1, policy_version 385176 (0.0009) [2023-12-27 08:18:23,808][20381] Updated weights for policy 1, policy_version 385186 (0.0009) [2023-12-27 08:18:23,854][20381] Updated weights for policy 1, policy_version 385196 (0.0010) [2023-12-27 08:18:24,199][20347] Updated weights for policy 0, policy_version 385319 (0.0007) [2023-12-27 08:18:24,260][20347] Updated weights for policy 0, policy_version 385329 (0.0007) [2023-12-27 08:18:24,315][20347] Updated weights for policy 0, policy_version 385339 (0.0008) [2023-12-27 08:18:24,565][20381] Updated weights for policy 1, policy_version 385206 (0.0010) [2023-12-27 08:18:24,617][20381] Updated weights for policy 1, policy_version 385216 (0.0005) [2023-12-27 08:18:24,663][20381] Updated weights for policy 1, policy_version 385226 (0.0005) [2023-12-27 08:18:25,001][20347] Updated weights for policy 0, policy_version 385349 (0.0008) [2023-12-27 08:18:25,056][20347] Updated weights for policy 0, policy_version 385359 (0.0007) [2023-12-27 08:18:25,114][20347] Updated weights for policy 0, policy_version 385369 (0.0008) [2023-12-27 08:18:25,223][20381] Updated weights for policy 1, policy_version 385236 (0.0006) [2023-12-27 08:18:25,276][20381] Updated weights for policy 1, policy_version 385246 (0.0006) [2023-12-27 08:18:25,321][20381] Updated weights for policy 1, policy_version 385256 (0.0005) [2023-12-27 08:18:25,776][20347] Updated weights for policy 0, policy_version 385379 (0.0007) [2023-12-27 08:18:25,838][20347] Updated weights for policy 0, policy_version 385389 (0.0008) [2023-12-27 08:18:25,862][20381] Updated weights for policy 1, policy_version 385266 (0.0006) [2023-12-27 08:18:25,895][20347] Updated weights for policy 0, policy_version 385399 (0.0006) [2023-12-27 08:18:25,912][20381] Updated weights for policy 1, policy_version 385276 (0.0006) [2023-12-27 08:18:25,967][20381] Updated weights for policy 1, policy_version 385286 (0.0007) [2023-12-27 08:18:26,019][20381] Updated weights for policy 1, policy_version 385296 (0.0007) [2023-12-27 08:18:26,442][20347] Updated weights for policy 0, policy_version 385409 (0.0007) [2023-12-27 08:18:26,494][20347] Updated weights for policy 0, policy_version 385419 (0.0008) [2023-12-27 08:18:26,541][20347] Updated weights for policy 0, policy_version 385429 (0.0007) [2023-12-27 08:18:26,591][20347] Updated weights for policy 0, policy_version 385439 (0.0005) [2023-12-27 08:18:26,719][20381] Updated weights for policy 1, policy_version 385306 (0.0008) [2023-12-27 08:18:26,771][20381] Updated weights for policy 1, policy_version 385316 (0.0008) [2023-12-27 08:18:26,824][20381] Updated weights for policy 1, policy_version 385326 (0.0005) [2023-12-27 08:18:27,219][20347] Updated weights for policy 0, policy_version 385449 (0.0006) [2023-12-27 08:18:27,279][20347] Updated weights for policy 0, policy_version 385459 (0.0005) [2023-12-27 08:18:27,334][20347] Updated weights for policy 0, policy_version 385469 (0.0005) [2023-12-27 08:18:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 197361664. Throughput: 0: 10260.2, 1: 10401.4. Samples: 197346592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:27,363][19170] Avg episode reward: [(0, '980.880'), (1, '818.430')] [2023-12-27 08:18:27,491][20381] Updated weights for policy 1, policy_version 385336 (0.0005) [2023-12-27 08:18:27,546][20381] Updated weights for policy 1, policy_version 385346 (0.0005) [2023-12-27 08:18:27,603][20381] Updated weights for policy 1, policy_version 385356 (0.0005) [2023-12-27 08:18:27,885][20347] Updated weights for policy 0, policy_version 385479 (0.0006) [2023-12-27 08:18:27,971][20347] Updated weights for policy 0, policy_version 385489 (0.0005) [2023-12-27 08:18:28,027][20347] Updated weights for policy 0, policy_version 385499 (0.0005) [2023-12-27 08:18:28,142][20381] Updated weights for policy 1, policy_version 385366 (0.0005) [2023-12-27 08:18:28,199][20381] Updated weights for policy 1, policy_version 385376 (0.0007) [2023-12-27 08:18:28,254][20381] Updated weights for policy 1, policy_version 385386 (0.0005) [2023-12-27 08:18:28,551][20347] Updated weights for policy 0, policy_version 385509 (0.0007) [2023-12-27 08:18:28,602][20347] Updated weights for policy 0, policy_version 385519 (0.0005) [2023-12-27 08:18:28,661][20347] Updated weights for policy 0, policy_version 385529 (0.0007) [2023-12-27 08:18:28,872][20381] Updated weights for policy 1, policy_version 385396 (0.0007) [2023-12-27 08:18:28,929][20381] Updated weights for policy 1, policy_version 385406 (0.0011) [2023-12-27 08:18:28,988][20381] Updated weights for policy 1, policy_version 385416 (0.0010) [2023-12-27 08:18:29,275][20347] Updated weights for policy 0, policy_version 385539 (0.0009) [2023-12-27 08:18:29,343][20347] Updated weights for policy 0, policy_version 385549 (0.0006) [2023-12-27 08:18:29,403][20347] Updated weights for policy 0, policy_version 385559 (0.0008) [2023-12-27 08:18:29,621][20381] Updated weights for policy 1, policy_version 385426 (0.0009) [2023-12-27 08:18:29,682][20381] Updated weights for policy 1, policy_version 385436 (0.0005) [2023-12-27 08:18:29,740][20381] Updated weights for policy 1, policy_version 385446 (0.0006) [2023-12-27 08:18:29,788][20381] Updated weights for policy 1, policy_version 385456 (0.0007) [2023-12-27 08:18:30,066][20347] Updated weights for policy 0, policy_version 385569 (0.0008) [2023-12-27 08:18:30,127][20347] Updated weights for policy 0, policy_version 385579 (0.0010) [2023-12-27 08:18:30,182][20347] Updated weights for policy 0, policy_version 385589 (0.0009) [2023-12-27 08:18:30,236][20347] Updated weights for policy 0, policy_version 385599 (0.0005) [2023-12-27 08:18:30,446][20381] Updated weights for policy 1, policy_version 385466 (0.0008) [2023-12-27 08:18:30,507][20381] Updated weights for policy 1, policy_version 385476 (0.0008) [2023-12-27 08:18:30,570][20381] Updated weights for policy 1, policy_version 385486 (0.0008) [2023-12-27 08:18:30,769][20347] Updated weights for policy 0, policy_version 385609 (0.0005) [2023-12-27 08:18:30,806][20291] KL-divergence is very high: 173.9146 [2023-12-27 08:18:30,822][20347] Updated weights for policy 0, policy_version 385619 (0.0005) [2023-12-27 08:18:30,851][20291] KL-divergence is very high: 146.7871 [2023-12-27 08:18:30,874][20347] Updated weights for policy 0, policy_version 385629 (0.0009) [2023-12-27 08:18:31,177][20381] Updated weights for policy 1, policy_version 385496 (0.0007) [2023-12-27 08:18:31,237][20381] Updated weights for policy 1, policy_version 385506 (0.0009) [2023-12-27 08:18:31,296][20381] Updated weights for policy 1, policy_version 385516 (0.0008) [2023-12-27 08:18:31,506][20347] Updated weights for policy 0, policy_version 385639 (0.0008) [2023-12-27 08:18:31,553][20347] Updated weights for policy 0, policy_version 385649 (0.0006) [2023-12-27 08:18:31,599][20347] Updated weights for policy 0, policy_version 385659 (0.0008) [2023-12-27 08:18:32,081][20381] Updated weights for policy 1, policy_version 385526 (0.0009) [2023-12-27 08:18:32,137][20381] Updated weights for policy 1, policy_version 385536 (0.0008) [2023-12-27 08:18:32,191][20381] Updated weights for policy 1, policy_version 385546 (0.0007) [2023-12-27 08:18:32,245][20347] Updated weights for policy 0, policy_version 385669 (0.0008) [2023-12-27 08:18:32,304][20347] Updated weights for policy 0, policy_version 385679 (0.0006) [2023-12-27 08:18:32,362][20347] Updated weights for policy 0, policy_version 385689 (0.0007) [2023-12-27 08:18:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 197468160. Throughput: 0: 10471.5, 1: 10428.1. Samples: 197478640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:32,363][19170] Avg episode reward: [(0, '971.040'), (1, '913.740')] [2023-12-27 08:18:32,922][20381] Updated weights for policy 1, policy_version 385556 (0.0009) [2023-12-27 08:18:32,976][20381] Updated weights for policy 1, policy_version 385566 (0.0010) [2023-12-27 08:18:33,009][20347] Updated weights for policy 0, policy_version 385699 (0.0007) [2023-12-27 08:18:33,033][20381] Updated weights for policy 1, policy_version 385576 (0.0010) [2023-12-27 08:18:33,060][20347] Updated weights for policy 0, policy_version 385709 (0.0005) [2023-12-27 08:18:33,106][20347] Updated weights for policy 0, policy_version 385719 (0.0007) [2023-12-27 08:18:33,619][20381] Updated weights for policy 1, policy_version 385586 (0.0010) [2023-12-27 08:18:33,679][20381] Updated weights for policy 1, policy_version 385596 (0.0007) [2023-12-27 08:18:33,737][20381] Updated weights for policy 1, policy_version 385606 (0.0006) [2023-12-27 08:18:33,792][20381] Updated weights for policy 1, policy_version 385616 (0.0005) [2023-12-27 08:18:33,829][20347] Updated weights for policy 0, policy_version 385729 (0.0007) [2023-12-27 08:18:33,881][20347] Updated weights for policy 0, policy_version 385739 (0.0008) [2023-12-27 08:18:33,930][20347] Updated weights for policy 0, policy_version 385750 (0.0007) [2023-12-27 08:18:33,978][20347] Updated weights for policy 0, policy_version 385760 (0.0007) [2023-12-27 08:18:34,390][20381] Updated weights for policy 1, policy_version 385626 (0.0006) [2023-12-27 08:18:34,452][20381] Updated weights for policy 1, policy_version 385636 (0.0007) [2023-12-27 08:18:34,509][20381] Updated weights for policy 1, policy_version 385646 (0.0008) [2023-12-27 08:18:34,710][20347] Updated weights for policy 0, policy_version 385770 (0.0008) [2023-12-27 08:18:34,770][20347] Updated weights for policy 0, policy_version 385780 (0.0008) [2023-12-27 08:18:34,836][20347] Updated weights for policy 0, policy_version 385790 (0.0008) [2023-12-27 08:18:35,108][20381] Updated weights for policy 1, policy_version 385656 (0.0007) [2023-12-27 08:18:35,156][20381] Updated weights for policy 1, policy_version 385666 (0.0006) [2023-12-27 08:18:35,205][20381] Updated weights for policy 1, policy_version 385676 (0.0005) [2023-12-27 08:18:35,514][20347] Updated weights for policy 0, policy_version 385800 (0.0008) [2023-12-27 08:18:35,572][20347] Updated weights for policy 0, policy_version 385810 (0.0008) [2023-12-27 08:18:35,619][20347] Updated weights for policy 0, policy_version 385820 (0.0007) [2023-12-27 08:18:35,835][20381] Updated weights for policy 1, policy_version 385686 (0.0007) [2023-12-27 08:18:35,885][20381] Updated weights for policy 1, policy_version 385696 (0.0007) [2023-12-27 08:18:35,946][20381] Updated weights for policy 1, policy_version 385706 (0.0008) [2023-12-27 08:18:36,293][20347] Updated weights for policy 0, policy_version 385830 (0.0006) [2023-12-27 08:18:36,354][20347] Updated weights for policy 0, policy_version 385840 (0.0006) [2023-12-27 08:18:36,403][20347] Updated weights for policy 0, policy_version 385850 (0.0007) [2023-12-27 08:18:36,661][20381] Updated weights for policy 1, policy_version 385716 (0.0007) [2023-12-27 08:18:36,721][20381] Updated weights for policy 1, policy_version 385726 (0.0008) [2023-12-27 08:18:36,781][20381] Updated weights for policy 1, policy_version 385736 (0.0006) [2023-12-27 08:18:37,054][20347] Updated weights for policy 0, policy_version 385860 (0.0007) [2023-12-27 08:18:37,105][20347] Updated weights for policy 0, policy_version 385870 (0.0007) [2023-12-27 08:18:37,160][20347] Updated weights for policy 0, policy_version 385880 (0.0006) [2023-12-27 08:18:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 197574656. Throughput: 0: 10408.4, 1: 10496.8. Samples: 197541788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:37,363][19170] Avg episode reward: [(0, '961.300'), (1, '942.620')] [2023-12-27 08:18:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000385744_98770944.pth... [2023-12-27 08:18:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000385888_98803712.pth... [2023-12-27 08:18:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000384528_98459648.pth [2023-12-27 08:18:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000384672_98492416.pth [2023-12-27 08:18:37,490][20381] Updated weights for policy 1, policy_version 385746 (0.0008) [2023-12-27 08:18:37,537][20381] Updated weights for policy 1, policy_version 385756 (0.0007) [2023-12-27 08:18:37,596][20381] Updated weights for policy 1, policy_version 385766 (0.0006) [2023-12-27 08:18:37,658][20381] Updated weights for policy 1, policy_version 385776 (0.0005) [2023-12-27 08:18:37,829][20347] Updated weights for policy 0, policy_version 385890 (0.0007) [2023-12-27 08:18:37,880][20347] Updated weights for policy 0, policy_version 385900 (0.0008) [2023-12-27 08:18:37,935][20347] Updated weights for policy 0, policy_version 385910 (0.0008) [2023-12-27 08:18:37,985][20347] Updated weights for policy 0, policy_version 385920 (0.0007) [2023-12-27 08:18:38,336][20381] Updated weights for policy 1, policy_version 385786 (0.0008) [2023-12-27 08:18:38,396][20381] Updated weights for policy 1, policy_version 385796 (0.0010) [2023-12-27 08:18:38,450][20381] Updated weights for policy 1, policy_version 385806 (0.0011) [2023-12-27 08:18:38,707][20347] Updated weights for policy 0, policy_version 385930 (0.0008) [2023-12-27 08:18:38,762][20347] Updated weights for policy 0, policy_version 385940 (0.0007) [2023-12-27 08:18:38,816][20347] Updated weights for policy 0, policy_version 385950 (0.0007) [2023-12-27 08:18:39,120][20381] Updated weights for policy 1, policy_version 385816 (0.0006) [2023-12-27 08:18:39,185][20381] Updated weights for policy 1, policy_version 385826 (0.0006) [2023-12-27 08:18:39,247][20381] Updated weights for policy 1, policy_version 385836 (0.0011) [2023-12-27 08:18:39,546][20347] Updated weights for policy 0, policy_version 385960 (0.0007) [2023-12-27 08:18:39,600][20347] Updated weights for policy 0, policy_version 385970 (0.0008) [2023-12-27 08:18:39,663][20347] Updated weights for policy 0, policy_version 385980 (0.0006) [2023-12-27 08:18:39,941][20381] Updated weights for policy 1, policy_version 385846 (0.0009) [2023-12-27 08:18:39,997][20381] Updated weights for policy 1, policy_version 385856 (0.0007) [2023-12-27 08:18:40,057][20381] Updated weights for policy 1, policy_version 385866 (0.0007) [2023-12-27 08:18:40,319][20347] Updated weights for policy 0, policy_version 385990 (0.0007) [2023-12-27 08:18:40,379][20347] Updated weights for policy 0, policy_version 386000 (0.0008) [2023-12-27 08:18:40,446][20347] Updated weights for policy 0, policy_version 386010 (0.0007) [2023-12-27 08:18:40,694][20381] Updated weights for policy 1, policy_version 385876 (0.0007) [2023-12-27 08:18:40,742][20381] Updated weights for policy 1, policy_version 385886 (0.0008) [2023-12-27 08:18:40,790][20381] Updated weights for policy 1, policy_version 385896 (0.0007) [2023-12-27 08:18:41,081][20347] Updated weights for policy 0, policy_version 386020 (0.0007) [2023-12-27 08:18:41,140][20347] Updated weights for policy 0, policy_version 386030 (0.0007) [2023-12-27 08:18:41,194][20347] Updated weights for policy 0, policy_version 386040 (0.0007) [2023-12-27 08:18:41,526][20381] Updated weights for policy 1, policy_version 385906 (0.0007) [2023-12-27 08:18:41,584][20381] Updated weights for policy 1, policy_version 385916 (0.0006) [2023-12-27 08:18:41,649][20381] Updated weights for policy 1, policy_version 385926 (0.0008) [2023-12-27 08:18:41,706][20381] Updated weights for policy 1, policy_version 385936 (0.0008) [2023-12-27 08:18:41,914][20347] Updated weights for policy 0, policy_version 386050 (0.0006) [2023-12-27 08:18:41,971][20347] Updated weights for policy 0, policy_version 386060 (0.0005) [2023-12-27 08:18:42,024][20347] Updated weights for policy 0, policy_version 386070 (0.0006) [2023-12-27 08:18:42,073][20347] Updated weights for policy 0, policy_version 386080 (0.0007) [2023-12-27 08:18:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 197672960. Throughput: 0: 10392.6, 1: 10412.9. Samples: 197664864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:42,363][19170] Avg episode reward: [(0, '951.590'), (1, '934.050')] [2023-12-27 08:18:42,392][20381] Updated weights for policy 1, policy_version 385946 (0.0007) [2023-12-27 08:18:42,453][20381] Updated weights for policy 1, policy_version 385956 (0.0006) [2023-12-27 08:18:42,512][20381] Updated weights for policy 1, policy_version 385966 (0.0008) [2023-12-27 08:18:42,687][20347] Updated weights for policy 0, policy_version 386090 (0.0008) [2023-12-27 08:18:42,741][20347] Updated weights for policy 0, policy_version 386100 (0.0007) [2023-12-27 08:18:42,790][20347] Updated weights for policy 0, policy_version 386110 (0.0007) [2023-12-27 08:18:43,118][20381] Updated weights for policy 1, policy_version 385976 (0.0006) [2023-12-27 08:18:43,178][20381] Updated weights for policy 1, policy_version 385986 (0.0006) [2023-12-27 08:18:43,237][20381] Updated weights for policy 1, policy_version 385996 (0.0005) [2023-12-27 08:18:43,512][20347] Updated weights for policy 0, policy_version 386120 (0.0008) [2023-12-27 08:18:43,564][20347] Updated weights for policy 0, policy_version 386130 (0.0007) [2023-12-27 08:18:43,619][20347] Updated weights for policy 0, policy_version 386140 (0.0008) [2023-12-27 08:18:43,802][20381] Updated weights for policy 1, policy_version 386006 (0.0006) [2023-12-27 08:18:43,867][20381] Updated weights for policy 1, policy_version 386016 (0.0008) [2023-12-27 08:18:43,926][20381] Updated weights for policy 1, policy_version 386026 (0.0008) [2023-12-27 08:18:44,305][20347] Updated weights for policy 0, policy_version 386150 (0.0007) [2023-12-27 08:18:44,364][20347] Updated weights for policy 0, policy_version 386160 (0.0008) [2023-12-27 08:18:44,418][20347] Updated weights for policy 0, policy_version 386170 (0.0007) [2023-12-27 08:18:44,618][20381] Updated weights for policy 1, policy_version 386036 (0.0008) [2023-12-27 08:18:44,678][20381] Updated weights for policy 1, policy_version 386046 (0.0008) [2023-12-27 08:18:44,724][20381] Updated weights for policy 1, policy_version 386056 (0.0007) [2023-12-27 08:18:45,114][20347] Updated weights for policy 0, policy_version 386180 (0.0007) [2023-12-27 08:18:45,164][20347] Updated weights for policy 0, policy_version 386190 (0.0006) [2023-12-27 08:18:45,216][20347] Updated weights for policy 0, policy_version 386200 (0.0005) [2023-12-27 08:18:45,433][20381] Updated weights for policy 1, policy_version 386066 (0.0007) [2023-12-27 08:18:45,499][20381] Updated weights for policy 1, policy_version 386076 (0.0008) [2023-12-27 08:18:45,559][20381] Updated weights for policy 1, policy_version 386086 (0.0006) [2023-12-27 08:18:45,617][20381] Updated weights for policy 1, policy_version 386096 (0.0008) [2023-12-27 08:18:45,885][20347] Updated weights for policy 0, policy_version 386210 (0.0007) [2023-12-27 08:18:45,938][20347] Updated weights for policy 0, policy_version 386220 (0.0008) [2023-12-27 08:18:45,991][20347] Updated weights for policy 0, policy_version 386230 (0.0007) [2023-12-27 08:18:46,053][20347] Updated weights for policy 0, policy_version 386240 (0.0008) [2023-12-27 08:18:46,310][20381] Updated weights for policy 1, policy_version 386106 (0.0007) [2023-12-27 08:18:46,360][20381] Updated weights for policy 1, policy_version 386116 (0.0008) [2023-12-27 08:18:46,406][20381] Updated weights for policy 1, policy_version 386126 (0.0008) [2023-12-27 08:18:46,744][20347] Updated weights for policy 0, policy_version 386250 (0.0008) [2023-12-27 08:18:46,790][20347] Updated weights for policy 0, policy_version 386260 (0.0006) [2023-12-27 08:18:46,841][20347] Updated weights for policy 0, policy_version 386270 (0.0007) [2023-12-27 08:18:47,028][20381] Updated weights for policy 1, policy_version 386136 (0.0006) [2023-12-27 08:18:47,085][20381] Updated weights for policy 1, policy_version 386146 (0.0007) [2023-12-27 08:18:47,148][20381] Updated weights for policy 1, policy_version 386156 (0.0007) [2023-12-27 08:18:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 197779456. Throughput: 0: 10362.5, 1: 10389.0. Samples: 197787996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:47,364][19170] Avg episode reward: [(0, '961.380'), (1, '962.950')] [2023-12-27 08:18:47,563][20347] Updated weights for policy 0, policy_version 386280 (0.0007) [2023-12-27 08:18:47,608][20347] Updated weights for policy 0, policy_version 386290 (0.0007) [2023-12-27 08:18:47,661][20347] Updated weights for policy 0, policy_version 386300 (0.0007) [2023-12-27 08:18:47,828][20381] Updated weights for policy 1, policy_version 386166 (0.0008) [2023-12-27 08:18:47,876][20381] Updated weights for policy 1, policy_version 386176 (0.0008) [2023-12-27 08:18:47,918][20381] Updated weights for policy 1, policy_version 386186 (0.0007) [2023-12-27 08:18:48,364][20347] Updated weights for policy 0, policy_version 386310 (0.0007) [2023-12-27 08:18:48,420][20347] Updated weights for policy 0, policy_version 386320 (0.0005) [2023-12-27 08:18:48,475][20347] Updated weights for policy 0, policy_version 386330 (0.0006) [2023-12-27 08:18:48,668][20381] Updated weights for policy 1, policy_version 386196 (0.0007) [2023-12-27 08:18:48,722][20381] Updated weights for policy 1, policy_version 386206 (0.0008) [2023-12-27 08:18:48,782][20381] Updated weights for policy 1, policy_version 386216 (0.0008) [2023-12-27 08:18:49,057][20347] Updated weights for policy 0, policy_version 386340 (0.0006) [2023-12-27 08:18:49,117][20347] Updated weights for policy 0, policy_version 386350 (0.0005) [2023-12-27 08:18:49,174][20347] Updated weights for policy 0, policy_version 386360 (0.0007) [2023-12-27 08:18:49,523][20381] Updated weights for policy 1, policy_version 386226 (0.0008) [2023-12-27 08:18:49,585][20381] Updated weights for policy 1, policy_version 386236 (0.0006) [2023-12-27 08:18:49,649][20381] Updated weights for policy 1, policy_version 386246 (0.0005) [2023-12-27 08:18:49,708][20381] Updated weights for policy 1, policy_version 386256 (0.0005) [2023-12-27 08:18:49,833][20347] Updated weights for policy 0, policy_version 386370 (0.0007) [2023-12-27 08:18:49,888][20347] Updated weights for policy 0, policy_version 386380 (0.0007) [2023-12-27 08:18:49,955][20347] Updated weights for policy 0, policy_version 386390 (0.0007) [2023-12-27 08:18:50,015][20347] Updated weights for policy 0, policy_version 386400 (0.0007) [2023-12-27 08:18:50,337][20381] Updated weights for policy 1, policy_version 386266 (0.0008) [2023-12-27 08:18:50,394][20381] Updated weights for policy 1, policy_version 386276 (0.0009) [2023-12-27 08:18:50,459][20381] Updated weights for policy 1, policy_version 386286 (0.0008) [2023-12-27 08:18:50,670][20347] Updated weights for policy 0, policy_version 386410 (0.0008) [2023-12-27 08:18:50,728][20347] Updated weights for policy 0, policy_version 386420 (0.0007) [2023-12-27 08:18:50,790][20347] Updated weights for policy 0, policy_version 386430 (0.0007) [2023-12-27 08:18:51,173][20381] Updated weights for policy 1, policy_version 386296 (0.0010) [2023-12-27 08:18:51,234][20381] Updated weights for policy 1, policy_version 386306 (0.0011) [2023-12-27 08:18:51,297][20381] Updated weights for policy 1, policy_version 386316 (0.0008) [2023-12-27 08:18:51,462][20347] Updated weights for policy 0, policy_version 386440 (0.0008) [2023-12-27 08:18:51,512][20347] Updated weights for policy 0, policy_version 386450 (0.0006) [2023-12-27 08:18:51,571][20347] Updated weights for policy 0, policy_version 386460 (0.0007) [2023-12-27 08:18:51,990][20381] Updated weights for policy 1, policy_version 386326 (0.0007) [2023-12-27 08:18:52,041][20381] Updated weights for policy 1, policy_version 386336 (0.0006) [2023-12-27 08:18:52,087][20381] Updated weights for policy 1, policy_version 386346 (0.0005) [2023-12-27 08:18:52,266][20347] Updated weights for policy 0, policy_version 386470 (0.0008) [2023-12-27 08:18:52,315][20347] Updated weights for policy 0, policy_version 386480 (0.0008) [2023-12-27 08:18:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 197877760. Throughput: 0: 10303.2, 1: 10346.0. Samples: 197848768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:52,363][19170] Avg episode reward: [(0, '961.380'), (1, '981.160')] [2023-12-27 08:18:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000386352_98926592.pth... [2023-12-27 08:18:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000385136_98615296.pth [2023-12-27 08:18:52,375][20347] Updated weights for policy 0, policy_version 386490 (0.0008) [2023-12-27 08:18:52,409][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000386496_98959360.pth... [2023-12-27 08:18:52,413][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000385248_98639872.pth [2023-12-27 08:18:52,721][20381] Updated weights for policy 1, policy_version 386356 (0.0006) [2023-12-27 08:18:52,781][20381] Updated weights for policy 1, policy_version 386366 (0.0005) [2023-12-27 08:18:52,841][20381] Updated weights for policy 1, policy_version 386376 (0.0008) [2023-12-27 08:18:53,012][20347] Updated weights for policy 0, policy_version 386500 (0.0006) [2023-12-27 08:18:53,073][20347] Updated weights for policy 0, policy_version 386510 (0.0008) [2023-12-27 08:18:53,131][20347] Updated weights for policy 0, policy_version 386520 (0.0009) [2023-12-27 08:18:53,452][20381] Updated weights for policy 1, policy_version 386386 (0.0009) [2023-12-27 08:18:53,503][20381] Updated weights for policy 1, policy_version 386396 (0.0007) [2023-12-27 08:18:53,564][20381] Updated weights for policy 1, policy_version 386406 (0.0007) [2023-12-27 08:18:53,619][20381] Updated weights for policy 1, policy_version 386416 (0.0007) [2023-12-27 08:18:53,736][20347] Updated weights for policy 0, policy_version 386530 (0.0009) [2023-12-27 08:18:53,783][20347] Updated weights for policy 0, policy_version 386540 (0.0008) [2023-12-27 08:18:53,835][20347] Updated weights for policy 0, policy_version 386550 (0.0007) [2023-12-27 08:18:53,886][20347] Updated weights for policy 0, policy_version 386560 (0.0010) [2023-12-27 08:18:54,296][20381] Updated weights for policy 1, policy_version 386426 (0.0008) [2023-12-27 08:18:54,358][20381] Updated weights for policy 1, policy_version 386436 (0.0008) [2023-12-27 08:18:54,415][20381] Updated weights for policy 1, policy_version 386446 (0.0007) [2023-12-27 08:18:54,557][20347] Updated weights for policy 0, policy_version 386570 (0.0008) [2023-12-27 08:18:54,618][20347] Updated weights for policy 0, policy_version 386580 (0.0008) [2023-12-27 08:18:54,672][20347] Updated weights for policy 0, policy_version 386590 (0.0008) [2023-12-27 08:18:55,090][20381] Updated weights for policy 1, policy_version 386456 (0.0007) [2023-12-27 08:18:55,147][20381] Updated weights for policy 1, policy_version 386466 (0.0007) [2023-12-27 08:18:55,198][20381] Updated weights for policy 1, policy_version 386476 (0.0011) [2023-12-27 08:18:55,334][20347] Updated weights for policy 0, policy_version 386600 (0.0010) [2023-12-27 08:18:55,391][20347] Updated weights for policy 0, policy_version 386610 (0.0005) [2023-12-27 08:18:55,452][20347] Updated weights for policy 0, policy_version 386620 (0.0005) [2023-12-27 08:18:55,853][20381] Updated weights for policy 1, policy_version 386486 (0.0008) [2023-12-27 08:18:55,912][20381] Updated weights for policy 1, policy_version 386496 (0.0005) [2023-12-27 08:18:55,967][20381] Updated weights for policy 1, policy_version 386506 (0.0005) [2023-12-27 08:18:56,091][20347] Updated weights for policy 0, policy_version 386630 (0.0008) [2023-12-27 08:18:56,154][20347] Updated weights for policy 0, policy_version 386640 (0.0008) [2023-12-27 08:18:56,216][20347] Updated weights for policy 0, policy_version 386650 (0.0008) [2023-12-27 08:18:56,562][20381] Updated weights for policy 1, policy_version 386516 (0.0007) [2023-12-27 08:18:56,617][20381] Updated weights for policy 1, policy_version 386526 (0.0010) [2023-12-27 08:18:56,662][20381] Updated weights for policy 1, policy_version 386536 (0.0010) [2023-12-27 08:18:56,842][20347] Updated weights for policy 0, policy_version 386660 (0.0007) [2023-12-27 08:18:56,897][20347] Updated weights for policy 0, policy_version 386670 (0.0010) [2023-12-27 08:18:56,948][20347] Updated weights for policy 0, policy_version 386680 (0.0010) [2023-12-27 08:18:57,236][20381] Updated weights for policy 1, policy_version 386546 (0.0010) [2023-12-27 08:18:57,295][20381] Updated weights for policy 1, policy_version 386556 (0.0010) [2023-12-27 08:18:57,359][20381] Updated weights for policy 1, policy_version 386566 (0.0011) [2023-12-27 08:18:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 197984256. Throughput: 0: 10390.0, 1: 10371.3. Samples: 197975260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:18:57,363][19170] Avg episode reward: [(0, '980.620'), (1, '990.330')] [2023-12-27 08:18:57,416][20381] Updated weights for policy 1, policy_version 386576 (0.0010) [2023-12-27 08:18:57,626][20347] Updated weights for policy 0, policy_version 386690 (0.0010) [2023-12-27 08:18:57,679][20347] Updated weights for policy 0, policy_version 386700 (0.0010) [2023-12-27 08:18:57,733][20347] Updated weights for policy 0, policy_version 386710 (0.0008) [2023-12-27 08:18:57,780][20347] Updated weights for policy 0, policy_version 386720 (0.0010) [2023-12-27 08:18:58,049][20381] Updated weights for policy 1, policy_version 386586 (0.0010) [2023-12-27 08:18:58,105][20381] Updated weights for policy 1, policy_version 386596 (0.0009) [2023-12-27 08:18:58,163][20381] Updated weights for policy 1, policy_version 386606 (0.0008) [2023-12-27 08:18:58,484][20347] Updated weights for policy 0, policy_version 386730 (0.0011) [2023-12-27 08:18:58,546][20347] Updated weights for policy 0, policy_version 386740 (0.0010) [2023-12-27 08:18:58,606][20347] Updated weights for policy 0, policy_version 386750 (0.0008) [2023-12-27 08:18:58,897][20381] Updated weights for policy 1, policy_version 386616 (0.0008) [2023-12-27 08:18:58,952][20381] Updated weights for policy 1, policy_version 386626 (0.0007) [2023-12-27 08:18:59,008][20381] Updated weights for policy 1, policy_version 386636 (0.0007) [2023-12-27 08:18:59,348][20347] Updated weights for policy 0, policy_version 386760 (0.0007) [2023-12-27 08:18:59,405][20347] Updated weights for policy 0, policy_version 386770 (0.0006) [2023-12-27 08:18:59,460][20347] Updated weights for policy 0, policy_version 386780 (0.0008) [2023-12-27 08:18:59,737][20381] Updated weights for policy 1, policy_version 386646 (0.0007) [2023-12-27 08:18:59,792][20381] Updated weights for policy 1, policy_version 386656 (0.0008) [2023-12-27 08:18:59,852][20381] Updated weights for policy 1, policy_version 386666 (0.0007) [2023-12-27 08:19:00,102][20347] Updated weights for policy 0, policy_version 386790 (0.0010) [2023-12-27 08:19:00,163][20347] Updated weights for policy 0, policy_version 386800 (0.0010) [2023-12-27 08:19:00,219][20347] Updated weights for policy 0, policy_version 386810 (0.0006) [2023-12-27 08:19:00,603][20381] Updated weights for policy 1, policy_version 386676 (0.0009) [2023-12-27 08:19:00,655][20381] Updated weights for policy 1, policy_version 386686 (0.0007) [2023-12-27 08:19:00,704][20381] Updated weights for policy 1, policy_version 386696 (0.0009) [2023-12-27 08:19:00,754][20347] Updated weights for policy 0, policy_version 386820 (0.0006) [2023-12-27 08:19:00,807][20347] Updated weights for policy 0, policy_version 386830 (0.0006) [2023-12-27 08:19:00,857][20347] Updated weights for policy 0, policy_version 386840 (0.0005) [2023-12-27 08:19:01,444][20381] Updated weights for policy 1, policy_version 386706 (0.0007) [2023-12-27 08:19:01,502][20381] Updated weights for policy 1, policy_version 386716 (0.0005) [2023-12-27 08:19:01,506][20292] KL-divergence is very high: 201.0726 [2023-12-27 08:19:01,526][20347] Updated weights for policy 0, policy_version 386850 (0.0008) [2023-12-27 08:19:01,528][20292] KL-divergence is very high: 130.2425 [2023-12-27 08:19:01,550][20381] Updated weights for policy 1, policy_version 386726 (0.0005) [2023-12-27 08:19:01,580][20347] Updated weights for policy 0, policy_version 386860 (0.0010) [2023-12-27 08:19:01,604][20381] Updated weights for policy 1, policy_version 386736 (0.0005) [2023-12-27 08:19:01,637][20347] Updated weights for policy 0, policy_version 386870 (0.0010) [2023-12-27 08:19:01,698][20347] Updated weights for policy 0, policy_version 386880 (0.0010) [2023-12-27 08:19:02,279][20381] Updated weights for policy 1, policy_version 386746 (0.0007) [2023-12-27 08:19:02,339][20381] Updated weights for policy 1, policy_version 386756 (0.0006) [2023-12-27 08:19:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 198082560. Throughput: 0: 10387.3, 1: 10354.8. Samples: 198099068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:02,363][19170] Avg episode reward: [(0, '952.280'), (1, '980.710')] [2023-12-27 08:19:02,393][20347] Updated weights for policy 0, policy_version 386890 (0.0010) [2023-12-27 08:19:02,403][20381] Updated weights for policy 1, policy_version 386766 (0.0007) [2023-12-27 08:19:02,447][20347] Updated weights for policy 0, policy_version 386900 (0.0010) [2023-12-27 08:19:02,502][20347] Updated weights for policy 0, policy_version 386910 (0.0010) [2023-12-27 08:19:03,054][20381] Updated weights for policy 1, policy_version 386776 (0.0007) [2023-12-27 08:19:03,099][20381] Updated weights for policy 1, policy_version 386786 (0.0007) [2023-12-27 08:19:03,130][20347] Updated weights for policy 0, policy_version 386920 (0.0010) [2023-12-27 08:19:03,143][20381] Updated weights for policy 1, policy_version 386796 (0.0005) [2023-12-27 08:19:03,182][20347] Updated weights for policy 0, policy_version 386930 (0.0009) [2023-12-27 08:19:03,240][20347] Updated weights for policy 0, policy_version 386940 (0.0008) [2023-12-27 08:19:03,874][20381] Updated weights for policy 1, policy_version 386806 (0.0008) [2023-12-27 08:19:03,875][20347] Updated weights for policy 0, policy_version 386950 (0.0010) [2023-12-27 08:19:03,927][20381] Updated weights for policy 1, policy_version 386816 (0.0006) [2023-12-27 08:19:03,928][20347] Updated weights for policy 0, policy_version 386960 (0.0009) [2023-12-27 08:19:03,989][20381] Updated weights for policy 1, policy_version 386826 (0.0006) [2023-12-27 08:19:03,991][20347] Updated weights for policy 0, policy_version 386970 (0.0010) [2023-12-27 08:19:04,608][20347] Updated weights for policy 0, policy_version 386980 (0.0008) [2023-12-27 08:19:04,661][20347] Updated weights for policy 0, policy_version 386990 (0.0005) [2023-12-27 08:19:04,716][20347] Updated weights for policy 0, policy_version 387000 (0.0006) [2023-12-27 08:19:04,720][20381] Updated weights for policy 1, policy_version 386836 (0.0007) [2023-12-27 08:19:04,767][20381] Updated weights for policy 1, policy_version 386846 (0.0007) [2023-12-27 08:19:04,823][20381] Updated weights for policy 1, policy_version 386856 (0.0007) [2023-12-27 08:19:05,300][20347] Updated weights for policy 0, policy_version 387010 (0.0006) [2023-12-27 08:19:05,344][20347] Updated weights for policy 0, policy_version 387020 (0.0010) [2023-12-27 08:19:05,389][20347] Updated weights for policy 0, policy_version 387030 (0.0010) [2023-12-27 08:19:05,436][20347] Updated weights for policy 0, policy_version 387040 (0.0010) [2023-12-27 08:19:05,549][20381] Updated weights for policy 1, policy_version 386866 (0.0008) [2023-12-27 08:19:05,600][20381] Updated weights for policy 1, policy_version 386876 (0.0007) [2023-12-27 08:19:05,648][20381] Updated weights for policy 1, policy_version 386886 (0.0007) [2023-12-27 08:19:05,652][20292] KL-divergence is very high: 136.8806 [2023-12-27 08:19:05,696][20381] Updated weights for policy 1, policy_version 386896 (0.0005) [2023-12-27 08:19:06,146][20347] Updated weights for policy 0, policy_version 387050 (0.0010) [2023-12-27 08:19:06,199][20347] Updated weights for policy 0, policy_version 387060 (0.0010) [2023-12-27 08:19:06,255][20347] Updated weights for policy 0, policy_version 387070 (0.0010) [2023-12-27 08:19:06,359][20381] Updated weights for policy 1, policy_version 386906 (0.0008) [2023-12-27 08:19:06,415][20381] Updated weights for policy 1, policy_version 386916 (0.0008) [2023-12-27 08:19:06,468][20381] Updated weights for policy 1, policy_version 386926 (0.0007) [2023-12-27 08:19:06,929][20347] Updated weights for policy 0, policy_version 387080 (0.0009) [2023-12-27 08:19:06,986][20347] Updated weights for policy 0, policy_version 387090 (0.0008) [2023-12-27 08:19:07,046][20347] Updated weights for policy 0, policy_version 387100 (0.0007) [2023-12-27 08:19:07,204][20381] Updated weights for policy 1, policy_version 386936 (0.0006) [2023-12-27 08:19:07,258][20381] Updated weights for policy 1, policy_version 386946 (0.0005) [2023-12-27 08:19:07,322][20381] Updated weights for policy 1, policy_version 386956 (0.0005) [2023-12-27 08:19:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 198197248. Throughput: 0: 10440.6, 1: 10332.1. Samples: 198161408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:07,363][19170] Avg episode reward: [(0, '893.090'), (1, '971.250')] [2023-12-27 08:19:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000386960_99082240.pth... [2023-12-27 08:19:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000387104_99115008.pth... [2023-12-27 08:19:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000385744_98770944.pth [2023-12-27 08:19:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000385888_98803712.pth [2023-12-27 08:19:07,766][20347] Updated weights for policy 0, policy_version 387110 (0.0007) [2023-12-27 08:19:07,817][20347] Updated weights for policy 0, policy_version 387120 (0.0007) [2023-12-27 08:19:07,874][20347] Updated weights for policy 0, policy_version 387130 (0.0006) [2023-12-27 08:19:07,876][20381] Updated weights for policy 1, policy_version 386967 (0.0008) [2023-12-27 08:19:07,924][20381] Updated weights for policy 1, policy_version 386977 (0.0005) [2023-12-27 08:19:07,976][20381] Updated weights for policy 1, policy_version 386987 (0.0005) [2023-12-27 08:19:08,527][20347] Updated weights for policy 0, policy_version 387140 (0.0008) [2023-12-27 08:19:08,586][20347] Updated weights for policy 0, policy_version 387150 (0.0008) [2023-12-27 08:19:08,647][20347] Updated weights for policy 0, policy_version 387160 (0.0007) [2023-12-27 08:19:08,723][20381] Updated weights for policy 1, policy_version 386997 (0.0008) [2023-12-27 08:19:08,778][20381] Updated weights for policy 1, policy_version 387007 (0.0008) [2023-12-27 08:19:08,834][20381] Updated weights for policy 1, policy_version 387017 (0.0007) [2023-12-27 08:19:09,325][20347] Updated weights for policy 0, policy_version 387170 (0.0010) [2023-12-27 08:19:09,389][20347] Updated weights for policy 0, policy_version 387180 (0.0011) [2023-12-27 08:19:09,457][20347] Updated weights for policy 0, policy_version 387190 (0.0011) [2023-12-27 08:19:09,505][20381] Updated weights for policy 1, policy_version 387027 (0.0007) [2023-12-27 08:19:09,520][20347] Updated weights for policy 0, policy_version 387200 (0.0009) [2023-12-27 08:19:09,569][20381] Updated weights for policy 1, policy_version 387037 (0.0006) [2023-12-27 08:19:09,628][20381] Updated weights for policy 1, policy_version 387047 (0.0008) [2023-12-27 08:19:10,200][20347] Updated weights for policy 0, policy_version 387210 (0.0009) [2023-12-27 08:19:10,262][20347] Updated weights for policy 0, policy_version 387220 (0.0007) [2023-12-27 08:19:10,335][20347] Updated weights for policy 0, policy_version 387230 (0.0006) [2023-12-27 08:19:10,338][20381] Updated weights for policy 1, policy_version 387057 (0.0008) [2023-12-27 08:19:10,395][20381] Updated weights for policy 1, policy_version 387067 (0.0008) [2023-12-27 08:19:10,447][20381] Updated weights for policy 1, policy_version 387077 (0.0008) [2023-12-27 08:19:10,503][20381] Updated weights for policy 1, policy_version 387087 (0.0007) [2023-12-27 08:19:10,965][20347] Updated weights for policy 0, policy_version 387240 (0.0008) [2023-12-27 08:19:11,033][20347] Updated weights for policy 0, policy_version 387250 (0.0007) [2023-12-27 08:19:11,099][20347] Updated weights for policy 0, policy_version 387260 (0.0007) [2023-12-27 08:19:11,117][20381] Updated weights for policy 1, policy_version 387097 (0.0008) [2023-12-27 08:19:11,180][20381] Updated weights for policy 1, policy_version 387107 (0.0008) [2023-12-27 08:19:11,236][20381] Updated weights for policy 1, policy_version 387117 (0.0008) [2023-12-27 08:19:11,785][20347] Updated weights for policy 0, policy_version 387270 (0.0008) [2023-12-27 08:19:11,844][20347] Updated weights for policy 0, policy_version 387280 (0.0008) [2023-12-27 08:19:11,904][20347] Updated weights for policy 0, policy_version 387290 (0.0007) [2023-12-27 08:19:11,982][20381] Updated weights for policy 1, policy_version 387127 (0.0007) [2023-12-27 08:19:12,035][20381] Updated weights for policy 1, policy_version 387137 (0.0008) [2023-12-27 08:19:12,089][20381] Updated weights for policy 1, policy_version 387147 (0.0008) [2023-12-27 08:19:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 198295552. Throughput: 0: 10522.2, 1: 10326.2. Samples: 198284772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:12,363][19170] Avg episode reward: [(0, '963.400'), (1, '980.870')] [2023-12-27 08:19:12,608][20347] Updated weights for policy 0, policy_version 387300 (0.0008) [2023-12-27 08:19:12,666][20347] Updated weights for policy 0, policy_version 387310 (0.0008) [2023-12-27 08:19:12,726][20347] Updated weights for policy 0, policy_version 387320 (0.0006) [2023-12-27 08:19:12,755][20381] Updated weights for policy 1, policy_version 387157 (0.0009) [2023-12-27 08:19:12,813][20381] Updated weights for policy 1, policy_version 387167 (0.0011) [2023-12-27 08:19:12,873][20381] Updated weights for policy 1, policy_version 387177 (0.0010) [2023-12-27 08:19:13,400][20347] Updated weights for policy 0, policy_version 387330 (0.0009) [2023-12-27 08:19:13,440][20381] Updated weights for policy 1, policy_version 387187 (0.0011) [2023-12-27 08:19:13,460][20347] Updated weights for policy 0, policy_version 387340 (0.0010) [2023-12-27 08:19:13,487][20381] Updated weights for policy 1, policy_version 387197 (0.0009) [2023-12-27 08:19:13,509][20347] Updated weights for policy 0, policy_version 387350 (0.0010) [2023-12-27 08:19:13,539][20381] Updated weights for policy 1, policy_version 387207 (0.0007) [2023-12-27 08:19:13,561][20347] Updated weights for policy 0, policy_version 387360 (0.0010) [2023-12-27 08:19:14,073][20381] Updated weights for policy 1, policy_version 387217 (0.0008) [2023-12-27 08:19:14,125][20381] Updated weights for policy 1, policy_version 387227 (0.0006) [2023-12-27 08:19:14,178][20381] Updated weights for policy 1, policy_version 387237 (0.0005) [2023-12-27 08:19:14,235][20381] Updated weights for policy 1, policy_version 387247 (0.0006) [2023-12-27 08:19:14,250][20347] Updated weights for policy 0, policy_version 387370 (0.0008) [2023-12-27 08:19:14,297][20347] Updated weights for policy 0, policy_version 387380 (0.0007) [2023-12-27 08:19:14,348][20347] Updated weights for policy 0, policy_version 387390 (0.0007) [2023-12-27 08:19:14,915][20381] Updated weights for policy 1, policy_version 387257 (0.0010) [2023-12-27 08:19:14,977][20381] Updated weights for policy 1, policy_version 387267 (0.0008) [2023-12-27 08:19:15,033][20381] Updated weights for policy 1, policy_version 387277 (0.0006) [2023-12-27 08:19:15,052][20347] Updated weights for policy 0, policy_version 387400 (0.0010) [2023-12-27 08:19:15,106][20347] Updated weights for policy 0, policy_version 387410 (0.0010) [2023-12-27 08:19:15,163][20347] Updated weights for policy 0, policy_version 387420 (0.0011) [2023-12-27 08:19:15,675][20381] Updated weights for policy 1, policy_version 387287 (0.0005) [2023-12-27 08:19:15,729][20381] Updated weights for policy 1, policy_version 387297 (0.0005) [2023-12-27 08:19:15,780][20381] Updated weights for policy 1, policy_version 387307 (0.0005) [2023-12-27 08:19:15,826][20347] Updated weights for policy 0, policy_version 387430 (0.0008) [2023-12-27 08:19:15,881][20347] Updated weights for policy 0, policy_version 387440 (0.0008) [2023-12-27 08:19:15,929][20347] Updated weights for policy 0, policy_version 387450 (0.0008) [2023-12-27 08:19:16,401][20381] Updated weights for policy 1, policy_version 387317 (0.0006) [2023-12-27 08:19:16,459][20381] Updated weights for policy 1, policy_version 387327 (0.0008) [2023-12-27 08:19:16,518][20381] Updated weights for policy 1, policy_version 387337 (0.0009) [2023-12-27 08:19:16,592][20347] Updated weights for policy 0, policy_version 387460 (0.0007) [2023-12-27 08:19:16,645][20347] Updated weights for policy 0, policy_version 387470 (0.0008) [2023-12-27 08:19:16,690][20347] Updated weights for policy 0, policy_version 387480 (0.0009) [2023-12-27 08:19:17,179][20381] Updated weights for policy 1, policy_version 387347 (0.0008) [2023-12-27 08:19:17,234][20381] Updated weights for policy 1, policy_version 387357 (0.0007) [2023-12-27 08:19:17,293][20381] Updated weights for policy 1, policy_version 387367 (0.0008) [2023-12-27 08:19:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 198402048. Throughput: 0: 10340.4, 1: 10361.6. Samples: 198410228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:17,363][19170] Avg episode reward: [(0, '971.130'), (1, '1000.000')] [2023-12-27 08:19:17,372][20347] Updated weights for policy 0, policy_version 387490 (0.0008) [2023-12-27 08:19:17,424][20347] Updated weights for policy 0, policy_version 387500 (0.0007) [2023-12-27 08:19:17,485][20347] Updated weights for policy 0, policy_version 387510 (0.0005) [2023-12-27 08:19:17,541][20347] Updated weights for policy 0, policy_version 387520 (0.0008) [2023-12-27 08:19:17,970][20381] Updated weights for policy 1, policy_version 387377 (0.0008) [2023-12-27 08:19:18,033][20381] Updated weights for policy 1, policy_version 387387 (0.0008) [2023-12-27 08:19:18,095][20381] Updated weights for policy 1, policy_version 387397 (0.0008) [2023-12-27 08:19:18,123][20347] Updated weights for policy 0, policy_version 387530 (0.0009) [2023-12-27 08:19:18,148][20381] Updated weights for policy 1, policy_version 387407 (0.0006) [2023-12-27 08:19:18,176][20347] Updated weights for policy 0, policy_version 387540 (0.0011) [2023-12-27 08:19:18,229][20347] Updated weights for policy 0, policy_version 387550 (0.0007) [2023-12-27 08:19:18,775][20381] Updated weights for policy 1, policy_version 387417 (0.0007) [2023-12-27 08:19:18,823][20381] Updated weights for policy 1, policy_version 387427 (0.0007) [2023-12-27 08:19:18,861][20291] KL-divergence is very high: 251.4844 [2023-12-27 08:19:18,877][20381] Updated weights for policy 1, policy_version 387437 (0.0009) [2023-12-27 08:19:18,881][20347] Updated weights for policy 0, policy_version 387560 (0.0010) [2023-12-27 08:19:18,902][20291] KL-divergence is very high: 274.3757 [2023-12-27 08:19:18,930][20347] Updated weights for policy 0, policy_version 387570 (0.0010) [2023-12-27 08:19:18,938][20291] KL-divergence is very high: 194.4135 [2023-12-27 08:19:18,975][20291] KL-divergence is very high: 122.4799 [2023-12-27 08:19:18,975][20347] Updated weights for policy 0, policy_version 387580 (0.0010) [2023-12-27 08:19:19,609][20381] Updated weights for policy 1, policy_version 387447 (0.0007) [2023-12-27 08:19:19,671][20381] Updated weights for policy 1, policy_version 387457 (0.0008) [2023-12-27 08:19:19,684][20347] Updated weights for policy 0, policy_version 387590 (0.0010) [2023-12-27 08:19:19,726][20381] Updated weights for policy 1, policy_version 387467 (0.0010) [2023-12-27 08:19:19,747][20347] Updated weights for policy 0, policy_version 387600 (0.0011) [2023-12-27 08:19:19,808][20347] Updated weights for policy 0, policy_version 387610 (0.0010) [2023-12-27 08:19:20,480][20381] Updated weights for policy 1, policy_version 387477 (0.0009) [2023-12-27 08:19:20,489][20347] Updated weights for policy 0, policy_version 387620 (0.0010) [2023-12-27 08:19:20,538][20381] Updated weights for policy 1, policy_version 387487 (0.0005) [2023-12-27 08:19:20,552][20347] Updated weights for policy 0, policy_version 387630 (0.0010) [2023-12-27 08:19:20,598][20381] Updated weights for policy 1, policy_version 387497 (0.0008) [2023-12-27 08:19:20,614][20347] Updated weights for policy 0, policy_version 387640 (0.0007) [2023-12-27 08:19:21,311][20347] Updated weights for policy 0, policy_version 387650 (0.0006) [2023-12-27 08:19:21,330][20381] Updated weights for policy 1, policy_version 387507 (0.0007) [2023-12-27 08:19:21,375][20347] Updated weights for policy 0, policy_version 387660 (0.0009) [2023-12-27 08:19:21,398][20381] Updated weights for policy 1, policy_version 387517 (0.0008) [2023-12-27 08:19:21,436][20347] Updated weights for policy 0, policy_version 387670 (0.0008) [2023-12-27 08:19:21,464][20381] Updated weights for policy 1, policy_version 387527 (0.0008) [2023-12-27 08:19:21,497][20347] Updated weights for policy 0, policy_version 387680 (0.0006) [2023-12-27 08:19:22,149][20381] Updated weights for policy 1, policy_version 387537 (0.0007) [2023-12-27 08:19:22,204][20381] Updated weights for policy 1, policy_version 387547 (0.0007) [2023-12-27 08:19:22,232][20347] Updated weights for policy 0, policy_version 387690 (0.0007) [2023-12-27 08:19:22,263][20381] Updated weights for policy 1, policy_version 387557 (0.0006) [2023-12-27 08:19:22,288][20347] Updated weights for policy 0, policy_version 387700 (0.0007) [2023-12-27 08:19:22,324][20381] Updated weights for policy 1, policy_version 387567 (0.0009) [2023-12-27 08:19:22,354][20347] Updated weights for policy 0, policy_version 387710 (0.0007) [2023-12-27 08:19:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 198500352. Throughput: 0: 10376.9, 1: 10301.4. Samples: 198472312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:22,363][19170] Avg episode reward: [(0, '970.940'), (1, '1000.000')] [2023-12-27 08:19:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000387712_99270656.pth... [2023-12-27 08:19:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000387568_99237888.pth... [2023-12-27 08:19:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000386496_98959360.pth [2023-12-27 08:19:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000386352_98926592.pth [2023-12-27 08:19:23,040][20381] Updated weights for policy 1, policy_version 387577 (0.0007) [2023-12-27 08:19:23,076][20347] Updated weights for policy 0, policy_version 387720 (0.0007) [2023-12-27 08:19:23,106][20381] Updated weights for policy 1, policy_version 387587 (0.0007) [2023-12-27 08:19:23,138][20347] Updated weights for policy 0, policy_version 387730 (0.0006) [2023-12-27 08:19:23,161][20381] Updated weights for policy 1, policy_version 387597 (0.0007) [2023-12-27 08:19:23,195][20347] Updated weights for policy 0, policy_version 387740 (0.0006) [2023-12-27 08:19:23,766][20381] Updated weights for policy 1, policy_version 387607 (0.0005) [2023-12-27 08:19:23,823][20381] Updated weights for policy 1, policy_version 387617 (0.0005) [2023-12-27 08:19:23,868][20381] Updated weights for policy 1, policy_version 387627 (0.0007) [2023-12-27 08:19:23,921][20347] Updated weights for policy 0, policy_version 387750 (0.0008) [2023-12-27 08:19:23,978][20347] Updated weights for policy 0, policy_version 387760 (0.0008) [2023-12-27 08:19:24,026][20347] Updated weights for policy 0, policy_version 387770 (0.0007) [2023-12-27 08:19:24,423][20381] Updated weights for policy 1, policy_version 387637 (0.0007) [2023-12-27 08:19:24,478][20381] Updated weights for policy 1, policy_version 387647 (0.0008) [2023-12-27 08:19:24,528][20381] Updated weights for policy 1, policy_version 387657 (0.0008) [2023-12-27 08:19:24,712][20347] Updated weights for policy 0, policy_version 387780 (0.0008) [2023-12-27 08:19:24,757][20347] Updated weights for policy 0, policy_version 387790 (0.0007) [2023-12-27 08:19:24,805][20347] Updated weights for policy 0, policy_version 387800 (0.0007) [2023-12-27 08:19:25,223][20381] Updated weights for policy 1, policy_version 387667 (0.0007) [2023-12-27 08:19:25,279][20381] Updated weights for policy 1, policy_version 387677 (0.0006) [2023-12-27 08:19:25,332][20381] Updated weights for policy 1, policy_version 387687 (0.0007) [2023-12-27 08:19:25,463][20347] Updated weights for policy 0, policy_version 387810 (0.0008) [2023-12-27 08:19:25,518][20347] Updated weights for policy 0, policy_version 387820 (0.0010) [2023-12-27 08:19:25,575][20347] Updated weights for policy 0, policy_version 387830 (0.0010) [2023-12-27 08:19:25,630][20347] Updated weights for policy 0, policy_version 387840 (0.0007) [2023-12-27 08:19:25,954][20381] Updated weights for policy 1, policy_version 387697 (0.0007) [2023-12-27 08:19:26,011][20381] Updated weights for policy 1, policy_version 387707 (0.0008) [2023-12-27 08:19:26,066][20381] Updated weights for policy 1, policy_version 387717 (0.0007) [2023-12-27 08:19:26,119][20381] Updated weights for policy 1, policy_version 387727 (0.0008) [2023-12-27 08:19:26,307][20347] Updated weights for policy 0, policy_version 387850 (0.0010) [2023-12-27 08:19:26,368][20347] Updated weights for policy 0, policy_version 387860 (0.0010) [2023-12-27 08:19:26,430][20347] Updated weights for policy 0, policy_version 387870 (0.0010) [2023-12-27 08:19:26,756][20381] Updated weights for policy 1, policy_version 387737 (0.0009) [2023-12-27 08:19:26,810][20381] Updated weights for policy 1, policy_version 387747 (0.0007) [2023-12-27 08:19:26,863][20381] Updated weights for policy 1, policy_version 387757 (0.0005) [2023-12-27 08:19:27,018][20347] Updated weights for policy 0, policy_version 387880 (0.0006) [2023-12-27 08:19:27,081][20347] Updated weights for policy 0, policy_version 387890 (0.0005) [2023-12-27 08:19:27,139][20347] Updated weights for policy 0, policy_version 387900 (0.0005) [2023-12-27 08:19:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 198606848. Throughput: 0: 10327.1, 1: 10339.2. Samples: 198594848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:27,363][19170] Avg episode reward: [(0, '980.600'), (1, '1000.000')] [2023-12-27 08:19:27,539][20381] Updated weights for policy 1, policy_version 387767 (0.0007) [2023-12-27 08:19:27,595][20381] Updated weights for policy 1, policy_version 387777 (0.0007) [2023-12-27 08:19:27,650][20381] Updated weights for policy 1, policy_version 387787 (0.0010) [2023-12-27 08:19:27,718][20347] Updated weights for policy 0, policy_version 387910 (0.0008) [2023-12-27 08:19:27,765][20347] Updated weights for policy 0, policy_version 387920 (0.0010) [2023-12-27 08:19:27,824][20347] Updated weights for policy 0, policy_version 387930 (0.0006) [2023-12-27 08:19:28,276][20381] Updated weights for policy 1, policy_version 387797 (0.0008) [2023-12-27 08:19:28,340][20381] Updated weights for policy 1, policy_version 387807 (0.0006) [2023-12-27 08:19:28,397][20381] Updated weights for policy 1, policy_version 387817 (0.0006) [2023-12-27 08:19:28,445][20347] Updated weights for policy 0, policy_version 387940 (0.0008) [2023-12-27 08:19:28,497][20347] Updated weights for policy 0, policy_version 387950 (0.0010) [2023-12-27 08:19:28,548][20347] Updated weights for policy 0, policy_version 387960 (0.0010) [2023-12-27 08:19:28,948][20381] Updated weights for policy 1, policy_version 387827 (0.0010) [2023-12-27 08:19:28,999][20381] Updated weights for policy 1, policy_version 387837 (0.0010) [2023-12-27 08:19:29,047][20381] Updated weights for policy 1, policy_version 387847 (0.0010) [2023-12-27 08:19:29,194][20347] Updated weights for policy 0, policy_version 387970 (0.0010) [2023-12-27 08:19:29,253][20347] Updated weights for policy 0, policy_version 387980 (0.0009) [2023-12-27 08:19:29,312][20347] Updated weights for policy 0, policy_version 387990 (0.0008) [2023-12-27 08:19:29,374][20347] Updated weights for policy 0, policy_version 388000 (0.0008) [2023-12-27 08:19:29,739][20381] Updated weights for policy 1, policy_version 387857 (0.0010) [2023-12-27 08:19:29,793][20381] Updated weights for policy 1, policy_version 387867 (0.0010) [2023-12-27 08:19:29,845][20381] Updated weights for policy 1, policy_version 387877 (0.0010) [2023-12-27 08:19:29,905][20381] Updated weights for policy 1, policy_version 387887 (0.0010) [2023-12-27 08:19:30,079][20347] Updated weights for policy 0, policy_version 388010 (0.0010) [2023-12-27 08:19:30,143][20347] Updated weights for policy 0, policy_version 388020 (0.0010) [2023-12-27 08:19:30,207][20347] Updated weights for policy 0, policy_version 388030 (0.0010) [2023-12-27 08:19:30,581][20381] Updated weights for policy 1, policy_version 387897 (0.0010) [2023-12-27 08:19:30,627][20381] Updated weights for policy 1, policy_version 387907 (0.0010) [2023-12-27 08:19:30,671][20381] Updated weights for policy 1, policy_version 387917 (0.0010) [2023-12-27 08:19:30,821][20347] Updated weights for policy 0, policy_version 388040 (0.0010) [2023-12-27 08:19:30,869][20347] Updated weights for policy 0, policy_version 388050 (0.0010) [2023-12-27 08:19:30,924][20347] Updated weights for policy 0, policy_version 388060 (0.0008) [2023-12-27 08:19:31,380][20381] Updated weights for policy 1, policy_version 387927 (0.0009) [2023-12-27 08:19:31,435][20381] Updated weights for policy 1, policy_version 387937 (0.0006) [2023-12-27 08:19:31,483][20381] Updated weights for policy 1, policy_version 387947 (0.0007) [2023-12-27 08:19:31,638][20347] Updated weights for policy 0, policy_version 388070 (0.0009) [2023-12-27 08:19:31,700][20347] Updated weights for policy 0, policy_version 388080 (0.0010) [2023-12-27 08:19:31,758][20347] Updated weights for policy 0, policy_version 388090 (0.0010) [2023-12-27 08:19:32,185][20381] Updated weights for policy 1, policy_version 387957 (0.0007) [2023-12-27 08:19:32,250][20381] Updated weights for policy 1, policy_version 387967 (0.0008) [2023-12-27 08:19:32,312][20381] Updated weights for policy 1, policy_version 387977 (0.0008) [2023-12-27 08:19:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 198713344. Throughput: 0: 10399.7, 1: 10362.8. Samples: 198722304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:19:32,363][19170] Avg episode reward: [(0, '981.090'), (1, '1000.000')] [2023-12-27 08:19:32,469][20347] Updated weights for policy 0, policy_version 388100 (0.0010) [2023-12-27 08:19:32,534][20347] Updated weights for policy 0, policy_version 388110 (0.0010) [2023-12-27 08:19:32,592][20347] Updated weights for policy 0, policy_version 388120 (0.0010) [2023-12-27 08:19:33,026][20381] Updated weights for policy 1, policy_version 387987 (0.0007) [2023-12-27 08:19:33,076][20381] Updated weights for policy 1, policy_version 387997 (0.0007) [2023-12-27 08:19:33,133][20381] Updated weights for policy 1, policy_version 388007 (0.0007) [2023-12-27 08:19:33,192][20347] Updated weights for policy 0, policy_version 388130 (0.0010) [2023-12-27 08:19:33,245][20347] Updated weights for policy 0, policy_version 388140 (0.0005) [2023-12-27 08:19:33,291][20347] Updated weights for policy 0, policy_version 388150 (0.0005) [2023-12-27 08:19:33,334][20347] Updated weights for policy 0, policy_version 388160 (0.0005) [2023-12-27 08:19:33,732][20381] Updated weights for policy 1, policy_version 388017 (0.0007) [2023-12-27 08:19:33,777][20381] Updated weights for policy 1, policy_version 388027 (0.0008) [2023-12-27 08:19:33,831][20381] Updated weights for policy 1, policy_version 388037 (0.0006) [2023-12-27 08:19:33,862][20347] Updated weights for policy 0, policy_version 388170 (0.0010) [2023-12-27 08:19:33,875][20381] Updated weights for policy 1, policy_version 388047 (0.0005) [2023-12-27 08:19:33,912][20347] Updated weights for policy 0, policy_version 388180 (0.0010) [2023-12-27 08:19:33,960][20347] Updated weights for policy 0, policy_version 388190 (0.0010) [2023-12-27 08:19:34,577][20347] Updated weights for policy 0, policy_version 388200 (0.0010) [2023-12-27 08:19:34,628][20381] Updated weights for policy 1, policy_version 388057 (0.0007) [2023-12-27 08:19:34,634][20347] Updated weights for policy 0, policy_version 388210 (0.0011) [2023-12-27 08:19:34,680][20381] Updated weights for policy 1, policy_version 388067 (0.0005) [2023-12-27 08:19:34,690][20347] Updated weights for policy 0, policy_version 388220 (0.0010) [2023-12-27 08:19:34,727][20381] Updated weights for policy 1, policy_version 388077 (0.0006) [2023-12-27 08:19:35,386][20347] Updated weights for policy 0, policy_version 388230 (0.0009) [2023-12-27 08:19:35,445][20381] Updated weights for policy 1, policy_version 388087 (0.0007) [2023-12-27 08:19:35,447][20347] Updated weights for policy 0, policy_version 388240 (0.0006) [2023-12-27 08:19:35,498][20381] Updated weights for policy 1, policy_version 388097 (0.0006) [2023-12-27 08:19:35,509][20347] Updated weights for policy 0, policy_version 388250 (0.0007) [2023-12-27 08:19:35,557][20381] Updated weights for policy 1, policy_version 388107 (0.0006) [2023-12-27 08:19:36,167][20347] Updated weights for policy 0, policy_version 388260 (0.0005) [2023-12-27 08:19:36,220][20347] Updated weights for policy 0, policy_version 388270 (0.0005) [2023-12-27 08:19:36,257][20381] Updated weights for policy 1, policy_version 388117 (0.0008) [2023-12-27 08:19:36,275][20347] Updated weights for policy 0, policy_version 388280 (0.0006) [2023-12-27 08:19:36,319][20381] Updated weights for policy 1, policy_version 388127 (0.0007) [2023-12-27 08:19:36,379][20381] Updated weights for policy 1, policy_version 388137 (0.0009) [2023-12-27 08:19:36,843][20347] Updated weights for policy 0, policy_version 388290 (0.0006) [2023-12-27 08:19:36,903][20347] Updated weights for policy 0, policy_version 388300 (0.0006) [2023-12-27 08:19:36,967][20347] Updated weights for policy 0, policy_version 388310 (0.0006) [2023-12-27 08:19:37,018][20347] Updated weights for policy 0, policy_version 388320 (0.0005) [2023-12-27 08:19:37,082][20381] Updated weights for policy 1, policy_version 388147 (0.0009) [2023-12-27 08:19:37,136][20381] Updated weights for policy 1, policy_version 388157 (0.0010) [2023-12-27 08:19:37,189][20381] Updated weights for policy 1, policy_version 388167 (0.0009) [2023-12-27 08:19:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 198819840. Throughput: 0: 10457.4, 1: 10373.8. Samples: 198786168. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:19:37,363][19170] Avg episode reward: [(0, '980.820'), (1, '1000.000')] [2023-12-27 08:19:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000388176_99393536.pth... [2023-12-27 08:19:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000388320_99426304.pth... [2023-12-27 08:19:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000386960_99082240.pth [2023-12-27 08:19:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000387104_99115008.pth [2023-12-27 08:19:37,604][20347] Updated weights for policy 0, policy_version 388330 (0.0011) [2023-12-27 08:19:37,663][20347] Updated weights for policy 0, policy_version 388340 (0.0010) [2023-12-27 08:19:37,725][20347] Updated weights for policy 0, policy_version 388350 (0.0007) [2023-12-27 08:19:37,907][20381] Updated weights for policy 1, policy_version 388177 (0.0008) [2023-12-27 08:19:37,971][20381] Updated weights for policy 1, policy_version 388187 (0.0008) [2023-12-27 08:19:38,026][20381] Updated weights for policy 1, policy_version 388197 (0.0008) [2023-12-27 08:19:38,082][20381] Updated weights for policy 1, policy_version 388207 (0.0007) [2023-12-27 08:19:38,399][20347] Updated weights for policy 0, policy_version 388360 (0.0008) [2023-12-27 08:19:38,459][20347] Updated weights for policy 0, policy_version 388370 (0.0007) [2023-12-27 08:19:38,507][20347] Updated weights for policy 0, policy_version 388380 (0.0006) [2023-12-27 08:19:38,786][20381] Updated weights for policy 1, policy_version 388217 (0.0006) [2023-12-27 08:19:38,846][20381] Updated weights for policy 1, policy_version 388227 (0.0005) [2023-12-27 08:19:38,907][20381] Updated weights for policy 1, policy_version 388237 (0.0007) [2023-12-27 08:19:39,188][20347] Updated weights for policy 0, policy_version 388390 (0.0006) [2023-12-27 08:19:39,243][20347] Updated weights for policy 0, policy_version 388400 (0.0007) [2023-12-27 08:19:39,296][20347] Updated weights for policy 0, policy_version 388410 (0.0006) [2023-12-27 08:19:39,570][20381] Updated weights for policy 1, policy_version 388247 (0.0010) [2023-12-27 08:19:39,624][20381] Updated weights for policy 1, policy_version 388257 (0.0011) [2023-12-27 08:19:39,680][20381] Updated weights for policy 1, policy_version 388267 (0.0010) [2023-12-27 08:19:39,990][20347] Updated weights for policy 0, policy_version 388420 (0.0007) [2023-12-27 08:19:40,052][20347] Updated weights for policy 0, policy_version 388430 (0.0010) [2023-12-27 08:19:40,113][20347] Updated weights for policy 0, policy_version 388440 (0.0010) [2023-12-27 08:19:40,408][20381] Updated weights for policy 1, policy_version 388277 (0.0010) [2023-12-27 08:19:40,467][20381] Updated weights for policy 1, policy_version 388287 (0.0007) [2023-12-27 08:19:40,533][20381] Updated weights for policy 1, policy_version 388297 (0.0007) [2023-12-27 08:19:40,779][20347] Updated weights for policy 0, policy_version 388450 (0.0009) [2023-12-27 08:19:40,836][20347] Updated weights for policy 0, policy_version 388460 (0.0005) [2023-12-27 08:19:40,897][20347] Updated weights for policy 0, policy_version 388470 (0.0005) [2023-12-27 08:19:40,958][20347] Updated weights for policy 0, policy_version 388480 (0.0009) [2023-12-27 08:19:41,208][20381] Updated weights for policy 1, policy_version 388307 (0.0008) [2023-12-27 08:19:41,273][20381] Updated weights for policy 1, policy_version 388317 (0.0008) [2023-12-27 08:19:41,335][20381] Updated weights for policy 1, policy_version 388327 (0.0007) [2023-12-27 08:19:41,596][20347] Updated weights for policy 0, policy_version 388490 (0.0006) [2023-12-27 08:19:41,656][20347] Updated weights for policy 0, policy_version 388500 (0.0010) [2023-12-27 08:19:41,716][20347] Updated weights for policy 0, policy_version 388510 (0.0008) [2023-12-27 08:19:42,118][20381] Updated weights for policy 1, policy_version 388337 (0.0008) [2023-12-27 08:19:42,175][20381] Updated weights for policy 1, policy_version 388347 (0.0009) [2023-12-27 08:19:42,227][20381] Updated weights for policy 1, policy_version 388357 (0.0008) [2023-12-27 08:19:42,289][20381] Updated weights for policy 1, policy_version 388367 (0.0008) [2023-12-27 08:19:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 198918144. Throughput: 0: 10453.5, 1: 10295.7. Samples: 198908976. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:19:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:19:42,372][20347] Updated weights for policy 0, policy_version 388520 (0.0008) [2023-12-27 08:19:42,435][20347] Updated weights for policy 0, policy_version 388530 (0.0011) [2023-12-27 08:19:42,497][20347] Updated weights for policy 0, policy_version 388540 (0.0010) [2023-12-27 08:19:43,033][20381] Updated weights for policy 1, policy_version 388377 (0.0008) [2023-12-27 08:19:43,087][20381] Updated weights for policy 1, policy_version 388387 (0.0008) [2023-12-27 08:19:43,114][20347] Updated weights for policy 0, policy_version 388550 (0.0006) [2023-12-27 08:19:43,149][20381] Updated weights for policy 1, policy_version 388397 (0.0007) [2023-12-27 08:19:43,158][20347] Updated weights for policy 0, policy_version 388560 (0.0006) [2023-12-27 08:19:43,210][20347] Updated weights for policy 0, policy_version 388570 (0.0007) [2023-12-27 08:19:43,773][20381] Updated weights for policy 1, policy_version 388407 (0.0006) [2023-12-27 08:19:43,821][20381] Updated weights for policy 1, policy_version 388417 (0.0008) [2023-12-27 08:19:43,875][20381] Updated weights for policy 1, policy_version 388427 (0.0005) [2023-12-27 08:19:43,921][20347] Updated weights for policy 0, policy_version 388580 (0.0008) [2023-12-27 08:19:43,980][20347] Updated weights for policy 0, policy_version 388590 (0.0008) [2023-12-27 08:19:44,023][20347] Updated weights for policy 0, policy_version 388600 (0.0008) [2023-12-27 08:19:44,456][20381] Updated weights for policy 1, policy_version 388437 (0.0007) [2023-12-27 08:19:44,513][20381] Updated weights for policy 1, policy_version 388447 (0.0007) [2023-12-27 08:19:44,566][20381] Updated weights for policy 1, policy_version 388457 (0.0007) [2023-12-27 08:19:44,716][20347] Updated weights for policy 0, policy_version 388610 (0.0007) [2023-12-27 08:19:44,776][20347] Updated weights for policy 0, policy_version 388620 (0.0007) [2023-12-27 08:19:44,828][20347] Updated weights for policy 0, policy_version 388630 (0.0008) [2023-12-27 08:19:44,890][20347] Updated weights for policy 0, policy_version 388640 (0.0007) [2023-12-27 08:19:45,270][20381] Updated weights for policy 1, policy_version 388467 (0.0009) [2023-12-27 08:19:45,330][20381] Updated weights for policy 1, policy_version 388477 (0.0007) [2023-12-27 08:19:45,382][20381] Updated weights for policy 1, policy_version 388487 (0.0008) [2023-12-27 08:19:45,566][20347] Updated weights for policy 0, policy_version 388650 (0.0007) [2023-12-27 08:19:45,618][20347] Updated weights for policy 0, policy_version 388660 (0.0008) [2023-12-27 08:19:45,672][20347] Updated weights for policy 0, policy_version 388670 (0.0007) [2023-12-27 08:19:46,075][20381] Updated weights for policy 1, policy_version 388497 (0.0007) [2023-12-27 08:19:46,130][20381] Updated weights for policy 1, policy_version 388507 (0.0006) [2023-12-27 08:19:46,175][20381] Updated weights for policy 1, policy_version 388517 (0.0008) [2023-12-27 08:19:46,230][20381] Updated weights for policy 1, policy_version 388527 (0.0008) [2023-12-27 08:19:46,353][20347] Updated weights for policy 0, policy_version 388680 (0.0006) [2023-12-27 08:19:46,401][20347] Updated weights for policy 0, policy_version 388690 (0.0005) [2023-12-27 08:19:46,452][20347] Updated weights for policy 0, policy_version 388700 (0.0005) [2023-12-27 08:19:46,959][20381] Updated weights for policy 1, policy_version 388537 (0.0007) [2023-12-27 08:19:47,013][20381] Updated weights for policy 1, policy_version 388547 (0.0007) [2023-12-27 08:19:47,075][20381] Updated weights for policy 1, policy_version 388557 (0.0006) [2023-12-27 08:19:47,077][20347] Updated weights for policy 0, policy_version 388710 (0.0008) [2023-12-27 08:19:47,129][20347] Updated weights for policy 0, policy_version 388720 (0.0007) [2023-12-27 08:19:47,184][20347] Updated weights for policy 0, policy_version 388730 (0.0005) [2023-12-27 08:19:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 199024640. Throughput: 0: 10445.8, 1: 10281.5. Samples: 199031800. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:19:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:19:47,745][20347] Updated weights for policy 0, policy_version 388740 (0.0007) [2023-12-27 08:19:47,805][20347] Updated weights for policy 0, policy_version 388750 (0.0010) [2023-12-27 08:19:47,822][20381] Updated weights for policy 1, policy_version 388567 (0.0006) [2023-12-27 08:19:47,861][20347] Updated weights for policy 0, policy_version 388760 (0.0010) [2023-12-27 08:19:47,878][20381] Updated weights for policy 1, policy_version 388577 (0.0006) [2023-12-27 08:19:47,942][20381] Updated weights for policy 1, policy_version 388587 (0.0007) [2023-12-27 08:19:48,536][20347] Updated weights for policy 0, policy_version 388770 (0.0007) [2023-12-27 08:19:48,594][20347] Updated weights for policy 0, policy_version 388780 (0.0007) [2023-12-27 08:19:48,644][20347] Updated weights for policy 0, policy_version 388790 (0.0007) [2023-12-27 08:19:48,649][20381] Updated weights for policy 1, policy_version 388597 (0.0007) [2023-12-27 08:19:48,699][20347] Updated weights for policy 0, policy_version 388800 (0.0007) [2023-12-27 08:19:48,709][20381] Updated weights for policy 1, policy_version 388607 (0.0008) [2023-12-27 08:19:48,770][20381] Updated weights for policy 1, policy_version 388617 (0.0008) [2023-12-27 08:19:49,315][20347] Updated weights for policy 0, policy_version 388810 (0.0007) [2023-12-27 08:19:49,383][20347] Updated weights for policy 0, policy_version 388820 (0.0007) [2023-12-27 08:19:49,437][20347] Updated weights for policy 0, policy_version 388830 (0.0008) [2023-12-27 08:19:49,530][20381] Updated weights for policy 1, policy_version 388627 (0.0008) [2023-12-27 08:19:49,586][20381] Updated weights for policy 1, policy_version 388637 (0.0006) [2023-12-27 08:19:49,644][20381] Updated weights for policy 1, policy_version 388647 (0.0008) [2023-12-27 08:19:50,152][20347] Updated weights for policy 0, policy_version 388840 (0.0007) [2023-12-27 08:19:50,207][20347] Updated weights for policy 0, policy_version 388850 (0.0007) [2023-12-27 08:19:50,269][20347] Updated weights for policy 0, policy_version 388860 (0.0007) [2023-12-27 08:19:50,280][20381] Updated weights for policy 1, policy_version 388657 (0.0008) [2023-12-27 08:19:50,344][20381] Updated weights for policy 1, policy_version 388667 (0.0011) [2023-12-27 08:19:50,402][20381] Updated weights for policy 1, policy_version 388677 (0.0011) [2023-12-27 08:19:50,458][20381] Updated weights for policy 1, policy_version 388687 (0.0010) [2023-12-27 08:19:50,971][20347] Updated weights for policy 0, policy_version 388870 (0.0007) [2023-12-27 08:19:51,027][20347] Updated weights for policy 0, policy_version 388880 (0.0008) [2023-12-27 08:19:51,087][20347] Updated weights for policy 0, policy_version 388890 (0.0007) [2023-12-27 08:19:51,147][20381] Updated weights for policy 1, policy_version 388697 (0.0011) [2023-12-27 08:19:51,206][20381] Updated weights for policy 1, policy_version 388707 (0.0008) [2023-12-27 08:19:51,269][20381] Updated weights for policy 1, policy_version 388717 (0.0008) [2023-12-27 08:19:51,804][20347] Updated weights for policy 0, policy_version 388900 (0.0006) [2023-12-27 08:19:51,850][20347] Updated weights for policy 0, policy_version 388910 (0.0010) [2023-12-27 08:19:51,903][20347] Updated weights for policy 0, policy_version 388920 (0.0010) [2023-12-27 08:19:51,909][20381] Updated weights for policy 1, policy_version 388727 (0.0007) [2023-12-27 08:19:51,970][20381] Updated weights for policy 1, policy_version 388737 (0.0006) [2023-12-27 08:19:52,030][20381] Updated weights for policy 1, policy_version 388747 (0.0008) [2023-12-27 08:19:52,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 199122944. Throughput: 0: 10429.9, 1: 10280.3. Samples: 199093372. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:19:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:19:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000388752_99540992.pth... [2023-12-27 08:19:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000388928_99581952.pth... [2023-12-27 08:19:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000387568_99237888.pth [2023-12-27 08:19:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000388752_99540992.pth [2023-12-27 08:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000387712_99270656.pth [2023-12-27 08:19:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000388928_99581952.pth [2023-12-27 08:19:52,657][20347] Updated weights for policy 0, policy_version 388930 (0.0010) [2023-12-27 08:19:52,714][20347] Updated weights for policy 0, policy_version 388940 (0.0008) [2023-12-27 08:19:52,763][20381] Updated weights for policy 1, policy_version 388757 (0.0008) [2023-12-27 08:19:52,775][20347] Updated weights for policy 0, policy_version 388950 (0.0008) [2023-12-27 08:19:52,817][20381] Updated weights for policy 1, policy_version 388767 (0.0008) [2023-12-27 08:19:52,832][20347] Updated weights for policy 0, policy_version 388960 (0.0008) [2023-12-27 08:19:52,869][20381] Updated weights for policy 1, policy_version 388777 (0.0009) [2023-12-27 08:19:53,550][20347] Updated weights for policy 0, policy_version 388970 (0.0008) [2023-12-27 08:19:53,564][20381] Updated weights for policy 1, policy_version 388787 (0.0007) [2023-12-27 08:19:53,609][20347] Updated weights for policy 0, policy_version 388980 (0.0008) [2023-12-27 08:19:53,620][20381] Updated weights for policy 1, policy_version 388797 (0.0008) [2023-12-27 08:19:53,669][20347] Updated weights for policy 0, policy_version 388990 (0.0008) [2023-12-27 08:19:53,680][20381] Updated weights for policy 1, policy_version 388807 (0.0007) [2023-12-27 08:19:54,363][20347] Updated weights for policy 0, policy_version 389000 (0.0008) [2023-12-27 08:19:54,384][20381] Updated weights for policy 1, policy_version 388817 (0.0008) [2023-12-27 08:19:54,422][20347] Updated weights for policy 0, policy_version 389010 (0.0006) [2023-12-27 08:19:54,439][20381] Updated weights for policy 1, policy_version 388827 (0.0006) [2023-12-27 08:19:54,477][20347] Updated weights for policy 0, policy_version 389020 (0.0006) [2023-12-27 08:19:54,501][20381] Updated weights for policy 1, policy_version 388837 (0.0006) [2023-12-27 08:19:54,553][20381] Updated weights for policy 1, policy_version 388847 (0.0005) [2023-12-27 08:19:55,100][20347] Updated weights for policy 0, policy_version 389030 (0.0005) [2023-12-27 08:19:55,146][20347] Updated weights for policy 0, policy_version 389040 (0.0005) [2023-12-27 08:19:55,160][20381] Updated weights for policy 1, policy_version 388857 (0.0007) [2023-12-27 08:19:55,196][20347] Updated weights for policy 0, policy_version 389050 (0.0006) [2023-12-27 08:19:55,215][20381] Updated weights for policy 1, policy_version 388867 (0.0009) [2023-12-27 08:19:55,266][20381] Updated weights for policy 1, policy_version 388877 (0.0008) [2023-12-27 08:19:55,775][20347] Updated weights for policy 0, policy_version 389060 (0.0006) [2023-12-27 08:19:55,835][20347] Updated weights for policy 0, policy_version 389070 (0.0007) [2023-12-27 08:19:55,882][20347] Updated weights for policy 0, policy_version 389080 (0.0007) [2023-12-27 08:19:55,959][20381] Updated weights for policy 1, policy_version 388887 (0.0007) [2023-12-27 08:19:56,015][20381] Updated weights for policy 1, policy_version 388897 (0.0006) [2023-12-27 08:19:56,078][20381] Updated weights for policy 1, policy_version 388907 (0.0005) [2023-12-27 08:19:56,436][20347] Updated weights for policy 0, policy_version 389090 (0.0008) [2023-12-27 08:19:56,497][20347] Updated weights for policy 0, policy_version 389100 (0.0006) [2023-12-27 08:19:56,558][20347] Updated weights for policy 0, policy_version 389110 (0.0005) [2023-12-27 08:19:56,624][20347] Updated weights for policy 0, policy_version 389120 (0.0006) [2023-12-27 08:19:56,736][20381] Updated weights for policy 1, policy_version 388917 (0.0006) [2023-12-27 08:19:56,784][20381] Updated weights for policy 1, policy_version 388927 (0.0007) [2023-12-27 08:19:56,837][20381] Updated weights for policy 1, policy_version 388937 (0.0008) [2023-12-27 08:19:57,240][20347] Updated weights for policy 0, policy_version 389130 (0.0007) [2023-12-27 08:19:57,302][20347] Updated weights for policy 0, policy_version 389140 (0.0008) [2023-12-27 08:19:57,356][20347] Updated weights for policy 0, policy_version 389150 (0.0006) [2023-12-27 08:19:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 199221248. Throughput: 0: 10411.9, 1: 10267.7. Samples: 199215356. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:19:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:19:57,465][20381] Updated weights for policy 1, policy_version 388947 (0.0008) [2023-12-27 08:19:57,517][20381] Updated weights for policy 1, policy_version 388957 (0.0008) [2023-12-27 08:19:57,565][20381] Updated weights for policy 1, policy_version 388967 (0.0007) [2023-12-27 08:19:58,019][20347] Updated weights for policy 0, policy_version 389160 (0.0007) [2023-12-27 08:19:58,074][20347] Updated weights for policy 0, policy_version 389170 (0.0007) [2023-12-27 08:19:58,131][20347] Updated weights for policy 0, policy_version 389180 (0.0007) [2023-12-27 08:19:58,263][20381] Updated weights for policy 1, policy_version 388977 (0.0008) [2023-12-27 08:19:58,320][20381] Updated weights for policy 1, policy_version 388987 (0.0009) [2023-12-27 08:19:58,384][20381] Updated weights for policy 1, policy_version 388997 (0.0009) [2023-12-27 08:19:58,448][20381] Updated weights for policy 1, policy_version 389007 (0.0008) [2023-12-27 08:19:58,927][20347] Updated weights for policy 0, policy_version 389190 (0.0008) [2023-12-27 08:19:58,991][20347] Updated weights for policy 0, policy_version 389200 (0.0006) [2023-12-27 08:19:59,053][20347] Updated weights for policy 0, policy_version 389210 (0.0006) [2023-12-27 08:19:59,212][20381] Updated weights for policy 1, policy_version 389017 (0.0010) [2023-12-27 08:19:59,281][20381] Updated weights for policy 1, policy_version 389027 (0.0009) [2023-12-27 08:19:59,341][20381] Updated weights for policy 1, policy_version 389037 (0.0010) [2023-12-27 08:19:59,676][20347] Updated weights for policy 0, policy_version 389220 (0.0005) [2023-12-27 08:19:59,724][20347] Updated weights for policy 0, policy_version 389230 (0.0005) [2023-12-27 08:19:59,781][20347] Updated weights for policy 0, policy_version 389240 (0.0009) [2023-12-27 08:20:00,127][20381] Updated weights for policy 1, policy_version 389047 (0.0009) [2023-12-27 08:20:00,181][20381] Updated weights for policy 1, policy_version 389057 (0.0007) [2023-12-27 08:20:00,232][20292] KL-divergence is very high: 179.6447 [2023-12-27 08:20:00,245][20381] Updated weights for policy 1, policy_version 389067 (0.0009) [2023-12-27 08:20:00,384][20347] Updated weights for policy 0, policy_version 389250 (0.0009) [2023-12-27 08:20:00,435][20347] Updated weights for policy 0, policy_version 389260 (0.0005) [2023-12-27 08:20:00,490][20347] Updated weights for policy 0, policy_version 389270 (0.0005) [2023-12-27 08:20:00,540][20347] Updated weights for policy 0, policy_version 389280 (0.0005) [2023-12-27 08:20:00,887][20381] Updated weights for policy 1, policy_version 389077 (0.0007) [2023-12-27 08:20:00,932][20381] Updated weights for policy 1, policy_version 389087 (0.0005) [2023-12-27 08:20:00,989][20381] Updated weights for policy 1, policy_version 389097 (0.0005) [2023-12-27 08:20:01,056][20347] Updated weights for policy 0, policy_version 389290 (0.0008) [2023-12-27 08:20:01,109][20347] Updated weights for policy 0, policy_version 389300 (0.0007) [2023-12-27 08:20:01,166][20347] Updated weights for policy 0, policy_version 389310 (0.0007) [2023-12-27 08:20:01,649][20381] Updated weights for policy 1, policy_version 389107 (0.0008) [2023-12-27 08:20:01,715][20381] Updated weights for policy 1, policy_version 389117 (0.0008) [2023-12-27 08:20:01,782][20381] Updated weights for policy 1, policy_version 389127 (0.0007) [2023-12-27 08:20:01,876][20347] Updated weights for policy 0, policy_version 389320 (0.0006) [2023-12-27 08:20:01,931][20347] Updated weights for policy 0, policy_version 389330 (0.0008) [2023-12-27 08:20:01,976][20347] Updated weights for policy 0, policy_version 389340 (0.0007) [2023-12-27 08:20:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 199327744. Throughput: 0: 10498.2, 1: 10148.3. Samples: 199339320. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.740')] [2023-12-27 08:20:02,454][20381] Updated weights for policy 1, policy_version 389137 (0.0007) [2023-12-27 08:20:02,500][20381] Updated weights for policy 1, policy_version 389147 (0.0005) [2023-12-27 08:20:02,548][20381] Updated weights for policy 1, policy_version 389157 (0.0005) [2023-12-27 08:20:02,599][20381] Updated weights for policy 1, policy_version 389167 (0.0007) [2023-12-27 08:20:02,710][20347] Updated weights for policy 0, policy_version 389350 (0.0007) [2023-12-27 08:20:02,765][20347] Updated weights for policy 0, policy_version 389360 (0.0008) [2023-12-27 08:20:02,819][20347] Updated weights for policy 0, policy_version 389370 (0.0007) [2023-12-27 08:20:03,287][20381] Updated weights for policy 1, policy_version 389177 (0.0007) [2023-12-27 08:20:03,333][20381] Updated weights for policy 1, policy_version 389187 (0.0007) [2023-12-27 08:20:03,375][20381] Updated weights for policy 1, policy_version 389197 (0.0008) [2023-12-27 08:20:03,449][20347] Updated weights for policy 0, policy_version 389380 (0.0007) [2023-12-27 08:20:03,492][20347] Updated weights for policy 0, policy_version 389390 (0.0008) [2023-12-27 08:20:03,537][20347] Updated weights for policy 0, policy_version 389400 (0.0007) [2023-12-27 08:20:04,039][20381] Updated weights for policy 1, policy_version 389207 (0.0006) [2023-12-27 08:20:04,095][20381] Updated weights for policy 1, policy_version 389217 (0.0006) [2023-12-27 08:20:04,159][20381] Updated weights for policy 1, policy_version 389227 (0.0006) [2023-12-27 08:20:04,297][20347] Updated weights for policy 0, policy_version 389410 (0.0007) [2023-12-27 08:20:04,341][20347] Updated weights for policy 0, policy_version 389420 (0.0006) [2023-12-27 08:20:04,398][20347] Updated weights for policy 0, policy_version 389430 (0.0007) [2023-12-27 08:20:04,453][20347] Updated weights for policy 0, policy_version 389440 (0.0008) [2023-12-27 08:20:04,830][20381] Updated weights for policy 1, policy_version 389237 (0.0008) [2023-12-27 08:20:04,886][20381] Updated weights for policy 1, policy_version 389247 (0.0008) [2023-12-27 08:20:04,940][20381] Updated weights for policy 1, policy_version 389257 (0.0007) [2023-12-27 08:20:05,154][20347] Updated weights for policy 0, policy_version 389450 (0.0005) [2023-12-27 08:20:05,204][20347] Updated weights for policy 0, policy_version 389460 (0.0005) [2023-12-27 08:20:05,251][20347] Updated weights for policy 0, policy_version 389470 (0.0005) [2023-12-27 08:20:05,711][20381] Updated weights for policy 1, policy_version 389267 (0.0008) [2023-12-27 08:20:05,764][20381] Updated weights for policy 1, policy_version 389277 (0.0009) [2023-12-27 08:20:05,769][20347] Updated weights for policy 0, policy_version 389480 (0.0005) [2023-12-27 08:20:05,814][20381] Updated weights for policy 1, policy_version 389287 (0.0007) [2023-12-27 08:20:05,833][20347] Updated weights for policy 0, policy_version 389490 (0.0005) [2023-12-27 08:20:05,882][20347] Updated weights for policy 0, policy_version 389500 (0.0005) [2023-12-27 08:20:06,454][20347] Updated weights for policy 0, policy_version 389510 (0.0007) [2023-12-27 08:20:06,507][20347] Updated weights for policy 0, policy_version 389520 (0.0008) [2023-12-27 08:20:06,560][20347] Updated weights for policy 0, policy_version 389530 (0.0008) [2023-12-27 08:20:06,582][20381] Updated weights for policy 1, policy_version 389297 (0.0007) [2023-12-27 08:20:06,639][20381] Updated weights for policy 1, policy_version 389307 (0.0007) [2023-12-27 08:20:06,699][20381] Updated weights for policy 1, policy_version 389317 (0.0008) [2023-12-27 08:20:06,757][20381] Updated weights for policy 1, policy_version 389327 (0.0008) [2023-12-27 08:20:07,279][20347] Updated weights for policy 0, policy_version 389540 (0.0007) [2023-12-27 08:20:07,340][20347] Updated weights for policy 0, policy_version 389550 (0.0007) [2023-12-27 08:20:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 199426048. Throughput: 0: 10460.2, 1: 10164.8. Samples: 199400436. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.260')] [2023-12-27 08:20:07,393][20347] Updated weights for policy 0, policy_version 389560 (0.0007) [2023-12-27 08:20:07,414][20381] Updated weights for policy 1, policy_version 389337 (0.0006) [2023-12-27 08:20:07,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000389568_99745792.pth... [2023-12-27 08:20:07,436][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000388320_99426304.pth [2023-12-27 08:20:07,474][20381] Updated weights for policy 1, policy_version 389347 (0.0007) [2023-12-27 08:20:07,534][20381] Updated weights for policy 1, policy_version 389357 (0.0008) [2023-12-27 08:20:07,550][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000389360_99696640.pth... [2023-12-27 08:20:07,554][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000388176_99393536.pth [2023-12-27 08:20:08,081][20347] Updated weights for policy 0, policy_version 389570 (0.0007) [2023-12-27 08:20:08,135][20347] Updated weights for policy 0, policy_version 389580 (0.0007) [2023-12-27 08:20:08,189][20347] Updated weights for policy 0, policy_version 389590 (0.0008) [2023-12-27 08:20:08,232][20381] Updated weights for policy 1, policy_version 389367 (0.0007) [2023-12-27 08:20:08,241][20347] Updated weights for policy 0, policy_version 389600 (0.0008) [2023-12-27 08:20:08,286][20381] Updated weights for policy 1, policy_version 389377 (0.0007) [2023-12-27 08:20:08,344][20381] Updated weights for policy 1, policy_version 389387 (0.0007) [2023-12-27 08:20:08,908][20347] Updated weights for policy 0, policy_version 389610 (0.0010) [2023-12-27 08:20:08,964][20347] Updated weights for policy 0, policy_version 389620 (0.0009) [2023-12-27 08:20:09,021][20347] Updated weights for policy 0, policy_version 389630 (0.0009) [2023-12-27 08:20:09,054][20381] Updated weights for policy 1, policy_version 389397 (0.0006) [2023-12-27 08:20:09,110][20381] Updated weights for policy 1, policy_version 389407 (0.0008) [2023-12-27 08:20:09,171][20381] Updated weights for policy 1, policy_version 389417 (0.0009) [2023-12-27 08:20:09,709][20347] Updated weights for policy 0, policy_version 389640 (0.0011) [2023-12-27 08:20:09,765][20347] Updated weights for policy 0, policy_version 389650 (0.0010) [2023-12-27 08:20:09,816][20347] Updated weights for policy 0, policy_version 389660 (0.0010) [2023-12-27 08:20:09,889][20381] Updated weights for policy 1, policy_version 389427 (0.0009) [2023-12-27 08:20:09,951][20381] Updated weights for policy 1, policy_version 389437 (0.0011) [2023-12-27 08:20:10,010][20381] Updated weights for policy 1, policy_version 389447 (0.0010) [2023-12-27 08:20:10,523][20347] Updated weights for policy 0, policy_version 389670 (0.0007) [2023-12-27 08:20:10,574][20347] Updated weights for policy 0, policy_version 389680 (0.0005) [2023-12-27 08:20:10,634][20347] Updated weights for policy 0, policy_version 389690 (0.0005) [2023-12-27 08:20:10,685][20381] Updated weights for policy 1, policy_version 389457 (0.0010) [2023-12-27 08:20:10,732][20381] Updated weights for policy 1, policy_version 389467 (0.0005) [2023-12-27 08:20:10,779][20381] Updated weights for policy 1, policy_version 389477 (0.0006) [2023-12-27 08:20:10,827][20381] Updated weights for policy 1, policy_version 389487 (0.0007) [2023-12-27 08:20:11,181][20347] Updated weights for policy 0, policy_version 389700 (0.0005) [2023-12-27 08:20:11,234][20347] Updated weights for policy 0, policy_version 389710 (0.0005) [2023-12-27 08:20:11,299][20347] Updated weights for policy 0, policy_version 389720 (0.0008) [2023-12-27 08:20:11,562][20381] Updated weights for policy 1, policy_version 389497 (0.0011) [2023-12-27 08:20:11,623][20381] Updated weights for policy 1, policy_version 389507 (0.0012) [2023-12-27 08:20:11,681][20381] Updated weights for policy 1, policy_version 389518 (0.0010) [2023-12-27 08:20:11,932][20347] Updated weights for policy 0, policy_version 389730 (0.0006) [2023-12-27 08:20:11,997][20347] Updated weights for policy 0, policy_version 389740 (0.0008) [2023-12-27 08:20:12,061][20347] Updated weights for policy 0, policy_version 389750 (0.0008) [2023-12-27 08:20:12,114][20347] Updated weights for policy 0, policy_version 389760 (0.0006) [2023-12-27 08:20:12,313][20381] Updated weights for policy 1, policy_version 389528 (0.0007) [2023-12-27 08:20:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 199532544. Throughput: 0: 10584.1, 1: 10075.2. Samples: 199524516. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.520')] [2023-12-27 08:20:12,378][20381] Updated weights for policy 1, policy_version 389538 (0.0007) [2023-12-27 08:20:12,436][20381] Updated weights for policy 1, policy_version 389548 (0.0007) [2023-12-27 08:20:12,756][20347] Updated weights for policy 0, policy_version 389770 (0.0005) [2023-12-27 08:20:12,815][20347] Updated weights for policy 0, policy_version 389780 (0.0006) [2023-12-27 08:20:12,867][20347] Updated weights for policy 0, policy_version 389790 (0.0007) [2023-12-27 08:20:13,077][20381] Updated weights for policy 1, policy_version 389558 (0.0006) [2023-12-27 08:20:13,122][20381] Updated weights for policy 1, policy_version 389568 (0.0005) [2023-12-27 08:20:13,172][20381] Updated weights for policy 1, policy_version 389578 (0.0005) [2023-12-27 08:20:13,488][20347] Updated weights for policy 0, policy_version 389800 (0.0010) [2023-12-27 08:20:13,545][20347] Updated weights for policy 0, policy_version 389810 (0.0010) [2023-12-27 08:20:13,602][20347] Updated weights for policy 0, policy_version 389820 (0.0010) [2023-12-27 08:20:13,766][20381] Updated weights for policy 1, policy_version 389588 (0.0005) [2023-12-27 08:20:13,826][20381] Updated weights for policy 1, policy_version 389598 (0.0005) [2023-12-27 08:20:13,888][20381] Updated weights for policy 1, policy_version 389608 (0.0007) [2023-12-27 08:20:14,228][20347] Updated weights for policy 0, policy_version 389830 (0.0010) [2023-12-27 08:20:14,282][20347] Updated weights for policy 0, policy_version 389840 (0.0010) [2023-12-27 08:20:14,341][20347] Updated weights for policy 0, policy_version 389850 (0.0010) [2023-12-27 08:20:14,487][20381] Updated weights for policy 1, policy_version 389618 (0.0010) [2023-12-27 08:20:14,547][20381] Updated weights for policy 1, policy_version 389628 (0.0010) [2023-12-27 08:20:14,600][20381] Updated weights for policy 1, policy_version 389638 (0.0011) [2023-12-27 08:20:14,656][20381] Updated weights for policy 1, policy_version 389648 (0.0005) [2023-12-27 08:20:15,019][20347] Updated weights for policy 0, policy_version 389860 (0.0009) [2023-12-27 08:20:15,090][20347] Updated weights for policy 0, policy_version 389870 (0.0006) [2023-12-27 08:20:15,157][20347] Updated weights for policy 0, policy_version 389880 (0.0009) [2023-12-27 08:20:15,268][20381] Updated weights for policy 1, policy_version 389658 (0.0005) [2023-12-27 08:20:15,319][20381] Updated weights for policy 1, policy_version 389668 (0.0006) [2023-12-27 08:20:15,365][20381] Updated weights for policy 1, policy_version 389678 (0.0011) [2023-12-27 08:20:15,775][20347] Updated weights for policy 0, policy_version 389890 (0.0009) [2023-12-27 08:20:15,834][20347] Updated weights for policy 0, policy_version 389900 (0.0006) [2023-12-27 08:20:15,898][20347] Updated weights for policy 0, policy_version 389910 (0.0005) [2023-12-27 08:20:15,960][20347] Updated weights for policy 0, policy_version 389920 (0.0006) [2023-12-27 08:20:15,992][20381] Updated weights for policy 1, policy_version 389688 (0.0007) [2023-12-27 08:20:16,049][20381] Updated weights for policy 1, policy_version 389698 (0.0008) [2023-12-27 08:20:16,104][20381] Updated weights for policy 1, policy_version 389708 (0.0007) [2023-12-27 08:20:16,573][20347] Updated weights for policy 0, policy_version 389930 (0.0005) [2023-12-27 08:20:16,625][20347] Updated weights for policy 0, policy_version 389940 (0.0005) [2023-12-27 08:20:16,676][20347] Updated weights for policy 0, policy_version 389950 (0.0005) [2023-12-27 08:20:16,784][20381] Updated weights for policy 1, policy_version 389718 (0.0009) [2023-12-27 08:20:16,837][20381] Updated weights for policy 1, policy_version 389728 (0.0011) [2023-12-27 08:20:16,895][20381] Updated weights for policy 1, policy_version 389738 (0.0011) [2023-12-27 08:20:17,280][20347] Updated weights for policy 0, policy_version 389960 (0.0007) [2023-12-27 08:20:17,326][20347] Updated weights for policy 0, policy_version 389970 (0.0007) [2023-12-27 08:20:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 199639040. Throughput: 0: 10590.2, 1: 10088.9. Samples: 199652860. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.230')] [2023-12-27 08:20:17,378][20347] Updated weights for policy 0, policy_version 389980 (0.0005) [2023-12-27 08:20:17,599][20381] Updated weights for policy 1, policy_version 389748 (0.0011) [2023-12-27 08:20:17,648][20381] Updated weights for policy 1, policy_version 389758 (0.0011) [2023-12-27 08:20:17,694][20381] Updated weights for policy 1, policy_version 389768 (0.0010) [2023-12-27 08:20:17,927][20347] Updated weights for policy 0, policy_version 389990 (0.0005) [2023-12-27 08:20:17,986][20347] Updated weights for policy 0, policy_version 390000 (0.0008) [2023-12-27 08:20:18,049][20347] Updated weights for policy 0, policy_version 390010 (0.0009) [2023-12-27 08:20:18,318][20381] Updated weights for policy 1, policy_version 389778 (0.0010) [2023-12-27 08:20:18,382][20381] Updated weights for policy 1, policy_version 389788 (0.0010) [2023-12-27 08:20:18,444][20381] Updated weights for policy 1, policy_version 389798 (0.0011) [2023-12-27 08:20:18,500][20381] Updated weights for policy 1, policy_version 389808 (0.0011) [2023-12-27 08:20:18,675][20347] Updated weights for policy 0, policy_version 390020 (0.0010) [2023-12-27 08:20:18,732][20347] Updated weights for policy 0, policy_version 390030 (0.0011) [2023-12-27 08:20:18,782][20347] Updated weights for policy 0, policy_version 390040 (0.0008) [2023-12-27 08:20:19,177][20381] Updated weights for policy 1, policy_version 389818 (0.0010) [2023-12-27 08:20:19,236][20381] Updated weights for policy 1, policy_version 389828 (0.0011) [2023-12-27 08:20:19,295][20381] Updated weights for policy 1, policy_version 389838 (0.0011) [2023-12-27 08:20:19,422][20347] Updated weights for policy 0, policy_version 390050 (0.0006) [2023-12-27 08:20:19,480][20347] Updated weights for policy 0, policy_version 390060 (0.0006) [2023-12-27 08:20:19,538][20347] Updated weights for policy 0, policy_version 390070 (0.0010) [2023-12-27 08:20:19,594][20347] Updated weights for policy 0, policy_version 390080 (0.0010) [2023-12-27 08:20:19,998][20381] Updated weights for policy 1, policy_version 389848 (0.0011) [2023-12-27 08:20:20,052][20381] Updated weights for policy 1, policy_version 389858 (0.0011) [2023-12-27 08:20:20,112][20381] Updated weights for policy 1, policy_version 389868 (0.0011) [2023-12-27 08:20:20,338][20347] Updated weights for policy 0, policy_version 390090 (0.0009) [2023-12-27 08:20:20,392][20347] Updated weights for policy 0, policy_version 390100 (0.0011) [2023-12-27 08:20:20,449][20347] Updated weights for policy 0, policy_version 390110 (0.0009) [2023-12-27 08:20:20,818][20381] Updated weights for policy 1, policy_version 389878 (0.0011) [2023-12-27 08:20:20,889][20381] Updated weights for policy 1, policy_version 389888 (0.0011) [2023-12-27 08:20:20,958][20381] Updated weights for policy 1, policy_version 389898 (0.0010) [2023-12-27 08:20:21,071][20347] Updated weights for policy 0, policy_version 390120 (0.0007) [2023-12-27 08:20:21,130][20347] Updated weights for policy 0, policy_version 390130 (0.0006) [2023-12-27 08:20:21,188][20347] Updated weights for policy 0, policy_version 390140 (0.0007) [2023-12-27 08:20:21,701][20381] Updated weights for policy 1, policy_version 389908 (0.0009) [2023-12-27 08:20:21,762][20381] Updated weights for policy 1, policy_version 389918 (0.0008) [2023-12-27 08:20:21,824][20381] Updated weights for policy 1, policy_version 389928 (0.0008) [2023-12-27 08:20:21,869][20347] Updated weights for policy 0, policy_version 390150 (0.0007) [2023-12-27 08:20:21,929][20347] Updated weights for policy 0, policy_version 390160 (0.0008) [2023-12-27 08:20:21,983][20347] Updated weights for policy 0, policy_version 390170 (0.0009) [2023-12-27 08:20:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 199745536. Throughput: 0: 10572.6, 1: 10095.7. Samples: 199716244. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.830')] [2023-12-27 08:20:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000389936_99844096.pth... [2023-12-27 08:20:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000390176_99901440.pth... [2023-12-27 08:20:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000388752_99540992.pth [2023-12-27 08:20:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000388928_99581952.pth [2023-12-27 08:20:22,497][20381] Updated weights for policy 1, policy_version 389938 (0.0007) [2023-12-27 08:20:22,553][20381] Updated weights for policy 1, policy_version 389948 (0.0007) [2023-12-27 08:20:22,603][20381] Updated weights for policy 1, policy_version 389958 (0.0006) [2023-12-27 08:20:22,659][20381] Updated weights for policy 1, policy_version 389968 (0.0005) [2023-12-27 08:20:22,714][20347] Updated weights for policy 0, policy_version 390180 (0.0007) [2023-12-27 08:20:22,778][20347] Updated weights for policy 0, policy_version 390190 (0.0007) [2023-12-27 08:20:22,848][20347] Updated weights for policy 0, policy_version 390200 (0.0007) [2023-12-27 08:20:23,348][20381] Updated weights for policy 1, policy_version 389978 (0.0010) [2023-12-27 08:20:23,402][20347] Updated weights for policy 0, policy_version 390210 (0.0008) [2023-12-27 08:20:23,402][20381] Updated weights for policy 1, policy_version 389988 (0.0009) [2023-12-27 08:20:23,451][20381] Updated weights for policy 1, policy_version 389998 (0.0006) [2023-12-27 08:20:23,458][20347] Updated weights for policy 0, policy_version 390220 (0.0006) [2023-12-27 08:20:23,512][20347] Updated weights for policy 0, policy_version 390230 (0.0008) [2023-12-27 08:20:23,571][20347] Updated weights for policy 0, policy_version 390240 (0.0007) [2023-12-27 08:20:24,063][20381] Updated weights for policy 1, policy_version 390008 (0.0006) [2023-12-27 08:20:24,119][20381] Updated weights for policy 1, policy_version 390018 (0.0006) [2023-12-27 08:20:24,174][20381] Updated weights for policy 1, policy_version 390028 (0.0007) [2023-12-27 08:20:24,292][20347] Updated weights for policy 0, policy_version 390250 (0.0006) [2023-12-27 08:20:24,342][20347] Updated weights for policy 0, policy_version 390260 (0.0005) [2023-12-27 08:20:24,389][20347] Updated weights for policy 0, policy_version 390270 (0.0007) [2023-12-27 08:20:24,826][20381] Updated weights for policy 1, policy_version 390038 (0.0007) [2023-12-27 08:20:24,896][20381] Updated weights for policy 1, policy_version 390048 (0.0006) [2023-12-27 08:20:24,950][20381] Updated weights for policy 1, policy_version 390058 (0.0007) [2023-12-27 08:20:24,969][20347] Updated weights for policy 0, policy_version 390280 (0.0006) [2023-12-27 08:20:25,019][20347] Updated weights for policy 0, policy_version 390290 (0.0007) [2023-12-27 08:20:25,071][20347] Updated weights for policy 0, policy_version 390300 (0.0007) [2023-12-27 08:20:25,605][20381] Updated weights for policy 1, policy_version 390068 (0.0007) [2023-12-27 08:20:25,656][20381] Updated weights for policy 1, policy_version 390078 (0.0008) [2023-12-27 08:20:25,703][20381] Updated weights for policy 1, policy_version 390088 (0.0006) [2023-12-27 08:20:25,753][20347] Updated weights for policy 0, policy_version 390310 (0.0006) [2023-12-27 08:20:25,806][20347] Updated weights for policy 0, policy_version 390320 (0.0005) [2023-12-27 08:20:25,875][20347] Updated weights for policy 0, policy_version 390330 (0.0005) [2023-12-27 08:20:26,314][20381] Updated weights for policy 1, policy_version 390098 (0.0005) [2023-12-27 08:20:26,364][20381] Updated weights for policy 1, policy_version 390108 (0.0005) [2023-12-27 08:20:26,420][20381] Updated weights for policy 1, policy_version 390118 (0.0005) [2023-12-27 08:20:26,473][20347] Updated weights for policy 0, policy_version 390340 (0.0007) [2023-12-27 08:20:26,475][20381] Updated weights for policy 1, policy_version 390128 (0.0005) [2023-12-27 08:20:26,521][20347] Updated weights for policy 0, policy_version 390350 (0.0010) [2023-12-27 08:20:26,569][20347] Updated weights for policy 0, policy_version 390360 (0.0010) [2023-12-27 08:20:26,973][20381] Updated weights for policy 1, policy_version 390138 (0.0007) [2023-12-27 08:20:27,018][20381] Updated weights for policy 1, policy_version 390148 (0.0007) [2023-12-27 08:20:27,070][20381] Updated weights for policy 1, policy_version 390158 (0.0007) [2023-12-27 08:20:27,210][20347] Updated weights for policy 0, policy_version 390370 (0.0010) [2023-12-27 08:20:27,274][20347] Updated weights for policy 0, policy_version 390380 (0.0007) [2023-12-27 08:20:27,338][20347] Updated weights for policy 0, policy_version 390390 (0.0006) [2023-12-27 08:20:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 199852032. Throughput: 0: 10576.3, 1: 10156.9. Samples: 199841972. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:27,406][20347] Updated weights for policy 0, policy_version 390400 (0.0006) [2023-12-27 08:20:27,681][20381] Updated weights for policy 1, policy_version 390168 (0.0007) [2023-12-27 08:20:27,732][20381] Updated weights for policy 1, policy_version 390178 (0.0006) [2023-12-27 08:20:27,783][20381] Updated weights for policy 1, policy_version 390188 (0.0005) [2023-12-27 08:20:28,035][20347] Updated weights for policy 0, policy_version 390410 (0.0006) [2023-12-27 08:20:28,082][20347] Updated weights for policy 0, policy_version 390420 (0.0008) [2023-12-27 08:20:28,127][20347] Updated weights for policy 0, policy_version 390430 (0.0008) [2023-12-27 08:20:28,349][20381] Updated weights for policy 1, policy_version 390198 (0.0007) [2023-12-27 08:20:28,405][20381] Updated weights for policy 1, policy_version 390208 (0.0006) [2023-12-27 08:20:28,458][20381] Updated weights for policy 1, policy_version 390218 (0.0005) [2023-12-27 08:20:28,861][20347] Updated weights for policy 0, policy_version 390440 (0.0008) [2023-12-27 08:20:28,920][20347] Updated weights for policy 0, policy_version 390450 (0.0008) [2023-12-27 08:20:28,978][20347] Updated weights for policy 0, policy_version 390460 (0.0008) [2023-12-27 08:20:29,113][20381] Updated weights for policy 1, policy_version 390228 (0.0007) [2023-12-27 08:20:29,165][20381] Updated weights for policy 1, policy_version 390238 (0.0007) [2023-12-27 08:20:29,222][20381] Updated weights for policy 1, policy_version 390248 (0.0008) [2023-12-27 08:20:29,609][20347] Updated weights for policy 0, policy_version 390470 (0.0006) [2023-12-27 08:20:29,666][20347] Updated weights for policy 0, policy_version 390480 (0.0007) [2023-12-27 08:20:29,719][20347] Updated weights for policy 0, policy_version 390490 (0.0007) [2023-12-27 08:20:29,948][20381] Updated weights for policy 1, policy_version 390258 (0.0008) [2023-12-27 08:20:30,001][20381] Updated weights for policy 1, policy_version 390268 (0.0006) [2023-12-27 08:20:30,058][20381] Updated weights for policy 1, policy_version 390278 (0.0008) [2023-12-27 08:20:30,118][20381] Updated weights for policy 1, policy_version 390288 (0.0008) [2023-12-27 08:20:30,424][20347] Updated weights for policy 0, policy_version 390500 (0.0007) [2023-12-27 08:20:30,484][20347] Updated weights for policy 0, policy_version 390510 (0.0007) [2023-12-27 08:20:30,544][20347] Updated weights for policy 0, policy_version 390520 (0.0007) [2023-12-27 08:20:30,745][20381] Updated weights for policy 1, policy_version 390298 (0.0008) [2023-12-27 08:20:30,799][20381] Updated weights for policy 1, policy_version 390308 (0.0008) [2023-12-27 08:20:30,844][20381] Updated weights for policy 1, policy_version 390318 (0.0007) [2023-12-27 08:20:31,202][20347] Updated weights for policy 0, policy_version 390530 (0.0007) [2023-12-27 08:20:31,260][20347] Updated weights for policy 0, policy_version 390540 (0.0007) [2023-12-27 08:20:31,316][20347] Updated weights for policy 0, policy_version 390550 (0.0007) [2023-12-27 08:20:31,382][20347] Updated weights for policy 0, policy_version 390560 (0.0007) [2023-12-27 08:20:31,517][20381] Updated weights for policy 1, policy_version 390328 (0.0007) [2023-12-27 08:20:31,569][20381] Updated weights for policy 1, policy_version 390338 (0.0008) [2023-12-27 08:20:31,620][20381] Updated weights for policy 1, policy_version 390348 (0.0008) [2023-12-27 08:20:32,075][20347] Updated weights for policy 0, policy_version 390570 (0.0008) [2023-12-27 08:20:32,133][20347] Updated weights for policy 0, policy_version 390580 (0.0008) [2023-12-27 08:20:32,184][20347] Updated weights for policy 0, policy_version 390590 (0.0006) [2023-12-27 08:20:32,242][20381] Updated weights for policy 1, policy_version 390358 (0.0009) [2023-12-27 08:20:32,300][20381] Updated weights for policy 1, policy_version 390368 (0.0008) [2023-12-27 08:20:32,358][20381] Updated weights for policy 1, policy_version 390378 (0.0008) [2023-12-27 08:20:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 199958528. Throughput: 0: 10557.9, 1: 10295.1. Samples: 199970180. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:32,816][20347] Updated weights for policy 0, policy_version 390600 (0.0007) [2023-12-27 08:20:32,860][20347] Updated weights for policy 0, policy_version 390610 (0.0008) [2023-12-27 08:20:32,911][20347] Updated weights for policy 0, policy_version 390620 (0.0007) [2023-12-27 08:20:33,035][20381] Updated weights for policy 1, policy_version 390388 (0.0008) [2023-12-27 08:20:33,093][20381] Updated weights for policy 1, policy_version 390398 (0.0007) [2023-12-27 08:20:33,147][20381] Updated weights for policy 1, policy_version 390408 (0.0007) [2023-12-27 08:20:33,577][20347] Updated weights for policy 0, policy_version 390630 (0.0008) [2023-12-27 08:20:33,636][20347] Updated weights for policy 0, policy_version 390640 (0.0007) [2023-12-27 08:20:33,693][20347] Updated weights for policy 0, policy_version 390650 (0.0007) [2023-12-27 08:20:33,846][20381] Updated weights for policy 1, policy_version 390418 (0.0008) [2023-12-27 08:20:33,891][20381] Updated weights for policy 1, policy_version 390428 (0.0008) [2023-12-27 08:20:33,936][20381] Updated weights for policy 1, policy_version 390438 (0.0007) [2023-12-27 08:20:33,988][20381] Updated weights for policy 1, policy_version 390448 (0.0008) [2023-12-27 08:20:34,361][20347] Updated weights for policy 0, policy_version 390660 (0.0006) [2023-12-27 08:20:34,420][20347] Updated weights for policy 0, policy_version 390670 (0.0006) [2023-12-27 08:20:34,474][20347] Updated weights for policy 0, policy_version 390680 (0.0007) [2023-12-27 08:20:34,679][20381] Updated weights for policy 1, policy_version 390458 (0.0005) [2023-12-27 08:20:34,733][20381] Updated weights for policy 1, policy_version 390468 (0.0005) [2023-12-27 08:20:34,789][20381] Updated weights for policy 1, policy_version 390478 (0.0006) [2023-12-27 08:20:35,157][20347] Updated weights for policy 0, policy_version 390690 (0.0009) [2023-12-27 08:20:35,216][20347] Updated weights for policy 0, policy_version 390700 (0.0010) [2023-12-27 08:20:35,268][20347] Updated weights for policy 0, policy_version 390710 (0.0010) [2023-12-27 08:20:35,321][20347] Updated weights for policy 0, policy_version 390720 (0.0010) [2023-12-27 08:20:35,345][20381] Updated weights for policy 1, policy_version 390488 (0.0006) [2023-12-27 08:20:35,395][20381] Updated weights for policy 1, policy_version 390498 (0.0010) [2023-12-27 08:20:35,454][20381] Updated weights for policy 1, policy_version 390508 (0.0011) [2023-12-27 08:20:35,932][20347] Updated weights for policy 0, policy_version 390730 (0.0007) [2023-12-27 08:20:35,984][20347] Updated weights for policy 0, policy_version 390740 (0.0010) [2023-12-27 08:20:36,041][20347] Updated weights for policy 0, policy_version 390750 (0.0010) [2023-12-27 08:20:36,161][20381] Updated weights for policy 1, policy_version 390518 (0.0009) [2023-12-27 08:20:36,222][20381] Updated weights for policy 1, policy_version 390528 (0.0007) [2023-12-27 08:20:36,282][20381] Updated weights for policy 1, policy_version 390538 (0.0008) [2023-12-27 08:20:36,745][20347] Updated weights for policy 0, policy_version 390760 (0.0010) [2023-12-27 08:20:36,804][20347] Updated weights for policy 0, policy_version 390770 (0.0010) [2023-12-27 08:20:36,877][20347] Updated weights for policy 0, policy_version 390780 (0.0007) [2023-12-27 08:20:36,905][20381] Updated weights for policy 1, policy_version 390548 (0.0008) [2023-12-27 08:20:36,963][20381] Updated weights for policy 1, policy_version 390558 (0.0007) [2023-12-27 08:20:37,016][20381] Updated weights for policy 1, policy_version 390568 (0.0008) [2023-12-27 08:20:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 200065024. Throughput: 0: 10543.0, 1: 10346.0. Samples: 200033372. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000390576_100007936.pth... [2023-12-27 08:20:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000390784_100057088.pth... [2023-12-27 08:20:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000389360_99696640.pth [2023-12-27 08:20:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000389568_99745792.pth [2023-12-27 08:20:37,492][20347] Updated weights for policy 0, policy_version 390790 (0.0006) [2023-12-27 08:20:37,538][20347] Updated weights for policy 0, policy_version 390800 (0.0008) [2023-12-27 08:20:37,596][20347] Updated weights for policy 0, policy_version 390810 (0.0007) [2023-12-27 08:20:37,734][20381] Updated weights for policy 1, policy_version 390578 (0.0007) [2023-12-27 08:20:37,795][20381] Updated weights for policy 1, policy_version 390588 (0.0009) [2023-12-27 08:20:37,851][20381] Updated weights for policy 1, policy_version 390598 (0.0006) [2023-12-27 08:20:37,910][20381] Updated weights for policy 1, policy_version 390608 (0.0005) [2023-12-27 08:20:38,290][20347] Updated weights for policy 0, policy_version 390820 (0.0007) [2023-12-27 08:20:38,352][20347] Updated weights for policy 0, policy_version 390830 (0.0007) [2023-12-27 08:20:38,413][20347] Updated weights for policy 0, policy_version 390840 (0.0006) [2023-12-27 08:20:38,557][20381] Updated weights for policy 1, policy_version 390618 (0.0007) [2023-12-27 08:20:38,613][20381] Updated weights for policy 1, policy_version 390628 (0.0007) [2023-12-27 08:20:38,659][20381] Updated weights for policy 1, policy_version 390638 (0.0008) [2023-12-27 08:20:39,062][20347] Updated weights for policy 0, policy_version 390850 (0.0006) [2023-12-27 08:20:39,117][20347] Updated weights for policy 0, policy_version 390860 (0.0007) [2023-12-27 08:20:39,173][20347] Updated weights for policy 0, policy_version 390870 (0.0008) [2023-12-27 08:20:39,234][20347] Updated weights for policy 0, policy_version 390880 (0.0008) [2023-12-27 08:20:39,353][20381] Updated weights for policy 1, policy_version 390648 (0.0008) [2023-12-27 08:20:39,402][20381] Updated weights for policy 1, policy_version 390658 (0.0008) [2023-12-27 08:20:39,466][20381] Updated weights for policy 1, policy_version 390668 (0.0007) [2023-12-27 08:20:39,981][20347] Updated weights for policy 0, policy_version 390890 (0.0007) [2023-12-27 08:20:40,042][20347] Updated weights for policy 0, policy_version 390900 (0.0008) [2023-12-27 08:20:40,102][20347] Updated weights for policy 0, policy_version 390910 (0.0007) [2023-12-27 08:20:40,154][20381] Updated weights for policy 1, policy_version 390678 (0.0008) [2023-12-27 08:20:40,215][20381] Updated weights for policy 1, policy_version 390688 (0.0007) [2023-12-27 08:20:40,275][20381] Updated weights for policy 1, policy_version 390698 (0.0005) [2023-12-27 08:20:40,823][20347] Updated weights for policy 0, policy_version 390920 (0.0006) [2023-12-27 08:20:40,875][20381] Updated weights for policy 1, policy_version 390708 (0.0006) [2023-12-27 08:20:40,882][20347] Updated weights for policy 0, policy_version 390930 (0.0006) [2023-12-27 08:20:40,933][20381] Updated weights for policy 1, policy_version 390718 (0.0006) [2023-12-27 08:20:40,937][20347] Updated weights for policy 0, policy_version 390940 (0.0006) [2023-12-27 08:20:40,990][20381] Updated weights for policy 1, policy_version 390728 (0.0007) [2023-12-27 08:20:41,652][20381] Updated weights for policy 1, policy_version 390738 (0.0007) [2023-12-27 08:20:41,710][20347] Updated weights for policy 0, policy_version 390950 (0.0006) [2023-12-27 08:20:41,712][20381] Updated weights for policy 1, policy_version 390748 (0.0007) [2023-12-27 08:20:41,775][20381] Updated weights for policy 1, policy_version 390758 (0.0007) [2023-12-27 08:20:41,777][20347] Updated weights for policy 0, policy_version 390960 (0.0009) [2023-12-27 08:20:41,826][20381] Updated weights for policy 1, policy_version 390768 (0.0007) [2023-12-27 08:20:41,837][20347] Updated weights for policy 0, policy_version 390970 (0.0007) [2023-12-27 08:20:42,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 200163328. Throughput: 0: 10545.6, 1: 10357.8. Samples: 200156008. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:42,475][20381] Updated weights for policy 1, policy_version 390778 (0.0006) [2023-12-27 08:20:42,532][20381] Updated weights for policy 1, policy_version 390788 (0.0008) [2023-12-27 08:20:42,545][20347] Updated weights for policy 0, policy_version 390980 (0.0009) [2023-12-27 08:20:42,578][20381] Updated weights for policy 1, policy_version 390798 (0.0011) [2023-12-27 08:20:42,590][20347] Updated weights for policy 0, policy_version 390990 (0.0005) [2023-12-27 08:20:42,634][20347] Updated weights for policy 0, policy_version 391000 (0.0007) [2023-12-27 08:20:43,243][20347] Updated weights for policy 0, policy_version 391010 (0.0007) [2023-12-27 08:20:43,293][20347] Updated weights for policy 0, policy_version 391020 (0.0005) [2023-12-27 08:20:43,331][20381] Updated weights for policy 1, policy_version 390808 (0.0008) [2023-12-27 08:20:43,345][20347] Updated weights for policy 0, policy_version 391030 (0.0005) [2023-12-27 08:20:43,385][20381] Updated weights for policy 1, policy_version 390818 (0.0008) [2023-12-27 08:20:43,399][20347] Updated weights for policy 0, policy_version 391040 (0.0005) [2023-12-27 08:20:43,436][20381] Updated weights for policy 1, policy_version 390828 (0.0008) [2023-12-27 08:20:43,914][20347] Updated weights for policy 0, policy_version 391050 (0.0005) [2023-12-27 08:20:43,961][20347] Updated weights for policy 0, policy_version 391060 (0.0005) [2023-12-27 08:20:44,016][20347] Updated weights for policy 0, policy_version 391070 (0.0005) [2023-12-27 08:20:44,153][20381] Updated weights for policy 1, policy_version 390838 (0.0007) [2023-12-27 08:20:44,212][20381] Updated weights for policy 1, policy_version 390848 (0.0009) [2023-12-27 08:20:44,266][20381] Updated weights for policy 1, policy_version 390858 (0.0010) [2023-12-27 08:20:44,576][20347] Updated weights for policy 0, policy_version 391080 (0.0005) [2023-12-27 08:20:44,633][20347] Updated weights for policy 0, policy_version 391090 (0.0006) [2023-12-27 08:20:44,683][20347] Updated weights for policy 0, policy_version 391100 (0.0005) [2023-12-27 08:20:44,995][20381] Updated weights for policy 1, policy_version 390868 (0.0011) [2023-12-27 08:20:45,046][20381] Updated weights for policy 1, policy_version 390878 (0.0011) [2023-12-27 08:20:45,098][20381] Updated weights for policy 1, policy_version 390888 (0.0011) [2023-12-27 08:20:45,222][20347] Updated weights for policy 0, policy_version 391110 (0.0005) [2023-12-27 08:20:45,276][20347] Updated weights for policy 0, policy_version 391120 (0.0005) [2023-12-27 08:20:45,340][20347] Updated weights for policy 0, policy_version 391130 (0.0007) [2023-12-27 08:20:45,704][20381] Updated weights for policy 1, policy_version 390898 (0.0006) [2023-12-27 08:20:45,765][20381] Updated weights for policy 1, policy_version 390908 (0.0008) [2023-12-27 08:20:45,825][20381] Updated weights for policy 1, policy_version 390918 (0.0007) [2023-12-27 08:20:45,875][20381] Updated weights for policy 1, policy_version 390928 (0.0008) [2023-12-27 08:20:45,960][20347] Updated weights for policy 0, policy_version 391140 (0.0006) [2023-12-27 08:20:46,020][20347] Updated weights for policy 0, policy_version 391150 (0.0005) [2023-12-27 08:20:46,078][20347] Updated weights for policy 0, policy_version 391160 (0.0005) [2023-12-27 08:20:46,497][20381] Updated weights for policy 1, policy_version 390938 (0.0010) [2023-12-27 08:20:46,551][20381] Updated weights for policy 1, policy_version 390948 (0.0009) [2023-12-27 08:20:46,600][20381] Updated weights for policy 1, policy_version 390958 (0.0009) [2023-12-27 08:20:46,601][20347] Updated weights for policy 0, policy_version 391170 (0.0005) [2023-12-27 08:20:46,657][20347] Updated weights for policy 0, policy_version 391180 (0.0005) [2023-12-27 08:20:46,719][20347] Updated weights for policy 0, policy_version 391190 (0.0007) [2023-12-27 08:20:46,784][20347] Updated weights for policy 0, policy_version 391200 (0.0008) [2023-12-27 08:20:47,297][20381] Updated weights for policy 1, policy_version 390968 (0.0006) [2023-12-27 08:20:47,360][20381] Updated weights for policy 1, policy_version 390978 (0.0005) [2023-12-27 08:20:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 200269824. Throughput: 0: 10632.9, 1: 10407.3. Samples: 200286132. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:47,367][20347] Updated weights for policy 0, policy_version 391210 (0.0006) [2023-12-27 08:20:47,417][20381] Updated weights for policy 1, policy_version 390988 (0.0005) [2023-12-27 08:20:47,421][20347] Updated weights for policy 0, policy_version 391220 (0.0007) [2023-12-27 08:20:47,481][20347] Updated weights for policy 0, policy_version 391230 (0.0007) [2023-12-27 08:20:48,076][20381] Updated weights for policy 1, policy_version 390998 (0.0007) [2023-12-27 08:20:48,125][20381] Updated weights for policy 1, policy_version 391008 (0.0007) [2023-12-27 08:20:48,125][20347] Updated weights for policy 0, policy_version 391240 (0.0006) [2023-12-27 08:20:48,173][20381] Updated weights for policy 1, policy_version 391018 (0.0006) [2023-12-27 08:20:48,177][20347] Updated weights for policy 0, policy_version 391250 (0.0005) [2023-12-27 08:20:48,225][20347] Updated weights for policy 0, policy_version 391260 (0.0008) [2023-12-27 08:20:48,819][20381] Updated weights for policy 1, policy_version 391028 (0.0006) [2023-12-27 08:20:48,879][20381] Updated weights for policy 1, policy_version 391038 (0.0008) [2023-12-27 08:20:48,939][20347] Updated weights for policy 0, policy_version 391270 (0.0009) [2023-12-27 08:20:48,942][20381] Updated weights for policy 1, policy_version 391048 (0.0006) [2023-12-27 08:20:48,994][20347] Updated weights for policy 0, policy_version 391280 (0.0007) [2023-12-27 08:20:49,043][20347] Updated weights for policy 0, policy_version 391290 (0.0008) [2023-12-27 08:20:49,659][20381] Updated weights for policy 1, policy_version 391058 (0.0006) [2023-12-27 08:20:49,716][20381] Updated weights for policy 1, policy_version 391068 (0.0008) [2023-12-27 08:20:49,745][20347] Updated weights for policy 0, policy_version 391300 (0.0008) [2023-12-27 08:20:49,773][20381] Updated weights for policy 1, policy_version 391078 (0.0007) [2023-12-27 08:20:49,797][20347] Updated weights for policy 0, policy_version 391310 (0.0006) [2023-12-27 08:20:49,823][20381] Updated weights for policy 1, policy_version 391088 (0.0006) [2023-12-27 08:20:49,857][20347] Updated weights for policy 0, policy_version 391320 (0.0008) [2023-12-27 08:20:50,466][20347] Updated weights for policy 0, policy_version 391330 (0.0008) [2023-12-27 08:20:50,522][20347] Updated weights for policy 0, policy_version 391340 (0.0005) [2023-12-27 08:20:50,583][20347] Updated weights for policy 0, policy_version 391350 (0.0007) [2023-12-27 08:20:50,623][20381] Updated weights for policy 1, policy_version 391098 (0.0007) [2023-12-27 08:20:50,639][20347] Updated weights for policy 0, policy_version 391360 (0.0006) [2023-12-27 08:20:50,710][20381] Updated weights for policy 1, policy_version 391108 (0.0008) [2023-12-27 08:20:50,773][20381] Updated weights for policy 1, policy_version 391118 (0.0008) [2023-12-27 08:20:51,240][20347] Updated weights for policy 0, policy_version 391370 (0.0008) [2023-12-27 08:20:51,297][20347] Updated weights for policy 0, policy_version 391380 (0.0008) [2023-12-27 08:20:51,360][20347] Updated weights for policy 0, policy_version 391390 (0.0008) [2023-12-27 08:20:51,461][20381] Updated weights for policy 1, policy_version 391128 (0.0007) [2023-12-27 08:20:51,521][20381] Updated weights for policy 1, policy_version 391138 (0.0007) [2023-12-27 08:20:51,587][20381] Updated weights for policy 1, policy_version 391148 (0.0009) [2023-12-27 08:20:52,024][20347] Updated weights for policy 0, policy_version 391400 (0.0007) [2023-12-27 08:20:52,074][20347] Updated weights for policy 0, policy_version 391410 (0.0008) [2023-12-27 08:20:52,126][20347] Updated weights for policy 0, policy_version 391420 (0.0008) [2023-12-27 08:20:52,281][20381] Updated weights for policy 1, policy_version 391158 (0.0009) [2023-12-27 08:20:52,335][20381] Updated weights for policy 1, policy_version 391168 (0.0007) [2023-12-27 08:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 200376320. Throughput: 0: 10643.8, 1: 10393.6. Samples: 200347120. Policy #0 lag: (min: 19.0, avg: 28.0, max: 51.0) [2023-12-27 08:20:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000391424_100220928.pth... [2023-12-27 08:20:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000390176_99901440.pth [2023-12-27 08:20:52,394][20381] Updated weights for policy 1, policy_version 391178 (0.0007) [2023-12-27 08:20:52,420][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000391184_100163584.pth... [2023-12-27 08:20:52,423][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000389936_99844096.pth [2023-12-27 08:20:52,838][20347] Updated weights for policy 0, policy_version 391430 (0.0006) [2023-12-27 08:20:52,894][20347] Updated weights for policy 0, policy_version 391440 (0.0005) [2023-12-27 08:20:52,969][20347] Updated weights for policy 0, policy_version 391450 (0.0006) [2023-12-27 08:20:53,063][20381] Updated weights for policy 1, policy_version 391188 (0.0009) [2023-12-27 08:20:53,110][20381] Updated weights for policy 1, policy_version 391198 (0.0008) [2023-12-27 08:20:53,167][20381] Updated weights for policy 1, policy_version 391208 (0.0008) [2023-12-27 08:20:53,585][20347] Updated weights for policy 0, policy_version 391460 (0.0008) [2023-12-27 08:20:53,630][20347] Updated weights for policy 0, policy_version 391470 (0.0007) [2023-12-27 08:20:53,678][20347] Updated weights for policy 0, policy_version 391480 (0.0007) [2023-12-27 08:20:53,878][20381] Updated weights for policy 1, policy_version 391218 (0.0007) [2023-12-27 08:20:53,932][20381] Updated weights for policy 1, policy_version 391228 (0.0008) [2023-12-27 08:20:53,990][20381] Updated weights for policy 1, policy_version 391238 (0.0008) [2023-12-27 08:20:54,045][20381] Updated weights for policy 1, policy_version 391248 (0.0007) [2023-12-27 08:20:54,369][20347] Updated weights for policy 0, policy_version 391490 (0.0008) [2023-12-27 08:20:54,418][20347] Updated weights for policy 0, policy_version 391500 (0.0010) [2023-12-27 08:20:54,474][20347] Updated weights for policy 0, policy_version 391510 (0.0010) [2023-12-27 08:20:54,532][20347] Updated weights for policy 0, policy_version 391520 (0.0011) [2023-12-27 08:20:54,757][20381] Updated weights for policy 1, policy_version 391258 (0.0007) [2023-12-27 08:20:54,816][20381] Updated weights for policy 1, policy_version 391268 (0.0008) [2023-12-27 08:20:54,868][20381] Updated weights for policy 1, policy_version 391278 (0.0007) [2023-12-27 08:20:55,183][20347] Updated weights for policy 0, policy_version 391530 (0.0010) [2023-12-27 08:20:55,241][20347] Updated weights for policy 0, policy_version 391540 (0.0010) [2023-12-27 08:20:55,294][20347] Updated weights for policy 0, policy_version 391550 (0.0005) [2023-12-27 08:20:55,536][20381] Updated weights for policy 1, policy_version 391288 (0.0008) [2023-12-27 08:20:55,597][20381] Updated weights for policy 1, policy_version 391298 (0.0007) [2023-12-27 08:20:55,650][20381] Updated weights for policy 1, policy_version 391308 (0.0007) [2023-12-27 08:20:55,937][20347] Updated weights for policy 0, policy_version 391560 (0.0006) [2023-12-27 08:20:55,995][20347] Updated weights for policy 0, policy_version 391570 (0.0008) [2023-12-27 08:20:56,050][20347] Updated weights for policy 0, policy_version 391580 (0.0007) [2023-12-27 08:20:56,338][20381] Updated weights for policy 1, policy_version 391318 (0.0006) [2023-12-27 08:20:56,391][20381] Updated weights for policy 1, policy_version 391328 (0.0005) [2023-12-27 08:20:56,451][20381] Updated weights for policy 1, policy_version 391338 (0.0006) [2023-12-27 08:20:56,748][20347] Updated weights for policy 0, policy_version 391590 (0.0007) [2023-12-27 08:20:56,804][20347] Updated weights for policy 0, policy_version 391600 (0.0008) [2023-12-27 08:20:56,855][20347] Updated weights for policy 0, policy_version 391610 (0.0008) [2023-12-27 08:20:57,083][20381] Updated weights for policy 1, policy_version 391348 (0.0008) [2023-12-27 08:20:57,129][20381] Updated weights for policy 1, policy_version 391358 (0.0008) [2023-12-27 08:20:57,177][20381] Updated weights for policy 1, policy_version 391368 (0.0008) [2023-12-27 08:20:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 200482816. Throughput: 0: 10617.4, 1: 10408.4. Samples: 200470680. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:20:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:20:57,528][20347] Updated weights for policy 0, policy_version 391620 (0.0008) [2023-12-27 08:20:57,584][20347] Updated weights for policy 0, policy_version 391630 (0.0007) [2023-12-27 08:20:57,643][20347] Updated weights for policy 0, policy_version 391640 (0.0008) [2023-12-27 08:20:57,883][20381] Updated weights for policy 1, policy_version 391378 (0.0007) [2023-12-27 08:20:57,937][20381] Updated weights for policy 1, policy_version 391388 (0.0007) [2023-12-27 08:20:57,993][20381] Updated weights for policy 1, policy_version 391398 (0.0008) [2023-12-27 08:20:58,044][20381] Updated weights for policy 1, policy_version 391408 (0.0008) [2023-12-27 08:20:58,269][20347] Updated weights for policy 0, policy_version 391650 (0.0007) [2023-12-27 08:20:58,332][20347] Updated weights for policy 0, policy_version 391660 (0.0007) [2023-12-27 08:20:58,397][20347] Updated weights for policy 0, policy_version 391670 (0.0009) [2023-12-27 08:20:58,461][20347] Updated weights for policy 0, policy_version 391680 (0.0008) [2023-12-27 08:20:58,826][20381] Updated weights for policy 1, policy_version 391418 (0.0008) [2023-12-27 08:20:58,893][20381] Updated weights for policy 1, policy_version 391428 (0.0010) [2023-12-27 08:20:58,953][20381] Updated weights for policy 1, policy_version 391438 (0.0010) [2023-12-27 08:20:59,264][20347] Updated weights for policy 0, policy_version 391690 (0.0008) [2023-12-27 08:20:59,315][20347] Updated weights for policy 0, policy_version 391700 (0.0007) [2023-12-27 08:20:59,380][20347] Updated weights for policy 0, policy_version 391710 (0.0007) [2023-12-27 08:20:59,653][20381] Updated weights for policy 1, policy_version 391448 (0.0008) [2023-12-27 08:20:59,702][20381] Updated weights for policy 1, policy_version 391458 (0.0008) [2023-12-27 08:20:59,754][20381] Updated weights for policy 1, policy_version 391468 (0.0006) [2023-12-27 08:21:00,062][20347] Updated weights for policy 0, policy_version 391720 (0.0008) [2023-12-27 08:21:00,124][20347] Updated weights for policy 0, policy_version 391730 (0.0007) [2023-12-27 08:21:00,185][20347] Updated weights for policy 0, policy_version 391740 (0.0008) [2023-12-27 08:21:00,414][20381] Updated weights for policy 1, policy_version 391478 (0.0007) [2023-12-27 08:21:00,462][20381] Updated weights for policy 1, policy_version 391488 (0.0006) [2023-12-27 08:21:00,518][20381] Updated weights for policy 1, policy_version 391498 (0.0007) [2023-12-27 08:21:00,858][20347] Updated weights for policy 0, policy_version 391750 (0.0007) [2023-12-27 08:21:00,913][20347] Updated weights for policy 0, policy_version 391760 (0.0007) [2023-12-27 08:21:00,973][20347] Updated weights for policy 0, policy_version 391770 (0.0007) [2023-12-27 08:21:01,212][20381] Updated weights for policy 1, policy_version 391508 (0.0008) [2023-12-27 08:21:01,265][20381] Updated weights for policy 1, policy_version 391518 (0.0007) [2023-12-27 08:21:01,320][20381] Updated weights for policy 1, policy_version 391528 (0.0008) [2023-12-27 08:21:01,676][20347] Updated weights for policy 0, policy_version 391780 (0.0008) [2023-12-27 08:21:01,732][20347] Updated weights for policy 0, policy_version 391790 (0.0007) [2023-12-27 08:21:01,794][20347] Updated weights for policy 0, policy_version 391800 (0.0007) [2023-12-27 08:21:01,959][20381] Updated weights for policy 1, policy_version 391538 (0.0008) [2023-12-27 08:21:02,011][20381] Updated weights for policy 1, policy_version 391548 (0.0008) [2023-12-27 08:21:02,074][20381] Updated weights for policy 1, policy_version 391558 (0.0007) [2023-12-27 08:21:02,130][20381] Updated weights for policy 1, policy_version 391568 (0.0011) [2023-12-27 08:21:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 200581120. Throughput: 0: 10510.9, 1: 10340.3. Samples: 200591168. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:02,505][20347] Updated weights for policy 0, policy_version 391810 (0.0007) [2023-12-27 08:21:02,555][20347] Updated weights for policy 0, policy_version 391820 (0.0007) [2023-12-27 08:21:02,606][20347] Updated weights for policy 0, policy_version 391830 (0.0007) [2023-12-27 08:21:02,653][20347] Updated weights for policy 0, policy_version 391840 (0.0007) [2023-12-27 08:21:02,767][20381] Updated weights for policy 1, policy_version 391578 (0.0009) [2023-12-27 08:21:02,817][20381] Updated weights for policy 1, policy_version 391588 (0.0007) [2023-12-27 08:21:02,870][20381] Updated weights for policy 1, policy_version 391598 (0.0008) [2023-12-27 08:21:03,297][20347] Updated weights for policy 0, policy_version 391850 (0.0005) [2023-12-27 08:21:03,344][20347] Updated weights for policy 0, policy_version 391860 (0.0005) [2023-12-27 08:21:03,388][20347] Updated weights for policy 0, policy_version 391870 (0.0005) [2023-12-27 08:21:03,569][20381] Updated weights for policy 1, policy_version 391608 (0.0006) [2023-12-27 08:21:03,627][20381] Updated weights for policy 1, policy_version 391618 (0.0005) [2023-12-27 08:21:03,683][20381] Updated weights for policy 1, policy_version 391628 (0.0005) [2023-12-27 08:21:03,970][20347] Updated weights for policy 0, policy_version 391880 (0.0005) [2023-12-27 08:21:04,026][20347] Updated weights for policy 0, policy_version 391890 (0.0007) [2023-12-27 08:21:04,084][20347] Updated weights for policy 0, policy_version 391900 (0.0007) [2023-12-27 08:21:04,330][20381] Updated weights for policy 1, policy_version 391638 (0.0007) [2023-12-27 08:21:04,394][20381] Updated weights for policy 1, policy_version 391648 (0.0009) [2023-12-27 08:21:04,459][20381] Updated weights for policy 1, policy_version 391658 (0.0008) [2023-12-27 08:21:04,706][20347] Updated weights for policy 0, policy_version 391910 (0.0007) [2023-12-27 08:21:04,757][20347] Updated weights for policy 0, policy_version 391920 (0.0006) [2023-12-27 08:21:04,816][20347] Updated weights for policy 0, policy_version 391930 (0.0007) [2023-12-27 08:21:05,151][20381] Updated weights for policy 1, policy_version 391668 (0.0007) [2023-12-27 08:21:05,201][20381] Updated weights for policy 1, policy_version 391678 (0.0005) [2023-12-27 08:21:05,247][20381] Updated weights for policy 1, policy_version 391688 (0.0005) [2023-12-27 08:21:05,436][20347] Updated weights for policy 0, policy_version 391940 (0.0008) [2023-12-27 08:21:05,494][20347] Updated weights for policy 0, policy_version 391950 (0.0007) [2023-12-27 08:21:05,552][20347] Updated weights for policy 0, policy_version 391960 (0.0007) [2023-12-27 08:21:05,906][20381] Updated weights for policy 1, policy_version 391698 (0.0007) [2023-12-27 08:21:05,960][20381] Updated weights for policy 1, policy_version 391708 (0.0008) [2023-12-27 08:21:06,009][20381] Updated weights for policy 1, policy_version 391718 (0.0006) [2023-12-27 08:21:06,065][20381] Updated weights for policy 1, policy_version 391728 (0.0008) [2023-12-27 08:21:06,262][20347] Updated weights for policy 0, policy_version 391970 (0.0007) [2023-12-27 08:21:06,318][20347] Updated weights for policy 0, policy_version 391980 (0.0007) [2023-12-27 08:21:06,378][20347] Updated weights for policy 0, policy_version 391990 (0.0008) [2023-12-27 08:21:06,431][20347] Updated weights for policy 0, policy_version 392000 (0.0007) [2023-12-27 08:21:06,775][20381] Updated weights for policy 1, policy_version 391738 (0.0008) [2023-12-27 08:21:06,831][20381] Updated weights for policy 1, policy_version 391748 (0.0008) [2023-12-27 08:21:06,881][20381] Updated weights for policy 1, policy_version 391758 (0.0008) [2023-12-27 08:21:07,098][20347] Updated weights for policy 0, policy_version 392010 (0.0008) [2023-12-27 08:21:07,150][20347] Updated weights for policy 0, policy_version 392020 (0.0007) [2023-12-27 08:21:07,196][20347] Updated weights for policy 0, policy_version 392030 (0.0008) [2023-12-27 08:21:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 200687616. Throughput: 0: 10510.3, 1: 10350.3. Samples: 200654972. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000392032_100376576.pth... [2023-12-27 08:21:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000391760_100311040.pth... [2023-12-27 08:21:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000390784_100057088.pth [2023-12-27 08:21:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000390576_100007936.pth [2023-12-27 08:21:07,522][20381] Updated weights for policy 1, policy_version 391768 (0.0006) [2023-12-27 08:21:07,579][20381] Updated weights for policy 1, policy_version 391778 (0.0007) [2023-12-27 08:21:07,628][20381] Updated weights for policy 1, policy_version 391788 (0.0008) [2023-12-27 08:21:07,878][20347] Updated weights for policy 0, policy_version 392040 (0.0008) [2023-12-27 08:21:07,932][20347] Updated weights for policy 0, policy_version 392050 (0.0007) [2023-12-27 08:21:07,983][20347] Updated weights for policy 0, policy_version 392060 (0.0007) [2023-12-27 08:21:08,312][20381] Updated weights for policy 1, policy_version 391798 (0.0008) [2023-12-27 08:21:08,376][20381] Updated weights for policy 1, policy_version 391808 (0.0008) [2023-12-27 08:21:08,429][20381] Updated weights for policy 1, policy_version 391818 (0.0008) [2023-12-27 08:21:08,688][20347] Updated weights for policy 0, policy_version 392070 (0.0008) [2023-12-27 08:21:08,744][20347] Updated weights for policy 0, policy_version 392080 (0.0008) [2023-12-27 08:21:08,800][20347] Updated weights for policy 0, policy_version 392090 (0.0009) [2023-12-27 08:21:09,129][20381] Updated weights for policy 1, policy_version 391828 (0.0007) [2023-12-27 08:21:09,185][20381] Updated weights for policy 1, policy_version 391838 (0.0007) [2023-12-27 08:21:09,249][20381] Updated weights for policy 1, policy_version 391848 (0.0008) [2023-12-27 08:21:09,532][20347] Updated weights for policy 0, policy_version 392100 (0.0008) [2023-12-27 08:21:09,594][20347] Updated weights for policy 0, policy_version 392110 (0.0007) [2023-12-27 08:21:09,660][20347] Updated weights for policy 0, policy_version 392120 (0.0007) [2023-12-27 08:21:09,998][20381] Updated weights for policy 1, policy_version 391858 (0.0008) [2023-12-27 08:21:10,061][20381] Updated weights for policy 1, policy_version 391868 (0.0008) [2023-12-27 08:21:10,124][20381] Updated weights for policy 1, policy_version 391878 (0.0009) [2023-12-27 08:21:10,179][20381] Updated weights for policy 1, policy_version 391888 (0.0008) [2023-12-27 08:21:10,281][20347] Updated weights for policy 0, policy_version 392130 (0.0006) [2023-12-27 08:21:10,340][20347] Updated weights for policy 0, policy_version 392140 (0.0007) [2023-12-27 08:21:10,392][20347] Updated weights for policy 0, policy_version 392150 (0.0007) [2023-12-27 08:21:10,456][20347] Updated weights for policy 0, policy_version 392160 (0.0008) [2023-12-27 08:21:10,883][20381] Updated weights for policy 1, policy_version 391898 (0.0009) [2023-12-27 08:21:10,933][20381] Updated weights for policy 1, policy_version 391908 (0.0008) [2023-12-27 08:21:10,987][20381] Updated weights for policy 1, policy_version 391918 (0.0006) [2023-12-27 08:21:11,131][20347] Updated weights for policy 0, policy_version 392170 (0.0007) [2023-12-27 08:21:11,193][20347] Updated weights for policy 0, policy_version 392180 (0.0009) [2023-12-27 08:21:11,254][20347] Updated weights for policy 0, policy_version 392190 (0.0008) [2023-12-27 08:21:11,744][20381] Updated weights for policy 1, policy_version 391928 (0.0008) [2023-12-27 08:21:11,806][20381] Updated weights for policy 1, policy_version 391938 (0.0008) [2023-12-27 08:21:11,870][20381] Updated weights for policy 1, policy_version 391948 (0.0009) [2023-12-27 08:21:11,935][20347] Updated weights for policy 0, policy_version 392200 (0.0007) [2023-12-27 08:21:11,996][20347] Updated weights for policy 0, policy_version 392210 (0.0007) [2023-12-27 08:21:12,061][20347] Updated weights for policy 0, policy_version 392220 (0.0007) [2023-12-27 08:21:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 200785920. Throughput: 0: 10469.5, 1: 10300.6. Samples: 200776628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:12,595][20381] Updated weights for policy 1, policy_version 391958 (0.0011) [2023-12-27 08:21:12,658][20381] Updated weights for policy 1, policy_version 391968 (0.0010) [2023-12-27 08:21:12,721][20381] Updated weights for policy 1, policy_version 391978 (0.0005) [2023-12-27 08:21:12,785][20347] Updated weights for policy 0, policy_version 392230 (0.0008) [2023-12-27 08:21:12,850][20347] Updated weights for policy 0, policy_version 392240 (0.0007) [2023-12-27 08:21:12,909][20347] Updated weights for policy 0, policy_version 392250 (0.0006) [2023-12-27 08:21:13,327][20381] Updated weights for policy 1, policy_version 391988 (0.0006) [2023-12-27 08:21:13,381][20381] Updated weights for policy 1, policy_version 391998 (0.0008) [2023-12-27 08:21:13,430][20381] Updated weights for policy 1, policy_version 392008 (0.0007) [2023-12-27 08:21:13,546][20347] Updated weights for policy 0, policy_version 392260 (0.0008) [2023-12-27 08:21:13,601][20347] Updated weights for policy 0, policy_version 392270 (0.0008) [2023-12-27 08:21:13,650][20347] Updated weights for policy 0, policy_version 392280 (0.0007) [2023-12-27 08:21:14,107][20381] Updated weights for policy 1, policy_version 392018 (0.0008) [2023-12-27 08:21:14,162][20381] Updated weights for policy 1, policy_version 392028 (0.0006) [2023-12-27 08:21:14,215][20381] Updated weights for policy 1, policy_version 392038 (0.0006) [2023-12-27 08:21:14,265][20381] Updated weights for policy 1, policy_version 392048 (0.0005) [2023-12-27 08:21:14,348][20347] Updated weights for policy 0, policy_version 392290 (0.0007) [2023-12-27 08:21:14,389][20291] KL-divergence is very high: 683.1100 [2023-12-27 08:21:14,400][20347] Updated weights for policy 0, policy_version 392300 (0.0005) [2023-12-27 08:21:14,428][20291] KL-divergence is very high: 533.1257 [2023-12-27 08:21:14,448][20347] Updated weights for policy 0, policy_version 392310 (0.0006) [2023-12-27 08:21:14,468][20291] KL-divergence is very high: 120.0781 [2023-12-27 08:21:14,496][20347] Updated weights for policy 0, policy_version 392320 (0.0006) [2023-12-27 08:21:14,892][20381] Updated weights for policy 1, policy_version 392058 (0.0006) [2023-12-27 08:21:14,954][20381] Updated weights for policy 1, policy_version 392068 (0.0007) [2023-12-27 08:21:15,014][20381] Updated weights for policy 1, policy_version 392078 (0.0008) [2023-12-27 08:21:15,119][20347] Updated weights for policy 0, policy_version 392330 (0.0007) [2023-12-27 08:21:15,176][20347] Updated weights for policy 0, policy_version 392340 (0.0007) [2023-12-27 08:21:15,228][20347] Updated weights for policy 0, policy_version 392350 (0.0006) [2023-12-27 08:21:15,702][20381] Updated weights for policy 1, policy_version 392088 (0.0009) [2023-12-27 08:21:15,749][20381] Updated weights for policy 1, policy_version 392098 (0.0010) [2023-12-27 08:21:15,778][20347] Updated weights for policy 0, policy_version 392360 (0.0005) [2023-12-27 08:21:15,798][20381] Updated weights for policy 1, policy_version 392108 (0.0010) [2023-12-27 08:21:15,832][20347] Updated weights for policy 0, policy_version 392370 (0.0005) [2023-12-27 08:21:15,890][20347] Updated weights for policy 0, policy_version 392380 (0.0005) [2023-12-27 08:21:16,448][20381] Updated weights for policy 1, policy_version 392118 (0.0007) [2023-12-27 08:21:16,474][20347] Updated weights for policy 0, policy_version 392390 (0.0007) [2023-12-27 08:21:16,503][20381] Updated weights for policy 1, policy_version 392128 (0.0005) [2023-12-27 08:21:16,527][20347] Updated weights for policy 0, policy_version 392400 (0.0007) [2023-12-27 08:21:16,555][20381] Updated weights for policy 1, policy_version 392138 (0.0007) [2023-12-27 08:21:16,572][20347] Updated weights for policy 0, policy_version 392410 (0.0006) [2023-12-27 08:21:17,136][20381] Updated weights for policy 1, policy_version 392148 (0.0006) [2023-12-27 08:21:17,158][20347] Updated weights for policy 0, policy_version 392420 (0.0007) [2023-12-27 08:21:17,193][20381] Updated weights for policy 1, policy_version 392158 (0.0005) [2023-12-27 08:21:17,217][20347] Updated weights for policy 0, policy_version 392430 (0.0007) [2023-12-27 08:21:17,251][20381] Updated weights for policy 1, policy_version 392168 (0.0007) [2023-12-27 08:21:17,268][20347] Updated weights for policy 0, policy_version 392440 (0.0005) [2023-12-27 08:21:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 200900608. Throughput: 0: 10533.0, 1: 10227.7. Samples: 200904412. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:17,363][19170] Avg episode reward: [(0, '990.540'), (1, '990.410')] [2023-12-27 08:21:17,853][20381] Updated weights for policy 1, policy_version 392178 (0.0008) [2023-12-27 08:21:17,883][20347] Updated weights for policy 0, policy_version 392450 (0.0006) [2023-12-27 08:21:17,914][20381] Updated weights for policy 1, policy_version 392188 (0.0011) [2023-12-27 08:21:17,936][20347] Updated weights for policy 0, policy_version 392460 (0.0006) [2023-12-27 08:21:17,963][20381] Updated weights for policy 1, policy_version 392198 (0.0011) [2023-12-27 08:21:17,988][20347] Updated weights for policy 0, policy_version 392470 (0.0005) [2023-12-27 08:21:18,014][20381] Updated weights for policy 1, policy_version 392208 (0.0010) [2023-12-27 08:21:18,042][20347] Updated weights for policy 0, policy_version 392480 (0.0006) [2023-12-27 08:21:18,716][20381] Updated weights for policy 1, policy_version 392218 (0.0007) [2023-12-27 08:21:18,755][20347] Updated weights for policy 0, policy_version 392490 (0.0008) [2023-12-27 08:21:18,779][20381] Updated weights for policy 1, policy_version 392228 (0.0006) [2023-12-27 08:21:18,807][20347] Updated weights for policy 0, policy_version 392500 (0.0008) [2023-12-27 08:21:18,831][20381] Updated weights for policy 1, policy_version 392238 (0.0005) [2023-12-27 08:21:18,868][20347] Updated weights for policy 0, policy_version 392510 (0.0009) [2023-12-27 08:21:19,540][20381] Updated weights for policy 1, policy_version 392248 (0.0007) [2023-12-27 08:21:19,604][20381] Updated weights for policy 1, policy_version 392258 (0.0007) [2023-12-27 08:21:19,623][20347] Updated weights for policy 0, policy_version 392520 (0.0007) [2023-12-27 08:21:19,668][20381] Updated weights for policy 1, policy_version 392268 (0.0006) [2023-12-27 08:21:19,681][20347] Updated weights for policy 0, policy_version 392530 (0.0005) [2023-12-27 08:21:19,748][20347] Updated weights for policy 0, policy_version 392540 (0.0005) [2023-12-27 08:21:20,327][20347] Updated weights for policy 0, policy_version 392550 (0.0007) [2023-12-27 08:21:20,374][20381] Updated weights for policy 1, policy_version 392278 (0.0007) [2023-12-27 08:21:20,386][20347] Updated weights for policy 0, policy_version 392560 (0.0007) [2023-12-27 08:21:20,435][20381] Updated weights for policy 1, policy_version 392288 (0.0006) [2023-12-27 08:21:20,444][20347] Updated weights for policy 0, policy_version 392570 (0.0007) [2023-12-27 08:21:20,494][20381] Updated weights for policy 1, policy_version 392298 (0.0005) [2023-12-27 08:21:21,167][20347] Updated weights for policy 0, policy_version 392580 (0.0008) [2023-12-27 08:21:21,176][20381] Updated weights for policy 1, policy_version 392308 (0.0007) [2023-12-27 08:21:21,220][20347] Updated weights for policy 0, policy_version 392590 (0.0005) [2023-12-27 08:21:21,242][20381] Updated weights for policy 1, policy_version 392318 (0.0008) [2023-12-27 08:21:21,286][20347] Updated weights for policy 0, policy_version 392600 (0.0007) [2023-12-27 08:21:21,305][20381] Updated weights for policy 1, policy_version 392328 (0.0008) [2023-12-27 08:21:21,982][20381] Updated weights for policy 1, policy_version 392338 (0.0008) [2023-12-27 08:21:21,991][20347] Updated weights for policy 0, policy_version 392610 (0.0007) [2023-12-27 08:21:22,039][20381] Updated weights for policy 1, policy_version 392348 (0.0007) [2023-12-27 08:21:22,050][20347] Updated weights for policy 0, policy_version 392620 (0.0008) [2023-12-27 08:21:22,097][20381] Updated weights for policy 1, policy_version 392358 (0.0007) [2023-12-27 08:21:22,106][20347] Updated weights for policy 0, policy_version 392630 (0.0006) [2023-12-27 08:21:22,150][20381] Updated weights for policy 1, policy_version 392368 (0.0006) [2023-12-27 08:21:22,167][20347] Updated weights for policy 0, policy_version 392640 (0.0007) [2023-12-27 08:21:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 200998912. Throughput: 0: 10517.6, 1: 10201.0. Samples: 200965708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:22,363][19170] Avg episode reward: [(0, '990.540'), (1, '990.410')] [2023-12-27 08:21:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000392368_100466688.pth... [2023-12-27 08:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000392640_100532224.pth... [2023-12-27 08:21:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000391184_100163584.pth [2023-12-27 08:21:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000391424_100220928.pth [2023-12-27 08:21:22,810][20381] Updated weights for policy 1, policy_version 392378 (0.0005) [2023-12-27 08:21:22,866][20381] Updated weights for policy 1, policy_version 392388 (0.0006) [2023-12-27 08:21:22,900][20347] Updated weights for policy 0, policy_version 392650 (0.0007) [2023-12-27 08:21:22,928][20381] Updated weights for policy 1, policy_version 392398 (0.0006) [2023-12-27 08:21:22,961][20347] Updated weights for policy 0, policy_version 392660 (0.0008) [2023-12-27 08:21:23,015][20347] Updated weights for policy 0, policy_version 392670 (0.0008) [2023-12-27 08:21:23,503][20381] Updated weights for policy 1, policy_version 392408 (0.0006) [2023-12-27 08:21:23,552][20381] Updated weights for policy 1, policy_version 392418 (0.0007) [2023-12-27 08:21:23,602][20381] Updated weights for policy 1, policy_version 392428 (0.0008) [2023-12-27 08:21:23,737][20347] Updated weights for policy 0, policy_version 392680 (0.0008) [2023-12-27 08:21:23,793][20347] Updated weights for policy 0, policy_version 392690 (0.0008) [2023-12-27 08:21:23,849][20347] Updated weights for policy 0, policy_version 392700 (0.0007) [2023-12-27 08:21:24,270][20381] Updated weights for policy 1, policy_version 392438 (0.0006) [2023-12-27 08:21:24,326][20381] Updated weights for policy 1, policy_version 392448 (0.0005) [2023-12-27 08:21:24,379][20381] Updated weights for policy 1, policy_version 392458 (0.0008) [2023-12-27 08:21:24,550][20347] Updated weights for policy 0, policy_version 392710 (0.0007) [2023-12-27 08:21:24,609][20347] Updated weights for policy 0, policy_version 392720 (0.0006) [2023-12-27 08:21:24,665][20347] Updated weights for policy 0, policy_version 392730 (0.0005) [2023-12-27 08:21:25,086][20381] Updated weights for policy 1, policy_version 392468 (0.0008) [2023-12-27 08:21:25,132][20381] Updated weights for policy 1, policy_version 392478 (0.0008) [2023-12-27 08:21:25,181][20381] Updated weights for policy 1, policy_version 392488 (0.0008) [2023-12-27 08:21:25,291][20347] Updated weights for policy 0, policy_version 392740 (0.0006) [2023-12-27 08:21:25,336][20347] Updated weights for policy 0, policy_version 392750 (0.0007) [2023-12-27 08:21:25,381][20347] Updated weights for policy 0, policy_version 392760 (0.0008) [2023-12-27 08:21:25,893][20381] Updated weights for policy 1, policy_version 392498 (0.0008) [2023-12-27 08:21:25,949][20381] Updated weights for policy 1, policy_version 392508 (0.0007) [2023-12-27 08:21:26,008][20381] Updated weights for policy 1, policy_version 392518 (0.0007) [2023-12-27 08:21:26,066][20347] Updated weights for policy 0, policy_version 392770 (0.0007) [2023-12-27 08:21:26,070][20381] Updated weights for policy 1, policy_version 392528 (0.0008) [2023-12-27 08:21:26,118][20347] Updated weights for policy 0, policy_version 392780 (0.0007) [2023-12-27 08:21:26,171][20347] Updated weights for policy 0, policy_version 392790 (0.0005) [2023-12-27 08:21:26,227][20347] Updated weights for policy 0, policy_version 392800 (0.0005) [2023-12-27 08:21:26,708][20381] Updated weights for policy 1, policy_version 392538 (0.0008) [2023-12-27 08:21:26,761][20381] Updated weights for policy 1, policy_version 392548 (0.0008) [2023-12-27 08:21:26,820][20381] Updated weights for policy 1, policy_version 392558 (0.0007) [2023-12-27 08:21:26,842][20347] Updated weights for policy 0, policy_version 392810 (0.0007) [2023-12-27 08:21:26,898][20347] Updated weights for policy 0, policy_version 392820 (0.0007) [2023-12-27 08:21:26,948][20347] Updated weights for policy 0, policy_version 392830 (0.0008) [2023-12-27 08:21:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 201097216. Throughput: 0: 10505.9, 1: 10197.4. Samples: 201087652. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:27,492][20381] Updated weights for policy 1, policy_version 392568 (0.0006) [2023-12-27 08:21:27,550][20381] Updated weights for policy 1, policy_version 392578 (0.0008) [2023-12-27 08:21:27,601][20381] Updated weights for policy 1, policy_version 392588 (0.0008) [2023-12-27 08:21:27,643][20347] Updated weights for policy 0, policy_version 392840 (0.0006) [2023-12-27 08:21:27,699][20347] Updated weights for policy 0, policy_version 392850 (0.0007) [2023-12-27 08:21:27,757][20347] Updated weights for policy 0, policy_version 392860 (0.0007) [2023-12-27 08:21:28,283][20381] Updated weights for policy 1, policy_version 392598 (0.0008) [2023-12-27 08:21:28,342][20381] Updated weights for policy 1, policy_version 392608 (0.0008) [2023-12-27 08:21:28,395][20381] Updated weights for policy 1, policy_version 392618 (0.0008) [2023-12-27 08:21:28,423][20347] Updated weights for policy 0, policy_version 392870 (0.0006) [2023-12-27 08:21:28,478][20347] Updated weights for policy 0, policy_version 392880 (0.0008) [2023-12-27 08:21:28,533][20347] Updated weights for policy 0, policy_version 392890 (0.0007) [2023-12-27 08:21:29,057][20381] Updated weights for policy 1, policy_version 392628 (0.0007) [2023-12-27 08:21:29,112][20381] Updated weights for policy 1, policy_version 392638 (0.0007) [2023-12-27 08:21:29,165][20381] Updated weights for policy 1, policy_version 392648 (0.0007) [2023-12-27 08:21:29,226][20347] Updated weights for policy 0, policy_version 392900 (0.0008) [2023-12-27 08:21:29,278][20347] Updated weights for policy 0, policy_version 392910 (0.0007) [2023-12-27 08:21:29,333][20347] Updated weights for policy 0, policy_version 392920 (0.0007) [2023-12-27 08:21:29,905][20381] Updated weights for policy 1, policy_version 392658 (0.0008) [2023-12-27 08:21:29,972][20381] Updated weights for policy 1, policy_version 392668 (0.0008) [2023-12-27 08:21:29,976][20347] Updated weights for policy 0, policy_version 392930 (0.0008) [2023-12-27 08:21:30,030][20347] Updated weights for policy 0, policy_version 392940 (0.0006) [2023-12-27 08:21:30,039][20381] Updated weights for policy 1, policy_version 392678 (0.0008) [2023-12-27 08:21:30,089][20347] Updated weights for policy 0, policy_version 392950 (0.0006) [2023-12-27 08:21:30,097][20381] Updated weights for policy 1, policy_version 392688 (0.0007) [2023-12-27 08:21:30,146][20347] Updated weights for policy 0, policy_version 392960 (0.0006) [2023-12-27 08:21:30,756][20381] Updated weights for policy 1, policy_version 392698 (0.0006) [2023-12-27 08:21:30,769][20347] Updated weights for policy 0, policy_version 392970 (0.0010) [2023-12-27 08:21:30,807][20381] Updated weights for policy 1, policy_version 392708 (0.0006) [2023-12-27 08:21:30,823][20347] Updated weights for policy 0, policy_version 392980 (0.0010) [2023-12-27 08:21:30,872][20347] Updated weights for policy 0, policy_version 392990 (0.0010) [2023-12-27 08:21:30,872][20381] Updated weights for policy 1, policy_version 392718 (0.0006) [2023-12-27 08:21:31,555][20347] Updated weights for policy 0, policy_version 393000 (0.0008) [2023-12-27 08:21:31,586][20381] Updated weights for policy 1, policy_version 392728 (0.0006) [2023-12-27 08:21:31,609][20347] Updated weights for policy 0, policy_version 393010 (0.0008) [2023-12-27 08:21:31,649][20381] Updated weights for policy 1, policy_version 392738 (0.0008) [2023-12-27 08:21:31,670][20347] Updated weights for policy 0, policy_version 393020 (0.0007) [2023-12-27 08:21:31,716][20381] Updated weights for policy 1, policy_version 392748 (0.0008) [2023-12-27 08:21:32,290][20381] Updated weights for policy 1, policy_version 392758 (0.0005) [2023-12-27 08:21:32,344][20381] Updated weights for policy 1, policy_version 392768 (0.0008) [2023-12-27 08:21:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20716.1). Total num frames: 201195520. Throughput: 0: 10379.0, 1: 10197.3. Samples: 201212060. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:32,400][20347] Updated weights for policy 0, policy_version 393030 (0.0006) [2023-12-27 08:21:32,404][20381] Updated weights for policy 1, policy_version 392778 (0.0011) [2023-12-27 08:21:32,451][20347] Updated weights for policy 0, policy_version 393040 (0.0008) [2023-12-27 08:21:32,501][20347] Updated weights for policy 0, policy_version 393050 (0.0009) [2023-12-27 08:21:33,010][20381] Updated weights for policy 1, policy_version 392788 (0.0009) [2023-12-27 08:21:33,061][20381] Updated weights for policy 1, policy_version 392798 (0.0005) [2023-12-27 08:21:33,111][20381] Updated weights for policy 1, policy_version 392808 (0.0005) [2023-12-27 08:21:33,212][20347] Updated weights for policy 0, policy_version 393060 (0.0009) [2023-12-27 08:21:33,272][20347] Updated weights for policy 0, policy_version 393070 (0.0008) [2023-12-27 08:21:33,319][20347] Updated weights for policy 0, policy_version 393080 (0.0010) [2023-12-27 08:21:33,754][20381] Updated weights for policy 1, policy_version 392818 (0.0007) [2023-12-27 08:21:33,801][20381] Updated weights for policy 1, policy_version 392828 (0.0009) [2023-12-27 08:21:33,856][20381] Updated weights for policy 1, policy_version 392838 (0.0005) [2023-12-27 08:21:33,909][20381] Updated weights for policy 1, policy_version 392848 (0.0005) [2023-12-27 08:21:34,004][20347] Updated weights for policy 0, policy_version 393090 (0.0010) [2023-12-27 08:21:34,048][20347] Updated weights for policy 0, policy_version 393100 (0.0010) [2023-12-27 08:21:34,107][20347] Updated weights for policy 0, policy_version 393110 (0.0009) [2023-12-27 08:21:34,161][20347] Updated weights for policy 0, policy_version 393120 (0.0008) [2023-12-27 08:21:34,516][20381] Updated weights for policy 1, policy_version 392858 (0.0011) [2023-12-27 08:21:34,578][20381] Updated weights for policy 1, policy_version 392868 (0.0011) [2023-12-27 08:21:34,640][20381] Updated weights for policy 1, policy_version 392878 (0.0011) [2023-12-27 08:21:34,803][20347] Updated weights for policy 0, policy_version 393130 (0.0010) [2023-12-27 08:21:34,857][20347] Updated weights for policy 0, policy_version 393140 (0.0010) [2023-12-27 08:21:34,911][20347] Updated weights for policy 0, policy_version 393150 (0.0010) [2023-12-27 08:21:35,318][20381] Updated weights for policy 1, policy_version 392888 (0.0011) [2023-12-27 08:21:35,369][20381] Updated weights for policy 1, policy_version 392898 (0.0010) [2023-12-27 08:21:35,424][20381] Updated weights for policy 1, policy_version 392908 (0.0011) [2023-12-27 08:21:35,477][20347] Updated weights for policy 0, policy_version 393160 (0.0006) [2023-12-27 08:21:35,526][20347] Updated weights for policy 0, policy_version 393170 (0.0006) [2023-12-27 08:21:35,575][20347] Updated weights for policy 0, policy_version 393180 (0.0007) [2023-12-27 08:21:36,102][20381] Updated weights for policy 1, policy_version 392918 (0.0010) [2023-12-27 08:21:36,156][20381] Updated weights for policy 1, policy_version 392928 (0.0009) [2023-12-27 08:21:36,221][20381] Updated weights for policy 1, policy_version 392938 (0.0010) [2023-12-27 08:21:36,255][20347] Updated weights for policy 0, policy_version 393190 (0.0005) [2023-12-27 08:21:36,318][20347] Updated weights for policy 0, policy_version 393200 (0.0005) [2023-12-27 08:21:36,380][20347] Updated weights for policy 0, policy_version 393210 (0.0007) [2023-12-27 08:21:36,916][20381] Updated weights for policy 1, policy_version 392948 (0.0010) [2023-12-27 08:21:36,968][20381] Updated weights for policy 1, policy_version 392958 (0.0008) [2023-12-27 08:21:36,998][20347] Updated weights for policy 0, policy_version 393220 (0.0009) [2023-12-27 08:21:37,018][20381] Updated weights for policy 1, policy_version 392968 (0.0009) [2023-12-27 08:21:37,051][20347] Updated weights for policy 0, policy_version 393230 (0.0010) [2023-12-27 08:21:37,099][20347] Updated weights for policy 0, policy_version 393240 (0.0010) [2023-12-27 08:21:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 201310208. Throughput: 0: 10382.6, 1: 10245.4. Samples: 201275376. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000393248_100687872.pth... [2023-12-27 08:21:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000392976_100622336.pth... [2023-12-27 08:21:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000392032_100376576.pth [2023-12-27 08:21:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000391760_100311040.pth [2023-12-27 08:21:37,697][20381] Updated weights for policy 1, policy_version 392978 (0.0007) [2023-12-27 08:21:37,752][20381] Updated weights for policy 1, policy_version 392988 (0.0011) [2023-12-27 08:21:37,792][20347] Updated weights for policy 0, policy_version 393250 (0.0009) [2023-12-27 08:21:37,806][20381] Updated weights for policy 1, policy_version 392998 (0.0010) [2023-12-27 08:21:37,852][20347] Updated weights for policy 0, policy_version 393260 (0.0005) [2023-12-27 08:21:37,859][20381] Updated weights for policy 1, policy_version 393008 (0.0010) [2023-12-27 08:21:37,907][20347] Updated weights for policy 0, policy_version 393270 (0.0005) [2023-12-27 08:21:37,962][20347] Updated weights for policy 0, policy_version 393280 (0.0005) [2023-12-27 08:21:38,487][20381] Updated weights for policy 1, policy_version 393018 (0.0011) [2023-12-27 08:21:38,538][20381] Updated weights for policy 1, policy_version 393028 (0.0009) [2023-12-27 08:21:38,556][20347] Updated weights for policy 0, policy_version 393290 (0.0008) [2023-12-27 08:21:38,584][20381] Updated weights for policy 1, policy_version 393038 (0.0007) [2023-12-27 08:21:38,614][20347] Updated weights for policy 0, policy_version 393300 (0.0007) [2023-12-27 08:21:38,673][20347] Updated weights for policy 0, policy_version 393310 (0.0008) [2023-12-27 08:21:39,216][20381] Updated weights for policy 1, policy_version 393048 (0.0008) [2023-12-27 08:21:39,274][20381] Updated weights for policy 1, policy_version 393058 (0.0009) [2023-12-27 08:21:39,337][20381] Updated weights for policy 1, policy_version 393068 (0.0009) [2023-12-27 08:21:39,462][20347] Updated weights for policy 0, policy_version 393320 (0.0010) [2023-12-27 08:21:39,521][20347] Updated weights for policy 0, policy_version 393330 (0.0011) [2023-12-27 08:21:39,570][20347] Updated weights for policy 0, policy_version 393340 (0.0010) [2023-12-27 08:21:40,065][20381] Updated weights for policy 1, policy_version 393078 (0.0008) [2023-12-27 08:21:40,125][20381] Updated weights for policy 1, policy_version 393088 (0.0007) [2023-12-27 08:21:40,192][20381] Updated weights for policy 1, policy_version 393098 (0.0007) [2023-12-27 08:21:40,244][20347] Updated weights for policy 0, policy_version 393350 (0.0010) [2023-12-27 08:21:40,297][20347] Updated weights for policy 0, policy_version 393360 (0.0008) [2023-12-27 08:21:40,353][20347] Updated weights for policy 0, policy_version 393370 (0.0008) [2023-12-27 08:21:40,868][20381] Updated weights for policy 1, policy_version 393108 (0.0008) [2023-12-27 08:21:40,925][20381] Updated weights for policy 1, policy_version 393118 (0.0006) [2023-12-27 08:21:40,983][20381] Updated weights for policy 1, policy_version 393128 (0.0005) [2023-12-27 08:21:41,061][20347] Updated weights for policy 0, policy_version 393380 (0.0008) [2023-12-27 08:21:41,121][20347] Updated weights for policy 0, policy_version 393390 (0.0008) [2023-12-27 08:21:41,186][20347] Updated weights for policy 0, policy_version 393400 (0.0009) [2023-12-27 08:21:41,700][20381] Updated weights for policy 1, policy_version 393138 (0.0008) [2023-12-27 08:21:41,765][20381] Updated weights for policy 1, policy_version 393148 (0.0007) [2023-12-27 08:21:41,825][20381] Updated weights for policy 1, policy_version 393158 (0.0007) [2023-12-27 08:21:41,879][20381] Updated weights for policy 1, policy_version 393168 (0.0005) [2023-12-27 08:21:41,930][20347] Updated weights for policy 0, policy_version 393410 (0.0007) [2023-12-27 08:21:41,992][20347] Updated weights for policy 0, policy_version 393420 (0.0008) [2023-12-27 08:21:42,048][20347] Updated weights for policy 0, policy_version 393430 (0.0008) [2023-12-27 08:21:42,106][20347] Updated weights for policy 0, policy_version 393440 (0.0008) [2023-12-27 08:21:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 201408512. Throughput: 0: 10339.2, 1: 10297.8. Samples: 201399344. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:21:42,553][20381] Updated weights for policy 1, policy_version 393178 (0.0008) [2023-12-27 08:21:42,616][20381] Updated weights for policy 1, policy_version 393188 (0.0008) [2023-12-27 08:21:42,680][20381] Updated weights for policy 1, policy_version 393198 (0.0006) [2023-12-27 08:21:42,826][20347] Updated weights for policy 0, policy_version 393450 (0.0008) [2023-12-27 08:21:42,837][20291] KL-divergence is very high: 381.1461 [2023-12-27 08:21:42,882][20347] Updated weights for policy 0, policy_version 393460 (0.0008) [2023-12-27 08:21:42,945][20347] Updated weights for policy 0, policy_version 393470 (0.0008) [2023-12-27 08:21:43,226][20381] Updated weights for policy 1, policy_version 393208 (0.0005) [2023-12-27 08:21:43,282][20381] Updated weights for policy 1, policy_version 393218 (0.0006) [2023-12-27 08:21:43,335][20381] Updated weights for policy 1, policy_version 393228 (0.0005) [2023-12-27 08:21:43,705][20347] Updated weights for policy 0, policy_version 393480 (0.0008) [2023-12-27 08:21:43,752][20347] Updated weights for policy 0, policy_version 393490 (0.0007) [2023-12-27 08:21:43,798][20347] Updated weights for policy 0, policy_version 393500 (0.0008) [2023-12-27 08:21:43,870][20381] Updated weights for policy 1, policy_version 393238 (0.0005) [2023-12-27 08:21:43,921][20381] Updated weights for policy 1, policy_version 393248 (0.0005) [2023-12-27 08:21:43,972][20381] Updated weights for policy 1, policy_version 393258 (0.0005) [2023-12-27 08:21:44,510][20347] Updated weights for policy 0, policy_version 393510 (0.0008) [2023-12-27 08:21:44,567][20347] Updated weights for policy 0, policy_version 393520 (0.0007) [2023-12-27 08:21:44,573][20381] Updated weights for policy 1, policy_version 393268 (0.0006) [2023-12-27 08:21:44,596][20292] KL-divergence is very high: 108.7782 [2023-12-27 08:21:44,619][20347] Updated weights for policy 0, policy_version 393530 (0.0007) [2023-12-27 08:21:44,629][20381] Updated weights for policy 1, policy_version 393278 (0.0005) [2023-12-27 08:21:44,686][20381] Updated weights for policy 1, policy_version 393288 (0.0007) [2023-12-27 08:21:45,312][20347] Updated weights for policy 0, policy_version 393540 (0.0006) [2023-12-27 08:21:45,326][20381] Updated weights for policy 1, policy_version 393298 (0.0007) [2023-12-27 08:21:45,372][20347] Updated weights for policy 0, policy_version 393550 (0.0008) [2023-12-27 08:21:45,385][20381] Updated weights for policy 1, policy_version 393308 (0.0005) [2023-12-27 08:21:45,430][20347] Updated weights for policy 0, policy_version 393560 (0.0007) [2023-12-27 08:21:45,437][20381] Updated weights for policy 1, policy_version 393318 (0.0005) [2023-12-27 08:21:45,491][20381] Updated weights for policy 1, policy_version 393328 (0.0006) [2023-12-27 08:21:46,114][20381] Updated weights for policy 1, policy_version 393338 (0.0008) [2023-12-27 08:21:46,150][20347] Updated weights for policy 0, policy_version 393570 (0.0006) [2023-12-27 08:21:46,165][20381] Updated weights for policy 1, policy_version 393348 (0.0007) [2023-12-27 08:21:46,202][20347] Updated weights for policy 0, policy_version 393580 (0.0005) [2023-12-27 08:21:46,215][20381] Updated weights for policy 1, policy_version 393358 (0.0008) [2023-12-27 08:21:46,258][20347] Updated weights for policy 0, policy_version 393590 (0.0005) [2023-12-27 08:21:46,316][20347] Updated weights for policy 0, policy_version 393600 (0.0007) [2023-12-27 08:21:46,931][20381] Updated weights for policy 1, policy_version 393368 (0.0007) [2023-12-27 08:21:46,977][20347] Updated weights for policy 0, policy_version 393610 (0.0010) [2023-12-27 08:21:46,980][20381] Updated weights for policy 1, policy_version 393378 (0.0009) [2023-12-27 08:21:47,019][20347] Updated weights for policy 0, policy_version 393620 (0.0010) [2023-12-27 08:21:47,036][20381] Updated weights for policy 1, policy_version 393388 (0.0006) [2023-12-27 08:21:47,075][20347] Updated weights for policy 0, policy_version 393630 (0.0010) [2023-12-27 08:21:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 201515008. Throughput: 0: 10329.5, 1: 10397.1. Samples: 201523864. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:47,363][19170] Avg episode reward: [(0, '990.710'), (1, '990.540')] [2023-12-27 08:21:47,574][20381] Updated weights for policy 1, policy_version 393398 (0.0005) [2023-12-27 08:21:47,621][20381] Updated weights for policy 1, policy_version 393408 (0.0005) [2023-12-27 08:21:47,666][20381] Updated weights for policy 1, policy_version 393418 (0.0005) [2023-12-27 08:21:47,802][20347] Updated weights for policy 0, policy_version 393640 (0.0009) [2023-12-27 08:21:47,860][20347] Updated weights for policy 0, policy_version 393650 (0.0008) [2023-12-27 08:21:47,914][20347] Updated weights for policy 0, policy_version 393660 (0.0006) [2023-12-27 08:21:48,257][20381] Updated weights for policy 1, policy_version 393428 (0.0006) [2023-12-27 08:21:48,317][20381] Updated weights for policy 1, policy_version 393438 (0.0009) [2023-12-27 08:21:48,377][20381] Updated weights for policy 1, policy_version 393448 (0.0006) [2023-12-27 08:21:48,639][20347] Updated weights for policy 0, policy_version 393670 (0.0006) [2023-12-27 08:21:48,688][20347] Updated weights for policy 0, policy_version 393680 (0.0008) [2023-12-27 08:21:48,740][20347] Updated weights for policy 0, policy_version 393690 (0.0008) [2023-12-27 08:21:49,002][20381] Updated weights for policy 1, policy_version 393458 (0.0006) [2023-12-27 08:21:49,058][20381] Updated weights for policy 1, policy_version 393468 (0.0008) [2023-12-27 08:21:49,119][20381] Updated weights for policy 1, policy_version 393478 (0.0008) [2023-12-27 08:21:49,170][20381] Updated weights for policy 1, policy_version 393488 (0.0008) [2023-12-27 08:21:49,349][20347] Updated weights for policy 0, policy_version 393700 (0.0009) [2023-12-27 08:21:49,415][20347] Updated weights for policy 0, policy_version 393710 (0.0010) [2023-12-27 08:21:49,473][20347] Updated weights for policy 0, policy_version 393720 (0.0010) [2023-12-27 08:21:49,980][20381] Updated weights for policy 1, policy_version 393498 (0.0007) [2023-12-27 08:21:50,031][20381] Updated weights for policy 1, policy_version 393508 (0.0006) [2023-12-27 08:21:50,086][20381] Updated weights for policy 1, policy_version 393518 (0.0008) [2023-12-27 08:21:50,116][20347] Updated weights for policy 0, policy_version 393730 (0.0009) [2023-12-27 08:21:50,168][20291] KL-divergence is very high: 232.2428 [2023-12-27 08:21:50,174][20347] Updated weights for policy 0, policy_version 393740 (0.0006) [2023-12-27 08:21:50,215][20291] KL-divergence is very high: 179.1745 [2023-12-27 08:21:50,231][20347] Updated weights for policy 0, policy_version 393750 (0.0005) [2023-12-27 08:21:50,286][20347] Updated weights for policy 0, policy_version 393760 (0.0010) [2023-12-27 08:21:50,854][20381] Updated weights for policy 1, policy_version 393528 (0.0009) [2023-12-27 08:21:50,887][20347] Updated weights for policy 0, policy_version 393770 (0.0006) [2023-12-27 08:21:50,924][20381] Updated weights for policy 1, policy_version 393538 (0.0006) [2023-12-27 08:21:50,949][20347] Updated weights for policy 0, policy_version 393780 (0.0008) [2023-12-27 08:21:50,990][20381] Updated weights for policy 1, policy_version 393548 (0.0007) [2023-12-27 08:21:51,010][20347] Updated weights for policy 0, policy_version 393790 (0.0006) [2023-12-27 08:21:51,705][20347] Updated weights for policy 0, policy_version 393800 (0.0008) [2023-12-27 08:21:51,712][20381] Updated weights for policy 1, policy_version 393558 (0.0008) [2023-12-27 08:21:51,768][20381] Updated weights for policy 1, policy_version 393568 (0.0008) [2023-12-27 08:21:51,770][20347] Updated weights for policy 0, policy_version 393810 (0.0008) [2023-12-27 08:21:51,820][20381] Updated weights for policy 1, policy_version 393578 (0.0008) [2023-12-27 08:21:51,824][20347] Updated weights for policy 0, policy_version 393820 (0.0008) [2023-12-27 08:21:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 201613312. Throughput: 0: 10289.0, 1: 10408.7. Samples: 201586372. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:52,363][19170] Avg episode reward: [(0, '980.830'), (1, '990.540')] [2023-12-27 08:21:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000393584_100777984.pth... [2023-12-27 08:21:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000393824_100835328.pth... [2023-12-27 08:21:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000392368_100466688.pth [2023-12-27 08:21:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000392640_100532224.pth [2023-12-27 08:21:52,524][20381] Updated weights for policy 1, policy_version 393588 (0.0007) [2023-12-27 08:21:52,543][20347] Updated weights for policy 0, policy_version 393830 (0.0008) [2023-12-27 08:21:52,566][20291] KL-divergence is very high: 333.9294 [2023-12-27 08:21:52,573][20381] Updated weights for policy 1, policy_version 393598 (0.0008) [2023-12-27 08:21:52,602][20347] Updated weights for policy 0, policy_version 393840 (0.0007) [2023-12-27 08:21:52,620][20381] Updated weights for policy 1, policy_version 393608 (0.0006) [2023-12-27 08:21:52,661][20347] Updated weights for policy 0, policy_version 393850 (0.0007) [2023-12-27 08:21:53,324][20347] Updated weights for policy 0, policy_version 393860 (0.0008) [2023-12-27 08:21:53,328][20291] KL-divergence is very high: 114.5668 [2023-12-27 08:21:53,366][20381] Updated weights for policy 1, policy_version 393618 (0.0008) [2023-12-27 08:21:53,376][20347] Updated weights for policy 0, policy_version 393870 (0.0006) [2023-12-27 08:21:53,422][20381] Updated weights for policy 1, policy_version 393628 (0.0006) [2023-12-27 08:21:53,435][20347] Updated weights for policy 0, policy_version 393880 (0.0007) [2023-12-27 08:21:53,478][20381] Updated weights for policy 1, policy_version 393638 (0.0006) [2023-12-27 08:21:53,537][20381] Updated weights for policy 1, policy_version 393648 (0.0008) [2023-12-27 08:21:54,084][20347] Updated weights for policy 0, policy_version 393890 (0.0006) [2023-12-27 08:21:54,137][20347] Updated weights for policy 0, policy_version 393900 (0.0010) [2023-12-27 08:21:54,189][20347] Updated weights for policy 0, policy_version 393910 (0.0010) [2023-12-27 08:21:54,213][20381] Updated weights for policy 1, policy_version 393658 (0.0005) [2023-12-27 08:21:54,240][20347] Updated weights for policy 0, policy_version 393920 (0.0010) [2023-12-27 08:21:54,269][20381] Updated weights for policy 1, policy_version 393668 (0.0006) [2023-12-27 08:21:54,329][20381] Updated weights for policy 1, policy_version 393678 (0.0007) [2023-12-27 08:21:54,875][20347] Updated weights for policy 0, policy_version 393930 (0.0010) [2023-12-27 08:21:54,928][20347] Updated weights for policy 0, policy_version 393940 (0.0010) [2023-12-27 08:21:54,986][20347] Updated weights for policy 0, policy_version 393950 (0.0010) [2023-12-27 08:21:55,031][20381] Updated weights for policy 1, policy_version 393688 (0.0006) [2023-12-27 08:21:55,080][20381] Updated weights for policy 1, policy_version 393698 (0.0005) [2023-12-27 08:21:55,131][20381] Updated weights for policy 1, policy_version 393708 (0.0005) [2023-12-27 08:21:55,600][20347] Updated weights for policy 0, policy_version 393960 (0.0005) [2023-12-27 08:21:55,654][20347] Updated weights for policy 0, policy_version 393970 (0.0005) [2023-12-27 08:21:55,711][20347] Updated weights for policy 0, policy_version 393980 (0.0009) [2023-12-27 08:21:55,826][20381] Updated weights for policy 1, policy_version 393718 (0.0007) [2023-12-27 08:21:55,874][20381] Updated weights for policy 1, policy_version 393728 (0.0007) [2023-12-27 08:21:55,925][20381] Updated weights for policy 1, policy_version 393738 (0.0008) [2023-12-27 08:21:56,286][20347] Updated weights for policy 0, policy_version 393990 (0.0007) [2023-12-27 08:21:56,345][20347] Updated weights for policy 0, policy_version 394000 (0.0005) [2023-12-27 08:21:56,404][20347] Updated weights for policy 0, policy_version 394010 (0.0007) [2023-12-27 08:21:56,685][20381] Updated weights for policy 1, policy_version 393748 (0.0009) [2023-12-27 08:21:56,734][20381] Updated weights for policy 1, policy_version 393758 (0.0010) [2023-12-27 08:21:56,779][20381] Updated weights for policy 1, policy_version 393768 (0.0009) [2023-12-27 08:21:56,920][20347] Updated weights for policy 0, policy_version 394020 (0.0006) [2023-12-27 08:21:56,978][20347] Updated weights for policy 0, policy_version 394030 (0.0005) [2023-12-27 08:21:57,038][20347] Updated weights for policy 0, policy_version 394040 (0.0005) [2023-12-27 08:21:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 201719808. Throughput: 0: 10323.6, 1: 10373.6. Samples: 201708000. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:21:57,363][19170] Avg episode reward: [(0, '961.460'), (1, '1000.000')] [2023-12-27 08:21:57,486][20381] Updated weights for policy 1, policy_version 393778 (0.0008) [2023-12-27 08:21:57,531][20381] Updated weights for policy 1, policy_version 393788 (0.0005) [2023-12-27 08:21:57,586][20381] Updated weights for policy 1, policy_version 393798 (0.0009) [2023-12-27 08:21:57,635][20381] Updated weights for policy 1, policy_version 393808 (0.0010) [2023-12-27 08:21:57,660][20347] Updated weights for policy 0, policy_version 394050 (0.0006) [2023-12-27 08:21:57,708][20347] Updated weights for policy 0, policy_version 394060 (0.0010) [2023-12-27 08:21:57,764][20347] Updated weights for policy 0, policy_version 394070 (0.0010) [2023-12-27 08:21:57,815][20347] Updated weights for policy 0, policy_version 394080 (0.0010) [2023-12-27 08:21:58,331][20381] Updated weights for policy 1, policy_version 393818 (0.0010) [2023-12-27 08:21:58,397][20381] Updated weights for policy 1, policy_version 393828 (0.0009) [2023-12-27 08:21:58,465][20381] Updated weights for policy 1, policy_version 393838 (0.0008) [2023-12-27 08:21:58,534][20347] Updated weights for policy 0, policy_version 394090 (0.0009) [2023-12-27 08:21:58,597][20347] Updated weights for policy 0, policy_version 394100 (0.0010) [2023-12-27 08:21:58,660][20347] Updated weights for policy 0, policy_version 394110 (0.0011) [2023-12-27 08:21:59,259][20381] Updated weights for policy 1, policy_version 393848 (0.0010) [2023-12-27 08:21:59,322][20381] Updated weights for policy 1, policy_version 393858 (0.0010) [2023-12-27 08:21:59,376][20347] Updated weights for policy 0, policy_version 394120 (0.0009) [2023-12-27 08:21:59,383][20381] Updated weights for policy 1, policy_version 393868 (0.0010) [2023-12-27 08:21:59,432][20347] Updated weights for policy 0, policy_version 394131 (0.0008) [2023-12-27 08:21:59,487][20347] Updated weights for policy 0, policy_version 394141 (0.0012) [2023-12-27 08:22:00,024][20381] Updated weights for policy 1, policy_version 393878 (0.0008) [2023-12-27 08:22:00,087][20381] Updated weights for policy 1, policy_version 393888 (0.0008) [2023-12-27 08:22:00,147][20381] Updated weights for policy 1, policy_version 393898 (0.0008) [2023-12-27 08:22:00,214][20347] Updated weights for policy 0, policy_version 394151 (0.0008) [2023-12-27 08:22:00,266][20347] Updated weights for policy 0, policy_version 394161 (0.0009) [2023-12-27 08:22:00,322][20347] Updated weights for policy 0, policy_version 394171 (0.0008) [2023-12-27 08:22:00,753][20381] Updated weights for policy 1, policy_version 393908 (0.0008) [2023-12-27 08:22:00,800][20381] Updated weights for policy 1, policy_version 393918 (0.0005) [2023-12-27 08:22:00,848][20381] Updated weights for policy 1, policy_version 393928 (0.0005) [2023-12-27 08:22:01,100][20347] Updated weights for policy 0, policy_version 394181 (0.0007) [2023-12-27 08:22:01,162][20347] Updated weights for policy 0, policy_version 394191 (0.0007) [2023-12-27 08:22:01,210][20347] Updated weights for policy 0, policy_version 394201 (0.0007) [2023-12-27 08:22:01,444][20381] Updated weights for policy 1, policy_version 393938 (0.0005) [2023-12-27 08:22:01,511][20381] Updated weights for policy 1, policy_version 393948 (0.0007) [2023-12-27 08:22:01,566][20381] Updated weights for policy 1, policy_version 393958 (0.0008) [2023-12-27 08:22:01,625][20381] Updated weights for policy 1, policy_version 393968 (0.0007) [2023-12-27 08:22:01,938][20347] Updated weights for policy 0, policy_version 394211 (0.0007) [2023-12-27 08:22:01,996][20347] Updated weights for policy 0, policy_version 394221 (0.0008) [2023-12-27 08:22:02,053][20347] Updated weights for policy 0, policy_version 394231 (0.0007) [2023-12-27 08:22:02,267][20381] Updated weights for policy 1, policy_version 393978 (0.0006) [2023-12-27 08:22:02,329][20381] Updated weights for policy 1, policy_version 393988 (0.0007) [2023-12-27 08:22:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 201818112. Throughput: 0: 10249.3, 1: 10348.4. Samples: 201831304. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:22:02,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 08:22:02,390][20381] Updated weights for policy 1, policy_version 393998 (0.0007) [2023-12-27 08:22:02,726][20347] Updated weights for policy 0, policy_version 394241 (0.0007) [2023-12-27 08:22:02,776][20347] Updated weights for policy 0, policy_version 394251 (0.0007) [2023-12-27 08:22:02,822][20347] Updated weights for policy 0, policy_version 394261 (0.0008) [2023-12-27 08:22:02,869][20347] Updated weights for policy 0, policy_version 394271 (0.0005) [2023-12-27 08:22:03,065][20381] Updated weights for policy 1, policy_version 394008 (0.0008) [2023-12-27 08:22:03,118][20381] Updated weights for policy 1, policy_version 394018 (0.0008) [2023-12-27 08:22:03,169][20381] Updated weights for policy 1, policy_version 394028 (0.0008) [2023-12-27 08:22:03,577][20347] Updated weights for policy 0, policy_version 394281 (0.0009) [2023-12-27 08:22:03,591][20291] KL-divergence is very high: 215.4162 [2023-12-27 08:22:03,626][20347] Updated weights for policy 0, policy_version 394291 (0.0009) [2023-12-27 08:22:03,676][20347] Updated weights for policy 0, policy_version 394301 (0.0010) [2023-12-27 08:22:03,735][20381] Updated weights for policy 1, policy_version 394038 (0.0007) [2023-12-27 08:22:03,778][20381] Updated weights for policy 1, policy_version 394048 (0.0007) [2023-12-27 08:22:03,832][20381] Updated weights for policy 1, policy_version 394058 (0.0008) [2023-12-27 08:22:04,399][20347] Updated weights for policy 0, policy_version 394311 (0.0008) [2023-12-27 08:22:04,451][20347] Updated weights for policy 0, policy_version 394321 (0.0007) [2023-12-27 08:22:04,501][20381] Updated weights for policy 1, policy_version 394068 (0.0008) [2023-12-27 08:22:04,502][20347] Updated weights for policy 0, policy_version 394331 (0.0006) [2023-12-27 08:22:04,555][20381] Updated weights for policy 1, policy_version 394078 (0.0008) [2023-12-27 08:22:04,607][20381] Updated weights for policy 1, policy_version 394088 (0.0007) [2023-12-27 08:22:05,225][20347] Updated weights for policy 0, policy_version 394341 (0.0008) [2023-12-27 08:22:05,266][20381] Updated weights for policy 1, policy_version 394098 (0.0007) [2023-12-27 08:22:05,279][20347] Updated weights for policy 0, policy_version 394351 (0.0008) [2023-12-27 08:22:05,322][20381] Updated weights for policy 1, policy_version 394108 (0.0005) [2023-12-27 08:22:05,328][20347] Updated weights for policy 0, policy_version 394361 (0.0006) [2023-12-27 08:22:05,369][20381] Updated weights for policy 1, policy_version 394118 (0.0005) [2023-12-27 08:22:05,424][20381] Updated weights for policy 1, policy_version 394128 (0.0005) [2023-12-27 08:22:05,942][20347] Updated weights for policy 0, policy_version 394371 (0.0007) [2023-12-27 08:22:06,002][20347] Updated weights for policy 0, policy_version 394381 (0.0006) [2023-12-27 08:22:06,042][20381] Updated weights for policy 1, policy_version 394138 (0.0006) [2023-12-27 08:22:06,055][20347] Updated weights for policy 0, policy_version 394391 (0.0006) [2023-12-27 08:22:06,092][20381] Updated weights for policy 1, policy_version 394148 (0.0006) [2023-12-27 08:22:06,157][20381] Updated weights for policy 1, policy_version 394158 (0.0008) [2023-12-27 08:22:06,747][20347] Updated weights for policy 0, policy_version 394401 (0.0008) [2023-12-27 08:22:06,808][20347] Updated weights for policy 0, policy_version 394411 (0.0007) [2023-12-27 08:22:06,860][20381] Updated weights for policy 1, policy_version 394168 (0.0006) [2023-12-27 08:22:06,862][20347] Updated weights for policy 0, policy_version 394421 (0.0006) [2023-12-27 08:22:06,914][20347] Updated weights for policy 0, policy_version 394431 (0.0005) [2023-12-27 08:22:06,915][20381] Updated weights for policy 1, policy_version 394178 (0.0006) [2023-12-27 08:22:06,972][20381] Updated weights for policy 1, policy_version 394188 (0.0007) [2023-12-27 08:22:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.4, 300 sec: 20743.8). Total num frames: 201924608. Throughput: 0: 10230.6, 1: 10389.2. Samples: 201893604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:22:07,364][19170] Avg episode reward: [(0, '990.490'), (1, '1000.000')] [2023-12-27 08:22:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000394432_100990976.pth... [2023-12-27 08:22:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000394192_100933632.pth... [2023-12-27 08:22:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000393248_100687872.pth [2023-12-27 08:22:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000392976_100622336.pth [2023-12-27 08:22:07,608][20347] Updated weights for policy 0, policy_version 394441 (0.0007) [2023-12-27 08:22:07,659][20347] Updated weights for policy 0, policy_version 394451 (0.0008) [2023-12-27 08:22:07,667][20381] Updated weights for policy 1, policy_version 394198 (0.0006) [2023-12-27 08:22:07,714][20347] Updated weights for policy 0, policy_version 394461 (0.0008) [2023-12-27 08:22:07,717][20381] Updated weights for policy 1, policy_version 394208 (0.0005) [2023-12-27 08:22:07,766][20381] Updated weights for policy 1, policy_version 394218 (0.0005) [2023-12-27 08:22:08,394][20381] Updated weights for policy 1, policy_version 394228 (0.0006) [2023-12-27 08:22:08,430][20347] Updated weights for policy 0, policy_version 394471 (0.0007) [2023-12-27 08:22:08,453][20381] Updated weights for policy 1, policy_version 394238 (0.0007) [2023-12-27 08:22:08,457][20291] KL-divergence is very high: 280.0034 [2023-12-27 08:22:08,463][20291] KL-divergence is very high: 253.5575 [2023-12-27 08:22:08,468][20291] KL-divergence is very high: 147.5541 [2023-12-27 08:22:08,491][20347] Updated weights for policy 0, policy_version 394481 (0.0006) [2023-12-27 08:22:08,515][20381] Updated weights for policy 1, policy_version 394248 (0.0006) [2023-12-27 08:22:08,550][20347] Updated weights for policy 0, policy_version 394491 (0.0006) [2023-12-27 08:22:09,153][20381] Updated weights for policy 1, policy_version 394258 (0.0006) [2023-12-27 08:22:09,209][20381] Updated weights for policy 1, policy_version 394268 (0.0008) [2023-12-27 08:22:09,243][20291] KL-divergence is very high: 125.0295 [2023-12-27 08:22:09,265][20347] Updated weights for policy 0, policy_version 394501 (0.0006) [2023-12-27 08:22:09,271][20381] Updated weights for policy 1, policy_version 394278 (0.0007) [2023-12-27 08:22:09,284][20291] KL-divergence is very high: 247.1638 [2023-12-27 08:22:09,318][20347] Updated weights for policy 0, policy_version 394511 (0.0006) [2023-12-27 08:22:09,328][20381] Updated weights for policy 1, policy_version 394288 (0.0006) [2023-12-27 08:22:09,329][20291] KL-divergence is very high: 189.7809 [2023-12-27 08:22:09,336][20291] KL-divergence is very high: 169.8536 [2023-12-27 08:22:09,343][20291] KL-divergence is very high: 168.1189 [2023-12-27 08:22:09,356][20291] KL-divergence is very high: 179.6160 [2023-12-27 08:22:09,363][20291] KL-divergence is very high: 410.1864 [2023-12-27 08:22:09,377][20291] KL-divergence is very high: 213.7736 [2023-12-27 08:22:09,382][20347] Updated weights for policy 0, policy_version 394521 (0.0008) [2023-12-27 08:22:09,384][20291] KL-divergence is very high: 326.4409 [2023-12-27 08:22:10,015][20347] Updated weights for policy 0, policy_version 394531 (0.0006) [2023-12-27 08:22:10,054][20291] KL-divergence is very high: 177.4369 [2023-12-27 08:22:10,066][20291] KL-divergence is very high: 144.2642 [2023-12-27 08:22:10,080][20347] Updated weights for policy 0, policy_version 394541 (0.0006) [2023-12-27 08:22:10,107][20381] Updated weights for policy 1, policy_version 394298 (0.0008) [2023-12-27 08:22:10,140][20347] Updated weights for policy 0, policy_version 394551 (0.0006) [2023-12-27 08:22:10,165][20381] Updated weights for policy 1, policy_version 394308 (0.0006) [2023-12-27 08:22:10,226][20381] Updated weights for policy 1, policy_version 394318 (0.0007) [2023-12-27 08:22:10,762][20347] Updated weights for policy 0, policy_version 394561 (0.0006) [2023-12-27 08:22:10,811][20347] Updated weights for policy 0, policy_version 394571 (0.0005) [2023-12-27 08:22:10,857][20347] Updated weights for policy 0, policy_version 394581 (0.0005) [2023-12-27 08:22:10,905][20347] Updated weights for policy 0, policy_version 394591 (0.0005) [2023-12-27 08:22:10,979][20381] Updated weights for policy 1, policy_version 394328 (0.0008) [2023-12-27 08:22:11,034][20381] Updated weights for policy 1, policy_version 394338 (0.0008) [2023-12-27 08:22:11,101][20381] Updated weights for policy 1, policy_version 394348 (0.0009) [2023-12-27 08:22:11,527][20347] Updated weights for policy 0, policy_version 394601 (0.0006) [2023-12-27 08:22:11,590][20347] Updated weights for policy 0, policy_version 394611 (0.0007) [2023-12-27 08:22:11,654][20347] Updated weights for policy 0, policy_version 394621 (0.0007) [2023-12-27 08:22:11,879][20381] Updated weights for policy 1, policy_version 394358 (0.0008) [2023-12-27 08:22:11,935][20381] Updated weights for policy 1, policy_version 394368 (0.0008) [2023-12-27 08:22:12,001][20381] Updated weights for policy 1, policy_version 394378 (0.0006) [2023-12-27 08:22:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 202022912. Throughput: 0: 10273.4, 1: 10357.3. Samples: 202016036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 08:22:12,363][19170] Avg episode reward: [(0, '942.630'), (1, '1000.000')] [2023-12-27 08:22:12,367][20347] Updated weights for policy 0, policy_version 394631 (0.0009) [2023-12-27 08:22:12,429][20347] Updated weights for policy 0, policy_version 394641 (0.0007) [2023-12-27 08:22:12,498][20347] Updated weights for policy 0, policy_version 394651 (0.0008) [2023-12-27 08:22:12,628][20381] Updated weights for policy 1, policy_version 394388 (0.0006) [2023-12-27 08:22:12,689][20381] Updated weights for policy 1, policy_version 394398 (0.0006) [2023-12-27 08:22:12,746][20381] Updated weights for policy 1, policy_version 394408 (0.0006) [2023-12-27 08:22:13,166][20347] Updated weights for policy 0, policy_version 394661 (0.0009) [2023-12-27 08:22:13,224][20347] Updated weights for policy 0, policy_version 394671 (0.0007) [2023-12-27 08:22:13,282][20347] Updated weights for policy 0, policy_version 394681 (0.0007) [2023-12-27 08:22:13,331][20381] Updated weights for policy 1, policy_version 394418 (0.0006) [2023-12-27 08:22:13,384][20381] Updated weights for policy 1, policy_version 394428 (0.0008) [2023-12-27 08:22:13,430][20381] Updated weights for policy 1, policy_version 394438 (0.0008) [2023-12-27 08:22:13,479][20381] Updated weights for policy 1, policy_version 394448 (0.0008) [2023-12-27 08:22:13,935][20347] Updated weights for policy 0, policy_version 394691 (0.0007) [2023-12-27 08:22:13,989][20347] Updated weights for policy 0, policy_version 394701 (0.0005) [2023-12-27 08:22:14,034][20347] Updated weights for policy 0, policy_version 394711 (0.0005) [2023-12-27 08:22:14,166][20381] Updated weights for policy 1, policy_version 394458 (0.0006) [2023-12-27 08:22:14,223][20381] Updated weights for policy 1, policy_version 394468 (0.0005) [2023-12-27 08:22:14,281][20381] Updated weights for policy 1, policy_version 394478 (0.0005) [2023-12-27 08:22:14,672][20347] Updated weights for policy 0, policy_version 394721 (0.0006) [2023-12-27 08:22:14,727][20291] KL-divergence is very high: 497.0539 [2023-12-27 08:22:14,739][20347] Updated weights for policy 0, policy_version 394731 (0.0005) [2023-12-27 08:22:14,801][20347] Updated weights for policy 0, policy_version 394741 (0.0006) [2023-12-27 08:22:14,849][20347] Updated weights for policy 0, policy_version 394751 (0.0006) [2023-12-27 08:22:14,873][20381] Updated weights for policy 1, policy_version 394488 (0.0008) [2023-12-27 08:22:14,934][20381] Updated weights for policy 1, policy_version 394498 (0.0007) [2023-12-27 08:22:15,001][20381] Updated weights for policy 1, policy_version 394508 (0.0006) [2023-12-27 08:22:15,495][20291] KL-divergence is very high: 489.1382 [2023-12-27 08:22:15,531][20347] Updated weights for policy 0, policy_version 394761 (0.0009) [2023-12-27 08:22:15,534][20291] KL-divergence is very high: 995.9045 [2023-12-27 08:22:15,543][20291] KL-divergence is very high: 103.9081 [2023-12-27 08:22:15,561][20291] KL-divergence is very high: 115.4581 [2023-12-27 08:22:15,570][20291] KL-divergence is very high: 301.3123 [2023-12-27 08:22:15,576][20347] Updated weights for policy 0, policy_version 394771 (0.0008) [2023-12-27 08:22:15,580][20291] KL-divergence is very high: 422.6081 [2023-12-27 08:22:15,615][20291] KL-divergence is very high: 219.7304 [2023-12-27 08:22:15,626][20291] KL-divergence is very high: 124.3431 [2023-12-27 08:22:15,630][20347] Updated weights for policy 0, policy_version 394781 (0.0007) [2023-12-27 08:22:15,645][20381] Updated weights for policy 1, policy_version 394518 (0.0011) [2023-12-27 08:22:15,703][20381] Updated weights for policy 1, policy_version 394528 (0.0010) [2023-12-27 08:22:15,762][20381] Updated weights for policy 1, policy_version 394538 (0.0011) [2023-12-27 08:22:16,301][20347] Updated weights for policy 0, policy_version 394791 (0.0007) [2023-12-27 08:22:16,344][20381] Updated weights for policy 1, policy_version 394548 (0.0010) [2023-12-27 08:22:16,357][20347] Updated weights for policy 0, policy_version 394801 (0.0006) [2023-12-27 08:22:16,374][20291] KL-divergence is very high: 471.6846 [2023-12-27 08:22:16,395][20291] KL-divergence is very high: 4191.5103 [2023-12-27 08:22:16,400][20291] KL-divergence is very high: 189.7398 [2023-12-27 08:22:16,406][20381] Updated weights for policy 1, policy_version 394558 (0.0009) [2023-12-27 08:22:16,410][20347] Updated weights for policy 0, policy_version 394811 (0.0006) [2023-12-27 08:22:16,418][20291] KL-divergence is very high: 511.0488 [2023-12-27 08:22:16,459][20381] Updated weights for policy 1, policy_version 394568 (0.0005) [2023-12-27 08:22:17,085][20291] KL-divergence is very high: 930.6581 [2023-12-27 08:22:17,090][20291] KL-divergence is very high: 437.6829 [2023-12-27 08:22:17,092][20381] Updated weights for policy 1, policy_version 394578 (0.0006) [2023-12-27 08:22:17,114][20347] Updated weights for policy 0, policy_version 394821 (0.0008) [2023-12-27 08:22:17,128][20291] KL-divergence is very high: 999.6523 [2023-12-27 08:22:17,133][20291] KL-divergence is very high: 1116.9630 [2023-12-27 08:22:17,137][20381] Updated weights for policy 1, policy_version 394588 (0.0010) [2023-12-27 08:22:17,164][20347] Updated weights for policy 0, policy_version 394831 (0.0007) [2023-12-27 08:22:17,198][20381] Updated weights for policy 1, policy_version 394598 (0.0011) [2023-12-27 08:22:17,212][20291] KL-divergence is very high: 147.2586 [2023-12-27 08:22:17,218][20347] Updated weights for policy 0, policy_version 394841 (0.0006) [2023-12-27 08:22:17,254][20381] Updated weights for policy 1, policy_version 394608 (0.0010) [2023-12-27 08:22:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 202137600. Throughput: 0: 10260.4, 1: 10422.3. Samples: 202142784. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:17,363][19170] Avg episode reward: [(0, '888.550'), (1, '1000.000')] [2023-12-27 08:22:17,826][20291] KL-divergence is very high: 1230.2438 [2023-12-27 08:22:17,837][20347] Updated weights for policy 0, policy_version 394851 (0.0006) [2023-12-27 08:22:17,867][20291] KL-divergence is very high: 1121.1307 [2023-12-27 08:22:17,876][20381] Updated weights for policy 1, policy_version 394618 (0.0005) [2023-12-27 08:22:17,887][20347] Updated weights for policy 0, policy_version 394861 (0.0007) [2023-12-27 08:22:17,905][20291] KL-divergence is very high: 1207.4758 [2023-12-27 08:22:17,930][20381] Updated weights for policy 1, policy_version 394628 (0.0005) [2023-12-27 08:22:17,932][20347] Updated weights for policy 0, policy_version 394871 (0.0007) [2023-12-27 08:22:17,942][20291] KL-divergence is very high: 1201.2953 [2023-12-27 08:22:17,989][20381] Updated weights for policy 1, policy_version 394638 (0.0005) [2023-12-27 08:22:18,586][20347] Updated weights for policy 0, policy_version 394881 (0.0007) [2023-12-27 08:22:18,592][20381] Updated weights for policy 1, policy_version 394648 (0.0007) [2023-12-27 08:22:18,640][20347] Updated weights for policy 0, policy_version 394891 (0.0006) [2023-12-27 08:22:18,645][20381] Updated weights for policy 1, policy_version 394658 (0.0006) [2023-12-27 08:22:18,690][20347] Updated weights for policy 0, policy_version 394901 (0.0006) [2023-12-27 08:22:18,702][20381] Updated weights for policy 1, policy_version 394668 (0.0008) [2023-12-27 08:22:18,744][20347] Updated weights for policy 0, policy_version 394911 (0.0007) [2023-12-27 08:22:19,378][20381] Updated weights for policy 1, policy_version 394678 (0.0007) [2023-12-27 08:22:19,433][20381] Updated weights for policy 1, policy_version 394688 (0.0005) [2023-12-27 08:22:19,500][20381] Updated weights for policy 1, policy_version 394698 (0.0006) [2023-12-27 08:22:19,506][20347] Updated weights for policy 0, policy_version 394921 (0.0007) [2023-12-27 08:22:19,563][20347] Updated weights for policy 0, policy_version 394931 (0.0007) [2023-12-27 08:22:19,617][20347] Updated weights for policy 0, policy_version 394941 (0.0007) [2023-12-27 08:22:20,172][20381] Updated weights for policy 1, policy_version 394708 (0.0007) [2023-12-27 08:22:20,235][20381] Updated weights for policy 1, policy_version 394718 (0.0008) [2023-12-27 08:22:20,295][20381] Updated weights for policy 1, policy_version 394728 (0.0009) [2023-12-27 08:22:20,355][20347] Updated weights for policy 0, policy_version 394951 (0.0009) [2023-12-27 08:22:20,412][20347] Updated weights for policy 0, policy_version 394961 (0.0009) [2023-12-27 08:22:20,469][20347] Updated weights for policy 0, policy_version 394971 (0.0009) [2023-12-27 08:22:20,914][20381] Updated weights for policy 1, policy_version 394738 (0.0008) [2023-12-27 08:22:20,973][20381] Updated weights for policy 1, policy_version 394748 (0.0008) [2023-12-27 08:22:21,033][20381] Updated weights for policy 1, policy_version 394758 (0.0008) [2023-12-27 08:22:21,087][20381] Updated weights for policy 1, policy_version 394768 (0.0008) [2023-12-27 08:22:21,221][20347] Updated weights for policy 0, policy_version 394981 (0.0008) [2023-12-27 08:22:21,278][20347] Updated weights for policy 0, policy_version 394991 (0.0007) [2023-12-27 08:22:21,335][20347] Updated weights for policy 0, policy_version 395001 (0.0007) [2023-12-27 08:22:21,878][20381] Updated weights for policy 1, policy_version 394778 (0.0007) [2023-12-27 08:22:21,934][20381] Updated weights for policy 1, policy_version 394788 (0.0007) [2023-12-27 08:22:21,989][20381] Updated weights for policy 1, policy_version 394798 (0.0007) [2023-12-27 08:22:22,072][20347] Updated weights for policy 0, policy_version 395011 (0.0009) [2023-12-27 08:22:22,131][20347] Updated weights for policy 0, policy_version 395021 (0.0008) [2023-12-27 08:22:22,195][20347] Updated weights for policy 0, policy_version 395031 (0.0008) [2023-12-27 08:22:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 202235904. Throughput: 0: 10255.3, 1: 10409.2. Samples: 202205280. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:22,363][19170] Avg episode reward: [(0, '854.990'), (1, '1000.000')] [2023-12-27 08:22:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000394800_101089280.pth... [2023-12-27 08:22:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000395040_101146624.pth... [2023-12-27 08:22:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000393584_100777984.pth [2023-12-27 08:22:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000393824_100835328.pth [2023-12-27 08:22:22,695][20381] Updated weights for policy 1, policy_version 394808 (0.0008) [2023-12-27 08:22:22,757][20381] Updated weights for policy 1, policy_version 394818 (0.0007) [2023-12-27 08:22:22,819][20381] Updated weights for policy 1, policy_version 394828 (0.0008) [2023-12-27 08:22:22,916][20347] Updated weights for policy 0, policy_version 395041 (0.0007) [2023-12-27 08:22:22,974][20347] Updated weights for policy 0, policy_version 395051 (0.0006) [2023-12-27 08:22:23,034][20347] Updated weights for policy 0, policy_version 395061 (0.0006) [2023-12-27 08:22:23,093][20347] Updated weights for policy 0, policy_version 395071 (0.0008) [2023-12-27 08:22:23,488][20381] Updated weights for policy 1, policy_version 394838 (0.0007) [2023-12-27 08:22:23,547][20381] Updated weights for policy 1, policy_version 394848 (0.0007) [2023-12-27 08:22:23,597][20381] Updated weights for policy 1, policy_version 394858 (0.0006) [2023-12-27 08:22:23,733][20347] Updated weights for policy 0, policy_version 395081 (0.0010) [2023-12-27 08:22:23,788][20347] Updated weights for policy 0, policy_version 395091 (0.0010) [2023-12-27 08:22:23,843][20347] Updated weights for policy 0, policy_version 395101 (0.0010) [2023-12-27 08:22:24,255][20381] Updated weights for policy 1, policy_version 394868 (0.0008) [2023-12-27 08:22:24,310][20381] Updated weights for policy 1, policy_version 394878 (0.0011) [2023-12-27 08:22:24,365][20381] Updated weights for policy 1, policy_version 394888 (0.0011) [2023-12-27 08:22:24,511][20347] Updated weights for policy 0, policy_version 395111 (0.0008) [2023-12-27 08:22:24,569][20347] Updated weights for policy 0, policy_version 395121 (0.0011) [2023-12-27 08:22:24,627][20347] Updated weights for policy 0, policy_version 395131 (0.0010) [2023-12-27 08:22:25,110][20381] Updated weights for policy 1, policy_version 394898 (0.0010) [2023-12-27 08:22:25,157][20381] Updated weights for policy 1, policy_version 394908 (0.0007) [2023-12-27 08:22:25,205][20381] Updated weights for policy 1, policy_version 394918 (0.0007) [2023-12-27 08:22:25,254][20347] Updated weights for policy 0, policy_version 395141 (0.0010) [2023-12-27 08:22:25,256][20381] Updated weights for policy 1, policy_version 394928 (0.0006) [2023-12-27 08:22:25,299][20347] Updated weights for policy 0, policy_version 395151 (0.0010) [2023-12-27 08:22:25,350][20347] Updated weights for policy 0, policy_version 395161 (0.0010) [2023-12-27 08:22:25,888][20381] Updated weights for policy 1, policy_version 394938 (0.0005) [2023-12-27 08:22:25,942][20381] Updated weights for policy 1, policy_version 394948 (0.0007) [2023-12-27 08:22:25,997][20381] Updated weights for policy 1, policy_version 394958 (0.0008) [2023-12-27 08:22:26,038][20347] Updated weights for policy 0, policy_version 395171 (0.0010) [2023-12-27 08:22:26,092][20347] Updated weights for policy 0, policy_version 395181 (0.0010) [2023-12-27 08:22:26,151][20347] Updated weights for policy 0, policy_version 395191 (0.0010) [2023-12-27 08:22:26,664][20381] Updated weights for policy 1, policy_version 394968 (0.0008) [2023-12-27 08:22:26,721][20381] Updated weights for policy 1, policy_version 394978 (0.0007) [2023-12-27 08:22:26,734][20347] Updated weights for policy 0, policy_version 395201 (0.0010) [2023-12-27 08:22:26,772][20381] Updated weights for policy 1, policy_version 394988 (0.0005) [2023-12-27 08:22:26,774][20291] KL-divergence is very high: 159.1799 [2023-12-27 08:22:26,791][20347] Updated weights for policy 0, policy_version 395211 (0.0010) [2023-12-27 08:22:26,842][20347] Updated weights for policy 0, policy_version 395221 (0.0010) [2023-12-27 08:22:26,889][20347] Updated weights for policy 0, policy_version 395231 (0.0005) [2023-12-27 08:22:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 202334208. Throughput: 0: 10210.2, 1: 10392.3. Samples: 202326456. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:27,363][19170] Avg episode reward: [(0, '875.340'), (1, '1000.000')] [2023-12-27 08:22:27,431][20381] Updated weights for policy 1, policy_version 394998 (0.0005) [2023-12-27 08:22:27,482][20381] Updated weights for policy 1, policy_version 395008 (0.0006) [2023-12-27 08:22:27,515][20347] Updated weights for policy 0, policy_version 395241 (0.0010) [2023-12-27 08:22:27,532][20381] Updated weights for policy 1, policy_version 395018 (0.0008) [2023-12-27 08:22:27,557][20347] Updated weights for policy 0, policy_version 395251 (0.0010) [2023-12-27 08:22:27,609][20347] Updated weights for policy 0, policy_version 395261 (0.0010) [2023-12-27 08:22:28,213][20381] Updated weights for policy 1, policy_version 395028 (0.0008) [2023-12-27 08:22:28,265][20381] Updated weights for policy 1, policy_version 395038 (0.0007) [2023-12-27 08:22:28,269][20347] Updated weights for policy 0, policy_version 395271 (0.0006) [2023-12-27 08:22:28,314][20381] Updated weights for policy 1, policy_version 395048 (0.0008) [2023-12-27 08:22:28,320][20347] Updated weights for policy 0, policy_version 395281 (0.0005) [2023-12-27 08:22:28,381][20347] Updated weights for policy 0, policy_version 395291 (0.0008) [2023-12-27 08:22:28,952][20381] Updated weights for policy 1, policy_version 395058 (0.0007) [2023-12-27 08:22:29,003][20381] Updated weights for policy 1, policy_version 395068 (0.0005) [2023-12-27 08:22:29,037][20347] Updated weights for policy 0, policy_version 395301 (0.0007) [2023-12-27 08:22:29,061][20381] Updated weights for policy 1, policy_version 395078 (0.0007) [2023-12-27 08:22:29,108][20347] Updated weights for policy 0, policy_version 395311 (0.0010) [2023-12-27 08:22:29,109][20381] Updated weights for policy 1, policy_version 395088 (0.0007) [2023-12-27 08:22:29,174][20347] Updated weights for policy 0, policy_version 395321 (0.0010) [2023-12-27 08:22:29,735][20381] Updated weights for policy 1, policy_version 395098 (0.0011) [2023-12-27 08:22:29,788][20381] Updated weights for policy 1, policy_version 395108 (0.0011) [2023-12-27 08:22:29,830][20347] Updated weights for policy 0, policy_version 395331 (0.0010) [2023-12-27 08:22:29,856][20381] Updated weights for policy 1, policy_version 395118 (0.0009) [2023-12-27 08:22:29,886][20347] Updated weights for policy 0, policy_version 395341 (0.0008) [2023-12-27 08:22:29,943][20347] Updated weights for policy 0, policy_version 395351 (0.0008) [2023-12-27 08:22:30,538][20347] Updated weights for policy 0, policy_version 395361 (0.0009) [2023-12-27 08:22:30,585][20381] Updated weights for policy 1, policy_version 395128 (0.0006) [2023-12-27 08:22:30,598][20347] Updated weights for policy 0, policy_version 395371 (0.0008) [2023-12-27 08:22:30,635][20381] Updated weights for policy 1, policy_version 395138 (0.0005) [2023-12-27 08:22:30,658][20347] Updated weights for policy 0, policy_version 395381 (0.0005) [2023-12-27 08:22:30,685][20381] Updated weights for policy 1, policy_version 395148 (0.0005) [2023-12-27 08:22:30,710][20347] Updated weights for policy 0, policy_version 395391 (0.0005) [2023-12-27 08:22:31,285][20347] Updated weights for policy 0, policy_version 395401 (0.0007) [2023-12-27 08:22:31,348][20347] Updated weights for policy 0, policy_version 395411 (0.0007) [2023-12-27 08:22:31,374][20381] Updated weights for policy 1, policy_version 395158 (0.0006) [2023-12-27 08:22:31,409][20347] Updated weights for policy 0, policy_version 395421 (0.0008) [2023-12-27 08:22:31,428][20381] Updated weights for policy 1, policy_version 395168 (0.0010) [2023-12-27 08:22:31,482][20381] Updated weights for policy 1, policy_version 395178 (0.0005) [2023-12-27 08:22:32,121][20347] Updated weights for policy 0, policy_version 395431 (0.0007) [2023-12-27 08:22:32,153][20381] Updated weights for policy 1, policy_version 395188 (0.0005) [2023-12-27 08:22:32,174][20347] Updated weights for policy 0, policy_version 395441 (0.0006) [2023-12-27 08:22:32,210][20381] Updated weights for policy 1, policy_version 395198 (0.0010) [2023-12-27 08:22:32,225][20347] Updated weights for policy 0, policy_version 395451 (0.0006) [2023-12-27 08:22:32,267][20381] Updated weights for policy 1, policy_version 395208 (0.0010) [2023-12-27 08:22:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 202448896. Throughput: 0: 10349.7, 1: 10311.6. Samples: 202453620. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:32,363][19170] Avg episode reward: [(0, '830.050'), (1, '1000.000')] [2023-12-27 08:22:32,894][20381] Updated weights for policy 1, policy_version 395218 (0.0010) [2023-12-27 08:22:32,936][20347] Updated weights for policy 0, policy_version 395461 (0.0008) [2023-12-27 08:22:32,954][20381] Updated weights for policy 1, policy_version 395228 (0.0010) [2023-12-27 08:22:33,002][20347] Updated weights for policy 0, policy_version 395471 (0.0009) [2023-12-27 08:22:33,012][20381] Updated weights for policy 1, policy_version 395238 (0.0010) [2023-12-27 08:22:33,055][20347] Updated weights for policy 0, policy_version 395481 (0.0009) [2023-12-27 08:22:33,060][20381] Updated weights for policy 1, policy_version 395248 (0.0010) [2023-12-27 08:22:33,644][20347] Updated weights for policy 0, policy_version 395491 (0.0009) [2023-12-27 08:22:33,688][20347] Updated weights for policy 0, policy_version 395501 (0.0010) [2023-12-27 08:22:33,688][20381] Updated weights for policy 1, policy_version 395258 (0.0009) [2023-12-27 08:22:33,732][20347] Updated weights for policy 0, policy_version 395511 (0.0010) [2023-12-27 08:22:33,738][20381] Updated weights for policy 1, policy_version 395268 (0.0007) [2023-12-27 08:22:33,787][20381] Updated weights for policy 1, policy_version 395278 (0.0006) [2023-12-27 08:22:34,404][20381] Updated weights for policy 1, policy_version 395288 (0.0008) [2023-12-27 08:22:34,430][20347] Updated weights for policy 0, policy_version 395521 (0.0010) [2023-12-27 08:22:34,454][20381] Updated weights for policy 1, policy_version 395298 (0.0006) [2023-12-27 08:22:34,493][20347] Updated weights for policy 0, policy_version 395531 (0.0011) [2023-12-27 08:22:34,501][20381] Updated weights for policy 1, policy_version 395308 (0.0009) [2023-12-27 08:22:34,548][20347] Updated weights for policy 0, policy_version 395541 (0.0010) [2023-12-27 08:22:34,600][20347] Updated weights for policy 0, policy_version 395551 (0.0010) [2023-12-27 08:22:35,228][20347] Updated weights for policy 0, policy_version 395561 (0.0007) [2023-12-27 08:22:35,237][20381] Updated weights for policy 1, policy_version 395318 (0.0005) [2023-12-27 08:22:35,280][20347] Updated weights for policy 0, policy_version 395571 (0.0006) [2023-12-27 08:22:35,285][20381] Updated weights for policy 1, policy_version 395328 (0.0006) [2023-12-27 08:22:35,334][20347] Updated weights for policy 0, policy_version 395581 (0.0006) [2023-12-27 08:22:35,337][20381] Updated weights for policy 1, policy_version 395338 (0.0006) [2023-12-27 08:22:35,901][20347] Updated weights for policy 0, policy_version 395591 (0.0007) [2023-12-27 08:22:35,949][20347] Updated weights for policy 0, policy_version 395601 (0.0009) [2023-12-27 08:22:35,997][20347] Updated weights for policy 0, policy_version 395611 (0.0008) [2023-12-27 08:22:36,043][20381] Updated weights for policy 1, policy_version 395348 (0.0007) [2023-12-27 08:22:36,094][20381] Updated weights for policy 1, policy_version 395358 (0.0008) [2023-12-27 08:22:36,153][20381] Updated weights for policy 1, policy_version 395368 (0.0007) [2023-12-27 08:22:36,646][20347] Updated weights for policy 0, policy_version 395621 (0.0006) [2023-12-27 08:22:36,700][20347] Updated weights for policy 0, policy_version 395631 (0.0005) [2023-12-27 08:22:36,752][20347] Updated weights for policy 0, policy_version 395641 (0.0006) [2023-12-27 08:22:36,869][20381] Updated weights for policy 1, policy_version 395378 (0.0007) [2023-12-27 08:22:36,933][20381] Updated weights for policy 1, policy_version 395388 (0.0007) [2023-12-27 08:22:36,995][20381] Updated weights for policy 1, policy_version 395398 (0.0007) [2023-12-27 08:22:37,058][20381] Updated weights for policy 1, policy_version 395408 (0.0006) [2023-12-27 08:22:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 202547200. Throughput: 0: 10358.2, 1: 10320.0. Samples: 202516892. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:37,363][19170] Avg episode reward: [(0, '868.360'), (1, '1000.000')] [2023-12-27 08:22:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000395408_101244928.pth... [2023-12-27 08:22:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000395648_101302272.pth... [2023-12-27 08:22:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000394192_100933632.pth [2023-12-27 08:22:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000394432_100990976.pth [2023-12-27 08:22:37,452][20347] Updated weights for policy 0, policy_version 395651 (0.0008) [2023-12-27 08:22:37,504][20347] Updated weights for policy 0, policy_version 395661 (0.0008) [2023-12-27 08:22:37,556][20347] Updated weights for policy 0, policy_version 395671 (0.0007) [2023-12-27 08:22:37,611][20381] Updated weights for policy 1, policy_version 395418 (0.0008) [2023-12-27 08:22:37,675][20381] Updated weights for policy 1, policy_version 395428 (0.0008) [2023-12-27 08:22:37,728][20381] Updated weights for policy 1, policy_version 395438 (0.0008) [2023-12-27 08:22:38,270][20347] Updated weights for policy 0, policy_version 395681 (0.0008) [2023-12-27 08:22:38,328][20347] Updated weights for policy 0, policy_version 395691 (0.0008) [2023-12-27 08:22:38,386][20347] Updated weights for policy 0, policy_version 395701 (0.0007) [2023-12-27 08:22:38,433][20381] Updated weights for policy 1, policy_version 395448 (0.0008) [2023-12-27 08:22:38,435][20347] Updated weights for policy 0, policy_version 395711 (0.0007) [2023-12-27 08:22:38,487][20381] Updated weights for policy 1, policy_version 395458 (0.0008) [2023-12-27 08:22:38,542][20381] Updated weights for policy 1, policy_version 395468 (0.0008) [2023-12-27 08:22:39,135][20347] Updated weights for policy 0, policy_version 395721 (0.0008) [2023-12-27 08:22:39,176][20381] Updated weights for policy 1, policy_version 395478 (0.0007) [2023-12-27 08:22:39,189][20347] Updated weights for policy 0, policy_version 395731 (0.0010) [2023-12-27 08:22:39,238][20291] KL-divergence is very high: 379.5680 [2023-12-27 08:22:39,238][20381] Updated weights for policy 1, policy_version 395488 (0.0007) [2023-12-27 08:22:39,243][20347] Updated weights for policy 0, policy_version 395741 (0.0010) [2023-12-27 08:22:39,298][20381] Updated weights for policy 1, policy_version 395498 (0.0006) [2023-12-27 08:22:39,948][20347] Updated weights for policy 0, policy_version 395751 (0.0010) [2023-12-27 08:22:40,003][20347] Updated weights for policy 0, policy_version 395761 (0.0010) [2023-12-27 08:22:40,056][20347] Updated weights for policy 0, policy_version 395771 (0.0011) [2023-12-27 08:22:40,095][20381] Updated weights for policy 1, policy_version 395508 (0.0008) [2023-12-27 08:22:40,154][20381] Updated weights for policy 1, policy_version 395518 (0.0009) [2023-12-27 08:22:40,212][20381] Updated weights for policy 1, policy_version 395528 (0.0009) [2023-12-27 08:22:40,738][20347] Updated weights for policy 0, policy_version 395781 (0.0009) [2023-12-27 08:22:40,790][20347] Updated weights for policy 0, policy_version 395791 (0.0010) [2023-12-27 08:22:40,841][20347] Updated weights for policy 0, policy_version 395801 (0.0010) [2023-12-27 08:22:40,905][20381] Updated weights for policy 1, policy_version 395538 (0.0009) [2023-12-27 08:22:40,965][20381] Updated weights for policy 1, policy_version 395548 (0.0008) [2023-12-27 08:22:41,026][20381] Updated weights for policy 1, policy_version 395558 (0.0007) [2023-12-27 08:22:41,086][20381] Updated weights for policy 1, policy_version 395568 (0.0007) [2023-12-27 08:22:41,510][20347] Updated weights for policy 0, policy_version 395811 (0.0010) [2023-12-27 08:22:41,563][20347] Updated weights for policy 0, policy_version 395821 (0.0010) [2023-12-27 08:22:41,626][20347] Updated weights for policy 0, policy_version 395831 (0.0011) [2023-12-27 08:22:41,828][20381] Updated weights for policy 1, policy_version 395578 (0.0011) [2023-12-27 08:22:41,880][20381] Updated weights for policy 1, policy_version 395588 (0.0010) [2023-12-27 08:22:41,939][20381] Updated weights for policy 1, policy_version 395598 (0.0011) [2023-12-27 08:22:42,290][20347] Updated weights for policy 0, policy_version 395841 (0.0009) [2023-12-27 08:22:42,356][20347] Updated weights for policy 0, policy_version 395851 (0.0009) [2023-12-27 08:22:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 202645504. Throughput: 0: 10334.7, 1: 10367.9. Samples: 202639616. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:42,363][19170] Avg episode reward: [(0, '906.560'), (1, '1000.000')] [2023-12-27 08:22:42,412][20347] Updated weights for policy 0, policy_version 395861 (0.0010) [2023-12-27 08:22:42,466][20347] Updated weights for policy 0, policy_version 395871 (0.0010) [2023-12-27 08:22:42,610][20381] Updated weights for policy 1, policy_version 395608 (0.0009) [2023-12-27 08:22:42,656][20381] Updated weights for policy 1, policy_version 395618 (0.0008) [2023-12-27 08:22:42,705][20381] Updated weights for policy 1, policy_version 395628 (0.0009) [2023-12-27 08:22:42,988][20291] KL-divergence is very high: 120.1087 [2023-12-27 08:22:43,028][20347] Updated weights for policy 0, policy_version 395881 (0.0007) [2023-12-27 08:22:43,029][20291] KL-divergence is very high: 206.6564 [2023-12-27 08:22:43,087][20347] Updated weights for policy 0, policy_version 395891 (0.0008) [2023-12-27 08:22:43,135][20347] Updated weights for policy 0, policy_version 395901 (0.0007) [2023-12-27 08:22:43,431][20381] Updated weights for policy 1, policy_version 395638 (0.0008) [2023-12-27 08:22:43,480][20381] Updated weights for policy 1, policy_version 395648 (0.0009) [2023-12-27 08:22:43,531][20381] Updated weights for policy 1, policy_version 395658 (0.0008) [2023-12-27 08:22:43,791][20291] KL-divergence is very high: 185.1230 [2023-12-27 08:22:43,796][20347] Updated weights for policy 0, policy_version 395911 (0.0007) [2023-12-27 08:22:43,853][20347] Updated weights for policy 0, policy_version 395921 (0.0006) [2023-12-27 08:22:43,881][20291] KL-divergence is very high: 117.5548 [2023-12-27 08:22:43,909][20347] Updated weights for policy 0, policy_version 395931 (0.0005) [2023-12-27 08:22:43,926][20291] KL-divergence is very high: 109.0024 [2023-12-27 08:22:44,281][20381] Updated weights for policy 1, policy_version 395668 (0.0007) [2023-12-27 08:22:44,336][20381] Updated weights for policy 1, policy_version 395678 (0.0006) [2023-12-27 08:22:44,387][20381] Updated weights for policy 1, policy_version 395688 (0.0005) [2023-12-27 08:22:44,486][20347] Updated weights for policy 0, policy_version 395941 (0.0007) [2023-12-27 08:22:44,538][20347] Updated weights for policy 0, policy_version 395951 (0.0008) [2023-12-27 08:22:44,544][20291] KL-divergence is very high: 628.3212 [2023-12-27 08:22:44,558][20291] KL-divergence is very high: 553.5400 [2023-12-27 08:22:44,568][20291] KL-divergence is very high: 223.0707 [2023-12-27 08:22:44,583][20291] KL-divergence is very high: 364.8459 [2023-12-27 08:22:44,589][20347] Updated weights for policy 0, policy_version 395961 (0.0007) [2023-12-27 08:22:45,064][20381] Updated weights for policy 1, policy_version 395698 (0.0007) [2023-12-27 08:22:45,120][20381] Updated weights for policy 1, policy_version 395708 (0.0006) [2023-12-27 08:22:45,171][20381] Updated weights for policy 1, policy_version 395718 (0.0006) [2023-12-27 08:22:45,223][20381] Updated weights for policy 1, policy_version 395728 (0.0008) [2023-12-27 08:22:45,289][20347] Updated weights for policy 0, policy_version 395971 (0.0008) [2023-12-27 08:22:45,355][20347] Updated weights for policy 0, policy_version 395981 (0.0011) [2023-12-27 08:22:45,404][20291] KL-divergence is very high: 150.8369 [2023-12-27 08:22:45,417][20347] Updated weights for policy 0, policy_version 395991 (0.0010) [2023-12-27 08:22:45,858][20381] Updated weights for policy 1, policy_version 395738 (0.0006) [2023-12-27 08:22:45,904][20381] Updated weights for policy 1, policy_version 395748 (0.0010) [2023-12-27 08:22:45,953][20381] Updated weights for policy 1, policy_version 395758 (0.0010) [2023-12-27 08:22:46,046][20347] Updated weights for policy 0, policy_version 396001 (0.0010) [2023-12-27 08:22:46,106][20347] Updated weights for policy 0, policy_version 396011 (0.0006) [2023-12-27 08:22:46,158][20347] Updated weights for policy 0, policy_version 396021 (0.0009) [2023-12-27 08:22:46,220][20347] Updated weights for policy 0, policy_version 396031 (0.0006) [2023-12-27 08:22:46,552][20381] Updated weights for policy 1, policy_version 395768 (0.0010) [2023-12-27 08:22:46,609][20381] Updated weights for policy 1, policy_version 395778 (0.0010) [2023-12-27 08:22:46,657][20381] Updated weights for policy 1, policy_version 395788 (0.0010) [2023-12-27 08:22:46,853][20347] Updated weights for policy 0, policy_version 396041 (0.0007) [2023-12-27 08:22:46,903][20347] Updated weights for policy 0, policy_version 396051 (0.0007) [2023-12-27 08:22:46,961][20291] KL-divergence is very high: 160.2700 [2023-12-27 08:22:46,963][20347] Updated weights for policy 0, policy_version 396061 (0.0007) [2023-12-27 08:22:47,260][20381] Updated weights for policy 1, policy_version 395798 (0.0007) [2023-12-27 08:22:47,314][20381] Updated weights for policy 1, policy_version 395808 (0.0005) [2023-12-27 08:22:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 202752000. Throughput: 0: 10396.7, 1: 10379.1. Samples: 202766216. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:47,363][19170] Avg episode reward: [(0, '804.870'), (1, '1000.000')] [2023-12-27 08:22:47,371][20381] Updated weights for policy 1, policy_version 395818 (0.0008) [2023-12-27 08:22:47,585][20347] Updated weights for policy 0, policy_version 396071 (0.0007) [2023-12-27 08:22:47,643][20347] Updated weights for policy 0, policy_version 396081 (0.0007) [2023-12-27 08:22:47,688][20347] Updated weights for policy 0, policy_version 396091 (0.0007) [2023-12-27 08:22:48,002][20381] Updated weights for policy 1, policy_version 395828 (0.0010) [2023-12-27 08:22:48,051][20381] Updated weights for policy 1, policy_version 395838 (0.0010) [2023-12-27 08:22:48,107][20381] Updated weights for policy 1, policy_version 395848 (0.0010) [2023-12-27 08:22:48,363][20347] Updated weights for policy 0, policy_version 396101 (0.0008) [2023-12-27 08:22:48,423][20347] Updated weights for policy 0, policy_version 396111 (0.0007) [2023-12-27 08:22:48,479][20347] Updated weights for policy 0, policy_version 396121 (0.0008) [2023-12-27 08:22:48,766][20381] Updated weights for policy 1, policy_version 395858 (0.0008) [2023-12-27 08:22:48,835][20381] Updated weights for policy 1, policy_version 395868 (0.0007) [2023-12-27 08:22:48,904][20381] Updated weights for policy 1, policy_version 395878 (0.0007) [2023-12-27 08:22:48,961][20381] Updated weights for policy 1, policy_version 395888 (0.0008) [2023-12-27 08:22:49,153][20347] Updated weights for policy 0, policy_version 396131 (0.0009) [2023-12-27 08:22:49,208][20347] Updated weights for policy 0, policy_version 396141 (0.0007) [2023-12-27 08:22:49,272][20347] Updated weights for policy 0, policy_version 396151 (0.0007) [2023-12-27 08:22:49,650][20381] Updated weights for policy 1, policy_version 395898 (0.0008) [2023-12-27 08:22:49,699][20381] Updated weights for policy 1, policy_version 395908 (0.0006) [2023-12-27 08:22:49,752][20381] Updated weights for policy 1, policy_version 395918 (0.0008) [2023-12-27 08:22:49,952][20347] Updated weights for policy 0, policy_version 396161 (0.0008) [2023-12-27 08:22:50,013][20347] Updated weights for policy 0, policy_version 396171 (0.0008) [2023-12-27 08:22:50,079][20347] Updated weights for policy 0, policy_version 396181 (0.0009) [2023-12-27 08:22:50,142][20347] Updated weights for policy 0, policy_version 396191 (0.0008) [2023-12-27 08:22:50,418][20381] Updated weights for policy 1, policy_version 395928 (0.0007) [2023-12-27 08:22:50,453][20292] KL-divergence is very high: 489.7667 [2023-12-27 08:22:50,463][20381] Updated weights for policy 1, policy_version 395938 (0.0008) [2023-12-27 08:22:50,490][20292] KL-divergence is very high: 138.7862 [2023-12-27 08:22:50,512][20381] Updated weights for policy 1, policy_version 395948 (0.0008) [2023-12-27 08:22:50,828][20347] Updated weights for policy 0, policy_version 396201 (0.0007) [2023-12-27 08:22:50,888][20347] Updated weights for policy 0, policy_version 396211 (0.0008) [2023-12-27 08:22:50,951][20347] Updated weights for policy 0, policy_version 396221 (0.0008) [2023-12-27 08:22:51,208][20381] Updated weights for policy 1, policy_version 395958 (0.0007) [2023-12-27 08:22:51,276][20381] Updated weights for policy 1, policy_version 395968 (0.0007) [2023-12-27 08:22:51,337][20381] Updated weights for policy 1, policy_version 395978 (0.0007) [2023-12-27 08:22:51,668][20347] Updated weights for policy 0, policy_version 396231 (0.0008) [2023-12-27 08:22:51,727][20347] Updated weights for policy 0, policy_version 396241 (0.0009) [2023-12-27 08:22:51,781][20347] Updated weights for policy 0, policy_version 396251 (0.0007) [2023-12-27 08:22:51,786][20291] KL-divergence is very high: 256.3886 [2023-12-27 08:22:51,791][20291] KL-divergence is very high: 290.3680 [2023-12-27 08:22:51,802][20291] KL-divergence is very high: 135.6664 [2023-12-27 08:22:52,062][20381] Updated weights for policy 1, policy_version 395988 (0.0008) [2023-12-27 08:22:52,119][20381] Updated weights for policy 1, policy_version 395998 (0.0007) [2023-12-27 08:22:52,145][20292] KL-divergence is very high: 162.4250 [2023-12-27 08:22:52,181][20381] Updated weights for policy 1, policy_version 396008 (0.0009) [2023-12-27 08:22:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 202858496. Throughput: 0: 10422.7, 1: 10360.0. Samples: 202828824. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:52,363][19170] Avg episode reward: [(0, '671.620'), (1, '971.140')] [2023-12-27 08:22:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000396016_101400576.pth... [2023-12-27 08:22:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000396256_101457920.pth... [2023-12-27 08:22:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000394800_101089280.pth [2023-12-27 08:22:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000395040_101146624.pth [2023-12-27 08:22:52,404][20291] KL-divergence is very high: 130.5283 [2023-12-27 08:22:52,415][20291] KL-divergence is very high: 109.1370 [2023-12-27 08:22:52,421][20347] Updated weights for policy 0, policy_version 396261 (0.0007) [2023-12-27 08:22:52,422][20291] KL-divergence is very high: 102.9183 [2023-12-27 08:22:52,485][20347] Updated weights for policy 0, policy_version 396271 (0.0007) [2023-12-27 08:22:52,554][20347] Updated weights for policy 0, policy_version 396281 (0.0009) [2023-12-27 08:22:52,854][20381] Updated weights for policy 1, policy_version 396018 (0.0007) [2023-12-27 08:22:52,914][20381] Updated weights for policy 1, policy_version 396028 (0.0008) [2023-12-27 08:22:52,975][20381] Updated weights for policy 1, policy_version 396038 (0.0008) [2023-12-27 08:22:53,030][20381] Updated weights for policy 1, policy_version 396048 (0.0008) [2023-12-27 08:22:53,243][20347] Updated weights for policy 0, policy_version 396291 (0.0008) [2023-12-27 08:22:53,300][20347] Updated weights for policy 0, policy_version 396301 (0.0009) [2023-12-27 08:22:53,358][20347] Updated weights for policy 0, policy_version 396311 (0.0009) [2023-12-27 08:22:53,719][20381] Updated weights for policy 1, policy_version 396058 (0.0008) [2023-12-27 08:22:53,769][20381] Updated weights for policy 1, policy_version 396068 (0.0007) [2023-12-27 08:22:53,818][20381] Updated weights for policy 1, policy_version 396078 (0.0005) [2023-12-27 08:22:54,013][20347] Updated weights for policy 0, policy_version 396321 (0.0007) [2023-12-27 08:22:54,057][20347] Updated weights for policy 0, policy_version 396331 (0.0005) [2023-12-27 08:22:54,115][20347] Updated weights for policy 0, policy_version 396341 (0.0005) [2023-12-27 08:22:54,177][20347] Updated weights for policy 0, policy_version 396351 (0.0005) [2023-12-27 08:22:54,419][20381] Updated weights for policy 1, policy_version 396088 (0.0008) [2023-12-27 08:22:54,469][20381] Updated weights for policy 1, policy_version 396098 (0.0009) [2023-12-27 08:22:54,522][20381] Updated weights for policy 1, policy_version 396108 (0.0006) [2023-12-27 08:22:54,794][20347] Updated weights for policy 0, policy_version 396361 (0.0006) [2023-12-27 08:22:54,855][20347] Updated weights for policy 0, policy_version 396371 (0.0005) [2023-12-27 08:22:54,907][20347] Updated weights for policy 0, policy_version 396381 (0.0007) [2023-12-27 08:22:55,295][20381] Updated weights for policy 1, policy_version 396118 (0.0007) [2023-12-27 08:22:55,354][20381] Updated weights for policy 1, policy_version 396128 (0.0006) [2023-12-27 08:22:55,414][20381] Updated weights for policy 1, policy_version 396138 (0.0007) [2023-12-27 08:22:55,520][20347] Updated weights for policy 0, policy_version 396391 (0.0010) [2023-12-27 08:22:55,575][20347] Updated weights for policy 0, policy_version 396401 (0.0010) [2023-12-27 08:22:55,632][20347] Updated weights for policy 0, policy_version 396411 (0.0010) [2023-12-27 08:22:56,024][20381] Updated weights for policy 1, policy_version 396148 (0.0006) [2023-12-27 08:22:56,084][20381] Updated weights for policy 1, policy_version 396158 (0.0005) [2023-12-27 08:22:56,136][20381] Updated weights for policy 1, policy_version 396168 (0.0007) [2023-12-27 08:22:56,234][20347] Updated weights for policy 0, policy_version 396421 (0.0008) [2023-12-27 08:22:56,293][20347] Updated weights for policy 0, policy_version 396431 (0.0005) [2023-12-27 08:22:56,352][20347] Updated weights for policy 0, policy_version 396441 (0.0005) [2023-12-27 08:22:56,747][20381] Updated weights for policy 1, policy_version 396178 (0.0008) [2023-12-27 08:22:56,794][20381] Updated weights for policy 1, policy_version 396188 (0.0007) [2023-12-27 08:22:56,841][20381] Updated weights for policy 1, policy_version 396198 (0.0007) [2023-12-27 08:22:56,873][20292] KL-divergence is very high: 115.7257 [2023-12-27 08:22:56,887][20381] Updated weights for policy 1, policy_version 396208 (0.0007) [2023-12-27 08:22:56,966][20347] Updated weights for policy 0, policy_version 396451 (0.0007) [2023-12-27 08:22:57,019][20347] Updated weights for policy 0, policy_version 396461 (0.0010) [2023-12-27 08:22:57,070][20347] Updated weights for policy 0, policy_version 396471 (0.0010) [2023-12-27 08:22:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 202964992. Throughput: 0: 10433.4, 1: 10387.9. Samples: 202952992. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:22:57,363][19170] Avg episode reward: [(0, '531.340'), (1, '951.710')] [2023-12-27 08:22:57,498][20381] Updated weights for policy 1, policy_version 396218 (0.0005) [2023-12-27 08:22:57,546][20381] Updated weights for policy 1, policy_version 396228 (0.0005) [2023-12-27 08:22:57,595][20381] Updated weights for policy 1, policy_version 396238 (0.0005) [2023-12-27 08:22:57,734][20347] Updated weights for policy 0, policy_version 396481 (0.0010) [2023-12-27 08:22:57,785][20347] Updated weights for policy 0, policy_version 396491 (0.0005) [2023-12-27 08:22:57,833][20347] Updated weights for policy 0, policy_version 396501 (0.0005) [2023-12-27 08:22:57,881][20347] Updated weights for policy 0, policy_version 396511 (0.0005) [2023-12-27 08:22:58,126][20381] Updated weights for policy 1, policy_version 396248 (0.0007) [2023-12-27 08:22:58,186][20381] Updated weights for policy 1, policy_version 396258 (0.0007) [2023-12-27 08:22:58,246][20381] Updated weights for policy 1, policy_version 396268 (0.0007) [2023-12-27 08:22:58,542][20347] Updated weights for policy 0, policy_version 396521 (0.0007) [2023-12-27 08:22:58,607][20347] Updated weights for policy 0, policy_version 396531 (0.0006) [2023-12-27 08:22:58,669][20347] Updated weights for policy 0, policy_version 396541 (0.0009) [2023-12-27 08:22:59,023][20381] Updated weights for policy 1, policy_version 396278 (0.0008) [2023-12-27 08:22:59,080][20381] Updated weights for policy 1, policy_version 396288 (0.0007) [2023-12-27 08:22:59,138][20381] Updated weights for policy 1, policy_version 396298 (0.0008) [2023-12-27 08:22:59,387][20347] Updated weights for policy 0, policy_version 396551 (0.0009) [2023-12-27 08:22:59,447][20347] Updated weights for policy 0, policy_version 396561 (0.0006) [2023-12-27 08:22:59,502][20347] Updated weights for policy 0, policy_version 396571 (0.0010) [2023-12-27 08:22:59,809][20381] Updated weights for policy 1, policy_version 396308 (0.0008) [2023-12-27 08:22:59,872][20381] Updated weights for policy 1, policy_version 396318 (0.0007) [2023-12-27 08:22:59,921][20292] KL-divergence is very high: 152.4246 [2023-12-27 08:22:59,928][20292] KL-divergence is very high: 338.4825 [2023-12-27 08:22:59,934][20381] Updated weights for policy 1, policy_version 396328 (0.0008) [2023-12-27 08:23:00,162][20347] Updated weights for policy 0, policy_version 396581 (0.0009) [2023-12-27 08:23:00,228][20347] Updated weights for policy 0, policy_version 396591 (0.0006) [2023-12-27 08:23:00,294][20347] Updated weights for policy 0, policy_version 396601 (0.0005) [2023-12-27 08:23:00,626][20381] Updated weights for policy 1, policy_version 396338 (0.0007) [2023-12-27 08:23:00,672][20381] Updated weights for policy 1, policy_version 396348 (0.0008) [2023-12-27 08:23:00,721][20292] KL-divergence is very high: 237.2803 [2023-12-27 08:23:00,728][20381] Updated weights for policy 1, policy_version 396358 (0.0008) [2023-12-27 08:23:00,738][20292] KL-divergence is very high: 118.5708 [2023-12-27 08:23:00,780][20381] Updated weights for policy 1, policy_version 396368 (0.0009) [2023-12-27 08:23:00,870][20347] Updated weights for policy 0, policy_version 396611 (0.0005) [2023-12-27 08:23:00,921][20347] Updated weights for policy 0, policy_version 396621 (0.0005) [2023-12-27 08:23:00,965][20347] Updated weights for policy 0, policy_version 396631 (0.0005) [2023-12-27 08:23:01,534][20381] Updated weights for policy 1, policy_version 396378 (0.0007) [2023-12-27 08:23:01,589][20381] Updated weights for policy 1, policy_version 396388 (0.0008) [2023-12-27 08:23:01,631][20292] KL-divergence is very high: 403.6476 [2023-12-27 08:23:01,638][20347] Updated weights for policy 0, policy_version 396641 (0.0005) [2023-12-27 08:23:01,647][20381] Updated weights for policy 1, policy_version 396398 (0.0008) [2023-12-27 08:23:01,701][20347] Updated weights for policy 0, policy_version 396651 (0.0006) [2023-12-27 08:23:01,766][20347] Updated weights for policy 0, policy_version 396661 (0.0009) [2023-12-27 08:23:01,817][20291] KL-divergence is very high: 133.6198 [2023-12-27 08:23:01,830][20347] Updated weights for policy 0, policy_version 396671 (0.0008) [2023-12-27 08:23:02,355][20292] KL-divergence is very high: 295.9278 [2023-12-27 08:23:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20716.1). Total num frames: 203063296. Throughput: 0: 10465.3, 1: 10325.8. Samples: 203078384. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:02,363][19170] Avg episode reward: [(0, '666.730'), (1, '806.770')] [2023-12-27 08:23:02,366][20381] Updated weights for policy 1, policy_version 396408 (0.0010) [2023-12-27 08:23:02,403][20292] KL-divergence is very high: 138.6618 [2023-12-27 08:23:02,428][20381] Updated weights for policy 1, policy_version 396418 (0.0010) [2023-12-27 08:23:02,484][20381] Updated weights for policy 1, policy_version 396428 (0.0011) [2023-12-27 08:23:02,489][20292] KL-divergence is very high: 210.0197 [2023-12-27 08:23:02,496][20292] KL-divergence is very high: 373.3950 [2023-12-27 08:23:02,516][20347] Updated weights for policy 0, policy_version 396681 (0.0006) [2023-12-27 08:23:02,573][20347] Updated weights for policy 0, policy_version 396691 (0.0007) [2023-12-27 08:23:02,629][20347] Updated weights for policy 0, policy_version 396701 (0.0006) [2023-12-27 08:23:03,177][20381] Updated weights for policy 1, policy_version 396438 (0.0011) [2023-12-27 08:23:03,231][20381] Updated weights for policy 1, policy_version 396448 (0.0010) [2023-12-27 08:23:03,248][20292] KL-divergence is very high: 149.6178 [2023-12-27 08:23:03,288][20381] Updated weights for policy 1, policy_version 396458 (0.0009) [2023-12-27 08:23:03,318][20347] Updated weights for policy 0, policy_version 396711 (0.0006) [2023-12-27 08:23:03,376][20347] Updated weights for policy 0, policy_version 396721 (0.0006) [2023-12-27 08:23:03,425][20347] Updated weights for policy 0, policy_version 396731 (0.0007) [2023-12-27 08:23:03,991][20381] Updated weights for policy 1, policy_version 396468 (0.0009) [2023-12-27 08:23:04,052][20381] Updated weights for policy 1, policy_version 396478 (0.0008) [2023-12-27 08:23:04,105][20347] Updated weights for policy 0, policy_version 396741 (0.0006) [2023-12-27 08:23:04,112][20381] Updated weights for policy 1, policy_version 396488 (0.0008) [2023-12-27 08:23:04,158][20347] Updated weights for policy 0, policy_version 396751 (0.0006) [2023-12-27 08:23:04,218][20347] Updated weights for policy 0, policy_version 396761 (0.0007) [2023-12-27 08:23:04,810][20381] Updated weights for policy 1, policy_version 396498 (0.0007) [2023-12-27 08:23:04,867][20381] Updated weights for policy 1, policy_version 396508 (0.0007) [2023-12-27 08:23:04,885][20347] Updated weights for policy 0, policy_version 396771 (0.0007) [2023-12-27 08:23:04,919][20381] Updated weights for policy 1, policy_version 396518 (0.0007) [2023-12-27 08:23:04,943][20292] KL-divergence is very high: 290.9131 [2023-12-27 08:23:04,944][20347] Updated weights for policy 0, policy_version 396781 (0.0006) [2023-12-27 08:23:04,976][20381] Updated weights for policy 1, policy_version 396528 (0.0005) [2023-12-27 08:23:04,999][20347] Updated weights for policy 0, policy_version 396791 (0.0006) [2023-12-27 08:23:05,634][20347] Updated weights for policy 0, policy_version 396801 (0.0008) [2023-12-27 08:23:05,635][20381] Updated weights for policy 1, policy_version 396538 (0.0007) [2023-12-27 08:23:05,683][20381] Updated weights for policy 1, policy_version 396548 (0.0010) [2023-12-27 08:23:05,685][20347] Updated weights for policy 0, policy_version 396811 (0.0010) [2023-12-27 08:23:05,731][20381] Updated weights for policy 1, policy_version 396558 (0.0010) [2023-12-27 08:23:05,739][20347] Updated weights for policy 0, policy_version 396821 (0.0010) [2023-12-27 08:23:05,790][20347] Updated weights for policy 0, policy_version 396831 (0.0010) [2023-12-27 08:23:06,323][20292] KL-divergence is very high: 117.5872 [2023-12-27 08:23:06,330][20381] Updated weights for policy 1, policy_version 396568 (0.0011) [2023-12-27 08:23:06,389][20381] Updated weights for policy 1, policy_version 396578 (0.0010) [2023-12-27 08:23:06,449][20381] Updated weights for policy 1, policy_version 396588 (0.0011) [2023-12-27 08:23:06,458][20347] Updated weights for policy 0, policy_version 396841 (0.0010) [2023-12-27 08:23:06,520][20347] Updated weights for policy 0, policy_version 396851 (0.0010) [2023-12-27 08:23:06,576][20347] Updated weights for policy 0, policy_version 396861 (0.0010) [2023-12-27 08:23:07,076][20381] Updated weights for policy 1, policy_version 396598 (0.0007) [2023-12-27 08:23:07,121][20292] KL-divergence is very high: 549.2042 [2023-12-27 08:23:07,127][20292] KL-divergence is very high: 429.2156 [2023-12-27 08:23:07,133][20381] Updated weights for policy 1, policy_version 396608 (0.0005) [2023-12-27 08:23:07,180][20381] Updated weights for policy 1, policy_version 396618 (0.0005) [2023-12-27 08:23:07,226][20347] Updated weights for policy 0, policy_version 396871 (0.0010) [2023-12-27 08:23:07,275][20347] Updated weights for policy 0, policy_version 396881 (0.0010) [2023-12-27 08:23:07,324][20347] Updated weights for policy 0, policy_version 396891 (0.0010) [2023-12-27 08:23:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 203177984. Throughput: 0: 10466.6, 1: 10281.4. Samples: 203138936. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:07,363][19170] Avg episode reward: [(0, '628.730'), (1, '798.040')] [2023-12-27 08:23:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000396624_101556224.pth... [2023-12-27 08:23:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000396896_101621760.pth... [2023-12-27 08:23:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000395408_101244928.pth [2023-12-27 08:23:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000395648_101302272.pth [2023-12-27 08:23:07,779][20292] KL-divergence is very high: 349.2504 [2023-12-27 08:23:07,784][20292] KL-divergence is very high: 709.2925 [2023-12-27 08:23:07,788][20381] Updated weights for policy 1, policy_version 396628 (0.0007) [2023-12-27 08:23:07,816][20292] KL-divergence is very high: 199.4662 [2023-12-27 08:23:07,821][20292] KL-divergence is very high: 240.1807 [2023-12-27 08:23:07,836][20381] Updated weights for policy 1, policy_version 396638 (0.0010) [2023-12-27 08:23:07,860][20292] KL-divergence is very high: 265.4635 [2023-12-27 08:23:07,865][20292] KL-divergence is very high: 256.9663 [2023-12-27 08:23:07,887][20381] Updated weights for policy 1, policy_version 396648 (0.0010) [2023-12-27 08:23:07,896][20292] KL-divergence is very high: 203.3420 [2023-12-27 08:23:07,901][20292] KL-divergence is very high: 228.8922 [2023-12-27 08:23:08,006][20347] Updated weights for policy 0, policy_version 396901 (0.0009) [2023-12-27 08:23:08,059][20347] Updated weights for policy 0, policy_version 396911 (0.0008) [2023-12-27 08:23:08,114][20347] Updated weights for policy 0, policy_version 396921 (0.0007) [2023-12-27 08:23:08,602][20381] Updated weights for policy 1, policy_version 396658 (0.0011) [2023-12-27 08:23:08,672][20381] Updated weights for policy 1, policy_version 396668 (0.0011) [2023-12-27 08:23:08,734][20381] Updated weights for policy 1, policy_version 396678 (0.0011) [2023-12-27 08:23:08,796][20381] Updated weights for policy 1, policy_version 396688 (0.0011) [2023-12-27 08:23:08,838][20347] Updated weights for policy 0, policy_version 396931 (0.0007) [2023-12-27 08:23:08,885][20347] Updated weights for policy 0, policy_version 396941 (0.0007) [2023-12-27 08:23:08,932][20347] Updated weights for policy 0, policy_version 396951 (0.0007) [2023-12-27 08:23:09,481][20292] KL-divergence is very high: 260.6674 [2023-12-27 08:23:09,481][20381] Updated weights for policy 1, policy_version 396698 (0.0011) [2023-12-27 08:23:09,499][20292] KL-divergence is very high: 145.5024 [2023-12-27 08:23:09,545][20381] Updated weights for policy 1, policy_version 396708 (0.0011) [2023-12-27 08:23:09,611][20381] Updated weights for policy 1, policy_version 396718 (0.0011) [2023-12-27 08:23:09,675][20347] Updated weights for policy 0, policy_version 396961 (0.0007) [2023-12-27 08:23:09,731][20347] Updated weights for policy 0, policy_version 396971 (0.0007) [2023-12-27 08:23:09,787][20347] Updated weights for policy 0, policy_version 396981 (0.0010) [2023-12-27 08:23:09,850][20347] Updated weights for policy 0, policy_version 396991 (0.0008) [2023-12-27 08:23:10,287][20381] Updated weights for policy 1, policy_version 396728 (0.0010) [2023-12-27 08:23:10,342][20381] Updated weights for policy 1, policy_version 396738 (0.0010) [2023-12-27 08:23:10,396][20381] Updated weights for policy 1, policy_version 396748 (0.0011) [2023-12-27 08:23:10,505][20347] Updated weights for policy 0, policy_version 397001 (0.0010) [2023-12-27 08:23:10,565][20347] Updated weights for policy 0, policy_version 397011 (0.0010) [2023-12-27 08:23:10,627][20347] Updated weights for policy 0, policy_version 397021 (0.0010) [2023-12-27 08:23:11,085][20381] Updated weights for policy 1, policy_version 396758 (0.0009) [2023-12-27 08:23:11,143][20381] Updated weights for policy 1, policy_version 396768 (0.0008) [2023-12-27 08:23:11,203][20381] Updated weights for policy 1, policy_version 396778 (0.0008) [2023-12-27 08:23:11,302][20347] Updated weights for policy 0, policy_version 397031 (0.0009) [2023-12-27 08:23:11,360][20347] Updated weights for policy 0, policy_version 397041 (0.0007) [2023-12-27 08:23:11,429][20347] Updated weights for policy 0, policy_version 397051 (0.0007) [2023-12-27 08:23:11,927][20381] Updated weights for policy 1, policy_version 396788 (0.0007) [2023-12-27 08:23:11,989][20381] Updated weights for policy 1, policy_version 396798 (0.0006) [2023-12-27 08:23:12,050][20381] Updated weights for policy 1, policy_version 396808 (0.0008) [2023-12-27 08:23:12,096][20347] Updated weights for policy 0, policy_version 397061 (0.0008) [2023-12-27 08:23:12,156][20347] Updated weights for policy 0, policy_version 397071 (0.0008) [2023-12-27 08:23:12,215][20347] Updated weights for policy 0, policy_version 397081 (0.0007) [2023-12-27 08:23:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 203276288. Throughput: 0: 10496.2, 1: 10318.5. Samples: 203263116. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:12,363][19170] Avg episode reward: [(0, '634.350'), (1, '777.830')] [2023-12-27 08:23:12,728][20381] Updated weights for policy 1, policy_version 396818 (0.0007) [2023-12-27 08:23:12,782][20381] Updated weights for policy 1, policy_version 396828 (0.0006) [2023-12-27 08:23:12,835][20381] Updated weights for policy 1, policy_version 396838 (0.0005) [2023-12-27 08:23:12,881][20381] Updated weights for policy 1, policy_version 396848 (0.0005) [2023-12-27 08:23:12,900][20347] Updated weights for policy 0, policy_version 397091 (0.0007) [2023-12-27 08:23:12,955][20347] Updated weights for policy 0, policy_version 397101 (0.0007) [2023-12-27 08:23:13,014][20347] Updated weights for policy 0, policy_version 397111 (0.0008) [2023-12-27 08:23:13,516][20381] Updated weights for policy 1, policy_version 396858 (0.0007) [2023-12-27 08:23:13,578][20381] Updated weights for policy 1, policy_version 396868 (0.0005) [2023-12-27 08:23:13,611][20292] KL-divergence is very high: 103.8845 [2023-12-27 08:23:13,639][20381] Updated weights for policy 1, policy_version 396878 (0.0005) [2023-12-27 08:23:13,688][20347] Updated weights for policy 0, policy_version 397121 (0.0007) [2023-12-27 08:23:13,743][20347] Updated weights for policy 0, policy_version 397131 (0.0005) [2023-12-27 08:23:13,803][20347] Updated weights for policy 0, policy_version 397141 (0.0007) [2023-12-27 08:23:13,850][20347] Updated weights for policy 0, policy_version 397151 (0.0008) [2023-12-27 08:23:14,202][20381] Updated weights for policy 1, policy_version 396888 (0.0005) [2023-12-27 08:23:14,255][20381] Updated weights for policy 1, policy_version 396898 (0.0005) [2023-12-27 08:23:14,296][20292] KL-divergence is very high: 521.7761 [2023-12-27 08:23:14,311][20381] Updated weights for policy 1, policy_version 396908 (0.0006) [2023-12-27 08:23:14,548][20347] Updated weights for policy 0, policy_version 397161 (0.0007) [2023-12-27 08:23:14,609][20347] Updated weights for policy 0, policy_version 397171 (0.0010) [2023-12-27 08:23:14,670][20347] Updated weights for policy 0, policy_version 397181 (0.0010) [2023-12-27 08:23:14,905][20292] KL-divergence is very high: 212.3611 [2023-12-27 08:23:14,925][20381] Updated weights for policy 1, policy_version 396918 (0.0005) [2023-12-27 08:23:14,989][20381] Updated weights for policy 1, policy_version 396928 (0.0006) [2023-12-27 08:23:15,055][20381] Updated weights for policy 1, policy_version 396938 (0.0007) [2023-12-27 08:23:15,338][20347] Updated weights for policy 0, policy_version 397191 (0.0010) [2023-12-27 08:23:15,399][20347] Updated weights for policy 0, policy_version 397201 (0.0010) [2023-12-27 08:23:15,454][20347] Updated weights for policy 0, policy_version 397211 (0.0010) [2023-12-27 08:23:15,692][20292] KL-divergence is very high: 163.8620 [2023-12-27 08:23:15,705][20381] Updated weights for policy 1, policy_version 396948 (0.0007) [2023-12-27 08:23:15,730][20292] KL-divergence is very high: 121.7803 [2023-12-27 08:23:15,755][20381] Updated weights for policy 1, policy_version 396958 (0.0007) [2023-12-27 08:23:15,802][20381] Updated weights for policy 1, policy_version 396968 (0.0007) [2023-12-27 08:23:16,132][20347] Updated weights for policy 0, policy_version 397221 (0.0009) [2023-12-27 08:23:16,181][20347] Updated weights for policy 0, policy_version 397231 (0.0008) [2023-12-27 08:23:16,227][20347] Updated weights for policy 0, policy_version 397241 (0.0007) [2023-12-27 08:23:16,520][20381] Updated weights for policy 1, policy_version 396978 (0.0007) [2023-12-27 08:23:16,574][20381] Updated weights for policy 1, policy_version 396988 (0.0006) [2023-12-27 08:23:16,628][20381] Updated weights for policy 1, policy_version 396998 (0.0008) [2023-12-27 08:23:16,645][20292] KL-divergence is very high: 132.4907 [2023-12-27 08:23:16,684][20381] Updated weights for policy 1, policy_version 397008 (0.0006) [2023-12-27 08:23:16,859][20347] Updated weights for policy 0, policy_version 397251 (0.0007) [2023-12-27 08:23:16,922][20347] Updated weights for policy 0, policy_version 397261 (0.0005) [2023-12-27 08:23:16,973][20347] Updated weights for policy 0, policy_version 397271 (0.0005) [2023-12-27 08:23:17,250][20381] Updated weights for policy 1, policy_version 397018 (0.0008) [2023-12-27 08:23:17,303][20381] Updated weights for policy 1, policy_version 397028 (0.0008) [2023-12-27 08:23:17,352][20381] Updated weights for policy 1, policy_version 397038 (0.0007) [2023-12-27 08:23:17,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 203382784. Throughput: 0: 10424.5, 1: 10366.5. Samples: 203389216. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:17,364][19170] Avg episode reward: [(0, '636.540'), (1, '778.990')] [2023-12-27 08:23:17,581][20347] Updated weights for policy 0, policy_version 397281 (0.0005) [2023-12-27 08:23:17,633][20347] Updated weights for policy 0, policy_version 397291 (0.0008) [2023-12-27 08:23:17,693][20347] Updated weights for policy 0, policy_version 397301 (0.0007) [2023-12-27 08:23:17,750][20347] Updated weights for policy 0, policy_version 397311 (0.0008) [2023-12-27 08:23:18,019][20381] Updated weights for policy 1, policy_version 397048 (0.0006) [2023-12-27 08:23:18,070][20381] Updated weights for policy 1, policy_version 397058 (0.0007) [2023-12-27 08:23:18,138][20381] Updated weights for policy 1, policy_version 397068 (0.0008) [2023-12-27 08:23:18,343][20347] Updated weights for policy 0, policy_version 397321 (0.0007) [2023-12-27 08:23:18,406][20347] Updated weights for policy 0, policy_version 397331 (0.0006) [2023-12-27 08:23:18,460][20347] Updated weights for policy 0, policy_version 397341 (0.0006) [2023-12-27 08:23:18,852][20381] Updated weights for policy 1, policy_version 397078 (0.0006) [2023-12-27 08:23:18,912][20381] Updated weights for policy 1, policy_version 397088 (0.0008) [2023-12-27 08:23:18,963][20381] Updated weights for policy 1, policy_version 397098 (0.0007) [2023-12-27 08:23:19,134][20347] Updated weights for policy 0, policy_version 397351 (0.0008) [2023-12-27 08:23:19,194][20347] Updated weights for policy 0, policy_version 397361 (0.0007) [2023-12-27 08:23:19,258][20347] Updated weights for policy 0, policy_version 397371 (0.0008) [2023-12-27 08:23:19,645][20381] Updated weights for policy 1, policy_version 397108 (0.0007) [2023-12-27 08:23:19,669][20292] KL-divergence is very high: 133.9083 [2023-12-27 08:23:19,691][20292] KL-divergence is very high: 137.2888 [2023-12-27 08:23:19,696][20292] KL-divergence is very high: 121.9565 [2023-12-27 08:23:19,702][20381] Updated weights for policy 1, policy_version 397118 (0.0006) [2023-12-27 08:23:19,702][20292] KL-divergence is very high: 104.5948 [2023-12-27 08:23:19,711][20292] KL-divergence is very high: 104.1235 [2023-12-27 08:23:19,751][20381] Updated weights for policy 1, policy_version 397128 (0.0006) [2023-12-27 08:23:20,000][20347] Updated weights for policy 0, policy_version 397381 (0.0006) [2023-12-27 08:23:20,032][20291] KL-divergence is very high: 117.1302 [2023-12-27 08:23:20,058][20347] Updated weights for policy 0, policy_version 397391 (0.0010) [2023-12-27 08:23:20,111][20347] Updated weights for policy 0, policy_version 397401 (0.0010) [2023-12-27 08:23:20,438][20381] Updated weights for policy 1, policy_version 397138 (0.0005) [2023-12-27 08:23:20,498][20381] Updated weights for policy 1, policy_version 397148 (0.0005) [2023-12-27 08:23:20,550][20381] Updated weights for policy 1, policy_version 397158 (0.0007) [2023-12-27 08:23:20,614][20381] Updated weights for policy 1, policy_version 397168 (0.0008) [2023-12-27 08:23:20,810][20347] Updated weights for policy 0, policy_version 397411 (0.0010) [2023-12-27 08:23:20,868][20347] Updated weights for policy 0, policy_version 397421 (0.0005) [2023-12-27 08:23:20,926][20347] Updated weights for policy 0, policy_version 397431 (0.0005) [2023-12-27 08:23:21,319][20381] Updated weights for policy 1, policy_version 397178 (0.0008) [2023-12-27 08:23:21,386][20381] Updated weights for policy 1, policy_version 397188 (0.0008) [2023-12-27 08:23:21,457][20381] Updated weights for policy 1, policy_version 397198 (0.0009) [2023-12-27 08:23:21,573][20347] Updated weights for policy 0, policy_version 397441 (0.0006) [2023-12-27 08:23:21,634][20347] Updated weights for policy 0, policy_version 397451 (0.0009) [2023-12-27 08:23:21,695][20347] Updated weights for policy 0, policy_version 397461 (0.0008) [2023-12-27 08:23:21,762][20347] Updated weights for policy 0, policy_version 397471 (0.0008) [2023-12-27 08:23:22,136][20381] Updated weights for policy 1, policy_version 397208 (0.0008) [2023-12-27 08:23:22,189][20381] Updated weights for policy 1, policy_version 397218 (0.0009) [2023-12-27 08:23:22,249][20381] Updated weights for policy 1, policy_version 397228 (0.0008) [2023-12-27 08:23:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 203481088. Throughput: 0: 10425.8, 1: 10335.8. Samples: 203451160. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:22,363][19170] Avg episode reward: [(0, '545.800'), (1, '744.190')] [2023-12-27 08:23:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000397232_101711872.pth... [2023-12-27 08:23:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000397472_101769216.pth... [2023-12-27 08:23:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000396256_101457920.pth [2023-12-27 08:23:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000396016_101400576.pth [2023-12-27 08:23:22,450][20347] Updated weights for policy 0, policy_version 397481 (0.0006) [2023-12-27 08:23:22,514][20347] Updated weights for policy 0, policy_version 397491 (0.0006) [2023-12-27 08:23:22,573][20347] Updated weights for policy 0, policy_version 397501 (0.0005) [2023-12-27 08:23:23,007][20381] Updated weights for policy 1, policy_version 397238 (0.0008) [2023-12-27 08:23:23,055][20292] KL-divergence is very high: 105.0489 [2023-12-27 08:23:23,066][20381] Updated weights for policy 1, policy_version 397248 (0.0009) [2023-12-27 08:23:23,121][20381] Updated weights for policy 1, policy_version 397258 (0.0008) [2023-12-27 08:23:23,188][20347] Updated weights for policy 0, policy_version 397511 (0.0007) [2023-12-27 08:23:23,236][20347] Updated weights for policy 0, policy_version 397521 (0.0005) [2023-12-27 08:23:23,282][20347] Updated weights for policy 0, policy_version 397531 (0.0005) [2023-12-27 08:23:23,839][20381] Updated weights for policy 1, policy_version 397268 (0.0006) [2023-12-27 08:23:23,893][20381] Updated weights for policy 1, policy_version 397278 (0.0008) [2023-12-27 08:23:23,944][20381] Updated weights for policy 1, policy_version 397288 (0.0007) [2023-12-27 08:23:23,962][20347] Updated weights for policy 0, policy_version 397541 (0.0006) [2023-12-27 08:23:24,019][20347] Updated weights for policy 0, policy_version 397551 (0.0007) [2023-12-27 08:23:24,076][20347] Updated weights for policy 0, policy_version 397561 (0.0008) [2023-12-27 08:23:24,682][20381] Updated weights for policy 1, policy_version 397298 (0.0008) [2023-12-27 08:23:24,704][20347] Updated weights for policy 0, policy_version 397571 (0.0007) [2023-12-27 08:23:24,726][20292] KL-divergence is very high: 115.9904 [2023-12-27 08:23:24,731][20292] KL-divergence is very high: 125.0823 [2023-12-27 08:23:24,743][20381] Updated weights for policy 1, policy_version 397308 (0.0007) [2023-12-27 08:23:24,759][20347] Updated weights for policy 0, policy_version 397581 (0.0006) [2023-12-27 08:23:24,791][20381] Updated weights for policy 1, policy_version 397318 (0.0005) [2023-12-27 08:23:24,814][20347] Updated weights for policy 0, policy_version 397591 (0.0007) [2023-12-27 08:23:24,839][20381] Updated weights for policy 1, policy_version 397328 (0.0005) [2023-12-27 08:23:25,508][20347] Updated weights for policy 0, policy_version 397601 (0.0006) [2023-12-27 08:23:25,555][20381] Updated weights for policy 1, policy_version 397338 (0.0008) [2023-12-27 08:23:25,557][20347] Updated weights for policy 0, policy_version 397611 (0.0005) [2023-12-27 08:23:25,615][20347] Updated weights for policy 0, policy_version 397621 (0.0006) [2023-12-27 08:23:25,616][20381] Updated weights for policy 1, policy_version 397348 (0.0008) [2023-12-27 08:23:25,668][20347] Updated weights for policy 0, policy_version 397631 (0.0008) [2023-12-27 08:23:25,678][20381] Updated weights for policy 1, policy_version 397358 (0.0009) [2023-12-27 08:23:26,372][20347] Updated weights for policy 0, policy_version 397641 (0.0006) [2023-12-27 08:23:26,374][20381] Updated weights for policy 1, policy_version 397368 (0.0006) [2023-12-27 08:23:26,387][20292] KL-divergence is very high: 119.3261 [2023-12-27 08:23:26,422][20381] Updated weights for policy 1, policy_version 397378 (0.0007) [2023-12-27 08:23:26,435][20347] Updated weights for policy 0, policy_version 397651 (0.0006) [2023-12-27 08:23:26,466][20381] Updated weights for policy 1, policy_version 397388 (0.0005) [2023-12-27 08:23:26,471][20292] KL-divergence is very high: 113.2571 [2023-12-27 08:23:26,486][20347] Updated weights for policy 0, policy_version 397661 (0.0006) [2023-12-27 08:23:27,160][20381] Updated weights for policy 1, policy_version 397398 (0.0006) [2023-12-27 08:23:27,170][20347] Updated weights for policy 0, policy_version 397671 (0.0006) [2023-12-27 08:23:27,211][20381] Updated weights for policy 1, policy_version 397408 (0.0007) [2023-12-27 08:23:27,216][20347] Updated weights for policy 0, policy_version 397681 (0.0007) [2023-12-27 08:23:27,259][20381] Updated weights for policy 1, policy_version 397418 (0.0005) [2023-12-27 08:23:27,264][20347] Updated weights for policy 0, policy_version 397691 (0.0006) [2023-12-27 08:23:27,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 203587584. Throughput: 0: 10421.1, 1: 10287.1. Samples: 203571488. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:27,364][19170] Avg episode reward: [(0, '659.450'), (1, '748.850')] [2023-12-27 08:23:27,942][20347] Updated weights for policy 0, policy_version 397701 (0.0007) [2023-12-27 08:23:27,970][20381] Updated weights for policy 1, policy_version 397428 (0.0007) [2023-12-27 08:23:27,987][20347] Updated weights for policy 0, policy_version 397711 (0.0006) [2023-12-27 08:23:28,022][20381] Updated weights for policy 1, policy_version 397438 (0.0005) [2023-12-27 08:23:28,038][20347] Updated weights for policy 0, policy_version 397721 (0.0006) [2023-12-27 08:23:28,074][20381] Updated weights for policy 1, policy_version 397448 (0.0006) [2023-12-27 08:23:28,743][20347] Updated weights for policy 0, policy_version 397731 (0.0007) [2023-12-27 08:23:28,777][20381] Updated weights for policy 1, policy_version 397458 (0.0008) [2023-12-27 08:23:28,802][20347] Updated weights for policy 0, policy_version 397741 (0.0006) [2023-12-27 08:23:28,837][20381] Updated weights for policy 1, policy_version 397468 (0.0006) [2023-12-27 08:23:28,858][20347] Updated weights for policy 0, policy_version 397751 (0.0005) [2023-12-27 08:23:28,863][20292] KL-divergence is very high: 257.9341 [2023-12-27 08:23:28,889][20381] Updated weights for policy 1, policy_version 397478 (0.0006) [2023-12-27 08:23:28,940][20381] Updated weights for policy 1, policy_version 397488 (0.0007) [2023-12-27 08:23:29,563][20347] Updated weights for policy 0, policy_version 397761 (0.0006) [2023-12-27 08:23:29,601][20381] Updated weights for policy 1, policy_version 397498 (0.0008) [2023-12-27 08:23:29,622][20347] Updated weights for policy 0, policy_version 397771 (0.0007) [2023-12-27 08:23:29,657][20381] Updated weights for policy 1, policy_version 397508 (0.0006) [2023-12-27 08:23:29,682][20347] Updated weights for policy 0, policy_version 397781 (0.0006) [2023-12-27 08:23:29,714][20381] Updated weights for policy 1, policy_version 397518 (0.0007) [2023-12-27 08:23:29,737][20347] Updated weights for policy 0, policy_version 397791 (0.0007) [2023-12-27 08:23:30,369][20381] Updated weights for policy 1, policy_version 397528 (0.0007) [2023-12-27 08:23:30,408][20347] Updated weights for policy 0, policy_version 397801 (0.0010) [2023-12-27 08:23:30,427][20381] Updated weights for policy 1, policy_version 397538 (0.0006) [2023-12-27 08:23:30,467][20347] Updated weights for policy 0, policy_version 397811 (0.0007) [2023-12-27 08:23:30,482][20381] Updated weights for policy 1, policy_version 397548 (0.0007) [2023-12-27 08:23:30,523][20347] Updated weights for policy 0, policy_version 397821 (0.0005) [2023-12-27 08:23:31,039][20381] Updated weights for policy 1, policy_version 397558 (0.0007) [2023-12-27 08:23:31,088][20381] Updated weights for policy 1, policy_version 397568 (0.0008) [2023-12-27 08:23:31,146][20381] Updated weights for policy 1, policy_version 397578 (0.0008) [2023-12-27 08:23:31,168][20347] Updated weights for policy 0, policy_version 397831 (0.0007) [2023-12-27 08:23:31,222][20347] Updated weights for policy 0, policy_version 397841 (0.0005) [2023-12-27 08:23:31,286][20347] Updated weights for policy 0, policy_version 397851 (0.0006) [2023-12-27 08:23:31,842][20381] Updated weights for policy 1, policy_version 397588 (0.0007) [2023-12-27 08:23:31,893][20381] Updated weights for policy 1, policy_version 397598 (0.0008) [2023-12-27 08:23:31,934][20347] Updated weights for policy 0, policy_version 397861 (0.0008) [2023-12-27 08:23:31,939][20381] Updated weights for policy 1, policy_version 397608 (0.0008) [2023-12-27 08:23:31,989][20347] Updated weights for policy 0, policy_version 397871 (0.0006) [2023-12-27 08:23:32,043][20347] Updated weights for policy 0, policy_version 397881 (0.0007) [2023-12-27 08:23:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 203685888. Throughput: 0: 10364.4, 1: 10283.3. Samples: 203695364. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-12-27 08:23:32,363][19170] Avg episode reward: [(0, '628.950'), (1, '884.630')] [2023-12-27 08:23:32,619][20381] Updated weights for policy 1, policy_version 397618 (0.0007) [2023-12-27 08:23:32,678][20381] Updated weights for policy 1, policy_version 397628 (0.0005) [2023-12-27 08:23:32,684][20347] Updated weights for policy 0, policy_version 397891 (0.0008) [2023-12-27 08:23:32,737][20347] Updated weights for policy 0, policy_version 397901 (0.0011) [2023-12-27 08:23:32,739][20381] Updated weights for policy 1, policy_version 397638 (0.0006) [2023-12-27 08:23:32,800][20347] Updated weights for policy 0, policy_version 397911 (0.0010) [2023-12-27 08:23:32,802][20381] Updated weights for policy 1, policy_version 397648 (0.0006) [2023-12-27 08:23:33,351][20347] Updated weights for policy 0, policy_version 397921 (0.0010) [2023-12-27 08:23:33,407][20347] Updated weights for policy 0, policy_version 397931 (0.0005) [2023-12-27 08:23:33,427][20381] Updated weights for policy 1, policy_version 397658 (0.0005) [2023-12-27 08:23:33,459][20347] Updated weights for policy 0, policy_version 397941 (0.0005) [2023-12-27 08:23:33,474][20381] Updated weights for policy 1, policy_version 397668 (0.0005) [2023-12-27 08:23:33,509][20347] Updated weights for policy 0, policy_version 397951 (0.0010) [2023-12-27 08:23:33,529][20381] Updated weights for policy 1, policy_version 397678 (0.0005) [2023-12-27 08:23:34,104][20381] Updated weights for policy 1, policy_version 397688 (0.0006) [2023-12-27 08:23:34,161][20381] Updated weights for policy 1, policy_version 397698 (0.0008) [2023-12-27 08:23:34,177][20347] Updated weights for policy 0, policy_version 397961 (0.0008) [2023-12-27 08:23:34,214][20381] Updated weights for policy 1, policy_version 397708 (0.0009) [2023-12-27 08:23:34,232][20347] Updated weights for policy 0, policy_version 397971 (0.0005) [2023-12-27 08:23:34,293][20347] Updated weights for policy 0, policy_version 397981 (0.0007) [2023-12-27 08:23:34,934][20381] Updated weights for policy 1, policy_version 397718 (0.0008) [2023-12-27 08:23:34,959][20347] Updated weights for policy 0, policy_version 397991 (0.0007) [2023-12-27 08:23:34,991][20381] Updated weights for policy 1, policy_version 397728 (0.0008) [2023-12-27 08:23:35,018][20347] Updated weights for policy 0, policy_version 398001 (0.0008) [2023-12-27 08:23:35,050][20381] Updated weights for policy 1, policy_version 397738 (0.0007) [2023-12-27 08:23:35,073][20347] Updated weights for policy 0, policy_version 398011 (0.0007) [2023-12-27 08:23:35,633][20381] Updated weights for policy 1, policy_version 397748 (0.0007) [2023-12-27 08:23:35,689][20381] Updated weights for policy 1, policy_version 397758 (0.0005) [2023-12-27 08:23:35,723][20347] Updated weights for policy 0, policy_version 398021 (0.0007) [2023-12-27 08:23:35,748][20381] Updated weights for policy 1, policy_version 397768 (0.0005) [2023-12-27 08:23:35,780][20347] Updated weights for policy 0, policy_version 398031 (0.0006) [2023-12-27 08:23:35,834][20347] Updated weights for policy 0, policy_version 398041 (0.0008) [2023-12-27 08:23:36,350][20381] Updated weights for policy 1, policy_version 397778 (0.0006) [2023-12-27 08:23:36,410][20381] Updated weights for policy 1, policy_version 397788 (0.0005) [2023-12-27 08:23:36,467][20347] Updated weights for policy 0, policy_version 398051 (0.0009) [2023-12-27 08:23:36,473][20381] Updated weights for policy 1, policy_version 397798 (0.0006) [2023-12-27 08:23:36,521][20347] Updated weights for policy 0, policy_version 398061 (0.0006) [2023-12-27 08:23:36,530][20381] Updated weights for policy 1, policy_version 397808 (0.0006) [2023-12-27 08:23:36,583][20347] Updated weights for policy 0, policy_version 398071 (0.0007) [2023-12-27 08:23:37,169][20381] Updated weights for policy 1, policy_version 397818 (0.0007) [2023-12-27 08:23:37,217][20381] Updated weights for policy 1, policy_version 397828 (0.0010) [2023-12-27 08:23:37,246][20347] Updated weights for policy 0, policy_version 398081 (0.0007) [2023-12-27 08:23:37,270][20381] Updated weights for policy 1, policy_version 397838 (0.0008) [2023-12-27 08:23:37,299][20347] Updated weights for policy 0, policy_version 398091 (0.0005) [2023-12-27 08:23:37,349][20347] Updated weights for policy 0, policy_version 398101 (0.0005) [2023-12-27 08:23:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 203792384. Throughput: 0: 10394.2, 1: 10305.0. Samples: 203760284. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:23:37,363][19170] Avg episode reward: [(0, '597.800'), (1, '903.720')] [2023-12-27 08:23:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000397840_101867520.pth... [2023-12-27 08:23:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000396624_101556224.pth [2023-12-27 08:23:37,400][20347] Updated weights for policy 0, policy_version 398111 (0.0007) [2023-12-27 08:23:37,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000398112_101933056.pth... [2023-12-27 08:23:37,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000396896_101621760.pth [2023-12-27 08:23:37,926][20381] Updated weights for policy 1, policy_version 397848 (0.0005) [2023-12-27 08:23:37,976][20381] Updated weights for policy 1, policy_version 397858 (0.0005) [2023-12-27 08:23:38,031][20381] Updated weights for policy 1, policy_version 397868 (0.0005) [2023-12-27 08:23:38,137][20347] Updated weights for policy 0, policy_version 398121 (0.0008) [2023-12-27 08:23:38,188][20347] Updated weights for policy 0, policy_version 398131 (0.0008) [2023-12-27 08:23:38,237][20347] Updated weights for policy 0, policy_version 398141 (0.0005) [2023-12-27 08:23:38,629][20381] Updated weights for policy 1, policy_version 397878 (0.0005) [2023-12-27 08:23:38,686][20381] Updated weights for policy 1, policy_version 397888 (0.0006) [2023-12-27 08:23:38,743][20381] Updated weights for policy 1, policy_version 397898 (0.0008) [2023-12-27 08:23:38,960][20347] Updated weights for policy 0, policy_version 398151 (0.0007) [2023-12-27 08:23:39,021][20347] Updated weights for policy 0, policy_version 398161 (0.0008) [2023-12-27 08:23:39,078][20347] Updated weights for policy 0, policy_version 398171 (0.0007) [2023-12-27 08:23:39,376][20381] Updated weights for policy 1, policy_version 397908 (0.0008) [2023-12-27 08:23:39,438][20381] Updated weights for policy 1, policy_version 397918 (0.0007) [2023-12-27 08:23:39,500][20381] Updated weights for policy 1, policy_version 397928 (0.0008) [2023-12-27 08:23:39,780][20347] Updated weights for policy 0, policy_version 398181 (0.0008) [2023-12-27 08:23:39,841][20347] Updated weights for policy 0, policy_version 398191 (0.0009) [2023-12-27 08:23:39,903][20347] Updated weights for policy 0, policy_version 398201 (0.0006) [2023-12-27 08:23:40,200][20381] Updated weights for policy 1, policy_version 397938 (0.0007) [2023-12-27 08:23:40,258][20381] Updated weights for policy 1, policy_version 397948 (0.0008) [2023-12-27 08:23:40,313][20381] Updated weights for policy 1, policy_version 397958 (0.0006) [2023-12-27 08:23:40,374][20381] Updated weights for policy 1, policy_version 397968 (0.0008) [2023-12-27 08:23:40,656][20347] Updated weights for policy 0, policy_version 398211 (0.0008) [2023-12-27 08:23:40,711][20347] Updated weights for policy 0, policy_version 398221 (0.0008) [2023-12-27 08:23:40,771][20347] Updated weights for policy 0, policy_version 398231 (0.0008) [2023-12-27 08:23:41,027][20381] Updated weights for policy 1, policy_version 397978 (0.0008) [2023-12-27 08:23:41,093][20381] Updated weights for policy 1, policy_version 397988 (0.0006) [2023-12-27 08:23:41,158][20381] Updated weights for policy 1, policy_version 397998 (0.0008) [2023-12-27 08:23:41,474][20347] Updated weights for policy 0, policy_version 398241 (0.0007) [2023-12-27 08:23:41,537][20347] Updated weights for policy 0, policy_version 398251 (0.0005) [2023-12-27 08:23:41,596][20347] Updated weights for policy 0, policy_version 398261 (0.0010) [2023-12-27 08:23:41,662][20347] Updated weights for policy 0, policy_version 398271 (0.0010) [2023-12-27 08:23:41,889][20381] Updated weights for policy 1, policy_version 398008 (0.0007) [2023-12-27 08:23:41,937][20381] Updated weights for policy 1, policy_version 398018 (0.0007) [2023-12-27 08:23:41,987][20381] Updated weights for policy 1, policy_version 398028 (0.0007) [2023-12-27 08:23:42,333][20347] Updated weights for policy 0, policy_version 398281 (0.0011) [2023-12-27 08:23:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 203890688. Throughput: 0: 10334.2, 1: 10364.3. Samples: 203884428. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:23:42,363][19170] Avg episode reward: [(0, '577.560'), (1, '835.700')] [2023-12-27 08:23:42,390][20347] Updated weights for policy 0, policy_version 398291 (0.0010) [2023-12-27 08:23:42,442][20347] Updated weights for policy 0, policy_version 398301 (0.0010) [2023-12-27 08:23:42,716][20381] Updated weights for policy 1, policy_version 398038 (0.0007) [2023-12-27 08:23:42,766][20381] Updated weights for policy 1, policy_version 398048 (0.0007) [2023-12-27 08:23:42,823][20381] Updated weights for policy 1, policy_version 398058 (0.0007) [2023-12-27 08:23:43,109][20347] Updated weights for policy 0, policy_version 398311 (0.0009) [2023-12-27 08:23:43,171][20347] Updated weights for policy 0, policy_version 398321 (0.0008) [2023-12-27 08:23:43,238][20347] Updated weights for policy 0, policy_version 398331 (0.0010) [2023-12-27 08:23:43,562][20381] Updated weights for policy 1, policy_version 398068 (0.0008) [2023-12-27 08:23:43,629][20381] Updated weights for policy 1, policy_version 398078 (0.0009) [2023-12-27 08:23:43,681][20381] Updated weights for policy 1, policy_version 398088 (0.0007) [2023-12-27 08:23:43,810][20347] Updated weights for policy 0, policy_version 398341 (0.0008) [2023-12-27 08:23:43,858][20347] Updated weights for policy 0, policy_version 398351 (0.0005) [2023-12-27 08:23:43,915][20347] Updated weights for policy 0, policy_version 398361 (0.0008) [2023-12-27 08:23:44,390][20381] Updated weights for policy 1, policy_version 398098 (0.0009) [2023-12-27 08:23:44,446][20381] Updated weights for policy 1, policy_version 398108 (0.0006) [2023-12-27 08:23:44,503][20381] Updated weights for policy 1, policy_version 398118 (0.0007) [2023-12-27 08:23:44,557][20381] Updated weights for policy 1, policy_version 398128 (0.0007) [2023-12-27 08:23:44,562][20347] Updated weights for policy 0, policy_version 398371 (0.0009) [2023-12-27 08:23:44,624][20347] Updated weights for policy 0, policy_version 398381 (0.0007) [2023-12-27 08:23:44,668][20347] Updated weights for policy 0, policy_version 398391 (0.0007) [2023-12-27 08:23:45,260][20381] Updated weights for policy 1, policy_version 398138 (0.0008) [2023-12-27 08:23:45,309][20381] Updated weights for policy 1, policy_version 398148 (0.0008) [2023-12-27 08:23:45,357][20381] Updated weights for policy 1, policy_version 398158 (0.0007) [2023-12-27 08:23:45,358][20347] Updated weights for policy 0, policy_version 398401 (0.0008) [2023-12-27 08:23:45,412][20347] Updated weights for policy 0, policy_version 398411 (0.0010) [2023-12-27 08:23:45,471][20347] Updated weights for policy 0, policy_version 398421 (0.0010) [2023-12-27 08:23:45,533][20347] Updated weights for policy 0, policy_version 398431 (0.0010) [2023-12-27 08:23:46,058][20381] Updated weights for policy 1, policy_version 398168 (0.0007) [2023-12-27 08:23:46,122][20381] Updated weights for policy 1, policy_version 398178 (0.0006) [2023-12-27 08:23:46,171][20381] Updated weights for policy 1, policy_version 398188 (0.0005) [2023-12-27 08:23:46,187][20347] Updated weights for policy 0, policy_version 398441 (0.0010) [2023-12-27 08:23:46,241][20347] Updated weights for policy 0, policy_version 398451 (0.0010) [2023-12-27 08:23:46,300][20347] Updated weights for policy 0, policy_version 398461 (0.0010) [2023-12-27 08:23:46,763][20381] Updated weights for policy 1, policy_version 398198 (0.0006) [2023-12-27 08:23:46,822][20381] Updated weights for policy 1, policy_version 398208 (0.0006) [2023-12-27 08:23:46,872][20381] Updated weights for policy 1, policy_version 398218 (0.0005) [2023-12-27 08:23:46,967][20347] Updated weights for policy 0, policy_version 398471 (0.0007) [2023-12-27 08:23:47,013][20347] Updated weights for policy 0, policy_version 398481 (0.0005) [2023-12-27 08:23:47,067][20347] Updated weights for policy 0, policy_version 398491 (0.0005) [2023-12-27 08:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 203997184. Throughput: 0: 10318.7, 1: 10347.6. Samples: 204008364. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:23:47,363][19170] Avg episode reward: [(0, '582.150'), (1, '854.780')] [2023-12-27 08:23:47,453][20381] Updated weights for policy 1, policy_version 398228 (0.0006) [2023-12-27 08:23:47,509][20381] Updated weights for policy 1, policy_version 398238 (0.0006) [2023-12-27 08:23:47,573][20381] Updated weights for policy 1, policy_version 398248 (0.0005) [2023-12-27 08:23:47,650][20347] Updated weights for policy 0, policy_version 398501 (0.0008) [2023-12-27 08:23:47,704][20347] Updated weights for policy 0, policy_version 398511 (0.0010) [2023-12-27 08:23:47,762][20347] Updated weights for policy 0, policy_version 398521 (0.0010) [2023-12-27 08:23:48,183][20381] Updated weights for policy 1, policy_version 398258 (0.0007) [2023-12-27 08:23:48,232][20381] Updated weights for policy 1, policy_version 398268 (0.0007) [2023-12-27 08:23:48,283][20381] Updated weights for policy 1, policy_version 398278 (0.0005) [2023-12-27 08:23:48,334][20381] Updated weights for policy 1, policy_version 398288 (0.0006) [2023-12-27 08:23:48,360][20347] Updated weights for policy 0, policy_version 398531 (0.0010) [2023-12-27 08:23:48,419][20347] Updated weights for policy 0, policy_version 398541 (0.0010) [2023-12-27 08:23:48,479][20347] Updated weights for policy 0, policy_version 398551 (0.0008) [2023-12-27 08:23:49,024][20381] Updated weights for policy 1, policy_version 398298 (0.0011) [2023-12-27 08:23:49,049][20347] Updated weights for policy 0, policy_version 398561 (0.0010) [2023-12-27 08:23:49,084][20381] Updated weights for policy 1, policy_version 398308 (0.0011) [2023-12-27 08:23:49,108][20347] Updated weights for policy 0, policy_version 398571 (0.0008) [2023-12-27 08:23:49,146][20381] Updated weights for policy 1, policy_version 398318 (0.0008) [2023-12-27 08:23:49,164][20347] Updated weights for policy 0, policy_version 398581 (0.0010) [2023-12-27 08:23:49,224][20347] Updated weights for policy 0, policy_version 398591 (0.0010) [2023-12-27 08:23:49,726][20381] Updated weights for policy 1, policy_version 398328 (0.0006) [2023-12-27 08:23:49,782][20381] Updated weights for policy 1, policy_version 398338 (0.0008) [2023-12-27 08:23:49,840][20381] Updated weights for policy 1, policy_version 398348 (0.0011) [2023-12-27 08:23:49,889][20347] Updated weights for policy 0, policy_version 398601 (0.0007) [2023-12-27 08:23:49,937][20291] KL-divergence is very high: 148.9469 [2023-12-27 08:23:49,948][20291] KL-divergence is very high: 116.5748 [2023-12-27 08:23:49,953][20347] Updated weights for policy 0, policy_version 398611 (0.0010) [2023-12-27 08:23:50,014][20347] Updated weights for policy 0, policy_version 398621 (0.0010) [2023-12-27 08:23:50,506][20381] Updated weights for policy 1, policy_version 398358 (0.0011) [2023-12-27 08:23:50,559][20381] Updated weights for policy 1, policy_version 398368 (0.0010) [2023-12-27 08:23:50,622][20381] Updated weights for policy 1, policy_version 398378 (0.0011) [2023-12-27 08:23:50,663][20347] Updated weights for policy 0, policy_version 398631 (0.0009) [2023-12-27 08:23:50,718][20347] Updated weights for policy 0, policy_version 398641 (0.0010) [2023-12-27 08:23:50,772][20347] Updated weights for policy 0, policy_version 398651 (0.0010) [2023-12-27 08:23:51,356][20381] Updated weights for policy 1, policy_version 398388 (0.0010) [2023-12-27 08:23:51,419][20381] Updated weights for policy 1, policy_version 398398 (0.0007) [2023-12-27 08:23:51,471][20381] Updated weights for policy 1, policy_version 398408 (0.0008) [2023-12-27 08:23:51,497][20347] Updated weights for policy 0, policy_version 398661 (0.0010) [2023-12-27 08:23:51,555][20347] Updated weights for policy 0, policy_version 398671 (0.0010) [2023-12-27 08:23:51,614][20347] Updated weights for policy 0, policy_version 398681 (0.0010) [2023-12-27 08:23:52,144][20381] Updated weights for policy 1, policy_version 398418 (0.0010) [2023-12-27 08:23:52,204][20381] Updated weights for policy 1, policy_version 398428 (0.0011) [2023-12-27 08:23:52,260][20381] Updated weights for policy 1, policy_version 398438 (0.0010) [2023-12-27 08:23:52,295][20347] Updated weights for policy 0, policy_version 398691 (0.0007) [2023-12-27 08:23:52,316][20381] Updated weights for policy 1, policy_version 398448 (0.0011) [2023-12-27 08:23:52,361][20347] Updated weights for policy 0, policy_version 398701 (0.0008) [2023-12-27 08:23:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 204103680. Throughput: 0: 10384.1, 1: 10388.5. Samples: 204073704. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:23:52,363][19170] Avg episode reward: [(0, '523.210'), (1, '874.640')] [2023-12-27 08:23:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000398448_102023168.pth... [2023-12-27 08:23:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000397232_101711872.pth [2023-12-27 08:23:52,416][20347] Updated weights for policy 0, policy_version 398711 (0.0008) [2023-12-27 08:23:52,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000398720_102088704.pth... [2023-12-27 08:23:52,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000397472_101769216.pth [2023-12-27 08:23:53,047][20381] Updated weights for policy 1, policy_version 398458 (0.0009) [2023-12-27 08:23:53,101][20381] Updated weights for policy 1, policy_version 398468 (0.0005) [2023-12-27 08:23:53,143][20347] Updated weights for policy 0, policy_version 398721 (0.0008) [2023-12-27 08:23:53,151][20381] Updated weights for policy 1, policy_version 398478 (0.0008) [2023-12-27 08:23:53,199][20347] Updated weights for policy 0, policy_version 398731 (0.0007) [2023-12-27 08:23:53,257][20347] Updated weights for policy 0, policy_version 398741 (0.0007) [2023-12-27 08:23:53,310][20347] Updated weights for policy 0, policy_version 398751 (0.0007) [2023-12-27 08:23:53,833][20381] Updated weights for policy 1, policy_version 398488 (0.0011) [2023-12-27 08:23:53,881][20381] Updated weights for policy 1, policy_version 398498 (0.0009) [2023-12-27 08:23:53,932][20381] Updated weights for policy 1, policy_version 398508 (0.0007) [2023-12-27 08:23:53,982][20347] Updated weights for policy 0, policy_version 398761 (0.0007) [2023-12-27 08:23:54,033][20347] Updated weights for policy 0, policy_version 398771 (0.0008) [2023-12-27 08:23:54,084][20347] Updated weights for policy 0, policy_version 398781 (0.0008) [2023-12-27 08:23:54,608][20381] Updated weights for policy 1, policy_version 398518 (0.0009) [2023-12-27 08:23:54,657][20381] Updated weights for policy 1, policy_version 398528 (0.0009) [2023-12-27 08:23:54,712][20381] Updated weights for policy 1, policy_version 398538 (0.0006) [2023-12-27 08:23:54,748][20347] Updated weights for policy 0, policy_version 398791 (0.0006) [2023-12-27 08:23:54,814][20347] Updated weights for policy 0, policy_version 398801 (0.0007) [2023-12-27 08:23:54,868][20347] Updated weights for policy 0, policy_version 398811 (0.0007) [2023-12-27 08:23:55,345][20381] Updated weights for policy 1, policy_version 398548 (0.0006) [2023-12-27 08:23:55,404][20381] Updated weights for policy 1, policy_version 398558 (0.0009) [2023-12-27 08:23:55,456][20381] Updated weights for policy 1, policy_version 398568 (0.0011) [2023-12-27 08:23:55,504][20347] Updated weights for policy 0, policy_version 398821 (0.0008) [2023-12-27 08:23:55,565][20347] Updated weights for policy 0, policy_version 398831 (0.0008) [2023-12-27 08:23:55,620][20347] Updated weights for policy 0, policy_version 398841 (0.0008) [2023-12-27 08:23:56,105][20381] Updated weights for policy 1, policy_version 398578 (0.0010) [2023-12-27 08:23:56,177][20381] Updated weights for policy 1, policy_version 398588 (0.0011) [2023-12-27 08:23:56,238][20381] Updated weights for policy 1, policy_version 398598 (0.0011) [2023-12-27 08:23:56,289][20347] Updated weights for policy 0, policy_version 398851 (0.0007) [2023-12-27 08:23:56,294][20381] Updated weights for policy 1, policy_version 398608 (0.0007) [2023-12-27 08:23:56,341][20347] Updated weights for policy 0, policy_version 398861 (0.0007) [2023-12-27 08:23:56,400][20347] Updated weights for policy 0, policy_version 398871 (0.0008) [2023-12-27 08:23:56,934][20381] Updated weights for policy 1, policy_version 398618 (0.0005) [2023-12-27 08:23:56,999][20381] Updated weights for policy 1, policy_version 398628 (0.0006) [2023-12-27 08:23:57,057][20381] Updated weights for policy 1, policy_version 398638 (0.0005) [2023-12-27 08:23:57,080][20347] Updated weights for policy 0, policy_version 398881 (0.0007) [2023-12-27 08:23:57,125][20347] Updated weights for policy 0, policy_version 398891 (0.0007) [2023-12-27 08:23:57,174][20347] Updated weights for policy 0, policy_version 398901 (0.0006) [2023-12-27 08:23:57,229][20347] Updated weights for policy 0, policy_version 398911 (0.0006) [2023-12-27 08:23:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 204210176. Throughput: 0: 10370.1, 1: 10360.1. Samples: 204195972. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:23:57,363][19170] Avg episode reward: [(0, '548.800'), (1, '968.320')] [2023-12-27 08:23:57,693][20381] Updated weights for policy 1, policy_version 398648 (0.0010) [2023-12-27 08:23:57,745][20381] Updated weights for policy 1, policy_version 398658 (0.0010) [2023-12-27 08:23:57,783][20347] Updated weights for policy 0, policy_version 398921 (0.0009) [2023-12-27 08:23:57,793][20381] Updated weights for policy 1, policy_version 398668 (0.0010) [2023-12-27 08:23:57,841][20347] Updated weights for policy 0, policy_version 398931 (0.0010) [2023-12-27 08:23:57,895][20347] Updated weights for policy 0, policy_version 398941 (0.0010) [2023-12-27 08:23:58,514][20381] Updated weights for policy 1, policy_version 398678 (0.0010) [2023-12-27 08:23:58,564][20347] Updated weights for policy 0, policy_version 398951 (0.0007) [2023-12-27 08:23:58,578][20381] Updated weights for policy 1, policy_version 398688 (0.0008) [2023-12-27 08:23:58,633][20347] Updated weights for policy 0, policy_version 398961 (0.0006) [2023-12-27 08:23:58,638][20381] Updated weights for policy 1, policy_version 398698 (0.0008) [2023-12-27 08:23:58,693][20347] Updated weights for policy 0, policy_version 398971 (0.0008) [2023-12-27 08:23:59,384][20381] Updated weights for policy 1, policy_version 398708 (0.0008) [2023-12-27 08:23:59,413][20347] Updated weights for policy 0, policy_version 398981 (0.0010) [2023-12-27 08:23:59,435][20381] Updated weights for policy 1, policy_version 398718 (0.0008) [2023-12-27 08:23:59,470][20347] Updated weights for policy 0, policy_version 398991 (0.0010) [2023-12-27 08:23:59,492][20381] Updated weights for policy 1, policy_version 398728 (0.0008) [2023-12-27 08:23:59,531][20347] Updated weights for policy 0, policy_version 399001 (0.0010) [2023-12-27 08:24:00,137][20381] Updated weights for policy 1, policy_version 398738 (0.0006) [2023-12-27 08:24:00,151][20347] Updated weights for policy 0, policy_version 399011 (0.0009) [2023-12-27 08:24:00,187][20381] Updated weights for policy 1, policy_version 398748 (0.0006) [2023-12-27 08:24:00,204][20347] Updated weights for policy 0, policy_version 399021 (0.0005) [2023-12-27 08:24:00,239][20381] Updated weights for policy 1, policy_version 398758 (0.0005) [2023-12-27 08:24:00,266][20347] Updated weights for policy 0, policy_version 399031 (0.0006) [2023-12-27 08:24:00,298][20381] Updated weights for policy 1, policy_version 398768 (0.0007) [2023-12-27 08:24:00,869][20347] Updated weights for policy 0, policy_version 399041 (0.0007) [2023-12-27 08:24:00,914][20347] Updated weights for policy 0, policy_version 399051 (0.0006) [2023-12-27 08:24:00,935][20381] Updated weights for policy 1, policy_version 398778 (0.0006) [2023-12-27 08:24:00,961][20347] Updated weights for policy 0, policy_version 399061 (0.0005) [2023-12-27 08:24:00,980][20291] KL-divergence is very high: 119.8276 [2023-12-27 08:24:00,985][20381] Updated weights for policy 1, policy_version 398788 (0.0007) [2023-12-27 08:24:01,009][20347] Updated weights for policy 0, policy_version 399071 (0.0005) [2023-12-27 08:24:01,037][20381] Updated weights for policy 1, policy_version 398798 (0.0008) [2023-12-27 08:24:01,646][20347] Updated weights for policy 0, policy_version 399081 (0.0007) [2023-12-27 08:24:01,702][20347] Updated weights for policy 0, policy_version 399091 (0.0008) [2023-12-27 08:24:01,762][20347] Updated weights for policy 0, policy_version 399101 (0.0007) [2023-12-27 08:24:01,775][20381] Updated weights for policy 1, policy_version 398808 (0.0008) [2023-12-27 08:24:01,833][20381] Updated weights for policy 1, policy_version 398818 (0.0007) [2023-12-27 08:24:01,895][20381] Updated weights for policy 1, policy_version 398828 (0.0008) [2023-12-27 08:24:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 204308480. Throughput: 0: 10409.2, 1: 10292.9. Samples: 204320808. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:02,363][19170] Avg episode reward: [(0, '714.620'), (1, '996.970')] [2023-12-27 08:24:02,523][20347] Updated weights for policy 0, policy_version 399111 (0.0006) [2023-12-27 08:24:02,535][20381] Updated weights for policy 1, policy_version 398838 (0.0009) [2023-12-27 08:24:02,577][20347] Updated weights for policy 0, policy_version 399121 (0.0005) [2023-12-27 08:24:02,590][20381] Updated weights for policy 1, policy_version 398848 (0.0010) [2023-12-27 08:24:02,633][20347] Updated weights for policy 0, policy_version 399131 (0.0009) [2023-12-27 08:24:02,648][20381] Updated weights for policy 1, policy_version 398858 (0.0010) [2023-12-27 08:24:03,271][20347] Updated weights for policy 0, policy_version 399141 (0.0008) [2023-12-27 08:24:03,314][20381] Updated weights for policy 1, policy_version 398868 (0.0010) [2023-12-27 08:24:03,325][20347] Updated weights for policy 0, policy_version 399151 (0.0006) [2023-12-27 08:24:03,362][20381] Updated weights for policy 1, policy_version 398878 (0.0010) [2023-12-27 08:24:03,377][20347] Updated weights for policy 0, policy_version 399161 (0.0006) [2023-12-27 08:24:03,413][20381] Updated weights for policy 1, policy_version 398888 (0.0010) [2023-12-27 08:24:03,935][20347] Updated weights for policy 0, policy_version 399171 (0.0006) [2023-12-27 08:24:03,993][20347] Updated weights for policy 0, policy_version 399181 (0.0006) [2023-12-27 08:24:04,055][20347] Updated weights for policy 0, policy_version 399191 (0.0006) [2023-12-27 08:24:04,148][20381] Updated weights for policy 1, policy_version 398898 (0.0010) [2023-12-27 08:24:04,210][20381] Updated weights for policy 1, policy_version 398908 (0.0007) [2023-12-27 08:24:04,267][20381] Updated weights for policy 1, policy_version 398918 (0.0008) [2023-12-27 08:24:04,334][20381] Updated weights for policy 1, policy_version 398928 (0.0008) [2023-12-27 08:24:04,679][20347] Updated weights for policy 0, policy_version 399201 (0.0005) [2023-12-27 08:24:04,745][20347] Updated weights for policy 0, policy_version 399211 (0.0008) [2023-12-27 08:24:04,798][20347] Updated weights for policy 0, policy_version 399221 (0.0007) [2023-12-27 08:24:04,848][20347] Updated weights for policy 0, policy_version 399231 (0.0006) [2023-12-27 08:24:05,003][20381] Updated weights for policy 1, policy_version 398938 (0.0009) [2023-12-27 08:24:05,057][20381] Updated weights for policy 1, policy_version 398948 (0.0008) [2023-12-27 08:24:05,110][20381] Updated weights for policy 1, policy_version 398958 (0.0007) [2023-12-27 08:24:05,505][20347] Updated weights for policy 0, policy_version 399241 (0.0008) [2023-12-27 08:24:05,563][20347] Updated weights for policy 0, policy_version 399251 (0.0008) [2023-12-27 08:24:05,621][20347] Updated weights for policy 0, policy_version 399261 (0.0008) [2023-12-27 08:24:05,787][20381] Updated weights for policy 1, policy_version 398968 (0.0008) [2023-12-27 08:24:05,840][20381] Updated weights for policy 1, policy_version 398978 (0.0006) [2023-12-27 08:24:05,895][20381] Updated weights for policy 1, policy_version 398988 (0.0005) [2023-12-27 08:24:06,299][20347] Updated weights for policy 0, policy_version 399271 (0.0007) [2023-12-27 08:24:06,365][20347] Updated weights for policy 0, policy_version 399281 (0.0007) [2023-12-27 08:24:06,423][20347] Updated weights for policy 0, policy_version 399291 (0.0008) [2023-12-27 08:24:06,639][20381] Updated weights for policy 1, policy_version 398998 (0.0008) [2023-12-27 08:24:06,696][20381] Updated weights for policy 1, policy_version 399008 (0.0011) [2023-12-27 08:24:06,759][20381] Updated weights for policy 1, policy_version 399018 (0.0011) [2023-12-27 08:24:07,058][20347] Updated weights for policy 0, policy_version 399301 (0.0007) [2023-12-27 08:24:07,109][20347] Updated weights for policy 0, policy_version 399311 (0.0007) [2023-12-27 08:24:07,160][20347] Updated weights for policy 0, policy_version 399321 (0.0007) [2023-12-27 08:24:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 204414976. Throughput: 0: 10435.6, 1: 10285.1. Samples: 204383592. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:07,363][19170] Avg episode reward: [(0, '734.400'), (1, '981.190')] [2023-12-27 08:24:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000399328_102244352.pth... [2023-12-27 08:24:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000399024_102170624.pth... [2023-12-27 08:24:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000398112_101933056.pth [2023-12-27 08:24:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000397840_101867520.pth [2023-12-27 08:24:07,455][20381] Updated weights for policy 1, policy_version 399028 (0.0010) [2023-12-27 08:24:07,507][20381] Updated weights for policy 1, policy_version 399038 (0.0010) [2023-12-27 08:24:07,556][20381] Updated weights for policy 1, policy_version 399048 (0.0010) [2023-12-27 08:24:07,780][20347] Updated weights for policy 0, policy_version 399331 (0.0007) [2023-12-27 08:24:07,838][20347] Updated weights for policy 0, policy_version 399341 (0.0005) [2023-12-27 08:24:07,897][20347] Updated weights for policy 0, policy_version 399351 (0.0006) [2023-12-27 08:24:08,200][20381] Updated weights for policy 1, policy_version 399058 (0.0010) [2023-12-27 08:24:08,249][20381] Updated weights for policy 1, policy_version 399068 (0.0010) [2023-12-27 08:24:08,298][20381] Updated weights for policy 1, policy_version 399078 (0.0010) [2023-12-27 08:24:08,351][20381] Updated weights for policy 1, policy_version 399088 (0.0010) [2023-12-27 08:24:08,528][20347] Updated weights for policy 0, policy_version 399361 (0.0007) [2023-12-27 08:24:08,588][20347] Updated weights for policy 0, policy_version 399371 (0.0007) [2023-12-27 08:24:08,650][20347] Updated weights for policy 0, policy_version 399381 (0.0005) [2023-12-27 08:24:08,706][20347] Updated weights for policy 0, policy_version 399391 (0.0005) [2023-12-27 08:24:09,010][20381] Updated weights for policy 1, policy_version 399098 (0.0010) [2023-12-27 08:24:09,067][20381] Updated weights for policy 1, policy_version 399108 (0.0010) [2023-12-27 08:24:09,128][20381] Updated weights for policy 1, policy_version 399118 (0.0006) [2023-12-27 08:24:09,366][20347] Updated weights for policy 0, policy_version 399401 (0.0008) [2023-12-27 08:24:09,422][20347] Updated weights for policy 0, policy_version 399411 (0.0008) [2023-12-27 08:24:09,485][20347] Updated weights for policy 0, policy_version 399421 (0.0008) [2023-12-27 08:24:09,832][20381] Updated weights for policy 1, policy_version 399128 (0.0009) [2023-12-27 08:24:09,888][20381] Updated weights for policy 1, policy_version 399138 (0.0007) [2023-12-27 08:24:09,949][20381] Updated weights for policy 1, policy_version 399148 (0.0007) [2023-12-27 08:24:10,216][20347] Updated weights for policy 0, policy_version 399431 (0.0008) [2023-12-27 08:24:10,272][20347] Updated weights for policy 0, policy_version 399441 (0.0007) [2023-12-27 08:24:10,332][20347] Updated weights for policy 0, policy_version 399451 (0.0011) [2023-12-27 08:24:10,618][20381] Updated weights for policy 1, policy_version 399158 (0.0007) [2023-12-27 08:24:10,680][20381] Updated weights for policy 1, policy_version 399168 (0.0007) [2023-12-27 08:24:10,744][20381] Updated weights for policy 1, policy_version 399178 (0.0007) [2023-12-27 08:24:11,067][20347] Updated weights for policy 0, policy_version 399461 (0.0010) [2023-12-27 08:24:11,127][20347] Updated weights for policy 0, policy_version 399471 (0.0008) [2023-12-27 08:24:11,188][20347] Updated weights for policy 0, policy_version 399481 (0.0007) [2023-12-27 08:24:11,438][20381] Updated weights for policy 1, policy_version 399188 (0.0008) [2023-12-27 08:24:11,501][20381] Updated weights for policy 1, policy_version 399198 (0.0007) [2023-12-27 08:24:11,558][20381] Updated weights for policy 1, policy_version 399208 (0.0009) [2023-12-27 08:24:11,959][20347] Updated weights for policy 0, policy_version 399491 (0.0007) [2023-12-27 08:24:12,015][20347] Updated weights for policy 0, policy_version 399501 (0.0010) [2023-12-27 08:24:12,065][20347] Updated weights for policy 0, policy_version 399511 (0.0010) [2023-12-27 08:24:12,261][20381] Updated weights for policy 1, policy_version 399218 (0.0010) [2023-12-27 08:24:12,272][20292] KL-divergence is very high: 132.4879 [2023-12-27 08:24:12,321][20381] Updated weights for policy 1, policy_version 399228 (0.0008) [2023-12-27 08:24:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 204513280. Throughput: 0: 10437.8, 1: 10365.8. Samples: 204507648. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:12,363][19170] Avg episode reward: [(0, '684.550'), (1, '962.080')] [2023-12-27 08:24:12,379][20381] Updated weights for policy 1, policy_version 399238 (0.0008) [2023-12-27 08:24:12,409][20292] KL-divergence is very high: 139.1950 [2023-12-27 08:24:12,431][20381] Updated weights for policy 1, policy_version 399248 (0.0008) [2023-12-27 08:24:12,787][20347] Updated weights for policy 0, policy_version 399521 (0.0010) [2023-12-27 08:24:12,844][20347] Updated weights for policy 0, policy_version 399531 (0.0007) [2023-12-27 08:24:12,892][20347] Updated weights for policy 0, policy_version 399541 (0.0008) [2023-12-27 08:24:12,930][20291] KL-divergence is very high: 750.0426 [2023-12-27 08:24:12,941][20347] Updated weights for policy 0, policy_version 399551 (0.0007) [2023-12-27 08:24:13,141][20381] Updated weights for policy 1, policy_version 399258 (0.0007) [2023-12-27 08:24:13,195][20381] Updated weights for policy 1, policy_version 399268 (0.0007) [2023-12-27 08:24:13,245][20381] Updated weights for policy 1, policy_version 399278 (0.0007) [2023-12-27 08:24:13,625][20347] Updated weights for policy 0, policy_version 399561 (0.0009) [2023-12-27 08:24:13,669][20347] Updated weights for policy 0, policy_version 399571 (0.0010) [2023-12-27 08:24:13,721][20347] Updated weights for policy 0, policy_version 399581 (0.0008) [2023-12-27 08:24:13,920][20381] Updated weights for policy 1, policy_version 399288 (0.0005) [2023-12-27 08:24:13,973][20381] Updated weights for policy 1, policy_version 399298 (0.0005) [2023-12-27 08:24:14,031][20381] Updated weights for policy 1, policy_version 399308 (0.0005) [2023-12-27 08:24:14,332][20347] Updated weights for policy 0, policy_version 399591 (0.0008) [2023-12-27 08:24:14,393][20347] Updated weights for policy 0, policy_version 399601 (0.0010) [2023-12-27 08:24:14,441][20347] Updated weights for policy 0, policy_version 399611 (0.0010) [2023-12-27 08:24:14,547][20381] Updated weights for policy 1, policy_version 399318 (0.0007) [2023-12-27 08:24:14,595][20381] Updated weights for policy 1, policy_version 399328 (0.0007) [2023-12-27 08:24:14,645][20381] Updated weights for policy 1, policy_version 399338 (0.0007) [2023-12-27 08:24:15,070][20347] Updated weights for policy 0, policy_version 399621 (0.0010) [2023-12-27 08:24:15,128][20347] Updated weights for policy 0, policy_version 399631 (0.0010) [2023-12-27 08:24:15,186][20347] Updated weights for policy 0, policy_version 399641 (0.0010) [2023-12-27 08:24:15,361][20381] Updated weights for policy 1, policy_version 399348 (0.0007) [2023-12-27 08:24:15,409][20381] Updated weights for policy 1, policy_version 399358 (0.0007) [2023-12-27 08:24:15,466][20381] Updated weights for policy 1, policy_version 399368 (0.0007) [2023-12-27 08:24:15,862][20347] Updated weights for policy 0, policy_version 399651 (0.0010) [2023-12-27 08:24:15,912][20347] Updated weights for policy 0, policy_version 399661 (0.0010) [2023-12-27 08:24:15,962][20347] Updated weights for policy 0, policy_version 399671 (0.0010) [2023-12-27 08:24:16,102][20381] Updated weights for policy 1, policy_version 399378 (0.0008) [2023-12-27 08:24:16,152][20381] Updated weights for policy 1, policy_version 399388 (0.0007) [2023-12-27 08:24:16,206][20381] Updated weights for policy 1, policy_version 399398 (0.0005) [2023-12-27 08:24:16,261][20381] Updated weights for policy 1, policy_version 399408 (0.0006) [2023-12-27 08:24:16,608][20347] Updated weights for policy 0, policy_version 399681 (0.0010) [2023-12-27 08:24:16,666][20347] Updated weights for policy 0, policy_version 399691 (0.0010) [2023-12-27 08:24:16,717][20347] Updated weights for policy 0, policy_version 399701 (0.0010) [2023-12-27 08:24:16,778][20347] Updated weights for policy 0, policy_version 399711 (0.0011) [2023-12-27 08:24:16,861][20381] Updated weights for policy 1, policy_version 399418 (0.0005) [2023-12-27 08:24:16,917][20381] Updated weights for policy 1, policy_version 399428 (0.0005) [2023-12-27 08:24:16,965][20381] Updated weights for policy 1, policy_version 399438 (0.0006) [2023-12-27 08:24:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 204619776. Throughput: 0: 10451.4, 1: 10401.7. Samples: 204633752. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:17,363][19170] Avg episode reward: [(0, '701.150'), (1, '942.230')] [2023-12-27 08:24:17,465][20347] Updated weights for policy 0, policy_version 399721 (0.0010) [2023-12-27 08:24:17,526][20347] Updated weights for policy 0, policy_version 399731 (0.0010) [2023-12-27 08:24:17,579][20381] Updated weights for policy 1, policy_version 399448 (0.0009) [2023-12-27 08:24:17,589][20347] Updated weights for policy 0, policy_version 399741 (0.0010) [2023-12-27 08:24:17,623][20381] Updated weights for policy 1, policy_version 399458 (0.0010) [2023-12-27 08:24:17,675][20381] Updated weights for policy 1, policy_version 399468 (0.0010) [2023-12-27 08:24:18,141][20347] Updated weights for policy 0, policy_version 399751 (0.0007) [2023-12-27 08:24:18,196][20347] Updated weights for policy 0, policy_version 399761 (0.0006) [2023-12-27 08:24:18,249][20347] Updated weights for policy 0, policy_version 399771 (0.0005) [2023-12-27 08:24:18,327][20381] Updated weights for policy 1, policy_version 399478 (0.0009) [2023-12-27 08:24:18,391][20381] Updated weights for policy 1, policy_version 399488 (0.0008) [2023-12-27 08:24:18,448][20381] Updated weights for policy 1, policy_version 399498 (0.0008) [2023-12-27 08:24:18,841][20347] Updated weights for policy 0, policy_version 399781 (0.0008) [2023-12-27 08:24:18,887][20347] Updated weights for policy 0, policy_version 399791 (0.0010) [2023-12-27 08:24:18,937][20347] Updated weights for policy 0, policy_version 399801 (0.0010) [2023-12-27 08:24:19,115][20381] Updated weights for policy 1, policy_version 399508 (0.0008) [2023-12-27 08:24:19,170][20381] Updated weights for policy 1, policy_version 399518 (0.0010) [2023-12-27 08:24:19,225][20381] Updated weights for policy 1, policy_version 399528 (0.0008) [2023-12-27 08:24:19,596][20347] Updated weights for policy 0, policy_version 399811 (0.0007) [2023-12-27 08:24:19,658][20347] Updated weights for policy 0, policy_version 399821 (0.0006) [2023-12-27 08:24:19,715][20347] Updated weights for policy 0, policy_version 399831 (0.0005) [2023-12-27 08:24:19,895][20381] Updated weights for policy 1, policy_version 399538 (0.0007) [2023-12-27 08:24:19,955][20381] Updated weights for policy 1, policy_version 399548 (0.0007) [2023-12-27 08:24:20,014][20381] Updated weights for policy 1, policy_version 399558 (0.0008) [2023-12-27 08:24:20,075][20381] Updated weights for policy 1, policy_version 399568 (0.0006) [2023-12-27 08:24:20,372][20347] Updated weights for policy 0, policy_version 399841 (0.0010) [2023-12-27 08:24:20,432][20347] Updated weights for policy 0, policy_version 399851 (0.0010) [2023-12-27 08:24:20,488][20347] Updated weights for policy 0, policy_version 399861 (0.0010) [2023-12-27 08:24:20,538][20347] Updated weights for policy 0, policy_version 399871 (0.0010) [2023-12-27 08:24:20,741][20381] Updated weights for policy 1, policy_version 399578 (0.0007) [2023-12-27 08:24:20,798][20381] Updated weights for policy 1, policy_version 399588 (0.0008) [2023-12-27 08:24:20,853][20381] Updated weights for policy 1, policy_version 399598 (0.0006) [2023-12-27 08:24:21,216][20347] Updated weights for policy 0, policy_version 399881 (0.0008) [2023-12-27 08:24:21,277][20347] Updated weights for policy 0, policy_version 399891 (0.0008) [2023-12-27 08:24:21,344][20347] Updated weights for policy 0, policy_version 399901 (0.0007) [2023-12-27 08:24:21,555][20381] Updated weights for policy 1, policy_version 399608 (0.0010) [2023-12-27 08:24:21,609][20381] Updated weights for policy 1, policy_version 399618 (0.0011) [2023-12-27 08:24:21,669][20381] Updated weights for policy 1, policy_version 399628 (0.0011) [2023-12-27 08:24:22,062][20347] Updated weights for policy 0, policy_version 399911 (0.0008) [2023-12-27 08:24:22,107][20347] Updated weights for policy 0, policy_version 399921 (0.0007) [2023-12-27 08:24:22,160][20347] Updated weights for policy 0, policy_version 399931 (0.0007) [2023-12-27 08:24:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 204726272. Throughput: 0: 10464.5, 1: 10375.5. Samples: 204698084. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:22,363][19170] Avg episode reward: [(0, '786.120'), (1, '941.970')] [2023-12-27 08:24:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000399936_102400000.pth... [2023-12-27 08:24:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000398720_102088704.pth [2023-12-27 08:24:22,389][20381] Updated weights for policy 1, policy_version 399638 (0.0009) [2023-12-27 08:24:22,455][20381] Updated weights for policy 1, policy_version 399648 (0.0011) [2023-12-27 08:24:22,522][20381] Updated weights for policy 1, policy_version 399658 (0.0007) [2023-12-27 08:24:22,563][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000399664_102334464.pth... [2023-12-27 08:24:22,568][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000398448_102023168.pth [2023-12-27 08:24:22,838][20347] Updated weights for policy 0, policy_version 399941 (0.0008) [2023-12-27 08:24:22,902][20347] Updated weights for policy 0, policy_version 399951 (0.0007) [2023-12-27 08:24:22,968][20347] Updated weights for policy 0, policy_version 399961 (0.0007) [2023-12-27 08:24:23,154][20381] Updated weights for policy 1, policy_version 399668 (0.0006) [2023-12-27 08:24:23,205][20381] Updated weights for policy 1, policy_version 399678 (0.0009) [2023-12-27 08:24:23,253][20381] Updated weights for policy 1, policy_version 399688 (0.0006) [2023-12-27 08:24:23,595][20347] Updated weights for policy 0, policy_version 399971 (0.0006) [2023-12-27 08:24:23,661][20347] Updated weights for policy 0, policy_version 399981 (0.0005) [2023-12-27 08:24:23,714][20347] Updated weights for policy 0, policy_version 399991 (0.0005) [2023-12-27 08:24:23,929][20381] Updated weights for policy 1, policy_version 399698 (0.0006) [2023-12-27 08:24:23,992][20381] Updated weights for policy 1, policy_version 399708 (0.0011) [2023-12-27 08:24:24,050][20381] Updated weights for policy 1, policy_version 399718 (0.0011) [2023-12-27 08:24:24,110][20381] Updated weights for policy 1, policy_version 399728 (0.0011) [2023-12-27 08:24:24,242][20347] Updated weights for policy 0, policy_version 400001 (0.0005) [2023-12-27 08:24:24,290][20347] Updated weights for policy 0, policy_version 400011 (0.0007) [2023-12-27 08:24:24,348][20347] Updated weights for policy 0, policy_version 400021 (0.0006) [2023-12-27 08:24:24,409][20347] Updated weights for policy 0, policy_version 400031 (0.0006) [2023-12-27 08:24:24,754][20381] Updated weights for policy 1, policy_version 399738 (0.0008) [2023-12-27 08:24:24,809][20381] Updated weights for policy 1, policy_version 399748 (0.0007) [2023-12-27 08:24:24,860][20381] Updated weights for policy 1, policy_version 399758 (0.0007) [2023-12-27 08:24:25,069][20347] Updated weights for policy 0, policy_version 400041 (0.0008) [2023-12-27 08:24:25,124][20347] Updated weights for policy 0, policy_version 400051 (0.0005) [2023-12-27 08:24:25,187][20347] Updated weights for policy 0, policy_version 400061 (0.0008) [2023-12-27 08:24:25,531][20381] Updated weights for policy 1, policy_version 399768 (0.0009) [2023-12-27 08:24:25,594][20381] Updated weights for policy 1, policy_version 399778 (0.0009) [2023-12-27 08:24:25,663][20381] Updated weights for policy 1, policy_version 399788 (0.0009) [2023-12-27 08:24:25,847][20347] Updated weights for policy 0, policy_version 400071 (0.0008) [2023-12-27 08:24:25,905][20347] Updated weights for policy 0, policy_version 400081 (0.0008) [2023-12-27 08:24:25,956][20347] Updated weights for policy 0, policy_version 400091 (0.0008) [2023-12-27 08:24:26,350][20381] Updated weights for policy 1, policy_version 399798 (0.0009) [2023-12-27 08:24:26,386][20292] KL-divergence is very high: 175.1697 [2023-12-27 08:24:26,403][20381] Updated weights for policy 1, policy_version 399808 (0.0010) [2023-12-27 08:24:26,463][20381] Updated weights for policy 1, policy_version 399818 (0.0010) [2023-12-27 08:24:26,694][20347] Updated weights for policy 0, policy_version 400101 (0.0007) [2023-12-27 08:24:26,753][20347] Updated weights for policy 0, policy_version 400111 (0.0008) [2023-12-27 08:24:26,816][20347] Updated weights for policy 0, policy_version 400121 (0.0006) [2023-12-27 08:24:27,093][20381] Updated weights for policy 1, policy_version 399828 (0.0011) [2023-12-27 08:24:27,111][20292] KL-divergence is very high: 117.2127 [2023-12-27 08:24:27,129][20292] KL-divergence is very high: 184.0796 [2023-12-27 08:24:27,134][20292] KL-divergence is very high: 188.0757 [2023-12-27 08:24:27,144][20292] KL-divergence is very high: 128.5641 [2023-12-27 08:24:27,150][20292] KL-divergence is very high: 123.8677 [2023-12-27 08:24:27,157][20381] Updated weights for policy 1, policy_version 399838 (0.0011) [2023-12-27 08:24:27,175][20292] KL-divergence is very high: 111.9334 [2023-12-27 08:24:27,209][20381] Updated weights for policy 1, policy_version 399848 (0.0011) [2023-12-27 08:24:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 204832768. Throughput: 0: 10514.1, 1: 10306.4. Samples: 204821352. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:27,363][19170] Avg episode reward: [(0, '845.360'), (1, '915.800')] [2023-12-27 08:24:27,384][20347] Updated weights for policy 0, policy_version 400131 (0.0006) [2023-12-27 08:24:27,439][20347] Updated weights for policy 0, policy_version 400141 (0.0005) [2023-12-27 08:24:27,497][20347] Updated weights for policy 0, policy_version 400151 (0.0005) [2023-12-27 08:24:27,866][20381] Updated weights for policy 1, policy_version 399858 (0.0009) [2023-12-27 08:24:27,926][20381] Updated weights for policy 1, policy_version 399868 (0.0009) [2023-12-27 08:24:27,980][20381] Updated weights for policy 1, policy_version 399878 (0.0010) [2023-12-27 08:24:28,034][20381] Updated weights for policy 1, policy_version 399888 (0.0010) [2023-12-27 08:24:28,135][20347] Updated weights for policy 0, policy_version 400161 (0.0006) [2023-12-27 08:24:28,190][20347] Updated weights for policy 0, policy_version 400171 (0.0007) [2023-12-27 08:24:28,243][20347] Updated weights for policy 0, policy_version 400181 (0.0007) [2023-12-27 08:24:28,286][20347] Updated weights for policy 0, policy_version 400191 (0.0007) [2023-12-27 08:24:28,717][20381] Updated weights for policy 1, policy_version 399898 (0.0011) [2023-12-27 08:24:28,781][20381] Updated weights for policy 1, policy_version 399908 (0.0011) [2023-12-27 08:24:28,846][20381] Updated weights for policy 1, policy_version 399918 (0.0011) [2023-12-27 08:24:28,976][20347] Updated weights for policy 0, policy_version 400201 (0.0006) [2023-12-27 08:24:29,026][20347] Updated weights for policy 0, policy_version 400211 (0.0005) [2023-12-27 08:24:29,073][20347] Updated weights for policy 0, policy_version 400221 (0.0007) [2023-12-27 08:24:29,473][20381] Updated weights for policy 1, policy_version 399928 (0.0011) [2023-12-27 08:24:29,527][20381] Updated weights for policy 1, policy_version 399938 (0.0010) [2023-12-27 08:24:29,586][20381] Updated weights for policy 1, policy_version 399948 (0.0010) [2023-12-27 08:24:29,709][20347] Updated weights for policy 0, policy_version 400231 (0.0007) [2023-12-27 08:24:29,774][20347] Updated weights for policy 0, policy_version 400241 (0.0008) [2023-12-27 08:24:29,838][20347] Updated weights for policy 0, policy_version 400251 (0.0008) [2023-12-27 08:24:30,242][20381] Updated weights for policy 1, policy_version 399958 (0.0010) [2023-12-27 08:24:30,291][20381] Updated weights for policy 1, policy_version 399968 (0.0010) [2023-12-27 08:24:30,346][20381] Updated weights for policy 1, policy_version 399978 (0.0010) [2023-12-27 08:24:30,393][20347] Updated weights for policy 0, policy_version 400261 (0.0009) [2023-12-27 08:24:30,437][20347] Updated weights for policy 0, policy_version 400271 (0.0008) [2023-12-27 08:24:30,484][20347] Updated weights for policy 0, policy_version 400281 (0.0005) [2023-12-27 08:24:31,041][20347] Updated weights for policy 0, policy_version 400291 (0.0007) [2023-12-27 08:24:31,043][20381] Updated weights for policy 1, policy_version 399988 (0.0011) [2023-12-27 08:24:31,100][20381] Updated weights for policy 1, policy_version 399998 (0.0006) [2023-12-27 08:24:31,101][20347] Updated weights for policy 0, policy_version 400301 (0.0007) [2023-12-27 08:24:31,162][20347] Updated weights for policy 0, policy_version 400311 (0.0009) [2023-12-27 08:24:31,163][20381] Updated weights for policy 1, policy_version 400008 (0.0007) [2023-12-27 08:24:31,792][20381] Updated weights for policy 1, policy_version 400018 (0.0008) [2023-12-27 08:24:31,845][20347] Updated weights for policy 0, policy_version 400321 (0.0006) [2023-12-27 08:24:31,847][20381] Updated weights for policy 1, policy_version 400028 (0.0010) [2023-12-27 08:24:31,907][20347] Updated weights for policy 0, policy_version 400331 (0.0007) [2023-12-27 08:24:31,908][20381] Updated weights for policy 1, policy_version 400038 (0.0006) [2023-12-27 08:24:31,958][20347] Updated weights for policy 0, policy_version 400341 (0.0005) [2023-12-27 08:24:31,966][20381] Updated weights for policy 1, policy_version 400048 (0.0010) [2023-12-27 08:24:32,008][20347] Updated weights for policy 0, policy_version 400351 (0.0007) [2023-12-27 08:24:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 204939264. Throughput: 0: 10568.7, 1: 10338.2. Samples: 204949176. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:32,363][19170] Avg episode reward: [(0, '748.430'), (1, '687.210')] [2023-12-27 08:24:32,645][20347] Updated weights for policy 0, policy_version 400361 (0.0009) [2023-12-27 08:24:32,650][20381] Updated weights for policy 1, policy_version 400058 (0.0007) [2023-12-27 08:24:32,707][20347] Updated weights for policy 0, policy_version 400371 (0.0008) [2023-12-27 08:24:32,710][20381] Updated weights for policy 1, policy_version 400068 (0.0007) [2023-12-27 08:24:32,768][20381] Updated weights for policy 1, policy_version 400078 (0.0007) [2023-12-27 08:24:32,770][20347] Updated weights for policy 0, policy_version 400381 (0.0007) [2023-12-27 08:24:33,349][20347] Updated weights for policy 0, policy_version 400391 (0.0008) [2023-12-27 08:24:33,392][20347] Updated weights for policy 0, policy_version 400401 (0.0007) [2023-12-27 08:24:33,435][20347] Updated weights for policy 0, policy_version 400411 (0.0007) [2023-12-27 08:24:33,485][20381] Updated weights for policy 1, policy_version 400088 (0.0007) [2023-12-27 08:24:33,530][20381] Updated weights for policy 1, policy_version 400098 (0.0007) [2023-12-27 08:24:33,578][20381] Updated weights for policy 1, policy_version 400108 (0.0007) [2023-12-27 08:24:34,101][20347] Updated weights for policy 0, policy_version 400421 (0.0006) [2023-12-27 08:24:34,162][20347] Updated weights for policy 0, policy_version 400431 (0.0007) [2023-12-27 08:24:34,224][20347] Updated weights for policy 0, policy_version 400441 (0.0007) [2023-12-27 08:24:34,305][20381] Updated weights for policy 1, policy_version 400118 (0.0006) [2023-12-27 08:24:34,356][20381] Updated weights for policy 1, policy_version 400128 (0.0007) [2023-12-27 08:24:34,402][20381] Updated weights for policy 1, policy_version 400138 (0.0007) [2023-12-27 08:24:34,918][20347] Updated weights for policy 0, policy_version 400451 (0.0006) [2023-12-27 08:24:34,976][20347] Updated weights for policy 0, policy_version 400461 (0.0007) [2023-12-27 08:24:35,029][20347] Updated weights for policy 0, policy_version 400471 (0.0007) [2023-12-27 08:24:35,083][20381] Updated weights for policy 1, policy_version 400148 (0.0008) [2023-12-27 08:24:35,149][20381] Updated weights for policy 1, policy_version 400158 (0.0007) [2023-12-27 08:24:35,213][20381] Updated weights for policy 1, policy_version 400168 (0.0008) [2023-12-27 08:24:35,706][20347] Updated weights for policy 0, policy_version 400481 (0.0007) [2023-12-27 08:24:35,764][20347] Updated weights for policy 0, policy_version 400491 (0.0007) [2023-12-27 08:24:35,824][20347] Updated weights for policy 0, policy_version 400501 (0.0007) [2023-12-27 08:24:35,844][20381] Updated weights for policy 1, policy_version 400178 (0.0008) [2023-12-27 08:24:35,878][20347] Updated weights for policy 0, policy_version 400511 (0.0006) [2023-12-27 08:24:35,895][20381] Updated weights for policy 1, policy_version 400188 (0.0009) [2023-12-27 08:24:35,954][20381] Updated weights for policy 1, policy_version 400198 (0.0009) [2023-12-27 08:24:36,007][20381] Updated weights for policy 1, policy_version 400208 (0.0009) [2023-12-27 08:24:36,620][20291] KL-divergence is very high: 199.2204 [2023-12-27 08:24:36,628][20347] Updated weights for policy 0, policy_version 400521 (0.0008) [2023-12-27 08:24:36,634][20291] KL-divergence is very high: 173.7065 [2023-12-27 08:24:36,686][20347] Updated weights for policy 0, policy_version 400531 (0.0008) [2023-12-27 08:24:36,724][20381] Updated weights for policy 1, policy_version 400218 (0.0011) [2023-12-27 08:24:36,744][20347] Updated weights for policy 0, policy_version 400541 (0.0006) [2023-12-27 08:24:36,786][20381] Updated weights for policy 1, policy_version 400228 (0.0010) [2023-12-27 08:24:36,841][20381] Updated weights for policy 1, policy_version 400238 (0.0010) [2023-12-27 08:24:37,363][19170] Fps is (10 sec: 20479.0, 60 sec: 20752.9, 300 sec: 20743.8). Total num frames: 205037568. Throughput: 0: 10534.4, 1: 10292.9. Samples: 205010940. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:37,364][19170] Avg episode reward: [(0, '816.530'), (1, '806.930')] [2023-12-27 08:24:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000400240_102481920.pth... [2023-12-27 08:24:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000400544_102555648.pth... [2023-12-27 08:24:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000399024_102170624.pth [2023-12-27 08:24:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000399328_102244352.pth [2023-12-27 08:24:37,423][20347] Updated weights for policy 0, policy_version 400551 (0.0007) [2023-12-27 08:24:37,470][20347] Updated weights for policy 0, policy_version 400561 (0.0007) [2023-12-27 08:24:37,525][20347] Updated weights for policy 0, policy_version 400571 (0.0009) [2023-12-27 08:24:37,535][20381] Updated weights for policy 1, policy_version 400248 (0.0011) [2023-12-27 08:24:37,583][20381] Updated weights for policy 1, policy_version 400258 (0.0009) [2023-12-27 08:24:37,632][20381] Updated weights for policy 1, policy_version 400268 (0.0010) [2023-12-27 08:24:38,174][20347] Updated weights for policy 0, policy_version 400581 (0.0011) [2023-12-27 08:24:38,237][20347] Updated weights for policy 0, policy_version 400591 (0.0011) [2023-12-27 08:24:38,295][20347] Updated weights for policy 0, policy_version 400601 (0.0010) [2023-12-27 08:24:38,341][20381] Updated weights for policy 1, policy_version 400278 (0.0008) [2023-12-27 08:24:38,392][20381] Updated weights for policy 1, policy_version 400288 (0.0005) [2023-12-27 08:24:38,449][20381] Updated weights for policy 1, policy_version 400298 (0.0006) [2023-12-27 08:24:38,906][20347] Updated weights for policy 0, policy_version 400611 (0.0007) [2023-12-27 08:24:38,955][20347] Updated weights for policy 0, policy_version 400621 (0.0005) [2023-12-27 08:24:39,001][20347] Updated weights for policy 0, policy_version 400631 (0.0005) [2023-12-27 08:24:39,009][20381] Updated weights for policy 1, policy_version 400308 (0.0008) [2023-12-27 08:24:39,064][20381] Updated weights for policy 1, policy_version 400318 (0.0006) [2023-12-27 08:24:39,111][20381] Updated weights for policy 1, policy_version 400328 (0.0007) [2023-12-27 08:24:39,636][20347] Updated weights for policy 0, policy_version 400641 (0.0007) [2023-12-27 08:24:39,692][20347] Updated weights for policy 0, policy_version 400651 (0.0005) [2023-12-27 08:24:39,746][20347] Updated weights for policy 0, policy_version 400661 (0.0007) [2023-12-27 08:24:39,796][20347] Updated weights for policy 0, policy_version 400671 (0.0008) [2023-12-27 08:24:39,884][20381] Updated weights for policy 1, policy_version 400338 (0.0007) [2023-12-27 08:24:39,943][20381] Updated weights for policy 1, policy_version 400348 (0.0008) [2023-12-27 08:24:39,998][20381] Updated weights for policy 1, policy_version 400358 (0.0007) [2023-12-27 08:24:40,054][20381] Updated weights for policy 1, policy_version 400368 (0.0008) [2023-12-27 08:24:40,399][20347] Updated weights for policy 0, policy_version 400681 (0.0006) [2023-12-27 08:24:40,463][20347] Updated weights for policy 0, policy_version 400691 (0.0005) [2023-12-27 08:24:40,526][20347] Updated weights for policy 0, policy_version 400701 (0.0005) [2023-12-27 08:24:40,644][20381] Updated weights for policy 1, policy_version 400378 (0.0005) [2023-12-27 08:24:40,704][20381] Updated weights for policy 1, policy_version 400388 (0.0006) [2023-12-27 08:24:40,757][20381] Updated weights for policy 1, policy_version 400398 (0.0008) [2023-12-27 08:24:41,140][20347] Updated weights for policy 0, policy_version 400711 (0.0007) [2023-12-27 08:24:41,197][20347] Updated weights for policy 0, policy_version 400721 (0.0008) [2023-12-27 08:24:41,250][20347] Updated weights for policy 0, policy_version 400731 (0.0007) [2023-12-27 08:24:41,407][20381] Updated weights for policy 1, policy_version 400408 (0.0008) [2023-12-27 08:24:41,466][20381] Updated weights for policy 1, policy_version 400418 (0.0007) [2023-12-27 08:24:41,524][20381] Updated weights for policy 1, policy_version 400428 (0.0007) [2023-12-27 08:24:41,991][20347] Updated weights for policy 0, policy_version 400741 (0.0008) [2023-12-27 08:24:42,051][20347] Updated weights for policy 0, policy_version 400751 (0.0008) [2023-12-27 08:24:42,116][20347] Updated weights for policy 0, policy_version 400761 (0.0007) [2023-12-27 08:24:42,182][20381] Updated weights for policy 1, policy_version 400438 (0.0009) [2023-12-27 08:24:42,234][20381] Updated weights for policy 1, policy_version 400448 (0.0010) [2023-12-27 08:24:42,298][20381] Updated weights for policy 1, policy_version 400458 (0.0011) [2023-12-27 08:24:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 205152256. Throughput: 0: 10593.9, 1: 10324.1. Samples: 205137280. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:42,363][19170] Avg episode reward: [(0, '806.720'), (1, '904.520')] [2023-12-27 08:24:42,720][20347] Updated weights for policy 0, policy_version 400771 (0.0007) [2023-12-27 08:24:42,771][20347] Updated weights for policy 0, policy_version 400781 (0.0005) [2023-12-27 08:24:42,820][20347] Updated weights for policy 0, policy_version 400791 (0.0005) [2023-12-27 08:24:42,911][20381] Updated weights for policy 1, policy_version 400468 (0.0008) [2023-12-27 08:24:42,961][20381] Updated weights for policy 1, policy_version 400478 (0.0005) [2023-12-27 08:24:43,010][20381] Updated weights for policy 1, policy_version 400488 (0.0006) [2023-12-27 08:24:43,378][20347] Updated weights for policy 0, policy_version 400801 (0.0005) [2023-12-27 08:24:43,424][20291] KL-divergence is very high: 124.3756 [2023-12-27 08:24:43,436][20347] Updated weights for policy 0, policy_version 400811 (0.0005) [2023-12-27 08:24:43,472][20291] KL-divergence is very high: 144.7944 [2023-12-27 08:24:43,494][20347] Updated weights for policy 0, policy_version 400821 (0.0006) [2023-12-27 08:24:43,547][20347] Updated weights for policy 0, policy_version 400831 (0.0008) [2023-12-27 08:24:43,559][20381] Updated weights for policy 1, policy_version 400498 (0.0005) [2023-12-27 08:24:43,619][20381] Updated weights for policy 1, policy_version 400508 (0.0007) [2023-12-27 08:24:43,682][20381] Updated weights for policy 1, policy_version 400518 (0.0008) [2023-12-27 08:24:43,746][20381] Updated weights for policy 1, policy_version 400528 (0.0008) [2023-12-27 08:24:44,184][20347] Updated weights for policy 0, policy_version 400841 (0.0010) [2023-12-27 08:24:44,243][20347] Updated weights for policy 0, policy_version 400851 (0.0006) [2023-12-27 08:24:44,294][20347] Updated weights for policy 0, policy_version 400861 (0.0005) [2023-12-27 08:24:44,373][20381] Updated weights for policy 1, policy_version 400538 (0.0007) [2023-12-27 08:24:44,425][20381] Updated weights for policy 1, policy_version 400548 (0.0005) [2023-12-27 08:24:44,476][20381] Updated weights for policy 1, policy_version 400558 (0.0006) [2023-12-27 08:24:44,851][20347] Updated weights for policy 0, policy_version 400871 (0.0005) [2023-12-27 08:24:44,907][20347] Updated weights for policy 0, policy_version 400881 (0.0006) [2023-12-27 08:24:44,960][20347] Updated weights for policy 0, policy_version 400891 (0.0008) [2023-12-27 08:24:45,174][20381] Updated weights for policy 1, policy_version 400568 (0.0007) [2023-12-27 08:24:45,231][20381] Updated weights for policy 1, policy_version 400578 (0.0005) [2023-12-27 08:24:45,287][20381] Updated weights for policy 1, policy_version 400588 (0.0008) [2023-12-27 08:24:45,650][20347] Updated weights for policy 0, policy_version 400901 (0.0008) [2023-12-27 08:24:45,706][20347] Updated weights for policy 0, policy_version 400911 (0.0008) [2023-12-27 08:24:45,761][20347] Updated weights for policy 0, policy_version 400921 (0.0007) [2023-12-27 08:24:45,875][20381] Updated weights for policy 1, policy_version 400598 (0.0011) [2023-12-27 08:24:45,926][20381] Updated weights for policy 1, policy_version 400608 (0.0009) [2023-12-27 08:24:45,982][20381] Updated weights for policy 1, policy_version 400618 (0.0005) [2023-12-27 08:24:46,448][20347] Updated weights for policy 0, policy_version 400931 (0.0008) [2023-12-27 08:24:46,494][20347] Updated weights for policy 0, policy_version 400941 (0.0008) [2023-12-27 08:24:46,538][20347] Updated weights for policy 0, policy_version 400951 (0.0007) [2023-12-27 08:24:46,637][20381] Updated weights for policy 1, policy_version 400628 (0.0007) [2023-12-27 08:24:46,695][20381] Updated weights for policy 1, policy_version 400638 (0.0011) [2023-12-27 08:24:46,760][20381] Updated weights for policy 1, policy_version 400648 (0.0011) [2023-12-27 08:24:47,224][20347] Updated weights for policy 0, policy_version 400961 (0.0008) [2023-12-27 08:24:47,284][20347] Updated weights for policy 0, policy_version 400971 (0.0010) [2023-12-27 08:24:47,347][20347] Updated weights for policy 0, policy_version 400981 (0.0010) [2023-12-27 08:24:47,362][19170] Fps is (10 sec: 21300.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 205250560. Throughput: 0: 10605.3, 1: 10414.5. Samples: 205266700. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:47,363][19170] Avg episode reward: [(0, '662.580'), (1, '980.760')] [2023-12-27 08:24:47,401][20381] Updated weights for policy 1, policy_version 400658 (0.0009) [2023-12-27 08:24:47,407][20347] Updated weights for policy 0, policy_version 400991 (0.0010) [2023-12-27 08:24:47,447][20381] Updated weights for policy 1, policy_version 400668 (0.0010) [2023-12-27 08:24:47,495][20381] Updated weights for policy 1, policy_version 400678 (0.0010) [2023-12-27 08:24:47,543][20381] Updated weights for policy 1, policy_version 400688 (0.0010) [2023-12-27 08:24:48,066][20347] Updated weights for policy 0, policy_version 401001 (0.0010) [2023-12-27 08:24:48,125][20347] Updated weights for policy 0, policy_version 401011 (0.0011) [2023-12-27 08:24:48,183][20347] Updated weights for policy 0, policy_version 401021 (0.0011) [2023-12-27 08:24:48,203][20381] Updated weights for policy 1, policy_version 400698 (0.0011) [2023-12-27 08:24:48,262][20381] Updated weights for policy 1, policy_version 400708 (0.0007) [2023-12-27 08:24:48,317][20381] Updated weights for policy 1, policy_version 400718 (0.0008) [2023-12-27 08:24:48,872][20347] Updated weights for policy 0, policy_version 401031 (0.0010) [2023-12-27 08:24:48,932][20347] Updated weights for policy 0, policy_version 401041 (0.0011) [2023-12-27 08:24:48,985][20347] Updated weights for policy 0, policy_version 401051 (0.0011) [2023-12-27 08:24:48,987][20381] Updated weights for policy 1, policy_version 400728 (0.0006) [2023-12-27 08:24:49,033][20381] Updated weights for policy 1, policy_version 400738 (0.0005) [2023-12-27 08:24:49,079][20381] Updated weights for policy 1, policy_version 400748 (0.0005) [2023-12-27 08:24:49,614][20347] Updated weights for policy 0, policy_version 401061 (0.0010) [2023-12-27 08:24:49,672][20347] Updated weights for policy 0, policy_version 401071 (0.0010) [2023-12-27 08:24:49,732][20347] Updated weights for policy 0, policy_version 401081 (0.0006) [2023-12-27 08:24:49,808][20381] Updated weights for policy 1, policy_version 400758 (0.0007) [2023-12-27 08:24:49,872][20381] Updated weights for policy 1, policy_version 400768 (0.0009) [2023-12-27 08:24:49,924][20381] Updated weights for policy 1, policy_version 400778 (0.0009) [2023-12-27 08:24:50,324][20347] Updated weights for policy 0, policy_version 401091 (0.0008) [2023-12-27 08:24:50,385][20347] Updated weights for policy 0, policy_version 401101 (0.0010) [2023-12-27 08:24:50,441][20347] Updated weights for policy 0, policy_version 401111 (0.0008) [2023-12-27 08:24:50,627][20381] Updated weights for policy 1, policy_version 400788 (0.0009) [2023-12-27 08:24:50,682][20381] Updated weights for policy 1, policy_version 400798 (0.0007) [2023-12-27 08:24:50,733][20381] Updated weights for policy 1, policy_version 400808 (0.0007) [2023-12-27 08:24:51,133][20347] Updated weights for policy 0, policy_version 401121 (0.0007) [2023-12-27 08:24:51,186][20347] Updated weights for policy 0, policy_version 401131 (0.0010) [2023-12-27 08:24:51,251][20347] Updated weights for policy 0, policy_version 401141 (0.0010) [2023-12-27 08:24:51,320][20347] Updated weights for policy 0, policy_version 401151 (0.0010) [2023-12-27 08:24:51,479][20381] Updated weights for policy 1, policy_version 400818 (0.0008) [2023-12-27 08:24:51,531][20381] Updated weights for policy 1, policy_version 400828 (0.0008) [2023-12-27 08:24:51,583][20381] Updated weights for policy 1, policy_version 400838 (0.0008) [2023-12-27 08:24:51,642][20381] Updated weights for policy 1, policy_version 400848 (0.0008) [2023-12-27 08:24:52,015][20347] Updated weights for policy 0, policy_version 401161 (0.0011) [2023-12-27 08:24:52,070][20347] Updated weights for policy 0, policy_version 401171 (0.0010) [2023-12-27 08:24:52,127][20347] Updated weights for policy 0, policy_version 401181 (0.0010) [2023-12-27 08:24:52,323][20381] Updated weights for policy 1, policy_version 400858 (0.0005) [2023-12-27 08:24:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 205357056. Throughput: 0: 10572.7, 1: 10421.5. Samples: 205328332. Policy #0 lag: (min: 2.0, avg: 14.5, max: 34.0) [2023-12-27 08:24:52,363][19170] Avg episode reward: [(0, '678.500'), (1, '1000.000')] [2023-12-27 08:24:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000401184_102719488.pth... [2023-12-27 08:24:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000399936_102400000.pth [2023-12-27 08:24:52,388][20381] Updated weights for policy 1, policy_version 400868 (0.0008) [2023-12-27 08:24:52,440][20381] Updated weights for policy 1, policy_version 400878 (0.0007) [2023-12-27 08:24:52,449][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000400880_102645760.pth... [2023-12-27 08:24:52,452][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000399664_102334464.pth [2023-12-27 08:24:52,841][20347] Updated weights for policy 0, policy_version 401191 (0.0009) [2023-12-27 08:24:52,911][20347] Updated weights for policy 0, policy_version 401201 (0.0009) [2023-12-27 08:24:52,962][20347] Updated weights for policy 0, policy_version 401211 (0.0010) [2023-12-27 08:24:53,025][20381] Updated weights for policy 1, policy_version 400888 (0.0005) [2023-12-27 08:24:53,085][20381] Updated weights for policy 1, policy_version 400898 (0.0010) [2023-12-27 08:24:53,140][20381] Updated weights for policy 1, policy_version 400908 (0.0010) [2023-12-27 08:24:53,632][20347] Updated weights for policy 0, policy_version 401221 (0.0010) [2023-12-27 08:24:53,690][20347] Updated weights for policy 0, policy_version 401231 (0.0010) [2023-12-27 08:24:53,753][20347] Updated weights for policy 0, policy_version 401241 (0.0010) [2023-12-27 08:24:53,802][20381] Updated weights for policy 1, policy_version 400918 (0.0011) [2023-12-27 08:24:53,863][20381] Updated weights for policy 1, policy_version 400928 (0.0010) [2023-12-27 08:24:53,914][20381] Updated weights for policy 1, policy_version 400938 (0.0008) [2023-12-27 08:24:54,399][20347] Updated weights for policy 0, policy_version 401251 (0.0008) [2023-12-27 08:24:54,462][20347] Updated weights for policy 0, policy_version 401261 (0.0005) [2023-12-27 08:24:54,521][20347] Updated weights for policy 0, policy_version 401271 (0.0007) [2023-12-27 08:24:54,577][20381] Updated weights for policy 1, policy_version 400948 (0.0007) [2023-12-27 08:24:54,631][20381] Updated weights for policy 1, policy_version 400958 (0.0008) [2023-12-27 08:24:54,689][20381] Updated weights for policy 1, policy_version 400968 (0.0005) [2023-12-27 08:24:55,183][20347] Updated weights for policy 0, policy_version 401281 (0.0007) [2023-12-27 08:24:55,234][20347] Updated weights for policy 0, policy_version 401291 (0.0007) [2023-12-27 08:24:55,251][20381] Updated weights for policy 1, policy_version 400978 (0.0005) [2023-12-27 08:24:55,282][20347] Updated weights for policy 0, policy_version 401301 (0.0006) [2023-12-27 08:24:55,302][20381] Updated weights for policy 1, policy_version 400988 (0.0006) [2023-12-27 08:24:55,330][20347] Updated weights for policy 0, policy_version 401311 (0.0005) [2023-12-27 08:24:55,358][20381] Updated weights for policy 1, policy_version 400998 (0.0007) [2023-12-27 08:24:55,414][20381] Updated weights for policy 1, policy_version 401008 (0.0008) [2023-12-27 08:24:56,039][20347] Updated weights for policy 0, policy_version 401321 (0.0008) [2023-12-27 08:24:56,096][20347] Updated weights for policy 0, policy_version 401331 (0.0007) [2023-12-27 08:24:56,128][20381] Updated weights for policy 1, policy_version 401018 (0.0008) [2023-12-27 08:24:56,156][20347] Updated weights for policy 0, policy_version 401341 (0.0006) [2023-12-27 08:24:56,178][20381] Updated weights for policy 1, policy_version 401028 (0.0006) [2023-12-27 08:24:56,232][20381] Updated weights for policy 1, policy_version 401038 (0.0007) [2023-12-27 08:24:56,823][20347] Updated weights for policy 0, policy_version 401351 (0.0007) [2023-12-27 08:24:56,876][20347] Updated weights for policy 0, policy_version 401361 (0.0007) [2023-12-27 08:24:56,927][20381] Updated weights for policy 1, policy_version 401048 (0.0006) [2023-12-27 08:24:56,931][20347] Updated weights for policy 0, policy_version 401371 (0.0007) [2023-12-27 08:24:56,979][20381] Updated weights for policy 1, policy_version 401058 (0.0006) [2023-12-27 08:24:57,038][20381] Updated weights for policy 1, policy_version 401068 (0.0007) [2023-12-27 08:24:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 205463552. Throughput: 0: 10560.6, 1: 10442.8. Samples: 205452800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:24:57,363][19170] Avg episode reward: [(0, '651.330'), (1, '971.040')] [2023-12-27 08:24:57,594][20347] Updated weights for policy 0, policy_version 401381 (0.0007) [2023-12-27 08:24:57,652][20347] Updated weights for policy 0, policy_version 401391 (0.0008) [2023-12-27 08:24:57,706][20347] Updated weights for policy 0, policy_version 401401 (0.0007) [2023-12-27 08:24:57,719][20381] Updated weights for policy 1, policy_version 401078 (0.0009) [2023-12-27 08:24:57,777][20381] Updated weights for policy 1, policy_version 401088 (0.0007) [2023-12-27 08:24:57,822][20381] Updated weights for policy 1, policy_version 401098 (0.0008) [2023-12-27 08:24:58,445][20347] Updated weights for policy 0, policy_version 401411 (0.0006) [2023-12-27 08:24:58,503][20347] Updated weights for policy 0, policy_version 401421 (0.0009) [2023-12-27 08:24:58,527][20381] Updated weights for policy 1, policy_version 401108 (0.0008) [2023-12-27 08:24:58,569][20347] Updated weights for policy 0, policy_version 401431 (0.0009) [2023-12-27 08:24:58,593][20381] Updated weights for policy 1, policy_version 401118 (0.0008) [2023-12-27 08:24:58,655][20381] Updated weights for policy 1, policy_version 401128 (0.0008) [2023-12-27 08:24:59,289][20347] Updated weights for policy 0, policy_version 401441 (0.0006) [2023-12-27 08:24:59,354][20347] Updated weights for policy 0, policy_version 401451 (0.0009) [2023-12-27 08:24:59,402][20381] Updated weights for policy 1, policy_version 401138 (0.0008) [2023-12-27 08:24:59,411][20347] Updated weights for policy 0, policy_version 401461 (0.0010) [2023-12-27 08:24:59,450][20381] Updated weights for policy 1, policy_version 401148 (0.0007) [2023-12-27 08:24:59,476][20347] Updated weights for policy 0, policy_version 401471 (0.0010) [2023-12-27 08:24:59,510][20381] Updated weights for policy 1, policy_version 401158 (0.0007) [2023-12-27 08:24:59,565][20381] Updated weights for policy 1, policy_version 401168 (0.0007) [2023-12-27 08:25:00,103][20347] Updated weights for policy 0, policy_version 401481 (0.0010) [2023-12-27 08:25:00,162][20347] Updated weights for policy 0, policy_version 401491 (0.0009) [2023-12-27 08:25:00,221][20347] Updated weights for policy 0, policy_version 401501 (0.0008) [2023-12-27 08:25:00,238][20381] Updated weights for policy 1, policy_version 401178 (0.0007) [2023-12-27 08:25:00,293][20381] Updated weights for policy 1, policy_version 401188 (0.0007) [2023-12-27 08:25:00,351][20381] Updated weights for policy 1, policy_version 401198 (0.0008) [2023-12-27 08:25:00,860][20347] Updated weights for policy 0, policy_version 401511 (0.0008) [2023-12-27 08:25:00,914][20347] Updated weights for policy 0, policy_version 401521 (0.0005) [2023-12-27 08:25:00,960][20381] Updated weights for policy 1, policy_version 401208 (0.0008) [2023-12-27 08:25:00,972][20347] Updated weights for policy 0, policy_version 401531 (0.0006) [2023-12-27 08:25:01,022][20381] Updated weights for policy 1, policy_version 401218 (0.0007) [2023-12-27 08:25:01,080][20381] Updated weights for policy 1, policy_version 401228 (0.0008) [2023-12-27 08:25:01,572][20347] Updated weights for policy 0, policy_version 401541 (0.0008) [2023-12-27 08:25:01,635][20347] Updated weights for policy 0, policy_version 401551 (0.0007) [2023-12-27 08:25:01,689][20347] Updated weights for policy 0, policy_version 401561 (0.0006) [2023-12-27 08:25:01,823][20381] Updated weights for policy 1, policy_version 401238 (0.0008) [2023-12-27 08:25:01,879][20381] Updated weights for policy 1, policy_version 401248 (0.0007) [2023-12-27 08:25:01,931][20381] Updated weights for policy 1, policy_version 401258 (0.0007) [2023-12-27 08:25:02,241][20347] Updated weights for policy 0, policy_version 401571 (0.0007) [2023-12-27 08:25:02,302][20347] Updated weights for policy 0, policy_version 401581 (0.0005) [2023-12-27 08:25:02,362][20347] Updated weights for policy 0, policy_version 401591 (0.0007) [2023-12-27 08:25:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 205561856. Throughput: 0: 10567.8, 1: 10379.4. Samples: 205576380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:02,363][19170] Avg episode reward: [(0, '467.790'), (1, '942.470')] [2023-12-27 08:25:02,563][20381] Updated weights for policy 1, policy_version 401268 (0.0006) [2023-12-27 08:25:02,625][20381] Updated weights for policy 1, policy_version 401278 (0.0008) [2023-12-27 08:25:02,680][20381] Updated weights for policy 1, policy_version 401288 (0.0007) [2023-12-27 08:25:03,005][20347] Updated weights for policy 0, policy_version 401601 (0.0005) [2023-12-27 08:25:03,053][20347] Updated weights for policy 0, policy_version 401611 (0.0009) [2023-12-27 08:25:03,098][20347] Updated weights for policy 0, policy_version 401621 (0.0008) [2023-12-27 08:25:03,144][20347] Updated weights for policy 0, policy_version 401631 (0.0009) [2023-12-27 08:25:03,274][20381] Updated weights for policy 1, policy_version 401298 (0.0008) [2023-12-27 08:25:03,330][20381] Updated weights for policy 1, policy_version 401308 (0.0005) [2023-12-27 08:25:03,392][20381] Updated weights for policy 1, policy_version 401318 (0.0005) [2023-12-27 08:25:03,449][20381] Updated weights for policy 1, policy_version 401328 (0.0006) [2023-12-27 08:25:03,891][20347] Updated weights for policy 0, policy_version 401641 (0.0008) [2023-12-27 08:25:03,953][20347] Updated weights for policy 0, policy_version 401651 (0.0006) [2023-12-27 08:25:04,015][20347] Updated weights for policy 0, policy_version 401661 (0.0006) [2023-12-27 08:25:04,039][20381] Updated weights for policy 1, policy_version 401338 (0.0007) [2023-12-27 08:25:04,094][20381] Updated weights for policy 1, policy_version 401348 (0.0007) [2023-12-27 08:25:04,153][20381] Updated weights for policy 1, policy_version 401358 (0.0008) [2023-12-27 08:25:04,608][20347] Updated weights for policy 0, policy_version 401671 (0.0007) [2023-12-27 08:25:04,669][20347] Updated weights for policy 0, policy_version 401681 (0.0008) [2023-12-27 08:25:04,729][20347] Updated weights for policy 0, policy_version 401691 (0.0007) [2023-12-27 08:25:04,886][20381] Updated weights for policy 1, policy_version 401368 (0.0008) [2023-12-27 08:25:04,935][20381] Updated weights for policy 1, policy_version 401378 (0.0008) [2023-12-27 08:25:04,981][20381] Updated weights for policy 1, policy_version 401388 (0.0007) [2023-12-27 08:25:05,334][20347] Updated weights for policy 0, policy_version 401701 (0.0008) [2023-12-27 08:25:05,394][20347] Updated weights for policy 0, policy_version 401711 (0.0007) [2023-12-27 08:25:05,448][20347] Updated weights for policy 0, policy_version 401721 (0.0008) [2023-12-27 08:25:05,695][20381] Updated weights for policy 1, policy_version 401398 (0.0007) [2023-12-27 08:25:05,750][20381] Updated weights for policy 1, policy_version 401408 (0.0007) [2023-12-27 08:25:05,808][20381] Updated weights for policy 1, policy_version 401418 (0.0008) [2023-12-27 08:25:06,184][20347] Updated weights for policy 0, policy_version 401731 (0.0009) [2023-12-27 08:25:06,241][20347] Updated weights for policy 0, policy_version 401741 (0.0011) [2023-12-27 08:25:06,305][20347] Updated weights for policy 0, policy_version 401751 (0.0011) [2023-12-27 08:25:06,570][20381] Updated weights for policy 1, policy_version 401428 (0.0007) [2023-12-27 08:25:06,629][20381] Updated weights for policy 1, policy_version 401438 (0.0008) [2023-12-27 08:25:06,681][20381] Updated weights for policy 1, policy_version 401448 (0.0007) [2023-12-27 08:25:06,986][20347] Updated weights for policy 0, policy_version 401761 (0.0008) [2023-12-27 08:25:07,053][20347] Updated weights for policy 0, policy_version 401771 (0.0007) [2023-12-27 08:25:07,119][20347] Updated weights for policy 0, policy_version 401781 (0.0008) [2023-12-27 08:25:07,186][20347] Updated weights for policy 0, policy_version 401791 (0.0008) [2023-12-27 08:25:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 205668352. Throughput: 0: 10541.2, 1: 10382.7. Samples: 205639656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:07,363][19170] Avg episode reward: [(0, '443.890'), (1, '951.940')] [2023-12-27 08:25:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000401456_102793216.pth... [2023-12-27 08:25:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000401792_102875136.pth... [2023-12-27 08:25:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000400240_102481920.pth [2023-12-27 08:25:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000400544_102555648.pth [2023-12-27 08:25:07,403][20381] Updated weights for policy 1, policy_version 401458 (0.0007) [2023-12-27 08:25:07,473][20381] Updated weights for policy 1, policy_version 401468 (0.0006) [2023-12-27 08:25:07,540][20381] Updated weights for policy 1, policy_version 401478 (0.0008) [2023-12-27 08:25:07,599][20381] Updated weights for policy 1, policy_version 401488 (0.0008) [2023-12-27 08:25:07,816][20347] Updated weights for policy 0, policy_version 401801 (0.0007) [2023-12-27 08:25:07,877][20347] Updated weights for policy 0, policy_version 401811 (0.0008) [2023-12-27 08:25:07,932][20347] Updated weights for policy 0, policy_version 401821 (0.0010) [2023-12-27 08:25:08,261][20381] Updated weights for policy 1, policy_version 401498 (0.0005) [2023-12-27 08:25:08,326][20381] Updated weights for policy 1, policy_version 401508 (0.0006) [2023-12-27 08:25:08,390][20381] Updated weights for policy 1, policy_version 401518 (0.0007) [2023-12-27 08:25:08,565][20347] Updated weights for policy 0, policy_version 401831 (0.0010) [2023-12-27 08:25:08,615][20347] Updated weights for policy 0, policy_version 401841 (0.0010) [2023-12-27 08:25:08,667][20347] Updated weights for policy 0, policy_version 401851 (0.0010) [2023-12-27 08:25:09,040][20381] Updated weights for policy 1, policy_version 401528 (0.0008) [2023-12-27 08:25:09,096][20381] Updated weights for policy 1, policy_version 401538 (0.0007) [2023-12-27 08:25:09,148][20381] Updated weights for policy 1, policy_version 401548 (0.0006) [2023-12-27 08:25:09,367][20347] Updated weights for policy 0, policy_version 401861 (0.0010) [2023-12-27 08:25:09,428][20347] Updated weights for policy 0, policy_version 401871 (0.0011) [2023-12-27 08:25:09,484][20347] Updated weights for policy 0, policy_version 401881 (0.0010) [2023-12-27 08:25:09,830][20381] Updated weights for policy 1, policy_version 401558 (0.0007) [2023-12-27 08:25:09,891][20381] Updated weights for policy 1, policy_version 401568 (0.0007) [2023-12-27 08:25:09,956][20381] Updated weights for policy 1, policy_version 401578 (0.0007) [2023-12-27 08:25:10,192][20347] Updated weights for policy 0, policy_version 401891 (0.0010) [2023-12-27 08:25:10,256][20347] Updated weights for policy 0, policy_version 401901 (0.0009) [2023-12-27 08:25:10,322][20347] Updated weights for policy 0, policy_version 401911 (0.0011) [2023-12-27 08:25:10,608][20381] Updated weights for policy 1, policy_version 401588 (0.0008) [2023-12-27 08:25:10,670][20381] Updated weights for policy 1, policy_version 401598 (0.0007) [2023-12-27 08:25:10,729][20381] Updated weights for policy 1, policy_version 401608 (0.0006) [2023-12-27 08:25:10,950][20347] Updated weights for policy 0, policy_version 401921 (0.0010) [2023-12-27 08:25:11,014][20347] Updated weights for policy 0, policy_version 401931 (0.0005) [2023-12-27 08:25:11,076][20347] Updated weights for policy 0, policy_version 401941 (0.0006) [2023-12-27 08:25:11,138][20347] Updated weights for policy 0, policy_version 401951 (0.0009) [2023-12-27 08:25:11,432][20381] Updated weights for policy 1, policy_version 401618 (0.0007) [2023-12-27 08:25:11,491][20381] Updated weights for policy 1, policy_version 401628 (0.0006) [2023-12-27 08:25:11,555][20381] Updated weights for policy 1, policy_version 401638 (0.0008) [2023-12-27 08:25:11,619][20381] Updated weights for policy 1, policy_version 401648 (0.0008) [2023-12-27 08:25:11,771][20347] Updated weights for policy 0, policy_version 401961 (0.0006) [2023-12-27 08:25:11,827][20347] Updated weights for policy 0, policy_version 401971 (0.0005) [2023-12-27 08:25:11,878][20347] Updated weights for policy 0, policy_version 401981 (0.0005) [2023-12-27 08:25:12,321][20381] Updated weights for policy 1, policy_version 401658 (0.0007) [2023-12-27 08:25:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 205766656. Throughput: 0: 10520.1, 1: 10361.8. Samples: 205761036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:12,363][19170] Avg episode reward: [(0, '440.650'), (1, '932.640')] [2023-12-27 08:25:12,385][20381] Updated weights for policy 1, policy_version 401668 (0.0008) [2023-12-27 08:25:12,448][20381] Updated weights for policy 1, policy_version 401678 (0.0007) [2023-12-27 08:25:12,515][20347] Updated weights for policy 0, policy_version 401991 (0.0007) [2023-12-27 08:25:12,566][20347] Updated weights for policy 0, policy_version 402001 (0.0007) [2023-12-27 08:25:12,630][20347] Updated weights for policy 0, policy_version 402011 (0.0006) [2023-12-27 08:25:13,102][20381] Updated weights for policy 1, policy_version 401688 (0.0008) [2023-12-27 08:25:13,164][20381] Updated weights for policy 1, policy_version 401698 (0.0008) [2023-12-27 08:25:13,220][20381] Updated weights for policy 1, policy_version 401708 (0.0008) [2023-12-27 08:25:13,280][20347] Updated weights for policy 0, policy_version 402021 (0.0005) [2023-12-27 08:25:13,349][20347] Updated weights for policy 0, policy_version 402031 (0.0005) [2023-12-27 08:25:13,405][20347] Updated weights for policy 0, policy_version 402041 (0.0005) [2023-12-27 08:25:13,784][20381] Updated weights for policy 1, policy_version 401718 (0.0008) [2023-12-27 08:25:13,832][20381] Updated weights for policy 1, policy_version 401728 (0.0006) [2023-12-27 08:25:13,880][20381] Updated weights for policy 1, policy_version 401738 (0.0005) [2023-12-27 08:25:14,020][20347] Updated weights for policy 0, policy_version 402051 (0.0006) [2023-12-27 08:25:14,076][20347] Updated weights for policy 0, policy_version 402061 (0.0007) [2023-12-27 08:25:14,128][20347] Updated weights for policy 0, policy_version 402071 (0.0009) [2023-12-27 08:25:14,579][20381] Updated weights for policy 1, policy_version 401748 (0.0006) [2023-12-27 08:25:14,633][20381] Updated weights for policy 1, policy_version 401758 (0.0005) [2023-12-27 08:25:14,691][20381] Updated weights for policy 1, policy_version 401768 (0.0005) [2023-12-27 08:25:14,722][20347] Updated weights for policy 0, policy_version 402081 (0.0008) [2023-12-27 08:25:14,778][20347] Updated weights for policy 0, policy_version 402091 (0.0006) [2023-12-27 08:25:14,841][20347] Updated weights for policy 0, policy_version 402101 (0.0007) [2023-12-27 08:25:14,900][20347] Updated weights for policy 0, policy_version 402111 (0.0009) [2023-12-27 08:25:15,282][20381] Updated weights for policy 1, policy_version 401778 (0.0006) [2023-12-27 08:25:15,340][20381] Updated weights for policy 1, policy_version 401788 (0.0009) [2023-12-27 08:25:15,395][20381] Updated weights for policy 1, policy_version 401798 (0.0010) [2023-12-27 08:25:15,447][20381] Updated weights for policy 1, policy_version 401808 (0.0011) [2023-12-27 08:25:15,615][20347] Updated weights for policy 0, policy_version 402121 (0.0007) [2023-12-27 08:25:15,673][20347] Updated weights for policy 0, policy_version 402131 (0.0005) [2023-12-27 08:25:15,736][20347] Updated weights for policy 0, policy_version 402141 (0.0005) [2023-12-27 08:25:16,051][20381] Updated weights for policy 1, policy_version 401818 (0.0011) [2023-12-27 08:25:16,100][20381] Updated weights for policy 1, policy_version 401828 (0.0010) [2023-12-27 08:25:16,152][20381] Updated weights for policy 1, policy_version 401838 (0.0011) [2023-12-27 08:25:16,283][20347] Updated weights for policy 0, policy_version 402151 (0.0005) [2023-12-27 08:25:16,336][20347] Updated weights for policy 0, policy_version 402161 (0.0007) [2023-12-27 08:25:16,393][20347] Updated weights for policy 0, policy_version 402171 (0.0005) [2023-12-27 08:25:16,850][20381] Updated weights for policy 1, policy_version 401848 (0.0008) [2023-12-27 08:25:16,898][20381] Updated weights for policy 1, policy_version 401858 (0.0006) [2023-12-27 08:25:16,927][20347] Updated weights for policy 0, policy_version 402181 (0.0005) [2023-12-27 08:25:16,944][20381] Updated weights for policy 1, policy_version 401868 (0.0008) [2023-12-27 08:25:16,985][20347] Updated weights for policy 0, policy_version 402191 (0.0005) [2023-12-27 08:25:17,048][20347] Updated weights for policy 0, policy_version 402201 (0.0005) [2023-12-27 08:25:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.0, 300 sec: 20799.3). Total num frames: 205881344. Throughput: 0: 10532.2, 1: 10369.7. Samples: 205889768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:17,364][19170] Avg episode reward: [(0, '703.650'), (1, '913.670')] [2023-12-27 08:25:17,628][20347] Updated weights for policy 0, policy_version 402211 (0.0006) [2023-12-27 08:25:17,680][20347] Updated weights for policy 0, policy_version 402221 (0.0008) [2023-12-27 08:25:17,694][20381] Updated weights for policy 1, policy_version 401878 (0.0006) [2023-12-27 08:25:17,742][20347] Updated weights for policy 0, policy_version 402231 (0.0008) [2023-12-27 08:25:17,756][20381] Updated weights for policy 1, policy_version 401888 (0.0007) [2023-12-27 08:25:17,811][20381] Updated weights for policy 1, policy_version 401898 (0.0008) [2023-12-27 08:25:18,390][20347] Updated weights for policy 0, policy_version 402241 (0.0006) [2023-12-27 08:25:18,410][20381] Updated weights for policy 1, policy_version 401908 (0.0006) [2023-12-27 08:25:18,455][20347] Updated weights for policy 0, policy_version 402251 (0.0007) [2023-12-27 08:25:18,462][20381] Updated weights for policy 1, policy_version 401918 (0.0008) [2023-12-27 08:25:18,519][20381] Updated weights for policy 1, policy_version 401928 (0.0008) [2023-12-27 08:25:18,519][20347] Updated weights for policy 0, policy_version 402261 (0.0006) [2023-12-27 08:25:18,578][20347] Updated weights for policy 0, policy_version 402271 (0.0010) [2023-12-27 08:25:19,173][20347] Updated weights for policy 0, policy_version 402281 (0.0010) [2023-12-27 08:25:19,224][20347] Updated weights for policy 0, policy_version 402291 (0.0012) [2023-12-27 08:25:19,239][20381] Updated weights for policy 1, policy_version 401938 (0.0007) [2023-12-27 08:25:19,283][20347] Updated weights for policy 0, policy_version 402301 (0.0010) [2023-12-27 08:25:19,300][20381] Updated weights for policy 1, policy_version 401948 (0.0006) [2023-12-27 08:25:19,364][20381] Updated weights for policy 1, policy_version 401958 (0.0008) [2023-12-27 08:25:19,426][20381] Updated weights for policy 1, policy_version 401968 (0.0009) [2023-12-27 08:25:20,048][20347] Updated weights for policy 0, policy_version 402311 (0.0007) [2023-12-27 08:25:20,096][20381] Updated weights for policy 1, policy_version 401978 (0.0007) [2023-12-27 08:25:20,115][20347] Updated weights for policy 0, policy_version 402321 (0.0006) [2023-12-27 08:25:20,157][20381] Updated weights for policy 1, policy_version 401988 (0.0007) [2023-12-27 08:25:20,183][20347] Updated weights for policy 0, policy_version 402331 (0.0006) [2023-12-27 08:25:20,215][20381] Updated weights for policy 1, policy_version 401998 (0.0009) [2023-12-27 08:25:20,836][20347] Updated weights for policy 0, policy_version 402341 (0.0006) [2023-12-27 08:25:20,886][20347] Updated weights for policy 0, policy_version 402351 (0.0006) [2023-12-27 08:25:20,892][20381] Updated weights for policy 1, policy_version 402008 (0.0007) [2023-12-27 08:25:20,945][20347] Updated weights for policy 0, policy_version 402361 (0.0006) [2023-12-27 08:25:20,954][20381] Updated weights for policy 1, policy_version 402018 (0.0006) [2023-12-27 08:25:21,008][20381] Updated weights for policy 1, policy_version 402028 (0.0008) [2023-12-27 08:25:21,700][20381] Updated weights for policy 1, policy_version 402038 (0.0008) [2023-12-27 08:25:21,720][20347] Updated weights for policy 0, policy_version 402371 (0.0007) [2023-12-27 08:25:21,768][20381] Updated weights for policy 1, policy_version 402048 (0.0007) [2023-12-27 08:25:21,781][20347] Updated weights for policy 0, policy_version 402381 (0.0007) [2023-12-27 08:25:21,828][20381] Updated weights for policy 1, policy_version 402058 (0.0007) [2023-12-27 08:25:21,838][20347] Updated weights for policy 0, policy_version 402391 (0.0005) [2023-12-27 08:25:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 205979648. Throughput: 0: 10524.0, 1: 10386.0. Samples: 205951880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:22,363][19170] Avg episode reward: [(0, '816.830'), (1, '923.380')] [2023-12-27 08:25:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000402064_102948864.pth... [2023-12-27 08:25:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000402400_103030784.pth... [2023-12-27 08:25:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000400880_102645760.pth [2023-12-27 08:25:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000401184_102719488.pth [2023-12-27 08:25:22,521][20381] Updated weights for policy 1, policy_version 402068 (0.0007) [2023-12-27 08:25:22,548][20347] Updated weights for policy 0, policy_version 402401 (0.0007) [2023-12-27 08:25:22,584][20381] Updated weights for policy 1, policy_version 402078 (0.0009) [2023-12-27 08:25:22,606][20347] Updated weights for policy 0, policy_version 402411 (0.0006) [2023-12-27 08:25:22,641][20381] Updated weights for policy 1, policy_version 402088 (0.0007) [2023-12-27 08:25:22,668][20347] Updated weights for policy 0, policy_version 402421 (0.0007) [2023-12-27 08:25:22,724][20347] Updated weights for policy 0, policy_version 402431 (0.0007) [2023-12-27 08:25:23,333][20381] Updated weights for policy 1, policy_version 402098 (0.0008) [2023-12-27 08:25:23,384][20347] Updated weights for policy 0, policy_version 402441 (0.0006) [2023-12-27 08:25:23,393][20381] Updated weights for policy 1, policy_version 402108 (0.0008) [2023-12-27 08:25:23,420][20292] KL-divergence is very high: 262.2586 [2023-12-27 08:25:23,433][20347] Updated weights for policy 0, policy_version 402451 (0.0005) [2023-12-27 08:25:23,439][20381] Updated weights for policy 1, policy_version 402118 (0.0006) [2023-12-27 08:25:23,485][20347] Updated weights for policy 0, policy_version 402461 (0.0006) [2023-12-27 08:25:23,488][20381] Updated weights for policy 1, policy_version 402128 (0.0005) [2023-12-27 08:25:24,069][20347] Updated weights for policy 0, policy_version 402471 (0.0006) [2023-12-27 08:25:24,117][20347] Updated weights for policy 0, policy_version 402481 (0.0005) [2023-12-27 08:25:24,170][20347] Updated weights for policy 0, policy_version 402491 (0.0006) [2023-12-27 08:25:24,253][20381] Updated weights for policy 1, policy_version 402138 (0.0008) [2023-12-27 08:25:24,307][20381] Updated weights for policy 1, policy_version 402148 (0.0005) [2023-12-27 08:25:24,355][20381] Updated weights for policy 1, policy_version 402158 (0.0005) [2023-12-27 08:25:24,817][20347] Updated weights for policy 0, policy_version 402501 (0.0007) [2023-12-27 08:25:24,869][20347] Updated weights for policy 0, policy_version 402511 (0.0009) [2023-12-27 08:25:24,913][20347] Updated weights for policy 0, policy_version 402521 (0.0008) [2023-12-27 08:25:24,954][20381] Updated weights for policy 1, policy_version 402168 (0.0007) [2023-12-27 08:25:25,014][20381] Updated weights for policy 1, policy_version 402178 (0.0006) [2023-12-27 08:25:25,072][20381] Updated weights for policy 1, policy_version 402188 (0.0009) [2023-12-27 08:25:25,608][20347] Updated weights for policy 0, policy_version 402531 (0.0008) [2023-12-27 08:25:25,660][20347] Updated weights for policy 0, policy_version 402541 (0.0005) [2023-12-27 08:25:25,718][20347] Updated weights for policy 0, policy_version 402551 (0.0008) [2023-12-27 08:25:25,723][20381] Updated weights for policy 1, policy_version 402198 (0.0008) [2023-12-27 08:25:25,779][20381] Updated weights for policy 1, policy_version 402208 (0.0010) [2023-12-27 08:25:25,840][20381] Updated weights for policy 1, policy_version 402218 (0.0010) [2023-12-27 08:25:26,360][20347] Updated weights for policy 0, policy_version 402561 (0.0010) [2023-12-27 08:25:26,425][20347] Updated weights for policy 0, policy_version 402571 (0.0006) [2023-12-27 08:25:26,449][20381] Updated weights for policy 1, policy_version 402228 (0.0010) [2023-12-27 08:25:26,483][20347] Updated weights for policy 0, policy_version 402581 (0.0010) [2023-12-27 08:25:26,505][20381] Updated weights for policy 1, policy_version 402238 (0.0009) [2023-12-27 08:25:26,540][20347] Updated weights for policy 0, policy_version 402591 (0.0008) [2023-12-27 08:25:26,564][20381] Updated weights for policy 1, policy_version 402248 (0.0006) [2023-12-27 08:25:27,156][20347] Updated weights for policy 0, policy_version 402601 (0.0009) [2023-12-27 08:25:27,208][20347] Updated weights for policy 0, policy_version 402611 (0.0006) [2023-12-27 08:25:27,241][20381] Updated weights for policy 1, policy_version 402258 (0.0007) [2023-12-27 08:25:27,275][20347] Updated weights for policy 0, policy_version 402621 (0.0006) [2023-12-27 08:25:27,301][20381] Updated weights for policy 1, policy_version 402268 (0.0008) [2023-12-27 08:25:27,360][20381] Updated weights for policy 1, policy_version 402278 (0.0011) [2023-12-27 08:25:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 206086144. Throughput: 0: 10483.1, 1: 10358.5. Samples: 206075156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:27,363][19170] Avg episode reward: [(0, '875.360'), (1, '874.890')] [2023-12-27 08:25:27,424][20381] Updated weights for policy 1, policy_version 402288 (0.0005) [2023-12-27 08:25:27,883][20347] Updated weights for policy 0, policy_version 402631 (0.0006) [2023-12-27 08:25:27,940][20347] Updated weights for policy 0, policy_version 402641 (0.0008) [2023-12-27 08:25:27,966][20381] Updated weights for policy 1, policy_version 402298 (0.0005) [2023-12-27 08:25:27,984][20347] Updated weights for policy 0, policy_version 402651 (0.0007) [2023-12-27 08:25:28,015][20381] Updated weights for policy 1, policy_version 402308 (0.0006) [2023-12-27 08:25:28,065][20381] Updated weights for policy 1, policy_version 402318 (0.0007) [2023-12-27 08:25:28,656][20347] Updated weights for policy 0, policy_version 402661 (0.0007) [2023-12-27 08:25:28,721][20347] Updated weights for policy 0, policy_version 402671 (0.0007) [2023-12-27 08:25:28,770][20381] Updated weights for policy 1, policy_version 402328 (0.0010) [2023-12-27 08:25:28,775][20347] Updated weights for policy 0, policy_version 402681 (0.0006) [2023-12-27 08:25:28,829][20381] Updated weights for policy 1, policy_version 402338 (0.0010) [2023-12-27 08:25:28,892][20381] Updated weights for policy 1, policy_version 402348 (0.0011) [2023-12-27 08:25:29,480][20347] Updated weights for policy 0, policy_version 402691 (0.0005) [2023-12-27 08:25:29,528][20347] Updated weights for policy 0, policy_version 402701 (0.0007) [2023-12-27 08:25:29,578][20381] Updated weights for policy 1, policy_version 402358 (0.0009) [2023-12-27 08:25:29,584][20347] Updated weights for policy 0, policy_version 402711 (0.0007) [2023-12-27 08:25:29,635][20381] Updated weights for policy 1, policy_version 402368 (0.0006) [2023-12-27 08:25:29,685][20381] Updated weights for policy 1, policy_version 402378 (0.0008) [2023-12-27 08:25:30,278][20347] Updated weights for policy 0, policy_version 402721 (0.0006) [2023-12-27 08:25:30,327][20347] Updated weights for policy 0, policy_version 402731 (0.0007) [2023-12-27 08:25:30,360][20381] Updated weights for policy 1, policy_version 402388 (0.0007) [2023-12-27 08:25:30,384][20347] Updated weights for policy 0, policy_version 402741 (0.0008) [2023-12-27 08:25:30,413][20381] Updated weights for policy 1, policy_version 402398 (0.0005) [2023-12-27 08:25:30,438][20347] Updated weights for policy 0, policy_version 402751 (0.0007) [2023-12-27 08:25:30,470][20381] Updated weights for policy 1, policy_version 402408 (0.0005) [2023-12-27 08:25:30,988][20381] Updated weights for policy 1, policy_version 402418 (0.0005) [2023-12-27 08:25:31,058][20381] Updated weights for policy 1, policy_version 402428 (0.0006) [2023-12-27 08:25:31,119][20381] Updated weights for policy 1, policy_version 402438 (0.0007) [2023-12-27 08:25:31,169][20347] Updated weights for policy 0, policy_version 402761 (0.0007) [2023-12-27 08:25:31,177][20381] Updated weights for policy 1, policy_version 402448 (0.0006) [2023-12-27 08:25:31,222][20347] Updated weights for policy 0, policy_version 402771 (0.0006) [2023-12-27 08:25:31,279][20347] Updated weights for policy 0, policy_version 402781 (0.0008) [2023-12-27 08:25:31,804][20381] Updated weights for policy 1, policy_version 402458 (0.0005) [2023-12-27 08:25:31,858][20381] Updated weights for policy 1, policy_version 402468 (0.0006) [2023-12-27 08:25:31,908][20381] Updated weights for policy 1, policy_version 402478 (0.0005) [2023-12-27 08:25:32,019][20347] Updated weights for policy 0, policy_version 402791 (0.0008) [2023-12-27 08:25:32,073][20347] Updated weights for policy 0, policy_version 402801 (0.0007) [2023-12-27 08:25:32,123][20347] Updated weights for policy 0, policy_version 402811 (0.0008) [2023-12-27 08:25:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 206192640. Throughput: 0: 10427.0, 1: 10361.7. Samples: 206202192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:32,363][19170] Avg episode reward: [(0, '778.620'), (1, '913.410')] [2023-12-27 08:25:32,524][20381] Updated weights for policy 1, policy_version 402488 (0.0007) [2023-12-27 08:25:32,573][20381] Updated weights for policy 1, policy_version 402498 (0.0008) [2023-12-27 08:25:32,626][20381] Updated weights for policy 1, policy_version 402508 (0.0008) [2023-12-27 08:25:32,833][20347] Updated weights for policy 0, policy_version 402821 (0.0008) [2023-12-27 08:25:32,880][20347] Updated weights for policy 0, policy_version 402831 (0.0007) [2023-12-27 08:25:32,927][20347] Updated weights for policy 0, policy_version 402841 (0.0007) [2023-12-27 08:25:33,318][20381] Updated weights for policy 1, policy_version 402518 (0.0009) [2023-12-27 08:25:33,375][20381] Updated weights for policy 1, policy_version 402528 (0.0010) [2023-12-27 08:25:33,428][20381] Updated weights for policy 1, policy_version 402538 (0.0010) [2023-12-27 08:25:33,570][20347] Updated weights for policy 0, policy_version 402851 (0.0008) [2023-12-27 08:25:33,621][20347] Updated weights for policy 0, policy_version 402861 (0.0010) [2023-12-27 08:25:33,676][20347] Updated weights for policy 0, policy_version 402871 (0.0010) [2023-12-27 08:25:34,040][20381] Updated weights for policy 1, policy_version 402548 (0.0010) [2023-12-27 08:25:34,101][20381] Updated weights for policy 1, policy_version 402558 (0.0008) [2023-12-27 08:25:34,163][20381] Updated weights for policy 1, policy_version 402568 (0.0008) [2023-12-27 08:25:34,330][20347] Updated weights for policy 0, policy_version 402881 (0.0010) [2023-12-27 08:25:34,390][20347] Updated weights for policy 0, policy_version 402891 (0.0007) [2023-12-27 08:25:34,440][20347] Updated weights for policy 0, policy_version 402901 (0.0007) [2023-12-27 08:25:34,497][20347] Updated weights for policy 0, policy_version 402911 (0.0007) [2023-12-27 08:25:34,904][20381] Updated weights for policy 1, policy_version 402578 (0.0008) [2023-12-27 08:25:34,956][20381] Updated weights for policy 1, policy_version 402588 (0.0005) [2023-12-27 08:25:35,020][20381] Updated weights for policy 1, policy_version 402598 (0.0007) [2023-12-27 08:25:35,076][20381] Updated weights for policy 1, policy_version 402608 (0.0008) [2023-12-27 08:25:35,143][20347] Updated weights for policy 0, policy_version 402921 (0.0006) [2023-12-27 08:25:35,196][20347] Updated weights for policy 0, policy_version 402931 (0.0005) [2023-12-27 08:25:35,254][20347] Updated weights for policy 0, policy_version 402941 (0.0005) [2023-12-27 08:25:35,683][20381] Updated weights for policy 1, policy_version 402618 (0.0006) [2023-12-27 08:25:35,739][20381] Updated weights for policy 1, policy_version 402628 (0.0005) [2023-12-27 08:25:35,792][20347] Updated weights for policy 0, policy_version 402951 (0.0005) [2023-12-27 08:25:35,797][20381] Updated weights for policy 1, policy_version 402638 (0.0005) [2023-12-27 08:25:35,848][20347] Updated weights for policy 0, policy_version 402961 (0.0006) [2023-12-27 08:25:35,902][20347] Updated weights for policy 0, policy_version 402971 (0.0006) [2023-12-27 08:25:36,397][20381] Updated weights for policy 1, policy_version 402648 (0.0008) [2023-12-27 08:25:36,464][20381] Updated weights for policy 1, policy_version 402658 (0.0007) [2023-12-27 08:25:36,519][20381] Updated weights for policy 1, policy_version 402668 (0.0008) [2023-12-27 08:25:36,556][20347] Updated weights for policy 0, policy_version 402981 (0.0008) [2023-12-27 08:25:36,616][20347] Updated weights for policy 0, policy_version 402991 (0.0008) [2023-12-27 08:25:36,683][20347] Updated weights for policy 0, policy_version 403001 (0.0007) [2023-12-27 08:25:37,235][20381] Updated weights for policy 1, policy_version 402678 (0.0007) [2023-12-27 08:25:37,289][20381] Updated weights for policy 1, policy_version 402688 (0.0007) [2023-12-27 08:25:37,308][20347] Updated weights for policy 0, policy_version 403011 (0.0008) [2023-12-27 08:25:37,347][20381] Updated weights for policy 1, policy_version 402698 (0.0005) [2023-12-27 08:25:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.8, 300 sec: 20771.6). Total num frames: 206290944. Throughput: 0: 10434.6, 1: 10367.8. Samples: 206264440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:37,363][19170] Avg episode reward: [(0, '788.440'), (1, '942.420')] [2023-12-27 08:25:37,363][20347] Updated weights for policy 0, policy_version 403021 (0.0010) [2023-12-27 08:25:37,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000402704_103112704.pth... [2023-12-27 08:25:37,383][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000401456_102793216.pth [2023-12-27 08:25:37,431][20347] Updated weights for policy 0, policy_version 403031 (0.0010) [2023-12-27 08:25:37,488][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000403040_103194624.pth... [2023-12-27 08:25:37,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000401792_102875136.pth [2023-12-27 08:25:38,036][20381] Updated weights for policy 1, policy_version 402708 (0.0008) [2023-12-27 08:25:38,092][20381] Updated weights for policy 1, policy_version 402718 (0.0011) [2023-12-27 08:25:38,102][20347] Updated weights for policy 0, policy_version 403041 (0.0010) [2023-12-27 08:25:38,142][20381] Updated weights for policy 1, policy_version 402728 (0.0011) [2023-12-27 08:25:38,153][20347] Updated weights for policy 0, policy_version 403051 (0.0010) [2023-12-27 08:25:38,199][20347] Updated weights for policy 0, policy_version 403061 (0.0010) [2023-12-27 08:25:38,248][20347] Updated weights for policy 0, policy_version 403071 (0.0010) [2023-12-27 08:25:38,856][20381] Updated weights for policy 1, policy_version 402738 (0.0009) [2023-12-27 08:25:38,913][20381] Updated weights for policy 1, policy_version 402748 (0.0009) [2023-12-27 08:25:38,952][20347] Updated weights for policy 0, policy_version 403081 (0.0007) [2023-12-27 08:25:38,969][20381] Updated weights for policy 1, policy_version 402758 (0.0011) [2023-12-27 08:25:39,007][20347] Updated weights for policy 0, policy_version 403091 (0.0005) [2023-12-27 08:25:39,024][20381] Updated weights for policy 1, policy_version 402768 (0.0011) [2023-12-27 08:25:39,069][20347] Updated weights for policy 0, policy_version 403101 (0.0005) [2023-12-27 08:25:39,683][20347] Updated weights for policy 0, policy_version 403111 (0.0006) [2023-12-27 08:25:39,706][20381] Updated weights for policy 1, policy_version 402778 (0.0011) [2023-12-27 08:25:39,745][20347] Updated weights for policy 0, policy_version 403121 (0.0006) [2023-12-27 08:25:39,763][20381] Updated weights for policy 1, policy_version 402788 (0.0011) [2023-12-27 08:25:39,802][20347] Updated weights for policy 0, policy_version 403131 (0.0005) [2023-12-27 08:25:39,812][20381] Updated weights for policy 1, policy_version 402798 (0.0010) [2023-12-27 08:25:40,511][20381] Updated weights for policy 1, policy_version 402808 (0.0011) [2023-12-27 08:25:40,536][20347] Updated weights for policy 0, policy_version 403141 (0.0008) [2023-12-27 08:25:40,575][20381] Updated weights for policy 1, policy_version 402818 (0.0010) [2023-12-27 08:25:40,599][20347] Updated weights for policy 0, policy_version 403151 (0.0006) [2023-12-27 08:25:40,642][20381] Updated weights for policy 1, policy_version 402828 (0.0008) [2023-12-27 08:25:40,654][20347] Updated weights for policy 0, policy_version 403161 (0.0007) [2023-12-27 08:25:41,301][20381] Updated weights for policy 1, policy_version 402838 (0.0007) [2023-12-27 08:25:41,363][20381] Updated weights for policy 1, policy_version 402848 (0.0008) [2023-12-27 08:25:41,401][20347] Updated weights for policy 0, policy_version 403171 (0.0008) [2023-12-27 08:25:41,428][20381] Updated weights for policy 1, policy_version 402858 (0.0008) [2023-12-27 08:25:41,465][20347] Updated weights for policy 0, policy_version 403181 (0.0007) [2023-12-27 08:25:41,532][20347] Updated weights for policy 0, policy_version 403191 (0.0008) [2023-12-27 08:25:42,111][20381] Updated weights for policy 1, policy_version 402868 (0.0006) [2023-12-27 08:25:42,171][20381] Updated weights for policy 1, policy_version 402878 (0.0005) [2023-12-27 08:25:42,232][20381] Updated weights for policy 1, policy_version 402888 (0.0005) [2023-12-27 08:25:42,272][20347] Updated weights for policy 0, policy_version 403201 (0.0008) [2023-12-27 08:25:42,330][20347] Updated weights for policy 0, policy_version 403211 (0.0008) [2023-12-27 08:25:42,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 206397440. Throughput: 0: 10473.1, 1: 10340.7. Samples: 206389424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:42,363][19170] Avg episode reward: [(0, '845.460'), (1, '980.790')] [2023-12-27 08:25:42,392][20347] Updated weights for policy 0, policy_version 403221 (0.0007) [2023-12-27 08:25:42,449][20347] Updated weights for policy 0, policy_version 403231 (0.0007) [2023-12-27 08:25:42,869][20381] Updated weights for policy 1, policy_version 402898 (0.0007) [2023-12-27 08:25:42,932][20381] Updated weights for policy 1, policy_version 402908 (0.0005) [2023-12-27 08:25:42,990][20381] Updated weights for policy 1, policy_version 402918 (0.0006) [2023-12-27 08:25:43,051][20381] Updated weights for policy 1, policy_version 402928 (0.0008) [2023-12-27 08:25:43,185][20347] Updated weights for policy 0, policy_version 403241 (0.0007) [2023-12-27 08:25:43,236][20347] Updated weights for policy 0, policy_version 403251 (0.0008) [2023-12-27 08:25:43,302][20347] Updated weights for policy 0, policy_version 403261 (0.0008) [2023-12-27 08:25:43,662][20381] Updated weights for policy 1, policy_version 402938 (0.0010) [2023-12-27 08:25:43,723][20381] Updated weights for policy 1, policy_version 402948 (0.0010) [2023-12-27 08:25:43,775][20381] Updated weights for policy 1, policy_version 402958 (0.0010) [2023-12-27 08:25:43,999][20347] Updated weights for policy 0, policy_version 403271 (0.0007) [2023-12-27 08:25:44,051][20347] Updated weights for policy 0, policy_version 403281 (0.0008) [2023-12-27 08:25:44,099][20347] Updated weights for policy 0, policy_version 403291 (0.0007) [2023-12-27 08:25:44,471][20381] Updated weights for policy 1, policy_version 402968 (0.0008) [2023-12-27 08:25:44,527][20381] Updated weights for policy 1, policy_version 402978 (0.0008) [2023-12-27 08:25:44,590][20381] Updated weights for policy 1, policy_version 402988 (0.0009) [2023-12-27 08:25:44,729][20347] Updated weights for policy 0, policy_version 403301 (0.0006) [2023-12-27 08:25:44,789][20347] Updated weights for policy 0, policy_version 403311 (0.0007) [2023-12-27 08:25:44,844][20347] Updated weights for policy 0, policy_version 403321 (0.0008) [2023-12-27 08:25:45,315][20381] Updated weights for policy 1, policy_version 402998 (0.0010) [2023-12-27 08:25:45,378][20381] Updated weights for policy 1, policy_version 403008 (0.0010) [2023-12-27 08:25:45,432][20381] Updated weights for policy 1, policy_version 403018 (0.0010) [2023-12-27 08:25:45,545][20347] Updated weights for policy 0, policy_version 403331 (0.0008) [2023-12-27 08:25:45,596][20347] Updated weights for policy 0, policy_version 403341 (0.0007) [2023-12-27 08:25:45,647][20347] Updated weights for policy 0, policy_version 403351 (0.0007) [2023-12-27 08:25:46,098][20381] Updated weights for policy 1, policy_version 403028 (0.0010) [2023-12-27 08:25:46,146][20381] Updated weights for policy 1, policy_version 403038 (0.0010) [2023-12-27 08:25:46,202][20381] Updated weights for policy 1, policy_version 403048 (0.0010) [2023-12-27 08:25:46,359][20347] Updated weights for policy 0, policy_version 403361 (0.0007) [2023-12-27 08:25:46,420][20347] Updated weights for policy 0, policy_version 403371 (0.0007) [2023-12-27 08:25:46,499][20347] Updated weights for policy 0, policy_version 403381 (0.0008) [2023-12-27 08:25:46,554][20347] Updated weights for policy 0, policy_version 403391 (0.0007) [2023-12-27 08:25:46,879][20381] Updated weights for policy 1, policy_version 403058 (0.0010) [2023-12-27 08:25:46,931][20381] Updated weights for policy 1, policy_version 403068 (0.0010) [2023-12-27 08:25:46,982][20381] Updated weights for policy 1, policy_version 403078 (0.0010) [2023-12-27 08:25:47,043][20381] Updated weights for policy 1, policy_version 403088 (0.0010) [2023-12-27 08:25:47,233][20347] Updated weights for policy 0, policy_version 403401 (0.0007) [2023-12-27 08:25:47,287][20347] Updated weights for policy 0, policy_version 403411 (0.0007) [2023-12-27 08:25:47,334][20347] Updated weights for policy 0, policy_version 403421 (0.0007) [2023-12-27 08:25:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 206503936. Throughput: 0: 10408.5, 1: 10341.4. Samples: 206510120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:47,363][19170] Avg episode reward: [(0, '865.140'), (1, '971.260')] [2023-12-27 08:25:47,733][20381] Updated weights for policy 1, policy_version 403098 (0.0010) [2023-12-27 08:25:47,782][20381] Updated weights for policy 1, policy_version 403108 (0.0007) [2023-12-27 08:25:47,827][20381] Updated weights for policy 1, policy_version 403118 (0.0006) [2023-12-27 08:25:47,989][20347] Updated weights for policy 0, policy_version 403431 (0.0006) [2023-12-27 08:25:48,042][20347] Updated weights for policy 0, policy_version 403441 (0.0005) [2023-12-27 08:25:48,100][20347] Updated weights for policy 0, policy_version 403451 (0.0007) [2023-12-27 08:25:48,534][20381] Updated weights for policy 1, policy_version 403128 (0.0007) [2023-12-27 08:25:48,591][20381] Updated weights for policy 1, policy_version 403138 (0.0007) [2023-12-27 08:25:48,650][20381] Updated weights for policy 1, policy_version 403148 (0.0007) [2023-12-27 08:25:48,734][20347] Updated weights for policy 0, policy_version 403461 (0.0008) [2023-12-27 08:25:48,797][20347] Updated weights for policy 0, policy_version 403471 (0.0008) [2023-12-27 08:25:48,857][20347] Updated weights for policy 0, policy_version 403481 (0.0008) [2023-12-27 08:25:49,358][20381] Updated weights for policy 1, policy_version 403158 (0.0008) [2023-12-27 08:25:49,416][20381] Updated weights for policy 1, policy_version 403168 (0.0007) [2023-12-27 08:25:49,473][20381] Updated weights for policy 1, policy_version 403178 (0.0008) [2023-12-27 08:25:49,590][20347] Updated weights for policy 0, policy_version 403491 (0.0008) [2023-12-27 08:25:49,640][20347] Updated weights for policy 0, policy_version 403501 (0.0008) [2023-12-27 08:25:49,698][20347] Updated weights for policy 0, policy_version 403511 (0.0008) [2023-12-27 08:25:50,161][20381] Updated weights for policy 1, policy_version 403188 (0.0008) [2023-12-27 08:25:50,217][20381] Updated weights for policy 1, policy_version 403198 (0.0009) [2023-12-27 08:25:50,280][20381] Updated weights for policy 1, policy_version 403208 (0.0011) [2023-12-27 08:25:50,447][20347] Updated weights for policy 0, policy_version 403521 (0.0007) [2023-12-27 08:25:50,499][20347] Updated weights for policy 0, policy_version 403531 (0.0008) [2023-12-27 08:25:50,547][20347] Updated weights for policy 0, policy_version 403541 (0.0007) [2023-12-27 08:25:50,603][20347] Updated weights for policy 0, policy_version 403551 (0.0006) [2023-12-27 08:25:50,928][20381] Updated weights for policy 1, policy_version 403218 (0.0010) [2023-12-27 08:25:50,978][20381] Updated weights for policy 1, policy_version 403228 (0.0006) [2023-12-27 08:25:51,041][20381] Updated weights for policy 1, policy_version 403238 (0.0008) [2023-12-27 08:25:51,103][20381] Updated weights for policy 1, policy_version 403248 (0.0008) [2023-12-27 08:25:51,399][20347] Updated weights for policy 0, policy_version 403561 (0.0008) [2023-12-27 08:25:51,459][20347] Updated weights for policy 0, policy_version 403571 (0.0008) [2023-12-27 08:25:51,520][20347] Updated weights for policy 0, policy_version 403581 (0.0008) [2023-12-27 08:25:51,774][20381] Updated weights for policy 1, policy_version 403258 (0.0008) [2023-12-27 08:25:51,838][20381] Updated weights for policy 1, policy_version 403268 (0.0006) [2023-12-27 08:25:51,902][20381] Updated weights for policy 1, policy_version 403278 (0.0006) [2023-12-27 08:25:52,207][20347] Updated weights for policy 0, policy_version 403591 (0.0008) [2023-12-27 08:25:52,268][20347] Updated weights for policy 0, policy_version 403601 (0.0008) [2023-12-27 08:25:52,321][20347] Updated weights for policy 0, policy_version 403611 (0.0007) [2023-12-27 08:25:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 206602240. Throughput: 0: 10377.6, 1: 10317.6. Samples: 206570940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:52,363][19170] Avg episode reward: [(0, '865.110'), (1, '961.870')] [2023-12-27 08:25:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000403280_103260160.pth... [2023-12-27 08:25:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000403616_103342080.pth... [2023-12-27 08:25:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000402064_102948864.pth [2023-12-27 08:25:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000402400_103030784.pth [2023-12-27 08:25:52,537][20381] Updated weights for policy 1, policy_version 403288 (0.0008) [2023-12-27 08:25:52,602][20381] Updated weights for policy 1, policy_version 403298 (0.0007) [2023-12-27 08:25:52,666][20381] Updated weights for policy 1, policy_version 403308 (0.0007) [2023-12-27 08:25:52,955][20347] Updated weights for policy 0, policy_version 403621 (0.0006) [2023-12-27 08:25:53,016][20347] Updated weights for policy 0, policy_version 403631 (0.0007) [2023-12-27 08:25:53,075][20347] Updated weights for policy 0, policy_version 403641 (0.0006) [2023-12-27 08:25:53,306][20381] Updated weights for policy 1, policy_version 403318 (0.0006) [2023-12-27 08:25:53,362][20381] Updated weights for policy 1, policy_version 403328 (0.0005) [2023-12-27 08:25:53,428][20381] Updated weights for policy 1, policy_version 403338 (0.0005) [2023-12-27 08:25:53,724][20347] Updated weights for policy 0, policy_version 403651 (0.0008) [2023-12-27 08:25:53,779][20347] Updated weights for policy 0, policy_version 403661 (0.0007) [2023-12-27 08:25:53,837][20347] Updated weights for policy 0, policy_version 403671 (0.0010) [2023-12-27 08:25:53,955][20381] Updated weights for policy 1, policy_version 403348 (0.0006) [2023-12-27 08:25:54,022][20381] Updated weights for policy 1, policy_version 403358 (0.0008) [2023-12-27 08:25:54,081][20381] Updated weights for policy 1, policy_version 403368 (0.0006) [2023-12-27 08:25:54,487][20347] Updated weights for policy 0, policy_version 403681 (0.0010) [2023-12-27 08:25:54,540][20347] Updated weights for policy 0, policy_version 403691 (0.0009) [2023-12-27 08:25:54,596][20347] Updated weights for policy 0, policy_version 403701 (0.0009) [2023-12-27 08:25:54,654][20347] Updated weights for policy 0, policy_version 403711 (0.0010) [2023-12-27 08:25:54,704][20381] Updated weights for policy 1, policy_version 403378 (0.0006) [2023-12-27 08:25:54,750][20381] Updated weights for policy 1, policy_version 403388 (0.0007) [2023-12-27 08:25:54,798][20381] Updated weights for policy 1, policy_version 403398 (0.0007) [2023-12-27 08:25:54,847][20381] Updated weights for policy 1, policy_version 403408 (0.0008) [2023-12-27 08:25:55,230][20347] Updated weights for policy 0, policy_version 403721 (0.0010) [2023-12-27 08:25:55,291][20347] Updated weights for policy 0, policy_version 403731 (0.0010) [2023-12-27 08:25:55,336][20347] Updated weights for policy 0, policy_version 403741 (0.0010) [2023-12-27 08:25:55,511][20381] Updated weights for policy 1, policy_version 403418 (0.0008) [2023-12-27 08:25:55,566][20381] Updated weights for policy 1, policy_version 403428 (0.0007) [2023-12-27 08:25:55,629][20381] Updated weights for policy 1, policy_version 403438 (0.0008) [2023-12-27 08:25:55,977][20347] Updated weights for policy 0, policy_version 403751 (0.0007) [2023-12-27 08:25:56,029][20347] Updated weights for policy 0, policy_version 403761 (0.0005) [2023-12-27 08:25:56,087][20347] Updated weights for policy 0, policy_version 403771 (0.0008) [2023-12-27 08:25:56,331][20381] Updated weights for policy 1, policy_version 403448 (0.0005) [2023-12-27 08:25:56,382][20381] Updated weights for policy 1, policy_version 403458 (0.0005) [2023-12-27 08:25:56,437][20381] Updated weights for policy 1, policy_version 403468 (0.0007) [2023-12-27 08:25:56,674][20347] Updated weights for policy 0, policy_version 403781 (0.0010) [2023-12-27 08:25:56,723][20347] Updated weights for policy 0, policy_version 403791 (0.0010) [2023-12-27 08:25:56,771][20347] Updated weights for policy 0, policy_version 403801 (0.0010) [2023-12-27 08:25:57,118][20381] Updated weights for policy 1, policy_version 403478 (0.0011) [2023-12-27 08:25:57,166][20381] Updated weights for policy 1, policy_version 403488 (0.0010) [2023-12-27 08:25:57,217][20381] Updated weights for policy 1, policy_version 403498 (0.0010) [2023-12-27 08:25:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 206708736. Throughput: 0: 10404.2, 1: 10400.0. Samples: 206697224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:25:57,363][19170] Avg episode reward: [(0, '932.460'), (1, '952.080')] [2023-12-27 08:25:57,379][20347] Updated weights for policy 0, policy_version 403811 (0.0010) [2023-12-27 08:25:57,440][20347] Updated weights for policy 0, policy_version 403821 (0.0010) [2023-12-27 08:25:57,508][20347] Updated weights for policy 0, policy_version 403831 (0.0011) [2023-12-27 08:25:57,886][20381] Updated weights for policy 1, policy_version 403508 (0.0010) [2023-12-27 08:25:57,937][20381] Updated weights for policy 1, policy_version 403518 (0.0006) [2023-12-27 08:25:57,991][20381] Updated weights for policy 1, policy_version 403528 (0.0006) [2023-12-27 08:25:58,096][20347] Updated weights for policy 0, policy_version 403841 (0.0010) [2023-12-27 08:25:58,152][20347] Updated weights for policy 0, policy_version 403851 (0.0006) [2023-12-27 08:25:58,214][20347] Updated weights for policy 0, policy_version 403861 (0.0007) [2023-12-27 08:25:58,275][20347] Updated weights for policy 0, policy_version 403871 (0.0007) [2023-12-27 08:25:58,699][20381] Updated weights for policy 1, policy_version 403538 (0.0008) [2023-12-27 08:25:58,762][20381] Updated weights for policy 1, policy_version 403548 (0.0007) [2023-12-27 08:25:58,826][20381] Updated weights for policy 1, policy_version 403558 (0.0008) [2023-12-27 08:25:58,889][20381] Updated weights for policy 1, policy_version 403568 (0.0008) [2023-12-27 08:25:59,042][20347] Updated weights for policy 0, policy_version 403881 (0.0007) [2023-12-27 08:25:59,097][20347] Updated weights for policy 0, policy_version 403891 (0.0007) [2023-12-27 08:25:59,153][20347] Updated weights for policy 0, policy_version 403901 (0.0007) [2023-12-27 08:25:59,585][20381] Updated weights for policy 1, policy_version 403578 (0.0008) [2023-12-27 08:25:59,640][20381] Updated weights for policy 1, policy_version 403588 (0.0009) [2023-12-27 08:25:59,691][20381] Updated weights for policy 1, policy_version 403598 (0.0008) [2023-12-27 08:25:59,834][20347] Updated weights for policy 0, policy_version 403911 (0.0008) [2023-12-27 08:25:59,895][20347] Updated weights for policy 0, policy_version 403921 (0.0007) [2023-12-27 08:25:59,906][20291] KL-divergence is very high: 147.2010 [2023-12-27 08:25:59,956][20347] Updated weights for policy 0, policy_version 403931 (0.0008) [2023-12-27 08:26:00,359][20381] Updated weights for policy 1, policy_version 403608 (0.0008) [2023-12-27 08:26:00,408][20381] Updated weights for policy 1, policy_version 403618 (0.0008) [2023-12-27 08:26:00,455][20381] Updated weights for policy 1, policy_version 403628 (0.0007) [2023-12-27 08:26:00,631][20347] Updated weights for policy 0, policy_version 403941 (0.0007) [2023-12-27 08:26:00,685][20347] Updated weights for policy 0, policy_version 403951 (0.0007) [2023-12-27 08:26:00,742][20347] Updated weights for policy 0, policy_version 403961 (0.0007) [2023-12-27 08:26:01,151][20381] Updated weights for policy 1, policy_version 403638 (0.0006) [2023-12-27 08:26:01,198][20381] Updated weights for policy 1, policy_version 403648 (0.0006) [2023-12-27 08:26:01,243][20381] Updated weights for policy 1, policy_version 403658 (0.0006) [2023-12-27 08:26:01,391][20291] KL-divergence is very high: 125.5426 [2023-12-27 08:26:01,404][20347] Updated weights for policy 0, policy_version 403971 (0.0008) [2023-12-27 08:26:01,459][20347] Updated weights for policy 0, policy_version 403981 (0.0010) [2023-12-27 08:26:01,515][20347] Updated weights for policy 0, policy_version 403991 (0.0010) [2023-12-27 08:26:01,919][20381] Updated weights for policy 1, policy_version 403668 (0.0007) [2023-12-27 08:26:01,975][20381] Updated weights for policy 1, policy_version 403678 (0.0007) [2023-12-27 08:26:02,027][20381] Updated weights for policy 1, policy_version 403688 (0.0008) [2023-12-27 08:26:02,174][20347] Updated weights for policy 0, policy_version 404001 (0.0009) [2023-12-27 08:26:02,225][20347] Updated weights for policy 0, policy_version 404011 (0.0009) [2023-12-27 08:26:02,280][20347] Updated weights for policy 0, policy_version 404021 (0.0009) [2023-12-27 08:26:02,336][20347] Updated weights for policy 0, policy_version 404031 (0.0009) [2023-12-27 08:26:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 206815232. Throughput: 0: 10344.2, 1: 10338.9. Samples: 206820504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:26:02,363][19170] Avg episode reward: [(0, '839.410'), (1, '961.640')] [2023-12-27 08:26:02,752][20381] Updated weights for policy 1, policy_version 403698 (0.0008) [2023-12-27 08:26:02,808][20381] Updated weights for policy 1, policy_version 403708 (0.0007) [2023-12-27 08:26:02,867][20381] Updated weights for policy 1, policy_version 403718 (0.0008) [2023-12-27 08:26:02,928][20381] Updated weights for policy 1, policy_version 403728 (0.0008) [2023-12-27 08:26:03,011][20347] Updated weights for policy 0, policy_version 404041 (0.0009) [2023-12-27 08:26:03,065][20347] Updated weights for policy 0, policy_version 404051 (0.0010) [2023-12-27 08:26:03,119][20347] Updated weights for policy 0, policy_version 404061 (0.0010) [2023-12-27 08:26:03,607][20381] Updated weights for policy 1, policy_version 403738 (0.0007) [2023-12-27 08:26:03,658][20381] Updated weights for policy 1, policy_version 403748 (0.0007) [2023-12-27 08:26:03,717][20381] Updated weights for policy 1, policy_version 403758 (0.0008) [2023-12-27 08:26:03,782][20347] Updated weights for policy 0, policy_version 404071 (0.0008) [2023-12-27 08:26:03,836][20347] Updated weights for policy 0, policy_version 404081 (0.0006) [2023-12-27 08:26:03,903][20347] Updated weights for policy 0, policy_version 404091 (0.0006) [2023-12-27 08:26:04,436][20381] Updated weights for policy 1, policy_version 403768 (0.0008) [2023-12-27 08:26:04,497][20381] Updated weights for policy 1, policy_version 403778 (0.0008) [2023-12-27 08:26:04,543][20347] Updated weights for policy 0, policy_version 404101 (0.0006) [2023-12-27 08:26:04,556][20381] Updated weights for policy 1, policy_version 403788 (0.0007) [2023-12-27 08:26:04,598][20347] Updated weights for policy 0, policy_version 404111 (0.0006) [2023-12-27 08:26:04,654][20347] Updated weights for policy 0, policy_version 404121 (0.0008) [2023-12-27 08:26:05,295][20381] Updated weights for policy 1, policy_version 403798 (0.0008) [2023-12-27 08:26:05,315][20347] Updated weights for policy 0, policy_version 404131 (0.0007) [2023-12-27 08:26:05,351][20381] Updated weights for policy 1, policy_version 403808 (0.0007) [2023-12-27 08:26:05,363][20347] Updated weights for policy 0, policy_version 404141 (0.0005) [2023-12-27 08:26:05,407][20381] Updated weights for policy 1, policy_version 403818 (0.0007) [2023-12-27 08:26:05,418][20347] Updated weights for policy 0, policy_version 404151 (0.0006) [2023-12-27 08:26:06,014][20347] Updated weights for policy 0, policy_version 404161 (0.0007) [2023-12-27 08:26:06,069][20347] Updated weights for policy 0, policy_version 404171 (0.0005) [2023-12-27 08:26:06,118][20381] Updated weights for policy 1, policy_version 403828 (0.0008) [2023-12-27 08:26:06,131][20347] Updated weights for policy 0, policy_version 404181 (0.0007) [2023-12-27 08:26:06,179][20381] Updated weights for policy 1, policy_version 403838 (0.0007) [2023-12-27 08:26:06,196][20347] Updated weights for policy 0, policy_version 404191 (0.0006) [2023-12-27 08:26:06,235][20381] Updated weights for policy 1, policy_version 403848 (0.0007) [2023-12-27 08:26:06,882][20381] Updated weights for policy 1, policy_version 403858 (0.0008) [2023-12-27 08:26:06,887][20347] Updated weights for policy 0, policy_version 404201 (0.0007) [2023-12-27 08:26:06,895][20291] KL-divergence is very high: 125.7788 [2023-12-27 08:26:06,940][20381] Updated weights for policy 1, policy_version 403868 (0.0006) [2023-12-27 08:26:06,947][20347] Updated weights for policy 0, policy_version 404211 (0.0010) [2023-12-27 08:26:06,994][20381] Updated weights for policy 1, policy_version 403878 (0.0006) [2023-12-27 08:26:07,010][20347] Updated weights for policy 0, policy_version 404221 (0.0010) [2023-12-27 08:26:07,052][20381] Updated weights for policy 1, policy_version 403888 (0.0005) [2023-12-27 08:26:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 206913536. Throughput: 0: 10331.4, 1: 10318.6. Samples: 206881128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:26:07,363][19170] Avg episode reward: [(0, '669.940'), (1, '981.240')] [2023-12-27 08:26:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000403888_103415808.pth... [2023-12-27 08:26:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000404224_103497728.pth... [2023-12-27 08:26:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000402704_103112704.pth [2023-12-27 08:26:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000403040_103194624.pth [2023-12-27 08:26:07,678][20347] Updated weights for policy 0, policy_version 404231 (0.0009) [2023-12-27 08:26:07,685][20381] Updated weights for policy 1, policy_version 403898 (0.0005) [2023-12-27 08:26:07,734][20347] Updated weights for policy 0, policy_version 404241 (0.0007) [2023-12-27 08:26:07,743][20381] Updated weights for policy 1, policy_version 403908 (0.0006) [2023-12-27 08:26:07,791][20347] Updated weights for policy 0, policy_version 404251 (0.0007) [2023-12-27 08:26:07,805][20381] Updated weights for policy 1, policy_version 403918 (0.0006) [2023-12-27 08:26:08,419][20347] Updated weights for policy 0, policy_version 404261 (0.0008) [2023-12-27 08:26:08,486][20347] Updated weights for policy 0, policy_version 404271 (0.0007) [2023-12-27 08:26:08,511][20381] Updated weights for policy 1, policy_version 403928 (0.0006) [2023-12-27 08:26:08,549][20347] Updated weights for policy 0, policy_version 404281 (0.0009) [2023-12-27 08:26:08,567][20381] Updated weights for policy 1, policy_version 403938 (0.0008) [2023-12-27 08:26:08,621][20381] Updated weights for policy 1, policy_version 403948 (0.0007) [2023-12-27 08:26:09,216][20347] Updated weights for policy 0, policy_version 404291 (0.0007) [2023-12-27 08:26:09,282][20347] Updated weights for policy 0, policy_version 404301 (0.0008) [2023-12-27 08:26:09,346][20347] Updated weights for policy 0, policy_version 404311 (0.0008) [2023-12-27 08:26:09,378][20381] Updated weights for policy 1, policy_version 403958 (0.0007) [2023-12-27 08:26:09,448][20381] Updated weights for policy 1, policy_version 403968 (0.0008) [2023-12-27 08:26:09,517][20381] Updated weights for policy 1, policy_version 403978 (0.0008) [2023-12-27 08:26:09,982][20347] Updated weights for policy 0, policy_version 404321 (0.0008) [2023-12-27 08:26:10,044][20347] Updated weights for policy 0, policy_version 404331 (0.0008) [2023-12-27 08:26:10,104][20347] Updated weights for policy 0, policy_version 404341 (0.0008) [2023-12-27 08:26:10,167][20347] Updated weights for policy 0, policy_version 404351 (0.0008) [2023-12-27 08:26:10,235][20381] Updated weights for policy 1, policy_version 403988 (0.0009) [2023-12-27 08:26:10,291][20381] Updated weights for policy 1, policy_version 403998 (0.0007) [2023-12-27 08:26:10,351][20381] Updated weights for policy 1, policy_version 404008 (0.0007) [2023-12-27 08:26:10,876][20347] Updated weights for policy 0, policy_version 404361 (0.0007) [2023-12-27 08:26:10,928][20347] Updated weights for policy 0, policy_version 404371 (0.0008) [2023-12-27 08:26:10,978][20347] Updated weights for policy 0, policy_version 404381 (0.0008) [2023-12-27 08:26:11,047][20381] Updated weights for policy 1, policy_version 404018 (0.0008) [2023-12-27 08:26:11,100][20381] Updated weights for policy 1, policy_version 404028 (0.0008) [2023-12-27 08:26:11,167][20381] Updated weights for policy 1, policy_version 404038 (0.0008) [2023-12-27 08:26:11,224][20381] Updated weights for policy 1, policy_version 404048 (0.0008) [2023-12-27 08:26:11,690][20347] Updated weights for policy 0, policy_version 404391 (0.0008) [2023-12-27 08:26:11,756][20347] Updated weights for policy 0, policy_version 404401 (0.0008) [2023-12-27 08:26:11,815][20347] Updated weights for policy 0, policy_version 404411 (0.0007) [2023-12-27 08:26:12,041][20381] Updated weights for policy 1, policy_version 404058 (0.0008) [2023-12-27 08:26:12,091][20381] Updated weights for policy 1, policy_version 404068 (0.0008) [2023-12-27 08:26:12,143][20381] Updated weights for policy 1, policy_version 404078 (0.0008) [2023-12-27 08:26:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 207011840. Throughput: 0: 10341.7, 1: 10262.7. Samples: 207002352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:26:12,363][19170] Avg episode reward: [(0, '782.050'), (1, '971.380')] [2023-12-27 08:26:12,504][20347] Updated weights for policy 0, policy_version 404421 (0.0008) [2023-12-27 08:26:12,551][20347] Updated weights for policy 0, policy_version 404431 (0.0008) [2023-12-27 08:26:12,592][20347] Updated weights for policy 0, policy_version 404441 (0.0007) [2023-12-27 08:26:12,875][20381] Updated weights for policy 1, policy_version 404088 (0.0006) [2023-12-27 08:26:12,930][20381] Updated weights for policy 1, policy_version 404098 (0.0007) [2023-12-27 08:26:12,980][20381] Updated weights for policy 1, policy_version 404108 (0.0005) [2023-12-27 08:26:13,316][20347] Updated weights for policy 0, policy_version 404451 (0.0006) [2023-12-27 08:26:13,377][20347] Updated weights for policy 0, policy_version 404461 (0.0005) [2023-12-27 08:26:13,430][20347] Updated weights for policy 0, policy_version 404471 (0.0008) [2023-12-27 08:26:13,622][20381] Updated weights for policy 1, policy_version 404118 (0.0007) [2023-12-27 08:26:13,668][20381] Updated weights for policy 1, policy_version 404128 (0.0008) [2023-12-27 08:26:13,719][20381] Updated weights for policy 1, policy_version 404138 (0.0009) [2023-12-27 08:26:14,030][20347] Updated weights for policy 0, policy_version 404481 (0.0007) [2023-12-27 08:26:14,088][20347] Updated weights for policy 0, policy_version 404491 (0.0010) [2023-12-27 08:26:14,149][20347] Updated weights for policy 0, policy_version 404501 (0.0010) [2023-12-27 08:26:14,209][20347] Updated weights for policy 0, policy_version 404511 (0.0009) [2023-12-27 08:26:14,474][20381] Updated weights for policy 1, policy_version 404148 (0.0008) [2023-12-27 08:26:14,527][20381] Updated weights for policy 1, policy_version 404158 (0.0008) [2023-12-27 08:26:14,584][20381] Updated weights for policy 1, policy_version 404168 (0.0008) [2023-12-27 08:26:14,816][20347] Updated weights for policy 0, policy_version 404521 (0.0006) [2023-12-27 08:26:14,875][20347] Updated weights for policy 0, policy_version 404531 (0.0006) [2023-12-27 08:26:14,937][20347] Updated weights for policy 0, policy_version 404541 (0.0011) [2023-12-27 08:26:15,321][20381] Updated weights for policy 1, policy_version 404178 (0.0008) [2023-12-27 08:26:15,381][20381] Updated weights for policy 1, policy_version 404188 (0.0005) [2023-12-27 08:26:15,433][20381] Updated weights for policy 1, policy_version 404198 (0.0005) [2023-12-27 08:26:15,484][20381] Updated weights for policy 1, policy_version 404208 (0.0005) [2023-12-27 08:26:15,592][20347] Updated weights for policy 0, policy_version 404551 (0.0010) [2023-12-27 08:26:15,643][20347] Updated weights for policy 0, policy_version 404561 (0.0010) [2023-12-27 08:26:15,688][20347] Updated weights for policy 0, policy_version 404571 (0.0010) [2023-12-27 08:26:16,055][20381] Updated weights for policy 1, policy_version 404218 (0.0008) [2023-12-27 08:26:16,115][20381] Updated weights for policy 1, policy_version 404228 (0.0008) [2023-12-27 08:26:16,171][20381] Updated weights for policy 1, policy_version 404238 (0.0008) [2023-12-27 08:26:16,336][20347] Updated weights for policy 0, policy_version 404581 (0.0009) [2023-12-27 08:26:16,391][20347] Updated weights for policy 0, policy_version 404591 (0.0008) [2023-12-27 08:26:16,447][20347] Updated weights for policy 0, policy_version 404601 (0.0005) [2023-12-27 08:26:16,858][20381] Updated weights for policy 1, policy_version 404248 (0.0009) [2023-12-27 08:26:16,919][20381] Updated weights for policy 1, policy_version 404258 (0.0011) [2023-12-27 08:26:16,976][20381] Updated weights for policy 1, policy_version 404268 (0.0011) [2023-12-27 08:26:17,050][20347] Updated weights for policy 0, policy_version 404611 (0.0006) [2023-12-27 08:26:17,102][20347] Updated weights for policy 0, policy_version 404621 (0.0007) [2023-12-27 08:26:17,154][20347] Updated weights for policy 0, policy_version 404631 (0.0007) [2023-12-27 08:26:17,178][20291] KL-divergence is very high: 118.7153 [2023-12-27 08:26:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 207118336. Throughput: 0: 10396.5, 1: 10146.1. Samples: 207126604. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:17,363][19170] Avg episode reward: [(0, '700.810'), (1, '961.450')] [2023-12-27 08:26:17,661][20381] Updated weights for policy 1, policy_version 404278 (0.0011) [2023-12-27 08:26:17,709][20381] Updated weights for policy 1, policy_version 404288 (0.0010) [2023-12-27 08:26:17,755][20381] Updated weights for policy 1, policy_version 404298 (0.0010) [2023-12-27 08:26:17,803][20347] Updated weights for policy 0, policy_version 404641 (0.0007) [2023-12-27 08:26:17,859][20347] Updated weights for policy 0, policy_version 404651 (0.0008) [2023-12-27 08:26:17,866][20291] KL-divergence is very high: 168.9579 [2023-12-27 08:26:17,893][20291] KL-divergence is very high: 104.0096 [2023-12-27 08:26:17,904][20291] KL-divergence is very high: 108.0534 [2023-12-27 08:26:17,909][20291] KL-divergence is very high: 150.9411 [2023-12-27 08:26:17,914][20347] Updated weights for policy 0, policy_version 404661 (0.0005) [2023-12-27 08:26:17,962][20347] Updated weights for policy 0, policy_version 404671 (0.0006) [2023-12-27 08:26:18,420][20381] Updated weights for policy 1, policy_version 404308 (0.0011) [2023-12-27 08:26:18,480][20381] Updated weights for policy 1, policy_version 404318 (0.0009) [2023-12-27 08:26:18,537][20381] Updated weights for policy 1, policy_version 404328 (0.0010) [2023-12-27 08:26:18,592][20347] Updated weights for policy 0, policy_version 404681 (0.0006) [2023-12-27 08:26:18,660][20347] Updated weights for policy 0, policy_version 404691 (0.0006) [2023-12-27 08:26:18,723][20347] Updated weights for policy 0, policy_version 404701 (0.0008) [2023-12-27 08:26:19,238][20381] Updated weights for policy 1, policy_version 404338 (0.0010) [2023-12-27 08:26:19,299][20381] Updated weights for policy 1, policy_version 404348 (0.0008) [2023-12-27 08:26:19,362][20381] Updated weights for policy 1, policy_version 404358 (0.0007) [2023-12-27 08:26:19,375][20347] Updated weights for policy 0, policy_version 404711 (0.0006) [2023-12-27 08:26:19,421][20381] Updated weights for policy 1, policy_version 404368 (0.0007) [2023-12-27 08:26:19,440][20347] Updated weights for policy 0, policy_version 404721 (0.0008) [2023-12-27 08:26:19,508][20347] Updated weights for policy 0, policy_version 404731 (0.0008) [2023-12-27 08:26:20,138][20381] Updated weights for policy 1, policy_version 404378 (0.0008) [2023-12-27 08:26:20,166][20347] Updated weights for policy 0, policy_version 404741 (0.0008) [2023-12-27 08:26:20,207][20381] Updated weights for policy 1, policy_version 404388 (0.0007) [2023-12-27 08:26:20,228][20347] Updated weights for policy 0, policy_version 404751 (0.0007) [2023-12-27 08:26:20,274][20381] Updated weights for policy 1, policy_version 404398 (0.0007) [2023-12-27 08:26:20,293][20347] Updated weights for policy 0, policy_version 404761 (0.0007) [2023-12-27 08:26:20,896][20381] Updated weights for policy 1, policy_version 404408 (0.0010) [2023-12-27 08:26:20,921][20347] Updated weights for policy 0, policy_version 404771 (0.0007) [2023-12-27 08:26:20,952][20381] Updated weights for policy 1, policy_version 404418 (0.0010) [2023-12-27 08:26:20,973][20347] Updated weights for policy 0, policy_version 404781 (0.0007) [2023-12-27 08:26:21,012][20381] Updated weights for policy 1, policy_version 404428 (0.0007) [2023-12-27 08:26:21,033][20347] Updated weights for policy 0, policy_version 404791 (0.0006) [2023-12-27 08:26:21,723][20381] Updated weights for policy 1, policy_version 404438 (0.0010) [2023-12-27 08:26:21,789][20381] Updated weights for policy 1, policy_version 404448 (0.0009) [2023-12-27 08:26:21,840][20347] Updated weights for policy 0, policy_version 404801 (0.0008) [2023-12-27 08:26:21,855][20381] Updated weights for policy 1, policy_version 404458 (0.0011) [2023-12-27 08:26:21,903][20347] Updated weights for policy 0, policy_version 404811 (0.0008) [2023-12-27 08:26:21,968][20347] Updated weights for policy 0, policy_version 404821 (0.0008) [2023-12-27 08:26:22,031][20347] Updated weights for policy 0, policy_version 404831 (0.0008) [2023-12-27 08:26:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 207216640. Throughput: 0: 10396.1, 1: 10130.7. Samples: 207188144. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:22,363][19170] Avg episode reward: [(0, '694.880'), (1, '961.800')] [2023-12-27 08:26:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000404464_103563264.pth... [2023-12-27 08:26:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000404832_103653376.pth... [2023-12-27 08:26:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000403280_103260160.pth [2023-12-27 08:26:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000403616_103342080.pth [2023-12-27 08:26:22,528][20381] Updated weights for policy 1, policy_version 404468 (0.0011) [2023-12-27 08:26:22,581][20381] Updated weights for policy 1, policy_version 404478 (0.0011) [2023-12-27 08:26:22,639][20381] Updated weights for policy 1, policy_version 404488 (0.0010) [2023-12-27 08:26:22,720][20347] Updated weights for policy 0, policy_version 404841 (0.0007) [2023-12-27 08:26:22,782][20347] Updated weights for policy 0, policy_version 404851 (0.0007) [2023-12-27 08:26:22,828][20347] Updated weights for policy 0, policy_version 404861 (0.0007) [2023-12-27 08:26:23,309][20381] Updated weights for policy 1, policy_version 404498 (0.0010) [2023-12-27 08:26:23,357][20381] Updated weights for policy 1, policy_version 404508 (0.0006) [2023-12-27 08:26:23,406][20381] Updated weights for policy 1, policy_version 404518 (0.0011) [2023-12-27 08:26:23,451][20347] Updated weights for policy 0, policy_version 404871 (0.0006) [2023-12-27 08:26:23,458][20381] Updated weights for policy 1, policy_version 404528 (0.0010) [2023-12-27 08:26:23,505][20347] Updated weights for policy 0, policy_version 404881 (0.0006) [2023-12-27 08:26:23,554][20347] Updated weights for policy 0, policy_version 404891 (0.0005) [2023-12-27 08:26:24,163][20347] Updated weights for policy 0, policy_version 404901 (0.0005) [2023-12-27 08:26:24,166][20381] Updated weights for policy 1, policy_version 404538 (0.0011) [2023-12-27 08:26:24,218][20381] Updated weights for policy 1, policy_version 404548 (0.0011) [2023-12-27 08:26:24,225][20347] Updated weights for policy 0, policy_version 404911 (0.0009) [2023-12-27 08:26:24,267][20381] Updated weights for policy 1, policy_version 404558 (0.0010) [2023-12-27 08:26:24,279][20347] Updated weights for policy 0, policy_version 404921 (0.0010) [2023-12-27 08:26:24,940][20347] Updated weights for policy 0, policy_version 404931 (0.0009) [2023-12-27 08:26:24,981][20381] Updated weights for policy 1, policy_version 404568 (0.0011) [2023-12-27 08:26:24,996][20347] Updated weights for policy 0, policy_version 404941 (0.0007) [2023-12-27 08:26:25,040][20381] Updated weights for policy 1, policy_version 404578 (0.0011) [2023-12-27 08:26:25,054][20347] Updated weights for policy 0, policy_version 404951 (0.0008) [2023-12-27 08:26:25,084][20381] Updated weights for policy 1, policy_version 404588 (0.0010) [2023-12-27 08:26:25,639][20347] Updated weights for policy 0, policy_version 404961 (0.0006) [2023-12-27 08:26:25,693][20347] Updated weights for policy 0, policy_version 404971 (0.0008) [2023-12-27 08:26:25,743][20381] Updated weights for policy 1, policy_version 404598 (0.0010) [2023-12-27 08:26:25,747][20347] Updated weights for policy 0, policy_version 404981 (0.0005) [2023-12-27 08:26:25,794][20347] Updated weights for policy 0, policy_version 404991 (0.0005) [2023-12-27 08:26:25,795][20381] Updated weights for policy 1, policy_version 404608 (0.0010) [2023-12-27 08:26:25,847][20381] Updated weights for policy 1, policy_version 404618 (0.0011) [2023-12-27 08:26:26,357][20347] Updated weights for policy 0, policy_version 405001 (0.0007) [2023-12-27 08:26:26,412][20347] Updated weights for policy 0, policy_version 405011 (0.0007) [2023-12-27 08:26:26,465][20347] Updated weights for policy 0, policy_version 405021 (0.0005) [2023-12-27 08:26:26,554][20381] Updated weights for policy 1, policy_version 404628 (0.0010) [2023-12-27 08:26:26,604][20381] Updated weights for policy 1, policy_version 404638 (0.0010) [2023-12-27 08:26:26,658][20381] Updated weights for policy 1, policy_version 404648 (0.0010) [2023-12-27 08:26:27,110][20347] Updated weights for policy 0, policy_version 405031 (0.0007) [2023-12-27 08:26:27,160][20347] Updated weights for policy 0, policy_version 405041 (0.0008) [2023-12-27 08:26:27,216][20347] Updated weights for policy 0, policy_version 405051 (0.0007) [2023-12-27 08:26:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 207323136. Throughput: 0: 10393.8, 1: 10117.2. Samples: 207312416. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:27,363][19170] Avg episode reward: [(0, '663.680'), (1, '953.030')] [2023-12-27 08:26:27,384][20381] Updated weights for policy 1, policy_version 404658 (0.0010) [2023-12-27 08:26:27,432][20381] Updated weights for policy 1, policy_version 404668 (0.0010) [2023-12-27 08:26:27,481][20381] Updated weights for policy 1, policy_version 404678 (0.0010) [2023-12-27 08:26:27,533][20381] Updated weights for policy 1, policy_version 404688 (0.0011) [2023-12-27 08:26:27,844][20347] Updated weights for policy 0, policy_version 405061 (0.0005) [2023-12-27 08:26:27,859][20291] KL-divergence is very high: 107.6682 [2023-12-27 08:26:27,892][20347] Updated weights for policy 0, policy_version 405071 (0.0005) [2023-12-27 08:26:27,944][20347] Updated weights for policy 0, policy_version 405081 (0.0005) [2023-12-27 08:26:28,202][20381] Updated weights for policy 1, policy_version 404698 (0.0005) [2023-12-27 08:26:28,257][20381] Updated weights for policy 1, policy_version 404708 (0.0005) [2023-12-27 08:26:28,303][20381] Updated weights for policy 1, policy_version 404718 (0.0007) [2023-12-27 08:26:28,556][20347] Updated weights for policy 0, policy_version 405091 (0.0006) [2023-12-27 08:26:28,602][20291] KL-divergence is very high: 167.7206 [2023-12-27 08:26:28,613][20347] Updated weights for policy 0, policy_version 405101 (0.0008) [2023-12-27 08:26:28,670][20347] Updated weights for policy 0, policy_version 405111 (0.0007) [2023-12-27 08:26:28,860][20381] Updated weights for policy 1, policy_version 404728 (0.0010) [2023-12-27 08:26:28,920][20381] Updated weights for policy 1, policy_version 404738 (0.0011) [2023-12-27 08:26:28,977][20381] Updated weights for policy 1, policy_version 404748 (0.0011) [2023-12-27 08:26:29,400][20347] Updated weights for policy 0, policy_version 405121 (0.0007) [2023-12-27 08:26:29,461][20347] Updated weights for policy 0, policy_version 405131 (0.0008) [2023-12-27 08:26:29,514][20347] Updated weights for policy 0, policy_version 405141 (0.0007) [2023-12-27 08:26:29,579][20347] Updated weights for policy 0, policy_version 405151 (0.0007) [2023-12-27 08:26:29,687][20381] Updated weights for policy 1, policy_version 404758 (0.0011) [2023-12-27 08:26:29,739][20381] Updated weights for policy 1, policy_version 404768 (0.0010) [2023-12-27 08:26:29,787][20381] Updated weights for policy 1, policy_version 404778 (0.0011) [2023-12-27 08:26:30,197][20347] Updated weights for policy 0, policy_version 405161 (0.0010) [2023-12-27 08:26:30,248][20347] Updated weights for policy 0, policy_version 405171 (0.0010) [2023-12-27 08:26:30,303][20347] Updated weights for policy 0, policy_version 405181 (0.0010) [2023-12-27 08:26:30,513][20381] Updated weights for policy 1, policy_version 404788 (0.0009) [2023-12-27 08:26:30,569][20381] Updated weights for policy 1, policy_version 404798 (0.0011) [2023-12-27 08:26:30,632][20381] Updated weights for policy 1, policy_version 404808 (0.0010) [2023-12-27 08:26:30,927][20347] Updated weights for policy 0, policy_version 405191 (0.0008) [2023-12-27 08:26:30,985][20347] Updated weights for policy 0, policy_version 405201 (0.0007) [2023-12-27 08:26:31,046][20347] Updated weights for policy 0, policy_version 405211 (0.0009) [2023-12-27 08:26:31,301][20381] Updated weights for policy 1, policy_version 404818 (0.0009) [2023-12-27 08:26:31,366][20381] Updated weights for policy 1, policy_version 404828 (0.0010) [2023-12-27 08:26:31,425][20381] Updated weights for policy 1, policy_version 404838 (0.0007) [2023-12-27 08:26:31,476][20381] Updated weights for policy 1, policy_version 404848 (0.0008) [2023-12-27 08:26:31,778][20347] Updated weights for policy 0, policy_version 405221 (0.0008) [2023-12-27 08:26:31,839][20347] Updated weights for policy 0, policy_version 405231 (0.0007) [2023-12-27 08:26:31,891][20347] Updated weights for policy 0, policy_version 405241 (0.0008) [2023-12-27 08:26:32,170][20381] Updated weights for policy 1, policy_version 404858 (0.0008) [2023-12-27 08:26:32,244][20381] Updated weights for policy 1, policy_version 404868 (0.0009) [2023-12-27 08:26:32,303][20381] Updated weights for policy 1, policy_version 404878 (0.0008) [2023-12-27 08:26:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 207429632. Throughput: 0: 10486.7, 1: 10131.8. Samples: 207437948. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:32,363][19170] Avg episode reward: [(0, '696.040'), (1, '971.960')] [2023-12-27 08:26:32,533][20347] Updated weights for policy 0, policy_version 405251 (0.0007) [2023-12-27 08:26:32,581][20347] Updated weights for policy 0, policy_version 405261 (0.0010) [2023-12-27 08:26:32,639][20347] Updated weights for policy 0, policy_version 405271 (0.0010) [2023-12-27 08:26:33,028][20381] Updated weights for policy 1, policy_version 404888 (0.0006) [2023-12-27 08:26:33,082][20381] Updated weights for policy 1, policy_version 404898 (0.0005) [2023-12-27 08:26:33,125][20381] Updated weights for policy 1, policy_version 404908 (0.0005) [2023-12-27 08:26:33,231][20347] Updated weights for policy 0, policy_version 405281 (0.0007) [2023-12-27 08:26:33,291][20347] Updated weights for policy 0, policy_version 405291 (0.0010) [2023-12-27 08:26:33,351][20347] Updated weights for policy 0, policy_version 405301 (0.0010) [2023-12-27 08:26:33,405][20347] Updated weights for policy 0, policy_version 405311 (0.0010) [2023-12-27 08:26:33,748][20381] Updated weights for policy 1, policy_version 404918 (0.0006) [2023-12-27 08:26:33,791][20381] Updated weights for policy 1, policy_version 404928 (0.0007) [2023-12-27 08:26:33,834][20381] Updated weights for policy 1, policy_version 404938 (0.0007) [2023-12-27 08:26:34,029][20347] Updated weights for policy 0, policy_version 405321 (0.0006) [2023-12-27 08:26:34,087][20347] Updated weights for policy 0, policy_version 405331 (0.0005) [2023-12-27 08:26:34,141][20347] Updated weights for policy 0, policy_version 405341 (0.0005) [2023-12-27 08:26:34,530][20381] Updated weights for policy 1, policy_version 404948 (0.0008) [2023-12-27 08:26:34,574][20292] KL-divergence is very high: 162.9476 [2023-12-27 08:26:34,579][20292] KL-divergence is very high: 133.9875 [2023-12-27 08:26:34,584][20292] KL-divergence is very high: 147.8246 [2023-12-27 08:26:34,589][20381] Updated weights for policy 1, policy_version 404958 (0.0008) [2023-12-27 08:26:34,619][20292] KL-divergence is very high: 112.6017 [2023-12-27 08:26:34,649][20381] Updated weights for policy 1, policy_version 404968 (0.0008) [2023-12-27 08:26:34,773][20347] Updated weights for policy 0, policy_version 405351 (0.0009) [2023-12-27 08:26:34,828][20347] Updated weights for policy 0, policy_version 405361 (0.0010) [2023-12-27 08:26:34,889][20347] Updated weights for policy 0, policy_version 405371 (0.0010) [2023-12-27 08:26:35,368][20292] KL-divergence is very high: 130.9255 [2023-12-27 08:26:35,377][20381] Updated weights for policy 1, policy_version 404978 (0.0007) [2023-12-27 08:26:35,409][20292] KL-divergence is very high: 107.6832 [2023-12-27 08:26:35,431][20381] Updated weights for policy 1, policy_version 404988 (0.0007) [2023-12-27 08:26:35,455][20292] KL-divergence is very high: 171.2113 [2023-12-27 08:26:35,485][20381] Updated weights for policy 1, policy_version 404998 (0.0008) [2023-12-27 08:26:35,527][20347] Updated weights for policy 0, policy_version 405381 (0.0010) [2023-12-27 08:26:35,535][20381] Updated weights for policy 1, policy_version 405008 (0.0006) [2023-12-27 08:26:35,581][20347] Updated weights for policy 0, policy_version 405391 (0.0010) [2023-12-27 08:26:35,630][20347] Updated weights for policy 0, policy_version 405401 (0.0010) [2023-12-27 08:26:36,246][20381] Updated weights for policy 1, policy_version 405018 (0.0007) [2023-12-27 08:26:36,310][20381] Updated weights for policy 1, policy_version 405028 (0.0008) [2023-12-27 08:26:36,379][20381] Updated weights for policy 1, policy_version 405038 (0.0008) [2023-12-27 08:26:36,380][20347] Updated weights for policy 0, policy_version 405411 (0.0009) [2023-12-27 08:26:36,432][20347] Updated weights for policy 0, policy_version 405421 (0.0007) [2023-12-27 08:26:36,488][20347] Updated weights for policy 0, policy_version 405431 (0.0010) [2023-12-27 08:26:37,086][20381] Updated weights for policy 1, policy_version 405048 (0.0009) [2023-12-27 08:26:37,099][20347] Updated weights for policy 0, policy_version 405441 (0.0010) [2023-12-27 08:26:37,147][20381] Updated weights for policy 1, policy_version 405058 (0.0009) [2023-12-27 08:26:37,155][20347] Updated weights for policy 0, policy_version 405451 (0.0005) [2023-12-27 08:26:37,202][20381] Updated weights for policy 1, policy_version 405068 (0.0007) [2023-12-27 08:26:37,211][20347] Updated weights for policy 0, policy_version 405461 (0.0009) [2023-12-27 08:26:37,274][20347] Updated weights for policy 0, policy_version 405471 (0.0010) [2023-12-27 08:26:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 207536128. Throughput: 0: 10533.4, 1: 10136.3. Samples: 207501076. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:37,363][19170] Avg episode reward: [(0, '645.500'), (1, '935.720')] [2023-12-27 08:26:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000405472_103817216.pth... [2023-12-27 08:26:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000405072_103718912.pth... [2023-12-27 08:26:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000404224_103497728.pth [2023-12-27 08:26:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000403888_103415808.pth [2023-12-27 08:26:37,875][20347] Updated weights for policy 0, policy_version 405481 (0.0010) [2023-12-27 08:26:37,924][20381] Updated weights for policy 1, policy_version 405078 (0.0006) [2023-12-27 08:26:37,936][20347] Updated weights for policy 0, policy_version 405491 (0.0010) [2023-12-27 08:26:37,969][20381] Updated weights for policy 1, policy_version 405088 (0.0007) [2023-12-27 08:26:37,991][20347] Updated weights for policy 0, policy_version 405501 (0.0010) [2023-12-27 08:26:38,020][20381] Updated weights for policy 1, policy_version 405098 (0.0008) [2023-12-27 08:26:38,633][20347] Updated weights for policy 0, policy_version 405511 (0.0009) [2023-12-27 08:26:38,691][20347] Updated weights for policy 0, policy_version 405521 (0.0010) [2023-12-27 08:26:38,734][20381] Updated weights for policy 1, policy_version 405108 (0.0008) [2023-12-27 08:26:38,743][20347] Updated weights for policy 0, policy_version 405531 (0.0010) [2023-12-27 08:26:38,787][20381] Updated weights for policy 1, policy_version 405118 (0.0006) [2023-12-27 08:26:38,846][20381] Updated weights for policy 1, policy_version 405128 (0.0007) [2023-12-27 08:26:39,467][20347] Updated weights for policy 0, policy_version 405541 (0.0009) [2023-12-27 08:26:39,535][20347] Updated weights for policy 0, policy_version 405551 (0.0010) [2023-12-27 08:26:39,557][20381] Updated weights for policy 1, policy_version 405138 (0.0008) [2023-12-27 08:26:39,603][20347] Updated weights for policy 0, policy_version 405561 (0.0009) [2023-12-27 08:26:39,620][20381] Updated weights for policy 1, policy_version 405148 (0.0009) [2023-12-27 08:26:39,684][20381] Updated weights for policy 1, policy_version 405158 (0.0008) [2023-12-27 08:26:39,742][20381] Updated weights for policy 1, policy_version 405168 (0.0007) [2023-12-27 08:26:40,309][20347] Updated weights for policy 0, policy_version 405571 (0.0008) [2023-12-27 08:26:40,368][20347] Updated weights for policy 0, policy_version 405581 (0.0006) [2023-12-27 08:26:40,428][20347] Updated weights for policy 0, policy_version 405591 (0.0006) [2023-12-27 08:26:40,469][20381] Updated weights for policy 1, policy_version 405178 (0.0008) [2023-12-27 08:26:40,518][20292] KL-divergence is very high: 177.7302 [2023-12-27 08:26:40,529][20381] Updated weights for policy 1, policy_version 405188 (0.0008) [2023-12-27 08:26:40,590][20381] Updated weights for policy 1, policy_version 405198 (0.0008) [2023-12-27 08:26:41,030][20347] Updated weights for policy 0, policy_version 405601 (0.0006) [2023-12-27 08:26:41,094][20347] Updated weights for policy 0, policy_version 405611 (0.0008) [2023-12-27 08:26:41,165][20347] Updated weights for policy 0, policy_version 405621 (0.0007) [2023-12-27 08:26:41,215][20347] Updated weights for policy 0, policy_version 405631 (0.0007) [2023-12-27 08:26:41,300][20292] KL-divergence is very high: 108.7579 [2023-12-27 08:26:41,309][20381] Updated weights for policy 1, policy_version 405208 (0.0008) [2023-12-27 08:26:41,369][20381] Updated weights for policy 1, policy_version 405218 (0.0008) [2023-12-27 08:26:41,426][20292] KL-divergence is very high: 101.6844 [2023-12-27 08:26:41,432][20381] Updated weights for policy 1, policy_version 405228 (0.0008) [2023-12-27 08:26:41,911][20347] Updated weights for policy 0, policy_version 405641 (0.0008) [2023-12-27 08:26:41,973][20347] Updated weights for policy 0, policy_version 405651 (0.0008) [2023-12-27 08:26:42,024][20347] Updated weights for policy 0, policy_version 405661 (0.0007) [2023-12-27 08:26:42,176][20381] Updated weights for policy 1, policy_version 405238 (0.0008) [2023-12-27 08:26:42,229][20381] Updated weights for policy 1, policy_version 405248 (0.0008) [2023-12-27 08:26:42,292][20381] Updated weights for policy 1, policy_version 405258 (0.0008) [2023-12-27 08:26:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 207634432. Throughput: 0: 10544.2, 1: 10004.6. Samples: 207621920. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:42,363][19170] Avg episode reward: [(0, '369.340'), (1, '858.210')] [2023-12-27 08:26:42,729][20347] Updated weights for policy 0, policy_version 405671 (0.0006) [2023-12-27 08:26:42,781][20347] Updated weights for policy 0, policy_version 405681 (0.0005) [2023-12-27 08:26:42,840][20347] Updated weights for policy 0, policy_version 405691 (0.0005) [2023-12-27 08:26:43,012][20381] Updated weights for policy 1, policy_version 405268 (0.0007) [2023-12-27 08:26:43,068][20381] Updated weights for policy 1, policy_version 405278 (0.0007) [2023-12-27 08:26:43,125][20381] Updated weights for policy 1, policy_version 405288 (0.0008) [2023-12-27 08:26:43,431][20347] Updated weights for policy 0, policy_version 405701 (0.0005) [2023-12-27 08:26:43,484][20347] Updated weights for policy 0, policy_version 405711 (0.0009) [2023-12-27 08:26:43,532][20347] Updated weights for policy 0, policy_version 405721 (0.0009) [2023-12-27 08:26:43,785][20381] Updated weights for policy 1, policy_version 405298 (0.0008) [2023-12-27 08:26:43,833][20381] Updated weights for policy 1, policy_version 405308 (0.0007) [2023-12-27 08:26:43,879][20381] Updated weights for policy 1, policy_version 405318 (0.0007) [2023-12-27 08:26:43,926][20381] Updated weights for policy 1, policy_version 405328 (0.0007) [2023-12-27 08:26:44,218][20347] Updated weights for policy 0, policy_version 405731 (0.0008) [2023-12-27 08:26:44,282][20347] Updated weights for policy 0, policy_version 405741 (0.0010) [2023-12-27 08:26:44,340][20347] Updated weights for policy 0, policy_version 405751 (0.0010) [2023-12-27 08:26:44,547][20381] Updated weights for policy 1, policy_version 405338 (0.0007) [2023-12-27 08:26:44,605][20381] Updated weights for policy 1, policy_version 405348 (0.0008) [2023-12-27 08:26:44,668][20381] Updated weights for policy 1, policy_version 405358 (0.0008) [2023-12-27 08:26:44,969][20347] Updated weights for policy 0, policy_version 405761 (0.0007) [2023-12-27 08:26:45,034][20347] Updated weights for policy 0, policy_version 405771 (0.0008) [2023-12-27 08:26:45,091][20347] Updated weights for policy 0, policy_version 405781 (0.0010) [2023-12-27 08:26:45,140][20347] Updated weights for policy 0, policy_version 405791 (0.0010) [2023-12-27 08:26:45,368][20381] Updated weights for policy 1, policy_version 405368 (0.0009) [2023-12-27 08:26:45,428][20381] Updated weights for policy 1, policy_version 405378 (0.0009) [2023-12-27 08:26:45,485][20381] Updated weights for policy 1, policy_version 405388 (0.0005) [2023-12-27 08:26:45,806][20347] Updated weights for policy 0, policy_version 405801 (0.0006) [2023-12-27 08:26:45,864][20347] Updated weights for policy 0, policy_version 405811 (0.0005) [2023-12-27 08:26:45,924][20347] Updated weights for policy 0, policy_version 405821 (0.0008) [2023-12-27 08:26:46,104][20381] Updated weights for policy 1, policy_version 405398 (0.0007) [2023-12-27 08:26:46,161][20381] Updated weights for policy 1, policy_version 405408 (0.0005) [2023-12-27 08:26:46,217][20381] Updated weights for policy 1, policy_version 405418 (0.0005) [2023-12-27 08:26:46,555][20347] Updated weights for policy 0, policy_version 405831 (0.0010) [2023-12-27 08:26:46,609][20347] Updated weights for policy 0, policy_version 405841 (0.0010) [2023-12-27 08:26:46,661][20347] Updated weights for policy 0, policy_version 405851 (0.0010) [2023-12-27 08:26:46,818][20381] Updated weights for policy 1, policy_version 405428 (0.0006) [2023-12-27 08:26:46,876][20381] Updated weights for policy 1, policy_version 405438 (0.0007) [2023-12-27 08:26:46,930][20381] Updated weights for policy 1, policy_version 405448 (0.0006) [2023-12-27 08:26:47,264][20347] Updated weights for policy 0, policy_version 405861 (0.0008) [2023-12-27 08:26:47,328][20347] Updated weights for policy 0, policy_version 405871 (0.0010) [2023-12-27 08:26:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 207732736. Throughput: 0: 10521.8, 1: 10062.7. Samples: 207746804. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:47,363][19170] Avg episode reward: [(0, '671.710'), (1, '847.210')] [2023-12-27 08:26:47,385][20347] Updated weights for policy 0, policy_version 405881 (0.0010) [2023-12-27 08:26:47,574][20381] Updated weights for policy 1, policy_version 405458 (0.0006) [2023-12-27 08:26:47,622][20381] Updated weights for policy 1, policy_version 405468 (0.0010) [2023-12-27 08:26:47,670][20381] Updated weights for policy 1, policy_version 405478 (0.0010) [2023-12-27 08:26:47,725][20381] Updated weights for policy 1, policy_version 405488 (0.0010) [2023-12-27 08:26:47,987][20347] Updated weights for policy 0, policy_version 405891 (0.0010) [2023-12-27 08:26:48,035][20347] Updated weights for policy 0, policy_version 405901 (0.0010) [2023-12-27 08:26:48,082][20347] Updated weights for policy 0, policy_version 405911 (0.0006) [2023-12-27 08:26:48,428][20381] Updated weights for policy 1, policy_version 405498 (0.0009) [2023-12-27 08:26:48,483][20381] Updated weights for policy 1, policy_version 405508 (0.0007) [2023-12-27 08:26:48,539][20381] Updated weights for policy 1, policy_version 405518 (0.0007) [2023-12-27 08:26:48,740][20347] Updated weights for policy 0, policy_version 405921 (0.0006) [2023-12-27 08:26:48,793][20347] Updated weights for policy 0, policy_version 405931 (0.0006) [2023-12-27 08:26:48,848][20347] Updated weights for policy 0, policy_version 405941 (0.0006) [2023-12-27 08:26:48,910][20347] Updated weights for policy 0, policy_version 405951 (0.0010) [2023-12-27 08:26:49,217][20381] Updated weights for policy 1, policy_version 405528 (0.0006) [2023-12-27 08:26:49,276][20381] Updated weights for policy 1, policy_version 405538 (0.0008) [2023-12-27 08:26:49,342][20381] Updated weights for policy 1, policy_version 405548 (0.0008) [2023-12-27 08:26:49,571][20347] Updated weights for policy 0, policy_version 405961 (0.0010) [2023-12-27 08:26:49,629][20347] Updated weights for policy 0, policy_version 405971 (0.0010) [2023-12-27 08:26:49,684][20347] Updated weights for policy 0, policy_version 405981 (0.0010) [2023-12-27 08:26:50,001][20381] Updated weights for policy 1, policy_version 405558 (0.0008) [2023-12-27 08:26:50,056][20381] Updated weights for policy 1, policy_version 405568 (0.0007) [2023-12-27 08:26:50,108][20381] Updated weights for policy 1, policy_version 405578 (0.0009) [2023-12-27 08:26:50,349][20347] Updated weights for policy 0, policy_version 405991 (0.0011) [2023-12-27 08:26:50,409][20347] Updated weights for policy 0, policy_version 406001 (0.0011) [2023-12-27 08:26:50,467][20347] Updated weights for policy 0, policy_version 406011 (0.0010) [2023-12-27 08:26:50,788][20381] Updated weights for policy 1, policy_version 405588 (0.0009) [2023-12-27 08:26:50,839][20381] Updated weights for policy 1, policy_version 405598 (0.0008) [2023-12-27 08:26:50,890][20381] Updated weights for policy 1, policy_version 405608 (0.0008) [2023-12-27 08:26:51,170][20347] Updated weights for policy 0, policy_version 406021 (0.0009) [2023-12-27 08:26:51,228][20347] Updated weights for policy 0, policy_version 406031 (0.0008) [2023-12-27 08:26:51,286][20347] Updated weights for policy 0, policy_version 406041 (0.0009) [2023-12-27 08:26:51,571][20381] Updated weights for policy 1, policy_version 405618 (0.0008) [2023-12-27 08:26:51,634][20381] Updated weights for policy 1, policy_version 405628 (0.0008) [2023-12-27 08:26:51,696][20381] Updated weights for policy 1, policy_version 405638 (0.0007) [2023-12-27 08:26:51,763][20381] Updated weights for policy 1, policy_version 405648 (0.0007) [2023-12-27 08:26:52,072][20347] Updated weights for policy 0, policy_version 406051 (0.0009) [2023-12-27 08:26:52,126][20347] Updated weights for policy 0, policy_version 406061 (0.0006) [2023-12-27 08:26:52,183][20347] Updated weights for policy 0, policy_version 406071 (0.0007) [2023-12-27 08:26:52,363][19170] Fps is (10 sec: 20478.9, 60 sec: 20616.4, 300 sec: 20743.8). Total num frames: 207839232. Throughput: 0: 10558.9, 1: 10080.0. Samples: 207809892. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:52,364][19170] Avg episode reward: [(0, '790.870'), (1, '923.090')] [2023-12-27 08:26:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000406080_103972864.pth... [2023-12-27 08:26:52,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000404832_103653376.pth [2023-12-27 08:26:52,425][20381] Updated weights for policy 1, policy_version 405658 (0.0008) [2023-12-27 08:26:52,483][20381] Updated weights for policy 1, policy_version 405668 (0.0007) [2023-12-27 08:26:52,541][20381] Updated weights for policy 1, policy_version 405678 (0.0007) [2023-12-27 08:26:52,551][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000405680_103874560.pth... [2023-12-27 08:26:52,555][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000404464_103563264.pth [2023-12-27 08:26:52,911][20347] Updated weights for policy 0, policy_version 406081 (0.0006) [2023-12-27 08:26:52,970][20347] Updated weights for policy 0, policy_version 406091 (0.0010) [2023-12-27 08:26:53,027][20347] Updated weights for policy 0, policy_version 406101 (0.0011) [2023-12-27 08:26:53,086][20347] Updated weights for policy 0, policy_version 406111 (0.0010) [2023-12-27 08:26:53,243][20381] Updated weights for policy 1, policy_version 405688 (0.0009) [2023-12-27 08:26:53,296][20381] Updated weights for policy 1, policy_version 405698 (0.0011) [2023-12-27 08:26:53,347][20381] Updated weights for policy 1, policy_version 405708 (0.0010) [2023-12-27 08:26:53,765][20347] Updated weights for policy 0, policy_version 406121 (0.0010) [2023-12-27 08:26:53,819][20347] Updated weights for policy 0, policy_version 406131 (0.0010) [2023-12-27 08:26:53,877][20347] Updated weights for policy 0, policy_version 406141 (0.0010) [2023-12-27 08:26:54,031][20381] Updated weights for policy 1, policy_version 405718 (0.0007) [2023-12-27 08:26:54,078][20381] Updated weights for policy 1, policy_version 405728 (0.0007) [2023-12-27 08:26:54,131][20381] Updated weights for policy 1, policy_version 405738 (0.0011) [2023-12-27 08:26:54,543][20347] Updated weights for policy 0, policy_version 406151 (0.0010) [2023-12-27 08:26:54,601][20347] Updated weights for policy 0, policy_version 406161 (0.0010) [2023-12-27 08:26:54,662][20347] Updated weights for policy 0, policy_version 406171 (0.0010) [2023-12-27 08:26:54,701][20381] Updated weights for policy 1, policy_version 405748 (0.0005) [2023-12-27 08:26:54,750][20381] Updated weights for policy 1, policy_version 405758 (0.0005) [2023-12-27 08:26:54,810][20381] Updated weights for policy 1, policy_version 405768 (0.0005) [2023-12-27 08:26:55,372][20381] Updated weights for policy 1, policy_version 405778 (0.0006) [2023-12-27 08:26:55,378][20347] Updated weights for policy 0, policy_version 406181 (0.0008) [2023-12-27 08:26:55,433][20381] Updated weights for policy 1, policy_version 405788 (0.0007) [2023-12-27 08:26:55,438][20347] Updated weights for policy 0, policy_version 406191 (0.0008) [2023-12-27 08:26:55,492][20347] Updated weights for policy 0, policy_version 406201 (0.0006) [2023-12-27 08:26:55,495][20381] Updated weights for policy 1, policy_version 405798 (0.0007) [2023-12-27 08:26:55,550][20381] Updated weights for policy 1, policy_version 405808 (0.0011) [2023-12-27 08:26:56,156][20347] Updated weights for policy 0, policy_version 406211 (0.0007) [2023-12-27 08:26:56,160][20381] Updated weights for policy 1, policy_version 405818 (0.0008) [2023-12-27 08:26:56,208][20347] Updated weights for policy 0, policy_version 406221 (0.0006) [2023-12-27 08:26:56,215][20381] Updated weights for policy 1, policy_version 405828 (0.0009) [2023-12-27 08:26:56,253][20347] Updated weights for policy 0, policy_version 406231 (0.0007) [2023-12-27 08:26:56,268][20381] Updated weights for policy 1, policy_version 405838 (0.0011) [2023-12-27 08:26:56,892][20347] Updated weights for policy 0, policy_version 406241 (0.0006) [2023-12-27 08:26:56,939][20347] Updated weights for policy 0, policy_version 406251 (0.0006) [2023-12-27 08:26:56,960][20381] Updated weights for policy 1, policy_version 405848 (0.0007) [2023-12-27 08:26:56,985][20347] Updated weights for policy 0, policy_version 406261 (0.0005) [2023-12-27 08:26:57,015][20381] Updated weights for policy 1, policy_version 405858 (0.0006) [2023-12-27 08:26:57,042][20347] Updated weights for policy 0, policy_version 406271 (0.0008) [2023-12-27 08:26:57,074][20381] Updated weights for policy 1, policy_version 405868 (0.0007) [2023-12-27 08:26:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 207945728. Throughput: 0: 10494.2, 1: 10180.5. Samples: 207932708. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:26:57,363][19170] Avg episode reward: [(0, '831.280'), (1, '913.270')] [2023-12-27 08:26:57,682][20347] Updated weights for policy 0, policy_version 406281 (0.0010) [2023-12-27 08:26:57,722][20381] Updated weights for policy 1, policy_version 405878 (0.0006) [2023-12-27 08:26:57,745][20347] Updated weights for policy 0, policy_version 406291 (0.0010) [2023-12-27 08:26:57,776][20381] Updated weights for policy 1, policy_version 405888 (0.0006) [2023-12-27 08:26:57,798][20347] Updated weights for policy 0, policy_version 406301 (0.0010) [2023-12-27 08:26:57,837][20381] Updated weights for policy 1, policy_version 405898 (0.0008) [2023-12-27 08:26:58,472][20347] Updated weights for policy 0, policy_version 406311 (0.0010) [2023-12-27 08:26:58,532][20347] Updated weights for policy 0, policy_version 406321 (0.0009) [2023-12-27 08:26:58,557][20381] Updated weights for policy 1, policy_version 405908 (0.0007) [2023-12-27 08:26:58,587][20347] Updated weights for policy 0, policy_version 406331 (0.0007) [2023-12-27 08:26:58,617][20381] Updated weights for policy 1, policy_version 405918 (0.0008) [2023-12-27 08:26:58,679][20381] Updated weights for policy 1, policy_version 405928 (0.0008) [2023-12-27 08:26:59,313][20347] Updated weights for policy 0, policy_version 406341 (0.0007) [2023-12-27 08:26:59,376][20347] Updated weights for policy 0, policy_version 406351 (0.0007) [2023-12-27 08:26:59,395][20381] Updated weights for policy 1, policy_version 405938 (0.0008) [2023-12-27 08:26:59,437][20347] Updated weights for policy 0, policy_version 406361 (0.0008) [2023-12-27 08:26:59,444][20381] Updated weights for policy 1, policy_version 405948 (0.0005) [2023-12-27 08:26:59,501][20381] Updated weights for policy 1, policy_version 405958 (0.0005) [2023-12-27 08:26:59,554][20381] Updated weights for policy 1, policy_version 405968 (0.0005) [2023-12-27 08:27:00,184][20347] Updated weights for policy 0, policy_version 406371 (0.0007) [2023-12-27 08:27:00,185][20381] Updated weights for policy 1, policy_version 405978 (0.0007) [2023-12-27 08:27:00,236][20347] Updated weights for policy 0, policy_version 406381 (0.0005) [2023-12-27 08:27:00,238][20381] Updated weights for policy 1, policy_version 405988 (0.0008) [2023-12-27 08:27:00,282][20381] Updated weights for policy 1, policy_version 405998 (0.0005) [2023-12-27 08:27:00,288][20347] Updated weights for policy 0, policy_version 406391 (0.0006) [2023-12-27 08:27:00,918][20381] Updated weights for policy 1, policy_version 406008 (0.0010) [2023-12-27 08:27:00,981][20381] Updated weights for policy 1, policy_version 406018 (0.0010) [2023-12-27 08:27:00,983][20347] Updated weights for policy 0, policy_version 406401 (0.0007) [2023-12-27 08:27:01,038][20381] Updated weights for policy 1, policy_version 406028 (0.0011) [2023-12-27 08:27:01,043][20347] Updated weights for policy 0, policy_version 406411 (0.0006) [2023-12-27 08:27:01,098][20347] Updated weights for policy 0, policy_version 406421 (0.0007) [2023-12-27 08:27:01,155][20347] Updated weights for policy 0, policy_version 406431 (0.0007) [2023-12-27 08:27:01,786][20381] Updated weights for policy 1, policy_version 406038 (0.0011) [2023-12-27 08:27:01,834][20347] Updated weights for policy 0, policy_version 406441 (0.0008) [2023-12-27 08:27:01,845][20381] Updated weights for policy 1, policy_version 406048 (0.0011) [2023-12-27 08:27:01,892][20347] Updated weights for policy 0, policy_version 406451 (0.0006) [2023-12-27 08:27:01,902][20381] Updated weights for policy 1, policy_version 406058 (0.0011) [2023-12-27 08:27:01,946][20347] Updated weights for policy 0, policy_version 406461 (0.0005) [2023-12-27 08:27:02,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 208044032. Throughput: 0: 10427.0, 1: 10207.1. Samples: 208055140. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:02,363][19170] Avg episode reward: [(0, '802.130'), (1, '932.500')] [2023-12-27 08:27:02,550][20347] Updated weights for policy 0, policy_version 406471 (0.0007) [2023-12-27 08:27:02,570][20381] Updated weights for policy 1, policy_version 406068 (0.0008) [2023-12-27 08:27:02,599][20347] Updated weights for policy 0, policy_version 406481 (0.0006) [2023-12-27 08:27:02,630][20381] Updated weights for policy 1, policy_version 406078 (0.0006) [2023-12-27 08:27:02,658][20347] Updated weights for policy 0, policy_version 406491 (0.0006) [2023-12-27 08:27:02,685][20381] Updated weights for policy 1, policy_version 406088 (0.0005) [2023-12-27 08:27:03,248][20381] Updated weights for policy 1, policy_version 406098 (0.0006) [2023-12-27 08:27:03,295][20381] Updated weights for policy 1, policy_version 406108 (0.0007) [2023-12-27 08:27:03,308][20347] Updated weights for policy 0, policy_version 406501 (0.0009) [2023-12-27 08:27:03,340][20381] Updated weights for policy 1, policy_version 406118 (0.0005) [2023-12-27 08:27:03,356][20347] Updated weights for policy 0, policy_version 406511 (0.0010) [2023-12-27 08:27:03,389][20381] Updated weights for policy 1, policy_version 406128 (0.0008) [2023-12-27 08:27:03,400][20347] Updated weights for policy 0, policy_version 406521 (0.0010) [2023-12-27 08:27:04,003][20381] Updated weights for policy 1, policy_version 406138 (0.0009) [2023-12-27 08:27:04,061][20381] Updated weights for policy 1, policy_version 406148 (0.0010) [2023-12-27 08:27:04,064][20347] Updated weights for policy 0, policy_version 406531 (0.0010) [2023-12-27 08:27:04,120][20381] Updated weights for policy 1, policy_version 406158 (0.0005) [2023-12-27 08:27:04,124][20347] Updated weights for policy 0, policy_version 406541 (0.0011) [2023-12-27 08:27:04,178][20347] Updated weights for policy 0, policy_version 406551 (0.0008) [2023-12-27 08:27:04,679][20381] Updated weights for policy 1, policy_version 406168 (0.0007) [2023-12-27 08:27:04,729][20381] Updated weights for policy 1, policy_version 406178 (0.0006) [2023-12-27 08:27:04,781][20381] Updated weights for policy 1, policy_version 406188 (0.0005) [2023-12-27 08:27:04,814][20347] Updated weights for policy 0, policy_version 406561 (0.0006) [2023-12-27 08:27:04,861][20347] Updated weights for policy 0, policy_version 406571 (0.0010) [2023-12-27 08:27:04,910][20347] Updated weights for policy 0, policy_version 406581 (0.0010) [2023-12-27 08:27:04,971][20347] Updated weights for policy 0, policy_version 406591 (0.0010) [2023-12-27 08:27:05,372][20381] Updated weights for policy 1, policy_version 406198 (0.0008) [2023-12-27 08:27:05,422][20381] Updated weights for policy 1, policy_version 406208 (0.0010) [2023-12-27 08:27:05,473][20381] Updated weights for policy 1, policy_version 406218 (0.0010) [2023-12-27 08:27:05,607][20347] Updated weights for policy 0, policy_version 406601 (0.0010) [2023-12-27 08:27:05,655][20347] Updated weights for policy 0, policy_version 406611 (0.0010) [2023-12-27 08:27:05,709][20347] Updated weights for policy 0, policy_version 406621 (0.0010) [2023-12-27 08:27:06,094][20381] Updated weights for policy 1, policy_version 406228 (0.0011) [2023-12-27 08:27:06,156][20381] Updated weights for policy 1, policy_version 406238 (0.0011) [2023-12-27 08:27:06,164][20292] KL-divergence is very high: 107.1370 [2023-12-27 08:27:06,221][20381] Updated weights for policy 1, policy_version 406248 (0.0010) [2023-12-27 08:27:06,410][20347] Updated weights for policy 0, policy_version 406631 (0.0009) [2023-12-27 08:27:06,465][20347] Updated weights for policy 0, policy_version 406641 (0.0005) [2023-12-27 08:27:06,527][20347] Updated weights for policy 0, policy_version 406651 (0.0005) [2023-12-27 08:27:06,907][20381] Updated weights for policy 1, policy_version 406258 (0.0011) [2023-12-27 08:27:06,962][20381] Updated weights for policy 1, policy_version 406268 (0.0010) [2023-12-27 08:27:07,017][20381] Updated weights for policy 1, policy_version 406278 (0.0010) [2023-12-27 08:27:07,073][20381] Updated weights for policy 1, policy_version 406288 (0.0010) [2023-12-27 08:27:07,106][20347] Updated weights for policy 0, policy_version 406661 (0.0008) [2023-12-27 08:27:07,152][20347] Updated weights for policy 0, policy_version 406671 (0.0011) [2023-12-27 08:27:07,210][20347] Updated weights for policy 0, policy_version 406681 (0.0008) [2023-12-27 08:27:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 208158720. Throughput: 0: 10440.5, 1: 10313.5. Samples: 208122076. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:07,364][19170] Avg episode reward: [(0, '866.120'), (1, '914.230')] [2023-12-27 08:27:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000406288_104030208.pth... [2023-12-27 08:27:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000406688_104128512.pth... [2023-12-27 08:27:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000405072_103718912.pth [2023-12-27 08:27:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000405472_103817216.pth [2023-12-27 08:27:07,651][20381] Updated weights for policy 1, policy_version 406298 (0.0010) [2023-12-27 08:27:07,702][20381] Updated weights for policy 1, policy_version 406308 (0.0010) [2023-12-27 08:27:07,772][20381] Updated weights for policy 1, policy_version 406318 (0.0007) [2023-12-27 08:27:07,842][20347] Updated weights for policy 0, policy_version 406691 (0.0010) [2023-12-27 08:27:07,891][20347] Updated weights for policy 0, policy_version 406701 (0.0010) [2023-12-27 08:27:07,942][20347] Updated weights for policy 0, policy_version 406711 (0.0010) [2023-12-27 08:27:08,449][20381] Updated weights for policy 1, policy_version 406328 (0.0008) [2023-12-27 08:27:08,507][20381] Updated weights for policy 1, policy_version 406338 (0.0008) [2023-12-27 08:27:08,569][20381] Updated weights for policy 1, policy_version 406348 (0.0008) [2023-12-27 08:27:08,641][20347] Updated weights for policy 0, policy_version 406721 (0.0010) [2023-12-27 08:27:08,708][20347] Updated weights for policy 0, policy_version 406731 (0.0008) [2023-12-27 08:27:08,765][20347] Updated weights for policy 0, policy_version 406741 (0.0010) [2023-12-27 08:27:08,823][20347] Updated weights for policy 0, policy_version 406751 (0.0010) [2023-12-27 08:27:09,247][20381] Updated weights for policy 1, policy_version 406358 (0.0008) [2023-12-27 08:27:09,302][20381] Updated weights for policy 1, policy_version 406368 (0.0009) [2023-12-27 08:27:09,364][20381] Updated weights for policy 1, policy_version 406378 (0.0009) [2023-12-27 08:27:09,538][20347] Updated weights for policy 0, policy_version 406761 (0.0007) [2023-12-27 08:27:09,608][20347] Updated weights for policy 0, policy_version 406771 (0.0006) [2023-12-27 08:27:09,676][20347] Updated weights for policy 0, policy_version 406781 (0.0006) [2023-12-27 08:27:10,101][20381] Updated weights for policy 1, policy_version 406388 (0.0008) [2023-12-27 08:27:10,162][20381] Updated weights for policy 1, policy_version 406398 (0.0008) [2023-12-27 08:27:10,215][20381] Updated weights for policy 1, policy_version 406408 (0.0009) [2023-12-27 08:27:10,258][20347] Updated weights for policy 0, policy_version 406791 (0.0006) [2023-12-27 08:27:10,316][20347] Updated weights for policy 0, policy_version 406801 (0.0008) [2023-12-27 08:27:10,380][20347] Updated weights for policy 0, policy_version 406811 (0.0006) [2023-12-27 08:27:10,913][20381] Updated weights for policy 1, policy_version 406418 (0.0007) [2023-12-27 08:27:10,973][20381] Updated weights for policy 1, policy_version 406428 (0.0011) [2023-12-27 08:27:11,014][20347] Updated weights for policy 0, policy_version 406821 (0.0006) [2023-12-27 08:27:11,035][20381] Updated weights for policy 1, policy_version 406438 (0.0011) [2023-12-27 08:27:11,078][20347] Updated weights for policy 0, policy_version 406831 (0.0008) [2023-12-27 08:27:11,090][20381] Updated weights for policy 1, policy_version 406448 (0.0011) [2023-12-27 08:27:11,139][20347] Updated weights for policy 0, policy_version 406841 (0.0007) [2023-12-27 08:27:11,747][20381] Updated weights for policy 1, policy_version 406458 (0.0008) [2023-12-27 08:27:11,808][20381] Updated weights for policy 1, policy_version 406468 (0.0008) [2023-12-27 08:27:11,868][20381] Updated weights for policy 1, policy_version 406478 (0.0007) [2023-12-27 08:27:11,881][20347] Updated weights for policy 0, policy_version 406851 (0.0007) [2023-12-27 08:27:11,945][20347] Updated weights for policy 0, policy_version 406861 (0.0008) [2023-12-27 08:27:12,006][20347] Updated weights for policy 0, policy_version 406871 (0.0008) [2023-12-27 08:27:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 208257024. Throughput: 0: 10457.9, 1: 10326.4. Samples: 208247708. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:12,363][19170] Avg episode reward: [(0, '941.970'), (1, '914.000')] [2023-12-27 08:27:12,610][20381] Updated weights for policy 1, policy_version 406488 (0.0006) [2023-12-27 08:27:12,626][20347] Updated weights for policy 0, policy_version 406881 (0.0006) [2023-12-27 08:27:12,670][20381] Updated weights for policy 1, policy_version 406498 (0.0007) [2023-12-27 08:27:12,685][20347] Updated weights for policy 0, policy_version 406891 (0.0006) [2023-12-27 08:27:12,731][20381] Updated weights for policy 1, policy_version 406508 (0.0006) [2023-12-27 08:27:12,748][20347] Updated weights for policy 0, policy_version 406901 (0.0007) [2023-12-27 08:27:12,808][20347] Updated weights for policy 0, policy_version 406911 (0.0009) [2023-12-27 08:27:13,267][20381] Updated weights for policy 1, policy_version 406518 (0.0008) [2023-12-27 08:27:13,316][20381] Updated weights for policy 1, policy_version 406528 (0.0010) [2023-12-27 08:27:13,366][20381] Updated weights for policy 1, policy_version 406538 (0.0010) [2023-12-27 08:27:13,474][20347] Updated weights for policy 0, policy_version 406921 (0.0008) [2023-12-27 08:27:13,530][20347] Updated weights for policy 0, policy_version 406931 (0.0008) [2023-12-27 08:27:13,581][20347] Updated weights for policy 0, policy_version 406941 (0.0007) [2023-12-27 08:27:14,047][20381] Updated weights for policy 1, policy_version 406548 (0.0008) [2023-12-27 08:27:14,104][20381] Updated weights for policy 1, policy_version 406558 (0.0010) [2023-12-27 08:27:14,163][20381] Updated weights for policy 1, policy_version 406568 (0.0010) [2023-12-27 08:27:14,211][20347] Updated weights for policy 0, policy_version 406951 (0.0009) [2023-12-27 08:27:14,268][20347] Updated weights for policy 0, policy_version 406961 (0.0007) [2023-12-27 08:27:14,324][20347] Updated weights for policy 0, policy_version 406971 (0.0007) [2023-12-27 08:27:14,769][20381] Updated weights for policy 1, policy_version 406578 (0.0010) [2023-12-27 08:27:14,834][20381] Updated weights for policy 1, policy_version 406588 (0.0007) [2023-12-27 08:27:14,897][20381] Updated weights for policy 1, policy_version 406598 (0.0007) [2023-12-27 08:27:14,961][20381] Updated weights for policy 1, policy_version 406608 (0.0008) [2023-12-27 08:27:14,987][20347] Updated weights for policy 0, policy_version 406981 (0.0006) [2023-12-27 08:27:15,051][20347] Updated weights for policy 0, policy_version 406991 (0.0006) [2023-12-27 08:27:15,117][20347] Updated weights for policy 0, policy_version 407001 (0.0006) [2023-12-27 08:27:15,658][20381] Updated weights for policy 1, policy_version 406618 (0.0006) [2023-12-27 08:27:15,661][20347] Updated weights for policy 0, policy_version 407011 (0.0006) [2023-12-27 08:27:15,717][20381] Updated weights for policy 1, policy_version 406628 (0.0005) [2023-12-27 08:27:15,718][20347] Updated weights for policy 0, policy_version 407021 (0.0008) [2023-12-27 08:27:15,772][20347] Updated weights for policy 0, policy_version 407031 (0.0007) [2023-12-27 08:27:15,776][20381] Updated weights for policy 1, policy_version 406638 (0.0005) [2023-12-27 08:27:16,407][20381] Updated weights for policy 1, policy_version 406648 (0.0005) [2023-12-27 08:27:16,470][20381] Updated weights for policy 1, policy_version 406658 (0.0006) [2023-12-27 08:27:16,497][20347] Updated weights for policy 0, policy_version 407041 (0.0007) [2023-12-27 08:27:16,531][20381] Updated weights for policy 1, policy_version 406668 (0.0009) [2023-12-27 08:27:16,562][20347] Updated weights for policy 0, policy_version 407051 (0.0007) [2023-12-27 08:27:16,618][20347] Updated weights for policy 0, policy_version 407061 (0.0007) [2023-12-27 08:27:16,675][20347] Updated weights for policy 0, policy_version 407071 (0.0008) [2023-12-27 08:27:17,171][20381] Updated weights for policy 1, policy_version 406678 (0.0007) [2023-12-27 08:27:17,227][20381] Updated weights for policy 1, policy_version 406688 (0.0008) [2023-12-27 08:27:17,283][20381] Updated weights for policy 1, policy_version 406698 (0.0007) [2023-12-27 08:27:17,336][20347] Updated weights for policy 0, policy_version 407081 (0.0006) [2023-12-27 08:27:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 208363520. Throughput: 0: 10427.3, 1: 10354.3. Samples: 208373120. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:17,363][19170] Avg episode reward: [(0, '904.050'), (1, '922.810')] [2023-12-27 08:27:17,383][20347] Updated weights for policy 0, policy_version 407091 (0.0006) [2023-12-27 08:27:17,438][20347] Updated weights for policy 0, policy_version 407101 (0.0008) [2023-12-27 08:27:17,898][20381] Updated weights for policy 1, policy_version 406708 (0.0006) [2023-12-27 08:27:17,943][20381] Updated weights for policy 1, policy_version 406718 (0.0005) [2023-12-27 08:27:17,957][20292] KL-divergence is very high: 104.3647 [2023-12-27 08:27:17,988][20381] Updated weights for policy 1, policy_version 406728 (0.0005) [2023-12-27 08:27:18,047][20347] Updated weights for policy 0, policy_version 407111 (0.0011) [2023-12-27 08:27:18,100][20347] Updated weights for policy 0, policy_version 407121 (0.0009) [2023-12-27 08:27:18,157][20347] Updated weights for policy 0, policy_version 407131 (0.0008) [2023-12-27 08:27:18,665][20381] Updated weights for policy 1, policy_version 406738 (0.0006) [2023-12-27 08:27:18,721][20381] Updated weights for policy 1, policy_version 406748 (0.0008) [2023-12-27 08:27:18,777][20381] Updated weights for policy 1, policy_version 406758 (0.0008) [2023-12-27 08:27:18,823][20347] Updated weights for policy 0, policy_version 407141 (0.0008) [2023-12-27 08:27:18,835][20381] Updated weights for policy 1, policy_version 406768 (0.0007) [2023-12-27 08:27:18,872][20347] Updated weights for policy 0, policy_version 407151 (0.0005) [2023-12-27 08:27:18,920][20347] Updated weights for policy 0, policy_version 407161 (0.0005) [2023-12-27 08:27:19,558][20347] Updated weights for policy 0, policy_version 407171 (0.0005) [2023-12-27 08:27:19,587][20381] Updated weights for policy 1, policy_version 406778 (0.0008) [2023-12-27 08:27:19,615][20347] Updated weights for policy 0, policy_version 407181 (0.0007) [2023-12-27 08:27:19,637][20381] Updated weights for policy 1, policy_version 406788 (0.0006) [2023-12-27 08:27:19,662][20347] Updated weights for policy 0, policy_version 407191 (0.0006) [2023-12-27 08:27:19,687][20381] Updated weights for policy 1, policy_version 406798 (0.0006) [2023-12-27 08:27:20,378][20381] Updated weights for policy 1, policy_version 406808 (0.0007) [2023-12-27 08:27:20,400][20347] Updated weights for policy 0, policy_version 407201 (0.0006) [2023-12-27 08:27:20,428][20381] Updated weights for policy 1, policy_version 406818 (0.0006) [2023-12-27 08:27:20,447][20347] Updated weights for policy 0, policy_version 407211 (0.0006) [2023-12-27 08:27:20,471][20381] Updated weights for policy 1, policy_version 406828 (0.0005) [2023-12-27 08:27:20,501][20347] Updated weights for policy 0, policy_version 407221 (0.0007) [2023-12-27 08:27:20,546][20347] Updated weights for policy 0, policy_version 407231 (0.0007) [2023-12-27 08:27:21,154][20381] Updated weights for policy 1, policy_version 406838 (0.0009) [2023-12-27 08:27:21,214][20381] Updated weights for policy 1, policy_version 406848 (0.0011) [2023-12-27 08:27:21,248][20347] Updated weights for policy 0, policy_version 407241 (0.0006) [2023-12-27 08:27:21,276][20381] Updated weights for policy 1, policy_version 406858 (0.0011) [2023-12-27 08:27:21,314][20347] Updated weights for policy 0, policy_version 407251 (0.0006) [2023-12-27 08:27:21,382][20347] Updated weights for policy 0, policy_version 407261 (0.0008) [2023-12-27 08:27:22,036][20381] Updated weights for policy 1, policy_version 406868 (0.0010) [2023-12-27 08:27:22,056][20347] Updated weights for policy 0, policy_version 407271 (0.0009) [2023-12-27 08:27:22,105][20381] Updated weights for policy 1, policy_version 406878 (0.0007) [2023-12-27 08:27:22,111][20347] Updated weights for policy 0, policy_version 407281 (0.0007) [2023-12-27 08:27:22,164][20347] Updated weights for policy 0, policy_version 407291 (0.0007) [2023-12-27 08:27:22,171][20381] Updated weights for policy 1, policy_version 406888 (0.0007) [2023-12-27 08:27:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 208470016. Throughput: 0: 10407.7, 1: 10362.3. Samples: 208435728. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:22,363][19170] Avg episode reward: [(0, '894.630'), (1, '904.560')] [2023-12-27 08:27:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000407296_104284160.pth... [2023-12-27 08:27:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000406896_104185856.pth... [2023-12-27 08:27:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000406080_103972864.pth [2023-12-27 08:27:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000405680_103874560.pth [2023-12-27 08:27:22,806][20347] Updated weights for policy 0, policy_version 407301 (0.0009) [2023-12-27 08:27:22,865][20347] Updated weights for policy 0, policy_version 407311 (0.0011) [2023-12-27 08:27:22,894][20381] Updated weights for policy 1, policy_version 406898 (0.0007) [2023-12-27 08:27:22,915][20347] Updated weights for policy 0, policy_version 407321 (0.0010) [2023-12-27 08:27:22,956][20381] Updated weights for policy 1, policy_version 406908 (0.0006) [2023-12-27 08:27:23,020][20381] Updated weights for policy 1, policy_version 406918 (0.0007) [2023-12-27 08:27:23,083][20381] Updated weights for policy 1, policy_version 406928 (0.0006) [2023-12-27 08:27:23,613][20347] Updated weights for policy 0, policy_version 407331 (0.0010) [2023-12-27 08:27:23,646][20381] Updated weights for policy 1, policy_version 406938 (0.0010) [2023-12-27 08:27:23,659][20347] Updated weights for policy 0, policy_version 407341 (0.0010) [2023-12-27 08:27:23,701][20381] Updated weights for policy 1, policy_version 406948 (0.0007) [2023-12-27 08:27:23,711][20347] Updated weights for policy 0, policy_version 407351 (0.0010) [2023-12-27 08:27:23,759][20381] Updated weights for policy 1, policy_version 406958 (0.0007) [2023-12-27 08:27:24,331][20347] Updated weights for policy 0, policy_version 407361 (0.0010) [2023-12-27 08:27:24,396][20347] Updated weights for policy 0, policy_version 407371 (0.0005) [2023-12-27 08:27:24,429][20381] Updated weights for policy 1, policy_version 406968 (0.0008) [2023-12-27 08:27:24,457][20347] Updated weights for policy 0, policy_version 407381 (0.0005) [2023-12-27 08:27:24,481][20381] Updated weights for policy 1, policy_version 406978 (0.0010) [2023-12-27 08:27:24,513][20347] Updated weights for policy 0, policy_version 407391 (0.0006) [2023-12-27 08:27:24,535][20381] Updated weights for policy 1, policy_version 406988 (0.0010) [2023-12-27 08:27:25,130][20347] Updated weights for policy 0, policy_version 407401 (0.0006) [2023-12-27 08:27:25,143][20381] Updated weights for policy 1, policy_version 406998 (0.0007) [2023-12-27 08:27:25,188][20347] Updated weights for policy 0, policy_version 407411 (0.0006) [2023-12-27 08:27:25,208][20381] Updated weights for policy 1, policy_version 407008 (0.0006) [2023-12-27 08:27:25,237][20347] Updated weights for policy 0, policy_version 407421 (0.0005) [2023-12-27 08:27:25,267][20381] Updated weights for policy 1, policy_version 407018 (0.0005) [2023-12-27 08:27:25,811][20381] Updated weights for policy 1, policy_version 407028 (0.0005) [2023-12-27 08:27:25,860][20381] Updated weights for policy 1, policy_version 407038 (0.0005) [2023-12-27 08:27:25,892][20347] Updated weights for policy 0, policy_version 407431 (0.0007) [2023-12-27 08:27:25,913][20381] Updated weights for policy 1, policy_version 407048 (0.0005) [2023-12-27 08:27:25,951][20347] Updated weights for policy 0, policy_version 407441 (0.0007) [2023-12-27 08:27:26,016][20347] Updated weights for policy 0, policy_version 407451 (0.0005) [2023-12-27 08:27:26,498][20381] Updated weights for policy 1, policy_version 407058 (0.0010) [2023-12-27 08:27:26,544][20381] Updated weights for policy 1, policy_version 407068 (0.0011) [2023-12-27 08:27:26,582][20347] Updated weights for policy 0, policy_version 407461 (0.0006) [2023-12-27 08:27:26,602][20381] Updated weights for policy 1, policy_version 407078 (0.0010) [2023-12-27 08:27:26,626][20347] Updated weights for policy 0, policy_version 407471 (0.0006) [2023-12-27 08:27:26,650][20381] Updated weights for policy 1, policy_version 407088 (0.0010) [2023-12-27 08:27:26,675][20347] Updated weights for policy 0, policy_version 407481 (0.0006) [2023-12-27 08:27:27,308][20347] Updated weights for policy 0, policy_version 407491 (0.0008) [2023-12-27 08:27:27,331][20381] Updated weights for policy 1, policy_version 407098 (0.0007) [2023-12-27 08:27:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 208568320. Throughput: 0: 10390.9, 1: 10503.0. Samples: 208562148. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:27,363][19170] Avg episode reward: [(0, '933.020'), (1, '905.410')] [2023-12-27 08:27:27,367][20347] Updated weights for policy 0, policy_version 407501 (0.0009) [2023-12-27 08:27:27,385][20381] Updated weights for policy 1, policy_version 407108 (0.0011) [2023-12-27 08:27:27,421][20347] Updated weights for policy 0, policy_version 407511 (0.0010) [2023-12-27 08:27:27,431][20381] Updated weights for policy 1, policy_version 407118 (0.0010) [2023-12-27 08:27:27,997][20347] Updated weights for policy 0, policy_version 407521 (0.0010) [2023-12-27 08:27:28,056][20347] Updated weights for policy 0, policy_version 407531 (0.0005) [2023-12-27 08:27:28,061][20291] KL-divergence is very high: 149.4353 [2023-12-27 08:27:28,113][20347] Updated weights for policy 0, policy_version 407541 (0.0005) [2023-12-27 08:27:28,126][20381] Updated weights for policy 1, policy_version 407128 (0.0011) [2023-12-27 08:27:28,167][20347] Updated weights for policy 0, policy_version 407551 (0.0005) [2023-12-27 08:27:28,175][20381] Updated weights for policy 1, policy_version 407138 (0.0010) [2023-12-27 08:27:28,226][20381] Updated weights for policy 1, policy_version 407148 (0.0010) [2023-12-27 08:27:28,702][20347] Updated weights for policy 0, policy_version 407561 (0.0007) [2023-12-27 08:27:28,764][20347] Updated weights for policy 0, policy_version 407571 (0.0006) [2023-12-27 08:27:28,815][20347] Updated weights for policy 0, policy_version 407581 (0.0006) [2023-12-27 08:27:28,903][20381] Updated weights for policy 1, policy_version 407158 (0.0011) [2023-12-27 08:27:28,949][20381] Updated weights for policy 1, policy_version 407168 (0.0010) [2023-12-27 08:27:28,995][20381] Updated weights for policy 1, policy_version 407178 (0.0010) [2023-12-27 08:27:29,498][20347] Updated weights for policy 0, policy_version 407591 (0.0006) [2023-12-27 08:27:29,560][20347] Updated weights for policy 0, policy_version 407601 (0.0005) [2023-12-27 08:27:29,619][20347] Updated weights for policy 0, policy_version 407611 (0.0007) [2023-12-27 08:27:29,691][20381] Updated weights for policy 1, policy_version 407188 (0.0008) [2023-12-27 08:27:29,738][20381] Updated weights for policy 1, policy_version 407198 (0.0005) [2023-12-27 08:27:29,799][20381] Updated weights for policy 1, policy_version 407208 (0.0011) [2023-12-27 08:27:30,275][20347] Updated weights for policy 0, policy_version 407621 (0.0008) [2023-12-27 08:27:30,324][20347] Updated weights for policy 0, policy_version 407631 (0.0007) [2023-12-27 08:27:30,382][20347] Updated weights for policy 0, policy_version 407641 (0.0008) [2023-12-27 08:27:30,437][20381] Updated weights for policy 1, policy_version 407218 (0.0011) [2023-12-27 08:27:30,486][20381] Updated weights for policy 1, policy_version 407228 (0.0007) [2023-12-27 08:27:30,549][20381] Updated weights for policy 1, policy_version 407238 (0.0006) [2023-12-27 08:27:30,612][20381] Updated weights for policy 1, policy_version 407248 (0.0005) [2023-12-27 08:27:31,089][20347] Updated weights for policy 0, policy_version 407651 (0.0007) [2023-12-27 08:27:31,155][20347] Updated weights for policy 0, policy_version 407661 (0.0008) [2023-12-27 08:27:31,199][20381] Updated weights for policy 1, policy_version 407258 (0.0006) [2023-12-27 08:27:31,215][20347] Updated weights for policy 0, policy_version 407671 (0.0006) [2023-12-27 08:27:31,253][20381] Updated weights for policy 1, policy_version 407268 (0.0006) [2023-12-27 08:27:31,316][20381] Updated weights for policy 1, policy_version 407278 (0.0008) [2023-12-27 08:27:31,907][20347] Updated weights for policy 0, policy_version 407681 (0.0007) [2023-12-27 08:27:31,953][20347] Updated weights for policy 0, policy_version 407691 (0.0005) [2023-12-27 08:27:32,000][20347] Updated weights for policy 0, policy_version 407701 (0.0005) [2023-12-27 08:27:32,004][20381] Updated weights for policy 1, policy_version 407288 (0.0007) [2023-12-27 08:27:32,053][20347] Updated weights for policy 0, policy_version 407711 (0.0008) [2023-12-27 08:27:32,059][20381] Updated weights for policy 1, policy_version 407298 (0.0006) [2023-12-27 08:27:32,120][20381] Updated weights for policy 1, policy_version 407308 (0.0008) [2023-12-27 08:27:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 208683008. Throughput: 0: 10468.7, 1: 10505.2. Samples: 208690628. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:32,363][19170] Avg episode reward: [(0, '971.490'), (1, '904.660')] [2023-12-27 08:27:32,736][20347] Updated weights for policy 0, policy_version 407721 (0.0009) [2023-12-27 08:27:32,794][20347] Updated weights for policy 0, policy_version 407731 (0.0010) [2023-12-27 08:27:32,811][20381] Updated weights for policy 1, policy_version 407318 (0.0009) [2023-12-27 08:27:32,851][20347] Updated weights for policy 0, policy_version 407741 (0.0010) [2023-12-27 08:27:32,866][20381] Updated weights for policy 1, policy_version 407328 (0.0008) [2023-12-27 08:27:32,922][20381] Updated weights for policy 1, policy_version 407338 (0.0005) [2023-12-27 08:27:33,456][20347] Updated weights for policy 0, policy_version 407751 (0.0010) [2023-12-27 08:27:33,516][20347] Updated weights for policy 0, policy_version 407761 (0.0010) [2023-12-27 08:27:33,548][20381] Updated weights for policy 1, policy_version 407348 (0.0009) [2023-12-27 08:27:33,568][20347] Updated weights for policy 0, policy_version 407771 (0.0010) [2023-12-27 08:27:33,593][20381] Updated weights for policy 1, policy_version 407358 (0.0010) [2023-12-27 08:27:33,647][20381] Updated weights for policy 1, policy_version 407368 (0.0010) [2023-12-27 08:27:34,227][20347] Updated weights for policy 0, policy_version 407781 (0.0008) [2023-12-27 08:27:34,286][20347] Updated weights for policy 0, policy_version 407791 (0.0008) [2023-12-27 08:27:34,335][20381] Updated weights for policy 1, policy_version 407378 (0.0009) [2023-12-27 08:27:34,348][20347] Updated weights for policy 0, policy_version 407801 (0.0011) [2023-12-27 08:27:34,402][20381] Updated weights for policy 1, policy_version 407388 (0.0006) [2023-12-27 08:27:34,461][20381] Updated weights for policy 1, policy_version 407398 (0.0011) [2023-12-27 08:27:34,516][20381] Updated weights for policy 1, policy_version 407408 (0.0010) [2023-12-27 08:27:35,016][20347] Updated weights for policy 0, policy_version 407811 (0.0009) [2023-12-27 08:27:35,058][20291] KL-divergence is very high: 229.1292 [2023-12-27 08:27:35,063][20291] KL-divergence is very high: 153.4143 [2023-12-27 08:27:35,073][20347] Updated weights for policy 0, policy_version 407821 (0.0005) [2023-12-27 08:27:35,125][20347] Updated weights for policy 0, policy_version 407831 (0.0005) [2023-12-27 08:27:35,132][20381] Updated weights for policy 1, policy_version 407418 (0.0005) [2023-12-27 08:27:35,186][20381] Updated weights for policy 1, policy_version 407428 (0.0005) [2023-12-27 08:27:35,239][20381] Updated weights for policy 1, policy_version 407438 (0.0008) [2023-12-27 08:27:35,704][20347] Updated weights for policy 0, policy_version 407841 (0.0010) [2023-12-27 08:27:35,764][20347] Updated weights for policy 0, policy_version 407851 (0.0010) [2023-12-27 08:27:35,819][20347] Updated weights for policy 0, policy_version 407861 (0.0010) [2023-12-27 08:27:35,820][20381] Updated weights for policy 1, policy_version 407448 (0.0006) [2023-12-27 08:27:35,871][20381] Updated weights for policy 1, policy_version 407458 (0.0005) [2023-12-27 08:27:35,875][20347] Updated weights for policy 0, policy_version 407871 (0.0010) [2023-12-27 08:27:35,930][20381] Updated weights for policy 1, policy_version 407468 (0.0007) [2023-12-27 08:27:36,500][20347] Updated weights for policy 0, policy_version 407881 (0.0011) [2023-12-27 08:27:36,556][20347] Updated weights for policy 0, policy_version 407891 (0.0010) [2023-12-27 08:27:36,601][20381] Updated weights for policy 1, policy_version 407478 (0.0009) [2023-12-27 08:27:36,610][20347] Updated weights for policy 0, policy_version 407901 (0.0007) [2023-12-27 08:27:36,656][20381] Updated weights for policy 1, policy_version 407488 (0.0008) [2023-12-27 08:27:36,704][20381] Updated weights for policy 1, policy_version 407498 (0.0008) [2023-12-27 08:27:37,213][20347] Updated weights for policy 0, policy_version 407911 (0.0007) [2023-12-27 08:27:37,274][20347] Updated weights for policy 0, policy_version 407921 (0.0007) [2023-12-27 08:27:37,334][20347] Updated weights for policy 0, policy_version 407931 (0.0008) [2023-12-27 08:27:37,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 208789504. Throughput: 0: 10444.0, 1: 10541.2. Samples: 208754212. Policy #0 lag: (min: 31.0, avg: 33.2, max: 63.0) [2023-12-27 08:27:37,363][19170] Avg episode reward: [(0, '912.820'), (1, '933.330')] [2023-12-27 08:27:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000407504_104341504.pth... [2023-12-27 08:27:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000407936_104448000.pth... [2023-12-27 08:27:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000406688_104128512.pth [2023-12-27 08:27:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000406288_104030208.pth [2023-12-27 08:27:37,409][20381] Updated weights for policy 1, policy_version 407508 (0.0007) [2023-12-27 08:27:37,469][20381] Updated weights for policy 1, policy_version 407518 (0.0007) [2023-12-27 08:27:37,523][20381] Updated weights for policy 1, policy_version 407528 (0.0008) [2023-12-27 08:27:38,019][20347] Updated weights for policy 0, policy_version 407941 (0.0009) [2023-12-27 08:27:38,077][20347] Updated weights for policy 0, policy_version 407951 (0.0008) [2023-12-27 08:27:38,126][20347] Updated weights for policy 0, policy_version 407961 (0.0008) [2023-12-27 08:27:38,161][20381] Updated weights for policy 1, policy_version 407538 (0.0007) [2023-12-27 08:27:38,210][20381] Updated weights for policy 1, policy_version 407548 (0.0007) [2023-12-27 08:27:38,267][20381] Updated weights for policy 1, policy_version 407558 (0.0008) [2023-12-27 08:27:38,323][20381] Updated weights for policy 1, policy_version 407568 (0.0008) [2023-12-27 08:27:38,877][20347] Updated weights for policy 0, policy_version 407971 (0.0008) [2023-12-27 08:27:38,925][20347] Updated weights for policy 0, policy_version 407981 (0.0007) [2023-12-27 08:27:38,981][20347] Updated weights for policy 0, policy_version 407991 (0.0007) [2023-12-27 08:27:39,033][20381] Updated weights for policy 1, policy_version 407578 (0.0010) [2023-12-27 08:27:39,079][20381] Updated weights for policy 1, policy_version 407588 (0.0008) [2023-12-27 08:27:39,125][20381] Updated weights for policy 1, policy_version 407598 (0.0005) [2023-12-27 08:27:39,663][20347] Updated weights for policy 0, policy_version 408001 (0.0006) [2023-12-27 08:27:39,727][20347] Updated weights for policy 0, policy_version 408011 (0.0007) [2023-12-27 08:27:39,789][20347] Updated weights for policy 0, policy_version 408021 (0.0006) [2023-12-27 08:27:39,812][20381] Updated weights for policy 1, policy_version 407608 (0.0010) [2023-12-27 08:27:39,851][20347] Updated weights for policy 0, policy_version 408031 (0.0007) [2023-12-27 08:27:39,880][20381] Updated weights for policy 1, policy_version 407618 (0.0011) [2023-12-27 08:27:39,944][20381] Updated weights for policy 1, policy_version 407628 (0.0011) [2023-12-27 08:27:40,513][20347] Updated weights for policy 0, policy_version 408041 (0.0007) [2023-12-27 08:27:40,566][20347] Updated weights for policy 0, policy_version 408051 (0.0007) [2023-12-27 08:27:40,619][20347] Updated weights for policy 0, policy_version 408061 (0.0007) [2023-12-27 08:27:40,634][20381] Updated weights for policy 1, policy_version 407638 (0.0011) [2023-12-27 08:27:40,692][20381] Updated weights for policy 1, policy_version 407648 (0.0010) [2023-12-27 08:27:40,744][20381] Updated weights for policy 1, policy_version 407658 (0.0010) [2023-12-27 08:27:41,326][20347] Updated weights for policy 0, policy_version 408071 (0.0007) [2023-12-27 08:27:41,392][20347] Updated weights for policy 0, policy_version 408081 (0.0008) [2023-12-27 08:27:41,455][20381] Updated weights for policy 1, policy_version 407668 (0.0010) [2023-12-27 08:27:41,456][20347] Updated weights for policy 0, policy_version 408091 (0.0006) [2023-12-27 08:27:41,515][20381] Updated weights for policy 1, policy_version 407678 (0.0010) [2023-12-27 08:27:41,576][20381] Updated weights for policy 1, policy_version 407688 (0.0010) [2023-12-27 08:27:42,166][20347] Updated weights for policy 0, policy_version 408101 (0.0007) [2023-12-27 08:27:42,215][20347] Updated weights for policy 0, policy_version 408111 (0.0009) [2023-12-27 08:27:42,268][20347] Updated weights for policy 0, policy_version 408121 (0.0008) [2023-12-27 08:27:42,272][20381] Updated weights for policy 1, policy_version 407698 (0.0008) [2023-12-27 08:27:42,329][20381] Updated weights for policy 1, policy_version 407708 (0.0007) [2023-12-27 08:27:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 208887808. Throughput: 0: 10513.7, 1: 10508.2. Samples: 208878700. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:27:42,363][19170] Avg episode reward: [(0, '836.090'), (1, '943.570')] [2023-12-27 08:27:42,396][20381] Updated weights for policy 1, policy_version 407718 (0.0011) [2023-12-27 08:27:42,463][20381] Updated weights for policy 1, policy_version 407728 (0.0008) [2023-12-27 08:27:42,951][20347] Updated weights for policy 0, policy_version 408131 (0.0009) [2023-12-27 08:27:43,000][20347] Updated weights for policy 0, policy_version 408141 (0.0010) [2023-12-27 08:27:43,048][20347] Updated weights for policy 0, policy_version 408151 (0.0010) [2023-12-27 08:27:43,096][20381] Updated weights for policy 1, policy_version 407738 (0.0007) [2023-12-27 08:27:43,142][20292] KL-divergence is very high: 165.4393 [2023-12-27 08:27:43,159][20381] Updated weights for policy 1, policy_version 407748 (0.0005) [2023-12-27 08:27:43,221][20381] Updated weights for policy 1, policy_version 407758 (0.0005) [2023-12-27 08:27:43,738][20381] Updated weights for policy 1, policy_version 407768 (0.0007) [2023-12-27 08:27:43,749][20347] Updated weights for policy 0, policy_version 408161 (0.0011) [2023-12-27 08:27:43,792][20381] Updated weights for policy 1, policy_version 407778 (0.0006) [2023-12-27 08:27:43,808][20347] Updated weights for policy 0, policy_version 408171 (0.0010) [2023-12-27 08:27:43,850][20381] Updated weights for policy 1, policy_version 407788 (0.0006) [2023-12-27 08:27:43,866][20347] Updated weights for policy 0, policy_version 408181 (0.0011) [2023-12-27 08:27:43,928][20347] Updated weights for policy 0, policy_version 408191 (0.0010) [2023-12-27 08:27:44,465][20381] Updated weights for policy 1, policy_version 407798 (0.0007) [2023-12-27 08:27:44,518][20381] Updated weights for policy 1, policy_version 407808 (0.0008) [2023-12-27 08:27:44,571][20381] Updated weights for policy 1, policy_version 407818 (0.0008) [2023-12-27 08:27:44,602][20347] Updated weights for policy 0, policy_version 408201 (0.0009) [2023-12-27 08:27:44,657][20347] Updated weights for policy 0, policy_version 408211 (0.0010) [2023-12-27 08:27:44,712][20347] Updated weights for policy 0, policy_version 408221 (0.0010) [2023-12-27 08:27:45,207][20381] Updated weights for policy 1, policy_version 407828 (0.0006) [2023-12-27 08:27:45,258][20381] Updated weights for policy 1, policy_version 407838 (0.0007) [2023-12-27 08:27:45,319][20381] Updated weights for policy 1, policy_version 407848 (0.0007) [2023-12-27 08:27:45,432][20347] Updated weights for policy 0, policy_version 408231 (0.0010) [2023-12-27 08:27:45,487][20347] Updated weights for policy 0, policy_version 408241 (0.0010) [2023-12-27 08:27:45,540][20347] Updated weights for policy 0, policy_version 408251 (0.0010) [2023-12-27 08:27:46,014][20381] Updated weights for policy 1, policy_version 407858 (0.0007) [2023-12-27 08:27:46,067][20381] Updated weights for policy 1, policy_version 407868 (0.0006) [2023-12-27 08:27:46,115][20381] Updated weights for policy 1, policy_version 407878 (0.0005) [2023-12-27 08:27:46,166][20381] Updated weights for policy 1, policy_version 407888 (0.0005) [2023-12-27 08:27:46,202][20347] Updated weights for policy 0, policy_version 408261 (0.0010) [2023-12-27 08:27:46,248][20347] Updated weights for policy 0, policy_version 408271 (0.0009) [2023-12-27 08:27:46,294][20347] Updated weights for policy 0, policy_version 408281 (0.0008) [2023-12-27 08:27:46,815][20381] Updated weights for policy 1, policy_version 407898 (0.0007) [2023-12-27 08:27:46,878][20381] Updated weights for policy 1, policy_version 407908 (0.0008) [2023-12-27 08:27:46,924][20381] Updated weights for policy 1, policy_version 407918 (0.0010) [2023-12-27 08:27:46,971][20347] Updated weights for policy 0, policy_version 408291 (0.0008) [2023-12-27 08:27:47,019][20347] Updated weights for policy 0, policy_version 408301 (0.0008) [2023-12-27 08:27:47,075][20347] Updated weights for policy 0, policy_version 408311 (0.0008) [2023-12-27 08:27:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 208994304. Throughput: 0: 10505.5, 1: 10553.3. Samples: 209002788. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:27:47,364][19170] Avg episode reward: [(0, '777.570'), (1, '915.070')] [2023-12-27 08:27:47,637][20381] Updated weights for policy 1, policy_version 407928 (0.0007) [2023-12-27 08:27:47,685][20381] Updated weights for policy 1, policy_version 407938 (0.0008) [2023-12-27 08:27:47,698][20347] Updated weights for policy 0, policy_version 408321 (0.0007) [2023-12-27 08:27:47,741][20381] Updated weights for policy 1, policy_version 407948 (0.0008) [2023-12-27 08:27:47,743][20347] Updated weights for policy 0, policy_version 408331 (0.0005) [2023-12-27 08:27:47,793][20347] Updated weights for policy 0, policy_version 408341 (0.0005) [2023-12-27 08:27:47,843][20347] Updated weights for policy 0, policy_version 408351 (0.0010) [2023-12-27 08:27:48,440][20347] Updated weights for policy 0, policy_version 408361 (0.0010) [2023-12-27 08:27:48,498][20347] Updated weights for policy 0, policy_version 408371 (0.0010) [2023-12-27 08:27:48,511][20381] Updated weights for policy 1, policy_version 407958 (0.0008) [2023-12-27 08:27:48,551][20347] Updated weights for policy 0, policy_version 408381 (0.0011) [2023-12-27 08:27:48,567][20381] Updated weights for policy 1, policy_version 407968 (0.0005) [2023-12-27 08:27:48,620][20381] Updated weights for policy 1, policy_version 407978 (0.0008) [2023-12-27 08:27:49,249][20347] Updated weights for policy 0, policy_version 408391 (0.0010) [2023-12-27 08:27:49,306][20347] Updated weights for policy 0, policy_version 408401 (0.0009) [2023-12-27 08:27:49,319][20381] Updated weights for policy 1, policy_version 407988 (0.0007) [2023-12-27 08:27:49,365][20347] Updated weights for policy 0, policy_version 408411 (0.0008) [2023-12-27 08:27:49,381][20381] Updated weights for policy 1, policy_version 407998 (0.0007) [2023-12-27 08:27:49,436][20381] Updated weights for policy 1, policy_version 408008 (0.0006) [2023-12-27 08:27:50,084][20381] Updated weights for policy 1, policy_version 408018 (0.0007) [2023-12-27 08:27:50,099][20347] Updated weights for policy 0, policy_version 408421 (0.0007) [2023-12-27 08:27:50,140][20381] Updated weights for policy 1, policy_version 408028 (0.0005) [2023-12-27 08:27:50,147][20347] Updated weights for policy 0, policy_version 408431 (0.0008) [2023-12-27 08:27:50,201][20381] Updated weights for policy 1, policy_version 408038 (0.0006) [2023-12-27 08:27:50,204][20347] Updated weights for policy 0, policy_version 408441 (0.0008) [2023-12-27 08:27:50,258][20381] Updated weights for policy 1, policy_version 408048 (0.0006) [2023-12-27 08:27:50,884][20381] Updated weights for policy 1, policy_version 408058 (0.0008) [2023-12-27 08:27:50,914][20347] Updated weights for policy 0, policy_version 408451 (0.0009) [2023-12-27 08:27:50,938][20381] Updated weights for policy 1, policy_version 408068 (0.0006) [2023-12-27 08:27:50,973][20347] Updated weights for policy 0, policy_version 408461 (0.0011) [2023-12-27 08:27:50,993][20381] Updated weights for policy 1, policy_version 408078 (0.0006) [2023-12-27 08:27:51,031][20347] Updated weights for policy 0, policy_version 408471 (0.0010) [2023-12-27 08:27:51,663][20381] Updated weights for policy 1, policy_version 408088 (0.0007) [2023-12-27 08:27:51,733][20381] Updated weights for policy 1, policy_version 408098 (0.0007) [2023-12-27 08:27:51,765][20347] Updated weights for policy 0, policy_version 408481 (0.0007) [2023-12-27 08:27:51,791][20381] Updated weights for policy 1, policy_version 408108 (0.0007) [2023-12-27 08:27:51,816][20347] Updated weights for policy 0, policy_version 408491 (0.0006) [2023-12-27 08:27:51,872][20347] Updated weights for policy 0, policy_version 408501 (0.0008) [2023-12-27 08:27:51,928][20347] Updated weights for policy 0, policy_version 408511 (0.0007) [2023-12-27 08:27:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.8, 300 sec: 20771.6). Total num frames: 209092608. Throughput: 0: 10500.0, 1: 10445.2. Samples: 209064604. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:27:52,363][19170] Avg episode reward: [(0, '796.970'), (1, '914.990')] [2023-12-27 08:27:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000408112_104497152.pth... [2023-12-27 08:27:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000408512_104595456.pth... [2023-12-27 08:27:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000407296_104284160.pth [2023-12-27 08:27:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000406896_104185856.pth [2023-12-27 08:27:52,448][20381] Updated weights for policy 1, policy_version 408118 (0.0008) [2023-12-27 08:27:52,508][20381] Updated weights for policy 1, policy_version 408128 (0.0007) [2023-12-27 08:27:52,572][20381] Updated weights for policy 1, policy_version 408138 (0.0007) [2023-12-27 08:27:52,619][20347] Updated weights for policy 0, policy_version 408521 (0.0008) [2023-12-27 08:27:52,678][20347] Updated weights for policy 0, policy_version 408531 (0.0005) [2023-12-27 08:27:52,732][20347] Updated weights for policy 0, policy_version 408541 (0.0005) [2023-12-27 08:27:53,241][20381] Updated weights for policy 1, policy_version 408148 (0.0006) [2023-12-27 08:27:53,287][20381] Updated weights for policy 1, policy_version 408158 (0.0008) [2023-12-27 08:27:53,342][20381] Updated weights for policy 1, policy_version 408168 (0.0007) [2023-12-27 08:27:53,370][20347] Updated weights for policy 0, policy_version 408551 (0.0007) [2023-12-27 08:27:53,417][20347] Updated weights for policy 0, policy_version 408561 (0.0008) [2023-12-27 08:27:53,469][20347] Updated weights for policy 0, policy_version 408571 (0.0008) [2023-12-27 08:27:53,925][20381] Updated weights for policy 1, policy_version 408178 (0.0006) [2023-12-27 08:27:53,980][20381] Updated weights for policy 1, policy_version 408188 (0.0005) [2023-12-27 08:27:54,028][20381] Updated weights for policy 1, policy_version 408198 (0.0007) [2023-12-27 08:27:54,076][20381] Updated weights for policy 1, policy_version 408208 (0.0008) [2023-12-27 08:27:54,080][20347] Updated weights for policy 0, policy_version 408581 (0.0007) [2023-12-27 08:27:54,135][20347] Updated weights for policy 0, policy_version 408591 (0.0005) [2023-12-27 08:27:54,197][20347] Updated weights for policy 0, policy_version 408601 (0.0006) [2023-12-27 08:27:54,718][20347] Updated weights for policy 0, policy_version 408611 (0.0006) [2023-12-27 08:27:54,770][20347] Updated weights for policy 0, policy_version 408621 (0.0006) [2023-12-27 08:27:54,822][20347] Updated weights for policy 0, policy_version 408631 (0.0008) [2023-12-27 08:27:54,829][20381] Updated weights for policy 1, policy_version 408218 (0.0006) [2023-12-27 08:27:54,889][20381] Updated weights for policy 1, policy_version 408228 (0.0006) [2023-12-27 08:27:54,940][20381] Updated weights for policy 1, policy_version 408238 (0.0006) [2023-12-27 08:27:55,392][20347] Updated weights for policy 0, policy_version 408641 (0.0006) [2023-12-27 08:27:55,432][20291] KL-divergence is very high: 163.6911 [2023-12-27 08:27:55,448][20347] Updated weights for policy 0, policy_version 408651 (0.0005) [2023-12-27 08:27:55,506][20347] Updated weights for policy 0, policy_version 408661 (0.0007) [2023-12-27 08:27:55,549][20347] Updated weights for policy 0, policy_version 408671 (0.0008) [2023-12-27 08:27:55,633][20381] Updated weights for policy 1, policy_version 408248 (0.0007) [2023-12-27 08:27:55,690][20381] Updated weights for policy 1, policy_version 408258 (0.0008) [2023-12-27 08:27:55,750][20381] Updated weights for policy 1, policy_version 408268 (0.0009) [2023-12-27 08:27:56,125][20291] KL-divergence is very high: 156.9882 [2023-12-27 08:27:56,178][20347] Updated weights for policy 0, policy_version 408681 (0.0006) [2023-12-27 08:27:56,233][20347] Updated weights for policy 0, policy_version 408691 (0.0005) [2023-12-27 08:27:56,284][20347] Updated weights for policy 0, policy_version 408701 (0.0006) [2023-12-27 08:27:56,408][20381] Updated weights for policy 1, policy_version 408278 (0.0007) [2023-12-27 08:27:56,453][20381] Updated weights for policy 1, policy_version 408288 (0.0008) [2023-12-27 08:27:56,507][20381] Updated weights for policy 1, policy_version 408298 (0.0006) [2023-12-27 08:27:56,943][20347] Updated weights for policy 0, policy_version 408711 (0.0008) [2023-12-27 08:27:57,000][20347] Updated weights for policy 0, policy_version 408721 (0.0007) [2023-12-27 08:27:57,058][20347] Updated weights for policy 0, policy_version 408731 (0.0005) [2023-12-27 08:27:57,115][20381] Updated weights for policy 1, policy_version 408308 (0.0007) [2023-12-27 08:27:57,176][20381] Updated weights for policy 1, policy_version 408318 (0.0005) [2023-12-27 08:27:57,234][20381] Updated weights for policy 1, policy_version 408328 (0.0006) [2023-12-27 08:27:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 209207296. Throughput: 0: 10513.1, 1: 10455.6. Samples: 209191304. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:27:57,363][19170] Avg episode reward: [(0, '787.950'), (1, '941.850')] [2023-12-27 08:27:57,651][20347] Updated weights for policy 0, policy_version 408741 (0.0008) [2023-12-27 08:27:57,701][20347] Updated weights for policy 0, policy_version 408751 (0.0007) [2023-12-27 08:27:57,753][20347] Updated weights for policy 0, policy_version 408761 (0.0009) [2023-12-27 08:27:57,877][20381] Updated weights for policy 1, policy_version 408338 (0.0011) [2023-12-27 08:27:57,922][20381] Updated weights for policy 1, policy_version 408348 (0.0010) [2023-12-27 08:27:57,970][20381] Updated weights for policy 1, policy_version 408358 (0.0010) [2023-12-27 08:27:58,026][20381] Updated weights for policy 1, policy_version 408368 (0.0010) [2023-12-27 08:27:58,449][20347] Updated weights for policy 0, policy_version 408771 (0.0010) [2023-12-27 08:27:58,516][20347] Updated weights for policy 0, policy_version 408781 (0.0008) [2023-12-27 08:27:58,578][20347] Updated weights for policy 0, policy_version 408791 (0.0008) [2023-12-27 08:27:58,604][20291] KL-divergence is very high: 174.2054 [2023-12-27 08:27:58,783][20381] Updated weights for policy 1, policy_version 408378 (0.0010) [2023-12-27 08:27:58,846][20381] Updated weights for policy 1, policy_version 408388 (0.0008) [2023-12-27 08:27:58,925][20381] Updated weights for policy 1, policy_version 408399 (0.0010) [2023-12-27 08:27:59,339][20347] Updated weights for policy 0, policy_version 408801 (0.0007) [2023-12-27 08:27:59,403][20347] Updated weights for policy 0, policy_version 408811 (0.0008) [2023-12-27 08:27:59,452][20347] Updated weights for policy 0, policy_version 408821 (0.0006) [2023-12-27 08:27:59,512][20347] Updated weights for policy 0, policy_version 408831 (0.0005) [2023-12-27 08:27:59,719][20381] Updated weights for policy 1, policy_version 408409 (0.0010) [2023-12-27 08:27:59,770][20381] Updated weights for policy 1, policy_version 408419 (0.0010) [2023-12-27 08:27:59,834][20381] Updated weights for policy 1, policy_version 408429 (0.0008) [2023-12-27 08:28:00,196][20347] Updated weights for policy 0, policy_version 408841 (0.0007) [2023-12-27 08:28:00,251][20347] Updated weights for policy 0, policy_version 408851 (0.0010) [2023-12-27 08:28:00,293][20291] KL-divergence is very high: 147.8273 [2023-12-27 08:28:00,305][20347] Updated weights for policy 0, policy_version 408861 (0.0010) [2023-12-27 08:28:00,509][20381] Updated weights for policy 1, policy_version 408439 (0.0010) [2023-12-27 08:28:00,564][20381] Updated weights for policy 1, policy_version 408449 (0.0010) [2023-12-27 08:28:00,623][20381] Updated weights for policy 1, policy_version 408459 (0.0010) [2023-12-27 08:28:00,944][20347] Updated weights for policy 0, policy_version 408871 (0.0007) [2023-12-27 08:28:00,999][20347] Updated weights for policy 0, policy_version 408881 (0.0005) [2023-12-27 08:28:01,056][20347] Updated weights for policy 0, policy_version 408891 (0.0007) [2023-12-27 08:28:01,269][20381] Updated weights for policy 1, policy_version 408469 (0.0011) [2023-12-27 08:28:01,319][20381] Updated weights for policy 1, policy_version 408479 (0.0010) [2023-12-27 08:28:01,380][20381] Updated weights for policy 1, policy_version 408489 (0.0011) [2023-12-27 08:28:01,677][20347] Updated weights for policy 0, policy_version 408901 (0.0006) [2023-12-27 08:28:01,738][20347] Updated weights for policy 0, policy_version 408911 (0.0007) [2023-12-27 08:28:01,796][20347] Updated weights for policy 0, policy_version 408921 (0.0007) [2023-12-27 08:28:02,128][20381] Updated weights for policy 1, policy_version 408499 (0.0009) [2023-12-27 08:28:02,184][20381] Updated weights for policy 1, policy_version 408509 (0.0007) [2023-12-27 08:28:02,240][20381] Updated weights for policy 1, policy_version 408519 (0.0007) [2023-12-27 08:28:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 209305600. Throughput: 0: 10506.5, 1: 10396.2. Samples: 209313744. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:02,363][19170] Avg episode reward: [(0, '798.550'), (1, '836.630')] [2023-12-27 08:28:02,510][20347] Updated weights for policy 0, policy_version 408931 (0.0009) [2023-12-27 08:28:02,566][20347] Updated weights for policy 0, policy_version 408941 (0.0008) [2023-12-27 08:28:02,625][20347] Updated weights for policy 0, policy_version 408951 (0.0008) [2023-12-27 08:28:02,958][20381] Updated weights for policy 1, policy_version 408529 (0.0008) [2023-12-27 08:28:03,019][20381] Updated weights for policy 1, policy_version 408539 (0.0007) [2023-12-27 08:28:03,089][20381] Updated weights for policy 1, policy_version 408549 (0.0007) [2023-12-27 08:28:03,153][20381] Updated weights for policy 1, policy_version 408559 (0.0007) [2023-12-27 08:28:03,247][20347] Updated weights for policy 0, policy_version 408961 (0.0008) [2023-12-27 08:28:03,296][20347] Updated weights for policy 0, policy_version 408971 (0.0005) [2023-12-27 08:28:03,350][20347] Updated weights for policy 0, policy_version 408981 (0.0006) [2023-12-27 08:28:03,403][20347] Updated weights for policy 0, policy_version 408991 (0.0005) [2023-12-27 08:28:03,764][20381] Updated weights for policy 1, policy_version 408569 (0.0008) [2023-12-27 08:28:03,815][20381] Updated weights for policy 1, policy_version 408579 (0.0007) [2023-12-27 08:28:03,868][20381] Updated weights for policy 1, policy_version 408589 (0.0007) [2023-12-27 08:28:03,963][20347] Updated weights for policy 0, policy_version 409001 (0.0009) [2023-12-27 08:28:04,018][20347] Updated weights for policy 0, policy_version 409011 (0.0009) [2023-12-27 08:28:04,064][20347] Updated weights for policy 0, policy_version 409021 (0.0007) [2023-12-27 08:28:04,511][20381] Updated weights for policy 1, policy_version 408599 (0.0006) [2023-12-27 08:28:04,569][20381] Updated weights for policy 1, policy_version 408609 (0.0005) [2023-12-27 08:28:04,622][20381] Updated weights for policy 1, policy_version 408619 (0.0007) [2023-12-27 08:28:04,771][20347] Updated weights for policy 0, policy_version 409031 (0.0008) [2023-12-27 08:28:04,832][20347] Updated weights for policy 0, policy_version 409041 (0.0006) [2023-12-27 08:28:04,895][20347] Updated weights for policy 0, policy_version 409051 (0.0005) [2023-12-27 08:28:05,333][20381] Updated weights for policy 1, policy_version 408629 (0.0008) [2023-12-27 08:28:05,386][20381] Updated weights for policy 1, policy_version 408639 (0.0008) [2023-12-27 08:28:05,436][20347] Updated weights for policy 0, policy_version 409061 (0.0005) [2023-12-27 08:28:05,444][20381] Updated weights for policy 1, policy_version 408649 (0.0008) [2023-12-27 08:28:05,462][20292] KL-divergence is very high: 124.9117 [2023-12-27 08:28:05,463][20291] KL-divergence is very high: 713.5872 [2023-12-27 08:28:05,497][20347] Updated weights for policy 0, policy_version 409071 (0.0008) [2023-12-27 08:28:05,561][20347] Updated weights for policy 0, policy_version 409081 (0.0010) [2023-12-27 08:28:05,563][20291] KL-divergence is very high: 104.1564 [2023-12-27 08:28:06,094][20381] Updated weights for policy 1, policy_version 408659 (0.0007) [2023-12-27 08:28:06,141][20292] KL-divergence is very high: 171.2881 [2023-12-27 08:28:06,157][20381] Updated weights for policy 1, policy_version 408669 (0.0007) [2023-12-27 08:28:06,179][20347] Updated weights for policy 0, policy_version 409091 (0.0010) [2023-12-27 08:28:06,215][20381] Updated weights for policy 1, policy_version 408679 (0.0007) [2023-12-27 08:28:06,236][20347] Updated weights for policy 0, policy_version 409101 (0.0010) [2023-12-27 08:28:06,295][20347] Updated weights for policy 0, policy_version 409111 (0.0010) [2023-12-27 08:28:06,903][20347] Updated weights for policy 0, policy_version 409121 (0.0010) [2023-12-27 08:28:06,931][20381] Updated weights for policy 1, policy_version 408689 (0.0007) [2023-12-27 08:28:06,958][20347] Updated weights for policy 0, policy_version 409131 (0.0005) [2023-12-27 08:28:06,985][20381] Updated weights for policy 1, policy_version 408699 (0.0008) [2023-12-27 08:28:07,016][20347] Updated weights for policy 0, policy_version 409141 (0.0006) [2023-12-27 08:28:07,037][20381] Updated weights for policy 1, policy_version 408709 (0.0006) [2023-12-27 08:28:07,068][20347] Updated weights for policy 0, policy_version 409151 (0.0010) [2023-12-27 08:28:07,095][20381] Updated weights for policy 1, policy_version 408719 (0.0006) [2023-12-27 08:28:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20799.4). Total num frames: 209412096. Throughput: 0: 10522.3, 1: 10398.8. Samples: 209377176. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:07,363][19170] Avg episode reward: [(0, '760.650'), (1, '714.930')] [2023-12-27 08:28:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000409152_104759296.pth... [2023-12-27 08:28:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000408720_104652800.pth... [2023-12-27 08:28:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000407936_104448000.pth [2023-12-27 08:28:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000407504_104341504.pth [2023-12-27 08:28:07,649][20347] Updated weights for policy 0, policy_version 409161 (0.0007) [2023-12-27 08:28:07,691][20292] KL-divergence is very high: 110.5304 [2023-12-27 08:28:07,693][20347] Updated weights for policy 0, policy_version 409171 (0.0005) [2023-12-27 08:28:07,699][20381] Updated weights for policy 1, policy_version 408729 (0.0006) [2023-12-27 08:28:07,740][20347] Updated weights for policy 0, policy_version 409181 (0.0005) [2023-12-27 08:28:07,757][20381] Updated weights for policy 1, policy_version 408739 (0.0007) [2023-12-27 08:28:07,811][20381] Updated weights for policy 1, policy_version 408749 (0.0008) [2023-12-27 08:28:08,305][20347] Updated weights for policy 0, policy_version 409191 (0.0005) [2023-12-27 08:28:08,368][20347] Updated weights for policy 0, policy_version 409201 (0.0007) [2023-12-27 08:28:08,430][20347] Updated weights for policy 0, policy_version 409211 (0.0007) [2023-12-27 08:28:08,490][20381] Updated weights for policy 1, policy_version 408759 (0.0008) [2023-12-27 08:28:08,547][20381] Updated weights for policy 1, policy_version 408769 (0.0007) [2023-12-27 08:28:08,606][20381] Updated weights for policy 1, policy_version 408779 (0.0008) [2023-12-27 08:28:09,071][20347] Updated weights for policy 0, policy_version 409221 (0.0006) [2023-12-27 08:28:09,126][20347] Updated weights for policy 0, policy_version 409231 (0.0007) [2023-12-27 08:28:09,179][20347] Updated weights for policy 0, policy_version 409241 (0.0008) [2023-12-27 08:28:09,271][20381] Updated weights for policy 1, policy_version 408789 (0.0009) [2023-12-27 08:28:09,330][20381] Updated weights for policy 1, policy_version 408799 (0.0011) [2023-12-27 08:28:09,394][20381] Updated weights for policy 1, policy_version 408809 (0.0011) [2023-12-27 08:28:09,892][20347] Updated weights for policy 0, policy_version 409251 (0.0008) [2023-12-27 08:28:09,951][20347] Updated weights for policy 0, policy_version 409261 (0.0008) [2023-12-27 08:28:10,009][20347] Updated weights for policy 0, policy_version 409271 (0.0007) [2023-12-27 08:28:10,134][20381] Updated weights for policy 1, policy_version 408819 (0.0009) [2023-12-27 08:28:10,196][20381] Updated weights for policy 1, policy_version 408829 (0.0006) [2023-12-27 08:28:10,257][20381] Updated weights for policy 1, policy_version 408839 (0.0005) [2023-12-27 08:28:10,729][20347] Updated weights for policy 0, policy_version 409281 (0.0008) [2023-12-27 08:28:10,789][20347] Updated weights for policy 0, policy_version 409291 (0.0008) [2023-12-27 08:28:10,848][20347] Updated weights for policy 0, policy_version 409301 (0.0007) [2023-12-27 08:28:10,878][20381] Updated weights for policy 1, policy_version 408849 (0.0006) [2023-12-27 08:28:10,907][20347] Updated weights for policy 0, policy_version 409311 (0.0006) [2023-12-27 08:28:10,936][20381] Updated weights for policy 1, policy_version 408859 (0.0006) [2023-12-27 08:28:10,993][20381] Updated weights for policy 1, policy_version 408869 (0.0008) [2023-12-27 08:28:11,056][20381] Updated weights for policy 1, policy_version 408879 (0.0007) [2023-12-27 08:28:11,596][20347] Updated weights for policy 0, policy_version 409321 (0.0007) [2023-12-27 08:28:11,657][20347] Updated weights for policy 0, policy_version 409331 (0.0007) [2023-12-27 08:28:11,745][20347] Updated weights for policy 0, policy_version 409341 (0.0008) [2023-12-27 08:28:11,855][20381] Updated weights for policy 1, policy_version 408889 (0.0009) [2023-12-27 08:28:11,917][20381] Updated weights for policy 1, policy_version 408899 (0.0008) [2023-12-27 08:28:11,978][20381] Updated weights for policy 1, policy_version 408909 (0.0007) [2023-12-27 08:28:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 209510400. Throughput: 0: 10583.8, 1: 10334.4. Samples: 209503468. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:12,363][19170] Avg episode reward: [(0, '816.580'), (1, '696.450')] [2023-12-27 08:28:12,417][20347] Updated weights for policy 0, policy_version 409351 (0.0007) [2023-12-27 08:28:12,481][20347] Updated weights for policy 0, policy_version 409361 (0.0009) [2023-12-27 08:28:12,544][20347] Updated weights for policy 0, policy_version 409371 (0.0009) [2023-12-27 08:28:12,726][20381] Updated weights for policy 1, policy_version 408919 (0.0008) [2023-12-27 08:28:12,781][20381] Updated weights for policy 1, policy_version 408929 (0.0007) [2023-12-27 08:28:12,834][20381] Updated weights for policy 1, policy_version 408939 (0.0009) [2023-12-27 08:28:13,167][20347] Updated weights for policy 0, policy_version 409381 (0.0010) [2023-12-27 08:28:13,229][20347] Updated weights for policy 0, policy_version 409391 (0.0005) [2023-12-27 08:28:13,289][20347] Updated weights for policy 0, policy_version 409401 (0.0010) [2023-12-27 08:28:13,561][20381] Updated weights for policy 1, policy_version 408949 (0.0009) [2023-12-27 08:28:13,615][20381] Updated weights for policy 1, policy_version 408959 (0.0008) [2023-12-27 08:28:13,669][20381] Updated weights for policy 1, policy_version 408969 (0.0007) [2023-12-27 08:28:13,945][20347] Updated weights for policy 0, policy_version 409411 (0.0009) [2023-12-27 08:28:14,000][20347] Updated weights for policy 0, policy_version 409421 (0.0005) [2023-12-27 08:28:14,054][20347] Updated weights for policy 0, policy_version 409431 (0.0005) [2023-12-27 08:28:14,323][20381] Updated weights for policy 1, policy_version 408979 (0.0008) [2023-12-27 08:28:14,367][20381] Updated weights for policy 1, policy_version 408989 (0.0008) [2023-12-27 08:28:14,417][20381] Updated weights for policy 1, policy_version 408999 (0.0007) [2023-12-27 08:28:14,622][20347] Updated weights for policy 0, policy_version 409441 (0.0006) [2023-12-27 08:28:14,683][20347] Updated weights for policy 0, policy_version 409451 (0.0008) [2023-12-27 08:28:14,742][20347] Updated weights for policy 0, policy_version 409461 (0.0008) [2023-12-27 08:28:14,807][20347] Updated weights for policy 0, policy_version 409471 (0.0007) [2023-12-27 08:28:15,091][20381] Updated weights for policy 1, policy_version 409009 (0.0007) [2023-12-27 08:28:15,149][20381] Updated weights for policy 1, policy_version 409019 (0.0008) [2023-12-27 08:28:15,210][20381] Updated weights for policy 1, policy_version 409029 (0.0006) [2023-12-27 08:28:15,272][20381] Updated weights for policy 1, policy_version 409039 (0.0007) [2023-12-27 08:28:15,536][20347] Updated weights for policy 0, policy_version 409481 (0.0008) [2023-12-27 08:28:15,598][20347] Updated weights for policy 0, policy_version 409491 (0.0011) [2023-12-27 08:28:15,660][20347] Updated weights for policy 0, policy_version 409501 (0.0010) [2023-12-27 08:28:15,880][20381] Updated weights for policy 1, policy_version 409049 (0.0007) [2023-12-27 08:28:15,930][20381] Updated weights for policy 1, policy_version 409059 (0.0008) [2023-12-27 08:28:15,991][20381] Updated weights for policy 1, policy_version 409069 (0.0008) [2023-12-27 08:28:16,259][20347] Updated weights for policy 0, policy_version 409511 (0.0007) [2023-12-27 08:28:16,316][20347] Updated weights for policy 0, policy_version 409521 (0.0005) [2023-12-27 08:28:16,374][20347] Updated weights for policy 0, policy_version 409531 (0.0006) [2023-12-27 08:28:16,555][20381] Updated weights for policy 1, policy_version 409079 (0.0009) [2023-12-27 08:28:16,613][20381] Updated weights for policy 1, policy_version 409089 (0.0008) [2023-12-27 08:28:16,664][20381] Updated weights for policy 1, policy_version 409099 (0.0007) [2023-12-27 08:28:17,014][20347] Updated weights for policy 0, policy_version 409541 (0.0008) [2023-12-27 08:28:17,068][20347] Updated weights for policy 0, policy_version 409551 (0.0007) [2023-12-27 08:28:17,118][20347] Updated weights for policy 0, policy_version 409561 (0.0005) [2023-12-27 08:28:17,214][20381] Updated weights for policy 1, policy_version 409109 (0.0005) [2023-12-27 08:28:17,273][20381] Updated weights for policy 1, policy_version 409119 (0.0005) [2023-12-27 08:28:17,328][20381] Updated weights for policy 1, policy_version 409129 (0.0005) [2023-12-27 08:28:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 209616896. Throughput: 0: 10533.5, 1: 10334.7. Samples: 209629704. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:17,364][19170] Avg episode reward: [(0, '903.390'), (1, '709.110')] [2023-12-27 08:28:17,788][20347] Updated weights for policy 0, policy_version 409571 (0.0008) [2023-12-27 08:28:17,839][20347] Updated weights for policy 0, policy_version 409581 (0.0010) [2023-12-27 08:28:17,896][20347] Updated weights for policy 0, policy_version 409591 (0.0010) [2023-12-27 08:28:17,919][20381] Updated weights for policy 1, policy_version 409139 (0.0007) [2023-12-27 08:28:17,976][20381] Updated weights for policy 1, policy_version 409149 (0.0010) [2023-12-27 08:28:17,986][20292] KL-divergence is very high: 110.5768 [2023-12-27 08:28:18,028][20381] Updated weights for policy 1, policy_version 409159 (0.0010) [2023-12-27 08:28:18,546][20347] Updated weights for policy 0, policy_version 409601 (0.0010) [2023-12-27 08:28:18,616][20347] Updated weights for policy 0, policy_version 409611 (0.0008) [2023-12-27 08:28:18,647][20381] Updated weights for policy 1, policy_version 409169 (0.0010) [2023-12-27 08:28:18,679][20347] Updated weights for policy 0, policy_version 409621 (0.0007) [2023-12-27 08:28:18,708][20381] Updated weights for policy 1, policy_version 409179 (0.0006) [2023-12-27 08:28:18,736][20347] Updated weights for policy 0, policy_version 409631 (0.0007) [2023-12-27 08:28:18,764][20381] Updated weights for policy 1, policy_version 409189 (0.0006) [2023-12-27 08:28:18,820][20381] Updated weights for policy 1, policy_version 409199 (0.0008) [2023-12-27 08:28:19,332][20347] Updated weights for policy 0, policy_version 409641 (0.0007) [2023-12-27 08:28:19,393][20347] Updated weights for policy 0, policy_version 409651 (0.0005) [2023-12-27 08:28:19,448][20347] Updated weights for policy 0, policy_version 409661 (0.0006) [2023-12-27 08:28:19,506][20381] Updated weights for policy 1, policy_version 409209 (0.0009) [2023-12-27 08:28:19,556][20381] Updated weights for policy 1, policy_version 409219 (0.0008) [2023-12-27 08:28:19,608][20381] Updated weights for policy 1, policy_version 409229 (0.0008) [2023-12-27 08:28:20,140][20347] Updated weights for policy 0, policy_version 409671 (0.0006) [2023-12-27 08:28:20,204][20347] Updated weights for policy 0, policy_version 409681 (0.0008) [2023-12-27 08:28:20,261][20347] Updated weights for policy 0, policy_version 409691 (0.0008) [2023-12-27 08:28:20,328][20381] Updated weights for policy 1, policy_version 409239 (0.0006) [2023-12-27 08:28:20,383][20381] Updated weights for policy 1, policy_version 409249 (0.0008) [2023-12-27 08:28:20,436][20381] Updated weights for policy 1, policy_version 409259 (0.0008) [2023-12-27 08:28:20,912][20347] Updated weights for policy 0, policy_version 409701 (0.0008) [2023-12-27 08:28:20,970][20347] Updated weights for policy 0, policy_version 409711 (0.0006) [2023-12-27 08:28:21,036][20347] Updated weights for policy 0, policy_version 409721 (0.0008) [2023-12-27 08:28:21,168][20381] Updated weights for policy 1, policy_version 409269 (0.0007) [2023-12-27 08:28:21,223][20381] Updated weights for policy 1, policy_version 409279 (0.0007) [2023-12-27 08:28:21,275][20381] Updated weights for policy 1, policy_version 409289 (0.0008) [2023-12-27 08:28:21,761][20347] Updated weights for policy 0, policy_version 409731 (0.0007) [2023-12-27 08:28:21,819][20347] Updated weights for policy 0, policy_version 409741 (0.0007) [2023-12-27 08:28:21,872][20347] Updated weights for policy 0, policy_version 409751 (0.0008) [2023-12-27 08:28:21,970][20381] Updated weights for policy 1, policy_version 409299 (0.0006) [2023-12-27 08:28:22,030][20381] Updated weights for policy 1, policy_version 409309 (0.0008) [2023-12-27 08:28:22,088][20381] Updated weights for policy 1, policy_version 409319 (0.0008) [2023-12-27 08:28:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 209723392. Throughput: 0: 10531.6, 1: 10327.1. Samples: 209692856. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:22,363][19170] Avg episode reward: [(0, '952.120'), (1, '717.840')] [2023-12-27 08:28:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000409328_104808448.pth... [2023-12-27 08:28:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000409760_104914944.pth... [2023-12-27 08:28:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000408512_104595456.pth [2023-12-27 08:28:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000408112_104497152.pth [2023-12-27 08:28:22,585][20347] Updated weights for policy 0, policy_version 409761 (0.0007) [2023-12-27 08:28:22,642][20347] Updated weights for policy 0, policy_version 409771 (0.0006) [2023-12-27 08:28:22,704][20347] Updated weights for policy 0, policy_version 409781 (0.0005) [2023-12-27 08:28:22,744][20381] Updated weights for policy 1, policy_version 409329 (0.0007) [2023-12-27 08:28:22,759][20347] Updated weights for policy 0, policy_version 409791 (0.0006) [2023-12-27 08:28:22,806][20381] Updated weights for policy 1, policy_version 409339 (0.0008) [2023-12-27 08:28:22,869][20381] Updated weights for policy 1, policy_version 409349 (0.0008) [2023-12-27 08:28:22,928][20381] Updated weights for policy 1, policy_version 409359 (0.0009) [2023-12-27 08:28:23,344][20347] Updated weights for policy 0, policy_version 409801 (0.0007) [2023-12-27 08:28:23,392][20347] Updated weights for policy 0, policy_version 409811 (0.0007) [2023-12-27 08:28:23,441][20347] Updated weights for policy 0, policy_version 409821 (0.0008) [2023-12-27 08:28:23,610][20381] Updated weights for policy 1, policy_version 409369 (0.0008) [2023-12-27 08:28:23,663][20381] Updated weights for policy 1, policy_version 409379 (0.0007) [2023-12-27 08:28:23,722][20381] Updated weights for policy 1, policy_version 409389 (0.0008) [2023-12-27 08:28:24,133][20347] Updated weights for policy 0, policy_version 409831 (0.0006) [2023-12-27 08:28:24,188][20347] Updated weights for policy 0, policy_version 409841 (0.0007) [2023-12-27 08:28:24,234][20347] Updated weights for policy 0, policy_version 409851 (0.0009) [2023-12-27 08:28:24,425][20381] Updated weights for policy 1, policy_version 409399 (0.0008) [2023-12-27 08:28:24,478][20381] Updated weights for policy 1, policy_version 409409 (0.0009) [2023-12-27 08:28:24,538][20381] Updated weights for policy 1, policy_version 409419 (0.0007) [2023-12-27 08:28:24,791][20347] Updated weights for policy 0, policy_version 409861 (0.0008) [2023-12-27 08:28:24,840][20347] Updated weights for policy 0, policy_version 409871 (0.0007) [2023-12-27 08:28:24,886][20347] Updated weights for policy 0, policy_version 409881 (0.0008) [2023-12-27 08:28:25,209][20381] Updated weights for policy 1, policy_version 409429 (0.0009) [2023-12-27 08:28:25,259][20381] Updated weights for policy 1, policy_version 409439 (0.0008) [2023-12-27 08:28:25,317][20381] Updated weights for policy 1, policy_version 409449 (0.0007) [2023-12-27 08:28:25,550][20347] Updated weights for policy 0, policy_version 409891 (0.0008) [2023-12-27 08:28:25,600][20347] Updated weights for policy 0, policy_version 409901 (0.0009) [2023-12-27 08:28:25,657][20347] Updated weights for policy 0, policy_version 409911 (0.0009) [2023-12-27 08:28:25,905][20381] Updated weights for policy 1, policy_version 409459 (0.0006) [2023-12-27 08:28:25,954][20381] Updated weights for policy 1, policy_version 409469 (0.0008) [2023-12-27 08:28:26,001][20381] Updated weights for policy 1, policy_version 409479 (0.0008) [2023-12-27 08:28:26,379][20347] Updated weights for policy 0, policy_version 409921 (0.0008) [2023-12-27 08:28:26,434][20347] Updated weights for policy 0, policy_version 409931 (0.0008) [2023-12-27 08:28:26,489][20347] Updated weights for policy 0, policy_version 409941 (0.0008) [2023-12-27 08:28:26,539][20347] Updated weights for policy 0, policy_version 409951 (0.0007) [2023-12-27 08:28:26,685][20381] Updated weights for policy 1, policy_version 409489 (0.0007) [2023-12-27 08:28:26,739][20381] Updated weights for policy 1, policy_version 409499 (0.0010) [2023-12-27 08:28:26,787][20381] Updated weights for policy 1, policy_version 409509 (0.0010) [2023-12-27 08:28:26,829][20381] Updated weights for policy 1, policy_version 409519 (0.0010) [2023-12-27 08:28:27,192][20347] Updated weights for policy 0, policy_version 409961 (0.0005) [2023-12-27 08:28:27,248][20347] Updated weights for policy 0, policy_version 409971 (0.0006) [2023-12-27 08:28:27,300][20347] Updated weights for policy 0, policy_version 409981 (0.0005) [2023-12-27 08:28:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 209829888. Throughput: 0: 10531.3, 1: 10308.8. Samples: 209816504. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:27,363][19170] Avg episode reward: [(0, '923.060'), (1, '726.520')] [2023-12-27 08:28:27,490][20292] KL-divergence is very high: 111.5241 [2023-12-27 08:28:27,502][20381] Updated weights for policy 1, policy_version 409529 (0.0009) [2023-12-27 08:28:27,580][20381] Updated weights for policy 1, policy_version 409539 (0.0010) [2023-12-27 08:28:27,632][20381] Updated weights for policy 1, policy_version 409549 (0.0009) [2023-12-27 08:28:27,900][20347] Updated weights for policy 0, policy_version 409991 (0.0007) [2023-12-27 08:28:27,951][20347] Updated weights for policy 0, policy_version 410001 (0.0008) [2023-12-27 08:28:27,996][20347] Updated weights for policy 0, policy_version 410011 (0.0008) [2023-12-27 08:28:28,178][20381] Updated weights for policy 1, policy_version 409559 (0.0008) [2023-12-27 08:28:28,245][20381] Updated weights for policy 1, policy_version 409569 (0.0008) [2023-12-27 08:28:28,298][20381] Updated weights for policy 1, policy_version 409579 (0.0008) [2023-12-27 08:28:28,714][20347] Updated weights for policy 0, policy_version 410021 (0.0009) [2023-12-27 08:28:28,769][20347] Updated weights for policy 0, policy_version 410031 (0.0010) [2023-12-27 08:28:28,829][20347] Updated weights for policy 0, policy_version 410041 (0.0010) [2023-12-27 08:28:28,948][20381] Updated weights for policy 1, policy_version 409589 (0.0007) [2023-12-27 08:28:28,994][20381] Updated weights for policy 1, policy_version 409599 (0.0007) [2023-12-27 08:28:29,043][20381] Updated weights for policy 1, policy_version 409609 (0.0006) [2023-12-27 08:28:29,461][20347] Updated weights for policy 0, policy_version 410051 (0.0008) [2023-12-27 08:28:29,521][20347] Updated weights for policy 0, policy_version 410061 (0.0010) [2023-12-27 08:28:29,572][20347] Updated weights for policy 0, policy_version 410071 (0.0010) [2023-12-27 08:28:29,744][20381] Updated weights for policy 1, policy_version 409619 (0.0008) [2023-12-27 08:28:29,797][20381] Updated weights for policy 1, policy_version 409629 (0.0008) [2023-12-27 08:28:29,853][20381] Updated weights for policy 1, policy_version 409639 (0.0008) [2023-12-27 08:28:30,231][20347] Updated weights for policy 0, policy_version 410081 (0.0006) [2023-12-27 08:28:30,287][20347] Updated weights for policy 0, policy_version 410091 (0.0009) [2023-12-27 08:28:30,350][20347] Updated weights for policy 0, policy_version 410101 (0.0006) [2023-12-27 08:28:30,411][20347] Updated weights for policy 0, policy_version 410111 (0.0006) [2023-12-27 08:28:30,583][20381] Updated weights for policy 1, policy_version 409649 (0.0007) [2023-12-27 08:28:30,630][20381] Updated weights for policy 1, policy_version 409659 (0.0007) [2023-12-27 08:28:30,686][20381] Updated weights for policy 1, policy_version 409669 (0.0008) [2023-12-27 08:28:30,744][20381] Updated weights for policy 1, policy_version 409679 (0.0006) [2023-12-27 08:28:31,028][20347] Updated weights for policy 0, policy_version 410121 (0.0010) [2023-12-27 08:28:31,085][20347] Updated weights for policy 0, policy_version 410131 (0.0010) [2023-12-27 08:28:31,147][20347] Updated weights for policy 0, policy_version 410141 (0.0010) [2023-12-27 08:28:31,394][20381] Updated weights for policy 1, policy_version 409689 (0.0008) [2023-12-27 08:28:31,454][20381] Updated weights for policy 1, policy_version 409699 (0.0008) [2023-12-27 08:28:31,504][20381] Updated weights for policy 1, policy_version 409709 (0.0008) [2023-12-27 08:28:31,863][20347] Updated weights for policy 0, policy_version 410151 (0.0008) [2023-12-27 08:28:31,921][20347] Updated weights for policy 0, policy_version 410161 (0.0007) [2023-12-27 08:28:31,974][20347] Updated weights for policy 0, policy_version 410171 (0.0007) [2023-12-27 08:28:32,222][20381] Updated weights for policy 1, policy_version 409719 (0.0008) [2023-12-27 08:28:32,284][20381] Updated weights for policy 1, policy_version 409729 (0.0008) [2023-12-27 08:28:32,334][20381] Updated weights for policy 1, policy_version 409739 (0.0007) [2023-12-27 08:28:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 209928192. Throughput: 0: 10582.5, 1: 10293.6. Samples: 209942212. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:32,363][19170] Avg episode reward: [(0, '883.420'), (1, '847.820')] [2023-12-27 08:28:32,709][20347] Updated weights for policy 0, policy_version 410181 (0.0008) [2023-12-27 08:28:32,768][20347] Updated weights for policy 0, policy_version 410191 (0.0007) [2023-12-27 08:28:32,830][20347] Updated weights for policy 0, policy_version 410201 (0.0008) [2023-12-27 08:28:32,991][20381] Updated weights for policy 1, policy_version 409749 (0.0009) [2023-12-27 08:28:33,035][20381] Updated weights for policy 1, policy_version 409759 (0.0008) [2023-12-27 08:28:33,087][20381] Updated weights for policy 1, policy_version 409769 (0.0010) [2023-12-27 08:28:33,536][20347] Updated weights for policy 0, policy_version 410211 (0.0007) [2023-12-27 08:28:33,582][20347] Updated weights for policy 0, policy_version 410221 (0.0006) [2023-12-27 08:28:33,633][20347] Updated weights for policy 0, policy_version 410231 (0.0005) [2023-12-27 08:28:33,704][20381] Updated weights for policy 1, policy_version 409779 (0.0010) [2023-12-27 08:28:33,751][20381] Updated weights for policy 1, policy_version 409789 (0.0008) [2023-12-27 08:28:33,804][20381] Updated weights for policy 1, policy_version 409799 (0.0005) [2023-12-27 08:28:34,237][20347] Updated weights for policy 0, policy_version 410241 (0.0005) [2023-12-27 08:28:34,292][20347] Updated weights for policy 0, policy_version 410251 (0.0005) [2023-12-27 08:28:34,353][20347] Updated weights for policy 0, policy_version 410261 (0.0010) [2023-12-27 08:28:34,417][20347] Updated weights for policy 0, policy_version 410271 (0.0008) [2023-12-27 08:28:34,436][20381] Updated weights for policy 1, policy_version 409809 (0.0006) [2023-12-27 08:28:34,494][20381] Updated weights for policy 1, policy_version 409819 (0.0008) [2023-12-27 08:28:34,546][20381] Updated weights for policy 1, policy_version 409829 (0.0007) [2023-12-27 08:28:34,604][20381] Updated weights for policy 1, policy_version 409839 (0.0007) [2023-12-27 08:28:35,023][20347] Updated weights for policy 0, policy_version 410281 (0.0006) [2023-12-27 08:28:35,074][20347] Updated weights for policy 0, policy_version 410291 (0.0005) [2023-12-27 08:28:35,133][20347] Updated weights for policy 0, policy_version 410301 (0.0005) [2023-12-27 08:28:35,323][20381] Updated weights for policy 1, policy_version 409849 (0.0006) [2023-12-27 08:28:35,372][20381] Updated weights for policy 1, policy_version 409859 (0.0005) [2023-12-27 08:28:35,420][20381] Updated weights for policy 1, policy_version 409869 (0.0005) [2023-12-27 08:28:35,697][20347] Updated weights for policy 0, policy_version 410311 (0.0009) [2023-12-27 08:28:35,750][20347] Updated weights for policy 0, policy_version 410321 (0.0010) [2023-12-27 08:28:35,810][20347] Updated weights for policy 0, policy_version 410331 (0.0010) [2023-12-27 08:28:36,001][20381] Updated weights for policy 1, policy_version 409879 (0.0007) [2023-12-27 08:28:36,076][20381] Updated weights for policy 1, policy_version 409889 (0.0008) [2023-12-27 08:28:36,136][20381] Updated weights for policy 1, policy_version 409899 (0.0008) [2023-12-27 08:28:36,488][20347] Updated weights for policy 0, policy_version 410341 (0.0009) [2023-12-27 08:28:36,547][20347] Updated weights for policy 0, policy_version 410351 (0.0008) [2023-12-27 08:28:36,608][20347] Updated weights for policy 0, policy_version 410361 (0.0008) [2023-12-27 08:28:36,831][20381] Updated weights for policy 1, policy_version 409909 (0.0006) [2023-12-27 08:28:36,892][20381] Updated weights for policy 1, policy_version 409919 (0.0008) [2023-12-27 08:28:36,954][20381] Updated weights for policy 1, policy_version 409929 (0.0006) [2023-12-27 08:28:37,250][20347] Updated weights for policy 0, policy_version 410371 (0.0009) [2023-12-27 08:28:37,306][20347] Updated weights for policy 0, policy_version 410381 (0.0010) [2023-12-27 08:28:37,360][20347] Updated weights for policy 0, policy_version 410391 (0.0009) [2023-12-27 08:28:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 210034688. Throughput: 0: 10586.6, 1: 10333.9. Samples: 210006024. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:37,363][19170] Avg episode reward: [(0, '854.690'), (1, '888.870')] [2023-12-27 08:28:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000409936_104964096.pth... [2023-12-27 08:28:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000408720_104652800.pth [2023-12-27 08:28:37,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000410400_105078784.pth... [2023-12-27 08:28:37,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000409152_104759296.pth [2023-12-27 08:28:37,604][20381] Updated weights for policy 1, policy_version 409939 (0.0009) [2023-12-27 08:28:37,662][20381] Updated weights for policy 1, policy_version 409949 (0.0008) [2023-12-27 08:28:37,723][20381] Updated weights for policy 1, policy_version 409959 (0.0010) [2023-12-27 08:28:38,007][20347] Updated weights for policy 0, policy_version 410401 (0.0007) [2023-12-27 08:28:38,059][20347] Updated weights for policy 0, policy_version 410411 (0.0010) [2023-12-27 08:28:38,110][20347] Updated weights for policy 0, policy_version 410421 (0.0005) [2023-12-27 08:28:38,162][20347] Updated weights for policy 0, policy_version 410431 (0.0010) [2023-12-27 08:28:38,330][20381] Updated weights for policy 1, policy_version 409969 (0.0010) [2023-12-27 08:28:38,391][20381] Updated weights for policy 1, policy_version 409979 (0.0011) [2023-12-27 08:28:38,440][20381] Updated weights for policy 1, policy_version 409989 (0.0010) [2023-12-27 08:28:38,501][20381] Updated weights for policy 1, policy_version 409999 (0.0010) [2023-12-27 08:28:38,835][20347] Updated weights for policy 0, policy_version 410441 (0.0007) [2023-12-27 08:28:38,893][20347] Updated weights for policy 0, policy_version 410451 (0.0010) [2023-12-27 08:28:38,950][20347] Updated weights for policy 0, policy_version 410461 (0.0010) [2023-12-27 08:28:39,149][20381] Updated weights for policy 1, policy_version 410009 (0.0006) [2023-12-27 08:28:39,202][20381] Updated weights for policy 1, policy_version 410019 (0.0006) [2023-12-27 08:28:39,266][20381] Updated weights for policy 1, policy_version 410029 (0.0008) [2023-12-27 08:28:39,664][20347] Updated weights for policy 0, policy_version 410471 (0.0008) [2023-12-27 08:28:39,727][20347] Updated weights for policy 0, policy_version 410481 (0.0007) [2023-12-27 08:28:39,790][20347] Updated weights for policy 0, policy_version 410491 (0.0008) [2023-12-27 08:28:39,987][20381] Updated weights for policy 1, policy_version 410039 (0.0008) [2023-12-27 08:28:40,049][20381] Updated weights for policy 1, policy_version 410049 (0.0007) [2023-12-27 08:28:40,112][20381] Updated weights for policy 1, policy_version 410059 (0.0007) [2023-12-27 08:28:40,503][20347] Updated weights for policy 0, policy_version 410501 (0.0007) [2023-12-27 08:28:40,561][20347] Updated weights for policy 0, policy_version 410511 (0.0006) [2023-12-27 08:28:40,619][20347] Updated weights for policy 0, policy_version 410521 (0.0006) [2023-12-27 08:28:40,801][20381] Updated weights for policy 1, policy_version 410069 (0.0007) [2023-12-27 08:28:40,865][20381] Updated weights for policy 1, policy_version 410079 (0.0005) [2023-12-27 08:28:40,917][20381] Updated weights for policy 1, policy_version 410089 (0.0006) [2023-12-27 08:28:41,256][20347] Updated weights for policy 0, policy_version 410531 (0.0006) [2023-12-27 08:28:41,316][20347] Updated weights for policy 0, policy_version 410541 (0.0008) [2023-12-27 08:28:41,382][20347] Updated weights for policy 0, policy_version 410551 (0.0007) [2023-12-27 08:28:41,634][20381] Updated weights for policy 1, policy_version 410099 (0.0008) [2023-12-27 08:28:41,695][20381] Updated weights for policy 1, policy_version 410109 (0.0008) [2023-12-27 08:28:41,764][20381] Updated weights for policy 1, policy_version 410119 (0.0007) [2023-12-27 08:28:42,051][20347] Updated weights for policy 0, policy_version 410561 (0.0007) [2023-12-27 08:28:42,103][20347] Updated weights for policy 0, policy_version 410571 (0.0007) [2023-12-27 08:28:42,163][20347] Updated weights for policy 0, policy_version 410581 (0.0009) [2023-12-27 08:28:42,220][20347] Updated weights for policy 0, policy_version 410591 (0.0008) [2023-12-27 08:28:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 210141184. Throughput: 0: 10539.6, 1: 10345.4. Samples: 210131124. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:42,363][19170] Avg episode reward: [(0, '827.060'), (1, '907.330')] [2023-12-27 08:28:42,392][20381] Updated weights for policy 1, policy_version 410129 (0.0008) [2023-12-27 08:28:42,453][20381] Updated weights for policy 1, policy_version 410139 (0.0008) [2023-12-27 08:28:42,511][20381] Updated weights for policy 1, policy_version 410149 (0.0007) [2023-12-27 08:28:42,570][20381] Updated weights for policy 1, policy_version 410159 (0.0008) [2023-12-27 08:28:42,896][20347] Updated weights for policy 0, policy_version 410601 (0.0007) [2023-12-27 08:28:42,946][20347] Updated weights for policy 0, policy_version 410611 (0.0006) [2023-12-27 08:28:42,995][20347] Updated weights for policy 0, policy_version 410621 (0.0005) [2023-12-27 08:28:43,268][20381] Updated weights for policy 1, policy_version 410169 (0.0008) [2023-12-27 08:28:43,331][20381] Updated weights for policy 1, policy_version 410179 (0.0007) [2023-12-27 08:28:43,389][20381] Updated weights for policy 1, policy_version 410189 (0.0008) [2023-12-27 08:28:43,673][20347] Updated weights for policy 0, policy_version 410631 (0.0007) [2023-12-27 08:28:43,721][20347] Updated weights for policy 0, policy_version 410641 (0.0007) [2023-12-27 08:28:43,768][20347] Updated weights for policy 0, policy_version 410651 (0.0007) [2023-12-27 08:28:44,094][20381] Updated weights for policy 1, policy_version 410199 (0.0008) [2023-12-27 08:28:44,146][20381] Updated weights for policy 1, policy_version 410209 (0.0007) [2023-12-27 08:28:44,195][20381] Updated weights for policy 1, policy_version 410219 (0.0007) [2023-12-27 08:28:44,473][20347] Updated weights for policy 0, policy_version 410661 (0.0008) [2023-12-27 08:28:44,534][20347] Updated weights for policy 0, policy_version 410671 (0.0007) [2023-12-27 08:28:44,588][20347] Updated weights for policy 0, policy_version 410681 (0.0006) [2023-12-27 08:28:44,898][20381] Updated weights for policy 1, policy_version 410229 (0.0009) [2023-12-27 08:28:44,964][20381] Updated weights for policy 1, policy_version 410239 (0.0010) [2023-12-27 08:28:45,025][20381] Updated weights for policy 1, policy_version 410249 (0.0007) [2023-12-27 08:28:45,169][20347] Updated weights for policy 0, policy_version 410691 (0.0006) [2023-12-27 08:28:45,231][20347] Updated weights for policy 0, policy_version 410701 (0.0006) [2023-12-27 08:28:45,295][20347] Updated weights for policy 0, policy_version 410711 (0.0008) [2023-12-27 08:28:45,629][20381] Updated weights for policy 1, policy_version 410259 (0.0007) [2023-12-27 08:28:45,674][20381] Updated weights for policy 1, policy_version 410269 (0.0010) [2023-12-27 08:28:45,723][20381] Updated weights for policy 1, policy_version 410279 (0.0010) [2023-12-27 08:28:45,985][20347] Updated weights for policy 0, policy_version 410721 (0.0009) [2023-12-27 08:28:46,041][20347] Updated weights for policy 0, policy_version 410731 (0.0007) [2023-12-27 08:28:46,093][20347] Updated weights for policy 0, policy_version 410741 (0.0007) [2023-12-27 08:28:46,142][20347] Updated weights for policy 0, policy_version 410751 (0.0008) [2023-12-27 08:28:46,387][20381] Updated weights for policy 1, policy_version 410289 (0.0005) [2023-12-27 08:28:46,438][20381] Updated weights for policy 1, policy_version 410299 (0.0007) [2023-12-27 08:28:46,492][20381] Updated weights for policy 1, policy_version 410309 (0.0007) [2023-12-27 08:28:46,548][20381] Updated weights for policy 1, policy_version 410319 (0.0007) [2023-12-27 08:28:46,839][20347] Updated weights for policy 0, policy_version 410761 (0.0008) [2023-12-27 08:28:46,891][20347] Updated weights for policy 0, policy_version 410771 (0.0008) [2023-12-27 08:28:46,946][20347] Updated weights for policy 0, policy_version 410781 (0.0008) [2023-12-27 08:28:47,150][20381] Updated weights for policy 1, policy_version 410329 (0.0008) [2023-12-27 08:28:47,197][20381] Updated weights for policy 1, policy_version 410339 (0.0009) [2023-12-27 08:28:47,246][20381] Updated weights for policy 1, policy_version 410349 (0.0009) [2023-12-27 08:28:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20827.1). Total num frames: 210247680. Throughput: 0: 10529.5, 1: 10388.9. Samples: 210255068. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:47,363][19170] Avg episode reward: [(0, '865.910'), (1, '934.800')] [2023-12-27 08:28:47,645][20347] Updated weights for policy 0, policy_version 410791 (0.0008) [2023-12-27 08:28:47,690][20347] Updated weights for policy 0, policy_version 410801 (0.0008) [2023-12-27 08:28:47,735][20347] Updated weights for policy 0, policy_version 410811 (0.0007) [2023-12-27 08:28:47,931][20381] Updated weights for policy 1, policy_version 410359 (0.0008) [2023-12-27 08:28:47,986][20381] Updated weights for policy 1, policy_version 410369 (0.0008) [2023-12-27 08:28:48,038][20381] Updated weights for policy 1, policy_version 410379 (0.0008) [2023-12-27 08:28:48,475][20347] Updated weights for policy 0, policy_version 410821 (0.0007) [2023-12-27 08:28:48,534][20347] Updated weights for policy 0, policy_version 410831 (0.0006) [2023-12-27 08:28:48,592][20347] Updated weights for policy 0, policy_version 410841 (0.0007) [2023-12-27 08:28:48,683][20381] Updated weights for policy 1, policy_version 410389 (0.0007) [2023-12-27 08:28:48,743][20381] Updated weights for policy 1, policy_version 410399 (0.0008) [2023-12-27 08:28:48,790][20381] Updated weights for policy 1, policy_version 410409 (0.0008) [2023-12-27 08:28:49,192][20347] Updated weights for policy 0, policy_version 410851 (0.0006) [2023-12-27 08:28:49,254][20347] Updated weights for policy 0, policy_version 410861 (0.0008) [2023-12-27 08:28:49,315][20347] Updated weights for policy 0, policy_version 410871 (0.0010) [2023-12-27 08:28:49,386][20381] Updated weights for policy 1, policy_version 410419 (0.0008) [2023-12-27 08:28:49,448][20381] Updated weights for policy 1, policy_version 410429 (0.0007) [2023-12-27 08:28:49,493][20381] Updated weights for policy 1, policy_version 410439 (0.0008) [2023-12-27 08:28:50,034][20347] Updated weights for policy 0, policy_version 410881 (0.0009) [2023-12-27 08:28:50,086][20347] Updated weights for policy 0, policy_version 410891 (0.0011) [2023-12-27 08:28:50,138][20347] Updated weights for policy 0, policy_version 410901 (0.0010) [2023-12-27 08:28:50,196][20347] Updated weights for policy 0, policy_version 410911 (0.0010) [2023-12-27 08:28:50,236][20381] Updated weights for policy 1, policy_version 410449 (0.0007) [2023-12-27 08:28:50,287][20381] Updated weights for policy 1, policy_version 410459 (0.0007) [2023-12-27 08:28:50,341][20381] Updated weights for policy 1, policy_version 410469 (0.0007) [2023-12-27 08:28:50,407][20381] Updated weights for policy 1, policy_version 410479 (0.0007) [2023-12-27 08:28:50,843][20347] Updated weights for policy 0, policy_version 410921 (0.0006) [2023-12-27 08:28:50,902][20347] Updated weights for policy 0, policy_version 410931 (0.0005) [2023-12-27 08:28:50,965][20347] Updated weights for policy 0, policy_version 410941 (0.0005) [2023-12-27 08:28:51,171][20381] Updated weights for policy 1, policy_version 410489 (0.0008) [2023-12-27 08:28:51,233][20381] Updated weights for policy 1, policy_version 410499 (0.0006) [2023-12-27 08:28:51,295][20381] Updated weights for policy 1, policy_version 410509 (0.0011) [2023-12-27 08:28:51,592][20347] Updated weights for policy 0, policy_version 410951 (0.0008) [2023-12-27 08:28:51,657][20347] Updated weights for policy 0, policy_version 410961 (0.0008) [2023-12-27 08:28:51,721][20347] Updated weights for policy 0, policy_version 410971 (0.0007) [2023-12-27 08:28:51,986][20381] Updated weights for policy 1, policy_version 410519 (0.0007) [2023-12-27 08:28:52,035][20381] Updated weights for policy 1, policy_version 410529 (0.0005) [2023-12-27 08:28:52,085][20381] Updated weights for policy 1, policy_version 410539 (0.0005) [2023-12-27 08:28:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 210345984. Throughput: 0: 10493.0, 1: 10405.1. Samples: 210317596. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:52,363][19170] Avg episode reward: [(0, '903.860'), (1, '944.400')] [2023-12-27 08:28:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000410544_105119744.pth... [2023-12-27 08:28:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000409328_104808448.pth [2023-12-27 08:28:52,386][20347] Updated weights for policy 0, policy_version 410981 (0.0008) [2023-12-27 08:28:52,449][20347] Updated weights for policy 0, policy_version 410991 (0.0008) [2023-12-27 08:28:52,513][20347] Updated weights for policy 0, policy_version 411001 (0.0007) [2023-12-27 08:28:52,556][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000411008_105234432.pth... [2023-12-27 08:28:52,561][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000409760_104914944.pth [2023-12-27 08:28:52,651][20381] Updated weights for policy 1, policy_version 410549 (0.0007) [2023-12-27 08:28:52,702][20381] Updated weights for policy 1, policy_version 410559 (0.0009) [2023-12-27 08:28:52,758][20381] Updated weights for policy 1, policy_version 410569 (0.0009) [2023-12-27 08:28:53,108][20347] Updated weights for policy 0, policy_version 411011 (0.0008) [2023-12-27 08:28:53,166][20347] Updated weights for policy 0, policy_version 411021 (0.0006) [2023-12-27 08:28:53,222][20347] Updated weights for policy 0, policy_version 411031 (0.0005) [2023-12-27 08:28:53,439][20381] Updated weights for policy 1, policy_version 410579 (0.0009) [2023-12-27 08:28:53,495][20381] Updated weights for policy 1, policy_version 410589 (0.0005) [2023-12-27 08:28:53,544][20381] Updated weights for policy 1, policy_version 410599 (0.0005) [2023-12-27 08:28:53,842][20347] Updated weights for policy 0, policy_version 411041 (0.0005) [2023-12-27 08:28:53,899][20347] Updated weights for policy 0, policy_version 411051 (0.0005) [2023-12-27 08:28:53,961][20347] Updated weights for policy 0, policy_version 411061 (0.0005) [2023-12-27 08:28:54,020][20347] Updated weights for policy 0, policy_version 411071 (0.0005) [2023-12-27 08:28:54,161][20381] Updated weights for policy 1, policy_version 410609 (0.0005) [2023-12-27 08:28:54,221][20381] Updated weights for policy 1, policy_version 410619 (0.0007) [2023-12-27 08:28:54,282][20381] Updated weights for policy 1, policy_version 410629 (0.0006) [2023-12-27 08:28:54,330][20381] Updated weights for policy 1, policy_version 410639 (0.0008) [2023-12-27 08:28:54,598][20347] Updated weights for policy 0, policy_version 411081 (0.0005) [2023-12-27 08:28:54,647][20347] Updated weights for policy 0, policy_version 411091 (0.0006) [2023-12-27 08:28:54,697][20347] Updated weights for policy 0, policy_version 411101 (0.0006) [2023-12-27 08:28:55,000][20381] Updated weights for policy 1, policy_version 410649 (0.0007) [2023-12-27 08:28:55,062][20381] Updated weights for policy 1, policy_version 410659 (0.0007) [2023-12-27 08:28:55,119][20381] Updated weights for policy 1, policy_version 410669 (0.0008) [2023-12-27 08:28:55,360][20347] Updated weights for policy 0, policy_version 411111 (0.0008) [2023-12-27 08:28:55,413][20347] Updated weights for policy 0, policy_version 411121 (0.0008) [2023-12-27 08:28:55,458][20347] Updated weights for policy 0, policy_version 411131 (0.0005) [2023-12-27 08:28:55,778][20381] Updated weights for policy 1, policy_version 410679 (0.0007) [2023-12-27 08:28:55,835][20381] Updated weights for policy 1, policy_version 410689 (0.0007) [2023-12-27 08:28:55,882][20381] Updated weights for policy 1, policy_version 410699 (0.0008) [2023-12-27 08:28:56,087][20347] Updated weights for policy 0, policy_version 411141 (0.0007) [2023-12-27 08:28:56,137][20347] Updated weights for policy 0, policy_version 411151 (0.0006) [2023-12-27 08:28:56,182][20347] Updated weights for policy 0, policy_version 411161 (0.0005) [2023-12-27 08:28:56,578][20381] Updated weights for policy 1, policy_version 410709 (0.0007) [2023-12-27 08:28:56,643][20381] Updated weights for policy 1, policy_version 410719 (0.0007) [2023-12-27 08:28:56,716][20381] Updated weights for policy 1, policy_version 410729 (0.0007) [2023-12-27 08:28:56,789][20347] Updated weights for policy 0, policy_version 411171 (0.0006) [2023-12-27 08:28:56,848][20347] Updated weights for policy 0, policy_version 411181 (0.0011) [2023-12-27 08:28:56,903][20347] Updated weights for policy 0, policy_version 411191 (0.0010) [2023-12-27 08:28:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 210452480. Throughput: 0: 10488.5, 1: 10431.6. Samples: 210444872. Policy #0 lag: (min: 22.0, avg: 26.6, max: 54.0) [2023-12-27 08:28:57,363][19170] Avg episode reward: [(0, '923.060'), (1, '972.620')] [2023-12-27 08:28:57,377][20381] Updated weights for policy 1, policy_version 410739 (0.0007) [2023-12-27 08:28:57,442][20381] Updated weights for policy 1, policy_version 410749 (0.0008) [2023-12-27 08:28:57,493][20381] Updated weights for policy 1, policy_version 410759 (0.0007) [2023-12-27 08:28:57,580][20347] Updated weights for policy 0, policy_version 411201 (0.0010) [2023-12-27 08:28:57,628][20347] Updated weights for policy 0, policy_version 411211 (0.0009) [2023-12-27 08:28:57,676][20347] Updated weights for policy 0, policy_version 411221 (0.0009) [2023-12-27 08:28:57,729][20347] Updated weights for policy 0, policy_version 411231 (0.0008) [2023-12-27 08:28:58,096][20381] Updated weights for policy 1, policy_version 410769 (0.0006) [2023-12-27 08:28:58,150][20381] Updated weights for policy 1, policy_version 410779 (0.0006) [2023-12-27 08:28:58,216][20381] Updated weights for policy 1, policy_version 410789 (0.0007) [2023-12-27 08:28:58,281][20381] Updated weights for policy 1, policy_version 410799 (0.0007) [2023-12-27 08:28:58,460][20347] Updated weights for policy 0, policy_version 411241 (0.0008) [2023-12-27 08:28:58,524][20347] Updated weights for policy 0, policy_version 411251 (0.0008) [2023-12-27 08:28:58,582][20347] Updated weights for policy 0, policy_version 411261 (0.0008) [2023-12-27 08:28:58,964][20381] Updated weights for policy 1, policy_version 410809 (0.0008) [2023-12-27 08:28:59,018][20381] Updated weights for policy 1, policy_version 410819 (0.0006) [2023-12-27 08:28:59,077][20381] Updated weights for policy 1, policy_version 410829 (0.0005) [2023-12-27 08:28:59,356][20347] Updated weights for policy 0, policy_version 411271 (0.0007) [2023-12-27 08:28:59,416][20347] Updated weights for policy 0, policy_version 411281 (0.0008) [2023-12-27 08:28:59,469][20347] Updated weights for policy 0, policy_version 411292 (0.0009) [2023-12-27 08:28:59,764][20381] Updated weights for policy 1, policy_version 410839 (0.0007) [2023-12-27 08:28:59,821][20381] Updated weights for policy 1, policy_version 410849 (0.0006) [2023-12-27 08:28:59,881][20381] Updated weights for policy 1, policy_version 410859 (0.0008) [2023-12-27 08:29:00,222][20347] Updated weights for policy 0, policy_version 411302 (0.0008) [2023-12-27 08:29:00,274][20347] Updated weights for policy 0, policy_version 411312 (0.0008) [2023-12-27 08:29:00,322][20347] Updated weights for policy 0, policy_version 411322 (0.0008) [2023-12-27 08:29:00,521][20381] Updated weights for policy 1, policy_version 410869 (0.0008) [2023-12-27 08:29:00,576][20381] Updated weights for policy 1, policy_version 410879 (0.0007) [2023-12-27 08:29:00,632][20381] Updated weights for policy 1, policy_version 410889 (0.0007) [2023-12-27 08:29:01,017][20347] Updated weights for policy 0, policy_version 411332 (0.0007) [2023-12-27 08:29:01,054][20291] KL-divergence is very high: 114.9763 [2023-12-27 08:29:01,079][20347] Updated weights for policy 0, policy_version 411342 (0.0007) [2023-12-27 08:29:01,138][20347] Updated weights for policy 0, policy_version 411352 (0.0008) [2023-12-27 08:29:01,341][20381] Updated weights for policy 1, policy_version 410899 (0.0007) [2023-12-27 08:29:01,411][20381] Updated weights for policy 1, policy_version 410909 (0.0008) [2023-12-27 08:29:01,420][20292] KL-divergence is very high: 122.0206 [2023-12-27 08:29:01,477][20381] Updated weights for policy 1, policy_version 410919 (0.0005) [2023-12-27 08:29:01,751][20347] Updated weights for policy 0, policy_version 411362 (0.0008) [2023-12-27 08:29:01,802][20347] Updated weights for policy 0, policy_version 411372 (0.0008) [2023-12-27 08:29:01,849][20347] Updated weights for policy 0, policy_version 411382 (0.0008) [2023-12-27 08:29:01,908][20347] Updated weights for policy 0, policy_version 411392 (0.0007) [2023-12-27 08:29:02,154][20381] Updated weights for policy 1, policy_version 410929 (0.0006) [2023-12-27 08:29:02,215][20381] Updated weights for policy 1, policy_version 410939 (0.0006) [2023-12-27 08:29:02,282][20381] Updated weights for policy 1, policy_version 410949 (0.0006) [2023-12-27 08:29:02,352][20381] Updated weights for policy 1, policy_version 410959 (0.0006) [2023-12-27 08:29:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 210558976. Throughput: 0: 10426.4, 1: 10403.4. Samples: 210567044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:02,363][19170] Avg episode reward: [(0, '943.050'), (1, '972.590')] [2023-12-27 08:29:02,645][20347] Updated weights for policy 0, policy_version 411402 (0.0008) [2023-12-27 08:29:02,693][20347] Updated weights for policy 0, policy_version 411412 (0.0008) [2023-12-27 08:29:02,749][20347] Updated weights for policy 0, policy_version 411422 (0.0008) [2023-12-27 08:29:02,996][20381] Updated weights for policy 1, policy_version 410969 (0.0007) [2023-12-27 08:29:03,053][20381] Updated weights for policy 1, policy_version 410979 (0.0006) [2023-12-27 08:29:03,108][20381] Updated weights for policy 1, policy_version 410989 (0.0005) [2023-12-27 08:29:03,468][20347] Updated weights for policy 0, policy_version 411432 (0.0008) [2023-12-27 08:29:03,521][20347] Updated weights for policy 0, policy_version 411442 (0.0006) [2023-12-27 08:29:03,576][20347] Updated weights for policy 0, policy_version 411452 (0.0006) [2023-12-27 08:29:03,709][20381] Updated weights for policy 1, policy_version 410999 (0.0006) [2023-12-27 08:29:03,757][20381] Updated weights for policy 1, policy_version 411009 (0.0009) [2023-12-27 08:29:03,807][20381] Updated weights for policy 1, policy_version 411019 (0.0008) [2023-12-27 08:29:04,261][20347] Updated weights for policy 0, policy_version 411462 (0.0010) [2023-12-27 08:29:04,322][20347] Updated weights for policy 0, policy_version 411472 (0.0010) [2023-12-27 08:29:04,380][20347] Updated weights for policy 0, policy_version 411482 (0.0010) [2023-12-27 08:29:04,522][20381] Updated weights for policy 1, policy_version 411029 (0.0008) [2023-12-27 08:29:04,583][20381] Updated weights for policy 1, policy_version 411039 (0.0007) [2023-12-27 08:29:04,619][20292] KL-divergence is very high: 147.5683 [2023-12-27 08:29:04,642][20381] Updated weights for policy 1, policy_version 411049 (0.0007) [2023-12-27 08:29:05,016][20347] Updated weights for policy 0, policy_version 411492 (0.0010) [2023-12-27 08:29:05,076][20347] Updated weights for policy 0, policy_version 411502 (0.0010) [2023-12-27 08:29:05,134][20347] Updated weights for policy 0, policy_version 411512 (0.0010) [2023-12-27 08:29:05,370][20381] Updated weights for policy 1, policy_version 411059 (0.0008) [2023-12-27 08:29:05,429][20381] Updated weights for policy 1, policy_version 411069 (0.0008) [2023-12-27 08:29:05,489][20381] Updated weights for policy 1, policy_version 411079 (0.0008) [2023-12-27 08:29:05,689][20347] Updated weights for policy 0, policy_version 411522 (0.0007) [2023-12-27 08:29:05,746][20347] Updated weights for policy 0, policy_version 411532 (0.0010) [2023-12-27 08:29:05,804][20347] Updated weights for policy 0, policy_version 411542 (0.0010) [2023-12-27 08:29:05,861][20347] Updated weights for policy 0, policy_version 411552 (0.0010) [2023-12-27 08:29:06,216][20381] Updated weights for policy 1, policy_version 411089 (0.0007) [2023-12-27 08:29:06,264][20381] Updated weights for policy 1, policy_version 411099 (0.0008) [2023-12-27 08:29:06,318][20381] Updated weights for policy 1, policy_version 411109 (0.0008) [2023-12-27 08:29:06,368][20381] Updated weights for policy 1, policy_version 411119 (0.0007) [2023-12-27 08:29:06,485][20347] Updated weights for policy 0, policy_version 411562 (0.0005) [2023-12-27 08:29:06,545][20347] Updated weights for policy 0, policy_version 411572 (0.0007) [2023-12-27 08:29:06,605][20347] Updated weights for policy 0, policy_version 411582 (0.0008) [2023-12-27 08:29:07,127][20381] Updated weights for policy 1, policy_version 411129 (0.0007) [2023-12-27 08:29:07,183][20381] Updated weights for policy 1, policy_version 411139 (0.0008) [2023-12-27 08:29:07,215][20347] Updated weights for policy 0, policy_version 411592 (0.0005) [2023-12-27 08:29:07,237][20381] Updated weights for policy 1, policy_version 411149 (0.0007) [2023-12-27 08:29:07,259][20347] Updated weights for policy 0, policy_version 411602 (0.0005) [2023-12-27 08:29:07,306][20347] Updated weights for policy 0, policy_version 411612 (0.0007) [2023-12-27 08:29:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 210665472. Throughput: 0: 10410.7, 1: 10384.5. Samples: 210628636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:07,363][19170] Avg episode reward: [(0, '894.830'), (1, '944.190')] [2023-12-27 08:29:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000411152_105275392.pth... [2023-12-27 08:29:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000411616_105390080.pth... [2023-12-27 08:29:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000409936_104964096.pth [2023-12-27 08:29:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000410400_105078784.pth [2023-12-27 08:29:07,887][20381] Updated weights for policy 1, policy_version 411159 (0.0006) [2023-12-27 08:29:07,928][20347] Updated weights for policy 0, policy_version 411622 (0.0007) [2023-12-27 08:29:07,937][20381] Updated weights for policy 1, policy_version 411169 (0.0005) [2023-12-27 08:29:07,981][20347] Updated weights for policy 0, policy_version 411632 (0.0005) [2023-12-27 08:29:07,984][20381] Updated weights for policy 1, policy_version 411179 (0.0008) [2023-12-27 08:29:08,032][20347] Updated weights for policy 0, policy_version 411642 (0.0006) [2023-12-27 08:29:08,634][20381] Updated weights for policy 1, policy_version 411189 (0.0009) [2023-12-27 08:29:08,641][20347] Updated weights for policy 0, policy_version 411652 (0.0006) [2023-12-27 08:29:08,679][20292] KL-divergence is very high: 162.8349 [2023-12-27 08:29:08,685][20381] Updated weights for policy 1, policy_version 411199 (0.0008) [2023-12-27 08:29:08,686][20292] KL-divergence is very high: 152.5723 [2023-12-27 08:29:08,703][20347] Updated weights for policy 0, policy_version 411662 (0.0007) [2023-12-27 08:29:08,738][20381] Updated weights for policy 1, policy_version 411209 (0.0008) [2023-12-27 08:29:08,765][20347] Updated weights for policy 0, policy_version 411672 (0.0006) [2023-12-27 08:29:09,371][20347] Updated weights for policy 0, policy_version 411682 (0.0007) [2023-12-27 08:29:09,433][20347] Updated weights for policy 0, policy_version 411692 (0.0009) [2023-12-27 08:29:09,482][20381] Updated weights for policy 1, policy_version 411219 (0.0006) [2023-12-27 08:29:09,495][20347] Updated weights for policy 0, policy_version 411702 (0.0007) [2023-12-27 08:29:09,543][20381] Updated weights for policy 1, policy_version 411229 (0.0006) [2023-12-27 08:29:09,549][20347] Updated weights for policy 0, policy_version 411712 (0.0007) [2023-12-27 08:29:09,593][20381] Updated weights for policy 1, policy_version 411239 (0.0007) [2023-12-27 08:29:10,235][20347] Updated weights for policy 0, policy_version 411722 (0.0008) [2023-12-27 08:29:10,291][20347] Updated weights for policy 0, policy_version 411732 (0.0007) [2023-12-27 08:29:10,347][20347] Updated weights for policy 0, policy_version 411742 (0.0008) [2023-12-27 08:29:10,349][20381] Updated weights for policy 1, policy_version 411249 (0.0007) [2023-12-27 08:29:10,406][20381] Updated weights for policy 1, policy_version 411259 (0.0007) [2023-12-27 08:29:10,463][20381] Updated weights for policy 1, policy_version 411269 (0.0008) [2023-12-27 08:29:10,519][20381] Updated weights for policy 1, policy_version 411279 (0.0007) [2023-12-27 08:29:11,036][20347] Updated weights for policy 0, policy_version 411752 (0.0007) [2023-12-27 08:29:11,091][20347] Updated weights for policy 0, policy_version 411762 (0.0008) [2023-12-27 08:29:11,150][20347] Updated weights for policy 0, policy_version 411772 (0.0008) [2023-12-27 08:29:11,231][20381] Updated weights for policy 1, policy_version 411289 (0.0007) [2023-12-27 08:29:11,286][20381] Updated weights for policy 1, policy_version 411299 (0.0008) [2023-12-27 08:29:11,345][20381] Updated weights for policy 1, policy_version 411309 (0.0007) [2023-12-27 08:29:11,878][20347] Updated weights for policy 0, policy_version 411782 (0.0008) [2023-12-27 08:29:11,931][20347] Updated weights for policy 0, policy_version 411792 (0.0006) [2023-12-27 08:29:11,992][20347] Updated weights for policy 0, policy_version 411802 (0.0006) [2023-12-27 08:29:12,066][20381] Updated weights for policy 1, policy_version 411319 (0.0008) [2023-12-27 08:29:12,126][20381] Updated weights for policy 1, policy_version 411329 (0.0008) [2023-12-27 08:29:12,186][20381] Updated weights for policy 1, policy_version 411339 (0.0008) [2023-12-27 08:29:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 210763776. Throughput: 0: 10472.0, 1: 10336.2. Samples: 210752872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:12,363][19170] Avg episode reward: [(0, '923.090'), (1, '788.590')] [2023-12-27 08:29:12,661][20347] Updated weights for policy 0, policy_version 411812 (0.0007) [2023-12-27 08:29:12,718][20347] Updated weights for policy 0, policy_version 411822 (0.0007) [2023-12-27 08:29:12,776][20347] Updated weights for policy 0, policy_version 411832 (0.0007) [2023-12-27 08:29:12,872][20381] Updated weights for policy 1, policy_version 411349 (0.0008) [2023-12-27 08:29:12,921][20381] Updated weights for policy 1, policy_version 411359 (0.0007) [2023-12-27 08:29:12,970][20381] Updated weights for policy 1, policy_version 411369 (0.0006) [2023-12-27 08:29:13,481][20347] Updated weights for policy 0, policy_version 411842 (0.0008) [2023-12-27 08:29:13,535][20381] Updated weights for policy 1, policy_version 411379 (0.0006) [2023-12-27 08:29:13,538][20347] Updated weights for policy 0, policy_version 411852 (0.0006) [2023-12-27 08:29:13,584][20381] Updated weights for policy 1, policy_version 411389 (0.0007) [2023-12-27 08:29:13,589][20347] Updated weights for policy 0, policy_version 411862 (0.0006) [2023-12-27 08:29:13,633][20381] Updated weights for policy 1, policy_version 411399 (0.0006) [2023-12-27 08:29:13,639][20347] Updated weights for policy 0, policy_version 411872 (0.0005) [2023-12-27 08:29:14,266][20347] Updated weights for policy 0, policy_version 411882 (0.0010) [2023-12-27 08:29:14,315][20347] Updated weights for policy 0, policy_version 411892 (0.0010) [2023-12-27 08:29:14,325][20381] Updated weights for policy 1, policy_version 411409 (0.0007) [2023-12-27 08:29:14,363][20347] Updated weights for policy 0, policy_version 411902 (0.0010) [2023-12-27 08:29:14,384][20381] Updated weights for policy 1, policy_version 411419 (0.0006) [2023-12-27 08:29:14,442][20381] Updated weights for policy 1, policy_version 411429 (0.0007) [2023-12-27 08:29:14,495][20381] Updated weights for policy 1, policy_version 411439 (0.0008) [2023-12-27 08:29:15,066][20347] Updated weights for policy 0, policy_version 411912 (0.0007) [2023-12-27 08:29:15,130][20347] Updated weights for policy 0, policy_version 411922 (0.0008) [2023-12-27 08:29:15,192][20347] Updated weights for policy 0, policy_version 411932 (0.0008) [2023-12-27 08:29:15,223][20381] Updated weights for policy 1, policy_version 411449 (0.0006) [2023-12-27 08:29:15,285][20381] Updated weights for policy 1, policy_version 411459 (0.0007) [2023-12-27 08:29:15,344][20381] Updated weights for policy 1, policy_version 411469 (0.0006) [2023-12-27 08:29:15,820][20347] Updated weights for policy 0, policy_version 411942 (0.0010) [2023-12-27 08:29:15,871][20347] Updated weights for policy 0, policy_version 411952 (0.0010) [2023-12-27 08:29:15,928][20347] Updated weights for policy 0, policy_version 411962 (0.0010) [2023-12-27 08:29:15,947][20381] Updated weights for policy 1, policy_version 411479 (0.0009) [2023-12-27 08:29:16,000][20381] Updated weights for policy 1, policy_version 411489 (0.0011) [2023-12-27 08:29:16,055][20381] Updated weights for policy 1, policy_version 411499 (0.0007) [2023-12-27 08:29:16,603][20347] Updated weights for policy 0, policy_version 411972 (0.0010) [2023-12-27 08:29:16,650][20347] Updated weights for policy 0, policy_version 411982 (0.0010) [2023-12-27 08:29:16,684][20381] Updated weights for policy 1, policy_version 411509 (0.0008) [2023-12-27 08:29:16,708][20347] Updated weights for policy 0, policy_version 411992 (0.0010) [2023-12-27 08:29:16,742][20381] Updated weights for policy 1, policy_version 411519 (0.0010) [2023-12-27 08:29:16,801][20381] Updated weights for policy 1, policy_version 411529 (0.0010) [2023-12-27 08:29:17,348][20381] Updated weights for policy 1, policy_version 411539 (0.0007) [2023-12-27 08:29:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 210862080. Throughput: 0: 10442.5, 1: 10350.4. Samples: 210877896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:17,364][19170] Avg episode reward: [(0, '971.310'), (1, '798.830')] [2023-12-27 08:29:17,378][20347] Updated weights for policy 0, policy_version 412002 (0.0010) [2023-12-27 08:29:17,406][20381] Updated weights for policy 1, policy_version 411549 (0.0010) [2023-12-27 08:29:17,436][20347] Updated weights for policy 0, policy_version 412012 (0.0008) [2023-12-27 08:29:17,465][20381] Updated weights for policy 1, policy_version 411559 (0.0010) [2023-12-27 08:29:17,497][20347] Updated weights for policy 0, policy_version 412022 (0.0006) [2023-12-27 08:29:17,554][20347] Updated weights for policy 0, policy_version 412032 (0.0005) [2023-12-27 08:29:18,081][20381] Updated weights for policy 1, policy_version 411569 (0.0010) [2023-12-27 08:29:18,137][20381] Updated weights for policy 1, policy_version 411579 (0.0008) [2023-12-27 08:29:18,180][20347] Updated weights for policy 0, policy_version 412042 (0.0010) [2023-12-27 08:29:18,187][20381] Updated weights for policy 1, policy_version 411589 (0.0006) [2023-12-27 08:29:18,235][20347] Updated weights for policy 0, policy_version 412052 (0.0010) [2023-12-27 08:29:18,239][20381] Updated weights for policy 1, policy_version 411599 (0.0005) [2023-12-27 08:29:18,284][20347] Updated weights for policy 0, policy_version 412062 (0.0010) [2023-12-27 08:29:18,859][20381] Updated weights for policy 1, policy_version 411609 (0.0010) [2023-12-27 08:29:18,903][20381] Updated weights for policy 1, policy_version 411619 (0.0010) [2023-12-27 08:29:18,921][20347] Updated weights for policy 0, policy_version 412072 (0.0010) [2023-12-27 08:29:18,954][20381] Updated weights for policy 1, policy_version 411629 (0.0011) [2023-12-27 08:29:18,963][20291] KL-divergence is very high: 150.9035 [2023-12-27 08:29:18,973][20347] Updated weights for policy 0, policy_version 412082 (0.0010) [2023-12-27 08:29:19,030][20347] Updated weights for policy 0, policy_version 412092 (0.0010) [2023-12-27 08:29:19,702][20381] Updated weights for policy 1, policy_version 411639 (0.0008) [2023-12-27 08:29:19,735][20347] Updated weights for policy 0, policy_version 412102 (0.0010) [2023-12-27 08:29:19,759][20381] Updated weights for policy 1, policy_version 411649 (0.0007) [2023-12-27 08:29:19,788][20347] Updated weights for policy 0, policy_version 412112 (0.0010) [2023-12-27 08:29:19,821][20381] Updated weights for policy 1, policy_version 411659 (0.0008) [2023-12-27 08:29:19,850][20347] Updated weights for policy 0, policy_version 412122 (0.0010) [2023-12-27 08:29:20,461][20381] Updated weights for policy 1, policy_version 411669 (0.0009) [2023-12-27 08:29:20,513][20381] Updated weights for policy 1, policy_version 411679 (0.0010) [2023-12-27 08:29:20,572][20381] Updated weights for policy 1, policy_version 411689 (0.0009) [2023-12-27 08:29:20,579][20347] Updated weights for policy 0, policy_version 412132 (0.0009) [2023-12-27 08:29:20,642][20347] Updated weights for policy 0, policy_version 412142 (0.0009) [2023-12-27 08:29:20,707][20347] Updated weights for policy 0, policy_version 412152 (0.0008) [2023-12-27 08:29:21,252][20381] Updated weights for policy 1, policy_version 411699 (0.0008) [2023-12-27 08:29:21,318][20381] Updated weights for policy 1, policy_version 411709 (0.0007) [2023-12-27 08:29:21,340][20347] Updated weights for policy 0, policy_version 412162 (0.0008) [2023-12-27 08:29:21,387][20381] Updated weights for policy 1, policy_version 411719 (0.0010) [2023-12-27 08:29:21,407][20347] Updated weights for policy 0, policy_version 412172 (0.0010) [2023-12-27 08:29:21,467][20347] Updated weights for policy 0, policy_version 412182 (0.0007) [2023-12-27 08:29:21,528][20347] Updated weights for policy 0, policy_version 412192 (0.0008) [2023-12-27 08:29:22,008][20381] Updated weights for policy 1, policy_version 411729 (0.0009) [2023-12-27 08:29:22,058][20381] Updated weights for policy 1, policy_version 411739 (0.0010) [2023-12-27 08:29:22,117][20381] Updated weights for policy 1, policy_version 411749 (0.0011) [2023-12-27 08:29:22,173][20381] Updated weights for policy 1, policy_version 411759 (0.0011) [2023-12-27 08:29:22,235][20347] Updated weights for policy 0, policy_version 412202 (0.0010) [2023-12-27 08:29:22,299][20347] Updated weights for policy 0, policy_version 412212 (0.0010) [2023-12-27 08:29:22,358][20347] Updated weights for policy 0, policy_version 412222 (0.0008) [2023-12-27 08:29:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 210968576. Throughput: 0: 10420.7, 1: 10348.8. Samples: 210940652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:22,363][19170] Avg episode reward: [(0, '952.150'), (1, '865.410')] [2023-12-27 08:29:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000411760_105431040.pth... [2023-12-27 08:29:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000412224_105545728.pth... [2023-12-27 08:29:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000411008_105234432.pth [2023-12-27 08:29:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000410544_105119744.pth [2023-12-27 08:29:22,874][20381] Updated weights for policy 1, policy_version 411769 (0.0011) [2023-12-27 08:29:22,937][20381] Updated weights for policy 1, policy_version 411779 (0.0011) [2023-12-27 08:29:22,995][20381] Updated weights for policy 1, policy_version 411789 (0.0011) [2023-12-27 08:29:23,048][20347] Updated weights for policy 0, policy_version 412232 (0.0009) [2023-12-27 08:29:23,097][20347] Updated weights for policy 0, policy_version 412242 (0.0010) [2023-12-27 08:29:23,145][20347] Updated weights for policy 0, policy_version 412252 (0.0010) [2023-12-27 08:29:23,673][20381] Updated weights for policy 1, policy_version 411799 (0.0007) [2023-12-27 08:29:23,725][20381] Updated weights for policy 1, policy_version 411809 (0.0009) [2023-12-27 08:29:23,782][20381] Updated weights for policy 1, policy_version 411819 (0.0010) [2023-12-27 08:29:23,796][20347] Updated weights for policy 0, policy_version 412262 (0.0010) [2023-12-27 08:29:23,854][20347] Updated weights for policy 0, policy_version 412272 (0.0010) [2023-12-27 08:29:23,915][20347] Updated weights for policy 0, policy_version 412282 (0.0010) [2023-12-27 08:29:24,371][20381] Updated weights for policy 1, policy_version 411829 (0.0010) [2023-12-27 08:29:24,426][20381] Updated weights for policy 1, policy_version 411839 (0.0011) [2023-12-27 08:29:24,487][20381] Updated weights for policy 1, policy_version 411849 (0.0010) [2023-12-27 08:29:24,575][20347] Updated weights for policy 0, policy_version 412292 (0.0010) [2023-12-27 08:29:24,619][20347] Updated weights for policy 0, policy_version 412302 (0.0009) [2023-12-27 08:29:24,674][20347] Updated weights for policy 0, policy_version 412312 (0.0005) [2023-12-27 08:29:25,171][20381] Updated weights for policy 1, policy_version 411859 (0.0010) [2023-12-27 08:29:25,223][20381] Updated weights for policy 1, policy_version 411869 (0.0009) [2023-12-27 08:29:25,275][20381] Updated weights for policy 1, policy_version 411879 (0.0008) [2023-12-27 08:29:25,282][20347] Updated weights for policy 0, policy_version 412322 (0.0008) [2023-12-27 08:29:25,333][20347] Updated weights for policy 0, policy_version 412332 (0.0010) [2023-12-27 08:29:25,385][20347] Updated weights for policy 0, policy_version 412342 (0.0010) [2023-12-27 08:29:25,436][20347] Updated weights for policy 0, policy_version 412352 (0.0010) [2023-12-27 08:29:25,919][20381] Updated weights for policy 1, policy_version 411889 (0.0006) [2023-12-27 08:29:25,973][20381] Updated weights for policy 1, policy_version 411899 (0.0011) [2023-12-27 08:29:26,026][20381] Updated weights for policy 1, policy_version 411909 (0.0009) [2023-12-27 08:29:26,046][20347] Updated weights for policy 0, policy_version 412362 (0.0010) [2023-12-27 08:29:26,082][20381] Updated weights for policy 1, policy_version 411919 (0.0005) [2023-12-27 08:29:26,094][20347] Updated weights for policy 0, policy_version 412372 (0.0009) [2023-12-27 08:29:26,149][20347] Updated weights for policy 0, policy_version 412382 (0.0005) [2023-12-27 08:29:26,708][20381] Updated weights for policy 1, policy_version 411929 (0.0010) [2023-12-27 08:29:26,762][20381] Updated weights for policy 1, policy_version 411939 (0.0010) [2023-12-27 08:29:26,780][20347] Updated weights for policy 0, policy_version 412392 (0.0008) [2023-12-27 08:29:26,816][20381] Updated weights for policy 1, policy_version 411949 (0.0007) [2023-12-27 08:29:26,843][20347] Updated weights for policy 0, policy_version 412402 (0.0008) [2023-12-27 08:29:26,906][20347] Updated weights for policy 0, policy_version 412412 (0.0008) [2023-12-27 08:29:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 211075072. Throughput: 0: 10434.2, 1: 10361.3. Samples: 211066920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:27,363][19170] Avg episode reward: [(0, '961.690'), (1, '903.590')] [2023-12-27 08:29:27,400][20381] Updated weights for policy 1, policy_version 411959 (0.0005) [2023-12-27 08:29:27,457][20381] Updated weights for policy 1, policy_version 411969 (0.0006) [2023-12-27 08:29:27,472][20347] Updated weights for policy 0, policy_version 412422 (0.0008) [2023-12-27 08:29:27,507][20381] Updated weights for policy 1, policy_version 411979 (0.0006) [2023-12-27 08:29:27,521][20347] Updated weights for policy 0, policy_version 412432 (0.0008) [2023-12-27 08:29:27,569][20347] Updated weights for policy 0, policy_version 412442 (0.0007) [2023-12-27 08:29:28,041][20381] Updated weights for policy 1, policy_version 411989 (0.0008) [2023-12-27 08:29:28,091][20381] Updated weights for policy 1, policy_version 411999 (0.0011) [2023-12-27 08:29:28,140][20381] Updated weights for policy 1, policy_version 412009 (0.0010) [2023-12-27 08:29:28,228][20347] Updated weights for policy 0, policy_version 412452 (0.0006) [2023-12-27 08:29:28,284][20347] Updated weights for policy 0, policy_version 412462 (0.0008) [2023-12-27 08:29:28,343][20347] Updated weights for policy 0, policy_version 412472 (0.0006) [2023-12-27 08:29:28,842][20381] Updated weights for policy 1, policy_version 412019 (0.0009) [2023-12-27 08:29:28,901][20381] Updated weights for policy 1, policy_version 412029 (0.0006) [2023-12-27 08:29:28,952][20381] Updated weights for policy 1, policy_version 412039 (0.0010) [2023-12-27 08:29:29,015][20347] Updated weights for policy 0, policy_version 412482 (0.0010) [2023-12-27 08:29:29,071][20347] Updated weights for policy 0, policy_version 412492 (0.0006) [2023-12-27 08:29:29,117][20347] Updated weights for policy 0, policy_version 412502 (0.0007) [2023-12-27 08:29:29,160][20347] Updated weights for policy 0, policy_version 412512 (0.0008) [2023-12-27 08:29:29,571][20381] Updated weights for policy 1, policy_version 412049 (0.0011) [2023-12-27 08:29:29,629][20381] Updated weights for policy 1, policy_version 412059 (0.0010) [2023-12-27 08:29:29,688][20381] Updated weights for policy 1, policy_version 412069 (0.0010) [2023-12-27 08:29:29,749][20381] Updated weights for policy 1, policy_version 412079 (0.0010) [2023-12-27 08:29:29,864][20347] Updated weights for policy 0, policy_version 412522 (0.0008) [2023-12-27 08:29:29,925][20347] Updated weights for policy 0, policy_version 412532 (0.0008) [2023-12-27 08:29:29,987][20347] Updated weights for policy 0, policy_version 412542 (0.0008) [2023-12-27 08:29:30,402][20381] Updated weights for policy 1, policy_version 412089 (0.0008) [2023-12-27 08:29:30,461][20381] Updated weights for policy 1, policy_version 412099 (0.0007) [2023-12-27 08:29:30,514][20381] Updated weights for policy 1, policy_version 412109 (0.0008) [2023-12-27 08:29:30,640][20347] Updated weights for policy 0, policy_version 412552 (0.0008) [2023-12-27 08:29:30,695][20347] Updated weights for policy 0, policy_version 412562 (0.0007) [2023-12-27 08:29:30,755][20347] Updated weights for policy 0, policy_version 412572 (0.0009) [2023-12-27 08:29:31,109][20381] Updated weights for policy 1, policy_version 412119 (0.0007) [2023-12-27 08:29:31,162][20381] Updated weights for policy 1, policy_version 412129 (0.0008) [2023-12-27 08:29:31,215][20381] Updated weights for policy 1, policy_version 412139 (0.0008) [2023-12-27 08:29:31,453][20347] Updated weights for policy 0, policy_version 412582 (0.0010) [2023-12-27 08:29:31,509][20347] Updated weights for policy 0, policy_version 412592 (0.0010) [2023-12-27 08:29:31,575][20347] Updated weights for policy 0, policy_version 412602 (0.0010) [2023-12-27 08:29:31,915][20381] Updated weights for policy 1, policy_version 412149 (0.0007) [2023-12-27 08:29:31,972][20381] Updated weights for policy 1, policy_version 412159 (0.0006) [2023-12-27 08:29:32,032][20381] Updated weights for policy 1, policy_version 412169 (0.0008) [2023-12-27 08:29:32,226][20347] Updated weights for policy 0, policy_version 412612 (0.0009) [2023-12-27 08:29:32,292][20347] Updated weights for policy 0, policy_version 412622 (0.0010) [2023-12-27 08:29:32,346][20347] Updated weights for policy 0, policy_version 412632 (0.0010) [2023-12-27 08:29:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.2). Total num frames: 211181568. Throughput: 0: 10471.0, 1: 10425.8. Samples: 211195424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:32,363][19170] Avg episode reward: [(0, '952.270'), (1, '922.930')] [2023-12-27 08:29:32,688][20381] Updated weights for policy 1, policy_version 412179 (0.0009) [2023-12-27 08:29:32,739][20381] Updated weights for policy 1, policy_version 412189 (0.0006) [2023-12-27 08:29:32,790][20381] Updated weights for policy 1, policy_version 412199 (0.0009) [2023-12-27 08:29:32,999][20347] Updated weights for policy 0, policy_version 412642 (0.0010) [2023-12-27 08:29:33,057][20347] Updated weights for policy 0, policy_version 412652 (0.0011) [2023-12-27 08:29:33,126][20347] Updated weights for policy 0, policy_version 412662 (0.0008) [2023-12-27 08:29:33,189][20347] Updated weights for policy 0, policy_version 412672 (0.0010) [2023-12-27 08:29:33,341][20381] Updated weights for policy 1, policy_version 412209 (0.0005) [2023-12-27 08:29:33,400][20381] Updated weights for policy 1, policy_version 412219 (0.0005) [2023-12-27 08:29:33,456][20381] Updated weights for policy 1, policy_version 412229 (0.0006) [2023-12-27 08:29:33,511][20381] Updated weights for policy 1, policy_version 412239 (0.0011) [2023-12-27 08:29:33,812][20347] Updated weights for policy 0, policy_version 412682 (0.0010) [2023-12-27 08:29:33,857][20347] Updated weights for policy 0, policy_version 412692 (0.0010) [2023-12-27 08:29:33,902][20347] Updated weights for policy 0, policy_version 412702 (0.0010) [2023-12-27 08:29:34,157][20381] Updated weights for policy 1, policy_version 412249 (0.0008) [2023-12-27 08:29:34,220][20381] Updated weights for policy 1, policy_version 412259 (0.0007) [2023-12-27 08:29:34,281][20381] Updated weights for policy 1, policy_version 412269 (0.0008) [2023-12-27 08:29:34,619][20347] Updated weights for policy 0, policy_version 412712 (0.0008) [2023-12-27 08:29:34,675][20347] Updated weights for policy 0, policy_version 412722 (0.0008) [2023-12-27 08:29:34,727][20347] Updated weights for policy 0, policy_version 412732 (0.0007) [2023-12-27 08:29:34,926][20381] Updated weights for policy 1, policy_version 412279 (0.0008) [2023-12-27 08:29:34,980][20381] Updated weights for policy 1, policy_version 412289 (0.0006) [2023-12-27 08:29:35,037][20381] Updated weights for policy 1, policy_version 412299 (0.0006) [2023-12-27 08:29:35,366][20347] Updated weights for policy 0, policy_version 412742 (0.0006) [2023-12-27 08:29:35,414][20347] Updated weights for policy 0, policy_version 412752 (0.0005) [2023-12-27 08:29:35,462][20347] Updated weights for policy 0, policy_version 412762 (0.0005) [2023-12-27 08:29:35,673][20381] Updated weights for policy 1, policy_version 412309 (0.0008) [2023-12-27 08:29:35,725][20381] Updated weights for policy 1, policy_version 412319 (0.0009) [2023-12-27 08:29:35,770][20381] Updated weights for policy 1, policy_version 412329 (0.0009) [2023-12-27 08:29:36,075][20347] Updated weights for policy 0, policy_version 412772 (0.0007) [2023-12-27 08:29:36,145][20347] Updated weights for policy 0, policy_version 412782 (0.0008) [2023-12-27 08:29:36,199][20347] Updated weights for policy 0, policy_version 412792 (0.0007) [2023-12-27 08:29:36,527][20381] Updated weights for policy 1, policy_version 412339 (0.0008) [2023-12-27 08:29:36,576][20381] Updated weights for policy 1, policy_version 412349 (0.0007) [2023-12-27 08:29:36,632][20381] Updated weights for policy 1, policy_version 412359 (0.0008) [2023-12-27 08:29:36,897][20347] Updated weights for policy 0, policy_version 412802 (0.0008) [2023-12-27 08:29:36,958][20347] Updated weights for policy 0, policy_version 412812 (0.0008) [2023-12-27 08:29:37,020][20347] Updated weights for policy 0, policy_version 412822 (0.0008) [2023-12-27 08:29:37,072][20347] Updated weights for policy 0, policy_version 412832 (0.0007) [2023-12-27 08:29:37,339][20381] Updated weights for policy 1, policy_version 412369 (0.0007) [2023-12-27 08:29:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 211288064. Throughput: 0: 10468.8, 1: 10452.7. Samples: 211259056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:37,363][19170] Avg episode reward: [(0, '952.380'), (1, '951.930')] [2023-12-27 08:29:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000412832_105701376.pth... [2023-12-27 08:29:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000411616_105390080.pth [2023-12-27 08:29:37,392][20381] Updated weights for policy 1, policy_version 412379 (0.0007) [2023-12-27 08:29:37,445][20381] Updated weights for policy 1, policy_version 412389 (0.0008) [2023-12-27 08:29:37,503][20381] Updated weights for policy 1, policy_version 412399 (0.0008) [2023-12-27 08:29:37,506][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000412400_105594880.pth... [2023-12-27 08:29:37,509][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000411152_105275392.pth [2023-12-27 08:29:37,678][20347] Updated weights for policy 0, policy_version 412842 (0.0006) [2023-12-27 08:29:37,731][20347] Updated weights for policy 0, policy_version 412852 (0.0006) [2023-12-27 08:29:37,795][20347] Updated weights for policy 0, policy_version 412862 (0.0005) [2023-12-27 08:29:38,242][20381] Updated weights for policy 1, policy_version 412409 (0.0008) [2023-12-27 08:29:38,305][20381] Updated weights for policy 1, policy_version 412419 (0.0008) [2023-12-27 08:29:38,338][20347] Updated weights for policy 0, policy_version 412872 (0.0007) [2023-12-27 08:29:38,366][20381] Updated weights for policy 1, policy_version 412429 (0.0007) [2023-12-27 08:29:38,392][20347] Updated weights for policy 0, policy_version 412882 (0.0006) [2023-12-27 08:29:38,444][20347] Updated weights for policy 0, policy_version 412892 (0.0007) [2023-12-27 08:29:39,052][20381] Updated weights for policy 1, policy_version 412439 (0.0007) [2023-12-27 08:29:39,111][20381] Updated weights for policy 1, policy_version 412449 (0.0006) [2023-12-27 08:29:39,131][20347] Updated weights for policy 0, policy_version 412902 (0.0007) [2023-12-27 08:29:39,165][20381] Updated weights for policy 1, policy_version 412459 (0.0006) [2023-12-27 08:29:39,184][20347] Updated weights for policy 0, policy_version 412912 (0.0006) [2023-12-27 08:29:39,249][20347] Updated weights for policy 0, policy_version 412922 (0.0008) [2023-12-27 08:29:39,824][20381] Updated weights for policy 1, policy_version 412469 (0.0007) [2023-12-27 08:29:39,890][20381] Updated weights for policy 1, policy_version 412479 (0.0006) [2023-12-27 08:29:39,955][20381] Updated weights for policy 1, policy_version 412489 (0.0007) [2023-12-27 08:29:40,003][20347] Updated weights for policy 0, policy_version 412932 (0.0008) [2023-12-27 08:29:40,062][20347] Updated weights for policy 0, policy_version 412942 (0.0008) [2023-12-27 08:29:40,122][20347] Updated weights for policy 0, policy_version 412952 (0.0008) [2023-12-27 08:29:40,647][20381] Updated weights for policy 1, policy_version 412499 (0.0006) [2023-12-27 08:29:40,707][20381] Updated weights for policy 1, policy_version 412509 (0.0005) [2023-12-27 08:29:40,760][20381] Updated weights for policy 1, policy_version 412519 (0.0008) [2023-12-27 08:29:40,777][20347] Updated weights for policy 0, policy_version 412962 (0.0007) [2023-12-27 08:29:40,834][20347] Updated weights for policy 0, policy_version 412972 (0.0006) [2023-12-27 08:29:40,884][20347] Updated weights for policy 0, policy_version 412982 (0.0005) [2023-12-27 08:29:40,941][20347] Updated weights for policy 0, policy_version 412992 (0.0005) [2023-12-27 08:29:41,432][20381] Updated weights for policy 1, policy_version 412529 (0.0007) [2023-12-27 08:29:41,491][20381] Updated weights for policy 1, policy_version 412539 (0.0008) [2023-12-27 08:29:41,552][20381] Updated weights for policy 1, policy_version 412549 (0.0008) [2023-12-27 08:29:41,608][20381] Updated weights for policy 1, policy_version 412559 (0.0007) [2023-12-27 08:29:41,635][20347] Updated weights for policy 0, policy_version 413002 (0.0011) [2023-12-27 08:29:41,695][20347] Updated weights for policy 0, policy_version 413012 (0.0011) [2023-12-27 08:29:41,756][20347] Updated weights for policy 0, policy_version 413022 (0.0009) [2023-12-27 08:29:42,313][20381] Updated weights for policy 1, policy_version 412569 (0.0007) [2023-12-27 08:29:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 211386368. Throughput: 0: 10446.5, 1: 10401.0. Samples: 211383008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:42,363][19170] Avg episode reward: [(0, '942.640'), (1, '971.190')] [2023-12-27 08:29:42,381][20381] Updated weights for policy 1, policy_version 412579 (0.0008) [2023-12-27 08:29:42,441][20381] Updated weights for policy 1, policy_version 412589 (0.0007) [2023-12-27 08:29:42,444][20347] Updated weights for policy 0, policy_version 413032 (0.0010) [2023-12-27 08:29:42,499][20347] Updated weights for policy 0, policy_version 413042 (0.0010) [2023-12-27 08:29:42,561][20347] Updated weights for policy 0, policy_version 413052 (0.0010) [2023-12-27 08:29:43,126][20381] Updated weights for policy 1, policy_version 412599 (0.0008) [2023-12-27 08:29:43,186][20381] Updated weights for policy 1, policy_version 412609 (0.0005) [2023-12-27 08:29:43,228][20347] Updated weights for policy 0, policy_version 413062 (0.0008) [2023-12-27 08:29:43,244][20381] Updated weights for policy 1, policy_version 412619 (0.0008) [2023-12-27 08:29:43,276][20347] Updated weights for policy 0, policy_version 413072 (0.0005) [2023-12-27 08:29:43,333][20347] Updated weights for policy 0, policy_version 413082 (0.0005) [2023-12-27 08:29:43,914][20381] Updated weights for policy 1, policy_version 412629 (0.0008) [2023-12-27 08:29:43,974][20381] Updated weights for policy 1, policy_version 412639 (0.0008) [2023-12-27 08:29:43,994][20347] Updated weights for policy 0, policy_version 413092 (0.0007) [2023-12-27 08:29:44,019][20381] Updated weights for policy 1, policy_version 412649 (0.0007) [2023-12-27 08:29:44,048][20347] Updated weights for policy 0, policy_version 413102 (0.0007) [2023-12-27 08:29:44,103][20347] Updated weights for policy 0, policy_version 413112 (0.0007) [2023-12-27 08:29:44,658][20381] Updated weights for policy 1, policy_version 412659 (0.0006) [2023-12-27 08:29:44,710][20381] Updated weights for policy 1, policy_version 412669 (0.0005) [2023-12-27 08:29:44,758][20381] Updated weights for policy 1, policy_version 412679 (0.0006) [2023-12-27 08:29:44,803][20347] Updated weights for policy 0, policy_version 413122 (0.0006) [2023-12-27 08:29:44,859][20347] Updated weights for policy 0, policy_version 413132 (0.0006) [2023-12-27 08:29:44,918][20347] Updated weights for policy 0, policy_version 413142 (0.0006) [2023-12-27 08:29:44,981][20347] Updated weights for policy 0, policy_version 413152 (0.0007) [2023-12-27 08:29:45,436][20381] Updated weights for policy 1, policy_version 412689 (0.0010) [2023-12-27 08:29:45,492][20381] Updated weights for policy 1, policy_version 412699 (0.0008) [2023-12-27 08:29:45,549][20381] Updated weights for policy 1, policy_version 412709 (0.0005) [2023-12-27 08:29:45,606][20381] Updated weights for policy 1, policy_version 412719 (0.0005) [2023-12-27 08:29:45,642][20347] Updated weights for policy 0, policy_version 413162 (0.0007) [2023-12-27 08:29:45,695][20347] Updated weights for policy 0, policy_version 413172 (0.0008) [2023-12-27 08:29:45,747][20347] Updated weights for policy 0, policy_version 413182 (0.0005) [2023-12-27 08:29:46,228][20381] Updated weights for policy 1, policy_version 412729 (0.0008) [2023-12-27 08:29:46,283][20381] Updated weights for policy 1, policy_version 412739 (0.0007) [2023-12-27 08:29:46,328][20381] Updated weights for policy 1, policy_version 412749 (0.0008) [2023-12-27 08:29:46,406][20347] Updated weights for policy 0, policy_version 413192 (0.0008) [2023-12-27 08:29:46,466][20347] Updated weights for policy 0, policy_version 413202 (0.0008) [2023-12-27 08:29:46,516][20347] Updated weights for policy 0, policy_version 413212 (0.0008) [2023-12-27 08:29:46,944][20381] Updated weights for policy 1, policy_version 412759 (0.0007) [2023-12-27 08:29:46,998][20381] Updated weights for policy 1, policy_version 412769 (0.0008) [2023-12-27 08:29:47,047][20381] Updated weights for policy 1, policy_version 412779 (0.0008) [2023-12-27 08:29:47,234][20347] Updated weights for policy 0, policy_version 413222 (0.0007) [2023-12-27 08:29:47,294][20347] Updated weights for policy 0, policy_version 413232 (0.0008) [2023-12-27 08:29:47,355][20347] Updated weights for policy 0, policy_version 413242 (0.0007) [2023-12-27 08:29:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 211492864. Throughput: 0: 10458.3, 1: 10403.2. Samples: 211505812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:47,363][19170] Avg episode reward: [(0, '914.180'), (1, '980.710')] [2023-12-27 08:29:47,708][20381] Updated weights for policy 1, policy_version 412789 (0.0007) [2023-12-27 08:29:47,759][20381] Updated weights for policy 1, policy_version 412799 (0.0007) [2023-12-27 08:29:47,808][20381] Updated weights for policy 1, policy_version 412809 (0.0008) [2023-12-27 08:29:48,042][20347] Updated weights for policy 0, policy_version 413252 (0.0007) [2023-12-27 08:29:48,099][20347] Updated weights for policy 0, policy_version 413262 (0.0007) [2023-12-27 08:29:48,100][20291] KL-divergence is very high: 124.9184 [2023-12-27 08:29:48,161][20347] Updated weights for policy 0, policy_version 413272 (0.0010) [2023-12-27 08:29:48,193][20291] KL-divergence is very high: 348.6769 [2023-12-27 08:29:48,516][20381] Updated weights for policy 1, policy_version 412819 (0.0007) [2023-12-27 08:29:48,579][20381] Updated weights for policy 1, policy_version 412829 (0.0008) [2023-12-27 08:29:48,636][20381] Updated weights for policy 1, policy_version 412839 (0.0007) [2023-12-27 08:29:48,846][20347] Updated weights for policy 0, policy_version 413282 (0.0009) [2023-12-27 08:29:48,892][20291] KL-divergence is very high: 134.5253 [2023-12-27 08:29:48,910][20347] Updated weights for policy 0, policy_version 413292 (0.0005) [2023-12-27 08:29:48,940][20291] KL-divergence is very high: 122.7760 [2023-12-27 08:29:48,969][20347] Updated weights for policy 0, policy_version 413302 (0.0005) [2023-12-27 08:29:48,988][20291] KL-divergence is very high: 289.4132 [2023-12-27 08:29:49,029][20347] Updated weights for policy 0, policy_version 413312 (0.0009) [2023-12-27 08:29:49,338][20381] Updated weights for policy 1, policy_version 412849 (0.0008) [2023-12-27 08:29:49,401][20381] Updated weights for policy 1, policy_version 412859 (0.0008) [2023-12-27 08:29:49,445][20381] Updated weights for policy 1, policy_version 412869 (0.0007) [2023-12-27 08:29:49,493][20381] Updated weights for policy 1, policy_version 412879 (0.0007) [2023-12-27 08:29:49,662][20347] Updated weights for policy 0, policy_version 413322 (0.0010) [2023-12-27 08:29:49,719][20347] Updated weights for policy 0, policy_version 413332 (0.0010) [2023-12-27 08:29:49,778][20347] Updated weights for policy 0, policy_version 413342 (0.0010) [2023-12-27 08:29:50,227][20381] Updated weights for policy 1, policy_version 412889 (0.0007) [2023-12-27 08:29:50,276][20381] Updated weights for policy 1, policy_version 412899 (0.0008) [2023-12-27 08:29:50,322][20381] Updated weights for policy 1, policy_version 412909 (0.0007) [2023-12-27 08:29:50,491][20347] Updated weights for policy 0, policy_version 413352 (0.0008) [2023-12-27 08:29:50,492][20291] KL-divergence is very high: 126.0916 [2023-12-27 08:29:50,545][20347] Updated weights for policy 0, policy_version 413362 (0.0010) [2023-12-27 08:29:50,558][20291] KL-divergence is very high: 101.7732 [2023-12-27 08:29:50,604][20347] Updated weights for policy 0, policy_version 413372 (0.0010) [2023-12-27 08:29:51,069][20381] Updated weights for policy 1, policy_version 412919 (0.0007) [2023-12-27 08:29:51,130][20381] Updated weights for policy 1, policy_version 412929 (0.0007) [2023-12-27 08:29:51,190][20381] Updated weights for policy 1, policy_version 412939 (0.0007) [2023-12-27 08:29:51,206][20347] Updated weights for policy 0, policy_version 413382 (0.0011) [2023-12-27 08:29:51,272][20347] Updated weights for policy 0, policy_version 413392 (0.0011) [2023-12-27 08:29:51,342][20347] Updated weights for policy 0, policy_version 413402 (0.0010) [2023-12-27 08:29:51,829][20381] Updated weights for policy 1, policy_version 412949 (0.0007) [2023-12-27 08:29:51,881][20381] Updated weights for policy 1, policy_version 412959 (0.0007) [2023-12-27 08:29:51,932][20347] Updated weights for policy 0, policy_version 413412 (0.0009) [2023-12-27 08:29:51,941][20381] Updated weights for policy 1, policy_version 412969 (0.0009) [2023-12-27 08:29:51,986][20347] Updated weights for policy 0, policy_version 413422 (0.0007) [2023-12-27 08:29:52,042][20347] Updated weights for policy 0, policy_version 413432 (0.0008) [2023-12-27 08:29:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 211599360. Throughput: 0: 10446.7, 1: 10387.6. Samples: 211566184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:52,363][19170] Avg episode reward: [(0, '817.130'), (1, '957.670')] [2023-12-27 08:29:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000412976_105742336.pth... [2023-12-27 08:29:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000413440_105857024.pth... [2023-12-27 08:29:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000411760_105431040.pth [2023-12-27 08:29:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000412224_105545728.pth [2023-12-27 08:29:52,648][20381] Updated weights for policy 1, policy_version 412979 (0.0007) [2023-12-27 08:29:52,692][20381] Updated weights for policy 1, policy_version 412989 (0.0008) [2023-12-27 08:29:52,742][20381] Updated weights for policy 1, policy_version 412999 (0.0006) [2023-12-27 08:29:52,756][20347] Updated weights for policy 0, policy_version 413442 (0.0007) [2023-12-27 08:29:52,810][20347] Updated weights for policy 0, policy_version 413452 (0.0007) [2023-12-27 08:29:52,828][20291] KL-divergence is very high: 100.3233 [2023-12-27 08:29:52,861][20347] Updated weights for policy 0, policy_version 413462 (0.0010) [2023-12-27 08:29:52,909][20347] Updated weights for policy 0, policy_version 413472 (0.0009) [2023-12-27 08:29:53,295][20381] Updated weights for policy 1, policy_version 413009 (0.0005) [2023-12-27 08:29:53,340][20292] KL-divergence is very high: 143.4626 [2023-12-27 08:29:53,349][20381] Updated weights for policy 1, policy_version 413019 (0.0006) [2023-12-27 08:29:53,413][20381] Updated weights for policy 1, policy_version 413029 (0.0008) [2023-12-27 08:29:53,467][20381] Updated weights for policy 1, policy_version 413039 (0.0008) [2023-12-27 08:29:53,644][20347] Updated weights for policy 0, policy_version 413482 (0.0005) [2023-12-27 08:29:53,700][20347] Updated weights for policy 0, policy_version 413492 (0.0005) [2023-12-27 08:29:53,757][20347] Updated weights for policy 0, policy_version 413502 (0.0006) [2023-12-27 08:29:54,140][20381] Updated weights for policy 1, policy_version 413049 (0.0009) [2023-12-27 08:29:54,189][20381] Updated weights for policy 1, policy_version 413059 (0.0006) [2023-12-27 08:29:54,251][20381] Updated weights for policy 1, policy_version 413069 (0.0006) [2023-12-27 08:29:54,275][20347] Updated weights for policy 0, policy_version 413512 (0.0008) [2023-12-27 08:29:54,338][20347] Updated weights for policy 0, policy_version 413522 (0.0008) [2023-12-27 08:29:54,400][20347] Updated weights for policy 0, policy_version 413532 (0.0008) [2023-12-27 08:29:54,845][20381] Updated weights for policy 1, policy_version 413079 (0.0008) [2023-12-27 08:29:54,913][20381] Updated weights for policy 1, policy_version 413089 (0.0009) [2023-12-27 08:29:54,970][20381] Updated weights for policy 1, policy_version 413099 (0.0009) [2023-12-27 08:29:55,077][20347] Updated weights for policy 0, policy_version 413542 (0.0007) [2023-12-27 08:29:55,127][20347] Updated weights for policy 0, policy_version 413552 (0.0008) [2023-12-27 08:29:55,175][20347] Updated weights for policy 0, policy_version 413562 (0.0008) [2023-12-27 08:29:55,650][20381] Updated weights for policy 1, policy_version 413109 (0.0010) [2023-12-27 08:29:55,699][20381] Updated weights for policy 1, policy_version 413119 (0.0010) [2023-12-27 08:29:55,753][20381] Updated weights for policy 1, policy_version 413129 (0.0010) [2023-12-27 08:29:55,814][20347] Updated weights for policy 0, policy_version 413572 (0.0006) [2023-12-27 08:29:55,865][20347] Updated weights for policy 0, policy_version 413582 (0.0005) [2023-12-27 08:29:55,924][20347] Updated weights for policy 0, policy_version 413592 (0.0006) [2023-12-27 08:29:56,418][20381] Updated weights for policy 1, policy_version 413139 (0.0010) [2023-12-27 08:29:56,466][20381] Updated weights for policy 1, policy_version 413149 (0.0010) [2023-12-27 08:29:56,467][20347] Updated weights for policy 0, policy_version 413602 (0.0005) [2023-12-27 08:29:56,515][20347] Updated weights for policy 0, policy_version 413612 (0.0005) [2023-12-27 08:29:56,521][20381] Updated weights for policy 1, policy_version 413159 (0.0010) [2023-12-27 08:29:56,563][20347] Updated weights for policy 0, policy_version 413622 (0.0005) [2023-12-27 08:29:56,607][20347] Updated weights for policy 0, policy_version 413632 (0.0007) [2023-12-27 08:29:57,101][20381] Updated weights for policy 1, policy_version 413169 (0.0009) [2023-12-27 08:29:57,163][20381] Updated weights for policy 1, policy_version 413179 (0.0007) [2023-12-27 08:29:57,212][20347] Updated weights for policy 0, policy_version 413642 (0.0005) [2023-12-27 08:29:57,224][20381] Updated weights for policy 1, policy_version 413189 (0.0005) [2023-12-27 08:29:57,274][20347] Updated weights for policy 0, policy_version 413652 (0.0007) [2023-12-27 08:29:57,277][20381] Updated weights for policy 1, policy_version 413199 (0.0009) [2023-12-27 08:29:57,336][20347] Updated weights for policy 0, policy_version 413662 (0.0007) [2023-12-27 08:29:57,363][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 211714048. Throughput: 0: 10426.6, 1: 10474.9. Samples: 211693444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:29:57,363][19170] Avg episode reward: [(0, '898.960'), (1, '801.180')] [2023-12-27 08:29:57,866][20381] Updated weights for policy 1, policy_version 413209 (0.0011) [2023-12-27 08:29:57,871][20347] Updated weights for policy 0, policy_version 413672 (0.0005) [2023-12-27 08:29:57,877][20291] KL-divergence is very high: 116.2434 [2023-12-27 08:29:57,921][20381] Updated weights for policy 1, policy_version 413219 (0.0010) [2023-12-27 08:29:57,921][20347] Updated weights for policy 0, policy_version 413682 (0.0005) [2023-12-27 08:29:57,969][20347] Updated weights for policy 0, policy_version 413692 (0.0005) [2023-12-27 08:29:57,975][20381] Updated weights for policy 1, policy_version 413229 (0.0010) [2023-12-27 08:29:58,684][20347] Updated weights for policy 0, policy_version 413702 (0.0006) [2023-12-27 08:29:58,733][20381] Updated weights for policy 1, policy_version 413239 (0.0010) [2023-12-27 08:29:58,748][20347] Updated weights for policy 0, policy_version 413712 (0.0007) [2023-12-27 08:29:58,806][20381] Updated weights for policy 1, policy_version 413250 (0.0008) [2023-12-27 08:29:58,814][20347] Updated weights for policy 0, policy_version 413722 (0.0007) [2023-12-27 08:29:58,871][20381] Updated weights for policy 1, policy_version 413260 (0.0007) [2023-12-27 08:29:59,533][20347] Updated weights for policy 0, policy_version 413732 (0.0007) [2023-12-27 08:29:59,586][20347] Updated weights for policy 0, policy_version 413742 (0.0007) [2023-12-27 08:29:59,595][20381] Updated weights for policy 1, policy_version 413270 (0.0007) [2023-12-27 08:29:59,645][20347] Updated weights for policy 0, policy_version 413752 (0.0007) [2023-12-27 08:29:59,655][20381] Updated weights for policy 1, policy_version 413280 (0.0007) [2023-12-27 08:29:59,714][20381] Updated weights for policy 1, policy_version 413290 (0.0005) [2023-12-27 08:30:00,316][20381] Updated weights for policy 1, policy_version 413300 (0.0007) [2023-12-27 08:30:00,335][20347] Updated weights for policy 0, policy_version 413762 (0.0008) [2023-12-27 08:30:00,371][20381] Updated weights for policy 1, policy_version 413310 (0.0007) [2023-12-27 08:30:00,383][20347] Updated weights for policy 0, policy_version 413772 (0.0010) [2023-12-27 08:30:00,417][20381] Updated weights for policy 1, policy_version 413320 (0.0005) [2023-12-27 08:30:00,432][20347] Updated weights for policy 0, policy_version 413782 (0.0010) [2023-12-27 08:30:00,487][20347] Updated weights for policy 0, policy_version 413792 (0.0007) [2023-12-27 08:30:01,104][20381] Updated weights for policy 1, policy_version 413330 (0.0006) [2023-12-27 08:30:01,159][20381] Updated weights for policy 1, policy_version 413340 (0.0008) [2023-12-27 08:30:01,168][20347] Updated weights for policy 0, policy_version 413802 (0.0008) [2023-12-27 08:30:01,211][20381] Updated weights for policy 1, policy_version 413350 (0.0008) [2023-12-27 08:30:01,219][20347] Updated weights for policy 0, policy_version 413812 (0.0007) [2023-12-27 08:30:01,272][20381] Updated weights for policy 1, policy_version 413360 (0.0008) [2023-12-27 08:30:01,279][20347] Updated weights for policy 0, policy_version 413822 (0.0006) [2023-12-27 08:30:01,974][20347] Updated weights for policy 0, policy_version 413832 (0.0007) [2023-12-27 08:30:01,979][20291] KL-divergence is very high: 440.8420 [2023-12-27 08:30:01,985][20381] Updated weights for policy 1, policy_version 413370 (0.0010) [2023-12-27 08:30:01,989][20291] KL-divergence is very high: 1385.5933 [2023-12-27 08:30:01,999][20291] KL-divergence is very high: 911.1165 [2023-12-27 08:30:02,006][20291] KL-divergence is very high: 493.3147 [2023-12-27 08:30:02,030][20347] Updated weights for policy 0, policy_version 413842 (0.0006) [2023-12-27 08:30:02,035][20291] KL-divergence is very high: 147.5979 [2023-12-27 08:30:02,037][20381] Updated weights for policy 1, policy_version 413380 (0.0010) [2023-12-27 08:30:02,048][20291] KL-divergence is very high: 170.9990 [2023-12-27 08:30:02,091][20347] Updated weights for policy 0, policy_version 413852 (0.0005) [2023-12-27 08:30:02,093][20381] Updated weights for policy 1, policy_version 413390 (0.0011) [2023-12-27 08:30:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 211812352. Throughput: 0: 10467.6, 1: 10441.5. Samples: 211818800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:30:02,363][19170] Avg episode reward: [(0, '892.820'), (1, '768.530')] [2023-12-27 08:30:02,743][20291] KL-divergence is very high: 134.0135 [2023-12-27 08:30:02,770][20347] Updated weights for policy 0, policy_version 413862 (0.0007) [2023-12-27 08:30:02,786][20291] KL-divergence is very high: 107.5758 [2023-12-27 08:30:02,788][20381] Updated weights for policy 1, policy_version 413400 (0.0010) [2023-12-27 08:30:02,827][20347] Updated weights for policy 0, policy_version 413872 (0.0007) [2023-12-27 08:30:02,831][20291] KL-divergence is very high: 111.9536 [2023-12-27 08:30:02,839][20381] Updated weights for policy 1, policy_version 413410 (0.0011) [2023-12-27 08:30:02,874][20291] KL-divergence is very high: 101.8531 [2023-12-27 08:30:02,878][20347] Updated weights for policy 0, policy_version 413882 (0.0005) [2023-12-27 08:30:02,884][20381] Updated weights for policy 1, policy_version 413420 (0.0010) [2023-12-27 08:30:03,524][20381] Updated weights for policy 1, policy_version 413430 (0.0011) [2023-12-27 08:30:03,573][20347] Updated weights for policy 0, policy_version 413892 (0.0005) [2023-12-27 08:30:03,579][20381] Updated weights for policy 1, policy_version 413440 (0.0011) [2023-12-27 08:30:03,614][20291] KL-divergence is very high: 113.1151 [2023-12-27 08:30:03,624][20347] Updated weights for policy 0, policy_version 413902 (0.0005) [2023-12-27 08:30:03,633][20381] Updated weights for policy 1, policy_version 413450 (0.0010) [2023-12-27 08:30:03,677][20347] Updated weights for policy 0, policy_version 413912 (0.0009) [2023-12-27 08:30:04,314][20381] Updated weights for policy 1, policy_version 413460 (0.0011) [2023-12-27 08:30:04,347][20292] KL-divergence is very high: 127.5033 [2023-12-27 08:30:04,379][20381] Updated weights for policy 1, policy_version 413470 (0.0006) [2023-12-27 08:30:04,400][20347] Updated weights for policy 0, policy_version 413922 (0.0007) [2023-12-27 08:30:04,443][20381] Updated weights for policy 1, policy_version 413480 (0.0008) [2023-12-27 08:30:04,457][20347] Updated weights for policy 0, policy_version 413932 (0.0006) [2023-12-27 08:30:04,518][20347] Updated weights for policy 0, policy_version 413942 (0.0006) [2023-12-27 08:30:04,575][20347] Updated weights for policy 0, policy_version 413952 (0.0007) [2023-12-27 08:30:05,120][20381] Updated weights for policy 1, policy_version 413490 (0.0010) [2023-12-27 08:30:05,190][20381] Updated weights for policy 1, policy_version 413500 (0.0009) [2023-12-27 08:30:05,251][20381] Updated weights for policy 1, policy_version 413510 (0.0010) [2023-12-27 08:30:05,265][20347] Updated weights for policy 0, policy_version 413962 (0.0006) [2023-12-27 08:30:05,315][20381] Updated weights for policy 1, policy_version 413520 (0.0011) [2023-12-27 08:30:05,324][20347] Updated weights for policy 0, policy_version 413972 (0.0009) [2023-12-27 08:30:05,383][20347] Updated weights for policy 0, policy_version 413982 (0.0008) [2023-12-27 08:30:05,914][20381] Updated weights for policy 1, policy_version 413530 (0.0010) [2023-12-27 08:30:05,970][20381] Updated weights for policy 1, policy_version 413540 (0.0009) [2023-12-27 08:30:06,027][20381] Updated weights for policy 1, policy_version 413550 (0.0007) [2023-12-27 08:30:06,094][20347] Updated weights for policy 0, policy_version 413992 (0.0006) [2023-12-27 08:30:06,159][20347] Updated weights for policy 0, policy_version 414002 (0.0008) [2023-12-27 08:30:06,222][20347] Updated weights for policy 0, policy_version 414012 (0.0007) [2023-12-27 08:30:06,756][20381] Updated weights for policy 1, policy_version 413560 (0.0007) [2023-12-27 08:30:06,803][20381] Updated weights for policy 1, policy_version 413570 (0.0008) [2023-12-27 08:30:06,852][20381] Updated weights for policy 1, policy_version 413580 (0.0007) [2023-12-27 08:30:06,912][20347] Updated weights for policy 0, policy_version 414022 (0.0008) [2023-12-27 08:30:06,961][20347] Updated weights for policy 0, policy_version 414032 (0.0008) [2023-12-27 08:30:07,019][20347] Updated weights for policy 0, policy_version 414042 (0.0008) [2023-12-27 08:30:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 211910656. Throughput: 0: 10447.8, 1: 10426.8. Samples: 211880008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:30:07,363][19170] Avg episode reward: [(0, '839.550'), (1, '840.260')] [2023-12-27 08:30:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000413584_105897984.pth... [2023-12-27 08:30:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000414048_106012672.pth... [2023-12-27 08:30:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000412832_105701376.pth [2023-12-27 08:30:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000412400_105594880.pth [2023-12-27 08:30:07,523][20381] Updated weights for policy 1, policy_version 413590 (0.0009) [2023-12-27 08:30:07,575][20381] Updated weights for policy 1, policy_version 413600 (0.0010) [2023-12-27 08:30:07,617][20381] Updated weights for policy 1, policy_version 413610 (0.0010) [2023-12-27 08:30:07,741][20347] Updated weights for policy 0, policy_version 414052 (0.0006) [2023-12-27 08:30:07,793][20347] Updated weights for policy 0, policy_version 414062 (0.0009) [2023-12-27 08:30:07,852][20347] Updated weights for policy 0, policy_version 414072 (0.0007) [2023-12-27 08:30:08,276][20381] Updated weights for policy 1, policy_version 413620 (0.0008) [2023-12-27 08:30:08,339][20381] Updated weights for policy 1, policy_version 413630 (0.0007) [2023-12-27 08:30:08,397][20381] Updated weights for policy 1, policy_version 413640 (0.0007) [2023-12-27 08:30:08,539][20347] Updated weights for policy 0, policy_version 414082 (0.0007) [2023-12-27 08:30:08,595][20347] Updated weights for policy 0, policy_version 414092 (0.0008) [2023-12-27 08:30:08,650][20347] Updated weights for policy 0, policy_version 414102 (0.0007) [2023-12-27 08:30:08,704][20347] Updated weights for policy 0, policy_version 414112 (0.0007) [2023-12-27 08:30:09,050][20381] Updated weights for policy 1, policy_version 413650 (0.0009) [2023-12-27 08:30:09,108][20381] Updated weights for policy 1, policy_version 413660 (0.0010) [2023-12-27 08:30:09,166][20381] Updated weights for policy 1, policy_version 413670 (0.0010) [2023-12-27 08:30:09,229][20381] Updated weights for policy 1, policy_version 413680 (0.0007) [2023-12-27 08:30:09,408][20347] Updated weights for policy 0, policy_version 414122 (0.0010) [2023-12-27 08:30:09,475][20347] Updated weights for policy 0, policy_version 414132 (0.0008) [2023-12-27 08:30:09,538][20347] Updated weights for policy 0, policy_version 414142 (0.0007) [2023-12-27 08:30:09,879][20381] Updated weights for policy 1, policy_version 413690 (0.0007) [2023-12-27 08:30:09,940][20381] Updated weights for policy 1, policy_version 413700 (0.0007) [2023-12-27 08:30:10,002][20381] Updated weights for policy 1, policy_version 413710 (0.0006) [2023-12-27 08:30:10,216][20347] Updated weights for policy 0, policy_version 414152 (0.0006) [2023-12-27 08:30:10,279][20347] Updated weights for policy 0, policy_version 414162 (0.0006) [2023-12-27 08:30:10,332][20347] Updated weights for policy 0, policy_version 414172 (0.0005) [2023-12-27 08:30:10,645][20381] Updated weights for policy 1, policy_version 413720 (0.0007) [2023-12-27 08:30:10,703][20381] Updated weights for policy 1, policy_version 413730 (0.0007) [2023-12-27 08:30:10,769][20381] Updated weights for policy 1, policy_version 413740 (0.0006) [2023-12-27 08:30:10,940][20347] Updated weights for policy 0, policy_version 414182 (0.0007) [2023-12-27 08:30:10,998][20347] Updated weights for policy 0, policy_version 414192 (0.0007) [2023-12-27 08:30:11,057][20347] Updated weights for policy 0, policy_version 414202 (0.0007) [2023-12-27 08:30:11,522][20381] Updated weights for policy 1, policy_version 413750 (0.0006) [2023-12-27 08:30:11,578][20381] Updated weights for policy 1, policy_version 413760 (0.0008) [2023-12-27 08:30:11,639][20381] Updated weights for policy 1, policy_version 413770 (0.0009) [2023-12-27 08:30:11,777][20347] Updated weights for policy 0, policy_version 414212 (0.0007) [2023-12-27 08:30:11,838][20347] Updated weights for policy 0, policy_version 414222 (0.0006) [2023-12-27 08:30:11,893][20347] Updated weights for policy 0, policy_version 414232 (0.0008) [2023-12-27 08:30:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 212008960. Throughput: 0: 10384.6, 1: 10412.7. Samples: 212002800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:30:12,363][19170] Avg episode reward: [(0, '802.900'), (1, '827.940')] [2023-12-27 08:30:12,369][20381] Updated weights for policy 1, policy_version 413780 (0.0009) [2023-12-27 08:30:12,422][20381] Updated weights for policy 1, policy_version 413790 (0.0007) [2023-12-27 08:30:12,473][20381] Updated weights for policy 1, policy_version 413800 (0.0006) [2023-12-27 08:30:12,568][20347] Updated weights for policy 0, policy_version 414242 (0.0007) [2023-12-27 08:30:12,622][20347] Updated weights for policy 0, policy_version 414252 (0.0007) [2023-12-27 08:30:12,679][20347] Updated weights for policy 0, policy_version 414262 (0.0008) [2023-12-27 08:30:12,737][20347] Updated weights for policy 0, policy_version 414272 (0.0007) [2023-12-27 08:30:13,099][20381] Updated weights for policy 1, policy_version 413810 (0.0008) [2023-12-27 08:30:13,162][20381] Updated weights for policy 1, policy_version 413820 (0.0010) [2023-12-27 08:30:13,213][20381] Updated weights for policy 1, policy_version 413830 (0.0010) [2023-12-27 08:30:13,273][20381] Updated weights for policy 1, policy_version 413840 (0.0010) [2023-12-27 08:30:13,387][20347] Updated weights for policy 0, policy_version 414282 (0.0005) [2023-12-27 08:30:13,442][20347] Updated weights for policy 0, policy_version 414292 (0.0008) [2023-12-27 08:30:13,502][20347] Updated weights for policy 0, policy_version 414302 (0.0008) [2023-12-27 08:30:13,919][20381] Updated weights for policy 1, policy_version 413850 (0.0007) [2023-12-27 08:30:13,951][20292] KL-divergence is very high: 194.0038 [2023-12-27 08:30:13,981][20381] Updated weights for policy 1, policy_version 413860 (0.0008) [2023-12-27 08:30:14,043][20381] Updated weights for policy 1, policy_version 413870 (0.0006) [2023-12-27 08:30:14,253][20347] Updated weights for policy 0, policy_version 414312 (0.0008) [2023-12-27 08:30:14,316][20347] Updated weights for policy 0, policy_version 414322 (0.0008) [2023-12-27 08:30:14,377][20347] Updated weights for policy 0, policy_version 414332 (0.0007) [2023-12-27 08:30:14,690][20381] Updated weights for policy 1, policy_version 413880 (0.0007) [2023-12-27 08:30:14,740][20381] Updated weights for policy 1, policy_version 413890 (0.0008) [2023-12-27 08:30:14,798][20381] Updated weights for policy 1, policy_version 413900 (0.0007) [2023-12-27 08:30:15,083][20347] Updated weights for policy 0, policy_version 414342 (0.0007) [2023-12-27 08:30:15,138][20347] Updated weights for policy 0, policy_version 414352 (0.0008) [2023-12-27 08:30:15,196][20347] Updated weights for policy 0, policy_version 414362 (0.0006) [2023-12-27 08:30:15,484][20381] Updated weights for policy 1, policy_version 413910 (0.0007) [2023-12-27 08:30:15,503][20292] KL-divergence is very high: 113.1168 [2023-12-27 08:30:15,540][20381] Updated weights for policy 1, policy_version 413920 (0.0007) [2023-12-27 08:30:15,595][20381] Updated weights for policy 1, policy_version 413930 (0.0008) [2023-12-27 08:30:15,865][20347] Updated weights for policy 0, policy_version 414372 (0.0005) [2023-12-27 08:30:15,917][20347] Updated weights for policy 0, policy_version 414382 (0.0005) [2023-12-27 08:30:15,966][20347] Updated weights for policy 0, policy_version 414392 (0.0008) [2023-12-27 08:30:16,290][20381] Updated weights for policy 1, policy_version 413940 (0.0008) [2023-12-27 08:30:16,339][20381] Updated weights for policy 1, policy_version 413950 (0.0008) [2023-12-27 08:30:16,400][20381] Updated weights for policy 1, policy_version 413960 (0.0006) [2023-12-27 08:30:16,687][20347] Updated weights for policy 0, policy_version 414402 (0.0008) [2023-12-27 08:30:16,747][20347] Updated weights for policy 0, policy_version 414412 (0.0007) [2023-12-27 08:30:16,811][20347] Updated weights for policy 0, policy_version 414422 (0.0008) [2023-12-27 08:30:16,867][20347] Updated weights for policy 0, policy_version 414432 (0.0008) [2023-12-27 08:30:16,975][20381] Updated weights for policy 1, policy_version 413970 (0.0006) [2023-12-27 08:30:17,032][20381] Updated weights for policy 1, policy_version 413980 (0.0007) [2023-12-27 08:30:17,091][20381] Updated weights for policy 1, policy_version 413990 (0.0008) [2023-12-27 08:30:17,147][20381] Updated weights for policy 1, policy_version 414000 (0.0008) [2023-12-27 08:30:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 212115456. Throughput: 0: 10302.2, 1: 10338.8. Samples: 212124272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:30:17,364][19170] Avg episode reward: [(0, '828.380'), (1, '760.640')] [2023-12-27 08:30:17,572][20347] Updated weights for policy 0, policy_version 414442 (0.0008) [2023-12-27 08:30:17,628][20347] Updated weights for policy 0, policy_version 414452 (0.0007) [2023-12-27 08:30:17,683][20347] Updated weights for policy 0, policy_version 414462 (0.0007) [2023-12-27 08:30:17,814][20381] Updated weights for policy 1, policy_version 414010 (0.0007) [2023-12-27 08:30:17,866][20381] Updated weights for policy 1, policy_version 414020 (0.0007) [2023-12-27 08:30:17,925][20381] Updated weights for policy 1, policy_version 414030 (0.0008) [2023-12-27 08:30:18,357][20347] Updated weights for policy 0, policy_version 414472 (0.0009) [2023-12-27 08:30:18,417][20347] Updated weights for policy 0, policy_version 414482 (0.0007) [2023-12-27 08:30:18,484][20347] Updated weights for policy 0, policy_version 414492 (0.0007) [2023-12-27 08:30:18,646][20381] Updated weights for policy 1, policy_version 414040 (0.0006) [2023-12-27 08:30:18,699][20381] Updated weights for policy 1, policy_version 414050 (0.0006) [2023-12-27 08:30:18,756][20381] Updated weights for policy 1, policy_version 414060 (0.0007) [2023-12-27 08:30:19,122][20347] Updated weights for policy 0, policy_version 414502 (0.0011) [2023-12-27 08:30:19,189][20347] Updated weights for policy 0, policy_version 414512 (0.0010) [2023-12-27 08:30:19,262][20347] Updated weights for policy 0, policy_version 414522 (0.0008) [2023-12-27 08:30:19,493][20381] Updated weights for policy 1, policy_version 414070 (0.0008) [2023-12-27 08:30:19,558][20381] Updated weights for policy 1, policy_version 414080 (0.0007) [2023-12-27 08:30:19,621][20381] Updated weights for policy 1, policy_version 414090 (0.0008) [2023-12-27 08:30:19,977][20347] Updated weights for policy 0, policy_version 414532 (0.0010) [2023-12-27 08:30:20,035][20347] Updated weights for policy 0, policy_version 414542 (0.0010) [2023-12-27 08:30:20,092][20347] Updated weights for policy 0, policy_version 414552 (0.0008) [2023-12-27 08:30:20,365][20381] Updated weights for policy 1, policy_version 414100 (0.0008) [2023-12-27 08:30:20,420][20381] Updated weights for policy 1, policy_version 414110 (0.0006) [2023-12-27 08:30:20,478][20381] Updated weights for policy 1, policy_version 414120 (0.0005) [2023-12-27 08:30:20,767][20347] Updated weights for policy 0, policy_version 414562 (0.0008) [2023-12-27 08:30:20,837][20347] Updated weights for policy 0, policy_version 414572 (0.0009) [2023-12-27 08:30:20,903][20347] Updated weights for policy 0, policy_version 414582 (0.0009) [2023-12-27 08:30:20,980][20347] Updated weights for policy 0, policy_version 414592 (0.0006) [2023-12-27 08:30:21,257][20381] Updated weights for policy 1, policy_version 414130 (0.0007) [2023-12-27 08:30:21,320][20381] Updated weights for policy 1, policy_version 414140 (0.0008) [2023-12-27 08:30:21,393][20381] Updated weights for policy 1, policy_version 414150 (0.0008) [2023-12-27 08:30:21,456][20381] Updated weights for policy 1, policy_version 414160 (0.0008) [2023-12-27 08:30:21,771][20347] Updated weights for policy 0, policy_version 414602 (0.0007) [2023-12-27 08:30:21,838][20347] Updated weights for policy 0, policy_version 414612 (0.0008) [2023-12-27 08:30:21,902][20347] Updated weights for policy 0, policy_version 414622 (0.0008) [2023-12-27 08:30:22,179][20381] Updated weights for policy 1, policy_version 414170 (0.0008) [2023-12-27 08:30:22,239][20381] Updated weights for policy 1, policy_version 414180 (0.0008) [2023-12-27 08:30:22,297][20381] Updated weights for policy 1, policy_version 414190 (0.0007) [2023-12-27 08:30:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 212213760. Throughput: 0: 10291.7, 1: 10255.1. Samples: 212183664. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:22,363][19170] Avg episode reward: [(0, '859.570'), (1, '807.900')] [2023-12-27 08:30:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000414192_106053632.pth... [2023-12-27 08:30:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000414624_106160128.pth... [2023-12-27 08:30:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000412976_105742336.pth [2023-12-27 08:30:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000413440_105857024.pth [2023-12-27 08:30:22,621][20347] Updated weights for policy 0, policy_version 414632 (0.0008) [2023-12-27 08:30:22,691][20347] Updated weights for policy 0, policy_version 414642 (0.0008) [2023-12-27 08:30:22,758][20347] Updated weights for policy 0, policy_version 414652 (0.0009) [2023-12-27 08:30:22,996][20381] Updated weights for policy 1, policy_version 414200 (0.0007) [2023-12-27 08:30:23,052][20381] Updated weights for policy 1, policy_version 414210 (0.0007) [2023-12-27 08:30:23,110][20381] Updated weights for policy 1, policy_version 414220 (0.0008) [2023-12-27 08:30:23,356][20347] Updated weights for policy 0, policy_version 414662 (0.0006) [2023-12-27 08:30:23,418][20347] Updated weights for policy 0, policy_version 414672 (0.0007) [2023-12-27 08:30:23,475][20347] Updated weights for policy 0, policy_version 414682 (0.0006) [2023-12-27 08:30:23,810][20381] Updated weights for policy 1, policy_version 414230 (0.0007) [2023-12-27 08:30:23,863][20381] Updated weights for policy 1, policy_version 414240 (0.0007) [2023-12-27 08:30:23,908][20381] Updated weights for policy 1, policy_version 414250 (0.0007) [2023-12-27 08:30:24,155][20347] Updated weights for policy 0, policy_version 414692 (0.0007) [2023-12-27 08:30:24,216][20347] Updated weights for policy 0, policy_version 414702 (0.0008) [2023-12-27 08:30:24,271][20347] Updated weights for policy 0, policy_version 414712 (0.0005) [2023-12-27 08:30:24,631][20381] Updated weights for policy 1, policy_version 414260 (0.0008) [2023-12-27 08:30:24,686][20381] Updated weights for policy 1, policy_version 414270 (0.0007) [2023-12-27 08:30:24,743][20381] Updated weights for policy 1, policy_version 414280 (0.0008) [2023-12-27 08:30:24,918][20347] Updated weights for policy 0, policy_version 414722 (0.0007) [2023-12-27 08:30:24,971][20347] Updated weights for policy 0, policy_version 414732 (0.0007) [2023-12-27 08:30:25,026][20347] Updated weights for policy 0, policy_version 414742 (0.0007) [2023-12-27 08:30:25,084][20347] Updated weights for policy 0, policy_version 414752 (0.0007) [2023-12-27 08:30:25,444][20381] Updated weights for policy 1, policy_version 414290 (0.0008) [2023-12-27 08:30:25,492][20381] Updated weights for policy 1, policy_version 414300 (0.0007) [2023-12-27 08:30:25,537][20381] Updated weights for policy 1, policy_version 414310 (0.0007) [2023-12-27 08:30:25,582][20381] Updated weights for policy 1, policy_version 414320 (0.0008) [2023-12-27 08:30:25,767][20347] Updated weights for policy 0, policy_version 414762 (0.0009) [2023-12-27 08:30:25,816][20347] Updated weights for policy 0, policy_version 414772 (0.0008) [2023-12-27 08:30:25,870][20347] Updated weights for policy 0, policy_version 414782 (0.0008) [2023-12-27 08:30:26,158][20381] Updated weights for policy 1, policy_version 414330 (0.0008) [2023-12-27 08:30:26,220][20381] Updated weights for policy 1, policy_version 414340 (0.0007) [2023-12-27 08:30:26,282][20381] Updated weights for policy 1, policy_version 414350 (0.0007) [2023-12-27 08:30:26,549][20347] Updated weights for policy 0, policy_version 414792 (0.0006) [2023-12-27 08:30:26,601][20347] Updated weights for policy 0, policy_version 414802 (0.0006) [2023-12-27 08:30:26,649][20347] Updated weights for policy 0, policy_version 414812 (0.0007) [2023-12-27 08:30:26,948][20381] Updated weights for policy 1, policy_version 414360 (0.0007) [2023-12-27 08:30:26,991][20381] Updated weights for policy 1, policy_version 414370 (0.0006) [2023-12-27 08:30:27,042][20381] Updated weights for policy 1, policy_version 414380 (0.0005) [2023-12-27 08:30:27,358][20347] Updated weights for policy 0, policy_version 414822 (0.0007) [2023-12-27 08:30:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 212312064. Throughput: 0: 10197.6, 1: 10234.9. Samples: 212302468. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:27,363][19170] Avg episode reward: [(0, '885.030'), (1, '788.250')] [2023-12-27 08:30:27,408][20347] Updated weights for policy 0, policy_version 414832 (0.0005) [2023-12-27 08:30:27,453][20347] Updated weights for policy 0, policy_version 414842 (0.0007) [2023-12-27 08:30:27,687][20381] Updated weights for policy 1, policy_version 414390 (0.0006) [2023-12-27 08:30:27,746][20381] Updated weights for policy 1, policy_version 414400 (0.0005) [2023-12-27 08:30:27,807][20381] Updated weights for policy 1, policy_version 414410 (0.0005) [2023-12-27 08:30:28,083][20347] Updated weights for policy 0, policy_version 414852 (0.0006) [2023-12-27 08:30:28,132][20347] Updated weights for policy 0, policy_version 414862 (0.0005) [2023-12-27 08:30:28,180][20347] Updated weights for policy 0, policy_version 414872 (0.0007) [2023-12-27 08:30:28,398][20381] Updated weights for policy 1, policy_version 414420 (0.0008) [2023-12-27 08:30:28,445][20381] Updated weights for policy 1, policy_version 414430 (0.0007) [2023-12-27 08:30:28,494][20381] Updated weights for policy 1, policy_version 414440 (0.0009) [2023-12-27 08:30:28,739][20347] Updated weights for policy 0, policy_version 414882 (0.0006) [2023-12-27 08:30:28,799][20347] Updated weights for policy 0, policy_version 414892 (0.0007) [2023-12-27 08:30:28,858][20347] Updated weights for policy 0, policy_version 414902 (0.0007) [2023-12-27 08:30:28,912][20347] Updated weights for policy 0, policy_version 414912 (0.0006) [2023-12-27 08:30:29,180][20381] Updated weights for policy 1, policy_version 414450 (0.0008) [2023-12-27 08:30:29,243][20381] Updated weights for policy 1, policy_version 414460 (0.0008) [2023-12-27 08:30:29,306][20381] Updated weights for policy 1, policy_version 414470 (0.0007) [2023-12-27 08:30:29,372][20381] Updated weights for policy 1, policy_version 414480 (0.0008) [2023-12-27 08:30:29,522][20347] Updated weights for policy 0, policy_version 414922 (0.0008) [2023-12-27 08:30:29,576][20347] Updated weights for policy 0, policy_version 414932 (0.0006) [2023-12-27 08:30:29,646][20347] Updated weights for policy 0, policy_version 414942 (0.0005) [2023-12-27 08:30:30,022][20381] Updated weights for policy 1, policy_version 414490 (0.0007) [2023-12-27 08:30:30,076][20381] Updated weights for policy 1, policy_version 414500 (0.0007) [2023-12-27 08:30:30,136][20381] Updated weights for policy 1, policy_version 414510 (0.0007) [2023-12-27 08:30:30,267][20347] Updated weights for policy 0, policy_version 414952 (0.0005) [2023-12-27 08:30:30,315][20347] Updated weights for policy 0, policy_version 414962 (0.0005) [2023-12-27 08:30:30,370][20347] Updated weights for policy 0, policy_version 414972 (0.0005) [2023-12-27 08:30:30,785][20381] Updated weights for policy 1, policy_version 414520 (0.0006) [2023-12-27 08:30:30,830][20381] Updated weights for policy 1, policy_version 414530 (0.0005) [2023-12-27 08:30:30,873][20381] Updated weights for policy 1, policy_version 414540 (0.0005) [2023-12-27 08:30:30,995][20347] Updated weights for policy 0, policy_version 414982 (0.0009) [2023-12-27 08:30:31,046][20347] Updated weights for policy 0, policy_version 414992 (0.0010) [2023-12-27 08:30:31,099][20347] Updated weights for policy 0, policy_version 415002 (0.0010) [2023-12-27 08:30:31,515][20381] Updated weights for policy 1, policy_version 414550 (0.0007) [2023-12-27 08:30:31,568][20381] Updated weights for policy 1, policy_version 414560 (0.0005) [2023-12-27 08:30:31,620][20381] Updated weights for policy 1, policy_version 414570 (0.0010) [2023-12-27 08:30:31,769][20347] Updated weights for policy 0, policy_version 415012 (0.0010) [2023-12-27 08:30:31,822][20347] Updated weights for policy 0, policy_version 415022 (0.0010) [2023-12-27 08:30:31,886][20347] Updated weights for policy 0, policy_version 415032 (0.0010) [2023-12-27 08:30:31,892][20291] KL-divergence is very high: 160.1354 [2023-12-27 08:30:32,307][20381] Updated weights for policy 1, policy_version 414580 (0.0010) [2023-12-27 08:30:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 212418560. Throughput: 0: 10293.9, 1: 10283.4. Samples: 212431788. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:32,363][19170] Avg episode reward: [(0, '823.870'), (1, '827.750')] [2023-12-27 08:30:32,375][20381] Updated weights for policy 1, policy_version 414590 (0.0008) [2023-12-27 08:30:32,435][20381] Updated weights for policy 1, policy_version 414600 (0.0008) [2023-12-27 08:30:32,533][20291] KL-divergence is very high: 201.9456 [2023-12-27 08:30:32,541][20291] KL-divergence is very high: 182.9099 [2023-12-27 08:30:32,547][20347] Updated weights for policy 0, policy_version 415042 (0.0011) [2023-12-27 08:30:32,607][20347] Updated weights for policy 0, policy_version 415052 (0.0010) [2023-12-27 08:30:32,663][20347] Updated weights for policy 0, policy_version 415062 (0.0010) [2023-12-27 08:30:32,718][20347] Updated weights for policy 0, policy_version 415072 (0.0010) [2023-12-27 08:30:33,060][20381] Updated weights for policy 1, policy_version 414610 (0.0006) [2023-12-27 08:30:33,117][20381] Updated weights for policy 1, policy_version 414620 (0.0006) [2023-12-27 08:30:33,170][20381] Updated weights for policy 1, policy_version 414630 (0.0006) [2023-12-27 08:30:33,217][20381] Updated weights for policy 1, policy_version 414640 (0.0007) [2023-12-27 08:30:33,375][20347] Updated weights for policy 0, policy_version 415082 (0.0010) [2023-12-27 08:30:33,423][20347] Updated weights for policy 0, policy_version 415092 (0.0010) [2023-12-27 08:30:33,473][20347] Updated weights for policy 0, policy_version 415102 (0.0010) [2023-12-27 08:30:33,863][20381] Updated weights for policy 1, policy_version 414650 (0.0007) [2023-12-27 08:30:33,919][20381] Updated weights for policy 1, policy_version 414660 (0.0007) [2023-12-27 08:30:33,972][20381] Updated weights for policy 1, policy_version 414670 (0.0007) [2023-12-27 08:30:34,169][20347] Updated weights for policy 0, policy_version 415112 (0.0010) [2023-12-27 08:30:34,232][20347] Updated weights for policy 0, policy_version 415122 (0.0009) [2023-12-27 08:30:34,292][20347] Updated weights for policy 0, policy_version 415132 (0.0009) [2023-12-27 08:30:34,712][20381] Updated weights for policy 1, policy_version 414680 (0.0006) [2023-12-27 08:30:34,769][20381] Updated weights for policy 1, policy_version 414690 (0.0008) [2023-12-27 08:30:34,828][20381] Updated weights for policy 1, policy_version 414700 (0.0007) [2023-12-27 08:30:34,858][20347] Updated weights for policy 0, policy_version 415142 (0.0008) [2023-12-27 08:30:34,914][20347] Updated weights for policy 0, policy_version 415152 (0.0010) [2023-12-27 08:30:34,976][20347] Updated weights for policy 0, policy_version 415162 (0.0010) [2023-12-27 08:30:35,488][20381] Updated weights for policy 1, policy_version 414710 (0.0007) [2023-12-27 08:30:35,535][20381] Updated weights for policy 1, policy_version 414720 (0.0007) [2023-12-27 08:30:35,585][20381] Updated weights for policy 1, policy_version 414730 (0.0008) [2023-12-27 08:30:35,610][20347] Updated weights for policy 0, policy_version 415172 (0.0009) [2023-12-27 08:30:35,662][20347] Updated weights for policy 0, policy_version 415182 (0.0010) [2023-12-27 08:30:35,707][20347] Updated weights for policy 0, policy_version 415192 (0.0010) [2023-12-27 08:30:36,315][20381] Updated weights for policy 1, policy_version 414740 (0.0008) [2023-12-27 08:30:36,378][20347] Updated weights for policy 0, policy_version 415202 (0.0010) [2023-12-27 08:30:36,380][20381] Updated weights for policy 1, policy_version 414750 (0.0007) [2023-12-27 08:30:36,434][20347] Updated weights for policy 0, policy_version 415212 (0.0010) [2023-12-27 08:30:36,439][20381] Updated weights for policy 1, policy_version 414760 (0.0005) [2023-12-27 08:30:36,494][20347] Updated weights for policy 0, policy_version 415222 (0.0011) [2023-12-27 08:30:36,558][20347] Updated weights for policy 0, policy_version 415232 (0.0011) [2023-12-27 08:30:37,097][20381] Updated weights for policy 1, policy_version 414770 (0.0008) [2023-12-27 08:30:37,159][20381] Updated weights for policy 1, policy_version 414780 (0.0007) [2023-12-27 08:30:37,216][20381] Updated weights for policy 1, policy_version 414790 (0.0007) [2023-12-27 08:30:37,272][20347] Updated weights for policy 0, policy_version 415242 (0.0011) [2023-12-27 08:30:37,277][20381] Updated weights for policy 1, policy_version 414800 (0.0007) [2023-12-27 08:30:37,326][20347] Updated weights for policy 0, policy_version 415252 (0.0010) [2023-12-27 08:30:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 212525056. Throughput: 0: 10334.7, 1: 10301.8. Samples: 212494824. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:37,363][19170] Avg episode reward: [(0, '761.600'), (1, '845.900')] [2023-12-27 08:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000414800_106209280.pth... [2023-12-27 08:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000413584_105897984.pth [2023-12-27 08:30:37,374][20347] Updated weights for policy 0, policy_version 415262 (0.0010) [2023-12-27 08:30:37,384][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000415264_106323968.pth... [2023-12-27 08:30:37,387][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000414048_106012672.pth [2023-12-27 08:30:37,887][20381] Updated weights for policy 1, policy_version 414810 (0.0005) [2023-12-27 08:30:37,939][20381] Updated weights for policy 1, policy_version 414820 (0.0005) [2023-12-27 08:30:37,987][20347] Updated weights for policy 0, policy_version 415272 (0.0008) [2023-12-27 08:30:37,990][20381] Updated weights for policy 1, policy_version 414830 (0.0005) [2023-12-27 08:30:38,048][20347] Updated weights for policy 0, policy_version 415282 (0.0008) [2023-12-27 08:30:38,107][20347] Updated weights for policy 0, policy_version 415292 (0.0008) [2023-12-27 08:30:38,574][20381] Updated weights for policy 1, policy_version 414840 (0.0010) [2023-12-27 08:30:38,627][20381] Updated weights for policy 1, policy_version 414850 (0.0007) [2023-12-27 08:30:38,684][20381] Updated weights for policy 1, policy_version 414860 (0.0006) [2023-12-27 08:30:38,763][20347] Updated weights for policy 0, policy_version 415302 (0.0006) [2023-12-27 08:30:38,859][20347] Updated weights for policy 0, policy_version 415312 (0.0007) [2023-12-27 08:30:38,917][20347] Updated weights for policy 0, policy_version 415322 (0.0008) [2023-12-27 08:30:39,333][20381] Updated weights for policy 1, policy_version 414870 (0.0008) [2023-12-27 08:30:39,398][20381] Updated weights for policy 1, policy_version 414880 (0.0008) [2023-12-27 08:30:39,454][20381] Updated weights for policy 1, policy_version 414890 (0.0007) [2023-12-27 08:30:39,564][20347] Updated weights for policy 0, policy_version 415332 (0.0007) [2023-12-27 08:30:39,626][20347] Updated weights for policy 0, policy_version 415342 (0.0008) [2023-12-27 08:30:39,687][20347] Updated weights for policy 0, policy_version 415352 (0.0006) [2023-12-27 08:30:40,128][20381] Updated weights for policy 1, policy_version 414900 (0.0008) [2023-12-27 08:30:40,187][20381] Updated weights for policy 1, policy_version 414910 (0.0008) [2023-12-27 08:30:40,244][20381] Updated weights for policy 1, policy_version 414920 (0.0007) [2023-12-27 08:30:40,401][20347] Updated weights for policy 0, policy_version 415362 (0.0008) [2023-12-27 08:30:40,463][20347] Updated weights for policy 0, policy_version 415372 (0.0008) [2023-12-27 08:30:40,525][20347] Updated weights for policy 0, policy_version 415382 (0.0008) [2023-12-27 08:30:40,583][20347] Updated weights for policy 0, policy_version 415392 (0.0008) [2023-12-27 08:30:40,853][20381] Updated weights for policy 1, policy_version 414930 (0.0008) [2023-12-27 08:30:40,902][20381] Updated weights for policy 1, policy_version 414940 (0.0009) [2023-12-27 08:30:40,949][20381] Updated weights for policy 1, policy_version 414950 (0.0007) [2023-12-27 08:30:40,997][20381] Updated weights for policy 1, policy_version 414960 (0.0007) [2023-12-27 08:30:41,252][20347] Updated weights for policy 0, policy_version 415402 (0.0010) [2023-12-27 08:30:41,312][20347] Updated weights for policy 0, policy_version 415412 (0.0011) [2023-12-27 08:30:41,379][20347] Updated weights for policy 0, policy_version 415422 (0.0011) [2023-12-27 08:30:41,758][20381] Updated weights for policy 1, policy_version 414970 (0.0008) [2023-12-27 08:30:41,816][20381] Updated weights for policy 1, policy_version 414980 (0.0007) [2023-12-27 08:30:41,872][20381] Updated weights for policy 1, policy_version 414990 (0.0008) [2023-12-27 08:30:42,076][20347] Updated weights for policy 0, policy_version 415432 (0.0008) [2023-12-27 08:30:42,136][20347] Updated weights for policy 0, policy_version 415442 (0.0009) [2023-12-27 08:30:42,201][20347] Updated weights for policy 0, policy_version 415452 (0.0011) [2023-12-27 08:30:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 212631552. Throughput: 0: 10291.7, 1: 10303.1. Samples: 212620208. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:42,363][19170] Avg episode reward: [(0, '761.670'), (1, '884.560')] [2023-12-27 08:30:42,533][20381] Updated weights for policy 1, policy_version 415000 (0.0008) [2023-12-27 08:30:42,595][20381] Updated weights for policy 1, policy_version 415010 (0.0008) [2023-12-27 08:30:42,665][20381] Updated weights for policy 1, policy_version 415020 (0.0008) [2023-12-27 08:30:42,909][20347] Updated weights for policy 0, policy_version 415462 (0.0009) [2023-12-27 08:30:42,963][20347] Updated weights for policy 0, policy_version 415472 (0.0007) [2023-12-27 08:30:43,023][20347] Updated weights for policy 0, policy_version 415482 (0.0008) [2023-12-27 08:30:43,277][20381] Updated weights for policy 1, policy_version 415030 (0.0009) [2023-12-27 08:30:43,324][20381] Updated weights for policy 1, policy_version 415040 (0.0010) [2023-12-27 08:30:43,382][20381] Updated weights for policy 1, policy_version 415050 (0.0010) [2023-12-27 08:30:43,721][20347] Updated weights for policy 0, policy_version 415492 (0.0008) [2023-12-27 08:30:43,782][20347] Updated weights for policy 0, policy_version 415502 (0.0007) [2023-12-27 08:30:43,838][20347] Updated weights for policy 0, policy_version 415512 (0.0010) [2023-12-27 08:30:44,062][20381] Updated weights for policy 1, policy_version 415060 (0.0011) [2023-12-27 08:30:44,118][20381] Updated weights for policy 1, policy_version 415070 (0.0011) [2023-12-27 08:30:44,177][20381] Updated weights for policy 1, policy_version 415080 (0.0009) [2023-12-27 08:30:44,497][20347] Updated weights for policy 0, policy_version 415522 (0.0009) [2023-12-27 08:30:44,557][20347] Updated weights for policy 0, policy_version 415532 (0.0007) [2023-12-27 08:30:44,624][20347] Updated weights for policy 0, policy_version 415542 (0.0008) [2023-12-27 08:30:44,684][20347] Updated weights for policy 0, policy_version 415552 (0.0008) [2023-12-27 08:30:44,873][20381] Updated weights for policy 1, policy_version 415090 (0.0008) [2023-12-27 08:30:44,933][20381] Updated weights for policy 1, policy_version 415100 (0.0011) [2023-12-27 08:30:44,990][20381] Updated weights for policy 1, policy_version 415110 (0.0011) [2023-12-27 08:30:45,052][20381] Updated weights for policy 1, policy_version 415120 (0.0011) [2023-12-27 08:30:45,381][20347] Updated weights for policy 0, policy_version 415562 (0.0008) [2023-12-27 08:30:45,445][20347] Updated weights for policy 0, policy_version 415572 (0.0006) [2023-12-27 08:30:45,510][20347] Updated weights for policy 0, policy_version 415582 (0.0005) [2023-12-27 08:30:45,678][20381] Updated weights for policy 1, policy_version 415130 (0.0005) [2023-12-27 08:30:45,725][20381] Updated weights for policy 1, policy_version 415140 (0.0005) [2023-12-27 08:30:45,776][20381] Updated weights for policy 1, policy_version 415150 (0.0005) [2023-12-27 08:30:46,052][20347] Updated weights for policy 0, policy_version 415592 (0.0009) [2023-12-27 08:30:46,108][20347] Updated weights for policy 0, policy_version 415602 (0.0010) [2023-12-27 08:30:46,164][20347] Updated weights for policy 0, policy_version 415612 (0.0010) [2023-12-27 08:30:46,394][20381] Updated weights for policy 1, policy_version 415160 (0.0010) [2023-12-27 08:30:46,400][20292] KL-divergence is very high: 108.2326 [2023-12-27 08:30:46,452][20381] Updated weights for policy 1, policy_version 415170 (0.0010) [2023-12-27 08:30:46,510][20381] Updated weights for policy 1, policy_version 415180 (0.0011) [2023-12-27 08:30:46,822][20347] Updated weights for policy 0, policy_version 415622 (0.0010) [2023-12-27 08:30:46,886][20347] Updated weights for policy 0, policy_version 415632 (0.0010) [2023-12-27 08:30:46,940][20347] Updated weights for policy 0, policy_version 415642 (0.0010) [2023-12-27 08:30:47,196][20381] Updated weights for policy 1, policy_version 415190 (0.0010) [2023-12-27 08:30:47,252][20381] Updated weights for policy 1, policy_version 415200 (0.0010) [2023-12-27 08:30:47,310][20381] Updated weights for policy 1, policy_version 415210 (0.0010) [2023-12-27 08:30:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 212738048. Throughput: 0: 10242.4, 1: 10330.1. Samples: 212744568. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:47,364][19170] Avg episode reward: [(0, '827.170'), (1, '971.220')] [2023-12-27 08:30:47,601][20347] Updated weights for policy 0, policy_version 415652 (0.0008) [2023-12-27 08:30:47,661][20347] Updated weights for policy 0, policy_version 415662 (0.0007) [2023-12-27 08:30:47,717][20347] Updated weights for policy 0, policy_version 415672 (0.0010) [2023-12-27 08:30:48,003][20381] Updated weights for policy 1, policy_version 415220 (0.0011) [2023-12-27 08:30:48,061][20381] Updated weights for policy 1, policy_version 415230 (0.0011) [2023-12-27 08:30:48,117][20381] Updated weights for policy 1, policy_version 415240 (0.0011) [2023-12-27 08:30:48,368][20347] Updated weights for policy 0, policy_version 415682 (0.0010) [2023-12-27 08:30:48,436][20347] Updated weights for policy 0, policy_version 415692 (0.0011) [2023-12-27 08:30:48,447][20291] KL-divergence is very high: 928.2639 [2023-12-27 08:30:48,494][20347] Updated weights for policy 0, policy_version 415702 (0.0010) [2023-12-27 08:30:48,495][20291] KL-divergence is very high: 282.0373 [2023-12-27 08:30:48,555][20347] Updated weights for policy 0, policy_version 415712 (0.0010) [2023-12-27 08:30:48,807][20381] Updated weights for policy 1, policy_version 415250 (0.0011) [2023-12-27 08:30:48,868][20381] Updated weights for policy 1, policy_version 415260 (0.0011) [2023-12-27 08:30:48,933][20381] Updated weights for policy 1, policy_version 415270 (0.0011) [2023-12-27 08:30:48,992][20381] Updated weights for policy 1, policy_version 415280 (0.0010) [2023-12-27 08:30:49,197][20347] Updated weights for policy 0, policy_version 415722 (0.0006) [2023-12-27 08:30:49,261][20347] Updated weights for policy 0, policy_version 415732 (0.0007) [2023-12-27 08:30:49,331][20347] Updated weights for policy 0, policy_version 415742 (0.0008) [2023-12-27 08:30:49,703][20381] Updated weights for policy 1, policy_version 415290 (0.0010) [2023-12-27 08:30:49,767][20381] Updated weights for policy 1, policy_version 415300 (0.0011) [2023-12-27 08:30:49,819][20381] Updated weights for policy 1, policy_version 415310 (0.0010) [2023-12-27 08:30:49,935][20347] Updated weights for policy 0, policy_version 415752 (0.0008) [2023-12-27 08:30:49,993][20347] Updated weights for policy 0, policy_version 415762 (0.0006) [2023-12-27 08:30:50,052][20347] Updated weights for policy 0, policy_version 415772 (0.0007) [2023-12-27 08:30:50,505][20381] Updated weights for policy 1, policy_version 415320 (0.0010) [2023-12-27 08:30:50,556][20381] Updated weights for policy 1, policy_version 415330 (0.0010) [2023-12-27 08:30:50,614][20381] Updated weights for policy 1, policy_version 415340 (0.0010) [2023-12-27 08:30:50,710][20347] Updated weights for policy 0, policy_version 415782 (0.0009) [2023-12-27 08:30:50,768][20347] Updated weights for policy 0, policy_version 415792 (0.0010) [2023-12-27 08:30:50,826][20347] Updated weights for policy 0, policy_version 415802 (0.0010) [2023-12-27 08:30:51,282][20381] Updated weights for policy 1, policy_version 415350 (0.0010) [2023-12-27 08:30:51,333][20381] Updated weights for policy 1, policy_version 415360 (0.0009) [2023-12-27 08:30:51,394][20381] Updated weights for policy 1, policy_version 415370 (0.0008) [2023-12-27 08:30:51,494][20347] Updated weights for policy 0, policy_version 415812 (0.0009) [2023-12-27 08:30:51,547][20347] Updated weights for policy 0, policy_version 415822 (0.0005) [2023-12-27 08:30:51,604][20347] Updated weights for policy 0, policy_version 415832 (0.0008) [2023-12-27 08:30:52,087][20381] Updated weights for policy 1, policy_version 415380 (0.0006) [2023-12-27 08:30:52,137][20381] Updated weights for policy 1, policy_version 415390 (0.0008) [2023-12-27 08:30:52,190][20381] Updated weights for policy 1, policy_version 415400 (0.0007) [2023-12-27 08:30:52,281][20347] Updated weights for policy 0, policy_version 415842 (0.0007) [2023-12-27 08:30:52,340][20347] Updated weights for policy 0, policy_version 415852 (0.0008) [2023-12-27 08:30:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 212836352. Throughput: 0: 10277.5, 1: 10294.2. Samples: 212805736. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:52,363][19170] Avg episode reward: [(0, '860.510'), (1, '981.650')] [2023-12-27 08:30:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000415408_106364928.pth... [2023-12-27 08:30:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000414192_106053632.pth [2023-12-27 08:30:52,403][20347] Updated weights for policy 0, policy_version 415862 (0.0008) [2023-12-27 08:30:52,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000415872_106479616.pth... [2023-12-27 08:30:52,466][20347] Updated weights for policy 0, policy_version 415872 (0.0008) [2023-12-27 08:30:52,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000414624_106160128.pth [2023-12-27 08:30:52,879][20381] Updated weights for policy 1, policy_version 415410 (0.0008) [2023-12-27 08:30:52,933][20381] Updated weights for policy 1, policy_version 415420 (0.0007) [2023-12-27 08:30:52,991][20381] Updated weights for policy 1, policy_version 415430 (0.0008) [2023-12-27 08:30:53,051][20381] Updated weights for policy 1, policy_version 415440 (0.0008) [2023-12-27 08:30:53,185][20347] Updated weights for policy 0, policy_version 415882 (0.0008) [2023-12-27 08:30:53,247][20347] Updated weights for policy 0, policy_version 415892 (0.0007) [2023-12-27 08:30:53,304][20347] Updated weights for policy 0, policy_version 415902 (0.0008) [2023-12-27 08:30:53,714][20381] Updated weights for policy 1, policy_version 415450 (0.0008) [2023-12-27 08:30:53,768][20381] Updated weights for policy 1, policy_version 415460 (0.0008) [2023-12-27 08:30:53,816][20381] Updated weights for policy 1, policy_version 415470 (0.0007) [2023-12-27 08:30:54,002][20347] Updated weights for policy 0, policy_version 415912 (0.0008) [2023-12-27 08:30:54,063][20347] Updated weights for policy 0, policy_version 415922 (0.0007) [2023-12-27 08:30:54,117][20347] Updated weights for policy 0, policy_version 415932 (0.0007) [2023-12-27 08:30:54,516][20381] Updated weights for policy 1, policy_version 415480 (0.0007) [2023-12-27 08:30:54,580][20381] Updated weights for policy 1, policy_version 415490 (0.0007) [2023-12-27 08:30:54,661][20381] Updated weights for policy 1, policy_version 415500 (0.0008) [2023-12-27 08:30:54,810][20347] Updated weights for policy 0, policy_version 415942 (0.0008) [2023-12-27 08:30:54,875][20347] Updated weights for policy 0, policy_version 415952 (0.0008) [2023-12-27 08:30:54,939][20347] Updated weights for policy 0, policy_version 415962 (0.0008) [2023-12-27 08:30:55,286][20381] Updated weights for policy 1, policy_version 415510 (0.0009) [2023-12-27 08:30:55,334][20381] Updated weights for policy 1, policy_version 415520 (0.0010) [2023-12-27 08:30:55,379][20381] Updated weights for policy 1, policy_version 415530 (0.0010) [2023-12-27 08:30:55,587][20347] Updated weights for policy 0, policy_version 415972 (0.0007) [2023-12-27 08:30:55,641][20347] Updated weights for policy 0, policy_version 415982 (0.0005) [2023-12-27 08:30:55,703][20347] Updated weights for policy 0, policy_version 415992 (0.0005) [2023-12-27 08:30:56,063][20381] Updated weights for policy 1, policy_version 415540 (0.0008) [2023-12-27 08:30:56,116][20381] Updated weights for policy 1, policy_version 415550 (0.0005) [2023-12-27 08:30:56,170][20381] Updated weights for policy 1, policy_version 415560 (0.0009) [2023-12-27 08:30:56,252][20347] Updated weights for policy 0, policy_version 416002 (0.0006) [2023-12-27 08:30:56,296][20347] Updated weights for policy 0, policy_version 416012 (0.0010) [2023-12-27 08:30:56,340][20347] Updated weights for policy 0, policy_version 416022 (0.0010) [2023-12-27 08:30:56,381][20347] Updated weights for policy 0, policy_version 416032 (0.0010) [2023-12-27 08:30:56,831][20381] Updated weights for policy 1, policy_version 415570 (0.0010) [2023-12-27 08:30:56,884][20381] Updated weights for policy 1, policy_version 415580 (0.0010) [2023-12-27 08:30:56,940][20381] Updated weights for policy 1, policy_version 415590 (0.0010) [2023-12-27 08:30:56,991][20381] Updated weights for policy 1, policy_version 415600 (0.0010) [2023-12-27 08:30:57,072][20347] Updated weights for policy 0, policy_version 416042 (0.0005) [2023-12-27 08:30:57,125][20347] Updated weights for policy 0, policy_version 416052 (0.0005) [2023-12-27 08:30:57,167][20347] Updated weights for policy 0, policy_version 416062 (0.0005) [2023-12-27 08:30:57,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 212942848. Throughput: 0: 10313.0, 1: 10289.0. Samples: 212929892. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:30:57,364][19170] Avg episode reward: [(0, '891.980'), (1, '991.240')] [2023-12-27 08:30:57,674][20381] Updated weights for policy 1, policy_version 415610 (0.0010) [2023-12-27 08:30:57,726][20381] Updated weights for policy 1, policy_version 415620 (0.0010) [2023-12-27 08:30:57,745][20347] Updated weights for policy 0, policy_version 416072 (0.0008) [2023-12-27 08:30:57,776][20381] Updated weights for policy 1, policy_version 415630 (0.0010) [2023-12-27 08:30:57,790][20347] Updated weights for policy 0, policy_version 416082 (0.0010) [2023-12-27 08:30:57,843][20347] Updated weights for policy 0, policy_version 416092 (0.0010) [2023-12-27 08:30:58,513][20381] Updated weights for policy 1, policy_version 415640 (0.0009) [2023-12-27 08:30:58,554][20347] Updated weights for policy 0, policy_version 416102 (0.0010) [2023-12-27 08:30:58,573][20381] Updated weights for policy 1, policy_version 415650 (0.0009) [2023-12-27 08:30:58,622][20347] Updated weights for policy 0, policy_version 416112 (0.0009) [2023-12-27 08:30:58,628][20381] Updated weights for policy 1, policy_version 415660 (0.0007) [2023-12-27 08:30:58,681][20347] Updated weights for policy 0, policy_version 416122 (0.0007) [2023-12-27 08:30:59,455][20347] Updated weights for policy 0, policy_version 416132 (0.0007) [2023-12-27 08:30:59,474][20381] Updated weights for policy 1, policy_version 415670 (0.0007) [2023-12-27 08:30:59,510][20347] Updated weights for policy 0, policy_version 416142 (0.0005) [2023-12-27 08:30:59,523][20381] Updated weights for policy 1, policy_version 415680 (0.0008) [2023-12-27 08:30:59,563][20347] Updated weights for policy 0, policy_version 416152 (0.0005) [2023-12-27 08:30:59,571][20381] Updated weights for policy 1, policy_version 415690 (0.0007) [2023-12-27 08:31:00,230][20347] Updated weights for policy 0, policy_version 416162 (0.0006) [2023-12-27 08:31:00,266][20381] Updated weights for policy 1, policy_version 415700 (0.0007) [2023-12-27 08:31:00,291][20347] Updated weights for policy 0, policy_version 416172 (0.0007) [2023-12-27 08:31:00,331][20381] Updated weights for policy 1, policy_version 415710 (0.0007) [2023-12-27 08:31:00,349][20347] Updated weights for policy 0, policy_version 416182 (0.0006) [2023-12-27 08:31:00,392][20381] Updated weights for policy 1, policy_version 415720 (0.0007) [2023-12-27 08:31:00,409][20347] Updated weights for policy 0, policy_version 416192 (0.0006) [2023-12-27 08:31:00,973][20347] Updated weights for policy 0, policy_version 416202 (0.0005) [2023-12-27 08:31:00,987][20381] Updated weights for policy 1, policy_version 415730 (0.0008) [2023-12-27 08:31:01,025][20347] Updated weights for policy 0, policy_version 416212 (0.0007) [2023-12-27 08:31:01,043][20381] Updated weights for policy 1, policy_version 415740 (0.0007) [2023-12-27 08:31:01,085][20347] Updated weights for policy 0, policy_version 416222 (0.0007) [2023-12-27 08:31:01,096][20381] Updated weights for policy 1, policy_version 415750 (0.0008) [2023-12-27 08:31:01,162][20381] Updated weights for policy 1, policy_version 415760 (0.0008) [2023-12-27 08:31:01,782][20347] Updated weights for policy 0, policy_version 416232 (0.0008) [2023-12-27 08:31:01,832][20381] Updated weights for policy 1, policy_version 415770 (0.0006) [2023-12-27 08:31:01,837][20347] Updated weights for policy 0, policy_version 416242 (0.0008) [2023-12-27 08:31:01,887][20381] Updated weights for policy 1, policy_version 415780 (0.0006) [2023-12-27 08:31:01,892][20347] Updated weights for policy 0, policy_version 416252 (0.0008) [2023-12-27 08:31:01,930][20381] Updated weights for policy 1, policy_version 415790 (0.0006) [2023-12-27 08:31:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 213041152. Throughput: 0: 10373.0, 1: 10256.5. Samples: 213052596. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:02,363][19170] Avg episode reward: [(0, '952.830'), (1, '980.820')] [2023-12-27 08:31:02,582][20347] Updated weights for policy 0, policy_version 416262 (0.0006) [2023-12-27 08:31:02,622][20381] Updated weights for policy 1, policy_version 415800 (0.0008) [2023-12-27 08:31:02,634][20347] Updated weights for policy 0, policy_version 416272 (0.0010) [2023-12-27 08:31:02,682][20381] Updated weights for policy 1, policy_version 415810 (0.0009) [2023-12-27 08:31:02,692][20347] Updated weights for policy 0, policy_version 416282 (0.0010) [2023-12-27 08:31:02,739][20381] Updated weights for policy 1, policy_version 415820 (0.0006) [2023-12-27 08:31:03,308][20347] Updated weights for policy 0, policy_version 416292 (0.0008) [2023-12-27 08:31:03,344][20381] Updated weights for policy 1, policy_version 415830 (0.0007) [2023-12-27 08:31:03,365][20347] Updated weights for policy 0, policy_version 416302 (0.0005) [2023-12-27 08:31:03,402][20381] Updated weights for policy 1, policy_version 415840 (0.0007) [2023-12-27 08:31:03,424][20347] Updated weights for policy 0, policy_version 416312 (0.0010) [2023-12-27 08:31:03,465][20381] Updated weights for policy 1, policy_version 415850 (0.0006) [2023-12-27 08:31:04,104][20347] Updated weights for policy 0, policy_version 416322 (0.0010) [2023-12-27 08:31:04,136][20381] Updated weights for policy 1, policy_version 415860 (0.0006) [2023-12-27 08:31:04,171][20347] Updated weights for policy 0, policy_version 416332 (0.0010) [2023-12-27 08:31:04,191][20381] Updated weights for policy 1, policy_version 415870 (0.0006) [2023-12-27 08:31:04,227][20347] Updated weights for policy 0, policy_version 416342 (0.0011) [2023-12-27 08:31:04,255][20381] Updated weights for policy 1, policy_version 415880 (0.0006) [2023-12-27 08:31:04,283][20347] Updated weights for policy 0, policy_version 416352 (0.0010) [2023-12-27 08:31:04,891][20381] Updated weights for policy 1, policy_version 415890 (0.0006) [2023-12-27 08:31:04,944][20381] Updated weights for policy 1, policy_version 415900 (0.0006) [2023-12-27 08:31:04,968][20347] Updated weights for policy 0, policy_version 416362 (0.0007) [2023-12-27 08:31:04,996][20381] Updated weights for policy 1, policy_version 415910 (0.0005) [2023-12-27 08:31:05,017][20347] Updated weights for policy 0, policy_version 416372 (0.0006) [2023-12-27 08:31:05,051][20381] Updated weights for policy 1, policy_version 415920 (0.0008) [2023-12-27 08:31:05,071][20347] Updated weights for policy 0, policy_version 416382 (0.0009) [2023-12-27 08:31:05,664][20381] Updated weights for policy 1, policy_version 415930 (0.0007) [2023-12-27 08:31:05,718][20381] Updated weights for policy 1, policy_version 415940 (0.0007) [2023-12-27 08:31:05,771][20381] Updated weights for policy 1, policy_version 415950 (0.0007) [2023-12-27 08:31:05,802][20347] Updated weights for policy 0, policy_version 416392 (0.0010) [2023-12-27 08:31:05,850][20347] Updated weights for policy 0, policy_version 416402 (0.0010) [2023-12-27 08:31:05,895][20347] Updated weights for policy 0, policy_version 416412 (0.0010) [2023-12-27 08:31:06,454][20381] Updated weights for policy 1, policy_version 415960 (0.0006) [2023-12-27 08:31:06,513][20381] Updated weights for policy 1, policy_version 415970 (0.0007) [2023-12-27 08:31:06,571][20381] Updated weights for policy 1, policy_version 415980 (0.0007) [2023-12-27 08:31:06,572][20347] Updated weights for policy 0, policy_version 416422 (0.0008) [2023-12-27 08:31:06,634][20347] Updated weights for policy 0, policy_version 416432 (0.0011) [2023-12-27 08:31:06,690][20347] Updated weights for policy 0, policy_version 416442 (0.0011) [2023-12-27 08:31:07,233][20381] Updated weights for policy 1, policy_version 415990 (0.0009) [2023-12-27 08:31:07,279][20381] Updated weights for policy 1, policy_version 416000 (0.0010) [2023-12-27 08:31:07,328][20381] Updated weights for policy 1, policy_version 416010 (0.0011) [2023-12-27 08:31:07,339][20347] Updated weights for policy 0, policy_version 416452 (0.0009) [2023-12-27 08:31:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20799.3). Total num frames: 213147648. Throughput: 0: 10389.8, 1: 10321.7. Samples: 213115688. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:07,364][19170] Avg episode reward: [(0, '971.340'), (1, '961.840')] [2023-12-27 08:31:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000416016_106520576.pth... [2023-12-27 08:31:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000414800_106209280.pth [2023-12-27 08:31:07,391][20347] Updated weights for policy 0, policy_version 416462 (0.0008) [2023-12-27 08:31:07,445][20347] Updated weights for policy 0, policy_version 416472 (0.0011) [2023-12-27 08:31:07,491][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000416480_106635264.pth... [2023-12-27 08:31:07,495][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000415264_106323968.pth [2023-12-27 08:31:08,023][20381] Updated weights for policy 1, policy_version 416020 (0.0009) [2023-12-27 08:31:08,071][20381] Updated weights for policy 1, policy_version 416030 (0.0007) [2023-12-27 08:31:08,122][20381] Updated weights for policy 1, policy_version 416040 (0.0007) [2023-12-27 08:31:08,140][20347] Updated weights for policy 0, policy_version 416482 (0.0007) [2023-12-27 08:31:08,201][20347] Updated weights for policy 0, policy_version 416492 (0.0008) [2023-12-27 08:31:08,258][20347] Updated weights for policy 0, policy_version 416502 (0.0008) [2023-12-27 08:31:08,310][20347] Updated weights for policy 0, policy_version 416512 (0.0007) [2023-12-27 08:31:08,792][20381] Updated weights for policy 1, policy_version 416050 (0.0008) [2023-12-27 08:31:08,851][20381] Updated weights for policy 1, policy_version 416060 (0.0009) [2023-12-27 08:31:08,903][20381] Updated weights for policy 1, policy_version 416070 (0.0005) [2023-12-27 08:31:08,962][20381] Updated weights for policy 1, policy_version 416080 (0.0007) [2023-12-27 08:31:08,993][20347] Updated weights for policy 0, policy_version 416522 (0.0008) [2023-12-27 08:31:09,059][20347] Updated weights for policy 0, policy_version 416532 (0.0009) [2023-12-27 08:31:09,118][20347] Updated weights for policy 0, policy_version 416542 (0.0006) [2023-12-27 08:31:09,615][20381] Updated weights for policy 1, policy_version 416090 (0.0008) [2023-12-27 08:31:09,668][20381] Updated weights for policy 1, policy_version 416100 (0.0008) [2023-12-27 08:31:09,728][20381] Updated weights for policy 1, policy_version 416110 (0.0007) [2023-12-27 08:31:09,796][20347] Updated weights for policy 0, policy_version 416552 (0.0007) [2023-12-27 08:31:09,865][20347] Updated weights for policy 0, policy_version 416562 (0.0007) [2023-12-27 08:31:09,924][20347] Updated weights for policy 0, policy_version 416572 (0.0008) [2023-12-27 08:31:10,410][20381] Updated weights for policy 1, policy_version 416120 (0.0007) [2023-12-27 08:31:10,465][20381] Updated weights for policy 1, policy_version 416130 (0.0008) [2023-12-27 08:31:10,515][20381] Updated weights for policy 1, policy_version 416140 (0.0009) [2023-12-27 08:31:10,599][20347] Updated weights for policy 0, policy_version 416582 (0.0007) [2023-12-27 08:31:10,664][20347] Updated weights for policy 0, policy_version 416592 (0.0008) [2023-12-27 08:31:10,719][20347] Updated weights for policy 0, policy_version 416602 (0.0006) [2023-12-27 08:31:11,203][20381] Updated weights for policy 1, policy_version 416150 (0.0009) [2023-12-27 08:31:11,261][20381] Updated weights for policy 1, policy_version 416160 (0.0008) [2023-12-27 08:31:11,314][20381] Updated weights for policy 1, policy_version 416170 (0.0008) [2023-12-27 08:31:11,358][20347] Updated weights for policy 0, policy_version 416612 (0.0006) [2023-12-27 08:31:11,424][20347] Updated weights for policy 0, policy_version 416622 (0.0007) [2023-12-27 08:31:11,484][20347] Updated weights for policy 0, policy_version 416632 (0.0009) [2023-12-27 08:31:12,056][20381] Updated weights for policy 1, policy_version 416180 (0.0008) [2023-12-27 08:31:12,108][20347] Updated weights for policy 0, policy_version 416642 (0.0007) [2023-12-27 08:31:12,118][20381] Updated weights for policy 1, policy_version 416190 (0.0008) [2023-12-27 08:31:12,169][20347] Updated weights for policy 0, policy_version 416652 (0.0006) [2023-12-27 08:31:12,176][20381] Updated weights for policy 1, policy_version 416200 (0.0007) [2023-12-27 08:31:12,229][20347] Updated weights for policy 0, policy_version 416662 (0.0007) [2023-12-27 08:31:12,293][20347] Updated weights for policy 0, policy_version 416672 (0.0009) [2023-12-27 08:31:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 213254144. Throughput: 0: 10424.9, 1: 10380.3. Samples: 213238704. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:12,363][19170] Avg episode reward: [(0, '954.250'), (1, '981.020')] [2023-12-27 08:31:12,911][20381] Updated weights for policy 1, policy_version 416210 (0.0006) [2023-12-27 08:31:12,959][20381] Updated weights for policy 1, policy_version 416220 (0.0010) [2023-12-27 08:31:12,988][20347] Updated weights for policy 0, policy_version 416682 (0.0007) [2023-12-27 08:31:13,012][20381] Updated weights for policy 1, policy_version 416230 (0.0010) [2023-12-27 08:31:13,043][20347] Updated weights for policy 0, policy_version 416692 (0.0006) [2023-12-27 08:31:13,063][20381] Updated weights for policy 1, policy_version 416240 (0.0010) [2023-12-27 08:31:13,099][20347] Updated weights for policy 0, policy_version 416702 (0.0007) [2023-12-27 08:31:13,753][20381] Updated weights for policy 1, policy_version 416250 (0.0010) [2023-12-27 08:31:13,798][20347] Updated weights for policy 0, policy_version 416712 (0.0006) [2023-12-27 08:31:13,811][20381] Updated weights for policy 1, policy_version 416260 (0.0010) [2023-12-27 08:31:13,852][20347] Updated weights for policy 0, policy_version 416722 (0.0006) [2023-12-27 08:31:13,871][20381] Updated weights for policy 1, policy_version 416270 (0.0010) [2023-12-27 08:31:13,901][20347] Updated weights for policy 0, policy_version 416732 (0.0007) [2023-12-27 08:31:14,559][20381] Updated weights for policy 1, policy_version 416280 (0.0009) [2023-12-27 08:31:14,602][20347] Updated weights for policy 0, policy_version 416742 (0.0007) [2023-12-27 08:31:14,616][20381] Updated weights for policy 1, policy_version 416290 (0.0007) [2023-12-27 08:31:14,668][20347] Updated weights for policy 0, policy_version 416752 (0.0007) [2023-12-27 08:31:14,678][20381] Updated weights for policy 1, policy_version 416300 (0.0005) [2023-12-27 08:31:14,723][20347] Updated weights for policy 0, policy_version 416762 (0.0008) [2023-12-27 08:31:15,360][20347] Updated weights for policy 0, policy_version 416772 (0.0007) [2023-12-27 08:31:15,371][20381] Updated weights for policy 1, policy_version 416310 (0.0007) [2023-12-27 08:31:15,412][20347] Updated weights for policy 0, policy_version 416782 (0.0006) [2023-12-27 08:31:15,427][20381] Updated weights for policy 1, policy_version 416320 (0.0008) [2023-12-27 08:31:15,468][20347] Updated weights for policy 0, policy_version 416792 (0.0010) [2023-12-27 08:31:15,485][20381] Updated weights for policy 1, policy_version 416330 (0.0006) [2023-12-27 08:31:15,499][20291] KL-divergence is very high: 242.6036 [2023-12-27 08:31:16,104][20347] Updated weights for policy 0, policy_version 416802 (0.0010) [2023-12-27 08:31:16,153][20347] Updated weights for policy 0, policy_version 416812 (0.0005) [2023-12-27 08:31:16,206][20381] Updated weights for policy 1, policy_version 416340 (0.0006) [2023-12-27 08:31:16,207][20347] Updated weights for policy 0, policy_version 416822 (0.0005) [2023-12-27 08:31:16,251][20347] Updated weights for policy 0, policy_version 416832 (0.0005) [2023-12-27 08:31:16,261][20381] Updated weights for policy 1, policy_version 416350 (0.0007) [2023-12-27 08:31:16,317][20381] Updated weights for policy 1, policy_version 416360 (0.0008) [2023-12-27 08:31:16,852][20347] Updated weights for policy 0, policy_version 416842 (0.0005) [2023-12-27 08:31:16,912][20347] Updated weights for policy 0, policy_version 416852 (0.0005) [2023-12-27 08:31:16,969][20347] Updated weights for policy 0, policy_version 416862 (0.0005) [2023-12-27 08:31:17,014][20381] Updated weights for policy 1, policy_version 416370 (0.0008) [2023-12-27 08:31:17,068][20381] Updated weights for policy 1, policy_version 416380 (0.0006) [2023-12-27 08:31:17,118][20381] Updated weights for policy 1, policy_version 416390 (0.0005) [2023-12-27 08:31:17,168][20381] Updated weights for policy 1, policy_version 416400 (0.0007) [2023-12-27 08:31:17,363][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 213352448. Throughput: 0: 10391.8, 1: 10270.4. Samples: 213361588. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:17,364][19170] Avg episode reward: [(0, '973.680'), (1, '980.920')] [2023-12-27 08:31:17,568][20347] Updated weights for policy 0, policy_version 416872 (0.0006) [2023-12-27 08:31:17,625][20347] Updated weights for policy 0, policy_version 416882 (0.0007) [2023-12-27 08:31:17,684][20347] Updated weights for policy 0, policy_version 416892 (0.0006) [2023-12-27 08:31:17,835][20381] Updated weights for policy 1, policy_version 416410 (0.0007) [2023-12-27 08:31:17,896][20381] Updated weights for policy 1, policy_version 416420 (0.0007) [2023-12-27 08:31:17,947][20381] Updated weights for policy 1, policy_version 416430 (0.0005) [2023-12-27 08:31:18,339][20347] Updated weights for policy 0, policy_version 416902 (0.0009) [2023-12-27 08:31:18,401][20347] Updated weights for policy 0, policy_version 416912 (0.0008) [2023-12-27 08:31:18,456][20347] Updated weights for policy 0, policy_version 416922 (0.0007) [2023-12-27 08:31:18,573][20381] Updated weights for policy 1, policy_version 416440 (0.0007) [2023-12-27 08:31:18,628][20381] Updated weights for policy 1, policy_version 416450 (0.0007) [2023-12-27 08:31:18,679][20381] Updated weights for policy 1, policy_version 416460 (0.0008) [2023-12-27 08:31:19,146][20347] Updated weights for policy 0, policy_version 416932 (0.0008) [2023-12-27 08:31:19,199][20347] Updated weights for policy 0, policy_version 416942 (0.0009) [2023-12-27 08:31:19,261][20347] Updated weights for policy 0, policy_version 416952 (0.0009) [2023-12-27 08:31:19,391][20381] Updated weights for policy 1, policy_version 416470 (0.0006) [2023-12-27 08:31:19,450][20381] Updated weights for policy 1, policy_version 416480 (0.0005) [2023-12-27 08:31:19,515][20381] Updated weights for policy 1, policy_version 416490 (0.0006) [2023-12-27 08:31:19,982][20347] Updated weights for policy 0, policy_version 416962 (0.0009) [2023-12-27 08:31:20,045][20347] Updated weights for policy 0, policy_version 416972 (0.0006) [2023-12-27 08:31:20,115][20347] Updated weights for policy 0, policy_version 416982 (0.0010) [2023-12-27 08:31:20,156][20381] Updated weights for policy 1, policy_version 416500 (0.0006) [2023-12-27 08:31:20,181][20347] Updated weights for policy 0, policy_version 416992 (0.0010) [2023-12-27 08:31:20,220][20381] Updated weights for policy 1, policy_version 416510 (0.0008) [2023-12-27 08:31:20,287][20381] Updated weights for policy 1, policy_version 416520 (0.0008) [2023-12-27 08:31:20,831][20347] Updated weights for policy 0, policy_version 417002 (0.0009) [2023-12-27 08:31:20,893][20347] Updated weights for policy 0, policy_version 417012 (0.0006) [2023-12-27 08:31:20,956][20347] Updated weights for policy 0, policy_version 417022 (0.0006) [2023-12-27 08:31:20,961][20381] Updated weights for policy 1, policy_version 416530 (0.0008) [2023-12-27 08:31:21,014][20381] Updated weights for policy 1, policy_version 416540 (0.0008) [2023-12-27 08:31:21,075][20381] Updated weights for policy 1, policy_version 416550 (0.0008) [2023-12-27 08:31:21,137][20381] Updated weights for policy 1, policy_version 416560 (0.0009) [2023-12-27 08:31:21,666][20347] Updated weights for policy 0, policy_version 417032 (0.0011) [2023-12-27 08:31:21,735][20347] Updated weights for policy 0, policy_version 417042 (0.0009) [2023-12-27 08:31:21,799][20347] Updated weights for policy 0, policy_version 417052 (0.0008) [2023-12-27 08:31:21,845][20381] Updated weights for policy 1, policy_version 416570 (0.0008) [2023-12-27 08:31:21,900][20381] Updated weights for policy 1, policy_version 416580 (0.0008) [2023-12-27 08:31:21,961][20381] Updated weights for policy 1, policy_version 416590 (0.0007) [2023-12-27 08:31:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 213450752. Throughput: 0: 10360.0, 1: 10277.1. Samples: 213423492. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:22,363][19170] Avg episode reward: [(0, '981.370'), (1, '990.440')] [2023-12-27 08:31:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000416592_106668032.pth... [2023-12-27 08:31:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000417056_106782720.pth... [2023-12-27 08:31:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000415872_106479616.pth [2023-12-27 08:31:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000415408_106364928.pth [2023-12-27 08:31:22,421][20347] Updated weights for policy 0, policy_version 417062 (0.0009) [2023-12-27 08:31:22,476][20347] Updated weights for policy 0, policy_version 417072 (0.0008) [2023-12-27 08:31:22,541][20347] Updated weights for policy 0, policy_version 417082 (0.0009) [2023-12-27 08:31:22,727][20381] Updated weights for policy 1, policy_version 416600 (0.0007) [2023-12-27 08:31:22,781][20381] Updated weights for policy 1, policy_version 416610 (0.0006) [2023-12-27 08:31:22,828][20381] Updated weights for policy 1, policy_version 416620 (0.0006) [2023-12-27 08:31:23,208][20347] Updated weights for policy 0, policy_version 417092 (0.0008) [2023-12-27 08:31:23,269][20347] Updated weights for policy 0, policy_version 417102 (0.0010) [2023-12-27 08:31:23,326][20347] Updated weights for policy 0, policy_version 417112 (0.0010) [2023-12-27 08:31:23,428][20381] Updated weights for policy 1, policy_version 416630 (0.0007) [2023-12-27 08:31:23,485][20381] Updated weights for policy 1, policy_version 416640 (0.0007) [2023-12-27 08:31:23,536][20381] Updated weights for policy 1, policy_version 416650 (0.0007) [2023-12-27 08:31:23,993][20347] Updated weights for policy 0, policy_version 417122 (0.0010) [2023-12-27 08:31:24,050][20347] Updated weights for policy 0, policy_version 417132 (0.0010) [2023-12-27 08:31:24,110][20347] Updated weights for policy 0, policy_version 417142 (0.0006) [2023-12-27 08:31:24,170][20347] Updated weights for policy 0, policy_version 417152 (0.0010) [2023-12-27 08:31:24,256][20381] Updated weights for policy 1, policy_version 416660 (0.0007) [2023-12-27 08:31:24,320][20381] Updated weights for policy 1, policy_version 416670 (0.0005) [2023-12-27 08:31:24,377][20381] Updated weights for policy 1, policy_version 416680 (0.0005) [2023-12-27 08:31:24,834][20347] Updated weights for policy 0, policy_version 417162 (0.0007) [2023-12-27 08:31:24,897][20347] Updated weights for policy 0, policy_version 417172 (0.0008) [2023-12-27 08:31:24,911][20381] Updated weights for policy 1, policy_version 416690 (0.0006) [2023-12-27 08:31:24,955][20347] Updated weights for policy 0, policy_version 417182 (0.0005) [2023-12-27 08:31:24,970][20381] Updated weights for policy 1, policy_version 416700 (0.0010) [2023-12-27 08:31:25,034][20381] Updated weights for policy 1, policy_version 416710 (0.0009) [2023-12-27 08:31:25,093][20381] Updated weights for policy 1, policy_version 416720 (0.0005) [2023-12-27 08:31:25,510][20347] Updated weights for policy 0, policy_version 417192 (0.0005) [2023-12-27 08:31:25,555][20347] Updated weights for policy 0, policy_version 417202 (0.0005) [2023-12-27 08:31:25,612][20347] Updated weights for policy 0, policy_version 417212 (0.0005) [2023-12-27 08:31:25,658][20381] Updated weights for policy 1, policy_version 416730 (0.0010) [2023-12-27 08:31:25,713][20381] Updated weights for policy 1, policy_version 416740 (0.0010) [2023-12-27 08:31:25,768][20381] Updated weights for policy 1, policy_version 416750 (0.0010) [2023-12-27 08:31:26,143][20347] Updated weights for policy 0, policy_version 417222 (0.0007) [2023-12-27 08:31:26,198][20347] Updated weights for policy 0, policy_version 417232 (0.0006) [2023-12-27 08:31:26,257][20347] Updated weights for policy 0, policy_version 417242 (0.0006) [2023-12-27 08:31:26,468][20381] Updated weights for policy 1, policy_version 416760 (0.0009) [2023-12-27 08:31:26,522][20381] Updated weights for policy 1, policy_version 416770 (0.0010) [2023-12-27 08:31:26,576][20381] Updated weights for policy 1, policy_version 416780 (0.0010) [2023-12-27 08:31:26,858][20347] Updated weights for policy 0, policy_version 417252 (0.0007) [2023-12-27 08:31:26,913][20347] Updated weights for policy 0, policy_version 417262 (0.0005) [2023-12-27 08:31:26,965][20347] Updated weights for policy 0, policy_version 417272 (0.0005) [2023-12-27 08:31:27,267][20381] Updated weights for policy 1, policy_version 416790 (0.0010) [2023-12-27 08:31:27,315][20381] Updated weights for policy 1, policy_version 416800 (0.0010) [2023-12-27 08:31:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 213557248. Throughput: 0: 10390.4, 1: 10265.1. Samples: 213549704. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:27,363][19170] Avg episode reward: [(0, '980.960'), (1, '980.780')] [2023-12-27 08:31:27,364][20381] Updated weights for policy 1, policy_version 416810 (0.0010) [2023-12-27 08:31:27,570][20347] Updated weights for policy 0, policy_version 417282 (0.0006) [2023-12-27 08:31:27,615][20347] Updated weights for policy 0, policy_version 417292 (0.0008) [2023-12-27 08:31:27,663][20347] Updated weights for policy 0, policy_version 417302 (0.0006) [2023-12-27 08:31:27,678][20291] KL-divergence is very high: 792.6099 [2023-12-27 08:31:27,714][20347] Updated weights for policy 0, policy_version 417312 (0.0010) [2023-12-27 08:31:28,036][20381] Updated weights for policy 1, policy_version 416820 (0.0010) [2023-12-27 08:31:28,096][20381] Updated weights for policy 1, policy_version 416830 (0.0010) [2023-12-27 08:31:28,147][20381] Updated weights for policy 1, policy_version 416840 (0.0010) [2023-12-27 08:31:28,369][20347] Updated weights for policy 0, policy_version 417322 (0.0009) [2023-12-27 08:31:28,427][20347] Updated weights for policy 0, policy_version 417332 (0.0008) [2023-12-27 08:31:28,486][20347] Updated weights for policy 0, policy_version 417342 (0.0009) [2023-12-27 08:31:28,752][20381] Updated weights for policy 1, policy_version 416850 (0.0009) [2023-12-27 08:31:28,811][20381] Updated weights for policy 1, policy_version 416860 (0.0005) [2023-12-27 08:31:28,868][20381] Updated weights for policy 1, policy_version 416870 (0.0006) [2023-12-27 08:31:28,923][20381] Updated weights for policy 1, policy_version 416880 (0.0005) [2023-12-27 08:31:29,183][20347] Updated weights for policy 0, policy_version 417352 (0.0010) [2023-12-27 08:31:29,246][20347] Updated weights for policy 0, policy_version 417362 (0.0009) [2023-12-27 08:31:29,289][20291] KL-divergence is very high: 117.4890 [2023-12-27 08:31:29,304][20347] Updated weights for policy 0, policy_version 417372 (0.0007) [2023-12-27 08:31:29,535][20381] Updated weights for policy 1, policy_version 416890 (0.0007) [2023-12-27 08:31:29,589][20381] Updated weights for policy 1, policy_version 416900 (0.0008) [2023-12-27 08:31:29,645][20381] Updated weights for policy 1, policy_version 416910 (0.0007) [2023-12-27 08:31:30,012][20347] Updated weights for policy 0, policy_version 417382 (0.0010) [2023-12-27 08:31:30,073][20347] Updated weights for policy 0, policy_version 417392 (0.0007) [2023-12-27 08:31:30,130][20347] Updated weights for policy 0, policy_version 417402 (0.0008) [2023-12-27 08:31:30,314][20381] Updated weights for policy 1, policy_version 416920 (0.0007) [2023-12-27 08:31:30,368][20381] Updated weights for policy 1, policy_version 416930 (0.0007) [2023-12-27 08:31:30,414][20381] Updated weights for policy 1, policy_version 416940 (0.0008) [2023-12-27 08:31:30,757][20347] Updated weights for policy 0, policy_version 417412 (0.0007) [2023-12-27 08:31:30,805][20347] Updated weights for policy 0, policy_version 417422 (0.0010) [2023-12-27 08:31:30,856][20347] Updated weights for policy 0, policy_version 417432 (0.0010) [2023-12-27 08:31:31,037][20381] Updated weights for policy 1, policy_version 416950 (0.0007) [2023-12-27 08:31:31,099][20381] Updated weights for policy 1, policy_version 416960 (0.0007) [2023-12-27 08:31:31,163][20381] Updated weights for policy 1, policy_version 416970 (0.0007) [2023-12-27 08:31:31,564][20347] Updated weights for policy 0, policy_version 417442 (0.0009) [2023-12-27 08:31:31,626][20347] Updated weights for policy 0, policy_version 417452 (0.0008) [2023-12-27 08:31:31,684][20347] Updated weights for policy 0, policy_version 417462 (0.0006) [2023-12-27 08:31:31,753][20347] Updated weights for policy 0, policy_version 417472 (0.0007) [2023-12-27 08:31:31,824][20381] Updated weights for policy 1, policy_version 416980 (0.0006) [2023-12-27 08:31:31,877][20381] Updated weights for policy 1, policy_version 416990 (0.0008) [2023-12-27 08:31:31,930][20381] Updated weights for policy 1, policy_version 417000 (0.0008) [2023-12-27 08:31:32,311][20347] Updated weights for policy 0, policy_version 417482 (0.0007) [2023-12-27 08:31:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 213663744. Throughput: 0: 10435.9, 1: 10277.3. Samples: 213676656. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:32,363][19170] Avg episode reward: [(0, '963.230'), (1, '980.780')] [2023-12-27 08:31:32,372][20347] Updated weights for policy 0, policy_version 417492 (0.0007) [2023-12-27 08:31:32,425][20347] Updated weights for policy 0, policy_version 417502 (0.0005) [2023-12-27 08:31:32,652][20381] Updated weights for policy 1, policy_version 417010 (0.0009) [2023-12-27 08:31:32,696][20381] Updated weights for policy 1, policy_version 417020 (0.0007) [2023-12-27 08:31:32,742][20381] Updated weights for policy 1, policy_version 417030 (0.0008) [2023-12-27 08:31:32,791][20381] Updated weights for policy 1, policy_version 417040 (0.0008) [2023-12-27 08:31:33,008][20347] Updated weights for policy 0, policy_version 417512 (0.0005) [2023-12-27 08:31:33,066][20347] Updated weights for policy 0, policy_version 417522 (0.0006) [2023-12-27 08:31:33,110][20347] Updated weights for policy 0, policy_version 417532 (0.0008) [2023-12-27 08:31:33,449][20381] Updated weights for policy 1, policy_version 417050 (0.0008) [2023-12-27 08:31:33,505][20381] Updated weights for policy 1, policy_version 417060 (0.0005) [2023-12-27 08:31:33,559][20381] Updated weights for policy 1, policy_version 417070 (0.0005) [2023-12-27 08:31:33,809][20347] Updated weights for policy 0, policy_version 417542 (0.0007) [2023-12-27 08:31:33,857][20347] Updated weights for policy 0, policy_version 417552 (0.0007) [2023-12-27 08:31:33,910][20347] Updated weights for policy 0, policy_version 417562 (0.0008) [2023-12-27 08:31:34,184][20381] Updated weights for policy 1, policy_version 417080 (0.0009) [2023-12-27 08:31:34,233][20381] Updated weights for policy 1, policy_version 417090 (0.0007) [2023-12-27 08:31:34,282][20381] Updated weights for policy 1, policy_version 417100 (0.0007) [2023-12-27 08:31:34,624][20347] Updated weights for policy 0, policy_version 417572 (0.0008) [2023-12-27 08:31:34,674][20347] Updated weights for policy 0, policy_version 417582 (0.0007) [2023-12-27 08:31:34,721][20347] Updated weights for policy 0, policy_version 417592 (0.0008) [2023-12-27 08:31:34,997][20381] Updated weights for policy 1, policy_version 417110 (0.0006) [2023-12-27 08:31:35,050][20381] Updated weights for policy 1, policy_version 417120 (0.0006) [2023-12-27 08:31:35,108][20381] Updated weights for policy 1, policy_version 417130 (0.0005) [2023-12-27 08:31:35,505][20347] Updated weights for policy 0, policy_version 417602 (0.0008) [2023-12-27 08:31:35,555][20347] Updated weights for policy 0, policy_version 417612 (0.0009) [2023-12-27 08:31:35,605][20347] Updated weights for policy 0, policy_version 417622 (0.0009) [2023-12-27 08:31:35,661][20347] Updated weights for policy 0, policy_version 417632 (0.0007) [2023-12-27 08:31:35,663][20381] Updated weights for policy 1, policy_version 417140 (0.0007) [2023-12-27 08:31:35,714][20381] Updated weights for policy 1, policy_version 417150 (0.0010) [2023-12-27 08:31:35,766][20381] Updated weights for policy 1, policy_version 417160 (0.0010) [2023-12-27 08:31:36,350][20347] Updated weights for policy 0, policy_version 417642 (0.0007) [2023-12-27 08:31:36,409][20347] Updated weights for policy 0, policy_version 417652 (0.0007) [2023-12-27 08:31:36,469][20381] Updated weights for policy 1, policy_version 417170 (0.0007) [2023-12-27 08:31:36,477][20347] Updated weights for policy 0, policy_version 417662 (0.0007) [2023-12-27 08:31:36,521][20381] Updated weights for policy 1, policy_version 417180 (0.0007) [2023-12-27 08:31:36,582][20381] Updated weights for policy 1, policy_version 417190 (0.0007) [2023-12-27 08:31:36,639][20381] Updated weights for policy 1, policy_version 417200 (0.0007) [2023-12-27 08:31:37,182][20347] Updated weights for policy 0, policy_version 417672 (0.0010) [2023-12-27 08:31:37,245][20347] Updated weights for policy 0, policy_version 417682 (0.0010) [2023-12-27 08:31:37,305][20347] Updated weights for policy 0, policy_version 417692 (0.0010) [2023-12-27 08:31:37,349][20381] Updated weights for policy 1, policy_version 417210 (0.0006) [2023-12-27 08:31:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 213770240. Throughput: 0: 10427.5, 1: 10328.4. Samples: 213739748. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:37,363][19170] Avg episode reward: [(0, '924.780'), (1, '1000.000')] [2023-12-27 08:31:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000417696_106946560.pth... [2023-12-27 08:31:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000416480_106635264.pth [2023-12-27 08:31:37,408][20381] Updated weights for policy 1, policy_version 417220 (0.0007) [2023-12-27 08:31:37,465][20381] Updated weights for policy 1, policy_version 417230 (0.0005) [2023-12-27 08:31:37,475][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000417232_106831872.pth... [2023-12-27 08:31:37,479][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000416016_106520576.pth [2023-12-27 08:31:37,986][20347] Updated weights for policy 0, policy_version 417702 (0.0010) [2023-12-27 08:31:38,043][20347] Updated weights for policy 0, policy_version 417712 (0.0010) [2023-12-27 08:31:38,092][20381] Updated weights for policy 1, policy_version 417240 (0.0007) [2023-12-27 08:31:38,097][20347] Updated weights for policy 0, policy_version 417722 (0.0010) [2023-12-27 08:31:38,138][20381] Updated weights for policy 1, policy_version 417250 (0.0006) [2023-12-27 08:31:38,186][20381] Updated weights for policy 1, policy_version 417260 (0.0006) [2023-12-27 08:31:38,773][20347] Updated weights for policy 0, policy_version 417732 (0.0010) [2023-12-27 08:31:38,838][20291] KL-divergence is very high: 279.5594 [2023-12-27 08:31:38,838][20347] Updated weights for policy 0, policy_version 417742 (0.0010) [2023-12-27 08:31:38,858][20381] Updated weights for policy 1, policy_version 417270 (0.0007) [2023-12-27 08:31:38,900][20347] Updated weights for policy 0, policy_version 417752 (0.0010) [2023-12-27 08:31:38,920][20381] Updated weights for policy 1, policy_version 417280 (0.0006) [2023-12-27 08:31:38,976][20381] Updated weights for policy 1, policy_version 417290 (0.0006) [2023-12-27 08:31:39,609][20347] Updated weights for policy 0, policy_version 417762 (0.0010) [2023-12-27 08:31:39,657][20347] Updated weights for policy 0, policy_version 417772 (0.0007) [2023-12-27 08:31:39,714][20347] Updated weights for policy 0, policy_version 417782 (0.0008) [2023-12-27 08:31:39,724][20381] Updated weights for policy 1, policy_version 417300 (0.0008) [2023-12-27 08:31:39,770][20347] Updated weights for policy 0, policy_version 417792 (0.0007) [2023-12-27 08:31:39,781][20381] Updated weights for policy 1, policy_version 417310 (0.0007) [2023-12-27 08:31:39,845][20381] Updated weights for policy 1, policy_version 417320 (0.0008) [2023-12-27 08:31:40,449][20347] Updated weights for policy 0, policy_version 417802 (0.0009) [2023-12-27 08:31:40,509][20347] Updated weights for policy 0, policy_version 417812 (0.0010) [2023-12-27 08:31:40,567][20381] Updated weights for policy 1, policy_version 417330 (0.0009) [2023-12-27 08:31:40,572][20347] Updated weights for policy 0, policy_version 417822 (0.0010) [2023-12-27 08:31:40,625][20381] Updated weights for policy 1, policy_version 417340 (0.0010) [2023-12-27 08:31:40,683][20381] Updated weights for policy 1, policy_version 417350 (0.0010) [2023-12-27 08:31:40,736][20381] Updated weights for policy 1, policy_version 417360 (0.0010) [2023-12-27 08:31:41,248][20347] Updated weights for policy 0, policy_version 417832 (0.0010) [2023-12-27 08:31:41,313][20347] Updated weights for policy 0, policy_version 417842 (0.0011) [2023-12-27 08:31:41,375][20347] Updated weights for policy 0, policy_version 417852 (0.0009) [2023-12-27 08:31:41,408][20381] Updated weights for policy 1, policy_version 417370 (0.0007) [2023-12-27 08:31:41,476][20381] Updated weights for policy 1, policy_version 417380 (0.0007) [2023-12-27 08:31:41,535][20381] Updated weights for policy 1, policy_version 417390 (0.0007) [2023-12-27 08:31:42,018][20347] Updated weights for policy 0, policy_version 417862 (0.0006) [2023-12-27 08:31:42,068][20347] Updated weights for policy 0, policy_version 417872 (0.0007) [2023-12-27 08:31:42,117][20347] Updated weights for policy 0, policy_version 417882 (0.0008) [2023-12-27 08:31:42,245][20381] Updated weights for policy 1, policy_version 417400 (0.0008) [2023-12-27 08:31:42,307][20381] Updated weights for policy 1, policy_version 417410 (0.0008) [2023-12-27 08:31:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 213868544. Throughput: 0: 10393.6, 1: 10305.7. Samples: 213861360. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) [2023-12-27 08:31:42,363][19170] Avg episode reward: [(0, '923.110'), (1, '990.980')] [2023-12-27 08:31:42,370][20381] Updated weights for policy 1, policy_version 417420 (0.0009) [2023-12-27 08:31:42,727][20347] Updated weights for policy 0, policy_version 417892 (0.0007) [2023-12-27 08:31:42,787][20347] Updated weights for policy 0, policy_version 417902 (0.0007) [2023-12-27 08:31:42,846][20347] Updated weights for policy 0, policy_version 417912 (0.0009) [2023-12-27 08:31:43,037][20381] Updated weights for policy 1, policy_version 417430 (0.0006) [2023-12-27 08:31:43,100][20381] Updated weights for policy 1, policy_version 417440 (0.0006) [2023-12-27 08:31:43,162][20381] Updated weights for policy 1, policy_version 417450 (0.0006) [2023-12-27 08:31:43,557][20347] Updated weights for policy 0, policy_version 417922 (0.0008) [2023-12-27 08:31:43,615][20347] Updated weights for policy 0, policy_version 417932 (0.0005) [2023-12-27 08:31:43,664][20347] Updated weights for policy 0, policy_version 417942 (0.0005) [2023-12-27 08:31:43,718][20347] Updated weights for policy 0, policy_version 417952 (0.0006) [2023-12-27 08:31:43,792][20381] Updated weights for policy 1, policy_version 417460 (0.0006) [2023-12-27 08:31:43,851][20381] Updated weights for policy 1, policy_version 417470 (0.0005) [2023-12-27 08:31:43,902][20381] Updated weights for policy 1, policy_version 417480 (0.0006) [2023-12-27 08:31:44,340][20347] Updated weights for policy 0, policy_version 417962 (0.0005) [2023-12-27 08:31:44,406][20347] Updated weights for policy 0, policy_version 417972 (0.0007) [2023-12-27 08:31:44,466][20347] Updated weights for policy 0, policy_version 417982 (0.0008) [2023-12-27 08:31:44,547][20381] Updated weights for policy 1, policy_version 417490 (0.0006) [2023-12-27 08:31:44,595][20381] Updated weights for policy 1, policy_version 417500 (0.0008) [2023-12-27 08:31:44,650][20381] Updated weights for policy 1, policy_version 417510 (0.0007) [2023-12-27 08:31:44,699][20381] Updated weights for policy 1, policy_version 417520 (0.0008) [2023-12-27 08:31:45,081][20347] Updated weights for policy 0, policy_version 417992 (0.0008) [2023-12-27 08:31:45,140][20347] Updated weights for policy 0, policy_version 418002 (0.0008) [2023-12-27 08:31:45,198][20347] Updated weights for policy 0, policy_version 418012 (0.0007) [2023-12-27 08:31:45,444][20381] Updated weights for policy 1, policy_version 417530 (0.0006) [2023-12-27 08:31:45,505][20381] Updated weights for policy 1, policy_version 417540 (0.0008) [2023-12-27 08:31:45,560][20381] Updated weights for policy 1, policy_version 417550 (0.0010) [2023-12-27 08:31:45,843][20347] Updated weights for policy 0, policy_version 418022 (0.0008) [2023-12-27 08:31:45,908][20347] Updated weights for policy 0, policy_version 418032 (0.0008) [2023-12-27 08:31:45,972][20347] Updated weights for policy 0, policy_version 418042 (0.0007) [2023-12-27 08:31:46,136][20381] Updated weights for policy 1, policy_version 417560 (0.0011) [2023-12-27 08:31:46,188][20381] Updated weights for policy 1, policy_version 417570 (0.0010) [2023-12-27 08:31:46,252][20381] Updated weights for policy 1, policy_version 417580 (0.0010) [2023-12-27 08:31:46,601][20347] Updated weights for policy 0, policy_version 418052 (0.0008) [2023-12-27 08:31:46,661][20347] Updated weights for policy 0, policy_version 418062 (0.0008) [2023-12-27 08:31:46,713][20347] Updated weights for policy 0, policy_version 418072 (0.0007) [2023-12-27 08:31:46,833][20381] Updated weights for policy 1, policy_version 417590 (0.0007) [2023-12-27 08:31:46,896][20381] Updated weights for policy 1, policy_version 417600 (0.0008) [2023-12-27 08:31:46,957][20381] Updated weights for policy 1, policy_version 417610 (0.0007) [2023-12-27 08:31:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 213975040. Throughput: 0: 10415.9, 1: 10372.5. Samples: 213988076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:31:47,363][19170] Avg episode reward: [(0, '971.340'), (1, '954.340')] [2023-12-27 08:31:47,423][20347] Updated weights for policy 0, policy_version 418082 (0.0008) [2023-12-27 08:31:47,473][20347] Updated weights for policy 0, policy_version 418092 (0.0007) [2023-12-27 08:31:47,527][20347] Updated weights for policy 0, policy_version 418102 (0.0007) [2023-12-27 08:31:47,570][20381] Updated weights for policy 1, policy_version 417620 (0.0009) [2023-12-27 08:31:47,580][20347] Updated weights for policy 0, policy_version 418112 (0.0006) [2023-12-27 08:31:47,618][20381] Updated weights for policy 1, policy_version 417630 (0.0010) [2023-12-27 08:31:47,672][20381] Updated weights for policy 1, policy_version 417640 (0.0010) [2023-12-27 08:31:48,283][20347] Updated weights for policy 0, policy_version 418122 (0.0008) [2023-12-27 08:31:48,348][20347] Updated weights for policy 0, policy_version 418132 (0.0007) [2023-12-27 08:31:48,361][20381] Updated weights for policy 1, policy_version 417650 (0.0010) [2023-12-27 08:31:48,404][20347] Updated weights for policy 0, policy_version 418142 (0.0007) [2023-12-27 08:31:48,408][20381] Updated weights for policy 1, policy_version 417660 (0.0008) [2023-12-27 08:31:48,460][20381] Updated weights for policy 1, policy_version 417670 (0.0006) [2023-12-27 08:31:48,521][20381] Updated weights for policy 1, policy_version 417680 (0.0005) [2023-12-27 08:31:48,990][20347] Updated weights for policy 0, policy_version 418152 (0.0008) [2023-12-27 08:31:49,048][20347] Updated weights for policy 0, policy_version 418162 (0.0009) [2023-12-27 08:31:49,101][20347] Updated weights for policy 0, policy_version 418172 (0.0008) [2023-12-27 08:31:49,145][20381] Updated weights for policy 1, policy_version 417690 (0.0005) [2023-12-27 08:31:49,199][20381] Updated weights for policy 1, policy_version 417700 (0.0008) [2023-12-27 08:31:49,263][20381] Updated weights for policy 1, policy_version 417710 (0.0009) [2023-12-27 08:31:49,780][20347] Updated weights for policy 0, policy_version 418182 (0.0007) [2023-12-27 08:31:49,839][20347] Updated weights for policy 0, policy_version 418192 (0.0007) [2023-12-27 08:31:49,900][20347] Updated weights for policy 0, policy_version 418202 (0.0007) [2023-12-27 08:31:49,969][20381] Updated weights for policy 1, policy_version 417720 (0.0007) [2023-12-27 08:31:50,029][20381] Updated weights for policy 1, policy_version 417730 (0.0008) [2023-12-27 08:31:50,088][20381] Updated weights for policy 1, policy_version 417740 (0.0006) [2023-12-27 08:31:50,637][20347] Updated weights for policy 0, policy_version 418212 (0.0008) [2023-12-27 08:31:50,701][20347] Updated weights for policy 0, policy_version 418222 (0.0008) [2023-12-27 08:31:50,703][20381] Updated weights for policy 1, policy_version 417750 (0.0006) [2023-12-27 08:31:50,757][20381] Updated weights for policy 1, policy_version 417760 (0.0007) [2023-12-27 08:31:50,760][20347] Updated weights for policy 0, policy_version 418232 (0.0008) [2023-12-27 08:31:50,815][20381] Updated weights for policy 1, policy_version 417770 (0.0007) [2023-12-27 08:31:51,478][20347] Updated weights for policy 0, policy_version 418242 (0.0006) [2023-12-27 08:31:51,502][20381] Updated weights for policy 1, policy_version 417780 (0.0007) [2023-12-27 08:31:51,531][20347] Updated weights for policy 0, policy_version 418252 (0.0007) [2023-12-27 08:31:51,552][20381] Updated weights for policy 1, policy_version 417790 (0.0005) [2023-12-27 08:31:51,584][20347] Updated weights for policy 0, policy_version 418262 (0.0006) [2023-12-27 08:31:51,606][20381] Updated weights for policy 1, policy_version 417800 (0.0006) [2023-12-27 08:31:51,648][20347] Updated weights for policy 0, policy_version 418272 (0.0007) [2023-12-27 08:31:52,247][20347] Updated weights for policy 0, policy_version 418282 (0.0006) [2023-12-27 08:31:52,305][20347] Updated weights for policy 0, policy_version 418292 (0.0008) [2023-12-27 08:31:52,343][20381] Updated weights for policy 1, policy_version 417810 (0.0008) [2023-12-27 08:31:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 214073344. Throughput: 0: 10415.9, 1: 10352.4. Samples: 214050256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:31:52,363][19170] Avg episode reward: [(0, '990.440'), (1, '963.360')] [2023-12-27 08:31:52,369][20347] Updated weights for policy 0, policy_version 418302 (0.0007) [2023-12-27 08:31:52,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000418304_107102208.pth... [2023-12-27 08:31:52,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000417056_106782720.pth [2023-12-27 08:31:52,404][20381] Updated weights for policy 1, policy_version 417820 (0.0007) [2023-12-27 08:31:52,473][20381] Updated weights for policy 1, policy_version 417830 (0.0006) [2023-12-27 08:31:52,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000417840_106987520.pth... [2023-12-27 08:31:52,538][20381] Updated weights for policy 1, policy_version 417840 (0.0007) [2023-12-27 08:31:52,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000416592_106668032.pth [2023-12-27 08:31:53,000][20347] Updated weights for policy 0, policy_version 418312 (0.0006) [2023-12-27 08:31:53,061][20347] Updated weights for policy 0, policy_version 418322 (0.0005) [2023-12-27 08:31:53,123][20347] Updated weights for policy 0, policy_version 418332 (0.0006) [2023-12-27 08:31:53,212][20381] Updated weights for policy 1, policy_version 417850 (0.0007) [2023-12-27 08:31:53,265][20381] Updated weights for policy 1, policy_version 417860 (0.0008) [2023-12-27 08:31:53,321][20381] Updated weights for policy 1, policy_version 417870 (0.0008) [2023-12-27 08:31:53,769][20347] Updated weights for policy 0, policy_version 418342 (0.0007) [2023-12-27 08:31:53,820][20347] Updated weights for policy 0, policy_version 418352 (0.0007) [2023-12-27 08:31:53,867][20347] Updated weights for policy 0, policy_version 418362 (0.0007) [2023-12-27 08:31:54,003][20381] Updated weights for policy 1, policy_version 417880 (0.0010) [2023-12-27 08:31:54,052][20381] Updated weights for policy 1, policy_version 417890 (0.0010) [2023-12-27 08:31:54,106][20381] Updated weights for policy 1, policy_version 417900 (0.0010) [2023-12-27 08:31:54,575][20347] Updated weights for policy 0, policy_version 418372 (0.0007) [2023-12-27 08:31:54,632][20347] Updated weights for policy 0, policy_version 418382 (0.0007) [2023-12-27 08:31:54,678][20347] Updated weights for policy 0, policy_version 418392 (0.0008) [2023-12-27 08:31:54,797][20381] Updated weights for policy 1, policy_version 417910 (0.0008) [2023-12-27 08:31:54,858][20381] Updated weights for policy 1, policy_version 417920 (0.0008) [2023-12-27 08:31:54,925][20381] Updated weights for policy 1, policy_version 417930 (0.0008) [2023-12-27 08:31:55,388][20347] Updated weights for policy 0, policy_version 418402 (0.0008) [2023-12-27 08:31:55,438][20347] Updated weights for policy 0, policy_version 418412 (0.0008) [2023-12-27 08:31:55,482][20347] Updated weights for policy 0, policy_version 418422 (0.0007) [2023-12-27 08:31:55,530][20347] Updated weights for policy 0, policy_version 418432 (0.0007) [2023-12-27 08:31:55,583][20381] Updated weights for policy 1, policy_version 417940 (0.0009) [2023-12-27 08:31:55,642][20381] Updated weights for policy 1, policy_version 417950 (0.0009) [2023-12-27 08:31:55,696][20381] Updated weights for policy 1, policy_version 417960 (0.0008) [2023-12-27 08:31:56,265][20381] Updated weights for policy 1, policy_version 417970 (0.0010) [2023-12-27 08:31:56,311][20347] Updated weights for policy 0, policy_version 418442 (0.0008) [2023-12-27 08:31:56,324][20381] Updated weights for policy 1, policy_version 417980 (0.0010) [2023-12-27 08:31:56,367][20347] Updated weights for policy 0, policy_version 418452 (0.0005) [2023-12-27 08:31:56,379][20381] Updated weights for policy 1, policy_version 417990 (0.0010) [2023-12-27 08:31:56,415][20347] Updated weights for policy 0, policy_version 418462 (0.0005) [2023-12-27 08:31:56,435][20381] Updated weights for policy 1, policy_version 418000 (0.0010) [2023-12-27 08:31:57,014][20381] Updated weights for policy 1, policy_version 418010 (0.0009) [2023-12-27 08:31:57,062][20381] Updated weights for policy 1, policy_version 418020 (0.0010) [2023-12-27 08:31:57,116][20381] Updated weights for policy 1, policy_version 418030 (0.0010) [2023-12-27 08:31:57,151][20347] Updated weights for policy 0, policy_version 418472 (0.0006) [2023-12-27 08:31:57,204][20347] Updated weights for policy 0, policy_version 418482 (0.0007) [2023-12-27 08:31:57,254][20347] Updated weights for policy 0, policy_version 418492 (0.0007) [2023-12-27 08:31:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 214188032. Throughput: 0: 10415.6, 1: 10359.6. Samples: 214173592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:31:57,363][19170] Avg episode reward: [(0, '980.830'), (1, '990.370')] [2023-12-27 08:31:57,769][20381] Updated weights for policy 1, policy_version 418040 (0.0006) [2023-12-27 08:31:57,819][20381] Updated weights for policy 1, policy_version 418050 (0.0005) [2023-12-27 08:31:57,869][20381] Updated weights for policy 1, policy_version 418060 (0.0005) [2023-12-27 08:31:57,965][20347] Updated weights for policy 0, policy_version 418502 (0.0007) [2023-12-27 08:31:58,010][20347] Updated weights for policy 0, policy_version 418512 (0.0008) [2023-12-27 08:31:58,058][20347] Updated weights for policy 0, policy_version 418522 (0.0008) [2023-12-27 08:31:58,562][20381] Updated weights for policy 1, policy_version 418070 (0.0008) [2023-12-27 08:31:58,623][20381] Updated weights for policy 1, policy_version 418080 (0.0009) [2023-12-27 08:31:58,681][20381] Updated weights for policy 1, policy_version 418090 (0.0007) [2023-12-27 08:31:58,732][20347] Updated weights for policy 0, policy_version 418532 (0.0006) [2023-12-27 08:31:58,794][20347] Updated weights for policy 0, policy_version 418542 (0.0008) [2023-12-27 08:31:58,857][20347] Updated weights for policy 0, policy_version 418552 (0.0008) [2023-12-27 08:31:59,442][20381] Updated weights for policy 1, policy_version 418100 (0.0007) [2023-12-27 08:31:59,496][20381] Updated weights for policy 1, policy_version 418110 (0.0007) [2023-12-27 08:31:59,556][20381] Updated weights for policy 1, policy_version 418120 (0.0008) [2023-12-27 08:31:59,561][20347] Updated weights for policy 0, policy_version 418562 (0.0008) [2023-12-27 08:31:59,617][20347] Updated weights for policy 0, policy_version 418572 (0.0006) [2023-12-27 08:31:59,674][20347] Updated weights for policy 0, policy_version 418582 (0.0009) [2023-12-27 08:31:59,729][20347] Updated weights for policy 0, policy_version 418592 (0.0006) [2023-12-27 08:32:00,256][20381] Updated weights for policy 1, policy_version 418130 (0.0007) [2023-12-27 08:32:00,313][20381] Updated weights for policy 1, policy_version 418140 (0.0007) [2023-12-27 08:32:00,371][20381] Updated weights for policy 1, policy_version 418150 (0.0007) [2023-12-27 08:32:00,403][20347] Updated weights for policy 0, policy_version 418602 (0.0010) [2023-12-27 08:32:00,423][20381] Updated weights for policy 1, policy_version 418160 (0.0006) [2023-12-27 08:32:00,455][20347] Updated weights for policy 0, policy_version 418612 (0.0010) [2023-12-27 08:32:00,505][20347] Updated weights for policy 0, policy_version 418622 (0.0010) [2023-12-27 08:32:01,039][20381] Updated weights for policy 1, policy_version 418170 (0.0006) [2023-12-27 08:32:01,099][20381] Updated weights for policy 1, policy_version 418180 (0.0007) [2023-12-27 08:32:01,156][20347] Updated weights for policy 0, policy_version 418632 (0.0008) [2023-12-27 08:32:01,166][20381] Updated weights for policy 1, policy_version 418190 (0.0007) [2023-12-27 08:32:01,212][20347] Updated weights for policy 0, policy_version 418642 (0.0010) [2023-12-27 08:32:01,268][20347] Updated weights for policy 0, policy_version 418652 (0.0010) [2023-12-27 08:32:01,863][20381] Updated weights for policy 1, policy_version 418200 (0.0009) [2023-12-27 08:32:01,909][20381] Updated weights for policy 1, policy_version 418210 (0.0006) [2023-12-27 08:32:01,936][20347] Updated weights for policy 0, policy_version 418662 (0.0007) [2023-12-27 08:32:01,958][20381] Updated weights for policy 1, policy_version 418220 (0.0008) [2023-12-27 08:32:01,993][20347] Updated weights for policy 0, policy_version 418672 (0.0009) [2023-12-27 08:32:02,026][20291] KL-divergence is very high: 424.4830 [2023-12-27 08:32:02,049][20347] Updated weights for policy 0, policy_version 418682 (0.0010) [2023-12-27 08:32:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 214286336. Throughput: 0: 10338.4, 1: 10413.1. Samples: 214295400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:02,363][19170] Avg episode reward: [(0, '972.560'), (1, '947.390')] [2023-12-27 08:32:02,695][20381] Updated weights for policy 1, policy_version 418230 (0.0007) [2023-12-27 08:32:02,746][20381] Updated weights for policy 1, policy_version 418240 (0.0006) [2023-12-27 08:32:02,782][20347] Updated weights for policy 0, policy_version 418692 (0.0008) [2023-12-27 08:32:02,809][20381] Updated weights for policy 1, policy_version 418250 (0.0008) [2023-12-27 08:32:02,831][20347] Updated weights for policy 0, policy_version 418702 (0.0007) [2023-12-27 08:32:02,884][20347] Updated weights for policy 0, policy_version 418712 (0.0007) [2023-12-27 08:32:03,483][20381] Updated weights for policy 1, policy_version 418260 (0.0007) [2023-12-27 08:32:03,537][20381] Updated weights for policy 1, policy_version 418270 (0.0008) [2023-12-27 08:32:03,544][20347] Updated weights for policy 0, policy_version 418722 (0.0008) [2023-12-27 08:32:03,587][20381] Updated weights for policy 1, policy_version 418280 (0.0006) [2023-12-27 08:32:03,593][20347] Updated weights for policy 0, policy_version 418732 (0.0006) [2023-12-27 08:32:03,637][20347] Updated weights for policy 0, policy_version 418742 (0.0006) [2023-12-27 08:32:03,688][20347] Updated weights for policy 0, policy_version 418752 (0.0007) [2023-12-27 08:32:04,234][20381] Updated weights for policy 1, policy_version 418290 (0.0007) [2023-12-27 08:32:04,293][20381] Updated weights for policy 1, policy_version 418300 (0.0006) [2023-12-27 08:32:04,349][20381] Updated weights for policy 1, policy_version 418310 (0.0008) [2023-12-27 08:32:04,403][20381] Updated weights for policy 1, policy_version 418320 (0.0006) [2023-12-27 08:32:04,416][20347] Updated weights for policy 0, policy_version 418762 (0.0008) [2023-12-27 08:32:04,474][20347] Updated weights for policy 0, policy_version 418772 (0.0008) [2023-12-27 08:32:04,525][20347] Updated weights for policy 0, policy_version 418782 (0.0007) [2023-12-27 08:32:05,014][20381] Updated weights for policy 1, policy_version 418330 (0.0007) [2023-12-27 08:32:05,062][20381] Updated weights for policy 1, policy_version 418340 (0.0008) [2023-12-27 08:32:05,111][20381] Updated weights for policy 1, policy_version 418350 (0.0005) [2023-12-27 08:32:05,162][20347] Updated weights for policy 0, policy_version 418792 (0.0006) [2023-12-27 08:32:05,211][20347] Updated weights for policy 0, policy_version 418802 (0.0005) [2023-12-27 08:32:05,265][20347] Updated weights for policy 0, policy_version 418812 (0.0005) [2023-12-27 08:32:05,674][20381] Updated weights for policy 1, policy_version 418360 (0.0005) [2023-12-27 08:32:05,733][20381] Updated weights for policy 1, policy_version 418370 (0.0005) [2023-12-27 08:32:05,789][20381] Updated weights for policy 1, policy_version 418380 (0.0005) [2023-12-27 08:32:05,887][20347] Updated weights for policy 0, policy_version 418822 (0.0007) [2023-12-27 08:32:05,935][20347] Updated weights for policy 0, policy_version 418832 (0.0010) [2023-12-27 08:32:05,980][20347] Updated weights for policy 0, policy_version 418842 (0.0010) [2023-12-27 08:32:06,351][20381] Updated weights for policy 1, policy_version 418390 (0.0006) [2023-12-27 08:32:06,404][20381] Updated weights for policy 1, policy_version 418400 (0.0008) [2023-12-27 08:32:06,460][20381] Updated weights for policy 1, policy_version 418410 (0.0007) [2023-12-27 08:32:06,653][20347] Updated weights for policy 0, policy_version 418852 (0.0010) [2023-12-27 08:32:06,705][20347] Updated weights for policy 0, policy_version 418862 (0.0010) [2023-12-27 08:32:06,760][20347] Updated weights for policy 0, policy_version 418872 (0.0007) [2023-12-27 08:32:07,195][20381] Updated weights for policy 1, policy_version 418420 (0.0007) [2023-12-27 08:32:07,260][20381] Updated weights for policy 1, policy_version 418430 (0.0008) [2023-12-27 08:32:07,315][20381] Updated weights for policy 1, policy_version 418440 (0.0011) [2023-12-27 08:32:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.2, 300 sec: 20799.4). Total num frames: 214392832. Throughput: 0: 10335.6, 1: 10427.7. Samples: 214357836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:07,363][19170] Avg episode reward: [(0, '972.610'), (1, '852.940')] [2023-12-27 08:32:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000418448_107143168.pth... [2023-12-27 08:32:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000418880_107249664.pth... [2023-12-27 08:32:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000417232_106831872.pth [2023-12-27 08:32:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000417696_106946560.pth [2023-12-27 08:32:07,397][20347] Updated weights for policy 0, policy_version 418882 (0.0009) [2023-12-27 08:32:07,445][20347] Updated weights for policy 0, policy_version 418892 (0.0010) [2023-12-27 08:32:07,487][20347] Updated weights for policy 0, policy_version 418902 (0.0010) [2023-12-27 08:32:07,550][20347] Updated weights for policy 0, policy_version 418912 (0.0010) [2023-12-27 08:32:07,946][20381] Updated weights for policy 1, policy_version 418450 (0.0011) [2023-12-27 08:32:07,992][20381] Updated weights for policy 1, policy_version 418460 (0.0010) [2023-12-27 08:32:08,041][20381] Updated weights for policy 1, policy_version 418470 (0.0007) [2023-12-27 08:32:08,092][20381] Updated weights for policy 1, policy_version 418480 (0.0005) [2023-12-27 08:32:08,239][20347] Updated weights for policy 0, policy_version 418922 (0.0010) [2023-12-27 08:32:08,287][20347] Updated weights for policy 0, policy_version 418932 (0.0010) [2023-12-27 08:32:08,343][20347] Updated weights for policy 0, policy_version 418942 (0.0010) [2023-12-27 08:32:08,769][20381] Updated weights for policy 1, policy_version 418490 (0.0009) [2023-12-27 08:32:08,808][20292] KL-divergence is very high: 211.7474 [2023-12-27 08:32:08,814][20292] KL-divergence is very high: 203.5084 [2023-12-27 08:32:08,826][20381] Updated weights for policy 1, policy_version 418500 (0.0011) [2023-12-27 08:32:08,878][20381] Updated weights for policy 1, policy_version 418510 (0.0011) [2023-12-27 08:32:09,032][20347] Updated weights for policy 0, policy_version 418952 (0.0010) [2023-12-27 08:32:09,090][20347] Updated weights for policy 0, policy_version 418962 (0.0008) [2023-12-27 08:32:09,149][20347] Updated weights for policy 0, policy_version 418972 (0.0005) [2023-12-27 08:32:09,579][20381] Updated weights for policy 1, policy_version 418520 (0.0009) [2023-12-27 08:32:09,635][20381] Updated weights for policy 1, policy_version 418530 (0.0008) [2023-12-27 08:32:09,694][20381] Updated weights for policy 1, policy_version 418540 (0.0008) [2023-12-27 08:32:09,807][20347] Updated weights for policy 0, policy_version 418982 (0.0007) [2023-12-27 08:32:09,855][20291] KL-divergence is very high: 154.1309 [2023-12-27 08:32:09,873][20347] Updated weights for policy 0, policy_version 418992 (0.0009) [2023-12-27 08:32:09,940][20347] Updated weights for policy 0, policy_version 419002 (0.0008) [2023-12-27 08:32:10,439][20381] Updated weights for policy 1, policy_version 418550 (0.0011) [2023-12-27 08:32:10,499][20381] Updated weights for policy 1, policy_version 418560 (0.0011) [2023-12-27 08:32:10,558][20381] Updated weights for policy 1, policy_version 418570 (0.0011) [2023-12-27 08:32:10,579][20347] Updated weights for policy 0, policy_version 419012 (0.0007) [2023-12-27 08:32:10,640][20347] Updated weights for policy 0, policy_version 419022 (0.0007) [2023-12-27 08:32:10,701][20347] Updated weights for policy 0, policy_version 419032 (0.0007) [2023-12-27 08:32:11,229][20381] Updated weights for policy 1, policy_version 418580 (0.0009) [2023-12-27 08:32:11,285][20381] Updated weights for policy 1, policy_version 418590 (0.0008) [2023-12-27 08:32:11,334][20381] Updated weights for policy 1, policy_version 418600 (0.0008) [2023-12-27 08:32:11,430][20347] Updated weights for policy 0, policy_version 419042 (0.0007) [2023-12-27 08:32:11,486][20347] Updated weights for policy 0, policy_version 419052 (0.0005) [2023-12-27 08:32:11,547][20347] Updated weights for policy 0, policy_version 419062 (0.0010) [2023-12-27 08:32:11,610][20347] Updated weights for policy 0, policy_version 419072 (0.0010) [2023-12-27 08:32:12,090][20381] Updated weights for policy 1, policy_version 418610 (0.0008) [2023-12-27 08:32:12,153][20381] Updated weights for policy 1, policy_version 418620 (0.0007) [2023-12-27 08:32:12,215][20381] Updated weights for policy 1, policy_version 418630 (0.0008) [2023-12-27 08:32:12,278][20381] Updated weights for policy 1, policy_version 418640 (0.0007) [2023-12-27 08:32:12,314][20347] Updated weights for policy 0, policy_version 419082 (0.0007) [2023-12-27 08:32:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 214491136. Throughput: 0: 10310.9, 1: 10442.0. Samples: 214483588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:12,363][19170] Avg episode reward: [(0, '961.850'), (1, '829.250')] [2023-12-27 08:32:12,376][20347] Updated weights for policy 0, policy_version 419092 (0.0007) [2023-12-27 08:32:12,429][20347] Updated weights for policy 0, policy_version 419102 (0.0009) [2023-12-27 08:32:12,912][20292] KL-divergence is very high: 104.5527 [2023-12-27 08:32:12,917][20381] Updated weights for policy 1, policy_version 418650 (0.0009) [2023-12-27 08:32:12,977][20381] Updated weights for policy 1, policy_version 418660 (0.0008) [2023-12-27 08:32:13,031][20381] Updated weights for policy 1, policy_version 418670 (0.0009) [2023-12-27 08:32:13,069][20347] Updated weights for policy 0, policy_version 419112 (0.0006) [2023-12-27 08:32:13,115][20347] Updated weights for policy 0, policy_version 419122 (0.0005) [2023-12-27 08:32:13,158][20347] Updated weights for policy 0, policy_version 419132 (0.0005) [2023-12-27 08:32:13,696][20381] Updated weights for policy 1, policy_version 418680 (0.0010) [2023-12-27 08:32:13,747][20381] Updated weights for policy 1, policy_version 418690 (0.0010) [2023-12-27 08:32:13,766][20347] Updated weights for policy 0, policy_version 419142 (0.0008) [2023-12-27 08:32:13,801][20381] Updated weights for policy 1, policy_version 418700 (0.0010) [2023-12-27 08:32:13,820][20347] Updated weights for policy 0, policy_version 419152 (0.0010) [2023-12-27 08:32:13,875][20347] Updated weights for policy 0, policy_version 419162 (0.0010) [2023-12-27 08:32:14,455][20347] Updated weights for policy 0, policy_version 419172 (0.0010) [2023-12-27 08:32:14,459][20381] Updated weights for policy 1, policy_version 418710 (0.0010) [2023-12-27 08:32:14,505][20381] Updated weights for policy 1, policy_version 418720 (0.0010) [2023-12-27 08:32:14,515][20347] Updated weights for policy 0, policy_version 419182 (0.0010) [2023-12-27 08:32:14,554][20381] Updated weights for policy 1, policy_version 418730 (0.0010) [2023-12-27 08:32:14,573][20347] Updated weights for policy 0, policy_version 419192 (0.0005) [2023-12-27 08:32:15,210][20381] Updated weights for policy 1, policy_version 418740 (0.0008) [2023-12-27 08:32:15,239][20347] Updated weights for policy 0, policy_version 419202 (0.0006) [2023-12-27 08:32:15,269][20381] Updated weights for policy 1, policy_version 418750 (0.0007) [2023-12-27 08:32:15,288][20347] Updated weights for policy 0, policy_version 419212 (0.0008) [2023-12-27 08:32:15,334][20381] Updated weights for policy 1, policy_version 418760 (0.0008) [2023-12-27 08:32:15,339][20347] Updated weights for policy 0, policy_version 419222 (0.0005) [2023-12-27 08:32:15,389][20347] Updated weights for policy 0, policy_version 419232 (0.0006) [2023-12-27 08:32:16,001][20381] Updated weights for policy 1, policy_version 418770 (0.0008) [2023-12-27 08:32:16,052][20381] Updated weights for policy 1, policy_version 418780 (0.0007) [2023-12-27 08:32:16,093][20347] Updated weights for policy 0, policy_version 419242 (0.0005) [2023-12-27 08:32:16,109][20381] Updated weights for policy 1, policy_version 418790 (0.0007) [2023-12-27 08:32:16,155][20347] Updated weights for policy 0, policy_version 419252 (0.0005) [2023-12-27 08:32:16,173][20381] Updated weights for policy 1, policy_version 418800 (0.0007) [2023-12-27 08:32:16,207][20347] Updated weights for policy 0, policy_version 419262 (0.0008) [2023-12-27 08:32:16,758][20381] Updated weights for policy 1, policy_version 418810 (0.0007) [2023-12-27 08:32:16,812][20381] Updated weights for policy 1, policy_version 418820 (0.0008) [2023-12-27 08:32:16,856][20381] Updated weights for policy 1, policy_version 418830 (0.0007) [2023-12-27 08:32:16,915][20347] Updated weights for policy 0, policy_version 419272 (0.0006) [2023-12-27 08:32:16,974][20347] Updated weights for policy 0, policy_version 419282 (0.0007) [2023-12-27 08:32:17,039][20347] Updated weights for policy 0, policy_version 419292 (0.0006) [2023-12-27 08:32:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 214597632. Throughput: 0: 10296.0, 1: 10413.5. Samples: 214608584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:17,363][19170] Avg episode reward: [(0, '932.960'), (1, '844.030')] [2023-12-27 08:32:17,580][20381] Updated weights for policy 1, policy_version 418840 (0.0008) [2023-12-27 08:32:17,644][20347] Updated weights for policy 0, policy_version 419302 (0.0009) [2023-12-27 08:32:17,644][20381] Updated weights for policy 1, policy_version 418850 (0.0009) [2023-12-27 08:32:17,699][20381] Updated weights for policy 1, policy_version 418860 (0.0009) [2023-12-27 08:32:17,702][20347] Updated weights for policy 0, policy_version 419312 (0.0010) [2023-12-27 08:32:17,768][20347] Updated weights for policy 0, policy_version 419322 (0.0010) [2023-12-27 08:32:18,313][20381] Updated weights for policy 1, policy_version 418870 (0.0007) [2023-12-27 08:32:18,353][20347] Updated weights for policy 0, policy_version 419332 (0.0010) [2023-12-27 08:32:18,373][20381] Updated weights for policy 1, policy_version 418880 (0.0008) [2023-12-27 08:32:18,413][20347] Updated weights for policy 0, policy_version 419342 (0.0008) [2023-12-27 08:32:18,433][20381] Updated weights for policy 1, policy_version 418890 (0.0005) [2023-12-27 08:32:18,472][20347] Updated weights for policy 0, policy_version 419352 (0.0007) [2023-12-27 08:32:18,965][20381] Updated weights for policy 1, policy_version 418900 (0.0006) [2023-12-27 08:32:19,026][20381] Updated weights for policy 1, policy_version 418910 (0.0008) [2023-12-27 08:32:19,060][20347] Updated weights for policy 0, policy_version 419362 (0.0006) [2023-12-27 08:32:19,078][20381] Updated weights for policy 1, policy_version 418920 (0.0009) [2023-12-27 08:32:19,115][20347] Updated weights for policy 0, policy_version 419372 (0.0010) [2023-12-27 08:32:19,171][20347] Updated weights for policy 0, policy_version 419382 (0.0011) [2023-12-27 08:32:19,233][20347] Updated weights for policy 0, policy_version 419392 (0.0010) [2023-12-27 08:32:19,705][20381] Updated weights for policy 1, policy_version 418930 (0.0008) [2023-12-27 08:32:19,764][20381] Updated weights for policy 1, policy_version 418940 (0.0007) [2023-12-27 08:32:19,830][20381] Updated weights for policy 1, policy_version 418950 (0.0008) [2023-12-27 08:32:19,895][20381] Updated weights for policy 1, policy_version 418960 (0.0008) [2023-12-27 08:32:19,966][20347] Updated weights for policy 0, policy_version 419402 (0.0011) [2023-12-27 08:32:20,012][20347] Updated weights for policy 0, policy_version 419412 (0.0010) [2023-12-27 08:32:20,067][20347] Updated weights for policy 0, policy_version 419422 (0.0010) [2023-12-27 08:32:20,610][20381] Updated weights for policy 1, policy_version 418970 (0.0008) [2023-12-27 08:32:20,666][20381] Updated weights for policy 1, policy_version 418980 (0.0007) [2023-12-27 08:32:20,722][20381] Updated weights for policy 1, policy_version 418990 (0.0006) [2023-12-27 08:32:20,761][20347] Updated weights for policy 0, policy_version 419432 (0.0009) [2023-12-27 08:32:20,819][20347] Updated weights for policy 0, policy_version 419442 (0.0008) [2023-12-27 08:32:20,880][20347] Updated weights for policy 0, policy_version 419452 (0.0008) [2023-12-27 08:32:21,443][20381] Updated weights for policy 1, policy_version 419000 (0.0007) [2023-12-27 08:32:21,496][20381] Updated weights for policy 1, policy_version 419010 (0.0008) [2023-12-27 08:32:21,560][20381] Updated weights for policy 1, policy_version 419020 (0.0007) [2023-12-27 08:32:21,601][20347] Updated weights for policy 0, policy_version 419462 (0.0009) [2023-12-27 08:32:21,664][20347] Updated weights for policy 0, policy_version 419472 (0.0011) [2023-12-27 08:32:21,726][20347] Updated weights for policy 0, policy_version 419482 (0.0009) [2023-12-27 08:32:22,216][20381] Updated weights for policy 1, policy_version 419030 (0.0006) [2023-12-27 08:32:22,278][20381] Updated weights for policy 1, policy_version 419040 (0.0007) [2023-12-27 08:32:22,340][20381] Updated weights for policy 1, policy_version 419050 (0.0006) [2023-12-27 08:32:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 214695936. Throughput: 0: 10314.0, 1: 10431.6. Samples: 214673296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:22,363][19170] Avg episode reward: [(0, '913.890'), (1, '827.100')] [2023-12-27 08:32:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000419488_107405312.pth... [2023-12-27 08:32:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000418304_107102208.pth [2023-12-27 08:32:22,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000419056_107298816.pth... [2023-12-27 08:32:22,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000417840_106987520.pth [2023-12-27 08:32:22,457][20347] Updated weights for policy 0, policy_version 419492 (0.0009) [2023-12-27 08:32:22,511][20347] Updated weights for policy 0, policy_version 419502 (0.0007) [2023-12-27 08:32:22,567][20347] Updated weights for policy 0, policy_version 419512 (0.0009) [2023-12-27 08:32:22,957][20381] Updated weights for policy 1, policy_version 419060 (0.0008) [2023-12-27 08:32:23,014][20381] Updated weights for policy 1, policy_version 419070 (0.0008) [2023-12-27 08:32:23,073][20381] Updated weights for policy 1, policy_version 419080 (0.0007) [2023-12-27 08:32:23,319][20347] Updated weights for policy 0, policy_version 419522 (0.0007) [2023-12-27 08:32:23,375][20347] Updated weights for policy 0, policy_version 419532 (0.0008) [2023-12-27 08:32:23,432][20347] Updated weights for policy 0, policy_version 419542 (0.0007) [2023-12-27 08:32:23,486][20347] Updated weights for policy 0, policy_version 419552 (0.0008) [2023-12-27 08:32:23,776][20381] Updated weights for policy 1, policy_version 419090 (0.0007) [2023-12-27 08:32:23,828][20381] Updated weights for policy 1, policy_version 419100 (0.0008) [2023-12-27 08:32:23,876][20381] Updated weights for policy 1, policy_version 419110 (0.0007) [2023-12-27 08:32:23,925][20381] Updated weights for policy 1, policy_version 419120 (0.0005) [2023-12-27 08:32:24,204][20347] Updated weights for policy 0, policy_version 419562 (0.0009) [2023-12-27 08:32:24,252][20347] Updated weights for policy 0, policy_version 419572 (0.0008) [2023-12-27 08:32:24,299][20347] Updated weights for policy 0, policy_version 419582 (0.0007) [2023-12-27 08:32:24,516][20381] Updated weights for policy 1, policy_version 419130 (0.0005) [2023-12-27 08:32:24,575][20381] Updated weights for policy 1, policy_version 419140 (0.0005) [2023-12-27 08:32:24,625][20381] Updated weights for policy 1, policy_version 419150 (0.0007) [2023-12-27 08:32:25,021][20347] Updated weights for policy 0, policy_version 419592 (0.0007) [2023-12-27 08:32:25,076][20347] Updated weights for policy 0, policy_version 419602 (0.0007) [2023-12-27 08:32:25,131][20347] Updated weights for policy 0, policy_version 419612 (0.0007) [2023-12-27 08:32:25,282][20381] Updated weights for policy 1, policy_version 419160 (0.0007) [2023-12-27 08:32:25,330][20381] Updated weights for policy 1, policy_version 419170 (0.0008) [2023-12-27 08:32:25,395][20381] Updated weights for policy 1, policy_version 419180 (0.0007) [2023-12-27 08:32:25,833][20347] Updated weights for policy 0, policy_version 419622 (0.0007) [2023-12-27 08:32:25,887][20347] Updated weights for policy 0, policy_version 419632 (0.0008) [2023-12-27 08:32:25,940][20347] Updated weights for policy 0, policy_version 419642 (0.0007) [2023-12-27 08:32:26,066][20381] Updated weights for policy 1, policy_version 419190 (0.0007) [2023-12-27 08:32:26,115][20381] Updated weights for policy 1, policy_version 419200 (0.0008) [2023-12-27 08:32:26,171][20381] Updated weights for policy 1, policy_version 419210 (0.0007) [2023-12-27 08:32:26,616][20347] Updated weights for policy 0, policy_version 419652 (0.0008) [2023-12-27 08:32:26,665][20347] Updated weights for policy 0, policy_version 419662 (0.0009) [2023-12-27 08:32:26,714][20347] Updated weights for policy 0, policy_version 419672 (0.0008) [2023-12-27 08:32:26,791][20381] Updated weights for policy 1, policy_version 419220 (0.0007) [2023-12-27 08:32:26,848][20381] Updated weights for policy 1, policy_version 419230 (0.0006) [2023-12-27 08:32:26,900][20381] Updated weights for policy 1, policy_version 419240 (0.0007) [2023-12-27 08:32:27,356][20347] Updated weights for policy 0, policy_version 419682 (0.0009) [2023-12-27 08:32:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 214802432. Throughput: 0: 10283.4, 1: 10444.6. Samples: 214794116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:27,363][19170] Avg episode reward: [(0, '856.460'), (1, '535.910')] [2023-12-27 08:32:27,401][20347] Updated weights for policy 0, policy_version 419692 (0.0008) [2023-12-27 08:32:27,447][20347] Updated weights for policy 0, policy_version 419702 (0.0008) [2023-12-27 08:32:27,492][20347] Updated weights for policy 0, policy_version 419712 (0.0007) [2023-12-27 08:32:27,573][20381] Updated weights for policy 1, policy_version 419250 (0.0007) [2023-12-27 08:32:27,623][20381] Updated weights for policy 1, policy_version 419260 (0.0007) [2023-12-27 08:32:27,672][20381] Updated weights for policy 1, policy_version 419270 (0.0009) [2023-12-27 08:32:27,724][20381] Updated weights for policy 1, policy_version 419280 (0.0010) [2023-12-27 08:32:28,254][20347] Updated weights for policy 0, policy_version 419722 (0.0007) [2023-12-27 08:32:28,307][20347] Updated weights for policy 0, policy_version 419732 (0.0007) [2023-12-27 08:32:28,375][20347] Updated weights for policy 0, policy_version 419742 (0.0006) [2023-12-27 08:32:28,391][20381] Updated weights for policy 1, policy_version 419290 (0.0011) [2023-12-27 08:32:28,440][20381] Updated weights for policy 1, policy_version 419300 (0.0010) [2023-12-27 08:32:28,496][20381] Updated weights for policy 1, policy_version 419310 (0.0007) [2023-12-27 08:32:29,065][20347] Updated weights for policy 0, policy_version 419752 (0.0005) [2023-12-27 08:32:29,118][20347] Updated weights for policy 0, policy_version 419762 (0.0005) [2023-12-27 08:32:29,169][20347] Updated weights for policy 0, policy_version 419772 (0.0005) [2023-12-27 08:32:29,191][20381] Updated weights for policy 1, policy_version 419320 (0.0008) [2023-12-27 08:32:29,250][20381] Updated weights for policy 1, policy_version 419330 (0.0008) [2023-12-27 08:32:29,311][20381] Updated weights for policy 1, policy_version 419340 (0.0008) [2023-12-27 08:32:29,821][20347] Updated weights for policy 0, policy_version 419782 (0.0007) [2023-12-27 08:32:29,879][20347] Updated weights for policy 0, policy_version 419792 (0.0007) [2023-12-27 08:32:29,940][20347] Updated weights for policy 0, policy_version 419802 (0.0010) [2023-12-27 08:32:29,998][20381] Updated weights for policy 1, policy_version 419350 (0.0007) [2023-12-27 08:32:30,059][20381] Updated weights for policy 1, policy_version 419360 (0.0010) [2023-12-27 08:32:30,116][20381] Updated weights for policy 1, policy_version 419370 (0.0010) [2023-12-27 08:32:30,613][20347] Updated weights for policy 0, policy_version 419812 (0.0010) [2023-12-27 08:32:30,671][20347] Updated weights for policy 0, policy_version 419822 (0.0010) [2023-12-27 08:32:30,719][20347] Updated weights for policy 0, policy_version 419832 (0.0010) [2023-12-27 08:32:30,720][20381] Updated weights for policy 1, policy_version 419380 (0.0010) [2023-12-27 08:32:30,774][20381] Updated weights for policy 1, policy_version 419390 (0.0010) [2023-12-27 08:32:30,827][20381] Updated weights for policy 1, policy_version 419400 (0.0010) [2023-12-27 08:32:31,314][20347] Updated weights for policy 0, policy_version 419842 (0.0008) [2023-12-27 08:32:31,324][20291] KL-divergence is very high: 138.3175 [2023-12-27 08:32:31,375][20291] KL-divergence is very high: 548.9588 [2023-12-27 08:32:31,378][20347] Updated weights for policy 0, policy_version 419852 (0.0009) [2023-12-27 08:32:31,431][20347] Updated weights for policy 0, policy_version 419862 (0.0007) [2023-12-27 08:32:31,456][20381] Updated weights for policy 1, policy_version 419410 (0.0010) [2023-12-27 08:32:31,471][20291] KL-divergence is very high: 170.1566 [2023-12-27 08:32:31,496][20347] Updated weights for policy 0, policy_version 419872 (0.0006) [2023-12-27 08:32:31,502][20381] Updated weights for policy 1, policy_version 419420 (0.0006) [2023-12-27 08:32:31,551][20381] Updated weights for policy 1, policy_version 419430 (0.0007) [2023-12-27 08:32:31,608][20381] Updated weights for policy 1, policy_version 419440 (0.0008) [2023-12-27 08:32:32,088][20291] KL-divergence is very high: 252.2020 [2023-12-27 08:32:32,094][20291] KL-divergence is very high: 273.4230 [2023-12-27 08:32:32,105][20291] KL-divergence is very high: 265.9053 [2023-12-27 08:32:32,112][20291] KL-divergence is very high: 136.5342 [2023-12-27 08:32:32,136][20291] KL-divergence is very high: 213.3152 [2023-12-27 08:32:32,141][20347] Updated weights for policy 0, policy_version 419882 (0.0008) [2023-12-27 08:32:32,142][20291] KL-divergence is very high: 146.8381 [2023-12-27 08:32:32,155][20291] KL-divergence is very high: 112.2576 [2023-12-27 08:32:32,184][20291] KL-divergence is very high: 203.7618 [2023-12-27 08:32:32,190][20291] KL-divergence is very high: 237.2284 [2023-12-27 08:32:32,200][20347] Updated weights for policy 0, policy_version 419892 (0.0007) [2023-12-27 08:32:32,202][20291] KL-divergence is very high: 172.4311 [2023-12-27 08:32:32,230][20291] KL-divergence is very high: 307.1802 [2023-12-27 08:32:32,235][20291] KL-divergence is very high: 264.3242 [2023-12-27 08:32:32,246][20291] KL-divergence is very high: 106.1955 [2023-12-27 08:32:32,258][20347] Updated weights for policy 0, policy_version 419902 (0.0007) [2023-12-27 08:32:32,342][20381] Updated weights for policy 1, policy_version 419450 (0.0006) [2023-12-27 08:32:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 214908928. Throughput: 0: 10273.8, 1: 10444.3. Samples: 214920388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:32,363][19170] Avg episode reward: [(0, '737.770'), (1, '563.660')] [2023-12-27 08:32:32,404][20381] Updated weights for policy 1, policy_version 419460 (0.0007) [2023-12-27 08:32:32,464][20381] Updated weights for policy 1, policy_version 419470 (0.0008) [2023-12-27 08:32:32,874][20291] KL-divergence is very high: 198.6368 [2023-12-27 08:32:32,910][20347] Updated weights for policy 0, policy_version 419912 (0.0009) [2023-12-27 08:32:32,933][20291] KL-divergence is very high: 186.0192 [2023-12-27 08:32:32,938][20291] KL-divergence is very high: 198.0759 [2023-12-27 08:32:32,943][20291] KL-divergence is very high: 193.7173 [2023-12-27 08:32:32,958][20347] Updated weights for policy 0, policy_version 419922 (0.0007) [2023-12-27 08:32:33,003][20291] KL-divergence is very high: 191.0962 [2023-12-27 08:32:33,019][20347] Updated weights for policy 0, policy_version 419932 (0.0005) [2023-12-27 08:32:33,024][20291] KL-divergence is very high: 136.1443 [2023-12-27 08:32:33,046][20381] Updated weights for policy 1, policy_version 419480 (0.0006) [2023-12-27 08:32:33,108][20381] Updated weights for policy 1, policy_version 419490 (0.0005) [2023-12-27 08:32:33,157][20381] Updated weights for policy 1, policy_version 419500 (0.0007) [2023-12-27 08:32:33,656][20347] Updated weights for policy 0, policy_version 419942 (0.0005) [2023-12-27 08:32:33,718][20347] Updated weights for policy 0, policy_version 419952 (0.0005) [2023-12-27 08:32:33,753][20381] Updated weights for policy 1, policy_version 419510 (0.0005) [2023-12-27 08:32:33,785][20347] Updated weights for policy 0, policy_version 419962 (0.0005) [2023-12-27 08:32:33,799][20381] Updated weights for policy 1, policy_version 419520 (0.0005) [2023-12-27 08:32:33,845][20381] Updated weights for policy 1, policy_version 419530 (0.0005) [2023-12-27 08:32:34,355][20347] Updated weights for policy 0, policy_version 419972 (0.0005) [2023-12-27 08:32:34,415][20347] Updated weights for policy 0, policy_version 419982 (0.0008) [2023-12-27 08:32:34,471][20347] Updated weights for policy 0, policy_version 419992 (0.0010) [2023-12-27 08:32:34,490][20381] Updated weights for policy 1, policy_version 419540 (0.0005) [2023-12-27 08:32:34,550][20381] Updated weights for policy 1, policy_version 419550 (0.0006) [2023-12-27 08:32:34,611][20381] Updated weights for policy 1, policy_version 419560 (0.0008) [2023-12-27 08:32:35,129][20347] Updated weights for policy 0, policy_version 420002 (0.0009) [2023-12-27 08:32:35,179][20347] Updated weights for policy 0, policy_version 420012 (0.0006) [2023-12-27 08:32:35,191][20291] KL-divergence is very high: 214.9674 [2023-12-27 08:32:35,213][20291] KL-divergence is very high: 191.4628 [2023-12-27 08:32:35,216][20381] Updated weights for policy 1, policy_version 419570 (0.0007) [2023-12-27 08:32:35,233][20347] Updated weights for policy 0, policy_version 420022 (0.0005) [2023-12-27 08:32:35,234][20291] KL-divergence is very high: 173.7592 [2023-12-27 08:32:35,265][20381] Updated weights for policy 1, policy_version 419580 (0.0008) [2023-12-27 08:32:35,285][20347] Updated weights for policy 0, policy_version 420032 (0.0005) [2023-12-27 08:32:35,317][20381] Updated weights for policy 1, policy_version 419590 (0.0008) [2023-12-27 08:32:35,364][20381] Updated weights for policy 1, policy_version 419600 (0.0009) [2023-12-27 08:32:35,907][20347] Updated weights for policy 0, policy_version 420042 (0.0007) [2023-12-27 08:32:35,955][20347] Updated weights for policy 0, policy_version 420052 (0.0008) [2023-12-27 08:32:36,005][20381] Updated weights for policy 1, policy_version 419610 (0.0005) [2023-12-27 08:32:36,007][20347] Updated weights for policy 0, policy_version 420062 (0.0008) [2023-12-27 08:32:36,058][20381] Updated weights for policy 1, policy_version 419620 (0.0005) [2023-12-27 08:32:36,116][20381] Updated weights for policy 1, policy_version 419630 (0.0006) [2023-12-27 08:32:36,738][20347] Updated weights for policy 0, policy_version 420072 (0.0008) [2023-12-27 08:32:36,752][20381] Updated weights for policy 1, policy_version 419640 (0.0007) [2023-12-27 08:32:36,800][20347] Updated weights for policy 0, policy_version 420082 (0.0006) [2023-12-27 08:32:36,812][20381] Updated weights for policy 1, policy_version 419650 (0.0007) [2023-12-27 08:32:36,866][20347] Updated weights for policy 0, policy_version 420092 (0.0006) [2023-12-27 08:32:36,879][20381] Updated weights for policy 1, policy_version 419660 (0.0006) [2023-12-27 08:32:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 215015424. Throughput: 0: 10301.9, 1: 10498.3. Samples: 214986264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:37,363][19170] Avg episode reward: [(0, '744.550'), (1, '734.750')] [2023-12-27 08:32:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000419664_107454464.pth... [2023-12-27 08:32:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000420096_107560960.pth... [2023-12-27 08:32:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000418448_107143168.pth [2023-12-27 08:32:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000418880_107249664.pth [2023-12-27 08:32:37,480][20347] Updated weights for policy 0, policy_version 420102 (0.0006) [2023-12-27 08:32:37,527][20347] Updated weights for policy 0, policy_version 420112 (0.0005) [2023-12-27 08:32:37,581][20347] Updated weights for policy 0, policy_version 420122 (0.0005) [2023-12-27 08:32:37,613][20381] Updated weights for policy 1, policy_version 419670 (0.0008) [2023-12-27 08:32:37,669][20381] Updated weights for policy 1, policy_version 419680 (0.0009) [2023-12-27 08:32:37,718][20381] Updated weights for policy 1, policy_version 419690 (0.0009) [2023-12-27 08:32:38,171][20347] Updated weights for policy 0, policy_version 420132 (0.0006) [2023-12-27 08:32:38,230][20347] Updated weights for policy 0, policy_version 420142 (0.0008) [2023-12-27 08:32:38,288][20347] Updated weights for policy 0, policy_version 420152 (0.0007) [2023-12-27 08:32:38,440][20381] Updated weights for policy 1, policy_version 419700 (0.0007) [2023-12-27 08:32:38,501][20381] Updated weights for policy 1, policy_version 419710 (0.0008) [2023-12-27 08:32:38,564][20381] Updated weights for policy 1, policy_version 419720 (0.0007) [2023-12-27 08:32:38,908][20347] Updated weights for policy 0, policy_version 420162 (0.0008) [2023-12-27 08:32:38,964][20347] Updated weights for policy 0, policy_version 420172 (0.0008) [2023-12-27 08:32:39,022][20347] Updated weights for policy 0, policy_version 420182 (0.0007) [2023-12-27 08:32:39,072][20347] Updated weights for policy 0, policy_version 420192 (0.0007) [2023-12-27 08:32:39,252][20381] Updated weights for policy 1, policy_version 419730 (0.0008) [2023-12-27 08:32:39,306][20381] Updated weights for policy 1, policy_version 419740 (0.0008) [2023-12-27 08:32:39,384][20381] Updated weights for policy 1, policy_version 419750 (0.0008) [2023-12-27 08:32:39,450][20381] Updated weights for policy 1, policy_version 419760 (0.0007) [2023-12-27 08:32:39,784][20347] Updated weights for policy 0, policy_version 420202 (0.0008) [2023-12-27 08:32:39,844][20347] Updated weights for policy 0, policy_version 420212 (0.0008) [2023-12-27 08:32:39,895][20347] Updated weights for policy 0, policy_version 420222 (0.0007) [2023-12-27 08:32:40,100][20381] Updated weights for policy 1, policy_version 419770 (0.0008) [2023-12-27 08:32:40,149][20381] Updated weights for policy 1, policy_version 419780 (0.0008) [2023-12-27 08:32:40,205][20381] Updated weights for policy 1, policy_version 419790 (0.0008) [2023-12-27 08:32:40,583][20347] Updated weights for policy 0, policy_version 420232 (0.0007) [2023-12-27 08:32:40,650][20347] Updated weights for policy 0, policy_version 420242 (0.0007) [2023-12-27 08:32:40,718][20347] Updated weights for policy 0, policy_version 420252 (0.0008) [2023-12-27 08:32:40,919][20381] Updated weights for policy 1, policy_version 419800 (0.0007) [2023-12-27 08:32:40,980][20381] Updated weights for policy 1, policy_version 419810 (0.0008) [2023-12-27 08:32:41,046][20381] Updated weights for policy 1, policy_version 419820 (0.0007) [2023-12-27 08:32:41,430][20347] Updated weights for policy 0, policy_version 420262 (0.0008) [2023-12-27 08:32:41,479][20347] Updated weights for policy 0, policy_version 420272 (0.0008) [2023-12-27 08:32:41,526][20347] Updated weights for policy 0, policy_version 420282 (0.0008) [2023-12-27 08:32:41,767][20381] Updated weights for policy 1, policy_version 419830 (0.0010) [2023-12-27 08:32:41,820][20381] Updated weights for policy 1, policy_version 419840 (0.0011) [2023-12-27 08:32:41,869][20381] Updated weights for policy 1, policy_version 419850 (0.0010) [2023-12-27 08:32:42,292][20347] Updated weights for policy 0, policy_version 420292 (0.0007) [2023-12-27 08:32:42,352][20347] Updated weights for policy 0, policy_version 420302 (0.0008) [2023-12-27 08:32:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 215113728. Throughput: 0: 10343.5, 1: 10453.1. Samples: 215109440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:42,363][19170] Avg episode reward: [(0, '834.800'), (1, '824.310')] [2023-12-27 08:32:42,417][20347] Updated weights for policy 0, policy_version 420312 (0.0007) [2023-12-27 08:32:42,589][20381] Updated weights for policy 1, policy_version 419860 (0.0008) [2023-12-27 08:32:42,647][20381] Updated weights for policy 1, policy_version 419870 (0.0008) [2023-12-27 08:32:42,703][20381] Updated weights for policy 1, policy_version 419880 (0.0008) [2023-12-27 08:32:43,072][20347] Updated weights for policy 0, policy_version 420322 (0.0008) [2023-12-27 08:32:43,120][20347] Updated weights for policy 0, policy_version 420332 (0.0007) [2023-12-27 08:32:43,165][20347] Updated weights for policy 0, policy_version 420342 (0.0007) [2023-12-27 08:32:43,207][20347] Updated weights for policy 0, policy_version 420352 (0.0007) [2023-12-27 08:32:43,342][20381] Updated weights for policy 1, policy_version 419890 (0.0008) [2023-12-27 08:32:43,397][20381] Updated weights for policy 1, policy_version 419900 (0.0008) [2023-12-27 08:32:43,453][20381] Updated weights for policy 1, policy_version 419910 (0.0007) [2023-12-27 08:32:43,512][20381] Updated weights for policy 1, policy_version 419920 (0.0005) [2023-12-27 08:32:43,957][20347] Updated weights for policy 0, policy_version 420362 (0.0007) [2023-12-27 08:32:44,014][20347] Updated weights for policy 0, policy_version 420372 (0.0006) [2023-12-27 08:32:44,073][20347] Updated weights for policy 0, policy_version 420382 (0.0008) [2023-12-27 08:32:44,110][20381] Updated weights for policy 1, policy_version 419930 (0.0006) [2023-12-27 08:32:44,158][20381] Updated weights for policy 1, policy_version 419940 (0.0005) [2023-12-27 08:32:44,215][20381] Updated weights for policy 1, policy_version 419950 (0.0006) [2023-12-27 08:32:44,739][20347] Updated weights for policy 0, policy_version 420392 (0.0008) [2023-12-27 08:32:44,801][20347] Updated weights for policy 0, policy_version 420402 (0.0009) [2023-12-27 08:32:44,861][20347] Updated weights for policy 0, policy_version 420412 (0.0008) [2023-12-27 08:32:44,886][20381] Updated weights for policy 1, policy_version 419960 (0.0007) [2023-12-27 08:32:44,939][20381] Updated weights for policy 1, policy_version 419970 (0.0008) [2023-12-27 08:32:45,001][20381] Updated weights for policy 1, policy_version 419980 (0.0007) [2023-12-27 08:32:45,589][20347] Updated weights for policy 0, policy_version 420422 (0.0007) [2023-12-27 08:32:45,638][20347] Updated weights for policy 0, policy_version 420432 (0.0009) [2023-12-27 08:32:45,655][20381] Updated weights for policy 1, policy_version 419990 (0.0006) [2023-12-27 08:32:45,685][20347] Updated weights for policy 0, policy_version 420442 (0.0008) [2023-12-27 08:32:45,701][20381] Updated weights for policy 1, policy_version 420000 (0.0005) [2023-12-27 08:32:45,748][20381] Updated weights for policy 1, policy_version 420010 (0.0005) [2023-12-27 08:32:46,280][20381] Updated weights for policy 1, policy_version 420020 (0.0005) [2023-12-27 08:32:46,337][20381] Updated weights for policy 1, policy_version 420030 (0.0006) [2023-12-27 08:32:46,392][20381] Updated weights for policy 1, policy_version 420040 (0.0007) [2023-12-27 08:32:46,467][20347] Updated weights for policy 0, policy_version 420452 (0.0007) [2023-12-27 08:32:46,516][20347] Updated weights for policy 0, policy_version 420462 (0.0008) [2023-12-27 08:32:46,557][20291] KL-divergence is very high: 174.5866 [2023-12-27 08:32:46,566][20291] KL-divergence is very high: 145.2783 [2023-12-27 08:32:46,568][20347] Updated weights for policy 0, policy_version 420472 (0.0007) [2023-12-27 08:32:47,052][20381] Updated weights for policy 1, policy_version 420050 (0.0008) [2023-12-27 08:32:47,101][20381] Updated weights for policy 1, policy_version 420060 (0.0008) [2023-12-27 08:32:47,147][20381] Updated weights for policy 1, policy_version 420070 (0.0009) [2023-12-27 08:32:47,199][20381] Updated weights for policy 1, policy_version 420080 (0.0009) [2023-12-27 08:32:47,211][20347] Updated weights for policy 0, policy_version 420482 (0.0007) [2023-12-27 08:32:47,264][20347] Updated weights for policy 0, policy_version 420492 (0.0006) [2023-12-27 08:32:47,311][20347] Updated weights for policy 0, policy_version 420502 (0.0007) [2023-12-27 08:32:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 215228416. Throughput: 0: 10306.2, 1: 10506.3. Samples: 215231964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:47,363][19170] Avg episode reward: [(0, '713.940'), (1, '854.940')] [2023-12-27 08:32:47,363][20347] Updated weights for policy 0, policy_version 420512 (0.0007) [2023-12-27 08:32:47,890][20381] Updated weights for policy 1, policy_version 420090 (0.0006) [2023-12-27 08:32:47,945][20381] Updated weights for policy 1, policy_version 420100 (0.0006) [2023-12-27 08:32:48,004][20381] Updated weights for policy 1, policy_version 420110 (0.0008) [2023-12-27 08:32:48,072][20347] Updated weights for policy 0, policy_version 420522 (0.0008) [2023-12-27 08:32:48,132][20347] Updated weights for policy 0, policy_version 420532 (0.0007) [2023-12-27 08:32:48,190][20347] Updated weights for policy 0, policy_version 420542 (0.0007) [2023-12-27 08:32:48,721][20381] Updated weights for policy 1, policy_version 420120 (0.0009) [2023-12-27 08:32:48,782][20381] Updated weights for policy 1, policy_version 420130 (0.0009) [2023-12-27 08:32:48,800][20347] Updated weights for policy 0, policy_version 420552 (0.0006) [2023-12-27 08:32:48,839][20381] Updated weights for policy 1, policy_version 420140 (0.0006) [2023-12-27 08:32:48,864][20347] Updated weights for policy 0, policy_version 420562 (0.0006) [2023-12-27 08:32:48,915][20347] Updated weights for policy 0, policy_version 420572 (0.0008) [2023-12-27 08:32:49,559][20381] Updated weights for policy 1, policy_version 420150 (0.0007) [2023-12-27 08:32:49,609][20381] Updated weights for policy 1, policy_version 420160 (0.0008) [2023-12-27 08:32:49,612][20347] Updated weights for policy 0, policy_version 420582 (0.0008) [2023-12-27 08:32:49,663][20381] Updated weights for policy 1, policy_version 420170 (0.0007) [2023-12-27 08:32:49,667][20347] Updated weights for policy 0, policy_version 420592 (0.0007) [2023-12-27 08:32:49,721][20347] Updated weights for policy 0, policy_version 420602 (0.0007) [2023-12-27 08:32:50,359][20347] Updated weights for policy 0, policy_version 420612 (0.0008) [2023-12-27 08:32:50,383][20381] Updated weights for policy 1, policy_version 420180 (0.0006) [2023-12-27 08:32:50,419][20347] Updated weights for policy 0, policy_version 420622 (0.0006) [2023-12-27 08:32:50,443][20381] Updated weights for policy 1, policy_version 420190 (0.0006) [2023-12-27 08:32:50,471][20347] Updated weights for policy 0, policy_version 420632 (0.0006) [2023-12-27 08:32:50,488][20381] Updated weights for policy 1, policy_version 420200 (0.0006) [2023-12-27 08:32:51,173][20347] Updated weights for policy 0, policy_version 420642 (0.0006) [2023-12-27 08:32:51,190][20381] Updated weights for policy 1, policy_version 420210 (0.0006) [2023-12-27 08:32:51,232][20347] Updated weights for policy 0, policy_version 420652 (0.0008) [2023-12-27 08:32:51,250][20381] Updated weights for policy 1, policy_version 420220 (0.0007) [2023-12-27 08:32:51,292][20347] Updated weights for policy 0, policy_version 420662 (0.0006) [2023-12-27 08:32:51,313][20381] Updated weights for policy 1, policy_version 420230 (0.0008) [2023-12-27 08:32:51,359][20347] Updated weights for policy 0, policy_version 420672 (0.0007) [2023-12-27 08:32:51,377][20381] Updated weights for policy 1, policy_version 420240 (0.0007) [2023-12-27 08:32:52,016][20381] Updated weights for policy 1, policy_version 420250 (0.0006) [2023-12-27 08:32:52,065][20381] Updated weights for policy 1, policy_version 420260 (0.0007) [2023-12-27 08:32:52,121][20381] Updated weights for policy 1, policy_version 420270 (0.0005) [2023-12-27 08:32:52,126][20347] Updated weights for policy 0, policy_version 420682 (0.0007) [2023-12-27 08:32:52,187][20347] Updated weights for policy 0, policy_version 420692 (0.0008) [2023-12-27 08:32:52,248][20347] Updated weights for policy 0, policy_version 420702 (0.0006) [2023-12-27 08:32:52,364][19170] Fps is (10 sec: 21295.6, 60 sec: 20889.0, 300 sec: 20743.7). Total num frames: 215326720. Throughput: 0: 10322.7, 1: 10459.7. Samples: 215293080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:52,365][19170] Avg episode reward: [(0, '712.450'), (1, '951.710')] [2023-12-27 08:32:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000420272_107610112.pth... [2023-12-27 08:32:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000420704_107716608.pth... [2023-12-27 08:32:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000419056_107298816.pth [2023-12-27 08:32:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000419488_107405312.pth [2023-12-27 08:32:52,833][20381] Updated weights for policy 1, policy_version 420280 (0.0008) [2023-12-27 08:32:52,891][20381] Updated weights for policy 1, policy_version 420290 (0.0007) [2023-12-27 08:32:52,943][20381] Updated weights for policy 1, policy_version 420300 (0.0007) [2023-12-27 08:32:52,944][20347] Updated weights for policy 0, policy_version 420712 (0.0007) [2023-12-27 08:32:53,004][20347] Updated weights for policy 0, policy_version 420722 (0.0007) [2023-12-27 08:32:53,053][20347] Updated weights for policy 0, policy_version 420732 (0.0007) [2023-12-27 08:32:53,566][20381] Updated weights for policy 1, policy_version 420310 (0.0007) [2023-12-27 08:32:53,624][20381] Updated weights for policy 1, policy_version 420320 (0.0006) [2023-12-27 08:32:53,681][20381] Updated weights for policy 1, policy_version 420330 (0.0005) [2023-12-27 08:32:53,689][20347] Updated weights for policy 0, policy_version 420742 (0.0007) [2023-12-27 08:32:53,744][20347] Updated weights for policy 0, policy_version 420752 (0.0008) [2023-12-27 08:32:53,800][20347] Updated weights for policy 0, policy_version 420762 (0.0008) [2023-12-27 08:32:54,318][20381] Updated weights for policy 1, policy_version 420340 (0.0006) [2023-12-27 08:32:54,373][20381] Updated weights for policy 1, policy_version 420350 (0.0008) [2023-12-27 08:32:54,423][20347] Updated weights for policy 0, policy_version 420772 (0.0007) [2023-12-27 08:32:54,433][20381] Updated weights for policy 1, policy_version 420360 (0.0007) [2023-12-27 08:32:54,471][20347] Updated weights for policy 0, policy_version 420782 (0.0006) [2023-12-27 08:32:54,519][20347] Updated weights for policy 0, policy_version 420792 (0.0005) [2023-12-27 08:32:55,098][20347] Updated weights for policy 0, policy_version 420802 (0.0006) [2023-12-27 08:32:55,140][20381] Updated weights for policy 1, policy_version 420370 (0.0007) [2023-12-27 08:32:55,153][20347] Updated weights for policy 0, policy_version 420812 (0.0008) [2023-12-27 08:32:55,191][20381] Updated weights for policy 1, policy_version 420380 (0.0006) [2023-12-27 08:32:55,210][20347] Updated weights for policy 0, policy_version 420822 (0.0010) [2023-12-27 08:32:55,239][20381] Updated weights for policy 1, policy_version 420390 (0.0007) [2023-12-27 08:32:55,262][20347] Updated weights for policy 0, policy_version 420832 (0.0010) [2023-12-27 08:32:55,286][20381] Updated weights for policy 1, policy_version 420400 (0.0006) [2023-12-27 08:32:55,918][20381] Updated weights for policy 1, policy_version 420410 (0.0006) [2023-12-27 08:32:55,940][20347] Updated weights for policy 0, policy_version 420842 (0.0007) [2023-12-27 08:32:55,969][20381] Updated weights for policy 1, policy_version 420420 (0.0006) [2023-12-27 08:32:55,997][20347] Updated weights for policy 0, policy_version 420852 (0.0006) [2023-12-27 08:32:56,029][20381] Updated weights for policy 1, policy_version 420430 (0.0005) [2023-12-27 08:32:56,051][20347] Updated weights for policy 0, policy_version 420862 (0.0009) [2023-12-27 08:32:56,643][20381] Updated weights for policy 1, policy_version 420440 (0.0006) [2023-12-27 08:32:56,694][20381] Updated weights for policy 1, policy_version 420450 (0.0007) [2023-12-27 08:32:56,703][20347] Updated weights for policy 0, policy_version 420872 (0.0010) [2023-12-27 08:32:56,745][20381] Updated weights for policy 1, policy_version 420460 (0.0006) [2023-12-27 08:32:56,761][20347] Updated weights for policy 0, policy_version 420882 (0.0010) [2023-12-27 08:32:56,808][20347] Updated weights for policy 0, policy_version 420892 (0.0010) [2023-12-27 08:32:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 215425024. Throughput: 0: 10331.8, 1: 10458.0. Samples: 215419128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:32:57,363][19170] Avg episode reward: [(0, '836.940'), (1, '942.060')] [2023-12-27 08:32:57,441][20347] Updated weights for policy 0, policy_version 420902 (0.0008) [2023-12-27 08:32:57,456][20381] Updated weights for policy 1, policy_version 420470 (0.0007) [2023-12-27 08:32:57,499][20347] Updated weights for policy 0, policy_version 420912 (0.0006) [2023-12-27 08:32:57,506][20381] Updated weights for policy 1, policy_version 420480 (0.0006) [2023-12-27 08:32:57,556][20381] Updated weights for policy 1, policy_version 420490 (0.0007) [2023-12-27 08:32:57,562][20347] Updated weights for policy 0, policy_version 420922 (0.0006) [2023-12-27 08:32:58,185][20347] Updated weights for policy 0, policy_version 420932 (0.0007) [2023-12-27 08:32:58,246][20347] Updated weights for policy 0, policy_version 420942 (0.0008) [2023-12-27 08:32:58,273][20381] Updated weights for policy 1, policy_version 420500 (0.0008) [2023-12-27 08:32:58,306][20347] Updated weights for policy 0, policy_version 420952 (0.0006) [2023-12-27 08:32:58,336][20381] Updated weights for policy 1, policy_version 420510 (0.0007) [2023-12-27 08:32:58,401][20381] Updated weights for policy 1, policy_version 420520 (0.0008) [2023-12-27 08:32:59,033][20347] Updated weights for policy 0, policy_version 420962 (0.0009) [2023-12-27 08:32:59,087][20347] Updated weights for policy 0, policy_version 420972 (0.0007) [2023-12-27 08:32:59,119][20381] Updated weights for policy 1, policy_version 420530 (0.0007) [2023-12-27 08:32:59,145][20347] Updated weights for policy 0, policy_version 420982 (0.0010) [2023-12-27 08:32:59,177][20381] Updated weights for policy 1, policy_version 420540 (0.0008) [2023-12-27 08:32:59,194][20347] Updated weights for policy 0, policy_version 420992 (0.0010) [2023-12-27 08:32:59,241][20381] Updated weights for policy 1, policy_version 420550 (0.0010) [2023-12-27 08:32:59,303][20381] Updated weights for policy 1, policy_version 420560 (0.0008) [2023-12-27 08:32:59,888][20347] Updated weights for policy 0, policy_version 421002 (0.0010) [2023-12-27 08:32:59,913][20381] Updated weights for policy 1, policy_version 420570 (0.0007) [2023-12-27 08:32:59,947][20347] Updated weights for policy 0, policy_version 421012 (0.0010) [2023-12-27 08:32:59,974][20381] Updated weights for policy 1, policy_version 420580 (0.0010) [2023-12-27 08:33:00,000][20347] Updated weights for policy 0, policy_version 421022 (0.0010) [2023-12-27 08:33:00,030][20381] Updated weights for policy 1, policy_version 420590 (0.0011) [2023-12-27 08:33:00,632][20347] Updated weights for policy 0, policy_version 421032 (0.0006) [2023-12-27 08:33:00,692][20347] Updated weights for policy 0, policy_version 421042 (0.0006) [2023-12-27 08:33:00,713][20381] Updated weights for policy 1, policy_version 420600 (0.0007) [2023-12-27 08:33:00,749][20347] Updated weights for policy 0, policy_version 421052 (0.0008) [2023-12-27 08:33:00,759][20381] Updated weights for policy 1, policy_version 420610 (0.0005) [2023-12-27 08:33:00,817][20381] Updated weights for policy 1, policy_version 420620 (0.0005) [2023-12-27 08:33:01,374][20381] Updated weights for policy 1, policy_version 420630 (0.0009) [2023-12-27 08:33:01,384][20347] Updated weights for policy 0, policy_version 421062 (0.0012) [2023-12-27 08:33:01,438][20381] Updated weights for policy 1, policy_version 420640 (0.0010) [2023-12-27 08:33:01,439][20347] Updated weights for policy 0, policy_version 421072 (0.0008) [2023-12-27 08:33:01,497][20347] Updated weights for policy 0, policy_version 421082 (0.0007) [2023-12-27 08:33:01,502][20381] Updated weights for policy 1, policy_version 420650 (0.0011) [2023-12-27 08:33:02,085][20347] Updated weights for policy 0, policy_version 421092 (0.0007) [2023-12-27 08:33:02,140][20347] Updated weights for policy 0, policy_version 421102 (0.0009) [2023-12-27 08:33:02,181][20381] Updated weights for policy 1, policy_version 420660 (0.0010) [2023-12-27 08:33:02,205][20347] Updated weights for policy 0, policy_version 421112 (0.0010) [2023-12-27 08:33:02,230][20381] Updated weights for policy 1, policy_version 420670 (0.0008) [2023-12-27 08:33:02,287][20381] Updated weights for policy 1, policy_version 420680 (0.0006) [2023-12-27 08:33:02,362][19170] Fps is (10 sec: 21303.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 215539712. Throughput: 0: 10350.0, 1: 10450.6. Samples: 215544612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:33:02,363][19170] Avg episode reward: [(0, '827.830'), (1, '951.640')] [2023-12-27 08:33:02,866][20347] Updated weights for policy 0, policy_version 421122 (0.0009) [2023-12-27 08:33:02,921][20347] Updated weights for policy 0, policy_version 421132 (0.0005) [2023-12-27 08:33:02,954][20381] Updated weights for policy 1, policy_version 420690 (0.0008) [2023-12-27 08:33:02,978][20347] Updated weights for policy 0, policy_version 421142 (0.0005) [2023-12-27 08:33:03,004][20381] Updated weights for policy 1, policy_version 420700 (0.0011) [2023-12-27 08:33:03,027][20347] Updated weights for policy 0, policy_version 421152 (0.0006) [2023-12-27 08:33:03,052][20381] Updated weights for policy 1, policy_version 420710 (0.0010) [2023-12-27 08:33:03,104][20381] Updated weights for policy 1, policy_version 420720 (0.0010) [2023-12-27 08:33:03,645][20347] Updated weights for policy 0, policy_version 421162 (0.0006) [2023-12-27 08:33:03,702][20347] Updated weights for policy 0, policy_version 421172 (0.0006) [2023-12-27 08:33:03,743][20381] Updated weights for policy 1, policy_version 420730 (0.0010) [2023-12-27 08:33:03,755][20347] Updated weights for policy 0, policy_version 421182 (0.0005) [2023-12-27 08:33:03,801][20381] Updated weights for policy 1, policy_version 420740 (0.0010) [2023-12-27 08:33:03,867][20381] Updated weights for policy 1, policy_version 420750 (0.0009) [2023-12-27 08:33:04,428][20347] Updated weights for policy 0, policy_version 421192 (0.0007) [2023-12-27 08:33:04,487][20347] Updated weights for policy 0, policy_version 421202 (0.0007) [2023-12-27 08:33:04,541][20381] Updated weights for policy 1, policy_version 420760 (0.0010) [2023-12-27 08:33:04,543][20347] Updated weights for policy 0, policy_version 421212 (0.0007) [2023-12-27 08:33:04,599][20381] Updated weights for policy 1, policy_version 420770 (0.0011) [2023-12-27 08:33:04,658][20381] Updated weights for policy 1, policy_version 420780 (0.0011) [2023-12-27 08:33:05,181][20347] Updated weights for policy 0, policy_version 421222 (0.0007) [2023-12-27 08:33:05,237][20347] Updated weights for policy 0, policy_version 421232 (0.0005) [2023-12-27 08:33:05,295][20347] Updated weights for policy 0, policy_version 421242 (0.0005) [2023-12-27 08:33:05,368][20381] Updated weights for policy 1, policy_version 420790 (0.0011) [2023-12-27 08:33:05,427][20381] Updated weights for policy 1, policy_version 420800 (0.0009) [2023-12-27 08:33:05,479][20381] Updated weights for policy 1, policy_version 420810 (0.0010) [2023-12-27 08:33:05,925][20347] Updated weights for policy 0, policy_version 421252 (0.0007) [2023-12-27 08:33:05,974][20347] Updated weights for policy 0, policy_version 421262 (0.0007) [2023-12-27 08:33:06,042][20347] Updated weights for policy 0, policy_version 421272 (0.0007) [2023-12-27 08:33:06,140][20381] Updated weights for policy 1, policy_version 420820 (0.0011) [2023-12-27 08:33:06,190][20381] Updated weights for policy 1, policy_version 420830 (0.0011) [2023-12-27 08:33:06,247][20381] Updated weights for policy 1, policy_version 420840 (0.0009) [2023-12-27 08:33:06,696][20347] Updated weights for policy 0, policy_version 421282 (0.0007) [2023-12-27 08:33:06,755][20347] Updated weights for policy 0, policy_version 421292 (0.0008) [2023-12-27 08:33:06,815][20347] Updated weights for policy 0, policy_version 421302 (0.0006) [2023-12-27 08:33:06,853][20381] Updated weights for policy 1, policy_version 420850 (0.0011) [2023-12-27 08:33:06,881][20347] Updated weights for policy 0, policy_version 421312 (0.0007) [2023-12-27 08:33:06,907][20381] Updated weights for policy 1, policy_version 420860 (0.0010) [2023-12-27 08:33:06,964][20381] Updated weights for policy 1, policy_version 420870 (0.0008) [2023-12-27 08:33:07,021][20381] Updated weights for policy 1, policy_version 420880 (0.0008) [2023-12-27 08:33:07,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 215638016. Throughput: 0: 10342.3, 1: 10419.2. Samples: 215607564. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:07,363][19170] Avg episode reward: [(0, '790.560'), (1, '951.800')] [2023-12-27 08:33:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000421312_107872256.pth... [2023-12-27 08:33:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000420880_107765760.pth... [2023-12-27 08:33:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000420096_107560960.pth [2023-12-27 08:33:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000419664_107454464.pth [2023-12-27 08:33:07,569][20347] Updated weights for policy 0, policy_version 421322 (0.0008) [2023-12-27 08:33:07,625][20347] Updated weights for policy 0, policy_version 421332 (0.0008) [2023-12-27 08:33:07,674][20381] Updated weights for policy 1, policy_version 420890 (0.0011) [2023-12-27 08:33:07,687][20347] Updated weights for policy 0, policy_version 421342 (0.0006) [2023-12-27 08:33:07,734][20381] Updated weights for policy 1, policy_version 420900 (0.0011) [2023-12-27 08:33:07,785][20381] Updated weights for policy 1, policy_version 420910 (0.0010) [2023-12-27 08:33:08,363][20347] Updated weights for policy 0, policy_version 421352 (0.0007) [2023-12-27 08:33:08,413][20347] Updated weights for policy 0, policy_version 421362 (0.0007) [2023-12-27 08:33:08,423][20381] Updated weights for policy 1, policy_version 420920 (0.0008) [2023-12-27 08:33:08,466][20347] Updated weights for policy 0, policy_version 421372 (0.0007) [2023-12-27 08:33:08,476][20381] Updated weights for policy 1, policy_version 420930 (0.0006) [2023-12-27 08:33:08,525][20381] Updated weights for policy 1, policy_version 420940 (0.0008) [2023-12-27 08:33:09,120][20347] Updated weights for policy 0, policy_version 421382 (0.0007) [2023-12-27 08:33:09,181][20347] Updated weights for policy 0, policy_version 421392 (0.0007) [2023-12-27 08:33:09,201][20381] Updated weights for policy 1, policy_version 420950 (0.0006) [2023-12-27 08:33:09,235][20347] Updated weights for policy 0, policy_version 421402 (0.0007) [2023-12-27 08:33:09,262][20381] Updated weights for policy 1, policy_version 420960 (0.0008) [2023-12-27 08:33:09,320][20381] Updated weights for policy 1, policy_version 420970 (0.0006) [2023-12-27 08:33:09,957][20347] Updated weights for policy 0, policy_version 421412 (0.0008) [2023-12-27 08:33:10,019][20347] Updated weights for policy 0, policy_version 421422 (0.0007) [2023-12-27 08:33:10,032][20381] Updated weights for policy 1, policy_version 420980 (0.0007) [2023-12-27 08:33:10,078][20347] Updated weights for policy 0, policy_version 421432 (0.0008) [2023-12-27 08:33:10,092][20381] Updated weights for policy 1, policy_version 420990 (0.0007) [2023-12-27 08:33:10,152][20381] Updated weights for policy 1, policy_version 421000 (0.0007) [2023-12-27 08:33:10,773][20347] Updated weights for policy 0, policy_version 421442 (0.0006) [2023-12-27 08:33:10,811][20381] Updated weights for policy 1, policy_version 421010 (0.0007) [2023-12-27 08:33:10,832][20347] Updated weights for policy 0, policy_version 421452 (0.0007) [2023-12-27 08:33:10,866][20381] Updated weights for policy 1, policy_version 421020 (0.0005) [2023-12-27 08:33:10,896][20347] Updated weights for policy 0, policy_version 421462 (0.0007) [2023-12-27 08:33:10,929][20381] Updated weights for policy 1, policy_version 421030 (0.0005) [2023-12-27 08:33:10,950][20347] Updated weights for policy 0, policy_version 421472 (0.0008) [2023-12-27 08:33:10,982][20381] Updated weights for policy 1, policy_version 421040 (0.0005) [2023-12-27 08:33:11,591][20381] Updated weights for policy 1, policy_version 421050 (0.0007) [2023-12-27 08:33:11,655][20381] Updated weights for policy 1, policy_version 421060 (0.0008) [2023-12-27 08:33:11,690][20347] Updated weights for policy 0, policy_version 421482 (0.0008) [2023-12-27 08:33:11,715][20381] Updated weights for policy 1, policy_version 421070 (0.0008) [2023-12-27 08:33:11,762][20347] Updated weights for policy 0, policy_version 421492 (0.0007) [2023-12-27 08:33:11,829][20347] Updated weights for policy 0, policy_version 421502 (0.0009) [2023-12-27 08:33:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 215736320. Throughput: 0: 10384.0, 1: 10439.9. Samples: 215731192. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:12,363][19170] Avg episode reward: [(0, '740.780'), (1, '951.860')] [2023-12-27 08:33:12,395][20381] Updated weights for policy 1, policy_version 421080 (0.0008) [2023-12-27 08:33:12,451][20381] Updated weights for policy 1, policy_version 421090 (0.0008) [2023-12-27 08:33:12,505][20381] Updated weights for policy 1, policy_version 421100 (0.0006) [2023-12-27 08:33:12,524][20347] Updated weights for policy 0, policy_version 421512 (0.0007) [2023-12-27 08:33:12,576][20347] Updated weights for policy 0, policy_version 421522 (0.0010) [2023-12-27 08:33:12,626][20347] Updated weights for policy 0, policy_version 421532 (0.0009) [2023-12-27 08:33:13,158][20381] Updated weights for policy 1, policy_version 421110 (0.0006) [2023-12-27 08:33:13,218][20381] Updated weights for policy 1, policy_version 421120 (0.0006) [2023-12-27 08:33:13,270][20381] Updated weights for policy 1, policy_version 421130 (0.0008) [2023-12-27 08:33:13,306][20347] Updated weights for policy 0, policy_version 421542 (0.0007) [2023-12-27 08:33:13,357][20347] Updated weights for policy 0, policy_version 421552 (0.0008) [2023-12-27 08:33:13,407][20347] Updated weights for policy 0, policy_version 421562 (0.0008) [2023-12-27 08:33:13,933][20381] Updated weights for policy 1, policy_version 421140 (0.0009) [2023-12-27 08:33:13,988][20381] Updated weights for policy 1, policy_version 421150 (0.0005) [2023-12-27 08:33:14,014][20347] Updated weights for policy 0, policy_version 421572 (0.0008) [2023-12-27 08:33:14,044][20381] Updated weights for policy 1, policy_version 421160 (0.0009) [2023-12-27 08:33:14,068][20347] Updated weights for policy 0, policy_version 421582 (0.0006) [2023-12-27 08:33:14,126][20347] Updated weights for policy 0, policy_version 421592 (0.0007) [2023-12-27 08:33:14,670][20381] Updated weights for policy 1, policy_version 421170 (0.0010) [2023-12-27 08:33:14,728][20381] Updated weights for policy 1, policy_version 421180 (0.0005) [2023-12-27 08:33:14,731][20347] Updated weights for policy 0, policy_version 421602 (0.0007) [2023-12-27 08:33:14,795][20347] Updated weights for policy 0, policy_version 421612 (0.0007) [2023-12-27 08:33:14,797][20381] Updated weights for policy 1, policy_version 421190 (0.0007) [2023-12-27 08:33:14,856][20381] Updated weights for policy 1, policy_version 421200 (0.0008) [2023-12-27 08:33:14,859][20347] Updated weights for policy 0, policy_version 421622 (0.0007) [2023-12-27 08:33:14,922][20347] Updated weights for policy 0, policy_version 421632 (0.0007) [2023-12-27 08:33:15,442][20381] Updated weights for policy 1, policy_version 421210 (0.0008) [2023-12-27 08:33:15,503][20381] Updated weights for policy 1, policy_version 421220 (0.0008) [2023-12-27 08:33:15,559][20381] Updated weights for policy 1, policy_version 421230 (0.0007) [2023-12-27 08:33:15,632][20347] Updated weights for policy 0, policy_version 421642 (0.0007) [2023-12-27 08:33:15,681][20347] Updated weights for policy 0, policy_version 421652 (0.0007) [2023-12-27 08:33:15,743][20347] Updated weights for policy 0, policy_version 421662 (0.0008) [2023-12-27 08:33:16,140][20381] Updated weights for policy 1, policy_version 421240 (0.0006) [2023-12-27 08:33:16,191][20381] Updated weights for policy 1, policy_version 421250 (0.0006) [2023-12-27 08:33:16,237][20381] Updated weights for policy 1, policy_version 421260 (0.0007) [2023-12-27 08:33:16,465][20347] Updated weights for policy 0, policy_version 421672 (0.0008) [2023-12-27 08:33:16,513][20347] Updated weights for policy 0, policy_version 421682 (0.0007) [2023-12-27 08:33:16,564][20347] Updated weights for policy 0, policy_version 421692 (0.0007) [2023-12-27 08:33:16,913][20381] Updated weights for policy 1, policy_version 421270 (0.0007) [2023-12-27 08:33:16,959][20381] Updated weights for policy 1, policy_version 421280 (0.0005) [2023-12-27 08:33:17,015][20381] Updated weights for policy 1, policy_version 421290 (0.0007) [2023-12-27 08:33:17,232][20347] Updated weights for policy 0, policy_version 421702 (0.0007) [2023-12-27 08:33:17,280][20347] Updated weights for policy 0, policy_version 421712 (0.0007) [2023-12-27 08:33:17,336][20347] Updated weights for policy 0, policy_version 421722 (0.0008) [2023-12-27 08:33:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 215842816. Throughput: 0: 10337.7, 1: 10460.7. Samples: 215856316. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:17,363][19170] Avg episode reward: [(0, '828.240'), (1, '990.370')] [2023-12-27 08:33:17,634][20381] Updated weights for policy 1, policy_version 421300 (0.0006) [2023-12-27 08:33:17,694][20381] Updated weights for policy 1, policy_version 421310 (0.0005) [2023-12-27 08:33:17,749][20381] Updated weights for policy 1, policy_version 421320 (0.0005) [2023-12-27 08:33:18,038][20347] Updated weights for policy 0, policy_version 421732 (0.0007) [2023-12-27 08:33:18,092][20347] Updated weights for policy 0, policy_version 421742 (0.0005) [2023-12-27 08:33:18,148][20347] Updated weights for policy 0, policy_version 421752 (0.0006) [2023-12-27 08:33:18,342][20381] Updated weights for policy 1, policy_version 421330 (0.0006) [2023-12-27 08:33:18,403][20381] Updated weights for policy 1, policy_version 421340 (0.0007) [2023-12-27 08:33:18,458][20381] Updated weights for policy 1, policy_version 421350 (0.0005) [2023-12-27 08:33:18,508][20381] Updated weights for policy 1, policy_version 421360 (0.0006) [2023-12-27 08:33:18,783][20347] Updated weights for policy 0, policy_version 421762 (0.0008) [2023-12-27 08:33:18,842][20347] Updated weights for policy 0, policy_version 421772 (0.0008) [2023-12-27 08:33:18,900][20347] Updated weights for policy 0, policy_version 421782 (0.0008) [2023-12-27 08:33:18,958][20347] Updated weights for policy 0, policy_version 421792 (0.0008) [2023-12-27 08:33:19,127][20381] Updated weights for policy 1, policy_version 421370 (0.0007) [2023-12-27 08:33:19,182][20381] Updated weights for policy 1, policy_version 421380 (0.0008) [2023-12-27 08:33:19,241][20381] Updated weights for policy 1, policy_version 421390 (0.0008) [2023-12-27 08:33:19,671][20347] Updated weights for policy 0, policy_version 421802 (0.0006) [2023-12-27 08:33:19,735][20347] Updated weights for policy 0, policy_version 421812 (0.0006) [2023-12-27 08:33:19,791][20347] Updated weights for policy 0, policy_version 421822 (0.0009) [2023-12-27 08:33:19,996][20381] Updated weights for policy 1, policy_version 421400 (0.0008) [2023-12-27 08:33:20,057][20381] Updated weights for policy 1, policy_version 421410 (0.0007) [2023-12-27 08:33:20,114][20381] Updated weights for policy 1, policy_version 421420 (0.0008) [2023-12-27 08:33:20,464][20347] Updated weights for policy 0, policy_version 421832 (0.0008) [2023-12-27 08:33:20,521][20347] Updated weights for policy 0, policy_version 421842 (0.0007) [2023-12-27 08:33:20,586][20347] Updated weights for policy 0, policy_version 421852 (0.0008) [2023-12-27 08:33:20,884][20381] Updated weights for policy 1, policy_version 421430 (0.0008) [2023-12-27 08:33:20,937][20381] Updated weights for policy 1, policy_version 421440 (0.0007) [2023-12-27 08:33:20,991][20381] Updated weights for policy 1, policy_version 421450 (0.0007) [2023-12-27 08:33:21,242][20347] Updated weights for policy 0, policy_version 421862 (0.0006) [2023-12-27 08:33:21,306][20347] Updated weights for policy 0, policy_version 421872 (0.0008) [2023-12-27 08:33:21,324][20291] KL-divergence is very high: 112.3401 [2023-12-27 08:33:21,372][20347] Updated weights for policy 0, policy_version 421882 (0.0008) [2023-12-27 08:33:21,760][20381] Updated weights for policy 1, policy_version 421460 (0.0008) [2023-12-27 08:33:21,816][20381] Updated weights for policy 1, policy_version 421470 (0.0008) [2023-12-27 08:33:21,869][20381] Updated weights for policy 1, policy_version 421480 (0.0008) [2023-12-27 08:33:22,092][20347] Updated weights for policy 0, policy_version 421892 (0.0008) [2023-12-27 08:33:22,142][20347] Updated weights for policy 0, policy_version 421902 (0.0008) [2023-12-27 08:33:22,198][20347] Updated weights for policy 0, policy_version 421912 (0.0007) [2023-12-27 08:33:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 215949312. Throughput: 0: 10303.5, 1: 10414.6. Samples: 215918580. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:22,363][19170] Avg episode reward: [(0, '857.750'), (1, '942.240')] [2023-12-27 08:33:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000421920_108027904.pth... [2023-12-27 08:33:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000421488_107921408.pth... [2023-12-27 08:33:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000420704_107716608.pth [2023-12-27 08:33:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000420272_107610112.pth [2023-12-27 08:33:22,596][20381] Updated weights for policy 1, policy_version 421490 (0.0008) [2023-12-27 08:33:22,655][20381] Updated weights for policy 1, policy_version 421500 (0.0008) [2023-12-27 08:33:22,711][20381] Updated weights for policy 1, policy_version 421510 (0.0007) [2023-12-27 08:33:22,765][20381] Updated weights for policy 1, policy_version 421520 (0.0005) [2023-12-27 08:33:22,887][20347] Updated weights for policy 0, policy_version 421922 (0.0006) [2023-12-27 08:33:22,947][20347] Updated weights for policy 0, policy_version 421932 (0.0005) [2023-12-27 08:33:23,001][20347] Updated weights for policy 0, policy_version 421942 (0.0005) [2023-12-27 08:33:23,060][20347] Updated weights for policy 0, policy_version 421952 (0.0007) [2023-12-27 08:33:23,393][20381] Updated weights for policy 1, policy_version 421530 (0.0007) [2023-12-27 08:33:23,446][20381] Updated weights for policy 1, policy_version 421540 (0.0008) [2023-12-27 08:33:23,499][20381] Updated weights for policy 1, policy_version 421550 (0.0007) [2023-12-27 08:33:23,621][20347] Updated weights for policy 0, policy_version 421962 (0.0007) [2023-12-27 08:33:23,677][20347] Updated weights for policy 0, policy_version 421972 (0.0008) [2023-12-27 08:33:23,735][20347] Updated weights for policy 0, policy_version 421982 (0.0008) [2023-12-27 08:33:24,133][20381] Updated weights for policy 1, policy_version 421560 (0.0006) [2023-12-27 08:33:24,188][20381] Updated weights for policy 1, policy_version 421570 (0.0005) [2023-12-27 08:33:24,241][20381] Updated weights for policy 1, policy_version 421580 (0.0005) [2023-12-27 08:33:24,414][20347] Updated weights for policy 0, policy_version 421992 (0.0006) [2023-12-27 08:33:24,473][20347] Updated weights for policy 0, policy_version 422002 (0.0009) [2023-12-27 08:33:24,531][20347] Updated weights for policy 0, policy_version 422012 (0.0009) [2023-12-27 08:33:24,838][20381] Updated weights for policy 1, policy_version 421590 (0.0008) [2023-12-27 08:33:24,882][20381] Updated weights for policy 1, policy_version 421600 (0.0007) [2023-12-27 08:33:24,933][20381] Updated weights for policy 1, policy_version 421610 (0.0008) [2023-12-27 08:33:25,107][20347] Updated weights for policy 0, policy_version 422022 (0.0010) [2023-12-27 08:33:25,158][20347] Updated weights for policy 0, policy_version 422032 (0.0010) [2023-12-27 08:33:25,213][20347] Updated weights for policy 0, policy_version 422042 (0.0010) [2023-12-27 08:33:25,579][20381] Updated weights for policy 1, policy_version 421620 (0.0008) [2023-12-27 08:33:25,643][20381] Updated weights for policy 1, policy_version 421630 (0.0008) [2023-12-27 08:33:25,699][20381] Updated weights for policy 1, policy_version 421640 (0.0005) [2023-12-27 08:33:25,836][20347] Updated weights for policy 0, policy_version 422052 (0.0010) [2023-12-27 08:33:25,886][20347] Updated weights for policy 0, policy_version 422062 (0.0010) [2023-12-27 08:33:25,945][20347] Updated weights for policy 0, policy_version 422072 (0.0011) [2023-12-27 08:33:25,980][20291] KL-divergence is very high: 163.3109 [2023-12-27 08:33:26,266][20381] Updated weights for policy 1, policy_version 421650 (0.0005) [2023-12-27 08:33:26,325][20381] Updated weights for policy 1, policy_version 421660 (0.0005) [2023-12-27 08:33:26,371][20381] Updated weights for policy 1, policy_version 421670 (0.0005) [2023-12-27 08:33:26,418][20381] Updated weights for policy 1, policy_version 421680 (0.0007) [2023-12-27 08:33:26,589][20347] Updated weights for policy 0, policy_version 422082 (0.0010) [2023-12-27 08:33:26,646][20347] Updated weights for policy 0, policy_version 422092 (0.0010) [2023-12-27 08:33:26,706][20347] Updated weights for policy 0, policy_version 422102 (0.0010) [2023-12-27 08:33:26,766][20347] Updated weights for policy 0, policy_version 422112 (0.0010) [2023-12-27 08:33:26,937][20381] Updated weights for policy 1, policy_version 421690 (0.0005) [2023-12-27 08:33:26,988][20381] Updated weights for policy 1, policy_version 421700 (0.0005) [2023-12-27 08:33:27,037][20381] Updated weights for policy 1, policy_version 421710 (0.0007) [2023-12-27 08:33:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 216055808. Throughput: 0: 10308.5, 1: 10476.4. Samples: 216044756. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:27,363][19170] Avg episode reward: [(0, '886.010'), (1, '942.060')] [2023-12-27 08:33:27,415][20347] Updated weights for policy 0, policy_version 422122 (0.0010) [2023-12-27 08:33:27,466][20347] Updated weights for policy 0, policy_version 422132 (0.0010) [2023-12-27 08:33:27,515][20347] Updated weights for policy 0, policy_version 422142 (0.0010) [2023-12-27 08:33:27,664][20381] Updated weights for policy 1, policy_version 421720 (0.0007) [2023-12-27 08:33:27,725][20381] Updated weights for policy 1, policy_version 421730 (0.0005) [2023-12-27 08:33:27,781][20381] Updated weights for policy 1, policy_version 421740 (0.0005) [2023-12-27 08:33:28,217][20347] Updated weights for policy 0, policy_version 422152 (0.0010) [2023-12-27 08:33:28,280][20347] Updated weights for policy 0, policy_version 422162 (0.0010) [2023-12-27 08:33:28,294][20381] Updated weights for policy 1, policy_version 421750 (0.0005) [2023-12-27 08:33:28,332][20347] Updated weights for policy 0, policy_version 422172 (0.0010) [2023-12-27 08:33:28,357][20381] Updated weights for policy 1, policy_version 421760 (0.0007) [2023-12-27 08:33:28,420][20381] Updated weights for policy 1, policy_version 421770 (0.0008) [2023-12-27 08:33:28,974][20381] Updated weights for policy 1, policy_version 421780 (0.0009) [2023-12-27 08:33:29,005][20347] Updated weights for policy 0, policy_version 422182 (0.0010) [2023-12-27 08:33:29,026][20381] Updated weights for policy 1, policy_version 421790 (0.0005) [2023-12-27 08:33:29,054][20347] Updated weights for policy 0, policy_version 422192 (0.0010) [2023-12-27 08:33:29,073][20381] Updated weights for policy 1, policy_version 421800 (0.0005) [2023-12-27 08:33:29,108][20347] Updated weights for policy 0, policy_version 422202 (0.0010) [2023-12-27 08:33:29,703][20381] Updated weights for policy 1, policy_version 421810 (0.0005) [2023-12-27 08:33:29,751][20381] Updated weights for policy 1, policy_version 421820 (0.0009) [2023-12-27 08:33:29,809][20381] Updated weights for policy 1, policy_version 421830 (0.0008) [2023-12-27 08:33:29,820][20347] Updated weights for policy 0, policy_version 422212 (0.0010) [2023-12-27 08:33:29,869][20381] Updated weights for policy 1, policy_version 421840 (0.0010) [2023-12-27 08:33:29,884][20347] Updated weights for policy 0, policy_version 422222 (0.0007) [2023-12-27 08:33:29,943][20347] Updated weights for policy 0, policy_version 422232 (0.0008) [2023-12-27 08:33:30,493][20381] Updated weights for policy 1, policy_version 421850 (0.0008) [2023-12-27 08:33:30,558][20381] Updated weights for policy 1, policy_version 421860 (0.0006) [2023-12-27 08:33:30,610][20347] Updated weights for policy 0, policy_version 422242 (0.0010) [2023-12-27 08:33:30,617][20381] Updated weights for policy 1, policy_version 421870 (0.0006) [2023-12-27 08:33:30,657][20347] Updated weights for policy 0, policy_version 422252 (0.0009) [2023-12-27 08:33:30,710][20347] Updated weights for policy 0, policy_version 422262 (0.0007) [2023-12-27 08:33:30,759][20347] Updated weights for policy 0, policy_version 422272 (0.0009) [2023-12-27 08:33:31,231][20381] Updated weights for policy 1, policy_version 421880 (0.0007) [2023-12-27 08:33:31,291][20381] Updated weights for policy 1, policy_version 421890 (0.0008) [2023-12-27 08:33:31,348][20381] Updated weights for policy 1, policy_version 421900 (0.0006) [2023-12-27 08:33:31,500][20347] Updated weights for policy 0, policy_version 422282 (0.0007) [2023-12-27 08:33:31,558][20347] Updated weights for policy 0, policy_version 422292 (0.0008) [2023-12-27 08:33:31,617][20347] Updated weights for policy 0, policy_version 422302 (0.0008) [2023-12-27 08:33:32,030][20381] Updated weights for policy 1, policy_version 421910 (0.0007) [2023-12-27 08:33:32,085][20381] Updated weights for policy 1, policy_version 421920 (0.0005) [2023-12-27 08:33:32,134][20381] Updated weights for policy 1, policy_version 421930 (0.0005) [2023-12-27 08:33:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 216162304. Throughput: 0: 10354.9, 1: 10581.0. Samples: 216174080. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:32,363][19170] Avg episode reward: [(0, '971.780'), (1, '932.480')] [2023-12-27 08:33:32,375][20347] Updated weights for policy 0, policy_version 422312 (0.0008) [2023-12-27 08:33:32,429][20347] Updated weights for policy 0, policy_version 422322 (0.0006) [2023-12-27 08:33:32,479][20347] Updated weights for policy 0, policy_version 422332 (0.0008) [2023-12-27 08:33:32,747][20381] Updated weights for policy 1, policy_version 421940 (0.0005) [2023-12-27 08:33:32,810][20381] Updated weights for policy 1, policy_version 421950 (0.0005) [2023-12-27 08:33:32,871][20381] Updated weights for policy 1, policy_version 421960 (0.0007) [2023-12-27 08:33:33,132][20347] Updated weights for policy 0, policy_version 422342 (0.0006) [2023-12-27 08:33:33,183][20347] Updated weights for policy 0, policy_version 422352 (0.0006) [2023-12-27 08:33:33,229][20347] Updated weights for policy 0, policy_version 422362 (0.0008) [2023-12-27 08:33:33,453][20381] Updated weights for policy 1, policy_version 421970 (0.0007) [2023-12-27 08:33:33,506][20381] Updated weights for policy 1, policy_version 421980 (0.0007) [2023-12-27 08:33:33,555][20381] Updated weights for policy 1, policy_version 421990 (0.0007) [2023-12-27 08:33:33,606][20381] Updated weights for policy 1, policy_version 422000 (0.0008) [2023-12-27 08:33:33,950][20347] Updated weights for policy 0, policy_version 422372 (0.0008) [2023-12-27 08:33:34,007][20347] Updated weights for policy 0, policy_version 422382 (0.0005) [2023-12-27 08:33:34,066][20347] Updated weights for policy 0, policy_version 422392 (0.0005) [2023-12-27 08:33:34,206][20381] Updated weights for policy 1, policy_version 422010 (0.0008) [2023-12-27 08:33:34,260][20381] Updated weights for policy 1, policy_version 422020 (0.0006) [2023-12-27 08:33:34,321][20381] Updated weights for policy 1, policy_version 422030 (0.0007) [2023-12-27 08:33:34,732][20347] Updated weights for policy 0, policy_version 422402 (0.0007) [2023-12-27 08:33:34,797][20347] Updated weights for policy 0, policy_version 422412 (0.0008) [2023-12-27 08:33:34,860][20347] Updated weights for policy 0, policy_version 422422 (0.0008) [2023-12-27 08:33:34,923][20347] Updated weights for policy 0, policy_version 422432 (0.0007) [2023-12-27 08:33:34,969][20381] Updated weights for policy 1, policy_version 422040 (0.0008) [2023-12-27 08:33:35,025][20381] Updated weights for policy 1, policy_version 422050 (0.0008) [2023-12-27 08:33:35,083][20381] Updated weights for policy 1, policy_version 422060 (0.0008) [2023-12-27 08:33:35,634][20347] Updated weights for policy 0, policy_version 422442 (0.0009) [2023-12-27 08:33:35,682][20381] Updated weights for policy 1, policy_version 422070 (0.0006) [2023-12-27 08:33:35,692][20347] Updated weights for policy 0, policy_version 422452 (0.0008) [2023-12-27 08:33:35,740][20381] Updated weights for policy 1, policy_version 422080 (0.0006) [2023-12-27 08:33:35,742][20347] Updated weights for policy 0, policy_version 422462 (0.0006) [2023-12-27 08:33:35,795][20381] Updated weights for policy 1, policy_version 422090 (0.0007) [2023-12-27 08:33:36,430][20347] Updated weights for policy 0, policy_version 422472 (0.0008) [2023-12-27 08:33:36,462][20381] Updated weights for policy 1, policy_version 422100 (0.0009) [2023-12-27 08:33:36,487][20347] Updated weights for policy 0, policy_version 422482 (0.0009) [2023-12-27 08:33:36,512][20381] Updated weights for policy 1, policy_version 422110 (0.0009) [2023-12-27 08:33:36,548][20347] Updated weights for policy 0, policy_version 422492 (0.0007) [2023-12-27 08:33:36,568][20381] Updated weights for policy 1, policy_version 422120 (0.0008) [2023-12-27 08:33:37,224][20381] Updated weights for policy 1, policy_version 422130 (0.0008) [2023-12-27 08:33:37,278][20347] Updated weights for policy 0, policy_version 422502 (0.0006) [2023-12-27 08:33:37,284][20381] Updated weights for policy 1, policy_version 422140 (0.0008) [2023-12-27 08:33:37,329][20347] Updated weights for policy 0, policy_version 422512 (0.0006) [2023-12-27 08:33:37,334][20381] Updated weights for policy 1, policy_version 422150 (0.0008) [2023-12-27 08:33:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 216260608. Throughput: 0: 10336.7, 1: 10644.8. Samples: 216237212. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:37,363][19170] Avg episode reward: [(0, '990.360'), (1, '883.960')] [2023-12-27 08:33:37,378][20347] Updated weights for policy 0, policy_version 422522 (0.0008) [2023-12-27 08:33:37,386][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000422160_108093440.pth... [2023-12-27 08:33:37,386][20381] Updated weights for policy 1, policy_version 422160 (0.0007) [2023-12-27 08:33:37,389][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000420880_107765760.pth [2023-12-27 08:33:37,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000422528_108183552.pth... [2023-12-27 08:33:37,411][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000421312_107872256.pth [2023-12-27 08:33:38,060][20381] Updated weights for policy 1, policy_version 422170 (0.0006) [2023-12-27 08:33:38,070][20292] KL-divergence is very high: 104.8009 [2023-12-27 08:33:38,088][20347] Updated weights for policy 0, policy_version 422532 (0.0007) [2023-12-27 08:33:38,112][20381] Updated weights for policy 1, policy_version 422180 (0.0006) [2023-12-27 08:33:38,141][20347] Updated weights for policy 0, policy_version 422542 (0.0006) [2023-12-27 08:33:38,162][20381] Updated weights for policy 1, policy_version 422190 (0.0005) [2023-12-27 08:33:38,194][20347] Updated weights for policy 0, policy_version 422552 (0.0006) [2023-12-27 08:33:38,846][20381] Updated weights for policy 1, policy_version 422200 (0.0008) [2023-12-27 08:33:38,893][20347] Updated weights for policy 0, policy_version 422562 (0.0007) [2023-12-27 08:33:38,907][20381] Updated weights for policy 1, policy_version 422210 (0.0008) [2023-12-27 08:33:38,951][20347] Updated weights for policy 0, policy_version 422572 (0.0006) [2023-12-27 08:33:38,964][20381] Updated weights for policy 1, policy_version 422220 (0.0006) [2023-12-27 08:33:39,004][20347] Updated weights for policy 0, policy_version 422582 (0.0006) [2023-12-27 08:33:39,058][20347] Updated weights for policy 0, policy_version 422592 (0.0008) [2023-12-27 08:33:39,625][20381] Updated weights for policy 1, policy_version 422230 (0.0006) [2023-12-27 08:33:39,678][20381] Updated weights for policy 1, policy_version 422240 (0.0007) [2023-12-27 08:33:39,733][20381] Updated weights for policy 1, policy_version 422250 (0.0008) [2023-12-27 08:33:39,804][20347] Updated weights for policy 0, policy_version 422602 (0.0007) [2023-12-27 08:33:39,873][20347] Updated weights for policy 0, policy_version 422612 (0.0008) [2023-12-27 08:33:39,934][20347] Updated weights for policy 0, policy_version 422622 (0.0009) [2023-12-27 08:33:40,334][20381] Updated weights for policy 1, policy_version 422260 (0.0007) [2023-12-27 08:33:40,396][20381] Updated weights for policy 1, policy_version 422270 (0.0008) [2023-12-27 08:33:40,458][20381] Updated weights for policy 1, policy_version 422280 (0.0008) [2023-12-27 08:33:40,684][20347] Updated weights for policy 0, policy_version 422632 (0.0008) [2023-12-27 08:33:40,733][20347] Updated weights for policy 0, policy_version 422642 (0.0008) [2023-12-27 08:33:40,781][20347] Updated weights for policy 0, policy_version 422652 (0.0008) [2023-12-27 08:33:41,060][20381] Updated weights for policy 1, policy_version 422290 (0.0008) [2023-12-27 08:33:41,131][20381] Updated weights for policy 1, policy_version 422300 (0.0008) [2023-12-27 08:33:41,194][20381] Updated weights for policy 1, policy_version 422310 (0.0008) [2023-12-27 08:33:41,258][20381] Updated weights for policy 1, policy_version 422320 (0.0008) [2023-12-27 08:33:41,458][20347] Updated weights for policy 0, policy_version 422662 (0.0008) [2023-12-27 08:33:41,520][20347] Updated weights for policy 0, policy_version 422672 (0.0007) [2023-12-27 08:33:41,570][20347] Updated weights for policy 0, policy_version 422682 (0.0006) [2023-12-27 08:33:41,969][20381] Updated weights for policy 1, policy_version 422330 (0.0007) [2023-12-27 08:33:42,040][20381] Updated weights for policy 1, policy_version 422340 (0.0007) [2023-12-27 08:33:42,094][20381] Updated weights for policy 1, policy_version 422350 (0.0005) [2023-12-27 08:33:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 216367104. Throughput: 0: 10240.7, 1: 10664.3. Samples: 216359856. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:42,363][19170] Avg episode reward: [(0, '990.520'), (1, '865.320')] [2023-12-27 08:33:42,372][20347] Updated weights for policy 0, policy_version 422692 (0.0009) [2023-12-27 08:33:42,425][20347] Updated weights for policy 0, policy_version 422702 (0.0007) [2023-12-27 08:33:42,482][20347] Updated weights for policy 0, policy_version 422712 (0.0007) [2023-12-27 08:33:42,767][20381] Updated weights for policy 1, policy_version 422360 (0.0007) [2023-12-27 08:33:42,830][20381] Updated weights for policy 1, policy_version 422370 (0.0008) [2023-12-27 08:33:42,898][20381] Updated weights for policy 1, policy_version 422380 (0.0008) [2023-12-27 08:33:43,187][20347] Updated weights for policy 0, policy_version 422722 (0.0009) [2023-12-27 08:33:43,239][20347] Updated weights for policy 0, policy_version 422732 (0.0009) [2023-12-27 08:33:43,311][20347] Updated weights for policy 0, policy_version 422742 (0.0011) [2023-12-27 08:33:43,370][20347] Updated weights for policy 0, policy_version 422752 (0.0010) [2023-12-27 08:33:43,508][20381] Updated weights for policy 1, policy_version 422390 (0.0009) [2023-12-27 08:33:43,556][20381] Updated weights for policy 1, policy_version 422400 (0.0010) [2023-12-27 08:33:43,562][20292] KL-divergence is very high: 171.4574 [2023-12-27 08:33:43,610][20381] Updated weights for policy 1, policy_version 422410 (0.0010) [2023-12-27 08:33:43,993][20347] Updated weights for policy 0, policy_version 422762 (0.0010) [2023-12-27 08:33:44,039][20347] Updated weights for policy 0, policy_version 422772 (0.0008) [2023-12-27 08:33:44,096][20347] Updated weights for policy 0, policy_version 422782 (0.0006) [2023-12-27 08:33:44,296][20381] Updated weights for policy 1, policy_version 422420 (0.0010) [2023-12-27 08:33:44,353][20381] Updated weights for policy 1, policy_version 422430 (0.0007) [2023-12-27 08:33:44,418][20381] Updated weights for policy 1, policy_version 422440 (0.0009) [2023-12-27 08:33:44,681][20347] Updated weights for policy 0, policy_version 422792 (0.0008) [2023-12-27 08:33:44,732][20347] Updated weights for policy 0, policy_version 422802 (0.0005) [2023-12-27 08:33:44,796][20347] Updated weights for policy 0, policy_version 422812 (0.0007) [2023-12-27 08:33:45,095][20381] Updated weights for policy 1, policy_version 422450 (0.0007) [2023-12-27 08:33:45,154][20381] Updated weights for policy 1, policy_version 422460 (0.0012) [2023-12-27 08:33:45,206][20381] Updated weights for policy 1, policy_version 422470 (0.0010) [2023-12-27 08:33:45,258][20381] Updated weights for policy 1, policy_version 422480 (0.0010) [2023-12-27 08:33:45,499][20347] Updated weights for policy 0, policy_version 422822 (0.0008) [2023-12-27 08:33:45,553][20347] Updated weights for policy 0, policy_version 422832 (0.0008) [2023-12-27 08:33:45,604][20347] Updated weights for policy 0, policy_version 422842 (0.0007) [2023-12-27 08:33:45,927][20381] Updated weights for policy 1, policy_version 422490 (0.0011) [2023-12-27 08:33:45,974][20381] Updated weights for policy 1, policy_version 422500 (0.0010) [2023-12-27 08:33:46,022][20381] Updated weights for policy 1, policy_version 422510 (0.0007) [2023-12-27 08:33:46,319][20347] Updated weights for policy 0, policy_version 422852 (0.0008) [2023-12-27 08:33:46,370][20347] Updated weights for policy 0, policy_version 422862 (0.0005) [2023-12-27 08:33:46,415][20347] Updated weights for policy 0, policy_version 422872 (0.0005) [2023-12-27 08:33:46,691][20381] Updated weights for policy 1, policy_version 422520 (0.0006) [2023-12-27 08:33:46,750][20381] Updated weights for policy 1, policy_version 422530 (0.0006) [2023-12-27 08:33:46,807][20381] Updated weights for policy 1, policy_version 422540 (0.0005) [2023-12-27 08:33:47,109][20347] Updated weights for policy 0, policy_version 422882 (0.0006) [2023-12-27 08:33:47,168][20347] Updated weights for policy 0, policy_version 422892 (0.0010) [2023-12-27 08:33:47,226][20347] Updated weights for policy 0, policy_version 422902 (0.0009) [2023-12-27 08:33:47,280][20347] Updated weights for policy 0, policy_version 422912 (0.0010) [2023-12-27 08:33:47,363][20381] Updated weights for policy 1, policy_version 422550 (0.0008) [2023-12-27 08:33:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 216473600. Throughput: 0: 10177.1, 1: 10682.9. Samples: 216483316. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:47,363][19170] Avg episode reward: [(0, '971.330'), (1, '740.300')] [2023-12-27 08:33:47,408][20381] Updated weights for policy 1, policy_version 422560 (0.0010) [2023-12-27 08:33:47,463][20381] Updated weights for policy 1, policy_version 422570 (0.0010) [2023-12-27 08:33:47,863][20347] Updated weights for policy 0, policy_version 422922 (0.0007) [2023-12-27 08:33:47,919][20347] Updated weights for policy 0, policy_version 422932 (0.0007) [2023-12-27 08:33:47,972][20347] Updated weights for policy 0, policy_version 422942 (0.0009) [2023-12-27 08:33:48,124][20381] Updated weights for policy 1, policy_version 422580 (0.0009) [2023-12-27 08:33:48,177][20381] Updated weights for policy 1, policy_version 422590 (0.0009) [2023-12-27 08:33:48,237][20381] Updated weights for policy 1, policy_version 422600 (0.0008) [2023-12-27 08:33:48,644][20347] Updated weights for policy 0, policy_version 422952 (0.0006) [2023-12-27 08:33:48,699][20347] Updated weights for policy 0, policy_version 422962 (0.0006) [2023-12-27 08:33:48,756][20347] Updated weights for policy 0, policy_version 422972 (0.0006) [2023-12-27 08:33:48,979][20381] Updated weights for policy 1, policy_version 422610 (0.0008) [2023-12-27 08:33:49,023][20381] Updated weights for policy 1, policy_version 422620 (0.0009) [2023-12-27 08:33:49,077][20381] Updated weights for policy 1, policy_version 422630 (0.0010) [2023-12-27 08:33:49,124][20381] Updated weights for policy 1, policy_version 422640 (0.0008) [2023-12-27 08:33:49,329][20347] Updated weights for policy 0, policy_version 422982 (0.0007) [2023-12-27 08:33:49,395][20347] Updated weights for policy 0, policy_version 422992 (0.0007) [2023-12-27 08:33:49,455][20347] Updated weights for policy 0, policy_version 423002 (0.0007) [2023-12-27 08:33:49,911][20381] Updated weights for policy 1, policy_version 422650 (0.0008) [2023-12-27 08:33:49,973][20381] Updated weights for policy 1, policy_version 422660 (0.0007) [2023-12-27 08:33:50,029][20381] Updated weights for policy 1, policy_version 422670 (0.0006) [2023-12-27 08:33:50,168][20347] Updated weights for policy 0, policy_version 423012 (0.0006) [2023-12-27 08:33:50,233][20347] Updated weights for policy 0, policy_version 423022 (0.0006) [2023-12-27 08:33:50,295][20347] Updated weights for policy 0, policy_version 423032 (0.0005) [2023-12-27 08:33:50,685][20381] Updated weights for policy 1, policy_version 422680 (0.0007) [2023-12-27 08:33:50,733][20381] Updated weights for policy 1, policy_version 422690 (0.0007) [2023-12-27 08:33:50,781][20381] Updated weights for policy 1, policy_version 422700 (0.0007) [2023-12-27 08:33:50,927][20347] Updated weights for policy 0, policy_version 423042 (0.0009) [2023-12-27 08:33:50,989][20347] Updated weights for policy 0, policy_version 423052 (0.0009) [2023-12-27 08:33:51,058][20347] Updated weights for policy 0, policy_version 423062 (0.0008) [2023-12-27 08:33:51,113][20347] Updated weights for policy 0, policy_version 423072 (0.0007) [2023-12-27 08:33:51,478][20381] Updated weights for policy 1, policy_version 422710 (0.0008) [2023-12-27 08:33:51,532][20381] Updated weights for policy 1, policy_version 422720 (0.0007) [2023-12-27 08:33:51,587][20381] Updated weights for policy 1, policy_version 422730 (0.0007) [2023-12-27 08:33:51,765][20347] Updated weights for policy 0, policy_version 423082 (0.0008) [2023-12-27 08:33:51,823][20347] Updated weights for policy 0, policy_version 423092 (0.0008) [2023-12-27 08:33:51,881][20347] Updated weights for policy 0, policy_version 423102 (0.0008) [2023-12-27 08:33:52,280][20381] Updated weights for policy 1, policy_version 422740 (0.0008) [2023-12-27 08:33:52,336][20381] Updated weights for policy 1, policy_version 422750 (0.0009) [2023-12-27 08:33:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.6, 300 sec: 20743.8). Total num frames: 216571904. Throughput: 0: 10189.1, 1: 10651.1. Samples: 216545372. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:52,363][19170] Avg episode reward: [(0, '971.310'), (1, '690.580')] [2023-12-27 08:33:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000423104_108331008.pth... [2023-12-27 08:33:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000421920_108027904.pth [2023-12-27 08:33:52,400][20381] Updated weights for policy 1, policy_version 422760 (0.0008) [2023-12-27 08:33:52,446][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000422768_108249088.pth... [2023-12-27 08:33:52,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000421488_107921408.pth [2023-12-27 08:33:52,604][20347] Updated weights for policy 0, policy_version 423112 (0.0008) [2023-12-27 08:33:52,668][20347] Updated weights for policy 0, policy_version 423122 (0.0007) [2023-12-27 08:33:52,729][20347] Updated weights for policy 0, policy_version 423132 (0.0008) [2023-12-27 08:33:53,070][20381] Updated weights for policy 1, policy_version 422770 (0.0007) [2023-12-27 08:33:53,124][20381] Updated weights for policy 1, policy_version 422780 (0.0006) [2023-12-27 08:33:53,180][20381] Updated weights for policy 1, policy_version 422790 (0.0005) [2023-12-27 08:33:53,238][20381] Updated weights for policy 1, policy_version 422800 (0.0005) [2023-12-27 08:33:53,449][20347] Updated weights for policy 0, policy_version 423142 (0.0010) [2023-12-27 08:33:53,510][20347] Updated weights for policy 0, policy_version 423152 (0.0010) [2023-12-27 08:33:53,558][20347] Updated weights for policy 0, policy_version 423162 (0.0008) [2023-12-27 08:33:53,845][20381] Updated weights for policy 1, policy_version 422810 (0.0007) [2023-12-27 08:33:53,899][20381] Updated weights for policy 1, policy_version 422820 (0.0008) [2023-12-27 08:33:53,950][20381] Updated weights for policy 1, policy_version 422830 (0.0010) [2023-12-27 08:33:54,175][20347] Updated weights for policy 0, policy_version 423172 (0.0008) [2023-12-27 08:33:54,240][20347] Updated weights for policy 0, policy_version 423182 (0.0010) [2023-12-27 08:33:54,299][20347] Updated weights for policy 0, policy_version 423192 (0.0006) [2023-12-27 08:33:54,688][20381] Updated weights for policy 1, policy_version 422840 (0.0009) [2023-12-27 08:33:54,748][20381] Updated weights for policy 1, policy_version 422850 (0.0008) [2023-12-27 08:33:54,797][20381] Updated weights for policy 1, policy_version 422860 (0.0008) [2023-12-27 08:33:54,882][20347] Updated weights for policy 0, policy_version 423202 (0.0006) [2023-12-27 08:33:54,930][20347] Updated weights for policy 0, policy_version 423212 (0.0010) [2023-12-27 08:33:54,983][20347] Updated weights for policy 0, policy_version 423222 (0.0010) [2023-12-27 08:33:55,045][20347] Updated weights for policy 0, policy_version 423232 (0.0010) [2023-12-27 08:33:55,548][20381] Updated weights for policy 1, policy_version 422870 (0.0008) [2023-12-27 08:33:55,602][20381] Updated weights for policy 1, policy_version 422880 (0.0007) [2023-12-27 08:33:55,661][20381] Updated weights for policy 1, policy_version 422890 (0.0008) [2023-12-27 08:33:55,695][20347] Updated weights for policy 0, policy_version 423242 (0.0010) [2023-12-27 08:33:55,740][20347] Updated weights for policy 0, policy_version 423252 (0.0010) [2023-12-27 08:33:55,794][20347] Updated weights for policy 0, policy_version 423262 (0.0010) [2023-12-27 08:33:56,340][20381] Updated weights for policy 1, policy_version 422900 (0.0009) [2023-12-27 08:33:56,388][20381] Updated weights for policy 1, policy_version 422910 (0.0007) [2023-12-27 08:33:56,437][20381] Updated weights for policy 1, policy_version 422920 (0.0007) [2023-12-27 08:33:56,443][20347] Updated weights for policy 0, policy_version 423272 (0.0010) [2023-12-27 08:33:56,494][20347] Updated weights for policy 0, policy_version 423282 (0.0010) [2023-12-27 08:33:56,542][20347] Updated weights for policy 0, policy_version 423292 (0.0010) [2023-12-27 08:33:57,149][20381] Updated weights for policy 1, policy_version 422930 (0.0006) [2023-12-27 08:33:57,197][20381] Updated weights for policy 1, policy_version 422940 (0.0005) [2023-12-27 08:33:57,199][20292] KL-divergence is very high: 340.6993 [2023-12-27 08:33:57,228][20347] Updated weights for policy 0, policy_version 423302 (0.0010) [2023-12-27 08:33:57,254][20381] Updated weights for policy 1, policy_version 422950 (0.0010) [2023-12-27 08:33:57,285][20347] Updated weights for policy 0, policy_version 423312 (0.0010) [2023-12-27 08:33:57,303][20381] Updated weights for policy 1, policy_version 422960 (0.0009) [2023-12-27 08:33:57,342][20347] Updated weights for policy 0, policy_version 423322 (0.0008) [2023-12-27 08:33:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 216678400. Throughput: 0: 10222.2, 1: 10604.7. Samples: 216668404. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:33:57,363][19170] Avg episode reward: [(0, '990.500'), (1, '529.690')] [2023-12-27 08:33:57,919][20381] Updated weights for policy 1, policy_version 422970 (0.0008) [2023-12-27 08:33:57,971][20381] Updated weights for policy 1, policy_version 422980 (0.0007) [2023-12-27 08:33:57,996][20347] Updated weights for policy 0, policy_version 423332 (0.0010) [2023-12-27 08:33:58,016][20381] Updated weights for policy 1, policy_version 422990 (0.0006) [2023-12-27 08:33:58,047][20347] Updated weights for policy 0, policy_version 423342 (0.0010) [2023-12-27 08:33:58,107][20347] Updated weights for policy 0, policy_version 423352 (0.0010) [2023-12-27 08:33:58,788][20381] Updated weights for policy 1, policy_version 423000 (0.0007) [2023-12-27 08:33:58,833][20347] Updated weights for policy 0, policy_version 423362 (0.0009) [2023-12-27 08:33:58,851][20381] Updated weights for policy 1, policy_version 423010 (0.0008) [2023-12-27 08:33:58,898][20347] Updated weights for policy 0, policy_version 423372 (0.0008) [2023-12-27 08:33:58,917][20381] Updated weights for policy 1, policy_version 423020 (0.0007) [2023-12-27 08:33:58,969][20347] Updated weights for policy 0, policy_version 423382 (0.0007) [2023-12-27 08:33:59,031][20347] Updated weights for policy 0, policy_version 423392 (0.0005) [2023-12-27 08:33:59,612][20381] Updated weights for policy 1, policy_version 423030 (0.0006) [2023-12-27 08:33:59,669][20381] Updated weights for policy 1, policy_version 423040 (0.0006) [2023-12-27 08:33:59,689][20347] Updated weights for policy 0, policy_version 423402 (0.0008) [2023-12-27 08:33:59,726][20381] Updated weights for policy 1, policy_version 423050 (0.0005) [2023-12-27 08:33:59,747][20347] Updated weights for policy 0, policy_version 423412 (0.0008) [2023-12-27 08:33:59,804][20347] Updated weights for policy 0, policy_version 423422 (0.0008) [2023-12-27 08:34:00,392][20347] Updated weights for policy 0, policy_version 423432 (0.0006) [2023-12-27 08:34:00,408][20381] Updated weights for policy 1, policy_version 423060 (0.0006) [2023-12-27 08:34:00,447][20347] Updated weights for policy 0, policy_version 423442 (0.0006) [2023-12-27 08:34:00,462][20381] Updated weights for policy 1, policy_version 423070 (0.0006) [2023-12-27 08:34:00,507][20347] Updated weights for policy 0, policy_version 423452 (0.0006) [2023-12-27 08:34:00,509][20381] Updated weights for policy 1, policy_version 423080 (0.0007) [2023-12-27 08:34:01,090][20347] Updated weights for policy 0, policy_version 423462 (0.0008) [2023-12-27 08:34:01,157][20347] Updated weights for policy 0, policy_version 423472 (0.0008) [2023-12-27 08:34:01,209][20347] Updated weights for policy 0, policy_version 423482 (0.0007) [2023-12-27 08:34:01,214][20381] Updated weights for policy 1, policy_version 423090 (0.0007) [2023-12-27 08:34:01,272][20381] Updated weights for policy 1, policy_version 423100 (0.0011) [2023-12-27 08:34:01,325][20381] Updated weights for policy 1, policy_version 423110 (0.0010) [2023-12-27 08:34:01,390][20381] Updated weights for policy 1, policy_version 423120 (0.0009) [2023-12-27 08:34:01,950][20347] Updated weights for policy 0, policy_version 423492 (0.0007) [2023-12-27 08:34:02,003][20347] Updated weights for policy 0, policy_version 423502 (0.0007) [2023-12-27 08:34:02,061][20347] Updated weights for policy 0, policy_version 423512 (0.0005) [2023-12-27 08:34:02,085][20381] Updated weights for policy 1, policy_version 423130 (0.0011) [2023-12-27 08:34:02,147][20381] Updated weights for policy 1, policy_version 423140 (0.0011) [2023-12-27 08:34:02,213][20381] Updated weights for policy 1, policy_version 423150 (0.0010) [2023-12-27 08:34:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 216784896. Throughput: 0: 10259.7, 1: 10510.9. Samples: 216790996. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:34:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '620.400')] [2023-12-27 08:34:02,663][20347] Updated weights for policy 0, policy_version 423522 (0.0006) [2023-12-27 08:34:02,722][20347] Updated weights for policy 0, policy_version 423532 (0.0009) [2023-12-27 08:34:02,767][20347] Updated weights for policy 0, policy_version 423542 (0.0010) [2023-12-27 08:34:02,820][20347] Updated weights for policy 0, policy_version 423552 (0.0007) [2023-12-27 08:34:02,880][20292] KL-divergence is very high: 103.5688 [2023-12-27 08:34:02,888][20381] Updated weights for policy 1, policy_version 423160 (0.0011) [2023-12-27 08:34:02,946][20381] Updated weights for policy 1, policy_version 423170 (0.0011) [2023-12-27 08:34:03,002][20381] Updated weights for policy 1, policy_version 423180 (0.0010) [2023-12-27 08:34:03,408][20347] Updated weights for policy 0, policy_version 423562 (0.0006) [2023-12-27 08:34:03,460][20347] Updated weights for policy 0, policy_version 423572 (0.0006) [2023-12-27 08:34:03,515][20347] Updated weights for policy 0, policy_version 423582 (0.0005) [2023-12-27 08:34:03,699][20381] Updated weights for policy 1, policy_version 423190 (0.0009) [2023-12-27 08:34:03,758][20381] Updated weights for policy 1, policy_version 423200 (0.0005) [2023-12-27 08:34:03,813][20381] Updated weights for policy 1, policy_version 423210 (0.0005) [2023-12-27 08:34:04,056][20347] Updated weights for policy 0, policy_version 423592 (0.0005) [2023-12-27 08:34:04,119][20347] Updated weights for policy 0, policy_version 423602 (0.0005) [2023-12-27 08:34:04,176][20347] Updated weights for policy 0, policy_version 423612 (0.0009) [2023-12-27 08:34:04,372][20381] Updated weights for policy 1, policy_version 423220 (0.0007) [2023-12-27 08:34:04,436][20381] Updated weights for policy 1, policy_version 423230 (0.0010) [2023-12-27 08:34:04,495][20381] Updated weights for policy 1, policy_version 423240 (0.0010) [2023-12-27 08:34:04,751][20347] Updated weights for policy 0, policy_version 423622 (0.0007) [2023-12-27 08:34:04,806][20347] Updated weights for policy 0, policy_version 423632 (0.0008) [2023-12-27 08:34:04,863][20347] Updated weights for policy 0, policy_version 423642 (0.0007) [2023-12-27 08:34:05,099][20381] Updated weights for policy 1, policy_version 423250 (0.0008) [2023-12-27 08:34:05,158][20381] Updated weights for policy 1, policy_version 423260 (0.0008) [2023-12-27 08:34:05,212][20381] Updated weights for policy 1, policy_version 423270 (0.0008) [2023-12-27 08:34:05,258][20381] Updated weights for policy 1, policy_version 423280 (0.0008) [2023-12-27 08:34:05,532][20347] Updated weights for policy 0, policy_version 423652 (0.0007) [2023-12-27 08:34:05,580][20347] Updated weights for policy 0, policy_version 423662 (0.0006) [2023-12-27 08:34:05,628][20347] Updated weights for policy 0, policy_version 423672 (0.0008) [2023-12-27 08:34:05,849][20381] Updated weights for policy 1, policy_version 423290 (0.0005) [2023-12-27 08:34:05,895][20381] Updated weights for policy 1, policy_version 423300 (0.0006) [2023-12-27 08:34:05,940][20381] Updated weights for policy 1, policy_version 423310 (0.0007) [2023-12-27 08:34:06,358][20347] Updated weights for policy 0, policy_version 423682 (0.0009) [2023-12-27 08:34:06,423][20347] Updated weights for policy 0, policy_version 423692 (0.0008) [2023-12-27 08:34:06,492][20347] Updated weights for policy 0, policy_version 423702 (0.0008) [2023-12-27 08:34:06,560][20347] Updated weights for policy 0, policy_version 423712 (0.0007) [2023-12-27 08:34:06,612][20381] Updated weights for policy 1, policy_version 423320 (0.0006) [2023-12-27 08:34:06,670][20381] Updated weights for policy 1, policy_version 423330 (0.0006) [2023-12-27 08:34:06,733][20381] Updated weights for policy 1, policy_version 423340 (0.0006) [2023-12-27 08:34:07,259][20347] Updated weights for policy 0, policy_version 423722 (0.0008) [2023-12-27 08:34:07,321][20347] Updated weights for policy 0, policy_version 423733 (0.0007) [2023-12-27 08:34:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 216883200. Throughput: 0: 10349.1, 1: 10525.6. Samples: 216857940. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:34:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '612.140')] [2023-12-27 08:34:07,379][20347] Updated weights for policy 0, policy_version 423743 (0.0005) [2023-12-27 08:34:07,383][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000423744_108494848.pth... [2023-12-27 08:34:07,387][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000422528_108183552.pth [2023-12-27 08:34:07,388][20381] Updated weights for policy 1, policy_version 423350 (0.0009) [2023-12-27 08:34:07,440][20381] Updated weights for policy 1, policy_version 423360 (0.0011) [2023-12-27 08:34:07,489][20381] Updated weights for policy 1, policy_version 423370 (0.0010) [2023-12-27 08:34:07,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000423376_108404736.pth... [2023-12-27 08:34:07,528][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000422160_108093440.pth [2023-12-27 08:34:08,019][20347] Updated weights for policy 0, policy_version 423753 (0.0007) [2023-12-27 08:34:08,074][20347] Updated weights for policy 0, policy_version 423763 (0.0009) [2023-12-27 08:34:08,094][20381] Updated weights for policy 1, policy_version 423380 (0.0007) [2023-12-27 08:34:08,124][20347] Updated weights for policy 0, policy_version 423773 (0.0007) [2023-12-27 08:34:08,152][20381] Updated weights for policy 1, policy_version 423390 (0.0009) [2023-12-27 08:34:08,209][20381] Updated weights for policy 1, policy_version 423400 (0.0005) [2023-12-27 08:34:08,800][20347] Updated weights for policy 0, policy_version 423783 (0.0008) [2023-12-27 08:34:08,855][20347] Updated weights for policy 0, policy_version 423793 (0.0008) [2023-12-27 08:34:08,871][20381] Updated weights for policy 1, policy_version 423410 (0.0006) [2023-12-27 08:34:08,910][20347] Updated weights for policy 0, policy_version 423803 (0.0007) [2023-12-27 08:34:08,921][20381] Updated weights for policy 1, policy_version 423420 (0.0006) [2023-12-27 08:34:08,966][20381] Updated weights for policy 1, policy_version 423430 (0.0007) [2023-12-27 08:34:09,014][20381] Updated weights for policy 1, policy_version 423440 (0.0007) [2023-12-27 08:34:09,574][20347] Updated weights for policy 0, policy_version 423813 (0.0006) [2023-12-27 08:34:09,640][20347] Updated weights for policy 0, policy_version 423823 (0.0006) [2023-12-27 08:34:09,707][20347] Updated weights for policy 0, policy_version 423833 (0.0008) [2023-12-27 08:34:09,708][20381] Updated weights for policy 1, policy_version 423450 (0.0008) [2023-12-27 08:34:09,765][20381] Updated weights for policy 1, policy_version 423460 (0.0007) [2023-12-27 08:34:09,825][20381] Updated weights for policy 1, policy_version 423470 (0.0007) [2023-12-27 08:34:10,312][20347] Updated weights for policy 0, policy_version 423843 (0.0010) [2023-12-27 08:34:10,384][20347] Updated weights for policy 0, policy_version 423853 (0.0008) [2023-12-27 08:34:10,436][20347] Updated weights for policy 0, policy_version 423863 (0.0008) [2023-12-27 08:34:10,534][20381] Updated weights for policy 1, policy_version 423480 (0.0007) [2023-12-27 08:34:10,596][20381] Updated weights for policy 1, policy_version 423490 (0.0007) [2023-12-27 08:34:10,641][20381] Updated weights for policy 1, policy_version 423500 (0.0007) [2023-12-27 08:34:11,100][20347] Updated weights for policy 0, policy_version 423873 (0.0007) [2023-12-27 08:34:11,168][20347] Updated weights for policy 0, policy_version 423883 (0.0008) [2023-12-27 08:34:11,236][20347] Updated weights for policy 0, policy_version 423893 (0.0006) [2023-12-27 08:34:11,300][20347] Updated weights for policy 0, policy_version 423903 (0.0008) [2023-12-27 08:34:11,393][20381] Updated weights for policy 1, policy_version 423510 (0.0008) [2023-12-27 08:34:11,451][20381] Updated weights for policy 1, policy_version 423520 (0.0008) [2023-12-27 08:34:11,512][20381] Updated weights for policy 1, policy_version 423530 (0.0006) [2023-12-27 08:34:12,004][20347] Updated weights for policy 0, policy_version 423913 (0.0008) [2023-12-27 08:34:12,067][20347] Updated weights for policy 0, policy_version 423923 (0.0008) [2023-12-27 08:34:12,123][20347] Updated weights for policy 0, policy_version 423933 (0.0008) [2023-12-27 08:34:12,152][20381] Updated weights for policy 1, policy_version 423540 (0.0006) [2023-12-27 08:34:12,212][20381] Updated weights for policy 1, policy_version 423550 (0.0008) [2023-12-27 08:34:12,271][20381] Updated weights for policy 1, policy_version 423560 (0.0008) [2023-12-27 08:34:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 216997888. Throughput: 0: 10321.4, 1: 10533.1. Samples: 216983212. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:34:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '747.550')] [2023-12-27 08:34:12,780][20347] Updated weights for policy 0, policy_version 423943 (0.0007) [2023-12-27 08:34:12,840][20347] Updated weights for policy 0, policy_version 423953 (0.0008) [2023-12-27 08:34:12,901][20347] Updated weights for policy 0, policy_version 423963 (0.0007) [2023-12-27 08:34:12,993][20381] Updated weights for policy 1, policy_version 423570 (0.0008) [2023-12-27 08:34:13,049][20381] Updated weights for policy 1, policy_version 423580 (0.0007) [2023-12-27 08:34:13,121][20381] Updated weights for policy 1, policy_version 423590 (0.0007) [2023-12-27 08:34:13,176][20381] Updated weights for policy 1, policy_version 423600 (0.0008) [2023-12-27 08:34:13,525][20347] Updated weights for policy 0, policy_version 423973 (0.0007) [2023-12-27 08:34:13,592][20347] Updated weights for policy 0, policy_version 423983 (0.0006) [2023-12-27 08:34:13,648][20347] Updated weights for policy 0, policy_version 423993 (0.0008) [2023-12-27 08:34:13,842][20381] Updated weights for policy 1, policy_version 423610 (0.0005) [2023-12-27 08:34:13,899][20381] Updated weights for policy 1, policy_version 423620 (0.0005) [2023-12-27 08:34:13,967][20381] Updated weights for policy 1, policy_version 423630 (0.0005) [2023-12-27 08:34:14,279][20347] Updated weights for policy 0, policy_version 424003 (0.0007) [2023-12-27 08:34:14,329][20347] Updated weights for policy 0, policy_version 424013 (0.0007) [2023-12-27 08:34:14,384][20347] Updated weights for policy 0, policy_version 424023 (0.0007) [2023-12-27 08:34:14,589][20381] Updated weights for policy 1, policy_version 423640 (0.0008) [2023-12-27 08:34:14,631][20381] Updated weights for policy 1, policy_version 423650 (0.0007) [2023-12-27 08:34:14,675][20381] Updated weights for policy 1, policy_version 423660 (0.0007) [2023-12-27 08:34:15,058][20347] Updated weights for policy 0, policy_version 424033 (0.0006) [2023-12-27 08:34:15,112][20347] Updated weights for policy 0, policy_version 424043 (0.0005) [2023-12-27 08:34:15,179][20347] Updated weights for policy 0, policy_version 424053 (0.0008) [2023-12-27 08:34:15,250][20347] Updated weights for policy 0, policy_version 424063 (0.0008) [2023-12-27 08:34:15,444][20381] Updated weights for policy 1, policy_version 423670 (0.0008) [2023-12-27 08:34:15,506][20381] Updated weights for policy 1, policy_version 423680 (0.0007) [2023-12-27 08:34:15,573][20381] Updated weights for policy 1, policy_version 423690 (0.0008) [2023-12-27 08:34:15,864][20347] Updated weights for policy 0, policy_version 424073 (0.0006) [2023-12-27 08:34:15,924][20347] Updated weights for policy 0, policy_version 424083 (0.0005) [2023-12-27 08:34:15,987][20347] Updated weights for policy 0, policy_version 424093 (0.0005) [2023-12-27 08:34:16,299][20381] Updated weights for policy 1, policy_version 423700 (0.0007) [2023-12-27 08:34:16,344][20381] Updated weights for policy 1, policy_version 423710 (0.0008) [2023-12-27 08:34:16,396][20381] Updated weights for policy 1, policy_version 423720 (0.0008) [2023-12-27 08:34:16,526][20347] Updated weights for policy 0, policy_version 424103 (0.0006) [2023-12-27 08:34:16,580][20347] Updated weights for policy 0, policy_version 424113 (0.0007) [2023-12-27 08:34:16,632][20347] Updated weights for policy 0, policy_version 424123 (0.0008) [2023-12-27 08:34:17,138][20381] Updated weights for policy 1, policy_version 423730 (0.0008) [2023-12-27 08:34:17,193][20381] Updated weights for policy 1, policy_version 423740 (0.0011) [2023-12-27 08:34:17,225][20347] Updated weights for policy 0, policy_version 424133 (0.0007) [2023-12-27 08:34:17,239][20381] Updated weights for policy 1, policy_version 423750 (0.0010) [2023-12-27 08:34:17,276][20347] Updated weights for policy 0, policy_version 424143 (0.0005) [2023-12-27 08:34:17,294][20381] Updated weights for policy 1, policy_version 423760 (0.0011) [2023-12-27 08:34:17,329][20347] Updated weights for policy 0, policy_version 424153 (0.0005) [2023-12-27 08:34:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 217096192. Throughput: 0: 10392.0, 1: 10326.7. Samples: 217106420. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:34:17,363][19170] Avg episode reward: [(0, '990.400'), (1, '776.850')] [2023-12-27 08:34:17,936][20347] Updated weights for policy 0, policy_version 424163 (0.0007) [2023-12-27 08:34:17,982][20381] Updated weights for policy 1, policy_version 423770 (0.0011) [2023-12-27 08:34:17,986][20347] Updated weights for policy 0, policy_version 424173 (0.0010) [2023-12-27 08:34:18,034][20381] Updated weights for policy 1, policy_version 423780 (0.0011) [2023-12-27 08:34:18,042][20347] Updated weights for policy 0, policy_version 424183 (0.0010) [2023-12-27 08:34:18,086][20381] Updated weights for policy 1, policy_version 423790 (0.0010) [2023-12-27 08:34:18,744][20347] Updated weights for policy 0, policy_version 424193 (0.0010) [2023-12-27 08:34:18,763][20381] Updated weights for policy 1, policy_version 423800 (0.0008) [2023-12-27 08:34:18,797][20347] Updated weights for policy 0, policy_version 424203 (0.0009) [2023-12-27 08:34:18,815][20381] Updated weights for policy 1, policy_version 423810 (0.0008) [2023-12-27 08:34:18,856][20347] Updated weights for policy 0, policy_version 424213 (0.0007) [2023-12-27 08:34:18,867][20381] Updated weights for policy 1, policy_version 423820 (0.0007) [2023-12-27 08:34:18,912][20347] Updated weights for policy 0, policy_version 424223 (0.0007) [2023-12-27 08:34:19,559][20381] Updated weights for policy 1, policy_version 423830 (0.0007) [2023-12-27 08:34:19,591][20347] Updated weights for policy 0, policy_version 424233 (0.0010) [2023-12-27 08:34:19,619][20381] Updated weights for policy 1, policy_version 423840 (0.0010) [2023-12-27 08:34:19,650][20347] Updated weights for policy 0, policy_version 424243 (0.0010) [2023-12-27 08:34:19,671][20381] Updated weights for policy 1, policy_version 423850 (0.0008) [2023-12-27 08:34:19,716][20347] Updated weights for policy 0, policy_version 424253 (0.0010) [2023-12-27 08:34:20,338][20381] Updated weights for policy 1, policy_version 423860 (0.0006) [2023-12-27 08:34:20,393][20347] Updated weights for policy 0, policy_version 424263 (0.0007) [2023-12-27 08:34:20,395][20381] Updated weights for policy 1, policy_version 423870 (0.0007) [2023-12-27 08:34:20,452][20381] Updated weights for policy 1, policy_version 423880 (0.0005) [2023-12-27 08:34:20,454][20347] Updated weights for policy 0, policy_version 424273 (0.0006) [2023-12-27 08:34:20,513][20347] Updated weights for policy 0, policy_version 424283 (0.0006) [2023-12-27 08:34:21,094][20381] Updated weights for policy 1, policy_version 423890 (0.0006) [2023-12-27 08:34:21,159][20381] Updated weights for policy 1, policy_version 423900 (0.0009) [2023-12-27 08:34:21,219][20347] Updated weights for policy 0, policy_version 424293 (0.0008) [2023-12-27 08:34:21,221][20381] Updated weights for policy 1, policy_version 423910 (0.0007) [2023-12-27 08:34:21,280][20347] Updated weights for policy 0, policy_version 424303 (0.0007) [2023-12-27 08:34:21,282][20381] Updated weights for policy 1, policy_version 423920 (0.0008) [2023-12-27 08:34:21,340][20347] Updated weights for policy 0, policy_version 424313 (0.0008) [2023-12-27 08:34:22,034][20381] Updated weights for policy 1, policy_version 423930 (0.0006) [2023-12-27 08:34:22,085][20347] Updated weights for policy 0, policy_version 424323 (0.0008) [2023-12-27 08:34:22,092][20381] Updated weights for policy 1, policy_version 423940 (0.0006) [2023-12-27 08:34:22,142][20347] Updated weights for policy 0, policy_version 424333 (0.0009) [2023-12-27 08:34:22,146][20381] Updated weights for policy 1, policy_version 423950 (0.0007) [2023-12-27 08:34:22,198][20347] Updated weights for policy 0, policy_version 424343 (0.0008) [2023-12-27 08:34:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 217202688. Throughput: 0: 10403.5, 1: 10287.4. Samples: 217168300. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 08:34:22,363][19170] Avg episode reward: [(0, '990.400'), (1, '700.320')] [2023-12-27 08:34:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000423952_108552192.pth... [2023-12-27 08:34:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000424352_108650496.pth... [2023-12-27 08:34:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000422768_108249088.pth [2023-12-27 08:34:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000423104_108331008.pth [2023-12-27 08:34:22,811][20381] Updated weights for policy 1, policy_version 423960 (0.0010) [2023-12-27 08:34:22,850][20347] Updated weights for policy 0, policy_version 424353 (0.0008) [2023-12-27 08:34:22,869][20381] Updated weights for policy 1, policy_version 423970 (0.0009) [2023-12-27 08:34:22,912][20347] Updated weights for policy 0, policy_version 424363 (0.0005) [2023-12-27 08:34:22,918][20381] Updated weights for policy 1, policy_version 423980 (0.0006) [2023-12-27 08:34:22,962][20347] Updated weights for policy 0, policy_version 424373 (0.0006) [2023-12-27 08:34:23,009][20347] Updated weights for policy 0, policy_version 424383 (0.0005) [2023-12-27 08:34:23,548][20347] Updated weights for policy 0, policy_version 424393 (0.0005) [2023-12-27 08:34:23,594][20381] Updated weights for policy 1, policy_version 423990 (0.0009) [2023-12-27 08:34:23,600][20347] Updated weights for policy 0, policy_version 424403 (0.0005) [2023-12-27 08:34:23,642][20381] Updated weights for policy 1, policy_version 424000 (0.0010) [2023-12-27 08:34:23,664][20347] Updated weights for policy 0, policy_version 424413 (0.0006) [2023-12-27 08:34:23,697][20381] Updated weights for policy 1, policy_version 424010 (0.0010) [2023-12-27 08:34:24,316][20347] Updated weights for policy 0, policy_version 424423 (0.0007) [2023-12-27 08:34:24,353][20381] Updated weights for policy 1, policy_version 424020 (0.0007) [2023-12-27 08:34:24,368][20347] Updated weights for policy 0, policy_version 424433 (0.0007) [2023-12-27 08:34:24,404][20381] Updated weights for policy 1, policy_version 424030 (0.0010) [2023-12-27 08:34:24,423][20347] Updated weights for policy 0, policy_version 424443 (0.0007) [2023-12-27 08:34:24,461][20381] Updated weights for policy 1, policy_version 424040 (0.0010) [2023-12-27 08:34:25,094][20347] Updated weights for policy 0, policy_version 424453 (0.0007) [2023-12-27 08:34:25,115][20381] Updated weights for policy 1, policy_version 424050 (0.0008) [2023-12-27 08:34:25,143][20347] Updated weights for policy 0, policy_version 424463 (0.0007) [2023-12-27 08:34:25,167][20381] Updated weights for policy 1, policy_version 424060 (0.0005) [2023-12-27 08:34:25,203][20347] Updated weights for policy 0, policy_version 424473 (0.0007) [2023-12-27 08:34:25,224][20381] Updated weights for policy 1, policy_version 424070 (0.0006) [2023-12-27 08:34:25,283][20381] Updated weights for policy 1, policy_version 424080 (0.0007) [2023-12-27 08:34:25,854][20347] Updated weights for policy 0, policy_version 424483 (0.0006) [2023-12-27 08:34:25,911][20347] Updated weights for policy 0, policy_version 424493 (0.0007) [2023-12-27 08:34:25,942][20381] Updated weights for policy 1, policy_version 424090 (0.0006) [2023-12-27 08:34:25,960][20347] Updated weights for policy 0, policy_version 424503 (0.0006) [2023-12-27 08:34:25,989][20381] Updated weights for policy 1, policy_version 424100 (0.0005) [2023-12-27 08:34:26,040][20381] Updated weights for policy 1, policy_version 424110 (0.0006) [2023-12-27 08:34:26,615][20381] Updated weights for policy 1, policy_version 424120 (0.0008) [2023-12-27 08:34:26,667][20347] Updated weights for policy 0, policy_version 424513 (0.0006) [2023-12-27 08:34:26,671][20381] Updated weights for policy 1, policy_version 424130 (0.0008) [2023-12-27 08:34:26,712][20347] Updated weights for policy 0, policy_version 424523 (0.0010) [2023-12-27 08:34:26,724][20381] Updated weights for policy 1, policy_version 424140 (0.0009) [2023-12-27 08:34:26,763][20347] Updated weights for policy 0, policy_version 424533 (0.0010) [2023-12-27 08:34:26,814][20347] Updated weights for policy 0, policy_version 424543 (0.0010) [2023-12-27 08:34:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 217300992. Throughput: 0: 10495.3, 1: 10246.2. Samples: 217293224. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:27,363][19170] Avg episode reward: [(0, '990.400'), (1, '688.650')] [2023-12-27 08:34:27,409][20381] Updated weights for policy 1, policy_version 424150 (0.0006) [2023-12-27 08:34:27,461][20381] Updated weights for policy 1, policy_version 424160 (0.0005) [2023-12-27 08:34:27,498][20347] Updated weights for policy 0, policy_version 424553 (0.0007) [2023-12-27 08:34:27,523][20381] Updated weights for policy 1, policy_version 424170 (0.0007) [2023-12-27 08:34:27,568][20347] Updated weights for policy 0, policy_version 424563 (0.0006) [2023-12-27 08:34:27,617][20347] Updated weights for policy 0, policy_version 424573 (0.0008) [2023-12-27 08:34:28,048][20381] Updated weights for policy 1, policy_version 424180 (0.0006) [2023-12-27 08:34:28,102][20381] Updated weights for policy 1, policy_version 424190 (0.0005) [2023-12-27 08:34:28,154][20381] Updated weights for policy 1, policy_version 424200 (0.0005) [2023-12-27 08:34:28,243][20347] Updated weights for policy 0, policy_version 424583 (0.0007) [2023-12-27 08:34:28,299][20347] Updated weights for policy 0, policy_version 424593 (0.0005) [2023-12-27 08:34:28,357][20347] Updated weights for policy 0, policy_version 424603 (0.0006) [2023-12-27 08:34:28,740][20381] Updated weights for policy 1, policy_version 424210 (0.0006) [2023-12-27 08:34:28,796][20381] Updated weights for policy 1, policy_version 424220 (0.0007) [2023-12-27 08:34:28,854][20381] Updated weights for policy 1, policy_version 424230 (0.0007) [2023-12-27 08:34:28,913][20381] Updated weights for policy 1, policy_version 424240 (0.0007) [2023-12-27 08:34:29,043][20347] Updated weights for policy 0, policy_version 424613 (0.0007) [2023-12-27 08:34:29,095][20347] Updated weights for policy 0, policy_version 424623 (0.0008) [2023-12-27 08:34:29,147][20347] Updated weights for policy 0, policy_version 424633 (0.0008) [2023-12-27 08:34:29,499][20381] Updated weights for policy 1, policy_version 424250 (0.0007) [2023-12-27 08:34:29,555][20381] Updated weights for policy 1, policy_version 424260 (0.0008) [2023-12-27 08:34:29,607][20381] Updated weights for policy 1, policy_version 424270 (0.0008) [2023-12-27 08:34:29,838][20347] Updated weights for policy 0, policy_version 424643 (0.0008) [2023-12-27 08:34:29,896][20347] Updated weights for policy 0, policy_version 424653 (0.0006) [2023-12-27 08:34:29,958][20347] Updated weights for policy 0, policy_version 424663 (0.0007) [2023-12-27 08:34:30,309][20381] Updated weights for policy 1, policy_version 424280 (0.0006) [2023-12-27 08:34:30,362][20381] Updated weights for policy 1, policy_version 424290 (0.0005) [2023-12-27 08:34:30,419][20381] Updated weights for policy 1, policy_version 424300 (0.0008) [2023-12-27 08:34:30,644][20347] Updated weights for policy 0, policy_version 424673 (0.0006) [2023-12-27 08:34:30,692][20347] Updated weights for policy 0, policy_version 424683 (0.0007) [2023-12-27 08:34:30,737][20347] Updated weights for policy 0, policy_version 424693 (0.0008) [2023-12-27 08:34:30,784][20347] Updated weights for policy 0, policy_version 424703 (0.0007) [2023-12-27 08:34:30,980][20381] Updated weights for policy 1, policy_version 424310 (0.0006) [2023-12-27 08:34:31,032][20381] Updated weights for policy 1, policy_version 424320 (0.0006) [2023-12-27 08:34:31,091][20381] Updated weights for policy 1, policy_version 424330 (0.0008) [2023-12-27 08:34:31,616][20347] Updated weights for policy 0, policy_version 424713 (0.0008) [2023-12-27 08:34:31,681][20347] Updated weights for policy 0, policy_version 424723 (0.0008) [2023-12-27 08:34:31,747][20347] Updated weights for policy 0, policy_version 424733 (0.0008) [2023-12-27 08:34:31,937][20381] Updated weights for policy 1, policy_version 424340 (0.0008) [2023-12-27 08:34:32,002][20381] Updated weights for policy 1, policy_version 424350 (0.0008) [2023-12-27 08:34:32,068][20381] Updated weights for policy 1, policy_version 424360 (0.0008) [2023-12-27 08:34:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 217407488. Throughput: 0: 10471.9, 1: 10285.2. Samples: 217417384. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '721.730')] [2023-12-27 08:34:32,697][20347] Updated weights for policy 0, policy_version 424743 (0.0008) [2023-12-27 08:34:32,770][20347] Updated weights for policy 0, policy_version 424753 (0.0008) [2023-12-27 08:34:32,835][20347] Updated weights for policy 0, policy_version 424763 (0.0008) [2023-12-27 08:34:33,070][20381] Updated weights for policy 1, policy_version 424371 (0.0007) [2023-12-27 08:34:33,144][20381] Updated weights for policy 1, policy_version 424381 (0.0008) [2023-12-27 08:34:33,206][20381] Updated weights for policy 1, policy_version 424391 (0.0008) [2023-12-27 08:34:33,644][20347] Updated weights for policy 0, policy_version 424773 (0.0008) [2023-12-27 08:34:33,714][20347] Updated weights for policy 0, policy_version 424783 (0.0010) [2023-12-27 08:34:33,782][20347] Updated weights for policy 0, policy_version 424793 (0.0009) [2023-12-27 08:34:33,991][20381] Updated weights for policy 1, policy_version 424401 (0.0008) [2023-12-27 08:34:34,058][20381] Updated weights for policy 1, policy_version 424411 (0.0008) [2023-12-27 08:34:34,119][20381] Updated weights for policy 1, policy_version 424421 (0.0008) [2023-12-27 08:34:34,191][20381] Updated weights for policy 1, policy_version 424431 (0.0010) [2023-12-27 08:34:34,532][20347] Updated weights for policy 0, policy_version 424803 (0.0010) [2023-12-27 08:34:34,604][20347] Updated weights for policy 0, policy_version 424813 (0.0008) [2023-12-27 08:34:34,670][20347] Updated weights for policy 0, policy_version 424823 (0.0010) [2023-12-27 08:34:34,985][20381] Updated weights for policy 1, policy_version 424441 (0.0009) [2023-12-27 08:34:35,046][20381] Updated weights for policy 1, policy_version 424451 (0.0008) [2023-12-27 08:34:35,123][20381] Updated weights for policy 1, policy_version 424461 (0.0008) [2023-12-27 08:34:35,534][20347] Updated weights for policy 0, policy_version 424833 (0.0009) [2023-12-27 08:34:35,605][20347] Updated weights for policy 0, policy_version 424843 (0.0008) [2023-12-27 08:34:35,679][20347] Updated weights for policy 0, policy_version 424853 (0.0008) [2023-12-27 08:34:35,761][20347] Updated weights for policy 0, policy_version 424863 (0.0008) [2023-12-27 08:34:36,185][20381] Updated weights for policy 1, policy_version 424471 (0.0008) [2023-12-27 08:34:36,256][20381] Updated weights for policy 1, policy_version 424481 (0.0008) [2023-12-27 08:34:36,324][20381] Updated weights for policy 1, policy_version 424491 (0.0007) [2023-12-27 08:34:36,700][20347] Updated weights for policy 0, policy_version 424873 (0.0009) [2023-12-27 08:34:36,768][20347] Updated weights for policy 0, policy_version 424883 (0.0008) [2023-12-27 08:34:36,836][20347] Updated weights for policy 0, policy_version 424893 (0.0008) [2023-12-27 08:34:37,188][20381] Updated weights for policy 1, policy_version 424501 (0.0008) [2023-12-27 08:34:37,252][20381] Updated weights for policy 1, policy_version 424511 (0.0009) [2023-12-27 08:34:37,322][20381] Updated weights for policy 1, policy_version 424521 (0.0010) [2023-12-27 08:34:37,362][19170] Fps is (10 sec: 18022.2, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 217481216. Throughput: 0: 10313.5, 1: 10174.6. Samples: 217467336. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '782.570')] [2023-12-27 08:34:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000424896_108789760.pth... [2023-12-27 08:34:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000424528_108699648.pth... [2023-12-27 08:34:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000423744_108494848.pth [2023-12-27 08:34:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000423376_108404736.pth [2023-12-27 08:34:37,704][20347] Updated weights for policy 0, policy_version 424903 (0.0009) [2023-12-27 08:34:37,775][20347] Updated weights for policy 0, policy_version 424913 (0.0009) [2023-12-27 08:34:37,849][20347] Updated weights for policy 0, policy_version 424923 (0.0009) [2023-12-27 08:34:38,190][20381] Updated weights for policy 1, policy_version 424531 (0.0009) [2023-12-27 08:34:38,263][20381] Updated weights for policy 1, policy_version 424541 (0.0008) [2023-12-27 08:34:38,340][20381] Updated weights for policy 1, policy_version 424551 (0.0009) [2023-12-27 08:34:38,749][20347] Updated weights for policy 0, policy_version 424933 (0.0010) [2023-12-27 08:34:38,822][20347] Updated weights for policy 0, policy_version 424943 (0.0009) [2023-12-27 08:34:38,896][20347] Updated weights for policy 0, policy_version 424953 (0.0010) [2023-12-27 08:34:39,213][20381] Updated weights for policy 1, policy_version 424561 (0.0009) [2023-12-27 08:34:39,287][20381] Updated weights for policy 1, policy_version 424571 (0.0007) [2023-12-27 08:34:39,359][20381] Updated weights for policy 1, policy_version 424581 (0.0009) [2023-12-27 08:34:39,442][20381] Updated weights for policy 1, policy_version 424591 (0.0010) [2023-12-27 08:34:39,787][20347] Updated weights for policy 0, policy_version 424963 (0.0009) [2023-12-27 08:34:39,868][20347] Updated weights for policy 0, policy_version 424973 (0.0009) [2023-12-27 08:34:39,937][20347] Updated weights for policy 0, policy_version 424983 (0.0009) [2023-12-27 08:34:40,338][20381] Updated weights for policy 1, policy_version 424601 (0.0009) [2023-12-27 08:34:40,411][20381] Updated weights for policy 1, policy_version 424611 (0.0008) [2023-12-27 08:34:40,486][20381] Updated weights for policy 1, policy_version 424621 (0.0008) [2023-12-27 08:34:40,798][20347] Updated weights for policy 0, policy_version 424993 (0.0008) [2023-12-27 08:34:40,877][20347] Updated weights for policy 0, policy_version 425003 (0.0008) [2023-12-27 08:34:40,940][20347] Updated weights for policy 0, policy_version 425013 (0.0008) [2023-12-27 08:34:41,018][20347] Updated weights for policy 0, policy_version 425023 (0.0009) [2023-12-27 08:34:41,381][20381] Updated weights for policy 1, policy_version 424631 (0.0008) [2023-12-27 08:34:41,466][20381] Updated weights for policy 1, policy_version 424641 (0.0009) [2023-12-27 08:34:41,538][20381] Updated weights for policy 1, policy_version 424651 (0.0009) [2023-12-27 08:34:41,915][20347] Updated weights for policy 0, policy_version 425033 (0.0009) [2023-12-27 08:34:41,984][20347] Updated weights for policy 0, policy_version 425043 (0.0008) [2023-12-27 08:34:42,061][20347] Updated weights for policy 0, policy_version 425053 (0.0008) [2023-12-27 08:34:42,362][19170] Fps is (10 sec: 15564.7, 60 sec: 19933.9, 300 sec: 20577.2). Total num frames: 217563136. Throughput: 0: 9953.3, 1: 9851.4. Samples: 217559620. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.910')] [2023-12-27 08:34:42,398][20381] Updated weights for policy 1, policy_version 424661 (0.0009) [2023-12-27 08:34:42,470][20381] Updated weights for policy 1, policy_version 424671 (0.0008) [2023-12-27 08:34:42,543][20381] Updated weights for policy 1, policy_version 424681 (0.0008) [2023-12-27 08:34:42,894][20347] Updated weights for policy 0, policy_version 425063 (0.0008) [2023-12-27 08:34:42,969][20347] Updated weights for policy 0, policy_version 425073 (0.0009) [2023-12-27 08:34:43,040][20347] Updated weights for policy 0, policy_version 425083 (0.0009) [2023-12-27 08:34:43,405][20381] Updated weights for policy 1, policy_version 424691 (0.0008) [2023-12-27 08:34:43,473][20381] Updated weights for policy 1, policy_version 424701 (0.0009) [2023-12-27 08:34:43,548][20381] Updated weights for policy 1, policy_version 424711 (0.0009) [2023-12-27 08:34:43,885][20347] Updated weights for policy 0, policy_version 425093 (0.0008) [2023-12-27 08:34:43,960][20347] Updated weights for policy 0, policy_version 425103 (0.0008) [2023-12-27 08:34:44,030][20347] Updated weights for policy 0, policy_version 425113 (0.0009) [2023-12-27 08:34:44,364][20381] Updated weights for policy 1, policy_version 424721 (0.0008) [2023-12-27 08:34:44,433][20381] Updated weights for policy 1, policy_version 424731 (0.0008) [2023-12-27 08:34:44,506][20381] Updated weights for policy 1, policy_version 424741 (0.0009) [2023-12-27 08:34:44,574][20381] Updated weights for policy 1, policy_version 424751 (0.0008) [2023-12-27 08:34:44,850][20347] Updated weights for policy 0, policy_version 425123 (0.0008) [2023-12-27 08:34:44,919][20347] Updated weights for policy 0, policy_version 425133 (0.0008) [2023-12-27 08:34:44,994][20347] Updated weights for policy 0, policy_version 425143 (0.0008) [2023-12-27 08:34:45,424][20381] Updated weights for policy 1, policy_version 424761 (0.0009) [2023-12-27 08:34:45,507][20381] Updated weights for policy 1, policy_version 424772 (0.0009) [2023-12-27 08:34:45,575][20381] Updated weights for policy 1, policy_version 424782 (0.0007) [2023-12-27 08:34:45,828][20347] Updated weights for policy 0, policy_version 425153 (0.0009) [2023-12-27 08:34:45,903][20347] Updated weights for policy 0, policy_version 425163 (0.0009) [2023-12-27 08:34:45,984][20347] Updated weights for policy 0, policy_version 425174 (0.0008) [2023-12-27 08:34:46,054][20347] Updated weights for policy 0, policy_version 425184 (0.0008) [2023-12-27 08:34:46,407][20381] Updated weights for policy 1, policy_version 424792 (0.0008) [2023-12-27 08:34:46,480][20381] Updated weights for policy 1, policy_version 424802 (0.0009) [2023-12-27 08:34:46,553][20381] Updated weights for policy 1, policy_version 424812 (0.0009) [2023-12-27 08:34:46,911][20347] Updated weights for policy 0, policy_version 425194 (0.0008) [2023-12-27 08:34:46,981][20347] Updated weights for policy 0, policy_version 425204 (0.0008) [2023-12-27 08:34:47,053][20347] Updated weights for policy 0, policy_version 425214 (0.0008) [2023-12-27 08:34:47,359][20381] Updated weights for policy 1, policy_version 424822 (0.0008) [2023-12-27 08:34:47,362][19170] Fps is (10 sec: 16384.1, 60 sec: 19524.3, 300 sec: 20493.9). Total num frames: 217645056. Throughput: 0: 9648.1, 1: 9610.9. Samples: 217657652. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.610')] [2023-12-27 08:34:47,428][20381] Updated weights for policy 1, policy_version 424832 (0.0008) [2023-12-27 08:34:47,498][20381] Updated weights for policy 1, policy_version 424842 (0.0009) [2023-12-27 08:34:47,876][20347] Updated weights for policy 0, policy_version 425224 (0.0009) [2023-12-27 08:34:47,950][20347] Updated weights for policy 0, policy_version 425234 (0.0009) [2023-12-27 08:34:48,024][20347] Updated weights for policy 0, policy_version 425244 (0.0009) [2023-12-27 08:34:48,305][20381] Updated weights for policy 1, policy_version 424852 (0.0009) [2023-12-27 08:34:48,379][20381] Updated weights for policy 1, policy_version 424862 (0.0009) [2023-12-27 08:34:48,453][20381] Updated weights for policy 1, policy_version 424872 (0.0009) [2023-12-27 08:34:48,878][20347] Updated weights for policy 0, policy_version 425254 (0.0009) [2023-12-27 08:34:48,953][20347] Updated weights for policy 0, policy_version 425264 (0.0009) [2023-12-27 08:34:49,028][20347] Updated weights for policy 0, policy_version 425274 (0.0010) [2023-12-27 08:34:49,296][20381] Updated weights for policy 1, policy_version 424882 (0.0008) [2023-12-27 08:34:49,371][20381] Updated weights for policy 1, policy_version 424892 (0.0009) [2023-12-27 08:34:49,441][20381] Updated weights for policy 1, policy_version 424902 (0.0008) [2023-12-27 08:34:49,516][20381] Updated weights for policy 1, policy_version 424912 (0.0008) [2023-12-27 08:34:49,878][20347] Updated weights for policy 0, policy_version 425284 (0.0009) [2023-12-27 08:34:49,948][20347] Updated weights for policy 0, policy_version 425294 (0.0008) [2023-12-27 08:34:50,024][20347] Updated weights for policy 0, policy_version 425304 (0.0008) [2023-12-27 08:34:50,356][20381] Updated weights for policy 1, policy_version 424922 (0.0008) [2023-12-27 08:34:50,431][20381] Updated weights for policy 1, policy_version 424932 (0.0008) [2023-12-27 08:34:50,499][20381] Updated weights for policy 1, policy_version 424942 (0.0008) [2023-12-27 08:34:50,838][20347] Updated weights for policy 0, policy_version 425314 (0.0008) [2023-12-27 08:34:50,912][20347] Updated weights for policy 0, policy_version 425324 (0.0009) [2023-12-27 08:34:50,986][20347] Updated weights for policy 0, policy_version 425334 (0.0009) [2023-12-27 08:34:51,060][20347] Updated weights for policy 0, policy_version 425344 (0.0008) [2023-12-27 08:34:51,312][20381] Updated weights for policy 1, policy_version 424952 (0.0008) [2023-12-27 08:34:51,391][20381] Updated weights for policy 1, policy_version 424962 (0.0008) [2023-12-27 08:34:51,468][20381] Updated weights for policy 1, policy_version 424972 (0.0008) [2023-12-27 08:34:51,929][20347] Updated weights for policy 0, policy_version 425354 (0.0009) [2023-12-27 08:34:51,997][20347] Updated weights for policy 0, policy_version 425364 (0.0008) [2023-12-27 08:34:52,073][20347] Updated weights for policy 0, policy_version 425374 (0.0008) [2023-12-27 08:34:52,281][20381] Updated weights for policy 1, policy_version 424982 (0.0008) [2023-12-27 08:34:52,359][20381] Updated weights for policy 1, policy_version 424992 (0.0009) [2023-12-27 08:34:52,363][19170] Fps is (10 sec: 16382.7, 60 sec: 19251.0, 300 sec: 20382.8). Total num frames: 217726976. Throughput: 0: 9411.6, 1: 9450.2. Samples: 217706736. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:52,364][19170] Avg episode reward: [(0, '1000.000'), (1, '922.750')] [2023-12-27 08:34:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000425376_108912640.pth... [2023-12-27 08:34:52,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000424352_108650496.pth [2023-12-27 08:34:52,435][20381] Updated weights for policy 1, policy_version 425002 (0.0009) [2023-12-27 08:34:52,477][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000425008_108822528.pth... [2023-12-27 08:34:52,482][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000423952_108552192.pth [2023-12-27 08:34:52,866][20347] Updated weights for policy 0, policy_version 425384 (0.0008) [2023-12-27 08:34:52,936][20347] Updated weights for policy 0, policy_version 425394 (0.0008) [2023-12-27 08:34:53,010][20347] Updated weights for policy 0, policy_version 425404 (0.0008) [2023-12-27 08:34:53,242][20381] Updated weights for policy 1, policy_version 425012 (0.0009) [2023-12-27 08:34:53,316][20381] Updated weights for policy 1, policy_version 425022 (0.0009) [2023-12-27 08:34:53,384][20381] Updated weights for policy 1, policy_version 425032 (0.0009) [2023-12-27 08:34:53,810][20347] Updated weights for policy 0, policy_version 425414 (0.0009) [2023-12-27 08:34:53,880][20347] Updated weights for policy 0, policy_version 425424 (0.0009) [2023-12-27 08:34:53,952][20347] Updated weights for policy 0, policy_version 425434 (0.0010) [2023-12-27 08:34:54,198][20381] Updated weights for policy 1, policy_version 425042 (0.0008) [2023-12-27 08:34:54,265][20381] Updated weights for policy 1, policy_version 425052 (0.0008) [2023-12-27 08:34:54,334][20381] Updated weights for policy 1, policy_version 425062 (0.0008) [2023-12-27 08:34:54,410][20381] Updated weights for policy 1, policy_version 425072 (0.0009) [2023-12-27 08:34:54,767][20347] Updated weights for policy 0, policy_version 425444 (0.0009) [2023-12-27 08:34:54,835][20347] Updated weights for policy 0, policy_version 425454 (0.0010) [2023-12-27 08:34:54,906][20347] Updated weights for policy 0, policy_version 425464 (0.0009) [2023-12-27 08:34:55,219][20381] Updated weights for policy 1, policy_version 425082 (0.0009) [2023-12-27 08:34:55,287][20381] Updated weights for policy 1, policy_version 425092 (0.0008) [2023-12-27 08:34:55,358][20381] Updated weights for policy 1, policy_version 425102 (0.0008) [2023-12-27 08:34:55,741][20347] Updated weights for policy 0, policy_version 425474 (0.0010) [2023-12-27 08:34:55,813][20347] Updated weights for policy 0, policy_version 425484 (0.0009) [2023-12-27 08:34:55,878][20347] Updated weights for policy 0, policy_version 425494 (0.0008) [2023-12-27 08:34:55,950][20347] Updated weights for policy 0, policy_version 425504 (0.0008) [2023-12-27 08:34:56,145][20381] Updated weights for policy 1, policy_version 425112 (0.0008) [2023-12-27 08:34:56,214][20381] Updated weights for policy 1, policy_version 425122 (0.0008) [2023-12-27 08:34:56,279][20381] Updated weights for policy 1, policy_version 425132 (0.0007) [2023-12-27 08:34:56,736][20347] Updated weights for policy 0, policy_version 425514 (0.0009) [2023-12-27 08:34:56,806][20347] Updated weights for policy 0, policy_version 425524 (0.0008) [2023-12-27 08:34:56,873][20347] Updated weights for policy 0, policy_version 425534 (0.0008) [2023-12-27 08:34:57,047][20381] Updated weights for policy 1, policy_version 425142 (0.0008) [2023-12-27 08:34:57,117][20381] Updated weights for policy 1, policy_version 425152 (0.0008) [2023-12-27 08:34:57,186][20381] Updated weights for policy 1, policy_version 425162 (0.0008) [2023-12-27 08:34:57,362][19170] Fps is (10 sec: 17203.2, 60 sec: 18978.1, 300 sec: 20355.0). Total num frames: 217817088. Throughput: 0: 9142.1, 1: 9162.9. Samples: 217806936. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:34:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.430')] [2023-12-27 08:34:57,629][20347] Updated weights for policy 0, policy_version 425544 (0.0008) [2023-12-27 08:34:57,693][20347] Updated weights for policy 0, policy_version 425554 (0.0008) [2023-12-27 08:34:57,760][20347] Updated weights for policy 0, policy_version 425564 (0.0008) [2023-12-27 08:34:57,971][20381] Updated weights for policy 1, policy_version 425172 (0.0009) [2023-12-27 08:34:58,037][20381] Updated weights for policy 1, policy_version 425182 (0.0007) [2023-12-27 08:34:58,105][20381] Updated weights for policy 1, policy_version 425192 (0.0009) [2023-12-27 08:34:58,580][20347] Updated weights for policy 0, policy_version 425574 (0.0008) [2023-12-27 08:34:58,659][20347] Updated weights for policy 0, policy_version 425584 (0.0009) [2023-12-27 08:34:58,727][20347] Updated weights for policy 0, policy_version 425594 (0.0009) [2023-12-27 08:34:59,013][20381] Updated weights for policy 1, policy_version 425202 (0.0008) [2023-12-27 08:34:59,082][20381] Updated weights for policy 1, policy_version 425212 (0.0008) [2023-12-27 08:34:59,143][20381] Updated weights for policy 1, policy_version 425222 (0.0008) [2023-12-27 08:34:59,206][20381] Updated weights for policy 1, policy_version 425232 (0.0008) [2023-12-27 08:34:59,563][20347] Updated weights for policy 0, policy_version 425604 (0.0008) [2023-12-27 08:34:59,615][20347] Updated weights for policy 0, policy_version 425614 (0.0007) [2023-12-27 08:34:59,675][20347] Updated weights for policy 0, policy_version 425624 (0.0005) [2023-12-27 08:34:59,935][20381] Updated weights for policy 1, policy_version 425242 (0.0008) [2023-12-27 08:35:00,016][20381] Updated weights for policy 1, policy_version 425252 (0.0008) [2023-12-27 08:35:00,087][20381] Updated weights for policy 1, policy_version 425262 (0.0009) [2023-12-27 08:35:00,500][20347] Updated weights for policy 0, policy_version 425634 (0.0006) [2023-12-27 08:35:00,576][20347] Updated weights for policy 0, policy_version 425644 (0.0009) [2023-12-27 08:35:00,644][20347] Updated weights for policy 0, policy_version 425654 (0.0010) [2023-12-27 08:35:00,699][20347] Updated weights for policy 0, policy_version 425664 (0.0009) [2023-12-27 08:35:00,924][20381] Updated weights for policy 1, policy_version 425272 (0.0008) [2023-12-27 08:35:00,991][20381] Updated weights for policy 1, policy_version 425282 (0.0008) [2023-12-27 08:35:01,061][20381] Updated weights for policy 1, policy_version 425292 (0.0009) [2023-12-27 08:35:01,602][20347] Updated weights for policy 0, policy_version 425674 (0.0008) [2023-12-27 08:35:01,673][20347] Updated weights for policy 0, policy_version 425684 (0.0009) [2023-12-27 08:35:01,743][20347] Updated weights for policy 0, policy_version 425694 (0.0008) [2023-12-27 08:35:01,942][20381] Updated weights for policy 1, policy_version 425302 (0.0009) [2023-12-27 08:35:02,019][20381] Updated weights for policy 1, policy_version 425312 (0.0008) [2023-12-27 08:35:02,090][20381] Updated weights for policy 1, policy_version 425322 (0.0009) [2023-12-27 08:35:02,362][19170] Fps is (10 sec: 17204.6, 60 sec: 18568.5, 300 sec: 20299.5). Total num frames: 217899008. Throughput: 0: 8856.2, 1: 8977.7. Samples: 217908944. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '942.150')] [2023-12-27 08:35:02,583][20347] Updated weights for policy 0, policy_version 425704 (0.0008) [2023-12-27 08:35:02,659][20347] Updated weights for policy 0, policy_version 425714 (0.0008) [2023-12-27 08:35:02,731][20347] Updated weights for policy 0, policy_version 425724 (0.0008) [2023-12-27 08:35:02,931][20381] Updated weights for policy 1, policy_version 425332 (0.0009) [2023-12-27 08:35:03,004][20381] Updated weights for policy 1, policy_version 425342 (0.0009) [2023-12-27 08:35:03,076][20381] Updated weights for policy 1, policy_version 425352 (0.0008) [2023-12-27 08:35:03,515][20347] Updated weights for policy 0, policy_version 425734 (0.0008) [2023-12-27 08:35:03,583][20347] Updated weights for policy 0, policy_version 425744 (0.0008) [2023-12-27 08:35:03,650][20347] Updated weights for policy 0, policy_version 425754 (0.0008) [2023-12-27 08:35:03,908][20381] Updated weights for policy 1, policy_version 425362 (0.0008) [2023-12-27 08:35:03,978][20381] Updated weights for policy 1, policy_version 425372 (0.0009) [2023-12-27 08:35:04,049][20381] Updated weights for policy 1, policy_version 425382 (0.0008) [2023-12-27 08:35:04,119][20381] Updated weights for policy 1, policy_version 425392 (0.0009) [2023-12-27 08:35:04,565][20347] Updated weights for policy 0, policy_version 425764 (0.0009) [2023-12-27 08:35:04,636][20347] Updated weights for policy 0, policy_version 425774 (0.0010) [2023-12-27 08:35:04,708][20347] Updated weights for policy 0, policy_version 425784 (0.0009) [2023-12-27 08:35:04,967][20381] Updated weights for policy 1, policy_version 425402 (0.0009) [2023-12-27 08:35:05,042][20381] Updated weights for policy 1, policy_version 425412 (0.0009) [2023-12-27 08:35:05,111][20381] Updated weights for policy 1, policy_version 425422 (0.0009) [2023-12-27 08:35:05,572][20347] Updated weights for policy 0, policy_version 425794 (0.0009) [2023-12-27 08:35:05,643][20347] Updated weights for policy 0, policy_version 425804 (0.0008) [2023-12-27 08:35:05,717][20347] Updated weights for policy 0, policy_version 425814 (0.0008) [2023-12-27 08:35:05,789][20347] Updated weights for policy 0, policy_version 425824 (0.0008) [2023-12-27 08:35:05,972][20381] Updated weights for policy 1, policy_version 425432 (0.0009) [2023-12-27 08:35:06,041][20381] Updated weights for policy 1, policy_version 425442 (0.0009) [2023-12-27 08:35:06,112][20381] Updated weights for policy 1, policy_version 425452 (0.0010) [2023-12-27 08:35:06,627][20347] Updated weights for policy 0, policy_version 425834 (0.0009) [2023-12-27 08:35:06,701][20347] Updated weights for policy 0, policy_version 425844 (0.0008) [2023-12-27 08:35:06,773][20347] Updated weights for policy 0, policy_version 425854 (0.0009) [2023-12-27 08:35:06,951][20381] Updated weights for policy 1, policy_version 425462 (0.0009) [2023-12-27 08:35:07,023][20381] Updated weights for policy 1, policy_version 425472 (0.0009) [2023-12-27 08:35:07,099][20381] Updated weights for policy 1, policy_version 425482 (0.0008) [2023-12-27 08:35:07,362][19170] Fps is (10 sec: 16383.9, 60 sec: 18295.4, 300 sec: 20244.0). Total num frames: 217980928. Throughput: 0: 8716.6, 1: 8834.8. Samples: 217958116. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.830')] [2023-12-27 08:35:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000425488_108945408.pth... [2023-12-27 08:35:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000425856_109035520.pth... [2023-12-27 08:35:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000424528_108699648.pth [2023-12-27 08:35:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000424896_108789760.pth [2023-12-27 08:35:07,613][20347] Updated weights for policy 0, policy_version 425864 (0.0009) [2023-12-27 08:35:07,684][20347] Updated weights for policy 0, policy_version 425874 (0.0009) [2023-12-27 08:35:07,749][20347] Updated weights for policy 0, policy_version 425884 (0.0009) [2023-12-27 08:35:07,900][20381] Updated weights for policy 1, policy_version 425492 (0.0008) [2023-12-27 08:35:07,975][20381] Updated weights for policy 1, policy_version 425502 (0.0009) [2023-12-27 08:35:08,048][20381] Updated weights for policy 1, policy_version 425512 (0.0008) [2023-12-27 08:35:08,648][20347] Updated weights for policy 0, policy_version 425894 (0.0009) [2023-12-27 08:35:08,724][20347] Updated weights for policy 0, policy_version 425904 (0.0009) [2023-12-27 08:35:08,798][20347] Updated weights for policy 0, policy_version 425914 (0.0009) [2023-12-27 08:35:08,891][20381] Updated weights for policy 1, policy_version 425522 (0.0009) [2023-12-27 08:35:08,963][20381] Updated weights for policy 1, policy_version 425532 (0.0009) [2023-12-27 08:35:09,041][20381] Updated weights for policy 1, policy_version 425542 (0.0010) [2023-12-27 08:35:09,111][20381] Updated weights for policy 1, policy_version 425552 (0.0008) [2023-12-27 08:35:09,729][20347] Updated weights for policy 0, policy_version 425924 (0.0009) [2023-12-27 08:35:09,799][20347] Updated weights for policy 0, policy_version 425934 (0.0008) [2023-12-27 08:35:09,872][20347] Updated weights for policy 0, policy_version 425944 (0.0010) [2023-12-27 08:35:10,032][20381] Updated weights for policy 1, policy_version 425563 (0.0009) [2023-12-27 08:35:10,104][20381] Updated weights for policy 1, policy_version 425573 (0.0010) [2023-12-27 08:35:10,173][20381] Updated weights for policy 1, policy_version 425583 (0.0010) [2023-12-27 08:35:10,752][20347] Updated weights for policy 0, policy_version 425954 (0.0008) [2023-12-27 08:35:10,821][20347] Updated weights for policy 0, policy_version 425964 (0.0008) [2023-12-27 08:35:10,895][20347] Updated weights for policy 0, policy_version 425974 (0.0009) [2023-12-27 08:35:10,965][20347] Updated weights for policy 0, policy_version 425984 (0.0008) [2023-12-27 08:35:10,982][20381] Updated weights for policy 1, policy_version 425593 (0.0010) [2023-12-27 08:35:11,059][20381] Updated weights for policy 1, policy_version 425603 (0.0009) [2023-12-27 08:35:11,134][20381] Updated weights for policy 1, policy_version 425613 (0.0010) [2023-12-27 08:35:11,820][20347] Updated weights for policy 0, policy_version 425994 (0.0008) [2023-12-27 08:35:11,891][20347] Updated weights for policy 0, policy_version 426004 (0.0009) [2023-12-27 08:35:11,969][20347] Updated weights for policy 0, policy_version 426014 (0.0008) [2023-12-27 08:35:11,996][20381] Updated weights for policy 1, policy_version 425623 (0.0009) [2023-12-27 08:35:12,070][20381] Updated weights for policy 1, policy_version 425633 (0.0008) [2023-12-27 08:35:12,140][20381] Updated weights for policy 1, policy_version 425643 (0.0009) [2023-12-27 08:35:12,362][19170] Fps is (10 sec: 16384.0, 60 sec: 17749.3, 300 sec: 20160.7). Total num frames: 218062848. Throughput: 0: 8389.6, 1: 8545.9. Samples: 218055324. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.060')] [2023-12-27 08:35:12,814][20347] Updated weights for policy 0, policy_version 426024 (0.0008) [2023-12-27 08:35:12,886][20347] Updated weights for policy 0, policy_version 426034 (0.0008) [2023-12-27 08:35:12,958][20347] Updated weights for policy 0, policy_version 426044 (0.0009) [2023-12-27 08:35:12,970][20381] Updated weights for policy 1, policy_version 425653 (0.0008) [2023-12-27 08:35:13,040][20381] Updated weights for policy 1, policy_version 425663 (0.0008) [2023-12-27 08:35:13,110][20381] Updated weights for policy 1, policy_version 425673 (0.0009) [2023-12-27 08:35:13,759][20347] Updated weights for policy 0, policy_version 426054 (0.0009) [2023-12-27 08:35:13,822][20347] Updated weights for policy 0, policy_version 426064 (0.0008) [2023-12-27 08:35:13,884][20381] Updated weights for policy 1, policy_version 425683 (0.0008) [2023-12-27 08:35:13,890][20347] Updated weights for policy 0, policy_version 426074 (0.0008) [2023-12-27 08:35:13,955][20381] Updated weights for policy 1, policy_version 425693 (0.0008) [2023-12-27 08:35:14,022][20381] Updated weights for policy 1, policy_version 425703 (0.0008) [2023-12-27 08:35:14,669][20347] Updated weights for policy 0, policy_version 426084 (0.0008) [2023-12-27 08:35:14,739][20347] Updated weights for policy 0, policy_version 426094 (0.0007) [2023-12-27 08:35:14,807][20347] Updated weights for policy 0, policy_version 426104 (0.0008) [2023-12-27 08:35:14,821][20381] Updated weights for policy 1, policy_version 425713 (0.0008) [2023-12-27 08:35:14,872][20381] Updated weights for policy 1, policy_version 425723 (0.0006) [2023-12-27 08:35:14,927][20381] Updated weights for policy 1, policy_version 425733 (0.0007) [2023-12-27 08:35:14,983][20381] Updated weights for policy 1, policy_version 425743 (0.0007) [2023-12-27 08:35:15,539][20347] Updated weights for policy 0, policy_version 426114 (0.0009) [2023-12-27 08:35:15,595][20347] Updated weights for policy 0, policy_version 426124 (0.0009) [2023-12-27 08:35:15,647][20347] Updated weights for policy 0, policy_version 426134 (0.0007) [2023-12-27 08:35:15,700][20347] Updated weights for policy 0, policy_version 426144 (0.0010) [2023-12-27 08:35:15,721][20381] Updated weights for policy 1, policy_version 425753 (0.0006) [2023-12-27 08:35:15,779][20381] Updated weights for policy 1, policy_version 425763 (0.0007) [2023-12-27 08:35:15,831][20381] Updated weights for policy 1, policy_version 425773 (0.0008) [2023-12-27 08:35:16,349][20347] Updated weights for policy 0, policy_version 426154 (0.0008) [2023-12-27 08:35:16,404][20347] Updated weights for policy 0, policy_version 426164 (0.0008) [2023-12-27 08:35:16,463][20347] Updated weights for policy 0, policy_version 426174 (0.0007) [2023-12-27 08:35:16,469][20381] Updated weights for policy 1, policy_version 425783 (0.0005) [2023-12-27 08:35:16,528][20381] Updated weights for policy 1, policy_version 425793 (0.0008) [2023-12-27 08:35:16,580][20381] Updated weights for policy 1, policy_version 425803 (0.0007) [2023-12-27 08:35:17,147][20347] Updated weights for policy 0, policy_version 426184 (0.0008) [2023-12-27 08:35:17,201][20347] Updated weights for policy 0, policy_version 426194 (0.0008) [2023-12-27 08:35:17,229][20381] Updated weights for policy 1, policy_version 425813 (0.0007) [2023-12-27 08:35:17,258][20347] Updated weights for policy 0, policy_version 426204 (0.0008) [2023-12-27 08:35:17,287][20381] Updated weights for policy 1, policy_version 425823 (0.0006) [2023-12-27 08:35:17,346][20381] Updated weights for policy 1, policy_version 425833 (0.0008) [2023-12-27 08:35:17,362][19170] Fps is (10 sec: 17203.4, 60 sec: 17612.8, 300 sec: 20132.9). Total num frames: 218152960. Throughput: 0: 8272.3, 1: 8336.4. Samples: 218164772. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.330')] [2023-12-27 08:35:17,952][20347] Updated weights for policy 0, policy_version 426214 (0.0005) [2023-12-27 08:35:17,985][20381] Updated weights for policy 1, policy_version 425843 (0.0007) [2023-12-27 08:35:18,011][20347] Updated weights for policy 0, policy_version 426224 (0.0006) [2023-12-27 08:35:18,043][20381] Updated weights for policy 1, policy_version 425853 (0.0008) [2023-12-27 08:35:18,070][20347] Updated weights for policy 0, policy_version 426234 (0.0006) [2023-12-27 08:35:18,097][20381] Updated weights for policy 1, policy_version 425863 (0.0007) [2023-12-27 08:35:18,622][20347] Updated weights for policy 0, policy_version 426244 (0.0007) [2023-12-27 08:35:18,675][20347] Updated weights for policy 0, policy_version 426254 (0.0008) [2023-12-27 08:35:18,726][20347] Updated weights for policy 0, policy_version 426264 (0.0007) [2023-12-27 08:35:18,796][20381] Updated weights for policy 1, policy_version 425873 (0.0009) [2023-12-27 08:35:18,857][20381] Updated weights for policy 1, policy_version 425883 (0.0008) [2023-12-27 08:35:18,915][20381] Updated weights for policy 1, policy_version 425893 (0.0007) [2023-12-27 08:35:18,968][20381] Updated weights for policy 1, policy_version 425903 (0.0007) [2023-12-27 08:35:19,422][20347] Updated weights for policy 0, policy_version 426274 (0.0007) [2023-12-27 08:35:19,484][20347] Updated weights for policy 0, policy_version 426284 (0.0007) [2023-12-27 08:35:19,549][20347] Updated weights for policy 0, policy_version 426294 (0.0007) [2023-12-27 08:35:19,612][20347] Updated weights for policy 0, policy_version 426304 (0.0007) [2023-12-27 08:35:19,670][20381] Updated weights for policy 1, policy_version 425913 (0.0005) [2023-12-27 08:35:19,725][20381] Updated weights for policy 1, policy_version 425923 (0.0005) [2023-12-27 08:35:19,779][20381] Updated weights for policy 1, policy_version 425933 (0.0005) [2023-12-27 08:35:20,297][20347] Updated weights for policy 0, policy_version 426314 (0.0008) [2023-12-27 08:35:20,350][20347] Updated weights for policy 0, policy_version 426324 (0.0008) [2023-12-27 08:35:20,402][20381] Updated weights for policy 1, policy_version 425943 (0.0006) [2023-12-27 08:35:20,408][20347] Updated weights for policy 0, policy_version 426334 (0.0006) [2023-12-27 08:35:20,456][20381] Updated weights for policy 1, policy_version 425953 (0.0007) [2023-12-27 08:35:20,522][20381] Updated weights for policy 1, policy_version 425963 (0.0008) [2023-12-27 08:35:21,126][20347] Updated weights for policy 0, policy_version 426344 (0.0008) [2023-12-27 08:35:21,190][20347] Updated weights for policy 0, policy_version 426354 (0.0008) [2023-12-27 08:35:21,216][20381] Updated weights for policy 1, policy_version 425973 (0.0008) [2023-12-27 08:35:21,254][20347] Updated weights for policy 0, policy_version 426364 (0.0008) [2023-12-27 08:35:21,270][20381] Updated weights for policy 1, policy_version 425983 (0.0007) [2023-12-27 08:35:21,324][20381] Updated weights for policy 1, policy_version 425993 (0.0007) [2023-12-27 08:35:22,027][20347] Updated weights for policy 0, policy_version 426374 (0.0007) [2023-12-27 08:35:22,037][20381] Updated weights for policy 1, policy_version 426003 (0.0008) [2023-12-27 08:35:22,087][20347] Updated weights for policy 0, policy_version 426384 (0.0006) [2023-12-27 08:35:22,112][20381] Updated weights for policy 1, policy_version 426013 (0.0006) [2023-12-27 08:35:22,154][20347] Updated weights for policy 0, policy_version 426394 (0.0007) [2023-12-27 08:35:22,171][20381] Updated weights for policy 1, policy_version 426023 (0.0008) [2023-12-27 08:35:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 17612.8, 300 sec: 20160.6). Total num frames: 218259456. Throughput: 0: 8403.5, 1: 8482.0. Samples: 218227184. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:22,363][19170] Avg episode reward: [(0, '990.560'), (1, '970.880')] [2023-12-27 08:35:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000426032_109084672.pth... [2023-12-27 08:35:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000426400_109174784.pth... [2023-12-27 08:35:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000425008_108822528.pth [2023-12-27 08:35:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000425376_108912640.pth [2023-12-27 08:35:22,823][20347] Updated weights for policy 0, policy_version 426404 (0.0007) [2023-12-27 08:35:22,871][20381] Updated weights for policy 1, policy_version 426033 (0.0008) [2023-12-27 08:35:22,881][20347] Updated weights for policy 0, policy_version 426414 (0.0007) [2023-12-27 08:35:22,922][20381] Updated weights for policy 1, policy_version 426043 (0.0006) [2023-12-27 08:35:22,931][20347] Updated weights for policy 0, policy_version 426424 (0.0006) [2023-12-27 08:35:22,973][20381] Updated weights for policy 1, policy_version 426053 (0.0006) [2023-12-27 08:35:23,024][20381] Updated weights for policy 1, policy_version 426063 (0.0007) [2023-12-27 08:35:23,584][20347] Updated weights for policy 0, policy_version 426434 (0.0006) [2023-12-27 08:35:23,633][20347] Updated weights for policy 0, policy_version 426444 (0.0007) [2023-12-27 08:35:23,681][20347] Updated weights for policy 0, policy_version 426454 (0.0008) [2023-12-27 08:35:23,719][20381] Updated weights for policy 1, policy_version 426073 (0.0007) [2023-12-27 08:35:23,733][20347] Updated weights for policy 0, policy_version 426464 (0.0007) [2023-12-27 08:35:23,766][20381] Updated weights for policy 1, policy_version 426083 (0.0008) [2023-12-27 08:35:23,817][20381] Updated weights for policy 1, policy_version 426093 (0.0007) [2023-12-27 08:35:24,361][20347] Updated weights for policy 0, policy_version 426474 (0.0008) [2023-12-27 08:35:24,413][20347] Updated weights for policy 0, policy_version 426484 (0.0008) [2023-12-27 08:35:24,466][20347] Updated weights for policy 0, policy_version 426494 (0.0007) [2023-12-27 08:35:24,550][20381] Updated weights for policy 1, policy_version 426103 (0.0007) [2023-12-27 08:35:24,606][20381] Updated weights for policy 1, policy_version 426113 (0.0007) [2023-12-27 08:35:24,664][20381] Updated weights for policy 1, policy_version 426123 (0.0008) [2023-12-27 08:35:25,131][20347] Updated weights for policy 0, policy_version 426504 (0.0007) [2023-12-27 08:35:25,194][20347] Updated weights for policy 0, policy_version 426514 (0.0007) [2023-12-27 08:35:25,248][20347] Updated weights for policy 0, policy_version 426524 (0.0007) [2023-12-27 08:35:25,315][20381] Updated weights for policy 1, policy_version 426133 (0.0007) [2023-12-27 08:35:25,367][20381] Updated weights for policy 1, policy_version 426143 (0.0007) [2023-12-27 08:35:25,425][20381] Updated weights for policy 1, policy_version 426153 (0.0008) [2023-12-27 08:35:25,908][20347] Updated weights for policy 0, policy_version 426534 (0.0008) [2023-12-27 08:35:25,960][20347] Updated weights for policy 0, policy_version 426544 (0.0007) [2023-12-27 08:35:26,018][20347] Updated weights for policy 0, policy_version 426554 (0.0008) [2023-12-27 08:35:26,133][20381] Updated weights for policy 1, policy_version 426163 (0.0008) [2023-12-27 08:35:26,189][20381] Updated weights for policy 1, policy_version 426173 (0.0008) [2023-12-27 08:35:26,239][20381] Updated weights for policy 1, policy_version 426183 (0.0007) [2023-12-27 08:35:26,713][20347] Updated weights for policy 0, policy_version 426564 (0.0008) [2023-12-27 08:35:26,764][20347] Updated weights for policy 0, policy_version 426574 (0.0007) [2023-12-27 08:35:26,815][20347] Updated weights for policy 0, policy_version 426584 (0.0007) [2023-12-27 08:35:26,912][20381] Updated weights for policy 1, policy_version 426193 (0.0006) [2023-12-27 08:35:26,968][20381] Updated weights for policy 1, policy_version 426203 (0.0008) [2023-12-27 08:35:27,020][20381] Updated weights for policy 1, policy_version 426213 (0.0007) [2023-12-27 08:35:27,070][20381] Updated weights for policy 1, policy_version 426223 (0.0008) [2023-12-27 08:35:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 17612.8, 300 sec: 20132.9). Total num frames: 218357760. Throughput: 0: 8731.1, 1: 8780.5. Samples: 218347636. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:27,363][19170] Avg episode reward: [(0, '980.950'), (1, '980.610')] [2023-12-27 08:35:27,528][20347] Updated weights for policy 0, policy_version 426594 (0.0007) [2023-12-27 08:35:27,582][20347] Updated weights for policy 0, policy_version 426604 (0.0007) [2023-12-27 08:35:27,630][20347] Updated weights for policy 0, policy_version 426614 (0.0007) [2023-12-27 08:35:27,689][20347] Updated weights for policy 0, policy_version 426624 (0.0007) [2023-12-27 08:35:27,698][20381] Updated weights for policy 1, policy_version 426233 (0.0009) [2023-12-27 08:35:27,748][20381] Updated weights for policy 1, policy_version 426243 (0.0008) [2023-12-27 08:35:27,799][20381] Updated weights for policy 1, policy_version 426253 (0.0008) [2023-12-27 08:35:28,388][20347] Updated weights for policy 0, policy_version 426634 (0.0007) [2023-12-27 08:35:28,444][20347] Updated weights for policy 0, policy_version 426644 (0.0007) [2023-12-27 08:35:28,482][20381] Updated weights for policy 1, policy_version 426263 (0.0006) [2023-12-27 08:35:28,496][20347] Updated weights for policy 0, policy_version 426654 (0.0007) [2023-12-27 08:35:28,538][20381] Updated weights for policy 1, policy_version 426273 (0.0005) [2023-12-27 08:35:28,594][20381] Updated weights for policy 1, policy_version 426283 (0.0005) [2023-12-27 08:35:29,202][20381] Updated weights for policy 1, policy_version 426293 (0.0006) [2023-12-27 08:35:29,209][20347] Updated weights for policy 0, policy_version 426664 (0.0008) [2023-12-27 08:35:29,227][20291] KL-divergence is very high: 174.4062 [2023-12-27 08:35:29,265][20381] Updated weights for policy 1, policy_version 426303 (0.0006) [2023-12-27 08:35:29,268][20347] Updated weights for policy 0, policy_version 426674 (0.0008) [2023-12-27 08:35:29,318][20291] KL-divergence is very high: 409.0377 [2023-12-27 08:35:29,320][20381] Updated weights for policy 1, policy_version 426313 (0.0008) [2023-12-27 08:35:29,324][20347] Updated weights for policy 0, policy_version 426684 (0.0009) [2023-12-27 08:35:29,947][20347] Updated weights for policy 0, policy_version 426694 (0.0007) [2023-12-27 08:35:29,958][20291] KL-divergence is very high: 126.6196 [2023-12-27 08:35:29,976][20381] Updated weights for policy 1, policy_version 426323 (0.0007) [2023-12-27 08:35:30,000][20347] Updated weights for policy 0, policy_version 426704 (0.0008) [2023-12-27 08:35:30,037][20381] Updated weights for policy 1, policy_version 426333 (0.0006) [2023-12-27 08:35:30,045][20291] KL-divergence is very high: 246.8911 [2023-12-27 08:35:30,057][20347] Updated weights for policy 0, policy_version 426714 (0.0008) [2023-12-27 08:35:30,094][20381] Updated weights for policy 1, policy_version 426343 (0.0005) [2023-12-27 08:35:30,671][20381] Updated weights for policy 1, policy_version 426353 (0.0006) [2023-12-27 08:35:30,723][20381] Updated weights for policy 1, policy_version 426363 (0.0007) [2023-12-27 08:35:30,769][20381] Updated weights for policy 1, policy_version 426373 (0.0007) [2023-12-27 08:35:30,791][20347] Updated weights for policy 0, policy_version 426724 (0.0009) [2023-12-27 08:35:30,818][20381] Updated weights for policy 1, policy_version 426383 (0.0006) [2023-12-27 08:35:30,842][20347] Updated weights for policy 0, policy_version 426734 (0.0010) [2023-12-27 08:35:30,886][20347] Updated weights for policy 0, policy_version 426744 (0.0010) [2023-12-27 08:35:31,533][20381] Updated weights for policy 1, policy_version 426393 (0.0005) [2023-12-27 08:35:31,588][20381] Updated weights for policy 1, policy_version 426403 (0.0005) [2023-12-27 08:35:31,589][20347] Updated weights for policy 0, policy_version 426754 (0.0010) [2023-12-27 08:35:31,648][20347] Updated weights for policy 0, policy_version 426764 (0.0008) [2023-12-27 08:35:31,656][20381] Updated weights for policy 1, policy_version 426413 (0.0006) [2023-12-27 08:35:31,710][20347] Updated weights for policy 0, policy_version 426774 (0.0008) [2023-12-27 08:35:31,762][20347] Updated weights for policy 0, policy_version 426784 (0.0010) [2023-12-27 08:35:32,317][20381] Updated weights for policy 1, policy_version 426423 (0.0009) [2023-12-27 08:35:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 17476.3, 300 sec: 20105.1). Total num frames: 218456064. Throughput: 0: 8990.1, 1: 9106.7. Samples: 218472008. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:32,363][19170] Avg episode reward: [(0, '951.790'), (1, '1000.000')] [2023-12-27 08:35:32,380][20381] Updated weights for policy 1, policy_version 426433 (0.0009) [2023-12-27 08:35:32,433][20347] Updated weights for policy 0, policy_version 426794 (0.0009) [2023-12-27 08:35:32,439][20381] Updated weights for policy 1, policy_version 426443 (0.0008) [2023-12-27 08:35:32,490][20347] Updated weights for policy 0, policy_version 426804 (0.0005) [2023-12-27 08:35:32,548][20347] Updated weights for policy 0, policy_version 426814 (0.0008) [2023-12-27 08:35:33,043][20381] Updated weights for policy 1, policy_version 426453 (0.0006) [2023-12-27 08:35:33,102][20381] Updated weights for policy 1, policy_version 426463 (0.0006) [2023-12-27 08:35:33,134][20347] Updated weights for policy 0, policy_version 426824 (0.0010) [2023-12-27 08:35:33,151][20381] Updated weights for policy 1, policy_version 426473 (0.0005) [2023-12-27 08:35:33,182][20347] Updated weights for policy 0, policy_version 426834 (0.0009) [2023-12-27 08:35:33,229][20347] Updated weights for policy 0, policy_version 426844 (0.0008) [2023-12-27 08:35:33,674][20381] Updated weights for policy 1, policy_version 426483 (0.0005) [2023-12-27 08:35:33,736][20381] Updated weights for policy 1, policy_version 426493 (0.0006) [2023-12-27 08:35:33,791][20381] Updated weights for policy 1, policy_version 426503 (0.0010) [2023-12-27 08:35:33,795][20347] Updated weights for policy 0, policy_version 426854 (0.0006) [2023-12-27 08:35:33,850][20347] Updated weights for policy 0, policy_version 426864 (0.0007) [2023-12-27 08:35:33,904][20347] Updated weights for policy 0, policy_version 426874 (0.0010) [2023-12-27 08:35:34,442][20381] Updated weights for policy 1, policy_version 426513 (0.0010) [2023-12-27 08:35:34,496][20381] Updated weights for policy 1, policy_version 426523 (0.0006) [2023-12-27 08:35:34,553][20381] Updated weights for policy 1, policy_version 426533 (0.0009) [2023-12-27 08:35:34,568][20347] Updated weights for policy 0, policy_version 426884 (0.0010) [2023-12-27 08:35:34,611][20381] Updated weights for policy 1, policy_version 426543 (0.0006) [2023-12-27 08:35:34,624][20347] Updated weights for policy 0, policy_version 426894 (0.0010) [2023-12-27 08:35:34,682][20347] Updated weights for policy 0, policy_version 426904 (0.0010) [2023-12-27 08:35:35,212][20381] Updated weights for policy 1, policy_version 426553 (0.0005) [2023-12-27 08:35:35,266][20381] Updated weights for policy 1, policy_version 426563 (0.0005) [2023-12-27 08:35:35,322][20381] Updated weights for policy 1, policy_version 426573 (0.0006) [2023-12-27 08:35:35,333][20347] Updated weights for policy 0, policy_version 426914 (0.0010) [2023-12-27 08:35:35,391][20347] Updated weights for policy 0, policy_version 426924 (0.0010) [2023-12-27 08:35:35,446][20347] Updated weights for policy 0, policy_version 426934 (0.0008) [2023-12-27 08:35:35,499][20347] Updated weights for policy 0, policy_version 426944 (0.0005) [2023-12-27 08:35:35,987][20381] Updated weights for policy 1, policy_version 426583 (0.0007) [2023-12-27 08:35:36,046][20381] Updated weights for policy 1, policy_version 426593 (0.0006) [2023-12-27 08:35:36,100][20347] Updated weights for policy 0, policy_version 426954 (0.0009) [2023-12-27 08:35:36,113][20381] Updated weights for policy 1, policy_version 426603 (0.0006) [2023-12-27 08:35:36,161][20347] Updated weights for policy 0, policy_version 426964 (0.0011) [2023-12-27 08:35:36,220][20347] Updated weights for policy 0, policy_version 426974 (0.0010) [2023-12-27 08:35:36,757][20381] Updated weights for policy 1, policy_version 426613 (0.0006) [2023-12-27 08:35:36,813][20381] Updated weights for policy 1, policy_version 426623 (0.0007) [2023-12-27 08:35:36,868][20381] Updated weights for policy 1, policy_version 426633 (0.0007) [2023-12-27 08:35:36,918][20347] Updated weights for policy 0, policy_version 426984 (0.0011) [2023-12-27 08:35:36,977][20347] Updated weights for policy 0, policy_version 426994 (0.0010) [2023-12-27 08:35:37,035][20347] Updated weights for policy 0, policy_version 427004 (0.0010) [2023-12-27 08:35:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 18159.0, 300 sec: 20132.9). Total num frames: 218570752. Throughput: 0: 9195.9, 1: 9305.5. Samples: 218539284. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:37,363][19170] Avg episode reward: [(0, '970.990'), (1, '1000.000')] [2023-12-27 08:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000426640_109240320.pth... [2023-12-27 08:35:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000427008_109330432.pth... [2023-12-27 08:35:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000425488_108945408.pth [2023-12-27 08:35:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000425856_109035520.pth [2023-12-27 08:35:37,501][20381] Updated weights for policy 1, policy_version 426643 (0.0008) [2023-12-27 08:35:37,562][20381] Updated weights for policy 1, policy_version 426653 (0.0007) [2023-12-27 08:35:37,623][20381] Updated weights for policy 1, policy_version 426663 (0.0007) [2023-12-27 08:35:37,714][20347] Updated weights for policy 0, policy_version 427014 (0.0010) [2023-12-27 08:35:37,766][20347] Updated weights for policy 0, policy_version 427024 (0.0010) [2023-12-27 08:35:37,815][20347] Updated weights for policy 0, policy_version 427034 (0.0010) [2023-12-27 08:35:38,262][20381] Updated weights for policy 1, policy_version 426673 (0.0008) [2023-12-27 08:35:38,318][20381] Updated weights for policy 1, policy_version 426683 (0.0007) [2023-12-27 08:35:38,384][20381] Updated weights for policy 1, policy_version 426693 (0.0008) [2023-12-27 08:35:38,394][20347] Updated weights for policy 0, policy_version 427044 (0.0010) [2023-12-27 08:35:38,447][20347] Updated weights for policy 0, policy_version 427054 (0.0009) [2023-12-27 08:35:38,450][20381] Updated weights for policy 1, policy_version 426703 (0.0007) [2023-12-27 08:35:38,505][20347] Updated weights for policy 0, policy_version 427064 (0.0006) [2023-12-27 08:35:39,100][20381] Updated weights for policy 1, policy_version 426713 (0.0007) [2023-12-27 08:35:39,158][20381] Updated weights for policy 1, policy_version 426723 (0.0007) [2023-12-27 08:35:39,219][20381] Updated weights for policy 1, policy_version 426733 (0.0008) [2023-12-27 08:35:39,239][20347] Updated weights for policy 0, policy_version 427074 (0.0008) [2023-12-27 08:35:39,299][20347] Updated weights for policy 0, policy_version 427084 (0.0008) [2023-12-27 08:35:39,359][20347] Updated weights for policy 0, policy_version 427094 (0.0008) [2023-12-27 08:35:39,416][20347] Updated weights for policy 0, policy_version 427104 (0.0008) [2023-12-27 08:35:39,958][20381] Updated weights for policy 1, policy_version 426743 (0.0007) [2023-12-27 08:35:40,018][20381] Updated weights for policy 1, policy_version 426753 (0.0008) [2023-12-27 08:35:40,081][20381] Updated weights for policy 1, policy_version 426763 (0.0008) [2023-12-27 08:35:40,157][20347] Updated weights for policy 0, policy_version 427114 (0.0008) [2023-12-27 08:35:40,214][20347] Updated weights for policy 0, policy_version 427124 (0.0008) [2023-12-27 08:35:40,271][20347] Updated weights for policy 0, policy_version 427134 (0.0008) [2023-12-27 08:35:40,774][20381] Updated weights for policy 1, policy_version 426773 (0.0007) [2023-12-27 08:35:40,832][20381] Updated weights for policy 1, policy_version 426783 (0.0008) [2023-12-27 08:35:40,890][20381] Updated weights for policy 1, policy_version 426793 (0.0007) [2023-12-27 08:35:40,968][20347] Updated weights for policy 0, policy_version 427144 (0.0007) [2023-12-27 08:35:40,987][20291] KL-divergence is very high: 149.4784 [2023-12-27 08:35:41,018][20347] Updated weights for policy 0, policy_version 427154 (0.0008) [2023-12-27 08:35:41,082][20347] Updated weights for policy 0, policy_version 427164 (0.0008) [2023-12-27 08:35:41,083][20291] KL-divergence is very high: 152.4692 [2023-12-27 08:35:41,576][20381] Updated weights for policy 1, policy_version 426803 (0.0007) [2023-12-27 08:35:41,640][20381] Updated weights for policy 1, policy_version 426813 (0.0008) [2023-12-27 08:35:41,704][20381] Updated weights for policy 1, policy_version 426823 (0.0008) [2023-12-27 08:35:41,877][20347] Updated weights for policy 0, policy_version 427174 (0.0008) [2023-12-27 08:35:41,921][20291] KL-divergence is very high: 341.1527 [2023-12-27 08:35:41,927][20347] Updated weights for policy 0, policy_version 427184 (0.0007) [2023-12-27 08:35:41,975][20347] Updated weights for policy 0, policy_version 427194 (0.0009) [2023-12-27 08:35:42,360][20381] Updated weights for policy 1, policy_version 426833 (0.0007) [2023-12-27 08:35:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 18432.0, 300 sec: 20105.1). Total num frames: 218669056. Throughput: 0: 9442.8, 1: 9552.5. Samples: 218661728. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:42,363][19170] Avg episode reward: [(0, '990.560'), (1, '1000.000')] [2023-12-27 08:35:42,425][20381] Updated weights for policy 1, policy_version 426843 (0.0007) [2023-12-27 08:35:42,484][20381] Updated weights for policy 1, policy_version 426853 (0.0005) [2023-12-27 08:35:42,552][20381] Updated weights for policy 1, policy_version 426863 (0.0008) [2023-12-27 08:35:42,744][20347] Updated weights for policy 0, policy_version 427204 (0.0008) [2023-12-27 08:35:42,793][20347] Updated weights for policy 0, policy_version 427214 (0.0008) [2023-12-27 08:35:42,846][20347] Updated weights for policy 0, policy_version 427224 (0.0008) [2023-12-27 08:35:43,158][20381] Updated weights for policy 1, policy_version 426873 (0.0006) [2023-12-27 08:35:43,222][20381] Updated weights for policy 1, policy_version 426883 (0.0005) [2023-12-27 08:35:43,288][20381] Updated weights for policy 1, policy_version 426893 (0.0005) [2023-12-27 08:35:43,579][20347] Updated weights for policy 0, policy_version 427234 (0.0007) [2023-12-27 08:35:43,643][20347] Updated weights for policy 0, policy_version 427244 (0.0005) [2023-12-27 08:35:43,699][20347] Updated weights for policy 0, policy_version 427254 (0.0005) [2023-12-27 08:35:43,756][20347] Updated weights for policy 0, policy_version 427264 (0.0010) [2023-12-27 08:35:43,852][20381] Updated weights for policy 1, policy_version 426903 (0.0005) [2023-12-27 08:35:43,902][20381] Updated weights for policy 1, policy_version 426913 (0.0007) [2023-12-27 08:35:43,951][20381] Updated weights for policy 1, policy_version 426923 (0.0010) [2023-12-27 08:35:44,388][20347] Updated weights for policy 0, policy_version 427274 (0.0010) [2023-12-27 08:35:44,441][20347] Updated weights for policy 0, policy_version 427284 (0.0008) [2023-12-27 08:35:44,494][20347] Updated weights for policy 0, policy_version 427294 (0.0006) [2023-12-27 08:35:44,630][20381] Updated weights for policy 1, policy_version 426933 (0.0011) [2023-12-27 08:35:44,689][20381] Updated weights for policy 1, policy_version 426943 (0.0011) [2023-12-27 08:35:44,747][20381] Updated weights for policy 1, policy_version 426953 (0.0011) [2023-12-27 08:35:45,153][20347] Updated weights for policy 0, policy_version 427304 (0.0006) [2023-12-27 08:35:45,214][20347] Updated weights for policy 0, policy_version 427314 (0.0010) [2023-12-27 08:35:45,278][20347] Updated weights for policy 0, policy_version 427324 (0.0009) [2023-12-27 08:35:45,414][20381] Updated weights for policy 1, policy_version 426963 (0.0009) [2023-12-27 08:35:45,477][20381] Updated weights for policy 1, policy_version 426973 (0.0005) [2023-12-27 08:35:45,527][20381] Updated weights for policy 1, policy_version 426983 (0.0007) [2023-12-27 08:35:45,834][20347] Updated weights for policy 0, policy_version 427334 (0.0007) [2023-12-27 08:35:45,890][20347] Updated weights for policy 0, policy_version 427344 (0.0006) [2023-12-27 08:35:45,948][20347] Updated weights for policy 0, policy_version 427354 (0.0006) [2023-12-27 08:35:46,154][20381] Updated weights for policy 1, policy_version 426993 (0.0011) [2023-12-27 08:35:46,210][20381] Updated weights for policy 1, policy_version 427003 (0.0011) [2023-12-27 08:35:46,263][20381] Updated weights for policy 1, policy_version 427013 (0.0011) [2023-12-27 08:35:46,320][20381] Updated weights for policy 1, policy_version 427023 (0.0011) [2023-12-27 08:35:46,509][20347] Updated weights for policy 0, policy_version 427364 (0.0006) [2023-12-27 08:35:46,567][20347] Updated weights for policy 0, policy_version 427374 (0.0008) [2023-12-27 08:35:46,620][20347] Updated weights for policy 0, policy_version 427384 (0.0007) [2023-12-27 08:35:47,029][20381] Updated weights for policy 1, policy_version 427033 (0.0011) [2023-12-27 08:35:47,075][20381] Updated weights for policy 1, policy_version 427043 (0.0010) [2023-12-27 08:35:47,124][20381] Updated weights for policy 1, policy_version 427053 (0.0010) [2023-12-27 08:35:47,295][20347] Updated weights for policy 0, policy_version 427394 (0.0007) [2023-12-27 08:35:47,348][20347] Updated weights for policy 0, policy_version 427404 (0.0006) [2023-12-27 08:35:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 18841.5, 300 sec: 20132.9). Total num frames: 218775552. Throughput: 0: 9686.6, 1: 9832.6. Samples: 218787312. Policy #0 lag: (min: 24.0, avg: 52.4, max: 56.0) [2023-12-27 08:35:47,364][19170] Avg episode reward: [(0, '990.560'), (1, '1000.000')] [2023-12-27 08:35:47,401][20347] Updated weights for policy 0, policy_version 427414 (0.0007) [2023-12-27 08:35:47,459][20347] Updated weights for policy 0, policy_version 427424 (0.0005) [2023-12-27 08:35:47,760][20381] Updated weights for policy 1, policy_version 427063 (0.0007) [2023-12-27 08:35:47,806][20381] Updated weights for policy 1, policy_version 427073 (0.0005) [2023-12-27 08:35:47,865][20381] Updated weights for policy 1, policy_version 427083 (0.0006) [2023-12-27 08:35:48,092][20347] Updated weights for policy 0, policy_version 427434 (0.0008) [2023-12-27 08:35:48,138][20347] Updated weights for policy 0, policy_version 427444 (0.0007) [2023-12-27 08:35:48,195][20347] Updated weights for policy 0, policy_version 427454 (0.0007) [2023-12-27 08:35:48,441][20381] Updated weights for policy 1, policy_version 427093 (0.0009) [2023-12-27 08:35:48,495][20381] Updated weights for policy 1, policy_version 427103 (0.0009) [2023-12-27 08:35:48,544][20381] Updated weights for policy 1, policy_version 427113 (0.0011) [2023-12-27 08:35:48,875][20347] Updated weights for policy 0, policy_version 427464 (0.0007) [2023-12-27 08:35:48,941][20347] Updated weights for policy 0, policy_version 427474 (0.0008) [2023-12-27 08:35:49,001][20347] Updated weights for policy 0, policy_version 427484 (0.0008) [2023-12-27 08:35:49,168][20381] Updated weights for policy 1, policy_version 427123 (0.0011) [2023-12-27 08:35:49,225][20381] Updated weights for policy 1, policy_version 427133 (0.0009) [2023-12-27 08:35:49,283][20381] Updated weights for policy 1, policy_version 427143 (0.0011) [2023-12-27 08:35:49,728][20347] Updated weights for policy 0, policy_version 427494 (0.0007) [2023-12-27 08:35:49,776][20347] Updated weights for policy 0, policy_version 427504 (0.0007) [2023-12-27 08:35:49,821][20347] Updated weights for policy 0, policy_version 427514 (0.0007) [2023-12-27 08:35:49,990][20381] Updated weights for policy 1, policy_version 427153 (0.0008) [2023-12-27 08:35:50,040][20381] Updated weights for policy 1, policy_version 427163 (0.0010) [2023-12-27 08:35:50,086][20381] Updated weights for policy 1, policy_version 427173 (0.0010) [2023-12-27 08:35:50,138][20381] Updated weights for policy 1, policy_version 427183 (0.0010) [2023-12-27 08:35:50,567][20347] Updated weights for policy 0, policy_version 427524 (0.0008) [2023-12-27 08:35:50,629][20347] Updated weights for policy 0, policy_version 427534 (0.0007) [2023-12-27 08:35:50,686][20347] Updated weights for policy 0, policy_version 427544 (0.0007) [2023-12-27 08:35:50,842][20381] Updated weights for policy 1, policy_version 427193 (0.0010) [2023-12-27 08:35:50,897][20381] Updated weights for policy 1, policy_version 427203 (0.0010) [2023-12-27 08:35:50,956][20381] Updated weights for policy 1, policy_version 427213 (0.0011) [2023-12-27 08:35:51,405][20347] Updated weights for policy 0, policy_version 427554 (0.0007) [2023-12-27 08:35:51,457][20347] Updated weights for policy 0, policy_version 427564 (0.0007) [2023-12-27 08:35:51,513][20347] Updated weights for policy 0, policy_version 427574 (0.0007) [2023-12-27 08:35:51,570][20347] Updated weights for policy 0, policy_version 427584 (0.0008) [2023-12-27 08:35:51,659][20381] Updated weights for policy 1, policy_version 427223 (0.0010) [2023-12-27 08:35:51,718][20381] Updated weights for policy 1, policy_version 427233 (0.0006) [2023-12-27 08:35:51,777][20381] Updated weights for policy 1, policy_version 427243 (0.0010) [2023-12-27 08:35:52,313][20347] Updated weights for policy 0, policy_version 427594 (0.0006) [2023-12-27 08:35:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19114.9, 300 sec: 20105.1). Total num frames: 218873856. Throughput: 0: 9816.8, 1: 10011.7. Samples: 218850396. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:35:52,363][19170] Avg episode reward: [(0, '990.530'), (1, '1000.000')] [2023-12-27 08:35:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000427248_109395968.pth... [2023-12-27 08:35:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000426032_109084672.pth [2023-12-27 08:35:52,377][20347] Updated weights for policy 0, policy_version 427604 (0.0007) [2023-12-27 08:35:52,440][20347] Updated weights for policy 0, policy_version 427614 (0.0006) [2023-12-27 08:35:52,449][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000427616_109486080.pth... [2023-12-27 08:35:52,452][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000426400_109174784.pth [2023-12-27 08:35:52,466][20381] Updated weights for policy 1, policy_version 427253 (0.0011) [2023-12-27 08:35:52,527][20381] Updated weights for policy 1, policy_version 427263 (0.0007) [2023-12-27 08:35:52,580][20381] Updated weights for policy 1, policy_version 427273 (0.0010) [2023-12-27 08:35:53,096][20347] Updated weights for policy 0, policy_version 427624 (0.0007) [2023-12-27 08:35:53,156][20347] Updated weights for policy 0, policy_version 427634 (0.0007) [2023-12-27 08:35:53,214][20347] Updated weights for policy 0, policy_version 427644 (0.0007) [2023-12-27 08:35:53,220][20381] Updated weights for policy 1, policy_version 427283 (0.0009) [2023-12-27 08:35:53,276][20381] Updated weights for policy 1, policy_version 427293 (0.0010) [2023-12-27 08:35:53,334][20381] Updated weights for policy 1, policy_version 427303 (0.0010) [2023-12-27 08:35:53,807][20347] Updated weights for policy 0, policy_version 427654 (0.0008) [2023-12-27 08:35:53,867][20347] Updated weights for policy 0, policy_version 427664 (0.0007) [2023-12-27 08:35:53,920][20347] Updated weights for policy 0, policy_version 427674 (0.0006) [2023-12-27 08:35:54,010][20381] Updated weights for policy 1, policy_version 427313 (0.0010) [2023-12-27 08:35:54,072][20381] Updated weights for policy 1, policy_version 427323 (0.0011) [2023-12-27 08:35:54,122][20381] Updated weights for policy 1, policy_version 427333 (0.0011) [2023-12-27 08:35:54,175][20381] Updated weights for policy 1, policy_version 427343 (0.0011) [2023-12-27 08:35:54,567][20347] Updated weights for policy 0, policy_version 427684 (0.0010) [2023-12-27 08:35:54,623][20347] Updated weights for policy 0, policy_version 427694 (0.0005) [2023-12-27 08:35:54,682][20347] Updated weights for policy 0, policy_version 427704 (0.0010) [2023-12-27 08:35:54,800][20381] Updated weights for policy 1, policy_version 427353 (0.0010) [2023-12-27 08:35:54,851][20381] Updated weights for policy 1, policy_version 427363 (0.0010) [2023-12-27 08:35:54,908][20381] Updated weights for policy 1, policy_version 427373 (0.0010) [2023-12-27 08:35:55,341][20347] Updated weights for policy 0, policy_version 427714 (0.0010) [2023-12-27 08:35:55,394][20347] Updated weights for policy 0, policy_version 427724 (0.0008) [2023-12-27 08:35:55,442][20347] Updated weights for policy 0, policy_version 427734 (0.0010) [2023-12-27 08:35:55,492][20347] Updated weights for policy 0, policy_version 427744 (0.0010) [2023-12-27 08:35:55,577][20381] Updated weights for policy 1, policy_version 427383 (0.0008) [2023-12-27 08:35:55,626][20381] Updated weights for policy 1, policy_version 427393 (0.0008) [2023-12-27 08:35:55,678][20381] Updated weights for policy 1, policy_version 427403 (0.0007) [2023-12-27 08:35:56,188][20347] Updated weights for policy 0, policy_version 427754 (0.0006) [2023-12-27 08:35:56,235][20347] Updated weights for policy 0, policy_version 427764 (0.0005) [2023-12-27 08:35:56,288][20347] Updated weights for policy 0, policy_version 427774 (0.0005) [2023-12-27 08:35:56,340][20381] Updated weights for policy 1, policy_version 427413 (0.0009) [2023-12-27 08:35:56,392][20381] Updated weights for policy 1, policy_version 427423 (0.0009) [2023-12-27 08:35:56,443][20381] Updated weights for policy 1, policy_version 427433 (0.0008) [2023-12-27 08:35:56,909][20347] Updated weights for policy 0, policy_version 427784 (0.0007) [2023-12-27 08:35:56,968][20347] Updated weights for policy 0, policy_version 427794 (0.0008) [2023-12-27 08:35:57,015][20347] Updated weights for policy 0, policy_version 427804 (0.0008) [2023-12-27 08:35:57,111][20381] Updated weights for policy 1, policy_version 427443 (0.0008) [2023-12-27 08:35:57,164][20381] Updated weights for policy 1, policy_version 427453 (0.0005) [2023-12-27 08:35:57,213][20381] Updated weights for policy 1, policy_version 427463 (0.0008) [2023-12-27 08:35:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 19524.3, 300 sec: 20160.7). Total num frames: 218988544. Throughput: 0: 10114.0, 1: 10301.6. Samples: 218974024. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:35:57,363][19170] Avg episode reward: [(0, '990.530'), (1, '1000.000')] [2023-12-27 08:35:57,676][20347] Updated weights for policy 0, policy_version 427814 (0.0009) [2023-12-27 08:35:57,724][20347] Updated weights for policy 0, policy_version 427824 (0.0009) [2023-12-27 08:35:57,783][20347] Updated weights for policy 0, policy_version 427834 (0.0008) [2023-12-27 08:35:57,872][20381] Updated weights for policy 1, policy_version 427473 (0.0008) [2023-12-27 08:35:57,928][20381] Updated weights for policy 1, policy_version 427483 (0.0008) [2023-12-27 08:35:57,985][20381] Updated weights for policy 1, policy_version 427493 (0.0008) [2023-12-27 08:35:58,029][20381] Updated weights for policy 1, policy_version 427503 (0.0007) [2023-12-27 08:35:58,466][20347] Updated weights for policy 0, policy_version 427844 (0.0008) [2023-12-27 08:35:58,532][20347] Updated weights for policy 0, policy_version 427854 (0.0007) [2023-12-27 08:35:58,593][20347] Updated weights for policy 0, policy_version 427864 (0.0007) [2023-12-27 08:35:58,791][20381] Updated weights for policy 1, policy_version 427513 (0.0007) [2023-12-27 08:35:58,861][20381] Updated weights for policy 1, policy_version 427523 (0.0008) [2023-12-27 08:35:58,925][20381] Updated weights for policy 1, policy_version 427533 (0.0007) [2023-12-27 08:35:59,368][20347] Updated weights for policy 0, policy_version 427874 (0.0007) [2023-12-27 08:35:59,426][20347] Updated weights for policy 0, policy_version 427884 (0.0008) [2023-12-27 08:35:59,480][20347] Updated weights for policy 0, policy_version 427894 (0.0007) [2023-12-27 08:35:59,529][20347] Updated weights for policy 0, policy_version 427904 (0.0005) [2023-12-27 08:35:59,603][20381] Updated weights for policy 1, policy_version 427543 (0.0008) [2023-12-27 08:35:59,657][20381] Updated weights for policy 1, policy_version 427553 (0.0007) [2023-12-27 08:35:59,714][20381] Updated weights for policy 1, policy_version 427563 (0.0007) [2023-12-27 08:36:00,224][20347] Updated weights for policy 0, policy_version 427914 (0.0009) [2023-12-27 08:36:00,269][20347] Updated weights for policy 0, policy_version 427924 (0.0009) [2023-12-27 08:36:00,317][20347] Updated weights for policy 0, policy_version 427934 (0.0007) [2023-12-27 08:36:00,340][20381] Updated weights for policy 1, policy_version 427573 (0.0007) [2023-12-27 08:36:00,381][20381] Updated weights for policy 1, policy_version 427583 (0.0007) [2023-12-27 08:36:00,434][20381] Updated weights for policy 1, policy_version 427593 (0.0006) [2023-12-27 08:36:00,975][20347] Updated weights for policy 0, policy_version 427944 (0.0010) [2023-12-27 08:36:01,026][20347] Updated weights for policy 0, policy_version 427954 (0.0010) [2023-12-27 08:36:01,084][20347] Updated weights for policy 0, policy_version 427964 (0.0010) [2023-12-27 08:36:01,138][20381] Updated weights for policy 1, policy_version 427603 (0.0009) [2023-12-27 08:36:01,197][20381] Updated weights for policy 1, policy_version 427613 (0.0006) [2023-12-27 08:36:01,241][20381] Updated weights for policy 1, policy_version 427623 (0.0007) [2023-12-27 08:36:01,817][20347] Updated weights for policy 0, policy_version 427974 (0.0009) [2023-12-27 08:36:01,869][20347] Updated weights for policy 0, policy_version 427984 (0.0007) [2023-12-27 08:36:01,921][20347] Updated weights for policy 0, policy_version 427994 (0.0008) [2023-12-27 08:36:01,953][20381] Updated weights for policy 1, policy_version 427633 (0.0008) [2023-12-27 08:36:02,018][20381] Updated weights for policy 1, policy_version 427643 (0.0006) [2023-12-27 08:36:02,071][20381] Updated weights for policy 1, policy_version 427653 (0.0007) [2023-12-27 08:36:02,126][20381] Updated weights for policy 1, policy_version 427663 (0.0008) [2023-12-27 08:36:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 20132.9). Total num frames: 219086848. Throughput: 0: 10244.9, 1: 10435.0. Samples: 219095368. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:02,666][20347] Updated weights for policy 0, policy_version 428004 (0.0007) [2023-12-27 08:36:02,704][20381] Updated weights for policy 1, policy_version 427673 (0.0006) [2023-12-27 08:36:02,725][20347] Updated weights for policy 0, policy_version 428014 (0.0007) [2023-12-27 08:36:02,760][20381] Updated weights for policy 1, policy_version 427683 (0.0005) [2023-12-27 08:36:02,776][20347] Updated weights for policy 0, policy_version 428024 (0.0008) [2023-12-27 08:36:02,822][20381] Updated weights for policy 1, policy_version 427693 (0.0007) [2023-12-27 08:36:03,453][20381] Updated weights for policy 1, policy_version 427703 (0.0006) [2023-12-27 08:36:03,469][20347] Updated weights for policy 0, policy_version 428034 (0.0006) [2023-12-27 08:36:03,510][20381] Updated weights for policy 1, policy_version 427713 (0.0007) [2023-12-27 08:36:03,521][20347] Updated weights for policy 0, policy_version 428044 (0.0005) [2023-12-27 08:36:03,558][20381] Updated weights for policy 1, policy_version 427723 (0.0007) [2023-12-27 08:36:03,571][20347] Updated weights for policy 0, policy_version 428054 (0.0005) [2023-12-27 08:36:03,629][20347] Updated weights for policy 0, policy_version 428064 (0.0006) [2023-12-27 08:36:04,237][20381] Updated weights for policy 1, policy_version 427733 (0.0007) [2023-12-27 08:36:04,272][20347] Updated weights for policy 0, policy_version 428074 (0.0008) [2023-12-27 08:36:04,297][20381] Updated weights for policy 1, policy_version 427743 (0.0006) [2023-12-27 08:36:04,325][20347] Updated weights for policy 0, policy_version 428084 (0.0011) [2023-12-27 08:36:04,359][20381] Updated weights for policy 1, policy_version 427753 (0.0006) [2023-12-27 08:36:04,390][20347] Updated weights for policy 0, policy_version 428094 (0.0009) [2023-12-27 08:36:05,054][20381] Updated weights for policy 1, policy_version 427763 (0.0008) [2023-12-27 08:36:05,075][20347] Updated weights for policy 0, policy_version 428104 (0.0009) [2023-12-27 08:36:05,103][20381] Updated weights for policy 1, policy_version 427773 (0.0005) [2023-12-27 08:36:05,134][20347] Updated weights for policy 0, policy_version 428114 (0.0007) [2023-12-27 08:36:05,158][20381] Updated weights for policy 1, policy_version 427783 (0.0006) [2023-12-27 08:36:05,191][20347] Updated weights for policy 0, policy_version 428124 (0.0007) [2023-12-27 08:36:05,719][20381] Updated weights for policy 1, policy_version 427793 (0.0007) [2023-12-27 08:36:05,774][20381] Updated weights for policy 1, policy_version 427803 (0.0005) [2023-12-27 08:36:05,828][20381] Updated weights for policy 1, policy_version 427813 (0.0007) [2023-12-27 08:36:05,850][20347] Updated weights for policy 0, policy_version 428134 (0.0006) [2023-12-27 08:36:05,875][20381] Updated weights for policy 1, policy_version 427823 (0.0008) [2023-12-27 08:36:05,911][20347] Updated weights for policy 0, policy_version 428144 (0.0006) [2023-12-27 08:36:05,959][20347] Updated weights for policy 0, policy_version 428154 (0.0007) [2023-12-27 08:36:06,532][20381] Updated weights for policy 1, policy_version 427833 (0.0008) [2023-12-27 08:36:06,597][20381] Updated weights for policy 1, policy_version 427843 (0.0008) [2023-12-27 08:36:06,661][20381] Updated weights for policy 1, policy_version 427853 (0.0008) [2023-12-27 08:36:06,665][20347] Updated weights for policy 0, policy_version 428164 (0.0009) [2023-12-27 08:36:06,726][20347] Updated weights for policy 0, policy_version 428174 (0.0007) [2023-12-27 08:36:06,785][20347] Updated weights for policy 0, policy_version 428184 (0.0008) [2023-12-27 08:36:07,291][20381] Updated weights for policy 1, policy_version 427863 (0.0007) [2023-12-27 08:36:07,336][20381] Updated weights for policy 1, policy_version 427873 (0.0007) [2023-12-27 08:36:07,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20070.3, 300 sec: 20105.1). Total num frames: 219185152. Throughput: 0: 10230.6, 1: 10439.2. Samples: 219157332. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000428192_109633536.pth... [2023-12-27 08:36:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000427008_109330432.pth [2023-12-27 08:36:07,386][20381] Updated weights for policy 1, policy_version 427883 (0.0006) [2023-12-27 08:36:07,410][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000427888_109559808.pth... [2023-12-27 08:36:07,414][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000426640_109240320.pth [2023-12-27 08:36:07,506][20347] Updated weights for policy 0, policy_version 428194 (0.0007) [2023-12-27 08:36:07,554][20347] Updated weights for policy 0, policy_version 428204 (0.0007) [2023-12-27 08:36:07,605][20347] Updated weights for policy 0, policy_version 428214 (0.0007) [2023-12-27 08:36:07,659][20347] Updated weights for policy 0, policy_version 428224 (0.0007) [2023-12-27 08:36:07,975][20381] Updated weights for policy 1, policy_version 427893 (0.0006) [2023-12-27 08:36:08,023][20381] Updated weights for policy 1, policy_version 427903 (0.0006) [2023-12-27 08:36:08,078][20381] Updated weights for policy 1, policy_version 427913 (0.0008) [2023-12-27 08:36:08,365][20347] Updated weights for policy 0, policy_version 428234 (0.0007) [2023-12-27 08:36:08,426][20347] Updated weights for policy 0, policy_version 428244 (0.0008) [2023-12-27 08:36:08,485][20347] Updated weights for policy 0, policy_version 428254 (0.0008) [2023-12-27 08:36:08,801][20381] Updated weights for policy 1, policy_version 427923 (0.0008) [2023-12-27 08:36:08,854][20381] Updated weights for policy 1, policy_version 427933 (0.0008) [2023-12-27 08:36:08,909][20381] Updated weights for policy 1, policy_version 427943 (0.0008) [2023-12-27 08:36:09,167][20347] Updated weights for policy 0, policy_version 428264 (0.0006) [2023-12-27 08:36:09,235][20347] Updated weights for policy 0, policy_version 428274 (0.0007) [2023-12-27 08:36:09,290][20347] Updated weights for policy 0, policy_version 428284 (0.0007) [2023-12-27 08:36:09,617][20381] Updated weights for policy 1, policy_version 427953 (0.0007) [2023-12-27 08:36:09,675][20381] Updated weights for policy 1, policy_version 427963 (0.0005) [2023-12-27 08:36:09,727][20381] Updated weights for policy 1, policy_version 427973 (0.0005) [2023-12-27 08:36:09,791][20381] Updated weights for policy 1, policy_version 427983 (0.0010) [2023-12-27 08:36:09,937][20347] Updated weights for policy 0, policy_version 428294 (0.0007) [2023-12-27 08:36:09,999][20347] Updated weights for policy 0, policy_version 428304 (0.0008) [2023-12-27 08:36:10,057][20347] Updated weights for policy 0, policy_version 428314 (0.0007) [2023-12-27 08:36:10,465][20381] Updated weights for policy 1, policy_version 427993 (0.0008) [2023-12-27 08:36:10,525][20381] Updated weights for policy 1, policy_version 428003 (0.0007) [2023-12-27 08:36:10,583][20381] Updated weights for policy 1, policy_version 428013 (0.0007) [2023-12-27 08:36:10,776][20347] Updated weights for policy 0, policy_version 428324 (0.0008) [2023-12-27 08:36:10,830][20347] Updated weights for policy 0, policy_version 428334 (0.0005) [2023-12-27 08:36:10,895][20347] Updated weights for policy 0, policy_version 428344 (0.0007) [2023-12-27 08:36:11,260][20381] Updated weights for policy 1, policy_version 428023 (0.0010) [2023-12-27 08:36:11,316][20381] Updated weights for policy 1, policy_version 428033 (0.0010) [2023-12-27 08:36:11,385][20381] Updated weights for policy 1, policy_version 428043 (0.0010) [2023-12-27 08:36:11,583][20347] Updated weights for policy 0, policy_version 428354 (0.0007) [2023-12-27 08:36:11,647][20347] Updated weights for policy 0, policy_version 428364 (0.0007) [2023-12-27 08:36:11,707][20347] Updated weights for policy 0, policy_version 428374 (0.0006) [2023-12-27 08:36:11,773][20347] Updated weights for policy 0, policy_version 428384 (0.0010) [2023-12-27 08:36:12,121][20381] Updated weights for policy 1, policy_version 428053 (0.0009) [2023-12-27 08:36:12,179][20381] Updated weights for policy 1, policy_version 428063 (0.0009) [2023-12-27 08:36:12,231][20381] Updated weights for policy 1, policy_version 428073 (0.0007) [2023-12-27 08:36:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20132.9). Total num frames: 219291648. Throughput: 0: 10211.6, 1: 10539.4. Samples: 219281432. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:12,420][20347] Updated weights for policy 0, policy_version 428394 (0.0008) [2023-12-27 08:36:12,479][20347] Updated weights for policy 0, policy_version 428404 (0.0007) [2023-12-27 08:36:12,546][20347] Updated weights for policy 0, policy_version 428414 (0.0008) [2023-12-27 08:36:12,904][20381] Updated weights for policy 1, policy_version 428083 (0.0007) [2023-12-27 08:36:12,965][20381] Updated weights for policy 1, policy_version 428093 (0.0007) [2023-12-27 08:36:13,028][20381] Updated weights for policy 1, policy_version 428103 (0.0007) [2023-12-27 08:36:13,220][20347] Updated weights for policy 0, policy_version 428424 (0.0006) [2023-12-27 08:36:13,280][20347] Updated weights for policy 0, policy_version 428434 (0.0006) [2023-12-27 08:36:13,351][20347] Updated weights for policy 0, policy_version 428444 (0.0005) [2023-12-27 08:36:13,595][20381] Updated weights for policy 1, policy_version 428113 (0.0005) [2023-12-27 08:36:13,651][20381] Updated weights for policy 1, policy_version 428123 (0.0005) [2023-12-27 08:36:13,713][20381] Updated weights for policy 1, policy_version 428133 (0.0006) [2023-12-27 08:36:13,761][20381] Updated weights for policy 1, policy_version 428143 (0.0007) [2023-12-27 08:36:13,849][20347] Updated weights for policy 0, policy_version 428454 (0.0006) [2023-12-27 08:36:13,903][20347] Updated weights for policy 0, policy_version 428464 (0.0006) [2023-12-27 08:36:13,952][20347] Updated weights for policy 0, policy_version 428474 (0.0010) [2023-12-27 08:36:14,386][20381] Updated weights for policy 1, policy_version 428153 (0.0006) [2023-12-27 08:36:14,440][20381] Updated weights for policy 1, policy_version 428163 (0.0005) [2023-12-27 08:36:14,487][20381] Updated weights for policy 1, policy_version 428173 (0.0007) [2023-12-27 08:36:14,628][20347] Updated weights for policy 0, policy_version 428484 (0.0011) [2023-12-27 08:36:14,681][20347] Updated weights for policy 0, policy_version 428494 (0.0008) [2023-12-27 08:36:14,725][20347] Updated weights for policy 0, policy_version 428504 (0.0008) [2023-12-27 08:36:15,140][20381] Updated weights for policy 1, policy_version 428183 (0.0008) [2023-12-27 08:36:15,199][20381] Updated weights for policy 1, policy_version 428193 (0.0008) [2023-12-27 08:36:15,261][20381] Updated weights for policy 1, policy_version 428203 (0.0007) [2023-12-27 08:36:15,405][20347] Updated weights for policy 0, policy_version 428514 (0.0008) [2023-12-27 08:36:15,465][20347] Updated weights for policy 0, policy_version 428524 (0.0009) [2023-12-27 08:36:15,521][20347] Updated weights for policy 0, policy_version 428534 (0.0005) [2023-12-27 08:36:15,581][20347] Updated weights for policy 0, policy_version 428544 (0.0006) [2023-12-27 08:36:15,940][20381] Updated weights for policy 1, policy_version 428213 (0.0009) [2023-12-27 08:36:16,002][20381] Updated weights for policy 1, policy_version 428223 (0.0010) [2023-12-27 08:36:16,056][20381] Updated weights for policy 1, policy_version 428233 (0.0010) [2023-12-27 08:36:16,177][20347] Updated weights for policy 0, policy_version 428554 (0.0008) [2023-12-27 08:36:16,225][20347] Updated weights for policy 0, policy_version 428564 (0.0010) [2023-12-27 08:36:16,273][20347] Updated weights for policy 0, policy_version 428574 (0.0010) [2023-12-27 08:36:16,671][20381] Updated weights for policy 1, policy_version 428243 (0.0010) [2023-12-27 08:36:16,730][20381] Updated weights for policy 1, policy_version 428253 (0.0011) [2023-12-27 08:36:16,789][20381] Updated weights for policy 1, policy_version 428263 (0.0010) [2023-12-27 08:36:16,888][20347] Updated weights for policy 0, policy_version 428584 (0.0010) [2023-12-27 08:36:16,937][20347] Updated weights for policy 0, policy_version 428594 (0.0009) [2023-12-27 08:36:16,988][20347] Updated weights for policy 0, policy_version 428604 (0.0008) [2023-12-27 08:36:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.0, 300 sec: 20160.7). Total num frames: 219398144. Throughput: 0: 10305.3, 1: 10508.1. Samples: 219408612. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:17,480][20381] Updated weights for policy 1, policy_version 428273 (0.0010) [2023-12-27 08:36:17,542][20381] Updated weights for policy 1, policy_version 428283 (0.0006) [2023-12-27 08:36:17,602][20381] Updated weights for policy 1, policy_version 428293 (0.0006) [2023-12-27 08:36:17,629][20347] Updated weights for policy 0, policy_version 428614 (0.0008) [2023-12-27 08:36:17,664][20381] Updated weights for policy 1, policy_version 428303 (0.0010) [2023-12-27 08:36:17,687][20347] Updated weights for policy 0, policy_version 428624 (0.0005) [2023-12-27 08:36:17,742][20347] Updated weights for policy 0, policy_version 428634 (0.0005) [2023-12-27 08:36:18,222][20381] Updated weights for policy 1, policy_version 428313 (0.0009) [2023-12-27 08:36:18,267][20381] Updated weights for policy 1, policy_version 428323 (0.0010) [2023-12-27 08:36:18,299][20347] Updated weights for policy 0, policy_version 428644 (0.0005) [2023-12-27 08:36:18,318][20381] Updated weights for policy 1, policy_version 428333 (0.0007) [2023-12-27 08:36:18,360][20347] Updated weights for policy 0, policy_version 428654 (0.0008) [2023-12-27 08:36:18,419][20347] Updated weights for policy 0, policy_version 428664 (0.0006) [2023-12-27 08:36:19,030][20347] Updated weights for policy 0, policy_version 428674 (0.0007) [2023-12-27 08:36:19,063][20381] Updated weights for policy 1, policy_version 428343 (0.0006) [2023-12-27 08:36:19,088][20347] Updated weights for policy 0, policy_version 428684 (0.0006) [2023-12-27 08:36:19,108][20381] Updated weights for policy 1, policy_version 428353 (0.0007) [2023-12-27 08:36:19,144][20347] Updated weights for policy 0, policy_version 428694 (0.0005) [2023-12-27 08:36:19,159][20381] Updated weights for policy 1, policy_version 428363 (0.0006) [2023-12-27 08:36:19,197][20347] Updated weights for policy 0, policy_version 428704 (0.0006) [2023-12-27 08:36:19,858][20347] Updated weights for policy 0, policy_version 428714 (0.0007) [2023-12-27 08:36:19,907][20381] Updated weights for policy 1, policy_version 428373 (0.0007) [2023-12-27 08:36:19,917][20347] Updated weights for policy 0, policy_version 428724 (0.0006) [2023-12-27 08:36:19,968][20381] Updated weights for policy 1, policy_version 428383 (0.0006) [2023-12-27 08:36:19,974][20347] Updated weights for policy 0, policy_version 428734 (0.0007) [2023-12-27 08:36:20,030][20381] Updated weights for policy 1, policy_version 428393 (0.0006) [2023-12-27 08:36:20,688][20381] Updated weights for policy 1, policy_version 428403 (0.0006) [2023-12-27 08:36:20,719][20347] Updated weights for policy 0, policy_version 428744 (0.0009) [2023-12-27 08:36:20,748][20381] Updated weights for policy 1, policy_version 428413 (0.0006) [2023-12-27 08:36:20,779][20347] Updated weights for policy 0, policy_version 428754 (0.0010) [2023-12-27 08:36:20,807][20381] Updated weights for policy 1, policy_version 428423 (0.0005) [2023-12-27 08:36:20,841][20347] Updated weights for policy 0, policy_version 428764 (0.0010) [2023-12-27 08:36:21,550][20381] Updated weights for policy 1, policy_version 428433 (0.0006) [2023-12-27 08:36:21,615][20347] Updated weights for policy 0, policy_version 428774 (0.0010) [2023-12-27 08:36:21,615][20381] Updated weights for policy 1, policy_version 428443 (0.0009) [2023-12-27 08:36:21,678][20381] Updated weights for policy 1, policy_version 428453 (0.0007) [2023-12-27 08:36:21,680][20347] Updated weights for policy 0, policy_version 428784 (0.0008) [2023-12-27 08:36:21,740][20381] Updated weights for policy 1, policy_version 428463 (0.0008) [2023-12-27 08:36:21,745][20347] Updated weights for policy 0, policy_version 428794 (0.0008) [2023-12-27 08:36:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20132.9). Total num frames: 219496448. Throughput: 0: 10292.6, 1: 10440.8. Samples: 219472288. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000428464_109707264.pth... [2023-12-27 08:36:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000427248_109395968.pth [2023-12-27 08:36:22,385][20347] Updated weights for policy 0, policy_version 428804 (0.0008) [2023-12-27 08:36:22,445][20347] Updated weights for policy 0, policy_version 428814 (0.0008) [2023-12-27 08:36:22,451][20381] Updated weights for policy 1, policy_version 428473 (0.0007) [2023-12-27 08:36:22,502][20347] Updated weights for policy 0, policy_version 428824 (0.0006) [2023-12-27 08:36:22,512][20381] Updated weights for policy 1, policy_version 428483 (0.0008) [2023-12-27 08:36:22,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000428832_109797376.pth... [2023-12-27 08:36:22,548][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000427616_109486080.pth [2023-12-27 08:36:22,573][20381] Updated weights for policy 1, policy_version 428493 (0.0008) [2023-12-27 08:36:23,119][20347] Updated weights for policy 0, policy_version 428834 (0.0006) [2023-12-27 08:36:23,179][20347] Updated weights for policy 0, policy_version 428844 (0.0007) [2023-12-27 08:36:23,230][20347] Updated weights for policy 0, policy_version 428854 (0.0007) [2023-12-27 08:36:23,265][20381] Updated weights for policy 1, policy_version 428503 (0.0008) [2023-12-27 08:36:23,293][20347] Updated weights for policy 0, policy_version 428864 (0.0007) [2023-12-27 08:36:23,318][20381] Updated weights for policy 1, policy_version 428513 (0.0007) [2023-12-27 08:36:23,370][20381] Updated weights for policy 1, policy_version 428523 (0.0008) [2023-12-27 08:36:23,901][20347] Updated weights for policy 0, policy_version 428874 (0.0005) [2023-12-27 08:36:23,958][20347] Updated weights for policy 0, policy_version 428884 (0.0005) [2023-12-27 08:36:24,017][20347] Updated weights for policy 0, policy_version 428894 (0.0005) [2023-12-27 08:36:24,116][20381] Updated weights for policy 1, policy_version 428533 (0.0008) [2023-12-27 08:36:24,179][20381] Updated weights for policy 1, policy_version 428543 (0.0007) [2023-12-27 08:36:24,230][20381] Updated weights for policy 1, policy_version 428553 (0.0008) [2023-12-27 08:36:24,621][20347] Updated weights for policy 0, policy_version 428904 (0.0010) [2023-12-27 08:36:24,669][20347] Updated weights for policy 0, policy_version 428914 (0.0010) [2023-12-27 08:36:24,717][20347] Updated weights for policy 0, policy_version 428924 (0.0010) [2023-12-27 08:36:24,892][20381] Updated weights for policy 1, policy_version 428563 (0.0006) [2023-12-27 08:36:24,941][20381] Updated weights for policy 1, policy_version 428573 (0.0007) [2023-12-27 08:36:24,990][20381] Updated weights for policy 1, policy_version 428583 (0.0007) [2023-12-27 08:36:25,342][20347] Updated weights for policy 0, policy_version 428934 (0.0007) [2023-12-27 08:36:25,390][20347] Updated weights for policy 0, policy_version 428944 (0.0005) [2023-12-27 08:36:25,439][20347] Updated weights for policy 0, policy_version 428954 (0.0007) [2023-12-27 08:36:25,700][20381] Updated weights for policy 1, policy_version 428593 (0.0009) [2023-12-27 08:36:25,755][20381] Updated weights for policy 1, policy_version 428603 (0.0008) [2023-12-27 08:36:25,806][20381] Updated weights for policy 1, policy_version 428613 (0.0007) [2023-12-27 08:36:25,854][20381] Updated weights for policy 1, policy_version 428623 (0.0007) [2023-12-27 08:36:26,090][20347] Updated weights for policy 0, policy_version 428964 (0.0011) [2023-12-27 08:36:26,145][20347] Updated weights for policy 0, policy_version 428974 (0.0010) [2023-12-27 08:36:26,201][20347] Updated weights for policy 0, policy_version 428984 (0.0010) [2023-12-27 08:36:26,560][20381] Updated weights for policy 1, policy_version 428633 (0.0007) [2023-12-27 08:36:26,619][20381] Updated weights for policy 1, policy_version 428643 (0.0006) [2023-12-27 08:36:26,671][20381] Updated weights for policy 1, policy_version 428653 (0.0007) [2023-12-27 08:36:26,852][20347] Updated weights for policy 0, policy_version 428994 (0.0009) [2023-12-27 08:36:26,903][20347] Updated weights for policy 0, policy_version 429004 (0.0006) [2023-12-27 08:36:26,948][20347] Updated weights for policy 0, policy_version 429014 (0.0010) [2023-12-27 08:36:27,002][20347] Updated weights for policy 0, policy_version 429024 (0.0010) [2023-12-27 08:36:27,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20132.9). Total num frames: 219602944. Throughput: 0: 10352.2, 1: 10390.4. Samples: 219595148. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:27,370][20381] Updated weights for policy 1, policy_version 428663 (0.0007) [2023-12-27 08:36:27,426][20381] Updated weights for policy 1, policy_version 428673 (0.0007) [2023-12-27 08:36:27,480][20381] Updated weights for policy 1, policy_version 428683 (0.0005) [2023-12-27 08:36:27,676][20347] Updated weights for policy 0, policy_version 429034 (0.0010) [2023-12-27 08:36:27,730][20347] Updated weights for policy 0, policy_version 429044 (0.0010) [2023-12-27 08:36:27,787][20347] Updated weights for policy 0, policy_version 429054 (0.0010) [2023-12-27 08:36:28,160][20381] Updated weights for policy 1, policy_version 428693 (0.0007) [2023-12-27 08:36:28,213][20381] Updated weights for policy 1, policy_version 428703 (0.0007) [2023-12-27 08:36:28,263][20381] Updated weights for policy 1, policy_version 428713 (0.0007) [2023-12-27 08:36:28,426][20347] Updated weights for policy 0, policy_version 429064 (0.0009) [2023-12-27 08:36:28,477][20347] Updated weights for policy 0, policy_version 429074 (0.0010) [2023-12-27 08:36:28,525][20347] Updated weights for policy 0, policy_version 429084 (0.0010) [2023-12-27 08:36:29,022][20381] Updated weights for policy 1, policy_version 428723 (0.0008) [2023-12-27 08:36:29,094][20381] Updated weights for policy 1, policy_version 428733 (0.0010) [2023-12-27 08:36:29,159][20381] Updated weights for policy 1, policy_version 428743 (0.0008) [2023-12-27 08:36:29,312][20347] Updated weights for policy 0, policy_version 429094 (0.0010) [2023-12-27 08:36:29,385][20347] Updated weights for policy 0, policy_version 429104 (0.0008) [2023-12-27 08:36:29,453][20347] Updated weights for policy 0, policy_version 429114 (0.0007) [2023-12-27 08:36:29,976][20381] Updated weights for policy 1, policy_version 428753 (0.0008) [2023-12-27 08:36:30,043][20381] Updated weights for policy 1, policy_version 428763 (0.0008) [2023-12-27 08:36:30,110][20381] Updated weights for policy 1, policy_version 428773 (0.0009) [2023-12-27 08:36:30,184][20381] Updated weights for policy 1, policy_version 428783 (0.0008) [2023-12-27 08:36:30,277][20347] Updated weights for policy 0, policy_version 429124 (0.0010) [2023-12-27 08:36:30,341][20347] Updated weights for policy 0, policy_version 429134 (0.0008) [2023-12-27 08:36:30,410][20347] Updated weights for policy 0, policy_version 429144 (0.0008) [2023-12-27 08:36:31,097][20381] Updated weights for policy 1, policy_version 428793 (0.0008) [2023-12-27 08:36:31,164][20381] Updated weights for policy 1, policy_version 428803 (0.0008) [2023-12-27 08:36:31,237][20381] Updated weights for policy 1, policy_version 428813 (0.0008) [2023-12-27 08:36:31,320][20347] Updated weights for policy 0, policy_version 429154 (0.0008) [2023-12-27 08:36:31,386][20347] Updated weights for policy 0, policy_version 429164 (0.0008) [2023-12-27 08:36:31,456][20347] Updated weights for policy 0, policy_version 429174 (0.0008) [2023-12-27 08:36:32,096][20381] Updated weights for policy 1, policy_version 428823 (0.0008) [2023-12-27 08:36:32,163][20381] Updated weights for policy 1, policy_version 428833 (0.0008) [2023-12-27 08:36:32,231][20381] Updated weights for policy 1, policy_version 428843 (0.0008) [2023-12-27 08:36:32,295][20347] Updated weights for policy 0, policy_version 429185 (0.0007) [2023-12-27 08:36:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20077.3). Total num frames: 219693056. Throughput: 0: 10195.8, 1: 10165.7. Samples: 219703576. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:32,365][20347] Updated weights for policy 0, policy_version 429195 (0.0008) [2023-12-27 08:36:32,432][20347] Updated weights for policy 0, policy_version 429205 (0.0008) [2023-12-27 08:36:32,494][20347] Updated weights for policy 0, policy_version 429215 (0.0008) [2023-12-27 08:36:32,998][20381] Updated weights for policy 1, policy_version 428853 (0.0008) [2023-12-27 08:36:33,072][20381] Updated weights for policy 1, policy_version 428863 (0.0007) [2023-12-27 08:36:33,139][20381] Updated weights for policy 1, policy_version 428873 (0.0008) [2023-12-27 08:36:33,295][20347] Updated weights for policy 0, policy_version 429225 (0.0007) [2023-12-27 08:36:33,360][20347] Updated weights for policy 0, policy_version 429235 (0.0008) [2023-12-27 08:36:33,422][20347] Updated weights for policy 0, policy_version 429245 (0.0007) [2023-12-27 08:36:33,922][20381] Updated weights for policy 1, policy_version 428883 (0.0008) [2023-12-27 08:36:33,991][20381] Updated weights for policy 1, policy_version 428893 (0.0009) [2023-12-27 08:36:34,068][20381] Updated weights for policy 1, policy_version 428903 (0.0008) [2023-12-27 08:36:34,395][20347] Updated weights for policy 0, policy_version 429255 (0.0008) [2023-12-27 08:36:34,469][20347] Updated weights for policy 0, policy_version 429265 (0.0008) [2023-12-27 08:36:34,541][20347] Updated weights for policy 0, policy_version 429275 (0.0008) [2023-12-27 08:36:35,122][20381] Updated weights for policy 1, policy_version 428913 (0.0008) [2023-12-27 08:36:35,188][20381] Updated weights for policy 1, policy_version 428923 (0.0008) [2023-12-27 08:36:35,258][20381] Updated weights for policy 1, policy_version 428933 (0.0007) [2023-12-27 08:36:35,322][20381] Updated weights for policy 1, policy_version 428943 (0.0008) [2023-12-27 08:36:35,432][20347] Updated weights for policy 0, policy_version 429285 (0.0009) [2023-12-27 08:36:35,492][20347] Updated weights for policy 0, policy_version 429295 (0.0008) [2023-12-27 08:36:35,557][20347] Updated weights for policy 0, policy_version 429305 (0.0008) [2023-12-27 08:36:36,071][20381] Updated weights for policy 1, policy_version 428953 (0.0008) [2023-12-27 08:36:36,144][20381] Updated weights for policy 1, policy_version 428963 (0.0008) [2023-12-27 08:36:36,207][20381] Updated weights for policy 1, policy_version 428973 (0.0008) [2023-12-27 08:36:36,332][20347] Updated weights for policy 0, policy_version 429315 (0.0008) [2023-12-27 08:36:36,405][20347] Updated weights for policy 0, policy_version 429325 (0.0010) [2023-12-27 08:36:36,474][20347] Updated weights for policy 0, policy_version 429335 (0.0011) [2023-12-27 08:36:37,007][20381] Updated weights for policy 1, policy_version 428983 (0.0008) [2023-12-27 08:36:37,075][20381] Updated weights for policy 1, policy_version 428993 (0.0009) [2023-12-27 08:36:37,145][20381] Updated weights for policy 1, policy_version 429003 (0.0008) [2023-12-27 08:36:37,284][20347] Updated weights for policy 0, policy_version 429345 (0.0010) [2023-12-27 08:36:37,355][20347] Updated weights for policy 0, policy_version 429355 (0.0008) [2023-12-27 08:36:37,362][19170] Fps is (10 sec: 17203.3, 60 sec: 20070.4, 300 sec: 20021.8). Total num frames: 219774976. Throughput: 0: 10051.5, 1: 9979.8. Samples: 219751808. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:36:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429008_109846528.pth... [2023-12-27 08:36:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000427888_109559808.pth [2023-12-27 08:36:37,426][20347] Updated weights for policy 0, policy_version 429365 (0.0009) [2023-12-27 08:36:37,496][20347] Updated weights for policy 0, policy_version 429375 (0.0008) [2023-12-27 08:36:37,502][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000429376_109936640.pth... [2023-12-27 08:36:37,506][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000428192_109633536.pth [2023-12-27 08:36:37,925][20381] Updated weights for policy 1, policy_version 429013 (0.0009) [2023-12-27 08:36:37,996][20381] Updated weights for policy 1, policy_version 429023 (0.0008) [2023-12-27 08:36:38,065][20381] Updated weights for policy 1, policy_version 429033 (0.0008) [2023-12-27 08:36:38,283][20347] Updated weights for policy 0, policy_version 429385 (0.0009) [2023-12-27 08:36:38,368][20347] Updated weights for policy 0, policy_version 429395 (0.0009) [2023-12-27 08:36:38,438][20347] Updated weights for policy 0, policy_version 429405 (0.0008) [2023-12-27 08:36:38,860][20381] Updated weights for policy 1, policy_version 429043 (0.0008) [2023-12-27 08:36:38,929][20381] Updated weights for policy 1, policy_version 429053 (0.0008) [2023-12-27 08:36:38,998][20381] Updated weights for policy 1, policy_version 429063 (0.0007) [2023-12-27 08:36:39,289][20347] Updated weights for policy 0, policy_version 429415 (0.0009) [2023-12-27 08:36:39,367][20347] Updated weights for policy 0, policy_version 429425 (0.0008) [2023-12-27 08:36:39,443][20347] Updated weights for policy 0, policy_version 429435 (0.0009) [2023-12-27 08:36:39,952][20381] Updated weights for policy 1, policy_version 429073 (0.0009) [2023-12-27 08:36:40,030][20381] Updated weights for policy 1, policy_version 429083 (0.0009) [2023-12-27 08:36:40,106][20381] Updated weights for policy 1, policy_version 429093 (0.0009) [2023-12-27 08:36:40,181][20381] Updated weights for policy 1, policy_version 429103 (0.0008) [2023-12-27 08:36:40,389][20347] Updated weights for policy 0, policy_version 429445 (0.0010) [2023-12-27 08:36:40,481][20347] Updated weights for policy 0, policy_version 429456 (0.0008) [2023-12-27 08:36:40,556][20347] Updated weights for policy 0, policy_version 429466 (0.0009) [2023-12-27 08:36:41,170][20381] Updated weights for policy 1, policy_version 429113 (0.0009) [2023-12-27 08:36:41,250][20381] Updated weights for policy 1, policy_version 429123 (0.0009) [2023-12-27 08:36:41,321][20381] Updated weights for policy 1, policy_version 429133 (0.0009) [2023-12-27 08:36:41,519][20347] Updated weights for policy 0, policy_version 429476 (0.0009) [2023-12-27 08:36:41,593][20347] Updated weights for policy 0, policy_version 429486 (0.0009) [2023-12-27 08:36:41,662][20347] Updated weights for policy 0, policy_version 429496 (0.0009) [2023-12-27 08:36:42,225][20381] Updated weights for policy 1, policy_version 429143 (0.0009) [2023-12-27 08:36:42,302][20381] Updated weights for policy 1, policy_version 429153 (0.0009) [2023-12-27 08:36:42,362][19170] Fps is (10 sec: 15564.7, 60 sec: 19660.8, 300 sec: 19910.7). Total num frames: 219848704. Throughput: 0: 9775.9, 1: 9697.0. Samples: 219850308. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:42,363][19170] Avg episode reward: [(0, '990.360'), (1, '1000.000')] [2023-12-27 08:36:42,376][20381] Updated weights for policy 1, policy_version 429163 (0.0009) [2023-12-27 08:36:42,537][20347] Updated weights for policy 0, policy_version 429506 (0.0009) [2023-12-27 08:36:42,604][20347] Updated weights for policy 0, policy_version 429516 (0.0009) [2023-12-27 08:36:42,675][20347] Updated weights for policy 0, policy_version 429526 (0.0009) [2023-12-27 08:36:42,746][20347] Updated weights for policy 0, policy_version 429536 (0.0009) [2023-12-27 08:36:43,262][20381] Updated weights for policy 1, policy_version 429173 (0.0009) [2023-12-27 08:36:43,333][20381] Updated weights for policy 1, policy_version 429183 (0.0009) [2023-12-27 08:36:43,402][20381] Updated weights for policy 1, policy_version 429193 (0.0009) [2023-12-27 08:36:43,612][20347] Updated weights for policy 0, policy_version 429546 (0.0008) [2023-12-27 08:36:43,683][20347] Updated weights for policy 0, policy_version 429556 (0.0009) [2023-12-27 08:36:43,757][20347] Updated weights for policy 0, policy_version 429566 (0.0009) [2023-12-27 08:36:44,265][20381] Updated weights for policy 1, policy_version 429203 (0.0009) [2023-12-27 08:36:44,338][20381] Updated weights for policy 1, policy_version 429213 (0.0009) [2023-12-27 08:36:44,405][20381] Updated weights for policy 1, policy_version 429223 (0.0009) [2023-12-27 08:36:44,625][20347] Updated weights for policy 0, policy_version 429576 (0.0008) [2023-12-27 08:36:44,694][20347] Updated weights for policy 0, policy_version 429586 (0.0008) [2023-12-27 08:36:44,769][20347] Updated weights for policy 0, policy_version 429596 (0.0009) [2023-12-27 08:36:45,245][20381] Updated weights for policy 1, policy_version 429233 (0.0009) [2023-12-27 08:36:45,317][20381] Updated weights for policy 1, policy_version 429243 (0.0009) [2023-12-27 08:36:45,396][20381] Updated weights for policy 1, policy_version 429253 (0.0008) [2023-12-27 08:36:45,472][20381] Updated weights for policy 1, policy_version 429263 (0.0008) [2023-12-27 08:36:45,639][20347] Updated weights for policy 0, policy_version 429606 (0.0009) [2023-12-27 08:36:45,716][20347] Updated weights for policy 0, policy_version 429616 (0.0009) [2023-12-27 08:36:45,785][20347] Updated weights for policy 0, policy_version 429626 (0.0009) [2023-12-27 08:36:46,277][20381] Updated weights for policy 1, policy_version 429273 (0.0009) [2023-12-27 08:36:46,344][20381] Updated weights for policy 1, policy_version 429283 (0.0009) [2023-12-27 08:36:46,416][20381] Updated weights for policy 1, policy_version 429293 (0.0009) [2023-12-27 08:36:46,651][20347] Updated weights for policy 0, policy_version 429636 (0.0008) [2023-12-27 08:36:46,720][20347] Updated weights for policy 0, policy_version 429646 (0.0009) [2023-12-27 08:36:46,794][20347] Updated weights for policy 0, policy_version 429656 (0.0009) [2023-12-27 08:36:47,259][20381] Updated weights for policy 1, policy_version 429303 (0.0009) [2023-12-27 08:36:47,335][20381] Updated weights for policy 1, policy_version 429313 (0.0008) [2023-12-27 08:36:47,362][19170] Fps is (10 sec: 15564.8, 60 sec: 19251.2, 300 sec: 19855.2). Total num frames: 219930624. Throughput: 0: 9501.4, 1: 9407.0. Samples: 219946248. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:47,363][19170] Avg episode reward: [(0, '990.360'), (1, '1000.000')] [2023-12-27 08:36:47,407][20381] Updated weights for policy 1, policy_version 429323 (0.0008) [2023-12-27 08:36:47,652][20347] Updated weights for policy 0, policy_version 429666 (0.0008) [2023-12-27 08:36:47,722][20347] Updated weights for policy 0, policy_version 429676 (0.0009) [2023-12-27 08:36:47,797][20347] Updated weights for policy 0, policy_version 429686 (0.0008) [2023-12-27 08:36:47,872][20347] Updated weights for policy 0, policy_version 429696 (0.0008) [2023-12-27 08:36:48,203][20381] Updated weights for policy 1, policy_version 429333 (0.0008) [2023-12-27 08:36:48,274][20381] Updated weights for policy 1, policy_version 429343 (0.0009) [2023-12-27 08:36:48,350][20381] Updated weights for policy 1, policy_version 429353 (0.0008) [2023-12-27 08:36:48,703][20347] Updated weights for policy 0, policy_version 429706 (0.0009) [2023-12-27 08:36:48,777][20347] Updated weights for policy 0, policy_version 429716 (0.0010) [2023-12-27 08:36:48,851][20347] Updated weights for policy 0, policy_version 429726 (0.0008) [2023-12-27 08:36:49,185][20381] Updated weights for policy 1, policy_version 429363 (0.0010) [2023-12-27 08:36:49,261][20381] Updated weights for policy 1, policy_version 429373 (0.0009) [2023-12-27 08:36:49,338][20381] Updated weights for policy 1, policy_version 429383 (0.0011) [2023-12-27 08:36:49,717][20347] Updated weights for policy 0, policy_version 429736 (0.0009) [2023-12-27 08:36:49,790][20347] Updated weights for policy 0, policy_version 429746 (0.0009) [2023-12-27 08:36:49,861][20347] Updated weights for policy 0, policy_version 429756 (0.0008) [2023-12-27 08:36:50,263][20381] Updated weights for policy 1, policy_version 429393 (0.0009) [2023-12-27 08:36:50,334][20381] Updated weights for policy 1, policy_version 429403 (0.0009) [2023-12-27 08:36:50,412][20381] Updated weights for policy 1, policy_version 429413 (0.0008) [2023-12-27 08:36:50,484][20381] Updated weights for policy 1, policy_version 429423 (0.0009) [2023-12-27 08:36:50,692][20347] Updated weights for policy 0, policy_version 429766 (0.0008) [2023-12-27 08:36:50,769][20347] Updated weights for policy 0, policy_version 429776 (0.0009) [2023-12-27 08:36:50,838][20347] Updated weights for policy 0, policy_version 429786 (0.0009) [2023-12-27 08:36:51,323][20381] Updated weights for policy 1, policy_version 429433 (0.0008) [2023-12-27 08:36:51,395][20381] Updated weights for policy 1, policy_version 429443 (0.0009) [2023-12-27 08:36:51,464][20381] Updated weights for policy 1, policy_version 429453 (0.0007) [2023-12-27 08:36:51,679][20347] Updated weights for policy 0, policy_version 429796 (0.0008) [2023-12-27 08:36:51,763][20347] Updated weights for policy 0, policy_version 429806 (0.0009) [2023-12-27 08:36:51,832][20347] Updated weights for policy 0, policy_version 429816 (0.0008) [2023-12-27 08:36:52,308][20381] Updated weights for policy 1, policy_version 429463 (0.0007) [2023-12-27 08:36:52,363][19170] Fps is (10 sec: 16383.5, 60 sec: 18978.0, 300 sec: 19744.1). Total num frames: 220012544. Throughput: 0: 9360.2, 1: 9245.1. Samples: 219994572. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:52,364][19170] Avg episode reward: [(0, '990.360'), (1, '1000.000')] [2023-12-27 08:36:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000429824_110051328.pth... [2023-12-27 08:36:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000428832_109797376.pth [2023-12-27 08:36:52,384][20381] Updated weights for policy 1, policy_version 429473 (0.0009) [2023-12-27 08:36:52,456][20381] Updated weights for policy 1, policy_version 429483 (0.0008) [2023-12-27 08:36:52,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429488_109969408.pth... [2023-12-27 08:36:52,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000428464_109707264.pth [2023-12-27 08:36:52,680][20347] Updated weights for policy 0, policy_version 429826 (0.0008) [2023-12-27 08:36:52,753][20347] Updated weights for policy 0, policy_version 429836 (0.0007) [2023-12-27 08:36:52,824][20347] Updated weights for policy 0, policy_version 429846 (0.0008) [2023-12-27 08:36:52,901][20347] Updated weights for policy 0, policy_version 429856 (0.0008) [2023-12-27 08:36:53,281][20381] Updated weights for policy 1, policy_version 429493 (0.0009) [2023-12-27 08:36:53,349][20381] Updated weights for policy 1, policy_version 429503 (0.0009) [2023-12-27 08:36:53,419][20381] Updated weights for policy 1, policy_version 429513 (0.0008) [2023-12-27 08:36:53,695][20347] Updated weights for policy 0, policy_version 429866 (0.0008) [2023-12-27 08:36:53,766][20347] Updated weights for policy 0, policy_version 429876 (0.0008) [2023-12-27 08:36:53,837][20347] Updated weights for policy 0, policy_version 429886 (0.0008) [2023-12-27 08:36:54,206][20381] Updated weights for policy 1, policy_version 429523 (0.0008) [2023-12-27 08:36:54,273][20381] Updated weights for policy 1, policy_version 429533 (0.0010) [2023-12-27 08:36:54,339][20381] Updated weights for policy 1, policy_version 429543 (0.0009) [2023-12-27 08:36:54,632][20347] Updated weights for policy 0, policy_version 429896 (0.0010) [2023-12-27 08:36:54,703][20347] Updated weights for policy 0, policy_version 429906 (0.0010) [2023-12-27 08:36:54,772][20347] Updated weights for policy 0, policy_version 429916 (0.0009) [2023-12-27 08:36:55,145][20381] Updated weights for policy 1, policy_version 429553 (0.0008) [2023-12-27 08:36:55,217][20381] Updated weights for policy 1, policy_version 429563 (0.0008) [2023-12-27 08:36:55,288][20381] Updated weights for policy 1, policy_version 429573 (0.0008) [2023-12-27 08:36:55,357][20381] Updated weights for policy 1, policy_version 429583 (0.0009) [2023-12-27 08:36:55,557][20347] Updated weights for policy 0, policy_version 429926 (0.0009) [2023-12-27 08:36:55,631][20347] Updated weights for policy 0, policy_version 429936 (0.0009) [2023-12-27 08:36:55,703][20347] Updated weights for policy 0, policy_version 429946 (0.0009) [2023-12-27 08:36:56,132][20381] Updated weights for policy 1, policy_version 429593 (0.0008) [2023-12-27 08:36:56,197][20381] Updated weights for policy 1, policy_version 429603 (0.0008) [2023-12-27 08:36:56,265][20381] Updated weights for policy 1, policy_version 429613 (0.0008) [2023-12-27 08:36:56,488][20347] Updated weights for policy 0, policy_version 429956 (0.0008) [2023-12-27 08:36:56,552][20347] Updated weights for policy 0, policy_version 429966 (0.0008) [2023-12-27 08:36:56,618][20347] Updated weights for policy 0, policy_version 429976 (0.0008) [2023-12-27 08:36:57,024][20381] Updated weights for policy 1, policy_version 429623 (0.0009) [2023-12-27 08:36:57,087][20381] Updated weights for policy 1, policy_version 429633 (0.0007) [2023-12-27 08:36:57,155][20381] Updated weights for policy 1, policy_version 429643 (0.0009) [2023-12-27 08:36:57,362][19170] Fps is (10 sec: 17203.1, 60 sec: 18568.5, 300 sec: 19716.3). Total num frames: 220102656. Throughput: 0: 9152.1, 1: 8945.7. Samples: 220095832. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:36:57,363][19170] Avg episode reward: [(0, '990.450'), (1, '1000.000')] [2023-12-27 08:36:57,422][20347] Updated weights for policy 0, policy_version 429986 (0.0008) [2023-12-27 08:36:57,493][20347] Updated weights for policy 0, policy_version 429996 (0.0008) [2023-12-27 08:36:57,562][20347] Updated weights for policy 0, policy_version 430006 (0.0008) [2023-12-27 08:36:57,631][20347] Updated weights for policy 0, policy_version 430016 (0.0008) [2023-12-27 08:36:57,936][20381] Updated weights for policy 1, policy_version 429653 (0.0008) [2023-12-27 08:36:58,007][20381] Updated weights for policy 1, policy_version 429663 (0.0009) [2023-12-27 08:36:58,073][20381] Updated weights for policy 1, policy_version 429673 (0.0007) [2023-12-27 08:36:58,398][20347] Updated weights for policy 0, policy_version 430026 (0.0008) [2023-12-27 08:36:58,467][20347] Updated weights for policy 0, policy_version 430036 (0.0008) [2023-12-27 08:36:58,541][20347] Updated weights for policy 0, policy_version 430046 (0.0008) [2023-12-27 08:36:58,839][20381] Updated weights for policy 1, policy_version 429683 (0.0007) [2023-12-27 08:36:58,904][20381] Updated weights for policy 1, policy_version 429693 (0.0008) [2023-12-27 08:36:58,974][20381] Updated weights for policy 1, policy_version 429703 (0.0007) [2023-12-27 08:36:59,339][20347] Updated weights for policy 0, policy_version 430056 (0.0007) [2023-12-27 08:36:59,410][20347] Updated weights for policy 0, policy_version 430066 (0.0008) [2023-12-27 08:36:59,466][20347] Updated weights for policy 0, policy_version 430076 (0.0008) [2023-12-27 08:36:59,788][20381] Updated weights for policy 1, policy_version 429713 (0.0010) [2023-12-27 08:36:59,864][20381] Updated weights for policy 1, policy_version 429723 (0.0010) [2023-12-27 08:36:59,936][20381] Updated weights for policy 1, policy_version 429733 (0.0010) [2023-12-27 08:37:00,018][20381] Updated weights for policy 1, policy_version 429743 (0.0009) [2023-12-27 08:37:00,307][20347] Updated weights for policy 0, policy_version 430086 (0.0008) [2023-12-27 08:37:00,381][20347] Updated weights for policy 0, policy_version 430096 (0.0007) [2023-12-27 08:37:00,452][20347] Updated weights for policy 0, policy_version 430106 (0.0009) [2023-12-27 08:37:00,890][20381] Updated weights for policy 1, policy_version 429753 (0.0009) [2023-12-27 08:37:00,957][20381] Updated weights for policy 1, policy_version 429763 (0.0009) [2023-12-27 08:37:01,030][20381] Updated weights for policy 1, policy_version 429773 (0.0009) [2023-12-27 08:37:01,289][20347] Updated weights for policy 0, policy_version 430116 (0.0009) [2023-12-27 08:37:01,363][20347] Updated weights for policy 0, policy_version 430126 (0.0008) [2023-12-27 08:37:01,437][20347] Updated weights for policy 0, policy_version 430136 (0.0008) [2023-12-27 08:37:01,871][20381] Updated weights for policy 1, policy_version 429783 (0.0009) [2023-12-27 08:37:01,943][20381] Updated weights for policy 1, policy_version 429793 (0.0008) [2023-12-27 08:37:02,015][20381] Updated weights for policy 1, policy_version 429803 (0.0009) [2023-12-27 08:37:02,263][20347] Updated weights for policy 0, policy_version 430146 (0.0009) [2023-12-27 08:37:02,337][20347] Updated weights for policy 0, policy_version 430156 (0.0008) [2023-12-27 08:37:02,362][19170] Fps is (10 sec: 17203.6, 60 sec: 18295.4, 300 sec: 19633.0). Total num frames: 220184576. Throughput: 0: 8845.5, 1: 8688.5. Samples: 220197644. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:37:02,363][19170] Avg episode reward: [(0, '971.210'), (1, '1000.000')] [2023-12-27 08:37:02,406][20347] Updated weights for policy 0, policy_version 430166 (0.0008) [2023-12-27 08:37:02,477][20347] Updated weights for policy 0, policy_version 430176 (0.0009) [2023-12-27 08:37:02,839][20381] Updated weights for policy 1, policy_version 429813 (0.0009) [2023-12-27 08:37:02,914][20381] Updated weights for policy 1, policy_version 429823 (0.0008) [2023-12-27 08:37:02,985][20381] Updated weights for policy 1, policy_version 429833 (0.0008) [2023-12-27 08:37:03,298][20347] Updated weights for policy 0, policy_version 430186 (0.0009) [2023-12-27 08:37:03,369][20347] Updated weights for policy 0, policy_version 430196 (0.0008) [2023-12-27 08:37:03,442][20347] Updated weights for policy 0, policy_version 430206 (0.0009) [2023-12-27 08:37:03,799][20381] Updated weights for policy 1, policy_version 429843 (0.0009) [2023-12-27 08:37:03,876][20381] Updated weights for policy 1, policy_version 429853 (0.0009) [2023-12-27 08:37:03,949][20381] Updated weights for policy 1, policy_version 429863 (0.0008) [2023-12-27 08:37:04,287][20347] Updated weights for policy 0, policy_version 430216 (0.0009) [2023-12-27 08:37:04,360][20347] Updated weights for policy 0, policy_version 430226 (0.0008) [2023-12-27 08:37:04,435][20347] Updated weights for policy 0, policy_version 430236 (0.0008) [2023-12-27 08:37:04,784][20381] Updated weights for policy 1, policy_version 429873 (0.0009) [2023-12-27 08:37:04,853][20381] Updated weights for policy 1, policy_version 429883 (0.0009) [2023-12-27 08:37:04,930][20381] Updated weights for policy 1, policy_version 429893 (0.0009) [2023-12-27 08:37:05,001][20381] Updated weights for policy 1, policy_version 429903 (0.0008) [2023-12-27 08:37:05,304][20347] Updated weights for policy 0, policy_version 430246 (0.0009) [2023-12-27 08:37:05,398][20347] Updated weights for policy 0, policy_version 430256 (0.0008) [2023-12-27 08:37:05,467][20347] Updated weights for policy 0, policy_version 430266 (0.0009) [2023-12-27 08:37:05,815][20381] Updated weights for policy 1, policy_version 429913 (0.0009) [2023-12-27 08:37:05,889][20381] Updated weights for policy 1, policy_version 429923 (0.0008) [2023-12-27 08:37:05,959][20381] Updated weights for policy 1, policy_version 429933 (0.0009) [2023-12-27 08:37:06,385][20347] Updated weights for policy 0, policy_version 430276 (0.0009) [2023-12-27 08:37:06,459][20291] KL-divergence is very high: 627.3004 [2023-12-27 08:37:06,459][20347] Updated weights for policy 0, policy_version 430286 (0.0009) [2023-12-27 08:37:06,531][20347] Updated weights for policy 0, policy_version 430296 (0.0009) [2023-12-27 08:37:06,803][20381] Updated weights for policy 1, policy_version 429943 (0.0010) [2023-12-27 08:37:06,878][20381] Updated weights for policy 1, policy_version 429953 (0.0009) [2023-12-27 08:37:06,951][20381] Updated weights for policy 1, policy_version 429963 (0.0008) [2023-12-27 08:37:07,362][19170] Fps is (10 sec: 16384.0, 60 sec: 18022.5, 300 sec: 19577.5). Total num frames: 220266496. Throughput: 0: 8660.3, 1: 8559.0. Samples: 220247156. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 08:37:07,363][19170] Avg episode reward: [(0, '961.710'), (1, '1000.000')] [2023-12-27 08:37:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429968_110092288.pth... [2023-12-27 08:37:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000430304_110174208.pth... [2023-12-27 08:37:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429008_109846528.pth [2023-12-27 08:37:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000429376_109936640.pth [2023-12-27 08:37:07,413][20347] Updated weights for policy 0, policy_version 430306 (0.0009) [2023-12-27 08:37:07,487][20347] Updated weights for policy 0, policy_version 430316 (0.0008) [2023-12-27 08:37:07,557][20347] Updated weights for policy 0, policy_version 430326 (0.0009) [2023-12-27 08:37:07,631][20347] Updated weights for policy 0, policy_version 430336 (0.0008) [2023-12-27 08:37:07,780][20381] Updated weights for policy 1, policy_version 429973 (0.0009) [2023-12-27 08:37:07,852][20381] Updated weights for policy 1, policy_version 429983 (0.0009) [2023-12-27 08:37:07,924][20381] Updated weights for policy 1, policy_version 429993 (0.0009) [2023-12-27 08:37:08,467][20347] Updated weights for policy 0, policy_version 430346 (0.0009) [2023-12-27 08:37:08,542][20347] Updated weights for policy 0, policy_version 430356 (0.0008) [2023-12-27 08:37:08,612][20347] Updated weights for policy 0, policy_version 430366 (0.0008) [2023-12-27 08:37:08,744][20381] Updated weights for policy 1, policy_version 430003 (0.0009) [2023-12-27 08:37:08,813][20381] Updated weights for policy 1, policy_version 430013 (0.0009) [2023-12-27 08:37:08,885][20381] Updated weights for policy 1, policy_version 430023 (0.0008) [2023-12-27 08:37:09,480][20347] Updated weights for policy 0, policy_version 430376 (0.0008) [2023-12-27 08:37:09,552][20347] Updated weights for policy 0, policy_version 430386 (0.0008) [2023-12-27 08:37:09,623][20347] Updated weights for policy 0, policy_version 430396 (0.0008) [2023-12-27 08:37:09,748][20381] Updated weights for policy 1, policy_version 430033 (0.0009) [2023-12-27 08:37:09,826][20381] Updated weights for policy 1, policy_version 430043 (0.0008) [2023-12-27 08:37:09,897][20381] Updated weights for policy 1, policy_version 430053 (0.0008) [2023-12-27 08:37:09,974][20381] Updated weights for policy 1, policy_version 430063 (0.0008) [2023-12-27 08:37:10,496][20347] Updated weights for policy 0, policy_version 430406 (0.0009) [2023-12-27 08:37:10,560][20347] Updated weights for policy 0, policy_version 430416 (0.0009) [2023-12-27 08:37:10,635][20347] Updated weights for policy 0, policy_version 430426 (0.0010) [2023-12-27 08:37:10,808][20381] Updated weights for policy 1, policy_version 430073 (0.0009) [2023-12-27 08:37:10,880][20381] Updated weights for policy 1, policy_version 430083 (0.0009) [2023-12-27 08:37:10,948][20381] Updated weights for policy 1, policy_version 430093 (0.0008) [2023-12-27 08:37:11,561][20347] Updated weights for policy 0, policy_version 430436 (0.0010) [2023-12-27 08:37:11,634][20347] Updated weights for policy 0, policy_version 430446 (0.0010) [2023-12-27 08:37:11,706][20347] Updated weights for policy 0, policy_version 430456 (0.0010) [2023-12-27 08:37:11,830][20381] Updated weights for policy 1, policy_version 430103 (0.0009) [2023-12-27 08:37:11,902][20381] Updated weights for policy 1, policy_version 430113 (0.0009) [2023-12-27 08:37:11,977][20381] Updated weights for policy 1, policy_version 430123 (0.0009) [2023-12-27 08:37:12,362][19170] Fps is (10 sec: 16384.1, 60 sec: 17612.8, 300 sec: 19494.2). Total num frames: 220348416. Throughput: 0: 8302.1, 1: 8340.5. Samples: 220344068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:12,363][19170] Avg episode reward: [(0, '961.880'), (1, '990.370')] [2023-12-27 08:37:12,527][20347] Updated weights for policy 0, policy_version 430466 (0.0010) [2023-12-27 08:37:12,596][20347] Updated weights for policy 0, policy_version 430476 (0.0010) [2023-12-27 08:37:12,661][20347] Updated weights for policy 0, policy_version 430486 (0.0010) [2023-12-27 08:37:12,734][20347] Updated weights for policy 0, policy_version 430496 (0.0010) [2023-12-27 08:37:12,848][20381] Updated weights for policy 1, policy_version 430133 (0.0009) [2023-12-27 08:37:12,917][20381] Updated weights for policy 1, policy_version 430143 (0.0009) [2023-12-27 08:37:12,992][20381] Updated weights for policy 1, policy_version 430153 (0.0009) [2023-12-27 08:37:13,537][20347] Updated weights for policy 0, policy_version 430506 (0.0010) [2023-12-27 08:37:13,606][20347] Updated weights for policy 0, policy_version 430516 (0.0009) [2023-12-27 08:37:13,673][20347] Updated weights for policy 0, policy_version 430526 (0.0010) [2023-12-27 08:37:13,785][20381] Updated weights for policy 1, policy_version 430163 (0.0007) [2023-12-27 08:37:13,851][20381] Updated weights for policy 1, policy_version 430173 (0.0010) [2023-12-27 08:37:13,920][20381] Updated weights for policy 1, policy_version 430183 (0.0010) [2023-12-27 08:37:14,427][20347] Updated weights for policy 0, policy_version 430536 (0.0009) [2023-12-27 08:37:14,497][20347] Updated weights for policy 0, policy_version 430546 (0.0011) [2023-12-27 08:37:14,561][20347] Updated weights for policy 0, policy_version 430556 (0.0010) [2023-12-27 08:37:14,694][20381] Updated weights for policy 1, policy_version 430193 (0.0010) [2023-12-27 08:37:14,759][20381] Updated weights for policy 1, policy_version 430203 (0.0010) [2023-12-27 08:37:14,830][20381] Updated weights for policy 1, policy_version 430213 (0.0008) [2023-12-27 08:37:14,894][20381] Updated weights for policy 1, policy_version 430223 (0.0008) [2023-12-27 08:37:15,321][20347] Updated weights for policy 0, policy_version 430566 (0.0009) [2023-12-27 08:37:15,385][20347] Updated weights for policy 0, policy_version 430576 (0.0007) [2023-12-27 08:37:15,446][20347] Updated weights for policy 0, policy_version 430586 (0.0007) [2023-12-27 08:37:15,609][20381] Updated weights for policy 1, policy_version 430233 (0.0009) [2023-12-27 08:37:15,664][20381] Updated weights for policy 1, policy_version 430243 (0.0006) [2023-12-27 08:37:15,726][20381] Updated weights for policy 1, policy_version 430253 (0.0006) [2023-12-27 08:37:16,022][20347] Updated weights for policy 0, policy_version 430596 (0.0006) [2023-12-27 08:37:16,094][20347] Updated weights for policy 0, policy_version 430606 (0.0007) [2023-12-27 08:37:16,154][20347] Updated weights for policy 0, policy_version 430616 (0.0005) [2023-12-27 08:37:16,324][20381] Updated weights for policy 1, policy_version 430263 (0.0008) [2023-12-27 08:37:16,386][20381] Updated weights for policy 1, policy_version 430273 (0.0008) [2023-12-27 08:37:16,440][20381] Updated weights for policy 1, policy_version 430283 (0.0008) [2023-12-27 08:37:16,688][20347] Updated weights for policy 0, policy_version 430626 (0.0008) [2023-12-27 08:37:16,755][20347] Updated weights for policy 0, policy_version 430636 (0.0008) [2023-12-27 08:37:16,813][20347] Updated weights for policy 0, policy_version 430646 (0.0008) [2023-12-27 08:37:16,873][20347] Updated weights for policy 0, policy_version 430656 (0.0005) [2023-12-27 08:37:17,025][20381] Updated weights for policy 1, policy_version 430293 (0.0006) [2023-12-27 08:37:17,084][20381] Updated weights for policy 1, policy_version 430303 (0.0005) [2023-12-27 08:37:17,140][20381] Updated weights for policy 1, policy_version 430313 (0.0006) [2023-12-27 08:37:17,362][19170] Fps is (10 sec: 18022.5, 60 sec: 17476.3, 300 sec: 19494.2). Total num frames: 220446720. Throughput: 0: 8342.9, 1: 8397.5. Samples: 220456896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:17,363][19170] Avg episode reward: [(0, '923.180'), (1, '990.370')] [2023-12-27 08:37:17,486][20347] Updated weights for policy 0, policy_version 430666 (0.0008) [2023-12-27 08:37:17,545][20347] Updated weights for policy 0, policy_version 430676 (0.0006) [2023-12-27 08:37:17,613][20347] Updated weights for policy 0, policy_version 430686 (0.0006) [2023-12-27 08:37:17,729][20381] Updated weights for policy 1, policy_version 430323 (0.0006) [2023-12-27 08:37:17,801][20381] Updated weights for policy 1, policy_version 430333 (0.0009) [2023-12-27 08:37:17,871][20381] Updated weights for policy 1, policy_version 430343 (0.0008) [2023-12-27 08:37:18,200][20347] Updated weights for policy 0, policy_version 430696 (0.0007) [2023-12-27 08:37:18,253][20347] Updated weights for policy 0, policy_version 430706 (0.0009) [2023-12-27 08:37:18,302][20347] Updated weights for policy 0, policy_version 430716 (0.0009) [2023-12-27 08:37:18,592][20381] Updated weights for policy 1, policy_version 430353 (0.0008) [2023-12-27 08:37:18,648][20381] Updated weights for policy 1, policy_version 430363 (0.0006) [2023-12-27 08:37:18,696][20381] Updated weights for policy 1, policy_version 430373 (0.0007) [2023-12-27 08:37:18,747][20381] Updated weights for policy 1, policy_version 430383 (0.0008) [2023-12-27 08:37:19,006][20347] Updated weights for policy 0, policy_version 430726 (0.0007) [2023-12-27 08:37:19,049][20291] KL-divergence is very high: 103.6660 [2023-12-27 08:37:19,055][20291] KL-divergence is very high: 101.7393 [2023-12-27 08:37:19,066][20347] Updated weights for policy 0, policy_version 430736 (0.0007) [2023-12-27 08:37:19,127][20347] Updated weights for policy 0, policy_version 430746 (0.0006) [2023-12-27 08:37:19,151][20291] KL-divergence is very high: 110.9739 [2023-12-27 08:37:19,466][20381] Updated weights for policy 1, policy_version 430393 (0.0008) [2023-12-27 08:37:19,527][20381] Updated weights for policy 1, policy_version 430403 (0.0009) [2023-12-27 08:37:19,589][20381] Updated weights for policy 1, policy_version 430413 (0.0010) [2023-12-27 08:37:19,801][20347] Updated weights for policy 0, policy_version 430756 (0.0007) [2023-12-27 08:37:19,859][20347] Updated weights for policy 0, policy_version 430766 (0.0008) [2023-12-27 08:37:19,917][20347] Updated weights for policy 0, policy_version 430776 (0.0008) [2023-12-27 08:37:20,286][20381] Updated weights for policy 1, policy_version 430423 (0.0010) [2023-12-27 08:37:20,348][20381] Updated weights for policy 1, policy_version 430433 (0.0006) [2023-12-27 08:37:20,413][20381] Updated weights for policy 1, policy_version 430443 (0.0011) [2023-12-27 08:37:20,645][20347] Updated weights for policy 0, policy_version 430786 (0.0008) [2023-12-27 08:37:20,710][20347] Updated weights for policy 0, policy_version 430796 (0.0008) [2023-12-27 08:37:20,769][20347] Updated weights for policy 0, policy_version 430806 (0.0005) [2023-12-27 08:37:20,834][20347] Updated weights for policy 0, policy_version 430816 (0.0006) [2023-12-27 08:37:21,055][20381] Updated weights for policy 1, policy_version 430453 (0.0009) [2023-12-27 08:37:21,115][20381] Updated weights for policy 1, policy_version 430463 (0.0008) [2023-12-27 08:37:21,168][20381] Updated weights for policy 1, policy_version 430473 (0.0007) [2023-12-27 08:37:21,524][20347] Updated weights for policy 0, policy_version 430826 (0.0008) [2023-12-27 08:37:21,570][20347] Updated weights for policy 0, policy_version 430836 (0.0008) [2023-12-27 08:37:21,627][20347] Updated weights for policy 0, policy_version 430846 (0.0008) [2023-12-27 08:37:21,899][20381] Updated weights for policy 1, policy_version 430483 (0.0008) [2023-12-27 08:37:21,956][20381] Updated weights for policy 1, policy_version 430493 (0.0009) [2023-12-27 08:37:22,015][20381] Updated weights for policy 1, policy_version 430503 (0.0010) [2023-12-27 08:37:22,328][20347] Updated weights for policy 0, policy_version 430856 (0.0007) [2023-12-27 08:37:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 17476.3, 300 sec: 19466.4). Total num frames: 220545024. Throughput: 0: 8497.8, 1: 8527.9. Samples: 220517964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:22,363][19170] Avg episode reward: [(0, '922.670'), (1, '1000.000')] [2023-12-27 08:37:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000430512_110231552.pth... [2023-12-27 08:37:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429488_109969408.pth [2023-12-27 08:37:22,392][20347] Updated weights for policy 0, policy_version 430866 (0.0008) [2023-12-27 08:37:22,458][20347] Updated weights for policy 0, policy_version 430876 (0.0008) [2023-12-27 08:37:22,482][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000430880_110321664.pth... [2023-12-27 08:37:22,486][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000429824_110051328.pth [2023-12-27 08:37:22,715][20381] Updated weights for policy 1, policy_version 430513 (0.0010) [2023-12-27 08:37:22,778][20381] Updated weights for policy 1, policy_version 430523 (0.0007) [2023-12-27 08:37:22,841][20381] Updated weights for policy 1, policy_version 430533 (0.0010) [2023-12-27 08:37:22,897][20381] Updated weights for policy 1, policy_version 430543 (0.0007) [2023-12-27 08:37:23,173][20347] Updated weights for policy 0, policy_version 430886 (0.0009) [2023-12-27 08:37:23,222][20347] Updated weights for policy 0, policy_version 430896 (0.0008) [2023-12-27 08:37:23,288][20347] Updated weights for policy 0, policy_version 430906 (0.0008) [2023-12-27 08:37:23,433][20381] Updated weights for policy 1, policy_version 430553 (0.0006) [2023-12-27 08:37:23,484][20381] Updated weights for policy 1, policy_version 430563 (0.0005) [2023-12-27 08:37:23,537][20381] Updated weights for policy 1, policy_version 430573 (0.0006) [2023-12-27 08:37:24,037][20347] Updated weights for policy 0, policy_version 430916 (0.0006) [2023-12-27 08:37:24,045][20381] Updated weights for policy 1, policy_version 430583 (0.0009) [2023-12-27 08:37:24,084][20347] Updated weights for policy 0, policy_version 430926 (0.0005) [2023-12-27 08:37:24,093][20381] Updated weights for policy 1, policy_version 430593 (0.0010) [2023-12-27 08:37:24,137][20347] Updated weights for policy 0, policy_version 430936 (0.0005) [2023-12-27 08:37:24,152][20381] Updated weights for policy 1, policy_version 430603 (0.0011) [2023-12-27 08:37:24,779][20347] Updated weights for policy 0, policy_version 430946 (0.0006) [2023-12-27 08:37:24,815][20381] Updated weights for policy 1, policy_version 430613 (0.0011) [2023-12-27 08:37:24,833][20347] Updated weights for policy 0, policy_version 430956 (0.0005) [2023-12-27 08:37:24,864][20381] Updated weights for policy 1, policy_version 430623 (0.0010) [2023-12-27 08:37:24,885][20347] Updated weights for policy 0, policy_version 430966 (0.0008) [2023-12-27 08:37:24,915][20381] Updated weights for policy 1, policy_version 430633 (0.0010) [2023-12-27 08:37:24,937][20347] Updated weights for policy 0, policy_version 430976 (0.0009) [2023-12-27 08:37:25,592][20381] Updated weights for policy 1, policy_version 430643 (0.0010) [2023-12-27 08:37:25,612][20347] Updated weights for policy 0, policy_version 430986 (0.0006) [2023-12-27 08:37:25,640][20381] Updated weights for policy 1, policy_version 430653 (0.0006) [2023-12-27 08:37:25,664][20347] Updated weights for policy 0, policy_version 430996 (0.0005) [2023-12-27 08:37:25,685][20381] Updated weights for policy 1, policy_version 430663 (0.0005) [2023-12-27 08:37:25,718][20347] Updated weights for policy 0, policy_version 431006 (0.0005) [2023-12-27 08:37:26,383][20381] Updated weights for policy 1, policy_version 430673 (0.0007) [2023-12-27 08:37:26,419][20347] Updated weights for policy 0, policy_version 431016 (0.0007) [2023-12-27 08:37:26,445][20381] Updated weights for policy 1, policy_version 430683 (0.0010) [2023-12-27 08:37:26,475][20347] Updated weights for policy 0, policy_version 431026 (0.0008) [2023-12-27 08:37:26,494][20381] Updated weights for policy 1, policy_version 430693 (0.0008) [2023-12-27 08:37:26,529][20347] Updated weights for policy 0, policy_version 431036 (0.0005) [2023-12-27 08:37:26,549][20381] Updated weights for policy 1, policy_version 430703 (0.0006) [2023-12-27 08:37:27,213][20347] Updated weights for policy 0, policy_version 431046 (0.0006) [2023-12-27 08:37:27,256][20381] Updated weights for policy 1, policy_version 430713 (0.0005) [2023-12-27 08:37:27,270][20347] Updated weights for policy 0, policy_version 431056 (0.0006) [2023-12-27 08:37:27,299][20291] KL-divergence is very high: 302.2535 [2023-12-27 08:37:27,315][20381] Updated weights for policy 1, policy_version 430723 (0.0005) [2023-12-27 08:37:27,331][20347] Updated weights for policy 0, policy_version 431066 (0.0008) [2023-12-27 08:37:27,344][20291] KL-divergence is very high: 109.3490 [2023-12-27 08:37:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 17339.8, 300 sec: 19438.6). Total num frames: 220643328. Throughput: 0: 8737.1, 1: 8853.3. Samples: 220641876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:27,363][19170] Avg episode reward: [(0, '913.650'), (1, '1000.000')] [2023-12-27 08:37:27,368][20381] Updated weights for policy 1, policy_version 430733 (0.0010) [2023-12-27 08:37:27,976][20291] KL-divergence is very high: 201.7439 [2023-12-27 08:37:27,985][20347] Updated weights for policy 0, policy_version 431076 (0.0007) [2023-12-27 08:37:28,013][20381] Updated weights for policy 1, policy_version 430743 (0.0009) [2023-12-27 08:37:28,016][20291] KL-divergence is very high: 338.8498 [2023-12-27 08:37:28,034][20347] Updated weights for policy 0, policy_version 431086 (0.0006) [2023-12-27 08:37:28,055][20291] KL-divergence is very high: 128.7209 [2023-12-27 08:37:28,062][20381] Updated weights for policy 1, policy_version 430753 (0.0010) [2023-12-27 08:37:28,086][20347] Updated weights for policy 0, policy_version 431096 (0.0005) [2023-12-27 08:37:28,098][20291] KL-divergence is very high: 103.1964 [2023-12-27 08:37:28,108][20381] Updated weights for policy 1, policy_version 430763 (0.0010) [2023-12-27 08:37:28,791][20347] Updated weights for policy 0, policy_version 431106 (0.0007) [2023-12-27 08:37:28,811][20381] Updated weights for policy 1, policy_version 430773 (0.0010) [2023-12-27 08:37:28,836][20291] KL-divergence is very high: 160.8399 [2023-12-27 08:37:28,847][20347] Updated weights for policy 0, policy_version 431116 (0.0006) [2023-12-27 08:37:28,863][20381] Updated weights for policy 1, policy_version 430783 (0.0011) [2023-12-27 08:37:28,899][20347] Updated weights for policy 0, policy_version 431126 (0.0006) [2023-12-27 08:37:28,916][20381] Updated weights for policy 1, policy_version 430793 (0.0011) [2023-12-27 08:37:28,944][20347] Updated weights for policy 0, policy_version 431136 (0.0006) [2023-12-27 08:37:29,591][20347] Updated weights for policy 0, policy_version 431146 (0.0008) [2023-12-27 08:37:29,599][20291] KL-divergence is very high: 218.1263 [2023-12-27 08:37:29,620][20381] Updated weights for policy 1, policy_version 430803 (0.0009) [2023-12-27 08:37:29,645][20291] KL-divergence is very high: 169.2094 [2023-12-27 08:37:29,650][20347] Updated weights for policy 0, policy_version 431156 (0.0006) [2023-12-27 08:37:29,678][20381] Updated weights for policy 1, policy_version 430813 (0.0007) [2023-12-27 08:37:29,693][20291] KL-divergence is very high: 319.8217 [2023-12-27 08:37:29,700][20291] KL-divergence is very high: 105.6373 [2023-12-27 08:37:29,710][20347] Updated weights for policy 0, policy_version 431166 (0.0008) [2023-12-27 08:37:29,731][20381] Updated weights for policy 1, policy_version 430823 (0.0005) [2023-12-27 08:37:30,361][20291] KL-divergence is very high: 208.2617 [2023-12-27 08:37:30,368][20381] Updated weights for policy 1, policy_version 430833 (0.0007) [2023-12-27 08:37:30,390][20347] Updated weights for policy 0, policy_version 431176 (0.0008) [2023-12-27 08:37:30,405][20291] KL-divergence is very high: 136.2775 [2023-12-27 08:37:30,417][20381] Updated weights for policy 1, policy_version 430843 (0.0005) [2023-12-27 08:37:30,450][20347] Updated weights for policy 0, policy_version 431186 (0.0008) [2023-12-27 08:37:30,471][20381] Updated weights for policy 1, policy_version 430853 (0.0005) [2023-12-27 08:37:30,505][20347] Updated weights for policy 0, policy_version 431196 (0.0007) [2023-12-27 08:37:30,525][20381] Updated weights for policy 1, policy_version 430863 (0.0005) [2023-12-27 08:37:31,102][20381] Updated weights for policy 1, policy_version 430873 (0.0007) [2023-12-27 08:37:31,169][20381] Updated weights for policy 1, policy_version 430883 (0.0009) [2023-12-27 08:37:31,197][20347] Updated weights for policy 0, policy_version 431206 (0.0006) [2023-12-27 08:37:31,225][20381] Updated weights for policy 1, policy_version 430893 (0.0007) [2023-12-27 08:37:31,250][20347] Updated weights for policy 0, policy_version 431216 (0.0007) [2023-12-27 08:37:31,308][20347] Updated weights for policy 0, policy_version 431226 (0.0007) [2023-12-27 08:37:31,882][20381] Updated weights for policy 1, policy_version 430903 (0.0010) [2023-12-27 08:37:31,934][20381] Updated weights for policy 1, policy_version 430913 (0.0007) [2023-12-27 08:37:31,989][20381] Updated weights for policy 1, policy_version 430923 (0.0006) [2023-12-27 08:37:32,010][20347] Updated weights for policy 0, policy_version 431236 (0.0007) [2023-12-27 08:37:32,069][20347] Updated weights for policy 0, policy_version 431246 (0.0007) [2023-12-27 08:37:32,130][20347] Updated weights for policy 0, policy_version 431256 (0.0007) [2023-12-27 08:37:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 17749.3, 300 sec: 19466.4). Total num frames: 220758016. Throughput: 0: 9033.7, 1: 9177.9. Samples: 220765768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:32,363][19170] Avg episode reward: [(0, '890.010'), (1, '1000.000')] [2023-12-27 08:37:32,672][20381] Updated weights for policy 1, policy_version 430933 (0.0007) [2023-12-27 08:37:32,731][20381] Updated weights for policy 1, policy_version 430943 (0.0008) [2023-12-27 08:37:32,792][20381] Updated weights for policy 1, policy_version 430953 (0.0007) [2023-12-27 08:37:32,848][20347] Updated weights for policy 0, policy_version 431266 (0.0007) [2023-12-27 08:37:32,898][20347] Updated weights for policy 0, policy_version 431276 (0.0009) [2023-12-27 08:37:32,946][20347] Updated weights for policy 0, policy_version 431286 (0.0010) [2023-12-27 08:37:32,993][20347] Updated weights for policy 0, policy_version 431296 (0.0010) [2023-12-27 08:37:33,480][20381] Updated weights for policy 1, policy_version 430963 (0.0008) [2023-12-27 08:37:33,523][20381] Updated weights for policy 1, policy_version 430973 (0.0007) [2023-12-27 08:37:33,580][20381] Updated weights for policy 1, policy_version 430983 (0.0007) [2023-12-27 08:37:33,703][20347] Updated weights for policy 0, policy_version 431306 (0.0010) [2023-12-27 08:37:33,734][20291] KL-divergence is very high: 126.7760 [2023-12-27 08:37:33,747][20347] Updated weights for policy 0, policy_version 431316 (0.0010) [2023-12-27 08:37:33,795][20347] Updated weights for policy 0, policy_version 431326 (0.0010) [2023-12-27 08:37:34,270][20381] Updated weights for policy 1, policy_version 430993 (0.0007) [2023-12-27 08:37:34,325][20381] Updated weights for policy 1, policy_version 431003 (0.0007) [2023-12-27 08:37:34,381][20381] Updated weights for policy 1, policy_version 431013 (0.0008) [2023-12-27 08:37:34,442][20381] Updated weights for policy 1, policy_version 431023 (0.0008) [2023-12-27 08:37:34,475][20347] Updated weights for policy 0, policy_version 431336 (0.0007) [2023-12-27 08:37:34,530][20347] Updated weights for policy 0, policy_version 431346 (0.0010) [2023-12-27 08:37:34,594][20347] Updated weights for policy 0, policy_version 431356 (0.0010) [2023-12-27 08:37:35,137][20381] Updated weights for policy 1, policy_version 431033 (0.0007) [2023-12-27 08:37:35,184][20381] Updated weights for policy 1, policy_version 431043 (0.0007) [2023-12-27 08:37:35,223][20347] Updated weights for policy 0, policy_version 431366 (0.0005) [2023-12-27 08:37:35,242][20381] Updated weights for policy 1, policy_version 431053 (0.0007) [2023-12-27 08:37:35,283][20347] Updated weights for policy 0, policy_version 431376 (0.0006) [2023-12-27 08:37:35,332][20347] Updated weights for policy 0, policy_version 431386 (0.0008) [2023-12-27 08:37:35,909][20381] Updated weights for policy 1, policy_version 431063 (0.0008) [2023-12-27 08:37:35,956][20381] Updated weights for policy 1, policy_version 431073 (0.0007) [2023-12-27 08:37:35,980][20347] Updated weights for policy 0, policy_version 431396 (0.0007) [2023-12-27 08:37:36,005][20381] Updated weights for policy 1, policy_version 431083 (0.0005) [2023-12-27 08:37:36,029][20347] Updated weights for policy 0, policy_version 431406 (0.0010) [2023-12-27 08:37:36,080][20347] Updated weights for policy 0, policy_version 431416 (0.0010) [2023-12-27 08:37:36,742][20381] Updated weights for policy 1, policy_version 431093 (0.0007) [2023-12-27 08:37:36,784][20347] Updated weights for policy 0, policy_version 431426 (0.0010) [2023-12-27 08:37:36,793][20381] Updated weights for policy 1, policy_version 431103 (0.0007) [2023-12-27 08:37:36,839][20347] Updated weights for policy 0, policy_version 431436 (0.0010) [2023-12-27 08:37:36,851][20381] Updated weights for policy 1, policy_version 431113 (0.0005) [2023-12-27 08:37:36,903][20347] Updated weights for policy 0, policy_version 431446 (0.0010) [2023-12-27 08:37:36,966][20347] Updated weights for policy 0, policy_version 431456 (0.0010) [2023-12-27 08:37:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 18022.4, 300 sec: 19466.4). Total num frames: 220856320. Throughput: 0: 9182.1, 1: 9303.3. Samples: 220826408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:37,363][19170] Avg episode reward: [(0, '886.040'), (1, '1000.000')] [2023-12-27 08:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000431120_110387200.pth... [2023-12-27 08:37:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000431456_110469120.pth... [2023-12-27 08:37:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000429968_110092288.pth [2023-12-27 08:37:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000430304_110174208.pth [2023-12-27 08:37:37,515][20381] Updated weights for policy 1, policy_version 431123 (0.0007) [2023-12-27 08:37:37,575][20381] Updated weights for policy 1, policy_version 431133 (0.0007) [2023-12-27 08:37:37,584][20347] Updated weights for policy 0, policy_version 431466 (0.0010) [2023-12-27 08:37:37,625][20381] Updated weights for policy 1, policy_version 431143 (0.0008) [2023-12-27 08:37:37,633][20347] Updated weights for policy 0, policy_version 431476 (0.0010) [2023-12-27 08:37:37,691][20347] Updated weights for policy 0, policy_version 431486 (0.0005) [2023-12-27 08:37:38,299][20381] Updated weights for policy 1, policy_version 431153 (0.0007) [2023-12-27 08:37:38,304][20347] Updated weights for policy 0, policy_version 431496 (0.0009) [2023-12-27 08:37:38,359][20381] Updated weights for policy 1, policy_version 431163 (0.0006) [2023-12-27 08:37:38,365][20347] Updated weights for policy 0, policy_version 431506 (0.0010) [2023-12-27 08:37:38,411][20381] Updated weights for policy 1, policy_version 431173 (0.0005) [2023-12-27 08:37:38,425][20347] Updated weights for policy 0, policy_version 431516 (0.0010) [2023-12-27 08:37:38,468][20381] Updated weights for policy 1, policy_version 431183 (0.0006) [2023-12-27 08:37:39,113][20347] Updated weights for policy 0, policy_version 431526 (0.0010) [2023-12-27 08:37:39,158][20347] Updated weights for policy 0, policy_version 431536 (0.0010) [2023-12-27 08:37:39,189][20381] Updated weights for policy 1, policy_version 431193 (0.0007) [2023-12-27 08:37:39,210][20347] Updated weights for policy 0, policy_version 431546 (0.0010) [2023-12-27 08:37:39,254][20381] Updated weights for policy 1, policy_version 431203 (0.0007) [2023-12-27 08:37:39,303][20381] Updated weights for policy 1, policy_version 431213 (0.0008) [2023-12-27 08:37:39,921][20347] Updated weights for policy 0, policy_version 431556 (0.0011) [2023-12-27 08:37:39,985][20347] Updated weights for policy 0, policy_version 431566 (0.0010) [2023-12-27 08:37:40,007][20381] Updated weights for policy 1, policy_version 431223 (0.0008) [2023-12-27 08:37:40,051][20347] Updated weights for policy 0, policy_version 431576 (0.0011) [2023-12-27 08:37:40,065][20381] Updated weights for policy 1, policy_version 431233 (0.0006) [2023-12-27 08:37:40,119][20381] Updated weights for policy 1, policy_version 431243 (0.0006) [2023-12-27 08:37:40,735][20347] Updated weights for policy 0, policy_version 431586 (0.0011) [2023-12-27 08:37:40,783][20347] Updated weights for policy 0, policy_version 431596 (0.0010) [2023-12-27 08:37:40,832][20347] Updated weights for policy 0, policy_version 431606 (0.0010) [2023-12-27 08:37:40,839][20381] Updated weights for policy 1, policy_version 431253 (0.0007) [2023-12-27 08:37:40,886][20347] Updated weights for policy 0, policy_version 431616 (0.0010) [2023-12-27 08:37:40,891][20381] Updated weights for policy 1, policy_version 431263 (0.0007) [2023-12-27 08:37:40,945][20381] Updated weights for policy 1, policy_version 431273 (0.0008) [2023-12-27 08:37:41,654][20347] Updated weights for policy 0, policy_version 431626 (0.0008) [2023-12-27 08:37:41,657][20381] Updated weights for policy 1, policy_version 431283 (0.0007) [2023-12-27 08:37:41,709][20347] Updated weights for policy 0, policy_version 431636 (0.0007) [2023-12-27 08:37:41,716][20381] Updated weights for policy 1, policy_version 431293 (0.0006) [2023-12-27 08:37:41,774][20347] Updated weights for policy 0, policy_version 431646 (0.0008) [2023-12-27 08:37:41,776][20381] Updated weights for policy 1, policy_version 431303 (0.0007) [2023-12-27 08:37:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 18432.0, 300 sec: 19410.9). Total num frames: 220954624. Throughput: 0: 9428.4, 1: 9512.9. Samples: 220948188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:42,363][19170] Avg episode reward: [(0, '924.970'), (1, '1000.000')] [2023-12-27 08:37:42,430][20347] Updated weights for policy 0, policy_version 431656 (0.0009) [2023-12-27 08:37:42,480][20381] Updated weights for policy 1, policy_version 431313 (0.0007) [2023-12-27 08:37:42,495][20347] Updated weights for policy 0, policy_version 431666 (0.0007) [2023-12-27 08:37:42,542][20381] Updated weights for policy 1, policy_version 431323 (0.0005) [2023-12-27 08:37:42,562][20347] Updated weights for policy 0, policy_version 431676 (0.0007) [2023-12-27 08:37:42,600][20381] Updated weights for policy 1, policy_version 431333 (0.0006) [2023-12-27 08:37:42,662][20381] Updated weights for policy 1, policy_version 431343 (0.0006) [2023-12-27 08:37:43,230][20381] Updated weights for policy 1, policy_version 431353 (0.0005) [2023-12-27 08:37:43,282][20381] Updated weights for policy 1, policy_version 431363 (0.0008) [2023-12-27 08:37:43,291][20347] Updated weights for policy 0, policy_version 431686 (0.0007) [2023-12-27 08:37:43,342][20381] Updated weights for policy 1, policy_version 431373 (0.0011) [2023-12-27 08:37:43,343][20347] Updated weights for policy 0, policy_version 431696 (0.0005) [2023-12-27 08:37:43,394][20347] Updated weights for policy 0, policy_version 431706 (0.0006) [2023-12-27 08:37:44,028][20381] Updated weights for policy 1, policy_version 431383 (0.0007) [2023-12-27 08:37:44,086][20381] Updated weights for policy 1, policy_version 431393 (0.0006) [2023-12-27 08:37:44,088][20347] Updated weights for policy 0, policy_version 431716 (0.0008) [2023-12-27 08:37:44,142][20347] Updated weights for policy 0, policy_version 431726 (0.0007) [2023-12-27 08:37:44,142][20381] Updated weights for policy 1, policy_version 431403 (0.0006) [2023-12-27 08:37:44,193][20347] Updated weights for policy 0, policy_version 431736 (0.0007) [2023-12-27 08:37:44,791][20381] Updated weights for policy 1, policy_version 431413 (0.0007) [2023-12-27 08:37:44,840][20381] Updated weights for policy 1, policy_version 431423 (0.0006) [2023-12-27 08:37:44,886][20381] Updated weights for policy 1, policy_version 431433 (0.0007) [2023-12-27 08:37:44,904][20347] Updated weights for policy 0, policy_version 431746 (0.0008) [2023-12-27 08:37:44,967][20347] Updated weights for policy 0, policy_version 431756 (0.0007) [2023-12-27 08:37:45,026][20347] Updated weights for policy 0, policy_version 431766 (0.0008) [2023-12-27 08:37:45,089][20347] Updated weights for policy 0, policy_version 431776 (0.0008) [2023-12-27 08:37:45,601][20381] Updated weights for policy 1, policy_version 431443 (0.0007) [2023-12-27 08:37:45,667][20381] Updated weights for policy 1, policy_version 431453 (0.0008) [2023-12-27 08:37:45,716][20381] Updated weights for policy 1, policy_version 431463 (0.0006) [2023-12-27 08:37:45,804][20347] Updated weights for policy 0, policy_version 431786 (0.0008) [2023-12-27 08:37:45,816][20291] KL-divergence is very high: 123.7806 [2023-12-27 08:37:45,867][20347] Updated weights for policy 0, policy_version 431796 (0.0008) [2023-12-27 08:37:45,927][20347] Updated weights for policy 0, policy_version 431806 (0.0007) [2023-12-27 08:37:46,374][20381] Updated weights for policy 1, policy_version 431473 (0.0007) [2023-12-27 08:37:46,419][20381] Updated weights for policy 1, policy_version 431483 (0.0007) [2023-12-27 08:37:46,478][20381] Updated weights for policy 1, policy_version 431493 (0.0007) [2023-12-27 08:37:46,527][20381] Updated weights for policy 1, policy_version 431503 (0.0007) [2023-12-27 08:37:46,552][20347] Updated weights for policy 0, policy_version 431816 (0.0007) [2023-12-27 08:37:46,609][20347] Updated weights for policy 0, policy_version 431826 (0.0007) [2023-12-27 08:37:46,667][20347] Updated weights for policy 0, policy_version 431836 (0.0008) [2023-12-27 08:37:47,197][20381] Updated weights for policy 1, policy_version 431513 (0.0008) [2023-12-27 08:37:47,240][20381] Updated weights for policy 1, policy_version 431523 (0.0007) [2023-12-27 08:37:47,297][20381] Updated weights for policy 1, policy_version 431533 (0.0008) [2023-12-27 08:37:47,351][20347] Updated weights for policy 0, policy_version 431846 (0.0008) [2023-12-27 08:37:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 18841.6, 300 sec: 19438.8). Total num frames: 221061120. Throughput: 0: 9623.4, 1: 9770.1. Samples: 221070348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:47,363][19170] Avg episode reward: [(0, '953.200'), (1, '1000.000')] [2023-12-27 08:37:47,398][20347] Updated weights for policy 0, policy_version 431856 (0.0009) [2023-12-27 08:37:47,457][20347] Updated weights for policy 0, policy_version 431866 (0.0005) [2023-12-27 08:37:48,033][20381] Updated weights for policy 1, policy_version 431543 (0.0006) [2023-12-27 08:37:48,090][20381] Updated weights for policy 1, policy_version 431553 (0.0006) [2023-12-27 08:37:48,116][20347] Updated weights for policy 0, policy_version 431876 (0.0006) [2023-12-27 08:37:48,152][20381] Updated weights for policy 1, policy_version 431563 (0.0006) [2023-12-27 08:37:48,178][20347] Updated weights for policy 0, policy_version 431886 (0.0007) [2023-12-27 08:37:48,227][20347] Updated weights for policy 0, policy_version 431896 (0.0009) [2023-12-27 08:37:48,789][20381] Updated weights for policy 1, policy_version 431573 (0.0007) [2023-12-27 08:37:48,846][20381] Updated weights for policy 1, policy_version 431583 (0.0008) [2023-12-27 08:37:48,896][20381] Updated weights for policy 1, policy_version 431593 (0.0007) [2023-12-27 08:37:48,952][20347] Updated weights for policy 0, policy_version 431906 (0.0007) [2023-12-27 08:37:49,009][20347] Updated weights for policy 0, policy_version 431916 (0.0007) [2023-12-27 08:37:49,066][20347] Updated weights for policy 0, policy_version 431926 (0.0005) [2023-12-27 08:37:49,116][20347] Updated weights for policy 0, policy_version 431936 (0.0006) [2023-12-27 08:37:49,636][20381] Updated weights for policy 1, policy_version 431603 (0.0008) [2023-12-27 08:37:49,684][20381] Updated weights for policy 1, policy_version 431613 (0.0007) [2023-12-27 08:37:49,736][20347] Updated weights for policy 0, policy_version 431946 (0.0008) [2023-12-27 08:37:49,736][20381] Updated weights for policy 1, policy_version 431623 (0.0008) [2023-12-27 08:37:49,802][20347] Updated weights for policy 0, policy_version 431956 (0.0007) [2023-12-27 08:37:49,866][20347] Updated weights for policy 0, policy_version 431966 (0.0008) [2023-12-27 08:37:50,459][20381] Updated weights for policy 1, policy_version 431633 (0.0006) [2023-12-27 08:37:50,519][20381] Updated weights for policy 1, policy_version 431643 (0.0007) [2023-12-27 08:37:50,546][20347] Updated weights for policy 0, policy_version 431976 (0.0006) [2023-12-27 08:37:50,579][20381] Updated weights for policy 1, policy_version 431653 (0.0007) [2023-12-27 08:37:50,606][20347] Updated weights for policy 0, policy_version 431986 (0.0008) [2023-12-27 08:37:50,638][20381] Updated weights for policy 1, policy_version 431663 (0.0006) [2023-12-27 08:37:50,662][20347] Updated weights for policy 0, policy_version 431996 (0.0008) [2023-12-27 08:37:51,292][20347] Updated weights for policy 0, policy_version 432006 (0.0008) [2023-12-27 08:37:51,305][20381] Updated weights for policy 1, policy_version 431673 (0.0008) [2023-12-27 08:37:51,344][20347] Updated weights for policy 0, policy_version 432016 (0.0009) [2023-12-27 08:37:51,360][20381] Updated weights for policy 1, policy_version 431683 (0.0009) [2023-12-27 08:37:51,404][20347] Updated weights for policy 0, policy_version 432026 (0.0009) [2023-12-27 08:37:51,418][20381] Updated weights for policy 1, policy_version 431693 (0.0009) [2023-12-27 08:37:52,111][20381] Updated weights for policy 1, policy_version 431703 (0.0008) [2023-12-27 08:37:52,160][20347] Updated weights for policy 0, policy_version 432036 (0.0008) [2023-12-27 08:37:52,169][20381] Updated weights for policy 1, policy_version 431713 (0.0006) [2023-12-27 08:37:52,216][20347] Updated weights for policy 0, policy_version 432046 (0.0005) [2023-12-27 08:37:52,231][20381] Updated weights for policy 1, policy_version 431723 (0.0007) [2023-12-27 08:37:52,282][20347] Updated weights for policy 0, policy_version 432056 (0.0007) [2023-12-27 08:37:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 19251.3, 300 sec: 19466.4). Total num frames: 221167616. Throughput: 0: 9774.1, 1: 9872.2. Samples: 221131244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:52,363][19170] Avg episode reward: [(0, '933.270'), (1, '1000.000')] [2023-12-27 08:37:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000431728_110542848.pth... [2023-12-27 08:37:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000432064_110624768.pth... [2023-12-27 08:37:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000430880_110321664.pth [2023-12-27 08:37:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000430512_110231552.pth [2023-12-27 08:37:52,877][20381] Updated weights for policy 1, policy_version 431733 (0.0008) [2023-12-27 08:37:52,931][20347] Updated weights for policy 0, policy_version 432066 (0.0006) [2023-12-27 08:37:52,933][20381] Updated weights for policy 1, policy_version 431743 (0.0007) [2023-12-27 08:37:52,976][20347] Updated weights for policy 0, policy_version 432076 (0.0006) [2023-12-27 08:37:52,981][20291] KL-divergence is very high: 124.9487 [2023-12-27 08:37:52,986][20381] Updated weights for policy 1, policy_version 431753 (0.0005) [2023-12-27 08:37:53,024][20347] Updated weights for policy 0, policy_version 432086 (0.0005) [2023-12-27 08:37:53,072][20347] Updated weights for policy 0, policy_version 432096 (0.0007) [2023-12-27 08:37:53,579][20381] Updated weights for policy 1, policy_version 431763 (0.0006) [2023-12-27 08:37:53,627][20381] Updated weights for policy 1, policy_version 431773 (0.0008) [2023-12-27 08:37:53,677][20381] Updated weights for policy 1, policy_version 431783 (0.0010) [2023-12-27 08:37:53,793][20347] Updated weights for policy 0, policy_version 432106 (0.0007) [2023-12-27 08:37:53,850][20347] Updated weights for policy 0, policy_version 432116 (0.0010) [2023-12-27 08:37:53,911][20347] Updated weights for policy 0, policy_version 432126 (0.0010) [2023-12-27 08:37:54,327][20381] Updated weights for policy 1, policy_version 431793 (0.0010) [2023-12-27 08:37:54,386][20381] Updated weights for policy 1, policy_version 431803 (0.0010) [2023-12-27 08:37:54,450][20381] Updated weights for policy 1, policy_version 431813 (0.0010) [2023-12-27 08:37:54,486][20347] Updated weights for policy 0, policy_version 432136 (0.0006) [2023-12-27 08:37:54,504][20381] Updated weights for policy 1, policy_version 431823 (0.0010) [2023-12-27 08:37:54,539][20347] Updated weights for policy 0, policy_version 432146 (0.0005) [2023-12-27 08:37:54,592][20347] Updated weights for policy 0, policy_version 432156 (0.0005) [2023-12-27 08:37:55,136][20347] Updated weights for policy 0, policy_version 432166 (0.0008) [2023-12-27 08:37:55,175][20381] Updated weights for policy 1, policy_version 431833 (0.0006) [2023-12-27 08:37:55,185][20347] Updated weights for policy 0, policy_version 432176 (0.0010) [2023-12-27 08:37:55,232][20381] Updated weights for policy 1, policy_version 431843 (0.0007) [2023-12-27 08:37:55,240][20347] Updated weights for policy 0, policy_version 432186 (0.0008) [2023-12-27 08:37:55,286][20381] Updated weights for policy 1, policy_version 431853 (0.0006) [2023-12-27 08:37:55,838][20381] Updated weights for policy 1, policy_version 431863 (0.0008) [2023-12-27 08:37:55,893][20381] Updated weights for policy 1, policy_version 431873 (0.0010) [2023-12-27 08:37:55,894][20347] Updated weights for policy 0, policy_version 432196 (0.0005) [2023-12-27 08:37:55,946][20381] Updated weights for policy 1, policy_version 431883 (0.0011) [2023-12-27 08:37:55,952][20347] Updated weights for policy 0, policy_version 432206 (0.0007) [2023-12-27 08:37:56,006][20347] Updated weights for policy 0, policy_version 432216 (0.0007) [2023-12-27 08:37:56,570][20381] Updated weights for policy 1, policy_version 431893 (0.0009) [2023-12-27 08:37:56,622][20381] Updated weights for policy 1, policy_version 431903 (0.0005) [2023-12-27 08:37:56,661][20347] Updated weights for policy 0, policy_version 432226 (0.0007) [2023-12-27 08:37:56,666][20381] Updated weights for policy 1, policy_version 431913 (0.0006) [2023-12-27 08:37:56,717][20347] Updated weights for policy 0, policy_version 432236 (0.0005) [2023-12-27 08:37:56,770][20347] Updated weights for policy 0, policy_version 432246 (0.0007) [2023-12-27 08:37:56,819][20347] Updated weights for policy 0, policy_version 432256 (0.0008) [2023-12-27 08:37:57,291][20381] Updated weights for policy 1, policy_version 431923 (0.0007) [2023-12-27 08:37:57,339][20381] Updated weights for policy 1, policy_version 431933 (0.0010) [2023-12-27 08:37:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 221265920. Throughput: 0: 10133.3, 1: 10187.8. Samples: 221258512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:37:57,363][19170] Avg episode reward: [(0, '932.500'), (1, '1000.000')] [2023-12-27 08:37:57,396][20381] Updated weights for policy 1, policy_version 431943 (0.0010) [2023-12-27 08:37:57,466][20347] Updated weights for policy 0, policy_version 432266 (0.0008) [2023-12-27 08:37:57,510][20347] Updated weights for policy 0, policy_version 432276 (0.0008) [2023-12-27 08:37:57,558][20347] Updated weights for policy 0, policy_version 432286 (0.0008) [2023-12-27 08:37:57,948][20381] Updated weights for policy 1, policy_version 431953 (0.0009) [2023-12-27 08:37:57,995][20381] Updated weights for policy 1, policy_version 431963 (0.0005) [2023-12-27 08:37:58,076][20381] Updated weights for policy 1, policy_version 431973 (0.0008) [2023-12-27 08:37:58,130][20381] Updated weights for policy 1, policy_version 431983 (0.0010) [2023-12-27 08:37:58,266][20347] Updated weights for policy 0, policy_version 432296 (0.0007) [2023-12-27 08:37:58,325][20347] Updated weights for policy 0, policy_version 432306 (0.0007) [2023-12-27 08:37:58,386][20347] Updated weights for policy 0, policy_version 432316 (0.0008) [2023-12-27 08:37:58,885][20381] Updated weights for policy 1, policy_version 431993 (0.0008) [2023-12-27 08:37:58,932][20381] Updated weights for policy 1, policy_version 432003 (0.0006) [2023-12-27 08:37:58,995][20381] Updated weights for policy 1, policy_version 432013 (0.0008) [2023-12-27 08:37:59,129][20347] Updated weights for policy 0, policy_version 432326 (0.0007) [2023-12-27 08:37:59,185][20347] Updated weights for policy 0, policy_version 432336 (0.0007) [2023-12-27 08:37:59,244][20347] Updated weights for policy 0, policy_version 432346 (0.0010) [2023-12-27 08:37:59,736][20381] Updated weights for policy 1, policy_version 432023 (0.0009) [2023-12-27 08:37:59,797][20381] Updated weights for policy 1, policy_version 432033 (0.0008) [2023-12-27 08:37:59,822][20347] Updated weights for policy 0, policy_version 432356 (0.0010) [2023-12-27 08:37:59,860][20381] Updated weights for policy 1, policy_version 432043 (0.0008) [2023-12-27 08:37:59,883][20347] Updated weights for policy 0, policy_version 432366 (0.0008) [2023-12-27 08:37:59,944][20347] Updated weights for policy 0, policy_version 432376 (0.0007) [2023-12-27 08:38:00,517][20347] Updated weights for policy 0, policy_version 432386 (0.0008) [2023-12-27 08:38:00,520][20381] Updated weights for policy 1, policy_version 432053 (0.0008) [2023-12-27 08:38:00,566][20381] Updated weights for policy 1, policy_version 432063 (0.0007) [2023-12-27 08:38:00,579][20347] Updated weights for policy 0, policy_version 432396 (0.0005) [2023-12-27 08:38:00,616][20381] Updated weights for policy 1, policy_version 432073 (0.0007) [2023-12-27 08:38:00,640][20347] Updated weights for policy 0, policy_version 432406 (0.0005) [2023-12-27 08:38:00,689][20347] Updated weights for policy 0, policy_version 432416 (0.0005) [2023-12-27 08:38:01,292][20347] Updated weights for policy 0, policy_version 432426 (0.0010) [2023-12-27 08:38:01,348][20347] Updated weights for policy 0, policy_version 432436 (0.0010) [2023-12-27 08:38:01,391][20381] Updated weights for policy 1, policy_version 432083 (0.0008) [2023-12-27 08:38:01,410][20347] Updated weights for policy 0, policy_version 432446 (0.0008) [2023-12-27 08:38:01,452][20381] Updated weights for policy 1, policy_version 432093 (0.0007) [2023-12-27 08:38:01,510][20381] Updated weights for policy 1, policy_version 432103 (0.0008) [2023-12-27 08:38:02,122][20381] Updated weights for policy 1, policy_version 432113 (0.0008) [2023-12-27 08:38:02,135][20347] Updated weights for policy 0, policy_version 432456 (0.0007) [2023-12-27 08:38:02,179][20381] Updated weights for policy 1, policy_version 432123 (0.0006) [2023-12-27 08:38:02,188][20347] Updated weights for policy 0, policy_version 432466 (0.0008) [2023-12-27 08:38:02,233][20381] Updated weights for policy 1, policy_version 432133 (0.0005) [2023-12-27 08:38:02,233][20347] Updated weights for policy 0, policy_version 432476 (0.0007) [2023-12-27 08:38:02,288][20381] Updated weights for policy 1, policy_version 432143 (0.0006) [2023-12-27 08:38:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 19933.9, 300 sec: 19466.4). Total num frames: 221380608. Throughput: 0: 10277.5, 1: 10344.9. Samples: 221384904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:02,363][19170] Avg episode reward: [(0, '951.740'), (1, '1000.000')] [2023-12-27 08:38:02,896][20381] Updated weights for policy 1, policy_version 432153 (0.0005) [2023-12-27 08:38:02,955][20381] Updated weights for policy 1, policy_version 432163 (0.0006) [2023-12-27 08:38:02,959][20347] Updated weights for policy 0, policy_version 432486 (0.0006) [2023-12-27 08:38:03,012][20381] Updated weights for policy 1, policy_version 432173 (0.0005) [2023-12-27 08:38:03,017][20347] Updated weights for policy 0, policy_version 432496 (0.0006) [2023-12-27 08:38:03,072][20347] Updated weights for policy 0, policy_version 432506 (0.0007) [2023-12-27 08:38:03,605][20381] Updated weights for policy 1, policy_version 432183 (0.0005) [2023-12-27 08:38:03,646][20347] Updated weights for policy 0, policy_version 432516 (0.0007) [2023-12-27 08:38:03,652][20381] Updated weights for policy 1, policy_version 432193 (0.0006) [2023-12-27 08:38:03,692][20347] Updated weights for policy 0, policy_version 432526 (0.0006) [2023-12-27 08:38:03,698][20381] Updated weights for policy 1, policy_version 432203 (0.0005) [2023-12-27 08:38:03,739][20347] Updated weights for policy 0, policy_version 432536 (0.0007) [2023-12-27 08:38:04,330][20381] Updated weights for policy 1, policy_version 432213 (0.0007) [2023-12-27 08:38:04,390][20381] Updated weights for policy 1, policy_version 432223 (0.0008) [2023-12-27 08:38:04,449][20381] Updated weights for policy 1, policy_version 432233 (0.0007) [2023-12-27 08:38:04,451][20347] Updated weights for policy 0, policy_version 432546 (0.0007) [2023-12-27 08:38:04,501][20347] Updated weights for policy 0, policy_version 432556 (0.0006) [2023-12-27 08:38:04,548][20347] Updated weights for policy 0, policy_version 432566 (0.0008) [2023-12-27 08:38:04,597][20347] Updated weights for policy 0, policy_version 432576 (0.0007) [2023-12-27 08:38:05,086][20381] Updated weights for policy 1, policy_version 432243 (0.0006) [2023-12-27 08:38:05,137][20381] Updated weights for policy 1, policy_version 432253 (0.0006) [2023-12-27 08:38:05,186][20381] Updated weights for policy 1, policy_version 432263 (0.0007) [2023-12-27 08:38:05,279][20347] Updated weights for policy 0, policy_version 432586 (0.0008) [2023-12-27 08:38:05,328][20347] Updated weights for policy 0, policy_version 432596 (0.0008) [2023-12-27 08:38:05,376][20347] Updated weights for policy 0, policy_version 432606 (0.0007) [2023-12-27 08:38:05,900][20381] Updated weights for policy 1, policy_version 432273 (0.0008) [2023-12-27 08:38:05,960][20381] Updated weights for policy 1, policy_version 432283 (0.0007) [2023-12-27 08:38:05,988][20347] Updated weights for policy 0, policy_version 432616 (0.0006) [2023-12-27 08:38:06,005][20381] Updated weights for policy 1, policy_version 432293 (0.0007) [2023-12-27 08:38:06,036][20347] Updated weights for policy 0, policy_version 432626 (0.0005) [2023-12-27 08:38:06,054][20381] Updated weights for policy 1, policy_version 432303 (0.0007) [2023-12-27 08:38:06,091][20347] Updated weights for policy 0, policy_version 432636 (0.0005) [2023-12-27 08:38:06,720][20347] Updated weights for policy 0, policy_version 432646 (0.0008) [2023-12-27 08:38:06,784][20347] Updated weights for policy 0, policy_version 432656 (0.0008) [2023-12-27 08:38:06,843][20347] Updated weights for policy 0, policy_version 432666 (0.0005) [2023-12-27 08:38:06,868][20381] Updated weights for policy 1, policy_version 432313 (0.0007) [2023-12-27 08:38:06,915][20381] Updated weights for policy 1, policy_version 432323 (0.0007) [2023-12-27 08:38:06,974][20381] Updated weights for policy 1, policy_version 432333 (0.0007) [2023-12-27 08:38:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20207.0, 300 sec: 19466.4). Total num frames: 221478912. Throughput: 0: 10283.9, 1: 10413.2. Samples: 221449336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:07,363][19170] Avg episode reward: [(0, '971.160'), (1, '1000.000')] [2023-12-27 08:38:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000432672_110780416.pth... [2023-12-27 08:38:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000432336_110698496.pth... [2023-12-27 08:38:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000431456_110469120.pth [2023-12-27 08:38:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000431120_110387200.pth [2023-12-27 08:38:07,508][20347] Updated weights for policy 0, policy_version 432676 (0.0006) [2023-12-27 08:38:07,560][20347] Updated weights for policy 0, policy_version 432686 (0.0008) [2023-12-27 08:38:07,605][20347] Updated weights for policy 0, policy_version 432696 (0.0005) [2023-12-27 08:38:07,668][20381] Updated weights for policy 1, policy_version 432343 (0.0007) [2023-12-27 08:38:07,729][20381] Updated weights for policy 1, policy_version 432353 (0.0010) [2023-12-27 08:38:07,791][20381] Updated weights for policy 1, policy_version 432363 (0.0010) [2023-12-27 08:38:08,174][20347] Updated weights for policy 0, policy_version 432706 (0.0006) [2023-12-27 08:38:08,232][20347] Updated weights for policy 0, policy_version 432716 (0.0005) [2023-12-27 08:38:08,293][20347] Updated weights for policy 0, policy_version 432726 (0.0007) [2023-12-27 08:38:08,356][20347] Updated weights for policy 0, policy_version 432736 (0.0006) [2023-12-27 08:38:08,445][20381] Updated weights for policy 1, policy_version 432373 (0.0011) [2023-12-27 08:38:08,500][20381] Updated weights for policy 1, policy_version 432383 (0.0010) [2023-12-27 08:38:08,562][20381] Updated weights for policy 1, policy_version 432393 (0.0010) [2023-12-27 08:38:08,949][20347] Updated weights for policy 0, policy_version 432746 (0.0006) [2023-12-27 08:38:09,006][20347] Updated weights for policy 0, policy_version 432756 (0.0005) [2023-12-27 08:38:09,063][20347] Updated weights for policy 0, policy_version 432766 (0.0008) [2023-12-27 08:38:09,232][20381] Updated weights for policy 1, policy_version 432403 (0.0009) [2023-12-27 08:38:09,286][20381] Updated weights for policy 1, policy_version 432413 (0.0008) [2023-12-27 08:38:09,336][20381] Updated weights for policy 1, policy_version 432423 (0.0008) [2023-12-27 08:38:09,689][20347] Updated weights for policy 0, policy_version 432776 (0.0006) [2023-12-27 08:38:09,752][20347] Updated weights for policy 0, policy_version 432786 (0.0006) [2023-12-27 08:38:09,812][20347] Updated weights for policy 0, policy_version 432796 (0.0006) [2023-12-27 08:38:10,113][20381] Updated weights for policy 1, policy_version 432433 (0.0009) [2023-12-27 08:38:10,167][20381] Updated weights for policy 1, policy_version 432443 (0.0008) [2023-12-27 08:38:10,221][20381] Updated weights for policy 1, policy_version 432453 (0.0008) [2023-12-27 08:38:10,267][20381] Updated weights for policy 1, policy_version 432463 (0.0008) [2023-12-27 08:38:10,492][20347] Updated weights for policy 0, policy_version 432806 (0.0008) [2023-12-27 08:38:10,549][20347] Updated weights for policy 0, policy_version 432816 (0.0009) [2023-12-27 08:38:10,605][20347] Updated weights for policy 0, policy_version 432826 (0.0008) [2023-12-27 08:38:10,971][20381] Updated weights for policy 1, policy_version 432473 (0.0007) [2023-12-27 08:38:11,017][20381] Updated weights for policy 1, policy_version 432483 (0.0006) [2023-12-27 08:38:11,079][20381] Updated weights for policy 1, policy_version 432493 (0.0008) [2023-12-27 08:38:11,352][20347] Updated weights for policy 0, policy_version 432836 (0.0008) [2023-12-27 08:38:11,418][20347] Updated weights for policy 0, policy_version 432846 (0.0009) [2023-12-27 08:38:11,470][20347] Updated weights for policy 0, policy_version 432856 (0.0007) [2023-12-27 08:38:11,791][20381] Updated weights for policy 1, policy_version 432503 (0.0008) [2023-12-27 08:38:11,846][20381] Updated weights for policy 1, policy_version 432513 (0.0008) [2023-12-27 08:38:11,905][20381] Updated weights for policy 1, policy_version 432523 (0.0007) [2023-12-27 08:38:12,221][20347] Updated weights for policy 0, policy_version 432866 (0.0008) [2023-12-27 08:38:12,280][20347] Updated weights for policy 0, policy_version 432876 (0.0009) [2023-12-27 08:38:12,340][20347] Updated weights for policy 0, policy_version 432886 (0.0009) [2023-12-27 08:38:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 19438.6). Total num frames: 221577216. Throughput: 0: 10400.1, 1: 10301.2. Samples: 221573432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:12,363][19170] Avg episode reward: [(0, '980.880'), (1, '1000.000')] [2023-12-27 08:38:12,401][20347] Updated weights for policy 0, policy_version 432896 (0.0010) [2023-12-27 08:38:12,626][20381] Updated weights for policy 1, policy_version 432533 (0.0009) [2023-12-27 08:38:12,678][20381] Updated weights for policy 1, policy_version 432543 (0.0007) [2023-12-27 08:38:12,724][20381] Updated weights for policy 1, policy_version 432553 (0.0010) [2023-12-27 08:38:13,051][20347] Updated weights for policy 0, policy_version 432906 (0.0009) [2023-12-27 08:38:13,107][20347] Updated weights for policy 0, policy_version 432916 (0.0005) [2023-12-27 08:38:13,164][20347] Updated weights for policy 0, policy_version 432926 (0.0005) [2023-12-27 08:38:13,494][20381] Updated weights for policy 1, policy_version 432563 (0.0007) [2023-12-27 08:38:13,556][20381] Updated weights for policy 1, policy_version 432573 (0.0008) [2023-12-27 08:38:13,611][20381] Updated weights for policy 1, policy_version 432583 (0.0008) [2023-12-27 08:38:13,692][20347] Updated weights for policy 0, policy_version 432936 (0.0006) [2023-12-27 08:38:13,738][20347] Updated weights for policy 0, policy_version 432946 (0.0005) [2023-12-27 08:38:13,783][20347] Updated weights for policy 0, policy_version 432956 (0.0005) [2023-12-27 08:38:14,333][20381] Updated weights for policy 1, policy_version 432593 (0.0007) [2023-12-27 08:38:14,386][20381] Updated weights for policy 1, policy_version 432603 (0.0008) [2023-12-27 08:38:14,441][20381] Updated weights for policy 1, policy_version 432613 (0.0006) [2023-12-27 08:38:14,451][20347] Updated weights for policy 0, policy_version 432966 (0.0010) [2023-12-27 08:38:14,493][20381] Updated weights for policy 1, policy_version 432623 (0.0005) [2023-12-27 08:38:14,503][20347] Updated weights for policy 0, policy_version 432976 (0.0010) [2023-12-27 08:38:14,555][20347] Updated weights for policy 0, policy_version 432986 (0.0010) [2023-12-27 08:38:15,205][20381] Updated weights for policy 1, policy_version 432633 (0.0007) [2023-12-27 08:38:15,263][20347] Updated weights for policy 0, policy_version 432996 (0.0010) [2023-12-27 08:38:15,265][20381] Updated weights for policy 1, policy_version 432643 (0.0007) [2023-12-27 08:38:15,318][20381] Updated weights for policy 1, policy_version 432653 (0.0005) [2023-12-27 08:38:15,320][20347] Updated weights for policy 0, policy_version 433006 (0.0010) [2023-12-27 08:38:15,372][20347] Updated weights for policy 0, policy_version 433016 (0.0010) [2023-12-27 08:38:16,024][20381] Updated weights for policy 1, policy_version 432663 (0.0007) [2023-12-27 08:38:16,072][20381] Updated weights for policy 1, policy_version 432673 (0.0008) [2023-12-27 08:38:16,075][20347] Updated weights for policy 0, policy_version 433026 (0.0011) [2023-12-27 08:38:16,126][20347] Updated weights for policy 0, policy_version 433036 (0.0010) [2023-12-27 08:38:16,127][20381] Updated weights for policy 1, policy_version 432683 (0.0005) [2023-12-27 08:38:16,180][20347] Updated weights for policy 0, policy_version 433046 (0.0010) [2023-12-27 08:38:16,228][20347] Updated weights for policy 0, policy_version 433056 (0.0010) [2023-12-27 08:38:16,845][20381] Updated weights for policy 1, policy_version 432693 (0.0007) [2023-12-27 08:38:16,895][20381] Updated weights for policy 1, policy_version 432703 (0.0007) [2023-12-27 08:38:16,904][20347] Updated weights for policy 0, policy_version 433066 (0.0010) [2023-12-27 08:38:16,953][20381] Updated weights for policy 1, policy_version 432713 (0.0005) [2023-12-27 08:38:16,958][20347] Updated weights for policy 0, policy_version 433076 (0.0010) [2023-12-27 08:38:17,010][20347] Updated weights for policy 0, policy_version 433086 (0.0010) [2023-12-27 08:38:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 19438.6). Total num frames: 221683712. Throughput: 0: 10412.7, 1: 10207.4. Samples: 221693672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:17,363][19170] Avg episode reward: [(0, '980.900'), (1, '1000.000')] [2023-12-27 08:38:17,656][20381] Updated weights for policy 1, policy_version 432723 (0.0006) [2023-12-27 08:38:17,683][20347] Updated weights for policy 0, policy_version 433096 (0.0010) [2023-12-27 08:38:17,707][20381] Updated weights for policy 1, policy_version 432733 (0.0010) [2023-12-27 08:38:17,741][20347] Updated weights for policy 0, policy_version 433106 (0.0010) [2023-12-27 08:38:17,765][20381] Updated weights for policy 1, policy_version 432743 (0.0010) [2023-12-27 08:38:17,792][20347] Updated weights for policy 0, policy_version 433116 (0.0010) [2023-12-27 08:38:18,467][20381] Updated weights for policy 1, policy_version 432753 (0.0010) [2023-12-27 08:38:18,509][20347] Updated weights for policy 0, policy_version 433126 (0.0010) [2023-12-27 08:38:18,523][20381] Updated weights for policy 1, policy_version 432763 (0.0011) [2023-12-27 08:38:18,558][20347] Updated weights for policy 0, policy_version 433136 (0.0010) [2023-12-27 08:38:18,575][20381] Updated weights for policy 1, policy_version 432773 (0.0010) [2023-12-27 08:38:18,621][20347] Updated weights for policy 0, policy_version 433146 (0.0011) [2023-12-27 08:38:18,639][20381] Updated weights for policy 1, policy_version 432783 (0.0010) [2023-12-27 08:38:19,233][20347] Updated weights for policy 0, policy_version 433156 (0.0010) [2023-12-27 08:38:19,296][20347] Updated weights for policy 0, policy_version 433166 (0.0008) [2023-12-27 08:38:19,297][20381] Updated weights for policy 1, policy_version 432793 (0.0010) [2023-12-27 08:38:19,359][20347] Updated weights for policy 0, policy_version 433176 (0.0008) [2023-12-27 08:38:19,362][20381] Updated weights for policy 1, policy_version 432803 (0.0009) [2023-12-27 08:38:19,432][20381] Updated weights for policy 1, policy_version 432813 (0.0006) [2023-12-27 08:38:20,031][20381] Updated weights for policy 1, policy_version 432823 (0.0008) [2023-12-27 08:38:20,092][20381] Updated weights for policy 1, policy_version 432833 (0.0006) [2023-12-27 08:38:20,095][20347] Updated weights for policy 0, policy_version 433186 (0.0008) [2023-12-27 08:38:20,153][20381] Updated weights for policy 1, policy_version 432843 (0.0006) [2023-12-27 08:38:20,167][20347] Updated weights for policy 0, policy_version 433196 (0.0008) [2023-12-27 08:38:20,231][20347] Updated weights for policy 0, policy_version 433206 (0.0006) [2023-12-27 08:38:20,289][20347] Updated weights for policy 0, policy_version 433216 (0.0010) [2023-12-27 08:38:20,848][20381] Updated weights for policy 1, policy_version 432853 (0.0009) [2023-12-27 08:38:20,907][20381] Updated weights for policy 1, policy_version 432863 (0.0007) [2023-12-27 08:38:20,931][20347] Updated weights for policy 0, policy_version 433226 (0.0007) [2023-12-27 08:38:20,949][20291] KL-divergence is very high: 151.2511 [2023-12-27 08:38:20,963][20381] Updated weights for policy 1, policy_version 432873 (0.0008) [2023-12-27 08:38:20,985][20347] Updated weights for policy 0, policy_version 433236 (0.0007) [2023-12-27 08:38:21,044][20347] Updated weights for policy 0, policy_version 433246 (0.0006) [2023-12-27 08:38:21,659][20381] Updated weights for policy 1, policy_version 432883 (0.0007) [2023-12-27 08:38:21,722][20381] Updated weights for policy 1, policy_version 432893 (0.0007) [2023-12-27 08:38:21,763][20347] Updated weights for policy 0, policy_version 433256 (0.0008) [2023-12-27 08:38:21,771][20381] Updated weights for policy 1, policy_version 432903 (0.0007) [2023-12-27 08:38:21,823][20347] Updated weights for policy 0, policy_version 433266 (0.0008) [2023-12-27 08:38:21,881][20347] Updated weights for policy 0, policy_version 433276 (0.0007) [2023-12-27 08:38:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 19410.9). Total num frames: 221782016. Throughput: 0: 10404.2, 1: 10225.9. Samples: 221754760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:22,363][19170] Avg episode reward: [(0, '971.250'), (1, '971.010')] [2023-12-27 08:38:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000432912_110845952.pth... [2023-12-27 08:38:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000433280_110936064.pth... [2023-12-27 08:38:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000431728_110542848.pth [2023-12-27 08:38:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000432064_110624768.pth [2023-12-27 08:38:22,520][20347] Updated weights for policy 0, policy_version 433286 (0.0007) [2023-12-27 08:38:22,579][20381] Updated weights for policy 1, policy_version 432913 (0.0007) [2023-12-27 08:38:22,581][20347] Updated weights for policy 0, policy_version 433296 (0.0008) [2023-12-27 08:38:22,623][20292] KL-divergence is very high: 280.2740 [2023-12-27 08:38:22,637][20381] Updated weights for policy 1, policy_version 432923 (0.0006) [2023-12-27 08:38:22,638][20347] Updated weights for policy 0, policy_version 433306 (0.0008) [2023-12-27 08:38:22,668][20292] KL-divergence is very high: 159.8806 [2023-12-27 08:38:22,698][20381] Updated weights for policy 1, policy_version 432933 (0.0007) [2023-12-27 08:38:22,759][20381] Updated weights for policy 1, policy_version 432943 (0.0008) [2023-12-27 08:38:23,323][20347] Updated weights for policy 0, policy_version 433316 (0.0007) [2023-12-27 08:38:23,390][20347] Updated weights for policy 0, policy_version 433326 (0.0005) [2023-12-27 08:38:23,444][20381] Updated weights for policy 1, policy_version 432953 (0.0006) [2023-12-27 08:38:23,455][20347] Updated weights for policy 0, policy_version 433336 (0.0005) [2023-12-27 08:38:23,502][20381] Updated weights for policy 1, policy_version 432963 (0.0006) [2023-12-27 08:38:23,564][20381] Updated weights for policy 1, policy_version 432973 (0.0008) [2023-12-27 08:38:24,026][20347] Updated weights for policy 0, policy_version 433346 (0.0006) [2023-12-27 08:38:24,074][20347] Updated weights for policy 0, policy_version 433356 (0.0007) [2023-12-27 08:38:24,133][20347] Updated weights for policy 0, policy_version 433366 (0.0005) [2023-12-27 08:38:24,190][20347] Updated weights for policy 0, policy_version 433376 (0.0006) [2023-12-27 08:38:24,252][20381] Updated weights for policy 1, policy_version 432983 (0.0008) [2023-12-27 08:38:24,314][20381] Updated weights for policy 1, policy_version 432993 (0.0008) [2023-12-27 08:38:24,371][20381] Updated weights for policy 1, policy_version 433003 (0.0008) [2023-12-27 08:38:24,850][20347] Updated weights for policy 0, policy_version 433386 (0.0007) [2023-12-27 08:38:24,909][20347] Updated weights for policy 0, policy_version 433396 (0.0008) [2023-12-27 08:38:24,967][20347] Updated weights for policy 0, policy_version 433406 (0.0007) [2023-12-27 08:38:25,053][20381] Updated weights for policy 1, policy_version 433013 (0.0007) [2023-12-27 08:38:25,102][20381] Updated weights for policy 1, policy_version 433023 (0.0007) [2023-12-27 08:38:25,147][20381] Updated weights for policy 1, policy_version 433033 (0.0008) [2023-12-27 08:38:25,657][20347] Updated weights for policy 0, policy_version 433416 (0.0007) [2023-12-27 08:38:25,712][20347] Updated weights for policy 0, policy_version 433426 (0.0007) [2023-12-27 08:38:25,758][20347] Updated weights for policy 0, policy_version 433436 (0.0007) [2023-12-27 08:38:25,854][20381] Updated weights for policy 1, policy_version 433043 (0.0007) [2023-12-27 08:38:25,906][20381] Updated weights for policy 1, policy_version 433053 (0.0007) [2023-12-27 08:38:25,963][20381] Updated weights for policy 1, policy_version 433063 (0.0008) [2023-12-27 08:38:26,431][20347] Updated weights for policy 0, policy_version 433446 (0.0007) [2023-12-27 08:38:26,485][20347] Updated weights for policy 0, policy_version 433456 (0.0007) [2023-12-27 08:38:26,544][20347] Updated weights for policy 0, policy_version 433466 (0.0007) [2023-12-27 08:38:26,663][20381] Updated weights for policy 1, policy_version 433073 (0.0007) [2023-12-27 08:38:26,718][20381] Updated weights for policy 1, policy_version 433083 (0.0008) [2023-12-27 08:38:26,770][20381] Updated weights for policy 1, policy_version 433093 (0.0008) [2023-12-27 08:38:26,826][20381] Updated weights for policy 1, policy_version 433103 (0.0008) [2023-12-27 08:38:27,219][20347] Updated weights for policy 0, policy_version 433476 (0.0007) [2023-12-27 08:38:27,240][20291] KL-divergence is very high: 166.6525 [2023-12-27 08:38:27,274][20347] Updated weights for policy 0, policy_version 433486 (0.0007) [2023-12-27 08:38:27,323][20347] Updated weights for policy 0, policy_version 433496 (0.0008) [2023-12-27 08:38:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 19383.1). Total num frames: 221880320. Throughput: 0: 10423.8, 1: 10205.1. Samples: 221876488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:27,363][19170] Avg episode reward: [(0, '961.400'), (1, '893.430')] [2023-12-27 08:38:27,479][20381] Updated weights for policy 1, policy_version 433113 (0.0007) [2023-12-27 08:38:27,523][20381] Updated weights for policy 1, policy_version 433123 (0.0008) [2023-12-27 08:38:27,567][20381] Updated weights for policy 1, policy_version 433133 (0.0007) [2023-12-27 08:38:28,035][20347] Updated weights for policy 0, policy_version 433506 (0.0008) [2023-12-27 08:38:28,090][20347] Updated weights for policy 0, policy_version 433516 (0.0008) [2023-12-27 08:38:28,151][20347] Updated weights for policy 0, policy_version 433526 (0.0007) [2023-12-27 08:38:28,208][20381] Updated weights for policy 1, policy_version 433143 (0.0007) [2023-12-27 08:38:28,210][20347] Updated weights for policy 0, policy_version 433536 (0.0007) [2023-12-27 08:38:28,261][20381] Updated weights for policy 1, policy_version 433153 (0.0007) [2023-12-27 08:38:28,315][20381] Updated weights for policy 1, policy_version 433163 (0.0006) [2023-12-27 08:38:28,903][20347] Updated weights for policy 0, policy_version 433546 (0.0007) [2023-12-27 08:38:28,956][20347] Updated weights for policy 0, policy_version 433556 (0.0007) [2023-12-27 08:38:28,978][20381] Updated weights for policy 1, policy_version 433173 (0.0008) [2023-12-27 08:38:29,011][20347] Updated weights for policy 0, policy_version 433566 (0.0006) [2023-12-27 08:38:29,024][20381] Updated weights for policy 1, policy_version 433183 (0.0007) [2023-12-27 08:38:29,068][20381] Updated weights for policy 1, policy_version 433193 (0.0008) [2023-12-27 08:38:29,713][20347] Updated weights for policy 0, policy_version 433576 (0.0007) [2023-12-27 08:38:29,765][20347] Updated weights for policy 0, policy_version 433586 (0.0007) [2023-12-27 08:38:29,775][20381] Updated weights for policy 1, policy_version 433203 (0.0007) [2023-12-27 08:38:29,818][20347] Updated weights for policy 0, policy_version 433596 (0.0006) [2023-12-27 08:38:29,836][20381] Updated weights for policy 1, policy_version 433213 (0.0008) [2023-12-27 08:38:29,897][20381] Updated weights for policy 1, policy_version 433223 (0.0007) [2023-12-27 08:38:30,533][20347] Updated weights for policy 0, policy_version 433606 (0.0009) [2023-12-27 08:38:30,554][20381] Updated weights for policy 1, policy_version 433233 (0.0013) [2023-12-27 08:38:30,584][20347] Updated weights for policy 0, policy_version 433616 (0.0006) [2023-12-27 08:38:30,609][20381] Updated weights for policy 1, policy_version 433243 (0.0008) [2023-12-27 08:38:30,642][20347] Updated weights for policy 0, policy_version 433626 (0.0007) [2023-12-27 08:38:30,667][20381] Updated weights for policy 1, policy_version 433253 (0.0009) [2023-12-27 08:38:30,719][20381] Updated weights for policy 1, policy_version 433263 (0.0007) [2023-12-27 08:38:31,312][20381] Updated weights for policy 1, policy_version 433273 (0.0007) [2023-12-27 08:38:31,358][20347] Updated weights for policy 0, policy_version 433636 (0.0008) [2023-12-27 08:38:31,376][20381] Updated weights for policy 1, policy_version 433283 (0.0009) [2023-12-27 08:38:31,420][20347] Updated weights for policy 0, policy_version 433646 (0.0006) [2023-12-27 08:38:31,433][20381] Updated weights for policy 1, policy_version 433293 (0.0011) [2023-12-27 08:38:31,472][20347] Updated weights for policy 0, policy_version 433656 (0.0006) [2023-12-27 08:38:32,054][20381] Updated weights for policy 1, policy_version 433303 (0.0007) [2023-12-27 08:38:32,112][20381] Updated weights for policy 1, policy_version 433313 (0.0006) [2023-12-27 08:38:32,161][20381] Updated weights for policy 1, policy_version 433323 (0.0008) [2023-12-27 08:38:32,175][20347] Updated weights for policy 0, policy_version 433666 (0.0007) [2023-12-27 08:38:32,233][20347] Updated weights for policy 0, policy_version 433676 (0.0007) [2023-12-27 08:38:32,296][20347] Updated weights for policy 0, policy_version 433686 (0.0007) [2023-12-27 08:38:32,358][20347] Updated weights for policy 0, policy_version 433696 (0.0008) [2023-12-27 08:38:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 19438.7). Total num frames: 221995008. Throughput: 0: 10421.8, 1: 10229.0. Samples: 221999632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:38:32,363][19170] Avg episode reward: [(0, '971.100'), (1, '845.010')] [2023-12-27 08:38:32,730][20381] Updated weights for policy 1, policy_version 433333 (0.0006) [2023-12-27 08:38:32,791][20381] Updated weights for policy 1, policy_version 433343 (0.0009) [2023-12-27 08:38:32,844][20381] Updated weights for policy 1, policy_version 433353 (0.0010) [2023-12-27 08:38:33,073][20347] Updated weights for policy 0, policy_version 433706 (0.0007) [2023-12-27 08:38:33,125][20347] Updated weights for policy 0, policy_version 433716 (0.0007) [2023-12-27 08:38:33,179][20347] Updated weights for policy 0, policy_version 433726 (0.0007) [2023-12-27 08:38:33,536][20381] Updated weights for policy 1, policy_version 433363 (0.0010) [2023-12-27 08:38:33,590][20381] Updated weights for policy 1, policy_version 433373 (0.0010) [2023-12-27 08:38:33,596][20292] KL-divergence is very high: 427.5237 [2023-12-27 08:38:33,644][20381] Updated weights for policy 1, policy_version 433383 (0.0010) [2023-12-27 08:38:33,886][20347] Updated weights for policy 0, policy_version 433736 (0.0008) [2023-12-27 08:38:33,937][20347] Updated weights for policy 0, policy_version 433746 (0.0008) [2023-12-27 08:38:33,988][20347] Updated weights for policy 0, policy_version 433756 (0.0008) [2023-12-27 08:38:34,237][20381] Updated weights for policy 1, policy_version 433393 (0.0009) [2023-12-27 08:38:34,296][20381] Updated weights for policy 1, policy_version 433403 (0.0008) [2023-12-27 08:38:34,354][20381] Updated weights for policy 1, policy_version 433413 (0.0008) [2023-12-27 08:38:34,411][20381] Updated weights for policy 1, policy_version 433423 (0.0007) [2023-12-27 08:38:34,704][20347] Updated weights for policy 0, policy_version 433766 (0.0007) [2023-12-27 08:38:34,772][20347] Updated weights for policy 0, policy_version 433776 (0.0005) [2023-12-27 08:38:34,834][20347] Updated weights for policy 0, policy_version 433786 (0.0006) [2023-12-27 08:38:35,149][20381] Updated weights for policy 1, policy_version 433433 (0.0007) [2023-12-27 08:38:35,205][20381] Updated weights for policy 1, policy_version 433443 (0.0008) [2023-12-27 08:38:35,257][20381] Updated weights for policy 1, policy_version 433453 (0.0007) [2023-12-27 08:38:35,426][20347] Updated weights for policy 0, policy_version 433796 (0.0006) [2023-12-27 08:38:35,477][20347] Updated weights for policy 0, policy_version 433806 (0.0005) [2023-12-27 08:38:35,534][20347] Updated weights for policy 0, policy_version 433816 (0.0010) [2023-12-27 08:38:35,998][20381] Updated weights for policy 1, policy_version 433463 (0.0007) [2023-12-27 08:38:36,061][20292] KL-divergence is very high: 363.1822 [2023-12-27 08:38:36,061][20381] Updated weights for policy 1, policy_version 433473 (0.0008) [2023-12-27 08:38:36,101][20347] Updated weights for policy 0, policy_version 433826 (0.0010) [2023-12-27 08:38:36,108][20292] KL-divergence is very high: 261.1934 [2023-12-27 08:38:36,121][20381] Updated weights for policy 1, policy_version 433483 (0.0008) [2023-12-27 08:38:36,163][20347] Updated weights for policy 0, policy_version 433836 (0.0010) [2023-12-27 08:38:36,221][20347] Updated weights for policy 0, policy_version 433846 (0.0010) [2023-12-27 08:38:36,274][20347] Updated weights for policy 0, policy_version 433856 (0.0010) [2023-12-27 08:38:36,809][20381] Updated weights for policy 1, policy_version 433493 (0.0007) [2023-12-27 08:38:36,869][20381] Updated weights for policy 1, policy_version 433503 (0.0006) [2023-12-27 08:38:36,922][20381] Updated weights for policy 1, policy_version 433513 (0.0007) [2023-12-27 08:38:36,958][20347] Updated weights for policy 0, policy_version 433866 (0.0008) [2023-12-27 08:38:37,025][20347] Updated weights for policy 0, policy_version 433876 (0.0008) [2023-12-27 08:38:37,085][20347] Updated weights for policy 0, policy_version 433886 (0.0009) [2023-12-27 08:38:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 19410.9). Total num frames: 222093312. Throughput: 0: 10394.0, 1: 10276.2. Samples: 222061400. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:38:37,363][19170] Avg episode reward: [(0, '980.870'), (1, '884.810')] [2023-12-27 08:38:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000433520_111001600.pth... [2023-12-27 08:38:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000433888_111091712.pth... [2023-12-27 08:38:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000432336_110698496.pth [2023-12-27 08:38:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000432672_110780416.pth [2023-12-27 08:38:37,527][20381] Updated weights for policy 1, policy_version 433523 (0.0006) [2023-12-27 08:38:37,582][20381] Updated weights for policy 1, policy_version 433533 (0.0005) [2023-12-27 08:38:37,642][20381] Updated weights for policy 1, policy_version 433543 (0.0005) [2023-12-27 08:38:37,804][20347] Updated weights for policy 0, policy_version 433896 (0.0008) [2023-12-27 08:38:37,856][20347] Updated weights for policy 0, policy_version 433906 (0.0007) [2023-12-27 08:38:37,911][20347] Updated weights for policy 0, policy_version 433916 (0.0007) [2023-12-27 08:38:38,308][20381] Updated weights for policy 1, policy_version 433553 (0.0007) [2023-12-27 08:38:38,370][20381] Updated weights for policy 1, policy_version 433563 (0.0008) [2023-12-27 08:38:38,427][20381] Updated weights for policy 1, policy_version 433573 (0.0007) [2023-12-27 08:38:38,476][20381] Updated weights for policy 1, policy_version 433583 (0.0007) [2023-12-27 08:38:38,605][20347] Updated weights for policy 0, policy_version 433926 (0.0006) [2023-12-27 08:38:38,664][20347] Updated weights for policy 0, policy_version 433936 (0.0005) [2023-12-27 08:38:38,713][20347] Updated weights for policy 0, policy_version 433946 (0.0005) [2023-12-27 08:38:39,217][20381] Updated weights for policy 1, policy_version 433593 (0.0007) [2023-12-27 08:38:39,279][20381] Updated weights for policy 1, policy_version 433603 (0.0009) [2023-12-27 08:38:39,342][20347] Updated weights for policy 0, policy_version 433956 (0.0007) [2023-12-27 08:38:39,344][20381] Updated weights for policy 1, policy_version 433613 (0.0008) [2023-12-27 08:38:39,413][20347] Updated weights for policy 0, policy_version 433966 (0.0007) [2023-12-27 08:38:39,478][20347] Updated weights for policy 0, policy_version 433976 (0.0007) [2023-12-27 08:38:40,052][20381] Updated weights for policy 1, policy_version 433623 (0.0008) [2023-12-27 08:38:40,112][20381] Updated weights for policy 1, policy_version 433633 (0.0007) [2023-12-27 08:38:40,157][20347] Updated weights for policy 0, policy_version 433986 (0.0007) [2023-12-27 08:38:40,175][20381] Updated weights for policy 1, policy_version 433643 (0.0008) [2023-12-27 08:38:40,211][20347] Updated weights for policy 0, policy_version 433996 (0.0008) [2023-12-27 08:38:40,275][20347] Updated weights for policy 0, policy_version 434006 (0.0007) [2023-12-27 08:38:40,331][20347] Updated weights for policy 0, policy_version 434016 (0.0008) [2023-12-27 08:38:40,848][20381] Updated weights for policy 1, policy_version 433653 (0.0008) [2023-12-27 08:38:40,908][20381] Updated weights for policy 1, policy_version 433663 (0.0007) [2023-12-27 08:38:40,968][20381] Updated weights for policy 1, policy_version 433673 (0.0008) [2023-12-27 08:38:41,032][20347] Updated weights for policy 0, policy_version 434026 (0.0009) [2023-12-27 08:38:41,084][20347] Updated weights for policy 0, policy_version 434036 (0.0008) [2023-12-27 08:38:41,137][20347] Updated weights for policy 0, policy_version 434046 (0.0008) [2023-12-27 08:38:41,680][20381] Updated weights for policy 1, policy_version 433683 (0.0009) [2023-12-27 08:38:41,747][20381] Updated weights for policy 1, policy_version 433693 (0.0007) [2023-12-27 08:38:41,814][20381] Updated weights for policy 1, policy_version 433703 (0.0008) [2023-12-27 08:38:41,892][20347] Updated weights for policy 0, policy_version 434056 (0.0007) [2023-12-27 08:38:41,941][20347] Updated weights for policy 0, policy_version 434066 (0.0005) [2023-12-27 08:38:41,994][20347] Updated weights for policy 0, policy_version 434076 (0.0006) [2023-12-27 08:38:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 19383.1). Total num frames: 222191616. Throughput: 0: 10354.8, 1: 10205.4. Samples: 222183720. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:38:42,363][19170] Avg episode reward: [(0, '922.970'), (1, '922.470')] [2023-12-27 08:38:42,557][20381] Updated weights for policy 1, policy_version 433713 (0.0007) [2023-12-27 08:38:42,594][20347] Updated weights for policy 0, policy_version 434086 (0.0008) [2023-12-27 08:38:42,610][20381] Updated weights for policy 1, policy_version 433723 (0.0009) [2023-12-27 08:38:42,643][20347] Updated weights for policy 0, policy_version 434096 (0.0007) [2023-12-27 08:38:42,661][20381] Updated weights for policy 1, policy_version 433733 (0.0010) [2023-12-27 08:38:42,693][20347] Updated weights for policy 0, policy_version 434106 (0.0006) [2023-12-27 08:38:42,716][20381] Updated weights for policy 1, policy_version 433743 (0.0010) [2023-12-27 08:38:43,306][20381] Updated weights for policy 1, policy_version 433753 (0.0010) [2023-12-27 08:38:43,352][20381] Updated weights for policy 1, policy_version 433763 (0.0010) [2023-12-27 08:38:43,397][20381] Updated weights for policy 1, policy_version 433773 (0.0010) [2023-12-27 08:38:43,399][20347] Updated weights for policy 0, policy_version 434116 (0.0007) [2023-12-27 08:38:43,458][20347] Updated weights for policy 0, policy_version 434126 (0.0006) [2023-12-27 08:38:43,522][20347] Updated weights for policy 0, policy_version 434136 (0.0007) [2023-12-27 08:38:44,046][20381] Updated weights for policy 1, policy_version 433783 (0.0007) [2023-12-27 08:38:44,112][20381] Updated weights for policy 1, policy_version 433793 (0.0008) [2023-12-27 08:38:44,177][20381] Updated weights for policy 1, policy_version 433803 (0.0008) [2023-12-27 08:38:44,220][20347] Updated weights for policy 0, policy_version 434146 (0.0007) [2023-12-27 08:38:44,267][20347] Updated weights for policy 0, policy_version 434156 (0.0009) [2023-12-27 08:38:44,323][20347] Updated weights for policy 0, policy_version 434166 (0.0007) [2023-12-27 08:38:44,333][20291] KL-divergence is very high: 282.9991 [2023-12-27 08:38:44,351][20291] KL-divergence is very high: 219.2417 [2023-12-27 08:38:44,382][20347] Updated weights for policy 0, policy_version 434176 (0.0008) [2023-12-27 08:38:44,827][20381] Updated weights for policy 1, policy_version 433813 (0.0008) [2023-12-27 08:38:44,889][20381] Updated weights for policy 1, policy_version 433823 (0.0007) [2023-12-27 08:38:44,955][20381] Updated weights for policy 1, policy_version 433833 (0.0006) [2023-12-27 08:38:45,070][20347] Updated weights for policy 0, policy_version 434186 (0.0007) [2023-12-27 08:38:45,128][20347] Updated weights for policy 0, policy_version 434196 (0.0007) [2023-12-27 08:38:45,184][20347] Updated weights for policy 0, policy_version 434206 (0.0008) [2023-12-27 08:38:45,550][20381] Updated weights for policy 1, policy_version 433843 (0.0006) [2023-12-27 08:38:45,614][20381] Updated weights for policy 1, policy_version 433853 (0.0005) [2023-12-27 08:38:45,669][20381] Updated weights for policy 1, policy_version 433863 (0.0007) [2023-12-27 08:38:45,799][20347] Updated weights for policy 0, policy_version 434216 (0.0008) [2023-12-27 08:38:45,855][20347] Updated weights for policy 0, policy_version 434226 (0.0007) [2023-12-27 08:38:45,916][20347] Updated weights for policy 0, policy_version 434236 (0.0007) [2023-12-27 08:38:46,324][20381] Updated weights for policy 1, policy_version 433873 (0.0006) [2023-12-27 08:38:46,384][20381] Updated weights for policy 1, policy_version 433883 (0.0007) [2023-12-27 08:38:46,434][20381] Updated weights for policy 1, policy_version 433893 (0.0008) [2023-12-27 08:38:46,485][20381] Updated weights for policy 1, policy_version 433903 (0.0008) [2023-12-27 08:38:46,600][20347] Updated weights for policy 0, policy_version 434246 (0.0008) [2023-12-27 08:38:46,659][20347] Updated weights for policy 0, policy_version 434256 (0.0008) [2023-12-27 08:38:46,714][20347] Updated weights for policy 0, policy_version 434266 (0.0008) [2023-12-27 08:38:47,157][20381] Updated weights for policy 1, policy_version 433913 (0.0007) [2023-12-27 08:38:47,215][20381] Updated weights for policy 1, policy_version 433923 (0.0008) [2023-12-27 08:38:47,266][20381] Updated weights for policy 1, policy_version 433933 (0.0008) [2023-12-27 08:38:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 19410.9). Total num frames: 222298112. Throughput: 0: 10320.7, 1: 10199.9. Samples: 222308328. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:38:47,363][19170] Avg episode reward: [(0, '922.960'), (1, '923.000')] [2023-12-27 08:38:47,380][20347] Updated weights for policy 0, policy_version 434276 (0.0007) [2023-12-27 08:38:47,424][20347] Updated weights for policy 0, policy_version 434286 (0.0007) [2023-12-27 08:38:47,479][20347] Updated weights for policy 0, policy_version 434296 (0.0008) [2023-12-27 08:38:47,976][20381] Updated weights for policy 1, policy_version 433943 (0.0006) [2023-12-27 08:38:48,038][20381] Updated weights for policy 1, policy_version 433953 (0.0007) [2023-12-27 08:38:48,090][20381] Updated weights for policy 1, policy_version 433963 (0.0008) [2023-12-27 08:38:48,174][20347] Updated weights for policy 0, policy_version 434306 (0.0008) [2023-12-27 08:38:48,223][20347] Updated weights for policy 0, policy_version 434316 (0.0007) [2023-12-27 08:38:48,267][20347] Updated weights for policy 0, policy_version 434326 (0.0007) [2023-12-27 08:38:48,319][20347] Updated weights for policy 0, policy_version 434336 (0.0007) [2023-12-27 08:38:48,763][20381] Updated weights for policy 1, policy_version 433973 (0.0008) [2023-12-27 08:38:48,824][20381] Updated weights for policy 1, policy_version 433983 (0.0008) [2023-12-27 08:38:48,891][20381] Updated weights for policy 1, policy_version 433993 (0.0007) [2023-12-27 08:38:49,021][20291] KL-divergence is very high: 124.8992 [2023-12-27 08:38:49,067][20347] Updated weights for policy 0, policy_version 434346 (0.0006) [2023-12-27 08:38:49,068][20291] KL-divergence is very high: 105.2799 [2023-12-27 08:38:49,122][20347] Updated weights for policy 0, policy_version 434356 (0.0007) [2023-12-27 08:38:49,179][20347] Updated weights for policy 0, policy_version 434366 (0.0007) [2023-12-27 08:38:49,586][20381] Updated weights for policy 1, policy_version 434003 (0.0007) [2023-12-27 08:38:49,649][20381] Updated weights for policy 1, policy_version 434013 (0.0008) [2023-12-27 08:38:49,702][20381] Updated weights for policy 1, policy_version 434023 (0.0008) [2023-12-27 08:38:49,860][20347] Updated weights for policy 0, policy_version 434376 (0.0008) [2023-12-27 08:38:49,921][20347] Updated weights for policy 0, policy_version 434386 (0.0008) [2023-12-27 08:38:49,972][20347] Updated weights for policy 0, policy_version 434396 (0.0006) [2023-12-27 08:38:50,450][20381] Updated weights for policy 1, policy_version 434033 (0.0008) [2023-12-27 08:38:50,503][20381] Updated weights for policy 1, policy_version 434043 (0.0008) [2023-12-27 08:38:50,563][20381] Updated weights for policy 1, policy_version 434053 (0.0008) [2023-12-27 08:38:50,617][20381] Updated weights for policy 1, policy_version 434063 (0.0008) [2023-12-27 08:38:50,655][20347] Updated weights for policy 0, policy_version 434406 (0.0007) [2023-12-27 08:38:50,707][20347] Updated weights for policy 0, policy_version 434416 (0.0008) [2023-12-27 08:38:50,756][20347] Updated weights for policy 0, policy_version 434426 (0.0008) [2023-12-27 08:38:51,325][20381] Updated weights for policy 1, policy_version 434073 (0.0007) [2023-12-27 08:38:51,389][20381] Updated weights for policy 1, policy_version 434083 (0.0007) [2023-12-27 08:38:51,448][20381] Updated weights for policy 1, policy_version 434093 (0.0008) [2023-12-27 08:38:51,470][20347] Updated weights for policy 0, policy_version 434436 (0.0008) [2023-12-27 08:38:51,526][20347] Updated weights for policy 0, policy_version 434446 (0.0007) [2023-12-27 08:38:51,579][20347] Updated weights for policy 0, policy_version 434456 (0.0009) [2023-12-27 08:38:52,089][20381] Updated weights for policy 1, policy_version 434103 (0.0007) [2023-12-27 08:38:52,144][20381] Updated weights for policy 1, policy_version 434113 (0.0006) [2023-12-27 08:38:52,196][20381] Updated weights for policy 1, policy_version 434123 (0.0005) [2023-12-27 08:38:52,305][20347] Updated weights for policy 0, policy_version 434466 (0.0007) [2023-12-27 08:38:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 19383.1). Total num frames: 222396416. Throughput: 0: 10289.2, 1: 10118.2. Samples: 222367672. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:38:52,363][20291] KL-divergence is very high: 103.2223 [2023-12-27 08:38:52,363][19170] Avg episode reward: [(0, '874.360'), (1, '952.080')] [2023-12-27 08:38:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000434128_111157248.pth... [2023-12-27 08:38:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000432912_110845952.pth [2023-12-27 08:38:52,377][20347] Updated weights for policy 0, policy_version 434476 (0.0008) [2023-12-27 08:38:52,432][20347] Updated weights for policy 0, policy_version 434486 (0.0005) [2023-12-27 08:38:52,491][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000434496_111247360.pth... [2023-12-27 08:38:52,493][20347] Updated weights for policy 0, policy_version 434496 (0.0007) [2023-12-27 08:38:52,494][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000433280_110936064.pth [2023-12-27 08:38:52,873][20381] Updated weights for policy 1, policy_version 434133 (0.0007) [2023-12-27 08:38:52,925][20381] Updated weights for policy 1, policy_version 434143 (0.0007) [2023-12-27 08:38:52,974][20381] Updated weights for policy 1, policy_version 434153 (0.0005) [2023-12-27 08:38:53,129][20347] Updated weights for policy 0, policy_version 434506 (0.0010) [2023-12-27 08:38:53,186][20347] Updated weights for policy 0, policy_version 434516 (0.0010) [2023-12-27 08:38:53,239][20347] Updated weights for policy 0, policy_version 434526 (0.0010) [2023-12-27 08:38:53,670][20381] Updated weights for policy 1, policy_version 434163 (0.0008) [2023-12-27 08:38:53,726][20381] Updated weights for policy 1, policy_version 434173 (0.0008) [2023-12-27 08:38:53,785][20381] Updated weights for policy 1, policy_version 434183 (0.0008) [2023-12-27 08:38:53,862][20347] Updated weights for policy 0, policy_version 434536 (0.0007) [2023-12-27 08:38:53,920][20347] Updated weights for policy 0, policy_version 434546 (0.0005) [2023-12-27 08:38:53,977][20347] Updated weights for policy 0, policy_version 434556 (0.0006) [2023-12-27 08:38:54,475][20381] Updated weights for policy 1, policy_version 434193 (0.0007) [2023-12-27 08:38:54,535][20381] Updated weights for policy 1, policy_version 434203 (0.0008) [2023-12-27 08:38:54,593][20381] Updated weights for policy 1, policy_version 434213 (0.0008) [2023-12-27 08:38:54,602][20347] Updated weights for policy 0, policy_version 434566 (0.0007) [2023-12-27 08:38:54,650][20381] Updated weights for policy 1, policy_version 434223 (0.0007) [2023-12-27 08:38:54,656][20347] Updated weights for policy 0, policy_version 434576 (0.0005) [2023-12-27 08:38:54,714][20347] Updated weights for policy 0, policy_version 434586 (0.0006) [2023-12-27 08:38:55,293][20347] Updated weights for policy 0, policy_version 434596 (0.0007) [2023-12-27 08:38:55,352][20347] Updated weights for policy 0, policy_version 434606 (0.0005) [2023-12-27 08:38:55,399][20381] Updated weights for policy 1, policy_version 434233 (0.0007) [2023-12-27 08:38:55,407][20347] Updated weights for policy 0, policy_version 434616 (0.0006) [2023-12-27 08:38:55,458][20381] Updated weights for policy 1, policy_version 434243 (0.0007) [2023-12-27 08:38:55,516][20381] Updated weights for policy 1, policy_version 434253 (0.0007) [2023-12-27 08:38:55,958][20347] Updated weights for policy 0, policy_version 434626 (0.0006) [2023-12-27 08:38:56,016][20347] Updated weights for policy 0, policy_version 434636 (0.0005) [2023-12-27 08:38:56,071][20347] Updated weights for policy 0, policy_version 434646 (0.0010) [2023-12-27 08:38:56,129][20347] Updated weights for policy 0, policy_version 434656 (0.0010) [2023-12-27 08:38:56,156][20381] Updated weights for policy 1, policy_version 434263 (0.0007) [2023-12-27 08:38:56,212][20381] Updated weights for policy 1, policy_version 434273 (0.0008) [2023-12-27 08:38:56,268][20381] Updated weights for policy 1, policy_version 434283 (0.0010) [2023-12-27 08:38:56,768][20347] Updated weights for policy 0, policy_version 434666 (0.0005) [2023-12-27 08:38:56,822][20347] Updated weights for policy 0, policy_version 434676 (0.0007) [2023-12-27 08:38:56,878][20347] Updated weights for policy 0, policy_version 434686 (0.0007) [2023-12-27 08:38:56,905][20381] Updated weights for policy 1, policy_version 434293 (0.0010) [2023-12-27 08:38:56,961][20381] Updated weights for policy 1, policy_version 434303 (0.0008) [2023-12-27 08:38:57,020][20381] Updated weights for policy 1, policy_version 434313 (0.0008) [2023-12-27 08:38:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 19383.1). Total num frames: 222502912. Throughput: 0: 10275.3, 1: 10158.3. Samples: 222492944. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:38:57,363][19170] Avg episode reward: [(0, '874.250'), (1, '980.900')] [2023-12-27 08:38:57,489][20347] Updated weights for policy 0, policy_version 434696 (0.0008) [2023-12-27 08:38:57,552][20347] Updated weights for policy 0, policy_version 434706 (0.0008) [2023-12-27 08:38:57,607][20347] Updated weights for policy 0, policy_version 434716 (0.0007) [2023-12-27 08:38:57,677][20381] Updated weights for policy 1, policy_version 434323 (0.0008) [2023-12-27 08:38:57,731][20381] Updated weights for policy 1, policy_version 434333 (0.0007) [2023-12-27 08:38:57,788][20381] Updated weights for policy 1, policy_version 434343 (0.0006) [2023-12-27 08:38:58,292][20347] Updated weights for policy 0, policy_version 434726 (0.0007) [2023-12-27 08:38:58,357][20347] Updated weights for policy 0, policy_version 434736 (0.0009) [2023-12-27 08:38:58,419][20347] Updated weights for policy 0, policy_version 434746 (0.0008) [2023-12-27 08:38:58,421][20381] Updated weights for policy 1, policy_version 434353 (0.0007) [2023-12-27 08:38:58,484][20381] Updated weights for policy 1, policy_version 434363 (0.0008) [2023-12-27 08:38:58,543][20381] Updated weights for policy 1, policy_version 434373 (0.0008) [2023-12-27 08:38:58,598][20381] Updated weights for policy 1, policy_version 434383 (0.0007) [2023-12-27 08:38:59,103][20347] Updated weights for policy 0, policy_version 434756 (0.0008) [2023-12-27 08:38:59,168][20347] Updated weights for policy 0, policy_version 434766 (0.0008) [2023-12-27 08:38:59,226][20347] Updated weights for policy 0, policy_version 434776 (0.0009) [2023-12-27 08:38:59,349][20381] Updated weights for policy 1, policy_version 434393 (0.0008) [2023-12-27 08:38:59,404][20381] Updated weights for policy 1, policy_version 434403 (0.0008) [2023-12-27 08:38:59,460][20381] Updated weights for policy 1, policy_version 434413 (0.0008) [2023-12-27 08:38:59,962][20347] Updated weights for policy 0, policy_version 434786 (0.0008) [2023-12-27 08:39:00,015][20347] Updated weights for policy 0, policy_version 434796 (0.0008) [2023-12-27 08:39:00,068][20347] Updated weights for policy 0, policy_version 434806 (0.0008) [2023-12-27 08:39:00,121][20347] Updated weights for policy 0, policy_version 434816 (0.0007) [2023-12-27 08:39:00,163][20381] Updated weights for policy 1, policy_version 434423 (0.0006) [2023-12-27 08:39:00,217][20381] Updated weights for policy 1, policy_version 434433 (0.0007) [2023-12-27 08:39:00,274][20381] Updated weights for policy 1, policy_version 434443 (0.0006) [2023-12-27 08:39:00,798][20347] Updated weights for policy 0, policy_version 434826 (0.0006) [2023-12-27 08:39:00,854][20347] Updated weights for policy 0, policy_version 434836 (0.0006) [2023-12-27 08:39:00,883][20381] Updated weights for policy 1, policy_version 434453 (0.0008) [2023-12-27 08:39:00,901][20347] Updated weights for policy 0, policy_version 434846 (0.0005) [2023-12-27 08:39:00,939][20381] Updated weights for policy 1, policy_version 434463 (0.0010) [2023-12-27 08:39:00,987][20381] Updated weights for policy 1, policy_version 434473 (0.0005) [2023-12-27 08:39:01,527][20347] Updated weights for policy 0, policy_version 434856 (0.0008) [2023-12-27 08:39:01,577][20347] Updated weights for policy 0, policy_version 434866 (0.0009) [2023-12-27 08:39:01,639][20347] Updated weights for policy 0, policy_version 434876 (0.0006) [2023-12-27 08:39:01,640][20381] Updated weights for policy 1, policy_version 434483 (0.0010) [2023-12-27 08:39:01,709][20381] Updated weights for policy 1, policy_version 434493 (0.0011) [2023-12-27 08:39:01,767][20381] Updated weights for policy 1, policy_version 434503 (0.0011) [2023-12-27 08:39:02,318][20347] Updated weights for policy 0, policy_version 434886 (0.0009) [2023-12-27 08:39:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 19383.1). Total num frames: 222601216. Throughput: 0: 10278.1, 1: 10238.9. Samples: 222616940. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:02,363][19170] Avg episode reward: [(0, '824.880'), (1, '980.700')] [2023-12-27 08:39:02,382][20347] Updated weights for policy 0, policy_version 434896 (0.0007) [2023-12-27 08:39:02,423][20381] Updated weights for policy 1, policy_version 434513 (0.0010) [2023-12-27 08:39:02,430][20347] Updated weights for policy 0, policy_version 434906 (0.0008) [2023-12-27 08:39:02,478][20381] Updated weights for policy 1, policy_version 434523 (0.0011) [2023-12-27 08:39:02,536][20381] Updated weights for policy 1, policy_version 434533 (0.0010) [2023-12-27 08:39:02,596][20381] Updated weights for policy 1, policy_version 434543 (0.0010) [2023-12-27 08:39:03,127][20347] Updated weights for policy 0, policy_version 434916 (0.0009) [2023-12-27 08:39:03,193][20347] Updated weights for policy 0, policy_version 434926 (0.0008) [2023-12-27 08:39:03,256][20347] Updated weights for policy 0, policy_version 434936 (0.0008) [2023-12-27 08:39:03,277][20381] Updated weights for policy 1, policy_version 434553 (0.0011) [2023-12-27 08:39:03,322][20381] Updated weights for policy 1, policy_version 434563 (0.0010) [2023-12-27 08:39:03,367][20381] Updated weights for policy 1, policy_version 434573 (0.0010) [2023-12-27 08:39:03,899][20347] Updated weights for policy 0, policy_version 434946 (0.0009) [2023-12-27 08:39:03,955][20347] Updated weights for policy 0, policy_version 434956 (0.0006) [2023-12-27 08:39:04,002][20347] Updated weights for policy 0, policy_version 434966 (0.0008) [2023-12-27 08:39:04,027][20291] KL-divergence is very high: 130.2018 [2023-12-27 08:39:04,053][20347] Updated weights for policy 0, policy_version 434976 (0.0007) [2023-12-27 08:39:04,063][20381] Updated weights for policy 1, policy_version 434583 (0.0010) [2023-12-27 08:39:04,130][20381] Updated weights for policy 1, policy_version 434593 (0.0010) [2023-12-27 08:39:04,179][20381] Updated weights for policy 1, policy_version 434603 (0.0010) [2023-12-27 08:39:04,721][20347] Updated weights for policy 0, policy_version 434986 (0.0010) [2023-12-27 08:39:04,783][20347] Updated weights for policy 0, policy_version 434996 (0.0010) [2023-12-27 08:39:04,843][20347] Updated weights for policy 0, policy_version 435006 (0.0010) [2023-12-27 08:39:04,844][20381] Updated weights for policy 1, policy_version 434613 (0.0010) [2023-12-27 08:39:04,899][20381] Updated weights for policy 1, policy_version 434623 (0.0010) [2023-12-27 08:39:04,964][20381] Updated weights for policy 1, policy_version 434633 (0.0010) [2023-12-27 08:39:05,468][20347] Updated weights for policy 0, policy_version 435016 (0.0008) [2023-12-27 08:39:05,531][20347] Updated weights for policy 0, policy_version 435026 (0.0010) [2023-12-27 08:39:05,544][20381] Updated weights for policy 1, policy_version 434643 (0.0009) [2023-12-27 08:39:05,582][20347] Updated weights for policy 0, policy_version 435036 (0.0010) [2023-12-27 08:39:05,596][20381] Updated weights for policy 1, policy_version 434653 (0.0005) [2023-12-27 08:39:05,650][20381] Updated weights for policy 1, policy_version 434663 (0.0005) [2023-12-27 08:39:06,171][20347] Updated weights for policy 0, policy_version 435046 (0.0009) [2023-12-27 08:39:06,229][20347] Updated weights for policy 0, policy_version 435056 (0.0007) [2023-12-27 08:39:06,234][20381] Updated weights for policy 1, policy_version 434673 (0.0006) [2023-12-27 08:39:06,288][20347] Updated weights for policy 0, policy_version 435066 (0.0006) [2023-12-27 08:39:06,290][20381] Updated weights for policy 1, policy_version 434683 (0.0010) [2023-12-27 08:39:06,343][20381] Updated weights for policy 1, policy_version 434693 (0.0010) [2023-12-27 08:39:06,401][20381] Updated weights for policy 1, policy_version 434703 (0.0010) [2023-12-27 08:39:07,014][20347] Updated weights for policy 0, policy_version 435076 (0.0008) [2023-12-27 08:39:07,036][20381] Updated weights for policy 1, policy_version 434713 (0.0006) [2023-12-27 08:39:07,040][20291] KL-divergence is very high: 111.0075 [2023-12-27 08:39:07,066][20347] Updated weights for policy 0, policy_version 435086 (0.0010) [2023-12-27 08:39:07,082][20291] KL-divergence is very high: 120.2589 [2023-12-27 08:39:07,091][20381] Updated weights for policy 1, policy_version 434723 (0.0007) [2023-12-27 08:39:07,121][20347] Updated weights for policy 0, policy_version 435096 (0.0010) [2023-12-27 08:39:07,125][20291] KL-divergence is very high: 133.5874 [2023-12-27 08:39:07,143][20381] Updated weights for policy 1, policy_version 434733 (0.0007) [2023-12-27 08:39:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 19383.1). Total num frames: 222715904. Throughput: 0: 10293.0, 1: 10244.7. Samples: 222678956. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:07,363][19170] Avg episode reward: [(0, '198.160'), (1, '903.020')] [2023-12-27 08:39:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000434736_111312896.pth... [2023-12-27 08:39:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000435104_111403008.pth... [2023-12-27 08:39:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000433520_111001600.pth [2023-12-27 08:39:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000433888_111091712.pth [2023-12-27 08:39:07,732][20347] Updated weights for policy 0, policy_version 435106 (0.0010) [2023-12-27 08:39:07,758][20381] Updated weights for policy 1, policy_version 434743 (0.0009) [2023-12-27 08:39:07,794][20347] Updated weights for policy 0, policy_version 435116 (0.0005) [2023-12-27 08:39:07,803][20381] Updated weights for policy 1, policy_version 434753 (0.0010) [2023-12-27 08:39:07,853][20347] Updated weights for policy 0, policy_version 435126 (0.0005) [2023-12-27 08:39:07,857][20381] Updated weights for policy 1, policy_version 434763 (0.0010) [2023-12-27 08:39:07,902][20347] Updated weights for policy 0, policy_version 435136 (0.0005) [2023-12-27 08:39:08,463][20347] Updated weights for policy 0, policy_version 435146 (0.0007) [2023-12-27 08:39:08,525][20347] Updated weights for policy 0, policy_version 435156 (0.0005) [2023-12-27 08:39:08,528][20381] Updated weights for policy 1, policy_version 434773 (0.0008) [2023-12-27 08:39:08,579][20347] Updated weights for policy 0, policy_version 435166 (0.0006) [2023-12-27 08:39:08,582][20381] Updated weights for policy 1, policy_version 434783 (0.0009) [2023-12-27 08:39:08,635][20381] Updated weights for policy 1, policy_version 434793 (0.0010) [2023-12-27 08:39:09,253][20381] Updated weights for policy 1, policy_version 434803 (0.0007) [2023-12-27 08:39:09,273][20347] Updated weights for policy 0, policy_version 435176 (0.0006) [2023-12-27 08:39:09,311][20381] Updated weights for policy 1, policy_version 434813 (0.0008) [2023-12-27 08:39:09,333][20347] Updated weights for policy 0, policy_version 435186 (0.0006) [2023-12-27 08:39:09,377][20381] Updated weights for policy 1, policy_version 434823 (0.0008) [2023-12-27 08:39:09,399][20347] Updated weights for policy 0, policy_version 435196 (0.0007) [2023-12-27 08:39:10,083][20381] Updated weights for policy 1, policy_version 434833 (0.0010) [2023-12-27 08:39:10,121][20347] Updated weights for policy 0, policy_version 435206 (0.0008) [2023-12-27 08:39:10,143][20381] Updated weights for policy 1, policy_version 434843 (0.0009) [2023-12-27 08:39:10,187][20347] Updated weights for policy 0, policy_version 435216 (0.0007) [2023-12-27 08:39:10,204][20381] Updated weights for policy 1, policy_version 434853 (0.0007) [2023-12-27 08:39:10,253][20347] Updated weights for policy 0, policy_version 435226 (0.0006) [2023-12-27 08:39:10,272][20381] Updated weights for policy 1, policy_version 434863 (0.0006) [2023-12-27 08:39:10,903][20381] Updated weights for policy 1, policy_version 434873 (0.0006) [2023-12-27 08:39:10,940][20347] Updated weights for policy 0, policy_version 435236 (0.0008) [2023-12-27 08:39:10,959][20381] Updated weights for policy 1, policy_version 434883 (0.0005) [2023-12-27 08:39:10,998][20347] Updated weights for policy 0, policy_version 435246 (0.0007) [2023-12-27 08:39:11,017][20381] Updated weights for policy 1, policy_version 434893 (0.0007) [2023-12-27 08:39:11,065][20347] Updated weights for policy 0, policy_version 435256 (0.0007) [2023-12-27 08:39:11,696][20381] Updated weights for policy 1, policy_version 434903 (0.0011) [2023-12-27 08:39:11,724][20292] KL-divergence is very high: 196.0168 [2023-12-27 08:39:11,764][20381] Updated weights for policy 1, policy_version 434913 (0.0011) [2023-12-27 08:39:11,808][20347] Updated weights for policy 0, policy_version 435266 (0.0008) [2023-12-27 08:39:11,830][20381] Updated weights for policy 1, policy_version 434923 (0.0011) [2023-12-27 08:39:11,866][20347] Updated weights for policy 0, policy_version 435276 (0.0006) [2023-12-27 08:39:11,919][20347] Updated weights for policy 0, policy_version 435286 (0.0005) [2023-12-27 08:39:11,975][20347] Updated weights for policy 0, policy_version 435296 (0.0007) [2023-12-27 08:39:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 19383.1). Total num frames: 222814208. Throughput: 0: 10294.9, 1: 10374.0. Samples: 222806588. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:12,363][19170] Avg episode reward: [(0, '210.860'), (1, '845.280')] [2023-12-27 08:39:12,447][20381] Updated weights for policy 1, policy_version 434933 (0.0011) [2023-12-27 08:39:12,503][20381] Updated weights for policy 1, policy_version 434943 (0.0010) [2023-12-27 08:39:12,558][20381] Updated weights for policy 1, policy_version 434953 (0.0010) [2023-12-27 08:39:12,670][20347] Updated weights for policy 0, policy_version 435306 (0.0007) [2023-12-27 08:39:12,732][20347] Updated weights for policy 0, policy_version 435316 (0.0007) [2023-12-27 08:39:12,780][20347] Updated weights for policy 0, policy_version 435326 (0.0008) [2023-12-27 08:39:13,177][20381] Updated weights for policy 1, policy_version 434963 (0.0009) [2023-12-27 08:39:13,227][20381] Updated weights for policy 1, policy_version 434973 (0.0008) [2023-12-27 08:39:13,285][20381] Updated weights for policy 1, policy_version 434983 (0.0008) [2023-12-27 08:39:13,470][20347] Updated weights for policy 0, policy_version 435336 (0.0006) [2023-12-27 08:39:13,521][20347] Updated weights for policy 0, policy_version 435346 (0.0005) [2023-12-27 08:39:13,574][20347] Updated weights for policy 0, policy_version 435356 (0.0008) [2023-12-27 08:39:13,967][20381] Updated weights for policy 1, policy_version 434993 (0.0007) [2023-12-27 08:39:14,035][20381] Updated weights for policy 1, policy_version 435003 (0.0007) [2023-12-27 08:39:14,098][20381] Updated weights for policy 1, policy_version 435013 (0.0008) [2023-12-27 08:39:14,153][20381] Updated weights for policy 1, policy_version 435023 (0.0007) [2023-12-27 08:39:14,213][20347] Updated weights for policy 0, policy_version 435366 (0.0010) [2023-12-27 08:39:14,267][20347] Updated weights for policy 0, policy_version 435376 (0.0010) [2023-12-27 08:39:14,317][20347] Updated weights for policy 0, policy_version 435386 (0.0010) [2023-12-27 08:39:14,827][20381] Updated weights for policy 1, policy_version 435033 (0.0007) [2023-12-27 08:39:14,840][20292] KL-divergence is very high: 229.7462 [2023-12-27 08:39:14,892][20381] Updated weights for policy 1, policy_version 435043 (0.0006) [2023-12-27 08:39:14,906][20292] KL-divergence is very high: 211.1610 [2023-12-27 08:39:14,958][20381] Updated weights for policy 1, policy_version 435053 (0.0007) [2023-12-27 08:39:14,991][20347] Updated weights for policy 0, policy_version 435396 (0.0009) [2023-12-27 08:39:15,056][20347] Updated weights for policy 0, policy_version 435406 (0.0007) [2023-12-27 08:39:15,120][20347] Updated weights for policy 0, policy_version 435416 (0.0006) [2023-12-27 08:39:15,625][20381] Updated weights for policy 1, policy_version 435063 (0.0007) [2023-12-27 08:39:15,676][20381] Updated weights for policy 1, policy_version 435073 (0.0005) [2023-12-27 08:39:15,736][20381] Updated weights for policy 1, policy_version 435083 (0.0007) [2023-12-27 08:39:15,770][20347] Updated weights for policy 0, policy_version 435426 (0.0007) [2023-12-27 08:39:15,833][20347] Updated weights for policy 0, policy_version 435436 (0.0008) [2023-12-27 08:39:15,899][20347] Updated weights for policy 0, policy_version 435446 (0.0008) [2023-12-27 08:39:15,960][20347] Updated weights for policy 0, policy_version 435456 (0.0010) [2023-12-27 08:39:16,400][20381] Updated weights for policy 1, policy_version 435093 (0.0007) [2023-12-27 08:39:16,450][20381] Updated weights for policy 1, policy_version 435103 (0.0007) [2023-12-27 08:39:16,495][20381] Updated weights for policy 1, policy_version 435113 (0.0008) [2023-12-27 08:39:16,584][20347] Updated weights for policy 0, policy_version 435466 (0.0006) [2023-12-27 08:39:16,639][20347] Updated weights for policy 0, policy_version 435476 (0.0006) [2023-12-27 08:39:16,689][20347] Updated weights for policy 0, policy_version 435486 (0.0009) [2023-12-27 08:39:17,137][20381] Updated weights for policy 1, policy_version 435123 (0.0006) [2023-12-27 08:39:17,196][20381] Updated weights for policy 1, policy_version 435133 (0.0006) [2023-12-27 08:39:17,251][20381] Updated weights for policy 1, policy_version 435143 (0.0005) [2023-12-27 08:39:17,259][20347] Updated weights for policy 0, policy_version 435496 (0.0008) [2023-12-27 08:39:17,311][20347] Updated weights for policy 0, policy_version 435506 (0.0009) [2023-12-27 08:39:17,358][20347] Updated weights for policy 0, policy_version 435516 (0.0009) [2023-12-27 08:39:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 19383.1). Total num frames: 222920704. Throughput: 0: 10364.9, 1: 10353.2. Samples: 222931944. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:17,363][19170] Avg episode reward: [(0, '126.070'), (1, '727.150')] [2023-12-27 08:39:17,915][20381] Updated weights for policy 1, policy_version 435153 (0.0006) [2023-12-27 08:39:17,977][20381] Updated weights for policy 1, policy_version 435163 (0.0008) [2023-12-27 08:39:18,039][20381] Updated weights for policy 1, policy_version 435173 (0.0007) [2023-12-27 08:39:18,050][20347] Updated weights for policy 0, policy_version 435526 (0.0007) [2023-12-27 08:39:18,097][20381] Updated weights for policy 1, policy_version 435183 (0.0006) [2023-12-27 08:39:18,114][20347] Updated weights for policy 0, policy_version 435536 (0.0006) [2023-12-27 08:39:18,160][20347] Updated weights for policy 0, policy_version 435546 (0.0006) [2023-12-27 08:39:18,751][20381] Updated weights for policy 1, policy_version 435193 (0.0007) [2023-12-27 08:39:18,810][20381] Updated weights for policy 1, policy_version 435203 (0.0008) [2023-12-27 08:39:18,828][20347] Updated weights for policy 0, policy_version 435556 (0.0007) [2023-12-27 08:39:18,873][20381] Updated weights for policy 1, policy_version 435213 (0.0008) [2023-12-27 08:39:18,891][20347] Updated weights for policy 0, policy_version 435566 (0.0006) [2023-12-27 08:39:18,948][20347] Updated weights for policy 0, policy_version 435576 (0.0007) [2023-12-27 08:39:19,568][20347] Updated weights for policy 0, policy_version 435586 (0.0007) [2023-12-27 08:39:19,616][20381] Updated weights for policy 1, policy_version 435223 (0.0008) [2023-12-27 08:39:19,629][20347] Updated weights for policy 0, policy_version 435596 (0.0005) [2023-12-27 08:39:19,679][20381] Updated weights for policy 1, policy_version 435233 (0.0008) [2023-12-27 08:39:19,689][20347] Updated weights for policy 0, policy_version 435606 (0.0006) [2023-12-27 08:39:19,738][20381] Updated weights for policy 1, policy_version 435243 (0.0008) [2023-12-27 08:39:19,739][20347] Updated weights for policy 0, policy_version 435616 (0.0006) [2023-12-27 08:39:20,345][20347] Updated weights for policy 0, policy_version 435626 (0.0007) [2023-12-27 08:39:20,404][20347] Updated weights for policy 0, policy_version 435636 (0.0008) [2023-12-27 08:39:20,464][20347] Updated weights for policy 0, policy_version 435646 (0.0008) [2023-12-27 08:39:20,491][20381] Updated weights for policy 1, policy_version 435253 (0.0008) [2023-12-27 08:39:20,555][20381] Updated weights for policy 1, policy_version 435263 (0.0008) [2023-12-27 08:39:20,610][20381] Updated weights for policy 1, policy_version 435273 (0.0008) [2023-12-27 08:39:21,136][20347] Updated weights for policy 0, policy_version 435656 (0.0007) [2023-12-27 08:39:21,200][20347] Updated weights for policy 0, policy_version 435666 (0.0007) [2023-12-27 08:39:21,260][20347] Updated weights for policy 0, policy_version 435676 (0.0007) [2023-12-27 08:39:21,367][20381] Updated weights for policy 1, policy_version 435283 (0.0008) [2023-12-27 08:39:21,424][20381] Updated weights for policy 1, policy_version 435293 (0.0007) [2023-12-27 08:39:21,483][20381] Updated weights for policy 1, policy_version 435303 (0.0008) [2023-12-27 08:39:21,922][20347] Updated weights for policy 0, policy_version 435686 (0.0008) [2023-12-27 08:39:21,982][20347] Updated weights for policy 0, policy_version 435696 (0.0008) [2023-12-27 08:39:22,038][20347] Updated weights for policy 0, policy_version 435706 (0.0009) [2023-12-27 08:39:22,192][20381] Updated weights for policy 1, policy_version 435313 (0.0008) [2023-12-27 08:39:22,248][20381] Updated weights for policy 1, policy_version 435323 (0.0008) [2023-12-27 08:39:22,307][20381] Updated weights for policy 1, policy_version 435333 (0.0008) [2023-12-27 08:39:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 19383.1). Total num frames: 223019008. Throughput: 0: 10406.8, 1: 10292.1. Samples: 222992848. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:22,363][19170] Avg episode reward: [(0, '496.960'), (1, '720.390')] [2023-12-27 08:39:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000435712_111558656.pth... [2023-12-27 08:39:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000434496_111247360.pth [2023-12-27 08:39:22,377][20381] Updated weights for policy 1, policy_version 435343 (0.0008) [2023-12-27 08:39:22,379][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000435344_111468544.pth... [2023-12-27 08:39:22,384][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000434128_111157248.pth [2023-12-27 08:39:22,822][20347] Updated weights for policy 0, policy_version 435716 (0.0009) [2023-12-27 08:39:22,882][20347] Updated weights for policy 0, policy_version 435726 (0.0008) [2023-12-27 08:39:22,940][20347] Updated weights for policy 0, policy_version 435736 (0.0008) [2023-12-27 08:39:22,983][20381] Updated weights for policy 1, policy_version 435353 (0.0006) [2023-12-27 08:39:23,045][20381] Updated weights for policy 1, policy_version 435363 (0.0007) [2023-12-27 08:39:23,099][20381] Updated weights for policy 1, policy_version 435373 (0.0008) [2023-12-27 08:39:23,670][20347] Updated weights for policy 0, policy_version 435746 (0.0007) [2023-12-27 08:39:23,687][20381] Updated weights for policy 1, policy_version 435383 (0.0006) [2023-12-27 08:39:23,718][20347] Updated weights for policy 0, policy_version 435756 (0.0010) [2023-12-27 08:39:23,732][20381] Updated weights for policy 1, policy_version 435393 (0.0005) [2023-12-27 08:39:23,764][20347] Updated weights for policy 0, policy_version 435766 (0.0009) [2023-12-27 08:39:23,780][20381] Updated weights for policy 1, policy_version 435403 (0.0007) [2023-12-27 08:39:23,824][20347] Updated weights for policy 0, policy_version 435776 (0.0006) [2023-12-27 08:39:24,433][20381] Updated weights for policy 1, policy_version 435413 (0.0007) [2023-12-27 08:39:24,483][20347] Updated weights for policy 0, policy_version 435786 (0.0006) [2023-12-27 08:39:24,489][20381] Updated weights for policy 1, policy_version 435423 (0.0007) [2023-12-27 08:39:24,538][20347] Updated weights for policy 0, policy_version 435796 (0.0006) [2023-12-27 08:39:24,546][20381] Updated weights for policy 1, policy_version 435433 (0.0006) [2023-12-27 08:39:24,597][20347] Updated weights for policy 0, policy_version 435806 (0.0006) [2023-12-27 08:39:25,175][20381] Updated weights for policy 1, policy_version 435443 (0.0006) [2023-12-27 08:39:25,220][20347] Updated weights for policy 0, policy_version 435816 (0.0006) [2023-12-27 08:39:25,237][20381] Updated weights for policy 1, policy_version 435453 (0.0007) [2023-12-27 08:39:25,272][20347] Updated weights for policy 0, policy_version 435826 (0.0006) [2023-12-27 08:39:25,296][20381] Updated weights for policy 1, policy_version 435463 (0.0006) [2023-12-27 08:39:25,318][20347] Updated weights for policy 0, policy_version 435836 (0.0005) [2023-12-27 08:39:25,845][20381] Updated weights for policy 1, policy_version 435473 (0.0007) [2023-12-27 08:39:25,903][20381] Updated weights for policy 1, policy_version 435483 (0.0005) [2023-12-27 08:39:25,957][20381] Updated weights for policy 1, policy_version 435493 (0.0005) [2023-12-27 08:39:26,021][20381] Updated weights for policy 1, policy_version 435503 (0.0007) [2023-12-27 08:39:26,039][20347] Updated weights for policy 0, policy_version 435846 (0.0005) [2023-12-27 08:39:26,099][20347] Updated weights for policy 0, policy_version 435856 (0.0005) [2023-12-27 08:39:26,152][20347] Updated weights for policy 0, policy_version 435866 (0.0007) [2023-12-27 08:39:26,618][20381] Updated weights for policy 1, policy_version 435513 (0.0005) [2023-12-27 08:39:26,665][20381] Updated weights for policy 1, policy_version 435523 (0.0005) [2023-12-27 08:39:26,716][20381] Updated weights for policy 1, policy_version 435533 (0.0005) [2023-12-27 08:39:26,845][20347] Updated weights for policy 0, policy_version 435876 (0.0008) [2023-12-27 08:39:26,899][20347] Updated weights for policy 0, policy_version 435886 (0.0007) [2023-12-27 08:39:26,953][20347] Updated weights for policy 0, policy_version 435896 (0.0007) [2023-12-27 08:39:27,323][20381] Updated weights for policy 1, policy_version 435543 (0.0010) [2023-12-27 08:39:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 19383.1). Total num frames: 223125504. Throughput: 0: 10371.3, 1: 10373.1. Samples: 223117216. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:27,363][19170] Avg episode reward: [(0, '407.850'), (1, '817.150')] [2023-12-27 08:39:27,371][20381] Updated weights for policy 1, policy_version 435553 (0.0010) [2023-12-27 08:39:27,419][20381] Updated weights for policy 1, policy_version 435563 (0.0010) [2023-12-27 08:39:27,597][20347] Updated weights for policy 0, policy_version 435906 (0.0008) [2023-12-27 08:39:27,655][20347] Updated weights for policy 0, policy_version 435916 (0.0007) [2023-12-27 08:39:27,712][20347] Updated weights for policy 0, policy_version 435926 (0.0007) [2023-12-27 08:39:27,769][20347] Updated weights for policy 0, policy_version 435936 (0.0008) [2023-12-27 08:39:28,004][20381] Updated weights for policy 1, policy_version 435573 (0.0008) [2023-12-27 08:39:28,057][20381] Updated weights for policy 1, policy_version 435583 (0.0005) [2023-12-27 08:39:28,114][20381] Updated weights for policy 1, policy_version 435593 (0.0006) [2023-12-27 08:39:28,507][20347] Updated weights for policy 0, policy_version 435946 (0.0007) [2023-12-27 08:39:28,556][20347] Updated weights for policy 0, policy_version 435956 (0.0008) [2023-12-27 08:39:28,603][20347] Updated weights for policy 0, policy_version 435966 (0.0007) [2023-12-27 08:39:28,772][20381] Updated weights for policy 1, policy_version 435603 (0.0010) [2023-12-27 08:39:28,817][20381] Updated weights for policy 1, policy_version 435613 (0.0010) [2023-12-27 08:39:28,863][20381] Updated weights for policy 1, policy_version 435623 (0.0010) [2023-12-27 08:39:29,264][20347] Updated weights for policy 0, policy_version 435976 (0.0008) [2023-12-27 08:39:29,316][20347] Updated weights for policy 0, policy_version 435986 (0.0008) [2023-12-27 08:39:29,374][20347] Updated weights for policy 0, policy_version 435996 (0.0008) [2023-12-27 08:39:29,577][20381] Updated weights for policy 1, policy_version 435633 (0.0010) [2023-12-27 08:39:29,628][20381] Updated weights for policy 1, policy_version 435643 (0.0010) [2023-12-27 08:39:29,683][20381] Updated weights for policy 1, policy_version 435653 (0.0010) [2023-12-27 08:39:29,742][20381] Updated weights for policy 1, policy_version 435663 (0.0010) [2023-12-27 08:39:30,031][20347] Updated weights for policy 0, policy_version 436006 (0.0008) [2023-12-27 08:39:30,086][20347] Updated weights for policy 0, policy_version 436016 (0.0007) [2023-12-27 08:39:30,143][20347] Updated weights for policy 0, policy_version 436027 (0.0009) [2023-12-27 08:39:30,381][20381] Updated weights for policy 1, policy_version 435673 (0.0008) [2023-12-27 08:39:30,440][20381] Updated weights for policy 1, policy_version 435683 (0.0007) [2023-12-27 08:39:30,489][20381] Updated weights for policy 1, policy_version 435693 (0.0008) [2023-12-27 08:39:30,813][20347] Updated weights for policy 0, policy_version 436037 (0.0006) [2023-12-27 08:39:30,869][20347] Updated weights for policy 0, policy_version 436047 (0.0005) [2023-12-27 08:39:30,918][20347] Updated weights for policy 0, policy_version 436057 (0.0005) [2023-12-27 08:39:31,113][20381] Updated weights for policy 1, policy_version 435703 (0.0007) [2023-12-27 08:39:31,183][20381] Updated weights for policy 1, policy_version 435713 (0.0007) [2023-12-27 08:39:31,233][20381] Updated weights for policy 1, policy_version 435723 (0.0008) [2023-12-27 08:39:31,600][20347] Updated weights for policy 0, policy_version 436067 (0.0006) [2023-12-27 08:39:31,659][20347] Updated weights for policy 0, policy_version 436077 (0.0007) [2023-12-27 08:39:31,719][20347] Updated weights for policy 0, policy_version 436087 (0.0007) [2023-12-27 08:39:31,899][20381] Updated weights for policy 1, policy_version 435733 (0.0008) [2023-12-27 08:39:31,960][20381] Updated weights for policy 1, policy_version 435743 (0.0007) [2023-12-27 08:39:32,019][20381] Updated weights for policy 1, policy_version 435753 (0.0008) [2023-12-27 08:39:32,293][20347] Updated weights for policy 0, policy_version 436097 (0.0007) [2023-12-27 08:39:32,361][20347] Updated weights for policy 0, policy_version 436107 (0.0008) [2023-12-27 08:39:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 19494.2). Total num frames: 223232000. Throughput: 0: 10378.5, 1: 10411.9. Samples: 223243896. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:32,363][19170] Avg episode reward: [(0, '438.170'), (1, '865.210')] [2023-12-27 08:39:32,423][20347] Updated weights for policy 0, policy_version 436117 (0.0009) [2023-12-27 08:39:32,482][20347] Updated weights for policy 0, policy_version 436127 (0.0006) [2023-12-27 08:39:32,705][20381] Updated weights for policy 1, policy_version 435763 (0.0008) [2023-12-27 08:39:32,765][20381] Updated weights for policy 1, policy_version 435773 (0.0007) [2023-12-27 08:39:32,828][20381] Updated weights for policy 1, policy_version 435783 (0.0007) [2023-12-27 08:39:33,121][20347] Updated weights for policy 0, policy_version 436137 (0.0008) [2023-12-27 08:39:33,176][20347] Updated weights for policy 0, policy_version 436147 (0.0007) [2023-12-27 08:39:33,232][20347] Updated weights for policy 0, policy_version 436157 (0.0007) [2023-12-27 08:39:33,487][20381] Updated weights for policy 1, policy_version 435793 (0.0008) [2023-12-27 08:39:33,547][20381] Updated weights for policy 1, policy_version 435803 (0.0011) [2023-12-27 08:39:33,606][20381] Updated weights for policy 1, policy_version 435813 (0.0011) [2023-12-27 08:39:33,661][20381] Updated weights for policy 1, policy_version 435823 (0.0008) [2023-12-27 08:39:33,887][20347] Updated weights for policy 0, policy_version 436167 (0.0006) [2023-12-27 08:39:33,940][20347] Updated weights for policy 0, policy_version 436177 (0.0007) [2023-12-27 08:39:33,991][20347] Updated weights for policy 0, policy_version 436187 (0.0009) [2023-12-27 08:39:34,297][20381] Updated weights for policy 1, policy_version 435833 (0.0008) [2023-12-27 08:39:34,347][20381] Updated weights for policy 1, policy_version 435843 (0.0008) [2023-12-27 08:39:34,402][20381] Updated weights for policy 1, policy_version 435853 (0.0008) [2023-12-27 08:39:34,613][20347] Updated weights for policy 0, policy_version 436197 (0.0007) [2023-12-27 08:39:34,663][20347] Updated weights for policy 0, policy_version 436207 (0.0008) [2023-12-27 08:39:34,733][20347] Updated weights for policy 0, policy_version 436217 (0.0007) [2023-12-27 08:39:35,082][20381] Updated weights for policy 1, policy_version 435863 (0.0006) [2023-12-27 08:39:35,132][20381] Updated weights for policy 1, policy_version 435873 (0.0005) [2023-12-27 08:39:35,196][20381] Updated weights for policy 1, policy_version 435883 (0.0005) [2023-12-27 08:39:35,431][20347] Updated weights for policy 0, policy_version 436227 (0.0008) [2023-12-27 08:39:35,482][20347] Updated weights for policy 0, policy_version 436237 (0.0008) [2023-12-27 08:39:35,545][20347] Updated weights for policy 0, policy_version 436247 (0.0009) [2023-12-27 08:39:35,780][20381] Updated weights for policy 1, policy_version 435893 (0.0006) [2023-12-27 08:39:35,833][20381] Updated weights for policy 1, policy_version 435903 (0.0007) [2023-12-27 08:39:35,895][20381] Updated weights for policy 1, policy_version 435913 (0.0008) [2023-12-27 08:39:36,250][20347] Updated weights for policy 0, policy_version 436257 (0.0008) [2023-12-27 08:39:36,315][20347] Updated weights for policy 0, policy_version 436267 (0.0008) [2023-12-27 08:39:36,379][20347] Updated weights for policy 0, policy_version 436277 (0.0008) [2023-12-27 08:39:36,442][20347] Updated weights for policy 0, policy_version 436287 (0.0007) [2023-12-27 08:39:36,610][20381] Updated weights for policy 1, policy_version 435923 (0.0008) [2023-12-27 08:39:36,667][20381] Updated weights for policy 1, policy_version 435933 (0.0008) [2023-12-27 08:39:36,729][20381] Updated weights for policy 1, policy_version 435943 (0.0008) [2023-12-27 08:39:37,141][20347] Updated weights for policy 0, policy_version 436297 (0.0008) [2023-12-27 08:39:37,189][20347] Updated weights for policy 0, policy_version 436307 (0.0006) [2023-12-27 08:39:37,241][20347] Updated weights for policy 0, policy_version 436317 (0.0007) [2023-12-27 08:39:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 19577.5). Total num frames: 223338496. Throughput: 0: 10422.8, 1: 10457.8. Samples: 223307296. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:37,363][19170] Avg episode reward: [(0, '613.490'), (1, '895.030')] [2023-12-27 08:39:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000435952_111624192.pth... [2023-12-27 08:39:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000436320_111714304.pth... [2023-12-27 08:39:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000434736_111312896.pth [2023-12-27 08:39:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000435104_111403008.pth [2023-12-27 08:39:37,409][20381] Updated weights for policy 1, policy_version 435953 (0.0008) [2023-12-27 08:39:37,464][20381] Updated weights for policy 1, policy_version 435963 (0.0008) [2023-12-27 08:39:37,521][20381] Updated weights for policy 1, policy_version 435973 (0.0008) [2023-12-27 08:39:37,573][20381] Updated weights for policy 1, policy_version 435983 (0.0008) [2023-12-27 08:39:37,936][20347] Updated weights for policy 0, policy_version 436327 (0.0007) [2023-12-27 08:39:37,985][20347] Updated weights for policy 0, policy_version 436337 (0.0006) [2023-12-27 08:39:38,038][20347] Updated weights for policy 0, policy_version 436347 (0.0005) [2023-12-27 08:39:38,210][20381] Updated weights for policy 1, policy_version 435993 (0.0006) [2023-12-27 08:39:38,258][20381] Updated weights for policy 1, policy_version 436003 (0.0010) [2023-12-27 08:39:38,312][20381] Updated weights for policy 1, policy_version 436013 (0.0010) [2023-12-27 08:39:38,719][20347] Updated weights for policy 0, policy_version 436357 (0.0007) [2023-12-27 08:39:38,770][20347] Updated weights for policy 0, policy_version 436367 (0.0007) [2023-12-27 08:39:38,826][20347] Updated weights for policy 0, policy_version 436377 (0.0008) [2023-12-27 08:39:38,970][20381] Updated weights for policy 1, policy_version 436023 (0.0010) [2023-12-27 08:39:39,022][20381] Updated weights for policy 1, policy_version 436033 (0.0010) [2023-12-27 08:39:39,067][20381] Updated weights for policy 1, policy_version 436043 (0.0010) [2023-12-27 08:39:39,550][20347] Updated weights for policy 0, policy_version 436387 (0.0007) [2023-12-27 08:39:39,611][20347] Updated weights for policy 0, policy_version 436397 (0.0009) [2023-12-27 08:39:39,669][20347] Updated weights for policy 0, policy_version 436407 (0.0008) [2023-12-27 08:39:39,718][20381] Updated weights for policy 1, policy_version 436053 (0.0008) [2023-12-27 08:39:39,778][20381] Updated weights for policy 1, policy_version 436063 (0.0006) [2023-12-27 08:39:39,845][20381] Updated weights for policy 1, policy_version 436073 (0.0007) [2023-12-27 08:39:40,431][20347] Updated weights for policy 0, policy_version 436417 (0.0008) [2023-12-27 08:39:40,494][20347] Updated weights for policy 0, policy_version 436427 (0.0008) [2023-12-27 08:39:40,501][20381] Updated weights for policy 1, policy_version 436083 (0.0008) [2023-12-27 08:39:40,550][20347] Updated weights for policy 0, policy_version 436437 (0.0007) [2023-12-27 08:39:40,551][20381] Updated weights for policy 1, policy_version 436093 (0.0011) [2023-12-27 08:39:40,597][20381] Updated weights for policy 1, policy_version 436103 (0.0010) [2023-12-27 08:39:40,609][20347] Updated weights for policy 0, policy_version 436447 (0.0007) [2023-12-27 08:39:41,207][20381] Updated weights for policy 1, policy_version 436113 (0.0010) [2023-12-27 08:39:41,266][20381] Updated weights for policy 1, policy_version 436123 (0.0008) [2023-12-27 08:39:41,327][20381] Updated weights for policy 1, policy_version 436133 (0.0007) [2023-12-27 08:39:41,352][20347] Updated weights for policy 0, policy_version 436457 (0.0008) [2023-12-27 08:39:41,393][20381] Updated weights for policy 1, policy_version 436143 (0.0007) [2023-12-27 08:39:41,412][20347] Updated weights for policy 0, policy_version 436467 (0.0008) [2023-12-27 08:39:41,471][20347] Updated weights for policy 0, policy_version 436477 (0.0008) [2023-12-27 08:39:42,116][20381] Updated weights for policy 1, policy_version 436153 (0.0006) [2023-12-27 08:39:42,172][20381] Updated weights for policy 1, policy_version 436163 (0.0005) [2023-12-27 08:39:42,182][20347] Updated weights for policy 0, policy_version 436487 (0.0007) [2023-12-27 08:39:42,229][20381] Updated weights for policy 1, policy_version 436173 (0.0005) [2023-12-27 08:39:42,237][20347] Updated weights for policy 0, policy_version 436497 (0.0007) [2023-12-27 08:39:42,294][20347] Updated weights for policy 0, policy_version 436507 (0.0008) [2023-12-27 08:39:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 19660.8). Total num frames: 223444992. Throughput: 0: 10293.5, 1: 10524.2. Samples: 223429736. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:42,363][19170] Avg episode reward: [(0, '625.360'), (1, '875.550')] [2023-12-27 08:39:42,878][20381] Updated weights for policy 1, policy_version 436183 (0.0007) [2023-12-27 08:39:42,935][20381] Updated weights for policy 1, policy_version 436193 (0.0008) [2023-12-27 08:39:42,991][20381] Updated weights for policy 1, policy_version 436203 (0.0007) [2023-12-27 08:39:42,996][20347] Updated weights for policy 0, policy_version 436517 (0.0008) [2023-12-27 08:39:43,056][20347] Updated weights for policy 0, policy_version 436527 (0.0008) [2023-12-27 08:39:43,115][20347] Updated weights for policy 0, policy_version 436537 (0.0007) [2023-12-27 08:39:43,662][20381] Updated weights for policy 1, policy_version 436213 (0.0006) [2023-12-27 08:39:43,719][20381] Updated weights for policy 1, policy_version 436223 (0.0005) [2023-12-27 08:39:43,772][20381] Updated weights for policy 1, policy_version 436233 (0.0005) [2023-12-27 08:39:43,816][20347] Updated weights for policy 0, policy_version 436547 (0.0007) [2023-12-27 08:39:43,879][20347] Updated weights for policy 0, policy_version 436557 (0.0008) [2023-12-27 08:39:43,935][20347] Updated weights for policy 0, policy_version 436567 (0.0008) [2023-12-27 08:39:44,345][20381] Updated weights for policy 1, policy_version 436243 (0.0006) [2023-12-27 08:39:44,406][20381] Updated weights for policy 1, policy_version 436253 (0.0008) [2023-12-27 08:39:44,462][20381] Updated weights for policy 1, policy_version 436263 (0.0007) [2023-12-27 08:39:44,645][20347] Updated weights for policy 0, policy_version 436577 (0.0008) [2023-12-27 08:39:44,706][20347] Updated weights for policy 0, policy_version 436587 (0.0007) [2023-12-27 08:39:44,770][20347] Updated weights for policy 0, policy_version 436597 (0.0008) [2023-12-27 08:39:44,829][20347] Updated weights for policy 0, policy_version 436607 (0.0007) [2023-12-27 08:39:45,050][20381] Updated weights for policy 1, policy_version 436273 (0.0006) [2023-12-27 08:39:45,104][20381] Updated weights for policy 1, policy_version 436283 (0.0008) [2023-12-27 08:39:45,160][20381] Updated weights for policy 1, policy_version 436293 (0.0007) [2023-12-27 08:39:45,221][20381] Updated weights for policy 1, policy_version 436303 (0.0008) [2023-12-27 08:39:45,556][20347] Updated weights for policy 0, policy_version 436617 (0.0007) [2023-12-27 08:39:45,612][20347] Updated weights for policy 0, policy_version 436627 (0.0008) [2023-12-27 08:39:45,668][20347] Updated weights for policy 0, policy_version 436637 (0.0008) [2023-12-27 08:39:45,924][20381] Updated weights for policy 1, policy_version 436313 (0.0008) [2023-12-27 08:39:45,977][20381] Updated weights for policy 1, policy_version 436323 (0.0007) [2023-12-27 08:39:46,032][20381] Updated weights for policy 1, policy_version 436333 (0.0008) [2023-12-27 08:39:46,370][20347] Updated weights for policy 0, policy_version 436647 (0.0008) [2023-12-27 08:39:46,430][20347] Updated weights for policy 0, policy_version 436657 (0.0007) [2023-12-27 08:39:46,488][20347] Updated weights for policy 0, policy_version 436667 (0.0007) [2023-12-27 08:39:46,704][20381] Updated weights for policy 1, policy_version 436343 (0.0006) [2023-12-27 08:39:46,759][20381] Updated weights for policy 1, policy_version 436353 (0.0005) [2023-12-27 08:39:46,827][20381] Updated weights for policy 1, policy_version 436363 (0.0008) [2023-12-27 08:39:47,187][20347] Updated weights for policy 0, policy_version 436677 (0.0006) [2023-12-27 08:39:47,237][20347] Updated weights for policy 0, policy_version 436687 (0.0005) [2023-12-27 08:39:47,293][20347] Updated weights for policy 0, policy_version 436697 (0.0005) [2023-12-27 08:39:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 19716.4). Total num frames: 223543296. Throughput: 0: 10225.0, 1: 10542.7. Samples: 223551484. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:47,363][19170] Avg episode reward: [(0, '596.790'), (1, '905.170')] [2023-12-27 08:39:47,496][20381] Updated weights for policy 1, policy_version 436373 (0.0009) [2023-12-27 08:39:47,540][20381] Updated weights for policy 1, policy_version 436383 (0.0009) [2023-12-27 08:39:47,587][20381] Updated weights for policy 1, policy_version 436393 (0.0010) [2023-12-27 08:39:47,867][20347] Updated weights for policy 0, policy_version 436707 (0.0008) [2023-12-27 08:39:47,929][20347] Updated weights for policy 0, policy_version 436717 (0.0006) [2023-12-27 08:39:47,983][20347] Updated weights for policy 0, policy_version 436727 (0.0005) [2023-12-27 08:39:48,316][20381] Updated weights for policy 1, policy_version 436403 (0.0008) [2023-12-27 08:39:48,375][20381] Updated weights for policy 1, policy_version 436413 (0.0008) [2023-12-27 08:39:48,431][20381] Updated weights for policy 1, policy_version 436423 (0.0007) [2023-12-27 08:39:48,628][20347] Updated weights for policy 0, policy_version 436737 (0.0006) [2023-12-27 08:39:48,693][20347] Updated weights for policy 0, policy_version 436747 (0.0007) [2023-12-27 08:39:48,744][20347] Updated weights for policy 0, policy_version 436757 (0.0007) [2023-12-27 08:39:48,798][20347] Updated weights for policy 0, policy_version 436767 (0.0007) [2023-12-27 08:39:49,167][20381] Updated weights for policy 1, policy_version 436433 (0.0008) [2023-12-27 08:39:49,232][20381] Updated weights for policy 1, policy_version 436443 (0.0008) [2023-12-27 08:39:49,295][20381] Updated weights for policy 1, policy_version 436453 (0.0007) [2023-12-27 08:39:49,368][20381] Updated weights for policy 1, policy_version 436463 (0.0007) [2023-12-27 08:39:49,482][20347] Updated weights for policy 0, policy_version 436777 (0.0008) [2023-12-27 08:39:49,541][20347] Updated weights for policy 0, policy_version 436787 (0.0007) [2023-12-27 08:39:49,601][20347] Updated weights for policy 0, policy_version 436797 (0.0006) [2023-12-27 08:39:50,058][20381] Updated weights for policy 1, policy_version 436473 (0.0007) [2023-12-27 08:39:50,115][20381] Updated weights for policy 1, policy_version 436483 (0.0008) [2023-12-27 08:39:50,169][20381] Updated weights for policy 1, policy_version 436493 (0.0008) [2023-12-27 08:39:50,297][20347] Updated weights for policy 0, policy_version 436807 (0.0007) [2023-12-27 08:39:50,347][20347] Updated weights for policy 0, policy_version 436817 (0.0008) [2023-12-27 08:39:50,399][20347] Updated weights for policy 0, policy_version 436827 (0.0007) [2023-12-27 08:39:50,898][20381] Updated weights for policy 1, policy_version 436503 (0.0008) [2023-12-27 08:39:50,958][20381] Updated weights for policy 1, policy_version 436513 (0.0008) [2023-12-27 08:39:51,011][20381] Updated weights for policy 1, policy_version 436523 (0.0007) [2023-12-27 08:39:51,130][20347] Updated weights for policy 0, policy_version 436837 (0.0007) [2023-12-27 08:39:51,190][20347] Updated weights for policy 0, policy_version 436847 (0.0008) [2023-12-27 08:39:51,258][20347] Updated weights for policy 0, policy_version 436857 (0.0008) [2023-12-27 08:39:51,731][20381] Updated weights for policy 1, policy_version 436533 (0.0008) [2023-12-27 08:39:51,789][20381] Updated weights for policy 1, policy_version 436543 (0.0010) [2023-12-27 08:39:51,847][20381] Updated weights for policy 1, policy_version 436553 (0.0007) [2023-12-27 08:39:52,018][20347] Updated weights for policy 0, policy_version 436867 (0.0008) [2023-12-27 08:39:52,083][20347] Updated weights for policy 0, policy_version 436877 (0.0005) [2023-12-27 08:39:52,138][20347] Updated weights for policy 0, policy_version 436887 (0.0008) [2023-12-27 08:39:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 19744.1). Total num frames: 223641600. Throughput: 0: 10235.9, 1: 10495.3. Samples: 223611864. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:52,363][19170] Avg episode reward: [(0, '710.260'), (1, '885.650')] [2023-12-27 08:39:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000436560_111779840.pth... [2023-12-27 08:39:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000436896_111861760.pth... [2023-12-27 08:39:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000435712_111558656.pth [2023-12-27 08:39:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000435344_111468544.pth [2023-12-27 08:39:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000436896_111861760.pth [2023-12-27 08:39:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000436560_111779840.pth [2023-12-27 08:39:52,570][20381] Updated weights for policy 1, policy_version 436563 (0.0007) [2023-12-27 08:39:52,621][20381] Updated weights for policy 1, policy_version 436573 (0.0007) [2023-12-27 08:39:52,677][20381] Updated weights for policy 1, policy_version 436583 (0.0008) [2023-12-27 08:39:52,840][20347] Updated weights for policy 0, policy_version 436897 (0.0008) [2023-12-27 08:39:52,893][20347] Updated weights for policy 0, policy_version 436907 (0.0006) [2023-12-27 08:39:52,945][20347] Updated weights for policy 0, policy_version 436917 (0.0008) [2023-12-27 08:39:52,993][20347] Updated weights for policy 0, policy_version 436927 (0.0007) [2023-12-27 08:39:53,388][20381] Updated weights for policy 1, policy_version 436593 (0.0008) [2023-12-27 08:39:53,446][20381] Updated weights for policy 1, policy_version 436603 (0.0011) [2023-12-27 08:39:53,495][20381] Updated weights for policy 1, policy_version 436613 (0.0010) [2023-12-27 08:39:53,552][20381] Updated weights for policy 1, policy_version 436623 (0.0010) [2023-12-27 08:39:53,704][20347] Updated weights for policy 0, policy_version 436937 (0.0007) [2023-12-27 08:39:53,752][20347] Updated weights for policy 0, policy_version 436947 (0.0007) [2023-12-27 08:39:53,809][20347] Updated weights for policy 0, policy_version 436957 (0.0007) [2023-12-27 08:39:54,240][20381] Updated weights for policy 1, policy_version 436633 (0.0010) [2023-12-27 08:39:54,292][20381] Updated weights for policy 1, policy_version 436643 (0.0010) [2023-12-27 08:39:54,350][20381] Updated weights for policy 1, policy_version 436653 (0.0010) [2023-12-27 08:39:54,528][20347] Updated weights for policy 0, policy_version 436967 (0.0008) [2023-12-27 08:39:54,589][20347] Updated weights for policy 0, policy_version 436977 (0.0007) [2023-12-27 08:39:54,643][20347] Updated weights for policy 0, policy_version 436987 (0.0007) [2023-12-27 08:39:55,044][20381] Updated weights for policy 1, policy_version 436663 (0.0011) [2023-12-27 08:39:55,102][20381] Updated weights for policy 1, policy_version 436673 (0.0010) [2023-12-27 08:39:55,158][20381] Updated weights for policy 1, policy_version 436683 (0.0011) [2023-12-27 08:39:55,337][20347] Updated weights for policy 0, policy_version 436997 (0.0007) [2023-12-27 08:39:55,385][20347] Updated weights for policy 0, policy_version 437007 (0.0007) [2023-12-27 08:39:55,432][20347] Updated weights for policy 0, policy_version 437017 (0.0007) [2023-12-27 08:39:55,840][20381] Updated weights for policy 1, policy_version 436693 (0.0009) [2023-12-27 08:39:55,888][20381] Updated weights for policy 1, policy_version 436703 (0.0007) [2023-12-27 08:39:55,939][20381] Updated weights for policy 1, policy_version 436713 (0.0007) [2023-12-27 08:39:56,143][20347] Updated weights for policy 0, policy_version 437027 (0.0007) [2023-12-27 08:39:56,198][20347] Updated weights for policy 0, policy_version 437037 (0.0008) [2023-12-27 08:39:56,254][20347] Updated weights for policy 0, policy_version 437047 (0.0008) [2023-12-27 08:39:56,638][20381] Updated weights for policy 1, policy_version 436723 (0.0008) [2023-12-27 08:39:56,695][20381] Updated weights for policy 1, policy_version 436733 (0.0010) [2023-12-27 08:39:56,752][20381] Updated weights for policy 1, policy_version 436743 (0.0010) [2023-12-27 08:39:56,865][20347] Updated weights for policy 0, policy_version 437057 (0.0007) [2023-12-27 08:39:56,921][20347] Updated weights for policy 0, policy_version 437067 (0.0007) [2023-12-27 08:39:56,969][20347] Updated weights for policy 0, policy_version 437077 (0.0007) [2023-12-27 08:39:57,012][20347] Updated weights for policy 0, policy_version 437087 (0.0007) [2023-12-27 08:39:57,363][19170] Fps is (10 sec: 19660.0, 60 sec: 20616.5, 300 sec: 19799.6). Total num frames: 223739904. Throughput: 0: 10151.3, 1: 10385.7. Samples: 223730760. Policy #0 lag: (min: 0.0, avg: 29.1, max: 32.0) [2023-12-27 08:39:57,364][19170] Avg episode reward: [(0, '748.660'), (1, '865.860')] [2023-12-27 08:39:57,395][20381] Updated weights for policy 1, policy_version 436753 (0.0010) [2023-12-27 08:39:57,458][20381] Updated weights for policy 1, policy_version 436763 (0.0007) [2023-12-27 08:39:57,519][20381] Updated weights for policy 1, policy_version 436773 (0.0008) [2023-12-27 08:39:57,583][20381] Updated weights for policy 1, policy_version 436783 (0.0008) [2023-12-27 08:39:57,748][20347] Updated weights for policy 0, policy_version 437097 (0.0008) [2023-12-27 08:39:57,802][20347] Updated weights for policy 0, policy_version 437107 (0.0007) [2023-12-27 08:39:57,861][20347] Updated weights for policy 0, policy_version 437117 (0.0008) [2023-12-27 08:39:58,114][20381] Updated weights for policy 1, policy_version 436793 (0.0005) [2023-12-27 08:39:58,167][20381] Updated weights for policy 1, policy_version 436803 (0.0006) [2023-12-27 08:39:58,230][20381] Updated weights for policy 1, policy_version 436813 (0.0008) [2023-12-27 08:39:58,587][20347] Updated weights for policy 0, policy_version 437127 (0.0007) [2023-12-27 08:39:58,648][20347] Updated weights for policy 0, policy_version 437137 (0.0006) [2023-12-27 08:39:58,715][20347] Updated weights for policy 0, policy_version 437147 (0.0008) [2023-12-27 08:39:58,984][20381] Updated weights for policy 1, policy_version 436823 (0.0008) [2023-12-27 08:39:59,046][20381] Updated weights for policy 1, policy_version 436833 (0.0008) [2023-12-27 08:39:59,101][20381] Updated weights for policy 1, policy_version 436843 (0.0008) [2023-12-27 08:39:59,431][20347] Updated weights for policy 0, policy_version 437157 (0.0006) [2023-12-27 08:39:59,483][20347] Updated weights for policy 0, policy_version 437167 (0.0005) [2023-12-27 08:39:59,534][20347] Updated weights for policy 0, policy_version 437177 (0.0005) [2023-12-27 08:39:59,817][20381] Updated weights for policy 1, policy_version 436853 (0.0007) [2023-12-27 08:39:59,878][20381] Updated weights for policy 1, policy_version 436863 (0.0008) [2023-12-27 08:39:59,944][20381] Updated weights for policy 1, policy_version 436873 (0.0009) [2023-12-27 08:40:00,136][20347] Updated weights for policy 0, policy_version 437187 (0.0006) [2023-12-27 08:40:00,186][20347] Updated weights for policy 0, policy_version 437197 (0.0009) [2023-12-27 08:40:00,230][20347] Updated weights for policy 0, policy_version 437207 (0.0008) [2023-12-27 08:40:00,568][20381] Updated weights for policy 1, policy_version 436883 (0.0008) [2023-12-27 08:40:00,619][20381] Updated weights for policy 1, policy_version 436893 (0.0008) [2023-12-27 08:40:00,663][20381] Updated weights for policy 1, policy_version 436903 (0.0007) [2023-12-27 08:40:00,943][20347] Updated weights for policy 0, policy_version 437217 (0.0007) [2023-12-27 08:40:00,997][20347] Updated weights for policy 0, policy_version 437227 (0.0007) [2023-12-27 08:40:01,057][20347] Updated weights for policy 0, policy_version 437237 (0.0008) [2023-12-27 08:40:01,106][20347] Updated weights for policy 0, policy_version 437247 (0.0007) [2023-12-27 08:40:01,361][20381] Updated weights for policy 1, policy_version 436913 (0.0008) [2023-12-27 08:40:01,414][20381] Updated weights for policy 1, policy_version 436923 (0.0009) [2023-12-27 08:40:01,475][20381] Updated weights for policy 1, policy_version 436933 (0.0009) [2023-12-27 08:40:01,526][20381] Updated weights for policy 1, policy_version 436943 (0.0010) [2023-12-27 08:40:01,871][20347] Updated weights for policy 0, policy_version 437257 (0.0008) [2023-12-27 08:40:01,934][20347] Updated weights for policy 0, policy_version 437267 (0.0009) [2023-12-27 08:40:01,991][20347] Updated weights for policy 0, policy_version 437277 (0.0007) [2023-12-27 08:40:02,214][20381] Updated weights for policy 1, policy_version 436953 (0.0010) [2023-12-27 08:40:02,275][20381] Updated weights for policy 1, policy_version 436963 (0.0011) [2023-12-27 08:40:02,344][20381] Updated weights for policy 1, policy_version 436973 (0.0010) [2023-12-27 08:40:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 19883.0). Total num frames: 223846400. Throughput: 0: 10102.1, 1: 10361.0. Samples: 223852784. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:02,363][19170] Avg episode reward: [(0, '797.300'), (1, '913.520')] [2023-12-27 08:40:02,597][20347] Updated weights for policy 0, policy_version 437287 (0.0009) [2023-12-27 08:40:02,649][20347] Updated weights for policy 0, policy_version 437297 (0.0010) [2023-12-27 08:40:02,701][20347] Updated weights for policy 0, policy_version 437307 (0.0010) [2023-12-27 08:40:02,962][20381] Updated weights for policy 1, policy_version 436983 (0.0011) [2023-12-27 08:40:03,012][20381] Updated weights for policy 1, policy_version 436993 (0.0010) [2023-12-27 08:40:03,067][20381] Updated weights for policy 1, policy_version 437003 (0.0005) [2023-12-27 08:40:03,338][20347] Updated weights for policy 0, policy_version 437317 (0.0010) [2023-12-27 08:40:03,395][20347] Updated weights for policy 0, policy_version 437327 (0.0010) [2023-12-27 08:40:03,449][20347] Updated weights for policy 0, policy_version 437337 (0.0006) [2023-12-27 08:40:03,705][20381] Updated weights for policy 1, policy_version 437013 (0.0010) [2023-12-27 08:40:03,753][20381] Updated weights for policy 1, policy_version 437023 (0.0009) [2023-12-27 08:40:03,820][20381] Updated weights for policy 1, policy_version 437033 (0.0009) [2023-12-27 08:40:04,091][20347] Updated weights for policy 0, policy_version 437347 (0.0007) [2023-12-27 08:40:04,162][20347] Updated weights for policy 0, policy_version 437357 (0.0006) [2023-12-27 08:40:04,221][20347] Updated weights for policy 0, policy_version 437367 (0.0007) [2023-12-27 08:40:04,442][20381] Updated weights for policy 1, policy_version 437043 (0.0009) [2023-12-27 08:40:04,500][20381] Updated weights for policy 1, policy_version 437053 (0.0009) [2023-12-27 08:40:04,545][20381] Updated weights for policy 1, policy_version 437063 (0.0008) [2023-12-27 08:40:04,854][20347] Updated weights for policy 0, policy_version 437377 (0.0008) [2023-12-27 08:40:04,915][20347] Updated weights for policy 0, policy_version 437387 (0.0007) [2023-12-27 08:40:04,972][20347] Updated weights for policy 0, policy_version 437397 (0.0007) [2023-12-27 08:40:05,030][20347] Updated weights for policy 0, policy_version 437407 (0.0005) [2023-12-27 08:40:05,275][20381] Updated weights for policy 1, policy_version 437073 (0.0007) [2023-12-27 08:40:05,323][20381] Updated weights for policy 1, policy_version 437083 (0.0008) [2023-12-27 08:40:05,372][20381] Updated weights for policy 1, policy_version 437093 (0.0008) [2023-12-27 08:40:05,425][20381] Updated weights for policy 1, policy_version 437103 (0.0008) [2023-12-27 08:40:05,632][20347] Updated weights for policy 0, policy_version 437417 (0.0007) [2023-12-27 08:40:05,686][20347] Updated weights for policy 0, policy_version 437427 (0.0007) [2023-12-27 08:40:05,734][20347] Updated weights for policy 0, policy_version 437437 (0.0008) [2023-12-27 08:40:06,118][20381] Updated weights for policy 1, policy_version 437113 (0.0007) [2023-12-27 08:40:06,180][20381] Updated weights for policy 1, policy_version 437123 (0.0008) [2023-12-27 08:40:06,226][20381] Updated weights for policy 1, policy_version 437133 (0.0008) [2023-12-27 08:40:06,400][20347] Updated weights for policy 0, policy_version 437447 (0.0008) [2023-12-27 08:40:06,456][20347] Updated weights for policy 0, policy_version 437457 (0.0008) [2023-12-27 08:40:06,512][20347] Updated weights for policy 0, policy_version 437467 (0.0007) [2023-12-27 08:40:06,965][20381] Updated weights for policy 1, policy_version 437143 (0.0010) [2023-12-27 08:40:07,012][20381] Updated weights for policy 1, policy_version 437153 (0.0009) [2023-12-27 08:40:07,063][20381] Updated weights for policy 1, policy_version 437163 (0.0006) [2023-12-27 08:40:07,094][20347] Updated weights for policy 0, policy_version 437477 (0.0007) [2023-12-27 08:40:07,146][20347] Updated weights for policy 0, policy_version 437487 (0.0005) [2023-12-27 08:40:07,197][20347] Updated weights for policy 0, policy_version 437497 (0.0005) [2023-12-27 08:40:07,363][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 19966.3). Total num frames: 223952896. Throughput: 0: 10114.3, 1: 10420.7. Samples: 223916924. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:07,363][19170] Avg episode reward: [(0, '865.150'), (1, '933.240')] [2023-12-27 08:40:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000437168_111935488.pth... [2023-12-27 08:40:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000437504_112017408.pth... [2023-12-27 08:40:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000435952_111624192.pth [2023-12-27 08:40:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000436320_111714304.pth [2023-12-27 08:40:07,761][20347] Updated weights for policy 0, policy_version 437507 (0.0006) [2023-12-27 08:40:07,783][20291] KL-divergence is very high: 123.0662 [2023-12-27 08:40:07,789][20291] KL-divergence is very high: 161.8307 [2023-12-27 08:40:07,795][20291] KL-divergence is very high: 144.7178 [2023-12-27 08:40:07,801][20291] KL-divergence is very high: 170.4841 [2023-12-27 08:40:07,811][20381] Updated weights for policy 1, policy_version 437173 (0.0006) [2023-12-27 08:40:07,824][20347] Updated weights for policy 0, policy_version 437517 (0.0009) [2023-12-27 08:40:07,860][20381] Updated weights for policy 1, policy_version 437183 (0.0006) [2023-12-27 08:40:07,880][20347] Updated weights for policy 0, policy_version 437527 (0.0005) [2023-12-27 08:40:07,881][20291] KL-divergence is very high: 146.7921 [2023-12-27 08:40:07,913][20381] Updated weights for policy 1, policy_version 437193 (0.0005) [2023-12-27 08:40:08,484][20347] Updated weights for policy 0, policy_version 437537 (0.0006) [2023-12-27 08:40:08,536][20291] KL-divergence is very high: 123.2055 [2023-12-27 08:40:08,548][20347] Updated weights for policy 0, policy_version 437547 (0.0010) [2023-12-27 08:40:08,613][20347] Updated weights for policy 0, policy_version 437557 (0.0007) [2023-12-27 08:40:08,618][20381] Updated weights for policy 1, policy_version 437203 (0.0009) [2023-12-27 08:40:08,670][20381] Updated weights for policy 1, policy_version 437213 (0.0005) [2023-12-27 08:40:08,672][20347] Updated weights for policy 0, policy_version 437567 (0.0010) [2023-12-27 08:40:08,729][20381] Updated weights for policy 1, policy_version 437223 (0.0008) [2023-12-27 08:40:09,259][20347] Updated weights for policy 0, policy_version 437577 (0.0009) [2023-12-27 08:40:09,309][20347] Updated weights for policy 0, policy_version 437587 (0.0008) [2023-12-27 08:40:09,375][20347] Updated weights for policy 0, policy_version 437597 (0.0010) [2023-12-27 08:40:09,449][20381] Updated weights for policy 1, policy_version 437233 (0.0006) [2023-12-27 08:40:09,505][20381] Updated weights for policy 1, policy_version 437243 (0.0007) [2023-12-27 08:40:09,564][20381] Updated weights for policy 1, policy_version 437253 (0.0007) [2023-12-27 08:40:09,626][20381] Updated weights for policy 1, policy_version 437263 (0.0007) [2023-12-27 08:40:10,082][20347] Updated weights for policy 0, policy_version 437607 (0.0010) [2023-12-27 08:40:10,105][20291] KL-divergence is very high: 180.5004 [2023-12-27 08:40:10,140][20347] Updated weights for policy 0, policy_version 437617 (0.0010) [2023-12-27 08:40:10,200][20347] Updated weights for policy 0, policy_version 437627 (0.0010) [2023-12-27 08:40:10,368][20381] Updated weights for policy 1, policy_version 437273 (0.0007) [2023-12-27 08:40:10,430][20381] Updated weights for policy 1, policy_version 437283 (0.0008) [2023-12-27 08:40:10,494][20381] Updated weights for policy 1, policy_version 437293 (0.0008) [2023-12-27 08:40:10,875][20347] Updated weights for policy 0, policy_version 437637 (0.0007) [2023-12-27 08:40:10,937][20347] Updated weights for policy 0, policy_version 437647 (0.0007) [2023-12-27 08:40:10,992][20347] Updated weights for policy 0, policy_version 437657 (0.0010) [2023-12-27 08:40:11,166][20381] Updated weights for policy 1, policy_version 437303 (0.0008) [2023-12-27 08:40:11,219][20381] Updated weights for policy 1, policy_version 437313 (0.0007) [2023-12-27 08:40:11,281][20381] Updated weights for policy 1, policy_version 437323 (0.0008) [2023-12-27 08:40:11,677][20347] Updated weights for policy 0, policy_version 437667 (0.0010) [2023-12-27 08:40:11,740][20347] Updated weights for policy 0, policy_version 437677 (0.0008) [2023-12-27 08:40:11,807][20347] Updated weights for policy 0, policy_version 437687 (0.0008) [2023-12-27 08:40:11,996][20381] Updated weights for policy 1, policy_version 437333 (0.0006) [2023-12-27 08:40:12,056][20381] Updated weights for policy 1, policy_version 437343 (0.0006) [2023-12-27 08:40:12,119][20381] Updated weights for policy 1, policy_version 437353 (0.0007) [2023-12-27 08:40:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 19994.0). Total num frames: 224051200. Throughput: 0: 10236.3, 1: 10305.9. Samples: 224041612. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:12,363][19170] Avg episode reward: [(0, '856.360'), (1, '942.940')] [2023-12-27 08:40:12,442][20347] Updated weights for policy 0, policy_version 437697 (0.0008) [2023-12-27 08:40:12,497][20347] Updated weights for policy 0, policy_version 437707 (0.0011) [2023-12-27 08:40:12,557][20347] Updated weights for policy 0, policy_version 437717 (0.0008) [2023-12-27 08:40:12,621][20347] Updated weights for policy 0, policy_version 437727 (0.0006) [2023-12-27 08:40:12,750][20381] Updated weights for policy 1, policy_version 437363 (0.0007) [2023-12-27 08:40:12,807][20381] Updated weights for policy 1, policy_version 437373 (0.0007) [2023-12-27 08:40:12,861][20381] Updated weights for policy 1, policy_version 437383 (0.0007) [2023-12-27 08:40:13,283][20347] Updated weights for policy 0, policy_version 437737 (0.0007) [2023-12-27 08:40:13,335][20347] Updated weights for policy 0, policy_version 437747 (0.0006) [2023-12-27 08:40:13,386][20347] Updated weights for policy 0, policy_version 437757 (0.0007) [2023-12-27 08:40:13,558][20381] Updated weights for policy 1, policy_version 437393 (0.0007) [2023-12-27 08:40:13,616][20381] Updated weights for policy 1, policy_version 437403 (0.0011) [2023-12-27 08:40:13,683][20381] Updated weights for policy 1, policy_version 437413 (0.0011) [2023-12-27 08:40:13,739][20381] Updated weights for policy 1, policy_version 437423 (0.0011) [2023-12-27 08:40:13,995][20347] Updated weights for policy 0, policy_version 437767 (0.0006) [2023-12-27 08:40:14,046][20347] Updated weights for policy 0, policy_version 437777 (0.0006) [2023-12-27 08:40:14,103][20347] Updated weights for policy 0, policy_version 437787 (0.0009) [2023-12-27 08:40:14,387][20381] Updated weights for policy 1, policy_version 437433 (0.0006) [2023-12-27 08:40:14,439][20381] Updated weights for policy 1, policy_version 437443 (0.0005) [2023-12-27 08:40:14,493][20381] Updated weights for policy 1, policy_version 437453 (0.0005) [2023-12-27 08:40:14,794][20347] Updated weights for policy 0, policy_version 437797 (0.0008) [2023-12-27 08:40:14,843][20347] Updated weights for policy 0, policy_version 437807 (0.0010) [2023-12-27 08:40:14,905][20347] Updated weights for policy 0, policy_version 437817 (0.0010) [2023-12-27 08:40:15,060][20381] Updated weights for policy 1, policy_version 437463 (0.0009) [2023-12-27 08:40:15,116][20381] Updated weights for policy 1, policy_version 437473 (0.0011) [2023-12-27 08:40:15,175][20381] Updated weights for policy 1, policy_version 437483 (0.0011) [2023-12-27 08:40:15,490][20347] Updated weights for policy 0, policy_version 437827 (0.0009) [2023-12-27 08:40:15,541][20347] Updated weights for policy 0, policy_version 437837 (0.0010) [2023-12-27 08:40:15,597][20347] Updated weights for policy 0, policy_version 437847 (0.0010) [2023-12-27 08:40:15,867][20381] Updated weights for policy 1, policy_version 437493 (0.0011) [2023-12-27 08:40:15,915][20381] Updated weights for policy 1, policy_version 437503 (0.0010) [2023-12-27 08:40:15,961][20381] Updated weights for policy 1, policy_version 437513 (0.0010) [2023-12-27 08:40:16,182][20347] Updated weights for policy 0, policy_version 437857 (0.0010) [2023-12-27 08:40:16,233][20347] Updated weights for policy 0, policy_version 437867 (0.0010) [2023-12-27 08:40:16,287][20347] Updated weights for policy 0, policy_version 437877 (0.0010) [2023-12-27 08:40:16,341][20347] Updated weights for policy 0, policy_version 437887 (0.0010) [2023-12-27 08:40:16,667][20381] Updated weights for policy 1, policy_version 437523 (0.0011) [2023-12-27 08:40:16,716][20381] Updated weights for policy 1, policy_version 437533 (0.0010) [2023-12-27 08:40:16,775][20381] Updated weights for policy 1, policy_version 437543 (0.0010) [2023-12-27 08:40:16,934][20347] Updated weights for policy 0, policy_version 437897 (0.0006) [2023-12-27 08:40:16,992][20347] Updated weights for policy 0, policy_version 437907 (0.0005) [2023-12-27 08:40:17,044][20347] Updated weights for policy 0, policy_version 437917 (0.0005) [2023-12-27 08:40:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 19994.0). Total num frames: 224157696. Throughput: 0: 10303.0, 1: 10246.8. Samples: 224168640. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:17,363][19170] Avg episode reward: [(0, '800.070'), (1, '952.660')] [2023-12-27 08:40:17,495][20381] Updated weights for policy 1, policy_version 437553 (0.0008) [2023-12-27 08:40:17,550][20381] Updated weights for policy 1, policy_version 437563 (0.0010) [2023-12-27 08:40:17,606][20381] Updated weights for policy 1, policy_version 437573 (0.0011) [2023-12-27 08:40:17,662][20381] Updated weights for policy 1, policy_version 437583 (0.0011) [2023-12-27 08:40:17,676][20347] Updated weights for policy 0, policy_version 437927 (0.0009) [2023-12-27 08:40:17,733][20347] Updated weights for policy 0, policy_version 437937 (0.0010) [2023-12-27 08:40:17,793][20347] Updated weights for policy 0, policy_version 437947 (0.0010) [2023-12-27 08:40:18,353][20381] Updated weights for policy 1, policy_version 437593 (0.0009) [2023-12-27 08:40:18,415][20381] Updated weights for policy 1, policy_version 437603 (0.0010) [2023-12-27 08:40:18,422][20347] Updated weights for policy 0, policy_version 437957 (0.0009) [2023-12-27 08:40:18,471][20381] Updated weights for policy 1, policy_version 437613 (0.0005) [2023-12-27 08:40:18,490][20347] Updated weights for policy 0, policy_version 437967 (0.0009) [2023-12-27 08:40:18,554][20347] Updated weights for policy 0, policy_version 437977 (0.0010) [2023-12-27 08:40:19,099][20381] Updated weights for policy 1, policy_version 437623 (0.0008) [2023-12-27 08:40:19,154][20381] Updated weights for policy 1, policy_version 437633 (0.0007) [2023-12-27 08:40:19,211][20381] Updated weights for policy 1, policy_version 437643 (0.0008) [2023-12-27 08:40:19,221][20347] Updated weights for policy 0, policy_version 437987 (0.0009) [2023-12-27 08:40:19,283][20347] Updated weights for policy 0, policy_version 437997 (0.0008) [2023-12-27 08:40:19,345][20347] Updated weights for policy 0, policy_version 438007 (0.0008) [2023-12-27 08:40:19,927][20381] Updated weights for policy 1, policy_version 437653 (0.0008) [2023-12-27 08:40:19,991][20381] Updated weights for policy 1, policy_version 437663 (0.0006) [2023-12-27 08:40:20,055][20381] Updated weights for policy 1, policy_version 437673 (0.0007) [2023-12-27 08:40:20,082][20347] Updated weights for policy 0, policy_version 438017 (0.0009) [2023-12-27 08:40:20,148][20347] Updated weights for policy 0, policy_version 438027 (0.0007) [2023-12-27 08:40:20,208][20347] Updated weights for policy 0, policy_version 438037 (0.0007) [2023-12-27 08:40:20,268][20347] Updated weights for policy 0, policy_version 438047 (0.0008) [2023-12-27 08:40:20,725][20381] Updated weights for policy 1, policy_version 437683 (0.0006) [2023-12-27 08:40:20,787][20381] Updated weights for policy 1, policy_version 437693 (0.0008) [2023-12-27 08:40:20,838][20381] Updated weights for policy 1, policy_version 437703 (0.0008) [2023-12-27 08:40:20,952][20347] Updated weights for policy 0, policy_version 438057 (0.0008) [2023-12-27 08:40:21,005][20347] Updated weights for policy 0, policy_version 438067 (0.0007) [2023-12-27 08:40:21,069][20347] Updated weights for policy 0, policy_version 438077 (0.0007) [2023-12-27 08:40:21,591][20381] Updated weights for policy 1, policy_version 437713 (0.0008) [2023-12-27 08:40:21,657][20381] Updated weights for policy 1, policy_version 437723 (0.0008) [2023-12-27 08:40:21,719][20381] Updated weights for policy 1, policy_version 437733 (0.0007) [2023-12-27 08:40:21,778][20381] Updated weights for policy 1, policy_version 437743 (0.0006) [2023-12-27 08:40:21,828][20347] Updated weights for policy 0, policy_version 438087 (0.0007) [2023-12-27 08:40:21,875][20291] KL-divergence is very high: 153.6295 [2023-12-27 08:40:21,887][20347] Updated weights for policy 0, policy_version 438097 (0.0007) [2023-12-27 08:40:21,950][20347] Updated weights for policy 0, policy_version 438107 (0.0007) [2023-12-27 08:40:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 19994.0). Total num frames: 224256000. Throughput: 0: 10273.4, 1: 10235.2. Samples: 224230184. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:22,363][19170] Avg episode reward: [(0, '586.320'), (1, '914.730')] [2023-12-27 08:40:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000437744_112082944.pth... [2023-12-27 08:40:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000438112_112173056.pth... [2023-12-27 08:40:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000436560_111779840.pth [2023-12-27 08:40:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000436896_111861760.pth [2023-12-27 08:40:22,459][20381] Updated weights for policy 1, policy_version 437753 (0.0008) [2023-12-27 08:40:22,514][20381] Updated weights for policy 1, policy_version 437763 (0.0006) [2023-12-27 08:40:22,565][20381] Updated weights for policy 1, policy_version 437773 (0.0006) [2023-12-27 08:40:22,681][20347] Updated weights for policy 0, policy_version 438117 (0.0008) [2023-12-27 08:40:22,744][20347] Updated weights for policy 0, policy_version 438127 (0.0009) [2023-12-27 08:40:22,808][20347] Updated weights for policy 0, policy_version 438137 (0.0008) [2023-12-27 08:40:23,167][20381] Updated weights for policy 1, policy_version 437783 (0.0007) [2023-12-27 08:40:23,217][20381] Updated weights for policy 1, policy_version 437793 (0.0008) [2023-12-27 08:40:23,262][20381] Updated weights for policy 1, policy_version 437803 (0.0009) [2023-12-27 08:40:23,459][20347] Updated weights for policy 0, policy_version 438147 (0.0007) [2023-12-27 08:40:23,520][20347] Updated weights for policy 0, policy_version 438157 (0.0006) [2023-12-27 08:40:23,575][20347] Updated weights for policy 0, policy_version 438167 (0.0008) [2023-12-27 08:40:24,019][20381] Updated weights for policy 1, policy_version 437813 (0.0009) [2023-12-27 08:40:24,066][20381] Updated weights for policy 1, policy_version 437823 (0.0010) [2023-12-27 08:40:24,113][20381] Updated weights for policy 1, policy_version 437833 (0.0007) [2023-12-27 08:40:24,171][20347] Updated weights for policy 0, policy_version 438177 (0.0009) [2023-12-27 08:40:24,232][20347] Updated weights for policy 0, policy_version 438187 (0.0008) [2023-12-27 08:40:24,296][20347] Updated weights for policy 0, policy_version 438197 (0.0008) [2023-12-27 08:40:24,368][20347] Updated weights for policy 0, policy_version 438207 (0.0008) [2023-12-27 08:40:24,793][20381] Updated weights for policy 1, policy_version 437843 (0.0007) [2023-12-27 08:40:24,846][20381] Updated weights for policy 1, policy_version 437853 (0.0007) [2023-12-27 08:40:24,905][20381] Updated weights for policy 1, policy_version 437863 (0.0007) [2023-12-27 08:40:25,016][20347] Updated weights for policy 0, policy_version 438217 (0.0008) [2023-12-27 08:40:25,078][20347] Updated weights for policy 0, policy_version 438227 (0.0007) [2023-12-27 08:40:25,131][20347] Updated weights for policy 0, policy_version 438237 (0.0006) [2023-12-27 08:40:25,603][20381] Updated weights for policy 1, policy_version 437873 (0.0007) [2023-12-27 08:40:25,651][20381] Updated weights for policy 1, policy_version 437883 (0.0007) [2023-12-27 08:40:25,710][20381] Updated weights for policy 1, policy_version 437893 (0.0005) [2023-12-27 08:40:25,748][20347] Updated weights for policy 0, policy_version 438247 (0.0009) [2023-12-27 08:40:25,767][20381] Updated weights for policy 1, policy_version 437903 (0.0005) [2023-12-27 08:40:25,803][20347] Updated weights for policy 0, policy_version 438257 (0.0010) [2023-12-27 08:40:25,848][20291] KL-divergence is very high: 104.9098 [2023-12-27 08:40:25,853][20291] KL-divergence is very high: 142.3526 [2023-12-27 08:40:25,855][20347] Updated weights for policy 0, policy_version 438267 (0.0010) [2023-12-27 08:40:26,348][20381] Updated weights for policy 1, policy_version 437913 (0.0007) [2023-12-27 08:40:26,401][20381] Updated weights for policy 1, policy_version 437923 (0.0007) [2023-12-27 08:40:26,411][20292] KL-divergence is very high: 195.9470 [2023-12-27 08:40:26,460][20292] KL-divergence is very high: 142.5467 [2023-12-27 08:40:26,481][20381] Updated weights for policy 1, policy_version 437933 (0.0006) [2023-12-27 08:40:26,554][20347] Updated weights for policy 0, policy_version 438277 (0.0010) [2023-12-27 08:40:26,598][20347] Updated weights for policy 0, policy_version 438287 (0.0010) [2023-12-27 08:40:26,660][20347] Updated weights for policy 0, policy_version 438297 (0.0009) [2023-12-27 08:40:27,010][20381] Updated weights for policy 1, policy_version 437943 (0.0005) [2023-12-27 08:40:27,064][20381] Updated weights for policy 1, policy_version 437953 (0.0007) [2023-12-27 08:40:27,108][20381] Updated weights for policy 1, policy_version 437963 (0.0008) [2023-12-27 08:40:27,268][20347] Updated weights for policy 0, policy_version 438307 (0.0009) [2023-12-27 08:40:27,313][20347] Updated weights for policy 0, policy_version 438317 (0.0010) [2023-12-27 08:40:27,358][20347] Updated weights for policy 0, policy_version 438327 (0.0006) [2023-12-27 08:40:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20021.8). Total num frames: 224362496. Throughput: 0: 10321.7, 1: 10188.4. Samples: 224352692. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:27,363][19170] Avg episode reward: [(0, '682.360'), (1, '924.710')] [2023-12-27 08:40:27,722][20381] Updated weights for policy 1, policy_version 437973 (0.0007) [2023-12-27 08:40:27,766][20381] Updated weights for policy 1, policy_version 437983 (0.0007) [2023-12-27 08:40:27,817][20381] Updated weights for policy 1, policy_version 437993 (0.0007) [2023-12-27 08:40:28,050][20347] Updated weights for policy 0, policy_version 438337 (0.0009) [2023-12-27 08:40:28,101][20347] Updated weights for policy 0, policy_version 438347 (0.0010) [2023-12-27 08:40:28,161][20347] Updated weights for policy 0, policy_version 438357 (0.0010) [2023-12-27 08:40:28,216][20347] Updated weights for policy 0, policy_version 438367 (0.0010) [2023-12-27 08:40:28,471][20381] Updated weights for policy 1, policy_version 438003 (0.0007) [2023-12-27 08:40:28,515][20381] Updated weights for policy 1, policy_version 438013 (0.0008) [2023-12-27 08:40:28,563][20381] Updated weights for policy 1, policy_version 438023 (0.0007) [2023-12-27 08:40:28,885][20347] Updated weights for policy 0, policy_version 438377 (0.0010) [2023-12-27 08:40:28,943][20347] Updated weights for policy 0, policy_version 438387 (0.0010) [2023-12-27 08:40:29,003][20347] Updated weights for policy 0, policy_version 438397 (0.0010) [2023-12-27 08:40:29,215][20381] Updated weights for policy 1, policy_version 438033 (0.0007) [2023-12-27 08:40:29,274][20381] Updated weights for policy 1, policy_version 438043 (0.0008) [2023-12-27 08:40:29,322][20381] Updated weights for policy 1, policy_version 438053 (0.0008) [2023-12-27 08:40:29,383][20381] Updated weights for policy 1, policy_version 438063 (0.0008) [2023-12-27 08:40:29,672][20347] Updated weights for policy 0, policy_version 438407 (0.0008) [2023-12-27 08:40:29,738][20347] Updated weights for policy 0, policy_version 438417 (0.0006) [2023-12-27 08:40:29,796][20347] Updated weights for policy 0, policy_version 438427 (0.0010) [2023-12-27 08:40:30,099][20381] Updated weights for policy 1, policy_version 438073 (0.0007) [2023-12-27 08:40:30,156][20381] Updated weights for policy 1, policy_version 438083 (0.0007) [2023-12-27 08:40:30,213][20381] Updated weights for policy 1, policy_version 438093 (0.0008) [2023-12-27 08:40:30,437][20347] Updated weights for policy 0, policy_version 438437 (0.0007) [2023-12-27 08:40:30,487][20347] Updated weights for policy 0, policy_version 438447 (0.0006) [2023-12-27 08:40:30,535][20347] Updated weights for policy 0, policy_version 438457 (0.0005) [2023-12-27 08:40:30,845][20381] Updated weights for policy 1, policy_version 438103 (0.0007) [2023-12-27 08:40:30,893][20381] Updated weights for policy 1, policy_version 438113 (0.0007) [2023-12-27 08:40:30,941][20381] Updated weights for policy 1, policy_version 438123 (0.0007) [2023-12-27 08:40:31,185][20347] Updated weights for policy 0, policy_version 438467 (0.0006) [2023-12-27 08:40:31,241][20347] Updated weights for policy 0, policy_version 438477 (0.0007) [2023-12-27 08:40:31,298][20347] Updated weights for policy 0, policy_version 438487 (0.0009) [2023-12-27 08:40:31,570][20381] Updated weights for policy 1, policy_version 438133 (0.0008) [2023-12-27 08:40:31,629][20381] Updated weights for policy 1, policy_version 438143 (0.0008) [2023-12-27 08:40:31,691][20381] Updated weights for policy 1, policy_version 438153 (0.0006) [2023-12-27 08:40:32,033][20347] Updated weights for policy 0, policy_version 438497 (0.0007) [2023-12-27 08:40:32,091][20347] Updated weights for policy 0, policy_version 438507 (0.0007) [2023-12-27 08:40:32,150][20347] Updated weights for policy 0, policy_version 438517 (0.0008) [2023-12-27 08:40:32,175][20291] KL-divergence is very high: 254.5379 [2023-12-27 08:40:32,189][20291] KL-divergence is very high: 312.6254 [2023-12-27 08:40:32,213][20347] Updated weights for policy 0, policy_version 438527 (0.0006) [2023-12-27 08:40:32,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 19994.0). Total num frames: 224468992. Throughput: 0: 10406.6, 1: 10222.7. Samples: 224479808. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:32,364][19170] Avg episode reward: [(0, '865.410'), (1, '933.570')] [2023-12-27 08:40:32,390][20381] Updated weights for policy 1, policy_version 438163 (0.0008) [2023-12-27 08:40:32,447][20381] Updated weights for policy 1, policy_version 438173 (0.0007) [2023-12-27 08:40:32,515][20381] Updated weights for policy 1, policy_version 438183 (0.0005) [2023-12-27 08:40:32,844][20347] Updated weights for policy 0, policy_version 438537 (0.0010) [2023-12-27 08:40:32,889][20347] Updated weights for policy 0, policy_version 438547 (0.0010) [2023-12-27 08:40:32,949][20347] Updated weights for policy 0, policy_version 438557 (0.0010) [2023-12-27 08:40:33,111][20381] Updated weights for policy 1, policy_version 438193 (0.0008) [2023-12-27 08:40:33,170][20381] Updated weights for policy 1, policy_version 438203 (0.0010) [2023-12-27 08:40:33,230][20381] Updated weights for policy 1, policy_version 438213 (0.0010) [2023-12-27 08:40:33,289][20381] Updated weights for policy 1, policy_version 438223 (0.0010) [2023-12-27 08:40:33,586][20347] Updated weights for policy 0, policy_version 438567 (0.0010) [2023-12-27 08:40:33,634][20347] Updated weights for policy 0, policy_version 438577 (0.0010) [2023-12-27 08:40:33,680][20347] Updated weights for policy 0, policy_version 438587 (0.0010) [2023-12-27 08:40:33,911][20381] Updated weights for policy 1, policy_version 438233 (0.0010) [2023-12-27 08:40:33,958][20381] Updated weights for policy 1, policy_version 438243 (0.0010) [2023-12-27 08:40:34,009][20381] Updated weights for policy 1, policy_version 438253 (0.0010) [2023-12-27 08:40:34,427][20347] Updated weights for policy 0, policy_version 438597 (0.0009) [2023-12-27 08:40:34,487][20347] Updated weights for policy 0, policy_version 438607 (0.0008) [2023-12-27 08:40:34,555][20347] Updated weights for policy 0, policy_version 438617 (0.0008) [2023-12-27 08:40:34,655][20381] Updated weights for policy 1, policy_version 438263 (0.0008) [2023-12-27 08:40:34,707][20381] Updated weights for policy 1, policy_version 438273 (0.0010) [2023-12-27 08:40:34,759][20381] Updated weights for policy 1, policy_version 438283 (0.0006) [2023-12-27 08:40:35,215][20347] Updated weights for policy 0, policy_version 438627 (0.0010) [2023-12-27 08:40:35,272][20347] Updated weights for policy 0, policy_version 438637 (0.0010) [2023-12-27 08:40:35,327][20347] Updated weights for policy 0, policy_version 438647 (0.0011) [2023-12-27 08:40:35,360][20381] Updated weights for policy 1, policy_version 438293 (0.0005) [2023-12-27 08:40:35,409][20381] Updated weights for policy 1, policy_version 438303 (0.0005) [2023-12-27 08:40:35,459][20381] Updated weights for policy 1, policy_version 438313 (0.0005) [2023-12-27 08:40:35,979][20381] Updated weights for policy 1, policy_version 438323 (0.0005) [2023-12-27 08:40:36,010][20347] Updated weights for policy 0, policy_version 438657 (0.0010) [2023-12-27 08:40:36,032][20381] Updated weights for policy 1, policy_version 438333 (0.0006) [2023-12-27 08:40:36,061][20347] Updated weights for policy 0, policy_version 438667 (0.0009) [2023-12-27 08:40:36,089][20381] Updated weights for policy 1, policy_version 438343 (0.0011) [2023-12-27 08:40:36,118][20347] Updated weights for policy 0, policy_version 438677 (0.0010) [2023-12-27 08:40:36,179][20347] Updated weights for policy 0, policy_version 438687 (0.0010) [2023-12-27 08:40:36,764][20381] Updated weights for policy 1, policy_version 438353 (0.0011) [2023-12-27 08:40:36,820][20381] Updated weights for policy 1, policy_version 438363 (0.0011) [2023-12-27 08:40:36,873][20347] Updated weights for policy 0, policy_version 438697 (0.0010) [2023-12-27 08:40:36,877][20381] Updated weights for policy 1, policy_version 438373 (0.0011) [2023-12-27 08:40:36,928][20347] Updated weights for policy 0, policy_version 438707 (0.0010) [2023-12-27 08:40:36,936][20381] Updated weights for policy 1, policy_version 438383 (0.0011) [2023-12-27 08:40:36,986][20347] Updated weights for policy 0, policy_version 438717 (0.0010) [2023-12-27 08:40:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20021.8). Total num frames: 224575488. Throughput: 0: 10395.9, 1: 10311.9. Samples: 224543712. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:37,363][19170] Avg episode reward: [(0, '855.920'), (1, '971.120')] [2023-12-27 08:40:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000438384_112246784.pth... [2023-12-27 08:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000438720_112328704.pth... [2023-12-27 08:40:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000437168_111935488.pth [2023-12-27 08:40:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000437504_112017408.pth [2023-12-27 08:40:37,636][20381] Updated weights for policy 1, policy_version 438393 (0.0011) [2023-12-27 08:40:37,665][20347] Updated weights for policy 0, policy_version 438727 (0.0010) [2023-12-27 08:40:37,687][20381] Updated weights for policy 1, policy_version 438403 (0.0010) [2023-12-27 08:40:37,725][20347] Updated weights for policy 0, policy_version 438737 (0.0010) [2023-12-27 08:40:37,743][20381] Updated weights for policy 1, policy_version 438413 (0.0011) [2023-12-27 08:40:37,779][20347] Updated weights for policy 0, policy_version 438747 (0.0010) [2023-12-27 08:40:38,430][20381] Updated weights for policy 1, policy_version 438423 (0.0007) [2023-12-27 08:40:38,450][20347] Updated weights for policy 0, policy_version 438757 (0.0010) [2023-12-27 08:40:38,489][20381] Updated weights for policy 1, policy_version 438433 (0.0011) [2023-12-27 08:40:38,513][20347] Updated weights for policy 0, policy_version 438767 (0.0007) [2023-12-27 08:40:38,546][20381] Updated weights for policy 1, policy_version 438443 (0.0011) [2023-12-27 08:40:38,569][20347] Updated weights for policy 0, policy_version 438777 (0.0010) [2023-12-27 08:40:39,211][20381] Updated weights for policy 1, policy_version 438453 (0.0010) [2023-12-27 08:40:39,248][20347] Updated weights for policy 0, policy_version 438787 (0.0010) [2023-12-27 08:40:39,272][20381] Updated weights for policy 1, policy_version 438463 (0.0007) [2023-12-27 08:40:39,299][20347] Updated weights for policy 0, policy_version 438797 (0.0008) [2023-12-27 08:40:39,323][20381] Updated weights for policy 1, policy_version 438473 (0.0009) [2023-12-27 08:40:39,362][20347] Updated weights for policy 0, policy_version 438807 (0.0008) [2023-12-27 08:40:40,048][20381] Updated weights for policy 1, policy_version 438483 (0.0008) [2023-12-27 08:40:40,089][20347] Updated weights for policy 0, policy_version 438817 (0.0010) [2023-12-27 08:40:40,102][20381] Updated weights for policy 1, policy_version 438493 (0.0009) [2023-12-27 08:40:40,138][20347] Updated weights for policy 0, policy_version 438827 (0.0009) [2023-12-27 08:40:40,152][20381] Updated weights for policy 1, policy_version 438503 (0.0009) [2023-12-27 08:40:40,191][20347] Updated weights for policy 0, policy_version 438837 (0.0009) [2023-12-27 08:40:40,252][20347] Updated weights for policy 0, policy_version 438847 (0.0009) [2023-12-27 08:40:40,843][20381] Updated weights for policy 1, policy_version 438513 (0.0009) [2023-12-27 08:40:40,884][20347] Updated weights for policy 0, policy_version 438857 (0.0010) [2023-12-27 08:40:40,902][20381] Updated weights for policy 1, policy_version 438523 (0.0008) [2023-12-27 08:40:40,934][20347] Updated weights for policy 0, policy_version 438867 (0.0010) [2023-12-27 08:40:40,953][20381] Updated weights for policy 1, policy_version 438533 (0.0010) [2023-12-27 08:40:40,986][20347] Updated weights for policy 0, policy_version 438877 (0.0010) [2023-12-27 08:40:41,005][20381] Updated weights for policy 1, policy_version 438543 (0.0011) [2023-12-27 08:40:41,714][20347] Updated weights for policy 0, policy_version 438887 (0.0011) [2023-12-27 08:40:41,722][20381] Updated weights for policy 1, policy_version 438553 (0.0009) [2023-12-27 08:40:41,768][20347] Updated weights for policy 0, policy_version 438897 (0.0011) [2023-12-27 08:40:41,779][20381] Updated weights for policy 1, policy_version 438563 (0.0007) [2023-12-27 08:40:41,814][20347] Updated weights for policy 0, policy_version 438907 (0.0010) [2023-12-27 08:40:41,824][20381] Updated weights for policy 1, policy_version 438573 (0.0006) [2023-12-27 08:40:42,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 19994.0). Total num frames: 224673792. Throughput: 0: 10439.2, 1: 10358.4. Samples: 224666644. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:42,363][19170] Avg episode reward: [(0, '798.850'), (1, '961.420')] [2023-12-27 08:40:42,522][20381] Updated weights for policy 1, policy_version 438583 (0.0010) [2023-12-27 08:40:42,542][20347] Updated weights for policy 0, policy_version 438917 (0.0010) [2023-12-27 08:40:42,578][20381] Updated weights for policy 1, policy_version 438593 (0.0010) [2023-12-27 08:40:42,597][20347] Updated weights for policy 0, policy_version 438927 (0.0010) [2023-12-27 08:40:42,637][20381] Updated weights for policy 1, policy_version 438603 (0.0011) [2023-12-27 08:40:42,652][20347] Updated weights for policy 0, policy_version 438937 (0.0010) [2023-12-27 08:40:43,338][20347] Updated weights for policy 0, policy_version 438947 (0.0010) [2023-12-27 08:40:43,339][20381] Updated weights for policy 1, policy_version 438613 (0.0010) [2023-12-27 08:40:43,391][20381] Updated weights for policy 1, policy_version 438623 (0.0010) [2023-12-27 08:40:43,391][20347] Updated weights for policy 0, policy_version 438957 (0.0010) [2023-12-27 08:40:43,441][20381] Updated weights for policy 1, policy_version 438633 (0.0010) [2023-12-27 08:40:43,446][20347] Updated weights for policy 0, policy_version 438967 (0.0010) [2023-12-27 08:40:44,118][20347] Updated weights for policy 0, policy_version 438977 (0.0010) [2023-12-27 08:40:44,120][20381] Updated weights for policy 1, policy_version 438643 (0.0006) [2023-12-27 08:40:44,172][20381] Updated weights for policy 1, policy_version 438653 (0.0006) [2023-12-27 08:40:44,176][20347] Updated weights for policy 0, policy_version 438987 (0.0010) [2023-12-27 08:40:44,222][20381] Updated weights for policy 1, policy_version 438663 (0.0005) [2023-12-27 08:40:44,224][20347] Updated weights for policy 0, policy_version 438997 (0.0010) [2023-12-27 08:40:44,286][20347] Updated weights for policy 0, policy_version 439007 (0.0010) [2023-12-27 08:40:44,770][20381] Updated weights for policy 1, policy_version 438673 (0.0006) [2023-12-27 08:40:44,811][20292] KL-divergence is very high: 182.1966 [2023-12-27 08:40:44,831][20381] Updated weights for policy 1, policy_version 438683 (0.0008) [2023-12-27 08:40:44,894][20381] Updated weights for policy 1, policy_version 438693 (0.0009) [2023-12-27 08:40:44,936][20347] Updated weights for policy 0, policy_version 439017 (0.0011) [2023-12-27 08:40:44,957][20381] Updated weights for policy 1, policy_version 438703 (0.0010) [2023-12-27 08:40:45,000][20347] Updated weights for policy 0, policy_version 439027 (0.0010) [2023-12-27 08:40:45,060][20347] Updated weights for policy 0, policy_version 439037 (0.0010) [2023-12-27 08:40:45,566][20381] Updated weights for policy 1, policy_version 438713 (0.0010) [2023-12-27 08:40:45,616][20381] Updated weights for policy 1, policy_version 438723 (0.0005) [2023-12-27 08:40:45,670][20381] Updated weights for policy 1, policy_version 438733 (0.0005) [2023-12-27 08:40:45,736][20347] Updated weights for policy 0, policy_version 439047 (0.0010) [2023-12-27 08:40:45,784][20347] Updated weights for policy 0, policy_version 439057 (0.0010) [2023-12-27 08:40:45,834][20347] Updated weights for policy 0, policy_version 439067 (0.0010) [2023-12-27 08:40:46,279][20381] Updated weights for policy 1, policy_version 438743 (0.0005) [2023-12-27 08:40:46,333][20381] Updated weights for policy 1, policy_version 438753 (0.0005) [2023-12-27 08:40:46,376][20381] Updated weights for policy 1, policy_version 438763 (0.0005) [2023-12-27 08:40:46,521][20347] Updated weights for policy 0, policy_version 439077 (0.0010) [2023-12-27 08:40:46,569][20347] Updated weights for policy 0, policy_version 439087 (0.0010) [2023-12-27 08:40:46,616][20347] Updated weights for policy 0, policy_version 439097 (0.0010) [2023-12-27 08:40:46,919][20381] Updated weights for policy 1, policy_version 438773 (0.0007) [2023-12-27 08:40:46,980][20381] Updated weights for policy 1, policy_version 438783 (0.0007) [2023-12-27 08:40:47,028][20381] Updated weights for policy 1, policy_version 438793 (0.0010) [2023-12-27 08:40:47,305][20347] Updated weights for policy 0, policy_version 439107 (0.0010) [2023-12-27 08:40:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20021.8). Total num frames: 224780288. Throughput: 0: 10450.0, 1: 10434.3. Samples: 224792576. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:47,363][19170] Avg episode reward: [(0, '866.800'), (1, '865.690')] [2023-12-27 08:40:47,366][20347] Updated weights for policy 0, policy_version 439117 (0.0005) [2023-12-27 08:40:47,421][20347] Updated weights for policy 0, policy_version 439127 (0.0006) [2023-12-27 08:40:47,656][20381] Updated weights for policy 1, policy_version 438803 (0.0010) [2023-12-27 08:40:47,714][20381] Updated weights for policy 1, policy_version 438813 (0.0011) [2023-12-27 08:40:47,769][20381] Updated weights for policy 1, policy_version 438823 (0.0010) [2023-12-27 08:40:48,059][20347] Updated weights for policy 0, policy_version 439137 (0.0010) [2023-12-27 08:40:48,113][20347] Updated weights for policy 0, policy_version 439147 (0.0010) [2023-12-27 08:40:48,180][20347] Updated weights for policy 0, policy_version 439157 (0.0010) [2023-12-27 08:40:48,239][20347] Updated weights for policy 0, policy_version 439167 (0.0010) [2023-12-27 08:40:48,367][20381] Updated weights for policy 1, policy_version 438833 (0.0007) [2023-12-27 08:40:48,422][20381] Updated weights for policy 1, policy_version 438843 (0.0008) [2023-12-27 08:40:48,480][20381] Updated weights for policy 1, policy_version 438853 (0.0005) [2023-12-27 08:40:48,536][20381] Updated weights for policy 1, policy_version 438863 (0.0008) [2023-12-27 08:40:48,932][20347] Updated weights for policy 0, policy_version 439177 (0.0007) [2023-12-27 08:40:48,983][20347] Updated weights for policy 0, policy_version 439187 (0.0008) [2023-12-27 08:40:49,042][20347] Updated weights for policy 0, policy_version 439197 (0.0007) [2023-12-27 08:40:49,186][20381] Updated weights for policy 1, policy_version 438873 (0.0011) [2023-12-27 08:40:49,250][20381] Updated weights for policy 1, policy_version 438883 (0.0009) [2023-12-27 08:40:49,307][20381] Updated weights for policy 1, policy_version 438893 (0.0011) [2023-12-27 08:40:49,681][20347] Updated weights for policy 0, policy_version 439207 (0.0010) [2023-12-27 08:40:49,736][20347] Updated weights for policy 0, policy_version 439217 (0.0010) [2023-12-27 08:40:49,787][20347] Updated weights for policy 0, policy_version 439227 (0.0010) [2023-12-27 08:40:49,908][20381] Updated weights for policy 1, policy_version 438903 (0.0007) [2023-12-27 08:40:49,970][20381] Updated weights for policy 1, policy_version 438913 (0.0008) [2023-12-27 08:40:50,025][20381] Updated weights for policy 1, policy_version 438923 (0.0005) [2023-12-27 08:40:50,499][20347] Updated weights for policy 0, policy_version 439237 (0.0010) [2023-12-27 08:40:50,546][20347] Updated weights for policy 0, policy_version 439247 (0.0010) [2023-12-27 08:40:50,602][20347] Updated weights for policy 0, policy_version 439257 (0.0008) [2023-12-27 08:40:50,700][20381] Updated weights for policy 1, policy_version 438933 (0.0006) [2023-12-27 08:40:50,734][20292] KL-divergence is very high: 153.0985 [2023-12-27 08:40:50,759][20381] Updated weights for policy 1, policy_version 438943 (0.0011) [2023-12-27 08:40:50,820][20381] Updated weights for policy 1, policy_version 438953 (0.0008) [2023-12-27 08:40:50,833][20292] KL-divergence is very high: 137.4616 [2023-12-27 08:40:51,280][20347] Updated weights for policy 0, policy_version 439267 (0.0007) [2023-12-27 08:40:51,331][20347] Updated weights for policy 0, policy_version 439277 (0.0010) [2023-12-27 08:40:51,392][20347] Updated weights for policy 0, policy_version 439287 (0.0009) [2023-12-27 08:40:51,502][20381] Updated weights for policy 1, policy_version 438963 (0.0008) [2023-12-27 08:40:51,559][20381] Updated weights for policy 1, policy_version 438973 (0.0011) [2023-12-27 08:40:51,621][20381] Updated weights for policy 1, policy_version 438983 (0.0011) [2023-12-27 08:40:52,076][20347] Updated weights for policy 0, policy_version 439297 (0.0010) [2023-12-27 08:40:52,140][20347] Updated weights for policy 0, policy_version 439307 (0.0010) [2023-12-27 08:40:52,199][20347] Updated weights for policy 0, policy_version 439317 (0.0006) [2023-12-27 08:40:52,256][20347] Updated weights for policy 0, policy_version 439327 (0.0006) [2023-12-27 08:40:52,343][20381] Updated weights for policy 1, policy_version 438993 (0.0008) [2023-12-27 08:40:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 19994.0). Total num frames: 224886784. Throughput: 0: 10419.2, 1: 10471.2. Samples: 224856988. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:52,363][19170] Avg episode reward: [(0, '828.740'), (1, '807.480')] [2023-12-27 08:40:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000439328_112484352.pth... [2023-12-27 08:40:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000438112_112173056.pth [2023-12-27 08:40:52,401][20381] Updated weights for policy 1, policy_version 439003 (0.0007) [2023-12-27 08:40:52,458][20381] Updated weights for policy 1, policy_version 439013 (0.0007) [2023-12-27 08:40:52,522][20381] Updated weights for policy 1, policy_version 439023 (0.0008) [2023-12-27 08:40:52,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000439024_112410624.pth... [2023-12-27 08:40:52,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000437744_112082944.pth [2023-12-27 08:40:52,897][20347] Updated weights for policy 0, policy_version 439337 (0.0007) [2023-12-27 08:40:52,951][20347] Updated weights for policy 0, policy_version 439347 (0.0007) [2023-12-27 08:40:52,959][20291] KL-divergence is very high: 108.9356 [2023-12-27 08:40:53,010][20347] Updated weights for policy 0, policy_version 439357 (0.0008) [2023-12-27 08:40:53,228][20381] Updated weights for policy 1, policy_version 439033 (0.0009) [2023-12-27 08:40:53,291][20381] Updated weights for policy 1, policy_version 439043 (0.0010) [2023-12-27 08:40:53,351][20381] Updated weights for policy 1, policy_version 439053 (0.0010) [2023-12-27 08:40:53,682][20347] Updated weights for policy 0, policy_version 439367 (0.0008) [2023-12-27 08:40:53,738][20347] Updated weights for policy 0, policy_version 439377 (0.0007) [2023-12-27 08:40:53,798][20347] Updated weights for policy 0, policy_version 439387 (0.0008) [2023-12-27 08:40:53,986][20381] Updated weights for policy 1, policy_version 439063 (0.0007) [2023-12-27 08:40:54,046][20381] Updated weights for policy 1, policy_version 439073 (0.0006) [2023-12-27 08:40:54,110][20381] Updated weights for policy 1, policy_version 439083 (0.0007) [2023-12-27 08:40:54,443][20347] Updated weights for policy 0, policy_version 439397 (0.0007) [2023-12-27 08:40:54,500][20347] Updated weights for policy 0, policy_version 439407 (0.0008) [2023-12-27 08:40:54,558][20347] Updated weights for policy 0, policy_version 439417 (0.0008) [2023-12-27 08:40:54,679][20381] Updated weights for policy 1, policy_version 439093 (0.0010) [2023-12-27 08:40:54,731][20381] Updated weights for policy 1, policy_version 439103 (0.0010) [2023-12-27 08:40:54,787][20381] Updated weights for policy 1, policy_version 439113 (0.0010) [2023-12-27 08:40:55,236][20347] Updated weights for policy 0, policy_version 439427 (0.0006) [2023-12-27 08:40:55,286][20347] Updated weights for policy 0, policy_version 439437 (0.0007) [2023-12-27 08:40:55,330][20347] Updated weights for policy 0, policy_version 439447 (0.0007) [2023-12-27 08:40:55,464][20381] Updated weights for policy 1, policy_version 439123 (0.0009) [2023-12-27 08:40:55,518][20381] Updated weights for policy 1, policy_version 439133 (0.0008) [2023-12-27 08:40:55,571][20381] Updated weights for policy 1, policy_version 439143 (0.0007) [2023-12-27 08:40:56,015][20347] Updated weights for policy 0, policy_version 439457 (0.0008) [2023-12-27 08:40:56,069][20347] Updated weights for policy 0, policy_version 439467 (0.0007) [2023-12-27 08:40:56,122][20347] Updated weights for policy 0, policy_version 439477 (0.0008) [2023-12-27 08:40:56,174][20347] Updated weights for policy 0, policy_version 439487 (0.0006) [2023-12-27 08:40:56,297][20381] Updated weights for policy 1, policy_version 439153 (0.0008) [2023-12-27 08:40:56,360][20381] Updated weights for policy 1, policy_version 439163 (0.0008) [2023-12-27 08:40:56,409][20381] Updated weights for policy 1, policy_version 439173 (0.0009) [2023-12-27 08:40:56,467][20381] Updated weights for policy 1, policy_version 439183 (0.0006) [2023-12-27 08:40:56,749][20347] Updated weights for policy 0, policy_version 439497 (0.0005) [2023-12-27 08:40:56,802][20347] Updated weights for policy 0, policy_version 439507 (0.0005) [2023-12-27 08:40:56,847][20347] Updated weights for policy 0, policy_version 439517 (0.0006) [2023-12-27 08:40:57,061][20381] Updated weights for policy 1, policy_version 439193 (0.0008) [2023-12-27 08:40:57,113][20381] Updated weights for policy 1, policy_version 439203 (0.0008) [2023-12-27 08:40:57,169][20381] Updated weights for policy 1, policy_version 439213 (0.0008) [2023-12-27 08:40:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.7, 300 sec: 20021.8). Total num frames: 224993280. Throughput: 0: 10324.6, 1: 10521.0. Samples: 224979664. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:40:57,363][19170] Avg episode reward: [(0, '751.810'), (1, '913.030')] [2023-12-27 08:40:57,468][20347] Updated weights for policy 0, policy_version 439527 (0.0007) [2023-12-27 08:40:57,518][20347] Updated weights for policy 0, policy_version 439537 (0.0007) [2023-12-27 08:40:57,568][20347] Updated weights for policy 0, policy_version 439547 (0.0006) [2023-12-27 08:40:57,849][20381] Updated weights for policy 1, policy_version 439223 (0.0006) [2023-12-27 08:40:57,904][20381] Updated weights for policy 1, policy_version 439233 (0.0005) [2023-12-27 08:40:57,967][20381] Updated weights for policy 1, policy_version 439243 (0.0007) [2023-12-27 08:40:58,293][20347] Updated weights for policy 0, policy_version 439557 (0.0007) [2023-12-27 08:40:58,358][20347] Updated weights for policy 0, policy_version 439567 (0.0007) [2023-12-27 08:40:58,422][20347] Updated weights for policy 0, policy_version 439577 (0.0008) [2023-12-27 08:40:58,609][20381] Updated weights for policy 1, policy_version 439253 (0.0009) [2023-12-27 08:40:58,669][20381] Updated weights for policy 1, policy_version 439263 (0.0010) [2023-12-27 08:40:58,729][20381] Updated weights for policy 1, policy_version 439273 (0.0010) [2023-12-27 08:40:59,082][20347] Updated weights for policy 0, policy_version 439587 (0.0007) [2023-12-27 08:40:59,150][20347] Updated weights for policy 0, policy_version 439597 (0.0010) [2023-12-27 08:40:59,206][20347] Updated weights for policy 0, policy_version 439607 (0.0010) [2023-12-27 08:40:59,395][20381] Updated weights for policy 1, policy_version 439283 (0.0009) [2023-12-27 08:40:59,453][20381] Updated weights for policy 1, policy_version 439293 (0.0005) [2023-12-27 08:40:59,510][20381] Updated weights for policy 1, policy_version 439303 (0.0005) [2023-12-27 08:40:59,876][20347] Updated weights for policy 0, policy_version 439617 (0.0010) [2023-12-27 08:40:59,932][20347] Updated weights for policy 0, policy_version 439627 (0.0008) [2023-12-27 08:40:59,986][20347] Updated weights for policy 0, policy_version 439637 (0.0007) [2023-12-27 08:41:00,054][20347] Updated weights for policy 0, policy_version 439648 (0.0009) [2023-12-27 08:41:00,127][20381] Updated weights for policy 1, policy_version 439313 (0.0006) [2023-12-27 08:41:00,191][20381] Updated weights for policy 1, policy_version 439323 (0.0009) [2023-12-27 08:41:00,249][20381] Updated weights for policy 1, policy_version 439333 (0.0010) [2023-12-27 08:41:00,307][20381] Updated weights for policy 1, policy_version 439343 (0.0010) [2023-12-27 08:41:00,764][20347] Updated weights for policy 0, policy_version 439658 (0.0011) [2023-12-27 08:41:00,820][20347] Updated weights for policy 0, policy_version 439668 (0.0010) [2023-12-27 08:41:00,881][20347] Updated weights for policy 0, policy_version 439678 (0.0010) [2023-12-27 08:41:00,926][20381] Updated weights for policy 1, policy_version 439353 (0.0006) [2023-12-27 08:41:00,988][20381] Updated weights for policy 1, policy_version 439363 (0.0007) [2023-12-27 08:41:01,056][20381] Updated weights for policy 1, policy_version 439373 (0.0009) [2023-12-27 08:41:01,541][20347] Updated weights for policy 0, policy_version 439688 (0.0010) [2023-12-27 08:41:01,600][20347] Updated weights for policy 0, policy_version 439698 (0.0010) [2023-12-27 08:41:01,661][20347] Updated weights for policy 0, policy_version 439708 (0.0009) [2023-12-27 08:41:01,685][20381] Updated weights for policy 1, policy_version 439383 (0.0005) [2023-12-27 08:41:01,744][20381] Updated weights for policy 1, policy_version 439393 (0.0007) [2023-12-27 08:41:01,808][20381] Updated weights for policy 1, policy_version 439403 (0.0008) [2023-12-27 08:41:02,274][20347] Updated weights for policy 0, policy_version 439718 (0.0009) [2023-12-27 08:41:02,339][20347] Updated weights for policy 0, policy_version 439728 (0.0008) [2023-12-27 08:41:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20021.8). Total num frames: 225091584. Throughput: 0: 10268.7, 1: 10549.1. Samples: 225105440. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:41:02,363][19170] Avg episode reward: [(0, '818.520'), (1, '903.130')] [2023-12-27 08:41:02,408][20347] Updated weights for policy 0, policy_version 439738 (0.0008) [2023-12-27 08:41:02,519][20381] Updated weights for policy 1, policy_version 439413 (0.0008) [2023-12-27 08:41:02,583][20381] Updated weights for policy 1, policy_version 439423 (0.0009) [2023-12-27 08:41:02,633][20381] Updated weights for policy 1, policy_version 439433 (0.0006) [2023-12-27 08:41:03,043][20347] Updated weights for policy 0, policy_version 439748 (0.0007) [2023-12-27 08:41:03,097][20347] Updated weights for policy 0, policy_version 439758 (0.0005) [2023-12-27 08:41:03,152][20347] Updated weights for policy 0, policy_version 439768 (0.0007) [2023-12-27 08:41:03,290][20381] Updated weights for policy 1, policy_version 439443 (0.0007) [2023-12-27 08:41:03,343][20381] Updated weights for policy 1, policy_version 439453 (0.0007) [2023-12-27 08:41:03,393][20381] Updated weights for policy 1, policy_version 439463 (0.0007) [2023-12-27 08:41:03,742][20347] Updated weights for policy 0, policy_version 439778 (0.0008) [2023-12-27 08:41:03,803][20347] Updated weights for policy 0, policy_version 439788 (0.0010) [2023-12-27 08:41:03,868][20347] Updated weights for policy 0, policy_version 439798 (0.0009) [2023-12-27 08:41:03,929][20347] Updated weights for policy 0, policy_version 439808 (0.0008) [2023-12-27 08:41:04,058][20381] Updated weights for policy 1, policy_version 439473 (0.0007) [2023-12-27 08:41:04,112][20381] Updated weights for policy 1, policy_version 439483 (0.0008) [2023-12-27 08:41:04,170][20381] Updated weights for policy 1, policy_version 439493 (0.0007) [2023-12-27 08:41:04,227][20381] Updated weights for policy 1, policy_version 439503 (0.0007) [2023-12-27 08:41:04,576][20347] Updated weights for policy 0, policy_version 439818 (0.0005) [2023-12-27 08:41:04,629][20347] Updated weights for policy 0, policy_version 439828 (0.0010) [2023-12-27 08:41:04,689][20347] Updated weights for policy 0, policy_version 439838 (0.0010) [2023-12-27 08:41:04,970][20381] Updated weights for policy 1, policy_version 439513 (0.0008) [2023-12-27 08:41:05,026][20381] Updated weights for policy 1, policy_version 439523 (0.0006) [2023-12-27 08:41:05,079][20381] Updated weights for policy 1, policy_version 439533 (0.0007) [2023-12-27 08:41:05,306][20347] Updated weights for policy 0, policy_version 439848 (0.0007) [2023-12-27 08:41:05,359][20347] Updated weights for policy 0, policy_version 439858 (0.0008) [2023-12-27 08:41:05,419][20347] Updated weights for policy 0, policy_version 439868 (0.0006) [2023-12-27 08:41:05,806][20381] Updated weights for policy 1, policy_version 439543 (0.0009) [2023-12-27 08:41:05,867][20381] Updated weights for policy 1, policy_version 439553 (0.0010) [2023-12-27 08:41:05,922][20381] Updated weights for policy 1, policy_version 439563 (0.0010) [2023-12-27 08:41:06,000][20347] Updated weights for policy 0, policy_version 439878 (0.0006) [2023-12-27 08:41:06,065][20347] Updated weights for policy 0, policy_version 439888 (0.0006) [2023-12-27 08:41:06,132][20347] Updated weights for policy 0, policy_version 439898 (0.0008) [2023-12-27 08:41:06,625][20381] Updated weights for policy 1, policy_version 439573 (0.0008) [2023-12-27 08:41:06,684][20381] Updated weights for policy 1, policy_version 439583 (0.0008) [2023-12-27 08:41:06,686][20292] KL-divergence is very high: 164.5443 [2023-12-27 08:41:06,745][20381] Updated weights for policy 1, policy_version 439593 (0.0007) [2023-12-27 08:41:06,807][20347] Updated weights for policy 0, policy_version 439908 (0.0006) [2023-12-27 08:41:06,866][20347] Updated weights for policy 0, policy_version 439918 (0.0005) [2023-12-27 08:41:06,915][20347] Updated weights for policy 0, policy_version 439928 (0.0008) [2023-12-27 08:41:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20021.8). Total num frames: 225198080. Throughput: 0: 10314.6, 1: 10529.0. Samples: 225168144. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:41:07,363][19170] Avg episode reward: [(0, '866.390'), (1, '893.200')] [2023-12-27 08:41:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000439936_112640000.pth... [2023-12-27 08:41:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000439600_112558080.pth... [2023-12-27 08:41:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000438720_112328704.pth [2023-12-27 08:41:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000438384_112246784.pth [2023-12-27 08:41:07,411][20381] Updated weights for policy 1, policy_version 439603 (0.0008) [2023-12-27 08:41:07,468][20381] Updated weights for policy 1, policy_version 439613 (0.0010) [2023-12-27 08:41:07,506][20347] Updated weights for policy 0, policy_version 439938 (0.0009) [2023-12-27 08:41:07,530][20381] Updated weights for policy 1, policy_version 439623 (0.0011) [2023-12-27 08:41:07,553][20347] Updated weights for policy 0, policy_version 439948 (0.0007) [2023-12-27 08:41:07,602][20347] Updated weights for policy 0, policy_version 439958 (0.0005) [2023-12-27 08:41:07,650][20347] Updated weights for policy 0, policy_version 439968 (0.0005) [2023-12-27 08:41:08,201][20381] Updated weights for policy 1, policy_version 439633 (0.0010) [2023-12-27 08:41:08,248][20347] Updated weights for policy 0, policy_version 439978 (0.0006) [2023-12-27 08:41:08,254][20381] Updated weights for policy 1, policy_version 439643 (0.0010) [2023-12-27 08:41:08,304][20347] Updated weights for policy 0, policy_version 439988 (0.0006) [2023-12-27 08:41:08,313][20381] Updated weights for policy 1, policy_version 439653 (0.0011) [2023-12-27 08:41:08,362][20347] Updated weights for policy 0, policy_version 439998 (0.0009) [2023-12-27 08:41:08,373][20381] Updated weights for policy 1, policy_version 439663 (0.0008) [2023-12-27 08:41:08,993][20381] Updated weights for policy 1, policy_version 439673 (0.0009) [2023-12-27 08:41:09,006][20347] Updated weights for policy 0, policy_version 440008 (0.0010) [2023-12-27 08:41:09,041][20381] Updated weights for policy 1, policy_version 439683 (0.0010) [2023-12-27 08:41:09,060][20347] Updated weights for policy 0, policy_version 440018 (0.0010) [2023-12-27 08:41:09,100][20381] Updated weights for policy 1, policy_version 439693 (0.0010) [2023-12-27 08:41:09,119][20347] Updated weights for policy 0, policy_version 440028 (0.0010) [2023-12-27 08:41:09,769][20347] Updated weights for policy 0, policy_version 440038 (0.0010) [2023-12-27 08:41:09,801][20381] Updated weights for policy 1, policy_version 439703 (0.0011) [2023-12-27 08:41:09,827][20347] Updated weights for policy 0, policy_version 440048 (0.0010) [2023-12-27 08:41:09,878][20381] Updated weights for policy 1, policy_version 439713 (0.0011) [2023-12-27 08:41:09,889][20347] Updated weights for policy 0, policy_version 440058 (0.0011) [2023-12-27 08:41:09,944][20381] Updated weights for policy 1, policy_version 439723 (0.0009) [2023-12-27 08:41:10,586][20347] Updated weights for policy 0, policy_version 440068 (0.0011) [2023-12-27 08:41:10,629][20381] Updated weights for policy 1, policy_version 439733 (0.0006) [2023-12-27 08:41:10,648][20347] Updated weights for policy 0, policy_version 440078 (0.0010) [2023-12-27 08:41:10,690][20381] Updated weights for policy 1, policy_version 439743 (0.0006) [2023-12-27 08:41:10,710][20347] Updated weights for policy 0, policy_version 440088 (0.0010) [2023-12-27 08:41:10,753][20381] Updated weights for policy 1, policy_version 439753 (0.0006) [2023-12-27 08:41:11,422][20347] Updated weights for policy 0, policy_version 440098 (0.0010) [2023-12-27 08:41:11,468][20381] Updated weights for policy 1, policy_version 439763 (0.0007) [2023-12-27 08:41:11,481][20347] Updated weights for policy 0, policy_version 440108 (0.0007) [2023-12-27 08:41:11,526][20381] Updated weights for policy 1, policy_version 439773 (0.0006) [2023-12-27 08:41:11,539][20347] Updated weights for policy 0, policy_version 440118 (0.0006) [2023-12-27 08:41:11,584][20381] Updated weights for policy 1, policy_version 439783 (0.0006) [2023-12-27 08:41:11,597][20347] Updated weights for policy 0, policy_version 440128 (0.0006) [2023-12-27 08:41:12,218][20347] Updated weights for policy 0, policy_version 440138 (0.0006) [2023-12-27 08:41:12,262][20381] Updated weights for policy 1, policy_version 439793 (0.0007) [2023-12-27 08:41:12,275][20347] Updated weights for policy 0, policy_version 440148 (0.0007) [2023-12-27 08:41:12,313][20381] Updated weights for policy 1, policy_version 439803 (0.0007) [2023-12-27 08:41:12,331][20347] Updated weights for policy 0, policy_version 440158 (0.0007) [2023-12-27 08:41:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20021.8). Total num frames: 225304576. Throughput: 0: 10398.2, 1: 10509.3. Samples: 225293532. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:41:12,363][19170] Avg episode reward: [(0, '856.540'), (1, '913.010')] [2023-12-27 08:41:12,374][20381] Updated weights for policy 1, policy_version 439813 (0.0008) [2023-12-27 08:41:12,430][20381] Updated weights for policy 1, policy_version 439823 (0.0007) [2023-12-27 08:41:12,984][20291] KL-divergence is very high: 227.4308 [2023-12-27 08:41:12,989][20347] Updated weights for policy 0, policy_version 440168 (0.0008) [2023-12-27 08:41:13,047][20347] Updated weights for policy 0, policy_version 440178 (0.0007) [2023-12-27 08:41:13,106][20347] Updated weights for policy 0, policy_version 440188 (0.0007) [2023-12-27 08:41:13,140][20381] Updated weights for policy 1, policy_version 439833 (0.0008) [2023-12-27 08:41:13,198][20381] Updated weights for policy 1, policy_version 439843 (0.0008) [2023-12-27 08:41:13,257][20381] Updated weights for policy 1, policy_version 439853 (0.0007) [2023-12-27 08:41:13,753][20347] Updated weights for policy 0, policy_version 440198 (0.0006) [2023-12-27 08:41:13,799][20347] Updated weights for policy 0, policy_version 440208 (0.0007) [2023-12-27 08:41:13,847][20347] Updated weights for policy 0, policy_version 440218 (0.0010) [2023-12-27 08:41:13,963][20381] Updated weights for policy 1, policy_version 439863 (0.0007) [2023-12-27 08:41:14,014][20381] Updated weights for policy 1, policy_version 439873 (0.0007) [2023-12-27 08:41:14,061][20381] Updated weights for policy 1, policy_version 439883 (0.0007) [2023-12-27 08:41:14,507][20347] Updated weights for policy 0, policy_version 440228 (0.0009) [2023-12-27 08:41:14,567][20347] Updated weights for policy 0, policy_version 440238 (0.0008) [2023-12-27 08:41:14,625][20347] Updated weights for policy 0, policy_version 440248 (0.0007) [2023-12-27 08:41:14,771][20381] Updated weights for policy 1, policy_version 439893 (0.0008) [2023-12-27 08:41:14,834][20381] Updated weights for policy 1, policy_version 439903 (0.0008) [2023-12-27 08:41:14,890][20381] Updated weights for policy 1, policy_version 439913 (0.0008) [2023-12-27 08:41:15,258][20347] Updated weights for policy 0, policy_version 440258 (0.0007) [2023-12-27 08:41:15,318][20347] Updated weights for policy 0, policy_version 440268 (0.0007) [2023-12-27 08:41:15,379][20347] Updated weights for policy 0, policy_version 440278 (0.0006) [2023-12-27 08:41:15,435][20347] Updated weights for policy 0, policy_version 440288 (0.0007) [2023-12-27 08:41:15,604][20381] Updated weights for policy 1, policy_version 439923 (0.0007) [2023-12-27 08:41:15,659][20381] Updated weights for policy 1, policy_version 439933 (0.0007) [2023-12-27 08:41:15,713][20381] Updated weights for policy 1, policy_version 439943 (0.0007) [2023-12-27 08:41:16,089][20347] Updated weights for policy 0, policy_version 440298 (0.0008) [2023-12-27 08:41:16,140][20347] Updated weights for policy 0, policy_version 440308 (0.0007) [2023-12-27 08:41:16,192][20347] Updated weights for policy 0, policy_version 440318 (0.0007) [2023-12-27 08:41:16,403][20381] Updated weights for policy 1, policy_version 439953 (0.0009) [2023-12-27 08:41:16,460][20381] Updated weights for policy 1, policy_version 439963 (0.0008) [2023-12-27 08:41:16,515][20381] Updated weights for policy 1, policy_version 439973 (0.0007) [2023-12-27 08:41:16,565][20381] Updated weights for policy 1, policy_version 439983 (0.0008) [2023-12-27 08:41:16,875][20347] Updated weights for policy 0, policy_version 440328 (0.0007) [2023-12-27 08:41:16,930][20347] Updated weights for policy 0, policy_version 440338 (0.0007) [2023-12-27 08:41:16,994][20347] Updated weights for policy 0, policy_version 440348 (0.0008) [2023-12-27 08:41:17,290][20381] Updated weights for policy 1, policy_version 439993 (0.0007) [2023-12-27 08:41:17,335][20381] Updated weights for policy 1, policy_version 440003 (0.0008) [2023-12-27 08:41:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20021.8). Total num frames: 225402880. Throughput: 0: 10408.1, 1: 10392.7. Samples: 225415844. Policy #0 lag: (min: 9.0, avg: 23.6, max: 41.0) [2023-12-27 08:41:17,363][19170] Avg episode reward: [(0, '791.300'), (1, '760.970')] [2023-12-27 08:41:17,381][20381] Updated weights for policy 1, policy_version 440013 (0.0008) [2023-12-27 08:41:17,628][20347] Updated weights for policy 0, policy_version 440358 (0.0007) [2023-12-27 08:41:17,679][20347] Updated weights for policy 0, policy_version 440368 (0.0007) [2023-12-27 08:41:17,730][20347] Updated weights for policy 0, policy_version 440378 (0.0008) [2023-12-27 08:41:18,124][20381] Updated weights for policy 1, policy_version 440023 (0.0007) [2023-12-27 08:41:18,180][20381] Updated weights for policy 1, policy_version 440033 (0.0005) [2023-12-27 08:41:18,239][20381] Updated weights for policy 1, policy_version 440043 (0.0008) [2023-12-27 08:41:18,411][20347] Updated weights for policy 0, policy_version 440388 (0.0008) [2023-12-27 08:41:18,463][20347] Updated weights for policy 0, policy_version 440398 (0.0008) [2023-12-27 08:41:18,515][20347] Updated weights for policy 0, policy_version 440408 (0.0008) [2023-12-27 08:41:18,936][20381] Updated weights for policy 1, policy_version 440053 (0.0008) [2023-12-27 08:41:18,983][20381] Updated weights for policy 1, policy_version 440063 (0.0007) [2023-12-27 08:41:19,034][20381] Updated weights for policy 1, policy_version 440073 (0.0008) [2023-12-27 08:41:19,198][20347] Updated weights for policy 0, policy_version 440418 (0.0006) [2023-12-27 08:41:19,258][20347] Updated weights for policy 0, policy_version 440428 (0.0008) [2023-12-27 08:41:19,315][20347] Updated weights for policy 0, policy_version 440438 (0.0008) [2023-12-27 08:41:19,374][20347] Updated weights for policy 0, policy_version 440448 (0.0008) [2023-12-27 08:41:19,848][20381] Updated weights for policy 1, policy_version 440083 (0.0008) [2023-12-27 08:41:19,911][20381] Updated weights for policy 1, policy_version 440093 (0.0008) [2023-12-27 08:41:19,976][20381] Updated weights for policy 1, policy_version 440103 (0.0007) [2023-12-27 08:41:20,013][20347] Updated weights for policy 0, policy_version 440458 (0.0008) [2023-12-27 08:41:20,077][20347] Updated weights for policy 0, policy_version 440468 (0.0009) [2023-12-27 08:41:20,140][20347] Updated weights for policy 0, policy_version 440478 (0.0009) [2023-12-27 08:41:20,665][20381] Updated weights for policy 1, policy_version 440113 (0.0007) [2023-12-27 08:41:20,730][20381] Updated weights for policy 1, policy_version 440123 (0.0007) [2023-12-27 08:41:20,792][20381] Updated weights for policy 1, policy_version 440133 (0.0011) [2023-12-27 08:41:20,826][20347] Updated weights for policy 0, policy_version 440488 (0.0008) [2023-12-27 08:41:20,846][20292] KL-divergence is very high: 100.4490 [2023-12-27 08:41:20,853][20381] Updated weights for policy 1, policy_version 440143 (0.0011) [2023-12-27 08:41:20,882][20347] Updated weights for policy 0, policy_version 440498 (0.0007) [2023-12-27 08:41:20,940][20347] Updated weights for policy 0, policy_version 440508 (0.0006) [2023-12-27 08:41:21,565][20381] Updated weights for policy 1, policy_version 440153 (0.0009) [2023-12-27 08:41:21,628][20381] Updated weights for policy 1, policy_version 440163 (0.0008) [2023-12-27 08:41:21,636][20347] Updated weights for policy 0, policy_version 440518 (0.0007) [2023-12-27 08:41:21,687][20381] Updated weights for policy 1, policy_version 440173 (0.0007) [2023-12-27 08:41:21,698][20347] Updated weights for policy 0, policy_version 440528 (0.0006) [2023-12-27 08:41:21,756][20347] Updated weights for policy 0, policy_version 440538 (0.0008) [2023-12-27 08:41:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 19994.0). Total num frames: 225501184. Throughput: 0: 10415.8, 1: 10303.7. Samples: 225476092. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:22,363][19170] Avg episode reward: [(0, '481.250'), (1, '776.570')] [2023-12-27 08:41:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000440544_112795648.pth... [2023-12-27 08:41:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000439328_112484352.pth [2023-12-27 08:41:22,404][20381] Updated weights for policy 1, policy_version 440183 (0.0008) [2023-12-27 08:41:22,462][20381] Updated weights for policy 1, policy_version 440193 (0.0008) [2023-12-27 08:41:22,522][20381] Updated weights for policy 1, policy_version 440203 (0.0007) [2023-12-27 08:41:22,525][20347] Updated weights for policy 0, policy_version 440548 (0.0007) [2023-12-27 08:41:22,550][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000440208_112713728.pth... [2023-12-27 08:41:22,553][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000439024_112410624.pth [2023-12-27 08:41:22,581][20347] Updated weights for policy 0, policy_version 440558 (0.0008) [2023-12-27 08:41:22,641][20347] Updated weights for policy 0, policy_version 440568 (0.0007) [2023-12-27 08:41:23,189][20381] Updated weights for policy 1, policy_version 440213 (0.0006) [2023-12-27 08:41:23,245][20381] Updated weights for policy 1, policy_version 440223 (0.0010) [2023-12-27 08:41:23,309][20347] Updated weights for policy 0, policy_version 440578 (0.0007) [2023-12-27 08:41:23,311][20381] Updated weights for policy 1, policy_version 440233 (0.0011) [2023-12-27 08:41:23,367][20347] Updated weights for policy 0, policy_version 440588 (0.0008) [2023-12-27 08:41:23,425][20347] Updated weights for policy 0, policy_version 440598 (0.0010) [2023-12-27 08:41:23,486][20347] Updated weights for policy 0, policy_version 440608 (0.0010) [2023-12-27 08:41:23,986][20381] Updated weights for policy 1, policy_version 440243 (0.0009) [2023-12-27 08:41:24,046][20381] Updated weights for policy 1, policy_version 440253 (0.0007) [2023-12-27 08:41:24,099][20381] Updated weights for policy 1, policy_version 440263 (0.0009) [2023-12-27 08:41:24,155][20347] Updated weights for policy 0, policy_version 440618 (0.0008) [2023-12-27 08:41:24,204][20347] Updated weights for policy 0, policy_version 440628 (0.0010) [2023-12-27 08:41:24,249][20347] Updated weights for policy 0, policy_version 440638 (0.0010) [2023-12-27 08:41:24,660][20381] Updated weights for policy 1, policy_version 440273 (0.0006) [2023-12-27 08:41:24,730][20381] Updated weights for policy 1, policy_version 440283 (0.0011) [2023-12-27 08:41:24,780][20381] Updated weights for policy 1, policy_version 440293 (0.0011) [2023-12-27 08:41:24,829][20381] Updated weights for policy 1, policy_version 440303 (0.0011) [2023-12-27 08:41:24,905][20347] Updated weights for policy 0, policy_version 440648 (0.0006) [2023-12-27 08:41:24,971][20347] Updated weights for policy 0, policy_version 440658 (0.0007) [2023-12-27 08:41:25,018][20347] Updated weights for policy 0, policy_version 440668 (0.0007) [2023-12-27 08:41:25,494][20381] Updated weights for policy 1, policy_version 440313 (0.0010) [2023-12-27 08:41:25,548][20381] Updated weights for policy 1, policy_version 440323 (0.0010) [2023-12-27 08:41:25,585][20347] Updated weights for policy 0, policy_version 440678 (0.0006) [2023-12-27 08:41:25,610][20381] Updated weights for policy 1, policy_version 440333 (0.0011) [2023-12-27 08:41:25,631][20347] Updated weights for policy 0, policy_version 440688 (0.0005) [2023-12-27 08:41:25,680][20347] Updated weights for policy 0, policy_version 440698 (0.0005) [2023-12-27 08:41:26,289][20381] Updated weights for policy 1, policy_version 440343 (0.0007) [2023-12-27 08:41:26,317][20347] Updated weights for policy 0, policy_version 440708 (0.0006) [2023-12-27 08:41:26,340][20381] Updated weights for policy 1, policy_version 440353 (0.0006) [2023-12-27 08:41:26,370][20347] Updated weights for policy 0, policy_version 440718 (0.0005) [2023-12-27 08:41:26,392][20381] Updated weights for policy 1, policy_version 440363 (0.0007) [2023-12-27 08:41:26,421][20347] Updated weights for policy 0, policy_version 440728 (0.0005) [2023-12-27 08:41:26,986][20347] Updated weights for policy 0, policy_version 440738 (0.0006) [2023-12-27 08:41:27,034][20347] Updated weights for policy 0, policy_version 440748 (0.0008) [2023-12-27 08:41:27,082][20347] Updated weights for policy 0, policy_version 440758 (0.0010) [2023-12-27 08:41:27,084][20381] Updated weights for policy 1, policy_version 440373 (0.0009) [2023-12-27 08:41:27,133][20347] Updated weights for policy 0, policy_version 440768 (0.0010) [2023-12-27 08:41:27,138][20381] Updated weights for policy 1, policy_version 440383 (0.0007) [2023-12-27 08:41:27,187][20381] Updated weights for policy 1, policy_version 440393 (0.0007) [2023-12-27 08:41:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20077.3). Total num frames: 225615872. Throughput: 0: 10443.9, 1: 10290.2. Samples: 225599680. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:27,363][19170] Avg episode reward: [(0, '646.800'), (1, '896.520')] [2023-12-27 08:41:27,782][20347] Updated weights for policy 0, policy_version 440778 (0.0008) [2023-12-27 08:41:27,787][20381] Updated weights for policy 1, policy_version 440403 (0.0008) [2023-12-27 08:41:27,833][20347] Updated weights for policy 0, policy_version 440788 (0.0007) [2023-12-27 08:41:27,840][20381] Updated weights for policy 1, policy_version 440413 (0.0006) [2023-12-27 08:41:27,887][20381] Updated weights for policy 1, policy_version 440423 (0.0005) [2023-12-27 08:41:27,890][20347] Updated weights for policy 0, policy_version 440798 (0.0007) [2023-12-27 08:41:28,450][20381] Updated weights for policy 1, policy_version 440433 (0.0005) [2023-12-27 08:41:28,510][20381] Updated weights for policy 1, policy_version 440443 (0.0010) [2023-12-27 08:41:28,564][20381] Updated weights for policy 1, policy_version 440453 (0.0010) [2023-12-27 08:41:28,581][20347] Updated weights for policy 0, policy_version 440808 (0.0006) [2023-12-27 08:41:28,616][20381] Updated weights for policy 1, policy_version 440463 (0.0008) [2023-12-27 08:41:28,644][20347] Updated weights for policy 0, policy_version 440818 (0.0007) [2023-12-27 08:41:28,711][20347] Updated weights for policy 0, policy_version 440828 (0.0008) [2023-12-27 08:41:29,213][20381] Updated weights for policy 1, policy_version 440473 (0.0007) [2023-12-27 08:41:29,272][20381] Updated weights for policy 1, policy_version 440483 (0.0009) [2023-12-27 08:41:29,283][20347] Updated weights for policy 0, policy_version 440838 (0.0006) [2023-12-27 08:41:29,332][20381] Updated weights for policy 1, policy_version 440493 (0.0007) [2023-12-27 08:41:29,342][20347] Updated weights for policy 0, policy_version 440848 (0.0009) [2023-12-27 08:41:29,401][20347] Updated weights for policy 0, policy_version 440858 (0.0007) [2023-12-27 08:41:30,055][20381] Updated weights for policy 1, policy_version 440503 (0.0007) [2023-12-27 08:41:30,089][20347] Updated weights for policy 0, policy_version 440868 (0.0009) [2023-12-27 08:41:30,111][20381] Updated weights for policy 1, policy_version 440513 (0.0008) [2023-12-27 08:41:30,147][20347] Updated weights for policy 0, policy_version 440878 (0.0010) [2023-12-27 08:41:30,161][20381] Updated weights for policy 1, policy_version 440523 (0.0007) [2023-12-27 08:41:30,202][20347] Updated weights for policy 0, policy_version 440888 (0.0010) [2023-12-27 08:41:30,817][20347] Updated weights for policy 0, policy_version 440898 (0.0009) [2023-12-27 08:41:30,833][20381] Updated weights for policy 1, policy_version 440533 (0.0007) [2023-12-27 08:41:30,876][20347] Updated weights for policy 0, policy_version 440908 (0.0005) [2023-12-27 08:41:30,881][20381] Updated weights for policy 1, policy_version 440543 (0.0007) [2023-12-27 08:41:30,932][20347] Updated weights for policy 0, policy_version 440918 (0.0005) [2023-12-27 08:41:30,933][20381] Updated weights for policy 1, policy_version 440553 (0.0007) [2023-12-27 08:41:30,988][20347] Updated weights for policy 0, policy_version 440928 (0.0005) [2023-12-27 08:41:31,601][20347] Updated weights for policy 0, policy_version 440938 (0.0005) [2023-12-27 08:41:31,648][20381] Updated weights for policy 1, policy_version 440563 (0.0008) [2023-12-27 08:41:31,668][20347] Updated weights for policy 0, policy_version 440948 (0.0008) [2023-12-27 08:41:31,672][20292] KL-divergence is very high: 146.3917 [2023-12-27 08:41:31,700][20381] Updated weights for policy 1, policy_version 440573 (0.0006) [2023-12-27 08:41:31,712][20292] KL-divergence is very high: 122.5271 [2023-12-27 08:41:31,725][20347] Updated weights for policy 0, policy_version 440958 (0.0009) [2023-12-27 08:41:31,763][20381] Updated weights for policy 1, policy_version 440583 (0.0008) [2023-12-27 08:41:32,304][20347] Updated weights for policy 0, policy_version 440968 (0.0009) [2023-12-27 08:41:32,362][20347] Updated weights for policy 0, policy_version 440978 (0.0010) [2023-12-27 08:41:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20132.9). Total num frames: 225714176. Throughput: 0: 10548.6, 1: 10250.6. Samples: 225728544. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:32,363][19170] Avg episode reward: [(0, '778.320'), (1, '903.920')] [2023-12-27 08:41:32,423][20347] Updated weights for policy 0, policy_version 440988 (0.0009) [2023-12-27 08:41:32,486][20381] Updated weights for policy 1, policy_version 440593 (0.0008) [2023-12-27 08:41:32,540][20381] Updated weights for policy 1, policy_version 440603 (0.0009) [2023-12-27 08:41:32,599][20381] Updated weights for policy 1, policy_version 440613 (0.0009) [2023-12-27 08:41:32,650][20381] Updated weights for policy 1, policy_version 440623 (0.0008) [2023-12-27 08:41:33,033][20347] Updated weights for policy 0, policy_version 440998 (0.0007) [2023-12-27 08:41:33,090][20347] Updated weights for policy 0, policy_version 441008 (0.0005) [2023-12-27 08:41:33,146][20347] Updated weights for policy 0, policy_version 441018 (0.0005) [2023-12-27 08:41:33,388][20381] Updated weights for policy 1, policy_version 440633 (0.0007) [2023-12-27 08:41:33,448][20381] Updated weights for policy 1, policy_version 440643 (0.0011) [2023-12-27 08:41:33,508][20381] Updated weights for policy 1, policy_version 440653 (0.0009) [2023-12-27 08:41:33,760][20347] Updated weights for policy 0, policy_version 441028 (0.0007) [2023-12-27 08:41:33,819][20347] Updated weights for policy 0, policy_version 441038 (0.0010) [2023-12-27 08:41:33,877][20347] Updated weights for policy 0, policy_version 441048 (0.0007) [2023-12-27 08:41:34,104][20381] Updated weights for policy 1, policy_version 440663 (0.0009) [2023-12-27 08:41:34,164][20381] Updated weights for policy 1, policy_version 440673 (0.0011) [2023-12-27 08:41:34,232][20381] Updated weights for policy 1, policy_version 440683 (0.0011) [2023-12-27 08:41:34,519][20347] Updated weights for policy 0, policy_version 441058 (0.0009) [2023-12-27 08:41:34,578][20347] Updated weights for policy 0, policy_version 441068 (0.0010) [2023-12-27 08:41:34,631][20347] Updated weights for policy 0, policy_version 441078 (0.0010) [2023-12-27 08:41:34,682][20347] Updated weights for policy 0, policy_version 441088 (0.0008) [2023-12-27 08:41:34,927][20381] Updated weights for policy 1, policy_version 440693 (0.0008) [2023-12-27 08:41:34,991][20381] Updated weights for policy 1, policy_version 440703 (0.0005) [2023-12-27 08:41:35,052][20381] Updated weights for policy 1, policy_version 440713 (0.0007) [2023-12-27 08:41:35,378][20347] Updated weights for policy 0, policy_version 441098 (0.0010) [2023-12-27 08:41:35,433][20347] Updated weights for policy 0, policy_version 441108 (0.0010) [2023-12-27 08:41:35,487][20347] Updated weights for policy 0, policy_version 441118 (0.0010) [2023-12-27 08:41:35,599][20381] Updated weights for policy 1, policy_version 440723 (0.0006) [2023-12-27 08:41:35,655][20381] Updated weights for policy 1, policy_version 440733 (0.0008) [2023-12-27 08:41:35,705][20381] Updated weights for policy 1, policy_version 440743 (0.0007) [2023-12-27 08:41:36,148][20347] Updated weights for policy 0, policy_version 441128 (0.0010) [2023-12-27 08:41:36,202][20347] Updated weights for policy 0, policy_version 441138 (0.0009) [2023-12-27 08:41:36,248][20347] Updated weights for policy 0, policy_version 441148 (0.0010) [2023-12-27 08:41:36,415][20381] Updated weights for policy 1, policy_version 440753 (0.0007) [2023-12-27 08:41:36,472][20381] Updated weights for policy 1, policy_version 440763 (0.0005) [2023-12-27 08:41:36,534][20381] Updated weights for policy 1, policy_version 440773 (0.0006) [2023-12-27 08:41:36,589][20381] Updated weights for policy 1, policy_version 440783 (0.0008) [2023-12-27 08:41:36,963][20347] Updated weights for policy 0, policy_version 441158 (0.0011) [2023-12-27 08:41:37,018][20347] Updated weights for policy 0, policy_version 441168 (0.0010) [2023-12-27 08:41:37,074][20347] Updated weights for policy 0, policy_version 441178 (0.0010) [2023-12-27 08:41:37,136][20381] Updated weights for policy 1, policy_version 440793 (0.0007) [2023-12-27 08:41:37,188][20381] Updated weights for policy 1, policy_version 440803 (0.0007) [2023-12-27 08:41:37,243][20381] Updated weights for policy 1, policy_version 440813 (0.0008) [2023-12-27 08:41:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20271.7). Total num frames: 225828864. Throughput: 0: 10577.3, 1: 10202.0. Samples: 225792052. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:37,363][19170] Avg episode reward: [(0, '816.600'), (1, '924.130')] [2023-12-27 08:41:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000440816_112869376.pth... [2023-12-27 08:41:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000441184_112959488.pth... [2023-12-27 08:41:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000439600_112558080.pth [2023-12-27 08:41:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000439936_112640000.pth [2023-12-27 08:41:37,773][20347] Updated weights for policy 0, policy_version 441188 (0.0010) [2023-12-27 08:41:37,826][20347] Updated weights for policy 0, policy_version 441198 (0.0011) [2023-12-27 08:41:37,880][20347] Updated weights for policy 0, policy_version 441208 (0.0010) [2023-12-27 08:41:37,929][20381] Updated weights for policy 1, policy_version 440823 (0.0006) [2023-12-27 08:41:37,977][20381] Updated weights for policy 1, policy_version 440833 (0.0005) [2023-12-27 08:41:38,030][20381] Updated weights for policy 1, policy_version 440843 (0.0005) [2023-12-27 08:41:38,496][20347] Updated weights for policy 0, policy_version 441218 (0.0009) [2023-12-27 08:41:38,552][20347] Updated weights for policy 0, policy_version 441228 (0.0006) [2023-12-27 08:41:38,612][20347] Updated weights for policy 0, policy_version 441238 (0.0005) [2023-12-27 08:41:38,671][20347] Updated weights for policy 0, policy_version 441248 (0.0006) [2023-12-27 08:41:38,686][20381] Updated weights for policy 1, policy_version 440853 (0.0006) [2023-12-27 08:41:38,739][20381] Updated weights for policy 1, policy_version 440863 (0.0008) [2023-12-27 08:41:38,800][20381] Updated weights for policy 1, policy_version 440873 (0.0007) [2023-12-27 08:41:39,305][20347] Updated weights for policy 0, policy_version 441258 (0.0006) [2023-12-27 08:41:39,367][20347] Updated weights for policy 0, policy_version 441268 (0.0009) [2023-12-27 08:41:39,431][20347] Updated weights for policy 0, policy_version 441278 (0.0007) [2023-12-27 08:41:39,529][20381] Updated weights for policy 1, policy_version 440883 (0.0007) [2023-12-27 08:41:39,593][20381] Updated weights for policy 1, policy_version 440893 (0.0006) [2023-12-27 08:41:39,659][20381] Updated weights for policy 1, policy_version 440903 (0.0006) [2023-12-27 08:41:40,131][20347] Updated weights for policy 0, policy_version 441288 (0.0006) [2023-12-27 08:41:40,197][20347] Updated weights for policy 0, policy_version 441298 (0.0007) [2023-12-27 08:41:40,266][20347] Updated weights for policy 0, policy_version 441308 (0.0010) [2023-12-27 08:41:40,304][20381] Updated weights for policy 1, policy_version 440913 (0.0006) [2023-12-27 08:41:40,365][20381] Updated weights for policy 1, policy_version 440923 (0.0008) [2023-12-27 08:41:40,420][20381] Updated weights for policy 1, policy_version 440933 (0.0008) [2023-12-27 08:41:40,486][20381] Updated weights for policy 1, policy_version 440943 (0.0006) [2023-12-27 08:41:40,887][20347] Updated weights for policy 0, policy_version 441318 (0.0007) [2023-12-27 08:41:40,941][20347] Updated weights for policy 0, policy_version 441328 (0.0005) [2023-12-27 08:41:40,998][20347] Updated weights for policy 0, policy_version 441338 (0.0005) [2023-12-27 08:41:41,089][20381] Updated weights for policy 1, policy_version 440953 (0.0010) [2023-12-27 08:41:41,156][20381] Updated weights for policy 1, policy_version 440963 (0.0013) [2023-12-27 08:41:41,222][20381] Updated weights for policy 1, policy_version 440973 (0.0009) [2023-12-27 08:41:41,652][20347] Updated weights for policy 0, policy_version 441348 (0.0008) [2023-12-27 08:41:41,708][20347] Updated weights for policy 0, policy_version 441358 (0.0009) [2023-12-27 08:41:41,763][20347] Updated weights for policy 0, policy_version 441368 (0.0009) [2023-12-27 08:41:41,928][20381] Updated weights for policy 1, policy_version 440983 (0.0011) [2023-12-27 08:41:41,995][20381] Updated weights for policy 1, policy_version 440993 (0.0009) [2023-12-27 08:41:42,061][20381] Updated weights for policy 1, policy_version 441003 (0.0008) [2023-12-27 08:41:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20327.3). Total num frames: 225927168. Throughput: 0: 10602.3, 1: 10246.0. Samples: 225917840. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:42,363][19170] Avg episode reward: [(0, '747.910'), (1, '951.830')] [2023-12-27 08:41:42,499][20347] Updated weights for policy 0, policy_version 441378 (0.0009) [2023-12-27 08:41:42,551][20347] Updated weights for policy 0, policy_version 441388 (0.0009) [2023-12-27 08:41:42,602][20347] Updated weights for policy 0, policy_version 441398 (0.0007) [2023-12-27 08:41:42,666][20347] Updated weights for policy 0, policy_version 441408 (0.0007) [2023-12-27 08:41:42,668][20381] Updated weights for policy 1, policy_version 441013 (0.0009) [2023-12-27 08:41:42,714][20381] Updated weights for policy 1, policy_version 441023 (0.0010) [2023-12-27 08:41:42,769][20381] Updated weights for policy 1, policy_version 441033 (0.0010) [2023-12-27 08:41:43,383][20347] Updated weights for policy 0, policy_version 441418 (0.0008) [2023-12-27 08:41:43,428][20347] Updated weights for policy 0, policy_version 441428 (0.0010) [2023-12-27 08:41:43,472][20381] Updated weights for policy 1, policy_version 441043 (0.0011) [2023-12-27 08:41:43,476][20347] Updated weights for policy 0, policy_version 441438 (0.0010) [2023-12-27 08:41:43,530][20381] Updated weights for policy 1, policy_version 441053 (0.0011) [2023-12-27 08:41:43,584][20381] Updated weights for policy 1, policy_version 441063 (0.0011) [2023-12-27 08:41:44,129][20347] Updated weights for policy 0, policy_version 441448 (0.0006) [2023-12-27 08:41:44,184][20347] Updated weights for policy 0, policy_version 441458 (0.0005) [2023-12-27 08:41:44,250][20347] Updated weights for policy 0, policy_version 441468 (0.0009) [2023-12-27 08:41:44,260][20381] Updated weights for policy 1, policy_version 441073 (0.0010) [2023-12-27 08:41:44,312][20381] Updated weights for policy 1, policy_version 441083 (0.0005) [2023-12-27 08:41:44,366][20381] Updated weights for policy 1, policy_version 441093 (0.0008) [2023-12-27 08:41:44,417][20381] Updated weights for policy 1, policy_version 441103 (0.0005) [2023-12-27 08:41:44,829][20347] Updated weights for policy 0, policy_version 441478 (0.0010) [2023-12-27 08:41:44,886][20347] Updated weights for policy 0, policy_version 441488 (0.0011) [2023-12-27 08:41:44,939][20347] Updated weights for policy 0, policy_version 441498 (0.0010) [2023-12-27 08:41:45,030][20381] Updated weights for policy 1, policy_version 441113 (0.0008) [2023-12-27 08:41:45,093][20381] Updated weights for policy 1, policy_version 441123 (0.0009) [2023-12-27 08:41:45,155][20381] Updated weights for policy 1, policy_version 441133 (0.0011) [2023-12-27 08:41:45,629][20347] Updated weights for policy 0, policy_version 441508 (0.0011) [2023-12-27 08:41:45,683][20347] Updated weights for policy 0, policy_version 441518 (0.0010) [2023-12-27 08:41:45,741][20347] Updated weights for policy 0, policy_version 441528 (0.0010) [2023-12-27 08:41:45,787][20381] Updated weights for policy 1, policy_version 441143 (0.0007) [2023-12-27 08:41:45,848][20381] Updated weights for policy 1, policy_version 441153 (0.0008) [2023-12-27 08:41:45,901][20381] Updated weights for policy 1, policy_version 441163 (0.0007) [2023-12-27 08:41:46,354][20347] Updated weights for policy 0, policy_version 441538 (0.0009) [2023-12-27 08:41:46,412][20347] Updated weights for policy 0, policy_version 441548 (0.0006) [2023-12-27 08:41:46,473][20347] Updated weights for policy 0, policy_version 441558 (0.0010) [2023-12-27 08:41:46,531][20347] Updated weights for policy 0, policy_version 441568 (0.0010) [2023-12-27 08:41:46,569][20381] Updated weights for policy 1, policy_version 441173 (0.0008) [2023-12-27 08:41:46,626][20381] Updated weights for policy 1, policy_version 441183 (0.0009) [2023-12-27 08:41:46,689][20381] Updated weights for policy 1, policy_version 441193 (0.0006) [2023-12-27 08:41:47,080][20347] Updated weights for policy 0, policy_version 441578 (0.0009) [2023-12-27 08:41:47,138][20347] Updated weights for policy 0, policy_version 441588 (0.0008) [2023-12-27 08:41:47,194][20347] Updated weights for policy 0, policy_version 441598 (0.0007) [2023-12-27 08:41:47,215][20381] Updated weights for policy 1, policy_version 441203 (0.0005) [2023-12-27 08:41:47,272][20381] Updated weights for policy 1, policy_version 441213 (0.0005) [2023-12-27 08:41:47,322][20381] Updated weights for policy 1, policy_version 441223 (0.0005) [2023-12-27 08:41:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20410.6). Total num frames: 226033664. Throughput: 0: 10622.6, 1: 10267.3. Samples: 226045484. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:47,363][19170] Avg episode reward: [(0, '777.820'), (1, '980.640')] [2023-12-27 08:41:47,838][20347] Updated weights for policy 0, policy_version 441608 (0.0006) [2023-12-27 08:41:47,894][20381] Updated weights for policy 1, policy_version 441233 (0.0006) [2023-12-27 08:41:47,895][20347] Updated weights for policy 0, policy_version 441618 (0.0008) [2023-12-27 08:41:47,943][20347] Updated weights for policy 0, policy_version 441628 (0.0007) [2023-12-27 08:41:47,948][20381] Updated weights for policy 1, policy_version 441243 (0.0008) [2023-12-27 08:41:48,009][20381] Updated weights for policy 1, policy_version 441253 (0.0010) [2023-12-27 08:41:48,064][20381] Updated weights for policy 1, policy_version 441263 (0.0008) [2023-12-27 08:41:48,676][20347] Updated weights for policy 0, policy_version 441638 (0.0005) [2023-12-27 08:41:48,682][20381] Updated weights for policy 1, policy_version 441273 (0.0011) [2023-12-27 08:41:48,732][20347] Updated weights for policy 0, policy_version 441648 (0.0005) [2023-12-27 08:41:48,734][20381] Updated weights for policy 1, policy_version 441283 (0.0010) [2023-12-27 08:41:48,783][20381] Updated weights for policy 1, policy_version 441293 (0.0010) [2023-12-27 08:41:48,792][20347] Updated weights for policy 0, policy_version 441658 (0.0006) [2023-12-27 08:41:49,443][20381] Updated weights for policy 1, policy_version 441303 (0.0010) [2023-12-27 08:41:49,502][20381] Updated weights for policy 1, policy_version 441313 (0.0010) [2023-12-27 08:41:49,511][20347] Updated weights for policy 0, policy_version 441668 (0.0008) [2023-12-27 08:41:49,554][20381] Updated weights for policy 1, policy_version 441323 (0.0010) [2023-12-27 08:41:49,569][20347] Updated weights for policy 0, policy_version 441678 (0.0010) [2023-12-27 08:41:49,631][20347] Updated weights for policy 0, policy_version 441688 (0.0010) [2023-12-27 08:41:50,316][20347] Updated weights for policy 0, policy_version 441698 (0.0010) [2023-12-27 08:41:50,328][20381] Updated weights for policy 1, policy_version 441333 (0.0009) [2023-12-27 08:41:50,372][20347] Updated weights for policy 0, policy_version 441708 (0.0010) [2023-12-27 08:41:50,385][20381] Updated weights for policy 1, policy_version 441343 (0.0005) [2023-12-27 08:41:50,428][20347] Updated weights for policy 0, policy_version 441718 (0.0010) [2023-12-27 08:41:50,445][20381] Updated weights for policy 1, policy_version 441353 (0.0005) [2023-12-27 08:41:50,480][20347] Updated weights for policy 0, policy_version 441728 (0.0010) [2023-12-27 08:41:51,158][20381] Updated weights for policy 1, policy_version 441363 (0.0006) [2023-12-27 08:41:51,194][20347] Updated weights for policy 0, policy_version 441738 (0.0007) [2023-12-27 08:41:51,210][20381] Updated weights for policy 1, policy_version 441373 (0.0005) [2023-12-27 08:41:51,254][20347] Updated weights for policy 0, policy_version 441748 (0.0007) [2023-12-27 08:41:51,273][20381] Updated weights for policy 1, policy_version 441383 (0.0008) [2023-12-27 08:41:51,315][20347] Updated weights for policy 0, policy_version 441758 (0.0005) [2023-12-27 08:41:51,927][20381] Updated weights for policy 1, policy_version 441393 (0.0008) [2023-12-27 08:41:51,989][20381] Updated weights for policy 1, policy_version 441403 (0.0008) [2023-12-27 08:41:52,034][20347] Updated weights for policy 0, policy_version 441768 (0.0006) [2023-12-27 08:41:52,047][20381] Updated weights for policy 1, policy_version 441413 (0.0005) [2023-12-27 08:41:52,088][20347] Updated weights for policy 0, policy_version 441778 (0.0007) [2023-12-27 08:41:52,109][20381] Updated weights for policy 1, policy_version 441423 (0.0006) [2023-12-27 08:41:52,150][20347] Updated weights for policy 0, policy_version 441788 (0.0007) [2023-12-27 08:41:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20466.1). Total num frames: 226140160. Throughput: 0: 10561.2, 1: 10312.2. Samples: 226107444. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:52,363][19170] Avg episode reward: [(0, '758.660'), (1, '951.620')] [2023-12-27 08:41:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000441424_113025024.pth... [2023-12-27 08:41:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000441792_113115136.pth... [2023-12-27 08:41:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000440208_112713728.pth [2023-12-27 08:41:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000440544_112795648.pth [2023-12-27 08:41:52,742][20381] Updated weights for policy 1, policy_version 441433 (0.0007) [2023-12-27 08:41:52,799][20381] Updated weights for policy 1, policy_version 441443 (0.0008) [2023-12-27 08:41:52,855][20381] Updated weights for policy 1, policy_version 441453 (0.0007) [2023-12-27 08:41:52,882][20347] Updated weights for policy 0, policy_version 441798 (0.0008) [2023-12-27 08:41:52,939][20347] Updated weights for policy 0, policy_version 441808 (0.0007) [2023-12-27 08:41:52,989][20347] Updated weights for policy 0, policy_version 441818 (0.0008) [2023-12-27 08:41:53,498][20381] Updated weights for policy 1, policy_version 441463 (0.0006) [2023-12-27 08:41:53,550][20381] Updated weights for policy 1, policy_version 441473 (0.0009) [2023-12-27 08:41:53,604][20381] Updated weights for policy 1, policy_version 441483 (0.0011) [2023-12-27 08:41:53,701][20347] Updated weights for policy 0, policy_version 441828 (0.0008) [2023-12-27 08:41:53,750][20347] Updated weights for policy 0, policy_version 441838 (0.0007) [2023-12-27 08:41:53,808][20347] Updated weights for policy 0, policy_version 441848 (0.0007) [2023-12-27 08:41:54,259][20381] Updated weights for policy 1, policy_version 441493 (0.0010) [2023-12-27 08:41:54,311][20381] Updated weights for policy 1, policy_version 441503 (0.0011) [2023-12-27 08:41:54,381][20381] Updated weights for policy 1, policy_version 441513 (0.0011) [2023-12-27 08:41:54,535][20347] Updated weights for policy 0, policy_version 441858 (0.0006) [2023-12-27 08:41:54,587][20347] Updated weights for policy 0, policy_version 441868 (0.0007) [2023-12-27 08:41:54,641][20347] Updated weights for policy 0, policy_version 441878 (0.0008) [2023-12-27 08:41:54,692][20347] Updated weights for policy 0, policy_version 441888 (0.0007) [2023-12-27 08:41:55,077][20381] Updated weights for policy 1, policy_version 441523 (0.0011) [2023-12-27 08:41:55,133][20381] Updated weights for policy 1, policy_version 441533 (0.0011) [2023-12-27 08:41:55,187][20381] Updated weights for policy 1, policy_version 441543 (0.0011) [2023-12-27 08:41:55,421][20347] Updated weights for policy 0, policy_version 441898 (0.0007) [2023-12-27 08:41:55,476][20347] Updated weights for policy 0, policy_version 441908 (0.0007) [2023-12-27 08:41:55,531][20347] Updated weights for policy 0, policy_version 441918 (0.0006) [2023-12-27 08:41:55,854][20381] Updated weights for policy 1, policy_version 441553 (0.0010) [2023-12-27 08:41:55,911][20381] Updated weights for policy 1, policy_version 441563 (0.0005) [2023-12-27 08:41:55,960][20381] Updated weights for policy 1, policy_version 441573 (0.0008) [2023-12-27 08:41:56,009][20381] Updated weights for policy 1, policy_version 441583 (0.0008) [2023-12-27 08:41:56,210][20347] Updated weights for policy 0, policy_version 441928 (0.0007) [2023-12-27 08:41:56,263][20347] Updated weights for policy 0, policy_version 441938 (0.0008) [2023-12-27 08:41:56,314][20347] Updated weights for policy 0, policy_version 441948 (0.0007) [2023-12-27 08:41:56,656][20381] Updated weights for policy 1, policy_version 441593 (0.0006) [2023-12-27 08:41:56,714][20381] Updated weights for policy 1, policy_version 441603 (0.0005) [2023-12-27 08:41:56,777][20381] Updated weights for policy 1, policy_version 441613 (0.0005) [2023-12-27 08:41:57,015][20347] Updated weights for policy 0, policy_version 441958 (0.0008) [2023-12-27 08:41:57,069][20347] Updated weights for policy 0, policy_version 441968 (0.0007) [2023-12-27 08:41:57,126][20347] Updated weights for policy 0, policy_version 441978 (0.0008) [2023-12-27 08:41:57,325][20381] Updated weights for policy 1, policy_version 441623 (0.0007) [2023-12-27 08:41:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 226238464. Throughput: 0: 10430.2, 1: 10340.2. Samples: 226228204. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:41:57,363][19170] Avg episode reward: [(0, '807.070'), (1, '951.710')] [2023-12-27 08:41:57,388][20381] Updated weights for policy 1, policy_version 441633 (0.0007) [2023-12-27 08:41:57,438][20381] Updated weights for policy 1, policy_version 441643 (0.0007) [2023-12-27 08:41:57,815][20347] Updated weights for policy 0, policy_version 441988 (0.0007) [2023-12-27 08:41:57,862][20347] Updated weights for policy 0, policy_version 441998 (0.0007) [2023-12-27 08:41:57,913][20347] Updated weights for policy 0, policy_version 442008 (0.0005) [2023-12-27 08:41:58,074][20381] Updated weights for policy 1, policy_version 441653 (0.0007) [2023-12-27 08:41:58,131][20381] Updated weights for policy 1, policy_version 441663 (0.0008) [2023-12-27 08:41:58,195][20381] Updated weights for policy 1, policy_version 441673 (0.0007) [2023-12-27 08:41:58,629][20347] Updated weights for policy 0, policy_version 442018 (0.0006) [2023-12-27 08:41:58,689][20347] Updated weights for policy 0, policy_version 442028 (0.0008) [2023-12-27 08:41:58,748][20347] Updated weights for policy 0, policy_version 442038 (0.0007) [2023-12-27 08:41:58,814][20347] Updated weights for policy 0, policy_version 442048 (0.0007) [2023-12-27 08:41:58,949][20381] Updated weights for policy 1, policy_version 441683 (0.0008) [2023-12-27 08:41:59,004][20381] Updated weights for policy 1, policy_version 441693 (0.0010) [2023-12-27 08:41:59,063][20381] Updated weights for policy 1, policy_version 441703 (0.0010) [2023-12-27 08:41:59,643][20347] Updated weights for policy 0, policy_version 442058 (0.0007) [2023-12-27 08:41:59,691][20347] Updated weights for policy 0, policy_version 442068 (0.0007) [2023-12-27 08:41:59,747][20347] Updated weights for policy 0, policy_version 442078 (0.0008) [2023-12-27 08:41:59,765][20381] Updated weights for policy 1, policy_version 441713 (0.0008) [2023-12-27 08:41:59,817][20381] Updated weights for policy 1, policy_version 441723 (0.0007) [2023-12-27 08:41:59,880][20381] Updated weights for policy 1, policy_version 441733 (0.0008) [2023-12-27 08:41:59,943][20381] Updated weights for policy 1, policy_version 441743 (0.0008) [2023-12-27 08:42:00,414][20347] Updated weights for policy 0, policy_version 442088 (0.0008) [2023-12-27 08:42:00,476][20347] Updated weights for policy 0, policy_version 442098 (0.0007) [2023-12-27 08:42:00,530][20347] Updated weights for policy 0, policy_version 442108 (0.0007) [2023-12-27 08:42:00,578][20381] Updated weights for policy 1, policy_version 441753 (0.0008) [2023-12-27 08:42:00,641][20381] Updated weights for policy 1, policy_version 441763 (0.0005) [2023-12-27 08:42:00,702][20381] Updated weights for policy 1, policy_version 441773 (0.0005) [2023-12-27 08:42:01,245][20347] Updated weights for policy 0, policy_version 442118 (0.0008) [2023-12-27 08:42:01,303][20347] Updated weights for policy 0, policy_version 442128 (0.0008) [2023-12-27 08:42:01,321][20381] Updated weights for policy 1, policy_version 441783 (0.0007) [2023-12-27 08:42:01,365][20347] Updated weights for policy 0, policy_version 442138 (0.0007) [2023-12-27 08:42:01,377][20381] Updated weights for policy 1, policy_version 441793 (0.0008) [2023-12-27 08:42:01,431][20381] Updated weights for policy 1, policy_version 441803 (0.0008) [2023-12-27 08:42:02,058][20347] Updated weights for policy 0, policy_version 442148 (0.0006) [2023-12-27 08:42:02,108][20347] Updated weights for policy 0, policy_version 442158 (0.0005) [2023-12-27 08:42:02,160][20347] Updated weights for policy 0, policy_version 442168 (0.0006) [2023-12-27 08:42:02,186][20381] Updated weights for policy 1, policy_version 441813 (0.0008) [2023-12-27 08:42:02,241][20381] Updated weights for policy 1, policy_version 441823 (0.0007) [2023-12-27 08:42:02,304][20381] Updated weights for policy 1, policy_version 441833 (0.0008) [2023-12-27 08:42:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 226344960. Throughput: 0: 10353.5, 1: 10405.4. Samples: 226349996. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:02,363][19170] Avg episode reward: [(0, '865.220'), (1, '951.900')] [2023-12-27 08:42:02,770][20347] Updated weights for policy 0, policy_version 442178 (0.0007) [2023-12-27 08:42:02,852][20347] Updated weights for policy 0, policy_version 442188 (0.0007) [2023-12-27 08:42:02,909][20347] Updated weights for policy 0, policy_version 442198 (0.0006) [2023-12-27 08:42:02,966][20347] Updated weights for policy 0, policy_version 442208 (0.0007) [2023-12-27 08:42:03,022][20381] Updated weights for policy 1, policy_version 441843 (0.0007) [2023-12-27 08:42:03,066][20381] Updated weights for policy 1, policy_version 441853 (0.0007) [2023-12-27 08:42:03,114][20381] Updated weights for policy 1, policy_version 441863 (0.0007) [2023-12-27 08:42:03,608][20347] Updated weights for policy 0, policy_version 442218 (0.0007) [2023-12-27 08:42:03,662][20347] Updated weights for policy 0, policy_version 442228 (0.0007) [2023-12-27 08:42:03,715][20347] Updated weights for policy 0, policy_version 442238 (0.0007) [2023-12-27 08:42:03,788][20381] Updated weights for policy 1, policy_version 441873 (0.0008) [2023-12-27 08:42:03,838][20381] Updated weights for policy 1, policy_version 441883 (0.0010) [2023-12-27 08:42:03,897][20381] Updated weights for policy 1, policy_version 441893 (0.0010) [2023-12-27 08:42:03,945][20381] Updated weights for policy 1, policy_version 441903 (0.0010) [2023-12-27 08:42:04,419][20347] Updated weights for policy 0, policy_version 442248 (0.0007) [2023-12-27 08:42:04,481][20347] Updated weights for policy 0, policy_version 442258 (0.0007) [2023-12-27 08:42:04,541][20347] Updated weights for policy 0, policy_version 442268 (0.0008) [2023-12-27 08:42:04,661][20381] Updated weights for policy 1, policy_version 441913 (0.0010) [2023-12-27 08:42:04,714][20381] Updated weights for policy 1, policy_version 441923 (0.0011) [2023-12-27 08:42:04,762][20381] Updated weights for policy 1, policy_version 441933 (0.0010) [2023-12-27 08:42:05,228][20347] Updated weights for policy 0, policy_version 442278 (0.0008) [2023-12-27 08:42:05,284][20347] Updated weights for policy 0, policy_version 442288 (0.0007) [2023-12-27 08:42:05,339][20347] Updated weights for policy 0, policy_version 442298 (0.0007) [2023-12-27 08:42:05,458][20381] Updated weights for policy 1, policy_version 441943 (0.0011) [2023-12-27 08:42:05,520][20381] Updated weights for policy 1, policy_version 441953 (0.0011) [2023-12-27 08:42:05,579][20381] Updated weights for policy 1, policy_version 441963 (0.0011) [2023-12-27 08:42:06,061][20347] Updated weights for policy 0, policy_version 442308 (0.0006) [2023-12-27 08:42:06,119][20347] Updated weights for policy 0, policy_version 442318 (0.0007) [2023-12-27 08:42:06,168][20381] Updated weights for policy 1, policy_version 441973 (0.0010) [2023-12-27 08:42:06,179][20347] Updated weights for policy 0, policy_version 442328 (0.0007) [2023-12-27 08:42:06,226][20381] Updated weights for policy 1, policy_version 441983 (0.0010) [2023-12-27 08:42:06,285][20381] Updated weights for policy 1, policy_version 441993 (0.0011) [2023-12-27 08:42:06,852][20347] Updated weights for policy 0, policy_version 442338 (0.0007) [2023-12-27 08:42:06,910][20347] Updated weights for policy 0, policy_version 442348 (0.0005) [2023-12-27 08:42:06,973][20347] Updated weights for policy 0, policy_version 442358 (0.0005) [2023-12-27 08:42:06,979][20381] Updated weights for policy 1, policy_version 442003 (0.0011) [2023-12-27 08:42:07,033][20347] Updated weights for policy 0, policy_version 442368 (0.0005) [2023-12-27 08:42:07,034][20381] Updated weights for policy 1, policy_version 442013 (0.0011) [2023-12-27 08:42:07,096][20381] Updated weights for policy 1, policy_version 442023 (0.0011) [2023-12-27 08:42:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 226443264. Throughput: 0: 10345.0, 1: 10433.5. Samples: 226411124. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:07,363][19170] Avg episode reward: [(0, '913.370'), (1, '971.100')] [2023-12-27 08:42:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000442032_113180672.pth... [2023-12-27 08:42:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000442368_113262592.pth... [2023-12-27 08:42:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000440816_112869376.pth [2023-12-27 08:42:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000441184_112959488.pth [2023-12-27 08:42:07,650][20347] Updated weights for policy 0, policy_version 442378 (0.0010) [2023-12-27 08:42:07,710][20347] Updated weights for policy 0, policy_version 442388 (0.0011) [2023-12-27 08:42:07,756][20347] Updated weights for policy 0, policy_version 442398 (0.0010) [2023-12-27 08:42:07,761][20381] Updated weights for policy 1, policy_version 442033 (0.0011) [2023-12-27 08:42:07,813][20381] Updated weights for policy 1, policy_version 442043 (0.0011) [2023-12-27 08:42:07,861][20381] Updated weights for policy 1, policy_version 442053 (0.0010) [2023-12-27 08:42:07,917][20381] Updated weights for policy 1, policy_version 442063 (0.0011) [2023-12-27 08:42:08,426][20347] Updated weights for policy 0, policy_version 442408 (0.0007) [2023-12-27 08:42:08,487][20347] Updated weights for policy 0, policy_version 442418 (0.0007) [2023-12-27 08:42:08,541][20347] Updated weights for policy 0, policy_version 442428 (0.0007) [2023-12-27 08:42:08,595][20381] Updated weights for policy 1, policy_version 442073 (0.0007) [2023-12-27 08:42:08,652][20381] Updated weights for policy 1, policy_version 442083 (0.0008) [2023-12-27 08:42:08,711][20381] Updated weights for policy 1, policy_version 442093 (0.0011) [2023-12-27 08:42:09,212][20347] Updated weights for policy 0, policy_version 442438 (0.0007) [2023-12-27 08:42:09,274][20347] Updated weights for policy 0, policy_version 442448 (0.0007) [2023-12-27 08:42:09,337][20347] Updated weights for policy 0, policy_version 442458 (0.0007) [2023-12-27 08:42:09,403][20381] Updated weights for policy 1, policy_version 442103 (0.0009) [2023-12-27 08:42:09,464][20381] Updated weights for policy 1, policy_version 442113 (0.0008) [2023-12-27 08:42:09,525][20381] Updated weights for policy 1, policy_version 442123 (0.0008) [2023-12-27 08:42:10,071][20347] Updated weights for policy 0, policy_version 442468 (0.0006) [2023-12-27 08:42:10,120][20347] Updated weights for policy 0, policy_version 442478 (0.0006) [2023-12-27 08:42:10,170][20347] Updated weights for policy 0, policy_version 442488 (0.0007) [2023-12-27 08:42:10,297][20381] Updated weights for policy 1, policy_version 442133 (0.0009) [2023-12-27 08:42:10,359][20381] Updated weights for policy 1, policy_version 442143 (0.0006) [2023-12-27 08:42:10,413][20381] Updated weights for policy 1, policy_version 442153 (0.0007) [2023-12-27 08:42:10,883][20347] Updated weights for policy 0, policy_version 442498 (0.0006) [2023-12-27 08:42:10,944][20347] Updated weights for policy 0, policy_version 442508 (0.0010) [2023-12-27 08:42:10,994][20347] Updated weights for policy 0, policy_version 442518 (0.0010) [2023-12-27 08:42:11,043][20381] Updated weights for policy 1, policy_version 442163 (0.0007) [2023-12-27 08:42:11,054][20347] Updated weights for policy 0, policy_version 442528 (0.0009) [2023-12-27 08:42:11,102][20381] Updated weights for policy 1, policy_version 442173 (0.0011) [2023-12-27 08:42:11,162][20381] Updated weights for policy 1, policy_version 442183 (0.0011) [2023-12-27 08:42:11,803][20347] Updated weights for policy 0, policy_version 442538 (0.0007) [2023-12-27 08:42:11,872][20347] Updated weights for policy 0, policy_version 442548 (0.0007) [2023-12-27 08:42:11,908][20381] Updated weights for policy 1, policy_version 442193 (0.0010) [2023-12-27 08:42:11,927][20347] Updated weights for policy 0, policy_version 442558 (0.0008) [2023-12-27 08:42:11,966][20381] Updated weights for policy 1, policy_version 442203 (0.0006) [2023-12-27 08:42:12,024][20381] Updated weights for policy 1, policy_version 442213 (0.0006) [2023-12-27 08:42:12,080][20381] Updated weights for policy 1, policy_version 442223 (0.0008) [2023-12-27 08:42:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 226541568. Throughput: 0: 10311.7, 1: 10416.6. Samples: 226532456. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:12,363][19170] Avg episode reward: [(0, '951.990'), (1, '1000.000')] [2023-12-27 08:42:12,586][20347] Updated weights for policy 0, policy_version 442568 (0.0006) [2023-12-27 08:42:12,647][20347] Updated weights for policy 0, policy_version 442578 (0.0007) [2023-12-27 08:42:12,708][20347] Updated weights for policy 0, policy_version 442588 (0.0005) [2023-12-27 08:42:12,784][20381] Updated weights for policy 1, policy_version 442233 (0.0007) [2023-12-27 08:42:12,837][20381] Updated weights for policy 1, policy_version 442243 (0.0009) [2023-12-27 08:42:12,886][20381] Updated weights for policy 1, policy_version 442253 (0.0010) [2023-12-27 08:42:13,289][20347] Updated weights for policy 0, policy_version 442598 (0.0007) [2023-12-27 08:42:13,341][20347] Updated weights for policy 0, policy_version 442608 (0.0008) [2023-12-27 08:42:13,403][20347] Updated weights for policy 0, policy_version 442618 (0.0010) [2023-12-27 08:42:13,570][20381] Updated weights for policy 1, policy_version 442263 (0.0008) [2023-12-27 08:42:13,615][20381] Updated weights for policy 1, policy_version 442273 (0.0008) [2023-12-27 08:42:13,671][20381] Updated weights for policy 1, policy_version 442283 (0.0007) [2023-12-27 08:42:14,013][20347] Updated weights for policy 0, policy_version 442628 (0.0009) [2023-12-27 08:42:14,072][20347] Updated weights for policy 0, policy_version 442638 (0.0007) [2023-12-27 08:42:14,135][20347] Updated weights for policy 0, policy_version 442648 (0.0006) [2023-12-27 08:42:14,278][20381] Updated weights for policy 1, policy_version 442293 (0.0006) [2023-12-27 08:42:14,329][20381] Updated weights for policy 1, policy_version 442303 (0.0007) [2023-12-27 08:42:14,380][20381] Updated weights for policy 1, policy_version 442313 (0.0007) [2023-12-27 08:42:14,780][20347] Updated weights for policy 0, policy_version 442658 (0.0006) [2023-12-27 08:42:14,843][20347] Updated weights for policy 0, policy_version 442668 (0.0010) [2023-12-27 08:42:14,902][20347] Updated weights for policy 0, policy_version 442678 (0.0008) [2023-12-27 08:42:14,965][20347] Updated weights for policy 0, policy_version 442688 (0.0007) [2023-12-27 08:42:15,091][20381] Updated weights for policy 1, policy_version 442323 (0.0007) [2023-12-27 08:42:15,149][20381] Updated weights for policy 1, policy_version 442333 (0.0007) [2023-12-27 08:42:15,206][20381] Updated weights for policy 1, policy_version 442343 (0.0009) [2023-12-27 08:42:15,644][20347] Updated weights for policy 0, policy_version 442698 (0.0010) [2023-12-27 08:42:15,699][20347] Updated weights for policy 0, policy_version 442708 (0.0010) [2023-12-27 08:42:15,757][20347] Updated weights for policy 0, policy_version 442718 (0.0010) [2023-12-27 08:42:15,875][20381] Updated weights for policy 1, policy_version 442353 (0.0007) [2023-12-27 08:42:15,922][20381] Updated weights for policy 1, policy_version 442363 (0.0010) [2023-12-27 08:42:15,967][20381] Updated weights for policy 1, policy_version 442373 (0.0010) [2023-12-27 08:42:16,014][20381] Updated weights for policy 1, policy_version 442383 (0.0009) [2023-12-27 08:42:16,421][20347] Updated weights for policy 0, policy_version 442728 (0.0010) [2023-12-27 08:42:16,478][20347] Updated weights for policy 0, policy_version 442738 (0.0009) [2023-12-27 08:42:16,533][20347] Updated weights for policy 0, policy_version 442748 (0.0006) [2023-12-27 08:42:16,609][20381] Updated weights for policy 1, policy_version 442393 (0.0007) [2023-12-27 08:42:16,662][20381] Updated weights for policy 1, policy_version 442403 (0.0009) [2023-12-27 08:42:16,706][20381] Updated weights for policy 1, policy_version 442413 (0.0008) [2023-12-27 08:42:17,241][20347] Updated weights for policy 0, policy_version 442758 (0.0007) [2023-12-27 08:42:17,285][20347] Updated weights for policy 0, policy_version 442768 (0.0008) [2023-12-27 08:42:17,332][20347] Updated weights for policy 0, policy_version 442778 (0.0007) [2023-12-27 08:42:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 226639872. Throughput: 0: 10237.3, 1: 10400.4. Samples: 226657236. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:17,363][19170] Avg episode reward: [(0, '990.330'), (1, '1000.000')] [2023-12-27 08:42:17,389][20381] Updated weights for policy 1, policy_version 442423 (0.0010) [2023-12-27 08:42:17,447][20381] Updated weights for policy 1, policy_version 442433 (0.0010) [2023-12-27 08:42:17,498][20381] Updated weights for policy 1, policy_version 442443 (0.0008) [2023-12-27 08:42:18,045][20347] Updated weights for policy 0, policy_version 442788 (0.0006) [2023-12-27 08:42:18,098][20347] Updated weights for policy 0, policy_version 442798 (0.0007) [2023-12-27 08:42:18,110][20381] Updated weights for policy 1, policy_version 442453 (0.0005) [2023-12-27 08:42:18,148][20347] Updated weights for policy 0, policy_version 442808 (0.0007) [2023-12-27 08:42:18,163][20381] Updated weights for policy 1, policy_version 442463 (0.0006) [2023-12-27 08:42:18,212][20381] Updated weights for policy 1, policy_version 442473 (0.0008) [2023-12-27 08:42:18,875][20347] Updated weights for policy 0, policy_version 442818 (0.0008) [2023-12-27 08:42:18,877][20381] Updated weights for policy 1, policy_version 442483 (0.0010) [2023-12-27 08:42:18,930][20347] Updated weights for policy 0, policy_version 442828 (0.0005) [2023-12-27 08:42:18,932][20381] Updated weights for policy 1, policy_version 442493 (0.0010) [2023-12-27 08:42:18,987][20381] Updated weights for policy 1, policy_version 442503 (0.0010) [2023-12-27 08:42:18,992][20347] Updated weights for policy 0, policy_version 442838 (0.0006) [2023-12-27 08:42:19,046][20347] Updated weights for policy 0, policy_version 442848 (0.0008) [2023-12-27 08:42:19,673][20381] Updated weights for policy 1, policy_version 442513 (0.0010) [2023-12-27 08:42:19,736][20381] Updated weights for policy 1, policy_version 442523 (0.0007) [2023-12-27 08:42:19,789][20347] Updated weights for policy 0, policy_version 442858 (0.0011) [2023-12-27 08:42:19,798][20381] Updated weights for policy 1, policy_version 442533 (0.0006) [2023-12-27 08:42:19,846][20347] Updated weights for policy 0, policy_version 442868 (0.0009) [2023-12-27 08:42:19,857][20381] Updated weights for policy 1, policy_version 442543 (0.0008) [2023-12-27 08:42:19,902][20347] Updated weights for policy 0, policy_version 442878 (0.0008) [2023-12-27 08:42:20,496][20381] Updated weights for policy 1, policy_version 442553 (0.0007) [2023-12-27 08:42:20,546][20381] Updated weights for policy 1, policy_version 442563 (0.0007) [2023-12-27 08:42:20,609][20381] Updated weights for policy 1, policy_version 442573 (0.0008) [2023-12-27 08:42:20,638][20347] Updated weights for policy 0, policy_version 442888 (0.0010) [2023-12-27 08:42:20,701][20347] Updated weights for policy 0, policy_version 442898 (0.0007) [2023-12-27 08:42:20,770][20347] Updated weights for policy 0, policy_version 442908 (0.0008) [2023-12-27 08:42:21,354][20381] Updated weights for policy 1, policy_version 442583 (0.0008) [2023-12-27 08:42:21,410][20347] Updated weights for policy 0, policy_version 442918 (0.0009) [2023-12-27 08:42:21,422][20381] Updated weights for policy 1, policy_version 442593 (0.0008) [2023-12-27 08:42:21,469][20347] Updated weights for policy 0, policy_version 442928 (0.0006) [2023-12-27 08:42:21,483][20381] Updated weights for policy 1, policy_version 442603 (0.0008) [2023-12-27 08:42:21,520][20291] KL-divergence is very high: 192.8040 [2023-12-27 08:42:21,530][20347] Updated weights for policy 0, policy_version 442938 (0.0006) [2023-12-27 08:42:22,107][20347] Updated weights for policy 0, policy_version 442948 (0.0006) [2023-12-27 08:42:22,169][20347] Updated weights for policy 0, policy_version 442958 (0.0008) [2023-12-27 08:42:22,216][20381] Updated weights for policy 1, policy_version 442613 (0.0008) [2023-12-27 08:42:22,223][20347] Updated weights for policy 0, policy_version 442968 (0.0006) [2023-12-27 08:42:22,277][20381] Updated weights for policy 1, policy_version 442623 (0.0007) [2023-12-27 08:42:22,334][20381] Updated weights for policy 1, policy_version 442633 (0.0008) [2023-12-27 08:42:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 226746368. Throughput: 0: 10158.2, 1: 10406.0. Samples: 226717444. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:22,363][19170] Avg episode reward: [(0, '963.100'), (1, '990.300')] [2023-12-27 08:42:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000442976_113418240.pth... [2023-12-27 08:42:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000441792_113115136.pth [2023-12-27 08:42:22,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000442640_113336320.pth... [2023-12-27 08:42:22,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000441424_113025024.pth [2023-12-27 08:42:22,943][20347] Updated weights for policy 0, policy_version 442978 (0.0007) [2023-12-27 08:42:22,989][20347] Updated weights for policy 0, policy_version 442988 (0.0006) [2023-12-27 08:42:23,044][20347] Updated weights for policy 0, policy_version 442998 (0.0006) [2023-12-27 08:42:23,080][20381] Updated weights for policy 1, policy_version 442643 (0.0008) [2023-12-27 08:42:23,097][20347] Updated weights for policy 0, policy_version 443008 (0.0006) [2023-12-27 08:42:23,137][20381] Updated weights for policy 1, policy_version 442653 (0.0007) [2023-12-27 08:42:23,198][20381] Updated weights for policy 1, policy_version 442663 (0.0008) [2023-12-27 08:42:23,802][20347] Updated weights for policy 0, policy_version 443018 (0.0007) [2023-12-27 08:42:23,846][20347] Updated weights for policy 0, policy_version 443028 (0.0008) [2023-12-27 08:42:23,886][20381] Updated weights for policy 1, policy_version 442673 (0.0008) [2023-12-27 08:42:23,892][20292] KL-divergence is very high: 173.6753 [2023-12-27 08:42:23,896][20347] Updated weights for policy 0, policy_version 443038 (0.0007) [2023-12-27 08:42:23,942][20381] Updated weights for policy 1, policy_version 442683 (0.0009) [2023-12-27 08:42:23,982][20292] KL-divergence is very high: 111.0032 [2023-12-27 08:42:23,998][20381] Updated weights for policy 1, policy_version 442693 (0.0010) [2023-12-27 08:42:24,045][20381] Updated weights for policy 1, policy_version 442703 (0.0010) [2023-12-27 08:42:24,621][20347] Updated weights for policy 0, policy_version 443048 (0.0008) [2023-12-27 08:42:24,687][20347] Updated weights for policy 0, policy_version 443058 (0.0006) [2023-12-27 08:42:24,739][20347] Updated weights for policy 0, policy_version 443068 (0.0005) [2023-12-27 08:42:24,754][20381] Updated weights for policy 1, policy_version 442713 (0.0008) [2023-12-27 08:42:24,810][20381] Updated weights for policy 1, policy_version 442723 (0.0008) [2023-12-27 08:42:24,861][20381] Updated weights for policy 1, policy_version 442733 (0.0010) [2023-12-27 08:42:25,313][20347] Updated weights for policy 0, policy_version 443078 (0.0005) [2023-12-27 08:42:25,372][20347] Updated weights for policy 0, policy_version 443088 (0.0005) [2023-12-27 08:42:25,427][20347] Updated weights for policy 0, policy_version 443098 (0.0006) [2023-12-27 08:42:25,566][20381] Updated weights for policy 1, policy_version 442743 (0.0008) [2023-12-27 08:42:25,614][20381] Updated weights for policy 1, policy_version 442753 (0.0007) [2023-12-27 08:42:25,668][20381] Updated weights for policy 1, policy_version 442763 (0.0007) [2023-12-27 08:42:26,062][20347] Updated weights for policy 0, policy_version 443108 (0.0010) [2023-12-27 08:42:26,124][20347] Updated weights for policy 0, policy_version 443118 (0.0009) [2023-12-27 08:42:26,189][20347] Updated weights for policy 0, policy_version 443128 (0.0010) [2023-12-27 08:42:26,355][20381] Updated weights for policy 1, policy_version 442773 (0.0009) [2023-12-27 08:42:26,411][20381] Updated weights for policy 1, policy_version 442783 (0.0011) [2023-12-27 08:42:26,470][20381] Updated weights for policy 1, policy_version 442793 (0.0011) [2023-12-27 08:42:26,814][20347] Updated weights for policy 0, policy_version 443138 (0.0011) [2023-12-27 08:42:26,864][20347] Updated weights for policy 0, policy_version 443148 (0.0009) [2023-12-27 08:42:26,918][20347] Updated weights for policy 0, policy_version 443158 (0.0007) [2023-12-27 08:42:26,969][20347] Updated weights for policy 0, policy_version 443168 (0.0007) [2023-12-27 08:42:27,054][20381] Updated weights for policy 1, policy_version 442803 (0.0009) [2023-12-27 08:42:27,104][20381] Updated weights for policy 1, policy_version 442813 (0.0005) [2023-12-27 08:42:27,156][20381] Updated weights for policy 1, policy_version 442823 (0.0005) [2023-12-27 08:42:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 226852864. Throughput: 0: 10159.8, 1: 10311.3. Samples: 226839036. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:27,363][19170] Avg episode reward: [(0, '924.750'), (1, '980.600')] [2023-12-27 08:42:27,701][20347] Updated weights for policy 0, policy_version 443178 (0.0005) [2023-12-27 08:42:27,745][20381] Updated weights for policy 1, policy_version 442833 (0.0006) [2023-12-27 08:42:27,751][20347] Updated weights for policy 0, policy_version 443188 (0.0007) [2023-12-27 08:42:27,797][20381] Updated weights for policy 1, policy_version 442843 (0.0005) [2023-12-27 08:42:27,806][20347] Updated weights for policy 0, policy_version 443198 (0.0006) [2023-12-27 08:42:27,854][20381] Updated weights for policy 1, policy_version 442853 (0.0006) [2023-12-27 08:42:27,904][20381] Updated weights for policy 1, policy_version 442863 (0.0008) [2023-12-27 08:42:28,506][20347] Updated weights for policy 0, policy_version 443208 (0.0006) [2023-12-27 08:42:28,561][20347] Updated weights for policy 0, policy_version 443218 (0.0007) [2023-12-27 08:42:28,574][20381] Updated weights for policy 1, policy_version 442873 (0.0007) [2023-12-27 08:42:28,614][20347] Updated weights for policy 0, policy_version 443228 (0.0006) [2023-12-27 08:42:28,623][20381] Updated weights for policy 1, policy_version 442883 (0.0006) [2023-12-27 08:42:28,672][20381] Updated weights for policy 1, policy_version 442893 (0.0006) [2023-12-27 08:42:29,325][20347] Updated weights for policy 0, policy_version 443238 (0.0006) [2023-12-27 08:42:29,385][20381] Updated weights for policy 1, policy_version 442903 (0.0009) [2023-12-27 08:42:29,390][20347] Updated weights for policy 0, policy_version 443248 (0.0008) [2023-12-27 08:42:29,442][20381] Updated weights for policy 1, policy_version 442913 (0.0011) [2023-12-27 08:42:29,445][20347] Updated weights for policy 0, policy_version 443258 (0.0006) [2023-12-27 08:42:29,504][20381] Updated weights for policy 1, policy_version 442923 (0.0011) [2023-12-27 08:42:30,095][20381] Updated weights for policy 1, policy_version 442933 (0.0010) [2023-12-27 08:42:30,109][20292] KL-divergence is very high: 102.7050 [2023-12-27 08:42:30,123][20347] Updated weights for policy 0, policy_version 443268 (0.0005) [2023-12-27 08:42:30,140][20381] Updated weights for policy 1, policy_version 442943 (0.0007) [2023-12-27 08:42:30,173][20347] Updated weights for policy 0, policy_version 443278 (0.0007) [2023-12-27 08:42:30,187][20381] Updated weights for policy 1, policy_version 442953 (0.0007) [2023-12-27 08:42:30,222][20347] Updated weights for policy 0, policy_version 443288 (0.0006) [2023-12-27 08:42:30,883][20381] Updated weights for policy 1, policy_version 442963 (0.0008) [2023-12-27 08:42:30,910][20347] Updated weights for policy 0, policy_version 443298 (0.0007) [2023-12-27 08:42:30,933][20381] Updated weights for policy 1, policy_version 442973 (0.0007) [2023-12-27 08:42:30,961][20347] Updated weights for policy 0, policy_version 443308 (0.0006) [2023-12-27 08:42:30,988][20381] Updated weights for policy 1, policy_version 442983 (0.0006) [2023-12-27 08:42:31,014][20347] Updated weights for policy 0, policy_version 443318 (0.0005) [2023-12-27 08:42:31,075][20347] Updated weights for policy 0, policy_version 443328 (0.0008) [2023-12-27 08:42:31,664][20381] Updated weights for policy 1, policy_version 442993 (0.0008) [2023-12-27 08:42:31,726][20381] Updated weights for policy 1, policy_version 443003 (0.0011) [2023-12-27 08:42:31,730][20347] Updated weights for policy 0, policy_version 443338 (0.0008) [2023-12-27 08:42:31,779][20381] Updated weights for policy 1, policy_version 443013 (0.0011) [2023-12-27 08:42:31,791][20347] Updated weights for policy 0, policy_version 443348 (0.0007) [2023-12-27 08:42:31,828][20381] Updated weights for policy 1, policy_version 443023 (0.0010) [2023-12-27 08:42:31,852][20347] Updated weights for policy 0, policy_version 443358 (0.0008) [2023-12-27 08:42:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 226951168. Throughput: 0: 10100.5, 1: 10308.8. Samples: 226963904. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:32,363][19170] Avg episode reward: [(0, '942.460'), (1, '980.680')] [2023-12-27 08:42:32,521][20347] Updated weights for policy 0, policy_version 443368 (0.0008) [2023-12-27 08:42:32,525][20381] Updated weights for policy 1, policy_version 443033 (0.0010) [2023-12-27 08:42:32,575][20347] Updated weights for policy 0, policy_version 443378 (0.0008) [2023-12-27 08:42:32,582][20381] Updated weights for policy 1, policy_version 443043 (0.0010) [2023-12-27 08:42:32,632][20381] Updated weights for policy 1, policy_version 443053 (0.0007) [2023-12-27 08:42:32,634][20347] Updated weights for policy 0, policy_version 443388 (0.0007) [2023-12-27 08:42:33,228][20347] Updated weights for policy 0, policy_version 443398 (0.0005) [2023-12-27 08:42:33,271][20381] Updated weights for policy 1, policy_version 443063 (0.0007) [2023-12-27 08:42:33,278][20347] Updated weights for policy 0, policy_version 443408 (0.0005) [2023-12-27 08:42:33,329][20381] Updated weights for policy 1, policy_version 443073 (0.0008) [2023-12-27 08:42:33,334][20347] Updated weights for policy 0, policy_version 443418 (0.0006) [2023-12-27 08:42:33,384][20381] Updated weights for policy 1, policy_version 443083 (0.0007) [2023-12-27 08:42:33,923][20347] Updated weights for policy 0, policy_version 443428 (0.0006) [2023-12-27 08:42:33,976][20347] Updated weights for policy 0, policy_version 443438 (0.0005) [2023-12-27 08:42:34,029][20381] Updated weights for policy 1, policy_version 443093 (0.0007) [2023-12-27 08:42:34,031][20347] Updated weights for policy 0, policy_version 443448 (0.0005) [2023-12-27 08:42:34,077][20381] Updated weights for policy 1, policy_version 443103 (0.0007) [2023-12-27 08:42:34,123][20381] Updated weights for policy 1, policy_version 443113 (0.0007) [2023-12-27 08:42:34,650][20347] Updated weights for policy 0, policy_version 443458 (0.0007) [2023-12-27 08:42:34,708][20347] Updated weights for policy 0, policy_version 443468 (0.0010) [2023-12-27 08:42:34,766][20347] Updated weights for policy 0, policy_version 443478 (0.0010) [2023-12-27 08:42:34,820][20347] Updated weights for policy 0, policy_version 443488 (0.0009) [2023-12-27 08:42:34,844][20381] Updated weights for policy 1, policy_version 443123 (0.0007) [2023-12-27 08:42:34,892][20381] Updated weights for policy 1, policy_version 443133 (0.0008) [2023-12-27 08:42:34,937][20381] Updated weights for policy 1, policy_version 443143 (0.0008) [2023-12-27 08:42:35,506][20347] Updated weights for policy 0, policy_version 443498 (0.0011) [2023-12-27 08:42:35,565][20347] Updated weights for policy 0, policy_version 443508 (0.0011) [2023-12-27 08:42:35,618][20347] Updated weights for policy 0, policy_version 443518 (0.0010) [2023-12-27 08:42:35,678][20381] Updated weights for policy 1, policy_version 443153 (0.0007) [2023-12-27 08:42:35,729][20381] Updated weights for policy 1, policy_version 443163 (0.0008) [2023-12-27 08:42:35,781][20381] Updated weights for policy 1, policy_version 443173 (0.0008) [2023-12-27 08:42:35,836][20381] Updated weights for policy 1, policy_version 443183 (0.0007) [2023-12-27 08:42:36,318][20347] Updated weights for policy 0, policy_version 443528 (0.0006) [2023-12-27 08:42:36,376][20347] Updated weights for policy 0, policy_version 443538 (0.0007) [2023-12-27 08:42:36,435][20347] Updated weights for policy 0, policy_version 443548 (0.0008) [2023-12-27 08:42:36,569][20381] Updated weights for policy 1, policy_version 443193 (0.0008) [2023-12-27 08:42:36,627][20381] Updated weights for policy 1, policy_version 443203 (0.0008) [2023-12-27 08:42:36,685][20381] Updated weights for policy 1, policy_version 443213 (0.0009) [2023-12-27 08:42:37,042][20347] Updated weights for policy 0, policy_version 443558 (0.0008) [2023-12-27 08:42:37,102][20347] Updated weights for policy 0, policy_version 443568 (0.0007) [2023-12-27 08:42:37,157][20347] Updated weights for policy 0, policy_version 443578 (0.0008) [2023-12-27 08:42:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20688.3). Total num frames: 227057664. Throughput: 0: 10173.4, 1: 10281.6. Samples: 227027920. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:37,363][19170] Avg episode reward: [(0, '990.370'), (1, '990.380')] [2023-12-27 08:42:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000443584_113573888.pth... [2023-12-27 08:42:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000442368_113262592.pth [2023-12-27 08:42:37,411][20381] Updated weights for policy 1, policy_version 443223 (0.0008) [2023-12-27 08:42:37,464][20381] Updated weights for policy 1, policy_version 443233 (0.0007) [2023-12-27 08:42:37,521][20381] Updated weights for policy 1, policy_version 443243 (0.0007) [2023-12-27 08:42:37,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000443248_113491968.pth... [2023-12-27 08:42:37,551][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000442032_113180672.pth [2023-12-27 08:42:37,848][20347] Updated weights for policy 0, policy_version 443588 (0.0008) [2023-12-27 08:42:37,901][20347] Updated weights for policy 0, policy_version 443598 (0.0008) [2023-12-27 08:42:37,955][20347] Updated weights for policy 0, policy_version 443608 (0.0007) [2023-12-27 08:42:38,230][20381] Updated weights for policy 1, policy_version 443253 (0.0008) [2023-12-27 08:42:38,285][20381] Updated weights for policy 1, policy_version 443263 (0.0007) [2023-12-27 08:42:38,352][20381] Updated weights for policy 1, policy_version 443273 (0.0007) [2023-12-27 08:42:38,661][20347] Updated weights for policy 0, policy_version 443618 (0.0008) [2023-12-27 08:42:38,716][20347] Updated weights for policy 0, policy_version 443628 (0.0007) [2023-12-27 08:42:38,765][20347] Updated weights for policy 0, policy_version 443638 (0.0007) [2023-12-27 08:42:38,821][20347] Updated weights for policy 0, policy_version 443648 (0.0007) [2023-12-27 08:42:39,017][20381] Updated weights for policy 1, policy_version 443283 (0.0007) [2023-12-27 08:42:39,071][20381] Updated weights for policy 1, policy_version 443293 (0.0008) [2023-12-27 08:42:39,125][20381] Updated weights for policy 1, policy_version 443303 (0.0008) [2023-12-27 08:42:39,554][20347] Updated weights for policy 0, policy_version 443658 (0.0010) [2023-12-27 08:42:39,610][20347] Updated weights for policy 0, policy_version 443668 (0.0010) [2023-12-27 08:42:39,666][20347] Updated weights for policy 0, policy_version 443678 (0.0010) [2023-12-27 08:42:39,799][20381] Updated weights for policy 1, policy_version 443313 (0.0008) [2023-12-27 08:42:39,869][20381] Updated weights for policy 1, policy_version 443323 (0.0008) [2023-12-27 08:42:39,934][20381] Updated weights for policy 1, policy_version 443333 (0.0007) [2023-12-27 08:42:39,995][20381] Updated weights for policy 1, policy_version 443343 (0.0008) [2023-12-27 08:42:40,374][20347] Updated weights for policy 0, policy_version 443688 (0.0009) [2023-12-27 08:42:40,437][20347] Updated weights for policy 0, policy_version 443698 (0.0008) [2023-12-27 08:42:40,489][20347] Updated weights for policy 0, policy_version 443708 (0.0007) [2023-12-27 08:42:40,715][20381] Updated weights for policy 1, policy_version 443353 (0.0007) [2023-12-27 08:42:40,771][20381] Updated weights for policy 1, policy_version 443363 (0.0008) [2023-12-27 08:42:40,827][20381] Updated weights for policy 1, policy_version 443373 (0.0008) [2023-12-27 08:42:41,190][20347] Updated weights for policy 0, policy_version 443718 (0.0007) [2023-12-27 08:42:41,254][20347] Updated weights for policy 0, policy_version 443728 (0.0008) [2023-12-27 08:42:41,275][20291] KL-divergence is very high: 123.0528 [2023-12-27 08:42:41,314][20347] Updated weights for policy 0, policy_version 443738 (0.0008) [2023-12-27 08:42:41,556][20381] Updated weights for policy 1, policy_version 443383 (0.0006) [2023-12-27 08:42:41,617][20381] Updated weights for policy 1, policy_version 443393 (0.0006) [2023-12-27 08:42:41,678][20381] Updated weights for policy 1, policy_version 443403 (0.0007) [2023-12-27 08:42:42,070][20347] Updated weights for policy 0, policy_version 443748 (0.0008) [2023-12-27 08:42:42,123][20347] Updated weights for policy 0, policy_version 443758 (0.0008) [2023-12-27 08:42:42,176][20347] Updated weights for policy 0, policy_version 443768 (0.0008) [2023-12-27 08:42:42,330][20381] Updated weights for policy 1, policy_version 443413 (0.0007) [2023-12-27 08:42:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 227155968. Throughput: 0: 10204.3, 1: 10212.4. Samples: 227146952. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 08:42:42,363][19170] Avg episode reward: [(0, '981.760'), (1, '1000.000')] [2023-12-27 08:42:42,392][20381] Updated weights for policy 1, policy_version 443423 (0.0006) [2023-12-27 08:42:42,447][20381] Updated weights for policy 1, policy_version 443433 (0.0005) [2023-12-27 08:42:42,933][20347] Updated weights for policy 0, policy_version 443778 (0.0007) [2023-12-27 08:42:42,992][20347] Updated weights for policy 0, policy_version 443788 (0.0008) [2023-12-27 08:42:43,028][20381] Updated weights for policy 1, policy_version 443443 (0.0005) [2023-12-27 08:42:43,046][20347] Updated weights for policy 0, policy_version 443798 (0.0008) [2023-12-27 08:42:43,081][20381] Updated weights for policy 1, policy_version 443453 (0.0005) [2023-12-27 08:42:43,091][20347] Updated weights for policy 0, policy_version 443808 (0.0007) [2023-12-27 08:42:43,139][20381] Updated weights for policy 1, policy_version 443463 (0.0006) [2023-12-27 08:42:43,675][20381] Updated weights for policy 1, policy_version 443473 (0.0006) [2023-12-27 08:42:43,736][20381] Updated weights for policy 1, policy_version 443483 (0.0005) [2023-12-27 08:42:43,786][20381] Updated weights for policy 1, policy_version 443493 (0.0010) [2023-12-27 08:42:43,835][20381] Updated weights for policy 1, policy_version 443503 (0.0010) [2023-12-27 08:42:43,844][20347] Updated weights for policy 0, policy_version 443818 (0.0005) [2023-12-27 08:42:43,895][20347] Updated weights for policy 0, policy_version 443828 (0.0007) [2023-12-27 08:42:43,952][20347] Updated weights for policy 0, policy_version 443838 (0.0007) [2023-12-27 08:42:44,429][20381] Updated weights for policy 1, policy_version 443513 (0.0008) [2023-12-27 08:42:44,482][20381] Updated weights for policy 1, policy_version 443523 (0.0008) [2023-12-27 08:42:44,537][20381] Updated weights for policy 1, policy_version 443533 (0.0007) [2023-12-27 08:42:44,696][20347] Updated weights for policy 0, policy_version 443848 (0.0007) [2023-12-27 08:42:44,759][20347] Updated weights for policy 0, policy_version 443858 (0.0008) [2023-12-27 08:42:44,818][20347] Updated weights for policy 0, policy_version 443868 (0.0007) [2023-12-27 08:42:45,168][20381] Updated weights for policy 1, policy_version 443543 (0.0007) [2023-12-27 08:42:45,234][20381] Updated weights for policy 1, policy_version 443553 (0.0006) [2023-12-27 08:42:45,294][20381] Updated weights for policy 1, policy_version 443563 (0.0008) [2023-12-27 08:42:45,550][20347] Updated weights for policy 0, policy_version 443878 (0.0008) [2023-12-27 08:42:45,610][20347] Updated weights for policy 0, policy_version 443888 (0.0008) [2023-12-27 08:42:45,669][20347] Updated weights for policy 0, policy_version 443898 (0.0007) [2023-12-27 08:42:45,893][20381] Updated weights for policy 1, policy_version 443573 (0.0007) [2023-12-27 08:42:45,942][20381] Updated weights for policy 1, policy_version 443583 (0.0007) [2023-12-27 08:42:45,999][20381] Updated weights for policy 1, policy_version 443593 (0.0008) [2023-12-27 08:42:46,355][20347] Updated weights for policy 0, policy_version 443908 (0.0009) [2023-12-27 08:42:46,411][20347] Updated weights for policy 0, policy_version 443918 (0.0010) [2023-12-27 08:42:46,466][20347] Updated weights for policy 0, policy_version 443928 (0.0010) [2023-12-27 08:42:46,701][20381] Updated weights for policy 1, policy_version 443603 (0.0009) [2023-12-27 08:42:46,757][20381] Updated weights for policy 1, policy_version 443613 (0.0008) [2023-12-27 08:42:46,807][20381] Updated weights for policy 1, policy_version 443623 (0.0007) [2023-12-27 08:42:47,072][20347] Updated weights for policy 0, policy_version 443938 (0.0009) [2023-12-27 08:42:47,125][20347] Updated weights for policy 0, policy_version 443948 (0.0009) [2023-12-27 08:42:47,188][20347] Updated weights for policy 0, policy_version 443958 (0.0010) [2023-12-27 08:42:47,249][20347] Updated weights for policy 0, policy_version 443968 (0.0010) [2023-12-27 08:42:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 227262464. Throughput: 0: 10171.5, 1: 10296.3. Samples: 227271044. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:42:47,363][19170] Avg episode reward: [(0, '962.640'), (1, '990.590')] [2023-12-27 08:42:47,475][20381] Updated weights for policy 1, policy_version 443633 (0.0008) [2023-12-27 08:42:47,524][20381] Updated weights for policy 1, policy_version 443643 (0.0005) [2023-12-27 08:42:47,584][20381] Updated weights for policy 1, policy_version 443653 (0.0005) [2023-12-27 08:42:47,637][20381] Updated weights for policy 1, policy_version 443663 (0.0005) [2023-12-27 08:42:47,833][20347] Updated weights for policy 0, policy_version 443978 (0.0008) [2023-12-27 08:42:47,893][20347] Updated weights for policy 0, policy_version 443988 (0.0007) [2023-12-27 08:42:47,943][20347] Updated weights for policy 0, policy_version 443998 (0.0006) [2023-12-27 08:42:48,156][20381] Updated weights for policy 1, policy_version 443673 (0.0007) [2023-12-27 08:42:48,205][20381] Updated weights for policy 1, policy_version 443683 (0.0008) [2023-12-27 08:42:48,252][20381] Updated weights for policy 1, policy_version 443693 (0.0008) [2023-12-27 08:42:48,617][20347] Updated weights for policy 0, policy_version 444008 (0.0008) [2023-12-27 08:42:48,674][20347] Updated weights for policy 0, policy_version 444018 (0.0007) [2023-12-27 08:42:48,734][20347] Updated weights for policy 0, policy_version 444028 (0.0006) [2023-12-27 08:42:49,021][20381] Updated weights for policy 1, policy_version 443703 (0.0008) [2023-12-27 08:42:49,072][20381] Updated weights for policy 1, policy_version 443713 (0.0010) [2023-12-27 08:42:49,125][20381] Updated weights for policy 1, policy_version 443723 (0.0009) [2023-12-27 08:42:49,366][20347] Updated weights for policy 0, policy_version 444038 (0.0008) [2023-12-27 08:42:49,435][20347] Updated weights for policy 0, policy_version 444048 (0.0007) [2023-12-27 08:42:49,496][20347] Updated weights for policy 0, policy_version 444058 (0.0006) [2023-12-27 08:42:49,896][20381] Updated weights for policy 1, policy_version 443733 (0.0007) [2023-12-27 08:42:49,962][20381] Updated weights for policy 1, policy_version 443743 (0.0007) [2023-12-27 08:42:50,016][20381] Updated weights for policy 1, policy_version 443753 (0.0007) [2023-12-27 08:42:50,166][20347] Updated weights for policy 0, policy_version 444068 (0.0008) [2023-12-27 08:42:50,225][20347] Updated weights for policy 0, policy_version 444078 (0.0008) [2023-12-27 08:42:50,289][20347] Updated weights for policy 0, policy_version 444088 (0.0009) [2023-12-27 08:42:50,739][20381] Updated weights for policy 1, policy_version 443763 (0.0008) [2023-12-27 08:42:50,790][20381] Updated weights for policy 1, policy_version 443773 (0.0007) [2023-12-27 08:42:50,849][20381] Updated weights for policy 1, policy_version 443783 (0.0008) [2023-12-27 08:42:50,942][20347] Updated weights for policy 0, policy_version 444098 (0.0010) [2023-12-27 08:42:50,991][20347] Updated weights for policy 0, policy_version 444108 (0.0010) [2023-12-27 08:42:51,048][20347] Updated weights for policy 0, policy_version 444118 (0.0011) [2023-12-27 08:42:51,110][20347] Updated weights for policy 0, policy_version 444128 (0.0010) [2023-12-27 08:42:51,609][20381] Updated weights for policy 1, policy_version 443793 (0.0008) [2023-12-27 08:42:51,676][20381] Updated weights for policy 1, policy_version 443803 (0.0010) [2023-12-27 08:42:51,736][20381] Updated weights for policy 1, policy_version 443813 (0.0010) [2023-12-27 08:42:51,799][20381] Updated weights for policy 1, policy_version 443823 (0.0008) [2023-12-27 08:42:51,824][20347] Updated weights for policy 0, policy_version 444138 (0.0010) [2023-12-27 08:42:51,881][20347] Updated weights for policy 0, policy_version 444148 (0.0009) [2023-12-27 08:42:51,935][20347] Updated weights for policy 0, policy_version 444158 (0.0007) [2023-12-27 08:42:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 227360768. Throughput: 0: 10204.3, 1: 10305.7. Samples: 227334072. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:42:52,363][19170] Avg episode reward: [(0, '972.270'), (1, '990.590')] [2023-12-27 08:42:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000443824_113639424.pth... [2023-12-27 08:42:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000444160_113721344.pth... [2023-12-27 08:42:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000442640_113336320.pth [2023-12-27 08:42:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000442976_113418240.pth [2023-12-27 08:42:52,451][20381] Updated weights for policy 1, policy_version 443833 (0.0006) [2023-12-27 08:42:52,516][20381] Updated weights for policy 1, policy_version 443843 (0.0006) [2023-12-27 08:42:52,572][20381] Updated weights for policy 1, policy_version 443853 (0.0006) [2023-12-27 08:42:52,577][20347] Updated weights for policy 0, policy_version 444168 (0.0010) [2023-12-27 08:42:52,626][20347] Updated weights for policy 0, policy_version 444178 (0.0010) [2023-12-27 08:42:52,672][20347] Updated weights for policy 0, policy_version 444188 (0.0010) [2023-12-27 08:42:53,189][20381] Updated weights for policy 1, policy_version 443863 (0.0007) [2023-12-27 08:42:53,234][20381] Updated weights for policy 1, policy_version 443873 (0.0007) [2023-12-27 08:42:53,285][20381] Updated weights for policy 1, policy_version 443883 (0.0007) [2023-12-27 08:42:53,362][20347] Updated weights for policy 0, policy_version 444198 (0.0010) [2023-12-27 08:42:53,413][20347] Updated weights for policy 0, policy_version 444208 (0.0010) [2023-12-27 08:42:53,460][20347] Updated weights for policy 0, policy_version 444218 (0.0006) [2023-12-27 08:42:54,019][20381] Updated weights for policy 1, policy_version 443893 (0.0007) [2023-12-27 08:42:54,073][20381] Updated weights for policy 1, policy_version 443903 (0.0008) [2023-12-27 08:42:54,135][20381] Updated weights for policy 1, policy_version 443913 (0.0006) [2023-12-27 08:42:54,137][20347] Updated weights for policy 0, policy_version 444228 (0.0008) [2023-12-27 08:42:54,170][20291] KL-divergence is very high: 158.6227 [2023-12-27 08:42:54,193][20347] Updated weights for policy 0, policy_version 444238 (0.0010) [2023-12-27 08:42:54,249][20347] Updated weights for policy 0, policy_version 444248 (0.0011) [2023-12-27 08:42:54,832][20381] Updated weights for policy 1, policy_version 443923 (0.0006) [2023-12-27 08:42:54,887][20381] Updated weights for policy 1, policy_version 443933 (0.0006) [2023-12-27 08:42:54,931][20347] Updated weights for policy 0, policy_version 444258 (0.0010) [2023-12-27 08:42:54,954][20381] Updated weights for policy 1, policy_version 443943 (0.0007) [2023-12-27 08:42:54,984][20347] Updated weights for policy 0, policy_version 444268 (0.0011) [2023-12-27 08:42:55,044][20347] Updated weights for policy 0, policy_version 444278 (0.0011) [2023-12-27 08:42:55,101][20347] Updated weights for policy 0, policy_version 444288 (0.0010) [2023-12-27 08:42:55,627][20381] Updated weights for policy 1, policy_version 443953 (0.0008) [2023-12-27 08:42:55,690][20381] Updated weights for policy 1, policy_version 443963 (0.0006) [2023-12-27 08:42:55,736][20347] Updated weights for policy 0, policy_version 444298 (0.0010) [2023-12-27 08:42:55,750][20381] Updated weights for policy 1, policy_version 443973 (0.0006) [2023-12-27 08:42:55,786][20347] Updated weights for policy 0, policy_version 444308 (0.0008) [2023-12-27 08:42:55,804][20381] Updated weights for policy 1, policy_version 443983 (0.0005) [2023-12-27 08:42:55,847][20347] Updated weights for policy 0, policy_version 444318 (0.0010) [2023-12-27 08:42:56,394][20381] Updated weights for policy 1, policy_version 443993 (0.0007) [2023-12-27 08:42:56,447][20381] Updated weights for policy 1, policy_version 444003 (0.0007) [2023-12-27 08:42:56,457][20347] Updated weights for policy 0, policy_version 444328 (0.0009) [2023-12-27 08:42:56,500][20381] Updated weights for policy 1, policy_version 444013 (0.0005) [2023-12-27 08:42:56,505][20347] Updated weights for policy 0, policy_version 444338 (0.0010) [2023-12-27 08:42:56,553][20347] Updated weights for policy 0, policy_version 444348 (0.0010) [2023-12-27 08:42:57,156][20381] Updated weights for policy 1, policy_version 444023 (0.0007) [2023-12-27 08:42:57,211][20381] Updated weights for policy 1, policy_version 444033 (0.0007) [2023-12-27 08:42:57,248][20347] Updated weights for policy 0, policy_version 444358 (0.0008) [2023-12-27 08:42:57,269][20381] Updated weights for policy 1, policy_version 444043 (0.0008) [2023-12-27 08:42:57,309][20347] Updated weights for policy 0, policy_version 444368 (0.0006) [2023-12-27 08:42:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 227467264. Throughput: 0: 10243.5, 1: 10320.2. Samples: 227457820. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:42:57,363][19170] Avg episode reward: [(0, '982.510'), (1, '991.000')] [2023-12-27 08:42:57,364][20347] Updated weights for policy 0, policy_version 444378 (0.0010) [2023-12-27 08:42:57,902][20347] Updated weights for policy 0, policy_version 444388 (0.0009) [2023-12-27 08:42:57,946][20381] Updated weights for policy 1, policy_version 444053 (0.0007) [2023-12-27 08:42:57,954][20347] Updated weights for policy 0, policy_version 444398 (0.0005) [2023-12-27 08:42:57,994][20381] Updated weights for policy 1, policy_version 444063 (0.0006) [2023-12-27 08:42:58,006][20347] Updated weights for policy 0, policy_version 444408 (0.0010) [2023-12-27 08:42:58,043][20381] Updated weights for policy 1, policy_version 444073 (0.0006) [2023-12-27 08:42:58,698][20381] Updated weights for policy 1, policy_version 444083 (0.0006) [2023-12-27 08:42:58,737][20347] Updated weights for policy 0, policy_version 444418 (0.0010) [2023-12-27 08:42:58,755][20381] Updated weights for policy 1, policy_version 444093 (0.0007) [2023-12-27 08:42:58,798][20347] Updated weights for policy 0, policy_version 444428 (0.0008) [2023-12-27 08:42:58,820][20381] Updated weights for policy 1, policy_version 444103 (0.0007) [2023-12-27 08:42:58,857][20347] Updated weights for policy 0, policy_version 444438 (0.0007) [2023-12-27 08:42:58,912][20347] Updated weights for policy 0, policy_version 444448 (0.0007) [2023-12-27 08:42:59,459][20381] Updated weights for policy 1, policy_version 444113 (0.0007) [2023-12-27 08:42:59,508][20381] Updated weights for policy 1, policy_version 444123 (0.0007) [2023-12-27 08:42:59,556][20381] Updated weights for policy 1, policy_version 444133 (0.0007) [2023-12-27 08:42:59,615][20347] Updated weights for policy 0, policy_version 444458 (0.0006) [2023-12-27 08:42:59,621][20381] Updated weights for policy 1, policy_version 444143 (0.0007) [2023-12-27 08:42:59,672][20347] Updated weights for policy 0, policy_version 444468 (0.0007) [2023-12-27 08:42:59,730][20347] Updated weights for policy 0, policy_version 444478 (0.0008) [2023-12-27 08:43:00,236][20381] Updated weights for policy 1, policy_version 444153 (0.0007) [2023-12-27 08:43:00,285][20381] Updated weights for policy 1, policy_version 444163 (0.0007) [2023-12-27 08:43:00,335][20381] Updated weights for policy 1, policy_version 444173 (0.0007) [2023-12-27 08:43:00,465][20347] Updated weights for policy 0, policy_version 444488 (0.0008) [2023-12-27 08:43:00,479][20291] KL-divergence is very high: 111.0579 [2023-12-27 08:43:00,498][20291] KL-divergence is very high: 429.9592 [2023-12-27 08:43:00,502][20291] KL-divergence is very high: 315.0678 [2023-12-27 08:43:00,517][20347] Updated weights for policy 0, policy_version 444498 (0.0009) [2023-12-27 08:43:00,572][20347] Updated weights for policy 0, policy_version 444508 (0.0007) [2023-12-27 08:43:00,586][20291] KL-divergence is very high: 124.6863 [2023-12-27 08:43:00,955][20381] Updated weights for policy 1, policy_version 444183 (0.0007) [2023-12-27 08:43:01,006][20381] Updated weights for policy 1, policy_version 444193 (0.0008) [2023-12-27 08:43:01,071][20381] Updated weights for policy 1, policy_version 444203 (0.0008) [2023-12-27 08:43:01,289][20347] Updated weights for policy 0, policy_version 444518 (0.0009) [2023-12-27 08:43:01,352][20347] Updated weights for policy 0, policy_version 444528 (0.0010) [2023-12-27 08:43:01,408][20347] Updated weights for policy 0, policy_version 444538 (0.0011) [2023-12-27 08:43:01,783][20381] Updated weights for policy 1, policy_version 444213 (0.0008) [2023-12-27 08:43:01,842][20381] Updated weights for policy 1, policy_version 444223 (0.0006) [2023-12-27 08:43:01,905][20381] Updated weights for policy 1, policy_version 444233 (0.0005) [2023-12-27 08:43:02,063][20347] Updated weights for policy 0, policy_version 444548 (0.0008) [2023-12-27 08:43:02,118][20347] Updated weights for policy 0, policy_version 444558 (0.0009) [2023-12-27 08:43:02,176][20347] Updated weights for policy 0, policy_version 444568 (0.0010) [2023-12-27 08:43:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 227573760. Throughput: 0: 10224.3, 1: 10351.7. Samples: 227583156. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:02,363][19170] Avg episode reward: [(0, '962.390'), (1, '991.000')] [2023-12-27 08:43:02,565][20381] Updated weights for policy 1, policy_version 444243 (0.0005) [2023-12-27 08:43:02,620][20381] Updated weights for policy 1, policy_version 444253 (0.0005) [2023-12-27 08:43:02,671][20381] Updated weights for policy 1, policy_version 444263 (0.0005) [2023-12-27 08:43:02,833][20347] Updated weights for policy 0, policy_version 444578 (0.0010) [2023-12-27 08:43:02,882][20347] Updated weights for policy 0, policy_version 444588 (0.0007) [2023-12-27 08:43:02,930][20347] Updated weights for policy 0, policy_version 444598 (0.0008) [2023-12-27 08:43:02,987][20347] Updated weights for policy 0, policy_version 444608 (0.0006) [2023-12-27 08:43:03,331][20381] Updated weights for policy 1, policy_version 444273 (0.0007) [2023-12-27 08:43:03,380][20381] Updated weights for policy 1, policy_version 444283 (0.0007) [2023-12-27 08:43:03,429][20381] Updated weights for policy 1, policy_version 444293 (0.0007) [2023-12-27 08:43:03,483][20381] Updated weights for policy 1, policy_version 444303 (0.0008) [2023-12-27 08:43:03,633][20347] Updated weights for policy 0, policy_version 444618 (0.0009) [2023-12-27 08:43:03,682][20347] Updated weights for policy 0, policy_version 444628 (0.0005) [2023-12-27 08:43:03,738][20347] Updated weights for policy 0, policy_version 444638 (0.0007) [2023-12-27 08:43:04,170][20381] Updated weights for policy 1, policy_version 444313 (0.0007) [2023-12-27 08:43:04,223][20381] Updated weights for policy 1, policy_version 444323 (0.0007) [2023-12-27 08:43:04,283][20381] Updated weights for policy 1, policy_version 444333 (0.0007) [2023-12-27 08:43:04,411][20347] Updated weights for policy 0, policy_version 444648 (0.0010) [2023-12-27 08:43:04,474][20347] Updated weights for policy 0, policy_version 444658 (0.0010) [2023-12-27 08:43:04,539][20347] Updated weights for policy 0, policy_version 444668 (0.0010) [2023-12-27 08:43:04,950][20381] Updated weights for policy 1, policy_version 444343 (0.0009) [2023-12-27 08:43:05,030][20381] Updated weights for policy 1, policy_version 444353 (0.0010) [2023-12-27 08:43:05,088][20381] Updated weights for policy 1, policy_version 444363 (0.0010) [2023-12-27 08:43:05,206][20347] Updated weights for policy 0, policy_version 444678 (0.0008) [2023-12-27 08:43:05,264][20347] Updated weights for policy 0, policy_version 444688 (0.0009) [2023-12-27 08:43:05,328][20347] Updated weights for policy 0, policy_version 444698 (0.0010) [2023-12-27 08:43:05,730][20381] Updated weights for policy 1, policy_version 444373 (0.0006) [2023-12-27 08:43:05,786][20381] Updated weights for policy 1, policy_version 444383 (0.0010) [2023-12-27 08:43:05,840][20381] Updated weights for policy 1, policy_version 444393 (0.0006) [2023-12-27 08:43:06,003][20347] Updated weights for policy 0, policy_version 444708 (0.0010) [2023-12-27 08:43:06,057][20347] Updated weights for policy 0, policy_version 444718 (0.0006) [2023-12-27 08:43:06,122][20347] Updated weights for policy 0, policy_version 444728 (0.0009) [2023-12-27 08:43:06,464][20381] Updated weights for policy 1, policy_version 444403 (0.0007) [2023-12-27 08:43:06,520][20381] Updated weights for policy 1, policy_version 444413 (0.0011) [2023-12-27 08:43:06,573][20381] Updated weights for policy 1, policy_version 444423 (0.0011) [2023-12-27 08:43:06,780][20347] Updated weights for policy 0, policy_version 444738 (0.0011) [2023-12-27 08:43:06,825][20347] Updated weights for policy 0, policy_version 444748 (0.0010) [2023-12-27 08:43:06,881][20347] Updated weights for policy 0, policy_version 444758 (0.0010) [2023-12-27 08:43:06,941][20347] Updated weights for policy 0, policy_version 444768 (0.0007) [2023-12-27 08:43:07,230][20381] Updated weights for policy 1, policy_version 444433 (0.0010) [2023-12-27 08:43:07,291][20381] Updated weights for policy 1, policy_version 444443 (0.0007) [2023-12-27 08:43:07,344][20381] Updated weights for policy 1, policy_version 444453 (0.0007) [2023-12-27 08:43:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 227672064. Throughput: 0: 10280.5, 1: 10336.6. Samples: 227645212. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:07,363][19170] Avg episode reward: [(0, '971.240'), (1, '1000.000')] [2023-12-27 08:43:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000444768_113876992.pth... [2023-12-27 08:43:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000443584_113573888.pth [2023-12-27 08:43:07,402][20381] Updated weights for policy 1, policy_version 444463 (0.0007) [2023-12-27 08:43:07,407][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000444464_113803264.pth... [2023-12-27 08:43:07,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000443248_113491968.pth [2023-12-27 08:43:07,622][20347] Updated weights for policy 0, policy_version 444778 (0.0008) [2023-12-27 08:43:07,679][20347] Updated weights for policy 0, policy_version 444788 (0.0010) [2023-12-27 08:43:07,744][20347] Updated weights for policy 0, policy_version 444798 (0.0011) [2023-12-27 08:43:08,069][20381] Updated weights for policy 1, policy_version 444473 (0.0009) [2023-12-27 08:43:08,121][20381] Updated weights for policy 1, policy_version 444483 (0.0007) [2023-12-27 08:43:08,178][20381] Updated weights for policy 1, policy_version 444493 (0.0008) [2023-12-27 08:43:08,369][20347] Updated weights for policy 0, policy_version 444808 (0.0009) [2023-12-27 08:43:08,428][20347] Updated weights for policy 0, policy_version 444818 (0.0006) [2023-12-27 08:43:08,490][20347] Updated weights for policy 0, policy_version 444828 (0.0007) [2023-12-27 08:43:08,792][20381] Updated weights for policy 1, policy_version 444503 (0.0006) [2023-12-27 08:43:08,848][20381] Updated weights for policy 1, policy_version 444513 (0.0008) [2023-12-27 08:43:08,904][20381] Updated weights for policy 1, policy_version 444523 (0.0008) [2023-12-27 08:43:09,201][20347] Updated weights for policy 0, policy_version 444838 (0.0008) [2023-12-27 08:43:09,264][20347] Updated weights for policy 0, policy_version 444848 (0.0007) [2023-12-27 08:43:09,316][20347] Updated weights for policy 0, policy_version 444858 (0.0006) [2023-12-27 08:43:09,550][20381] Updated weights for policy 1, policy_version 444533 (0.0006) [2023-12-27 08:43:09,604][20381] Updated weights for policy 1, policy_version 444543 (0.0008) [2023-12-27 08:43:09,663][20381] Updated weights for policy 1, policy_version 444553 (0.0008) [2023-12-27 08:43:10,044][20347] Updated weights for policy 0, policy_version 444868 (0.0008) [2023-12-27 08:43:10,105][20347] Updated weights for policy 0, policy_version 444878 (0.0008) [2023-12-27 08:43:10,168][20347] Updated weights for policy 0, policy_version 444888 (0.0007) [2023-12-27 08:43:10,349][20381] Updated weights for policy 1, policy_version 444563 (0.0009) [2023-12-27 08:43:10,405][20381] Updated weights for policy 1, policy_version 444573 (0.0008) [2023-12-27 08:43:10,464][20381] Updated weights for policy 1, policy_version 444583 (0.0007) [2023-12-27 08:43:10,836][20347] Updated weights for policy 0, policy_version 444898 (0.0006) [2023-12-27 08:43:10,883][20347] Updated weights for policy 0, policy_version 444908 (0.0008) [2023-12-27 08:43:10,937][20347] Updated weights for policy 0, policy_version 444918 (0.0008) [2023-12-27 08:43:10,990][20347] Updated weights for policy 0, policy_version 444928 (0.0008) [2023-12-27 08:43:11,136][20381] Updated weights for policy 1, policy_version 444593 (0.0008) [2023-12-27 08:43:11,196][20381] Updated weights for policy 1, policy_version 444603 (0.0008) [2023-12-27 08:43:11,251][20381] Updated weights for policy 1, policy_version 444613 (0.0007) [2023-12-27 08:43:11,311][20381] Updated weights for policy 1, policy_version 444623 (0.0008) [2023-12-27 08:43:11,742][20347] Updated weights for policy 0, policy_version 444938 (0.0008) [2023-12-27 08:43:11,804][20347] Updated weights for policy 0, policy_version 444948 (0.0007) [2023-12-27 08:43:11,871][20347] Updated weights for policy 0, policy_version 444958 (0.0006) [2023-12-27 08:43:12,027][20381] Updated weights for policy 1, policy_version 444633 (0.0005) [2023-12-27 08:43:12,080][20381] Updated weights for policy 1, policy_version 444643 (0.0006) [2023-12-27 08:43:12,140][20381] Updated weights for policy 1, policy_version 444653 (0.0006) [2023-12-27 08:43:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 227778560. Throughput: 0: 10220.7, 1: 10424.3. Samples: 227768060. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:12,363][19170] Avg episode reward: [(0, '990.300'), (1, '1000.000')] [2023-12-27 08:43:12,603][20347] Updated weights for policy 0, policy_version 444968 (0.0007) [2023-12-27 08:43:12,666][20347] Updated weights for policy 0, policy_version 444978 (0.0006) [2023-12-27 08:43:12,728][20347] Updated weights for policy 0, policy_version 444988 (0.0007) [2023-12-27 08:43:12,802][20381] Updated weights for policy 1, policy_version 444663 (0.0007) [2023-12-27 08:43:12,839][20292] KL-divergence is very high: 304.5860 [2023-12-27 08:43:12,860][20381] Updated weights for policy 1, policy_version 444673 (0.0007) [2023-12-27 08:43:12,918][20381] Updated weights for policy 1, policy_version 444683 (0.0007) [2023-12-27 08:43:13,400][20347] Updated weights for policy 0, policy_version 444998 (0.0007) [2023-12-27 08:43:13,463][20347] Updated weights for policy 0, policy_version 445008 (0.0007) [2023-12-27 08:43:13,532][20347] Updated weights for policy 0, policy_version 445018 (0.0008) [2023-12-27 08:43:13,616][20381] Updated weights for policy 1, policy_version 444693 (0.0007) [2023-12-27 08:43:13,666][20381] Updated weights for policy 1, policy_version 444703 (0.0008) [2023-12-27 08:43:13,718][20381] Updated weights for policy 1, policy_version 444713 (0.0008) [2023-12-27 08:43:14,176][20347] Updated weights for policy 0, policy_version 445028 (0.0008) [2023-12-27 08:43:14,238][20347] Updated weights for policy 0, policy_version 445038 (0.0007) [2023-12-27 08:43:14,294][20347] Updated weights for policy 0, policy_version 445048 (0.0007) [2023-12-27 08:43:14,448][20381] Updated weights for policy 1, policy_version 444723 (0.0008) [2023-12-27 08:43:14,497][20381] Updated weights for policy 1, policy_version 444733 (0.0007) [2023-12-27 08:43:14,560][20381] Updated weights for policy 1, policy_version 444743 (0.0007) [2023-12-27 08:43:14,930][20347] Updated weights for policy 0, policy_version 445058 (0.0007) [2023-12-27 08:43:14,995][20347] Updated weights for policy 0, policy_version 445068 (0.0008) [2023-12-27 08:43:15,056][20347] Updated weights for policy 0, policy_version 445078 (0.0006) [2023-12-27 08:43:15,114][20347] Updated weights for policy 0, policy_version 445088 (0.0006) [2023-12-27 08:43:15,270][20381] Updated weights for policy 1, policy_version 444753 (0.0007) [2023-12-27 08:43:15,324][20381] Updated weights for policy 1, policy_version 444763 (0.0005) [2023-12-27 08:43:15,382][20381] Updated weights for policy 1, policy_version 444773 (0.0006) [2023-12-27 08:43:15,437][20381] Updated weights for policy 1, policy_version 444783 (0.0005) [2023-12-27 08:43:15,780][20347] Updated weights for policy 0, policy_version 445098 (0.0005) [2023-12-27 08:43:15,832][20347] Updated weights for policy 0, policy_version 445108 (0.0005) [2023-12-27 08:43:15,877][20347] Updated weights for policy 0, policy_version 445118 (0.0005) [2023-12-27 08:43:15,996][20381] Updated weights for policy 1, policy_version 444793 (0.0006) [2023-12-27 08:43:16,054][20381] Updated weights for policy 1, policy_version 444803 (0.0007) [2023-12-27 08:43:16,108][20381] Updated weights for policy 1, policy_version 444813 (0.0007) [2023-12-27 08:43:16,493][20347] Updated weights for policy 0, policy_version 445128 (0.0005) [2023-12-27 08:43:16,543][20347] Updated weights for policy 0, policy_version 445138 (0.0005) [2023-12-27 08:43:16,594][20347] Updated weights for policy 0, policy_version 445148 (0.0005) [2023-12-27 08:43:16,809][20381] Updated weights for policy 1, policy_version 444823 (0.0007) [2023-12-27 08:43:16,867][20381] Updated weights for policy 1, policy_version 444833 (0.0006) [2023-12-27 08:43:16,915][20381] Updated weights for policy 1, policy_version 444843 (0.0005) [2023-12-27 08:43:17,209][20347] Updated weights for policy 0, policy_version 445158 (0.0008) [2023-12-27 08:43:17,260][20347] Updated weights for policy 0, policy_version 445168 (0.0010) [2023-12-27 08:43:17,308][20347] Updated weights for policy 0, policy_version 445178 (0.0010) [2023-12-27 08:43:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 227885056. Throughput: 0: 10255.1, 1: 10373.9. Samples: 227892212. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.420')] [2023-12-27 08:43:17,573][20381] Updated weights for policy 1, policy_version 444853 (0.0005) [2023-12-27 08:43:17,626][20381] Updated weights for policy 1, policy_version 444863 (0.0005) [2023-12-27 08:43:17,676][20381] Updated weights for policy 1, policy_version 444873 (0.0007) [2023-12-27 08:43:17,991][20347] Updated weights for policy 0, policy_version 445188 (0.0009) [2023-12-27 08:43:18,049][20347] Updated weights for policy 0, policy_version 445198 (0.0008) [2023-12-27 08:43:18,111][20347] Updated weights for policy 0, policy_version 445208 (0.0006) [2023-12-27 08:43:18,337][20381] Updated weights for policy 1, policy_version 444883 (0.0007) [2023-12-27 08:43:18,399][20381] Updated weights for policy 1, policy_version 444893 (0.0009) [2023-12-27 08:43:18,453][20381] Updated weights for policy 1, policy_version 444903 (0.0008) [2023-12-27 08:43:18,763][20347] Updated weights for policy 0, policy_version 445218 (0.0008) [2023-12-27 08:43:18,826][20347] Updated weights for policy 0, policy_version 445228 (0.0005) [2023-12-27 08:43:18,889][20347] Updated weights for policy 0, policy_version 445238 (0.0006) [2023-12-27 08:43:18,950][20347] Updated weights for policy 0, policy_version 445248 (0.0005) [2023-12-27 08:43:19,178][20381] Updated weights for policy 1, policy_version 444913 (0.0007) [2023-12-27 08:43:19,238][20381] Updated weights for policy 1, policy_version 444923 (0.0007) [2023-12-27 08:43:19,299][20381] Updated weights for policy 1, policy_version 444933 (0.0007) [2023-12-27 08:43:19,361][20381] Updated weights for policy 1, policy_version 444943 (0.0009) [2023-12-27 08:43:19,510][20347] Updated weights for policy 0, policy_version 445258 (0.0007) [2023-12-27 08:43:19,563][20347] Updated weights for policy 0, policy_version 445268 (0.0008) [2023-12-27 08:43:19,618][20347] Updated weights for policy 0, policy_version 445278 (0.0007) [2023-12-27 08:43:20,066][20381] Updated weights for policy 1, policy_version 444953 (0.0007) [2023-12-27 08:43:20,121][20381] Updated weights for policy 1, policy_version 444963 (0.0008) [2023-12-27 08:43:20,172][20381] Updated weights for policy 1, policy_version 444973 (0.0008) [2023-12-27 08:43:20,330][20347] Updated weights for policy 0, policy_version 445288 (0.0008) [2023-12-27 08:43:20,387][20347] Updated weights for policy 0, policy_version 445298 (0.0008) [2023-12-27 08:43:20,447][20347] Updated weights for policy 0, policy_version 445308 (0.0008) [2023-12-27 08:43:20,904][20381] Updated weights for policy 1, policy_version 444983 (0.0008) [2023-12-27 08:43:20,959][20381] Updated weights for policy 1, policy_version 444993 (0.0007) [2023-12-27 08:43:21,006][20381] Updated weights for policy 1, policy_version 445003 (0.0007) [2023-12-27 08:43:21,210][20347] Updated weights for policy 0, policy_version 445318 (0.0008) [2023-12-27 08:43:21,270][20347] Updated weights for policy 0, policy_version 445328 (0.0009) [2023-12-27 08:43:21,327][20347] Updated weights for policy 0, policy_version 445338 (0.0007) [2023-12-27 08:43:21,760][20381] Updated weights for policy 1, policy_version 445013 (0.0009) [2023-12-27 08:43:21,818][20381] Updated weights for policy 1, policy_version 445023 (0.0009) [2023-12-27 08:43:21,883][20381] Updated weights for policy 1, policy_version 445033 (0.0007) [2023-12-27 08:43:21,953][20347] Updated weights for policy 0, policy_version 445348 (0.0009) [2023-12-27 08:43:22,012][20347] Updated weights for policy 0, policy_version 445358 (0.0010) [2023-12-27 08:43:22,071][20347] Updated weights for policy 0, policy_version 445368 (0.0005) [2023-12-27 08:43:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 227983360. Throughput: 0: 10222.0, 1: 10344.0. Samples: 227953388. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.200')] [2023-12-27 08:43:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000445040_113950720.pth... [2023-12-27 08:43:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000445376_114032640.pth... [2023-12-27 08:43:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000443824_113639424.pth [2023-12-27 08:43:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000444160_113721344.pth [2023-12-27 08:43:22,506][20381] Updated weights for policy 1, policy_version 445043 (0.0006) [2023-12-27 08:43:22,569][20381] Updated weights for policy 1, policy_version 445053 (0.0007) [2023-12-27 08:43:22,587][20292] KL-divergence is very high: 100.1029 [2023-12-27 08:43:22,634][20381] Updated weights for policy 1, policy_version 445063 (0.0008) [2023-12-27 08:43:22,723][20347] Updated weights for policy 0, policy_version 445378 (0.0007) [2023-12-27 08:43:22,789][20347] Updated weights for policy 0, policy_version 445388 (0.0008) [2023-12-27 08:43:22,852][20347] Updated weights for policy 0, policy_version 445398 (0.0006) [2023-12-27 08:43:22,914][20347] Updated weights for policy 0, policy_version 445408 (0.0011) [2023-12-27 08:43:23,372][20381] Updated weights for policy 1, policy_version 445073 (0.0008) [2023-12-27 08:43:23,418][20381] Updated weights for policy 1, policy_version 445083 (0.0010) [2023-12-27 08:43:23,472][20381] Updated weights for policy 1, policy_version 445093 (0.0008) [2023-12-27 08:43:23,483][20347] Updated weights for policy 0, policy_version 445418 (0.0005) [2023-12-27 08:43:23,524][20381] Updated weights for policy 1, policy_version 445103 (0.0006) [2023-12-27 08:43:23,540][20347] Updated weights for policy 0, policy_version 445428 (0.0010) [2023-12-27 08:43:23,600][20347] Updated weights for policy 0, policy_version 445438 (0.0010) [2023-12-27 08:43:24,225][20347] Updated weights for policy 0, policy_version 445448 (0.0010) [2023-12-27 08:43:24,231][20381] Updated weights for policy 1, policy_version 445113 (0.0006) [2023-12-27 08:43:24,282][20381] Updated weights for policy 1, policy_version 445123 (0.0009) [2023-12-27 08:43:24,288][20347] Updated weights for policy 0, policy_version 445458 (0.0008) [2023-12-27 08:43:24,339][20381] Updated weights for policy 1, policy_version 445133 (0.0008) [2023-12-27 08:43:24,345][20347] Updated weights for policy 0, policy_version 445468 (0.0006) [2023-12-27 08:43:24,960][20347] Updated weights for policy 0, policy_version 445478 (0.0008) [2023-12-27 08:43:25,008][20347] Updated weights for policy 0, policy_version 445488 (0.0010) [2023-12-27 08:43:25,033][20381] Updated weights for policy 1, policy_version 445143 (0.0006) [2023-12-27 08:43:25,060][20347] Updated weights for policy 0, policy_version 445498 (0.0009) [2023-12-27 08:43:25,090][20381] Updated weights for policy 1, policy_version 445153 (0.0007) [2023-12-27 08:43:25,149][20381] Updated weights for policy 1, policy_version 445163 (0.0008) [2023-12-27 08:43:25,714][20347] Updated weights for policy 0, policy_version 445508 (0.0005) [2023-12-27 08:43:25,767][20347] Updated weights for policy 0, policy_version 445518 (0.0006) [2023-12-27 08:43:25,826][20347] Updated weights for policy 0, policy_version 445528 (0.0010) [2023-12-27 08:43:25,867][20381] Updated weights for policy 1, policy_version 445173 (0.0007) [2023-12-27 08:43:25,922][20381] Updated weights for policy 1, policy_version 445183 (0.0007) [2023-12-27 08:43:25,981][20381] Updated weights for policy 1, policy_version 445193 (0.0007) [2023-12-27 08:43:26,434][20347] Updated weights for policy 0, policy_version 445538 (0.0011) [2023-12-27 08:43:26,486][20347] Updated weights for policy 0, policy_version 445548 (0.0010) [2023-12-27 08:43:26,539][20347] Updated weights for policy 0, policy_version 445558 (0.0009) [2023-12-27 08:43:26,597][20347] Updated weights for policy 0, policy_version 445568 (0.0007) [2023-12-27 08:43:26,614][20381] Updated weights for policy 1, policy_version 445203 (0.0006) [2023-12-27 08:43:26,662][20381] Updated weights for policy 1, policy_version 445213 (0.0005) [2023-12-27 08:43:26,714][20381] Updated weights for policy 1, policy_version 445223 (0.0005) [2023-12-27 08:43:27,180][20347] Updated weights for policy 0, policy_version 445578 (0.0009) [2023-12-27 08:43:27,237][20347] Updated weights for policy 0, policy_version 445588 (0.0009) [2023-12-27 08:43:27,284][20347] Updated weights for policy 0, policy_version 445598 (0.0009) [2023-12-27 08:43:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 228089856. Throughput: 0: 10303.7, 1: 10345.3. Samples: 228076160. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.120')] [2023-12-27 08:43:27,399][20381] Updated weights for policy 1, policy_version 445233 (0.0007) [2023-12-27 08:43:27,452][20381] Updated weights for policy 1, policy_version 445243 (0.0007) [2023-12-27 08:43:27,505][20381] Updated weights for policy 1, policy_version 445253 (0.0005) [2023-12-27 08:43:27,550][20381] Updated weights for policy 1, policy_version 445263 (0.0005) [2023-12-27 08:43:27,969][20347] Updated weights for policy 0, policy_version 445608 (0.0010) [2023-12-27 08:43:28,025][20347] Updated weights for policy 0, policy_version 445618 (0.0010) [2023-12-27 08:43:28,079][20347] Updated weights for policy 0, policy_version 445628 (0.0010) [2023-12-27 08:43:28,121][20381] Updated weights for policy 1, policy_version 445273 (0.0006) [2023-12-27 08:43:28,179][20381] Updated weights for policy 1, policy_version 445283 (0.0007) [2023-12-27 08:43:28,228][20381] Updated weights for policy 1, policy_version 445293 (0.0005) [2023-12-27 08:43:28,710][20347] Updated weights for policy 0, policy_version 445638 (0.0010) [2023-12-27 08:43:28,763][20347] Updated weights for policy 0, policy_version 445648 (0.0010) [2023-12-27 08:43:28,818][20347] Updated weights for policy 0, policy_version 445658 (0.0010) [2023-12-27 08:43:28,863][20381] Updated weights for policy 1, policy_version 445303 (0.0007) [2023-12-27 08:43:28,919][20381] Updated weights for policy 1, policy_version 445313 (0.0008) [2023-12-27 08:43:28,975][20381] Updated weights for policy 1, policy_version 445323 (0.0008) [2023-12-27 08:43:29,493][20347] Updated weights for policy 0, policy_version 445668 (0.0008) [2023-12-27 08:43:29,548][20347] Updated weights for policy 0, policy_version 445678 (0.0006) [2023-12-27 08:43:29,610][20347] Updated weights for policy 0, policy_version 445688 (0.0006) [2023-12-27 08:43:29,665][20381] Updated weights for policy 1, policy_version 445333 (0.0008) [2023-12-27 08:43:29,720][20381] Updated weights for policy 1, policy_version 445343 (0.0011) [2023-12-27 08:43:29,782][20381] Updated weights for policy 1, policy_version 445353 (0.0010) [2023-12-27 08:43:30,253][20347] Updated weights for policy 0, policy_version 445698 (0.0006) [2023-12-27 08:43:30,314][20347] Updated weights for policy 0, policy_version 445708 (0.0007) [2023-12-27 08:43:30,372][20347] Updated weights for policy 0, policy_version 445718 (0.0007) [2023-12-27 08:43:30,424][20347] Updated weights for policy 0, policy_version 445728 (0.0007) [2023-12-27 08:43:30,469][20381] Updated weights for policy 1, policy_version 445363 (0.0009) [2023-12-27 08:43:30,527][20381] Updated weights for policy 1, policy_version 445373 (0.0006) [2023-12-27 08:43:30,584][20381] Updated weights for policy 1, policy_version 445383 (0.0005) [2023-12-27 08:43:31,091][20347] Updated weights for policy 0, policy_version 445738 (0.0007) [2023-12-27 08:43:31,150][20347] Updated weights for policy 0, policy_version 445748 (0.0007) [2023-12-27 08:43:31,184][20381] Updated weights for policy 1, policy_version 445393 (0.0005) [2023-12-27 08:43:31,212][20347] Updated weights for policy 0, policy_version 445758 (0.0008) [2023-12-27 08:43:31,232][20381] Updated weights for policy 1, policy_version 445403 (0.0007) [2023-12-27 08:43:31,292][20381] Updated weights for policy 1, policy_version 445413 (0.0011) [2023-12-27 08:43:31,349][20381] Updated weights for policy 1, policy_version 445423 (0.0010) [2023-12-27 08:43:31,910][20347] Updated weights for policy 0, policy_version 445768 (0.0007) [2023-12-27 08:43:31,973][20347] Updated weights for policy 0, policy_version 445778 (0.0008) [2023-12-27 08:43:32,008][20381] Updated weights for policy 1, policy_version 445433 (0.0005) [2023-12-27 08:43:32,021][20347] Updated weights for policy 0, policy_version 445788 (0.0007) [2023-12-27 08:43:32,057][20381] Updated weights for policy 1, policy_version 445443 (0.0007) [2023-12-27 08:43:32,116][20381] Updated weights for policy 1, policy_version 445453 (0.0007) [2023-12-27 08:43:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 228196352. Throughput: 0: 10421.3, 1: 10315.3. Samples: 228204188. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:32,363][19170] Avg episode reward: [(0, '990.480'), (1, '971.180')] [2023-12-27 08:43:32,740][20347] Updated weights for policy 0, policy_version 445798 (0.0006) [2023-12-27 08:43:32,758][20381] Updated weights for policy 1, policy_version 445463 (0.0007) [2023-12-27 08:43:32,802][20347] Updated weights for policy 0, policy_version 445808 (0.0006) [2023-12-27 08:43:32,811][20381] Updated weights for policy 1, policy_version 445473 (0.0006) [2023-12-27 08:43:32,855][20347] Updated weights for policy 0, policy_version 445818 (0.0008) [2023-12-27 08:43:32,872][20381] Updated weights for policy 1, policy_version 445483 (0.0006) [2023-12-27 08:43:33,512][20347] Updated weights for policy 0, policy_version 445828 (0.0009) [2023-12-27 08:43:33,541][20381] Updated weights for policy 1, policy_version 445493 (0.0008) [2023-12-27 08:43:33,561][20347] Updated weights for policy 0, policy_version 445838 (0.0008) [2023-12-27 08:43:33,600][20381] Updated weights for policy 1, policy_version 445503 (0.0006) [2023-12-27 08:43:33,616][20347] Updated weights for policy 0, policy_version 445848 (0.0006) [2023-12-27 08:43:33,654][20381] Updated weights for policy 1, policy_version 445513 (0.0005) [2023-12-27 08:43:34,234][20381] Updated weights for policy 1, policy_version 445523 (0.0006) [2023-12-27 08:43:34,295][20292] KL-divergence is very high: 121.1803 [2023-12-27 08:43:34,296][20381] Updated weights for policy 1, policy_version 445533 (0.0006) [2023-12-27 08:43:34,298][20347] Updated weights for policy 0, policy_version 445858 (0.0006) [2023-12-27 08:43:34,300][20292] KL-divergence is very high: 105.6318 [2023-12-27 08:43:34,354][20347] Updated weights for policy 0, policy_version 445868 (0.0006) [2023-12-27 08:43:34,356][20381] Updated weights for policy 1, policy_version 445543 (0.0005) [2023-12-27 08:43:34,411][20347] Updated weights for policy 0, policy_version 445878 (0.0006) [2023-12-27 08:43:34,461][20347] Updated weights for policy 0, policy_version 445888 (0.0008) [2023-12-27 08:43:34,954][20381] Updated weights for policy 1, policy_version 445553 (0.0006) [2023-12-27 08:43:35,010][20381] Updated weights for policy 1, policy_version 445563 (0.0008) [2023-12-27 08:43:35,060][20381] Updated weights for policy 1, policy_version 445573 (0.0007) [2023-12-27 08:43:35,114][20381] Updated weights for policy 1, policy_version 445583 (0.0008) [2023-12-27 08:43:35,203][20347] Updated weights for policy 0, policy_version 445898 (0.0008) [2023-12-27 08:43:35,247][20347] Updated weights for policy 0, policy_version 445908 (0.0007) [2023-12-27 08:43:35,302][20347] Updated weights for policy 0, policy_version 445918 (0.0008) [2023-12-27 08:43:35,813][20381] Updated weights for policy 1, policy_version 445593 (0.0008) [2023-12-27 08:43:35,872][20381] Updated weights for policy 1, policy_version 445603 (0.0007) [2023-12-27 08:43:35,928][20381] Updated weights for policy 1, policy_version 445613 (0.0008) [2023-12-27 08:43:36,011][20347] Updated weights for policy 0, policy_version 445928 (0.0007) [2023-12-27 08:43:36,076][20347] Updated weights for policy 0, policy_version 445938 (0.0008) [2023-12-27 08:43:36,139][20347] Updated weights for policy 0, policy_version 445948 (0.0007) [2023-12-27 08:43:36,605][20381] Updated weights for policy 1, policy_version 445623 (0.0008) [2023-12-27 08:43:36,611][20292] KL-divergence is very high: 335.6378 [2023-12-27 08:43:36,662][20381] Updated weights for policy 1, policy_version 445633 (0.0008) [2023-12-27 08:43:36,694][20292] KL-divergence is very high: 2050.1431 [2023-12-27 08:43:36,708][20381] Updated weights for policy 1, policy_version 445643 (0.0008) [2023-12-27 08:43:36,813][20347] Updated weights for policy 0, policy_version 445958 (0.0005) [2023-12-27 08:43:36,870][20347] Updated weights for policy 0, policy_version 445968 (0.0007) [2023-12-27 08:43:36,921][20347] Updated weights for policy 0, policy_version 445978 (0.0009) [2023-12-27 08:43:37,346][20381] Updated weights for policy 1, policy_version 445653 (0.0008) [2023-12-27 08:43:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.5, 300 sec: 20688.2). Total num frames: 228294656. Throughput: 0: 10372.5, 1: 10361.3. Samples: 228267096. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:37,364][19170] Avg episode reward: [(0, '990.480'), (1, '923.050')] [2023-12-27 08:43:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000445984_114188288.pth... [2023-12-27 08:43:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000444768_113876992.pth [2023-12-27 08:43:37,415][20381] Updated weights for policy 1, policy_version 445663 (0.0008) [2023-12-27 08:43:37,483][20381] Updated weights for policy 1, policy_version 445673 (0.0009) [2023-12-27 08:43:37,505][20292] KL-divergence is very high: 833.1284 [2023-12-27 08:43:37,512][20292] KL-divergence is very high: 649.0063 [2023-12-27 08:43:37,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000445680_114114560.pth... [2023-12-27 08:43:37,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000444464_113803264.pth [2023-12-27 08:43:37,565][20347] Updated weights for policy 0, policy_version 445988 (0.0008) [2023-12-27 08:43:37,611][20347] Updated weights for policy 0, policy_version 445998 (0.0006) [2023-12-27 08:43:37,659][20347] Updated weights for policy 0, policy_version 446008 (0.0008) [2023-12-27 08:43:38,162][20292] KL-divergence is very high: 116.7606 [2023-12-27 08:43:38,175][20381] Updated weights for policy 1, policy_version 445683 (0.0008) [2023-12-27 08:43:38,183][20292] KL-divergence is very high: 185.8544 [2023-12-27 08:43:38,214][20292] KL-divergence is very high: 188.2561 [2023-12-27 08:43:38,235][20381] Updated weights for policy 1, policy_version 445693 (0.0008) [2023-12-27 08:43:38,290][20381] Updated weights for policy 1, policy_version 445703 (0.0008) [2023-12-27 08:43:38,296][20347] Updated weights for policy 0, policy_version 446018 (0.0007) [2023-12-27 08:43:38,356][20347] Updated weights for policy 0, policy_version 446028 (0.0006) [2023-12-27 08:43:38,416][20347] Updated weights for policy 0, policy_version 446038 (0.0006) [2023-12-27 08:43:38,467][20347] Updated weights for policy 0, policy_version 446048 (0.0005) [2023-12-27 08:43:39,044][20381] Updated weights for policy 1, policy_version 445713 (0.0007) [2023-12-27 08:43:39,059][20347] Updated weights for policy 0, policy_version 446058 (0.0006) [2023-12-27 08:43:39,102][20381] Updated weights for policy 1, policy_version 445723 (0.0008) [2023-12-27 08:43:39,119][20347] Updated weights for policy 0, policy_version 446068 (0.0006) [2023-12-27 08:43:39,163][20381] Updated weights for policy 1, policy_version 445733 (0.0006) [2023-12-27 08:43:39,180][20347] Updated weights for policy 0, policy_version 446078 (0.0006) [2023-12-27 08:43:39,218][20381] Updated weights for policy 1, policy_version 445743 (0.0007) [2023-12-27 08:43:39,863][20347] Updated weights for policy 0, policy_version 446088 (0.0008) [2023-12-27 08:43:39,923][20347] Updated weights for policy 0, policy_version 446098 (0.0008) [2023-12-27 08:43:39,966][20292] KL-divergence is very high: 139.4882 [2023-12-27 08:43:39,966][20381] Updated weights for policy 1, policy_version 445753 (0.0006) [2023-12-27 08:43:39,983][20347] Updated weights for policy 0, policy_version 446108 (0.0007) [2023-12-27 08:43:39,989][20292] KL-divergence is very high: 118.4271 [2023-12-27 08:43:40,022][20381] Updated weights for policy 1, policy_version 445763 (0.0007) [2023-12-27 08:43:40,051][20292] KL-divergence is very high: 207.1391 [2023-12-27 08:43:40,073][20381] Updated weights for policy 1, policy_version 445773 (0.0008) [2023-12-27 08:43:40,074][20292] KL-divergence is very high: 228.4085 [2023-12-27 08:43:40,579][20347] Updated weights for policy 0, policy_version 446118 (0.0006) [2023-12-27 08:43:40,634][20347] Updated weights for policy 0, policy_version 446128 (0.0005) [2023-12-27 08:43:40,685][20347] Updated weights for policy 0, policy_version 446138 (0.0006) [2023-12-27 08:43:40,797][20292] KL-divergence is very high: 401.5699 [2023-12-27 08:43:40,818][20292] KL-divergence is very high: 185.9865 [2023-12-27 08:43:40,833][20381] Updated weights for policy 1, policy_version 445783 (0.0008) [2023-12-27 08:43:40,837][20292] KL-divergence is very high: 155.1203 [2023-12-27 08:43:40,856][20292] KL-divergence is very high: 321.9438 [2023-12-27 08:43:40,882][20292] KL-divergence is very high: 129.4137 [2023-12-27 08:43:40,890][20381] Updated weights for policy 1, policy_version 445793 (0.0008) [2023-12-27 08:43:40,905][20292] KL-divergence is very high: 161.0839 [2023-12-27 08:43:40,928][20292] KL-divergence is very high: 168.0275 [2023-12-27 08:43:40,947][20381] Updated weights for policy 1, policy_version 445803 (0.0008) [2023-12-27 08:43:41,341][20347] Updated weights for policy 0, policy_version 446148 (0.0008) [2023-12-27 08:43:41,405][20347] Updated weights for policy 0, policy_version 446158 (0.0010) [2023-12-27 08:43:41,458][20347] Updated weights for policy 0, policy_version 446168 (0.0006) [2023-12-27 08:43:41,723][20381] Updated weights for policy 1, policy_version 445813 (0.0008) [2023-12-27 08:43:41,780][20381] Updated weights for policy 1, policy_version 445823 (0.0007) [2023-12-27 08:43:41,836][20381] Updated weights for policy 1, policy_version 445833 (0.0008) [2023-12-27 08:43:42,073][20347] Updated weights for policy 0, policy_version 446178 (0.0006) [2023-12-27 08:43:42,133][20347] Updated weights for policy 0, policy_version 446188 (0.0008) [2023-12-27 08:43:42,194][20347] Updated weights for policy 0, policy_version 446198 (0.0005) [2023-12-27 08:43:42,254][20347] Updated weights for policy 0, policy_version 446208 (0.0008) [2023-12-27 08:43:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 228401152. Throughput: 0: 10413.2, 1: 10299.6. Samples: 228389900. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:42,363][19170] Avg episode reward: [(0, '990.470'), (1, '817.160')] [2023-12-27 08:43:42,581][20381] Updated weights for policy 1, policy_version 445843 (0.0008) [2023-12-27 08:43:42,641][20381] Updated weights for policy 1, policy_version 445853 (0.0008) [2023-12-27 08:43:42,700][20381] Updated weights for policy 1, policy_version 445863 (0.0006) [2023-12-27 08:43:42,879][20347] Updated weights for policy 0, policy_version 446218 (0.0009) [2023-12-27 08:43:42,924][20347] Updated weights for policy 0, policy_version 446228 (0.0007) [2023-12-27 08:43:42,973][20347] Updated weights for policy 0, policy_version 446238 (0.0009) [2023-12-27 08:43:43,261][20381] Updated weights for policy 1, policy_version 445873 (0.0007) [2023-12-27 08:43:43,314][20381] Updated weights for policy 1, policy_version 445883 (0.0006) [2023-12-27 08:43:43,363][20381] Updated weights for policy 1, policy_version 445893 (0.0005) [2023-12-27 08:43:43,420][20381] Updated weights for policy 1, policy_version 445903 (0.0005) [2023-12-27 08:43:43,737][20347] Updated weights for policy 0, policy_version 446248 (0.0008) [2023-12-27 08:43:43,784][20347] Updated weights for policy 0, policy_version 446258 (0.0007) [2023-12-27 08:43:43,830][20347] Updated weights for policy 0, policy_version 446268 (0.0008) [2023-12-27 08:43:44,049][20381] Updated weights for policy 1, policy_version 445913 (0.0010) [2023-12-27 08:43:44,105][20381] Updated weights for policy 1, policy_version 445923 (0.0011) [2023-12-27 08:43:44,165][20381] Updated weights for policy 1, policy_version 445933 (0.0009) [2023-12-27 08:43:44,506][20347] Updated weights for policy 0, policy_version 446278 (0.0007) [2023-12-27 08:43:44,566][20347] Updated weights for policy 0, policy_version 446288 (0.0008) [2023-12-27 08:43:44,625][20347] Updated weights for policy 0, policy_version 446298 (0.0008) [2023-12-27 08:43:44,842][20381] Updated weights for policy 1, policy_version 445943 (0.0007) [2023-12-27 08:43:44,843][20292] KL-divergence is very high: 118.3221 [2023-12-27 08:43:44,849][20292] KL-divergence is very high: 128.7149 [2023-12-27 08:43:44,855][20292] KL-divergence is very high: 125.2525 [2023-12-27 08:43:44,861][20292] KL-divergence is very high: 101.4914 [2023-12-27 08:43:44,903][20381] Updated weights for policy 1, policy_version 445953 (0.0006) [2023-12-27 08:43:44,954][20381] Updated weights for policy 1, policy_version 445963 (0.0005) [2023-12-27 08:43:44,979][20292] KL-divergence is very high: 274.2208 [2023-12-27 08:43:45,262][20347] Updated weights for policy 0, policy_version 446308 (0.0007) [2023-12-27 08:43:45,311][20347] Updated weights for policy 0, policy_version 446318 (0.0007) [2023-12-27 08:43:45,361][20347] Updated weights for policy 0, policy_version 446328 (0.0006) [2023-12-27 08:43:45,578][20381] Updated weights for policy 1, policy_version 445973 (0.0007) [2023-12-27 08:43:45,629][20381] Updated weights for policy 1, policy_version 445983 (0.0008) [2023-12-27 08:43:45,677][20381] Updated weights for policy 1, policy_version 445993 (0.0008) [2023-12-27 08:43:45,996][20347] Updated weights for policy 0, policy_version 446338 (0.0006) [2023-12-27 08:43:46,053][20347] Updated weights for policy 0, policy_version 446348 (0.0007) [2023-12-27 08:43:46,106][20347] Updated weights for policy 0, policy_version 446358 (0.0008) [2023-12-27 08:43:46,156][20347] Updated weights for policy 0, policy_version 446368 (0.0006) [2023-12-27 08:43:46,356][20381] Updated weights for policy 1, policy_version 446003 (0.0008) [2023-12-27 08:43:46,410][20381] Updated weights for policy 1, policy_version 446013 (0.0007) [2023-12-27 08:43:46,462][20381] Updated weights for policy 1, policy_version 446023 (0.0008) [2023-12-27 08:43:46,785][20347] Updated weights for policy 0, policy_version 446378 (0.0008) [2023-12-27 08:43:46,841][20347] Updated weights for policy 0, policy_version 446388 (0.0008) [2023-12-27 08:43:46,896][20347] Updated weights for policy 0, policy_version 446398 (0.0007) [2023-12-27 08:43:47,094][20381] Updated weights for policy 1, policy_version 446033 (0.0008) [2023-12-27 08:43:47,143][20381] Updated weights for policy 1, policy_version 446043 (0.0010) [2023-12-27 08:43:47,201][20381] Updated weights for policy 1, policy_version 446053 (0.0010) [2023-12-27 08:43:47,252][20381] Updated weights for policy 1, policy_version 446063 (0.0007) [2023-12-27 08:43:47,363][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 228507648. Throughput: 0: 10437.0, 1: 10273.3. Samples: 228515124. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:47,364][19170] Avg episode reward: [(0, '971.940'), (1, '780.680')] [2023-12-27 08:43:47,597][20347] Updated weights for policy 0, policy_version 446408 (0.0008) [2023-12-27 08:43:47,652][20347] Updated weights for policy 0, policy_version 446418 (0.0007) [2023-12-27 08:43:47,702][20347] Updated weights for policy 0, policy_version 446428 (0.0005) [2023-12-27 08:43:47,930][20381] Updated weights for policy 1, policy_version 446073 (0.0010) [2023-12-27 08:43:47,974][20381] Updated weights for policy 1, policy_version 446083 (0.0010) [2023-12-27 08:43:48,032][20381] Updated weights for policy 1, policy_version 446093 (0.0010) [2023-12-27 08:43:48,361][20347] Updated weights for policy 0, policy_version 446438 (0.0008) [2023-12-27 08:43:48,422][20347] Updated weights for policy 0, policy_version 446448 (0.0010) [2023-12-27 08:43:48,477][20347] Updated weights for policy 0, policy_version 446458 (0.0010) [2023-12-27 08:43:48,655][20381] Updated weights for policy 1, policy_version 446103 (0.0011) [2023-12-27 08:43:48,707][20381] Updated weights for policy 1, policy_version 446113 (0.0010) [2023-12-27 08:43:48,752][20381] Updated weights for policy 1, policy_version 446123 (0.0010) [2023-12-27 08:43:49,104][20347] Updated weights for policy 0, policy_version 446468 (0.0010) [2023-12-27 08:43:49,152][20347] Updated weights for policy 0, policy_version 446478 (0.0010) [2023-12-27 08:43:49,200][20347] Updated weights for policy 0, policy_version 446488 (0.0010) [2023-12-27 08:43:49,399][20381] Updated weights for policy 1, policy_version 446133 (0.0010) [2023-12-27 08:43:49,455][20381] Updated weights for policy 1, policy_version 446143 (0.0011) [2023-12-27 08:43:49,520][20381] Updated weights for policy 1, policy_version 446153 (0.0011) [2023-12-27 08:43:49,902][20347] Updated weights for policy 0, policy_version 446498 (0.0010) [2023-12-27 08:43:49,965][20347] Updated weights for policy 0, policy_version 446508 (0.0008) [2023-12-27 08:43:50,030][20347] Updated weights for policy 0, policy_version 446518 (0.0006) [2023-12-27 08:43:50,096][20347] Updated weights for policy 0, policy_version 446528 (0.0008) [2023-12-27 08:43:50,177][20381] Updated weights for policy 1, policy_version 446163 (0.0010) [2023-12-27 08:43:50,241][20381] Updated weights for policy 1, policy_version 446173 (0.0008) [2023-12-27 08:43:50,301][20381] Updated weights for policy 1, policy_version 446183 (0.0008) [2023-12-27 08:43:50,700][20347] Updated weights for policy 0, policy_version 446538 (0.0007) [2023-12-27 08:43:50,761][20347] Updated weights for policy 0, policy_version 446548 (0.0008) [2023-12-27 08:43:50,820][20347] Updated weights for policy 0, policy_version 446558 (0.0007) [2023-12-27 08:43:50,960][20381] Updated weights for policy 1, policy_version 446193 (0.0008) [2023-12-27 08:43:51,019][20381] Updated weights for policy 1, policy_version 446203 (0.0010) [2023-12-27 08:43:51,081][20381] Updated weights for policy 1, policy_version 446213 (0.0010) [2023-12-27 08:43:51,141][20381] Updated weights for policy 1, policy_version 446223 (0.0010) [2023-12-27 08:43:51,466][20347] Updated weights for policy 0, policy_version 446568 (0.0005) [2023-12-27 08:43:51,515][20347] Updated weights for policy 0, policy_version 446578 (0.0005) [2023-12-27 08:43:51,578][20347] Updated weights for policy 0, policy_version 446588 (0.0006) [2023-12-27 08:43:51,762][20381] Updated weights for policy 1, policy_version 446233 (0.0009) [2023-12-27 08:43:51,820][20381] Updated weights for policy 1, policy_version 446244 (0.0009) [2023-12-27 08:43:51,884][20381] Updated weights for policy 1, policy_version 446254 (0.0005) [2023-12-27 08:43:52,218][20347] Updated weights for policy 0, policy_version 446598 (0.0006) [2023-12-27 08:43:52,285][20347] Updated weights for policy 0, policy_version 446608 (0.0007) [2023-12-27 08:43:52,342][20347] Updated weights for policy 0, policy_version 446618 (0.0008) [2023-12-27 08:43:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 228605952. Throughput: 0: 10448.4, 1: 10308.9. Samples: 228579292. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:52,363][19170] Avg episode reward: [(0, '981.470'), (1, '914.720')] [2023-12-27 08:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000446256_114262016.pth... [2023-12-27 08:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000445040_113950720.pth [2023-12-27 08:43:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000446624_114352128.pth... [2023-12-27 08:43:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000445376_114032640.pth [2023-12-27 08:43:52,529][20381] Updated weights for policy 1, policy_version 446264 (0.0009) [2023-12-27 08:43:52,578][20381] Updated weights for policy 1, policy_version 446274 (0.0010) [2023-12-27 08:43:52,627][20381] Updated weights for policy 1, policy_version 446284 (0.0010) [2023-12-27 08:43:52,971][20347] Updated weights for policy 0, policy_version 446628 (0.0007) [2023-12-27 08:43:53,031][20347] Updated weights for policy 0, policy_version 446638 (0.0007) [2023-12-27 08:43:53,078][20347] Updated weights for policy 0, policy_version 446648 (0.0007) [2023-12-27 08:43:53,310][20381] Updated weights for policy 1, policy_version 446294 (0.0010) [2023-12-27 08:43:53,362][20381] Updated weights for policy 1, policy_version 446304 (0.0010) [2023-12-27 08:43:53,411][20381] Updated weights for policy 1, policy_version 446314 (0.0010) [2023-12-27 08:43:53,635][20347] Updated weights for policy 0, policy_version 446658 (0.0006) [2023-12-27 08:43:53,695][20347] Updated weights for policy 0, policy_version 446668 (0.0008) [2023-12-27 08:43:53,752][20347] Updated weights for policy 0, policy_version 446678 (0.0008) [2023-12-27 08:43:53,807][20347] Updated weights for policy 0, policy_version 446688 (0.0008) [2023-12-27 08:43:54,054][20381] Updated weights for policy 1, policy_version 446324 (0.0008) [2023-12-27 08:43:54,111][20381] Updated weights for policy 1, policy_version 446334 (0.0005) [2023-12-27 08:43:54,164][20381] Updated weights for policy 1, policy_version 446344 (0.0006) [2023-12-27 08:43:54,471][20347] Updated weights for policy 0, policy_version 446698 (0.0007) [2023-12-27 08:43:54,508][20291] KL-divergence is very high: 160.5692 [2023-12-27 08:43:54,519][20347] Updated weights for policy 0, policy_version 446708 (0.0007) [2023-12-27 08:43:54,573][20347] Updated weights for policy 0, policy_version 446718 (0.0007) [2023-12-27 08:43:54,772][20381] Updated weights for policy 1, policy_version 446354 (0.0006) [2023-12-27 08:43:54,828][20381] Updated weights for policy 1, policy_version 446364 (0.0005) [2023-12-27 08:43:54,885][20381] Updated weights for policy 1, policy_version 446374 (0.0005) [2023-12-27 08:43:54,938][20381] Updated weights for policy 1, policy_version 446384 (0.0006) [2023-12-27 08:43:55,276][20347] Updated weights for policy 0, policy_version 446728 (0.0010) [2023-12-27 08:43:55,327][20347] Updated weights for policy 0, policy_version 446738 (0.0009) [2023-12-27 08:43:55,382][20347] Updated weights for policy 0, policy_version 446748 (0.0007) [2023-12-27 08:43:55,512][20381] Updated weights for policy 1, policy_version 446394 (0.0005) [2023-12-27 08:43:55,559][20381] Updated weights for policy 1, policy_version 446404 (0.0006) [2023-12-27 08:43:55,611][20381] Updated weights for policy 1, policy_version 446414 (0.0007) [2023-12-27 08:43:55,986][20347] Updated weights for policy 0, policy_version 446758 (0.0010) [2023-12-27 08:43:56,048][20347] Updated weights for policy 0, policy_version 446768 (0.0010) [2023-12-27 08:43:56,112][20347] Updated weights for policy 0, policy_version 446778 (0.0010) [2023-12-27 08:43:56,196][20381] Updated weights for policy 1, policy_version 446424 (0.0005) [2023-12-27 08:43:56,247][20381] Updated weights for policy 1, policy_version 446434 (0.0007) [2023-12-27 08:43:56,304][20381] Updated weights for policy 1, policy_version 446444 (0.0010) [2023-12-27 08:43:56,655][20347] Updated weights for policy 0, policy_version 446788 (0.0008) [2023-12-27 08:43:56,712][20347] Updated weights for policy 0, policy_version 446798 (0.0010) [2023-12-27 08:43:56,770][20347] Updated weights for policy 0, policy_version 446808 (0.0010) [2023-12-27 08:43:56,968][20381] Updated weights for policy 1, policy_version 446454 (0.0007) [2023-12-27 08:43:57,032][20381] Updated weights for policy 1, policy_version 446464 (0.0006) [2023-12-27 08:43:57,079][20381] Updated weights for policy 1, policy_version 446474 (0.0005) [2023-12-27 08:43:57,332][20347] Updated weights for policy 0, policy_version 446818 (0.0010) [2023-12-27 08:43:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 228720640. Throughput: 0: 10545.2, 1: 10383.7. Samples: 228709856. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:43:57,363][19170] Avg episode reward: [(0, '982.070'), (1, '912.510')] [2023-12-27 08:43:57,380][20347] Updated weights for policy 0, policy_version 446828 (0.0005) [2023-12-27 08:43:57,435][20347] Updated weights for policy 0, policy_version 446838 (0.0005) [2023-12-27 08:43:57,490][20347] Updated weights for policy 0, policy_version 446848 (0.0005) [2023-12-27 08:43:57,686][20381] Updated weights for policy 1, policy_version 446484 (0.0008) [2023-12-27 08:43:57,737][20381] Updated weights for policy 1, policy_version 446494 (0.0010) [2023-12-27 08:43:57,789][20381] Updated weights for policy 1, policy_version 446504 (0.0007) [2023-12-27 08:43:58,113][20347] Updated weights for policy 0, policy_version 446858 (0.0006) [2023-12-27 08:43:58,175][20347] Updated weights for policy 0, policy_version 446868 (0.0008) [2023-12-27 08:43:58,236][20347] Updated weights for policy 0, policy_version 446878 (0.0010) [2023-12-27 08:43:58,463][20381] Updated weights for policy 1, policy_version 446514 (0.0010) [2023-12-27 08:43:58,526][20381] Updated weights for policy 1, policy_version 446524 (0.0011) [2023-12-27 08:43:58,589][20381] Updated weights for policy 1, policy_version 446534 (0.0009) [2023-12-27 08:43:58,649][20381] Updated weights for policy 1, policy_version 446544 (0.0009) [2023-12-27 08:43:58,992][20347] Updated weights for policy 0, policy_version 446888 (0.0007) [2023-12-27 08:43:59,052][20347] Updated weights for policy 0, policy_version 446898 (0.0006) [2023-12-27 08:43:59,115][20347] Updated weights for policy 0, policy_version 446908 (0.0006) [2023-12-27 08:43:59,413][20381] Updated weights for policy 1, policy_version 446554 (0.0010) [2023-12-27 08:43:59,470][20381] Updated weights for policy 1, policy_version 446564 (0.0011) [2023-12-27 08:43:59,526][20381] Updated weights for policy 1, policy_version 446574 (0.0009) [2023-12-27 08:43:59,745][20347] Updated weights for policy 0, policy_version 446918 (0.0006) [2023-12-27 08:43:59,797][20347] Updated weights for policy 0, policy_version 446928 (0.0006) [2023-12-27 08:43:59,855][20347] Updated weights for policy 0, policy_version 446938 (0.0007) [2023-12-27 08:44:00,241][20381] Updated weights for policy 1, policy_version 446584 (0.0007) [2023-12-27 08:44:00,295][20381] Updated weights for policy 1, policy_version 446594 (0.0008) [2023-12-27 08:44:00,350][20381] Updated weights for policy 1, policy_version 446604 (0.0008) [2023-12-27 08:44:00,485][20347] Updated weights for policy 0, policy_version 446948 (0.0007) [2023-12-27 08:44:00,533][20347] Updated weights for policy 0, policy_version 446958 (0.0008) [2023-12-27 08:44:00,594][20347] Updated weights for policy 0, policy_version 446968 (0.0008) [2023-12-27 08:44:00,975][20381] Updated weights for policy 1, policy_version 446614 (0.0006) [2023-12-27 08:44:01,027][20381] Updated weights for policy 1, policy_version 446624 (0.0007) [2023-12-27 08:44:01,085][20381] Updated weights for policy 1, policy_version 446634 (0.0007) [2023-12-27 08:44:01,324][20347] Updated weights for policy 0, policy_version 446978 (0.0007) [2023-12-27 08:44:01,391][20347] Updated weights for policy 0, policy_version 446988 (0.0008) [2023-12-27 08:44:01,455][20347] Updated weights for policy 0, policy_version 446998 (0.0006) [2023-12-27 08:44:01,515][20347] Updated weights for policy 0, policy_version 447008 (0.0007) [2023-12-27 08:44:01,781][20381] Updated weights for policy 1, policy_version 446644 (0.0007) [2023-12-27 08:44:01,842][20381] Updated weights for policy 1, policy_version 446654 (0.0007) [2023-12-27 08:44:01,904][20381] Updated weights for policy 1, policy_version 446664 (0.0007) [2023-12-27 08:44:02,184][20347] Updated weights for policy 0, policy_version 447018 (0.0010) [2023-12-27 08:44:02,244][20347] Updated weights for policy 0, policy_version 447028 (0.0010) [2023-12-27 08:44:02,314][20347] Updated weights for policy 0, policy_version 447038 (0.0010) [2023-12-27 08:44:02,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 228827136. Throughput: 0: 10568.4, 1: 10384.5. Samples: 228835096. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:44:02,363][19170] Avg episode reward: [(0, '991.040'), (1, '931.940')] [2023-12-27 08:44:02,531][20381] Updated weights for policy 1, policy_version 446674 (0.0008) [2023-12-27 08:44:02,589][20381] Updated weights for policy 1, policy_version 446684 (0.0010) [2023-12-27 08:44:02,649][20381] Updated weights for policy 1, policy_version 446694 (0.0010) [2023-12-27 08:44:02,707][20381] Updated weights for policy 1, policy_version 446704 (0.0010) [2023-12-27 08:44:02,885][20347] Updated weights for policy 0, policy_version 447048 (0.0010) [2023-12-27 08:44:02,930][20347] Updated weights for policy 0, policy_version 447058 (0.0010) [2023-12-27 08:44:02,981][20347] Updated weights for policy 0, policy_version 447068 (0.0010) [2023-12-27 08:44:03,391][20381] Updated weights for policy 1, policy_version 446714 (0.0010) [2023-12-27 08:44:03,448][20381] Updated weights for policy 1, policy_version 446724 (0.0010) [2023-12-27 08:44:03,498][20381] Updated weights for policy 1, policy_version 446734 (0.0010) [2023-12-27 08:44:03,624][20347] Updated weights for policy 0, policy_version 447078 (0.0007) [2023-12-27 08:44:03,674][20347] Updated weights for policy 0, policy_version 447088 (0.0005) [2023-12-27 08:44:03,725][20347] Updated weights for policy 0, policy_version 447098 (0.0005) [2023-12-27 08:44:04,190][20381] Updated weights for policy 1, policy_version 446744 (0.0011) [2023-12-27 08:44:04,250][20381] Updated weights for policy 1, policy_version 446754 (0.0011) [2023-12-27 08:44:04,307][20347] Updated weights for policy 0, policy_version 447108 (0.0006) [2023-12-27 08:44:04,307][20381] Updated weights for policy 1, policy_version 446764 (0.0010) [2023-12-27 08:44:04,364][20347] Updated weights for policy 0, policy_version 447118 (0.0006) [2023-12-27 08:44:04,421][20347] Updated weights for policy 0, policy_version 447128 (0.0006) [2023-12-27 08:44:05,027][20381] Updated weights for policy 1, policy_version 446774 (0.0008) [2023-12-27 08:44:05,053][20347] Updated weights for policy 0, policy_version 447138 (0.0009) [2023-12-27 08:44:05,076][20292] KL-divergence is very high: 994.7666 [2023-12-27 08:44:05,076][20381] Updated weights for policy 1, policy_version 446784 (0.0006) [2023-12-27 08:44:05,105][20347] Updated weights for policy 0, policy_version 447148 (0.0008) [2023-12-27 08:44:05,114][20292] KL-divergence is very high: 554.7024 [2023-12-27 08:44:05,125][20381] Updated weights for policy 1, policy_version 446794 (0.0005) [2023-12-27 08:44:05,152][20347] Updated weights for policy 0, policy_version 447158 (0.0006) [2023-12-27 08:44:05,205][20347] Updated weights for policy 0, policy_version 447168 (0.0006) [2023-12-27 08:44:05,712][20381] Updated weights for policy 1, policy_version 446804 (0.0006) [2023-12-27 08:44:05,776][20381] Updated weights for policy 1, policy_version 446814 (0.0007) [2023-12-27 08:44:05,800][20347] Updated weights for policy 0, policy_version 447178 (0.0007) [2023-12-27 08:44:05,834][20381] Updated weights for policy 1, policy_version 446824 (0.0010) [2023-12-27 08:44:05,855][20347] Updated weights for policy 0, policy_version 447188 (0.0007) [2023-12-27 08:44:05,906][20347] Updated weights for policy 0, policy_version 447198 (0.0007) [2023-12-27 08:44:06,514][20381] Updated weights for policy 1, policy_version 446834 (0.0007) [2023-12-27 08:44:06,571][20381] Updated weights for policy 1, policy_version 446844 (0.0010) [2023-12-27 08:44:06,607][20347] Updated weights for policy 0, policy_version 447208 (0.0007) [2023-12-27 08:44:06,631][20381] Updated weights for policy 1, policy_version 446854 (0.0006) [2023-12-27 08:44:06,653][20347] Updated weights for policy 0, policy_version 447218 (0.0008) [2023-12-27 08:44:06,698][20381] Updated weights for policy 1, policy_version 446864 (0.0006) [2023-12-27 08:44:06,703][20347] Updated weights for policy 0, policy_version 447228 (0.0008) [2023-12-27 08:44:07,312][20381] Updated weights for policy 1, policy_version 446874 (0.0008) [2023-12-27 08:44:07,362][20381] Updated weights for policy 1, policy_version 446884 (0.0008) [2023-12-27 08:44:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 228925440. Throughput: 0: 10617.6, 1: 10401.8. Samples: 228899260. Policy #0 lag: (min: 4.0, avg: 15.5, max: 36.0) [2023-12-27 08:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '952.060')] [2023-12-27 08:44:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000447232_114507776.pth... [2023-12-27 08:44:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000445984_114188288.pth [2023-12-27 08:44:07,412][20381] Updated weights for policy 1, policy_version 446894 (0.0008) [2023-12-27 08:44:07,420][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000446896_114425856.pth... [2023-12-27 08:44:07,423][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000445680_114114560.pth [2023-12-27 08:44:07,443][20347] Updated weights for policy 0, policy_version 447238 (0.0008) [2023-12-27 08:44:07,493][20347] Updated weights for policy 0, policy_version 447248 (0.0007) [2023-12-27 08:44:07,546][20347] Updated weights for policy 0, policy_version 447258 (0.0008) [2023-12-27 08:44:08,124][20381] Updated weights for policy 1, policy_version 446904 (0.0008) [2023-12-27 08:44:08,180][20381] Updated weights for policy 1, policy_version 446914 (0.0007) [2023-12-27 08:44:08,223][20347] Updated weights for policy 0, policy_version 447268 (0.0007) [2023-12-27 08:44:08,228][20381] Updated weights for policy 1, policy_version 446924 (0.0007) [2023-12-27 08:44:08,280][20347] Updated weights for policy 0, policy_version 447278 (0.0005) [2023-12-27 08:44:08,338][20347] Updated weights for policy 0, policy_version 447288 (0.0006) [2023-12-27 08:44:08,920][20381] Updated weights for policy 1, policy_version 446934 (0.0006) [2023-12-27 08:44:08,981][20381] Updated weights for policy 1, policy_version 446944 (0.0005) [2023-12-27 08:44:09,021][20347] Updated weights for policy 0, policy_version 447298 (0.0008) [2023-12-27 08:44:09,041][20381] Updated weights for policy 1, policy_version 446954 (0.0006) [2023-12-27 08:44:09,078][20347] Updated weights for policy 0, policy_version 447308 (0.0009) [2023-12-27 08:44:09,135][20347] Updated weights for policy 0, policy_version 447318 (0.0007) [2023-12-27 08:44:09,189][20347] Updated weights for policy 0, policy_version 447328 (0.0007) [2023-12-27 08:44:09,701][20381] Updated weights for policy 1, policy_version 446964 (0.0009) [2023-12-27 08:44:09,752][20381] Updated weights for policy 1, policy_version 446974 (0.0008) [2023-12-27 08:44:09,796][20381] Updated weights for policy 1, policy_version 446984 (0.0008) [2023-12-27 08:44:09,901][20347] Updated weights for policy 0, policy_version 447338 (0.0008) [2023-12-27 08:44:09,961][20347] Updated weights for policy 0, policy_version 447348 (0.0008) [2023-12-27 08:44:10,020][20347] Updated weights for policy 0, policy_version 447358 (0.0008) [2023-12-27 08:44:10,467][20381] Updated weights for policy 1, policy_version 446994 (0.0007) [2023-12-27 08:44:10,519][20381] Updated weights for policy 1, policy_version 447004 (0.0008) [2023-12-27 08:44:10,572][20381] Updated weights for policy 1, policy_version 447014 (0.0011) [2023-12-27 08:44:10,628][20381] Updated weights for policy 1, policy_version 447024 (0.0010) [2023-12-27 08:44:10,757][20347] Updated weights for policy 0, policy_version 447368 (0.0008) [2023-12-27 08:44:10,817][20347] Updated weights for policy 0, policy_version 447378 (0.0007) [2023-12-27 08:44:10,872][20347] Updated weights for policy 0, policy_version 447388 (0.0007) [2023-12-27 08:44:11,338][20381] Updated weights for policy 1, policy_version 447034 (0.0011) [2023-12-27 08:44:11,407][20381] Updated weights for policy 1, policy_version 447044 (0.0008) [2023-12-27 08:44:11,468][20381] Updated weights for policy 1, policy_version 447054 (0.0010) [2023-12-27 08:44:11,591][20347] Updated weights for policy 0, policy_version 447398 (0.0007) [2023-12-27 08:44:11,650][20347] Updated weights for policy 0, policy_version 447408 (0.0008) [2023-12-27 08:44:11,717][20347] Updated weights for policy 0, policy_version 447418 (0.0009) [2023-12-27 08:44:12,135][20381] Updated weights for policy 1, policy_version 447064 (0.0007) [2023-12-27 08:44:12,197][20381] Updated weights for policy 1, policy_version 447074 (0.0006) [2023-12-27 08:44:12,258][20381] Updated weights for policy 1, policy_version 447084 (0.0008) [2023-12-27 08:44:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 229031936. Throughput: 0: 10545.0, 1: 10479.9. Samples: 229022284. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '962.010')] [2023-12-27 08:44:12,410][20347] Updated weights for policy 0, policy_version 447428 (0.0009) [2023-12-27 08:44:12,469][20347] Updated weights for policy 0, policy_version 447438 (0.0010) [2023-12-27 08:44:12,524][20347] Updated weights for policy 0, policy_version 447448 (0.0010) [2023-12-27 08:44:12,920][20381] Updated weights for policy 1, policy_version 447094 (0.0011) [2023-12-27 08:44:12,978][20381] Updated weights for policy 1, policy_version 447104 (0.0010) [2023-12-27 08:44:13,030][20381] Updated weights for policy 1, policy_version 447114 (0.0010) [2023-12-27 08:44:13,175][20347] Updated weights for policy 0, policy_version 447458 (0.0008) [2023-12-27 08:44:13,230][20347] Updated weights for policy 0, policy_version 447468 (0.0009) [2023-12-27 08:44:13,284][20347] Updated weights for policy 0, policy_version 447478 (0.0006) [2023-12-27 08:44:13,337][20347] Updated weights for policy 0, policy_version 447488 (0.0008) [2023-12-27 08:44:13,696][20381] Updated weights for policy 1, policy_version 447124 (0.0010) [2023-12-27 08:44:13,747][20381] Updated weights for policy 1, policy_version 447134 (0.0010) [2023-12-27 08:44:13,803][20381] Updated weights for policy 1, policy_version 447144 (0.0010) [2023-12-27 08:44:14,014][20347] Updated weights for policy 0, policy_version 447498 (0.0005) [2023-12-27 08:44:14,076][20347] Updated weights for policy 0, policy_version 447508 (0.0005) [2023-12-27 08:44:14,143][20347] Updated weights for policy 0, policy_version 447518 (0.0008) [2023-12-27 08:44:14,417][20381] Updated weights for policy 1, policy_version 447154 (0.0010) [2023-12-27 08:44:14,481][20381] Updated weights for policy 1, policy_version 447164 (0.0011) [2023-12-27 08:44:14,537][20381] Updated weights for policy 1, policy_version 447174 (0.0010) [2023-12-27 08:44:14,595][20381] Updated weights for policy 1, policy_version 447184 (0.0010) [2023-12-27 08:44:14,717][20347] Updated weights for policy 0, policy_version 447528 (0.0006) [2023-12-27 08:44:14,777][20347] Updated weights for policy 0, policy_version 447538 (0.0007) [2023-12-27 08:44:14,831][20347] Updated weights for policy 0, policy_version 447548 (0.0006) [2023-12-27 08:44:15,283][20381] Updated weights for policy 1, policy_version 447194 (0.0011) [2023-12-27 08:44:15,318][20292] KL-divergence is very high: 124.8913 [2023-12-27 08:44:15,324][20292] KL-divergence is very high: 626.2085 [2023-12-27 08:44:15,340][20381] Updated weights for policy 1, policy_version 447204 (0.0011) [2023-12-27 08:44:15,371][20292] KL-divergence is very high: 225.2061 [2023-12-27 08:44:15,399][20381] Updated weights for policy 1, policy_version 447214 (0.0011) [2023-12-27 08:44:15,488][20347] Updated weights for policy 0, policy_version 447558 (0.0007) [2023-12-27 08:44:15,540][20347] Updated weights for policy 0, policy_version 447568 (0.0010) [2023-12-27 08:44:15,588][20347] Updated weights for policy 0, policy_version 447578 (0.0010) [2023-12-27 08:44:16,079][20381] Updated weights for policy 1, policy_version 447224 (0.0006) [2023-12-27 08:44:16,128][20381] Updated weights for policy 1, policy_version 447234 (0.0006) [2023-12-27 08:44:16,159][20347] Updated weights for policy 0, policy_version 447588 (0.0009) [2023-12-27 08:44:16,176][20381] Updated weights for policy 1, policy_version 447244 (0.0010) [2023-12-27 08:44:16,213][20347] Updated weights for policy 0, policy_version 447598 (0.0008) [2023-12-27 08:44:16,261][20347] Updated weights for policy 0, policy_version 447608 (0.0008) [2023-12-27 08:44:16,747][20381] Updated weights for policy 1, policy_version 447254 (0.0009) [2023-12-27 08:44:16,796][20381] Updated weights for policy 1, policy_version 447264 (0.0010) [2023-12-27 08:44:16,850][20381] Updated weights for policy 1, policy_version 447274 (0.0010) [2023-12-27 08:44:16,954][20347] Updated weights for policy 0, policy_version 447618 (0.0008) [2023-12-27 08:44:17,017][20347] Updated weights for policy 0, policy_version 447628 (0.0010) [2023-12-27 08:44:17,076][20347] Updated weights for policy 0, policy_version 447638 (0.0010) [2023-12-27 08:44:17,137][20347] Updated weights for policy 0, policy_version 447648 (0.0010) [2023-12-27 08:44:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 229138432. Throughput: 0: 10547.2, 1: 10462.6. Samples: 229149628. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.520')] [2023-12-27 08:44:17,475][20381] Updated weights for policy 1, policy_version 447284 (0.0007) [2023-12-27 08:44:17,533][20381] Updated weights for policy 1, policy_version 447294 (0.0010) [2023-12-27 08:44:17,588][20381] Updated weights for policy 1, policy_version 447304 (0.0010) [2023-12-27 08:44:17,802][20347] Updated weights for policy 0, policy_version 447658 (0.0010) [2023-12-27 08:44:17,857][20347] Updated weights for policy 0, policy_version 447668 (0.0010) [2023-12-27 08:44:17,914][20347] Updated weights for policy 0, policy_version 447678 (0.0010) [2023-12-27 08:44:18,168][20381] Updated weights for policy 1, policy_version 447314 (0.0009) [2023-12-27 08:44:18,226][20381] Updated weights for policy 1, policy_version 447324 (0.0005) [2023-12-27 08:44:18,279][20381] Updated weights for policy 1, policy_version 447334 (0.0005) [2023-12-27 08:44:18,337][20381] Updated weights for policy 1, policy_version 447344 (0.0006) [2023-12-27 08:44:18,578][20347] Updated weights for policy 0, policy_version 447688 (0.0010) [2023-12-27 08:44:18,633][20347] Updated weights for policy 0, policy_version 447698 (0.0010) [2023-12-27 08:44:18,688][20347] Updated weights for policy 0, policy_version 447708 (0.0006) [2023-12-27 08:44:18,927][20381] Updated weights for policy 1, policy_version 447354 (0.0007) [2023-12-27 08:44:18,977][20381] Updated weights for policy 1, policy_version 447364 (0.0008) [2023-12-27 08:44:19,025][20381] Updated weights for policy 1, policy_version 447374 (0.0007) [2023-12-27 08:44:19,333][20347] Updated weights for policy 0, policy_version 447718 (0.0008) [2023-12-27 08:44:19,389][20347] Updated weights for policy 0, policy_version 447728 (0.0010) [2023-12-27 08:44:19,438][20347] Updated weights for policy 0, policy_version 447738 (0.0010) [2023-12-27 08:44:19,726][20381] Updated weights for policy 1, policy_version 447384 (0.0010) [2023-12-27 08:44:19,788][20381] Updated weights for policy 1, policy_version 447394 (0.0011) [2023-12-27 08:44:19,845][20381] Updated weights for policy 1, policy_version 447404 (0.0011) [2023-12-27 08:44:20,125][20347] Updated weights for policy 0, policy_version 447748 (0.0010) [2023-12-27 08:44:20,180][20347] Updated weights for policy 0, policy_version 447758 (0.0010) [2023-12-27 08:44:20,236][20347] Updated weights for policy 0, policy_version 447768 (0.0011) [2023-12-27 08:44:20,508][20381] Updated weights for policy 1, policy_version 447414 (0.0009) [2023-12-27 08:44:20,569][20381] Updated weights for policy 1, policy_version 447424 (0.0008) [2023-12-27 08:44:20,634][20381] Updated weights for policy 1, policy_version 447434 (0.0008) [2023-12-27 08:44:20,936][20347] Updated weights for policy 0, policy_version 447778 (0.0010) [2023-12-27 08:44:20,986][20347] Updated weights for policy 0, policy_version 447788 (0.0009) [2023-12-27 08:44:21,047][20347] Updated weights for policy 0, policy_version 447798 (0.0008) [2023-12-27 08:44:21,096][20347] Updated weights for policy 0, policy_version 447808 (0.0005) [2023-12-27 08:44:21,321][20381] Updated weights for policy 1, policy_version 447444 (0.0006) [2023-12-27 08:44:21,396][20381] Updated weights for policy 1, policy_version 447454 (0.0008) [2023-12-27 08:44:21,453][20381] Updated weights for policy 1, policy_version 447464 (0.0008) [2023-12-27 08:44:21,859][20347] Updated weights for policy 0, policy_version 447818 (0.0008) [2023-12-27 08:44:21,922][20347] Updated weights for policy 0, policy_version 447828 (0.0006) [2023-12-27 08:44:21,982][20347] Updated weights for policy 0, policy_version 447838 (0.0007) [2023-12-27 08:44:22,148][20381] Updated weights for policy 1, policy_version 447474 (0.0007) [2023-12-27 08:44:22,207][20381] Updated weights for policy 1, policy_version 447484 (0.0008) [2023-12-27 08:44:22,271][20381] Updated weights for policy 1, policy_version 447494 (0.0008) [2023-12-27 08:44:22,331][20381] Updated weights for policy 1, policy_version 447504 (0.0008) [2023-12-27 08:44:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 229244928. Throughput: 0: 10564.1, 1: 10454.2. Samples: 229212916. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '942.130')] [2023-12-27 08:44:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000447504_114581504.pth... [2023-12-27 08:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000447840_114663424.pth... [2023-12-27 08:44:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000446256_114262016.pth [2023-12-27 08:44:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000446624_114352128.pth [2023-12-27 08:44:22,646][20347] Updated weights for policy 0, policy_version 447848 (0.0008) [2023-12-27 08:44:22,709][20347] Updated weights for policy 0, policy_version 447858 (0.0008) [2023-12-27 08:44:22,763][20347] Updated weights for policy 0, policy_version 447868 (0.0008) [2023-12-27 08:44:22,989][20381] Updated weights for policy 1, policy_version 447514 (0.0006) [2023-12-27 08:44:23,044][20381] Updated weights for policy 1, policy_version 447524 (0.0008) [2023-12-27 08:44:23,106][20381] Updated weights for policy 1, policy_version 447534 (0.0007) [2023-12-27 08:44:23,460][20347] Updated weights for policy 0, policy_version 447878 (0.0007) [2023-12-27 08:44:23,516][20347] Updated weights for policy 0, policy_version 447888 (0.0007) [2023-12-27 08:44:23,566][20347] Updated weights for policy 0, policy_version 447898 (0.0008) [2023-12-27 08:44:23,716][20381] Updated weights for policy 1, policy_version 447544 (0.0008) [2023-12-27 08:44:23,765][20381] Updated weights for policy 1, policy_version 447554 (0.0008) [2023-12-27 08:44:23,815][20381] Updated weights for policy 1, policy_version 447564 (0.0005) [2023-12-27 08:44:24,324][20347] Updated weights for policy 0, policy_version 447908 (0.0007) [2023-12-27 08:44:24,376][20381] Updated weights for policy 1, policy_version 447574 (0.0006) [2023-12-27 08:44:24,378][20347] Updated weights for policy 0, policy_version 447918 (0.0008) [2023-12-27 08:44:24,431][20381] Updated weights for policy 1, policy_version 447584 (0.0005) [2023-12-27 08:44:24,432][20347] Updated weights for policy 0, policy_version 447928 (0.0007) [2023-12-27 08:44:24,488][20381] Updated weights for policy 1, policy_version 447594 (0.0006) [2023-12-27 08:44:25,115][20347] Updated weights for policy 0, policy_version 447938 (0.0007) [2023-12-27 08:44:25,125][20381] Updated weights for policy 1, policy_version 447604 (0.0005) [2023-12-27 08:44:25,172][20347] Updated weights for policy 0, policy_version 447948 (0.0007) [2023-12-27 08:44:25,179][20381] Updated weights for policy 1, policy_version 447614 (0.0005) [2023-12-27 08:44:25,225][20381] Updated weights for policy 1, policy_version 447624 (0.0005) [2023-12-27 08:44:25,227][20347] Updated weights for policy 0, policy_version 447958 (0.0007) [2023-12-27 08:44:25,279][20347] Updated weights for policy 0, policy_version 447968 (0.0006) [2023-12-27 08:44:25,820][20347] Updated weights for policy 0, policy_version 447978 (0.0005) [2023-12-27 08:44:25,880][20347] Updated weights for policy 0, policy_version 447988 (0.0006) [2023-12-27 08:44:25,887][20381] Updated weights for policy 1, policy_version 447634 (0.0006) [2023-12-27 08:44:25,935][20347] Updated weights for policy 0, policy_version 447998 (0.0007) [2023-12-27 08:44:25,939][20381] Updated weights for policy 1, policy_version 447644 (0.0010) [2023-12-27 08:44:25,990][20381] Updated weights for policy 1, policy_version 447654 (0.0010) [2023-12-27 08:44:26,040][20381] Updated weights for policy 1, policy_version 447664 (0.0010) [2023-12-27 08:44:26,477][20347] Updated weights for policy 0, policy_version 448008 (0.0006) [2023-12-27 08:44:26,524][20347] Updated weights for policy 0, policy_version 448018 (0.0007) [2023-12-27 08:44:26,574][20347] Updated weights for policy 0, policy_version 448028 (0.0007) [2023-12-27 08:44:26,583][20291] KL-divergence is very high: 138.5831 [2023-12-27 08:44:26,587][20291] KL-divergence is very high: 116.8349 [2023-12-27 08:44:26,728][20381] Updated weights for policy 1, policy_version 447674 (0.0010) [2023-12-27 08:44:26,775][20381] Updated weights for policy 1, policy_version 447684 (0.0010) [2023-12-27 08:44:26,827][20381] Updated weights for policy 1, policy_version 447694 (0.0010) [2023-12-27 08:44:27,275][20347] Updated weights for policy 0, policy_version 448038 (0.0006) [2023-12-27 08:44:27,336][20347] Updated weights for policy 0, policy_version 448048 (0.0006) [2023-12-27 08:44:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 229343232. Throughput: 0: 10491.2, 1: 10559.7. Samples: 229337188. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:27,363][19170] Avg episode reward: [(0, '990.720'), (1, '951.690')] [2023-12-27 08:44:27,398][20347] Updated weights for policy 0, policy_version 448058 (0.0008) [2023-12-27 08:44:27,536][20381] Updated weights for policy 1, policy_version 447704 (0.0010) [2023-12-27 08:44:27,581][20381] Updated weights for policy 1, policy_version 447714 (0.0010) [2023-12-27 08:44:27,629][20381] Updated weights for policy 1, policy_version 447724 (0.0010) [2023-12-27 08:44:28,086][20347] Updated weights for policy 0, policy_version 448068 (0.0007) [2023-12-27 08:44:28,143][20347] Updated weights for policy 0, policy_version 448078 (0.0007) [2023-12-27 08:44:28,193][20347] Updated weights for policy 0, policy_version 448088 (0.0007) [2023-12-27 08:44:28,252][20381] Updated weights for policy 1, policy_version 447734 (0.0010) [2023-12-27 08:44:28,305][20381] Updated weights for policy 1, policy_version 447744 (0.0010) [2023-12-27 08:44:28,365][20381] Updated weights for policy 1, policy_version 447754 (0.0010) [2023-12-27 08:44:28,896][20347] Updated weights for policy 0, policy_version 448098 (0.0007) [2023-12-27 08:44:28,942][20347] Updated weights for policy 0, policy_version 448108 (0.0007) [2023-12-27 08:44:28,996][20347] Updated weights for policy 0, policy_version 448118 (0.0007) [2023-12-27 08:44:29,049][20347] Updated weights for policy 0, policy_version 448128 (0.0007) [2023-12-27 08:44:29,054][20381] Updated weights for policy 1, policy_version 447764 (0.0010) [2023-12-27 08:44:29,111][20381] Updated weights for policy 1, policy_version 447774 (0.0009) [2023-12-27 08:44:29,165][20381] Updated weights for policy 1, policy_version 447784 (0.0005) [2023-12-27 08:44:29,750][20347] Updated weights for policy 0, policy_version 448138 (0.0005) [2023-12-27 08:44:29,798][20381] Updated weights for policy 1, policy_version 447794 (0.0006) [2023-12-27 08:44:29,807][20347] Updated weights for policy 0, policy_version 448148 (0.0005) [2023-12-27 08:44:29,858][20381] Updated weights for policy 1, policy_version 447804 (0.0009) [2023-12-27 08:44:29,872][20347] Updated weights for policy 0, policy_version 448158 (0.0008) [2023-12-27 08:44:29,914][20381] Updated weights for policy 1, policy_version 447814 (0.0008) [2023-12-27 08:44:29,966][20381] Updated weights for policy 1, policy_version 447824 (0.0007) [2023-12-27 08:44:30,535][20347] Updated weights for policy 0, policy_version 448168 (0.0010) [2023-12-27 08:44:30,582][20381] Updated weights for policy 1, policy_version 447834 (0.0008) [2023-12-27 08:44:30,587][20347] Updated weights for policy 0, policy_version 448178 (0.0010) [2023-12-27 08:44:30,635][20381] Updated weights for policy 1, policy_version 447844 (0.0008) [2023-12-27 08:44:30,638][20347] Updated weights for policy 0, policy_version 448188 (0.0010) [2023-12-27 08:44:30,691][20381] Updated weights for policy 1, policy_version 447854 (0.0009) [2023-12-27 08:44:31,316][20381] Updated weights for policy 1, policy_version 447864 (0.0006) [2023-12-27 08:44:31,341][20347] Updated weights for policy 0, policy_version 448198 (0.0009) [2023-12-27 08:44:31,383][20381] Updated weights for policy 1, policy_version 447874 (0.0008) [2023-12-27 08:44:31,400][20347] Updated weights for policy 0, policy_version 448208 (0.0008) [2023-12-27 08:44:31,446][20381] Updated weights for policy 1, policy_version 447884 (0.0008) [2023-12-27 08:44:31,451][20347] Updated weights for policy 0, policy_version 448218 (0.0007) [2023-12-27 08:44:32,075][20381] Updated weights for policy 1, policy_version 447894 (0.0006) [2023-12-27 08:44:32,136][20381] Updated weights for policy 1, policy_version 447904 (0.0006) [2023-12-27 08:44:32,194][20347] Updated weights for policy 0, policy_version 448228 (0.0006) [2023-12-27 08:44:32,203][20381] Updated weights for policy 1, policy_version 447914 (0.0008) [2023-12-27 08:44:32,252][20347] Updated weights for policy 0, policy_version 448238 (0.0010) [2023-12-27 08:44:32,305][20347] Updated weights for policy 0, policy_version 448248 (0.0009) [2023-12-27 08:44:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 229457920. Throughput: 0: 10466.3, 1: 10582.4. Samples: 229462308. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:32,363][19170] Avg episode reward: [(0, '990.720'), (1, '971.020')] [2023-12-27 08:44:32,802][20381] Updated weights for policy 1, policy_version 447924 (0.0008) [2023-12-27 08:44:32,865][20381] Updated weights for policy 1, policy_version 447934 (0.0008) [2023-12-27 08:44:32,928][20381] Updated weights for policy 1, policy_version 447944 (0.0008) [2023-12-27 08:44:32,976][20347] Updated weights for policy 0, policy_version 448258 (0.0007) [2023-12-27 08:44:33,030][20347] Updated weights for policy 0, policy_version 448268 (0.0010) [2023-12-27 08:44:33,083][20347] Updated weights for policy 0, policy_version 448278 (0.0010) [2023-12-27 08:44:33,137][20347] Updated weights for policy 0, policy_version 448288 (0.0010) [2023-12-27 08:44:33,483][20381] Updated weights for policy 1, policy_version 447954 (0.0007) [2023-12-27 08:44:33,535][20381] Updated weights for policy 1, policy_version 447964 (0.0007) [2023-12-27 08:44:33,592][20381] Updated weights for policy 1, policy_version 447974 (0.0007) [2023-12-27 08:44:33,642][20381] Updated weights for policy 1, policy_version 447984 (0.0007) [2023-12-27 08:44:33,822][20347] Updated weights for policy 0, policy_version 448298 (0.0010) [2023-12-27 08:44:33,868][20347] Updated weights for policy 0, policy_version 448308 (0.0009) [2023-12-27 08:44:33,914][20347] Updated weights for policy 0, policy_version 448318 (0.0005) [2023-12-27 08:44:34,217][20381] Updated weights for policy 1, policy_version 447994 (0.0008) [2023-12-27 08:44:34,283][20381] Updated weights for policy 1, policy_version 448004 (0.0009) [2023-12-27 08:44:34,339][20381] Updated weights for policy 1, policy_version 448014 (0.0008) [2023-12-27 08:44:34,535][20347] Updated weights for policy 0, policy_version 448328 (0.0007) [2023-12-27 08:44:34,599][20347] Updated weights for policy 0, policy_version 448338 (0.0007) [2023-12-27 08:44:34,663][20347] Updated weights for policy 0, policy_version 448348 (0.0006) [2023-12-27 08:44:35,014][20381] Updated weights for policy 1, policy_version 448024 (0.0006) [2023-12-27 08:44:35,062][20381] Updated weights for policy 1, policy_version 448034 (0.0005) [2023-12-27 08:44:35,112][20381] Updated weights for policy 1, policy_version 448044 (0.0005) [2023-12-27 08:44:35,230][20347] Updated weights for policy 0, policy_version 448358 (0.0006) [2023-12-27 08:44:35,286][20347] Updated weights for policy 0, policy_version 448368 (0.0006) [2023-12-27 08:44:35,352][20347] Updated weights for policy 0, policy_version 448378 (0.0005) [2023-12-27 08:44:35,676][20381] Updated weights for policy 1, policy_version 448054 (0.0008) [2023-12-27 08:44:35,729][20381] Updated weights for policy 1, policy_version 448064 (0.0008) [2023-12-27 08:44:35,776][20381] Updated weights for policy 1, policy_version 448074 (0.0007) [2023-12-27 08:44:36,019][20347] Updated weights for policy 0, policy_version 448388 (0.0006) [2023-12-27 08:44:36,076][20347] Updated weights for policy 0, policy_version 448398 (0.0005) [2023-12-27 08:44:36,138][20347] Updated weights for policy 0, policy_version 448408 (0.0008) [2023-12-27 08:44:36,548][20381] Updated weights for policy 1, policy_version 448084 (0.0007) [2023-12-27 08:44:36,604][20381] Updated weights for policy 1, policy_version 448094 (0.0009) [2023-12-27 08:44:36,672][20381] Updated weights for policy 1, policy_version 448104 (0.0005) [2023-12-27 08:44:36,728][20347] Updated weights for policy 0, policy_version 448418 (0.0007) [2023-12-27 08:44:36,796][20347] Updated weights for policy 0, policy_version 448428 (0.0006) [2023-12-27 08:44:36,847][20347] Updated weights for policy 0, policy_version 448438 (0.0006) [2023-12-27 08:44:36,900][20347] Updated weights for policy 0, policy_version 448448 (0.0010) [2023-12-27 08:44:37,305][20381] Updated weights for policy 1, policy_version 448114 (0.0006) [2023-12-27 08:44:37,354][20381] Updated weights for policy 1, policy_version 448124 (0.0006) [2023-12-27 08:44:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.2, 300 sec: 20716.0). Total num frames: 229556224. Throughput: 0: 10471.3, 1: 10609.7. Samples: 229527936. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.390')] [2023-12-27 08:44:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000448448_114819072.pth... [2023-12-27 08:44:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000447232_114507776.pth [2023-12-27 08:44:37,410][20381] Updated weights for policy 1, policy_version 448134 (0.0008) [2023-12-27 08:44:37,462][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000448144_114745344.pth... [2023-12-27 08:44:37,463][20381] Updated weights for policy 1, policy_version 448144 (0.0011) [2023-12-27 08:44:37,466][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000446896_114425856.pth [2023-12-27 08:44:37,552][20347] Updated weights for policy 0, policy_version 448458 (0.0010) [2023-12-27 08:44:37,604][20347] Updated weights for policy 0, policy_version 448468 (0.0010) [2023-12-27 08:44:37,650][20347] Updated weights for policy 0, policy_version 448478 (0.0010) [2023-12-27 08:44:38,150][20381] Updated weights for policy 1, policy_version 448154 (0.0005) [2023-12-27 08:44:38,205][20381] Updated weights for policy 1, policy_version 448164 (0.0005) [2023-12-27 08:44:38,257][20381] Updated weights for policy 1, policy_version 448174 (0.0006) [2023-12-27 08:44:38,358][20347] Updated weights for policy 0, policy_version 448488 (0.0010) [2023-12-27 08:44:38,404][20347] Updated weights for policy 0, policy_version 448498 (0.0010) [2023-12-27 08:44:38,449][20347] Updated weights for policy 0, policy_version 448508 (0.0010) [2023-12-27 08:44:38,909][20381] Updated weights for policy 1, policy_version 448184 (0.0007) [2023-12-27 08:44:38,956][20381] Updated weights for policy 1, policy_version 448194 (0.0007) [2023-12-27 08:44:39,001][20381] Updated weights for policy 1, policy_version 448204 (0.0007) [2023-12-27 08:44:39,158][20347] Updated weights for policy 0, policy_version 448518 (0.0007) [2023-12-27 08:44:39,216][20347] Updated weights for policy 0, policy_version 448528 (0.0006) [2023-12-27 08:44:39,279][20347] Updated weights for policy 0, policy_version 448538 (0.0010) [2023-12-27 08:44:39,628][20381] Updated weights for policy 1, policy_version 448214 (0.0007) [2023-12-27 08:44:39,678][20381] Updated weights for policy 1, policy_version 448224 (0.0008) [2023-12-27 08:44:39,735][20381] Updated weights for policy 1, policy_version 448234 (0.0010) [2023-12-27 08:44:39,881][20347] Updated weights for policy 0, policy_version 448548 (0.0009) [2023-12-27 08:44:39,944][20347] Updated weights for policy 0, policy_version 448558 (0.0008) [2023-12-27 08:44:39,998][20347] Updated weights for policy 0, policy_version 448568 (0.0008) [2023-12-27 08:44:40,454][20381] Updated weights for policy 1, policy_version 448244 (0.0007) [2023-12-27 08:44:40,513][20381] Updated weights for policy 1, policy_version 448254 (0.0008) [2023-12-27 08:44:40,573][20381] Updated weights for policy 1, policy_version 448264 (0.0006) [2023-12-27 08:44:40,654][20347] Updated weights for policy 0, policy_version 448578 (0.0007) [2023-12-27 08:44:40,709][20347] Updated weights for policy 0, policy_version 448588 (0.0007) [2023-12-27 08:44:40,764][20347] Updated weights for policy 0, policy_version 448598 (0.0007) [2023-12-27 08:44:40,819][20347] Updated weights for policy 0, policy_version 448608 (0.0008) [2023-12-27 08:44:41,255][20381] Updated weights for policy 1, policy_version 448274 (0.0008) [2023-12-27 08:44:41,315][20381] Updated weights for policy 1, policy_version 448284 (0.0007) [2023-12-27 08:44:41,384][20381] Updated weights for policy 1, policy_version 448294 (0.0008) [2023-12-27 08:44:41,442][20381] Updated weights for policy 1, policy_version 448304 (0.0008) [2023-12-27 08:44:41,482][20347] Updated weights for policy 0, policy_version 448618 (0.0010) [2023-12-27 08:44:41,532][20347] Updated weights for policy 0, policy_version 448628 (0.0011) [2023-12-27 08:44:41,585][20347] Updated weights for policy 0, policy_version 448638 (0.0011) [2023-12-27 08:44:42,128][20381] Updated weights for policy 1, policy_version 448314 (0.0007) [2023-12-27 08:44:42,185][20381] Updated weights for policy 1, policy_version 448324 (0.0009) [2023-12-27 08:44:42,238][20381] Updated weights for policy 1, policy_version 448334 (0.0009) [2023-12-27 08:44:42,285][20347] Updated weights for policy 0, policy_version 448648 (0.0009) [2023-12-27 08:44:42,345][20347] Updated weights for policy 0, policy_version 448658 (0.0011) [2023-12-27 08:44:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 229662720. Throughput: 0: 10437.9, 1: 10529.0. Samples: 229653364. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.800')] [2023-12-27 08:44:42,411][20347] Updated weights for policy 0, policy_version 448668 (0.0010) [2023-12-27 08:44:42,935][20381] Updated weights for policy 1, policy_version 448344 (0.0008) [2023-12-27 08:44:42,984][20381] Updated weights for policy 1, policy_version 448354 (0.0008) [2023-12-27 08:44:43,042][20381] Updated weights for policy 1, policy_version 448364 (0.0008) [2023-12-27 08:44:43,095][20347] Updated weights for policy 0, policy_version 448678 (0.0011) [2023-12-27 08:44:43,151][20347] Updated weights for policy 0, policy_version 448688 (0.0010) [2023-12-27 08:44:43,206][20347] Updated weights for policy 0, policy_version 448698 (0.0010) [2023-12-27 08:44:43,726][20381] Updated weights for policy 1, policy_version 448374 (0.0006) [2023-12-27 08:44:43,780][20381] Updated weights for policy 1, policy_version 448384 (0.0007) [2023-12-27 08:44:43,827][20381] Updated weights for policy 1, policy_version 448394 (0.0007) [2023-12-27 08:44:43,856][20347] Updated weights for policy 0, policy_version 448708 (0.0010) [2023-12-27 08:44:43,901][20347] Updated weights for policy 0, policy_version 448718 (0.0010) [2023-12-27 08:44:43,956][20347] Updated weights for policy 0, policy_version 448728 (0.0010) [2023-12-27 08:44:44,553][20381] Updated weights for policy 1, policy_version 448404 (0.0007) [2023-12-27 08:44:44,608][20347] Updated weights for policy 0, policy_version 448738 (0.0010) [2023-12-27 08:44:44,613][20381] Updated weights for policy 1, policy_version 448414 (0.0008) [2023-12-27 08:44:44,668][20381] Updated weights for policy 1, policy_version 448424 (0.0005) [2023-12-27 08:44:44,668][20347] Updated weights for policy 0, policy_version 448748 (0.0006) [2023-12-27 08:44:44,733][20347] Updated weights for policy 0, policy_version 448758 (0.0005) [2023-12-27 08:44:44,799][20347] Updated weights for policy 0, policy_version 448768 (0.0010) [2023-12-27 08:44:45,331][20381] Updated weights for policy 1, policy_version 448434 (0.0006) [2023-12-27 08:44:45,383][20381] Updated weights for policy 1, policy_version 448444 (0.0010) [2023-12-27 08:44:45,433][20381] Updated weights for policy 1, policy_version 448454 (0.0011) [2023-12-27 08:44:45,438][20347] Updated weights for policy 0, policy_version 448778 (0.0011) [2023-12-27 08:44:45,485][20381] Updated weights for policy 1, policy_version 448464 (0.0007) [2023-12-27 08:44:45,494][20347] Updated weights for policy 0, policy_version 448788 (0.0008) [2023-12-27 08:44:45,546][20347] Updated weights for policy 0, policy_version 448798 (0.0008) [2023-12-27 08:44:46,120][20381] Updated weights for policy 1, policy_version 448474 (0.0006) [2023-12-27 08:44:46,141][20347] Updated weights for policy 0, policy_version 448808 (0.0009) [2023-12-27 08:44:46,172][20381] Updated weights for policy 1, policy_version 448484 (0.0006) [2023-12-27 08:44:46,195][20347] Updated weights for policy 0, policy_version 448818 (0.0010) [2023-12-27 08:44:46,227][20381] Updated weights for policy 1, policy_version 448494 (0.0006) [2023-12-27 08:44:46,248][20347] Updated weights for policy 0, policy_version 448828 (0.0010) [2023-12-27 08:44:46,836][20347] Updated weights for policy 0, policy_version 448838 (0.0007) [2023-12-27 08:44:46,884][20347] Updated weights for policy 0, policy_version 448848 (0.0006) [2023-12-27 08:44:46,909][20381] Updated weights for policy 1, policy_version 448504 (0.0010) [2023-12-27 08:44:46,932][20347] Updated weights for policy 0, policy_version 448858 (0.0008) [2023-12-27 08:44:46,955][20381] Updated weights for policy 1, policy_version 448514 (0.0010) [2023-12-27 08:44:47,010][20381] Updated weights for policy 1, policy_version 448524 (0.0009) [2023-12-27 08:44:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 229769216. Throughput: 0: 10442.7, 1: 10520.1. Samples: 229778424. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.410')] [2023-12-27 08:44:47,591][20347] Updated weights for policy 0, policy_version 448868 (0.0007) [2023-12-27 08:44:47,615][20381] Updated weights for policy 1, policy_version 448534 (0.0008) [2023-12-27 08:44:47,639][20347] Updated weights for policy 0, policy_version 448878 (0.0010) [2023-12-27 08:44:47,678][20381] Updated weights for policy 1, policy_version 448544 (0.0011) [2023-12-27 08:44:47,687][20347] Updated weights for policy 0, policy_version 448888 (0.0007) [2023-12-27 08:44:47,735][20381] Updated weights for policy 1, policy_version 448554 (0.0011) [2023-12-27 08:44:48,256][20347] Updated weights for policy 0, policy_version 448898 (0.0009) [2023-12-27 08:44:48,317][20347] Updated weights for policy 0, policy_version 448908 (0.0005) [2023-12-27 08:44:48,374][20347] Updated weights for policy 0, policy_version 448918 (0.0008) [2023-12-27 08:44:48,391][20381] Updated weights for policy 1, policy_version 448564 (0.0011) [2023-12-27 08:44:48,428][20347] Updated weights for policy 0, policy_version 448928 (0.0006) [2023-12-27 08:44:48,448][20381] Updated weights for policy 1, policy_version 448574 (0.0010) [2023-12-27 08:44:48,502][20381] Updated weights for policy 1, policy_version 448584 (0.0007) [2023-12-27 08:44:49,003][20347] Updated weights for policy 0, policy_version 448938 (0.0005) [2023-12-27 08:44:49,057][20347] Updated weights for policy 0, policy_version 448948 (0.0005) [2023-12-27 08:44:49,111][20347] Updated weights for policy 0, policy_version 448958 (0.0005) [2023-12-27 08:44:49,141][20381] Updated weights for policy 1, policy_version 448594 (0.0009) [2023-12-27 08:44:49,201][20381] Updated weights for policy 1, policy_version 448604 (0.0006) [2023-12-27 08:44:49,262][20381] Updated weights for policy 1, policy_version 448614 (0.0006) [2023-12-27 08:44:49,320][20381] Updated weights for policy 1, policy_version 448624 (0.0006) [2023-12-27 08:44:49,688][20347] Updated weights for policy 0, policy_version 448968 (0.0006) [2023-12-27 08:44:49,755][20347] Updated weights for policy 0, policy_version 448978 (0.0007) [2023-12-27 08:44:49,814][20347] Updated weights for policy 0, policy_version 448988 (0.0008) [2023-12-27 08:44:50,092][20381] Updated weights for policy 1, policy_version 448634 (0.0008) [2023-12-27 08:44:50,149][20381] Updated weights for policy 1, policy_version 448644 (0.0007) [2023-12-27 08:44:50,201][20381] Updated weights for policy 1, policy_version 448654 (0.0008) [2023-12-27 08:44:50,467][20347] Updated weights for policy 0, policy_version 448998 (0.0007) [2023-12-27 08:44:50,529][20347] Updated weights for policy 0, policy_version 449008 (0.0005) [2023-12-27 08:44:50,586][20347] Updated weights for policy 0, policy_version 449018 (0.0005) [2023-12-27 08:44:50,910][20381] Updated weights for policy 1, policy_version 448664 (0.0010) [2023-12-27 08:44:50,967][20381] Updated weights for policy 1, policy_version 448674 (0.0007) [2023-12-27 08:44:51,020][20381] Updated weights for policy 1, policy_version 448684 (0.0006) [2023-12-27 08:44:51,234][20347] Updated weights for policy 0, policy_version 449028 (0.0006) [2023-12-27 08:44:51,287][20347] Updated weights for policy 0, policy_version 449038 (0.0007) [2023-12-27 08:44:51,340][20347] Updated weights for policy 0, policy_version 449048 (0.0006) [2023-12-27 08:44:51,745][20381] Updated weights for policy 1, policy_version 448694 (0.0007) [2023-12-27 08:44:51,797][20381] Updated weights for policy 1, policy_version 448704 (0.0008) [2023-12-27 08:44:51,847][20381] Updated weights for policy 1, policy_version 448714 (0.0008) [2023-12-27 08:44:52,056][20347] Updated weights for policy 0, policy_version 449058 (0.0009) [2023-12-27 08:44:52,109][20347] Updated weights for policy 0, policy_version 449068 (0.0010) [2023-12-27 08:44:52,161][20347] Updated weights for policy 0, policy_version 449078 (0.0010) [2023-12-27 08:44:52,212][20347] Updated weights for policy 0, policy_version 449088 (0.0010) [2023-12-27 08:44:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21162.6, 300 sec: 20799.4). Total num frames: 229875712. Throughput: 0: 10450.8, 1: 10538.2. Samples: 229843768. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:44:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000448720_114892800.pth... [2023-12-27 08:44:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000449088_114982912.pth... [2023-12-27 08:44:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000447504_114581504.pth [2023-12-27 08:44:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000447840_114663424.pth [2023-12-27 08:44:52,603][20381] Updated weights for policy 1, policy_version 448724 (0.0008) [2023-12-27 08:44:52,660][20381] Updated weights for policy 1, policy_version 448734 (0.0007) [2023-12-27 08:44:52,711][20381] Updated weights for policy 1, policy_version 448744 (0.0007) [2023-12-27 08:44:52,917][20347] Updated weights for policy 0, policy_version 449098 (0.0009) [2023-12-27 08:44:52,985][20347] Updated weights for policy 0, policy_version 449108 (0.0007) [2023-12-27 08:44:53,042][20347] Updated weights for policy 0, policy_version 449118 (0.0008) [2023-12-27 08:44:53,302][20381] Updated weights for policy 1, policy_version 448754 (0.0007) [2023-12-27 08:44:53,361][20381] Updated weights for policy 1, policy_version 448764 (0.0007) [2023-12-27 08:44:53,417][20381] Updated weights for policy 1, policy_version 448774 (0.0008) [2023-12-27 08:44:53,471][20381] Updated weights for policy 1, policy_version 448784 (0.0006) [2023-12-27 08:44:53,771][20347] Updated weights for policy 0, policy_version 449128 (0.0008) [2023-12-27 08:44:53,823][20347] Updated weights for policy 0, policy_version 449138 (0.0006) [2023-12-27 08:44:53,871][20347] Updated weights for policy 0, policy_version 449148 (0.0007) [2023-12-27 08:44:54,084][20381] Updated weights for policy 1, policy_version 448794 (0.0007) [2023-12-27 08:44:54,139][20381] Updated weights for policy 1, policy_version 448804 (0.0007) [2023-12-27 08:44:54,203][20381] Updated weights for policy 1, policy_version 448814 (0.0005) [2023-12-27 08:44:54,553][20347] Updated weights for policy 0, policy_version 449158 (0.0008) [2023-12-27 08:44:54,610][20347] Updated weights for policy 0, policy_version 449168 (0.0008) [2023-12-27 08:44:54,656][20347] Updated weights for policy 0, policy_version 449178 (0.0009) [2023-12-27 08:44:54,784][20381] Updated weights for policy 1, policy_version 448824 (0.0007) [2023-12-27 08:44:54,844][20381] Updated weights for policy 1, policy_version 448834 (0.0008) [2023-12-27 08:44:54,893][20381] Updated weights for policy 1, policy_version 448844 (0.0007) [2023-12-27 08:44:55,374][20347] Updated weights for policy 0, policy_version 449188 (0.0009) [2023-12-27 08:44:55,425][20347] Updated weights for policy 0, policy_version 449198 (0.0007) [2023-12-27 08:44:55,479][20381] Updated weights for policy 1, policy_version 448854 (0.0006) [2023-12-27 08:44:55,486][20347] Updated weights for policy 0, policy_version 449208 (0.0007) [2023-12-27 08:44:55,537][20381] Updated weights for policy 1, policy_version 448864 (0.0005) [2023-12-27 08:44:55,593][20381] Updated weights for policy 1, policy_version 448874 (0.0005) [2023-12-27 08:44:56,157][20381] Updated weights for policy 1, policy_version 448884 (0.0005) [2023-12-27 08:44:56,200][20347] Updated weights for policy 0, policy_version 449218 (0.0007) [2023-12-27 08:44:56,211][20381] Updated weights for policy 1, policy_version 448894 (0.0007) [2023-12-27 08:44:56,254][20347] Updated weights for policy 0, policy_version 449228 (0.0005) [2023-12-27 08:44:56,270][20381] Updated weights for policy 1, policy_version 448904 (0.0008) [2023-12-27 08:44:56,310][20347] Updated weights for policy 0, policy_version 449238 (0.0005) [2023-12-27 08:44:56,368][20347] Updated weights for policy 0, policy_version 449248 (0.0005) [2023-12-27 08:44:56,961][20347] Updated weights for policy 0, policy_version 449258 (0.0008) [2023-12-27 08:44:56,968][20381] Updated weights for policy 1, policy_version 448914 (0.0007) [2023-12-27 08:44:57,010][20347] Updated weights for policy 0, policy_version 449268 (0.0006) [2023-12-27 08:44:57,019][20381] Updated weights for policy 1, policy_version 448924 (0.0006) [2023-12-27 08:44:57,054][20347] Updated weights for policy 0, policy_version 449278 (0.0005) [2023-12-27 08:44:57,073][20381] Updated weights for policy 1, policy_version 448934 (0.0007) [2023-12-27 08:44:57,127][20381] Updated weights for policy 1, policy_version 448944 (0.0007) [2023-12-27 08:44:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 229982208. Throughput: 0: 10447.0, 1: 10589.3. Samples: 229968916. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:44:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.700')] [2023-12-27 08:44:57,718][20347] Updated weights for policy 0, policy_version 449288 (0.0010) [2023-12-27 08:44:57,723][20381] Updated weights for policy 1, policy_version 448954 (0.0007) [2023-12-27 08:44:57,769][20347] Updated weights for policy 0, policy_version 449298 (0.0010) [2023-12-27 08:44:57,782][20381] Updated weights for policy 1, policy_version 448964 (0.0006) [2023-12-27 08:44:57,819][20347] Updated weights for policy 0, policy_version 449308 (0.0010) [2023-12-27 08:44:57,835][20381] Updated weights for policy 1, policy_version 448974 (0.0007) [2023-12-27 08:44:58,544][20381] Updated weights for policy 1, policy_version 448984 (0.0007) [2023-12-27 08:44:58,545][20347] Updated weights for policy 0, policy_version 449318 (0.0009) [2023-12-27 08:44:58,602][20381] Updated weights for policy 1, policy_version 448994 (0.0007) [2023-12-27 08:44:58,604][20347] Updated weights for policy 0, policy_version 449328 (0.0006) [2023-12-27 08:44:58,664][20347] Updated weights for policy 0, policy_version 449338 (0.0006) [2023-12-27 08:44:58,665][20381] Updated weights for policy 1, policy_version 449004 (0.0008) [2023-12-27 08:44:59,364][20381] Updated weights for policy 1, policy_version 449014 (0.0008) [2023-12-27 08:44:59,391][20347] Updated weights for policy 0, policy_version 449348 (0.0007) [2023-12-27 08:44:59,418][20381] Updated weights for policy 1, policy_version 449024 (0.0007) [2023-12-27 08:44:59,437][20347] Updated weights for policy 0, policy_version 449358 (0.0008) [2023-12-27 08:44:59,474][20381] Updated weights for policy 1, policy_version 449034 (0.0010) [2023-12-27 08:44:59,496][20347] Updated weights for policy 0, policy_version 449368 (0.0009) [2023-12-27 08:45:00,133][20381] Updated weights for policy 1, policy_version 449044 (0.0011) [2023-12-27 08:45:00,155][20347] Updated weights for policy 0, policy_version 449378 (0.0008) [2023-12-27 08:45:00,193][20381] Updated weights for policy 1, policy_version 449054 (0.0011) [2023-12-27 08:45:00,212][20347] Updated weights for policy 0, policy_version 449388 (0.0009) [2023-12-27 08:45:00,251][20381] Updated weights for policy 1, policy_version 449064 (0.0010) [2023-12-27 08:45:00,267][20347] Updated weights for policy 0, policy_version 449398 (0.0006) [2023-12-27 08:45:00,321][20347] Updated weights for policy 0, policy_version 449408 (0.0006) [2023-12-27 08:45:00,861][20381] Updated weights for policy 1, policy_version 449074 (0.0009) [2023-12-27 08:45:00,908][20381] Updated weights for policy 1, policy_version 449084 (0.0010) [2023-12-27 08:45:00,913][20347] Updated weights for policy 0, policy_version 449418 (0.0006) [2023-12-27 08:45:00,960][20381] Updated weights for policy 1, policy_version 449094 (0.0010) [2023-12-27 08:45:00,961][20347] Updated weights for policy 0, policy_version 449428 (0.0005) [2023-12-27 08:45:01,012][20347] Updated weights for policy 0, policy_version 449438 (0.0005) [2023-12-27 08:45:01,014][20381] Updated weights for policy 1, policy_version 449104 (0.0010) [2023-12-27 08:45:01,705][20347] Updated weights for policy 0, policy_version 449448 (0.0007) [2023-12-27 08:45:01,711][20381] Updated weights for policy 1, policy_version 449114 (0.0006) [2023-12-27 08:45:01,767][20347] Updated weights for policy 0, policy_version 449458 (0.0007) [2023-12-27 08:45:01,772][20381] Updated weights for policy 1, policy_version 449124 (0.0007) [2023-12-27 08:45:01,817][20347] Updated weights for policy 0, policy_version 449468 (0.0006) [2023-12-27 08:45:01,822][20381] Updated weights for policy 1, policy_version 449134 (0.0006) [2023-12-27 08:45:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 230080512. Throughput: 0: 10414.2, 1: 10553.1. Samples: 230093156. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.490')] [2023-12-27 08:45:02,503][20381] Updated weights for policy 1, policy_version 449144 (0.0007) [2023-12-27 08:45:02,522][20347] Updated weights for policy 0, policy_version 449478 (0.0006) [2023-12-27 08:45:02,555][20381] Updated weights for policy 1, policy_version 449154 (0.0007) [2023-12-27 08:45:02,573][20347] Updated weights for policy 0, policy_version 449488 (0.0005) [2023-12-27 08:45:02,612][20381] Updated weights for policy 1, policy_version 449164 (0.0008) [2023-12-27 08:45:02,622][20347] Updated weights for policy 0, policy_version 449498 (0.0005) [2023-12-27 08:45:03,208][20347] Updated weights for policy 0, policy_version 449508 (0.0005) [2023-12-27 08:45:03,262][20347] Updated weights for policy 0, policy_version 449518 (0.0005) [2023-12-27 08:45:03,328][20347] Updated weights for policy 0, policy_version 449528 (0.0008) [2023-12-27 08:45:03,330][20381] Updated weights for policy 1, policy_version 449174 (0.0006) [2023-12-27 08:45:03,378][20381] Updated weights for policy 1, policy_version 449184 (0.0005) [2023-12-27 08:45:03,439][20381] Updated weights for policy 1, policy_version 449194 (0.0006) [2023-12-27 08:45:04,002][20347] Updated weights for policy 0, policy_version 449538 (0.0010) [2023-12-27 08:45:04,054][20347] Updated weights for policy 0, policy_version 449548 (0.0009) [2023-12-27 08:45:04,060][20381] Updated weights for policy 1, policy_version 449204 (0.0007) [2023-12-27 08:45:04,108][20347] Updated weights for policy 0, policy_version 449558 (0.0008) [2023-12-27 08:45:04,130][20381] Updated weights for policy 1, policy_version 449214 (0.0007) [2023-12-27 08:45:04,171][20347] Updated weights for policy 0, policy_version 449568 (0.0007) [2023-12-27 08:45:04,195][20381] Updated weights for policy 1, policy_version 449224 (0.0007) [2023-12-27 08:45:04,814][20381] Updated weights for policy 1, policy_version 449234 (0.0008) [2023-12-27 08:45:04,821][20347] Updated weights for policy 0, policy_version 449578 (0.0005) [2023-12-27 08:45:04,862][20381] Updated weights for policy 1, policy_version 449244 (0.0010) [2023-12-27 08:45:04,873][20347] Updated weights for policy 0, policy_version 449588 (0.0005) [2023-12-27 08:45:04,926][20381] Updated weights for policy 1, policy_version 449254 (0.0010) [2023-12-27 08:45:04,926][20347] Updated weights for policy 0, policy_version 449598 (0.0005) [2023-12-27 08:45:04,983][20381] Updated weights for policy 1, policy_version 449264 (0.0010) [2023-12-27 08:45:05,523][20347] Updated weights for policy 0, policy_version 449608 (0.0007) [2023-12-27 08:45:05,582][20347] Updated weights for policy 0, policy_version 449618 (0.0010) [2023-12-27 08:45:05,597][20381] Updated weights for policy 1, policy_version 449274 (0.0005) [2023-12-27 08:45:05,639][20347] Updated weights for policy 0, policy_version 449628 (0.0010) [2023-12-27 08:45:05,648][20381] Updated weights for policy 1, policy_version 449284 (0.0005) [2023-12-27 08:45:05,697][20381] Updated weights for policy 1, policy_version 449294 (0.0006) [2023-12-27 08:45:06,249][20381] Updated weights for policy 1, policy_version 449304 (0.0011) [2023-12-27 08:45:06,255][20347] Updated weights for policy 0, policy_version 449638 (0.0008) [2023-12-27 08:45:06,300][20381] Updated weights for policy 1, policy_version 449314 (0.0010) [2023-12-27 08:45:06,311][20347] Updated weights for policy 0, policy_version 449648 (0.0008) [2023-12-27 08:45:06,346][20381] Updated weights for policy 1, policy_version 449324 (0.0010) [2023-12-27 08:45:06,367][20347] Updated weights for policy 0, policy_version 449658 (0.0007) [2023-12-27 08:45:07,009][20347] Updated weights for policy 0, policy_version 449668 (0.0007) [2023-12-27 08:45:07,046][20381] Updated weights for policy 1, policy_version 449334 (0.0010) [2023-12-27 08:45:07,058][20347] Updated weights for policy 0, policy_version 449678 (0.0006) [2023-12-27 08:45:07,114][20381] Updated weights for policy 1, policy_version 449344 (0.0009) [2023-12-27 08:45:07,122][20347] Updated weights for policy 0, policy_version 449688 (0.0007) [2023-12-27 08:45:07,160][20381] Updated weights for policy 1, policy_version 449354 (0.0006) [2023-12-27 08:45:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.7, 300 sec: 20827.1). Total num frames: 230195200. Throughput: 0: 10446.4, 1: 10522.7. Samples: 230156524. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.400')] [2023-12-27 08:45:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000449360_115056640.pth... [2023-12-27 08:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000449696_115138560.pth... [2023-12-27 08:45:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000448144_114745344.pth [2023-12-27 08:45:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000448448_114819072.pth [2023-12-27 08:45:07,782][20347] Updated weights for policy 0, policy_version 449698 (0.0010) [2023-12-27 08:45:07,787][20381] Updated weights for policy 1, policy_version 449364 (0.0008) [2023-12-27 08:45:07,834][20347] Updated weights for policy 0, policy_version 449708 (0.0010) [2023-12-27 08:45:07,845][20381] Updated weights for policy 1, policy_version 449374 (0.0010) [2023-12-27 08:45:07,892][20347] Updated weights for policy 0, policy_version 449718 (0.0010) [2023-12-27 08:45:07,897][20292] KL-divergence is very high: 138.4316 [2023-12-27 08:45:07,903][20381] Updated weights for policy 1, policy_version 449384 (0.0010) [2023-12-27 08:45:07,946][20347] Updated weights for policy 0, policy_version 449728 (0.0010) [2023-12-27 08:45:08,613][20381] Updated weights for policy 1, policy_version 449394 (0.0009) [2023-12-27 08:45:08,639][20347] Updated weights for policy 0, policy_version 449738 (0.0008) [2023-12-27 08:45:08,675][20381] Updated weights for policy 1, policy_version 449404 (0.0007) [2023-12-27 08:45:08,700][20347] Updated weights for policy 0, policy_version 449748 (0.0007) [2023-12-27 08:45:08,740][20381] Updated weights for policy 1, policy_version 449414 (0.0006) [2023-12-27 08:45:08,753][20292] KL-divergence is very high: 109.3890 [2023-12-27 08:45:08,756][20347] Updated weights for policy 0, policy_version 449758 (0.0008) [2023-12-27 08:45:08,801][20381] Updated weights for policy 1, policy_version 449424 (0.0007) [2023-12-27 08:45:09,432][20381] Updated weights for policy 1, policy_version 449434 (0.0008) [2023-12-27 08:45:09,458][20347] Updated weights for policy 0, policy_version 449768 (0.0008) [2023-12-27 08:45:09,483][20381] Updated weights for policy 1, policy_version 449444 (0.0007) [2023-12-27 08:45:09,514][20347] Updated weights for policy 0, policy_version 449778 (0.0010) [2023-12-27 08:45:09,531][20381] Updated weights for policy 1, policy_version 449454 (0.0006) [2023-12-27 08:45:09,570][20347] Updated weights for policy 0, policy_version 449788 (0.0010) [2023-12-27 08:45:10,235][20381] Updated weights for policy 1, policy_version 449464 (0.0010) [2023-12-27 08:45:10,241][20347] Updated weights for policy 0, policy_version 449798 (0.0010) [2023-12-27 08:45:10,292][20381] Updated weights for policy 1, policy_version 449474 (0.0011) [2023-12-27 08:45:10,296][20347] Updated weights for policy 0, policy_version 449808 (0.0009) [2023-12-27 08:45:10,351][20381] Updated weights for policy 1, policy_version 449484 (0.0010) [2023-12-27 08:45:10,355][20347] Updated weights for policy 0, policy_version 449818 (0.0010) [2023-12-27 08:45:11,010][20347] Updated weights for policy 0, policy_version 449828 (0.0010) [2023-12-27 08:45:11,052][20381] Updated weights for policy 1, policy_version 449494 (0.0007) [2023-12-27 08:45:11,070][20347] Updated weights for policy 0, policy_version 449838 (0.0008) [2023-12-27 08:45:11,113][20381] Updated weights for policy 1, policy_version 449504 (0.0008) [2023-12-27 08:45:11,136][20347] Updated weights for policy 0, policy_version 449848 (0.0007) [2023-12-27 08:45:11,176][20381] Updated weights for policy 1, policy_version 449514 (0.0009) [2023-12-27 08:45:11,872][20347] Updated weights for policy 0, policy_version 449858 (0.0007) [2023-12-27 08:45:11,912][20381] Updated weights for policy 1, policy_version 449524 (0.0007) [2023-12-27 08:45:11,923][20347] Updated weights for policy 0, policy_version 449868 (0.0007) [2023-12-27 08:45:11,970][20381] Updated weights for policy 1, policy_version 449534 (0.0006) [2023-12-27 08:45:11,974][20347] Updated weights for policy 0, policy_version 449878 (0.0007) [2023-12-27 08:45:12,019][20381] Updated weights for policy 1, policy_version 449544 (0.0006) [2023-12-27 08:45:12,025][20347] Updated weights for policy 0, policy_version 449888 (0.0006) [2023-12-27 08:45:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 230293504. Throughput: 0: 10503.1, 1: 10520.1. Samples: 230283232. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.250')] [2023-12-27 08:45:12,651][20381] Updated weights for policy 1, policy_version 449554 (0.0007) [2023-12-27 08:45:12,706][20381] Updated weights for policy 1, policy_version 449564 (0.0007) [2023-12-27 08:45:12,716][20347] Updated weights for policy 0, policy_version 449898 (0.0007) [2023-12-27 08:45:12,766][20381] Updated weights for policy 1, policy_version 449574 (0.0007) [2023-12-27 08:45:12,776][20347] Updated weights for policy 0, policy_version 449908 (0.0006) [2023-12-27 08:45:12,818][20381] Updated weights for policy 1, policy_version 449584 (0.0006) [2023-12-27 08:45:12,833][20347] Updated weights for policy 0, policy_version 449918 (0.0006) [2023-12-27 08:45:13,408][20381] Updated weights for policy 1, policy_version 449594 (0.0008) [2023-12-27 08:45:13,464][20381] Updated weights for policy 1, policy_version 449604 (0.0007) [2023-12-27 08:45:13,514][20381] Updated weights for policy 1, policy_version 449614 (0.0008) [2023-12-27 08:45:13,515][20347] Updated weights for policy 0, policy_version 449928 (0.0006) [2023-12-27 08:45:13,579][20347] Updated weights for policy 0, policy_version 449938 (0.0008) [2023-12-27 08:45:13,645][20347] Updated weights for policy 0, policy_version 449948 (0.0007) [2023-12-27 08:45:14,196][20381] Updated weights for policy 1, policy_version 449624 (0.0008) [2023-12-27 08:45:14,257][20381] Updated weights for policy 1, policy_version 449634 (0.0008) [2023-12-27 08:45:14,258][20347] Updated weights for policy 0, policy_version 449958 (0.0008) [2023-12-27 08:45:14,313][20347] Updated weights for policy 0, policy_version 449968 (0.0006) [2023-12-27 08:45:14,320][20381] Updated weights for policy 1, policy_version 449644 (0.0006) [2023-12-27 08:45:14,363][20347] Updated weights for policy 0, policy_version 449978 (0.0007) [2023-12-27 08:45:14,997][20381] Updated weights for policy 1, policy_version 449654 (0.0009) [2023-12-27 08:45:15,045][20381] Updated weights for policy 1, policy_version 449664 (0.0008) [2023-12-27 08:45:15,056][20347] Updated weights for policy 0, policy_version 449988 (0.0007) [2023-12-27 08:45:15,096][20381] Updated weights for policy 1, policy_version 449674 (0.0008) [2023-12-27 08:45:15,113][20347] Updated weights for policy 0, policy_version 449998 (0.0007) [2023-12-27 08:45:15,165][20347] Updated weights for policy 0, policy_version 450008 (0.0007) [2023-12-27 08:45:15,714][20381] Updated weights for policy 1, policy_version 449684 (0.0005) [2023-12-27 08:45:15,762][20381] Updated weights for policy 1, policy_version 449694 (0.0005) [2023-12-27 08:45:15,808][20381] Updated weights for policy 1, policy_version 449704 (0.0005) [2023-12-27 08:45:15,878][20347] Updated weights for policy 0, policy_version 450018 (0.0007) [2023-12-27 08:45:15,936][20347] Updated weights for policy 0, policy_version 450028 (0.0008) [2023-12-27 08:45:15,996][20347] Updated weights for policy 0, policy_version 450038 (0.0005) [2023-12-27 08:45:16,052][20347] Updated weights for policy 0, policy_version 450048 (0.0005) [2023-12-27 08:45:16,408][20381] Updated weights for policy 1, policy_version 449714 (0.0006) [2023-12-27 08:45:16,457][20381] Updated weights for policy 1, policy_version 449724 (0.0005) [2023-12-27 08:45:16,505][20381] Updated weights for policy 1, policy_version 449734 (0.0005) [2023-12-27 08:45:16,562][20381] Updated weights for policy 1, policy_version 449744 (0.0006) [2023-12-27 08:45:16,717][20347] Updated weights for policy 0, policy_version 450058 (0.0010) [2023-12-27 08:45:16,771][20347] Updated weights for policy 0, policy_version 450068 (0.0010) [2023-12-27 08:45:16,825][20347] Updated weights for policy 0, policy_version 450078 (0.0008) [2023-12-27 08:45:17,252][20381] Updated weights for policy 1, policy_version 449754 (0.0007) [2023-12-27 08:45:17,306][20381] Updated weights for policy 1, policy_version 449764 (0.0008) [2023-12-27 08:45:17,361][20381] Updated weights for policy 1, policy_version 449774 (0.0008) [2023-12-27 08:45:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 230391808. Throughput: 0: 10491.6, 1: 10526.1. Samples: 230408104. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '962.140')] [2023-12-27 08:45:17,439][20347] Updated weights for policy 0, policy_version 450088 (0.0008) [2023-12-27 08:45:17,498][20347] Updated weights for policy 0, policy_version 450098 (0.0008) [2023-12-27 08:45:17,555][20347] Updated weights for policy 0, policy_version 450108 (0.0008) [2023-12-27 08:45:18,060][20381] Updated weights for policy 1, policy_version 449784 (0.0008) [2023-12-27 08:45:18,110][20381] Updated weights for policy 1, policy_version 449794 (0.0008) [2023-12-27 08:45:18,165][20381] Updated weights for policy 1, policy_version 449804 (0.0008) [2023-12-27 08:45:18,204][20347] Updated weights for policy 0, policy_version 450118 (0.0010) [2023-12-27 08:45:18,260][20347] Updated weights for policy 0, policy_version 450128 (0.0011) [2023-12-27 08:45:18,322][20347] Updated weights for policy 0, policy_version 450138 (0.0010) [2023-12-27 08:45:18,836][20381] Updated weights for policy 1, policy_version 449814 (0.0008) [2023-12-27 08:45:18,900][20381] Updated weights for policy 1, policy_version 449824 (0.0008) [2023-12-27 08:45:18,955][20381] Updated weights for policy 1, policy_version 449834 (0.0007) [2023-12-27 08:45:19,027][20347] Updated weights for policy 0, policy_version 450148 (0.0009) [2023-12-27 08:45:19,086][20347] Updated weights for policy 0, policy_version 450158 (0.0007) [2023-12-27 08:45:19,147][20347] Updated weights for policy 0, policy_version 450168 (0.0008) [2023-12-27 08:45:19,666][20381] Updated weights for policy 1, policy_version 449844 (0.0008) [2023-12-27 08:45:19,723][20381] Updated weights for policy 1, policy_version 449854 (0.0008) [2023-12-27 08:45:19,776][20381] Updated weights for policy 1, policy_version 449864 (0.0007) [2023-12-27 08:45:19,884][20347] Updated weights for policy 0, policy_version 450178 (0.0007) [2023-12-27 08:45:19,948][20347] Updated weights for policy 0, policy_version 450188 (0.0007) [2023-12-27 08:45:20,006][20347] Updated weights for policy 0, policy_version 450198 (0.0008) [2023-12-27 08:45:20,071][20347] Updated weights for policy 0, policy_version 450208 (0.0007) [2023-12-27 08:45:20,496][20381] Updated weights for policy 1, policy_version 449874 (0.0007) [2023-12-27 08:45:20,556][20381] Updated weights for policy 1, policy_version 449884 (0.0008) [2023-12-27 08:45:20,625][20381] Updated weights for policy 1, policy_version 449894 (0.0006) [2023-12-27 08:45:20,688][20381] Updated weights for policy 1, policy_version 449904 (0.0007) [2023-12-27 08:45:20,810][20347] Updated weights for policy 0, policy_version 450218 (0.0007) [2023-12-27 08:45:20,874][20347] Updated weights for policy 0, policy_version 450228 (0.0007) [2023-12-27 08:45:20,930][20347] Updated weights for policy 0, policy_version 450238 (0.0007) [2023-12-27 08:45:21,436][20381] Updated weights for policy 1, policy_version 449914 (0.0009) [2023-12-27 08:45:21,495][20381] Updated weights for policy 1, policy_version 449924 (0.0009) [2023-12-27 08:45:21,552][20381] Updated weights for policy 1, policy_version 449934 (0.0009) [2023-12-27 08:45:21,668][20347] Updated weights for policy 0, policy_version 450248 (0.0009) [2023-12-27 08:45:21,733][20347] Updated weights for policy 0, policy_version 450258 (0.0008) [2023-12-27 08:45:21,797][20347] Updated weights for policy 0, policy_version 450268 (0.0008) [2023-12-27 08:45:22,313][20381] Updated weights for policy 1, policy_version 449944 (0.0008) [2023-12-27 08:45:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 230490112. Throughput: 0: 10445.2, 1: 10434.8. Samples: 230467536. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.870')] [2023-12-27 08:45:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000450272_115286016.pth... [2023-12-27 08:45:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000449088_114982912.pth [2023-12-27 08:45:22,377][20381] Updated weights for policy 1, policy_version 449954 (0.0007) [2023-12-27 08:45:22,422][20347] Updated weights for policy 0, policy_version 450278 (0.0007) [2023-12-27 08:45:22,439][20381] Updated weights for policy 1, policy_version 449964 (0.0007) [2023-12-27 08:45:22,459][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000449968_115212288.pth... [2023-12-27 08:45:22,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000448720_114892800.pth [2023-12-27 08:45:22,478][20347] Updated weights for policy 0, policy_version 450288 (0.0006) [2023-12-27 08:45:22,532][20347] Updated weights for policy 0, policy_version 450298 (0.0007) [2023-12-27 08:45:23,117][20381] Updated weights for policy 1, policy_version 449974 (0.0008) [2023-12-27 08:45:23,180][20381] Updated weights for policy 1, policy_version 449984 (0.0008) [2023-12-27 08:45:23,237][20381] Updated weights for policy 1, policy_version 449994 (0.0007) [2023-12-27 08:45:23,270][20347] Updated weights for policy 0, policy_version 450308 (0.0008) [2023-12-27 08:45:23,327][20347] Updated weights for policy 0, policy_version 450318 (0.0008) [2023-12-27 08:45:23,389][20347] Updated weights for policy 0, policy_version 450328 (0.0008) [2023-12-27 08:45:23,832][20381] Updated weights for policy 1, policy_version 450004 (0.0007) [2023-12-27 08:45:23,884][20381] Updated weights for policy 1, policy_version 450014 (0.0007) [2023-12-27 08:45:23,942][20381] Updated weights for policy 1, policy_version 450024 (0.0008) [2023-12-27 08:45:24,127][20347] Updated weights for policy 0, policy_version 450338 (0.0008) [2023-12-27 08:45:24,184][20347] Updated weights for policy 0, policy_version 450348 (0.0009) [2023-12-27 08:45:24,243][20347] Updated weights for policy 0, policy_version 450358 (0.0007) [2023-12-27 08:45:24,299][20347] Updated weights for policy 0, policy_version 450368 (0.0008) [2023-12-27 08:45:24,628][20381] Updated weights for policy 1, policy_version 450034 (0.0008) [2023-12-27 08:45:24,691][20381] Updated weights for policy 1, policy_version 450044 (0.0006) [2023-12-27 08:45:24,752][20381] Updated weights for policy 1, policy_version 450054 (0.0008) [2023-12-27 08:45:24,808][20381] Updated weights for policy 1, policy_version 450064 (0.0008) [2023-12-27 08:45:25,019][20347] Updated weights for policy 0, policy_version 450378 (0.0009) [2023-12-27 08:45:25,076][20347] Updated weights for policy 0, policy_version 450388 (0.0006) [2023-12-27 08:45:25,137][20347] Updated weights for policy 0, policy_version 450398 (0.0008) [2023-12-27 08:45:25,435][20381] Updated weights for policy 1, policy_version 450074 (0.0005) [2023-12-27 08:45:25,486][20381] Updated weights for policy 1, policy_version 450084 (0.0005) [2023-12-27 08:45:25,530][20381] Updated weights for policy 1, policy_version 450094 (0.0005) [2023-12-27 08:45:25,766][20347] Updated weights for policy 0, policy_version 450408 (0.0008) [2023-12-27 08:45:25,820][20347] Updated weights for policy 0, policy_version 450418 (0.0007) [2023-12-27 08:45:25,878][20347] Updated weights for policy 0, policy_version 450428 (0.0006) [2023-12-27 08:45:26,177][20381] Updated weights for policy 1, policy_version 450104 (0.0010) [2023-12-27 08:45:26,231][20381] Updated weights for policy 1, policy_version 450114 (0.0010) [2023-12-27 08:45:26,288][20381] Updated weights for policy 1, policy_version 450124 (0.0010) [2023-12-27 08:45:26,558][20347] Updated weights for policy 0, policy_version 450438 (0.0007) [2023-12-27 08:45:26,600][20347] Updated weights for policy 0, policy_version 450448 (0.0007) [2023-12-27 08:45:26,647][20347] Updated weights for policy 0, policy_version 450458 (0.0007) [2023-12-27 08:45:26,937][20381] Updated weights for policy 1, policy_version 450134 (0.0010) [2023-12-27 08:45:26,990][20381] Updated weights for policy 1, policy_version 450144 (0.0010) [2023-12-27 08:45:27,044][20381] Updated weights for policy 1, policy_version 450154 (0.0010) [2023-12-27 08:45:27,341][20347] Updated weights for policy 0, policy_version 450468 (0.0006) [2023-12-27 08:45:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 230596608. Throughput: 0: 10355.8, 1: 10406.2. Samples: 230587656. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-12-27 08:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.600')] [2023-12-27 08:45:27,399][20347] Updated weights for policy 0, policy_version 450478 (0.0005) [2023-12-27 08:45:27,455][20347] Updated weights for policy 0, policy_version 450488 (0.0008) [2023-12-27 08:45:27,722][20381] Updated weights for policy 1, policy_version 450164 (0.0008) [2023-12-27 08:45:27,767][20381] Updated weights for policy 1, policy_version 450174 (0.0005) [2023-12-27 08:45:27,823][20381] Updated weights for policy 1, policy_version 450184 (0.0006) [2023-12-27 08:45:28,102][20347] Updated weights for policy 0, policy_version 450498 (0.0007) [2023-12-27 08:45:28,147][20347] Updated weights for policy 0, policy_version 450508 (0.0005) [2023-12-27 08:45:28,192][20347] Updated weights for policy 0, policy_version 450518 (0.0005) [2023-12-27 08:45:28,246][20347] Updated weights for policy 0, policy_version 450528 (0.0005) [2023-12-27 08:45:28,511][20381] Updated weights for policy 1, policy_version 450194 (0.0010) [2023-12-27 08:45:28,568][20381] Updated weights for policy 1, policy_version 450204 (0.0007) [2023-12-27 08:45:28,624][20381] Updated weights for policy 1, policy_version 450214 (0.0007) [2023-12-27 08:45:28,683][20381] Updated weights for policy 1, policy_version 450224 (0.0008) [2023-12-27 08:45:28,885][20347] Updated weights for policy 0, policy_version 450538 (0.0008) [2023-12-27 08:45:28,941][20347] Updated weights for policy 0, policy_version 450548 (0.0008) [2023-12-27 08:45:28,996][20347] Updated weights for policy 0, policy_version 450558 (0.0008) [2023-12-27 08:45:29,414][20381] Updated weights for policy 1, policy_version 450234 (0.0008) [2023-12-27 08:45:29,470][20381] Updated weights for policy 1, policy_version 450244 (0.0008) [2023-12-27 08:45:29,530][20381] Updated weights for policy 1, policy_version 450254 (0.0007) [2023-12-27 08:45:29,646][20347] Updated weights for policy 0, policy_version 450568 (0.0008) [2023-12-27 08:45:29,705][20347] Updated weights for policy 0, policy_version 450578 (0.0009) [2023-12-27 08:45:29,766][20347] Updated weights for policy 0, policy_version 450588 (0.0009) [2023-12-27 08:45:30,184][20381] Updated weights for policy 1, policy_version 450264 (0.0007) [2023-12-27 08:45:30,237][20381] Updated weights for policy 1, policy_version 450274 (0.0008) [2023-12-27 08:45:30,302][20381] Updated weights for policy 1, policy_version 450284 (0.0008) [2023-12-27 08:45:30,445][20347] Updated weights for policy 0, policy_version 450598 (0.0006) [2023-12-27 08:45:30,490][20347] Updated weights for policy 0, policy_version 450608 (0.0006) [2023-12-27 08:45:30,547][20347] Updated weights for policy 0, policy_version 450618 (0.0007) [2023-12-27 08:45:30,997][20381] Updated weights for policy 1, policy_version 450294 (0.0008) [2023-12-27 08:45:31,057][20381] Updated weights for policy 1, policy_version 450304 (0.0006) [2023-12-27 08:45:31,141][20381] Updated weights for policy 1, policy_version 450314 (0.0007) [2023-12-27 08:45:31,226][20347] Updated weights for policy 0, policy_version 450628 (0.0008) [2023-12-27 08:45:31,284][20347] Updated weights for policy 0, policy_version 450638 (0.0009) [2023-12-27 08:45:31,344][20347] Updated weights for policy 0, policy_version 450648 (0.0007) [2023-12-27 08:45:31,807][20381] Updated weights for policy 1, policy_version 450324 (0.0006) [2023-12-27 08:45:31,857][20381] Updated weights for policy 1, policy_version 450334 (0.0006) [2023-12-27 08:45:31,905][20381] Updated weights for policy 1, policy_version 450344 (0.0007) [2023-12-27 08:45:32,050][20347] Updated weights for policy 0, policy_version 450658 (0.0008) [2023-12-27 08:45:32,103][20347] Updated weights for policy 0, policy_version 450668 (0.0008) [2023-12-27 08:45:32,159][20347] Updated weights for policy 0, policy_version 450678 (0.0008) [2023-12-27 08:45:32,217][20347] Updated weights for policy 0, policy_version 450688 (0.0008) [2023-12-27 08:45:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 230703104. Throughput: 0: 10318.7, 1: 10406.4. Samples: 230711052. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.990')] [2023-12-27 08:45:32,518][20381] Updated weights for policy 1, policy_version 450354 (0.0005) [2023-12-27 08:45:32,577][20381] Updated weights for policy 1, policy_version 450364 (0.0005) [2023-12-27 08:45:32,640][20381] Updated weights for policy 1, policy_version 450374 (0.0005) [2023-12-27 08:45:32,698][20381] Updated weights for policy 1, policy_version 450384 (0.0005) [2023-12-27 08:45:32,842][20347] Updated weights for policy 0, policy_version 450698 (0.0008) [2023-12-27 08:45:32,894][20347] Updated weights for policy 0, policy_version 450708 (0.0008) [2023-12-27 08:45:32,952][20347] Updated weights for policy 0, policy_version 450718 (0.0008) [2023-12-27 08:45:33,273][20381] Updated weights for policy 1, policy_version 450394 (0.0006) [2023-12-27 08:45:33,328][20381] Updated weights for policy 1, policy_version 450404 (0.0005) [2023-12-27 08:45:33,377][20381] Updated weights for policy 1, policy_version 450414 (0.0007) [2023-12-27 08:45:33,649][20347] Updated weights for policy 0, policy_version 450728 (0.0007) [2023-12-27 08:45:33,704][20347] Updated weights for policy 0, policy_version 450738 (0.0008) [2023-12-27 08:45:33,748][20347] Updated weights for policy 0, policy_version 450748 (0.0008) [2023-12-27 08:45:33,984][20381] Updated weights for policy 1, policy_version 450424 (0.0010) [2023-12-27 08:45:34,029][20381] Updated weights for policy 1, policy_version 450434 (0.0010) [2023-12-27 08:45:34,080][20381] Updated weights for policy 1, policy_version 450444 (0.0010) [2023-12-27 08:45:34,483][20347] Updated weights for policy 0, policy_version 450758 (0.0008) [2023-12-27 08:45:34,547][20347] Updated weights for policy 0, policy_version 450768 (0.0008) [2023-12-27 08:45:34,602][20347] Updated weights for policy 0, policy_version 450778 (0.0007) [2023-12-27 08:45:34,792][20381] Updated weights for policy 1, policy_version 450454 (0.0009) [2023-12-27 08:45:34,856][20381] Updated weights for policy 1, policy_version 450464 (0.0008) [2023-12-27 08:45:34,914][20381] Updated weights for policy 1, policy_version 450474 (0.0007) [2023-12-27 08:45:35,251][20347] Updated weights for policy 0, policy_version 450788 (0.0007) [2023-12-27 08:45:35,283][20291] KL-divergence is very high: 415.1922 [2023-12-27 08:45:35,303][20291] KL-divergence is very high: 1189.4081 [2023-12-27 08:45:35,315][20347] Updated weights for policy 0, policy_version 450798 (0.0008) [2023-12-27 08:45:35,354][20291] KL-divergence is very high: 152.0042 [2023-12-27 08:45:35,379][20347] Updated weights for policy 0, policy_version 450808 (0.0008) [2023-12-27 08:45:35,402][20291] KL-divergence is very high: 209.5488 [2023-12-27 08:45:35,495][20381] Updated weights for policy 1, policy_version 450484 (0.0005) [2023-12-27 08:45:35,543][20381] Updated weights for policy 1, policy_version 450494 (0.0008) [2023-12-27 08:45:35,602][20381] Updated weights for policy 1, policy_version 450504 (0.0006) [2023-12-27 08:45:36,086][20347] Updated weights for policy 0, policy_version 450818 (0.0007) [2023-12-27 08:45:36,150][20347] Updated weights for policy 0, policy_version 450828 (0.0008) [2023-12-27 08:45:36,209][20347] Updated weights for policy 0, policy_version 450838 (0.0007) [2023-12-27 08:45:36,264][20347] Updated weights for policy 0, policy_version 450848 (0.0007) [2023-12-27 08:45:36,313][20381] Updated weights for policy 1, policy_version 450514 (0.0008) [2023-12-27 08:45:36,364][20381] Updated weights for policy 1, policy_version 450524 (0.0010) [2023-12-27 08:45:36,429][20381] Updated weights for policy 1, policy_version 450534 (0.0011) [2023-12-27 08:45:36,477][20381] Updated weights for policy 1, policy_version 450544 (0.0009) [2023-12-27 08:45:36,939][20347] Updated weights for policy 0, policy_version 450858 (0.0006) [2023-12-27 08:45:37,002][20347] Updated weights for policy 0, policy_version 450868 (0.0008) [2023-12-27 08:45:37,058][20347] Updated weights for policy 0, policy_version 450878 (0.0006) [2023-12-27 08:45:37,105][20381] Updated weights for policy 1, policy_version 450554 (0.0008) [2023-12-27 08:45:37,151][20381] Updated weights for policy 1, policy_version 450564 (0.0006) [2023-12-27 08:45:37,210][20381] Updated weights for policy 1, policy_version 450574 (0.0006) [2023-12-27 08:45:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 230809600. Throughput: 0: 10245.4, 1: 10447.5. Samples: 230774952. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:37,364][19170] Avg episode reward: [(0, '980.820'), (1, '980.630')] [2023-12-27 08:45:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000450576_115367936.pth... [2023-12-27 08:45:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000450880_115441664.pth... [2023-12-27 08:45:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000449360_115056640.pth [2023-12-27 08:45:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000449696_115138560.pth [2023-12-27 08:45:37,742][20347] Updated weights for policy 0, policy_version 450888 (0.0007) [2023-12-27 08:45:37,796][20347] Updated weights for policy 0, policy_version 450898 (0.0007) [2023-12-27 08:45:37,834][20381] Updated weights for policy 1, policy_version 450584 (0.0006) [2023-12-27 08:45:37,851][20347] Updated weights for policy 0, policy_version 450908 (0.0007) [2023-12-27 08:45:37,883][20381] Updated weights for policy 1, policy_version 450594 (0.0010) [2023-12-27 08:45:37,938][20381] Updated weights for policy 1, policy_version 450604 (0.0010) [2023-12-27 08:45:38,479][20347] Updated weights for policy 0, policy_version 450918 (0.0006) [2023-12-27 08:45:38,539][20347] Updated weights for policy 0, policy_version 450928 (0.0007) [2023-12-27 08:45:38,594][20347] Updated weights for policy 0, policy_version 450938 (0.0008) [2023-12-27 08:45:38,645][20381] Updated weights for policy 1, policy_version 450614 (0.0009) [2023-12-27 08:45:38,694][20381] Updated weights for policy 1, policy_version 450624 (0.0007) [2023-12-27 08:45:38,752][20381] Updated weights for policy 1, policy_version 450634 (0.0008) [2023-12-27 08:45:39,271][20347] Updated weights for policy 0, policy_version 450948 (0.0007) [2023-12-27 08:45:39,327][20347] Updated weights for policy 0, policy_version 450958 (0.0008) [2023-12-27 08:45:39,393][20347] Updated weights for policy 0, policy_version 450968 (0.0009) [2023-12-27 08:45:39,477][20381] Updated weights for policy 1, policy_version 450644 (0.0007) [2023-12-27 08:45:39,536][20381] Updated weights for policy 1, policy_version 450654 (0.0007) [2023-12-27 08:45:39,591][20381] Updated weights for policy 1, policy_version 450664 (0.0006) [2023-12-27 08:45:40,138][20347] Updated weights for policy 0, policy_version 450978 (0.0008) [2023-12-27 08:45:40,198][20347] Updated weights for policy 0, policy_version 450988 (0.0008) [2023-12-27 08:45:40,220][20381] Updated weights for policy 1, policy_version 450674 (0.0006) [2023-12-27 08:45:40,259][20347] Updated weights for policy 0, policy_version 450998 (0.0006) [2023-12-27 08:45:40,281][20381] Updated weights for policy 1, policy_version 450684 (0.0006) [2023-12-27 08:45:40,291][20292] KL-divergence is very high: 361.0922 [2023-12-27 08:45:40,297][20292] KL-divergence is very high: 3124.8086 [2023-12-27 08:45:40,317][20347] Updated weights for policy 0, policy_version 451008 (0.0006) [2023-12-27 08:45:40,332][20381] Updated weights for policy 1, policy_version 450694 (0.0007) [2023-12-27 08:45:40,333][20292] KL-divergence is very high: 682.2164 [2023-12-27 08:45:40,377][20292] KL-divergence is very high: 608.1602 [2023-12-27 08:45:40,387][20381] Updated weights for policy 1, policy_version 450704 (0.0007) [2023-12-27 08:45:40,942][20347] Updated weights for policy 0, policy_version 451018 (0.0005) [2023-12-27 08:45:41,004][20347] Updated weights for policy 0, policy_version 451028 (0.0006) [2023-12-27 08:45:41,026][20292] KL-divergence is very high: 135.9697 [2023-12-27 08:45:41,062][20347] Updated weights for policy 0, policy_version 451038 (0.0008) [2023-12-27 08:45:41,076][20381] Updated weights for policy 1, policy_version 450714 (0.0008) [2023-12-27 08:45:41,077][20292] KL-divergence is very high: 719.7435 [2023-12-27 08:45:41,123][20292] KL-divergence is very high: 124.9539 [2023-12-27 08:45:41,134][20381] Updated weights for policy 1, policy_version 450724 (0.0009) [2023-12-27 08:45:41,194][20381] Updated weights for policy 1, policy_version 450734 (0.0008) [2023-12-27 08:45:41,778][20347] Updated weights for policy 0, policy_version 451048 (0.0008) [2023-12-27 08:45:41,837][20347] Updated weights for policy 0, policy_version 451058 (0.0008) [2023-12-27 08:45:41,895][20347] Updated weights for policy 0, policy_version 451068 (0.0007) [2023-12-27 08:45:41,916][20381] Updated weights for policy 1, policy_version 450744 (0.0006) [2023-12-27 08:45:41,968][20381] Updated weights for policy 1, policy_version 450754 (0.0007) [2023-12-27 08:45:42,024][20381] Updated weights for policy 1, policy_version 450764 (0.0008) [2023-12-27 08:45:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 230907904. Throughput: 0: 10253.7, 1: 10382.6. Samples: 230897548. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:42,363][19170] Avg episode reward: [(0, '980.820'), (1, '904.570')] [2023-12-27 08:45:42,549][20347] Updated weights for policy 0, policy_version 451078 (0.0007) [2023-12-27 08:45:42,605][20347] Updated weights for policy 0, policy_version 451088 (0.0007) [2023-12-27 08:45:42,663][20347] Updated weights for policy 0, policy_version 451098 (0.0007) [2023-12-27 08:45:42,733][20381] Updated weights for policy 1, policy_version 450774 (0.0009) [2023-12-27 08:45:42,787][20381] Updated weights for policy 1, policy_version 450784 (0.0011) [2023-12-27 08:45:42,849][20381] Updated weights for policy 1, policy_version 450794 (0.0006) [2023-12-27 08:45:43,269][20347] Updated weights for policy 0, policy_version 451108 (0.0008) [2023-12-27 08:45:43,323][20347] Updated weights for policy 0, policy_version 451118 (0.0007) [2023-12-27 08:45:43,377][20347] Updated weights for policy 0, policy_version 451128 (0.0007) [2023-12-27 08:45:43,500][20381] Updated weights for policy 1, policy_version 450804 (0.0005) [2023-12-27 08:45:43,554][20381] Updated weights for policy 1, policy_version 450814 (0.0005) [2023-12-27 08:45:43,607][20381] Updated weights for policy 1, policy_version 450824 (0.0005) [2023-12-27 08:45:44,043][20347] Updated weights for policy 0, policy_version 451138 (0.0007) [2023-12-27 08:45:44,103][20347] Updated weights for policy 0, policy_version 451148 (0.0007) [2023-12-27 08:45:44,158][20347] Updated weights for policy 0, policy_version 451158 (0.0007) [2023-12-27 08:45:44,216][20347] Updated weights for policy 0, policy_version 451168 (0.0007) [2023-12-27 08:45:44,241][20381] Updated weights for policy 1, policy_version 450834 (0.0008) [2023-12-27 08:45:44,286][20381] Updated weights for policy 1, policy_version 450844 (0.0010) [2023-12-27 08:45:44,333][20381] Updated weights for policy 1, policy_version 450854 (0.0010) [2023-12-27 08:45:44,385][20381] Updated weights for policy 1, policy_version 450864 (0.0010) [2023-12-27 08:45:44,817][20347] Updated weights for policy 0, policy_version 451178 (0.0008) [2023-12-27 08:45:44,873][20347] Updated weights for policy 0, policy_version 451188 (0.0008) [2023-12-27 08:45:44,936][20347] Updated weights for policy 0, policy_version 451198 (0.0008) [2023-12-27 08:45:45,060][20381] Updated weights for policy 1, policy_version 450874 (0.0011) [2023-12-27 08:45:45,119][20381] Updated weights for policy 1, policy_version 450884 (0.0009) [2023-12-27 08:45:45,176][20381] Updated weights for policy 1, policy_version 450894 (0.0011) [2023-12-27 08:45:45,638][20347] Updated weights for policy 0, policy_version 451208 (0.0008) [2023-12-27 08:45:45,686][20291] KL-divergence is very high: 187.3580 [2023-12-27 08:45:45,687][20347] Updated weights for policy 0, policy_version 451218 (0.0007) [2023-12-27 08:45:45,739][20347] Updated weights for policy 0, policy_version 451228 (0.0008) [2023-12-27 08:45:45,879][20381] Updated weights for policy 1, policy_version 450904 (0.0011) [2023-12-27 08:45:45,935][20381] Updated weights for policy 1, policy_version 450914 (0.0010) [2023-12-27 08:45:45,989][20381] Updated weights for policy 1, policy_version 450924 (0.0005) [2023-12-27 08:45:46,409][20347] Updated weights for policy 0, policy_version 451238 (0.0007) [2023-12-27 08:45:46,454][20347] Updated weights for policy 0, policy_version 451248 (0.0006) [2023-12-27 08:45:46,501][20347] Updated weights for policy 0, policy_version 451258 (0.0005) [2023-12-27 08:45:46,509][20291] KL-divergence is very high: 303.5316 [2023-12-27 08:45:46,585][20381] Updated weights for policy 1, policy_version 450934 (0.0005) [2023-12-27 08:45:46,631][20381] Updated weights for policy 1, policy_version 450944 (0.0007) [2023-12-27 08:45:46,675][20381] Updated weights for policy 1, policy_version 450954 (0.0007) [2023-12-27 08:45:47,094][20347] Updated weights for policy 0, policy_version 451268 (0.0006) [2023-12-27 08:45:47,149][20347] Updated weights for policy 0, policy_version 451278 (0.0007) [2023-12-27 08:45:47,201][20347] Updated weights for policy 0, policy_version 451288 (0.0006) [2023-12-27 08:45:47,360][20381] Updated weights for policy 1, policy_version 450964 (0.0007) [2023-12-27 08:45:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 231014400. Throughput: 0: 10294.2, 1: 10393.8. Samples: 231024116. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:47,363][19170] Avg episode reward: [(0, '962.960'), (1, '915.190')] [2023-12-27 08:45:47,420][20381] Updated weights for policy 1, policy_version 450974 (0.0008) [2023-12-27 08:45:47,479][20381] Updated weights for policy 1, policy_version 450984 (0.0008) [2023-12-27 08:45:47,820][20347] Updated weights for policy 0, policy_version 451298 (0.0006) [2023-12-27 08:45:47,856][20291] KL-divergence is very high: 473.5626 [2023-12-27 08:45:47,872][20347] Updated weights for policy 0, policy_version 451308 (0.0010) [2023-12-27 08:45:47,898][20291] KL-divergence is very high: 193.2157 [2023-12-27 08:45:47,923][20347] Updated weights for policy 0, policy_version 451318 (0.0009) [2023-12-27 08:45:47,939][20291] KL-divergence is very high: 151.7088 [2023-12-27 08:45:47,974][20347] Updated weights for policy 0, policy_version 451328 (0.0008) [2023-12-27 08:45:48,112][20381] Updated weights for policy 1, policy_version 450994 (0.0009) [2023-12-27 08:45:48,160][20381] Updated weights for policy 1, policy_version 451004 (0.0007) [2023-12-27 08:45:48,218][20381] Updated weights for policy 1, policy_version 451014 (0.0008) [2023-12-27 08:45:48,265][20381] Updated weights for policy 1, policy_version 451024 (0.0007) [2023-12-27 08:45:48,716][20347] Updated weights for policy 0, policy_version 451338 (0.0010) [2023-12-27 08:45:48,774][20347] Updated weights for policy 0, policy_version 451348 (0.0010) [2023-12-27 08:45:48,834][20347] Updated weights for policy 0, policy_version 451358 (0.0005) [2023-12-27 08:45:48,982][20381] Updated weights for policy 1, policy_version 451034 (0.0008) [2023-12-27 08:45:49,038][20381] Updated weights for policy 1, policy_version 451044 (0.0007) [2023-12-27 08:45:49,095][20381] Updated weights for policy 1, policy_version 451054 (0.0007) [2023-12-27 08:45:49,474][20347] Updated weights for policy 0, policy_version 451368 (0.0005) [2023-12-27 08:45:49,541][20347] Updated weights for policy 0, policy_version 451378 (0.0005) [2023-12-27 08:45:49,606][20347] Updated weights for policy 0, policy_version 451388 (0.0006) [2023-12-27 08:45:49,798][20381] Updated weights for policy 1, policy_version 451064 (0.0007) [2023-12-27 08:45:49,856][20381] Updated weights for policy 1, policy_version 451074 (0.0007) [2023-12-27 08:45:49,918][20381] Updated weights for policy 1, policy_version 451084 (0.0009) [2023-12-27 08:45:50,173][20347] Updated weights for policy 0, policy_version 451398 (0.0006) [2023-12-27 08:45:50,222][20347] Updated weights for policy 0, policy_version 451408 (0.0007) [2023-12-27 08:45:50,273][20347] Updated weights for policy 0, policy_version 451418 (0.0009) [2023-12-27 08:45:50,612][20381] Updated weights for policy 1, policy_version 451094 (0.0008) [2023-12-27 08:45:50,675][20381] Updated weights for policy 1, policy_version 451104 (0.0007) [2023-12-27 08:45:50,741][20381] Updated weights for policy 1, policy_version 451114 (0.0008) [2023-12-27 08:45:50,939][20347] Updated weights for policy 0, policy_version 451428 (0.0007) [2023-12-27 08:45:50,997][20347] Updated weights for policy 0, policy_version 451438 (0.0005) [2023-12-27 08:45:51,066][20347] Updated weights for policy 0, policy_version 451448 (0.0007) [2023-12-27 08:45:51,479][20381] Updated weights for policy 1, policy_version 451124 (0.0007) [2023-12-27 08:45:51,542][20381] Updated weights for policy 1, policy_version 451134 (0.0008) [2023-12-27 08:45:51,606][20381] Updated weights for policy 1, policy_version 451144 (0.0008) [2023-12-27 08:45:51,695][20347] Updated weights for policy 0, policy_version 451458 (0.0007) [2023-12-27 08:45:51,747][20347] Updated weights for policy 0, policy_version 451468 (0.0008) [2023-12-27 08:45:51,799][20347] Updated weights for policy 0, policy_version 451478 (0.0005) [2023-12-27 08:45:51,861][20347] Updated weights for policy 0, policy_version 451488 (0.0005) [2023-12-27 08:45:52,312][20381] Updated weights for policy 1, policy_version 451154 (0.0007) [2023-12-27 08:45:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 231112704. Throughput: 0: 10280.5, 1: 10376.9. Samples: 231086112. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:52,363][19170] Avg episode reward: [(0, '943.780'), (1, '942.040')] [2023-12-27 08:45:52,367][20381] Updated weights for policy 1, policy_version 451164 (0.0007) [2023-12-27 08:45:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000451488_115597312.pth... [2023-12-27 08:45:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000450272_115286016.pth [2023-12-27 08:45:52,430][20381] Updated weights for policy 1, policy_version 451174 (0.0008) [2023-12-27 08:45:52,490][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000451184_115523584.pth... [2023-12-27 08:45:52,492][20381] Updated weights for policy 1, policy_version 451184 (0.0007) [2023-12-27 08:45:52,493][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000449968_115212288.pth [2023-12-27 08:45:52,498][20347] Updated weights for policy 0, policy_version 451498 (0.0006) [2023-12-27 08:45:52,552][20347] Updated weights for policy 0, policy_version 451508 (0.0008) [2023-12-27 08:45:52,607][20347] Updated weights for policy 0, policy_version 451518 (0.0007) [2023-12-27 08:45:53,153][20381] Updated weights for policy 1, policy_version 451194 (0.0007) [2023-12-27 08:45:53,202][20381] Updated weights for policy 1, policy_version 451204 (0.0008) [2023-12-27 08:45:53,254][20381] Updated weights for policy 1, policy_version 451214 (0.0008) [2023-12-27 08:45:53,287][20347] Updated weights for policy 0, policy_version 451528 (0.0006) [2023-12-27 08:45:53,336][20347] Updated weights for policy 0, policy_version 451538 (0.0005) [2023-12-27 08:45:53,383][20347] Updated weights for policy 0, policy_version 451548 (0.0005) [2023-12-27 08:45:53,923][20347] Updated weights for policy 0, policy_version 451558 (0.0006) [2023-12-27 08:45:53,972][20347] Updated weights for policy 0, policy_version 451568 (0.0007) [2023-12-27 08:45:54,013][20381] Updated weights for policy 1, policy_version 451224 (0.0006) [2023-12-27 08:45:54,023][20347] Updated weights for policy 0, policy_version 451578 (0.0007) [2023-12-27 08:45:54,066][20381] Updated weights for policy 1, policy_version 451234 (0.0006) [2023-12-27 08:45:54,115][20381] Updated weights for policy 1, policy_version 451244 (0.0008) [2023-12-27 08:45:54,629][20347] Updated weights for policy 0, policy_version 451588 (0.0006) [2023-12-27 08:45:54,685][20347] Updated weights for policy 0, policy_version 451598 (0.0007) [2023-12-27 08:45:54,740][20347] Updated weights for policy 0, policy_version 451608 (0.0007) [2023-12-27 08:45:54,850][20381] Updated weights for policy 1, policy_version 451254 (0.0009) [2023-12-27 08:45:54,899][20381] Updated weights for policy 1, policy_version 451264 (0.0008) [2023-12-27 08:45:54,943][20381] Updated weights for policy 1, policy_version 451274 (0.0005) [2023-12-27 08:45:55,377][20347] Updated weights for policy 0, policy_version 451618 (0.0006) [2023-12-27 08:45:55,439][20347] Updated weights for policy 0, policy_version 451628 (0.0006) [2023-12-27 08:45:55,495][20347] Updated weights for policy 0, policy_version 451638 (0.0007) [2023-12-27 08:45:55,542][20347] Updated weights for policy 0, policy_version 451648 (0.0007) [2023-12-27 08:45:55,646][20381] Updated weights for policy 1, policy_version 451284 (0.0006) [2023-12-27 08:45:55,709][20381] Updated weights for policy 1, policy_version 451294 (0.0007) [2023-12-27 08:45:55,755][20381] Updated weights for policy 1, policy_version 451304 (0.0008) [2023-12-27 08:45:56,162][20347] Updated weights for policy 0, policy_version 451658 (0.0005) [2023-12-27 08:45:56,225][20347] Updated weights for policy 0, policy_version 451668 (0.0006) [2023-12-27 08:45:56,276][20381] Updated weights for policy 1, policy_version 451314 (0.0005) [2023-12-27 08:45:56,280][20347] Updated weights for policy 0, policy_version 451678 (0.0007) [2023-12-27 08:45:56,339][20381] Updated weights for policy 1, policy_version 451324 (0.0005) [2023-12-27 08:45:56,403][20381] Updated weights for policy 1, policy_version 451334 (0.0007) [2023-12-27 08:45:56,461][20381] Updated weights for policy 1, policy_version 451344 (0.0006) [2023-12-27 08:45:56,835][20347] Updated weights for policy 0, policy_version 451688 (0.0007) [2023-12-27 08:45:56,879][20347] Updated weights for policy 0, policy_version 451698 (0.0009) [2023-12-27 08:45:56,925][20347] Updated weights for policy 0, policy_version 451708 (0.0007) [2023-12-27 08:45:57,033][20381] Updated weights for policy 1, policy_version 451354 (0.0009) [2023-12-27 08:45:57,092][20381] Updated weights for policy 1, policy_version 451364 (0.0008) [2023-12-27 08:45:57,148][20381] Updated weights for policy 1, policy_version 451374 (0.0005) [2023-12-27 08:45:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 231227392. Throughput: 0: 10339.7, 1: 10328.7. Samples: 231213308. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:45:57,363][19170] Avg episode reward: [(0, '980.820'), (1, '941.940')] [2023-12-27 08:45:57,544][20347] Updated weights for policy 0, policy_version 451718 (0.0008) [2023-12-27 08:45:57,601][20347] Updated weights for policy 0, policy_version 451728 (0.0007) [2023-12-27 08:45:57,654][20347] Updated weights for policy 0, policy_version 451738 (0.0007) [2023-12-27 08:45:57,821][20381] Updated weights for policy 1, policy_version 451384 (0.0010) [2023-12-27 08:45:57,872][20381] Updated weights for policy 1, policy_version 451394 (0.0006) [2023-12-27 08:45:57,930][20381] Updated weights for policy 1, policy_version 451404 (0.0006) [2023-12-27 08:45:58,355][20347] Updated weights for policy 0, policy_version 451748 (0.0007) [2023-12-27 08:45:58,418][20347] Updated weights for policy 0, policy_version 451758 (0.0007) [2023-12-27 08:45:58,477][20347] Updated weights for policy 0, policy_version 451768 (0.0008) [2023-12-27 08:45:58,613][20381] Updated weights for policy 1, policy_version 451414 (0.0006) [2023-12-27 08:45:58,665][20292] KL-divergence is very high: 393.8300 [2023-12-27 08:45:58,678][20381] Updated weights for policy 1, policy_version 451424 (0.0010) [2023-12-27 08:45:58,716][20292] KL-divergence is very high: 451.8894 [2023-12-27 08:45:58,743][20381] Updated weights for policy 1, policy_version 451434 (0.0009) [2023-12-27 08:45:58,769][20292] KL-divergence is very high: 276.9128 [2023-12-27 08:45:59,192][20347] Updated weights for policy 0, policy_version 451778 (0.0008) [2023-12-27 08:45:59,262][20347] Updated weights for policy 0, policy_version 451788 (0.0007) [2023-12-27 08:45:59,311][20347] Updated weights for policy 0, policy_version 451798 (0.0008) [2023-12-27 08:45:59,379][20347] Updated weights for policy 0, policy_version 451808 (0.0008) [2023-12-27 08:45:59,461][20381] Updated weights for policy 1, policy_version 451444 (0.0010) [2023-12-27 08:45:59,505][20381] Updated weights for policy 1, policy_version 451454 (0.0007) [2023-12-27 08:45:59,567][20381] Updated weights for policy 1, policy_version 451464 (0.0008) [2023-12-27 08:46:00,040][20347] Updated weights for policy 0, policy_version 451818 (0.0007) [2023-12-27 08:46:00,087][20347] Updated weights for policy 0, policy_version 451828 (0.0007) [2023-12-27 08:46:00,109][20291] KL-divergence is very high: 1034.8132 [2023-12-27 08:46:00,141][20347] Updated weights for policy 0, policy_version 451838 (0.0005) [2023-12-27 08:46:00,302][20381] Updated weights for policy 1, policy_version 451474 (0.0008) [2023-12-27 08:46:00,359][20381] Updated weights for policy 1, policy_version 451484 (0.0007) [2023-12-27 08:46:00,416][20381] Updated weights for policy 1, policy_version 451494 (0.0007) [2023-12-27 08:46:00,461][20381] Updated weights for policy 1, policy_version 451504 (0.0009) [2023-12-27 08:46:00,731][20347] Updated weights for policy 0, policy_version 451848 (0.0008) [2023-12-27 08:46:00,785][20347] Updated weights for policy 0, policy_version 451858 (0.0006) [2023-12-27 08:46:00,838][20347] Updated weights for policy 0, policy_version 451868 (0.0008) [2023-12-27 08:46:01,118][20381] Updated weights for policy 1, policy_version 451514 (0.0007) [2023-12-27 08:46:01,176][20381] Updated weights for policy 1, policy_version 451524 (0.0007) [2023-12-27 08:46:01,228][20381] Updated weights for policy 1, policy_version 451534 (0.0008) [2023-12-27 08:46:01,536][20347] Updated weights for policy 0, policy_version 451878 (0.0008) [2023-12-27 08:46:01,593][20347] Updated weights for policy 0, policy_version 451888 (0.0007) [2023-12-27 08:46:01,654][20347] Updated weights for policy 0, policy_version 451898 (0.0008) [2023-12-27 08:46:01,944][20381] Updated weights for policy 1, policy_version 451544 (0.0008) [2023-12-27 08:46:02,006][20381] Updated weights for policy 1, policy_version 451554 (0.0007) [2023-12-27 08:46:02,068][20381] Updated weights for policy 1, policy_version 451564 (0.0008) [2023-12-27 08:46:02,360][20347] Updated weights for policy 0, policy_version 451908 (0.0008) [2023-12-27 08:46:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 231325696. Throughput: 0: 10393.2, 1: 10249.2. Samples: 231337016. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:02,363][19170] Avg episode reward: [(0, '933.430'), (1, '961.550')] [2023-12-27 08:46:02,388][20291] KL-divergence is very high: 359.7722 [2023-12-27 08:46:02,402][20291] KL-divergence is very high: 403.9375 [2023-12-27 08:46:02,407][20291] KL-divergence is very high: 282.0369 [2023-12-27 08:46:02,412][20347] Updated weights for policy 0, policy_version 451918 (0.0007) [2023-12-27 08:46:02,426][20291] KL-divergence is very high: 179.0643 [2023-12-27 08:46:02,442][20291] KL-divergence is very high: 217.4931 [2023-12-27 08:46:02,446][20291] KL-divergence is very high: 212.1865 [2023-12-27 08:46:02,460][20347] Updated weights for policy 0, policy_version 451928 (0.0008) [2023-12-27 08:46:02,758][20381] Updated weights for policy 1, policy_version 451574 (0.0008) [2023-12-27 08:46:02,813][20381] Updated weights for policy 1, policy_version 451584 (0.0008) [2023-12-27 08:46:02,870][20381] Updated weights for policy 1, policy_version 451594 (0.0008) [2023-12-27 08:46:03,117][20347] Updated weights for policy 0, policy_version 451938 (0.0008) [2023-12-27 08:46:03,140][20291] KL-divergence is very high: 338.7599 [2023-12-27 08:46:03,167][20347] Updated weights for policy 0, policy_version 451948 (0.0007) [2023-12-27 08:46:03,177][20291] KL-divergence is very high: 414.2118 [2023-12-27 08:46:03,212][20291] KL-divergence is very high: 186.2685 [2023-12-27 08:46:03,213][20347] Updated weights for policy 0, policy_version 451958 (0.0006) [2023-12-27 08:46:03,248][20291] KL-divergence is very high: 154.6495 [2023-12-27 08:46:03,258][20347] Updated weights for policy 0, policy_version 451968 (0.0007) [2023-12-27 08:46:03,534][20381] Updated weights for policy 1, policy_version 451604 (0.0007) [2023-12-27 08:46:03,588][20381] Updated weights for policy 1, policy_version 451614 (0.0006) [2023-12-27 08:46:03,642][20381] Updated weights for policy 1, policy_version 451624 (0.0006) [2023-12-27 08:46:03,989][20347] Updated weights for policy 0, policy_version 451978 (0.0007) [2023-12-27 08:46:04,049][20347] Updated weights for policy 0, policy_version 451988 (0.0008) [2023-12-27 08:46:04,109][20347] Updated weights for policy 0, policy_version 451998 (0.0008) [2023-12-27 08:46:04,271][20381] Updated weights for policy 1, policy_version 451634 (0.0006) [2023-12-27 08:46:04,334][20381] Updated weights for policy 1, policy_version 451644 (0.0007) [2023-12-27 08:46:04,392][20381] Updated weights for policy 1, policy_version 451654 (0.0010) [2023-12-27 08:46:04,451][20381] Updated weights for policy 1, policy_version 451664 (0.0011) [2023-12-27 08:46:04,842][20347] Updated weights for policy 0, policy_version 452008 (0.0008) [2023-12-27 08:46:04,896][20347] Updated weights for policy 0, policy_version 452018 (0.0007) [2023-12-27 08:46:04,945][20347] Updated weights for policy 0, policy_version 452028 (0.0009) [2023-12-27 08:46:05,033][20381] Updated weights for policy 1, policy_version 451674 (0.0010) [2023-12-27 08:46:05,081][20381] Updated weights for policy 1, policy_version 451684 (0.0010) [2023-12-27 08:46:05,130][20381] Updated weights for policy 1, policy_version 451694 (0.0010) [2023-12-27 08:46:05,669][20347] Updated weights for policy 0, policy_version 452038 (0.0008) [2023-12-27 08:46:05,723][20347] Updated weights for policy 0, policy_version 452048 (0.0008) [2023-12-27 08:46:05,734][20381] Updated weights for policy 1, policy_version 451704 (0.0006) [2023-12-27 08:46:05,769][20347] Updated weights for policy 0, policy_version 452058 (0.0007) [2023-12-27 08:46:05,786][20381] Updated weights for policy 1, policy_version 451714 (0.0007) [2023-12-27 08:46:05,834][20381] Updated weights for policy 1, policy_version 451724 (0.0010) [2023-12-27 08:46:06,419][20381] Updated weights for policy 1, policy_version 451734 (0.0008) [2023-12-27 08:46:06,486][20381] Updated weights for policy 1, policy_version 451744 (0.0008) [2023-12-27 08:46:06,547][20381] Updated weights for policy 1, policy_version 451754 (0.0008) [2023-12-27 08:46:06,567][20347] Updated weights for policy 0, policy_version 452068 (0.0006) [2023-12-27 08:46:06,629][20347] Updated weights for policy 0, policy_version 452078 (0.0007) [2023-12-27 08:46:06,685][20347] Updated weights for policy 0, policy_version 452088 (0.0007) [2023-12-27 08:46:07,278][20381] Updated weights for policy 1, policy_version 451764 (0.0008) [2023-12-27 08:46:07,335][20381] Updated weights for policy 1, policy_version 451774 (0.0006) [2023-12-27 08:46:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 231424000. Throughput: 0: 10388.0, 1: 10308.5. Samples: 231398876. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:07,363][19170] Avg episode reward: [(0, '827.320'), (1, '952.730')] [2023-12-27 08:46:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000452096_115752960.pth... [2023-12-27 08:46:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000450880_115441664.pth [2023-12-27 08:46:07,396][20381] Updated weights for policy 1, policy_version 451784 (0.0005) [2023-12-27 08:46:07,414][20347] Updated weights for policy 0, policy_version 452098 (0.0010) [2023-12-27 08:46:07,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000451792_115679232.pth... [2023-12-27 08:46:07,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000450576_115367936.pth [2023-12-27 08:46:07,476][20347] Updated weights for policy 0, policy_version 452108 (0.0007) [2023-12-27 08:46:07,525][20347] Updated weights for policy 0, policy_version 452118 (0.0007) [2023-12-27 08:46:07,569][20347] Updated weights for policy 0, policy_version 452128 (0.0007) [2023-12-27 08:46:07,970][20381] Updated weights for policy 1, policy_version 451794 (0.0007) [2023-12-27 08:46:08,029][20381] Updated weights for policy 1, policy_version 451804 (0.0010) [2023-12-27 08:46:08,084][20381] Updated weights for policy 1, policy_version 451814 (0.0007) [2023-12-27 08:46:08,092][20292] KL-divergence is very high: 170.4180 [2023-12-27 08:46:08,143][20292] KL-divergence is very high: 120.2473 [2023-12-27 08:46:08,148][20381] Updated weights for policy 1, policy_version 451824 (0.0006) [2023-12-27 08:46:08,306][20347] Updated weights for policy 0, policy_version 452138 (0.0007) [2023-12-27 08:46:08,372][20347] Updated weights for policy 0, policy_version 452148 (0.0008) [2023-12-27 08:46:08,436][20347] Updated weights for policy 0, policy_version 452158 (0.0008) [2023-12-27 08:46:08,807][20381] Updated weights for policy 1, policy_version 451834 (0.0009) [2023-12-27 08:46:08,854][20381] Updated weights for policy 1, policy_version 451844 (0.0009) [2023-12-27 08:46:08,909][20381] Updated weights for policy 1, policy_version 451854 (0.0007) [2023-12-27 08:46:09,115][20347] Updated weights for policy 0, policy_version 452168 (0.0006) [2023-12-27 08:46:09,170][20347] Updated weights for policy 0, policy_version 452178 (0.0005) [2023-12-27 08:46:09,232][20347] Updated weights for policy 0, policy_version 452188 (0.0009) [2023-12-27 08:46:09,553][20381] Updated weights for policy 1, policy_version 451864 (0.0008) [2023-12-27 08:46:09,606][20381] Updated weights for policy 1, policy_version 451874 (0.0009) [2023-12-27 08:46:09,667][20381] Updated weights for policy 1, policy_version 451884 (0.0009) [2023-12-27 08:46:09,899][20347] Updated weights for policy 0, policy_version 452198 (0.0008) [2023-12-27 08:46:09,966][20347] Updated weights for policy 0, policy_version 452208 (0.0008) [2023-12-27 08:46:10,033][20347] Updated weights for policy 0, policy_version 452218 (0.0008) [2023-12-27 08:46:10,416][20381] Updated weights for policy 1, policy_version 451894 (0.0007) [2023-12-27 08:46:10,469][20381] Updated weights for policy 1, policy_version 451904 (0.0007) [2023-12-27 08:46:10,522][20292] KL-divergence is very high: 180.2783 [2023-12-27 08:46:10,523][20381] Updated weights for policy 1, policy_version 451914 (0.0006) [2023-12-27 08:46:10,711][20347] Updated weights for policy 0, policy_version 452228 (0.0008) [2023-12-27 08:46:10,762][20347] Updated weights for policy 0, policy_version 452238 (0.0008) [2023-12-27 08:46:10,813][20347] Updated weights for policy 0, policy_version 452248 (0.0006) [2023-12-27 08:46:11,170][20381] Updated weights for policy 1, policy_version 451924 (0.0006) [2023-12-27 08:46:11,228][20381] Updated weights for policy 1, policy_version 451934 (0.0006) [2023-12-27 08:46:11,279][20381] Updated weights for policy 1, policy_version 451944 (0.0006) [2023-12-27 08:46:11,572][20347] Updated weights for policy 0, policy_version 452258 (0.0006) [2023-12-27 08:46:11,620][20347] Updated weights for policy 0, policy_version 452268 (0.0007) [2023-12-27 08:46:11,679][20347] Updated weights for policy 0, policy_version 452278 (0.0008) [2023-12-27 08:46:11,743][20347] Updated weights for policy 0, policy_version 452288 (0.0009) [2023-12-27 08:46:11,937][20381] Updated weights for policy 1, policy_version 451954 (0.0007) [2023-12-27 08:46:11,993][20381] Updated weights for policy 1, policy_version 451964 (0.0008) [2023-12-27 08:46:12,042][20381] Updated weights for policy 1, policy_version 451974 (0.0009) [2023-12-27 08:46:12,089][20381] Updated weights for policy 1, policy_version 451984 (0.0008) [2023-12-27 08:46:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 231530496. Throughput: 0: 10397.0, 1: 10379.0. Samples: 231522576. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:12,363][19170] Avg episode reward: [(0, '828.290'), (1, '915.010')] [2023-12-27 08:46:12,415][20347] Updated weights for policy 0, policy_version 452298 (0.0007) [2023-12-27 08:46:12,479][20347] Updated weights for policy 0, policy_version 452308 (0.0008) [2023-12-27 08:46:12,540][20347] Updated weights for policy 0, policy_version 452318 (0.0005) [2023-12-27 08:46:12,806][20381] Updated weights for policy 1, policy_version 451994 (0.0009) [2023-12-27 08:46:12,861][20381] Updated weights for policy 1, policy_version 452004 (0.0006) [2023-12-27 08:46:12,906][20381] Updated weights for policy 1, policy_version 452014 (0.0008) [2023-12-27 08:46:13,119][20347] Updated weights for policy 0, policy_version 452328 (0.0007) [2023-12-27 08:46:13,132][20291] KL-divergence is very high: 148.3791 [2023-12-27 08:46:13,156][20291] KL-divergence is very high: 107.4409 [2023-12-27 08:46:13,181][20347] Updated weights for policy 0, policy_version 452338 (0.0006) [2023-12-27 08:46:13,238][20347] Updated weights for policy 0, policy_version 452348 (0.0005) [2023-12-27 08:46:13,590][20381] Updated weights for policy 1, policy_version 452024 (0.0007) [2023-12-27 08:46:13,640][20381] Updated weights for policy 1, policy_version 452034 (0.0007) [2023-12-27 08:46:13,693][20381] Updated weights for policy 1, policy_version 452044 (0.0008) [2023-12-27 08:46:13,849][20347] Updated weights for policy 0, policy_version 452358 (0.0008) [2023-12-27 08:46:13,903][20347] Updated weights for policy 0, policy_version 452368 (0.0010) [2023-12-27 08:46:13,954][20347] Updated weights for policy 0, policy_version 452378 (0.0010) [2023-12-27 08:46:14,387][20381] Updated weights for policy 1, policy_version 452054 (0.0008) [2023-12-27 08:46:14,434][20381] Updated weights for policy 1, policy_version 452064 (0.0007) [2023-12-27 08:46:14,492][20381] Updated weights for policy 1, policy_version 452074 (0.0007) [2023-12-27 08:46:14,586][20347] Updated weights for policy 0, policy_version 452388 (0.0010) [2023-12-27 08:46:14,640][20347] Updated weights for policy 0, policy_version 452398 (0.0010) [2023-12-27 08:46:14,694][20347] Updated weights for policy 0, policy_version 452408 (0.0010) [2023-12-27 08:46:15,215][20381] Updated weights for policy 1, policy_version 452084 (0.0007) [2023-12-27 08:46:15,273][20381] Updated weights for policy 1, policy_version 452094 (0.0009) [2023-12-27 08:46:15,335][20381] Updated weights for policy 1, policy_version 452104 (0.0007) [2023-12-27 08:46:15,357][20347] Updated weights for policy 0, policy_version 452418 (0.0010) [2023-12-27 08:46:15,416][20347] Updated weights for policy 0, policy_version 452428 (0.0007) [2023-12-27 08:46:15,467][20347] Updated weights for policy 0, policy_version 452438 (0.0006) [2023-12-27 08:46:15,517][20347] Updated weights for policy 0, policy_version 452448 (0.0005) [2023-12-27 08:46:16,054][20381] Updated weights for policy 1, policy_version 452114 (0.0007) [2023-12-27 08:46:16,110][20381] Updated weights for policy 1, policy_version 452124 (0.0005) [2023-12-27 08:46:16,155][20347] Updated weights for policy 0, policy_version 452458 (0.0010) [2023-12-27 08:46:16,164][20381] Updated weights for policy 1, policy_version 452134 (0.0005) [2023-12-27 08:46:16,212][20347] Updated weights for policy 0, policy_version 452468 (0.0008) [2023-12-27 08:46:16,222][20381] Updated weights for policy 1, policy_version 452144 (0.0006) [2023-12-27 08:46:16,262][20347] Updated weights for policy 0, policy_version 452478 (0.0010) [2023-12-27 08:46:16,882][20381] Updated weights for policy 1, policy_version 452154 (0.0008) [2023-12-27 08:46:16,924][20347] Updated weights for policy 0, policy_version 452488 (0.0006) [2023-12-27 08:46:16,931][20381] Updated weights for policy 1, policy_version 452164 (0.0008) [2023-12-27 08:46:16,977][20347] Updated weights for policy 0, policy_version 452498 (0.0005) [2023-12-27 08:46:16,985][20381] Updated weights for policy 1, policy_version 452174 (0.0008) [2023-12-27 08:46:17,023][20347] Updated weights for policy 0, policy_version 452508 (0.0005) [2023-12-27 08:46:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 231636992. Throughput: 0: 10441.3, 1: 10344.0. Samples: 231646392. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:17,364][19170] Avg episode reward: [(0, '830.490'), (1, '915.010')] [2023-12-27 08:46:17,550][20347] Updated weights for policy 0, policy_version 452518 (0.0005) [2023-12-27 08:46:17,596][20347] Updated weights for policy 0, policy_version 452528 (0.0005) [2023-12-27 08:46:17,642][20347] Updated weights for policy 0, policy_version 452538 (0.0005) [2023-12-27 08:46:17,764][20381] Updated weights for policy 1, policy_version 452184 (0.0006) [2023-12-27 08:46:17,818][20381] Updated weights for policy 1, policy_version 452194 (0.0005) [2023-12-27 08:46:17,864][20381] Updated weights for policy 1, policy_version 452204 (0.0005) [2023-12-27 08:46:18,203][20347] Updated weights for policy 0, policy_version 452548 (0.0007) [2023-12-27 08:46:18,204][20291] KL-divergence is very high: 100.4928 [2023-12-27 08:46:18,209][20291] KL-divergence is very high: 116.2100 [2023-12-27 08:46:18,263][20347] Updated weights for policy 0, policy_version 452558 (0.0010) [2023-12-27 08:46:18,305][20291] KL-divergence is very high: 102.6824 [2023-12-27 08:46:18,320][20347] Updated weights for policy 0, policy_version 452568 (0.0010) [2023-12-27 08:46:18,520][20381] Updated weights for policy 1, policy_version 452214 (0.0006) [2023-12-27 08:46:18,569][20381] Updated weights for policy 1, policy_version 452224 (0.0007) [2023-12-27 08:46:18,621][20381] Updated weights for policy 1, policy_version 452234 (0.0008) [2023-12-27 08:46:18,963][20347] Updated weights for policy 0, policy_version 452578 (0.0010) [2023-12-27 08:46:19,013][20347] Updated weights for policy 0, policy_version 452588 (0.0010) [2023-12-27 08:46:19,061][20347] Updated weights for policy 0, policy_version 452598 (0.0010) [2023-12-27 08:46:19,109][20347] Updated weights for policy 0, policy_version 452608 (0.0010) [2023-12-27 08:46:19,353][20381] Updated weights for policy 1, policy_version 452244 (0.0008) [2023-12-27 08:46:19,415][20381] Updated weights for policy 1, policy_version 452254 (0.0007) [2023-12-27 08:46:19,477][20381] Updated weights for policy 1, policy_version 452264 (0.0008) [2023-12-27 08:46:19,842][20347] Updated weights for policy 0, policy_version 452618 (0.0011) [2023-12-27 08:46:19,899][20347] Updated weights for policy 0, policy_version 452628 (0.0010) [2023-12-27 08:46:19,964][20347] Updated weights for policy 0, policy_version 452638 (0.0011) [2023-12-27 08:46:20,172][20381] Updated weights for policy 1, policy_version 452274 (0.0008) [2023-12-27 08:46:20,228][20381] Updated weights for policy 1, policy_version 452284 (0.0010) [2023-12-27 08:46:20,276][20381] Updated weights for policy 1, policy_version 452294 (0.0010) [2023-12-27 08:46:20,328][20381] Updated weights for policy 1, policy_version 452304 (0.0010) [2023-12-27 08:46:20,642][20347] Updated weights for policy 0, policy_version 452648 (0.0010) [2023-12-27 08:46:20,705][20347] Updated weights for policy 0, policy_version 452658 (0.0008) [2023-12-27 08:46:20,769][20347] Updated weights for policy 0, policy_version 452668 (0.0005) [2023-12-27 08:46:21,010][20292] KL-divergence is very high: 144.6600 [2023-12-27 08:46:21,052][20381] Updated weights for policy 1, policy_version 452314 (0.0011) [2023-12-27 08:46:21,068][20292] KL-divergence is very high: 133.0521 [2023-12-27 08:46:21,122][20381] Updated weights for policy 1, policy_version 452324 (0.0009) [2023-12-27 08:46:21,184][20381] Updated weights for policy 1, policy_version 452334 (0.0009) [2023-12-27 08:46:21,429][20347] Updated weights for policy 0, policy_version 452678 (0.0006) [2023-12-27 08:46:21,490][20347] Updated weights for policy 0, policy_version 452688 (0.0007) [2023-12-27 08:46:21,549][20347] Updated weights for policy 0, policy_version 452698 (0.0008) [2023-12-27 08:46:21,812][20381] Updated weights for policy 1, policy_version 452344 (0.0007) [2023-12-27 08:46:21,869][20381] Updated weights for policy 1, policy_version 452354 (0.0008) [2023-12-27 08:46:21,933][20381] Updated weights for policy 1, policy_version 452364 (0.0007) [2023-12-27 08:46:22,252][20347] Updated weights for policy 0, policy_version 452708 (0.0008) [2023-12-27 08:46:22,314][20347] Updated weights for policy 0, policy_version 452718 (0.0009) [2023-12-27 08:46:22,340][20291] KL-divergence is very high: 223.0683 [2023-12-27 08:46:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 231735296. Throughput: 0: 10493.8, 1: 10273.0. Samples: 231709456. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:22,363][19170] Avg episode reward: [(0, '828.360'), (1, '895.280')] [2023-12-27 08:46:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000452368_115826688.pth... [2023-12-27 08:46:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000451184_115523584.pth [2023-12-27 08:46:22,376][20347] Updated weights for policy 0, policy_version 452728 (0.0009) [2023-12-27 08:46:22,421][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000452736_115916800.pth... [2023-12-27 08:46:22,425][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000451488_115597312.pth [2023-12-27 08:46:22,643][20381] Updated weights for policy 1, policy_version 452374 (0.0007) [2023-12-27 08:46:22,701][20381] Updated weights for policy 1, policy_version 452384 (0.0007) [2023-12-27 08:46:22,761][20381] Updated weights for policy 1, policy_version 452394 (0.0008) [2023-12-27 08:46:23,061][20347] Updated weights for policy 0, policy_version 452738 (0.0008) [2023-12-27 08:46:23,089][20291] KL-divergence is very high: 174.4751 [2023-12-27 08:46:23,115][20347] Updated weights for policy 0, policy_version 452748 (0.0005) [2023-12-27 08:46:23,167][20347] Updated weights for policy 0, policy_version 452758 (0.0005) [2023-12-27 08:46:23,218][20347] Updated weights for policy 0, policy_version 452768 (0.0005) [2023-12-27 08:46:23,440][20381] Updated weights for policy 1, policy_version 452404 (0.0008) [2023-12-27 08:46:23,488][20381] Updated weights for policy 1, policy_version 452414 (0.0009) [2023-12-27 08:46:23,543][20381] Updated weights for policy 1, policy_version 452424 (0.0007) [2023-12-27 08:46:23,743][20347] Updated weights for policy 0, policy_version 452778 (0.0005) [2023-12-27 08:46:23,796][20347] Updated weights for policy 0, policy_version 452788 (0.0005) [2023-12-27 08:46:23,842][20347] Updated weights for policy 0, policy_version 452798 (0.0005) [2023-12-27 08:46:24,265][20381] Updated weights for policy 1, policy_version 452434 (0.0008) [2023-12-27 08:46:24,325][20381] Updated weights for policy 1, policy_version 452444 (0.0008) [2023-12-27 08:46:24,382][20381] Updated weights for policy 1, policy_version 452454 (0.0008) [2023-12-27 08:46:24,441][20381] Updated weights for policy 1, policy_version 452464 (0.0008) [2023-12-27 08:46:24,467][20347] Updated weights for policy 0, policy_version 452808 (0.0007) [2023-12-27 08:46:24,517][20347] Updated weights for policy 0, policy_version 452818 (0.0006) [2023-12-27 08:46:24,567][20347] Updated weights for policy 0, policy_version 452828 (0.0007) [2023-12-27 08:46:25,136][20381] Updated weights for policy 1, policy_version 452474 (0.0007) [2023-12-27 08:46:25,172][20347] Updated weights for policy 0, policy_version 452838 (0.0006) [2023-12-27 08:46:25,195][20381] Updated weights for policy 1, policy_version 452484 (0.0010) [2023-12-27 08:46:25,232][20347] Updated weights for policy 0, policy_version 452848 (0.0005) [2023-12-27 08:46:25,254][20381] Updated weights for policy 1, policy_version 452494 (0.0011) [2023-12-27 08:46:25,289][20347] Updated weights for policy 0, policy_version 452858 (0.0005) [2023-12-27 08:46:25,901][20347] Updated weights for policy 0, policy_version 452868 (0.0006) [2023-12-27 08:46:25,922][20381] Updated weights for policy 1, policy_version 452504 (0.0007) [2023-12-27 08:46:25,951][20347] Updated weights for policy 0, policy_version 452878 (0.0006) [2023-12-27 08:46:25,976][20381] Updated weights for policy 1, policy_version 452514 (0.0006) [2023-12-27 08:46:25,998][20347] Updated weights for policy 0, policy_version 452888 (0.0007) [2023-12-27 08:46:26,022][20381] Updated weights for policy 1, policy_version 452524 (0.0011) [2023-12-27 08:46:26,594][20347] Updated weights for policy 0, policy_version 452898 (0.0006) [2023-12-27 08:46:26,640][20347] Updated weights for policy 0, policy_version 452908 (0.0007) [2023-12-27 08:46:26,682][20381] Updated weights for policy 1, policy_version 452534 (0.0011) [2023-12-27 08:46:26,689][20347] Updated weights for policy 0, policy_version 452918 (0.0008) [2023-12-27 08:46:26,733][20381] Updated weights for policy 1, policy_version 452544 (0.0010) [2023-12-27 08:46:26,738][20347] Updated weights for policy 0, policy_version 452928 (0.0008) [2023-12-27 08:46:26,787][20381] Updated weights for policy 1, policy_version 452554 (0.0010) [2023-12-27 08:46:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 231841792. Throughput: 0: 10574.9, 1: 10244.3. Samples: 231834412. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:27,363][19170] Avg episode reward: [(0, '770.470'), (1, '791.710')] [2023-12-27 08:46:27,391][20347] Updated weights for policy 0, policy_version 452938 (0.0007) [2023-12-27 08:46:27,440][20347] Updated weights for policy 0, policy_version 452948 (0.0005) [2023-12-27 08:46:27,456][20381] Updated weights for policy 1, policy_version 452564 (0.0011) [2023-12-27 08:46:27,490][20347] Updated weights for policy 0, policy_version 452958 (0.0007) [2023-12-27 08:46:27,490][20291] KL-divergence is very high: 180.2438 [2023-12-27 08:46:27,495][20291] KL-divergence is very high: 187.8319 [2023-12-27 08:46:27,504][20381] Updated weights for policy 1, policy_version 452574 (0.0010) [2023-12-27 08:46:27,553][20381] Updated weights for policy 1, policy_version 452584 (0.0010) [2023-12-27 08:46:28,169][20347] Updated weights for policy 0, policy_version 452968 (0.0008) [2023-12-27 08:46:28,204][20291] KL-divergence is very high: 126.9831 [2023-12-27 08:46:28,210][20291] KL-divergence is very high: 135.0689 [2023-12-27 08:46:28,228][20347] Updated weights for policy 0, policy_version 452978 (0.0007) [2023-12-27 08:46:28,246][20381] Updated weights for policy 1, policy_version 452594 (0.0010) [2023-12-27 08:46:28,277][20347] Updated weights for policy 0, policy_version 452988 (0.0006) [2023-12-27 08:46:28,304][20381] Updated weights for policy 1, policy_version 452604 (0.0011) [2023-12-27 08:46:28,361][20381] Updated weights for policy 1, policy_version 452614 (0.0010) [2023-12-27 08:46:28,421][20381] Updated weights for policy 1, policy_version 452624 (0.0011) [2023-12-27 08:46:28,994][20347] Updated weights for policy 0, policy_version 452998 (0.0008) [2023-12-27 08:46:29,021][20291] KL-divergence is very high: 148.0271 [2023-12-27 08:46:29,044][20347] Updated weights for policy 0, policy_version 453008 (0.0006) [2023-12-27 08:46:29,057][20291] KL-divergence is very high: 109.5869 [2023-12-27 08:46:29,062][20381] Updated weights for policy 1, policy_version 452634 (0.0010) [2023-12-27 08:46:29,090][20347] Updated weights for policy 0, policy_version 453018 (0.0006) [2023-12-27 08:46:29,120][20381] Updated weights for policy 1, policy_version 452644 (0.0010) [2023-12-27 08:46:29,174][20381] Updated weights for policy 1, policy_version 452654 (0.0011) [2023-12-27 08:46:29,811][20347] Updated weights for policy 0, policy_version 453028 (0.0007) [2023-12-27 08:46:29,875][20381] Updated weights for policy 1, policy_version 452664 (0.0011) [2023-12-27 08:46:29,878][20347] Updated weights for policy 0, policy_version 453038 (0.0009) [2023-12-27 08:46:29,937][20347] Updated weights for policy 0, policy_version 453048 (0.0006) [2023-12-27 08:46:29,938][20291] KL-divergence is very high: 194.4084 [2023-12-27 08:46:29,942][20381] Updated weights for policy 1, policy_version 452674 (0.0010) [2023-12-27 08:46:29,945][20291] KL-divergence is very high: 182.4546 [2023-12-27 08:46:29,950][20291] KL-divergence is very high: 247.7190 [2023-12-27 08:46:30,000][20381] Updated weights for policy 1, policy_version 452684 (0.0008) [2023-12-27 08:46:30,626][20347] Updated weights for policy 0, policy_version 453058 (0.0006) [2023-12-27 08:46:30,630][20291] KL-divergence is very high: 308.8715 [2023-12-27 08:46:30,670][20291] KL-divergence is very high: 183.0335 [2023-12-27 08:46:30,675][20347] Updated weights for policy 0, policy_version 453068 (0.0010) [2023-12-27 08:46:30,682][20381] Updated weights for policy 1, policy_version 452694 (0.0005) [2023-12-27 08:46:30,717][20291] KL-divergence is very high: 194.2536 [2023-12-27 08:46:30,734][20347] Updated weights for policy 0, policy_version 453078 (0.0010) [2023-12-27 08:46:30,741][20381] Updated weights for policy 1, policy_version 452704 (0.0005) [2023-12-27 08:46:30,761][20291] KL-divergence is very high: 181.0929 [2023-12-27 08:46:30,788][20347] Updated weights for policy 0, policy_version 453088 (0.0010) [2023-12-27 08:46:30,801][20381] Updated weights for policy 1, policy_version 452714 (0.0006) [2023-12-27 08:46:31,408][20381] Updated weights for policy 1, policy_version 452724 (0.0007) [2023-12-27 08:46:31,468][20381] Updated weights for policy 1, policy_version 452734 (0.0009) [2023-12-27 08:46:31,500][20347] Updated weights for policy 0, policy_version 453098 (0.0006) [2023-12-27 08:46:31,527][20381] Updated weights for policy 1, policy_version 452744 (0.0007) [2023-12-27 08:46:31,553][20347] Updated weights for policy 0, policy_version 453108 (0.0005) [2023-12-27 08:46:31,610][20347] Updated weights for policy 0, policy_version 453118 (0.0006) [2023-12-27 08:46:32,190][20381] Updated weights for policy 1, policy_version 452754 (0.0008) [2023-12-27 08:46:32,249][20381] Updated weights for policy 1, policy_version 452764 (0.0010) [2023-12-27 08:46:32,305][20381] Updated weights for policy 1, policy_version 452774 (0.0010) [2023-12-27 08:46:32,315][20347] Updated weights for policy 0, policy_version 453128 (0.0009) [2023-12-27 08:46:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 231948288. Throughput: 0: 10527.3, 1: 10232.9. Samples: 231958324. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:32,362][20381] Updated weights for policy 1, policy_version 452784 (0.0008) [2023-12-27 08:46:32,363][19170] Avg episode reward: [(0, '621.790'), (1, '746.530')] [2023-12-27 08:46:32,375][20347] Updated weights for policy 0, policy_version 453138 (0.0010) [2023-12-27 08:46:32,430][20347] Updated weights for policy 0, policy_version 453148 (0.0010) [2023-12-27 08:46:32,964][20381] Updated weights for policy 1, policy_version 452794 (0.0010) [2023-12-27 08:46:33,015][20381] Updated weights for policy 1, policy_version 452804 (0.0008) [2023-12-27 08:46:33,066][20381] Updated weights for policy 1, policy_version 452814 (0.0010) [2023-12-27 08:46:33,101][20347] Updated weights for policy 0, policy_version 453158 (0.0009) [2023-12-27 08:46:33,158][20347] Updated weights for policy 0, policy_version 453168 (0.0007) [2023-12-27 08:46:33,210][20347] Updated weights for policy 0, policy_version 453178 (0.0008) [2023-12-27 08:46:33,714][20381] Updated weights for policy 1, policy_version 452824 (0.0010) [2023-12-27 08:46:33,770][20381] Updated weights for policy 1, policy_version 452834 (0.0010) [2023-12-27 08:46:33,814][20381] Updated weights for policy 1, policy_version 452844 (0.0008) [2023-12-27 08:46:33,930][20347] Updated weights for policy 0, policy_version 453188 (0.0007) [2023-12-27 08:46:33,991][20347] Updated weights for policy 0, policy_version 453198 (0.0008) [2023-12-27 08:46:34,052][20347] Updated weights for policy 0, policy_version 453208 (0.0008) [2023-12-27 08:46:34,425][20381] Updated weights for policy 1, policy_version 452854 (0.0008) [2023-12-27 08:46:34,480][20381] Updated weights for policy 1, policy_version 452864 (0.0010) [2023-12-27 08:46:34,536][20381] Updated weights for policy 1, policy_version 452874 (0.0010) [2023-12-27 08:46:34,721][20347] Updated weights for policy 0, policy_version 453218 (0.0009) [2023-12-27 08:46:34,783][20347] Updated weights for policy 0, policy_version 453228 (0.0007) [2023-12-27 08:46:34,846][20347] Updated weights for policy 0, policy_version 453238 (0.0008) [2023-12-27 08:46:34,896][20347] Updated weights for policy 0, policy_version 453248 (0.0009) [2023-12-27 08:46:35,135][20381] Updated weights for policy 1, policy_version 452884 (0.0011) [2023-12-27 08:46:35,196][20381] Updated weights for policy 1, policy_version 452894 (0.0010) [2023-12-27 08:46:35,253][20381] Updated weights for policy 1, policy_version 452904 (0.0006) [2023-12-27 08:46:35,623][20347] Updated weights for policy 0, policy_version 453258 (0.0007) [2023-12-27 08:46:35,671][20347] Updated weights for policy 0, policy_version 453268 (0.0007) [2023-12-27 08:46:35,729][20347] Updated weights for policy 0, policy_version 453278 (0.0007) [2023-12-27 08:46:35,877][20381] Updated weights for policy 1, policy_version 452914 (0.0005) [2023-12-27 08:46:35,939][20381] Updated weights for policy 1, policy_version 452924 (0.0008) [2023-12-27 08:46:35,997][20381] Updated weights for policy 1, policy_version 452934 (0.0010) [2023-12-27 08:46:36,054][20381] Updated weights for policy 1, policy_version 452944 (0.0005) [2023-12-27 08:46:36,402][20347] Updated weights for policy 0, policy_version 453288 (0.0009) [2023-12-27 08:46:36,453][20347] Updated weights for policy 0, policy_version 453298 (0.0006) [2023-12-27 08:46:36,509][20347] Updated weights for policy 0, policy_version 453308 (0.0010) [2023-12-27 08:46:36,751][20381] Updated weights for policy 1, policy_version 452954 (0.0011) [2023-12-27 08:46:36,801][20381] Updated weights for policy 1, policy_version 452964 (0.0011) [2023-12-27 08:46:36,845][20381] Updated weights for policy 1, policy_version 452974 (0.0008) [2023-12-27 08:46:37,210][20347] Updated weights for policy 0, policy_version 453318 (0.0010) [2023-12-27 08:46:37,275][20347] Updated weights for policy 0, policy_version 453328 (0.0010) [2023-12-27 08:46:37,337][20347] Updated weights for policy 0, policy_version 453338 (0.0008) [2023-12-27 08:46:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 232046592. Throughput: 0: 10476.9, 1: 10303.8. Samples: 232021240. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:37,363][19170] Avg episode reward: [(0, '481.010'), (1, '764.020')] [2023-12-27 08:46:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000452976_115982336.pth... [2023-12-27 08:46:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000451792_115679232.pth [2023-12-27 08:46:37,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000453344_116072448.pth... [2023-12-27 08:46:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000452096_115752960.pth [2023-12-27 08:46:37,557][20381] Updated weights for policy 1, policy_version 452984 (0.0008) [2023-12-27 08:46:37,610][20381] Updated weights for policy 1, policy_version 452994 (0.0007) [2023-12-27 08:46:37,674][20381] Updated weights for policy 1, policy_version 453004 (0.0005) [2023-12-27 08:46:37,960][20347] Updated weights for policy 0, policy_version 453348 (0.0007) [2023-12-27 08:46:38,019][20347] Updated weights for policy 0, policy_version 453358 (0.0005) [2023-12-27 08:46:38,080][20347] Updated weights for policy 0, policy_version 453368 (0.0005) [2023-12-27 08:46:38,321][20381] Updated weights for policy 1, policy_version 453014 (0.0010) [2023-12-27 08:46:38,374][20381] Updated weights for policy 1, policy_version 453024 (0.0010) [2023-12-27 08:46:38,424][20381] Updated weights for policy 1, policy_version 453034 (0.0008) [2023-12-27 08:46:38,706][20347] Updated weights for policy 0, policy_version 453378 (0.0007) [2023-12-27 08:46:38,768][20347] Updated weights for policy 0, policy_version 453388 (0.0010) [2023-12-27 08:46:38,827][20347] Updated weights for policy 0, policy_version 453398 (0.0010) [2023-12-27 08:46:38,886][20347] Updated weights for policy 0, policy_version 453408 (0.0006) [2023-12-27 08:46:39,085][20381] Updated weights for policy 1, policy_version 453044 (0.0008) [2023-12-27 08:46:39,147][20381] Updated weights for policy 1, policy_version 453054 (0.0011) [2023-12-27 08:46:39,207][20381] Updated weights for policy 1, policy_version 453064 (0.0010) [2023-12-27 08:46:39,552][20347] Updated weights for policy 0, policy_version 453418 (0.0009) [2023-12-27 08:46:39,615][20291] KL-divergence is very high: 111.7477 [2023-12-27 08:46:39,622][20291] KL-divergence is very high: 111.1541 [2023-12-27 08:46:39,652][20347] Updated weights for policy 0, policy_version 453428 (0.0009) [2023-12-27 08:46:39,716][20347] Updated weights for policy 0, policy_version 453438 (0.0007) [2023-12-27 08:46:39,877][20381] Updated weights for policy 1, policy_version 453074 (0.0008) [2023-12-27 08:46:39,933][20381] Updated weights for policy 1, policy_version 453084 (0.0007) [2023-12-27 08:46:39,995][20381] Updated weights for policy 1, policy_version 453094 (0.0009) [2023-12-27 08:46:40,048][20381] Updated weights for policy 1, policy_version 453104 (0.0007) [2023-12-27 08:46:40,386][20347] Updated weights for policy 0, policy_version 453448 (0.0008) [2023-12-27 08:46:40,442][20347] Updated weights for policy 0, policy_version 453458 (0.0007) [2023-12-27 08:46:40,443][20291] KL-divergence is very high: 158.2298 [2023-12-27 08:46:40,495][20347] Updated weights for policy 0, policy_version 453468 (0.0008) [2023-12-27 08:46:40,746][20381] Updated weights for policy 1, policy_version 453114 (0.0009) [2023-12-27 08:46:40,797][20381] Updated weights for policy 1, policy_version 453124 (0.0007) [2023-12-27 08:46:40,851][20381] Updated weights for policy 1, policy_version 453134 (0.0008) [2023-12-27 08:46:41,058][20347] Updated weights for policy 0, policy_version 453478 (0.0007) [2023-12-27 08:46:41,130][20347] Updated weights for policy 0, policy_version 453488 (0.0009) [2023-12-27 08:46:41,190][20347] Updated weights for policy 0, policy_version 453498 (0.0009) [2023-12-27 08:46:41,629][20381] Updated weights for policy 1, policy_version 453144 (0.0008) [2023-12-27 08:46:41,688][20381] Updated weights for policy 1, policy_version 453154 (0.0008) [2023-12-27 08:46:41,748][20381] Updated weights for policy 1, policy_version 453164 (0.0009) [2023-12-27 08:46:41,882][20347] Updated weights for policy 0, policy_version 453508 (0.0008) [2023-12-27 08:46:41,946][20347] Updated weights for policy 0, policy_version 453518 (0.0008) [2023-12-27 08:46:42,005][20347] Updated weights for policy 0, policy_version 453528 (0.0008) [2023-12-27 08:46:42,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 232153088. Throughput: 0: 10403.2, 1: 10308.1. Samples: 232145316. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:42,363][19170] Avg episode reward: [(0, '398.360'), (1, '876.760')] [2023-12-27 08:46:42,409][20381] Updated weights for policy 1, policy_version 453174 (0.0008) [2023-12-27 08:46:42,460][20381] Updated weights for policy 1, policy_version 453184 (0.0007) [2023-12-27 08:46:42,510][20381] Updated weights for policy 1, policy_version 453194 (0.0007) [2023-12-27 08:46:42,645][20347] Updated weights for policy 0, policy_version 453538 (0.0007) [2023-12-27 08:46:42,710][20347] Updated weights for policy 0, policy_version 453548 (0.0007) [2023-12-27 08:46:42,773][20347] Updated weights for policy 0, policy_version 453558 (0.0007) [2023-12-27 08:46:42,829][20347] Updated weights for policy 0, policy_version 453568 (0.0008) [2023-12-27 08:46:43,209][20381] Updated weights for policy 1, policy_version 453204 (0.0007) [2023-12-27 08:46:43,273][20381] Updated weights for policy 1, policy_version 453214 (0.0008) [2023-12-27 08:46:43,320][20381] Updated weights for policy 1, policy_version 453224 (0.0007) [2023-12-27 08:46:43,523][20347] Updated weights for policy 0, policy_version 453578 (0.0009) [2023-12-27 08:46:43,576][20347] Updated weights for policy 0, policy_version 453588 (0.0007) [2023-12-27 08:46:43,631][20347] Updated weights for policy 0, policy_version 453598 (0.0009) [2023-12-27 08:46:43,921][20381] Updated weights for policy 1, policy_version 453234 (0.0006) [2023-12-27 08:46:43,979][20381] Updated weights for policy 1, policy_version 453244 (0.0006) [2023-12-27 08:46:44,026][20381] Updated weights for policy 1, policy_version 453254 (0.0007) [2023-12-27 08:46:44,076][20381] Updated weights for policy 1, policy_version 453264 (0.0007) [2023-12-27 08:46:44,356][20347] Updated weights for policy 0, policy_version 453608 (0.0008) [2023-12-27 08:46:44,408][20347] Updated weights for policy 0, policy_version 453618 (0.0008) [2023-12-27 08:46:44,470][20347] Updated weights for policy 0, policy_version 453628 (0.0008) [2023-12-27 08:46:44,754][20381] Updated weights for policy 1, policy_version 453274 (0.0009) [2023-12-27 08:46:44,818][20381] Updated weights for policy 1, policy_version 453284 (0.0008) [2023-12-27 08:46:44,872][20381] Updated weights for policy 1, policy_version 453294 (0.0008) [2023-12-27 08:46:45,104][20347] Updated weights for policy 0, policy_version 453638 (0.0009) [2023-12-27 08:46:45,156][20347] Updated weights for policy 0, policy_version 453648 (0.0010) [2023-12-27 08:46:45,210][20347] Updated weights for policy 0, policy_version 453658 (0.0010) [2023-12-27 08:46:45,577][20381] Updated weights for policy 1, policy_version 453304 (0.0007) [2023-12-27 08:46:45,626][20381] Updated weights for policy 1, policy_version 453314 (0.0008) [2023-12-27 08:46:45,678][20381] Updated weights for policy 1, policy_version 453324 (0.0008) [2023-12-27 08:46:45,880][20347] Updated weights for policy 0, policy_version 453668 (0.0010) [2023-12-27 08:46:45,934][20347] Updated weights for policy 0, policy_version 453678 (0.0010) [2023-12-27 08:46:45,979][20347] Updated weights for policy 0, policy_version 453688 (0.0010) [2023-12-27 08:46:46,395][20381] Updated weights for policy 1, policy_version 453334 (0.0007) [2023-12-27 08:46:46,451][20381] Updated weights for policy 1, policy_version 453344 (0.0005) [2023-12-27 08:46:46,499][20381] Updated weights for policy 1, policy_version 453354 (0.0005) [2023-12-27 08:46:46,670][20347] Updated weights for policy 0, policy_version 453698 (0.0010) [2023-12-27 08:46:46,718][20347] Updated weights for policy 0, policy_version 453708 (0.0010) [2023-12-27 08:46:46,762][20347] Updated weights for policy 0, policy_version 453718 (0.0010) [2023-12-27 08:46:46,810][20347] Updated weights for policy 0, policy_version 453728 (0.0010) [2023-12-27 08:46:47,129][20381] Updated weights for policy 1, policy_version 453364 (0.0006) [2023-12-27 08:46:47,174][20381] Updated weights for policy 1, policy_version 453374 (0.0005) [2023-12-27 08:46:47,227][20381] Updated weights for policy 1, policy_version 453384 (0.0007) [2023-12-27 08:46:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 232259584. Throughput: 0: 10353.6, 1: 10338.5. Samples: 232268164. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:47,364][19170] Avg episode reward: [(0, '214.960'), (1, '933.190')] [2023-12-27 08:46:47,502][20347] Updated weights for policy 0, policy_version 453738 (0.0010) [2023-12-27 08:46:47,548][20347] Updated weights for policy 0, policy_version 453748 (0.0010) [2023-12-27 08:46:47,593][20347] Updated weights for policy 0, policy_version 453758 (0.0010) [2023-12-27 08:46:47,823][20381] Updated weights for policy 1, policy_version 453394 (0.0010) [2023-12-27 08:46:47,881][20381] Updated weights for policy 1, policy_version 453404 (0.0006) [2023-12-27 08:46:47,931][20381] Updated weights for policy 1, policy_version 453414 (0.0005) [2023-12-27 08:46:47,979][20381] Updated weights for policy 1, policy_version 453424 (0.0005) [2023-12-27 08:46:48,299][20347] Updated weights for policy 0, policy_version 453768 (0.0010) [2023-12-27 08:46:48,364][20347] Updated weights for policy 0, policy_version 453778 (0.0010) [2023-12-27 08:46:48,430][20347] Updated weights for policy 0, policy_version 453788 (0.0011) [2023-12-27 08:46:48,541][20381] Updated weights for policy 1, policy_version 453434 (0.0005) [2023-12-27 08:46:48,604][20381] Updated weights for policy 1, policy_version 453444 (0.0009) [2023-12-27 08:46:48,669][20381] Updated weights for policy 1, policy_version 453454 (0.0010) [2023-12-27 08:46:49,045][20347] Updated weights for policy 0, policy_version 453798 (0.0010) [2023-12-27 08:46:49,099][20347] Updated weights for policy 0, policy_version 453808 (0.0010) [2023-12-27 08:46:49,158][20347] Updated weights for policy 0, policy_version 453818 (0.0011) [2023-12-27 08:46:49,316][20381] Updated weights for policy 1, policy_version 453464 (0.0009) [2023-12-27 08:46:49,378][20381] Updated weights for policy 1, policy_version 453474 (0.0008) [2023-12-27 08:46:49,427][20381] Updated weights for policy 1, policy_version 453484 (0.0006) [2023-12-27 08:46:49,915][20347] Updated weights for policy 0, policy_version 453828 (0.0010) [2023-12-27 08:46:49,975][20347] Updated weights for policy 0, policy_version 453838 (0.0007) [2023-12-27 08:46:50,034][20347] Updated weights for policy 0, policy_version 453848 (0.0007) [2023-12-27 08:46:50,107][20381] Updated weights for policy 1, policy_version 453494 (0.0006) [2023-12-27 08:46:50,167][20381] Updated weights for policy 1, policy_version 453504 (0.0006) [2023-12-27 08:46:50,226][20381] Updated weights for policy 1, policy_version 453514 (0.0005) [2023-12-27 08:46:50,782][20347] Updated weights for policy 0, policy_version 453858 (0.0007) [2023-12-27 08:46:50,814][20381] Updated weights for policy 1, policy_version 453524 (0.0008) [2023-12-27 08:46:50,832][20347] Updated weights for policy 0, policy_version 453868 (0.0007) [2023-12-27 08:46:50,863][20381] Updated weights for policy 1, policy_version 453534 (0.0007) [2023-12-27 08:46:50,881][20347] Updated weights for policy 0, policy_version 453878 (0.0006) [2023-12-27 08:46:50,924][20381] Updated weights for policy 1, policy_version 453544 (0.0006) [2023-12-27 08:46:50,937][20347] Updated weights for policy 0, policy_version 453888 (0.0005) [2023-12-27 08:46:51,600][20381] Updated weights for policy 1, policy_version 453554 (0.0007) [2023-12-27 08:46:51,662][20347] Updated weights for policy 0, policy_version 453898 (0.0007) [2023-12-27 08:46:51,663][20381] Updated weights for policy 1, policy_version 453564 (0.0007) [2023-12-27 08:46:51,719][20381] Updated weights for policy 1, policy_version 453574 (0.0006) [2023-12-27 08:46:51,723][20347] Updated weights for policy 0, policy_version 453908 (0.0007) [2023-12-27 08:46:51,780][20381] Updated weights for policy 1, policy_version 453584 (0.0006) [2023-12-27 08:46:51,782][20347] Updated weights for policy 0, policy_version 453918 (0.0007) [2023-12-27 08:46:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 232357888. Throughput: 0: 10363.6, 1: 10364.3. Samples: 232331632. Policy #0 lag: (min: 21.0, avg: 28.6, max: 53.0) [2023-12-27 08:46:52,363][19170] Avg episode reward: [(0, '100.800'), (1, '905.980')] [2023-12-27 08:46:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000453584_116137984.pth... [2023-12-27 08:46:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000453920_116219904.pth... [2023-12-27 08:46:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000452368_115826688.pth [2023-12-27 08:46:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000452736_115916800.pth [2023-12-27 08:46:52,447][20347] Updated weights for policy 0, policy_version 453928 (0.0006) [2023-12-27 08:46:52,479][20381] Updated weights for policy 1, policy_version 453594 (0.0006) [2023-12-27 08:46:52,507][20347] Updated weights for policy 0, policy_version 453938 (0.0007) [2023-12-27 08:46:52,529][20381] Updated weights for policy 1, policy_version 453604 (0.0006) [2023-12-27 08:46:52,567][20347] Updated weights for policy 0, policy_version 453948 (0.0008) [2023-12-27 08:46:52,590][20381] Updated weights for policy 1, policy_version 453614 (0.0006) [2023-12-27 08:46:53,198][20381] Updated weights for policy 1, policy_version 453624 (0.0007) [2023-12-27 08:46:53,248][20381] Updated weights for policy 1, policy_version 453634 (0.0006) [2023-12-27 08:46:53,249][20347] Updated weights for policy 0, policy_version 453958 (0.0006) [2023-12-27 08:46:53,296][20381] Updated weights for policy 1, policy_version 453644 (0.0006) [2023-12-27 08:46:53,299][20347] Updated weights for policy 0, policy_version 453968 (0.0006) [2023-12-27 08:46:53,358][20347] Updated weights for policy 0, policy_version 453978 (0.0005) [2023-12-27 08:46:53,929][20347] Updated weights for policy 0, policy_version 453988 (0.0006) [2023-12-27 08:46:53,981][20347] Updated weights for policy 0, policy_version 453998 (0.0008) [2023-12-27 08:46:54,025][20381] Updated weights for policy 1, policy_version 453654 (0.0006) [2023-12-27 08:46:54,038][20347] Updated weights for policy 0, policy_version 454008 (0.0006) [2023-12-27 08:46:54,087][20381] Updated weights for policy 1, policy_version 453664 (0.0007) [2023-12-27 08:46:54,145][20381] Updated weights for policy 1, policy_version 453674 (0.0008) [2023-12-27 08:46:54,718][20347] Updated weights for policy 0, policy_version 454018 (0.0006) [2023-12-27 08:46:54,772][20347] Updated weights for policy 0, policy_version 454028 (0.0008) [2023-12-27 08:46:54,821][20347] Updated weights for policy 0, policy_version 454038 (0.0006) [2023-12-27 08:46:54,851][20381] Updated weights for policy 1, policy_version 453684 (0.0008) [2023-12-27 08:46:54,880][20347] Updated weights for policy 0, policy_version 454048 (0.0005) [2023-12-27 08:46:54,902][20381] Updated weights for policy 1, policy_version 453694 (0.0008) [2023-12-27 08:46:54,952][20381] Updated weights for policy 1, policy_version 453704 (0.0010) [2023-12-27 08:46:55,431][20347] Updated weights for policy 0, policy_version 454058 (0.0007) [2023-12-27 08:46:55,480][20347] Updated weights for policy 0, policy_version 454068 (0.0007) [2023-12-27 08:46:55,531][20347] Updated weights for policy 0, policy_version 454078 (0.0009) [2023-12-27 08:46:55,611][20381] Updated weights for policy 1, policy_version 453714 (0.0008) [2023-12-27 08:46:55,671][20381] Updated weights for policy 1, policy_version 453724 (0.0007) [2023-12-27 08:46:55,672][20292] KL-divergence is very high: 198.0331 [2023-12-27 08:46:55,719][20292] KL-divergence is very high: 129.1914 [2023-12-27 08:46:55,730][20381] Updated weights for policy 1, policy_version 453734 (0.0007) [2023-12-27 08:46:55,780][20381] Updated weights for policy 1, policy_version 453744 (0.0008) [2023-12-27 08:46:56,138][20347] Updated weights for policy 0, policy_version 454088 (0.0007) [2023-12-27 08:46:56,192][20347] Updated weights for policy 0, policy_version 454098 (0.0007) [2023-12-27 08:46:56,247][20347] Updated weights for policy 0, policy_version 454108 (0.0007) [2023-12-27 08:46:56,506][20292] KL-divergence is very high: 112.6586 [2023-12-27 08:46:56,512][20381] Updated weights for policy 1, policy_version 453754 (0.0008) [2023-12-27 08:46:56,560][20381] Updated weights for policy 1, policy_version 453764 (0.0008) [2023-12-27 08:46:56,616][20381] Updated weights for policy 1, policy_version 453774 (0.0007) [2023-12-27 08:46:56,821][20347] Updated weights for policy 0, policy_version 454118 (0.0006) [2023-12-27 08:46:56,873][20347] Updated weights for policy 0, policy_version 454128 (0.0005) [2023-12-27 08:46:56,928][20347] Updated weights for policy 0, policy_version 454138 (0.0005) [2023-12-27 08:46:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 232464384. Throughput: 0: 10455.7, 1: 10305.5. Samples: 232456828. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:46:57,363][19170] Avg episode reward: [(0, '316.380'), (1, '886.690')] [2023-12-27 08:46:57,399][20381] Updated weights for policy 1, policy_version 453784 (0.0008) [2023-12-27 08:46:57,451][20381] Updated weights for policy 1, policy_version 453794 (0.0008) [2023-12-27 08:46:57,488][20347] Updated weights for policy 0, policy_version 454148 (0.0006) [2023-12-27 08:46:57,509][20381] Updated weights for policy 1, policy_version 453804 (0.0006) [2023-12-27 08:46:57,540][20347] Updated weights for policy 0, policy_version 454158 (0.0006) [2023-12-27 08:46:57,585][20347] Updated weights for policy 0, policy_version 454168 (0.0007) [2023-12-27 08:46:58,162][20347] Updated weights for policy 0, policy_version 454178 (0.0007) [2023-12-27 08:46:58,200][20381] Updated weights for policy 1, policy_version 453814 (0.0008) [2023-12-27 08:46:58,219][20347] Updated weights for policy 0, policy_version 454188 (0.0008) [2023-12-27 08:46:58,259][20381] Updated weights for policy 1, policy_version 453824 (0.0010) [2023-12-27 08:46:58,277][20347] Updated weights for policy 0, policy_version 454198 (0.0006) [2023-12-27 08:46:58,317][20381] Updated weights for policy 1, policy_version 453834 (0.0011) [2023-12-27 08:46:58,341][20347] Updated weights for policy 0, policy_version 454208 (0.0006) [2023-12-27 08:46:59,070][20347] Updated weights for policy 0, policy_version 454218 (0.0008) [2023-12-27 08:46:59,106][20381] Updated weights for policy 1, policy_version 453844 (0.0008) [2023-12-27 08:46:59,129][20347] Updated weights for policy 0, policy_version 454228 (0.0007) [2023-12-27 08:46:59,162][20381] Updated weights for policy 1, policy_version 453854 (0.0007) [2023-12-27 08:46:59,164][20292] KL-divergence is very high: 135.0835 [2023-12-27 08:46:59,190][20347] Updated weights for policy 0, policy_version 454238 (0.0007) [2023-12-27 08:46:59,218][20381] Updated weights for policy 1, policy_version 453864 (0.0008) [2023-12-27 08:46:59,851][20347] Updated weights for policy 0, policy_version 454248 (0.0006) [2023-12-27 08:46:59,918][20347] Updated weights for policy 0, policy_version 454258 (0.0007) [2023-12-27 08:46:59,971][20381] Updated weights for policy 1, policy_version 453874 (0.0007) [2023-12-27 08:46:59,983][20347] Updated weights for policy 0, policy_version 454268 (0.0007) [2023-12-27 08:47:00,032][20381] Updated weights for policy 1, policy_version 453884 (0.0008) [2023-12-27 08:47:00,089][20381] Updated weights for policy 1, policy_version 453894 (0.0008) [2023-12-27 08:47:00,149][20381] Updated weights for policy 1, policy_version 453904 (0.0008) [2023-12-27 08:47:00,536][20347] Updated weights for policy 0, policy_version 454278 (0.0005) [2023-12-27 08:47:00,591][20347] Updated weights for policy 0, policy_version 454288 (0.0006) [2023-12-27 08:47:00,650][20347] Updated weights for policy 0, policy_version 454298 (0.0005) [2023-12-27 08:47:00,867][20381] Updated weights for policy 1, policy_version 453914 (0.0008) [2023-12-27 08:47:00,916][20381] Updated weights for policy 1, policy_version 453924 (0.0009) [2023-12-27 08:47:00,973][20381] Updated weights for policy 1, policy_version 453934 (0.0007) [2023-12-27 08:47:01,240][20347] Updated weights for policy 0, policy_version 454308 (0.0007) [2023-12-27 08:47:01,307][20347] Updated weights for policy 0, policy_version 454318 (0.0008) [2023-12-27 08:47:01,368][20347] Updated weights for policy 0, policy_version 454328 (0.0008) [2023-12-27 08:47:01,679][20381] Updated weights for policy 1, policy_version 453944 (0.0007) [2023-12-27 08:47:01,738][20381] Updated weights for policy 1, policy_version 453954 (0.0007) [2023-12-27 08:47:01,786][20381] Updated weights for policy 1, policy_version 453964 (0.0007) [2023-12-27 08:47:01,961][20347] Updated weights for policy 0, policy_version 454338 (0.0006) [2023-12-27 08:47:02,014][20347] Updated weights for policy 0, policy_version 454348 (0.0008) [2023-12-27 08:47:02,071][20347] Updated weights for policy 0, policy_version 454358 (0.0008) [2023-12-27 08:47:02,121][20347] Updated weights for policy 0, policy_version 454368 (0.0008) [2023-12-27 08:47:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 232570880. Throughput: 0: 10528.1, 1: 10252.1. Samples: 232581500. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:02,363][19170] Avg episode reward: [(0, '322.360'), (1, '760.410')] [2023-12-27 08:47:02,501][20381] Updated weights for policy 1, policy_version 453974 (0.0008) [2023-12-27 08:47:02,557][20381] Updated weights for policy 1, policy_version 453984 (0.0008) [2023-12-27 08:47:02,606][20381] Updated weights for policy 1, policy_version 453994 (0.0007) [2023-12-27 08:47:02,822][20347] Updated weights for policy 0, policy_version 454378 (0.0008) [2023-12-27 08:47:02,878][20347] Updated weights for policy 0, policy_version 454388 (0.0007) [2023-12-27 08:47:02,926][20347] Updated weights for policy 0, policy_version 454398 (0.0005) [2023-12-27 08:47:03,320][20381] Updated weights for policy 1, policy_version 454004 (0.0008) [2023-12-27 08:47:03,378][20381] Updated weights for policy 1, policy_version 454014 (0.0008) [2023-12-27 08:47:03,429][20381] Updated weights for policy 1, policy_version 454024 (0.0008) [2023-12-27 08:47:03,543][20347] Updated weights for policy 0, policy_version 454408 (0.0008) [2023-12-27 08:47:03,599][20347] Updated weights for policy 0, policy_version 454418 (0.0008) [2023-12-27 08:47:03,657][20347] Updated weights for policy 0, policy_version 454428 (0.0008) [2023-12-27 08:47:04,106][20381] Updated weights for policy 1, policy_version 454034 (0.0008) [2023-12-27 08:47:04,167][20381] Updated weights for policy 1, policy_version 454044 (0.0011) [2023-12-27 08:47:04,219][20381] Updated weights for policy 1, policy_version 454054 (0.0010) [2023-12-27 08:47:04,286][20381] Updated weights for policy 1, policy_version 454064 (0.0010) [2023-12-27 08:47:04,325][20347] Updated weights for policy 0, policy_version 454438 (0.0007) [2023-12-27 08:47:04,391][20347] Updated weights for policy 0, policy_version 454448 (0.0008) [2023-12-27 08:47:04,459][20347] Updated weights for policy 0, policy_version 454458 (0.0008) [2023-12-27 08:47:04,996][20381] Updated weights for policy 1, policy_version 454074 (0.0007) [2023-12-27 08:47:05,010][20347] Updated weights for policy 0, policy_version 454468 (0.0007) [2023-12-27 08:47:05,054][20381] Updated weights for policy 1, policy_version 454084 (0.0007) [2023-12-27 08:47:05,075][20347] Updated weights for policy 0, policy_version 454478 (0.0006) [2023-12-27 08:47:05,115][20381] Updated weights for policy 1, policy_version 454094 (0.0008) [2023-12-27 08:47:05,139][20347] Updated weights for policy 0, policy_version 454488 (0.0007) [2023-12-27 08:47:05,695][20347] Updated weights for policy 0, policy_version 454498 (0.0007) [2023-12-27 08:47:05,730][20381] Updated weights for policy 1, policy_version 454104 (0.0010) [2023-12-27 08:47:05,751][20347] Updated weights for policy 0, policy_version 454508 (0.0006) [2023-12-27 08:47:05,786][20381] Updated weights for policy 1, policy_version 454114 (0.0011) [2023-12-27 08:47:05,808][20347] Updated weights for policy 0, policy_version 454518 (0.0007) [2023-12-27 08:47:05,835][20381] Updated weights for policy 1, policy_version 454124 (0.0010) [2023-12-27 08:47:05,856][20347] Updated weights for policy 0, policy_version 454528 (0.0008) [2023-12-27 08:47:06,480][20347] Updated weights for policy 0, policy_version 454538 (0.0011) [2023-12-27 08:47:06,520][20381] Updated weights for policy 1, policy_version 454134 (0.0011) [2023-12-27 08:47:06,539][20347] Updated weights for policy 0, policy_version 454548 (0.0010) [2023-12-27 08:47:06,578][20381] Updated weights for policy 1, policy_version 454144 (0.0010) [2023-12-27 08:47:06,596][20347] Updated weights for policy 0, policy_version 454558 (0.0011) [2023-12-27 08:47:06,638][20381] Updated weights for policy 1, policy_version 454154 (0.0011) [2023-12-27 08:47:07,203][20347] Updated weights for policy 0, policy_version 454568 (0.0008) [2023-12-27 08:47:07,268][20347] Updated weights for policy 0, policy_version 454578 (0.0008) [2023-12-27 08:47:07,289][20381] Updated weights for policy 1, policy_version 454164 (0.0009) [2023-12-27 08:47:07,327][20347] Updated weights for policy 0, policy_version 454588 (0.0007) [2023-12-27 08:47:07,352][20381] Updated weights for policy 1, policy_version 454174 (0.0006) [2023-12-27 08:47:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 232677376. Throughput: 0: 10506.9, 1: 10257.7. Samples: 232643860. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:07,363][19170] Avg episode reward: [(0, '273.720'), (1, '759.910')] [2023-12-27 08:47:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000454592_116391936.pth... [2023-12-27 08:47:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000453344_116072448.pth [2023-12-27 08:47:07,414][20381] Updated weights for policy 1, policy_version 454184 (0.0006) [2023-12-27 08:47:07,460][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000454192_116293632.pth... [2023-12-27 08:47:07,464][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000452976_115982336.pth [2023-12-27 08:47:08,014][20347] Updated weights for policy 0, policy_version 454598 (0.0007) [2023-12-27 08:47:08,075][20347] Updated weights for policy 0, policy_version 454608 (0.0008) [2023-12-27 08:47:08,115][20381] Updated weights for policy 1, policy_version 454194 (0.0008) [2023-12-27 08:47:08,124][20347] Updated weights for policy 0, policy_version 454618 (0.0010) [2023-12-27 08:47:08,165][20381] Updated weights for policy 1, policy_version 454204 (0.0006) [2023-12-27 08:47:08,213][20381] Updated weights for policy 1, policy_version 454214 (0.0007) [2023-12-27 08:47:08,258][20381] Updated weights for policy 1, policy_version 454224 (0.0007) [2023-12-27 08:47:08,802][20347] Updated weights for policy 0, policy_version 454628 (0.0010) [2023-12-27 08:47:08,863][20347] Updated weights for policy 0, policy_version 454638 (0.0010) [2023-12-27 08:47:08,920][20347] Updated weights for policy 0, policy_version 454648 (0.0006) [2023-12-27 08:47:08,989][20381] Updated weights for policy 1, policy_version 454234 (0.0008) [2023-12-27 08:47:09,044][20381] Updated weights for policy 1, policy_version 454244 (0.0009) [2023-12-27 08:47:09,095][20381] Updated weights for policy 1, policy_version 454254 (0.0008) [2023-12-27 08:47:09,558][20347] Updated weights for policy 0, policy_version 454658 (0.0005) [2023-12-27 08:47:09,616][20347] Updated weights for policy 0, policy_version 454668 (0.0007) [2023-12-27 08:47:09,671][20347] Updated weights for policy 0, policy_version 454678 (0.0008) [2023-12-27 08:47:09,729][20347] Updated weights for policy 0, policy_version 454688 (0.0007) [2023-12-27 08:47:09,883][20381] Updated weights for policy 1, policy_version 454264 (0.0007) [2023-12-27 08:47:09,940][20381] Updated weights for policy 1, policy_version 454274 (0.0008) [2023-12-27 08:47:09,942][20292] KL-divergence is very high: 115.7097 [2023-12-27 08:47:09,996][20381] Updated weights for policy 1, policy_version 454284 (0.0008) [2023-12-27 08:47:10,435][20347] Updated weights for policy 0, policy_version 454698 (0.0006) [2023-12-27 08:47:10,494][20347] Updated weights for policy 0, policy_version 454708 (0.0007) [2023-12-27 08:47:10,554][20347] Updated weights for policy 0, policy_version 454718 (0.0006) [2023-12-27 08:47:10,745][20381] Updated weights for policy 1, policy_version 454294 (0.0006) [2023-12-27 08:47:10,791][20381] Updated weights for policy 1, policy_version 454304 (0.0005) [2023-12-27 08:47:10,857][20381] Updated weights for policy 1, policy_version 454314 (0.0011) [2023-12-27 08:47:11,208][20347] Updated weights for policy 0, policy_version 454728 (0.0008) [2023-12-27 08:47:11,256][20347] Updated weights for policy 0, policy_version 454738 (0.0007) [2023-12-27 08:47:11,304][20347] Updated weights for policy 0, policy_version 454748 (0.0008) [2023-12-27 08:47:11,566][20381] Updated weights for policy 1, policy_version 454324 (0.0009) [2023-12-27 08:47:11,635][20381] Updated weights for policy 1, policy_version 454334 (0.0008) [2023-12-27 08:47:11,691][20381] Updated weights for policy 1, policy_version 454344 (0.0010) [2023-12-27 08:47:12,093][20347] Updated weights for policy 0, policy_version 454758 (0.0007) [2023-12-27 08:47:12,144][20347] Updated weights for policy 0, policy_version 454768 (0.0006) [2023-12-27 08:47:12,202][20347] Updated weights for policy 0, policy_version 454778 (0.0008) [2023-12-27 08:47:12,315][20381] Updated weights for policy 1, policy_version 454354 (0.0010) [2023-12-27 08:47:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 232775680. Throughput: 0: 10492.6, 1: 10231.0. Samples: 232766976. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:12,363][19170] Avg episode reward: [(0, '271.200'), (1, '856.100')] [2023-12-27 08:47:12,379][20381] Updated weights for policy 1, policy_version 454364 (0.0009) [2023-12-27 08:47:12,431][20381] Updated weights for policy 1, policy_version 454374 (0.0008) [2023-12-27 08:47:12,475][20381] Updated weights for policy 1, policy_version 454384 (0.0008) [2023-12-27 08:47:12,888][20347] Updated weights for policy 0, policy_version 454788 (0.0007) [2023-12-27 08:47:12,940][20347] Updated weights for policy 0, policy_version 454798 (0.0005) [2023-12-27 08:47:13,000][20347] Updated weights for policy 0, policy_version 454808 (0.0005) [2023-12-27 08:47:13,230][20381] Updated weights for policy 1, policy_version 454394 (0.0008) [2023-12-27 08:47:13,285][20381] Updated weights for policy 1, policy_version 454404 (0.0010) [2023-12-27 08:47:13,333][20381] Updated weights for policy 1, policy_version 454414 (0.0008) [2023-12-27 08:47:13,532][20347] Updated weights for policy 0, policy_version 454818 (0.0006) [2023-12-27 08:47:13,578][20347] Updated weights for policy 0, policy_version 454828 (0.0007) [2023-12-27 08:47:13,627][20347] Updated weights for policy 0, policy_version 454838 (0.0006) [2023-12-27 08:47:13,677][20347] Updated weights for policy 0, policy_version 454848 (0.0007) [2023-12-27 08:47:14,074][20381] Updated weights for policy 1, policy_version 454424 (0.0008) [2023-12-27 08:47:14,126][20381] Updated weights for policy 1, policy_version 454434 (0.0008) [2023-12-27 08:47:14,183][20381] Updated weights for policy 1, policy_version 454444 (0.0007) [2023-12-27 08:47:14,323][20347] Updated weights for policy 0, policy_version 454858 (0.0005) [2023-12-27 08:47:14,370][20347] Updated weights for policy 0, policy_version 454868 (0.0005) [2023-12-27 08:47:14,417][20347] Updated weights for policy 0, policy_version 454878 (0.0007) [2023-12-27 08:47:14,830][20381] Updated weights for policy 1, policy_version 454454 (0.0008) [2023-12-27 08:47:14,885][20381] Updated weights for policy 1, policy_version 454464 (0.0007) [2023-12-27 08:47:14,940][20381] Updated weights for policy 1, policy_version 454474 (0.0008) [2023-12-27 08:47:15,109][20347] Updated weights for policy 0, policy_version 454888 (0.0010) [2023-12-27 08:47:15,170][20347] Updated weights for policy 0, policy_version 454898 (0.0010) [2023-12-27 08:47:15,235][20347] Updated weights for policy 0, policy_version 454908 (0.0010) [2023-12-27 08:47:15,658][20381] Updated weights for policy 1, policy_version 454484 (0.0007) [2023-12-27 08:47:15,709][20381] Updated weights for policy 1, policy_version 454494 (0.0007) [2023-12-27 08:47:15,761][20381] Updated weights for policy 1, policy_version 454504 (0.0008) [2023-12-27 08:47:15,900][20347] Updated weights for policy 0, policy_version 454918 (0.0010) [2023-12-27 08:47:15,947][20347] Updated weights for policy 0, policy_version 454928 (0.0009) [2023-12-27 08:47:15,994][20347] Updated weights for policy 0, policy_version 454938 (0.0009) [2023-12-27 08:47:16,499][20381] Updated weights for policy 1, policy_version 454514 (0.0008) [2023-12-27 08:47:16,556][20381] Updated weights for policy 1, policy_version 454524 (0.0007) [2023-12-27 08:47:16,610][20381] Updated weights for policy 1, policy_version 454534 (0.0008) [2023-12-27 08:47:16,626][20347] Updated weights for policy 0, policy_version 454948 (0.0008) [2023-12-27 08:47:16,664][20381] Updated weights for policy 1, policy_version 454544 (0.0006) [2023-12-27 08:47:16,681][20347] Updated weights for policy 0, policy_version 454958 (0.0006) [2023-12-27 08:47:16,736][20347] Updated weights for policy 0, policy_version 454968 (0.0007) [2023-12-27 08:47:17,272][20381] Updated weights for policy 1, policy_version 454554 (0.0010) [2023-12-27 08:47:17,331][20381] Updated weights for policy 1, policy_version 454564 (0.0007) [2023-12-27 08:47:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 232873984. Throughput: 0: 10514.6, 1: 10193.1. Samples: 232890168. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:17,363][19170] Avg episode reward: [(0, '180.650'), (1, '820.950')] [2023-12-27 08:47:17,376][20347] Updated weights for policy 0, policy_version 454978 (0.0008) [2023-12-27 08:47:17,386][20381] Updated weights for policy 1, policy_version 454574 (0.0009) [2023-12-27 08:47:17,433][20347] Updated weights for policy 0, policy_version 454988 (0.0005) [2023-12-27 08:47:17,491][20347] Updated weights for policy 0, policy_version 454998 (0.0006) [2023-12-27 08:47:17,545][20347] Updated weights for policy 0, policy_version 455008 (0.0005) [2023-12-27 08:47:18,091][20381] Updated weights for policy 1, policy_version 454584 (0.0008) [2023-12-27 08:47:18,124][20347] Updated weights for policy 0, policy_version 455018 (0.0010) [2023-12-27 08:47:18,151][20381] Updated weights for policy 1, policy_version 454594 (0.0006) [2023-12-27 08:47:18,184][20347] Updated weights for policy 0, policy_version 455028 (0.0007) [2023-12-27 08:47:18,213][20381] Updated weights for policy 1, policy_version 454604 (0.0006) [2023-12-27 08:47:18,243][20347] Updated weights for policy 0, policy_version 455038 (0.0007) [2023-12-27 08:47:18,905][20347] Updated weights for policy 0, policy_version 455048 (0.0010) [2023-12-27 08:47:18,937][20381] Updated weights for policy 1, policy_version 454614 (0.0007) [2023-12-27 08:47:18,960][20347] Updated weights for policy 0, policy_version 455058 (0.0010) [2023-12-27 08:47:18,991][20381] Updated weights for policy 1, policy_version 454624 (0.0005) [2023-12-27 08:47:19,011][20347] Updated weights for policy 0, policy_version 455068 (0.0010) [2023-12-27 08:47:19,046][20381] Updated weights for policy 1, policy_version 454634 (0.0006) [2023-12-27 08:47:19,739][20347] Updated weights for policy 0, policy_version 455078 (0.0010) [2023-12-27 08:47:19,773][20381] Updated weights for policy 1, policy_version 454644 (0.0008) [2023-12-27 08:47:19,789][20347] Updated weights for policy 0, policy_version 455088 (0.0010) [2023-12-27 08:47:19,828][20381] Updated weights for policy 1, policy_version 454654 (0.0009) [2023-12-27 08:47:19,852][20347] Updated weights for policy 0, policy_version 455098 (0.0010) [2023-12-27 08:47:19,887][20381] Updated weights for policy 1, policy_version 454664 (0.0006) [2023-12-27 08:47:20,554][20347] Updated weights for policy 0, policy_version 455108 (0.0011) [2023-12-27 08:47:20,618][20347] Updated weights for policy 0, policy_version 455118 (0.0009) [2023-12-27 08:47:20,618][20381] Updated weights for policy 1, policy_version 454674 (0.0008) [2023-12-27 08:47:20,671][20347] Updated weights for policy 0, policy_version 455128 (0.0007) [2023-12-27 08:47:20,676][20381] Updated weights for policy 1, policy_version 454684 (0.0006) [2023-12-27 08:47:20,738][20381] Updated weights for policy 1, policy_version 454694 (0.0007) [2023-12-27 08:47:20,800][20381] Updated weights for policy 1, policy_version 454704 (0.0008) [2023-12-27 08:47:21,402][20347] Updated weights for policy 0, policy_version 455138 (0.0009) [2023-12-27 08:47:21,459][20347] Updated weights for policy 0, policy_version 455148 (0.0009) [2023-12-27 08:47:21,521][20347] Updated weights for policy 0, policy_version 455158 (0.0009) [2023-12-27 08:47:21,558][20381] Updated weights for policy 1, policy_version 454714 (0.0009) [2023-12-27 08:47:21,579][20347] Updated weights for policy 0, policy_version 455168 (0.0009) [2023-12-27 08:47:21,620][20381] Updated weights for policy 1, policy_version 454724 (0.0009) [2023-12-27 08:47:21,680][20381] Updated weights for policy 1, policy_version 454734 (0.0008) [2023-12-27 08:47:22,316][20347] Updated weights for policy 0, policy_version 455178 (0.0011) [2023-12-27 08:47:22,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 232972288. Throughput: 0: 10566.3, 1: 10094.8. Samples: 232950992. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:22,363][19170] Avg episode reward: [(0, '194.560'), (1, '736.970')] [2023-12-27 08:47:22,383][20347] Updated weights for policy 0, policy_version 455188 (0.0010) [2023-12-27 08:47:22,424][20381] Updated weights for policy 1, policy_version 454744 (0.0009) [2023-12-27 08:47:22,448][20347] Updated weights for policy 0, policy_version 455198 (0.0009) [2023-12-27 08:47:22,457][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000455200_116547584.pth... [2023-12-27 08:47:22,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000453920_116219904.pth [2023-12-27 08:47:22,478][20381] Updated weights for policy 1, policy_version 454754 (0.0006) [2023-12-27 08:47:22,523][20381] Updated weights for policy 1, policy_version 454764 (0.0005) [2023-12-27 08:47:22,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000454768_116441088.pth... [2023-12-27 08:47:22,543][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000453584_116137984.pth [2023-12-27 08:47:23,114][20347] Updated weights for policy 0, policy_version 455208 (0.0009) [2023-12-27 08:47:23,167][20347] Updated weights for policy 0, policy_version 455218 (0.0007) [2023-12-27 08:47:23,217][20347] Updated weights for policy 0, policy_version 455228 (0.0006) [2023-12-27 08:47:23,257][20381] Updated weights for policy 1, policy_version 454774 (0.0008) [2023-12-27 08:47:23,313][20381] Updated weights for policy 1, policy_version 454784 (0.0008) [2023-12-27 08:47:23,374][20381] Updated weights for policy 1, policy_version 454794 (0.0008) [2023-12-27 08:47:23,870][20347] Updated weights for policy 0, policy_version 455238 (0.0008) [2023-12-27 08:47:23,918][20347] Updated weights for policy 0, policy_version 455248 (0.0008) [2023-12-27 08:47:23,967][20347] Updated weights for policy 0, policy_version 455258 (0.0009) [2023-12-27 08:47:24,073][20381] Updated weights for policy 1, policy_version 454804 (0.0008) [2023-12-27 08:47:24,124][20381] Updated weights for policy 1, policy_version 454814 (0.0008) [2023-12-27 08:47:24,178][20381] Updated weights for policy 1, policy_version 454824 (0.0008) [2023-12-27 08:47:24,634][20347] Updated weights for policy 0, policy_version 455268 (0.0007) [2023-12-27 08:47:24,688][20347] Updated weights for policy 0, policy_version 455278 (0.0007) [2023-12-27 08:47:24,742][20347] Updated weights for policy 0, policy_version 455288 (0.0008) [2023-12-27 08:47:24,903][20381] Updated weights for policy 1, policy_version 454834 (0.0008) [2023-12-27 08:47:24,950][20381] Updated weights for policy 1, policy_version 454844 (0.0008) [2023-12-27 08:47:24,995][20381] Updated weights for policy 1, policy_version 454854 (0.0008) [2023-12-27 08:47:25,045][20381] Updated weights for policy 1, policy_version 454864 (0.0009) [2023-12-27 08:47:25,310][20347] Updated weights for policy 0, policy_version 455298 (0.0007) [2023-12-27 08:47:25,368][20347] Updated weights for policy 0, policy_version 455308 (0.0007) [2023-12-27 08:47:25,421][20347] Updated weights for policy 0, policy_version 455318 (0.0008) [2023-12-27 08:47:25,472][20347] Updated weights for policy 0, policy_version 455328 (0.0007) [2023-12-27 08:47:25,730][20381] Updated weights for policy 1, policy_version 454874 (0.0006) [2023-12-27 08:47:25,775][20381] Updated weights for policy 1, policy_version 454884 (0.0008) [2023-12-27 08:47:25,828][20381] Updated weights for policy 1, policy_version 454894 (0.0007) [2023-12-27 08:47:26,144][20347] Updated weights for policy 0, policy_version 455338 (0.0005) [2023-12-27 08:47:26,204][20347] Updated weights for policy 0, policy_version 455348 (0.0005) [2023-12-27 08:47:26,266][20347] Updated weights for policy 0, policy_version 455358 (0.0005) [2023-12-27 08:47:26,595][20381] Updated weights for policy 1, policy_version 454904 (0.0007) [2023-12-27 08:47:26,641][20381] Updated weights for policy 1, policy_version 454914 (0.0007) [2023-12-27 08:47:26,689][20381] Updated weights for policy 1, policy_version 454924 (0.0008) [2023-12-27 08:47:26,821][20347] Updated weights for policy 0, policy_version 455368 (0.0007) [2023-12-27 08:47:26,867][20347] Updated weights for policy 0, policy_version 455378 (0.0007) [2023-12-27 08:47:26,912][20347] Updated weights for policy 0, policy_version 455388 (0.0008) [2023-12-27 08:47:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 233078784. Throughput: 0: 10538.5, 1: 10025.3. Samples: 233070688. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:27,363][20381] Updated weights for policy 1, policy_version 454934 (0.0007) [2023-12-27 08:47:27,363][19170] Avg episode reward: [(0, '198.830'), (1, '732.100')] [2023-12-27 08:47:27,421][20381] Updated weights for policy 1, policy_version 454944 (0.0005) [2023-12-27 08:47:27,482][20381] Updated weights for policy 1, policy_version 454954 (0.0009) [2023-12-27 08:47:27,637][20347] Updated weights for policy 0, policy_version 455398 (0.0007) [2023-12-27 08:47:27,686][20347] Updated weights for policy 0, policy_version 455408 (0.0005) [2023-12-27 08:47:27,737][20347] Updated weights for policy 0, policy_version 455418 (0.0006) [2023-12-27 08:47:28,045][20381] Updated weights for policy 1, policy_version 454964 (0.0010) [2023-12-27 08:47:28,103][20381] Updated weights for policy 1, policy_version 454974 (0.0010) [2023-12-27 08:47:28,163][20381] Updated weights for policy 1, policy_version 454984 (0.0009) [2023-12-27 08:47:28,351][20347] Updated weights for policy 0, policy_version 455428 (0.0008) [2023-12-27 08:47:28,409][20347] Updated weights for policy 0, policy_version 455438 (0.0007) [2023-12-27 08:47:28,469][20347] Updated weights for policy 0, policy_version 455448 (0.0007) [2023-12-27 08:47:28,807][20381] Updated weights for policy 1, policy_version 454994 (0.0006) [2023-12-27 08:47:28,862][20381] Updated weights for policy 1, policy_version 455004 (0.0005) [2023-12-27 08:47:28,913][20381] Updated weights for policy 1, policy_version 455014 (0.0005) [2023-12-27 08:47:28,962][20381] Updated weights for policy 1, policy_version 455024 (0.0009) [2023-12-27 08:47:29,071][20347] Updated weights for policy 0, policy_version 455458 (0.0007) [2023-12-27 08:47:29,128][20347] Updated weights for policy 0, policy_version 455468 (0.0007) [2023-12-27 08:47:29,177][20347] Updated weights for policy 0, policy_version 455478 (0.0007) [2023-12-27 08:47:29,237][20347] Updated weights for policy 0, policy_version 455488 (0.0006) [2023-12-27 08:47:29,604][20381] Updated weights for policy 1, policy_version 455034 (0.0011) [2023-12-27 08:47:29,658][20381] Updated weights for policy 1, policy_version 455044 (0.0011) [2023-12-27 08:47:29,713][20381] Updated weights for policy 1, policy_version 455054 (0.0010) [2023-12-27 08:47:29,908][20347] Updated weights for policy 0, policy_version 455498 (0.0007) [2023-12-27 08:47:29,975][20347] Updated weights for policy 0, policy_version 455508 (0.0008) [2023-12-27 08:47:30,022][20347] Updated weights for policy 0, policy_version 455518 (0.0008) [2023-12-27 08:47:30,386][20381] Updated weights for policy 1, policy_version 455064 (0.0010) [2023-12-27 08:47:30,443][20381] Updated weights for policy 1, policy_version 455074 (0.0010) [2023-12-27 08:47:30,493][20381] Updated weights for policy 1, policy_version 455084 (0.0010) [2023-12-27 08:47:30,644][20347] Updated weights for policy 0, policy_version 455528 (0.0006) [2023-12-27 08:47:30,692][20347] Updated weights for policy 0, policy_version 455538 (0.0007) [2023-12-27 08:47:30,745][20347] Updated weights for policy 0, policy_version 455548 (0.0007) [2023-12-27 08:47:31,147][20381] Updated weights for policy 1, policy_version 455094 (0.0008) [2023-12-27 08:47:31,205][20381] Updated weights for policy 1, policy_version 455104 (0.0006) [2023-12-27 08:47:31,270][20381] Updated weights for policy 1, policy_version 455114 (0.0006) [2023-12-27 08:47:31,458][20347] Updated weights for policy 0, policy_version 455558 (0.0006) [2023-12-27 08:47:31,518][20347] Updated weights for policy 0, policy_version 455568 (0.0006) [2023-12-27 08:47:31,572][20347] Updated weights for policy 0, policy_version 455578 (0.0007) [2023-12-27 08:47:31,921][20381] Updated weights for policy 1, policy_version 455124 (0.0007) [2023-12-27 08:47:31,975][20381] Updated weights for policy 1, policy_version 455134 (0.0008) [2023-12-27 08:47:32,030][20381] Updated weights for policy 1, policy_version 455144 (0.0007) [2023-12-27 08:47:32,274][20347] Updated weights for policy 0, policy_version 455588 (0.0007) [2023-12-27 08:47:32,329][20347] Updated weights for policy 0, policy_version 455598 (0.0008) [2023-12-27 08:47:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 233185280. Throughput: 0: 10619.9, 1: 10075.7. Samples: 233199464. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:32,363][19170] Avg episode reward: [(0, '276.350'), (1, '702.460')] [2023-12-27 08:47:32,386][20347] Updated weights for policy 0, policy_version 455608 (0.0007) [2023-12-27 08:47:32,620][20381] Updated weights for policy 1, policy_version 455154 (0.0006) [2023-12-27 08:47:32,666][20381] Updated weights for policy 1, policy_version 455164 (0.0008) [2023-12-27 08:47:32,725][20381] Updated weights for policy 1, policy_version 455174 (0.0008) [2023-12-27 08:47:32,783][20381] Updated weights for policy 1, policy_version 455184 (0.0005) [2023-12-27 08:47:33,062][20347] Updated weights for policy 0, policy_version 455618 (0.0007) [2023-12-27 08:47:33,124][20347] Updated weights for policy 0, policy_version 455628 (0.0008) [2023-12-27 08:47:33,177][20347] Updated weights for policy 0, policy_version 455638 (0.0007) [2023-12-27 08:47:33,224][20347] Updated weights for policy 0, policy_version 455648 (0.0007) [2023-12-27 08:47:33,436][20381] Updated weights for policy 1, policy_version 455194 (0.0008) [2023-12-27 08:47:33,498][20381] Updated weights for policy 1, policy_version 455204 (0.0008) [2023-12-27 08:47:33,554][20381] Updated weights for policy 1, policy_version 455214 (0.0007) [2023-12-27 08:47:33,885][20347] Updated weights for policy 0, policy_version 455658 (0.0005) [2023-12-27 08:47:33,933][20347] Updated weights for policy 0, policy_version 455668 (0.0006) [2023-12-27 08:47:33,978][20347] Updated weights for policy 0, policy_version 455678 (0.0007) [2023-12-27 08:47:34,273][20381] Updated weights for policy 1, policy_version 455224 (0.0009) [2023-12-27 08:47:34,337][20381] Updated weights for policy 1, policy_version 455234 (0.0008) [2023-12-27 08:47:34,395][20381] Updated weights for policy 1, policy_version 455244 (0.0008) [2023-12-27 08:47:34,592][20347] Updated weights for policy 0, policy_version 455688 (0.0007) [2023-12-27 08:47:34,639][20347] Updated weights for policy 0, policy_version 455698 (0.0007) [2023-12-27 08:47:34,686][20347] Updated weights for policy 0, policy_version 455708 (0.0007) [2023-12-27 08:47:35,115][20381] Updated weights for policy 1, policy_version 455254 (0.0008) [2023-12-27 08:47:35,157][20381] Updated weights for policy 1, policy_version 455264 (0.0008) [2023-12-27 08:47:35,212][20381] Updated weights for policy 1, policy_version 455274 (0.0008) [2023-12-27 08:47:35,402][20347] Updated weights for policy 0, policy_version 455718 (0.0007) [2023-12-27 08:47:35,453][20347] Updated weights for policy 0, policy_version 455728 (0.0008) [2023-12-27 08:47:35,510][20347] Updated weights for policy 0, policy_version 455738 (0.0006) [2023-12-27 08:47:35,834][20381] Updated weights for policy 1, policy_version 455284 (0.0007) [2023-12-27 08:47:35,890][20381] Updated weights for policy 1, policy_version 455294 (0.0005) [2023-12-27 08:47:35,949][20381] Updated weights for policy 1, policy_version 455304 (0.0007) [2023-12-27 08:47:36,213][20347] Updated weights for policy 0, policy_version 455748 (0.0006) [2023-12-27 08:47:36,260][20347] Updated weights for policy 0, policy_version 455758 (0.0010) [2023-12-27 08:47:36,306][20347] Updated weights for policy 0, policy_version 455768 (0.0009) [2023-12-27 08:47:36,549][20381] Updated weights for policy 1, policy_version 455314 (0.0007) [2023-12-27 08:47:36,615][20381] Updated weights for policy 1, policy_version 455324 (0.0006) [2023-12-27 08:47:36,679][20381] Updated weights for policy 1, policy_version 455334 (0.0007) [2023-12-27 08:47:36,741][20381] Updated weights for policy 1, policy_version 455344 (0.0008) [2023-12-27 08:47:37,085][20347] Updated weights for policy 0, policy_version 455778 (0.0008) [2023-12-27 08:47:37,144][20347] Updated weights for policy 0, policy_version 455788 (0.0007) [2023-12-27 08:47:37,205][20347] Updated weights for policy 0, policy_version 455798 (0.0007) [2023-12-27 08:47:37,265][20347] Updated weights for policy 0, policy_version 455808 (0.0007) [2023-12-27 08:47:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 233291776. Throughput: 0: 10642.5, 1: 10010.8. Samples: 233261028. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:37,363][19170] Avg episode reward: [(0, '279.610'), (1, '855.570')] [2023-12-27 08:47:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000455808_116703232.pth... [2023-12-27 08:47:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000454592_116391936.pth [2023-12-27 08:47:37,416][20381] Updated weights for policy 1, policy_version 455354 (0.0008) [2023-12-27 08:47:37,474][20381] Updated weights for policy 1, policy_version 455364 (0.0007) [2023-12-27 08:47:37,538][20381] Updated weights for policy 1, policy_version 455374 (0.0008) [2023-12-27 08:47:37,551][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000455376_116596736.pth... [2023-12-27 08:47:37,556][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000454192_116293632.pth [2023-12-27 08:47:37,966][20347] Updated weights for policy 0, policy_version 455818 (0.0007) [2023-12-27 08:47:38,013][20347] Updated weights for policy 0, policy_version 455828 (0.0007) [2023-12-27 08:47:38,066][20347] Updated weights for policy 0, policy_version 455838 (0.0007) [2023-12-27 08:47:38,250][20381] Updated weights for policy 1, policy_version 455384 (0.0010) [2023-12-27 08:47:38,305][20381] Updated weights for policy 1, policy_version 455394 (0.0010) [2023-12-27 08:47:38,368][20381] Updated weights for policy 1, policy_version 455404 (0.0009) [2023-12-27 08:47:38,806][20347] Updated weights for policy 0, policy_version 455848 (0.0007) [2023-12-27 08:47:38,857][20347] Updated weights for policy 0, policy_version 455858 (0.0007) [2023-12-27 08:47:38,903][20347] Updated weights for policy 0, policy_version 455868 (0.0007) [2023-12-27 08:47:39,071][20381] Updated weights for policy 1, policy_version 455414 (0.0010) [2023-12-27 08:47:39,124][20381] Updated weights for policy 1, policy_version 455424 (0.0009) [2023-12-27 08:47:39,174][20381] Updated weights for policy 1, policy_version 455434 (0.0010) [2023-12-27 08:47:39,628][20347] Updated weights for policy 0, policy_version 455878 (0.0007) [2023-12-27 08:47:39,683][20347] Updated weights for policy 0, policy_version 455888 (0.0007) [2023-12-27 08:47:39,741][20347] Updated weights for policy 0, policy_version 455898 (0.0007) [2023-12-27 08:47:39,890][20381] Updated weights for policy 1, policy_version 455444 (0.0010) [2023-12-27 08:47:39,952][20381] Updated weights for policy 1, policy_version 455454 (0.0008) [2023-12-27 08:47:40,016][20381] Updated weights for policy 1, policy_version 455464 (0.0008) [2023-12-27 08:47:40,464][20347] Updated weights for policy 0, policy_version 455908 (0.0008) [2023-12-27 08:47:40,530][20347] Updated weights for policy 0, policy_version 455918 (0.0007) [2023-12-27 08:47:40,591][20347] Updated weights for policy 0, policy_version 455928 (0.0007) [2023-12-27 08:47:40,720][20381] Updated weights for policy 1, policy_version 455474 (0.0008) [2023-12-27 08:47:40,782][20381] Updated weights for policy 1, policy_version 455484 (0.0011) [2023-12-27 08:47:40,846][20381] Updated weights for policy 1, policy_version 455494 (0.0010) [2023-12-27 08:47:40,853][20292] KL-divergence is very high: 114.9458 [2023-12-27 08:47:40,911][20381] Updated weights for policy 1, policy_version 455504 (0.0011) [2023-12-27 08:47:41,320][20347] Updated weights for policy 0, policy_version 455938 (0.0008) [2023-12-27 08:47:41,393][20347] Updated weights for policy 0, policy_version 455948 (0.0010) [2023-12-27 08:47:41,451][20347] Updated weights for policy 0, policy_version 455958 (0.0011) [2023-12-27 08:47:41,508][20347] Updated weights for policy 0, policy_version 455968 (0.0008) [2023-12-27 08:47:41,611][20381] Updated weights for policy 1, policy_version 455514 (0.0011) [2023-12-27 08:47:41,678][20381] Updated weights for policy 1, policy_version 455524 (0.0011) [2023-12-27 08:47:41,740][20381] Updated weights for policy 1, policy_version 455534 (0.0011) [2023-12-27 08:47:42,228][20347] Updated weights for policy 0, policy_version 455978 (0.0006) [2023-12-27 08:47:42,289][20347] Updated weights for policy 0, policy_version 455988 (0.0008) [2023-12-27 08:47:42,347][20347] Updated weights for policy 0, policy_version 455998 (0.0010) [2023-12-27 08:47:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 233390080. Throughput: 0: 10536.5, 1: 10000.4. Samples: 233380992. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:42,363][19170] Avg episode reward: [(0, '313.240'), (1, '817.310')] [2023-12-27 08:47:42,403][20381] Updated weights for policy 1, policy_version 455544 (0.0006) [2023-12-27 08:47:42,467][20381] Updated weights for policy 1, policy_version 455554 (0.0006) [2023-12-27 08:47:42,532][20381] Updated weights for policy 1, policy_version 455564 (0.0006) [2023-12-27 08:47:43,010][20347] Updated weights for policy 0, policy_version 456008 (0.0010) [2023-12-27 08:47:43,068][20347] Updated weights for policy 0, policy_version 456018 (0.0010) [2023-12-27 08:47:43,076][20381] Updated weights for policy 1, policy_version 455574 (0.0009) [2023-12-27 08:47:43,113][20347] Updated weights for policy 0, policy_version 456028 (0.0010) [2023-12-27 08:47:43,126][20381] Updated weights for policy 1, policy_version 455584 (0.0011) [2023-12-27 08:47:43,173][20381] Updated weights for policy 1, policy_version 455594 (0.0010) [2023-12-27 08:47:43,776][20347] Updated weights for policy 0, policy_version 456038 (0.0010) [2023-12-27 08:47:43,821][20347] Updated weights for policy 0, policy_version 456048 (0.0010) [2023-12-27 08:47:43,842][20381] Updated weights for policy 1, policy_version 455604 (0.0010) [2023-12-27 08:47:43,873][20347] Updated weights for policy 0, policy_version 456058 (0.0010) [2023-12-27 08:47:43,894][20381] Updated weights for policy 1, policy_version 455614 (0.0007) [2023-12-27 08:47:43,944][20381] Updated weights for policy 1, policy_version 455624 (0.0005) [2023-12-27 08:47:44,546][20381] Updated weights for policy 1, policy_version 455634 (0.0005) [2023-12-27 08:47:44,572][20347] Updated weights for policy 0, policy_version 456068 (0.0008) [2023-12-27 08:47:44,608][20381] Updated weights for policy 1, policy_version 455644 (0.0005) [2023-12-27 08:47:44,630][20347] Updated weights for policy 0, policy_version 456078 (0.0005) [2023-12-27 08:47:44,668][20381] Updated weights for policy 1, policy_version 455654 (0.0010) [2023-12-27 08:47:44,681][20347] Updated weights for policy 0, policy_version 456088 (0.0005) [2023-12-27 08:47:44,722][20381] Updated weights for policy 1, policy_version 455664 (0.0010) [2023-12-27 08:47:45,332][20347] Updated weights for policy 0, policy_version 456098 (0.0005) [2023-12-27 08:47:45,381][20381] Updated weights for policy 1, policy_version 455674 (0.0011) [2023-12-27 08:47:45,383][20347] Updated weights for policy 0, policy_version 456108 (0.0005) [2023-12-27 08:47:45,436][20381] Updated weights for policy 1, policy_version 455684 (0.0010) [2023-12-27 08:47:45,437][20347] Updated weights for policy 0, policy_version 456118 (0.0008) [2023-12-27 08:47:45,489][20347] Updated weights for policy 0, policy_version 456128 (0.0006) [2023-12-27 08:47:45,495][20381] Updated weights for policy 1, policy_version 455694 (0.0011) [2023-12-27 08:47:46,108][20381] Updated weights for policy 1, policy_version 455704 (0.0010) [2023-12-27 08:47:46,157][20381] Updated weights for policy 1, policy_version 455714 (0.0010) [2023-12-27 08:47:46,186][20347] Updated weights for policy 0, policy_version 456138 (0.0005) [2023-12-27 08:47:46,209][20381] Updated weights for policy 1, policy_version 455724 (0.0010) [2023-12-27 08:47:46,235][20347] Updated weights for policy 0, policy_version 456148 (0.0005) [2023-12-27 08:47:46,283][20347] Updated weights for policy 0, policy_version 456158 (0.0007) [2023-12-27 08:47:46,841][20381] Updated weights for policy 1, policy_version 455734 (0.0007) [2023-12-27 08:47:46,896][20381] Updated weights for policy 1, policy_version 455744 (0.0005) [2023-12-27 08:47:46,942][20381] Updated weights for policy 1, policy_version 455754 (0.0005) [2023-12-27 08:47:46,943][20347] Updated weights for policy 0, policy_version 456168 (0.0006) [2023-12-27 08:47:46,990][20347] Updated weights for policy 0, policy_version 456178 (0.0008) [2023-12-27 08:47:47,044][20347] Updated weights for policy 0, policy_version 456188 (0.0007) [2023-12-27 08:47:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 233496576. Throughput: 0: 10391.7, 1: 10173.1. Samples: 233506916. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:47,363][19170] Avg episode reward: [(0, '344.150'), (1, '779.250')] [2023-12-27 08:47:47,478][20381] Updated weights for policy 1, policy_version 455764 (0.0005) [2023-12-27 08:47:47,530][20381] Updated weights for policy 1, policy_version 455774 (0.0008) [2023-12-27 08:47:47,583][20381] Updated weights for policy 1, policy_version 455784 (0.0006) [2023-12-27 08:47:47,801][20347] Updated weights for policy 0, policy_version 456198 (0.0008) [2023-12-27 08:47:47,858][20347] Updated weights for policy 0, policy_version 456208 (0.0007) [2023-12-27 08:47:47,912][20347] Updated weights for policy 0, policy_version 456218 (0.0007) [2023-12-27 08:47:48,208][20381] Updated weights for policy 1, policy_version 455794 (0.0006) [2023-12-27 08:47:48,253][20381] Updated weights for policy 1, policy_version 455804 (0.0010) [2023-12-27 08:47:48,314][20381] Updated weights for policy 1, policy_version 455814 (0.0010) [2023-12-27 08:47:48,379][20381] Updated weights for policy 1, policy_version 455824 (0.0011) [2023-12-27 08:47:48,579][20347] Updated weights for policy 0, policy_version 456228 (0.0008) [2023-12-27 08:47:48,638][20347] Updated weights for policy 0, policy_version 456238 (0.0008) [2023-12-27 08:47:48,699][20347] Updated weights for policy 0, policy_version 456248 (0.0007) [2023-12-27 08:47:49,028][20381] Updated weights for policy 1, policy_version 455834 (0.0007) [2023-12-27 08:47:49,087][20381] Updated weights for policy 1, policy_version 455844 (0.0011) [2023-12-27 08:47:49,150][20381] Updated weights for policy 1, policy_version 455854 (0.0011) [2023-12-27 08:47:49,416][20347] Updated weights for policy 0, policy_version 456258 (0.0008) [2023-12-27 08:47:49,468][20347] Updated weights for policy 0, policy_version 456268 (0.0008) [2023-12-27 08:47:49,519][20347] Updated weights for policy 0, policy_version 456278 (0.0008) [2023-12-27 08:47:49,571][20347] Updated weights for policy 0, policy_version 456288 (0.0008) [2023-12-27 08:47:49,805][20381] Updated weights for policy 1, policy_version 455864 (0.0009) [2023-12-27 08:47:49,867][20381] Updated weights for policy 1, policy_version 455874 (0.0007) [2023-12-27 08:47:49,930][20381] Updated weights for policy 1, policy_version 455884 (0.0008) [2023-12-27 08:47:50,252][20347] Updated weights for policy 0, policy_version 456298 (0.0008) [2023-12-27 08:47:50,311][20347] Updated weights for policy 0, policy_version 456308 (0.0008) [2023-12-27 08:47:50,369][20347] Updated weights for policy 0, policy_version 456318 (0.0009) [2023-12-27 08:47:50,582][20381] Updated weights for policy 1, policy_version 455894 (0.0008) [2023-12-27 08:47:50,639][20381] Updated weights for policy 1, policy_version 455904 (0.0007) [2023-12-27 08:47:50,695][20381] Updated weights for policy 1, policy_version 455914 (0.0009) [2023-12-27 08:47:51,109][20347] Updated weights for policy 0, policy_version 456328 (0.0008) [2023-12-27 08:47:51,171][20347] Updated weights for policy 0, policy_version 456338 (0.0009) [2023-12-27 08:47:51,230][20347] Updated weights for policy 0, policy_version 456348 (0.0008) [2023-12-27 08:47:51,392][20381] Updated weights for policy 1, policy_version 455924 (0.0008) [2023-12-27 08:47:51,452][20381] Updated weights for policy 1, policy_version 455934 (0.0006) [2023-12-27 08:47:51,514][20381] Updated weights for policy 1, policy_version 455944 (0.0006) [2023-12-27 08:47:51,944][20347] Updated weights for policy 0, policy_version 456358 (0.0007) [2023-12-27 08:47:52,001][20347] Updated weights for policy 0, policy_version 456368 (0.0009) [2023-12-27 08:47:52,054][20347] Updated weights for policy 0, policy_version 456378 (0.0008) [2023-12-27 08:47:52,151][20381] Updated weights for policy 1, policy_version 455954 (0.0009) [2023-12-27 08:47:52,204][20381] Updated weights for policy 1, policy_version 455964 (0.0011) [2023-12-27 08:47:52,262][20381] Updated weights for policy 1, policy_version 455974 (0.0010) [2023-12-27 08:47:52,319][20381] Updated weights for policy 1, policy_version 455984 (0.0010) [2023-12-27 08:47:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 233603072. Throughput: 0: 10336.7, 1: 10210.1. Samples: 233568468. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:52,363][19170] Avg episode reward: [(0, '342.350'), (1, '806.550')] [2023-12-27 08:47:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000455984_116752384.pth... [2023-12-27 08:47:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000456384_116850688.pth... [2023-12-27 08:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000454768_116441088.pth [2023-12-27 08:47:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000455200_116547584.pth [2023-12-27 08:47:52,786][20347] Updated weights for policy 0, policy_version 456388 (0.0009) [2023-12-27 08:47:52,835][20347] Updated weights for policy 0, policy_version 456398 (0.0007) [2023-12-27 08:47:52,888][20347] Updated weights for policy 0, policy_version 456408 (0.0008) [2023-12-27 08:47:52,983][20381] Updated weights for policy 1, policy_version 455994 (0.0006) [2023-12-27 08:47:53,037][20381] Updated weights for policy 1, policy_version 456004 (0.0005) [2023-12-27 08:47:53,090][20381] Updated weights for policy 1, policy_version 456014 (0.0006) [2023-12-27 08:47:53,582][20347] Updated weights for policy 0, policy_version 456418 (0.0007) [2023-12-27 08:47:53,630][20347] Updated weights for policy 0, policy_version 456428 (0.0006) [2023-12-27 08:47:53,679][20347] Updated weights for policy 0, policy_version 456438 (0.0007) [2023-12-27 08:47:53,733][20347] Updated weights for policy 0, policy_version 456448 (0.0005) [2023-12-27 08:47:53,776][20381] Updated weights for policy 1, policy_version 456024 (0.0007) [2023-12-27 08:47:53,826][20381] Updated weights for policy 1, policy_version 456034 (0.0008) [2023-12-27 08:47:53,874][20381] Updated weights for policy 1, policy_version 456044 (0.0007) [2023-12-27 08:47:54,326][20347] Updated weights for policy 0, policy_version 456458 (0.0008) [2023-12-27 08:47:54,388][20347] Updated weights for policy 0, policy_version 456468 (0.0007) [2023-12-27 08:47:54,443][20347] Updated weights for policy 0, policy_version 456478 (0.0007) [2023-12-27 08:47:54,560][20381] Updated weights for policy 1, policy_version 456054 (0.0008) [2023-12-27 08:47:54,624][20381] Updated weights for policy 1, policy_version 456064 (0.0008) [2023-12-27 08:47:54,675][20381] Updated weights for policy 1, policy_version 456074 (0.0007) [2023-12-27 08:47:55,108][20347] Updated weights for policy 0, policy_version 456488 (0.0007) [2023-12-27 08:47:55,162][20347] Updated weights for policy 0, policy_version 456498 (0.0008) [2023-12-27 08:47:55,211][20347] Updated weights for policy 0, policy_version 456508 (0.0009) [2023-12-27 08:47:55,318][20381] Updated weights for policy 1, policy_version 456084 (0.0006) [2023-12-27 08:47:55,373][20381] Updated weights for policy 1, policy_version 456094 (0.0005) [2023-12-27 08:47:55,378][20292] KL-divergence is very high: 135.9122 [2023-12-27 08:47:55,384][20292] KL-divergence is very high: 136.2787 [2023-12-27 08:47:55,427][20381] Updated weights for policy 1, policy_version 456104 (0.0005) [2023-12-27 08:47:55,977][20381] Updated weights for policy 1, policy_version 456114 (0.0005) [2023-12-27 08:47:55,980][20347] Updated weights for policy 0, policy_version 456518 (0.0008) [2023-12-27 08:47:56,026][20381] Updated weights for policy 1, policy_version 456124 (0.0005) [2023-12-27 08:47:56,036][20347] Updated weights for policy 0, policy_version 456528 (0.0006) [2023-12-27 08:47:56,082][20381] Updated weights for policy 1, policy_version 456134 (0.0007) [2023-12-27 08:47:56,088][20347] Updated weights for policy 0, policy_version 456538 (0.0006) [2023-12-27 08:47:56,129][20381] Updated weights for policy 1, policy_version 456144 (0.0005) [2023-12-27 08:47:56,681][20381] Updated weights for policy 1, policy_version 456154 (0.0005) [2023-12-27 08:47:56,733][20381] Updated weights for policy 1, policy_version 456164 (0.0005) [2023-12-27 08:47:56,770][20347] Updated weights for policy 0, policy_version 456548 (0.0007) [2023-12-27 08:47:56,781][20381] Updated weights for policy 1, policy_version 456174 (0.0005) [2023-12-27 08:47:56,831][20347] Updated weights for policy 0, policy_version 456558 (0.0008) [2023-12-27 08:47:56,893][20347] Updated weights for policy 0, policy_version 456568 (0.0007) [2023-12-27 08:47:57,355][20381] Updated weights for policy 1, policy_version 456184 (0.0005) [2023-12-27 08:47:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 233701376. Throughput: 0: 10248.1, 1: 10311.1. Samples: 233692140. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:47:57,363][19170] Avg episode reward: [(0, '306.940'), (1, '720.510')] [2023-12-27 08:47:57,401][20381] Updated weights for policy 1, policy_version 456194 (0.0005) [2023-12-27 08:47:57,452][20381] Updated weights for policy 1, policy_version 456204 (0.0005) [2023-12-27 08:47:57,542][20347] Updated weights for policy 0, policy_version 456578 (0.0008) [2023-12-27 08:47:57,590][20347] Updated weights for policy 0, policy_version 456588 (0.0006) [2023-12-27 08:47:57,637][20347] Updated weights for policy 0, policy_version 456598 (0.0007) [2023-12-27 08:47:57,685][20347] Updated weights for policy 0, policy_version 456608 (0.0005) [2023-12-27 08:47:58,090][20381] Updated weights for policy 1, policy_version 456214 (0.0006) [2023-12-27 08:47:58,155][20381] Updated weights for policy 1, policy_version 456224 (0.0005) [2023-12-27 08:47:58,221][20381] Updated weights for policy 1, policy_version 456234 (0.0008) [2023-12-27 08:47:58,343][20347] Updated weights for policy 0, policy_version 456618 (0.0008) [2023-12-27 08:47:58,416][20347] Updated weights for policy 0, policy_version 456628 (0.0008) [2023-12-27 08:47:58,475][20347] Updated weights for policy 0, policy_version 456638 (0.0007) [2023-12-27 08:47:58,921][20381] Updated weights for policy 1, policy_version 456244 (0.0007) [2023-12-27 08:47:58,991][20381] Updated weights for policy 1, policy_version 456254 (0.0007) [2023-12-27 08:47:59,047][20381] Updated weights for policy 1, policy_version 456264 (0.0007) [2023-12-27 08:47:59,230][20347] Updated weights for policy 0, policy_version 456648 (0.0008) [2023-12-27 08:47:59,298][20347] Updated weights for policy 0, policy_version 456658 (0.0007) [2023-12-27 08:47:59,364][20347] Updated weights for policy 0, policy_version 456668 (0.0008) [2023-12-27 08:47:59,808][20381] Updated weights for policy 1, policy_version 456274 (0.0008) [2023-12-27 08:47:59,866][20381] Updated weights for policy 1, policy_version 456284 (0.0006) [2023-12-27 08:47:59,929][20381] Updated weights for policy 1, policy_version 456294 (0.0007) [2023-12-27 08:47:59,982][20381] Updated weights for policy 1, policy_version 456304 (0.0006) [2023-12-27 08:48:00,054][20347] Updated weights for policy 0, policy_version 456678 (0.0006) [2023-12-27 08:48:00,114][20347] Updated weights for policy 0, policy_version 456688 (0.0007) [2023-12-27 08:48:00,168][20347] Updated weights for policy 0, policy_version 456698 (0.0006) [2023-12-27 08:48:00,627][20381] Updated weights for policy 1, policy_version 456314 (0.0005) [2023-12-27 08:48:00,673][20381] Updated weights for policy 1, policy_version 456324 (0.0006) [2023-12-27 08:48:00,731][20381] Updated weights for policy 1, policy_version 456334 (0.0006) [2023-12-27 08:48:00,748][20347] Updated weights for policy 0, policy_version 456708 (0.0005) [2023-12-27 08:48:00,804][20347] Updated weights for policy 0, policy_version 456718 (0.0005) [2023-12-27 08:48:00,867][20347] Updated weights for policy 0, policy_version 456728 (0.0005) [2023-12-27 08:48:01,339][20381] Updated weights for policy 1, policy_version 456344 (0.0006) [2023-12-27 08:48:01,407][20381] Updated weights for policy 1, policy_version 456354 (0.0008) [2023-12-27 08:48:01,455][20347] Updated weights for policy 0, policy_version 456738 (0.0006) [2023-12-27 08:48:01,463][20381] Updated weights for policy 1, policy_version 456364 (0.0005) [2023-12-27 08:48:01,510][20347] Updated weights for policy 0, policy_version 456748 (0.0010) [2023-12-27 08:48:01,571][20347] Updated weights for policy 0, policy_version 456758 (0.0011) [2023-12-27 08:48:01,632][20347] Updated weights for policy 0, policy_version 456768 (0.0011) [2023-12-27 08:48:02,085][20381] Updated weights for policy 1, policy_version 456374 (0.0008) [2023-12-27 08:48:02,143][20381] Updated weights for policy 1, policy_version 456384 (0.0007) [2023-12-27 08:48:02,203][20381] Updated weights for policy 1, policy_version 456394 (0.0007) [2023-12-27 08:48:02,304][20347] Updated weights for policy 0, policy_version 456778 (0.0008) [2023-12-27 08:48:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 233807872. Throughput: 0: 10238.8, 1: 10395.0. Samples: 233818688. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:48:02,363][19170] Avg episode reward: [(0, '296.610'), (1, '573.510')] [2023-12-27 08:48:02,366][20347] Updated weights for policy 0, policy_version 456788 (0.0008) [2023-12-27 08:48:02,423][20347] Updated weights for policy 0, policy_version 456798 (0.0008) [2023-12-27 08:48:02,883][20381] Updated weights for policy 1, policy_version 456404 (0.0006) [2023-12-27 08:48:02,941][20381] Updated weights for policy 1, policy_version 456414 (0.0007) [2023-12-27 08:48:02,992][20381] Updated weights for policy 1, policy_version 456424 (0.0007) [2023-12-27 08:48:03,089][20347] Updated weights for policy 0, policy_version 456808 (0.0008) [2023-12-27 08:48:03,151][20347] Updated weights for policy 0, policy_version 456818 (0.0008) [2023-12-27 08:48:03,210][20347] Updated weights for policy 0, policy_version 456828 (0.0007) [2023-12-27 08:48:03,556][20381] Updated weights for policy 1, policy_version 456434 (0.0006) [2023-12-27 08:48:03,606][20381] Updated weights for policy 1, policy_version 456444 (0.0005) [2023-12-27 08:48:03,665][20381] Updated weights for policy 1, policy_version 456454 (0.0006) [2023-12-27 08:48:03,712][20381] Updated weights for policy 1, policy_version 456464 (0.0008) [2023-12-27 08:48:03,867][20347] Updated weights for policy 0, policy_version 456838 (0.0008) [2023-12-27 08:48:03,926][20347] Updated weights for policy 0, policy_version 456848 (0.0008) [2023-12-27 08:48:03,984][20347] Updated weights for policy 0, policy_version 456858 (0.0010) [2023-12-27 08:48:04,310][20381] Updated weights for policy 1, policy_version 456474 (0.0005) [2023-12-27 08:48:04,376][20381] Updated weights for policy 1, policy_version 456484 (0.0009) [2023-12-27 08:48:04,437][20381] Updated weights for policy 1, policy_version 456494 (0.0011) [2023-12-27 08:48:04,637][20347] Updated weights for policy 0, policy_version 456868 (0.0009) [2023-12-27 08:48:04,698][20347] Updated weights for policy 0, policy_version 456878 (0.0007) [2023-12-27 08:48:04,751][20347] Updated weights for policy 0, policy_version 456888 (0.0006) [2023-12-27 08:48:05,082][20381] Updated weights for policy 1, policy_version 456504 (0.0010) [2023-12-27 08:48:05,130][20381] Updated weights for policy 1, policy_version 456514 (0.0010) [2023-12-27 08:48:05,181][20381] Updated weights for policy 1, policy_version 456524 (0.0010) [2023-12-27 08:48:05,357][20347] Updated weights for policy 0, policy_version 456898 (0.0010) [2023-12-27 08:48:05,409][20347] Updated weights for policy 0, policy_version 456908 (0.0010) [2023-12-27 08:48:05,463][20347] Updated weights for policy 0, policy_version 456918 (0.0010) [2023-12-27 08:48:05,520][20347] Updated weights for policy 0, policy_version 456928 (0.0010) [2023-12-27 08:48:05,901][20381] Updated weights for policy 1, policy_version 456534 (0.0008) [2023-12-27 08:48:05,957][20381] Updated weights for policy 1, policy_version 456544 (0.0005) [2023-12-27 08:48:06,011][20381] Updated weights for policy 1, policy_version 456554 (0.0005) [2023-12-27 08:48:06,130][20347] Updated weights for policy 0, policy_version 456938 (0.0011) [2023-12-27 08:48:06,189][20347] Updated weights for policy 0, policy_version 456948 (0.0010) [2023-12-27 08:48:06,252][20347] Updated weights for policy 0, policy_version 456958 (0.0011) [2023-12-27 08:48:06,543][20381] Updated weights for policy 1, policy_version 456564 (0.0005) [2023-12-27 08:48:06,600][20381] Updated weights for policy 1, policy_version 456574 (0.0005) [2023-12-27 08:48:06,657][20381] Updated weights for policy 1, policy_version 456584 (0.0006) [2023-12-27 08:48:06,939][20347] Updated weights for policy 0, policy_version 456968 (0.0009) [2023-12-27 08:48:06,989][20347] Updated weights for policy 0, policy_version 456978 (0.0009) [2023-12-27 08:48:07,042][20347] Updated weights for policy 0, policy_version 456988 (0.0007) [2023-12-27 08:48:07,174][20381] Updated weights for policy 1, policy_version 456594 (0.0005) [2023-12-27 08:48:07,223][20381] Updated weights for policy 1, policy_version 456604 (0.0007) [2023-12-27 08:48:07,275][20381] Updated weights for policy 1, policy_version 456614 (0.0005) [2023-12-27 08:48:07,333][20381] Updated weights for policy 1, policy_version 456624 (0.0005) [2023-12-27 08:48:07,362][19170] Fps is (10 sec: 22118.1, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 233922560. Throughput: 0: 10233.5, 1: 10487.6. Samples: 233883440. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:48:07,363][19170] Avg episode reward: [(0, '281.670'), (1, '529.620')] [2023-12-27 08:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000456624_116916224.pth... [2023-12-27 08:48:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000456992_117006336.pth... [2023-12-27 08:48:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000455376_116596736.pth [2023-12-27 08:48:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000455808_116703232.pth [2023-12-27 08:48:07,705][20347] Updated weights for policy 0, policy_version 456998 (0.0008) [2023-12-27 08:48:07,770][20347] Updated weights for policy 0, policy_version 457008 (0.0006) [2023-12-27 08:48:07,834][20347] Updated weights for policy 0, policy_version 457018 (0.0008) [2023-12-27 08:48:07,942][20381] Updated weights for policy 1, policy_version 456634 (0.0007) [2023-12-27 08:48:07,997][20381] Updated weights for policy 1, policy_version 456644 (0.0006) [2023-12-27 08:48:08,046][20381] Updated weights for policy 1, policy_version 456654 (0.0007) [2023-12-27 08:48:08,494][20347] Updated weights for policy 0, policy_version 457028 (0.0007) [2023-12-27 08:48:08,540][20347] Updated weights for policy 0, policy_version 457038 (0.0005) [2023-12-27 08:48:08,590][20347] Updated weights for policy 0, policy_version 457048 (0.0009) [2023-12-27 08:48:08,752][20381] Updated weights for policy 1, policy_version 456664 (0.0006) [2023-12-27 08:48:08,816][20381] Updated weights for policy 1, policy_version 456674 (0.0005) [2023-12-27 08:48:08,871][20381] Updated weights for policy 1, policy_version 456684 (0.0007) [2023-12-27 08:48:09,358][20347] Updated weights for policy 0, policy_version 457058 (0.0009) [2023-12-27 08:48:09,413][20347] Updated weights for policy 0, policy_version 457068 (0.0008) [2023-12-27 08:48:09,451][20381] Updated weights for policy 1, policy_version 456694 (0.0006) [2023-12-27 08:48:09,471][20347] Updated weights for policy 0, policy_version 457078 (0.0008) [2023-12-27 08:48:09,509][20381] Updated weights for policy 1, policy_version 456704 (0.0005) [2023-12-27 08:48:09,521][20347] Updated weights for policy 0, policy_version 457088 (0.0008) [2023-12-27 08:48:09,557][20381] Updated weights for policy 1, policy_version 456714 (0.0006) [2023-12-27 08:48:10,227][20381] Updated weights for policy 1, policy_version 456724 (0.0006) [2023-12-27 08:48:10,277][20347] Updated weights for policy 0, policy_version 457098 (0.0007) [2023-12-27 08:48:10,293][20381] Updated weights for policy 1, policy_version 456734 (0.0008) [2023-12-27 08:48:10,342][20347] Updated weights for policy 0, policy_version 457108 (0.0008) [2023-12-27 08:48:10,352][20381] Updated weights for policy 1, policy_version 456744 (0.0005) [2023-12-27 08:48:10,404][20347] Updated weights for policy 0, policy_version 457118 (0.0008) [2023-12-27 08:48:10,971][20381] Updated weights for policy 1, policy_version 456754 (0.0007) [2023-12-27 08:48:10,999][20347] Updated weights for policy 0, policy_version 457128 (0.0006) [2023-12-27 08:48:11,024][20381] Updated weights for policy 1, policy_version 456764 (0.0008) [2023-12-27 08:48:11,061][20347] Updated weights for policy 0, policy_version 457138 (0.0006) [2023-12-27 08:48:11,090][20381] Updated weights for policy 1, policy_version 456774 (0.0007) [2023-12-27 08:48:11,127][20347] Updated weights for policy 0, policy_version 457148 (0.0007) [2023-12-27 08:48:11,163][20381] Updated weights for policy 1, policy_version 456784 (0.0008) [2023-12-27 08:48:11,801][20347] Updated weights for policy 0, policy_version 457158 (0.0007) [2023-12-27 08:48:11,864][20347] Updated weights for policy 0, policy_version 457168 (0.0006) [2023-12-27 08:48:11,881][20381] Updated weights for policy 1, policy_version 456794 (0.0008) [2023-12-27 08:48:11,925][20347] Updated weights for policy 0, policy_version 457178 (0.0007) [2023-12-27 08:48:11,948][20381] Updated weights for policy 1, policy_version 456804 (0.0006) [2023-12-27 08:48:12,012][20381] Updated weights for policy 1, policy_version 456814 (0.0007) [2023-12-27 08:48:12,363][19170] Fps is (10 sec: 21297.8, 60 sec: 20752.8, 300 sec: 20799.3). Total num frames: 234020864. Throughput: 0: 10234.0, 1: 10671.5. Samples: 234011448. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:48:12,364][19170] Avg episode reward: [(0, '297.610'), (1, '207.550')] [2023-12-27 08:48:12,605][20347] Updated weights for policy 0, policy_version 457188 (0.0006) [2023-12-27 08:48:12,664][20347] Updated weights for policy 0, policy_version 457198 (0.0007) [2023-12-27 08:48:12,689][20381] Updated weights for policy 1, policy_version 456824 (0.0007) [2023-12-27 08:48:12,722][20347] Updated weights for policy 0, policy_version 457208 (0.0006) [2023-12-27 08:48:12,749][20381] Updated weights for policy 1, policy_version 456834 (0.0007) [2023-12-27 08:48:12,808][20381] Updated weights for policy 1, policy_version 456844 (0.0008) [2023-12-27 08:48:13,332][20347] Updated weights for policy 0, policy_version 457218 (0.0009) [2023-12-27 08:48:13,386][20347] Updated weights for policy 0, policy_version 457228 (0.0008) [2023-12-27 08:48:13,443][20347] Updated weights for policy 0, policy_version 457238 (0.0009) [2023-12-27 08:48:13,490][20381] Updated weights for policy 1, policy_version 456854 (0.0009) [2023-12-27 08:48:13,499][20347] Updated weights for policy 0, policy_version 457248 (0.0006) [2023-12-27 08:48:13,545][20381] Updated weights for policy 1, policy_version 456864 (0.0007) [2023-12-27 08:48:13,599][20381] Updated weights for policy 1, policy_version 456874 (0.0009) [2023-12-27 08:48:14,228][20347] Updated weights for policy 0, policy_version 457258 (0.0008) [2023-12-27 08:48:14,231][20381] Updated weights for policy 1, policy_version 456884 (0.0007) [2023-12-27 08:48:14,287][20347] Updated weights for policy 0, policy_version 457268 (0.0007) [2023-12-27 08:48:14,292][20381] Updated weights for policy 1, policy_version 456894 (0.0007) [2023-12-27 08:48:14,339][20347] Updated weights for policy 0, policy_version 457278 (0.0006) [2023-12-27 08:48:14,348][20381] Updated weights for policy 1, policy_version 456904 (0.0007) [2023-12-27 08:48:14,903][20381] Updated weights for policy 1, policy_version 456914 (0.0005) [2023-12-27 08:48:14,962][20381] Updated weights for policy 1, policy_version 456924 (0.0007) [2023-12-27 08:48:15,019][20381] Updated weights for policy 1, policy_version 456934 (0.0008) [2023-12-27 08:48:15,050][20347] Updated weights for policy 0, policy_version 457288 (0.0006) [2023-12-27 08:48:15,074][20381] Updated weights for policy 1, policy_version 456944 (0.0008) [2023-12-27 08:48:15,111][20347] Updated weights for policy 0, policy_version 457298 (0.0006) [2023-12-27 08:48:15,171][20347] Updated weights for policy 0, policy_version 457308 (0.0007) [2023-12-27 08:48:15,706][20381] Updated weights for policy 1, policy_version 456954 (0.0011) [2023-12-27 08:48:15,758][20381] Updated weights for policy 1, policy_version 456964 (0.0011) [2023-12-27 08:48:15,807][20381] Updated weights for policy 1, policy_version 456974 (0.0011) [2023-12-27 08:48:15,868][20347] Updated weights for policy 0, policy_version 457318 (0.0009) [2023-12-27 08:48:15,927][20347] Updated weights for policy 0, policy_version 457328 (0.0008) [2023-12-27 08:48:15,988][20347] Updated weights for policy 0, policy_version 457338 (0.0008) [2023-12-27 08:48:16,429][20381] Updated weights for policy 1, policy_version 456984 (0.0010) [2023-12-27 08:48:16,487][20381] Updated weights for policy 1, policy_version 456994 (0.0011) [2023-12-27 08:48:16,552][20381] Updated weights for policy 1, policy_version 457004 (0.0011) [2023-12-27 08:48:16,688][20347] Updated weights for policy 0, policy_version 457348 (0.0007) [2023-12-27 08:48:16,746][20347] Updated weights for policy 0, policy_version 457358 (0.0006) [2023-12-27 08:48:16,792][20347] Updated weights for policy 0, policy_version 457368 (0.0007) [2023-12-27 08:48:17,225][20381] Updated weights for policy 1, policy_version 457014 (0.0007) [2023-12-27 08:48:17,273][20381] Updated weights for policy 1, policy_version 457024 (0.0005) [2023-12-27 08:48:17,316][20381] Updated weights for policy 1, policy_version 457034 (0.0005) [2023-12-27 08:48:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 234127360. Throughput: 0: 10139.0, 1: 10660.7. Samples: 234135452. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-12-27 08:48:17,363][19170] Avg episode reward: [(0, '338.220'), (1, '422.490')] [2023-12-27 08:48:17,495][20347] Updated weights for policy 0, policy_version 457378 (0.0008) [2023-12-27 08:48:17,543][20347] Updated weights for policy 0, policy_version 457388 (0.0006) [2023-12-27 08:48:17,594][20347] Updated weights for policy 0, policy_version 457398 (0.0005) [2023-12-27 08:48:17,649][20347] Updated weights for policy 0, policy_version 457408 (0.0005) [2023-12-27 08:48:17,937][20381] Updated weights for policy 1, policy_version 457044 (0.0005) [2023-12-27 08:48:17,992][20381] Updated weights for policy 1, policy_version 457054 (0.0005) [2023-12-27 08:48:18,053][20381] Updated weights for policy 1, policy_version 457064 (0.0005) [2023-12-27 08:48:18,222][20347] Updated weights for policy 0, policy_version 457418 (0.0008) [2023-12-27 08:48:18,272][20347] Updated weights for policy 0, policy_version 457428 (0.0007) [2023-12-27 08:48:18,322][20347] Updated weights for policy 0, policy_version 457438 (0.0007) [2023-12-27 08:48:18,679][20381] Updated weights for policy 1, policy_version 457074 (0.0011) [2023-12-27 08:48:18,741][20381] Updated weights for policy 1, policy_version 457084 (0.0011) [2023-12-27 08:48:18,800][20381] Updated weights for policy 1, policy_version 457094 (0.0009) [2023-12-27 08:48:18,854][20381] Updated weights for policy 1, policy_version 457104 (0.0008) [2023-12-27 08:48:18,982][20347] Updated weights for policy 0, policy_version 457448 (0.0007) [2023-12-27 08:48:19,036][20347] Updated weights for policy 0, policy_version 457458 (0.0007) [2023-12-27 08:48:19,095][20347] Updated weights for policy 0, policy_version 457468 (0.0007) [2023-12-27 08:48:19,563][20381] Updated weights for policy 1, policy_version 457114 (0.0008) [2023-12-27 08:48:19,622][20381] Updated weights for policy 1, policy_version 457124 (0.0007) [2023-12-27 08:48:19,686][20381] Updated weights for policy 1, policy_version 457134 (0.0006) [2023-12-27 08:48:19,789][20347] Updated weights for policy 0, policy_version 457478 (0.0009) [2023-12-27 08:48:19,852][20347] Updated weights for policy 0, policy_version 457488 (0.0008) [2023-12-27 08:48:19,915][20347] Updated weights for policy 0, policy_version 457498 (0.0008) [2023-12-27 08:48:20,368][20381] Updated weights for policy 1, policy_version 457144 (0.0007) [2023-12-27 08:48:20,425][20381] Updated weights for policy 1, policy_version 457154 (0.0008) [2023-12-27 08:48:20,477][20381] Updated weights for policy 1, policy_version 457164 (0.0007) [2023-12-27 08:48:20,606][20347] Updated weights for policy 0, policy_version 457508 (0.0007) [2023-12-27 08:48:20,663][20347] Updated weights for policy 0, policy_version 457518 (0.0009) [2023-12-27 08:48:20,723][20347] Updated weights for policy 0, policy_version 457528 (0.0008) [2023-12-27 08:48:21,086][20381] Updated weights for policy 1, policy_version 457174 (0.0008) [2023-12-27 08:48:21,147][20381] Updated weights for policy 1, policy_version 457184 (0.0008) [2023-12-27 08:48:21,201][20381] Updated weights for policy 1, policy_version 457194 (0.0008) [2023-12-27 08:48:21,371][20347] Updated weights for policy 0, policy_version 457538 (0.0006) [2023-12-27 08:48:21,428][20347] Updated weights for policy 0, policy_version 457548 (0.0006) [2023-12-27 08:48:21,491][20347] Updated weights for policy 0, policy_version 457558 (0.0006) [2023-12-27 08:48:21,548][20347] Updated weights for policy 0, policy_version 457568 (0.0006) [2023-12-27 08:48:21,948][20381] Updated weights for policy 1, policy_version 457204 (0.0007) [2023-12-27 08:48:22,013][20381] Updated weights for policy 1, policy_version 457214 (0.0006) [2023-12-27 08:48:22,079][20381] Updated weights for policy 1, policy_version 457224 (0.0007) [2023-12-27 08:48:22,227][20347] Updated weights for policy 0, policy_version 457578 (0.0008) [2023-12-27 08:48:22,289][20347] Updated weights for policy 0, policy_version 457588 (0.0009) [2023-12-27 08:48:22,345][20347] Updated weights for policy 0, policy_version 457598 (0.0007) [2023-12-27 08:48:22,362][19170] Fps is (10 sec: 21300.6, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 234233856. Throughput: 0: 10148.4, 1: 10684.7. Samples: 234198520. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:22,363][19170] Avg episode reward: [(0, '344.490'), (1, '556.820')] [2023-12-27 08:48:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000457232_117071872.pth... [2023-12-27 08:48:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000457600_117161984.pth... [2023-12-27 08:48:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000455984_116752384.pth [2023-12-27 08:48:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000456384_116850688.pth [2023-12-27 08:48:22,700][20381] Updated weights for policy 1, policy_version 457234 (0.0006) [2023-12-27 08:48:22,750][20381] Updated weights for policy 1, policy_version 457244 (0.0006) [2023-12-27 08:48:22,795][20381] Updated weights for policy 1, policy_version 457254 (0.0008) [2023-12-27 08:48:22,846][20381] Updated weights for policy 1, policy_version 457264 (0.0007) [2023-12-27 08:48:23,059][20347] Updated weights for policy 0, policy_version 457608 (0.0007) [2023-12-27 08:48:23,119][20347] Updated weights for policy 0, policy_version 457618 (0.0008) [2023-12-27 08:48:23,177][20347] Updated weights for policy 0, policy_version 457628 (0.0008) [2023-12-27 08:48:23,573][20381] Updated weights for policy 1, policy_version 457274 (0.0007) [2023-12-27 08:48:23,634][20381] Updated weights for policy 1, policy_version 457284 (0.0005) [2023-12-27 08:48:23,696][20381] Updated weights for policy 1, policy_version 457294 (0.0006) [2023-12-27 08:48:23,843][20347] Updated weights for policy 0, policy_version 457638 (0.0006) [2023-12-27 08:48:23,901][20347] Updated weights for policy 0, policy_version 457648 (0.0008) [2023-12-27 08:48:23,956][20347] Updated weights for policy 0, policy_version 457658 (0.0008) [2023-12-27 08:48:24,377][20381] Updated weights for policy 1, policy_version 457304 (0.0007) [2023-12-27 08:48:24,431][20381] Updated weights for policy 1, policy_version 457314 (0.0007) [2023-12-27 08:48:24,488][20381] Updated weights for policy 1, policy_version 457324 (0.0008) [2023-12-27 08:48:24,528][20347] Updated weights for policy 0, policy_version 457668 (0.0008) [2023-12-27 08:48:24,576][20347] Updated weights for policy 0, policy_version 457678 (0.0008) [2023-12-27 08:48:24,635][20347] Updated weights for policy 0, policy_version 457688 (0.0008) [2023-12-27 08:48:25,171][20381] Updated weights for policy 1, policy_version 457334 (0.0007) [2023-12-27 08:48:25,198][20347] Updated weights for policy 0, policy_version 457698 (0.0008) [2023-12-27 08:48:25,224][20381] Updated weights for policy 1, policy_version 457344 (0.0005) [2023-12-27 08:48:25,260][20347] Updated weights for policy 0, policy_version 457708 (0.0007) [2023-12-27 08:48:25,283][20381] Updated weights for policy 1, policy_version 457354 (0.0005) [2023-12-27 08:48:25,311][20347] Updated weights for policy 0, policy_version 457718 (0.0008) [2023-12-27 08:48:25,359][20347] Updated weights for policy 0, policy_version 457728 (0.0007) [2023-12-27 08:48:25,922][20381] Updated weights for policy 1, policy_version 457364 (0.0007) [2023-12-27 08:48:25,972][20381] Updated weights for policy 1, policy_version 457374 (0.0010) [2023-12-27 08:48:26,003][20347] Updated weights for policy 0, policy_version 457738 (0.0005) [2023-12-27 08:48:26,024][20381] Updated weights for policy 1, policy_version 457384 (0.0010) [2023-12-27 08:48:26,053][20347] Updated weights for policy 0, policy_version 457748 (0.0006) [2023-12-27 08:48:26,103][20347] Updated weights for policy 0, policy_version 457758 (0.0007) [2023-12-27 08:48:26,683][20381] Updated weights for policy 1, policy_version 457394 (0.0009) [2023-12-27 08:48:26,733][20381] Updated weights for policy 1, policy_version 457404 (0.0010) [2023-12-27 08:48:26,786][20347] Updated weights for policy 0, policy_version 457768 (0.0005) [2023-12-27 08:48:26,787][20381] Updated weights for policy 1, policy_version 457414 (0.0010) [2023-12-27 08:48:26,838][20381] Updated weights for policy 1, policy_version 457424 (0.0010) [2023-12-27 08:48:26,842][20347] Updated weights for policy 0, policy_version 457778 (0.0005) [2023-12-27 08:48:26,904][20347] Updated weights for policy 0, policy_version 457788 (0.0006) [2023-12-27 08:48:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 234332160. Throughput: 0: 10256.4, 1: 10689.8. Samples: 234323568. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:27,363][19170] Avg episode reward: [(0, '300.460'), (1, '545.350')] [2023-12-27 08:48:27,454][20347] Updated weights for policy 0, policy_version 457798 (0.0007) [2023-12-27 08:48:27,506][20347] Updated weights for policy 0, policy_version 457808 (0.0006) [2023-12-27 08:48:27,515][20381] Updated weights for policy 1, policy_version 457434 (0.0009) [2023-12-27 08:48:27,563][20347] Updated weights for policy 0, policy_version 457818 (0.0006) [2023-12-27 08:48:27,572][20381] Updated weights for policy 1, policy_version 457444 (0.0007) [2023-12-27 08:48:27,625][20381] Updated weights for policy 1, policy_version 457454 (0.0007) [2023-12-27 08:48:28,164][20347] Updated weights for policy 0, policy_version 457828 (0.0006) [2023-12-27 08:48:28,211][20347] Updated weights for policy 0, policy_version 457838 (0.0008) [2023-12-27 08:48:28,245][20381] Updated weights for policy 1, policy_version 457464 (0.0005) [2023-12-27 08:48:28,262][20347] Updated weights for policy 0, policy_version 457848 (0.0007) [2023-12-27 08:48:28,292][20381] Updated weights for policy 1, policy_version 457474 (0.0005) [2023-12-27 08:48:28,349][20381] Updated weights for policy 1, policy_version 457484 (0.0008) [2023-12-27 08:48:28,981][20347] Updated weights for policy 0, policy_version 457858 (0.0007) [2023-12-27 08:48:28,989][20381] Updated weights for policy 1, policy_version 457494 (0.0009) [2023-12-27 08:48:29,042][20347] Updated weights for policy 0, policy_version 457868 (0.0008) [2023-12-27 08:48:29,047][20381] Updated weights for policy 1, policy_version 457504 (0.0007) [2023-12-27 08:48:29,098][20347] Updated weights for policy 0, policy_version 457878 (0.0008) [2023-12-27 08:48:29,101][20381] Updated weights for policy 1, policy_version 457514 (0.0005) [2023-12-27 08:48:29,152][20347] Updated weights for policy 0, policy_version 457888 (0.0007) [2023-12-27 08:48:29,765][20381] Updated weights for policy 1, policy_version 457524 (0.0006) [2023-12-27 08:48:29,814][20381] Updated weights for policy 1, policy_version 457534 (0.0007) [2023-12-27 08:48:29,840][20347] Updated weights for policy 0, policy_version 457898 (0.0007) [2023-12-27 08:48:29,872][20381] Updated weights for policy 1, policy_version 457544 (0.0008) [2023-12-27 08:48:29,891][20347] Updated weights for policy 0, policy_version 457908 (0.0006) [2023-12-27 08:48:29,948][20347] Updated weights for policy 0, policy_version 457918 (0.0006) [2023-12-27 08:48:30,550][20381] Updated weights for policy 1, policy_version 457554 (0.0007) [2023-12-27 08:48:30,593][20347] Updated weights for policy 0, policy_version 457928 (0.0006) [2023-12-27 08:48:30,603][20381] Updated weights for policy 1, policy_version 457564 (0.0006) [2023-12-27 08:48:30,654][20347] Updated weights for policy 0, policy_version 457938 (0.0006) [2023-12-27 08:48:30,659][20381] Updated weights for policy 1, policy_version 457574 (0.0008) [2023-12-27 08:48:30,712][20381] Updated weights for policy 1, policy_version 457584 (0.0007) [2023-12-27 08:48:30,717][20347] Updated weights for policy 0, policy_version 457948 (0.0006) [2023-12-27 08:48:31,375][20381] Updated weights for policy 1, policy_version 457594 (0.0007) [2023-12-27 08:48:31,399][20347] Updated weights for policy 0, policy_version 457958 (0.0008) [2023-12-27 08:48:31,434][20381] Updated weights for policy 1, policy_version 457604 (0.0006) [2023-12-27 08:48:31,453][20347] Updated weights for policy 0, policy_version 457968 (0.0005) [2023-12-27 08:48:31,490][20381] Updated weights for policy 1, policy_version 457614 (0.0005) [2023-12-27 08:48:31,508][20347] Updated weights for policy 0, policy_version 457978 (0.0007) [2023-12-27 08:48:32,079][20381] Updated weights for policy 1, policy_version 457624 (0.0007) [2023-12-27 08:48:32,130][20381] Updated weights for policy 1, policy_version 457634 (0.0007) [2023-12-27 08:48:32,177][20347] Updated weights for policy 0, policy_version 457988 (0.0007) [2023-12-27 08:48:32,181][20381] Updated weights for policy 1, policy_version 457644 (0.0006) [2023-12-27 08:48:32,224][20347] Updated weights for policy 0, policy_version 457998 (0.0006) [2023-12-27 08:48:32,279][20347] Updated weights for policy 0, policy_version 458008 (0.0006) [2023-12-27 08:48:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 234446848. Throughput: 0: 10323.8, 1: 10678.0. Samples: 234452000. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:32,363][19170] Avg episode reward: [(0, '290.070'), (1, '575.300')] [2023-12-27 08:48:32,803][20381] Updated weights for policy 1, policy_version 457654 (0.0006) [2023-12-27 08:48:32,857][20381] Updated weights for policy 1, policy_version 457664 (0.0005) [2023-12-27 08:48:32,902][20347] Updated weights for policy 0, policy_version 458018 (0.0007) [2023-12-27 08:48:32,917][20381] Updated weights for policy 1, policy_version 457674 (0.0005) [2023-12-27 08:48:32,954][20347] Updated weights for policy 0, policy_version 458028 (0.0006) [2023-12-27 08:48:33,013][20347] Updated weights for policy 0, policy_version 458038 (0.0006) [2023-12-27 08:48:33,066][20347] Updated weights for policy 0, policy_version 458048 (0.0007) [2023-12-27 08:48:33,557][20381] Updated weights for policy 1, policy_version 457684 (0.0005) [2023-12-27 08:48:33,616][20381] Updated weights for policy 1, policy_version 457694 (0.0008) [2023-12-27 08:48:33,676][20381] Updated weights for policy 1, policy_version 457704 (0.0007) [2023-12-27 08:48:33,697][20347] Updated weights for policy 0, policy_version 458058 (0.0007) [2023-12-27 08:48:33,747][20347] Updated weights for policy 0, policy_version 458068 (0.0010) [2023-12-27 08:48:33,804][20347] Updated weights for policy 0, policy_version 458078 (0.0010) [2023-12-27 08:48:34,367][20381] Updated weights for policy 1, policy_version 457714 (0.0008) [2023-12-27 08:48:34,427][20381] Updated weights for policy 1, policy_version 457724 (0.0008) [2023-12-27 08:48:34,457][20347] Updated weights for policy 0, policy_version 458088 (0.0008) [2023-12-27 08:48:34,481][20381] Updated weights for policy 1, policy_version 457734 (0.0007) [2023-12-27 08:48:34,515][20347] Updated weights for policy 0, policy_version 458098 (0.0009) [2023-12-27 08:48:34,539][20381] Updated weights for policy 1, policy_version 457744 (0.0007) [2023-12-27 08:48:34,579][20347] Updated weights for policy 0, policy_version 458108 (0.0008) [2023-12-27 08:48:35,234][20381] Updated weights for policy 1, policy_version 457754 (0.0008) [2023-12-27 08:48:35,273][20347] Updated weights for policy 0, policy_version 458118 (0.0009) [2023-12-27 08:48:35,283][20381] Updated weights for policy 1, policy_version 457764 (0.0007) [2023-12-27 08:48:35,325][20347] Updated weights for policy 0, policy_version 458128 (0.0006) [2023-12-27 08:48:35,338][20381] Updated weights for policy 1, policy_version 457774 (0.0007) [2023-12-27 08:48:35,371][20347] Updated weights for policy 0, policy_version 458138 (0.0006) [2023-12-27 08:48:36,054][20381] Updated weights for policy 1, policy_version 457784 (0.0007) [2023-12-27 08:48:36,076][20347] Updated weights for policy 0, policy_version 458148 (0.0007) [2023-12-27 08:48:36,108][20381] Updated weights for policy 1, policy_version 457794 (0.0007) [2023-12-27 08:48:36,140][20347] Updated weights for policy 0, policy_version 458158 (0.0007) [2023-12-27 08:48:36,168][20381] Updated weights for policy 1, policy_version 457804 (0.0007) [2023-12-27 08:48:36,199][20347] Updated weights for policy 0, policy_version 458168 (0.0008) [2023-12-27 08:48:36,869][20347] Updated weights for policy 0, policy_version 458178 (0.0007) [2023-12-27 08:48:36,890][20381] Updated weights for policy 1, policy_version 457814 (0.0007) [2023-12-27 08:48:36,926][20347] Updated weights for policy 0, policy_version 458188 (0.0007) [2023-12-27 08:48:36,944][20381] Updated weights for policy 1, policy_version 457824 (0.0006) [2023-12-27 08:48:36,986][20347] Updated weights for policy 0, policy_version 458198 (0.0006) [2023-12-27 08:48:36,995][20381] Updated weights for policy 1, policy_version 457834 (0.0007) [2023-12-27 08:48:37,042][20347] Updated weights for policy 0, policy_version 458208 (0.0006) [2023-12-27 08:48:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 234545152. Throughput: 0: 10369.7, 1: 10668.9. Samples: 234515212. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:37,364][19170] Avg episode reward: [(0, '278.000'), (1, '689.980')] [2023-12-27 08:48:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000457840_117227520.pth... [2023-12-27 08:48:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000458208_117317632.pth... [2023-12-27 08:48:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000456624_116916224.pth [2023-12-27 08:48:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000456992_117006336.pth [2023-12-27 08:48:37,706][20347] Updated weights for policy 0, policy_version 458218 (0.0005) [2023-12-27 08:48:37,736][20381] Updated weights for policy 1, policy_version 457844 (0.0006) [2023-12-27 08:48:37,754][20347] Updated weights for policy 0, policy_version 458228 (0.0005) [2023-12-27 08:48:37,796][20381] Updated weights for policy 1, policy_version 457854 (0.0008) [2023-12-27 08:48:37,814][20347] Updated weights for policy 0, policy_version 458238 (0.0005) [2023-12-27 08:48:37,864][20381] Updated weights for policy 1, policy_version 457864 (0.0008) [2023-12-27 08:48:38,421][20347] Updated weights for policy 0, policy_version 458248 (0.0007) [2023-12-27 08:48:38,467][20347] Updated weights for policy 0, policy_version 458258 (0.0008) [2023-12-27 08:48:38,515][20347] Updated weights for policy 0, policy_version 458268 (0.0008) [2023-12-27 08:48:38,546][20381] Updated weights for policy 1, policy_version 457874 (0.0009) [2023-12-27 08:48:38,601][20381] Updated weights for policy 1, policy_version 457884 (0.0008) [2023-12-27 08:48:38,653][20381] Updated weights for policy 1, policy_version 457894 (0.0009) [2023-12-27 08:48:38,712][20381] Updated weights for policy 1, policy_version 457904 (0.0007) [2023-12-27 08:48:39,212][20347] Updated weights for policy 0, policy_version 458278 (0.0009) [2023-12-27 08:48:39,271][20347] Updated weights for policy 0, policy_version 458288 (0.0007) [2023-12-27 08:48:39,333][20347] Updated weights for policy 0, policy_version 458298 (0.0008) [2023-12-27 08:48:39,419][20381] Updated weights for policy 1, policy_version 457914 (0.0007) [2023-12-27 08:48:39,483][20381] Updated weights for policy 1, policy_version 457924 (0.0008) [2023-12-27 08:48:39,539][20381] Updated weights for policy 1, policy_version 457934 (0.0008) [2023-12-27 08:48:40,027][20347] Updated weights for policy 0, policy_version 458308 (0.0009) [2023-12-27 08:48:40,087][20347] Updated weights for policy 0, policy_version 458318 (0.0010) [2023-12-27 08:48:40,142][20347] Updated weights for policy 0, policy_version 458328 (0.0010) [2023-12-27 08:48:40,295][20381] Updated weights for policy 1, policy_version 457944 (0.0010) [2023-12-27 08:48:40,356][20381] Updated weights for policy 1, policy_version 457954 (0.0010) [2023-12-27 08:48:40,414][20381] Updated weights for policy 1, policy_version 457964 (0.0011) [2023-12-27 08:48:40,821][20347] Updated weights for policy 0, policy_version 458338 (0.0009) [2023-12-27 08:48:40,875][20347] Updated weights for policy 0, policy_version 458348 (0.0007) [2023-12-27 08:48:40,935][20347] Updated weights for policy 0, policy_version 458358 (0.0010) [2023-12-27 08:48:40,999][20347] Updated weights for policy 0, policy_version 458368 (0.0008) [2023-12-27 08:48:41,012][20381] Updated weights for policy 1, policy_version 457974 (0.0010) [2023-12-27 08:48:41,070][20381] Updated weights for policy 1, policy_version 457984 (0.0008) [2023-12-27 08:48:41,131][20381] Updated weights for policy 1, policy_version 457994 (0.0008) [2023-12-27 08:48:41,664][20347] Updated weights for policy 0, policy_version 458378 (0.0010) [2023-12-27 08:48:41,726][20347] Updated weights for policy 0, policy_version 458388 (0.0008) [2023-12-27 08:48:41,786][20347] Updated weights for policy 0, policy_version 458398 (0.0010) [2023-12-27 08:48:41,846][20381] Updated weights for policy 1, policy_version 458004 (0.0007) [2023-12-27 08:48:41,904][20381] Updated weights for policy 1, policy_version 458014 (0.0005) [2023-12-27 08:48:41,959][20381] Updated weights for policy 1, policy_version 458024 (0.0007) [2023-12-27 08:48:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 234643456. Throughput: 0: 10412.2, 1: 10561.3. Samples: 234635948. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:42,363][19170] Avg episode reward: [(0, '263.370'), (1, '700.300')] [2023-12-27 08:48:42,410][20347] Updated weights for policy 0, policy_version 458408 (0.0007) [2023-12-27 08:48:42,476][20347] Updated weights for policy 0, policy_version 458418 (0.0006) [2023-12-27 08:48:42,531][20347] Updated weights for policy 0, policy_version 458428 (0.0008) [2023-12-27 08:48:42,640][20381] Updated weights for policy 1, policy_version 458034 (0.0008) [2023-12-27 08:48:42,695][20381] Updated weights for policy 1, policy_version 458044 (0.0006) [2023-12-27 08:48:42,750][20381] Updated weights for policy 1, policy_version 458054 (0.0005) [2023-12-27 08:48:42,809][20381] Updated weights for policy 1, policy_version 458064 (0.0006) [2023-12-27 08:48:43,163][20347] Updated weights for policy 0, policy_version 458438 (0.0007) [2023-12-27 08:48:43,217][20347] Updated weights for policy 0, policy_version 458448 (0.0007) [2023-12-27 08:48:43,281][20347] Updated weights for policy 0, policy_version 458458 (0.0005) [2023-12-27 08:48:43,480][20381] Updated weights for policy 1, policy_version 458074 (0.0011) [2023-12-27 08:48:43,540][20381] Updated weights for policy 1, policy_version 458084 (0.0010) [2023-12-27 08:48:43,595][20381] Updated weights for policy 1, policy_version 458094 (0.0010) [2023-12-27 08:48:43,859][20347] Updated weights for policy 0, policy_version 458468 (0.0005) [2023-12-27 08:48:43,913][20347] Updated weights for policy 0, policy_version 458478 (0.0005) [2023-12-27 08:48:43,975][20347] Updated weights for policy 0, policy_version 458488 (0.0005) [2023-12-27 08:48:44,258][20381] Updated weights for policy 1, policy_version 458104 (0.0011) [2023-12-27 08:48:44,317][20381] Updated weights for policy 1, policy_version 458114 (0.0010) [2023-12-27 08:48:44,372][20381] Updated weights for policy 1, policy_version 458124 (0.0010) [2023-12-27 08:48:44,575][20347] Updated weights for policy 0, policy_version 458498 (0.0006) [2023-12-27 08:48:44,633][20347] Updated weights for policy 0, policy_version 458508 (0.0010) [2023-12-27 08:48:44,687][20347] Updated weights for policy 0, policy_version 458518 (0.0010) [2023-12-27 08:48:44,746][20347] Updated weights for policy 0, policy_version 458528 (0.0007) [2023-12-27 08:48:45,068][20381] Updated weights for policy 1, policy_version 458134 (0.0011) [2023-12-27 08:48:45,124][20381] Updated weights for policy 1, policy_version 458144 (0.0011) [2023-12-27 08:48:45,188][20381] Updated weights for policy 1, policy_version 458154 (0.0011) [2023-12-27 08:48:45,379][20347] Updated weights for policy 0, policy_version 458538 (0.0010) [2023-12-27 08:48:45,429][20347] Updated weights for policy 0, policy_version 458548 (0.0010) [2023-12-27 08:48:45,477][20347] Updated weights for policy 0, policy_version 458558 (0.0010) [2023-12-27 08:48:45,887][20381] Updated weights for policy 1, policy_version 458164 (0.0011) [2023-12-27 08:48:45,948][20381] Updated weights for policy 1, policy_version 458174 (0.0010) [2023-12-27 08:48:45,997][20381] Updated weights for policy 1, policy_version 458184 (0.0010) [2023-12-27 08:48:46,178][20347] Updated weights for policy 0, policy_version 458568 (0.0011) [2023-12-27 08:48:46,226][20347] Updated weights for policy 0, policy_version 458578 (0.0010) [2023-12-27 08:48:46,274][20347] Updated weights for policy 0, policy_version 458588 (0.0010) [2023-12-27 08:48:46,589][20381] Updated weights for policy 1, policy_version 458194 (0.0010) [2023-12-27 08:48:46,643][20381] Updated weights for policy 1, policy_version 458204 (0.0008) [2023-12-27 08:48:46,688][20381] Updated weights for policy 1, policy_version 458214 (0.0008) [2023-12-27 08:48:46,738][20381] Updated weights for policy 1, policy_version 458224 (0.0009) [2023-12-27 08:48:46,859][20347] Updated weights for policy 0, policy_version 458598 (0.0010) [2023-12-27 08:48:46,907][20347] Updated weights for policy 0, policy_version 458608 (0.0010) [2023-12-27 08:48:46,959][20347] Updated weights for policy 0, policy_version 458618 (0.0010) [2023-12-27 08:48:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 234749952. Throughput: 0: 10465.9, 1: 10493.9. Samples: 234761876. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:47,363][19170] Avg episode reward: [(0, '279.820'), (1, '786.670')] [2023-12-27 08:48:47,449][20381] Updated weights for policy 1, policy_version 458234 (0.0009) [2023-12-27 08:48:47,501][20381] Updated weights for policy 1, policy_version 458244 (0.0008) [2023-12-27 08:48:47,555][20347] Updated weights for policy 0, policy_version 458628 (0.0011) [2023-12-27 08:48:47,557][20381] Updated weights for policy 1, policy_version 458254 (0.0006) [2023-12-27 08:48:47,610][20347] Updated weights for policy 0, policy_version 458638 (0.0010) [2023-12-27 08:48:47,660][20347] Updated weights for policy 0, policy_version 458648 (0.0009) [2023-12-27 08:48:48,275][20381] Updated weights for policy 1, policy_version 458264 (0.0007) [2023-12-27 08:48:48,288][20347] Updated weights for policy 0, policy_version 458658 (0.0007) [2023-12-27 08:48:48,318][20381] Updated weights for policy 1, policy_version 458274 (0.0006) [2023-12-27 08:48:48,348][20347] Updated weights for policy 0, policy_version 458668 (0.0006) [2023-12-27 08:48:48,375][20381] Updated weights for policy 1, policy_version 458284 (0.0007) [2023-12-27 08:48:48,407][20347] Updated weights for policy 0, policy_version 458678 (0.0006) [2023-12-27 08:48:48,467][20347] Updated weights for policy 0, policy_version 458688 (0.0006) [2023-12-27 08:48:49,055][20347] Updated weights for policy 0, policy_version 458698 (0.0005) [2023-12-27 08:48:49,113][20347] Updated weights for policy 0, policy_version 458708 (0.0005) [2023-12-27 08:48:49,134][20381] Updated weights for policy 1, policy_version 458294 (0.0007) [2023-12-27 08:48:49,167][20347] Updated weights for policy 0, policy_version 458718 (0.0005) [2023-12-27 08:48:49,193][20381] Updated weights for policy 1, policy_version 458304 (0.0008) [2023-12-27 08:48:49,256][20381] Updated weights for policy 1, policy_version 458314 (0.0009) [2023-12-27 08:48:49,820][20347] Updated weights for policy 0, policy_version 458728 (0.0007) [2023-12-27 08:48:49,885][20381] Updated weights for policy 1, policy_version 458324 (0.0007) [2023-12-27 08:48:49,886][20347] Updated weights for policy 0, policy_version 458738 (0.0007) [2023-12-27 08:48:49,946][20381] Updated weights for policy 1, policy_version 458334 (0.0007) [2023-12-27 08:48:49,948][20347] Updated weights for policy 0, policy_version 458748 (0.0007) [2023-12-27 08:48:49,998][20381] Updated weights for policy 1, policy_version 458344 (0.0007) [2023-12-27 08:48:50,623][20347] Updated weights for policy 0, policy_version 458758 (0.0007) [2023-12-27 08:48:50,674][20347] Updated weights for policy 0, policy_version 458768 (0.0006) [2023-12-27 08:48:50,708][20381] Updated weights for policy 1, policy_version 458354 (0.0008) [2023-12-27 08:48:50,732][20347] Updated weights for policy 0, policy_version 458778 (0.0007) [2023-12-27 08:48:50,761][20381] Updated weights for policy 1, policy_version 458364 (0.0005) [2023-12-27 08:48:50,816][20381] Updated weights for policy 1, policy_version 458374 (0.0007) [2023-12-27 08:48:50,872][20381] Updated weights for policy 1, policy_version 458384 (0.0008) [2023-12-27 08:48:51,421][20347] Updated weights for policy 0, policy_version 458788 (0.0007) [2023-12-27 08:48:51,477][20347] Updated weights for policy 0, policy_version 458798 (0.0007) [2023-12-27 08:48:51,534][20347] Updated weights for policy 0, policy_version 458808 (0.0007) [2023-12-27 08:48:51,649][20381] Updated weights for policy 1, policy_version 458394 (0.0009) [2023-12-27 08:48:51,717][20381] Updated weights for policy 1, policy_version 458404 (0.0007) [2023-12-27 08:48:51,784][20381] Updated weights for policy 1, policy_version 458414 (0.0008) [2023-12-27 08:48:52,176][20347] Updated weights for policy 0, policy_version 458818 (0.0006) [2023-12-27 08:48:52,233][20347] Updated weights for policy 0, policy_version 458828 (0.0008) [2023-12-27 08:48:52,286][20347] Updated weights for policy 0, policy_version 458838 (0.0008) [2023-12-27 08:48:52,339][20347] Updated weights for policy 0, policy_version 458848 (0.0007) [2023-12-27 08:48:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 234856448. Throughput: 0: 10495.7, 1: 10426.5. Samples: 234824940. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:52,363][19170] Avg episode reward: [(0, '287.790'), (1, '806.790')] [2023-12-27 08:48:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000458416_117374976.pth... [2023-12-27 08:48:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000458848_117481472.pth... [2023-12-27 08:48:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000457600_117161984.pth [2023-12-27 08:48:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000457232_117071872.pth [2023-12-27 08:48:52,485][20381] Updated weights for policy 1, policy_version 458424 (0.0008) [2023-12-27 08:48:52,544][20381] Updated weights for policy 1, policy_version 458434 (0.0008) [2023-12-27 08:48:52,586][20381] Updated weights for policy 1, policy_version 458444 (0.0008) [2023-12-27 08:48:53,000][20347] Updated weights for policy 0, policy_version 458858 (0.0008) [2023-12-27 08:48:53,060][20347] Updated weights for policy 0, policy_version 458868 (0.0007) [2023-12-27 08:48:53,106][20347] Updated weights for policy 0, policy_version 458878 (0.0008) [2023-12-27 08:48:53,274][20381] Updated weights for policy 1, policy_version 458454 (0.0008) [2023-12-27 08:48:53,323][20381] Updated weights for policy 1, policy_version 458464 (0.0007) [2023-12-27 08:48:53,369][20381] Updated weights for policy 1, policy_version 458474 (0.0007) [2023-12-27 08:48:53,823][20347] Updated weights for policy 0, policy_version 458888 (0.0008) [2023-12-27 08:48:53,875][20347] Updated weights for policy 0, policy_version 458898 (0.0010) [2023-12-27 08:48:53,931][20347] Updated weights for policy 0, policy_version 458908 (0.0009) [2023-12-27 08:48:54,088][20381] Updated weights for policy 1, policy_version 458484 (0.0006) [2023-12-27 08:48:54,142][20381] Updated weights for policy 1, policy_version 458494 (0.0005) [2023-12-27 08:48:54,203][20381] Updated weights for policy 1, policy_version 458504 (0.0006) [2023-12-27 08:48:54,575][20347] Updated weights for policy 0, policy_version 458918 (0.0006) [2023-12-27 08:48:54,627][20347] Updated weights for policy 0, policy_version 458928 (0.0008) [2023-12-27 08:48:54,684][20347] Updated weights for policy 0, policy_version 458938 (0.0006) [2023-12-27 08:48:54,822][20381] Updated weights for policy 1, policy_version 458514 (0.0008) [2023-12-27 08:48:54,878][20381] Updated weights for policy 1, policy_version 458524 (0.0006) [2023-12-27 08:48:54,936][20381] Updated weights for policy 1, policy_version 458534 (0.0011) [2023-12-27 08:48:54,986][20381] Updated weights for policy 1, policy_version 458544 (0.0011) [2023-12-27 08:48:55,372][20347] Updated weights for policy 0, policy_version 458948 (0.0010) [2023-12-27 08:48:55,425][20347] Updated weights for policy 0, policy_version 458958 (0.0010) [2023-12-27 08:48:55,484][20347] Updated weights for policy 0, policy_version 458968 (0.0010) [2023-12-27 08:48:55,673][20381] Updated weights for policy 1, policy_version 458554 (0.0011) [2023-12-27 08:48:55,735][20381] Updated weights for policy 1, policy_version 458564 (0.0011) [2023-12-27 08:48:55,794][20381] Updated weights for policy 1, policy_version 458574 (0.0010) [2023-12-27 08:48:56,154][20347] Updated weights for policy 0, policy_version 458978 (0.0010) [2023-12-27 08:48:56,212][20347] Updated weights for policy 0, policy_version 458988 (0.0010) [2023-12-27 08:48:56,270][20347] Updated weights for policy 0, policy_version 458998 (0.0010) [2023-12-27 08:48:56,323][20347] Updated weights for policy 0, policy_version 459008 (0.0010) [2023-12-27 08:48:56,409][20381] Updated weights for policy 1, policy_version 458584 (0.0010) [2023-12-27 08:48:56,462][20381] Updated weights for policy 1, policy_version 458594 (0.0010) [2023-12-27 08:48:56,523][20381] Updated weights for policy 1, policy_version 458604 (0.0009) [2023-12-27 08:48:56,983][20347] Updated weights for policy 0, policy_version 459018 (0.0010) [2023-12-27 08:48:57,038][20347] Updated weights for policy 0, policy_version 459028 (0.0010) [2023-12-27 08:48:57,091][20347] Updated weights for policy 0, policy_version 459038 (0.0010) [2023-12-27 08:48:57,131][20381] Updated weights for policy 1, policy_version 458614 (0.0009) [2023-12-27 08:48:57,185][20381] Updated weights for policy 1, policy_version 458624 (0.0009) [2023-12-27 08:48:57,244][20381] Updated weights for policy 1, policy_version 458634 (0.0005) [2023-12-27 08:48:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 234962944. Throughput: 0: 10515.4, 1: 10296.1. Samples: 234947952. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:48:57,363][19170] Avg episode reward: [(0, '245.300'), (1, '689.120')] [2023-12-27 08:48:57,756][20347] Updated weights for policy 0, policy_version 459048 (0.0009) [2023-12-27 08:48:57,813][20347] Updated weights for policy 0, policy_version 459058 (0.0007) [2023-12-27 08:48:57,830][20381] Updated weights for policy 1, policy_version 458644 (0.0006) [2023-12-27 08:48:57,866][20347] Updated weights for policy 0, policy_version 459068 (0.0005) [2023-12-27 08:48:57,879][20381] Updated weights for policy 1, policy_version 458654 (0.0007) [2023-12-27 08:48:57,923][20381] Updated weights for policy 1, policy_version 458664 (0.0006) [2023-12-27 08:48:58,439][20347] Updated weights for policy 0, policy_version 459078 (0.0008) [2023-12-27 08:48:58,497][20347] Updated weights for policy 0, policy_version 459088 (0.0010) [2023-12-27 08:48:58,566][20347] Updated weights for policy 0, policy_version 459098 (0.0010) [2023-12-27 08:48:58,600][20381] Updated weights for policy 1, policy_version 458674 (0.0009) [2023-12-27 08:48:58,664][20381] Updated weights for policy 1, policy_version 458684 (0.0007) [2023-12-27 08:48:58,719][20381] Updated weights for policy 1, policy_version 458694 (0.0008) [2023-12-27 08:48:58,775][20381] Updated weights for policy 1, policy_version 458704 (0.0008) [2023-12-27 08:48:59,346][20347] Updated weights for policy 0, policy_version 459108 (0.0008) [2023-12-27 08:48:59,408][20347] Updated weights for policy 0, policy_version 459118 (0.0008) [2023-12-27 08:48:59,462][20347] Updated weights for policy 0, policy_version 459128 (0.0008) [2023-12-27 08:48:59,494][20381] Updated weights for policy 1, policy_version 458714 (0.0011) [2023-12-27 08:48:59,550][20381] Updated weights for policy 1, policy_version 458724 (0.0008) [2023-12-27 08:48:59,610][20381] Updated weights for policy 1, policy_version 458734 (0.0008) [2023-12-27 08:49:00,113][20347] Updated weights for policy 0, policy_version 459138 (0.0008) [2023-12-27 08:49:00,172][20347] Updated weights for policy 0, policy_version 459148 (0.0011) [2023-12-27 08:49:00,206][20381] Updated weights for policy 1, policy_version 458744 (0.0006) [2023-12-27 08:49:00,233][20347] Updated weights for policy 0, policy_version 459158 (0.0010) [2023-12-27 08:49:00,272][20381] Updated weights for policy 1, policy_version 458754 (0.0005) [2023-12-27 08:49:00,285][20347] Updated weights for policy 0, policy_version 459168 (0.0010) [2023-12-27 08:49:00,318][20381] Updated weights for policy 1, policy_version 458764 (0.0005) [2023-12-27 08:49:00,949][20381] Updated weights for policy 1, policy_version 458774 (0.0010) [2023-12-27 08:49:00,951][20347] Updated weights for policy 0, policy_version 459178 (0.0005) [2023-12-27 08:49:01,004][20381] Updated weights for policy 1, policy_version 458784 (0.0007) [2023-12-27 08:49:01,012][20347] Updated weights for policy 0, policy_version 459188 (0.0008) [2023-12-27 08:49:01,066][20381] Updated weights for policy 1, policy_version 458794 (0.0010) [2023-12-27 08:49:01,077][20347] Updated weights for policy 0, policy_version 459198 (0.0010) [2023-12-27 08:49:01,761][20347] Updated weights for policy 0, policy_version 459208 (0.0009) [2023-12-27 08:49:01,780][20381] Updated weights for policy 1, policy_version 458804 (0.0010) [2023-12-27 08:49:01,817][20347] Updated weights for policy 0, policy_version 459218 (0.0010) [2023-12-27 08:49:01,831][20381] Updated weights for policy 1, policy_version 458814 (0.0008) [2023-12-27 08:49:01,874][20347] Updated weights for policy 0, policy_version 459228 (0.0010) [2023-12-27 08:49:01,875][20381] Updated weights for policy 1, policy_version 458824 (0.0008) [2023-12-27 08:49:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 235061248. Throughput: 0: 10541.5, 1: 10296.7. Samples: 235073172. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:02,363][19170] Avg episode reward: [(0, '242.640'), (1, '700.250')] [2023-12-27 08:49:02,506][20381] Updated weights for policy 1, policy_version 458834 (0.0007) [2023-12-27 08:49:02,527][20347] Updated weights for policy 0, policy_version 459238 (0.0009) [2023-12-27 08:49:02,564][20381] Updated weights for policy 1, policy_version 458844 (0.0006) [2023-12-27 08:49:02,599][20347] Updated weights for policy 0, policy_version 459248 (0.0007) [2023-12-27 08:49:02,617][20381] Updated weights for policy 1, policy_version 458854 (0.0007) [2023-12-27 08:49:02,656][20347] Updated weights for policy 0, policy_version 459258 (0.0008) [2023-12-27 08:49:02,666][20381] Updated weights for policy 1, policy_version 458864 (0.0006) [2023-12-27 08:49:03,303][20347] Updated weights for policy 0, policy_version 459268 (0.0009) [2023-12-27 08:49:03,341][20381] Updated weights for policy 1, policy_version 458874 (0.0006) [2023-12-27 08:49:03,368][20347] Updated weights for policy 0, policy_version 459278 (0.0006) [2023-12-27 08:49:03,389][20381] Updated weights for policy 1, policy_version 458884 (0.0006) [2023-12-27 08:49:03,430][20347] Updated weights for policy 0, policy_version 459288 (0.0009) [2023-12-27 08:49:03,440][20381] Updated weights for policy 1, policy_version 458894 (0.0006) [2023-12-27 08:49:04,090][20347] Updated weights for policy 0, policy_version 459298 (0.0010) [2023-12-27 08:49:04,141][20381] Updated weights for policy 1, policy_version 458904 (0.0008) [2023-12-27 08:49:04,162][20347] Updated weights for policy 0, policy_version 459308 (0.0007) [2023-12-27 08:49:04,203][20381] Updated weights for policy 1, policy_version 458914 (0.0008) [2023-12-27 08:49:04,228][20347] Updated weights for policy 0, policy_version 459318 (0.0007) [2023-12-27 08:49:04,261][20381] Updated weights for policy 1, policy_version 458924 (0.0007) [2023-12-27 08:49:04,284][20347] Updated weights for policy 0, policy_version 459328 (0.0006) [2023-12-27 08:49:04,882][20347] Updated weights for policy 0, policy_version 459338 (0.0007) [2023-12-27 08:49:04,932][20347] Updated weights for policy 0, policy_version 459348 (0.0005) [2023-12-27 08:49:04,983][20347] Updated weights for policy 0, policy_version 459358 (0.0005) [2023-12-27 08:49:04,997][20381] Updated weights for policy 1, policy_version 458934 (0.0008) [2023-12-27 08:49:05,046][20381] Updated weights for policy 1, policy_version 458944 (0.0008) [2023-12-27 08:49:05,092][20381] Updated weights for policy 1, policy_version 458954 (0.0009) [2023-12-27 08:49:05,604][20347] Updated weights for policy 0, policy_version 459368 (0.0007) [2023-12-27 08:49:05,654][20347] Updated weights for policy 0, policy_version 459378 (0.0007) [2023-12-27 08:49:05,730][20347] Updated weights for policy 0, policy_version 459388 (0.0008) [2023-12-27 08:49:05,816][20381] Updated weights for policy 1, policy_version 458964 (0.0008) [2023-12-27 08:49:05,862][20381] Updated weights for policy 1, policy_version 458974 (0.0008) [2023-12-27 08:49:05,905][20381] Updated weights for policy 1, policy_version 458984 (0.0007) [2023-12-27 08:49:06,373][20347] Updated weights for policy 0, policy_version 459398 (0.0007) [2023-12-27 08:49:06,433][20347] Updated weights for policy 0, policy_version 459408 (0.0008) [2023-12-27 08:49:06,493][20347] Updated weights for policy 0, policy_version 459418 (0.0008) [2023-12-27 08:49:06,700][20381] Updated weights for policy 1, policy_version 458994 (0.0008) [2023-12-27 08:49:06,758][20381] Updated weights for policy 1, policy_version 459004 (0.0008) [2023-12-27 08:49:06,808][20381] Updated weights for policy 1, policy_version 459014 (0.0008) [2023-12-27 08:49:06,860][20381] Updated weights for policy 1, policy_version 459024 (0.0008) [2023-12-27 08:49:07,187][20347] Updated weights for policy 0, policy_version 459428 (0.0008) [2023-12-27 08:49:07,235][20347] Updated weights for policy 0, policy_version 459438 (0.0008) [2023-12-27 08:49:07,285][20347] Updated weights for policy 0, policy_version 459448 (0.0006) [2023-12-27 08:49:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 235167744. Throughput: 0: 10542.8, 1: 10266.7. Samples: 235134944. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:07,363][19170] Avg episode reward: [(0, '318.920'), (1, '649.660')] [2023-12-27 08:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000459024_117530624.pth... [2023-12-27 08:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000459456_117637120.pth... [2023-12-27 08:49:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000458208_117317632.pth [2023-12-27 08:49:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000457840_117227520.pth [2023-12-27 08:49:07,541][20381] Updated weights for policy 1, policy_version 459034 (0.0008) [2023-12-27 08:49:07,600][20381] Updated weights for policy 1, policy_version 459044 (0.0008) [2023-12-27 08:49:07,648][20381] Updated weights for policy 1, policy_version 459054 (0.0008) [2023-12-27 08:49:07,974][20347] Updated weights for policy 0, policy_version 459458 (0.0007) [2023-12-27 08:49:08,029][20347] Updated weights for policy 0, policy_version 459468 (0.0007) [2023-12-27 08:49:08,084][20347] Updated weights for policy 0, policy_version 459478 (0.0007) [2023-12-27 08:49:08,137][20347] Updated weights for policy 0, policy_version 459488 (0.0008) [2023-12-27 08:49:08,381][20381] Updated weights for policy 1, policy_version 459064 (0.0008) [2023-12-27 08:49:08,431][20381] Updated weights for policy 1, policy_version 459074 (0.0008) [2023-12-27 08:49:08,484][20381] Updated weights for policy 1, policy_version 459084 (0.0008) [2023-12-27 08:49:08,860][20347] Updated weights for policy 0, policy_version 459498 (0.0010) [2023-12-27 08:49:08,908][20347] Updated weights for policy 0, policy_version 459508 (0.0010) [2023-12-27 08:49:08,957][20347] Updated weights for policy 0, policy_version 459518 (0.0010) [2023-12-27 08:49:09,203][20381] Updated weights for policy 1, policy_version 459094 (0.0007) [2023-12-27 08:49:09,266][20381] Updated weights for policy 1, policy_version 459104 (0.0008) [2023-12-27 08:49:09,323][20381] Updated weights for policy 1, policy_version 459114 (0.0008) [2023-12-27 08:49:09,625][20347] Updated weights for policy 0, policy_version 459528 (0.0007) [2023-12-27 08:49:09,678][20347] Updated weights for policy 0, policy_version 459538 (0.0005) [2023-12-27 08:49:09,736][20347] Updated weights for policy 0, policy_version 459548 (0.0008) [2023-12-27 08:49:10,055][20381] Updated weights for policy 1, policy_version 459124 (0.0007) [2023-12-27 08:49:10,117][20381] Updated weights for policy 1, policy_version 459134 (0.0007) [2023-12-27 08:49:10,182][20381] Updated weights for policy 1, policy_version 459144 (0.0008) [2023-12-27 08:49:10,416][20347] Updated weights for policy 0, policy_version 459558 (0.0009) [2023-12-27 08:49:10,479][20347] Updated weights for policy 0, policy_version 459568 (0.0011) [2023-12-27 08:49:10,543][20347] Updated weights for policy 0, policy_version 459578 (0.0011) [2023-12-27 08:49:10,861][20381] Updated weights for policy 1, policy_version 459154 (0.0009) [2023-12-27 08:49:10,912][20381] Updated weights for policy 1, policy_version 459164 (0.0010) [2023-12-27 08:49:10,957][20381] Updated weights for policy 1, policy_version 459174 (0.0010) [2023-12-27 08:49:11,019][20381] Updated weights for policy 1, policy_version 459184 (0.0007) [2023-12-27 08:49:11,173][20347] Updated weights for policy 0, policy_version 459588 (0.0009) [2023-12-27 08:49:11,233][20347] Updated weights for policy 0, policy_version 459598 (0.0008) [2023-12-27 08:49:11,293][20347] Updated weights for policy 0, policy_version 459608 (0.0008) [2023-12-27 08:49:11,754][20381] Updated weights for policy 1, policy_version 459194 (0.0009) [2023-12-27 08:49:11,818][20381] Updated weights for policy 1, policy_version 459204 (0.0011) [2023-12-27 08:49:11,885][20381] Updated weights for policy 1, policy_version 459214 (0.0011) [2023-12-27 08:49:11,964][20347] Updated weights for policy 0, policy_version 459618 (0.0009) [2023-12-27 08:49:12,017][20347] Updated weights for policy 0, policy_version 459628 (0.0007) [2023-12-27 08:49:12,078][20347] Updated weights for policy 0, policy_version 459638 (0.0010) [2023-12-27 08:49:12,140][20347] Updated weights for policy 0, policy_version 459648 (0.0008) [2023-12-27 08:49:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.3, 300 sec: 20771.6). Total num frames: 235266048. Throughput: 0: 10514.3, 1: 10207.4. Samples: 235256048. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:12,363][19170] Avg episode reward: [(0, '328.950'), (1, '785.940')] [2023-12-27 08:49:12,581][20381] Updated weights for policy 1, policy_version 459224 (0.0010) [2023-12-27 08:49:12,634][20381] Updated weights for policy 1, policy_version 459234 (0.0010) [2023-12-27 08:49:12,686][20381] Updated weights for policy 1, policy_version 459244 (0.0010) [2023-12-27 08:49:12,815][20347] Updated weights for policy 0, policy_version 459658 (0.0008) [2023-12-27 08:49:12,870][20347] Updated weights for policy 0, policy_version 459668 (0.0007) [2023-12-27 08:49:12,917][20347] Updated weights for policy 0, policy_version 459678 (0.0005) [2023-12-27 08:49:13,368][20381] Updated weights for policy 1, policy_version 459254 (0.0007) [2023-12-27 08:49:13,426][20381] Updated weights for policy 1, policy_version 459264 (0.0006) [2023-12-27 08:49:13,480][20381] Updated weights for policy 1, policy_version 459274 (0.0007) [2023-12-27 08:49:13,557][20347] Updated weights for policy 0, policy_version 459688 (0.0009) [2023-12-27 08:49:13,609][20347] Updated weights for policy 0, policy_version 459698 (0.0010) [2023-12-27 08:49:13,659][20347] Updated weights for policy 0, policy_version 459708 (0.0009) [2023-12-27 08:49:14,151][20381] Updated weights for policy 1, policy_version 459284 (0.0008) [2023-12-27 08:49:14,198][20381] Updated weights for policy 1, policy_version 459294 (0.0010) [2023-12-27 08:49:14,249][20381] Updated weights for policy 1, policy_version 459304 (0.0009) [2023-12-27 08:49:14,294][20347] Updated weights for policy 0, policy_version 459718 (0.0007) [2023-12-27 08:49:14,345][20347] Updated weights for policy 0, policy_version 459728 (0.0007) [2023-12-27 08:49:14,399][20347] Updated weights for policy 0, policy_version 459738 (0.0007) [2023-12-27 08:49:14,926][20381] Updated weights for policy 1, policy_version 459314 (0.0010) [2023-12-27 08:49:14,988][20381] Updated weights for policy 1, policy_version 459324 (0.0007) [2023-12-27 08:49:15,045][20381] Updated weights for policy 1, policy_version 459334 (0.0008) [2023-12-27 08:49:15,100][20381] Updated weights for policy 1, policy_version 459344 (0.0007) [2023-12-27 08:49:15,110][20347] Updated weights for policy 0, policy_version 459748 (0.0007) [2023-12-27 08:49:15,160][20347] Updated weights for policy 0, policy_version 459758 (0.0008) [2023-12-27 08:49:15,218][20347] Updated weights for policy 0, policy_version 459768 (0.0007) [2023-12-27 08:49:15,795][20381] Updated weights for policy 1, policy_version 459354 (0.0006) [2023-12-27 08:49:15,853][20381] Updated weights for policy 1, policy_version 459364 (0.0006) [2023-12-27 08:49:15,891][20347] Updated weights for policy 0, policy_version 459778 (0.0007) [2023-12-27 08:49:15,911][20381] Updated weights for policy 1, policy_version 459374 (0.0005) [2023-12-27 08:49:15,947][20347] Updated weights for policy 0, policy_version 459788 (0.0006) [2023-12-27 08:49:16,007][20347] Updated weights for policy 0, policy_version 459798 (0.0007) [2023-12-27 08:49:16,051][20347] Updated weights for policy 0, policy_version 459808 (0.0006) [2023-12-27 08:49:16,541][20381] Updated weights for policy 1, policy_version 459384 (0.0009) [2023-12-27 08:49:16,590][20381] Updated weights for policy 1, policy_version 459394 (0.0010) [2023-12-27 08:49:16,645][20381] Updated weights for policy 1, policy_version 459404 (0.0010) [2023-12-27 08:49:16,740][20347] Updated weights for policy 0, policy_version 459818 (0.0006) [2023-12-27 08:49:16,786][20347] Updated weights for policy 0, policy_version 459828 (0.0007) [2023-12-27 08:49:16,844][20347] Updated weights for policy 0, policy_version 459838 (0.0009) [2023-12-27 08:49:17,320][20381] Updated weights for policy 1, policy_version 459414 (0.0010) [2023-12-27 08:49:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 235364352. Throughput: 0: 10486.2, 1: 10132.7. Samples: 235379852. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:17,363][19170] Avg episode reward: [(0, '358.230'), (1, '786.840')] [2023-12-27 08:49:17,371][20381] Updated weights for policy 1, policy_version 459424 (0.0005) [2023-12-27 08:49:17,414][20347] Updated weights for policy 0, policy_version 459848 (0.0009) [2023-12-27 08:49:17,428][20381] Updated weights for policy 1, policy_version 459434 (0.0009) [2023-12-27 08:49:17,468][20347] Updated weights for policy 0, policy_version 459858 (0.0010) [2023-12-27 08:49:17,526][20347] Updated weights for policy 0, policy_version 459868 (0.0010) [2023-12-27 08:49:18,102][20381] Updated weights for policy 1, policy_version 459444 (0.0010) [2023-12-27 08:49:18,160][20381] Updated weights for policy 1, policy_version 459454 (0.0010) [2023-12-27 08:49:18,204][20347] Updated weights for policy 0, policy_version 459878 (0.0010) [2023-12-27 08:49:18,223][20381] Updated weights for policy 1, policy_version 459464 (0.0008) [2023-12-27 08:49:18,259][20347] Updated weights for policy 0, policy_version 459888 (0.0010) [2023-12-27 08:49:18,313][20347] Updated weights for policy 0, policy_version 459898 (0.0009) [2023-12-27 08:49:18,903][20381] Updated weights for policy 1, policy_version 459474 (0.0008) [2023-12-27 08:49:18,946][20381] Updated weights for policy 1, policy_version 459484 (0.0008) [2023-12-27 08:49:18,995][20347] Updated weights for policy 0, policy_version 459908 (0.0008) [2023-12-27 08:49:19,000][20381] Updated weights for policy 1, policy_version 459494 (0.0008) [2023-12-27 08:49:19,051][20381] Updated weights for policy 1, policy_version 459504 (0.0006) [2023-12-27 08:49:19,055][20347] Updated weights for policy 0, policy_version 459918 (0.0010) [2023-12-27 08:49:19,116][20347] Updated weights for policy 0, policy_version 459928 (0.0010) [2023-12-27 08:49:19,704][20381] Updated weights for policy 1, policy_version 459514 (0.0011) [2023-12-27 08:49:19,761][20347] Updated weights for policy 0, policy_version 459938 (0.0010) [2023-12-27 08:49:19,764][20381] Updated weights for policy 1, policy_version 459524 (0.0011) [2023-12-27 08:49:19,818][20347] Updated weights for policy 0, policy_version 459948 (0.0010) [2023-12-27 08:49:19,822][20381] Updated weights for policy 1, policy_version 459534 (0.0011) [2023-12-27 08:49:19,882][20347] Updated weights for policy 0, policy_version 459958 (0.0009) [2023-12-27 08:49:19,940][20347] Updated weights for policy 0, policy_version 459968 (0.0008) [2023-12-27 08:49:20,535][20381] Updated weights for policy 1, policy_version 459544 (0.0008) [2023-12-27 08:49:20,599][20381] Updated weights for policy 1, policy_version 459554 (0.0008) [2023-12-27 08:49:20,622][20347] Updated weights for policy 0, policy_version 459978 (0.0011) [2023-12-27 08:49:20,654][20381] Updated weights for policy 1, policy_version 459564 (0.0008) [2023-12-27 08:49:20,680][20347] Updated weights for policy 0, policy_version 459988 (0.0011) [2023-12-27 08:49:20,730][20347] Updated weights for policy 0, policy_version 459998 (0.0010) [2023-12-27 08:49:21,363][20381] Updated weights for policy 1, policy_version 459574 (0.0011) [2023-12-27 08:49:21,404][20347] Updated weights for policy 0, policy_version 460008 (0.0008) [2023-12-27 08:49:21,429][20381] Updated weights for policy 1, policy_version 459584 (0.0006) [2023-12-27 08:49:21,461][20347] Updated weights for policy 0, policy_version 460018 (0.0007) [2023-12-27 08:49:21,487][20381] Updated weights for policy 1, policy_version 459594 (0.0007) [2023-12-27 08:49:21,522][20347] Updated weights for policy 0, policy_version 460028 (0.0009) [2023-12-27 08:49:22,192][20347] Updated weights for policy 0, policy_version 460038 (0.0008) [2023-12-27 08:49:22,254][20381] Updated weights for policy 1, policy_version 459604 (0.0008) [2023-12-27 08:49:22,257][20347] Updated weights for policy 0, policy_version 460048 (0.0007) [2023-12-27 08:49:22,316][20347] Updated weights for policy 0, policy_version 460058 (0.0008) [2023-12-27 08:49:22,319][20381] Updated weights for policy 1, policy_version 459614 (0.0011) [2023-12-27 08:49:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 235470848. Throughput: 0: 10470.8, 1: 10118.7. Samples: 235441736. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:22,363][19170] Avg episode reward: [(0, '389.190'), (1, '845.310')] [2023-12-27 08:49:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000460064_117792768.pth... [2023-12-27 08:49:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000458848_117481472.pth [2023-12-27 08:49:22,389][20381] Updated weights for policy 1, policy_version 459624 (0.0010) [2023-12-27 08:49:22,439][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000459632_117686272.pth... [2023-12-27 08:49:22,442][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000458416_117374976.pth [2023-12-27 08:49:22,968][20347] Updated weights for policy 0, policy_version 460068 (0.0008) [2023-12-27 08:49:23,026][20347] Updated weights for policy 0, policy_version 460078 (0.0007) [2023-12-27 08:49:23,070][20381] Updated weights for policy 1, policy_version 459634 (0.0008) [2023-12-27 08:49:23,084][20347] Updated weights for policy 0, policy_version 460088 (0.0007) [2023-12-27 08:49:23,117][20381] Updated weights for policy 1, policy_version 459644 (0.0008) [2023-12-27 08:49:23,171][20381] Updated weights for policy 1, policy_version 459654 (0.0010) [2023-12-27 08:49:23,227][20381] Updated weights for policy 1, policy_version 459664 (0.0010) [2023-12-27 08:49:23,727][20347] Updated weights for policy 0, policy_version 460098 (0.0006) [2023-12-27 08:49:23,774][20347] Updated weights for policy 0, policy_version 460108 (0.0007) [2023-12-27 08:49:23,828][20347] Updated weights for policy 0, policy_version 460118 (0.0007) [2023-12-27 08:49:23,879][20347] Updated weights for policy 0, policy_version 460128 (0.0005) [2023-12-27 08:49:23,926][20381] Updated weights for policy 1, policy_version 459674 (0.0010) [2023-12-27 08:49:23,977][20381] Updated weights for policy 1, policy_version 459684 (0.0010) [2023-12-27 08:49:24,022][20381] Updated weights for policy 1, policy_version 459694 (0.0010) [2023-12-27 08:49:24,592][20381] Updated weights for policy 1, policy_version 459704 (0.0006) [2023-12-27 08:49:24,619][20347] Updated weights for policy 0, policy_version 460138 (0.0007) [2023-12-27 08:49:24,651][20381] Updated weights for policy 1, policy_version 459714 (0.0005) [2023-12-27 08:49:24,675][20347] Updated weights for policy 0, policy_version 460148 (0.0007) [2023-12-27 08:49:24,706][20381] Updated weights for policy 1, policy_version 459724 (0.0005) [2023-12-27 08:49:24,731][20347] Updated weights for policy 0, policy_version 460158 (0.0007) [2023-12-27 08:49:25,314][20381] Updated weights for policy 1, policy_version 459734 (0.0007) [2023-12-27 08:49:25,359][20381] Updated weights for policy 1, policy_version 459744 (0.0008) [2023-12-27 08:49:25,395][20347] Updated weights for policy 0, policy_version 460168 (0.0007) [2023-12-27 08:49:25,411][20381] Updated weights for policy 1, policy_version 459754 (0.0006) [2023-12-27 08:49:25,440][20347] Updated weights for policy 0, policy_version 460178 (0.0005) [2023-12-27 08:49:25,490][20347] Updated weights for policy 0, policy_version 460188 (0.0006) [2023-12-27 08:49:26,045][20292] KL-divergence is very high: 107.0438 [2023-12-27 08:49:26,061][20381] Updated weights for policy 1, policy_version 459764 (0.0005) [2023-12-27 08:49:26,090][20292] KL-divergence is very high: 137.1809 [2023-12-27 08:49:26,119][20381] Updated weights for policy 1, policy_version 459774 (0.0006) [2023-12-27 08:49:26,170][20381] Updated weights for policy 1, policy_version 459784 (0.0008) [2023-12-27 08:49:26,221][20347] Updated weights for policy 0, policy_version 460198 (0.0007) [2023-12-27 08:49:26,275][20347] Updated weights for policy 0, policy_version 460208 (0.0007) [2023-12-27 08:49:26,327][20347] Updated weights for policy 0, policy_version 460218 (0.0007) [2023-12-27 08:49:26,823][20381] Updated weights for policy 1, policy_version 459794 (0.0007) [2023-12-27 08:49:26,884][20381] Updated weights for policy 1, policy_version 459804 (0.0007) [2023-12-27 08:49:26,945][20381] Updated weights for policy 1, policy_version 459814 (0.0007) [2023-12-27 08:49:27,001][20381] Updated weights for policy 1, policy_version 459824 (0.0007) [2023-12-27 08:49:27,010][20347] Updated weights for policy 0, policy_version 460228 (0.0007) [2023-12-27 08:49:27,064][20347] Updated weights for policy 0, policy_version 460238 (0.0007) [2023-12-27 08:49:27,113][20347] Updated weights for policy 0, policy_version 460248 (0.0007) [2023-12-27 08:49:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 235577344. Throughput: 0: 10468.8, 1: 10193.0. Samples: 235565728. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:27,363][19170] Avg episode reward: [(0, '391.770'), (1, '825.570')] [2023-12-27 08:49:27,685][20381] Updated weights for policy 1, policy_version 459834 (0.0007) [2023-12-27 08:49:27,726][20347] Updated weights for policy 0, policy_version 460258 (0.0008) [2023-12-27 08:49:27,743][20381] Updated weights for policy 1, policy_version 459844 (0.0005) [2023-12-27 08:49:27,783][20347] Updated weights for policy 0, policy_version 460268 (0.0009) [2023-12-27 08:49:27,798][20381] Updated weights for policy 1, policy_version 459854 (0.0005) [2023-12-27 08:49:27,836][20347] Updated weights for policy 0, policy_version 460278 (0.0007) [2023-12-27 08:49:27,881][20347] Updated weights for policy 0, policy_version 460288 (0.0009) [2023-12-27 08:49:28,411][20381] Updated weights for policy 1, policy_version 459864 (0.0007) [2023-12-27 08:49:28,461][20381] Updated weights for policy 1, policy_version 459874 (0.0006) [2023-12-27 08:49:28,517][20381] Updated weights for policy 1, policy_version 459884 (0.0007) [2023-12-27 08:49:28,546][20347] Updated weights for policy 0, policy_version 460298 (0.0005) [2023-12-27 08:49:28,602][20347] Updated weights for policy 0, policy_version 460308 (0.0008) [2023-12-27 08:49:28,660][20347] Updated weights for policy 0, policy_version 460318 (0.0006) [2023-12-27 08:49:29,234][20347] Updated weights for policy 0, policy_version 460328 (0.0007) [2023-12-27 08:49:29,260][20381] Updated weights for policy 1, policy_version 459894 (0.0007) [2023-12-27 08:49:29,294][20347] Updated weights for policy 0, policy_version 460338 (0.0006) [2023-12-27 08:49:29,320][20381] Updated weights for policy 1, policy_version 459904 (0.0007) [2023-12-27 08:49:29,354][20347] Updated weights for policy 0, policy_version 460348 (0.0006) [2023-12-27 08:49:29,379][20381] Updated weights for policy 1, policy_version 459914 (0.0006) [2023-12-27 08:49:30,016][20347] Updated weights for policy 0, policy_version 460358 (0.0007) [2023-12-27 08:49:30,072][20347] Updated weights for policy 0, policy_version 460368 (0.0006) [2023-12-27 08:49:30,080][20381] Updated weights for policy 1, policy_version 459924 (0.0008) [2023-12-27 08:49:30,132][20347] Updated weights for policy 0, policy_version 460378 (0.0007) [2023-12-27 08:49:30,136][20381] Updated weights for policy 1, policy_version 459934 (0.0006) [2023-12-27 08:49:30,195][20381] Updated weights for policy 1, policy_version 459944 (0.0006) [2023-12-27 08:49:30,771][20381] Updated weights for policy 1, policy_version 459954 (0.0006) [2023-12-27 08:49:30,825][20381] Updated weights for policy 1, policy_version 459964 (0.0007) [2023-12-27 08:49:30,869][20347] Updated weights for policy 0, policy_version 460388 (0.0008) [2023-12-27 08:49:30,883][20381] Updated weights for policy 1, policy_version 459974 (0.0008) [2023-12-27 08:49:30,917][20347] Updated weights for policy 0, policy_version 460398 (0.0008) [2023-12-27 08:49:30,940][20381] Updated weights for policy 1, policy_version 459984 (0.0007) [2023-12-27 08:49:30,967][20347] Updated weights for policy 0, policy_version 460408 (0.0007) [2023-12-27 08:49:31,610][20381] Updated weights for policy 1, policy_version 459994 (0.0007) [2023-12-27 08:49:31,654][20347] Updated weights for policy 0, policy_version 460418 (0.0007) [2023-12-27 08:49:31,668][20381] Updated weights for policy 1, policy_version 460004 (0.0010) [2023-12-27 08:49:31,715][20347] Updated weights for policy 0, policy_version 460428 (0.0006) [2023-12-27 08:49:31,735][20381] Updated weights for policy 1, policy_version 460014 (0.0012) [2023-12-27 08:49:31,779][20347] Updated weights for policy 0, policy_version 460438 (0.0006) [2023-12-27 08:49:31,841][20347] Updated weights for policy 0, policy_version 460448 (0.0006) [2023-12-27 08:49:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 235675648. Throughput: 0: 10434.2, 1: 10214.9. Samples: 235691080. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:32,363][19170] Avg episode reward: [(0, '401.170'), (1, '805.960')] [2023-12-27 08:49:32,375][20381] Updated weights for policy 1, policy_version 460024 (0.0009) [2023-12-27 08:49:32,430][20381] Updated weights for policy 1, policy_version 460034 (0.0009) [2023-12-27 08:49:32,472][20347] Updated weights for policy 0, policy_version 460458 (0.0007) [2023-12-27 08:49:32,486][20381] Updated weights for policy 1, policy_version 460044 (0.0007) [2023-12-27 08:49:32,522][20347] Updated weights for policy 0, policy_version 460468 (0.0007) [2023-12-27 08:49:32,582][20347] Updated weights for policy 0, policy_version 460478 (0.0008) [2023-12-27 08:49:33,156][20381] Updated weights for policy 1, policy_version 460054 (0.0009) [2023-12-27 08:49:33,205][20381] Updated weights for policy 1, policy_version 460064 (0.0010) [2023-12-27 08:49:33,214][20347] Updated weights for policy 0, policy_version 460488 (0.0006) [2023-12-27 08:49:33,257][20381] Updated weights for policy 1, policy_version 460074 (0.0011) [2023-12-27 08:49:33,269][20347] Updated weights for policy 0, policy_version 460498 (0.0005) [2023-12-27 08:49:33,320][20347] Updated weights for policy 0, policy_version 460508 (0.0005) [2023-12-27 08:49:33,829][20381] Updated weights for policy 1, policy_version 460084 (0.0009) [2023-12-27 08:49:33,882][20381] Updated weights for policy 1, policy_version 460094 (0.0007) [2023-12-27 08:49:33,927][20381] Updated weights for policy 1, policy_version 460104 (0.0010) [2023-12-27 08:49:33,949][20347] Updated weights for policy 0, policy_version 460518 (0.0005) [2023-12-27 08:49:34,002][20347] Updated weights for policy 0, policy_version 460528 (0.0005) [2023-12-27 08:49:34,056][20347] Updated weights for policy 0, policy_version 460538 (0.0005) [2023-12-27 08:49:34,580][20381] Updated weights for policy 1, policy_version 460114 (0.0009) [2023-12-27 08:49:34,636][20381] Updated weights for policy 1, policy_version 460124 (0.0008) [2023-12-27 08:49:34,687][20381] Updated weights for policy 1, policy_version 460134 (0.0007) [2023-12-27 08:49:34,734][20347] Updated weights for policy 0, policy_version 460548 (0.0007) [2023-12-27 08:49:34,744][20381] Updated weights for policy 1, policy_version 460144 (0.0006) [2023-12-27 08:49:34,786][20347] Updated weights for policy 0, policy_version 460558 (0.0007) [2023-12-27 08:49:34,836][20347] Updated weights for policy 0, policy_version 460568 (0.0008) [2023-12-27 08:49:35,442][20381] Updated weights for policy 1, policy_version 460154 (0.0007) [2023-12-27 08:49:35,486][20347] Updated weights for policy 0, policy_version 460578 (0.0006) [2023-12-27 08:49:35,499][20381] Updated weights for policy 1, policy_version 460164 (0.0007) [2023-12-27 08:49:35,538][20347] Updated weights for policy 0, policy_version 460588 (0.0006) [2023-12-27 08:49:35,551][20381] Updated weights for policy 1, policy_version 460174 (0.0006) [2023-12-27 08:49:35,592][20347] Updated weights for policy 0, policy_version 460598 (0.0006) [2023-12-27 08:49:35,639][20347] Updated weights for policy 0, policy_version 460608 (0.0009) [2023-12-27 08:49:36,236][20381] Updated weights for policy 1, policy_version 460184 (0.0007) [2023-12-27 08:49:36,289][20381] Updated weights for policy 1, policy_version 460194 (0.0007) [2023-12-27 08:49:36,346][20347] Updated weights for policy 0, policy_version 460618 (0.0007) [2023-12-27 08:49:36,347][20381] Updated weights for policy 1, policy_version 460204 (0.0008) [2023-12-27 08:49:36,401][20347] Updated weights for policy 0, policy_version 460628 (0.0007) [2023-12-27 08:49:36,452][20347] Updated weights for policy 0, policy_version 460638 (0.0008) [2023-12-27 08:49:36,995][20381] Updated weights for policy 1, policy_version 460214 (0.0005) [2023-12-27 08:49:37,055][20381] Updated weights for policy 1, policy_version 460224 (0.0007) [2023-12-27 08:49:37,117][20381] Updated weights for policy 1, policy_version 460234 (0.0005) [2023-12-27 08:49:37,204][20347] Updated weights for policy 0, policy_version 460648 (0.0007) [2023-12-27 08:49:37,263][20347] Updated weights for policy 0, policy_version 460658 (0.0008) [2023-12-27 08:49:37,321][20347] Updated weights for policy 0, policy_version 460668 (0.0008) [2023-12-27 08:49:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 235790336. Throughput: 0: 10396.3, 1: 10263.0. Samples: 235754608. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:37,363][19170] Avg episode reward: [(0, '433.290'), (1, '795.790')] [2023-12-27 08:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000460240_117841920.pth... [2023-12-27 08:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000460672_117948416.pth... [2023-12-27 08:49:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000459024_117530624.pth [2023-12-27 08:49:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000459456_117637120.pth [2023-12-27 08:49:37,702][20381] Updated weights for policy 1, policy_version 460244 (0.0006) [2023-12-27 08:49:37,761][20381] Updated weights for policy 1, policy_version 460254 (0.0008) [2023-12-27 08:49:37,821][20381] Updated weights for policy 1, policy_version 460264 (0.0007) [2023-12-27 08:49:38,035][20347] Updated weights for policy 0, policy_version 460678 (0.0007) [2023-12-27 08:49:38,080][20347] Updated weights for policy 0, policy_version 460688 (0.0007) [2023-12-27 08:49:38,140][20347] Updated weights for policy 0, policy_version 460698 (0.0007) [2023-12-27 08:49:38,491][20381] Updated weights for policy 1, policy_version 460274 (0.0007) [2023-12-27 08:49:38,553][20381] Updated weights for policy 1, policy_version 460284 (0.0007) [2023-12-27 08:49:38,610][20381] Updated weights for policy 1, policy_version 460294 (0.0008) [2023-12-27 08:49:38,677][20381] Updated weights for policy 1, policy_version 460304 (0.0007) [2023-12-27 08:49:38,837][20347] Updated weights for policy 0, policy_version 460708 (0.0008) [2023-12-27 08:49:38,902][20347] Updated weights for policy 0, policy_version 460718 (0.0008) [2023-12-27 08:49:38,961][20347] Updated weights for policy 0, policy_version 460728 (0.0008) [2023-12-27 08:49:39,284][20381] Updated weights for policy 1, policy_version 460314 (0.0008) [2023-12-27 08:49:39,350][20381] Updated weights for policy 1, policy_version 460324 (0.0011) [2023-12-27 08:49:39,415][20381] Updated weights for policy 1, policy_version 460334 (0.0011) [2023-12-27 08:49:39,596][20347] Updated weights for policy 0, policy_version 460738 (0.0008) [2023-12-27 08:49:39,660][20347] Updated weights for policy 0, policy_version 460748 (0.0007) [2023-12-27 08:49:39,721][20347] Updated weights for policy 0, policy_version 460758 (0.0007) [2023-12-27 08:49:39,782][20347] Updated weights for policy 0, policy_version 460768 (0.0008) [2023-12-27 08:49:40,085][20381] Updated weights for policy 1, policy_version 460344 (0.0009) [2023-12-27 08:49:40,145][20381] Updated weights for policy 1, policy_version 460354 (0.0008) [2023-12-27 08:49:40,206][20381] Updated weights for policy 1, policy_version 460364 (0.0008) [2023-12-27 08:49:40,438][20347] Updated weights for policy 0, policy_version 460778 (0.0008) [2023-12-27 08:49:40,495][20347] Updated weights for policy 0, policy_version 460788 (0.0008) [2023-12-27 08:49:40,552][20347] Updated weights for policy 0, policy_version 460798 (0.0008) [2023-12-27 08:49:40,866][20381] Updated weights for policy 1, policy_version 460374 (0.0007) [2023-12-27 08:49:40,917][20381] Updated weights for policy 1, policy_version 460384 (0.0007) [2023-12-27 08:49:40,980][20381] Updated weights for policy 1, policy_version 460394 (0.0006) [2023-12-27 08:49:41,224][20347] Updated weights for policy 0, policy_version 460808 (0.0007) [2023-12-27 08:49:41,285][20347] Updated weights for policy 0, policy_version 460818 (0.0008) [2023-12-27 08:49:41,343][20347] Updated weights for policy 0, policy_version 460828 (0.0008) [2023-12-27 08:49:41,732][20381] Updated weights for policy 1, policy_version 460404 (0.0008) [2023-12-27 08:49:41,797][20381] Updated weights for policy 1, policy_version 460414 (0.0008) [2023-12-27 08:49:41,857][20381] Updated weights for policy 1, policy_version 460424 (0.0008) [2023-12-27 08:49:42,074][20347] Updated weights for policy 0, policy_version 460838 (0.0008) [2023-12-27 08:49:42,131][20347] Updated weights for policy 0, policy_version 460848 (0.0008) [2023-12-27 08:49:42,181][20347] Updated weights for policy 0, policy_version 460858 (0.0007) [2023-12-27 08:49:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 235888640. Throughput: 0: 10359.4, 1: 10311.8. Samples: 235878156. Policy #0 lag: (min: 1.0, avg: 21.6, max: 33.0) [2023-12-27 08:49:42,363][19170] Avg episode reward: [(0, '487.500'), (1, '815.170')] [2023-12-27 08:49:42,474][20381] Updated weights for policy 1, policy_version 460434 (0.0008) [2023-12-27 08:49:42,528][20381] Updated weights for policy 1, policy_version 460444 (0.0005) [2023-12-27 08:49:42,583][20381] Updated weights for policy 1, policy_version 460454 (0.0006) [2023-12-27 08:49:42,641][20381] Updated weights for policy 1, policy_version 460464 (0.0008) [2023-12-27 08:49:42,924][20347] Updated weights for policy 0, policy_version 460868 (0.0007) [2023-12-27 08:49:42,981][20347] Updated weights for policy 0, policy_version 460878 (0.0007) [2023-12-27 08:49:43,032][20347] Updated weights for policy 0, policy_version 460888 (0.0007) [2023-12-27 08:49:43,288][20381] Updated weights for policy 1, policy_version 460474 (0.0007) [2023-12-27 08:49:43,335][20381] Updated weights for policy 1, policy_version 460484 (0.0010) [2023-12-27 08:49:43,390][20381] Updated weights for policy 1, policy_version 460494 (0.0009) [2023-12-27 08:49:43,747][20347] Updated weights for policy 0, policy_version 460898 (0.0007) [2023-12-27 08:49:43,812][20347] Updated weights for policy 0, policy_version 460908 (0.0005) [2023-12-27 08:49:43,870][20347] Updated weights for policy 0, policy_version 460918 (0.0005) [2023-12-27 08:49:43,929][20347] Updated weights for policy 0, policy_version 460928 (0.0005) [2023-12-27 08:49:44,009][20381] Updated weights for policy 1, policy_version 460504 (0.0011) [2023-12-27 08:49:44,060][20381] Updated weights for policy 1, policy_version 460514 (0.0008) [2023-12-27 08:49:44,117][20381] Updated weights for policy 1, policy_version 460524 (0.0006) [2023-12-27 08:49:44,512][20347] Updated weights for policy 0, policy_version 460938 (0.0010) [2023-12-27 08:49:44,578][20347] Updated weights for policy 0, policy_version 460948 (0.0010) [2023-12-27 08:49:44,640][20347] Updated weights for policy 0, policy_version 460958 (0.0010) [2023-12-27 08:49:44,761][20381] Updated weights for policy 1, policy_version 460534 (0.0008) [2023-12-27 08:49:44,820][20381] Updated weights for policy 1, policy_version 460544 (0.0009) [2023-12-27 08:49:44,873][20381] Updated weights for policy 1, policy_version 460554 (0.0009) [2023-12-27 08:49:45,305][20347] Updated weights for policy 0, policy_version 460968 (0.0010) [2023-12-27 08:49:45,351][20347] Updated weights for policy 0, policy_version 460978 (0.0010) [2023-12-27 08:49:45,397][20347] Updated weights for policy 0, policy_version 460988 (0.0010) [2023-12-27 08:49:45,509][20381] Updated weights for policy 1, policy_version 460564 (0.0006) [2023-12-27 08:49:45,565][20381] Updated weights for policy 1, policy_version 460574 (0.0005) [2023-12-27 08:49:45,617][20381] Updated weights for policy 1, policy_version 460584 (0.0005) [2023-12-27 08:49:46,075][20347] Updated weights for policy 0, policy_version 460998 (0.0008) [2023-12-27 08:49:46,125][20347] Updated weights for policy 0, policy_version 461008 (0.0006) [2023-12-27 08:49:46,170][20347] Updated weights for policy 0, policy_version 461018 (0.0009) [2023-12-27 08:49:46,217][20381] Updated weights for policy 1, policy_version 460594 (0.0005) [2023-12-27 08:49:46,264][20381] Updated weights for policy 1, policy_version 460604 (0.0010) [2023-12-27 08:49:46,314][20381] Updated weights for policy 1, policy_version 460615 (0.0009) [2023-12-27 08:49:46,884][20347] Updated weights for policy 0, policy_version 461028 (0.0008) [2023-12-27 08:49:46,938][20347] Updated weights for policy 0, policy_version 461038 (0.0005) [2023-12-27 08:49:46,945][20381] Updated weights for policy 1, policy_version 460625 (0.0006) [2023-12-27 08:49:46,987][20347] Updated weights for policy 0, policy_version 461048 (0.0007) [2023-12-27 08:49:47,003][20381] Updated weights for policy 1, policy_version 460635 (0.0007) [2023-12-27 08:49:47,062][20381] Updated weights for policy 1, policy_version 460645 (0.0008) [2023-12-27 08:49:47,119][20381] Updated weights for policy 1, policy_version 460655 (0.0007) [2023-12-27 08:49:47,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 235995136. Throughput: 0: 10347.0, 1: 10317.3. Samples: 236003076. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:49:47,364][19170] Avg episode reward: [(0, '403.970'), (1, '824.670')] [2023-12-27 08:49:47,612][20347] Updated weights for policy 0, policy_version 461058 (0.0007) [2023-12-27 08:49:47,675][20347] Updated weights for policy 0, policy_version 461068 (0.0005) [2023-12-27 08:49:47,736][20347] Updated weights for policy 0, policy_version 461078 (0.0007) [2023-12-27 08:49:47,766][20381] Updated weights for policy 1, policy_version 460665 (0.0005) [2023-12-27 08:49:47,790][20347] Updated weights for policy 0, policy_version 461088 (0.0006) [2023-12-27 08:49:47,815][20381] Updated weights for policy 1, policy_version 460675 (0.0010) [2023-12-27 08:49:47,863][20381] Updated weights for policy 1, policy_version 460685 (0.0010) [2023-12-27 08:49:48,385][20347] Updated weights for policy 0, policy_version 461098 (0.0008) [2023-12-27 08:49:48,440][20347] Updated weights for policy 0, policy_version 461108 (0.0007) [2023-12-27 08:49:48,506][20347] Updated weights for policy 0, policy_version 461118 (0.0007) [2023-12-27 08:49:48,545][20381] Updated weights for policy 1, policy_version 460695 (0.0007) [2023-12-27 08:49:48,603][20381] Updated weights for policy 1, policy_version 460705 (0.0005) [2023-12-27 08:49:48,664][20381] Updated weights for policy 1, policy_version 460715 (0.0006) [2023-12-27 08:49:49,208][20347] Updated weights for policy 0, policy_version 461128 (0.0008) [2023-12-27 08:49:49,271][20347] Updated weights for policy 0, policy_version 461138 (0.0008) [2023-12-27 08:49:49,309][20381] Updated weights for policy 1, policy_version 460725 (0.0007) [2023-12-27 08:49:49,327][20347] Updated weights for policy 0, policy_version 461148 (0.0006) [2023-12-27 08:49:49,371][20381] Updated weights for policy 1, policy_version 460735 (0.0008) [2023-12-27 08:49:49,427][20381] Updated weights for policy 1, policy_version 460745 (0.0008) [2023-12-27 08:49:50,032][20347] Updated weights for policy 0, policy_version 461158 (0.0008) [2023-12-27 08:49:50,080][20347] Updated weights for policy 0, policy_version 461168 (0.0009) [2023-12-27 08:49:50,110][20381] Updated weights for policy 1, policy_version 460755 (0.0007) [2023-12-27 08:49:50,138][20347] Updated weights for policy 0, policy_version 461178 (0.0007) [2023-12-27 08:49:50,173][20381] Updated weights for policy 1, policy_version 460765 (0.0007) [2023-12-27 08:49:50,227][20381] Updated weights for policy 1, policy_version 460775 (0.0007) [2023-12-27 08:49:50,823][20381] Updated weights for policy 1, policy_version 460785 (0.0008) [2023-12-27 08:49:50,868][20347] Updated weights for policy 0, policy_version 461188 (0.0007) [2023-12-27 08:49:50,877][20381] Updated weights for policy 1, policy_version 460795 (0.0008) [2023-12-27 08:49:50,921][20347] Updated weights for policy 0, policy_version 461198 (0.0005) [2023-12-27 08:49:50,930][20381] Updated weights for policy 1, policy_version 460805 (0.0007) [2023-12-27 08:49:50,974][20347] Updated weights for policy 0, policy_version 461208 (0.0006) [2023-12-27 08:49:50,989][20381] Updated weights for policy 1, policy_version 460815 (0.0008) [2023-12-27 08:49:51,736][20381] Updated weights for policy 1, policy_version 460825 (0.0008) [2023-12-27 08:49:51,767][20347] Updated weights for policy 0, policy_version 461218 (0.0009) [2023-12-27 08:49:51,794][20381] Updated weights for policy 1, policy_version 460835 (0.0007) [2023-12-27 08:49:51,827][20347] Updated weights for policy 0, policy_version 461228 (0.0006) [2023-12-27 08:49:51,845][20381] Updated weights for policy 1, policy_version 460845 (0.0006) [2023-12-27 08:49:51,880][20347] Updated weights for policy 0, policy_version 461238 (0.0008) [2023-12-27 08:49:51,933][20347] Updated weights for policy 0, policy_version 461248 (0.0008) [2023-12-27 08:49:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 236093440. Throughput: 0: 10338.4, 1: 10346.1. Samples: 236065748. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:49:52,363][19170] Avg episode reward: [(0, '553.870'), (1, '815.340')] [2023-12-27 08:49:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000460848_117997568.pth... [2023-12-27 08:49:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000461248_118095872.pth... [2023-12-27 08:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000459632_117686272.pth [2023-12-27 08:49:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000460064_117792768.pth [2023-12-27 08:49:52,526][20381] Updated weights for policy 1, policy_version 460855 (0.0008) [2023-12-27 08:49:52,580][20381] Updated weights for policy 1, policy_version 460865 (0.0007) [2023-12-27 08:49:52,599][20347] Updated weights for policy 0, policy_version 461258 (0.0006) [2023-12-27 08:49:52,631][20381] Updated weights for policy 1, policy_version 460875 (0.0007) [2023-12-27 08:49:52,661][20347] Updated weights for policy 0, policy_version 461268 (0.0008) [2023-12-27 08:49:52,717][20347] Updated weights for policy 0, policy_version 461278 (0.0009) [2023-12-27 08:49:53,243][20381] Updated weights for policy 1, policy_version 460885 (0.0008) [2023-12-27 08:49:53,299][20381] Updated weights for policy 1, policy_version 460895 (0.0008) [2023-12-27 08:49:53,358][20381] Updated weights for policy 1, policy_version 460905 (0.0007) [2023-12-27 08:49:53,389][20347] Updated weights for policy 0, policy_version 461288 (0.0008) [2023-12-27 08:49:53,446][20347] Updated weights for policy 0, policy_version 461298 (0.0006) [2023-12-27 08:49:53,502][20347] Updated weights for policy 0, policy_version 461308 (0.0006) [2023-12-27 08:49:54,055][20381] Updated weights for policy 1, policy_version 460915 (0.0007) [2023-12-27 08:49:54,119][20381] Updated weights for policy 1, policy_version 460925 (0.0009) [2023-12-27 08:49:54,179][20381] Updated weights for policy 1, policy_version 460935 (0.0010) [2023-12-27 08:49:54,213][20347] Updated weights for policy 0, policy_version 461318 (0.0007) [2023-12-27 08:49:54,275][20347] Updated weights for policy 0, policy_version 461328 (0.0007) [2023-12-27 08:49:54,335][20347] Updated weights for policy 0, policy_version 461338 (0.0008) [2023-12-27 08:49:54,745][20381] Updated weights for policy 1, policy_version 460945 (0.0008) [2023-12-27 08:49:54,802][20381] Updated weights for policy 1, policy_version 460955 (0.0005) [2023-12-27 08:49:54,855][20381] Updated weights for policy 1, policy_version 460965 (0.0005) [2023-12-27 08:49:54,899][20381] Updated weights for policy 1, policy_version 460975 (0.0005) [2023-12-27 08:49:54,994][20347] Updated weights for policy 0, policy_version 461348 (0.0005) [2023-12-27 08:49:55,052][20347] Updated weights for policy 0, policy_version 461358 (0.0005) [2023-12-27 08:49:55,111][20347] Updated weights for policy 0, policy_version 461368 (0.0005) [2023-12-27 08:49:55,498][20381] Updated weights for policy 1, policy_version 460985 (0.0010) [2023-12-27 08:49:55,555][20381] Updated weights for policy 1, policy_version 460995 (0.0010) [2023-12-27 08:49:55,601][20381] Updated weights for policy 1, policy_version 461005 (0.0007) [2023-12-27 08:49:55,658][20347] Updated weights for policy 0, policy_version 461378 (0.0006) [2023-12-27 08:49:55,710][20347] Updated weights for policy 0, policy_version 461388 (0.0006) [2023-12-27 08:49:55,764][20347] Updated weights for policy 0, policy_version 461398 (0.0006) [2023-12-27 08:49:55,820][20347] Updated weights for policy 0, policy_version 461408 (0.0005) [2023-12-27 08:49:56,293][20381] Updated weights for policy 1, policy_version 461015 (0.0009) [2023-12-27 08:49:56,343][20347] Updated weights for policy 0, policy_version 461418 (0.0007) [2023-12-27 08:49:56,348][20381] Updated weights for policy 1, policy_version 461025 (0.0010) [2023-12-27 08:49:56,398][20347] Updated weights for policy 0, policy_version 461428 (0.0007) [2023-12-27 08:49:56,403][20381] Updated weights for policy 1, policy_version 461035 (0.0010) [2023-12-27 08:49:56,448][20347] Updated weights for policy 0, policy_version 461438 (0.0005) [2023-12-27 08:49:57,076][20381] Updated weights for policy 1, policy_version 461045 (0.0010) [2023-12-27 08:49:57,102][20347] Updated weights for policy 0, policy_version 461448 (0.0006) [2023-12-27 08:49:57,127][20381] Updated weights for policy 1, policy_version 461055 (0.0010) [2023-12-27 08:49:57,158][20347] Updated weights for policy 0, policy_version 461458 (0.0005) [2023-12-27 08:49:57,182][20381] Updated weights for policy 1, policy_version 461065 (0.0010) [2023-12-27 08:49:57,212][20347] Updated weights for policy 0, policy_version 461468 (0.0005) [2023-12-27 08:49:57,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 236208128. Throughput: 0: 10337.2, 1: 10470.1. Samples: 236192376. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:49:57,363][19170] Avg episode reward: [(0, '722.110'), (1, '728.370')] [2023-12-27 08:49:57,840][20381] Updated weights for policy 1, policy_version 461075 (0.0011) [2023-12-27 08:49:57,851][20347] Updated weights for policy 0, policy_version 461478 (0.0005) [2023-12-27 08:49:57,894][20381] Updated weights for policy 1, policy_version 461085 (0.0010) [2023-12-27 08:49:57,904][20347] Updated weights for policy 0, policy_version 461488 (0.0009) [2023-12-27 08:49:57,945][20381] Updated weights for policy 1, policy_version 461095 (0.0010) [2023-12-27 08:49:57,958][20347] Updated weights for policy 0, policy_version 461498 (0.0010) [2023-12-27 08:49:58,640][20347] Updated weights for policy 0, policy_version 461508 (0.0010) [2023-12-27 08:49:58,662][20381] Updated weights for policy 1, policy_version 461105 (0.0010) [2023-12-27 08:49:58,701][20347] Updated weights for policy 0, policy_version 461518 (0.0010) [2023-12-27 08:49:58,721][20381] Updated weights for policy 1, policy_version 461115 (0.0010) [2023-12-27 08:49:58,760][20347] Updated weights for policy 0, policy_version 461528 (0.0009) [2023-12-27 08:49:58,787][20381] Updated weights for policy 1, policy_version 461125 (0.0009) [2023-12-27 08:49:58,846][20381] Updated weights for policy 1, policy_version 461135 (0.0011) [2023-12-27 08:49:59,451][20347] Updated weights for policy 0, policy_version 461538 (0.0007) [2023-12-27 08:49:59,512][20381] Updated weights for policy 1, policy_version 461145 (0.0006) [2023-12-27 08:49:59,515][20347] Updated weights for policy 0, policy_version 461548 (0.0005) [2023-12-27 08:49:59,569][20381] Updated weights for policy 1, policy_version 461155 (0.0007) [2023-12-27 08:49:59,576][20347] Updated weights for policy 0, policy_version 461558 (0.0005) [2023-12-27 08:49:59,626][20381] Updated weights for policy 1, policy_version 461165 (0.0007) [2023-12-27 08:49:59,640][20347] Updated weights for policy 0, policy_version 461568 (0.0007) [2023-12-27 08:50:00,238][20381] Updated weights for policy 1, policy_version 461175 (0.0007) [2023-12-27 08:50:00,296][20381] Updated weights for policy 1, policy_version 461185 (0.0006) [2023-12-27 08:50:00,325][20347] Updated weights for policy 0, policy_version 461578 (0.0008) [2023-12-27 08:50:00,352][20381] Updated weights for policy 1, policy_version 461195 (0.0008) [2023-12-27 08:50:00,389][20347] Updated weights for policy 0, policy_version 461588 (0.0007) [2023-12-27 08:50:00,453][20347] Updated weights for policy 0, policy_version 461598 (0.0008) [2023-12-27 08:50:01,000][20347] Updated weights for policy 0, policy_version 461609 (0.0008) [2023-12-27 08:50:01,053][20347] Updated weights for policy 0, policy_version 461619 (0.0007) [2023-12-27 08:50:01,097][20381] Updated weights for policy 1, policy_version 461205 (0.0007) [2023-12-27 08:50:01,115][20347] Updated weights for policy 0, policy_version 461629 (0.0005) [2023-12-27 08:50:01,151][20381] Updated weights for policy 1, policy_version 461215 (0.0007) [2023-12-27 08:50:01,214][20381] Updated weights for policy 1, policy_version 461225 (0.0006) [2023-12-27 08:50:01,829][20347] Updated weights for policy 0, policy_version 461639 (0.0007) [2023-12-27 08:50:01,888][20381] Updated weights for policy 1, policy_version 461235 (0.0006) [2023-12-27 08:50:01,890][20347] Updated weights for policy 0, policy_version 461649 (0.0008) [2023-12-27 08:50:01,939][20381] Updated weights for policy 1, policy_version 461245 (0.0007) [2023-12-27 08:50:01,943][20292] KL-divergence is very high: 121.5457 [2023-12-27 08:50:01,944][20347] Updated weights for policy 0, policy_version 461659 (0.0008) [2023-12-27 08:50:01,996][20381] Updated weights for policy 1, policy_version 461255 (0.0008) [2023-12-27 08:50:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 236306432. Throughput: 0: 10340.5, 1: 10459.5. Samples: 236315856. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:02,363][19170] Avg episode reward: [(0, '563.220'), (1, '749.860')] [2023-12-27 08:50:02,610][20347] Updated weights for policy 0, policy_version 461669 (0.0007) [2023-12-27 08:50:02,668][20347] Updated weights for policy 0, policy_version 461679 (0.0008) [2023-12-27 08:50:02,729][20347] Updated weights for policy 0, policy_version 461689 (0.0011) [2023-12-27 08:50:02,735][20381] Updated weights for policy 1, policy_version 461265 (0.0007) [2023-12-27 08:50:02,790][20381] Updated weights for policy 1, policy_version 461275 (0.0008) [2023-12-27 08:50:02,849][20381] Updated weights for policy 1, policy_version 461285 (0.0008) [2023-12-27 08:50:02,910][20381] Updated weights for policy 1, policy_version 461295 (0.0007) [2023-12-27 08:50:03,309][20347] Updated weights for policy 0, policy_version 461699 (0.0009) [2023-12-27 08:50:03,366][20347] Updated weights for policy 0, policy_version 461709 (0.0010) [2023-12-27 08:50:03,410][20347] Updated weights for policy 0, policy_version 461719 (0.0010) [2023-12-27 08:50:03,622][20381] Updated weights for policy 1, policy_version 461305 (0.0008) [2023-12-27 08:50:03,672][20381] Updated weights for policy 1, policy_version 461315 (0.0007) [2023-12-27 08:50:03,723][20381] Updated weights for policy 1, policy_version 461325 (0.0007) [2023-12-27 08:50:04,056][20347] Updated weights for policy 0, policy_version 461729 (0.0006) [2023-12-27 08:50:04,121][20347] Updated weights for policy 0, policy_version 461739 (0.0008) [2023-12-27 08:50:04,183][20347] Updated weights for policy 0, policy_version 461749 (0.0006) [2023-12-27 08:50:04,248][20347] Updated weights for policy 0, policy_version 461759 (0.0007) [2023-12-27 08:50:04,444][20381] Updated weights for policy 1, policy_version 461335 (0.0008) [2023-12-27 08:50:04,497][20381] Updated weights for policy 1, policy_version 461345 (0.0007) [2023-12-27 08:50:04,545][20381] Updated weights for policy 1, policy_version 461355 (0.0007) [2023-12-27 08:50:04,886][20347] Updated weights for policy 0, policy_version 461769 (0.0010) [2023-12-27 08:50:04,935][20347] Updated weights for policy 0, policy_version 461779 (0.0010) [2023-12-27 08:50:04,982][20347] Updated weights for policy 0, policy_version 461789 (0.0010) [2023-12-27 08:50:05,166][20381] Updated weights for policy 1, policy_version 461365 (0.0007) [2023-12-27 08:50:05,221][20381] Updated weights for policy 1, policy_version 461375 (0.0008) [2023-12-27 08:50:05,277][20381] Updated weights for policy 1, policy_version 461385 (0.0009) [2023-12-27 08:50:05,569][20347] Updated weights for policy 0, policy_version 461799 (0.0008) [2023-12-27 08:50:05,623][20347] Updated weights for policy 0, policy_version 461809 (0.0007) [2023-12-27 08:50:05,685][20347] Updated weights for policy 0, policy_version 461819 (0.0005) [2023-12-27 08:50:06,025][20381] Updated weights for policy 1, policy_version 461395 (0.0008) [2023-12-27 08:50:06,079][20381] Updated weights for policy 1, policy_version 461405 (0.0007) [2023-12-27 08:50:06,138][20381] Updated weights for policy 1, policy_version 461415 (0.0008) [2023-12-27 08:50:06,328][20347] Updated weights for policy 0, policy_version 461829 (0.0007) [2023-12-27 08:50:06,391][20347] Updated weights for policy 0, policy_version 461839 (0.0008) [2023-12-27 08:50:06,458][20347] Updated weights for policy 0, policy_version 461849 (0.0008) [2023-12-27 08:50:06,868][20381] Updated weights for policy 1, policy_version 461425 (0.0008) [2023-12-27 08:50:06,932][20381] Updated weights for policy 1, policy_version 461435 (0.0007) [2023-12-27 08:50:06,986][20381] Updated weights for policy 1, policy_version 461445 (0.0006) [2023-12-27 08:50:07,053][20381] Updated weights for policy 1, policy_version 461455 (0.0005) [2023-12-27 08:50:07,129][20347] Updated weights for policy 0, policy_version 461859 (0.0008) [2023-12-27 08:50:07,189][20347] Updated weights for policy 0, policy_version 461869 (0.0007) [2023-12-27 08:50:07,252][20347] Updated weights for policy 0, policy_version 461879 (0.0005) [2023-12-27 08:50:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 236412928. Throughput: 0: 10371.5, 1: 10447.7. Samples: 236378596. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:07,363][19170] Avg episode reward: [(0, '498.000'), (1, '875.520')] [2023-12-27 08:50:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000461888_118259712.pth... [2023-12-27 08:50:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000461456_118153216.pth... [2023-12-27 08:50:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000460672_117948416.pth [2023-12-27 08:50:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000460240_117841920.pth [2023-12-27 08:50:07,664][20381] Updated weights for policy 1, policy_version 461465 (0.0006) [2023-12-27 08:50:07,718][20381] Updated weights for policy 1, policy_version 461475 (0.0006) [2023-12-27 08:50:07,771][20381] Updated weights for policy 1, policy_version 461485 (0.0008) [2023-12-27 08:50:07,808][20347] Updated weights for policy 0, policy_version 461889 (0.0007) [2023-12-27 08:50:07,859][20347] Updated weights for policy 0, policy_version 461899 (0.0008) [2023-12-27 08:50:07,910][20347] Updated weights for policy 0, policy_version 461909 (0.0007) [2023-12-27 08:50:07,957][20347] Updated weights for policy 0, policy_version 461919 (0.0007) [2023-12-27 08:50:08,373][20381] Updated weights for policy 1, policy_version 461495 (0.0008) [2023-12-27 08:50:08,433][20381] Updated weights for policy 1, policy_version 461505 (0.0009) [2023-12-27 08:50:08,501][20381] Updated weights for policy 1, policy_version 461515 (0.0008) [2023-12-27 08:50:08,657][20347] Updated weights for policy 0, policy_version 461929 (0.0008) [2023-12-27 08:50:08,722][20347] Updated weights for policy 0, policy_version 461939 (0.0010) [2023-12-27 08:50:08,788][20347] Updated weights for policy 0, policy_version 461949 (0.0008) [2023-12-27 08:50:09,176][20381] Updated weights for policy 1, policy_version 461525 (0.0008) [2023-12-27 08:50:09,235][20292] KL-divergence is very high: 409.4260 [2023-12-27 08:50:09,235][20381] Updated weights for policy 1, policy_version 461535 (0.0008) [2023-12-27 08:50:09,280][20292] KL-divergence is very high: 133.3361 [2023-12-27 08:50:09,291][20381] Updated weights for policy 1, policy_version 461545 (0.0007) [2023-12-27 08:50:09,447][20347] Updated weights for policy 0, policy_version 461959 (0.0007) [2023-12-27 08:50:09,508][20347] Updated weights for policy 0, policy_version 461969 (0.0007) [2023-12-27 08:50:09,569][20347] Updated weights for policy 0, policy_version 461979 (0.0008) [2023-12-27 08:50:10,007][20381] Updated weights for policy 1, policy_version 461555 (0.0008) [2023-12-27 08:50:10,068][20381] Updated weights for policy 1, policy_version 461565 (0.0008) [2023-12-27 08:50:10,133][20381] Updated weights for policy 1, policy_version 461575 (0.0008) [2023-12-27 08:50:10,262][20347] Updated weights for policy 0, policy_version 461989 (0.0006) [2023-12-27 08:50:10,317][20347] Updated weights for policy 0, policy_version 461999 (0.0007) [2023-12-27 08:50:10,384][20347] Updated weights for policy 0, policy_version 462009 (0.0006) [2023-12-27 08:50:10,827][20381] Updated weights for policy 1, policy_version 461585 (0.0008) [2023-12-27 08:50:10,882][20381] Updated weights for policy 1, policy_version 461595 (0.0006) [2023-12-27 08:50:10,938][20381] Updated weights for policy 1, policy_version 461605 (0.0010) [2023-12-27 08:50:11,001][20381] Updated weights for policy 1, policy_version 461615 (0.0011) [2023-12-27 08:50:11,081][20347] Updated weights for policy 0, policy_version 462019 (0.0008) [2023-12-27 08:50:11,139][20347] Updated weights for policy 0, policy_version 462029 (0.0007) [2023-12-27 08:50:11,198][20347] Updated weights for policy 0, policy_version 462039 (0.0007) [2023-12-27 08:50:11,776][20381] Updated weights for policy 1, policy_version 461625 (0.0009) [2023-12-27 08:50:11,837][20381] Updated weights for policy 1, policy_version 461635 (0.0008) [2023-12-27 08:50:11,893][20381] Updated weights for policy 1, policy_version 461645 (0.0008) [2023-12-27 08:50:11,963][20347] Updated weights for policy 0, policy_version 462049 (0.0007) [2023-12-27 08:50:12,021][20347] Updated weights for policy 0, policy_version 462059 (0.0007) [2023-12-27 08:50:12,067][20347] Updated weights for policy 0, policy_version 462069 (0.0008) [2023-12-27 08:50:12,120][20347] Updated weights for policy 0, policy_version 462079 (0.0007) [2023-12-27 08:50:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 236511232. Throughput: 0: 10405.6, 1: 10399.4. Samples: 236501956. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:12,363][19170] Avg episode reward: [(0, '520.070'), (1, '884.340')] [2023-12-27 08:50:12,625][20381] Updated weights for policy 1, policy_version 461655 (0.0010) [2023-12-27 08:50:12,657][20292] KL-divergence is very high: 153.3073 [2023-12-27 08:50:12,680][20381] Updated weights for policy 1, policy_version 461665 (0.0009) [2023-12-27 08:50:12,686][20292] KL-divergence is very high: 556.2742 [2023-12-27 08:50:12,705][20292] KL-divergence is very high: 286.8571 [2023-12-27 08:50:12,738][20381] Updated weights for policy 1, policy_version 461675 (0.0008) [2023-12-27 08:50:12,751][20292] KL-divergence is very high: 298.9727 [2023-12-27 08:50:12,786][20347] Updated weights for policy 0, policy_version 462089 (0.0006) [2023-12-27 08:50:12,847][20347] Updated weights for policy 0, policy_version 462099 (0.0008) [2023-12-27 08:50:12,909][20347] Updated weights for policy 0, policy_version 462109 (0.0006) [2023-12-27 08:50:13,504][20347] Updated weights for policy 0, policy_version 462119 (0.0006) [2023-12-27 08:50:13,506][20381] Updated weights for policy 1, policy_version 461685 (0.0007) [2023-12-27 08:50:13,554][20347] Updated weights for policy 0, policy_version 462129 (0.0006) [2023-12-27 08:50:13,556][20381] Updated weights for policy 1, policy_version 461695 (0.0006) [2023-12-27 08:50:13,601][20347] Updated weights for policy 0, policy_version 462139 (0.0005) [2023-12-27 08:50:13,606][20381] Updated weights for policy 1, policy_version 461705 (0.0006) [2023-12-27 08:50:14,253][20347] Updated weights for policy 0, policy_version 462149 (0.0006) [2023-12-27 08:50:14,306][20381] Updated weights for policy 1, policy_version 461715 (0.0009) [2023-12-27 08:50:14,309][20347] Updated weights for policy 0, policy_version 462159 (0.0007) [2023-12-27 08:50:14,360][20381] Updated weights for policy 1, policy_version 461725 (0.0006) [2023-12-27 08:50:14,368][20347] Updated weights for policy 0, policy_version 462169 (0.0008) [2023-12-27 08:50:14,416][20381] Updated weights for policy 1, policy_version 461735 (0.0006) [2023-12-27 08:50:15,036][20347] Updated weights for policy 0, policy_version 462179 (0.0006) [2023-12-27 08:50:15,098][20347] Updated weights for policy 0, policy_version 462189 (0.0008) [2023-12-27 08:50:15,144][20381] Updated weights for policy 1, policy_version 461745 (0.0007) [2023-12-27 08:50:15,154][20347] Updated weights for policy 0, policy_version 462199 (0.0007) [2023-12-27 08:50:15,202][20381] Updated weights for policy 1, policy_version 461755 (0.0006) [2023-12-27 08:50:15,252][20381] Updated weights for policy 1, policy_version 461765 (0.0008) [2023-12-27 08:50:15,312][20381] Updated weights for policy 1, policy_version 461775 (0.0008) [2023-12-27 08:50:15,834][20347] Updated weights for policy 0, policy_version 462209 (0.0007) [2023-12-27 08:50:15,882][20347] Updated weights for policy 0, policy_version 462219 (0.0008) [2023-12-27 08:50:15,926][20347] Updated weights for policy 0, policy_version 462229 (0.0007) [2023-12-27 08:50:15,986][20381] Updated weights for policy 1, policy_version 461785 (0.0006) [2023-12-27 08:50:15,987][20347] Updated weights for policy 0, policy_version 462239 (0.0006) [2023-12-27 08:50:16,039][20381] Updated weights for policy 1, policy_version 461795 (0.0007) [2023-12-27 08:50:16,103][20381] Updated weights for policy 1, policy_version 461805 (0.0008) [2023-12-27 08:50:16,672][20347] Updated weights for policy 0, policy_version 462249 (0.0007) [2023-12-27 08:50:16,723][20347] Updated weights for policy 0, policy_version 462259 (0.0007) [2023-12-27 08:50:16,769][20381] Updated weights for policy 1, policy_version 461815 (0.0009) [2023-12-27 08:50:16,774][20347] Updated weights for policy 0, policy_version 462269 (0.0005) [2023-12-27 08:50:16,826][20381] Updated weights for policy 1, policy_version 461825 (0.0009) [2023-12-27 08:50:16,872][20381] Updated weights for policy 1, policy_version 461835 (0.0005) [2023-12-27 08:50:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 236609536. Throughput: 0: 10375.1, 1: 10331.6. Samples: 236622880. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:17,363][19170] Avg episode reward: [(0, '699.860'), (1, '922.680')] [2023-12-27 08:50:17,452][20347] Updated weights for policy 0, policy_version 462279 (0.0007) [2023-12-27 08:50:17,503][20347] Updated weights for policy 0, policy_version 462289 (0.0007) [2023-12-27 08:50:17,515][20381] Updated weights for policy 1, policy_version 461845 (0.0008) [2023-12-27 08:50:17,554][20347] Updated weights for policy 0, policy_version 462299 (0.0005) [2023-12-27 08:50:17,567][20381] Updated weights for policy 1, policy_version 461855 (0.0009) [2023-12-27 08:50:17,618][20381] Updated weights for policy 1, policy_version 461865 (0.0010) [2023-12-27 08:50:18,239][20347] Updated weights for policy 0, policy_version 462309 (0.0006) [2023-12-27 08:50:18,251][20381] Updated weights for policy 1, policy_version 461875 (0.0009) [2023-12-27 08:50:18,292][20347] Updated weights for policy 0, policy_version 462319 (0.0006) [2023-12-27 08:50:18,307][20381] Updated weights for policy 1, policy_version 461885 (0.0008) [2023-12-27 08:50:18,350][20347] Updated weights for policy 0, policy_version 462329 (0.0008) [2023-12-27 08:50:18,369][20381] Updated weights for policy 1, policy_version 461895 (0.0009) [2023-12-27 08:50:19,069][20381] Updated weights for policy 1, policy_version 461905 (0.0011) [2023-12-27 08:50:19,087][20347] Updated weights for policy 0, policy_version 462339 (0.0006) [2023-12-27 08:50:19,125][20381] Updated weights for policy 1, policy_version 461915 (0.0010) [2023-12-27 08:50:19,148][20347] Updated weights for policy 0, policy_version 462349 (0.0006) [2023-12-27 08:50:19,171][20381] Updated weights for policy 1, policy_version 461925 (0.0008) [2023-12-27 08:50:19,206][20347] Updated weights for policy 0, policy_version 462359 (0.0010) [2023-12-27 08:50:19,229][20381] Updated weights for policy 1, policy_version 461935 (0.0006) [2023-12-27 08:50:19,859][20381] Updated weights for policy 1, policy_version 461945 (0.0008) [2023-12-27 08:50:19,881][20347] Updated weights for policy 0, policy_version 462369 (0.0009) [2023-12-27 08:50:19,887][20292] KL-divergence is very high: 220.2143 [2023-12-27 08:50:19,899][20292] KL-divergence is very high: 221.8660 [2023-12-27 08:50:19,927][20381] Updated weights for policy 1, policy_version 461955 (0.0007) [2023-12-27 08:50:19,948][20347] Updated weights for policy 0, policy_version 462379 (0.0009) [2023-12-27 08:50:19,974][20292] KL-divergence is very high: 144.8761 [2023-12-27 08:50:19,987][20381] Updated weights for policy 1, policy_version 461965 (0.0007) [2023-12-27 08:50:20,005][20347] Updated weights for policy 0, policy_version 462389 (0.0011) [2023-12-27 08:50:20,066][20347] Updated weights for policy 0, policy_version 462399 (0.0011) [2023-12-27 08:50:20,614][20381] Updated weights for policy 1, policy_version 461975 (0.0007) [2023-12-27 08:50:20,665][20381] Updated weights for policy 1, policy_version 461985 (0.0008) [2023-12-27 08:50:20,715][20347] Updated weights for policy 0, policy_version 462409 (0.0009) [2023-12-27 08:50:20,722][20381] Updated weights for policy 1, policy_version 461995 (0.0008) [2023-12-27 08:50:20,781][20347] Updated weights for policy 0, policy_version 462419 (0.0005) [2023-12-27 08:50:20,845][20347] Updated weights for policy 0, policy_version 462429 (0.0010) [2023-12-27 08:50:21,428][20381] Updated weights for policy 1, policy_version 462005 (0.0008) [2023-12-27 08:50:21,480][20381] Updated weights for policy 1, policy_version 462015 (0.0007) [2023-12-27 08:50:21,525][20347] Updated weights for policy 0, policy_version 462439 (0.0008) [2023-12-27 08:50:21,545][20381] Updated weights for policy 1, policy_version 462025 (0.0007) [2023-12-27 08:50:21,585][20347] Updated weights for policy 0, policy_version 462449 (0.0006) [2023-12-27 08:50:21,650][20347] Updated weights for policy 0, policy_version 462459 (0.0008) [2023-12-27 08:50:22,196][20381] Updated weights for policy 1, policy_version 462035 (0.0006) [2023-12-27 08:50:22,259][20381] Updated weights for policy 1, policy_version 462045 (0.0006) [2023-12-27 08:50:22,298][20347] Updated weights for policy 0, policy_version 462469 (0.0006) [2023-12-27 08:50:22,319][20381] Updated weights for policy 1, policy_version 462055 (0.0007) [2023-12-27 08:50:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 236707840. Throughput: 0: 10354.1, 1: 10321.0. Samples: 236684988. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:22,363][19170] Avg episode reward: [(0, '824.160'), (1, '912.850')] [2023-12-27 08:50:22,364][20347] Updated weights for policy 0, policy_version 462479 (0.0007) [2023-12-27 08:50:22,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000462064_118308864.pth... [2023-12-27 08:50:22,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000460848_117997568.pth [2023-12-27 08:50:22,415][20347] Updated weights for policy 0, policy_version 462489 (0.0006) [2023-12-27 08:50:22,451][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000462496_118415360.pth... [2023-12-27 08:50:22,455][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000461248_118095872.pth [2023-12-27 08:50:22,958][20381] Updated weights for policy 1, policy_version 462065 (0.0008) [2023-12-27 08:50:23,017][20381] Updated weights for policy 1, policy_version 462075 (0.0008) [2023-12-27 08:50:23,081][20347] Updated weights for policy 0, policy_version 462499 (0.0006) [2023-12-27 08:50:23,083][20381] Updated weights for policy 1, policy_version 462085 (0.0008) [2023-12-27 08:50:23,133][20347] Updated weights for policy 0, policy_version 462509 (0.0005) [2023-12-27 08:50:23,133][20381] Updated weights for policy 1, policy_version 462095 (0.0008) [2023-12-27 08:50:23,180][20347] Updated weights for policy 0, policy_version 462519 (0.0006) [2023-12-27 08:50:23,725][20381] Updated weights for policy 1, policy_version 462105 (0.0008) [2023-12-27 08:50:23,777][20381] Updated weights for policy 1, policy_version 462115 (0.0009) [2023-12-27 08:50:23,795][20347] Updated weights for policy 0, policy_version 462529 (0.0008) [2023-12-27 08:50:23,830][20381] Updated weights for policy 1, policy_version 462125 (0.0006) [2023-12-27 08:50:23,847][20347] Updated weights for policy 0, policy_version 462539 (0.0010) [2023-12-27 08:50:23,901][20347] Updated weights for policy 0, policy_version 462549 (0.0010) [2023-12-27 08:50:23,959][20347] Updated weights for policy 0, policy_version 462559 (0.0008) [2023-12-27 08:50:24,508][20381] Updated weights for policy 1, policy_version 462135 (0.0006) [2023-12-27 08:50:24,524][20292] KL-divergence is very high: 119.7104 [2023-12-27 08:50:24,555][20292] KL-divergence is very high: 403.3813 [2023-12-27 08:50:24,568][20381] Updated weights for policy 1, policy_version 462145 (0.0008) [2023-12-27 08:50:24,573][20292] KL-divergence is very high: 324.3925 [2023-12-27 08:50:24,601][20292] KL-divergence is very high: 136.7077 [2023-12-27 08:50:24,621][20381] Updated weights for policy 1, policy_version 462155 (0.0006) [2023-12-27 08:50:24,664][20347] Updated weights for policy 0, policy_version 462569 (0.0011) [2023-12-27 08:50:24,720][20347] Updated weights for policy 0, policy_version 462579 (0.0008) [2023-12-27 08:50:24,769][20347] Updated weights for policy 0, policy_version 462589 (0.0006) [2023-12-27 08:50:25,242][20381] Updated weights for policy 1, policy_version 462165 (0.0006) [2023-12-27 08:50:25,292][20381] Updated weights for policy 1, policy_version 462175 (0.0007) [2023-12-27 08:50:25,339][20381] Updated weights for policy 1, policy_version 462185 (0.0007) [2023-12-27 08:50:25,397][20347] Updated weights for policy 0, policy_version 462599 (0.0008) [2023-12-27 08:50:25,458][20347] Updated weights for policy 0, policy_version 462609 (0.0011) [2023-12-27 08:50:25,516][20347] Updated weights for policy 0, policy_version 462619 (0.0010) [2023-12-27 08:50:26,053][20381] Updated weights for policy 1, policy_version 462195 (0.0007) [2023-12-27 08:50:26,087][20292] KL-divergence is very high: 193.2923 [2023-12-27 08:50:26,092][20292] KL-divergence is very high: 223.5239 [2023-12-27 08:50:26,115][20381] Updated weights for policy 1, policy_version 462205 (0.0007) [2023-12-27 08:50:26,128][20347] Updated weights for policy 0, policy_version 462629 (0.0008) [2023-12-27 08:50:26,133][20292] KL-divergence is very high: 151.5881 [2023-12-27 08:50:26,138][20292] KL-divergence is very high: 137.9573 [2023-12-27 08:50:26,168][20381] Updated weights for policy 1, policy_version 462215 (0.0005) [2023-12-27 08:50:26,181][20347] Updated weights for policy 0, policy_version 462639 (0.0006) [2023-12-27 08:50:26,230][20347] Updated weights for policy 0, policy_version 462649 (0.0006) [2023-12-27 08:50:26,848][20347] Updated weights for policy 0, policy_version 462659 (0.0007) [2023-12-27 08:50:26,885][20381] Updated weights for policy 1, policy_version 462225 (0.0005) [2023-12-27 08:50:26,905][20347] Updated weights for policy 0, policy_version 462669 (0.0007) [2023-12-27 08:50:26,961][20347] Updated weights for policy 0, policy_version 462679 (0.0005) [2023-12-27 08:50:26,970][20381] Updated weights for policy 1, policy_version 462235 (0.0008) [2023-12-27 08:50:26,980][20292] KL-divergence is very high: 137.2564 [2023-12-27 08:50:27,020][20292] KL-divergence is very high: 139.2767 [2023-12-27 08:50:27,026][20381] Updated weights for policy 1, policy_version 462245 (0.0007) [2023-12-27 08:50:27,058][20292] KL-divergence is very high: 107.0237 [2023-12-27 08:50:27,073][20381] Updated weights for policy 1, policy_version 462255 (0.0008) [2023-12-27 08:50:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 236822528. Throughput: 0: 10419.9, 1: 10323.7. Samples: 236811624. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:27,364][19170] Avg episode reward: [(0, '863.890'), (1, '864.420')] [2023-12-27 08:50:27,520][20347] Updated weights for policy 0, policy_version 462689 (0.0006) [2023-12-27 08:50:27,582][20347] Updated weights for policy 0, policy_version 462699 (0.0007) [2023-12-27 08:50:27,643][20347] Updated weights for policy 0, policy_version 462709 (0.0008) [2023-12-27 08:50:27,699][20347] Updated weights for policy 0, policy_version 462719 (0.0007) [2023-12-27 08:50:27,743][20381] Updated weights for policy 1, policy_version 462265 (0.0006) [2023-12-27 08:50:27,799][20381] Updated weights for policy 1, policy_version 462275 (0.0005) [2023-12-27 08:50:27,858][20381] Updated weights for policy 1, policy_version 462285 (0.0005) [2023-12-27 08:50:28,369][20381] Updated weights for policy 1, policy_version 462295 (0.0008) [2023-12-27 08:50:28,384][20347] Updated weights for policy 0, policy_version 462729 (0.0008) [2023-12-27 08:50:28,419][20381] Updated weights for policy 1, policy_version 462305 (0.0008) [2023-12-27 08:50:28,439][20347] Updated weights for policy 0, policy_version 462739 (0.0008) [2023-12-27 08:50:28,465][20381] Updated weights for policy 1, policy_version 462315 (0.0010) [2023-12-27 08:50:28,491][20347] Updated weights for policy 0, policy_version 462749 (0.0008) [2023-12-27 08:50:29,063][20381] Updated weights for policy 1, policy_version 462325 (0.0010) [2023-12-27 08:50:29,118][20381] Updated weights for policy 1, policy_version 462335 (0.0010) [2023-12-27 08:50:29,163][20381] Updated weights for policy 1, policy_version 462345 (0.0010) [2023-12-27 08:50:29,214][20347] Updated weights for policy 0, policy_version 462759 (0.0007) [2023-12-27 08:50:29,277][20347] Updated weights for policy 0, policy_version 462769 (0.0007) [2023-12-27 08:50:29,338][20347] Updated weights for policy 0, policy_version 462779 (0.0006) [2023-12-27 08:50:29,870][20381] Updated weights for policy 1, policy_version 462355 (0.0009) [2023-12-27 08:50:29,939][20347] Updated weights for policy 0, policy_version 462789 (0.0007) [2023-12-27 08:50:29,942][20381] Updated weights for policy 1, policy_version 462365 (0.0007) [2023-12-27 08:50:29,994][20347] Updated weights for policy 0, policy_version 462799 (0.0008) [2023-12-27 08:50:29,997][20381] Updated weights for policy 1, policy_version 462375 (0.0006) [2023-12-27 08:50:30,051][20347] Updated weights for policy 0, policy_version 462809 (0.0006) [2023-12-27 08:50:30,610][20381] Updated weights for policy 1, policy_version 462385 (0.0007) [2023-12-27 08:50:30,663][20381] Updated weights for policy 1, policy_version 462395 (0.0006) [2023-12-27 08:50:30,719][20381] Updated weights for policy 1, policy_version 462405 (0.0005) [2023-12-27 08:50:30,760][20347] Updated weights for policy 0, policy_version 462819 (0.0008) [2023-12-27 08:50:30,781][20381] Updated weights for policy 1, policy_version 462415 (0.0007) [2023-12-27 08:50:30,814][20347] Updated weights for policy 0, policy_version 462829 (0.0006) [2023-12-27 08:50:30,872][20347] Updated weights for policy 0, policy_version 462839 (0.0008) [2023-12-27 08:50:31,406][20381] Updated weights for policy 1, policy_version 462425 (0.0008) [2023-12-27 08:50:31,464][20381] Updated weights for policy 1, policy_version 462435 (0.0008) [2023-12-27 08:50:31,516][20381] Updated weights for policy 1, policy_version 462445 (0.0006) [2023-12-27 08:50:31,637][20347] Updated weights for policy 0, policy_version 462849 (0.0008) [2023-12-27 08:50:31,691][20347] Updated weights for policy 0, policy_version 462859 (0.0008) [2023-12-27 08:50:31,751][20347] Updated weights for policy 0, policy_version 462869 (0.0008) [2023-12-27 08:50:31,808][20347] Updated weights for policy 0, policy_version 462879 (0.0007) [2023-12-27 08:50:32,106][20381] Updated weights for policy 1, policy_version 462455 (0.0006) [2023-12-27 08:50:32,159][20381] Updated weights for policy 1, policy_version 462465 (0.0005) [2023-12-27 08:50:32,210][20381] Updated weights for policy 1, policy_version 462475 (0.0005) [2023-12-27 08:50:32,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 236929024. Throughput: 0: 10434.3, 1: 10360.1. Samples: 236938816. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:32,363][19170] Avg episode reward: [(0, '912.570'), (1, '855.490')] [2023-12-27 08:50:32,542][20347] Updated weights for policy 0, policy_version 462889 (0.0008) [2023-12-27 08:50:32,602][20347] Updated weights for policy 0, policy_version 462899 (0.0007) [2023-12-27 08:50:32,662][20347] Updated weights for policy 0, policy_version 462909 (0.0007) [2023-12-27 08:50:32,856][20381] Updated weights for policy 1, policy_version 462485 (0.0008) [2023-12-27 08:50:32,910][20381] Updated weights for policy 1, policy_version 462495 (0.0009) [2023-12-27 08:50:32,963][20381] Updated weights for policy 1, policy_version 462505 (0.0005) [2023-12-27 08:50:33,392][20347] Updated weights for policy 0, policy_version 462919 (0.0006) [2023-12-27 08:50:33,434][20347] Updated weights for policy 0, policy_version 462929 (0.0007) [2023-12-27 08:50:33,480][20347] Updated weights for policy 0, policy_version 462939 (0.0007) [2023-12-27 08:50:33,495][20381] Updated weights for policy 1, policy_version 462515 (0.0006) [2023-12-27 08:50:33,543][20381] Updated weights for policy 1, policy_version 462525 (0.0010) [2023-12-27 08:50:33,594][20381] Updated weights for policy 1, policy_version 462535 (0.0010) [2023-12-27 08:50:34,194][20347] Updated weights for policy 0, policy_version 462949 (0.0008) [2023-12-27 08:50:34,257][20347] Updated weights for policy 0, policy_version 462959 (0.0008) [2023-12-27 08:50:34,257][20381] Updated weights for policy 1, policy_version 462545 (0.0010) [2023-12-27 08:50:34,314][20347] Updated weights for policy 0, policy_version 462969 (0.0008) [2023-12-27 08:50:34,315][20381] Updated weights for policy 1, policy_version 462555 (0.0006) [2023-12-27 08:50:34,371][20381] Updated weights for policy 1, policy_version 462565 (0.0006) [2023-12-27 08:50:34,422][20381] Updated weights for policy 1, policy_version 462575 (0.0007) [2023-12-27 08:50:35,009][20347] Updated weights for policy 0, policy_version 462979 (0.0006) [2023-12-27 08:50:35,066][20347] Updated weights for policy 0, policy_version 462989 (0.0007) [2023-12-27 08:50:35,111][20381] Updated weights for policy 1, policy_version 462585 (0.0010) [2023-12-27 08:50:35,117][20347] Updated weights for policy 0, policy_version 462999 (0.0007) [2023-12-27 08:50:35,167][20381] Updated weights for policy 1, policy_version 462595 (0.0011) [2023-12-27 08:50:35,228][20381] Updated weights for policy 1, policy_version 462605 (0.0010) [2023-12-27 08:50:35,834][20347] Updated weights for policy 0, policy_version 463009 (0.0006) [2023-12-27 08:50:35,848][20381] Updated weights for policy 1, policy_version 462615 (0.0011) [2023-12-27 08:50:35,881][20347] Updated weights for policy 0, policy_version 463019 (0.0008) [2023-12-27 08:50:35,897][20381] Updated weights for policy 1, policy_version 462625 (0.0010) [2023-12-27 08:50:35,928][20347] Updated weights for policy 0, policy_version 463029 (0.0007) [2023-12-27 08:50:35,954][20381] Updated weights for policy 1, policy_version 462635 (0.0008) [2023-12-27 08:50:35,975][20347] Updated weights for policy 0, policy_version 463039 (0.0008) [2023-12-27 08:50:36,675][20347] Updated weights for policy 0, policy_version 463049 (0.0007) [2023-12-27 08:50:36,706][20381] Updated weights for policy 1, policy_version 462645 (0.0008) [2023-12-27 08:50:36,733][20347] Updated weights for policy 0, policy_version 463059 (0.0009) [2023-12-27 08:50:36,763][20381] Updated weights for policy 1, policy_version 462655 (0.0010) [2023-12-27 08:50:36,791][20347] Updated weights for policy 0, policy_version 463069 (0.0006) [2023-12-27 08:50:36,819][20381] Updated weights for policy 1, policy_version 462665 (0.0010) [2023-12-27 08:50:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 237027328. Throughput: 0: 10391.5, 1: 10388.6. Samples: 237000848. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:37,363][19170] Avg episode reward: [(0, '951.490'), (1, '835.200')] [2023-12-27 08:50:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000462672_118464512.pth... [2023-12-27 08:50:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000463072_118562816.pth... [2023-12-27 08:50:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000461888_118259712.pth [2023-12-27 08:50:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000461456_118153216.pth [2023-12-27 08:50:37,490][20381] Updated weights for policy 1, policy_version 462675 (0.0008) [2023-12-27 08:50:37,507][20347] Updated weights for policy 0, policy_version 463079 (0.0007) [2023-12-27 08:50:37,542][20381] Updated weights for policy 1, policy_version 462685 (0.0010) [2023-12-27 08:50:37,573][20347] Updated weights for policy 0, policy_version 463089 (0.0007) [2023-12-27 08:50:37,595][20381] Updated weights for policy 1, policy_version 462695 (0.0010) [2023-12-27 08:50:37,637][20347] Updated weights for policy 0, policy_version 463099 (0.0006) [2023-12-27 08:50:38,229][20381] Updated weights for policy 1, policy_version 462705 (0.0010) [2023-12-27 08:50:38,281][20347] Updated weights for policy 0, policy_version 463109 (0.0006) [2023-12-27 08:50:38,287][20381] Updated weights for policy 1, policy_version 462715 (0.0007) [2023-12-27 08:50:38,335][20347] Updated weights for policy 0, policy_version 463119 (0.0006) [2023-12-27 08:50:38,348][20381] Updated weights for policy 1, policy_version 462725 (0.0007) [2023-12-27 08:50:38,393][20347] Updated weights for policy 0, policy_version 463129 (0.0007) [2023-12-27 08:50:38,407][20381] Updated weights for policy 1, policy_version 462735 (0.0007) [2023-12-27 08:50:39,027][20347] Updated weights for policy 0, policy_version 463139 (0.0008) [2023-12-27 08:50:39,062][20381] Updated weights for policy 1, policy_version 462745 (0.0008) [2023-12-27 08:50:39,091][20347] Updated weights for policy 0, policy_version 463149 (0.0008) [2023-12-27 08:50:39,123][20381] Updated weights for policy 1, policy_version 462755 (0.0006) [2023-12-27 08:50:39,150][20347] Updated weights for policy 0, policy_version 463159 (0.0008) [2023-12-27 08:50:39,178][20381] Updated weights for policy 1, policy_version 462765 (0.0005) [2023-12-27 08:50:39,864][20381] Updated weights for policy 1, policy_version 462775 (0.0008) [2023-12-27 08:50:39,875][20347] Updated weights for policy 0, policy_version 463169 (0.0006) [2023-12-27 08:50:39,918][20381] Updated weights for policy 1, policy_version 462785 (0.0007) [2023-12-27 08:50:39,937][20347] Updated weights for policy 0, policy_version 463179 (0.0006) [2023-12-27 08:50:39,982][20381] Updated weights for policy 1, policy_version 462795 (0.0006) [2023-12-27 08:50:39,995][20347] Updated weights for policy 0, policy_version 463189 (0.0008) [2023-12-27 08:50:40,055][20347] Updated weights for policy 0, policy_version 463199 (0.0007) [2023-12-27 08:50:40,680][20381] Updated weights for policy 1, policy_version 462805 (0.0008) [2023-12-27 08:50:40,727][20347] Updated weights for policy 0, policy_version 463209 (0.0007) [2023-12-27 08:50:40,738][20381] Updated weights for policy 1, policy_version 462815 (0.0011) [2023-12-27 08:50:40,781][20347] Updated weights for policy 0, policy_version 463219 (0.0008) [2023-12-27 08:50:40,790][20381] Updated weights for policy 1, policy_version 462825 (0.0010) [2023-12-27 08:50:40,829][20347] Updated weights for policy 0, policy_version 463229 (0.0006) [2023-12-27 08:50:41,526][20347] Updated weights for policy 0, policy_version 463239 (0.0007) [2023-12-27 08:50:41,561][20381] Updated weights for policy 1, policy_version 462835 (0.0010) [2023-12-27 08:50:41,584][20347] Updated weights for policy 0, policy_version 463249 (0.0006) [2023-12-27 08:50:41,621][20381] Updated weights for policy 1, policy_version 462845 (0.0008) [2023-12-27 08:50:41,640][20347] Updated weights for policy 0, policy_version 463259 (0.0007) [2023-12-27 08:50:41,683][20381] Updated weights for policy 1, policy_version 462855 (0.0008) [2023-12-27 08:50:42,319][20347] Updated weights for policy 0, policy_version 463269 (0.0007) [2023-12-27 08:50:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 237125632. Throughput: 0: 10351.3, 1: 10321.1. Samples: 237122632. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:42,363][19170] Avg episode reward: [(0, '946.130'), (1, '892.900')] [2023-12-27 08:50:42,385][20347] Updated weights for policy 0, policy_version 463279 (0.0008) [2023-12-27 08:50:42,434][20381] Updated weights for policy 1, policy_version 462865 (0.0008) [2023-12-27 08:50:42,444][20347] Updated weights for policy 0, policy_version 463289 (0.0007) [2023-12-27 08:50:42,495][20381] Updated weights for policy 1, policy_version 462875 (0.0006) [2023-12-27 08:50:42,552][20381] Updated weights for policy 1, policy_version 462885 (0.0007) [2023-12-27 08:50:42,604][20381] Updated weights for policy 1, policy_version 462895 (0.0007) [2023-12-27 08:50:43,125][20347] Updated weights for policy 0, policy_version 463299 (0.0008) [2023-12-27 08:50:43,170][20347] Updated weights for policy 0, policy_version 463309 (0.0007) [2023-12-27 08:50:43,225][20347] Updated weights for policy 0, policy_version 463319 (0.0007) [2023-12-27 08:50:43,303][20381] Updated weights for policy 1, policy_version 462905 (0.0006) [2023-12-27 08:50:43,355][20381] Updated weights for policy 1, policy_version 462915 (0.0006) [2023-12-27 08:50:43,412][20381] Updated weights for policy 1, policy_version 462925 (0.0005) [2023-12-27 08:50:43,911][20347] Updated weights for policy 0, policy_version 463329 (0.0007) [2023-12-27 08:50:43,966][20347] Updated weights for policy 0, policy_version 463339 (0.0010) [2023-12-27 08:50:44,016][20381] Updated weights for policy 1, policy_version 462935 (0.0009) [2023-12-27 08:50:44,021][20347] Updated weights for policy 0, policy_version 463349 (0.0006) [2023-12-27 08:50:44,070][20347] Updated weights for policy 0, policy_version 463359 (0.0007) [2023-12-27 08:50:44,076][20381] Updated weights for policy 1, policy_version 462945 (0.0010) [2023-12-27 08:50:44,133][20381] Updated weights for policy 1, policy_version 462955 (0.0010) [2023-12-27 08:50:44,712][20347] Updated weights for policy 0, policy_version 463369 (0.0005) [2023-12-27 08:50:44,762][20347] Updated weights for policy 0, policy_version 463379 (0.0005) [2023-12-27 08:50:44,798][20381] Updated weights for policy 1, policy_version 462965 (0.0009) [2023-12-27 08:50:44,820][20347] Updated weights for policy 0, policy_version 463389 (0.0010) [2023-12-27 08:50:44,849][20381] Updated weights for policy 1, policy_version 462975 (0.0008) [2023-12-27 08:50:44,898][20381] Updated weights for policy 1, policy_version 462985 (0.0008) [2023-12-27 08:50:45,412][20347] Updated weights for policy 0, policy_version 463399 (0.0010) [2023-12-27 08:50:45,464][20347] Updated weights for policy 0, policy_version 463409 (0.0010) [2023-12-27 08:50:45,524][20347] Updated weights for policy 0, policy_version 463419 (0.0010) [2023-12-27 08:50:45,580][20381] Updated weights for policy 1, policy_version 462995 (0.0007) [2023-12-27 08:50:45,642][20381] Updated weights for policy 1, policy_version 463005 (0.0008) [2023-12-27 08:50:45,705][20381] Updated weights for policy 1, policy_version 463015 (0.0008) [2023-12-27 08:50:46,152][20347] Updated weights for policy 0, policy_version 463429 (0.0008) [2023-12-27 08:50:46,210][20347] Updated weights for policy 0, policy_version 463439 (0.0007) [2023-12-27 08:50:46,268][20347] Updated weights for policy 0, policy_version 463449 (0.0007) [2023-12-27 08:50:46,272][20381] Updated weights for policy 1, policy_version 463025 (0.0008) [2023-12-27 08:50:46,323][20381] Updated weights for policy 1, policy_version 463035 (0.0010) [2023-12-27 08:50:46,375][20381] Updated weights for policy 1, policy_version 463045 (0.0007) [2023-12-27 08:50:46,421][20381] Updated weights for policy 1, policy_version 463055 (0.0007) [2023-12-27 08:50:46,972][20347] Updated weights for policy 0, policy_version 463459 (0.0007) [2023-12-27 08:50:47,035][20347] Updated weights for policy 0, policy_version 463469 (0.0005) [2023-12-27 08:50:47,076][20381] Updated weights for policy 1, policy_version 463065 (0.0005) [2023-12-27 08:50:47,093][20347] Updated weights for policy 0, policy_version 463479 (0.0005) [2023-12-27 08:50:47,136][20381] Updated weights for policy 1, policy_version 463075 (0.0006) [2023-12-27 08:50:47,195][20381] Updated weights for policy 1, policy_version 463085 (0.0008) [2023-12-27 08:50:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 237240320. Throughput: 0: 10357.0, 1: 10354.0. Samples: 237247852. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:47,363][19170] Avg episode reward: [(0, '961.780'), (1, '961.070')] [2023-12-27 08:50:47,709][20347] Updated weights for policy 0, policy_version 463489 (0.0009) [2023-12-27 08:50:47,733][20381] Updated weights for policy 1, policy_version 463095 (0.0006) [2023-12-27 08:50:47,764][20347] Updated weights for policy 0, policy_version 463499 (0.0010) [2023-12-27 08:50:47,785][20381] Updated weights for policy 1, policy_version 463105 (0.0006) [2023-12-27 08:50:47,817][20347] Updated weights for policy 0, policy_version 463509 (0.0010) [2023-12-27 08:50:47,834][20381] Updated weights for policy 1, policy_version 463115 (0.0006) [2023-12-27 08:50:47,865][20347] Updated weights for policy 0, policy_version 463519 (0.0010) [2023-12-27 08:50:48,496][20381] Updated weights for policy 1, policy_version 463125 (0.0006) [2023-12-27 08:50:48,528][20347] Updated weights for policy 0, policy_version 463529 (0.0010) [2023-12-27 08:50:48,546][20381] Updated weights for policy 1, policy_version 463135 (0.0006) [2023-12-27 08:50:48,583][20347] Updated weights for policy 0, policy_version 463539 (0.0010) [2023-12-27 08:50:48,589][20381] Updated weights for policy 1, policy_version 463145 (0.0006) [2023-12-27 08:50:48,641][20347] Updated weights for policy 0, policy_version 463549 (0.0010) [2023-12-27 08:50:49,319][20381] Updated weights for policy 1, policy_version 463155 (0.0007) [2023-12-27 08:50:49,330][20347] Updated weights for policy 0, policy_version 463559 (0.0011) [2023-12-27 08:50:49,382][20381] Updated weights for policy 1, policy_version 463165 (0.0007) [2023-12-27 08:50:49,388][20347] Updated weights for policy 0, policy_version 463569 (0.0010) [2023-12-27 08:50:49,431][20381] Updated weights for policy 1, policy_version 463175 (0.0008) [2023-12-27 08:50:49,438][20347] Updated weights for policy 0, policy_version 463579 (0.0010) [2023-12-27 08:50:50,112][20381] Updated weights for policy 1, policy_version 463185 (0.0007) [2023-12-27 08:50:50,142][20347] Updated weights for policy 0, policy_version 463589 (0.0011) [2023-12-27 08:50:50,166][20381] Updated weights for policy 1, policy_version 463195 (0.0007) [2023-12-27 08:50:50,201][20347] Updated weights for policy 0, policy_version 463599 (0.0010) [2023-12-27 08:50:50,220][20381] Updated weights for policy 1, policy_version 463205 (0.0006) [2023-12-27 08:50:50,258][20347] Updated weights for policy 0, policy_version 463609 (0.0011) [2023-12-27 08:50:50,271][20381] Updated weights for policy 1, policy_version 463215 (0.0007) [2023-12-27 08:50:50,930][20381] Updated weights for policy 1, policy_version 463225 (0.0010) [2023-12-27 08:50:50,959][20347] Updated weights for policy 0, policy_version 463619 (0.0009) [2023-12-27 08:50:50,988][20381] Updated weights for policy 1, policy_version 463235 (0.0011) [2023-12-27 08:50:51,019][20347] Updated weights for policy 0, policy_version 463629 (0.0007) [2023-12-27 08:50:51,047][20381] Updated weights for policy 1, policy_version 463245 (0.0009) [2023-12-27 08:50:51,084][20347] Updated weights for policy 0, policy_version 463639 (0.0007) [2023-12-27 08:50:51,686][20381] Updated weights for policy 1, policy_version 463255 (0.0007) [2023-12-27 08:50:51,749][20381] Updated weights for policy 1, policy_version 463265 (0.0007) [2023-12-27 08:50:51,808][20381] Updated weights for policy 1, policy_version 463275 (0.0007) [2023-12-27 08:50:51,819][20347] Updated weights for policy 0, policy_version 463649 (0.0008) [2023-12-27 08:50:51,879][20347] Updated weights for policy 0, policy_version 463659 (0.0008) [2023-12-27 08:50:51,947][20347] Updated weights for policy 0, policy_version 463669 (0.0007) [2023-12-27 08:50:52,009][20347] Updated weights for policy 0, policy_version 463679 (0.0007) [2023-12-27 08:50:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 237338624. Throughput: 0: 10318.8, 1: 10395.3. Samples: 237310732. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:52,363][19170] Avg episode reward: [(0, '996.280'), (1, '990.280')] [2023-12-27 08:50:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000463280_118620160.pth... [2023-12-27 08:50:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000463680_118718464.pth... [2023-12-27 08:50:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000462496_118415360.pth [2023-12-27 08:50:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000462064_118308864.pth [2023-12-27 08:50:52,482][20381] Updated weights for policy 1, policy_version 463285 (0.0008) [2023-12-27 08:50:52,539][20381] Updated weights for policy 1, policy_version 463295 (0.0008) [2023-12-27 08:50:52,595][20381] Updated weights for policy 1, policy_version 463305 (0.0008) [2023-12-27 08:50:52,666][20347] Updated weights for policy 0, policy_version 463689 (0.0007) [2023-12-27 08:50:52,721][20347] Updated weights for policy 0, policy_version 463699 (0.0008) [2023-12-27 08:50:52,774][20347] Updated weights for policy 0, policy_version 463709 (0.0008) [2023-12-27 08:50:53,233][20381] Updated weights for policy 1, policy_version 463315 (0.0007) [2023-12-27 08:50:53,285][20381] Updated weights for policy 1, policy_version 463325 (0.0007) [2023-12-27 08:50:53,340][20381] Updated weights for policy 1, policy_version 463335 (0.0007) [2023-12-27 08:50:53,435][20347] Updated weights for policy 0, policy_version 463719 (0.0006) [2023-12-27 08:50:53,493][20347] Updated weights for policy 0, policy_version 463729 (0.0007) [2023-12-27 08:50:53,556][20347] Updated weights for policy 0, policy_version 463739 (0.0007) [2023-12-27 08:50:54,024][20381] Updated weights for policy 1, policy_version 463345 (0.0007) [2023-12-27 08:50:54,081][20381] Updated weights for policy 1, policy_version 463355 (0.0007) [2023-12-27 08:50:54,129][20381] Updated weights for policy 1, policy_version 463365 (0.0007) [2023-12-27 08:50:54,165][20347] Updated weights for policy 0, policy_version 463749 (0.0008) [2023-12-27 08:50:54,191][20381] Updated weights for policy 1, policy_version 463375 (0.0009) [2023-12-27 08:50:54,232][20347] Updated weights for policy 0, policy_version 463759 (0.0008) [2023-12-27 08:50:54,296][20347] Updated weights for policy 0, policy_version 463769 (0.0007) [2023-12-27 08:50:54,899][20381] Updated weights for policy 1, policy_version 463385 (0.0008) [2023-12-27 08:50:54,951][20381] Updated weights for policy 1, policy_version 463395 (0.0007) [2023-12-27 08:50:54,976][20347] Updated weights for policy 0, policy_version 463779 (0.0007) [2023-12-27 08:50:55,008][20381] Updated weights for policy 1, policy_version 463405 (0.0006) [2023-12-27 08:50:55,031][20347] Updated weights for policy 0, policy_version 463789 (0.0008) [2023-12-27 08:50:55,088][20347] Updated weights for policy 0, policy_version 463799 (0.0007) [2023-12-27 08:50:55,668][20381] Updated weights for policy 1, policy_version 463415 (0.0007) [2023-12-27 08:50:55,717][20381] Updated weights for policy 1, policy_version 463425 (0.0008) [2023-12-27 08:50:55,772][20347] Updated weights for policy 0, policy_version 463809 (0.0005) [2023-12-27 08:50:55,773][20381] Updated weights for policy 1, policy_version 463435 (0.0008) [2023-12-27 08:50:55,835][20347] Updated weights for policy 0, policy_version 463819 (0.0008) [2023-12-27 08:50:55,887][20347] Updated weights for policy 0, policy_version 463829 (0.0007) [2023-12-27 08:50:55,937][20347] Updated weights for policy 0, policy_version 463839 (0.0009) [2023-12-27 08:50:56,432][20381] Updated weights for policy 1, policy_version 463445 (0.0008) [2023-12-27 08:50:56,480][20381] Updated weights for policy 1, policy_version 463455 (0.0007) [2023-12-27 08:50:56,524][20381] Updated weights for policy 1, policy_version 463465 (0.0008) [2023-12-27 08:50:56,591][20347] Updated weights for policy 0, policy_version 463849 (0.0006) [2023-12-27 08:50:56,646][20347] Updated weights for policy 0, policy_version 463859 (0.0005) [2023-12-27 08:50:56,694][20347] Updated weights for policy 0, policy_version 463869 (0.0005) [2023-12-27 08:50:57,281][20381] Updated weights for policy 1, policy_version 463475 (0.0008) [2023-12-27 08:50:57,310][20347] Updated weights for policy 0, policy_version 463879 (0.0006) [2023-12-27 08:50:57,338][20381] Updated weights for policy 1, policy_version 463485 (0.0007) [2023-12-27 08:50:57,359][20347] Updated weights for policy 0, policy_version 463889 (0.0007) [2023-12-27 08:50:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 237436928. Throughput: 0: 10263.1, 1: 10442.2. Samples: 237433696. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:50:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.310')] [2023-12-27 08:50:57,390][20381] Updated weights for policy 1, policy_version 463495 (0.0006) [2023-12-27 08:50:57,411][20347] Updated weights for policy 0, policy_version 463899 (0.0007) [2023-12-27 08:50:58,031][20347] Updated weights for policy 0, policy_version 463909 (0.0006) [2023-12-27 08:50:58,058][20381] Updated weights for policy 1, policy_version 463505 (0.0007) [2023-12-27 08:50:58,081][20347] Updated weights for policy 0, policy_version 463919 (0.0006) [2023-12-27 08:50:58,120][20381] Updated weights for policy 1, policy_version 463515 (0.0005) [2023-12-27 08:50:58,134][20347] Updated weights for policy 0, policy_version 463929 (0.0006) [2023-12-27 08:50:58,181][20381] Updated weights for policy 1, policy_version 463525 (0.0007) [2023-12-27 08:50:58,242][20381] Updated weights for policy 1, policy_version 463535 (0.0008) [2023-12-27 08:50:58,852][20347] Updated weights for policy 0, policy_version 463939 (0.0007) [2023-12-27 08:50:58,919][20347] Updated weights for policy 0, policy_version 463949 (0.0009) [2023-12-27 08:50:58,976][20347] Updated weights for policy 0, policy_version 463959 (0.0007) [2023-12-27 08:50:59,021][20381] Updated weights for policy 1, policy_version 463545 (0.0008) [2023-12-27 08:50:59,080][20381] Updated weights for policy 1, policy_version 463555 (0.0008) [2023-12-27 08:50:59,127][20381] Updated weights for policy 1, policy_version 463565 (0.0008) [2023-12-27 08:50:59,710][20347] Updated weights for policy 0, policy_version 463969 (0.0006) [2023-12-27 08:50:59,764][20347] Updated weights for policy 0, policy_version 463979 (0.0008) [2023-12-27 08:50:59,832][20347] Updated weights for policy 0, policy_version 463989 (0.0008) [2023-12-27 08:50:59,836][20381] Updated weights for policy 1, policy_version 463575 (0.0006) [2023-12-27 08:50:59,891][20347] Updated weights for policy 0, policy_version 463999 (0.0007) [2023-12-27 08:50:59,896][20381] Updated weights for policy 1, policy_version 463585 (0.0007) [2023-12-27 08:50:59,954][20381] Updated weights for policy 1, policy_version 463595 (0.0008) [2023-12-27 08:51:00,574][20347] Updated weights for policy 0, policy_version 464009 (0.0006) [2023-12-27 08:51:00,580][20381] Updated weights for policy 1, policy_version 463605 (0.0007) [2023-12-27 08:51:00,631][20347] Updated weights for policy 0, policy_version 464019 (0.0006) [2023-12-27 08:51:00,632][20381] Updated weights for policy 1, policy_version 463615 (0.0005) [2023-12-27 08:51:00,686][20381] Updated weights for policy 1, policy_version 463625 (0.0008) [2023-12-27 08:51:00,687][20347] Updated weights for policy 0, policy_version 464029 (0.0007) [2023-12-27 08:51:01,276][20381] Updated weights for policy 1, policy_version 463635 (0.0007) [2023-12-27 08:51:01,312][20347] Updated weights for policy 0, policy_version 464039 (0.0010) [2023-12-27 08:51:01,328][20381] Updated weights for policy 1, policy_version 463645 (0.0011) [2023-12-27 08:51:01,374][20347] Updated weights for policy 0, policy_version 464049 (0.0009) [2023-12-27 08:51:01,392][20381] Updated weights for policy 1, policy_version 463655 (0.0008) [2023-12-27 08:51:01,433][20347] Updated weights for policy 0, policy_version 464059 (0.0008) [2023-12-27 08:51:02,071][20381] Updated weights for policy 1, policy_version 463665 (0.0008) [2023-12-27 08:51:02,126][20381] Updated weights for policy 1, policy_version 463675 (0.0010) [2023-12-27 08:51:02,135][20347] Updated weights for policy 0, policy_version 464069 (0.0007) [2023-12-27 08:51:02,173][20381] Updated weights for policy 1, policy_version 463685 (0.0008) [2023-12-27 08:51:02,198][20347] Updated weights for policy 0, policy_version 464079 (0.0007) [2023-12-27 08:51:02,229][20381] Updated weights for policy 1, policy_version 463695 (0.0009) [2023-12-27 08:51:02,260][20347] Updated weights for policy 0, policy_version 464089 (0.0006) [2023-12-27 08:51:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 237551616. Throughput: 0: 10280.8, 1: 10471.6. Samples: 237556740. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:51:02,363][19170] Avg episode reward: [(0, '993.610'), (1, '970.980')] [2023-12-27 08:51:02,856][20347] Updated weights for policy 0, policy_version 464099 (0.0008) [2023-12-27 08:51:02,909][20347] Updated weights for policy 0, policy_version 464109 (0.0006) [2023-12-27 08:51:02,930][20381] Updated weights for policy 1, policy_version 463705 (0.0007) [2023-12-27 08:51:02,964][20347] Updated weights for policy 0, policy_version 464119 (0.0006) [2023-12-27 08:51:02,985][20381] Updated weights for policy 1, policy_version 463715 (0.0006) [2023-12-27 08:51:03,038][20381] Updated weights for policy 1, policy_version 463725 (0.0007) [2023-12-27 08:51:03,630][20347] Updated weights for policy 0, policy_version 464129 (0.0007) [2023-12-27 08:51:03,687][20347] Updated weights for policy 0, policy_version 464139 (0.0007) [2023-12-27 08:51:03,720][20381] Updated weights for policy 1, policy_version 463735 (0.0007) [2023-12-27 08:51:03,744][20347] Updated weights for policy 0, policy_version 464149 (0.0006) [2023-12-27 08:51:03,771][20381] Updated weights for policy 1, policy_version 463745 (0.0006) [2023-12-27 08:51:03,806][20347] Updated weights for policy 0, policy_version 464159 (0.0007) [2023-12-27 08:51:03,823][20381] Updated weights for policy 1, policy_version 463755 (0.0006) [2023-12-27 08:51:04,457][20347] Updated weights for policy 0, policy_version 464169 (0.0007) [2023-12-27 08:51:04,511][20347] Updated weights for policy 0, policy_version 464179 (0.0009) [2023-12-27 08:51:04,515][20381] Updated weights for policy 1, policy_version 463765 (0.0006) [2023-12-27 08:51:04,564][20347] Updated weights for policy 0, policy_version 464189 (0.0010) [2023-12-27 08:51:04,569][20381] Updated weights for policy 1, policy_version 463775 (0.0007) [2023-12-27 08:51:04,628][20381] Updated weights for policy 1, policy_version 463785 (0.0008) [2023-12-27 08:51:05,148][20347] Updated weights for policy 0, policy_version 464199 (0.0007) [2023-12-27 08:51:05,199][20347] Updated weights for policy 0, policy_version 464209 (0.0005) [2023-12-27 08:51:05,260][20347] Updated weights for policy 0, policy_version 464219 (0.0005) [2023-12-27 08:51:05,292][20381] Updated weights for policy 1, policy_version 463795 (0.0008) [2023-12-27 08:51:05,341][20381] Updated weights for policy 1, policy_version 463805 (0.0009) [2023-12-27 08:51:05,390][20381] Updated weights for policy 1, policy_version 463815 (0.0009) [2023-12-27 08:51:05,774][20347] Updated weights for policy 0, policy_version 464229 (0.0006) [2023-12-27 08:51:05,836][20347] Updated weights for policy 0, policy_version 464239 (0.0008) [2023-12-27 08:51:05,897][20347] Updated weights for policy 0, policy_version 464249 (0.0008) [2023-12-27 08:51:06,136][20381] Updated weights for policy 1, policy_version 463825 (0.0008) [2023-12-27 08:51:06,202][20381] Updated weights for policy 1, policy_version 463835 (0.0008) [2023-12-27 08:51:06,267][20381] Updated weights for policy 1, policy_version 463845 (0.0008) [2023-12-27 08:51:06,337][20381] Updated weights for policy 1, policy_version 463855 (0.0008) [2023-12-27 08:51:06,523][20347] Updated weights for policy 0, policy_version 464259 (0.0008) [2023-12-27 08:51:06,585][20347] Updated weights for policy 0, policy_version 464269 (0.0007) [2023-12-27 08:51:06,638][20347] Updated weights for policy 0, policy_version 464279 (0.0008) [2023-12-27 08:51:07,026][20381] Updated weights for policy 1, policy_version 463865 (0.0008) [2023-12-27 08:51:07,083][20381] Updated weights for policy 1, policy_version 463875 (0.0008) [2023-12-27 08:51:07,145][20381] Updated weights for policy 1, policy_version 463885 (0.0007) [2023-12-27 08:51:07,294][20347] Updated weights for policy 0, policy_version 464289 (0.0007) [2023-12-27 08:51:07,353][20347] Updated weights for policy 0, policy_version 464299 (0.0008) [2023-12-27 08:51:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 237649920. Throughput: 0: 10325.1, 1: 10445.8. Samples: 237619684. Policy #0 lag: (min: 23.0, avg: 24.1, max: 48.0) [2023-12-27 08:51:07,363][19170] Avg episode reward: [(0, '992.160'), (1, '942.430')] [2023-12-27 08:51:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000463888_118775808.pth... [2023-12-27 08:51:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000462672_118464512.pth [2023-12-27 08:51:07,425][20347] Updated weights for policy 0, policy_version 464309 (0.0009) [2023-12-27 08:51:07,491][20347] Updated weights for policy 0, policy_version 464319 (0.0007) [2023-12-27 08:51:07,498][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000464320_118882304.pth... [2023-12-27 08:51:07,503][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000463072_118562816.pth [2023-12-27 08:51:07,823][20381] Updated weights for policy 1, policy_version 463895 (0.0006) [2023-12-27 08:51:07,869][20381] Updated weights for policy 1, policy_version 463905 (0.0008) [2023-12-27 08:51:07,915][20381] Updated weights for policy 1, policy_version 463915 (0.0008) [2023-12-27 08:51:08,153][20347] Updated weights for policy 0, policy_version 464329 (0.0007) [2023-12-27 08:51:08,204][20347] Updated weights for policy 0, policy_version 464339 (0.0008) [2023-12-27 08:51:08,265][20347] Updated weights for policy 0, policy_version 464349 (0.0007) [2023-12-27 08:51:08,558][20381] Updated weights for policy 1, policy_version 463925 (0.0006) [2023-12-27 08:51:08,613][20381] Updated weights for policy 1, policy_version 463935 (0.0005) [2023-12-27 08:51:08,677][20381] Updated weights for policy 1, policy_version 463945 (0.0007) [2023-12-27 08:51:08,972][20347] Updated weights for policy 0, policy_version 464359 (0.0006) [2023-12-27 08:51:09,034][20347] Updated weights for policy 0, policy_version 464369 (0.0005) [2023-12-27 08:51:09,093][20347] Updated weights for policy 0, policy_version 464379 (0.0006) [2023-12-27 08:51:09,290][20381] Updated weights for policy 1, policy_version 463955 (0.0008) [2023-12-27 08:51:09,355][20381] Updated weights for policy 1, policy_version 463965 (0.0007) [2023-12-27 08:51:09,420][20381] Updated weights for policy 1, policy_version 463975 (0.0008) [2023-12-27 08:51:09,717][20347] Updated weights for policy 0, policy_version 464389 (0.0007) [2023-12-27 08:51:09,784][20347] Updated weights for policy 0, policy_version 464399 (0.0006) [2023-12-27 08:51:09,849][20347] Updated weights for policy 0, policy_version 464409 (0.0007) [2023-12-27 08:51:10,122][20381] Updated weights for policy 1, policy_version 463985 (0.0009) [2023-12-27 08:51:10,182][20381] Updated weights for policy 1, policy_version 463995 (0.0008) [2023-12-27 08:51:10,240][20381] Updated weights for policy 1, policy_version 464005 (0.0008) [2023-12-27 08:51:10,300][20381] Updated weights for policy 1, policy_version 464015 (0.0008) [2023-12-27 08:51:10,499][20347] Updated weights for policy 0, policy_version 464419 (0.0008) [2023-12-27 08:51:10,553][20347] Updated weights for policy 0, policy_version 464429 (0.0008) [2023-12-27 08:51:10,605][20347] Updated weights for policy 0, policy_version 464439 (0.0008) [2023-12-27 08:51:10,966][20381] Updated weights for policy 1, policy_version 464025 (0.0008) [2023-12-27 08:51:11,013][20381] Updated weights for policy 1, policy_version 464035 (0.0008) [2023-12-27 08:51:11,076][20381] Updated weights for policy 1, policy_version 464045 (0.0009) [2023-12-27 08:51:11,331][20347] Updated weights for policy 0, policy_version 464449 (0.0008) [2023-12-27 08:51:11,400][20347] Updated weights for policy 0, policy_version 464459 (0.0008) [2023-12-27 08:51:11,464][20347] Updated weights for policy 0, policy_version 464469 (0.0008) [2023-12-27 08:51:11,518][20347] Updated weights for policy 0, policy_version 464479 (0.0008) [2023-12-27 08:51:11,810][20381] Updated weights for policy 1, policy_version 464055 (0.0006) [2023-12-27 08:51:11,866][20381] Updated weights for policy 1, policy_version 464065 (0.0008) [2023-12-27 08:51:11,930][20381] Updated weights for policy 1, policy_version 464075 (0.0008) [2023-12-27 08:51:12,218][20347] Updated weights for policy 0, policy_version 464489 (0.0008) [2023-12-27 08:51:12,277][20347] Updated weights for policy 0, policy_version 464499 (0.0007) [2023-12-27 08:51:12,338][20347] Updated weights for policy 0, policy_version 464509 (0.0008) [2023-12-27 08:51:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 237756416. Throughput: 0: 10319.1, 1: 10393.6. Samples: 237743692. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:12,363][19170] Avg episode reward: [(0, '998.550'), (1, '961.760')] [2023-12-27 08:51:12,636][20381] Updated weights for policy 1, policy_version 464085 (0.0008) [2023-12-27 08:51:12,696][20381] Updated weights for policy 1, policy_version 464095 (0.0005) [2023-12-27 08:51:12,754][20381] Updated weights for policy 1, policy_version 464105 (0.0006) [2023-12-27 08:51:13,075][20347] Updated weights for policy 0, policy_version 464519 (0.0007) [2023-12-27 08:51:13,130][20347] Updated weights for policy 0, policy_version 464529 (0.0005) [2023-12-27 08:51:13,195][20347] Updated weights for policy 0, policy_version 464539 (0.0006) [2023-12-27 08:51:13,377][20381] Updated weights for policy 1, policy_version 464115 (0.0006) [2023-12-27 08:51:13,427][20381] Updated weights for policy 1, policy_version 464125 (0.0007) [2023-12-27 08:51:13,485][20381] Updated weights for policy 1, policy_version 464135 (0.0008) [2023-12-27 08:51:13,781][20347] Updated weights for policy 0, policy_version 464549 (0.0008) [2023-12-27 08:51:13,840][20347] Updated weights for policy 0, policy_version 464559 (0.0007) [2023-12-27 08:51:13,892][20347] Updated weights for policy 0, policy_version 464569 (0.0008) [2023-12-27 08:51:14,192][20381] Updated weights for policy 1, policy_version 464145 (0.0007) [2023-12-27 08:51:14,251][20381] Updated weights for policy 1, policy_version 464155 (0.0007) [2023-12-27 08:51:14,304][20381] Updated weights for policy 1, policy_version 464165 (0.0007) [2023-12-27 08:51:14,355][20381] Updated weights for policy 1, policy_version 464175 (0.0007) [2023-12-27 08:51:14,586][20347] Updated weights for policy 0, policy_version 464579 (0.0008) [2023-12-27 08:51:14,638][20347] Updated weights for policy 0, policy_version 464589 (0.0010) [2023-12-27 08:51:14,696][20347] Updated weights for policy 0, policy_version 464599 (0.0010) [2023-12-27 08:51:15,008][20381] Updated weights for policy 1, policy_version 464185 (0.0006) [2023-12-27 08:51:15,073][20381] Updated weights for policy 1, policy_version 464195 (0.0005) [2023-12-27 08:51:15,131][20381] Updated weights for policy 1, policy_version 464205 (0.0005) [2023-12-27 08:51:15,413][20347] Updated weights for policy 0, policy_version 464609 (0.0010) [2023-12-27 08:51:15,460][20347] Updated weights for policy 0, policy_version 464619 (0.0008) [2023-12-27 08:51:15,513][20347] Updated weights for policy 0, policy_version 464629 (0.0007) [2023-12-27 08:51:15,567][20347] Updated weights for policy 0, policy_version 464639 (0.0009) [2023-12-27 08:51:15,730][20381] Updated weights for policy 1, policy_version 464215 (0.0006) [2023-12-27 08:51:15,786][20381] Updated weights for policy 1, policy_version 464225 (0.0005) [2023-12-27 08:51:15,834][20381] Updated weights for policy 1, policy_version 464235 (0.0005) [2023-12-27 08:51:16,359][20381] Updated weights for policy 1, policy_version 464245 (0.0007) [2023-12-27 08:51:16,368][20347] Updated weights for policy 0, policy_version 464649 (0.0007) [2023-12-27 08:51:16,411][20381] Updated weights for policy 1, policy_version 464255 (0.0006) [2023-12-27 08:51:16,431][20347] Updated weights for policy 0, policy_version 464659 (0.0006) [2023-12-27 08:51:16,469][20381] Updated weights for policy 1, policy_version 464265 (0.0006) [2023-12-27 08:51:16,488][20347] Updated weights for policy 0, policy_version 464669 (0.0005) [2023-12-27 08:51:17,079][20381] Updated weights for policy 1, policy_version 464275 (0.0007) [2023-12-27 08:51:17,134][20381] Updated weights for policy 1, policy_version 464285 (0.0005) [2023-12-27 08:51:17,186][20381] Updated weights for policy 1, policy_version 464295 (0.0005) [2023-12-27 08:51:17,195][20347] Updated weights for policy 0, policy_version 464679 (0.0006) [2023-12-27 08:51:17,247][20347] Updated weights for policy 0, policy_version 464689 (0.0005) [2023-12-27 08:51:17,302][20347] Updated weights for policy 0, policy_version 464699 (0.0005) [2023-12-27 08:51:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 237862912. Throughput: 0: 10259.0, 1: 10376.3. Samples: 237867404. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.320')] [2023-12-27 08:51:17,810][20381] Updated weights for policy 1, policy_version 464305 (0.0006) [2023-12-27 08:51:17,870][20381] Updated weights for policy 1, policy_version 464315 (0.0006) [2023-12-27 08:51:17,925][20381] Updated weights for policy 1, policy_version 464325 (0.0010) [2023-12-27 08:51:17,949][20347] Updated weights for policy 0, policy_version 464709 (0.0005) [2023-12-27 08:51:17,982][20381] Updated weights for policy 1, policy_version 464335 (0.0005) [2023-12-27 08:51:18,006][20347] Updated weights for policy 0, policy_version 464719 (0.0005) [2023-12-27 08:51:18,061][20347] Updated weights for policy 0, policy_version 464729 (0.0006) [2023-12-27 08:51:18,648][20381] Updated weights for policy 1, policy_version 464345 (0.0010) [2023-12-27 08:51:18,711][20381] Updated weights for policy 1, policy_version 464355 (0.0011) [2023-12-27 08:51:18,730][20347] Updated weights for policy 0, policy_version 464739 (0.0007) [2023-12-27 08:51:18,769][20381] Updated weights for policy 1, policy_version 464365 (0.0011) [2023-12-27 08:51:18,790][20347] Updated weights for policy 0, policy_version 464749 (0.0005) [2023-12-27 08:51:18,845][20347] Updated weights for policy 0, policy_version 464759 (0.0007) [2023-12-27 08:51:19,417][20381] Updated weights for policy 1, policy_version 464375 (0.0010) [2023-12-27 08:51:19,464][20381] Updated weights for policy 1, policy_version 464385 (0.0008) [2023-12-27 08:51:19,517][20347] Updated weights for policy 0, policy_version 464769 (0.0007) [2023-12-27 08:51:19,521][20381] Updated weights for policy 1, policy_version 464395 (0.0008) [2023-12-27 08:51:19,578][20347] Updated weights for policy 0, policy_version 464779 (0.0009) [2023-12-27 08:51:19,640][20347] Updated weights for policy 0, policy_version 464789 (0.0010) [2023-12-27 08:51:19,700][20347] Updated weights for policy 0, policy_version 464799 (0.0010) [2023-12-27 08:51:20,250][20292] KL-divergence is very high: 151.5373 [2023-12-27 08:51:20,261][20381] Updated weights for policy 1, policy_version 464405 (0.0007) [2023-12-27 08:51:20,321][20381] Updated weights for policy 1, policy_version 464415 (0.0008) [2023-12-27 08:51:20,375][20381] Updated weights for policy 1, policy_version 464425 (0.0008) [2023-12-27 08:51:20,417][20347] Updated weights for policy 0, policy_version 464809 (0.0006) [2023-12-27 08:51:20,485][20347] Updated weights for policy 0, policy_version 464819 (0.0005) [2023-12-27 08:51:20,534][20347] Updated weights for policy 0, policy_version 464829 (0.0008) [2023-12-27 08:51:21,046][20381] Updated weights for policy 1, policy_version 464435 (0.0007) [2023-12-27 08:51:21,114][20381] Updated weights for policy 1, policy_version 464445 (0.0009) [2023-12-27 08:51:21,175][20381] Updated weights for policy 1, policy_version 464455 (0.0008) [2023-12-27 08:51:21,219][20347] Updated weights for policy 0, policy_version 464839 (0.0009) [2023-12-27 08:51:21,281][20347] Updated weights for policy 0, policy_version 464849 (0.0008) [2023-12-27 08:51:21,344][20347] Updated weights for policy 0, policy_version 464859 (0.0007) [2023-12-27 08:51:21,869][20381] Updated weights for policy 1, policy_version 464465 (0.0007) [2023-12-27 08:51:21,922][20381] Updated weights for policy 1, policy_version 464475 (0.0008) [2023-12-27 08:51:21,982][20381] Updated weights for policy 1, policy_version 464485 (0.0008) [2023-12-27 08:51:22,043][20381] Updated weights for policy 1, policy_version 464495 (0.0005) [2023-12-27 08:51:22,072][20347] Updated weights for policy 0, policy_version 464869 (0.0010) [2023-12-27 08:51:22,131][20347] Updated weights for policy 0, policy_version 464879 (0.0011) [2023-12-27 08:51:22,189][20347] Updated weights for policy 0, policy_version 464889 (0.0011) [2023-12-27 08:51:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 237961216. Throughput: 0: 10284.7, 1: 10332.8. Samples: 237928636. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.290')] [2023-12-27 08:51:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000464496_118931456.pth... [2023-12-27 08:51:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000464896_119029760.pth... [2023-12-27 08:51:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000463680_118718464.pth [2023-12-27 08:51:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000463280_118620160.pth [2023-12-27 08:51:22,724][20381] Updated weights for policy 1, policy_version 464505 (0.0007) [2023-12-27 08:51:22,779][20381] Updated weights for policy 1, policy_version 464515 (0.0006) [2023-12-27 08:51:22,832][20381] Updated weights for policy 1, policy_version 464525 (0.0008) [2023-12-27 08:51:22,914][20347] Updated weights for policy 0, policy_version 464899 (0.0010) [2023-12-27 08:51:22,964][20347] Updated weights for policy 0, policy_version 464909 (0.0008) [2023-12-27 08:51:23,013][20347] Updated weights for policy 0, policy_version 464919 (0.0008) [2023-12-27 08:51:23,525][20381] Updated weights for policy 1, policy_version 464535 (0.0008) [2023-12-27 08:51:23,581][20381] Updated weights for policy 1, policy_version 464545 (0.0006) [2023-12-27 08:51:23,633][20381] Updated weights for policy 1, policy_version 464555 (0.0005) [2023-12-27 08:51:23,731][20347] Updated weights for policy 0, policy_version 464929 (0.0008) [2023-12-27 08:51:23,781][20347] Updated weights for policy 0, policy_version 464939 (0.0008) [2023-12-27 08:51:23,838][20347] Updated weights for policy 0, policy_version 464949 (0.0007) [2023-12-27 08:51:23,887][20347] Updated weights for policy 0, policy_version 464959 (0.0008) [2023-12-27 08:51:24,249][20381] Updated weights for policy 1, policy_version 464565 (0.0005) [2023-12-27 08:51:24,313][20381] Updated weights for policy 1, policy_version 464575 (0.0008) [2023-12-27 08:51:24,368][20381] Updated weights for policy 1, policy_version 464585 (0.0008) [2023-12-27 08:51:24,565][20347] Updated weights for policy 0, policy_version 464969 (0.0009) [2023-12-27 08:51:24,620][20347] Updated weights for policy 0, policy_version 464979 (0.0008) [2023-12-27 08:51:24,677][20347] Updated weights for policy 0, policy_version 464989 (0.0007) [2023-12-27 08:51:25,054][20381] Updated weights for policy 1, policy_version 464595 (0.0008) [2023-12-27 08:51:25,120][20381] Updated weights for policy 1, policy_version 464605 (0.0009) [2023-12-27 08:51:25,187][20381] Updated weights for policy 1, policy_version 464615 (0.0008) [2023-12-27 08:51:25,405][20347] Updated weights for policy 0, policy_version 464999 (0.0006) [2023-12-27 08:51:25,456][20347] Updated weights for policy 0, policy_version 465009 (0.0005) [2023-12-27 08:51:25,522][20347] Updated weights for policy 0, policy_version 465019 (0.0005) [2023-12-27 08:51:25,776][20381] Updated weights for policy 1, policy_version 464625 (0.0008) [2023-12-27 08:51:25,832][20381] Updated weights for policy 1, policy_version 464635 (0.0005) [2023-12-27 08:51:25,886][20381] Updated weights for policy 1, policy_version 464645 (0.0005) [2023-12-27 08:51:25,939][20381] Updated weights for policy 1, policy_version 464655 (0.0005) [2023-12-27 08:51:26,159][20347] Updated weights for policy 0, policy_version 465029 (0.0007) [2023-12-27 08:51:26,212][20347] Updated weights for policy 0, policy_version 465039 (0.0005) [2023-12-27 08:51:26,273][20347] Updated weights for policy 0, policy_version 465049 (0.0005) [2023-12-27 08:51:26,557][20381] Updated weights for policy 1, policy_version 464665 (0.0006) [2023-12-27 08:51:26,606][20381] Updated weights for policy 1, policy_version 464675 (0.0005) [2023-12-27 08:51:26,662][20381] Updated weights for policy 1, policy_version 464685 (0.0005) [2023-12-27 08:51:26,949][20347] Updated weights for policy 0, policy_version 465059 (0.0006) [2023-12-27 08:51:27,000][20347] Updated weights for policy 0, policy_version 465069 (0.0009) [2023-12-27 08:51:27,051][20347] Updated weights for policy 0, policy_version 465079 (0.0006) [2023-12-27 08:51:27,200][20381] Updated weights for policy 1, policy_version 464695 (0.0006) [2023-12-27 08:51:27,254][20381] Updated weights for policy 1, policy_version 464705 (0.0005) [2023-12-27 08:51:27,307][20381] Updated weights for policy 1, policy_version 464715 (0.0005) [2023-12-27 08:51:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 238067712. Throughput: 0: 10267.6, 1: 10357.9. Samples: 238050776. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.610')] [2023-12-27 08:51:27,729][20347] Updated weights for policy 0, policy_version 465089 (0.0005) [2023-12-27 08:51:27,773][20347] Updated weights for policy 0, policy_version 465099 (0.0007) [2023-12-27 08:51:27,817][20347] Updated weights for policy 0, policy_version 465109 (0.0007) [2023-12-27 08:51:27,859][20381] Updated weights for policy 1, policy_version 464725 (0.0008) [2023-12-27 08:51:27,869][20347] Updated weights for policy 0, policy_version 465119 (0.0006) [2023-12-27 08:51:27,910][20381] Updated weights for policy 1, policy_version 464735 (0.0010) [2023-12-27 08:51:27,960][20381] Updated weights for policy 1, policy_version 464745 (0.0010) [2023-12-27 08:51:28,531][20381] Updated weights for policy 1, policy_version 464755 (0.0010) [2023-12-27 08:51:28,560][20347] Updated weights for policy 0, policy_version 465129 (0.0007) [2023-12-27 08:51:28,589][20381] Updated weights for policy 1, policy_version 464765 (0.0006) [2023-12-27 08:51:28,614][20347] Updated weights for policy 0, policy_version 465139 (0.0008) [2023-12-27 08:51:28,646][20381] Updated weights for policy 1, policy_version 464775 (0.0005) [2023-12-27 08:51:28,669][20347] Updated weights for policy 0, policy_version 465149 (0.0008) [2023-12-27 08:51:29,240][20381] Updated weights for policy 1, policy_version 464785 (0.0006) [2023-12-27 08:51:29,293][20381] Updated weights for policy 1, policy_version 464795 (0.0006) [2023-12-27 08:51:29,362][20381] Updated weights for policy 1, policy_version 464805 (0.0008) [2023-12-27 08:51:29,409][20381] Updated weights for policy 1, policy_version 464815 (0.0006) [2023-12-27 08:51:29,426][20347] Updated weights for policy 0, policy_version 465159 (0.0007) [2023-12-27 08:51:29,481][20347] Updated weights for policy 0, policy_version 465169 (0.0007) [2023-12-27 08:51:29,534][20347] Updated weights for policy 0, policy_version 465179 (0.0006) [2023-12-27 08:51:30,077][20381] Updated weights for policy 1, policy_version 464825 (0.0005) [2023-12-27 08:51:30,133][20381] Updated weights for policy 1, policy_version 464835 (0.0007) [2023-12-27 08:51:30,196][20381] Updated weights for policy 1, policy_version 464845 (0.0008) [2023-12-27 08:51:30,255][20347] Updated weights for policy 0, policy_version 465189 (0.0007) [2023-12-27 08:51:30,306][20347] Updated weights for policy 0, policy_version 465199 (0.0007) [2023-12-27 08:51:30,359][20347] Updated weights for policy 0, policy_version 465209 (0.0008) [2023-12-27 08:51:30,868][20381] Updated weights for policy 1, policy_version 464855 (0.0008) [2023-12-27 08:51:30,919][20381] Updated weights for policy 1, policy_version 464865 (0.0007) [2023-12-27 08:51:30,974][20381] Updated weights for policy 1, policy_version 464875 (0.0008) [2023-12-27 08:51:31,017][20347] Updated weights for policy 0, policy_version 465219 (0.0008) [2023-12-27 08:51:31,073][20347] Updated weights for policy 0, policy_version 465229 (0.0008) [2023-12-27 08:51:31,137][20347] Updated weights for policy 0, policy_version 465239 (0.0008) [2023-12-27 08:51:31,701][20381] Updated weights for policy 1, policy_version 464885 (0.0006) [2023-12-27 08:51:31,763][20381] Updated weights for policy 1, policy_version 464895 (0.0006) [2023-12-27 08:51:31,818][20347] Updated weights for policy 0, policy_version 465249 (0.0007) [2023-12-27 08:51:31,822][20381] Updated weights for policy 1, policy_version 464905 (0.0008) [2023-12-27 08:51:31,874][20347] Updated weights for policy 0, policy_version 465259 (0.0005) [2023-12-27 08:51:31,928][20347] Updated weights for policy 0, policy_version 465269 (0.0005) [2023-12-27 08:51:31,982][20347] Updated weights for policy 0, policy_version 465279 (0.0005) [2023-12-27 08:51:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 238166016. Throughput: 0: 10229.5, 1: 10433.5. Samples: 238177684. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.550')] [2023-12-27 08:51:32,527][20347] Updated weights for policy 0, policy_version 465289 (0.0005) [2023-12-27 08:51:32,569][20381] Updated weights for policy 1, policy_version 464915 (0.0007) [2023-12-27 08:51:32,583][20347] Updated weights for policy 0, policy_version 465299 (0.0005) [2023-12-27 08:51:32,622][20381] Updated weights for policy 1, policy_version 464925 (0.0007) [2023-12-27 08:51:32,642][20347] Updated weights for policy 0, policy_version 465309 (0.0006) [2023-12-27 08:51:32,675][20381] Updated weights for policy 1, policy_version 464935 (0.0007) [2023-12-27 08:51:33,224][20347] Updated weights for policy 0, policy_version 465319 (0.0005) [2023-12-27 08:51:33,270][20347] Updated weights for policy 0, policy_version 465329 (0.0005) [2023-12-27 08:51:33,321][20347] Updated weights for policy 0, policy_version 465339 (0.0005) [2023-12-27 08:51:33,361][20381] Updated weights for policy 1, policy_version 464945 (0.0007) [2023-12-27 08:51:33,410][20381] Updated weights for policy 1, policy_version 464955 (0.0008) [2023-12-27 08:51:33,459][20381] Updated weights for policy 1, policy_version 464965 (0.0008) [2023-12-27 08:51:33,504][20381] Updated weights for policy 1, policy_version 464975 (0.0007) [2023-12-27 08:51:33,936][20347] Updated weights for policy 0, policy_version 465349 (0.0006) [2023-12-27 08:51:33,981][20347] Updated weights for policy 0, policy_version 465359 (0.0005) [2023-12-27 08:51:34,033][20347] Updated weights for policy 0, policy_version 465369 (0.0005) [2023-12-27 08:51:34,272][20381] Updated weights for policy 1, policy_version 464985 (0.0008) [2023-12-27 08:51:34,331][20381] Updated weights for policy 1, policy_version 464995 (0.0008) [2023-12-27 08:51:34,388][20381] Updated weights for policy 1, policy_version 465005 (0.0009) [2023-12-27 08:51:34,593][20347] Updated weights for policy 0, policy_version 465379 (0.0006) [2023-12-27 08:51:34,652][20347] Updated weights for policy 0, policy_version 465389 (0.0008) [2023-12-27 08:51:34,711][20347] Updated weights for policy 0, policy_version 465399 (0.0007) [2023-12-27 08:51:35,087][20381] Updated weights for policy 1, policy_version 465015 (0.0008) [2023-12-27 08:51:35,132][20381] Updated weights for policy 1, policy_version 465025 (0.0007) [2023-12-27 08:51:35,181][20381] Updated weights for policy 1, policy_version 465035 (0.0008) [2023-12-27 08:51:35,389][20347] Updated weights for policy 0, policy_version 465409 (0.0008) [2023-12-27 08:51:35,444][20347] Updated weights for policy 0, policy_version 465419 (0.0008) [2023-12-27 08:51:35,508][20347] Updated weights for policy 0, policy_version 465429 (0.0007) [2023-12-27 08:51:35,571][20347] Updated weights for policy 0, policy_version 465439 (0.0008) [2023-12-27 08:51:35,855][20381] Updated weights for policy 1, policy_version 465045 (0.0009) [2023-12-27 08:51:35,907][20381] Updated weights for policy 1, policy_version 465055 (0.0010) [2023-12-27 08:51:35,937][20292] KL-divergence is very high: 150.6194 [2023-12-27 08:51:35,957][20381] Updated weights for policy 1, policy_version 465065 (0.0006) [2023-12-27 08:51:36,183][20347] Updated weights for policy 0, policy_version 465449 (0.0006) [2023-12-27 08:51:36,240][20347] Updated weights for policy 0, policy_version 465459 (0.0007) [2023-12-27 08:51:36,306][20347] Updated weights for policy 0, policy_version 465469 (0.0008) [2023-12-27 08:51:36,623][20381] Updated weights for policy 1, policy_version 465075 (0.0006) [2023-12-27 08:51:36,683][20381] Updated weights for policy 1, policy_version 465085 (0.0008) [2023-12-27 08:51:36,690][20292] KL-divergence is very high: 100.2590 [2023-12-27 08:51:36,740][20381] Updated weights for policy 1, policy_version 465095 (0.0008) [2023-12-27 08:51:36,984][20347] Updated weights for policy 0, policy_version 465479 (0.0008) [2023-12-27 08:51:37,032][20347] Updated weights for policy 0, policy_version 465489 (0.0006) [2023-12-27 08:51:37,078][20347] Updated weights for policy 0, policy_version 465499 (0.0005) [2023-12-27 08:51:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 238272512. Throughput: 0: 10312.1, 1: 10368.1. Samples: 238241348. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '932.410')] [2023-12-27 08:51:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000465104_119087104.pth... [2023-12-27 08:51:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000465504_119185408.pth... [2023-12-27 08:51:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000464320_118882304.pth [2023-12-27 08:51:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000463888_118775808.pth [2023-12-27 08:51:37,395][20381] Updated weights for policy 1, policy_version 465105 (0.0007) [2023-12-27 08:51:37,456][20381] Updated weights for policy 1, policy_version 465115 (0.0007) [2023-12-27 08:51:37,484][20292] KL-divergence is very high: 165.4068 [2023-12-27 08:51:37,517][20381] Updated weights for policy 1, policy_version 465125 (0.0007) [2023-12-27 08:51:37,581][20381] Updated weights for policy 1, policy_version 465135 (0.0006) [2023-12-27 08:51:37,717][20347] Updated weights for policy 0, policy_version 465509 (0.0006) [2023-12-27 08:51:37,784][20347] Updated weights for policy 0, policy_version 465519 (0.0008) [2023-12-27 08:51:37,838][20347] Updated weights for policy 0, policy_version 465529 (0.0007) [2023-12-27 08:51:38,149][20381] Updated weights for policy 1, policy_version 465145 (0.0006) [2023-12-27 08:51:38,212][20381] Updated weights for policy 1, policy_version 465155 (0.0006) [2023-12-27 08:51:38,267][20381] Updated weights for policy 1, policy_version 465165 (0.0007) [2023-12-27 08:51:38,546][20347] Updated weights for policy 0, policy_version 465539 (0.0007) [2023-12-27 08:51:38,605][20347] Updated weights for policy 0, policy_version 465549 (0.0007) [2023-12-27 08:51:38,660][20347] Updated weights for policy 0, policy_version 465559 (0.0008) [2023-12-27 08:51:38,938][20381] Updated weights for policy 1, policy_version 465175 (0.0007) [2023-12-27 08:51:38,983][20381] Updated weights for policy 1, policy_version 465185 (0.0008) [2023-12-27 08:51:39,029][20381] Updated weights for policy 1, policy_version 465195 (0.0007) [2023-12-27 08:51:39,346][20347] Updated weights for policy 0, policy_version 465569 (0.0007) [2023-12-27 08:51:39,414][20347] Updated weights for policy 0, policy_version 465579 (0.0009) [2023-12-27 08:51:39,470][20347] Updated weights for policy 0, policy_version 465589 (0.0009) [2023-12-27 08:51:39,526][20347] Updated weights for policy 0, policy_version 465599 (0.0008) [2023-12-27 08:51:39,767][20381] Updated weights for policy 1, policy_version 465205 (0.0009) [2023-12-27 08:51:39,820][20381] Updated weights for policy 1, policy_version 465215 (0.0010) [2023-12-27 08:51:39,878][20381] Updated weights for policy 1, policy_version 465225 (0.0010) [2023-12-27 08:51:40,267][20347] Updated weights for policy 0, policy_version 465609 (0.0010) [2023-12-27 08:51:40,322][20347] Updated weights for policy 0, policy_version 465619 (0.0008) [2023-12-27 08:51:40,379][20347] Updated weights for policy 0, policy_version 465629 (0.0011) [2023-12-27 08:51:40,552][20381] Updated weights for policy 1, policy_version 465235 (0.0011) [2023-12-27 08:51:40,610][20381] Updated weights for policy 1, policy_version 465245 (0.0011) [2023-12-27 08:51:40,668][20381] Updated weights for policy 1, policy_version 465255 (0.0011) [2023-12-27 08:51:41,032][20347] Updated weights for policy 0, policy_version 465639 (0.0011) [2023-12-27 08:51:41,086][20347] Updated weights for policy 0, policy_version 465649 (0.0011) [2023-12-27 08:51:41,154][20347] Updated weights for policy 0, policy_version 465659 (0.0009) [2023-12-27 08:51:41,365][20381] Updated weights for policy 1, policy_version 465265 (0.0011) [2023-12-27 08:51:41,431][20381] Updated weights for policy 1, policy_version 465275 (0.0008) [2023-12-27 08:51:41,484][20381] Updated weights for policy 1, policy_version 465285 (0.0007) [2023-12-27 08:51:41,532][20381] Updated weights for policy 1, policy_version 465295 (0.0007) [2023-12-27 08:51:41,874][20347] Updated weights for policy 0, policy_version 465669 (0.0007) [2023-12-27 08:51:41,921][20347] Updated weights for policy 0, policy_version 465679 (0.0007) [2023-12-27 08:51:41,943][20291] KL-divergence is very high: 276.8824 [2023-12-27 08:51:41,978][20347] Updated weights for policy 0, policy_version 465689 (0.0007) [2023-12-27 08:51:42,245][20381] Updated weights for policy 1, policy_version 465305 (0.0008) [2023-12-27 08:51:42,301][20381] Updated weights for policy 1, policy_version 465315 (0.0007) [2023-12-27 08:51:42,359][20381] Updated weights for policy 1, policy_version 465325 (0.0008) [2023-12-27 08:51:42,364][19170] Fps is (10 sec: 20477.3, 60 sec: 20752.6, 300 sec: 20716.0). Total num frames: 238370816. Throughput: 0: 10327.3, 1: 10378.6. Samples: 238365492. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:42,365][19170] Avg episode reward: [(0, '990.520'), (1, '951.880')] [2023-12-27 08:51:42,655][20347] Updated weights for policy 0, policy_version 465699 (0.0006) [2023-12-27 08:51:42,719][20347] Updated weights for policy 0, policy_version 465709 (0.0007) [2023-12-27 08:51:42,770][20347] Updated weights for policy 0, policy_version 465719 (0.0007) [2023-12-27 08:51:42,994][20381] Updated weights for policy 1, policy_version 465335 (0.0007) [2023-12-27 08:51:43,052][20381] Updated weights for policy 1, policy_version 465345 (0.0007) [2023-12-27 08:51:43,111][20381] Updated weights for policy 1, policy_version 465355 (0.0008) [2023-12-27 08:51:43,412][20347] Updated weights for policy 0, policy_version 465729 (0.0007) [2023-12-27 08:51:43,466][20347] Updated weights for policy 0, policy_version 465739 (0.0008) [2023-12-27 08:51:43,520][20347] Updated weights for policy 0, policy_version 465749 (0.0007) [2023-12-27 08:51:43,575][20347] Updated weights for policy 0, policy_version 465759 (0.0007) [2023-12-27 08:51:43,814][20381] Updated weights for policy 1, policy_version 465365 (0.0008) [2023-12-27 08:51:43,864][20381] Updated weights for policy 1, policy_version 465375 (0.0008) [2023-12-27 08:51:43,917][20381] Updated weights for policy 1, policy_version 465385 (0.0007) [2023-12-27 08:51:44,240][20347] Updated weights for policy 0, policy_version 465769 (0.0006) [2023-12-27 08:51:44,300][20347] Updated weights for policy 0, policy_version 465779 (0.0008) [2023-12-27 08:51:44,352][20347] Updated weights for policy 0, policy_version 465789 (0.0008) [2023-12-27 08:51:44,567][20381] Updated weights for policy 1, policy_version 465395 (0.0005) [2023-12-27 08:51:44,620][20381] Updated weights for policy 1, policy_version 465405 (0.0005) [2023-12-27 08:51:44,677][20381] Updated weights for policy 1, policy_version 465415 (0.0006) [2023-12-27 08:51:45,057][20347] Updated weights for policy 0, policy_version 465799 (0.0007) [2023-12-27 08:51:45,117][20347] Updated weights for policy 0, policy_version 465809 (0.0007) [2023-12-27 08:51:45,170][20347] Updated weights for policy 0, policy_version 465819 (0.0007) [2023-12-27 08:51:45,366][20381] Updated weights for policy 1, policy_version 465425 (0.0008) [2023-12-27 08:51:45,420][20381] Updated weights for policy 1, policy_version 465435 (0.0008) [2023-12-27 08:51:45,483][20381] Updated weights for policy 1, policy_version 465445 (0.0008) [2023-12-27 08:51:45,543][20381] Updated weights for policy 1, policy_version 465455 (0.0008) [2023-12-27 08:51:45,792][20347] Updated weights for policy 0, policy_version 465829 (0.0007) [2023-12-27 08:51:45,841][20347] Updated weights for policy 0, policy_version 465839 (0.0005) [2023-12-27 08:51:45,895][20347] Updated weights for policy 0, policy_version 465849 (0.0005) [2023-12-27 08:51:46,239][20381] Updated weights for policy 1, policy_version 465465 (0.0009) [2023-12-27 08:51:46,292][20381] Updated weights for policy 1, policy_version 465475 (0.0010) [2023-12-27 08:51:46,353][20381] Updated weights for policy 1, policy_version 465485 (0.0008) [2023-12-27 08:51:46,415][20347] Updated weights for policy 0, policy_version 465859 (0.0005) [2023-12-27 08:51:46,467][20347] Updated weights for policy 0, policy_version 465869 (0.0005) [2023-12-27 08:51:46,513][20347] Updated weights for policy 0, policy_version 465879 (0.0005) [2023-12-27 08:51:47,056][20381] Updated weights for policy 1, policy_version 465495 (0.0006) [2023-12-27 08:51:47,063][20347] Updated weights for policy 0, policy_version 465889 (0.0006) [2023-12-27 08:51:47,073][20292] KL-divergence is very high: 347.8377 [2023-12-27 08:51:47,110][20347] Updated weights for policy 0, policy_version 465899 (0.0008) [2023-12-27 08:51:47,115][20381] Updated weights for policy 1, policy_version 465505 (0.0005) [2023-12-27 08:51:47,155][20347] Updated weights for policy 0, policy_version 465909 (0.0006) [2023-12-27 08:51:47,173][20381] Updated weights for policy 1, policy_version 465515 (0.0005) [2023-12-27 08:51:47,199][20347] Updated weights for policy 0, policy_version 465919 (0.0008) [2023-12-27 08:51:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 238485504. Throughput: 0: 10384.2, 1: 10391.6. Samples: 238491648. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:47,363][19170] Avg episode reward: [(0, '961.890'), (1, '923.630')] [2023-12-27 08:51:47,768][20381] Updated weights for policy 1, policy_version 465525 (0.0008) [2023-12-27 08:51:47,832][20381] Updated weights for policy 1, policy_version 465535 (0.0010) [2023-12-27 08:51:47,856][20347] Updated weights for policy 0, policy_version 465929 (0.0005) [2023-12-27 08:51:47,890][20381] Updated weights for policy 1, policy_version 465545 (0.0010) [2023-12-27 08:51:47,916][20347] Updated weights for policy 0, policy_version 465939 (0.0007) [2023-12-27 08:51:47,977][20347] Updated weights for policy 0, policy_version 465949 (0.0010) [2023-12-27 08:51:48,489][20381] Updated weights for policy 1, policy_version 465555 (0.0009) [2023-12-27 08:51:48,534][20381] Updated weights for policy 1, policy_version 465565 (0.0007) [2023-12-27 08:51:48,561][20292] KL-divergence is very high: 146.6197 [2023-12-27 08:51:48,587][20381] Updated weights for policy 1, policy_version 465575 (0.0008) [2023-12-27 08:51:48,589][20347] Updated weights for policy 0, policy_version 465959 (0.0007) [2023-12-27 08:51:48,645][20347] Updated weights for policy 0, policy_version 465969 (0.0006) [2023-12-27 08:51:48,710][20347] Updated weights for policy 0, policy_version 465979 (0.0007) [2023-12-27 08:51:48,727][20291] KL-divergence is very high: 1172.0535 [2023-12-27 08:51:49,230][20381] Updated weights for policy 1, policy_version 465585 (0.0008) [2023-12-27 08:51:49,290][20381] Updated weights for policy 1, policy_version 465595 (0.0011) [2023-12-27 08:51:49,333][20292] KL-divergence is very high: 192.2716 [2023-12-27 08:51:49,353][20381] Updated weights for policy 1, policy_version 465605 (0.0008) [2023-12-27 08:51:49,359][20292] KL-divergence is very high: 564.3342 [2023-12-27 08:51:49,374][20291] KL-divergence is very high: 124.4363 [2023-12-27 08:51:49,379][20347] Updated weights for policy 0, policy_version 465989 (0.0008) [2023-12-27 08:51:49,381][20291] KL-divergence is very high: 190.8867 [2023-12-27 08:51:49,385][20292] KL-divergence is very high: 114.6277 [2023-12-27 08:51:49,388][20291] KL-divergence is very high: 125.1843 [2023-12-27 08:51:49,408][20292] KL-divergence is very high: 215.0251 [2023-12-27 08:51:49,412][20381] Updated weights for policy 1, policy_version 465615 (0.0009) [2023-12-27 08:51:49,440][20347] Updated weights for policy 0, policy_version 465999 (0.0006) [2023-12-27 08:51:49,501][20347] Updated weights for policy 0, policy_version 466009 (0.0007) [2023-12-27 08:51:50,006][20292] KL-divergence is very high: 163.0649 [2023-12-27 08:51:50,062][20381] Updated weights for policy 1, policy_version 465625 (0.0008) [2023-12-27 08:51:50,110][20347] Updated weights for policy 0, policy_version 466019 (0.0006) [2023-12-27 08:51:50,122][20381] Updated weights for policy 1, policy_version 465635 (0.0007) [2023-12-27 08:51:50,167][20347] Updated weights for policy 0, policy_version 466029 (0.0006) [2023-12-27 08:51:50,180][20381] Updated weights for policy 1, policy_version 465645 (0.0008) [2023-12-27 08:51:50,189][20292] KL-divergence is very high: 120.1784 [2023-12-27 08:51:50,223][20347] Updated weights for policy 0, policy_version 466039 (0.0006) [2023-12-27 08:51:50,837][20347] Updated weights for policy 0, policy_version 466049 (0.0005) [2023-12-27 08:51:50,879][20381] Updated weights for policy 1, policy_version 465655 (0.0008) [2023-12-27 08:51:50,899][20347] Updated weights for policy 0, policy_version 466059 (0.0007) [2023-12-27 08:51:50,933][20291] KL-divergence is very high: 109.1404 [2023-12-27 08:51:50,934][20381] Updated weights for policy 1, policy_version 465665 (0.0009) [2023-12-27 08:51:50,940][20291] KL-divergence is very high: 220.6086 [2023-12-27 08:51:50,956][20347] Updated weights for policy 0, policy_version 466069 (0.0006) [2023-12-27 08:51:50,983][20381] Updated weights for policy 1, policy_version 465675 (0.0008) [2023-12-27 08:51:50,993][20291] KL-divergence is very high: 100.9870 [2023-12-27 08:51:50,998][20291] KL-divergence is very high: 120.0444 [2023-12-27 08:51:51,018][20347] Updated weights for policy 0, policy_version 466079 (0.0006) [2023-12-27 08:51:51,643][20381] Updated weights for policy 1, policy_version 465685 (0.0008) [2023-12-27 08:51:51,698][20291] KL-divergence is very high: 163.0461 [2023-12-27 08:51:51,712][20381] Updated weights for policy 1, policy_version 465695 (0.0008) [2023-12-27 08:51:51,719][20347] Updated weights for policy 0, policy_version 466089 (0.0007) [2023-12-27 08:51:51,768][20381] Updated weights for policy 1, policy_version 465705 (0.0007) [2023-12-27 08:51:51,778][20347] Updated weights for policy 0, policy_version 466099 (0.0006) [2023-12-27 08:51:51,793][20291] KL-divergence is very high: 134.2616 [2023-12-27 08:51:51,798][20291] KL-divergence is very high: 275.9229 [2023-12-27 08:51:51,824][20291] KL-divergence is very high: 153.0196 [2023-12-27 08:51:51,834][20291] KL-divergence is very high: 234.5507 [2023-12-27 08:51:51,835][20347] Updated weights for policy 0, policy_version 466109 (0.0006) [2023-12-27 08:51:51,839][20291] KL-divergence is very high: 174.4973 [2023-12-27 08:51:52,340][20381] Updated weights for policy 1, policy_version 465715 (0.0005) [2023-12-27 08:51:52,362][19170] Fps is (10 sec: 21301.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 238583808. Throughput: 0: 10381.7, 1: 10422.2. Samples: 238555860. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:52,363][19170] Avg episode reward: [(0, '497.200'), (1, '532.190')] [2023-12-27 08:51:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000466112_119341056.pth... [2023-12-27 08:51:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000464896_119029760.pth [2023-12-27 08:51:52,404][20381] Updated weights for policy 1, policy_version 465725 (0.0008) [2023-12-27 08:51:52,457][20381] Updated weights for policy 1, policy_version 465735 (0.0008) [2023-12-27 08:51:52,501][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000465744_119250944.pth... [2023-12-27 08:51:52,506][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000464496_118931456.pth [2023-12-27 08:51:52,539][20291] KL-divergence is very high: 280.2458 [2023-12-27 08:51:52,544][20291] KL-divergence is very high: 157.1637 [2023-12-27 08:51:52,550][20291] KL-divergence is very high: 617.2709 [2023-12-27 08:51:52,555][20291] KL-divergence is very high: 109.9375 [2023-12-27 08:51:52,565][20291] KL-divergence is very high: 203.2164 [2023-12-27 08:51:52,568][20347] Updated weights for policy 0, policy_version 466119 (0.0009) [2023-12-27 08:51:52,583][20291] KL-divergence is very high: 397.0382 [2023-12-27 08:51:52,608][20291] KL-divergence is very high: 164.3969 [2023-12-27 08:51:52,618][20347] Updated weights for policy 0, policy_version 466129 (0.0008) [2023-12-27 08:51:52,623][20291] KL-divergence is very high: 145.8484 [2023-12-27 08:51:52,634][20291] KL-divergence is very high: 198.9096 [2023-12-27 08:51:52,652][20291] KL-divergence is very high: 142.6622 [2023-12-27 08:51:52,668][20291] KL-divergence is very high: 221.4632 [2023-12-27 08:51:52,675][20347] Updated weights for policy 0, policy_version 466139 (0.0009) [2023-12-27 08:51:52,680][20291] KL-divergence is very high: 139.8458 [2023-12-27 08:51:52,694][20291] KL-divergence is very high: 168.7769 [2023-12-27 08:51:53,092][20381] Updated weights for policy 1, policy_version 465745 (0.0006) [2023-12-27 08:51:53,155][20381] Updated weights for policy 1, policy_version 465755 (0.0010) [2023-12-27 08:51:53,220][20381] Updated weights for policy 1, policy_version 465765 (0.0011) [2023-12-27 08:51:53,279][20381] Updated weights for policy 1, policy_version 465775 (0.0011) [2023-12-27 08:51:53,323][20291] KL-divergence is very high: 140.1311 [2023-12-27 08:51:53,329][20291] KL-divergence is very high: 129.2577 [2023-12-27 08:51:53,344][20347] Updated weights for policy 0, policy_version 466149 (0.0007) [2023-12-27 08:51:53,353][20291] KL-divergence is very high: 136.1819 [2023-12-27 08:51:53,369][20291] KL-divergence is very high: 148.8527 [2023-12-27 08:51:53,374][20291] KL-divergence is very high: 115.3443 [2023-12-27 08:51:53,400][20347] Updated weights for policy 0, policy_version 466159 (0.0005) [2023-12-27 08:51:53,410][20291] KL-divergence is very high: 172.9464 [2023-12-27 08:51:53,415][20291] KL-divergence is very high: 205.6457 [2023-12-27 08:51:53,454][20347] Updated weights for policy 0, policy_version 466169 (0.0005) [2023-12-27 08:51:53,457][20291] KL-divergence is very high: 111.4260 [2023-12-27 08:51:53,463][20291] KL-divergence is very high: 107.2136 [2023-12-27 08:51:53,932][20381] Updated weights for policy 1, policy_version 465785 (0.0010) [2023-12-27 08:51:53,981][20381] Updated weights for policy 1, policy_version 465795 (0.0010) [2023-12-27 08:51:54,034][20381] Updated weights for policy 1, policy_version 465805 (0.0010) [2023-12-27 08:51:54,092][20347] Updated weights for policy 0, policy_version 466179 (0.0007) [2023-12-27 08:51:54,145][20347] Updated weights for policy 0, policy_version 466189 (0.0010) [2023-12-27 08:51:54,200][20347] Updated weights for policy 0, policy_version 466199 (0.0010) [2023-12-27 08:51:54,680][20381] Updated weights for policy 1, policy_version 465815 (0.0009) [2023-12-27 08:51:54,730][20381] Updated weights for policy 1, policy_version 465825 (0.0007) [2023-12-27 08:51:54,787][20381] Updated weights for policy 1, policy_version 465835 (0.0005) [2023-12-27 08:51:54,889][20347] Updated weights for policy 0, policy_version 466209 (0.0010) [2023-12-27 08:51:54,944][20347] Updated weights for policy 0, policy_version 466219 (0.0010) [2023-12-27 08:51:55,004][20347] Updated weights for policy 0, policy_version 466229 (0.0011) [2023-12-27 08:51:55,057][20347] Updated weights for policy 0, policy_version 466239 (0.0010) [2023-12-27 08:51:55,422][20381] Updated weights for policy 1, policy_version 465845 (0.0007) [2023-12-27 08:51:55,478][20381] Updated weights for policy 1, policy_version 465855 (0.0006) [2023-12-27 08:51:55,527][20381] Updated weights for policy 1, policy_version 465865 (0.0007) [2023-12-27 08:51:55,703][20347] Updated weights for policy 0, policy_version 466249 (0.0006) [2023-12-27 08:51:55,754][20347] Updated weights for policy 0, policy_version 466259 (0.0007) [2023-12-27 08:51:55,809][20347] Updated weights for policy 0, policy_version 466269 (0.0007) [2023-12-27 08:51:56,186][20381] Updated weights for policy 1, policy_version 465875 (0.0009) [2023-12-27 08:51:56,239][20381] Updated weights for policy 1, policy_version 465885 (0.0009) [2023-12-27 08:51:56,297][20381] Updated weights for policy 1, policy_version 465895 (0.0010) [2023-12-27 08:51:56,438][20347] Updated weights for policy 0, policy_version 466279 (0.0006) [2023-12-27 08:51:56,485][20347] Updated weights for policy 0, policy_version 466289 (0.0006) [2023-12-27 08:51:56,539][20347] Updated weights for policy 0, policy_version 466299 (0.0007) [2023-12-27 08:51:56,925][20381] Updated weights for policy 1, policy_version 465905 (0.0010) [2023-12-27 08:51:56,988][20381] Updated weights for policy 1, policy_version 465915 (0.0005) [2023-12-27 08:51:57,040][20381] Updated weights for policy 1, policy_version 465925 (0.0005) [2023-12-27 08:51:57,093][20381] Updated weights for policy 1, policy_version 465935 (0.0005) [2023-12-27 08:51:57,207][20347] Updated weights for policy 0, policy_version 466309 (0.0009) [2023-12-27 08:51:57,255][20347] Updated weights for policy 0, policy_version 466319 (0.0010) [2023-12-27 08:51:57,303][20347] Updated weights for policy 0, policy_version 466329 (0.0010) [2023-12-27 08:51:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 238698496. Throughput: 0: 10358.6, 1: 10511.1. Samples: 238682828. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:51:57,363][19170] Avg episode reward: [(0, '305.550'), (1, '288.060')] [2023-12-27 08:51:57,699][20381] Updated weights for policy 1, policy_version 465945 (0.0008) [2023-12-27 08:51:57,753][20381] Updated weights for policy 1, policy_version 465955 (0.0008) [2023-12-27 08:51:57,803][20381] Updated weights for policy 1, policy_version 465965 (0.0007) [2023-12-27 08:51:57,958][20347] Updated weights for policy 0, policy_version 466339 (0.0010) [2023-12-27 08:51:58,009][20347] Updated weights for policy 0, policy_version 466349 (0.0010) [2023-12-27 08:51:58,063][20347] Updated weights for policy 0, policy_version 466359 (0.0010) [2023-12-27 08:51:58,475][20381] Updated weights for policy 1, policy_version 465975 (0.0007) [2023-12-27 08:51:58,535][20381] Updated weights for policy 1, policy_version 465985 (0.0008) [2023-12-27 08:51:58,597][20381] Updated weights for policy 1, policy_version 465995 (0.0008) [2023-12-27 08:51:58,773][20347] Updated weights for policy 0, policy_version 466369 (0.0010) [2023-12-27 08:51:58,844][20347] Updated weights for policy 0, policy_version 466379 (0.0009) [2023-12-27 08:51:58,911][20347] Updated weights for policy 0, policy_version 466389 (0.0009) [2023-12-27 08:51:58,970][20347] Updated weights for policy 0, policy_version 466399 (0.0010) [2023-12-27 08:51:59,321][20381] Updated weights for policy 1, policy_version 466005 (0.0008) [2023-12-27 08:51:59,379][20381] Updated weights for policy 1, policy_version 466015 (0.0008) [2023-12-27 08:51:59,427][20381] Updated weights for policy 1, policy_version 466025 (0.0007) [2023-12-27 08:51:59,682][20347] Updated weights for policy 0, policy_version 466409 (0.0010) [2023-12-27 08:51:59,747][20347] Updated weights for policy 0, policy_version 466419 (0.0011) [2023-12-27 08:51:59,799][20347] Updated weights for policy 0, policy_version 466429 (0.0010) [2023-12-27 08:52:00,096][20381] Updated weights for policy 1, policy_version 466035 (0.0008) [2023-12-27 08:52:00,151][20381] Updated weights for policy 1, policy_version 466045 (0.0010) [2023-12-27 08:52:00,206][20381] Updated weights for policy 1, policy_version 466055 (0.0010) [2023-12-27 08:52:00,478][20347] Updated weights for policy 0, policy_version 466439 (0.0010) [2023-12-27 08:52:00,536][20347] Updated weights for policy 0, policy_version 466449 (0.0010) [2023-12-27 08:52:00,591][20347] Updated weights for policy 0, policy_version 466459 (0.0010) [2023-12-27 08:52:00,875][20381] Updated weights for policy 1, policy_version 466065 (0.0010) [2023-12-27 08:52:00,937][20381] Updated weights for policy 1, policy_version 466075 (0.0011) [2023-12-27 08:52:00,995][20381] Updated weights for policy 1, policy_version 466085 (0.0011) [2023-12-27 08:52:01,054][20381] Updated weights for policy 1, policy_version 466095 (0.0011) [2023-12-27 08:52:01,278][20347] Updated weights for policy 0, policy_version 466469 (0.0010) [2023-12-27 08:52:01,333][20347] Updated weights for policy 0, policy_version 466479 (0.0010) [2023-12-27 08:52:01,399][20347] Updated weights for policy 0, policy_version 466489 (0.0011) [2023-12-27 08:52:01,754][20381] Updated weights for policy 1, policy_version 466105 (0.0008) [2023-12-27 08:52:01,810][20381] Updated weights for policy 1, policy_version 466115 (0.0007) [2023-12-27 08:52:01,869][20381] Updated weights for policy 1, policy_version 466125 (0.0007) [2023-12-27 08:52:02,094][20347] Updated weights for policy 0, policy_version 466499 (0.0008) [2023-12-27 08:52:02,151][20347] Updated weights for policy 0, policy_version 466509 (0.0008) [2023-12-27 08:52:02,204][20347] Updated weights for policy 0, policy_version 466519 (0.0009) [2023-12-27 08:52:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 238796800. Throughput: 0: 10396.6, 1: 10454.5. Samples: 238805704. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:02,363][19170] Avg episode reward: [(0, '468.640'), (1, '529.790')] [2023-12-27 08:52:02,472][20381] Updated weights for policy 1, policy_version 466135 (0.0006) [2023-12-27 08:52:02,521][20381] Updated weights for policy 1, policy_version 466145 (0.0005) [2023-12-27 08:52:02,572][20381] Updated weights for policy 1, policy_version 466155 (0.0005) [2023-12-27 08:52:02,960][20347] Updated weights for policy 0, policy_version 466529 (0.0008) [2023-12-27 08:52:03,014][20347] Updated weights for policy 0, policy_version 466539 (0.0007) [2023-12-27 08:52:03,059][20347] Updated weights for policy 0, policy_version 466549 (0.0010) [2023-12-27 08:52:03,108][20347] Updated weights for policy 0, policy_version 466559 (0.0010) [2023-12-27 08:52:03,156][20381] Updated weights for policy 1, policy_version 466165 (0.0005) [2023-12-27 08:52:03,207][20381] Updated weights for policy 1, policy_version 466175 (0.0005) [2023-12-27 08:52:03,251][20381] Updated weights for policy 1, policy_version 466185 (0.0005) [2023-12-27 08:52:03,789][20347] Updated weights for policy 0, policy_version 466569 (0.0005) [2023-12-27 08:52:03,838][20381] Updated weights for policy 1, policy_version 466195 (0.0005) [2023-12-27 08:52:03,849][20347] Updated weights for policy 0, policy_version 466579 (0.0007) [2023-12-27 08:52:03,897][20381] Updated weights for policy 1, policy_version 466205 (0.0006) [2023-12-27 08:52:03,910][20347] Updated weights for policy 0, policy_version 466589 (0.0008) [2023-12-27 08:52:03,958][20381] Updated weights for policy 1, policy_version 466215 (0.0007) [2023-12-27 08:52:04,572][20381] Updated weights for policy 1, policy_version 466225 (0.0007) [2023-12-27 08:52:04,596][20347] Updated weights for policy 0, policy_version 466599 (0.0006) [2023-12-27 08:52:04,636][20381] Updated weights for policy 1, policy_version 466235 (0.0007) [2023-12-27 08:52:04,651][20347] Updated weights for policy 0, policy_version 466609 (0.0005) [2023-12-27 08:52:04,699][20381] Updated weights for policy 1, policy_version 466245 (0.0008) [2023-12-27 08:52:04,710][20347] Updated weights for policy 0, policy_version 466619 (0.0005) [2023-12-27 08:52:04,759][20381] Updated weights for policy 1, policy_version 466255 (0.0008) [2023-12-27 08:52:05,276][20347] Updated weights for policy 0, policy_version 466629 (0.0006) [2023-12-27 08:52:05,333][20347] Updated weights for policy 0, policy_version 466639 (0.0006) [2023-12-27 08:52:05,385][20347] Updated weights for policy 0, policy_version 466649 (0.0009) [2023-12-27 08:52:05,461][20381] Updated weights for policy 1, policy_version 466265 (0.0006) [2023-12-27 08:52:05,512][20381] Updated weights for policy 1, policy_version 466275 (0.0008) [2023-12-27 08:52:05,570][20381] Updated weights for policy 1, policy_version 466285 (0.0007) [2023-12-27 08:52:06,054][20347] Updated weights for policy 0, policy_version 466659 (0.0010) [2023-12-27 08:52:06,113][20347] Updated weights for policy 0, policy_version 466669 (0.0010) [2023-12-27 08:52:06,173][20347] Updated weights for policy 0, policy_version 466679 (0.0011) [2023-12-27 08:52:06,277][20381] Updated weights for policy 1, policy_version 466295 (0.0007) [2023-12-27 08:52:06,335][20381] Updated weights for policy 1, policy_version 466305 (0.0005) [2023-12-27 08:52:06,384][20381] Updated weights for policy 1, policy_version 466315 (0.0007) [2023-12-27 08:52:06,865][20347] Updated weights for policy 0, policy_version 466689 (0.0010) [2023-12-27 08:52:06,921][20347] Updated weights for policy 0, policy_version 466699 (0.0011) [2023-12-27 08:52:06,980][20347] Updated weights for policy 0, policy_version 466709 (0.0010) [2023-12-27 08:52:07,042][20347] Updated weights for policy 0, policy_version 466719 (0.0011) [2023-12-27 08:52:07,084][20381] Updated weights for policy 1, policy_version 466325 (0.0007) [2023-12-27 08:52:07,129][20381] Updated weights for policy 1, policy_version 466335 (0.0007) [2023-12-27 08:52:07,174][20381] Updated weights for policy 1, policy_version 466345 (0.0007) [2023-12-27 08:52:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 238903296. Throughput: 0: 10398.2, 1: 10507.2. Samples: 238869380. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:07,363][19170] Avg episode reward: [(0, '539.430'), (1, '519.700')] [2023-12-27 08:52:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000466352_119406592.pth... [2023-12-27 08:52:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000466720_119496704.pth... [2023-12-27 08:52:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000465104_119087104.pth [2023-12-27 08:52:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000465504_119185408.pth [2023-12-27 08:52:07,717][20347] Updated weights for policy 0, policy_version 466729 (0.0010) [2023-12-27 08:52:07,777][20347] Updated weights for policy 0, policy_version 466739 (0.0008) [2023-12-27 08:52:07,829][20347] Updated weights for policy 0, policy_version 466749 (0.0007) [2023-12-27 08:52:07,887][20381] Updated weights for policy 1, policy_version 466355 (0.0007) [2023-12-27 08:52:07,938][20381] Updated weights for policy 1, policy_version 466365 (0.0007) [2023-12-27 08:52:07,996][20381] Updated weights for policy 1, policy_version 466375 (0.0007) [2023-12-27 08:52:08,525][20347] Updated weights for policy 0, policy_version 466759 (0.0008) [2023-12-27 08:52:08,588][20347] Updated weights for policy 0, policy_version 466769 (0.0006) [2023-12-27 08:52:08,649][20347] Updated weights for policy 0, policy_version 466779 (0.0007) [2023-12-27 08:52:08,682][20381] Updated weights for policy 1, policy_version 466385 (0.0007) [2023-12-27 08:52:08,740][20381] Updated weights for policy 1, policy_version 466395 (0.0005) [2023-12-27 08:52:08,801][20381] Updated weights for policy 1, policy_version 466405 (0.0006) [2023-12-27 08:52:08,859][20381] Updated weights for policy 1, policy_version 466415 (0.0007) [2023-12-27 08:52:09,329][20347] Updated weights for policy 0, policy_version 466789 (0.0008) [2023-12-27 08:52:09,400][20347] Updated weights for policy 0, policy_version 466799 (0.0008) [2023-12-27 08:52:09,460][20347] Updated weights for policy 0, policy_version 466809 (0.0010) [2023-12-27 08:52:09,477][20381] Updated weights for policy 1, policy_version 466425 (0.0006) [2023-12-27 08:52:09,538][20381] Updated weights for policy 1, policy_version 466435 (0.0007) [2023-12-27 08:52:09,599][20381] Updated weights for policy 1, policy_version 466445 (0.0007) [2023-12-27 08:52:10,155][20347] Updated weights for policy 0, policy_version 466819 (0.0010) [2023-12-27 08:52:10,216][20347] Updated weights for policy 0, policy_version 466829 (0.0010) [2023-12-27 08:52:10,268][20381] Updated weights for policy 1, policy_version 466455 (0.0007) [2023-12-27 08:52:10,277][20347] Updated weights for policy 0, policy_version 466839 (0.0011) [2023-12-27 08:52:10,321][20381] Updated weights for policy 1, policy_version 466465 (0.0006) [2023-12-27 08:52:10,379][20381] Updated weights for policy 1, policy_version 466475 (0.0007) [2023-12-27 08:52:10,909][20347] Updated weights for policy 0, policy_version 466849 (0.0011) [2023-12-27 08:52:10,963][20347] Updated weights for policy 0, policy_version 466859 (0.0010) [2023-12-27 08:52:11,015][20347] Updated weights for policy 0, policy_version 466869 (0.0010) [2023-12-27 08:52:11,078][20347] Updated weights for policy 0, policy_version 466879 (0.0008) [2023-12-27 08:52:11,105][20381] Updated weights for policy 1, policy_version 466485 (0.0008) [2023-12-27 08:52:11,171][20381] Updated weights for policy 1, policy_version 466495 (0.0008) [2023-12-27 08:52:11,231][20381] Updated weights for policy 1, policy_version 466505 (0.0007) [2023-12-27 08:52:11,787][20347] Updated weights for policy 0, policy_version 466889 (0.0006) [2023-12-27 08:52:11,847][20347] Updated weights for policy 0, policy_version 466899 (0.0008) [2023-12-27 08:52:11,916][20347] Updated weights for policy 0, policy_version 466909 (0.0007) [2023-12-27 08:52:11,921][20381] Updated weights for policy 1, policy_version 466515 (0.0008) [2023-12-27 08:52:11,980][20381] Updated weights for policy 1, policy_version 466525 (0.0007) [2023-12-27 08:52:12,042][20381] Updated weights for policy 1, policy_version 466535 (0.0007) [2023-12-27 08:52:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 239001600. Throughput: 0: 10423.4, 1: 10464.3. Samples: 238990724. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:12,363][19170] Avg episode reward: [(0, '486.240'), (1, '663.800')] [2023-12-27 08:52:12,580][20347] Updated weights for policy 0, policy_version 466919 (0.0007) [2023-12-27 08:52:12,634][20347] Updated weights for policy 0, policy_version 466929 (0.0008) [2023-12-27 08:52:12,689][20347] Updated weights for policy 0, policy_version 466939 (0.0006) [2023-12-27 08:52:12,730][20381] Updated weights for policy 1, policy_version 466545 (0.0006) [2023-12-27 08:52:12,798][20381] Updated weights for policy 1, policy_version 466555 (0.0008) [2023-12-27 08:52:12,863][20381] Updated weights for policy 1, policy_version 466565 (0.0007) [2023-12-27 08:52:12,927][20381] Updated weights for policy 1, policy_version 466575 (0.0008) [2023-12-27 08:52:13,372][20347] Updated weights for policy 0, policy_version 466949 (0.0009) [2023-12-27 08:52:13,418][20347] Updated weights for policy 0, policy_version 466959 (0.0010) [2023-12-27 08:52:13,470][20347] Updated weights for policy 0, policy_version 466969 (0.0010) [2023-12-27 08:52:13,524][20381] Updated weights for policy 1, policy_version 466585 (0.0005) [2023-12-27 08:52:13,572][20381] Updated weights for policy 1, policy_version 466595 (0.0005) [2023-12-27 08:52:13,629][20381] Updated weights for policy 1, policy_version 466605 (0.0008) [2023-12-27 08:52:14,159][20347] Updated weights for policy 0, policy_version 466979 (0.0010) [2023-12-27 08:52:14,215][20347] Updated weights for policy 0, policy_version 466989 (0.0010) [2023-12-27 08:52:14,273][20347] Updated weights for policy 0, policy_version 466999 (0.0010) [2023-12-27 08:52:14,316][20381] Updated weights for policy 1, policy_version 466615 (0.0009) [2023-12-27 08:52:14,368][20381] Updated weights for policy 1, policy_version 466625 (0.0009) [2023-12-27 08:52:14,413][20381] Updated weights for policy 1, policy_version 466635 (0.0008) [2023-12-27 08:52:14,906][20347] Updated weights for policy 0, policy_version 467009 (0.0006) [2023-12-27 08:52:14,965][20347] Updated weights for policy 0, policy_version 467019 (0.0006) [2023-12-27 08:52:15,026][20347] Updated weights for policy 0, policy_version 467029 (0.0006) [2023-12-27 08:52:15,074][20347] Updated weights for policy 0, policy_version 467039 (0.0006) [2023-12-27 08:52:15,182][20381] Updated weights for policy 1, policy_version 466645 (0.0009) [2023-12-27 08:52:15,237][20381] Updated weights for policy 1, policy_version 466655 (0.0008) [2023-12-27 08:52:15,290][20381] Updated weights for policy 1, policy_version 466665 (0.0006) [2023-12-27 08:52:15,707][20347] Updated weights for policy 0, policy_version 467049 (0.0006) [2023-12-27 08:52:15,753][20347] Updated weights for policy 0, policy_version 467059 (0.0007) [2023-12-27 08:52:15,801][20347] Updated weights for policy 0, policy_version 467069 (0.0007) [2023-12-27 08:52:15,902][20381] Updated weights for policy 1, policy_version 466675 (0.0005) [2023-12-27 08:52:15,961][20381] Updated weights for policy 1, policy_version 466685 (0.0005) [2023-12-27 08:52:16,009][20381] Updated weights for policy 1, policy_version 466695 (0.0005) [2023-12-27 08:52:16,394][20347] Updated weights for policy 0, policy_version 467079 (0.0008) [2023-12-27 08:52:16,453][20347] Updated weights for policy 0, policy_version 467089 (0.0008) [2023-12-27 08:52:16,513][20347] Updated weights for policy 0, policy_version 467099 (0.0010) [2023-12-27 08:52:16,583][20381] Updated weights for policy 1, policy_version 466705 (0.0005) [2023-12-27 08:52:16,652][20381] Updated weights for policy 1, policy_version 466715 (0.0010) [2023-12-27 08:52:16,718][20381] Updated weights for policy 1, policy_version 466725 (0.0010) [2023-12-27 08:52:16,786][20381] Updated weights for policy 1, policy_version 466735 (0.0009) [2023-12-27 08:52:17,250][20347] Updated weights for policy 0, policy_version 467109 (0.0009) [2023-12-27 08:52:17,315][20347] Updated weights for policy 0, policy_version 467119 (0.0010) [2023-12-27 08:52:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 239099904. Throughput: 0: 10453.1, 1: 10385.6. Samples: 239115424. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:17,363][19170] Avg episode reward: [(0, '355.670'), (1, '741.700')] [2023-12-27 08:52:17,382][20347] Updated weights for policy 0, policy_version 467129 (0.0010) [2023-12-27 08:52:17,531][20381] Updated weights for policy 1, policy_version 466745 (0.0010) [2023-12-27 08:52:17,597][20381] Updated weights for policy 1, policy_version 466755 (0.0008) [2023-12-27 08:52:17,666][20381] Updated weights for policy 1, policy_version 466765 (0.0011) [2023-12-27 08:52:18,172][20347] Updated weights for policy 0, policy_version 467139 (0.0009) [2023-12-27 08:52:18,237][20347] Updated weights for policy 0, policy_version 467149 (0.0008) [2023-12-27 08:52:18,307][20347] Updated weights for policy 0, policy_version 467159 (0.0008) [2023-12-27 08:52:18,447][20381] Updated weights for policy 1, policy_version 466775 (0.0009) [2023-12-27 08:52:18,509][20381] Updated weights for policy 1, policy_version 466785 (0.0008) [2023-12-27 08:52:18,577][20381] Updated weights for policy 1, policy_version 466795 (0.0007) [2023-12-27 08:52:19,249][20347] Updated weights for policy 0, policy_version 467169 (0.0008) [2023-12-27 08:52:19,323][20347] Updated weights for policy 0, policy_version 467179 (0.0008) [2023-12-27 08:52:19,394][20347] Updated weights for policy 0, policy_version 467189 (0.0009) [2023-12-27 08:52:19,470][20347] Updated weights for policy 0, policy_version 467199 (0.0008) [2023-12-27 08:52:19,492][20381] Updated weights for policy 1, policy_version 466805 (0.0008) [2023-12-27 08:52:19,569][20381] Updated weights for policy 1, policy_version 466815 (0.0008) [2023-12-27 08:52:19,636][20381] Updated weights for policy 1, policy_version 466825 (0.0010) [2023-12-27 08:52:20,319][20347] Updated weights for policy 0, policy_version 467209 (0.0010) [2023-12-27 08:52:20,382][20347] Updated weights for policy 0, policy_version 467219 (0.0009) [2023-12-27 08:52:20,426][20381] Updated weights for policy 1, policy_version 466835 (0.0007) [2023-12-27 08:52:20,447][20347] Updated weights for policy 0, policy_version 467229 (0.0009) [2023-12-27 08:52:20,495][20381] Updated weights for policy 1, policy_version 466845 (0.0007) [2023-12-27 08:52:20,560][20381] Updated weights for policy 1, policy_version 466855 (0.0008) [2023-12-27 08:52:21,350][20381] Updated weights for policy 1, policy_version 466865 (0.0007) [2023-12-27 08:52:21,356][20347] Updated weights for policy 0, policy_version 467239 (0.0009) [2023-12-27 08:52:21,418][20347] Updated weights for policy 0, policy_version 467249 (0.0010) [2023-12-27 08:52:21,421][20381] Updated weights for policy 1, policy_version 466875 (0.0007) [2023-12-27 08:52:21,479][20347] Updated weights for policy 0, policy_version 467259 (0.0008) [2023-12-27 08:52:21,490][20381] Updated weights for policy 1, policy_version 466885 (0.0007) [2023-12-27 08:52:21,552][20381] Updated weights for policy 1, policy_version 466895 (0.0008) [2023-12-27 08:52:22,326][20347] Updated weights for policy 0, policy_version 467269 (0.0009) [2023-12-27 08:52:22,362][19170] Fps is (10 sec: 18022.3, 60 sec: 20343.5, 300 sec: 20688.3). Total num frames: 239181824. Throughput: 0: 10227.3, 1: 10293.1. Samples: 239164764. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:22,363][19170] Avg episode reward: [(0, '362.630'), (1, '769.750')] [2023-12-27 08:52:22,403][20347] Updated weights for policy 0, policy_version 467279 (0.0008) [2023-12-27 08:52:22,407][20381] Updated weights for policy 1, policy_version 466905 (0.0008) [2023-12-27 08:52:22,472][20381] Updated weights for policy 1, policy_version 466915 (0.0008) [2023-12-27 08:52:22,490][20347] Updated weights for policy 0, policy_version 467289 (0.0011) [2023-12-27 08:52:22,539][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000467296_119644160.pth... [2023-12-27 08:52:22,544][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000466112_119341056.pth [2023-12-27 08:52:22,561][20381] Updated weights for policy 1, policy_version 466926 (0.0008) [2023-12-27 08:52:22,582][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000466928_119554048.pth... [2023-12-27 08:52:22,586][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000465744_119250944.pth [2023-12-27 08:52:23,616][20347] Updated weights for policy 0, policy_version 467299 (0.0019) [2023-12-27 08:52:23,689][20347] Updated weights for policy 0, policy_version 467309 (0.0008) [2023-12-27 08:52:23,705][20381] Updated weights for policy 1, policy_version 466936 (0.0008) [2023-12-27 08:52:23,758][20347] Updated weights for policy 0, policy_version 467319 (0.0008) [2023-12-27 08:52:23,769][20381] Updated weights for policy 1, policy_version 466946 (0.0007) [2023-12-27 08:52:23,834][20381] Updated weights for policy 1, policy_version 466956 (0.0007) [2023-12-27 08:52:24,553][20347] Updated weights for policy 0, policy_version 467329 (0.0009) [2023-12-27 08:52:24,610][20381] Updated weights for policy 1, policy_version 466966 (0.0008) [2023-12-27 08:52:24,619][20347] Updated weights for policy 0, policy_version 467339 (0.0008) [2023-12-27 08:52:24,674][20381] Updated weights for policy 1, policy_version 466976 (0.0008) [2023-12-27 08:52:24,688][20347] Updated weights for policy 0, policy_version 467349 (0.0007) [2023-12-27 08:52:24,739][20381] Updated weights for policy 1, policy_version 466986 (0.0008) [2023-12-27 08:52:24,755][20347] Updated weights for policy 0, policy_version 467359 (0.0007) [2023-12-27 08:52:25,553][20381] Updated weights for policy 1, policy_version 466996 (0.0008) [2023-12-27 08:52:25,576][20347] Updated weights for policy 0, policy_version 467369 (0.0007) [2023-12-27 08:52:25,619][20381] Updated weights for policy 1, policy_version 467006 (0.0011) [2023-12-27 08:52:25,646][20347] Updated weights for policy 0, policy_version 467379 (0.0009) [2023-12-27 08:52:25,684][20381] Updated weights for policy 1, policy_version 467016 (0.0009) [2023-12-27 08:52:25,711][20347] Updated weights for policy 0, policy_version 467389 (0.0007) [2023-12-27 08:52:26,432][20381] Updated weights for policy 1, policy_version 467026 (0.0008) [2023-12-27 08:52:26,460][20347] Updated weights for policy 0, policy_version 467399 (0.0008) [2023-12-27 08:52:26,497][20381] Updated weights for policy 1, policy_version 467036 (0.0009) [2023-12-27 08:52:26,523][20347] Updated weights for policy 0, policy_version 467409 (0.0008) [2023-12-27 08:52:26,558][20381] Updated weights for policy 1, policy_version 467046 (0.0010) [2023-12-27 08:52:26,586][20347] Updated weights for policy 0, policy_version 467419 (0.0007) [2023-12-27 08:52:26,621][20381] Updated weights for policy 1, policy_version 467056 (0.0008) [2023-12-27 08:52:27,354][20347] Updated weights for policy 0, policy_version 467429 (0.0007) [2023-12-27 08:52:27,362][19170] Fps is (10 sec: 16383.6, 60 sec: 19933.8, 300 sec: 20605.0). Total num frames: 239263744. Throughput: 0: 9922.1, 1: 9990.0. Samples: 239261512. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:27,363][19170] Avg episode reward: [(0, '402.710'), (1, '777.890')] [2023-12-27 08:52:27,388][20381] Updated weights for policy 1, policy_version 467066 (0.0009) [2023-12-27 08:52:27,423][20347] Updated weights for policy 0, policy_version 467439 (0.0007) [2023-12-27 08:52:27,453][20381] Updated weights for policy 1, policy_version 467076 (0.0010) [2023-12-27 08:52:27,491][20347] Updated weights for policy 0, policy_version 467449 (0.0008) [2023-12-27 08:52:27,515][20381] Updated weights for policy 1, policy_version 467086 (0.0011) [2023-12-27 08:52:28,267][20347] Updated weights for policy 0, policy_version 467459 (0.0008) [2023-12-27 08:52:28,299][20381] Updated weights for policy 1, policy_version 467096 (0.0011) [2023-12-27 08:52:28,336][20347] Updated weights for policy 0, policy_version 467469 (0.0009) [2023-12-27 08:52:28,371][20381] Updated weights for policy 1, policy_version 467106 (0.0009) [2023-12-27 08:52:28,409][20347] Updated weights for policy 0, policy_version 467479 (0.0008) [2023-12-27 08:52:28,442][20381] Updated weights for policy 1, policy_version 467116 (0.0008) [2023-12-27 08:52:29,310][20347] Updated weights for policy 0, policy_version 467489 (0.0008) [2023-12-27 08:52:29,382][20381] Updated weights for policy 1, policy_version 467126 (0.0009) [2023-12-27 08:52:29,382][20347] Updated weights for policy 0, policy_version 467499 (0.0009) [2023-12-27 08:52:29,453][20381] Updated weights for policy 1, policy_version 467136 (0.0009) [2023-12-27 08:52:29,458][20347] Updated weights for policy 0, policy_version 467509 (0.0008) [2023-12-27 08:52:29,524][20381] Updated weights for policy 1, policy_version 467146 (0.0009) [2023-12-27 08:52:29,531][20347] Updated weights for policy 0, policy_version 467519 (0.0008) [2023-12-27 08:52:30,353][20347] Updated weights for policy 0, policy_version 467529 (0.0009) [2023-12-27 08:52:30,410][20381] Updated weights for policy 1, policy_version 467156 (0.0009) [2023-12-27 08:52:30,425][20347] Updated weights for policy 0, policy_version 467539 (0.0008) [2023-12-27 08:52:30,478][20381] Updated weights for policy 1, policy_version 467166 (0.0009) [2023-12-27 08:52:30,504][20347] Updated weights for policy 0, policy_version 467549 (0.0008) [2023-12-27 08:52:30,551][20381] Updated weights for policy 1, policy_version 467176 (0.0008) [2023-12-27 08:52:31,378][20381] Updated weights for policy 1, policy_version 467186 (0.0009) [2023-12-27 08:52:31,393][20347] Updated weights for policy 0, policy_version 467559 (0.0009) [2023-12-27 08:52:31,460][20381] Updated weights for policy 1, policy_version 467196 (0.0009) [2023-12-27 08:52:31,468][20347] Updated weights for policy 0, policy_version 467569 (0.0009) [2023-12-27 08:52:31,529][20381] Updated weights for policy 1, policy_version 467206 (0.0009) [2023-12-27 08:52:31,534][20347] Updated weights for policy 0, policy_version 467579 (0.0009) [2023-12-27 08:52:31,596][20381] Updated weights for policy 1, policy_version 467216 (0.0009) [2023-12-27 08:52:32,362][19170] Fps is (10 sec: 16384.0, 60 sec: 19660.8, 300 sec: 20521.6). Total num frames: 239345664. Throughput: 0: 9594.1, 1: 9741.0. Samples: 239361732. Policy #0 lag: (min: 3.0, avg: 16.8, max: 35.0) [2023-12-27 08:52:32,363][19170] Avg episode reward: [(0, '533.920'), (1, '787.450')] [2023-12-27 08:52:32,385][20347] Updated weights for policy 0, policy_version 467589 (0.0008) [2023-12-27 08:52:32,452][20381] Updated weights for policy 1, policy_version 467226 (0.0009) [2023-12-27 08:52:32,459][20347] Updated weights for policy 0, policy_version 467599 (0.0010) [2023-12-27 08:52:32,523][20381] Updated weights for policy 1, policy_version 467236 (0.0009) [2023-12-27 08:52:32,529][20347] Updated weights for policy 0, policy_version 467609 (0.0009) [2023-12-27 08:52:32,593][20381] Updated weights for policy 1, policy_version 467246 (0.0009) [2023-12-27 08:52:33,371][20347] Updated weights for policy 0, policy_version 467619 (0.0008) [2023-12-27 08:52:33,386][20381] Updated weights for policy 1, policy_version 467256 (0.0009) [2023-12-27 08:52:33,440][20347] Updated weights for policy 0, policy_version 467629 (0.0008) [2023-12-27 08:52:33,455][20381] Updated weights for policy 1, policy_version 467266 (0.0008) [2023-12-27 08:52:33,511][20347] Updated weights for policy 0, policy_version 467639 (0.0009) [2023-12-27 08:52:33,524][20381] Updated weights for policy 1, policy_version 467276 (0.0009) [2023-12-27 08:52:34,320][20347] Updated weights for policy 0, policy_version 467649 (0.0008) [2023-12-27 08:52:34,333][20381] Updated weights for policy 1, policy_version 467286 (0.0008) [2023-12-27 08:52:34,391][20347] Updated weights for policy 0, policy_version 467659 (0.0008) [2023-12-27 08:52:34,404][20381] Updated weights for policy 1, policy_version 467296 (0.0008) [2023-12-27 08:52:34,461][20347] Updated weights for policy 0, policy_version 467669 (0.0008) [2023-12-27 08:52:34,476][20381] Updated weights for policy 1, policy_version 467306 (0.0008) [2023-12-27 08:52:34,541][20347] Updated weights for policy 0, policy_version 467679 (0.0008) [2023-12-27 08:52:35,268][20381] Updated weights for policy 1, policy_version 467316 (0.0008) [2023-12-27 08:52:35,340][20381] Updated weights for policy 1, policy_version 467326 (0.0008) [2023-12-27 08:52:35,348][20347] Updated weights for policy 0, policy_version 467689 (0.0008) [2023-12-27 08:52:35,418][20381] Updated weights for policy 1, policy_version 467336 (0.0008) [2023-12-27 08:52:35,421][20347] Updated weights for policy 0, policy_version 467699 (0.0009) [2023-12-27 08:52:35,493][20347] Updated weights for policy 0, policy_version 467709 (0.0008) [2023-12-27 08:52:36,199][20381] Updated weights for policy 1, policy_version 467346 (0.0008) [2023-12-27 08:52:36,272][20381] Updated weights for policy 1, policy_version 467356 (0.0008) [2023-12-27 08:52:36,295][20347] Updated weights for policy 0, policy_version 467719 (0.0008) [2023-12-27 08:52:36,344][20381] Updated weights for policy 1, policy_version 467366 (0.0008) [2023-12-27 08:52:36,368][20347] Updated weights for policy 0, policy_version 467729 (0.0008) [2023-12-27 08:52:36,419][20381] Updated weights for policy 1, policy_version 467376 (0.0008) [2023-12-27 08:52:36,439][20347] Updated weights for policy 0, policy_version 467739 (0.0009) [2023-12-27 08:52:37,220][20381] Updated weights for policy 1, policy_version 467386 (0.0008) [2023-12-27 08:52:37,249][20347] Updated weights for policy 0, policy_version 467749 (0.0008) [2023-12-27 08:52:37,290][20381] Updated weights for policy 1, policy_version 467396 (0.0008) [2023-12-27 08:52:37,320][20347] Updated weights for policy 0, policy_version 467759 (0.0009) [2023-12-27 08:52:37,356][20381] Updated weights for policy 1, policy_version 467406 (0.0008) [2023-12-27 08:52:37,362][19170] Fps is (10 sec: 16383.8, 60 sec: 19251.2, 300 sec: 20466.1). Total num frames: 239427584. Throughput: 0: 9425.3, 1: 9600.3. Samples: 239412016. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:52:37,363][19170] Avg episode reward: [(0, '495.190'), (1, '845.160')] [2023-12-27 08:52:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000467408_119676928.pth... [2023-12-27 08:52:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000466352_119406592.pth [2023-12-27 08:52:37,389][20347] Updated weights for policy 0, policy_version 467769 (0.0009) [2023-12-27 08:52:37,439][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000467776_119767040.pth... [2023-12-27 08:52:37,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000466720_119496704.pth [2023-12-27 08:52:38,201][20381] Updated weights for policy 1, policy_version 467416 (0.0009) [2023-12-27 08:52:38,219][20347] Updated weights for policy 0, policy_version 467779 (0.0009) [2023-12-27 08:52:38,273][20381] Updated weights for policy 1, policy_version 467426 (0.0008) [2023-12-27 08:52:38,287][20347] Updated weights for policy 0, policy_version 467789 (0.0009) [2023-12-27 08:52:38,347][20381] Updated weights for policy 1, policy_version 467436 (0.0009) [2023-12-27 08:52:38,374][20347] Updated weights for policy 0, policy_version 467799 (0.0009) [2023-12-27 08:52:39,190][20347] Updated weights for policy 0, policy_version 467809 (0.0008) [2023-12-27 08:52:39,193][20381] Updated weights for policy 1, policy_version 467446 (0.0009) [2023-12-27 08:52:39,259][20347] Updated weights for policy 0, policy_version 467819 (0.0007) [2023-12-27 08:52:39,263][20381] Updated weights for policy 1, policy_version 467456 (0.0009) [2023-12-27 08:52:39,330][20381] Updated weights for policy 1, policy_version 467466 (0.0008) [2023-12-27 08:52:39,331][20347] Updated weights for policy 0, policy_version 467829 (0.0008) [2023-12-27 08:52:39,407][20347] Updated weights for policy 0, policy_version 467839 (0.0008) [2023-12-27 08:52:40,242][20381] Updated weights for policy 1, policy_version 467476 (0.0009) [2023-12-27 08:52:40,262][20347] Updated weights for policy 0, policy_version 467849 (0.0008) [2023-12-27 08:52:40,312][20381] Updated weights for policy 1, policy_version 467486 (0.0009) [2023-12-27 08:52:40,334][20347] Updated weights for policy 0, policy_version 467859 (0.0009) [2023-12-27 08:52:40,383][20381] Updated weights for policy 1, policy_version 467496 (0.0008) [2023-12-27 08:52:40,409][20347] Updated weights for policy 0, policy_version 467869 (0.0008) [2023-12-27 08:52:41,252][20347] Updated weights for policy 0, policy_version 467879 (0.0008) [2023-12-27 08:52:41,301][20381] Updated weights for policy 1, policy_version 467506 (0.0009) [2023-12-27 08:52:41,322][20347] Updated weights for policy 0, policy_version 467889 (0.0009) [2023-12-27 08:52:41,367][20381] Updated weights for policy 1, policy_version 467516 (0.0008) [2023-12-27 08:52:41,395][20347] Updated weights for policy 0, policy_version 467899 (0.0007) [2023-12-27 08:52:41,449][20381] Updated weights for policy 1, policy_version 467526 (0.0009) [2023-12-27 08:52:41,527][20381] Updated weights for policy 1, policy_version 467536 (0.0009) [2023-12-27 08:52:42,234][20347] Updated weights for policy 0, policy_version 467909 (0.0009) [2023-12-27 08:52:42,308][20347] Updated weights for policy 0, policy_version 467919 (0.0008) [2023-12-27 08:52:42,362][19170] Fps is (10 sec: 16384.0, 60 sec: 18978.5, 300 sec: 20382.8). Total num frames: 239509504. Throughput: 0: 9137.6, 1: 9253.9. Samples: 239510448. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:52:42,363][19170] Avg episode reward: [(0, '404.040'), (1, '893.380')] [2023-12-27 08:52:42,381][20347] Updated weights for policy 0, policy_version 467929 (0.0008) [2023-12-27 08:52:42,392][20381] Updated weights for policy 1, policy_version 467546 (0.0008) [2023-12-27 08:52:42,461][20381] Updated weights for policy 1, policy_version 467556 (0.0008) [2023-12-27 08:52:42,534][20381] Updated weights for policy 1, policy_version 467566 (0.0009) [2023-12-27 08:52:43,187][20347] Updated weights for policy 0, policy_version 467939 (0.0008) [2023-12-27 08:52:43,252][20347] Updated weights for policy 0, policy_version 467949 (0.0008) [2023-12-27 08:52:43,323][20347] Updated weights for policy 0, policy_version 467959 (0.0008) [2023-12-27 08:52:43,348][20381] Updated weights for policy 1, policy_version 467576 (0.0008) [2023-12-27 08:52:43,417][20381] Updated weights for policy 1, policy_version 467586 (0.0008) [2023-12-27 08:52:43,488][20381] Updated weights for policy 1, policy_version 467596 (0.0007) [2023-12-27 08:52:44,108][20347] Updated weights for policy 0, policy_version 467969 (0.0008) [2023-12-27 08:52:44,177][20347] Updated weights for policy 0, policy_version 467979 (0.0008) [2023-12-27 08:52:44,249][20347] Updated weights for policy 0, policy_version 467989 (0.0008) [2023-12-27 08:52:44,267][20381] Updated weights for policy 1, policy_version 467606 (0.0009) [2023-12-27 08:52:44,320][20347] Updated weights for policy 0, policy_version 467999 (0.0007) [2023-12-27 08:52:44,336][20381] Updated weights for policy 1, policy_version 467616 (0.0008) [2023-12-27 08:52:44,406][20381] Updated weights for policy 1, policy_version 467626 (0.0008) [2023-12-27 08:52:45,122][20347] Updated weights for policy 0, policy_version 468009 (0.0009) [2023-12-27 08:52:45,181][20381] Updated weights for policy 1, policy_version 467636 (0.0008) [2023-12-27 08:52:45,187][20347] Updated weights for policy 0, policy_version 468019 (0.0007) [2023-12-27 08:52:45,249][20381] Updated weights for policy 1, policy_version 467646 (0.0008) [2023-12-27 08:52:45,256][20347] Updated weights for policy 0, policy_version 468029 (0.0007) [2023-12-27 08:52:45,318][20381] Updated weights for policy 1, policy_version 467656 (0.0008) [2023-12-27 08:52:46,025][20347] Updated weights for policy 0, policy_version 468039 (0.0008) [2023-12-27 08:52:46,094][20347] Updated weights for policy 0, policy_version 468049 (0.0008) [2023-12-27 08:52:46,105][20381] Updated weights for policy 1, policy_version 467666 (0.0008) [2023-12-27 08:52:46,167][20347] Updated weights for policy 0, policy_version 468059 (0.0007) [2023-12-27 08:52:46,171][20381] Updated weights for policy 1, policy_version 467676 (0.0008) [2023-12-27 08:52:46,242][20381] Updated weights for policy 1, policy_version 467686 (0.0009) [2023-12-27 08:52:46,308][20381] Updated weights for policy 1, policy_version 467696 (0.0008) [2023-12-27 08:52:46,928][20347] Updated weights for policy 0, policy_version 468069 (0.0009) [2023-12-27 08:52:46,999][20347] Updated weights for policy 0, policy_version 468079 (0.0008) [2023-12-27 08:52:47,061][20347] Updated weights for policy 0, policy_version 468089 (0.0007) [2023-12-27 08:52:47,062][20381] Updated weights for policy 1, policy_version 467706 (0.0007) [2023-12-27 08:52:47,132][20381] Updated weights for policy 1, policy_version 467716 (0.0007) [2023-12-27 08:52:47,187][20381] Updated weights for policy 1, policy_version 467726 (0.0007) [2023-12-27 08:52:47,362][19170] Fps is (10 sec: 18022.6, 60 sec: 18705.0, 300 sec: 20355.0). Total num frames: 239607808. Throughput: 0: 8936.0, 1: 9011.6. Samples: 239613344. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:52:47,363][19170] Avg episode reward: [(0, '361.340'), (1, '903.020')] [2023-12-27 08:52:47,717][20347] Updated weights for policy 0, policy_version 468099 (0.0008) [2023-12-27 08:52:47,765][20347] Updated weights for policy 0, policy_version 468109 (0.0007) [2023-12-27 08:52:47,827][20347] Updated weights for policy 0, policy_version 468119 (0.0007) [2023-12-27 08:52:47,897][20381] Updated weights for policy 1, policy_version 467736 (0.0010) [2023-12-27 08:52:47,947][20381] Updated weights for policy 1, policy_version 467746 (0.0010) [2023-12-27 08:52:48,020][20381] Updated weights for policy 1, policy_version 467756 (0.0008) [2023-12-27 08:52:48,689][20347] Updated weights for policy 0, policy_version 468129 (0.0008) [2023-12-27 08:52:48,757][20347] Updated weights for policy 0, policy_version 468139 (0.0009) [2023-12-27 08:52:48,827][20347] Updated weights for policy 0, policy_version 468149 (0.0009) [2023-12-27 08:52:48,902][20381] Updated weights for policy 1, policy_version 467766 (0.0009) [2023-12-27 08:52:48,902][20347] Updated weights for policy 0, policy_version 468159 (0.0009) [2023-12-27 08:52:48,976][20381] Updated weights for policy 1, policy_version 467776 (0.0009) [2023-12-27 08:52:49,054][20381] Updated weights for policy 1, policy_version 467786 (0.0009) [2023-12-27 08:52:49,781][20347] Updated weights for policy 0, policy_version 468169 (0.0009) [2023-12-27 08:52:49,857][20347] Updated weights for policy 0, policy_version 468179 (0.0009) [2023-12-27 08:52:49,898][20381] Updated weights for policy 1, policy_version 467796 (0.0009) [2023-12-27 08:52:49,928][20347] Updated weights for policy 0, policy_version 468189 (0.0008) [2023-12-27 08:52:49,967][20381] Updated weights for policy 1, policy_version 467806 (0.0010) [2023-12-27 08:52:50,043][20381] Updated weights for policy 1, policy_version 467816 (0.0008) [2023-12-27 08:52:50,835][20347] Updated weights for policy 0, policy_version 468199 (0.0009) [2023-12-27 08:52:50,911][20347] Updated weights for policy 0, policy_version 468209 (0.0009) [2023-12-27 08:52:50,926][20381] Updated weights for policy 1, policy_version 467826 (0.0008) [2023-12-27 08:52:51,003][20347] Updated weights for policy 0, policy_version 468219 (0.0018) [2023-12-27 08:52:51,003][20381] Updated weights for policy 1, policy_version 467836 (0.0009) [2023-12-27 08:52:51,074][20381] Updated weights for policy 1, policy_version 467846 (0.0009) [2023-12-27 08:52:51,155][20381] Updated weights for policy 1, policy_version 467856 (0.0009) [2023-12-27 08:52:51,931][20347] Updated weights for policy 0, policy_version 468229 (0.0009) [2023-12-27 08:52:52,004][20347] Updated weights for policy 0, policy_version 468239 (0.0009) [2023-12-27 08:52:52,070][20381] Updated weights for policy 1, policy_version 467866 (0.0009) [2023-12-27 08:52:52,077][20347] Updated weights for policy 0, policy_version 468249 (0.0008) [2023-12-27 08:52:52,144][20381] Updated weights for policy 1, policy_version 467876 (0.0008) [2023-12-27 08:52:52,212][20381] Updated weights for policy 1, policy_version 467886 (0.0008) [2023-12-27 08:52:52,362][19170] Fps is (10 sec: 18022.4, 60 sec: 18432.0, 300 sec: 20299.5). Total num frames: 239689728. Throughput: 0: 8825.1, 1: 8840.3. Samples: 239664324. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:52:52,363][19170] Avg episode reward: [(0, '345.630'), (1, '941.870')] [2023-12-27 08:52:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000467888_119799808.pth... [2023-12-27 08:52:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000468256_119889920.pth... [2023-12-27 08:52:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000466928_119554048.pth [2023-12-27 08:52:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000467296_119644160.pth [2023-12-27 08:52:52,966][20347] Updated weights for policy 0, policy_version 468259 (0.0008) [2023-12-27 08:52:53,038][20347] Updated weights for policy 0, policy_version 468269 (0.0008) [2023-12-27 08:52:53,066][20381] Updated weights for policy 1, policy_version 467896 (0.0009) [2023-12-27 08:52:53,110][20347] Updated weights for policy 0, policy_version 468279 (0.0008) [2023-12-27 08:52:53,137][20381] Updated weights for policy 1, policy_version 467906 (0.0009) [2023-12-27 08:52:53,210][20381] Updated weights for policy 1, policy_version 467916 (0.0008) [2023-12-27 08:52:53,941][20347] Updated weights for policy 0, policy_version 468289 (0.0009) [2023-12-27 08:52:54,013][20347] Updated weights for policy 0, policy_version 468299 (0.0009) [2023-12-27 08:52:54,014][20381] Updated weights for policy 1, policy_version 467926 (0.0009) [2023-12-27 08:52:54,086][20347] Updated weights for policy 0, policy_version 468309 (0.0008) [2023-12-27 08:52:54,086][20381] Updated weights for policy 1, policy_version 467936 (0.0009) [2023-12-27 08:52:54,153][20381] Updated weights for policy 1, policy_version 467946 (0.0010) [2023-12-27 08:52:54,155][20347] Updated weights for policy 0, policy_version 468319 (0.0008) [2023-12-27 08:52:54,961][20381] Updated weights for policy 1, policy_version 467956 (0.0008) [2023-12-27 08:52:55,012][20347] Updated weights for policy 0, policy_version 468329 (0.0010) [2023-12-27 08:52:55,036][20381] Updated weights for policy 1, policy_version 467966 (0.0008) [2023-12-27 08:52:55,080][20347] Updated weights for policy 0, policy_version 468339 (0.0010) [2023-12-27 08:52:55,104][20381] Updated weights for policy 1, policy_version 467976 (0.0008) [2023-12-27 08:52:55,153][20347] Updated weights for policy 0, policy_version 468349 (0.0010) [2023-12-27 08:52:55,964][20381] Updated weights for policy 1, policy_version 467986 (0.0008) [2023-12-27 08:52:56,005][20347] Updated weights for policy 0, policy_version 468359 (0.0009) [2023-12-27 08:52:56,045][20381] Updated weights for policy 1, policy_version 467997 (0.0009) [2023-12-27 08:52:56,077][20347] Updated weights for policy 0, policy_version 468369 (0.0009) [2023-12-27 08:52:56,112][20381] Updated weights for policy 1, policy_version 468007 (0.0008) [2023-12-27 08:52:56,148][20347] Updated weights for policy 0, policy_version 468379 (0.0009) [2023-12-27 08:52:56,923][20381] Updated weights for policy 1, policy_version 468017 (0.0009) [2023-12-27 08:52:56,935][20347] Updated weights for policy 0, policy_version 468389 (0.0009) [2023-12-27 08:52:56,994][20381] Updated weights for policy 1, policy_version 468027 (0.0009) [2023-12-27 08:52:57,009][20347] Updated weights for policy 0, policy_version 468399 (0.0008) [2023-12-27 08:52:57,063][20381] Updated weights for policy 1, policy_version 468037 (0.0008) [2023-12-27 08:52:57,080][20347] Updated weights for policy 0, policy_version 468409 (0.0008) [2023-12-27 08:52:57,133][20381] Updated weights for policy 1, policy_version 468047 (0.0008) [2023-12-27 08:52:57,362][19170] Fps is (10 sec: 16384.0, 60 sec: 17885.9, 300 sec: 20216.2). Total num frames: 239771648. Throughput: 0: 8542.8, 1: 8579.2. Samples: 239761216. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:52:57,363][19170] Avg episode reward: [(0, '260.850'), (1, '942.660')] [2023-12-27 08:52:57,889][20347] Updated weights for policy 0, policy_version 468419 (0.0009) [2023-12-27 08:52:57,945][20381] Updated weights for policy 1, policy_version 468057 (0.0008) [2023-12-27 08:52:57,960][20347] Updated weights for policy 0, policy_version 468429 (0.0008) [2023-12-27 08:52:58,012][20381] Updated weights for policy 1, policy_version 468067 (0.0008) [2023-12-27 08:52:58,027][20347] Updated weights for policy 0, policy_version 468439 (0.0008) [2023-12-27 08:52:58,080][20381] Updated weights for policy 1, policy_version 468077 (0.0008) [2023-12-27 08:52:58,859][20347] Updated weights for policy 0, policy_version 468449 (0.0008) [2023-12-27 08:52:58,924][20381] Updated weights for policy 1, policy_version 468087 (0.0008) [2023-12-27 08:52:58,927][20347] Updated weights for policy 0, policy_version 468459 (0.0008) [2023-12-27 08:52:58,998][20381] Updated weights for policy 1, policy_version 468097 (0.0008) [2023-12-27 08:52:58,999][20347] Updated weights for policy 0, policy_version 468469 (0.0008) [2023-12-27 08:52:59,070][20347] Updated weights for policy 0, policy_version 468479 (0.0008) [2023-12-27 08:52:59,070][20381] Updated weights for policy 1, policy_version 468107 (0.0008) [2023-12-27 08:52:59,897][20381] Updated weights for policy 1, policy_version 468117 (0.0008) [2023-12-27 08:52:59,930][20347] Updated weights for policy 0, policy_version 468489 (0.0009) [2023-12-27 08:52:59,971][20381] Updated weights for policy 1, policy_version 468127 (0.0008) [2023-12-27 08:53:00,002][20347] Updated weights for policy 0, policy_version 468499 (0.0008) [2023-12-27 08:53:00,045][20381] Updated weights for policy 1, policy_version 468137 (0.0009) [2023-12-27 08:53:00,070][20347] Updated weights for policy 0, policy_version 468509 (0.0008) [2023-12-27 08:53:00,828][20381] Updated weights for policy 1, policy_version 468147 (0.0008) [2023-12-27 08:53:00,867][20347] Updated weights for policy 0, policy_version 468519 (0.0009) [2023-12-27 08:53:00,894][20381] Updated weights for policy 1, policy_version 468157 (0.0009) [2023-12-27 08:53:00,933][20347] Updated weights for policy 0, policy_version 468529 (0.0008) [2023-12-27 08:53:00,960][20381] Updated weights for policy 1, policy_version 468167 (0.0008) [2023-12-27 08:53:00,999][20347] Updated weights for policy 0, policy_version 468539 (0.0009) [2023-12-27 08:53:01,768][20381] Updated weights for policy 1, policy_version 468177 (0.0008) [2023-12-27 08:53:01,833][20347] Updated weights for policy 0, policy_version 468549 (0.0008) [2023-12-27 08:53:01,840][20381] Updated weights for policy 1, policy_version 468187 (0.0008) [2023-12-27 08:53:01,900][20347] Updated weights for policy 0, policy_version 468559 (0.0007) [2023-12-27 08:53:01,936][20381] Updated weights for policy 1, policy_version 468197 (0.0008) [2023-12-27 08:53:01,969][20347] Updated weights for policy 0, policy_version 468569 (0.0008) [2023-12-27 08:53:02,004][20381] Updated weights for policy 1, policy_version 468207 (0.0007) [2023-12-27 08:53:02,362][19170] Fps is (10 sec: 16384.1, 60 sec: 17612.8, 300 sec: 20105.1). Total num frames: 239853568. Throughput: 0: 8270.8, 1: 8332.0. Samples: 239862552. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:02,363][19170] Avg episode reward: [(0, '291.820'), (1, '942.650')] [2023-12-27 08:53:02,742][20381] Updated weights for policy 1, policy_version 468217 (0.0008) [2023-12-27 08:53:02,759][20347] Updated weights for policy 0, policy_version 468579 (0.0009) [2023-12-27 08:53:02,804][20381] Updated weights for policy 1, policy_version 468227 (0.0008) [2023-12-27 08:53:02,821][20347] Updated weights for policy 0, policy_version 468589 (0.0008) [2023-12-27 08:53:02,866][20381] Updated weights for policy 1, policy_version 468237 (0.0009) [2023-12-27 08:53:02,885][20347] Updated weights for policy 0, policy_version 468599 (0.0007) [2023-12-27 08:53:03,537][20381] Updated weights for policy 1, policy_version 468247 (0.0007) [2023-12-27 08:53:03,587][20381] Updated weights for policy 1, policy_version 468257 (0.0008) [2023-12-27 08:53:03,618][20347] Updated weights for policy 0, policy_version 468609 (0.0007) [2023-12-27 08:53:03,639][20381] Updated weights for policy 1, policy_version 468267 (0.0007) [2023-12-27 08:53:03,670][20347] Updated weights for policy 0, policy_version 468619 (0.0007) [2023-12-27 08:53:03,713][20347] Updated weights for policy 0, policy_version 468629 (0.0007) [2023-12-27 08:53:03,759][20347] Updated weights for policy 0, policy_version 468639 (0.0007) [2023-12-27 08:53:04,280][20381] Updated weights for policy 1, policy_version 468277 (0.0006) [2023-12-27 08:53:04,339][20381] Updated weights for policy 1, policy_version 468287 (0.0007) [2023-12-27 08:53:04,399][20381] Updated weights for policy 1, policy_version 468297 (0.0008) [2023-12-27 08:53:04,429][20347] Updated weights for policy 0, policy_version 468649 (0.0007) [2023-12-27 08:53:04,482][20347] Updated weights for policy 0, policy_version 468659 (0.0009) [2023-12-27 08:53:04,540][20347] Updated weights for policy 0, policy_version 468669 (0.0008) [2023-12-27 08:53:05,031][20381] Updated weights for policy 1, policy_version 468307 (0.0007) [2023-12-27 08:53:05,084][20381] Updated weights for policy 1, policy_version 468317 (0.0008) [2023-12-27 08:53:05,137][20381] Updated weights for policy 1, policy_version 468327 (0.0008) [2023-12-27 08:53:05,163][20347] Updated weights for policy 0, policy_version 468679 (0.0006) [2023-12-27 08:53:05,215][20347] Updated weights for policy 0, policy_version 468689 (0.0005) [2023-12-27 08:53:05,264][20347] Updated weights for policy 0, policy_version 468699 (0.0005) [2023-12-27 08:53:05,839][20347] Updated weights for policy 0, policy_version 468709 (0.0008) [2023-12-27 08:53:05,867][20381] Updated weights for policy 1, policy_version 468337 (0.0008) [2023-12-27 08:53:05,891][20347] Updated weights for policy 0, policy_version 468719 (0.0010) [2023-12-27 08:53:05,918][20381] Updated weights for policy 1, policy_version 468347 (0.0008) [2023-12-27 08:53:05,956][20347] Updated weights for policy 0, policy_version 468729 (0.0010) [2023-12-27 08:53:05,966][20381] Updated weights for policy 1, policy_version 468357 (0.0008) [2023-12-27 08:53:06,017][20381] Updated weights for policy 1, policy_version 468367 (0.0006) [2023-12-27 08:53:06,564][20347] Updated weights for policy 0, policy_version 468739 (0.0010) [2023-12-27 08:53:06,625][20347] Updated weights for policy 0, policy_version 468749 (0.0011) [2023-12-27 08:53:06,672][20381] Updated weights for policy 1, policy_version 468377 (0.0008) [2023-12-27 08:53:06,685][20347] Updated weights for policy 0, policy_version 468759 (0.0011) [2023-12-27 08:53:06,723][20381] Updated weights for policy 1, policy_version 468387 (0.0011) [2023-12-27 08:53:06,769][20381] Updated weights for policy 1, policy_version 468397 (0.0010) [2023-12-27 08:53:07,348][20347] Updated weights for policy 0, policy_version 468769 (0.0010) [2023-12-27 08:53:07,362][19170] Fps is (10 sec: 18022.5, 60 sec: 17476.3, 300 sec: 20105.2). Total num frames: 239951872. Throughput: 0: 8391.1, 1: 8456.5. Samples: 239922904. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:07,363][19170] Avg episode reward: [(0, '331.100'), (1, '942.690')] [2023-12-27 08:53:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000468400_119930880.pth... [2023-12-27 08:53:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000467408_119676928.pth [2023-12-27 08:53:07,408][20347] Updated weights for policy 0, policy_version 468779 (0.0007) [2023-12-27 08:53:07,458][20347] Updated weights for policy 0, policy_version 468789 (0.0008) [2023-12-27 08:53:07,472][20381] Updated weights for policy 1, policy_version 468407 (0.0008) [2023-12-27 08:53:07,515][20347] Updated weights for policy 0, policy_version 468799 (0.0007) [2023-12-27 08:53:07,516][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000468800_120029184.pth... [2023-12-27 08:53:07,519][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000467776_119767040.pth [2023-12-27 08:53:07,528][20381] Updated weights for policy 1, policy_version 468417 (0.0005) [2023-12-27 08:53:07,582][20381] Updated weights for policy 1, policy_version 468427 (0.0007) [2023-12-27 08:53:08,179][20347] Updated weights for policy 0, policy_version 468809 (0.0007) [2023-12-27 08:53:08,181][20381] Updated weights for policy 1, policy_version 468437 (0.0007) [2023-12-27 08:53:08,234][20381] Updated weights for policy 1, policy_version 468447 (0.0005) [2023-12-27 08:53:08,235][20347] Updated weights for policy 0, policy_version 468819 (0.0006) [2023-12-27 08:53:08,288][20381] Updated weights for policy 1, policy_version 468457 (0.0005) [2023-12-27 08:53:08,293][20347] Updated weights for policy 0, policy_version 468829 (0.0006) [2023-12-27 08:53:08,885][20347] Updated weights for policy 0, policy_version 468839 (0.0007) [2023-12-27 08:53:08,931][20347] Updated weights for policy 0, policy_version 468849 (0.0005) [2023-12-27 08:53:08,968][20381] Updated weights for policy 1, policy_version 468467 (0.0008) [2023-12-27 08:53:08,981][20347] Updated weights for policy 0, policy_version 468859 (0.0006) [2023-12-27 08:53:09,017][20381] Updated weights for policy 1, policy_version 468477 (0.0010) [2023-12-27 08:53:09,072][20381] Updated weights for policy 1, policy_version 468487 (0.0010) [2023-12-27 08:53:09,656][20347] Updated weights for policy 0, policy_version 468869 (0.0007) [2023-12-27 08:53:09,713][20347] Updated weights for policy 0, policy_version 468879 (0.0007) [2023-12-27 08:53:09,778][20381] Updated weights for policy 1, policy_version 468497 (0.0008) [2023-12-27 08:53:09,780][20347] Updated weights for policy 0, policy_version 468889 (0.0007) [2023-12-27 08:53:09,845][20381] Updated weights for policy 1, policy_version 468507 (0.0007) [2023-12-27 08:53:09,910][20381] Updated weights for policy 1, policy_version 468517 (0.0009) [2023-12-27 08:53:09,973][20381] Updated weights for policy 1, policy_version 468527 (0.0008) [2023-12-27 08:53:10,526][20347] Updated weights for policy 0, policy_version 468899 (0.0007) [2023-12-27 08:53:10,587][20347] Updated weights for policy 0, policy_version 468909 (0.0008) [2023-12-27 08:53:10,647][20347] Updated weights for policy 0, policy_version 468919 (0.0009) [2023-12-27 08:53:10,699][20381] Updated weights for policy 1, policy_version 468537 (0.0006) [2023-12-27 08:53:10,756][20381] Updated weights for policy 1, policy_version 468547 (0.0005) [2023-12-27 08:53:10,818][20381] Updated weights for policy 1, policy_version 468557 (0.0005) [2023-12-27 08:53:11,338][20347] Updated weights for policy 0, policy_version 468929 (0.0007) [2023-12-27 08:53:11,418][20347] Updated weights for policy 0, policy_version 468939 (0.0009) [2023-12-27 08:53:11,494][20347] Updated weights for policy 0, policy_version 468950 (0.0007) [2023-12-27 08:53:11,509][20381] Updated weights for policy 1, policy_version 468567 (0.0006) [2023-12-27 08:53:11,563][20347] Updated weights for policy 0, policy_version 468960 (0.0008) [2023-12-27 08:53:11,572][20381] Updated weights for policy 1, policy_version 468577 (0.0008) [2023-12-27 08:53:11,644][20381] Updated weights for policy 1, policy_version 468587 (0.0008) [2023-12-27 08:53:12,302][20291] KL-divergence is very high: 101.3066 [2023-12-27 08:53:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 17476.3, 300 sec: 20077.3). Total num frames: 240050176. Throughput: 0: 8742.0, 1: 8740.0. Samples: 240048200. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:12,363][19170] Avg episode reward: [(0, '335.390'), (1, '951.760')] [2023-12-27 08:53:12,375][20347] Updated weights for policy 0, policy_version 468970 (0.0009) [2023-12-27 08:53:12,446][20347] Updated weights for policy 0, policy_version 468980 (0.0010) [2023-12-27 08:53:12,483][20381] Updated weights for policy 1, policy_version 468597 (0.0008) [2023-12-27 08:53:12,518][20347] Updated weights for policy 0, policy_version 468990 (0.0009) [2023-12-27 08:53:12,548][20381] Updated weights for policy 1, policy_version 468607 (0.0008) [2023-12-27 08:53:12,617][20381] Updated weights for policy 1, policy_version 468617 (0.0009) [2023-12-27 08:53:13,465][20347] Updated weights for policy 0, policy_version 469000 (0.0008) [2023-12-27 08:53:13,531][20347] Updated weights for policy 0, policy_version 469010 (0.0007) [2023-12-27 08:53:13,542][20381] Updated weights for policy 1, policy_version 468627 (0.0008) [2023-12-27 08:53:13,598][20347] Updated weights for policy 0, policy_version 469020 (0.0008) [2023-12-27 08:53:13,610][20381] Updated weights for policy 1, policy_version 468637 (0.0007) [2023-12-27 08:53:13,679][20381] Updated weights for policy 1, policy_version 468647 (0.0008) [2023-12-27 08:53:14,425][20347] Updated weights for policy 0, policy_version 469030 (0.0009) [2023-12-27 08:53:14,458][20381] Updated weights for policy 1, policy_version 468657 (0.0008) [2023-12-27 08:53:14,485][20347] Updated weights for policy 0, policy_version 469040 (0.0008) [2023-12-27 08:53:14,526][20381] Updated weights for policy 1, policy_version 468667 (0.0007) [2023-12-27 08:53:14,552][20347] Updated weights for policy 0, policy_version 469050 (0.0008) [2023-12-27 08:53:14,591][20381] Updated weights for policy 1, policy_version 468677 (0.0008) [2023-12-27 08:53:14,664][20381] Updated weights for policy 1, policy_version 468687 (0.0008) [2023-12-27 08:53:15,518][20347] Updated weights for policy 0, policy_version 469060 (0.0009) [2023-12-27 08:53:15,574][20381] Updated weights for policy 1, policy_version 468697 (0.0009) [2023-12-27 08:53:15,587][20347] Updated weights for policy 0, policy_version 469070 (0.0009) [2023-12-27 08:53:15,642][20381] Updated weights for policy 1, policy_version 468707 (0.0007) [2023-12-27 08:53:15,653][20347] Updated weights for policy 0, policy_version 469080 (0.0007) [2023-12-27 08:53:15,707][20381] Updated weights for policy 1, policy_version 468717 (0.0007) [2023-12-27 08:53:16,444][20347] Updated weights for policy 0, policy_version 469090 (0.0007) [2023-12-27 08:53:16,463][20381] Updated weights for policy 1, policy_version 468727 (0.0008) [2023-12-27 08:53:16,507][20347] Updated weights for policy 0, policy_version 469100 (0.0007) [2023-12-27 08:53:16,528][20381] Updated weights for policy 1, policy_version 468737 (0.0007) [2023-12-27 08:53:16,578][20347] Updated weights for policy 0, policy_version 469110 (0.0008) [2023-12-27 08:53:16,596][20381] Updated weights for policy 1, policy_version 468747 (0.0008) [2023-12-27 08:53:16,646][20347] Updated weights for policy 0, policy_version 469120 (0.0008) [2023-12-27 08:53:17,362][19170] Fps is (10 sec: 18022.3, 60 sec: 17203.1, 300 sec: 19994.0). Total num frames: 240132096. Throughput: 0: 8698.5, 1: 8718.5. Samples: 240145496. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:17,363][19170] Avg episode reward: [(0, '293.570'), (1, '961.410')] [2023-12-27 08:53:17,569][20381] Updated weights for policy 1, policy_version 468757 (0.0009) [2023-12-27 08:53:17,611][20347] Updated weights for policy 0, policy_version 469130 (0.0008) [2023-12-27 08:53:17,641][20381] Updated weights for policy 1, policy_version 468767 (0.0009) [2023-12-27 08:53:17,679][20347] Updated weights for policy 0, policy_version 469140 (0.0008) [2023-12-27 08:53:17,711][20381] Updated weights for policy 1, policy_version 468777 (0.0009) [2023-12-27 08:53:17,750][20347] Updated weights for policy 0, policy_version 469150 (0.0008) [2023-12-27 08:53:18,614][20381] Updated weights for policy 1, policy_version 468787 (0.0009) [2023-12-27 08:53:18,633][20347] Updated weights for policy 0, policy_version 469160 (0.0009) [2023-12-27 08:53:18,684][20381] Updated weights for policy 1, policy_version 468797 (0.0008) [2023-12-27 08:53:18,695][20347] Updated weights for policy 0, policy_version 469170 (0.0009) [2023-12-27 08:53:18,746][20381] Updated weights for policy 1, policy_version 468807 (0.0008) [2023-12-27 08:53:18,761][20347] Updated weights for policy 0, policy_version 469180 (0.0010) [2023-12-27 08:53:19,563][20381] Updated weights for policy 1, policy_version 468817 (0.0008) [2023-12-27 08:53:19,613][20347] Updated weights for policy 0, policy_version 469190 (0.0010) [2023-12-27 08:53:19,638][20381] Updated weights for policy 1, policy_version 468827 (0.0008) [2023-12-27 08:53:19,684][20347] Updated weights for policy 0, policy_version 469200 (0.0009) [2023-12-27 08:53:19,712][20381] Updated weights for policy 1, policy_version 468837 (0.0009) [2023-12-27 08:53:19,751][20347] Updated weights for policy 0, policy_version 469210 (0.0009) [2023-12-27 08:53:19,782][20381] Updated weights for policy 1, policy_version 468847 (0.0009) [2023-12-27 08:53:20,614][20347] Updated weights for policy 0, policy_version 469220 (0.0009) [2023-12-27 08:53:20,615][20381] Updated weights for policy 1, policy_version 468857 (0.0008) [2023-12-27 08:53:20,685][20347] Updated weights for policy 0, policy_version 469230 (0.0008) [2023-12-27 08:53:20,688][20381] Updated weights for policy 1, policy_version 468867 (0.0008) [2023-12-27 08:53:20,757][20347] Updated weights for policy 0, policy_version 469240 (0.0008) [2023-12-27 08:53:20,761][20381] Updated weights for policy 1, policy_version 468877 (0.0009) [2023-12-27 08:53:21,622][20347] Updated weights for policy 0, policy_version 469250 (0.0009) [2023-12-27 08:53:21,698][20381] Updated weights for policy 1, policy_version 468887 (0.0009) [2023-12-27 08:53:21,703][20347] Updated weights for policy 0, policy_version 469260 (0.0009) [2023-12-27 08:53:21,778][20381] Updated weights for policy 1, policy_version 468897 (0.0008) [2023-12-27 08:53:21,783][20347] Updated weights for policy 0, policy_version 469270 (0.0009) [2023-12-27 08:53:21,849][20381] Updated weights for policy 1, policy_version 468907 (0.0008) [2023-12-27 08:53:21,856][20347] Updated weights for policy 0, policy_version 469280 (0.0008) [2023-12-27 08:53:22,362][19170] Fps is (10 sec: 16383.9, 60 sec: 17203.2, 300 sec: 19938.5). Total num frames: 240214016. Throughput: 0: 8671.9, 1: 8681.9. Samples: 240192936. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:22,363][19170] Avg episode reward: [(0, '372.050'), (1, '990.340')] [2023-12-27 08:53:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000468912_120061952.pth... [2023-12-27 08:53:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000469280_120152064.pth... [2023-12-27 08:53:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000467888_119799808.pth [2023-12-27 08:53:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000468256_119889920.pth [2023-12-27 08:53:22,793][20381] Updated weights for policy 1, policy_version 468917 (0.0008) [2023-12-27 08:53:22,836][20347] Updated weights for policy 0, policy_version 469290 (0.0008) [2023-12-27 08:53:22,867][20381] Updated weights for policy 1, policy_version 468927 (0.0009) [2023-12-27 08:53:22,908][20347] Updated weights for policy 0, policy_version 469300 (0.0009) [2023-12-27 08:53:22,940][20381] Updated weights for policy 1, policy_version 468937 (0.0008) [2023-12-27 08:53:22,982][20347] Updated weights for policy 0, policy_version 469310 (0.0009) [2023-12-27 08:53:23,790][20381] Updated weights for policy 1, policy_version 468947 (0.0008) [2023-12-27 08:53:23,824][20347] Updated weights for policy 0, policy_version 469320 (0.0008) [2023-12-27 08:53:23,859][20381] Updated weights for policy 1, policy_version 468957 (0.0008) [2023-12-27 08:53:23,893][20347] Updated weights for policy 0, policy_version 469330 (0.0008) [2023-12-27 08:53:23,926][20381] Updated weights for policy 1, policy_version 468967 (0.0008) [2023-12-27 08:53:23,965][20347] Updated weights for policy 0, policy_version 469340 (0.0008) [2023-12-27 08:53:24,755][20381] Updated weights for policy 1, policy_version 468977 (0.0009) [2023-12-27 08:53:24,795][20347] Updated weights for policy 0, policy_version 469350 (0.0009) [2023-12-27 08:53:24,827][20381] Updated weights for policy 1, policy_version 468987 (0.0008) [2023-12-27 08:53:24,876][20347] Updated weights for policy 0, policy_version 469360 (0.0009) [2023-12-27 08:53:24,898][20381] Updated weights for policy 1, policy_version 468997 (0.0009) [2023-12-27 08:53:24,946][20347] Updated weights for policy 0, policy_version 469370 (0.0009) [2023-12-27 08:53:24,966][20381] Updated weights for policy 1, policy_version 469007 (0.0009) [2023-12-27 08:53:25,788][20347] Updated weights for policy 0, policy_version 469380 (0.0010) [2023-12-27 08:53:25,817][20381] Updated weights for policy 1, policy_version 469017 (0.0009) [2023-12-27 08:53:25,863][20347] Updated weights for policy 0, policy_version 469390 (0.0009) [2023-12-27 08:53:25,892][20381] Updated weights for policy 1, policy_version 469027 (0.0008) [2023-12-27 08:53:25,930][20347] Updated weights for policy 0, policy_version 469400 (0.0010) [2023-12-27 08:53:25,962][20381] Updated weights for policy 1, policy_version 469037 (0.0009) [2023-12-27 08:53:26,770][20381] Updated weights for policy 1, policy_version 469047 (0.0008) [2023-12-27 08:53:26,819][20347] Updated weights for policy 0, policy_version 469410 (0.0009) [2023-12-27 08:53:26,841][20381] Updated weights for policy 1, policy_version 469057 (0.0008) [2023-12-27 08:53:26,890][20347] Updated weights for policy 0, policy_version 469420 (0.0009) [2023-12-27 08:53:26,913][20381] Updated weights for policy 1, policy_version 469067 (0.0008) [2023-12-27 08:53:26,960][20347] Updated weights for policy 0, policy_version 469430 (0.0009) [2023-12-27 08:53:27,031][20347] Updated weights for policy 0, policy_version 469440 (0.0009) [2023-12-27 08:53:27,362][19170] Fps is (10 sec: 16383.9, 60 sec: 17203.2, 300 sec: 19827.4). Total num frames: 240295936. Throughput: 0: 8640.6, 1: 8657.2. Samples: 240288852. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:27,363][19170] Avg episode reward: [(0, '351.150'), (1, '990.270')] [2023-12-27 08:53:27,753][20381] Updated weights for policy 1, policy_version 469077 (0.0009) [2023-12-27 08:53:27,821][20381] Updated weights for policy 1, policy_version 469087 (0.0009) [2023-12-27 08:53:27,887][20381] Updated weights for policy 1, policy_version 469097 (0.0009) [2023-12-27 08:53:27,921][20347] Updated weights for policy 0, policy_version 469450 (0.0009) [2023-12-27 08:53:27,994][20347] Updated weights for policy 0, policy_version 469460 (0.0009) [2023-12-27 08:53:28,067][20347] Updated weights for policy 0, policy_version 469470 (0.0009) [2023-12-27 08:53:28,792][20381] Updated weights for policy 1, policy_version 469107 (0.0012) [2023-12-27 08:53:28,868][20381] Updated weights for policy 1, policy_version 469117 (0.0010) [2023-12-27 08:53:28,923][20347] Updated weights for policy 0, policy_version 469480 (0.0008) [2023-12-27 08:53:28,941][20381] Updated weights for policy 1, policy_version 469127 (0.0010) [2023-12-27 08:53:28,992][20347] Updated weights for policy 0, policy_version 469490 (0.0008) [2023-12-27 08:53:29,067][20347] Updated weights for policy 0, policy_version 469500 (0.0008) [2023-12-27 08:53:29,776][20381] Updated weights for policy 1, policy_version 469137 (0.0011) [2023-12-27 08:53:29,849][20381] Updated weights for policy 1, policy_version 469147 (0.0009) [2023-12-27 08:53:29,919][20381] Updated weights for policy 1, policy_version 469157 (0.0009) [2023-12-27 08:53:29,921][20347] Updated weights for policy 0, policy_version 469510 (0.0009) [2023-12-27 08:53:29,991][20381] Updated weights for policy 1, policy_version 469167 (0.0009) [2023-12-27 08:53:29,994][20347] Updated weights for policy 0, policy_version 469520 (0.0009) [2023-12-27 08:53:30,069][20347] Updated weights for policy 0, policy_version 469530 (0.0009) [2023-12-27 08:53:30,835][20381] Updated weights for policy 1, policy_version 469177 (0.0008) [2023-12-27 08:53:30,879][20292] KL-divergence is very high: 431.5969 [2023-12-27 08:53:30,887][20292] KL-divergence is very high: 284.8658 [2023-12-27 08:53:30,908][20381] Updated weights for policy 1, policy_version 469187 (0.0009) [2023-12-27 08:53:30,929][20347] Updated weights for policy 0, policy_version 469540 (0.0009) [2023-12-27 08:53:30,938][20292] KL-divergence is very high: 224.9976 [2023-12-27 08:53:30,946][20292] KL-divergence is very high: 202.1465 [2023-12-27 08:53:30,983][20381] Updated weights for policy 1, policy_version 469197 (0.0009) [2023-12-27 08:53:30,999][20292] KL-divergence is very high: 282.1264 [2023-12-27 08:53:31,002][20347] Updated weights for policy 0, policy_version 469550 (0.0008) [2023-12-27 08:53:31,078][20347] Updated weights for policy 0, policy_version 469560 (0.0009) [2023-12-27 08:53:31,849][20381] Updated weights for policy 1, policy_version 469207 (0.0008) [2023-12-27 08:53:31,917][20381] Updated weights for policy 1, policy_version 469217 (0.0008) [2023-12-27 08:53:31,927][20347] Updated weights for policy 0, policy_version 469570 (0.0008) [2023-12-27 08:53:31,988][20381] Updated weights for policy 1, policy_version 469227 (0.0008) [2023-12-27 08:53:31,997][20347] Updated weights for policy 0, policy_version 469580 (0.0008) [2023-12-27 08:53:32,069][20347] Updated weights for policy 0, policy_version 469590 (0.0008) [2023-12-27 08:53:32,139][20347] Updated weights for policy 0, policy_version 469600 (0.0009) [2023-12-27 08:53:32,362][19170] Fps is (10 sec: 16384.1, 60 sec: 17203.2, 300 sec: 19771.9). Total num frames: 240377856. Throughput: 0: 8563.3, 1: 8601.7. Samples: 240385768. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:32,363][19170] Avg episode reward: [(0, '403.490'), (1, '970.910')] [2023-12-27 08:53:32,794][20381] Updated weights for policy 1, policy_version 469237 (0.0009) [2023-12-27 08:53:32,864][20381] Updated weights for policy 1, policy_version 469247 (0.0009) [2023-12-27 08:53:32,931][20381] Updated weights for policy 1, policy_version 469257 (0.0008) [2023-12-27 08:53:32,948][20347] Updated weights for policy 0, policy_version 469610 (0.0008) [2023-12-27 08:53:33,018][20347] Updated weights for policy 0, policy_version 469620 (0.0008) [2023-12-27 08:53:33,086][20347] Updated weights for policy 0, policy_version 469630 (0.0009) [2023-12-27 08:53:33,724][20381] Updated weights for policy 1, policy_version 469267 (0.0008) [2023-12-27 08:53:33,794][20381] Updated weights for policy 1, policy_version 469277 (0.0009) [2023-12-27 08:53:33,867][20381] Updated weights for policy 1, policy_version 469287 (0.0009) [2023-12-27 08:53:33,896][20347] Updated weights for policy 0, policy_version 469640 (0.0008) [2023-12-27 08:53:33,967][20347] Updated weights for policy 0, policy_version 469650 (0.0009) [2023-12-27 08:53:34,035][20347] Updated weights for policy 0, policy_version 469660 (0.0009) [2023-12-27 08:53:34,686][20381] Updated weights for policy 1, policy_version 469297 (0.0009) [2023-12-27 08:53:34,755][20381] Updated weights for policy 1, policy_version 469307 (0.0007) [2023-12-27 08:53:34,823][20381] Updated weights for policy 1, policy_version 469317 (0.0008) [2023-12-27 08:53:34,857][20347] Updated weights for policy 0, policy_version 469670 (0.0008) [2023-12-27 08:53:34,891][20381] Updated weights for policy 1, policy_version 469327 (0.0009) [2023-12-27 08:53:34,930][20347] Updated weights for policy 0, policy_version 469680 (0.0008) [2023-12-27 08:53:35,000][20347] Updated weights for policy 0, policy_version 469690 (0.0007) [2023-12-27 08:53:35,679][20381] Updated weights for policy 1, policy_version 469337 (0.0008) [2023-12-27 08:53:35,750][20381] Updated weights for policy 1, policy_version 469347 (0.0009) [2023-12-27 08:53:35,768][20347] Updated weights for policy 0, policy_version 469700 (0.0009) [2023-12-27 08:53:35,818][20381] Updated weights for policy 1, policy_version 469357 (0.0008) [2023-12-27 08:53:35,836][20347] Updated weights for policy 0, policy_version 469710 (0.0008) [2023-12-27 08:53:35,908][20347] Updated weights for policy 0, policy_version 469720 (0.0009) [2023-12-27 08:53:36,651][20381] Updated weights for policy 1, policy_version 469367 (0.0008) [2023-12-27 08:53:36,722][20381] Updated weights for policy 1, policy_version 469377 (0.0009) [2023-12-27 08:53:36,736][20347] Updated weights for policy 0, policy_version 469730 (0.0008) [2023-12-27 08:53:36,797][20381] Updated weights for policy 1, policy_version 469387 (0.0008) [2023-12-27 08:53:36,804][20347] Updated weights for policy 0, policy_version 469740 (0.0008) [2023-12-27 08:53:36,875][20347] Updated weights for policy 0, policy_version 469750 (0.0008) [2023-12-27 08:53:36,950][20347] Updated weights for policy 0, policy_version 469760 (0.0008) [2023-12-27 08:53:37,362][19170] Fps is (10 sec: 16384.1, 60 sec: 17203.2, 300 sec: 19716.3). Total num frames: 240459776. Throughput: 0: 8557.3, 1: 8610.1. Samples: 240436860. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:37,363][19170] Avg episode reward: [(0, '376.160'), (1, '980.640')] [2023-12-27 08:53:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000469392_120184832.pth... [2023-12-27 08:53:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000469760_120274944.pth... [2023-12-27 08:53:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000468400_119930880.pth [2023-12-27 08:53:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000468800_120029184.pth [2023-12-27 08:53:37,592][20381] Updated weights for policy 1, policy_version 469397 (0.0008) [2023-12-27 08:53:37,659][20381] Updated weights for policy 1, policy_version 469407 (0.0008) [2023-12-27 08:53:37,730][20381] Updated weights for policy 1, policy_version 469417 (0.0008) [2023-12-27 08:53:37,753][20347] Updated weights for policy 0, policy_version 469770 (0.0007) [2023-12-27 08:53:37,823][20347] Updated weights for policy 0, policy_version 469780 (0.0008) [2023-12-27 08:53:37,888][20347] Updated weights for policy 0, policy_version 469790 (0.0008) [2023-12-27 08:53:38,533][20381] Updated weights for policy 1, policy_version 469427 (0.0008) [2023-12-27 08:53:38,604][20381] Updated weights for policy 1, policy_version 469437 (0.0008) [2023-12-27 08:53:38,673][20381] Updated weights for policy 1, policy_version 469447 (0.0008) [2023-12-27 08:53:38,678][20347] Updated weights for policy 0, policy_version 469800 (0.0008) [2023-12-27 08:53:38,750][20347] Updated weights for policy 0, policy_version 469810 (0.0007) [2023-12-27 08:53:38,820][20347] Updated weights for policy 0, policy_version 469820 (0.0008) [2023-12-27 08:53:39,493][20381] Updated weights for policy 1, policy_version 469457 (0.0008) [2023-12-27 08:53:39,564][20381] Updated weights for policy 1, policy_version 469467 (0.0008) [2023-12-27 08:53:39,600][20347] Updated weights for policy 0, policy_version 469830 (0.0008) [2023-12-27 08:53:39,633][20381] Updated weights for policy 1, policy_version 469477 (0.0007) [2023-12-27 08:53:39,669][20347] Updated weights for policy 0, policy_version 469840 (0.0008) [2023-12-27 08:53:39,704][20381] Updated weights for policy 1, policy_version 469487 (0.0009) [2023-12-27 08:53:39,740][20347] Updated weights for policy 0, policy_version 469850 (0.0008) [2023-12-27 08:53:40,460][20347] Updated weights for policy 0, policy_version 469860 (0.0008) [2023-12-27 08:53:40,491][20381] Updated weights for policy 1, policy_version 469497 (0.0007) [2023-12-27 08:53:40,525][20347] Updated weights for policy 0, policy_version 469870 (0.0007) [2023-12-27 08:53:40,557][20381] Updated weights for policy 1, policy_version 469507 (0.0008) [2023-12-27 08:53:40,582][20347] Updated weights for policy 0, policy_version 469880 (0.0007) [2023-12-27 08:53:40,621][20381] Updated weights for policy 1, policy_version 469517 (0.0007) [2023-12-27 08:53:41,380][20347] Updated weights for policy 0, policy_version 469890 (0.0007) [2023-12-27 08:53:41,449][20381] Updated weights for policy 1, policy_version 469527 (0.0008) [2023-12-27 08:53:41,457][20347] Updated weights for policy 0, policy_version 469900 (0.0008) [2023-12-27 08:53:41,525][20381] Updated weights for policy 1, policy_version 469537 (0.0009) [2023-12-27 08:53:41,526][20347] Updated weights for policy 0, policy_version 469910 (0.0008) [2023-12-27 08:53:41,601][20381] Updated weights for policy 1, policy_version 469547 (0.0008) [2023-12-27 08:53:41,613][20347] Updated weights for policy 0, policy_version 469920 (0.0008) [2023-12-27 08:53:42,362][19170] Fps is (10 sec: 16384.0, 60 sec: 17203.2, 300 sec: 19633.0). Total num frames: 240541696. Throughput: 0: 8650.8, 1: 8676.9. Samples: 240540960. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:42,363][19170] Avg episode reward: [(0, '408.880'), (1, '1000.000')] [2023-12-27 08:53:42,560][20347] Updated weights for policy 0, policy_version 469930 (0.0009) [2023-12-27 08:53:42,587][20381] Updated weights for policy 1, policy_version 469557 (0.0008) [2023-12-27 08:53:42,637][20347] Updated weights for policy 0, policy_version 469940 (0.0009) [2023-12-27 08:53:42,662][20381] Updated weights for policy 1, policy_version 469567 (0.0011) [2023-12-27 08:53:42,706][20347] Updated weights for policy 0, policy_version 469950 (0.0009) [2023-12-27 08:53:42,737][20381] Updated weights for policy 1, policy_version 469577 (0.0009) [2023-12-27 08:53:43,532][20347] Updated weights for policy 0, policy_version 469960 (0.0008) [2023-12-27 08:53:43,571][20381] Updated weights for policy 1, policy_version 469587 (0.0009) [2023-12-27 08:53:43,603][20347] Updated weights for policy 0, policy_version 469970 (0.0009) [2023-12-27 08:53:43,646][20381] Updated weights for policy 1, policy_version 469597 (0.0008) [2023-12-27 08:53:43,675][20347] Updated weights for policy 0, policy_version 469980 (0.0008) [2023-12-27 08:53:43,716][20381] Updated weights for policy 1, policy_version 469607 (0.0009) [2023-12-27 08:53:44,526][20347] Updated weights for policy 0, policy_version 469990 (0.0009) [2023-12-27 08:53:44,554][20381] Updated weights for policy 1, policy_version 469617 (0.0009) [2023-12-27 08:53:44,597][20347] Updated weights for policy 0, policy_version 470000 (0.0009) [2023-12-27 08:53:44,627][20381] Updated weights for policy 1, policy_version 469627 (0.0008) [2023-12-27 08:53:44,668][20347] Updated weights for policy 0, policy_version 470010 (0.0008) [2023-12-27 08:53:44,702][20381] Updated weights for policy 1, policy_version 469637 (0.0013) [2023-12-27 08:53:44,777][20381] Updated weights for policy 1, policy_version 469647 (0.0009) [2023-12-27 08:53:45,536][20347] Updated weights for policy 0, policy_version 470020 (0.0009) [2023-12-27 08:53:45,609][20347] Updated weights for policy 0, policy_version 470030 (0.0009) [2023-12-27 08:53:45,631][20381] Updated weights for policy 1, policy_version 469657 (0.0009) [2023-12-27 08:53:45,680][20347] Updated weights for policy 0, policy_version 470040 (0.0008) [2023-12-27 08:53:45,707][20381] Updated weights for policy 1, policy_version 469667 (0.0008) [2023-12-27 08:53:45,774][20381] Updated weights for policy 1, policy_version 469677 (0.0008) [2023-12-27 08:53:46,538][20347] Updated weights for policy 0, policy_version 470050 (0.0009) [2023-12-27 08:53:46,610][20347] Updated weights for policy 0, policy_version 470060 (0.0009) [2023-12-27 08:53:46,622][20381] Updated weights for policy 1, policy_version 469687 (0.0008) [2023-12-27 08:53:46,679][20347] Updated weights for policy 0, policy_version 470070 (0.0009) [2023-12-27 08:53:46,693][20381] Updated weights for policy 1, policy_version 469697 (0.0009) [2023-12-27 08:53:46,750][20347] Updated weights for policy 0, policy_version 470080 (0.0009) [2023-12-27 08:53:46,762][20381] Updated weights for policy 1, policy_version 469707 (0.0008) [2023-12-27 08:53:47,362][19170] Fps is (10 sec: 16384.0, 60 sec: 16930.1, 300 sec: 19549.7). Total num frames: 240623616. Throughput: 0: 8597.0, 1: 8604.0. Samples: 240636596. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:47,363][19170] Avg episode reward: [(0, '497.640'), (1, '1000.000')] [2023-12-27 08:53:47,579][20347] Updated weights for policy 0, policy_version 470090 (0.0008) [2023-12-27 08:53:47,612][20381] Updated weights for policy 1, policy_version 469717 (0.0008) [2023-12-27 08:53:47,652][20347] Updated weights for policy 0, policy_version 470100 (0.0008) [2023-12-27 08:53:47,682][20381] Updated weights for policy 1, policy_version 469727 (0.0008) [2023-12-27 08:53:47,726][20347] Updated weights for policy 0, policy_version 470110 (0.0008) [2023-12-27 08:53:47,761][20381] Updated weights for policy 1, policy_version 469738 (0.0009) [2023-12-27 08:53:48,583][20347] Updated weights for policy 0, policy_version 470120 (0.0010) [2023-12-27 08:53:48,648][20381] Updated weights for policy 1, policy_version 469748 (0.0011) [2023-12-27 08:53:48,652][20347] Updated weights for policy 0, policy_version 470130 (0.0008) [2023-12-27 08:53:48,720][20381] Updated weights for policy 1, policy_version 469758 (0.0010) [2023-12-27 08:53:48,722][20347] Updated weights for policy 0, policy_version 470140 (0.0009) [2023-12-27 08:53:48,789][20381] Updated weights for policy 1, policy_version 469768 (0.0010) [2023-12-27 08:53:49,607][20347] Updated weights for policy 0, policy_version 470150 (0.0009) [2023-12-27 08:53:49,664][20381] Updated weights for policy 1, policy_version 469778 (0.0010) [2023-12-27 08:53:49,679][20347] Updated weights for policy 0, policy_version 470160 (0.0009) [2023-12-27 08:53:49,738][20381] Updated weights for policy 1, policy_version 469788 (0.0009) [2023-12-27 08:53:49,756][20347] Updated weights for policy 0, policy_version 470170 (0.0008) [2023-12-27 08:53:49,811][20381] Updated weights for policy 1, policy_version 469798 (0.0009) [2023-12-27 08:53:49,881][20381] Updated weights for policy 1, policy_version 469808 (0.0008) [2023-12-27 08:53:50,638][20347] Updated weights for policy 0, policy_version 470180 (0.0008) [2023-12-27 08:53:50,714][20347] Updated weights for policy 0, policy_version 470190 (0.0008) [2023-12-27 08:53:50,725][20381] Updated weights for policy 1, policy_version 469818 (0.0007) [2023-12-27 08:53:50,787][20347] Updated weights for policy 0, policy_version 470200 (0.0008) [2023-12-27 08:53:50,790][20381] Updated weights for policy 1, policy_version 469828 (0.0011) [2023-12-27 08:53:50,856][20381] Updated weights for policy 1, policy_version 469838 (0.0009) [2023-12-27 08:53:51,647][20347] Updated weights for policy 0, policy_version 470210 (0.0009) [2023-12-27 08:53:51,719][20381] Updated weights for policy 1, policy_version 469848 (0.0009) [2023-12-27 08:53:51,720][20347] Updated weights for policy 0, policy_version 470220 (0.0009) [2023-12-27 08:53:51,791][20347] Updated weights for policy 0, policy_version 470230 (0.0009) [2023-12-27 08:53:51,797][20381] Updated weights for policy 1, policy_version 469858 (0.0009) [2023-12-27 08:53:51,864][20347] Updated weights for policy 0, policy_version 470240 (0.0008) [2023-12-27 08:53:51,867][20381] Updated weights for policy 1, policy_version 469868 (0.0009) [2023-12-27 08:53:52,362][19170] Fps is (10 sec: 16383.8, 60 sec: 16930.1, 300 sec: 19466.4). Total num frames: 240705536. Throughput: 0: 8470.6, 1: 8454.3. Samples: 240684524. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:52,363][19170] Avg episode reward: [(0, '540.500'), (1, '1000.000')] [2023-12-27 08:53:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000469872_120307712.pth... [2023-12-27 08:53:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000470240_120397824.pth... [2023-12-27 08:53:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000468912_120061952.pth [2023-12-27 08:53:52,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000469280_120152064.pth [2023-12-27 08:53:52,717][20347] Updated weights for policy 0, policy_version 470250 (0.0008) [2023-12-27 08:53:52,733][20381] Updated weights for policy 1, policy_version 469878 (0.0008) [2023-12-27 08:53:52,783][20347] Updated weights for policy 0, policy_version 470260 (0.0008) [2023-12-27 08:53:52,803][20381] Updated weights for policy 1, policy_version 469888 (0.0008) [2023-12-27 08:53:52,851][20347] Updated weights for policy 0, policy_version 470270 (0.0008) [2023-12-27 08:53:52,869][20381] Updated weights for policy 1, policy_version 469898 (0.0008) [2023-12-27 08:53:53,641][20347] Updated weights for policy 0, policy_version 470280 (0.0009) [2023-12-27 08:53:53,696][20381] Updated weights for policy 1, policy_version 469908 (0.0009) [2023-12-27 08:53:53,709][20347] Updated weights for policy 0, policy_version 470290 (0.0008) [2023-12-27 08:53:53,764][20381] Updated weights for policy 1, policy_version 469918 (0.0009) [2023-12-27 08:53:53,780][20347] Updated weights for policy 0, policy_version 470300 (0.0009) [2023-12-27 08:53:53,831][20381] Updated weights for policy 1, policy_version 469928 (0.0008) [2023-12-27 08:53:54,574][20347] Updated weights for policy 0, policy_version 470310 (0.0009) [2023-12-27 08:53:54,633][20381] Updated weights for policy 1, policy_version 469938 (0.0008) [2023-12-27 08:53:54,639][20347] Updated weights for policy 0, policy_version 470320 (0.0008) [2023-12-27 08:53:54,696][20381] Updated weights for policy 1, policy_version 469948 (0.0008) [2023-12-27 08:53:54,708][20347] Updated weights for policy 0, policy_version 470330 (0.0010) [2023-12-27 08:53:54,763][20381] Updated weights for policy 1, policy_version 469958 (0.0008) [2023-12-27 08:53:54,826][20381] Updated weights for policy 1, policy_version 469968 (0.0008) [2023-12-27 08:53:55,490][20347] Updated weights for policy 0, policy_version 470340 (0.0010) [2023-12-27 08:53:55,554][20347] Updated weights for policy 0, policy_version 470350 (0.0011) [2023-12-27 08:53:55,620][20347] Updated weights for policy 0, policy_version 470360 (0.0010) [2023-12-27 08:53:55,624][20381] Updated weights for policy 1, policy_version 469978 (0.0008) [2023-12-27 08:53:55,694][20381] Updated weights for policy 1, policy_version 469988 (0.0007) [2023-12-27 08:53:55,765][20381] Updated weights for policy 1, policy_version 469998 (0.0008) [2023-12-27 08:53:56,423][20381] Updated weights for policy 1, policy_version 470008 (0.0007) [2023-12-27 08:53:56,430][20347] Updated weights for policy 0, policy_version 470370 (0.0012) [2023-12-27 08:53:56,476][20381] Updated weights for policy 1, policy_version 470018 (0.0006) [2023-12-27 08:53:56,485][20347] Updated weights for policy 0, policy_version 470380 (0.0010) [2023-12-27 08:53:56,529][20381] Updated weights for policy 1, policy_version 470028 (0.0005) [2023-12-27 08:53:56,534][20347] Updated weights for policy 0, policy_version 470390 (0.0010) [2023-12-27 08:53:56,593][20347] Updated weights for policy 0, policy_version 470400 (0.0010) [2023-12-27 08:53:57,234][20381] Updated weights for policy 1, policy_version 470038 (0.0007) [2023-12-27 08:53:57,267][20347] Updated weights for policy 0, policy_version 470410 (0.0005) [2023-12-27 08:53:57,292][20381] Updated weights for policy 1, policy_version 470048 (0.0008) [2023-12-27 08:53:57,320][20347] Updated weights for policy 0, policy_version 470420 (0.0006) [2023-12-27 08:53:57,346][20381] Updated weights for policy 1, policy_version 470058 (0.0008) [2023-12-27 08:53:57,363][19170] Fps is (10 sec: 16383.8, 60 sec: 16930.1, 300 sec: 19410.9). Total num frames: 240787456. Throughput: 0: 8180.6, 1: 8200.5. Samples: 240785352. Policy #0 lag: (min: 22.0, avg: 33.5, max: 54.0) [2023-12-27 08:53:57,364][19170] Avg episode reward: [(0, '394.670'), (1, '1000.000')] [2023-12-27 08:53:57,370][20347] Updated weights for policy 0, policy_version 470430 (0.0007) [2023-12-27 08:53:57,981][20347] Updated weights for policy 0, policy_version 470440 (0.0009) [2023-12-27 08:53:58,034][20347] Updated weights for policy 0, policy_version 470450 (0.0010) [2023-12-27 08:53:58,072][20381] Updated weights for policy 1, policy_version 470068 (0.0007) [2023-12-27 08:53:58,095][20347] Updated weights for policy 0, policy_version 470460 (0.0010) [2023-12-27 08:53:58,127][20381] Updated weights for policy 1, policy_version 470078 (0.0007) [2023-12-27 08:53:58,187][20381] Updated weights for policy 1, policy_version 470088 (0.0007) [2023-12-27 08:53:58,855][20347] Updated weights for policy 0, policy_version 470470 (0.0010) [2023-12-27 08:53:58,907][20381] Updated weights for policy 1, policy_version 470098 (0.0008) [2023-12-27 08:53:58,916][20347] Updated weights for policy 0, policy_version 470480 (0.0011) [2023-12-27 08:53:58,960][20381] Updated weights for policy 1, policy_version 470108 (0.0007) [2023-12-27 08:53:58,972][20347] Updated weights for policy 0, policy_version 470490 (0.0010) [2023-12-27 08:53:59,019][20381] Updated weights for policy 1, policy_version 470118 (0.0006) [2023-12-27 08:53:59,078][20381] Updated weights for policy 1, policy_version 470128 (0.0007) [2023-12-27 08:53:59,687][20347] Updated weights for policy 0, policy_version 470500 (0.0011) [2023-12-27 08:53:59,750][20347] Updated weights for policy 0, policy_version 470510 (0.0006) [2023-12-27 08:53:59,760][20381] Updated weights for policy 1, policy_version 470138 (0.0006) [2023-12-27 08:53:59,809][20347] Updated weights for policy 0, policy_version 470520 (0.0010) [2023-12-27 08:53:59,819][20381] Updated weights for policy 1, policy_version 470148 (0.0006) [2023-12-27 08:53:59,879][20381] Updated weights for policy 1, policy_version 470158 (0.0007) [2023-12-27 08:54:00,489][20347] Updated weights for policy 0, policy_version 470530 (0.0010) [2023-12-27 08:54:00,549][20347] Updated weights for policy 0, policy_version 470540 (0.0010) [2023-12-27 08:54:00,602][20347] Updated weights for policy 0, policy_version 470550 (0.0008) [2023-12-27 08:54:00,604][20381] Updated weights for policy 1, policy_version 470168 (0.0007) [2023-12-27 08:54:00,657][20347] Updated weights for policy 0, policy_version 470560 (0.0006) [2023-12-27 08:54:00,666][20381] Updated weights for policy 1, policy_version 470178 (0.0008) [2023-12-27 08:54:00,721][20381] Updated weights for policy 1, policy_version 470188 (0.0007) [2023-12-27 08:54:01,260][20347] Updated weights for policy 0, policy_version 470570 (0.0009) [2023-12-27 08:54:01,314][20347] Updated weights for policy 0, policy_version 470580 (0.0006) [2023-12-27 08:54:01,380][20347] Updated weights for policy 0, policy_version 470590 (0.0007) [2023-12-27 08:54:01,394][20381] Updated weights for policy 1, policy_version 470198 (0.0007) [2023-12-27 08:54:01,456][20381] Updated weights for policy 1, policy_version 470208 (0.0008) [2023-12-27 08:54:01,514][20381] Updated weights for policy 1, policy_version 470218 (0.0010) [2023-12-27 08:54:02,072][20347] Updated weights for policy 0, policy_version 470600 (0.0007) [2023-12-27 08:54:02,127][20347] Updated weights for policy 0, policy_version 470610 (0.0008) [2023-12-27 08:54:02,176][20381] Updated weights for policy 1, policy_version 470228 (0.0010) [2023-12-27 08:54:02,183][20347] Updated weights for policy 0, policy_version 470620 (0.0008) [2023-12-27 08:54:02,235][20381] Updated weights for policy 1, policy_version 470238 (0.0010) [2023-12-27 08:54:02,305][20381] Updated weights for policy 1, policy_version 470248 (0.0010) [2023-12-27 08:54:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 17476.3, 300 sec: 19438.6). Total num frames: 240902144. Throughput: 0: 8468.8, 1: 8442.1. Samples: 240906488. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:02,363][19170] Avg episode reward: [(0, '531.880'), (1, '1000.000')] [2023-12-27 08:54:02,893][20347] Updated weights for policy 0, policy_version 470630 (0.0008) [2023-12-27 08:54:02,952][20347] Updated weights for policy 0, policy_version 470640 (0.0008) [2023-12-27 08:54:02,967][20381] Updated weights for policy 1, policy_version 470258 (0.0009) [2023-12-27 08:54:03,007][20347] Updated weights for policy 0, policy_version 470650 (0.0006) [2023-12-27 08:54:03,022][20381] Updated weights for policy 1, policy_version 470268 (0.0006) [2023-12-27 08:54:03,070][20381] Updated weights for policy 1, policy_version 470278 (0.0005) [2023-12-27 08:54:03,124][20381] Updated weights for policy 1, policy_version 470288 (0.0007) [2023-12-27 08:54:03,647][20347] Updated weights for policy 0, policy_version 470660 (0.0007) [2023-12-27 08:54:03,704][20347] Updated weights for policy 0, policy_version 470670 (0.0008) [2023-12-27 08:54:03,726][20381] Updated weights for policy 1, policy_version 470298 (0.0010) [2023-12-27 08:54:03,757][20347] Updated weights for policy 0, policy_version 470680 (0.0006) [2023-12-27 08:54:03,784][20381] Updated weights for policy 1, policy_version 470308 (0.0010) [2023-12-27 08:54:03,843][20381] Updated weights for policy 1, policy_version 470318 (0.0011) [2023-12-27 08:54:04,424][20347] Updated weights for policy 0, policy_version 470690 (0.0006) [2023-12-27 08:54:04,482][20347] Updated weights for policy 0, policy_version 470700 (0.0007) [2023-12-27 08:54:04,507][20381] Updated weights for policy 1, policy_version 470328 (0.0007) [2023-12-27 08:54:04,541][20347] Updated weights for policy 0, policy_version 470710 (0.0007) [2023-12-27 08:54:04,565][20381] Updated weights for policy 1, policy_version 470338 (0.0009) [2023-12-27 08:54:04,590][20347] Updated weights for policy 0, policy_version 470720 (0.0006) [2023-12-27 08:54:04,624][20381] Updated weights for policy 1, policy_version 470348 (0.0007) [2023-12-27 08:54:05,294][20347] Updated weights for policy 0, policy_version 470730 (0.0006) [2023-12-27 08:54:05,309][20381] Updated weights for policy 1, policy_version 470358 (0.0007) [2023-12-27 08:54:05,353][20347] Updated weights for policy 0, policy_version 470740 (0.0005) [2023-12-27 08:54:05,356][20381] Updated weights for policy 1, policy_version 470368 (0.0008) [2023-12-27 08:54:05,401][20347] Updated weights for policy 0, policy_version 470750 (0.0005) [2023-12-27 08:54:05,412][20381] Updated weights for policy 1, policy_version 470378 (0.0007) [2023-12-27 08:54:06,076][20347] Updated weights for policy 0, policy_version 470760 (0.0007) [2023-12-27 08:54:06,089][20381] Updated weights for policy 1, policy_version 470388 (0.0009) [2023-12-27 08:54:06,136][20347] Updated weights for policy 0, policy_version 470770 (0.0006) [2023-12-27 08:54:06,148][20381] Updated weights for policy 1, policy_version 470398 (0.0008) [2023-12-27 08:54:06,195][20347] Updated weights for policy 0, policy_version 470780 (0.0006) [2023-12-27 08:54:06,205][20381] Updated weights for policy 1, policy_version 470408 (0.0007) [2023-12-27 08:54:06,859][20347] Updated weights for policy 0, policy_version 470790 (0.0006) [2023-12-27 08:54:06,909][20347] Updated weights for policy 0, policy_version 470800 (0.0007) [2023-12-27 08:54:06,926][20381] Updated weights for policy 1, policy_version 470418 (0.0009) [2023-12-27 08:54:06,964][20347] Updated weights for policy 0, policy_version 470810 (0.0006) [2023-12-27 08:54:06,974][20381] Updated weights for policy 1, policy_version 470428 (0.0006) [2023-12-27 08:54:07,030][20381] Updated weights for policy 1, policy_version 470438 (0.0008) [2023-12-27 08:54:07,086][20381] Updated weights for policy 1, policy_version 470448 (0.0008) [2023-12-27 08:54:07,363][19170] Fps is (10 sec: 21299.2, 60 sec: 17476.2, 300 sec: 19438.6). Total num frames: 241000448. Throughput: 0: 8623.8, 1: 8610.8. Samples: 240968496. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:07,364][19170] Avg episode reward: [(0, '598.360'), (1, '1000.000')] [2023-12-27 08:54:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000470448_120455168.pth... [2023-12-27 08:54:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000470816_120545280.pth... [2023-12-27 08:54:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000469392_120184832.pth [2023-12-27 08:54:07,401][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000469760_120274944.pth [2023-12-27 08:54:07,600][20347] Updated weights for policy 0, policy_version 470820 (0.0006) [2023-12-27 08:54:07,656][20347] Updated weights for policy 0, policy_version 470830 (0.0005) [2023-12-27 08:54:07,715][20347] Updated weights for policy 0, policy_version 470840 (0.0006) [2023-12-27 08:54:07,751][20381] Updated weights for policy 1, policy_version 470458 (0.0007) [2023-12-27 08:54:07,824][20381] Updated weights for policy 1, policy_version 470468 (0.0011) [2023-12-27 08:54:07,875][20381] Updated weights for policy 1, policy_version 470478 (0.0010) [2023-12-27 08:54:08,384][20347] Updated weights for policy 0, policy_version 470850 (0.0006) [2023-12-27 08:54:08,445][20347] Updated weights for policy 0, policy_version 470860 (0.0007) [2023-12-27 08:54:08,506][20347] Updated weights for policy 0, policy_version 470870 (0.0008) [2023-12-27 08:54:08,571][20381] Updated weights for policy 1, policy_version 470488 (0.0011) [2023-12-27 08:54:08,572][20347] Updated weights for policy 0, policy_version 470880 (0.0006) [2023-12-27 08:54:08,620][20381] Updated weights for policy 1, policy_version 470498 (0.0010) [2023-12-27 08:54:08,668][20381] Updated weights for policy 1, policy_version 470508 (0.0010) [2023-12-27 08:54:09,261][20347] Updated weights for policy 0, policy_version 470890 (0.0007) [2023-12-27 08:54:09,319][20347] Updated weights for policy 0, policy_version 470900 (0.0007) [2023-12-27 08:54:09,387][20347] Updated weights for policy 0, policy_version 470910 (0.0007) [2023-12-27 08:54:09,402][20381] Updated weights for policy 1, policy_version 470518 (0.0010) [2023-12-27 08:54:09,462][20381] Updated weights for policy 1, policy_version 470528 (0.0008) [2023-12-27 08:54:09,522][20381] Updated weights for policy 1, policy_version 470538 (0.0008) [2023-12-27 08:54:10,115][20347] Updated weights for policy 0, policy_version 470920 (0.0009) [2023-12-27 08:54:10,173][20347] Updated weights for policy 0, policy_version 470930 (0.0008) [2023-12-27 08:54:10,195][20381] Updated weights for policy 1, policy_version 470548 (0.0009) [2023-12-27 08:54:10,224][20347] Updated weights for policy 0, policy_version 470940 (0.0007) [2023-12-27 08:54:10,254][20381] Updated weights for policy 1, policy_version 470558 (0.0010) [2023-12-27 08:54:10,310][20381] Updated weights for policy 1, policy_version 470568 (0.0011) [2023-12-27 08:54:10,943][20347] Updated weights for policy 0, policy_version 470950 (0.0008) [2023-12-27 08:54:11,000][20381] Updated weights for policy 1, policy_version 470578 (0.0011) [2023-12-27 08:54:11,001][20347] Updated weights for policy 0, policy_version 470960 (0.0007) [2023-12-27 08:54:11,066][20381] Updated weights for policy 1, policy_version 470588 (0.0009) [2023-12-27 08:54:11,066][20347] Updated weights for policy 0, policy_version 470970 (0.0007) [2023-12-27 08:54:11,130][20381] Updated weights for policy 1, policy_version 470598 (0.0011) [2023-12-27 08:54:11,189][20381] Updated weights for policy 1, policy_version 470608 (0.0008) [2023-12-27 08:54:11,788][20347] Updated weights for policy 0, policy_version 470980 (0.0008) [2023-12-27 08:54:11,845][20347] Updated weights for policy 0, policy_version 470990 (0.0011) [2023-12-27 08:54:11,895][20347] Updated weights for policy 0, policy_version 471000 (0.0010) [2023-12-27 08:54:11,920][20381] Updated weights for policy 1, policy_version 470618 (0.0007) [2023-12-27 08:54:11,979][20381] Updated weights for policy 1, policy_version 470628 (0.0009) [2023-12-27 08:54:12,040][20381] Updated weights for policy 1, policy_version 470638 (0.0008) [2023-12-27 08:54:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 17476.3, 300 sec: 19438.6). Total num frames: 241098752. Throughput: 0: 8893.2, 1: 8878.3. Samples: 241088568. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:12,363][19170] Avg episode reward: [(0, '815.270'), (1, '1000.000')] [2023-12-27 08:54:12,629][20347] Updated weights for policy 0, policy_version 471010 (0.0010) [2023-12-27 08:54:12,694][20347] Updated weights for policy 0, policy_version 471020 (0.0008) [2023-12-27 08:54:12,748][20347] Updated weights for policy 0, policy_version 471030 (0.0006) [2023-12-27 08:54:12,750][20381] Updated weights for policy 1, policy_version 470648 (0.0010) [2023-12-27 08:54:12,804][20347] Updated weights for policy 0, policy_version 471040 (0.0005) [2023-12-27 08:54:12,809][20381] Updated weights for policy 1, policy_version 470658 (0.0011) [2023-12-27 08:54:12,873][20381] Updated weights for policy 1, policy_version 470668 (0.0011) [2023-12-27 08:54:13,459][20347] Updated weights for policy 0, policy_version 471050 (0.0006) [2023-12-27 08:54:13,514][20347] Updated weights for policy 0, policy_version 471060 (0.0006) [2023-12-27 08:54:13,572][20347] Updated weights for policy 0, policy_version 471070 (0.0006) [2023-12-27 08:54:13,574][20381] Updated weights for policy 1, policy_version 470678 (0.0011) [2023-12-27 08:54:13,632][20381] Updated weights for policy 1, policy_version 470688 (0.0010) [2023-12-27 08:54:13,693][20381] Updated weights for policy 1, policy_version 470698 (0.0005) [2023-12-27 08:54:14,164][20347] Updated weights for policy 0, policy_version 471080 (0.0006) [2023-12-27 08:54:14,220][20347] Updated weights for policy 0, policy_version 471090 (0.0007) [2023-12-27 08:54:14,268][20347] Updated weights for policy 0, policy_version 471100 (0.0008) [2023-12-27 08:54:14,279][20381] Updated weights for policy 1, policy_version 470708 (0.0005) [2023-12-27 08:54:14,328][20381] Updated weights for policy 1, policy_version 470718 (0.0006) [2023-12-27 08:54:14,389][20381] Updated weights for policy 1, policy_version 470728 (0.0007) [2023-12-27 08:54:14,906][20347] Updated weights for policy 0, policy_version 471110 (0.0007) [2023-12-27 08:54:14,967][20347] Updated weights for policy 0, policy_version 471120 (0.0007) [2023-12-27 08:54:15,022][20347] Updated weights for policy 0, policy_version 471130 (0.0007) [2023-12-27 08:54:15,062][20381] Updated weights for policy 1, policy_version 470738 (0.0008) [2023-12-27 08:54:15,118][20381] Updated weights for policy 1, policy_version 470748 (0.0010) [2023-12-27 08:54:15,179][20381] Updated weights for policy 1, policy_version 470758 (0.0009) [2023-12-27 08:54:15,239][20381] Updated weights for policy 1, policy_version 470768 (0.0010) [2023-12-27 08:54:15,602][20347] Updated weights for policy 0, policy_version 471140 (0.0008) [2023-12-27 08:54:15,658][20347] Updated weights for policy 0, policy_version 471150 (0.0010) [2023-12-27 08:54:15,703][20347] Updated weights for policy 0, policy_version 471160 (0.0010) [2023-12-27 08:54:15,954][20381] Updated weights for policy 1, policy_version 470778 (0.0010) [2023-12-27 08:54:16,011][20381] Updated weights for policy 1, policy_version 470788 (0.0010) [2023-12-27 08:54:16,069][20381] Updated weights for policy 1, policy_version 470798 (0.0010) [2023-12-27 08:54:16,338][20347] Updated weights for policy 0, policy_version 471170 (0.0010) [2023-12-27 08:54:16,390][20347] Updated weights for policy 0, policy_version 471180 (0.0010) [2023-12-27 08:54:16,449][20347] Updated weights for policy 0, policy_version 471190 (0.0010) [2023-12-27 08:54:16,504][20347] Updated weights for policy 0, policy_version 471200 (0.0010) [2023-12-27 08:54:16,765][20381] Updated weights for policy 1, policy_version 470808 (0.0009) [2023-12-27 08:54:16,812][20381] Updated weights for policy 1, policy_version 470818 (0.0007) [2023-12-27 08:54:16,864][20381] Updated weights for policy 1, policy_version 470828 (0.0008) [2023-12-27 08:54:17,190][20347] Updated weights for policy 0, policy_version 471210 (0.0005) [2023-12-27 08:54:17,243][20347] Updated weights for policy 0, policy_version 471220 (0.0005) [2023-12-27 08:54:17,304][20347] Updated weights for policy 0, policy_version 471230 (0.0005) [2023-12-27 08:54:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 17885.9, 300 sec: 19438.6). Total num frames: 241205248. Throughput: 0: 9231.6, 1: 9138.0. Samples: 241212396. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:17,363][19170] Avg episode reward: [(0, '863.670'), (1, '1000.000')] [2023-12-27 08:54:17,512][20381] Updated weights for policy 1, policy_version 470838 (0.0007) [2023-12-27 08:54:17,571][20381] Updated weights for policy 1, policy_version 470848 (0.0008) [2023-12-27 08:54:17,632][20381] Updated weights for policy 1, policy_version 470858 (0.0006) [2023-12-27 08:54:17,949][20347] Updated weights for policy 0, policy_version 471240 (0.0009) [2023-12-27 08:54:18,009][20347] Updated weights for policy 0, policy_version 471250 (0.0007) [2023-12-27 08:54:18,064][20347] Updated weights for policy 0, policy_version 471260 (0.0006) [2023-12-27 08:54:18,280][20381] Updated weights for policy 1, policy_version 470868 (0.0005) [2023-12-27 08:54:18,332][20381] Updated weights for policy 1, policy_version 470878 (0.0006) [2023-12-27 08:54:18,389][20381] Updated weights for policy 1, policy_version 470888 (0.0006) [2023-12-27 08:54:18,676][20347] Updated weights for policy 0, policy_version 471270 (0.0009) [2023-12-27 08:54:18,729][20347] Updated weights for policy 0, policy_version 471280 (0.0009) [2023-12-27 08:54:18,786][20347] Updated weights for policy 0, policy_version 471290 (0.0005) [2023-12-27 08:54:19,081][20381] Updated weights for policy 1, policy_version 470898 (0.0007) [2023-12-27 08:54:19,137][20381] Updated weights for policy 1, policy_version 470908 (0.0008) [2023-12-27 08:54:19,189][20381] Updated weights for policy 1, policy_version 470918 (0.0008) [2023-12-27 08:54:19,259][20381] Updated weights for policy 1, policy_version 470928 (0.0008) [2023-12-27 08:54:19,439][20347] Updated weights for policy 0, policy_version 471300 (0.0007) [2023-12-27 08:54:19,501][20347] Updated weights for policy 0, policy_version 471310 (0.0008) [2023-12-27 08:54:19,564][20347] Updated weights for policy 0, policy_version 471320 (0.0006) [2023-12-27 08:54:20,046][20381] Updated weights for policy 1, policy_version 470938 (0.0007) [2023-12-27 08:54:20,093][20381] Updated weights for policy 1, policy_version 470948 (0.0009) [2023-12-27 08:54:20,143][20381] Updated weights for policy 1, policy_version 470958 (0.0008) [2023-12-27 08:54:20,222][20347] Updated weights for policy 0, policy_version 471330 (0.0006) [2023-12-27 08:54:20,288][20347] Updated weights for policy 0, policy_version 471340 (0.0008) [2023-12-27 08:54:20,345][20347] Updated weights for policy 0, policy_version 471350 (0.0005) [2023-12-27 08:54:20,403][20347] Updated weights for policy 0, policy_version 471360 (0.0006) [2023-12-27 08:54:20,862][20381] Updated weights for policy 1, policy_version 470968 (0.0007) [2023-12-27 08:54:20,927][20381] Updated weights for policy 1, policy_version 470978 (0.0006) [2023-12-27 08:54:20,994][20381] Updated weights for policy 1, policy_version 470988 (0.0010) [2023-12-27 08:54:21,066][20347] Updated weights for policy 0, policy_version 471370 (0.0008) [2023-12-27 08:54:21,128][20347] Updated weights for policy 0, policy_version 471380 (0.0008) [2023-12-27 08:54:21,188][20347] Updated weights for policy 0, policy_version 471390 (0.0008) [2023-12-27 08:54:21,734][20381] Updated weights for policy 1, policy_version 470998 (0.0009) [2023-12-27 08:54:21,794][20381] Updated weights for policy 1, policy_version 471008 (0.0007) [2023-12-27 08:54:21,849][20381] Updated weights for policy 1, policy_version 471018 (0.0008) [2023-12-27 08:54:21,970][20347] Updated weights for policy 0, policy_version 471400 (0.0009) [2023-12-27 08:54:22,027][20347] Updated weights for policy 0, policy_version 471410 (0.0008) [2023-12-27 08:54:22,086][20347] Updated weights for policy 0, policy_version 471420 (0.0008) [2023-12-27 08:54:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 18158.9, 300 sec: 19410.9). Total num frames: 241303552. Throughput: 0: 9374.1, 1: 9224.2. Samples: 241273784. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:22,363][19170] Avg episode reward: [(0, '883.280'), (1, '1000.000')] [2023-12-27 08:54:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000471024_120602624.pth... [2023-12-27 08:54:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000471424_120700928.pth... [2023-12-27 08:54:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000470240_120397824.pth [2023-12-27 08:54:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000469872_120307712.pth [2023-12-27 08:54:22,553][20381] Updated weights for policy 1, policy_version 471028 (0.0008) [2023-12-27 08:54:22,607][20381] Updated weights for policy 1, policy_version 471038 (0.0008) [2023-12-27 08:54:22,657][20381] Updated weights for policy 1, policy_version 471048 (0.0008) [2023-12-27 08:54:22,788][20347] Updated weights for policy 0, policy_version 471430 (0.0008) [2023-12-27 08:54:22,849][20347] Updated weights for policy 0, policy_version 471440 (0.0008) [2023-12-27 08:54:22,883][20291] KL-divergence is very high: 103.8916 [2023-12-27 08:54:22,903][20347] Updated weights for policy 0, policy_version 471450 (0.0008) [2023-12-27 08:54:22,922][20291] KL-divergence is very high: 103.5399 [2023-12-27 08:54:23,368][20381] Updated weights for policy 1, policy_version 471058 (0.0007) [2023-12-27 08:54:23,428][20381] Updated weights for policy 1, policy_version 471068 (0.0007) [2023-12-27 08:54:23,485][20381] Updated weights for policy 1, policy_version 471078 (0.0008) [2023-12-27 08:54:23,538][20381] Updated weights for policy 1, policy_version 471088 (0.0005) [2023-12-27 08:54:23,638][20347] Updated weights for policy 0, policy_version 471460 (0.0008) [2023-12-27 08:54:23,696][20347] Updated weights for policy 0, policy_version 471470 (0.0007) [2023-12-27 08:54:23,745][20347] Updated weights for policy 0, policy_version 471480 (0.0008) [2023-12-27 08:54:24,171][20381] Updated weights for policy 1, policy_version 471098 (0.0007) [2023-12-27 08:54:24,230][20381] Updated weights for policy 1, policy_version 471108 (0.0005) [2023-12-27 08:54:24,278][20381] Updated weights for policy 1, policy_version 471118 (0.0005) [2023-12-27 08:54:24,495][20347] Updated weights for policy 0, policy_version 471490 (0.0008) [2023-12-27 08:54:24,538][20347] Updated weights for policy 0, policy_version 471500 (0.0008) [2023-12-27 08:54:24,593][20347] Updated weights for policy 0, policy_version 471510 (0.0008) [2023-12-27 08:54:24,648][20347] Updated weights for policy 0, policy_version 471520 (0.0008) [2023-12-27 08:54:24,847][20381] Updated weights for policy 1, policy_version 471128 (0.0005) [2023-12-27 08:54:24,892][20381] Updated weights for policy 1, policy_version 471138 (0.0006) [2023-12-27 08:54:24,953][20381] Updated weights for policy 1, policy_version 471148 (0.0007) [2023-12-27 08:54:25,304][20347] Updated weights for policy 0, policy_version 471530 (0.0005) [2023-12-27 08:54:25,360][20347] Updated weights for policy 0, policy_version 471540 (0.0009) [2023-12-27 08:54:25,412][20347] Updated weights for policy 0, policy_version 471550 (0.0010) [2023-12-27 08:54:25,589][20381] Updated weights for policy 1, policy_version 471158 (0.0008) [2023-12-27 08:54:25,646][20381] Updated weights for policy 1, policy_version 471168 (0.0007) [2023-12-27 08:54:25,701][20381] Updated weights for policy 1, policy_version 471178 (0.0007) [2023-12-27 08:54:26,008][20347] Updated weights for policy 0, policy_version 471560 (0.0007) [2023-12-27 08:54:26,058][20347] Updated weights for policy 0, policy_version 471570 (0.0008) [2023-12-27 08:54:26,108][20347] Updated weights for policy 0, policy_version 471580 (0.0007) [2023-12-27 08:54:26,362][20381] Updated weights for policy 1, policy_version 471188 (0.0006) [2023-12-27 08:54:26,417][20381] Updated weights for policy 1, policy_version 471198 (0.0007) [2023-12-27 08:54:26,476][20381] Updated weights for policy 1, policy_version 471208 (0.0005) [2023-12-27 08:54:26,770][20347] Updated weights for policy 0, policy_version 471590 (0.0008) [2023-12-27 08:54:26,823][20347] Updated weights for policy 0, policy_version 471600 (0.0010) [2023-12-27 08:54:26,878][20347] Updated weights for policy 0, policy_version 471610 (0.0005) [2023-12-27 08:54:27,079][20381] Updated weights for policy 1, policy_version 471218 (0.0006) [2023-12-27 08:54:27,120][20292] KL-divergence is very high: 133.7661 [2023-12-27 08:54:27,134][20381] Updated weights for policy 1, policy_version 471228 (0.0007) [2023-12-27 08:54:27,186][20381] Updated weights for policy 1, policy_version 471238 (0.0008) [2023-12-27 08:54:27,244][20381] Updated weights for policy 1, policy_version 471248 (0.0007) [2023-12-27 08:54:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 18568.5, 300 sec: 19438.6). Total num frames: 241410048. Throughput: 0: 9549.9, 1: 9459.8. Samples: 241396400. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:27,363][19170] Avg episode reward: [(0, '863.390'), (1, '990.380')] [2023-12-27 08:54:27,522][20347] Updated weights for policy 0, policy_version 471620 (0.0009) [2023-12-27 08:54:27,573][20347] Updated weights for policy 0, policy_version 471630 (0.0010) [2023-12-27 08:54:27,623][20347] Updated weights for policy 0, policy_version 471640 (0.0010) [2023-12-27 08:54:27,831][20381] Updated weights for policy 1, policy_version 471258 (0.0007) [2023-12-27 08:54:27,892][20381] Updated weights for policy 1, policy_version 471268 (0.0007) [2023-12-27 08:54:27,946][20381] Updated weights for policy 1, policy_version 471278 (0.0007) [2023-12-27 08:54:28,291][20347] Updated weights for policy 0, policy_version 471650 (0.0010) [2023-12-27 08:54:28,344][20347] Updated weights for policy 0, policy_version 471660 (0.0010) [2023-12-27 08:54:28,400][20347] Updated weights for policy 0, policy_version 471670 (0.0009) [2023-12-27 08:54:28,449][20347] Updated weights for policy 0, policy_version 471680 (0.0010) [2023-12-27 08:54:28,577][20381] Updated weights for policy 1, policy_version 471288 (0.0008) [2023-12-27 08:54:28,633][20381] Updated weights for policy 1, policy_version 471298 (0.0007) [2023-12-27 08:54:28,694][20381] Updated weights for policy 1, policy_version 471308 (0.0008) [2023-12-27 08:54:29,127][20347] Updated weights for policy 0, policy_version 471690 (0.0006) [2023-12-27 08:54:29,174][20347] Updated weights for policy 0, policy_version 471700 (0.0006) [2023-12-27 08:54:29,229][20347] Updated weights for policy 0, policy_version 471710 (0.0006) [2023-12-27 08:54:29,395][20381] Updated weights for policy 1, policy_version 471318 (0.0008) [2023-12-27 08:54:29,452][20381] Updated weights for policy 1, policy_version 471328 (0.0008) [2023-12-27 08:54:29,509][20381] Updated weights for policy 1, policy_version 471338 (0.0006) [2023-12-27 08:54:29,881][20347] Updated weights for policy 0, policy_version 471720 (0.0008) [2023-12-27 08:54:29,946][20347] Updated weights for policy 0, policy_version 471730 (0.0008) [2023-12-27 08:54:30,000][20347] Updated weights for policy 0, policy_version 471740 (0.0008) [2023-12-27 08:54:30,093][20381] Updated weights for policy 1, policy_version 471348 (0.0005) [2023-12-27 08:54:30,151][20381] Updated weights for policy 1, policy_version 471358 (0.0006) [2023-12-27 08:54:30,207][20381] Updated weights for policy 1, policy_version 471368 (0.0008) [2023-12-27 08:54:30,690][20347] Updated weights for policy 0, policy_version 471750 (0.0006) [2023-12-27 08:54:30,742][20347] Updated weights for policy 0, policy_version 471760 (0.0008) [2023-12-27 08:54:30,793][20347] Updated weights for policy 0, policy_version 471770 (0.0010) [2023-12-27 08:54:30,844][20381] Updated weights for policy 1, policy_version 471378 (0.0007) [2023-12-27 08:54:30,893][20381] Updated weights for policy 1, policy_version 471388 (0.0007) [2023-12-27 08:54:30,948][20381] Updated weights for policy 1, policy_version 471398 (0.0008) [2023-12-27 08:54:31,000][20381] Updated weights for policy 1, policy_version 471408 (0.0007) [2023-12-27 08:54:31,464][20347] Updated weights for policy 0, policy_version 471780 (0.0010) [2023-12-27 08:54:31,517][20347] Updated weights for policy 0, policy_version 471790 (0.0010) [2023-12-27 08:54:31,574][20347] Updated weights for policy 0, policy_version 471800 (0.0010) [2023-12-27 08:54:31,741][20381] Updated weights for policy 1, policy_version 471418 (0.0008) [2023-12-27 08:54:31,792][20381] Updated weights for policy 1, policy_version 471428 (0.0007) [2023-12-27 08:54:31,854][20381] Updated weights for policy 1, policy_version 471438 (0.0007) [2023-12-27 08:54:32,281][20347] Updated weights for policy 0, policy_version 471810 (0.0011) [2023-12-27 08:54:32,337][20347] Updated weights for policy 0, policy_version 471820 (0.0010) [2023-12-27 08:54:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 18841.6, 300 sec: 19383.1). Total num frames: 241508352. Throughput: 0: 9882.6, 1: 9813.6. Samples: 241522924. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:32,363][19170] Avg episode reward: [(0, '872.910'), (1, '990.380')] [2023-12-27 08:54:32,401][20347] Updated weights for policy 0, policy_version 471830 (0.0007) [2023-12-27 08:54:32,463][20347] Updated weights for policy 0, policy_version 471840 (0.0005) [2023-12-27 08:54:32,574][20381] Updated weights for policy 1, policy_version 471448 (0.0007) [2023-12-27 08:54:32,633][20381] Updated weights for policy 1, policy_version 471458 (0.0007) [2023-12-27 08:54:32,690][20381] Updated weights for policy 1, policy_version 471468 (0.0008) [2023-12-27 08:54:33,106][20347] Updated weights for policy 0, policy_version 471850 (0.0010) [2023-12-27 08:54:33,159][20347] Updated weights for policy 0, policy_version 471860 (0.0010) [2023-12-27 08:54:33,218][20347] Updated weights for policy 0, policy_version 471870 (0.0010) [2023-12-27 08:54:33,370][20381] Updated weights for policy 1, policy_version 471478 (0.0007) [2023-12-27 08:54:33,419][20381] Updated weights for policy 1, policy_version 471488 (0.0008) [2023-12-27 08:54:33,469][20381] Updated weights for policy 1, policy_version 471498 (0.0007) [2023-12-27 08:54:33,890][20347] Updated weights for policy 0, policy_version 471880 (0.0010) [2023-12-27 08:54:33,943][20347] Updated weights for policy 0, policy_version 471890 (0.0010) [2023-12-27 08:54:33,988][20347] Updated weights for policy 0, policy_version 471900 (0.0010) [2023-12-27 08:54:34,130][20381] Updated weights for policy 1, policy_version 471508 (0.0007) [2023-12-27 08:54:34,186][20381] Updated weights for policy 1, policy_version 471518 (0.0007) [2023-12-27 08:54:34,241][20381] Updated weights for policy 1, policy_version 471528 (0.0008) [2023-12-27 08:54:34,701][20347] Updated weights for policy 0, policy_version 471910 (0.0010) [2023-12-27 08:54:34,759][20347] Updated weights for policy 0, policy_version 471920 (0.0010) [2023-12-27 08:54:34,765][20291] KL-divergence is very high: 272.3126 [2023-12-27 08:54:34,807][20291] KL-divergence is very high: 348.3279 [2023-12-27 08:54:34,813][20347] Updated weights for policy 0, policy_version 471930 (0.0010) [2023-12-27 08:54:34,925][20381] Updated weights for policy 1, policy_version 471538 (0.0008) [2023-12-27 08:54:34,982][20381] Updated weights for policy 1, policy_version 471548 (0.0008) [2023-12-27 08:54:35,039][20381] Updated weights for policy 1, policy_version 471558 (0.0009) [2023-12-27 08:54:35,087][20381] Updated weights for policy 1, policy_version 471568 (0.0009) [2023-12-27 08:54:35,397][20347] Updated weights for policy 0, policy_version 471940 (0.0010) [2023-12-27 08:54:35,448][20347] Updated weights for policy 0, policy_version 471950 (0.0010) [2023-12-27 08:54:35,494][20347] Updated weights for policy 0, policy_version 471960 (0.0010) [2023-12-27 08:54:35,824][20381] Updated weights for policy 1, policy_version 471578 (0.0008) [2023-12-27 08:54:35,878][20381] Updated weights for policy 1, policy_version 471588 (0.0008) [2023-12-27 08:54:35,936][20381] Updated weights for policy 1, policy_version 471598 (0.0008) [2023-12-27 08:54:36,137][20347] Updated weights for policy 0, policy_version 471970 (0.0009) [2023-12-27 08:54:36,197][20347] Updated weights for policy 0, policy_version 471980 (0.0007) [2023-12-27 08:54:36,252][20347] Updated weights for policy 0, policy_version 471990 (0.0006) [2023-12-27 08:54:36,311][20347] Updated weights for policy 0, policy_version 472000 (0.0005) [2023-12-27 08:54:36,691][20381] Updated weights for policy 1, policy_version 471608 (0.0008) [2023-12-27 08:54:36,751][20381] Updated weights for policy 1, policy_version 471618 (0.0009) [2023-12-27 08:54:36,812][20381] Updated weights for policy 1, policy_version 471628 (0.0009) [2023-12-27 08:54:36,932][20347] Updated weights for policy 0, policy_version 472010 (0.0005) [2023-12-27 08:54:36,993][20347] Updated weights for policy 0, policy_version 472020 (0.0007) [2023-12-27 08:54:37,054][20347] Updated weights for policy 0, policy_version 472030 (0.0008) [2023-12-27 08:54:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 241614848. Throughput: 0: 10033.7, 1: 9956.2. Samples: 241584068. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:37,363][19170] Avg episode reward: [(0, '931.760'), (1, '1000.000')] [2023-12-27 08:54:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000471632_120758272.pth... [2023-12-27 08:54:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000472032_120856576.pth... [2023-12-27 08:54:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000470816_120545280.pth [2023-12-27 08:54:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000470448_120455168.pth [2023-12-27 08:54:37,580][20381] Updated weights for policy 1, policy_version 471638 (0.0008) [2023-12-27 08:54:37,642][20381] Updated weights for policy 1, policy_version 471648 (0.0007) [2023-12-27 08:54:37,644][20347] Updated weights for policy 0, policy_version 472040 (0.0005) [2023-12-27 08:54:37,701][20347] Updated weights for policy 0, policy_version 472050 (0.0006) [2023-12-27 08:54:37,705][20381] Updated weights for policy 1, policy_version 471658 (0.0008) [2023-12-27 08:54:37,756][20347] Updated weights for policy 0, policy_version 472060 (0.0006) [2023-12-27 08:54:38,398][20381] Updated weights for policy 1, policy_version 471668 (0.0007) [2023-12-27 08:54:38,415][20347] Updated weights for policy 0, policy_version 472070 (0.0006) [2023-12-27 08:54:38,453][20381] Updated weights for policy 1, policy_version 471678 (0.0007) [2023-12-27 08:54:38,482][20347] Updated weights for policy 0, policy_version 472080 (0.0006) [2023-12-27 08:54:38,514][20381] Updated weights for policy 1, policy_version 471688 (0.0007) [2023-12-27 08:54:38,553][20347] Updated weights for policy 0, policy_version 472090 (0.0005) [2023-12-27 08:54:39,069][20347] Updated weights for policy 0, policy_version 472100 (0.0006) [2023-12-27 08:54:39,128][20347] Updated weights for policy 0, policy_version 472110 (0.0006) [2023-12-27 08:54:39,157][20381] Updated weights for policy 1, policy_version 471698 (0.0008) [2023-12-27 08:54:39,190][20347] Updated weights for policy 0, policy_version 472120 (0.0005) [2023-12-27 08:54:39,214][20381] Updated weights for policy 1, policy_version 471708 (0.0007) [2023-12-27 08:54:39,227][20292] KL-divergence is very high: 154.5612 [2023-12-27 08:54:39,278][20381] Updated weights for policy 1, policy_version 471718 (0.0007) [2023-12-27 08:54:39,332][20292] KL-divergence is very high: 130.6204 [2023-12-27 08:54:39,344][20381] Updated weights for policy 1, policy_version 471728 (0.0007) [2023-12-27 08:54:39,838][20347] Updated weights for policy 0, policy_version 472130 (0.0007) [2023-12-27 08:54:39,894][20347] Updated weights for policy 0, policy_version 472140 (0.0007) [2023-12-27 08:54:39,954][20347] Updated weights for policy 0, policy_version 472150 (0.0007) [2023-12-27 08:54:40,012][20347] Updated weights for policy 0, policy_version 472160 (0.0006) [2023-12-27 08:54:40,082][20381] Updated weights for policy 1, policy_version 471738 (0.0008) [2023-12-27 08:54:40,139][20381] Updated weights for policy 1, policy_version 471748 (0.0008) [2023-12-27 08:54:40,199][20381] Updated weights for policy 1, policy_version 471758 (0.0008) [2023-12-27 08:54:40,687][20347] Updated weights for policy 0, policy_version 472170 (0.0007) [2023-12-27 08:54:40,744][20347] Updated weights for policy 0, policy_version 472180 (0.0006) [2023-12-27 08:54:40,805][20347] Updated weights for policy 0, policy_version 472190 (0.0008) [2023-12-27 08:54:40,822][20381] Updated weights for policy 1, policy_version 471768 (0.0006) [2023-12-27 08:54:40,872][20381] Updated weights for policy 1, policy_version 471778 (0.0005) [2023-12-27 08:54:40,929][20381] Updated weights for policy 1, policy_version 471788 (0.0006) [2023-12-27 08:54:41,514][20347] Updated weights for policy 0, policy_version 472200 (0.0010) [2023-12-27 08:54:41,574][20347] Updated weights for policy 0, policy_version 472210 (0.0010) [2023-12-27 08:54:41,622][20381] Updated weights for policy 1, policy_version 471798 (0.0007) [2023-12-27 08:54:41,640][20347] Updated weights for policy 0, policy_version 472220 (0.0010) [2023-12-27 08:54:41,683][20381] Updated weights for policy 1, policy_version 471808 (0.0007) [2023-12-27 08:54:41,753][20381] Updated weights for policy 1, policy_version 471818 (0.0008) [2023-12-27 08:54:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 241713152. Throughput: 0: 10358.8, 1: 10163.7. Samples: 241708860. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.460')] [2023-12-27 08:54:42,379][20347] Updated weights for policy 0, policy_version 472230 (0.0009) [2023-12-27 08:54:42,433][20347] Updated weights for policy 0, policy_version 472240 (0.0007) [2023-12-27 08:54:42,434][20381] Updated weights for policy 1, policy_version 471828 (0.0008) [2023-12-27 08:54:42,486][20347] Updated weights for policy 0, policy_version 472250 (0.0008) [2023-12-27 08:54:42,491][20381] Updated weights for policy 1, policy_version 471838 (0.0008) [2023-12-27 08:54:42,550][20381] Updated weights for policy 1, policy_version 471848 (0.0009) [2023-12-27 08:54:43,192][20381] Updated weights for policy 1, policy_version 471858 (0.0011) [2023-12-27 08:54:43,208][20347] Updated weights for policy 0, policy_version 472260 (0.0007) [2023-12-27 08:54:43,250][20381] Updated weights for policy 1, policy_version 471868 (0.0011) [2023-12-27 08:54:43,263][20347] Updated weights for policy 0, policy_version 472270 (0.0006) [2023-12-27 08:54:43,302][20381] Updated weights for policy 1, policy_version 471878 (0.0010) [2023-12-27 08:54:43,314][20347] Updated weights for policy 0, policy_version 472280 (0.0005) [2023-12-27 08:54:43,355][20381] Updated weights for policy 1, policy_version 471888 (0.0010) [2023-12-27 08:54:43,868][20347] Updated weights for policy 0, policy_version 472290 (0.0005) [2023-12-27 08:54:43,918][20347] Updated weights for policy 0, policy_version 472300 (0.0008) [2023-12-27 08:54:43,972][20347] Updated weights for policy 0, policy_version 472310 (0.0005) [2023-12-27 08:54:44,021][20381] Updated weights for policy 1, policy_version 471898 (0.0006) [2023-12-27 08:54:44,034][20347] Updated weights for policy 0, policy_version 472320 (0.0005) [2023-12-27 08:54:44,071][20381] Updated weights for policy 1, policy_version 471908 (0.0006) [2023-12-27 08:54:44,119][20381] Updated weights for policy 1, policy_version 471918 (0.0007) [2023-12-27 08:54:44,632][20347] Updated weights for policy 0, policy_version 472330 (0.0010) [2023-12-27 08:54:44,680][20347] Updated weights for policy 0, policy_version 472340 (0.0010) [2023-12-27 08:54:44,728][20347] Updated weights for policy 0, policy_version 472350 (0.0009) [2023-12-27 08:54:44,841][20381] Updated weights for policy 1, policy_version 471928 (0.0008) [2023-12-27 08:54:44,902][20381] Updated weights for policy 1, policy_version 471938 (0.0007) [2023-12-27 08:54:44,951][20381] Updated weights for policy 1, policy_version 471948 (0.0007) [2023-12-27 08:54:45,445][20347] Updated weights for policy 0, policy_version 472360 (0.0009) [2023-12-27 08:54:45,500][20347] Updated weights for policy 0, policy_version 472370 (0.0010) [2023-12-27 08:54:45,556][20347] Updated weights for policy 0, policy_version 472380 (0.0010) [2023-12-27 08:54:45,648][20381] Updated weights for policy 1, policy_version 471958 (0.0007) [2023-12-27 08:54:45,712][20381] Updated weights for policy 1, policy_version 471968 (0.0007) [2023-12-27 08:54:45,773][20381] Updated weights for policy 1, policy_version 471978 (0.0006) [2023-12-27 08:54:46,228][20347] Updated weights for policy 0, policy_version 472390 (0.0007) [2023-12-27 08:54:46,288][20347] Updated weights for policy 0, policy_version 472400 (0.0005) [2023-12-27 08:54:46,344][20347] Updated weights for policy 0, policy_version 472410 (0.0005) [2023-12-27 08:54:46,431][20381] Updated weights for policy 1, policy_version 471988 (0.0006) [2023-12-27 08:54:46,492][20381] Updated weights for policy 1, policy_version 471998 (0.0007) [2023-12-27 08:54:46,548][20381] Updated weights for policy 1, policy_version 472008 (0.0007) [2023-12-27 08:54:46,964][20347] Updated weights for policy 0, policy_version 472420 (0.0007) [2023-12-27 08:54:47,012][20347] Updated weights for policy 0, policy_version 472430 (0.0010) [2023-12-27 08:54:47,064][20347] Updated weights for policy 0, policy_version 472440 (0.0010) [2023-12-27 08:54:47,142][20381] Updated weights for policy 1, policy_version 472018 (0.0006) [2023-12-27 08:54:47,200][20381] Updated weights for policy 1, policy_version 472028 (0.0008) [2023-12-27 08:54:47,262][20381] Updated weights for policy 1, policy_version 472038 (0.0007) [2023-12-27 08:54:47,310][20381] Updated weights for policy 1, policy_version 472048 (0.0008) [2023-12-27 08:54:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20070.4, 300 sec: 19438.6). Total num frames: 241827840. Throughput: 0: 10388.9, 1: 10210.6. Samples: 241833464. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.460')] [2023-12-27 08:54:47,759][20347] Updated weights for policy 0, policy_version 472450 (0.0010) [2023-12-27 08:54:47,806][20347] Updated weights for policy 0, policy_version 472460 (0.0010) [2023-12-27 08:54:47,855][20347] Updated weights for policy 0, policy_version 472470 (0.0005) [2023-12-27 08:54:47,911][20347] Updated weights for policy 0, policy_version 472480 (0.0005) [2023-12-27 08:54:47,937][20381] Updated weights for policy 1, policy_version 472058 (0.0007) [2023-12-27 08:54:47,984][20381] Updated weights for policy 1, policy_version 472068 (0.0007) [2023-12-27 08:54:48,035][20381] Updated weights for policy 1, policy_version 472078 (0.0005) [2023-12-27 08:54:48,532][20347] Updated weights for policy 0, policy_version 472490 (0.0010) [2023-12-27 08:54:48,590][20347] Updated weights for policy 0, policy_version 472500 (0.0010) [2023-12-27 08:54:48,649][20347] Updated weights for policy 0, policy_version 472510 (0.0011) [2023-12-27 08:54:48,738][20381] Updated weights for policy 1, policy_version 472088 (0.0007) [2023-12-27 08:54:48,794][20381] Updated weights for policy 1, policy_version 472098 (0.0007) [2023-12-27 08:54:48,848][20381] Updated weights for policy 1, policy_version 472108 (0.0005) [2023-12-27 08:54:49,341][20347] Updated weights for policy 0, policy_version 472520 (0.0010) [2023-12-27 08:54:49,406][20347] Updated weights for policy 0, policy_version 472530 (0.0011) [2023-12-27 08:54:49,465][20347] Updated weights for policy 0, policy_version 472540 (0.0010) [2023-12-27 08:54:49,526][20381] Updated weights for policy 1, policy_version 472118 (0.0007) [2023-12-27 08:54:49,581][20381] Updated weights for policy 1, policy_version 472128 (0.0007) [2023-12-27 08:54:49,643][20381] Updated weights for policy 1, policy_version 472138 (0.0008) [2023-12-27 08:54:50,168][20347] Updated weights for policy 0, policy_version 472550 (0.0008) [2023-12-27 08:54:50,225][20347] Updated weights for policy 0, policy_version 472560 (0.0007) [2023-12-27 08:54:50,281][20347] Updated weights for policy 0, policy_version 472570 (0.0008) [2023-12-27 08:54:50,362][20381] Updated weights for policy 1, policy_version 472148 (0.0009) [2023-12-27 08:54:50,411][20381] Updated weights for policy 1, policy_version 472158 (0.0010) [2023-12-27 08:54:50,466][20381] Updated weights for policy 1, policy_version 472168 (0.0009) [2023-12-27 08:54:50,998][20347] Updated weights for policy 0, policy_version 472580 (0.0009) [2023-12-27 08:54:51,060][20347] Updated weights for policy 0, policy_version 472590 (0.0007) [2023-12-27 08:54:51,122][20347] Updated weights for policy 0, policy_version 472600 (0.0010) [2023-12-27 08:54:51,159][20381] Updated weights for policy 1, policy_version 472178 (0.0011) [2023-12-27 08:54:51,213][20381] Updated weights for policy 1, policy_version 472188 (0.0011) [2023-12-27 08:54:51,274][20381] Updated weights for policy 1, policy_version 472198 (0.0009) [2023-12-27 08:54:51,334][20381] Updated weights for policy 1, policy_version 472208 (0.0007) [2023-12-27 08:54:51,772][20347] Updated weights for policy 0, policy_version 472610 (0.0007) [2023-12-27 08:54:51,837][20347] Updated weights for policy 0, policy_version 472620 (0.0006) [2023-12-27 08:54:51,897][20347] Updated weights for policy 0, policy_version 472630 (0.0006) [2023-12-27 08:54:51,962][20347] Updated weights for policy 0, policy_version 472640 (0.0007) [2023-12-27 08:54:52,045][20381] Updated weights for policy 1, policy_version 472218 (0.0009) [2023-12-27 08:54:52,107][20381] Updated weights for policy 1, policy_version 472228 (0.0010) [2023-12-27 08:54:52,172][20381] Updated weights for policy 1, policy_version 472238 (0.0011) [2023-12-27 08:54:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 19383.1). Total num frames: 241926144. Throughput: 0: 10397.2, 1: 10195.3. Samples: 241895152. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:52,363][19170] Avg episode reward: [(0, '921.640'), (1, '1000.000')] [2023-12-27 08:54:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000472240_120913920.pth... [2023-12-27 08:54:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000472640_121012224.pth... [2023-12-27 08:54:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000471024_120602624.pth [2023-12-27 08:54:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000471424_120700928.pth [2023-12-27 08:54:52,614][20347] Updated weights for policy 0, policy_version 472650 (0.0006) [2023-12-27 08:54:52,672][20347] Updated weights for policy 0, policy_version 472660 (0.0006) [2023-12-27 08:54:52,728][20347] Updated weights for policy 0, policy_version 472670 (0.0007) [2023-12-27 08:54:52,837][20381] Updated weights for policy 1, policy_version 472248 (0.0006) [2023-12-27 08:54:52,888][20381] Updated weights for policy 1, policy_version 472258 (0.0005) [2023-12-27 08:54:52,949][20381] Updated weights for policy 1, policy_version 472268 (0.0009) [2023-12-27 08:54:53,329][20347] Updated weights for policy 0, policy_version 472680 (0.0006) [2023-12-27 08:54:53,387][20347] Updated weights for policy 0, policy_version 472690 (0.0007) [2023-12-27 08:54:53,434][20347] Updated weights for policy 0, policy_version 472700 (0.0007) [2023-12-27 08:54:53,607][20381] Updated weights for policy 1, policy_version 472278 (0.0010) [2023-12-27 08:54:53,656][20381] Updated weights for policy 1, policy_version 472288 (0.0010) [2023-12-27 08:54:53,710][20381] Updated weights for policy 1, policy_version 472298 (0.0008) [2023-12-27 08:54:54,087][20347] Updated weights for policy 0, policy_version 472710 (0.0008) [2023-12-27 08:54:54,150][20347] Updated weights for policy 0, policy_version 472720 (0.0007) [2023-12-27 08:54:54,199][20347] Updated weights for policy 0, policy_version 472730 (0.0010) [2023-12-27 08:54:54,332][20381] Updated weights for policy 1, policy_version 472308 (0.0007) [2023-12-27 08:54:54,387][20381] Updated weights for policy 1, policy_version 472318 (0.0010) [2023-12-27 08:54:54,442][20381] Updated weights for policy 1, policy_version 472328 (0.0010) [2023-12-27 08:54:54,858][20347] Updated weights for policy 0, policy_version 472740 (0.0010) [2023-12-27 08:54:54,903][20347] Updated weights for policy 0, policy_version 472750 (0.0010) [2023-12-27 08:54:54,967][20347] Updated weights for policy 0, policy_version 472760 (0.0010) [2023-12-27 08:54:55,126][20381] Updated weights for policy 1, policy_version 472338 (0.0010) [2023-12-27 08:54:55,173][20381] Updated weights for policy 1, policy_version 472348 (0.0005) [2023-12-27 08:54:55,226][20381] Updated weights for policy 1, policy_version 472358 (0.0005) [2023-12-27 08:54:55,274][20381] Updated weights for policy 1, policy_version 472368 (0.0005) [2023-12-27 08:54:55,607][20347] Updated weights for policy 0, policy_version 472770 (0.0006) [2023-12-27 08:54:55,664][20347] Updated weights for policy 0, policy_version 472780 (0.0007) [2023-12-27 08:54:55,722][20347] Updated weights for policy 0, policy_version 472790 (0.0006) [2023-12-27 08:54:55,781][20347] Updated weights for policy 0, policy_version 472800 (0.0007) [2023-12-27 08:54:55,880][20381] Updated weights for policy 1, policy_version 472378 (0.0008) [2023-12-27 08:54:55,928][20381] Updated weights for policy 1, policy_version 472388 (0.0007) [2023-12-27 08:54:55,985][20381] Updated weights for policy 1, policy_version 472398 (0.0005) [2023-12-27 08:54:56,456][20347] Updated weights for policy 0, policy_version 472810 (0.0005) [2023-12-27 08:54:56,501][20347] Updated weights for policy 0, policy_version 472820 (0.0006) [2023-12-27 08:54:56,558][20347] Updated weights for policy 0, policy_version 472830 (0.0008) [2023-12-27 08:54:56,586][20381] Updated weights for policy 1, policy_version 472408 (0.0007) [2023-12-27 08:54:56,643][20381] Updated weights for policy 1, policy_version 472418 (0.0008) [2023-12-27 08:54:56,690][20381] Updated weights for policy 1, policy_version 472428 (0.0008) [2023-12-27 08:54:57,113][20347] Updated weights for policy 0, policy_version 472840 (0.0007) [2023-12-27 08:54:57,157][20347] Updated weights for policy 0, policy_version 472850 (0.0010) [2023-12-27 08:54:57,204][20347] Updated weights for policy 0, policy_version 472860 (0.0007) [2023-12-27 08:54:57,284][20381] Updated weights for policy 1, policy_version 472438 (0.0007) [2023-12-27 08:54:57,342][20381] Updated weights for policy 1, policy_version 472448 (0.0006) [2023-12-27 08:54:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 19410.9). Total num frames: 242032640. Throughput: 0: 10450.7, 1: 10268.5. Samples: 242020936. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:54:57,363][19170] Avg episode reward: [(0, '781.520'), (1, '1000.000')] [2023-12-27 08:54:57,400][20381] Updated weights for policy 1, policy_version 472458 (0.0005) [2023-12-27 08:54:57,902][20381] Updated weights for policy 1, policy_version 472468 (0.0006) [2023-12-27 08:54:57,937][20347] Updated weights for policy 0, policy_version 472870 (0.0008) [2023-12-27 08:54:57,962][20381] Updated weights for policy 1, policy_version 472478 (0.0007) [2023-12-27 08:54:57,995][20347] Updated weights for policy 0, policy_version 472880 (0.0006) [2023-12-27 08:54:58,018][20381] Updated weights for policy 1, policy_version 472488 (0.0007) [2023-12-27 08:54:58,043][20347] Updated weights for policy 0, policy_version 472890 (0.0005) [2023-12-27 08:54:58,751][20381] Updated weights for policy 1, policy_version 472498 (0.0007) [2023-12-27 08:54:58,796][20347] Updated weights for policy 0, policy_version 472900 (0.0007) [2023-12-27 08:54:58,817][20381] Updated weights for policy 1, policy_version 472508 (0.0006) [2023-12-27 08:54:58,859][20347] Updated weights for policy 0, policy_version 472910 (0.0009) [2023-12-27 08:54:58,880][20381] Updated weights for policy 1, policy_version 472518 (0.0008) [2023-12-27 08:54:58,921][20347] Updated weights for policy 0, policy_version 472920 (0.0007) [2023-12-27 08:54:58,942][20381] Updated weights for policy 1, policy_version 472528 (0.0006) [2023-12-27 08:54:59,527][20381] Updated weights for policy 1, policy_version 472538 (0.0005) [2023-12-27 08:54:59,558][20347] Updated weights for policy 0, policy_version 472930 (0.0006) [2023-12-27 08:54:59,592][20381] Updated weights for policy 1, policy_version 472548 (0.0005) [2023-12-27 08:54:59,622][20347] Updated weights for policy 0, policy_version 472940 (0.0008) [2023-12-27 08:54:59,644][20381] Updated weights for policy 1, policy_version 472558 (0.0007) [2023-12-27 08:54:59,677][20347] Updated weights for policy 0, policy_version 472950 (0.0006) [2023-12-27 08:54:59,726][20347] Updated weights for policy 0, policy_version 472960 (0.0006) [2023-12-27 08:55:00,312][20381] Updated weights for policy 1, policy_version 472568 (0.0009) [2023-12-27 08:55:00,343][20347] Updated weights for policy 0, policy_version 472970 (0.0005) [2023-12-27 08:55:00,367][20381] Updated weights for policy 1, policy_version 472578 (0.0010) [2023-12-27 08:55:00,398][20347] Updated weights for policy 0, policy_version 472980 (0.0005) [2023-12-27 08:55:00,415][20381] Updated weights for policy 1, policy_version 472588 (0.0010) [2023-12-27 08:55:00,449][20347] Updated weights for policy 0, policy_version 472990 (0.0006) [2023-12-27 08:55:01,075][20347] Updated weights for policy 0, policy_version 473000 (0.0008) [2023-12-27 08:55:01,087][20381] Updated weights for policy 1, policy_version 472598 (0.0007) [2023-12-27 08:55:01,136][20347] Updated weights for policy 0, policy_version 473010 (0.0006) [2023-12-27 08:55:01,149][20381] Updated weights for policy 1, policy_version 472608 (0.0007) [2023-12-27 08:55:01,193][20347] Updated weights for policy 0, policy_version 473020 (0.0006) [2023-12-27 08:55:01,202][20381] Updated weights for policy 1, policy_version 472618 (0.0006) [2023-12-27 08:55:01,910][20347] Updated weights for policy 0, policy_version 473030 (0.0007) [2023-12-27 08:55:01,933][20381] Updated weights for policy 1, policy_version 472628 (0.0007) [2023-12-27 08:55:01,968][20347] Updated weights for policy 0, policy_version 473040 (0.0005) [2023-12-27 08:55:01,992][20381] Updated weights for policy 1, policy_version 472638 (0.0007) [2023-12-27 08:55:02,019][20347] Updated weights for policy 0, policy_version 473050 (0.0005) [2023-12-27 08:55:02,049][20381] Updated weights for policy 1, policy_version 472648 (0.0007) [2023-12-27 08:55:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 19410.9). Total num frames: 242139136. Throughput: 0: 10440.9, 1: 10346.8. Samples: 242147844. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:55:02,363][19170] Avg episode reward: [(0, '816.480'), (1, '1000.000')] [2023-12-27 08:55:02,620][20347] Updated weights for policy 0, policy_version 473060 (0.0008) [2023-12-27 08:55:02,683][20347] Updated weights for policy 0, policy_version 473070 (0.0007) [2023-12-27 08:55:02,747][20347] Updated weights for policy 0, policy_version 473080 (0.0007) [2023-12-27 08:55:02,764][20381] Updated weights for policy 1, policy_version 472658 (0.0007) [2023-12-27 08:55:02,823][20381] Updated weights for policy 1, policy_version 472668 (0.0005) [2023-12-27 08:55:02,872][20381] Updated weights for policy 1, policy_version 472678 (0.0005) [2023-12-27 08:55:02,928][20381] Updated weights for policy 1, policy_version 472688 (0.0009) [2023-12-27 08:55:03,340][20347] Updated weights for policy 0, policy_version 473090 (0.0006) [2023-12-27 08:55:03,386][20347] Updated weights for policy 0, policy_version 473100 (0.0005) [2023-12-27 08:55:03,432][20347] Updated weights for policy 0, policy_version 473110 (0.0005) [2023-12-27 08:55:03,484][20347] Updated weights for policy 0, policy_version 473120 (0.0005) [2023-12-27 08:55:03,487][20381] Updated weights for policy 1, policy_version 472698 (0.0005) [2023-12-27 08:55:03,532][20381] Updated weights for policy 1, policy_version 472708 (0.0006) [2023-12-27 08:55:03,578][20381] Updated weights for policy 1, policy_version 472718 (0.0010) [2023-12-27 08:55:04,096][20347] Updated weights for policy 0, policy_version 473130 (0.0007) [2023-12-27 08:55:04,164][20347] Updated weights for policy 0, policy_version 473140 (0.0008) [2023-12-27 08:55:04,228][20347] Updated weights for policy 0, policy_version 473150 (0.0007) [2023-12-27 08:55:04,252][20381] Updated weights for policy 1, policy_version 472728 (0.0011) [2023-12-27 08:55:04,309][20381] Updated weights for policy 1, policy_version 472738 (0.0010) [2023-12-27 08:55:04,369][20381] Updated weights for policy 1, policy_version 472748 (0.0011) [2023-12-27 08:55:04,880][20347] Updated weights for policy 0, policy_version 473160 (0.0006) [2023-12-27 08:55:04,932][20347] Updated weights for policy 0, policy_version 473170 (0.0006) [2023-12-27 08:55:04,990][20381] Updated weights for policy 1, policy_version 472758 (0.0009) [2023-12-27 08:55:04,991][20347] Updated weights for policy 0, policy_version 473180 (0.0007) [2023-12-27 08:55:05,048][20381] Updated weights for policy 1, policy_version 472768 (0.0009) [2023-12-27 08:55:05,106][20381] Updated weights for policy 1, policy_version 472778 (0.0005) [2023-12-27 08:55:05,642][20381] Updated weights for policy 1, policy_version 472788 (0.0005) [2023-12-27 08:55:05,701][20381] Updated weights for policy 1, policy_version 472798 (0.0006) [2023-12-27 08:55:05,702][20347] Updated weights for policy 0, policy_version 473190 (0.0009) [2023-12-27 08:55:05,759][20381] Updated weights for policy 1, policy_version 472808 (0.0005) [2023-12-27 08:55:05,763][20347] Updated weights for policy 0, policy_version 473200 (0.0010) [2023-12-27 08:55:05,824][20347] Updated weights for policy 0, policy_version 473210 (0.0007) [2023-12-27 08:55:06,364][20381] Updated weights for policy 1, policy_version 472818 (0.0006) [2023-12-27 08:55:06,418][20381] Updated weights for policy 1, policy_version 472828 (0.0011) [2023-12-27 08:55:06,454][20347] Updated weights for policy 0, policy_version 473220 (0.0007) [2023-12-27 08:55:06,476][20381] Updated weights for policy 1, policy_version 472838 (0.0010) [2023-12-27 08:55:06,514][20347] Updated weights for policy 0, policy_version 473230 (0.0011) [2023-12-27 08:55:06,537][20381] Updated weights for policy 1, policy_version 472848 (0.0011) [2023-12-27 08:55:06,573][20347] Updated weights for policy 0, policy_version 473240 (0.0010) [2023-12-27 08:55:07,170][20381] Updated weights for policy 1, policy_version 472858 (0.0007) [2023-12-27 08:55:07,232][20381] Updated weights for policy 1, policy_version 472868 (0.0006) [2023-12-27 08:55:07,249][20347] Updated weights for policy 0, policy_version 473250 (0.0010) [2023-12-27 08:55:07,288][20381] Updated weights for policy 1, policy_version 472878 (0.0005) [2023-12-27 08:55:07,298][20347] Updated weights for policy 0, policy_version 473260 (0.0010) [2023-12-27 08:55:07,346][20347] Updated weights for policy 0, policy_version 473270 (0.0010) [2023-12-27 08:55:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 19438.6). Total num frames: 242245632. Throughput: 0: 10469.6, 1: 10421.0. Samples: 242213860. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:55:07,363][19170] Avg episode reward: [(0, '928.260'), (1, '1000.000')] [2023-12-27 08:55:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000472880_121077760.pth... [2023-12-27 08:55:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000471632_120758272.pth [2023-12-27 08:55:07,398][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000473280_121176064.pth... [2023-12-27 08:55:07,401][20347] Updated weights for policy 0, policy_version 473280 (0.0009) [2023-12-27 08:55:07,402][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000472032_120856576.pth [2023-12-27 08:55:07,920][20381] Updated weights for policy 1, policy_version 472888 (0.0006) [2023-12-27 08:55:07,976][20381] Updated weights for policy 1, policy_version 472898 (0.0006) [2023-12-27 08:55:08,026][20347] Updated weights for policy 0, policy_version 473290 (0.0006) [2023-12-27 08:55:08,030][20381] Updated weights for policy 1, policy_version 472908 (0.0008) [2023-12-27 08:55:08,084][20347] Updated weights for policy 0, policy_version 473300 (0.0006) [2023-12-27 08:55:08,146][20347] Updated weights for policy 0, policy_version 473310 (0.0006) [2023-12-27 08:55:08,714][20381] Updated weights for policy 1, policy_version 472918 (0.0008) [2023-12-27 08:55:08,766][20381] Updated weights for policy 1, policy_version 472928 (0.0007) [2023-12-27 08:55:08,781][20347] Updated weights for policy 0, policy_version 473320 (0.0007) [2023-12-27 08:55:08,817][20381] Updated weights for policy 1, policy_version 472938 (0.0008) [2023-12-27 08:55:08,839][20347] Updated weights for policy 0, policy_version 473330 (0.0007) [2023-12-27 08:55:08,908][20347] Updated weights for policy 0, policy_version 473340 (0.0007) [2023-12-27 08:55:09,535][20347] Updated weights for policy 0, policy_version 473350 (0.0008) [2023-12-27 08:55:09,590][20381] Updated weights for policy 1, policy_version 472948 (0.0007) [2023-12-27 08:55:09,597][20347] Updated weights for policy 0, policy_version 473360 (0.0006) [2023-12-27 08:55:09,636][20381] Updated weights for policy 1, policy_version 472958 (0.0007) [2023-12-27 08:55:09,656][20347] Updated weights for policy 0, policy_version 473370 (0.0007) [2023-12-27 08:55:09,696][20381] Updated weights for policy 1, policy_version 472968 (0.0007) [2023-12-27 08:55:10,370][20347] Updated weights for policy 0, policy_version 473380 (0.0005) [2023-12-27 08:55:10,418][20381] Updated weights for policy 1, policy_version 472978 (0.0008) [2023-12-27 08:55:10,433][20347] Updated weights for policy 0, policy_version 473390 (0.0008) [2023-12-27 08:55:10,479][20381] Updated weights for policy 1, policy_version 472988 (0.0008) [2023-12-27 08:55:10,495][20347] Updated weights for policy 0, policy_version 473400 (0.0006) [2023-12-27 08:55:10,546][20381] Updated weights for policy 1, policy_version 472998 (0.0008) [2023-12-27 08:55:10,617][20381] Updated weights for policy 1, policy_version 473008 (0.0007) [2023-12-27 08:55:11,118][20347] Updated weights for policy 0, policy_version 473410 (0.0008) [2023-12-27 08:55:11,188][20347] Updated weights for policy 0, policy_version 473420 (0.0010) [2023-12-27 08:55:11,248][20347] Updated weights for policy 0, policy_version 473430 (0.0010) [2023-12-27 08:55:11,306][20347] Updated weights for policy 0, policy_version 473440 (0.0008) [2023-12-27 08:55:11,311][20381] Updated weights for policy 1, policy_version 473018 (0.0006) [2023-12-27 08:55:11,383][20381] Updated weights for policy 1, policy_version 473028 (0.0008) [2023-12-27 08:55:11,445][20381] Updated weights for policy 1, policy_version 473038 (0.0008) [2023-12-27 08:55:12,047][20347] Updated weights for policy 0, policy_version 473450 (0.0006) [2023-12-27 08:55:12,091][20381] Updated weights for policy 1, policy_version 473048 (0.0007) [2023-12-27 08:55:12,112][20347] Updated weights for policy 0, policy_version 473460 (0.0006) [2023-12-27 08:55:12,156][20381] Updated weights for policy 1, policy_version 473058 (0.0006) [2023-12-27 08:55:12,180][20347] Updated weights for policy 0, policy_version 473470 (0.0006) [2023-12-27 08:55:12,217][20381] Updated weights for policy 1, policy_version 473068 (0.0006) [2023-12-27 08:55:12,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 19466.4). Total num frames: 242352128. Throughput: 0: 10534.5, 1: 10411.1. Samples: 242338952. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:55:12,363][19170] Avg episode reward: [(0, '971.220'), (1, '1000.000')] [2023-12-27 08:55:12,827][20347] Updated weights for policy 0, policy_version 473480 (0.0007) [2023-12-27 08:55:12,879][20347] Updated weights for policy 0, policy_version 473490 (0.0007) [2023-12-27 08:55:12,900][20381] Updated weights for policy 1, policy_version 473078 (0.0006) [2023-12-27 08:55:12,929][20347] Updated weights for policy 0, policy_version 473500 (0.0006) [2023-12-27 08:55:12,955][20381] Updated weights for policy 1, policy_version 473088 (0.0007) [2023-12-27 08:55:13,014][20381] Updated weights for policy 1, policy_version 473098 (0.0005) [2023-12-27 08:55:13,548][20347] Updated weights for policy 0, policy_version 473510 (0.0006) [2023-12-27 08:55:13,550][20381] Updated weights for policy 1, policy_version 473108 (0.0005) [2023-12-27 08:55:13,601][20347] Updated weights for policy 0, policy_version 473520 (0.0006) [2023-12-27 08:55:13,603][20381] Updated weights for policy 1, policy_version 473118 (0.0006) [2023-12-27 08:55:13,656][20347] Updated weights for policy 0, policy_version 473530 (0.0007) [2023-12-27 08:55:13,658][20381] Updated weights for policy 1, policy_version 473128 (0.0005) [2023-12-27 08:55:14,264][20347] Updated weights for policy 0, policy_version 473540 (0.0005) [2023-12-27 08:55:14,315][20347] Updated weights for policy 0, policy_version 473550 (0.0008) [2023-12-27 08:55:14,346][20381] Updated weights for policy 1, policy_version 473138 (0.0006) [2023-12-27 08:55:14,374][20347] Updated weights for policy 0, policy_version 473560 (0.0006) [2023-12-27 08:55:14,402][20381] Updated weights for policy 1, policy_version 473148 (0.0006) [2023-12-27 08:55:14,455][20381] Updated weights for policy 1, policy_version 473158 (0.0007) [2023-12-27 08:55:14,504][20381] Updated weights for policy 1, policy_version 473168 (0.0007) [2023-12-27 08:55:15,061][20347] Updated weights for policy 0, policy_version 473570 (0.0006) [2023-12-27 08:55:15,117][20347] Updated weights for policy 0, policy_version 473580 (0.0008) [2023-12-27 08:55:15,173][20347] Updated weights for policy 0, policy_version 473590 (0.0007) [2023-12-27 08:55:15,205][20381] Updated weights for policy 1, policy_version 473178 (0.0006) [2023-12-27 08:55:15,233][20347] Updated weights for policy 0, policy_version 473600 (0.0006) [2023-12-27 08:55:15,259][20381] Updated weights for policy 1, policy_version 473188 (0.0006) [2023-12-27 08:55:15,318][20381] Updated weights for policy 1, policy_version 473198 (0.0007) [2023-12-27 08:55:15,943][20347] Updated weights for policy 0, policy_version 473610 (0.0008) [2023-12-27 08:55:15,969][20381] Updated weights for policy 1, policy_version 473208 (0.0006) [2023-12-27 08:55:15,991][20347] Updated weights for policy 0, policy_version 473620 (0.0006) [2023-12-27 08:55:16,023][20381] Updated weights for policy 1, policy_version 473218 (0.0008) [2023-12-27 08:55:16,036][20347] Updated weights for policy 0, policy_version 473630 (0.0007) [2023-12-27 08:55:16,079][20381] Updated weights for policy 1, policy_version 473228 (0.0007) [2023-12-27 08:55:16,678][20381] Updated weights for policy 1, policy_version 473238 (0.0009) [2023-12-27 08:55:16,699][20347] Updated weights for policy 0, policy_version 473640 (0.0007) [2023-12-27 08:55:16,737][20381] Updated weights for policy 1, policy_version 473248 (0.0010) [2023-12-27 08:55:16,751][20347] Updated weights for policy 0, policy_version 473650 (0.0006) [2023-12-27 08:55:16,788][20381] Updated weights for policy 1, policy_version 473258 (0.0006) [2023-12-27 08:55:16,802][20347] Updated weights for policy 0, policy_version 473660 (0.0006) [2023-12-27 08:55:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 19466.4). Total num frames: 242450432. Throughput: 0: 10503.7, 1: 10423.0. Samples: 242464624. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:55:17,363][19170] Avg episode reward: [(0, '961.530'), (1, '1000.000')] [2023-12-27 08:55:17,376][20381] Updated weights for policy 1, policy_version 473268 (0.0008) [2023-12-27 08:55:17,421][20347] Updated weights for policy 0, policy_version 473670 (0.0006) [2023-12-27 08:55:17,434][20381] Updated weights for policy 1, policy_version 473278 (0.0011) [2023-12-27 08:55:17,469][20347] Updated weights for policy 0, policy_version 473680 (0.0006) [2023-12-27 08:55:17,489][20381] Updated weights for policy 1, policy_version 473288 (0.0010) [2023-12-27 08:55:17,518][20347] Updated weights for policy 0, policy_version 473690 (0.0009) [2023-12-27 08:55:18,095][20381] Updated weights for policy 1, policy_version 473298 (0.0009) [2023-12-27 08:55:18,152][20347] Updated weights for policy 0, policy_version 473700 (0.0010) [2023-12-27 08:55:18,156][20381] Updated weights for policy 1, policy_version 473308 (0.0005) [2023-12-27 08:55:18,203][20347] Updated weights for policy 0, policy_version 473710 (0.0010) [2023-12-27 08:55:18,211][20381] Updated weights for policy 1, policy_version 473318 (0.0010) [2023-12-27 08:55:18,254][20347] Updated weights for policy 0, policy_version 473720 (0.0010) [2023-12-27 08:55:18,258][20381] Updated weights for policy 1, policy_version 473328 (0.0010) [2023-12-27 08:55:18,883][20381] Updated weights for policy 1, policy_version 473338 (0.0006) [2023-12-27 08:55:18,920][20347] Updated weights for policy 0, policy_version 473730 (0.0010) [2023-12-27 08:55:18,942][20381] Updated weights for policy 1, policy_version 473348 (0.0005) [2023-12-27 08:55:18,967][20347] Updated weights for policy 0, policy_version 473740 (0.0009) [2023-12-27 08:55:18,999][20381] Updated weights for policy 1, policy_version 473358 (0.0005) [2023-12-27 08:55:19,011][20347] Updated weights for policy 0, policy_version 473750 (0.0007) [2023-12-27 08:55:19,056][20347] Updated weights for policy 0, policy_version 473760 (0.0010) [2023-12-27 08:55:19,638][20381] Updated weights for policy 1, policy_version 473368 (0.0010) [2023-12-27 08:55:19,688][20381] Updated weights for policy 1, policy_version 473378 (0.0011) [2023-12-27 08:55:19,746][20381] Updated weights for policy 1, policy_version 473388 (0.0011) [2023-12-27 08:55:19,752][20347] Updated weights for policy 0, policy_version 473770 (0.0008) [2023-12-27 08:55:19,815][20347] Updated weights for policy 0, policy_version 473780 (0.0008) [2023-12-27 08:55:19,881][20347] Updated weights for policy 0, policy_version 473790 (0.0006) [2023-12-27 08:55:20,482][20381] Updated weights for policy 1, policy_version 473398 (0.0007) [2023-12-27 08:55:20,533][20381] Updated weights for policy 1, policy_version 473408 (0.0005) [2023-12-27 08:55:20,546][20347] Updated weights for policy 0, policy_version 473800 (0.0006) [2023-12-27 08:55:20,587][20381] Updated weights for policy 1, policy_version 473418 (0.0007) [2023-12-27 08:55:20,608][20347] Updated weights for policy 0, policy_version 473810 (0.0006) [2023-12-27 08:55:20,666][20347] Updated weights for policy 0, policy_version 473820 (0.0006) [2023-12-27 08:55:21,334][20381] Updated weights for policy 1, policy_version 473428 (0.0008) [2023-12-27 08:55:21,357][20347] Updated weights for policy 0, policy_version 473830 (0.0008) [2023-12-27 08:55:21,398][20381] Updated weights for policy 1, policy_version 473438 (0.0008) [2023-12-27 08:55:21,421][20347] Updated weights for policy 0, policy_version 473840 (0.0007) [2023-12-27 08:55:21,456][20381] Updated weights for policy 1, policy_version 473448 (0.0006) [2023-12-27 08:55:21,474][20347] Updated weights for policy 0, policy_version 473850 (0.0008) [2023-12-27 08:55:22,064][20381] Updated weights for policy 1, policy_version 473458 (0.0006) [2023-12-27 08:55:22,126][20381] Updated weights for policy 1, policy_version 473468 (0.0008) [2023-12-27 08:55:22,184][20381] Updated weights for policy 1, policy_version 473478 (0.0007) [2023-12-27 08:55:22,229][20347] Updated weights for policy 0, policy_version 473860 (0.0007) [2023-12-27 08:55:22,244][20381] Updated weights for policy 1, policy_version 473488 (0.0007) [2023-12-27 08:55:22,289][20347] Updated weights for policy 0, policy_version 473870 (0.0009) [2023-12-27 08:55:22,349][20347] Updated weights for policy 0, policy_version 473880 (0.0007) [2023-12-27 08:55:22,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20889.6, 300 sec: 19438.6). Total num frames: 242556928. Throughput: 0: 10532.4, 1: 10467.4. Samples: 242529060. Policy #0 lag: (min: 8.0, avg: 35.0, max: 40.0) [2023-12-27 08:55:22,363][19170] Avg episode reward: [(0, '951.670'), (1, '1000.000')] [2023-12-27 08:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000473488_121233408.pth... [2023-12-27 08:55:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000472240_120913920.pth [2023-12-27 08:55:22,395][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000473888_121331712.pth... [2023-12-27 08:55:22,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000472640_121012224.pth [2023-12-27 08:55:22,898][20381] Updated weights for policy 1, policy_version 473498 (0.0008) [2023-12-27 08:55:22,948][20381] Updated weights for policy 1, policy_version 473508 (0.0008) [2023-12-27 08:55:22,995][20381] Updated weights for policy 1, policy_version 473518 (0.0008) [2023-12-27 08:55:23,089][20347] Updated weights for policy 0, policy_version 473890 (0.0008) [2023-12-27 08:55:23,140][20347] Updated weights for policy 0, policy_version 473900 (0.0007) [2023-12-27 08:55:23,198][20347] Updated weights for policy 0, policy_version 473910 (0.0008) [2023-12-27 08:55:23,250][20347] Updated weights for policy 0, policy_version 473920 (0.0008) [2023-12-27 08:55:23,635][20381] Updated weights for policy 1, policy_version 473528 (0.0006) [2023-12-27 08:55:23,681][20381] Updated weights for policy 1, policy_version 473538 (0.0010) [2023-12-27 08:55:23,725][20381] Updated weights for policy 1, policy_version 473548 (0.0008) [2023-12-27 08:55:23,916][20347] Updated weights for policy 0, policy_version 473930 (0.0006) [2023-12-27 08:55:23,968][20347] Updated weights for policy 0, policy_version 473940 (0.0007) [2023-12-27 08:55:24,013][20347] Updated weights for policy 0, policy_version 473950 (0.0009) [2023-12-27 08:55:24,345][20381] Updated weights for policy 1, policy_version 473558 (0.0007) [2023-12-27 08:55:24,394][20381] Updated weights for policy 1, policy_version 473568 (0.0010) [2023-12-27 08:55:24,442][20381] Updated weights for policy 1, policy_version 473578 (0.0010) [2023-12-27 08:55:24,730][20347] Updated weights for policy 0, policy_version 473960 (0.0006) [2023-12-27 08:55:24,792][20347] Updated weights for policy 0, policy_version 473970 (0.0007) [2023-12-27 08:55:24,845][20347] Updated weights for policy 0, policy_version 473980 (0.0007) [2023-12-27 08:55:25,098][20381] Updated weights for policy 1, policy_version 473588 (0.0009) [2023-12-27 08:55:25,149][20381] Updated weights for policy 1, policy_version 473598 (0.0008) [2023-12-27 08:55:25,199][20381] Updated weights for policy 1, policy_version 473608 (0.0007) [2023-12-27 08:55:25,508][20347] Updated weights for policy 0, policy_version 473990 (0.0006) [2023-12-27 08:55:25,575][20347] Updated weights for policy 0, policy_version 474000 (0.0008) [2023-12-27 08:55:25,633][20347] Updated weights for policy 0, policy_version 474010 (0.0005) [2023-12-27 08:55:25,893][20381] Updated weights for policy 1, policy_version 473618 (0.0008) [2023-12-27 08:55:25,952][20381] Updated weights for policy 1, policy_version 473628 (0.0008) [2023-12-27 08:55:26,005][20381] Updated weights for policy 1, policy_version 473638 (0.0008) [2023-12-27 08:55:26,063][20381] Updated weights for policy 1, policy_version 473648 (0.0008) [2023-12-27 08:55:26,229][20347] Updated weights for policy 0, policy_version 474020 (0.0007) [2023-12-27 08:55:26,280][20347] Updated weights for policy 0, policy_version 474030 (0.0005) [2023-12-27 08:55:26,331][20347] Updated weights for policy 0, policy_version 474040 (0.0005) [2023-12-27 08:55:26,754][20381] Updated weights for policy 1, policy_version 473658 (0.0007) [2023-12-27 08:55:26,813][20381] Updated weights for policy 1, policy_version 473668 (0.0008) [2023-12-27 08:55:26,879][20381] Updated weights for policy 1, policy_version 473678 (0.0008) [2023-12-27 08:55:26,957][20347] Updated weights for policy 0, policy_version 474050 (0.0005) [2023-12-27 08:55:27,017][20347] Updated weights for policy 0, policy_version 474060 (0.0006) [2023-12-27 08:55:27,078][20347] Updated weights for policy 0, policy_version 474070 (0.0008) [2023-12-27 08:55:27,143][20347] Updated weights for policy 0, policy_version 474080 (0.0008) [2023-12-27 08:55:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 19438.6). Total num frames: 242663424. Throughput: 0: 10436.2, 1: 10534.7. Samples: 242652556. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:27,363][19170] Avg episode reward: [(0, '951.640'), (1, '1000.000')] [2023-12-27 08:55:27,506][20381] Updated weights for policy 1, policy_version 473688 (0.0007) [2023-12-27 08:55:27,558][20381] Updated weights for policy 1, policy_version 473698 (0.0008) [2023-12-27 08:55:27,613][20381] Updated weights for policy 1, policy_version 473708 (0.0007) [2023-12-27 08:55:27,805][20347] Updated weights for policy 0, policy_version 474090 (0.0007) [2023-12-27 08:55:27,865][20347] Updated weights for policy 0, policy_version 474100 (0.0007) [2023-12-27 08:55:27,920][20347] Updated weights for policy 0, policy_version 474110 (0.0007) [2023-12-27 08:55:28,275][20381] Updated weights for policy 1, policy_version 473718 (0.0009) [2023-12-27 08:55:28,320][20381] Updated weights for policy 1, policy_version 473728 (0.0010) [2023-12-27 08:55:28,382][20381] Updated weights for policy 1, policy_version 473738 (0.0008) [2023-12-27 08:55:28,577][20347] Updated weights for policy 0, policy_version 474120 (0.0007) [2023-12-27 08:55:28,630][20347] Updated weights for policy 0, policy_version 474130 (0.0008) [2023-12-27 08:55:28,676][20347] Updated weights for policy 0, policy_version 474140 (0.0005) [2023-12-27 08:55:29,039][20381] Updated weights for policy 1, policy_version 473748 (0.0011) [2023-12-27 08:55:29,093][20381] Updated weights for policy 1, policy_version 473758 (0.0010) [2023-12-27 08:55:29,148][20381] Updated weights for policy 1, policy_version 473768 (0.0010) [2023-12-27 08:55:29,306][20347] Updated weights for policy 0, policy_version 474150 (0.0006) [2023-12-27 08:55:29,366][20347] Updated weights for policy 0, policy_version 474160 (0.0007) [2023-12-27 08:55:29,424][20347] Updated weights for policy 0, policy_version 474170 (0.0007) [2023-12-27 08:55:29,848][20381] Updated weights for policy 1, policy_version 473778 (0.0010) [2023-12-27 08:55:29,907][20381] Updated weights for policy 1, policy_version 473788 (0.0010) [2023-12-27 08:55:29,967][20381] Updated weights for policy 1, policy_version 473798 (0.0008) [2023-12-27 08:55:30,022][20381] Updated weights for policy 1, policy_version 473808 (0.0008) [2023-12-27 08:55:30,049][20347] Updated weights for policy 0, policy_version 474180 (0.0007) [2023-12-27 08:55:30,099][20347] Updated weights for policy 0, policy_version 474190 (0.0008) [2023-12-27 08:55:30,149][20347] Updated weights for policy 0, policy_version 474200 (0.0009) [2023-12-27 08:55:30,682][20381] Updated weights for policy 1, policy_version 473818 (0.0007) [2023-12-27 08:55:30,732][20381] Updated weights for policy 1, policy_version 473828 (0.0005) [2023-12-27 08:55:30,782][20381] Updated weights for policy 1, policy_version 473838 (0.0005) [2023-12-27 08:55:30,886][20347] Updated weights for policy 0, policy_version 474210 (0.0007) [2023-12-27 08:55:30,931][20347] Updated weights for policy 0, policy_version 474220 (0.0006) [2023-12-27 08:55:30,986][20347] Updated weights for policy 0, policy_version 474230 (0.0005) [2023-12-27 08:55:31,050][20347] Updated weights for policy 0, policy_version 474240 (0.0006) [2023-12-27 08:55:31,429][20381] Updated weights for policy 1, policy_version 473848 (0.0007) [2023-12-27 08:55:31,496][20381] Updated weights for policy 1, policy_version 473858 (0.0008) [2023-12-27 08:55:31,559][20381] Updated weights for policy 1, policy_version 473868 (0.0008) [2023-12-27 08:55:31,743][20347] Updated weights for policy 0, policy_version 474250 (0.0008) [2023-12-27 08:55:31,794][20347] Updated weights for policy 0, policy_version 474261 (0.0007) [2023-12-27 08:55:31,842][20347] Updated weights for policy 0, policy_version 474271 (0.0009) [2023-12-27 08:55:32,163][20381] Updated weights for policy 1, policy_version 473878 (0.0009) [2023-12-27 08:55:32,216][20381] Updated weights for policy 1, policy_version 473888 (0.0010) [2023-12-27 08:55:32,276][20381] Updated weights for policy 1, policy_version 473898 (0.0010) [2023-12-27 08:55:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 19466.4). Total num frames: 242769920. Throughput: 0: 10427.1, 1: 10563.1. Samples: 242778024. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:32,363][19170] Avg episode reward: [(0, '902.900'), (1, '1000.000')] [2023-12-27 08:55:32,566][20347] Updated weights for policy 0, policy_version 474281 (0.0008) [2023-12-27 08:55:32,614][20347] Updated weights for policy 0, policy_version 474291 (0.0007) [2023-12-27 08:55:32,664][20347] Updated weights for policy 0, policy_version 474301 (0.0007) [2023-12-27 08:55:32,974][20381] Updated weights for policy 1, policy_version 473908 (0.0010) [2023-12-27 08:55:33,019][20381] Updated weights for policy 1, policy_version 473918 (0.0010) [2023-12-27 08:55:33,068][20381] Updated weights for policy 1, policy_version 473928 (0.0007) [2023-12-27 08:55:33,362][20347] Updated weights for policy 0, policy_version 474311 (0.0007) [2023-12-27 08:55:33,417][20347] Updated weights for policy 0, policy_version 474321 (0.0007) [2023-12-27 08:55:33,471][20347] Updated weights for policy 0, policy_version 474331 (0.0007) [2023-12-27 08:55:33,641][20381] Updated weights for policy 1, policy_version 473938 (0.0006) [2023-12-27 08:55:33,704][20381] Updated weights for policy 1, policy_version 473948 (0.0008) [2023-12-27 08:55:33,768][20381] Updated weights for policy 1, policy_version 473958 (0.0008) [2023-12-27 08:55:33,828][20381] Updated weights for policy 1, policy_version 473968 (0.0008) [2023-12-27 08:55:34,159][20347] Updated weights for policy 0, policy_version 474341 (0.0007) [2023-12-27 08:55:34,225][20347] Updated weights for policy 0, policy_version 474351 (0.0009) [2023-12-27 08:55:34,295][20347] Updated weights for policy 0, policy_version 474361 (0.0008) [2023-12-27 08:55:34,462][20381] Updated weights for policy 1, policy_version 473978 (0.0011) [2023-12-27 08:55:34,520][20381] Updated weights for policy 1, policy_version 473988 (0.0009) [2023-12-27 08:55:34,577][20381] Updated weights for policy 1, policy_version 473998 (0.0006) [2023-12-27 08:55:35,027][20347] Updated weights for policy 0, policy_version 474371 (0.0008) [2023-12-27 08:55:35,089][20347] Updated weights for policy 0, policy_version 474381 (0.0007) [2023-12-27 08:55:35,154][20347] Updated weights for policy 0, policy_version 474391 (0.0007) [2023-12-27 08:55:35,224][20381] Updated weights for policy 1, policy_version 474008 (0.0005) [2023-12-27 08:55:35,282][20381] Updated weights for policy 1, policy_version 474018 (0.0005) [2023-12-27 08:55:35,340][20381] Updated weights for policy 1, policy_version 474028 (0.0009) [2023-12-27 08:55:35,831][20347] Updated weights for policy 0, policy_version 474401 (0.0007) [2023-12-27 08:55:35,881][20347] Updated weights for policy 0, policy_version 474411 (0.0006) [2023-12-27 08:55:35,942][20347] Updated weights for policy 0, policy_version 474421 (0.0005) [2023-12-27 08:55:35,966][20381] Updated weights for policy 1, policy_version 474038 (0.0008) [2023-12-27 08:55:35,996][20347] Updated weights for policy 0, policy_version 474431 (0.0005) [2023-12-27 08:55:36,019][20381] Updated weights for policy 1, policy_version 474048 (0.0008) [2023-12-27 08:55:36,076][20381] Updated weights for policy 1, policy_version 474058 (0.0008) [2023-12-27 08:55:36,664][20347] Updated weights for policy 0, policy_version 474441 (0.0005) [2023-12-27 08:55:36,723][20347] Updated weights for policy 0, policy_version 474451 (0.0007) [2023-12-27 08:55:36,765][20347] Updated weights for policy 0, policy_version 474461 (0.0007) [2023-12-27 08:55:36,809][20381] Updated weights for policy 1, policy_version 474068 (0.0008) [2023-12-27 08:55:36,868][20381] Updated weights for policy 1, policy_version 474078 (0.0008) [2023-12-27 08:55:36,931][20381] Updated weights for policy 1, policy_version 474088 (0.0008) [2023-12-27 08:55:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 19466.4). Total num frames: 242868224. Throughput: 0: 10407.1, 1: 10597.2. Samples: 242840348. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:37,363][19170] Avg episode reward: [(0, '912.850'), (1, '1000.000')] [2023-12-27 08:55:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000474096_121389056.pth... [2023-12-27 08:55:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000472880_121077760.pth [2023-12-27 08:55:37,405][20347] Updated weights for policy 0, policy_version 474471 (0.0008) [2023-12-27 08:55:37,456][20347] Updated weights for policy 0, policy_version 474481 (0.0007) [2023-12-27 08:55:37,513][20347] Updated weights for policy 0, policy_version 474491 (0.0008) [2023-12-27 08:55:37,540][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000474496_121487360.pth... [2023-12-27 08:55:37,544][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000473280_121176064.pth [2023-12-27 08:55:37,629][20381] Updated weights for policy 1, policy_version 474098 (0.0007) [2023-12-27 08:55:37,688][20381] Updated weights for policy 1, policy_version 474108 (0.0011) [2023-12-27 08:55:37,744][20381] Updated weights for policy 1, policy_version 474118 (0.0008) [2023-12-27 08:55:37,795][20381] Updated weights for policy 1, policy_version 474128 (0.0006) [2023-12-27 08:55:38,195][20347] Updated weights for policy 0, policy_version 474501 (0.0006) [2023-12-27 08:55:38,249][20347] Updated weights for policy 0, policy_version 474511 (0.0005) [2023-12-27 08:55:38,300][20347] Updated weights for policy 0, policy_version 474521 (0.0005) [2023-12-27 08:55:38,473][20381] Updated weights for policy 1, policy_version 474138 (0.0011) [2023-12-27 08:55:38,530][20381] Updated weights for policy 1, policy_version 474148 (0.0011) [2023-12-27 08:55:38,604][20381] Updated weights for policy 1, policy_version 474158 (0.0010) [2023-12-27 08:55:38,978][20347] Updated weights for policy 0, policy_version 474531 (0.0007) [2023-12-27 08:55:39,032][20347] Updated weights for policy 0, policy_version 474541 (0.0007) [2023-12-27 08:55:39,089][20347] Updated weights for policy 0, policy_version 474551 (0.0007) [2023-12-27 08:55:39,282][20381] Updated weights for policy 1, policy_version 474168 (0.0011) [2023-12-27 08:55:39,339][20381] Updated weights for policy 1, policy_version 474178 (0.0011) [2023-12-27 08:55:39,407][20381] Updated weights for policy 1, policy_version 474188 (0.0011) [2023-12-27 08:55:39,822][20347] Updated weights for policy 0, policy_version 474561 (0.0007) [2023-12-27 08:55:39,882][20347] Updated weights for policy 0, policy_version 474571 (0.0009) [2023-12-27 08:55:39,946][20347] Updated weights for policy 0, policy_version 474581 (0.0008) [2023-12-27 08:55:40,008][20347] Updated weights for policy 0, policy_version 474591 (0.0008) [2023-12-27 08:55:40,077][20381] Updated weights for policy 1, policy_version 474198 (0.0006) [2023-12-27 08:55:40,133][20381] Updated weights for policy 1, policy_version 474208 (0.0007) [2023-12-27 08:55:40,188][20381] Updated weights for policy 1, policy_version 474218 (0.0008) [2023-12-27 08:55:40,716][20347] Updated weights for policy 0, policy_version 474601 (0.0009) [2023-12-27 08:55:40,764][20347] Updated weights for policy 0, policy_version 474611 (0.0007) [2023-12-27 08:55:40,821][20381] Updated weights for policy 1, policy_version 474228 (0.0006) [2023-12-27 08:55:40,826][20347] Updated weights for policy 0, policy_version 474621 (0.0007) [2023-12-27 08:55:40,887][20381] Updated weights for policy 1, policy_version 474238 (0.0005) [2023-12-27 08:55:40,947][20381] Updated weights for policy 1, policy_version 474248 (0.0006) [2023-12-27 08:55:41,589][20347] Updated weights for policy 0, policy_version 474631 (0.0008) [2023-12-27 08:55:41,599][20381] Updated weights for policy 1, policy_version 474258 (0.0006) [2023-12-27 08:55:41,652][20347] Updated weights for policy 0, policy_version 474641 (0.0007) [2023-12-27 08:55:41,665][20381] Updated weights for policy 1, policy_version 474268 (0.0007) [2023-12-27 08:55:41,714][20347] Updated weights for policy 0, policy_version 474651 (0.0007) [2023-12-27 08:55:41,718][20381] Updated weights for policy 1, policy_version 474278 (0.0009) [2023-12-27 08:55:41,766][20381] Updated weights for policy 1, policy_version 474288 (0.0008) [2023-12-27 08:55:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20889.6, 300 sec: 19410.9). Total num frames: 242966528. Throughput: 0: 10356.6, 1: 10563.8. Samples: 242962356. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:42,363][19170] Avg episode reward: [(0, '961.510'), (1, '990.520')] [2023-12-27 08:55:42,394][20347] Updated weights for policy 0, policy_version 474661 (0.0007) [2023-12-27 08:55:42,455][20347] Updated weights for policy 0, policy_version 474671 (0.0007) [2023-12-27 08:55:42,514][20347] Updated weights for policy 0, policy_version 474681 (0.0006) [2023-12-27 08:55:42,525][20381] Updated weights for policy 1, policy_version 474298 (0.0008) [2023-12-27 08:55:42,590][20381] Updated weights for policy 1, policy_version 474308 (0.0006) [2023-12-27 08:55:42,655][20381] Updated weights for policy 1, policy_version 474318 (0.0006) [2023-12-27 08:55:43,184][20347] Updated weights for policy 0, policy_version 474691 (0.0007) [2023-12-27 08:55:43,237][20347] Updated weights for policy 0, policy_version 474701 (0.0007) [2023-12-27 08:55:43,271][20381] Updated weights for policy 1, policy_version 474328 (0.0010) [2023-12-27 08:55:43,291][20347] Updated weights for policy 0, policy_version 474711 (0.0005) [2023-12-27 08:55:43,328][20381] Updated weights for policy 1, policy_version 474338 (0.0009) [2023-12-27 08:55:43,391][20381] Updated weights for policy 1, policy_version 474348 (0.0010) [2023-12-27 08:55:43,981][20347] Updated weights for policy 0, policy_version 474721 (0.0007) [2023-12-27 08:55:44,034][20381] Updated weights for policy 1, policy_version 474358 (0.0010) [2023-12-27 08:55:44,037][20347] Updated weights for policy 0, policy_version 474731 (0.0006) [2023-12-27 08:55:44,088][20381] Updated weights for policy 1, policy_version 474368 (0.0009) [2023-12-27 08:55:44,092][20347] Updated weights for policy 0, policy_version 474741 (0.0007) [2023-12-27 08:55:44,145][20381] Updated weights for policy 1, policy_version 474378 (0.0009) [2023-12-27 08:55:44,146][20347] Updated weights for policy 0, policy_version 474751 (0.0007) [2023-12-27 08:55:44,819][20381] Updated weights for policy 1, policy_version 474388 (0.0009) [2023-12-27 08:55:44,828][20347] Updated weights for policy 0, policy_version 474761 (0.0007) [2023-12-27 08:55:44,874][20381] Updated weights for policy 1, policy_version 474398 (0.0006) [2023-12-27 08:55:44,888][20347] Updated weights for policy 0, policy_version 474771 (0.0007) [2023-12-27 08:55:44,926][20381] Updated weights for policy 1, policy_version 474408 (0.0006) [2023-12-27 08:55:44,947][20347] Updated weights for policy 0, policy_version 474781 (0.0007) [2023-12-27 08:55:45,547][20381] Updated weights for policy 1, policy_version 474418 (0.0007) [2023-12-27 08:55:45,603][20381] Updated weights for policy 1, policy_version 474428 (0.0010) [2023-12-27 08:55:45,661][20381] Updated weights for policy 1, policy_version 474438 (0.0010) [2023-12-27 08:55:45,707][20347] Updated weights for policy 0, policy_version 474791 (0.0006) [2023-12-27 08:55:45,719][20381] Updated weights for policy 1, policy_version 474448 (0.0010) [2023-12-27 08:55:45,759][20347] Updated weights for policy 0, policy_version 474801 (0.0006) [2023-12-27 08:55:45,814][20347] Updated weights for policy 0, policy_version 474811 (0.0007) [2023-12-27 08:55:46,390][20381] Updated weights for policy 1, policy_version 474458 (0.0010) [2023-12-27 08:55:46,446][20381] Updated weights for policy 1, policy_version 474468 (0.0011) [2023-12-27 08:55:46,509][20381] Updated weights for policy 1, policy_version 474478 (0.0010) [2023-12-27 08:55:46,510][20347] Updated weights for policy 0, policy_version 474821 (0.0007) [2023-12-27 08:55:46,561][20347] Updated weights for policy 0, policy_version 474831 (0.0007) [2023-12-27 08:55:46,604][20347] Updated weights for policy 0, policy_version 474841 (0.0007) [2023-12-27 08:55:47,163][20381] Updated weights for policy 1, policy_version 474488 (0.0011) [2023-12-27 08:55:47,219][20381] Updated weights for policy 1, policy_version 474498 (0.0010) [2023-12-27 08:55:47,268][20381] Updated weights for policy 1, policy_version 474508 (0.0010) [2023-12-27 08:55:47,302][20347] Updated weights for policy 0, policy_version 474851 (0.0007) [2023-12-27 08:55:47,353][20347] Updated weights for policy 0, policy_version 474861 (0.0007) [2023-12-27 08:55:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 19438.7). Total num frames: 243073024. Throughput: 0: 10280.1, 1: 10526.2. Samples: 243084124. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:47,363][19170] Avg episode reward: [(0, '980.800'), (1, '990.520')] [2023-12-27 08:55:47,409][20347] Updated weights for policy 0, policy_version 474871 (0.0007) [2023-12-27 08:55:47,900][20381] Updated weights for policy 1, policy_version 474518 (0.0007) [2023-12-27 08:55:47,958][20381] Updated weights for policy 1, policy_version 474528 (0.0008) [2023-12-27 08:55:48,012][20381] Updated weights for policy 1, policy_version 474538 (0.0008) [2023-12-27 08:55:48,084][20347] Updated weights for policy 0, policy_version 474881 (0.0007) [2023-12-27 08:55:48,146][20347] Updated weights for policy 0, policy_version 474891 (0.0008) [2023-12-27 08:55:48,202][20347] Updated weights for policy 0, policy_version 474901 (0.0008) [2023-12-27 08:55:48,260][20347] Updated weights for policy 0, policy_version 474911 (0.0008) [2023-12-27 08:55:48,688][20381] Updated weights for policy 1, policy_version 474548 (0.0007) [2023-12-27 08:55:48,746][20381] Updated weights for policy 1, policy_version 474558 (0.0007) [2023-12-27 08:55:48,792][20381] Updated weights for policy 1, policy_version 474568 (0.0008) [2023-12-27 08:55:48,922][20347] Updated weights for policy 0, policy_version 474921 (0.0008) [2023-12-27 08:55:48,976][20347] Updated weights for policy 0, policy_version 474931 (0.0007) [2023-12-27 08:55:49,028][20347] Updated weights for policy 0, policy_version 474941 (0.0007) [2023-12-27 08:55:49,397][20381] Updated weights for policy 1, policy_version 474578 (0.0008) [2023-12-27 08:55:49,447][20381] Updated weights for policy 1, policy_version 474588 (0.0006) [2023-12-27 08:55:49,495][20381] Updated weights for policy 1, policy_version 474598 (0.0006) [2023-12-27 08:55:49,554][20381] Updated weights for policy 1, policy_version 474608 (0.0005) [2023-12-27 08:55:49,809][20347] Updated weights for policy 0, policy_version 474951 (0.0007) [2023-12-27 08:55:49,874][20347] Updated weights for policy 0, policy_version 474961 (0.0007) [2023-12-27 08:55:49,930][20347] Updated weights for policy 0, policy_version 474971 (0.0008) [2023-12-27 08:55:50,148][20381] Updated weights for policy 1, policy_version 474618 (0.0007) [2023-12-27 08:55:50,203][20381] Updated weights for policy 1, policy_version 474628 (0.0005) [2023-12-27 08:55:50,258][20381] Updated weights for policy 1, policy_version 474638 (0.0006) [2023-12-27 08:55:50,540][20347] Updated weights for policy 0, policy_version 474981 (0.0006) [2023-12-27 08:55:50,601][20347] Updated weights for policy 0, policy_version 474991 (0.0008) [2023-12-27 08:55:50,666][20347] Updated weights for policy 0, policy_version 475001 (0.0006) [2023-12-27 08:55:50,972][20381] Updated weights for policy 1, policy_version 474648 (0.0006) [2023-12-27 08:55:51,032][20381] Updated weights for policy 1, policy_version 474658 (0.0007) [2023-12-27 08:55:51,090][20381] Updated weights for policy 1, policy_version 474668 (0.0008) [2023-12-27 08:55:51,353][20347] Updated weights for policy 0, policy_version 475011 (0.0007) [2023-12-27 08:55:51,417][20347] Updated weights for policy 0, policy_version 475021 (0.0008) [2023-12-27 08:55:51,472][20347] Updated weights for policy 0, policy_version 475031 (0.0007) [2023-12-27 08:55:51,769][20381] Updated weights for policy 1, policy_version 474678 (0.0006) [2023-12-27 08:55:51,824][20381] Updated weights for policy 1, policy_version 474688 (0.0006) [2023-12-27 08:55:51,873][20381] Updated weights for policy 1, policy_version 474698 (0.0007) [2023-12-27 08:55:52,187][20347] Updated weights for policy 0, policy_version 475041 (0.0008) [2023-12-27 08:55:52,237][20347] Updated weights for policy 0, policy_version 475051 (0.0009) [2023-12-27 08:55:52,292][20347] Updated weights for policy 0, policy_version 475061 (0.0008) [2023-12-27 08:55:52,346][20347] Updated weights for policy 0, policy_version 475071 (0.0008) [2023-12-27 08:55:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 19466.4). Total num frames: 243179520. Throughput: 0: 10206.0, 1: 10522.4. Samples: 243146636. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:52,363][19170] Avg episode reward: [(0, '950.390'), (1, '1000.000')] [2023-12-27 08:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000474704_121544704.pth... [2023-12-27 08:55:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000475072_121634816.pth... [2023-12-27 08:55:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000473488_121233408.pth [2023-12-27 08:55:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000473888_121331712.pth [2023-12-27 08:55:52,498][20381] Updated weights for policy 1, policy_version 474708 (0.0008) [2023-12-27 08:55:52,555][20381] Updated weights for policy 1, policy_version 474718 (0.0007) [2023-12-27 08:55:52,619][20381] Updated weights for policy 1, policy_version 474728 (0.0008) [2023-12-27 08:55:52,952][20347] Updated weights for policy 0, policy_version 475081 (0.0006) [2023-12-27 08:55:53,009][20347] Updated weights for policy 0, policy_version 475091 (0.0005) [2023-12-27 08:55:53,059][20347] Updated weights for policy 0, policy_version 475101 (0.0006) [2023-12-27 08:55:53,331][20381] Updated weights for policy 1, policy_version 474738 (0.0007) [2023-12-27 08:55:53,382][20381] Updated weights for policy 1, policy_version 474748 (0.0005) [2023-12-27 08:55:53,433][20381] Updated weights for policy 1, policy_version 474758 (0.0005) [2023-12-27 08:55:53,487][20381] Updated weights for policy 1, policy_version 474768 (0.0007) [2023-12-27 08:55:53,715][20347] Updated weights for policy 0, policy_version 475111 (0.0008) [2023-12-27 08:55:53,769][20347] Updated weights for policy 0, policy_version 475121 (0.0006) [2023-12-27 08:55:53,832][20347] Updated weights for policy 0, policy_version 475131 (0.0005) [2023-12-27 08:55:54,057][20381] Updated weights for policy 1, policy_version 474778 (0.0005) [2023-12-27 08:55:54,104][20381] Updated weights for policy 1, policy_version 474788 (0.0006) [2023-12-27 08:55:54,152][20381] Updated weights for policy 1, policy_version 474798 (0.0008) [2023-12-27 08:55:54,484][20347] Updated weights for policy 0, policy_version 475141 (0.0006) [2023-12-27 08:55:54,539][20347] Updated weights for policy 0, policy_version 475151 (0.0007) [2023-12-27 08:55:54,599][20347] Updated weights for policy 0, policy_version 475161 (0.0007) [2023-12-27 08:55:54,797][20381] Updated weights for policy 1, policy_version 474808 (0.0006) [2023-12-27 08:55:54,849][20381] Updated weights for policy 1, policy_version 474818 (0.0005) [2023-12-27 08:55:54,899][20381] Updated weights for policy 1, policy_version 474828 (0.0005) [2023-12-27 08:55:55,276][20347] Updated weights for policy 0, policy_version 475171 (0.0008) [2023-12-27 08:55:55,330][20347] Updated weights for policy 0, policy_version 475181 (0.0006) [2023-12-27 08:55:55,387][20347] Updated weights for policy 0, policy_version 475191 (0.0005) [2023-12-27 08:55:55,560][20381] Updated weights for policy 1, policy_version 474838 (0.0006) [2023-12-27 08:55:55,613][20381] Updated weights for policy 1, policy_version 474848 (0.0005) [2023-12-27 08:55:55,664][20381] Updated weights for policy 1, policy_version 474858 (0.0005) [2023-12-27 08:55:56,043][20347] Updated weights for policy 0, policy_version 475201 (0.0005) [2023-12-27 08:55:56,106][20347] Updated weights for policy 0, policy_version 475211 (0.0005) [2023-12-27 08:55:56,158][20347] Updated weights for policy 0, policy_version 475221 (0.0006) [2023-12-27 08:55:56,208][20347] Updated weights for policy 0, policy_version 475231 (0.0007) [2023-12-27 08:55:56,326][20381] Updated weights for policy 1, policy_version 474868 (0.0007) [2023-12-27 08:55:56,375][20381] Updated weights for policy 1, policy_version 474878 (0.0005) [2023-12-27 08:55:56,425][20381] Updated weights for policy 1, policy_version 474888 (0.0006) [2023-12-27 08:55:56,780][20347] Updated weights for policy 0, policy_version 475241 (0.0006) [2023-12-27 08:55:56,834][20347] Updated weights for policy 0, policy_version 475251 (0.0006) [2023-12-27 08:55:56,890][20347] Updated weights for policy 0, policy_version 475261 (0.0009) [2023-12-27 08:55:57,030][20381] Updated weights for policy 1, policy_version 474898 (0.0008) [2023-12-27 08:55:57,081][20381] Updated weights for policy 1, policy_version 474908 (0.0005) [2023-12-27 08:55:57,140][20381] Updated weights for policy 1, policy_version 474918 (0.0005) [2023-12-27 08:55:57,197][20381] Updated weights for policy 1, policy_version 474928 (0.0007) [2023-12-27 08:55:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 19438.6). Total num frames: 243286016. Throughput: 0: 10200.2, 1: 10555.7. Samples: 243272972. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:55:57,363][19170] Avg episode reward: [(0, '902.760'), (1, '1000.000')] [2023-12-27 08:55:57,547][20347] Updated weights for policy 0, policy_version 475271 (0.0006) [2023-12-27 08:55:57,602][20347] Updated weights for policy 0, policy_version 475281 (0.0006) [2023-12-27 08:55:57,653][20347] Updated weights for policy 0, policy_version 475291 (0.0007) [2023-12-27 08:55:57,846][20381] Updated weights for policy 1, policy_version 474938 (0.0010) [2023-12-27 08:55:57,899][20381] Updated weights for policy 1, policy_version 474948 (0.0010) [2023-12-27 08:55:57,955][20381] Updated weights for policy 1, policy_version 474958 (0.0010) [2023-12-27 08:55:58,349][20347] Updated weights for policy 0, policy_version 475301 (0.0007) [2023-12-27 08:55:58,407][20347] Updated weights for policy 0, policy_version 475311 (0.0008) [2023-12-27 08:55:58,467][20347] Updated weights for policy 0, policy_version 475321 (0.0007) [2023-12-27 08:55:58,639][20381] Updated weights for policy 1, policy_version 474968 (0.0011) [2023-12-27 08:55:58,702][20381] Updated weights for policy 1, policy_version 474978 (0.0010) [2023-12-27 08:55:58,768][20381] Updated weights for policy 1, policy_version 474988 (0.0011) [2023-12-27 08:55:59,223][20347] Updated weights for policy 0, policy_version 475331 (0.0008) [2023-12-27 08:55:59,283][20347] Updated weights for policy 0, policy_version 475341 (0.0008) [2023-12-27 08:55:59,343][20347] Updated weights for policy 0, policy_version 475351 (0.0008) [2023-12-27 08:55:59,458][20381] Updated weights for policy 1, policy_version 474998 (0.0007) [2023-12-27 08:55:59,526][20381] Updated weights for policy 1, policy_version 475008 (0.0005) [2023-12-27 08:55:59,592][20381] Updated weights for policy 1, policy_version 475018 (0.0006) [2023-12-27 08:56:00,045][20347] Updated weights for policy 0, policy_version 475361 (0.0008) [2023-12-27 08:56:00,089][20347] Updated weights for policy 0, policy_version 475371 (0.0007) [2023-12-27 08:56:00,133][20347] Updated weights for policy 0, policy_version 475381 (0.0007) [2023-12-27 08:56:00,184][20347] Updated weights for policy 0, policy_version 475391 (0.0007) [2023-12-27 08:56:00,263][20381] Updated weights for policy 1, policy_version 475028 (0.0009) [2023-12-27 08:56:00,317][20381] Updated weights for policy 1, policy_version 475038 (0.0010) [2023-12-27 08:56:00,371][20381] Updated weights for policy 1, policy_version 475048 (0.0010) [2023-12-27 08:56:00,829][20347] Updated weights for policy 0, policy_version 475401 (0.0010) [2023-12-27 08:56:00,889][20347] Updated weights for policy 0, policy_version 475411 (0.0010) [2023-12-27 08:56:00,943][20347] Updated weights for policy 0, policy_version 475421 (0.0010) [2023-12-27 08:56:01,043][20381] Updated weights for policy 1, policy_version 475058 (0.0010) [2023-12-27 08:56:01,107][20381] Updated weights for policy 1, policy_version 475068 (0.0006) [2023-12-27 08:56:01,164][20381] Updated weights for policy 1, policy_version 475078 (0.0007) [2023-12-27 08:56:01,223][20381] Updated weights for policy 1, policy_version 475088 (0.0006) [2023-12-27 08:56:01,625][20347] Updated weights for policy 0, policy_version 475431 (0.0008) [2023-12-27 08:56:01,687][20347] Updated weights for policy 0, policy_version 475441 (0.0010) [2023-12-27 08:56:01,744][20347] Updated weights for policy 0, policy_version 475451 (0.0011) [2023-12-27 08:56:01,902][20381] Updated weights for policy 1, policy_version 475098 (0.0010) [2023-12-27 08:56:01,964][20381] Updated weights for policy 1, policy_version 475108 (0.0010) [2023-12-27 08:56:02,024][20381] Updated weights for policy 1, policy_version 475118 (0.0011) [2023-12-27 08:56:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 19438.7). Total num frames: 243384320. Throughput: 0: 10206.4, 1: 10501.2. Samples: 243396468. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:02,363][19170] Avg episode reward: [(0, '952.280'), (1, '1000.000')] [2023-12-27 08:56:02,409][20347] Updated weights for policy 0, policy_version 475461 (0.0009) [2023-12-27 08:56:02,467][20347] Updated weights for policy 0, policy_version 475471 (0.0007) [2023-12-27 08:56:02,529][20347] Updated weights for policy 0, policy_version 475481 (0.0007) [2023-12-27 08:56:02,704][20381] Updated weights for policy 1, policy_version 475128 (0.0011) [2023-12-27 08:56:02,762][20381] Updated weights for policy 1, policy_version 475138 (0.0011) [2023-12-27 08:56:02,817][20381] Updated weights for policy 1, policy_version 475148 (0.0010) [2023-12-27 08:56:03,152][20347] Updated weights for policy 0, policy_version 475491 (0.0010) [2023-12-27 08:56:03,206][20347] Updated weights for policy 0, policy_version 475501 (0.0010) [2023-12-27 08:56:03,259][20347] Updated weights for policy 0, policy_version 475511 (0.0010) [2023-12-27 08:56:03,484][20381] Updated weights for policy 1, policy_version 475158 (0.0010) [2023-12-27 08:56:03,545][20381] Updated weights for policy 1, policy_version 475168 (0.0011) [2023-12-27 08:56:03,599][20381] Updated weights for policy 1, policy_version 475178 (0.0010) [2023-12-27 08:56:03,935][20347] Updated weights for policy 0, policy_version 475521 (0.0010) [2023-12-27 08:56:03,990][20347] Updated weights for policy 0, policy_version 475531 (0.0008) [2023-12-27 08:56:04,053][20347] Updated weights for policy 0, policy_version 475541 (0.0008) [2023-12-27 08:56:04,110][20347] Updated weights for policy 0, policy_version 475551 (0.0008) [2023-12-27 08:56:04,281][20381] Updated weights for policy 1, policy_version 475188 (0.0011) [2023-12-27 08:56:04,340][20381] Updated weights for policy 1, policy_version 475198 (0.0011) [2023-12-27 08:56:04,400][20381] Updated weights for policy 1, policy_version 475208 (0.0011) [2023-12-27 08:56:04,821][20347] Updated weights for policy 0, policy_version 475561 (0.0006) [2023-12-27 08:56:04,878][20347] Updated weights for policy 0, policy_version 475571 (0.0005) [2023-12-27 08:56:04,941][20347] Updated weights for policy 0, policy_version 475581 (0.0005) [2023-12-27 08:56:05,087][20381] Updated weights for policy 1, policy_version 475218 (0.0011) [2023-12-27 08:56:05,139][20381] Updated weights for policy 1, policy_version 475228 (0.0010) [2023-12-27 08:56:05,187][20381] Updated weights for policy 1, policy_version 475238 (0.0010) [2023-12-27 08:56:05,251][20381] Updated weights for policy 1, policy_version 475248 (0.0010) [2023-12-27 08:56:05,557][20347] Updated weights for policy 0, policy_version 475591 (0.0009) [2023-12-27 08:56:05,603][20347] Updated weights for policy 0, policy_version 475601 (0.0008) [2023-12-27 08:56:05,653][20347] Updated weights for policy 0, policy_version 475611 (0.0007) [2023-12-27 08:56:05,897][20381] Updated weights for policy 1, policy_version 475258 (0.0007) [2023-12-27 08:56:05,958][20381] Updated weights for policy 1, policy_version 475268 (0.0007) [2023-12-27 08:56:06,020][20381] Updated weights for policy 1, policy_version 475278 (0.0011) [2023-12-27 08:56:06,364][20347] Updated weights for policy 0, policy_version 475621 (0.0008) [2023-12-27 08:56:06,423][20347] Updated weights for policy 0, policy_version 475631 (0.0006) [2023-12-27 08:56:06,485][20347] Updated weights for policy 0, policy_version 475641 (0.0007) [2023-12-27 08:56:06,698][20381] Updated weights for policy 1, policy_version 475288 (0.0011) [2023-12-27 08:56:06,751][20381] Updated weights for policy 1, policy_version 475298 (0.0011) [2023-12-27 08:56:06,813][20381] Updated weights for policy 1, policy_version 475308 (0.0011) [2023-12-27 08:56:07,084][20347] Updated weights for policy 0, policy_version 475651 (0.0007) [2023-12-27 08:56:07,137][20347] Updated weights for policy 0, policy_version 475661 (0.0009) [2023-12-27 08:56:07,188][20347] Updated weights for policy 0, policy_version 475671 (0.0010) [2023-12-27 08:56:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 19438.6). Total num frames: 243490816. Throughput: 0: 10182.9, 1: 10462.1. Samples: 243458088. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:07,364][19170] Avg episode reward: [(0, '955.570'), (1, '1000.000')] [2023-12-27 08:56:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000475680_121790464.pth... [2023-12-27 08:56:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000475312_121700352.pth... [2023-12-27 08:56:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000474496_121487360.pth [2023-12-27 08:56:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000474096_121389056.pth [2023-12-27 08:56:07,437][20381] Updated weights for policy 1, policy_version 475318 (0.0011) [2023-12-27 08:56:07,495][20381] Updated weights for policy 1, policy_version 475328 (0.0010) [2023-12-27 08:56:07,553][20381] Updated weights for policy 1, policy_version 475338 (0.0010) [2023-12-27 08:56:07,812][20347] Updated weights for policy 0, policy_version 475682 (0.0010) [2023-12-27 08:56:07,873][20347] Updated weights for policy 0, policy_version 475692 (0.0006) [2023-12-27 08:56:07,928][20347] Updated weights for policy 0, policy_version 475702 (0.0005) [2023-12-27 08:56:07,987][20347] Updated weights for policy 0, policy_version 475712 (0.0005) [2023-12-27 08:56:08,235][20381] Updated weights for policy 1, policy_version 475348 (0.0010) [2023-12-27 08:56:08,283][20381] Updated weights for policy 1, policy_version 475358 (0.0010) [2023-12-27 08:56:08,337][20381] Updated weights for policy 1, policy_version 475368 (0.0011) [2023-12-27 08:56:08,588][20347] Updated weights for policy 0, policy_version 475722 (0.0006) [2023-12-27 08:56:08,643][20347] Updated weights for policy 0, policy_version 475732 (0.0007) [2023-12-27 08:56:08,691][20347] Updated weights for policy 0, policy_version 475742 (0.0007) [2023-12-27 08:56:09,022][20381] Updated weights for policy 1, policy_version 475378 (0.0010) [2023-12-27 08:56:09,091][20381] Updated weights for policy 1, policy_version 475388 (0.0006) [2023-12-27 08:56:09,146][20381] Updated weights for policy 1, policy_version 475398 (0.0006) [2023-12-27 08:56:09,199][20381] Updated weights for policy 1, policy_version 475408 (0.0006) [2023-12-27 08:56:09,343][20347] Updated weights for policy 0, policy_version 475752 (0.0009) [2023-12-27 08:56:09,405][20347] Updated weights for policy 0, policy_version 475762 (0.0009) [2023-12-27 08:56:09,469][20347] Updated weights for policy 0, policy_version 475772 (0.0007) [2023-12-27 08:56:09,957][20381] Updated weights for policy 1, policy_version 475418 (0.0009) [2023-12-27 08:56:10,021][20381] Updated weights for policy 1, policy_version 475428 (0.0009) [2023-12-27 08:56:10,085][20381] Updated weights for policy 1, policy_version 475438 (0.0009) [2023-12-27 08:56:10,130][20347] Updated weights for policy 0, policy_version 475782 (0.0006) [2023-12-27 08:56:10,195][20347] Updated weights for policy 0, policy_version 475792 (0.0007) [2023-12-27 08:56:10,252][20347] Updated weights for policy 0, policy_version 475802 (0.0005) [2023-12-27 08:56:10,846][20381] Updated weights for policy 1, policy_version 475448 (0.0010) [2023-12-27 08:56:10,899][20381] Updated weights for policy 1, policy_version 475458 (0.0010) [2023-12-27 08:56:10,907][20347] Updated weights for policy 0, policy_version 475812 (0.0008) [2023-12-27 08:56:10,948][20381] Updated weights for policy 1, policy_version 475468 (0.0010) [2023-12-27 08:56:10,958][20347] Updated weights for policy 0, policy_version 475822 (0.0005) [2023-12-27 08:56:11,013][20347] Updated weights for policy 0, policy_version 475832 (0.0005) [2023-12-27 08:56:11,667][20381] Updated weights for policy 1, policy_version 475478 (0.0011) [2023-12-27 08:56:11,727][20381] Updated weights for policy 1, policy_version 475488 (0.0010) [2023-12-27 08:56:11,754][20347] Updated weights for policy 0, policy_version 475842 (0.0006) [2023-12-27 08:56:11,797][20381] Updated weights for policy 1, policy_version 475498 (0.0007) [2023-12-27 08:56:11,818][20347] Updated weights for policy 0, policy_version 475852 (0.0006) [2023-12-27 08:56:11,880][20347] Updated weights for policy 0, policy_version 475862 (0.0007) [2023-12-27 08:56:11,936][20347] Updated weights for policy 0, policy_version 475872 (0.0008) [2023-12-27 08:56:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 19410.9). Total num frames: 243589120. Throughput: 0: 10271.0, 1: 10398.4. Samples: 243582680. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:12,363][19170] Avg episode reward: [(0, '981.410'), (1, '1000.000')] [2023-12-27 08:56:12,466][20381] Updated weights for policy 1, policy_version 475508 (0.0008) [2023-12-27 08:56:12,518][20381] Updated weights for policy 1, policy_version 475518 (0.0005) [2023-12-27 08:56:12,567][20381] Updated weights for policy 1, policy_version 475528 (0.0007) [2023-12-27 08:56:12,647][20347] Updated weights for policy 0, policy_version 475882 (0.0006) [2023-12-27 08:56:12,702][20347] Updated weights for policy 0, policy_version 475892 (0.0007) [2023-12-27 08:56:12,756][20347] Updated weights for policy 0, policy_version 475902 (0.0007) [2023-12-27 08:56:13,161][20381] Updated weights for policy 1, policy_version 475538 (0.0009) [2023-12-27 08:56:13,223][20381] Updated weights for policy 1, policy_version 475548 (0.0006) [2023-12-27 08:56:13,284][20381] Updated weights for policy 1, policy_version 475558 (0.0010) [2023-12-27 08:56:13,338][20381] Updated weights for policy 1, policy_version 475568 (0.0011) [2023-12-27 08:56:13,465][20347] Updated weights for policy 0, policy_version 475912 (0.0008) [2023-12-27 08:56:13,520][20347] Updated weights for policy 0, policy_version 475922 (0.0010) [2023-12-27 08:56:13,579][20347] Updated weights for policy 0, policy_version 475932 (0.0010) [2023-12-27 08:56:13,975][20381] Updated weights for policy 1, policy_version 475578 (0.0011) [2023-12-27 08:56:14,029][20381] Updated weights for policy 1, policy_version 475588 (0.0010) [2023-12-27 08:56:14,086][20381] Updated weights for policy 1, policy_version 475598 (0.0005) [2023-12-27 08:56:14,206][20347] Updated weights for policy 0, policy_version 475942 (0.0010) [2023-12-27 08:56:14,264][20347] Updated weights for policy 0, policy_version 475952 (0.0010) [2023-12-27 08:56:14,321][20347] Updated weights for policy 0, policy_version 475962 (0.0009) [2023-12-27 08:56:14,769][20381] Updated weights for policy 1, policy_version 475608 (0.0006) [2023-12-27 08:56:14,829][20381] Updated weights for policy 1, policy_version 475618 (0.0010) [2023-12-27 08:56:14,890][20381] Updated weights for policy 1, policy_version 475628 (0.0010) [2023-12-27 08:56:14,919][20347] Updated weights for policy 0, policy_version 475972 (0.0005) [2023-12-27 08:56:14,982][20347] Updated weights for policy 0, policy_version 475982 (0.0007) [2023-12-27 08:56:15,038][20347] Updated weights for policy 0, policy_version 475992 (0.0008) [2023-12-27 08:56:15,591][20381] Updated weights for policy 1, policy_version 475638 (0.0010) [2023-12-27 08:56:15,643][20381] Updated weights for policy 1, policy_version 475648 (0.0010) [2023-12-27 08:56:15,661][20347] Updated weights for policy 0, policy_version 476002 (0.0007) [2023-12-27 08:56:15,705][20381] Updated weights for policy 1, policy_version 475658 (0.0011) [2023-12-27 08:56:15,719][20347] Updated weights for policy 0, policy_version 476012 (0.0007) [2023-12-27 08:56:15,766][20347] Updated weights for policy 0, policy_version 476022 (0.0007) [2023-12-27 08:56:15,820][20347] Updated weights for policy 0, policy_version 476032 (0.0007) [2023-12-27 08:56:16,401][20381] Updated weights for policy 1, policy_version 475668 (0.0009) [2023-12-27 08:56:16,458][20381] Updated weights for policy 1, policy_version 475678 (0.0006) [2023-12-27 08:56:16,510][20381] Updated weights for policy 1, policy_version 475688 (0.0006) [2023-12-27 08:56:16,527][20347] Updated weights for policy 0, policy_version 476042 (0.0007) [2023-12-27 08:56:16,577][20347] Updated weights for policy 0, policy_version 476052 (0.0006) [2023-12-27 08:56:16,622][20347] Updated weights for policy 0, policy_version 476062 (0.0007) [2023-12-27 08:56:17,110][20381] Updated weights for policy 1, policy_version 475698 (0.0006) [2023-12-27 08:56:17,164][20381] Updated weights for policy 1, policy_version 475708 (0.0006) [2023-12-27 08:56:17,210][20381] Updated weights for policy 1, policy_version 475718 (0.0007) [2023-12-27 08:56:17,256][20381] Updated weights for policy 1, policy_version 475728 (0.0005) [2023-12-27 08:56:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.0, 300 sec: 19438.6). Total num frames: 243695616. Throughput: 0: 10252.0, 1: 10391.1. Samples: 243706964. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:17,363][19170] Avg episode reward: [(0, '989.830'), (1, '1000.000')] [2023-12-27 08:56:17,364][20347] Updated weights for policy 0, policy_version 476072 (0.0009) [2023-12-27 08:56:17,415][20347] Updated weights for policy 0, policy_version 476082 (0.0008) [2023-12-27 08:56:17,462][20347] Updated weights for policy 0, policy_version 476092 (0.0010) [2023-12-27 08:56:17,835][20381] Updated weights for policy 1, policy_version 475738 (0.0005) [2023-12-27 08:56:17,882][20381] Updated weights for policy 1, policy_version 475748 (0.0006) [2023-12-27 08:56:17,934][20381] Updated weights for policy 1, policy_version 475758 (0.0007) [2023-12-27 08:56:18,194][20347] Updated weights for policy 0, policy_version 476102 (0.0007) [2023-12-27 08:56:18,258][20347] Updated weights for policy 0, policy_version 476112 (0.0005) [2023-12-27 08:56:18,318][20347] Updated weights for policy 0, policy_version 476122 (0.0006) [2023-12-27 08:56:18,605][20381] Updated weights for policy 1, policy_version 475768 (0.0006) [2023-12-27 08:56:18,656][20381] Updated weights for policy 1, policy_version 475778 (0.0007) [2023-12-27 08:56:18,716][20381] Updated weights for policy 1, policy_version 475788 (0.0008) [2023-12-27 08:56:18,913][20347] Updated weights for policy 0, policy_version 476132 (0.0007) [2023-12-27 08:56:18,962][20347] Updated weights for policy 0, policy_version 476142 (0.0005) [2023-12-27 08:56:19,009][20347] Updated weights for policy 0, policy_version 476152 (0.0005) [2023-12-27 08:56:19,376][20381] Updated weights for policy 1, policy_version 475798 (0.0007) [2023-12-27 08:56:19,431][20381] Updated weights for policy 1, policy_version 475808 (0.0007) [2023-12-27 08:56:19,486][20381] Updated weights for policy 1, policy_version 475818 (0.0008) [2023-12-27 08:56:19,686][20347] Updated weights for policy 0, policy_version 476162 (0.0006) [2023-12-27 08:56:19,741][20347] Updated weights for policy 0, policy_version 476172 (0.0006) [2023-12-27 08:56:19,797][20347] Updated weights for policy 0, policy_version 476182 (0.0007) [2023-12-27 08:56:19,858][20347] Updated weights for policy 0, policy_version 476192 (0.0007) [2023-12-27 08:56:20,143][20381] Updated weights for policy 1, policy_version 475828 (0.0009) [2023-12-27 08:56:20,206][20381] Updated weights for policy 1, policy_version 475838 (0.0011) [2023-12-27 08:56:20,264][20381] Updated weights for policy 1, policy_version 475848 (0.0010) [2023-12-27 08:56:20,565][20347] Updated weights for policy 0, policy_version 476202 (0.0008) [2023-12-27 08:56:20,629][20347] Updated weights for policy 0, policy_version 476212 (0.0007) [2023-12-27 08:56:20,701][20347] Updated weights for policy 0, policy_version 476222 (0.0006) [2023-12-27 08:56:20,894][20381] Updated weights for policy 1, policy_version 475858 (0.0010) [2023-12-27 08:56:20,950][20381] Updated weights for policy 1, policy_version 475868 (0.0008) [2023-12-27 08:56:21,009][20381] Updated weights for policy 1, policy_version 475878 (0.0008) [2023-12-27 08:56:21,073][20381] Updated weights for policy 1, policy_version 475888 (0.0006) [2023-12-27 08:56:21,354][20347] Updated weights for policy 0, policy_version 476232 (0.0008) [2023-12-27 08:56:21,427][20347] Updated weights for policy 0, policy_version 476242 (0.0008) [2023-12-27 08:56:21,492][20347] Updated weights for policy 0, policy_version 476252 (0.0007) [2023-12-27 08:56:21,786][20381] Updated weights for policy 1, policy_version 475898 (0.0011) [2023-12-27 08:56:21,842][20381] Updated weights for policy 1, policy_version 475908 (0.0011) [2023-12-27 08:56:21,899][20381] Updated weights for policy 1, policy_version 475918 (0.0011) [2023-12-27 08:56:22,173][20347] Updated weights for policy 0, policy_version 476262 (0.0008) [2023-12-27 08:56:22,235][20347] Updated weights for policy 0, policy_version 476272 (0.0006) [2023-12-27 08:56:22,297][20347] Updated weights for policy 0, policy_version 476282 (0.0007) [2023-12-27 08:56:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 19438.6). Total num frames: 243802112. Throughput: 0: 10275.2, 1: 10393.4. Samples: 243770436. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:22,363][19170] Avg episode reward: [(0, '951.690'), (1, '1000.000')] [2023-12-27 08:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000475920_121856000.pth... [2023-12-27 08:56:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000476288_121946112.pth... [2023-12-27 08:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000474704_121544704.pth [2023-12-27 08:56:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000475072_121634816.pth [2023-12-27 08:56:22,612][20381] Updated weights for policy 1, policy_version 475928 (0.0008) [2023-12-27 08:56:22,675][20381] Updated weights for policy 1, policy_version 475938 (0.0008) [2023-12-27 08:56:22,735][20381] Updated weights for policy 1, policy_version 475948 (0.0006) [2023-12-27 08:56:22,973][20347] Updated weights for policy 0, policy_version 476292 (0.0006) [2023-12-27 08:56:23,035][20347] Updated weights for policy 0, policy_version 476302 (0.0007) [2023-12-27 08:56:23,087][20347] Updated weights for policy 0, policy_version 476312 (0.0008) [2023-12-27 08:56:23,418][20381] Updated weights for policy 1, policy_version 475958 (0.0011) [2023-12-27 08:56:23,468][20381] Updated weights for policy 1, policy_version 475968 (0.0010) [2023-12-27 08:56:23,523][20381] Updated weights for policy 1, policy_version 475978 (0.0010) [2023-12-27 08:56:23,691][20347] Updated weights for policy 0, policy_version 476322 (0.0007) [2023-12-27 08:56:23,743][20347] Updated weights for policy 0, policy_version 476332 (0.0010) [2023-12-27 08:56:23,797][20347] Updated weights for policy 0, policy_version 476342 (0.0010) [2023-12-27 08:56:23,859][20347] Updated weights for policy 0, policy_version 476352 (0.0009) [2023-12-27 08:56:24,130][20381] Updated weights for policy 1, policy_version 475988 (0.0008) [2023-12-27 08:56:24,189][20381] Updated weights for policy 1, policy_version 475998 (0.0006) [2023-12-27 08:56:24,242][20381] Updated weights for policy 1, policy_version 476008 (0.0007) [2023-12-27 08:56:24,503][20347] Updated weights for policy 0, policy_version 476362 (0.0010) [2023-12-27 08:56:24,551][20347] Updated weights for policy 0, policy_version 476372 (0.0010) [2023-12-27 08:56:24,621][20347] Updated weights for policy 0, policy_version 476382 (0.0010) [2023-12-27 08:56:24,771][20381] Updated weights for policy 1, policy_version 476018 (0.0006) [2023-12-27 08:56:24,822][20381] Updated weights for policy 1, policy_version 476028 (0.0006) [2023-12-27 08:56:24,867][20381] Updated weights for policy 1, policy_version 476038 (0.0007) [2023-12-27 08:56:24,917][20381] Updated weights for policy 1, policy_version 476048 (0.0008) [2023-12-27 08:56:25,269][20347] Updated weights for policy 0, policy_version 476392 (0.0009) [2023-12-27 08:56:25,338][20347] Updated weights for policy 0, policy_version 476402 (0.0006) [2023-12-27 08:56:25,399][20347] Updated weights for policy 0, policy_version 476412 (0.0008) [2023-12-27 08:56:25,641][20381] Updated weights for policy 1, policy_version 476058 (0.0010) [2023-12-27 08:56:25,689][20381] Updated weights for policy 1, policy_version 476068 (0.0010) [2023-12-27 08:56:25,750][20381] Updated weights for policy 1, policy_version 476078 (0.0010) [2023-12-27 08:56:26,047][20347] Updated weights for policy 0, policy_version 476422 (0.0010) [2023-12-27 08:56:26,096][20347] Updated weights for policy 0, policy_version 476432 (0.0010) [2023-12-27 08:56:26,147][20347] Updated weights for policy 0, policy_version 476442 (0.0010) [2023-12-27 08:56:26,413][20381] Updated weights for policy 1, policy_version 476088 (0.0006) [2023-12-27 08:56:26,467][20381] Updated weights for policy 1, policy_version 476098 (0.0005) [2023-12-27 08:56:26,520][20381] Updated weights for policy 1, policy_version 476108 (0.0005) [2023-12-27 08:56:26,825][20347] Updated weights for policy 0, policy_version 476452 (0.0010) [2023-12-27 08:56:26,876][20347] Updated weights for policy 0, policy_version 476462 (0.0010) [2023-12-27 08:56:26,926][20347] Updated weights for policy 0, policy_version 476472 (0.0010) [2023-12-27 08:56:27,092][20381] Updated weights for policy 1, policy_version 476118 (0.0007) [2023-12-27 08:56:27,144][20381] Updated weights for policy 1, policy_version 476128 (0.0008) [2023-12-27 08:56:27,194][20381] Updated weights for policy 1, policy_version 476138 (0.0007) [2023-12-27 08:56:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 19466.4). Total num frames: 243908608. Throughput: 0: 10313.4, 1: 10421.5. Samples: 243895428. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:27,363][19170] Avg episode reward: [(0, '952.230'), (1, '1000.000')] [2023-12-27 08:56:27,563][20347] Updated weights for policy 0, policy_version 476482 (0.0009) [2023-12-27 08:56:27,612][20347] Updated weights for policy 0, policy_version 476492 (0.0010) [2023-12-27 08:56:27,667][20347] Updated weights for policy 0, policy_version 476502 (0.0010) [2023-12-27 08:56:27,714][20347] Updated weights for policy 0, policy_version 476512 (0.0010) [2023-12-27 08:56:27,927][20381] Updated weights for policy 1, policy_version 476148 (0.0007) [2023-12-27 08:56:27,980][20381] Updated weights for policy 1, policy_version 476158 (0.0007) [2023-12-27 08:56:28,037][20381] Updated weights for policy 1, policy_version 476168 (0.0008) [2023-12-27 08:56:28,388][20347] Updated weights for policy 0, policy_version 476522 (0.0010) [2023-12-27 08:56:28,440][20347] Updated weights for policy 0, policy_version 476532 (0.0010) [2023-12-27 08:56:28,488][20347] Updated weights for policy 0, policy_version 476542 (0.0010) [2023-12-27 08:56:28,670][20381] Updated weights for policy 1, policy_version 476178 (0.0008) [2023-12-27 08:56:28,728][20381] Updated weights for policy 1, policy_version 476188 (0.0010) [2023-12-27 08:56:28,782][20381] Updated weights for policy 1, policy_version 476198 (0.0010) [2023-12-27 08:56:28,836][20381] Updated weights for policy 1, policy_version 476208 (0.0009) [2023-12-27 08:56:29,176][20347] Updated weights for policy 0, policy_version 476552 (0.0007) [2023-12-27 08:56:29,235][20347] Updated weights for policy 0, policy_version 476562 (0.0008) [2023-12-27 08:56:29,294][20347] Updated weights for policy 0, policy_version 476572 (0.0006) [2023-12-27 08:56:29,460][20381] Updated weights for policy 1, policy_version 476218 (0.0008) [2023-12-27 08:56:29,513][20381] Updated weights for policy 1, policy_version 476228 (0.0008) [2023-12-27 08:56:29,564][20381] Updated weights for policy 1, policy_version 476238 (0.0010) [2023-12-27 08:56:29,868][20347] Updated weights for policy 0, policy_version 476582 (0.0007) [2023-12-27 08:56:29,928][20347] Updated weights for policy 0, policy_version 476592 (0.0008) [2023-12-27 08:56:29,993][20347] Updated weights for policy 0, policy_version 476602 (0.0009) [2023-12-27 08:56:30,261][20381] Updated weights for policy 1, policy_version 476248 (0.0007) [2023-12-27 08:56:30,317][20381] Updated weights for policy 1, policy_version 476258 (0.0005) [2023-12-27 08:56:30,371][20381] Updated weights for policy 1, policy_version 476268 (0.0006) [2023-12-27 08:56:30,563][20347] Updated weights for policy 0, policy_version 476612 (0.0006) [2023-12-27 08:56:30,612][20347] Updated weights for policy 0, policy_version 476622 (0.0007) [2023-12-27 08:56:30,661][20347] Updated weights for policy 0, policy_version 476632 (0.0007) [2023-12-27 08:56:30,984][20381] Updated weights for policy 1, policy_version 476278 (0.0006) [2023-12-27 08:56:31,040][20381] Updated weights for policy 1, policy_version 476288 (0.0006) [2023-12-27 08:56:31,098][20381] Updated weights for policy 1, policy_version 476298 (0.0005) [2023-12-27 08:56:31,404][20347] Updated weights for policy 0, policy_version 476642 (0.0008) [2023-12-27 08:56:31,453][20347] Updated weights for policy 0, policy_version 476652 (0.0006) [2023-12-27 08:56:31,498][20347] Updated weights for policy 0, policy_version 476662 (0.0007) [2023-12-27 08:56:31,548][20347] Updated weights for policy 0, policy_version 476672 (0.0006) [2023-12-27 08:56:31,757][20381] Updated weights for policy 1, policy_version 476308 (0.0008) [2023-12-27 08:56:31,809][20381] Updated weights for policy 1, policy_version 476318 (0.0010) [2023-12-27 08:56:31,861][20381] Updated weights for policy 1, policy_version 476328 (0.0010) [2023-12-27 08:56:32,186][20347] Updated weights for policy 0, policy_version 476682 (0.0005) [2023-12-27 08:56:32,239][20347] Updated weights for policy 0, policy_version 476692 (0.0005) [2023-12-27 08:56:32,296][20347] Updated weights for policy 0, policy_version 476702 (0.0008) [2023-12-27 08:56:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 19466.4). Total num frames: 244015104. Throughput: 0: 10417.5, 1: 10436.9. Samples: 244022572. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:32,363][19170] Avg episode reward: [(0, '980.720'), (1, '1000.000')] [2023-12-27 08:56:32,523][20381] Updated weights for policy 1, policy_version 476338 (0.0009) [2023-12-27 08:56:32,571][20381] Updated weights for policy 1, policy_version 476348 (0.0006) [2023-12-27 08:56:32,623][20381] Updated weights for policy 1, policy_version 476358 (0.0006) [2023-12-27 08:56:32,685][20381] Updated weights for policy 1, policy_version 476368 (0.0005) [2023-12-27 08:56:32,969][20347] Updated weights for policy 0, policy_version 476712 (0.0007) [2023-12-27 08:56:33,021][20347] Updated weights for policy 0, policy_version 476722 (0.0007) [2023-12-27 08:56:33,069][20347] Updated weights for policy 0, policy_version 476732 (0.0008) [2023-12-27 08:56:33,252][20381] Updated weights for policy 1, policy_version 476378 (0.0007) [2023-12-27 08:56:33,304][20381] Updated weights for policy 1, policy_version 476388 (0.0008) [2023-12-27 08:56:33,350][20381] Updated weights for policy 1, policy_version 476398 (0.0008) [2023-12-27 08:56:33,747][20347] Updated weights for policy 0, policy_version 476742 (0.0009) [2023-12-27 08:56:33,807][20347] Updated weights for policy 0, policy_version 476752 (0.0010) [2023-12-27 08:56:33,861][20347] Updated weights for policy 0, policy_version 476762 (0.0009) [2023-12-27 08:56:33,935][20381] Updated weights for policy 1, policy_version 476408 (0.0008) [2023-12-27 08:56:33,991][20381] Updated weights for policy 1, policy_version 476418 (0.0007) [2023-12-27 08:56:34,037][20381] Updated weights for policy 1, policy_version 476428 (0.0007) [2023-12-27 08:56:34,498][20347] Updated weights for policy 0, policy_version 476772 (0.0006) [2023-12-27 08:56:34,587][20347] Updated weights for policy 0, policy_version 476782 (0.0008) [2023-12-27 08:56:34,647][20347] Updated weights for policy 0, policy_version 476792 (0.0009) [2023-12-27 08:56:34,716][20381] Updated weights for policy 1, policy_version 476438 (0.0006) [2023-12-27 08:56:34,769][20381] Updated weights for policy 1, policy_version 476448 (0.0005) [2023-12-27 08:56:34,827][20381] Updated weights for policy 1, policy_version 476458 (0.0007) [2023-12-27 08:56:35,335][20347] Updated weights for policy 0, policy_version 476802 (0.0007) [2023-12-27 08:56:35,393][20347] Updated weights for policy 0, policy_version 476812 (0.0008) [2023-12-27 08:56:35,448][20347] Updated weights for policy 0, policy_version 476822 (0.0007) [2023-12-27 08:56:35,461][20381] Updated weights for policy 1, policy_version 476468 (0.0007) [2023-12-27 08:56:35,500][20347] Updated weights for policy 0, policy_version 476832 (0.0005) [2023-12-27 08:56:35,513][20381] Updated weights for policy 1, policy_version 476478 (0.0006) [2023-12-27 08:56:35,561][20381] Updated weights for policy 1, policy_version 476488 (0.0007) [2023-12-27 08:56:36,157][20347] Updated weights for policy 0, policy_version 476842 (0.0008) [2023-12-27 08:56:36,215][20347] Updated weights for policy 0, policy_version 476852 (0.0007) [2023-12-27 08:56:36,238][20381] Updated weights for policy 1, policy_version 476498 (0.0008) [2023-12-27 08:56:36,278][20347] Updated weights for policy 0, policy_version 476862 (0.0007) [2023-12-27 08:56:36,296][20381] Updated weights for policy 1, policy_version 476508 (0.0008) [2023-12-27 08:56:36,359][20381] Updated weights for policy 1, policy_version 476518 (0.0009) [2023-12-27 08:56:36,418][20381] Updated weights for policy 1, policy_version 476528 (0.0008) [2023-12-27 08:56:36,900][20347] Updated weights for policy 0, policy_version 476872 (0.0009) [2023-12-27 08:56:36,953][20347] Updated weights for policy 0, policy_version 476882 (0.0009) [2023-12-27 08:56:37,008][20347] Updated weights for policy 0, policy_version 476892 (0.0006) [2023-12-27 08:56:37,164][20381] Updated weights for policy 1, policy_version 476538 (0.0007) [2023-12-27 08:56:37,219][20381] Updated weights for policy 1, policy_version 476548 (0.0009) [2023-12-27 08:56:37,272][20381] Updated weights for policy 1, policy_version 476558 (0.0008) [2023-12-27 08:56:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 19494.3). Total num frames: 244121600. Throughput: 0: 10443.5, 1: 10461.9. Samples: 244087384. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:37,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 08:56:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000476896_122101760.pth... [2023-12-27 08:56:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000476560_122019840.pth... [2023-12-27 08:56:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000475680_121790464.pth [2023-12-27 08:56:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000475312_121700352.pth [2023-12-27 08:56:37,655][20347] Updated weights for policy 0, policy_version 476902 (0.0007) [2023-12-27 08:56:37,707][20347] Updated weights for policy 0, policy_version 476912 (0.0007) [2023-12-27 08:56:37,763][20347] Updated weights for policy 0, policy_version 476922 (0.0008) [2023-12-27 08:56:37,997][20381] Updated weights for policy 1, policy_version 476568 (0.0008) [2023-12-27 08:56:38,059][20381] Updated weights for policy 1, policy_version 476578 (0.0006) [2023-12-27 08:56:38,115][20381] Updated weights for policy 1, policy_version 476588 (0.0007) [2023-12-27 08:56:38,461][20347] Updated weights for policy 0, policy_version 476932 (0.0008) [2023-12-27 08:56:38,523][20347] Updated weights for policy 0, policy_version 476942 (0.0008) [2023-12-27 08:56:38,579][20347] Updated weights for policy 0, policy_version 476952 (0.0006) [2023-12-27 08:56:38,796][20381] Updated weights for policy 1, policy_version 476598 (0.0006) [2023-12-27 08:56:38,860][20381] Updated weights for policy 1, policy_version 476608 (0.0006) [2023-12-27 08:56:38,917][20381] Updated weights for policy 1, policy_version 476618 (0.0008) [2023-12-27 08:56:39,264][20347] Updated weights for policy 0, policy_version 476962 (0.0006) [2023-12-27 08:56:39,328][20347] Updated weights for policy 0, policy_version 476972 (0.0006) [2023-12-27 08:56:39,398][20347] Updated weights for policy 0, policy_version 476982 (0.0006) [2023-12-27 08:56:39,459][20347] Updated weights for policy 0, policy_version 476992 (0.0005) [2023-12-27 08:56:39,514][20381] Updated weights for policy 1, policy_version 476628 (0.0008) [2023-12-27 08:56:39,579][20381] Updated weights for policy 1, policy_version 476638 (0.0008) [2023-12-27 08:56:39,632][20381] Updated weights for policy 1, policy_version 476648 (0.0010) [2023-12-27 08:56:40,173][20347] Updated weights for policy 0, policy_version 477002 (0.0006) [2023-12-27 08:56:40,230][20347] Updated weights for policy 0, policy_version 477012 (0.0008) [2023-12-27 08:56:40,281][20347] Updated weights for policy 0, policy_version 477022 (0.0007) [2023-12-27 08:56:40,324][20381] Updated weights for policy 1, policy_version 476658 (0.0011) [2023-12-27 08:56:40,384][20381] Updated weights for policy 1, policy_version 476668 (0.0011) [2023-12-27 08:56:40,441][20381] Updated weights for policy 1, policy_version 476678 (0.0007) [2023-12-27 08:56:40,501][20381] Updated weights for policy 1, policy_version 476688 (0.0006) [2023-12-27 08:56:40,997][20347] Updated weights for policy 0, policy_version 477032 (0.0006) [2023-12-27 08:56:41,063][20347] Updated weights for policy 0, policy_version 477042 (0.0008) [2023-12-27 08:56:41,123][20347] Updated weights for policy 0, policy_version 477052 (0.0008) [2023-12-27 08:56:41,213][20381] Updated weights for policy 1, policy_version 476698 (0.0008) [2023-12-27 08:56:41,275][20381] Updated weights for policy 1, policy_version 476708 (0.0008) [2023-12-27 08:56:41,330][20381] Updated weights for policy 1, policy_version 476718 (0.0008) [2023-12-27 08:56:41,798][20347] Updated weights for policy 0, policy_version 477062 (0.0006) [2023-12-27 08:56:41,856][20347] Updated weights for policy 0, policy_version 477072 (0.0005) [2023-12-27 08:56:41,908][20347] Updated weights for policy 0, policy_version 477082 (0.0005) [2023-12-27 08:56:42,086][20381] Updated weights for policy 1, policy_version 476728 (0.0007) [2023-12-27 08:56:42,142][20381] Updated weights for policy 1, policy_version 476738 (0.0006) [2023-12-27 08:56:42,196][20381] Updated weights for policy 1, policy_version 476748 (0.0006) [2023-12-27 08:56:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 19438.6). Total num frames: 244219904. Throughput: 0: 10417.1, 1: 10388.2. Samples: 244209212. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:56:42,528][20347] Updated weights for policy 0, policy_version 477092 (0.0005) [2023-12-27 08:56:42,585][20347] Updated weights for policy 0, policy_version 477102 (0.0005) [2023-12-27 08:56:42,651][20347] Updated weights for policy 0, policy_version 477112 (0.0005) [2023-12-27 08:56:42,868][20381] Updated weights for policy 1, policy_version 476758 (0.0005) [2023-12-27 08:56:42,922][20381] Updated weights for policy 1, policy_version 476768 (0.0005) [2023-12-27 08:56:42,979][20381] Updated weights for policy 1, policy_version 476778 (0.0005) [2023-12-27 08:56:43,246][20347] Updated weights for policy 0, policy_version 477122 (0.0006) [2023-12-27 08:56:43,302][20347] Updated weights for policy 0, policy_version 477132 (0.0007) [2023-12-27 08:56:43,347][20347] Updated weights for policy 0, policy_version 477142 (0.0005) [2023-12-27 08:56:43,395][20347] Updated weights for policy 0, policy_version 477152 (0.0005) [2023-12-27 08:56:43,595][20381] Updated weights for policy 1, policy_version 476788 (0.0006) [2023-12-27 08:56:43,641][20381] Updated weights for policy 1, policy_version 476798 (0.0008) [2023-12-27 08:56:43,684][20381] Updated weights for policy 1, policy_version 476808 (0.0007) [2023-12-27 08:56:44,060][20347] Updated weights for policy 0, policy_version 477162 (0.0010) [2023-12-27 08:56:44,116][20347] Updated weights for policy 0, policy_version 477172 (0.0010) [2023-12-27 08:56:44,172][20347] Updated weights for policy 0, policy_version 477182 (0.0010) [2023-12-27 08:56:44,418][20381] Updated weights for policy 1, policy_version 476818 (0.0007) [2023-12-27 08:56:44,471][20381] Updated weights for policy 1, policy_version 476828 (0.0008) [2023-12-27 08:56:44,526][20381] Updated weights for policy 1, policy_version 476838 (0.0008) [2023-12-27 08:56:44,576][20381] Updated weights for policy 1, policy_version 476848 (0.0005) [2023-12-27 08:56:44,824][20347] Updated weights for policy 0, policy_version 477192 (0.0009) [2023-12-27 08:56:44,874][20347] Updated weights for policy 0, policy_version 477202 (0.0011) [2023-12-27 08:56:44,927][20347] Updated weights for policy 0, policy_version 477212 (0.0011) [2023-12-27 08:56:45,211][20381] Updated weights for policy 1, policy_version 476858 (0.0005) [2023-12-27 08:56:45,260][20381] Updated weights for policy 1, policy_version 476868 (0.0008) [2023-12-27 08:56:45,313][20381] Updated weights for policy 1, policy_version 476878 (0.0007) [2023-12-27 08:56:45,614][20347] Updated weights for policy 0, policy_version 477222 (0.0011) [2023-12-27 08:56:45,672][20347] Updated weights for policy 0, policy_version 477232 (0.0010) [2023-12-27 08:56:45,731][20347] Updated weights for policy 0, policy_version 477242 (0.0006) [2023-12-27 08:56:45,943][20381] Updated weights for policy 1, policy_version 476888 (0.0006) [2023-12-27 08:56:45,998][20381] Updated weights for policy 1, policy_version 476898 (0.0007) [2023-12-27 08:56:46,049][20381] Updated weights for policy 1, policy_version 476908 (0.0007) [2023-12-27 08:56:46,311][20347] Updated weights for policy 0, policy_version 477252 (0.0007) [2023-12-27 08:56:46,365][20347] Updated weights for policy 0, policy_version 477262 (0.0010) [2023-12-27 08:56:46,417][20347] Updated weights for policy 0, policy_version 477272 (0.0010) [2023-12-27 08:56:46,726][20381] Updated weights for policy 1, policy_version 476918 (0.0007) [2023-12-27 08:56:46,772][20381] Updated weights for policy 1, policy_version 476928 (0.0006) [2023-12-27 08:56:46,821][20381] Updated weights for policy 1, policy_version 476938 (0.0005) [2023-12-27 08:56:47,120][20347] Updated weights for policy 0, policy_version 477282 (0.0008) [2023-12-27 08:56:47,172][20347] Updated weights for policy 0, policy_version 477292 (0.0006) [2023-12-27 08:56:47,219][20347] Updated weights for policy 0, policy_version 477302 (0.0007) [2023-12-27 08:56:47,277][20347] Updated weights for policy 0, policy_version 477312 (0.0006) [2023-12-27 08:56:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 19466.4). Total num frames: 244326400. Throughput: 0: 10461.0, 1: 10392.5. Samples: 244334880. Policy #0 lag: (min: 15.0, avg: 19.4, max: 47.0) [2023-12-27 08:56:47,363][19170] Avg episode reward: [(0, '990.920'), (1, '1000.000')] [2023-12-27 08:56:47,518][20381] Updated weights for policy 1, policy_version 476948 (0.0008) [2023-12-27 08:56:47,577][20381] Updated weights for policy 1, policy_version 476958 (0.0008) [2023-12-27 08:56:47,632][20381] Updated weights for policy 1, policy_version 476968 (0.0007) [2023-12-27 08:56:47,902][20347] Updated weights for policy 0, policy_version 477322 (0.0005) [2023-12-27 08:56:47,958][20347] Updated weights for policy 0, policy_version 477332 (0.0006) [2023-12-27 08:56:48,015][20347] Updated weights for policy 0, policy_version 477342 (0.0005) [2023-12-27 08:56:48,354][20381] Updated weights for policy 1, policy_version 476978 (0.0008) [2023-12-27 08:56:48,412][20381] Updated weights for policy 1, policy_version 476988 (0.0006) [2023-12-27 08:56:48,471][20381] Updated weights for policy 1, policy_version 476998 (0.0006) [2023-12-27 08:56:48,526][20381] Updated weights for policy 1, policy_version 477008 (0.0010) [2023-12-27 08:56:48,557][20347] Updated weights for policy 0, policy_version 477352 (0.0009) [2023-12-27 08:56:48,613][20347] Updated weights for policy 0, policy_version 477362 (0.0010) [2023-12-27 08:56:48,673][20347] Updated weights for policy 0, policy_version 477372 (0.0010) [2023-12-27 08:56:49,198][20381] Updated weights for policy 1, policy_version 477018 (0.0008) [2023-12-27 08:56:49,266][20381] Updated weights for policy 1, policy_version 477028 (0.0007) [2023-12-27 08:56:49,328][20381] Updated weights for policy 1, policy_version 477038 (0.0006) [2023-12-27 08:56:49,389][20347] Updated weights for policy 0, policy_version 477382 (0.0009) [2023-12-27 08:56:49,443][20347] Updated weights for policy 0, policy_version 477392 (0.0007) [2023-12-27 08:56:49,499][20347] Updated weights for policy 0, policy_version 477402 (0.0007) [2023-12-27 08:56:49,936][20381] Updated weights for policy 1, policy_version 477048 (0.0008) [2023-12-27 08:56:49,991][20381] Updated weights for policy 1, policy_version 477058 (0.0006) [2023-12-27 08:56:50,041][20381] Updated weights for policy 1, policy_version 477068 (0.0006) [2023-12-27 08:56:50,206][20347] Updated weights for policy 0, policy_version 477412 (0.0011) [2023-12-27 08:56:50,255][20347] Updated weights for policy 0, policy_version 477422 (0.0010) [2023-12-27 08:56:50,307][20347] Updated weights for policy 0, policy_version 477432 (0.0010) [2023-12-27 08:56:50,732][20381] Updated weights for policy 1, policy_version 477078 (0.0008) [2023-12-27 08:56:50,786][20381] Updated weights for policy 1, policy_version 477088 (0.0009) [2023-12-27 08:56:50,841][20381] Updated weights for policy 1, policy_version 477098 (0.0008) [2023-12-27 08:56:50,965][20347] Updated weights for policy 0, policy_version 477442 (0.0009) [2023-12-27 08:56:51,025][20347] Updated weights for policy 0, policy_version 477452 (0.0008) [2023-12-27 08:56:51,081][20347] Updated weights for policy 0, policy_version 477462 (0.0007) [2023-12-27 08:56:51,138][20347] Updated weights for policy 0, policy_version 477472 (0.0007) [2023-12-27 08:56:51,533][20381] Updated weights for policy 1, policy_version 477108 (0.0009) [2023-12-27 08:56:51,584][20381] Updated weights for policy 1, policy_version 477118 (0.0009) [2023-12-27 08:56:51,640][20381] Updated weights for policy 1, policy_version 477128 (0.0007) [2023-12-27 08:56:51,807][20347] Updated weights for policy 0, policy_version 477482 (0.0008) [2023-12-27 08:56:51,856][20347] Updated weights for policy 0, policy_version 477492 (0.0008) [2023-12-27 08:56:51,910][20347] Updated weights for policy 0, policy_version 477502 (0.0008) [2023-12-27 08:56:52,310][20381] Updated weights for policy 1, policy_version 477138 (0.0007) [2023-12-27 08:56:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 19410.9). Total num frames: 244424704. Throughput: 0: 10478.9, 1: 10401.0. Samples: 244397680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:56:52,363][19170] Avg episode reward: [(0, '990.920'), (1, '1000.000')] [2023-12-27 08:56:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000477504_122257408.pth... [2023-12-27 08:56:52,372][20381] Updated weights for policy 1, policy_version 477148 (0.0008) [2023-12-27 08:56:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000476288_121946112.pth [2023-12-27 08:56:52,425][20381] Updated weights for policy 1, policy_version 477158 (0.0005) [2023-12-27 08:56:52,483][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000477168_122175488.pth... [2023-12-27 08:56:52,485][20381] Updated weights for policy 1, policy_version 477168 (0.0007) [2023-12-27 08:56:52,486][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000475920_121856000.pth [2023-12-27 08:56:52,659][20347] Updated weights for policy 0, policy_version 477512 (0.0007) [2023-12-27 08:56:52,709][20347] Updated weights for policy 0, policy_version 477522 (0.0008) [2023-12-27 08:56:52,764][20347] Updated weights for policy 0, policy_version 477532 (0.0007) [2023-12-27 08:56:53,168][20381] Updated weights for policy 1, policy_version 477178 (0.0008) [2023-12-27 08:56:53,231][20381] Updated weights for policy 1, policy_version 477188 (0.0007) [2023-12-27 08:56:53,289][20381] Updated weights for policy 1, policy_version 477198 (0.0008) [2023-12-27 08:56:53,440][20347] Updated weights for policy 0, policy_version 477542 (0.0007) [2023-12-27 08:56:53,491][20347] Updated weights for policy 0, policy_version 477552 (0.0007) [2023-12-27 08:56:53,542][20347] Updated weights for policy 0, policy_version 477562 (0.0007) [2023-12-27 08:56:54,004][20381] Updated weights for policy 1, policy_version 477208 (0.0009) [2023-12-27 08:56:54,057][20381] Updated weights for policy 1, policy_version 477218 (0.0009) [2023-12-27 08:56:54,108][20381] Updated weights for policy 1, policy_version 477228 (0.0007) [2023-12-27 08:56:54,120][20347] Updated weights for policy 0, policy_version 477572 (0.0008) [2023-12-27 08:56:54,173][20347] Updated weights for policy 0, policy_version 477582 (0.0006) [2023-12-27 08:56:54,226][20347] Updated weights for policy 0, policy_version 477592 (0.0008) [2023-12-27 08:56:54,825][20347] Updated weights for policy 0, policy_version 477602 (0.0009) [2023-12-27 08:56:54,826][20381] Updated weights for policy 1, policy_version 477238 (0.0006) [2023-12-27 08:56:54,874][20347] Updated weights for policy 0, policy_version 477612 (0.0010) [2023-12-27 08:56:54,882][20381] Updated weights for policy 1, policy_version 477248 (0.0009) [2023-12-27 08:56:54,930][20347] Updated weights for policy 0, policy_version 477622 (0.0009) [2023-12-27 08:56:54,934][20381] Updated weights for policy 1, policy_version 477258 (0.0007) [2023-12-27 08:56:54,989][20347] Updated weights for policy 0, policy_version 477632 (0.0006) [2023-12-27 08:56:55,578][20381] Updated weights for policy 1, policy_version 477268 (0.0007) [2023-12-27 08:56:55,624][20381] Updated weights for policy 1, policy_version 477278 (0.0005) [2023-12-27 08:56:55,633][20347] Updated weights for policy 0, policy_version 477642 (0.0010) [2023-12-27 08:56:55,671][20381] Updated weights for policy 1, policy_version 477288 (0.0005) [2023-12-27 08:56:55,681][20347] Updated weights for policy 0, policy_version 477652 (0.0010) [2023-12-27 08:56:55,739][20347] Updated weights for policy 0, policy_version 477662 (0.0010) [2023-12-27 08:56:56,340][20381] Updated weights for policy 1, policy_version 477298 (0.0006) [2023-12-27 08:56:56,389][20381] Updated weights for policy 1, policy_version 477308 (0.0008) [2023-12-27 08:56:56,409][20347] Updated weights for policy 0, policy_version 477672 (0.0010) [2023-12-27 08:56:56,436][20381] Updated weights for policy 1, policy_version 477318 (0.0005) [2023-12-27 08:56:56,466][20347] Updated weights for policy 0, policy_version 477682 (0.0010) [2023-12-27 08:56:56,493][20381] Updated weights for policy 1, policy_version 477328 (0.0005) [2023-12-27 08:56:56,520][20347] Updated weights for policy 0, policy_version 477692 (0.0008) [2023-12-27 08:56:57,184][20381] Updated weights for policy 1, policy_version 477338 (0.0007) [2023-12-27 08:56:57,200][20347] Updated weights for policy 0, policy_version 477702 (0.0009) [2023-12-27 08:56:57,239][20381] Updated weights for policy 1, policy_version 477348 (0.0007) [2023-12-27 08:56:57,256][20347] Updated weights for policy 0, policy_version 477712 (0.0009) [2023-12-27 08:56:57,298][20381] Updated weights for policy 1, policy_version 477358 (0.0006) [2023-12-27 08:56:57,319][20347] Updated weights for policy 0, policy_version 477722 (0.0011) [2023-12-27 08:56:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 19466.4). Total num frames: 244539392. Throughput: 0: 10450.8, 1: 10425.3. Samples: 244522108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:56:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:56:57,963][20347] Updated weights for policy 0, policy_version 477732 (0.0010) [2023-12-27 08:56:58,007][20381] Updated weights for policy 1, policy_version 477368 (0.0007) [2023-12-27 08:56:58,017][20347] Updated weights for policy 0, policy_version 477742 (0.0010) [2023-12-27 08:56:58,065][20381] Updated weights for policy 1, policy_version 477378 (0.0006) [2023-12-27 08:56:58,071][20347] Updated weights for policy 0, policy_version 477752 (0.0010) [2023-12-27 08:56:58,118][20381] Updated weights for policy 1, policy_version 477388 (0.0005) [2023-12-27 08:56:58,746][20347] Updated weights for policy 0, policy_version 477762 (0.0009) [2023-12-27 08:56:58,811][20347] Updated weights for policy 0, policy_version 477772 (0.0007) [2023-12-27 08:56:58,820][20381] Updated weights for policy 1, policy_version 477398 (0.0008) [2023-12-27 08:56:58,868][20347] Updated weights for policy 0, policy_version 477782 (0.0011) [2023-12-27 08:56:58,883][20381] Updated weights for policy 1, policy_version 477408 (0.0008) [2023-12-27 08:56:58,928][20347] Updated weights for policy 0, policy_version 477792 (0.0010) [2023-12-27 08:56:58,945][20381] Updated weights for policy 1, policy_version 477418 (0.0007) [2023-12-27 08:56:59,588][20347] Updated weights for policy 0, policy_version 477802 (0.0005) [2023-12-27 08:56:59,602][20381] Updated weights for policy 1, policy_version 477428 (0.0006) [2023-12-27 08:56:59,650][20347] Updated weights for policy 0, policy_version 477812 (0.0008) [2023-12-27 08:56:59,651][20381] Updated weights for policy 1, policy_version 477438 (0.0007) [2023-12-27 08:56:59,702][20381] Updated weights for policy 1, policy_version 477448 (0.0005) [2023-12-27 08:56:59,709][20347] Updated weights for policy 0, policy_version 477822 (0.0011) [2023-12-27 08:57:00,329][20381] Updated weights for policy 1, policy_version 477458 (0.0007) [2023-12-27 08:57:00,360][20347] Updated weights for policy 0, policy_version 477832 (0.0010) [2023-12-27 08:57:00,378][20381] Updated weights for policy 1, policy_version 477468 (0.0007) [2023-12-27 08:57:00,422][20347] Updated weights for policy 0, policy_version 477842 (0.0011) [2023-12-27 08:57:00,435][20381] Updated weights for policy 1, policy_version 477478 (0.0008) [2023-12-27 08:57:00,476][20347] Updated weights for policy 0, policy_version 477852 (0.0010) [2023-12-27 08:57:00,492][20381] Updated weights for policy 1, policy_version 477488 (0.0005) [2023-12-27 08:57:01,069][20347] Updated weights for policy 0, policy_version 477862 (0.0009) [2023-12-27 08:57:01,119][20347] Updated weights for policy 0, policy_version 477872 (0.0010) [2023-12-27 08:57:01,131][20381] Updated weights for policy 1, policy_version 477498 (0.0010) [2023-12-27 08:57:01,175][20347] Updated weights for policy 0, policy_version 477882 (0.0009) [2023-12-27 08:57:01,192][20381] Updated weights for policy 1, policy_version 477508 (0.0009) [2023-12-27 08:57:01,254][20381] Updated weights for policy 1, policy_version 477518 (0.0010) [2023-12-27 08:57:01,823][20347] Updated weights for policy 0, policy_version 477892 (0.0008) [2023-12-27 08:57:01,876][20347] Updated weights for policy 0, policy_version 477902 (0.0008) [2023-12-27 08:57:01,932][20347] Updated weights for policy 0, policy_version 477912 (0.0005) [2023-12-27 08:57:01,991][20381] Updated weights for policy 1, policy_version 477528 (0.0007) [2023-12-27 08:57:02,056][20381] Updated weights for policy 1, policy_version 477538 (0.0006) [2023-12-27 08:57:02,109][20381] Updated weights for policy 1, policy_version 477548 (0.0011) [2023-12-27 08:57:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 19438.6). Total num frames: 244637696. Throughput: 0: 10491.4, 1: 10408.3. Samples: 244647448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:02,551][20347] Updated weights for policy 0, policy_version 477922 (0.0006) [2023-12-27 08:57:02,600][20347] Updated weights for policy 0, policy_version 477932 (0.0005) [2023-12-27 08:57:02,655][20347] Updated weights for policy 0, policy_version 477942 (0.0005) [2023-12-27 08:57:02,712][20347] Updated weights for policy 0, policy_version 477952 (0.0006) [2023-12-27 08:57:02,785][20381] Updated weights for policy 1, policy_version 477558 (0.0010) [2023-12-27 08:57:02,840][20381] Updated weights for policy 1, policy_version 477568 (0.0010) [2023-12-27 08:57:02,897][20381] Updated weights for policy 1, policy_version 477578 (0.0010) [2023-12-27 08:57:03,352][20347] Updated weights for policy 0, policy_version 477962 (0.0006) [2023-12-27 08:57:03,405][20347] Updated weights for policy 0, policy_version 477972 (0.0007) [2023-12-27 08:57:03,458][20347] Updated weights for policy 0, policy_version 477982 (0.0008) [2023-12-27 08:57:03,474][20381] Updated weights for policy 1, policy_version 477588 (0.0008) [2023-12-27 08:57:03,523][20381] Updated weights for policy 1, policy_version 477598 (0.0005) [2023-12-27 08:57:03,566][20381] Updated weights for policy 1, policy_version 477608 (0.0005) [2023-12-27 08:57:04,102][20347] Updated weights for policy 0, policy_version 477992 (0.0008) [2023-12-27 08:57:04,169][20347] Updated weights for policy 0, policy_version 478002 (0.0008) [2023-12-27 08:57:04,229][20347] Updated weights for policy 0, policy_version 478012 (0.0008) [2023-12-27 08:57:04,229][20381] Updated weights for policy 1, policy_version 477618 (0.0006) [2023-12-27 08:57:04,286][20381] Updated weights for policy 1, policy_version 477628 (0.0009) [2023-12-27 08:57:04,342][20381] Updated weights for policy 1, policy_version 477638 (0.0008) [2023-12-27 08:57:04,401][20381] Updated weights for policy 1, policy_version 477648 (0.0008) [2023-12-27 08:57:04,896][20347] Updated weights for policy 0, policy_version 478022 (0.0007) [2023-12-27 08:57:04,951][20347] Updated weights for policy 0, policy_version 478032 (0.0005) [2023-12-27 08:57:05,009][20347] Updated weights for policy 0, policy_version 478042 (0.0006) [2023-12-27 08:57:05,099][20381] Updated weights for policy 1, policy_version 477658 (0.0005) [2023-12-27 08:57:05,147][20381] Updated weights for policy 1, policy_version 477668 (0.0005) [2023-12-27 08:57:05,200][20381] Updated weights for policy 1, policy_version 477678 (0.0005) [2023-12-27 08:57:05,641][20347] Updated weights for policy 0, policy_version 478052 (0.0008) [2023-12-27 08:57:05,690][20347] Updated weights for policy 0, policy_version 478062 (0.0010) [2023-12-27 08:57:05,742][20347] Updated weights for policy 0, policy_version 478072 (0.0010) [2023-12-27 08:57:05,779][20381] Updated weights for policy 1, policy_version 477688 (0.0008) [2023-12-27 08:57:05,832][20381] Updated weights for policy 1, policy_version 477698 (0.0007) [2023-12-27 08:57:05,889][20381] Updated weights for policy 1, policy_version 477708 (0.0007) [2023-12-27 08:57:06,475][20381] Updated weights for policy 1, policy_version 477718 (0.0006) [2023-12-27 08:57:06,487][20347] Updated weights for policy 0, policy_version 478082 (0.0009) [2023-12-27 08:57:06,529][20381] Updated weights for policy 1, policy_version 477728 (0.0005) [2023-12-27 08:57:06,547][20347] Updated weights for policy 0, policy_version 478092 (0.0008) [2023-12-27 08:57:06,581][20381] Updated weights for policy 1, policy_version 477738 (0.0005) [2023-12-27 08:57:06,607][20347] Updated weights for policy 0, policy_version 478102 (0.0007) [2023-12-27 08:57:06,673][20347] Updated weights for policy 0, policy_version 478112 (0.0007) [2023-12-27 08:57:07,131][20381] Updated weights for policy 1, policy_version 477748 (0.0005) [2023-12-27 08:57:07,180][20381] Updated weights for policy 1, policy_version 477758 (0.0006) [2023-12-27 08:57:07,232][20381] Updated weights for policy 1, policy_version 477768 (0.0007) [2023-12-27 08:57:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.7, 300 sec: 19466.4). Total num frames: 244744192. Throughput: 0: 10515.8, 1: 10394.7. Samples: 244711408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000477776_122331136.pth... [2023-12-27 08:57:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000476560_122019840.pth [2023-12-27 08:57:07,405][20347] Updated weights for policy 0, policy_version 478122 (0.0010) [2023-12-27 08:57:07,458][20347] Updated weights for policy 0, policy_version 478132 (0.0008) [2023-12-27 08:57:07,515][20347] Updated weights for policy 0, policy_version 478142 (0.0005) [2023-12-27 08:57:07,528][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000478144_122421248.pth... [2023-12-27 08:57:07,532][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000476896_122101760.pth [2023-12-27 08:57:07,899][20381] Updated weights for policy 1, policy_version 477778 (0.0008) [2023-12-27 08:57:07,954][20381] Updated weights for policy 1, policy_version 477788 (0.0006) [2023-12-27 08:57:08,018][20381] Updated weights for policy 1, policy_version 477798 (0.0005) [2023-12-27 08:57:08,084][20381] Updated weights for policy 1, policy_version 477808 (0.0005) [2023-12-27 08:57:08,101][20347] Updated weights for policy 0, policy_version 478152 (0.0007) [2023-12-27 08:57:08,168][20347] Updated weights for policy 0, policy_version 478162 (0.0008) [2023-12-27 08:57:08,232][20347] Updated weights for policy 0, policy_version 478172 (0.0008) [2023-12-27 08:57:08,722][20381] Updated weights for policy 1, policy_version 477818 (0.0007) [2023-12-27 08:57:08,779][20381] Updated weights for policy 1, policy_version 477828 (0.0008) [2023-12-27 08:57:08,843][20381] Updated weights for policy 1, policy_version 477838 (0.0007) [2023-12-27 08:57:08,880][20347] Updated weights for policy 0, policy_version 478182 (0.0010) [2023-12-27 08:57:08,943][20347] Updated weights for policy 0, policy_version 478192 (0.0005) [2023-12-27 08:57:09,001][20347] Updated weights for policy 0, policy_version 478202 (0.0005) [2023-12-27 08:57:09,517][20381] Updated weights for policy 1, policy_version 477848 (0.0008) [2023-12-27 08:57:09,578][20381] Updated weights for policy 1, policy_version 477858 (0.0008) [2023-12-27 08:57:09,638][20381] Updated weights for policy 1, policy_version 477868 (0.0007) [2023-12-27 08:57:09,651][20347] Updated weights for policy 0, policy_version 478212 (0.0010) [2023-12-27 08:57:09,714][20347] Updated weights for policy 0, policy_version 478222 (0.0011) [2023-12-27 08:57:09,770][20347] Updated weights for policy 0, policy_version 478232 (0.0010) [2023-12-27 08:57:10,355][20381] Updated weights for policy 1, policy_version 477878 (0.0007) [2023-12-27 08:57:10,393][20347] Updated weights for policy 0, policy_version 478242 (0.0007) [2023-12-27 08:57:10,416][20381] Updated weights for policy 1, policy_version 477888 (0.0008) [2023-12-27 08:57:10,455][20347] Updated weights for policy 0, policy_version 478252 (0.0006) [2023-12-27 08:57:10,473][20381] Updated weights for policy 1, policy_version 477898 (0.0007) [2023-12-27 08:57:10,511][20347] Updated weights for policy 0, policy_version 478262 (0.0007) [2023-12-27 08:57:10,572][20347] Updated weights for policy 0, policy_version 478272 (0.0007) [2023-12-27 08:57:11,191][20381] Updated weights for policy 1, policy_version 477908 (0.0007) [2023-12-27 08:57:11,219][20347] Updated weights for policy 0, policy_version 478282 (0.0011) [2023-12-27 08:57:11,253][20381] Updated weights for policy 1, policy_version 477918 (0.0008) [2023-12-27 08:57:11,292][20347] Updated weights for policy 0, policy_version 478292 (0.0008) [2023-12-27 08:57:11,308][20381] Updated weights for policy 1, policy_version 477928 (0.0007) [2023-12-27 08:57:11,358][20347] Updated weights for policy 0, policy_version 478302 (0.0008) [2023-12-27 08:57:12,066][20347] Updated weights for policy 0, policy_version 478312 (0.0008) [2023-12-27 08:57:12,069][20381] Updated weights for policy 1, policy_version 477938 (0.0007) [2023-12-27 08:57:12,119][20381] Updated weights for policy 1, policy_version 477948 (0.0007) [2023-12-27 08:57:12,119][20347] Updated weights for policy 0, policy_version 478322 (0.0007) [2023-12-27 08:57:12,172][20347] Updated weights for policy 0, policy_version 478332 (0.0005) [2023-12-27 08:57:12,173][20381] Updated weights for policy 1, policy_version 477958 (0.0008) [2023-12-27 08:57:12,219][20381] Updated weights for policy 1, policy_version 477968 (0.0008) [2023-12-27 08:57:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 19494.2). Total num frames: 244850688. Throughput: 0: 10535.4, 1: 10409.9. Samples: 244837972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:12,806][20347] Updated weights for policy 0, policy_version 478342 (0.0006) [2023-12-27 08:57:12,865][20347] Updated weights for policy 0, policy_version 478352 (0.0010) [2023-12-27 08:57:12,921][20347] Updated weights for policy 0, policy_version 478362 (0.0007) [2023-12-27 08:57:12,968][20381] Updated weights for policy 1, policy_version 477978 (0.0006) [2023-12-27 08:57:13,034][20381] Updated weights for policy 1, policy_version 477988 (0.0008) [2023-12-27 08:57:13,088][20381] Updated weights for policy 1, policy_version 477998 (0.0008) [2023-12-27 08:57:13,496][20347] Updated weights for policy 0, policy_version 478372 (0.0009) [2023-12-27 08:57:13,554][20347] Updated weights for policy 0, policy_version 478382 (0.0008) [2023-12-27 08:57:13,605][20347] Updated weights for policy 0, policy_version 478392 (0.0010) [2023-12-27 08:57:13,781][20381] Updated weights for policy 1, policy_version 478008 (0.0008) [2023-12-27 08:57:13,826][20381] Updated weights for policy 1, policy_version 478018 (0.0008) [2023-12-27 08:57:13,885][20381] Updated weights for policy 1, policy_version 478028 (0.0006) [2023-12-27 08:57:14,264][20347] Updated weights for policy 0, policy_version 478402 (0.0010) [2023-12-27 08:57:14,307][20347] Updated weights for policy 0, policy_version 478412 (0.0009) [2023-12-27 08:57:14,358][20347] Updated weights for policy 0, policy_version 478422 (0.0005) [2023-12-27 08:57:14,419][20347] Updated weights for policy 0, policy_version 478432 (0.0007) [2023-12-27 08:57:14,530][20381] Updated weights for policy 1, policy_version 478038 (0.0007) [2023-12-27 08:57:14,581][20381] Updated weights for policy 1, policy_version 478048 (0.0008) [2023-12-27 08:57:14,627][20381] Updated weights for policy 1, policy_version 478058 (0.0007) [2023-12-27 08:57:15,113][20347] Updated weights for policy 0, policy_version 478442 (0.0008) [2023-12-27 08:57:15,174][20347] Updated weights for policy 0, policy_version 478452 (0.0007) [2023-12-27 08:57:15,229][20347] Updated weights for policy 0, policy_version 478462 (0.0008) [2023-12-27 08:57:15,361][20381] Updated weights for policy 1, policy_version 478068 (0.0007) [2023-12-27 08:57:15,419][20381] Updated weights for policy 1, policy_version 478078 (0.0008) [2023-12-27 08:57:15,468][20381] Updated weights for policy 1, policy_version 478088 (0.0007) [2023-12-27 08:57:15,789][20347] Updated weights for policy 0, policy_version 478472 (0.0008) [2023-12-27 08:57:15,838][20347] Updated weights for policy 0, policy_version 478482 (0.0007) [2023-12-27 08:57:15,892][20347] Updated weights for policy 0, policy_version 478492 (0.0008) [2023-12-27 08:57:16,155][20381] Updated weights for policy 1, policy_version 478098 (0.0008) [2023-12-27 08:57:16,208][20381] Updated weights for policy 1, policy_version 478108 (0.0009) [2023-12-27 08:57:16,261][20381] Updated weights for policy 1, policy_version 478118 (0.0008) [2023-12-27 08:57:16,306][20381] Updated weights for policy 1, policy_version 478128 (0.0008) [2023-12-27 08:57:16,588][20347] Updated weights for policy 0, policy_version 478502 (0.0007) [2023-12-27 08:57:16,649][20347] Updated weights for policy 0, policy_version 478512 (0.0007) [2023-12-27 08:57:16,703][20347] Updated weights for policy 0, policy_version 478522 (0.0007) [2023-12-27 08:57:16,994][20381] Updated weights for policy 1, policy_version 478138 (0.0007) [2023-12-27 08:57:17,049][20381] Updated weights for policy 1, policy_version 478148 (0.0011) [2023-12-27 08:57:17,098][20381] Updated weights for policy 1, policy_version 478158 (0.0011) [2023-12-27 08:57:17,331][20347] Updated weights for policy 0, policy_version 478532 (0.0006) [2023-12-27 08:57:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 19549.7). Total num frames: 244948992. Throughput: 0: 10524.8, 1: 10334.2. Samples: 244961228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:17,394][20347] Updated weights for policy 0, policy_version 478542 (0.0007) [2023-12-27 08:57:17,460][20347] Updated weights for policy 0, policy_version 478552 (0.0007) [2023-12-27 08:57:17,738][20381] Updated weights for policy 1, policy_version 478168 (0.0006) [2023-12-27 08:57:17,801][20381] Updated weights for policy 1, policy_version 478178 (0.0009) [2023-12-27 08:57:17,854][20381] Updated weights for policy 1, policy_version 478188 (0.0011) [2023-12-27 08:57:18,068][20347] Updated weights for policy 0, policy_version 478562 (0.0009) [2023-12-27 08:57:18,132][20347] Updated weights for policy 0, policy_version 478572 (0.0007) [2023-12-27 08:57:18,190][20347] Updated weights for policy 0, policy_version 478582 (0.0008) [2023-12-27 08:57:18,251][20347] Updated weights for policy 0, policy_version 478592 (0.0009) [2023-12-27 08:57:18,479][20381] Updated weights for policy 1, policy_version 478198 (0.0008) [2023-12-27 08:57:18,538][20381] Updated weights for policy 1, policy_version 478208 (0.0008) [2023-12-27 08:57:18,599][20381] Updated weights for policy 1, policy_version 478218 (0.0007) [2023-12-27 08:57:18,898][20347] Updated weights for policy 0, policy_version 478602 (0.0007) [2023-12-27 08:57:18,960][20347] Updated weights for policy 0, policy_version 478612 (0.0007) [2023-12-27 08:57:19,006][20347] Updated weights for policy 0, policy_version 478622 (0.0007) [2023-12-27 08:57:19,191][20381] Updated weights for policy 1, policy_version 478228 (0.0007) [2023-12-27 08:57:19,245][20381] Updated weights for policy 1, policy_version 478238 (0.0007) [2023-12-27 08:57:19,303][20381] Updated weights for policy 1, policy_version 478248 (0.0009) [2023-12-27 08:57:19,744][20347] Updated weights for policy 0, policy_version 478632 (0.0008) [2023-12-27 08:57:19,807][20347] Updated weights for policy 0, policy_version 478642 (0.0008) [2023-12-27 08:57:19,870][20347] Updated weights for policy 0, policy_version 478652 (0.0008) [2023-12-27 08:57:19,984][20381] Updated weights for policy 1, policy_version 478258 (0.0010) [2023-12-27 08:57:20,042][20381] Updated weights for policy 1, policy_version 478268 (0.0009) [2023-12-27 08:57:20,093][20381] Updated weights for policy 1, policy_version 478278 (0.0008) [2023-12-27 08:57:20,149][20381] Updated weights for policy 1, policy_version 478288 (0.0008) [2023-12-27 08:57:20,491][20347] Updated weights for policy 0, policy_version 478662 (0.0008) [2023-12-27 08:57:20,549][20347] Updated weights for policy 0, policy_version 478672 (0.0008) [2023-12-27 08:57:20,613][20347] Updated weights for policy 0, policy_version 478682 (0.0007) [2023-12-27 08:57:20,902][20381] Updated weights for policy 1, policy_version 478298 (0.0007) [2023-12-27 08:57:20,962][20381] Updated weights for policy 1, policy_version 478308 (0.0005) [2023-12-27 08:57:21,021][20381] Updated weights for policy 1, policy_version 478318 (0.0007) [2023-12-27 08:57:21,334][20347] Updated weights for policy 0, policy_version 478692 (0.0008) [2023-12-27 08:57:21,405][20347] Updated weights for policy 0, policy_version 478702 (0.0007) [2023-12-27 08:57:21,465][20347] Updated weights for policy 0, policy_version 478712 (0.0007) [2023-12-27 08:57:21,675][20381] Updated weights for policy 1, policy_version 478328 (0.0008) [2023-12-27 08:57:21,741][20381] Updated weights for policy 1, policy_version 478338 (0.0008) [2023-12-27 08:57:21,808][20381] Updated weights for policy 1, policy_version 478348 (0.0007) [2023-12-27 08:57:22,181][20347] Updated weights for policy 0, policy_version 478722 (0.0007) [2023-12-27 08:57:22,227][20347] Updated weights for policy 0, policy_version 478732 (0.0005) [2023-12-27 08:57:22,286][20347] Updated weights for policy 0, policy_version 478742 (0.0006) [2023-12-27 08:57:22,353][20347] Updated weights for policy 0, policy_version 478752 (0.0007) [2023-12-27 08:57:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 19633.0). Total num frames: 245055488. Throughput: 0: 10516.8, 1: 10288.6. Samples: 245023628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000478352_122478592.pth... [2023-12-27 08:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000478752_122576896.pth... [2023-12-27 08:57:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000477168_122175488.pth [2023-12-27 08:57:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000477504_122257408.pth [2023-12-27 08:57:22,449][20381] Updated weights for policy 1, policy_version 478358 (0.0007) [2023-12-27 08:57:22,515][20381] Updated weights for policy 1, policy_version 478368 (0.0008) [2023-12-27 08:57:22,581][20381] Updated weights for policy 1, policy_version 478378 (0.0008) [2023-12-27 08:57:23,008][20347] Updated weights for policy 0, policy_version 478762 (0.0009) [2023-12-27 08:57:23,065][20347] Updated weights for policy 0, policy_version 478772 (0.0008) [2023-12-27 08:57:23,124][20347] Updated weights for policy 0, policy_version 478782 (0.0008) [2023-12-27 08:57:23,201][20381] Updated weights for policy 1, policy_version 478388 (0.0007) [2023-12-27 08:57:23,243][20381] Updated weights for policy 1, policy_version 478398 (0.0007) [2023-12-27 08:57:23,288][20381] Updated weights for policy 1, policy_version 478408 (0.0007) [2023-12-27 08:57:23,833][20347] Updated weights for policy 0, policy_version 478792 (0.0007) [2023-12-27 08:57:23,892][20347] Updated weights for policy 0, policy_version 478802 (0.0006) [2023-12-27 08:57:23,952][20347] Updated weights for policy 0, policy_version 478812 (0.0005) [2023-12-27 08:57:24,041][20381] Updated weights for policy 1, policy_version 478418 (0.0008) [2023-12-27 08:57:24,087][20381] Updated weights for policy 1, policy_version 478428 (0.0008) [2023-12-27 08:57:24,135][20381] Updated weights for policy 1, policy_version 478438 (0.0007) [2023-12-27 08:57:24,184][20381] Updated weights for policy 1, policy_version 478448 (0.0007) [2023-12-27 08:57:24,517][20347] Updated weights for policy 0, policy_version 478822 (0.0008) [2023-12-27 08:57:24,571][20347] Updated weights for policy 0, policy_version 478832 (0.0010) [2023-12-27 08:57:24,621][20347] Updated weights for policy 0, policy_version 478842 (0.0008) [2023-12-27 08:57:24,942][20381] Updated weights for policy 1, policy_version 478458 (0.0008) [2023-12-27 08:57:24,993][20381] Updated weights for policy 1, policy_version 478468 (0.0007) [2023-12-27 08:57:25,040][20381] Updated weights for policy 1, policy_version 478478 (0.0008) [2023-12-27 08:57:25,243][20347] Updated weights for policy 0, policy_version 478852 (0.0007) [2023-12-27 08:57:25,298][20347] Updated weights for policy 0, policy_version 478862 (0.0010) [2023-12-27 08:57:25,358][20347] Updated weights for policy 0, policy_version 478872 (0.0010) [2023-12-27 08:57:25,664][20381] Updated weights for policy 1, policy_version 478488 (0.0008) [2023-12-27 08:57:25,720][20381] Updated weights for policy 1, policy_version 478498 (0.0007) [2023-12-27 08:57:25,777][20381] Updated weights for policy 1, policy_version 478508 (0.0007) [2023-12-27 08:57:26,001][20347] Updated weights for policy 0, policy_version 478882 (0.0010) [2023-12-27 08:57:26,050][20347] Updated weights for policy 0, policy_version 478892 (0.0009) [2023-12-27 08:57:26,111][20347] Updated weights for policy 0, policy_version 478902 (0.0010) [2023-12-27 08:57:26,165][20347] Updated weights for policy 0, policy_version 478912 (0.0010) [2023-12-27 08:57:26,488][20381] Updated weights for policy 1, policy_version 478518 (0.0007) [2023-12-27 08:57:26,540][20381] Updated weights for policy 1, policy_version 478528 (0.0007) [2023-12-27 08:57:26,587][20381] Updated weights for policy 1, policy_version 478538 (0.0008) [2023-12-27 08:57:26,808][20347] Updated weights for policy 0, policy_version 478922 (0.0010) [2023-12-27 08:57:26,862][20347] Updated weights for policy 0, policy_version 478932 (0.0010) [2023-12-27 08:57:26,913][20347] Updated weights for policy 0, policy_version 478942 (0.0009) [2023-12-27 08:57:27,198][20381] Updated weights for policy 1, policy_version 478548 (0.0007) [2023-12-27 08:57:27,246][20381] Updated weights for policy 1, policy_version 478558 (0.0005) [2023-12-27 08:57:27,291][20381] Updated weights for policy 1, policy_version 478568 (0.0006) [2023-12-27 08:57:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 19716.3). Total num frames: 245161984. Throughput: 0: 10541.0, 1: 10306.1. Samples: 245147332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:27,561][20347] Updated weights for policy 0, policy_version 478952 (0.0010) [2023-12-27 08:57:27,610][20347] Updated weights for policy 0, policy_version 478962 (0.0010) [2023-12-27 08:57:27,656][20347] Updated weights for policy 0, policy_version 478972 (0.0010) [2023-12-27 08:57:27,992][20381] Updated weights for policy 1, policy_version 478578 (0.0008) [2023-12-27 08:57:28,042][20381] Updated weights for policy 1, policy_version 478588 (0.0009) [2023-12-27 08:57:28,089][20381] Updated weights for policy 1, policy_version 478598 (0.0010) [2023-12-27 08:57:28,142][20381] Updated weights for policy 1, policy_version 478608 (0.0009) [2023-12-27 08:57:28,245][20347] Updated weights for policy 0, policy_version 478982 (0.0007) [2023-12-27 08:57:28,300][20347] Updated weights for policy 0, policy_version 478992 (0.0005) [2023-12-27 08:57:28,358][20347] Updated weights for policy 0, policy_version 479002 (0.0007) [2023-12-27 08:57:28,878][20381] Updated weights for policy 1, policy_version 478618 (0.0007) [2023-12-27 08:57:28,907][20347] Updated weights for policy 0, policy_version 479012 (0.0006) [2023-12-27 08:57:28,928][20381] Updated weights for policy 1, policy_version 478628 (0.0006) [2023-12-27 08:57:28,965][20347] Updated weights for policy 0, policy_version 479022 (0.0008) [2023-12-27 08:57:28,978][20381] Updated weights for policy 1, policy_version 478638 (0.0008) [2023-12-27 08:57:29,021][20347] Updated weights for policy 0, policy_version 479032 (0.0007) [2023-12-27 08:57:29,661][20347] Updated weights for policy 0, policy_version 479042 (0.0007) [2023-12-27 08:57:29,706][20347] Updated weights for policy 0, policy_version 479052 (0.0008) [2023-12-27 08:57:29,751][20381] Updated weights for policy 1, policy_version 478648 (0.0008) [2023-12-27 08:57:29,760][20347] Updated weights for policy 0, policy_version 479062 (0.0008) [2023-12-27 08:57:29,807][20381] Updated weights for policy 1, policy_version 478658 (0.0006) [2023-12-27 08:57:29,817][20347] Updated weights for policy 0, policy_version 479072 (0.0006) [2023-12-27 08:57:29,869][20381] Updated weights for policy 1, policy_version 478668 (0.0008) [2023-12-27 08:57:30,499][20347] Updated weights for policy 0, policy_version 479082 (0.0010) [2023-12-27 08:57:30,515][20381] Updated weights for policy 1, policy_version 478678 (0.0007) [2023-12-27 08:57:30,547][20347] Updated weights for policy 0, policy_version 479092 (0.0010) [2023-12-27 08:57:30,566][20381] Updated weights for policy 1, policy_version 478688 (0.0007) [2023-12-27 08:57:30,604][20347] Updated weights for policy 0, policy_version 479102 (0.0010) [2023-12-27 08:57:30,611][20381] Updated weights for policy 1, policy_version 478698 (0.0009) [2023-12-27 08:57:31,295][20347] Updated weights for policy 0, policy_version 479112 (0.0007) [2023-12-27 08:57:31,314][20381] Updated weights for policy 1, policy_version 478708 (0.0008) [2023-12-27 08:57:31,359][20347] Updated weights for policy 0, policy_version 479122 (0.0006) [2023-12-27 08:57:31,370][20381] Updated weights for policy 1, policy_version 478718 (0.0007) [2023-12-27 08:57:31,421][20347] Updated weights for policy 0, policy_version 479132 (0.0007) [2023-12-27 08:57:31,427][20381] Updated weights for policy 1, policy_version 478728 (0.0007) [2023-12-27 08:57:32,036][20347] Updated weights for policy 0, policy_version 479142 (0.0005) [2023-12-27 08:57:32,100][20347] Updated weights for policy 0, policy_version 479152 (0.0006) [2023-12-27 08:57:32,108][20381] Updated weights for policy 1, policy_version 478738 (0.0007) [2023-12-27 08:57:32,155][20347] Updated weights for policy 0, policy_version 479162 (0.0007) [2023-12-27 08:57:32,160][20381] Updated weights for policy 1, policy_version 478748 (0.0006) [2023-12-27 08:57:32,212][20381] Updated weights for policy 1, policy_version 478758 (0.0006) [2023-12-27 08:57:32,266][20381] Updated weights for policy 1, policy_version 478768 (0.0007) [2023-12-27 08:57:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 19799.7). Total num frames: 245268480. Throughput: 0: 10576.7, 1: 10276.6. Samples: 245273280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:32,803][20347] Updated weights for policy 0, policy_version 479172 (0.0005) [2023-12-27 08:57:32,853][20347] Updated weights for policy 0, policy_version 479182 (0.0005) [2023-12-27 08:57:32,900][20347] Updated weights for policy 0, policy_version 479192 (0.0005) [2023-12-27 08:57:32,944][20381] Updated weights for policy 1, policy_version 478778 (0.0007) [2023-12-27 08:57:32,994][20381] Updated weights for policy 1, policy_version 478788 (0.0007) [2023-12-27 08:57:33,040][20381] Updated weights for policy 1, policy_version 478798 (0.0010) [2023-12-27 08:57:33,451][20347] Updated weights for policy 0, policy_version 479202 (0.0006) [2023-12-27 08:57:33,494][20347] Updated weights for policy 0, policy_version 479212 (0.0007) [2023-12-27 08:57:33,543][20347] Updated weights for policy 0, policy_version 479222 (0.0007) [2023-12-27 08:57:33,586][20347] Updated weights for policy 0, policy_version 479232 (0.0008) [2023-12-27 08:57:33,708][20381] Updated weights for policy 1, policy_version 478808 (0.0006) [2023-12-27 08:57:33,737][20292] KL-divergence is very high: 377.0671 [2023-12-27 08:57:33,762][20292] KL-divergence is very high: 136.2601 [2023-12-27 08:57:33,768][20381] Updated weights for policy 1, policy_version 478818 (0.0005) [2023-12-27 08:57:33,829][20381] Updated weights for policy 1, policy_version 478828 (0.0007) [2023-12-27 08:57:34,207][20347] Updated weights for policy 0, policy_version 479242 (0.0011) [2023-12-27 08:57:34,263][20347] Updated weights for policy 0, policy_version 479252 (0.0010) [2023-12-27 08:57:34,315][20347] Updated weights for policy 0, policy_version 479262 (0.0010) [2023-12-27 08:57:34,404][20381] Updated weights for policy 1, policy_version 478838 (0.0006) [2023-12-27 08:57:34,462][20381] Updated weights for policy 1, policy_version 478848 (0.0008) [2023-12-27 08:57:34,521][20381] Updated weights for policy 1, policy_version 478858 (0.0008) [2023-12-27 08:57:34,964][20347] Updated weights for policy 0, policy_version 479272 (0.0006) [2023-12-27 08:57:35,022][20347] Updated weights for policy 0, policy_version 479282 (0.0005) [2023-12-27 08:57:35,081][20347] Updated weights for policy 0, policy_version 479292 (0.0006) [2023-12-27 08:57:35,150][20381] Updated weights for policy 1, policy_version 478868 (0.0008) [2023-12-27 08:57:35,206][20381] Updated weights for policy 1, policy_version 478878 (0.0008) [2023-12-27 08:57:35,262][20381] Updated weights for policy 1, policy_version 478888 (0.0008) [2023-12-27 08:57:35,738][20347] Updated weights for policy 0, policy_version 479302 (0.0007) [2023-12-27 08:57:35,787][20347] Updated weights for policy 0, policy_version 479312 (0.0009) [2023-12-27 08:57:35,833][20381] Updated weights for policy 1, policy_version 478898 (0.0008) [2023-12-27 08:57:35,840][20347] Updated weights for policy 0, policy_version 479322 (0.0008) [2023-12-27 08:57:35,884][20381] Updated weights for policy 1, policy_version 478908 (0.0007) [2023-12-27 08:57:35,933][20381] Updated weights for policy 1, policy_version 478918 (0.0008) [2023-12-27 08:57:35,990][20381] Updated weights for policy 1, policy_version 478928 (0.0006) [2023-12-27 08:57:36,507][20347] Updated weights for policy 0, policy_version 479332 (0.0008) [2023-12-27 08:57:36,576][20347] Updated weights for policy 0, policy_version 479342 (0.0010) [2023-12-27 08:57:36,579][20381] Updated weights for policy 1, policy_version 478938 (0.0005) [2023-12-27 08:57:36,634][20347] Updated weights for policy 0, policy_version 479352 (0.0009) [2023-12-27 08:57:36,635][20381] Updated weights for policy 1, policy_version 478948 (0.0007) [2023-12-27 08:57:36,696][20381] Updated weights for policy 1, policy_version 478958 (0.0006) [2023-12-27 08:57:37,216][20347] Updated weights for policy 0, policy_version 479362 (0.0007) [2023-12-27 08:57:37,264][20381] Updated weights for policy 1, policy_version 478968 (0.0007) [2023-12-27 08:57:37,266][20347] Updated weights for policy 0, policy_version 479372 (0.0005) [2023-12-27 08:57:37,316][20381] Updated weights for policy 1, policy_version 478978 (0.0008) [2023-12-27 08:57:37,326][20347] Updated weights for policy 0, policy_version 479382 (0.0007) [2023-12-27 08:57:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 19855.2). Total num frames: 245366784. Throughput: 0: 10608.4, 1: 10311.6. Samples: 245339076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.460')] [2023-12-27 08:57:37,372][20381] Updated weights for policy 1, policy_version 478988 (0.0007) [2023-12-27 08:57:37,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000479392_122740736.pth... [2023-12-27 08:57:37,382][20347] Updated weights for policy 0, policy_version 479392 (0.0006) [2023-12-27 08:57:37,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000478144_122421248.pth [2023-12-27 08:57:37,393][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000478992_122642432.pth... [2023-12-27 08:57:37,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000477776_122331136.pth [2023-12-27 08:57:37,933][20347] Updated weights for policy 0, policy_version 479403 (0.0007) [2023-12-27 08:57:37,989][20347] Updated weights for policy 0, policy_version 479413 (0.0009) [2023-12-27 08:57:38,052][20347] Updated weights for policy 0, policy_version 479423 (0.0005) [2023-12-27 08:57:38,115][20381] Updated weights for policy 1, policy_version 478998 (0.0008) [2023-12-27 08:57:38,170][20381] Updated weights for policy 1, policy_version 479008 (0.0010) [2023-12-27 08:57:38,219][20381] Updated weights for policy 1, policy_version 479018 (0.0010) [2023-12-27 08:57:38,700][20347] Updated weights for policy 0, policy_version 479433 (0.0010) [2023-12-27 08:57:38,755][20347] Updated weights for policy 0, policy_version 479443 (0.0010) [2023-12-27 08:57:38,806][20347] Updated weights for policy 0, policy_version 479453 (0.0010) [2023-12-27 08:57:38,893][20381] Updated weights for policy 1, policy_version 479028 (0.0009) [2023-12-27 08:57:38,949][20381] Updated weights for policy 1, policy_version 479038 (0.0008) [2023-12-27 08:57:39,009][20381] Updated weights for policy 1, policy_version 479048 (0.0008) [2023-12-27 08:57:39,508][20347] Updated weights for policy 0, policy_version 479463 (0.0010) [2023-12-27 08:57:39,571][20347] Updated weights for policy 0, policy_version 479473 (0.0011) [2023-12-27 08:57:39,633][20347] Updated weights for policy 0, policy_version 479483 (0.0010) [2023-12-27 08:57:39,681][20381] Updated weights for policy 1, policy_version 479058 (0.0007) [2023-12-27 08:57:39,738][20381] Updated weights for policy 1, policy_version 479068 (0.0005) [2023-12-27 08:57:39,800][20381] Updated weights for policy 1, policy_version 479078 (0.0005) [2023-12-27 08:57:39,868][20381] Updated weights for policy 1, policy_version 479088 (0.0006) [2023-12-27 08:57:40,301][20347] Updated weights for policy 0, policy_version 479493 (0.0008) [2023-12-27 08:57:40,362][20347] Updated weights for policy 0, policy_version 479503 (0.0008) [2023-12-27 08:57:40,419][20347] Updated weights for policy 0, policy_version 479513 (0.0010) [2023-12-27 08:57:40,519][20381] Updated weights for policy 1, policy_version 479098 (0.0006) [2023-12-27 08:57:40,570][20381] Updated weights for policy 1, policy_version 479108 (0.0009) [2023-12-27 08:57:40,623][20381] Updated weights for policy 1, policy_version 479118 (0.0009) [2023-12-27 08:57:41,022][20347] Updated weights for policy 0, policy_version 479523 (0.0010) [2023-12-27 08:57:41,087][20347] Updated weights for policy 0, policy_version 479533 (0.0010) [2023-12-27 08:57:41,151][20347] Updated weights for policy 0, policy_version 479543 (0.0010) [2023-12-27 08:57:41,433][20381] Updated weights for policy 1, policy_version 479128 (0.0010) [2023-12-27 08:57:41,496][20381] Updated weights for policy 1, policy_version 479138 (0.0010) [2023-12-27 08:57:41,555][20381] Updated weights for policy 1, policy_version 479148 (0.0008) [2023-12-27 08:57:41,877][20347] Updated weights for policy 0, policy_version 479553 (0.0008) [2023-12-27 08:57:41,941][20347] Updated weights for policy 0, policy_version 479563 (0.0008) [2023-12-27 08:57:41,998][20347] Updated weights for policy 0, policy_version 479573 (0.0008) [2023-12-27 08:57:42,056][20347] Updated weights for policy 0, policy_version 479583 (0.0008) [2023-12-27 08:57:42,242][20381] Updated weights for policy 1, policy_version 479158 (0.0008) [2023-12-27 08:57:42,302][20381] Updated weights for policy 1, policy_version 479168 (0.0008) [2023-12-27 08:57:42,362][20381] Updated weights for policy 1, policy_version 479178 (0.0008) [2023-12-27 08:57:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 19883.0). Total num frames: 245473280. Throughput: 0: 10640.2, 1: 10355.1. Samples: 245466896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.460')] [2023-12-27 08:57:42,779][20347] Updated weights for policy 0, policy_version 479593 (0.0007) [2023-12-27 08:57:42,832][20347] Updated weights for policy 0, policy_version 479603 (0.0005) [2023-12-27 08:57:42,886][20347] Updated weights for policy 0, policy_version 479613 (0.0007) [2023-12-27 08:57:43,139][20381] Updated weights for policy 1, policy_version 479188 (0.0009) [2023-12-27 08:57:43,186][20381] Updated weights for policy 1, policy_version 479198 (0.0010) [2023-12-27 08:57:43,238][20381] Updated weights for policy 1, policy_version 479208 (0.0007) [2023-12-27 08:57:43,433][20347] Updated weights for policy 0, policy_version 479623 (0.0008) [2023-12-27 08:57:43,478][20347] Updated weights for policy 0, policy_version 479633 (0.0007) [2023-12-27 08:57:43,523][20347] Updated weights for policy 0, policy_version 479643 (0.0007) [2023-12-27 08:57:43,949][20381] Updated weights for policy 1, policy_version 479218 (0.0007) [2023-12-27 08:57:44,005][20381] Updated weights for policy 1, policy_version 479228 (0.0005) [2023-12-27 08:57:44,048][20381] Updated weights for policy 1, policy_version 479238 (0.0005) [2023-12-27 08:57:44,094][20381] Updated weights for policy 1, policy_version 479248 (0.0005) [2023-12-27 08:57:44,190][20347] Updated weights for policy 0, policy_version 479653 (0.0007) [2023-12-27 08:57:44,249][20347] Updated weights for policy 0, policy_version 479663 (0.0006) [2023-12-27 08:57:44,310][20347] Updated weights for policy 0, policy_version 479673 (0.0006) [2023-12-27 08:57:44,780][20381] Updated weights for policy 1, policy_version 479258 (0.0007) [2023-12-27 08:57:44,842][20381] Updated weights for policy 1, policy_version 479268 (0.0008) [2023-12-27 08:57:44,905][20381] Updated weights for policy 1, policy_version 479278 (0.0007) [2023-12-27 08:57:44,962][20347] Updated weights for policy 0, policy_version 479683 (0.0006) [2023-12-27 08:57:45,024][20347] Updated weights for policy 0, policy_version 479693 (0.0008) [2023-12-27 08:57:45,086][20347] Updated weights for policy 0, policy_version 479703 (0.0007) [2023-12-27 08:57:45,611][20381] Updated weights for policy 1, policy_version 479288 (0.0007) [2023-12-27 08:57:45,670][20381] Updated weights for policy 1, policy_version 479298 (0.0008) [2023-12-27 08:57:45,721][20381] Updated weights for policy 1, policy_version 479308 (0.0006) [2023-12-27 08:57:45,797][20347] Updated weights for policy 0, policy_version 479713 (0.0006) [2023-12-27 08:57:45,848][20347] Updated weights for policy 0, policy_version 479723 (0.0008) [2023-12-27 08:57:45,902][20347] Updated weights for policy 0, policy_version 479733 (0.0008) [2023-12-27 08:57:45,953][20347] Updated weights for policy 0, policy_version 479743 (0.0008) [2023-12-27 08:57:46,376][20381] Updated weights for policy 1, policy_version 479318 (0.0007) [2023-12-27 08:57:46,421][20381] Updated weights for policy 1, policy_version 479328 (0.0008) [2023-12-27 08:57:46,469][20381] Updated weights for policy 1, policy_version 479338 (0.0007) [2023-12-27 08:57:46,653][20347] Updated weights for policy 0, policy_version 479753 (0.0007) [2023-12-27 08:57:46,702][20347] Updated weights for policy 0, policy_version 479763 (0.0007) [2023-12-27 08:57:46,761][20347] Updated weights for policy 0, policy_version 479773 (0.0008) [2023-12-27 08:57:47,150][20381] Updated weights for policy 1, policy_version 479348 (0.0007) [2023-12-27 08:57:47,197][20381] Updated weights for policy 1, policy_version 479358 (0.0009) [2023-12-27 08:57:47,242][20381] Updated weights for policy 1, policy_version 479368 (0.0008) [2023-12-27 08:57:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 19966.3). Total num frames: 245579776. Throughput: 0: 10600.7, 1: 10311.9. Samples: 245588516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:47,384][20347] Updated weights for policy 0, policy_version 479783 (0.0006) [2023-12-27 08:57:47,444][20347] Updated weights for policy 0, policy_version 479793 (0.0007) [2023-12-27 08:57:47,495][20347] Updated weights for policy 0, policy_version 479803 (0.0008) [2023-12-27 08:57:47,971][20381] Updated weights for policy 1, policy_version 479378 (0.0008) [2023-12-27 08:57:48,026][20381] Updated weights for policy 1, policy_version 479388 (0.0008) [2023-12-27 08:57:48,079][20381] Updated weights for policy 1, policy_version 479398 (0.0007) [2023-12-27 08:57:48,080][20347] Updated weights for policy 0, policy_version 479813 (0.0008) [2023-12-27 08:57:48,136][20347] Updated weights for policy 0, policy_version 479823 (0.0010) [2023-12-27 08:57:48,138][20381] Updated weights for policy 1, policy_version 479408 (0.0005) [2023-12-27 08:57:48,196][20347] Updated weights for policy 0, policy_version 479833 (0.0008) [2023-12-27 08:57:48,778][20381] Updated weights for policy 1, policy_version 479418 (0.0010) [2023-12-27 08:57:48,813][20347] Updated weights for policy 0, policy_version 479843 (0.0005) [2023-12-27 08:57:48,834][20381] Updated weights for policy 1, policy_version 479428 (0.0011) [2023-12-27 08:57:48,871][20347] Updated weights for policy 0, policy_version 479853 (0.0010) [2023-12-27 08:57:48,887][20381] Updated weights for policy 1, policy_version 479438 (0.0007) [2023-12-27 08:57:48,927][20347] Updated weights for policy 0, policy_version 479863 (0.0008) [2023-12-27 08:57:49,573][20381] Updated weights for policy 1, policy_version 479448 (0.0009) [2023-12-27 08:57:49,607][20347] Updated weights for policy 0, policy_version 479873 (0.0010) [2023-12-27 08:57:49,622][20381] Updated weights for policy 1, policy_version 479458 (0.0009) [2023-12-27 08:57:49,656][20347] Updated weights for policy 0, policy_version 479883 (0.0010) [2023-12-27 08:57:49,677][20381] Updated weights for policy 1, policy_version 479468 (0.0007) [2023-12-27 08:57:49,714][20347] Updated weights for policy 0, policy_version 479893 (0.0010) [2023-12-27 08:57:49,769][20347] Updated weights for policy 0, policy_version 479903 (0.0010) [2023-12-27 08:57:50,344][20381] Updated weights for policy 1, policy_version 479478 (0.0010) [2023-12-27 08:57:50,400][20381] Updated weights for policy 1, policy_version 479488 (0.0011) [2023-12-27 08:57:50,437][20347] Updated weights for policy 0, policy_version 479913 (0.0006) [2023-12-27 08:57:50,451][20381] Updated weights for policy 1, policy_version 479498 (0.0006) [2023-12-27 08:57:50,492][20347] Updated weights for policy 0, policy_version 479923 (0.0010) [2023-12-27 08:57:50,549][20347] Updated weights for policy 0, policy_version 479933 (0.0010) [2023-12-27 08:57:51,098][20381] Updated weights for policy 1, policy_version 479508 (0.0006) [2023-12-27 08:57:51,166][20381] Updated weights for policy 1, policy_version 479518 (0.0007) [2023-12-27 08:57:51,198][20347] Updated weights for policy 0, policy_version 479943 (0.0007) [2023-12-27 08:57:51,231][20381] Updated weights for policy 1, policy_version 479528 (0.0007) [2023-12-27 08:57:51,259][20347] Updated weights for policy 0, policy_version 479953 (0.0006) [2023-12-27 08:57:51,315][20347] Updated weights for policy 0, policy_version 479963 (0.0007) [2023-12-27 08:57:51,854][20381] Updated weights for policy 1, policy_version 479538 (0.0007) [2023-12-27 08:57:51,910][20381] Updated weights for policy 1, policy_version 479548 (0.0007) [2023-12-27 08:57:51,969][20381] Updated weights for policy 1, policy_version 479558 (0.0007) [2023-12-27 08:57:52,021][20381] Updated weights for policy 1, policy_version 479568 (0.0007) [2023-12-27 08:57:52,026][20347] Updated weights for policy 0, policy_version 479973 (0.0007) [2023-12-27 08:57:52,089][20347] Updated weights for policy 0, policy_version 479983 (0.0005) [2023-12-27 08:57:52,138][20347] Updated weights for policy 0, policy_version 479993 (0.0008) [2023-12-27 08:57:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20049.6). Total num frames: 245686272. Throughput: 0: 10612.5, 1: 10295.6. Samples: 245652276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000479568_122789888.pth... [2023-12-27 08:57:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000480000_122896384.pth... [2023-12-27 08:57:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000478352_122478592.pth [2023-12-27 08:57:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000478752_122576896.pth [2023-12-27 08:57:52,642][20381] Updated weights for policy 1, policy_version 479578 (0.0006) [2023-12-27 08:57:52,702][20381] Updated weights for policy 1, policy_version 479588 (0.0006) [2023-12-27 08:57:52,762][20381] Updated weights for policy 1, policy_version 479598 (0.0005) [2023-12-27 08:57:52,832][20347] Updated weights for policy 0, policy_version 480003 (0.0007) [2023-12-27 08:57:52,896][20347] Updated weights for policy 0, policy_version 480013 (0.0005) [2023-12-27 08:57:52,954][20347] Updated weights for policy 0, policy_version 480023 (0.0006) [2023-12-27 08:57:53,344][20381] Updated weights for policy 1, policy_version 479608 (0.0007) [2023-12-27 08:57:53,406][20381] Updated weights for policy 1, policy_version 479618 (0.0008) [2023-12-27 08:57:53,456][20381] Updated weights for policy 1, policy_version 479628 (0.0007) [2023-12-27 08:57:53,583][20347] Updated weights for policy 0, policy_version 480033 (0.0008) [2023-12-27 08:57:53,638][20347] Updated weights for policy 0, policy_version 480043 (0.0006) [2023-12-27 08:57:53,698][20347] Updated weights for policy 0, policy_version 480053 (0.0008) [2023-12-27 08:57:53,747][20347] Updated weights for policy 0, policy_version 480063 (0.0008) [2023-12-27 08:57:54,180][20381] Updated weights for policy 1, policy_version 479638 (0.0008) [2023-12-27 08:57:54,238][20381] Updated weights for policy 1, policy_version 479648 (0.0007) [2023-12-27 08:57:54,290][20381] Updated weights for policy 1, policy_version 479658 (0.0009) [2023-12-27 08:57:54,323][20347] Updated weights for policy 0, policy_version 480073 (0.0006) [2023-12-27 08:57:54,374][20347] Updated weights for policy 0, policy_version 480083 (0.0007) [2023-12-27 08:57:54,424][20347] Updated weights for policy 0, policy_version 480093 (0.0007) [2023-12-27 08:57:55,015][20381] Updated weights for policy 1, policy_version 479668 (0.0008) [2023-12-27 08:57:55,065][20347] Updated weights for policy 0, policy_version 480103 (0.0006) [2023-12-27 08:57:55,070][20381] Updated weights for policy 1, policy_version 479678 (0.0008) [2023-12-27 08:57:55,125][20347] Updated weights for policy 0, policy_version 480113 (0.0006) [2023-12-27 08:57:55,126][20381] Updated weights for policy 1, policy_version 479688 (0.0007) [2023-12-27 08:57:55,189][20347] Updated weights for policy 0, policy_version 480123 (0.0008) [2023-12-27 08:57:55,835][20381] Updated weights for policy 1, policy_version 479698 (0.0007) [2023-12-27 08:57:55,841][20347] Updated weights for policy 0, policy_version 480134 (0.0007) [2023-12-27 08:57:55,884][20381] Updated weights for policy 1, policy_version 479708 (0.0007) [2023-12-27 08:57:55,895][20347] Updated weights for policy 0, policy_version 480144 (0.0005) [2023-12-27 08:57:55,929][20381] Updated weights for policy 1, policy_version 479718 (0.0007) [2023-12-27 08:57:55,950][20347] Updated weights for policy 0, policy_version 480154 (0.0009) [2023-12-27 08:57:55,978][20381] Updated weights for policy 1, policy_version 479728 (0.0005) [2023-12-27 08:57:56,615][20347] Updated weights for policy 0, policy_version 480164 (0.0010) [2023-12-27 08:57:56,623][20381] Updated weights for policy 1, policy_version 479738 (0.0011) [2023-12-27 08:57:56,667][20381] Updated weights for policy 1, policy_version 479748 (0.0010) [2023-12-27 08:57:56,671][20347] Updated weights for policy 0, policy_version 480174 (0.0010) [2023-12-27 08:57:56,712][20381] Updated weights for policy 1, policy_version 479758 (0.0010) [2023-12-27 08:57:56,728][20347] Updated weights for policy 0, policy_version 480184 (0.0010) [2023-12-27 08:57:57,354][20381] Updated weights for policy 1, policy_version 479768 (0.0007) [2023-12-27 08:57:57,358][20347] Updated weights for policy 0, policy_version 480194 (0.0010) [2023-12-27 08:57:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20105.1). Total num frames: 245784576. Throughput: 0: 10627.0, 1: 10260.0. Samples: 245777888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:57:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:57:57,402][20347] Updated weights for policy 0, policy_version 480204 (0.0010) [2023-12-27 08:57:57,404][20381] Updated weights for policy 1, policy_version 479778 (0.0005) [2023-12-27 08:57:57,451][20347] Updated weights for policy 0, policy_version 480214 (0.0010) [2023-12-27 08:57:57,462][20381] Updated weights for policy 1, policy_version 479788 (0.0007) [2023-12-27 08:57:57,505][20347] Updated weights for policy 0, policy_version 480224 (0.0010) [2023-12-27 08:57:58,138][20381] Updated weights for policy 1, policy_version 479798 (0.0011) [2023-12-27 08:57:58,176][20347] Updated weights for policy 0, policy_version 480234 (0.0010) [2023-12-27 08:57:58,197][20381] Updated weights for policy 1, policy_version 479808 (0.0008) [2023-12-27 08:57:58,240][20347] Updated weights for policy 0, policy_version 480244 (0.0009) [2023-12-27 08:57:58,260][20381] Updated weights for policy 1, policy_version 479818 (0.0008) [2023-12-27 08:57:58,300][20347] Updated weights for policy 0, policy_version 480254 (0.0008) [2023-12-27 08:57:59,041][20381] Updated weights for policy 1, policy_version 479828 (0.0006) [2023-12-27 08:57:59,097][20347] Updated weights for policy 0, policy_version 480264 (0.0009) [2023-12-27 08:57:59,098][20381] Updated weights for policy 1, policy_version 479838 (0.0007) [2023-12-27 08:57:59,155][20347] Updated weights for policy 0, policy_version 480274 (0.0010) [2023-12-27 08:57:59,157][20381] Updated weights for policy 1, policy_version 479848 (0.0006) [2023-12-27 08:57:59,216][20347] Updated weights for policy 0, policy_version 480284 (0.0010) [2023-12-27 08:57:59,826][20347] Updated weights for policy 0, policy_version 480294 (0.0007) [2023-12-27 08:57:59,829][20381] Updated weights for policy 1, policy_version 479858 (0.0007) [2023-12-27 08:57:59,877][20347] Updated weights for policy 0, policy_version 480304 (0.0008) [2023-12-27 08:57:59,886][20381] Updated weights for policy 1, policy_version 479868 (0.0007) [2023-12-27 08:57:59,929][20347] Updated weights for policy 0, policy_version 480314 (0.0008) [2023-12-27 08:57:59,944][20381] Updated weights for policy 1, policy_version 479878 (0.0006) [2023-12-27 08:58:00,004][20381] Updated weights for policy 1, policy_version 479888 (0.0007) [2023-12-27 08:58:00,636][20381] Updated weights for policy 1, policy_version 479898 (0.0007) [2023-12-27 08:58:00,669][20347] Updated weights for policy 0, policy_version 480324 (0.0008) [2023-12-27 08:58:00,690][20381] Updated weights for policy 1, policy_version 479908 (0.0008) [2023-12-27 08:58:00,719][20347] Updated weights for policy 0, policy_version 480334 (0.0006) [2023-12-27 08:58:00,745][20381] Updated weights for policy 1, policy_version 479918 (0.0007) [2023-12-27 08:58:00,767][20347] Updated weights for policy 0, policy_version 480344 (0.0006) [2023-12-27 08:58:01,403][20381] Updated weights for policy 1, policy_version 479928 (0.0007) [2023-12-27 08:58:01,418][20347] Updated weights for policy 0, policy_version 480354 (0.0008) [2023-12-27 08:58:01,460][20381] Updated weights for policy 1, policy_version 479938 (0.0006) [2023-12-27 08:58:01,473][20347] Updated weights for policy 0, policy_version 480364 (0.0010) [2023-12-27 08:58:01,514][20381] Updated weights for policy 1, policy_version 479948 (0.0007) [2023-12-27 08:58:01,532][20347] Updated weights for policy 0, policy_version 480374 (0.0006) [2023-12-27 08:58:01,578][20347] Updated weights for policy 0, policy_version 480384 (0.0009) [2023-12-27 08:58:02,167][20381] Updated weights for policy 1, policy_version 479958 (0.0007) [2023-12-27 08:58:02,219][20381] Updated weights for policy 1, policy_version 479968 (0.0006) [2023-12-27 08:58:02,235][20347] Updated weights for policy 0, policy_version 480394 (0.0010) [2023-12-27 08:58:02,284][20381] Updated weights for policy 1, policy_version 479978 (0.0006) [2023-12-27 08:58:02,297][20347] Updated weights for policy 0, policy_version 480404 (0.0009) [2023-12-27 08:58:02,352][20347] Updated weights for policy 0, policy_version 480414 (0.0007) [2023-12-27 08:58:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20160.6). Total num frames: 245899264. Throughput: 0: 10594.8, 1: 10326.0. Samples: 245902668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:02,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.510')] [2023-12-27 08:58:02,941][20381] Updated weights for policy 1, policy_version 479988 (0.0007) [2023-12-27 08:58:02,973][20347] Updated weights for policy 0, policy_version 480424 (0.0006) [2023-12-27 08:58:02,993][20381] Updated weights for policy 1, policy_version 479998 (0.0007) [2023-12-27 08:58:03,036][20347] Updated weights for policy 0, policy_version 480434 (0.0008) [2023-12-27 08:58:03,050][20381] Updated weights for policy 1, policy_version 480008 (0.0007) [2023-12-27 08:58:03,097][20347] Updated weights for policy 0, policy_version 480444 (0.0007) [2023-12-27 08:58:03,672][20347] Updated weights for policy 0, policy_version 480454 (0.0005) [2023-12-27 08:58:03,726][20347] Updated weights for policy 0, policy_version 480464 (0.0005) [2023-12-27 08:58:03,769][20381] Updated weights for policy 1, policy_version 480018 (0.0008) [2023-12-27 08:58:03,783][20347] Updated weights for policy 0, policy_version 480474 (0.0005) [2023-12-27 08:58:03,814][20381] Updated weights for policy 1, policy_version 480028 (0.0010) [2023-12-27 08:58:03,870][20381] Updated weights for policy 1, policy_version 480038 (0.0011) [2023-12-27 08:58:03,932][20381] Updated weights for policy 1, policy_version 480048 (0.0011) [2023-12-27 08:58:04,406][20347] Updated weights for policy 0, policy_version 480484 (0.0006) [2023-12-27 08:58:04,455][20347] Updated weights for policy 0, policy_version 480494 (0.0007) [2023-12-27 08:58:04,506][20347] Updated weights for policy 0, policy_version 480504 (0.0006) [2023-12-27 08:58:04,595][20381] Updated weights for policy 1, policy_version 480058 (0.0010) [2023-12-27 08:58:04,644][20381] Updated weights for policy 1, policy_version 480068 (0.0010) [2023-12-27 08:58:04,692][20381] Updated weights for policy 1, policy_version 480078 (0.0011) [2023-12-27 08:58:05,041][20347] Updated weights for policy 0, policy_version 480514 (0.0005) [2023-12-27 08:58:05,087][20347] Updated weights for policy 0, policy_version 480524 (0.0005) [2023-12-27 08:58:05,131][20347] Updated weights for policy 0, policy_version 480534 (0.0005) [2023-12-27 08:58:05,182][20347] Updated weights for policy 0, policy_version 480544 (0.0005) [2023-12-27 08:58:05,405][20381] Updated weights for policy 1, policy_version 480088 (0.0010) [2023-12-27 08:58:05,451][20381] Updated weights for policy 1, policy_version 480098 (0.0010) [2023-12-27 08:58:05,503][20381] Updated weights for policy 1, policy_version 480108 (0.0007) [2023-12-27 08:58:05,756][20347] Updated weights for policy 0, policy_version 480554 (0.0005) [2023-12-27 08:58:05,808][20347] Updated weights for policy 0, policy_version 480564 (0.0005) [2023-12-27 08:58:05,859][20347] Updated weights for policy 0, policy_version 480574 (0.0005) [2023-12-27 08:58:06,154][20381] Updated weights for policy 1, policy_version 480118 (0.0008) [2023-12-27 08:58:06,213][20381] Updated weights for policy 1, policy_version 480128 (0.0005) [2023-12-27 08:58:06,266][20381] Updated weights for policy 1, policy_version 480138 (0.0005) [2023-12-27 08:58:06,505][20347] Updated weights for policy 0, policy_version 480584 (0.0009) [2023-12-27 08:58:06,559][20347] Updated weights for policy 0, policy_version 480594 (0.0005) [2023-12-27 08:58:06,618][20347] Updated weights for policy 0, policy_version 480604 (0.0006) [2023-12-27 08:58:06,875][20381] Updated weights for policy 1, policy_version 480148 (0.0005) [2023-12-27 08:58:06,936][20381] Updated weights for policy 1, policy_version 480158 (0.0008) [2023-12-27 08:58:07,001][20381] Updated weights for policy 1, policy_version 480168 (0.0008) [2023-12-27 08:58:07,222][20347] Updated weights for policy 0, policy_version 480614 (0.0008) [2023-12-27 08:58:07,284][20347] Updated weights for policy 0, policy_version 480624 (0.0009) [2023-12-27 08:58:07,338][20347] Updated weights for policy 0, policy_version 480634 (0.0010) [2023-12-27 08:58:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20160.7). Total num frames: 245997568. Throughput: 0: 10681.6, 1: 10297.6. Samples: 245967692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.510')] [2023-12-27 08:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000480176_122945536.pth... [2023-12-27 08:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000480640_123060224.pth... [2023-12-27 08:58:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000478992_122642432.pth [2023-12-27 08:58:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000479392_122740736.pth [2023-12-27 08:58:07,603][20381] Updated weights for policy 1, policy_version 480178 (0.0007) [2023-12-27 08:58:07,663][20381] Updated weights for policy 1, policy_version 480188 (0.0006) [2023-12-27 08:58:07,722][20381] Updated weights for policy 1, policy_version 480198 (0.0006) [2023-12-27 08:58:07,775][20381] Updated weights for policy 1, policy_version 480208 (0.0005) [2023-12-27 08:58:08,004][20347] Updated weights for policy 0, policy_version 480644 (0.0011) [2023-12-27 08:58:08,058][20347] Updated weights for policy 0, policy_version 480654 (0.0010) [2023-12-27 08:58:08,110][20347] Updated weights for policy 0, policy_version 480664 (0.0010) [2023-12-27 08:58:08,376][20381] Updated weights for policy 1, policy_version 480218 (0.0008) [2023-12-27 08:58:08,428][20381] Updated weights for policy 1, policy_version 480228 (0.0007) [2023-12-27 08:58:08,482][20381] Updated weights for policy 1, policy_version 480238 (0.0005) [2023-12-27 08:58:08,812][20347] Updated weights for policy 0, policy_version 480674 (0.0010) [2023-12-27 08:58:08,866][20347] Updated weights for policy 0, policy_version 480684 (0.0010) [2023-12-27 08:58:08,925][20347] Updated weights for policy 0, policy_version 480694 (0.0010) [2023-12-27 08:58:08,980][20347] Updated weights for policy 0, policy_version 480704 (0.0010) [2023-12-27 08:58:09,151][20381] Updated weights for policy 1, policy_version 480248 (0.0007) [2023-12-27 08:58:09,214][20381] Updated weights for policy 1, policy_version 480258 (0.0007) [2023-12-27 08:58:09,277][20381] Updated weights for policy 1, policy_version 480268 (0.0007) [2023-12-27 08:58:09,653][20347] Updated weights for policy 0, policy_version 480714 (0.0010) [2023-12-27 08:58:09,707][20347] Updated weights for policy 0, policy_version 480724 (0.0007) [2023-12-27 08:58:09,764][20347] Updated weights for policy 0, policy_version 480734 (0.0005) [2023-12-27 08:58:10,081][20381] Updated weights for policy 1, policy_version 480278 (0.0010) [2023-12-27 08:58:10,145][20381] Updated weights for policy 1, policy_version 480288 (0.0011) [2023-12-27 08:58:10,207][20381] Updated weights for policy 1, policy_version 480298 (0.0011) [2023-12-27 08:58:10,453][20347] Updated weights for policy 0, policy_version 480744 (0.0008) [2023-12-27 08:58:10,509][20347] Updated weights for policy 0, policy_version 480754 (0.0005) [2023-12-27 08:58:10,564][20347] Updated weights for policy 0, policy_version 480764 (0.0005) [2023-12-27 08:58:10,808][20381] Updated weights for policy 1, policy_version 480308 (0.0010) [2023-12-27 08:58:10,858][20381] Updated weights for policy 1, policy_version 480318 (0.0011) [2023-12-27 08:58:10,908][20381] Updated weights for policy 1, policy_version 480328 (0.0011) [2023-12-27 08:58:11,228][20347] Updated weights for policy 0, policy_version 480774 (0.0008) [2023-12-27 08:58:11,292][20347] Updated weights for policy 0, policy_version 480784 (0.0008) [2023-12-27 08:58:11,355][20347] Updated weights for policy 0, policy_version 480794 (0.0008) [2023-12-27 08:58:11,620][20381] Updated weights for policy 1, policy_version 480338 (0.0010) [2023-12-27 08:58:11,674][20381] Updated weights for policy 1, policy_version 480348 (0.0008) [2023-12-27 08:58:11,737][20381] Updated weights for policy 1, policy_version 480358 (0.0008) [2023-12-27 08:58:11,804][20381] Updated weights for policy 1, policy_version 480368 (0.0008) [2023-12-27 08:58:12,081][20347] Updated weights for policy 0, policy_version 480804 (0.0007) [2023-12-27 08:58:12,134][20347] Updated weights for policy 0, policy_version 480814 (0.0008) [2023-12-27 08:58:12,189][20347] Updated weights for policy 0, policy_version 480824 (0.0007) [2023-12-27 08:58:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20244.0). Total num frames: 246104064. Throughput: 0: 10704.4, 1: 10359.0. Samples: 246095184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 08:58:12,534][20381] Updated weights for policy 1, policy_version 480378 (0.0005) [2023-12-27 08:58:12,594][20381] Updated weights for policy 1, policy_version 480388 (0.0005) [2023-12-27 08:58:12,655][20381] Updated weights for policy 1, policy_version 480398 (0.0006) [2023-12-27 08:58:12,923][20347] Updated weights for policy 0, policy_version 480834 (0.0008) [2023-12-27 08:58:12,984][20347] Updated weights for policy 0, policy_version 480844 (0.0007) [2023-12-27 08:58:13,047][20347] Updated weights for policy 0, policy_version 480854 (0.0007) [2023-12-27 08:58:13,102][20347] Updated weights for policy 0, policy_version 480864 (0.0008) [2023-12-27 08:58:13,313][20381] Updated weights for policy 1, policy_version 480408 (0.0007) [2023-12-27 08:58:13,366][20381] Updated weights for policy 1, policy_version 480418 (0.0007) [2023-12-27 08:58:13,419][20381] Updated weights for policy 1, policy_version 480428 (0.0010) [2023-12-27 08:58:13,753][20347] Updated weights for policy 0, policy_version 480874 (0.0006) [2023-12-27 08:58:13,821][20347] Updated weights for policy 0, policy_version 480884 (0.0007) [2023-12-27 08:58:13,886][20347] Updated weights for policy 0, policy_version 480894 (0.0009) [2023-12-27 08:58:14,011][20381] Updated weights for policy 1, policy_version 480438 (0.0007) [2023-12-27 08:58:14,059][20381] Updated weights for policy 1, policy_version 480448 (0.0005) [2023-12-27 08:58:14,111][20381] Updated weights for policy 1, policy_version 480458 (0.0006) [2023-12-27 08:58:14,497][20347] Updated weights for policy 0, policy_version 480904 (0.0007) [2023-12-27 08:58:14,547][20347] Updated weights for policy 0, policy_version 480914 (0.0009) [2023-12-27 08:58:14,604][20347] Updated weights for policy 0, policy_version 480924 (0.0010) [2023-12-27 08:58:14,686][20381] Updated weights for policy 1, policy_version 480468 (0.0010) [2023-12-27 08:58:14,702][20292] KL-divergence is very high: 181.8820 [2023-12-27 08:58:14,731][20381] Updated weights for policy 1, policy_version 480478 (0.0010) [2023-12-27 08:58:14,789][20381] Updated weights for policy 1, policy_version 480488 (0.0010) [2023-12-27 08:58:15,301][20347] Updated weights for policy 0, policy_version 480934 (0.0009) [2023-12-27 08:58:15,350][20347] Updated weights for policy 0, policy_version 480944 (0.0007) [2023-12-27 08:58:15,404][20347] Updated weights for policy 0, policy_version 480954 (0.0005) [2023-12-27 08:58:15,415][20381] Updated weights for policy 1, policy_version 480498 (0.0006) [2023-12-27 08:58:15,472][20381] Updated weights for policy 1, policy_version 480508 (0.0006) [2023-12-27 08:58:15,533][20381] Updated weights for policy 1, policy_version 480518 (0.0005) [2023-12-27 08:58:15,594][20381] Updated weights for policy 1, policy_version 480528 (0.0005) [2023-12-27 08:58:16,096][20347] Updated weights for policy 0, policy_version 480964 (0.0007) [2023-12-27 08:58:16,134][20381] Updated weights for policy 1, policy_version 480538 (0.0006) [2023-12-27 08:58:16,155][20347] Updated weights for policy 0, policy_version 480974 (0.0008) [2023-12-27 08:58:16,197][20381] Updated weights for policy 1, policy_version 480548 (0.0009) [2023-12-27 08:58:16,215][20347] Updated weights for policy 0, policy_version 480984 (0.0006) [2023-12-27 08:58:16,256][20381] Updated weights for policy 1, policy_version 480558 (0.0006) [2023-12-27 08:58:16,883][20347] Updated weights for policy 0, policy_version 480994 (0.0007) [2023-12-27 08:58:16,916][20381] Updated weights for policy 1, policy_version 480568 (0.0006) [2023-12-27 08:58:16,940][20347] Updated weights for policy 0, policy_version 481004 (0.0006) [2023-12-27 08:58:16,971][20381] Updated weights for policy 1, policy_version 480578 (0.0005) [2023-12-27 08:58:16,990][20347] Updated weights for policy 0, policy_version 481014 (0.0007) [2023-12-27 08:58:17,019][20381] Updated weights for policy 1, policy_version 480588 (0.0005) [2023-12-27 08:58:17,044][20347] Updated weights for policy 0, policy_version 481024 (0.0007) [2023-12-27 08:58:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20327.3). Total num frames: 246210560. Throughput: 0: 10585.7, 1: 10456.5. Samples: 246220180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.290')] [2023-12-27 08:58:17,643][20381] Updated weights for policy 1, policy_version 480598 (0.0005) [2023-12-27 08:58:17,652][20347] Updated weights for policy 0, policy_version 481034 (0.0007) [2023-12-27 08:58:17,688][20381] Updated weights for policy 1, policy_version 480608 (0.0005) [2023-12-27 08:58:17,701][20347] Updated weights for policy 0, policy_version 481044 (0.0007) [2023-12-27 08:58:17,737][20381] Updated weights for policy 1, policy_version 480618 (0.0006) [2023-12-27 08:58:17,755][20347] Updated weights for policy 0, policy_version 481054 (0.0007) [2023-12-27 08:58:18,439][20381] Updated weights for policy 1, policy_version 480628 (0.0006) [2023-12-27 08:58:18,453][20347] Updated weights for policy 0, policy_version 481064 (0.0006) [2023-12-27 08:58:18,502][20381] Updated weights for policy 1, policy_version 480638 (0.0006) [2023-12-27 08:58:18,510][20347] Updated weights for policy 0, policy_version 481074 (0.0007) [2023-12-27 08:58:18,562][20381] Updated weights for policy 1, policy_version 480648 (0.0006) [2023-12-27 08:58:18,564][20347] Updated weights for policy 0, policy_version 481084 (0.0007) [2023-12-27 08:58:19,191][20381] Updated weights for policy 1, policy_version 480658 (0.0007) [2023-12-27 08:58:19,208][20347] Updated weights for policy 0, policy_version 481094 (0.0006) [2023-12-27 08:58:19,251][20381] Updated weights for policy 1, policy_version 480668 (0.0007) [2023-12-27 08:58:19,274][20347] Updated weights for policy 0, policy_version 481104 (0.0007) [2023-12-27 08:58:19,305][20381] Updated weights for policy 1, policy_version 480678 (0.0006) [2023-12-27 08:58:19,334][20347] Updated weights for policy 0, policy_version 481114 (0.0007) [2023-12-27 08:58:19,370][20381] Updated weights for policy 1, policy_version 480688 (0.0006) [2023-12-27 08:58:19,967][20347] Updated weights for policy 0, policy_version 481124 (0.0008) [2023-12-27 08:58:20,026][20347] Updated weights for policy 0, policy_version 481134 (0.0008) [2023-12-27 08:58:20,084][20347] Updated weights for policy 0, policy_version 481144 (0.0007) [2023-12-27 08:58:20,086][20381] Updated weights for policy 1, policy_version 480698 (0.0007) [2023-12-27 08:58:20,148][20381] Updated weights for policy 1, policy_version 480708 (0.0007) [2023-12-27 08:58:20,208][20381] Updated weights for policy 1, policy_version 480718 (0.0008) [2023-12-27 08:58:20,752][20347] Updated weights for policy 0, policy_version 481154 (0.0010) [2023-12-27 08:58:20,815][20347] Updated weights for policy 0, policy_version 481164 (0.0010) [2023-12-27 08:58:20,856][20381] Updated weights for policy 1, policy_version 480728 (0.0006) [2023-12-27 08:58:20,875][20347] Updated weights for policy 0, policy_version 481174 (0.0011) [2023-12-27 08:58:20,912][20381] Updated weights for policy 1, policy_version 480738 (0.0005) [2023-12-27 08:58:20,932][20347] Updated weights for policy 0, policy_version 481184 (0.0010) [2023-12-27 08:58:20,964][20381] Updated weights for policy 1, policy_version 480748 (0.0006) [2023-12-27 08:58:21,633][20347] Updated weights for policy 0, policy_version 481194 (0.0011) [2023-12-27 08:58:21,645][20292] KL-divergence is very high: 387.8586 [2023-12-27 08:58:21,659][20381] Updated weights for policy 1, policy_version 480758 (0.0007) [2023-12-27 08:58:21,672][20292] KL-divergence is very high: 136.9659 [2023-12-27 08:58:21,697][20347] Updated weights for policy 0, policy_version 481204 (0.0008) [2023-12-27 08:58:21,698][20292] KL-divergence is very high: 120.6810 [2023-12-27 08:58:21,725][20381] Updated weights for policy 1, policy_version 480768 (0.0007) [2023-12-27 08:58:21,769][20347] Updated weights for policy 0, policy_version 481214 (0.0007) [2023-12-27 08:58:21,798][20381] Updated weights for policy 1, policy_version 480778 (0.0007) [2023-12-27 08:58:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20382.8). Total num frames: 246308864. Throughput: 0: 10551.4, 1: 10414.5. Samples: 246282540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '942.400')] [2023-12-27 08:58:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000481216_123207680.pth... [2023-12-27 08:58:22,377][20381] Updated weights for policy 1, policy_version 480788 (0.0007) [2023-12-27 08:58:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000480000_122896384.pth [2023-12-27 08:58:22,439][20381] Updated weights for policy 1, policy_version 480798 (0.0007) [2023-12-27 08:58:22,495][20347] Updated weights for policy 0, policy_version 481224 (0.0006) [2023-12-27 08:58:22,498][20381] Updated weights for policy 1, policy_version 480808 (0.0008) [2023-12-27 08:58:22,543][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000480816_123109376.pth... [2023-12-27 08:58:22,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000479568_122789888.pth [2023-12-27 08:58:22,559][20347] Updated weights for policy 0, policy_version 481234 (0.0006) [2023-12-27 08:58:22,621][20347] Updated weights for policy 0, policy_version 481244 (0.0007) [2023-12-27 08:58:23,127][20381] Updated weights for policy 1, policy_version 480818 (0.0009) [2023-12-27 08:58:23,191][20381] Updated weights for policy 1, policy_version 480828 (0.0010) [2023-12-27 08:58:23,252][20381] Updated weights for policy 1, policy_version 480838 (0.0010) [2023-12-27 08:58:23,313][20381] Updated weights for policy 1, policy_version 480848 (0.0010) [2023-12-27 08:58:23,314][20347] Updated weights for policy 0, policy_version 481254 (0.0007) [2023-12-27 08:58:23,361][20347] Updated weights for policy 0, policy_version 481264 (0.0006) [2023-12-27 08:58:23,410][20347] Updated weights for policy 0, policy_version 481274 (0.0005) [2023-12-27 08:58:23,927][20381] Updated weights for policy 1, policy_version 480858 (0.0007) [2023-12-27 08:58:23,983][20381] Updated weights for policy 1, policy_version 480868 (0.0008) [2023-12-27 08:58:24,039][20381] Updated weights for policy 1, policy_version 480878 (0.0005) [2023-12-27 08:58:24,106][20347] Updated weights for policy 0, policy_version 481284 (0.0008) [2023-12-27 08:58:24,151][20347] Updated weights for policy 0, policy_version 481294 (0.0007) [2023-12-27 08:58:24,204][20347] Updated weights for policy 0, policy_version 481304 (0.0009) [2023-12-27 08:58:24,598][20381] Updated weights for policy 1, policy_version 480888 (0.0006) [2023-12-27 08:58:24,637][20292] KL-divergence is very high: 984.9580 [2023-12-27 08:58:24,667][20381] Updated weights for policy 1, policy_version 480898 (0.0009) [2023-12-27 08:58:24,685][20292] KL-divergence is very high: 274.3213 [2023-12-27 08:58:24,712][20381] Updated weights for policy 1, policy_version 480908 (0.0007) [2023-12-27 08:58:24,898][20347] Updated weights for policy 0, policy_version 481314 (0.0007) [2023-12-27 08:58:24,948][20347] Updated weights for policy 0, policy_version 481324 (0.0005) [2023-12-27 08:58:25,009][20347] Updated weights for policy 0, policy_version 481334 (0.0006) [2023-12-27 08:58:25,067][20347] Updated weights for policy 0, policy_version 481344 (0.0008) [2023-12-27 08:58:25,302][20381] Updated weights for policy 1, policy_version 480918 (0.0007) [2023-12-27 08:58:25,352][20381] Updated weights for policy 1, policy_version 480928 (0.0007) [2023-12-27 08:58:25,407][20381] Updated weights for policy 1, policy_version 480938 (0.0008) [2023-12-27 08:58:25,753][20347] Updated weights for policy 0, policy_version 481354 (0.0007) [2023-12-27 08:58:25,798][20347] Updated weights for policy 0, policy_version 481364 (0.0007) [2023-12-27 08:58:25,848][20347] Updated weights for policy 0, policy_version 481374 (0.0008) [2023-12-27 08:58:26,057][20381] Updated weights for policy 1, policy_version 480948 (0.0007) [2023-12-27 08:58:26,110][20381] Updated weights for policy 1, policy_version 480958 (0.0006) [2023-12-27 08:58:26,158][20381] Updated weights for policy 1, policy_version 480968 (0.0005) [2023-12-27 08:58:26,504][20347] Updated weights for policy 0, policy_version 481384 (0.0007) [2023-12-27 08:58:26,551][20347] Updated weights for policy 0, policy_version 481394 (0.0007) [2023-12-27 08:58:26,611][20347] Updated weights for policy 0, policy_version 481404 (0.0007) [2023-12-27 08:58:26,804][20381] Updated weights for policy 1, policy_version 480978 (0.0005) [2023-12-27 08:58:26,862][20381] Updated weights for policy 1, policy_version 480988 (0.0005) [2023-12-27 08:58:26,910][20381] Updated weights for policy 1, policy_version 480998 (0.0006) [2023-12-27 08:58:26,971][20381] Updated weights for policy 1, policy_version 481008 (0.0007) [2023-12-27 08:58:27,279][20347] Updated weights for policy 0, policy_version 481414 (0.0009) [2023-12-27 08:58:27,333][20347] Updated weights for policy 0, policy_version 481424 (0.0009) [2023-12-27 08:58:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20466.1). Total num frames: 246415360. Throughput: 0: 10436.7, 1: 10476.1. Samples: 246407972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '943.270')] [2023-12-27 08:58:27,385][20347] Updated weights for policy 0, policy_version 481434 (0.0009) [2023-12-27 08:58:27,508][20381] Updated weights for policy 1, policy_version 481018 (0.0007) [2023-12-27 08:58:27,558][20381] Updated weights for policy 1, policy_version 481028 (0.0007) [2023-12-27 08:58:27,608][20381] Updated weights for policy 1, policy_version 481038 (0.0007) [2023-12-27 08:58:27,962][20347] Updated weights for policy 0, policy_version 481444 (0.0007) [2023-12-27 08:58:28,033][20347] Updated weights for policy 0, policy_version 481455 (0.0007) [2023-12-27 08:58:28,094][20347] Updated weights for policy 0, policy_version 481465 (0.0005) [2023-12-27 08:58:28,255][20381] Updated weights for policy 1, policy_version 481048 (0.0007) [2023-12-27 08:58:28,310][20381] Updated weights for policy 1, policy_version 481058 (0.0008) [2023-12-27 08:58:28,372][20381] Updated weights for policy 1, policy_version 481068 (0.0007) [2023-12-27 08:58:28,642][20347] Updated weights for policy 0, policy_version 481475 (0.0006) [2023-12-27 08:58:28,695][20347] Updated weights for policy 0, policy_version 481485 (0.0007) [2023-12-27 08:58:28,750][20347] Updated weights for policy 0, policy_version 481495 (0.0006) [2023-12-27 08:58:29,071][20381] Updated weights for policy 1, policy_version 481078 (0.0007) [2023-12-27 08:58:29,124][20381] Updated weights for policy 1, policy_version 481088 (0.0009) [2023-12-27 08:58:29,187][20381] Updated weights for policy 1, policy_version 481098 (0.0005) [2023-12-27 08:58:29,467][20347] Updated weights for policy 0, policy_version 481505 (0.0008) [2023-12-27 08:58:29,522][20347] Updated weights for policy 0, policy_version 481515 (0.0010) [2023-12-27 08:58:29,580][20347] Updated weights for policy 0, policy_version 481525 (0.0010) [2023-12-27 08:58:29,643][20347] Updated weights for policy 0, policy_version 481535 (0.0008) [2023-12-27 08:58:29,880][20381] Updated weights for policy 1, policy_version 481108 (0.0008) [2023-12-27 08:58:29,936][20381] Updated weights for policy 1, policy_version 481118 (0.0011) [2023-12-27 08:58:29,985][20381] Updated weights for policy 1, policy_version 481128 (0.0010) [2023-12-27 08:58:30,291][20347] Updated weights for policy 0, policy_version 481545 (0.0007) [2023-12-27 08:58:30,355][20347] Updated weights for policy 0, policy_version 481555 (0.0007) [2023-12-27 08:58:30,415][20347] Updated weights for policy 0, policy_version 481565 (0.0007) [2023-12-27 08:58:30,661][20381] Updated weights for policy 1, policy_version 481138 (0.0010) [2023-12-27 08:58:30,712][20381] Updated weights for policy 1, policy_version 481148 (0.0010) [2023-12-27 08:58:30,773][20381] Updated weights for policy 1, policy_version 481158 (0.0010) [2023-12-27 08:58:30,836][20381] Updated weights for policy 1, policy_version 481168 (0.0010) [2023-12-27 08:58:31,045][20347] Updated weights for policy 0, policy_version 481575 (0.0007) [2023-12-27 08:58:31,103][20347] Updated weights for policy 0, policy_version 481585 (0.0007) [2023-12-27 08:58:31,161][20347] Updated weights for policy 0, policy_version 481595 (0.0006) [2023-12-27 08:58:31,553][20292] KL-divergence is very high: 620.4797 [2023-12-27 08:58:31,565][20381] Updated weights for policy 1, policy_version 481178 (0.0006) [2023-12-27 08:58:31,627][20381] Updated weights for policy 1, policy_version 481188 (0.0007) [2023-12-27 08:58:31,694][20381] Updated weights for policy 1, policy_version 481198 (0.0007) [2023-12-27 08:58:31,822][20347] Updated weights for policy 0, policy_version 481605 (0.0008) [2023-12-27 08:58:31,876][20347] Updated weights for policy 0, policy_version 481615 (0.0008) [2023-12-27 08:58:31,930][20347] Updated weights for policy 0, policy_version 481625 (0.0008) [2023-12-27 08:58:32,260][20381] Updated weights for policy 1, policy_version 481208 (0.0010) [2023-12-27 08:58:32,318][20381] Updated weights for policy 1, policy_version 481218 (0.0010) [2023-12-27 08:58:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 246521856. Throughput: 0: 10495.2, 1: 10553.2. Samples: 246535696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '925.710')] [2023-12-27 08:58:32,380][20381] Updated weights for policy 1, policy_version 481228 (0.0009) [2023-12-27 08:58:32,584][20347] Updated weights for policy 0, policy_version 481635 (0.0008) [2023-12-27 08:58:32,634][20347] Updated weights for policy 0, policy_version 481645 (0.0006) [2023-12-27 08:58:32,687][20347] Updated weights for policy 0, policy_version 481655 (0.0008) [2023-12-27 08:58:32,987][20381] Updated weights for policy 1, policy_version 481238 (0.0008) [2023-12-27 08:58:32,995][20292] KL-divergence is very high: 629.0388 [2023-12-27 08:58:33,014][20292] KL-divergence is very high: 229.0329 [2023-12-27 08:58:33,052][20381] Updated weights for policy 1, policy_version 481248 (0.0006) [2023-12-27 08:58:33,112][20381] Updated weights for policy 1, policy_version 481258 (0.0005) [2023-12-27 08:58:33,407][20347] Updated weights for policy 0, policy_version 481666 (0.0010) [2023-12-27 08:58:33,455][20347] Updated weights for policy 0, policy_version 481676 (0.0007) [2023-12-27 08:58:33,500][20347] Updated weights for policy 0, policy_version 481686 (0.0007) [2023-12-27 08:58:33,547][20347] Updated weights for policy 0, policy_version 481696 (0.0007) [2023-12-27 08:58:33,681][20381] Updated weights for policy 1, policy_version 481268 (0.0005) [2023-12-27 08:58:33,734][20381] Updated weights for policy 1, policy_version 481278 (0.0005) [2023-12-27 08:58:33,786][20381] Updated weights for policy 1, policy_version 481288 (0.0005) [2023-12-27 08:58:34,205][20347] Updated weights for policy 0, policy_version 481706 (0.0008) [2023-12-27 08:58:34,258][20347] Updated weights for policy 0, policy_version 481716 (0.0005) [2023-12-27 08:58:34,306][20347] Updated weights for policy 0, policy_version 481726 (0.0006) [2023-12-27 08:58:34,378][20381] Updated weights for policy 1, policy_version 481298 (0.0005) [2023-12-27 08:58:34,435][20381] Updated weights for policy 1, policy_version 481308 (0.0007) [2023-12-27 08:58:34,495][20381] Updated weights for policy 1, policy_version 481318 (0.0007) [2023-12-27 08:58:34,553][20381] Updated weights for policy 1, policy_version 481328 (0.0007) [2023-12-27 08:58:34,957][20347] Updated weights for policy 0, policy_version 481736 (0.0008) [2023-12-27 08:58:35,015][20347] Updated weights for policy 0, policy_version 481746 (0.0006) [2023-12-27 08:58:35,076][20347] Updated weights for policy 0, policy_version 481756 (0.0006) [2023-12-27 08:58:35,122][20381] Updated weights for policy 1, policy_version 481338 (0.0006) [2023-12-27 08:58:35,168][20381] Updated weights for policy 1, policy_version 481348 (0.0005) [2023-12-27 08:58:35,215][20381] Updated weights for policy 1, policy_version 481358 (0.0005) [2023-12-27 08:58:35,678][20347] Updated weights for policy 0, policy_version 481766 (0.0008) [2023-12-27 08:58:35,737][20347] Updated weights for policy 0, policy_version 481776 (0.0009) [2023-12-27 08:58:35,792][20347] Updated weights for policy 0, policy_version 481786 (0.0008) [2023-12-27 08:58:35,832][20292] KL-divergence is very high: 100.7262 [2023-12-27 08:58:35,832][20381] Updated weights for policy 1, policy_version 481368 (0.0006) [2023-12-27 08:58:35,878][20381] Updated weights for policy 1, policy_version 481378 (0.0006) [2023-12-27 08:58:35,924][20381] Updated weights for policy 1, policy_version 481388 (0.0005) [2023-12-27 08:58:36,450][20347] Updated weights for policy 0, policy_version 481796 (0.0006) [2023-12-27 08:58:36,513][20347] Updated weights for policy 0, policy_version 481806 (0.0008) [2023-12-27 08:58:36,575][20347] Updated weights for policy 0, policy_version 481816 (0.0008) [2023-12-27 08:58:36,606][20381] Updated weights for policy 1, policy_version 481398 (0.0009) [2023-12-27 08:58:36,665][20381] Updated weights for policy 1, policy_version 481408 (0.0010) [2023-12-27 08:58:36,730][20381] Updated weights for policy 1, policy_version 481418 (0.0010) [2023-12-27 08:58:37,235][20347] Updated weights for policy 0, policy_version 481826 (0.0007) [2023-12-27 08:58:37,284][20347] Updated weights for policy 0, policy_version 481836 (0.0005) [2023-12-27 08:58:37,344][20347] Updated weights for policy 0, policy_version 481846 (0.0007) [2023-12-27 08:58:37,356][20381] Updated weights for policy 1, policy_version 481428 (0.0011) [2023-12-27 08:58:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20632.7). Total num frames: 246628352. Throughput: 0: 10478.2, 1: 10625.7. Samples: 246601952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '904.870')] [2023-12-27 08:58:37,395][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000481856_123371520.pth... [2023-12-27 08:58:37,397][20347] Updated weights for policy 0, policy_version 481856 (0.0006) [2023-12-27 08:58:37,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000480640_123060224.pth [2023-12-27 08:58:37,404][20381] Updated weights for policy 1, policy_version 481438 (0.0008) [2023-12-27 08:58:37,457][20381] Updated weights for policy 1, policy_version 481448 (0.0011) [2023-12-27 08:58:37,500][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000481456_123273216.pth... [2023-12-27 08:58:37,504][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000480176_122945536.pth [2023-12-27 08:58:38,069][20347] Updated weights for policy 0, policy_version 481866 (0.0008) [2023-12-27 08:58:38,107][20381] Updated weights for policy 1, policy_version 481458 (0.0009) [2023-12-27 08:58:38,120][20347] Updated weights for policy 0, policy_version 481876 (0.0010) [2023-12-27 08:58:38,159][20381] Updated weights for policy 1, policy_version 481468 (0.0005) [2023-12-27 08:58:38,171][20347] Updated weights for policy 0, policy_version 481886 (0.0010) [2023-12-27 08:58:38,214][20381] Updated weights for policy 1, policy_version 481478 (0.0006) [2023-12-27 08:58:38,264][20381] Updated weights for policy 1, policy_version 481488 (0.0005) [2023-12-27 08:58:38,808][20347] Updated weights for policy 0, policy_version 481896 (0.0010) [2023-12-27 08:58:38,866][20347] Updated weights for policy 0, policy_version 481906 (0.0010) [2023-12-27 08:58:38,925][20347] Updated weights for policy 0, policy_version 481916 (0.0010) [2023-12-27 08:58:38,963][20381] Updated weights for policy 1, policy_version 481498 (0.0005) [2023-12-27 08:58:39,012][20381] Updated weights for policy 1, policy_version 481508 (0.0005) [2023-12-27 08:58:39,064][20381] Updated weights for policy 1, policy_version 481518 (0.0005) [2023-12-27 08:58:39,628][20347] Updated weights for policy 0, policy_version 481926 (0.0011) [2023-12-27 08:58:39,690][20347] Updated weights for policy 0, policy_version 481936 (0.0009) [2023-12-27 08:58:39,751][20347] Updated weights for policy 0, policy_version 481946 (0.0006) [2023-12-27 08:58:39,751][20381] Updated weights for policy 1, policy_version 481528 (0.0008) [2023-12-27 08:58:39,817][20381] Updated weights for policy 1, policy_version 481538 (0.0008) [2023-12-27 08:58:39,879][20381] Updated weights for policy 1, policy_version 481548 (0.0007) [2023-12-27 08:58:40,422][20347] Updated weights for policy 0, policy_version 481956 (0.0007) [2023-12-27 08:58:40,480][20347] Updated weights for policy 0, policy_version 481966 (0.0008) [2023-12-27 08:58:40,539][20347] Updated weights for policy 0, policy_version 481976 (0.0010) [2023-12-27 08:58:40,614][20381] Updated weights for policy 1, policy_version 481558 (0.0008) [2023-12-27 08:58:40,661][20381] Updated weights for policy 1, policy_version 481568 (0.0007) [2023-12-27 08:58:40,720][20381] Updated weights for policy 1, policy_version 481578 (0.0007) [2023-12-27 08:58:41,225][20347] Updated weights for policy 0, policy_version 481986 (0.0010) [2023-12-27 08:58:41,286][20347] Updated weights for policy 0, policy_version 481996 (0.0011) [2023-12-27 08:58:41,345][20347] Updated weights for policy 0, policy_version 482006 (0.0011) [2023-12-27 08:58:41,408][20347] Updated weights for policy 0, policy_version 482016 (0.0008) [2023-12-27 08:58:41,461][20381] Updated weights for policy 1, policy_version 481588 (0.0007) [2023-12-27 08:58:41,520][20381] Updated weights for policy 1, policy_version 481598 (0.0007) [2023-12-27 08:58:41,539][20292] KL-divergence is very high: 115.5719 [2023-12-27 08:58:41,583][20381] Updated weights for policy 1, policy_version 481608 (0.0008) [2023-12-27 08:58:42,082][20347] Updated weights for policy 0, policy_version 482026 (0.0005) [2023-12-27 08:58:42,146][20347] Updated weights for policy 0, policy_version 482036 (0.0005) [2023-12-27 08:58:42,200][20347] Updated weights for policy 0, policy_version 482046 (0.0005) [2023-12-27 08:58:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 246734848. Throughput: 0: 10447.5, 1: 10612.5. Samples: 246725592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '673.210')] [2023-12-27 08:58:42,371][20381] Updated weights for policy 1, policy_version 481618 (0.0008) [2023-12-27 08:58:42,429][20381] Updated weights for policy 1, policy_version 481628 (0.0011) [2023-12-27 08:58:42,479][20381] Updated weights for policy 1, policy_version 481638 (0.0011) [2023-12-27 08:58:42,532][20381] Updated weights for policy 1, policy_version 481648 (0.0010) [2023-12-27 08:58:42,786][20347] Updated weights for policy 0, policy_version 482056 (0.0007) [2023-12-27 08:58:42,850][20347] Updated weights for policy 0, policy_version 482066 (0.0006) [2023-12-27 08:58:42,921][20347] Updated weights for policy 0, policy_version 482076 (0.0008) [2023-12-27 08:58:43,272][20381] Updated weights for policy 1, policy_version 481658 (0.0007) [2023-12-27 08:58:43,331][20381] Updated weights for policy 1, policy_version 481668 (0.0007) [2023-12-27 08:58:43,382][20381] Updated weights for policy 1, policy_version 481678 (0.0009) [2023-12-27 08:58:43,526][20347] Updated weights for policy 0, policy_version 482086 (0.0008) [2023-12-27 08:58:43,585][20347] Updated weights for policy 0, policy_version 482096 (0.0010) [2023-12-27 08:58:43,645][20347] Updated weights for policy 0, policy_version 482106 (0.0011) [2023-12-27 08:58:44,054][20381] Updated weights for policy 1, policy_version 481688 (0.0008) [2023-12-27 08:58:44,105][20292] KL-divergence is very high: 110.5448 [2023-12-27 08:58:44,112][20381] Updated weights for policy 1, policy_version 481698 (0.0007) [2023-12-27 08:58:44,170][20381] Updated weights for policy 1, policy_version 481708 (0.0008) [2023-12-27 08:58:44,318][20347] Updated weights for policy 0, policy_version 482116 (0.0010) [2023-12-27 08:58:44,373][20347] Updated weights for policy 0, policy_version 482126 (0.0007) [2023-12-27 08:58:44,431][20347] Updated weights for policy 0, policy_version 482136 (0.0006) [2023-12-27 08:58:44,885][20381] Updated weights for policy 1, policy_version 481718 (0.0007) [2023-12-27 08:58:44,915][20292] KL-divergence is very high: 100.3821 [2023-12-27 08:58:44,946][20381] Updated weights for policy 1, policy_version 481728 (0.0007) [2023-12-27 08:58:44,983][20292] KL-divergence is very high: 146.9211 [2023-12-27 08:58:45,007][20381] Updated weights for policy 1, policy_version 481738 (0.0009) [2023-12-27 08:58:45,015][20292] KL-divergence is very high: 243.2867 [2023-12-27 08:58:45,028][20292] KL-divergence is very high: 176.8752 [2023-12-27 08:58:45,034][20292] KL-divergence is very high: 128.8704 [2023-12-27 08:58:45,047][20347] Updated weights for policy 0, policy_version 482146 (0.0010) [2023-12-27 08:58:45,107][20347] Updated weights for policy 0, policy_version 482156 (0.0005) [2023-12-27 08:58:45,161][20347] Updated weights for policy 0, policy_version 482166 (0.0008) [2023-12-27 08:58:45,210][20347] Updated weights for policy 0, policy_version 482176 (0.0010) [2023-12-27 08:58:45,645][20381] Updated weights for policy 1, policy_version 481748 (0.0009) [2023-12-27 08:58:45,688][20292] KL-divergence is very high: 231.3070 [2023-12-27 08:58:45,705][20381] Updated weights for policy 1, policy_version 481758 (0.0011) [2023-12-27 08:58:45,756][20381] Updated weights for policy 1, policy_version 481768 (0.0010) [2023-12-27 08:58:45,774][20292] KL-divergence is very high: 339.1086 [2023-12-27 08:58:45,836][20347] Updated weights for policy 0, policy_version 482186 (0.0005) [2023-12-27 08:58:45,888][20347] Updated weights for policy 0, policy_version 482196 (0.0005) [2023-12-27 08:58:45,953][20347] Updated weights for policy 0, policy_version 482206 (0.0005) [2023-12-27 08:58:46,337][20381] Updated weights for policy 1, policy_version 481778 (0.0008) [2023-12-27 08:58:46,379][20292] KL-divergence is very high: 225.3013 [2023-12-27 08:58:46,384][20292] KL-divergence is very high: 147.7829 [2023-12-27 08:58:46,391][20381] Updated weights for policy 1, policy_version 481788 (0.0005) [2023-12-27 08:58:46,440][20381] Updated weights for policy 1, policy_version 481798 (0.0006) [2023-12-27 08:58:46,488][20347] Updated weights for policy 0, policy_version 482216 (0.0006) [2023-12-27 08:58:46,490][20381] Updated weights for policy 1, policy_version 481808 (0.0009) [2023-12-27 08:58:46,539][20347] Updated weights for policy 0, policy_version 482226 (0.0005) [2023-12-27 08:58:46,602][20347] Updated weights for policy 0, policy_version 482236 (0.0010) [2023-12-27 08:58:47,117][20381] Updated weights for policy 1, policy_version 481818 (0.0006) [2023-12-27 08:58:47,178][20381] Updated weights for policy 1, policy_version 481828 (0.0008) [2023-12-27 08:58:47,227][20381] Updated weights for policy 1, policy_version 481838 (0.0010) [2023-12-27 08:58:47,296][20347] Updated weights for policy 0, policy_version 482246 (0.0009) [2023-12-27 08:58:47,339][20347] Updated weights for policy 0, policy_version 482256 (0.0007) [2023-12-27 08:58:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 246841344. Throughput: 0: 10510.2, 1: 10587.3. Samples: 246852052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '644.610')] [2023-12-27 08:58:47,396][20347] Updated weights for policy 0, policy_version 482266 (0.0008) [2023-12-27 08:58:47,834][20381] Updated weights for policy 1, policy_version 481848 (0.0006) [2023-12-27 08:58:47,883][20381] Updated weights for policy 1, policy_version 481858 (0.0006) [2023-12-27 08:58:47,935][20381] Updated weights for policy 1, policy_version 481868 (0.0005) [2023-12-27 08:58:48,076][20347] Updated weights for policy 0, policy_version 482276 (0.0007) [2023-12-27 08:58:48,123][20347] Updated weights for policy 0, policy_version 482286 (0.0007) [2023-12-27 08:58:48,173][20347] Updated weights for policy 0, policy_version 482296 (0.0007) [2023-12-27 08:58:48,584][20381] Updated weights for policy 1, policy_version 481878 (0.0006) [2023-12-27 08:58:48,633][20381] Updated weights for policy 1, policy_version 481888 (0.0008) [2023-12-27 08:58:48,690][20381] Updated weights for policy 1, policy_version 481898 (0.0008) [2023-12-27 08:58:48,892][20347] Updated weights for policy 0, policy_version 482306 (0.0006) [2023-12-27 08:58:48,951][20347] Updated weights for policy 0, policy_version 482316 (0.0007) [2023-12-27 08:58:49,010][20347] Updated weights for policy 0, policy_version 482326 (0.0007) [2023-12-27 08:58:49,063][20347] Updated weights for policy 0, policy_version 482336 (0.0008) [2023-12-27 08:58:49,388][20292] KL-divergence is very high: 164.7734 [2023-12-27 08:58:49,405][20381] Updated weights for policy 1, policy_version 481908 (0.0008) [2023-12-27 08:58:49,436][20292] KL-divergence is very high: 113.8683 [2023-12-27 08:58:49,466][20381] Updated weights for policy 1, policy_version 481918 (0.0008) [2023-12-27 08:58:49,486][20292] KL-divergence is very high: 123.6210 [2023-12-27 08:58:49,529][20381] Updated weights for policy 1, policy_version 481928 (0.0008) [2023-12-27 08:58:49,535][20292] KL-divergence is very high: 232.9545 [2023-12-27 08:58:49,777][20347] Updated weights for policy 0, policy_version 482346 (0.0007) [2023-12-27 08:58:49,833][20347] Updated weights for policy 0, policy_version 482356 (0.0010) [2023-12-27 08:58:49,893][20347] Updated weights for policy 0, policy_version 482367 (0.0007) [2023-12-27 08:58:50,150][20381] Updated weights for policy 1, policy_version 481938 (0.0007) [2023-12-27 08:58:50,204][20381] Updated weights for policy 1, policy_version 481948 (0.0008) [2023-12-27 08:58:50,260][20381] Updated weights for policy 1, policy_version 481958 (0.0007) [2023-12-27 08:58:50,312][20381] Updated weights for policy 1, policy_version 481968 (0.0007) [2023-12-27 08:58:50,592][20347] Updated weights for policy 0, policy_version 482377 (0.0010) [2023-12-27 08:58:50,655][20347] Updated weights for policy 0, policy_version 482387 (0.0006) [2023-12-27 08:58:50,716][20347] Updated weights for policy 0, policy_version 482397 (0.0008) [2023-12-27 08:58:50,970][20381] Updated weights for policy 1, policy_version 481978 (0.0005) [2023-12-27 08:58:51,033][20381] Updated weights for policy 1, policy_version 481988 (0.0007) [2023-12-27 08:58:51,096][20381] Updated weights for policy 1, policy_version 481998 (0.0006) [2023-12-27 08:58:51,334][20347] Updated weights for policy 0, policy_version 482407 (0.0008) [2023-12-27 08:58:51,403][20347] Updated weights for policy 0, policy_version 482417 (0.0010) [2023-12-27 08:58:51,458][20347] Updated weights for policy 0, policy_version 482427 (0.0010) [2023-12-27 08:58:51,735][20381] Updated weights for policy 1, policy_version 482008 (0.0008) [2023-12-27 08:58:51,749][20292] KL-divergence is very high: 450.1739 [2023-12-27 08:58:51,789][20381] Updated weights for policy 1, policy_version 482018 (0.0009) [2023-12-27 08:58:51,793][20292] KL-divergence is very high: 187.0419 [2023-12-27 08:58:51,842][20381] Updated weights for policy 1, policy_version 482028 (0.0009) [2023-12-27 08:58:52,149][20347] Updated weights for policy 0, policy_version 482437 (0.0009) [2023-12-27 08:58:52,202][20347] Updated weights for policy 0, policy_version 482447 (0.0006) [2023-12-27 08:58:52,269][20347] Updated weights for policy 0, policy_version 482457 (0.0007) [2023-12-27 08:58:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 246947840. Throughput: 0: 10405.3, 1: 10617.5. Samples: 246913720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '428.560')] [2023-12-27 08:58:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000482032_123420672.pth... [2023-12-27 08:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000482464_123527168.pth... [2023-12-27 08:58:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000481216_123207680.pth [2023-12-27 08:58:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000480816_123109376.pth [2023-12-27 08:58:52,479][20292] KL-divergence is very high: 183.3073 [2023-12-27 08:58:52,500][20381] Updated weights for policy 1, policy_version 482038 (0.0011) [2023-12-27 08:58:52,536][20292] KL-divergence is very high: 202.6234 [2023-12-27 08:58:52,541][20292] KL-divergence is very high: 135.8805 [2023-12-27 08:58:52,545][20381] Updated weights for policy 1, policy_version 482048 (0.0007) [2023-12-27 08:58:52,594][20381] Updated weights for policy 1, policy_version 482058 (0.0010) [2023-12-27 08:58:52,947][20347] Updated weights for policy 0, policy_version 482467 (0.0007) [2023-12-27 08:58:52,998][20347] Updated weights for policy 0, policy_version 482477 (0.0007) [2023-12-27 08:58:53,042][20347] Updated weights for policy 0, policy_version 482487 (0.0007) [2023-12-27 08:58:53,250][20292] KL-divergence is very high: 143.4267 [2023-12-27 08:58:53,256][20292] KL-divergence is very high: 139.3137 [2023-12-27 08:58:53,263][20292] KL-divergence is very high: 126.5562 [2023-12-27 08:58:53,270][20292] KL-divergence is very high: 139.0894 [2023-12-27 08:58:53,277][20381] Updated weights for policy 1, policy_version 482068 (0.0009) [2023-12-27 08:58:53,297][20292] KL-divergence is very high: 157.2094 [2023-12-27 08:58:53,302][20292] KL-divergence is very high: 139.2421 [2023-12-27 08:58:53,307][20292] KL-divergence is very high: 101.7283 [2023-12-27 08:58:53,313][20292] KL-divergence is very high: 197.0632 [2023-12-27 08:58:53,328][20381] Updated weights for policy 1, policy_version 482078 (0.0008) [2023-12-27 08:58:53,345][20292] KL-divergence is very high: 138.0481 [2023-12-27 08:58:53,350][20292] KL-divergence is very high: 215.8431 [2023-12-27 08:58:53,356][20292] KL-divergence is very high: 135.0827 [2023-12-27 08:58:53,362][20292] KL-divergence is very high: 177.0558 [2023-12-27 08:58:53,386][20292] KL-divergence is very high: 160.8464 [2023-12-27 08:58:53,387][20381] Updated weights for policy 1, policy_version 482088 (0.0005) [2023-12-27 08:58:53,391][20292] KL-divergence is very high: 166.8002 [2023-12-27 08:58:53,396][20292] KL-divergence is very high: 122.2799 [2023-12-27 08:58:53,401][20292] KL-divergence is very high: 176.2477 [2023-12-27 08:58:53,716][20347] Updated weights for policy 0, policy_version 482497 (0.0007) [2023-12-27 08:58:53,771][20347] Updated weights for policy 0, policy_version 482507 (0.0010) [2023-12-27 08:58:53,815][20347] Updated weights for policy 0, policy_version 482517 (0.0010) [2023-12-27 08:58:53,867][20347] Updated weights for policy 0, policy_version 482527 (0.0010) [2023-12-27 08:58:54,038][20381] Updated weights for policy 1, policy_version 482098 (0.0007) [2023-12-27 08:58:54,083][20381] Updated weights for policy 1, policy_version 482108 (0.0007) [2023-12-27 08:58:54,138][20381] Updated weights for policy 1, policy_version 482118 (0.0007) [2023-12-27 08:58:54,200][20381] Updated weights for policy 1, policy_version 482128 (0.0007) [2023-12-27 08:58:54,568][20347] Updated weights for policy 0, policy_version 482537 (0.0008) [2023-12-27 08:58:54,620][20347] Updated weights for policy 0, policy_version 482547 (0.0007) [2023-12-27 08:58:54,674][20347] Updated weights for policy 0, policy_version 482557 (0.0006) [2023-12-27 08:58:54,873][20381] Updated weights for policy 1, policy_version 482138 (0.0005) [2023-12-27 08:58:54,935][20381] Updated weights for policy 1, policy_version 482148 (0.0006) [2023-12-27 08:58:54,990][20381] Updated weights for policy 1, policy_version 482158 (0.0006) [2023-12-27 08:58:55,358][20347] Updated weights for policy 0, policy_version 482567 (0.0008) [2023-12-27 08:58:55,407][20347] Updated weights for policy 0, policy_version 482577 (0.0009) [2023-12-27 08:58:55,458][20347] Updated weights for policy 0, policy_version 482587 (0.0007) [2023-12-27 08:58:55,496][20381] Updated weights for policy 1, policy_version 482168 (0.0007) [2023-12-27 08:58:55,547][20381] Updated weights for policy 1, policy_version 482178 (0.0008) [2023-12-27 08:58:55,596][20381] Updated weights for policy 1, policy_version 482188 (0.0008) [2023-12-27 08:58:56,141][20347] Updated weights for policy 0, policy_version 482597 (0.0007) [2023-12-27 08:58:56,195][20347] Updated weights for policy 0, policy_version 482607 (0.0008) [2023-12-27 08:58:56,253][20347] Updated weights for policy 0, policy_version 482617 (0.0010) [2023-12-27 08:58:56,319][20381] Updated weights for policy 1, policy_version 482198 (0.0008) [2023-12-27 08:58:56,364][20381] Updated weights for policy 1, policy_version 482208 (0.0008) [2023-12-27 08:58:56,408][20381] Updated weights for policy 1, policy_version 482218 (0.0007) [2023-12-27 08:58:56,928][20347] Updated weights for policy 0, policy_version 482627 (0.0009) [2023-12-27 08:58:56,984][20381] Updated weights for policy 1, policy_version 482228 (0.0007) [2023-12-27 08:58:56,991][20347] Updated weights for policy 0, policy_version 482637 (0.0005) [2023-12-27 08:58:57,037][20381] Updated weights for policy 1, policy_version 482238 (0.0006) [2023-12-27 08:58:57,043][20347] Updated weights for policy 0, policy_version 482647 (0.0007) [2023-12-27 08:58:57,094][20381] Updated weights for policy 1, policy_version 482248 (0.0006) [2023-12-27 08:58:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21162.7, 300 sec: 20854.9). Total num frames: 247054336. Throughput: 0: 10368.2, 1: 10621.2. Samples: 247039704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:58:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '51.140')] [2023-12-27 08:58:57,634][20381] Updated weights for policy 1, policy_version 482258 (0.0005) [2023-12-27 08:58:57,686][20381] Updated weights for policy 1, policy_version 482268 (0.0005) [2023-12-27 08:58:57,706][20347] Updated weights for policy 0, policy_version 482657 (0.0008) [2023-12-27 08:58:57,742][20381] Updated weights for policy 1, policy_version 482278 (0.0005) [2023-12-27 08:58:57,756][20347] Updated weights for policy 0, policy_version 482667 (0.0009) [2023-12-27 08:58:57,793][20381] Updated weights for policy 1, policy_version 482288 (0.0005) [2023-12-27 08:58:57,813][20347] Updated weights for policy 0, policy_version 482677 (0.0008) [2023-12-27 08:58:57,876][20347] Updated weights for policy 0, policy_version 482687 (0.0010) [2023-12-27 08:58:58,458][20381] Updated weights for policy 1, policy_version 482298 (0.0008) [2023-12-27 08:58:58,521][20381] Updated weights for policy 1, policy_version 482308 (0.0008) [2023-12-27 08:58:58,574][20347] Updated weights for policy 0, policy_version 482697 (0.0008) [2023-12-27 08:58:58,585][20381] Updated weights for policy 1, policy_version 482318 (0.0009) [2023-12-27 08:58:58,636][20347] Updated weights for policy 0, policy_version 482707 (0.0008) [2023-12-27 08:58:58,701][20347] Updated weights for policy 0, policy_version 482717 (0.0008) [2023-12-27 08:58:59,362][20381] Updated weights for policy 1, policy_version 482328 (0.0009) [2023-12-27 08:58:59,427][20381] Updated weights for policy 1, policy_version 482338 (0.0006) [2023-12-27 08:58:59,459][20347] Updated weights for policy 0, policy_version 482727 (0.0008) [2023-12-27 08:58:59,478][20381] Updated weights for policy 1, policy_version 482348 (0.0005) [2023-12-27 08:58:59,512][20347] Updated weights for policy 0, policy_version 482737 (0.0007) [2023-12-27 08:58:59,605][20347] Updated weights for policy 0, policy_version 482747 (0.0007) [2023-12-27 08:59:00,080][20381] Updated weights for policy 1, policy_version 482358 (0.0007) [2023-12-27 08:59:00,132][20381] Updated weights for policy 1, policy_version 482368 (0.0008) [2023-12-27 08:59:00,185][20381] Updated weights for policy 1, policy_version 482378 (0.0006) [2023-12-27 08:59:00,300][20347] Updated weights for policy 0, policy_version 482757 (0.0008) [2023-12-27 08:59:00,362][20347] Updated weights for policy 0, policy_version 482767 (0.0008) [2023-12-27 08:59:00,424][20347] Updated weights for policy 0, policy_version 482777 (0.0007) [2023-12-27 08:59:00,770][20381] Updated weights for policy 1, policy_version 482388 (0.0006) [2023-12-27 08:59:00,819][20381] Updated weights for policy 1, policy_version 482398 (0.0007) [2023-12-27 08:59:00,873][20381] Updated weights for policy 1, policy_version 482408 (0.0008) [2023-12-27 08:59:01,036][20347] Updated weights for policy 0, policy_version 482787 (0.0006) [2023-12-27 08:59:01,093][20347] Updated weights for policy 0, policy_version 482797 (0.0007) [2023-12-27 08:59:01,152][20347] Updated weights for policy 0, policy_version 482807 (0.0008) [2023-12-27 08:59:01,537][20381] Updated weights for policy 1, policy_version 482418 (0.0009) [2023-12-27 08:59:01,590][20381] Updated weights for policy 1, policy_version 482428 (0.0009) [2023-12-27 08:59:01,647][20381] Updated weights for policy 1, policy_version 482438 (0.0007) [2023-12-27 08:59:01,702][20381] Updated weights for policy 1, policy_version 482448 (0.0006) [2023-12-27 08:59:01,818][20347] Updated weights for policy 0, policy_version 482817 (0.0007) [2023-12-27 08:59:01,868][20347] Updated weights for policy 0, policy_version 482827 (0.0006) [2023-12-27 08:59:01,915][20347] Updated weights for policy 0, policy_version 482837 (0.0005) [2023-12-27 08:59:01,971][20347] Updated weights for policy 0, policy_version 482847 (0.0005) [2023-12-27 08:59:02,352][20381] Updated weights for policy 1, policy_version 482458 (0.0011) [2023-12-27 08:59:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 247152640. Throughput: 0: 10375.3, 1: 10632.1. Samples: 247165512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '52.220')] [2023-12-27 08:59:02,410][20381] Updated weights for policy 1, policy_version 482468 (0.0010) [2023-12-27 08:59:02,462][20381] Updated weights for policy 1, policy_version 482478 (0.0006) [2023-12-27 08:59:02,572][20347] Updated weights for policy 0, policy_version 482857 (0.0008) [2023-12-27 08:59:02,627][20347] Updated weights for policy 0, policy_version 482867 (0.0010) [2023-12-27 08:59:02,687][20347] Updated weights for policy 0, policy_version 482877 (0.0006) [2023-12-27 08:59:03,023][20381] Updated weights for policy 1, policy_version 482488 (0.0005) [2023-12-27 08:59:03,076][20381] Updated weights for policy 1, policy_version 482498 (0.0009) [2023-12-27 08:59:03,130][20381] Updated weights for policy 1, policy_version 482508 (0.0010) [2023-12-27 08:59:03,243][20347] Updated weights for policy 0, policy_version 482887 (0.0005) [2023-12-27 08:59:03,288][20347] Updated weights for policy 0, policy_version 482897 (0.0005) [2023-12-27 08:59:03,337][20347] Updated weights for policy 0, policy_version 482907 (0.0007) [2023-12-27 08:59:03,790][20381] Updated weights for policy 1, policy_version 482518 (0.0010) [2023-12-27 08:59:03,851][20381] Updated weights for policy 1, policy_version 482528 (0.0008) [2023-12-27 08:59:03,910][20381] Updated weights for policy 1, policy_version 482538 (0.0011) [2023-12-27 08:59:03,987][20347] Updated weights for policy 0, policy_version 482917 (0.0009) [2023-12-27 08:59:04,045][20347] Updated weights for policy 0, policy_version 482927 (0.0007) [2023-12-27 08:59:04,104][20347] Updated weights for policy 0, policy_version 482937 (0.0008) [2023-12-27 08:59:04,591][20381] Updated weights for policy 1, policy_version 482548 (0.0009) [2023-12-27 08:59:04,648][20381] Updated weights for policy 1, policy_version 482558 (0.0008) [2023-12-27 08:59:04,705][20381] Updated weights for policy 1, policy_version 482568 (0.0008) [2023-12-27 08:59:04,817][20347] Updated weights for policy 0, policy_version 482947 (0.0008) [2023-12-27 08:59:04,866][20347] Updated weights for policy 0, policy_version 482957 (0.0005) [2023-12-27 08:59:04,917][20347] Updated weights for policy 0, policy_version 482967 (0.0005) [2023-12-27 08:59:05,410][20381] Updated weights for policy 1, policy_version 482578 (0.0006) [2023-12-27 08:59:05,479][20381] Updated weights for policy 1, policy_version 482588 (0.0006) [2023-12-27 08:59:05,504][20347] Updated weights for policy 0, policy_version 482977 (0.0006) [2023-12-27 08:59:05,539][20381] Updated weights for policy 1, policy_version 482598 (0.0006) [2023-12-27 08:59:05,552][20347] Updated weights for policy 0, policy_version 482987 (0.0010) [2023-12-27 08:59:05,593][20381] Updated weights for policy 1, policy_version 482608 (0.0006) [2023-12-27 08:59:05,602][20347] Updated weights for policy 0, policy_version 482997 (0.0010) [2023-12-27 08:59:05,652][20347] Updated weights for policy 0, policy_version 483007 (0.0010) [2023-12-27 08:59:06,197][20381] Updated weights for policy 1, policy_version 482618 (0.0009) [2023-12-27 08:59:06,259][20381] Updated weights for policy 1, policy_version 482628 (0.0008) [2023-12-27 08:59:06,312][20347] Updated weights for policy 0, policy_version 483017 (0.0010) [2023-12-27 08:59:06,314][20381] Updated weights for policy 1, policy_version 482638 (0.0008) [2023-12-27 08:59:06,373][20347] Updated weights for policy 0, policy_version 483027 (0.0010) [2023-12-27 08:59:06,434][20347] Updated weights for policy 0, policy_version 483037 (0.0010) [2023-12-27 08:59:06,973][20381] Updated weights for policy 1, policy_version 482648 (0.0006) [2023-12-27 08:59:07,022][20381] Updated weights for policy 1, policy_version 482658 (0.0008) [2023-12-27 08:59:07,079][20381] Updated weights for policy 1, policy_version 482668 (0.0008) [2023-12-27 08:59:07,104][20347] Updated weights for policy 0, policy_version 483047 (0.0010) [2023-12-27 08:59:07,156][20347] Updated weights for policy 0, policy_version 483057 (0.0009) [2023-12-27 08:59:07,209][20347] Updated weights for policy 0, policy_version 483067 (0.0010) [2023-12-27 08:59:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21162.7, 300 sec: 20910.4). Total num frames: 247267328. Throughput: 0: 10397.8, 1: 10664.6. Samples: 247230348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '66.230')] [2023-12-27 08:59:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000482672_123584512.pth... [2023-12-27 08:59:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000483072_123682816.pth... [2023-12-27 08:59:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000481456_123273216.pth [2023-12-27 08:59:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000481856_123371520.pth [2023-12-27 08:59:07,762][20381] Updated weights for policy 1, policy_version 482678 (0.0005) [2023-12-27 08:59:07,810][20381] Updated weights for policy 1, policy_version 482688 (0.0005) [2023-12-27 08:59:07,866][20381] Updated weights for policy 1, policy_version 482698 (0.0006) [2023-12-27 08:59:07,878][20347] Updated weights for policy 0, policy_version 483077 (0.0010) [2023-12-27 08:59:07,923][20347] Updated weights for policy 0, policy_version 483087 (0.0010) [2023-12-27 08:59:07,970][20347] Updated weights for policy 0, policy_version 483097 (0.0010) [2023-12-27 08:59:08,530][20381] Updated weights for policy 1, policy_version 482708 (0.0007) [2023-12-27 08:59:08,586][20381] Updated weights for policy 1, policy_version 482718 (0.0008) [2023-12-27 08:59:08,645][20381] Updated weights for policy 1, policy_version 482728 (0.0008) [2023-12-27 08:59:08,691][20347] Updated weights for policy 0, policy_version 483107 (0.0010) [2023-12-27 08:59:08,753][20347] Updated weights for policy 0, policy_version 483117 (0.0007) [2023-12-27 08:59:08,814][20347] Updated weights for policy 0, policy_version 483127 (0.0009) [2023-12-27 08:59:09,373][20381] Updated weights for policy 1, policy_version 482738 (0.0008) [2023-12-27 08:59:09,436][20381] Updated weights for policy 1, policy_version 482748 (0.0010) [2023-12-27 08:59:09,478][20347] Updated weights for policy 0, policy_version 483137 (0.0010) [2023-12-27 08:59:09,496][20381] Updated weights for policy 1, policy_version 482758 (0.0011) [2023-12-27 08:59:09,536][20347] Updated weights for policy 0, policy_version 483147 (0.0006) [2023-12-27 08:59:09,552][20381] Updated weights for policy 1, policy_version 482768 (0.0011) [2023-12-27 08:59:09,592][20347] Updated weights for policy 0, policy_version 483157 (0.0008) [2023-12-27 08:59:09,657][20347] Updated weights for policy 0, policy_version 483167 (0.0007) [2023-12-27 08:59:10,208][20381] Updated weights for policy 1, policy_version 482778 (0.0010) [2023-12-27 08:59:10,261][20381] Updated weights for policy 1, policy_version 482788 (0.0010) [2023-12-27 08:59:10,321][20381] Updated weights for policy 1, policy_version 482798 (0.0009) [2023-12-27 08:59:10,342][20347] Updated weights for policy 0, policy_version 483177 (0.0009) [2023-12-27 08:59:10,407][20347] Updated weights for policy 0, policy_version 483187 (0.0008) [2023-12-27 08:59:10,465][20347] Updated weights for policy 0, policy_version 483197 (0.0008) [2023-12-27 08:59:10,952][20381] Updated weights for policy 1, policy_version 482808 (0.0009) [2023-12-27 08:59:11,021][20381] Updated weights for policy 1, policy_version 482818 (0.0006) [2023-12-27 08:59:11,086][20381] Updated weights for policy 1, policy_version 482828 (0.0009) [2023-12-27 08:59:11,241][20347] Updated weights for policy 0, policy_version 483207 (0.0008) [2023-12-27 08:59:11,304][20347] Updated weights for policy 0, policy_version 483217 (0.0006) [2023-12-27 08:59:11,369][20347] Updated weights for policy 0, policy_version 483227 (0.0007) [2023-12-27 08:59:11,773][20381] Updated weights for policy 1, policy_version 482838 (0.0010) [2023-12-27 08:59:11,829][20381] Updated weights for policy 1, policy_version 482848 (0.0010) [2023-12-27 08:59:11,886][20381] Updated weights for policy 1, policy_version 482858 (0.0009) [2023-12-27 08:59:12,086][20347] Updated weights for policy 0, policy_version 483237 (0.0006) [2023-12-27 08:59:12,138][20347] Updated weights for policy 0, policy_version 483247 (0.0007) [2023-12-27 08:59:12,201][20347] Updated weights for policy 0, policy_version 483257 (0.0007) [2023-12-27 08:59:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 247365632. Throughput: 0: 10450.0, 1: 10578.3. Samples: 247354248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '100.450')] [2023-12-27 08:59:12,587][20381] Updated weights for policy 1, policy_version 482868 (0.0008) [2023-12-27 08:59:12,642][20381] Updated weights for policy 1, policy_version 482878 (0.0011) [2023-12-27 08:59:12,702][20381] Updated weights for policy 1, policy_version 482888 (0.0011) [2023-12-27 08:59:12,883][20347] Updated weights for policy 0, policy_version 483267 (0.0008) [2023-12-27 08:59:12,937][20347] Updated weights for policy 0, policy_version 483277 (0.0010) [2023-12-27 08:59:12,991][20347] Updated weights for policy 0, policy_version 483287 (0.0010) [2023-12-27 08:59:13,381][20381] Updated weights for policy 1, policy_version 482898 (0.0011) [2023-12-27 08:59:13,440][20381] Updated weights for policy 1, policy_version 482908 (0.0011) [2023-12-27 08:59:13,498][20381] Updated weights for policy 1, policy_version 482918 (0.0011) [2023-12-27 08:59:13,553][20381] Updated weights for policy 1, policy_version 482928 (0.0011) [2023-12-27 08:59:13,679][20347] Updated weights for policy 0, policy_version 483297 (0.0010) [2023-12-27 08:59:13,728][20347] Updated weights for policy 0, policy_version 483307 (0.0010) [2023-12-27 08:59:13,781][20347] Updated weights for policy 0, policy_version 483317 (0.0009) [2023-12-27 08:59:13,842][20347] Updated weights for policy 0, policy_version 483327 (0.0007) [2023-12-27 08:59:14,237][20381] Updated weights for policy 1, policy_version 482938 (0.0011) [2023-12-27 08:59:14,292][20381] Updated weights for policy 1, policy_version 482948 (0.0010) [2023-12-27 08:59:14,350][20381] Updated weights for policy 1, policy_version 482958 (0.0011) [2023-12-27 08:59:14,485][20347] Updated weights for policy 0, policy_version 483337 (0.0005) [2023-12-27 08:59:14,539][20347] Updated weights for policy 0, policy_version 483347 (0.0006) [2023-12-27 08:59:14,595][20347] Updated weights for policy 0, policy_version 483357 (0.0008) [2023-12-27 08:59:15,008][20381] Updated weights for policy 1, policy_version 482968 (0.0008) [2023-12-27 08:59:15,069][20381] Updated weights for policy 1, policy_version 482978 (0.0011) [2023-12-27 08:59:15,130][20381] Updated weights for policy 1, policy_version 482988 (0.0011) [2023-12-27 08:59:15,290][20347] Updated weights for policy 0, policy_version 483367 (0.0008) [2023-12-27 08:59:15,350][20347] Updated weights for policy 0, policy_version 483377 (0.0008) [2023-12-27 08:59:15,401][20347] Updated weights for policy 0, policy_version 483387 (0.0007) [2023-12-27 08:59:15,812][20381] Updated weights for policy 1, policy_version 482998 (0.0011) [2023-12-27 08:59:15,861][20381] Updated weights for policy 1, policy_version 483008 (0.0011) [2023-12-27 08:59:15,914][20381] Updated weights for policy 1, policy_version 483018 (0.0007) [2023-12-27 08:59:16,096][20347] Updated weights for policy 0, policy_version 483397 (0.0006) [2023-12-27 08:59:16,150][20347] Updated weights for policy 0, policy_version 483407 (0.0005) [2023-12-27 08:59:16,200][20347] Updated weights for policy 0, policy_version 483417 (0.0005) [2023-12-27 08:59:16,599][20381] Updated weights for policy 1, policy_version 483028 (0.0011) [2023-12-27 08:59:16,645][20381] Updated weights for policy 1, policy_version 483038 (0.0010) [2023-12-27 08:59:16,696][20381] Updated weights for policy 1, policy_version 483048 (0.0011) [2023-12-27 08:59:16,769][20347] Updated weights for policy 0, policy_version 483427 (0.0006) [2023-12-27 08:59:16,820][20347] Updated weights for policy 0, policy_version 483437 (0.0005) [2023-12-27 08:59:16,872][20347] Updated weights for policy 0, policy_version 483447 (0.0005) [2023-12-27 08:59:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 247463936. Throughput: 0: 10401.4, 1: 10513.3. Samples: 247476860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '184.520')] [2023-12-27 08:59:17,403][20381] Updated weights for policy 1, policy_version 483058 (0.0010) [2023-12-27 08:59:17,462][20381] Updated weights for policy 1, policy_version 483068 (0.0011) [2023-12-27 08:59:17,484][20347] Updated weights for policy 0, policy_version 483457 (0.0008) [2023-12-27 08:59:17,517][20381] Updated weights for policy 1, policy_version 483078 (0.0011) [2023-12-27 08:59:17,538][20347] Updated weights for policy 0, policy_version 483467 (0.0007) [2023-12-27 08:59:17,567][20381] Updated weights for policy 1, policy_version 483088 (0.0010) [2023-12-27 08:59:17,599][20347] Updated weights for policy 0, policy_version 483477 (0.0006) [2023-12-27 08:59:17,663][20347] Updated weights for policy 0, policy_version 483487 (0.0007) [2023-12-27 08:59:18,195][20381] Updated weights for policy 1, policy_version 483098 (0.0008) [2023-12-27 08:59:18,258][20381] Updated weights for policy 1, policy_version 483108 (0.0008) [2023-12-27 08:59:18,319][20381] Updated weights for policy 1, policy_version 483118 (0.0008) [2023-12-27 08:59:18,368][20347] Updated weights for policy 0, policy_version 483497 (0.0008) [2023-12-27 08:59:18,421][20347] Updated weights for policy 0, policy_version 483507 (0.0008) [2023-12-27 08:59:18,480][20347] Updated weights for policy 0, policy_version 483517 (0.0007) [2023-12-27 08:59:18,975][20381] Updated weights for policy 1, policy_version 483128 (0.0008) [2023-12-27 08:59:19,030][20381] Updated weights for policy 1, policy_version 483138 (0.0006) [2023-12-27 08:59:19,088][20381] Updated weights for policy 1, policy_version 483148 (0.0006) [2023-12-27 08:59:19,122][20347] Updated weights for policy 0, policy_version 483527 (0.0008) [2023-12-27 08:59:19,167][20347] Updated weights for policy 0, policy_version 483537 (0.0010) [2023-12-27 08:59:19,212][20347] Updated weights for policy 0, policy_version 483547 (0.0010) [2023-12-27 08:59:19,783][20381] Updated weights for policy 1, policy_version 483158 (0.0007) [2023-12-27 08:59:19,848][20381] Updated weights for policy 1, policy_version 483168 (0.0008) [2023-12-27 08:59:19,908][20381] Updated weights for policy 1, policy_version 483178 (0.0008) [2023-12-27 08:59:19,945][20347] Updated weights for policy 0, policy_version 483557 (0.0010) [2023-12-27 08:59:20,009][20347] Updated weights for policy 0, policy_version 483567 (0.0010) [2023-12-27 08:59:20,065][20347] Updated weights for policy 0, policy_version 483577 (0.0010) [2023-12-27 08:59:20,659][20381] Updated weights for policy 1, policy_version 483188 (0.0009) [2023-12-27 08:59:20,689][20347] Updated weights for policy 0, policy_version 483587 (0.0007) [2023-12-27 08:59:20,710][20381] Updated weights for policy 1, policy_version 483198 (0.0006) [2023-12-27 08:59:20,746][20347] Updated weights for policy 0, policy_version 483597 (0.0010) [2023-12-27 08:59:20,759][20381] Updated weights for policy 1, policy_version 483208 (0.0005) [2023-12-27 08:59:20,805][20347] Updated weights for policy 0, policy_version 483607 (0.0010) [2023-12-27 08:59:21,450][20347] Updated weights for policy 0, policy_version 483617 (0.0010) [2023-12-27 08:59:21,507][20347] Updated weights for policy 0, policy_version 483627 (0.0009) [2023-12-27 08:59:21,527][20381] Updated weights for policy 1, policy_version 483218 (0.0007) [2023-12-27 08:59:21,562][20347] Updated weights for policy 0, policy_version 483637 (0.0010) [2023-12-27 08:59:21,583][20381] Updated weights for policy 1, policy_version 483228 (0.0006) [2023-12-27 08:59:21,619][20347] Updated weights for policy 0, policy_version 483647 (0.0010) [2023-12-27 08:59:21,640][20381] Updated weights for policy 1, policy_version 483238 (0.0007) [2023-12-27 08:59:21,703][20381] Updated weights for policy 1, policy_version 483248 (0.0008) [2023-12-27 08:59:22,235][20347] Updated weights for policy 0, policy_version 483657 (0.0006) [2023-12-27 08:59:22,298][20347] Updated weights for policy 0, policy_version 483667 (0.0006) [2023-12-27 08:59:22,360][20347] Updated weights for policy 0, policy_version 483677 (0.0010) [2023-12-27 08:59:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 247562240. Throughput: 0: 10372.8, 1: 10424.4. Samples: 247537828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '103.230')] [2023-12-27 08:59:22,373][20381] Updated weights for policy 1, policy_version 483258 (0.0008) [2023-12-27 08:59:22,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000483680_123838464.pth... [2023-12-27 08:59:22,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000482464_123527168.pth [2023-12-27 08:59:22,430][20381] Updated weights for policy 1, policy_version 483268 (0.0008) [2023-12-27 08:59:22,484][20381] Updated weights for policy 1, policy_version 483278 (0.0007) [2023-12-27 08:59:22,495][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000483280_123740160.pth... [2023-12-27 08:59:22,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000482032_123420672.pth [2023-12-27 08:59:23,051][20347] Updated weights for policy 0, policy_version 483687 (0.0011) [2023-12-27 08:59:23,101][20347] Updated weights for policy 0, policy_version 483697 (0.0008) [2023-12-27 08:59:23,153][20347] Updated weights for policy 0, policy_version 483707 (0.0005) [2023-12-27 08:59:23,199][20381] Updated weights for policy 1, policy_version 483288 (0.0009) [2023-12-27 08:59:23,262][20381] Updated weights for policy 1, policy_version 483298 (0.0006) [2023-12-27 08:59:23,314][20381] Updated weights for policy 1, policy_version 483308 (0.0006) [2023-12-27 08:59:23,796][20347] Updated weights for policy 0, policy_version 483717 (0.0009) [2023-12-27 08:59:23,841][20347] Updated weights for policy 0, policy_version 483727 (0.0010) [2023-12-27 08:59:23,895][20347] Updated weights for policy 0, policy_version 483737 (0.0010) [2023-12-27 08:59:23,968][20381] Updated weights for policy 1, policy_version 483318 (0.0006) [2023-12-27 08:59:24,024][20381] Updated weights for policy 1, policy_version 483328 (0.0005) [2023-12-27 08:59:24,078][20381] Updated weights for policy 1, policy_version 483338 (0.0005) [2023-12-27 08:59:24,508][20347] Updated weights for policy 0, policy_version 483747 (0.0005) [2023-12-27 08:59:24,569][20347] Updated weights for policy 0, policy_version 483757 (0.0007) [2023-12-27 08:59:24,598][20381] Updated weights for policy 1, policy_version 483348 (0.0005) [2023-12-27 08:59:24,629][20347] Updated weights for policy 0, policy_version 483767 (0.0007) [2023-12-27 08:59:24,653][20381] Updated weights for policy 1, policy_version 483358 (0.0005) [2023-12-27 08:59:24,709][20381] Updated weights for policy 1, policy_version 483368 (0.0006) [2023-12-27 08:59:25,264][20381] Updated weights for policy 1, policy_version 483378 (0.0006) [2023-12-27 08:59:25,273][20347] Updated weights for policy 0, policy_version 483777 (0.0007) [2023-12-27 08:59:25,316][20381] Updated weights for policy 1, policy_version 483388 (0.0008) [2023-12-27 08:59:25,332][20347] Updated weights for policy 0, policy_version 483787 (0.0005) [2023-12-27 08:59:25,369][20381] Updated weights for policy 1, policy_version 483398 (0.0007) [2023-12-27 08:59:25,387][20347] Updated weights for policy 0, policy_version 483797 (0.0005) [2023-12-27 08:59:25,420][20381] Updated weights for policy 1, policy_version 483408 (0.0006) [2023-12-27 08:59:25,437][20347] Updated weights for policy 0, policy_version 483807 (0.0007) [2023-12-27 08:59:25,977][20347] Updated weights for policy 0, policy_version 483817 (0.0008) [2023-12-27 08:59:26,025][20347] Updated weights for policy 0, policy_version 483827 (0.0007) [2023-12-27 08:59:26,075][20347] Updated weights for policy 0, policy_version 483837 (0.0007) [2023-12-27 08:59:26,159][20381] Updated weights for policy 1, policy_version 483418 (0.0008) [2023-12-27 08:59:26,215][20381] Updated weights for policy 1, policy_version 483428 (0.0007) [2023-12-27 08:59:26,271][20381] Updated weights for policy 1, policy_version 483438 (0.0008) [2023-12-27 08:59:26,784][20347] Updated weights for policy 0, policy_version 483847 (0.0007) [2023-12-27 08:59:26,831][20347] Updated weights for policy 0, policy_version 483857 (0.0007) [2023-12-27 08:59:26,878][20347] Updated weights for policy 0, policy_version 483867 (0.0007) [2023-12-27 08:59:26,964][20381] Updated weights for policy 1, policy_version 483448 (0.0008) [2023-12-27 08:59:27,016][20381] Updated weights for policy 1, policy_version 483458 (0.0008) [2023-12-27 08:59:27,071][20381] Updated weights for policy 1, policy_version 483468 (0.0007) [2023-12-27 08:59:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 247676928. Throughput: 0: 10437.7, 1: 10441.6. Samples: 247665160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '413.910')] [2023-12-27 08:59:27,528][20347] Updated weights for policy 0, policy_version 483877 (0.0007) [2023-12-27 08:59:27,574][20347] Updated weights for policy 0, policy_version 483887 (0.0008) [2023-12-27 08:59:27,628][20347] Updated weights for policy 0, policy_version 483897 (0.0007) [2023-12-27 08:59:27,732][20381] Updated weights for policy 1, policy_version 483478 (0.0008) [2023-12-27 08:59:27,780][20381] Updated weights for policy 1, policy_version 483488 (0.0008) [2023-12-27 08:59:27,829][20381] Updated weights for policy 1, policy_version 483498 (0.0008) [2023-12-27 08:59:28,307][20347] Updated weights for policy 0, policy_version 483907 (0.0007) [2023-12-27 08:59:28,368][20347] Updated weights for policy 0, policy_version 483917 (0.0005) [2023-12-27 08:59:28,423][20347] Updated weights for policy 0, policy_version 483927 (0.0007) [2023-12-27 08:59:28,514][20381] Updated weights for policy 1, policy_version 483508 (0.0009) [2023-12-27 08:59:28,563][20381] Updated weights for policy 1, policy_version 483518 (0.0009) [2023-12-27 08:59:28,617][20381] Updated weights for policy 1, policy_version 483528 (0.0009) [2023-12-27 08:59:28,996][20347] Updated weights for policy 0, policy_version 483937 (0.0006) [2023-12-27 08:59:29,046][20347] Updated weights for policy 0, policy_version 483947 (0.0005) [2023-12-27 08:59:29,094][20347] Updated weights for policy 0, policy_version 483957 (0.0005) [2023-12-27 08:59:29,140][20347] Updated weights for policy 0, policy_version 483967 (0.0005) [2023-12-27 08:59:29,394][20381] Updated weights for policy 1, policy_version 483538 (0.0008) [2023-12-27 08:59:29,444][20381] Updated weights for policy 1, policy_version 483548 (0.0007) [2023-12-27 08:59:29,501][20381] Updated weights for policy 1, policy_version 483558 (0.0007) [2023-12-27 08:59:29,562][20381] Updated weights for policy 1, policy_version 483568 (0.0007) [2023-12-27 08:59:29,815][20347] Updated weights for policy 0, policy_version 483977 (0.0009) [2023-12-27 08:59:29,878][20347] Updated weights for policy 0, policy_version 483987 (0.0010) [2023-12-27 08:59:29,936][20347] Updated weights for policy 0, policy_version 483997 (0.0010) [2023-12-27 08:59:30,283][20381] Updated weights for policy 1, policy_version 483578 (0.0005) [2023-12-27 08:59:30,340][20381] Updated weights for policy 1, policy_version 483588 (0.0007) [2023-12-27 08:59:30,399][20381] Updated weights for policy 1, policy_version 483598 (0.0010) [2023-12-27 08:59:30,585][20347] Updated weights for policy 0, policy_version 484007 (0.0007) [2023-12-27 08:59:30,641][20347] Updated weights for policy 0, policy_version 484017 (0.0005) [2023-12-27 08:59:30,689][20347] Updated weights for policy 0, policy_version 484027 (0.0005) [2023-12-27 08:59:31,010][20381] Updated weights for policy 1, policy_version 483608 (0.0006) [2023-12-27 08:59:31,068][20381] Updated weights for policy 1, policy_version 483618 (0.0008) [2023-12-27 08:59:31,127][20381] Updated weights for policy 1, policy_version 483628 (0.0007) [2023-12-27 08:59:31,334][20347] Updated weights for policy 0, policy_version 484037 (0.0008) [2023-12-27 08:59:31,395][20347] Updated weights for policy 0, policy_version 484047 (0.0007) [2023-12-27 08:59:31,451][20347] Updated weights for policy 0, policy_version 484057 (0.0007) [2023-12-27 08:59:31,801][20381] Updated weights for policy 1, policy_version 483638 (0.0007) [2023-12-27 08:59:31,858][20381] Updated weights for policy 1, policy_version 483648 (0.0005) [2023-12-27 08:59:31,911][20381] Updated weights for policy 1, policy_version 483658 (0.0007) [2023-12-27 08:59:32,099][20347] Updated weights for policy 0, policy_version 484067 (0.0007) [2023-12-27 08:59:32,151][20347] Updated weights for policy 0, policy_version 484077 (0.0008) [2023-12-27 08:59:32,196][20347] Updated weights for policy 0, policy_version 484087 (0.0006) [2023-12-27 08:59:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.1, 300 sec: 20910.4). Total num frames: 247783424. Throughput: 0: 10421.7, 1: 10436.8. Samples: 247790684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '127.680')] [2023-12-27 08:59:32,494][20381] Updated weights for policy 1, policy_version 483668 (0.0007) [2023-12-27 08:59:32,546][20381] Updated weights for policy 1, policy_version 483678 (0.0005) [2023-12-27 08:59:32,590][20381] Updated weights for policy 1, policy_version 483688 (0.0005) [2023-12-27 08:59:32,940][20347] Updated weights for policy 0, policy_version 484097 (0.0006) [2023-12-27 08:59:32,989][20347] Updated weights for policy 0, policy_version 484107 (0.0010) [2023-12-27 08:59:33,039][20347] Updated weights for policy 0, policy_version 484117 (0.0010) [2023-12-27 08:59:33,087][20347] Updated weights for policy 0, policy_version 484127 (0.0010) [2023-12-27 08:59:33,172][20381] Updated weights for policy 1, policy_version 483698 (0.0009) [2023-12-27 08:59:33,230][20381] Updated weights for policy 1, policy_version 483708 (0.0005) [2023-12-27 08:59:33,292][20381] Updated weights for policy 1, policy_version 483718 (0.0007) [2023-12-27 08:59:33,350][20381] Updated weights for policy 1, policy_version 483728 (0.0007) [2023-12-27 08:59:33,756][20347] Updated weights for policy 0, policy_version 484137 (0.0010) [2023-12-27 08:59:33,813][20347] Updated weights for policy 0, policy_version 484147 (0.0008) [2023-12-27 08:59:33,840][20381] Updated weights for policy 1, policy_version 483738 (0.0009) [2023-12-27 08:59:33,868][20347] Updated weights for policy 0, policy_version 484157 (0.0005) [2023-12-27 08:59:33,895][20381] Updated weights for policy 1, policy_version 483748 (0.0010) [2023-12-27 08:59:33,946][20381] Updated weights for policy 1, policy_version 483758 (0.0010) [2023-12-27 08:59:34,547][20347] Updated weights for policy 0, policy_version 484167 (0.0006) [2023-12-27 08:59:34,557][20381] Updated weights for policy 1, policy_version 483768 (0.0011) [2023-12-27 08:59:34,602][20347] Updated weights for policy 0, policy_version 484177 (0.0007) [2023-12-27 08:59:34,610][20381] Updated weights for policy 1, policy_version 483778 (0.0010) [2023-12-27 08:59:34,657][20347] Updated weights for policy 0, policy_version 484187 (0.0006) [2023-12-27 08:59:34,663][20381] Updated weights for policy 1, policy_version 483788 (0.0010) [2023-12-27 08:59:35,261][20381] Updated weights for policy 1, policy_version 483798 (0.0006) [2023-12-27 08:59:35,319][20381] Updated weights for policy 1, policy_version 483808 (0.0006) [2023-12-27 08:59:35,377][20381] Updated weights for policy 1, policy_version 483818 (0.0007) [2023-12-27 08:59:35,408][20347] Updated weights for policy 0, policy_version 484197 (0.0008) [2023-12-27 08:59:35,470][20347] Updated weights for policy 0, policy_version 484207 (0.0006) [2023-12-27 08:59:35,523][20347] Updated weights for policy 0, policy_version 484217 (0.0005) [2023-12-27 08:59:35,969][20381] Updated weights for policy 1, policy_version 483828 (0.0005) [2023-12-27 08:59:36,034][20381] Updated weights for policy 1, policy_version 483838 (0.0005) [2023-12-27 08:59:36,096][20381] Updated weights for policy 1, policy_version 483848 (0.0005) [2023-12-27 08:59:36,105][20347] Updated weights for policy 0, policy_version 484227 (0.0006) [2023-12-27 08:59:36,162][20347] Updated weights for policy 0, policy_version 484237 (0.0007) [2023-12-27 08:59:36,222][20347] Updated weights for policy 0, policy_version 484247 (0.0006) [2023-12-27 08:59:36,732][20381] Updated weights for policy 1, policy_version 483858 (0.0009) [2023-12-27 08:59:36,788][20381] Updated weights for policy 1, policy_version 483868 (0.0008) [2023-12-27 08:59:36,790][20347] Updated weights for policy 0, policy_version 484257 (0.0006) [2023-12-27 08:59:36,845][20381] Updated weights for policy 1, policy_version 483878 (0.0006) [2023-12-27 08:59:36,854][20347] Updated weights for policy 0, policy_version 484267 (0.0006) [2023-12-27 08:59:36,904][20381] Updated weights for policy 1, policy_version 483888 (0.0005) [2023-12-27 08:59:36,907][20347] Updated weights for policy 0, policy_version 484277 (0.0008) [2023-12-27 08:59:36,963][20347] Updated weights for policy 0, policy_version 484287 (0.0008) [2023-12-27 08:59:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20938.2). Total num frames: 247889920. Throughput: 0: 10426.4, 1: 10524.3. Samples: 247856500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '70.460')] [2023-12-27 08:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000483888_123895808.pth... [2023-12-27 08:59:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000484288_123994112.pth... [2023-12-27 08:59:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000483072_123682816.pth [2023-12-27 08:59:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000482672_123584512.pth [2023-12-27 08:59:37,551][20381] Updated weights for policy 1, policy_version 483898 (0.0008) [2023-12-27 08:59:37,605][20381] Updated weights for policy 1, policy_version 483908 (0.0007) [2023-12-27 08:59:37,655][20381] Updated weights for policy 1, policy_version 483918 (0.0007) [2023-12-27 08:59:37,681][20347] Updated weights for policy 0, policy_version 484297 (0.0007) [2023-12-27 08:59:37,739][20347] Updated weights for policy 0, policy_version 484307 (0.0008) [2023-12-27 08:59:37,793][20347] Updated weights for policy 0, policy_version 484317 (0.0007) [2023-12-27 08:59:38,342][20381] Updated weights for policy 1, policy_version 483928 (0.0008) [2023-12-27 08:59:38,401][20381] Updated weights for policy 1, policy_version 483938 (0.0007) [2023-12-27 08:59:38,446][20347] Updated weights for policy 0, policy_version 484327 (0.0008) [2023-12-27 08:59:38,454][20381] Updated weights for policy 1, policy_version 483948 (0.0010) [2023-12-27 08:59:38,513][20347] Updated weights for policy 0, policy_version 484337 (0.0007) [2023-12-27 08:59:38,570][20347] Updated weights for policy 0, policy_version 484347 (0.0007) [2023-12-27 08:59:39,116][20381] Updated weights for policy 1, policy_version 483958 (0.0010) [2023-12-27 08:59:39,179][20381] Updated weights for policy 1, policy_version 483968 (0.0007) [2023-12-27 08:59:39,209][20347] Updated weights for policy 0, policy_version 484357 (0.0008) [2023-12-27 08:59:39,245][20381] Updated weights for policy 1, policy_version 483978 (0.0008) [2023-12-27 08:59:39,272][20347] Updated weights for policy 0, policy_version 484367 (0.0009) [2023-12-27 08:59:39,329][20347] Updated weights for policy 0, policy_version 484377 (0.0007) [2023-12-27 08:59:39,993][20381] Updated weights for policy 1, policy_version 483988 (0.0009) [2023-12-27 08:59:40,011][20347] Updated weights for policy 0, policy_version 484387 (0.0007) [2023-12-27 08:59:40,054][20381] Updated weights for policy 1, policy_version 483998 (0.0009) [2023-12-27 08:59:40,067][20347] Updated weights for policy 0, policy_version 484397 (0.0009) [2023-12-27 08:59:40,111][20381] Updated weights for policy 1, policy_version 484008 (0.0009) [2023-12-27 08:59:40,120][20347] Updated weights for policy 0, policy_version 484407 (0.0009) [2023-12-27 08:59:40,738][20381] Updated weights for policy 1, policy_version 484018 (0.0009) [2023-12-27 08:59:40,794][20381] Updated weights for policy 1, policy_version 484028 (0.0011) [2023-12-27 08:59:40,843][20347] Updated weights for policy 0, policy_version 484417 (0.0009) [2023-12-27 08:59:40,853][20381] Updated weights for policy 1, policy_version 484038 (0.0011) [2023-12-27 08:59:40,897][20347] Updated weights for policy 0, policy_version 484427 (0.0006) [2023-12-27 08:59:40,908][20381] Updated weights for policy 1, policy_version 484048 (0.0007) [2023-12-27 08:59:40,959][20347] Updated weights for policy 0, policy_version 484437 (0.0006) [2023-12-27 08:59:41,020][20347] Updated weights for policy 0, policy_version 484447 (0.0007) [2023-12-27 08:59:41,617][20381] Updated weights for policy 1, policy_version 484058 (0.0008) [2023-12-27 08:59:41,687][20381] Updated weights for policy 1, policy_version 484068 (0.0007) [2023-12-27 08:59:41,693][20347] Updated weights for policy 0, policy_version 484457 (0.0008) [2023-12-27 08:59:41,748][20381] Updated weights for policy 1, policy_version 484078 (0.0007) [2023-12-27 08:59:41,766][20347] Updated weights for policy 0, policy_version 484467 (0.0008) [2023-12-27 08:59:41,820][20347] Updated weights for policy 0, policy_version 484477 (0.0007) [2023-12-27 08:59:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 247988224. Throughput: 0: 10432.5, 1: 10491.9. Samples: 247981304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '191.530')] [2023-12-27 08:59:42,439][20381] Updated weights for policy 1, policy_version 484088 (0.0006) [2023-12-27 08:59:42,485][20347] Updated weights for policy 0, policy_version 484487 (0.0008) [2023-12-27 08:59:42,499][20381] Updated weights for policy 1, policy_version 484098 (0.0006) [2023-12-27 08:59:42,545][20347] Updated weights for policy 0, policy_version 484497 (0.0007) [2023-12-27 08:59:42,549][20381] Updated weights for policy 1, policy_version 484108 (0.0011) [2023-12-27 08:59:42,600][20347] Updated weights for policy 0, policy_version 484507 (0.0007) [2023-12-27 08:59:43,157][20381] Updated weights for policy 1, policy_version 484118 (0.0007) [2023-12-27 08:59:43,207][20381] Updated weights for policy 1, policy_version 484128 (0.0008) [2023-12-27 08:59:43,259][20381] Updated weights for policy 1, policy_version 484138 (0.0010) [2023-12-27 08:59:43,328][20347] Updated weights for policy 0, policy_version 484517 (0.0007) [2023-12-27 08:59:43,379][20347] Updated weights for policy 0, policy_version 484527 (0.0007) [2023-12-27 08:59:43,436][20347] Updated weights for policy 0, policy_version 484537 (0.0007) [2023-12-27 08:59:43,951][20381] Updated weights for policy 1, policy_version 484148 (0.0011) [2023-12-27 08:59:44,000][20381] Updated weights for policy 1, policy_version 484158 (0.0010) [2023-12-27 08:59:44,047][20381] Updated weights for policy 1, policy_version 484168 (0.0010) [2023-12-27 08:59:44,117][20347] Updated weights for policy 0, policy_version 484547 (0.0007) [2023-12-27 08:59:44,179][20347] Updated weights for policy 0, policy_version 484557 (0.0006) [2023-12-27 08:59:44,235][20347] Updated weights for policy 0, policy_version 484567 (0.0006) [2023-12-27 08:59:44,751][20381] Updated weights for policy 1, policy_version 484178 (0.0010) [2023-12-27 08:59:44,811][20381] Updated weights for policy 1, policy_version 484188 (0.0010) [2023-12-27 08:59:44,869][20381] Updated weights for policy 1, policy_version 484198 (0.0011) [2023-12-27 08:59:44,911][20347] Updated weights for policy 0, policy_version 484577 (0.0007) [2023-12-27 08:59:44,930][20381] Updated weights for policy 1, policy_version 484208 (0.0011) [2023-12-27 08:59:44,976][20347] Updated weights for policy 0, policy_version 484587 (0.0010) [2023-12-27 08:59:45,033][20347] Updated weights for policy 0, policy_version 484597 (0.0011) [2023-12-27 08:59:45,091][20347] Updated weights for policy 0, policy_version 484607 (0.0010) [2023-12-27 08:59:45,612][20381] Updated weights for policy 1, policy_version 484218 (0.0007) [2023-12-27 08:59:45,675][20381] Updated weights for policy 1, policy_version 484228 (0.0009) [2023-12-27 08:59:45,730][20381] Updated weights for policy 1, policy_version 484238 (0.0005) [2023-12-27 08:59:45,745][20347] Updated weights for policy 0, policy_version 484617 (0.0007) [2023-12-27 08:59:45,801][20347] Updated weights for policy 0, policy_version 484627 (0.0009) [2023-12-27 08:59:45,849][20347] Updated weights for policy 0, policy_version 484637 (0.0008) [2023-12-27 08:59:46,296][20381] Updated weights for policy 1, policy_version 484248 (0.0006) [2023-12-27 08:59:46,359][20381] Updated weights for policy 1, policy_version 484258 (0.0007) [2023-12-27 08:59:46,420][20381] Updated weights for policy 1, policy_version 484268 (0.0008) [2023-12-27 08:59:46,587][20347] Updated weights for policy 0, policy_version 484647 (0.0008) [2023-12-27 08:59:46,643][20347] Updated weights for policy 0, policy_version 484657 (0.0007) [2023-12-27 08:59:46,710][20347] Updated weights for policy 0, policy_version 484667 (0.0007) [2023-12-27 08:59:46,975][20381] Updated weights for policy 1, policy_version 484278 (0.0007) [2023-12-27 08:59:47,026][20381] Updated weights for policy 1, policy_version 484288 (0.0007) [2023-12-27 08:59:47,081][20381] Updated weights for policy 1, policy_version 484298 (0.0007) [2023-12-27 08:59:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 248094720. Throughput: 0: 10420.8, 1: 10444.1. Samples: 248104432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '411.470')] [2023-12-27 08:59:47,427][20347] Updated weights for policy 0, policy_version 484677 (0.0008) [2023-12-27 08:59:47,481][20347] Updated weights for policy 0, policy_version 484687 (0.0007) [2023-12-27 08:59:47,538][20347] Updated weights for policy 0, policy_version 484697 (0.0005) [2023-12-27 08:59:47,786][20381] Updated weights for policy 1, policy_version 484308 (0.0008) [2023-12-27 08:59:47,842][20381] Updated weights for policy 1, policy_version 484318 (0.0008) [2023-12-27 08:59:47,900][20381] Updated weights for policy 1, policy_version 484328 (0.0007) [2023-12-27 08:59:48,176][20347] Updated weights for policy 0, policy_version 484707 (0.0005) [2023-12-27 08:59:48,243][20347] Updated weights for policy 0, policy_version 484717 (0.0005) [2023-12-27 08:59:48,303][20347] Updated weights for policy 0, policy_version 484727 (0.0008) [2023-12-27 08:59:48,610][20381] Updated weights for policy 1, policy_version 484338 (0.0008) [2023-12-27 08:59:48,666][20381] Updated weights for policy 1, policy_version 484348 (0.0010) [2023-12-27 08:59:48,715][20381] Updated weights for policy 1, policy_version 484358 (0.0010) [2023-12-27 08:59:48,772][20381] Updated weights for policy 1, policy_version 484368 (0.0008) [2023-12-27 08:59:48,924][20347] Updated weights for policy 0, policy_version 484737 (0.0009) [2023-12-27 08:59:48,982][20347] Updated weights for policy 0, policy_version 484747 (0.0010) [2023-12-27 08:59:49,046][20347] Updated weights for policy 0, policy_version 484757 (0.0010) [2023-12-27 08:59:49,108][20347] Updated weights for policy 0, policy_version 484767 (0.0011) [2023-12-27 08:59:49,433][20381] Updated weights for policy 1, policy_version 484378 (0.0006) [2023-12-27 08:59:49,479][20381] Updated weights for policy 1, policy_version 484388 (0.0006) [2023-12-27 08:59:49,525][20381] Updated weights for policy 1, policy_version 484398 (0.0007) [2023-12-27 08:59:49,812][20347] Updated weights for policy 0, policy_version 484777 (0.0011) [2023-12-27 08:59:49,878][20347] Updated weights for policy 0, policy_version 484787 (0.0010) [2023-12-27 08:59:49,942][20347] Updated weights for policy 0, policy_version 484797 (0.0008) [2023-12-27 08:59:50,274][20381] Updated weights for policy 1, policy_version 484408 (0.0008) [2023-12-27 08:59:50,330][20381] Updated weights for policy 1, policy_version 484418 (0.0008) [2023-12-27 08:59:50,381][20381] Updated weights for policy 1, policy_version 484428 (0.0008) [2023-12-27 08:59:50,594][20347] Updated weights for policy 0, policy_version 484807 (0.0009) [2023-12-27 08:59:50,649][20347] Updated weights for policy 0, policy_version 484817 (0.0010) [2023-12-27 08:59:50,704][20347] Updated weights for policy 0, policy_version 484827 (0.0010) [2023-12-27 08:59:51,066][20381] Updated weights for policy 1, policy_version 484438 (0.0008) [2023-12-27 08:59:51,127][20381] Updated weights for policy 1, policy_version 484448 (0.0007) [2023-12-27 08:59:51,182][20381] Updated weights for policy 1, policy_version 484458 (0.0008) [2023-12-27 08:59:51,322][20347] Updated weights for policy 0, policy_version 484837 (0.0010) [2023-12-27 08:59:51,392][20347] Updated weights for policy 0, policy_version 484847 (0.0010) [2023-12-27 08:59:51,449][20347] Updated weights for policy 0, policy_version 484857 (0.0011) [2023-12-27 08:59:51,885][20381] Updated weights for policy 1, policy_version 484468 (0.0006) [2023-12-27 08:59:51,949][20381] Updated weights for policy 1, policy_version 484478 (0.0005) [2023-12-27 08:59:52,012][20381] Updated weights for policy 1, policy_version 484488 (0.0006) [2023-12-27 08:59:52,170][20347] Updated weights for policy 0, policy_version 484867 (0.0009) [2023-12-27 08:59:52,228][20347] Updated weights for policy 0, policy_version 484877 (0.0008) [2023-12-27 08:59:52,284][20347] Updated weights for policy 0, policy_version 484887 (0.0008) [2023-12-27 08:59:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20910.4). Total num frames: 248201216. Throughput: 0: 10382.4, 1: 10410.0. Samples: 248166008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '374.440')] [2023-12-27 08:59:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000484896_124149760.pth... [2023-12-27 08:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000484496_124051456.pth... [2023-12-27 08:59:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000483680_123838464.pth [2023-12-27 08:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000483280_123740160.pth [2023-12-27 08:59:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000484896_124149760.pth [2023-12-27 08:59:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000484496_124051456.pth [2023-12-27 08:59:52,639][20381] Updated weights for policy 1, policy_version 484498 (0.0008) [2023-12-27 08:59:52,695][20381] Updated weights for policy 1, policy_version 484508 (0.0007) [2023-12-27 08:59:52,741][20381] Updated weights for policy 1, policy_version 484518 (0.0007) [2023-12-27 08:59:52,796][20381] Updated weights for policy 1, policy_version 484528 (0.0008) [2023-12-27 08:59:52,998][20347] Updated weights for policy 0, policy_version 484897 (0.0008) [2023-12-27 08:59:53,054][20347] Updated weights for policy 0, policy_version 484907 (0.0007) [2023-12-27 08:59:53,115][20347] Updated weights for policy 0, policy_version 484917 (0.0008) [2023-12-27 08:59:53,163][20347] Updated weights for policy 0, policy_version 484927 (0.0008) [2023-12-27 08:59:53,511][20381] Updated weights for policy 1, policy_version 484538 (0.0008) [2023-12-27 08:59:53,567][20381] Updated weights for policy 1, policy_version 484548 (0.0007) [2023-12-27 08:59:53,625][20381] Updated weights for policy 1, policy_version 484558 (0.0008) [2023-12-27 08:59:53,822][20347] Updated weights for policy 0, policy_version 484937 (0.0008) [2023-12-27 08:59:53,877][20347] Updated weights for policy 0, policy_version 484947 (0.0008) [2023-12-27 08:59:53,925][20347] Updated weights for policy 0, policy_version 484957 (0.0008) [2023-12-27 08:59:54,312][20381] Updated weights for policy 1, policy_version 484568 (0.0008) [2023-12-27 08:59:54,361][20381] Updated weights for policy 1, policy_version 484578 (0.0007) [2023-12-27 08:59:54,413][20381] Updated weights for policy 1, policy_version 484588 (0.0007) [2023-12-27 08:59:54,605][20347] Updated weights for policy 0, policy_version 484967 (0.0007) [2023-12-27 08:59:54,665][20347] Updated weights for policy 0, policy_version 484977 (0.0008) [2023-12-27 08:59:54,720][20347] Updated weights for policy 0, policy_version 484987 (0.0008) [2023-12-27 08:59:55,066][20381] Updated weights for policy 1, policy_version 484598 (0.0006) [2023-12-27 08:59:55,119][20381] Updated weights for policy 1, policy_version 484608 (0.0007) [2023-12-27 08:59:55,175][20381] Updated weights for policy 1, policy_version 484618 (0.0007) [2023-12-27 08:59:55,407][20347] Updated weights for policy 0, policy_version 484997 (0.0007) [2023-12-27 08:59:55,460][20347] Updated weights for policy 0, policy_version 485007 (0.0007) [2023-12-27 08:59:55,511][20347] Updated weights for policy 0, policy_version 485017 (0.0007) [2023-12-27 08:59:55,853][20381] Updated weights for policy 1, policy_version 484628 (0.0009) [2023-12-27 08:59:55,917][20381] Updated weights for policy 1, policy_version 484638 (0.0010) [2023-12-27 08:59:55,972][20381] Updated weights for policy 1, policy_version 484648 (0.0010) [2023-12-27 08:59:56,220][20347] Updated weights for policy 0, policy_version 485027 (0.0008) [2023-12-27 08:59:56,269][20347] Updated weights for policy 0, policy_version 485037 (0.0007) [2023-12-27 08:59:56,317][20347] Updated weights for policy 0, policy_version 485047 (0.0007) [2023-12-27 08:59:56,574][20381] Updated weights for policy 1, policy_version 484658 (0.0010) [2023-12-27 08:59:56,633][20381] Updated weights for policy 1, policy_version 484668 (0.0011) [2023-12-27 08:59:56,684][20381] Updated weights for policy 1, policy_version 484678 (0.0007) [2023-12-27 08:59:56,734][20381] Updated weights for policy 1, policy_version 484688 (0.0008) [2023-12-27 08:59:57,003][20347] Updated weights for policy 0, policy_version 485057 (0.0007) [2023-12-27 08:59:57,059][20347] Updated weights for policy 0, policy_version 485067 (0.0007) [2023-12-27 08:59:57,112][20347] Updated weights for policy 0, policy_version 485077 (0.0007) [2023-12-27 08:59:57,165][20347] Updated weights for policy 0, policy_version 485087 (0.0007) [2023-12-27 08:59:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20752.9, 300 sec: 20882.6). Total num frames: 248299520. Throughput: 0: 10374.9, 1: 10399.5. Samples: 248289100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 08:59:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '494.480')] [2023-12-27 08:59:57,394][20381] Updated weights for policy 1, policy_version 484698 (0.0009) [2023-12-27 08:59:57,451][20381] Updated weights for policy 1, policy_version 484708 (0.0008) [2023-12-27 08:59:57,505][20381] Updated weights for policy 1, policy_version 484718 (0.0010) [2023-12-27 08:59:57,753][20347] Updated weights for policy 0, policy_version 485097 (0.0005) [2023-12-27 08:59:57,804][20347] Updated weights for policy 0, policy_version 485107 (0.0006) [2023-12-27 08:59:57,852][20347] Updated weights for policy 0, policy_version 485117 (0.0005) [2023-12-27 08:59:58,174][20381] Updated weights for policy 1, policy_version 484728 (0.0010) [2023-12-27 08:59:58,234][20381] Updated weights for policy 1, policy_version 484738 (0.0007) [2023-12-27 08:59:58,290][20381] Updated weights for policy 1, policy_version 484748 (0.0007) [2023-12-27 08:59:58,478][20347] Updated weights for policy 0, policy_version 485127 (0.0007) [2023-12-27 08:59:58,538][20347] Updated weights for policy 0, policy_version 485137 (0.0008) [2023-12-27 08:59:58,593][20347] Updated weights for policy 0, policy_version 485147 (0.0006) [2023-12-27 08:59:59,090][20381] Updated weights for policy 1, policy_version 484758 (0.0007) [2023-12-27 08:59:59,150][20381] Updated weights for policy 1, policy_version 484768 (0.0008) [2023-12-27 08:59:59,211][20381] Updated weights for policy 1, policy_version 484778 (0.0008) [2023-12-27 08:59:59,387][20347] Updated weights for policy 0, policy_version 485157 (0.0006) [2023-12-27 08:59:59,437][20347] Updated weights for policy 0, policy_version 485167 (0.0006) [2023-12-27 08:59:59,501][20347] Updated weights for policy 0, policy_version 485177 (0.0005) [2023-12-27 08:59:59,866][20381] Updated weights for policy 1, policy_version 484788 (0.0007) [2023-12-27 08:59:59,928][20381] Updated weights for policy 1, policy_version 484798 (0.0006) [2023-12-27 08:59:59,989][20381] Updated weights for policy 1, policy_version 484808 (0.0006) [2023-12-27 09:00:00,110][20347] Updated weights for policy 0, policy_version 485187 (0.0007) [2023-12-27 09:00:00,159][20347] Updated weights for policy 0, policy_version 485197 (0.0010) [2023-12-27 09:00:00,208][20347] Updated weights for policy 0, policy_version 485207 (0.0010) [2023-12-27 09:00:00,579][20381] Updated weights for policy 1, policy_version 484818 (0.0006) [2023-12-27 09:00:00,631][20381] Updated weights for policy 1, policy_version 484828 (0.0006) [2023-12-27 09:00:00,682][20381] Updated weights for policy 1, policy_version 484838 (0.0007) [2023-12-27 09:00:00,737][20381] Updated weights for policy 1, policy_version 484848 (0.0007) [2023-12-27 09:00:00,835][20347] Updated weights for policy 0, policy_version 485217 (0.0005) [2023-12-27 09:00:00,888][20347] Updated weights for policy 0, policy_version 485227 (0.0007) [2023-12-27 09:00:00,945][20347] Updated weights for policy 0, policy_version 485237 (0.0007) [2023-12-27 09:00:01,002][20347] Updated weights for policy 0, policy_version 485247 (0.0007) [2023-12-27 09:00:01,458][20381] Updated weights for policy 1, policy_version 484858 (0.0010) [2023-12-27 09:00:01,516][20381] Updated weights for policy 1, policy_version 484868 (0.0007) [2023-12-27 09:00:01,574][20381] Updated weights for policy 1, policy_version 484878 (0.0008) [2023-12-27 09:00:01,649][20347] Updated weights for policy 0, policy_version 485257 (0.0007) [2023-12-27 09:00:01,707][20347] Updated weights for policy 0, policy_version 485267 (0.0006) [2023-12-27 09:00:01,765][20347] Updated weights for policy 0, policy_version 485277 (0.0008) [2023-12-27 09:00:02,255][20381] Updated weights for policy 1, policy_version 484888 (0.0007) [2023-12-27 09:00:02,314][20381] Updated weights for policy 1, policy_version 484898 (0.0008) [2023-12-27 09:00:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 248397824. Throughput: 0: 10402.1, 1: 10414.2. Samples: 248413596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '325.170')] [2023-12-27 09:00:02,374][20381] Updated weights for policy 1, policy_version 484908 (0.0008) [2023-12-27 09:00:02,453][20347] Updated weights for policy 0, policy_version 485287 (0.0008) [2023-12-27 09:00:02,506][20347] Updated weights for policy 0, policy_version 485297 (0.0010) [2023-12-27 09:00:02,551][20347] Updated weights for policy 0, policy_version 485307 (0.0010) [2023-12-27 09:00:03,063][20381] Updated weights for policy 1, policy_version 484918 (0.0010) [2023-12-27 09:00:03,115][20381] Updated weights for policy 1, policy_version 484928 (0.0010) [2023-12-27 09:00:03,163][20381] Updated weights for policy 1, policy_version 484938 (0.0010) [2023-12-27 09:00:03,246][20347] Updated weights for policy 0, policy_version 485317 (0.0009) [2023-12-27 09:00:03,310][20347] Updated weights for policy 0, policy_version 485327 (0.0010) [2023-12-27 09:00:03,354][20347] Updated weights for policy 0, policy_version 485337 (0.0010) [2023-12-27 09:00:03,761][20381] Updated weights for policy 1, policy_version 484948 (0.0009) [2023-12-27 09:00:03,815][20381] Updated weights for policy 1, policy_version 484958 (0.0007) [2023-12-27 09:00:03,876][20381] Updated weights for policy 1, policy_version 484968 (0.0008) [2023-12-27 09:00:04,010][20347] Updated weights for policy 0, policy_version 485347 (0.0009) [2023-12-27 09:00:04,067][20347] Updated weights for policy 0, policy_version 485357 (0.0010) [2023-12-27 09:00:04,125][20347] Updated weights for policy 0, policy_version 485367 (0.0007) [2023-12-27 09:00:04,560][20381] Updated weights for policy 1, policy_version 484978 (0.0007) [2023-12-27 09:00:04,615][20381] Updated weights for policy 1, policy_version 484988 (0.0005) [2023-12-27 09:00:04,673][20381] Updated weights for policy 1, policy_version 484998 (0.0005) [2023-12-27 09:00:04,730][20381] Updated weights for policy 1, policy_version 485008 (0.0008) [2023-12-27 09:00:04,782][20347] Updated weights for policy 0, policy_version 485377 (0.0006) [2023-12-27 09:00:04,833][20347] Updated weights for policy 0, policy_version 485387 (0.0010) [2023-12-27 09:00:04,887][20347] Updated weights for policy 0, policy_version 485397 (0.0010) [2023-12-27 09:00:04,931][20347] Updated weights for policy 0, policy_version 485407 (0.0010) [2023-12-27 09:00:05,368][20381] Updated weights for policy 1, policy_version 485018 (0.0005) [2023-12-27 09:00:05,422][20381] Updated weights for policy 1, policy_version 485028 (0.0007) [2023-12-27 09:00:05,469][20381] Updated weights for policy 1, policy_version 485038 (0.0010) [2023-12-27 09:00:05,570][20347] Updated weights for policy 0, policy_version 485417 (0.0010) [2023-12-27 09:00:05,627][20347] Updated weights for policy 0, policy_version 485427 (0.0010) [2023-12-27 09:00:05,687][20347] Updated weights for policy 0, policy_version 485437 (0.0010) [2023-12-27 09:00:06,148][20381] Updated weights for policy 1, policy_version 485048 (0.0011) [2023-12-27 09:00:06,204][20381] Updated weights for policy 1, policy_version 485058 (0.0011) [2023-12-27 09:00:06,265][20381] Updated weights for policy 1, policy_version 485068 (0.0010) [2023-12-27 09:00:06,326][20347] Updated weights for policy 0, policy_version 485447 (0.0007) [2023-12-27 09:00:06,385][20347] Updated weights for policy 0, policy_version 485457 (0.0008) [2023-12-27 09:00:06,451][20347] Updated weights for policy 0, policy_version 485467 (0.0009) [2023-12-27 09:00:06,932][20381] Updated weights for policy 1, policy_version 485078 (0.0009) [2023-12-27 09:00:06,993][20381] Updated weights for policy 1, policy_version 485088 (0.0011) [2023-12-27 09:00:07,054][20381] Updated weights for policy 1, policy_version 485098 (0.0010) [2023-12-27 09:00:07,080][20347] Updated weights for policy 0, policy_version 485477 (0.0007) [2023-12-27 09:00:07,147][20347] Updated weights for policy 0, policy_version 485487 (0.0008) [2023-12-27 09:00:07,198][20347] Updated weights for policy 0, policy_version 485497 (0.0011) [2023-12-27 09:00:07,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 248512512. Throughput: 0: 10408.0, 1: 10438.2. Samples: 248475908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '300.250')] [2023-12-27 09:00:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000485104_124207104.pth... [2023-12-27 09:00:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000485504_124305408.pth... [2023-12-27 09:00:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000483888_123895808.pth [2023-12-27 09:00:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000484288_123994112.pth [2023-12-27 09:00:07,639][20381] Updated weights for policy 1, policy_version 485108 (0.0006) [2023-12-27 09:00:07,695][20381] Updated weights for policy 1, policy_version 485118 (0.0005) [2023-12-27 09:00:07,751][20381] Updated weights for policy 1, policy_version 485128 (0.0007) [2023-12-27 09:00:07,871][20347] Updated weights for policy 0, policy_version 485507 (0.0009) [2023-12-27 09:00:07,933][20347] Updated weights for policy 0, policy_version 485517 (0.0008) [2023-12-27 09:00:07,997][20347] Updated weights for policy 0, policy_version 485527 (0.0007) [2023-12-27 09:00:08,295][20381] Updated weights for policy 1, policy_version 485138 (0.0005) [2023-12-27 09:00:08,353][20381] Updated weights for policy 1, policy_version 485148 (0.0007) [2023-12-27 09:00:08,409][20381] Updated weights for policy 1, policy_version 485158 (0.0008) [2023-12-27 09:00:08,462][20381] Updated weights for policy 1, policy_version 485168 (0.0008) [2023-12-27 09:00:08,703][20347] Updated weights for policy 0, policy_version 485537 (0.0008) [2023-12-27 09:00:08,754][20347] Updated weights for policy 0, policy_version 485547 (0.0007) [2023-12-27 09:00:08,811][20347] Updated weights for policy 0, policy_version 485557 (0.0007) [2023-12-27 09:00:08,870][20347] Updated weights for policy 0, policy_version 485567 (0.0005) [2023-12-27 09:00:09,151][20381] Updated weights for policy 1, policy_version 485178 (0.0008) [2023-12-27 09:00:09,216][20381] Updated weights for policy 1, policy_version 485188 (0.0008) [2023-12-27 09:00:09,285][20381] Updated weights for policy 1, policy_version 485198 (0.0008) [2023-12-27 09:00:09,544][20347] Updated weights for policy 0, policy_version 485577 (0.0006) [2023-12-27 09:00:09,607][20347] Updated weights for policy 0, policy_version 485587 (0.0007) [2023-12-27 09:00:09,671][20347] Updated weights for policy 0, policy_version 485597 (0.0006) [2023-12-27 09:00:10,051][20381] Updated weights for policy 1, policy_version 485208 (0.0008) [2023-12-27 09:00:10,112][20381] Updated weights for policy 1, policy_version 485218 (0.0008) [2023-12-27 09:00:10,173][20381] Updated weights for policy 1, policy_version 485228 (0.0008) [2023-12-27 09:00:10,267][20347] Updated weights for policy 0, policy_version 485607 (0.0008) [2023-12-27 09:00:10,328][20347] Updated weights for policy 0, policy_version 485617 (0.0006) [2023-12-27 09:00:10,378][20347] Updated weights for policy 0, policy_version 485627 (0.0006) [2023-12-27 09:00:10,884][20381] Updated weights for policy 1, policy_version 485238 (0.0008) [2023-12-27 09:00:10,934][20381] Updated weights for policy 1, policy_version 485248 (0.0008) [2023-12-27 09:00:10,996][20381] Updated weights for policy 1, policy_version 485258 (0.0007) [2023-12-27 09:00:10,998][20347] Updated weights for policy 0, policy_version 485637 (0.0007) [2023-12-27 09:00:11,060][20347] Updated weights for policy 0, policy_version 485647 (0.0007) [2023-12-27 09:00:11,118][20347] Updated weights for policy 0, policy_version 485657 (0.0008) [2023-12-27 09:00:11,778][20381] Updated weights for policy 1, policy_version 485268 (0.0007) [2023-12-27 09:00:11,824][20347] Updated weights for policy 0, policy_version 485667 (0.0007) [2023-12-27 09:00:11,844][20381] Updated weights for policy 1, policy_version 485278 (0.0007) [2023-12-27 09:00:11,886][20347] Updated weights for policy 0, policy_version 485677 (0.0006) [2023-12-27 09:00:11,903][20381] Updated weights for policy 1, policy_version 485288 (0.0007) [2023-12-27 09:00:11,942][20347] Updated weights for policy 0, policy_version 485687 (0.0006) [2023-12-27 09:00:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 248610816. Throughput: 0: 10369.9, 1: 10428.0. Samples: 248601064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '216.850')] [2023-12-27 09:00:12,598][20381] Updated weights for policy 1, policy_version 485298 (0.0005) [2023-12-27 09:00:12,614][20347] Updated weights for policy 0, policy_version 485697 (0.0007) [2023-12-27 09:00:12,655][20381] Updated weights for policy 1, policy_version 485308 (0.0005) [2023-12-27 09:00:12,669][20347] Updated weights for policy 0, policy_version 485707 (0.0009) [2023-12-27 09:00:12,705][20381] Updated weights for policy 1, policy_version 485318 (0.0006) [2023-12-27 09:00:12,727][20347] Updated weights for policy 0, policy_version 485717 (0.0007) [2023-12-27 09:00:12,759][20381] Updated weights for policy 1, policy_version 485328 (0.0005) [2023-12-27 09:00:12,789][20347] Updated weights for policy 0, policy_version 485727 (0.0008) [2023-12-27 09:00:13,329][20381] Updated weights for policy 1, policy_version 485338 (0.0007) [2023-12-27 09:00:13,378][20381] Updated weights for policy 1, policy_version 485348 (0.0009) [2023-12-27 09:00:13,426][20381] Updated weights for policy 1, policy_version 485358 (0.0010) [2023-12-27 09:00:13,505][20347] Updated weights for policy 0, policy_version 485737 (0.0006) [2023-12-27 09:00:13,557][20347] Updated weights for policy 0, policy_version 485747 (0.0006) [2023-12-27 09:00:13,611][20347] Updated weights for policy 0, policy_version 485757 (0.0007) [2023-12-27 09:00:14,042][20381] Updated weights for policy 1, policy_version 485368 (0.0009) [2023-12-27 09:00:14,094][20381] Updated weights for policy 1, policy_version 485378 (0.0007) [2023-12-27 09:00:14,160][20381] Updated weights for policy 1, policy_version 485388 (0.0008) [2023-12-27 09:00:14,398][20347] Updated weights for policy 0, policy_version 485767 (0.0008) [2023-12-27 09:00:14,458][20347] Updated weights for policy 0, policy_version 485777 (0.0007) [2023-12-27 09:00:14,512][20347] Updated weights for policy 0, policy_version 485787 (0.0007) [2023-12-27 09:00:14,910][20381] Updated weights for policy 1, policy_version 485398 (0.0010) [2023-12-27 09:00:14,969][20381] Updated weights for policy 1, policy_version 485408 (0.0011) [2023-12-27 09:00:15,028][20381] Updated weights for policy 1, policy_version 485418 (0.0011) [2023-12-27 09:00:15,240][20347] Updated weights for policy 0, policy_version 485797 (0.0008) [2023-12-27 09:00:15,299][20347] Updated weights for policy 0, policy_version 485807 (0.0007) [2023-12-27 09:00:15,351][20347] Updated weights for policy 0, policy_version 485817 (0.0007) [2023-12-27 09:00:15,749][20381] Updated weights for policy 1, policy_version 485428 (0.0011) [2023-12-27 09:00:15,812][20381] Updated weights for policy 1, policy_version 485438 (0.0011) [2023-12-27 09:00:15,868][20381] Updated weights for policy 1, policy_version 485448 (0.0011) [2023-12-27 09:00:16,058][20347] Updated weights for policy 0, policy_version 485827 (0.0008) [2023-12-27 09:00:16,115][20347] Updated weights for policy 0, policy_version 485837 (0.0008) [2023-12-27 09:00:16,186][20347] Updated weights for policy 0, policy_version 485847 (0.0008) [2023-12-27 09:00:16,555][20381] Updated weights for policy 1, policy_version 485458 (0.0010) [2023-12-27 09:00:16,616][20381] Updated weights for policy 1, policy_version 485468 (0.0008) [2023-12-27 09:00:16,666][20381] Updated weights for policy 1, policy_version 485478 (0.0011) [2023-12-27 09:00:16,715][20381] Updated weights for policy 1, policy_version 485488 (0.0010) [2023-12-27 09:00:16,847][20347] Updated weights for policy 0, policy_version 485857 (0.0007) [2023-12-27 09:00:16,900][20347] Updated weights for policy 0, policy_version 485867 (0.0007) [2023-12-27 09:00:16,944][20347] Updated weights for policy 0, policy_version 485877 (0.0007) [2023-12-27 09:00:16,998][20347] Updated weights for policy 0, policy_version 485887 (0.0007) [2023-12-27 09:00:17,325][20381] Updated weights for policy 1, policy_version 485498 (0.0005) [2023-12-27 09:00:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 248709120. Throughput: 0: 10239.2, 1: 10446.2. Samples: 248721524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:17,363][19170] Avg episode reward: [(0, '990.380'), (1, '354.930')] [2023-12-27 09:00:17,383][20381] Updated weights for policy 1, policy_version 485508 (0.0005) [2023-12-27 09:00:17,438][20381] Updated weights for policy 1, policy_version 485518 (0.0011) [2023-12-27 09:00:17,750][20291] KL-divergence is very high: 231.2722 [2023-12-27 09:00:17,751][20347] Updated weights for policy 0, policy_version 485897 (0.0008) [2023-12-27 09:00:17,774][20291] KL-divergence is very high: 2141.6099 [2023-12-27 09:00:17,799][20347] Updated weights for policy 0, policy_version 485907 (0.0008) [2023-12-27 09:00:17,813][20291] KL-divergence is very high: 186.8765 [2023-12-27 09:00:17,850][20347] Updated weights for policy 0, policy_version 485917 (0.0007) [2023-12-27 09:00:17,854][20291] KL-divergence is very high: 225.8988 [2023-12-27 09:00:18,103][20381] Updated weights for policy 1, policy_version 485528 (0.0010) [2023-12-27 09:00:18,159][20381] Updated weights for policy 1, policy_version 485538 (0.0011) [2023-12-27 09:00:18,212][20381] Updated weights for policy 1, policy_version 485548 (0.0011) [2023-12-27 09:00:18,559][20347] Updated weights for policy 0, policy_version 485927 (0.0007) [2023-12-27 09:00:18,615][20347] Updated weights for policy 0, policy_version 485937 (0.0008) [2023-12-27 09:00:18,677][20347] Updated weights for policy 0, policy_version 485947 (0.0007) [2023-12-27 09:00:18,934][20381] Updated weights for policy 1, policy_version 485558 (0.0011) [2023-12-27 09:00:18,986][20381] Updated weights for policy 1, policy_version 485568 (0.0009) [2023-12-27 09:00:19,036][20381] Updated weights for policy 1, policy_version 485578 (0.0011) [2023-12-27 09:00:19,328][20347] Updated weights for policy 0, policy_version 485957 (0.0008) [2023-12-27 09:00:19,391][20347] Updated weights for policy 0, policy_version 485967 (0.0007) [2023-12-27 09:00:19,452][20347] Updated weights for policy 0, policy_version 485977 (0.0008) [2023-12-27 09:00:19,696][20381] Updated weights for policy 1, policy_version 485588 (0.0009) [2023-12-27 09:00:19,745][20381] Updated weights for policy 1, policy_version 485598 (0.0008) [2023-12-27 09:00:19,797][20381] Updated weights for policy 1, policy_version 485608 (0.0007) [2023-12-27 09:00:20,185][20347] Updated weights for policy 0, policy_version 485987 (0.0009) [2023-12-27 09:00:20,203][20291] KL-divergence is very high: 178.4997 [2023-12-27 09:00:20,214][20291] KL-divergence is very high: 495.9051 [2023-12-27 09:00:20,244][20347] Updated weights for policy 0, policy_version 485997 (0.0010) [2023-12-27 09:00:20,248][20291] KL-divergence is very high: 194.5909 [2023-12-27 09:00:20,260][20291] KL-divergence is very high: 359.2184 [2023-12-27 09:00:20,303][20347] Updated weights for policy 0, policy_version 486007 (0.0010) [2023-12-27 09:00:20,312][20291] KL-divergence is very high: 482.0946 [2023-12-27 09:00:20,346][20291] KL-divergence is very high: 107.3793 [2023-12-27 09:00:20,531][20381] Updated weights for policy 1, policy_version 485618 (0.0007) [2023-12-27 09:00:20,584][20381] Updated weights for policy 1, policy_version 485628 (0.0008) [2023-12-27 09:00:20,643][20381] Updated weights for policy 1, policy_version 485638 (0.0008) [2023-12-27 09:00:20,699][20381] Updated weights for policy 1, policy_version 485648 (0.0007) [2023-12-27 09:00:20,989][20347] Updated weights for policy 0, policy_version 486017 (0.0010) [2023-12-27 09:00:21,044][20347] Updated weights for policy 0, policy_version 486027 (0.0010) [2023-12-27 09:00:21,108][20347] Updated weights for policy 0, policy_version 486037 (0.0009) [2023-12-27 09:00:21,181][20347] Updated weights for policy 0, policy_version 486047 (0.0006) [2023-12-27 09:00:21,396][20381] Updated weights for policy 1, policy_version 485658 (0.0008) [2023-12-27 09:00:21,454][20381] Updated weights for policy 1, policy_version 485668 (0.0009) [2023-12-27 09:00:21,510][20381] Updated weights for policy 1, policy_version 485678 (0.0008) [2023-12-27 09:00:21,877][20347] Updated weights for policy 0, policy_version 486057 (0.0007) [2023-12-27 09:00:21,911][20291] KL-divergence is very high: 177.4885 [2023-12-27 09:00:21,936][20347] Updated weights for policy 0, policy_version 486067 (0.0007) [2023-12-27 09:00:21,996][20347] Updated weights for policy 0, policy_version 486077 (0.0007) [2023-12-27 09:00:22,187][20381] Updated weights for policy 1, policy_version 485688 (0.0007) [2023-12-27 09:00:22,234][20381] Updated weights for policy 1, policy_version 485698 (0.0007) [2023-12-27 09:00:22,300][20381] Updated weights for policy 1, policy_version 485708 (0.0007) [2023-12-27 09:00:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 248815616. Throughput: 0: 10231.9, 1: 10320.9. Samples: 248781376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:22,363][19170] Avg episode reward: [(0, '846.020'), (1, '201.740')] [2023-12-27 09:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000485712_124362752.pth... [2023-12-27 09:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000486080_124452864.pth... [2023-12-27 09:00:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000484496_124051456.pth [2023-12-27 09:00:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000484896_124149760.pth [2023-12-27 09:00:22,724][20347] Updated weights for policy 0, policy_version 486087 (0.0009) [2023-12-27 09:00:22,781][20347] Updated weights for policy 0, policy_version 486097 (0.0010) [2023-12-27 09:00:22,841][20347] Updated weights for policy 0, policy_version 486107 (0.0010) [2023-12-27 09:00:22,993][20381] Updated weights for policy 1, policy_version 485718 (0.0008) [2023-12-27 09:00:23,048][20381] Updated weights for policy 1, policy_version 485728 (0.0006) [2023-12-27 09:00:23,105][20381] Updated weights for policy 1, policy_version 485738 (0.0007) [2023-12-27 09:00:23,490][20291] KL-divergence is very high: 103.7740 [2023-12-27 09:00:23,506][20347] Updated weights for policy 0, policy_version 486117 (0.0008) [2023-12-27 09:00:23,560][20347] Updated weights for policy 0, policy_version 486127 (0.0007) [2023-12-27 09:00:23,617][20347] Updated weights for policy 0, policy_version 486137 (0.0010) [2023-12-27 09:00:23,718][20381] Updated weights for policy 1, policy_version 485748 (0.0006) [2023-12-27 09:00:23,766][20381] Updated weights for policy 1, policy_version 485758 (0.0007) [2023-12-27 09:00:23,821][20381] Updated weights for policy 1, policy_version 485768 (0.0007) [2023-12-27 09:00:24,234][20347] Updated weights for policy 0, policy_version 486147 (0.0010) [2023-12-27 09:00:24,289][20347] Updated weights for policy 0, policy_version 486157 (0.0010) [2023-12-27 09:00:24,349][20347] Updated weights for policy 0, policy_version 486167 (0.0011) [2023-12-27 09:00:24,390][20291] KL-divergence is very high: 104.8144 [2023-12-27 09:00:24,447][20381] Updated weights for policy 1, policy_version 485778 (0.0008) [2023-12-27 09:00:24,504][20381] Updated weights for policy 1, policy_version 485788 (0.0011) [2023-12-27 09:00:24,553][20381] Updated weights for policy 1, policy_version 485798 (0.0007) [2023-12-27 09:00:24,606][20381] Updated weights for policy 1, policy_version 485808 (0.0011) [2023-12-27 09:00:24,979][20347] Updated weights for policy 0, policy_version 486177 (0.0011) [2023-12-27 09:00:24,984][20291] KL-divergence is very high: 120.4975 [2023-12-27 09:00:25,027][20347] Updated weights for policy 0, policy_version 486187 (0.0010) [2023-12-27 09:00:25,074][20347] Updated weights for policy 0, policy_version 486197 (0.0010) [2023-12-27 09:00:25,128][20347] Updated weights for policy 0, policy_version 486207 (0.0010) [2023-12-27 09:00:25,294][20381] Updated weights for policy 1, policy_version 485818 (0.0011) [2023-12-27 09:00:25,339][20381] Updated weights for policy 1, policy_version 485828 (0.0010) [2023-12-27 09:00:25,394][20381] Updated weights for policy 1, policy_version 485838 (0.0010) [2023-12-27 09:00:25,743][20347] Updated weights for policy 0, policy_version 486217 (0.0006) [2023-12-27 09:00:25,801][20347] Updated weights for policy 0, policy_version 486227 (0.0006) [2023-12-27 09:00:25,851][20347] Updated weights for policy 0, policy_version 486237 (0.0008) [2023-12-27 09:00:26,099][20381] Updated weights for policy 1, policy_version 485848 (0.0011) [2023-12-27 09:00:26,158][20381] Updated weights for policy 1, policy_version 485858 (0.0011) [2023-12-27 09:00:26,219][20381] Updated weights for policy 1, policy_version 485868 (0.0011) [2023-12-27 09:00:26,507][20347] Updated weights for policy 0, policy_version 486247 (0.0007) [2023-12-27 09:00:26,567][20347] Updated weights for policy 0, policy_version 486257 (0.0005) [2023-12-27 09:00:26,620][20347] Updated weights for policy 0, policy_version 486267 (0.0005) [2023-12-27 09:00:26,792][20381] Updated weights for policy 1, policy_version 485878 (0.0007) [2023-12-27 09:00:26,856][20381] Updated weights for policy 1, policy_version 485888 (0.0006) [2023-12-27 09:00:26,919][20381] Updated weights for policy 1, policy_version 485898 (0.0005) [2023-12-27 09:00:27,200][20347] Updated weights for policy 0, policy_version 486277 (0.0005) [2023-12-27 09:00:27,259][20347] Updated weights for policy 0, policy_version 486287 (0.0005) [2023-12-27 09:00:27,315][20347] Updated weights for policy 0, policy_version 486297 (0.0005) [2023-12-27 09:00:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 248922112. Throughput: 0: 10229.2, 1: 10304.9. Samples: 248905336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:27,363][19170] Avg episode reward: [(0, '608.890'), (1, '230.570')] [2023-12-27 09:00:27,530][20381] Updated weights for policy 1, policy_version 485908 (0.0007) [2023-12-27 09:00:27,575][20381] Updated weights for policy 1, policy_version 485918 (0.0010) [2023-12-27 09:00:27,624][20381] Updated weights for policy 1, policy_version 485928 (0.0010) [2023-12-27 09:00:27,918][20347] Updated weights for policy 0, policy_version 486307 (0.0007) [2023-12-27 09:00:27,972][20291] KL-divergence is very high: 192.4302 [2023-12-27 09:00:27,978][20347] Updated weights for policy 0, policy_version 486317 (0.0008) [2023-12-27 09:00:27,979][20291] KL-divergence is very high: 377.6275 [2023-12-27 09:00:27,991][20291] KL-divergence is very high: 512.6783 [2023-12-27 09:00:28,038][20347] Updated weights for policy 0, policy_version 486327 (0.0007) [2023-12-27 09:00:28,039][20291] KL-divergence is very high: 106.0473 [2023-12-27 09:00:28,094][20291] KL-divergence is very high: 101.5639 [2023-12-27 09:00:28,267][20381] Updated weights for policy 1, policy_version 485938 (0.0010) [2023-12-27 09:00:28,332][20381] Updated weights for policy 1, policy_version 485948 (0.0008) [2023-12-27 09:00:28,393][20381] Updated weights for policy 1, policy_version 485958 (0.0008) [2023-12-27 09:00:28,455][20381] Updated weights for policy 1, policy_version 485968 (0.0008) [2023-12-27 09:00:28,709][20347] Updated weights for policy 0, policy_version 486337 (0.0007) [2023-12-27 09:00:28,709][20291] KL-divergence is very high: 357.4133 [2023-12-27 09:00:28,747][20291] KL-divergence is very high: 134.6423 [2023-12-27 09:00:28,752][20291] KL-divergence is very high: 123.1205 [2023-12-27 09:00:28,758][20347] Updated weights for policy 0, policy_version 486347 (0.0006) [2023-12-27 09:00:28,792][20291] KL-divergence is very high: 141.6546 [2023-12-27 09:00:28,814][20347] Updated weights for policy 0, policy_version 486357 (0.0008) [2023-12-27 09:00:28,836][20291] KL-divergence is very high: 238.9047 [2023-12-27 09:00:28,866][20347] Updated weights for policy 0, policy_version 486367 (0.0006) [2023-12-27 09:00:29,134][20381] Updated weights for policy 1, policy_version 485978 (0.0008) [2023-12-27 09:00:29,195][20381] Updated weights for policy 1, policy_version 485988 (0.0006) [2023-12-27 09:00:29,257][20381] Updated weights for policy 1, policy_version 485998 (0.0008) [2023-12-27 09:00:29,605][20291] KL-divergence is very high: 125.9798 [2023-12-27 09:00:29,605][20347] Updated weights for policy 0, policy_version 486377 (0.0008) [2023-12-27 09:00:29,669][20347] Updated weights for policy 0, policy_version 486387 (0.0008) [2023-12-27 09:00:29,733][20347] Updated weights for policy 0, policy_version 486397 (0.0007) [2023-12-27 09:00:29,923][20381] Updated weights for policy 1, policy_version 486008 (0.0007) [2023-12-27 09:00:29,982][20381] Updated weights for policy 1, policy_version 486018 (0.0007) [2023-12-27 09:00:30,036][20381] Updated weights for policy 1, policy_version 486028 (0.0006) [2023-12-27 09:00:30,458][20347] Updated weights for policy 0, policy_version 486407 (0.0007) [2023-12-27 09:00:30,514][20347] Updated weights for policy 0, policy_version 486417 (0.0005) [2023-12-27 09:00:30,574][20347] Updated weights for policy 0, policy_version 486427 (0.0006) [2023-12-27 09:00:30,593][20291] KL-divergence is very high: 123.1609 [2023-12-27 09:00:30,708][20381] Updated weights for policy 1, policy_version 486038 (0.0006) [2023-12-27 09:00:30,756][20381] Updated weights for policy 1, policy_version 486048 (0.0005) [2023-12-27 09:00:30,804][20381] Updated weights for policy 1, policy_version 486058 (0.0006) [2023-12-27 09:00:31,269][20347] Updated weights for policy 0, policy_version 486437 (0.0007) [2023-12-27 09:00:31,289][20291] KL-divergence is very high: 122.2415 [2023-12-27 09:00:31,300][20291] KL-divergence is very high: 241.4358 [2023-12-27 09:00:31,312][20291] KL-divergence is very high: 123.1571 [2023-12-27 09:00:31,323][20347] Updated weights for policy 0, policy_version 486447 (0.0008) [2023-12-27 09:00:31,324][20291] KL-divergence is very high: 143.9356 [2023-12-27 09:00:31,337][20291] KL-divergence is very high: 105.5555 [2023-12-27 09:00:31,389][20347] Updated weights for policy 0, policy_version 486457 (0.0008) [2023-12-27 09:00:31,391][20291] KL-divergence is very high: 145.0602 [2023-12-27 09:00:31,480][20381] Updated weights for policy 1, policy_version 486068 (0.0006) [2023-12-27 09:00:31,562][20381] Updated weights for policy 1, policy_version 486078 (0.0006) [2023-12-27 09:00:31,621][20381] Updated weights for policy 1, policy_version 486088 (0.0006) [2023-12-27 09:00:32,045][20347] Updated weights for policy 0, policy_version 486467 (0.0007) [2023-12-27 09:00:32,065][20291] KL-divergence is very high: 249.0679 [2023-12-27 09:00:32,077][20291] KL-divergence is very high: 153.0938 [2023-12-27 09:00:32,107][20347] Updated weights for policy 0, policy_version 486477 (0.0007) [2023-12-27 09:00:32,126][20291] KL-divergence is very high: 113.4589 [2023-12-27 09:00:32,162][20291] KL-divergence is very high: 200.8936 [2023-12-27 09:00:32,169][20347] Updated weights for policy 0, policy_version 486487 (0.0008) [2023-12-27 09:00:32,211][20291] KL-divergence is very high: 116.1174 [2023-12-27 09:00:32,224][20381] Updated weights for policy 1, policy_version 486098 (0.0007) [2023-12-27 09:00:32,285][20381] Updated weights for policy 1, policy_version 486108 (0.0008) [2023-12-27 09:00:32,344][20381] Updated weights for policy 1, policy_version 486118 (0.0008) [2023-12-27 09:00:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 249020416. Throughput: 0: 10278.5, 1: 10317.9. Samples: 249031268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:32,365][19170] Avg episode reward: [(0, '424.590'), (1, '240.300')] [2023-12-27 09:00:32,411][20381] Updated weights for policy 1, policy_version 486128 (0.0008) [2023-12-27 09:00:32,855][20347] Updated weights for policy 0, policy_version 486497 (0.0007) [2023-12-27 09:00:32,916][20347] Updated weights for policy 0, policy_version 486507 (0.0005) [2023-12-27 09:00:32,951][20291] KL-divergence is very high: 129.5818 [2023-12-27 09:00:32,974][20347] Updated weights for policy 0, policy_version 486517 (0.0005) [2023-12-27 09:00:33,040][20347] Updated weights for policy 0, policy_version 486527 (0.0006) [2023-12-27 09:00:33,143][20381] Updated weights for policy 1, policy_version 486138 (0.0007) [2023-12-27 09:00:33,189][20381] Updated weights for policy 1, policy_version 486148 (0.0008) [2023-12-27 09:00:33,239][20381] Updated weights for policy 1, policy_version 486158 (0.0008) [2023-12-27 09:00:33,636][20347] Updated weights for policy 0, policy_version 486537 (0.0006) [2023-12-27 09:00:33,697][20347] Updated weights for policy 0, policy_version 486547 (0.0007) [2023-12-27 09:00:33,752][20347] Updated weights for policy 0, policy_version 486557 (0.0007) [2023-12-27 09:00:33,994][20381] Updated weights for policy 1, policy_version 486168 (0.0008) [2023-12-27 09:00:34,049][20381] Updated weights for policy 1, policy_version 486178 (0.0008) [2023-12-27 09:00:34,103][20381] Updated weights for policy 1, policy_version 486188 (0.0008) [2023-12-27 09:00:34,415][20347] Updated weights for policy 0, policy_version 486567 (0.0007) [2023-12-27 09:00:34,483][20347] Updated weights for policy 0, policy_version 486577 (0.0008) [2023-12-27 09:00:34,545][20347] Updated weights for policy 0, policy_version 486587 (0.0008) [2023-12-27 09:00:34,837][20381] Updated weights for policy 1, policy_version 486198 (0.0008) [2023-12-27 09:00:34,888][20381] Updated weights for policy 1, policy_version 486208 (0.0009) [2023-12-27 09:00:34,940][20381] Updated weights for policy 1, policy_version 486218 (0.0008) [2023-12-27 09:00:35,201][20347] Updated weights for policy 0, policy_version 486597 (0.0007) [2023-12-27 09:00:35,260][20347] Updated weights for policy 0, policy_version 486607 (0.0008) [2023-12-27 09:00:35,319][20347] Updated weights for policy 0, policy_version 486617 (0.0008) [2023-12-27 09:00:35,659][20381] Updated weights for policy 1, policy_version 486228 (0.0008) [2023-12-27 09:00:35,707][20381] Updated weights for policy 1, policy_version 486238 (0.0007) [2023-12-27 09:00:35,752][20381] Updated weights for policy 1, policy_version 486248 (0.0007) [2023-12-27 09:00:35,972][20347] Updated weights for policy 0, policy_version 486627 (0.0006) [2023-12-27 09:00:36,017][20347] Updated weights for policy 0, policy_version 486637 (0.0007) [2023-12-27 09:00:36,071][20347] Updated weights for policy 0, policy_version 486647 (0.0007) [2023-12-27 09:00:36,482][20381] Updated weights for policy 1, policy_version 486258 (0.0007) [2023-12-27 09:00:36,545][20381] Updated weights for policy 1, policy_version 486268 (0.0009) [2023-12-27 09:00:36,605][20381] Updated weights for policy 1, policy_version 486278 (0.0008) [2023-12-27 09:00:36,659][20381] Updated weights for policy 1, policy_version 486288 (0.0007) [2023-12-27 09:00:36,728][20347] Updated weights for policy 0, policy_version 486657 (0.0007) [2023-12-27 09:00:36,793][20347] Updated weights for policy 0, policy_version 486667 (0.0008) [2023-12-27 09:00:36,854][20347] Updated weights for policy 0, policy_version 486677 (0.0006) [2023-12-27 09:00:36,915][20347] Updated weights for policy 0, policy_version 486687 (0.0006) [2023-12-27 09:00:37,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20479.9, 300 sec: 20854.9). Total num frames: 249118720. Throughput: 0: 10282.4, 1: 10277.8. Samples: 249091216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:37,364][19170] Avg episode reward: [(0, '413.690'), (1, '160.940')] [2023-12-27 09:00:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000486688_124608512.pth... [2023-12-27 09:00:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000485504_124305408.pth [2023-12-27 09:00:37,397][20381] Updated weights for policy 1, policy_version 486298 (0.0011) [2023-12-27 09:00:37,447][20381] Updated weights for policy 1, policy_version 486308 (0.0010) [2023-12-27 09:00:37,494][20381] Updated weights for policy 1, policy_version 486318 (0.0010) [2023-12-27 09:00:37,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000486320_124518400.pth... [2023-12-27 09:00:37,509][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000485104_124207104.pth [2023-12-27 09:00:37,530][20347] Updated weights for policy 0, policy_version 486697 (0.0008) [2023-12-27 09:00:37,588][20347] Updated weights for policy 0, policy_version 486707 (0.0005) [2023-12-27 09:00:37,644][20291] KL-divergence is very high: 107.2422 [2023-12-27 09:00:37,648][20347] Updated weights for policy 0, policy_version 486717 (0.0005) [2023-12-27 09:00:38,204][20381] Updated weights for policy 1, policy_version 486328 (0.0008) [2023-12-27 09:00:38,233][20347] Updated weights for policy 0, policy_version 486727 (0.0008) [2023-12-27 09:00:38,241][20291] KL-divergence is very high: 193.2633 [2023-12-27 09:00:38,254][20381] Updated weights for policy 1, policy_version 486338 (0.0007) [2023-12-27 09:00:38,296][20347] Updated weights for policy 0, policy_version 486737 (0.0008) [2023-12-27 09:00:38,304][20291] KL-divergence is very high: 102.8220 [2023-12-27 09:00:38,311][20381] Updated weights for policy 1, policy_version 486348 (0.0006) [2023-12-27 09:00:38,365][20347] Updated weights for policy 0, policy_version 486747 (0.0008) [2023-12-27 09:00:38,936][20381] Updated weights for policy 1, policy_version 486358 (0.0007) [2023-12-27 09:00:38,996][20381] Updated weights for policy 1, policy_version 486368 (0.0008) [2023-12-27 09:00:39,057][20381] Updated weights for policy 1, policy_version 486378 (0.0007) [2023-12-27 09:00:39,080][20347] Updated weights for policy 0, policy_version 486757 (0.0006) [2023-12-27 09:00:39,130][20347] Updated weights for policy 0, policy_version 486767 (0.0007) [2023-12-27 09:00:39,181][20347] Updated weights for policy 0, policy_version 486777 (0.0010) [2023-12-27 09:00:39,718][20381] Updated weights for policy 1, policy_version 486388 (0.0007) [2023-12-27 09:00:39,774][20381] Updated weights for policy 1, policy_version 486398 (0.0008) [2023-12-27 09:00:39,835][20381] Updated weights for policy 1, policy_version 486408 (0.0007) [2023-12-27 09:00:39,879][20347] Updated weights for policy 0, policy_version 486787 (0.0009) [2023-12-27 09:00:39,943][20347] Updated weights for policy 0, policy_version 486797 (0.0007) [2023-12-27 09:00:39,995][20347] Updated weights for policy 0, policy_version 486807 (0.0005) [2023-12-27 09:00:40,579][20381] Updated weights for policy 1, policy_version 486418 (0.0008) [2023-12-27 09:00:40,624][20347] Updated weights for policy 0, policy_version 486817 (0.0006) [2023-12-27 09:00:40,630][20381] Updated weights for policy 1, policy_version 486428 (0.0007) [2023-12-27 09:00:40,674][20347] Updated weights for policy 0, policy_version 486827 (0.0005) [2023-12-27 09:00:40,683][20381] Updated weights for policy 1, policy_version 486438 (0.0006) [2023-12-27 09:00:40,723][20347] Updated weights for policy 0, policy_version 486837 (0.0006) [2023-12-27 09:00:40,744][20381] Updated weights for policy 1, policy_version 486448 (0.0005) [2023-12-27 09:00:40,770][20347] Updated weights for policy 0, policy_version 486847 (0.0007) [2023-12-27 09:00:41,451][20381] Updated weights for policy 1, policy_version 486458 (0.0008) [2023-12-27 09:00:41,514][20381] Updated weights for policy 1, policy_version 486468 (0.0006) [2023-12-27 09:00:41,528][20347] Updated weights for policy 0, policy_version 486857 (0.0006) [2023-12-27 09:00:41,544][20291] KL-divergence is very high: 117.6700 [2023-12-27 09:00:41,567][20291] KL-divergence is very high: 113.5283 [2023-12-27 09:00:41,572][20381] Updated weights for policy 1, policy_version 486478 (0.0006) [2023-12-27 09:00:41,580][20291] KL-divergence is very high: 154.4842 [2023-12-27 09:00:41,585][20347] Updated weights for policy 0, policy_version 486867 (0.0006) [2023-12-27 09:00:41,647][20347] Updated weights for policy 0, policy_version 486877 (0.0008) [2023-12-27 09:00:42,272][20347] Updated weights for policy 0, policy_version 486887 (0.0008) [2023-12-27 09:00:42,296][20381] Updated weights for policy 1, policy_version 486488 (0.0006) [2023-12-27 09:00:42,321][20347] Updated weights for policy 0, policy_version 486897 (0.0008) [2023-12-27 09:00:42,354][20381] Updated weights for policy 1, policy_version 486498 (0.0007) [2023-12-27 09:00:42,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.1, 300 sec: 20827.1). Total num frames: 249217024. Throughput: 0: 10303.0, 1: 10242.4. Samples: 249213636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:42,363][19170] Avg episode reward: [(0, '663.610'), (1, '132.560')] [2023-12-27 09:00:42,378][20347] Updated weights for policy 0, policy_version 486907 (0.0008) [2023-12-27 09:00:42,413][20381] Updated weights for policy 1, policy_version 486508 (0.0007) [2023-12-27 09:00:42,994][20347] Updated weights for policy 0, policy_version 486917 (0.0007) [2023-12-27 09:00:43,041][20347] Updated weights for policy 0, policy_version 486927 (0.0006) [2023-12-27 09:00:43,096][20347] Updated weights for policy 0, policy_version 486937 (0.0005) [2023-12-27 09:00:43,133][20381] Updated weights for policy 1, policy_version 486518 (0.0007) [2023-12-27 09:00:43,196][20381] Updated weights for policy 1, policy_version 486528 (0.0008) [2023-12-27 09:00:43,262][20381] Updated weights for policy 1, policy_version 486538 (0.0008) [2023-12-27 09:00:43,687][20347] Updated weights for policy 0, policy_version 486947 (0.0006) [2023-12-27 09:00:43,746][20347] Updated weights for policy 0, policy_version 486957 (0.0008) [2023-12-27 09:00:43,802][20347] Updated weights for policy 0, policy_version 486967 (0.0008) [2023-12-27 09:00:43,948][20381] Updated weights for policy 1, policy_version 486548 (0.0006) [2023-12-27 09:00:43,993][20381] Updated weights for policy 1, policy_version 486558 (0.0007) [2023-12-27 09:00:44,041][20381] Updated weights for policy 1, policy_version 486568 (0.0006) [2023-12-27 09:00:44,503][20347] Updated weights for policy 0, policy_version 486977 (0.0008) [2023-12-27 09:00:44,562][20347] Updated weights for policy 0, policy_version 486987 (0.0010) [2023-12-27 09:00:44,619][20347] Updated weights for policy 0, policy_version 486997 (0.0010) [2023-12-27 09:00:44,673][20347] Updated weights for policy 0, policy_version 487007 (0.0010) [2023-12-27 09:00:44,723][20381] Updated weights for policy 1, policy_version 486578 (0.0005) [2023-12-27 09:00:44,777][20381] Updated weights for policy 1, policy_version 486588 (0.0006) [2023-12-27 09:00:44,832][20381] Updated weights for policy 1, policy_version 486598 (0.0007) [2023-12-27 09:00:44,884][20381] Updated weights for policy 1, policy_version 486608 (0.0006) [2023-12-27 09:00:45,362][20347] Updated weights for policy 0, policy_version 487017 (0.0007) [2023-12-27 09:00:45,411][20347] Updated weights for policy 0, policy_version 487027 (0.0010) [2023-12-27 09:00:45,459][20347] Updated weights for policy 0, policy_version 487037 (0.0010) [2023-12-27 09:00:45,498][20381] Updated weights for policy 1, policy_version 486618 (0.0006) [2023-12-27 09:00:45,558][20381] Updated weights for policy 1, policy_version 486628 (0.0007) [2023-12-27 09:00:45,609][20381] Updated weights for policy 1, policy_version 486638 (0.0005) [2023-12-27 09:00:46,113][20347] Updated weights for policy 0, policy_version 487047 (0.0007) [2023-12-27 09:00:46,165][20347] Updated weights for policy 0, policy_version 487057 (0.0005) [2023-12-27 09:00:46,215][20347] Updated weights for policy 0, policy_version 487067 (0.0006) [2023-12-27 09:00:46,283][20381] Updated weights for policy 1, policy_version 486648 (0.0005) [2023-12-27 09:00:46,344][20381] Updated weights for policy 1, policy_version 486658 (0.0010) [2023-12-27 09:00:46,400][20381] Updated weights for policy 1, policy_version 486668 (0.0010) [2023-12-27 09:00:46,806][20347] Updated weights for policy 0, policy_version 487077 (0.0007) [2023-12-27 09:00:46,859][20347] Updated weights for policy 0, policy_version 487087 (0.0006) [2023-12-27 09:00:46,908][20347] Updated weights for policy 0, policy_version 487097 (0.0005) [2023-12-27 09:00:46,976][20381] Updated weights for policy 1, policy_version 486678 (0.0007) [2023-12-27 09:00:47,038][20381] Updated weights for policy 1, policy_version 486688 (0.0005) [2023-12-27 09:00:47,095][20381] Updated weights for policy 1, policy_version 486698 (0.0006) [2023-12-27 09:00:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 249331712. Throughput: 0: 10331.0, 1: 10264.2. Samples: 249340380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:47,363][19170] Avg episode reward: [(0, '783.190'), (1, '286.790')] [2023-12-27 09:00:47,453][20347] Updated weights for policy 0, policy_version 487107 (0.0007) [2023-12-27 09:00:47,507][20347] Updated weights for policy 0, policy_version 487117 (0.0010) [2023-12-27 09:00:47,552][20347] Updated weights for policy 0, policy_version 487127 (0.0010) [2023-12-27 09:00:47,688][20381] Updated weights for policy 1, policy_version 486708 (0.0006) [2023-12-27 09:00:47,747][20381] Updated weights for policy 1, policy_version 486718 (0.0006) [2023-12-27 09:00:47,810][20381] Updated weights for policy 1, policy_version 486728 (0.0005) [2023-12-27 09:00:48,187][20347] Updated weights for policy 0, policy_version 487137 (0.0010) [2023-12-27 09:00:48,248][20347] Updated weights for policy 0, policy_version 487147 (0.0010) [2023-12-27 09:00:48,309][20347] Updated weights for policy 0, policy_version 487157 (0.0006) [2023-12-27 09:00:48,370][20347] Updated weights for policy 0, policy_version 487167 (0.0010) [2023-12-27 09:00:48,381][20381] Updated weights for policy 1, policy_version 486738 (0.0005) [2023-12-27 09:00:48,431][20381] Updated weights for policy 1, policy_version 486748 (0.0006) [2023-12-27 09:00:48,485][20381] Updated weights for policy 1, policy_version 486758 (0.0007) [2023-12-27 09:00:48,534][20381] Updated weights for policy 1, policy_version 486768 (0.0008) [2023-12-27 09:00:49,020][20347] Updated weights for policy 0, policy_version 487177 (0.0008) [2023-12-27 09:00:49,072][20347] Updated weights for policy 0, policy_version 487187 (0.0010) [2023-12-27 09:00:49,122][20347] Updated weights for policy 0, policy_version 487197 (0.0008) [2023-12-27 09:00:49,223][20381] Updated weights for policy 1, policy_version 486778 (0.0006) [2023-12-27 09:00:49,282][20381] Updated weights for policy 1, policy_version 486788 (0.0007) [2023-12-27 09:00:49,345][20381] Updated weights for policy 1, policy_version 486798 (0.0007) [2023-12-27 09:00:49,864][20347] Updated weights for policy 0, policy_version 487207 (0.0009) [2023-12-27 09:00:49,921][20347] Updated weights for policy 0, policy_version 487217 (0.0009) [2023-12-27 09:00:49,981][20347] Updated weights for policy 0, policy_version 487227 (0.0010) [2023-12-27 09:00:50,031][20381] Updated weights for policy 1, policy_version 486808 (0.0007) [2023-12-27 09:00:50,090][20381] Updated weights for policy 1, policy_version 486818 (0.0007) [2023-12-27 09:00:50,157][20381] Updated weights for policy 1, policy_version 486828 (0.0008) [2023-12-27 09:00:50,697][20347] Updated weights for policy 0, policy_version 487237 (0.0011) [2023-12-27 09:00:50,756][20347] Updated weights for policy 0, policy_version 487247 (0.0009) [2023-12-27 09:00:50,796][20381] Updated weights for policy 1, policy_version 486838 (0.0007) [2023-12-27 09:00:50,805][20347] Updated weights for policy 0, policy_version 487257 (0.0007) [2023-12-27 09:00:50,846][20381] Updated weights for policy 1, policy_version 486848 (0.0010) [2023-12-27 09:00:50,895][20381] Updated weights for policy 1, policy_version 486858 (0.0010) [2023-12-27 09:00:51,552][20347] Updated weights for policy 0, policy_version 487267 (0.0007) [2023-12-27 09:00:51,604][20381] Updated weights for policy 1, policy_version 486868 (0.0011) [2023-12-27 09:00:51,612][20347] Updated weights for policy 0, policy_version 487277 (0.0007) [2023-12-27 09:00:51,671][20381] Updated weights for policy 1, policy_version 486878 (0.0008) [2023-12-27 09:00:51,671][20347] Updated weights for policy 0, policy_version 487287 (0.0010) [2023-12-27 09:00:51,739][20381] Updated weights for policy 1, policy_version 486888 (0.0009) [2023-12-27 09:00:52,339][20381] Updated weights for policy 1, policy_version 486898 (0.0009) [2023-12-27 09:00:52,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20827.1). Total num frames: 249430016. Throughput: 0: 10350.7, 1: 10268.4. Samples: 249403768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:52,363][19170] Avg episode reward: [(0, '847.950'), (1, '377.220')] [2023-12-27 09:00:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000487296_124764160.pth... [2023-12-27 09:00:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000486080_124452864.pth [2023-12-27 09:00:52,397][20347] Updated weights for policy 0, policy_version 487297 (0.0009) [2023-12-27 09:00:52,400][20381] Updated weights for policy 1, policy_version 486908 (0.0009) [2023-12-27 09:00:52,452][20347] Updated weights for policy 0, policy_version 487307 (0.0008) [2023-12-27 09:00:52,457][20381] Updated weights for policy 1, policy_version 486918 (0.0008) [2023-12-27 09:00:52,512][20347] Updated weights for policy 0, policy_version 487317 (0.0006) [2023-12-27 09:00:52,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000486928_124674048.pth... [2023-12-27 09:00:52,514][20381] Updated weights for policy 1, policy_version 486928 (0.0006) [2023-12-27 09:00:52,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000485712_124362752.pth [2023-12-27 09:00:52,570][20347] Updated weights for policy 0, policy_version 487327 (0.0008) [2023-12-27 09:00:53,232][20381] Updated weights for policy 1, policy_version 486938 (0.0007) [2023-12-27 09:00:53,264][20347] Updated weights for policy 0, policy_version 487337 (0.0007) [2023-12-27 09:00:53,289][20381] Updated weights for policy 1, policy_version 486948 (0.0005) [2023-12-27 09:00:53,327][20347] Updated weights for policy 0, policy_version 487347 (0.0005) [2023-12-27 09:00:53,348][20381] Updated weights for policy 1, policy_version 486958 (0.0005) [2023-12-27 09:00:53,383][20347] Updated weights for policy 0, policy_version 487357 (0.0008) [2023-12-27 09:00:53,956][20381] Updated weights for policy 1, policy_version 486968 (0.0007) [2023-12-27 09:00:54,014][20381] Updated weights for policy 1, policy_version 486978 (0.0008) [2023-12-27 09:00:54,050][20347] Updated weights for policy 0, policy_version 487367 (0.0007) [2023-12-27 09:00:54,072][20381] Updated weights for policy 1, policy_version 486988 (0.0009) [2023-12-27 09:00:54,105][20347] Updated weights for policy 0, policy_version 487377 (0.0007) [2023-12-27 09:00:54,160][20347] Updated weights for policy 0, policy_version 487387 (0.0005) [2023-12-27 09:00:54,769][20347] Updated weights for policy 0, policy_version 487397 (0.0005) [2023-12-27 09:00:54,796][20381] Updated weights for policy 1, policy_version 486998 (0.0006) [2023-12-27 09:00:54,819][20347] Updated weights for policy 0, policy_version 487407 (0.0006) [2023-12-27 09:00:54,855][20381] Updated weights for policy 1, policy_version 487008 (0.0005) [2023-12-27 09:00:54,870][20347] Updated weights for policy 0, policy_version 487417 (0.0007) [2023-12-27 09:00:54,913][20381] Updated weights for policy 1, policy_version 487018 (0.0006) [2023-12-27 09:00:55,536][20347] Updated weights for policy 0, policy_version 487427 (0.0006) [2023-12-27 09:00:55,583][20347] Updated weights for policy 0, policy_version 487437 (0.0007) [2023-12-27 09:00:55,592][20381] Updated weights for policy 1, policy_version 487028 (0.0007) [2023-12-27 09:00:55,633][20347] Updated weights for policy 0, policy_version 487447 (0.0006) [2023-12-27 09:00:55,642][20381] Updated weights for policy 1, policy_version 487038 (0.0006) [2023-12-27 09:00:55,688][20381] Updated weights for policy 1, policy_version 487048 (0.0008) [2023-12-27 09:00:56,341][20347] Updated weights for policy 0, policy_version 487457 (0.0007) [2023-12-27 09:00:56,394][20347] Updated weights for policy 0, policy_version 487467 (0.0007) [2023-12-27 09:00:56,396][20381] Updated weights for policy 1, policy_version 487058 (0.0010) [2023-12-27 09:00:56,443][20347] Updated weights for policy 0, policy_version 487477 (0.0007) [2023-12-27 09:00:56,445][20381] Updated weights for policy 1, policy_version 487068 (0.0005) [2023-12-27 09:00:56,492][20347] Updated weights for policy 0, policy_version 487487 (0.0006) [2023-12-27 09:00:56,496][20381] Updated weights for policy 1, policy_version 487078 (0.0005) [2023-12-27 09:00:56,550][20381] Updated weights for policy 1, policy_version 487088 (0.0005) [2023-12-27 09:00:57,154][20347] Updated weights for policy 0, policy_version 487497 (0.0006) [2023-12-27 09:00:57,173][20381] Updated weights for policy 1, policy_version 487098 (0.0011) [2023-12-27 09:00:57,213][20347] Updated weights for policy 0, policy_version 487507 (0.0007) [2023-12-27 09:00:57,228][20381] Updated weights for policy 1, policy_version 487108 (0.0010) [2023-12-27 09:00:57,262][20347] Updated weights for policy 0, policy_version 487517 (0.0006) [2023-12-27 09:00:57,282][20381] Updated weights for policy 1, policy_version 487118 (0.0010) [2023-12-27 09:00:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.2, 300 sec: 20882.7). Total num frames: 249544704. Throughput: 0: 10292.8, 1: 10268.6. Samples: 249526328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:00:57,363][19170] Avg episode reward: [(0, '757.770'), (1, '619.140')] [2023-12-27 09:00:57,860][20347] Updated weights for policy 0, policy_version 487527 (0.0005) [2023-12-27 09:00:57,924][20347] Updated weights for policy 0, policy_version 487537 (0.0005) [2023-12-27 09:00:57,966][20381] Updated weights for policy 1, policy_version 487128 (0.0006) [2023-12-27 09:00:57,975][20347] Updated weights for policy 0, policy_version 487547 (0.0005) [2023-12-27 09:00:58,021][20381] Updated weights for policy 1, policy_version 487138 (0.0006) [2023-12-27 09:00:58,079][20381] Updated weights for policy 1, policy_version 487148 (0.0005) [2023-12-27 09:00:58,646][20347] Updated weights for policy 0, policy_version 487557 (0.0006) [2023-12-27 09:00:58,709][20347] Updated weights for policy 0, policy_version 487567 (0.0007) [2023-12-27 09:00:58,754][20381] Updated weights for policy 1, policy_version 487158 (0.0008) [2023-12-27 09:00:58,781][20347] Updated weights for policy 0, policy_version 487577 (0.0007) [2023-12-27 09:00:58,821][20381] Updated weights for policy 1, policy_version 487168 (0.0012) [2023-12-27 09:00:58,893][20381] Updated weights for policy 1, policy_version 487178 (0.0009) [2023-12-27 09:00:59,512][20347] Updated weights for policy 0, policy_version 487587 (0.0006) [2023-12-27 09:00:59,575][20347] Updated weights for policy 0, policy_version 487597 (0.0006) [2023-12-27 09:00:59,647][20347] Updated weights for policy 0, policy_version 487607 (0.0005) [2023-12-27 09:00:59,653][20381] Updated weights for policy 1, policy_version 487188 (0.0008) [2023-12-27 09:00:59,712][20381] Updated weights for policy 1, policy_version 487198 (0.0006) [2023-12-27 09:00:59,767][20381] Updated weights for policy 1, policy_version 487208 (0.0006) [2023-12-27 09:01:00,289][20347] Updated weights for policy 0, policy_version 487617 (0.0007) [2023-12-27 09:01:00,347][20347] Updated weights for policy 0, policy_version 487627 (0.0008) [2023-12-27 09:01:00,414][20347] Updated weights for policy 0, policy_version 487637 (0.0007) [2023-12-27 09:01:00,428][20381] Updated weights for policy 1, policy_version 487218 (0.0009) [2023-12-27 09:01:00,463][20347] Updated weights for policy 0, policy_version 487647 (0.0006) [2023-12-27 09:01:00,483][20381] Updated weights for policy 1, policy_version 487228 (0.0006) [2023-12-27 09:01:00,535][20381] Updated weights for policy 1, policy_version 487238 (0.0008) [2023-12-27 09:01:00,581][20381] Updated weights for policy 1, policy_version 487248 (0.0007) [2023-12-27 09:01:01,164][20381] Updated weights for policy 1, policy_version 487258 (0.0011) [2023-12-27 09:01:01,206][20347] Updated weights for policy 0, policy_version 487657 (0.0006) [2023-12-27 09:01:01,213][20381] Updated weights for policy 1, policy_version 487268 (0.0009) [2023-12-27 09:01:01,265][20347] Updated weights for policy 0, policy_version 487667 (0.0008) [2023-12-27 09:01:01,274][20381] Updated weights for policy 1, policy_version 487278 (0.0006) [2023-12-27 09:01:01,324][20347] Updated weights for policy 0, policy_version 487677 (0.0007) [2023-12-27 09:01:01,985][20381] Updated weights for policy 1, policy_version 487288 (0.0010) [2023-12-27 09:01:02,004][20347] Updated weights for policy 0, policy_version 487687 (0.0007) [2023-12-27 09:01:02,044][20381] Updated weights for policy 1, policy_version 487298 (0.0010) [2023-12-27 09:01:02,064][20347] Updated weights for policy 0, policy_version 487697 (0.0005) [2023-12-27 09:01:02,102][20381] Updated weights for policy 1, policy_version 487308 (0.0008) [2023-12-27 09:01:02,123][20347] Updated weights for policy 0, policy_version 487707 (0.0006) [2023-12-27 09:01:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 249643008. Throughput: 0: 10366.7, 1: 10267.7. Samples: 249650076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:01:02,363][19170] Avg episode reward: [(0, '757.290'), (1, '580.010')] [2023-12-27 09:01:02,709][20347] Updated weights for policy 0, policy_version 487717 (0.0006) [2023-12-27 09:01:02,757][20381] Updated weights for policy 1, policy_version 487318 (0.0008) [2023-12-27 09:01:02,762][20347] Updated weights for policy 0, policy_version 487727 (0.0006) [2023-12-27 09:01:02,802][20381] Updated weights for policy 1, policy_version 487328 (0.0007) [2023-12-27 09:01:02,818][20347] Updated weights for policy 0, policy_version 487737 (0.0005) [2023-12-27 09:01:02,848][20381] Updated weights for policy 1, policy_version 487338 (0.0007) [2023-12-27 09:01:03,455][20347] Updated weights for policy 0, policy_version 487747 (0.0006) [2023-12-27 09:01:03,500][20347] Updated weights for policy 0, policy_version 487757 (0.0008) [2023-12-27 09:01:03,534][20381] Updated weights for policy 1, policy_version 487348 (0.0007) [2023-12-27 09:01:03,543][20347] Updated weights for policy 0, policy_version 487767 (0.0006) [2023-12-27 09:01:03,583][20381] Updated weights for policy 1, policy_version 487358 (0.0006) [2023-12-27 09:01:03,630][20381] Updated weights for policy 1, policy_version 487368 (0.0006) [2023-12-27 09:01:04,218][20347] Updated weights for policy 0, policy_version 487777 (0.0006) [2023-12-27 09:01:04,276][20347] Updated weights for policy 0, policy_version 487787 (0.0006) [2023-12-27 09:01:04,321][20381] Updated weights for policy 1, policy_version 487378 (0.0007) [2023-12-27 09:01:04,333][20347] Updated weights for policy 0, policy_version 487797 (0.0007) [2023-12-27 09:01:04,386][20381] Updated weights for policy 1, policy_version 487388 (0.0008) [2023-12-27 09:01:04,397][20347] Updated weights for policy 0, policy_version 487807 (0.0007) [2023-12-27 09:01:04,447][20381] Updated weights for policy 1, policy_version 487398 (0.0008) [2023-12-27 09:01:04,508][20381] Updated weights for policy 1, policy_version 487408 (0.0006) [2023-12-27 09:01:05,049][20347] Updated weights for policy 0, policy_version 487817 (0.0010) [2023-12-27 09:01:05,104][20347] Updated weights for policy 0, policy_version 487827 (0.0010) [2023-12-27 09:01:05,114][20381] Updated weights for policy 1, policy_version 487418 (0.0007) [2023-12-27 09:01:05,149][20347] Updated weights for policy 0, policy_version 487837 (0.0010) [2023-12-27 09:01:05,171][20381] Updated weights for policy 1, policy_version 487428 (0.0005) [2023-12-27 09:01:05,224][20381] Updated weights for policy 1, policy_version 487438 (0.0007) [2023-12-27 09:01:05,754][20347] Updated weights for policy 0, policy_version 487847 (0.0007) [2023-12-27 09:01:05,810][20347] Updated weights for policy 0, policy_version 487857 (0.0005) [2023-12-27 09:01:05,865][20347] Updated weights for policy 0, policy_version 487867 (0.0007) [2023-12-27 09:01:05,878][20381] Updated weights for policy 1, policy_version 487448 (0.0006) [2023-12-27 09:01:05,928][20381] Updated weights for policy 1, policy_version 487458 (0.0007) [2023-12-27 09:01:05,982][20381] Updated weights for policy 1, policy_version 487468 (0.0008) [2023-12-27 09:01:06,439][20347] Updated weights for policy 0, policy_version 487877 (0.0007) [2023-12-27 09:01:06,496][20347] Updated weights for policy 0, policy_version 487887 (0.0007) [2023-12-27 09:01:06,554][20347] Updated weights for policy 0, policy_version 487897 (0.0008) [2023-12-27 09:01:06,710][20381] Updated weights for policy 1, policy_version 487478 (0.0008) [2023-12-27 09:01:06,772][20381] Updated weights for policy 1, policy_version 487488 (0.0008) [2023-12-27 09:01:06,831][20381] Updated weights for policy 1, policy_version 487498 (0.0008) [2023-12-27 09:01:07,264][20347] Updated weights for policy 0, policy_version 487907 (0.0008) [2023-12-27 09:01:07,321][20347] Updated weights for policy 0, policy_version 487917 (0.0008) [2023-12-27 09:01:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20854.9). Total num frames: 249741312. Throughput: 0: 10429.1, 1: 10294.4. Samples: 249713932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:01:07,363][19170] Avg episode reward: [(0, '562.840'), (1, '694.590')] [2023-12-27 09:01:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000487504_124821504.pth... [2023-12-27 09:01:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000486320_124518400.pth [2023-12-27 09:01:07,378][20347] Updated weights for policy 0, policy_version 487927 (0.0007) [2023-12-27 09:01:07,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000487936_124928000.pth... [2023-12-27 09:01:07,429][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000486688_124608512.pth [2023-12-27 09:01:07,538][20381] Updated weights for policy 1, policy_version 487508 (0.0007) [2023-12-27 09:01:07,595][20381] Updated weights for policy 1, policy_version 487518 (0.0007) [2023-12-27 09:01:07,646][20381] Updated weights for policy 1, policy_version 487528 (0.0007) [2023-12-27 09:01:08,041][20347] Updated weights for policy 0, policy_version 487937 (0.0007) [2023-12-27 09:01:08,091][20347] Updated weights for policy 0, policy_version 487947 (0.0008) [2023-12-27 09:01:08,139][20347] Updated weights for policy 0, policy_version 487957 (0.0008) [2023-12-27 09:01:08,192][20347] Updated weights for policy 0, policy_version 487967 (0.0006) [2023-12-27 09:01:08,231][20381] Updated weights for policy 1, policy_version 487538 (0.0008) [2023-12-27 09:01:08,283][20381] Updated weights for policy 1, policy_version 487548 (0.0007) [2023-12-27 09:01:08,338][20381] Updated weights for policy 1, policy_version 487558 (0.0009) [2023-12-27 09:01:08,399][20381] Updated weights for policy 1, policy_version 487568 (0.0008) [2023-12-27 09:01:08,915][20347] Updated weights for policy 0, policy_version 487977 (0.0008) [2023-12-27 09:01:08,966][20347] Updated weights for policy 0, policy_version 487987 (0.0008) [2023-12-27 09:01:09,015][20347] Updated weights for policy 0, policy_version 487997 (0.0007) [2023-12-27 09:01:09,074][20381] Updated weights for policy 1, policy_version 487578 (0.0007) [2023-12-27 09:01:09,123][20381] Updated weights for policy 1, policy_version 487588 (0.0007) [2023-12-27 09:01:09,170][20381] Updated weights for policy 1, policy_version 487598 (0.0007) [2023-12-27 09:01:09,733][20347] Updated weights for policy 0, policy_version 488007 (0.0007) [2023-12-27 09:01:09,789][20347] Updated weights for policy 0, policy_version 488017 (0.0008) [2023-12-27 09:01:09,855][20347] Updated weights for policy 0, policy_version 488027 (0.0008) [2023-12-27 09:01:09,868][20381] Updated weights for policy 1, policy_version 487608 (0.0008) [2023-12-27 09:01:09,925][20381] Updated weights for policy 1, policy_version 487618 (0.0008) [2023-12-27 09:01:09,982][20381] Updated weights for policy 1, policy_version 487628 (0.0007) [2023-12-27 09:01:10,550][20347] Updated weights for policy 0, policy_version 488037 (0.0008) [2023-12-27 09:01:10,600][20347] Updated weights for policy 0, policy_version 488047 (0.0008) [2023-12-27 09:01:10,661][20347] Updated weights for policy 0, policy_version 488057 (0.0007) [2023-12-27 09:01:10,673][20381] Updated weights for policy 1, policy_version 487638 (0.0008) [2023-12-27 09:01:10,721][20381] Updated weights for policy 1, policy_version 487648 (0.0006) [2023-12-27 09:01:10,767][20381] Updated weights for policy 1, policy_version 487658 (0.0008) [2023-12-27 09:01:11,372][20347] Updated weights for policy 0, policy_version 488067 (0.0006) [2023-12-27 09:01:11,433][20347] Updated weights for policy 0, policy_version 488077 (0.0008) [2023-12-27 09:01:11,494][20347] Updated weights for policy 0, policy_version 488087 (0.0007) [2023-12-27 09:01:11,503][20381] Updated weights for policy 1, policy_version 487668 (0.0006) [2023-12-27 09:01:11,560][20381] Updated weights for policy 1, policy_version 487678 (0.0005) [2023-12-27 09:01:11,627][20381] Updated weights for policy 1, policy_version 487688 (0.0007) [2023-12-27 09:01:12,186][20347] Updated weights for policy 0, policy_version 488097 (0.0007) [2023-12-27 09:01:12,236][20347] Updated weights for policy 0, policy_version 488107 (0.0009) [2023-12-27 09:01:12,258][20381] Updated weights for policy 1, policy_version 487698 (0.0008) [2023-12-27 09:01:12,295][20347] Updated weights for policy 0, policy_version 488117 (0.0008) [2023-12-27 09:01:12,324][20381] Updated weights for policy 1, policy_version 487708 (0.0008) [2023-12-27 09:01:12,359][20347] Updated weights for policy 0, policy_version 488127 (0.0007) [2023-12-27 09:01:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20827.1). Total num frames: 249839616. Throughput: 0: 10418.6, 1: 10283.0. Samples: 249836908. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:12,363][19170] Avg episode reward: [(0, '478.500'), (1, '772.970')] [2023-12-27 09:01:12,392][20381] Updated weights for policy 1, policy_version 487718 (0.0008) [2023-12-27 09:01:12,442][20381] Updated weights for policy 1, policy_version 487728 (0.0006) [2023-12-27 09:01:13,019][20347] Updated weights for policy 0, policy_version 488137 (0.0007) [2023-12-27 09:01:13,067][20347] Updated weights for policy 0, policy_version 488147 (0.0009) [2023-12-27 09:01:13,119][20347] Updated weights for policy 0, policy_version 488157 (0.0009) [2023-12-27 09:01:13,148][20381] Updated weights for policy 1, policy_version 487738 (0.0006) [2023-12-27 09:01:13,201][20381] Updated weights for policy 1, policy_version 487748 (0.0007) [2023-12-27 09:01:13,253][20381] Updated weights for policy 1, policy_version 487758 (0.0005) [2023-12-27 09:01:13,708][20347] Updated weights for policy 0, policy_version 488167 (0.0009) [2023-12-27 09:01:13,755][20347] Updated weights for policy 0, policy_version 488177 (0.0007) [2023-12-27 09:01:13,798][20347] Updated weights for policy 0, policy_version 488187 (0.0007) [2023-12-27 09:01:13,876][20381] Updated weights for policy 1, policy_version 487768 (0.0009) [2023-12-27 09:01:13,921][20381] Updated weights for policy 1, policy_version 487778 (0.0010) [2023-12-27 09:01:13,974][20381] Updated weights for policy 1, policy_version 487788 (0.0010) [2023-12-27 09:01:14,432][20347] Updated weights for policy 0, policy_version 488197 (0.0007) [2023-12-27 09:01:14,496][20347] Updated weights for policy 0, policy_version 488207 (0.0008) [2023-12-27 09:01:14,559][20347] Updated weights for policy 0, policy_version 488217 (0.0008) [2023-12-27 09:01:14,658][20381] Updated weights for policy 1, policy_version 487798 (0.0010) [2023-12-27 09:01:14,720][20381] Updated weights for policy 1, policy_version 487808 (0.0010) [2023-12-27 09:01:14,780][20381] Updated weights for policy 1, policy_version 487818 (0.0009) [2023-12-27 09:01:15,225][20347] Updated weights for policy 0, policy_version 488227 (0.0007) [2023-12-27 09:01:15,279][20347] Updated weights for policy 0, policy_version 488237 (0.0007) [2023-12-27 09:01:15,329][20347] Updated weights for policy 0, policy_version 488247 (0.0006) [2023-12-27 09:01:15,462][20381] Updated weights for policy 1, policy_version 487828 (0.0008) [2023-12-27 09:01:15,522][20381] Updated weights for policy 1, policy_version 487838 (0.0010) [2023-12-27 09:01:15,583][20381] Updated weights for policy 1, policy_version 487848 (0.0010) [2023-12-27 09:01:15,960][20347] Updated weights for policy 0, policy_version 488257 (0.0007) [2023-12-27 09:01:16,017][20347] Updated weights for policy 0, policy_version 488267 (0.0010) [2023-12-27 09:01:16,069][20347] Updated weights for policy 0, policy_version 488277 (0.0010) [2023-12-27 09:01:16,121][20347] Updated weights for policy 0, policy_version 488287 (0.0009) [2023-12-27 09:01:16,215][20381] Updated weights for policy 1, policy_version 487858 (0.0009) [2023-12-27 09:01:16,263][20381] Updated weights for policy 1, policy_version 487868 (0.0009) [2023-12-27 09:01:16,321][20381] Updated weights for policy 1, policy_version 487878 (0.0010) [2023-12-27 09:01:16,374][20381] Updated weights for policy 1, policy_version 487888 (0.0010) [2023-12-27 09:01:16,840][20347] Updated weights for policy 0, policy_version 488297 (0.0008) [2023-12-27 09:01:16,902][20347] Updated weights for policy 0, policy_version 488307 (0.0008) [2023-12-27 09:01:16,950][20381] Updated weights for policy 1, policy_version 487898 (0.0007) [2023-12-27 09:01:16,965][20347] Updated weights for policy 0, policy_version 488317 (0.0008) [2023-12-27 09:01:16,997][20381] Updated weights for policy 1, policy_version 487908 (0.0008) [2023-12-27 09:01:17,044][20381] Updated weights for policy 1, policy_version 487918 (0.0006) [2023-12-27 09:01:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 249954304. Throughput: 0: 10437.0, 1: 10267.7. Samples: 249962976. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:17,363][19170] Avg episode reward: [(0, '465.560'), (1, '801.650')] [2023-12-27 09:01:17,571][20347] Updated weights for policy 0, policy_version 488327 (0.0007) [2023-12-27 09:01:17,631][20347] Updated weights for policy 0, policy_version 488337 (0.0007) [2023-12-27 09:01:17,641][20381] Updated weights for policy 1, policy_version 487928 (0.0009) [2023-12-27 09:01:17,687][20347] Updated weights for policy 0, policy_version 488347 (0.0005) [2023-12-27 09:01:17,689][20381] Updated weights for policy 1, policy_version 487938 (0.0010) [2023-12-27 09:01:17,744][20381] Updated weights for policy 1, policy_version 487948 (0.0009) [2023-12-27 09:01:18,290][20347] Updated weights for policy 0, policy_version 488357 (0.0006) [2023-12-27 09:01:18,350][20347] Updated weights for policy 0, policy_version 488367 (0.0007) [2023-12-27 09:01:18,380][20381] Updated weights for policy 1, policy_version 487958 (0.0006) [2023-12-27 09:01:18,403][20347] Updated weights for policy 0, policy_version 488377 (0.0008) [2023-12-27 09:01:18,441][20381] Updated weights for policy 1, policy_version 487968 (0.0009) [2023-12-27 09:01:18,496][20381] Updated weights for policy 1, policy_version 487978 (0.0008) [2023-12-27 09:01:19,032][20347] Updated weights for policy 0, policy_version 488387 (0.0009) [2023-12-27 09:01:19,086][20347] Updated weights for policy 0, policy_version 488397 (0.0007) [2023-12-27 09:01:19,147][20347] Updated weights for policy 0, policy_version 488407 (0.0007) [2023-12-27 09:01:19,185][20381] Updated weights for policy 1, policy_version 487988 (0.0007) [2023-12-27 09:01:19,242][20381] Updated weights for policy 1, policy_version 487998 (0.0007) [2023-12-27 09:01:19,297][20381] Updated weights for policy 1, policy_version 488008 (0.0008) [2023-12-27 09:01:19,849][20347] Updated weights for policy 0, policy_version 488417 (0.0007) [2023-12-27 09:01:19,924][20347] Updated weights for policy 0, policy_version 488427 (0.0007) [2023-12-27 09:01:19,984][20347] Updated weights for policy 0, policy_version 488437 (0.0007) [2023-12-27 09:01:20,021][20381] Updated weights for policy 1, policy_version 488018 (0.0008) [2023-12-27 09:01:20,043][20347] Updated weights for policy 0, policy_version 488447 (0.0006) [2023-12-27 09:01:20,080][20381] Updated weights for policy 1, policy_version 488028 (0.0008) [2023-12-27 09:01:20,142][20381] Updated weights for policy 1, policy_version 488038 (0.0008) [2023-12-27 09:01:20,202][20381] Updated weights for policy 1, policy_version 488048 (0.0008) [2023-12-27 09:01:20,747][20347] Updated weights for policy 0, policy_version 488457 (0.0007) [2023-12-27 09:01:20,804][20347] Updated weights for policy 0, policy_version 488467 (0.0007) [2023-12-27 09:01:20,860][20381] Updated weights for policy 1, policy_version 488058 (0.0008) [2023-12-27 09:01:20,860][20347] Updated weights for policy 0, policy_version 488477 (0.0007) [2023-12-27 09:01:20,909][20381] Updated weights for policy 1, policy_version 488068 (0.0008) [2023-12-27 09:01:20,961][20381] Updated weights for policy 1, policy_version 488078 (0.0008) [2023-12-27 09:01:21,571][20347] Updated weights for policy 0, policy_version 488487 (0.0007) [2023-12-27 09:01:21,635][20347] Updated weights for policy 0, policy_version 488497 (0.0007) [2023-12-27 09:01:21,690][20347] Updated weights for policy 0, policy_version 488507 (0.0008) [2023-12-27 09:01:21,728][20381] Updated weights for policy 1, policy_version 488088 (0.0007) [2023-12-27 09:01:21,793][20381] Updated weights for policy 1, policy_version 488098 (0.0008) [2023-12-27 09:01:21,845][20381] Updated weights for policy 1, policy_version 488108 (0.0006) [2023-12-27 09:01:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 250052608. Throughput: 0: 10450.3, 1: 10328.2. Samples: 250026244. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:22,363][19170] Avg episode reward: [(0, '819.320'), (1, '833.560')] [2023-12-27 09:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000488112_124977152.pth... [2023-12-27 09:01:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000488512_125075456.pth... [2023-12-27 09:01:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000486928_124674048.pth [2023-12-27 09:01:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000487296_124764160.pth [2023-12-27 09:01:22,464][20347] Updated weights for policy 0, policy_version 488517 (0.0007) [2023-12-27 09:01:22,526][20347] Updated weights for policy 0, policy_version 488527 (0.0007) [2023-12-27 09:01:22,542][20381] Updated weights for policy 1, policy_version 488118 (0.0008) [2023-12-27 09:01:22,588][20347] Updated weights for policy 0, policy_version 488537 (0.0008) [2023-12-27 09:01:22,593][20381] Updated weights for policy 1, policy_version 488128 (0.0008) [2023-12-27 09:01:22,648][20381] Updated weights for policy 1, policy_version 488138 (0.0006) [2023-12-27 09:01:23,265][20347] Updated weights for policy 0, policy_version 488547 (0.0006) [2023-12-27 09:01:23,317][20347] Updated weights for policy 0, policy_version 488557 (0.0005) [2023-12-27 09:01:23,357][20381] Updated weights for policy 1, policy_version 488148 (0.0008) [2023-12-27 09:01:23,370][20347] Updated weights for policy 0, policy_version 488567 (0.0005) [2023-12-27 09:01:23,422][20381] Updated weights for policy 1, policy_version 488158 (0.0005) [2023-12-27 09:01:23,482][20381] Updated weights for policy 1, policy_version 488168 (0.0005) [2023-12-27 09:01:23,993][20347] Updated weights for policy 0, policy_version 488577 (0.0006) [2023-12-27 09:01:24,016][20381] Updated weights for policy 1, policy_version 488178 (0.0007) [2023-12-27 09:01:24,048][20347] Updated weights for policy 0, policy_version 488587 (0.0005) [2023-12-27 09:01:24,071][20381] Updated weights for policy 1, policy_version 488188 (0.0010) [2023-12-27 09:01:24,109][20347] Updated weights for policy 0, policy_version 488597 (0.0006) [2023-12-27 09:01:24,137][20381] Updated weights for policy 1, policy_version 488198 (0.0010) [2023-12-27 09:01:24,175][20347] Updated weights for policy 0, policy_version 488607 (0.0006) [2023-12-27 09:01:24,202][20381] Updated weights for policy 1, policy_version 488208 (0.0011) [2023-12-27 09:01:24,769][20381] Updated weights for policy 1, policy_version 488218 (0.0011) [2023-12-27 09:01:24,815][20347] Updated weights for policy 0, policy_version 488617 (0.0006) [2023-12-27 09:01:24,818][20381] Updated weights for policy 1, policy_version 488228 (0.0008) [2023-12-27 09:01:24,866][20347] Updated weights for policy 0, policy_version 488627 (0.0008) [2023-12-27 09:01:24,875][20381] Updated weights for policy 1, policy_version 488238 (0.0006) [2023-12-27 09:01:24,911][20347] Updated weights for policy 0, policy_version 488637 (0.0010) [2023-12-27 09:01:25,482][20381] Updated weights for policy 1, policy_version 488248 (0.0005) [2023-12-27 09:01:25,531][20381] Updated weights for policy 1, policy_version 488258 (0.0005) [2023-12-27 09:01:25,581][20347] Updated weights for policy 0, policy_version 488647 (0.0010) [2023-12-27 09:01:25,589][20381] Updated weights for policy 1, policy_version 488268 (0.0005) [2023-12-27 09:01:25,642][20347] Updated weights for policy 0, policy_version 488657 (0.0010) [2023-12-27 09:01:25,709][20347] Updated weights for policy 0, policy_version 488667 (0.0010) [2023-12-27 09:01:26,197][20381] Updated weights for policy 1, policy_version 488278 (0.0008) [2023-12-27 09:01:26,251][20381] Updated weights for policy 1, policy_version 488288 (0.0010) [2023-12-27 09:01:26,307][20381] Updated weights for policy 1, policy_version 488298 (0.0011) [2023-12-27 09:01:26,357][20347] Updated weights for policy 0, policy_version 488677 (0.0010) [2023-12-27 09:01:26,411][20347] Updated weights for policy 0, policy_version 488687 (0.0010) [2023-12-27 09:01:26,462][20347] Updated weights for policy 0, policy_version 488697 (0.0010) [2023-12-27 09:01:26,900][20381] Updated weights for policy 1, policy_version 488308 (0.0008) [2023-12-27 09:01:26,958][20381] Updated weights for policy 1, policy_version 488318 (0.0005) [2023-12-27 09:01:27,017][20381] Updated weights for policy 1, policy_version 488328 (0.0006) [2023-12-27 09:01:27,140][20347] Updated weights for policy 0, policy_version 488707 (0.0009) [2023-12-27 09:01:27,199][20347] Updated weights for policy 0, policy_version 488717 (0.0008) [2023-12-27 09:01:27,262][20347] Updated weights for policy 0, policy_version 488727 (0.0009) [2023-12-27 09:01:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 250167296. Throughput: 0: 10410.0, 1: 10424.5. Samples: 250151192. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:27,363][19170] Avg episode reward: [(0, '845.670'), (1, '873.600')] [2023-12-27 09:01:27,611][20381] Updated weights for policy 1, policy_version 488338 (0.0007) [2023-12-27 09:01:27,654][20381] Updated weights for policy 1, policy_version 488348 (0.0005) [2023-12-27 09:01:27,705][20381] Updated weights for policy 1, policy_version 488358 (0.0005) [2023-12-27 09:01:27,760][20381] Updated weights for policy 1, policy_version 488368 (0.0005) [2023-12-27 09:01:27,899][20347] Updated weights for policy 0, policy_version 488737 (0.0010) [2023-12-27 09:01:27,946][20347] Updated weights for policy 0, policy_version 488747 (0.0010) [2023-12-27 09:01:27,991][20347] Updated weights for policy 0, policy_version 488757 (0.0010) [2023-12-27 09:01:28,038][20347] Updated weights for policy 0, policy_version 488767 (0.0010) [2023-12-27 09:01:28,335][20381] Updated weights for policy 1, policy_version 488378 (0.0007) [2023-12-27 09:01:28,387][20381] Updated weights for policy 1, policy_version 488388 (0.0010) [2023-12-27 09:01:28,444][20381] Updated weights for policy 1, policy_version 488398 (0.0010) [2023-12-27 09:01:28,736][20347] Updated weights for policy 0, policy_version 488777 (0.0010) [2023-12-27 09:01:28,785][20347] Updated weights for policy 0, policy_version 488787 (0.0011) [2023-12-27 09:01:28,833][20347] Updated weights for policy 0, policy_version 488797 (0.0010) [2023-12-27 09:01:29,096][20381] Updated weights for policy 1, policy_version 488408 (0.0010) [2023-12-27 09:01:29,147][20381] Updated weights for policy 1, policy_version 488418 (0.0010) [2023-12-27 09:01:29,198][20381] Updated weights for policy 1, policy_version 488428 (0.0010) [2023-12-27 09:01:29,524][20347] Updated weights for policy 0, policy_version 488807 (0.0010) [2023-12-27 09:01:29,583][20347] Updated weights for policy 0, policy_version 488817 (0.0011) [2023-12-27 09:01:29,628][20347] Updated weights for policy 0, policy_version 488827 (0.0010) [2023-12-27 09:01:29,876][20381] Updated weights for policy 1, policy_version 488438 (0.0008) [2023-12-27 09:01:29,938][20381] Updated weights for policy 1, policy_version 488448 (0.0007) [2023-12-27 09:01:29,999][20381] Updated weights for policy 1, policy_version 488458 (0.0007) [2023-12-27 09:01:30,328][20347] Updated weights for policy 0, policy_version 488837 (0.0010) [2023-12-27 09:01:30,379][20347] Updated weights for policy 0, policy_version 488847 (0.0010) [2023-12-27 09:01:30,433][20347] Updated weights for policy 0, policy_version 488857 (0.0010) [2023-12-27 09:01:30,641][20381] Updated weights for policy 1, policy_version 488468 (0.0007) [2023-12-27 09:01:30,685][20381] Updated weights for policy 1, policy_version 488478 (0.0008) [2023-12-27 09:01:30,745][20381] Updated weights for policy 1, policy_version 488488 (0.0010) [2023-12-27 09:01:31,099][20347] Updated weights for policy 0, policy_version 488867 (0.0009) [2023-12-27 09:01:31,162][20347] Updated weights for policy 0, policy_version 488877 (0.0007) [2023-12-27 09:01:31,219][20347] Updated weights for policy 0, policy_version 488887 (0.0006) [2023-12-27 09:01:31,318][20381] Updated weights for policy 1, policy_version 488498 (0.0009) [2023-12-27 09:01:31,383][20381] Updated weights for policy 1, policy_version 488508 (0.0008) [2023-12-27 09:01:31,433][20381] Updated weights for policy 1, policy_version 488518 (0.0006) [2023-12-27 09:01:31,490][20381] Updated weights for policy 1, policy_version 488528 (0.0009) [2023-12-27 09:01:31,879][20347] Updated weights for policy 0, policy_version 488897 (0.0010) [2023-12-27 09:01:31,930][20347] Updated weights for policy 0, policy_version 488907 (0.0009) [2023-12-27 09:01:31,991][20347] Updated weights for policy 0, policy_version 488917 (0.0008) [2023-12-27 09:01:32,053][20347] Updated weights for policy 0, policy_version 488927 (0.0007) [2023-12-27 09:01:32,130][20381] Updated weights for policy 1, policy_version 488538 (0.0008) [2023-12-27 09:01:32,179][20381] Updated weights for policy 1, policy_version 488548 (0.0007) [2023-12-27 09:01:32,229][20381] Updated weights for policy 1, policy_version 488558 (0.0007) [2023-12-27 09:01:32,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 250273792. Throughput: 0: 10354.9, 1: 10506.8. Samples: 250279156. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:32,363][19170] Avg episode reward: [(0, '922.950'), (1, '825.060')] [2023-12-27 09:01:32,734][20347] Updated weights for policy 0, policy_version 488937 (0.0007) [2023-12-27 09:01:32,792][20347] Updated weights for policy 0, policy_version 488947 (0.0008) [2023-12-27 09:01:32,849][20347] Updated weights for policy 0, policy_version 488957 (0.0008) [2023-12-27 09:01:32,926][20381] Updated weights for policy 1, policy_version 488568 (0.0007) [2023-12-27 09:01:32,984][20381] Updated weights for policy 1, policy_version 488578 (0.0008) [2023-12-27 09:01:33,039][20381] Updated weights for policy 1, policy_version 488588 (0.0008) [2023-12-27 09:01:33,475][20347] Updated weights for policy 0, policy_version 488967 (0.0007) [2023-12-27 09:01:33,522][20347] Updated weights for policy 0, policy_version 488977 (0.0008) [2023-12-27 09:01:33,568][20347] Updated weights for policy 0, policy_version 488987 (0.0008) [2023-12-27 09:01:33,740][20381] Updated weights for policy 1, policy_version 488598 (0.0006) [2023-12-27 09:01:33,787][20381] Updated weights for policy 1, policy_version 488608 (0.0005) [2023-12-27 09:01:33,835][20381] Updated weights for policy 1, policy_version 488618 (0.0005) [2023-12-27 09:01:34,221][20347] Updated weights for policy 0, policy_version 488997 (0.0009) [2023-12-27 09:01:34,281][20347] Updated weights for policy 0, policy_version 489007 (0.0010) [2023-12-27 09:01:34,343][20347] Updated weights for policy 0, policy_version 489017 (0.0010) [2023-12-27 09:01:34,457][20381] Updated weights for policy 1, policy_version 488628 (0.0007) [2023-12-27 09:01:34,516][20381] Updated weights for policy 1, policy_version 488638 (0.0011) [2023-12-27 09:01:34,577][20381] Updated weights for policy 1, policy_version 488648 (0.0011) [2023-12-27 09:01:34,966][20347] Updated weights for policy 0, policy_version 489027 (0.0009) [2023-12-27 09:01:35,025][20347] Updated weights for policy 0, policy_version 489037 (0.0007) [2023-12-27 09:01:35,080][20347] Updated weights for policy 0, policy_version 489047 (0.0008) [2023-12-27 09:01:35,232][20381] Updated weights for policy 1, policy_version 488658 (0.0011) [2023-12-27 09:01:35,290][20381] Updated weights for policy 1, policy_version 488668 (0.0011) [2023-12-27 09:01:35,339][20381] Updated weights for policy 1, policy_version 488678 (0.0010) [2023-12-27 09:01:35,389][20381] Updated weights for policy 1, policy_version 488688 (0.0011) [2023-12-27 09:01:35,705][20347] Updated weights for policy 0, policy_version 489057 (0.0006) [2023-12-27 09:01:35,761][20347] Updated weights for policy 0, policy_version 489067 (0.0007) [2023-12-27 09:01:35,811][20347] Updated weights for policy 0, policy_version 489077 (0.0007) [2023-12-27 09:01:35,869][20347] Updated weights for policy 0, policy_version 489087 (0.0006) [2023-12-27 09:01:36,057][20381] Updated weights for policy 1, policy_version 488698 (0.0007) [2023-12-27 09:01:36,119][20381] Updated weights for policy 1, policy_version 488708 (0.0008) [2023-12-27 09:01:36,177][20381] Updated weights for policy 1, policy_version 488718 (0.0007) [2023-12-27 09:01:36,504][20347] Updated weights for policy 0, policy_version 489097 (0.0008) [2023-12-27 09:01:36,562][20347] Updated weights for policy 0, policy_version 489107 (0.0008) [2023-12-27 09:01:36,614][20347] Updated weights for policy 0, policy_version 489117 (0.0007) [2023-12-27 09:01:36,898][20381] Updated weights for policy 1, policy_version 488728 (0.0008) [2023-12-27 09:01:36,955][20381] Updated weights for policy 1, policy_version 488738 (0.0008) [2023-12-27 09:01:37,010][20381] Updated weights for policy 1, policy_version 488748 (0.0007) [2023-12-27 09:01:37,336][20347] Updated weights for policy 0, policy_version 489127 (0.0006) [2023-12-27 09:01:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 250372096. Throughput: 0: 10369.1, 1: 10505.9. Samples: 250343140. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:37,363][19170] Avg episode reward: [(0, '961.760'), (1, '834.820')] [2023-12-27 09:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000488752_125140992.pth... [2023-12-27 09:01:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000487504_124821504.pth [2023-12-27 09:01:37,399][20347] Updated weights for policy 0, policy_version 489137 (0.0007) [2023-12-27 09:01:37,455][20347] Updated weights for policy 0, policy_version 489147 (0.0008) [2023-12-27 09:01:37,484][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000489152_125239296.pth... [2023-12-27 09:01:37,488][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000487936_124928000.pth [2023-12-27 09:01:37,635][20381] Updated weights for policy 1, policy_version 488758 (0.0007) [2023-12-27 09:01:37,687][20381] Updated weights for policy 1, policy_version 488768 (0.0008) [2023-12-27 09:01:37,737][20381] Updated weights for policy 1, policy_version 488778 (0.0008) [2023-12-27 09:01:38,118][20347] Updated weights for policy 0, policy_version 489157 (0.0008) [2023-12-27 09:01:38,167][20347] Updated weights for policy 0, policy_version 489167 (0.0007) [2023-12-27 09:01:38,226][20347] Updated weights for policy 0, policy_version 489177 (0.0008) [2023-12-27 09:01:38,384][20381] Updated weights for policy 1, policy_version 488788 (0.0008) [2023-12-27 09:01:38,437][20381] Updated weights for policy 1, policy_version 488798 (0.0008) [2023-12-27 09:01:38,489][20381] Updated weights for policy 1, policy_version 488808 (0.0006) [2023-12-27 09:01:38,985][20347] Updated weights for policy 0, policy_version 489187 (0.0007) [2023-12-27 09:01:39,047][20347] Updated weights for policy 0, policy_version 489197 (0.0005) [2023-12-27 09:01:39,106][20347] Updated weights for policy 0, policy_version 489207 (0.0005) [2023-12-27 09:01:39,107][20381] Updated weights for policy 1, policy_version 488818 (0.0006) [2023-12-27 09:01:39,161][20381] Updated weights for policy 1, policy_version 488828 (0.0008) [2023-12-27 09:01:39,222][20381] Updated weights for policy 1, policy_version 488838 (0.0009) [2023-12-27 09:01:39,286][20381] Updated weights for policy 1, policy_version 488848 (0.0007) [2023-12-27 09:01:39,743][20347] Updated weights for policy 0, policy_version 489217 (0.0006) [2023-12-27 09:01:39,802][20347] Updated weights for policy 0, policy_version 489227 (0.0008) [2023-12-27 09:01:39,867][20347] Updated weights for policy 0, policy_version 489237 (0.0008) [2023-12-27 09:01:39,925][20347] Updated weights for policy 0, policy_version 489247 (0.0007) [2023-12-27 09:01:40,054][20381] Updated weights for policy 1, policy_version 488858 (0.0008) [2023-12-27 09:01:40,106][20381] Updated weights for policy 1, policy_version 488868 (0.0008) [2023-12-27 09:01:40,160][20381] Updated weights for policy 1, policy_version 488878 (0.0007) [2023-12-27 09:01:40,573][20347] Updated weights for policy 0, policy_version 489257 (0.0006) [2023-12-27 09:01:40,625][20347] Updated weights for policy 0, policy_version 489267 (0.0009) [2023-12-27 09:01:40,676][20347] Updated weights for policy 0, policy_version 489277 (0.0010) [2023-12-27 09:01:40,868][20381] Updated weights for policy 1, policy_version 488888 (0.0006) [2023-12-27 09:01:40,932][20381] Updated weights for policy 1, policy_version 488898 (0.0005) [2023-12-27 09:01:40,988][20381] Updated weights for policy 1, policy_version 488908 (0.0005) [2023-12-27 09:01:41,340][20347] Updated weights for policy 0, policy_version 489287 (0.0010) [2023-12-27 09:01:41,400][20347] Updated weights for policy 0, policy_version 489297 (0.0010) [2023-12-27 09:01:41,459][20347] Updated weights for policy 0, policy_version 489307 (0.0010) [2023-12-27 09:01:41,623][20381] Updated weights for policy 1, policy_version 488918 (0.0008) [2023-12-27 09:01:41,680][20381] Updated weights for policy 1, policy_version 488928 (0.0008) [2023-12-27 09:01:41,746][20381] Updated weights for policy 1, policy_version 488938 (0.0008) [2023-12-27 09:01:42,195][20347] Updated weights for policy 0, policy_version 489317 (0.0010) [2023-12-27 09:01:42,258][20347] Updated weights for policy 0, policy_version 489327 (0.0008) [2023-12-27 09:01:42,314][20347] Updated weights for policy 0, policy_version 489337 (0.0008) [2023-12-27 09:01:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 250478592. Throughput: 0: 10394.3, 1: 10512.2. Samples: 250467120. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:42,363][19170] Avg episode reward: [(0, '904.670'), (1, '912.690')] [2023-12-27 09:01:42,437][20381] Updated weights for policy 1, policy_version 488948 (0.0007) [2023-12-27 09:01:42,493][20381] Updated weights for policy 1, policy_version 488958 (0.0008) [2023-12-27 09:01:42,548][20381] Updated weights for policy 1, policy_version 488968 (0.0007) [2023-12-27 09:01:42,967][20347] Updated weights for policy 0, policy_version 489347 (0.0007) [2023-12-27 09:01:43,029][20347] Updated weights for policy 0, policy_version 489357 (0.0008) [2023-12-27 09:01:43,093][20347] Updated weights for policy 0, policy_version 489367 (0.0010) [2023-12-27 09:01:43,207][20381] Updated weights for policy 1, policy_version 488978 (0.0007) [2023-12-27 09:01:43,267][20381] Updated weights for policy 1, policy_version 488988 (0.0005) [2023-12-27 09:01:43,328][20381] Updated weights for policy 1, policy_version 488998 (0.0005) [2023-12-27 09:01:43,386][20381] Updated weights for policy 1, policy_version 489008 (0.0005) [2023-12-27 09:01:43,805][20347] Updated weights for policy 0, policy_version 489377 (0.0007) [2023-12-27 09:01:43,855][20347] Updated weights for policy 0, policy_version 489387 (0.0006) [2023-12-27 09:01:43,878][20381] Updated weights for policy 1, policy_version 489018 (0.0007) [2023-12-27 09:01:43,901][20347] Updated weights for policy 0, policy_version 489397 (0.0008) [2023-12-27 09:01:43,933][20381] Updated weights for policy 1, policy_version 489028 (0.0006) [2023-12-27 09:01:43,946][20347] Updated weights for policy 0, policy_version 489407 (0.0005) [2023-12-27 09:01:43,983][20381] Updated weights for policy 1, policy_version 489038 (0.0007) [2023-12-27 09:01:44,596][20381] Updated weights for policy 1, policy_version 489048 (0.0009) [2023-12-27 09:01:44,649][20381] Updated weights for policy 1, policy_version 489058 (0.0010) [2023-12-27 09:01:44,675][20347] Updated weights for policy 0, policy_version 489417 (0.0006) [2023-12-27 09:01:44,710][20381] Updated weights for policy 1, policy_version 489068 (0.0009) [2023-12-27 09:01:44,732][20347] Updated weights for policy 0, policy_version 489427 (0.0007) [2023-12-27 09:01:44,797][20347] Updated weights for policy 0, policy_version 489437 (0.0008) [2023-12-27 09:01:45,411][20381] Updated weights for policy 1, policy_version 489078 (0.0008) [2023-12-27 09:01:45,446][20347] Updated weights for policy 0, policy_version 489447 (0.0007) [2023-12-27 09:01:45,468][20381] Updated weights for policy 1, policy_version 489088 (0.0006) [2023-12-27 09:01:45,508][20347] Updated weights for policy 0, policy_version 489457 (0.0007) [2023-12-27 09:01:45,526][20381] Updated weights for policy 1, policy_version 489098 (0.0007) [2023-12-27 09:01:45,565][20347] Updated weights for policy 0, policy_version 489467 (0.0006) [2023-12-27 09:01:46,160][20381] Updated weights for policy 1, policy_version 489108 (0.0006) [2023-12-27 09:01:46,208][20381] Updated weights for policy 1, policy_version 489118 (0.0005) [2023-12-27 09:01:46,256][20347] Updated weights for policy 0, policy_version 489477 (0.0006) [2023-12-27 09:01:46,259][20381] Updated weights for policy 1, policy_version 489128 (0.0006) [2023-12-27 09:01:46,303][20347] Updated weights for policy 0, policy_version 489487 (0.0005) [2023-12-27 09:01:46,365][20347] Updated weights for policy 0, policy_version 489497 (0.0007) [2023-12-27 09:01:46,820][20381] Updated weights for policy 1, policy_version 489138 (0.0005) [2023-12-27 09:01:46,877][20381] Updated weights for policy 1, policy_version 489148 (0.0005) [2023-12-27 09:01:46,935][20381] Updated weights for policy 1, policy_version 489158 (0.0005) [2023-12-27 09:01:46,987][20381] Updated weights for policy 1, policy_version 489168 (0.0005) [2023-12-27 09:01:47,013][20347] Updated weights for policy 0, policy_version 489507 (0.0007) [2023-12-27 09:01:47,059][20347] Updated weights for policy 0, policy_version 489517 (0.0005) [2023-12-27 09:01:47,108][20347] Updated weights for policy 0, policy_version 489527 (0.0005) [2023-12-27 09:01:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 250585088. Throughput: 0: 10371.8, 1: 10592.4. Samples: 250593468. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:47,363][19170] Avg episode reward: [(0, '885.450'), (1, '913.830')] [2023-12-27 09:01:47,640][20381] Updated weights for policy 1, policy_version 489178 (0.0008) [2023-12-27 09:01:47,695][20381] Updated weights for policy 1, policy_version 489188 (0.0008) [2023-12-27 09:01:47,697][20347] Updated weights for policy 0, policy_version 489537 (0.0006) [2023-12-27 09:01:47,743][20347] Updated weights for policy 0, policy_version 489547 (0.0007) [2023-12-27 09:01:47,757][20381] Updated weights for policy 1, policy_version 489198 (0.0005) [2023-12-27 09:01:47,796][20347] Updated weights for policy 0, policy_version 489557 (0.0006) [2023-12-27 09:01:47,852][20347] Updated weights for policy 0, policy_version 489567 (0.0005) [2023-12-27 09:01:48,386][20381] Updated weights for policy 1, policy_version 489208 (0.0007) [2023-12-27 09:01:48,447][20381] Updated weights for policy 1, policy_version 489218 (0.0008) [2023-12-27 09:01:48,498][20381] Updated weights for policy 1, policy_version 489228 (0.0007) [2023-12-27 09:01:48,523][20347] Updated weights for policy 0, policy_version 489577 (0.0007) [2023-12-27 09:01:48,582][20347] Updated weights for policy 0, policy_version 489587 (0.0007) [2023-12-27 09:01:48,642][20347] Updated weights for policy 0, policy_version 489597 (0.0006) [2023-12-27 09:01:49,171][20381] Updated weights for policy 1, policy_version 489238 (0.0007) [2023-12-27 09:01:49,223][20381] Updated weights for policy 1, policy_version 489248 (0.0007) [2023-12-27 09:01:49,282][20381] Updated weights for policy 1, policy_version 489258 (0.0007) [2023-12-27 09:01:49,346][20347] Updated weights for policy 0, policy_version 489607 (0.0007) [2023-12-27 09:01:49,416][20347] Updated weights for policy 0, policy_version 489617 (0.0007) [2023-12-27 09:01:49,477][20347] Updated weights for policy 0, policy_version 489627 (0.0008) [2023-12-27 09:01:49,937][20381] Updated weights for policy 1, policy_version 489268 (0.0008) [2023-12-27 09:01:49,997][20381] Updated weights for policy 1, policy_version 489278 (0.0009) [2023-12-27 09:01:50,056][20381] Updated weights for policy 1, policy_version 489288 (0.0008) [2023-12-27 09:01:50,170][20347] Updated weights for policy 0, policy_version 489637 (0.0007) [2023-12-27 09:01:50,231][20347] Updated weights for policy 0, policy_version 489647 (0.0007) [2023-12-27 09:01:50,279][20347] Updated weights for policy 0, policy_version 489657 (0.0007) [2023-12-27 09:01:50,766][20381] Updated weights for policy 1, policy_version 489298 (0.0008) [2023-12-27 09:01:50,827][20381] Updated weights for policy 1, policy_version 489308 (0.0007) [2023-12-27 09:01:50,890][20381] Updated weights for policy 1, policy_version 489318 (0.0008) [2023-12-27 09:01:50,947][20381] Updated weights for policy 1, policy_version 489328 (0.0008) [2023-12-27 09:01:50,966][20347] Updated weights for policy 0, policy_version 489667 (0.0008) [2023-12-27 09:01:51,026][20347] Updated weights for policy 0, policy_version 489677 (0.0008) [2023-12-27 09:01:51,091][20347] Updated weights for policy 0, policy_version 489687 (0.0008) [2023-12-27 09:01:51,647][20381] Updated weights for policy 1, policy_version 489338 (0.0007) [2023-12-27 09:01:51,701][20381] Updated weights for policy 1, policy_version 489348 (0.0007) [2023-12-27 09:01:51,770][20381] Updated weights for policy 1, policy_version 489358 (0.0007) [2023-12-27 09:01:51,830][20347] Updated weights for policy 0, policy_version 489697 (0.0008) [2023-12-27 09:01:51,887][20347] Updated weights for policy 0, policy_version 489707 (0.0006) [2023-12-27 09:01:51,943][20347] Updated weights for policy 0, policy_version 489717 (0.0006) [2023-12-27 09:01:52,001][20347] Updated weights for policy 0, policy_version 489727 (0.0008) [2023-12-27 09:01:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 250683392. Throughput: 0: 10338.2, 1: 10587.5. Samples: 250655592. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:52,363][19170] Avg episode reward: [(0, '884.810'), (1, '913.650')] [2023-12-27 09:01:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000489360_125296640.pth... [2023-12-27 09:01:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000489728_125386752.pth... [2023-12-27 09:01:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000488512_125075456.pth [2023-12-27 09:01:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000488112_124977152.pth [2023-12-27 09:01:52,460][20381] Updated weights for policy 1, policy_version 489368 (0.0006) [2023-12-27 09:01:52,515][20381] Updated weights for policy 1, policy_version 489378 (0.0007) [2023-12-27 09:01:52,572][20381] Updated weights for policy 1, policy_version 489388 (0.0008) [2023-12-27 09:01:52,685][20347] Updated weights for policy 0, policy_version 489737 (0.0008) [2023-12-27 09:01:52,745][20347] Updated weights for policy 0, policy_version 489747 (0.0007) [2023-12-27 09:01:52,809][20347] Updated weights for policy 0, policy_version 489757 (0.0008) [2023-12-27 09:01:53,237][20381] Updated weights for policy 1, policy_version 489398 (0.0005) [2023-12-27 09:01:53,284][20381] Updated weights for policy 1, policy_version 489408 (0.0007) [2023-12-27 09:01:53,341][20381] Updated weights for policy 1, policy_version 489418 (0.0008) [2023-12-27 09:01:53,515][20347] Updated weights for policy 0, policy_version 489767 (0.0007) [2023-12-27 09:01:53,568][20347] Updated weights for policy 0, policy_version 489777 (0.0007) [2023-12-27 09:01:53,626][20347] Updated weights for policy 0, policy_version 489787 (0.0007) [2023-12-27 09:01:54,012][20381] Updated weights for policy 1, policy_version 489428 (0.0009) [2023-12-27 09:01:54,057][20381] Updated weights for policy 1, policy_version 489438 (0.0010) [2023-12-27 09:01:54,114][20381] Updated weights for policy 1, policy_version 489448 (0.0010) [2023-12-27 09:01:54,324][20347] Updated weights for policy 0, policy_version 489797 (0.0007) [2023-12-27 09:01:54,386][20347] Updated weights for policy 0, policy_version 489807 (0.0008) [2023-12-27 09:01:54,436][20347] Updated weights for policy 0, policy_version 489817 (0.0007) [2023-12-27 09:01:54,758][20381] Updated weights for policy 1, policy_version 489458 (0.0010) [2023-12-27 09:01:54,817][20381] Updated weights for policy 1, policy_version 489468 (0.0008) [2023-12-27 09:01:54,872][20381] Updated weights for policy 1, policy_version 489478 (0.0007) [2023-12-27 09:01:54,931][20381] Updated weights for policy 1, policy_version 489488 (0.0008) [2023-12-27 09:01:55,141][20347] Updated weights for policy 0, policy_version 489827 (0.0008) [2023-12-27 09:01:55,192][20347] Updated weights for policy 0, policy_version 489837 (0.0008) [2023-12-27 09:01:55,242][20347] Updated weights for policy 0, policy_version 489847 (0.0008) [2023-12-27 09:01:55,578][20381] Updated weights for policy 1, policy_version 489498 (0.0008) [2023-12-27 09:01:55,636][20381] Updated weights for policy 1, policy_version 489508 (0.0008) [2023-12-27 09:01:55,692][20381] Updated weights for policy 1, policy_version 489518 (0.0009) [2023-12-27 09:01:55,867][20347] Updated weights for policy 0, policy_version 489857 (0.0007) [2023-12-27 09:01:55,922][20347] Updated weights for policy 0, policy_version 489867 (0.0005) [2023-12-27 09:01:55,979][20347] Updated weights for policy 0, policy_version 489877 (0.0005) [2023-12-27 09:01:56,027][20347] Updated weights for policy 0, policy_version 489887 (0.0008) [2023-12-27 09:01:56,376][20381] Updated weights for policy 1, policy_version 489528 (0.0006) [2023-12-27 09:01:56,430][20381] Updated weights for policy 1, policy_version 489538 (0.0007) [2023-12-27 09:01:56,475][20381] Updated weights for policy 1, policy_version 489548 (0.0006) [2023-12-27 09:01:56,643][20347] Updated weights for policy 0, policy_version 489897 (0.0005) [2023-12-27 09:01:56,688][20347] Updated weights for policy 0, policy_version 489907 (0.0006) [2023-12-27 09:01:56,747][20347] Updated weights for policy 0, policy_version 489917 (0.0007) [2023-12-27 09:01:57,112][20381] Updated weights for policy 1, policy_version 489558 (0.0006) [2023-12-27 09:01:57,161][20381] Updated weights for policy 1, policy_version 489568 (0.0008) [2023-12-27 09:01:57,212][20381] Updated weights for policy 1, policy_version 489578 (0.0007) [2023-12-27 09:01:57,326][20291] KL-divergence is very high: 124.5776 [2023-12-27 09:01:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 250789888. Throughput: 0: 10309.0, 1: 10601.6. Samples: 250777884. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:01:57,363][19170] Avg episode reward: [(0, '835.200'), (1, '912.810')] [2023-12-27 09:01:57,363][20347] Updated weights for policy 0, policy_version 489927 (0.0007) [2023-12-27 09:01:57,375][20291] KL-divergence is very high: 132.8109 [2023-12-27 09:01:57,414][20347] Updated weights for policy 0, policy_version 489937 (0.0007) [2023-12-27 09:01:57,463][20347] Updated weights for policy 0, policy_version 489947 (0.0009) [2023-12-27 09:01:57,886][20381] Updated weights for policy 1, policy_version 489588 (0.0008) [2023-12-27 09:01:57,941][20381] Updated weights for policy 1, policy_version 489598 (0.0009) [2023-12-27 09:01:57,996][20381] Updated weights for policy 1, policy_version 489608 (0.0008) [2023-12-27 09:01:58,091][20347] Updated weights for policy 0, policy_version 489957 (0.0008) [2023-12-27 09:01:58,150][20347] Updated weights for policy 0, policy_version 489967 (0.0006) [2023-12-27 09:01:58,214][20347] Updated weights for policy 0, policy_version 489977 (0.0008) [2023-12-27 09:01:58,710][20381] Updated weights for policy 1, policy_version 489618 (0.0008) [2023-12-27 09:01:58,772][20381] Updated weights for policy 1, policy_version 489628 (0.0008) [2023-12-27 09:01:58,833][20381] Updated weights for policy 1, policy_version 489638 (0.0007) [2023-12-27 09:01:58,902][20381] Updated weights for policy 1, policy_version 489648 (0.0008) [2023-12-27 09:01:58,929][20347] Updated weights for policy 0, policy_version 489987 (0.0007) [2023-12-27 09:01:58,989][20347] Updated weights for policy 0, policy_version 489997 (0.0008) [2023-12-27 09:01:59,046][20347] Updated weights for policy 0, policy_version 490007 (0.0008) [2023-12-27 09:01:59,599][20381] Updated weights for policy 1, policy_version 489658 (0.0007) [2023-12-27 09:01:59,655][20381] Updated weights for policy 1, policy_version 489668 (0.0007) [2023-12-27 09:01:59,716][20381] Updated weights for policy 1, policy_version 489678 (0.0005) [2023-12-27 09:01:59,769][20347] Updated weights for policy 0, policy_version 490017 (0.0008) [2023-12-27 09:01:59,829][20347] Updated weights for policy 0, policy_version 490027 (0.0009) [2023-12-27 09:01:59,891][20347] Updated weights for policy 0, policy_version 490037 (0.0008) [2023-12-27 09:01:59,950][20347] Updated weights for policy 0, policy_version 490047 (0.0008) [2023-12-27 09:02:00,303][20381] Updated weights for policy 1, policy_version 489688 (0.0006) [2023-12-27 09:02:00,359][20381] Updated weights for policy 1, policy_version 489698 (0.0008) [2023-12-27 09:02:00,414][20381] Updated weights for policy 1, policy_version 489708 (0.0006) [2023-12-27 09:02:00,634][20347] Updated weights for policy 0, policy_version 490057 (0.0007) [2023-12-27 09:02:00,683][20347] Updated weights for policy 0, policy_version 490067 (0.0007) [2023-12-27 09:02:00,732][20347] Updated weights for policy 0, policy_version 490077 (0.0009) [2023-12-27 09:02:00,918][20381] Updated weights for policy 1, policy_version 489718 (0.0005) [2023-12-27 09:02:00,963][20381] Updated weights for policy 1, policy_version 489728 (0.0005) [2023-12-27 09:02:01,018][20381] Updated weights for policy 1, policy_version 489738 (0.0006) [2023-12-27 09:02:01,446][20347] Updated weights for policy 0, policy_version 490087 (0.0008) [2023-12-27 09:02:01,498][20347] Updated weights for policy 0, policy_version 490097 (0.0008) [2023-12-27 09:02:01,550][20347] Updated weights for policy 0, policy_version 490107 (0.0008) [2023-12-27 09:02:01,695][20381] Updated weights for policy 1, policy_version 489748 (0.0008) [2023-12-27 09:02:01,759][20381] Updated weights for policy 1, policy_version 489758 (0.0007) [2023-12-27 09:02:01,818][20381] Updated weights for policy 1, policy_version 489768 (0.0007) [2023-12-27 09:02:02,209][20347] Updated weights for policy 0, policy_version 490117 (0.0008) [2023-12-27 09:02:02,268][20347] Updated weights for policy 0, policy_version 490127 (0.0008) [2023-12-27 09:02:02,323][20347] Updated weights for policy 0, policy_version 490137 (0.0008) [2023-12-27 09:02:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 250888192. Throughput: 0: 10291.8, 1: 10602.4. Samples: 250903212. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:02,363][19170] Avg episode reward: [(0, '883.770'), (1, '873.840')] [2023-12-27 09:02:02,504][20381] Updated weights for policy 1, policy_version 489778 (0.0008) [2023-12-27 09:02:02,556][20381] Updated weights for policy 1, policy_version 489788 (0.0007) [2023-12-27 09:02:02,608][20381] Updated weights for policy 1, policy_version 489798 (0.0007) [2023-12-27 09:02:02,656][20381] Updated weights for policy 1, policy_version 489808 (0.0007) [2023-12-27 09:02:03,040][20347] Updated weights for policy 0, policy_version 490147 (0.0008) [2023-12-27 09:02:03,097][20347] Updated weights for policy 0, policy_version 490157 (0.0008) [2023-12-27 09:02:03,154][20347] Updated weights for policy 0, policy_version 490167 (0.0008) [2023-12-27 09:02:03,333][20381] Updated weights for policy 1, policy_version 489818 (0.0008) [2023-12-27 09:02:03,381][20381] Updated weights for policy 1, policy_version 489828 (0.0006) [2023-12-27 09:02:03,427][20381] Updated weights for policy 1, policy_version 489838 (0.0007) [2023-12-27 09:02:03,818][20347] Updated weights for policy 0, policy_version 490177 (0.0007) [2023-12-27 09:02:03,872][20347] Updated weights for policy 0, policy_version 490187 (0.0007) [2023-12-27 09:02:03,921][20347] Updated weights for policy 0, policy_version 490197 (0.0008) [2023-12-27 09:02:03,984][20347] Updated weights for policy 0, policy_version 490207 (0.0008) [2023-12-27 09:02:04,081][20381] Updated weights for policy 1, policy_version 489848 (0.0007) [2023-12-27 09:02:04,139][20381] Updated weights for policy 1, policy_version 489858 (0.0008) [2023-12-27 09:02:04,201][20381] Updated weights for policy 1, policy_version 489868 (0.0008) [2023-12-27 09:02:04,724][20347] Updated weights for policy 0, policy_version 490217 (0.0009) [2023-12-27 09:02:04,778][20347] Updated weights for policy 0, policy_version 490227 (0.0010) [2023-12-27 09:02:04,830][20347] Updated weights for policy 0, policy_version 490237 (0.0010) [2023-12-27 09:02:04,907][20381] Updated weights for policy 1, policy_version 489878 (0.0010) [2023-12-27 09:02:04,971][20381] Updated weights for policy 1, policy_version 489888 (0.0010) [2023-12-27 09:02:05,010][20292] KL-divergence is very high: 111.2328 [2023-12-27 09:02:05,027][20381] Updated weights for policy 1, policy_version 489898 (0.0008) [2023-12-27 09:02:05,508][20347] Updated weights for policy 0, policy_version 490247 (0.0010) [2023-12-27 09:02:05,560][20347] Updated weights for policy 0, policy_version 490257 (0.0008) [2023-12-27 09:02:05,616][20347] Updated weights for policy 0, policy_version 490267 (0.0007) [2023-12-27 09:02:05,646][20381] Updated weights for policy 1, policy_version 489908 (0.0007) [2023-12-27 09:02:05,704][20381] Updated weights for policy 1, policy_version 489918 (0.0010) [2023-12-27 09:02:05,762][20381] Updated weights for policy 1, policy_version 489928 (0.0010) [2023-12-27 09:02:06,260][20347] Updated weights for policy 0, policy_version 490277 (0.0007) [2023-12-27 09:02:06,323][20347] Updated weights for policy 0, policy_version 490287 (0.0008) [2023-12-27 09:02:06,384][20347] Updated weights for policy 0, policy_version 490297 (0.0008) [2023-12-27 09:02:06,437][20381] Updated weights for policy 1, policy_version 489938 (0.0010) [2023-12-27 09:02:06,493][20381] Updated weights for policy 1, policy_version 489948 (0.0010) [2023-12-27 09:02:06,552][20381] Updated weights for policy 1, policy_version 489958 (0.0010) [2023-12-27 09:02:06,614][20381] Updated weights for policy 1, policy_version 489968 (0.0011) [2023-12-27 09:02:07,026][20347] Updated weights for policy 0, policy_version 490307 (0.0006) [2023-12-27 09:02:07,078][20347] Updated weights for policy 0, policy_version 490317 (0.0006) [2023-12-27 09:02:07,134][20347] Updated weights for policy 0, policy_version 490327 (0.0008) [2023-12-27 09:02:07,283][20381] Updated weights for policy 1, policy_version 489978 (0.0007) [2023-12-27 09:02:07,331][20381] Updated weights for policy 1, policy_version 489988 (0.0010) [2023-12-27 09:02:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 250994688. Throughput: 0: 10242.7, 1: 10605.6. Samples: 250964420. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:07,364][19170] Avg episode reward: [(0, '903.200'), (1, '844.530')] [2023-12-27 09:02:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000490336_125542400.pth... [2023-12-27 09:02:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000489152_125239296.pth [2023-12-27 09:02:07,385][20381] Updated weights for policy 1, policy_version 489998 (0.0010) [2023-12-27 09:02:07,394][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000490000_125460480.pth... [2023-12-27 09:02:07,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000488752_125140992.pth [2023-12-27 09:02:07,759][20347] Updated weights for policy 0, policy_version 490337 (0.0008) [2023-12-27 09:02:07,821][20347] Updated weights for policy 0, policy_version 490347 (0.0010) [2023-12-27 09:02:07,877][20347] Updated weights for policy 0, policy_version 490357 (0.0011) [2023-12-27 09:02:07,925][20347] Updated weights for policy 0, policy_version 490367 (0.0010) [2023-12-27 09:02:08,054][20381] Updated weights for policy 1, policy_version 490008 (0.0010) [2023-12-27 09:02:08,106][20381] Updated weights for policy 1, policy_version 490018 (0.0010) [2023-12-27 09:02:08,167][20381] Updated weights for policy 1, policy_version 490028 (0.0010) [2023-12-27 09:02:08,657][20347] Updated weights for policy 0, policy_version 490377 (0.0010) [2023-12-27 09:02:08,723][20347] Updated weights for policy 0, policy_version 490387 (0.0010) [2023-12-27 09:02:08,778][20347] Updated weights for policy 0, policy_version 490397 (0.0010) [2023-12-27 09:02:08,871][20381] Updated weights for policy 1, policy_version 490038 (0.0011) [2023-12-27 09:02:08,921][20381] Updated weights for policy 1, policy_version 490048 (0.0010) [2023-12-27 09:02:08,982][20381] Updated weights for policy 1, policy_version 490058 (0.0006) [2023-12-27 09:02:09,429][20347] Updated weights for policy 0, policy_version 490407 (0.0008) [2023-12-27 09:02:09,486][20347] Updated weights for policy 0, policy_version 490417 (0.0007) [2023-12-27 09:02:09,547][20347] Updated weights for policy 0, policy_version 490427 (0.0008) [2023-12-27 09:02:09,662][20381] Updated weights for policy 1, policy_version 490068 (0.0008) [2023-12-27 09:02:09,727][20381] Updated weights for policy 1, policy_version 490078 (0.0011) [2023-12-27 09:02:09,792][20381] Updated weights for policy 1, policy_version 490088 (0.0011) [2023-12-27 09:02:10,306][20347] Updated weights for policy 0, policy_version 490437 (0.0009) [2023-12-27 09:02:10,356][20381] Updated weights for policy 1, policy_version 490098 (0.0008) [2023-12-27 09:02:10,364][20347] Updated weights for policy 0, policy_version 490447 (0.0008) [2023-12-27 09:02:10,420][20381] Updated weights for policy 1, policy_version 490108 (0.0010) [2023-12-27 09:02:10,423][20347] Updated weights for policy 0, policy_version 490457 (0.0006) [2023-12-27 09:02:10,482][20381] Updated weights for policy 1, policy_version 490118 (0.0010) [2023-12-27 09:02:10,539][20381] Updated weights for policy 1, policy_version 490128 (0.0010) [2023-12-27 09:02:11,132][20347] Updated weights for policy 0, policy_version 490467 (0.0008) [2023-12-27 09:02:11,197][20347] Updated weights for policy 0, policy_version 490477 (0.0008) [2023-12-27 09:02:11,207][20381] Updated weights for policy 1, policy_version 490138 (0.0009) [2023-12-27 09:02:11,260][20347] Updated weights for policy 0, policy_version 490487 (0.0007) [2023-12-27 09:02:11,267][20381] Updated weights for policy 1, policy_version 490148 (0.0010) [2023-12-27 09:02:11,328][20381] Updated weights for policy 1, policy_version 490158 (0.0008) [2023-12-27 09:02:12,018][20381] Updated weights for policy 1, policy_version 490168 (0.0008) [2023-12-27 09:02:12,025][20347] Updated weights for policy 0, policy_version 490497 (0.0006) [2023-12-27 09:02:12,070][20381] Updated weights for policy 1, policy_version 490178 (0.0006) [2023-12-27 09:02:12,090][20347] Updated weights for policy 0, policy_version 490507 (0.0006) [2023-12-27 09:02:12,129][20381] Updated weights for policy 1, policy_version 490188 (0.0006) [2023-12-27 09:02:12,151][20347] Updated weights for policy 0, policy_version 490517 (0.0007) [2023-12-27 09:02:12,210][20347] Updated weights for policy 0, policy_version 490527 (0.0009) [2023-12-27 09:02:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 251101184. Throughput: 0: 10253.9, 1: 10561.2. Samples: 251087876. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:12,363][19170] Avg episode reward: [(0, '903.810'), (1, '854.220')] [2023-12-27 09:02:12,711][20381] Updated weights for policy 1, policy_version 490198 (0.0005) [2023-12-27 09:02:12,766][20381] Updated weights for policy 1, policy_version 490208 (0.0005) [2023-12-27 09:02:12,823][20381] Updated weights for policy 1, policy_version 490218 (0.0005) [2023-12-27 09:02:12,964][20347] Updated weights for policy 0, policy_version 490537 (0.0008) [2023-12-27 09:02:13,020][20347] Updated weights for policy 0, policy_version 490547 (0.0009) [2023-12-27 09:02:13,080][20347] Updated weights for policy 0, policy_version 490557 (0.0007) [2023-12-27 09:02:13,364][20381] Updated weights for policy 1, policy_version 490228 (0.0006) [2023-12-27 09:02:13,419][20381] Updated weights for policy 1, policy_version 490238 (0.0008) [2023-12-27 09:02:13,474][20381] Updated weights for policy 1, policy_version 490248 (0.0007) [2023-12-27 09:02:13,810][20347] Updated weights for policy 0, policy_version 490567 (0.0008) [2023-12-27 09:02:13,875][20347] Updated weights for policy 0, policy_version 490577 (0.0008) [2023-12-27 09:02:13,938][20347] Updated weights for policy 0, policy_version 490587 (0.0008) [2023-12-27 09:02:14,142][20381] Updated weights for policy 1, policy_version 490258 (0.0008) [2023-12-27 09:02:14,197][20381] Updated weights for policy 1, policy_version 490268 (0.0008) [2023-12-27 09:02:14,246][20381] Updated weights for policy 1, policy_version 490278 (0.0007) [2023-12-27 09:02:14,295][20381] Updated weights for policy 1, policy_version 490288 (0.0008) [2023-12-27 09:02:14,617][20347] Updated weights for policy 0, policy_version 490597 (0.0007) [2023-12-27 09:02:14,673][20347] Updated weights for policy 0, policy_version 490607 (0.0008) [2023-12-27 09:02:14,718][20347] Updated weights for policy 0, policy_version 490617 (0.0007) [2023-12-27 09:02:14,983][20381] Updated weights for policy 1, policy_version 490298 (0.0007) [2023-12-27 09:02:15,051][20381] Updated weights for policy 1, policy_version 490308 (0.0009) [2023-12-27 09:02:15,110][20381] Updated weights for policy 1, policy_version 490318 (0.0007) [2023-12-27 09:02:15,474][20347] Updated weights for policy 0, policy_version 490627 (0.0008) [2023-12-27 09:02:15,526][20347] Updated weights for policy 0, policy_version 490637 (0.0009) [2023-12-27 09:02:15,576][20347] Updated weights for policy 0, policy_version 490647 (0.0007) [2023-12-27 09:02:15,667][20381] Updated weights for policy 1, policy_version 490328 (0.0006) [2023-12-27 09:02:15,715][20381] Updated weights for policy 1, policy_version 490338 (0.0005) [2023-12-27 09:02:15,762][20381] Updated weights for policy 1, policy_version 490348 (0.0005) [2023-12-27 09:02:16,169][20347] Updated weights for policy 0, policy_version 490657 (0.0007) [2023-12-27 09:02:16,223][20347] Updated weights for policy 0, policy_version 490667 (0.0005) [2023-12-27 09:02:16,272][20347] Updated weights for policy 0, policy_version 490677 (0.0006) [2023-12-27 09:02:16,299][20381] Updated weights for policy 1, policy_version 490358 (0.0007) [2023-12-27 09:02:16,327][20347] Updated weights for policy 0, policy_version 490687 (0.0006) [2023-12-27 09:02:16,349][20381] Updated weights for policy 1, policy_version 490368 (0.0006) [2023-12-27 09:02:16,399][20381] Updated weights for policy 1, policy_version 490378 (0.0006) [2023-12-27 09:02:16,977][20347] Updated weights for policy 0, policy_version 490697 (0.0006) [2023-12-27 09:02:17,037][20347] Updated weights for policy 0, policy_version 490707 (0.0007) [2023-12-27 09:02:17,058][20381] Updated weights for policy 1, policy_version 490388 (0.0008) [2023-12-27 09:02:17,090][20347] Updated weights for policy 0, policy_version 490717 (0.0006) [2023-12-27 09:02:17,116][20381] Updated weights for policy 1, policy_version 490398 (0.0010) [2023-12-27 09:02:17,171][20381] Updated weights for policy 1, policy_version 490408 (0.0010) [2023-12-27 09:02:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 251207680. Throughput: 0: 10190.1, 1: 10582.7. Samples: 251213936. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:17,363][19170] Avg episode reward: [(0, '847.010'), (1, '873.860')] [2023-12-27 09:02:17,628][20347] Updated weights for policy 0, policy_version 490727 (0.0009) [2023-12-27 09:02:17,676][20347] Updated weights for policy 0, policy_version 490737 (0.0010) [2023-12-27 09:02:17,726][20381] Updated weights for policy 1, policy_version 490418 (0.0009) [2023-12-27 09:02:17,739][20347] Updated weights for policy 0, policy_version 490747 (0.0008) [2023-12-27 09:02:17,775][20381] Updated weights for policy 1, policy_version 490428 (0.0006) [2023-12-27 09:02:17,833][20381] Updated weights for policy 1, policy_version 490438 (0.0007) [2023-12-27 09:02:17,891][20381] Updated weights for policy 1, policy_version 490448 (0.0008) [2023-12-27 09:02:18,461][20347] Updated weights for policy 0, policy_version 490757 (0.0008) [2023-12-27 09:02:18,520][20347] Updated weights for policy 0, policy_version 490767 (0.0008) [2023-12-27 09:02:18,530][20381] Updated weights for policy 1, policy_version 490458 (0.0007) [2023-12-27 09:02:18,579][20347] Updated weights for policy 0, policy_version 490777 (0.0007) [2023-12-27 09:02:18,591][20381] Updated weights for policy 1, policy_version 490468 (0.0005) [2023-12-27 09:02:18,642][20381] Updated weights for policy 1, policy_version 490478 (0.0007) [2023-12-27 09:02:19,274][20347] Updated weights for policy 0, policy_version 490787 (0.0007) [2023-12-27 09:02:19,333][20381] Updated weights for policy 1, policy_version 490488 (0.0006) [2023-12-27 09:02:19,335][20347] Updated weights for policy 0, policy_version 490797 (0.0007) [2023-12-27 09:02:19,395][20381] Updated weights for policy 1, policy_version 490498 (0.0006) [2023-12-27 09:02:19,396][20347] Updated weights for policy 0, policy_version 490807 (0.0007) [2023-12-27 09:02:19,451][20381] Updated weights for policy 1, policy_version 490508 (0.0005) [2023-12-27 09:02:20,073][20347] Updated weights for policy 0, policy_version 490817 (0.0005) [2023-12-27 09:02:20,135][20347] Updated weights for policy 0, policy_version 490827 (0.0006) [2023-12-27 09:02:20,148][20381] Updated weights for policy 1, policy_version 490518 (0.0008) [2023-12-27 09:02:20,197][20347] Updated weights for policy 0, policy_version 490837 (0.0006) [2023-12-27 09:02:20,206][20381] Updated weights for policy 1, policy_version 490528 (0.0007) [2023-12-27 09:02:20,257][20347] Updated weights for policy 0, policy_version 490847 (0.0007) [2023-12-27 09:02:20,259][20381] Updated weights for policy 1, policy_version 490538 (0.0006) [2023-12-27 09:02:20,895][20381] Updated weights for policy 1, policy_version 490548 (0.0007) [2023-12-27 09:02:20,907][20347] Updated weights for policy 0, policy_version 490857 (0.0009) [2023-12-27 09:02:20,952][20381] Updated weights for policy 1, policy_version 490558 (0.0008) [2023-12-27 09:02:20,968][20347] Updated weights for policy 0, policy_version 490867 (0.0007) [2023-12-27 09:02:21,016][20381] Updated weights for policy 1, policy_version 490568 (0.0006) [2023-12-27 09:02:21,028][20347] Updated weights for policy 0, policy_version 490877 (0.0008) [2023-12-27 09:02:21,720][20381] Updated weights for policy 1, policy_version 490578 (0.0009) [2023-12-27 09:02:21,785][20381] Updated weights for policy 1, policy_version 490588 (0.0007) [2023-12-27 09:02:21,806][20347] Updated weights for policy 0, policy_version 490887 (0.0007) [2023-12-27 09:02:21,840][20381] Updated weights for policy 1, policy_version 490598 (0.0006) [2023-12-27 09:02:21,865][20347] Updated weights for policy 0, policy_version 490897 (0.0006) [2023-12-27 09:02:21,902][20381] Updated weights for policy 1, policy_version 490608 (0.0006) [2023-12-27 09:02:21,922][20347] Updated weights for policy 0, policy_version 490907 (0.0006) [2023-12-27 09:02:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 251305984. Throughput: 0: 10163.5, 1: 10589.2. Samples: 251277016. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:22,363][19170] Avg episode reward: [(0, '904.180'), (1, '836.370')] [2023-12-27 09:02:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000490608_125616128.pth... [2023-12-27 09:02:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000490912_125689856.pth... [2023-12-27 09:02:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000489728_125386752.pth [2023-12-27 09:02:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000489360_125296640.pth [2023-12-27 09:02:22,566][20381] Updated weights for policy 1, policy_version 490618 (0.0005) [2023-12-27 09:02:22,623][20381] Updated weights for policy 1, policy_version 490628 (0.0006) [2023-12-27 09:02:22,641][20347] Updated weights for policy 0, policy_version 490917 (0.0007) [2023-12-27 09:02:22,689][20381] Updated weights for policy 1, policy_version 490638 (0.0006) [2023-12-27 09:02:22,704][20347] Updated weights for policy 0, policy_version 490927 (0.0006) [2023-12-27 09:02:22,765][20347] Updated weights for policy 0, policy_version 490937 (0.0006) [2023-12-27 09:02:23,288][20381] Updated weights for policy 1, policy_version 490648 (0.0007) [2023-12-27 09:02:23,335][20381] Updated weights for policy 1, policy_version 490658 (0.0007) [2023-12-27 09:02:23,379][20381] Updated weights for policy 1, policy_version 490668 (0.0007) [2023-12-27 09:02:23,388][20347] Updated weights for policy 0, policy_version 490947 (0.0007) [2023-12-27 09:02:23,444][20347] Updated weights for policy 0, policy_version 490957 (0.0007) [2023-12-27 09:02:23,494][20347] Updated weights for policy 0, policy_version 490967 (0.0008) [2023-12-27 09:02:24,110][20381] Updated weights for policy 1, policy_version 490678 (0.0006) [2023-12-27 09:02:24,136][20347] Updated weights for policy 0, policy_version 490977 (0.0008) [2023-12-27 09:02:24,170][20381] Updated weights for policy 1, policy_version 490688 (0.0007) [2023-12-27 09:02:24,195][20347] Updated weights for policy 0, policy_version 490987 (0.0011) [2023-12-27 09:02:24,227][20381] Updated weights for policy 1, policy_version 490698 (0.0006) [2023-12-27 09:02:24,258][20347] Updated weights for policy 0, policy_version 490997 (0.0010) [2023-12-27 09:02:24,307][20347] Updated weights for policy 0, policy_version 491007 (0.0010) [2023-12-27 09:02:24,842][20381] Updated weights for policy 1, policy_version 490708 (0.0008) [2023-12-27 09:02:24,897][20381] Updated weights for policy 1, policy_version 490718 (0.0007) [2023-12-27 09:02:24,956][20381] Updated weights for policy 1, policy_version 490728 (0.0007) [2023-12-27 09:02:24,990][20347] Updated weights for policy 0, policy_version 491017 (0.0010) [2023-12-27 09:02:25,048][20347] Updated weights for policy 0, policy_version 491027 (0.0009) [2023-12-27 09:02:25,107][20347] Updated weights for policy 0, policy_version 491037 (0.0007) [2023-12-27 09:02:25,658][20381] Updated weights for policy 1, policy_version 490738 (0.0007) [2023-12-27 09:02:25,720][20347] Updated weights for policy 0, policy_version 491047 (0.0005) [2023-12-27 09:02:25,723][20381] Updated weights for policy 1, policy_version 490748 (0.0005) [2023-12-27 09:02:25,776][20347] Updated weights for policy 0, policy_version 491057 (0.0005) [2023-12-27 09:02:25,782][20381] Updated weights for policy 1, policy_version 490758 (0.0006) [2023-12-27 09:02:25,835][20347] Updated weights for policy 0, policy_version 491067 (0.0005) [2023-12-27 09:02:25,843][20381] Updated weights for policy 1, policy_version 490768 (0.0006) [2023-12-27 09:02:26,395][20381] Updated weights for policy 1, policy_version 490778 (0.0006) [2023-12-27 09:02:26,402][20347] Updated weights for policy 0, policy_version 491077 (0.0008) [2023-12-27 09:02:26,447][20381] Updated weights for policy 1, policy_version 490788 (0.0010) [2023-12-27 09:02:26,450][20347] Updated weights for policy 0, policy_version 491087 (0.0010) [2023-12-27 09:02:26,500][20381] Updated weights for policy 1, policy_version 490798 (0.0007) [2023-12-27 09:02:26,504][20347] Updated weights for policy 0, policy_version 491097 (0.0010) [2023-12-27 09:02:27,148][20381] Updated weights for policy 1, policy_version 490808 (0.0009) [2023-12-27 09:02:27,151][20347] Updated weights for policy 0, policy_version 491107 (0.0010) [2023-12-27 09:02:27,196][20381] Updated weights for policy 1, policy_version 490818 (0.0010) [2023-12-27 09:02:27,199][20347] Updated weights for policy 0, policy_version 491117 (0.0010) [2023-12-27 09:02:27,249][20347] Updated weights for policy 0, policy_version 491127 (0.0010) [2023-12-27 09:02:27,250][20381] Updated weights for policy 1, policy_version 490828 (0.0010) [2023-12-27 09:02:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 251420672. Throughput: 0: 10163.4, 1: 10614.5. Samples: 251402124. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:27,363][19170] Avg episode reward: [(0, '903.960'), (1, '894.520')] [2023-12-27 09:02:27,797][20347] Updated weights for policy 0, policy_version 491137 (0.0009) [2023-12-27 09:02:27,850][20347] Updated weights for policy 0, policy_version 491147 (0.0005) [2023-12-27 09:02:27,898][20347] Updated weights for policy 0, policy_version 491157 (0.0005) [2023-12-27 09:02:27,908][20381] Updated weights for policy 1, policy_version 490838 (0.0010) [2023-12-27 09:02:27,944][20347] Updated weights for policy 0, policy_version 491167 (0.0005) [2023-12-27 09:02:27,953][20381] Updated weights for policy 1, policy_version 490848 (0.0010) [2023-12-27 09:02:28,013][20381] Updated weights for policy 1, policy_version 490858 (0.0006) [2023-12-27 09:02:28,549][20347] Updated weights for policy 0, policy_version 491177 (0.0008) [2023-12-27 09:02:28,579][20381] Updated weights for policy 1, policy_version 490868 (0.0005) [2023-12-27 09:02:28,601][20347] Updated weights for policy 0, policy_version 491187 (0.0005) [2023-12-27 09:02:28,630][20381] Updated weights for policy 1, policy_version 490878 (0.0006) [2023-12-27 09:02:28,662][20347] Updated weights for policy 0, policy_version 491197 (0.0007) [2023-12-27 09:02:28,683][20381] Updated weights for policy 1, policy_version 490888 (0.0006) [2023-12-27 09:02:29,272][20347] Updated weights for policy 0, policy_version 491207 (0.0009) [2023-12-27 09:02:29,291][20381] Updated weights for policy 1, policy_version 490898 (0.0008) [2023-12-27 09:02:29,331][20347] Updated weights for policy 0, policy_version 491217 (0.0010) [2023-12-27 09:02:29,351][20381] Updated weights for policy 1, policy_version 490908 (0.0011) [2023-12-27 09:02:29,388][20347] Updated weights for policy 0, policy_version 491227 (0.0009) [2023-12-27 09:02:29,414][20381] Updated weights for policy 1, policy_version 490918 (0.0011) [2023-12-27 09:02:29,480][20381] Updated weights for policy 1, policy_version 490928 (0.0011) [2023-12-27 09:02:30,119][20347] Updated weights for policy 0, policy_version 491237 (0.0007) [2023-12-27 09:02:30,167][20381] Updated weights for policy 1, policy_version 490938 (0.0008) [2023-12-27 09:02:30,173][20347] Updated weights for policy 0, policy_version 491247 (0.0006) [2023-12-27 09:02:30,227][20381] Updated weights for policy 1, policy_version 490948 (0.0008) [2023-12-27 09:02:30,233][20347] Updated weights for policy 0, policy_version 491257 (0.0006) [2023-12-27 09:02:30,283][20381] Updated weights for policy 1, policy_version 490958 (0.0007) [2023-12-27 09:02:30,798][20347] Updated weights for policy 0, policy_version 491267 (0.0005) [2023-12-27 09:02:30,844][20347] Updated weights for policy 0, policy_version 491277 (0.0007) [2023-12-27 09:02:30,892][20347] Updated weights for policy 0, policy_version 491287 (0.0007) [2023-12-27 09:02:30,948][20381] Updated weights for policy 1, policy_version 490968 (0.0010) [2023-12-27 09:02:30,992][20381] Updated weights for policy 1, policy_version 490978 (0.0010) [2023-12-27 09:02:31,043][20381] Updated weights for policy 1, policy_version 490988 (0.0009) [2023-12-27 09:02:31,555][20347] Updated weights for policy 0, policy_version 491297 (0.0006) [2023-12-27 09:02:31,615][20347] Updated weights for policy 0, policy_version 491307 (0.0008) [2023-12-27 09:02:31,670][20347] Updated weights for policy 0, policy_version 491317 (0.0007) [2023-12-27 09:02:31,734][20347] Updated weights for policy 0, policy_version 491327 (0.0007) [2023-12-27 09:02:31,761][20381] Updated weights for policy 1, policy_version 490998 (0.0007) [2023-12-27 09:02:31,821][20381] Updated weights for policy 1, policy_version 491008 (0.0007) [2023-12-27 09:02:31,880][20381] Updated weights for policy 1, policy_version 491018 (0.0007) [2023-12-27 09:02:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 251518976. Throughput: 0: 10289.7, 1: 10562.3. Samples: 251531808. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-12-27 09:02:32,363][19170] Avg episode reward: [(0, '865.560'), (1, '941.870')] [2023-12-27 09:02:32,377][20347] Updated weights for policy 0, policy_version 491337 (0.0007) [2023-12-27 09:02:32,426][20347] Updated weights for policy 0, policy_version 491347 (0.0007) [2023-12-27 09:02:32,479][20347] Updated weights for policy 0, policy_version 491357 (0.0005) [2023-12-27 09:02:32,523][20381] Updated weights for policy 1, policy_version 491028 (0.0007) [2023-12-27 09:02:32,569][20381] Updated weights for policy 1, policy_version 491038 (0.0007) [2023-12-27 09:02:32,614][20381] Updated weights for policy 1, policy_version 491048 (0.0008) [2023-12-27 09:02:33,082][20347] Updated weights for policy 0, policy_version 491367 (0.0008) [2023-12-27 09:02:33,134][20347] Updated weights for policy 0, policy_version 491377 (0.0009) [2023-12-27 09:02:33,184][20347] Updated weights for policy 0, policy_version 491387 (0.0008) [2023-12-27 09:02:33,237][20381] Updated weights for policy 1, policy_version 491058 (0.0006) [2023-12-27 09:02:33,293][20381] Updated weights for policy 1, policy_version 491068 (0.0005) [2023-12-27 09:02:33,353][20381] Updated weights for policy 1, policy_version 491078 (0.0005) [2023-12-27 09:02:33,408][20381] Updated weights for policy 1, policy_version 491088 (0.0005) [2023-12-27 09:02:33,917][20347] Updated weights for policy 0, policy_version 491397 (0.0006) [2023-12-27 09:02:33,922][20381] Updated weights for policy 1, policy_version 491098 (0.0006) [2023-12-27 09:02:33,966][20347] Updated weights for policy 0, policy_version 491407 (0.0006) [2023-12-27 09:02:33,984][20381] Updated weights for policy 1, policy_version 491108 (0.0006) [2023-12-27 09:02:34,024][20347] Updated weights for policy 0, policy_version 491417 (0.0005) [2023-12-27 09:02:34,034][20381] Updated weights for policy 1, policy_version 491118 (0.0008) [2023-12-27 09:02:34,679][20347] Updated weights for policy 0, policy_version 491427 (0.0005) [2023-12-27 09:02:34,703][20381] Updated weights for policy 1, policy_version 491128 (0.0007) [2023-12-27 09:02:34,733][20347] Updated weights for policy 0, policy_version 491437 (0.0007) [2023-12-27 09:02:34,763][20381] Updated weights for policy 1, policy_version 491138 (0.0006) [2023-12-27 09:02:34,785][20347] Updated weights for policy 0, policy_version 491447 (0.0007) [2023-12-27 09:02:34,816][20381] Updated weights for policy 1, policy_version 491148 (0.0006) [2023-12-27 09:02:35,379][20381] Updated weights for policy 1, policy_version 491158 (0.0006) [2023-12-27 09:02:35,441][20381] Updated weights for policy 1, policy_version 491168 (0.0008) [2023-12-27 09:02:35,472][20347] Updated weights for policy 0, policy_version 491457 (0.0007) [2023-12-27 09:02:35,500][20381] Updated weights for policy 1, policy_version 491178 (0.0007) [2023-12-27 09:02:35,525][20347] Updated weights for policy 0, policy_version 491467 (0.0005) [2023-12-27 09:02:35,580][20347] Updated weights for policy 0, policy_version 491477 (0.0007) [2023-12-27 09:02:35,637][20347] Updated weights for policy 0, policy_version 491487 (0.0005) [2023-12-27 09:02:36,075][20381] Updated weights for policy 1, policy_version 491188 (0.0007) [2023-12-27 09:02:36,132][20381] Updated weights for policy 1, policy_version 491198 (0.0008) [2023-12-27 09:02:36,188][20347] Updated weights for policy 0, policy_version 491497 (0.0008) [2023-12-27 09:02:36,192][20381] Updated weights for policy 1, policy_version 491208 (0.0006) [2023-12-27 09:02:36,256][20347] Updated weights for policy 0, policy_version 491507 (0.0009) [2023-12-27 09:02:36,321][20347] Updated weights for policy 0, policy_version 491517 (0.0008) [2023-12-27 09:02:36,861][20381] Updated weights for policy 1, policy_version 491218 (0.0008) [2023-12-27 09:02:36,917][20381] Updated weights for policy 1, policy_version 491228 (0.0007) [2023-12-27 09:02:36,940][20347] Updated weights for policy 0, policy_version 491527 (0.0008) [2023-12-27 09:02:36,968][20381] Updated weights for policy 1, policy_version 491238 (0.0006) [2023-12-27 09:02:36,987][20347] Updated weights for policy 0, policy_version 491537 (0.0005) [2023-12-27 09:02:37,019][20381] Updated weights for policy 1, policy_version 491248 (0.0008) [2023-12-27 09:02:37,045][20347] Updated weights for policy 0, policy_version 491547 (0.0007) [2023-12-27 09:02:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20882.6). Total num frames: 251633664. Throughput: 0: 10294.3, 1: 10637.2. Samples: 251597512. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:02:37,363][19170] Avg episode reward: [(0, '932.980'), (1, '942.440')] [2023-12-27 09:02:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000491248_125779968.pth... [2023-12-27 09:02:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000491552_125853696.pth... [2023-12-27 09:02:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000490336_125542400.pth [2023-12-27 09:02:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000490000_125460480.pth [2023-12-27 09:02:37,744][20381] Updated weights for policy 1, policy_version 491258 (0.0008) [2023-12-27 09:02:37,749][20347] Updated weights for policy 0, policy_version 491557 (0.0007) [2023-12-27 09:02:37,794][20381] Updated weights for policy 1, policy_version 491268 (0.0007) [2023-12-27 09:02:37,807][20347] Updated weights for policy 0, policy_version 491567 (0.0008) [2023-12-27 09:02:37,844][20381] Updated weights for policy 1, policy_version 491278 (0.0006) [2023-12-27 09:02:37,857][20347] Updated weights for policy 0, policy_version 491577 (0.0005) [2023-12-27 09:02:38,517][20381] Updated weights for policy 1, policy_version 491288 (0.0007) [2023-12-27 09:02:38,542][20347] Updated weights for policy 0, policy_version 491587 (0.0008) [2023-12-27 09:02:38,575][20381] Updated weights for policy 1, policy_version 491298 (0.0006) [2023-12-27 09:02:38,592][20347] Updated weights for policy 0, policy_version 491597 (0.0010) [2023-12-27 09:02:38,629][20381] Updated weights for policy 1, policy_version 491308 (0.0007) [2023-12-27 09:02:38,645][20347] Updated weights for policy 0, policy_version 491607 (0.0006) [2023-12-27 09:02:39,283][20381] Updated weights for policy 1, policy_version 491318 (0.0008) [2023-12-27 09:02:39,309][20347] Updated weights for policy 0, policy_version 491617 (0.0009) [2023-12-27 09:02:39,349][20381] Updated weights for policy 1, policy_version 491328 (0.0008) [2023-12-27 09:02:39,374][20347] Updated weights for policy 0, policy_version 491627 (0.0008) [2023-12-27 09:02:39,411][20381] Updated weights for policy 1, policy_version 491338 (0.0006) [2023-12-27 09:02:39,436][20347] Updated weights for policy 0, policy_version 491637 (0.0007) [2023-12-27 09:02:39,500][20347] Updated weights for policy 0, policy_version 491647 (0.0007) [2023-12-27 09:02:40,123][20381] Updated weights for policy 1, policy_version 491348 (0.0008) [2023-12-27 09:02:40,187][20381] Updated weights for policy 1, policy_version 491358 (0.0011) [2023-12-27 09:02:40,233][20347] Updated weights for policy 0, policy_version 491657 (0.0008) [2023-12-27 09:02:40,246][20381] Updated weights for policy 1, policy_version 491368 (0.0010) [2023-12-27 09:02:40,282][20347] Updated weights for policy 0, policy_version 491667 (0.0005) [2023-12-27 09:02:40,342][20347] Updated weights for policy 0, policy_version 491677 (0.0007) [2023-12-27 09:02:40,867][20381] Updated weights for policy 1, policy_version 491378 (0.0009) [2023-12-27 09:02:40,924][20381] Updated weights for policy 1, policy_version 491388 (0.0005) [2023-12-27 09:02:40,982][20381] Updated weights for policy 1, policy_version 491398 (0.0010) [2023-12-27 09:02:41,041][20381] Updated weights for policy 1, policy_version 491408 (0.0011) [2023-12-27 09:02:41,043][20347] Updated weights for policy 0, policy_version 491687 (0.0008) [2023-12-27 09:02:41,098][20347] Updated weights for policy 0, policy_version 491697 (0.0008) [2023-12-27 09:02:41,160][20347] Updated weights for policy 0, policy_version 491707 (0.0010) [2023-12-27 09:02:41,695][20381] Updated weights for policy 1, policy_version 491418 (0.0007) [2023-12-27 09:02:41,755][20381] Updated weights for policy 1, policy_version 491428 (0.0007) [2023-12-27 09:02:41,816][20381] Updated weights for policy 1, policy_version 491438 (0.0007) [2023-12-27 09:02:41,938][20347] Updated weights for policy 0, policy_version 491717 (0.0009) [2023-12-27 09:02:41,999][20347] Updated weights for policy 0, policy_version 491727 (0.0008) [2023-12-27 09:02:42,049][20347] Updated weights for policy 0, policy_version 491737 (0.0008) [2023-12-27 09:02:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 251731968. Throughput: 0: 10338.9, 1: 10647.2. Samples: 251722260. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:02:42,363][19170] Avg episode reward: [(0, '932.960'), (1, '913.420')] [2023-12-27 09:02:42,475][20381] Updated weights for policy 1, policy_version 491448 (0.0008) [2023-12-27 09:02:42,525][20381] Updated weights for policy 1, policy_version 491458 (0.0008) [2023-12-27 09:02:42,574][20381] Updated weights for policy 1, policy_version 491468 (0.0007) [2023-12-27 09:02:42,780][20347] Updated weights for policy 0, policy_version 491747 (0.0008) [2023-12-27 09:02:42,833][20347] Updated weights for policy 0, policy_version 491757 (0.0007) [2023-12-27 09:02:42,891][20347] Updated weights for policy 0, policy_version 491767 (0.0007) [2023-12-27 09:02:43,255][20381] Updated weights for policy 1, policy_version 491478 (0.0007) [2023-12-27 09:02:43,310][20381] Updated weights for policy 1, policy_version 491488 (0.0009) [2023-12-27 09:02:43,367][20381] Updated weights for policy 1, policy_version 491498 (0.0005) [2023-12-27 09:02:43,563][20347] Updated weights for policy 0, policy_version 491777 (0.0008) [2023-12-27 09:02:43,622][20347] Updated weights for policy 0, policy_version 491787 (0.0005) [2023-12-27 09:02:43,681][20347] Updated weights for policy 0, policy_version 491797 (0.0005) [2023-12-27 09:02:43,725][20347] Updated weights for policy 0, policy_version 491807 (0.0005) [2023-12-27 09:02:44,079][20381] Updated weights for policy 1, policy_version 491508 (0.0006) [2023-12-27 09:02:44,133][20381] Updated weights for policy 1, policy_version 491518 (0.0007) [2023-12-27 09:02:44,188][20381] Updated weights for policy 1, policy_version 491528 (0.0007) [2023-12-27 09:02:44,313][20347] Updated weights for policy 0, policy_version 491817 (0.0010) [2023-12-27 09:02:44,371][20347] Updated weights for policy 0, policy_version 491827 (0.0010) [2023-12-27 09:02:44,419][20347] Updated weights for policy 0, policy_version 491837 (0.0010) [2023-12-27 09:02:44,856][20381] Updated weights for policy 1, policy_version 491538 (0.0007) [2023-12-27 09:02:44,918][20381] Updated weights for policy 1, policy_version 491548 (0.0006) [2023-12-27 09:02:44,978][20381] Updated weights for policy 1, policy_version 491558 (0.0007) [2023-12-27 09:02:45,039][20381] Updated weights for policy 1, policy_version 491568 (0.0007) [2023-12-27 09:02:45,049][20347] Updated weights for policy 0, policy_version 491847 (0.0010) [2023-12-27 09:02:45,101][20347] Updated weights for policy 0, policy_version 491857 (0.0010) [2023-12-27 09:02:45,154][20347] Updated weights for policy 0, policy_version 491867 (0.0010) [2023-12-27 09:02:45,759][20381] Updated weights for policy 1, policy_version 491578 (0.0008) [2023-12-27 09:02:45,803][20347] Updated weights for policy 0, policy_version 491877 (0.0008) [2023-12-27 09:02:45,816][20381] Updated weights for policy 1, policy_version 491588 (0.0008) [2023-12-27 09:02:45,857][20347] Updated weights for policy 0, policy_version 491887 (0.0005) [2023-12-27 09:02:45,876][20381] Updated weights for policy 1, policy_version 491598 (0.0007) [2023-12-27 09:02:45,906][20347] Updated weights for policy 0, policy_version 491897 (0.0005) [2023-12-27 09:02:46,516][20381] Updated weights for policy 1, policy_version 491608 (0.0008) [2023-12-27 09:02:46,574][20347] Updated weights for policy 0, policy_version 491907 (0.0005) [2023-12-27 09:02:46,583][20381] Updated weights for policy 1, policy_version 491618 (0.0006) [2023-12-27 09:02:46,629][20347] Updated weights for policy 0, policy_version 491917 (0.0006) [2023-12-27 09:02:46,642][20381] Updated weights for policy 1, policy_version 491628 (0.0006) [2023-12-27 09:02:46,691][20347] Updated weights for policy 0, policy_version 491927 (0.0008) [2023-12-27 09:02:47,304][20381] Updated weights for policy 1, policy_version 491638 (0.0008) [2023-12-27 09:02:47,360][20381] Updated weights for policy 1, policy_version 491648 (0.0007) [2023-12-27 09:02:47,361][20347] Updated weights for policy 0, policy_version 491937 (0.0008) [2023-12-27 09:02:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 251830272. Throughput: 0: 10340.1, 1: 10600.8. Samples: 251845552. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:02:47,363][19170] Avg episode reward: [(0, '923.280'), (1, '932.250')] [2023-12-27 09:02:47,416][20347] Updated weights for policy 0, policy_version 491947 (0.0005) [2023-12-27 09:02:47,418][20381] Updated weights for policy 1, policy_version 491658 (0.0006) [2023-12-27 09:02:47,470][20347] Updated weights for policy 0, policy_version 491957 (0.0005) [2023-12-27 09:02:47,523][20347] Updated weights for policy 0, policy_version 491967 (0.0005) [2023-12-27 09:02:48,113][20347] Updated weights for policy 0, policy_version 491977 (0.0010) [2023-12-27 09:02:48,120][20381] Updated weights for policy 1, policy_version 491668 (0.0007) [2023-12-27 09:02:48,172][20381] Updated weights for policy 1, policy_version 491678 (0.0006) [2023-12-27 09:02:48,172][20347] Updated weights for policy 0, policy_version 491987 (0.0008) [2023-12-27 09:02:48,221][20381] Updated weights for policy 1, policy_version 491688 (0.0008) [2023-12-27 09:02:48,231][20347] Updated weights for policy 0, policy_version 491997 (0.0010) [2023-12-27 09:02:48,919][20347] Updated weights for policy 0, policy_version 492007 (0.0008) [2023-12-27 09:02:48,950][20381] Updated weights for policy 1, policy_version 491698 (0.0007) [2023-12-27 09:02:48,974][20347] Updated weights for policy 0, policy_version 492017 (0.0009) [2023-12-27 09:02:49,005][20381] Updated weights for policy 1, policy_version 491708 (0.0008) [2023-12-27 09:02:49,026][20347] Updated weights for policy 0, policy_version 492027 (0.0010) [2023-12-27 09:02:49,056][20381] Updated weights for policy 1, policy_version 491718 (0.0008) [2023-12-27 09:02:49,117][20381] Updated weights for policy 1, policy_version 491728 (0.0008) [2023-12-27 09:02:49,732][20347] Updated weights for policy 0, policy_version 492037 (0.0010) [2023-12-27 09:02:49,792][20347] Updated weights for policy 0, policy_version 492047 (0.0010) [2023-12-27 09:02:49,831][20381] Updated weights for policy 1, policy_version 491738 (0.0008) [2023-12-27 09:02:49,852][20347] Updated weights for policy 0, policy_version 492057 (0.0010) [2023-12-27 09:02:49,888][20381] Updated weights for policy 1, policy_version 491748 (0.0007) [2023-12-27 09:02:49,954][20381] Updated weights for policy 1, policy_version 491758 (0.0006) [2023-12-27 09:02:50,529][20347] Updated weights for policy 0, policy_version 492067 (0.0009) [2023-12-27 09:02:50,539][20381] Updated weights for policy 1, policy_version 491768 (0.0005) [2023-12-27 09:02:50,593][20347] Updated weights for policy 0, policy_version 492077 (0.0007) [2023-12-27 09:02:50,598][20381] Updated weights for policy 1, policy_version 491778 (0.0008) [2023-12-27 09:02:50,646][20347] Updated weights for policy 0, policy_version 492087 (0.0007) [2023-12-27 09:02:50,653][20381] Updated weights for policy 1, policy_version 491788 (0.0006) [2023-12-27 09:02:51,274][20381] Updated weights for policy 1, policy_version 491798 (0.0008) [2023-12-27 09:02:51,293][20347] Updated weights for policy 0, policy_version 492097 (0.0008) [2023-12-27 09:02:51,323][20381] Updated weights for policy 1, policy_version 491808 (0.0007) [2023-12-27 09:02:51,355][20347] Updated weights for policy 0, policy_version 492107 (0.0007) [2023-12-27 09:02:51,381][20381] Updated weights for policy 1, policy_version 491818 (0.0007) [2023-12-27 09:02:51,418][20347] Updated weights for policy 0, policy_version 492117 (0.0007) [2023-12-27 09:02:51,477][20347] Updated weights for policy 0, policy_version 492127 (0.0006) [2023-12-27 09:02:52,078][20381] Updated weights for policy 1, policy_version 491828 (0.0007) [2023-12-27 09:02:52,138][20381] Updated weights for policy 1, policy_version 491838 (0.0008) [2023-12-27 09:02:52,175][20347] Updated weights for policy 0, policy_version 492137 (0.0006) [2023-12-27 09:02:52,192][20381] Updated weights for policy 1, policy_version 491848 (0.0006) [2023-12-27 09:02:52,218][20291] KL-divergence is very high: 180.5395 [2023-12-27 09:02:52,223][20347] Updated weights for policy 0, policy_version 492147 (0.0006) [2023-12-27 09:02:52,228][20291] KL-divergence is very high: 198.4438 [2023-12-27 09:02:52,279][20347] Updated weights for policy 0, policy_version 492157 (0.0006) [2023-12-27 09:02:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 251944960. Throughput: 0: 10365.5, 1: 10572.9. Samples: 251906648. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:02:52,363][19170] Avg episode reward: [(0, '933.030'), (1, '932.470')] [2023-12-27 09:02:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000491856_125935616.pth... [2023-12-27 09:02:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000492160_126009344.pth... [2023-12-27 09:02:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000490608_125616128.pth [2023-12-27 09:02:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000490912_125689856.pth [2023-12-27 09:02:52,832][20381] Updated weights for policy 1, policy_version 491858 (0.0006) [2023-12-27 09:02:52,892][20381] Updated weights for policy 1, policy_version 491868 (0.0008) [2023-12-27 09:02:52,953][20381] Updated weights for policy 1, policy_version 491878 (0.0007) [2023-12-27 09:02:52,991][20347] Updated weights for policy 0, policy_version 492167 (0.0007) [2023-12-27 09:02:53,008][20381] Updated weights for policy 1, policy_version 491888 (0.0005) [2023-12-27 09:02:53,051][20347] Updated weights for policy 0, policy_version 492177 (0.0008) [2023-12-27 09:02:53,106][20347] Updated weights for policy 0, policy_version 492187 (0.0007) [2023-12-27 09:02:53,125][20291] KL-divergence is very high: 303.3236 [2023-12-27 09:02:53,593][20381] Updated weights for policy 1, policy_version 491898 (0.0005) [2023-12-27 09:02:53,640][20381] Updated weights for policy 1, policy_version 491908 (0.0005) [2023-12-27 09:02:53,684][20381] Updated weights for policy 1, policy_version 491918 (0.0005) [2023-12-27 09:02:53,787][20347] Updated weights for policy 0, policy_version 492197 (0.0007) [2023-12-27 09:02:53,837][20347] Updated weights for policy 0, policy_version 492207 (0.0010) [2023-12-27 09:02:53,889][20347] Updated weights for policy 0, policy_version 492217 (0.0010) [2023-12-27 09:02:54,343][20381] Updated weights for policy 1, policy_version 491928 (0.0007) [2023-12-27 09:02:54,403][20381] Updated weights for policy 1, policy_version 491938 (0.0007) [2023-12-27 09:02:54,467][20381] Updated weights for policy 1, policy_version 491948 (0.0006) [2023-12-27 09:02:54,565][20347] Updated weights for policy 0, policy_version 492227 (0.0009) [2023-12-27 09:02:54,595][20291] KL-divergence is very high: 128.2054 [2023-12-27 09:02:54,622][20347] Updated weights for policy 0, policy_version 492237 (0.0010) [2023-12-27 09:02:54,670][20347] Updated weights for policy 0, policy_version 492247 (0.0010) [2023-12-27 09:02:55,116][20381] Updated weights for policy 1, policy_version 491958 (0.0006) [2023-12-27 09:02:55,176][20381] Updated weights for policy 1, policy_version 491968 (0.0008) [2023-12-27 09:02:55,238][20381] Updated weights for policy 1, policy_version 491978 (0.0011) [2023-12-27 09:02:55,273][20347] Updated weights for policy 0, policy_version 492257 (0.0006) [2023-12-27 09:02:55,329][20347] Updated weights for policy 0, policy_version 492267 (0.0005) [2023-12-27 09:02:55,388][20347] Updated weights for policy 0, policy_version 492277 (0.0006) [2023-12-27 09:02:55,441][20347] Updated weights for policy 0, policy_version 492287 (0.0008) [2023-12-27 09:02:55,822][20381] Updated weights for policy 1, policy_version 491988 (0.0008) [2023-12-27 09:02:55,870][20381] Updated weights for policy 1, policy_version 491998 (0.0005) [2023-12-27 09:02:55,927][20381] Updated weights for policy 1, policy_version 492008 (0.0007) [2023-12-27 09:02:56,057][20347] Updated weights for policy 0, policy_version 492297 (0.0009) [2023-12-27 09:02:56,115][20347] Updated weights for policy 0, policy_version 492307 (0.0010) [2023-12-27 09:02:56,176][20347] Updated weights for policy 0, policy_version 492317 (0.0010) [2023-12-27 09:02:56,506][20381] Updated weights for policy 1, policy_version 492018 (0.0010) [2023-12-27 09:02:56,563][20381] Updated weights for policy 1, policy_version 492028 (0.0010) [2023-12-27 09:02:56,579][20292] KL-divergence is very high: 158.9239 [2023-12-27 09:02:56,646][20381] Updated weights for policy 1, policy_version 492038 (0.0010) [2023-12-27 09:02:56,693][20381] Updated weights for policy 1, policy_version 492048 (0.0010) [2023-12-27 09:02:56,757][20347] Updated weights for policy 0, policy_version 492327 (0.0005) [2023-12-27 09:02:56,810][20347] Updated weights for policy 0, policy_version 492337 (0.0008) [2023-12-27 09:02:56,857][20347] Updated weights for policy 0, policy_version 492347 (0.0010) [2023-12-27 09:02:57,265][20381] Updated weights for policy 1, policy_version 492058 (0.0005) [2023-12-27 09:02:57,330][20381] Updated weights for policy 1, policy_version 492068 (0.0008) [2023-12-27 09:02:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 252043264. Throughput: 0: 10400.4, 1: 10640.3. Samples: 252034704. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:02:57,363][19170] Avg episode reward: [(0, '942.490'), (1, '970.970')] [2023-12-27 09:02:57,388][20381] Updated weights for policy 1, policy_version 492078 (0.0010) [2023-12-27 09:02:57,504][20347] Updated weights for policy 0, policy_version 492357 (0.0008) [2023-12-27 09:02:57,561][20347] Updated weights for policy 0, policy_version 492367 (0.0007) [2023-12-27 09:02:57,616][20347] Updated weights for policy 0, policy_version 492377 (0.0006) [2023-12-27 09:02:58,000][20381] Updated weights for policy 1, policy_version 492088 (0.0006) [2023-12-27 09:02:58,059][20381] Updated weights for policy 1, policy_version 492098 (0.0006) [2023-12-27 09:02:58,112][20381] Updated weights for policy 1, policy_version 492108 (0.0005) [2023-12-27 09:02:58,211][20347] Updated weights for policy 0, policy_version 492387 (0.0010) [2023-12-27 09:02:58,269][20347] Updated weights for policy 0, policy_version 492397 (0.0010) [2023-12-27 09:02:58,328][20347] Updated weights for policy 0, policy_version 492407 (0.0010) [2023-12-27 09:02:58,740][20381] Updated weights for policy 1, policy_version 492118 (0.0010) [2023-12-27 09:02:58,801][20381] Updated weights for policy 1, policy_version 492128 (0.0010) [2023-12-27 09:02:58,860][20381] Updated weights for policy 1, policy_version 492138 (0.0010) [2023-12-27 09:02:59,053][20347] Updated weights for policy 0, policy_version 492417 (0.0008) [2023-12-27 09:02:59,100][20347] Updated weights for policy 0, policy_version 492427 (0.0010) [2023-12-27 09:02:59,148][20347] Updated weights for policy 0, policy_version 492437 (0.0010) [2023-12-27 09:02:59,192][20347] Updated weights for policy 0, policy_version 492447 (0.0010) [2023-12-27 09:02:59,566][20381] Updated weights for policy 1, policy_version 492148 (0.0011) [2023-12-27 09:02:59,627][20381] Updated weights for policy 1, policy_version 492158 (0.0008) [2023-12-27 09:02:59,683][20381] Updated weights for policy 1, policy_version 492168 (0.0005) [2023-12-27 09:02:59,878][20347] Updated weights for policy 0, policy_version 492457 (0.0009) [2023-12-27 09:02:59,940][20347] Updated weights for policy 0, policy_version 492467 (0.0009) [2023-12-27 09:03:00,001][20347] Updated weights for policy 0, policy_version 492477 (0.0008) [2023-12-27 09:03:00,360][20381] Updated weights for policy 1, policy_version 492178 (0.0007) [2023-12-27 09:03:00,426][20381] Updated weights for policy 1, policy_version 492188 (0.0007) [2023-12-27 09:03:00,489][20381] Updated weights for policy 1, policy_version 492198 (0.0006) [2023-12-27 09:03:00,552][20381] Updated weights for policy 1, policy_version 492208 (0.0005) [2023-12-27 09:03:00,701][20347] Updated weights for policy 0, policy_version 492487 (0.0007) [2023-12-27 09:03:00,761][20347] Updated weights for policy 0, policy_version 492497 (0.0008) [2023-12-27 09:03:00,820][20347] Updated weights for policy 0, policy_version 492507 (0.0008) [2023-12-27 09:03:01,191][20381] Updated weights for policy 1, policy_version 492218 (0.0007) [2023-12-27 09:03:01,251][20381] Updated weights for policy 1, policy_version 492228 (0.0008) [2023-12-27 09:03:01,311][20381] Updated weights for policy 1, policy_version 492238 (0.0008) [2023-12-27 09:03:01,491][20347] Updated weights for policy 0, policy_version 492517 (0.0007) [2023-12-27 09:03:01,540][20347] Updated weights for policy 0, policy_version 492527 (0.0010) [2023-12-27 09:03:01,588][20347] Updated weights for policy 0, policy_version 492537 (0.0008) [2023-12-27 09:03:01,948][20381] Updated weights for policy 1, policy_version 492248 (0.0007) [2023-12-27 09:03:02,005][20381] Updated weights for policy 1, policy_version 492258 (0.0008) [2023-12-27 09:03:02,062][20381] Updated weights for policy 1, policy_version 492268 (0.0007) [2023-12-27 09:03:02,184][20347] Updated weights for policy 0, policy_version 492547 (0.0007) [2023-12-27 09:03:02,250][20347] Updated weights for policy 0, policy_version 492557 (0.0006) [2023-12-27 09:03:02,302][20347] Updated weights for policy 0, policy_version 492567 (0.0008) [2023-12-27 09:03:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21162.7, 300 sec: 20882.7). Total num frames: 252157952. Throughput: 0: 10494.9, 1: 10579.0. Samples: 252162264. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:02,363][19170] Avg episode reward: [(0, '942.460'), (1, '981.140')] [2023-12-27 09:03:02,733][20381] Updated weights for policy 1, policy_version 492278 (0.0007) [2023-12-27 09:03:02,777][20381] Updated weights for policy 1, policy_version 492288 (0.0008) [2023-12-27 09:03:02,816][20292] KL-divergence is very high: 157.0523 [2023-12-27 09:03:02,828][20381] Updated weights for policy 1, policy_version 492298 (0.0008) [2023-12-27 09:03:02,922][20347] Updated weights for policy 0, policy_version 492577 (0.0009) [2023-12-27 09:03:02,977][20347] Updated weights for policy 0, policy_version 492587 (0.0008) [2023-12-27 09:03:03,034][20347] Updated weights for policy 0, policy_version 492597 (0.0008) [2023-12-27 09:03:03,089][20347] Updated weights for policy 0, policy_version 492607 (0.0010) [2023-12-27 09:03:03,437][20381] Updated weights for policy 1, policy_version 492308 (0.0007) [2023-12-27 09:03:03,487][20381] Updated weights for policy 1, policy_version 492318 (0.0005) [2023-12-27 09:03:03,539][20381] Updated weights for policy 1, policy_version 492328 (0.0005) [2023-12-27 09:03:03,663][20347] Updated weights for policy 0, policy_version 492617 (0.0010) [2023-12-27 09:03:03,717][20347] Updated weights for policy 0, policy_version 492627 (0.0010) [2023-12-27 09:03:03,771][20347] Updated weights for policy 0, policy_version 492637 (0.0010) [2023-12-27 09:03:04,129][20381] Updated weights for policy 1, policy_version 492338 (0.0006) [2023-12-27 09:03:04,184][20381] Updated weights for policy 1, policy_version 492348 (0.0007) [2023-12-27 09:03:04,238][20381] Updated weights for policy 1, policy_version 492358 (0.0008) [2023-12-27 09:03:04,284][20381] Updated weights for policy 1, policy_version 492368 (0.0010) [2023-12-27 09:03:04,443][20347] Updated weights for policy 0, policy_version 492647 (0.0007) [2023-12-27 09:03:04,489][20291] KL-divergence is very high: 1632.8826 [2023-12-27 09:03:04,500][20347] Updated weights for policy 0, policy_version 492657 (0.0010) [2023-12-27 09:03:04,560][20347] Updated weights for policy 0, policy_version 492667 (0.0010) [2023-12-27 09:03:04,967][20381] Updated weights for policy 1, policy_version 492378 (0.0007) [2023-12-27 09:03:05,020][20381] Updated weights for policy 1, policy_version 492388 (0.0007) [2023-12-27 09:03:05,065][20381] Updated weights for policy 1, policy_version 492398 (0.0007) [2023-12-27 09:03:05,215][20291] KL-divergence is very high: 172.7323 [2023-12-27 09:03:05,221][20347] Updated weights for policy 0, policy_version 492677 (0.0010) [2023-12-27 09:03:05,266][20347] Updated weights for policy 0, policy_version 492687 (0.0007) [2023-12-27 09:03:05,289][20291] KL-divergence is very high: 107.1701 [2023-12-27 09:03:05,313][20347] Updated weights for policy 0, policy_version 492697 (0.0007) [2023-12-27 09:03:05,650][20381] Updated weights for policy 1, policy_version 492408 (0.0005) [2023-12-27 09:03:05,704][20381] Updated weights for policy 1, policy_version 492418 (0.0005) [2023-12-27 09:03:05,752][20381] Updated weights for policy 1, policy_version 492428 (0.0005) [2023-12-27 09:03:06,061][20347] Updated weights for policy 0, policy_version 492707 (0.0008) [2023-12-27 09:03:06,127][20347] Updated weights for policy 0, policy_version 492717 (0.0009) [2023-12-27 09:03:06,145][20291] KL-divergence is very high: 156.1549 [2023-12-27 09:03:06,186][20347] Updated weights for policy 0, policy_version 492727 (0.0010) [2023-12-27 09:03:06,363][20381] Updated weights for policy 1, policy_version 492438 (0.0006) [2023-12-27 09:03:06,425][20381] Updated weights for policy 1, policy_version 492448 (0.0008) [2023-12-27 09:03:06,487][20381] Updated weights for policy 1, policy_version 492458 (0.0006) [2023-12-27 09:03:06,847][20347] Updated weights for policy 0, policy_version 492737 (0.0010) [2023-12-27 09:03:06,901][20347] Updated weights for policy 0, policy_version 492747 (0.0007) [2023-12-27 09:03:06,950][20347] Updated weights for policy 0, policy_version 492757 (0.0007) [2023-12-27 09:03:07,004][20347] Updated weights for policy 0, policy_version 492767 (0.0006) [2023-12-27 09:03:07,197][20381] Updated weights for policy 1, policy_version 492468 (0.0008) [2023-12-27 09:03:07,250][20381] Updated weights for policy 1, policy_version 492478 (0.0007) [2023-12-27 09:03:07,301][20381] Updated weights for policy 1, policy_version 492488 (0.0007) [2023-12-27 09:03:07,362][19170] Fps is (10 sec: 22118.4, 60 sec: 21162.8, 300 sec: 20882.7). Total num frames: 252264448. Throughput: 0: 10534.0, 1: 10596.5. Samples: 252227884. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:07,363][19170] Avg episode reward: [(0, '823.140'), (1, '990.840')] [2023-12-27 09:03:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000492496_126099456.pth... [2023-12-27 09:03:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000492768_126164992.pth... [2023-12-27 09:03:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000491248_125779968.pth [2023-12-27 09:03:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000491552_125853696.pth [2023-12-27 09:03:07,651][20347] Updated weights for policy 0, policy_version 492777 (0.0007) [2023-12-27 09:03:07,710][20347] Updated weights for policy 0, policy_version 492787 (0.0005) [2023-12-27 09:03:07,766][20347] Updated weights for policy 0, policy_version 492797 (0.0005) [2023-12-27 09:03:07,956][20381] Updated weights for policy 1, policy_version 492498 (0.0007) [2023-12-27 09:03:08,009][20381] Updated weights for policy 1, policy_version 492508 (0.0008) [2023-12-27 09:03:08,055][20381] Updated weights for policy 1, policy_version 492518 (0.0010) [2023-12-27 09:03:08,107][20381] Updated weights for policy 1, policy_version 492528 (0.0009) [2023-12-27 09:03:08,304][20347] Updated weights for policy 0, policy_version 492807 (0.0008) [2023-12-27 09:03:08,359][20347] Updated weights for policy 0, policy_version 492817 (0.0007) [2023-12-27 09:03:08,410][20347] Updated weights for policy 0, policy_version 492827 (0.0007) [2023-12-27 09:03:08,845][20381] Updated weights for policy 1, policy_version 492538 (0.0007) [2023-12-27 09:03:08,894][20381] Updated weights for policy 1, policy_version 492548 (0.0007) [2023-12-27 09:03:08,948][20381] Updated weights for policy 1, policy_version 492558 (0.0008) [2023-12-27 09:03:09,123][20347] Updated weights for policy 0, policy_version 492837 (0.0006) [2023-12-27 09:03:09,176][20347] Updated weights for policy 0, policy_version 492847 (0.0005) [2023-12-27 09:03:09,239][20347] Updated weights for policy 0, policy_version 492857 (0.0007) [2023-12-27 09:03:09,735][20381] Updated weights for policy 1, policy_version 492568 (0.0008) [2023-12-27 09:03:09,792][20381] Updated weights for policy 1, policy_version 492578 (0.0007) [2023-12-27 09:03:09,862][20381] Updated weights for policy 1, policy_version 492588 (0.0008) [2023-12-27 09:03:09,921][20347] Updated weights for policy 0, policy_version 492867 (0.0007) [2023-12-27 09:03:09,989][20347] Updated weights for policy 0, policy_version 492877 (0.0007) [2023-12-27 09:03:10,048][20347] Updated weights for policy 0, policy_version 492887 (0.0007) [2023-12-27 09:03:10,568][20381] Updated weights for policy 1, policy_version 492598 (0.0009) [2023-12-27 09:03:10,630][20381] Updated weights for policy 1, policy_version 492608 (0.0010) [2023-12-27 09:03:10,692][20381] Updated weights for policy 1, policy_version 492618 (0.0011) [2023-12-27 09:03:10,756][20347] Updated weights for policy 0, policy_version 492897 (0.0008) [2023-12-27 09:03:10,809][20347] Updated weights for policy 0, policy_version 492907 (0.0007) [2023-12-27 09:03:10,876][20347] Updated weights for policy 0, policy_version 492917 (0.0007) [2023-12-27 09:03:10,933][20347] Updated weights for policy 0, policy_version 492927 (0.0007) [2023-12-27 09:03:11,389][20381] Updated weights for policy 1, policy_version 492628 (0.0013) [2023-12-27 09:03:11,444][20381] Updated weights for policy 1, policy_version 492638 (0.0010) [2023-12-27 09:03:11,511][20381] Updated weights for policy 1, policy_version 492648 (0.0010) [2023-12-27 09:03:11,615][20347] Updated weights for policy 0, policy_version 492937 (0.0008) [2023-12-27 09:03:11,674][20347] Updated weights for policy 0, policy_version 492947 (0.0011) [2023-12-27 09:03:11,736][20347] Updated weights for policy 0, policy_version 492957 (0.0010) [2023-12-27 09:03:12,212][20381] Updated weights for policy 1, policy_version 492658 (0.0011) [2023-12-27 09:03:12,262][20381] Updated weights for policy 1, policy_version 492668 (0.0007) [2023-12-27 09:03:12,320][20381] Updated weights for policy 1, policy_version 492678 (0.0007) [2023-12-27 09:03:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 252354560. Throughput: 0: 10522.8, 1: 10548.1. Samples: 252350316. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:12,363][19170] Avg episode reward: [(0, '778.030'), (1, '1000.000')] [2023-12-27 09:03:12,382][20381] Updated weights for policy 1, policy_version 492688 (0.0010) [2023-12-27 09:03:12,439][20347] Updated weights for policy 0, policy_version 492967 (0.0007) [2023-12-27 09:03:12,493][20347] Updated weights for policy 0, policy_version 492977 (0.0007) [2023-12-27 09:03:12,550][20347] Updated weights for policy 0, policy_version 492987 (0.0007) [2023-12-27 09:03:13,051][20381] Updated weights for policy 1, policy_version 492698 (0.0008) [2023-12-27 09:03:13,111][20381] Updated weights for policy 1, policy_version 492708 (0.0007) [2023-12-27 09:03:13,162][20381] Updated weights for policy 1, policy_version 492718 (0.0008) [2023-12-27 09:03:13,193][20347] Updated weights for policy 0, policy_version 492997 (0.0007) [2023-12-27 09:03:13,254][20347] Updated weights for policy 0, policy_version 493007 (0.0008) [2023-12-27 09:03:13,313][20347] Updated weights for policy 0, policy_version 493017 (0.0007) [2023-12-27 09:03:13,816][20381] Updated weights for policy 1, policy_version 492728 (0.0009) [2023-12-27 09:03:13,881][20381] Updated weights for policy 1, policy_version 492738 (0.0006) [2023-12-27 09:03:13,942][20381] Updated weights for policy 1, policy_version 492748 (0.0008) [2023-12-27 09:03:14,017][20347] Updated weights for policy 0, policy_version 493027 (0.0008) [2023-12-27 09:03:14,069][20347] Updated weights for policy 0, policy_version 493037 (0.0008) [2023-12-27 09:03:14,120][20347] Updated weights for policy 0, policy_version 493047 (0.0008) [2023-12-27 09:03:14,470][20381] Updated weights for policy 1, policy_version 492758 (0.0010) [2023-12-27 09:03:14,525][20381] Updated weights for policy 1, policy_version 492768 (0.0011) [2023-12-27 09:03:14,577][20381] Updated weights for policy 1, policy_version 492778 (0.0010) [2023-12-27 09:03:14,700][20347] Updated weights for policy 0, policy_version 493057 (0.0007) [2023-12-27 09:03:14,761][20347] Updated weights for policy 0, policy_version 493067 (0.0006) [2023-12-27 09:03:14,826][20347] Updated weights for policy 0, policy_version 493077 (0.0007) [2023-12-27 09:03:14,882][20347] Updated weights for policy 0, policy_version 493087 (0.0006) [2023-12-27 09:03:15,225][20381] Updated weights for policy 1, policy_version 492788 (0.0008) [2023-12-27 09:03:15,292][20381] Updated weights for policy 1, policy_version 492798 (0.0008) [2023-12-27 09:03:15,357][20381] Updated weights for policy 1, policy_version 492808 (0.0007) [2023-12-27 09:03:15,461][20347] Updated weights for policy 0, policy_version 493097 (0.0005) [2023-12-27 09:03:15,513][20347] Updated weights for policy 0, policy_version 493107 (0.0006) [2023-12-27 09:03:15,562][20347] Updated weights for policy 0, policy_version 493117 (0.0007) [2023-12-27 09:03:15,995][20381] Updated weights for policy 1, policy_version 492818 (0.0008) [2023-12-27 09:03:16,054][20381] Updated weights for policy 1, policy_version 492828 (0.0010) [2023-12-27 09:03:16,115][20381] Updated weights for policy 1, policy_version 492838 (0.0010) [2023-12-27 09:03:16,163][20381] Updated weights for policy 1, policy_version 492848 (0.0010) [2023-12-27 09:03:16,226][20347] Updated weights for policy 0, policy_version 493127 (0.0009) [2023-12-27 09:03:16,281][20347] Updated weights for policy 0, policy_version 493137 (0.0010) [2023-12-27 09:03:16,337][20347] Updated weights for policy 0, policy_version 493147 (0.0007) [2023-12-27 09:03:16,802][20381] Updated weights for policy 1, policy_version 492858 (0.0005) [2023-12-27 09:03:16,866][20381] Updated weights for policy 1, policy_version 492868 (0.0010) [2023-12-27 09:03:16,900][20347] Updated weights for policy 0, policy_version 493157 (0.0005) [2023-12-27 09:03:16,929][20381] Updated weights for policy 1, policy_version 492878 (0.0009) [2023-12-27 09:03:16,963][20347] Updated weights for policy 0, policy_version 493167 (0.0005) [2023-12-27 09:03:17,023][20347] Updated weights for policy 0, policy_version 493177 (0.0009) [2023-12-27 09:03:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 252469248. Throughput: 0: 10483.4, 1: 10562.1. Samples: 252478856. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:17,363][19170] Avg episode reward: [(0, '849.770'), (1, '1000.000')] [2023-12-27 09:03:17,559][20347] Updated weights for policy 0, policy_version 493187 (0.0008) [2023-12-27 09:03:17,583][20381] Updated weights for policy 1, policy_version 492888 (0.0006) [2023-12-27 09:03:17,615][20347] Updated weights for policy 0, policy_version 493197 (0.0006) [2023-12-27 09:03:17,641][20381] Updated weights for policy 1, policy_version 492898 (0.0006) [2023-12-27 09:03:17,666][20347] Updated weights for policy 0, policy_version 493207 (0.0005) [2023-12-27 09:03:17,694][20381] Updated weights for policy 1, policy_version 492908 (0.0007) [2023-12-27 09:03:18,252][20347] Updated weights for policy 0, policy_version 493217 (0.0006) [2023-12-27 09:03:18,307][20347] Updated weights for policy 0, policy_version 493227 (0.0007) [2023-12-27 09:03:18,328][20381] Updated weights for policy 1, policy_version 492918 (0.0008) [2023-12-27 09:03:18,363][20347] Updated weights for policy 0, policy_version 493237 (0.0006) [2023-12-27 09:03:18,381][20381] Updated weights for policy 1, policy_version 492928 (0.0007) [2023-12-27 09:03:18,420][20347] Updated weights for policy 0, policy_version 493247 (0.0007) [2023-12-27 09:03:18,430][20381] Updated weights for policy 1, policy_version 492938 (0.0005) [2023-12-27 09:03:19,038][20347] Updated weights for policy 0, policy_version 493257 (0.0007) [2023-12-27 09:03:19,096][20347] Updated weights for policy 0, policy_version 493267 (0.0007) [2023-12-27 09:03:19,148][20347] Updated weights for policy 0, policy_version 493277 (0.0007) [2023-12-27 09:03:19,175][20381] Updated weights for policy 1, policy_version 492948 (0.0008) [2023-12-27 09:03:19,234][20381] Updated weights for policy 1, policy_version 492958 (0.0007) [2023-12-27 09:03:19,294][20381] Updated weights for policy 1, policy_version 492968 (0.0008) [2023-12-27 09:03:19,820][20347] Updated weights for policy 0, policy_version 493287 (0.0008) [2023-12-27 09:03:19,883][20347] Updated weights for policy 0, policy_version 493297 (0.0007) [2023-12-27 09:03:19,948][20347] Updated weights for policy 0, policy_version 493307 (0.0008) [2023-12-27 09:03:20,026][20381] Updated weights for policy 1, policy_version 492978 (0.0008) [2023-12-27 09:03:20,079][20381] Updated weights for policy 1, policy_version 492988 (0.0011) [2023-12-27 09:03:20,139][20381] Updated weights for policy 1, policy_version 492998 (0.0011) [2023-12-27 09:03:20,196][20381] Updated weights for policy 1, policy_version 493008 (0.0011) [2023-12-27 09:03:20,655][20347] Updated weights for policy 0, policy_version 493317 (0.0008) [2023-12-27 09:03:20,711][20347] Updated weights for policy 0, policy_version 493327 (0.0007) [2023-12-27 09:03:20,775][20347] Updated weights for policy 0, policy_version 493337 (0.0008) [2023-12-27 09:03:20,908][20381] Updated weights for policy 1, policy_version 493018 (0.0008) [2023-12-27 09:03:20,965][20381] Updated weights for policy 1, policy_version 493028 (0.0008) [2023-12-27 09:03:21,030][20381] Updated weights for policy 1, policy_version 493038 (0.0008) [2023-12-27 09:03:21,545][20347] Updated weights for policy 0, policy_version 493347 (0.0008) [2023-12-27 09:03:21,593][20347] Updated weights for policy 0, policy_version 493357 (0.0008) [2023-12-27 09:03:21,656][20347] Updated weights for policy 0, policy_version 493367 (0.0007) [2023-12-27 09:03:21,676][20381] Updated weights for policy 1, policy_version 493048 (0.0007) [2023-12-27 09:03:21,737][20381] Updated weights for policy 1, policy_version 493058 (0.0008) [2023-12-27 09:03:21,797][20381] Updated weights for policy 1, policy_version 493068 (0.0008) [2023-12-27 09:03:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 252567552. Throughput: 0: 10526.7, 1: 10448.7. Samples: 252541404. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:22,363][19170] Avg episode reward: [(0, '848.670'), (1, '951.720')] [2023-12-27 09:03:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000493072_126246912.pth... [2023-12-27 09:03:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000493376_126320640.pth... [2023-12-27 09:03:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000491856_125935616.pth [2023-12-27 09:03:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000492160_126009344.pth [2023-12-27 09:03:22,403][20347] Updated weights for policy 0, policy_version 493377 (0.0007) [2023-12-27 09:03:22,466][20347] Updated weights for policy 0, policy_version 493387 (0.0007) [2023-12-27 09:03:22,487][20381] Updated weights for policy 1, policy_version 493078 (0.0008) [2023-12-27 09:03:22,527][20347] Updated weights for policy 0, policy_version 493397 (0.0007) [2023-12-27 09:03:22,547][20292] KL-divergence is very high: 443.6873 [2023-12-27 09:03:22,548][20381] Updated weights for policy 1, policy_version 493088 (0.0008) [2023-12-27 09:03:22,589][20347] Updated weights for policy 0, policy_version 493407 (0.0006) [2023-12-27 09:03:22,607][20381] Updated weights for policy 1, policy_version 493098 (0.0006) [2023-12-27 09:03:23,265][20347] Updated weights for policy 0, policy_version 493417 (0.0007) [2023-12-27 09:03:23,316][20381] Updated weights for policy 1, policy_version 493108 (0.0007) [2023-12-27 09:03:23,319][20347] Updated weights for policy 0, policy_version 493427 (0.0007) [2023-12-27 09:03:23,368][20381] Updated weights for policy 1, policy_version 493118 (0.0008) [2023-12-27 09:03:23,373][20347] Updated weights for policy 0, policy_version 493437 (0.0008) [2023-12-27 09:03:23,415][20381] Updated weights for policy 1, policy_version 493128 (0.0007) [2023-12-27 09:03:24,021][20347] Updated weights for policy 0, policy_version 493447 (0.0007) [2023-12-27 09:03:24,080][20347] Updated weights for policy 0, policy_version 493457 (0.0007) [2023-12-27 09:03:24,106][20291] KL-divergence is very high: 152.4969 [2023-12-27 09:03:24,114][20381] Updated weights for policy 1, policy_version 493138 (0.0007) [2023-12-27 09:03:24,129][20291] KL-divergence is very high: 161.4992 [2023-12-27 09:03:24,135][20347] Updated weights for policy 0, policy_version 493467 (0.0007) [2023-12-27 09:03:24,154][20291] KL-divergence is very high: 122.7571 [2023-12-27 09:03:24,169][20381] Updated weights for policy 1, policy_version 493148 (0.0007) [2023-12-27 09:03:24,227][20381] Updated weights for policy 1, policy_version 493158 (0.0006) [2023-12-27 09:03:24,275][20381] Updated weights for policy 1, policy_version 493168 (0.0005) [2023-12-27 09:03:24,804][20347] Updated weights for policy 0, policy_version 493477 (0.0007) [2023-12-27 09:03:24,828][20381] Updated weights for policy 1, policy_version 493178 (0.0008) [2023-12-27 09:03:24,863][20347] Updated weights for policy 0, policy_version 493487 (0.0005) [2023-12-27 09:03:24,885][20381] Updated weights for policy 1, policy_version 493188 (0.0006) [2023-12-27 09:03:24,923][20347] Updated weights for policy 0, policy_version 493497 (0.0008) [2023-12-27 09:03:24,937][20381] Updated weights for policy 1, policy_version 493198 (0.0005) [2023-12-27 09:03:25,496][20381] Updated weights for policy 1, policy_version 493208 (0.0006) [2023-12-27 09:03:25,531][20347] Updated weights for policy 0, policy_version 493507 (0.0007) [2023-12-27 09:03:25,549][20381] Updated weights for policy 1, policy_version 493218 (0.0008) [2023-12-27 09:03:25,585][20347] Updated weights for policy 0, policy_version 493517 (0.0005) [2023-12-27 09:03:25,595][20381] Updated weights for policy 1, policy_version 493228 (0.0008) [2023-12-27 09:03:25,641][20347] Updated weights for policy 0, policy_version 493527 (0.0008) [2023-12-27 09:03:26,174][20347] Updated weights for policy 0, policy_version 493537 (0.0010) [2023-12-27 09:03:26,229][20347] Updated weights for policy 0, policy_version 493547 (0.0010) [2023-12-27 09:03:26,281][20347] Updated weights for policy 0, policy_version 493557 (0.0010) [2023-12-27 09:03:26,298][20381] Updated weights for policy 1, policy_version 493238 (0.0007) [2023-12-27 09:03:26,345][20347] Updated weights for policy 0, policy_version 493567 (0.0010) [2023-12-27 09:03:26,347][20381] Updated weights for policy 1, policy_version 493248 (0.0006) [2023-12-27 09:03:26,397][20381] Updated weights for policy 1, policy_version 493258 (0.0006) [2023-12-27 09:03:26,972][20381] Updated weights for policy 1, policy_version 493268 (0.0007) [2023-12-27 09:03:26,989][20347] Updated weights for policy 0, policy_version 493577 (0.0008) [2023-12-27 09:03:27,021][20381] Updated weights for policy 1, policy_version 493278 (0.0011) [2023-12-27 09:03:27,040][20347] Updated weights for policy 0, policy_version 493587 (0.0010) [2023-12-27 09:03:27,066][20381] Updated weights for policy 1, policy_version 493288 (0.0008) [2023-12-27 09:03:27,093][20347] Updated weights for policy 0, policy_version 493597 (0.0010) [2023-12-27 09:03:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 252682240. Throughput: 0: 10518.2, 1: 10470.4. Samples: 252666748. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:27,363][19170] Avg episode reward: [(0, '875.850'), (1, '942.030')] [2023-12-27 09:03:27,763][20381] Updated weights for policy 1, policy_version 493298 (0.0006) [2023-12-27 09:03:27,771][20347] Updated weights for policy 0, policy_version 493607 (0.0010) [2023-12-27 09:03:27,810][20381] Updated weights for policy 1, policy_version 493308 (0.0006) [2023-12-27 09:03:27,826][20347] Updated weights for policy 0, policy_version 493617 (0.0009) [2023-12-27 09:03:27,856][20381] Updated weights for policy 1, policy_version 493318 (0.0007) [2023-12-27 09:03:27,870][20347] Updated weights for policy 0, policy_version 493627 (0.0009) [2023-12-27 09:03:27,912][20381] Updated weights for policy 1, policy_version 493328 (0.0007) [2023-12-27 09:03:28,503][20347] Updated weights for policy 0, policy_version 493637 (0.0008) [2023-12-27 09:03:28,547][20291] KL-divergence is very high: 144.9868 [2023-12-27 09:03:28,557][20347] Updated weights for policy 0, policy_version 493647 (0.0008) [2023-12-27 09:03:28,568][20291] KL-divergence is very high: 111.3026 [2023-12-27 09:03:28,595][20381] Updated weights for policy 1, policy_version 493338 (0.0006) [2023-12-27 09:03:28,612][20347] Updated weights for policy 0, policy_version 493657 (0.0010) [2023-12-27 09:03:28,648][20381] Updated weights for policy 1, policy_version 493348 (0.0005) [2023-12-27 09:03:28,704][20381] Updated weights for policy 1, policy_version 493358 (0.0006) [2023-12-27 09:03:29,233][20347] Updated weights for policy 0, policy_version 493667 (0.0010) [2023-12-27 09:03:29,269][20381] Updated weights for policy 1, policy_version 493368 (0.0007) [2023-12-27 09:03:29,301][20347] Updated weights for policy 0, policy_version 493677 (0.0007) [2023-12-27 09:03:29,327][20381] Updated weights for policy 1, policy_version 493378 (0.0008) [2023-12-27 09:03:29,369][20347] Updated weights for policy 0, policy_version 493687 (0.0007) [2023-12-27 09:03:29,389][20381] Updated weights for policy 1, policy_version 493388 (0.0007) [2023-12-27 09:03:29,978][20347] Updated weights for policy 0, policy_version 493697 (0.0008) [2023-12-27 09:03:30,053][20347] Updated weights for policy 0, policy_version 493707 (0.0009) [2023-12-27 09:03:30,095][20381] Updated weights for policy 1, policy_version 493398 (0.0006) [2023-12-27 09:03:30,117][20347] Updated weights for policy 0, policy_version 493717 (0.0006) [2023-12-27 09:03:30,158][20381] Updated weights for policy 1, policy_version 493408 (0.0006) [2023-12-27 09:03:30,179][20347] Updated weights for policy 0, policy_version 493727 (0.0006) [2023-12-27 09:03:30,223][20381] Updated weights for policy 1, policy_version 493418 (0.0006) [2023-12-27 09:03:30,791][20347] Updated weights for policy 0, policy_version 493737 (0.0007) [2023-12-27 09:03:30,807][20381] Updated weights for policy 1, policy_version 493428 (0.0007) [2023-12-27 09:03:30,842][20347] Updated weights for policy 0, policy_version 493747 (0.0006) [2023-12-27 09:03:30,871][20381] Updated weights for policy 1, policy_version 493438 (0.0009) [2023-12-27 09:03:30,904][20347] Updated weights for policy 0, policy_version 493757 (0.0006) [2023-12-27 09:03:30,933][20381] Updated weights for policy 1, policy_version 493448 (0.0006) [2023-12-27 09:03:31,572][20347] Updated weights for policy 0, policy_version 493767 (0.0007) [2023-12-27 09:03:31,634][20347] Updated weights for policy 0, policy_version 493777 (0.0007) [2023-12-27 09:03:31,647][20381] Updated weights for policy 1, policy_version 493458 (0.0007) [2023-12-27 09:03:31,690][20347] Updated weights for policy 0, policy_version 493787 (0.0007) [2023-12-27 09:03:31,705][20381] Updated weights for policy 1, policy_version 493468 (0.0007) [2023-12-27 09:03:31,760][20381] Updated weights for policy 1, policy_version 493478 (0.0008) [2023-12-27 09:03:31,818][20381] Updated weights for policy 1, policy_version 493488 (0.0008) [2023-12-27 09:03:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 252780544. Throughput: 0: 10552.8, 1: 10539.2. Samples: 252794692. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:32,363][19170] Avg episode reward: [(0, '856.560'), (1, '990.370')] [2023-12-27 09:03:32,389][20347] Updated weights for policy 0, policy_version 493797 (0.0007) [2023-12-27 09:03:32,442][20347] Updated weights for policy 0, policy_version 493807 (0.0010) [2023-12-27 09:03:32,459][20381] Updated weights for policy 1, policy_version 493498 (0.0005) [2023-12-27 09:03:32,497][20347] Updated weights for policy 0, policy_version 493817 (0.0010) [2023-12-27 09:03:32,517][20381] Updated weights for policy 1, policy_version 493508 (0.0006) [2023-12-27 09:03:32,577][20381] Updated weights for policy 1, policy_version 493518 (0.0008) [2023-12-27 09:03:33,194][20347] Updated weights for policy 0, policy_version 493827 (0.0008) [2023-12-27 09:03:33,244][20347] Updated weights for policy 0, policy_version 493837 (0.0006) [2023-12-27 09:03:33,249][20381] Updated weights for policy 1, policy_version 493528 (0.0008) [2023-12-27 09:03:33,292][20347] Updated weights for policy 0, policy_version 493847 (0.0006) [2023-12-27 09:03:33,303][20381] Updated weights for policy 1, policy_version 493538 (0.0009) [2023-12-27 09:03:33,360][20381] Updated weights for policy 1, policy_version 493548 (0.0005) [2023-12-27 09:03:33,894][20347] Updated weights for policy 0, policy_version 493857 (0.0006) [2023-12-27 09:03:33,954][20347] Updated weights for policy 0, policy_version 493867 (0.0007) [2023-12-27 09:03:34,008][20381] Updated weights for policy 1, policy_version 493558 (0.0005) [2023-12-27 09:03:34,014][20347] Updated weights for policy 0, policy_version 493877 (0.0006) [2023-12-27 09:03:34,062][20381] Updated weights for policy 1, policy_version 493568 (0.0006) [2023-12-27 09:03:34,074][20347] Updated weights for policy 0, policy_version 493887 (0.0007) [2023-12-27 09:03:34,117][20381] Updated weights for policy 1, policy_version 493578 (0.0007) [2023-12-27 09:03:34,687][20347] Updated weights for policy 0, policy_version 493897 (0.0008) [2023-12-27 09:03:34,735][20291] KL-divergence is very high: 127.2487 [2023-12-27 09:03:34,745][20347] Updated weights for policy 0, policy_version 493907 (0.0007) [2023-12-27 09:03:34,796][20347] Updated weights for policy 0, policy_version 493917 (0.0005) [2023-12-27 09:03:34,860][20381] Updated weights for policy 1, policy_version 493588 (0.0007) [2023-12-27 09:03:34,925][20381] Updated weights for policy 1, policy_version 493598 (0.0007) [2023-12-27 09:03:34,976][20381] Updated weights for policy 1, policy_version 493608 (0.0007) [2023-12-27 09:03:35,408][20347] Updated weights for policy 0, policy_version 493927 (0.0007) [2023-12-27 09:03:35,461][20347] Updated weights for policy 0, policy_version 493937 (0.0007) [2023-12-27 09:03:35,516][20347] Updated weights for policy 0, policy_version 493947 (0.0005) [2023-12-27 09:03:35,667][20381] Updated weights for policy 1, policy_version 493618 (0.0008) [2023-12-27 09:03:35,711][20381] Updated weights for policy 1, policy_version 493628 (0.0005) [2023-12-27 09:03:35,757][20381] Updated weights for policy 1, policy_version 493638 (0.0005) [2023-12-27 09:03:35,807][20381] Updated weights for policy 1, policy_version 493648 (0.0005) [2023-12-27 09:03:36,091][20347] Updated weights for policy 0, policy_version 493957 (0.0005) [2023-12-27 09:03:36,155][20347] Updated weights for policy 0, policy_version 493967 (0.0007) [2023-12-27 09:03:36,218][20347] Updated weights for policy 0, policy_version 493977 (0.0008) [2023-12-27 09:03:36,468][20381] Updated weights for policy 1, policy_version 493658 (0.0006) [2023-12-27 09:03:36,526][20381] Updated weights for policy 1, policy_version 493668 (0.0005) [2023-12-27 09:03:36,578][20381] Updated weights for policy 1, policy_version 493678 (0.0006) [2023-12-27 09:03:36,965][20347] Updated weights for policy 0, policy_version 493987 (0.0008) [2023-12-27 09:03:37,020][20347] Updated weights for policy 0, policy_version 493997 (0.0008) [2023-12-27 09:03:37,076][20347] Updated weights for policy 0, policy_version 494007 (0.0009) [2023-12-27 09:03:37,154][20381] Updated weights for policy 1, policy_version 493688 (0.0008) [2023-12-27 09:03:37,204][20381] Updated weights for policy 1, policy_version 493698 (0.0008) [2023-12-27 09:03:37,264][20381] Updated weights for policy 1, policy_version 493708 (0.0008) [2023-12-27 09:03:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20882.6). Total num frames: 252895232. Throughput: 0: 10587.1, 1: 10538.6. Samples: 252857304. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:37,364][19170] Avg episode reward: [(0, '770.140'), (1, '971.000')] [2023-12-27 09:03:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000494016_126484480.pth... [2023-12-27 09:03:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000493712_126410752.pth... [2023-12-27 09:03:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000492768_126164992.pth [2023-12-27 09:03:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000492496_126099456.pth [2023-12-27 09:03:37,701][20347] Updated weights for policy 0, policy_version 494017 (0.0008) [2023-12-27 09:03:37,753][20347] Updated weights for policy 0, policy_version 494027 (0.0006) [2023-12-27 09:03:37,802][20347] Updated weights for policy 0, policy_version 494037 (0.0005) [2023-12-27 09:03:37,848][20347] Updated weights for policy 0, policy_version 494047 (0.0005) [2023-12-27 09:03:37,945][20381] Updated weights for policy 1, policy_version 493718 (0.0009) [2023-12-27 09:03:37,994][20381] Updated weights for policy 1, policy_version 493728 (0.0009) [2023-12-27 09:03:38,041][20381] Updated weights for policy 1, policy_version 493738 (0.0009) [2023-12-27 09:03:38,491][20347] Updated weights for policy 0, policy_version 494057 (0.0007) [2023-12-27 09:03:38,552][20347] Updated weights for policy 0, policy_version 494067 (0.0007) [2023-12-27 09:03:38,607][20347] Updated weights for policy 0, policy_version 494077 (0.0007) [2023-12-27 09:03:38,712][20381] Updated weights for policy 1, policy_version 493748 (0.0011) [2023-12-27 09:03:38,764][20381] Updated weights for policy 1, policy_version 493758 (0.0010) [2023-12-27 09:03:38,816][20381] Updated weights for policy 1, policy_version 493768 (0.0010) [2023-12-27 09:03:39,259][20347] Updated weights for policy 0, policy_version 494087 (0.0008) [2023-12-27 09:03:39,319][20347] Updated weights for policy 0, policy_version 494097 (0.0006) [2023-12-27 09:03:39,381][20347] Updated weights for policy 0, policy_version 494107 (0.0008) [2023-12-27 09:03:39,562][20381] Updated weights for policy 1, policy_version 493778 (0.0011) [2023-12-27 09:03:39,627][20381] Updated weights for policy 1, policy_version 493788 (0.0011) [2023-12-27 09:03:39,686][20381] Updated weights for policy 1, policy_version 493798 (0.0011) [2023-12-27 09:03:39,736][20381] Updated weights for policy 1, policy_version 493808 (0.0011) [2023-12-27 09:03:40,077][20347] Updated weights for policy 0, policy_version 494117 (0.0006) [2023-12-27 09:03:40,137][20347] Updated weights for policy 0, policy_version 494127 (0.0007) [2023-12-27 09:03:40,198][20347] Updated weights for policy 0, policy_version 494137 (0.0008) [2023-12-27 09:03:40,438][20381] Updated weights for policy 1, policy_version 493818 (0.0006) [2023-12-27 09:03:40,489][20381] Updated weights for policy 1, policy_version 493828 (0.0005) [2023-12-27 09:03:40,545][20381] Updated weights for policy 1, policy_version 493838 (0.0005) [2023-12-27 09:03:40,889][20347] Updated weights for policy 0, policy_version 494147 (0.0008) [2023-12-27 09:03:40,949][20347] Updated weights for policy 0, policy_version 494157 (0.0007) [2023-12-27 09:03:41,010][20347] Updated weights for policy 0, policy_version 494167 (0.0008) [2023-12-27 09:03:41,072][20381] Updated weights for policy 1, policy_version 493848 (0.0007) [2023-12-27 09:03:41,134][20381] Updated weights for policy 1, policy_version 493858 (0.0006) [2023-12-27 09:03:41,198][20381] Updated weights for policy 1, policy_version 493868 (0.0009) [2023-12-27 09:03:41,710][20347] Updated weights for policy 0, policy_version 494177 (0.0007) [2023-12-27 09:03:41,768][20347] Updated weights for policy 0, policy_version 494187 (0.0008) [2023-12-27 09:03:41,821][20347] Updated weights for policy 0, policy_version 494197 (0.0010) [2023-12-27 09:03:41,880][20347] Updated weights for policy 0, policy_version 494207 (0.0011) [2023-12-27 09:03:41,900][20381] Updated weights for policy 1, policy_version 493878 (0.0008) [2023-12-27 09:03:41,960][20381] Updated weights for policy 1, policy_version 493888 (0.0007) [2023-12-27 09:03:42,023][20381] Updated weights for policy 1, policy_version 493898 (0.0007) [2023-12-27 09:03:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 252993536. Throughput: 0: 10576.6, 1: 10495.9. Samples: 252982968. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:42,363][19170] Avg episode reward: [(0, '837.080'), (1, '941.800')] [2023-12-27 09:03:42,544][20347] Updated weights for policy 0, policy_version 494217 (0.0007) [2023-12-27 09:03:42,592][20347] Updated weights for policy 0, policy_version 494227 (0.0010) [2023-12-27 09:03:42,647][20347] Updated weights for policy 0, policy_version 494237 (0.0010) [2023-12-27 09:03:42,720][20381] Updated weights for policy 1, policy_version 493908 (0.0007) [2023-12-27 09:03:42,770][20381] Updated weights for policy 1, policy_version 493918 (0.0005) [2023-12-27 09:03:42,818][20381] Updated weights for policy 1, policy_version 493928 (0.0010) [2023-12-27 09:03:43,276][20347] Updated weights for policy 0, policy_version 494247 (0.0007) [2023-12-27 09:03:43,322][20347] Updated weights for policy 0, policy_version 494257 (0.0005) [2023-12-27 09:03:43,376][20347] Updated weights for policy 0, policy_version 494267 (0.0005) [2023-12-27 09:03:43,491][20381] Updated weights for policy 1, policy_version 493938 (0.0010) [2023-12-27 09:03:43,548][20381] Updated weights for policy 1, policy_version 493948 (0.0010) [2023-12-27 09:03:43,594][20381] Updated weights for policy 1, policy_version 493958 (0.0010) [2023-12-27 09:03:43,640][20381] Updated weights for policy 1, policy_version 493968 (0.0010) [2023-12-27 09:03:43,970][20347] Updated weights for policy 0, policy_version 494277 (0.0009) [2023-12-27 09:03:44,020][20347] Updated weights for policy 0, policy_version 494287 (0.0006) [2023-12-27 09:03:44,070][20347] Updated weights for policy 0, policy_version 494297 (0.0005) [2023-12-27 09:03:44,314][20381] Updated weights for policy 1, policy_version 493978 (0.0009) [2023-12-27 09:03:44,368][20381] Updated weights for policy 1, policy_version 493988 (0.0010) [2023-12-27 09:03:44,426][20381] Updated weights for policy 1, policy_version 493998 (0.0010) [2023-12-27 09:03:44,779][20347] Updated weights for policy 0, policy_version 494307 (0.0007) [2023-12-27 09:03:44,832][20347] Updated weights for policy 0, policy_version 494317 (0.0007) [2023-12-27 09:03:44,881][20347] Updated weights for policy 0, policy_version 494327 (0.0007) [2023-12-27 09:03:44,973][20381] Updated weights for policy 1, policy_version 494008 (0.0008) [2023-12-27 09:03:45,035][20381] Updated weights for policy 1, policy_version 494018 (0.0007) [2023-12-27 09:03:45,086][20381] Updated weights for policy 1, policy_version 494028 (0.0008) [2023-12-27 09:03:45,541][20347] Updated weights for policy 0, policy_version 494337 (0.0008) [2023-12-27 09:03:45,597][20347] Updated weights for policy 0, policy_version 494347 (0.0008) [2023-12-27 09:03:45,652][20347] Updated weights for policy 0, policy_version 494357 (0.0008) [2023-12-27 09:03:45,714][20347] Updated weights for policy 0, policy_version 494367 (0.0008) [2023-12-27 09:03:45,792][20381] Updated weights for policy 1, policy_version 494038 (0.0008) [2023-12-27 09:03:45,847][20381] Updated weights for policy 1, policy_version 494048 (0.0008) [2023-12-27 09:03:45,895][20381] Updated weights for policy 1, policy_version 494058 (0.0010) [2023-12-27 09:03:46,381][20347] Updated weights for policy 0, policy_version 494377 (0.0008) [2023-12-27 09:03:46,437][20347] Updated weights for policy 0, policy_version 494387 (0.0008) [2023-12-27 09:03:46,496][20347] Updated weights for policy 0, policy_version 494397 (0.0007) [2023-12-27 09:03:46,607][20381] Updated weights for policy 1, policy_version 494068 (0.0009) [2023-12-27 09:03:46,656][20381] Updated weights for policy 1, policy_version 494078 (0.0008) [2023-12-27 09:03:46,712][20381] Updated weights for policy 1, policy_version 494088 (0.0009) [2023-12-27 09:03:47,123][20347] Updated weights for policy 0, policy_version 494407 (0.0007) [2023-12-27 09:03:47,172][20347] Updated weights for policy 0, policy_version 494417 (0.0007) [2023-12-27 09:03:47,223][20347] Updated weights for policy 0, policy_version 494427 (0.0007) [2023-12-27 09:03:47,279][20381] Updated weights for policy 1, policy_version 494098 (0.0008) [2023-12-27 09:03:47,337][20381] Updated weights for policy 1, policy_version 494108 (0.0005) [2023-12-27 09:03:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 21162.6, 300 sec: 20854.9). Total num frames: 253100032. Throughput: 0: 10581.2, 1: 10472.3. Samples: 253109676. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:47,364][19170] Avg episode reward: [(0, '741.100'), (1, '941.890')] [2023-12-27 09:03:47,395][20381] Updated weights for policy 1, policy_version 494118 (0.0006) [2023-12-27 09:03:47,440][20381] Updated weights for policy 1, policy_version 494128 (0.0005) [2023-12-27 09:03:47,973][20347] Updated weights for policy 0, policy_version 494437 (0.0008) [2023-12-27 09:03:48,029][20347] Updated weights for policy 0, policy_version 494447 (0.0006) [2023-12-27 09:03:48,035][20381] Updated weights for policy 1, policy_version 494138 (0.0010) [2023-12-27 09:03:48,077][20347] Updated weights for policy 0, policy_version 494457 (0.0005) [2023-12-27 09:03:48,082][20381] Updated weights for policy 1, policy_version 494148 (0.0010) [2023-12-27 09:03:48,139][20381] Updated weights for policy 1, policy_version 494158 (0.0010) [2023-12-27 09:03:48,788][20347] Updated weights for policy 0, policy_version 494467 (0.0006) [2023-12-27 09:03:48,847][20347] Updated weights for policy 0, policy_version 494477 (0.0009) [2023-12-27 09:03:48,850][20381] Updated weights for policy 1, policy_version 494168 (0.0007) [2023-12-27 09:03:48,903][20381] Updated weights for policy 1, policy_version 494178 (0.0005) [2023-12-27 09:03:48,910][20347] Updated weights for policy 0, policy_version 494487 (0.0007) [2023-12-27 09:03:48,959][20381] Updated weights for policy 1, policy_version 494188 (0.0006) [2023-12-27 09:03:49,630][20347] Updated weights for policy 0, policy_version 494497 (0.0007) [2023-12-27 09:03:49,633][20381] Updated weights for policy 1, policy_version 494198 (0.0007) [2023-12-27 09:03:49,689][20381] Updated weights for policy 1, policy_version 494208 (0.0007) [2023-12-27 09:03:49,690][20347] Updated weights for policy 0, policy_version 494507 (0.0007) [2023-12-27 09:03:49,749][20347] Updated weights for policy 0, policy_version 494517 (0.0006) [2023-12-27 09:03:49,752][20381] Updated weights for policy 1, policy_version 494218 (0.0006) [2023-12-27 09:03:49,808][20347] Updated weights for policy 0, policy_version 494527 (0.0007) [2023-12-27 09:03:50,383][20381] Updated weights for policy 1, policy_version 494228 (0.0006) [2023-12-27 09:03:50,444][20381] Updated weights for policy 1, policy_version 494238 (0.0006) [2023-12-27 09:03:50,495][20381] Updated weights for policy 1, policy_version 494248 (0.0005) [2023-12-27 09:03:50,570][20347] Updated weights for policy 0, policy_version 494537 (0.0008) [2023-12-27 09:03:50,626][20347] Updated weights for policy 0, policy_version 494547 (0.0008) [2023-12-27 09:03:50,682][20347] Updated weights for policy 0, policy_version 494557 (0.0006) [2023-12-27 09:03:51,205][20381] Updated weights for policy 1, policy_version 494258 (0.0006) [2023-12-27 09:03:51,258][20381] Updated weights for policy 1, policy_version 494268 (0.0011) [2023-12-27 09:03:51,317][20381] Updated weights for policy 1, policy_version 494278 (0.0010) [2023-12-27 09:03:51,333][20347] Updated weights for policy 0, policy_version 494567 (0.0010) [2023-12-27 09:03:51,343][20292] KL-divergence is very high: 127.8566 [2023-12-27 09:03:51,362][20292] KL-divergence is very high: 582.5214 [2023-12-27 09:03:51,381][20381] Updated weights for policy 1, policy_version 494288 (0.0011) [2023-12-27 09:03:51,400][20347] Updated weights for policy 0, policy_version 494577 (0.0011) [2023-12-27 09:03:51,468][20347] Updated weights for policy 0, policy_version 494587 (0.0007) [2023-12-27 09:03:52,063][20292] KL-divergence is very high: 192.0870 [2023-12-27 09:03:52,119][20381] Updated weights for policy 1, policy_version 494298 (0.0007) [2023-12-27 09:03:52,150][20292] KL-divergence is very high: 120.3445 [2023-12-27 09:03:52,167][20347] Updated weights for policy 0, policy_version 494597 (0.0009) [2023-12-27 09:03:52,170][20381] Updated weights for policy 1, policy_version 494308 (0.0006) [2023-12-27 09:03:52,223][20347] Updated weights for policy 0, policy_version 494607 (0.0011) [2023-12-27 09:03:52,229][20381] Updated weights for policy 1, policy_version 494318 (0.0005) [2023-12-27 09:03:52,280][20347] Updated weights for policy 0, policy_version 494617 (0.0011) [2023-12-27 09:03:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 253206528. Throughput: 0: 10486.9, 1: 10448.4. Samples: 253169976. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:52,363][19170] Avg episode reward: [(0, '673.690'), (1, '865.840')] [2023-12-27 09:03:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000494320_126566400.pth... [2023-12-27 09:03:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000494624_126640128.pth... [2023-12-27 09:03:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000493072_126246912.pth [2023-12-27 09:03:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000493376_126320640.pth [2023-12-27 09:03:52,867][20381] Updated weights for policy 1, policy_version 494328 (0.0007) [2023-12-27 09:03:52,914][20381] Updated weights for policy 1, policy_version 494338 (0.0008) [2023-12-27 09:03:52,965][20381] Updated weights for policy 1, policy_version 494348 (0.0007) [2023-12-27 09:03:52,970][20347] Updated weights for policy 0, policy_version 494627 (0.0011) [2023-12-27 09:03:53,033][20347] Updated weights for policy 0, policy_version 494637 (0.0010) [2023-12-27 09:03:53,088][20347] Updated weights for policy 0, policy_version 494647 (0.0010) [2023-12-27 09:03:53,642][20381] Updated weights for policy 1, policy_version 494358 (0.0007) [2023-12-27 09:03:53,686][20381] Updated weights for policy 1, policy_version 494368 (0.0007) [2023-12-27 09:03:53,734][20381] Updated weights for policy 1, policy_version 494378 (0.0007) [2023-12-27 09:03:53,768][20347] Updated weights for policy 0, policy_version 494657 (0.0010) [2023-12-27 09:03:53,844][20347] Updated weights for policy 0, policy_version 494667 (0.0010) [2023-12-27 09:03:53,895][20347] Updated weights for policy 0, policy_version 494677 (0.0010) [2023-12-27 09:03:53,951][20347] Updated weights for policy 0, policy_version 494687 (0.0010) [2023-12-27 09:03:54,393][20381] Updated weights for policy 1, policy_version 494388 (0.0006) [2023-12-27 09:03:54,451][20381] Updated weights for policy 1, policy_version 494398 (0.0007) [2023-12-27 09:03:54,510][20381] Updated weights for policy 1, policy_version 494408 (0.0007) [2023-12-27 09:03:54,619][20347] Updated weights for policy 0, policy_version 494697 (0.0010) [2023-12-27 09:03:54,683][20347] Updated weights for policy 0, policy_version 494707 (0.0010) [2023-12-27 09:03:54,732][20347] Updated weights for policy 0, policy_version 494717 (0.0010) [2023-12-27 09:03:55,059][20381] Updated weights for policy 1, policy_version 494418 (0.0008) [2023-12-27 09:03:55,113][20381] Updated weights for policy 1, policy_version 494428 (0.0009) [2023-12-27 09:03:55,171][20381] Updated weights for policy 1, policy_version 494438 (0.0008) [2023-12-27 09:03:55,215][20381] Updated weights for policy 1, policy_version 494448 (0.0007) [2023-12-27 09:03:55,299][20347] Updated weights for policy 0, policy_version 494727 (0.0007) [2023-12-27 09:03:55,357][20347] Updated weights for policy 0, policy_version 494737 (0.0010) [2023-12-27 09:03:55,420][20347] Updated weights for policy 0, policy_version 494747 (0.0007) [2023-12-27 09:03:55,962][20381] Updated weights for policy 1, policy_version 494458 (0.0007) [2023-12-27 09:03:56,014][20381] Updated weights for policy 1, policy_version 494468 (0.0007) [2023-12-27 09:03:56,020][20347] Updated weights for policy 0, policy_version 494757 (0.0005) [2023-12-27 09:03:56,064][20347] Updated weights for policy 0, policy_version 494767 (0.0006) [2023-12-27 09:03:56,069][20381] Updated weights for policy 1, policy_version 494478 (0.0006) [2023-12-27 09:03:56,111][20347] Updated weights for policy 0, policy_version 494777 (0.0006) [2023-12-27 09:03:56,686][20381] Updated weights for policy 1, policy_version 494488 (0.0005) [2023-12-27 09:03:56,741][20381] Updated weights for policy 1, policy_version 494498 (0.0005) [2023-12-27 09:03:56,743][20347] Updated weights for policy 0, policy_version 494787 (0.0007) [2023-12-27 09:03:56,794][20381] Updated weights for policy 1, policy_version 494508 (0.0005) [2023-12-27 09:03:56,803][20347] Updated weights for policy 0, policy_version 494797 (0.0007) [2023-12-27 09:03:56,849][20347] Updated weights for policy 0, policy_version 494807 (0.0007) [2023-12-27 09:03:57,340][20381] Updated weights for policy 1, policy_version 494518 (0.0007) [2023-12-27 09:03:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 253304832. Throughput: 0: 10506.4, 1: 10487.4. Samples: 253295036. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:03:57,363][19170] Avg episode reward: [(0, '696.080'), (1, '914.320')] [2023-12-27 09:03:57,396][20381] Updated weights for policy 1, policy_version 494528 (0.0009) [2023-12-27 09:03:57,452][20381] Updated weights for policy 1, policy_version 494538 (0.0008) [2023-12-27 09:03:57,479][20347] Updated weights for policy 0, policy_version 494817 (0.0008) [2023-12-27 09:03:57,535][20347] Updated weights for policy 0, policy_version 494827 (0.0005) [2023-12-27 09:03:57,587][20347] Updated weights for policy 0, policy_version 494837 (0.0010) [2023-12-27 09:03:57,637][20347] Updated weights for policy 0, policy_version 494847 (0.0010) [2023-12-27 09:03:58,168][20381] Updated weights for policy 1, policy_version 494548 (0.0007) [2023-12-27 09:03:58,231][20381] Updated weights for policy 1, policy_version 494558 (0.0007) [2023-12-27 09:03:58,276][20347] Updated weights for policy 0, policy_version 494857 (0.0011) [2023-12-27 09:03:58,293][20381] Updated weights for policy 1, policy_version 494568 (0.0006) [2023-12-27 09:03:58,333][20347] Updated weights for policy 0, policy_version 494867 (0.0009) [2023-12-27 09:03:58,403][20347] Updated weights for policy 0, policy_version 494877 (0.0009) [2023-12-27 09:03:59,003][20381] Updated weights for policy 1, policy_version 494578 (0.0007) [2023-12-27 09:03:59,056][20381] Updated weights for policy 1, policy_version 494588 (0.0008) [2023-12-27 09:03:59,115][20381] Updated weights for policy 1, policy_version 494598 (0.0007) [2023-12-27 09:03:59,180][20381] Updated weights for policy 1, policy_version 494608 (0.0007) [2023-12-27 09:03:59,182][20347] Updated weights for policy 0, policy_version 494887 (0.0009) [2023-12-27 09:03:59,247][20347] Updated weights for policy 0, policy_version 494897 (0.0010) [2023-12-27 09:03:59,307][20347] Updated weights for policy 0, policy_version 494907 (0.0007) [2023-12-27 09:03:59,951][20381] Updated weights for policy 1, policy_version 494618 (0.0007) [2023-12-27 09:03:59,979][20347] Updated weights for policy 0, policy_version 494917 (0.0010) [2023-12-27 09:04:00,012][20381] Updated weights for policy 1, policy_version 494628 (0.0007) [2023-12-27 09:04:00,046][20347] Updated weights for policy 0, policy_version 494927 (0.0008) [2023-12-27 09:04:00,073][20381] Updated weights for policy 1, policy_version 494638 (0.0007) [2023-12-27 09:04:00,111][20347] Updated weights for policy 0, policy_version 494937 (0.0006) [2023-12-27 09:04:00,704][20347] Updated weights for policy 0, policy_version 494947 (0.0006) [2023-12-27 09:04:00,768][20347] Updated weights for policy 0, policy_version 494957 (0.0006) [2023-12-27 09:04:00,777][20381] Updated weights for policy 1, policy_version 494648 (0.0006) [2023-12-27 09:04:00,828][20347] Updated weights for policy 0, policy_version 494967 (0.0005) [2023-12-27 09:04:00,837][20381] Updated weights for policy 1, policy_version 494658 (0.0005) [2023-12-27 09:04:00,894][20381] Updated weights for policy 1, policy_version 494668 (0.0006) [2023-12-27 09:04:01,493][20347] Updated weights for policy 0, policy_version 494977 (0.0005) [2023-12-27 09:04:01,516][20381] Updated weights for policy 1, policy_version 494678 (0.0008) [2023-12-27 09:04:01,550][20347] Updated weights for policy 0, policy_version 494987 (0.0008) [2023-12-27 09:04:01,574][20381] Updated weights for policy 1, policy_version 494688 (0.0006) [2023-12-27 09:04:01,598][20347] Updated weights for policy 0, policy_version 494997 (0.0007) [2023-12-27 09:04:01,638][20381] Updated weights for policy 1, policy_version 494698 (0.0007) [2023-12-27 09:04:01,660][20347] Updated weights for policy 0, policy_version 495007 (0.0008) [2023-12-27 09:04:02,279][20381] Updated weights for policy 1, policy_version 494708 (0.0006) [2023-12-27 09:04:02,283][20347] Updated weights for policy 0, policy_version 495017 (0.0007) [2023-12-27 09:04:02,337][20381] Updated weights for policy 1, policy_version 494718 (0.0006) [2023-12-27 09:04:02,345][20347] Updated weights for policy 0, policy_version 495027 (0.0006) [2023-12-27 09:04:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 253403136. Throughput: 0: 10463.2, 1: 10451.8. Samples: 253420032. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 09:04:02,363][19170] Avg episode reward: [(0, '593.480'), (1, '923.140')] [2023-12-27 09:04:02,396][20381] Updated weights for policy 1, policy_version 494728 (0.0006) [2023-12-27 09:04:02,406][20347] Updated weights for policy 0, policy_version 495037 (0.0007) [2023-12-27 09:04:03,031][20381] Updated weights for policy 1, policy_version 494738 (0.0008) [2023-12-27 09:04:03,037][20347] Updated weights for policy 0, policy_version 495047 (0.0006) [2023-12-27 09:04:03,079][20381] Updated weights for policy 1, policy_version 494748 (0.0006) [2023-12-27 09:04:03,088][20347] Updated weights for policy 0, policy_version 495057 (0.0006) [2023-12-27 09:04:03,130][20381] Updated weights for policy 1, policy_version 494758 (0.0005) [2023-12-27 09:04:03,142][20347] Updated weights for policy 0, policy_version 495067 (0.0005) [2023-12-27 09:04:03,188][20381] Updated weights for policy 1, policy_version 494768 (0.0007) [2023-12-27 09:04:03,789][20381] Updated weights for policy 1, policy_version 494778 (0.0011) [2023-12-27 09:04:03,849][20381] Updated weights for policy 1, policy_version 494788 (0.0009) [2023-12-27 09:04:03,856][20347] Updated weights for policy 0, policy_version 495077 (0.0007) [2023-12-27 09:04:03,903][20381] Updated weights for policy 1, policy_version 494798 (0.0006) [2023-12-27 09:04:03,916][20347] Updated weights for policy 0, policy_version 495087 (0.0007) [2023-12-27 09:04:03,979][20347] Updated weights for policy 0, policy_version 495097 (0.0008) [2023-12-27 09:04:04,531][20381] Updated weights for policy 1, policy_version 494808 (0.0009) [2023-12-27 09:04:04,588][20381] Updated weights for policy 1, policy_version 494818 (0.0011) [2023-12-27 09:04:04,643][20381] Updated weights for policy 1, policy_version 494828 (0.0011) [2023-12-27 09:04:04,686][20347] Updated weights for policy 0, policy_version 495107 (0.0007) [2023-12-27 09:04:04,737][20347] Updated weights for policy 0, policy_version 495117 (0.0007) [2023-12-27 09:04:04,794][20347] Updated weights for policy 0, policy_version 495127 (0.0007) [2023-12-27 09:04:05,310][20381] Updated weights for policy 1, policy_version 494838 (0.0008) [2023-12-27 09:04:05,363][20381] Updated weights for policy 1, policy_version 494848 (0.0005) [2023-12-27 09:04:05,415][20381] Updated weights for policy 1, policy_version 494858 (0.0006) [2023-12-27 09:04:05,499][20347] Updated weights for policy 0, policy_version 495137 (0.0008) [2023-12-27 09:04:05,547][20347] Updated weights for policy 0, policy_version 495147 (0.0007) [2023-12-27 09:04:05,602][20347] Updated weights for policy 0, policy_version 495157 (0.0008) [2023-12-27 09:04:05,653][20347] Updated weights for policy 0, policy_version 495167 (0.0007) [2023-12-27 09:04:05,997][20381] Updated weights for policy 1, policy_version 494868 (0.0006) [2023-12-27 09:04:06,056][20381] Updated weights for policy 1, policy_version 494878 (0.0006) [2023-12-27 09:04:06,113][20381] Updated weights for policy 1, policy_version 494888 (0.0007) [2023-12-27 09:04:06,419][20347] Updated weights for policy 0, policy_version 495177 (0.0006) [2023-12-27 09:04:06,467][20347] Updated weights for policy 0, policy_version 495187 (0.0009) [2023-12-27 09:04:06,517][20347] Updated weights for policy 0, policy_version 495197 (0.0007) [2023-12-27 09:04:06,702][20381] Updated weights for policy 1, policy_version 494898 (0.0007) [2023-12-27 09:04:06,761][20381] Updated weights for policy 1, policy_version 494908 (0.0005) [2023-12-27 09:04:06,820][20381] Updated weights for policy 1, policy_version 494918 (0.0005) [2023-12-27 09:04:06,872][20381] Updated weights for policy 1, policy_version 494928 (0.0006) [2023-12-27 09:04:07,259][20347] Updated weights for policy 0, policy_version 495207 (0.0007) [2023-12-27 09:04:07,310][20347] Updated weights for policy 0, policy_version 495217 (0.0005) [2023-12-27 09:04:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 253509632. Throughput: 0: 10404.3, 1: 10514.4. Samples: 253482744. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:07,363][19170] Avg episode reward: [(0, '435.150'), (1, '903.300')] [2023-12-27 09:04:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000494928_126722048.pth... [2023-12-27 09:04:07,369][20347] Updated weights for policy 0, policy_version 495227 (0.0007) [2023-12-27 09:04:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000493712_126410752.pth [2023-12-27 09:04:07,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000495232_126795776.pth... [2023-12-27 09:04:07,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000494016_126484480.pth [2023-12-27 09:04:07,486][20381] Updated weights for policy 1, policy_version 494938 (0.0006) [2023-12-27 09:04:07,530][20381] Updated weights for policy 1, policy_version 494948 (0.0005) [2023-12-27 09:04:07,586][20381] Updated weights for policy 1, policy_version 494958 (0.0011) [2023-12-27 09:04:08,044][20347] Updated weights for policy 0, policy_version 495237 (0.0007) [2023-12-27 09:04:08,097][20347] Updated weights for policy 0, policy_version 495247 (0.0009) [2023-12-27 09:04:08,152][20381] Updated weights for policy 1, policy_version 494968 (0.0006) [2023-12-27 09:04:08,157][20347] Updated weights for policy 0, policy_version 495257 (0.0007) [2023-12-27 09:04:08,203][20381] Updated weights for policy 1, policy_version 494978 (0.0009) [2023-12-27 09:04:08,259][20381] Updated weights for policy 1, policy_version 494988 (0.0006) [2023-12-27 09:04:08,857][20381] Updated weights for policy 1, policy_version 494998 (0.0009) [2023-12-27 09:04:08,896][20347] Updated weights for policy 0, policy_version 495267 (0.0007) [2023-12-27 09:04:08,920][20381] Updated weights for policy 1, policy_version 495008 (0.0009) [2023-12-27 09:04:08,955][20347] Updated weights for policy 0, policy_version 495277 (0.0005) [2023-12-27 09:04:08,978][20381] Updated weights for policy 1, policy_version 495018 (0.0007) [2023-12-27 09:04:09,012][20347] Updated weights for policy 0, policy_version 495287 (0.0005) [2023-12-27 09:04:09,630][20347] Updated weights for policy 0, policy_version 495297 (0.0007) [2023-12-27 09:04:09,691][20347] Updated weights for policy 0, policy_version 495307 (0.0006) [2023-12-27 09:04:09,692][20381] Updated weights for policy 1, policy_version 495028 (0.0007) [2023-12-27 09:04:09,746][20347] Updated weights for policy 0, policy_version 495317 (0.0006) [2023-12-27 09:04:09,751][20381] Updated weights for policy 1, policy_version 495038 (0.0008) [2023-12-27 09:04:09,800][20347] Updated weights for policy 0, policy_version 495327 (0.0006) [2023-12-27 09:04:09,809][20381] Updated weights for policy 1, policy_version 495048 (0.0008) [2023-12-27 09:04:10,563][20381] Updated weights for policy 1, policy_version 495058 (0.0009) [2023-12-27 09:04:10,598][20347] Updated weights for policy 0, policy_version 495337 (0.0010) [2023-12-27 09:04:10,617][20381] Updated weights for policy 1, policy_version 495068 (0.0010) [2023-12-27 09:04:10,657][20347] Updated weights for policy 0, policy_version 495347 (0.0011) [2023-12-27 09:04:10,672][20381] Updated weights for policy 1, policy_version 495078 (0.0011) [2023-12-27 09:04:10,716][20347] Updated weights for policy 0, policy_version 495357 (0.0010) [2023-12-27 09:04:10,724][20381] Updated weights for policy 1, policy_version 495088 (0.0011) [2023-12-27 09:04:11,396][20347] Updated weights for policy 0, policy_version 495367 (0.0011) [2023-12-27 09:04:11,427][20381] Updated weights for policy 1, policy_version 495098 (0.0009) [2023-12-27 09:04:11,463][20347] Updated weights for policy 0, policy_version 495377 (0.0011) [2023-12-27 09:04:11,489][20381] Updated weights for policy 1, policy_version 495108 (0.0005) [2023-12-27 09:04:11,527][20347] Updated weights for policy 0, policy_version 495387 (0.0009) [2023-12-27 09:04:11,548][20381] Updated weights for policy 1, policy_version 495118 (0.0007) [2023-12-27 09:04:12,162][20347] Updated weights for policy 0, policy_version 495397 (0.0008) [2023-12-27 09:04:12,214][20381] Updated weights for policy 1, policy_version 495128 (0.0011) [2023-12-27 09:04:12,226][20347] Updated weights for policy 0, policy_version 495407 (0.0007) [2023-12-27 09:04:12,279][20381] Updated weights for policy 1, policy_version 495138 (0.0012) [2023-12-27 09:04:12,289][20347] Updated weights for policy 0, policy_version 495417 (0.0006) [2023-12-27 09:04:12,339][20381] Updated weights for policy 1, policy_version 495148 (0.0010) [2023-12-27 09:04:12,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21162.6, 300 sec: 20882.7). Total num frames: 253624320. Throughput: 0: 10333.4, 1: 10543.7. Samples: 253606216. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:12,363][19170] Avg episode reward: [(0, '448.150'), (1, '922.690')] [2023-12-27 09:04:12,928][20347] Updated weights for policy 0, policy_version 495427 (0.0007) [2023-12-27 09:04:12,989][20347] Updated weights for policy 0, policy_version 495437 (0.0005) [2023-12-27 09:04:13,046][20381] Updated weights for policy 1, policy_version 495158 (0.0010) [2023-12-27 09:04:13,050][20347] Updated weights for policy 0, policy_version 495447 (0.0008) [2023-12-27 09:04:13,094][20381] Updated weights for policy 1, policy_version 495168 (0.0010) [2023-12-27 09:04:13,148][20381] Updated weights for policy 1, policy_version 495178 (0.0010) [2023-12-27 09:04:13,661][20347] Updated weights for policy 0, policy_version 495457 (0.0010) [2023-12-27 09:04:13,714][20347] Updated weights for policy 0, policy_version 495467 (0.0007) [2023-12-27 09:04:13,770][20347] Updated weights for policy 0, policy_version 495477 (0.0007) [2023-12-27 09:04:13,810][20381] Updated weights for policy 1, policy_version 495188 (0.0011) [2023-12-27 09:04:13,824][20347] Updated weights for policy 0, policy_version 495487 (0.0005) [2023-12-27 09:04:13,856][20381] Updated weights for policy 1, policy_version 495198 (0.0009) [2023-12-27 09:04:13,903][20381] Updated weights for policy 1, policy_version 495208 (0.0005) [2023-12-27 09:04:13,911][20292] KL-divergence is very high: 131.4956 [2023-12-27 09:04:13,915][20292] KL-divergence is very high: 111.1982 [2023-12-27 09:04:14,440][20347] Updated weights for policy 0, policy_version 495497 (0.0010) [2023-12-27 09:04:14,490][20347] Updated weights for policy 0, policy_version 495507 (0.0009) [2023-12-27 09:04:14,517][20381] Updated weights for policy 1, policy_version 495218 (0.0007) [2023-12-27 09:04:14,536][20347] Updated weights for policy 0, policy_version 495517 (0.0008) [2023-12-27 09:04:14,568][20381] Updated weights for policy 1, policy_version 495228 (0.0010) [2023-12-27 09:04:14,616][20381] Updated weights for policy 1, policy_version 495238 (0.0010) [2023-12-27 09:04:14,672][20381] Updated weights for policy 1, policy_version 495248 (0.0006) [2023-12-27 09:04:15,201][20347] Updated weights for policy 0, policy_version 495527 (0.0007) [2023-12-27 09:04:15,252][20347] Updated weights for policy 0, policy_version 495537 (0.0008) [2023-12-27 09:04:15,301][20347] Updated weights for policy 0, policy_version 495547 (0.0007) [2023-12-27 09:04:15,386][20381] Updated weights for policy 1, policy_version 495258 (0.0011) [2023-12-27 09:04:15,434][20381] Updated weights for policy 1, policy_version 495268 (0.0010) [2023-12-27 09:04:15,482][20381] Updated weights for policy 1, policy_version 495278 (0.0010) [2023-12-27 09:04:16,010][20347] Updated weights for policy 0, policy_version 495557 (0.0007) [2023-12-27 09:04:16,074][20347] Updated weights for policy 0, policy_version 495567 (0.0008) [2023-12-27 09:04:16,135][20347] Updated weights for policy 0, policy_version 495577 (0.0007) [2023-12-27 09:04:16,191][20381] Updated weights for policy 1, policy_version 495288 (0.0011) [2023-12-27 09:04:16,249][20381] Updated weights for policy 1, policy_version 495298 (0.0011) [2023-12-27 09:04:16,308][20381] Updated weights for policy 1, policy_version 495308 (0.0011) [2023-12-27 09:04:16,779][20347] Updated weights for policy 0, policy_version 495587 (0.0005) [2023-12-27 09:04:16,833][20347] Updated weights for policy 0, policy_version 495597 (0.0005) [2023-12-27 09:04:16,883][20347] Updated weights for policy 0, policy_version 495607 (0.0006) [2023-12-27 09:04:16,951][20381] Updated weights for policy 1, policy_version 495318 (0.0007) [2023-12-27 09:04:17,007][20381] Updated weights for policy 1, policy_version 495328 (0.0006) [2023-12-27 09:04:17,054][20381] Updated weights for policy 1, policy_version 495338 (0.0005) [2023-12-27 09:04:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 253722624. Throughput: 0: 10327.9, 1: 10506.5. Samples: 253732240. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:17,363][19170] Avg episode reward: [(0, '420.900'), (1, '884.240')] [2023-12-27 09:04:17,574][20381] Updated weights for policy 1, policy_version 495348 (0.0005) [2023-12-27 09:04:17,609][20347] Updated weights for policy 0, policy_version 495617 (0.0007) [2023-12-27 09:04:17,630][20381] Updated weights for policy 1, policy_version 495358 (0.0006) [2023-12-27 09:04:17,659][20347] Updated weights for policy 0, policy_version 495627 (0.0006) [2023-12-27 09:04:17,684][20381] Updated weights for policy 1, policy_version 495368 (0.0006) [2023-12-27 09:04:17,702][20347] Updated weights for policy 0, policy_version 495637 (0.0006) [2023-12-27 09:04:17,750][20347] Updated weights for policy 0, policy_version 495647 (0.0006) [2023-12-27 09:04:18,253][20381] Updated weights for policy 1, policy_version 495378 (0.0006) [2023-12-27 09:04:18,298][20381] Updated weights for policy 1, policy_version 495388 (0.0007) [2023-12-27 09:04:18,354][20381] Updated weights for policy 1, policy_version 495398 (0.0008) [2023-12-27 09:04:18,414][20381] Updated weights for policy 1, policy_version 495408 (0.0005) [2023-12-27 09:04:18,422][20347] Updated weights for policy 0, policy_version 495657 (0.0010) [2023-12-27 09:04:18,480][20347] Updated weights for policy 0, policy_version 495667 (0.0008) [2023-12-27 09:04:18,538][20347] Updated weights for policy 0, policy_version 495677 (0.0007) [2023-12-27 09:04:19,086][20381] Updated weights for policy 1, policy_version 495418 (0.0006) [2023-12-27 09:04:19,148][20381] Updated weights for policy 1, policy_version 495428 (0.0005) [2023-12-27 09:04:19,207][20381] Updated weights for policy 1, policy_version 495438 (0.0007) [2023-12-27 09:04:19,224][20347] Updated weights for policy 0, policy_version 495687 (0.0007) [2023-12-27 09:04:19,280][20347] Updated weights for policy 0, policy_version 495697 (0.0008) [2023-12-27 09:04:19,325][20347] Updated weights for policy 0, policy_version 495707 (0.0007) [2023-12-27 09:04:19,923][20381] Updated weights for policy 1, policy_version 495448 (0.0008) [2023-12-27 09:04:19,965][20347] Updated weights for policy 0, policy_version 495717 (0.0008) [2023-12-27 09:04:19,986][20381] Updated weights for policy 1, policy_version 495458 (0.0007) [2023-12-27 09:04:20,022][20347] Updated weights for policy 0, policy_version 495727 (0.0007) [2023-12-27 09:04:20,047][20381] Updated weights for policy 1, policy_version 495468 (0.0007) [2023-12-27 09:04:20,082][20347] Updated weights for policy 0, policy_version 495737 (0.0007) [2023-12-27 09:04:20,749][20381] Updated weights for policy 1, policy_version 495478 (0.0008) [2023-12-27 09:04:20,766][20347] Updated weights for policy 0, policy_version 495747 (0.0007) [2023-12-27 09:04:20,807][20381] Updated weights for policy 1, policy_version 495488 (0.0006) [2023-12-27 09:04:20,817][20347] Updated weights for policy 0, policy_version 495757 (0.0006) [2023-12-27 09:04:20,858][20381] Updated weights for policy 1, policy_version 495498 (0.0006) [2023-12-27 09:04:20,874][20347] Updated weights for policy 0, policy_version 495767 (0.0007) [2023-12-27 09:04:20,881][20292] KL-divergence is very high: 280.4125 [2023-12-27 09:04:21,577][20381] Updated weights for policy 1, policy_version 495508 (0.0005) [2023-12-27 09:04:21,628][20381] Updated weights for policy 1, policy_version 495518 (0.0007) [2023-12-27 09:04:21,651][20347] Updated weights for policy 0, policy_version 495777 (0.0007) [2023-12-27 09:04:21,677][20292] KL-divergence is very high: 115.4988 [2023-12-27 09:04:21,695][20381] Updated weights for policy 1, policy_version 495528 (0.0006) [2023-12-27 09:04:21,715][20347] Updated weights for policy 0, policy_version 495787 (0.0007) [2023-12-27 09:04:21,773][20347] Updated weights for policy 0, policy_version 495797 (0.0008) [2023-12-27 09:04:21,836][20347] Updated weights for policy 0, policy_version 495807 (0.0006) [2023-12-27 09:04:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 253820928. Throughput: 0: 10291.4, 1: 10548.0. Samples: 253795076. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:22,363][19170] Avg episode reward: [(0, '406.770'), (1, '893.630')] [2023-12-27 09:04:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000495536_126877696.pth... [2023-12-27 09:04:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000495808_126943232.pth... [2023-12-27 09:04:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000494320_126566400.pth [2023-12-27 09:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000494624_126640128.pth [2023-12-27 09:04:22,402][20381] Updated weights for policy 1, policy_version 495538 (0.0008) [2023-12-27 09:04:22,464][20381] Updated weights for policy 1, policy_version 495548 (0.0008) [2023-12-27 09:04:22,474][20347] Updated weights for policy 0, policy_version 495817 (0.0006) [2023-12-27 09:04:22,526][20347] Updated weights for policy 0, policy_version 495827 (0.0007) [2023-12-27 09:04:22,529][20381] Updated weights for policy 1, policy_version 495558 (0.0008) [2023-12-27 09:04:22,584][20347] Updated weights for policy 0, policy_version 495837 (0.0007) [2023-12-27 09:04:22,586][20381] Updated weights for policy 1, policy_version 495568 (0.0006) [2023-12-27 09:04:23,258][20347] Updated weights for policy 0, policy_version 495847 (0.0007) [2023-12-27 09:04:23,278][20381] Updated weights for policy 1, policy_version 495578 (0.0006) [2023-12-27 09:04:23,316][20347] Updated weights for policy 0, policy_version 495857 (0.0006) [2023-12-27 09:04:23,329][20381] Updated weights for policy 1, policy_version 495588 (0.0006) [2023-12-27 09:04:23,360][20347] Updated weights for policy 0, policy_version 495867 (0.0006) [2023-12-27 09:04:23,391][20381] Updated weights for policy 1, policy_version 495598 (0.0007) [2023-12-27 09:04:24,064][20347] Updated weights for policy 0, policy_version 495877 (0.0006) [2023-12-27 09:04:24,098][20381] Updated weights for policy 1, policy_version 495608 (0.0008) [2023-12-27 09:04:24,124][20347] Updated weights for policy 0, policy_version 495887 (0.0008) [2023-12-27 09:04:24,155][20381] Updated weights for policy 1, policy_version 495618 (0.0007) [2023-12-27 09:04:24,177][20347] Updated weights for policy 0, policy_version 495897 (0.0010) [2023-12-27 09:04:24,208][20381] Updated weights for policy 1, policy_version 495628 (0.0007) [2023-12-27 09:04:24,716][20347] Updated weights for policy 0, policy_version 495907 (0.0005) [2023-12-27 09:04:24,772][20347] Updated weights for policy 0, policy_version 495917 (0.0005) [2023-12-27 09:04:24,825][20347] Updated weights for policy 0, policy_version 495927 (0.0006) [2023-12-27 09:04:24,969][20381] Updated weights for policy 1, policy_version 495638 (0.0008) [2023-12-27 09:04:25,014][20381] Updated weights for policy 1, policy_version 495648 (0.0008) [2023-12-27 09:04:25,069][20381] Updated weights for policy 1, policy_version 495658 (0.0005) [2023-12-27 09:04:25,342][20347] Updated weights for policy 0, policy_version 495937 (0.0005) [2023-12-27 09:04:25,401][20347] Updated weights for policy 0, policy_version 495947 (0.0005) [2023-12-27 09:04:25,454][20347] Updated weights for policy 0, policy_version 495957 (0.0005) [2023-12-27 09:04:25,500][20347] Updated weights for policy 0, policy_version 495967 (0.0005) [2023-12-27 09:04:25,793][20381] Updated weights for policy 1, policy_version 495668 (0.0006) [2023-12-27 09:04:25,820][20292] KL-divergence is very high: 101.3864 [2023-12-27 09:04:25,855][20381] Updated weights for policy 1, policy_version 495678 (0.0005) [2023-12-27 09:04:25,914][20381] Updated weights for policy 1, policy_version 495688 (0.0007) [2023-12-27 09:04:26,115][20347] Updated weights for policy 0, policy_version 495977 (0.0010) [2023-12-27 09:04:26,166][20347] Updated weights for policy 0, policy_version 495987 (0.0010) [2023-12-27 09:04:26,221][20347] Updated weights for policy 0, policy_version 495997 (0.0010) [2023-12-27 09:04:26,510][20381] Updated weights for policy 1, policy_version 495698 (0.0008) [2023-12-27 09:04:26,568][20381] Updated weights for policy 1, policy_version 495708 (0.0007) [2023-12-27 09:04:26,622][20381] Updated weights for policy 1, policy_version 495718 (0.0007) [2023-12-27 09:04:26,673][20381] Updated weights for policy 1, policy_version 495728 (0.0006) [2023-12-27 09:04:26,904][20347] Updated weights for policy 0, policy_version 496007 (0.0008) [2023-12-27 09:04:26,961][20347] Updated weights for policy 0, policy_version 496017 (0.0007) [2023-12-27 09:04:27,018][20347] Updated weights for policy 0, policy_version 496027 (0.0007) [2023-12-27 09:04:27,297][20381] Updated weights for policy 1, policy_version 495738 (0.0005) [2023-12-27 09:04:27,350][20381] Updated weights for policy 1, policy_version 495748 (0.0008) [2023-12-27 09:04:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 253927424. Throughput: 0: 10341.9, 1: 10449.4. Samples: 253918576. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:27,363][19170] Avg episode reward: [(0, '473.120'), (1, '893.480')] [2023-12-27 09:04:27,395][20381] Updated weights for policy 1, policy_version 495758 (0.0005) [2023-12-27 09:04:27,691][20347] Updated weights for policy 0, policy_version 496037 (0.0007) [2023-12-27 09:04:27,741][20347] Updated weights for policy 0, policy_version 496047 (0.0006) [2023-12-27 09:04:27,796][20347] Updated weights for policy 0, policy_version 496057 (0.0006) [2023-12-27 09:04:27,982][20381] Updated weights for policy 1, policy_version 495768 (0.0006) [2023-12-27 09:04:28,031][20381] Updated weights for policy 1, policy_version 495778 (0.0005) [2023-12-27 09:04:28,089][20381] Updated weights for policy 1, policy_version 495788 (0.0007) [2023-12-27 09:04:28,470][20347] Updated weights for policy 0, policy_version 496067 (0.0008) [2023-12-27 09:04:28,527][20347] Updated weights for policy 0, policy_version 496077 (0.0007) [2023-12-27 09:04:28,576][20347] Updated weights for policy 0, policy_version 496087 (0.0010) [2023-12-27 09:04:28,627][20381] Updated weights for policy 1, policy_version 495798 (0.0006) [2023-12-27 09:04:28,681][20381] Updated weights for policy 1, policy_version 495808 (0.0005) [2023-12-27 09:04:28,736][20381] Updated weights for policy 1, policy_version 495818 (0.0005) [2023-12-27 09:04:29,153][20347] Updated weights for policy 0, policy_version 496097 (0.0006) [2023-12-27 09:04:29,210][20347] Updated weights for policy 0, policy_version 496107 (0.0009) [2023-12-27 09:04:29,269][20347] Updated weights for policy 0, policy_version 496117 (0.0008) [2023-12-27 09:04:29,328][20347] Updated weights for policy 0, policy_version 496127 (0.0006) [2023-12-27 09:04:29,360][20381] Updated weights for policy 1, policy_version 495828 (0.0009) [2023-12-27 09:04:29,415][20381] Updated weights for policy 1, policy_version 495838 (0.0008) [2023-12-27 09:04:29,463][20381] Updated weights for policy 1, policy_version 495848 (0.0008) [2023-12-27 09:04:30,011][20347] Updated weights for policy 0, policy_version 496137 (0.0007) [2023-12-27 09:04:30,070][20347] Updated weights for policy 0, policy_version 496147 (0.0007) [2023-12-27 09:04:30,125][20347] Updated weights for policy 0, policy_version 496157 (0.0007) [2023-12-27 09:04:30,170][20381] Updated weights for policy 1, policy_version 495858 (0.0008) [2023-12-27 09:04:30,225][20381] Updated weights for policy 1, policy_version 495868 (0.0008) [2023-12-27 09:04:30,279][20381] Updated weights for policy 1, policy_version 495878 (0.0008) [2023-12-27 09:04:30,331][20381] Updated weights for policy 1, policy_version 495888 (0.0008) [2023-12-27 09:04:30,817][20347] Updated weights for policy 0, policy_version 496167 (0.0006) [2023-12-27 09:04:30,871][20347] Updated weights for policy 0, policy_version 496177 (0.0008) [2023-12-27 09:04:30,925][20347] Updated weights for policy 0, policy_version 496187 (0.0007) [2023-12-27 09:04:31,053][20381] Updated weights for policy 1, policy_version 495898 (0.0009) [2023-12-27 09:04:31,105][20381] Updated weights for policy 1, policy_version 495908 (0.0008) [2023-12-27 09:04:31,163][20381] Updated weights for policy 1, policy_version 495918 (0.0008) [2023-12-27 09:04:31,645][20347] Updated weights for policy 0, policy_version 496197 (0.0010) [2023-12-27 09:04:31,706][20347] Updated weights for policy 0, policy_version 496207 (0.0008) [2023-12-27 09:04:31,764][20347] Updated weights for policy 0, policy_version 496217 (0.0009) [2023-12-27 09:04:31,854][20381] Updated weights for policy 1, policy_version 495928 (0.0008) [2023-12-27 09:04:31,913][20381] Updated weights for policy 1, policy_version 495938 (0.0007) [2023-12-27 09:04:31,966][20381] Updated weights for policy 1, policy_version 495948 (0.0006) [2023-12-27 09:04:32,351][20347] Updated weights for policy 0, policy_version 496227 (0.0008) [2023-12-27 09:04:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 254033920. Throughput: 0: 10307.0, 1: 10489.4. Samples: 254045516. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:32,363][19170] Avg episode reward: [(0, '520.710'), (1, '951.670')] [2023-12-27 09:04:32,412][20347] Updated weights for policy 0, policy_version 496237 (0.0011) [2023-12-27 09:04:32,457][20347] Updated weights for policy 0, policy_version 496247 (0.0010) [2023-12-27 09:04:32,648][20381] Updated weights for policy 1, policy_version 495958 (0.0006) [2023-12-27 09:04:32,707][20381] Updated weights for policy 1, policy_version 495968 (0.0006) [2023-12-27 09:04:32,758][20381] Updated weights for policy 1, policy_version 495978 (0.0006) [2023-12-27 09:04:33,165][20347] Updated weights for policy 0, policy_version 496257 (0.0010) [2023-12-27 09:04:33,218][20347] Updated weights for policy 0, policy_version 496267 (0.0010) [2023-12-27 09:04:33,268][20347] Updated weights for policy 0, policy_version 496277 (0.0010) [2023-12-27 09:04:33,322][20347] Updated weights for policy 0, policy_version 496287 (0.0010) [2023-12-27 09:04:33,420][20381] Updated weights for policy 1, policy_version 495988 (0.0007) [2023-12-27 09:04:33,466][20381] Updated weights for policy 1, policy_version 495998 (0.0007) [2023-12-27 09:04:33,506][20381] Updated weights for policy 1, policy_version 496008 (0.0007) [2023-12-27 09:04:34,010][20347] Updated weights for policy 0, policy_version 496297 (0.0008) [2023-12-27 09:04:34,061][20347] Updated weights for policy 0, policy_version 496307 (0.0007) [2023-12-27 09:04:34,115][20347] Updated weights for policy 0, policy_version 496317 (0.0007) [2023-12-27 09:04:34,172][20381] Updated weights for policy 1, policy_version 496018 (0.0007) [2023-12-27 09:04:34,228][20381] Updated weights for policy 1, policy_version 496028 (0.0008) [2023-12-27 09:04:34,274][20381] Updated weights for policy 1, policy_version 496038 (0.0008) [2023-12-27 09:04:34,335][20381] Updated weights for policy 1, policy_version 496048 (0.0005) [2023-12-27 09:04:34,840][20347] Updated weights for policy 0, policy_version 496327 (0.0008) [2023-12-27 09:04:34,899][20347] Updated weights for policy 0, policy_version 496337 (0.0008) [2023-12-27 09:04:34,955][20347] Updated weights for policy 0, policy_version 496347 (0.0007) [2023-12-27 09:04:35,013][20381] Updated weights for policy 1, policy_version 496058 (0.0007) [2023-12-27 09:04:35,059][20381] Updated weights for policy 1, policy_version 496068 (0.0007) [2023-12-27 09:04:35,108][20381] Updated weights for policy 1, policy_version 496078 (0.0008) [2023-12-27 09:04:35,657][20347] Updated weights for policy 0, policy_version 496357 (0.0007) [2023-12-27 09:04:35,708][20347] Updated weights for policy 0, policy_version 496367 (0.0008) [2023-12-27 09:04:35,756][20347] Updated weights for policy 0, policy_version 496377 (0.0007) [2023-12-27 09:04:35,813][20381] Updated weights for policy 1, policy_version 496088 (0.0007) [2023-12-27 09:04:35,859][20381] Updated weights for policy 1, policy_version 496098 (0.0008) [2023-12-27 09:04:35,911][20381] Updated weights for policy 1, policy_version 496108 (0.0008) [2023-12-27 09:04:36,488][20347] Updated weights for policy 0, policy_version 496387 (0.0007) [2023-12-27 09:04:36,548][20347] Updated weights for policy 0, policy_version 496397 (0.0007) [2023-12-27 09:04:36,607][20347] Updated weights for policy 0, policy_version 496407 (0.0007) [2023-12-27 09:04:36,634][20381] Updated weights for policy 1, policy_version 496118 (0.0009) [2023-12-27 09:04:36,694][20381] Updated weights for policy 1, policy_version 496128 (0.0006) [2023-12-27 09:04:36,753][20381] Updated weights for policy 1, policy_version 496138 (0.0006) [2023-12-27 09:04:37,342][20347] Updated weights for policy 0, policy_version 496417 (0.0009) [2023-12-27 09:04:37,358][20381] Updated weights for policy 1, policy_version 496148 (0.0007) [2023-12-27 09:04:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 254132224. Throughput: 0: 10338.5, 1: 10472.0. Samples: 254106448. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:37,363][19170] Avg episode reward: [(0, '513.190'), (1, '980.740')] [2023-12-27 09:04:37,394][20347] Updated weights for policy 0, policy_version 496427 (0.0010) [2023-12-27 09:04:37,407][20381] Updated weights for policy 1, policy_version 496158 (0.0006) [2023-12-27 09:04:37,453][20347] Updated weights for policy 0, policy_version 496437 (0.0010) [2023-12-27 09:04:37,453][20381] Updated weights for policy 1, policy_version 496168 (0.0007) [2023-12-27 09:04:37,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000496176_127041536.pth... [2023-12-27 09:04:37,500][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000494928_126722048.pth [2023-12-27 09:04:37,509][20347] Updated weights for policy 0, policy_version 496447 (0.0010) [2023-12-27 09:04:37,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000496448_127107072.pth... [2023-12-27 09:04:37,516][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000495232_126795776.pth [2023-12-27 09:04:38,140][20347] Updated weights for policy 0, policy_version 496457 (0.0008) [2023-12-27 09:04:38,196][20347] Updated weights for policy 0, policy_version 496467 (0.0010) [2023-12-27 09:04:38,212][20381] Updated weights for policy 1, policy_version 496178 (0.0008) [2023-12-27 09:04:38,244][20347] Updated weights for policy 0, policy_version 496477 (0.0010) [2023-12-27 09:04:38,268][20381] Updated weights for policy 1, policy_version 496188 (0.0005) [2023-12-27 09:04:38,336][20381] Updated weights for policy 1, policy_version 496198 (0.0008) [2023-12-27 09:04:38,401][20381] Updated weights for policy 1, policy_version 496208 (0.0008) [2023-12-27 09:04:38,927][20347] Updated weights for policy 0, policy_version 496487 (0.0010) [2023-12-27 09:04:38,995][20347] Updated weights for policy 0, policy_version 496497 (0.0008) [2023-12-27 09:04:39,066][20347] Updated weights for policy 0, policy_version 496507 (0.0006) [2023-12-27 09:04:39,127][20381] Updated weights for policy 1, policy_version 496218 (0.0008) [2023-12-27 09:04:39,182][20381] Updated weights for policy 1, policy_version 496228 (0.0008) [2023-12-27 09:04:39,244][20381] Updated weights for policy 1, policy_version 496238 (0.0006) [2023-12-27 09:04:39,762][20347] Updated weights for policy 0, policy_version 496517 (0.0007) [2023-12-27 09:04:39,820][20347] Updated weights for policy 0, policy_version 496527 (0.0008) [2023-12-27 09:04:39,886][20347] Updated weights for policy 0, policy_version 496537 (0.0007) [2023-12-27 09:04:39,937][20381] Updated weights for policy 1, policy_version 496248 (0.0008) [2023-12-27 09:04:39,997][20381] Updated weights for policy 1, policy_version 496258 (0.0008) [2023-12-27 09:04:40,002][20292] KL-divergence is very high: 115.2430 [2023-12-27 09:04:40,051][20381] Updated weights for policy 1, policy_version 496268 (0.0008) [2023-12-27 09:04:40,573][20347] Updated weights for policy 0, policy_version 496547 (0.0006) [2023-12-27 09:04:40,637][20347] Updated weights for policy 0, policy_version 496557 (0.0006) [2023-12-27 09:04:40,700][20347] Updated weights for policy 0, policy_version 496567 (0.0007) [2023-12-27 09:04:40,717][20381] Updated weights for policy 1, policy_version 496278 (0.0006) [2023-12-27 09:04:40,760][20381] Updated weights for policy 1, policy_version 496288 (0.0007) [2023-12-27 09:04:40,824][20381] Updated weights for policy 1, policy_version 496298 (0.0008) [2023-12-27 09:04:41,317][20347] Updated weights for policy 0, policy_version 496577 (0.0007) [2023-12-27 09:04:41,383][20347] Updated weights for policy 0, policy_version 496587 (0.0008) [2023-12-27 09:04:41,448][20347] Updated weights for policy 0, policy_version 496597 (0.0008) [2023-12-27 09:04:41,509][20347] Updated weights for policy 0, policy_version 496607 (0.0008) [2023-12-27 09:04:41,588][20381] Updated weights for policy 1, policy_version 496308 (0.0008) [2023-12-27 09:04:41,654][20381] Updated weights for policy 1, policy_version 496318 (0.0008) [2023-12-27 09:04:41,706][20381] Updated weights for policy 1, policy_version 496328 (0.0007) [2023-12-27 09:04:42,216][20347] Updated weights for policy 0, policy_version 496617 (0.0008) [2023-12-27 09:04:42,273][20347] Updated weights for policy 0, policy_version 496627 (0.0007) [2023-12-27 09:04:42,331][20347] Updated weights for policy 0, policy_version 496637 (0.0008) [2023-12-27 09:04:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 254238720. Throughput: 0: 10315.0, 1: 10419.1. Samples: 254228068. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:42,363][19170] Avg episode reward: [(0, '446.130'), (1, '980.760')] [2023-12-27 09:04:42,410][20381] Updated weights for policy 1, policy_version 496338 (0.0008) [2023-12-27 09:04:42,459][20381] Updated weights for policy 1, policy_version 496348 (0.0007) [2023-12-27 09:04:42,508][20381] Updated weights for policy 1, policy_version 496358 (0.0007) [2023-12-27 09:04:42,562][20381] Updated weights for policy 1, policy_version 496368 (0.0008) [2023-12-27 09:04:43,029][20347] Updated weights for policy 0, policy_version 496647 (0.0008) [2023-12-27 09:04:43,078][20347] Updated weights for policy 0, policy_version 496657 (0.0008) [2023-12-27 09:04:43,132][20347] Updated weights for policy 0, policy_version 496667 (0.0008) [2023-12-27 09:04:43,249][20381] Updated weights for policy 1, policy_version 496378 (0.0007) [2023-12-27 09:04:43,304][20381] Updated weights for policy 1, policy_version 496388 (0.0007) [2023-12-27 09:04:43,358][20381] Updated weights for policy 1, policy_version 496398 (0.0007) [2023-12-27 09:04:43,837][20347] Updated weights for policy 0, policy_version 496677 (0.0008) [2023-12-27 09:04:43,888][20347] Updated weights for policy 0, policy_version 496687 (0.0007) [2023-12-27 09:04:43,936][20347] Updated weights for policy 0, policy_version 496697 (0.0008) [2023-12-27 09:04:44,050][20381] Updated weights for policy 1, policy_version 496408 (0.0008) [2023-12-27 09:04:44,114][20381] Updated weights for policy 1, policy_version 496418 (0.0008) [2023-12-27 09:04:44,179][20381] Updated weights for policy 1, policy_version 496428 (0.0008) [2023-12-27 09:04:44,641][20347] Updated weights for policy 0, policy_version 496707 (0.0007) [2023-12-27 09:04:44,704][20347] Updated weights for policy 0, policy_version 496717 (0.0007) [2023-12-27 09:04:44,760][20347] Updated weights for policy 0, policy_version 496727 (0.0007) [2023-12-27 09:04:44,809][20381] Updated weights for policy 1, policy_version 496438 (0.0007) [2023-12-27 09:04:44,865][20381] Updated weights for policy 1, policy_version 496448 (0.0007) [2023-12-27 09:04:44,919][20381] Updated weights for policy 1, policy_version 496458 (0.0008) [2023-12-27 09:04:45,441][20347] Updated weights for policy 0, policy_version 496737 (0.0007) [2023-12-27 09:04:45,493][20347] Updated weights for policy 0, policy_version 496747 (0.0008) [2023-12-27 09:04:45,548][20347] Updated weights for policy 0, policy_version 496757 (0.0007) [2023-12-27 09:04:45,607][20347] Updated weights for policy 0, policy_version 496767 (0.0007) [2023-12-27 09:04:45,624][20381] Updated weights for policy 1, policy_version 496468 (0.0008) [2023-12-27 09:04:45,670][20381] Updated weights for policy 1, policy_version 496478 (0.0008) [2023-12-27 09:04:45,716][20381] Updated weights for policy 1, policy_version 496488 (0.0008) [2023-12-27 09:04:46,309][20347] Updated weights for policy 0, policy_version 496777 (0.0007) [2023-12-27 09:04:46,362][20347] Updated weights for policy 0, policy_version 496787 (0.0008) [2023-12-27 09:04:46,418][20381] Updated weights for policy 1, policy_version 496498 (0.0007) [2023-12-27 09:04:46,420][20347] Updated weights for policy 0, policy_version 496797 (0.0007) [2023-12-27 09:04:46,481][20381] Updated weights for policy 1, policy_version 496508 (0.0007) [2023-12-27 09:04:46,543][20381] Updated weights for policy 1, policy_version 496518 (0.0008) [2023-12-27 09:04:46,599][20381] Updated weights for policy 1, policy_version 496528 (0.0008) [2023-12-27 09:04:47,114][20347] Updated weights for policy 0, policy_version 496807 (0.0007) [2023-12-27 09:04:47,164][20347] Updated weights for policy 0, policy_version 496817 (0.0007) [2023-12-27 09:04:47,218][20347] Updated weights for policy 0, policy_version 496827 (0.0007) [2023-12-27 09:04:47,283][20381] Updated weights for policy 1, policy_version 496538 (0.0007) [2023-12-27 09:04:47,332][20381] Updated weights for policy 1, policy_version 496548 (0.0008) [2023-12-27 09:04:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 254337024. Throughput: 0: 10239.6, 1: 10374.9. Samples: 254347680. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:47,363][19170] Avg episode reward: [(0, '588.650'), (1, '961.520')] [2023-12-27 09:04:47,385][20381] Updated weights for policy 1, policy_version 496558 (0.0008) [2023-12-27 09:04:47,854][20347] Updated weights for policy 0, policy_version 496837 (0.0007) [2023-12-27 09:04:47,899][20347] Updated weights for policy 0, policy_version 496847 (0.0008) [2023-12-27 09:04:47,943][20347] Updated weights for policy 0, policy_version 496857 (0.0007) [2023-12-27 09:04:48,116][20381] Updated weights for policy 1, policy_version 496568 (0.0008) [2023-12-27 09:04:48,183][20381] Updated weights for policy 1, policy_version 496578 (0.0008) [2023-12-27 09:04:48,245][20381] Updated weights for policy 1, policy_version 496588 (0.0008) [2023-12-27 09:04:48,565][20347] Updated weights for policy 0, policy_version 496867 (0.0007) [2023-12-27 09:04:48,624][20347] Updated weights for policy 0, policy_version 496877 (0.0008) [2023-12-27 09:04:48,669][20347] Updated weights for policy 0, policy_version 496887 (0.0007) [2023-12-27 09:04:48,951][20381] Updated weights for policy 1, policy_version 496598 (0.0008) [2023-12-27 09:04:49,003][20381] Updated weights for policy 1, policy_version 496608 (0.0008) [2023-12-27 09:04:49,060][20381] Updated weights for policy 1, policy_version 496618 (0.0008) [2023-12-27 09:04:49,347][20347] Updated weights for policy 0, policy_version 496897 (0.0007) [2023-12-27 09:04:49,408][20347] Updated weights for policy 0, policy_version 496907 (0.0008) [2023-12-27 09:04:49,473][20347] Updated weights for policy 0, policy_version 496917 (0.0008) [2023-12-27 09:04:49,534][20347] Updated weights for policy 0, policy_version 496927 (0.0009) [2023-12-27 09:04:49,779][20381] Updated weights for policy 1, policy_version 496628 (0.0007) [2023-12-27 09:04:49,842][20381] Updated weights for policy 1, policy_version 496638 (0.0007) [2023-12-27 09:04:49,895][20381] Updated weights for policy 1, policy_version 496648 (0.0007) [2023-12-27 09:04:50,203][20347] Updated weights for policy 0, policy_version 496937 (0.0006) [2023-12-27 09:04:50,247][20347] Updated weights for policy 0, policy_version 496947 (0.0005) [2023-12-27 09:04:50,296][20347] Updated weights for policy 0, policy_version 496957 (0.0005) [2023-12-27 09:04:50,632][20381] Updated weights for policy 1, policy_version 496658 (0.0008) [2023-12-27 09:04:50,688][20381] Updated weights for policy 1, policy_version 496668 (0.0008) [2023-12-27 09:04:50,741][20381] Updated weights for policy 1, policy_version 496678 (0.0007) [2023-12-27 09:04:50,793][20381] Updated weights for policy 1, policy_version 496688 (0.0007) [2023-12-27 09:04:50,939][20347] Updated weights for policy 0, policy_version 496967 (0.0008) [2023-12-27 09:04:50,995][20347] Updated weights for policy 0, policy_version 496977 (0.0006) [2023-12-27 09:04:51,058][20347] Updated weights for policy 0, policy_version 496987 (0.0010) [2023-12-27 09:04:51,533][20381] Updated weights for policy 1, policy_version 496698 (0.0006) [2023-12-27 09:04:51,588][20381] Updated weights for policy 1, policy_version 496708 (0.0008) [2023-12-27 09:04:51,655][20381] Updated weights for policy 1, policy_version 496718 (0.0008) [2023-12-27 09:04:51,735][20347] Updated weights for policy 0, policy_version 496997 (0.0010) [2023-12-27 09:04:51,799][20347] Updated weights for policy 0, policy_version 497007 (0.0007) [2023-12-27 09:04:51,865][20347] Updated weights for policy 0, policy_version 497017 (0.0008) [2023-12-27 09:04:52,359][20381] Updated weights for policy 1, policy_version 496728 (0.0007) [2023-12-27 09:04:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20799.4). Total num frames: 254435328. Throughput: 0: 10270.7, 1: 10301.4. Samples: 254408488. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:52,363][19170] Avg episode reward: [(0, '484.220'), (1, '942.410')] [2023-12-27 09:04:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000497024_127254528.pth... [2023-12-27 09:04:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000495808_126943232.pth [2023-12-27 09:04:52,414][20381] Updated weights for policy 1, policy_version 496738 (0.0007) [2023-12-27 09:04:52,464][20381] Updated weights for policy 1, policy_version 496748 (0.0008) [2023-12-27 09:04:52,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000496752_127188992.pth... [2023-12-27 09:04:52,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000495536_126877696.pth [2023-12-27 09:04:52,506][20347] Updated weights for policy 0, policy_version 497027 (0.0008) [2023-12-27 09:04:52,566][20347] Updated weights for policy 0, policy_version 497037 (0.0006) [2023-12-27 09:04:52,629][20347] Updated weights for policy 0, policy_version 497047 (0.0005) [2023-12-27 09:04:53,142][20381] Updated weights for policy 1, policy_version 496758 (0.0007) [2023-12-27 09:04:53,206][20381] Updated weights for policy 1, policy_version 496768 (0.0006) [2023-12-27 09:04:53,252][20381] Updated weights for policy 1, policy_version 496778 (0.0005) [2023-12-27 09:04:53,275][20347] Updated weights for policy 0, policy_version 497057 (0.0010) [2023-12-27 09:04:53,321][20347] Updated weights for policy 0, policy_version 497067 (0.0010) [2023-12-27 09:04:53,370][20347] Updated weights for policy 0, policy_version 497077 (0.0010) [2023-12-27 09:04:53,425][20347] Updated weights for policy 0, policy_version 497087 (0.0010) [2023-12-27 09:04:53,817][20381] Updated weights for policy 1, policy_version 496788 (0.0007) [2023-12-27 09:04:53,862][20381] Updated weights for policy 1, policy_version 496798 (0.0010) [2023-12-27 09:04:53,914][20381] Updated weights for policy 1, policy_version 496808 (0.0009) [2023-12-27 09:04:54,076][20347] Updated weights for policy 0, policy_version 497097 (0.0011) [2023-12-27 09:04:54,127][20347] Updated weights for policy 0, policy_version 497107 (0.0010) [2023-12-27 09:04:54,186][20347] Updated weights for policy 0, policy_version 497117 (0.0010) [2023-12-27 09:04:54,600][20381] Updated weights for policy 1, policy_version 496818 (0.0006) [2023-12-27 09:04:54,661][20381] Updated weights for policy 1, policy_version 496828 (0.0009) [2023-12-27 09:04:54,722][20381] Updated weights for policy 1, policy_version 496838 (0.0010) [2023-12-27 09:04:54,783][20381] Updated weights for policy 1, policy_version 496848 (0.0010) [2023-12-27 09:04:54,844][20347] Updated weights for policy 0, policy_version 497127 (0.0007) [2023-12-27 09:04:54,913][20347] Updated weights for policy 0, policy_version 497137 (0.0007) [2023-12-27 09:04:54,974][20347] Updated weights for policy 0, policy_version 497147 (0.0010) [2023-12-27 09:04:55,464][20381] Updated weights for policy 1, policy_version 496858 (0.0009) [2023-12-27 09:04:55,524][20381] Updated weights for policy 1, policy_version 496868 (0.0009) [2023-12-27 09:04:55,585][20381] Updated weights for policy 1, policy_version 496878 (0.0010) [2023-12-27 09:04:55,616][20347] Updated weights for policy 0, policy_version 497157 (0.0009) [2023-12-27 09:04:55,668][20347] Updated weights for policy 0, policy_version 497167 (0.0010) [2023-12-27 09:04:55,717][20347] Updated weights for policy 0, policy_version 497177 (0.0010) [2023-12-27 09:04:56,240][20381] Updated weights for policy 1, policy_version 496888 (0.0010) [2023-12-27 09:04:56,302][20381] Updated weights for policy 1, policy_version 496898 (0.0010) [2023-12-27 09:04:56,352][20347] Updated weights for policy 0, policy_version 497187 (0.0009) [2023-12-27 09:04:56,360][20381] Updated weights for policy 1, policy_version 496908 (0.0010) [2023-12-27 09:04:56,408][20347] Updated weights for policy 0, policy_version 497197 (0.0005) [2023-12-27 09:04:56,459][20347] Updated weights for policy 0, policy_version 497207 (0.0005) [2023-12-27 09:04:57,016][20347] Updated weights for policy 0, policy_version 497217 (0.0005) [2023-12-27 09:04:57,069][20347] Updated weights for policy 0, policy_version 497227 (0.0005) [2023-12-27 09:04:57,071][20381] Updated weights for policy 1, policy_version 496918 (0.0009) [2023-12-27 09:04:57,122][20381] Updated weights for policy 1, policy_version 496928 (0.0007) [2023-12-27 09:04:57,127][20347] Updated weights for policy 0, policy_version 497237 (0.0007) [2023-12-27 09:04:57,169][20381] Updated weights for policy 1, policy_version 496938 (0.0006) [2023-12-27 09:04:57,181][20347] Updated weights for policy 0, policy_version 497247 (0.0010) [2023-12-27 09:04:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 254550016. Throughput: 0: 10371.0, 1: 10238.4. Samples: 254533640. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:04:57,363][19170] Avg episode reward: [(0, '371.570'), (1, '951.940')] [2023-12-27 09:04:57,756][20347] Updated weights for policy 0, policy_version 497257 (0.0008) [2023-12-27 09:04:57,799][20347] Updated weights for policy 0, policy_version 497267 (0.0007) [2023-12-27 09:04:57,845][20347] Updated weights for policy 0, policy_version 497277 (0.0007) [2023-12-27 09:04:57,889][20381] Updated weights for policy 1, policy_version 496948 (0.0007) [2023-12-27 09:04:57,934][20381] Updated weights for policy 1, policy_version 496958 (0.0007) [2023-12-27 09:04:57,980][20381] Updated weights for policy 1, policy_version 496968 (0.0005) [2023-12-27 09:04:58,625][20347] Updated weights for policy 0, policy_version 497287 (0.0006) [2023-12-27 09:04:58,633][20381] Updated weights for policy 1, policy_version 496978 (0.0006) [2023-12-27 09:04:58,681][20347] Updated weights for policy 0, policy_version 497297 (0.0007) [2023-12-27 09:04:58,696][20381] Updated weights for policy 1, policy_version 496988 (0.0008) [2023-12-27 09:04:58,743][20347] Updated weights for policy 0, policy_version 497307 (0.0007) [2023-12-27 09:04:58,758][20381] Updated weights for policy 1, policy_version 496998 (0.0008) [2023-12-27 09:04:58,824][20381] Updated weights for policy 1, policy_version 497008 (0.0008) [2023-12-27 09:04:59,401][20347] Updated weights for policy 0, policy_version 497317 (0.0007) [2023-12-27 09:04:59,450][20347] Updated weights for policy 0, policy_version 497327 (0.0008) [2023-12-27 09:04:59,506][20347] Updated weights for policy 0, policy_version 497337 (0.0008) [2023-12-27 09:04:59,592][20381] Updated weights for policy 1, policy_version 497018 (0.0007) [2023-12-27 09:04:59,639][20381] Updated weights for policy 1, policy_version 497028 (0.0007) [2023-12-27 09:04:59,692][20381] Updated weights for policy 1, policy_version 497038 (0.0005) [2023-12-27 09:05:00,198][20347] Updated weights for policy 0, policy_version 497347 (0.0007) [2023-12-27 09:05:00,243][20347] Updated weights for policy 0, policy_version 497357 (0.0008) [2023-12-27 09:05:00,289][20347] Updated weights for policy 0, policy_version 497367 (0.0007) [2023-12-27 09:05:00,394][20381] Updated weights for policy 1, policy_version 497048 (0.0008) [2023-12-27 09:05:00,451][20381] Updated weights for policy 1, policy_version 497058 (0.0006) [2023-12-27 09:05:00,504][20381] Updated weights for policy 1, policy_version 497068 (0.0006) [2023-12-27 09:05:00,941][20347] Updated weights for policy 0, policy_version 497377 (0.0007) [2023-12-27 09:05:00,991][20347] Updated weights for policy 0, policy_version 497387 (0.0010) [2023-12-27 09:05:01,051][20347] Updated weights for policy 0, policy_version 497397 (0.0008) [2023-12-27 09:05:01,060][20381] Updated weights for policy 1, policy_version 497078 (0.0011) [2023-12-27 09:05:01,099][20347] Updated weights for policy 0, policy_version 497407 (0.0006) [2023-12-27 09:05:01,119][20381] Updated weights for policy 1, policy_version 497088 (0.0006) [2023-12-27 09:05:01,183][20381] Updated weights for policy 1, policy_version 497098 (0.0009) [2023-12-27 09:05:01,780][20347] Updated weights for policy 0, policy_version 497417 (0.0008) [2023-12-27 09:05:01,825][20381] Updated weights for policy 1, policy_version 497108 (0.0008) [2023-12-27 09:05:01,832][20347] Updated weights for policy 0, policy_version 497427 (0.0009) [2023-12-27 09:05:01,874][20381] Updated weights for policy 1, policy_version 497118 (0.0010) [2023-12-27 09:05:01,888][20347] Updated weights for policy 0, policy_version 497437 (0.0008) [2023-12-27 09:05:01,932][20381] Updated weights for policy 1, policy_version 497128 (0.0005) [2023-12-27 09:05:02,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 254648320. Throughput: 0: 10380.4, 1: 10219.6. Samples: 254659244. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:02,363][19170] Avg episode reward: [(0, '620.530'), (1, '913.460')] [2023-12-27 09:05:02,574][20347] Updated weights for policy 0, policy_version 497447 (0.0007) [2023-12-27 09:05:02,595][20381] Updated weights for policy 1, policy_version 497138 (0.0005) [2023-12-27 09:05:02,632][20347] Updated weights for policy 0, policy_version 497457 (0.0007) [2023-12-27 09:05:02,652][20381] Updated weights for policy 1, policy_version 497148 (0.0007) [2023-12-27 09:05:02,690][20347] Updated weights for policy 0, policy_version 497467 (0.0008) [2023-12-27 09:05:02,710][20381] Updated weights for policy 1, policy_version 497158 (0.0005) [2023-12-27 09:05:02,766][20381] Updated weights for policy 1, policy_version 497168 (0.0008) [2023-12-27 09:05:03,324][20347] Updated weights for policy 0, policy_version 497477 (0.0008) [2023-12-27 09:05:03,385][20347] Updated weights for policy 0, policy_version 497487 (0.0010) [2023-12-27 09:05:03,414][20381] Updated weights for policy 1, policy_version 497178 (0.0007) [2023-12-27 09:05:03,435][20347] Updated weights for policy 0, policy_version 497497 (0.0009) [2023-12-27 09:05:03,467][20381] Updated weights for policy 1, policy_version 497188 (0.0005) [2023-12-27 09:05:03,513][20381] Updated weights for policy 1, policy_version 497198 (0.0005) [2023-12-27 09:05:04,012][20347] Updated weights for policy 0, policy_version 497507 (0.0006) [2023-12-27 09:05:04,048][20381] Updated weights for policy 1, policy_version 497208 (0.0007) [2023-12-27 09:05:04,062][20347] Updated weights for policy 0, policy_version 497517 (0.0006) [2023-12-27 09:05:04,104][20381] Updated weights for policy 1, policy_version 497218 (0.0007) [2023-12-27 09:05:04,109][20347] Updated weights for policy 0, policy_version 497527 (0.0006) [2023-12-27 09:05:04,157][20381] Updated weights for policy 1, policy_version 497228 (0.0007) [2023-12-27 09:05:04,827][20381] Updated weights for policy 1, policy_version 497238 (0.0008) [2023-12-27 09:05:04,860][20347] Updated weights for policy 0, policy_version 497537 (0.0006) [2023-12-27 09:05:04,877][20381] Updated weights for policy 1, policy_version 497248 (0.0006) [2023-12-27 09:05:04,916][20347] Updated weights for policy 0, policy_version 497547 (0.0007) [2023-12-27 09:05:04,932][20381] Updated weights for policy 1, policy_version 497258 (0.0006) [2023-12-27 09:05:04,972][20347] Updated weights for policy 0, policy_version 497557 (0.0007) [2023-12-27 09:05:05,034][20347] Updated weights for policy 0, policy_version 497567 (0.0007) [2023-12-27 09:05:05,672][20381] Updated weights for policy 1, policy_version 497268 (0.0007) [2023-12-27 09:05:05,676][20347] Updated weights for policy 0, policy_version 497577 (0.0006) [2023-12-27 09:05:05,720][20347] Updated weights for policy 0, policy_version 497587 (0.0005) [2023-12-27 09:05:05,730][20381] Updated weights for policy 1, policy_version 497278 (0.0008) [2023-12-27 09:05:05,766][20347] Updated weights for policy 0, policy_version 497597 (0.0005) [2023-12-27 09:05:05,780][20381] Updated weights for policy 1, policy_version 497288 (0.0008) [2023-12-27 09:05:06,347][20347] Updated weights for policy 0, policy_version 497607 (0.0006) [2023-12-27 09:05:06,406][20347] Updated weights for policy 0, policy_version 497617 (0.0006) [2023-12-27 09:05:06,449][20381] Updated weights for policy 1, policy_version 497298 (0.0008) [2023-12-27 09:05:06,462][20347] Updated weights for policy 0, policy_version 497627 (0.0005) [2023-12-27 09:05:06,513][20381] Updated weights for policy 1, policy_version 497308 (0.0008) [2023-12-27 09:05:06,568][20381] Updated weights for policy 1, policy_version 497318 (0.0008) [2023-12-27 09:05:06,632][20381] Updated weights for policy 1, policy_version 497328 (0.0009) [2023-12-27 09:05:06,995][20347] Updated weights for policy 0, policy_version 497637 (0.0006) [2023-12-27 09:05:07,052][20347] Updated weights for policy 0, policy_version 497647 (0.0007) [2023-12-27 09:05:07,116][20347] Updated weights for policy 0, policy_version 497657 (0.0006) [2023-12-27 09:05:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 254754816. Throughput: 0: 10399.8, 1: 10228.7. Samples: 254723360. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:07,364][19170] Avg episode reward: [(0, '686.080'), (1, '942.330')] [2023-12-27 09:05:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000497664_127418368.pth... [2023-12-27 09:05:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000496448_127107072.pth [2023-12-27 09:05:07,388][20381] Updated weights for policy 1, policy_version 497338 (0.0009) [2023-12-27 09:05:07,434][20381] Updated weights for policy 1, policy_version 497348 (0.0009) [2023-12-27 09:05:07,483][20381] Updated weights for policy 1, policy_version 497358 (0.0009) [2023-12-27 09:05:07,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000497360_127344640.pth... [2023-12-27 09:05:07,497][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000496176_127041536.pth [2023-12-27 09:05:07,688][20347] Updated weights for policy 0, policy_version 497667 (0.0009) [2023-12-27 09:05:07,745][20347] Updated weights for policy 0, policy_version 497677 (0.0008) [2023-12-27 09:05:07,800][20347] Updated weights for policy 0, policy_version 497687 (0.0008) [2023-12-27 09:05:08,195][20381] Updated weights for policy 1, policy_version 497368 (0.0006) [2023-12-27 09:05:08,261][20381] Updated weights for policy 1, policy_version 497378 (0.0005) [2023-12-27 09:05:08,325][20381] Updated weights for policy 1, policy_version 497388 (0.0006) [2023-12-27 09:05:08,432][20347] Updated weights for policy 0, policy_version 497697 (0.0007) [2023-12-27 09:05:08,490][20347] Updated weights for policy 0, policy_version 497707 (0.0006) [2023-12-27 09:05:08,552][20347] Updated weights for policy 0, policy_version 497717 (0.0006) [2023-12-27 09:05:08,611][20347] Updated weights for policy 0, policy_version 497727 (0.0008) [2023-12-27 09:05:08,848][20381] Updated weights for policy 1, policy_version 497398 (0.0007) [2023-12-27 09:05:08,911][20381] Updated weights for policy 1, policy_version 497408 (0.0006) [2023-12-27 09:05:08,970][20381] Updated weights for policy 1, policy_version 497418 (0.0005) [2023-12-27 09:05:09,287][20347] Updated weights for policy 0, policy_version 497737 (0.0008) [2023-12-27 09:05:09,339][20347] Updated weights for policy 0, policy_version 497747 (0.0007) [2023-12-27 09:05:09,408][20347] Updated weights for policy 0, policy_version 497757 (0.0008) [2023-12-27 09:05:09,591][20381] Updated weights for policy 1, policy_version 497428 (0.0005) [2023-12-27 09:05:09,646][20381] Updated weights for policy 1, policy_version 497438 (0.0005) [2023-12-27 09:05:09,699][20381] Updated weights for policy 1, policy_version 497448 (0.0007) [2023-12-27 09:05:10,163][20347] Updated weights for policy 0, policy_version 497767 (0.0009) [2023-12-27 09:05:10,226][20347] Updated weights for policy 0, policy_version 497777 (0.0010) [2023-12-27 09:05:10,289][20347] Updated weights for policy 0, policy_version 497787 (0.0010) [2023-12-27 09:05:10,389][20381] Updated weights for policy 1, policy_version 497458 (0.0008) [2023-12-27 09:05:10,450][20381] Updated weights for policy 1, policy_version 497468 (0.0008) [2023-12-27 09:05:10,507][20381] Updated weights for policy 1, policy_version 497478 (0.0005) [2023-12-27 09:05:10,562][20381] Updated weights for policy 1, policy_version 497488 (0.0005) [2023-12-27 09:05:10,964][20347] Updated weights for policy 0, policy_version 497797 (0.0008) [2023-12-27 09:05:11,031][20347] Updated weights for policy 0, policy_version 497807 (0.0007) [2023-12-27 09:05:11,094][20347] Updated weights for policy 0, policy_version 497817 (0.0007) [2023-12-27 09:05:11,240][20381] Updated weights for policy 1, policy_version 497498 (0.0006) [2023-12-27 09:05:11,291][20381] Updated weights for policy 1, policy_version 497508 (0.0006) [2023-12-27 09:05:11,352][20381] Updated weights for policy 1, policy_version 497518 (0.0007) [2023-12-27 09:05:11,817][20347] Updated weights for policy 0, policy_version 497827 (0.0008) [2023-12-27 09:05:11,873][20347] Updated weights for policy 0, policy_version 497837 (0.0006) [2023-12-27 09:05:11,922][20347] Updated weights for policy 0, policy_version 497847 (0.0006) [2023-12-27 09:05:12,060][20381] Updated weights for policy 1, policy_version 497528 (0.0008) [2023-12-27 09:05:12,117][20381] Updated weights for policy 1, policy_version 497538 (0.0007) [2023-12-27 09:05:12,170][20381] Updated weights for policy 1, policy_version 497548 (0.0008) [2023-12-27 09:05:12,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 254861312. Throughput: 0: 10405.3, 1: 10294.4. Samples: 254850064. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:12,363][19170] Avg episode reward: [(0, '779.200'), (1, '990.410')] [2023-12-27 09:05:12,634][20347] Updated weights for policy 0, policy_version 497857 (0.0007) [2023-12-27 09:05:12,693][20347] Updated weights for policy 0, policy_version 497867 (0.0008) [2023-12-27 09:05:12,745][20347] Updated weights for policy 0, policy_version 497877 (0.0008) [2023-12-27 09:05:12,805][20347] Updated weights for policy 0, policy_version 497887 (0.0008) [2023-12-27 09:05:12,830][20381] Updated weights for policy 1, policy_version 497558 (0.0007) [2023-12-27 09:05:12,881][20381] Updated weights for policy 1, policy_version 497568 (0.0007) [2023-12-27 09:05:12,940][20381] Updated weights for policy 1, policy_version 497578 (0.0007) [2023-12-27 09:05:13,504][20347] Updated weights for policy 0, policy_version 497897 (0.0007) [2023-12-27 09:05:13,519][20381] Updated weights for policy 1, policy_version 497588 (0.0007) [2023-12-27 09:05:13,561][20347] Updated weights for policy 0, policy_version 497907 (0.0006) [2023-12-27 09:05:13,571][20381] Updated weights for policy 1, policy_version 497598 (0.0006) [2023-12-27 09:05:13,614][20347] Updated weights for policy 0, policy_version 497917 (0.0006) [2023-12-27 09:05:13,623][20381] Updated weights for policy 1, policy_version 497608 (0.0006) [2023-12-27 09:05:14,255][20347] Updated weights for policy 0, policy_version 497927 (0.0007) [2023-12-27 09:05:14,257][20381] Updated weights for policy 1, policy_version 497618 (0.0007) [2023-12-27 09:05:14,309][20381] Updated weights for policy 1, policy_version 497628 (0.0005) [2023-12-27 09:05:14,314][20347] Updated weights for policy 0, policy_version 497937 (0.0008) [2023-12-27 09:05:14,368][20381] Updated weights for policy 1, policy_version 497638 (0.0008) [2023-12-27 09:05:14,369][20347] Updated weights for policy 0, policy_version 497947 (0.0009) [2023-12-27 09:05:14,428][20381] Updated weights for policy 1, policy_version 497648 (0.0007) [2023-12-27 09:05:15,051][20347] Updated weights for policy 0, policy_version 497957 (0.0007) [2023-12-27 09:05:15,068][20381] Updated weights for policy 1, policy_version 497658 (0.0008) [2023-12-27 09:05:15,109][20347] Updated weights for policy 0, policy_version 497967 (0.0005) [2023-12-27 09:05:15,122][20381] Updated weights for policy 1, policy_version 497668 (0.0008) [2023-12-27 09:05:15,168][20347] Updated weights for policy 0, policy_version 497977 (0.0005) [2023-12-27 09:05:15,180][20381] Updated weights for policy 1, policy_version 497678 (0.0009) [2023-12-27 09:05:15,736][20347] Updated weights for policy 0, policy_version 497987 (0.0006) [2023-12-27 09:05:15,794][20347] Updated weights for policy 0, policy_version 497997 (0.0007) [2023-12-27 09:05:15,843][20347] Updated weights for policy 0, policy_version 498007 (0.0006) [2023-12-27 09:05:15,850][20381] Updated weights for policy 1, policy_version 497688 (0.0006) [2023-12-27 09:05:15,900][20381] Updated weights for policy 1, policy_version 497698 (0.0006) [2023-12-27 09:05:15,955][20381] Updated weights for policy 1, policy_version 497708 (0.0007) [2023-12-27 09:05:16,523][20347] Updated weights for policy 0, policy_version 498017 (0.0007) [2023-12-27 09:05:16,585][20347] Updated weights for policy 0, policy_version 498027 (0.0010) [2023-12-27 09:05:16,638][20347] Updated weights for policy 0, policy_version 498037 (0.0011) [2023-12-27 09:05:16,659][20381] Updated weights for policy 1, policy_version 497718 (0.0006) [2023-12-27 09:05:16,696][20347] Updated weights for policy 0, policy_version 498047 (0.0008) [2023-12-27 09:05:16,719][20381] Updated weights for policy 1, policy_version 497728 (0.0008) [2023-12-27 09:05:16,775][20381] Updated weights for policy 1, policy_version 497738 (0.0007) [2023-12-27 09:05:17,309][20347] Updated weights for policy 0, policy_version 498057 (0.0007) [2023-12-27 09:05:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 254959616. Throughput: 0: 10417.0, 1: 10238.0. Samples: 254974988. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:17,363][19170] Avg episode reward: [(0, '799.050'), (1, '980.740')] [2023-12-27 09:05:17,363][20347] Updated weights for policy 0, policy_version 498067 (0.0007) [2023-12-27 09:05:17,418][20347] Updated weights for policy 0, policy_version 498077 (0.0007) [2023-12-27 09:05:17,496][20381] Updated weights for policy 1, policy_version 497748 (0.0008) [2023-12-27 09:05:17,557][20381] Updated weights for policy 1, policy_version 497758 (0.0009) [2023-12-27 09:05:17,612][20381] Updated weights for policy 1, policy_version 497768 (0.0009) [2023-12-27 09:05:18,049][20347] Updated weights for policy 0, policy_version 498087 (0.0007) [2023-12-27 09:05:18,093][20347] Updated weights for policy 0, policy_version 498097 (0.0006) [2023-12-27 09:05:18,141][20347] Updated weights for policy 0, policy_version 498107 (0.0005) [2023-12-27 09:05:18,329][20381] Updated weights for policy 1, policy_version 497778 (0.0007) [2023-12-27 09:05:18,394][20381] Updated weights for policy 1, policy_version 497788 (0.0008) [2023-12-27 09:05:18,453][20381] Updated weights for policy 1, policy_version 497798 (0.0008) [2023-12-27 09:05:18,513][20381] Updated weights for policy 1, policy_version 497808 (0.0008) [2023-12-27 09:05:18,809][20347] Updated weights for policy 0, policy_version 498117 (0.0006) [2023-12-27 09:05:18,868][20347] Updated weights for policy 0, policy_version 498127 (0.0007) [2023-12-27 09:05:18,922][20347] Updated weights for policy 0, policy_version 498137 (0.0007) [2023-12-27 09:05:19,251][20381] Updated weights for policy 1, policy_version 497818 (0.0008) [2023-12-27 09:05:19,305][20381] Updated weights for policy 1, policy_version 497828 (0.0008) [2023-12-27 09:05:19,371][20381] Updated weights for policy 1, policy_version 497838 (0.0008) [2023-12-27 09:05:19,564][20347] Updated weights for policy 0, policy_version 498147 (0.0007) [2023-12-27 09:05:19,622][20347] Updated weights for policy 0, policy_version 498157 (0.0007) [2023-12-27 09:05:19,683][20347] Updated weights for policy 0, policy_version 498167 (0.0008) [2023-12-27 09:05:20,053][20381] Updated weights for policy 1, policy_version 497848 (0.0007) [2023-12-27 09:05:20,072][20292] KL-divergence is very high: 132.1613 [2023-12-27 09:05:20,108][20381] Updated weights for policy 1, policy_version 497858 (0.0007) [2023-12-27 09:05:20,160][20381] Updated weights for policy 1, policy_version 497868 (0.0007) [2023-12-27 09:05:20,388][20347] Updated weights for policy 0, policy_version 498177 (0.0008) [2023-12-27 09:05:20,447][20347] Updated weights for policy 0, policy_version 498187 (0.0011) [2023-12-27 09:05:20,505][20347] Updated weights for policy 0, policy_version 498197 (0.0010) [2023-12-27 09:05:20,560][20347] Updated weights for policy 0, policy_version 498207 (0.0012) [2023-12-27 09:05:20,876][20381] Updated weights for policy 1, policy_version 497878 (0.0010) [2023-12-27 09:05:20,925][20381] Updated weights for policy 1, policy_version 497888 (0.0010) [2023-12-27 09:05:20,974][20381] Updated weights for policy 1, policy_version 497898 (0.0011) [2023-12-27 09:05:21,253][20347] Updated weights for policy 0, policy_version 498217 (0.0010) [2023-12-27 09:05:21,316][20347] Updated weights for policy 0, policy_version 498227 (0.0011) [2023-12-27 09:05:21,387][20347] Updated weights for policy 0, policy_version 498237 (0.0009) [2023-12-27 09:05:21,725][20381] Updated weights for policy 1, policy_version 497908 (0.0010) [2023-12-27 09:05:21,791][20381] Updated weights for policy 1, policy_version 497918 (0.0007) [2023-12-27 09:05:21,846][20381] Updated weights for policy 1, policy_version 497928 (0.0007) [2023-12-27 09:05:22,139][20347] Updated weights for policy 0, policy_version 498247 (0.0007) [2023-12-27 09:05:22,199][20347] Updated weights for policy 0, policy_version 498257 (0.0009) [2023-12-27 09:05:22,252][20347] Updated weights for policy 0, policy_version 498267 (0.0008) [2023-12-27 09:05:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 255066112. Throughput: 0: 10453.1, 1: 10199.0. Samples: 255035796. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:22,363][19170] Avg episode reward: [(0, '846.540'), (1, '961.520')] [2023-12-27 09:05:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000497936_127492096.pth... [2023-12-27 09:05:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000498272_127574016.pth... [2023-12-27 09:05:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000497024_127254528.pth [2023-12-27 09:05:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000496752_127188992.pth [2023-12-27 09:05:22,450][20381] Updated weights for policy 1, policy_version 497938 (0.0006) [2023-12-27 09:05:22,504][20381] Updated weights for policy 1, policy_version 497948 (0.0008) [2023-12-27 09:05:22,559][20381] Updated weights for policy 1, policy_version 497958 (0.0008) [2023-12-27 09:05:22,616][20381] Updated weights for policy 1, policy_version 497968 (0.0008) [2023-12-27 09:05:22,912][20347] Updated weights for policy 0, policy_version 498277 (0.0007) [2023-12-27 09:05:22,967][20347] Updated weights for policy 0, policy_version 498287 (0.0005) [2023-12-27 09:05:23,030][20347] Updated weights for policy 0, policy_version 498297 (0.0005) [2023-12-27 09:05:23,218][20381] Updated weights for policy 1, policy_version 497978 (0.0009) [2023-12-27 09:05:23,273][20381] Updated weights for policy 1, policy_version 497988 (0.0007) [2023-12-27 09:05:23,348][20381] Updated weights for policy 1, policy_version 497998 (0.0010) [2023-12-27 09:05:23,585][20347] Updated weights for policy 0, policy_version 498307 (0.0007) [2023-12-27 09:05:23,645][20347] Updated weights for policy 0, policy_version 498317 (0.0007) [2023-12-27 09:05:23,701][20347] Updated weights for policy 0, policy_version 498327 (0.0005) [2023-12-27 09:05:23,993][20381] Updated weights for policy 1, policy_version 498008 (0.0010) [2023-12-27 09:05:24,052][20381] Updated weights for policy 1, policy_version 498018 (0.0009) [2023-12-27 09:05:24,103][20381] Updated weights for policy 1, policy_version 498028 (0.0005) [2023-12-27 09:05:24,323][20347] Updated weights for policy 0, policy_version 498337 (0.0007) [2023-12-27 09:05:24,383][20347] Updated weights for policy 0, policy_version 498347 (0.0010) [2023-12-27 09:05:24,438][20347] Updated weights for policy 0, policy_version 498357 (0.0011) [2023-12-27 09:05:24,496][20347] Updated weights for policy 0, policy_version 498367 (0.0007) [2023-12-27 09:05:24,743][20381] Updated weights for policy 1, policy_version 498038 (0.0006) [2023-12-27 09:05:24,798][20381] Updated weights for policy 1, policy_version 498048 (0.0008) [2023-12-27 09:05:24,847][20381] Updated weights for policy 1, policy_version 498058 (0.0010) [2023-12-27 09:05:25,102][20347] Updated weights for policy 0, policy_version 498377 (0.0005) [2023-12-27 09:05:25,167][20347] Updated weights for policy 0, policy_version 498387 (0.0007) [2023-12-27 09:05:25,231][20347] Updated weights for policy 0, policy_version 498397 (0.0007) [2023-12-27 09:05:25,506][20381] Updated weights for policy 1, policy_version 498068 (0.0010) [2023-12-27 09:05:25,558][20381] Updated weights for policy 1, policy_version 498078 (0.0010) [2023-12-27 09:05:25,605][20292] KL-divergence is very high: 105.1255 [2023-12-27 09:05:25,611][20381] Updated weights for policy 1, policy_version 498088 (0.0008) [2023-12-27 09:05:25,858][20347] Updated weights for policy 0, policy_version 498407 (0.0009) [2023-12-27 09:05:25,924][20347] Updated weights for policy 0, policy_version 498417 (0.0008) [2023-12-27 09:05:25,991][20347] Updated weights for policy 0, policy_version 498427 (0.0008) [2023-12-27 09:05:26,179][20381] Updated weights for policy 1, policy_version 498098 (0.0006) [2023-12-27 09:05:26,249][20381] Updated weights for policy 1, policy_version 498108 (0.0007) [2023-12-27 09:05:26,309][20381] Updated weights for policy 1, policy_version 498118 (0.0010) [2023-12-27 09:05:26,359][20381] Updated weights for policy 1, policy_version 498128 (0.0010) [2023-12-27 09:05:26,580][20347] Updated weights for policy 0, policy_version 498437 (0.0007) [2023-12-27 09:05:26,634][20347] Updated weights for policy 0, policy_version 498447 (0.0007) [2023-12-27 09:05:26,686][20347] Updated weights for policy 0, policy_version 498457 (0.0007) [2023-12-27 09:05:26,943][20381] Updated weights for policy 1, policy_version 498138 (0.0008) [2023-12-27 09:05:26,987][20381] Updated weights for policy 1, policy_version 498148 (0.0007) [2023-12-27 09:05:27,031][20381] Updated weights for policy 1, policy_version 498158 (0.0005) [2023-12-27 09:05:27,354][20347] Updated weights for policy 0, policy_version 498467 (0.0007) [2023-12-27 09:05:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 255172608. Throughput: 0: 10472.8, 1: 10286.5. Samples: 255162236. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-12-27 09:05:27,363][19170] Avg episode reward: [(0, '923.950'), (1, '922.960')] [2023-12-27 09:05:27,402][20347] Updated weights for policy 0, policy_version 498477 (0.0007) [2023-12-27 09:05:27,450][20347] Updated weights for policy 0, policy_version 498487 (0.0005) [2023-12-27 09:05:27,720][20381] Updated weights for policy 1, policy_version 498168 (0.0009) [2023-12-27 09:05:27,778][20381] Updated weights for policy 1, policy_version 498178 (0.0010) [2023-12-27 09:05:27,839][20381] Updated weights for policy 1, policy_version 498188 (0.0010) [2023-12-27 09:05:28,013][20347] Updated weights for policy 0, policy_version 498497 (0.0006) [2023-12-27 09:05:28,061][20347] Updated weights for policy 0, policy_version 498507 (0.0008) [2023-12-27 09:05:28,106][20347] Updated weights for policy 0, policy_version 498517 (0.0007) [2023-12-27 09:05:28,156][20347] Updated weights for policy 0, policy_version 498527 (0.0007) [2023-12-27 09:05:28,533][20381] Updated weights for policy 1, policy_version 498198 (0.0010) [2023-12-27 09:05:28,593][20381] Updated weights for policy 1, policy_version 498208 (0.0010) [2023-12-27 09:05:28,651][20381] Updated weights for policy 1, policy_version 498218 (0.0011) [2023-12-27 09:05:28,835][20347] Updated weights for policy 0, policy_version 498537 (0.0010) [2023-12-27 09:05:28,883][20347] Updated weights for policy 0, policy_version 498547 (0.0010) [2023-12-27 09:05:28,928][20347] Updated weights for policy 0, policy_version 498557 (0.0010) [2023-12-27 09:05:29,310][20381] Updated weights for policy 1, policy_version 498228 (0.0010) [2023-12-27 09:05:29,378][20381] Updated weights for policy 1, policy_version 498238 (0.0008) [2023-12-27 09:05:29,440][20381] Updated weights for policy 1, policy_version 498248 (0.0007) [2023-12-27 09:05:29,596][20347] Updated weights for policy 0, policy_version 498567 (0.0007) [2023-12-27 09:05:29,655][20347] Updated weights for policy 0, policy_version 498577 (0.0005) [2023-12-27 09:05:29,718][20347] Updated weights for policy 0, policy_version 498587 (0.0005) [2023-12-27 09:05:30,117][20381] Updated weights for policy 1, policy_version 498258 (0.0007) [2023-12-27 09:05:30,171][20381] Updated weights for policy 1, policy_version 498268 (0.0008) [2023-12-27 09:05:30,220][20381] Updated weights for policy 1, policy_version 498278 (0.0010) [2023-12-27 09:05:30,271][20381] Updated weights for policy 1, policy_version 498288 (0.0010) [2023-12-27 09:05:30,298][20347] Updated weights for policy 0, policy_version 498597 (0.0006) [2023-12-27 09:05:30,360][20347] Updated weights for policy 0, policy_version 498607 (0.0008) [2023-12-27 09:05:30,413][20347] Updated weights for policy 0, policy_version 498617 (0.0007) [2023-12-27 09:05:30,886][20381] Updated weights for policy 1, policy_version 498298 (0.0005) [2023-12-27 09:05:30,931][20381] Updated weights for policy 1, policy_version 498308 (0.0005) [2023-12-27 09:05:30,986][20381] Updated weights for policy 1, policy_version 498318 (0.0005) [2023-12-27 09:05:31,075][20347] Updated weights for policy 0, policy_version 498627 (0.0007) [2023-12-27 09:05:31,137][20347] Updated weights for policy 0, policy_version 498637 (0.0007) [2023-12-27 09:05:31,197][20347] Updated weights for policy 0, policy_version 498647 (0.0008) [2023-12-27 09:05:31,618][20381] Updated weights for policy 1, policy_version 498328 (0.0007) [2023-12-27 09:05:31,680][20381] Updated weights for policy 1, policy_version 498338 (0.0008) [2023-12-27 09:05:31,727][20381] Updated weights for policy 1, policy_version 498348 (0.0007) [2023-12-27 09:05:31,904][20347] Updated weights for policy 0, policy_version 498657 (0.0009) [2023-12-27 09:05:31,960][20347] Updated weights for policy 0, policy_version 498667 (0.0007) [2023-12-27 09:05:32,018][20347] Updated weights for policy 0, policy_version 498677 (0.0010) [2023-12-27 09:05:32,076][20347] Updated weights for policy 0, policy_version 498687 (0.0010) [2023-12-27 09:05:32,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 255279104. Throughput: 0: 10599.8, 1: 10343.8. Samples: 255290144. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:32,363][19170] Avg episode reward: [(0, '894.830'), (1, '874.270')] [2023-12-27 09:05:32,428][20381] Updated weights for policy 1, policy_version 498358 (0.0009) [2023-12-27 09:05:32,487][20381] Updated weights for policy 1, policy_version 498368 (0.0008) [2023-12-27 09:05:32,544][20381] Updated weights for policy 1, policy_version 498378 (0.0007) [2023-12-27 09:05:32,716][20347] Updated weights for policy 0, policy_version 498697 (0.0010) [2023-12-27 09:05:32,775][20347] Updated weights for policy 0, policy_version 498707 (0.0010) [2023-12-27 09:05:32,830][20347] Updated weights for policy 0, policy_version 498717 (0.0010) [2023-12-27 09:05:33,231][20381] Updated weights for policy 1, policy_version 498388 (0.0007) [2023-12-27 09:05:33,286][20381] Updated weights for policy 1, policy_version 498398 (0.0007) [2023-12-27 09:05:33,334][20381] Updated weights for policy 1, policy_version 498408 (0.0007) [2023-12-27 09:05:33,476][20347] Updated weights for policy 0, policy_version 498727 (0.0011) [2023-12-27 09:05:33,527][20347] Updated weights for policy 0, policy_version 498737 (0.0010) [2023-12-27 09:05:33,585][20347] Updated weights for policy 0, policy_version 498747 (0.0010) [2023-12-27 09:05:34,066][20381] Updated weights for policy 1, policy_version 498418 (0.0007) [2023-12-27 09:05:34,133][20381] Updated weights for policy 1, policy_version 498428 (0.0007) [2023-12-27 09:05:34,191][20381] Updated weights for policy 1, policy_version 498438 (0.0008) [2023-12-27 09:05:34,192][20347] Updated weights for policy 0, policy_version 498757 (0.0008) [2023-12-27 09:05:34,242][20347] Updated weights for policy 0, policy_version 498767 (0.0006) [2023-12-27 09:05:34,248][20381] Updated weights for policy 1, policy_version 498448 (0.0006) [2023-12-27 09:05:34,303][20347] Updated weights for policy 0, policy_version 498777 (0.0008) [2023-12-27 09:05:34,913][20381] Updated weights for policy 1, policy_version 498458 (0.0005) [2023-12-27 09:05:34,972][20381] Updated weights for policy 1, policy_version 498468 (0.0005) [2023-12-27 09:05:34,973][20347] Updated weights for policy 0, policy_version 498787 (0.0008) [2023-12-27 09:05:35,030][20347] Updated weights for policy 0, policy_version 498797 (0.0007) [2023-12-27 09:05:35,034][20381] Updated weights for policy 1, policy_version 498478 (0.0006) [2023-12-27 09:05:35,078][20347] Updated weights for policy 0, policy_version 498807 (0.0007) [2023-12-27 09:05:35,567][20381] Updated weights for policy 1, policy_version 498488 (0.0009) [2023-12-27 09:05:35,613][20381] Updated weights for policy 1, policy_version 498498 (0.0009) [2023-12-27 09:05:35,630][20347] Updated weights for policy 0, policy_version 498817 (0.0005) [2023-12-27 09:05:35,673][20381] Updated weights for policy 1, policy_version 498508 (0.0007) [2023-12-27 09:05:35,683][20347] Updated weights for policy 0, policy_version 498827 (0.0005) [2023-12-27 09:05:35,743][20347] Updated weights for policy 0, policy_version 498837 (0.0005) [2023-12-27 09:05:35,802][20347] Updated weights for policy 0, policy_version 498847 (0.0005) [2023-12-27 09:05:36,336][20347] Updated weights for policy 0, policy_version 498857 (0.0007) [2023-12-27 09:05:36,368][20381] Updated weights for policy 1, policy_version 498518 (0.0007) [2023-12-27 09:05:36,390][20347] Updated weights for policy 0, policy_version 498867 (0.0006) [2023-12-27 09:05:36,431][20381] Updated weights for policy 1, policy_version 498528 (0.0008) [2023-12-27 09:05:36,445][20347] Updated weights for policy 0, policy_version 498877 (0.0007) [2023-12-27 09:05:36,493][20381] Updated weights for policy 1, policy_version 498538 (0.0007) [2023-12-27 09:05:37,138][20347] Updated weights for policy 0, policy_version 498887 (0.0009) [2023-12-27 09:05:37,177][20381] Updated weights for policy 1, policy_version 498548 (0.0007) [2023-12-27 09:05:37,195][20347] Updated weights for policy 0, policy_version 498897 (0.0010) [2023-12-27 09:05:37,234][20381] Updated weights for policy 1, policy_version 498558 (0.0005) [2023-12-27 09:05:37,255][20347] Updated weights for policy 0, policy_version 498907 (0.0010) [2023-12-27 09:05:37,287][20381] Updated weights for policy 1, policy_version 498568 (0.0006) [2023-12-27 09:05:37,362][19170] Fps is (10 sec: 22118.2, 60 sec: 21026.1, 300 sec: 20938.2). Total num frames: 255393792. Throughput: 0: 10597.3, 1: 10373.1. Samples: 255352160. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:37,363][19170] Avg episode reward: [(0, '855.290'), (1, '805.660')] [2023-12-27 09:05:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000498576_127655936.pth... [2023-12-27 09:05:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000498912_127737856.pth... [2023-12-27 09:05:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000497360_127344640.pth [2023-12-27 09:05:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000497664_127418368.pth [2023-12-27 09:05:37,945][20347] Updated weights for policy 0, policy_version 498917 (0.0010) [2023-12-27 09:05:37,989][20381] Updated weights for policy 1, policy_version 498578 (0.0007) [2023-12-27 09:05:38,007][20347] Updated weights for policy 0, policy_version 498927 (0.0007) [2023-12-27 09:05:38,062][20381] Updated weights for policy 1, policy_version 498588 (0.0006) [2023-12-27 09:05:38,068][20347] Updated weights for policy 0, policy_version 498937 (0.0007) [2023-12-27 09:05:38,112][20381] Updated weights for policy 1, policy_version 498598 (0.0006) [2023-12-27 09:05:38,176][20381] Updated weights for policy 1, policy_version 498608 (0.0007) [2023-12-27 09:05:38,744][20347] Updated weights for policy 0, policy_version 498947 (0.0008) [2023-12-27 09:05:38,748][20381] Updated weights for policy 1, policy_version 498618 (0.0005) [2023-12-27 09:05:38,802][20347] Updated weights for policy 0, policy_version 498957 (0.0008) [2023-12-27 09:05:38,806][20381] Updated weights for policy 1, policy_version 498628 (0.0006) [2023-12-27 09:05:38,859][20347] Updated weights for policy 0, policy_version 498967 (0.0007) [2023-12-27 09:05:38,865][20381] Updated weights for policy 1, policy_version 498638 (0.0006) [2023-12-27 09:05:39,569][20381] Updated weights for policy 1, policy_version 498648 (0.0006) [2023-12-27 09:05:39,620][20347] Updated weights for policy 0, policy_version 498977 (0.0008) [2023-12-27 09:05:39,633][20381] Updated weights for policy 1, policy_version 498658 (0.0007) [2023-12-27 09:05:39,681][20347] Updated weights for policy 0, policy_version 498987 (0.0008) [2023-12-27 09:05:39,694][20381] Updated weights for policy 1, policy_version 498668 (0.0006) [2023-12-27 09:05:39,737][20347] Updated weights for policy 0, policy_version 498997 (0.0007) [2023-12-27 09:05:39,786][20347] Updated weights for policy 0, policy_version 499007 (0.0007) [2023-12-27 09:05:40,375][20381] Updated weights for policy 1, policy_version 498678 (0.0007) [2023-12-27 09:05:40,429][20381] Updated weights for policy 1, policy_version 498688 (0.0008) [2023-12-27 09:05:40,484][20381] Updated weights for policy 1, policy_version 498698 (0.0008) [2023-12-27 09:05:40,538][20347] Updated weights for policy 0, policy_version 499017 (0.0007) [2023-12-27 09:05:40,589][20347] Updated weights for policy 0, policy_version 499027 (0.0007) [2023-12-27 09:05:40,639][20347] Updated weights for policy 0, policy_version 499037 (0.0007) [2023-12-27 09:05:41,199][20381] Updated weights for policy 1, policy_version 498708 (0.0006) [2023-12-27 09:05:41,264][20381] Updated weights for policy 1, policy_version 498718 (0.0007) [2023-12-27 09:05:41,305][20347] Updated weights for policy 0, policy_version 499047 (0.0006) [2023-12-27 09:05:41,322][20381] Updated weights for policy 1, policy_version 498728 (0.0006) [2023-12-27 09:05:41,371][20347] Updated weights for policy 0, policy_version 499057 (0.0008) [2023-12-27 09:05:41,430][20347] Updated weights for policy 0, policy_version 499067 (0.0007) [2023-12-27 09:05:42,066][20381] Updated weights for policy 1, policy_version 498738 (0.0008) [2023-12-27 09:05:42,100][20292] KL-divergence is very high: 208.3989 [2023-12-27 09:05:42,118][20381] Updated weights for policy 1, policy_version 498748 (0.0008) [2023-12-27 09:05:42,118][20347] Updated weights for policy 0, policy_version 499077 (0.0008) [2023-12-27 09:05:42,172][20381] Updated weights for policy 1, policy_version 498758 (0.0008) [2023-12-27 09:05:42,178][20347] Updated weights for policy 0, policy_version 499087 (0.0006) [2023-12-27 09:05:42,191][20292] KL-divergence is very high: 113.8522 [2023-12-27 09:05:42,225][20347] Updated weights for policy 0, policy_version 499097 (0.0008) [2023-12-27 09:05:42,230][20381] Updated weights for policy 1, policy_version 498768 (0.0008) [2023-12-27 09:05:42,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 255492096. Throughput: 0: 10584.3, 1: 10385.7. Samples: 255477288. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:42,363][19170] Avg episode reward: [(0, '883.700'), (1, '674.530')] [2023-12-27 09:05:42,886][20292] KL-divergence is very high: 103.7383 [2023-12-27 09:05:42,902][20347] Updated weights for policy 0, policy_version 499107 (0.0006) [2023-12-27 09:05:42,930][20381] Updated weights for policy 1, policy_version 498778 (0.0007) [2023-12-27 09:05:42,967][20347] Updated weights for policy 0, policy_version 499117 (0.0005) [2023-12-27 09:05:42,977][20381] Updated weights for policy 1, policy_version 498788 (0.0007) [2023-12-27 09:05:43,026][20381] Updated weights for policy 1, policy_version 498798 (0.0007) [2023-12-27 09:05:43,031][20347] Updated weights for policy 0, policy_version 499127 (0.0007) [2023-12-27 09:05:43,690][20347] Updated weights for policy 0, policy_version 499137 (0.0007) [2023-12-27 09:05:43,721][20381] Updated weights for policy 1, policy_version 498808 (0.0007) [2023-12-27 09:05:43,742][20347] Updated weights for policy 0, policy_version 499147 (0.0008) [2023-12-27 09:05:43,780][20381] Updated weights for policy 1, policy_version 498818 (0.0005) [2023-12-27 09:05:43,787][20347] Updated weights for policy 0, policy_version 499157 (0.0007) [2023-12-27 09:05:43,837][20347] Updated weights for policy 0, policy_version 499167 (0.0006) [2023-12-27 09:05:43,839][20381] Updated weights for policy 1, policy_version 498828 (0.0005) [2023-12-27 09:05:44,443][20381] Updated weights for policy 1, policy_version 498838 (0.0006) [2023-12-27 09:05:44,471][20347] Updated weights for policy 0, policy_version 499177 (0.0006) [2023-12-27 09:05:44,497][20381] Updated weights for policy 1, policy_version 498848 (0.0008) [2023-12-27 09:05:44,526][20347] Updated weights for policy 0, policy_version 499187 (0.0006) [2023-12-27 09:05:44,553][20381] Updated weights for policy 1, policy_version 498858 (0.0006) [2023-12-27 09:05:44,588][20347] Updated weights for policy 0, policy_version 499197 (0.0007) [2023-12-27 09:05:45,170][20381] Updated weights for policy 1, policy_version 498868 (0.0007) [2023-12-27 09:05:45,231][20381] Updated weights for policy 1, policy_version 498878 (0.0007) [2023-12-27 09:05:45,286][20381] Updated weights for policy 1, policy_version 498888 (0.0008) [2023-12-27 09:05:45,343][20347] Updated weights for policy 0, policy_version 499207 (0.0006) [2023-12-27 09:05:45,395][20347] Updated weights for policy 0, policy_version 499217 (0.0007) [2023-12-27 09:05:45,457][20347] Updated weights for policy 0, policy_version 499227 (0.0005) [2023-12-27 09:05:45,870][20381] Updated weights for policy 1, policy_version 498898 (0.0008) [2023-12-27 09:05:45,924][20381] Updated weights for policy 1, policy_version 498908 (0.0007) [2023-12-27 09:05:45,980][20381] Updated weights for policy 1, policy_version 498918 (0.0008) [2023-12-27 09:05:46,039][20381] Updated weights for policy 1, policy_version 498928 (0.0008) [2023-12-27 09:05:46,127][20347] Updated weights for policy 0, policy_version 499237 (0.0006) [2023-12-27 09:05:46,188][20347] Updated weights for policy 0, policy_version 499247 (0.0008) [2023-12-27 09:05:46,246][20347] Updated weights for policy 0, policy_version 499257 (0.0007) [2023-12-27 09:05:46,686][20381] Updated weights for policy 1, policy_version 498938 (0.0008) [2023-12-27 09:05:46,749][20381] Updated weights for policy 1, policy_version 498948 (0.0008) [2023-12-27 09:05:46,802][20381] Updated weights for policy 1, policy_version 498958 (0.0008) [2023-12-27 09:05:46,951][20347] Updated weights for policy 0, policy_version 499267 (0.0008) [2023-12-27 09:05:47,010][20347] Updated weights for policy 0, policy_version 499277 (0.0008) [2023-12-27 09:05:47,073][20347] Updated weights for policy 0, policy_version 499287 (0.0008) [2023-12-27 09:05:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 255590400. Throughput: 0: 10515.2, 1: 10400.1. Samples: 255600428. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:47,363][19170] Avg episode reward: [(0, '903.320'), (1, '449.380')] [2023-12-27 09:05:47,506][20381] Updated weights for policy 1, policy_version 498968 (0.0008) [2023-12-27 09:05:47,558][20381] Updated weights for policy 1, policy_version 498978 (0.0008) [2023-12-27 09:05:47,620][20381] Updated weights for policy 1, policy_version 498988 (0.0008) [2023-12-27 09:05:47,740][20347] Updated weights for policy 0, policy_version 499297 (0.0008) [2023-12-27 09:05:47,792][20347] Updated weights for policy 0, policy_version 499307 (0.0008) [2023-12-27 09:05:47,837][20347] Updated weights for policy 0, policy_version 499317 (0.0007) [2023-12-27 09:05:47,881][20347] Updated weights for policy 0, policy_version 499327 (0.0008) [2023-12-27 09:05:48,257][20381] Updated weights for policy 1, policy_version 498998 (0.0008) [2023-12-27 09:05:48,312][20381] Updated weights for policy 1, policy_version 499008 (0.0007) [2023-12-27 09:05:48,375][20381] Updated weights for policy 1, policy_version 499018 (0.0007) [2023-12-27 09:05:48,634][20347] Updated weights for policy 0, policy_version 499337 (0.0008) [2023-12-27 09:05:48,693][20347] Updated weights for policy 0, policy_version 499347 (0.0008) [2023-12-27 09:05:48,761][20347] Updated weights for policy 0, policy_version 499357 (0.0007) [2023-12-27 09:05:49,017][20381] Updated weights for policy 1, policy_version 499028 (0.0008) [2023-12-27 09:05:49,078][20381] Updated weights for policy 1, policy_version 499038 (0.0007) [2023-12-27 09:05:49,139][20381] Updated weights for policy 1, policy_version 499048 (0.0007) [2023-12-27 09:05:49,448][20347] Updated weights for policy 0, policy_version 499367 (0.0010) [2023-12-27 09:05:49,506][20347] Updated weights for policy 0, policy_version 499377 (0.0010) [2023-12-27 09:05:49,568][20347] Updated weights for policy 0, policy_version 499387 (0.0010) [2023-12-27 09:05:49,813][20381] Updated weights for policy 1, policy_version 499058 (0.0007) [2023-12-27 09:05:49,872][20381] Updated weights for policy 1, policy_version 499068 (0.0008) [2023-12-27 09:05:49,934][20381] Updated weights for policy 1, policy_version 499078 (0.0008) [2023-12-27 09:05:49,988][20381] Updated weights for policy 1, policy_version 499088 (0.0008) [2023-12-27 09:05:50,248][20347] Updated weights for policy 0, policy_version 499397 (0.0010) [2023-12-27 09:05:50,308][20347] Updated weights for policy 0, policy_version 499407 (0.0008) [2023-12-27 09:05:50,370][20347] Updated weights for policy 0, policy_version 499417 (0.0010) [2023-12-27 09:05:50,700][20381] Updated weights for policy 1, policy_version 499098 (0.0008) [2023-12-27 09:05:50,754][20381] Updated weights for policy 1, policy_version 499108 (0.0007) [2023-12-27 09:05:50,811][20381] Updated weights for policy 1, policy_version 499118 (0.0007) [2023-12-27 09:05:51,051][20347] Updated weights for policy 0, policy_version 499427 (0.0010) [2023-12-27 09:05:51,103][20347] Updated weights for policy 0, policy_version 499437 (0.0007) [2023-12-27 09:05:51,156][20347] Updated weights for policy 0, policy_version 499447 (0.0007) [2023-12-27 09:05:51,543][20381] Updated weights for policy 1, policy_version 499128 (0.0008) [2023-12-27 09:05:51,604][20381] Updated weights for policy 1, policy_version 499138 (0.0007) [2023-12-27 09:05:51,665][20381] Updated weights for policy 1, policy_version 499148 (0.0008) [2023-12-27 09:05:51,817][20347] Updated weights for policy 0, policy_version 499457 (0.0008) [2023-12-27 09:05:51,873][20347] Updated weights for policy 0, policy_version 499467 (0.0005) [2023-12-27 09:05:51,932][20347] Updated weights for policy 0, policy_version 499477 (0.0007) [2023-12-27 09:05:51,985][20347] Updated weights for policy 0, policy_version 499487 (0.0007) [2023-12-27 09:05:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 255688704. Throughput: 0: 10475.1, 1: 10364.8. Samples: 255661156. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:52,363][19170] Avg episode reward: [(0, '923.050'), (1, '593.970')] [2023-12-27 09:05:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000499488_127885312.pth... [2023-12-27 09:05:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000498272_127574016.pth [2023-12-27 09:05:52,395][20381] Updated weights for policy 1, policy_version 499158 (0.0008) [2023-12-27 09:05:52,459][20381] Updated weights for policy 1, policy_version 499168 (0.0009) [2023-12-27 09:05:52,514][20381] Updated weights for policy 1, policy_version 499178 (0.0007) [2023-12-27 09:05:52,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000499184_127811584.pth... [2023-12-27 09:05:52,550][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000497936_127492096.pth [2023-12-27 09:05:52,633][20347] Updated weights for policy 0, policy_version 499497 (0.0006) [2023-12-27 09:05:52,694][20347] Updated weights for policy 0, policy_version 499507 (0.0008) [2023-12-27 09:05:52,755][20347] Updated weights for policy 0, policy_version 499517 (0.0008) [2023-12-27 09:05:53,155][20381] Updated weights for policy 1, policy_version 499188 (0.0007) [2023-12-27 09:05:53,207][20381] Updated weights for policy 1, policy_version 499198 (0.0008) [2023-12-27 09:05:53,256][20381] Updated weights for policy 1, policy_version 499208 (0.0008) [2023-12-27 09:05:53,459][20347] Updated weights for policy 0, policy_version 499527 (0.0007) [2023-12-27 09:05:53,519][20347] Updated weights for policy 0, policy_version 499537 (0.0008) [2023-12-27 09:05:53,574][20347] Updated weights for policy 0, policy_version 499547 (0.0007) [2023-12-27 09:05:53,949][20381] Updated weights for policy 1, policy_version 499218 (0.0007) [2023-12-27 09:05:54,014][20381] Updated weights for policy 1, policy_version 499228 (0.0008) [2023-12-27 09:05:54,069][20381] Updated weights for policy 1, policy_version 499238 (0.0008) [2023-12-27 09:05:54,130][20381] Updated weights for policy 1, policy_version 499248 (0.0008) [2023-12-27 09:05:54,280][20347] Updated weights for policy 0, policy_version 499557 (0.0007) [2023-12-27 09:05:54,339][20347] Updated weights for policy 0, policy_version 499567 (0.0008) [2023-12-27 09:05:54,391][20347] Updated weights for policy 0, policy_version 499577 (0.0010) [2023-12-27 09:05:54,760][20381] Updated weights for policy 1, policy_version 499258 (0.0007) [2023-12-27 09:05:54,818][20381] Updated weights for policy 1, policy_version 499268 (0.0007) [2023-12-27 09:05:54,873][20381] Updated weights for policy 1, policy_version 499278 (0.0008) [2023-12-27 09:05:55,056][20347] Updated weights for policy 0, policy_version 499587 (0.0008) [2023-12-27 09:05:55,117][20347] Updated weights for policy 0, policy_version 499597 (0.0008) [2023-12-27 09:05:55,173][20347] Updated weights for policy 0, policy_version 499607 (0.0010) [2023-12-27 09:05:55,556][20381] Updated weights for policy 1, policy_version 499288 (0.0007) [2023-12-27 09:05:55,590][20292] KL-divergence is very high: 116.0687 [2023-12-27 09:05:55,607][20292] KL-divergence is very high: 142.6308 [2023-12-27 09:05:55,613][20381] Updated weights for policy 1, policy_version 499298 (0.0007) [2023-12-27 09:05:55,613][20292] KL-divergence is very high: 154.1668 [2023-12-27 09:05:55,671][20381] Updated weights for policy 1, policy_version 499308 (0.0007) [2023-12-27 09:05:55,869][20347] Updated weights for policy 0, policy_version 499617 (0.0010) [2023-12-27 09:05:55,926][20347] Updated weights for policy 0, policy_version 499627 (0.0010) [2023-12-27 09:05:55,978][20347] Updated weights for policy 0, policy_version 499637 (0.0010) [2023-12-27 09:05:56,037][20347] Updated weights for policy 0, policy_version 499647 (0.0010) [2023-12-27 09:05:56,309][20381] Updated weights for policy 1, policy_version 499318 (0.0008) [2023-12-27 09:05:56,363][20381] Updated weights for policy 1, policy_version 499328 (0.0007) [2023-12-27 09:05:56,411][20381] Updated weights for policy 1, policy_version 499338 (0.0007) [2023-12-27 09:05:56,702][20347] Updated weights for policy 0, policy_version 499657 (0.0010) [2023-12-27 09:05:56,747][20347] Updated weights for policy 0, policy_version 499667 (0.0006) [2023-12-27 09:05:56,795][20347] Updated weights for policy 0, policy_version 499677 (0.0005) [2023-12-27 09:05:57,103][20381] Updated weights for policy 1, policy_version 499348 (0.0007) [2023-12-27 09:05:57,150][20381] Updated weights for policy 1, policy_version 499358 (0.0007) [2023-12-27 09:05:57,200][20381] Updated weights for policy 1, policy_version 499368 (0.0007) [2023-12-27 09:05:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 255795200. Throughput: 0: 10384.7, 1: 10345.8. Samples: 255782936. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:05:57,363][19170] Avg episode reward: [(0, '952.040'), (1, '269.370')] [2023-12-27 09:05:57,443][20347] Updated weights for policy 0, policy_version 499687 (0.0008) [2023-12-27 09:05:57,497][20347] Updated weights for policy 0, policy_version 499697 (0.0008) [2023-12-27 09:05:57,550][20347] Updated weights for policy 0, policy_version 499707 (0.0005) [2023-12-27 09:05:57,909][20381] Updated weights for policy 1, policy_version 499378 (0.0008) [2023-12-27 09:05:57,959][20381] Updated weights for policy 1, policy_version 499388 (0.0007) [2023-12-27 09:05:58,012][20381] Updated weights for policy 1, policy_version 499398 (0.0007) [2023-12-27 09:05:58,072][20381] Updated weights for policy 1, policy_version 499408 (0.0007) [2023-12-27 09:05:58,192][20347] Updated weights for policy 0, policy_version 499717 (0.0008) [2023-12-27 09:05:58,252][20347] Updated weights for policy 0, policy_version 499727 (0.0008) [2023-12-27 09:05:58,317][20347] Updated weights for policy 0, policy_version 499737 (0.0008) [2023-12-27 09:05:58,856][20381] Updated weights for policy 1, policy_version 499418 (0.0006) [2023-12-27 09:05:58,918][20381] Updated weights for policy 1, policy_version 499428 (0.0007) [2023-12-27 09:05:58,976][20381] Updated weights for policy 1, policy_version 499438 (0.0007) [2023-12-27 09:05:59,029][20347] Updated weights for policy 0, policy_version 499747 (0.0008) [2023-12-27 09:05:59,080][20347] Updated weights for policy 0, policy_version 499757 (0.0010) [2023-12-27 09:05:59,129][20347] Updated weights for policy 0, policy_version 499767 (0.0009) [2023-12-27 09:05:59,595][20381] Updated weights for policy 1, policy_version 499448 (0.0010) [2023-12-27 09:05:59,641][20381] Updated weights for policy 1, policy_version 499458 (0.0010) [2023-12-27 09:05:59,686][20381] Updated weights for policy 1, policy_version 499468 (0.0010) [2023-12-27 09:05:59,732][20347] Updated weights for policy 0, policy_version 499777 (0.0006) [2023-12-27 09:05:59,790][20347] Updated weights for policy 0, policy_version 499787 (0.0010) [2023-12-27 09:05:59,845][20347] Updated weights for policy 0, policy_version 499797 (0.0009) [2023-12-27 09:05:59,904][20347] Updated weights for policy 0, policy_version 499807 (0.0006) [2023-12-27 09:06:00,409][20381] Updated weights for policy 1, policy_version 499478 (0.0011) [2023-12-27 09:06:00,470][20381] Updated weights for policy 1, policy_version 499488 (0.0010) [2023-12-27 09:06:00,521][20381] Updated weights for policy 1, policy_version 499498 (0.0010) [2023-12-27 09:06:00,553][20347] Updated weights for policy 0, policy_version 499817 (0.0010) [2023-12-27 09:06:00,609][20347] Updated weights for policy 0, policy_version 499827 (0.0011) [2023-12-27 09:06:00,658][20347] Updated weights for policy 0, policy_version 499837 (0.0010) [2023-12-27 09:06:01,148][20381] Updated weights for policy 1, policy_version 499508 (0.0009) [2023-12-27 09:06:01,205][20381] Updated weights for policy 1, policy_version 499518 (0.0008) [2023-12-27 09:06:01,266][20381] Updated weights for policy 1, policy_version 499528 (0.0007) [2023-12-27 09:06:01,313][20347] Updated weights for policy 0, policy_version 499847 (0.0008) [2023-12-27 09:06:01,374][20347] Updated weights for policy 0, policy_version 499857 (0.0009) [2023-12-27 09:06:01,420][20347] Updated weights for policy 0, policy_version 499867 (0.0010) [2023-12-27 09:06:01,967][20381] Updated weights for policy 1, policy_version 499538 (0.0006) [2023-12-27 09:06:02,020][20381] Updated weights for policy 1, policy_version 499548 (0.0005) [2023-12-27 09:06:02,071][20381] Updated weights for policy 1, policy_version 499558 (0.0005) [2023-12-27 09:06:02,122][20381] Updated weights for policy 1, policy_version 499568 (0.0006) [2023-12-27 09:06:02,132][20347] Updated weights for policy 0, policy_version 499877 (0.0010) [2023-12-27 09:06:02,184][20347] Updated weights for policy 0, policy_version 499887 (0.0011) [2023-12-27 09:06:02,233][20347] Updated weights for policy 0, policy_version 499897 (0.0010) [2023-12-27 09:06:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.7, 300 sec: 20882.7). Total num frames: 255901696. Throughput: 0: 10387.8, 1: 10317.9. Samples: 255906744. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:02,363][19170] Avg episode reward: [(0, '981.650'), (1, '314.840')] [2023-12-27 09:06:02,746][20381] Updated weights for policy 1, policy_version 499578 (0.0007) [2023-12-27 09:06:02,794][20381] Updated weights for policy 1, policy_version 499588 (0.0007) [2023-12-27 09:06:02,842][20381] Updated weights for policy 1, policy_version 499598 (0.0006) [2023-12-27 09:06:02,948][20347] Updated weights for policy 0, policy_version 499907 (0.0010) [2023-12-27 09:06:02,996][20347] Updated weights for policy 0, policy_version 499917 (0.0008) [2023-12-27 09:06:03,047][20347] Updated weights for policy 0, policy_version 499927 (0.0008) [2023-12-27 09:06:03,455][20381] Updated weights for policy 1, policy_version 499608 (0.0005) [2023-12-27 09:06:03,510][20381] Updated weights for policy 1, policy_version 499618 (0.0005) [2023-12-27 09:06:03,572][20381] Updated weights for policy 1, policy_version 499628 (0.0006) [2023-12-27 09:06:03,742][20347] Updated weights for policy 0, policy_version 499937 (0.0008) [2023-12-27 09:06:03,793][20347] Updated weights for policy 0, policy_version 499947 (0.0008) [2023-12-27 09:06:03,852][20347] Updated weights for policy 0, policy_version 499957 (0.0006) [2023-12-27 09:06:03,907][20347] Updated weights for policy 0, policy_version 499967 (0.0007) [2023-12-27 09:06:04,164][20381] Updated weights for policy 1, policy_version 499638 (0.0009) [2023-12-27 09:06:04,228][20381] Updated weights for policy 1, policy_version 499648 (0.0007) [2023-12-27 09:06:04,294][20381] Updated weights for policy 1, policy_version 499658 (0.0007) [2023-12-27 09:06:04,631][20347] Updated weights for policy 0, policy_version 499977 (0.0008) [2023-12-27 09:06:04,678][20347] Updated weights for policy 0, policy_version 499987 (0.0008) [2023-12-27 09:06:04,730][20347] Updated weights for policy 0, policy_version 499997 (0.0007) [2023-12-27 09:06:05,002][20381] Updated weights for policy 1, policy_version 499668 (0.0008) [2023-12-27 09:06:05,052][20381] Updated weights for policy 1, policy_version 499678 (0.0008) [2023-12-27 09:06:05,103][20381] Updated weights for policy 1, policy_version 499688 (0.0007) [2023-12-27 09:06:05,428][20347] Updated weights for policy 0, policy_version 500007 (0.0007) [2023-12-27 09:06:05,479][20347] Updated weights for policy 0, policy_version 500017 (0.0007) [2023-12-27 09:06:05,530][20347] Updated weights for policy 0, policy_version 500027 (0.0008) [2023-12-27 09:06:05,818][20381] Updated weights for policy 1, policy_version 499698 (0.0008) [2023-12-27 09:06:05,877][20381] Updated weights for policy 1, policy_version 499708 (0.0007) [2023-12-27 09:06:05,936][20381] Updated weights for policy 1, policy_version 499718 (0.0008) [2023-12-27 09:06:05,993][20381] Updated weights for policy 1, policy_version 499728 (0.0008) [2023-12-27 09:06:06,198][20347] Updated weights for policy 0, policy_version 500037 (0.0008) [2023-12-27 09:06:06,255][20347] Updated weights for policy 0, policy_version 500047 (0.0007) [2023-12-27 09:06:06,311][20347] Updated weights for policy 0, policy_version 500057 (0.0007) [2023-12-27 09:06:06,702][20381] Updated weights for policy 1, policy_version 499738 (0.0008) [2023-12-27 09:06:06,762][20381] Updated weights for policy 1, policy_version 499748 (0.0009) [2023-12-27 09:06:06,821][20381] Updated weights for policy 1, policy_version 499758 (0.0008) [2023-12-27 09:06:06,994][20347] Updated weights for policy 0, policy_version 500067 (0.0007) [2023-12-27 09:06:07,045][20347] Updated weights for policy 0, policy_version 500077 (0.0005) [2023-12-27 09:06:07,091][20347] Updated weights for policy 0, policy_version 500087 (0.0005) [2023-12-27 09:06:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.2, 300 sec: 20882.7). Total num frames: 256000000. Throughput: 0: 10347.2, 1: 10375.2. Samples: 255968304. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:07,363][19170] Avg episode reward: [(0, '943.630'), (1, '555.140')] [2023-12-27 09:06:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000500096_128040960.pth... [2023-12-27 09:06:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000499760_127959040.pth... [2023-12-27 09:06:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000498576_127655936.pth [2023-12-27 09:06:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000498912_127737856.pth [2023-12-27 09:06:07,524][20381] Updated weights for policy 1, policy_version 499768 (0.0008) [2023-12-27 09:06:07,572][20381] Updated weights for policy 1, policy_version 499778 (0.0008) [2023-12-27 09:06:07,624][20381] Updated weights for policy 1, policy_version 499788 (0.0007) [2023-12-27 09:06:07,710][20347] Updated weights for policy 0, policy_version 500097 (0.0007) [2023-12-27 09:06:07,762][20347] Updated weights for policy 0, policy_version 500107 (0.0007) [2023-12-27 09:06:07,821][20347] Updated weights for policy 0, policy_version 500117 (0.0006) [2023-12-27 09:06:07,875][20347] Updated weights for policy 0, policy_version 500127 (0.0005) [2023-12-27 09:06:08,259][20381] Updated weights for policy 1, policy_version 499798 (0.0008) [2023-12-27 09:06:08,315][20381] Updated weights for policy 1, policy_version 499808 (0.0008) [2023-12-27 09:06:08,379][20381] Updated weights for policy 1, policy_version 499818 (0.0007) [2023-12-27 09:06:08,549][20347] Updated weights for policy 0, policy_version 500137 (0.0006) [2023-12-27 09:06:08,601][20347] Updated weights for policy 0, policy_version 500147 (0.0007) [2023-12-27 09:06:08,663][20347] Updated weights for policy 0, policy_version 500157 (0.0007) [2023-12-27 09:06:09,082][20381] Updated weights for policy 1, policy_version 499828 (0.0007) [2023-12-27 09:06:09,134][20381] Updated weights for policy 1, policy_version 499838 (0.0008) [2023-12-27 09:06:09,183][20381] Updated weights for policy 1, policy_version 499848 (0.0008) [2023-12-27 09:06:09,347][20347] Updated weights for policy 0, policy_version 500167 (0.0008) [2023-12-27 09:06:09,407][20347] Updated weights for policy 0, policy_version 500177 (0.0007) [2023-12-27 09:06:09,470][20347] Updated weights for policy 0, policy_version 500187 (0.0008) [2023-12-27 09:06:09,928][20381] Updated weights for policy 1, policy_version 499858 (0.0008) [2023-12-27 09:06:09,988][20381] Updated weights for policy 1, policy_version 499868 (0.0008) [2023-12-27 09:06:10,047][20381] Updated weights for policy 1, policy_version 499878 (0.0008) [2023-12-27 09:06:10,106][20381] Updated weights for policy 1, policy_version 499888 (0.0008) [2023-12-27 09:06:10,190][20347] Updated weights for policy 0, policy_version 500197 (0.0009) [2023-12-27 09:06:10,247][20347] Updated weights for policy 0, policy_version 500207 (0.0008) [2023-12-27 09:06:10,296][20347] Updated weights for policy 0, policy_version 500217 (0.0007) [2023-12-27 09:06:10,827][20381] Updated weights for policy 1, policy_version 499898 (0.0008) [2023-12-27 09:06:10,886][20381] Updated weights for policy 1, policy_version 499908 (0.0007) [2023-12-27 09:06:10,936][20347] Updated weights for policy 0, policy_version 500227 (0.0007) [2023-12-27 09:06:10,939][20381] Updated weights for policy 1, policy_version 499918 (0.0008) [2023-12-27 09:06:10,993][20347] Updated weights for policy 0, policy_version 500237 (0.0006) [2023-12-27 09:06:11,058][20347] Updated weights for policy 0, policy_version 500247 (0.0007) [2023-12-27 09:06:11,722][20381] Updated weights for policy 1, policy_version 499928 (0.0008) [2023-12-27 09:06:11,784][20347] Updated weights for policy 0, policy_version 500257 (0.0006) [2023-12-27 09:06:11,788][20381] Updated weights for policy 1, policy_version 499938 (0.0007) [2023-12-27 09:06:11,837][20347] Updated weights for policy 0, policy_version 500267 (0.0006) [2023-12-27 09:06:11,849][20381] Updated weights for policy 1, policy_version 499948 (0.0007) [2023-12-27 09:06:11,894][20347] Updated weights for policy 0, policy_version 500277 (0.0005) [2023-12-27 09:06:11,951][20347] Updated weights for policy 0, policy_version 500287 (0.0007) [2023-12-27 09:06:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 256098304. Throughput: 0: 10365.2, 1: 10257.3. Samples: 256090244. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:12,363][19170] Avg episode reward: [(0, '963.160'), (1, '700.210')] [2023-12-27 09:06:12,518][20381] Updated weights for policy 1, policy_version 499958 (0.0006) [2023-12-27 09:06:12,564][20381] Updated weights for policy 1, policy_version 499968 (0.0005) [2023-12-27 09:06:12,624][20347] Updated weights for policy 0, policy_version 500297 (0.0007) [2023-12-27 09:06:12,629][20381] Updated weights for policy 1, policy_version 499978 (0.0006) [2023-12-27 09:06:12,673][20347] Updated weights for policy 0, policy_version 500307 (0.0006) [2023-12-27 09:06:12,729][20347] Updated weights for policy 0, policy_version 500317 (0.0008) [2023-12-27 09:06:13,291][20381] Updated weights for policy 1, policy_version 499988 (0.0006) [2023-12-27 09:06:13,339][20381] Updated weights for policy 1, policy_version 499998 (0.0007) [2023-12-27 09:06:13,387][20381] Updated weights for policy 1, policy_version 500008 (0.0007) [2023-12-27 09:06:13,430][20347] Updated weights for policy 0, policy_version 500327 (0.0006) [2023-12-27 09:06:13,483][20347] Updated weights for policy 0, policy_version 500337 (0.0009) [2023-12-27 09:06:13,540][20347] Updated weights for policy 0, policy_version 500347 (0.0008) [2023-12-27 09:06:14,041][20381] Updated weights for policy 1, policy_version 500018 (0.0006) [2023-12-27 09:06:14,090][20381] Updated weights for policy 1, policy_version 500028 (0.0009) [2023-12-27 09:06:14,137][20381] Updated weights for policy 1, policy_version 500038 (0.0008) [2023-12-27 09:06:14,187][20381] Updated weights for policy 1, policy_version 500048 (0.0008) [2023-12-27 09:06:14,255][20347] Updated weights for policy 0, policy_version 500357 (0.0007) [2023-12-27 09:06:14,303][20347] Updated weights for policy 0, policy_version 500367 (0.0007) [2023-12-27 09:06:14,364][20347] Updated weights for policy 0, policy_version 500377 (0.0007) [2023-12-27 09:06:14,873][20381] Updated weights for policy 1, policy_version 500058 (0.0009) [2023-12-27 09:06:14,930][20381] Updated weights for policy 1, policy_version 500068 (0.0010) [2023-12-27 09:06:14,990][20381] Updated weights for policy 1, policy_version 500078 (0.0011) [2023-12-27 09:06:15,084][20347] Updated weights for policy 0, policy_version 500387 (0.0007) [2023-12-27 09:06:15,151][20347] Updated weights for policy 0, policy_version 500397 (0.0007) [2023-12-27 09:06:15,215][20347] Updated weights for policy 0, policy_version 500407 (0.0008) [2023-12-27 09:06:15,640][20381] Updated weights for policy 1, policy_version 500088 (0.0011) [2023-12-27 09:06:15,695][20381] Updated weights for policy 1, policy_version 500098 (0.0010) [2023-12-27 09:06:15,749][20381] Updated weights for policy 1, policy_version 500108 (0.0010) [2023-12-27 09:06:15,887][20347] Updated weights for policy 0, policy_version 500417 (0.0008) [2023-12-27 09:06:15,942][20347] Updated weights for policy 0, policy_version 500427 (0.0009) [2023-12-27 09:06:15,991][20347] Updated weights for policy 0, policy_version 500437 (0.0007) [2023-12-27 09:06:16,053][20347] Updated weights for policy 0, policy_version 500447 (0.0008) [2023-12-27 09:06:16,312][20381] Updated weights for policy 1, policy_version 500118 (0.0007) [2023-12-27 09:06:16,364][20381] Updated weights for policy 1, policy_version 500128 (0.0009) [2023-12-27 09:06:16,422][20381] Updated weights for policy 1, policy_version 500138 (0.0011) [2023-12-27 09:06:16,810][20347] Updated weights for policy 0, policy_version 500457 (0.0006) [2023-12-27 09:06:16,863][20347] Updated weights for policy 0, policy_version 500467 (0.0008) [2023-12-27 09:06:16,918][20347] Updated weights for policy 0, policy_version 500477 (0.0007) [2023-12-27 09:06:17,024][20381] Updated weights for policy 1, policy_version 500148 (0.0011) [2023-12-27 09:06:17,081][20381] Updated weights for policy 1, policy_version 500158 (0.0009) [2023-12-27 09:06:17,140][20381] Updated weights for policy 1, policy_version 500168 (0.0005) [2023-12-27 09:06:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 256204800. Throughput: 0: 10238.1, 1: 10283.2. Samples: 256213600. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:17,363][19170] Avg episode reward: [(0, '964.040'), (1, '756.740')] [2023-12-27 09:06:17,616][20347] Updated weights for policy 0, policy_version 500487 (0.0007) [2023-12-27 09:06:17,659][20347] Updated weights for policy 0, policy_version 500497 (0.0007) [2023-12-27 09:06:17,706][20347] Updated weights for policy 0, policy_version 500507 (0.0007) [2023-12-27 09:06:17,803][20381] Updated weights for policy 1, policy_version 500178 (0.0010) [2023-12-27 09:06:17,855][20381] Updated weights for policy 1, policy_version 500188 (0.0010) [2023-12-27 09:06:17,900][20381] Updated weights for policy 1, policy_version 500198 (0.0010) [2023-12-27 09:06:17,946][20381] Updated weights for policy 1, policy_version 500208 (0.0010) [2023-12-27 09:06:18,372][20347] Updated weights for policy 0, policy_version 500517 (0.0007) [2023-12-27 09:06:18,423][20347] Updated weights for policy 0, policy_version 500527 (0.0008) [2023-12-27 09:06:18,482][20347] Updated weights for policy 0, policy_version 500537 (0.0007) [2023-12-27 09:06:18,643][20381] Updated weights for policy 1, policy_version 500218 (0.0011) [2023-12-27 09:06:18,709][20381] Updated weights for policy 1, policy_version 500228 (0.0007) [2023-12-27 09:06:18,770][20381] Updated weights for policy 1, policy_version 500238 (0.0007) [2023-12-27 09:06:19,231][20347] Updated weights for policy 0, policy_version 500547 (0.0008) [2023-12-27 09:06:19,287][20347] Updated weights for policy 0, policy_version 500557 (0.0007) [2023-12-27 09:06:19,348][20347] Updated weights for policy 0, policy_version 500567 (0.0007) [2023-12-27 09:06:19,364][20381] Updated weights for policy 1, policy_version 500248 (0.0007) [2023-12-27 09:06:19,435][20381] Updated weights for policy 1, policy_version 500258 (0.0008) [2023-12-27 09:06:19,505][20381] Updated weights for policy 1, policy_version 500268 (0.0008) [2023-12-27 09:06:19,965][20347] Updated weights for policy 0, policy_version 500577 (0.0009) [2023-12-27 09:06:20,031][20347] Updated weights for policy 0, policy_version 500587 (0.0007) [2023-12-27 09:06:20,098][20347] Updated weights for policy 0, policy_version 500597 (0.0007) [2023-12-27 09:06:20,164][20347] Updated weights for policy 0, policy_version 500607 (0.0006) [2023-12-27 09:06:20,225][20381] Updated weights for policy 1, policy_version 500278 (0.0008) [2023-12-27 09:06:20,285][20381] Updated weights for policy 1, policy_version 500288 (0.0006) [2023-12-27 09:06:20,343][20381] Updated weights for policy 1, policy_version 500298 (0.0010) [2023-12-27 09:06:20,851][20347] Updated weights for policy 0, policy_version 500617 (0.0009) [2023-12-27 09:06:20,906][20347] Updated weights for policy 0, policy_version 500627 (0.0008) [2023-12-27 09:06:20,950][20381] Updated weights for policy 1, policy_version 500308 (0.0008) [2023-12-27 09:06:20,987][20347] Updated weights for policy 0, policy_version 500637 (0.0006) [2023-12-27 09:06:21,012][20381] Updated weights for policy 1, policy_version 500318 (0.0006) [2023-12-27 09:06:21,073][20381] Updated weights for policy 1, policy_version 500328 (0.0008) [2023-12-27 09:06:21,658][20347] Updated weights for policy 0, policy_version 500647 (0.0008) [2023-12-27 09:06:21,711][20347] Updated weights for policy 0, policy_version 500657 (0.0008) [2023-12-27 09:06:21,780][20347] Updated weights for policy 0, policy_version 500667 (0.0008) [2023-12-27 09:06:21,851][20381] Updated weights for policy 1, policy_version 500338 (0.0008) [2023-12-27 09:06:21,917][20381] Updated weights for policy 1, policy_version 500348 (0.0005) [2023-12-27 09:06:21,974][20381] Updated weights for policy 1, policy_version 500358 (0.0010) [2023-12-27 09:06:22,033][20381] Updated weights for policy 1, policy_version 500368 (0.0011) [2023-12-27 09:06:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 256303104. Throughput: 0: 10218.4, 1: 10296.3. Samples: 256275320. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:22,363][19170] Avg episode reward: [(0, '972.630'), (1, '824.270')] [2023-12-27 09:06:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000500368_128114688.pth... [2023-12-27 09:06:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000500672_128188416.pth... [2023-12-27 09:06:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000499488_127885312.pth [2023-12-27 09:06:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000499184_127811584.pth [2023-12-27 09:06:22,546][20347] Updated weights for policy 0, policy_version 500677 (0.0009) [2023-12-27 09:06:22,596][20347] Updated weights for policy 0, policy_version 500687 (0.0010) [2023-12-27 09:06:22,654][20347] Updated weights for policy 0, policy_version 500697 (0.0009) [2023-12-27 09:06:22,674][20381] Updated weights for policy 1, policy_version 500378 (0.0011) [2023-12-27 09:06:22,734][20381] Updated weights for policy 1, policy_version 500388 (0.0011) [2023-12-27 09:06:22,788][20381] Updated weights for policy 1, policy_version 500398 (0.0010) [2023-12-27 09:06:23,273][20347] Updated weights for policy 0, policy_version 500707 (0.0009) [2023-12-27 09:06:23,327][20347] Updated weights for policy 0, policy_version 500717 (0.0005) [2023-12-27 09:06:23,377][20347] Updated weights for policy 0, policy_version 500727 (0.0006) [2023-12-27 09:06:23,391][20291] KL-divergence is very high: 124.7082 [2023-12-27 09:06:23,406][20291] KL-divergence is very high: 100.8469 [2023-12-27 09:06:23,412][20291] KL-divergence is very high: 653.5118 [2023-12-27 09:06:23,455][20381] Updated weights for policy 1, policy_version 500408 (0.0007) [2023-12-27 09:06:23,509][20381] Updated weights for policy 1, policy_version 500418 (0.0008) [2023-12-27 09:06:23,558][20381] Updated weights for policy 1, policy_version 500428 (0.0007) [2023-12-27 09:06:23,959][20347] Updated weights for policy 0, policy_version 500737 (0.0007) [2023-12-27 09:06:23,961][20291] KL-divergence is very high: 283.0211 [2023-12-27 09:06:24,008][20347] Updated weights for policy 0, policy_version 500747 (0.0008) [2023-12-27 09:06:24,035][20291] KL-divergence is very high: 159.5830 [2023-12-27 09:06:24,055][20347] Updated weights for policy 0, policy_version 500757 (0.0007) [2023-12-27 09:06:24,111][20347] Updated weights for policy 0, policy_version 500767 (0.0008) [2023-12-27 09:06:24,196][20381] Updated weights for policy 1, policy_version 500438 (0.0007) [2023-12-27 09:06:24,250][20381] Updated weights for policy 1, policy_version 500448 (0.0008) [2023-12-27 09:06:24,308][20381] Updated weights for policy 1, policy_version 500458 (0.0007) [2023-12-27 09:06:24,816][20291] KL-divergence is very high: 129.3492 [2023-12-27 09:06:24,821][20347] Updated weights for policy 0, policy_version 500777 (0.0006) [2023-12-27 09:06:24,876][20347] Updated weights for policy 0, policy_version 500787 (0.0005) [2023-12-27 09:06:24,931][20347] Updated weights for policy 0, policy_version 500797 (0.0005) [2023-12-27 09:06:25,013][20381] Updated weights for policy 1, policy_version 500468 (0.0008) [2023-12-27 09:06:25,069][20381] Updated weights for policy 1, policy_version 500478 (0.0008) [2023-12-27 09:06:25,118][20381] Updated weights for policy 1, policy_version 500488 (0.0008) [2023-12-27 09:06:25,514][20347] Updated weights for policy 0, policy_version 500807 (0.0006) [2023-12-27 09:06:25,571][20347] Updated weights for policy 0, policy_version 500817 (0.0007) [2023-12-27 09:06:25,620][20347] Updated weights for policy 0, policy_version 500827 (0.0007) [2023-12-27 09:06:25,833][20381] Updated weights for policy 1, policy_version 500498 (0.0008) [2023-12-27 09:06:25,889][20381] Updated weights for policy 1, policy_version 500508 (0.0007) [2023-12-27 09:06:25,945][20381] Updated weights for policy 1, policy_version 500518 (0.0008) [2023-12-27 09:06:25,991][20381] Updated weights for policy 1, policy_version 500528 (0.0008) [2023-12-27 09:06:26,251][20347] Updated weights for policy 0, policy_version 500837 (0.0007) [2023-12-27 09:06:26,303][20347] Updated weights for policy 0, policy_version 500847 (0.0005) [2023-12-27 09:06:26,365][20347] Updated weights for policy 0, policy_version 500857 (0.0005) [2023-12-27 09:06:26,722][20381] Updated weights for policy 1, policy_version 500538 (0.0008) [2023-12-27 09:06:26,769][20381] Updated weights for policy 1, policy_version 500548 (0.0007) [2023-12-27 09:06:26,819][20381] Updated weights for policy 1, policy_version 500558 (0.0007) [2023-12-27 09:06:26,975][20347] Updated weights for policy 0, policy_version 500867 (0.0006) [2023-12-27 09:06:27,020][20347] Updated weights for policy 0, policy_version 500877 (0.0007) [2023-12-27 09:06:27,075][20347] Updated weights for policy 0, policy_version 500887 (0.0008) [2023-12-27 09:06:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 256409600. Throughput: 0: 10215.6, 1: 10258.3. Samples: 256398612. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:27,363][19170] Avg episode reward: [(0, '953.290'), (1, '850.680')] [2023-12-27 09:06:27,488][20381] Updated weights for policy 1, policy_version 500568 (0.0005) [2023-12-27 09:06:27,536][20381] Updated weights for policy 1, policy_version 500578 (0.0005) [2023-12-27 09:06:27,588][20381] Updated weights for policy 1, policy_version 500588 (0.0006) [2023-12-27 09:06:27,676][20347] Updated weights for policy 0, policy_version 500897 (0.0007) [2023-12-27 09:06:27,732][20347] Updated weights for policy 0, policy_version 500907 (0.0005) [2023-12-27 09:06:27,782][20347] Updated weights for policy 0, policy_version 500917 (0.0007) [2023-12-27 09:06:27,831][20347] Updated weights for policy 0, policy_version 500927 (0.0006) [2023-12-27 09:06:28,263][20381] Updated weights for policy 1, policy_version 500598 (0.0010) [2023-12-27 09:06:28,306][20381] Updated weights for policy 1, policy_version 500608 (0.0010) [2023-12-27 09:06:28,361][20381] Updated weights for policy 1, policy_version 500618 (0.0011) [2023-12-27 09:06:28,469][20347] Updated weights for policy 0, policy_version 500937 (0.0005) [2023-12-27 09:06:28,532][20347] Updated weights for policy 0, policy_version 500947 (0.0010) [2023-12-27 09:06:28,594][20347] Updated weights for policy 0, policy_version 500957 (0.0010) [2023-12-27 09:06:29,025][20381] Updated weights for policy 1, policy_version 500628 (0.0010) [2023-12-27 09:06:29,083][20381] Updated weights for policy 1, policy_version 500638 (0.0010) [2023-12-27 09:06:29,142][20381] Updated weights for policy 1, policy_version 500648 (0.0011) [2023-12-27 09:06:29,240][20347] Updated weights for policy 0, policy_version 500967 (0.0011) [2023-12-27 09:06:29,293][20347] Updated weights for policy 0, policy_version 500977 (0.0010) [2023-12-27 09:06:29,352][20347] Updated weights for policy 0, policy_version 500987 (0.0012) [2023-12-27 09:06:29,774][20381] Updated weights for policy 1, policy_version 500658 (0.0010) [2023-12-27 09:06:29,837][20381] Updated weights for policy 1, policy_version 500668 (0.0007) [2023-12-27 09:06:29,898][20381] Updated weights for policy 1, policy_version 500678 (0.0007) [2023-12-27 09:06:29,958][20381] Updated weights for policy 1, policy_version 500688 (0.0008) [2023-12-27 09:06:30,014][20347] Updated weights for policy 0, policy_version 500997 (0.0008) [2023-12-27 09:06:30,066][20347] Updated weights for policy 0, policy_version 501007 (0.0007) [2023-12-27 09:06:30,115][20347] Updated weights for policy 0, policy_version 501017 (0.0008) [2023-12-27 09:06:30,565][20381] Updated weights for policy 1, policy_version 500698 (0.0005) [2023-12-27 09:06:30,625][20381] Updated weights for policy 1, policy_version 500708 (0.0005) [2023-12-27 09:06:30,682][20381] Updated weights for policy 1, policy_version 500718 (0.0005) [2023-12-27 09:06:30,854][20347] Updated weights for policy 0, policy_version 501027 (0.0007) [2023-12-27 09:06:30,898][20347] Updated weights for policy 0, policy_version 501037 (0.0005) [2023-12-27 09:06:30,945][20347] Updated weights for policy 0, policy_version 501047 (0.0005) [2023-12-27 09:06:31,221][20381] Updated weights for policy 1, policy_version 500728 (0.0006) [2023-12-27 09:06:31,278][20381] Updated weights for policy 1, policy_version 500738 (0.0006) [2023-12-27 09:06:31,340][20381] Updated weights for policy 1, policy_version 500748 (0.0006) [2023-12-27 09:06:31,671][20347] Updated weights for policy 0, policy_version 501057 (0.0006) [2023-12-27 09:06:31,734][20347] Updated weights for policy 0, policy_version 501067 (0.0008) [2023-12-27 09:06:31,785][20347] Updated weights for policy 0, policy_version 501077 (0.0008) [2023-12-27 09:06:31,836][20347] Updated weights for policy 0, policy_version 501087 (0.0008) [2023-12-27 09:06:31,961][20381] Updated weights for policy 1, policy_version 500758 (0.0006) [2023-12-27 09:06:32,013][20381] Updated weights for policy 1, policy_version 500768 (0.0005) [2023-12-27 09:06:32,065][20381] Updated weights for policy 1, policy_version 500778 (0.0005) [2023-12-27 09:06:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 256516096. Throughput: 0: 10260.8, 1: 10317.4. Samples: 256526448. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:32,363][19170] Avg episode reward: [(0, '934.150'), (1, '885.270')] [2023-12-27 09:06:32,566][20347] Updated weights for policy 0, policy_version 501097 (0.0007) [2023-12-27 09:06:32,616][20347] Updated weights for policy 0, policy_version 501107 (0.0008) [2023-12-27 09:06:32,662][20347] Updated weights for policy 0, policy_version 501117 (0.0010) [2023-12-27 09:06:32,707][20381] Updated weights for policy 1, policy_version 500788 (0.0007) [2023-12-27 09:06:32,762][20381] Updated weights for policy 1, policy_version 500798 (0.0008) [2023-12-27 09:06:32,807][20381] Updated weights for policy 1, policy_version 500808 (0.0007) [2023-12-27 09:06:33,348][20347] Updated weights for policy 0, policy_version 501127 (0.0007) [2023-12-27 09:06:33,393][20347] Updated weights for policy 0, policy_version 501137 (0.0009) [2023-12-27 09:06:33,431][20381] Updated weights for policy 1, policy_version 500818 (0.0007) [2023-12-27 09:06:33,445][20347] Updated weights for policy 0, policy_version 501147 (0.0006) [2023-12-27 09:06:33,487][20381] Updated weights for policy 1, policy_version 500828 (0.0006) [2023-12-27 09:06:33,540][20381] Updated weights for policy 1, policy_version 500838 (0.0007) [2023-12-27 09:06:33,596][20381] Updated weights for policy 1, policy_version 500848 (0.0005) [2023-12-27 09:06:34,127][20381] Updated weights for policy 1, policy_version 500858 (0.0005) [2023-12-27 09:06:34,169][20347] Updated weights for policy 0, policy_version 501157 (0.0006) [2023-12-27 09:06:34,187][20381] Updated weights for policy 1, policy_version 500868 (0.0007) [2023-12-27 09:06:34,217][20347] Updated weights for policy 0, policy_version 501167 (0.0007) [2023-12-27 09:06:34,243][20381] Updated weights for policy 1, policy_version 500878 (0.0007) [2023-12-27 09:06:34,277][20347] Updated weights for policy 0, policy_version 501177 (0.0007) [2023-12-27 09:06:34,871][20381] Updated weights for policy 1, policy_version 500888 (0.0007) [2023-12-27 09:06:34,928][20381] Updated weights for policy 1, policy_version 500898 (0.0005) [2023-12-27 09:06:34,984][20381] Updated weights for policy 1, policy_version 500908 (0.0005) [2023-12-27 09:06:35,003][20347] Updated weights for policy 0, policy_version 501187 (0.0007) [2023-12-27 09:06:35,062][20347] Updated weights for policy 0, policy_version 501197 (0.0005) [2023-12-27 09:06:35,124][20347] Updated weights for policy 0, policy_version 501207 (0.0005) [2023-12-27 09:06:35,621][20381] Updated weights for policy 1, policy_version 500918 (0.0007) [2023-12-27 09:06:35,675][20347] Updated weights for policy 0, policy_version 501217 (0.0006) [2023-12-27 09:06:35,677][20381] Updated weights for policy 1, policy_version 500928 (0.0007) [2023-12-27 09:06:35,731][20381] Updated weights for policy 1, policy_version 500938 (0.0007) [2023-12-27 09:06:35,734][20347] Updated weights for policy 0, policy_version 501227 (0.0005) [2023-12-27 09:06:35,798][20347] Updated weights for policy 0, policy_version 501237 (0.0005) [2023-12-27 09:06:35,860][20347] Updated weights for policy 0, policy_version 501247 (0.0005) [2023-12-27 09:06:36,358][20381] Updated weights for policy 1, policy_version 500948 (0.0006) [2023-12-27 09:06:36,423][20381] Updated weights for policy 1, policy_version 500958 (0.0008) [2023-12-27 09:06:36,483][20381] Updated weights for policy 1, policy_version 500968 (0.0008) [2023-12-27 09:06:36,527][20347] Updated weights for policy 0, policy_version 501257 (0.0009) [2023-12-27 09:06:36,576][20291] KL-divergence is very high: 123.0139 [2023-12-27 09:06:36,582][20347] Updated weights for policy 0, policy_version 501267 (0.0007) [2023-12-27 09:06:36,645][20347] Updated weights for policy 0, policy_version 501277 (0.0008) [2023-12-27 09:06:37,175][20381] Updated weights for policy 1, policy_version 500978 (0.0007) [2023-12-27 09:06:37,221][20381] Updated weights for policy 1, policy_version 500988 (0.0010) [2023-12-27 09:06:37,279][20381] Updated weights for policy 1, policy_version 500998 (0.0010) [2023-12-27 09:06:37,317][20347] Updated weights for policy 0, policy_version 501287 (0.0008) [2023-12-27 09:06:37,332][20381] Updated weights for policy 1, policy_version 501008 (0.0010) [2023-12-27 09:06:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20827.1). Total num frames: 256622592. Throughput: 0: 10265.2, 1: 10393.5. Samples: 256590796. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:37,363][19170] Avg episode reward: [(0, '933.860'), (1, '900.090')] [2023-12-27 09:06:37,364][20347] Updated weights for policy 0, policy_version 501297 (0.0008) [2023-12-27 09:06:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000501008_128278528.pth... [2023-12-27 09:06:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000499760_127959040.pth [2023-12-27 09:06:37,418][20347] Updated weights for policy 0, policy_version 501307 (0.0006) [2023-12-27 09:06:37,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000501312_128352256.pth... [2023-12-27 09:06:37,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000500096_128040960.pth [2023-12-27 09:06:38,036][20381] Updated weights for policy 1, policy_version 501018 (0.0011) [2023-12-27 09:06:38,084][20381] Updated weights for policy 1, policy_version 501028 (0.0010) [2023-12-27 09:06:38,119][20347] Updated weights for policy 0, policy_version 501317 (0.0005) [2023-12-27 09:06:38,134][20381] Updated weights for policy 1, policy_version 501038 (0.0008) [2023-12-27 09:06:38,170][20347] Updated weights for policy 0, policy_version 501327 (0.0008) [2023-12-27 09:06:38,225][20347] Updated weights for policy 0, policy_version 501337 (0.0006) [2023-12-27 09:06:38,799][20381] Updated weights for policy 1, policy_version 501048 (0.0010) [2023-12-27 09:06:38,857][20381] Updated weights for policy 1, policy_version 501058 (0.0011) [2023-12-27 09:06:38,913][20381] Updated weights for policy 1, policy_version 501068 (0.0011) [2023-12-27 09:06:38,919][20347] Updated weights for policy 0, policy_version 501347 (0.0006) [2023-12-27 09:06:38,975][20347] Updated weights for policy 0, policy_version 501357 (0.0007) [2023-12-27 09:06:39,033][20347] Updated weights for policy 0, policy_version 501367 (0.0007) [2023-12-27 09:06:39,606][20381] Updated weights for policy 1, policy_version 501078 (0.0009) [2023-12-27 09:06:39,659][20381] Updated weights for policy 1, policy_version 501088 (0.0008) [2023-12-27 09:06:39,716][20381] Updated weights for policy 1, policy_version 501098 (0.0008) [2023-12-27 09:06:39,744][20347] Updated weights for policy 0, policy_version 501377 (0.0007) [2023-12-27 09:06:39,804][20347] Updated weights for policy 0, policy_version 501387 (0.0008) [2023-12-27 09:06:39,870][20347] Updated weights for policy 0, policy_version 501397 (0.0006) [2023-12-27 09:06:39,934][20347] Updated weights for policy 0, policy_version 501407 (0.0009) [2023-12-27 09:06:40,393][20381] Updated weights for policy 1, policy_version 501108 (0.0009) [2023-12-27 09:06:40,457][20381] Updated weights for policy 1, policy_version 501118 (0.0009) [2023-12-27 09:06:40,512][20381] Updated weights for policy 1, policy_version 501128 (0.0010) [2023-12-27 09:06:40,591][20347] Updated weights for policy 0, policy_version 501417 (0.0008) [2023-12-27 09:06:40,646][20347] Updated weights for policy 0, policy_version 501427 (0.0007) [2023-12-27 09:06:40,693][20347] Updated weights for policy 0, policy_version 501437 (0.0008) [2023-12-27 09:06:41,161][20381] Updated weights for policy 1, policy_version 501138 (0.0006) [2023-12-27 09:06:41,223][20381] Updated weights for policy 1, policy_version 501148 (0.0010) [2023-12-27 09:06:41,290][20381] Updated weights for policy 1, policy_version 501158 (0.0011) [2023-12-27 09:06:41,354][20381] Updated weights for policy 1, policy_version 501168 (0.0011) [2023-12-27 09:06:41,393][20347] Updated weights for policy 0, policy_version 501447 (0.0009) [2023-12-27 09:06:41,451][20347] Updated weights for policy 0, policy_version 501457 (0.0011) [2023-12-27 09:06:41,507][20347] Updated weights for policy 0, policy_version 501467 (0.0007) [2023-12-27 09:06:42,072][20381] Updated weights for policy 1, policy_version 501178 (0.0011) [2023-12-27 09:06:42,135][20381] Updated weights for policy 1, policy_version 501188 (0.0011) [2023-12-27 09:06:42,158][20347] Updated weights for policy 0, policy_version 501477 (0.0007) [2023-12-27 09:06:42,192][20381] Updated weights for policy 1, policy_version 501198 (0.0011) [2023-12-27 09:06:42,214][20347] Updated weights for policy 0, policy_version 501487 (0.0007) [2023-12-27 09:06:42,276][20347] Updated weights for policy 0, policy_version 501497 (0.0008) [2023-12-27 09:06:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 256729088. Throughput: 0: 10282.7, 1: 10402.9. Samples: 256713784. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:42,363][19170] Avg episode reward: [(0, '962.150'), (1, '910.320')] [2023-12-27 09:06:42,881][20381] Updated weights for policy 1, policy_version 501208 (0.0011) [2023-12-27 09:06:42,935][20381] Updated weights for policy 1, policy_version 501218 (0.0011) [2023-12-27 09:06:42,968][20347] Updated weights for policy 0, policy_version 501507 (0.0008) [2023-12-27 09:06:42,988][20381] Updated weights for policy 1, policy_version 501228 (0.0009) [2023-12-27 09:06:43,028][20347] Updated weights for policy 0, policy_version 501517 (0.0010) [2023-12-27 09:06:43,078][20347] Updated weights for policy 0, policy_version 501527 (0.0010) [2023-12-27 09:06:43,642][20381] Updated weights for policy 1, policy_version 501238 (0.0007) [2023-12-27 09:06:43,699][20381] Updated weights for policy 1, policy_version 501248 (0.0005) [2023-12-27 09:06:43,725][20347] Updated weights for policy 0, policy_version 501537 (0.0010) [2023-12-27 09:06:43,755][20381] Updated weights for policy 1, policy_version 501258 (0.0005) [2023-12-27 09:06:43,779][20347] Updated weights for policy 0, policy_version 501547 (0.0007) [2023-12-27 09:06:43,830][20347] Updated weights for policy 0, policy_version 501557 (0.0007) [2023-12-27 09:06:43,880][20347] Updated weights for policy 0, policy_version 501567 (0.0007) [2023-12-27 09:06:44,303][20381] Updated weights for policy 1, policy_version 501268 (0.0007) [2023-12-27 09:06:44,349][20381] Updated weights for policy 1, policy_version 501278 (0.0005) [2023-12-27 09:06:44,398][20381] Updated weights for policy 1, policy_version 501288 (0.0006) [2023-12-27 09:06:44,645][20347] Updated weights for policy 0, policy_version 501577 (0.0007) [2023-12-27 09:06:44,704][20347] Updated weights for policy 0, policy_version 501587 (0.0005) [2023-12-27 09:06:44,756][20347] Updated weights for policy 0, policy_version 501597 (0.0005) [2023-12-27 09:06:44,993][20381] Updated weights for policy 1, policy_version 501298 (0.0006) [2023-12-27 09:06:45,054][20381] Updated weights for policy 1, policy_version 501308 (0.0007) [2023-12-27 09:06:45,110][20381] Updated weights for policy 1, policy_version 501318 (0.0006) [2023-12-27 09:06:45,161][20381] Updated weights for policy 1, policy_version 501328 (0.0007) [2023-12-27 09:06:45,376][20347] Updated weights for policy 0, policy_version 501607 (0.0009) [2023-12-27 09:06:45,440][20347] Updated weights for policy 0, policy_version 501617 (0.0008) [2023-12-27 09:06:45,500][20347] Updated weights for policy 0, policy_version 501627 (0.0008) [2023-12-27 09:06:45,808][20381] Updated weights for policy 1, policy_version 501338 (0.0006) [2023-12-27 09:06:45,855][20381] Updated weights for policy 1, policy_version 501348 (0.0007) [2023-12-27 09:06:45,906][20381] Updated weights for policy 1, policy_version 501358 (0.0008) [2023-12-27 09:06:46,146][20347] Updated weights for policy 0, policy_version 501637 (0.0009) [2023-12-27 09:06:46,204][20347] Updated weights for policy 0, policy_version 501647 (0.0010) [2023-12-27 09:06:46,268][20347] Updated weights for policy 0, policy_version 501657 (0.0009) [2023-12-27 09:06:46,527][20381] Updated weights for policy 1, policy_version 501368 (0.0005) [2023-12-27 09:06:46,584][20381] Updated weights for policy 1, policy_version 501378 (0.0006) [2023-12-27 09:06:46,645][20381] Updated weights for policy 1, policy_version 501388 (0.0007) [2023-12-27 09:06:46,955][20347] Updated weights for policy 0, policy_version 501667 (0.0007) [2023-12-27 09:06:47,004][20347] Updated weights for policy 0, policy_version 501677 (0.0007) [2023-12-27 09:06:47,051][20347] Updated weights for policy 0, policy_version 501687 (0.0007) [2023-12-27 09:06:47,292][20381] Updated weights for policy 1, policy_version 501398 (0.0008) [2023-12-27 09:06:47,345][20381] Updated weights for policy 1, policy_version 501408 (0.0006) [2023-12-27 09:06:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 256827392. Throughput: 0: 10254.7, 1: 10484.6. Samples: 256840016. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:47,363][19170] Avg episode reward: [(0, '932.880'), (1, '912.720')] [2023-12-27 09:06:47,407][20381] Updated weights for policy 1, policy_version 501418 (0.0005) [2023-12-27 09:06:47,838][20347] Updated weights for policy 0, policy_version 501697 (0.0007) [2023-12-27 09:06:47,897][20347] Updated weights for policy 0, policy_version 501707 (0.0007) [2023-12-27 09:06:47,945][20381] Updated weights for policy 1, policy_version 501428 (0.0005) [2023-12-27 09:06:47,945][20347] Updated weights for policy 0, policy_version 501717 (0.0008) [2023-12-27 09:06:48,000][20381] Updated weights for policy 1, policy_version 501438 (0.0005) [2023-12-27 09:06:48,004][20347] Updated weights for policy 0, policy_version 501727 (0.0008) [2023-12-27 09:06:48,061][20381] Updated weights for policy 1, policy_version 501448 (0.0007) [2023-12-27 09:06:48,675][20347] Updated weights for policy 0, policy_version 501737 (0.0007) [2023-12-27 09:06:48,734][20347] Updated weights for policy 0, policy_version 501747 (0.0008) [2023-12-27 09:06:48,753][20381] Updated weights for policy 1, policy_version 501458 (0.0008) [2023-12-27 09:06:48,784][20347] Updated weights for policy 0, policy_version 501757 (0.0007) [2023-12-27 09:06:48,809][20381] Updated weights for policy 1, policy_version 501468 (0.0008) [2023-12-27 09:06:48,860][20381] Updated weights for policy 1, policy_version 501478 (0.0011) [2023-12-27 09:06:48,917][20381] Updated weights for policy 1, policy_version 501488 (0.0009) [2023-12-27 09:06:49,421][20347] Updated weights for policy 0, policy_version 501767 (0.0007) [2023-12-27 09:06:49,470][20347] Updated weights for policy 0, policy_version 501777 (0.0007) [2023-12-27 09:06:49,519][20347] Updated weights for policy 0, policy_version 501787 (0.0009) [2023-12-27 09:06:49,579][20381] Updated weights for policy 1, policy_version 501498 (0.0009) [2023-12-27 09:06:49,645][20381] Updated weights for policy 1, policy_version 501508 (0.0007) [2023-12-27 09:06:49,695][20381] Updated weights for policy 1, policy_version 501518 (0.0008) [2023-12-27 09:06:50,256][20347] Updated weights for policy 0, policy_version 501797 (0.0007) [2023-12-27 09:06:50,323][20347] Updated weights for policy 0, policy_version 501807 (0.0005) [2023-12-27 09:06:50,375][20347] Updated weights for policy 0, policy_version 501817 (0.0005) [2023-12-27 09:06:50,395][20381] Updated weights for policy 1, policy_version 501528 (0.0006) [2023-12-27 09:06:50,452][20381] Updated weights for policy 1, policy_version 501538 (0.0006) [2023-12-27 09:06:50,499][20381] Updated weights for policy 1, policy_version 501548 (0.0007) [2023-12-27 09:06:51,009][20347] Updated weights for policy 0, policy_version 501827 (0.0010) [2023-12-27 09:06:51,069][20347] Updated weights for policy 0, policy_version 501837 (0.0009) [2023-12-27 09:06:51,078][20381] Updated weights for policy 1, policy_version 501558 (0.0008) [2023-12-27 09:06:51,128][20347] Updated weights for policy 0, policy_version 501847 (0.0010) [2023-12-27 09:06:51,141][20381] Updated weights for policy 1, policy_version 501568 (0.0006) [2023-12-27 09:06:51,197][20381] Updated weights for policy 1, policy_version 501578 (0.0007) [2023-12-27 09:06:51,777][20347] Updated weights for policy 0, policy_version 501857 (0.0007) [2023-12-27 09:06:51,838][20347] Updated weights for policy 0, policy_version 501867 (0.0005) [2023-12-27 09:06:51,890][20381] Updated weights for policy 1, policy_version 501588 (0.0008) [2023-12-27 09:06:51,894][20347] Updated weights for policy 0, policy_version 501877 (0.0005) [2023-12-27 09:06:51,949][20347] Updated weights for policy 0, policy_version 501887 (0.0006) [2023-12-27 09:06:51,949][20381] Updated weights for policy 1, policy_version 501598 (0.0009) [2023-12-27 09:06:52,009][20381] Updated weights for policy 1, policy_version 501608 (0.0007) [2023-12-27 09:06:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 256933888. Throughput: 0: 10252.0, 1: 10490.1. Samples: 256901704. Policy #0 lag: (min: 9.0, avg: 26.0, max: 41.0) [2023-12-27 09:06:52,363][19170] Avg episode reward: [(0, '932.910'), (1, '883.520')] [2023-12-27 09:06:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000501616_128434176.pth... [2023-12-27 09:06:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000501888_128499712.pth... [2023-12-27 09:06:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000500368_128114688.pth [2023-12-27 09:06:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000500672_128188416.pth [2023-12-27 09:06:52,564][20347] Updated weights for policy 0, policy_version 501897 (0.0007) [2023-12-27 09:06:52,625][20347] Updated weights for policy 0, policy_version 501907 (0.0007) [2023-12-27 09:06:52,673][20347] Updated weights for policy 0, policy_version 501917 (0.0008) [2023-12-27 09:06:52,739][20381] Updated weights for policy 1, policy_version 501618 (0.0009) [2023-12-27 09:06:52,795][20381] Updated weights for policy 1, policy_version 501628 (0.0007) [2023-12-27 09:06:52,846][20381] Updated weights for policy 1, policy_version 501638 (0.0008) [2023-12-27 09:06:52,909][20381] Updated weights for policy 1, policy_version 501648 (0.0007) [2023-12-27 09:06:53,331][20347] Updated weights for policy 0, policy_version 501927 (0.0009) [2023-12-27 09:06:53,386][20347] Updated weights for policy 0, policy_version 501937 (0.0010) [2023-12-27 09:06:53,447][20347] Updated weights for policy 0, policy_version 501947 (0.0009) [2023-12-27 09:06:53,625][20381] Updated weights for policy 1, policy_version 501658 (0.0008) [2023-12-27 09:06:53,683][20381] Updated weights for policy 1, policy_version 501668 (0.0007) [2023-12-27 09:06:53,740][20381] Updated weights for policy 1, policy_version 501678 (0.0008) [2023-12-27 09:06:54,133][20347] Updated weights for policy 0, policy_version 501957 (0.0009) [2023-12-27 09:06:54,187][20347] Updated weights for policy 0, policy_version 501967 (0.0009) [2023-12-27 09:06:54,236][20347] Updated weights for policy 0, policy_version 501977 (0.0010) [2023-12-27 09:06:54,443][20381] Updated weights for policy 1, policy_version 501688 (0.0007) [2023-12-27 09:06:54,490][20381] Updated weights for policy 1, policy_version 501698 (0.0007) [2023-12-27 09:06:54,537][20381] Updated weights for policy 1, policy_version 501708 (0.0007) [2023-12-27 09:06:54,918][20347] Updated weights for policy 0, policy_version 501987 (0.0010) [2023-12-27 09:06:54,972][20347] Updated weights for policy 0, policy_version 501997 (0.0008) [2023-12-27 09:06:55,032][20347] Updated weights for policy 0, policy_version 502007 (0.0007) [2023-12-27 09:06:55,236][20381] Updated weights for policy 1, policy_version 501718 (0.0007) [2023-12-27 09:06:55,278][20381] Updated weights for policy 1, policy_version 501728 (0.0008) [2023-12-27 09:06:55,329][20381] Updated weights for policy 1, policy_version 501738 (0.0007) [2023-12-27 09:06:55,689][20347] Updated weights for policy 0, policy_version 502017 (0.0008) [2023-12-27 09:06:55,748][20347] Updated weights for policy 0, policy_version 502027 (0.0007) [2023-12-27 09:06:55,804][20347] Updated weights for policy 0, policy_version 502037 (0.0007) [2023-12-27 09:06:55,858][20347] Updated weights for policy 0, policy_version 502047 (0.0007) [2023-12-27 09:06:56,059][20381] Updated weights for policy 1, policy_version 501748 (0.0008) [2023-12-27 09:06:56,112][20381] Updated weights for policy 1, policy_version 501758 (0.0008) [2023-12-27 09:06:56,164][20381] Updated weights for policy 1, policy_version 501768 (0.0007) [2023-12-27 09:06:56,503][20291] KL-divergence is very high: 1776.8019 [2023-12-27 09:06:56,509][20347] Updated weights for policy 0, policy_version 502057 (0.0007) [2023-12-27 09:06:56,539][20291] KL-divergence is very high: 646.6213 [2023-12-27 09:06:56,553][20347] Updated weights for policy 0, policy_version 502067 (0.0006) [2023-12-27 09:06:56,606][20347] Updated weights for policy 0, policy_version 502077 (0.0006) [2023-12-27 09:06:56,866][20381] Updated weights for policy 1, policy_version 501778 (0.0008) [2023-12-27 09:06:56,920][20381] Updated weights for policy 1, policy_version 501788 (0.0007) [2023-12-27 09:06:56,976][20381] Updated weights for policy 1, policy_version 501798 (0.0008) [2023-12-27 09:06:57,036][20381] Updated weights for policy 1, policy_version 501808 (0.0008) [2023-12-27 09:06:57,285][20347] Updated weights for policy 0, policy_version 502087 (0.0008) [2023-12-27 09:06:57,347][20347] Updated weights for policy 0, policy_version 502097 (0.0007) [2023-12-27 09:06:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 257032192. Throughput: 0: 10244.4, 1: 10508.7. Samples: 257024136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:06:57,363][19170] Avg episode reward: [(0, '970.050'), (1, '844.700')] [2023-12-27 09:06:57,401][20347] Updated weights for policy 0, policy_version 502107 (0.0007) [2023-12-27 09:06:57,715][20381] Updated weights for policy 1, policy_version 501818 (0.0008) [2023-12-27 09:06:57,758][20381] Updated weights for policy 1, policy_version 501828 (0.0005) [2023-12-27 09:06:57,813][20381] Updated weights for policy 1, policy_version 501838 (0.0007) [2023-12-27 09:06:58,089][20347] Updated weights for policy 0, policy_version 502117 (0.0008) [2023-12-27 09:06:58,146][20347] Updated weights for policy 0, policy_version 502127 (0.0007) [2023-12-27 09:06:58,203][20347] Updated weights for policy 0, policy_version 502137 (0.0007) [2023-12-27 09:06:58,561][20381] Updated weights for policy 1, policy_version 501848 (0.0007) [2023-12-27 09:06:58,622][20381] Updated weights for policy 1, policy_version 501858 (0.0007) [2023-12-27 09:06:58,681][20381] Updated weights for policy 1, policy_version 501868 (0.0008) [2023-12-27 09:06:58,953][20347] Updated weights for policy 0, policy_version 502147 (0.0007) [2023-12-27 09:06:59,019][20347] Updated weights for policy 0, policy_version 502157 (0.0007) [2023-12-27 09:06:59,080][20347] Updated weights for policy 0, policy_version 502167 (0.0007) [2023-12-27 09:06:59,474][20381] Updated weights for policy 1, policy_version 501878 (0.0006) [2023-12-27 09:06:59,535][20381] Updated weights for policy 1, policy_version 501888 (0.0005) [2023-12-27 09:06:59,595][20381] Updated weights for policy 1, policy_version 501898 (0.0006) [2023-12-27 09:06:59,802][20347] Updated weights for policy 0, policy_version 502177 (0.0007) [2023-12-27 09:06:59,859][20347] Updated weights for policy 0, policy_version 502187 (0.0008) [2023-12-27 09:06:59,918][20347] Updated weights for policy 0, policy_version 502197 (0.0008) [2023-12-27 09:06:59,977][20347] Updated weights for policy 0, policy_version 502207 (0.0007) [2023-12-27 09:07:00,191][20381] Updated weights for policy 1, policy_version 501908 (0.0006) [2023-12-27 09:07:00,251][20381] Updated weights for policy 1, policy_version 501918 (0.0005) [2023-12-27 09:07:00,310][20381] Updated weights for policy 1, policy_version 501928 (0.0005) [2023-12-27 09:07:00,705][20347] Updated weights for policy 0, policy_version 502217 (0.0007) [2023-12-27 09:07:00,762][20347] Updated weights for policy 0, policy_version 502227 (0.0007) [2023-12-27 09:07:00,817][20347] Updated weights for policy 0, policy_version 502237 (0.0008) [2023-12-27 09:07:00,894][20381] Updated weights for policy 1, policy_version 501938 (0.0007) [2023-12-27 09:07:00,945][20381] Updated weights for policy 1, policy_version 501948 (0.0010) [2023-12-27 09:07:00,989][20381] Updated weights for policy 1, policy_version 501958 (0.0010) [2023-12-27 09:07:01,046][20381] Updated weights for policy 1, policy_version 501968 (0.0011) [2023-12-27 09:07:01,527][20347] Updated weights for policy 0, policy_version 502247 (0.0007) [2023-12-27 09:07:01,581][20347] Updated weights for policy 0, policy_version 502257 (0.0007) [2023-12-27 09:07:01,638][20347] Updated weights for policy 0, policy_version 502267 (0.0007) [2023-12-27 09:07:01,750][20381] Updated weights for policy 1, policy_version 501978 (0.0008) [2023-12-27 09:07:01,813][20381] Updated weights for policy 1, policy_version 501988 (0.0007) [2023-12-27 09:07:01,871][20381] Updated weights for policy 1, policy_version 501998 (0.0010) [2023-12-27 09:07:02,319][20347] Updated weights for policy 0, policy_version 502277 (0.0007) [2023-12-27 09:07:02,359][20291] KL-divergence is very high: 178.0103 [2023-12-27 09:07:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20799.4). Total num frames: 257130496. Throughput: 0: 10244.7, 1: 10457.9. Samples: 257145216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:02,363][19170] Avg episode reward: [(0, '979.870'), (1, '883.570')] [2023-12-27 09:07:02,375][20347] Updated weights for policy 0, policy_version 502287 (0.0008) [2023-12-27 09:07:02,406][20291] KL-divergence is very high: 100.1248 [2023-12-27 09:07:02,433][20347] Updated weights for policy 0, policy_version 502297 (0.0006) [2023-12-27 09:07:02,532][20381] Updated weights for policy 1, policy_version 502008 (0.0008) [2023-12-27 09:07:02,582][20381] Updated weights for policy 1, policy_version 502018 (0.0008) [2023-12-27 09:07:02,628][20381] Updated weights for policy 1, policy_version 502028 (0.0008) [2023-12-27 09:07:03,037][20291] KL-divergence is very high: 103.3282 [2023-12-27 09:07:03,052][20347] Updated weights for policy 0, policy_version 502307 (0.0008) [2023-12-27 09:07:03,102][20347] Updated weights for policy 0, policy_version 502317 (0.0007) [2023-12-27 09:07:03,153][20347] Updated weights for policy 0, policy_version 502327 (0.0007) [2023-12-27 09:07:03,256][20381] Updated weights for policy 1, policy_version 502038 (0.0005) [2023-12-27 09:07:03,302][20381] Updated weights for policy 1, policy_version 502048 (0.0005) [2023-12-27 09:07:03,345][20381] Updated weights for policy 1, policy_version 502058 (0.0005) [2023-12-27 09:07:03,846][20347] Updated weights for policy 0, policy_version 502337 (0.0007) [2023-12-27 09:07:03,895][20381] Updated weights for policy 1, policy_version 502068 (0.0006) [2023-12-27 09:07:03,906][20347] Updated weights for policy 0, policy_version 502347 (0.0007) [2023-12-27 09:07:03,956][20381] Updated weights for policy 1, policy_version 502078 (0.0008) [2023-12-27 09:07:03,962][20347] Updated weights for policy 0, policy_version 502357 (0.0006) [2023-12-27 09:07:04,013][20381] Updated weights for policy 1, policy_version 502088 (0.0006) [2023-12-27 09:07:04,018][20347] Updated weights for policy 0, policy_version 502367 (0.0007) [2023-12-27 09:07:04,676][20347] Updated weights for policy 0, policy_version 502377 (0.0010) [2023-12-27 09:07:04,685][20381] Updated weights for policy 1, policy_version 502098 (0.0006) [2023-12-27 09:07:04,732][20347] Updated weights for policy 0, policy_version 502387 (0.0008) [2023-12-27 09:07:04,744][20381] Updated weights for policy 1, policy_version 502108 (0.0007) [2023-12-27 09:07:04,787][20347] Updated weights for policy 0, policy_version 502397 (0.0006) [2023-12-27 09:07:04,797][20381] Updated weights for policy 1, policy_version 502118 (0.0007) [2023-12-27 09:07:04,851][20381] Updated weights for policy 1, policy_version 502128 (0.0007) [2023-12-27 09:07:05,345][20347] Updated weights for policy 0, policy_version 502407 (0.0007) [2023-12-27 09:07:05,402][20347] Updated weights for policy 0, policy_version 502417 (0.0008) [2023-12-27 09:07:05,454][20347] Updated weights for policy 0, policy_version 502427 (0.0007) [2023-12-27 09:07:05,518][20381] Updated weights for policy 1, policy_version 502138 (0.0006) [2023-12-27 09:07:05,570][20381] Updated weights for policy 1, policy_version 502148 (0.0005) [2023-12-27 09:07:05,616][20381] Updated weights for policy 1, policy_version 502158 (0.0005) [2023-12-27 09:07:06,180][20347] Updated weights for policy 0, policy_version 502437 (0.0007) [2023-12-27 09:07:06,243][20347] Updated weights for policy 0, policy_version 502447 (0.0009) [2023-12-27 09:07:06,282][20381] Updated weights for policy 1, policy_version 502168 (0.0006) [2023-12-27 09:07:06,307][20347] Updated weights for policy 0, policy_version 502457 (0.0006) [2023-12-27 09:07:06,343][20381] Updated weights for policy 1, policy_version 502178 (0.0007) [2023-12-27 09:07:06,402][20381] Updated weights for policy 1, policy_version 502188 (0.0005) [2023-12-27 09:07:06,973][20347] Updated weights for policy 0, policy_version 502467 (0.0007) [2023-12-27 09:07:07,017][20381] Updated weights for policy 1, policy_version 502198 (0.0005) [2023-12-27 09:07:07,028][20347] Updated weights for policy 0, policy_version 502477 (0.0008) [2023-12-27 09:07:07,072][20381] Updated weights for policy 1, policy_version 502208 (0.0006) [2023-12-27 09:07:07,076][20347] Updated weights for policy 0, policy_version 502487 (0.0007) [2023-12-27 09:07:07,127][20381] Updated weights for policy 1, policy_version 502218 (0.0007) [2023-12-27 09:07:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 257245184. Throughput: 0: 10271.3, 1: 10503.1. Samples: 257210168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:07,363][19170] Avg episode reward: [(0, '951.130'), (1, '912.790')] [2023-12-27 09:07:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000502224_128589824.pth... [2023-12-27 09:07:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000502496_128655360.pth... [2023-12-27 09:07:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000501008_128278528.pth [2023-12-27 09:07:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000501312_128352256.pth [2023-12-27 09:07:07,782][20381] Updated weights for policy 1, policy_version 502228 (0.0007) [2023-12-27 09:07:07,785][20347] Updated weights for policy 0, policy_version 502497 (0.0007) [2023-12-27 09:07:07,839][20381] Updated weights for policy 1, policy_version 502238 (0.0005) [2023-12-27 09:07:07,844][20347] Updated weights for policy 0, policy_version 502507 (0.0006) [2023-12-27 09:07:07,895][20381] Updated weights for policy 1, policy_version 502248 (0.0005) [2023-12-27 09:07:07,904][20347] Updated weights for policy 0, policy_version 502517 (0.0006) [2023-12-27 09:07:07,958][20347] Updated weights for policy 0, policy_version 502527 (0.0006) [2023-12-27 09:07:08,549][20381] Updated weights for policy 1, policy_version 502258 (0.0006) [2023-12-27 09:07:08,616][20381] Updated weights for policy 1, policy_version 502268 (0.0007) [2023-12-27 09:07:08,666][20347] Updated weights for policy 0, policy_version 502537 (0.0007) [2023-12-27 09:07:08,678][20381] Updated weights for policy 1, policy_version 502278 (0.0005) [2023-12-27 09:07:08,725][20347] Updated weights for policy 0, policy_version 502547 (0.0007) [2023-12-27 09:07:08,740][20381] Updated weights for policy 1, policy_version 502288 (0.0006) [2023-12-27 09:07:08,785][20347] Updated weights for policy 0, policy_version 502557 (0.0007) [2023-12-27 09:07:09,368][20381] Updated weights for policy 1, policy_version 502298 (0.0007) [2023-12-27 09:07:09,430][20381] Updated weights for policy 1, policy_version 502308 (0.0008) [2023-12-27 09:07:09,469][20347] Updated weights for policy 0, policy_version 502567 (0.0008) [2023-12-27 09:07:09,497][20381] Updated weights for policy 1, policy_version 502318 (0.0008) [2023-12-27 09:07:09,531][20347] Updated weights for policy 0, policy_version 502577 (0.0006) [2023-12-27 09:07:09,592][20347] Updated weights for policy 0, policy_version 502587 (0.0008) [2023-12-27 09:07:10,202][20381] Updated weights for policy 1, policy_version 502328 (0.0007) [2023-12-27 09:07:10,265][20381] Updated weights for policy 1, policy_version 502338 (0.0006) [2023-12-27 09:07:10,318][20347] Updated weights for policy 0, policy_version 502597 (0.0007) [2023-12-27 09:07:10,324][20381] Updated weights for policy 1, policy_version 502348 (0.0006) [2023-12-27 09:07:10,372][20347] Updated weights for policy 0, policy_version 502607 (0.0007) [2023-12-27 09:07:10,424][20347] Updated weights for policy 0, policy_version 502617 (0.0008) [2023-12-27 09:07:10,927][20381] Updated weights for policy 1, policy_version 502358 (0.0006) [2023-12-27 09:07:10,979][20381] Updated weights for policy 1, policy_version 502368 (0.0005) [2023-12-27 09:07:11,045][20381] Updated weights for policy 1, policy_version 502378 (0.0006) [2023-12-27 09:07:11,177][20347] Updated weights for policy 0, policy_version 502627 (0.0008) [2023-12-27 09:07:11,228][20347] Updated weights for policy 0, policy_version 502637 (0.0009) [2023-12-27 09:07:11,290][20347] Updated weights for policy 0, policy_version 502647 (0.0011) [2023-12-27 09:07:11,754][20381] Updated weights for policy 1, policy_version 502388 (0.0009) [2023-12-27 09:07:11,814][20381] Updated weights for policy 1, policy_version 502398 (0.0011) [2023-12-27 09:07:11,872][20381] Updated weights for policy 1, policy_version 502408 (0.0010) [2023-12-27 09:07:12,036][20347] Updated weights for policy 0, policy_version 502657 (0.0007) [2023-12-27 09:07:12,104][20347] Updated weights for policy 0, policy_version 502667 (0.0008) [2023-12-27 09:07:12,165][20347] Updated weights for policy 0, policy_version 502677 (0.0007) [2023-12-27 09:07:12,221][20347] Updated weights for policy 0, policy_version 502687 (0.0008) [2023-12-27 09:07:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 257343488. Throughput: 0: 10200.7, 1: 10564.3. Samples: 257333036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:12,363][19170] Avg episode reward: [(0, '942.580'), (1, '912.720')] [2023-12-27 09:07:12,554][20381] Updated weights for policy 1, policy_version 502418 (0.0010) [2023-12-27 09:07:12,608][20381] Updated weights for policy 1, policy_version 502428 (0.0010) [2023-12-27 09:07:12,657][20381] Updated weights for policy 1, policy_version 502438 (0.0010) [2023-12-27 09:07:12,709][20381] Updated weights for policy 1, policy_version 502448 (0.0010) [2023-12-27 09:07:12,866][20347] Updated weights for policy 0, policy_version 502697 (0.0006) [2023-12-27 09:07:12,912][20347] Updated weights for policy 0, policy_version 502707 (0.0007) [2023-12-27 09:07:12,966][20347] Updated weights for policy 0, policy_version 502717 (0.0008) [2023-12-27 09:07:13,379][20381] Updated weights for policy 1, policy_version 502458 (0.0011) [2023-12-27 09:07:13,438][20381] Updated weights for policy 1, policy_version 502468 (0.0011) [2023-12-27 09:07:13,493][20381] Updated weights for policy 1, policy_version 502478 (0.0010) [2023-12-27 09:07:13,599][20347] Updated weights for policy 0, policy_version 502727 (0.0007) [2023-12-27 09:07:13,650][20347] Updated weights for policy 0, policy_version 502737 (0.0007) [2023-12-27 09:07:13,704][20347] Updated weights for policy 0, policy_version 502747 (0.0007) [2023-12-27 09:07:14,167][20381] Updated weights for policy 1, policy_version 502488 (0.0010) [2023-12-27 09:07:14,222][20381] Updated weights for policy 1, policy_version 502498 (0.0010) [2023-12-27 09:07:14,267][20381] Updated weights for policy 1, policy_version 502508 (0.0005) [2023-12-27 09:07:14,399][20347] Updated weights for policy 0, policy_version 502757 (0.0006) [2023-12-27 09:07:14,459][20347] Updated weights for policy 0, policy_version 502767 (0.0006) [2023-12-27 09:07:14,521][20347] Updated weights for policy 0, policy_version 502777 (0.0007) [2023-12-27 09:07:14,919][20381] Updated weights for policy 1, policy_version 502518 (0.0008) [2023-12-27 09:07:14,968][20381] Updated weights for policy 1, policy_version 502528 (0.0009) [2023-12-27 09:07:15,022][20381] Updated weights for policy 1, policy_version 502538 (0.0009) [2023-12-27 09:07:15,150][20347] Updated weights for policy 0, policy_version 502787 (0.0008) [2023-12-27 09:07:15,210][20347] Updated weights for policy 0, policy_version 502797 (0.0006) [2023-12-27 09:07:15,265][20347] Updated weights for policy 0, policy_version 502807 (0.0008) [2023-12-27 09:07:15,783][20381] Updated weights for policy 1, policy_version 502548 (0.0008) [2023-12-27 09:07:15,833][20381] Updated weights for policy 1, policy_version 502558 (0.0009) [2023-12-27 09:07:15,854][20347] Updated weights for policy 0, policy_version 502817 (0.0008) [2023-12-27 09:07:15,889][20381] Updated weights for policy 1, policy_version 502568 (0.0011) [2023-12-27 09:07:15,915][20347] Updated weights for policy 0, policy_version 502827 (0.0010) [2023-12-27 09:07:15,966][20347] Updated weights for policy 0, policy_version 502837 (0.0010) [2023-12-27 09:07:16,018][20347] Updated weights for policy 0, policy_version 502847 (0.0009) [2023-12-27 09:07:16,497][20381] Updated weights for policy 1, policy_version 502578 (0.0008) [2023-12-27 09:07:16,549][20381] Updated weights for policy 1, policy_version 502588 (0.0006) [2023-12-27 09:07:16,601][20381] Updated weights for policy 1, policy_version 502598 (0.0007) [2023-12-27 09:07:16,623][20347] Updated weights for policy 0, policy_version 502857 (0.0010) [2023-12-27 09:07:16,655][20381] Updated weights for policy 1, policy_version 502608 (0.0008) [2023-12-27 09:07:16,681][20347] Updated weights for policy 0, policy_version 502867 (0.0010) [2023-12-27 09:07:16,738][20347] Updated weights for policy 0, policy_version 502877 (0.0010) [2023-12-27 09:07:17,326][20381] Updated weights for policy 1, policy_version 502618 (0.0010) [2023-12-27 09:07:17,347][20347] Updated weights for policy 0, policy_version 502887 (0.0007) [2023-12-27 09:07:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 257441792. Throughput: 0: 10211.9, 1: 10492.9. Samples: 257458164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:17,363][19170] Avg episode reward: [(0, '951.720'), (1, '922.520')] [2023-12-27 09:07:17,371][20381] Updated weights for policy 1, policy_version 502628 (0.0010) [2023-12-27 09:07:17,413][20347] Updated weights for policy 0, policy_version 502897 (0.0007) [2023-12-27 09:07:17,427][20381] Updated weights for policy 1, policy_version 502638 (0.0007) [2023-12-27 09:07:17,473][20347] Updated weights for policy 0, policy_version 502907 (0.0010) [2023-12-27 09:07:18,099][20381] Updated weights for policy 1, policy_version 502648 (0.0009) [2023-12-27 09:07:18,109][20347] Updated weights for policy 0, policy_version 502917 (0.0008) [2023-12-27 09:07:18,158][20381] Updated weights for policy 1, policy_version 502658 (0.0010) [2023-12-27 09:07:18,169][20347] Updated weights for policy 0, policy_version 502927 (0.0008) [2023-12-27 09:07:18,217][20381] Updated weights for policy 1, policy_version 502668 (0.0011) [2023-12-27 09:07:18,231][20347] Updated weights for policy 0, policy_version 502937 (0.0008) [2023-12-27 09:07:18,855][20347] Updated weights for policy 0, policy_version 502947 (0.0008) [2023-12-27 09:07:18,907][20347] Updated weights for policy 0, policy_version 502957 (0.0005) [2023-12-27 09:07:18,912][20381] Updated weights for policy 1, policy_version 502678 (0.0011) [2023-12-27 09:07:18,956][20347] Updated weights for policy 0, policy_version 502967 (0.0008) [2023-12-27 09:07:18,967][20381] Updated weights for policy 1, policy_version 502688 (0.0010) [2023-12-27 09:07:19,018][20381] Updated weights for policy 1, policy_version 502698 (0.0010) [2023-12-27 09:07:19,634][20347] Updated weights for policy 0, policy_version 502977 (0.0006) [2023-12-27 09:07:19,683][20381] Updated weights for policy 1, policy_version 502708 (0.0010) [2023-12-27 09:07:19,697][20347] Updated weights for policy 0, policy_version 502987 (0.0007) [2023-12-27 09:07:19,739][20381] Updated weights for policy 1, policy_version 502718 (0.0010) [2023-12-27 09:07:19,752][20347] Updated weights for policy 0, policy_version 502997 (0.0007) [2023-12-27 09:07:19,795][20381] Updated weights for policy 1, policy_version 502728 (0.0010) [2023-12-27 09:07:19,808][20347] Updated weights for policy 0, policy_version 503007 (0.0005) [2023-12-27 09:07:20,471][20381] Updated weights for policy 1, policy_version 502738 (0.0009) [2023-12-27 09:07:20,529][20381] Updated weights for policy 1, policy_version 502748 (0.0010) [2023-12-27 09:07:20,534][20347] Updated weights for policy 0, policy_version 503017 (0.0005) [2023-12-27 09:07:20,590][20381] Updated weights for policy 1, policy_version 502758 (0.0009) [2023-12-27 09:07:20,594][20347] Updated weights for policy 0, policy_version 503027 (0.0008) [2023-12-27 09:07:20,644][20381] Updated weights for policy 1, policy_version 502768 (0.0010) [2023-12-27 09:07:20,646][20347] Updated weights for policy 0, policy_version 503037 (0.0005) [2023-12-27 09:07:21,276][20381] Updated weights for policy 1, policy_version 502778 (0.0011) [2023-12-27 09:07:21,332][20381] Updated weights for policy 1, policy_version 502788 (0.0010) [2023-12-27 09:07:21,379][20347] Updated weights for policy 0, policy_version 503047 (0.0008) [2023-12-27 09:07:21,396][20381] Updated weights for policy 1, policy_version 502798 (0.0008) [2023-12-27 09:07:21,443][20347] Updated weights for policy 0, policy_version 503057 (0.0006) [2023-12-27 09:07:21,501][20347] Updated weights for policy 0, policy_version 503067 (0.0005) [2023-12-27 09:07:22,085][20381] Updated weights for policy 1, policy_version 502808 (0.0008) [2023-12-27 09:07:22,144][20381] Updated weights for policy 1, policy_version 502818 (0.0006) [2023-12-27 09:07:22,175][20347] Updated weights for policy 0, policy_version 503077 (0.0006) [2023-12-27 09:07:22,202][20381] Updated weights for policy 1, policy_version 502828 (0.0006) [2023-12-27 09:07:22,224][20347] Updated weights for policy 0, policy_version 503087 (0.0007) [2023-12-27 09:07:22,290][20347] Updated weights for policy 0, policy_version 503097 (0.0008) [2023-12-27 09:07:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 257556480. Throughput: 0: 10246.6, 1: 10411.7. Samples: 257520420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:22,363][19170] Avg episode reward: [(0, '933.860'), (1, '922.550')] [2023-12-27 09:07:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000502832_128745472.pth... [2023-12-27 09:07:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000503104_128811008.pth... [2023-12-27 09:07:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000501616_128434176.pth [2023-12-27 09:07:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000501888_128499712.pth [2023-12-27 09:07:22,828][20381] Updated weights for policy 1, policy_version 502838 (0.0007) [2023-12-27 09:07:22,888][20381] Updated weights for policy 1, policy_version 502848 (0.0007) [2023-12-27 09:07:22,950][20381] Updated weights for policy 1, policy_version 502858 (0.0006) [2023-12-27 09:07:22,975][20347] Updated weights for policy 0, policy_version 503107 (0.0008) [2023-12-27 09:07:23,029][20347] Updated weights for policy 0, policy_version 503117 (0.0005) [2023-12-27 09:07:23,088][20347] Updated weights for policy 0, policy_version 503127 (0.0006) [2023-12-27 09:07:23,644][20381] Updated weights for policy 1, policy_version 502868 (0.0009) [2023-12-27 09:07:23,661][20347] Updated weights for policy 0, policy_version 503137 (0.0005) [2023-12-27 09:07:23,697][20381] Updated weights for policy 1, policy_version 502878 (0.0008) [2023-12-27 09:07:23,712][20347] Updated weights for policy 0, policy_version 503147 (0.0005) [2023-12-27 09:07:23,758][20381] Updated weights for policy 1, policy_version 502888 (0.0006) [2023-12-27 09:07:23,758][20347] Updated weights for policy 0, policy_version 503157 (0.0008) [2023-12-27 09:07:23,807][20347] Updated weights for policy 0, policy_version 503167 (0.0010) [2023-12-27 09:07:24,398][20381] Updated weights for policy 1, policy_version 502898 (0.0007) [2023-12-27 09:07:24,453][20381] Updated weights for policy 1, policy_version 502908 (0.0007) [2023-12-27 09:07:24,457][20347] Updated weights for policy 0, policy_version 503177 (0.0011) [2023-12-27 09:07:24,511][20381] Updated weights for policy 1, policy_version 502918 (0.0006) [2023-12-27 09:07:24,513][20347] Updated weights for policy 0, policy_version 503187 (0.0010) [2023-12-27 09:07:24,560][20381] Updated weights for policy 1, policy_version 502928 (0.0006) [2023-12-27 09:07:24,569][20347] Updated weights for policy 0, policy_version 503197 (0.0010) [2023-12-27 09:07:25,248][20347] Updated weights for policy 0, policy_version 503207 (0.0010) [2023-12-27 09:07:25,261][20381] Updated weights for policy 1, policy_version 502938 (0.0006) [2023-12-27 09:07:25,283][20291] KL-divergence is very high: 126.3311 [2023-12-27 09:07:25,292][20291] KL-divergence is very high: 122.8971 [2023-12-27 09:07:25,294][20347] Updated weights for policy 0, policy_version 503217 (0.0010) [2023-12-27 09:07:25,313][20381] Updated weights for policy 1, policy_version 502948 (0.0007) [2023-12-27 09:07:25,343][20347] Updated weights for policy 0, policy_version 503227 (0.0010) [2023-12-27 09:07:25,364][20381] Updated weights for policy 1, policy_version 502958 (0.0006) [2023-12-27 09:07:25,367][20291] KL-divergence is very high: 100.1924 [2023-12-27 09:07:26,000][20381] Updated weights for policy 1, policy_version 502968 (0.0005) [2023-12-27 09:07:26,061][20381] Updated weights for policy 1, policy_version 502978 (0.0005) [2023-12-27 09:07:26,067][20347] Updated weights for policy 0, policy_version 503237 (0.0010) [2023-12-27 09:07:26,111][20381] Updated weights for policy 1, policy_version 502988 (0.0006) [2023-12-27 09:07:26,121][20347] Updated weights for policy 0, policy_version 503247 (0.0008) [2023-12-27 09:07:26,177][20347] Updated weights for policy 0, policy_version 503257 (0.0009) [2023-12-27 09:07:26,702][20381] Updated weights for policy 1, policy_version 502998 (0.0008) [2023-12-27 09:07:26,747][20381] Updated weights for policy 1, policy_version 503008 (0.0009) [2023-12-27 09:07:26,799][20381] Updated weights for policy 1, policy_version 503018 (0.0008) [2023-12-27 09:07:26,839][20347] Updated weights for policy 0, policy_version 503267 (0.0007) [2023-12-27 09:07:26,889][20347] Updated weights for policy 0, policy_version 503277 (0.0006) [2023-12-27 09:07:26,939][20347] Updated weights for policy 0, policy_version 503287 (0.0005) [2023-12-27 09:07:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 257654784. Throughput: 0: 10245.1, 1: 10444.9. Samples: 257644832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:27,363][19170] Avg episode reward: [(0, '847.030'), (1, '941.780')] [2023-12-27 09:07:27,548][20381] Updated weights for policy 1, policy_version 503028 (0.0008) [2023-12-27 09:07:27,558][20347] Updated weights for policy 0, policy_version 503297 (0.0007) [2023-12-27 09:07:27,593][20381] Updated weights for policy 1, policy_version 503038 (0.0006) [2023-12-27 09:07:27,606][20347] Updated weights for policy 0, policy_version 503307 (0.0010) [2023-12-27 09:07:27,652][20381] Updated weights for policy 1, policy_version 503048 (0.0005) [2023-12-27 09:07:27,664][20347] Updated weights for policy 0, policy_version 503317 (0.0010) [2023-12-27 09:07:27,716][20347] Updated weights for policy 0, policy_version 503327 (0.0010) [2023-12-27 09:07:28,227][20381] Updated weights for policy 1, policy_version 503058 (0.0005) [2023-12-27 09:07:28,277][20381] Updated weights for policy 1, policy_version 503068 (0.0007) [2023-12-27 09:07:28,326][20381] Updated weights for policy 1, policy_version 503078 (0.0006) [2023-12-27 09:07:28,359][20347] Updated weights for policy 0, policy_version 503337 (0.0010) [2023-12-27 09:07:28,379][20381] Updated weights for policy 1, policy_version 503088 (0.0008) [2023-12-27 09:07:28,416][20347] Updated weights for policy 0, policy_version 503347 (0.0009) [2023-12-27 09:07:28,465][20347] Updated weights for policy 0, policy_version 503357 (0.0010) [2023-12-27 09:07:29,093][20347] Updated weights for policy 0, policy_version 503367 (0.0008) [2023-12-27 09:07:29,105][20381] Updated weights for policy 1, policy_version 503098 (0.0007) [2023-12-27 09:07:29,143][20347] Updated weights for policy 0, policy_version 503377 (0.0010) [2023-12-27 09:07:29,159][20381] Updated weights for policy 1, policy_version 503108 (0.0005) [2023-12-27 09:07:29,191][20347] Updated weights for policy 0, policy_version 503387 (0.0010) [2023-12-27 09:07:29,212][20381] Updated weights for policy 1, policy_version 503118 (0.0005) [2023-12-27 09:07:29,771][20347] Updated weights for policy 0, policy_version 503397 (0.0010) [2023-12-27 09:07:29,819][20347] Updated weights for policy 0, policy_version 503407 (0.0010) [2023-12-27 09:07:29,882][20347] Updated weights for policy 0, policy_version 503417 (0.0008) [2023-12-27 09:07:29,934][20381] Updated weights for policy 1, policy_version 503128 (0.0010) [2023-12-27 09:07:29,994][20381] Updated weights for policy 1, policy_version 503138 (0.0009) [2023-12-27 09:07:30,061][20381] Updated weights for policy 1, policy_version 503148 (0.0010) [2023-12-27 09:07:30,539][20347] Updated weights for policy 0, policy_version 503427 (0.0007) [2023-12-27 09:07:30,591][20347] Updated weights for policy 0, policy_version 503437 (0.0010) [2023-12-27 09:07:30,649][20347] Updated weights for policy 0, policy_version 503447 (0.0010) [2023-12-27 09:07:30,714][20381] Updated weights for policy 1, policy_version 503158 (0.0011) [2023-12-27 09:07:30,767][20381] Updated weights for policy 1, policy_version 503168 (0.0010) [2023-12-27 09:07:30,823][20381] Updated weights for policy 1, policy_version 503178 (0.0010) [2023-12-27 09:07:31,246][20347] Updated weights for policy 0, policy_version 503457 (0.0010) [2023-12-27 09:07:31,309][20347] Updated weights for policy 0, policy_version 503467 (0.0008) [2023-12-27 09:07:31,371][20347] Updated weights for policy 0, policy_version 503477 (0.0008) [2023-12-27 09:07:31,427][20347] Updated weights for policy 0, policy_version 503487 (0.0005) [2023-12-27 09:07:31,516][20381] Updated weights for policy 1, policy_version 503188 (0.0009) [2023-12-27 09:07:31,573][20381] Updated weights for policy 1, policy_version 503198 (0.0005) [2023-12-27 09:07:31,635][20381] Updated weights for policy 1, policy_version 503208 (0.0006) [2023-12-27 09:07:32,085][20347] Updated weights for policy 0, policy_version 503497 (0.0010) [2023-12-27 09:07:32,144][20347] Updated weights for policy 0, policy_version 503507 (0.0010) [2023-12-27 09:07:32,195][20347] Updated weights for policy 0, policy_version 503517 (0.0010) [2023-12-27 09:07:32,288][20381] Updated weights for policy 1, policy_version 503218 (0.0009) [2023-12-27 09:07:32,334][20381] Updated weights for policy 1, policy_version 503228 (0.0009) [2023-12-27 09:07:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 257761280. Throughput: 0: 10337.3, 1: 10374.0. Samples: 257772020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:32,363][19170] Avg episode reward: [(0, '876.810'), (1, '932.290')] [2023-12-27 09:07:32,396][20381] Updated weights for policy 1, policy_version 503238 (0.0006) [2023-12-27 09:07:32,448][20381] Updated weights for policy 1, policy_version 503248 (0.0005) [2023-12-27 09:07:32,832][20347] Updated weights for policy 0, policy_version 503527 (0.0010) [2023-12-27 09:07:32,833][20291] KL-divergence is very high: 155.1321 [2023-12-27 09:07:32,846][20291] KL-divergence is very high: 125.2721 [2023-12-27 09:07:32,896][20347] Updated weights for policy 0, policy_version 503537 (0.0010) [2023-12-27 09:07:32,938][20291] KL-divergence is very high: 548.3943 [2023-12-27 09:07:32,950][20291] KL-divergence is very high: 391.8084 [2023-12-27 09:07:32,966][20347] Updated weights for policy 0, policy_version 503547 (0.0010) [2023-12-27 09:07:33,118][20381] Updated weights for policy 1, policy_version 503258 (0.0005) [2023-12-27 09:07:33,174][20381] Updated weights for policy 1, policy_version 503268 (0.0005) [2023-12-27 09:07:33,236][20381] Updated weights for policy 1, policy_version 503278 (0.0005) [2023-12-27 09:07:33,520][20347] Updated weights for policy 0, policy_version 503557 (0.0010) [2023-12-27 09:07:33,568][20347] Updated weights for policy 0, policy_version 503567 (0.0010) [2023-12-27 09:07:33,622][20347] Updated weights for policy 0, policy_version 503577 (0.0010) [2023-12-27 09:07:33,855][20381] Updated weights for policy 1, policy_version 503288 (0.0006) [2023-12-27 09:07:33,910][20381] Updated weights for policy 1, policy_version 503298 (0.0005) [2023-12-27 09:07:33,967][20381] Updated weights for policy 1, policy_version 503308 (0.0005) [2023-12-27 09:07:34,308][20347] Updated weights for policy 0, policy_version 503587 (0.0010) [2023-12-27 09:07:34,370][20347] Updated weights for policy 0, policy_version 503597 (0.0010) [2023-12-27 09:07:34,438][20347] Updated weights for policy 0, policy_version 503607 (0.0010) [2023-12-27 09:07:34,611][20381] Updated weights for policy 1, policy_version 503318 (0.0005) [2023-12-27 09:07:34,673][20381] Updated weights for policy 1, policy_version 503328 (0.0005) [2023-12-27 09:07:34,731][20381] Updated weights for policy 1, policy_version 503338 (0.0007) [2023-12-27 09:07:35,113][20347] Updated weights for policy 0, policy_version 503617 (0.0011) [2023-12-27 09:07:35,160][20347] Updated weights for policy 0, policy_version 503627 (0.0010) [2023-12-27 09:07:35,215][20347] Updated weights for policy 0, policy_version 503637 (0.0010) [2023-12-27 09:07:35,273][20347] Updated weights for policy 0, policy_version 503647 (0.0008) [2023-12-27 09:07:35,401][20381] Updated weights for policy 1, policy_version 503348 (0.0007) [2023-12-27 09:07:35,458][20381] Updated weights for policy 1, policy_version 503358 (0.0008) [2023-12-27 09:07:35,519][20381] Updated weights for policy 1, policy_version 503368 (0.0007) [2023-12-27 09:07:35,900][20347] Updated weights for policy 0, policy_version 503657 (0.0010) [2023-12-27 09:07:35,954][20347] Updated weights for policy 0, policy_version 503667 (0.0010) [2023-12-27 09:07:36,009][20347] Updated weights for policy 0, policy_version 503677 (0.0007) [2023-12-27 09:07:36,211][20381] Updated weights for policy 1, policy_version 503378 (0.0008) [2023-12-27 09:07:36,265][20381] Updated weights for policy 1, policy_version 503388 (0.0009) [2023-12-27 09:07:36,319][20381] Updated weights for policy 1, policy_version 503398 (0.0006) [2023-12-27 09:07:36,376][20381] Updated weights for policy 1, policy_version 503408 (0.0005) [2023-12-27 09:07:36,675][20347] Updated weights for policy 0, policy_version 503687 (0.0009) [2023-12-27 09:07:36,740][20347] Updated weights for policy 0, policy_version 503697 (0.0006) [2023-12-27 09:07:36,805][20347] Updated weights for policy 0, policy_version 503707 (0.0011) [2023-12-27 09:07:37,030][20381] Updated weights for policy 1, policy_version 503418 (0.0006) [2023-12-27 09:07:37,079][20381] Updated weights for policy 1, policy_version 503428 (0.0005) [2023-12-27 09:07:37,121][20292] KL-divergence is very high: 104.6244 [2023-12-27 09:07:37,136][20381] Updated weights for policy 1, policy_version 503438 (0.0008) [2023-12-27 09:07:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 257867776. Throughput: 0: 10387.7, 1: 10377.6. Samples: 257836140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:37,363][19170] Avg episode reward: [(0, '928.740'), (1, '893.780')] [2023-12-27 09:07:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000503440_128901120.pth... [2023-12-27 09:07:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000503712_128966656.pth... [2023-12-27 09:07:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000502496_128655360.pth [2023-12-27 09:07:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000502224_128589824.pth [2023-12-27 09:07:37,464][20347] Updated weights for policy 0, policy_version 503717 (0.0010) [2023-12-27 09:07:37,526][20347] Updated weights for policy 0, policy_version 503727 (0.0011) [2023-12-27 09:07:37,580][20347] Updated weights for policy 0, policy_version 503737 (0.0010) [2023-12-27 09:07:37,755][20381] Updated weights for policy 1, policy_version 503448 (0.0006) [2023-12-27 09:07:37,808][20381] Updated weights for policy 1, policy_version 503458 (0.0006) [2023-12-27 09:07:37,860][20381] Updated weights for policy 1, policy_version 503468 (0.0005) [2023-12-27 09:07:38,241][20347] Updated weights for policy 0, policy_version 503747 (0.0008) [2023-12-27 09:07:38,290][20347] Updated weights for policy 0, policy_version 503757 (0.0010) [2023-12-27 09:07:38,348][20347] Updated weights for policy 0, policy_version 503767 (0.0010) [2023-12-27 09:07:38,433][20381] Updated weights for policy 1, policy_version 503478 (0.0007) [2023-12-27 09:07:38,496][20381] Updated weights for policy 1, policy_version 503488 (0.0008) [2023-12-27 09:07:38,562][20381] Updated weights for policy 1, policy_version 503498 (0.0007) [2023-12-27 09:07:39,059][20347] Updated weights for policy 0, policy_version 503777 (0.0010) [2023-12-27 09:07:39,124][20347] Updated weights for policy 0, policy_version 503787 (0.0010) [2023-12-27 09:07:39,188][20347] Updated weights for policy 0, policy_version 503797 (0.0010) [2023-12-27 09:07:39,199][20381] Updated weights for policy 1, policy_version 503508 (0.0008) [2023-12-27 09:07:39,254][20347] Updated weights for policy 0, policy_version 503807 (0.0008) [2023-12-27 09:07:39,262][20381] Updated weights for policy 1, policy_version 503518 (0.0011) [2023-12-27 09:07:39,315][20381] Updated weights for policy 1, policy_version 503528 (0.0011) [2023-12-27 09:07:39,939][20347] Updated weights for policy 0, policy_version 503817 (0.0008) [2023-12-27 09:07:39,987][20381] Updated weights for policy 1, policy_version 503538 (0.0010) [2023-12-27 09:07:40,003][20347] Updated weights for policy 0, policy_version 503827 (0.0008) [2023-12-27 09:07:40,043][20381] Updated weights for policy 1, policy_version 503548 (0.0006) [2023-12-27 09:07:40,068][20347] Updated weights for policy 0, policy_version 503837 (0.0008) [2023-12-27 09:07:40,101][20381] Updated weights for policy 1, policy_version 503558 (0.0006) [2023-12-27 09:07:40,161][20381] Updated weights for policy 1, policy_version 503568 (0.0006) [2023-12-27 09:07:40,710][20381] Updated weights for policy 1, policy_version 503578 (0.0006) [2023-12-27 09:07:40,765][20347] Updated weights for policy 0, policy_version 503847 (0.0007) [2023-12-27 09:07:40,769][20381] Updated weights for policy 1, policy_version 503588 (0.0005) [2023-12-27 09:07:40,823][20347] Updated weights for policy 0, policy_version 503857 (0.0008) [2023-12-27 09:07:40,826][20381] Updated weights for policy 1, policy_version 503598 (0.0006) [2023-12-27 09:07:40,874][20347] Updated weights for policy 0, policy_version 503867 (0.0007) [2023-12-27 09:07:41,470][20381] Updated weights for policy 1, policy_version 503608 (0.0006) [2023-12-27 09:07:41,529][20381] Updated weights for policy 1, policy_version 503618 (0.0008) [2023-12-27 09:07:41,593][20381] Updated weights for policy 1, policy_version 503628 (0.0008) [2023-12-27 09:07:41,603][20347] Updated weights for policy 0, policy_version 503877 (0.0007) [2023-12-27 09:07:41,668][20347] Updated weights for policy 0, policy_version 503887 (0.0008) [2023-12-27 09:07:41,735][20347] Updated weights for policy 0, policy_version 503897 (0.0007) [2023-12-27 09:07:42,240][20381] Updated weights for policy 1, policy_version 503638 (0.0007) [2023-12-27 09:07:42,297][20381] Updated weights for policy 1, policy_version 503648 (0.0007) [2023-12-27 09:07:42,353][20347] Updated weights for policy 0, policy_version 503907 (0.0007) [2023-12-27 09:07:42,361][20381] Updated weights for policy 1, policy_version 503658 (0.0008) [2023-12-27 09:07:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 257966080. Throughput: 0: 10345.4, 1: 10493.3. Samples: 257961880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:42,363][19170] Avg episode reward: [(0, '947.850'), (1, '942.120')] [2023-12-27 09:07:42,408][20347] Updated weights for policy 0, policy_version 503917 (0.0008) [2023-12-27 09:07:42,460][20347] Updated weights for policy 0, policy_version 503927 (0.0008) [2023-12-27 09:07:43,051][20381] Updated weights for policy 1, policy_version 503668 (0.0007) [2023-12-27 09:07:43,098][20381] Updated weights for policy 1, policy_version 503678 (0.0007) [2023-12-27 09:07:43,145][20381] Updated weights for policy 1, policy_version 503688 (0.0007) [2023-12-27 09:07:43,171][20347] Updated weights for policy 0, policy_version 503937 (0.0007) [2023-12-27 09:07:43,218][20347] Updated weights for policy 0, policy_version 503947 (0.0009) [2023-12-27 09:07:43,270][20347] Updated weights for policy 0, policy_version 503957 (0.0010) [2023-12-27 09:07:43,321][20347] Updated weights for policy 0, policy_version 503967 (0.0010) [2023-12-27 09:07:43,836][20381] Updated weights for policy 1, policy_version 503698 (0.0007) [2023-12-27 09:07:43,886][20381] Updated weights for policy 1, policy_version 503708 (0.0007) [2023-12-27 09:07:43,934][20381] Updated weights for policy 1, policy_version 503718 (0.0008) [2023-12-27 09:07:43,943][20292] KL-divergence is very high: 263.0997 [2023-12-27 09:07:43,963][20292] KL-divergence is very high: 252.2953 [2023-12-27 09:07:43,982][20347] Updated weights for policy 0, policy_version 503977 (0.0010) [2023-12-27 09:07:43,985][20381] Updated weights for policy 1, policy_version 503728 (0.0009) [2023-12-27 09:07:44,041][20347] Updated weights for policy 0, policy_version 503987 (0.0010) [2023-12-27 09:07:44,101][20347] Updated weights for policy 0, policy_version 503997 (0.0010) [2023-12-27 09:07:44,690][20381] Updated weights for policy 1, policy_version 503738 (0.0005) [2023-12-27 09:07:44,740][20381] Updated weights for policy 1, policy_version 503748 (0.0007) [2023-12-27 09:07:44,798][20347] Updated weights for policy 0, policy_version 504007 (0.0009) [2023-12-27 09:07:44,800][20381] Updated weights for policy 1, policy_version 503758 (0.0007) [2023-12-27 09:07:44,857][20347] Updated weights for policy 0, policy_version 504017 (0.0008) [2023-12-27 09:07:44,917][20347] Updated weights for policy 0, policy_version 504027 (0.0007) [2023-12-27 09:07:45,479][20381] Updated weights for policy 1, policy_version 503768 (0.0007) [2023-12-27 09:07:45,540][20381] Updated weights for policy 1, policy_version 503778 (0.0006) [2023-12-27 09:07:45,581][20347] Updated weights for policy 0, policy_version 504037 (0.0006) [2023-12-27 09:07:45,600][20381] Updated weights for policy 1, policy_version 503788 (0.0007) [2023-12-27 09:07:45,628][20347] Updated weights for policy 0, policy_version 504047 (0.0007) [2023-12-27 09:07:45,678][20291] KL-divergence is very high: 104.6220 [2023-12-27 09:07:45,684][20347] Updated weights for policy 0, policy_version 504057 (0.0008) [2023-12-27 09:07:46,242][20381] Updated weights for policy 1, policy_version 503798 (0.0007) [2023-12-27 09:07:46,289][20381] Updated weights for policy 1, policy_version 503808 (0.0007) [2023-12-27 09:07:46,298][20347] Updated weights for policy 0, policy_version 504067 (0.0008) [2023-12-27 09:07:46,350][20347] Updated weights for policy 0, policy_version 504077 (0.0009) [2023-12-27 09:07:46,350][20381] Updated weights for policy 1, policy_version 503818 (0.0007) [2023-12-27 09:07:46,413][20347] Updated weights for policy 0, policy_version 504087 (0.0006) [2023-12-27 09:07:47,030][20347] Updated weights for policy 0, policy_version 504097 (0.0005) [2023-12-27 09:07:47,061][20381] Updated weights for policy 1, policy_version 503828 (0.0007) [2023-12-27 09:07:47,084][20347] Updated weights for policy 0, policy_version 504107 (0.0010) [2023-12-27 09:07:47,115][20381] Updated weights for policy 1, policy_version 503838 (0.0005) [2023-12-27 09:07:47,139][20347] Updated weights for policy 0, policy_version 504117 (0.0010) [2023-12-27 09:07:47,160][20381] Updated weights for policy 1, policy_version 503848 (0.0006) [2023-12-27 09:07:47,192][20347] Updated weights for policy 0, policy_version 504127 (0.0009) [2023-12-27 09:07:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 258080768. Throughput: 0: 10425.6, 1: 10477.5. Samples: 258085856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:47,364][19170] Avg episode reward: [(0, '960.010'), (1, '969.250')] [2023-12-27 09:07:47,760][20381] Updated weights for policy 1, policy_version 503858 (0.0006) [2023-12-27 09:07:47,815][20381] Updated weights for policy 1, policy_version 503868 (0.0006) [2023-12-27 09:07:47,835][20347] Updated weights for policy 0, policy_version 504137 (0.0006) [2023-12-27 09:07:47,874][20381] Updated weights for policy 1, policy_version 503878 (0.0007) [2023-12-27 09:07:47,888][20347] Updated weights for policy 0, policy_version 504147 (0.0005) [2023-12-27 09:07:47,936][20381] Updated weights for policy 1, policy_version 503888 (0.0008) [2023-12-27 09:07:47,945][20347] Updated weights for policy 0, policy_version 504157 (0.0007) [2023-12-27 09:07:48,571][20381] Updated weights for policy 1, policy_version 503898 (0.0008) [2023-12-27 09:07:48,629][20381] Updated weights for policy 1, policy_version 503908 (0.0010) [2023-12-27 09:07:48,675][20347] Updated weights for policy 0, policy_version 504167 (0.0007) [2023-12-27 09:07:48,682][20381] Updated weights for policy 1, policy_version 503918 (0.0010) [2023-12-27 09:07:48,733][20347] Updated weights for policy 0, policy_version 504177 (0.0006) [2023-12-27 09:07:48,790][20347] Updated weights for policy 0, policy_version 504187 (0.0007) [2023-12-27 09:07:49,341][20381] Updated weights for policy 1, policy_version 503928 (0.0010) [2023-12-27 09:07:49,397][20381] Updated weights for policy 1, policy_version 503938 (0.0010) [2023-12-27 09:07:49,406][20347] Updated weights for policy 0, policy_version 504197 (0.0006) [2023-12-27 09:07:49,450][20381] Updated weights for policy 1, policy_version 503948 (0.0010) [2023-12-27 09:07:49,462][20347] Updated weights for policy 0, policy_version 504207 (0.0006) [2023-12-27 09:07:49,518][20347] Updated weights for policy 0, policy_version 504217 (0.0007) [2023-12-27 09:07:50,132][20381] Updated weights for policy 1, policy_version 503958 (0.0010) [2023-12-27 09:07:50,216][20381] Updated weights for policy 1, policy_version 503968 (0.0011) [2023-12-27 09:07:50,262][20347] Updated weights for policy 0, policy_version 504227 (0.0007) [2023-12-27 09:07:50,282][20381] Updated weights for policy 1, policy_version 503978 (0.0011) [2023-12-27 09:07:50,321][20347] Updated weights for policy 0, policy_version 504237 (0.0006) [2023-12-27 09:07:50,370][20347] Updated weights for policy 0, policy_version 504247 (0.0008) [2023-12-27 09:07:50,816][20381] Updated weights for policy 1, policy_version 503988 (0.0010) [2023-12-27 09:07:50,877][20381] Updated weights for policy 1, policy_version 503998 (0.0010) [2023-12-27 09:07:50,938][20381] Updated weights for policy 1, policy_version 504008 (0.0009) [2023-12-27 09:07:51,163][20347] Updated weights for policy 0, policy_version 504257 (0.0007) [2023-12-27 09:07:51,214][20347] Updated weights for policy 0, policy_version 504267 (0.0007) [2023-12-27 09:07:51,281][20347] Updated weights for policy 0, policy_version 504277 (0.0006) [2023-12-27 09:07:51,341][20347] Updated weights for policy 0, policy_version 504287 (0.0008) [2023-12-27 09:07:51,621][20381] Updated weights for policy 1, policy_version 504018 (0.0006) [2023-12-27 09:07:51,680][20381] Updated weights for policy 1, policy_version 504028 (0.0008) [2023-12-27 09:07:51,741][20381] Updated weights for policy 1, policy_version 504038 (0.0010) [2023-12-27 09:07:51,795][20381] Updated weights for policy 1, policy_version 504048 (0.0011) [2023-12-27 09:07:52,045][20347] Updated weights for policy 0, policy_version 504297 (0.0007) [2023-12-27 09:07:52,096][20347] Updated weights for policy 0, policy_version 504307 (0.0007) [2023-12-27 09:07:52,159][20347] Updated weights for policy 0, policy_version 504317 (0.0007) [2023-12-27 09:07:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 258179072. Throughput: 0: 10391.5, 1: 10460.5. Samples: 258148512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:52,363][19170] Avg episode reward: [(0, '982.720'), (1, '959.680')] [2023-12-27 09:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000504048_129056768.pth... [2023-12-27 09:07:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000504320_129122304.pth... [2023-12-27 09:07:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000502832_128745472.pth [2023-12-27 09:07:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000503104_128811008.pth [2023-12-27 09:07:52,522][20381] Updated weights for policy 1, policy_version 504058 (0.0011) [2023-12-27 09:07:52,586][20381] Updated weights for policy 1, policy_version 504068 (0.0011) [2023-12-27 09:07:52,655][20381] Updated weights for policy 1, policy_version 504078 (0.0011) [2023-12-27 09:07:52,836][20347] Updated weights for policy 0, policy_version 504327 (0.0007) [2023-12-27 09:07:52,884][20347] Updated weights for policy 0, policy_version 504337 (0.0007) [2023-12-27 09:07:52,937][20347] Updated weights for policy 0, policy_version 504347 (0.0007) [2023-12-27 09:07:53,293][20381] Updated weights for policy 1, policy_version 504088 (0.0009) [2023-12-27 09:07:53,358][20381] Updated weights for policy 1, policy_version 504098 (0.0008) [2023-12-27 09:07:53,416][20381] Updated weights for policy 1, policy_version 504108 (0.0008) [2023-12-27 09:07:53,641][20347] Updated weights for policy 0, policy_version 504357 (0.0008) [2023-12-27 09:07:53,694][20347] Updated weights for policy 0, policy_version 504367 (0.0008) [2023-12-27 09:07:53,741][20347] Updated weights for policy 0, policy_version 504377 (0.0009) [2023-12-27 09:07:53,975][20381] Updated weights for policy 1, policy_version 504118 (0.0006) [2023-12-27 09:07:54,033][20381] Updated weights for policy 1, policy_version 504128 (0.0008) [2023-12-27 09:07:54,092][20381] Updated weights for policy 1, policy_version 504138 (0.0008) [2023-12-27 09:07:54,424][20347] Updated weights for policy 0, policy_version 504387 (0.0008) [2023-12-27 09:07:54,479][20347] Updated weights for policy 0, policy_version 504397 (0.0008) [2023-12-27 09:07:54,531][20347] Updated weights for policy 0, policy_version 504407 (0.0008) [2023-12-27 09:07:54,665][20381] Updated weights for policy 1, policy_version 504148 (0.0007) [2023-12-27 09:07:54,718][20381] Updated weights for policy 1, policy_version 504158 (0.0008) [2023-12-27 09:07:54,774][20381] Updated weights for policy 1, policy_version 504168 (0.0007) [2023-12-27 09:07:55,245][20347] Updated weights for policy 0, policy_version 504417 (0.0008) [2023-12-27 09:07:55,298][20347] Updated weights for policy 0, policy_version 504427 (0.0008) [2023-12-27 09:07:55,352][20347] Updated weights for policy 0, policy_version 504437 (0.0008) [2023-12-27 09:07:55,401][20347] Updated weights for policy 0, policy_version 504447 (0.0008) [2023-12-27 09:07:55,452][20381] Updated weights for policy 1, policy_version 504178 (0.0008) [2023-12-27 09:07:55,510][20381] Updated weights for policy 1, policy_version 504188 (0.0007) [2023-12-27 09:07:55,568][20381] Updated weights for policy 1, policy_version 504198 (0.0008) [2023-12-27 09:07:55,626][20381] Updated weights for policy 1, policy_version 504208 (0.0007) [2023-12-27 09:07:56,127][20347] Updated weights for policy 0, policy_version 504457 (0.0009) [2023-12-27 09:07:56,187][20347] Updated weights for policy 0, policy_version 504467 (0.0008) [2023-12-27 09:07:56,237][20347] Updated weights for policy 0, policy_version 504477 (0.0007) [2023-12-27 09:07:56,268][20381] Updated weights for policy 1, policy_version 504218 (0.0007) [2023-12-27 09:07:56,330][20381] Updated weights for policy 1, policy_version 504228 (0.0007) [2023-12-27 09:07:56,380][20381] Updated weights for policy 1, policy_version 504238 (0.0007) [2023-12-27 09:07:56,916][20347] Updated weights for policy 0, policy_version 504487 (0.0008) [2023-12-27 09:07:56,967][20347] Updated weights for policy 0, policy_version 504497 (0.0008) [2023-12-27 09:07:57,017][20347] Updated weights for policy 0, policy_version 504507 (0.0008) [2023-12-27 09:07:57,037][20381] Updated weights for policy 1, policy_version 504248 (0.0005) [2023-12-27 09:07:57,093][20381] Updated weights for policy 1, policy_version 504258 (0.0005) [2023-12-27 09:07:57,148][20381] Updated weights for policy 1, policy_version 504268 (0.0007) [2023-12-27 09:07:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 258285568. Throughput: 0: 10394.6, 1: 10478.2. Samples: 258272312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:07:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.490')] [2023-12-27 09:07:57,683][20347] Updated weights for policy 0, policy_version 504517 (0.0006) [2023-12-27 09:07:57,743][20347] Updated weights for policy 0, policy_version 504527 (0.0008) [2023-12-27 09:07:57,757][20381] Updated weights for policy 1, policy_version 504278 (0.0009) [2023-12-27 09:07:57,792][20347] Updated weights for policy 0, policy_version 504537 (0.0010) [2023-12-27 09:07:57,807][20381] Updated weights for policy 1, policy_version 504288 (0.0010) [2023-12-27 09:07:57,855][20381] Updated weights for policy 1, policy_version 504298 (0.0010) [2023-12-27 09:07:58,468][20347] Updated weights for policy 0, policy_version 504547 (0.0009) [2023-12-27 09:07:58,496][20381] Updated weights for policy 1, policy_version 504308 (0.0010) [2023-12-27 09:07:58,534][20347] Updated weights for policy 0, policy_version 504557 (0.0008) [2023-12-27 09:07:58,552][20381] Updated weights for policy 1, policy_version 504318 (0.0011) [2023-12-27 09:07:58,601][20347] Updated weights for policy 0, policy_version 504567 (0.0009) [2023-12-27 09:07:58,611][20381] Updated weights for policy 1, policy_version 504328 (0.0010) [2023-12-27 09:07:59,252][20347] Updated weights for policy 0, policy_version 504577 (0.0010) [2023-12-27 09:07:59,312][20347] Updated weights for policy 0, policy_version 504587 (0.0008) [2023-12-27 09:07:59,336][20381] Updated weights for policy 1, policy_version 504338 (0.0011) [2023-12-27 09:07:59,373][20347] Updated weights for policy 0, policy_version 504597 (0.0009) [2023-12-27 09:07:59,397][20381] Updated weights for policy 1, policy_version 504348 (0.0011) [2023-12-27 09:07:59,435][20347] Updated weights for policy 0, policy_version 504607 (0.0008) [2023-12-27 09:07:59,449][20381] Updated weights for policy 1, policy_version 504358 (0.0011) [2023-12-27 09:07:59,505][20381] Updated weights for policy 1, policy_version 504368 (0.0010) [2023-12-27 09:08:00,059][20347] Updated weights for policy 0, policy_version 504617 (0.0005) [2023-12-27 09:08:00,123][20347] Updated weights for policy 0, policy_version 504627 (0.0005) [2023-12-27 09:08:00,184][20381] Updated weights for policy 1, policy_version 504378 (0.0011) [2023-12-27 09:08:00,187][20347] Updated weights for policy 0, policy_version 504637 (0.0007) [2023-12-27 09:08:00,242][20381] Updated weights for policy 1, policy_version 504388 (0.0010) [2023-12-27 09:08:00,296][20381] Updated weights for policy 1, policy_version 504398 (0.0007) [2023-12-27 09:08:00,809][20347] Updated weights for policy 0, policy_version 504647 (0.0010) [2023-12-27 09:08:00,859][20347] Updated weights for policy 0, policy_version 504657 (0.0010) [2023-12-27 09:08:00,884][20381] Updated weights for policy 1, policy_version 504408 (0.0005) [2023-12-27 09:08:00,910][20347] Updated weights for policy 0, policy_version 504667 (0.0010) [2023-12-27 09:08:00,934][20381] Updated weights for policy 1, policy_version 504418 (0.0005) [2023-12-27 09:08:00,990][20381] Updated weights for policy 1, policy_version 504428 (0.0005) [2023-12-27 09:08:01,591][20347] Updated weights for policy 0, policy_version 504677 (0.0010) [2023-12-27 09:08:01,601][20381] Updated weights for policy 1, policy_version 504438 (0.0006) [2023-12-27 09:08:01,645][20347] Updated weights for policy 0, policy_version 504687 (0.0008) [2023-12-27 09:08:01,664][20381] Updated weights for policy 1, policy_version 504448 (0.0008) [2023-12-27 09:08:01,705][20347] Updated weights for policy 0, policy_version 504697 (0.0009) [2023-12-27 09:08:01,731][20381] Updated weights for policy 1, policy_version 504458 (0.0008) [2023-12-27 09:08:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 258383872. Throughput: 0: 10376.4, 1: 10517.7. Samples: 258398400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:08:02,363][19170] Avg episode reward: [(0, '990.450'), (1, '990.430')] [2023-12-27 09:08:02,396][20381] Updated weights for policy 1, policy_version 504468 (0.0009) [2023-12-27 09:08:02,406][20347] Updated weights for policy 0, policy_version 504707 (0.0010) [2023-12-27 09:08:02,451][20381] Updated weights for policy 1, policy_version 504478 (0.0007) [2023-12-27 09:08:02,463][20347] Updated weights for policy 0, policy_version 504717 (0.0008) [2023-12-27 09:08:02,507][20381] Updated weights for policy 1, policy_version 504488 (0.0006) [2023-12-27 09:08:02,526][20347] Updated weights for policy 0, policy_version 504727 (0.0005) [2023-12-27 09:08:03,124][20381] Updated weights for policy 1, policy_version 504498 (0.0007) [2023-12-27 09:08:03,178][20381] Updated weights for policy 1, policy_version 504508 (0.0005) [2023-12-27 09:08:03,190][20347] Updated weights for policy 0, policy_version 504737 (0.0006) [2023-12-27 09:08:03,243][20381] Updated weights for policy 1, policy_version 504518 (0.0007) [2023-12-27 09:08:03,249][20347] Updated weights for policy 0, policy_version 504747 (0.0007) [2023-12-27 09:08:03,299][20381] Updated weights for policy 1, policy_version 504528 (0.0006) [2023-12-27 09:08:03,309][20347] Updated weights for policy 0, policy_version 504757 (0.0008) [2023-12-27 09:08:03,356][20347] Updated weights for policy 0, policy_version 504767 (0.0007) [2023-12-27 09:08:03,827][20381] Updated weights for policy 1, policy_version 504538 (0.0006) [2023-12-27 09:08:03,885][20381] Updated weights for policy 1, policy_version 504548 (0.0007) [2023-12-27 09:08:03,944][20381] Updated weights for policy 1, policy_version 504558 (0.0006) [2023-12-27 09:08:04,067][20347] Updated weights for policy 0, policy_version 504777 (0.0006) [2023-12-27 09:08:04,128][20347] Updated weights for policy 0, policy_version 504787 (0.0006) [2023-12-27 09:08:04,188][20347] Updated weights for policy 0, policy_version 504797 (0.0006) [2023-12-27 09:08:04,613][20381] Updated weights for policy 1, policy_version 504568 (0.0009) [2023-12-27 09:08:04,662][20381] Updated weights for policy 1, policy_version 504578 (0.0010) [2023-12-27 09:08:04,707][20381] Updated weights for policy 1, policy_version 504588 (0.0010) [2023-12-27 09:08:04,790][20347] Updated weights for policy 0, policy_version 504807 (0.0006) [2023-12-27 09:08:04,836][20347] Updated weights for policy 0, policy_version 504817 (0.0005) [2023-12-27 09:08:04,891][20347] Updated weights for policy 0, policy_version 504827 (0.0005) [2023-12-27 09:08:05,409][20381] Updated weights for policy 1, policy_version 504598 (0.0011) [2023-12-27 09:08:05,463][20381] Updated weights for policy 1, policy_version 504608 (0.0010) [2023-12-27 09:08:05,473][20347] Updated weights for policy 0, policy_version 504837 (0.0008) [2023-12-27 09:08:05,518][20381] Updated weights for policy 1, policy_version 504618 (0.0011) [2023-12-27 09:08:05,525][20347] Updated weights for policy 0, policy_version 504847 (0.0009) [2023-12-27 09:08:05,577][20347] Updated weights for policy 0, policy_version 504857 (0.0006) [2023-12-27 09:08:06,194][20381] Updated weights for policy 1, policy_version 504628 (0.0010) [2023-12-27 09:08:06,209][20347] Updated weights for policy 0, policy_version 504867 (0.0007) [2023-12-27 09:08:06,247][20381] Updated weights for policy 1, policy_version 504638 (0.0011) [2023-12-27 09:08:06,265][20347] Updated weights for policy 0, policy_version 504877 (0.0010) [2023-12-27 09:08:06,307][20381] Updated weights for policy 1, policy_version 504648 (0.0011) [2023-12-27 09:08:06,318][20347] Updated weights for policy 0, policy_version 504887 (0.0011) [2023-12-27 09:08:07,008][20347] Updated weights for policy 0, policy_version 504897 (0.0010) [2023-12-27 09:08:07,011][20381] Updated weights for policy 1, policy_version 504658 (0.0011) [2023-12-27 09:08:07,056][20347] Updated weights for policy 0, policy_version 504907 (0.0005) [2023-12-27 09:08:07,063][20381] Updated weights for policy 1, policy_version 504668 (0.0010) [2023-12-27 09:08:07,106][20347] Updated weights for policy 0, policy_version 504917 (0.0006) [2023-12-27 09:08:07,112][20381] Updated weights for policy 1, policy_version 504678 (0.0011) [2023-12-27 09:08:07,160][20347] Updated weights for policy 0, policy_version 504927 (0.0006) [2023-12-27 09:08:07,169][20381] Updated weights for policy 1, policy_version 504688 (0.0010) [2023-12-27 09:08:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 258498560. Throughput: 0: 10374.5, 1: 10570.8. Samples: 258462956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:08:07,363][19170] Avg episode reward: [(0, '980.860'), (1, '980.800')] [2023-12-27 09:08:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000504688_129220608.pth... [2023-12-27 09:08:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000504928_129277952.pth... [2023-12-27 09:08:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000503440_128901120.pth [2023-12-27 09:08:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000503712_128966656.pth [2023-12-27 09:08:07,818][20347] Updated weights for policy 0, policy_version 504937 (0.0010) [2023-12-27 09:08:07,867][20347] Updated weights for policy 0, policy_version 504947 (0.0008) [2023-12-27 09:08:07,874][20381] Updated weights for policy 1, policy_version 504698 (0.0011) [2023-12-27 09:08:07,918][20347] Updated weights for policy 0, policy_version 504957 (0.0010) [2023-12-27 09:08:07,926][20381] Updated weights for policy 1, policy_version 504708 (0.0010) [2023-12-27 09:08:07,971][20381] Updated weights for policy 1, policy_version 504718 (0.0010) [2023-12-27 09:08:08,614][20347] Updated weights for policy 0, policy_version 504967 (0.0010) [2023-12-27 09:08:08,669][20381] Updated weights for policy 1, policy_version 504728 (0.0007) [2023-12-27 09:08:08,673][20347] Updated weights for policy 0, policy_version 504977 (0.0009) [2023-12-27 09:08:08,724][20381] Updated weights for policy 1, policy_version 504738 (0.0006) [2023-12-27 09:08:08,729][20347] Updated weights for policy 0, policy_version 504987 (0.0008) [2023-12-27 09:08:08,781][20381] Updated weights for policy 1, policy_version 504748 (0.0007) [2023-12-27 09:08:09,402][20347] Updated weights for policy 0, policy_version 504997 (0.0007) [2023-12-27 09:08:09,436][20381] Updated weights for policy 1, policy_version 504758 (0.0008) [2023-12-27 09:08:09,457][20347] Updated weights for policy 0, policy_version 505007 (0.0008) [2023-12-27 09:08:09,493][20381] Updated weights for policy 1, policy_version 504768 (0.0005) [2023-12-27 09:08:09,517][20347] Updated weights for policy 0, policy_version 505017 (0.0008) [2023-12-27 09:08:09,560][20381] Updated weights for policy 1, policy_version 504778 (0.0007) [2023-12-27 09:08:10,240][20381] Updated weights for policy 1, policy_version 504788 (0.0008) [2023-12-27 09:08:10,291][20381] Updated weights for policy 1, policy_version 504798 (0.0007) [2023-12-27 09:08:10,301][20347] Updated weights for policy 0, policy_version 505027 (0.0007) [2023-12-27 09:08:10,343][20381] Updated weights for policy 1, policy_version 504808 (0.0006) [2023-12-27 09:08:10,353][20347] Updated weights for policy 0, policy_version 505037 (0.0006) [2023-12-27 09:08:10,415][20347] Updated weights for policy 0, policy_version 505047 (0.0007) [2023-12-27 09:08:11,044][20381] Updated weights for policy 1, policy_version 504818 (0.0008) [2023-12-27 09:08:11,104][20381] Updated weights for policy 1, policy_version 504828 (0.0006) [2023-12-27 09:08:11,115][20347] Updated weights for policy 0, policy_version 505057 (0.0008) [2023-12-27 09:08:11,169][20381] Updated weights for policy 1, policy_version 504838 (0.0007) [2023-12-27 09:08:11,185][20347] Updated weights for policy 0, policy_version 505067 (0.0008) [2023-12-27 09:08:11,228][20381] Updated weights for policy 1, policy_version 504848 (0.0007) [2023-12-27 09:08:11,247][20347] Updated weights for policy 0, policy_version 505077 (0.0007) [2023-12-27 09:08:11,302][20347] Updated weights for policy 0, policy_version 505087 (0.0008) [2023-12-27 09:08:11,944][20381] Updated weights for policy 1, policy_version 504858 (0.0007) [2023-12-27 09:08:11,995][20381] Updated weights for policy 1, policy_version 504868 (0.0007) [2023-12-27 09:08:12,031][20347] Updated weights for policy 0, policy_version 505097 (0.0010) [2023-12-27 09:08:12,051][20381] Updated weights for policy 1, policy_version 504878 (0.0006) [2023-12-27 09:08:12,077][20347] Updated weights for policy 0, policy_version 505107 (0.0010) [2023-12-27 09:08:12,129][20347] Updated weights for policy 0, policy_version 505117 (0.0010) [2023-12-27 09:08:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 258596864. Throughput: 0: 10380.1, 1: 10523.9. Samples: 258585512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:08:12,363][19170] Avg episode reward: [(0, '990.410'), (1, '980.730')] [2023-12-27 09:08:12,758][20347] Updated weights for policy 0, policy_version 505127 (0.0008) [2023-12-27 09:08:12,786][20381] Updated weights for policy 1, policy_version 504888 (0.0008) [2023-12-27 09:08:12,818][20347] Updated weights for policy 0, policy_version 505137 (0.0007) [2023-12-27 09:08:12,849][20381] Updated weights for policy 1, policy_version 504898 (0.0005) [2023-12-27 09:08:12,881][20347] Updated weights for policy 0, policy_version 505147 (0.0008) [2023-12-27 09:08:12,910][20381] Updated weights for policy 1, policy_version 504908 (0.0005) [2023-12-27 09:08:13,515][20347] Updated weights for policy 0, policy_version 505157 (0.0007) [2023-12-27 09:08:13,522][20381] Updated weights for policy 1, policy_version 504918 (0.0007) [2023-12-27 09:08:13,574][20347] Updated weights for policy 0, policy_version 505167 (0.0007) [2023-12-27 09:08:13,588][20381] Updated weights for policy 1, policy_version 504928 (0.0007) [2023-12-27 09:08:13,625][20347] Updated weights for policy 0, policy_version 505177 (0.0008) [2023-12-27 09:08:13,650][20381] Updated weights for policy 1, policy_version 504938 (0.0006) [2023-12-27 09:08:14,206][20347] Updated weights for policy 0, policy_version 505187 (0.0008) [2023-12-27 09:08:14,255][20347] Updated weights for policy 0, policy_version 505197 (0.0010) [2023-12-27 09:08:14,306][20347] Updated weights for policy 0, policy_version 505207 (0.0010) [2023-12-27 09:08:14,317][20381] Updated weights for policy 1, policy_version 504948 (0.0007) [2023-12-27 09:08:14,372][20381] Updated weights for policy 1, policy_version 504958 (0.0009) [2023-12-27 09:08:14,427][20381] Updated weights for policy 1, policy_version 504968 (0.0007) [2023-12-27 09:08:15,018][20347] Updated weights for policy 0, policy_version 505217 (0.0010) [2023-12-27 09:08:15,067][20347] Updated weights for policy 0, policy_version 505227 (0.0008) [2023-12-27 09:08:15,098][20381] Updated weights for policy 1, policy_version 504978 (0.0007) [2023-12-27 09:08:15,116][20347] Updated weights for policy 0, policy_version 505237 (0.0007) [2023-12-27 09:08:15,159][20381] Updated weights for policy 1, policy_version 504988 (0.0008) [2023-12-27 09:08:15,169][20347] Updated weights for policy 0, policy_version 505247 (0.0006) [2023-12-27 09:08:15,224][20381] Updated weights for policy 1, policy_version 504998 (0.0008) [2023-12-27 09:08:15,288][20381] Updated weights for policy 1, policy_version 505008 (0.0008) [2023-12-27 09:08:15,892][20347] Updated weights for policy 0, policy_version 505257 (0.0005) [2023-12-27 09:08:15,946][20347] Updated weights for policy 0, policy_version 505267 (0.0005) [2023-12-27 09:08:15,970][20381] Updated weights for policy 1, policy_version 505018 (0.0008) [2023-12-27 09:08:16,002][20347] Updated weights for policy 0, policy_version 505277 (0.0005) [2023-12-27 09:08:16,026][20381] Updated weights for policy 1, policy_version 505028 (0.0008) [2023-12-27 09:08:16,093][20381] Updated weights for policy 1, policy_version 505038 (0.0008) [2023-12-27 09:08:16,529][20347] Updated weights for policy 0, policy_version 505287 (0.0005) [2023-12-27 09:08:16,584][20347] Updated weights for policy 0, policy_version 505297 (0.0005) [2023-12-27 09:08:16,639][20347] Updated weights for policy 0, policy_version 505307 (0.0006) [2023-12-27 09:08:16,830][20381] Updated weights for policy 1, policy_version 505048 (0.0008) [2023-12-27 09:08:16,893][20381] Updated weights for policy 1, policy_version 505058 (0.0007) [2023-12-27 09:08:16,945][20381] Updated weights for policy 1, policy_version 505068 (0.0008) [2023-12-27 09:08:17,258][20347] Updated weights for policy 0, policy_version 505317 (0.0007) [2023-12-27 09:08:17,313][20347] Updated weights for policy 0, policy_version 505327 (0.0008) [2023-12-27 09:08:17,362][20347] Updated weights for policy 0, policy_version 505337 (0.0008) [2023-12-27 09:08:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 258695168. Throughput: 0: 10348.3, 1: 10487.2. Samples: 258709616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:08:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.720')] [2023-12-27 09:08:17,633][20381] Updated weights for policy 1, policy_version 505078 (0.0008) [2023-12-27 09:08:17,690][20381] Updated weights for policy 1, policy_version 505088 (0.0008) [2023-12-27 09:08:17,742][20381] Updated weights for policy 1, policy_version 505098 (0.0008) [2023-12-27 09:08:17,971][20347] Updated weights for policy 0, policy_version 505347 (0.0007) [2023-12-27 09:08:18,025][20347] Updated weights for policy 0, policy_version 505357 (0.0006) [2023-12-27 09:08:18,082][20347] Updated weights for policy 0, policy_version 505367 (0.0007) [2023-12-27 09:08:18,456][20381] Updated weights for policy 1, policy_version 505108 (0.0006) [2023-12-27 09:08:18,521][20381] Updated weights for policy 1, policy_version 505118 (0.0005) [2023-12-27 09:08:18,583][20381] Updated weights for policy 1, policy_version 505128 (0.0005) [2023-12-27 09:08:18,756][20347] Updated weights for policy 0, policy_version 505377 (0.0008) [2023-12-27 09:08:18,809][20347] Updated weights for policy 0, policy_version 505387 (0.0009) [2023-12-27 09:08:18,861][20347] Updated weights for policy 0, policy_version 505397 (0.0008) [2023-12-27 09:08:18,918][20347] Updated weights for policy 0, policy_version 505407 (0.0008) [2023-12-27 09:08:19,146][20381] Updated weights for policy 1, policy_version 505138 (0.0006) [2023-12-27 09:08:19,200][20381] Updated weights for policy 1, policy_version 505148 (0.0007) [2023-12-27 09:08:19,258][20381] Updated weights for policy 1, policy_version 505158 (0.0007) [2023-12-27 09:08:19,319][20381] Updated weights for policy 1, policy_version 505168 (0.0008) [2023-12-27 09:08:19,628][20347] Updated weights for policy 0, policy_version 505417 (0.0008) [2023-12-27 09:08:19,692][20347] Updated weights for policy 0, policy_version 505427 (0.0008) [2023-12-27 09:08:19,748][20347] Updated weights for policy 0, policy_version 505437 (0.0006) [2023-12-27 09:08:20,034][20381] Updated weights for policy 1, policy_version 505178 (0.0007) [2023-12-27 09:08:20,094][20381] Updated weights for policy 1, policy_version 505188 (0.0008) [2023-12-27 09:08:20,155][20381] Updated weights for policy 1, policy_version 505198 (0.0008) [2023-12-27 09:08:20,378][20347] Updated weights for policy 0, policy_version 505447 (0.0009) [2023-12-27 09:08:20,431][20347] Updated weights for policy 0, policy_version 505457 (0.0011) [2023-12-27 09:08:20,503][20347] Updated weights for policy 0, policy_version 505467 (0.0009) [2023-12-27 09:08:20,891][20381] Updated weights for policy 1, policy_version 505208 (0.0008) [2023-12-27 09:08:20,946][20381] Updated weights for policy 1, policy_version 505218 (0.0007) [2023-12-27 09:08:21,004][20381] Updated weights for policy 1, policy_version 505228 (0.0008) [2023-12-27 09:08:21,150][20347] Updated weights for policy 0, policy_version 505477 (0.0006) [2023-12-27 09:08:21,216][20347] Updated weights for policy 0, policy_version 505487 (0.0006) [2023-12-27 09:08:21,283][20347] Updated weights for policy 0, policy_version 505497 (0.0008) [2023-12-27 09:08:21,715][20381] Updated weights for policy 1, policy_version 505238 (0.0008) [2023-12-27 09:08:21,780][20381] Updated weights for policy 1, policy_version 505248 (0.0008) [2023-12-27 09:08:21,841][20381] Updated weights for policy 1, policy_version 505258 (0.0007) [2023-12-27 09:08:21,969][20347] Updated weights for policy 0, policy_version 505507 (0.0007) [2023-12-27 09:08:22,029][20347] Updated weights for policy 0, policy_version 505517 (0.0007) [2023-12-27 09:08:22,089][20347] Updated weights for policy 0, policy_version 505527 (0.0007) [2023-12-27 09:08:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 258801664. Throughput: 0: 10334.9, 1: 10443.5. Samples: 258771172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:08:22,363][19170] Avg episode reward: [(0, '990.490'), (1, '971.100')] [2023-12-27 09:08:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000505264_129368064.pth... [2023-12-27 09:08:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000505536_129433600.pth... [2023-12-27 09:08:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000504048_129056768.pth [2023-12-27 09:08:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000504320_129122304.pth [2023-12-27 09:08:22,524][20381] Updated weights for policy 1, policy_version 505268 (0.0007) [2023-12-27 09:08:22,584][20381] Updated weights for policy 1, policy_version 505278 (0.0006) [2023-12-27 09:08:22,642][20381] Updated weights for policy 1, policy_version 505288 (0.0007) [2023-12-27 09:08:22,816][20347] Updated weights for policy 0, policy_version 505537 (0.0008) [2023-12-27 09:08:22,875][20347] Updated weights for policy 0, policy_version 505547 (0.0007) [2023-12-27 09:08:22,940][20347] Updated weights for policy 0, policy_version 505557 (0.0008) [2023-12-27 09:08:22,993][20347] Updated weights for policy 0, policy_version 505567 (0.0008) [2023-12-27 09:08:23,270][20381] Updated weights for policy 1, policy_version 505298 (0.0009) [2023-12-27 09:08:23,328][20381] Updated weights for policy 1, policy_version 505308 (0.0011) [2023-12-27 09:08:23,377][20381] Updated weights for policy 1, policy_version 505318 (0.0011) [2023-12-27 09:08:23,429][20381] Updated weights for policy 1, policy_version 505328 (0.0011) [2023-12-27 09:08:23,704][20347] Updated weights for policy 0, policy_version 505577 (0.0009) [2023-12-27 09:08:23,759][20347] Updated weights for policy 0, policy_version 505588 (0.0007) [2023-12-27 09:08:23,820][20347] Updated weights for policy 0, policy_version 505598 (0.0009) [2023-12-27 09:08:24,016][20381] Updated weights for policy 1, policy_version 505338 (0.0011) [2023-12-27 09:08:24,076][20381] Updated weights for policy 1, policy_version 505348 (0.0010) [2023-12-27 09:08:24,133][20381] Updated weights for policy 1, policy_version 505358 (0.0010) [2023-12-27 09:08:24,495][20347] Updated weights for policy 0, policy_version 505608 (0.0009) [2023-12-27 09:08:24,544][20347] Updated weights for policy 0, policy_version 505618 (0.0010) [2023-12-27 09:08:24,599][20347] Updated weights for policy 0, policy_version 505628 (0.0010) [2023-12-27 09:08:24,802][20381] Updated weights for policy 1, policy_version 505368 (0.0010) [2023-12-27 09:08:24,856][20381] Updated weights for policy 1, policy_version 505378 (0.0010) [2023-12-27 09:08:24,910][20381] Updated weights for policy 1, policy_version 505388 (0.0010) [2023-12-27 09:08:25,211][20347] Updated weights for policy 0, policy_version 505638 (0.0007) [2023-12-27 09:08:25,273][20347] Updated weights for policy 0, policy_version 505648 (0.0007) [2023-12-27 09:08:25,338][20347] Updated weights for policy 0, policy_version 505658 (0.0009) [2023-12-27 09:08:25,546][20381] Updated weights for policy 1, policy_version 505398 (0.0010) [2023-12-27 09:08:25,601][20381] Updated weights for policy 1, policy_version 505408 (0.0010) [2023-12-27 09:08:25,659][20381] Updated weights for policy 1, policy_version 505418 (0.0011) [2023-12-27 09:08:26,035][20347] Updated weights for policy 0, policy_version 505668 (0.0009) [2023-12-27 09:08:26,085][20347] Updated weights for policy 0, policy_version 505678 (0.0010) [2023-12-27 09:08:26,131][20347] Updated weights for policy 0, policy_version 505688 (0.0010) [2023-12-27 09:08:26,335][20381] Updated weights for policy 1, policy_version 505428 (0.0011) [2023-12-27 09:08:26,388][20381] Updated weights for policy 1, policy_version 505438 (0.0010) [2023-12-27 09:08:26,439][20381] Updated weights for policy 1, policy_version 505448 (0.0010) [2023-12-27 09:08:26,824][20347] Updated weights for policy 0, policy_version 505698 (0.0010) [2023-12-27 09:08:26,869][20347] Updated weights for policy 0, policy_version 505708 (0.0010) [2023-12-27 09:08:26,922][20347] Updated weights for policy 0, policy_version 505718 (0.0010) [2023-12-27 09:08:26,981][20347] Updated weights for policy 0, policy_version 505728 (0.0007) [2023-12-27 09:08:27,022][20381] Updated weights for policy 1, policy_version 505458 (0.0010) [2023-12-27 09:08:27,072][20381] Updated weights for policy 1, policy_version 505468 (0.0010) [2023-12-27 09:08:27,116][20381] Updated weights for policy 1, policy_version 505478 (0.0006) [2023-12-27 09:08:27,164][20381] Updated weights for policy 1, policy_version 505488 (0.0010) [2023-12-27 09:08:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 258908160. Throughput: 0: 10341.9, 1: 10391.8. Samples: 258894896. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.060')] [2023-12-27 09:08:27,702][20347] Updated weights for policy 0, policy_version 505738 (0.0007) [2023-12-27 09:08:27,761][20347] Updated weights for policy 0, policy_version 505748 (0.0007) [2023-12-27 09:08:27,813][20347] Updated weights for policy 0, policy_version 505758 (0.0007) [2023-12-27 09:08:27,852][20381] Updated weights for policy 1, policy_version 505498 (0.0010) [2023-12-27 09:08:27,899][20381] Updated weights for policy 1, policy_version 505508 (0.0010) [2023-12-27 09:08:27,950][20381] Updated weights for policy 1, policy_version 505518 (0.0010) [2023-12-27 09:08:28,485][20347] Updated weights for policy 0, policy_version 505768 (0.0009) [2023-12-27 09:08:28,549][20347] Updated weights for policy 0, policy_version 505778 (0.0010) [2023-12-27 09:08:28,609][20347] Updated weights for policy 0, policy_version 505788 (0.0011) [2023-12-27 09:08:28,664][20381] Updated weights for policy 1, policy_version 505528 (0.0008) [2023-12-27 09:08:28,723][20381] Updated weights for policy 1, policy_version 505538 (0.0007) [2023-12-27 09:08:28,783][20381] Updated weights for policy 1, policy_version 505548 (0.0011) [2023-12-27 09:08:29,287][20347] Updated weights for policy 0, policy_version 505798 (0.0008) [2023-12-27 09:08:29,341][20347] Updated weights for policy 0, policy_version 505808 (0.0008) [2023-12-27 09:08:29,399][20347] Updated weights for policy 0, policy_version 505818 (0.0008) [2023-12-27 09:08:29,470][20381] Updated weights for policy 1, policy_version 505558 (0.0011) [2023-12-27 09:08:29,527][20381] Updated weights for policy 1, policy_version 505568 (0.0011) [2023-12-27 09:08:29,580][20381] Updated weights for policy 1, policy_version 505578 (0.0011) [2023-12-27 09:08:30,085][20347] Updated weights for policy 0, policy_version 505828 (0.0006) [2023-12-27 09:08:30,147][20347] Updated weights for policy 0, policy_version 505838 (0.0006) [2023-12-27 09:08:30,203][20347] Updated weights for policy 0, policy_version 505848 (0.0007) [2023-12-27 09:08:30,252][20381] Updated weights for policy 1, policy_version 505588 (0.0009) [2023-12-27 09:08:30,316][20381] Updated weights for policy 1, policy_version 505598 (0.0005) [2023-12-27 09:08:30,377][20381] Updated weights for policy 1, policy_version 505608 (0.0006) [2023-12-27 09:08:30,837][20347] Updated weights for policy 0, policy_version 505858 (0.0007) [2023-12-27 09:08:30,902][20347] Updated weights for policy 0, policy_version 505868 (0.0006) [2023-12-27 09:08:30,965][20347] Updated weights for policy 0, policy_version 505878 (0.0005) [2023-12-27 09:08:30,990][20381] Updated weights for policy 1, policy_version 505618 (0.0007) [2023-12-27 09:08:31,019][20347] Updated weights for policy 0, policy_version 505888 (0.0005) [2023-12-27 09:08:31,051][20381] Updated weights for policy 1, policy_version 505628 (0.0008) [2023-12-27 09:08:31,107][20381] Updated weights for policy 1, policy_version 505638 (0.0007) [2023-12-27 09:08:31,170][20381] Updated weights for policy 1, policy_version 505648 (0.0008) [2023-12-27 09:08:31,626][20347] Updated weights for policy 0, policy_version 505898 (0.0007) [2023-12-27 09:08:31,685][20347] Updated weights for policy 0, policy_version 505908 (0.0008) [2023-12-27 09:08:31,743][20347] Updated weights for policy 0, policy_version 505918 (0.0007) [2023-12-27 09:08:31,823][20381] Updated weights for policy 1, policy_version 505658 (0.0008) [2023-12-27 09:08:31,883][20381] Updated weights for policy 1, policy_version 505668 (0.0008) [2023-12-27 09:08:31,937][20381] Updated weights for policy 1, policy_version 505678 (0.0008) [2023-12-27 09:08:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 259006464. Throughput: 0: 10321.7, 1: 10434.5. Samples: 259019880. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:32,363][19170] Avg episode reward: [(0, '991.370'), (1, '980.710')] [2023-12-27 09:08:32,392][20347] Updated weights for policy 0, policy_version 505928 (0.0007) [2023-12-27 09:08:32,423][20291] KL-divergence is very high: 101.4728 [2023-12-27 09:08:32,448][20347] Updated weights for policy 0, policy_version 505938 (0.0007) [2023-12-27 09:08:32,505][20347] Updated weights for policy 0, policy_version 505948 (0.0008) [2023-12-27 09:08:32,594][20381] Updated weights for policy 1, policy_version 505688 (0.0007) [2023-12-27 09:08:32,648][20381] Updated weights for policy 1, policy_version 505698 (0.0008) [2023-12-27 09:08:32,706][20381] Updated weights for policy 1, policy_version 505708 (0.0008) [2023-12-27 09:08:33,190][20347] Updated weights for policy 0, policy_version 505958 (0.0007) [2023-12-27 09:08:33,241][20347] Updated weights for policy 0, policy_version 505968 (0.0007) [2023-12-27 09:08:33,296][20347] Updated weights for policy 0, policy_version 505978 (0.0009) [2023-12-27 09:08:33,382][20381] Updated weights for policy 1, policy_version 505718 (0.0008) [2023-12-27 09:08:33,434][20381] Updated weights for policy 1, policy_version 505728 (0.0008) [2023-12-27 09:08:33,489][20381] Updated weights for policy 1, policy_version 505738 (0.0008) [2023-12-27 09:08:33,966][20347] Updated weights for policy 0, policy_version 505988 (0.0007) [2023-12-27 09:08:34,022][20347] Updated weights for policy 0, policy_version 505998 (0.0005) [2023-12-27 09:08:34,078][20347] Updated weights for policy 0, policy_version 506008 (0.0006) [2023-12-27 09:08:34,166][20381] Updated weights for policy 1, policy_version 505748 (0.0008) [2023-12-27 09:08:34,215][20381] Updated weights for policy 1, policy_version 505758 (0.0008) [2023-12-27 09:08:34,268][20381] Updated weights for policy 1, policy_version 505768 (0.0008) [2023-12-27 09:08:34,756][20347] Updated weights for policy 0, policy_version 506018 (0.0006) [2023-12-27 09:08:34,811][20347] Updated weights for policy 0, policy_version 506028 (0.0006) [2023-12-27 09:08:34,872][20347] Updated weights for policy 0, policy_version 506038 (0.0006) [2023-12-27 09:08:34,930][20347] Updated weights for policy 0, policy_version 506048 (0.0008) [2023-12-27 09:08:34,959][20381] Updated weights for policy 1, policy_version 505778 (0.0008) [2023-12-27 09:08:35,008][20381] Updated weights for policy 1, policy_version 505788 (0.0007) [2023-12-27 09:08:35,060][20381] Updated weights for policy 1, policy_version 505798 (0.0008) [2023-12-27 09:08:35,112][20381] Updated weights for policy 1, policy_version 505808 (0.0008) [2023-12-27 09:08:35,552][20347] Updated weights for policy 0, policy_version 506058 (0.0007) [2023-12-27 09:08:35,614][20347] Updated weights for policy 0, policy_version 506068 (0.0006) [2023-12-27 09:08:35,683][20347] Updated weights for policy 0, policy_version 506078 (0.0005) [2023-12-27 09:08:35,821][20381] Updated weights for policy 1, policy_version 505818 (0.0007) [2023-12-27 09:08:35,881][20381] Updated weights for policy 1, policy_version 505828 (0.0009) [2023-12-27 09:08:35,937][20381] Updated weights for policy 1, policy_version 505838 (0.0009) [2023-12-27 09:08:36,276][20347] Updated weights for policy 0, policy_version 506088 (0.0007) [2023-12-27 09:08:36,335][20347] Updated weights for policy 0, policy_version 506098 (0.0008) [2023-12-27 09:08:36,392][20347] Updated weights for policy 0, policy_version 506108 (0.0008) [2023-12-27 09:08:36,583][20381] Updated weights for policy 1, policy_version 505848 (0.0009) [2023-12-27 09:08:36,647][20381] Updated weights for policy 1, policy_version 505858 (0.0008) [2023-12-27 09:08:36,708][20381] Updated weights for policy 1, policy_version 505868 (0.0007) [2023-12-27 09:08:37,096][20347] Updated weights for policy 0, policy_version 506118 (0.0008) [2023-12-27 09:08:37,151][20347] Updated weights for policy 0, policy_version 506128 (0.0007) [2023-12-27 09:08:37,203][20347] Updated weights for policy 0, policy_version 506138 (0.0008) [2023-12-27 09:08:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 259112960. Throughput: 0: 10335.9, 1: 10397.9. Samples: 259081532. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:37,363][19170] Avg episode reward: [(0, '991.370'), (1, '990.390')] [2023-12-27 09:08:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000506144_129589248.pth... [2023-12-27 09:08:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000504928_129277952.pth [2023-12-27 09:08:37,408][20381] Updated weights for policy 1, policy_version 505878 (0.0008) [2023-12-27 09:08:37,465][20381] Updated weights for policy 1, policy_version 505888 (0.0008) [2023-12-27 09:08:37,521][20381] Updated weights for policy 1, policy_version 505898 (0.0007) [2023-12-27 09:08:37,556][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000505904_129531904.pth... [2023-12-27 09:08:37,560][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000504688_129220608.pth [2023-12-27 09:08:37,864][20347] Updated weights for policy 0, policy_version 506148 (0.0009) [2023-12-27 09:08:37,919][20347] Updated weights for policy 0, policy_version 506158 (0.0010) [2023-12-27 09:08:37,976][20347] Updated weights for policy 0, policy_version 506168 (0.0009) [2023-12-27 09:08:38,140][20381] Updated weights for policy 1, policy_version 505908 (0.0008) [2023-12-27 09:08:38,204][20381] Updated weights for policy 1, policy_version 505918 (0.0010) [2023-12-27 09:08:38,252][20381] Updated weights for policy 1, policy_version 505928 (0.0010) [2023-12-27 09:08:38,604][20347] Updated weights for policy 0, policy_version 506178 (0.0006) [2023-12-27 09:08:38,661][20347] Updated weights for policy 0, policy_version 506188 (0.0007) [2023-12-27 09:08:38,720][20347] Updated weights for policy 0, policy_version 506198 (0.0007) [2023-12-27 09:08:38,784][20347] Updated weights for policy 0, policy_version 506208 (0.0005) [2023-12-27 09:08:38,894][20381] Updated weights for policy 1, policy_version 505938 (0.0010) [2023-12-27 09:08:38,945][20381] Updated weights for policy 1, policy_version 505948 (0.0011) [2023-12-27 09:08:39,000][20381] Updated weights for policy 1, policy_version 505958 (0.0010) [2023-12-27 09:08:39,053][20381] Updated weights for policy 1, policy_version 505968 (0.0011) [2023-12-27 09:08:39,441][20347] Updated weights for policy 0, policy_version 506218 (0.0007) [2023-12-27 09:08:39,492][20347] Updated weights for policy 0, policy_version 506228 (0.0007) [2023-12-27 09:08:39,547][20347] Updated weights for policy 0, policy_version 506238 (0.0007) [2023-12-27 09:08:39,781][20381] Updated weights for policy 1, policy_version 505978 (0.0007) [2023-12-27 09:08:39,795][20292] KL-divergence is very high: 175.0292 [2023-12-27 09:08:39,843][20381] Updated weights for policy 1, policy_version 505988 (0.0008) [2023-12-27 09:08:39,889][20292] KL-divergence is very high: 126.4142 [2023-12-27 09:08:39,901][20381] Updated weights for policy 1, policy_version 505998 (0.0008) [2023-12-27 09:08:40,271][20347] Updated weights for policy 0, policy_version 506248 (0.0007) [2023-12-27 09:08:40,333][20347] Updated weights for policy 0, policy_version 506258 (0.0007) [2023-12-27 09:08:40,393][20347] Updated weights for policy 0, policy_version 506268 (0.0008) [2023-12-27 09:08:40,567][20381] Updated weights for policy 1, policy_version 506008 (0.0006) [2023-12-27 09:08:40,618][20381] Updated weights for policy 1, policy_version 506018 (0.0005) [2023-12-27 09:08:40,672][20381] Updated weights for policy 1, policy_version 506028 (0.0006) [2023-12-27 09:08:41,029][20347] Updated weights for policy 0, policy_version 506278 (0.0007) [2023-12-27 09:08:41,087][20347] Updated weights for policy 0, policy_version 506288 (0.0008) [2023-12-27 09:08:41,145][20347] Updated weights for policy 0, policy_version 506298 (0.0009) [2023-12-27 09:08:41,354][20381] Updated weights for policy 1, policy_version 506038 (0.0009) [2023-12-27 09:08:41,417][20381] Updated weights for policy 1, policy_version 506048 (0.0007) [2023-12-27 09:08:41,481][20381] Updated weights for policy 1, policy_version 506058 (0.0006) [2023-12-27 09:08:41,876][20347] Updated weights for policy 0, policy_version 506308 (0.0008) [2023-12-27 09:08:41,932][20347] Updated weights for policy 0, policy_version 506318 (0.0008) [2023-12-27 09:08:41,991][20347] Updated weights for policy 0, policy_version 506328 (0.0008) [2023-12-27 09:08:42,132][20381] Updated weights for policy 1, policy_version 506068 (0.0008) [2023-12-27 09:08:42,191][20381] Updated weights for policy 1, policy_version 506078 (0.0008) [2023-12-27 09:08:42,246][20381] Updated weights for policy 1, policy_version 506088 (0.0006) [2023-12-27 09:08:42,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 259219456. Throughput: 0: 10418.2, 1: 10352.5. Samples: 259206996. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:42,363][19170] Avg episode reward: [(0, '990.510'), (1, '991.020')] [2023-12-27 09:08:42,740][20347] Updated weights for policy 0, policy_version 506338 (0.0007) [2023-12-27 09:08:42,791][20347] Updated weights for policy 0, policy_version 506348 (0.0007) [2023-12-27 09:08:42,843][20347] Updated weights for policy 0, policy_version 506358 (0.0008) [2023-12-27 09:08:42,872][20381] Updated weights for policy 1, policy_version 506098 (0.0008) [2023-12-27 09:08:42,897][20347] Updated weights for policy 0, policy_version 506368 (0.0008) [2023-12-27 09:08:42,930][20381] Updated weights for policy 1, policy_version 506108 (0.0006) [2023-12-27 09:08:42,980][20381] Updated weights for policy 1, policy_version 506118 (0.0005) [2023-12-27 09:08:43,036][20381] Updated weights for policy 1, policy_version 506128 (0.0005) [2023-12-27 09:08:43,581][20381] Updated weights for policy 1, policy_version 506138 (0.0007) [2023-12-27 09:08:43,633][20381] Updated weights for policy 1, policy_version 506148 (0.0008) [2023-12-27 09:08:43,659][20347] Updated weights for policy 0, policy_version 506378 (0.0008) [2023-12-27 09:08:43,684][20381] Updated weights for policy 1, policy_version 506158 (0.0005) [2023-12-27 09:08:43,716][20347] Updated weights for policy 0, policy_version 506388 (0.0007) [2023-12-27 09:08:43,768][20347] Updated weights for policy 0, policy_version 506398 (0.0008) [2023-12-27 09:08:44,269][20381] Updated weights for policy 1, policy_version 506168 (0.0010) [2023-12-27 09:08:44,327][20381] Updated weights for policy 1, policy_version 506178 (0.0011) [2023-12-27 09:08:44,388][20381] Updated weights for policy 1, policy_version 506188 (0.0010) [2023-12-27 09:08:44,491][20347] Updated weights for policy 0, policy_version 506408 (0.0008) [2023-12-27 09:08:44,544][20347] Updated weights for policy 0, policy_version 506418 (0.0008) [2023-12-27 09:08:44,597][20347] Updated weights for policy 0, policy_version 506428 (0.0008) [2023-12-27 09:08:45,080][20381] Updated weights for policy 1, policy_version 506198 (0.0009) [2023-12-27 09:08:45,144][20381] Updated weights for policy 1, policy_version 506208 (0.0007) [2023-12-27 09:08:45,203][20381] Updated weights for policy 1, policy_version 506218 (0.0010) [2023-12-27 09:08:45,304][20347] Updated weights for policy 0, policy_version 506438 (0.0006) [2023-12-27 09:08:45,367][20347] Updated weights for policy 0, policy_version 506448 (0.0007) [2023-12-27 09:08:45,424][20347] Updated weights for policy 0, policy_version 506458 (0.0006) [2023-12-27 09:08:45,816][20381] Updated weights for policy 1, policy_version 506228 (0.0008) [2023-12-27 09:08:45,867][20381] Updated weights for policy 1, policy_version 506238 (0.0005) [2023-12-27 09:08:45,918][20381] Updated weights for policy 1, policy_version 506248 (0.0005) [2023-12-27 09:08:46,043][20347] Updated weights for policy 0, policy_version 506468 (0.0008) [2023-12-27 09:08:46,097][20347] Updated weights for policy 0, policy_version 506478 (0.0010) [2023-12-27 09:08:46,147][20347] Updated weights for policy 0, policy_version 506488 (0.0010) [2023-12-27 09:08:46,522][20381] Updated weights for policy 1, policy_version 506258 (0.0007) [2023-12-27 09:08:46,580][20381] Updated weights for policy 1, policy_version 506268 (0.0010) [2023-12-27 09:08:46,646][20381] Updated weights for policy 1, policy_version 506278 (0.0010) [2023-12-27 09:08:46,710][20381] Updated weights for policy 1, policy_version 506288 (0.0010) [2023-12-27 09:08:46,821][20347] Updated weights for policy 0, policy_version 506498 (0.0010) [2023-12-27 09:08:46,876][20347] Updated weights for policy 0, policy_version 506508 (0.0010) [2023-12-27 09:08:46,933][20347] Updated weights for policy 0, policy_version 506518 (0.0010) [2023-12-27 09:08:46,984][20347] Updated weights for policy 0, policy_version 506528 (0.0010) [2023-12-27 09:08:47,262][20381] Updated weights for policy 1, policy_version 506298 (0.0008) [2023-12-27 09:08:47,319][20381] Updated weights for policy 1, policy_version 506308 (0.0006) [2023-12-27 09:08:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 259317760. Throughput: 0: 10342.0, 1: 10421.1. Samples: 259332740. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:47,363][19170] Avg episode reward: [(0, '971.300'), (1, '980.220')] [2023-12-27 09:08:47,368][20381] Updated weights for policy 1, policy_version 506318 (0.0005) [2023-12-27 09:08:47,672][20347] Updated weights for policy 0, policy_version 506538 (0.0007) [2023-12-27 09:08:47,719][20347] Updated weights for policy 0, policy_version 506548 (0.0008) [2023-12-27 09:08:47,777][20347] Updated weights for policy 0, policy_version 506558 (0.0005) [2023-12-27 09:08:47,932][20381] Updated weights for policy 1, policy_version 506328 (0.0009) [2023-12-27 09:08:47,986][20381] Updated weights for policy 1, policy_version 506338 (0.0010) [2023-12-27 09:08:48,035][20381] Updated weights for policy 1, policy_version 506348 (0.0010) [2023-12-27 09:08:48,361][20347] Updated weights for policy 0, policy_version 506568 (0.0008) [2023-12-27 09:08:48,423][20347] Updated weights for policy 0, policy_version 506578 (0.0008) [2023-12-27 09:08:48,482][20347] Updated weights for policy 0, policy_version 506588 (0.0008) [2023-12-27 09:08:48,724][20381] Updated weights for policy 1, policy_version 506358 (0.0010) [2023-12-27 09:08:48,794][20381] Updated weights for policy 1, policy_version 506368 (0.0006) [2023-12-27 09:08:48,863][20381] Updated weights for policy 1, policy_version 506378 (0.0006) [2023-12-27 09:08:49,160][20347] Updated weights for policy 0, policy_version 506598 (0.0008) [2023-12-27 09:08:49,218][20347] Updated weights for policy 0, policy_version 506608 (0.0007) [2023-12-27 09:08:49,274][20347] Updated weights for policy 0, policy_version 506618 (0.0008) [2023-12-27 09:08:49,431][20381] Updated weights for policy 1, policy_version 506388 (0.0007) [2023-12-27 09:08:49,484][20381] Updated weights for policy 1, policy_version 506398 (0.0011) [2023-12-27 09:08:49,544][20381] Updated weights for policy 1, policy_version 506408 (0.0010) [2023-12-27 09:08:49,980][20347] Updated weights for policy 0, policy_version 506628 (0.0007) [2023-12-27 09:08:50,034][20347] Updated weights for policy 0, policy_version 506638 (0.0007) [2023-12-27 09:08:50,094][20347] Updated weights for policy 0, policy_version 506648 (0.0008) [2023-12-27 09:08:50,197][20381] Updated weights for policy 1, policy_version 506418 (0.0010) [2023-12-27 09:08:50,253][20381] Updated weights for policy 1, policy_version 506428 (0.0005) [2023-12-27 09:08:50,310][20381] Updated weights for policy 1, policy_version 506438 (0.0005) [2023-12-27 09:08:50,368][20381] Updated weights for policy 1, policy_version 506448 (0.0006) [2023-12-27 09:08:50,769][20347] Updated weights for policy 0, policy_version 506658 (0.0008) [2023-12-27 09:08:50,825][20347] Updated weights for policy 0, policy_version 506668 (0.0008) [2023-12-27 09:08:50,885][20347] Updated weights for policy 0, policy_version 506678 (0.0006) [2023-12-27 09:08:50,944][20347] Updated weights for policy 0, policy_version 506688 (0.0006) [2023-12-27 09:08:50,995][20381] Updated weights for policy 1, policy_version 506458 (0.0008) [2023-12-27 09:08:51,059][20381] Updated weights for policy 1, policy_version 506468 (0.0008) [2023-12-27 09:08:51,120][20381] Updated weights for policy 1, policy_version 506478 (0.0007) [2023-12-27 09:08:51,611][20347] Updated weights for policy 0, policy_version 506698 (0.0008) [2023-12-27 09:08:51,674][20347] Updated weights for policy 0, policy_version 506708 (0.0008) [2023-12-27 09:08:51,738][20347] Updated weights for policy 0, policy_version 506718 (0.0009) [2023-12-27 09:08:51,849][20381] Updated weights for policy 1, policy_version 506488 (0.0008) [2023-12-27 09:08:51,915][20381] Updated weights for policy 1, policy_version 506498 (0.0008) [2023-12-27 09:08:51,967][20381] Updated weights for policy 1, policy_version 506508 (0.0008) [2023-12-27 09:08:52,363][19170] Fps is (10 sec: 20478.6, 60 sec: 20752.8, 300 sec: 20743.8). Total num frames: 259424256. Throughput: 0: 10337.6, 1: 10415.5. Samples: 259396860. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:52,364][19170] Avg episode reward: [(0, '971.310'), (1, '997.500')] [2023-12-27 09:08:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000506720_129736704.pth... [2023-12-27 09:08:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000506512_129687552.pth... [2023-12-27 09:08:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000505536_129433600.pth [2023-12-27 09:08:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000505264_129368064.pth [2023-12-27 09:08:52,447][20347] Updated weights for policy 0, policy_version 506728 (0.0007) [2023-12-27 09:08:52,507][20347] Updated weights for policy 0, policy_version 506738 (0.0008) [2023-12-27 09:08:52,567][20347] Updated weights for policy 0, policy_version 506748 (0.0008) [2023-12-27 09:08:52,601][20381] Updated weights for policy 1, policy_version 506518 (0.0007) [2023-12-27 09:08:52,649][20381] Updated weights for policy 1, policy_version 506528 (0.0008) [2023-12-27 09:08:52,697][20381] Updated weights for policy 1, policy_version 506538 (0.0009) [2023-12-27 09:08:53,243][20347] Updated weights for policy 0, policy_version 506758 (0.0008) [2023-12-27 09:08:53,296][20347] Updated weights for policy 0, policy_version 506768 (0.0007) [2023-12-27 09:08:53,352][20347] Updated weights for policy 0, policy_version 506778 (0.0007) [2023-12-27 09:08:53,424][20381] Updated weights for policy 1, policy_version 506548 (0.0008) [2023-12-27 09:08:53,479][20381] Updated weights for policy 1, policy_version 506558 (0.0010) [2023-12-27 09:08:53,536][20381] Updated weights for policy 1, policy_version 506568 (0.0010) [2023-12-27 09:08:53,956][20347] Updated weights for policy 0, policy_version 506788 (0.0006) [2023-12-27 09:08:54,012][20347] Updated weights for policy 0, policy_version 506798 (0.0005) [2023-12-27 09:08:54,064][20347] Updated weights for policy 0, policy_version 506808 (0.0005) [2023-12-27 09:08:54,115][20381] Updated weights for policy 1, policy_version 506578 (0.0010) [2023-12-27 09:08:54,176][20381] Updated weights for policy 1, policy_version 506588 (0.0007) [2023-12-27 09:08:54,235][20381] Updated weights for policy 1, policy_version 506598 (0.0007) [2023-12-27 09:08:54,293][20381] Updated weights for policy 1, policy_version 506608 (0.0009) [2023-12-27 09:08:54,706][20347] Updated weights for policy 0, policy_version 506818 (0.0005) [2023-12-27 09:08:54,752][20347] Updated weights for policy 0, policy_version 506828 (0.0005) [2023-12-27 09:08:54,805][20347] Updated weights for policy 0, policy_version 506838 (0.0010) [2023-12-27 09:08:54,859][20347] Updated weights for policy 0, policy_version 506848 (0.0010) [2023-12-27 09:08:54,909][20381] Updated weights for policy 1, policy_version 506618 (0.0007) [2023-12-27 09:08:54,960][20381] Updated weights for policy 1, policy_version 506628 (0.0007) [2023-12-27 09:08:55,014][20381] Updated weights for policy 1, policy_version 506638 (0.0007) [2023-12-27 09:08:55,440][20347] Updated weights for policy 0, policy_version 506858 (0.0010) [2023-12-27 09:08:55,491][20347] Updated weights for policy 0, policy_version 506868 (0.0010) [2023-12-27 09:08:55,552][20347] Updated weights for policy 0, policy_version 506878 (0.0010) [2023-12-27 09:08:55,658][20381] Updated weights for policy 1, policy_version 506648 (0.0006) [2023-12-27 09:08:55,708][20381] Updated weights for policy 1, policy_version 506658 (0.0006) [2023-12-27 09:08:55,757][20381] Updated weights for policy 1, policy_version 506668 (0.0008) [2023-12-27 09:08:56,114][20347] Updated weights for policy 0, policy_version 506888 (0.0006) [2023-12-27 09:08:56,165][20347] Updated weights for policy 0, policy_version 506898 (0.0005) [2023-12-27 09:08:56,223][20347] Updated weights for policy 0, policy_version 506908 (0.0008) [2023-12-27 09:08:56,465][20381] Updated weights for policy 1, policy_version 506678 (0.0006) [2023-12-27 09:08:56,517][20381] Updated weights for policy 1, policy_version 506688 (0.0007) [2023-12-27 09:08:56,561][20381] Updated weights for policy 1, policy_version 506698 (0.0005) [2023-12-27 09:08:56,837][20347] Updated weights for policy 0, policy_version 506918 (0.0010) [2023-12-27 09:08:56,885][20347] Updated weights for policy 0, policy_version 506928 (0.0010) [2023-12-27 09:08:56,948][20347] Updated weights for policy 0, policy_version 506938 (0.0010) [2023-12-27 09:08:57,185][20381] Updated weights for policy 1, policy_version 506708 (0.0006) [2023-12-27 09:08:57,238][20381] Updated weights for policy 1, policy_version 506718 (0.0005) [2023-12-27 09:08:57,287][20381] Updated weights for policy 1, policy_version 506728 (0.0006) [2023-12-27 09:08:57,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 259538944. Throughput: 0: 10393.2, 1: 10454.8. Samples: 259523676. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:08:57,363][19170] Avg episode reward: [(0, '990.630'), (1, '999.440')] [2023-12-27 09:08:57,593][20347] Updated weights for policy 0, policy_version 506948 (0.0010) [2023-12-27 09:08:57,638][20347] Updated weights for policy 0, policy_version 506958 (0.0010) [2023-12-27 09:08:57,695][20347] Updated weights for policy 0, policy_version 506968 (0.0010) [2023-12-27 09:08:57,855][20381] Updated weights for policy 1, policy_version 506738 (0.0007) [2023-12-27 09:08:57,905][20381] Updated weights for policy 1, policy_version 506748 (0.0005) [2023-12-27 09:08:57,959][20381] Updated weights for policy 1, policy_version 506758 (0.0010) [2023-12-27 09:08:58,010][20381] Updated weights for policy 1, policy_version 506768 (0.0010) [2023-12-27 09:08:58,372][20347] Updated weights for policy 0, policy_version 506978 (0.0010) [2023-12-27 09:08:58,435][20347] Updated weights for policy 0, policy_version 506988 (0.0008) [2023-12-27 09:08:58,498][20347] Updated weights for policy 0, policy_version 506998 (0.0008) [2023-12-27 09:08:58,559][20347] Updated weights for policy 0, policy_version 507008 (0.0008) [2023-12-27 09:08:58,710][20381] Updated weights for policy 1, policy_version 506778 (0.0011) [2023-12-27 09:08:58,766][20381] Updated weights for policy 1, policy_version 506788 (0.0010) [2023-12-27 09:08:58,825][20381] Updated weights for policy 1, policy_version 506798 (0.0009) [2023-12-27 09:08:59,270][20347] Updated weights for policy 0, policy_version 507018 (0.0007) [2023-12-27 09:08:59,327][20347] Updated weights for policy 0, policy_version 507028 (0.0007) [2023-12-27 09:08:59,384][20347] Updated weights for policy 0, policy_version 507038 (0.0010) [2023-12-27 09:08:59,515][20381] Updated weights for policy 1, policy_version 506808 (0.0008) [2023-12-27 09:08:59,574][20381] Updated weights for policy 1, policy_version 506818 (0.0008) [2023-12-27 09:08:59,632][20381] Updated weights for policy 1, policy_version 506828 (0.0008) [2023-12-27 09:09:00,031][20347] Updated weights for policy 0, policy_version 507048 (0.0008) [2023-12-27 09:09:00,088][20347] Updated weights for policy 0, policy_version 507058 (0.0007) [2023-12-27 09:09:00,142][20347] Updated weights for policy 0, policy_version 507068 (0.0007) [2023-12-27 09:09:00,350][20381] Updated weights for policy 1, policy_version 506838 (0.0008) [2023-12-27 09:09:00,408][20381] Updated weights for policy 1, policy_version 506848 (0.0007) [2023-12-27 09:09:00,466][20381] Updated weights for policy 1, policy_version 506858 (0.0008) [2023-12-27 09:09:00,830][20347] Updated weights for policy 0, policy_version 507078 (0.0009) [2023-12-27 09:09:00,887][20347] Updated weights for policy 0, policy_version 507088 (0.0010) [2023-12-27 09:09:00,940][20347] Updated weights for policy 0, policy_version 507098 (0.0009) [2023-12-27 09:09:01,178][20381] Updated weights for policy 1, policy_version 506868 (0.0007) [2023-12-27 09:09:01,234][20381] Updated weights for policy 1, policy_version 506878 (0.0007) [2023-12-27 09:09:01,295][20381] Updated weights for policy 1, policy_version 506888 (0.0008) [2023-12-27 09:09:01,607][20347] Updated weights for policy 0, policy_version 507108 (0.0007) [2023-12-27 09:09:01,667][20347] Updated weights for policy 0, policy_version 507118 (0.0008) [2023-12-27 09:09:01,727][20347] Updated weights for policy 0, policy_version 507128 (0.0007) [2023-12-27 09:09:02,029][20381] Updated weights for policy 1, policy_version 506898 (0.0008) [2023-12-27 09:09:02,088][20381] Updated weights for policy 1, policy_version 506908 (0.0007) [2023-12-27 09:09:02,145][20381] Updated weights for policy 1, policy_version 506918 (0.0006) [2023-12-27 09:09:02,201][20381] Updated weights for policy 1, policy_version 506928 (0.0006) [2023-12-27 09:09:02,332][20347] Updated weights for policy 0, policy_version 507138 (0.0007) [2023-12-27 09:09:02,362][19170] Fps is (10 sec: 21300.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 259637248. Throughput: 0: 10358.8, 1: 10494.7. Samples: 259648020. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:02,363][19170] Avg episode reward: [(0, '990.630'), (1, '1000.000')] [2023-12-27 09:09:02,394][20347] Updated weights for policy 0, policy_version 507148 (0.0010) [2023-12-27 09:09:02,454][20347] Updated weights for policy 0, policy_version 507158 (0.0010) [2023-12-27 09:09:02,514][20347] Updated weights for policy 0, policy_version 507168 (0.0010) [2023-12-27 09:09:02,890][20381] Updated weights for policy 1, policy_version 506938 (0.0007) [2023-12-27 09:09:02,940][20381] Updated weights for policy 1, policy_version 506948 (0.0007) [2023-12-27 09:09:03,003][20381] Updated weights for policy 1, policy_version 506958 (0.0006) [2023-12-27 09:09:03,195][20347] Updated weights for policy 0, policy_version 507178 (0.0008) [2023-12-27 09:09:03,255][20347] Updated weights for policy 0, policy_version 507188 (0.0008) [2023-12-27 09:09:03,316][20347] Updated weights for policy 0, policy_version 507198 (0.0008) [2023-12-27 09:09:03,596][20381] Updated weights for policy 1, policy_version 506968 (0.0007) [2023-12-27 09:09:03,651][20381] Updated weights for policy 1, policy_version 506978 (0.0008) [2023-12-27 09:09:03,702][20381] Updated weights for policy 1, policy_version 506988 (0.0007) [2023-12-27 09:09:04,021][20347] Updated weights for policy 0, policy_version 507208 (0.0007) [2023-12-27 09:09:04,083][20347] Updated weights for policy 0, policy_version 507218 (0.0008) [2023-12-27 09:09:04,139][20347] Updated weights for policy 0, policy_version 507228 (0.0008) [2023-12-27 09:09:04,406][20381] Updated weights for policy 1, policy_version 506998 (0.0007) [2023-12-27 09:09:04,461][20381] Updated weights for policy 1, policy_version 507008 (0.0008) [2023-12-27 09:09:04,517][20381] Updated weights for policy 1, policy_version 507018 (0.0007) [2023-12-27 09:09:04,830][20347] Updated weights for policy 0, policy_version 507238 (0.0007) [2023-12-27 09:09:04,879][20347] Updated weights for policy 0, policy_version 507248 (0.0008) [2023-12-27 09:09:04,934][20347] Updated weights for policy 0, policy_version 507258 (0.0008) [2023-12-27 09:09:05,197][20381] Updated weights for policy 1, policy_version 507028 (0.0007) [2023-12-27 09:09:05,254][20381] Updated weights for policy 1, policy_version 507038 (0.0009) [2023-12-27 09:09:05,309][20381] Updated weights for policy 1, policy_version 507048 (0.0008) [2023-12-27 09:09:05,637][20347] Updated weights for policy 0, policy_version 507268 (0.0008) [2023-12-27 09:09:05,694][20347] Updated weights for policy 0, policy_version 507278 (0.0008) [2023-12-27 09:09:05,750][20347] Updated weights for policy 0, policy_version 507288 (0.0008) [2023-12-27 09:09:06,000][20381] Updated weights for policy 1, policy_version 507058 (0.0008) [2023-12-27 09:09:06,056][20381] Updated weights for policy 1, policy_version 507068 (0.0008) [2023-12-27 09:09:06,106][20381] Updated weights for policy 1, policy_version 507078 (0.0007) [2023-12-27 09:09:06,164][20381] Updated weights for policy 1, policy_version 507088 (0.0007) [2023-12-27 09:09:06,383][20347] Updated weights for policy 0, policy_version 507298 (0.0007) [2023-12-27 09:09:06,441][20347] Updated weights for policy 0, policy_version 507308 (0.0008) [2023-12-27 09:09:06,500][20347] Updated weights for policy 0, policy_version 507318 (0.0008) [2023-12-27 09:09:06,556][20347] Updated weights for policy 0, policy_version 507328 (0.0008) [2023-12-27 09:09:06,871][20381] Updated weights for policy 1, policy_version 507098 (0.0008) [2023-12-27 09:09:06,931][20381] Updated weights for policy 1, policy_version 507108 (0.0007) [2023-12-27 09:09:06,991][20381] Updated weights for policy 1, policy_version 507118 (0.0008) [2023-12-27 09:09:07,233][20347] Updated weights for policy 0, policy_version 507338 (0.0007) [2023-12-27 09:09:07,293][20347] Updated weights for policy 0, policy_version 507348 (0.0007) [2023-12-27 09:09:07,355][20347] Updated weights for policy 0, policy_version 507358 (0.0008) [2023-12-27 09:09:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 259735552. Throughput: 0: 10331.4, 1: 10523.7. Samples: 259709652. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:07,363][19170] Avg episode reward: [(0, '990.320'), (1, '1000.000')] [2023-12-27 09:09:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000507360_129900544.pth... [2023-12-27 09:09:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000507120_129843200.pth... [2023-12-27 09:09:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000506144_129589248.pth [2023-12-27 09:09:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000505904_129531904.pth [2023-12-27 09:09:07,655][20381] Updated weights for policy 1, policy_version 507128 (0.0006) [2023-12-27 09:09:07,717][20381] Updated weights for policy 1, policy_version 507138 (0.0005) [2023-12-27 09:09:07,780][20381] Updated weights for policy 1, policy_version 507148 (0.0005) [2023-12-27 09:09:07,979][20347] Updated weights for policy 0, policy_version 507368 (0.0010) [2023-12-27 09:09:08,028][20347] Updated weights for policy 0, policy_version 507378 (0.0010) [2023-12-27 09:09:08,077][20347] Updated weights for policy 0, policy_version 507388 (0.0010) [2023-12-27 09:09:08,265][20381] Updated weights for policy 1, policy_version 507158 (0.0005) [2023-12-27 09:09:08,316][20381] Updated weights for policy 1, policy_version 507168 (0.0006) [2023-12-27 09:09:08,376][20381] Updated weights for policy 1, policy_version 507178 (0.0007) [2023-12-27 09:09:08,724][20347] Updated weights for policy 0, policy_version 507398 (0.0008) [2023-12-27 09:09:08,777][20347] Updated weights for policy 0, policy_version 507408 (0.0007) [2023-12-27 09:09:08,846][20347] Updated weights for policy 0, policy_version 507418 (0.0009) [2023-12-27 09:09:09,006][20381] Updated weights for policy 1, policy_version 507188 (0.0006) [2023-12-27 09:09:09,061][20381] Updated weights for policy 1, policy_version 507198 (0.0005) [2023-12-27 09:09:09,118][20381] Updated weights for policy 1, policy_version 507208 (0.0007) [2023-12-27 09:09:09,513][20347] Updated weights for policy 0, policy_version 507428 (0.0009) [2023-12-27 09:09:09,582][20347] Updated weights for policy 0, policy_version 507438 (0.0010) [2023-12-27 09:09:09,646][20347] Updated weights for policy 0, policy_version 507448 (0.0010) [2023-12-27 09:09:09,819][20381] Updated weights for policy 1, policy_version 507218 (0.0008) [2023-12-27 09:09:09,874][20381] Updated weights for policy 1, policy_version 507228 (0.0008) [2023-12-27 09:09:09,936][20381] Updated weights for policy 1, policy_version 507238 (0.0006) [2023-12-27 09:09:09,989][20381] Updated weights for policy 1, policy_version 507248 (0.0005) [2023-12-27 09:09:10,327][20347] Updated weights for policy 0, policy_version 507458 (0.0011) [2023-12-27 09:09:10,389][20347] Updated weights for policy 0, policy_version 507468 (0.0010) [2023-12-27 09:09:10,451][20347] Updated weights for policy 0, policy_version 507478 (0.0010) [2023-12-27 09:09:10,519][20347] Updated weights for policy 0, policy_version 507488 (0.0010) [2023-12-27 09:09:10,594][20381] Updated weights for policy 1, policy_version 507258 (0.0005) [2023-12-27 09:09:10,642][20381] Updated weights for policy 1, policy_version 507268 (0.0009) [2023-12-27 09:09:10,702][20381] Updated weights for policy 1, policy_version 507278 (0.0007) [2023-12-27 09:09:11,194][20347] Updated weights for policy 0, policy_version 507498 (0.0008) [2023-12-27 09:09:11,238][20347] Updated weights for policy 0, policy_version 507508 (0.0008) [2023-12-27 09:09:11,296][20347] Updated weights for policy 0, policy_version 507518 (0.0006) [2023-12-27 09:09:11,384][20381] Updated weights for policy 1, policy_version 507288 (0.0009) [2023-12-27 09:09:11,443][20381] Updated weights for policy 1, policy_version 507298 (0.0007) [2023-12-27 09:09:11,502][20381] Updated weights for policy 1, policy_version 507308 (0.0005) [2023-12-27 09:09:11,990][20347] Updated weights for policy 0, policy_version 507528 (0.0008) [2023-12-27 09:09:12,046][20347] Updated weights for policy 0, policy_version 507538 (0.0009) [2023-12-27 09:09:12,092][20347] Updated weights for policy 0, policy_version 507548 (0.0010) [2023-12-27 09:09:12,171][20381] Updated weights for policy 1, policy_version 507318 (0.0008) [2023-12-27 09:09:12,227][20381] Updated weights for policy 1, policy_version 507328 (0.0011) [2023-12-27 09:09:12,291][20381] Updated weights for policy 1, policy_version 507338 (0.0011) [2023-12-27 09:09:12,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 259850240. Throughput: 0: 10360.8, 1: 10551.9. Samples: 259835968. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:12,363][19170] Avg episode reward: [(0, '971.060'), (1, '1000.000')] [2023-12-27 09:09:12,794][20347] Updated weights for policy 0, policy_version 507558 (0.0009) [2023-12-27 09:09:12,848][20347] Updated weights for policy 0, policy_version 507568 (0.0007) [2023-12-27 09:09:12,905][20347] Updated weights for policy 0, policy_version 507578 (0.0005) [2023-12-27 09:09:13,012][20381] Updated weights for policy 1, policy_version 507348 (0.0011) [2023-12-27 09:09:13,068][20381] Updated weights for policy 1, policy_version 507358 (0.0010) [2023-12-27 09:09:13,116][20381] Updated weights for policy 1, policy_version 507368 (0.0010) [2023-12-27 09:09:13,465][20347] Updated weights for policy 0, policy_version 507588 (0.0005) [2023-12-27 09:09:13,523][20347] Updated weights for policy 0, policy_version 507598 (0.0006) [2023-12-27 09:09:13,570][20347] Updated weights for policy 0, policy_version 507608 (0.0005) [2023-12-27 09:09:13,804][20381] Updated weights for policy 1, policy_version 507378 (0.0011) [2023-12-27 09:09:13,859][20381] Updated weights for policy 1, policy_version 507388 (0.0010) [2023-12-27 09:09:13,908][20381] Updated weights for policy 1, policy_version 507398 (0.0010) [2023-12-27 09:09:13,956][20381] Updated weights for policy 1, policy_version 507408 (0.0010) [2023-12-27 09:09:14,177][20347] Updated weights for policy 0, policy_version 507618 (0.0006) [2023-12-27 09:09:14,233][20347] Updated weights for policy 0, policy_version 507628 (0.0007) [2023-12-27 09:09:14,294][20347] Updated weights for policy 0, policy_version 507638 (0.0007) [2023-12-27 09:09:14,352][20347] Updated weights for policy 0, policy_version 507648 (0.0007) [2023-12-27 09:09:14,655][20381] Updated weights for policy 1, policy_version 507418 (0.0005) [2023-12-27 09:09:14,710][20381] Updated weights for policy 1, policy_version 507428 (0.0005) [2023-12-27 09:09:14,766][20292] KL-divergence is very high: 127.8286 [2023-12-27 09:09:14,774][20381] Updated weights for policy 1, policy_version 507438 (0.0009) [2023-12-27 09:09:14,988][20347] Updated weights for policy 0, policy_version 507658 (0.0007) [2023-12-27 09:09:15,049][20347] Updated weights for policy 0, policy_version 507668 (0.0005) [2023-12-27 09:09:15,110][20347] Updated weights for policy 0, policy_version 507678 (0.0006) [2023-12-27 09:09:15,437][20381] Updated weights for policy 1, policy_version 507448 (0.0010) [2023-12-27 09:09:15,495][20381] Updated weights for policy 1, policy_version 507458 (0.0010) [2023-12-27 09:09:15,549][20381] Updated weights for policy 1, policy_version 507468 (0.0010) [2023-12-27 09:09:15,767][20347] Updated weights for policy 0, policy_version 507688 (0.0007) [2023-12-27 09:09:15,818][20347] Updated weights for policy 0, policy_version 507698 (0.0007) [2023-12-27 09:09:15,872][20347] Updated weights for policy 0, policy_version 507708 (0.0007) [2023-12-27 09:09:16,212][20381] Updated weights for policy 1, policy_version 507478 (0.0010) [2023-12-27 09:09:16,262][20381] Updated weights for policy 1, policy_version 507488 (0.0010) [2023-12-27 09:09:16,309][20381] Updated weights for policy 1, policy_version 507498 (0.0010) [2023-12-27 09:09:16,579][20347] Updated weights for policy 0, policy_version 507718 (0.0007) [2023-12-27 09:09:16,636][20347] Updated weights for policy 0, policy_version 507728 (0.0010) [2023-12-27 09:09:16,694][20347] Updated weights for policy 0, policy_version 507738 (0.0010) [2023-12-27 09:09:17,012][20381] Updated weights for policy 1, policy_version 507508 (0.0009) [2023-12-27 09:09:17,061][20381] Updated weights for policy 1, policy_version 507518 (0.0007) [2023-12-27 09:09:17,114][20381] Updated weights for policy 1, policy_version 507528 (0.0008) [2023-12-27 09:09:17,342][20347] Updated weights for policy 0, policy_version 507748 (0.0008) [2023-12-27 09:09:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 259948544. Throughput: 0: 10396.3, 1: 10493.3. Samples: 259959916. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:17,363][19170] Avg episode reward: [(0, '980.740'), (1, '991.130')] [2023-12-27 09:09:17,394][20347] Updated weights for policy 0, policy_version 507758 (0.0005) [2023-12-27 09:09:17,457][20347] Updated weights for policy 0, policy_version 507768 (0.0005) [2023-12-27 09:09:17,845][20381] Updated weights for policy 1, policy_version 507538 (0.0008) [2023-12-27 09:09:17,910][20381] Updated weights for policy 1, policy_version 507548 (0.0007) [2023-12-27 09:09:17,969][20381] Updated weights for policy 1, policy_version 507558 (0.0008) [2023-12-27 09:09:18,029][20381] Updated weights for policy 1, policy_version 507568 (0.0008) [2023-12-27 09:09:18,096][20347] Updated weights for policy 0, policy_version 507778 (0.0007) [2023-12-27 09:09:18,152][20347] Updated weights for policy 0, policy_version 507788 (0.0008) [2023-12-27 09:09:18,199][20347] Updated weights for policy 0, policy_version 507798 (0.0007) [2023-12-27 09:09:18,246][20347] Updated weights for policy 0, policy_version 507808 (0.0007) [2023-12-27 09:09:18,688][20381] Updated weights for policy 1, policy_version 507578 (0.0008) [2023-12-27 09:09:18,736][20381] Updated weights for policy 1, policy_version 507588 (0.0008) [2023-12-27 09:09:18,779][20381] Updated weights for policy 1, policy_version 507598 (0.0008) [2023-12-27 09:09:18,925][20347] Updated weights for policy 0, policy_version 507818 (0.0005) [2023-12-27 09:09:18,984][20347] Updated weights for policy 0, policy_version 507828 (0.0005) [2023-12-27 09:09:19,037][20347] Updated weights for policy 0, policy_version 507838 (0.0005) [2023-12-27 09:09:19,541][20381] Updated weights for policy 1, policy_version 507608 (0.0008) [2023-12-27 09:09:19,591][20381] Updated weights for policy 1, policy_version 507618 (0.0008) [2023-12-27 09:09:19,648][20381] Updated weights for policy 1, policy_version 507628 (0.0007) [2023-12-27 09:09:19,655][20347] Updated weights for policy 0, policy_version 507848 (0.0005) [2023-12-27 09:09:19,717][20347] Updated weights for policy 0, policy_version 507858 (0.0007) [2023-12-27 09:09:19,783][20347] Updated weights for policy 0, policy_version 507868 (0.0007) [2023-12-27 09:09:20,415][20381] Updated weights for policy 1, policy_version 507638 (0.0007) [2023-12-27 09:09:20,444][20347] Updated weights for policy 0, policy_version 507878 (0.0007) [2023-12-27 09:09:20,471][20381] Updated weights for policy 1, policy_version 507648 (0.0008) [2023-12-27 09:09:20,497][20347] Updated weights for policy 0, policy_version 507888 (0.0006) [2023-12-27 09:09:20,523][20381] Updated weights for policy 1, policy_version 507658 (0.0006) [2023-12-27 09:09:20,551][20347] Updated weights for policy 0, policy_version 507898 (0.0007) [2023-12-27 09:09:21,158][20347] Updated weights for policy 0, policy_version 507908 (0.0008) [2023-12-27 09:09:21,217][20347] Updated weights for policy 0, policy_version 507918 (0.0008) [2023-12-27 09:09:21,281][20381] Updated weights for policy 1, policy_version 507668 (0.0007) [2023-12-27 09:09:21,283][20347] Updated weights for policy 0, policy_version 507928 (0.0008) [2023-12-27 09:09:21,330][20381] Updated weights for policy 1, policy_version 507678 (0.0007) [2023-12-27 09:09:21,394][20381] Updated weights for policy 1, policy_version 507688 (0.0009) [2023-12-27 09:09:22,013][20347] Updated weights for policy 0, policy_version 507938 (0.0008) [2023-12-27 09:09:22,067][20347] Updated weights for policy 0, policy_version 507948 (0.0007) [2023-12-27 09:09:22,108][20381] Updated weights for policy 1, policy_version 507698 (0.0007) [2023-12-27 09:09:22,129][20347] Updated weights for policy 0, policy_version 507958 (0.0007) [2023-12-27 09:09:22,166][20381] Updated weights for policy 1, policy_version 507708 (0.0005) [2023-12-27 09:09:22,185][20347] Updated weights for policy 0, policy_version 507968 (0.0007) [2023-12-27 09:09:22,233][20381] Updated weights for policy 1, policy_version 507718 (0.0005) [2023-12-27 09:09:22,296][20381] Updated weights for policy 1, policy_version 507728 (0.0006) [2023-12-27 09:09:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 260055040. Throughput: 0: 10407.7, 1: 10468.8. Samples: 260020976. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '991.130')] [2023-12-27 09:09:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000507728_129998848.pth... [2023-12-27 09:09:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000507968_130056192.pth... [2023-12-27 09:09:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000506512_129687552.pth [2023-12-27 09:09:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000506720_129736704.pth [2023-12-27 09:09:22,837][20381] Updated weights for policy 1, policy_version 507738 (0.0006) [2023-12-27 09:09:22,896][20381] Updated weights for policy 1, policy_version 507748 (0.0007) [2023-12-27 09:09:22,934][20347] Updated weights for policy 0, policy_version 507978 (0.0010) [2023-12-27 09:09:22,955][20381] Updated weights for policy 1, policy_version 507758 (0.0006) [2023-12-27 09:09:22,990][20347] Updated weights for policy 0, policy_version 507988 (0.0010) [2023-12-27 09:09:23,038][20347] Updated weights for policy 0, policy_version 507998 (0.0010) [2023-12-27 09:09:23,632][20347] Updated weights for policy 0, policy_version 508008 (0.0006) [2023-12-27 09:09:23,653][20381] Updated weights for policy 1, policy_version 507768 (0.0006) [2023-12-27 09:09:23,696][20347] Updated weights for policy 0, policy_version 508018 (0.0006) [2023-12-27 09:09:23,711][20381] Updated weights for policy 1, policy_version 507778 (0.0006) [2023-12-27 09:09:23,747][20347] Updated weights for policy 0, policy_version 508028 (0.0010) [2023-12-27 09:09:23,764][20381] Updated weights for policy 1, policy_version 507788 (0.0005) [2023-12-27 09:09:24,342][20347] Updated weights for policy 0, policy_version 508038 (0.0008) [2023-12-27 09:09:24,361][20381] Updated weights for policy 1, policy_version 507798 (0.0008) [2023-12-27 09:09:24,397][20347] Updated weights for policy 0, policy_version 508048 (0.0006) [2023-12-27 09:09:24,417][20381] Updated weights for policy 1, policy_version 507808 (0.0010) [2023-12-27 09:09:24,452][20347] Updated weights for policy 0, policy_version 508058 (0.0005) [2023-12-27 09:09:24,473][20381] Updated weights for policy 1, policy_version 507818 (0.0010) [2023-12-27 09:09:25,024][20347] Updated weights for policy 0, policy_version 508068 (0.0005) [2023-12-27 09:09:25,074][20347] Updated weights for policy 0, policy_version 508078 (0.0006) [2023-12-27 09:09:25,126][20347] Updated weights for policy 0, policy_version 508088 (0.0008) [2023-12-27 09:09:25,142][20381] Updated weights for policy 1, policy_version 507828 (0.0008) [2023-12-27 09:09:25,196][20381] Updated weights for policy 1, policy_version 507838 (0.0006) [2023-12-27 09:09:25,243][20381] Updated weights for policy 1, policy_version 507848 (0.0008) [2023-12-27 09:09:25,728][20347] Updated weights for policy 0, policy_version 508098 (0.0008) [2023-12-27 09:09:25,789][20347] Updated weights for policy 0, policy_version 508108 (0.0005) [2023-12-27 09:09:25,852][20347] Updated weights for policy 0, policy_version 508118 (0.0005) [2023-12-27 09:09:25,912][20347] Updated weights for policy 0, policy_version 508128 (0.0005) [2023-12-27 09:09:25,974][20381] Updated weights for policy 1, policy_version 507858 (0.0007) [2023-12-27 09:09:26,023][20381] Updated weights for policy 1, policy_version 507868 (0.0008) [2023-12-27 09:09:26,080][20381] Updated weights for policy 1, policy_version 507878 (0.0008) [2023-12-27 09:09:26,127][20381] Updated weights for policy 1, policy_version 507888 (0.0009) [2023-12-27 09:09:26,458][20347] Updated weights for policy 0, policy_version 508138 (0.0008) [2023-12-27 09:09:26,520][20347] Updated weights for policy 0, policy_version 508148 (0.0008) [2023-12-27 09:09:26,573][20347] Updated weights for policy 0, policy_version 508158 (0.0007) [2023-12-27 09:09:26,860][20381] Updated weights for policy 1, policy_version 507898 (0.0008) [2023-12-27 09:09:26,913][20381] Updated weights for policy 1, policy_version 507908 (0.0008) [2023-12-27 09:09:26,963][20381] Updated weights for policy 1, policy_version 507918 (0.0007) [2023-12-27 09:09:27,234][20347] Updated weights for policy 0, policy_version 508168 (0.0008) [2023-12-27 09:09:27,288][20347] Updated weights for policy 0, policy_version 508178 (0.0008) [2023-12-27 09:09:27,343][20347] Updated weights for policy 0, policy_version 508188 (0.0006) [2023-12-27 09:09:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 260153344. Throughput: 0: 10444.4, 1: 10441.6. Samples: 260146864. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '991.130')] [2023-12-27 09:09:27,628][20381] Updated weights for policy 1, policy_version 507928 (0.0007) [2023-12-27 09:09:27,680][20381] Updated weights for policy 1, policy_version 507938 (0.0007) [2023-12-27 09:09:27,735][20292] KL-divergence is very high: 390.9547 [2023-12-27 09:09:27,742][20381] Updated weights for policy 1, policy_version 507948 (0.0008) [2023-12-27 09:09:28,033][20347] Updated weights for policy 0, policy_version 508198 (0.0008) [2023-12-27 09:09:28,089][20347] Updated weights for policy 0, policy_version 508208 (0.0007) [2023-12-27 09:09:28,143][20347] Updated weights for policy 0, policy_version 508218 (0.0007) [2023-12-27 09:09:28,386][20381] Updated weights for policy 1, policy_version 507958 (0.0008) [2023-12-27 09:09:28,443][20381] Updated weights for policy 1, policy_version 507968 (0.0007) [2023-12-27 09:09:28,503][20381] Updated weights for policy 1, policy_version 507978 (0.0007) [2023-12-27 09:09:28,846][20347] Updated weights for policy 0, policy_version 508228 (0.0008) [2023-12-27 09:09:28,897][20347] Updated weights for policy 0, policy_version 508238 (0.0010) [2023-12-27 09:09:28,950][20347] Updated weights for policy 0, policy_version 508248 (0.0010) [2023-12-27 09:09:29,192][20381] Updated weights for policy 1, policy_version 507988 (0.0006) [2023-12-27 09:09:29,254][20381] Updated weights for policy 1, policy_version 507998 (0.0006) [2023-12-27 09:09:29,311][20381] Updated weights for policy 1, policy_version 508008 (0.0007) [2023-12-27 09:09:29,593][20347] Updated weights for policy 0, policy_version 508258 (0.0009) [2023-12-27 09:09:29,643][20347] Updated weights for policy 0, policy_version 508268 (0.0008) [2023-12-27 09:09:29,689][20347] Updated weights for policy 0, policy_version 508278 (0.0008) [2023-12-27 09:09:29,738][20347] Updated weights for policy 0, policy_version 508288 (0.0008) [2023-12-27 09:09:29,968][20381] Updated weights for policy 1, policy_version 508018 (0.0008) [2023-12-27 09:09:30,019][20381] Updated weights for policy 1, policy_version 508028 (0.0007) [2023-12-27 09:09:30,071][20381] Updated weights for policy 1, policy_version 508038 (0.0008) [2023-12-27 09:09:30,119][20381] Updated weights for policy 1, policy_version 508048 (0.0007) [2023-12-27 09:09:30,500][20347] Updated weights for policy 0, policy_version 508298 (0.0007) [2023-12-27 09:09:30,548][20347] Updated weights for policy 0, policy_version 508308 (0.0007) [2023-12-27 09:09:30,600][20347] Updated weights for policy 0, policy_version 508318 (0.0005) [2023-12-27 09:09:30,773][20381] Updated weights for policy 1, policy_version 508058 (0.0010) [2023-12-27 09:09:30,829][20381] Updated weights for policy 1, policy_version 508068 (0.0010) [2023-12-27 09:09:30,880][20381] Updated weights for policy 1, policy_version 508078 (0.0010) [2023-12-27 09:09:31,240][20347] Updated weights for policy 0, policy_version 508328 (0.0007) [2023-12-27 09:09:31,295][20347] Updated weights for policy 0, policy_version 508338 (0.0006) [2023-12-27 09:09:31,351][20347] Updated weights for policy 0, policy_version 508348 (0.0007) [2023-12-27 09:09:31,585][20381] Updated weights for policy 1, policy_version 508088 (0.0010) [2023-12-27 09:09:31,643][20381] Updated weights for policy 1, policy_version 508098 (0.0010) [2023-12-27 09:09:31,702][20381] Updated weights for policy 1, policy_version 508108 (0.0010) [2023-12-27 09:09:31,968][20347] Updated weights for policy 0, policy_version 508358 (0.0006) [2023-12-27 09:09:32,027][20347] Updated weights for policy 0, policy_version 508368 (0.0008) [2023-12-27 09:09:32,086][20347] Updated weights for policy 0, policy_version 508378 (0.0007) [2023-12-27 09:09:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 260259840. Throughput: 0: 10518.8, 1: 10334.2. Samples: 260271124. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '991.130')] [2023-12-27 09:09:32,382][20381] Updated weights for policy 1, policy_version 508118 (0.0010) [2023-12-27 09:09:32,440][20381] Updated weights for policy 1, policy_version 508128 (0.0010) [2023-12-27 09:09:32,500][20381] Updated weights for policy 1, policy_version 508138 (0.0010) [2023-12-27 09:09:32,741][20347] Updated weights for policy 0, policy_version 508388 (0.0007) [2023-12-27 09:09:32,805][20347] Updated weights for policy 0, policy_version 508398 (0.0008) [2023-12-27 09:09:32,864][20347] Updated weights for policy 0, policy_version 508408 (0.0008) [2023-12-27 09:09:33,151][20381] Updated weights for policy 1, policy_version 508148 (0.0008) [2023-12-27 09:09:33,207][20381] Updated weights for policy 1, policy_version 508158 (0.0006) [2023-12-27 09:09:33,262][20381] Updated weights for policy 1, policy_version 508168 (0.0005) [2023-12-27 09:09:33,593][20347] Updated weights for policy 0, policy_version 508418 (0.0007) [2023-12-27 09:09:33,652][20347] Updated weights for policy 0, policy_version 508428 (0.0008) [2023-12-27 09:09:33,712][20347] Updated weights for policy 0, policy_version 508438 (0.0008) [2023-12-27 09:09:33,774][20381] Updated weights for policy 1, policy_version 508178 (0.0006) [2023-12-27 09:09:33,783][20347] Updated weights for policy 0, policy_version 508448 (0.0008) [2023-12-27 09:09:33,836][20381] Updated weights for policy 1, policy_version 508188 (0.0007) [2023-12-27 09:09:33,889][20381] Updated weights for policy 1, policy_version 508198 (0.0008) [2023-12-27 09:09:33,938][20381] Updated weights for policy 1, policy_version 508208 (0.0008) [2023-12-27 09:09:34,462][20347] Updated weights for policy 0, policy_version 508458 (0.0007) [2023-12-27 09:09:34,517][20347] Updated weights for policy 0, policy_version 508468 (0.0010) [2023-12-27 09:09:34,582][20347] Updated weights for policy 0, policy_version 508478 (0.0007) [2023-12-27 09:09:34,657][20381] Updated weights for policy 1, policy_version 508218 (0.0008) [2023-12-27 09:09:34,708][20381] Updated weights for policy 1, policy_version 508229 (0.0006) [2023-12-27 09:09:34,760][20381] Updated weights for policy 1, policy_version 508239 (0.0005) [2023-12-27 09:09:35,212][20347] Updated weights for policy 0, policy_version 508488 (0.0007) [2023-12-27 09:09:35,265][20347] Updated weights for policy 0, policy_version 508498 (0.0007) [2023-12-27 09:09:35,326][20347] Updated weights for policy 0, policy_version 508508 (0.0008) [2023-12-27 09:09:35,438][20381] Updated weights for policy 1, policy_version 508249 (0.0007) [2023-12-27 09:09:35,495][20381] Updated weights for policy 1, policy_version 508259 (0.0008) [2023-12-27 09:09:35,542][20381] Updated weights for policy 1, policy_version 508269 (0.0007) [2023-12-27 09:09:36,047][20347] Updated weights for policy 0, policy_version 508518 (0.0006) [2023-12-27 09:09:36,103][20347] Updated weights for policy 0, policy_version 508528 (0.0007) [2023-12-27 09:09:36,161][20381] Updated weights for policy 1, policy_version 508279 (0.0008) [2023-12-27 09:09:36,167][20347] Updated weights for policy 0, policy_version 508538 (0.0007) [2023-12-27 09:09:36,210][20381] Updated weights for policy 1, policy_version 508289 (0.0008) [2023-12-27 09:09:36,265][20381] Updated weights for policy 1, policy_version 508299 (0.0008) [2023-12-27 09:09:36,878][20347] Updated weights for policy 0, policy_version 508548 (0.0008) [2023-12-27 09:09:36,924][20347] Updated weights for policy 0, policy_version 508558 (0.0010) [2023-12-27 09:09:36,936][20381] Updated weights for policy 1, policy_version 508309 (0.0009) [2023-12-27 09:09:36,975][20347] Updated weights for policy 0, policy_version 508568 (0.0010) [2023-12-27 09:09:36,982][20381] Updated weights for policy 1, policy_version 508319 (0.0010) [2023-12-27 09:09:37,035][20381] Updated weights for policy 1, policy_version 508329 (0.0010) [2023-12-27 09:09:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 260366336. Throughput: 0: 10495.8, 1: 10326.4. Samples: 260333844. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:37,363][19170] Avg episode reward: [(0, '951.820'), (1, '1000.000')] [2023-12-27 09:09:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000508336_130154496.pth... [2023-12-27 09:09:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000508576_130211840.pth... [2023-12-27 09:09:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000507360_129900544.pth [2023-12-27 09:09:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000507120_129843200.pth [2023-12-27 09:09:37,635][20347] Updated weights for policy 0, policy_version 508578 (0.0009) [2023-12-27 09:09:37,688][20381] Updated weights for policy 1, policy_version 508339 (0.0009) [2023-12-27 09:09:37,689][20347] Updated weights for policy 0, policy_version 508588 (0.0006) [2023-12-27 09:09:37,745][20347] Updated weights for policy 0, policy_version 508598 (0.0006) [2023-12-27 09:09:37,745][20381] Updated weights for policy 1, policy_version 508349 (0.0008) [2023-12-27 09:09:37,791][20347] Updated weights for policy 0, policy_version 508608 (0.0010) [2023-12-27 09:09:37,804][20381] Updated weights for policy 1, policy_version 508359 (0.0007) [2023-12-27 09:09:38,394][20347] Updated weights for policy 0, policy_version 508618 (0.0007) [2023-12-27 09:09:38,456][20347] Updated weights for policy 0, policy_version 508628 (0.0011) [2023-12-27 09:09:38,494][20381] Updated weights for policy 1, policy_version 508369 (0.0007) [2023-12-27 09:09:38,514][20347] Updated weights for policy 0, policy_version 508638 (0.0010) [2023-12-27 09:09:38,551][20381] Updated weights for policy 1, policy_version 508379 (0.0006) [2023-12-27 09:09:38,608][20381] Updated weights for policy 1, policy_version 508389 (0.0007) [2023-12-27 09:09:38,662][20381] Updated weights for policy 1, policy_version 508399 (0.0008) [2023-12-27 09:09:39,170][20347] Updated weights for policy 0, policy_version 508648 (0.0011) [2023-12-27 09:09:39,236][20347] Updated weights for policy 0, policy_version 508658 (0.0009) [2023-12-27 09:09:39,292][20347] Updated weights for policy 0, policy_version 508668 (0.0006) [2023-12-27 09:09:39,346][20381] Updated weights for policy 1, policy_version 508409 (0.0008) [2023-12-27 09:09:39,411][20381] Updated weights for policy 1, policy_version 508419 (0.0008) [2023-12-27 09:09:39,466][20381] Updated weights for policy 1, policy_version 508429 (0.0008) [2023-12-27 09:09:40,026][20347] Updated weights for policy 0, policy_version 508678 (0.0006) [2023-12-27 09:09:40,077][20347] Updated weights for policy 0, policy_version 508688 (0.0006) [2023-12-27 09:09:40,132][20347] Updated weights for policy 0, policy_version 508698 (0.0006) [2023-12-27 09:09:40,202][20381] Updated weights for policy 1, policy_version 508439 (0.0009) [2023-12-27 09:09:40,266][20381] Updated weights for policy 1, policy_version 508449 (0.0007) [2023-12-27 09:09:40,322][20381] Updated weights for policy 1, policy_version 508459 (0.0008) [2023-12-27 09:09:40,819][20347] Updated weights for policy 0, policy_version 508708 (0.0008) [2023-12-27 09:09:40,873][20347] Updated weights for policy 0, policy_version 508718 (0.0008) [2023-12-27 09:09:40,924][20347] Updated weights for policy 0, policy_version 508728 (0.0008) [2023-12-27 09:09:41,047][20381] Updated weights for policy 1, policy_version 508469 (0.0008) [2023-12-27 09:09:41,111][20381] Updated weights for policy 1, policy_version 508479 (0.0006) [2023-12-27 09:09:41,173][20381] Updated weights for policy 1, policy_version 508489 (0.0007) [2023-12-27 09:09:41,647][20347] Updated weights for policy 0, policy_version 508738 (0.0007) [2023-12-27 09:09:41,706][20347] Updated weights for policy 0, policy_version 508748 (0.0008) [2023-12-27 09:09:41,772][20347] Updated weights for policy 0, policy_version 508758 (0.0007) [2023-12-27 09:09:41,832][20347] Updated weights for policy 0, policy_version 508768 (0.0007) [2023-12-27 09:09:41,897][20381] Updated weights for policy 1, policy_version 508499 (0.0008) [2023-12-27 09:09:41,957][20381] Updated weights for policy 1, policy_version 508509 (0.0006) [2023-12-27 09:09:42,016][20381] Updated weights for policy 1, policy_version 508519 (0.0006) [2023-12-27 09:09:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 260464640. Throughput: 0: 10431.5, 1: 10286.7. Samples: 260455992. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:42,363][19170] Avg episode reward: [(0, '951.750'), (1, '952.420')] [2023-12-27 09:09:42,498][20347] Updated weights for policy 0, policy_version 508778 (0.0008) [2023-12-27 09:09:42,548][20347] Updated weights for policy 0, policy_version 508788 (0.0007) [2023-12-27 09:09:42,605][20347] Updated weights for policy 0, policy_version 508798 (0.0008) [2023-12-27 09:09:42,611][20381] Updated weights for policy 1, policy_version 508529 (0.0010) [2023-12-27 09:09:42,671][20381] Updated weights for policy 1, policy_version 508539 (0.0006) [2023-12-27 09:09:42,721][20381] Updated weights for policy 1, policy_version 508549 (0.0005) [2023-12-27 09:09:42,770][20381] Updated weights for policy 1, policy_version 508559 (0.0005) [2023-12-27 09:09:43,246][20347] Updated weights for policy 0, policy_version 508808 (0.0009) [2023-12-27 09:09:43,285][20291] KL-divergence is very high: 113.8838 [2023-12-27 09:09:43,312][20347] Updated weights for policy 0, policy_version 508818 (0.0008) [2023-12-27 09:09:43,372][20347] Updated weights for policy 0, policy_version 508828 (0.0009) [2023-12-27 09:09:43,393][20381] Updated weights for policy 1, policy_version 508569 (0.0010) [2023-12-27 09:09:43,451][20381] Updated weights for policy 1, policy_version 508579 (0.0010) [2023-12-27 09:09:43,509][20381] Updated weights for policy 1, policy_version 508589 (0.0010) [2023-12-27 09:09:43,931][20347] Updated weights for policy 0, policy_version 508838 (0.0007) [2023-12-27 09:09:43,987][20347] Updated weights for policy 0, policy_version 508848 (0.0007) [2023-12-27 09:09:44,031][20347] Updated weights for policy 0, policy_version 508858 (0.0007) [2023-12-27 09:09:44,193][20381] Updated weights for policy 1, policy_version 508599 (0.0010) [2023-12-27 09:09:44,255][20381] Updated weights for policy 1, policy_version 508609 (0.0008) [2023-12-27 09:09:44,312][20381] Updated weights for policy 1, policy_version 508619 (0.0010) [2023-12-27 09:09:44,682][20347] Updated weights for policy 0, policy_version 508868 (0.0007) [2023-12-27 09:09:44,736][20347] Updated weights for policy 0, policy_version 508878 (0.0007) [2023-12-27 09:09:44,801][20347] Updated weights for policy 0, policy_version 508888 (0.0007) [2023-12-27 09:09:44,973][20381] Updated weights for policy 1, policy_version 508629 (0.0006) [2023-12-27 09:09:45,038][20381] Updated weights for policy 1, policy_version 508639 (0.0008) [2023-12-27 09:09:45,100][20381] Updated weights for policy 1, policy_version 508649 (0.0008) [2023-12-27 09:09:45,413][20347] Updated weights for policy 0, policy_version 508898 (0.0006) [2023-12-27 09:09:45,477][20347] Updated weights for policy 0, policy_version 508908 (0.0008) [2023-12-27 09:09:45,532][20347] Updated weights for policy 0, policy_version 508918 (0.0007) [2023-12-27 09:09:45,577][20347] Updated weights for policy 0, policy_version 508928 (0.0007) [2023-12-27 09:09:45,793][20381] Updated weights for policy 1, policy_version 508659 (0.0008) [2023-12-27 09:09:45,849][20381] Updated weights for policy 1, policy_version 508669 (0.0010) [2023-12-27 09:09:45,909][20381] Updated weights for policy 1, policy_version 508679 (0.0007) [2023-12-27 09:09:46,194][20291] KL-divergence is very high: 179.6447 [2023-12-27 09:09:46,206][20347] Updated weights for policy 0, policy_version 508938 (0.0007) [2023-12-27 09:09:46,212][20291] KL-divergence is very high: 460.9901 [2023-12-27 09:09:46,224][20291] KL-divergence is very high: 154.7919 [2023-12-27 09:09:46,258][20291] KL-divergence is very high: 224.8893 [2023-12-27 09:09:46,264][20347] Updated weights for policy 0, policy_version 508948 (0.0007) [2023-12-27 09:09:46,270][20291] KL-divergence is very high: 193.4661 [2023-12-27 09:09:46,315][20347] Updated weights for policy 0, policy_version 508958 (0.0007) [2023-12-27 09:09:46,593][20381] Updated weights for policy 1, policy_version 508689 (0.0008) [2023-12-27 09:09:46,656][20381] Updated weights for policy 1, policy_version 508699 (0.0005) [2023-12-27 09:09:46,713][20381] Updated weights for policy 1, policy_version 508709 (0.0006) [2023-12-27 09:09:46,763][20381] Updated weights for policy 1, policy_version 508719 (0.0007) [2023-12-27 09:09:47,001][20347] Updated weights for policy 0, policy_version 508968 (0.0009) [2023-12-27 09:09:47,053][20347] Updated weights for policy 0, policy_version 508978 (0.0010) [2023-12-27 09:09:47,107][20347] Updated weights for policy 0, policy_version 508988 (0.0008) [2023-12-27 09:09:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 260571136. Throughput: 0: 10469.1, 1: 10306.7. Samples: 260582932. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 09:09:47,363][19170] Avg episode reward: [(0, '972.360'), (1, '893.740')] [2023-12-27 09:09:47,365][20381] Updated weights for policy 1, policy_version 508729 (0.0005) [2023-12-27 09:09:47,418][20381] Updated weights for policy 1, policy_version 508739 (0.0007) [2023-12-27 09:09:47,471][20381] Updated weights for policy 1, policy_version 508749 (0.0007) [2023-12-27 09:09:47,696][20347] Updated weights for policy 0, policy_version 508998 (0.0005) [2023-12-27 09:09:47,749][20347] Updated weights for policy 0, policy_version 509008 (0.0009) [2023-12-27 09:09:47,806][20347] Updated weights for policy 0, policy_version 509018 (0.0011) [2023-12-27 09:09:48,052][20381] Updated weights for policy 1, policy_version 508759 (0.0007) [2023-12-27 09:09:48,110][20381] Updated weights for policy 1, policy_version 508769 (0.0008) [2023-12-27 09:09:48,168][20381] Updated weights for policy 1, policy_version 508779 (0.0007) [2023-12-27 09:09:48,490][20347] Updated weights for policy 0, policy_version 509028 (0.0009) [2023-12-27 09:09:48,549][20347] Updated weights for policy 0, policy_version 509038 (0.0009) [2023-12-27 09:09:48,600][20347] Updated weights for policy 0, policy_version 509048 (0.0010) [2023-12-27 09:09:48,816][20381] Updated weights for policy 1, policy_version 508789 (0.0007) [2023-12-27 09:09:48,873][20381] Updated weights for policy 1, policy_version 508799 (0.0006) [2023-12-27 09:09:48,932][20381] Updated weights for policy 1, policy_version 508809 (0.0008) [2023-12-27 09:09:49,260][20347] Updated weights for policy 0, policy_version 509058 (0.0010) [2023-12-27 09:09:49,320][20347] Updated weights for policy 0, policy_version 509068 (0.0007) [2023-12-27 09:09:49,387][20347] Updated weights for policy 0, policy_version 509078 (0.0008) [2023-12-27 09:09:49,453][20347] Updated weights for policy 0, policy_version 509088 (0.0009) [2023-12-27 09:09:49,642][20381] Updated weights for policy 1, policy_version 508819 (0.0009) [2023-12-27 09:09:49,698][20381] Updated weights for policy 1, policy_version 508829 (0.0011) [2023-12-27 09:09:49,760][20381] Updated weights for policy 1, policy_version 508839 (0.0011) [2023-12-27 09:09:50,110][20347] Updated weights for policy 0, policy_version 509098 (0.0008) [2023-12-27 09:09:50,167][20347] Updated weights for policy 0, policy_version 509108 (0.0008) [2023-12-27 09:09:50,221][20347] Updated weights for policy 0, policy_version 509118 (0.0011) [2023-12-27 09:09:50,482][20381] Updated weights for policy 1, policy_version 508849 (0.0011) [2023-12-27 09:09:50,529][20381] Updated weights for policy 1, policy_version 508859 (0.0010) [2023-12-27 09:09:50,583][20381] Updated weights for policy 1, policy_version 508869 (0.0010) [2023-12-27 09:09:50,641][20381] Updated weights for policy 1, policy_version 508879 (0.0008) [2023-12-27 09:09:50,916][20347] Updated weights for policy 0, policy_version 509128 (0.0008) [2023-12-27 09:09:50,966][20347] Updated weights for policy 0, policy_version 509138 (0.0008) [2023-12-27 09:09:51,015][20347] Updated weights for policy 0, policy_version 509148 (0.0009) [2023-12-27 09:09:51,306][20381] Updated weights for policy 1, policy_version 508889 (0.0008) [2023-12-27 09:09:51,368][20381] Updated weights for policy 1, policy_version 508899 (0.0008) [2023-12-27 09:09:51,432][20381] Updated weights for policy 1, policy_version 508909 (0.0011) [2023-12-27 09:09:51,783][20347] Updated weights for policy 0, policy_version 509159 (0.0008) [2023-12-27 09:09:51,830][20347] Updated weights for policy 0, policy_version 509169 (0.0007) [2023-12-27 09:09:51,874][20347] Updated weights for policy 0, policy_version 509179 (0.0007) [2023-12-27 09:09:52,138][20381] Updated weights for policy 1, policy_version 508919 (0.0007) [2023-12-27 09:09:52,202][20381] Updated weights for policy 1, policy_version 508929 (0.0006) [2023-12-27 09:09:52,262][20381] Updated weights for policy 1, policy_version 508939 (0.0007) [2023-12-27 09:09:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.8, 300 sec: 20771.6). Total num frames: 260677632. Throughput: 0: 10494.5, 1: 10304.2. Samples: 260645592. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:09:52,363][19170] Avg episode reward: [(0, '982.000'), (1, '914.700')] [2023-12-27 09:09:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000508944_130310144.pth... [2023-12-27 09:09:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000509184_130367488.pth... [2023-12-27 09:09:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000507728_129998848.pth [2023-12-27 09:09:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000507968_130056192.pth [2023-12-27 09:09:52,628][20347] Updated weights for policy 0, policy_version 509189 (0.0008) [2023-12-27 09:09:52,684][20347] Updated weights for policy 0, policy_version 509199 (0.0008) [2023-12-27 09:09:52,737][20347] Updated weights for policy 0, policy_version 509209 (0.0008) [2023-12-27 09:09:52,906][20381] Updated weights for policy 1, policy_version 508949 (0.0007) [2023-12-27 09:09:52,956][20381] Updated weights for policy 1, policy_version 508959 (0.0008) [2023-12-27 09:09:53,012][20381] Updated weights for policy 1, policy_version 508969 (0.0008) [2023-12-27 09:09:53,425][20347] Updated weights for policy 0, policy_version 509219 (0.0007) [2023-12-27 09:09:53,484][20347] Updated weights for policy 0, policy_version 509229 (0.0005) [2023-12-27 09:09:53,548][20347] Updated weights for policy 0, policy_version 509239 (0.0005) [2023-12-27 09:09:53,740][20381] Updated weights for policy 1, policy_version 508979 (0.0009) [2023-12-27 09:09:53,802][20381] Updated weights for policy 1, policy_version 508989 (0.0011) [2023-12-27 09:09:53,860][20381] Updated weights for policy 1, policy_version 508999 (0.0010) [2023-12-27 09:09:54,159][20347] Updated weights for policy 0, policy_version 509249 (0.0008) [2023-12-27 09:09:54,207][20347] Updated weights for policy 0, policy_version 509259 (0.0007) [2023-12-27 09:09:54,258][20347] Updated weights for policy 0, policy_version 509269 (0.0007) [2023-12-27 09:09:54,308][20347] Updated weights for policy 0, policy_version 509279 (0.0007) [2023-12-27 09:09:54,527][20381] Updated weights for policy 1, policy_version 509009 (0.0010) [2023-12-27 09:09:54,572][20381] Updated weights for policy 1, policy_version 509019 (0.0010) [2023-12-27 09:09:54,617][20381] Updated weights for policy 1, policy_version 509029 (0.0010) [2023-12-27 09:09:54,663][20381] Updated weights for policy 1, policy_version 509039 (0.0010) [2023-12-27 09:09:55,020][20347] Updated weights for policy 0, policy_version 509289 (0.0008) [2023-12-27 09:09:55,074][20347] Updated weights for policy 0, policy_version 509299 (0.0008) [2023-12-27 09:09:55,137][20347] Updated weights for policy 0, policy_version 509309 (0.0007) [2023-12-27 09:09:55,377][20381] Updated weights for policy 1, policy_version 509049 (0.0010) [2023-12-27 09:09:55,432][20381] Updated weights for policy 1, policy_version 509059 (0.0011) [2023-12-27 09:09:55,486][20381] Updated weights for policy 1, policy_version 509069 (0.0010) [2023-12-27 09:09:55,815][20347] Updated weights for policy 0, policy_version 509319 (0.0006) [2023-12-27 09:09:55,871][20347] Updated weights for policy 0, policy_version 509329 (0.0005) [2023-12-27 09:09:55,922][20347] Updated weights for policy 0, policy_version 509339 (0.0005) [2023-12-27 09:09:56,148][20381] Updated weights for policy 1, policy_version 509079 (0.0007) [2023-12-27 09:09:56,213][20381] Updated weights for policy 1, policy_version 509089 (0.0006) [2023-12-27 09:09:56,270][20381] Updated weights for policy 1, policy_version 509099 (0.0005) [2023-12-27 09:09:56,528][20347] Updated weights for policy 0, policy_version 509349 (0.0006) [2023-12-27 09:09:56,549][20291] KL-divergence is very high: 202.8183 [2023-12-27 09:09:56,580][20347] Updated weights for policy 0, policy_version 509359 (0.0007) [2023-12-27 09:09:56,628][20347] Updated weights for policy 0, policy_version 509369 (0.0008) [2023-12-27 09:09:56,814][20381] Updated weights for policy 1, policy_version 509109 (0.0005) [2023-12-27 09:09:56,861][20381] Updated weights for policy 1, policy_version 509119 (0.0005) [2023-12-27 09:09:56,915][20381] Updated weights for policy 1, policy_version 509129 (0.0005) [2023-12-27 09:09:57,335][20347] Updated weights for policy 0, policy_version 509379 (0.0007) [2023-12-27 09:09:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 260775936. Throughput: 0: 10458.7, 1: 10246.1. Samples: 260767684. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:09:57,363][19170] Avg episode reward: [(0, '995.500'), (1, '957.690')] [2023-12-27 09:09:57,392][20347] Updated weights for policy 0, policy_version 509389 (0.0010) [2023-12-27 09:09:57,446][20347] Updated weights for policy 0, policy_version 509399 (0.0010) [2023-12-27 09:09:57,481][20381] Updated weights for policy 1, policy_version 509139 (0.0005) [2023-12-27 09:09:57,541][20381] Updated weights for policy 1, policy_version 509149 (0.0005) [2023-12-27 09:09:57,592][20381] Updated weights for policy 1, policy_version 509159 (0.0008) [2023-12-27 09:09:58,099][20347] Updated weights for policy 0, policy_version 509409 (0.0010) [2023-12-27 09:09:58,157][20347] Updated weights for policy 0, policy_version 509419 (0.0008) [2023-12-27 09:09:58,191][20381] Updated weights for policy 1, policy_version 509169 (0.0010) [2023-12-27 09:09:58,218][20347] Updated weights for policy 0, policy_version 509429 (0.0008) [2023-12-27 09:09:58,256][20381] Updated weights for policy 1, policy_version 509179 (0.0008) [2023-12-27 09:09:58,282][20347] Updated weights for policy 0, policy_version 509439 (0.0010) [2023-12-27 09:09:58,317][20381] Updated weights for policy 1, policy_version 509189 (0.0007) [2023-12-27 09:09:58,385][20381] Updated weights for policy 1, policy_version 509199 (0.0009) [2023-12-27 09:09:59,017][20347] Updated weights for policy 0, policy_version 509449 (0.0010) [2023-12-27 09:09:59,065][20347] Updated weights for policy 0, policy_version 509459 (0.0009) [2023-12-27 09:09:59,106][20381] Updated weights for policy 1, policy_version 509209 (0.0008) [2023-12-27 09:09:59,119][20347] Updated weights for policy 0, policy_version 509469 (0.0010) [2023-12-27 09:09:59,163][20381] Updated weights for policy 1, policy_version 509219 (0.0006) [2023-12-27 09:09:59,216][20381] Updated weights for policy 1, policy_version 509229 (0.0005) [2023-12-27 09:09:59,808][20347] Updated weights for policy 0, policy_version 509479 (0.0008) [2023-12-27 09:09:59,863][20381] Updated weights for policy 1, policy_version 509239 (0.0008) [2023-12-27 09:09:59,870][20347] Updated weights for policy 0, policy_version 509489 (0.0007) [2023-12-27 09:09:59,921][20381] Updated weights for policy 1, policy_version 509249 (0.0009) [2023-12-27 09:09:59,929][20347] Updated weights for policy 0, policy_version 509499 (0.0007) [2023-12-27 09:09:59,981][20381] Updated weights for policy 1, policy_version 509259 (0.0006) [2023-12-27 09:10:00,605][20381] Updated weights for policy 1, policy_version 509269 (0.0008) [2023-12-27 09:10:00,641][20347] Updated weights for policy 0, policy_version 509509 (0.0007) [2023-12-27 09:10:00,656][20381] Updated weights for policy 1, policy_version 509279 (0.0010) [2023-12-27 09:10:00,675][20291] KL-divergence is very high: 205.8174 [2023-12-27 09:10:00,686][20291] KL-divergence is very high: 148.3038 [2023-12-27 09:10:00,695][20347] Updated weights for policy 0, policy_version 509519 (0.0007) [2023-12-27 09:10:00,711][20381] Updated weights for policy 1, policy_version 509289 (0.0010) [2023-12-27 09:10:00,742][20347] Updated weights for policy 0, policy_version 509529 (0.0006) [2023-12-27 09:10:00,751][20291] KL-divergence is very high: 373.3934 [2023-12-27 09:10:00,760][20291] KL-divergence is very high: 350.3304 [2023-12-27 09:10:01,332][20381] Updated weights for policy 1, policy_version 509299 (0.0007) [2023-12-27 09:10:01,394][20347] Updated weights for policy 0, policy_version 509539 (0.0007) [2023-12-27 09:10:01,402][20381] Updated weights for policy 1, policy_version 509309 (0.0008) [2023-12-27 09:10:01,454][20347] Updated weights for policy 0, policy_version 509549 (0.0006) [2023-12-27 09:10:01,456][20381] Updated weights for policy 1, policy_version 509319 (0.0006) [2023-12-27 09:10:01,513][20347] Updated weights for policy 0, policy_version 509559 (0.0007) [2023-12-27 09:10:02,075][20381] Updated weights for policy 1, policy_version 509329 (0.0005) [2023-12-27 09:10:02,138][20381] Updated weights for policy 1, policy_version 509339 (0.0008) [2023-12-27 09:10:02,185][20381] Updated weights for policy 1, policy_version 509349 (0.0007) [2023-12-27 09:10:02,234][20381] Updated weights for policy 1, policy_version 509359 (0.0007) [2023-12-27 09:10:02,256][20347] Updated weights for policy 0, policy_version 509569 (0.0008) [2023-12-27 09:10:02,319][20347] Updated weights for policy 0, policy_version 509579 (0.0010) [2023-12-27 09:10:02,324][20291] KL-divergence is very high: 1492.5830 [2023-12-27 09:10:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 260882432. Throughput: 0: 10386.8, 1: 10361.9. Samples: 260893604. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:02,363][19170] Avg episode reward: [(0, '966.630'), (1, '970.920')] [2023-12-27 09:10:02,375][20291] KL-divergence is very high: 301.2238 [2023-12-27 09:10:02,380][20347] Updated weights for policy 0, policy_version 509589 (0.0008) [2023-12-27 09:10:02,441][20347] Updated weights for policy 0, policy_version 509599 (0.0008) [2023-12-27 09:10:02,956][20381] Updated weights for policy 1, policy_version 509369 (0.0007) [2023-12-27 09:10:03,010][20381] Updated weights for policy 1, policy_version 509379 (0.0007) [2023-12-27 09:10:03,037][20347] Updated weights for policy 0, policy_version 509609 (0.0006) [2023-12-27 09:10:03,065][20381] Updated weights for policy 1, policy_version 509389 (0.0005) [2023-12-27 09:10:03,082][20347] Updated weights for policy 0, policy_version 509619 (0.0005) [2023-12-27 09:10:03,131][20347] Updated weights for policy 0, policy_version 509629 (0.0007) [2023-12-27 09:10:03,749][20381] Updated weights for policy 1, policy_version 509399 (0.0007) [2023-12-27 09:10:03,766][20347] Updated weights for policy 0, policy_version 509639 (0.0007) [2023-12-27 09:10:03,798][20381] Updated weights for policy 1, policy_version 509409 (0.0005) [2023-12-27 09:10:03,825][20347] Updated weights for policy 0, policy_version 509649 (0.0006) [2023-12-27 09:10:03,848][20381] Updated weights for policy 1, policy_version 509419 (0.0008) [2023-12-27 09:10:03,895][20347] Updated weights for policy 0, policy_version 509659 (0.0008) [2023-12-27 09:10:04,550][20381] Updated weights for policy 1, policy_version 509429 (0.0006) [2023-12-27 09:10:04,561][20347] Updated weights for policy 0, policy_version 509669 (0.0006) [2023-12-27 09:10:04,609][20381] Updated weights for policy 1, policy_version 509439 (0.0005) [2023-12-27 09:10:04,622][20347] Updated weights for policy 0, policy_version 509679 (0.0008) [2023-12-27 09:10:04,671][20381] Updated weights for policy 1, policy_version 509449 (0.0006) [2023-12-27 09:10:04,686][20347] Updated weights for policy 0, policy_version 509689 (0.0008) [2023-12-27 09:10:05,252][20381] Updated weights for policy 1, policy_version 509459 (0.0007) [2023-12-27 09:10:05,305][20381] Updated weights for policy 1, policy_version 509469 (0.0010) [2023-12-27 09:10:05,347][20347] Updated weights for policy 0, policy_version 509699 (0.0006) [2023-12-27 09:10:05,354][20381] Updated weights for policy 1, policy_version 509479 (0.0010) [2023-12-27 09:10:05,374][20291] KL-divergence is very high: 126.9217 [2023-12-27 09:10:05,380][20291] KL-divergence is very high: 109.0610 [2023-12-27 09:10:05,400][20347] Updated weights for policy 0, policy_version 509709 (0.0005) [2023-12-27 09:10:05,454][20347] Updated weights for policy 0, policy_version 509719 (0.0005) [2023-12-27 09:10:05,966][20381] Updated weights for policy 1, policy_version 509489 (0.0011) [2023-12-27 09:10:06,025][20381] Updated weights for policy 1, policy_version 509499 (0.0011) [2023-12-27 09:10:06,073][20347] Updated weights for policy 0, policy_version 509729 (0.0006) [2023-12-27 09:10:06,084][20381] Updated weights for policy 1, policy_version 509509 (0.0010) [2023-12-27 09:10:06,135][20347] Updated weights for policy 0, policy_version 509739 (0.0007) [2023-12-27 09:10:06,145][20381] Updated weights for policy 1, policy_version 509519 (0.0011) [2023-12-27 09:10:06,185][20347] Updated weights for policy 0, policy_version 509749 (0.0006) [2023-12-27 09:10:06,243][20347] Updated weights for policy 0, policy_version 509759 (0.0005) [2023-12-27 09:10:06,819][20381] Updated weights for policy 1, policy_version 509529 (0.0011) [2023-12-27 09:10:06,869][20381] Updated weights for policy 1, policy_version 509539 (0.0011) [2023-12-27 09:10:06,904][20347] Updated weights for policy 0, policy_version 509769 (0.0007) [2023-12-27 09:10:06,922][20381] Updated weights for policy 1, policy_version 509549 (0.0011) [2023-12-27 09:10:06,957][20347] Updated weights for policy 0, policy_version 509779 (0.0006) [2023-12-27 09:10:06,958][20291] KL-divergence is very high: 117.5483 [2023-12-27 09:10:06,989][20291] KL-divergence is very high: 228.2318 [2023-12-27 09:10:07,008][20291] KL-divergence is very high: 102.6642 [2023-12-27 09:10:07,021][20347] Updated weights for policy 0, policy_version 509789 (0.0006) [2023-12-27 09:10:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 260988928. Throughput: 0: 10389.6, 1: 10400.1. Samples: 260956512. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:07,363][19170] Avg episode reward: [(0, '865.530'), (1, '961.200')] [2023-12-27 09:10:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000509552_130465792.pth... [2023-12-27 09:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000509792_130523136.pth... [2023-12-27 09:10:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000508336_130154496.pth [2023-12-27 09:10:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000508576_130211840.pth [2023-12-27 09:10:07,629][20381] Updated weights for policy 1, policy_version 509559 (0.0009) [2023-12-27 09:10:07,684][20381] Updated weights for policy 1, policy_version 509569 (0.0006) [2023-12-27 09:10:07,704][20347] Updated weights for policy 0, policy_version 509799 (0.0007) [2023-12-27 09:10:07,741][20381] Updated weights for policy 1, policy_version 509579 (0.0005) [2023-12-27 09:10:07,764][20347] Updated weights for policy 0, policy_version 509809 (0.0008) [2023-12-27 09:10:07,826][20347] Updated weights for policy 0, policy_version 509819 (0.0008) [2023-12-27 09:10:08,306][20381] Updated weights for policy 1, policy_version 509589 (0.0008) [2023-12-27 09:10:08,369][20381] Updated weights for policy 1, policy_version 509599 (0.0010) [2023-12-27 09:10:08,432][20381] Updated weights for policy 1, policy_version 509609 (0.0011) [2023-12-27 09:10:08,484][20347] Updated weights for policy 0, policy_version 509829 (0.0007) [2023-12-27 09:10:08,542][20347] Updated weights for policy 0, policy_version 509839 (0.0007) [2023-12-27 09:10:08,600][20347] Updated weights for policy 0, policy_version 509849 (0.0007) [2023-12-27 09:10:09,141][20381] Updated weights for policy 1, policy_version 509619 (0.0009) [2023-12-27 09:10:09,198][20381] Updated weights for policy 1, policy_version 509629 (0.0007) [2023-12-27 09:10:09,262][20381] Updated weights for policy 1, policy_version 509639 (0.0009) [2023-12-27 09:10:09,293][20347] Updated weights for policy 0, policy_version 509859 (0.0008) [2023-12-27 09:10:09,353][20347] Updated weights for policy 0, policy_version 509869 (0.0008) [2023-12-27 09:10:09,419][20347] Updated weights for policy 0, policy_version 509879 (0.0007) [2023-12-27 09:10:09,932][20381] Updated weights for policy 1, policy_version 509649 (0.0007) [2023-12-27 09:10:10,005][20381] Updated weights for policy 1, policy_version 509659 (0.0007) [2023-12-27 09:10:10,063][20381] Updated weights for policy 1, policy_version 509669 (0.0010) [2023-12-27 09:10:10,120][20381] Updated weights for policy 1, policy_version 509679 (0.0006) [2023-12-27 09:10:10,160][20347] Updated weights for policy 0, policy_version 509889 (0.0007) [2023-12-27 09:10:10,220][20347] Updated weights for policy 0, policy_version 509899 (0.0008) [2023-12-27 09:10:10,279][20347] Updated weights for policy 0, policy_version 509909 (0.0008) [2023-12-27 09:10:10,343][20347] Updated weights for policy 0, policy_version 509919 (0.0007) [2023-12-27 09:10:10,728][20381] Updated weights for policy 1, policy_version 509689 (0.0010) [2023-12-27 09:10:10,788][20381] Updated weights for policy 1, policy_version 509699 (0.0008) [2023-12-27 09:10:10,841][20381] Updated weights for policy 1, policy_version 509709 (0.0010) [2023-12-27 09:10:11,075][20347] Updated weights for policy 0, policy_version 509929 (0.0008) [2023-12-27 09:10:11,142][20347] Updated weights for policy 0, policy_version 509939 (0.0008) [2023-12-27 09:10:11,202][20347] Updated weights for policy 0, policy_version 509949 (0.0007) [2023-12-27 09:10:11,548][20381] Updated weights for policy 1, policy_version 509719 (0.0010) [2023-12-27 09:10:11,601][20381] Updated weights for policy 1, policy_version 509729 (0.0010) [2023-12-27 09:10:11,663][20381] Updated weights for policy 1, policy_version 509739 (0.0010) [2023-12-27 09:10:11,941][20347] Updated weights for policy 0, policy_version 509959 (0.0007) [2023-12-27 09:10:11,989][20347] Updated weights for policy 0, policy_version 509969 (0.0007) [2023-12-27 09:10:12,041][20347] Updated weights for policy 0, policy_version 509979 (0.0007) [2023-12-27 09:10:12,343][20381] Updated weights for policy 1, policy_version 509749 (0.0008) [2023-12-27 09:10:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 261087232. Throughput: 0: 10287.5, 1: 10445.6. Samples: 261079856. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:12,363][19170] Avg episode reward: [(0, '768.370'), (1, '931.920')] [2023-12-27 09:10:12,402][20381] Updated weights for policy 1, policy_version 509759 (0.0010) [2023-12-27 09:10:12,454][20381] Updated weights for policy 1, policy_version 509769 (0.0010) [2023-12-27 09:10:12,700][20347] Updated weights for policy 0, policy_version 509989 (0.0008) [2023-12-27 09:10:12,760][20347] Updated weights for policy 0, policy_version 509999 (0.0008) [2023-12-27 09:10:12,817][20347] Updated weights for policy 0, policy_version 510009 (0.0006) [2023-12-27 09:10:13,021][20381] Updated weights for policy 1, policy_version 509779 (0.0009) [2023-12-27 09:10:13,076][20381] Updated weights for policy 1, policy_version 509789 (0.0006) [2023-12-27 09:10:13,131][20381] Updated weights for policy 1, policy_version 509799 (0.0005) [2023-12-27 09:10:13,413][20347] Updated weights for policy 0, policy_version 510019 (0.0007) [2023-12-27 09:10:13,472][20347] Updated weights for policy 0, policy_version 510029 (0.0006) [2023-12-27 09:10:13,527][20347] Updated weights for policy 0, policy_version 510039 (0.0006) [2023-12-27 09:10:13,771][20381] Updated weights for policy 1, policy_version 509809 (0.0008) [2023-12-27 09:10:13,829][20381] Updated weights for policy 1, policy_version 509819 (0.0008) [2023-12-27 09:10:13,878][20381] Updated weights for policy 1, policy_version 509829 (0.0006) [2023-12-27 09:10:13,938][20381] Updated weights for policy 1, policy_version 509839 (0.0006) [2023-12-27 09:10:14,153][20347] Updated weights for policy 0, policy_version 510049 (0.0006) [2023-12-27 09:10:14,217][20347] Updated weights for policy 0, policy_version 510059 (0.0007) [2023-12-27 09:10:14,276][20347] Updated weights for policy 0, policy_version 510069 (0.0007) [2023-12-27 09:10:14,335][20347] Updated weights for policy 0, policy_version 510079 (0.0008) [2023-12-27 09:10:14,566][20381] Updated weights for policy 1, policy_version 509849 (0.0005) [2023-12-27 09:10:14,615][20381] Updated weights for policy 1, policy_version 509859 (0.0005) [2023-12-27 09:10:14,664][20381] Updated weights for policy 1, policy_version 509869 (0.0008) [2023-12-27 09:10:14,965][20347] Updated weights for policy 0, policy_version 510089 (0.0010) [2023-12-27 09:10:15,024][20347] Updated weights for policy 0, policy_version 510099 (0.0010) [2023-12-27 09:10:15,085][20347] Updated weights for policy 0, policy_version 510109 (0.0010) [2023-12-27 09:10:15,308][20381] Updated weights for policy 1, policy_version 509879 (0.0007) [2023-12-27 09:10:15,374][20381] Updated weights for policy 1, policy_version 509889 (0.0008) [2023-12-27 09:10:15,436][20381] Updated weights for policy 1, policy_version 509899 (0.0008) [2023-12-27 09:10:15,706][20347] Updated weights for policy 0, policy_version 510119 (0.0006) [2023-12-27 09:10:15,765][20347] Updated weights for policy 0, policy_version 510129 (0.0006) [2023-12-27 09:10:15,829][20347] Updated weights for policy 0, policy_version 510139 (0.0006) [2023-12-27 09:10:16,086][20381] Updated weights for policy 1, policy_version 509909 (0.0008) [2023-12-27 09:10:16,165][20381] Updated weights for policy 1, policy_version 509919 (0.0008) [2023-12-27 09:10:16,229][20381] Updated weights for policy 1, policy_version 509929 (0.0008) [2023-12-27 09:10:16,423][20347] Updated weights for policy 0, policy_version 510149 (0.0008) [2023-12-27 09:10:16,480][20347] Updated weights for policy 0, policy_version 510159 (0.0010) [2023-12-27 09:10:16,538][20347] Updated weights for policy 0, policy_version 510169 (0.0010) [2023-12-27 09:10:16,881][20381] Updated weights for policy 1, policy_version 509939 (0.0009) [2023-12-27 09:10:16,940][20381] Updated weights for policy 1, policy_version 509949 (0.0011) [2023-12-27 09:10:16,998][20381] Updated weights for policy 1, policy_version 509959 (0.0011) [2023-12-27 09:10:17,226][20347] Updated weights for policy 0, policy_version 510179 (0.0010) [2023-12-27 09:10:17,290][20347] Updated weights for policy 0, policy_version 510189 (0.0010) [2023-12-27 09:10:17,346][20347] Updated weights for policy 0, policy_version 510199 (0.0010) [2023-12-27 09:10:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 261193728. Throughput: 0: 10319.7, 1: 10482.7. Samples: 261207228. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:17,363][19170] Avg episode reward: [(0, '616.390'), (1, '922.520')] [2023-12-27 09:10:17,642][20381] Updated weights for policy 1, policy_version 509969 (0.0011) [2023-12-27 09:10:17,690][20381] Updated weights for policy 1, policy_version 509979 (0.0010) [2023-12-27 09:10:17,738][20381] Updated weights for policy 1, policy_version 509989 (0.0010) [2023-12-27 09:10:17,786][20381] Updated weights for policy 1, policy_version 509999 (0.0010) [2023-12-27 09:10:17,996][20347] Updated weights for policy 0, policy_version 510209 (0.0010) [2023-12-27 09:10:18,055][20347] Updated weights for policy 0, policy_version 510219 (0.0007) [2023-12-27 09:10:18,114][20347] Updated weights for policy 0, policy_version 510229 (0.0010) [2023-12-27 09:10:18,174][20347] Updated weights for policy 0, policy_version 510239 (0.0010) [2023-12-27 09:10:18,457][20381] Updated weights for policy 1, policy_version 510009 (0.0011) [2023-12-27 09:10:18,512][20381] Updated weights for policy 1, policy_version 510019 (0.0011) [2023-12-27 09:10:18,576][20381] Updated weights for policy 1, policy_version 510029 (0.0007) [2023-12-27 09:10:18,832][20291] KL-divergence is very high: 135.4500 [2023-12-27 09:10:18,870][20347] Updated weights for policy 0, policy_version 510249 (0.0008) [2023-12-27 09:10:18,907][20291] KL-divergence is very high: 126.5933 [2023-12-27 09:10:18,931][20347] Updated weights for policy 0, policy_version 510259 (0.0011) [2023-12-27 09:10:18,977][20291] KL-divergence is very high: 102.3340 [2023-12-27 09:10:18,990][20347] Updated weights for policy 0, policy_version 510269 (0.0010) [2023-12-27 09:10:19,240][20381] Updated weights for policy 1, policy_version 510039 (0.0007) [2023-12-27 09:10:19,310][20381] Updated weights for policy 1, policy_version 510049 (0.0010) [2023-12-27 09:10:19,373][20381] Updated weights for policy 1, policy_version 510059 (0.0009) [2023-12-27 09:10:19,685][20347] Updated weights for policy 0, policy_version 510279 (0.0007) [2023-12-27 09:10:19,746][20347] Updated weights for policy 0, policy_version 510289 (0.0007) [2023-12-27 09:10:19,758][20291] KL-divergence is very high: 109.7665 [2023-12-27 09:10:19,801][20347] Updated weights for policy 0, policy_version 510299 (0.0006) [2023-12-27 09:10:20,154][20381] Updated weights for policy 1, policy_version 510069 (0.0009) [2023-12-27 09:10:20,214][20381] Updated weights for policy 1, policy_version 510079 (0.0008) [2023-12-27 09:10:20,270][20381] Updated weights for policy 1, policy_version 510089 (0.0008) [2023-12-27 09:10:20,475][20347] Updated weights for policy 0, policy_version 510309 (0.0008) [2023-12-27 09:10:20,520][20347] Updated weights for policy 0, policy_version 510319 (0.0008) [2023-12-27 09:10:20,568][20347] Updated weights for policy 0, policy_version 510329 (0.0007) [2023-12-27 09:10:20,960][20381] Updated weights for policy 1, policy_version 510099 (0.0010) [2023-12-27 09:10:21,015][20381] Updated weights for policy 1, policy_version 510109 (0.0011) [2023-12-27 09:10:21,078][20381] Updated weights for policy 1, policy_version 510119 (0.0008) [2023-12-27 09:10:21,249][20347] Updated weights for policy 0, policy_version 510339 (0.0008) [2023-12-27 09:10:21,312][20347] Updated weights for policy 0, policy_version 510349 (0.0010) [2023-12-27 09:10:21,374][20347] Updated weights for policy 0, policy_version 510359 (0.0008) [2023-12-27 09:10:21,811][20381] Updated weights for policy 1, policy_version 510129 (0.0009) [2023-12-27 09:10:21,871][20381] Updated weights for policy 1, policy_version 510139 (0.0006) [2023-12-27 09:10:21,927][20381] Updated weights for policy 1, policy_version 510149 (0.0008) [2023-12-27 09:10:21,987][20381] Updated weights for policy 1, policy_version 510159 (0.0008) [2023-12-27 09:10:22,147][20347] Updated weights for policy 0, policy_version 510369 (0.0010) [2023-12-27 09:10:22,208][20347] Updated weights for policy 0, policy_version 510379 (0.0009) [2023-12-27 09:10:22,272][20347] Updated weights for policy 0, policy_version 510389 (0.0009) [2023-12-27 09:10:22,321][20347] Updated weights for policy 0, policy_version 510399 (0.0009) [2023-12-27 09:10:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 261300224. Throughput: 0: 10316.9, 1: 10435.1. Samples: 261267684. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:22,363][19170] Avg episode reward: [(0, '337.120'), (1, '961.290')] [2023-12-27 09:10:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000510400_130678784.pth... [2023-12-27 09:10:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000510160_130621440.pth... [2023-12-27 09:10:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000509184_130367488.pth [2023-12-27 09:10:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000508944_130310144.pth [2023-12-27 09:10:22,819][20381] Updated weights for policy 1, policy_version 510169 (0.0007) [2023-12-27 09:10:22,875][20381] Updated weights for policy 1, policy_version 510179 (0.0007) [2023-12-27 09:10:22,924][20381] Updated weights for policy 1, policy_version 510189 (0.0006) [2023-12-27 09:10:23,166][20347] Updated weights for policy 0, policy_version 510409 (0.0008) [2023-12-27 09:10:23,223][20347] Updated weights for policy 0, policy_version 510419 (0.0007) [2023-12-27 09:10:23,284][20347] Updated weights for policy 0, policy_version 510429 (0.0007) [2023-12-27 09:10:23,607][20381] Updated weights for policy 1, policy_version 510199 (0.0006) [2023-12-27 09:10:23,665][20381] Updated weights for policy 1, policy_version 510209 (0.0007) [2023-12-27 09:10:23,719][20381] Updated weights for policy 1, policy_version 510219 (0.0007) [2023-12-27 09:10:23,949][20347] Updated weights for policy 0, policy_version 510439 (0.0009) [2023-12-27 09:10:24,004][20347] Updated weights for policy 0, policy_version 510449 (0.0010) [2023-12-27 09:10:24,061][20347] Updated weights for policy 0, policy_version 510459 (0.0010) [2023-12-27 09:10:24,317][20381] Updated weights for policy 1, policy_version 510229 (0.0006) [2023-12-27 09:10:24,366][20381] Updated weights for policy 1, policy_version 510239 (0.0005) [2023-12-27 09:10:24,410][20381] Updated weights for policy 1, policy_version 510249 (0.0005) [2023-12-27 09:10:24,750][20347] Updated weights for policy 0, policy_version 510469 (0.0009) [2023-12-27 09:10:24,802][20347] Updated weights for policy 0, policy_version 510479 (0.0010) [2023-12-27 09:10:24,863][20347] Updated weights for policy 0, policy_version 510489 (0.0010) [2023-12-27 09:10:24,988][20381] Updated weights for policy 1, policy_version 510259 (0.0009) [2023-12-27 09:10:25,044][20381] Updated weights for policy 1, policy_version 510269 (0.0007) [2023-12-27 09:10:25,098][20381] Updated weights for policy 1, policy_version 510279 (0.0005) [2023-12-27 09:10:25,548][20347] Updated weights for policy 0, policy_version 510499 (0.0010) [2023-12-27 09:10:25,599][20347] Updated weights for policy 0, policy_version 510509 (0.0010) [2023-12-27 09:10:25,653][20347] Updated weights for policy 0, policy_version 510519 (0.0010) [2023-12-27 09:10:25,674][20381] Updated weights for policy 1, policy_version 510289 (0.0006) [2023-12-27 09:10:25,732][20381] Updated weights for policy 1, policy_version 510299 (0.0010) [2023-12-27 09:10:25,784][20381] Updated weights for policy 1, policy_version 510309 (0.0008) [2023-12-27 09:10:25,833][20381] Updated weights for policy 1, policy_version 510319 (0.0005) [2023-12-27 09:10:26,323][20347] Updated weights for policy 0, policy_version 510529 (0.0010) [2023-12-27 09:10:26,380][20347] Updated weights for policy 0, policy_version 510539 (0.0010) [2023-12-27 09:10:26,438][20347] Updated weights for policy 0, policy_version 510549 (0.0006) [2023-12-27 09:10:26,463][20381] Updated weights for policy 1, policy_version 510329 (0.0010) [2023-12-27 09:10:26,490][20347] Updated weights for policy 0, policy_version 510559 (0.0006) [2023-12-27 09:10:26,514][20381] Updated weights for policy 1, policy_version 510339 (0.0010) [2023-12-27 09:10:26,571][20381] Updated weights for policy 1, policy_version 510349 (0.0010) [2023-12-27 09:10:27,053][20347] Updated weights for policy 0, policy_version 510569 (0.0005) [2023-12-27 09:10:27,104][20347] Updated weights for policy 0, policy_version 510579 (0.0005) [2023-12-27 09:10:27,156][20347] Updated weights for policy 0, policy_version 510589 (0.0006) [2023-12-27 09:10:27,178][20381] Updated weights for policy 1, policy_version 510359 (0.0010) [2023-12-27 09:10:27,223][20381] Updated weights for policy 1, policy_version 510369 (0.0010) [2023-12-27 09:10:27,264][20381] Updated weights for policy 1, policy_version 510379 (0.0010) [2023-12-27 09:10:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 261406720. Throughput: 0: 10263.9, 1: 10470.2. Samples: 261389028. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:27,363][19170] Avg episode reward: [(0, '345.280'), (1, '970.840')] [2023-12-27 09:10:27,796][20347] Updated weights for policy 0, policy_version 510599 (0.0007) [2023-12-27 09:10:27,842][20347] Updated weights for policy 0, policy_version 510609 (0.0006) [2023-12-27 09:10:27,900][20347] Updated weights for policy 0, policy_version 510619 (0.0005) [2023-12-27 09:10:27,920][20381] Updated weights for policy 1, policy_version 510389 (0.0010) [2023-12-27 09:10:27,968][20381] Updated weights for policy 1, policy_version 510399 (0.0010) [2023-12-27 09:10:28,019][20381] Updated weights for policy 1, policy_version 510409 (0.0010) [2023-12-27 09:10:28,594][20347] Updated weights for policy 0, policy_version 510629 (0.0007) [2023-12-27 09:10:28,656][20347] Updated weights for policy 0, policy_version 510639 (0.0008) [2023-12-27 09:10:28,665][20381] Updated weights for policy 1, policy_version 510419 (0.0009) [2023-12-27 09:10:28,714][20347] Updated weights for policy 0, policy_version 510649 (0.0007) [2023-12-27 09:10:28,717][20381] Updated weights for policy 1, policy_version 510429 (0.0007) [2023-12-27 09:10:28,780][20381] Updated weights for policy 1, policy_version 510439 (0.0007) [2023-12-27 09:10:29,356][20381] Updated weights for policy 1, policy_version 510449 (0.0007) [2023-12-27 09:10:29,418][20381] Updated weights for policy 1, policy_version 510459 (0.0010) [2023-12-27 09:10:29,427][20347] Updated weights for policy 0, policy_version 510659 (0.0007) [2023-12-27 09:10:29,476][20381] Updated weights for policy 1, policy_version 510469 (0.0010) [2023-12-27 09:10:29,478][20347] Updated weights for policy 0, policy_version 510669 (0.0005) [2023-12-27 09:10:29,533][20347] Updated weights for policy 0, policy_version 510679 (0.0007) [2023-12-27 09:10:29,535][20381] Updated weights for policy 1, policy_version 510479 (0.0006) [2023-12-27 09:10:30,133][20381] Updated weights for policy 1, policy_version 510489 (0.0011) [2023-12-27 09:10:30,160][20347] Updated weights for policy 0, policy_version 510689 (0.0006) [2023-12-27 09:10:30,192][20381] Updated weights for policy 1, policy_version 510499 (0.0011) [2023-12-27 09:10:30,217][20347] Updated weights for policy 0, policy_version 510699 (0.0006) [2023-12-27 09:10:30,248][20381] Updated weights for policy 1, policy_version 510509 (0.0007) [2023-12-27 09:10:30,275][20347] Updated weights for policy 0, policy_version 510709 (0.0007) [2023-12-27 09:10:30,330][20347] Updated weights for policy 0, policy_version 510719 (0.0008) [2023-12-27 09:10:30,865][20381] Updated weights for policy 1, policy_version 510519 (0.0005) [2023-12-27 09:10:30,922][20381] Updated weights for policy 1, policy_version 510529 (0.0009) [2023-12-27 09:10:30,976][20381] Updated weights for policy 1, policy_version 510539 (0.0010) [2023-12-27 09:10:30,977][20347] Updated weights for policy 0, policy_version 510729 (0.0006) [2023-12-27 09:10:31,035][20347] Updated weights for policy 0, policy_version 510739 (0.0007) [2023-12-27 09:10:31,087][20347] Updated weights for policy 0, policy_version 510749 (0.0007) [2023-12-27 09:10:31,617][20381] Updated weights for policy 1, policy_version 510549 (0.0008) [2023-12-27 09:10:31,674][20381] Updated weights for policy 1, policy_version 510559 (0.0007) [2023-12-27 09:10:31,740][20381] Updated weights for policy 1, policy_version 510569 (0.0007) [2023-12-27 09:10:31,808][20347] Updated weights for policy 0, policy_version 510759 (0.0006) [2023-12-27 09:10:31,862][20347] Updated weights for policy 0, policy_version 510769 (0.0008) [2023-12-27 09:10:31,912][20347] Updated weights for policy 0, policy_version 510779 (0.0007) [2023-12-27 09:10:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 261505024. Throughput: 0: 10233.9, 1: 10535.0. Samples: 261517536. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:32,363][19170] Avg episode reward: [(0, '176.230'), (1, '937.420')] [2023-12-27 09:10:32,432][20381] Updated weights for policy 1, policy_version 510579 (0.0007) [2023-12-27 09:10:32,489][20381] Updated weights for policy 1, policy_version 510589 (0.0006) [2023-12-27 09:10:32,558][20381] Updated weights for policy 1, policy_version 510599 (0.0006) [2023-12-27 09:10:32,636][20347] Updated weights for policy 0, policy_version 510789 (0.0007) [2023-12-27 09:10:32,697][20347] Updated weights for policy 0, policy_version 510799 (0.0006) [2023-12-27 09:10:32,753][20347] Updated weights for policy 0, policy_version 510809 (0.0007) [2023-12-27 09:10:33,204][20381] Updated weights for policy 1, policy_version 510609 (0.0007) [2023-12-27 09:10:33,260][20381] Updated weights for policy 1, policy_version 510619 (0.0007) [2023-12-27 09:10:33,322][20381] Updated weights for policy 1, policy_version 510629 (0.0005) [2023-12-27 09:10:33,378][20381] Updated weights for policy 1, policy_version 510639 (0.0005) [2023-12-27 09:10:33,443][20347] Updated weights for policy 0, policy_version 510819 (0.0007) [2023-12-27 09:10:33,488][20347] Updated weights for policy 0, policy_version 510829 (0.0007) [2023-12-27 09:10:33,537][20347] Updated weights for policy 0, policy_version 510839 (0.0008) [2023-12-27 09:10:33,941][20381] Updated weights for policy 1, policy_version 510649 (0.0007) [2023-12-27 09:10:33,998][20381] Updated weights for policy 1, policy_version 510659 (0.0007) [2023-12-27 09:10:34,049][20381] Updated weights for policy 1, policy_version 510669 (0.0007) [2023-12-27 09:10:34,265][20347] Updated weights for policy 0, policy_version 510849 (0.0008) [2023-12-27 09:10:34,332][20347] Updated weights for policy 0, policy_version 510859 (0.0008) [2023-12-27 09:10:34,394][20347] Updated weights for policy 0, policy_version 510869 (0.0008) [2023-12-27 09:10:34,450][20347] Updated weights for policy 0, policy_version 510879 (0.0008) [2023-12-27 09:10:34,690][20381] Updated weights for policy 1, policy_version 510679 (0.0008) [2023-12-27 09:10:34,749][20381] Updated weights for policy 1, policy_version 510689 (0.0007) [2023-12-27 09:10:34,810][20381] Updated weights for policy 1, policy_version 510699 (0.0008) [2023-12-27 09:10:35,179][20347] Updated weights for policy 0, policy_version 510889 (0.0008) [2023-12-27 09:10:35,225][20347] Updated weights for policy 0, policy_version 510899 (0.0008) [2023-12-27 09:10:35,283][20347] Updated weights for policy 0, policy_version 510909 (0.0007) [2023-12-27 09:10:35,446][20381] Updated weights for policy 1, policy_version 510709 (0.0008) [2023-12-27 09:10:35,500][20381] Updated weights for policy 1, policy_version 510719 (0.0007) [2023-12-27 09:10:35,558][20381] Updated weights for policy 1, policy_version 510729 (0.0008) [2023-12-27 09:10:35,963][20347] Updated weights for policy 0, policy_version 510919 (0.0008) [2023-12-27 09:10:36,023][20347] Updated weights for policy 0, policy_version 510929 (0.0007) [2023-12-27 09:10:36,073][20347] Updated weights for policy 0, policy_version 510939 (0.0007) [2023-12-27 09:10:36,167][20381] Updated weights for policy 1, policy_version 510739 (0.0006) [2023-12-27 09:10:36,221][20381] Updated weights for policy 1, policy_version 510749 (0.0008) [2023-12-27 09:10:36,276][20381] Updated weights for policy 1, policy_version 510759 (0.0008) [2023-12-27 09:10:36,809][20347] Updated weights for policy 0, policy_version 510949 (0.0009) [2023-12-27 09:10:36,856][20347] Updated weights for policy 0, policy_version 510959 (0.0010) [2023-12-27 09:10:36,911][20347] Updated weights for policy 0, policy_version 510969 (0.0009) [2023-12-27 09:10:36,915][20381] Updated weights for policy 1, policy_version 510769 (0.0008) [2023-12-27 09:10:36,967][20381] Updated weights for policy 1, policy_version 510779 (0.0006) [2023-12-27 09:10:37,015][20381] Updated weights for policy 1, policy_version 510789 (0.0008) [2023-12-27 09:10:37,071][20381] Updated weights for policy 1, policy_version 510799 (0.0005) [2023-12-27 09:10:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 261611520. Throughput: 0: 10186.6, 1: 10563.4. Samples: 261579344. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:37,364][19170] Avg episode reward: [(0, '161.280'), (1, '925.030')] [2023-12-27 09:10:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000510976_130826240.pth... [2023-12-27 09:10:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000510800_130785280.pth... [2023-12-27 09:10:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000509792_130523136.pth [2023-12-27 09:10:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000509552_130465792.pth [2023-12-27 09:10:37,565][20347] Updated weights for policy 0, policy_version 510979 (0.0007) [2023-12-27 09:10:37,613][20347] Updated weights for policy 0, policy_version 510989 (0.0008) [2023-12-27 09:10:37,615][20381] Updated weights for policy 1, policy_version 510809 (0.0005) [2023-12-27 09:10:37,664][20347] Updated weights for policy 0, policy_version 510999 (0.0008) [2023-12-27 09:10:37,671][20381] Updated weights for policy 1, policy_version 510819 (0.0005) [2023-12-27 09:10:37,733][20381] Updated weights for policy 1, policy_version 510829 (0.0005) [2023-12-27 09:10:38,340][20381] Updated weights for policy 1, policy_version 510839 (0.0007) [2023-12-27 09:10:38,406][20381] Updated weights for policy 1, policy_version 510849 (0.0007) [2023-12-27 09:10:38,423][20347] Updated weights for policy 0, policy_version 511009 (0.0007) [2023-12-27 09:10:38,476][20381] Updated weights for policy 1, policy_version 510859 (0.0006) [2023-12-27 09:10:38,485][20347] Updated weights for policy 0, policy_version 511019 (0.0007) [2023-12-27 09:10:38,545][20347] Updated weights for policy 0, policy_version 511029 (0.0006) [2023-12-27 09:10:38,604][20347] Updated weights for policy 0, policy_version 511039 (0.0005) [2023-12-27 09:10:39,156][20381] Updated weights for policy 1, policy_version 510869 (0.0007) [2023-12-27 09:10:39,203][20381] Updated weights for policy 1, policy_version 510879 (0.0008) [2023-12-27 09:10:39,261][20381] Updated weights for policy 1, policy_version 510889 (0.0008) [2023-12-27 09:10:39,271][20347] Updated weights for policy 0, policy_version 511049 (0.0006) [2023-12-27 09:10:39,339][20347] Updated weights for policy 0, policy_version 511059 (0.0007) [2023-12-27 09:10:39,404][20347] Updated weights for policy 0, policy_version 511069 (0.0008) [2023-12-27 09:10:39,936][20381] Updated weights for policy 1, policy_version 510899 (0.0008) [2023-12-27 09:10:39,995][20381] Updated weights for policy 1, policy_version 510909 (0.0006) [2023-12-27 09:10:40,052][20381] Updated weights for policy 1, policy_version 510919 (0.0005) [2023-12-27 09:10:40,091][20347] Updated weights for policy 0, policy_version 511079 (0.0006) [2023-12-27 09:10:40,149][20347] Updated weights for policy 0, policy_version 511089 (0.0007) [2023-12-27 09:10:40,202][20347] Updated weights for policy 0, policy_version 511099 (0.0008) [2023-12-27 09:10:40,685][20381] Updated weights for policy 1, policy_version 510929 (0.0007) [2023-12-27 09:10:40,744][20381] Updated weights for policy 1, policy_version 510939 (0.0007) [2023-12-27 09:10:40,810][20381] Updated weights for policy 1, policy_version 510949 (0.0008) [2023-12-27 09:10:40,874][20381] Updated weights for policy 1, policy_version 510959 (0.0008) [2023-12-27 09:10:40,898][20347] Updated weights for policy 0, policy_version 511109 (0.0007) [2023-12-27 09:10:40,960][20347] Updated weights for policy 0, policy_version 511119 (0.0005) [2023-12-27 09:10:41,018][20347] Updated weights for policy 0, policy_version 511129 (0.0006) [2023-12-27 09:10:41,606][20381] Updated weights for policy 1, policy_version 510969 (0.0007) [2023-12-27 09:10:41,666][20381] Updated weights for policy 1, policy_version 510979 (0.0008) [2023-12-27 09:10:41,707][20347] Updated weights for policy 0, policy_version 511139 (0.0008) [2023-12-27 09:10:41,722][20381] Updated weights for policy 1, policy_version 510989 (0.0007) [2023-12-27 09:10:41,772][20347] Updated weights for policy 0, policy_version 511149 (0.0007) [2023-12-27 09:10:41,839][20347] Updated weights for policy 0, policy_version 511159 (0.0005) [2023-12-27 09:10:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 261709824. Throughput: 0: 10172.1, 1: 10624.5. Samples: 261703528. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:42,363][19170] Avg episode reward: [(0, '153.070'), (1, '927.350')] [2023-12-27 09:10:42,422][20381] Updated weights for policy 1, policy_version 510999 (0.0009) [2023-12-27 09:10:42,479][20381] Updated weights for policy 1, policy_version 511009 (0.0008) [2023-12-27 09:10:42,495][20347] Updated weights for policy 0, policy_version 511169 (0.0006) [2023-12-27 09:10:42,543][20381] Updated weights for policy 1, policy_version 511019 (0.0006) [2023-12-27 09:10:42,546][20347] Updated weights for policy 0, policy_version 511179 (0.0006) [2023-12-27 09:10:42,597][20347] Updated weights for policy 0, policy_version 511189 (0.0008) [2023-12-27 09:10:42,656][20347] Updated weights for policy 0, policy_version 511199 (0.0008) [2023-12-27 09:10:43,148][20381] Updated weights for policy 1, policy_version 511029 (0.0006) [2023-12-27 09:10:43,195][20381] Updated weights for policy 1, policy_version 511039 (0.0005) [2023-12-27 09:10:43,239][20381] Updated weights for policy 1, policy_version 511049 (0.0007) [2023-12-27 09:10:43,406][20347] Updated weights for policy 0, policy_version 511209 (0.0007) [2023-12-27 09:10:43,464][20347] Updated weights for policy 0, policy_version 511219 (0.0008) [2023-12-27 09:10:43,526][20347] Updated weights for policy 0, policy_version 511229 (0.0009) [2023-12-27 09:10:43,829][20381] Updated weights for policy 1, policy_version 511059 (0.0009) [2023-12-27 09:10:43,889][20381] Updated weights for policy 1, policy_version 511069 (0.0005) [2023-12-27 09:10:43,942][20381] Updated weights for policy 1, policy_version 511079 (0.0005) [2023-12-27 09:10:44,283][20347] Updated weights for policy 0, policy_version 511239 (0.0007) [2023-12-27 09:10:44,330][20347] Updated weights for policy 0, policy_version 511249 (0.0008) [2023-12-27 09:10:44,381][20347] Updated weights for policy 0, policy_version 511259 (0.0007) [2023-12-27 09:10:44,478][20381] Updated weights for policy 1, policy_version 511089 (0.0006) [2023-12-27 09:10:44,537][20381] Updated weights for policy 1, policy_version 511099 (0.0007) [2023-12-27 09:10:44,588][20381] Updated weights for policy 1, policy_version 511109 (0.0007) [2023-12-27 09:10:44,650][20381] Updated weights for policy 1, policy_version 511119 (0.0008) [2023-12-27 09:10:45,075][20347] Updated weights for policy 0, policy_version 511269 (0.0007) [2023-12-27 09:10:45,127][20347] Updated weights for policy 0, policy_version 511279 (0.0007) [2023-12-27 09:10:45,188][20347] Updated weights for policy 0, policy_version 511289 (0.0005) [2023-12-27 09:10:45,339][20381] Updated weights for policy 1, policy_version 511129 (0.0007) [2023-12-27 09:10:45,400][20381] Updated weights for policy 1, policy_version 511139 (0.0009) [2023-12-27 09:10:45,454][20381] Updated weights for policy 1, policy_version 511149 (0.0008) [2023-12-27 09:10:45,758][20347] Updated weights for policy 0, policy_version 511299 (0.0006) [2023-12-27 09:10:45,808][20347] Updated weights for policy 0, policy_version 511309 (0.0008) [2023-12-27 09:10:45,867][20347] Updated weights for policy 0, policy_version 511319 (0.0008) [2023-12-27 09:10:46,119][20381] Updated weights for policy 1, policy_version 511159 (0.0008) [2023-12-27 09:10:46,171][20381] Updated weights for policy 1, policy_version 511169 (0.0007) [2023-12-27 09:10:46,232][20381] Updated weights for policy 1, policy_version 511179 (0.0007) [2023-12-27 09:10:46,599][20347] Updated weights for policy 0, policy_version 511329 (0.0007) [2023-12-27 09:10:46,661][20347] Updated weights for policy 0, policy_version 511339 (0.0008) [2023-12-27 09:10:46,712][20347] Updated weights for policy 0, policy_version 511349 (0.0008) [2023-12-27 09:10:46,757][20347] Updated weights for policy 0, policy_version 511359 (0.0007) [2023-12-27 09:10:46,845][20381] Updated weights for policy 1, policy_version 511189 (0.0007) [2023-12-27 09:10:46,889][20381] Updated weights for policy 1, policy_version 511199 (0.0007) [2023-12-27 09:10:46,936][20381] Updated weights for policy 1, policy_version 511209 (0.0005) [2023-12-27 09:10:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 261816320. Throughput: 0: 10159.8, 1: 10621.7. Samples: 261828768. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:47,363][19170] Avg episode reward: [(0, '194.420'), (1, '906.190')] [2023-12-27 09:10:47,508][20347] Updated weights for policy 0, policy_version 511369 (0.0008) [2023-12-27 09:10:47,563][20347] Updated weights for policy 0, policy_version 511379 (0.0007) [2023-12-27 09:10:47,583][20381] Updated weights for policy 1, policy_version 511219 (0.0005) [2023-12-27 09:10:47,619][20347] Updated weights for policy 0, policy_version 511389 (0.0006) [2023-12-27 09:10:47,638][20381] Updated weights for policy 1, policy_version 511229 (0.0008) [2023-12-27 09:10:47,685][20381] Updated weights for policy 1, policy_version 511239 (0.0008) [2023-12-27 09:10:48,280][20347] Updated weights for policy 0, policy_version 511399 (0.0008) [2023-12-27 09:10:48,348][20347] Updated weights for policy 0, policy_version 511409 (0.0007) [2023-12-27 09:10:48,354][20381] Updated weights for policy 1, policy_version 511249 (0.0007) [2023-12-27 09:10:48,412][20347] Updated weights for policy 0, policy_version 511419 (0.0007) [2023-12-27 09:10:48,413][20381] Updated weights for policy 1, policy_version 511259 (0.0007) [2023-12-27 09:10:48,469][20381] Updated weights for policy 1, policy_version 511269 (0.0006) [2023-12-27 09:10:48,531][20381] Updated weights for policy 1, policy_version 511279 (0.0005) [2023-12-27 09:10:49,062][20381] Updated weights for policy 1, policy_version 511289 (0.0005) [2023-12-27 09:10:49,105][20347] Updated weights for policy 0, policy_version 511429 (0.0007) [2023-12-27 09:10:49,123][20381] Updated weights for policy 1, policy_version 511299 (0.0005) [2023-12-27 09:10:49,165][20347] Updated weights for policy 0, policy_version 511439 (0.0006) [2023-12-27 09:10:49,175][20381] Updated weights for policy 1, policy_version 511309 (0.0005) [2023-12-27 09:10:49,225][20347] Updated weights for policy 0, policy_version 511449 (0.0007) [2023-12-27 09:10:49,821][20347] Updated weights for policy 0, policy_version 511459 (0.0006) [2023-12-27 09:10:49,827][20381] Updated weights for policy 1, policy_version 511319 (0.0007) [2023-12-27 09:10:49,885][20347] Updated weights for policy 0, policy_version 511469 (0.0006) [2023-12-27 09:10:49,890][20381] Updated weights for policy 1, policy_version 511329 (0.0006) [2023-12-27 09:10:49,945][20347] Updated weights for policy 0, policy_version 511479 (0.0006) [2023-12-27 09:10:49,952][20381] Updated weights for policy 1, policy_version 511339 (0.0006) [2023-12-27 09:10:50,611][20381] Updated weights for policy 1, policy_version 511349 (0.0007) [2023-12-27 09:10:50,656][20347] Updated weights for policy 0, policy_version 511489 (0.0008) [2023-12-27 09:10:50,670][20381] Updated weights for policy 1, policy_version 511359 (0.0007) [2023-12-27 09:10:50,718][20347] Updated weights for policy 0, policy_version 511499 (0.0008) [2023-12-27 09:10:50,721][20381] Updated weights for policy 1, policy_version 511369 (0.0006) [2023-12-27 09:10:50,778][20347] Updated weights for policy 0, policy_version 511509 (0.0009) [2023-12-27 09:10:50,827][20347] Updated weights for policy 0, policy_version 511519 (0.0007) [2023-12-27 09:10:51,414][20381] Updated weights for policy 1, policy_version 511379 (0.0007) [2023-12-27 09:10:51,464][20381] Updated weights for policy 1, policy_version 511389 (0.0009) [2023-12-27 09:10:51,518][20381] Updated weights for policy 1, policy_version 511399 (0.0010) [2023-12-27 09:10:51,615][20347] Updated weights for policy 0, policy_version 511529 (0.0010) [2023-12-27 09:10:51,685][20347] Updated weights for policy 0, policy_version 511539 (0.0010) [2023-12-27 09:10:51,743][20347] Updated weights for policy 0, policy_version 511549 (0.0010) [2023-12-27 09:10:52,178][20381] Updated weights for policy 1, policy_version 511409 (0.0010) [2023-12-27 09:10:52,237][20381] Updated weights for policy 1, policy_version 511419 (0.0010) [2023-12-27 09:10:52,298][20381] Updated weights for policy 1, policy_version 511429 (0.0008) [2023-12-27 09:10:52,353][20381] Updated weights for policy 1, policy_version 511439 (0.0010) [2023-12-27 09:10:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 261922816. Throughput: 0: 10131.8, 1: 10659.3. Samples: 261892112. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:52,363][19170] Avg episode reward: [(0, '173.380'), (1, '926.470')] [2023-12-27 09:10:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000511440_130949120.pth... [2023-12-27 09:10:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000511552_130973696.pth... [2023-12-27 09:10:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000510160_130621440.pth [2023-12-27 09:10:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000510400_130678784.pth [2023-12-27 09:10:52,432][20347] Updated weights for policy 0, policy_version 511559 (0.0010) [2023-12-27 09:10:52,494][20347] Updated weights for policy 0, policy_version 511569 (0.0010) [2023-12-27 09:10:52,559][20347] Updated weights for policy 0, policy_version 511579 (0.0010) [2023-12-27 09:10:53,047][20381] Updated weights for policy 1, policy_version 511449 (0.0009) [2023-12-27 09:10:53,090][20292] KL-divergence is very high: 292.9887 [2023-12-27 09:10:53,108][20381] Updated weights for policy 1, policy_version 511459 (0.0007) [2023-12-27 09:10:53,140][20292] KL-divergence is very high: 109.8756 [2023-12-27 09:10:53,171][20381] Updated weights for policy 1, policy_version 511469 (0.0009) [2023-12-27 09:10:53,219][20347] Updated weights for policy 0, policy_version 511589 (0.0008) [2023-12-27 09:10:53,276][20347] Updated weights for policy 0, policy_version 511599 (0.0005) [2023-12-27 09:10:53,338][20347] Updated weights for policy 0, policy_version 511609 (0.0006) [2023-12-27 09:10:53,814][20381] Updated weights for policy 1, policy_version 511479 (0.0008) [2023-12-27 09:10:53,863][20381] Updated weights for policy 1, policy_version 511489 (0.0009) [2023-12-27 09:10:53,911][20381] Updated weights for policy 1, policy_version 511499 (0.0007) [2023-12-27 09:10:53,949][20347] Updated weights for policy 0, policy_version 511619 (0.0007) [2023-12-27 09:10:54,008][20347] Updated weights for policy 0, policy_version 511629 (0.0008) [2023-12-27 09:10:54,068][20347] Updated weights for policy 0, policy_version 511639 (0.0008) [2023-12-27 09:10:54,534][20381] Updated weights for policy 1, policy_version 511509 (0.0008) [2023-12-27 09:10:54,582][20381] Updated weights for policy 1, policy_version 511519 (0.0007) [2023-12-27 09:10:54,630][20381] Updated weights for policy 1, policy_version 511529 (0.0005) [2023-12-27 09:10:54,787][20347] Updated weights for policy 0, policy_version 511649 (0.0007) [2023-12-27 09:10:54,850][20347] Updated weights for policy 0, policy_version 511659 (0.0006) [2023-12-27 09:10:54,910][20347] Updated weights for policy 0, policy_version 511669 (0.0005) [2023-12-27 09:10:54,976][20347] Updated weights for policy 0, policy_version 511679 (0.0007) [2023-12-27 09:10:55,273][20381] Updated weights for policy 1, policy_version 511539 (0.0005) [2023-12-27 09:10:55,334][20381] Updated weights for policy 1, policy_version 511549 (0.0005) [2023-12-27 09:10:55,391][20381] Updated weights for policy 1, policy_version 511559 (0.0007) [2023-12-27 09:10:55,421][20292] KL-divergence is very high: 488.4510 [2023-12-27 09:10:55,426][20292] KL-divergence is very high: 461.4441 [2023-12-27 09:10:55,431][20292] KL-divergence is very high: 373.1088 [2023-12-27 09:10:55,550][20347] Updated weights for policy 0, policy_version 511689 (0.0006) [2023-12-27 09:10:55,612][20347] Updated weights for policy 0, policy_version 511699 (0.0005) [2023-12-27 09:10:55,672][20347] Updated weights for policy 0, policy_version 511709 (0.0005) [2023-12-27 09:10:56,089][20381] Updated weights for policy 1, policy_version 511569 (0.0007) [2023-12-27 09:10:56,140][20381] Updated weights for policy 1, policy_version 511579 (0.0005) [2023-12-27 09:10:56,195][20347] Updated weights for policy 0, policy_version 511719 (0.0007) [2023-12-27 09:10:56,198][20381] Updated weights for policy 1, policy_version 511589 (0.0006) [2023-12-27 09:10:56,245][20347] Updated weights for policy 0, policy_version 511729 (0.0008) [2023-12-27 09:10:56,254][20381] Updated weights for policy 1, policy_version 511599 (0.0006) [2023-12-27 09:10:56,292][20347] Updated weights for policy 0, policy_version 511739 (0.0007) [2023-12-27 09:10:56,926][20381] Updated weights for policy 1, policy_version 511609 (0.0008) [2023-12-27 09:10:56,986][20381] Updated weights for policy 1, policy_version 511619 (0.0006) [2023-12-27 09:10:56,997][20347] Updated weights for policy 0, policy_version 511749 (0.0008) [2023-12-27 09:10:57,037][20381] Updated weights for policy 1, policy_version 511629 (0.0009) [2023-12-27 09:10:57,057][20347] Updated weights for policy 0, policy_version 511759 (0.0007) [2023-12-27 09:10:57,112][20347] Updated weights for policy 0, policy_version 511769 (0.0008) [2023-12-27 09:10:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 262029312. Throughput: 0: 10163.9, 1: 10652.7. Samples: 262016604. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:10:57,363][19170] Avg episode reward: [(0, '127.140'), (1, '902.520')] [2023-12-27 09:10:57,669][20381] Updated weights for policy 1, policy_version 511639 (0.0007) [2023-12-27 09:10:57,729][20381] Updated weights for policy 1, policy_version 511649 (0.0008) [2023-12-27 09:10:57,778][20347] Updated weights for policy 0, policy_version 511779 (0.0007) [2023-12-27 09:10:57,792][20381] Updated weights for policy 1, policy_version 511659 (0.0007) [2023-12-27 09:10:57,838][20347] Updated weights for policy 0, policy_version 511789 (0.0005) [2023-12-27 09:10:57,900][20347] Updated weights for policy 0, policy_version 511799 (0.0007) [2023-12-27 09:10:58,512][20381] Updated weights for policy 1, policy_version 511669 (0.0008) [2023-12-27 09:10:58,577][20381] Updated weights for policy 1, policy_version 511679 (0.0007) [2023-12-27 09:10:58,612][20347] Updated weights for policy 0, policy_version 511809 (0.0007) [2023-12-27 09:10:58,637][20381] Updated weights for policy 1, policy_version 511689 (0.0007) [2023-12-27 09:10:58,672][20347] Updated weights for policy 0, policy_version 511819 (0.0008) [2023-12-27 09:10:58,732][20347] Updated weights for policy 0, policy_version 511829 (0.0008) [2023-12-27 09:10:58,792][20347] Updated weights for policy 0, policy_version 511839 (0.0008) [2023-12-27 09:10:59,345][20381] Updated weights for policy 1, policy_version 511699 (0.0007) [2023-12-27 09:10:59,408][20381] Updated weights for policy 1, policy_version 511709 (0.0007) [2023-12-27 09:10:59,471][20381] Updated weights for policy 1, policy_version 511719 (0.0007) [2023-12-27 09:10:59,530][20347] Updated weights for policy 0, policy_version 511849 (0.0008) [2023-12-27 09:10:59,584][20347] Updated weights for policy 0, policy_version 511859 (0.0011) [2023-12-27 09:10:59,633][20347] Updated weights for policy 0, policy_version 511869 (0.0010) [2023-12-27 09:11:00,168][20381] Updated weights for policy 1, policy_version 511729 (0.0007) [2023-12-27 09:11:00,215][20381] Updated weights for policy 1, policy_version 511739 (0.0008) [2023-12-27 09:11:00,272][20381] Updated weights for policy 1, policy_version 511749 (0.0010) [2023-12-27 09:11:00,280][20347] Updated weights for policy 0, policy_version 511879 (0.0009) [2023-12-27 09:11:00,333][20381] Updated weights for policy 1, policy_version 511759 (0.0010) [2023-12-27 09:11:00,335][20347] Updated weights for policy 0, policy_version 511889 (0.0010) [2023-12-27 09:11:00,390][20347] Updated weights for policy 0, policy_version 511899 (0.0010) [2023-12-27 09:11:00,977][20381] Updated weights for policy 1, policy_version 511769 (0.0006) [2023-12-27 09:11:01,034][20381] Updated weights for policy 1, policy_version 511779 (0.0007) [2023-12-27 09:11:01,061][20347] Updated weights for policy 0, policy_version 511909 (0.0009) [2023-12-27 09:11:01,088][20381] Updated weights for policy 1, policy_version 511789 (0.0009) [2023-12-27 09:11:01,115][20347] Updated weights for policy 0, policy_version 511919 (0.0008) [2023-12-27 09:11:01,169][20347] Updated weights for policy 0, policy_version 511929 (0.0008) [2023-12-27 09:11:01,740][20381] Updated weights for policy 1, policy_version 511799 (0.0009) [2023-12-27 09:11:01,798][20381] Updated weights for policy 1, policy_version 511809 (0.0011) [2023-12-27 09:11:01,848][20381] Updated weights for policy 1, policy_version 511819 (0.0009) [2023-12-27 09:11:01,887][20347] Updated weights for policy 0, policy_version 511939 (0.0009) [2023-12-27 09:11:01,942][20347] Updated weights for policy 0, policy_version 511949 (0.0010) [2023-12-27 09:11:01,997][20347] Updated weights for policy 0, policy_version 511959 (0.0010) [2023-12-27 09:11:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 262127616. Throughput: 0: 10088.4, 1: 10596.5. Samples: 262138052. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:11:02,363][19170] Avg episode reward: [(0, '118.610'), (1, '894.660')] [2023-12-27 09:11:02,540][20381] Updated weights for policy 1, policy_version 511829 (0.0010) [2023-12-27 09:11:02,596][20381] Updated weights for policy 1, policy_version 511839 (0.0007) [2023-12-27 09:11:02,602][20347] Updated weights for policy 0, policy_version 511969 (0.0008) [2023-12-27 09:11:02,655][20347] Updated weights for policy 0, policy_version 511979 (0.0010) [2023-12-27 09:11:02,663][20381] Updated weights for policy 1, policy_version 511849 (0.0006) [2023-12-27 09:11:02,713][20347] Updated weights for policy 0, policy_version 511989 (0.0010) [2023-12-27 09:11:02,769][20347] Updated weights for policy 0, policy_version 511999 (0.0010) [2023-12-27 09:11:03,376][20381] Updated weights for policy 1, policy_version 511859 (0.0006) [2023-12-27 09:11:03,420][20347] Updated weights for policy 0, policy_version 512009 (0.0010) [2023-12-27 09:11:03,433][20381] Updated weights for policy 1, policy_version 511869 (0.0006) [2023-12-27 09:11:03,478][20347] Updated weights for policy 0, policy_version 512019 (0.0010) [2023-12-27 09:11:03,480][20381] Updated weights for policy 1, policy_version 511879 (0.0007) [2023-12-27 09:11:03,537][20347] Updated weights for policy 0, policy_version 512029 (0.0010) [2023-12-27 09:11:04,084][20381] Updated weights for policy 1, policy_version 511889 (0.0007) [2023-12-27 09:11:04,144][20381] Updated weights for policy 1, policy_version 511899 (0.0007) [2023-12-27 09:11:04,190][20347] Updated weights for policy 0, policy_version 512039 (0.0010) [2023-12-27 09:11:04,200][20381] Updated weights for policy 1, policy_version 511909 (0.0006) [2023-12-27 09:11:04,247][20347] Updated weights for policy 0, policy_version 512049 (0.0008) [2023-12-27 09:11:04,259][20381] Updated weights for policy 1, policy_version 511919 (0.0007) [2023-12-27 09:11:04,299][20347] Updated weights for policy 0, policy_version 512059 (0.0010) [2023-12-27 09:11:04,946][20381] Updated weights for policy 1, policy_version 511929 (0.0007) [2023-12-27 09:11:04,955][20347] Updated weights for policy 0, policy_version 512069 (0.0010) [2023-12-27 09:11:05,001][20381] Updated weights for policy 1, policy_version 511939 (0.0006) [2023-12-27 09:11:05,011][20347] Updated weights for policy 0, policy_version 512079 (0.0010) [2023-12-27 09:11:05,048][20381] Updated weights for policy 1, policy_version 511949 (0.0007) [2023-12-27 09:11:05,074][20347] Updated weights for policy 0, policy_version 512089 (0.0010) [2023-12-27 09:11:05,678][20347] Updated weights for policy 0, policy_version 512099 (0.0010) [2023-12-27 09:11:05,700][20381] Updated weights for policy 1, policy_version 511959 (0.0009) [2023-12-27 09:11:05,740][20347] Updated weights for policy 0, policy_version 512109 (0.0006) [2023-12-27 09:11:05,756][20381] Updated weights for policy 1, policy_version 511969 (0.0009) [2023-12-27 09:11:05,793][20347] Updated weights for policy 0, policy_version 512119 (0.0010) [2023-12-27 09:11:05,810][20381] Updated weights for policy 1, policy_version 511979 (0.0007) [2023-12-27 09:11:06,426][20347] Updated weights for policy 0, policy_version 512129 (0.0010) [2023-12-27 09:11:06,486][20347] Updated weights for policy 0, policy_version 512139 (0.0010) [2023-12-27 09:11:06,489][20381] Updated weights for policy 1, policy_version 511989 (0.0008) [2023-12-27 09:11:06,543][20347] Updated weights for policy 0, policy_version 512149 (0.0009) [2023-12-27 09:11:06,553][20381] Updated weights for policy 1, policy_version 511999 (0.0011) [2023-12-27 09:11:06,599][20347] Updated weights for policy 0, policy_version 512159 (0.0010) [2023-12-27 09:11:06,616][20381] Updated weights for policy 1, policy_version 512009 (0.0011) [2023-12-27 09:11:07,200][20347] Updated weights for policy 0, policy_version 512169 (0.0010) [2023-12-27 09:11:07,253][20347] Updated weights for policy 0, policy_version 512179 (0.0010) [2023-12-27 09:11:07,287][20381] Updated weights for policy 1, policy_version 512019 (0.0009) [2023-12-27 09:11:07,302][20347] Updated weights for policy 0, policy_version 512189 (0.0010) [2023-12-27 09:11:07,341][20381] Updated weights for policy 1, policy_version 512029 (0.0005) [2023-12-27 09:11:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 262234112. Throughput: 0: 10132.9, 1: 10609.4. Samples: 262201088. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:11:07,363][19170] Avg episode reward: [(0, '166.980'), (1, '932.430')] [2023-12-27 09:11:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000512192_131137536.pth... [2023-12-27 09:11:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000510976_130826240.pth [2023-12-27 09:11:07,391][20381] Updated weights for policy 1, policy_version 512039 (0.0008) [2023-12-27 09:11:07,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000512048_131104768.pth... [2023-12-27 09:11:07,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000510800_130785280.pth [2023-12-27 09:11:07,894][20347] Updated weights for policy 0, policy_version 512199 (0.0007) [2023-12-27 09:11:07,949][20347] Updated weights for policy 0, policy_version 512209 (0.0009) [2023-12-27 09:11:07,996][20347] Updated weights for policy 0, policy_version 512219 (0.0010) [2023-12-27 09:11:08,057][20381] Updated weights for policy 1, policy_version 512049 (0.0010) [2023-12-27 09:11:08,115][20381] Updated weights for policy 1, policy_version 512059 (0.0007) [2023-12-27 09:11:08,170][20381] Updated weights for policy 1, policy_version 512069 (0.0007) [2023-12-27 09:11:08,219][20381] Updated weights for policy 1, policy_version 512079 (0.0006) [2023-12-27 09:11:08,636][20347] Updated weights for policy 0, policy_version 512229 (0.0010) [2023-12-27 09:11:08,694][20347] Updated weights for policy 0, policy_version 512239 (0.0010) [2023-12-27 09:11:08,757][20347] Updated weights for policy 0, policy_version 512249 (0.0006) [2023-12-27 09:11:08,862][20381] Updated weights for policy 1, policy_version 512089 (0.0008) [2023-12-27 09:11:08,930][20381] Updated weights for policy 1, policy_version 512099 (0.0009) [2023-12-27 09:11:08,991][20381] Updated weights for policy 1, policy_version 512109 (0.0008) [2023-12-27 09:11:09,392][20347] Updated weights for policy 0, policy_version 512259 (0.0007) [2023-12-27 09:11:09,459][20347] Updated weights for policy 0, policy_version 512269 (0.0010) [2023-12-27 09:11:09,514][20347] Updated weights for policy 0, policy_version 512279 (0.0009) [2023-12-27 09:11:09,703][20381] Updated weights for policy 1, policy_version 512119 (0.0007) [2023-12-27 09:11:09,754][20381] Updated weights for policy 1, policy_version 512129 (0.0007) [2023-12-27 09:11:09,803][20381] Updated weights for policy 1, policy_version 512139 (0.0007) [2023-12-27 09:11:10,285][20347] Updated weights for policy 0, policy_version 512289 (0.0009) [2023-12-27 09:11:10,338][20347] Updated weights for policy 0, policy_version 512299 (0.0008) [2023-12-27 09:11:10,401][20347] Updated weights for policy 0, policy_version 512309 (0.0006) [2023-12-27 09:11:10,460][20347] Updated weights for policy 0, policy_version 512319 (0.0006) [2023-12-27 09:11:10,480][20381] Updated weights for policy 1, policy_version 512149 (0.0007) [2023-12-27 09:11:10,540][20381] Updated weights for policy 1, policy_version 512159 (0.0006) [2023-12-27 09:11:10,600][20381] Updated weights for policy 1, policy_version 512169 (0.0007) [2023-12-27 09:11:11,088][20347] Updated weights for policy 0, policy_version 512329 (0.0007) [2023-12-27 09:11:11,149][20347] Updated weights for policy 0, policy_version 512339 (0.0008) [2023-12-27 09:11:11,208][20347] Updated weights for policy 0, policy_version 512349 (0.0007) [2023-12-27 09:11:11,259][20381] Updated weights for policy 1, policy_version 512179 (0.0006) [2023-12-27 09:11:11,317][20381] Updated weights for policy 1, policy_version 512189 (0.0008) [2023-12-27 09:11:11,386][20381] Updated weights for policy 1, policy_version 512199 (0.0009) [2023-12-27 09:11:11,936][20347] Updated weights for policy 0, policy_version 512359 (0.0006) [2023-12-27 09:11:11,999][20347] Updated weights for policy 0, policy_version 512369 (0.0007) [2023-12-27 09:11:12,059][20347] Updated weights for policy 0, policy_version 512379 (0.0006) [2023-12-27 09:11:12,089][20381] Updated weights for policy 1, policy_version 512209 (0.0008) [2023-12-27 09:11:12,140][20381] Updated weights for policy 1, policy_version 512219 (0.0008) [2023-12-27 09:11:12,199][20381] Updated weights for policy 1, policy_version 512229 (0.0008) [2023-12-27 09:11:12,259][20381] Updated weights for policy 1, policy_version 512239 (0.0007) [2023-12-27 09:11:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 262340608. Throughput: 0: 10259.2, 1: 10598.9. Samples: 262327644. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:11:12,363][19170] Avg episode reward: [(0, '204.530'), (1, '961.360')] [2023-12-27 09:11:12,681][20347] Updated weights for policy 0, policy_version 512389 (0.0007) [2023-12-27 09:11:12,745][20347] Updated weights for policy 0, policy_version 512399 (0.0007) [2023-12-27 09:11:12,808][20347] Updated weights for policy 0, policy_version 512409 (0.0006) [2023-12-27 09:11:12,931][20381] Updated weights for policy 1, policy_version 512249 (0.0005) [2023-12-27 09:11:12,992][20381] Updated weights for policy 1, policy_version 512259 (0.0006) [2023-12-27 09:11:13,050][20381] Updated weights for policy 1, policy_version 512269 (0.0005) [2023-12-27 09:11:13,388][20347] Updated weights for policy 0, policy_version 512419 (0.0007) [2023-12-27 09:11:13,461][20347] Updated weights for policy 0, policy_version 512429 (0.0010) [2023-12-27 09:11:13,512][20347] Updated weights for policy 0, policy_version 512439 (0.0010) [2023-12-27 09:11:13,587][20381] Updated weights for policy 1, policy_version 512279 (0.0007) [2023-12-27 09:11:13,643][20381] Updated weights for policy 1, policy_version 512289 (0.0008) [2023-12-27 09:11:13,707][20381] Updated weights for policy 1, policy_version 512299 (0.0009) [2023-12-27 09:11:14,105][20347] Updated weights for policy 0, policy_version 512449 (0.0010) [2023-12-27 09:11:14,172][20347] Updated weights for policy 0, policy_version 512459 (0.0010) [2023-12-27 09:11:14,229][20347] Updated weights for policy 0, policy_version 512469 (0.0008) [2023-12-27 09:11:14,286][20347] Updated weights for policy 0, policy_version 512479 (0.0005) [2023-12-27 09:11:14,440][20381] Updated weights for policy 1, policy_version 512309 (0.0008) [2023-12-27 09:11:14,485][20381] Updated weights for policy 1, policy_version 512319 (0.0008) [2023-12-27 09:11:14,546][20381] Updated weights for policy 1, policy_version 512329 (0.0009) [2023-12-27 09:11:14,827][20347] Updated weights for policy 0, policy_version 512489 (0.0007) [2023-12-27 09:11:14,885][20347] Updated weights for policy 0, policy_version 512499 (0.0006) [2023-12-27 09:11:14,945][20347] Updated weights for policy 0, policy_version 512509 (0.0011) [2023-12-27 09:11:15,254][20381] Updated weights for policy 1, policy_version 512339 (0.0008) [2023-12-27 09:11:15,317][20381] Updated weights for policy 1, policy_version 512349 (0.0008) [2023-12-27 09:11:15,380][20381] Updated weights for policy 1, policy_version 512359 (0.0005) [2023-12-27 09:11:15,563][20347] Updated weights for policy 0, policy_version 512519 (0.0010) [2023-12-27 09:11:15,612][20347] Updated weights for policy 0, policy_version 512529 (0.0010) [2023-12-27 09:11:15,665][20347] Updated weights for policy 0, policy_version 512539 (0.0008) [2023-12-27 09:11:15,974][20381] Updated weights for policy 1, policy_version 512369 (0.0009) [2023-12-27 09:11:16,025][20381] Updated weights for policy 1, policy_version 512379 (0.0005) [2023-12-27 09:11:16,075][20381] Updated weights for policy 1, policy_version 512389 (0.0005) [2023-12-27 09:11:16,131][20381] Updated weights for policy 1, policy_version 512399 (0.0007) [2023-12-27 09:11:16,284][20347] Updated weights for policy 0, policy_version 512549 (0.0006) [2023-12-27 09:11:16,362][20347] Updated weights for policy 0, policy_version 512559 (0.0005) [2023-12-27 09:11:16,407][20347] Updated weights for policy 0, policy_version 512569 (0.0006) [2023-12-27 09:11:16,800][20381] Updated weights for policy 1, policy_version 512409 (0.0006) [2023-12-27 09:11:16,848][20381] Updated weights for policy 1, policy_version 512419 (0.0005) [2023-12-27 09:11:16,896][20381] Updated weights for policy 1, policy_version 512429 (0.0005) [2023-12-27 09:11:16,995][20347] Updated weights for policy 0, policy_version 512579 (0.0006) [2023-12-27 09:11:17,048][20347] Updated weights for policy 0, policy_version 512589 (0.0005) [2023-12-27 09:11:17,102][20347] Updated weights for policy 0, policy_version 512599 (0.0005) [2023-12-27 09:11:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 262447104. Throughput: 0: 10334.4, 1: 10538.3. Samples: 262456808. Policy #0 lag: (min: 20.0, avg: 23.4, max: 52.0) [2023-12-27 09:11:17,363][19170] Avg episode reward: [(0, '218.920'), (1, '932.880')] [2023-12-27 09:11:17,492][20381] Updated weights for policy 1, policy_version 512439 (0.0007) [2023-12-27 09:11:17,545][20381] Updated weights for policy 1, policy_version 512449 (0.0005) [2023-12-27 09:11:17,604][20381] Updated weights for policy 1, policy_version 512459 (0.0011) [2023-12-27 09:11:17,699][20347] Updated weights for policy 0, policy_version 512609 (0.0006) [2023-12-27 09:11:17,760][20347] Updated weights for policy 0, policy_version 512619 (0.0007) [2023-12-27 09:11:17,817][20347] Updated weights for policy 0, policy_version 512629 (0.0007) [2023-12-27 09:11:17,867][20347] Updated weights for policy 0, policy_version 512639 (0.0008) [2023-12-27 09:11:18,216][20381] Updated weights for policy 1, policy_version 512469 (0.0011) [2023-12-27 09:11:18,267][20381] Updated weights for policy 1, policy_version 512479 (0.0010) [2023-12-27 09:11:18,322][20381] Updated weights for policy 1, policy_version 512489 (0.0011) [2023-12-27 09:11:18,593][20347] Updated weights for policy 0, policy_version 512649 (0.0007) [2023-12-27 09:11:18,648][20347] Updated weights for policy 0, policy_version 512659 (0.0008) [2023-12-27 09:11:18,707][20347] Updated weights for policy 0, policy_version 512669 (0.0007) [2023-12-27 09:11:18,994][20381] Updated weights for policy 1, policy_version 512499 (0.0011) [2023-12-27 09:11:19,049][20381] Updated weights for policy 1, policy_version 512509 (0.0010) [2023-12-27 09:11:19,104][20381] Updated weights for policy 1, policy_version 512519 (0.0010) [2023-12-27 09:11:19,429][20347] Updated weights for policy 0, policy_version 512679 (0.0007) [2023-12-27 09:11:19,487][20347] Updated weights for policy 0, policy_version 512689 (0.0006) [2023-12-27 09:11:19,550][20347] Updated weights for policy 0, policy_version 512699 (0.0008) [2023-12-27 09:11:19,867][20381] Updated weights for policy 1, policy_version 512529 (0.0010) [2023-12-27 09:11:19,932][20381] Updated weights for policy 1, policy_version 512539 (0.0008) [2023-12-27 09:11:19,987][20381] Updated weights for policy 1, policy_version 512549 (0.0007) [2023-12-27 09:11:20,048][20381] Updated weights for policy 1, policy_version 512559 (0.0007) [2023-12-27 09:11:20,291][20347] Updated weights for policy 0, policy_version 512709 (0.0008) [2023-12-27 09:11:20,351][20347] Updated weights for policy 0, policy_version 512719 (0.0007) [2023-12-27 09:11:20,407][20347] Updated weights for policy 0, policy_version 512729 (0.0008) [2023-12-27 09:11:20,745][20381] Updated weights for policy 1, policy_version 512569 (0.0008) [2023-12-27 09:11:20,798][20381] Updated weights for policy 1, policy_version 512579 (0.0008) [2023-12-27 09:11:20,858][20381] Updated weights for policy 1, policy_version 512589 (0.0008) [2023-12-27 09:11:21,133][20347] Updated weights for policy 0, policy_version 512739 (0.0008) [2023-12-27 09:11:21,193][20347] Updated weights for policy 0, policy_version 512749 (0.0009) [2023-12-27 09:11:21,245][20347] Updated weights for policy 0, policy_version 512759 (0.0008) [2023-12-27 09:11:21,649][20381] Updated weights for policy 1, policy_version 512599 (0.0010) [2023-12-27 09:11:21,708][20381] Updated weights for policy 1, policy_version 512609 (0.0011) [2023-12-27 09:11:21,770][20381] Updated weights for policy 1, policy_version 512619 (0.0011) [2023-12-27 09:11:21,974][20347] Updated weights for policy 0, policy_version 512769 (0.0008) [2023-12-27 09:11:22,033][20347] Updated weights for policy 0, policy_version 512779 (0.0006) [2023-12-27 09:11:22,099][20347] Updated weights for policy 0, policy_version 512789 (0.0008) [2023-12-27 09:11:22,167][20347] Updated weights for policy 0, policy_version 512799 (0.0008) [2023-12-27 09:11:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 262545408. Throughput: 0: 10347.1, 1: 10498.7. Samples: 262517400. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:22,363][19170] Avg episode reward: [(0, '185.480'), (1, '846.120')] [2023-12-27 09:11:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000512624_131252224.pth... [2023-12-27 09:11:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000512800_131293184.pth... [2023-12-27 09:11:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000511440_130949120.pth [2023-12-27 09:11:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000511552_130973696.pth [2023-12-27 09:11:22,540][20381] Updated weights for policy 1, policy_version 512629 (0.0011) [2023-12-27 09:11:22,597][20381] Updated weights for policy 1, policy_version 512639 (0.0011) [2023-12-27 09:11:22,661][20381] Updated weights for policy 1, policy_version 512649 (0.0011) [2023-12-27 09:11:22,774][20347] Updated weights for policy 0, policy_version 512809 (0.0010) [2023-12-27 09:11:22,830][20347] Updated weights for policy 0, policy_version 512819 (0.0010) [2023-12-27 09:11:22,891][20347] Updated weights for policy 0, policy_version 512829 (0.0011) [2023-12-27 09:11:23,280][20381] Updated weights for policy 1, policy_version 512659 (0.0010) [2023-12-27 09:11:23,332][20381] Updated weights for policy 1, policy_version 512669 (0.0005) [2023-12-27 09:11:23,378][20381] Updated weights for policy 1, policy_version 512679 (0.0005) [2023-12-27 09:11:23,587][20347] Updated weights for policy 0, policy_version 512839 (0.0010) [2023-12-27 09:11:23,632][20347] Updated weights for policy 0, policy_version 512849 (0.0010) [2023-12-27 09:11:23,681][20347] Updated weights for policy 0, policy_version 512859 (0.0010) [2023-12-27 09:11:23,922][20381] Updated weights for policy 1, policy_version 512689 (0.0005) [2023-12-27 09:11:23,980][20381] Updated weights for policy 1, policy_version 512699 (0.0005) [2023-12-27 09:11:24,036][20381] Updated weights for policy 1, policy_version 512709 (0.0005) [2023-12-27 09:11:24,091][20381] Updated weights for policy 1, policy_version 512719 (0.0005) [2023-12-27 09:11:24,381][20347] Updated weights for policy 0, policy_version 512869 (0.0010) [2023-12-27 09:11:24,438][20347] Updated weights for policy 0, policy_version 512879 (0.0010) [2023-12-27 09:11:24,496][20347] Updated weights for policy 0, policy_version 512889 (0.0010) [2023-12-27 09:11:24,600][20381] Updated weights for policy 1, policy_version 512729 (0.0005) [2023-12-27 09:11:24,651][20381] Updated weights for policy 1, policy_version 512739 (0.0005) [2023-12-27 09:11:24,702][20381] Updated weights for policy 1, policy_version 512749 (0.0005) [2023-12-27 09:11:25,181][20347] Updated weights for policy 0, policy_version 512899 (0.0010) [2023-12-27 09:11:25,231][20347] Updated weights for policy 0, policy_version 512909 (0.0007) [2023-12-27 09:11:25,282][20347] Updated weights for policy 0, policy_version 512919 (0.0006) [2023-12-27 09:11:25,287][20381] Updated weights for policy 1, policy_version 512759 (0.0009) [2023-12-27 09:11:25,340][20381] Updated weights for policy 1, policy_version 512769 (0.0007) [2023-12-27 09:11:25,402][20381] Updated weights for policy 1, policy_version 512779 (0.0006) [2023-12-27 09:11:25,838][20347] Updated weights for policy 0, policy_version 512929 (0.0006) [2023-12-27 09:11:25,882][20347] Updated weights for policy 0, policy_version 512939 (0.0007) [2023-12-27 09:11:25,928][20347] Updated weights for policy 0, policy_version 512949 (0.0006) [2023-12-27 09:11:25,968][20381] Updated weights for policy 1, policy_version 512789 (0.0008) [2023-12-27 09:11:25,981][20347] Updated weights for policy 0, policy_version 512959 (0.0005) [2023-12-27 09:11:26,013][20381] Updated weights for policy 1, policy_version 512799 (0.0010) [2023-12-27 09:11:26,062][20381] Updated weights for policy 1, policy_version 512809 (0.0010) [2023-12-27 09:11:26,568][20347] Updated weights for policy 0, policy_version 512969 (0.0007) [2023-12-27 09:11:26,624][20347] Updated weights for policy 0, policy_version 512979 (0.0008) [2023-12-27 09:11:26,686][20347] Updated weights for policy 0, policy_version 512989 (0.0008) [2023-12-27 09:11:26,736][20381] Updated weights for policy 1, policy_version 512819 (0.0009) [2023-12-27 09:11:26,791][20381] Updated weights for policy 1, policy_version 512829 (0.0005) [2023-12-27 09:11:26,843][20381] Updated weights for policy 1, policy_version 512839 (0.0010) [2023-12-27 09:11:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 262651904. Throughput: 0: 10382.5, 1: 10516.1. Samples: 262643964. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:27,363][19170] Avg episode reward: [(0, '218.110'), (1, '874.630')] [2023-12-27 09:11:27,376][20347] Updated weights for policy 0, policy_version 512999 (0.0006) [2023-12-27 09:11:27,429][20347] Updated weights for policy 0, policy_version 513009 (0.0005) [2023-12-27 09:11:27,481][20347] Updated weights for policy 0, policy_version 513019 (0.0005) [2023-12-27 09:11:27,495][20381] Updated weights for policy 1, policy_version 512849 (0.0010) [2023-12-27 09:11:27,551][20381] Updated weights for policy 1, policy_version 512859 (0.0010) [2023-12-27 09:11:27,599][20381] Updated weights for policy 1, policy_version 512869 (0.0010) [2023-12-27 09:11:27,647][20381] Updated weights for policy 1, policy_version 512879 (0.0010) [2023-12-27 09:11:28,056][20347] Updated weights for policy 0, policy_version 513029 (0.0007) [2023-12-27 09:11:28,104][20347] Updated weights for policy 0, policy_version 513039 (0.0008) [2023-12-27 09:11:28,155][20347] Updated weights for policy 0, policy_version 513049 (0.0005) [2023-12-27 09:11:28,280][20381] Updated weights for policy 1, policy_version 512889 (0.0006) [2023-12-27 09:11:28,329][20381] Updated weights for policy 1, policy_version 512899 (0.0006) [2023-12-27 09:11:28,388][20381] Updated weights for policy 1, policy_version 512909 (0.0008) [2023-12-27 09:11:28,764][20347] Updated weights for policy 0, policy_version 513059 (0.0006) [2023-12-27 09:11:28,823][20347] Updated weights for policy 0, policy_version 513069 (0.0006) [2023-12-27 09:11:28,882][20347] Updated weights for policy 0, policy_version 513079 (0.0005) [2023-12-27 09:11:29,045][20381] Updated weights for policy 1, policy_version 512919 (0.0007) [2023-12-27 09:11:29,110][20381] Updated weights for policy 1, policy_version 512929 (0.0005) [2023-12-27 09:11:29,174][20381] Updated weights for policy 1, policy_version 512939 (0.0006) [2023-12-27 09:11:29,511][20347] Updated weights for policy 0, policy_version 513089 (0.0005) [2023-12-27 09:11:29,554][20347] Updated weights for policy 0, policy_version 513099 (0.0006) [2023-12-27 09:11:29,605][20347] Updated weights for policy 0, policy_version 513109 (0.0005) [2023-12-27 09:11:29,656][20347] Updated weights for policy 0, policy_version 513119 (0.0006) [2023-12-27 09:11:29,857][20381] Updated weights for policy 1, policy_version 512949 (0.0007) [2023-12-27 09:11:29,909][20381] Updated weights for policy 1, policy_version 512959 (0.0007) [2023-12-27 09:11:29,972][20381] Updated weights for policy 1, policy_version 512969 (0.0007) [2023-12-27 09:11:30,267][20347] Updated weights for policy 0, policy_version 513129 (0.0008) [2023-12-27 09:11:30,319][20347] Updated weights for policy 0, policy_version 513139 (0.0007) [2023-12-27 09:11:30,376][20347] Updated weights for policy 0, policy_version 513149 (0.0008) [2023-12-27 09:11:30,561][20381] Updated weights for policy 1, policy_version 512979 (0.0006) [2023-12-27 09:11:30,617][20381] Updated weights for policy 1, policy_version 512989 (0.0008) [2023-12-27 09:11:30,671][20381] Updated weights for policy 1, policy_version 512999 (0.0008) [2023-12-27 09:11:31,026][20347] Updated weights for policy 0, policy_version 513159 (0.0008) [2023-12-27 09:11:31,088][20347] Updated weights for policy 0, policy_version 513169 (0.0006) [2023-12-27 09:11:31,156][20347] Updated weights for policy 0, policy_version 513179 (0.0007) [2023-12-27 09:11:31,413][20381] Updated weights for policy 1, policy_version 513009 (0.0008) [2023-12-27 09:11:31,464][20381] Updated weights for policy 1, policy_version 513019 (0.0008) [2023-12-27 09:11:31,520][20381] Updated weights for policy 1, policy_version 513029 (0.0006) [2023-12-27 09:11:31,577][20381] Updated weights for policy 1, policy_version 513039 (0.0009) [2023-12-27 09:11:31,816][20347] Updated weights for policy 0, policy_version 513189 (0.0007) [2023-12-27 09:11:31,877][20347] Updated weights for policy 0, policy_version 513199 (0.0009) [2023-12-27 09:11:31,936][20347] Updated weights for policy 0, policy_version 513209 (0.0008) [2023-12-27 09:11:32,197][20381] Updated weights for policy 1, policy_version 513049 (0.0006) [2023-12-27 09:11:32,255][20381] Updated weights for policy 1, policy_version 513059 (0.0006) [2023-12-27 09:11:32,311][20381] Updated weights for policy 1, policy_version 513069 (0.0009) [2023-12-27 09:11:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 262766592. Throughput: 0: 10503.6, 1: 10484.8. Samples: 262773248. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:32,363][19170] Avg episode reward: [(0, '156.470'), (1, '942.240')] [2023-12-27 09:11:32,578][20347] Updated weights for policy 0, policy_version 513219 (0.0007) [2023-12-27 09:11:32,642][20347] Updated weights for policy 0, policy_version 513229 (0.0006) [2023-12-27 09:11:32,698][20347] Updated weights for policy 0, policy_version 513239 (0.0006) [2023-12-27 09:11:32,937][20381] Updated weights for policy 1, policy_version 513079 (0.0007) [2023-12-27 09:11:32,991][20381] Updated weights for policy 1, policy_version 513089 (0.0006) [2023-12-27 09:11:33,047][20381] Updated weights for policy 1, policy_version 513099 (0.0006) [2023-12-27 09:11:33,328][20347] Updated weights for policy 0, policy_version 513249 (0.0008) [2023-12-27 09:11:33,380][20347] Updated weights for policy 0, policy_version 513259 (0.0010) [2023-12-27 09:11:33,435][20347] Updated weights for policy 0, policy_version 513269 (0.0006) [2023-12-27 09:11:33,495][20347] Updated weights for policy 0, policy_version 513279 (0.0005) [2023-12-27 09:11:33,674][20381] Updated weights for policy 1, policy_version 513109 (0.0007) [2023-12-27 09:11:33,730][20381] Updated weights for policy 1, policy_version 513119 (0.0005) [2023-12-27 09:11:33,788][20381] Updated weights for policy 1, policy_version 513129 (0.0005) [2023-12-27 09:11:34,153][20347] Updated weights for policy 0, policy_version 513289 (0.0006) [2023-12-27 09:11:34,211][20347] Updated weights for policy 0, policy_version 513299 (0.0007) [2023-12-27 09:11:34,265][20347] Updated weights for policy 0, policy_version 513309 (0.0010) [2023-12-27 09:11:34,401][20381] Updated weights for policy 1, policy_version 513139 (0.0007) [2023-12-27 09:11:34,462][20381] Updated weights for policy 1, policy_version 513149 (0.0010) [2023-12-27 09:11:34,516][20381] Updated weights for policy 1, policy_version 513159 (0.0009) [2023-12-27 09:11:34,919][20347] Updated weights for policy 0, policy_version 513319 (0.0010) [2023-12-27 09:11:34,978][20347] Updated weights for policy 0, policy_version 513329 (0.0010) [2023-12-27 09:11:35,039][20347] Updated weights for policy 0, policy_version 513339 (0.0010) [2023-12-27 09:11:35,136][20381] Updated weights for policy 1, policy_version 513169 (0.0006) [2023-12-27 09:11:35,198][20381] Updated weights for policy 1, policy_version 513179 (0.0011) [2023-12-27 09:11:35,249][20381] Updated weights for policy 1, policy_version 513189 (0.0010) [2023-12-27 09:11:35,302][20381] Updated weights for policy 1, policy_version 513199 (0.0010) [2023-12-27 09:11:35,633][20347] Updated weights for policy 0, policy_version 513349 (0.0010) [2023-12-27 09:11:35,681][20347] Updated weights for policy 0, policy_version 513359 (0.0005) [2023-12-27 09:11:35,742][20347] Updated weights for policy 0, policy_version 513369 (0.0005) [2023-12-27 09:11:35,995][20381] Updated weights for policy 1, policy_version 513209 (0.0010) [2023-12-27 09:11:36,037][20381] Updated weights for policy 1, policy_version 513219 (0.0010) [2023-12-27 09:11:36,086][20381] Updated weights for policy 1, policy_version 513229 (0.0010) [2023-12-27 09:11:36,332][20347] Updated weights for policy 0, policy_version 513379 (0.0007) [2023-12-27 09:11:36,391][20347] Updated weights for policy 0, policy_version 513389 (0.0010) [2023-12-27 09:11:36,454][20347] Updated weights for policy 0, policy_version 513399 (0.0010) [2023-12-27 09:11:36,814][20381] Updated weights for policy 1, policy_version 513239 (0.0011) [2023-12-27 09:11:36,866][20381] Updated weights for policy 1, policy_version 513249 (0.0011) [2023-12-27 09:11:36,929][20381] Updated weights for policy 1, policy_version 513259 (0.0011) [2023-12-27 09:11:37,102][20347] Updated weights for policy 0, policy_version 513409 (0.0010) [2023-12-27 09:11:37,171][20347] Updated weights for policy 0, policy_version 513419 (0.0005) [2023-12-27 09:11:37,232][20347] Updated weights for policy 0, policy_version 513429 (0.0005) [2023-12-27 09:11:37,294][20347] Updated weights for policy 0, policy_version 513439 (0.0006) [2023-12-27 09:11:37,362][19170] Fps is (10 sec: 22118.2, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 262873088. Throughput: 0: 10539.6, 1: 10476.3. Samples: 262837828. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:37,363][19170] Avg episode reward: [(0, '181.350'), (1, '883.960')] [2023-12-27 09:11:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000513264_131416064.pth... [2023-12-27 09:11:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000513440_131457024.pth... [2023-12-27 09:11:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000512048_131104768.pth [2023-12-27 09:11:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000512192_131137536.pth [2023-12-27 09:11:37,637][20381] Updated weights for policy 1, policy_version 513269 (0.0011) [2023-12-27 09:11:37,702][20381] Updated weights for policy 1, policy_version 513279 (0.0011) [2023-12-27 09:11:37,765][20381] Updated weights for policy 1, policy_version 513289 (0.0011) [2023-12-27 09:11:37,797][20347] Updated weights for policy 0, policy_version 513449 (0.0005) [2023-12-27 09:11:37,858][20347] Updated weights for policy 0, policy_version 513459 (0.0005) [2023-12-27 09:11:37,919][20347] Updated weights for policy 0, policy_version 513469 (0.0005) [2023-12-27 09:11:38,438][20381] Updated weights for policy 1, policy_version 513299 (0.0011) [2023-12-27 09:11:38,465][20347] Updated weights for policy 0, policy_version 513479 (0.0006) [2023-12-27 09:11:38,498][20381] Updated weights for policy 1, policy_version 513309 (0.0011) [2023-12-27 09:11:38,527][20347] Updated weights for policy 0, policy_version 513489 (0.0005) [2023-12-27 09:11:38,551][20381] Updated weights for policy 1, policy_version 513319 (0.0010) [2023-12-27 09:11:38,587][20347] Updated weights for policy 0, policy_version 513499 (0.0010) [2023-12-27 09:11:39,212][20347] Updated weights for policy 0, policy_version 513509 (0.0009) [2023-12-27 09:11:39,257][20381] Updated weights for policy 1, policy_version 513329 (0.0011) [2023-12-27 09:11:39,272][20347] Updated weights for policy 0, policy_version 513519 (0.0008) [2023-12-27 09:11:39,317][20381] Updated weights for policy 1, policy_version 513339 (0.0011) [2023-12-27 09:11:39,329][20347] Updated weights for policy 0, policy_version 513529 (0.0009) [2023-12-27 09:11:39,384][20381] Updated weights for policy 1, policy_version 513349 (0.0010) [2023-12-27 09:11:39,443][20381] Updated weights for policy 1, policy_version 513359 (0.0010) [2023-12-27 09:11:40,038][20347] Updated weights for policy 0, policy_version 513539 (0.0009) [2023-12-27 09:11:40,094][20347] Updated weights for policy 0, policy_version 513549 (0.0010) [2023-12-27 09:11:40,151][20347] Updated weights for policy 0, policy_version 513559 (0.0010) [2023-12-27 09:11:40,152][20381] Updated weights for policy 1, policy_version 513369 (0.0010) [2023-12-27 09:11:40,204][20381] Updated weights for policy 1, policy_version 513379 (0.0011) [2023-12-27 09:11:40,255][20381] Updated weights for policy 1, policy_version 513389 (0.0011) [2023-12-27 09:11:40,839][20347] Updated weights for policy 0, policy_version 513569 (0.0010) [2023-12-27 09:11:40,887][20347] Updated weights for policy 0, policy_version 513579 (0.0005) [2023-12-27 09:11:40,944][20347] Updated weights for policy 0, policy_version 513589 (0.0005) [2023-12-27 09:11:40,976][20381] Updated weights for policy 1, policy_version 513399 (0.0009) [2023-12-27 09:11:40,994][20347] Updated weights for policy 0, policy_version 513599 (0.0006) [2023-12-27 09:11:41,033][20381] Updated weights for policy 1, policy_version 513409 (0.0007) [2023-12-27 09:11:41,094][20381] Updated weights for policy 1, policy_version 513419 (0.0006) [2023-12-27 09:11:41,627][20347] Updated weights for policy 0, policy_version 513609 (0.0011) [2023-12-27 09:11:41,692][20347] Updated weights for policy 0, policy_version 513619 (0.0010) [2023-12-27 09:11:41,754][20347] Updated weights for policy 0, policy_version 513629 (0.0007) [2023-12-27 09:11:41,796][20381] Updated weights for policy 1, policy_version 513429 (0.0008) [2023-12-27 09:11:41,855][20381] Updated weights for policy 1, policy_version 513439 (0.0008) [2023-12-27 09:11:41,913][20381] Updated weights for policy 1, policy_version 513449 (0.0007) [2023-12-27 09:11:42,362][19170] Fps is (10 sec: 20480.4, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 262971392. Throughput: 0: 10639.9, 1: 10388.6. Samples: 262962884. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:42,363][19170] Avg episode reward: [(0, '182.150'), (1, '912.780')] [2023-12-27 09:11:42,421][20347] Updated weights for policy 0, policy_version 513639 (0.0007) [2023-12-27 09:11:42,481][20347] Updated weights for policy 0, policy_version 513649 (0.0008) [2023-12-27 09:11:42,537][20347] Updated weights for policy 0, policy_version 513659 (0.0008) [2023-12-27 09:11:42,622][20381] Updated weights for policy 1, policy_version 513459 (0.0008) [2023-12-27 09:11:42,683][20381] Updated weights for policy 1, policy_version 513469 (0.0008) [2023-12-27 09:11:42,745][20381] Updated weights for policy 1, policy_version 513479 (0.0008) [2023-12-27 09:11:43,217][20347] Updated weights for policy 0, policy_version 513669 (0.0008) [2023-12-27 09:11:43,264][20347] Updated weights for policy 0, policy_version 513679 (0.0007) [2023-12-27 09:11:43,312][20347] Updated weights for policy 0, policy_version 513689 (0.0008) [2023-12-27 09:11:43,444][20381] Updated weights for policy 1, policy_version 513489 (0.0007) [2023-12-27 09:11:43,501][20381] Updated weights for policy 1, policy_version 513499 (0.0007) [2023-12-27 09:11:43,551][20381] Updated weights for policy 1, policy_version 513509 (0.0008) [2023-12-27 09:11:43,600][20381] Updated weights for policy 1, policy_version 513519 (0.0007) [2023-12-27 09:11:44,022][20347] Updated weights for policy 0, policy_version 513699 (0.0007) [2023-12-27 09:11:44,078][20347] Updated weights for policy 0, policy_version 513709 (0.0007) [2023-12-27 09:11:44,129][20347] Updated weights for policy 0, policy_version 513719 (0.0007) [2023-12-27 09:11:44,298][20381] Updated weights for policy 1, policy_version 513529 (0.0006) [2023-12-27 09:11:44,345][20381] Updated weights for policy 1, policy_version 513539 (0.0007) [2023-12-27 09:11:44,394][20381] Updated weights for policy 1, policy_version 513549 (0.0009) [2023-12-27 09:11:44,779][20347] Updated weights for policy 0, policy_version 513729 (0.0007) [2023-12-27 09:11:44,833][20347] Updated weights for policy 0, policy_version 513739 (0.0007) [2023-12-27 09:11:44,895][20347] Updated weights for policy 0, policy_version 513749 (0.0008) [2023-12-27 09:11:44,955][20347] Updated weights for policy 0, policy_version 513759 (0.0008) [2023-12-27 09:11:45,044][20381] Updated weights for policy 1, policy_version 513559 (0.0008) [2023-12-27 09:11:45,100][20381] Updated weights for policy 1, policy_version 513569 (0.0008) [2023-12-27 09:11:45,156][20381] Updated weights for policy 1, policy_version 513579 (0.0008) [2023-12-27 09:11:45,569][20347] Updated weights for policy 0, policy_version 513769 (0.0006) [2023-12-27 09:11:45,623][20347] Updated weights for policy 0, policy_version 513779 (0.0005) [2023-12-27 09:11:45,672][20347] Updated weights for policy 0, policy_version 513789 (0.0005) [2023-12-27 09:11:45,893][20381] Updated weights for policy 1, policy_version 513589 (0.0007) [2023-12-27 09:11:45,942][20381] Updated weights for policy 1, policy_version 513599 (0.0010) [2023-12-27 09:11:45,988][20381] Updated weights for policy 1, policy_version 513609 (0.0010) [2023-12-27 09:11:46,269][20347] Updated weights for policy 0, policy_version 513799 (0.0005) [2023-12-27 09:11:46,330][20347] Updated weights for policy 0, policy_version 513809 (0.0005) [2023-12-27 09:11:46,386][20347] Updated weights for policy 0, policy_version 513819 (0.0007) [2023-12-27 09:11:46,685][20381] Updated weights for policy 1, policy_version 513619 (0.0010) [2023-12-27 09:11:46,743][20381] Updated weights for policy 1, policy_version 513629 (0.0011) [2023-12-27 09:11:46,799][20381] Updated weights for policy 1, policy_version 513639 (0.0010) [2023-12-27 09:11:47,010][20347] Updated weights for policy 0, policy_version 513829 (0.0007) [2023-12-27 09:11:47,064][20347] Updated weights for policy 0, policy_version 513839 (0.0007) [2023-12-27 09:11:47,116][20347] Updated weights for policy 0, policy_version 513849 (0.0007) [2023-12-27 09:11:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 263077888. Throughput: 0: 10712.6, 1: 10375.4. Samples: 263087012. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:47,363][19170] Avg episode reward: [(0, '152.520'), (1, '951.500')] [2023-12-27 09:11:47,482][20381] Updated weights for policy 1, policy_version 513649 (0.0011) [2023-12-27 09:11:47,538][20381] Updated weights for policy 1, policy_version 513659 (0.0009) [2023-12-27 09:11:47,602][20381] Updated weights for policy 1, policy_version 513669 (0.0010) [2023-12-27 09:11:47,658][20381] Updated weights for policy 1, policy_version 513679 (0.0011) [2023-12-27 09:11:47,829][20347] Updated weights for policy 0, policy_version 513859 (0.0007) [2023-12-27 09:11:47,892][20347] Updated weights for policy 0, policy_version 513869 (0.0008) [2023-12-27 09:11:47,951][20347] Updated weights for policy 0, policy_version 513879 (0.0009) [2023-12-27 09:11:48,315][20381] Updated weights for policy 1, policy_version 513689 (0.0006) [2023-12-27 09:11:48,372][20381] Updated weights for policy 1, policy_version 513699 (0.0008) [2023-12-27 09:11:48,430][20381] Updated weights for policy 1, policy_version 513709 (0.0011) [2023-12-27 09:11:48,569][20347] Updated weights for policy 0, policy_version 513889 (0.0007) [2023-12-27 09:11:48,626][20347] Updated weights for policy 0, policy_version 513899 (0.0005) [2023-12-27 09:11:48,691][20347] Updated weights for policy 0, policy_version 513909 (0.0005) [2023-12-27 09:11:48,751][20347] Updated weights for policy 0, policy_version 513919 (0.0008) [2023-12-27 09:11:49,062][20381] Updated weights for policy 1, policy_version 513719 (0.0007) [2023-12-27 09:11:49,116][20381] Updated weights for policy 1, policy_version 513729 (0.0008) [2023-12-27 09:11:49,171][20381] Updated weights for policy 1, policy_version 513739 (0.0010) [2023-12-27 09:11:49,407][20347] Updated weights for policy 0, policy_version 513929 (0.0006) [2023-12-27 09:11:49,457][20347] Updated weights for policy 0, policy_version 513939 (0.0006) [2023-12-27 09:11:49,505][20347] Updated weights for policy 0, policy_version 513949 (0.0006) [2023-12-27 09:11:49,898][20381] Updated weights for policy 1, policy_version 513749 (0.0009) [2023-12-27 09:11:49,959][20381] Updated weights for policy 1, policy_version 513759 (0.0010) [2023-12-27 09:11:50,019][20381] Updated weights for policy 1, policy_version 513769 (0.0008) [2023-12-27 09:11:50,176][20347] Updated weights for policy 0, policy_version 513959 (0.0007) [2023-12-27 09:11:50,230][20347] Updated weights for policy 0, policy_version 513969 (0.0008) [2023-12-27 09:11:50,290][20347] Updated weights for policy 0, policy_version 513979 (0.0008) [2023-12-27 09:11:50,702][20381] Updated weights for policy 1, policy_version 513779 (0.0009) [2023-12-27 09:11:50,766][20381] Updated weights for policy 1, policy_version 513789 (0.0011) [2023-12-27 09:11:50,826][20381] Updated weights for policy 1, policy_version 513799 (0.0011) [2023-12-27 09:11:50,981][20347] Updated weights for policy 0, policy_version 513989 (0.0007) [2023-12-27 09:11:51,042][20347] Updated weights for policy 0, policy_version 513999 (0.0008) [2023-12-27 09:11:51,104][20347] Updated weights for policy 0, policy_version 514009 (0.0007) [2023-12-27 09:11:51,521][20381] Updated weights for policy 1, policy_version 513809 (0.0010) [2023-12-27 09:11:51,583][20381] Updated weights for policy 1, policy_version 513819 (0.0005) [2023-12-27 09:11:51,653][20381] Updated weights for policy 1, policy_version 513829 (0.0008) [2023-12-27 09:11:51,718][20381] Updated weights for policy 1, policy_version 513839 (0.0011) [2023-12-27 09:11:51,754][20347] Updated weights for policy 0, policy_version 514019 (0.0006) [2023-12-27 09:11:51,804][20347] Updated weights for policy 0, policy_version 514029 (0.0005) [2023-12-27 09:11:51,856][20347] Updated weights for policy 0, policy_version 514039 (0.0006) [2023-12-27 09:11:52,350][20381] Updated weights for policy 1, policy_version 513849 (0.0011) [2023-12-27 09:11:52,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 263176192. Throughput: 0: 10689.2, 1: 10361.5. Samples: 263148372. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:52,363][19170] Avg episode reward: [(0, '181.450'), (1, '966.760')] [2023-12-27 09:11:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000514048_131612672.pth... [2023-12-27 09:11:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000512800_131293184.pth [2023-12-27 09:11:52,411][20381] Updated weights for policy 1, policy_version 513859 (0.0011) [2023-12-27 09:11:52,468][20381] Updated weights for policy 1, policy_version 513869 (0.0011) [2023-12-27 09:11:52,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000513872_131571712.pth... [2023-12-27 09:11:52,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000512624_131252224.pth [2023-12-27 09:11:52,514][20347] Updated weights for policy 0, policy_version 514049 (0.0007) [2023-12-27 09:11:52,574][20347] Updated weights for policy 0, policy_version 514059 (0.0007) [2023-12-27 09:11:52,622][20347] Updated weights for policy 0, policy_version 514069 (0.0005) [2023-12-27 09:11:52,671][20347] Updated weights for policy 0, policy_version 514079 (0.0005) [2023-12-27 09:11:53,152][20381] Updated weights for policy 1, policy_version 513879 (0.0008) [2023-12-27 09:11:53,214][20381] Updated weights for policy 1, policy_version 513889 (0.0005) [2023-12-27 09:11:53,237][20347] Updated weights for policy 0, policy_version 514089 (0.0006) [2023-12-27 09:11:53,276][20381] Updated weights for policy 1, policy_version 513899 (0.0005) [2023-12-27 09:11:53,287][20347] Updated weights for policy 0, policy_version 514099 (0.0007) [2023-12-27 09:11:53,347][20347] Updated weights for policy 0, policy_version 514109 (0.0008) [2023-12-27 09:11:53,869][20381] Updated weights for policy 1, policy_version 513909 (0.0007) [2023-12-27 09:11:53,917][20381] Updated weights for policy 1, policy_version 513919 (0.0009) [2023-12-27 09:11:53,966][20381] Updated weights for policy 1, policy_version 513929 (0.0008) [2023-12-27 09:11:53,973][20347] Updated weights for policy 0, policy_version 514119 (0.0008) [2023-12-27 09:11:54,020][20347] Updated weights for policy 0, policy_version 514129 (0.0007) [2023-12-27 09:11:54,072][20347] Updated weights for policy 0, policy_version 514139 (0.0007) [2023-12-27 09:11:54,692][20381] Updated weights for policy 1, policy_version 513939 (0.0007) [2023-12-27 09:11:54,746][20381] Updated weights for policy 1, policy_version 513949 (0.0007) [2023-12-27 09:11:54,779][20347] Updated weights for policy 0, policy_version 514149 (0.0008) [2023-12-27 09:11:54,796][20381] Updated weights for policy 1, policy_version 513959 (0.0007) [2023-12-27 09:11:54,838][20347] Updated weights for policy 0, policy_version 514159 (0.0005) [2023-12-27 09:11:54,895][20347] Updated weights for policy 0, policy_version 514169 (0.0007) [2023-12-27 09:11:55,425][20381] Updated weights for policy 1, policy_version 513969 (0.0006) [2023-12-27 09:11:55,478][20381] Updated weights for policy 1, policy_version 513979 (0.0009) [2023-12-27 09:11:55,522][20381] Updated weights for policy 1, policy_version 513989 (0.0007) [2023-12-27 09:11:55,530][20347] Updated weights for policy 0, policy_version 514179 (0.0007) [2023-12-27 09:11:55,566][20381] Updated weights for policy 1, policy_version 513999 (0.0005) [2023-12-27 09:11:55,597][20347] Updated weights for policy 0, policy_version 514189 (0.0007) [2023-12-27 09:11:55,650][20347] Updated weights for policy 0, policy_version 514199 (0.0007) [2023-12-27 09:11:56,261][20347] Updated weights for policy 0, policy_version 514209 (0.0006) [2023-12-27 09:11:56,311][20381] Updated weights for policy 1, policy_version 514009 (0.0006) [2023-12-27 09:11:56,322][20347] Updated weights for policy 0, policy_version 514219 (0.0007) [2023-12-27 09:11:56,365][20381] Updated weights for policy 1, policy_version 514019 (0.0006) [2023-12-27 09:11:56,371][20347] Updated weights for policy 0, policy_version 514229 (0.0006) [2023-12-27 09:11:56,414][20347] Updated weights for policy 0, policy_version 514239 (0.0007) [2023-12-27 09:11:56,422][20381] Updated weights for policy 1, policy_version 514029 (0.0006) [2023-12-27 09:11:57,069][20381] Updated weights for policy 1, policy_version 514039 (0.0008) [2023-12-27 09:11:57,099][20347] Updated weights for policy 0, policy_version 514249 (0.0005) [2023-12-27 09:11:57,126][20381] Updated weights for policy 1, policy_version 514049 (0.0007) [2023-12-27 09:11:57,153][20347] Updated weights for policy 0, policy_version 514259 (0.0006) [2023-12-27 09:11:57,181][20381] Updated weights for policy 1, policy_version 514059 (0.0006) [2023-12-27 09:11:57,203][20347] Updated weights for policy 0, policy_version 514269 (0.0006) [2023-12-27 09:11:57,362][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 263290880. Throughput: 0: 10676.8, 1: 10360.5. Samples: 263274324. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:11:57,363][19170] Avg episode reward: [(0, '168.020'), (1, '927.890')] [2023-12-27 09:11:57,859][20347] Updated weights for policy 0, policy_version 514279 (0.0007) [2023-12-27 09:11:57,860][20381] Updated weights for policy 1, policy_version 514069 (0.0006) [2023-12-27 09:11:57,903][20347] Updated weights for policy 0, policy_version 514289 (0.0005) [2023-12-27 09:11:57,915][20381] Updated weights for policy 1, policy_version 514079 (0.0006) [2023-12-27 09:11:57,950][20347] Updated weights for policy 0, policy_version 514299 (0.0006) [2023-12-27 09:11:57,970][20381] Updated weights for policy 1, policy_version 514089 (0.0008) [2023-12-27 09:11:58,700][20347] Updated weights for policy 0, policy_version 514309 (0.0006) [2023-12-27 09:11:58,712][20381] Updated weights for policy 1, policy_version 514099 (0.0008) [2023-12-27 09:11:58,758][20347] Updated weights for policy 0, policy_version 514319 (0.0007) [2023-12-27 09:11:58,772][20381] Updated weights for policy 1, policy_version 514109 (0.0008) [2023-12-27 09:11:58,828][20347] Updated weights for policy 0, policy_version 514329 (0.0008) [2023-12-27 09:11:58,836][20381] Updated weights for policy 1, policy_version 514119 (0.0008) [2023-12-27 09:11:59,580][20347] Updated weights for policy 0, policy_version 514339 (0.0007) [2023-12-27 09:11:59,605][20381] Updated weights for policy 1, policy_version 514129 (0.0010) [2023-12-27 09:11:59,644][20347] Updated weights for policy 0, policy_version 514349 (0.0006) [2023-12-27 09:11:59,655][20381] Updated weights for policy 1, policy_version 514139 (0.0008) [2023-12-27 09:11:59,707][20347] Updated weights for policy 0, policy_version 514359 (0.0007) [2023-12-27 09:11:59,714][20381] Updated weights for policy 1, policy_version 514149 (0.0007) [2023-12-27 09:11:59,767][20381] Updated weights for policy 1, policy_version 514159 (0.0011) [2023-12-27 09:12:00,302][20347] Updated weights for policy 0, policy_version 514369 (0.0006) [2023-12-27 09:12:00,362][20347] Updated weights for policy 0, policy_version 514379 (0.0008) [2023-12-27 09:12:00,421][20347] Updated weights for policy 0, policy_version 514389 (0.0008) [2023-12-27 09:12:00,452][20381] Updated weights for policy 1, policy_version 514169 (0.0008) [2023-12-27 09:12:00,483][20347] Updated weights for policy 0, policy_version 514399 (0.0006) [2023-12-27 09:12:00,503][20381] Updated weights for policy 1, policy_version 514179 (0.0007) [2023-12-27 09:12:00,560][20381] Updated weights for policy 1, policy_version 514189 (0.0006) [2023-12-27 09:12:01,144][20347] Updated weights for policy 0, policy_version 514409 (0.0008) [2023-12-27 09:12:01,196][20347] Updated weights for policy 0, policy_version 514419 (0.0006) [2023-12-27 09:12:01,234][20381] Updated weights for policy 1, policy_version 514199 (0.0005) [2023-12-27 09:12:01,257][20347] Updated weights for policy 0, policy_version 514429 (0.0006) [2023-12-27 09:12:01,286][20381] Updated weights for policy 1, policy_version 514209 (0.0008) [2023-12-27 09:12:01,348][20381] Updated weights for policy 1, policy_version 514219 (0.0008) [2023-12-27 09:12:01,879][20347] Updated weights for policy 0, policy_version 514439 (0.0008) [2023-12-27 09:12:01,940][20347] Updated weights for policy 0, policy_version 514449 (0.0008) [2023-12-27 09:12:02,005][20347] Updated weights for policy 0, policy_version 514459 (0.0008) [2023-12-27 09:12:02,036][20381] Updated weights for policy 1, policy_version 514229 (0.0007) [2023-12-27 09:12:02,089][20381] Updated weights for policy 1, policy_version 514239 (0.0006) [2023-12-27 09:12:02,143][20381] Updated weights for policy 1, policy_version 514249 (0.0005) [2023-12-27 09:12:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 263389184. Throughput: 0: 10567.6, 1: 10306.0. Samples: 263396120. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:02,363][19170] Avg episode reward: [(0, '166.760'), (1, '912.730')] [2023-12-27 09:12:02,640][20347] Updated weights for policy 0, policy_version 514469 (0.0007) [2023-12-27 09:12:02,699][20347] Updated weights for policy 0, policy_version 514479 (0.0007) [2023-12-27 09:12:02,761][20347] Updated weights for policy 0, policy_version 514489 (0.0007) [2023-12-27 09:12:02,788][20381] Updated weights for policy 1, policy_version 514259 (0.0006) [2023-12-27 09:12:02,841][20381] Updated weights for policy 1, policy_version 514269 (0.0005) [2023-12-27 09:12:02,894][20381] Updated weights for policy 1, policy_version 514279 (0.0007) [2023-12-27 09:12:03,312][20347] Updated weights for policy 0, policy_version 514499 (0.0005) [2023-12-27 09:12:03,363][20347] Updated weights for policy 0, policy_version 514509 (0.0007) [2023-12-27 09:12:03,412][20347] Updated weights for policy 0, policy_version 514519 (0.0007) [2023-12-27 09:12:03,532][20381] Updated weights for policy 1, policy_version 514289 (0.0008) [2023-12-27 09:12:03,598][20381] Updated weights for policy 1, policy_version 514299 (0.0007) [2023-12-27 09:12:03,649][20381] Updated weights for policy 1, policy_version 514309 (0.0009) [2023-12-27 09:12:03,702][20381] Updated weights for policy 1, policy_version 514319 (0.0007) [2023-12-27 09:12:04,085][20347] Updated weights for policy 0, policy_version 514529 (0.0007) [2023-12-27 09:12:04,147][20347] Updated weights for policy 0, policy_version 514539 (0.0011) [2023-12-27 09:12:04,204][20347] Updated weights for policy 0, policy_version 514549 (0.0010) [2023-12-27 09:12:04,261][20347] Updated weights for policy 0, policy_version 514559 (0.0010) [2023-12-27 09:12:04,398][20381] Updated weights for policy 1, policy_version 514329 (0.0008) [2023-12-27 09:12:04,452][20381] Updated weights for policy 1, policy_version 514339 (0.0007) [2023-12-27 09:12:04,508][20381] Updated weights for policy 1, policy_version 514349 (0.0008) [2023-12-27 09:12:04,937][20347] Updated weights for policy 0, policy_version 514569 (0.0010) [2023-12-27 09:12:04,993][20347] Updated weights for policy 0, policy_version 514579 (0.0010) [2023-12-27 09:12:05,055][20347] Updated weights for policy 0, policy_version 514589 (0.0010) [2023-12-27 09:12:05,264][20381] Updated weights for policy 1, policy_version 514359 (0.0008) [2023-12-27 09:12:05,326][20381] Updated weights for policy 1, policy_version 514369 (0.0008) [2023-12-27 09:12:05,384][20381] Updated weights for policy 1, policy_version 514379 (0.0008) [2023-12-27 09:12:05,754][20347] Updated weights for policy 0, policy_version 514599 (0.0010) [2023-12-27 09:12:05,811][20347] Updated weights for policy 0, policy_version 514609 (0.0010) [2023-12-27 09:12:05,859][20347] Updated weights for policy 0, policy_version 514619 (0.0010) [2023-12-27 09:12:06,073][20381] Updated weights for policy 1, policy_version 514389 (0.0008) [2023-12-27 09:12:06,136][20381] Updated weights for policy 1, policy_version 514399 (0.0010) [2023-12-27 09:12:06,199][20381] Updated weights for policy 1, policy_version 514409 (0.0007) [2023-12-27 09:12:06,536][20347] Updated weights for policy 0, policy_version 514629 (0.0010) [2023-12-27 09:12:06,595][20347] Updated weights for policy 0, policy_version 514639 (0.0010) [2023-12-27 09:12:06,653][20347] Updated weights for policy 0, policy_version 514649 (0.0010) [2023-12-27 09:12:06,784][20381] Updated weights for policy 1, policy_version 514419 (0.0008) [2023-12-27 09:12:06,839][20381] Updated weights for policy 1, policy_version 514429 (0.0010) [2023-12-27 09:12:06,897][20381] Updated weights for policy 1, policy_version 514439 (0.0010) [2023-12-27 09:12:07,314][20347] Updated weights for policy 0, policy_version 514659 (0.0010) [2023-12-27 09:12:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 263487488. Throughput: 0: 10635.2, 1: 10302.3. Samples: 263459588. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:07,363][19170] Avg episode reward: [(0, '151.400'), (1, '902.850')] [2023-12-27 09:12:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000514448_131719168.pth... [2023-12-27 09:12:07,369][20347] Updated weights for policy 0, policy_version 514669 (0.0010) [2023-12-27 09:12:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000513264_131416064.pth [2023-12-27 09:12:07,435][20347] Updated weights for policy 0, policy_version 514679 (0.0010) [2023-12-27 09:12:07,481][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000514688_131776512.pth... [2023-12-27 09:12:07,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000513440_131457024.pth [2023-12-27 09:12:07,521][20381] Updated weights for policy 1, policy_version 514449 (0.0007) [2023-12-27 09:12:07,576][20381] Updated weights for policy 1, policy_version 514459 (0.0005) [2023-12-27 09:12:07,632][20381] Updated weights for policy 1, policy_version 514469 (0.0005) [2023-12-27 09:12:07,689][20381] Updated weights for policy 1, policy_version 514479 (0.0007) [2023-12-27 09:12:08,082][20347] Updated weights for policy 0, policy_version 514689 (0.0010) [2023-12-27 09:12:08,141][20347] Updated weights for policy 0, policy_version 514699 (0.0010) [2023-12-27 09:12:08,192][20347] Updated weights for policy 0, policy_version 514709 (0.0010) [2023-12-27 09:12:08,240][20347] Updated weights for policy 0, policy_version 514719 (0.0010) [2023-12-27 09:12:08,292][20381] Updated weights for policy 1, policy_version 514489 (0.0006) [2023-12-27 09:12:08,349][20381] Updated weights for policy 1, policy_version 514499 (0.0007) [2023-12-27 09:12:08,405][20381] Updated weights for policy 1, policy_version 514509 (0.0006) [2023-12-27 09:12:08,940][20347] Updated weights for policy 0, policy_version 514729 (0.0010) [2023-12-27 09:12:08,971][20381] Updated weights for policy 1, policy_version 514519 (0.0005) [2023-12-27 09:12:08,985][20347] Updated weights for policy 0, policy_version 514739 (0.0010) [2023-12-27 09:12:09,020][20381] Updated weights for policy 1, policy_version 514529 (0.0005) [2023-12-27 09:12:09,034][20347] Updated weights for policy 0, policy_version 514749 (0.0010) [2023-12-27 09:12:09,081][20381] Updated weights for policy 1, policy_version 514539 (0.0005) [2023-12-27 09:12:09,691][20381] Updated weights for policy 1, policy_version 514549 (0.0006) [2023-12-27 09:12:09,741][20381] Updated weights for policy 1, policy_version 514559 (0.0006) [2023-12-27 09:12:09,751][20347] Updated weights for policy 0, policy_version 514759 (0.0010) [2023-12-27 09:12:09,800][20381] Updated weights for policy 1, policy_version 514569 (0.0008) [2023-12-27 09:12:09,803][20347] Updated weights for policy 0, policy_version 514769 (0.0010) [2023-12-27 09:12:09,869][20347] Updated weights for policy 0, policy_version 514779 (0.0008) [2023-12-27 09:12:10,496][20381] Updated weights for policy 1, policy_version 514579 (0.0008) [2023-12-27 09:12:10,529][20347] Updated weights for policy 0, policy_version 514789 (0.0006) [2023-12-27 09:12:10,550][20381] Updated weights for policy 1, policy_version 514589 (0.0006) [2023-12-27 09:12:10,581][20347] Updated weights for policy 0, policy_version 514799 (0.0007) [2023-12-27 09:12:10,599][20381] Updated weights for policy 1, policy_version 514599 (0.0006) [2023-12-27 09:12:10,634][20347] Updated weights for policy 0, policy_version 514809 (0.0006) [2023-12-27 09:12:11,310][20381] Updated weights for policy 1, policy_version 514609 (0.0008) [2023-12-27 09:12:11,369][20347] Updated weights for policy 0, policy_version 514819 (0.0007) [2023-12-27 09:12:11,379][20381] Updated weights for policy 1, policy_version 514619 (0.0007) [2023-12-27 09:12:11,387][20292] KL-divergence is very high: 106.7387 [2023-12-27 09:12:11,434][20347] Updated weights for policy 0, policy_version 514829 (0.0008) [2023-12-27 09:12:11,438][20381] Updated weights for policy 1, policy_version 514629 (0.0006) [2023-12-27 09:12:11,495][20347] Updated weights for policy 0, policy_version 514839 (0.0007) [2023-12-27 09:12:11,502][20381] Updated weights for policy 1, policy_version 514639 (0.0006) [2023-12-27 09:12:12,154][20292] KL-divergence is very high: 103.5827 [2023-12-27 09:12:12,173][20381] Updated weights for policy 1, policy_version 514649 (0.0008) [2023-12-27 09:12:12,225][20347] Updated weights for policy 0, policy_version 514849 (0.0007) [2023-12-27 09:12:12,229][20381] Updated weights for policy 1, policy_version 514659 (0.0007) [2023-12-27 09:12:12,239][20292] KL-divergence is very high: 123.2788 [2023-12-27 09:12:12,286][20381] Updated weights for policy 1, policy_version 514669 (0.0006) [2023-12-27 09:12:12,294][20347] Updated weights for policy 0, policy_version 514859 (0.0008) [2023-12-27 09:12:12,349][20347] Updated weights for policy 0, policy_version 514869 (0.0008) [2023-12-27 09:12:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 263593984. Throughput: 0: 10622.9, 1: 10290.1. Samples: 263585052. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:12,363][19170] Avg episode reward: [(0, '190.570'), (1, '883.280')] [2023-12-27 09:12:12,404][20347] Updated weights for policy 0, policy_version 514879 (0.0007) [2023-12-27 09:12:12,998][20381] Updated weights for policy 1, policy_version 514679 (0.0009) [2023-12-27 09:12:13,057][20381] Updated weights for policy 1, policy_version 514689 (0.0007) [2023-12-27 09:12:13,085][20347] Updated weights for policy 0, policy_version 514889 (0.0006) [2023-12-27 09:12:13,107][20381] Updated weights for policy 1, policy_version 514699 (0.0006) [2023-12-27 09:12:13,142][20347] Updated weights for policy 0, policy_version 514899 (0.0006) [2023-12-27 09:12:13,198][20347] Updated weights for policy 0, policy_version 514909 (0.0008) [2023-12-27 09:12:13,837][20381] Updated weights for policy 1, policy_version 514709 (0.0006) [2023-12-27 09:12:13,850][20347] Updated weights for policy 0, policy_version 514919 (0.0007) [2023-12-27 09:12:13,893][20381] Updated weights for policy 1, policy_version 514719 (0.0005) [2023-12-27 09:12:13,909][20347] Updated weights for policy 0, policy_version 514929 (0.0007) [2023-12-27 09:12:13,948][20381] Updated weights for policy 1, policy_version 514729 (0.0006) [2023-12-27 09:12:13,961][20347] Updated weights for policy 0, policy_version 514939 (0.0005) [2023-12-27 09:12:14,602][20347] Updated weights for policy 0, policy_version 514949 (0.0005) [2023-12-27 09:12:14,654][20381] Updated weights for policy 1, policy_version 514739 (0.0007) [2023-12-27 09:12:14,657][20347] Updated weights for policy 0, policy_version 514959 (0.0006) [2023-12-27 09:12:14,707][20347] Updated weights for policy 0, policy_version 514969 (0.0009) [2023-12-27 09:12:14,709][20381] Updated weights for policy 1, policy_version 514749 (0.0008) [2023-12-27 09:12:14,767][20381] Updated weights for policy 1, policy_version 514759 (0.0006) [2023-12-27 09:12:15,362][20347] Updated weights for policy 0, policy_version 514979 (0.0006) [2023-12-27 09:12:15,411][20347] Updated weights for policy 0, policy_version 514989 (0.0008) [2023-12-27 09:12:15,452][20381] Updated weights for policy 1, policy_version 514769 (0.0008) [2023-12-27 09:12:15,462][20347] Updated weights for policy 0, policy_version 514999 (0.0007) [2023-12-27 09:12:15,508][20381] Updated weights for policy 1, policy_version 514779 (0.0007) [2023-12-27 09:12:15,565][20381] Updated weights for policy 1, policy_version 514789 (0.0007) [2023-12-27 09:12:15,622][20381] Updated weights for policy 1, policy_version 514799 (0.0007) [2023-12-27 09:12:16,123][20347] Updated weights for policy 0, policy_version 515009 (0.0006) [2023-12-27 09:12:16,184][20347] Updated weights for policy 0, policy_version 515019 (0.0007) [2023-12-27 09:12:16,246][20347] Updated weights for policy 0, policy_version 515029 (0.0009) [2023-12-27 09:12:16,296][20381] Updated weights for policy 1, policy_version 514809 (0.0006) [2023-12-27 09:12:16,302][20347] Updated weights for policy 0, policy_version 515039 (0.0011) [2023-12-27 09:12:16,348][20381] Updated weights for policy 1, policy_version 514819 (0.0007) [2023-12-27 09:12:16,401][20381] Updated weights for policy 1, policy_version 514829 (0.0007) [2023-12-27 09:12:16,926][20347] Updated weights for policy 0, policy_version 515049 (0.0010) [2023-12-27 09:12:16,978][20347] Updated weights for policy 0, policy_version 515059 (0.0010) [2023-12-27 09:12:17,030][20347] Updated weights for policy 0, policy_version 515069 (0.0010) [2023-12-27 09:12:17,102][20381] Updated weights for policy 1, policy_version 514839 (0.0008) [2023-12-27 09:12:17,150][20381] Updated weights for policy 1, policy_version 514849 (0.0007) [2023-12-27 09:12:17,207][20381] Updated weights for policy 1, policy_version 514859 (0.0007) [2023-12-27 09:12:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 263700480. Throughput: 0: 10550.9, 1: 10192.0. Samples: 263706676. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:17,363][19170] Avg episode reward: [(0, '186.740'), (1, '851.210')] [2023-12-27 09:12:17,707][20347] Updated weights for policy 0, policy_version 515079 (0.0007) [2023-12-27 09:12:17,760][20347] Updated weights for policy 0, policy_version 515089 (0.0010) [2023-12-27 09:12:17,812][20347] Updated weights for policy 0, policy_version 515099 (0.0010) [2023-12-27 09:12:17,902][20381] Updated weights for policy 1, policy_version 514869 (0.0007) [2023-12-27 09:12:17,954][20381] Updated weights for policy 1, policy_version 514879 (0.0008) [2023-12-27 09:12:18,005][20381] Updated weights for policy 1, policy_version 514889 (0.0007) [2023-12-27 09:12:18,452][20347] Updated weights for policy 0, policy_version 515109 (0.0009) [2023-12-27 09:12:18,507][20347] Updated weights for policy 0, policy_version 515119 (0.0008) [2023-12-27 09:12:18,565][20347] Updated weights for policy 0, policy_version 515129 (0.0008) [2023-12-27 09:12:18,695][20381] Updated weights for policy 1, policy_version 514899 (0.0008) [2023-12-27 09:12:18,754][20381] Updated weights for policy 1, policy_version 514909 (0.0007) [2023-12-27 09:12:18,815][20381] Updated weights for policy 1, policy_version 514919 (0.0008) [2023-12-27 09:12:19,297][20347] Updated weights for policy 0, policy_version 515139 (0.0008) [2023-12-27 09:12:19,353][20347] Updated weights for policy 0, policy_version 515149 (0.0007) [2023-12-27 09:12:19,406][20347] Updated weights for policy 0, policy_version 515159 (0.0007) [2023-12-27 09:12:19,525][20381] Updated weights for policy 1, policy_version 514929 (0.0007) [2023-12-27 09:12:19,586][20381] Updated weights for policy 1, policy_version 514939 (0.0009) [2023-12-27 09:12:19,636][20381] Updated weights for policy 1, policy_version 514949 (0.0008) [2023-12-27 09:12:19,694][20381] Updated weights for policy 1, policy_version 514959 (0.0008) [2023-12-27 09:12:20,079][20347] Updated weights for policy 0, policy_version 515169 (0.0005) [2023-12-27 09:12:20,142][20347] Updated weights for policy 0, policy_version 515179 (0.0007) [2023-12-27 09:12:20,201][20347] Updated weights for policy 0, policy_version 515189 (0.0007) [2023-12-27 09:12:20,260][20347] Updated weights for policy 0, policy_version 515199 (0.0008) [2023-12-27 09:12:20,362][20381] Updated weights for policy 1, policy_version 514969 (0.0007) [2023-12-27 09:12:20,426][20381] Updated weights for policy 1, policy_version 514979 (0.0008) [2023-12-27 09:12:20,485][20381] Updated weights for policy 1, policy_version 514989 (0.0007) [2023-12-27 09:12:20,964][20347] Updated weights for policy 0, policy_version 515209 (0.0007) [2023-12-27 09:12:21,026][20347] Updated weights for policy 0, policy_version 515219 (0.0008) [2023-12-27 09:12:21,092][20347] Updated weights for policy 0, policy_version 515229 (0.0008) [2023-12-27 09:12:21,158][20381] Updated weights for policy 1, policy_version 514999 (0.0008) [2023-12-27 09:12:21,222][20381] Updated weights for policy 1, policy_version 515009 (0.0008) [2023-12-27 09:12:21,289][20381] Updated weights for policy 1, policy_version 515019 (0.0008) [2023-12-27 09:12:21,807][20347] Updated weights for policy 0, policy_version 515239 (0.0006) [2023-12-27 09:12:21,862][20347] Updated weights for policy 0, policy_version 515249 (0.0005) [2023-12-27 09:12:21,922][20347] Updated weights for policy 0, policy_version 515259 (0.0006) [2023-12-27 09:12:22,010][20381] Updated weights for policy 1, policy_version 515029 (0.0007) [2023-12-27 09:12:22,070][20381] Updated weights for policy 1, policy_version 515039 (0.0008) [2023-12-27 09:12:22,129][20381] Updated weights for policy 1, policy_version 515049 (0.0008) [2023-12-27 09:12:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 263798784. Throughput: 0: 10524.1, 1: 10146.0. Samples: 263767984. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:22,363][19170] Avg episode reward: [(0, '198.760'), (1, '879.830')] [2023-12-27 09:12:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000515056_131874816.pth... [2023-12-27 09:12:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000515264_131923968.pth... [2023-12-27 09:12:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000513872_131571712.pth [2023-12-27 09:12:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000514048_131612672.pth [2023-12-27 09:12:22,554][20347] Updated weights for policy 0, policy_version 515269 (0.0006) [2023-12-27 09:12:22,609][20347] Updated weights for policy 0, policy_version 515279 (0.0008) [2023-12-27 09:12:22,674][20347] Updated weights for policy 0, policy_version 515289 (0.0007) [2023-12-27 09:12:22,909][20381] Updated weights for policy 1, policy_version 515059 (0.0008) [2023-12-27 09:12:22,964][20381] Updated weights for policy 1, policy_version 515069 (0.0008) [2023-12-27 09:12:23,026][20381] Updated weights for policy 1, policy_version 515079 (0.0008) [2023-12-27 09:12:23,317][20347] Updated weights for policy 0, policy_version 515299 (0.0007) [2023-12-27 09:12:23,372][20347] Updated weights for policy 0, policy_version 515309 (0.0008) [2023-12-27 09:12:23,432][20347] Updated weights for policy 0, policy_version 515319 (0.0008) [2023-12-27 09:12:23,716][20381] Updated weights for policy 1, policy_version 515089 (0.0008) [2023-12-27 09:12:23,765][20381] Updated weights for policy 1, policy_version 515099 (0.0007) [2023-12-27 09:12:23,819][20381] Updated weights for policy 1, policy_version 515109 (0.0007) [2023-12-27 09:12:23,874][20381] Updated weights for policy 1, policy_version 515119 (0.0009) [2023-12-27 09:12:24,042][20347] Updated weights for policy 0, policy_version 515329 (0.0007) [2023-12-27 09:12:24,090][20347] Updated weights for policy 0, policy_version 515339 (0.0007) [2023-12-27 09:12:24,135][20347] Updated weights for policy 0, policy_version 515349 (0.0008) [2023-12-27 09:12:24,182][20347] Updated weights for policy 0, policy_version 515359 (0.0007) [2023-12-27 09:12:24,573][20381] Updated weights for policy 1, policy_version 515129 (0.0007) [2023-12-27 09:12:24,624][20381] Updated weights for policy 1, policy_version 515139 (0.0008) [2023-12-27 09:12:24,676][20381] Updated weights for policy 1, policy_version 515149 (0.0008) [2023-12-27 09:12:24,853][20347] Updated weights for policy 0, policy_version 515369 (0.0008) [2023-12-27 09:12:24,906][20347] Updated weights for policy 0, policy_version 515379 (0.0007) [2023-12-27 09:12:24,951][20347] Updated weights for policy 0, policy_version 515389 (0.0007) [2023-12-27 09:12:25,381][20381] Updated weights for policy 1, policy_version 515159 (0.0007) [2023-12-27 09:12:25,442][20381] Updated weights for policy 1, policy_version 515169 (0.0008) [2023-12-27 09:12:25,504][20381] Updated weights for policy 1, policy_version 515179 (0.0010) [2023-12-27 09:12:25,599][20347] Updated weights for policy 0, policy_version 515399 (0.0008) [2023-12-27 09:12:25,656][20347] Updated weights for policy 0, policy_version 515409 (0.0008) [2023-12-27 09:12:25,711][20347] Updated weights for policy 0, policy_version 515419 (0.0007) [2023-12-27 09:12:26,173][20381] Updated weights for policy 1, policy_version 515189 (0.0010) [2023-12-27 09:12:26,230][20381] Updated weights for policy 1, policy_version 515199 (0.0011) [2023-12-27 09:12:26,290][20381] Updated weights for policy 1, policy_version 515209 (0.0011) [2023-12-27 09:12:26,420][20347] Updated weights for policy 0, policy_version 515429 (0.0007) [2023-12-27 09:12:26,470][20347] Updated weights for policy 0, policy_version 515439 (0.0007) [2023-12-27 09:12:26,515][20347] Updated weights for policy 0, policy_version 515449 (0.0008) [2023-12-27 09:12:26,899][20381] Updated weights for policy 1, policy_version 515219 (0.0010) [2023-12-27 09:12:26,957][20381] Updated weights for policy 1, policy_version 515229 (0.0010) [2023-12-27 09:12:27,018][20381] Updated weights for policy 1, policy_version 515239 (0.0010) [2023-12-27 09:12:27,150][20347] Updated weights for policy 0, policy_version 515459 (0.0009) [2023-12-27 09:12:27,200][20347] Updated weights for policy 0, policy_version 515469 (0.0006) [2023-12-27 09:12:27,251][20347] Updated weights for policy 0, policy_version 515479 (0.0008) [2023-12-27 09:12:27,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 263905280. Throughput: 0: 10427.7, 1: 10157.5. Samples: 263889228. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:27,364][19170] Avg episode reward: [(0, '169.480'), (1, '883.870')] [2023-12-27 09:12:27,671][20381] Updated weights for policy 1, policy_version 515249 (0.0007) [2023-12-27 09:12:27,719][20381] Updated weights for policy 1, policy_version 515259 (0.0006) [2023-12-27 09:12:27,776][20381] Updated weights for policy 1, policy_version 515269 (0.0005) [2023-12-27 09:12:27,836][20381] Updated weights for policy 1, policy_version 515279 (0.0005) [2023-12-27 09:12:27,941][20347] Updated weights for policy 0, policy_version 515489 (0.0008) [2023-12-27 09:12:28,003][20347] Updated weights for policy 0, policy_version 515499 (0.0005) [2023-12-27 09:12:28,047][20347] Updated weights for policy 0, policy_version 515509 (0.0008) [2023-12-27 09:12:28,094][20347] Updated weights for policy 0, policy_version 515519 (0.0007) [2023-12-27 09:12:28,454][20381] Updated weights for policy 1, policy_version 515289 (0.0005) [2023-12-27 09:12:28,523][20381] Updated weights for policy 1, policy_version 515299 (0.0005) [2023-12-27 09:12:28,577][20381] Updated weights for policy 1, policy_version 515309 (0.0007) [2023-12-27 09:12:28,828][20347] Updated weights for policy 0, policy_version 515529 (0.0005) [2023-12-27 09:12:28,889][20347] Updated weights for policy 0, policy_version 515539 (0.0008) [2023-12-27 09:12:28,938][20347] Updated weights for policy 0, policy_version 515549 (0.0009) [2023-12-27 09:12:29,103][20381] Updated weights for policy 1, policy_version 515319 (0.0005) [2023-12-27 09:12:29,152][20381] Updated weights for policy 1, policy_version 515329 (0.0006) [2023-12-27 09:12:29,203][20381] Updated weights for policy 1, policy_version 515339 (0.0005) [2023-12-27 09:12:29,646][20347] Updated weights for policy 0, policy_version 515559 (0.0008) [2023-12-27 09:12:29,703][20347] Updated weights for policy 0, policy_version 515569 (0.0007) [2023-12-27 09:12:29,759][20347] Updated weights for policy 0, policy_version 515579 (0.0008) [2023-12-27 09:12:29,815][20381] Updated weights for policy 1, policy_version 515349 (0.0007) [2023-12-27 09:12:29,877][20381] Updated weights for policy 1, policy_version 515359 (0.0007) [2023-12-27 09:12:29,939][20381] Updated weights for policy 1, policy_version 515369 (0.0008) [2023-12-27 09:12:30,431][20347] Updated weights for policy 0, policy_version 515589 (0.0006) [2023-12-27 09:12:30,483][20347] Updated weights for policy 0, policy_version 515599 (0.0008) [2023-12-27 09:12:30,527][20347] Updated weights for policy 0, policy_version 515609 (0.0007) [2023-12-27 09:12:30,602][20381] Updated weights for policy 1, policy_version 515379 (0.0008) [2023-12-27 09:12:30,657][20381] Updated weights for policy 1, policy_version 515389 (0.0008) [2023-12-27 09:12:30,706][20381] Updated weights for policy 1, policy_version 515399 (0.0008) [2023-12-27 09:12:31,260][20347] Updated weights for policy 0, policy_version 515619 (0.0007) [2023-12-27 09:12:31,317][20347] Updated weights for policy 0, policy_version 515629 (0.0008) [2023-12-27 09:12:31,337][20381] Updated weights for policy 1, policy_version 515409 (0.0008) [2023-12-27 09:12:31,384][20347] Updated weights for policy 0, policy_version 515639 (0.0007) [2023-12-27 09:12:31,397][20381] Updated weights for policy 1, policy_version 515419 (0.0009) [2023-12-27 09:12:31,452][20381] Updated weights for policy 1, policy_version 515429 (0.0007) [2023-12-27 09:12:31,509][20381] Updated weights for policy 1, policy_version 515439 (0.0007) [2023-12-27 09:12:32,110][20347] Updated weights for policy 0, policy_version 515649 (0.0007) [2023-12-27 09:12:32,155][20347] Updated weights for policy 0, policy_version 515659 (0.0008) [2023-12-27 09:12:32,206][20347] Updated weights for policy 0, policy_version 515669 (0.0008) [2023-12-27 09:12:32,234][20381] Updated weights for policy 1, policy_version 515449 (0.0006) [2023-12-27 09:12:32,263][20347] Updated weights for policy 0, policy_version 515679 (0.0006) [2023-12-27 09:12:32,295][20381] Updated weights for policy 1, policy_version 515459 (0.0008) [2023-12-27 09:12:32,360][20381] Updated weights for policy 1, policy_version 515469 (0.0006) [2023-12-27 09:12:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 264003584. Throughput: 0: 10353.8, 1: 10267.7. Samples: 264014984. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:32,363][19170] Avg episode reward: [(0, '211.610'), (1, '835.200')] [2023-12-27 09:12:32,888][20347] Updated weights for policy 0, policy_version 515689 (0.0007) [2023-12-27 09:12:32,942][20347] Updated weights for policy 0, policy_version 515699 (0.0007) [2023-12-27 09:12:32,994][20347] Updated weights for policy 0, policy_version 515709 (0.0006) [2023-12-27 09:12:32,995][20381] Updated weights for policy 1, policy_version 515479 (0.0006) [2023-12-27 09:12:33,049][20381] Updated weights for policy 1, policy_version 515489 (0.0007) [2023-12-27 09:12:33,105][20381] Updated weights for policy 1, policy_version 515499 (0.0005) [2023-12-27 09:12:33,558][20347] Updated weights for policy 0, policy_version 515719 (0.0007) [2023-12-27 09:12:33,602][20347] Updated weights for policy 0, policy_version 515729 (0.0007) [2023-12-27 09:12:33,646][20347] Updated weights for policy 0, policy_version 515739 (0.0007) [2023-12-27 09:12:33,750][20381] Updated weights for policy 1, policy_version 515509 (0.0007) [2023-12-27 09:12:33,806][20381] Updated weights for policy 1, policy_version 515519 (0.0006) [2023-12-27 09:12:33,859][20381] Updated weights for policy 1, policy_version 515529 (0.0005) [2023-12-27 09:12:34,285][20347] Updated weights for policy 0, policy_version 515749 (0.0007) [2023-12-27 09:12:34,343][20347] Updated weights for policy 0, policy_version 515759 (0.0008) [2023-12-27 09:12:34,403][20347] Updated weights for policy 0, policy_version 515769 (0.0007) [2023-12-27 09:12:34,541][20381] Updated weights for policy 1, policy_version 515539 (0.0006) [2023-12-27 09:12:34,601][20381] Updated weights for policy 1, policy_version 515549 (0.0008) [2023-12-27 09:12:34,655][20381] Updated weights for policy 1, policy_version 515559 (0.0008) [2023-12-27 09:12:35,098][20347] Updated weights for policy 0, policy_version 515779 (0.0008) [2023-12-27 09:12:35,148][20347] Updated weights for policy 0, policy_version 515789 (0.0007) [2023-12-27 09:12:35,201][20347] Updated weights for policy 0, policy_version 515799 (0.0006) [2023-12-27 09:12:35,343][20381] Updated weights for policy 1, policy_version 515569 (0.0008) [2023-12-27 09:12:35,399][20381] Updated weights for policy 1, policy_version 515579 (0.0011) [2023-12-27 09:12:35,462][20381] Updated weights for policy 1, policy_version 515589 (0.0011) [2023-12-27 09:12:35,520][20381] Updated weights for policy 1, policy_version 515599 (0.0011) [2023-12-27 09:12:35,830][20347] Updated weights for policy 0, policy_version 515809 (0.0008) [2023-12-27 09:12:35,885][20347] Updated weights for policy 0, policy_version 515819 (0.0008) [2023-12-27 09:12:35,943][20347] Updated weights for policy 0, policy_version 515829 (0.0007) [2023-12-27 09:12:35,999][20347] Updated weights for policy 0, policy_version 515839 (0.0007) [2023-12-27 09:12:36,198][20381] Updated weights for policy 1, policy_version 515609 (0.0006) [2023-12-27 09:12:36,261][20381] Updated weights for policy 1, policy_version 515619 (0.0006) [2023-12-27 09:12:36,323][20381] Updated weights for policy 1, policy_version 515629 (0.0011) [2023-12-27 09:12:36,682][20347] Updated weights for policy 0, policy_version 515849 (0.0007) [2023-12-27 09:12:36,731][20347] Updated weights for policy 0, policy_version 515859 (0.0008) [2023-12-27 09:12:36,780][20347] Updated weights for policy 0, policy_version 515869 (0.0008) [2023-12-27 09:12:36,938][20381] Updated weights for policy 1, policy_version 515639 (0.0007) [2023-12-27 09:12:36,992][20381] Updated weights for policy 1, policy_version 515649 (0.0005) [2023-12-27 09:12:37,046][20381] Updated weights for policy 1, policy_version 515659 (0.0006) [2023-12-27 09:12:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 264110080. Throughput: 0: 10398.6, 1: 10290.8. Samples: 264079396. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:37,363][19170] Avg episode reward: [(0, '194.140'), (1, '845.280')] [2023-12-27 09:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000515664_132030464.pth... [2023-12-27 09:12:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000515872_132079616.pth... [2023-12-27 09:12:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000514688_131776512.pth [2023-12-27 09:12:37,392][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000514448_131719168.pth [2023-12-27 09:12:37,503][20347] Updated weights for policy 0, policy_version 515879 (0.0007) [2023-12-27 09:12:37,549][20347] Updated weights for policy 0, policy_version 515889 (0.0008) [2023-12-27 09:12:37,605][20347] Updated weights for policy 0, policy_version 515899 (0.0009) [2023-12-27 09:12:37,666][20381] Updated weights for policy 1, policy_version 515669 (0.0008) [2023-12-27 09:12:37,719][20381] Updated weights for policy 1, policy_version 515679 (0.0005) [2023-12-27 09:12:37,772][20381] Updated weights for policy 1, policy_version 515689 (0.0008) [2023-12-27 09:12:38,342][20347] Updated weights for policy 0, policy_version 515909 (0.0009) [2023-12-27 09:12:38,387][20381] Updated weights for policy 1, policy_version 515699 (0.0008) [2023-12-27 09:12:38,396][20347] Updated weights for policy 0, policy_version 515919 (0.0007) [2023-12-27 09:12:38,443][20381] Updated weights for policy 1, policy_version 515709 (0.0006) [2023-12-27 09:12:38,449][20347] Updated weights for policy 0, policy_version 515929 (0.0006) [2023-12-27 09:12:38,496][20381] Updated weights for policy 1, policy_version 515719 (0.0006) [2023-12-27 09:12:39,165][20381] Updated weights for policy 1, policy_version 515729 (0.0007) [2023-12-27 09:12:39,169][20347] Updated weights for policy 0, policy_version 515939 (0.0007) [2023-12-27 09:12:39,220][20347] Updated weights for policy 0, policy_version 515949 (0.0007) [2023-12-27 09:12:39,231][20381] Updated weights for policy 1, policy_version 515739 (0.0006) [2023-12-27 09:12:39,282][20347] Updated weights for policy 0, policy_version 515959 (0.0007) [2023-12-27 09:12:39,292][20381] Updated weights for policy 1, policy_version 515749 (0.0006) [2023-12-27 09:12:39,356][20381] Updated weights for policy 1, policy_version 515759 (0.0008) [2023-12-27 09:12:40,031][20347] Updated weights for policy 0, policy_version 515969 (0.0008) [2023-12-27 09:12:40,045][20381] Updated weights for policy 1, policy_version 515769 (0.0007) [2023-12-27 09:12:40,085][20347] Updated weights for policy 0, policy_version 515979 (0.0006) [2023-12-27 09:12:40,103][20381] Updated weights for policy 1, policy_version 515779 (0.0006) [2023-12-27 09:12:40,146][20347] Updated weights for policy 0, policy_version 515989 (0.0006) [2023-12-27 09:12:40,164][20381] Updated weights for policy 1, policy_version 515789 (0.0007) [2023-12-27 09:12:40,204][20347] Updated weights for policy 0, policy_version 515999 (0.0008) [2023-12-27 09:12:40,795][20347] Updated weights for policy 0, policy_version 516009 (0.0007) [2023-12-27 09:12:40,850][20347] Updated weights for policy 0, policy_version 516019 (0.0006) [2023-12-27 09:12:40,888][20381] Updated weights for policy 1, policy_version 515799 (0.0007) [2023-12-27 09:12:40,911][20347] Updated weights for policy 0, policy_version 516029 (0.0006) [2023-12-27 09:12:40,936][20381] Updated weights for policy 1, policy_version 515809 (0.0006) [2023-12-27 09:12:40,988][20381] Updated weights for policy 1, policy_version 515819 (0.0007) [2023-12-27 09:12:41,632][20347] Updated weights for policy 0, policy_version 516039 (0.0007) [2023-12-27 09:12:41,692][20347] Updated weights for policy 0, policy_version 516049 (0.0008) [2023-12-27 09:12:41,756][20347] Updated weights for policy 0, policy_version 516059 (0.0008) [2023-12-27 09:12:41,763][20381] Updated weights for policy 1, policy_version 515829 (0.0007) [2023-12-27 09:12:41,814][20381] Updated weights for policy 1, policy_version 515839 (0.0006) [2023-12-27 09:12:41,877][20381] Updated weights for policy 1, policy_version 515849 (0.0007) [2023-12-27 09:12:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 264208384. Throughput: 0: 10313.1, 1: 10291.6. Samples: 264201532. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:42,363][19170] Avg episode reward: [(0, '155.510'), (1, '866.480')] [2023-12-27 09:12:42,538][20347] Updated weights for policy 0, policy_version 516069 (0.0008) [2023-12-27 09:12:42,567][20381] Updated weights for policy 1, policy_version 515859 (0.0007) [2023-12-27 09:12:42,596][20347] Updated weights for policy 0, policy_version 516079 (0.0006) [2023-12-27 09:12:42,626][20381] Updated weights for policy 1, policy_version 515869 (0.0008) [2023-12-27 09:12:42,651][20347] Updated weights for policy 0, policy_version 516089 (0.0007) [2023-12-27 09:12:42,679][20381] Updated weights for policy 1, policy_version 515879 (0.0005) [2023-12-27 09:12:43,276][20347] Updated weights for policy 0, policy_version 516099 (0.0006) [2023-12-27 09:12:43,308][20381] Updated weights for policy 1, policy_version 515889 (0.0007) [2023-12-27 09:12:43,332][20347] Updated weights for policy 0, policy_version 516109 (0.0006) [2023-12-27 09:12:43,356][20381] Updated weights for policy 1, policy_version 515899 (0.0008) [2023-12-27 09:12:43,386][20347] Updated weights for policy 0, policy_version 516119 (0.0006) [2023-12-27 09:12:43,407][20381] Updated weights for policy 1, policy_version 515909 (0.0008) [2023-12-27 09:12:43,448][20381] Updated weights for policy 1, policy_version 515919 (0.0006) [2023-12-27 09:12:43,982][20347] Updated weights for policy 0, policy_version 516129 (0.0007) [2023-12-27 09:12:44,035][20347] Updated weights for policy 0, policy_version 516139 (0.0005) [2023-12-27 09:12:44,093][20347] Updated weights for policy 0, policy_version 516149 (0.0005) [2023-12-27 09:12:44,104][20381] Updated weights for policy 1, policy_version 515929 (0.0007) [2023-12-27 09:12:44,154][20347] Updated weights for policy 0, policy_version 516159 (0.0006) [2023-12-27 09:12:44,164][20381] Updated weights for policy 1, policy_version 515939 (0.0006) [2023-12-27 09:12:44,219][20381] Updated weights for policy 1, policy_version 515949 (0.0007) [2023-12-27 09:12:44,706][20347] Updated weights for policy 0, policy_version 516169 (0.0007) [2023-12-27 09:12:44,768][20347] Updated weights for policy 0, policy_version 516179 (0.0009) [2023-12-27 09:12:44,832][20347] Updated weights for policy 0, policy_version 516189 (0.0008) [2023-12-27 09:12:44,950][20381] Updated weights for policy 1, policy_version 515959 (0.0008) [2023-12-27 09:12:45,015][20381] Updated weights for policy 1, policy_version 515969 (0.0008) [2023-12-27 09:12:45,083][20381] Updated weights for policy 1, policy_version 515979 (0.0007) [2023-12-27 09:12:45,418][20347] Updated weights for policy 0, policy_version 516199 (0.0009) [2023-12-27 09:12:45,469][20347] Updated weights for policy 0, policy_version 516209 (0.0010) [2023-12-27 09:12:45,524][20347] Updated weights for policy 0, policy_version 516219 (0.0010) [2023-12-27 09:12:45,769][20381] Updated weights for policy 1, policy_version 515989 (0.0008) [2023-12-27 09:12:45,823][20381] Updated weights for policy 1, policy_version 515999 (0.0007) [2023-12-27 09:12:45,880][20381] Updated weights for policy 1, policy_version 516009 (0.0008) [2023-12-27 09:12:46,168][20347] Updated weights for policy 0, policy_version 516229 (0.0009) [2023-12-27 09:12:46,229][20347] Updated weights for policy 0, policy_version 516239 (0.0007) [2023-12-27 09:12:46,279][20347] Updated weights for policy 0, policy_version 516249 (0.0007) [2023-12-27 09:12:46,597][20381] Updated weights for policy 1, policy_version 516019 (0.0009) [2023-12-27 09:12:46,658][20381] Updated weights for policy 1, policy_version 516029 (0.0010) [2023-12-27 09:12:46,719][20381] Updated weights for policy 1, policy_version 516039 (0.0010) [2023-12-27 09:12:46,878][20347] Updated weights for policy 0, policy_version 516259 (0.0007) [2023-12-27 09:12:46,928][20347] Updated weights for policy 0, policy_version 516269 (0.0008) [2023-12-27 09:12:46,988][20347] Updated weights for policy 0, policy_version 516279 (0.0008) [2023-12-27 09:12:47,342][20381] Updated weights for policy 1, policy_version 516049 (0.0010) [2023-12-27 09:12:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 264314880. Throughput: 0: 10379.9, 1: 10300.6. Samples: 264326744. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 09:12:47,363][19170] Avg episode reward: [(0, '181.010'), (1, '828.200')] [2023-12-27 09:12:47,390][20381] Updated weights for policy 1, policy_version 516059 (0.0008) [2023-12-27 09:12:47,443][20381] Updated weights for policy 1, policy_version 516069 (0.0007) [2023-12-27 09:12:47,493][20381] Updated weights for policy 1, policy_version 516079 (0.0007) [2023-12-27 09:12:47,690][20347] Updated weights for policy 0, policy_version 516289 (0.0008) [2023-12-27 09:12:47,743][20347] Updated weights for policy 0, policy_version 516299 (0.0005) [2023-12-27 09:12:47,799][20347] Updated weights for policy 0, policy_version 516309 (0.0005) [2023-12-27 09:12:47,847][20347] Updated weights for policy 0, policy_version 516319 (0.0005) [2023-12-27 09:12:48,194][20381] Updated weights for policy 1, policy_version 516089 (0.0007) [2023-12-27 09:12:48,245][20381] Updated weights for policy 1, policy_version 516099 (0.0008) [2023-12-27 09:12:48,290][20381] Updated weights for policy 1, policy_version 516109 (0.0008) [2023-12-27 09:12:48,385][20347] Updated weights for policy 0, policy_version 516329 (0.0007) [2023-12-27 09:12:48,450][20347] Updated weights for policy 0, policy_version 516339 (0.0007) [2023-12-27 09:12:48,512][20347] Updated weights for policy 0, policy_version 516349 (0.0008) [2023-12-27 09:12:48,880][20381] Updated weights for policy 1, policy_version 516119 (0.0007) [2023-12-27 09:12:48,939][20381] Updated weights for policy 1, policy_version 516129 (0.0005) [2023-12-27 09:12:49,001][20381] Updated weights for policy 1, policy_version 516139 (0.0007) [2023-12-27 09:12:49,156][20347] Updated weights for policy 0, policy_version 516359 (0.0007) [2023-12-27 09:12:49,216][20347] Updated weights for policy 0, policy_version 516369 (0.0008) [2023-12-27 09:12:49,275][20347] Updated weights for policy 0, policy_version 516379 (0.0008) [2023-12-27 09:12:49,693][20381] Updated weights for policy 1, policy_version 516149 (0.0008) [2023-12-27 09:12:49,750][20381] Updated weights for policy 1, policy_version 516159 (0.0008) [2023-12-27 09:12:49,808][20381] Updated weights for policy 1, policy_version 516169 (0.0006) [2023-12-27 09:12:49,963][20347] Updated weights for policy 0, policy_version 516389 (0.0007) [2023-12-27 09:12:50,012][20347] Updated weights for policy 0, policy_version 516399 (0.0007) [2023-12-27 09:12:50,066][20347] Updated weights for policy 0, policy_version 516409 (0.0008) [2023-12-27 09:12:50,505][20381] Updated weights for policy 1, policy_version 516179 (0.0010) [2023-12-27 09:12:50,555][20381] Updated weights for policy 1, policy_version 516189 (0.0010) [2023-12-27 09:12:50,619][20381] Updated weights for policy 1, policy_version 516199 (0.0011) [2023-12-27 09:12:50,776][20347] Updated weights for policy 0, policy_version 516419 (0.0007) [2023-12-27 09:12:50,826][20347] Updated weights for policy 0, policy_version 516429 (0.0007) [2023-12-27 09:12:50,883][20347] Updated weights for policy 0, policy_version 516439 (0.0008) [2023-12-27 09:12:51,261][20381] Updated weights for policy 1, policy_version 516209 (0.0009) [2023-12-27 09:12:51,324][20381] Updated weights for policy 1, policy_version 516219 (0.0008) [2023-12-27 09:12:51,390][20381] Updated weights for policy 1, policy_version 516229 (0.0008) [2023-12-27 09:12:51,453][20381] Updated weights for policy 1, policy_version 516239 (0.0007) [2023-12-27 09:12:51,614][20347] Updated weights for policy 0, policy_version 516449 (0.0007) [2023-12-27 09:12:51,684][20347] Updated weights for policy 0, policy_version 516459 (0.0008) [2023-12-27 09:12:51,748][20347] Updated weights for policy 0, policy_version 516469 (0.0008) [2023-12-27 09:12:51,796][20347] Updated weights for policy 0, policy_version 516479 (0.0007) [2023-12-27 09:12:52,091][20381] Updated weights for policy 1, policy_version 516249 (0.0007) [2023-12-27 09:12:52,148][20381] Updated weights for policy 1, policy_version 516259 (0.0007) [2023-12-27 09:12:52,204][20381] Updated weights for policy 1, policy_version 516269 (0.0005) [2023-12-27 09:12:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 264421376. Throughput: 0: 10363.2, 1: 10312.2. Samples: 264389980. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:12:52,363][19170] Avg episode reward: [(0, '214.370'), (1, '777.930')] [2023-12-27 09:12:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000516272_132186112.pth... [2023-12-27 09:12:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000516480_132235264.pth... [2023-12-27 09:12:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000515056_131874816.pth [2023-12-27 09:12:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000515264_131923968.pth [2023-12-27 09:12:52,526][20347] Updated weights for policy 0, policy_version 516489 (0.0008) [2023-12-27 09:12:52,589][20347] Updated weights for policy 0, policy_version 516499 (0.0009) [2023-12-27 09:12:52,646][20347] Updated weights for policy 0, policy_version 516509 (0.0008) [2023-12-27 09:12:52,856][20381] Updated weights for policy 1, policy_version 516279 (0.0007) [2023-12-27 09:12:52,915][20381] Updated weights for policy 1, policy_version 516289 (0.0008) [2023-12-27 09:12:52,972][20381] Updated weights for policy 1, policy_version 516299 (0.0007) [2023-12-27 09:12:53,345][20347] Updated weights for policy 0, policy_version 516519 (0.0008) [2023-12-27 09:12:53,399][20347] Updated weights for policy 0, policy_version 516529 (0.0007) [2023-12-27 09:12:53,456][20347] Updated weights for policy 0, policy_version 516539 (0.0006) [2023-12-27 09:12:53,647][20381] Updated weights for policy 1, policy_version 516309 (0.0007) [2023-12-27 09:12:53,700][20381] Updated weights for policy 1, policy_version 516319 (0.0007) [2023-12-27 09:12:53,749][20381] Updated weights for policy 1, policy_version 516329 (0.0008) [2023-12-27 09:12:54,142][20347] Updated weights for policy 0, policy_version 516549 (0.0007) [2023-12-27 09:12:54,197][20347] Updated weights for policy 0, policy_version 516559 (0.0008) [2023-12-27 09:12:54,254][20347] Updated weights for policy 0, policy_version 516569 (0.0008) [2023-12-27 09:12:54,472][20381] Updated weights for policy 1, policy_version 516339 (0.0007) [2023-12-27 09:12:54,534][20381] Updated weights for policy 1, policy_version 516349 (0.0008) [2023-12-27 09:12:54,591][20381] Updated weights for policy 1, policy_version 516359 (0.0008) [2023-12-27 09:12:54,953][20347] Updated weights for policy 0, policy_version 516579 (0.0007) [2023-12-27 09:12:55,014][20347] Updated weights for policy 0, policy_version 516589 (0.0006) [2023-12-27 09:12:55,071][20347] Updated weights for policy 0, policy_version 516599 (0.0005) [2023-12-27 09:12:55,313][20381] Updated weights for policy 1, policy_version 516369 (0.0008) [2023-12-27 09:12:55,365][20381] Updated weights for policy 1, policy_version 516379 (0.0008) [2023-12-27 09:12:55,422][20381] Updated weights for policy 1, policy_version 516389 (0.0008) [2023-12-27 09:12:55,480][20381] Updated weights for policy 1, policy_version 516399 (0.0007) [2023-12-27 09:12:55,673][20347] Updated weights for policy 0, policy_version 516609 (0.0006) [2023-12-27 09:12:55,729][20347] Updated weights for policy 0, policy_version 516619 (0.0008) [2023-12-27 09:12:55,784][20347] Updated weights for policy 0, policy_version 516629 (0.0008) [2023-12-27 09:12:55,839][20347] Updated weights for policy 0, policy_version 516639 (0.0007) [2023-12-27 09:12:56,160][20381] Updated weights for policy 1, policy_version 516409 (0.0007) [2023-12-27 09:12:56,215][20381] Updated weights for policy 1, policy_version 516419 (0.0007) [2023-12-27 09:12:56,265][20381] Updated weights for policy 1, policy_version 516429 (0.0007) [2023-12-27 09:12:56,513][20347] Updated weights for policy 0, policy_version 516649 (0.0006) [2023-12-27 09:12:56,557][20347] Updated weights for policy 0, policy_version 516659 (0.0005) [2023-12-27 09:12:56,603][20347] Updated weights for policy 0, policy_version 516669 (0.0007) [2023-12-27 09:12:57,009][20381] Updated weights for policy 1, policy_version 516439 (0.0007) [2023-12-27 09:12:57,072][20381] Updated weights for policy 1, policy_version 516449 (0.0007) [2023-12-27 09:12:57,125][20381] Updated weights for policy 1, policy_version 516459 (0.0007) [2023-12-27 09:12:57,174][20347] Updated weights for policy 0, policy_version 516679 (0.0010) [2023-12-27 09:12:57,222][20347] Updated weights for policy 0, policy_version 516689 (0.0010) [2023-12-27 09:12:57,276][20347] Updated weights for policy 0, policy_version 516699 (0.0010) [2023-12-27 09:12:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 264527872. Throughput: 0: 10335.0, 1: 10237.9. Samples: 264510836. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:12:57,363][19170] Avg episode reward: [(0, '193.330'), (1, '779.170')] [2023-12-27 09:12:57,821][20381] Updated weights for policy 1, policy_version 516469 (0.0007) [2023-12-27 09:12:57,849][20347] Updated weights for policy 0, policy_version 516709 (0.0009) [2023-12-27 09:12:57,878][20381] Updated weights for policy 1, policy_version 516479 (0.0005) [2023-12-27 09:12:57,909][20347] Updated weights for policy 0, policy_version 516719 (0.0005) [2023-12-27 09:12:57,932][20381] Updated weights for policy 1, policy_version 516489 (0.0006) [2023-12-27 09:12:57,973][20347] Updated weights for policy 0, policy_version 516729 (0.0005) [2023-12-27 09:12:58,625][20381] Updated weights for policy 1, policy_version 516499 (0.0007) [2023-12-27 09:12:58,654][20347] Updated weights for policy 0, policy_version 516739 (0.0006) [2023-12-27 09:12:58,685][20381] Updated weights for policy 1, policy_version 516509 (0.0007) [2023-12-27 09:12:58,717][20347] Updated weights for policy 0, policy_version 516749 (0.0007) [2023-12-27 09:12:58,752][20381] Updated weights for policy 1, policy_version 516519 (0.0008) [2023-12-27 09:12:58,778][20347] Updated weights for policy 0, policy_version 516759 (0.0007) [2023-12-27 09:12:59,504][20347] Updated weights for policy 0, policy_version 516769 (0.0008) [2023-12-27 09:12:59,508][20381] Updated weights for policy 1, policy_version 516529 (0.0009) [2023-12-27 09:12:59,563][20347] Updated weights for policy 0, policy_version 516779 (0.0005) [2023-12-27 09:12:59,567][20381] Updated weights for policy 1, policy_version 516539 (0.0007) [2023-12-27 09:12:59,621][20347] Updated weights for policy 0, policy_version 516789 (0.0006) [2023-12-27 09:12:59,626][20381] Updated weights for policy 1, policy_version 516549 (0.0006) [2023-12-27 09:12:59,674][20347] Updated weights for policy 0, policy_version 516799 (0.0006) [2023-12-27 09:12:59,676][20381] Updated weights for policy 1, policy_version 516559 (0.0006) [2023-12-27 09:13:00,321][20381] Updated weights for policy 1, policy_version 516569 (0.0007) [2023-12-27 09:13:00,374][20347] Updated weights for policy 0, policy_version 516809 (0.0007) [2023-12-27 09:13:00,374][20381] Updated weights for policy 1, policy_version 516579 (0.0007) [2023-12-27 09:13:00,431][20347] Updated weights for policy 0, policy_version 516819 (0.0005) [2023-12-27 09:13:00,433][20381] Updated weights for policy 1, policy_version 516589 (0.0005) [2023-12-27 09:13:00,486][20347] Updated weights for policy 0, policy_version 516829 (0.0007) [2023-12-27 09:13:01,036][20381] Updated weights for policy 1, policy_version 516599 (0.0008) [2023-12-27 09:13:01,091][20381] Updated weights for policy 1, policy_version 516609 (0.0008) [2023-12-27 09:13:01,141][20347] Updated weights for policy 0, policy_version 516839 (0.0006) [2023-12-27 09:13:01,143][20381] Updated weights for policy 1, policy_version 516619 (0.0007) [2023-12-27 09:13:01,200][20347] Updated weights for policy 0, policy_version 516849 (0.0009) [2023-12-27 09:13:01,261][20347] Updated weights for policy 0, policy_version 516859 (0.0007) [2023-12-27 09:13:01,872][20347] Updated weights for policy 0, policy_version 516869 (0.0008) [2023-12-27 09:13:01,906][20381] Updated weights for policy 1, policy_version 516629 (0.0007) [2023-12-27 09:13:01,922][20347] Updated weights for policy 0, policy_version 516879 (0.0005) [2023-12-27 09:13:01,959][20381] Updated weights for policy 1, policy_version 516639 (0.0007) [2023-12-27 09:13:01,973][20347] Updated weights for policy 0, policy_version 516889 (0.0005) [2023-12-27 09:13:02,013][20381] Updated weights for policy 1, policy_version 516649 (0.0007) [2023-12-27 09:13:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 264626176. Throughput: 0: 10380.0, 1: 10248.4. Samples: 264634952. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:02,363][19170] Avg episode reward: [(0, '265.880'), (1, '836.230')] [2023-12-27 09:13:02,681][20347] Updated weights for policy 0, policy_version 516899 (0.0006) [2023-12-27 09:13:02,684][20381] Updated weights for policy 1, policy_version 516659 (0.0007) [2023-12-27 09:13:02,742][20347] Updated weights for policy 0, policy_version 516909 (0.0006) [2023-12-27 09:13:02,747][20381] Updated weights for policy 1, policy_version 516669 (0.0006) [2023-12-27 09:13:02,791][20347] Updated weights for policy 0, policy_version 516919 (0.0005) [2023-12-27 09:13:02,801][20381] Updated weights for policy 1, policy_version 516679 (0.0008) [2023-12-27 09:13:03,358][20381] Updated weights for policy 1, policy_version 516689 (0.0007) [2023-12-27 09:13:03,413][20381] Updated weights for policy 1, policy_version 516699 (0.0005) [2023-12-27 09:13:03,461][20381] Updated weights for policy 1, policy_version 516709 (0.0006) [2023-12-27 09:13:03,478][20347] Updated weights for policy 0, policy_version 516929 (0.0007) [2023-12-27 09:13:03,504][20381] Updated weights for policy 1, policy_version 516719 (0.0006) [2023-12-27 09:13:03,532][20347] Updated weights for policy 0, policy_version 516939 (0.0005) [2023-12-27 09:13:03,580][20347] Updated weights for policy 0, policy_version 516949 (0.0005) [2023-12-27 09:13:03,631][20347] Updated weights for policy 0, policy_version 516959 (0.0006) [2023-12-27 09:13:04,089][20381] Updated weights for policy 1, policy_version 516729 (0.0006) [2023-12-27 09:13:04,148][20381] Updated weights for policy 1, policy_version 516739 (0.0006) [2023-12-27 09:13:04,210][20381] Updated weights for policy 1, policy_version 516749 (0.0007) [2023-12-27 09:13:04,363][20347] Updated weights for policy 0, policy_version 516969 (0.0007) [2023-12-27 09:13:04,423][20347] Updated weights for policy 0, policy_version 516979 (0.0008) [2023-12-27 09:13:04,484][20347] Updated weights for policy 0, policy_version 516989 (0.0008) [2023-12-27 09:13:04,863][20381] Updated weights for policy 1, policy_version 516759 (0.0005) [2023-12-27 09:13:04,918][20381] Updated weights for policy 1, policy_version 516769 (0.0007) [2023-12-27 09:13:04,966][20381] Updated weights for policy 1, policy_version 516779 (0.0010) [2023-12-27 09:13:05,094][20347] Updated weights for policy 0, policy_version 516999 (0.0007) [2023-12-27 09:13:05,151][20347] Updated weights for policy 0, policy_version 517009 (0.0007) [2023-12-27 09:13:05,202][20347] Updated weights for policy 0, policy_version 517019 (0.0007) [2023-12-27 09:13:05,622][20381] Updated weights for policy 1, policy_version 516789 (0.0009) [2023-12-27 09:13:05,682][20381] Updated weights for policy 1, policy_version 516799 (0.0008) [2023-12-27 09:13:05,745][20381] Updated weights for policy 1, policy_version 516809 (0.0007) [2023-12-27 09:13:05,886][20347] Updated weights for policy 0, policy_version 517029 (0.0006) [2023-12-27 09:13:05,947][20347] Updated weights for policy 0, policy_version 517039 (0.0008) [2023-12-27 09:13:06,005][20347] Updated weights for policy 0, policy_version 517049 (0.0008) [2023-12-27 09:13:06,420][20381] Updated weights for policy 1, policy_version 516819 (0.0008) [2023-12-27 09:13:06,477][20381] Updated weights for policy 1, policy_version 516829 (0.0008) [2023-12-27 09:13:06,535][20381] Updated weights for policy 1, policy_version 516839 (0.0008) [2023-12-27 09:13:06,700][20347] Updated weights for policy 0, policy_version 517059 (0.0007) [2023-12-27 09:13:06,758][20347] Updated weights for policy 0, policy_version 517069 (0.0007) [2023-12-27 09:13:06,814][20347] Updated weights for policy 0, policy_version 517079 (0.0008) [2023-12-27 09:13:07,234][20381] Updated weights for policy 1, policy_version 516849 (0.0008) [2023-12-27 09:13:07,296][20381] Updated weights for policy 1, policy_version 516859 (0.0011) [2023-12-27 09:13:07,355][20381] Updated weights for policy 1, policy_version 516869 (0.0011) [2023-12-27 09:13:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 264724480. Throughput: 0: 10373.8, 1: 10318.8. Samples: 264699152. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:07,363][19170] Avg episode reward: [(0, '254.650'), (1, '852.990')] [2023-12-27 09:13:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000517088_132390912.pth... [2023-12-27 09:13:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000515872_132079616.pth [2023-12-27 09:13:07,403][20381] Updated weights for policy 1, policy_version 516879 (0.0010) [2023-12-27 09:13:07,407][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000516880_132341760.pth... [2023-12-27 09:13:07,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000515664_132030464.pth [2023-12-27 09:13:07,475][20347] Updated weights for policy 0, policy_version 517089 (0.0008) [2023-12-27 09:13:07,524][20347] Updated weights for policy 0, policy_version 517099 (0.0008) [2023-12-27 09:13:07,576][20347] Updated weights for policy 0, policy_version 517109 (0.0008) [2023-12-27 09:13:07,631][20347] Updated weights for policy 0, policy_version 517119 (0.0007) [2023-12-27 09:13:07,970][20381] Updated weights for policy 1, policy_version 516889 (0.0006) [2023-12-27 09:13:08,022][20381] Updated weights for policy 1, policy_version 516899 (0.0006) [2023-12-27 09:13:08,072][20381] Updated weights for policy 1, policy_version 516909 (0.0005) [2023-12-27 09:13:08,409][20347] Updated weights for policy 0, policy_version 517129 (0.0007) [2023-12-27 09:13:08,467][20347] Updated weights for policy 0, policy_version 517139 (0.0007) [2023-12-27 09:13:08,518][20347] Updated weights for policy 0, policy_version 517149 (0.0007) [2023-12-27 09:13:08,632][20381] Updated weights for policy 1, policy_version 516919 (0.0006) [2023-12-27 09:13:08,695][20381] Updated weights for policy 1, policy_version 516929 (0.0007) [2023-12-27 09:13:08,751][20381] Updated weights for policy 1, policy_version 516939 (0.0011) [2023-12-27 09:13:09,209][20347] Updated weights for policy 0, policy_version 517159 (0.0008) [2023-12-27 09:13:09,271][20347] Updated weights for policy 0, policy_version 517169 (0.0007) [2023-12-27 09:13:09,327][20347] Updated weights for policy 0, policy_version 517179 (0.0008) [2023-12-27 09:13:09,424][20381] Updated weights for policy 1, policy_version 516949 (0.0010) [2023-12-27 09:13:09,480][20381] Updated weights for policy 1, policy_version 516959 (0.0011) [2023-12-27 09:13:09,533][20381] Updated weights for policy 1, policy_version 516969 (0.0011) [2023-12-27 09:13:10,064][20347] Updated weights for policy 0, policy_version 517189 (0.0008) [2023-12-27 09:13:10,118][20347] Updated weights for policy 0, policy_version 517199 (0.0008) [2023-12-27 09:13:10,178][20347] Updated weights for policy 0, policy_version 517209 (0.0008) [2023-12-27 09:13:10,251][20381] Updated weights for policy 1, policy_version 516979 (0.0009) [2023-12-27 09:13:10,321][20381] Updated weights for policy 1, policy_version 516989 (0.0009) [2023-12-27 09:13:10,390][20381] Updated weights for policy 1, policy_version 516999 (0.0009) [2023-12-27 09:13:10,835][20347] Updated weights for policy 0, policy_version 517219 (0.0007) [2023-12-27 09:13:10,894][20347] Updated weights for policy 0, policy_version 517229 (0.0006) [2023-12-27 09:13:10,951][20347] Updated weights for policy 0, policy_version 517239 (0.0008) [2023-12-27 09:13:11,003][20381] Updated weights for policy 1, policy_version 517009 (0.0009) [2023-12-27 09:13:11,060][20381] Updated weights for policy 1, policy_version 517019 (0.0007) [2023-12-27 09:13:11,122][20381] Updated weights for policy 1, policy_version 517029 (0.0007) [2023-12-27 09:13:11,190][20381] Updated weights for policy 1, policy_version 517039 (0.0008) [2023-12-27 09:13:11,641][20347] Updated weights for policy 0, policy_version 517249 (0.0008) [2023-12-27 09:13:11,703][20347] Updated weights for policy 0, policy_version 517259 (0.0008) [2023-12-27 09:13:11,766][20347] Updated weights for policy 0, policy_version 517269 (0.0008) [2023-12-27 09:13:11,815][20347] Updated weights for policy 0, policy_version 517279 (0.0007) [2023-12-27 09:13:11,970][20381] Updated weights for policy 1, policy_version 517049 (0.0007) [2023-12-27 09:13:12,028][20381] Updated weights for policy 1, policy_version 517059 (0.0008) [2023-12-27 09:13:12,090][20381] Updated weights for policy 1, policy_version 517069 (0.0008) [2023-12-27 09:13:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 264830976. Throughput: 0: 10312.2, 1: 10401.4. Samples: 264821332. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:12,363][19170] Avg episode reward: [(0, '286.180'), (1, '731.320')] [2023-12-27 09:13:12,433][20347] Updated weights for policy 0, policy_version 517289 (0.0007) [2023-12-27 09:13:12,487][20347] Updated weights for policy 0, policy_version 517299 (0.0007) [2023-12-27 09:13:12,547][20347] Updated weights for policy 0, policy_version 517309 (0.0008) [2023-12-27 09:13:12,808][20381] Updated weights for policy 1, policy_version 517079 (0.0008) [2023-12-27 09:13:12,867][20381] Updated weights for policy 1, policy_version 517089 (0.0007) [2023-12-27 09:13:12,929][20381] Updated weights for policy 1, policy_version 517099 (0.0007) [2023-12-27 09:13:13,250][20347] Updated weights for policy 0, policy_version 517319 (0.0006) [2023-12-27 09:13:13,310][20347] Updated weights for policy 0, policy_version 517329 (0.0006) [2023-12-27 09:13:13,371][20347] Updated weights for policy 0, policy_version 517339 (0.0007) [2023-12-27 09:13:13,590][20381] Updated weights for policy 1, policy_version 517109 (0.0007) [2023-12-27 09:13:13,639][20381] Updated weights for policy 1, policy_version 517119 (0.0005) [2023-12-27 09:13:13,686][20381] Updated weights for policy 1, policy_version 517129 (0.0009) [2023-12-27 09:13:13,986][20347] Updated weights for policy 0, policy_version 517349 (0.0008) [2023-12-27 09:13:14,044][20347] Updated weights for policy 0, policy_version 517359 (0.0005) [2023-12-27 09:13:14,096][20347] Updated weights for policy 0, policy_version 517369 (0.0005) [2023-12-27 09:13:14,344][20292] KL-divergence is very high: 116.8442 [2023-12-27 09:13:14,359][20381] Updated weights for policy 1, policy_version 517139 (0.0009) [2023-12-27 09:13:14,382][20292] KL-divergence is very high: 205.1658 [2023-12-27 09:13:14,406][20381] Updated weights for policy 1, policy_version 517149 (0.0007) [2023-12-27 09:13:14,455][20381] Updated weights for policy 1, policy_version 517159 (0.0007) [2023-12-27 09:13:14,765][20347] Updated weights for policy 0, policy_version 517379 (0.0008) [2023-12-27 09:13:14,824][20347] Updated weights for policy 0, policy_version 517389 (0.0007) [2023-12-27 09:13:14,884][20347] Updated weights for policy 0, policy_version 517399 (0.0007) [2023-12-27 09:13:15,214][20381] Updated weights for policy 1, policy_version 517169 (0.0007) [2023-12-27 09:13:15,272][20381] Updated weights for policy 1, policy_version 517179 (0.0008) [2023-12-27 09:13:15,326][20381] Updated weights for policy 1, policy_version 517189 (0.0008) [2023-12-27 09:13:15,382][20381] Updated weights for policy 1, policy_version 517199 (0.0008) [2023-12-27 09:13:15,566][20347] Updated weights for policy 0, policy_version 517409 (0.0007) [2023-12-27 09:13:15,622][20347] Updated weights for policy 0, policy_version 517419 (0.0008) [2023-12-27 09:13:15,681][20347] Updated weights for policy 0, policy_version 517429 (0.0008) [2023-12-27 09:13:15,739][20347] Updated weights for policy 0, policy_version 517439 (0.0007) [2023-12-27 09:13:16,041][20381] Updated weights for policy 1, policy_version 517209 (0.0008) [2023-12-27 09:13:16,096][20381] Updated weights for policy 1, policy_version 517219 (0.0009) [2023-12-27 09:13:16,148][20381] Updated weights for policy 1, policy_version 517229 (0.0009) [2023-12-27 09:13:16,289][20347] Updated weights for policy 0, policy_version 517449 (0.0005) [2023-12-27 09:13:16,337][20347] Updated weights for policy 0, policy_version 517459 (0.0006) [2023-12-27 09:13:16,401][20347] Updated weights for policy 0, policy_version 517469 (0.0006) [2023-12-27 09:13:16,865][20381] Updated weights for policy 1, policy_version 517239 (0.0009) [2023-12-27 09:13:16,919][20381] Updated weights for policy 1, policy_version 517249 (0.0007) [2023-12-27 09:13:16,974][20381] Updated weights for policy 1, policy_version 517259 (0.0008) [2023-12-27 09:13:17,057][20347] Updated weights for policy 0, policy_version 517479 (0.0007) [2023-12-27 09:13:17,120][20347] Updated weights for policy 0, policy_version 517489 (0.0005) [2023-12-27 09:13:17,168][20347] Updated weights for policy 0, policy_version 517499 (0.0005) [2023-12-27 09:13:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 264937472. Throughput: 0: 10381.0, 1: 10274.5. Samples: 264944480. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:17,363][19170] Avg episode reward: [(0, '282.210'), (1, '765.270')] [2023-12-27 09:13:17,642][20381] Updated weights for policy 1, policy_version 517269 (0.0008) [2023-12-27 09:13:17,689][20381] Updated weights for policy 1, policy_version 517279 (0.0008) [2023-12-27 09:13:17,742][20381] Updated weights for policy 1, policy_version 517289 (0.0008) [2023-12-27 09:13:17,760][20347] Updated weights for policy 0, policy_version 517509 (0.0005) [2023-12-27 09:13:17,813][20347] Updated weights for policy 0, policy_version 517519 (0.0005) [2023-12-27 09:13:17,874][20347] Updated weights for policy 0, policy_version 517529 (0.0007) [2023-12-27 09:13:18,404][20381] Updated weights for policy 1, policy_version 517299 (0.0008) [2023-12-27 09:13:18,461][20381] Updated weights for policy 1, policy_version 517309 (0.0008) [2023-12-27 09:13:18,491][20347] Updated weights for policy 0, policy_version 517539 (0.0006) [2023-12-27 09:13:18,518][20381] Updated weights for policy 1, policy_version 517319 (0.0007) [2023-12-27 09:13:18,551][20347] Updated weights for policy 0, policy_version 517549 (0.0005) [2023-12-27 09:13:18,612][20347] Updated weights for policy 0, policy_version 517559 (0.0005) [2023-12-27 09:13:19,216][20381] Updated weights for policy 1, policy_version 517329 (0.0008) [2023-12-27 09:13:19,268][20347] Updated weights for policy 0, policy_version 517569 (0.0005) [2023-12-27 09:13:19,281][20381] Updated weights for policy 1, policy_version 517339 (0.0008) [2023-12-27 09:13:19,330][20347] Updated weights for policy 0, policy_version 517579 (0.0008) [2023-12-27 09:13:19,340][20381] Updated weights for policy 1, policy_version 517349 (0.0007) [2023-12-27 09:13:19,392][20347] Updated weights for policy 0, policy_version 517589 (0.0007) [2023-12-27 09:13:19,403][20381] Updated weights for policy 1, policy_version 517359 (0.0010) [2023-12-27 09:13:19,448][20347] Updated weights for policy 0, policy_version 517599 (0.0008) [2023-12-27 09:13:20,093][20381] Updated weights for policy 1, policy_version 517369 (0.0006) [2023-12-27 09:13:20,145][20381] Updated weights for policy 1, policy_version 517379 (0.0006) [2023-12-27 09:13:20,156][20347] Updated weights for policy 0, policy_version 517609 (0.0006) [2023-12-27 09:13:20,198][20381] Updated weights for policy 1, policy_version 517389 (0.0009) [2023-12-27 09:13:20,219][20347] Updated weights for policy 0, policy_version 517619 (0.0006) [2023-12-27 09:13:20,278][20347] Updated weights for policy 0, policy_version 517629 (0.0007) [2023-12-27 09:13:20,849][20381] Updated weights for policy 1, policy_version 517399 (0.0008) [2023-12-27 09:13:20,912][20381] Updated weights for policy 1, policy_version 517409 (0.0005) [2023-12-27 09:13:20,948][20347] Updated weights for policy 0, policy_version 517639 (0.0009) [2023-12-27 09:13:20,976][20381] Updated weights for policy 1, policy_version 517419 (0.0006) [2023-12-27 09:13:21,005][20347] Updated weights for policy 0, policy_version 517649 (0.0005) [2023-12-27 09:13:21,069][20347] Updated weights for policy 0, policy_version 517659 (0.0010) [2023-12-27 09:13:21,632][20381] Updated weights for policy 1, policy_version 517429 (0.0010) [2023-12-27 09:13:21,687][20381] Updated weights for policy 1, policy_version 517439 (0.0011) [2023-12-27 09:13:21,753][20381] Updated weights for policy 1, policy_version 517449 (0.0009) [2023-12-27 09:13:21,815][20347] Updated weights for policy 0, policy_version 517669 (0.0007) [2023-12-27 09:13:21,870][20347] Updated weights for policy 0, policy_version 517679 (0.0005) [2023-12-27 09:13:21,922][20347] Updated weights for policy 0, policy_version 517689 (0.0006) [2023-12-27 09:13:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 265035776. Throughput: 0: 10351.1, 1: 10262.1. Samples: 265006988. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:22,363][19170] Avg episode reward: [(0, '334.850'), (1, '873.470')] [2023-12-27 09:13:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000517456_132489216.pth... [2023-12-27 09:13:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000517696_132546560.pth... [2023-12-27 09:13:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000516272_132186112.pth [2023-12-27 09:13:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000516480_132235264.pth [2023-12-27 09:13:22,517][20381] Updated weights for policy 1, policy_version 517459 (0.0011) [2023-12-27 09:13:22,546][20347] Updated weights for policy 0, policy_version 517699 (0.0007) [2023-12-27 09:13:22,579][20381] Updated weights for policy 1, policy_version 517469 (0.0011) [2023-12-27 09:13:22,606][20347] Updated weights for policy 0, policy_version 517709 (0.0007) [2023-12-27 09:13:22,635][20381] Updated weights for policy 1, policy_version 517479 (0.0010) [2023-12-27 09:13:22,662][20347] Updated weights for policy 0, policy_version 517719 (0.0005) [2023-12-27 09:13:23,329][20381] Updated weights for policy 1, policy_version 517489 (0.0011) [2023-12-27 09:13:23,362][20347] Updated weights for policy 0, policy_version 517729 (0.0006) [2023-12-27 09:13:23,383][20381] Updated weights for policy 1, policy_version 517499 (0.0010) [2023-12-27 09:13:23,417][20347] Updated weights for policy 0, policy_version 517739 (0.0005) [2023-12-27 09:13:23,437][20381] Updated weights for policy 1, policy_version 517509 (0.0010) [2023-12-27 09:13:23,472][20347] Updated weights for policy 0, policy_version 517749 (0.0005) [2023-12-27 09:13:23,489][20381] Updated weights for policy 1, policy_version 517519 (0.0010) [2023-12-27 09:13:23,527][20347] Updated weights for policy 0, policy_version 517759 (0.0006) [2023-12-27 09:13:24,159][20347] Updated weights for policy 0, policy_version 517769 (0.0006) [2023-12-27 09:13:24,187][20381] Updated weights for policy 1, policy_version 517529 (0.0011) [2023-12-27 09:13:24,215][20347] Updated weights for policy 0, policy_version 517779 (0.0005) [2023-12-27 09:13:24,247][20381] Updated weights for policy 1, policy_version 517539 (0.0011) [2023-12-27 09:13:24,264][20347] Updated weights for policy 0, policy_version 517789 (0.0006) [2023-12-27 09:13:24,305][20381] Updated weights for policy 1, policy_version 517549 (0.0011) [2023-12-27 09:13:24,945][20347] Updated weights for policy 0, policy_version 517799 (0.0008) [2023-12-27 09:13:24,983][20381] Updated weights for policy 1, policy_version 517559 (0.0010) [2023-12-27 09:13:24,996][20347] Updated weights for policy 0, policy_version 517809 (0.0005) [2023-12-27 09:13:25,034][20381] Updated weights for policy 1, policy_version 517569 (0.0007) [2023-12-27 09:13:25,057][20347] Updated weights for policy 0, policy_version 517819 (0.0006) [2023-12-27 09:13:25,082][20381] Updated weights for policy 1, policy_version 517579 (0.0008) [2023-12-27 09:13:25,751][20347] Updated weights for policy 0, policy_version 517829 (0.0007) [2023-12-27 09:13:25,760][20381] Updated weights for policy 1, policy_version 517589 (0.0009) [2023-12-27 09:13:25,799][20347] Updated weights for policy 0, policy_version 517839 (0.0006) [2023-12-27 09:13:25,817][20381] Updated weights for policy 1, policy_version 517599 (0.0007) [2023-12-27 09:13:25,848][20347] Updated weights for policy 0, policy_version 517849 (0.0006) [2023-12-27 09:13:25,868][20381] Updated weights for policy 1, policy_version 517609 (0.0007) [2023-12-27 09:13:26,516][20347] Updated weights for policy 0, policy_version 517859 (0.0006) [2023-12-27 09:13:26,539][20381] Updated weights for policy 1, policy_version 517619 (0.0008) [2023-12-27 09:13:26,565][20347] Updated weights for policy 0, policy_version 517869 (0.0005) [2023-12-27 09:13:26,587][20381] Updated weights for policy 1, policy_version 517629 (0.0010) [2023-12-27 09:13:26,624][20347] Updated weights for policy 0, policy_version 517879 (0.0010) [2023-12-27 09:13:26,644][20381] Updated weights for policy 1, policy_version 517639 (0.0010) [2023-12-27 09:13:27,199][20347] Updated weights for policy 0, policy_version 517889 (0.0010) [2023-12-27 09:13:27,250][20347] Updated weights for policy 0, policy_version 517899 (0.0010) [2023-12-27 09:13:27,310][20381] Updated weights for policy 1, policy_version 517649 (0.0007) [2023-12-27 09:13:27,313][20347] Updated weights for policy 0, policy_version 517909 (0.0010) [2023-12-27 09:13:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.1, 300 sec: 20771.6). Total num frames: 265134080. Throughput: 0: 10370.3, 1: 10238.1. Samples: 265128904. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:27,363][19170] Avg episode reward: [(0, '314.780'), (1, '854.070')] [2023-12-27 09:13:27,364][20381] Updated weights for policy 1, policy_version 517659 (0.0009) [2023-12-27 09:13:27,370][20347] Updated weights for policy 0, policy_version 517919 (0.0010) [2023-12-27 09:13:27,422][20381] Updated weights for policy 1, policy_version 517669 (0.0010) [2023-12-27 09:13:27,479][20381] Updated weights for policy 1, policy_version 517679 (0.0010) [2023-12-27 09:13:27,947][20347] Updated weights for policy 0, policy_version 517929 (0.0007) [2023-12-27 09:13:28,001][20347] Updated weights for policy 0, policy_version 517939 (0.0006) [2023-12-27 09:13:28,061][20347] Updated weights for policy 0, policy_version 517949 (0.0005) [2023-12-27 09:13:28,171][20381] Updated weights for policy 1, policy_version 517689 (0.0010) [2023-12-27 09:13:28,216][20381] Updated weights for policy 1, policy_version 517699 (0.0010) [2023-12-27 09:13:28,267][20381] Updated weights for policy 1, policy_version 517709 (0.0010) [2023-12-27 09:13:28,663][20347] Updated weights for policy 0, policy_version 517959 (0.0007) [2023-12-27 09:13:28,712][20347] Updated weights for policy 0, policy_version 517969 (0.0008) [2023-12-27 09:13:28,766][20347] Updated weights for policy 0, policy_version 517979 (0.0007) [2023-12-27 09:13:28,995][20381] Updated weights for policy 1, policy_version 517719 (0.0009) [2023-12-27 09:13:29,046][20381] Updated weights for policy 1, policy_version 517729 (0.0007) [2023-12-27 09:13:29,092][20381] Updated weights for policy 1, policy_version 517739 (0.0008) [2023-12-27 09:13:29,465][20347] Updated weights for policy 0, policy_version 517989 (0.0008) [2023-12-27 09:13:29,525][20347] Updated weights for policy 0, policy_version 517999 (0.0008) [2023-12-27 09:13:29,583][20347] Updated weights for policy 0, policy_version 518009 (0.0007) [2023-12-27 09:13:29,859][20381] Updated weights for policy 1, policy_version 517749 (0.0007) [2023-12-27 09:13:29,916][20381] Updated weights for policy 1, policy_version 517759 (0.0008) [2023-12-27 09:13:29,977][20381] Updated weights for policy 1, policy_version 517769 (0.0008) [2023-12-27 09:13:30,220][20347] Updated weights for policy 0, policy_version 518019 (0.0008) [2023-12-27 09:13:30,285][20347] Updated weights for policy 0, policy_version 518029 (0.0008) [2023-12-27 09:13:30,341][20347] Updated weights for policy 0, policy_version 518039 (0.0006) [2023-12-27 09:13:30,701][20381] Updated weights for policy 1, policy_version 517779 (0.0007) [2023-12-27 09:13:30,755][20381] Updated weights for policy 1, policy_version 517789 (0.0008) [2023-12-27 09:13:30,818][20381] Updated weights for policy 1, policy_version 517799 (0.0006) [2023-12-27 09:13:30,892][20347] Updated weights for policy 0, policy_version 518049 (0.0005) [2023-12-27 09:13:30,951][20347] Updated weights for policy 0, policy_version 518059 (0.0005) [2023-12-27 09:13:31,014][20347] Updated weights for policy 0, policy_version 518069 (0.0006) [2023-12-27 09:13:31,074][20347] Updated weights for policy 0, policy_version 518079 (0.0007) [2023-12-27 09:13:31,440][20381] Updated weights for policy 1, policy_version 517809 (0.0009) [2023-12-27 09:13:31,485][20381] Updated weights for policy 1, policy_version 517819 (0.0010) [2023-12-27 09:13:31,530][20381] Updated weights for policy 1, policy_version 517829 (0.0010) [2023-12-27 09:13:31,582][20381] Updated weights for policy 1, policy_version 517839 (0.0010) [2023-12-27 09:13:31,679][20347] Updated weights for policy 0, policy_version 518089 (0.0010) [2023-12-27 09:13:31,748][20347] Updated weights for policy 0, policy_version 518099 (0.0008) [2023-12-27 09:13:31,804][20347] Updated weights for policy 0, policy_version 518109 (0.0009) [2023-12-27 09:13:32,269][20381] Updated weights for policy 1, policy_version 517849 (0.0009) [2023-12-27 09:13:32,318][20381] Updated weights for policy 1, policy_version 517859 (0.0010) [2023-12-27 09:13:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 265240576. Throughput: 0: 10416.7, 1: 10242.8. Samples: 265256424. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:32,363][19170] Avg episode reward: [(0, '257.100'), (1, '825.370')] [2023-12-27 09:13:32,370][20381] Updated weights for policy 1, policy_version 517869 (0.0009) [2023-12-27 09:13:32,430][20347] Updated weights for policy 0, policy_version 518119 (0.0010) [2023-12-27 09:13:32,478][20347] Updated weights for policy 0, policy_version 518129 (0.0010) [2023-12-27 09:13:32,526][20347] Updated weights for policy 0, policy_version 518139 (0.0010) [2023-12-27 09:13:33,033][20381] Updated weights for policy 1, policy_version 517879 (0.0008) [2023-12-27 09:13:33,081][20381] Updated weights for policy 1, policy_version 517889 (0.0005) [2023-12-27 09:13:33,131][20381] Updated weights for policy 1, policy_version 517899 (0.0006) [2023-12-27 09:13:33,148][20347] Updated weights for policy 0, policy_version 518149 (0.0008) [2023-12-27 09:13:33,209][20347] Updated weights for policy 0, policy_version 518159 (0.0007) [2023-12-27 09:13:33,265][20347] Updated weights for policy 0, policy_version 518169 (0.0006) [2023-12-27 09:13:33,796][20381] Updated weights for policy 1, policy_version 517909 (0.0008) [2023-12-27 09:13:33,845][20381] Updated weights for policy 1, policy_version 517919 (0.0009) [2023-12-27 09:13:33,857][20347] Updated weights for policy 0, policy_version 518179 (0.0005) [2023-12-27 09:13:33,895][20381] Updated weights for policy 1, policy_version 517929 (0.0005) [2023-12-27 09:13:33,916][20347] Updated weights for policy 0, policy_version 518189 (0.0008) [2023-12-27 09:13:33,963][20347] Updated weights for policy 0, policy_version 518199 (0.0009) [2023-12-27 09:13:34,567][20381] Updated weights for policy 1, policy_version 517939 (0.0009) [2023-12-27 09:13:34,605][20347] Updated weights for policy 0, policy_version 518209 (0.0006) [2023-12-27 09:13:34,628][20381] Updated weights for policy 1, policy_version 517949 (0.0011) [2023-12-27 09:13:34,659][20347] Updated weights for policy 0, policy_version 518219 (0.0009) [2023-12-27 09:13:34,684][20381] Updated weights for policy 1, policy_version 517959 (0.0009) [2023-12-27 09:13:34,708][20347] Updated weights for policy 0, policy_version 518229 (0.0007) [2023-12-27 09:13:34,759][20347] Updated weights for policy 0, policy_version 518239 (0.0006) [2023-12-27 09:13:35,334][20381] Updated weights for policy 1, policy_version 517969 (0.0006) [2023-12-27 09:13:35,396][20381] Updated weights for policy 1, policy_version 517979 (0.0010) [2023-12-27 09:13:35,441][20381] Updated weights for policy 1, policy_version 517989 (0.0010) [2023-12-27 09:13:35,472][20347] Updated weights for policy 0, policy_version 518249 (0.0006) [2023-12-27 09:13:35,492][20381] Updated weights for policy 1, policy_version 517999 (0.0010) [2023-12-27 09:13:35,517][20347] Updated weights for policy 0, policy_version 518259 (0.0006) [2023-12-27 09:13:35,562][20347] Updated weights for policy 0, policy_version 518269 (0.0008) [2023-12-27 09:13:36,114][20381] Updated weights for policy 1, policy_version 518009 (0.0007) [2023-12-27 09:13:36,171][20381] Updated weights for policy 1, policy_version 518019 (0.0006) [2023-12-27 09:13:36,232][20381] Updated weights for policy 1, policy_version 518029 (0.0007) [2023-12-27 09:13:36,243][20347] Updated weights for policy 0, policy_version 518280 (0.0007) [2023-12-27 09:13:36,306][20347] Updated weights for policy 0, policy_version 518290 (0.0010) [2023-12-27 09:13:36,365][20347] Updated weights for policy 0, policy_version 518300 (0.0010) [2023-12-27 09:13:36,816][20381] Updated weights for policy 1, policy_version 518039 (0.0006) [2023-12-27 09:13:36,872][20381] Updated weights for policy 1, policy_version 518049 (0.0006) [2023-12-27 09:13:36,922][20381] Updated weights for policy 1, policy_version 518059 (0.0008) [2023-12-27 09:13:36,965][20347] Updated weights for policy 0, policy_version 518310 (0.0005) [2023-12-27 09:13:37,015][20347] Updated weights for policy 0, policy_version 518320 (0.0008) [2023-12-27 09:13:37,061][20347] Updated weights for policy 0, policy_version 518330 (0.0010) [2023-12-27 09:13:37,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 265355264. Throughput: 0: 10433.0, 1: 10252.0. Samples: 265320804. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:37,363][19170] Avg episode reward: [(0, '281.370'), (1, '802.260')] [2023-12-27 09:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000518064_132644864.pth... [2023-12-27 09:13:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000518336_132710400.pth... [2023-12-27 09:13:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000516880_132341760.pth [2023-12-27 09:13:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000517088_132390912.pth [2023-12-27 09:13:37,494][20381] Updated weights for policy 1, policy_version 518069 (0.0008) [2023-12-27 09:13:37,552][20381] Updated weights for policy 1, policy_version 518079 (0.0010) [2023-12-27 09:13:37,599][20381] Updated weights for policy 1, policy_version 518089 (0.0008) [2023-12-27 09:13:37,689][20347] Updated weights for policy 0, policy_version 518340 (0.0009) [2023-12-27 09:13:37,748][20347] Updated weights for policy 0, policy_version 518350 (0.0010) [2023-12-27 09:13:37,803][20347] Updated weights for policy 0, policy_version 518360 (0.0010) [2023-12-27 09:13:38,260][20381] Updated weights for policy 1, policy_version 518099 (0.0009) [2023-12-27 09:13:38,305][20381] Updated weights for policy 1, policy_version 518109 (0.0007) [2023-12-27 09:13:38,364][20381] Updated weights for policy 1, policy_version 518119 (0.0008) [2023-12-27 09:13:38,462][20347] Updated weights for policy 0, policy_version 518370 (0.0010) [2023-12-27 09:13:38,523][20347] Updated weights for policy 0, policy_version 518380 (0.0006) [2023-12-27 09:13:38,581][20347] Updated weights for policy 0, policy_version 518390 (0.0009) [2023-12-27 09:13:38,635][20347] Updated weights for policy 0, policy_version 518400 (0.0010) [2023-12-27 09:13:39,018][20381] Updated weights for policy 1, policy_version 518129 (0.0008) [2023-12-27 09:13:39,081][20381] Updated weights for policy 1, policy_version 518139 (0.0010) [2023-12-27 09:13:39,140][20381] Updated weights for policy 1, policy_version 518149 (0.0011) [2023-12-27 09:13:39,192][20381] Updated weights for policy 1, policy_version 518159 (0.0010) [2023-12-27 09:13:39,266][20347] Updated weights for policy 0, policy_version 518410 (0.0010) [2023-12-27 09:13:39,331][20347] Updated weights for policy 0, policy_version 518420 (0.0010) [2023-12-27 09:13:39,398][20347] Updated weights for policy 0, policy_version 518430 (0.0011) [2023-12-27 09:13:39,925][20381] Updated weights for policy 1, policy_version 518169 (0.0009) [2023-12-27 09:13:39,992][20381] Updated weights for policy 1, policy_version 518179 (0.0007) [2023-12-27 09:13:40,063][20381] Updated weights for policy 1, policy_version 518189 (0.0011) [2023-12-27 09:13:40,080][20347] Updated weights for policy 0, policy_version 518440 (0.0008) [2023-12-27 09:13:40,142][20347] Updated weights for policy 0, policy_version 518450 (0.0006) [2023-12-27 09:13:40,203][20347] Updated weights for policy 0, policy_version 518460 (0.0006) [2023-12-27 09:13:40,703][20381] Updated weights for policy 1, policy_version 518199 (0.0009) [2023-12-27 09:13:40,761][20381] Updated weights for policy 1, policy_version 518209 (0.0011) [2023-12-27 09:13:40,816][20381] Updated weights for policy 1, policy_version 518219 (0.0006) [2023-12-27 09:13:40,828][20347] Updated weights for policy 0, policy_version 518470 (0.0008) [2023-12-27 09:13:40,887][20347] Updated weights for policy 0, policy_version 518480 (0.0008) [2023-12-27 09:13:40,936][20347] Updated weights for policy 0, policy_version 518490 (0.0008) [2023-12-27 09:13:41,542][20381] Updated weights for policy 1, policy_version 518229 (0.0007) [2023-12-27 09:13:41,594][20381] Updated weights for policy 1, policy_version 518239 (0.0007) [2023-12-27 09:13:41,659][20381] Updated weights for policy 1, policy_version 518249 (0.0008) [2023-12-27 09:13:41,699][20347] Updated weights for policy 0, policy_version 518500 (0.0008) [2023-12-27 09:13:41,760][20347] Updated weights for policy 0, policy_version 518510 (0.0008) [2023-12-27 09:13:41,849][20347] Updated weights for policy 0, policy_version 518520 (0.0007) [2023-12-27 09:13:42,332][20381] Updated weights for policy 1, policy_version 518259 (0.0008) [2023-12-27 09:13:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 265453568. Throughput: 0: 10505.8, 1: 10318.7. Samples: 265447936. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:42,363][19170] Avg episode reward: [(0, '295.920'), (1, '746.830')] [2023-12-27 09:13:42,394][20381] Updated weights for policy 1, policy_version 518269 (0.0011) [2023-12-27 09:13:42,442][20381] Updated weights for policy 1, policy_version 518279 (0.0011) [2023-12-27 09:13:42,499][20347] Updated weights for policy 0, policy_version 518530 (0.0006) [2023-12-27 09:13:42,545][20347] Updated weights for policy 0, policy_version 518540 (0.0006) [2023-12-27 09:13:42,591][20347] Updated weights for policy 0, policy_version 518550 (0.0005) [2023-12-27 09:13:42,644][20347] Updated weights for policy 0, policy_version 518560 (0.0005) [2023-12-27 09:13:43,117][20381] Updated weights for policy 1, policy_version 518289 (0.0010) [2023-12-27 09:13:43,174][20381] Updated weights for policy 1, policy_version 518299 (0.0009) [2023-12-27 09:13:43,232][20381] Updated weights for policy 1, policy_version 518309 (0.0010) [2023-12-27 09:13:43,290][20381] Updated weights for policy 1, policy_version 518319 (0.0011) [2023-12-27 09:13:43,300][20347] Updated weights for policy 0, policy_version 518570 (0.0005) [2023-12-27 09:13:43,345][20347] Updated weights for policy 0, policy_version 518580 (0.0007) [2023-12-27 09:13:43,393][20347] Updated weights for policy 0, policy_version 518590 (0.0007) [2023-12-27 09:13:43,913][20381] Updated weights for policy 1, policy_version 518329 (0.0006) [2023-12-27 09:13:43,963][20381] Updated weights for policy 1, policy_version 518339 (0.0006) [2023-12-27 09:13:44,025][20381] Updated weights for policy 1, policy_version 518349 (0.0007) [2023-12-27 09:13:44,070][20347] Updated weights for policy 0, policy_version 518600 (0.0006) [2023-12-27 09:13:44,128][20347] Updated weights for policy 0, policy_version 518610 (0.0006) [2023-12-27 09:13:44,195][20347] Updated weights for policy 0, policy_version 518620 (0.0005) [2023-12-27 09:13:44,604][20381] Updated weights for policy 1, policy_version 518359 (0.0008) [2023-12-27 09:13:44,657][20381] Updated weights for policy 1, policy_version 518369 (0.0008) [2023-12-27 09:13:44,720][20381] Updated weights for policy 1, policy_version 518379 (0.0008) [2023-12-27 09:13:44,804][20347] Updated weights for policy 0, policy_version 518630 (0.0006) [2023-12-27 09:13:44,862][20347] Updated weights for policy 0, policy_version 518640 (0.0005) [2023-12-27 09:13:44,915][20347] Updated weights for policy 0, policy_version 518650 (0.0006) [2023-12-27 09:13:45,442][20381] Updated weights for policy 1, policy_version 518389 (0.0008) [2023-12-27 09:13:45,499][20381] Updated weights for policy 1, policy_version 518399 (0.0008) [2023-12-27 09:13:45,512][20347] Updated weights for policy 0, policy_version 518660 (0.0006) [2023-12-27 09:13:45,555][20381] Updated weights for policy 1, policy_version 518409 (0.0005) [2023-12-27 09:13:45,571][20347] Updated weights for policy 0, policy_version 518670 (0.0007) [2023-12-27 09:13:45,635][20347] Updated weights for policy 0, policy_version 518680 (0.0005) [2023-12-27 09:13:46,180][20381] Updated weights for policy 1, policy_version 518419 (0.0007) [2023-12-27 09:13:46,209][20347] Updated weights for policy 0, policy_version 518690 (0.0005) [2023-12-27 09:13:46,240][20381] Updated weights for policy 1, policy_version 518429 (0.0011) [2023-12-27 09:13:46,260][20347] Updated weights for policy 0, policy_version 518700 (0.0005) [2023-12-27 09:13:46,293][20381] Updated weights for policy 1, policy_version 518439 (0.0009) [2023-12-27 09:13:46,310][20347] Updated weights for policy 0, policy_version 518710 (0.0006) [2023-12-27 09:13:46,352][20347] Updated weights for policy 0, policy_version 518720 (0.0008) [2023-12-27 09:13:46,902][20381] Updated weights for policy 1, policy_version 518449 (0.0006) [2023-12-27 09:13:46,954][20381] Updated weights for policy 1, policy_version 518459 (0.0006) [2023-12-27 09:13:47,002][20381] Updated weights for policy 1, policy_version 518469 (0.0005) [2023-12-27 09:13:47,048][20381] Updated weights for policy 1, policy_version 518479 (0.0005) [2023-12-27 09:13:47,049][20347] Updated weights for policy 0, policy_version 518730 (0.0008) [2023-12-27 09:13:47,097][20347] Updated weights for policy 0, policy_version 518740 (0.0008) [2023-12-27 09:13:47,156][20347] Updated weights for policy 0, policy_version 518750 (0.0008) [2023-12-27 09:13:47,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.5, 300 sec: 20827.2). Total num frames: 265568256. Throughput: 0: 10508.7, 1: 10406.4. Samples: 265576132. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:47,363][19170] Avg episode reward: [(0, '341.240'), (1, '560.830')] [2023-12-27 09:13:47,646][20381] Updated weights for policy 1, policy_version 518489 (0.0010) [2023-12-27 09:13:47,698][20381] Updated weights for policy 1, policy_version 518499 (0.0007) [2023-12-27 09:13:47,752][20381] Updated weights for policy 1, policy_version 518509 (0.0005) [2023-12-27 09:13:47,883][20347] Updated weights for policy 0, policy_version 518760 (0.0010) [2023-12-27 09:13:47,941][20347] Updated weights for policy 0, policy_version 518770 (0.0007) [2023-12-27 09:13:48,002][20347] Updated weights for policy 0, policy_version 518780 (0.0008) [2023-12-27 09:13:48,320][20381] Updated weights for policy 1, policy_version 518519 (0.0007) [2023-12-27 09:13:48,382][20381] Updated weights for policy 1, policy_version 518529 (0.0007) [2023-12-27 09:13:48,437][20381] Updated weights for policy 1, policy_version 518539 (0.0007) [2023-12-27 09:13:48,661][20347] Updated weights for policy 0, policy_version 518790 (0.0010) [2023-12-27 09:13:48,709][20347] Updated weights for policy 0, policy_version 518800 (0.0010) [2023-12-27 09:13:48,762][20347] Updated weights for policy 0, policy_version 518810 (0.0010) [2023-12-27 09:13:49,142][20381] Updated weights for policy 1, policy_version 518549 (0.0008) [2023-12-27 09:13:49,200][20381] Updated weights for policy 1, policy_version 518559 (0.0008) [2023-12-27 09:13:49,267][20381] Updated weights for policy 1, policy_version 518569 (0.0008) [2023-12-27 09:13:49,453][20347] Updated weights for policy 0, policy_version 518820 (0.0009) [2023-12-27 09:13:49,506][20347] Updated weights for policy 0, policy_version 518830 (0.0008) [2023-12-27 09:13:49,557][20347] Updated weights for policy 0, policy_version 518840 (0.0008) [2023-12-27 09:13:49,928][20381] Updated weights for policy 1, policy_version 518579 (0.0008) [2023-12-27 09:13:49,987][20381] Updated weights for policy 1, policy_version 518589 (0.0008) [2023-12-27 09:13:50,038][20381] Updated weights for policy 1, policy_version 518599 (0.0008) [2023-12-27 09:13:50,309][20347] Updated weights for policy 0, policy_version 518850 (0.0009) [2023-12-27 09:13:50,360][20347] Updated weights for policy 0, policy_version 518860 (0.0008) [2023-12-27 09:13:50,421][20347] Updated weights for policy 0, policy_version 518870 (0.0008) [2023-12-27 09:13:50,484][20347] Updated weights for policy 0, policy_version 518880 (0.0009) [2023-12-27 09:13:50,746][20381] Updated weights for policy 1, policy_version 518609 (0.0008) [2023-12-27 09:13:50,802][20381] Updated weights for policy 1, policy_version 518619 (0.0009) [2023-12-27 09:13:50,859][20381] Updated weights for policy 1, policy_version 518629 (0.0007) [2023-12-27 09:13:50,910][20381] Updated weights for policy 1, policy_version 518639 (0.0007) [2023-12-27 09:13:51,133][20347] Updated weights for policy 0, policy_version 518890 (0.0008) [2023-12-27 09:13:51,194][20347] Updated weights for policy 0, policy_version 518900 (0.0009) [2023-12-27 09:13:51,249][20347] Updated weights for policy 0, policy_version 518910 (0.0007) [2023-12-27 09:13:51,665][20381] Updated weights for policy 1, policy_version 518649 (0.0006) [2023-12-27 09:13:51,723][20381] Updated weights for policy 1, policy_version 518659 (0.0008) [2023-12-27 09:13:51,786][20381] Updated weights for policy 1, policy_version 518669 (0.0008) [2023-12-27 09:13:51,929][20347] Updated weights for policy 0, policy_version 518920 (0.0006) [2023-12-27 09:13:51,992][20347] Updated weights for policy 0, policy_version 518930 (0.0008) [2023-12-27 09:13:52,054][20347] Updated weights for policy 0, policy_version 518940 (0.0007) [2023-12-27 09:13:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 265666560. Throughput: 0: 10488.1, 1: 10361.2. Samples: 265637368. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:52,363][19170] Avg episode reward: [(0, '358.540'), (1, '411.510')] [2023-12-27 09:13:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000518672_132800512.pth... [2023-12-27 09:13:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000518944_132866048.pth... [2023-12-27 09:13:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000517456_132489216.pth [2023-12-27 09:13:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000517696_132546560.pth [2023-12-27 09:13:52,492][20381] Updated weights for policy 1, policy_version 518679 (0.0008) [2023-12-27 09:13:52,548][20381] Updated weights for policy 1, policy_version 518689 (0.0007) [2023-12-27 09:13:52,607][20381] Updated weights for policy 1, policy_version 518699 (0.0006) [2023-12-27 09:13:52,662][20347] Updated weights for policy 0, policy_version 518950 (0.0008) [2023-12-27 09:13:52,712][20347] Updated weights for policy 0, policy_version 518960 (0.0009) [2023-12-27 09:13:52,772][20347] Updated weights for policy 0, policy_version 518970 (0.0007) [2023-12-27 09:13:53,147][20381] Updated weights for policy 1, policy_version 518709 (0.0005) [2023-12-27 09:13:53,203][20381] Updated weights for policy 1, policy_version 518719 (0.0005) [2023-12-27 09:13:53,249][20381] Updated weights for policy 1, policy_version 518729 (0.0005) [2023-12-27 09:13:53,353][20347] Updated weights for policy 0, policy_version 518980 (0.0005) [2023-12-27 09:13:53,416][20347] Updated weights for policy 0, policy_version 518990 (0.0005) [2023-12-27 09:13:53,475][20347] Updated weights for policy 0, policy_version 519000 (0.0007) [2023-12-27 09:13:53,864][20381] Updated weights for policy 1, policy_version 518739 (0.0007) [2023-12-27 09:13:53,924][20381] Updated weights for policy 1, policy_version 518749 (0.0008) [2023-12-27 09:13:53,974][20381] Updated weights for policy 1, policy_version 518759 (0.0007) [2023-12-27 09:13:54,065][20347] Updated weights for policy 0, policy_version 519010 (0.0007) [2023-12-27 09:13:54,120][20347] Updated weights for policy 0, policy_version 519020 (0.0008) [2023-12-27 09:13:54,165][20347] Updated weights for policy 0, policy_version 519030 (0.0007) [2023-12-27 09:13:54,218][20347] Updated weights for policy 0, policy_version 519040 (0.0009) [2023-12-27 09:13:54,651][20381] Updated weights for policy 1, policy_version 518769 (0.0008) [2023-12-27 09:13:54,701][20381] Updated weights for policy 1, policy_version 518779 (0.0010) [2023-12-27 09:13:54,761][20381] Updated weights for policy 1, policy_version 518789 (0.0008) [2023-12-27 09:13:54,819][20381] Updated weights for policy 1, policy_version 518799 (0.0008) [2023-12-27 09:13:54,892][20347] Updated weights for policy 0, policy_version 519050 (0.0008) [2023-12-27 09:13:54,946][20347] Updated weights for policy 0, policy_version 519060 (0.0008) [2023-12-27 09:13:55,011][20347] Updated weights for policy 0, policy_version 519070 (0.0010) [2023-12-27 09:13:55,496][20381] Updated weights for policy 1, policy_version 518809 (0.0008) [2023-12-27 09:13:55,557][20381] Updated weights for policy 1, policy_version 518819 (0.0007) [2023-12-27 09:13:55,608][20381] Updated weights for policy 1, policy_version 518829 (0.0007) [2023-12-27 09:13:55,677][20347] Updated weights for policy 0, policy_version 519080 (0.0007) [2023-12-27 09:13:55,728][20347] Updated weights for policy 0, policy_version 519090 (0.0007) [2023-12-27 09:13:55,779][20347] Updated weights for policy 0, policy_version 519100 (0.0007) [2023-12-27 09:13:56,308][20381] Updated weights for policy 1, policy_version 518839 (0.0011) [2023-12-27 09:13:56,364][20381] Updated weights for policy 1, policy_version 518849 (0.0010) [2023-12-27 09:13:56,413][20381] Updated weights for policy 1, policy_version 518859 (0.0010) [2023-12-27 09:13:56,510][20347] Updated weights for policy 0, policy_version 519110 (0.0007) [2023-12-27 09:13:56,562][20347] Updated weights for policy 0, policy_version 519120 (0.0007) [2023-12-27 09:13:56,613][20347] Updated weights for policy 0, policy_version 519130 (0.0007) [2023-12-27 09:13:57,114][20381] Updated weights for policy 1, policy_version 518869 (0.0010) [2023-12-27 09:13:57,169][20381] Updated weights for policy 1, policy_version 518879 (0.0010) [2023-12-27 09:13:57,224][20381] Updated weights for policy 1, policy_version 518889 (0.0010) [2023-12-27 09:13:57,306][20347] Updated weights for policy 0, policy_version 519140 (0.0007) [2023-12-27 09:13:57,359][20347] Updated weights for policy 0, policy_version 519150 (0.0007) [2023-12-27 09:13:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 265773056. Throughput: 0: 10590.3, 1: 10347.6. Samples: 265763536. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:13:57,363][19170] Avg episode reward: [(0, '395.780'), (1, '367.760')] [2023-12-27 09:13:57,412][20347] Updated weights for policy 0, policy_version 519160 (0.0007) [2023-12-27 09:13:57,897][20381] Updated weights for policy 1, policy_version 518899 (0.0010) [2023-12-27 09:13:57,942][20381] Updated weights for policy 1, policy_version 518909 (0.0010) [2023-12-27 09:13:57,993][20381] Updated weights for policy 1, policy_version 518919 (0.0010) [2023-12-27 09:13:58,103][20347] Updated weights for policy 0, policy_version 519170 (0.0007) [2023-12-27 09:13:58,157][20347] Updated weights for policy 0, policy_version 519180 (0.0008) [2023-12-27 09:13:58,224][20347] Updated weights for policy 0, policy_version 519190 (0.0007) [2023-12-27 09:13:58,279][20347] Updated weights for policy 0, policy_version 519200 (0.0007) [2023-12-27 09:13:58,727][20381] Updated weights for policy 1, policy_version 518929 (0.0010) [2023-12-27 09:13:58,786][20381] Updated weights for policy 1, policy_version 518939 (0.0008) [2023-12-27 09:13:58,849][20381] Updated weights for policy 1, policy_version 518949 (0.0008) [2023-12-27 09:13:58,904][20381] Updated weights for policy 1, policy_version 518959 (0.0006) [2023-12-27 09:13:58,983][20347] Updated weights for policy 0, policy_version 519210 (0.0008) [2023-12-27 09:13:59,026][20347] Updated weights for policy 0, policy_version 519220 (0.0007) [2023-12-27 09:13:59,091][20347] Updated weights for policy 0, policy_version 519230 (0.0010) [2023-12-27 09:13:59,512][20381] Updated weights for policy 1, policy_version 518969 (0.0005) [2023-12-27 09:13:59,558][20381] Updated weights for policy 1, policy_version 518979 (0.0005) [2023-12-27 09:13:59,614][20381] Updated weights for policy 1, policy_version 518989 (0.0006) [2023-12-27 09:13:59,804][20347] Updated weights for policy 0, policy_version 519240 (0.0010) [2023-12-27 09:13:59,858][20347] Updated weights for policy 0, policy_version 519250 (0.0010) [2023-12-27 09:13:59,902][20347] Updated weights for policy 0, policy_version 519260 (0.0008) [2023-12-27 09:14:00,334][20381] Updated weights for policy 1, policy_version 518999 (0.0007) [2023-12-27 09:14:00,392][20381] Updated weights for policy 1, policy_version 519009 (0.0007) [2023-12-27 09:14:00,445][20381] Updated weights for policy 1, policy_version 519019 (0.0007) [2023-12-27 09:14:00,595][20347] Updated weights for policy 0, policy_version 519270 (0.0008) [2023-12-27 09:14:00,646][20347] Updated weights for policy 0, policy_version 519280 (0.0010) [2023-12-27 09:14:00,693][20347] Updated weights for policy 0, policy_version 519290 (0.0010) [2023-12-27 09:14:01,115][20381] Updated weights for policy 1, policy_version 519029 (0.0007) [2023-12-27 09:14:01,176][20381] Updated weights for policy 1, policy_version 519039 (0.0008) [2023-12-27 09:14:01,236][20381] Updated weights for policy 1, policy_version 519049 (0.0008) [2023-12-27 09:14:01,375][20347] Updated weights for policy 0, policy_version 519300 (0.0008) [2023-12-27 09:14:01,431][20347] Updated weights for policy 0, policy_version 519310 (0.0008) [2023-12-27 09:14:01,487][20347] Updated weights for policy 0, policy_version 519320 (0.0005) [2023-12-27 09:14:01,917][20381] Updated weights for policy 1, policy_version 519059 (0.0008) [2023-12-27 09:14:01,963][20381] Updated weights for policy 1, policy_version 519069 (0.0007) [2023-12-27 09:14:02,009][20381] Updated weights for policy 1, policy_version 519079 (0.0008) [2023-12-27 09:14:02,164][20347] Updated weights for policy 0, policy_version 519330 (0.0006) [2023-12-27 09:14:02,209][20347] Updated weights for policy 0, policy_version 519340 (0.0008) [2023-12-27 09:14:02,263][20347] Updated weights for policy 0, policy_version 519350 (0.0007) [2023-12-27 09:14:02,323][20347] Updated weights for policy 0, policy_version 519360 (0.0007) [2023-12-27 09:14:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 265879552. Throughput: 0: 10530.9, 1: 10370.7. Samples: 265885052. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:14:02,363][19170] Avg episode reward: [(0, '442.080'), (1, '389.750')] [2023-12-27 09:14:02,649][20381] Updated weights for policy 1, policy_version 519089 (0.0007) [2023-12-27 09:14:02,707][20381] Updated weights for policy 1, policy_version 519099 (0.0006) [2023-12-27 09:14:02,766][20381] Updated weights for policy 1, policy_version 519109 (0.0006) [2023-12-27 09:14:02,824][20381] Updated weights for policy 1, policy_version 519119 (0.0005) [2023-12-27 09:14:03,057][20347] Updated weights for policy 0, policy_version 519370 (0.0008) [2023-12-27 09:14:03,105][20347] Updated weights for policy 0, policy_version 519380 (0.0007) [2023-12-27 09:14:03,156][20347] Updated weights for policy 0, policy_version 519390 (0.0007) [2023-12-27 09:14:03,444][20381] Updated weights for policy 1, policy_version 519129 (0.0008) [2023-12-27 09:14:03,501][20381] Updated weights for policy 1, policy_version 519139 (0.0007) [2023-12-27 09:14:03,555][20381] Updated weights for policy 1, policy_version 519149 (0.0008) [2023-12-27 09:14:03,847][20347] Updated weights for policy 0, policy_version 519400 (0.0008) [2023-12-27 09:14:03,913][20347] Updated weights for policy 0, policy_version 519410 (0.0007) [2023-12-27 09:14:03,975][20347] Updated weights for policy 0, policy_version 519420 (0.0008) [2023-12-27 09:14:04,244][20381] Updated weights for policy 1, policy_version 519159 (0.0010) [2023-12-27 09:14:04,301][20381] Updated weights for policy 1, policy_version 519169 (0.0011) [2023-12-27 09:14:04,368][20381] Updated weights for policy 1, policy_version 519179 (0.0010) [2023-12-27 09:14:04,748][20347] Updated weights for policy 0, policy_version 519430 (0.0009) [2023-12-27 09:14:04,803][20347] Updated weights for policy 0, policy_version 519440 (0.0007) [2023-12-27 09:14:04,860][20347] Updated weights for policy 0, policy_version 519450 (0.0007) [2023-12-27 09:14:05,055][20381] Updated weights for policy 1, policy_version 519189 (0.0009) [2023-12-27 09:14:05,110][20381] Updated weights for policy 1, policy_version 519199 (0.0008) [2023-12-27 09:14:05,161][20381] Updated weights for policy 1, policy_version 519209 (0.0007) [2023-12-27 09:14:05,521][20347] Updated weights for policy 0, policy_version 519460 (0.0008) [2023-12-27 09:14:05,569][20347] Updated weights for policy 0, policy_version 519470 (0.0010) [2023-12-27 09:14:05,613][20347] Updated weights for policy 0, policy_version 519480 (0.0010) [2023-12-27 09:14:05,826][20381] Updated weights for policy 1, policy_version 519219 (0.0007) [2023-12-27 09:14:05,884][20381] Updated weights for policy 1, policy_version 519229 (0.0010) [2023-12-27 09:14:05,938][20381] Updated weights for policy 1, policy_version 519239 (0.0010) [2023-12-27 09:14:06,326][20347] Updated weights for policy 0, policy_version 519490 (0.0010) [2023-12-27 09:14:06,395][20347] Updated weights for policy 0, policy_version 519500 (0.0008) [2023-12-27 09:14:06,454][20347] Updated weights for policy 0, policy_version 519510 (0.0007) [2023-12-27 09:14:06,523][20347] Updated weights for policy 0, policy_version 519520 (0.0008) [2023-12-27 09:14:06,628][20381] Updated weights for policy 1, policy_version 519249 (0.0010) [2023-12-27 09:14:06,687][20381] Updated weights for policy 1, policy_version 519259 (0.0010) [2023-12-27 09:14:06,737][20381] Updated weights for policy 1, policy_version 519269 (0.0011) [2023-12-27 09:14:06,786][20381] Updated weights for policy 1, policy_version 519279 (0.0010) [2023-12-27 09:14:07,206][20347] Updated weights for policy 0, policy_version 519530 (0.0007) [2023-12-27 09:14:07,268][20347] Updated weights for policy 0, policy_version 519540 (0.0007) [2023-12-27 09:14:07,330][20347] Updated weights for policy 0, policy_version 519550 (0.0008) [2023-12-27 09:14:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 265977856. Throughput: 0: 10472.9, 1: 10387.4. Samples: 265945704. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:14:07,363][19170] Avg episode reward: [(0, '428.740'), (1, '494.000')] [2023-12-27 09:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000519552_133021696.pth... [2023-12-27 09:14:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000519280_132956160.pth... [2023-12-27 09:14:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000518064_132644864.pth [2023-12-27 09:14:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000518336_132710400.pth [2023-12-27 09:14:07,493][20381] Updated weights for policy 1, policy_version 519289 (0.0011) [2023-12-27 09:14:07,545][20381] Updated weights for policy 1, policy_version 519299 (0.0010) [2023-12-27 09:14:07,600][20381] Updated weights for policy 1, policy_version 519309 (0.0011) [2023-12-27 09:14:07,949][20347] Updated weights for policy 0, policy_version 519560 (0.0007) [2023-12-27 09:14:07,998][20347] Updated weights for policy 0, policy_version 519570 (0.0007) [2023-12-27 09:14:08,049][20347] Updated weights for policy 0, policy_version 519580 (0.0006) [2023-12-27 09:14:08,215][20381] Updated weights for policy 1, policy_version 519319 (0.0009) [2023-12-27 09:14:08,279][20381] Updated weights for policy 1, policy_version 519329 (0.0008) [2023-12-27 09:14:08,349][20381] Updated weights for policy 1, policy_version 519339 (0.0006) [2023-12-27 09:14:08,724][20347] Updated weights for policy 0, policy_version 519590 (0.0007) [2023-12-27 09:14:08,784][20347] Updated weights for policy 0, policy_version 519600 (0.0007) [2023-12-27 09:14:08,849][20347] Updated weights for policy 0, policy_version 519610 (0.0007) [2023-12-27 09:14:09,012][20381] Updated weights for policy 1, policy_version 519349 (0.0010) [2023-12-27 09:14:09,065][20381] Updated weights for policy 1, policy_version 519359 (0.0007) [2023-12-27 09:14:09,115][20381] Updated weights for policy 1, policy_version 519369 (0.0005) [2023-12-27 09:14:09,554][20347] Updated weights for policy 0, policy_version 519620 (0.0007) [2023-12-27 09:14:09,609][20347] Updated weights for policy 0, policy_version 519630 (0.0008) [2023-12-27 09:14:09,663][20347] Updated weights for policy 0, policy_version 519640 (0.0010) [2023-12-27 09:14:09,794][20381] Updated weights for policy 1, policy_version 519379 (0.0007) [2023-12-27 09:14:09,863][20381] Updated weights for policy 1, policy_version 519389 (0.0010) [2023-12-27 09:14:09,925][20381] Updated weights for policy 1, policy_version 519399 (0.0009) [2023-12-27 09:14:10,359][20347] Updated weights for policy 0, policy_version 519650 (0.0008) [2023-12-27 09:14:10,411][20347] Updated weights for policy 0, policy_version 519660 (0.0008) [2023-12-27 09:14:10,461][20347] Updated weights for policy 0, policy_version 519670 (0.0009) [2023-12-27 09:14:10,520][20347] Updated weights for policy 0, policy_version 519680 (0.0009) [2023-12-27 09:14:10,597][20381] Updated weights for policy 1, policy_version 519409 (0.0009) [2023-12-27 09:14:10,660][20381] Updated weights for policy 1, policy_version 519419 (0.0008) [2023-12-27 09:14:10,721][20381] Updated weights for policy 1, policy_version 519429 (0.0007) [2023-12-27 09:14:10,780][20381] Updated weights for policy 1, policy_version 519439 (0.0007) [2023-12-27 09:14:11,264][20347] Updated weights for policy 0, policy_version 519690 (0.0006) [2023-12-27 09:14:11,328][20347] Updated weights for policy 0, policy_version 519700 (0.0006) [2023-12-27 09:14:11,399][20347] Updated weights for policy 0, policy_version 519710 (0.0008) [2023-12-27 09:14:11,475][20381] Updated weights for policy 1, policy_version 519449 (0.0008) [2023-12-27 09:14:11,531][20381] Updated weights for policy 1, policy_version 519459 (0.0008) [2023-12-27 09:14:11,594][20381] Updated weights for policy 1, policy_version 519469 (0.0008) [2023-12-27 09:14:12,086][20347] Updated weights for policy 0, policy_version 519720 (0.0008) [2023-12-27 09:14:12,152][20347] Updated weights for policy 0, policy_version 519730 (0.0008) [2023-12-27 09:14:12,205][20347] Updated weights for policy 0, policy_version 519740 (0.0009) [2023-12-27 09:14:12,309][20381] Updated weights for policy 1, policy_version 519479 (0.0007) [2023-12-27 09:14:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 266076160. Throughput: 0: 10465.2, 1: 10408.8. Samples: 266068236. Policy #0 lag: (min: 31.0, avg: 47.6, max: 63.0) [2023-12-27 09:14:12,363][19170] Avg episode reward: [(0, '433.280'), (1, '483.400')] [2023-12-27 09:14:12,364][20381] Updated weights for policy 1, policy_version 519489 (0.0007) [2023-12-27 09:14:12,423][20381] Updated weights for policy 1, policy_version 519499 (0.0007) [2023-12-27 09:14:12,909][20347] Updated weights for policy 0, policy_version 519750 (0.0010) [2023-12-27 09:14:12,957][20347] Updated weights for policy 0, policy_version 519760 (0.0010) [2023-12-27 09:14:13,002][20347] Updated weights for policy 0, policy_version 519770 (0.0010) [2023-12-27 09:14:13,063][20381] Updated weights for policy 1, policy_version 519509 (0.0008) [2023-12-27 09:14:13,111][20381] Updated weights for policy 1, policy_version 519519 (0.0007) [2023-12-27 09:14:13,158][20381] Updated weights for policy 1, policy_version 519529 (0.0007) [2023-12-27 09:14:13,646][20347] Updated weights for policy 0, policy_version 519780 (0.0010) [2023-12-27 09:14:13,700][20347] Updated weights for policy 0, policy_version 519790 (0.0006) [2023-12-27 09:14:13,757][20381] Updated weights for policy 1, policy_version 519539 (0.0006) [2023-12-27 09:14:13,757][20347] Updated weights for policy 0, policy_version 519800 (0.0005) [2023-12-27 09:14:13,810][20381] Updated weights for policy 1, policy_version 519549 (0.0009) [2023-12-27 09:14:13,862][20381] Updated weights for policy 1, policy_version 519559 (0.0009) [2023-12-27 09:14:14,389][20347] Updated weights for policy 0, policy_version 519810 (0.0010) [2023-12-27 09:14:14,437][20347] Updated weights for policy 0, policy_version 519820 (0.0010) [2023-12-27 09:14:14,483][20381] Updated weights for policy 1, policy_version 519569 (0.0010) [2023-12-27 09:14:14,495][20347] Updated weights for policy 0, policy_version 519830 (0.0010) [2023-12-27 09:14:14,545][20381] Updated weights for policy 1, policy_version 519579 (0.0006) [2023-12-27 09:14:14,549][20347] Updated weights for policy 0, policy_version 519840 (0.0010) [2023-12-27 09:14:14,608][20381] Updated weights for policy 1, policy_version 519589 (0.0005) [2023-12-27 09:14:14,665][20381] Updated weights for policy 1, policy_version 519599 (0.0007) [2023-12-27 09:14:15,243][20347] Updated weights for policy 0, policy_version 519850 (0.0011) [2023-12-27 09:14:15,308][20347] Updated weights for policy 0, policy_version 519860 (0.0011) [2023-12-27 09:14:15,321][20381] Updated weights for policy 1, policy_version 519609 (0.0007) [2023-12-27 09:14:15,370][20347] Updated weights for policy 0, policy_version 519870 (0.0007) [2023-12-27 09:14:15,375][20381] Updated weights for policy 1, policy_version 519619 (0.0008) [2023-12-27 09:14:15,438][20381] Updated weights for policy 1, policy_version 519629 (0.0008) [2023-12-27 09:14:15,964][20347] Updated weights for policy 0, policy_version 519880 (0.0007) [2023-12-27 09:14:16,017][20347] Updated weights for policy 0, policy_version 519890 (0.0006) [2023-12-27 09:14:16,064][20347] Updated weights for policy 0, policy_version 519900 (0.0005) [2023-12-27 09:14:16,157][20381] Updated weights for policy 1, policy_version 519639 (0.0008) [2023-12-27 09:14:16,216][20381] Updated weights for policy 1, policy_version 519649 (0.0008) [2023-12-27 09:14:16,263][20381] Updated weights for policy 1, policy_version 519659 (0.0008) [2023-12-27 09:14:16,770][20347] Updated weights for policy 0, policy_version 519910 (0.0008) [2023-12-27 09:14:16,830][20347] Updated weights for policy 0, policy_version 519920 (0.0006) [2023-12-27 09:14:16,875][20381] Updated weights for policy 1, policy_version 519669 (0.0007) [2023-12-27 09:14:16,892][20347] Updated weights for policy 0, policy_version 519930 (0.0006) [2023-12-27 09:14:16,937][20381] Updated weights for policy 1, policy_version 519679 (0.0006) [2023-12-27 09:14:16,996][20381] Updated weights for policy 1, policy_version 519689 (0.0007) [2023-12-27 09:14:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 266182656. Throughput: 0: 10365.9, 1: 10466.8. Samples: 266193892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:17,363][19170] Avg episode reward: [(0, '402.250'), (1, '444.770')] [2023-12-27 09:14:17,560][20347] Updated weights for policy 0, policy_version 519940 (0.0007) [2023-12-27 09:14:17,614][20381] Updated weights for policy 1, policy_version 519699 (0.0006) [2023-12-27 09:14:17,615][20347] Updated weights for policy 0, policy_version 519950 (0.0008) [2023-12-27 09:14:17,669][20381] Updated weights for policy 1, policy_version 519709 (0.0006) [2023-12-27 09:14:17,671][20347] Updated weights for policy 0, policy_version 519960 (0.0007) [2023-12-27 09:14:17,718][20381] Updated weights for policy 1, policy_version 519719 (0.0006) [2023-12-27 09:14:18,390][20381] Updated weights for policy 1, policy_version 519729 (0.0007) [2023-12-27 09:14:18,392][20347] Updated weights for policy 0, policy_version 519970 (0.0006) [2023-12-27 09:14:18,440][20381] Updated weights for policy 1, policy_version 519739 (0.0005) [2023-12-27 09:14:18,449][20347] Updated weights for policy 0, policy_version 519980 (0.0006) [2023-12-27 09:14:18,492][20381] Updated weights for policy 1, policy_version 519749 (0.0006) [2023-12-27 09:14:18,501][20347] Updated weights for policy 0, policy_version 519990 (0.0007) [2023-12-27 09:14:18,551][20381] Updated weights for policy 1, policy_version 519759 (0.0007) [2023-12-27 09:14:18,557][20347] Updated weights for policy 0, policy_version 520000 (0.0007) [2023-12-27 09:14:19,112][20381] Updated weights for policy 1, policy_version 519769 (0.0005) [2023-12-27 09:14:19,170][20381] Updated weights for policy 1, policy_version 519779 (0.0007) [2023-12-27 09:14:19,225][20381] Updated weights for policy 1, policy_version 519789 (0.0009) [2023-12-27 09:14:19,321][20347] Updated weights for policy 0, policy_version 520010 (0.0008) [2023-12-27 09:14:19,378][20347] Updated weights for policy 0, policy_version 520020 (0.0007) [2023-12-27 09:14:19,435][20347] Updated weights for policy 0, policy_version 520030 (0.0005) [2023-12-27 09:14:19,914][20381] Updated weights for policy 1, policy_version 519799 (0.0008) [2023-12-27 09:14:19,977][20381] Updated weights for policy 1, policy_version 519809 (0.0008) [2023-12-27 09:14:20,030][20381] Updated weights for policy 1, policy_version 519819 (0.0006) [2023-12-27 09:14:20,037][20347] Updated weights for policy 0, policy_version 520040 (0.0006) [2023-12-27 09:14:20,094][20347] Updated weights for policy 0, policy_version 520050 (0.0006) [2023-12-27 09:14:20,161][20347] Updated weights for policy 0, policy_version 520060 (0.0008) [2023-12-27 09:14:20,735][20381] Updated weights for policy 1, policy_version 519829 (0.0008) [2023-12-27 09:14:20,793][20381] Updated weights for policy 1, policy_version 519839 (0.0006) [2023-12-27 09:14:20,818][20347] Updated weights for policy 0, policy_version 520070 (0.0008) [2023-12-27 09:14:20,852][20381] Updated weights for policy 1, policy_version 519849 (0.0006) [2023-12-27 09:14:20,888][20347] Updated weights for policy 0, policy_version 520080 (0.0008) [2023-12-27 09:14:20,952][20347] Updated weights for policy 0, policy_version 520090 (0.0007) [2023-12-27 09:14:21,428][20381] Updated weights for policy 1, policy_version 519859 (0.0006) [2023-12-27 09:14:21,492][20381] Updated weights for policy 1, policy_version 519869 (0.0006) [2023-12-27 09:14:21,551][20381] Updated weights for policy 1, policy_version 519879 (0.0007) [2023-12-27 09:14:21,700][20347] Updated weights for policy 0, policy_version 520100 (0.0009) [2023-12-27 09:14:21,769][20347] Updated weights for policy 0, policy_version 520110 (0.0007) [2023-12-27 09:14:21,827][20347] Updated weights for policy 0, policy_version 520120 (0.0007) [2023-12-27 09:14:22,241][20381] Updated weights for policy 1, policy_version 519889 (0.0009) [2023-12-27 09:14:22,305][20381] Updated weights for policy 1, policy_version 519899 (0.0011) [2023-12-27 09:14:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 266280960. Throughput: 0: 10306.3, 1: 10471.4. Samples: 266255800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:22,363][19170] Avg episode reward: [(0, '369.370'), (1, '635.180')] [2023-12-27 09:14:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000520128_133169152.pth... [2023-12-27 09:14:22,373][20381] Updated weights for policy 1, policy_version 519909 (0.0011) [2023-12-27 09:14:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000518944_132866048.pth [2023-12-27 09:14:22,438][20381] Updated weights for policy 1, policy_version 519919 (0.0007) [2023-12-27 09:14:22,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000519920_133120000.pth... [2023-12-27 09:14:22,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000518672_132800512.pth [2023-12-27 09:14:22,539][20347] Updated weights for policy 0, policy_version 520130 (0.0008) [2023-12-27 09:14:22,600][20347] Updated weights for policy 0, policy_version 520140 (0.0007) [2023-12-27 09:14:22,648][20347] Updated weights for policy 0, policy_version 520150 (0.0007) [2023-12-27 09:14:22,697][20347] Updated weights for policy 0, policy_version 520160 (0.0008) [2023-12-27 09:14:23,098][20381] Updated weights for policy 1, policy_version 519929 (0.0006) [2023-12-27 09:14:23,144][20381] Updated weights for policy 1, policy_version 519939 (0.0005) [2023-12-27 09:14:23,201][20381] Updated weights for policy 1, policy_version 519949 (0.0006) [2023-12-27 09:14:23,345][20347] Updated weights for policy 0, policy_version 520170 (0.0007) [2023-12-27 09:14:23,393][20347] Updated weights for policy 0, policy_version 520180 (0.0007) [2023-12-27 09:14:23,444][20347] Updated weights for policy 0, policy_version 520190 (0.0007) [2023-12-27 09:14:23,822][20381] Updated weights for policy 1, policy_version 519959 (0.0010) [2023-12-27 09:14:23,867][20381] Updated weights for policy 1, policy_version 519969 (0.0010) [2023-12-27 09:14:23,918][20381] Updated weights for policy 1, policy_version 519979 (0.0010) [2023-12-27 09:14:24,092][20347] Updated weights for policy 0, policy_version 520200 (0.0010) [2023-12-27 09:14:24,147][20347] Updated weights for policy 0, policy_version 520210 (0.0006) [2023-12-27 09:14:24,203][20347] Updated weights for policy 0, policy_version 520220 (0.0005) [2023-12-27 09:14:24,603][20381] Updated weights for policy 1, policy_version 519989 (0.0008) [2023-12-27 09:14:24,654][20381] Updated weights for policy 1, policy_version 519999 (0.0005) [2023-12-27 09:14:24,705][20381] Updated weights for policy 1, policy_version 520009 (0.0007) [2023-12-27 09:14:24,780][20347] Updated weights for policy 0, policy_version 520230 (0.0006) [2023-12-27 09:14:24,829][20347] Updated weights for policy 0, policy_version 520240 (0.0006) [2023-12-27 09:14:24,887][20347] Updated weights for policy 0, policy_version 520250 (0.0009) [2023-12-27 09:14:25,374][20381] Updated weights for policy 1, policy_version 520019 (0.0010) [2023-12-27 09:14:25,425][20381] Updated weights for policy 1, policy_version 520029 (0.0010) [2023-12-27 09:14:25,480][20381] Updated weights for policy 1, policy_version 520039 (0.0011) [2023-12-27 09:14:25,528][20347] Updated weights for policy 0, policy_version 520260 (0.0007) [2023-12-27 09:14:25,582][20347] Updated weights for policy 0, policy_version 520270 (0.0010) [2023-12-27 09:14:25,633][20347] Updated weights for policy 0, policy_version 520280 (0.0010) [2023-12-27 09:14:26,162][20381] Updated weights for policy 1, policy_version 520049 (0.0011) [2023-12-27 09:14:26,221][20381] Updated weights for policy 1, policy_version 520059 (0.0010) [2023-12-27 09:14:26,288][20381] Updated weights for policy 1, policy_version 520069 (0.0010) [2023-12-27 09:14:26,290][20347] Updated weights for policy 0, policy_version 520290 (0.0009) [2023-12-27 09:14:26,336][20347] Updated weights for policy 0, policy_version 520300 (0.0006) [2023-12-27 09:14:26,340][20381] Updated weights for policy 1, policy_version 520079 (0.0010) [2023-12-27 09:14:26,390][20347] Updated weights for policy 0, policy_version 520310 (0.0009) [2023-12-27 09:14:26,454][20347] Updated weights for policy 0, policy_version 520320 (0.0006) [2023-12-27 09:14:26,947][20381] Updated weights for policy 1, policy_version 520089 (0.0010) [2023-12-27 09:14:26,999][20381] Updated weights for policy 1, policy_version 520099 (0.0008) [2023-12-27 09:14:27,001][20347] Updated weights for policy 0, policy_version 520330 (0.0006) [2023-12-27 09:14:27,052][20381] Updated weights for policy 1, policy_version 520109 (0.0005) [2023-12-27 09:14:27,056][20347] Updated weights for policy 0, policy_version 520340 (0.0008) [2023-12-27 09:14:27,120][20347] Updated weights for policy 0, policy_version 520350 (0.0007) [2023-12-27 09:14:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 266395648. Throughput: 0: 10290.1, 1: 10445.1. Samples: 266381020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:27,363][19170] Avg episode reward: [(0, '417.030'), (1, '673.010')] [2023-12-27 09:14:27,694][20381] Updated weights for policy 1, policy_version 520119 (0.0009) [2023-12-27 09:14:27,735][20347] Updated weights for policy 0, policy_version 520360 (0.0008) [2023-12-27 09:14:27,749][20381] Updated weights for policy 1, policy_version 520129 (0.0011) [2023-12-27 09:14:27,786][20347] Updated weights for policy 0, policy_version 520370 (0.0007) [2023-12-27 09:14:27,803][20381] Updated weights for policy 1, policy_version 520139 (0.0010) [2023-12-27 09:14:27,846][20347] Updated weights for policy 0, policy_version 520380 (0.0006) [2023-12-27 09:14:28,451][20381] Updated weights for policy 1, policy_version 520149 (0.0011) [2023-12-27 09:14:28,513][20381] Updated weights for policy 1, policy_version 520159 (0.0009) [2023-12-27 09:14:28,525][20347] Updated weights for policy 0, policy_version 520390 (0.0008) [2023-12-27 09:14:28,572][20381] Updated weights for policy 1, policy_version 520169 (0.0010) [2023-12-27 09:14:28,584][20347] Updated weights for policy 0, policy_version 520400 (0.0007) [2023-12-27 09:14:28,636][20347] Updated weights for policy 0, policy_version 520410 (0.0007) [2023-12-27 09:14:29,246][20381] Updated weights for policy 1, policy_version 520179 (0.0011) [2023-12-27 09:14:29,301][20381] Updated weights for policy 1, policy_version 520189 (0.0010) [2023-12-27 09:14:29,330][20347] Updated weights for policy 0, policy_version 520420 (0.0008) [2023-12-27 09:14:29,367][20381] Updated weights for policy 1, policy_version 520199 (0.0011) [2023-12-27 09:14:29,392][20347] Updated weights for policy 0, policy_version 520430 (0.0006) [2023-12-27 09:14:29,450][20347] Updated weights for policy 0, policy_version 520440 (0.0005) [2023-12-27 09:14:30,043][20381] Updated weights for policy 1, policy_version 520209 (0.0009) [2023-12-27 09:14:30,095][20381] Updated weights for policy 1, policy_version 520219 (0.0008) [2023-12-27 09:14:30,106][20347] Updated weights for policy 0, policy_version 520450 (0.0007) [2023-12-27 09:14:30,150][20381] Updated weights for policy 1, policy_version 520229 (0.0006) [2023-12-27 09:14:30,163][20347] Updated weights for policy 0, policy_version 520460 (0.0006) [2023-12-27 09:14:30,209][20347] Updated weights for policy 0, policy_version 520470 (0.0007) [2023-12-27 09:14:30,214][20381] Updated weights for policy 1, policy_version 520239 (0.0005) [2023-12-27 09:14:30,256][20347] Updated weights for policy 0, policy_version 520480 (0.0008) [2023-12-27 09:14:30,864][20381] Updated weights for policy 1, policy_version 520249 (0.0006) [2023-12-27 09:14:30,911][20347] Updated weights for policy 0, policy_version 520490 (0.0005) [2023-12-27 09:14:30,918][20381] Updated weights for policy 1, policy_version 520259 (0.0008) [2023-12-27 09:14:30,962][20347] Updated weights for policy 0, policy_version 520500 (0.0005) [2023-12-27 09:14:30,975][20381] Updated weights for policy 1, policy_version 520269 (0.0007) [2023-12-27 09:14:31,012][20347] Updated weights for policy 0, policy_version 520510 (0.0005) [2023-12-27 09:14:31,662][20347] Updated weights for policy 0, policy_version 520520 (0.0007) [2023-12-27 09:14:31,715][20347] Updated weights for policy 0, policy_version 520530 (0.0007) [2023-12-27 09:14:31,721][20381] Updated weights for policy 1, policy_version 520279 (0.0007) [2023-12-27 09:14:31,771][20381] Updated weights for policy 1, policy_version 520289 (0.0008) [2023-12-27 09:14:31,772][20347] Updated weights for policy 0, policy_version 520540 (0.0007) [2023-12-27 09:14:31,825][20381] Updated weights for policy 1, policy_version 520299 (0.0007) [2023-12-27 09:14:32,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 266493952. Throughput: 0: 10288.2, 1: 10397.0. Samples: 266506972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:32,363][19170] Avg episode reward: [(0, '529.980'), (1, '664.620')] [2023-12-27 09:14:32,412][20347] Updated weights for policy 0, policy_version 520550 (0.0008) [2023-12-27 09:14:32,471][20347] Updated weights for policy 0, policy_version 520560 (0.0008) [2023-12-27 09:14:32,525][20381] Updated weights for policy 1, policy_version 520309 (0.0007) [2023-12-27 09:14:32,528][20347] Updated weights for policy 0, policy_version 520570 (0.0008) [2023-12-27 09:14:32,581][20381] Updated weights for policy 1, policy_version 520319 (0.0005) [2023-12-27 09:14:32,634][20381] Updated weights for policy 1, policy_version 520329 (0.0005) [2023-12-27 09:14:33,235][20381] Updated weights for policy 1, policy_version 520339 (0.0005) [2023-12-27 09:14:33,252][20347] Updated weights for policy 0, policy_version 520580 (0.0006) [2023-12-27 09:14:33,287][20381] Updated weights for policy 1, policy_version 520349 (0.0007) [2023-12-27 09:14:33,305][20347] Updated weights for policy 0, policy_version 520590 (0.0005) [2023-12-27 09:14:33,339][20381] Updated weights for policy 1, policy_version 520359 (0.0006) [2023-12-27 09:14:33,352][20347] Updated weights for policy 0, policy_version 520600 (0.0005) [2023-12-27 09:14:33,888][20381] Updated weights for policy 1, policy_version 520369 (0.0006) [2023-12-27 09:14:33,951][20381] Updated weights for policy 1, policy_version 520379 (0.0008) [2023-12-27 09:14:34,004][20381] Updated weights for policy 1, policy_version 520389 (0.0007) [2023-12-27 09:14:34,017][20347] Updated weights for policy 0, policy_version 520610 (0.0006) [2023-12-27 09:14:34,059][20381] Updated weights for policy 1, policy_version 520399 (0.0005) [2023-12-27 09:14:34,066][20347] Updated weights for policy 0, policy_version 520620 (0.0010) [2023-12-27 09:14:34,131][20347] Updated weights for policy 0, policy_version 520630 (0.0009) [2023-12-27 09:14:34,188][20347] Updated weights for policy 0, policy_version 520640 (0.0010) [2023-12-27 09:14:34,675][20381] Updated weights for policy 1, policy_version 520409 (0.0006) [2023-12-27 09:14:34,724][20381] Updated weights for policy 1, policy_version 520419 (0.0006) [2023-12-27 09:14:34,782][20381] Updated weights for policy 1, policy_version 520429 (0.0007) [2023-12-27 09:14:34,895][20347] Updated weights for policy 0, policy_version 520650 (0.0011) [2023-12-27 09:14:34,955][20347] Updated weights for policy 0, policy_version 520660 (0.0011) [2023-12-27 09:14:35,022][20347] Updated weights for policy 0, policy_version 520670 (0.0010) [2023-12-27 09:14:35,416][20381] Updated weights for policy 1, policy_version 520439 (0.0006) [2023-12-27 09:14:35,469][20381] Updated weights for policy 1, policy_version 520449 (0.0005) [2023-12-27 09:14:35,527][20381] Updated weights for policy 1, policy_version 520459 (0.0005) [2023-12-27 09:14:35,654][20347] Updated weights for policy 0, policy_version 520680 (0.0010) [2023-12-27 09:14:35,716][20347] Updated weights for policy 0, policy_version 520690 (0.0010) [2023-12-27 09:14:35,781][20347] Updated weights for policy 0, policy_version 520700 (0.0010) [2023-12-27 09:14:36,147][20381] Updated weights for policy 1, policy_version 520469 (0.0006) [2023-12-27 09:14:36,207][20381] Updated weights for policy 1, policy_version 520479 (0.0007) [2023-12-27 09:14:36,271][20381] Updated weights for policy 1, policy_version 520489 (0.0009) [2023-12-27 09:14:36,446][20347] Updated weights for policy 0, policy_version 520710 (0.0010) [2023-12-27 09:14:36,502][20347] Updated weights for policy 0, policy_version 520720 (0.0011) [2023-12-27 09:14:36,553][20347] Updated weights for policy 0, policy_version 520730 (0.0010) [2023-12-27 09:14:36,981][20381] Updated weights for policy 1, policy_version 520499 (0.0007) [2023-12-27 09:14:37,042][20381] Updated weights for policy 1, policy_version 520509 (0.0006) [2023-12-27 09:14:37,100][20381] Updated weights for policy 1, policy_version 520519 (0.0005) [2023-12-27 09:14:37,247][20347] Updated weights for policy 0, policy_version 520740 (0.0010) [2023-12-27 09:14:37,305][20347] Updated weights for policy 0, policy_version 520750 (0.0010) [2023-12-27 09:14:37,361][20347] Updated weights for policy 0, policy_version 520760 (0.0010) [2023-12-27 09:14:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 266600448. Throughput: 0: 10300.4, 1: 10445.5. Samples: 266570932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:37,363][19170] Avg episode reward: [(0, '531.580'), (1, '584.490')] [2023-12-27 09:14:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000520528_133275648.pth... [2023-12-27 09:14:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000519280_132956160.pth [2023-12-27 09:14:37,399][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000520768_133332992.pth... [2023-12-27 09:14:37,402][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000519552_133021696.pth [2023-12-27 09:14:37,759][20381] Updated weights for policy 1, policy_version 520529 (0.0008) [2023-12-27 09:14:37,808][20381] Updated weights for policy 1, policy_version 520539 (0.0005) [2023-12-27 09:14:37,857][20381] Updated weights for policy 1, policy_version 520549 (0.0007) [2023-12-27 09:14:37,901][20381] Updated weights for policy 1, policy_version 520559 (0.0010) [2023-12-27 09:14:38,000][20347] Updated weights for policy 0, policy_version 520770 (0.0011) [2023-12-27 09:14:38,061][20347] Updated weights for policy 0, policy_version 520780 (0.0008) [2023-12-27 09:14:38,126][20347] Updated weights for policy 0, policy_version 520790 (0.0008) [2023-12-27 09:14:38,192][20347] Updated weights for policy 0, policy_version 520800 (0.0007) [2023-12-27 09:14:38,577][20381] Updated weights for policy 1, policy_version 520569 (0.0009) [2023-12-27 09:14:38,634][20381] Updated weights for policy 1, policy_version 520579 (0.0011) [2023-12-27 09:14:38,690][20381] Updated weights for policy 1, policy_version 520589 (0.0011) [2023-12-27 09:14:38,831][20347] Updated weights for policy 0, policy_version 520810 (0.0010) [2023-12-27 09:14:38,895][20347] Updated weights for policy 0, policy_version 520820 (0.0010) [2023-12-27 09:14:38,946][20347] Updated weights for policy 0, policy_version 520830 (0.0010) [2023-12-27 09:14:39,378][20381] Updated weights for policy 1, policy_version 520599 (0.0009) [2023-12-27 09:14:39,445][20381] Updated weights for policy 1, policy_version 520609 (0.0008) [2023-12-27 09:14:39,509][20381] Updated weights for policy 1, policy_version 520619 (0.0011) [2023-12-27 09:14:39,641][20347] Updated weights for policy 0, policy_version 520840 (0.0010) [2023-12-27 09:14:39,704][20347] Updated weights for policy 0, policy_version 520850 (0.0010) [2023-12-27 09:14:39,765][20347] Updated weights for policy 0, policy_version 520860 (0.0011) [2023-12-27 09:14:40,172][20381] Updated weights for policy 1, policy_version 520629 (0.0011) [2023-12-27 09:14:40,227][20381] Updated weights for policy 1, policy_version 520639 (0.0011) [2023-12-27 09:14:40,287][20381] Updated weights for policy 1, policy_version 520649 (0.0011) [2023-12-27 09:14:40,521][20347] Updated weights for policy 0, policy_version 520870 (0.0009) [2023-12-27 09:14:40,576][20347] Updated weights for policy 0, policy_version 520880 (0.0007) [2023-12-27 09:14:40,639][20347] Updated weights for policy 0, policy_version 520890 (0.0007) [2023-12-27 09:14:40,937][20381] Updated weights for policy 1, policy_version 520659 (0.0010) [2023-12-27 09:14:40,996][20381] Updated weights for policy 1, policy_version 520669 (0.0007) [2023-12-27 09:14:41,063][20381] Updated weights for policy 1, policy_version 520679 (0.0008) [2023-12-27 09:14:41,335][20347] Updated weights for policy 0, policy_version 520900 (0.0008) [2023-12-27 09:14:41,398][20347] Updated weights for policy 0, policy_version 520910 (0.0009) [2023-12-27 09:14:41,450][20347] Updated weights for policy 0, policy_version 520920 (0.0008) [2023-12-27 09:14:41,772][20381] Updated weights for policy 1, policy_version 520689 (0.0008) [2023-12-27 09:14:41,836][20381] Updated weights for policy 1, policy_version 520699 (0.0007) [2023-12-27 09:14:41,896][20381] Updated weights for policy 1, policy_version 520709 (0.0008) [2023-12-27 09:14:41,952][20381] Updated weights for policy 1, policy_version 520719 (0.0008) [2023-12-27 09:14:42,122][20347] Updated weights for policy 0, policy_version 520930 (0.0006) [2023-12-27 09:14:42,187][20347] Updated weights for policy 0, policy_version 520940 (0.0006) [2023-12-27 09:14:42,250][20347] Updated weights for policy 0, policy_version 520950 (0.0006) [2023-12-27 09:14:42,304][20347] Updated weights for policy 0, policy_version 520960 (0.0009) [2023-12-27 09:14:42,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 266706944. Throughput: 0: 10240.8, 1: 10435.1. Samples: 266693952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:42,363][19170] Avg episode reward: [(0, '608.450'), (1, '728.580')] [2023-12-27 09:14:42,604][20381] Updated weights for policy 1, policy_version 520729 (0.0007) [2023-12-27 09:14:42,668][20381] Updated weights for policy 1, policy_version 520739 (0.0008) [2023-12-27 09:14:42,729][20381] Updated weights for policy 1, policy_version 520749 (0.0006) [2023-12-27 09:14:42,856][20347] Updated weights for policy 0, policy_version 520970 (0.0006) [2023-12-27 09:14:42,907][20347] Updated weights for policy 0, policy_version 520980 (0.0007) [2023-12-27 09:14:42,956][20347] Updated weights for policy 0, policy_version 520990 (0.0008) [2023-12-27 09:14:43,403][20381] Updated weights for policy 1, policy_version 520759 (0.0008) [2023-12-27 09:14:43,458][20381] Updated weights for policy 1, policy_version 520769 (0.0008) [2023-12-27 09:14:43,515][20381] Updated weights for policy 1, policy_version 520779 (0.0007) [2023-12-27 09:14:43,670][20347] Updated weights for policy 0, policy_version 521000 (0.0007) [2023-12-27 09:14:43,719][20347] Updated weights for policy 0, policy_version 521010 (0.0007) [2023-12-27 09:14:43,768][20347] Updated weights for policy 0, policy_version 521020 (0.0008) [2023-12-27 09:14:44,147][20381] Updated weights for policy 1, policy_version 520789 (0.0008) [2023-12-27 09:14:44,204][20381] Updated weights for policy 1, policy_version 520799 (0.0010) [2023-12-27 09:14:44,259][20381] Updated weights for policy 1, policy_version 520809 (0.0010) [2023-12-27 09:14:44,516][20347] Updated weights for policy 0, policy_version 521030 (0.0008) [2023-12-27 09:14:44,568][20347] Updated weights for policy 0, policy_version 521040 (0.0008) [2023-12-27 09:14:44,623][20347] Updated weights for policy 0, policy_version 521050 (0.0005) [2023-12-27 09:14:44,848][20381] Updated weights for policy 1, policy_version 520819 (0.0010) [2023-12-27 09:14:44,916][20381] Updated weights for policy 1, policy_version 520829 (0.0006) [2023-12-27 09:14:44,983][20381] Updated weights for policy 1, policy_version 520839 (0.0008) [2023-12-27 09:14:45,295][20347] Updated weights for policy 0, policy_version 521060 (0.0007) [2023-12-27 09:14:45,350][20347] Updated weights for policy 0, policy_version 521070 (0.0008) [2023-12-27 09:14:45,408][20347] Updated weights for policy 0, policy_version 521080 (0.0007) [2023-12-27 09:14:45,577][20381] Updated weights for policy 1, policy_version 520849 (0.0007) [2023-12-27 09:14:45,638][20381] Updated weights for policy 1, policy_version 520859 (0.0011) [2023-12-27 09:14:45,686][20381] Updated weights for policy 1, policy_version 520869 (0.0010) [2023-12-27 09:14:45,738][20381] Updated weights for policy 1, policy_version 520879 (0.0010) [2023-12-27 09:14:46,057][20347] Updated weights for policy 0, policy_version 521090 (0.0007) [2023-12-27 09:14:46,115][20347] Updated weights for policy 0, policy_version 521100 (0.0005) [2023-12-27 09:14:46,181][20347] Updated weights for policy 0, policy_version 521110 (0.0005) [2023-12-27 09:14:46,230][20347] Updated weights for policy 0, policy_version 521120 (0.0005) [2023-12-27 09:14:46,437][20381] Updated weights for policy 1, policy_version 520889 (0.0010) [2023-12-27 09:14:46,492][20381] Updated weights for policy 1, policy_version 520899 (0.0010) [2023-12-27 09:14:46,546][20381] Updated weights for policy 1, policy_version 520909 (0.0006) [2023-12-27 09:14:46,830][20347] Updated weights for policy 0, policy_version 521130 (0.0005) [2023-12-27 09:14:46,885][20347] Updated weights for policy 0, policy_version 521140 (0.0005) [2023-12-27 09:14:46,934][20347] Updated weights for policy 0, policy_version 521150 (0.0005) [2023-12-27 09:14:47,199][20381] Updated weights for policy 1, policy_version 520919 (0.0007) [2023-12-27 09:14:47,260][20381] Updated weights for policy 1, policy_version 520929 (0.0005) [2023-12-27 09:14:47,309][20381] Updated weights for policy 1, policy_version 520939 (0.0005) [2023-12-27 09:14:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 266813440. Throughput: 0: 10290.0, 1: 10496.3. Samples: 266820440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:47,363][19170] Avg episode reward: [(0, '844.750'), (1, '661.250')] [2023-12-27 09:14:47,511][20347] Updated weights for policy 0, policy_version 521160 (0.0005) [2023-12-27 09:14:47,565][20347] Updated weights for policy 0, policy_version 521170 (0.0005) [2023-12-27 09:14:47,628][20347] Updated weights for policy 0, policy_version 521180 (0.0006) [2023-12-27 09:14:47,974][20381] Updated weights for policy 1, policy_version 520949 (0.0009) [2023-12-27 09:14:48,031][20381] Updated weights for policy 1, policy_version 520959 (0.0008) [2023-12-27 09:14:48,079][20381] Updated weights for policy 1, policy_version 520969 (0.0007) [2023-12-27 09:14:48,180][20347] Updated weights for policy 0, policy_version 521190 (0.0006) [2023-12-27 09:14:48,237][20347] Updated weights for policy 0, policy_version 521200 (0.0006) [2023-12-27 09:14:48,296][20347] Updated weights for policy 0, policy_version 521210 (0.0010) [2023-12-27 09:14:48,755][20381] Updated weights for policy 1, policy_version 520979 (0.0006) [2023-12-27 09:14:48,806][20381] Updated weights for policy 1, policy_version 520989 (0.0008) [2023-12-27 09:14:48,854][20381] Updated weights for policy 1, policy_version 520999 (0.0007) [2023-12-27 09:14:48,954][20347] Updated weights for policy 0, policy_version 521220 (0.0010) [2023-12-27 09:14:49,007][20347] Updated weights for policy 0, policy_version 521230 (0.0009) [2023-12-27 09:14:49,059][20347] Updated weights for policy 0, policy_version 521240 (0.0005) [2023-12-27 09:14:49,555][20381] Updated weights for policy 1, policy_version 521009 (0.0007) [2023-12-27 09:14:49,610][20381] Updated weights for policy 1, policy_version 521019 (0.0007) [2023-12-27 09:14:49,661][20381] Updated weights for policy 1, policy_version 521029 (0.0008) [2023-12-27 09:14:49,696][20347] Updated weights for policy 0, policy_version 521250 (0.0005) [2023-12-27 09:14:49,711][20381] Updated weights for policy 1, policy_version 521039 (0.0008) [2023-12-27 09:14:49,749][20347] Updated weights for policy 0, policy_version 521260 (0.0005) [2023-12-27 09:14:49,808][20347] Updated weights for policy 0, policy_version 521270 (0.0007) [2023-12-27 09:14:49,868][20347] Updated weights for policy 0, policy_version 521280 (0.0006) [2023-12-27 09:14:50,452][20381] Updated weights for policy 1, policy_version 521049 (0.0007) [2023-12-27 09:14:50,487][20347] Updated weights for policy 0, policy_version 521290 (0.0006) [2023-12-27 09:14:50,496][20381] Updated weights for policy 1, policy_version 521059 (0.0005) [2023-12-27 09:14:50,541][20381] Updated weights for policy 1, policy_version 521069 (0.0005) [2023-12-27 09:14:50,543][20347] Updated weights for policy 0, policy_version 521300 (0.0005) [2023-12-27 09:14:50,606][20347] Updated weights for policy 0, policy_version 521310 (0.0008) [2023-12-27 09:14:51,263][20381] Updated weights for policy 1, policy_version 521079 (0.0008) [2023-12-27 09:14:51,297][20347] Updated weights for policy 0, policy_version 521320 (0.0007) [2023-12-27 09:14:51,326][20381] Updated weights for policy 1, policy_version 521089 (0.0008) [2023-12-27 09:14:51,353][20347] Updated weights for policy 0, policy_version 521330 (0.0007) [2023-12-27 09:14:51,396][20381] Updated weights for policy 1, policy_version 521099 (0.0009) [2023-12-27 09:14:51,417][20347] Updated weights for policy 0, policy_version 521340 (0.0007) [2023-12-27 09:14:52,067][20381] Updated weights for policy 1, policy_version 521109 (0.0007) [2023-12-27 09:14:52,129][20347] Updated weights for policy 0, policy_version 521350 (0.0007) [2023-12-27 09:14:52,131][20381] Updated weights for policy 1, policy_version 521119 (0.0007) [2023-12-27 09:14:52,178][20381] Updated weights for policy 1, policy_version 521129 (0.0005) [2023-12-27 09:14:52,185][20347] Updated weights for policy 0, policy_version 521360 (0.0007) [2023-12-27 09:14:52,244][20347] Updated weights for policy 0, policy_version 521370 (0.0008) [2023-12-27 09:14:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 266919936. Throughput: 0: 10381.7, 1: 10467.6. Samples: 266883920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:52,363][19170] Avg episode reward: [(0, '875.770'), (1, '652.430')] [2023-12-27 09:14:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000521136_133431296.pth... [2023-12-27 09:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000521376_133488640.pth... [2023-12-27 09:14:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000519920_133120000.pth [2023-12-27 09:14:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000520128_133169152.pth [2023-12-27 09:14:52,779][20381] Updated weights for policy 1, policy_version 521139 (0.0006) [2023-12-27 09:14:52,837][20381] Updated weights for policy 1, policy_version 521149 (0.0008) [2023-12-27 09:14:52,893][20381] Updated weights for policy 1, policy_version 521159 (0.0006) [2023-12-27 09:14:52,950][20347] Updated weights for policy 0, policy_version 521380 (0.0008) [2023-12-27 09:14:53,000][20347] Updated weights for policy 0, policy_version 521390 (0.0008) [2023-12-27 09:14:53,051][20347] Updated weights for policy 0, policy_version 521400 (0.0008) [2023-12-27 09:14:53,558][20381] Updated weights for policy 1, policy_version 521169 (0.0008) [2023-12-27 09:14:53,608][20381] Updated weights for policy 1, policy_version 521179 (0.0007) [2023-12-27 09:14:53,649][20381] Updated weights for policy 1, policy_version 521189 (0.0007) [2023-12-27 09:14:53,696][20381] Updated weights for policy 1, policy_version 521199 (0.0006) [2023-12-27 09:14:53,770][20347] Updated weights for policy 0, policy_version 521410 (0.0008) [2023-12-27 09:14:53,829][20347] Updated weights for policy 0, policy_version 521420 (0.0005) [2023-12-27 09:14:53,877][20347] Updated weights for policy 0, policy_version 521430 (0.0006) [2023-12-27 09:14:53,931][20347] Updated weights for policy 0, policy_version 521440 (0.0008) [2023-12-27 09:14:54,371][20381] Updated weights for policy 1, policy_version 521209 (0.0008) [2023-12-27 09:14:54,416][20381] Updated weights for policy 1, policy_version 521219 (0.0007) [2023-12-27 09:14:54,468][20381] Updated weights for policy 1, policy_version 521229 (0.0008) [2023-12-27 09:14:54,597][20347] Updated weights for policy 0, policy_version 521450 (0.0008) [2023-12-27 09:14:54,649][20347] Updated weights for policy 0, policy_version 521460 (0.0008) [2023-12-27 09:14:54,707][20347] Updated weights for policy 0, policy_version 521470 (0.0007) [2023-12-27 09:14:55,131][20381] Updated weights for policy 1, policy_version 521239 (0.0008) [2023-12-27 09:14:55,184][20381] Updated weights for policy 1, policy_version 521249 (0.0008) [2023-12-27 09:14:55,235][20381] Updated weights for policy 1, policy_version 521259 (0.0007) [2023-12-27 09:14:55,430][20347] Updated weights for policy 0, policy_version 521480 (0.0010) [2023-12-27 09:14:55,484][20347] Updated weights for policy 0, policy_version 521490 (0.0006) [2023-12-27 09:14:55,547][20347] Updated weights for policy 0, policy_version 521500 (0.0008) [2023-12-27 09:14:55,907][20381] Updated weights for policy 1, policy_version 521269 (0.0006) [2023-12-27 09:14:55,968][20381] Updated weights for policy 1, policy_version 521279 (0.0005) [2023-12-27 09:14:56,014][20381] Updated weights for policy 1, policy_version 521289 (0.0005) [2023-12-27 09:14:56,135][20347] Updated weights for policy 0, policy_version 521510 (0.0007) [2023-12-27 09:14:56,198][20347] Updated weights for policy 0, policy_version 521520 (0.0005) [2023-12-27 09:14:56,250][20347] Updated weights for policy 0, policy_version 521530 (0.0005) [2023-12-27 09:14:56,583][20381] Updated weights for policy 1, policy_version 521299 (0.0005) [2023-12-27 09:14:56,631][20381] Updated weights for policy 1, policy_version 521309 (0.0005) [2023-12-27 09:14:56,684][20381] Updated weights for policy 1, policy_version 521319 (0.0005) [2023-12-27 09:14:56,819][20347] Updated weights for policy 0, policy_version 521540 (0.0006) [2023-12-27 09:14:56,866][20347] Updated weights for policy 0, policy_version 521550 (0.0007) [2023-12-27 09:14:56,910][20347] Updated weights for policy 0, policy_version 521560 (0.0006) [2023-12-27 09:14:57,325][20381] Updated weights for policy 1, policy_version 521329 (0.0005) [2023-12-27 09:14:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 267018240. Throughput: 0: 10388.5, 1: 10510.6. Samples: 267008692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:14:57,363][19170] Avg episode reward: [(0, '855.960'), (1, '499.730')] [2023-12-27 09:14:57,374][20381] Updated weights for policy 1, policy_version 521339 (0.0006) [2023-12-27 09:14:57,422][20381] Updated weights for policy 1, policy_version 521349 (0.0008) [2023-12-27 09:14:57,480][20381] Updated weights for policy 1, policy_version 521359 (0.0008) [2023-12-27 09:14:57,535][20347] Updated weights for policy 0, policy_version 521570 (0.0007) [2023-12-27 09:14:57,595][20347] Updated weights for policy 0, policy_version 521580 (0.0007) [2023-12-27 09:14:57,650][20347] Updated weights for policy 0, policy_version 521590 (0.0007) [2023-12-27 09:14:57,704][20347] Updated weights for policy 0, policy_version 521600 (0.0007) [2023-12-27 09:14:58,065][20381] Updated weights for policy 1, policy_version 521369 (0.0006) [2023-12-27 09:14:58,111][20381] Updated weights for policy 1, policy_version 521379 (0.0006) [2023-12-27 09:14:58,160][20381] Updated weights for policy 1, policy_version 521389 (0.0007) [2023-12-27 09:14:58,384][20347] Updated weights for policy 0, policy_version 521610 (0.0008) [2023-12-27 09:14:58,443][20347] Updated weights for policy 0, policy_version 521620 (0.0008) [2023-12-27 09:14:58,502][20347] Updated weights for policy 0, policy_version 521630 (0.0008) [2023-12-27 09:14:58,907][20381] Updated weights for policy 1, policy_version 521399 (0.0007) [2023-12-27 09:14:58,966][20381] Updated weights for policy 1, policy_version 521409 (0.0006) [2023-12-27 09:14:59,028][20381] Updated weights for policy 1, policy_version 521419 (0.0008) [2023-12-27 09:14:59,343][20347] Updated weights for policy 0, policy_version 521640 (0.0009) [2023-12-27 09:14:59,410][20347] Updated weights for policy 0, policy_version 521650 (0.0008) [2023-12-27 09:14:59,471][20347] Updated weights for policy 0, policy_version 521660 (0.0008) [2023-12-27 09:14:59,741][20381] Updated weights for policy 1, policy_version 521429 (0.0009) [2023-12-27 09:14:59,792][20381] Updated weights for policy 1, policy_version 521439 (0.0010) [2023-12-27 09:14:59,851][20381] Updated weights for policy 1, policy_version 521449 (0.0008) [2023-12-27 09:15:00,199][20347] Updated weights for policy 0, policy_version 521670 (0.0007) [2023-12-27 09:15:00,249][20347] Updated weights for policy 0, policy_version 521680 (0.0007) [2023-12-27 09:15:00,293][20347] Updated weights for policy 0, policy_version 521690 (0.0007) [2023-12-27 09:15:00,583][20381] Updated weights for policy 1, policy_version 521459 (0.0008) [2023-12-27 09:15:00,639][20381] Updated weights for policy 1, policy_version 521469 (0.0007) [2023-12-27 09:15:00,694][20381] Updated weights for policy 1, policy_version 521479 (0.0007) [2023-12-27 09:15:00,982][20347] Updated weights for policy 0, policy_version 521700 (0.0008) [2023-12-27 09:15:01,042][20347] Updated weights for policy 0, policy_version 521710 (0.0008) [2023-12-27 09:15:01,095][20347] Updated weights for policy 0, policy_version 521720 (0.0007) [2023-12-27 09:15:01,394][20381] Updated weights for policy 1, policy_version 521489 (0.0007) [2023-12-27 09:15:01,454][20381] Updated weights for policy 1, policy_version 521499 (0.0007) [2023-12-27 09:15:01,508][20381] Updated weights for policy 1, policy_version 521509 (0.0007) [2023-12-27 09:15:01,555][20381] Updated weights for policy 1, policy_version 521519 (0.0007) [2023-12-27 09:15:01,780][20347] Updated weights for policy 0, policy_version 521730 (0.0007) [2023-12-27 09:15:01,840][20347] Updated weights for policy 0, policy_version 521740 (0.0008) [2023-12-27 09:15:01,900][20347] Updated weights for policy 0, policy_version 521750 (0.0007) [2023-12-27 09:15:01,960][20347] Updated weights for policy 0, policy_version 521760 (0.0008) [2023-12-27 09:15:02,240][20381] Updated weights for policy 1, policy_version 521529 (0.0009) [2023-12-27 09:15:02,299][20381] Updated weights for policy 1, policy_version 521539 (0.0010) [2023-12-27 09:15:02,360][20381] Updated weights for policy 1, policy_version 521549 (0.0008) [2023-12-27 09:15:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 267116544. Throughput: 0: 10347.0, 1: 10475.1. Samples: 267130888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:02,363][19170] Avg episode reward: [(0, '849.630'), (1, '372.530')] [2023-12-27 09:15:02,670][20347] Updated weights for policy 0, policy_version 521770 (0.0008) [2023-12-27 09:15:02,728][20347] Updated weights for policy 0, policy_version 521780 (0.0007) [2023-12-27 09:15:02,789][20347] Updated weights for policy 0, policy_version 521790 (0.0009) [2023-12-27 09:15:02,974][20381] Updated weights for policy 1, policy_version 521559 (0.0009) [2023-12-27 09:15:03,022][20381] Updated weights for policy 1, policy_version 521569 (0.0010) [2023-12-27 09:15:03,080][20381] Updated weights for policy 1, policy_version 521579 (0.0010) [2023-12-27 09:15:03,451][20347] Updated weights for policy 0, policy_version 521800 (0.0006) [2023-12-27 09:15:03,511][20347] Updated weights for policy 0, policy_version 521810 (0.0005) [2023-12-27 09:15:03,570][20347] Updated weights for policy 0, policy_version 521820 (0.0005) [2023-12-27 09:15:03,778][20381] Updated weights for policy 1, policy_version 521589 (0.0008) [2023-12-27 09:15:03,830][20381] Updated weights for policy 1, policy_version 521599 (0.0007) [2023-12-27 09:15:03,883][20381] Updated weights for policy 1, policy_version 521609 (0.0008) [2023-12-27 09:15:04,156][20347] Updated weights for policy 0, policy_version 521830 (0.0006) [2023-12-27 09:15:04,218][20347] Updated weights for policy 0, policy_version 521840 (0.0006) [2023-12-27 09:15:04,278][20347] Updated weights for policy 0, policy_version 521850 (0.0007) [2023-12-27 09:15:04,642][20381] Updated weights for policy 1, policy_version 521619 (0.0008) [2023-12-27 09:15:04,719][20381] Updated weights for policy 1, policy_version 521629 (0.0007) [2023-12-27 09:15:04,778][20381] Updated weights for policy 1, policy_version 521639 (0.0005) [2023-12-27 09:15:04,944][20347] Updated weights for policy 0, policy_version 521860 (0.0006) [2023-12-27 09:15:04,993][20347] Updated weights for policy 0, policy_version 521870 (0.0005) [2023-12-27 09:15:05,056][20347] Updated weights for policy 0, policy_version 521880 (0.0007) [2023-12-27 09:15:05,344][20381] Updated weights for policy 1, policy_version 521649 (0.0006) [2023-12-27 09:15:05,403][20381] Updated weights for policy 1, policy_version 521659 (0.0007) [2023-12-27 09:15:05,469][20381] Updated weights for policy 1, policy_version 521669 (0.0008) [2023-12-27 09:15:05,532][20381] Updated weights for policy 1, policy_version 521679 (0.0008) [2023-12-27 09:15:05,630][20347] Updated weights for policy 0, policy_version 521890 (0.0007) [2023-12-27 09:15:05,686][20347] Updated weights for policy 0, policy_version 521900 (0.0005) [2023-12-27 09:15:05,741][20347] Updated weights for policy 0, policy_version 521910 (0.0005) [2023-12-27 09:15:05,803][20347] Updated weights for policy 0, policy_version 521920 (0.0006) [2023-12-27 09:15:06,205][20381] Updated weights for policy 1, policy_version 521689 (0.0010) [2023-12-27 09:15:06,266][20381] Updated weights for policy 1, policy_version 521699 (0.0008) [2023-12-27 09:15:06,329][20381] Updated weights for policy 1, policy_version 521709 (0.0008) [2023-12-27 09:15:06,434][20347] Updated weights for policy 0, policy_version 521930 (0.0007) [2023-12-27 09:15:06,496][20347] Updated weights for policy 0, policy_version 521940 (0.0006) [2023-12-27 09:15:06,554][20347] Updated weights for policy 0, policy_version 521950 (0.0007) [2023-12-27 09:15:06,999][20381] Updated weights for policy 1, policy_version 521719 (0.0008) [2023-12-27 09:15:07,050][20381] Updated weights for policy 1, policy_version 521729 (0.0010) [2023-12-27 09:15:07,103][20381] Updated weights for policy 1, policy_version 521739 (0.0005) [2023-12-27 09:15:07,217][20347] Updated weights for policy 0, policy_version 521960 (0.0006) [2023-12-27 09:15:07,274][20347] Updated weights for policy 0, policy_version 521970 (0.0006) [2023-12-27 09:15:07,339][20347] Updated weights for policy 0, policy_version 521980 (0.0006) [2023-12-27 09:15:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 267231232. Throughput: 0: 10379.2, 1: 10460.5. Samples: 267193584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:07,363][19170] Avg episode reward: [(0, '868.070'), (1, '459.940')] [2023-12-27 09:15:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000521984_133644288.pth... [2023-12-27 09:15:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000521744_133586944.pth... [2023-12-27 09:15:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000520768_133332992.pth [2023-12-27 09:15:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000520528_133275648.pth [2023-12-27 09:15:07,717][20381] Updated weights for policy 1, policy_version 521749 (0.0006) [2023-12-27 09:15:07,778][20381] Updated weights for policy 1, policy_version 521759 (0.0005) [2023-12-27 09:15:07,828][20381] Updated weights for policy 1, policy_version 521769 (0.0007) [2023-12-27 09:15:07,979][20347] Updated weights for policy 0, policy_version 521990 (0.0007) [2023-12-27 09:15:08,032][20347] Updated weights for policy 0, policy_version 522000 (0.0008) [2023-12-27 09:15:08,090][20347] Updated weights for policy 0, policy_version 522010 (0.0007) [2023-12-27 09:15:08,474][20381] Updated weights for policy 1, policy_version 521779 (0.0007) [2023-12-27 09:15:08,530][20381] Updated weights for policy 1, policy_version 521789 (0.0007) [2023-12-27 09:15:08,589][20381] Updated weights for policy 1, policy_version 521799 (0.0008) [2023-12-27 09:15:08,782][20347] Updated weights for policy 0, policy_version 522020 (0.0007) [2023-12-27 09:15:08,843][20347] Updated weights for policy 0, policy_version 522030 (0.0008) [2023-12-27 09:15:08,897][20347] Updated weights for policy 0, policy_version 522040 (0.0011) [2023-12-27 09:15:09,247][20381] Updated weights for policy 1, policy_version 521809 (0.0008) [2023-12-27 09:15:09,302][20381] Updated weights for policy 1, policy_version 521819 (0.0008) [2023-12-27 09:15:09,371][20381] Updated weights for policy 1, policy_version 521829 (0.0007) [2023-12-27 09:15:09,435][20381] Updated weights for policy 1, policy_version 521839 (0.0007) [2023-12-27 09:15:09,570][20347] Updated weights for policy 0, policy_version 522050 (0.0010) [2023-12-27 09:15:09,633][20347] Updated weights for policy 0, policy_version 522060 (0.0011) [2023-12-27 09:15:09,690][20347] Updated weights for policy 0, policy_version 522070 (0.0010) [2023-12-27 09:15:09,755][20347] Updated weights for policy 0, policy_version 522080 (0.0010) [2023-12-27 09:15:10,110][20381] Updated weights for policy 1, policy_version 521849 (0.0007) [2023-12-27 09:15:10,170][20381] Updated weights for policy 1, policy_version 521859 (0.0009) [2023-12-27 09:15:10,234][20381] Updated weights for policy 1, policy_version 521869 (0.0009) [2023-12-27 09:15:10,547][20347] Updated weights for policy 0, policy_version 522090 (0.0007) [2023-12-27 09:15:10,606][20347] Updated weights for policy 0, policy_version 522100 (0.0010) [2023-12-27 09:15:10,661][20347] Updated weights for policy 0, policy_version 522110 (0.0010) [2023-12-27 09:15:10,973][20381] Updated weights for policy 1, policy_version 521879 (0.0007) [2023-12-27 09:15:11,041][20381] Updated weights for policy 1, policy_version 521889 (0.0006) [2023-12-27 09:15:11,105][20381] Updated weights for policy 1, policy_version 521899 (0.0007) [2023-12-27 09:15:11,332][20347] Updated weights for policy 0, policy_version 522120 (0.0011) [2023-12-27 09:15:11,397][20347] Updated weights for policy 0, policy_version 522130 (0.0010) [2023-12-27 09:15:11,462][20347] Updated weights for policy 0, policy_version 522140 (0.0010) [2023-12-27 09:15:11,730][20381] Updated weights for policy 1, policy_version 521909 (0.0010) [2023-12-27 09:15:11,788][20381] Updated weights for policy 1, policy_version 521919 (0.0009) [2023-12-27 09:15:11,850][20381] Updated weights for policy 1, policy_version 521929 (0.0009) [2023-12-27 09:15:12,124][20347] Updated weights for policy 0, policy_version 522150 (0.0011) [2023-12-27 09:15:12,181][20347] Updated weights for policy 0, policy_version 522160 (0.0006) [2023-12-27 09:15:12,239][20347] Updated weights for policy 0, policy_version 522170 (0.0005) [2023-12-27 09:15:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 267329536. Throughput: 0: 10375.9, 1: 10431.1. Samples: 267317336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:12,366][19170] Avg episode reward: [(0, '870.590'), (1, '550.780')] [2023-12-27 09:15:12,526][20381] Updated weights for policy 1, policy_version 521939 (0.0009) [2023-12-27 09:15:12,579][20381] Updated weights for policy 1, policy_version 521949 (0.0011) [2023-12-27 09:15:12,639][20381] Updated weights for policy 1, policy_version 521959 (0.0006) [2023-12-27 09:15:12,912][20347] Updated weights for policy 0, policy_version 522180 (0.0010) [2023-12-27 09:15:12,964][20347] Updated weights for policy 0, policy_version 522190 (0.0010) [2023-12-27 09:15:13,009][20347] Updated weights for policy 0, policy_version 522200 (0.0010) [2023-12-27 09:15:13,294][20381] Updated weights for policy 1, policy_version 521969 (0.0010) [2023-12-27 09:15:13,354][20381] Updated weights for policy 1, policy_version 521979 (0.0005) [2023-12-27 09:15:13,409][20381] Updated weights for policy 1, policy_version 521989 (0.0007) [2023-12-27 09:15:13,457][20381] Updated weights for policy 1, policy_version 521999 (0.0010) [2023-12-27 09:15:13,678][20347] Updated weights for policy 0, policy_version 522210 (0.0010) [2023-12-27 09:15:13,733][20347] Updated weights for policy 0, policy_version 522220 (0.0010) [2023-12-27 09:15:13,780][20347] Updated weights for policy 0, policy_version 522230 (0.0010) [2023-12-27 09:15:13,834][20347] Updated weights for policy 0, policy_version 522240 (0.0010) [2023-12-27 09:15:14,087][20381] Updated weights for policy 1, policy_version 522009 (0.0008) [2023-12-27 09:15:14,144][20381] Updated weights for policy 1, policy_version 522019 (0.0007) [2023-12-27 09:15:14,196][20381] Updated weights for policy 1, policy_version 522029 (0.0008) [2023-12-27 09:15:14,502][20347] Updated weights for policy 0, policy_version 522250 (0.0010) [2023-12-27 09:15:14,559][20347] Updated weights for policy 0, policy_version 522260 (0.0010) [2023-12-27 09:15:14,608][20347] Updated weights for policy 0, policy_version 522270 (0.0010) [2023-12-27 09:15:14,915][20381] Updated weights for policy 1, policy_version 522039 (0.0010) [2023-12-27 09:15:14,980][20381] Updated weights for policy 1, policy_version 522049 (0.0011) [2023-12-27 09:15:15,038][20381] Updated weights for policy 1, policy_version 522059 (0.0011) [2023-12-27 09:15:15,321][20347] Updated weights for policy 0, policy_version 522280 (0.0009) [2023-12-27 09:15:15,377][20347] Updated weights for policy 0, policy_version 522290 (0.0007) [2023-12-27 09:15:15,444][20347] Updated weights for policy 0, policy_version 522300 (0.0007) [2023-12-27 09:15:15,769][20381] Updated weights for policy 1, policy_version 522069 (0.0008) [2023-12-27 09:15:15,822][20381] Updated weights for policy 1, policy_version 522079 (0.0005) [2023-12-27 09:15:15,869][20381] Updated weights for policy 1, policy_version 522089 (0.0007) [2023-12-27 09:15:16,034][20347] Updated weights for policy 0, policy_version 522310 (0.0006) [2023-12-27 09:15:16,088][20347] Updated weights for policy 0, policy_version 522320 (0.0005) [2023-12-27 09:15:16,140][20347] Updated weights for policy 0, policy_version 522330 (0.0005) [2023-12-27 09:15:16,571][20381] Updated weights for policy 1, policy_version 522099 (0.0007) [2023-12-27 09:15:16,616][20381] Updated weights for policy 1, policy_version 522109 (0.0007) [2023-12-27 09:15:16,661][20381] Updated weights for policy 1, policy_version 522119 (0.0007) [2023-12-27 09:15:16,764][20347] Updated weights for policy 0, policy_version 522340 (0.0007) [2023-12-27 09:15:16,823][20347] Updated weights for policy 0, policy_version 522350 (0.0007) [2023-12-27 09:15:16,879][20347] Updated weights for policy 0, policy_version 522360 (0.0005) [2023-12-27 09:15:17,336][20381] Updated weights for policy 1, policy_version 522129 (0.0007) [2023-12-27 09:15:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 267427840. Throughput: 0: 10344.2, 1: 10419.8. Samples: 267441344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:17,363][19170] Avg episode reward: [(0, '884.340'), (1, '587.740')] [2023-12-27 09:15:17,395][20381] Updated weights for policy 1, policy_version 522139 (0.0008) [2023-12-27 09:15:17,449][20381] Updated weights for policy 1, policy_version 522149 (0.0010) [2023-12-27 09:15:17,499][20381] Updated weights for policy 1, policy_version 522159 (0.0010) [2023-12-27 09:15:17,520][20347] Updated weights for policy 0, policy_version 522370 (0.0006) [2023-12-27 09:15:17,584][20347] Updated weights for policy 0, policy_version 522380 (0.0008) [2023-12-27 09:15:17,637][20347] Updated weights for policy 0, policy_version 522390 (0.0009) [2023-12-27 09:15:17,690][20347] Updated weights for policy 0, policy_version 522400 (0.0009) [2023-12-27 09:15:18,131][20381] Updated weights for policy 1, policy_version 522169 (0.0008) [2023-12-27 09:15:18,184][20381] Updated weights for policy 1, policy_version 522179 (0.0010) [2023-12-27 09:15:18,237][20381] Updated weights for policy 1, policy_version 522189 (0.0010) [2023-12-27 09:15:18,429][20347] Updated weights for policy 0, policy_version 522410 (0.0008) [2023-12-27 09:15:18,487][20347] Updated weights for policy 0, policy_version 522420 (0.0008) [2023-12-27 09:15:18,534][20347] Updated weights for policy 0, policy_version 522430 (0.0008) [2023-12-27 09:15:18,936][20381] Updated weights for policy 1, policy_version 522199 (0.0009) [2023-12-27 09:15:18,998][20381] Updated weights for policy 1, policy_version 522209 (0.0007) [2023-12-27 09:15:19,064][20381] Updated weights for policy 1, policy_version 522219 (0.0008) [2023-12-27 09:15:19,219][20347] Updated weights for policy 0, policy_version 522440 (0.0007) [2023-12-27 09:15:19,284][20347] Updated weights for policy 0, policy_version 522450 (0.0008) [2023-12-27 09:15:19,347][20347] Updated weights for policy 0, policy_version 522460 (0.0008) [2023-12-27 09:15:19,760][20381] Updated weights for policy 1, policy_version 522229 (0.0008) [2023-12-27 09:15:19,821][20381] Updated weights for policy 1, policy_version 522239 (0.0011) [2023-12-27 09:15:19,887][20381] Updated weights for policy 1, policy_version 522249 (0.0008) [2023-12-27 09:15:20,084][20347] Updated weights for policy 0, policy_version 522470 (0.0008) [2023-12-27 09:15:20,144][20347] Updated weights for policy 0, policy_version 522480 (0.0006) [2023-12-27 09:15:20,211][20347] Updated weights for policy 0, policy_version 522490 (0.0008) [2023-12-27 09:15:20,580][20381] Updated weights for policy 1, policy_version 522259 (0.0009) [2023-12-27 09:15:20,638][20381] Updated weights for policy 1, policy_version 522269 (0.0006) [2023-12-27 09:15:20,690][20381] Updated weights for policy 1, policy_version 522279 (0.0007) [2023-12-27 09:15:20,913][20347] Updated weights for policy 0, policy_version 522500 (0.0008) [2023-12-27 09:15:20,979][20347] Updated weights for policy 0, policy_version 522510 (0.0008) [2023-12-27 09:15:21,052][20347] Updated weights for policy 0, policy_version 522520 (0.0009) [2023-12-27 09:15:21,455][20381] Updated weights for policy 1, policy_version 522289 (0.0008) [2023-12-27 09:15:21,515][20381] Updated weights for policy 1, policy_version 522299 (0.0007) [2023-12-27 09:15:21,575][20381] Updated weights for policy 1, policy_version 522309 (0.0009) [2023-12-27 09:15:21,637][20381] Updated weights for policy 1, policy_version 522319 (0.0007) [2023-12-27 09:15:21,929][20347] Updated weights for policy 0, policy_version 522530 (0.0009) [2023-12-27 09:15:21,985][20347] Updated weights for policy 0, policy_version 522540 (0.0007) [2023-12-27 09:15:22,034][20347] Updated weights for policy 0, policy_version 522550 (0.0007) [2023-12-27 09:15:22,087][20347] Updated weights for policy 0, policy_version 522560 (0.0007) [2023-12-27 09:15:22,353][20381] Updated weights for policy 1, policy_version 522329 (0.0008) [2023-12-27 09:15:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 267526144. Throughput: 0: 10317.9, 1: 10354.1. Samples: 267501176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:22,363][19170] Avg episode reward: [(0, '952.650'), (1, '621.750')] [2023-12-27 09:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000522560_133791744.pth... [2023-12-27 09:15:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000521376_133488640.pth [2023-12-27 09:15:22,416][20381] Updated weights for policy 1, policy_version 522339 (0.0009) [2023-12-27 09:15:22,474][20381] Updated weights for policy 1, policy_version 522349 (0.0007) [2023-12-27 09:15:22,489][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000522352_133742592.pth... [2023-12-27 09:15:22,492][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000521136_133431296.pth [2023-12-27 09:15:22,858][20347] Updated weights for policy 0, policy_version 522570 (0.0007) [2023-12-27 09:15:22,913][20347] Updated weights for policy 0, policy_version 522580 (0.0007) [2023-12-27 09:15:22,972][20347] Updated weights for policy 0, policy_version 522590 (0.0006) [2023-12-27 09:15:23,179][20381] Updated weights for policy 1, policy_version 522359 (0.0008) [2023-12-27 09:15:23,230][20381] Updated weights for policy 1, policy_version 522369 (0.0007) [2023-12-27 09:15:23,273][20381] Updated weights for policy 1, policy_version 522379 (0.0006) [2023-12-27 09:15:23,641][20347] Updated weights for policy 0, policy_version 522600 (0.0006) [2023-12-27 09:15:23,702][20347] Updated weights for policy 0, policy_version 522610 (0.0009) [2023-12-27 09:15:23,767][20347] Updated weights for policy 0, policy_version 522620 (0.0010) [2023-12-27 09:15:23,941][20381] Updated weights for policy 1, policy_version 522389 (0.0007) [2023-12-27 09:15:23,994][20381] Updated weights for policy 1, policy_version 522399 (0.0008) [2023-12-27 09:15:24,051][20381] Updated weights for policy 1, policy_version 522409 (0.0009) [2023-12-27 09:15:24,398][20347] Updated weights for policy 0, policy_version 522630 (0.0010) [2023-12-27 09:15:24,460][20347] Updated weights for policy 0, policy_version 522640 (0.0010) [2023-12-27 09:15:24,519][20347] Updated weights for policy 0, policy_version 522650 (0.0010) [2023-12-27 09:15:24,723][20381] Updated weights for policy 1, policy_version 522419 (0.0009) [2023-12-27 09:15:24,784][20381] Updated weights for policy 1, policy_version 522429 (0.0005) [2023-12-27 09:15:24,846][20381] Updated weights for policy 1, policy_version 522439 (0.0011) [2023-12-27 09:15:25,141][20347] Updated weights for policy 0, policy_version 522660 (0.0011) [2023-12-27 09:15:25,206][20347] Updated weights for policy 0, policy_version 522670 (0.0008) [2023-12-27 09:15:25,268][20347] Updated weights for policy 0, policy_version 522680 (0.0010) [2023-12-27 09:15:25,502][20381] Updated weights for policy 1, policy_version 522449 (0.0010) [2023-12-27 09:15:25,547][20381] Updated weights for policy 1, policy_version 522459 (0.0008) [2023-12-27 09:15:25,601][20381] Updated weights for policy 1, policy_version 522469 (0.0007) [2023-12-27 09:15:25,664][20381] Updated weights for policy 1, policy_version 522479 (0.0008) [2023-12-27 09:15:25,880][20347] Updated weights for policy 0, policy_version 522690 (0.0010) [2023-12-27 09:15:25,931][20347] Updated weights for policy 0, policy_version 522700 (0.0010) [2023-12-27 09:15:25,995][20347] Updated weights for policy 0, policy_version 522710 (0.0008) [2023-12-27 09:15:26,047][20347] Updated weights for policy 0, policy_version 522720 (0.0010) [2023-12-27 09:15:26,365][20381] Updated weights for policy 1, policy_version 522489 (0.0007) [2023-12-27 09:15:26,419][20381] Updated weights for policy 1, policy_version 522499 (0.0008) [2023-12-27 09:15:26,476][20381] Updated weights for policy 1, policy_version 522509 (0.0007) [2023-12-27 09:15:26,729][20347] Updated weights for policy 0, policy_version 522730 (0.0005) [2023-12-27 09:15:26,776][20347] Updated weights for policy 0, policy_version 522740 (0.0006) [2023-12-27 09:15:26,828][20347] Updated weights for policy 0, policy_version 522750 (0.0007) [2023-12-27 09:15:27,204][20381] Updated weights for policy 1, policy_version 522519 (0.0009) [2023-12-27 09:15:27,261][20381] Updated weights for policy 1, policy_version 522529 (0.0007) [2023-12-27 09:15:27,318][20381] Updated weights for policy 1, policy_version 522539 (0.0007) [2023-12-27 09:15:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 267632640. Throughput: 0: 10276.2, 1: 10301.3. Samples: 267619936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:27,363][19170] Avg episode reward: [(0, '986.580'), (1, '717.620')] [2023-12-27 09:15:27,391][20347] Updated weights for policy 0, policy_version 522760 (0.0010) [2023-12-27 09:15:27,442][20347] Updated weights for policy 0, policy_version 522770 (0.0010) [2023-12-27 09:15:27,490][20347] Updated weights for policy 0, policy_version 522780 (0.0010) [2023-12-27 09:15:27,880][20381] Updated weights for policy 1, policy_version 522549 (0.0009) [2023-12-27 09:15:27,925][20381] Updated weights for policy 1, policy_version 522559 (0.0007) [2023-12-27 09:15:27,982][20381] Updated weights for policy 1, policy_version 522569 (0.0005) [2023-12-27 09:15:28,066][20347] Updated weights for policy 0, policy_version 522790 (0.0007) [2023-12-27 09:15:28,116][20347] Updated weights for policy 0, policy_version 522800 (0.0005) [2023-12-27 09:15:28,169][20347] Updated weights for policy 0, policy_version 522810 (0.0009) [2023-12-27 09:15:28,578][20381] Updated weights for policy 1, policy_version 522579 (0.0006) [2023-12-27 09:15:28,625][20381] Updated weights for policy 1, policy_version 522589 (0.0007) [2023-12-27 09:15:28,670][20381] Updated weights for policy 1, policy_version 522599 (0.0007) [2023-12-27 09:15:28,821][20347] Updated weights for policy 0, policy_version 522820 (0.0010) [2023-12-27 09:15:28,869][20347] Updated weights for policy 0, policy_version 522830 (0.0010) [2023-12-27 09:15:28,918][20347] Updated weights for policy 0, policy_version 522840 (0.0009) [2023-12-27 09:15:29,360][20381] Updated weights for policy 1, policy_version 522609 (0.0007) [2023-12-27 09:15:29,417][20381] Updated weights for policy 1, policy_version 522619 (0.0008) [2023-12-27 09:15:29,466][20381] Updated weights for policy 1, policy_version 522629 (0.0008) [2023-12-27 09:15:29,512][20381] Updated weights for policy 1, policy_version 522639 (0.0008) [2023-12-27 09:15:29,581][20347] Updated weights for policy 0, policy_version 522850 (0.0006) [2023-12-27 09:15:29,636][20347] Updated weights for policy 0, policy_version 522860 (0.0009) [2023-12-27 09:15:29,695][20347] Updated weights for policy 0, policy_version 522870 (0.0011) [2023-12-27 09:15:29,750][20347] Updated weights for policy 0, policy_version 522880 (0.0010) [2023-12-27 09:15:30,243][20381] Updated weights for policy 1, policy_version 522649 (0.0006) [2023-12-27 09:15:30,309][20381] Updated weights for policy 1, policy_version 522659 (0.0007) [2023-12-27 09:15:30,369][20381] Updated weights for policy 1, policy_version 522669 (0.0007) [2023-12-27 09:15:30,413][20347] Updated weights for policy 0, policy_version 522890 (0.0008) [2023-12-27 09:15:30,461][20347] Updated weights for policy 0, policy_version 522900 (0.0009) [2023-12-27 09:15:30,509][20347] Updated weights for policy 0, policy_version 522910 (0.0009) [2023-12-27 09:15:31,020][20381] Updated weights for policy 1, policy_version 522679 (0.0008) [2023-12-27 09:15:31,075][20381] Updated weights for policy 1, policy_version 522689 (0.0007) [2023-12-27 09:15:31,117][20347] Updated weights for policy 0, policy_version 522920 (0.0010) [2023-12-27 09:15:31,130][20381] Updated weights for policy 1, policy_version 522699 (0.0007) [2023-12-27 09:15:31,183][20347] Updated weights for policy 0, policy_version 522930 (0.0011) [2023-12-27 09:15:31,236][20347] Updated weights for policy 0, policy_version 522940 (0.0008) [2023-12-27 09:15:31,865][20347] Updated weights for policy 0, policy_version 522950 (0.0010) [2023-12-27 09:15:31,923][20347] Updated weights for policy 0, policy_version 522960 (0.0011) [2023-12-27 09:15:31,951][20381] Updated weights for policy 1, policy_version 522709 (0.0006) [2023-12-27 09:15:31,972][20347] Updated weights for policy 0, policy_version 522970 (0.0010) [2023-12-27 09:15:32,007][20381] Updated weights for policy 1, policy_version 522719 (0.0005) [2023-12-27 09:15:32,069][20381] Updated weights for policy 1, policy_version 522729 (0.0007) [2023-12-27 09:15:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 267739136. Throughput: 0: 10335.3, 1: 10241.5. Samples: 267746396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:32,363][19170] Avg episode reward: [(0, '977.090'), (1, '798.030')] [2023-12-27 09:15:32,627][20347] Updated weights for policy 0, policy_version 522980 (0.0010) [2023-12-27 09:15:32,681][20347] Updated weights for policy 0, policy_version 522990 (0.0010) [2023-12-27 09:15:32,738][20347] Updated weights for policy 0, policy_version 523000 (0.0010) [2023-12-27 09:15:32,766][20381] Updated weights for policy 1, policy_version 522739 (0.0007) [2023-12-27 09:15:32,827][20381] Updated weights for policy 1, policy_version 522749 (0.0008) [2023-12-27 09:15:32,875][20381] Updated weights for policy 1, policy_version 522759 (0.0007) [2023-12-27 09:15:33,410][20347] Updated weights for policy 0, policy_version 523010 (0.0008) [2023-12-27 09:15:33,460][20347] Updated weights for policy 0, policy_version 523020 (0.0010) [2023-12-27 09:15:33,519][20347] Updated weights for policy 0, policy_version 523030 (0.0010) [2023-12-27 09:15:33,573][20381] Updated weights for policy 1, policy_version 522769 (0.0007) [2023-12-27 09:15:33,576][20347] Updated weights for policy 0, policy_version 523040 (0.0010) [2023-12-27 09:15:33,626][20381] Updated weights for policy 1, policy_version 522779 (0.0007) [2023-12-27 09:15:33,680][20381] Updated weights for policy 1, policy_version 522789 (0.0007) [2023-12-27 09:15:33,734][20381] Updated weights for policy 1, policy_version 522799 (0.0007) [2023-12-27 09:15:34,267][20347] Updated weights for policy 0, policy_version 523050 (0.0007) [2023-12-27 09:15:34,326][20347] Updated weights for policy 0, policy_version 523060 (0.0007) [2023-12-27 09:15:34,383][20347] Updated weights for policy 0, policy_version 523070 (0.0007) [2023-12-27 09:15:34,392][20381] Updated weights for policy 1, policy_version 522809 (0.0007) [2023-12-27 09:15:34,461][20381] Updated weights for policy 1, policy_version 522819 (0.0008) [2023-12-27 09:15:34,524][20381] Updated weights for policy 1, policy_version 522829 (0.0008) [2023-12-27 09:15:35,088][20347] Updated weights for policy 0, policy_version 523080 (0.0009) [2023-12-27 09:15:35,128][20381] Updated weights for policy 1, policy_version 522839 (0.0008) [2023-12-27 09:15:35,150][20347] Updated weights for policy 0, policy_version 523090 (0.0008) [2023-12-27 09:15:35,185][20381] Updated weights for policy 1, policy_version 522849 (0.0007) [2023-12-27 09:15:35,205][20347] Updated weights for policy 0, policy_version 523100 (0.0008) [2023-12-27 09:15:35,241][20381] Updated weights for policy 1, policy_version 522859 (0.0006) [2023-12-27 09:15:35,794][20381] Updated weights for policy 1, policy_version 522869 (0.0007) [2023-12-27 09:15:35,853][20381] Updated weights for policy 1, policy_version 522879 (0.0008) [2023-12-27 09:15:35,913][20381] Updated weights for policy 1, policy_version 522889 (0.0008) [2023-12-27 09:15:35,930][20347] Updated weights for policy 0, policy_version 523110 (0.0007) [2023-12-27 09:15:35,978][20347] Updated weights for policy 0, policy_version 523120 (0.0007) [2023-12-27 09:15:36,039][20347] Updated weights for policy 0, policy_version 523130 (0.0008) [2023-12-27 09:15:36,640][20381] Updated weights for policy 1, policy_version 522899 (0.0006) [2023-12-27 09:15:36,691][20347] Updated weights for policy 0, policy_version 523140 (0.0008) [2023-12-27 09:15:36,705][20381] Updated weights for policy 1, policy_version 522909 (0.0007) [2023-12-27 09:15:36,753][20347] Updated weights for policy 0, policy_version 523150 (0.0006) [2023-12-27 09:15:36,766][20381] Updated weights for policy 1, policy_version 522919 (0.0008) [2023-12-27 09:15:36,803][20347] Updated weights for policy 0, policy_version 523160 (0.0006) [2023-12-27 09:15:37,363][19170] Fps is (10 sec: 20479.0, 60 sec: 20616.4, 300 sec: 20771.6). Total num frames: 267837440. Throughput: 0: 10280.9, 1: 10264.6. Samples: 267808476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:37,364][19170] Avg episode reward: [(0, '990.510'), (1, '756.220')] [2023-12-27 09:15:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000522928_133890048.pth... [2023-12-27 09:15:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000523168_133947392.pth... [2023-12-27 09:15:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000521744_133586944.pth [2023-12-27 09:15:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000521984_133644288.pth [2023-12-27 09:15:37,447][20381] Updated weights for policy 1, policy_version 522929 (0.0008) [2023-12-27 09:15:37,471][20347] Updated weights for policy 0, policy_version 523170 (0.0008) [2023-12-27 09:15:37,505][20381] Updated weights for policy 1, policy_version 522939 (0.0008) [2023-12-27 09:15:37,530][20347] Updated weights for policy 0, policy_version 523180 (0.0008) [2023-12-27 09:15:37,561][20381] Updated weights for policy 1, policy_version 522949 (0.0006) [2023-12-27 09:15:37,585][20347] Updated weights for policy 0, policy_version 523190 (0.0006) [2023-12-27 09:15:37,620][20381] Updated weights for policy 1, policy_version 522959 (0.0007) [2023-12-27 09:15:37,644][20347] Updated weights for policy 0, policy_version 523200 (0.0006) [2023-12-27 09:15:38,328][20381] Updated weights for policy 1, policy_version 522969 (0.0006) [2023-12-27 09:15:38,335][20347] Updated weights for policy 0, policy_version 523210 (0.0006) [2023-12-27 09:15:38,386][20381] Updated weights for policy 1, policy_version 522979 (0.0007) [2023-12-27 09:15:38,400][20347] Updated weights for policy 0, policy_version 523220 (0.0006) [2023-12-27 09:15:38,445][20381] Updated weights for policy 1, policy_version 522989 (0.0008) [2023-12-27 09:15:38,460][20347] Updated weights for policy 0, policy_version 523230 (0.0006) [2023-12-27 09:15:39,085][20347] Updated weights for policy 0, policy_version 523240 (0.0008) [2023-12-27 09:15:39,131][20381] Updated weights for policy 1, policy_version 522999 (0.0006) [2023-12-27 09:15:39,140][20347] Updated weights for policy 0, policy_version 523250 (0.0006) [2023-12-27 09:15:39,187][20381] Updated weights for policy 1, policy_version 523009 (0.0007) [2023-12-27 09:15:39,195][20347] Updated weights for policy 0, policy_version 523260 (0.0008) [2023-12-27 09:15:39,245][20381] Updated weights for policy 1, policy_version 523019 (0.0007) [2023-12-27 09:15:39,947][20347] Updated weights for policy 0, policy_version 523270 (0.0008) [2023-12-27 09:15:40,002][20347] Updated weights for policy 0, policy_version 523280 (0.0007) [2023-12-27 09:15:40,013][20381] Updated weights for policy 1, policy_version 523029 (0.0010) [2023-12-27 09:15:40,063][20347] Updated weights for policy 0, policy_version 523290 (0.0006) [2023-12-27 09:15:40,069][20381] Updated weights for policy 1, policy_version 523039 (0.0011) [2023-12-27 09:15:40,127][20381] Updated weights for policy 1, policy_version 523049 (0.0011) [2023-12-27 09:15:40,770][20347] Updated weights for policy 0, policy_version 523300 (0.0006) [2023-12-27 09:15:40,776][20381] Updated weights for policy 1, policy_version 523059 (0.0011) [2023-12-27 09:15:40,828][20381] Updated weights for policy 1, policy_version 523069 (0.0010) [2023-12-27 09:15:40,829][20347] Updated weights for policy 0, policy_version 523310 (0.0005) [2023-12-27 09:15:40,880][20381] Updated weights for policy 1, policy_version 523079 (0.0009) [2023-12-27 09:15:40,884][20347] Updated weights for policy 0, policy_version 523320 (0.0007) [2023-12-27 09:15:41,550][20381] Updated weights for policy 1, policy_version 523089 (0.0006) [2023-12-27 09:15:41,611][20381] Updated weights for policy 1, policy_version 523099 (0.0010) [2023-12-27 09:15:41,621][20347] Updated weights for policy 0, policy_version 523330 (0.0007) [2023-12-27 09:15:41,674][20381] Updated weights for policy 1, policy_version 523109 (0.0008) [2023-12-27 09:15:41,677][20347] Updated weights for policy 0, policy_version 523340 (0.0006) [2023-12-27 09:15:41,733][20381] Updated weights for policy 1, policy_version 523119 (0.0008) [2023-12-27 09:15:41,744][20347] Updated weights for policy 0, policy_version 523350 (0.0007) [2023-12-27 09:15:41,796][20347] Updated weights for policy 0, policy_version 523360 (0.0007) [2023-12-27 09:15:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 267935744. Throughput: 0: 10252.6, 1: 10206.7. Samples: 267929356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:15:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.900')] [2023-12-27 09:15:42,435][20381] Updated weights for policy 1, policy_version 523129 (0.0010) [2023-12-27 09:15:42,494][20381] Updated weights for policy 1, policy_version 523139 (0.0010) [2023-12-27 09:15:42,509][20347] Updated weights for policy 0, policy_version 523370 (0.0010) [2023-12-27 09:15:42,547][20381] Updated weights for policy 1, policy_version 523149 (0.0010) [2023-12-27 09:15:42,566][20347] Updated weights for policy 0, policy_version 523380 (0.0006) [2023-12-27 09:15:42,621][20347] Updated weights for policy 0, policy_version 523390 (0.0005) [2023-12-27 09:15:43,203][20381] Updated weights for policy 1, policy_version 523159 (0.0010) [2023-12-27 09:15:43,245][20381] Updated weights for policy 1, policy_version 523169 (0.0010) [2023-12-27 09:15:43,271][20347] Updated weights for policy 0, policy_version 523400 (0.0007) [2023-12-27 09:15:43,297][20381] Updated weights for policy 1, policy_version 523179 (0.0005) [2023-12-27 09:15:43,323][20347] Updated weights for policy 0, policy_version 523410 (0.0007) [2023-12-27 09:15:43,380][20347] Updated weights for policy 0, policy_version 523420 (0.0008) [2023-12-27 09:15:43,861][20381] Updated weights for policy 1, policy_version 523189 (0.0008) [2023-12-27 09:15:43,909][20381] Updated weights for policy 1, policy_version 523199 (0.0010) [2023-12-27 09:15:43,963][20381] Updated weights for policy 1, policy_version 523209 (0.0010) [2023-12-27 09:15:43,970][20347] Updated weights for policy 0, policy_version 523430 (0.0006) [2023-12-27 09:15:44,027][20347] Updated weights for policy 0, policy_version 523440 (0.0005) [2023-12-27 09:15:44,082][20347] Updated weights for policy 0, policy_version 523450 (0.0005) [2023-12-27 09:15:44,629][20381] Updated weights for policy 1, policy_version 523219 (0.0009) [2023-12-27 09:15:44,644][20347] Updated weights for policy 0, policy_version 523460 (0.0006) [2023-12-27 09:15:44,684][20381] Updated weights for policy 1, policy_version 523229 (0.0006) [2023-12-27 09:15:44,702][20347] Updated weights for policy 0, policy_version 523470 (0.0005) [2023-12-27 09:15:44,743][20381] Updated weights for policy 1, policy_version 523239 (0.0007) [2023-12-27 09:15:44,764][20347] Updated weights for policy 0, policy_version 523480 (0.0008) [2023-12-27 09:15:45,364][20347] Updated weights for policy 0, policy_version 523490 (0.0007) [2023-12-27 09:15:45,433][20381] Updated weights for policy 1, policy_version 523249 (0.0008) [2023-12-27 09:15:45,434][20347] Updated weights for policy 0, policy_version 523500 (0.0008) [2023-12-27 09:15:45,486][20381] Updated weights for policy 1, policy_version 523259 (0.0006) [2023-12-27 09:15:45,500][20347] Updated weights for policy 0, policy_version 523510 (0.0008) [2023-12-27 09:15:45,542][20381] Updated weights for policy 1, policy_version 523269 (0.0005) [2023-12-27 09:15:45,563][20347] Updated weights for policy 0, policy_version 523520 (0.0008) [2023-12-27 09:15:45,604][20381] Updated weights for policy 1, policy_version 523279 (0.0007) [2023-12-27 09:15:46,181][20347] Updated weights for policy 0, policy_version 523530 (0.0010) [2023-12-27 09:15:46,239][20347] Updated weights for policy 0, policy_version 523540 (0.0010) [2023-12-27 09:15:46,259][20381] Updated weights for policy 1, policy_version 523289 (0.0006) [2023-12-27 09:15:46,293][20347] Updated weights for policy 0, policy_version 523550 (0.0010) [2023-12-27 09:15:46,312][20381] Updated weights for policy 1, policy_version 523299 (0.0007) [2023-12-27 09:15:46,368][20381] Updated weights for policy 1, policy_version 523309 (0.0008) [2023-12-27 09:15:46,900][20347] Updated weights for policy 0, policy_version 523560 (0.0005) [2023-12-27 09:15:46,963][20347] Updated weights for policy 0, policy_version 523570 (0.0005) [2023-12-27 09:15:47,019][20347] Updated weights for policy 0, policy_version 523580 (0.0009) [2023-12-27 09:15:47,025][20381] Updated weights for policy 1, policy_version 523319 (0.0006) [2023-12-27 09:15:47,071][20381] Updated weights for policy 1, policy_version 523329 (0.0006) [2023-12-27 09:15:47,125][20381] Updated weights for policy 1, policy_version 523339 (0.0007) [2023-12-27 09:15:47,362][19170] Fps is (10 sec: 21300.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 268050432. Throughput: 0: 10365.9, 1: 10217.7. Samples: 268057148. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:15:47,363][19170] Avg episode reward: [(0, '999.400'), (1, '767.130')] [2023-12-27 09:15:47,642][20347] Updated weights for policy 0, policy_version 523590 (0.0008) [2023-12-27 09:15:47,696][20347] Updated weights for policy 0, policy_version 523600 (0.0005) [2023-12-27 09:15:47,756][20347] Updated weights for policy 0, policy_version 523610 (0.0005) [2023-12-27 09:15:47,841][20381] Updated weights for policy 1, policy_version 523349 (0.0007) [2023-12-27 09:15:47,892][20381] Updated weights for policy 1, policy_version 523359 (0.0008) [2023-12-27 09:15:47,937][20381] Updated weights for policy 1, policy_version 523369 (0.0009) [2023-12-27 09:15:48,289][20347] Updated weights for policy 0, policy_version 523620 (0.0007) [2023-12-27 09:15:48,352][20347] Updated weights for policy 0, policy_version 523630 (0.0006) [2023-12-27 09:15:48,416][20347] Updated weights for policy 0, policy_version 523640 (0.0009) [2023-12-27 09:15:48,737][20381] Updated weights for policy 1, policy_version 523379 (0.0008) [2023-12-27 09:15:48,795][20381] Updated weights for policy 1, policy_version 523389 (0.0009) [2023-12-27 09:15:48,847][20381] Updated weights for policy 1, policy_version 523399 (0.0007) [2023-12-27 09:15:49,071][20347] Updated weights for policy 0, policy_version 523650 (0.0008) [2023-12-27 09:15:49,128][20347] Updated weights for policy 0, policy_version 523660 (0.0005) [2023-12-27 09:15:49,186][20347] Updated weights for policy 0, policy_version 523670 (0.0008) [2023-12-27 09:15:49,250][20347] Updated weights for policy 0, policy_version 523680 (0.0011) [2023-12-27 09:15:49,567][20381] Updated weights for policy 1, policy_version 523409 (0.0008) [2023-12-27 09:15:49,620][20381] Updated weights for policy 1, policy_version 523419 (0.0011) [2023-12-27 09:15:49,678][20381] Updated weights for policy 1, policy_version 523429 (0.0008) [2023-12-27 09:15:49,736][20381] Updated weights for policy 1, policy_version 523439 (0.0006) [2023-12-27 09:15:49,869][20347] Updated weights for policy 0, policy_version 523690 (0.0009) [2023-12-27 09:15:49,934][20347] Updated weights for policy 0, policy_version 523700 (0.0006) [2023-12-27 09:15:49,994][20347] Updated weights for policy 0, policy_version 523710 (0.0007) [2023-12-27 09:15:50,444][20381] Updated weights for policy 1, policy_version 523449 (0.0010) [2023-12-27 09:15:50,499][20381] Updated weights for policy 1, policy_version 523459 (0.0011) [2023-12-27 09:15:50,552][20381] Updated weights for policy 1, policy_version 523469 (0.0011) [2023-12-27 09:15:50,656][20347] Updated weights for policy 0, policy_version 523720 (0.0008) [2023-12-27 09:15:50,710][20347] Updated weights for policy 0, policy_version 523730 (0.0007) [2023-12-27 09:15:50,770][20347] Updated weights for policy 0, policy_version 523740 (0.0008) [2023-12-27 09:15:51,260][20381] Updated weights for policy 1, policy_version 523479 (0.0009) [2023-12-27 09:15:51,312][20381] Updated weights for policy 1, policy_version 523489 (0.0008) [2023-12-27 09:15:51,372][20381] Updated weights for policy 1, policy_version 523499 (0.0008) [2023-12-27 09:15:51,475][20347] Updated weights for policy 0, policy_version 523750 (0.0007) [2023-12-27 09:15:51,530][20347] Updated weights for policy 0, policy_version 523760 (0.0007) [2023-12-27 09:15:51,584][20347] Updated weights for policy 0, policy_version 523770 (0.0007) [2023-12-27 09:15:52,129][20381] Updated weights for policy 1, policy_version 523509 (0.0011) [2023-12-27 09:15:52,189][20381] Updated weights for policy 1, policy_version 523519 (0.0011) [2023-12-27 09:15:52,245][20381] Updated weights for policy 1, policy_version 523529 (0.0011) [2023-12-27 09:15:52,281][20347] Updated weights for policy 0, policy_version 523780 (0.0007) [2023-12-27 09:15:52,341][20347] Updated weights for policy 0, policy_version 523790 (0.0008) [2023-12-27 09:15:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 268148736. Throughput: 0: 10401.1, 1: 10173.9. Samples: 268119464. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:15:52,363][19170] Avg episode reward: [(0, '999.400'), (1, '815.800')] [2023-12-27 09:15:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000523536_134045696.pth... [2023-12-27 09:15:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000522352_133742592.pth [2023-12-27 09:15:52,401][20347] Updated weights for policy 0, policy_version 523800 (0.0008) [2023-12-27 09:15:52,443][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000523808_134111232.pth... [2023-12-27 09:15:52,447][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000522560_133791744.pth [2023-12-27 09:15:52,918][20381] Updated weights for policy 1, policy_version 523539 (0.0009) [2023-12-27 09:15:52,961][20381] Updated weights for policy 1, policy_version 523549 (0.0010) [2023-12-27 09:15:53,013][20381] Updated weights for policy 1, policy_version 523559 (0.0011) [2023-12-27 09:15:53,076][20347] Updated weights for policy 0, policy_version 523810 (0.0008) [2023-12-27 09:15:53,122][20347] Updated weights for policy 0, policy_version 523820 (0.0005) [2023-12-27 09:15:53,169][20347] Updated weights for policy 0, policy_version 523830 (0.0007) [2023-12-27 09:15:53,227][20347] Updated weights for policy 0, policy_version 523840 (0.0005) [2023-12-27 09:15:53,676][20381] Updated weights for policy 1, policy_version 523569 (0.0011) [2023-12-27 09:15:53,741][20381] Updated weights for policy 1, policy_version 523579 (0.0011) [2023-12-27 09:15:53,782][20347] Updated weights for policy 0, policy_version 523850 (0.0009) [2023-12-27 09:15:53,797][20381] Updated weights for policy 1, policy_version 523589 (0.0011) [2023-12-27 09:15:53,836][20347] Updated weights for policy 0, policy_version 523860 (0.0006) [2023-12-27 09:15:53,852][20381] Updated weights for policy 1, policy_version 523599 (0.0011) [2023-12-27 09:15:53,892][20347] Updated weights for policy 0, policy_version 523870 (0.0005) [2023-12-27 09:15:54,457][20347] Updated weights for policy 0, policy_version 523880 (0.0006) [2023-12-27 09:15:54,480][20381] Updated weights for policy 1, policy_version 523609 (0.0006) [2023-12-27 09:15:54,510][20347] Updated weights for policy 0, policy_version 523890 (0.0007) [2023-12-27 09:15:54,528][20381] Updated weights for policy 1, policy_version 523619 (0.0005) [2023-12-27 09:15:54,565][20347] Updated weights for policy 0, policy_version 523900 (0.0007) [2023-12-27 09:15:54,574][20381] Updated weights for policy 1, policy_version 523629 (0.0007) [2023-12-27 09:15:55,224][20381] Updated weights for policy 1, policy_version 523639 (0.0006) [2023-12-27 09:15:55,249][20347] Updated weights for policy 0, policy_version 523910 (0.0006) [2023-12-27 09:15:55,272][20381] Updated weights for policy 1, policy_version 523649 (0.0008) [2023-12-27 09:15:55,297][20347] Updated weights for policy 0, policy_version 523920 (0.0007) [2023-12-27 09:15:55,318][20381] Updated weights for policy 1, policy_version 523659 (0.0006) [2023-12-27 09:15:55,346][20347] Updated weights for policy 0, policy_version 523930 (0.0006) [2023-12-27 09:15:55,954][20347] Updated weights for policy 0, policy_version 523940 (0.0007) [2023-12-27 09:15:55,974][20381] Updated weights for policy 1, policy_version 523669 (0.0007) [2023-12-27 09:15:56,009][20347] Updated weights for policy 0, policy_version 523950 (0.0010) [2023-12-27 09:15:56,028][20381] Updated weights for policy 1, policy_version 523679 (0.0007) [2023-12-27 09:15:56,062][20347] Updated weights for policy 0, policy_version 523960 (0.0010) [2023-12-27 09:15:56,077][20381] Updated weights for policy 1, policy_version 523689 (0.0008) [2023-12-27 09:15:56,642][20347] Updated weights for policy 0, policy_version 523970 (0.0009) [2023-12-27 09:15:56,706][20347] Updated weights for policy 0, policy_version 523980 (0.0009) [2023-12-27 09:15:56,772][20347] Updated weights for policy 0, policy_version 523990 (0.0007) [2023-12-27 09:15:56,793][20381] Updated weights for policy 1, policy_version 523699 (0.0007) [2023-12-27 09:15:56,831][20347] Updated weights for policy 0, policy_version 524000 (0.0010) [2023-12-27 09:15:56,859][20381] Updated weights for policy 1, policy_version 523709 (0.0008) [2023-12-27 09:15:56,917][20381] Updated weights for policy 1, policy_version 523719 (0.0008) [2023-12-27 09:15:57,362][20347] Updated weights for policy 0, policy_version 524010 (0.0005) [2023-12-27 09:15:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 268255232. Throughput: 0: 10442.3, 1: 10180.5. Samples: 268245364. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:15:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '631.090')] [2023-12-27 09:15:57,427][20347] Updated weights for policy 0, policy_version 524020 (0.0005) [2023-12-27 09:15:57,477][20347] Updated weights for policy 0, policy_version 524030 (0.0009) [2023-12-27 09:15:57,599][20381] Updated weights for policy 1, policy_version 523729 (0.0007) [2023-12-27 09:15:57,647][20381] Updated weights for policy 1, policy_version 523739 (0.0008) [2023-12-27 09:15:57,703][20381] Updated weights for policy 1, policy_version 523749 (0.0006) [2023-12-27 09:15:57,758][20381] Updated weights for policy 1, policy_version 523759 (0.0006) [2023-12-27 09:15:58,074][20347] Updated weights for policy 0, policy_version 524040 (0.0006) [2023-12-27 09:15:58,120][20347] Updated weights for policy 0, policy_version 524050 (0.0005) [2023-12-27 09:15:58,174][20347] Updated weights for policy 0, policy_version 524060 (0.0006) [2023-12-27 09:15:58,409][20381] Updated weights for policy 1, policy_version 523769 (0.0008) [2023-12-27 09:15:58,461][20381] Updated weights for policy 1, policy_version 523779 (0.0008) [2023-12-27 09:15:58,513][20381] Updated weights for policy 1, policy_version 523789 (0.0008) [2023-12-27 09:15:58,952][20347] Updated weights for policy 0, policy_version 524070 (0.0006) [2023-12-27 09:15:59,007][20347] Updated weights for policy 0, policy_version 524080 (0.0007) [2023-12-27 09:15:59,066][20347] Updated weights for policy 0, policy_version 524090 (0.0007) [2023-12-27 09:15:59,314][20381] Updated weights for policy 1, policy_version 523799 (0.0008) [2023-12-27 09:15:59,376][20381] Updated weights for policy 1, policy_version 523809 (0.0007) [2023-12-27 09:15:59,432][20381] Updated weights for policy 1, policy_version 523819 (0.0008) [2023-12-27 09:15:59,850][20347] Updated weights for policy 0, policy_version 524100 (0.0008) [2023-12-27 09:15:59,906][20347] Updated weights for policy 0, policy_version 524110 (0.0008) [2023-12-27 09:15:59,972][20347] Updated weights for policy 0, policy_version 524120 (0.0008) [2023-12-27 09:16:00,097][20381] Updated weights for policy 1, policy_version 523829 (0.0006) [2023-12-27 09:16:00,145][20381] Updated weights for policy 1, policy_version 523839 (0.0005) [2023-12-27 09:16:00,202][20381] Updated weights for policy 1, policy_version 523849 (0.0007) [2023-12-27 09:16:00,627][20347] Updated weights for policy 0, policy_version 524130 (0.0008) [2023-12-27 09:16:00,677][20347] Updated weights for policy 0, policy_version 524140 (0.0007) [2023-12-27 09:16:00,731][20347] Updated weights for policy 0, policy_version 524150 (0.0008) [2023-12-27 09:16:00,778][20347] Updated weights for policy 0, policy_version 524160 (0.0010) [2023-12-27 09:16:00,864][20381] Updated weights for policy 1, policy_version 523859 (0.0007) [2023-12-27 09:16:00,912][20381] Updated weights for policy 1, policy_version 523869 (0.0007) [2023-12-27 09:16:00,961][20381] Updated weights for policy 1, policy_version 523879 (0.0008) [2023-12-27 09:16:01,501][20347] Updated weights for policy 0, policy_version 524170 (0.0006) [2023-12-27 09:16:01,560][20347] Updated weights for policy 0, policy_version 524180 (0.0005) [2023-12-27 09:16:01,624][20347] Updated weights for policy 0, policy_version 524190 (0.0006) [2023-12-27 09:16:01,650][20381] Updated weights for policy 1, policy_version 523889 (0.0008) [2023-12-27 09:16:01,708][20381] Updated weights for policy 1, policy_version 523899 (0.0007) [2023-12-27 09:16:01,775][20381] Updated weights for policy 1, policy_version 523909 (0.0008) [2023-12-27 09:16:01,833][20381] Updated weights for policy 1, policy_version 523919 (0.0006) [2023-12-27 09:16:02,260][20347] Updated weights for policy 0, policy_version 524200 (0.0007) [2023-12-27 09:16:02,322][20347] Updated weights for policy 0, policy_version 524210 (0.0008) [2023-12-27 09:16:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 268353536. Throughput: 0: 10456.4, 1: 10187.6. Samples: 268370324. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:02,363][19170] Avg episode reward: [(0, '906.110'), (1, '746.490')] [2023-12-27 09:16:02,380][20347] Updated weights for policy 0, policy_version 524220 (0.0007) [2023-12-27 09:16:02,409][20381] Updated weights for policy 1, policy_version 523929 (0.0008) [2023-12-27 09:16:02,469][20381] Updated weights for policy 1, policy_version 523939 (0.0007) [2023-12-27 09:16:02,526][20381] Updated weights for policy 1, policy_version 523949 (0.0008) [2023-12-27 09:16:03,071][20347] Updated weights for policy 0, policy_version 524230 (0.0007) [2023-12-27 09:16:03,124][20347] Updated weights for policy 0, policy_version 524240 (0.0007) [2023-12-27 09:16:03,180][20381] Updated weights for policy 1, policy_version 523959 (0.0007) [2023-12-27 09:16:03,182][20347] Updated weights for policy 0, policy_version 524250 (0.0006) [2023-12-27 09:16:03,231][20381] Updated weights for policy 1, policy_version 523969 (0.0005) [2023-12-27 09:16:03,292][20381] Updated weights for policy 1, policy_version 523979 (0.0006) [2023-12-27 09:16:03,814][20347] Updated weights for policy 0, policy_version 524260 (0.0006) [2023-12-27 09:16:03,874][20347] Updated weights for policy 0, policy_version 524270 (0.0007) [2023-12-27 09:16:03,903][20381] Updated weights for policy 1, policy_version 523989 (0.0007) [2023-12-27 09:16:03,927][20347] Updated weights for policy 0, policy_version 524280 (0.0006) [2023-12-27 09:16:03,956][20381] Updated weights for policy 1, policy_version 523999 (0.0008) [2023-12-27 09:16:04,008][20381] Updated weights for policy 1, policy_version 524009 (0.0008) [2023-12-27 09:16:04,653][20347] Updated weights for policy 0, policy_version 524290 (0.0007) [2023-12-27 09:16:04,700][20381] Updated weights for policy 1, policy_version 524019 (0.0007) [2023-12-27 09:16:04,702][20347] Updated weights for policy 0, policy_version 524300 (0.0008) [2023-12-27 09:16:04,761][20381] Updated weights for policy 1, policy_version 524029 (0.0006) [2023-12-27 09:16:04,763][20347] Updated weights for policy 0, policy_version 524310 (0.0007) [2023-12-27 09:16:04,819][20381] Updated weights for policy 1, policy_version 524039 (0.0005) [2023-12-27 09:16:04,821][20347] Updated weights for policy 0, policy_version 524320 (0.0007) [2023-12-27 09:16:05,405][20347] Updated weights for policy 0, policy_version 524330 (0.0008) [2023-12-27 09:16:05,450][20347] Updated weights for policy 0, policy_version 524340 (0.0009) [2023-12-27 09:16:05,494][20381] Updated weights for policy 1, policy_version 524049 (0.0007) [2023-12-27 09:16:05,495][20347] Updated weights for policy 0, policy_version 524350 (0.0007) [2023-12-27 09:16:05,544][20381] Updated weights for policy 1, policy_version 524059 (0.0007) [2023-12-27 09:16:05,592][20381] Updated weights for policy 1, policy_version 524069 (0.0008) [2023-12-27 09:16:05,642][20381] Updated weights for policy 1, policy_version 524079 (0.0008) [2023-12-27 09:16:06,155][20347] Updated weights for policy 0, policy_version 524360 (0.0008) [2023-12-27 09:16:06,217][20347] Updated weights for policy 0, policy_version 524370 (0.0009) [2023-12-27 09:16:06,283][20347] Updated weights for policy 0, policy_version 524380 (0.0011) [2023-12-27 09:16:06,374][20381] Updated weights for policy 1, policy_version 524089 (0.0010) [2023-12-27 09:16:06,431][20381] Updated weights for policy 1, policy_version 524099 (0.0010) [2023-12-27 09:16:06,488][20381] Updated weights for policy 1, policy_version 524109 (0.0006) [2023-12-27 09:16:06,965][20347] Updated weights for policy 0, policy_version 524390 (0.0008) [2023-12-27 09:16:07,019][20347] Updated weights for policy 0, policy_version 524400 (0.0008) [2023-12-27 09:16:07,074][20347] Updated weights for policy 0, policy_version 524410 (0.0007) [2023-12-27 09:16:07,163][20381] Updated weights for policy 1, policy_version 524119 (0.0009) [2023-12-27 09:16:07,224][20381] Updated weights for policy 1, policy_version 524129 (0.0008) [2023-12-27 09:16:07,283][20381] Updated weights for policy 1, policy_version 524139 (0.0008) [2023-12-27 09:16:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 268468224. Throughput: 0: 10478.2, 1: 10198.1. Samples: 268431612. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:07,363][19170] Avg episode reward: [(0, '890.770'), (1, '681.780')] [2023-12-27 09:16:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000524144_134201344.pth... [2023-12-27 09:16:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000524416_134266880.pth... [2023-12-27 09:16:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000522928_133890048.pth [2023-12-27 09:16:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000523168_133947392.pth [2023-12-27 09:16:07,770][20347] Updated weights for policy 0, policy_version 524420 (0.0008) [2023-12-27 09:16:07,815][20347] Updated weights for policy 0, policy_version 524430 (0.0007) [2023-12-27 09:16:07,865][20347] Updated weights for policy 0, policy_version 524440 (0.0007) [2023-12-27 09:16:07,970][20381] Updated weights for policy 1, policy_version 524149 (0.0009) [2023-12-27 09:16:08,018][20381] Updated weights for policy 1, policy_version 524159 (0.0008) [2023-12-27 09:16:08,068][20381] Updated weights for policy 1, policy_version 524169 (0.0007) [2023-12-27 09:16:08,537][20347] Updated weights for policy 0, policy_version 524450 (0.0006) [2023-12-27 09:16:08,596][20347] Updated weights for policy 0, policy_version 524460 (0.0005) [2023-12-27 09:16:08,652][20347] Updated weights for policy 0, policy_version 524470 (0.0006) [2023-12-27 09:16:08,709][20347] Updated weights for policy 0, policy_version 524480 (0.0006) [2023-12-27 09:16:08,775][20381] Updated weights for policy 1, policy_version 524179 (0.0009) [2023-12-27 09:16:08,824][20381] Updated weights for policy 1, policy_version 524189 (0.0005) [2023-12-27 09:16:08,878][20381] Updated weights for policy 1, policy_version 524199 (0.0009) [2023-12-27 09:16:09,485][20347] Updated weights for policy 0, policy_version 524490 (0.0009) [2023-12-27 09:16:09,499][20381] Updated weights for policy 1, policy_version 524209 (0.0010) [2023-12-27 09:16:09,546][20347] Updated weights for policy 0, policy_version 524500 (0.0006) [2023-12-27 09:16:09,548][20381] Updated weights for policy 1, policy_version 524219 (0.0008) [2023-12-27 09:16:09,597][20381] Updated weights for policy 1, policy_version 524229 (0.0011) [2023-12-27 09:16:09,607][20347] Updated weights for policy 0, policy_version 524510 (0.0006) [2023-12-27 09:16:09,654][20381] Updated weights for policy 1, policy_version 524239 (0.0011) [2023-12-27 09:16:10,312][20347] Updated weights for policy 0, policy_version 524520 (0.0007) [2023-12-27 09:16:10,369][20381] Updated weights for policy 1, policy_version 524249 (0.0011) [2023-12-27 09:16:10,372][20347] Updated weights for policy 0, policy_version 524530 (0.0006) [2023-12-27 09:16:10,446][20381] Updated weights for policy 1, policy_version 524260 (0.0010) [2023-12-27 09:16:10,449][20347] Updated weights for policy 0, policy_version 524540 (0.0006) [2023-12-27 09:16:10,499][20381] Updated weights for policy 1, policy_version 524270 (0.0007) [2023-12-27 09:16:11,092][20347] Updated weights for policy 0, policy_version 524550 (0.0007) [2023-12-27 09:16:11,152][20347] Updated weights for policy 0, policy_version 524560 (0.0010) [2023-12-27 09:16:11,204][20381] Updated weights for policy 1, policy_version 524280 (0.0010) [2023-12-27 09:16:11,214][20347] Updated weights for policy 0, policy_version 524570 (0.0009) [2023-12-27 09:16:11,268][20381] Updated weights for policy 1, policy_version 524290 (0.0011) [2023-12-27 09:16:11,323][20381] Updated weights for policy 1, policy_version 524300 (0.0010) [2023-12-27 09:16:11,927][20347] Updated weights for policy 0, policy_version 524580 (0.0009) [2023-12-27 09:16:11,985][20347] Updated weights for policy 0, policy_version 524590 (0.0006) [2023-12-27 09:16:12,029][20381] Updated weights for policy 1, policy_version 524310 (0.0008) [2023-12-27 09:16:12,035][20347] Updated weights for policy 0, policy_version 524600 (0.0006) [2023-12-27 09:16:12,096][20381] Updated weights for policy 1, policy_version 524320 (0.0008) [2023-12-27 09:16:12,157][20381] Updated weights for policy 1, policy_version 524330 (0.0007) [2023-12-27 09:16:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 268566528. Throughput: 0: 10530.6, 1: 10227.4. Samples: 268554052. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:12,363][19170] Avg episode reward: [(0, '978.780'), (1, '658.300')] [2023-12-27 09:16:12,698][20347] Updated weights for policy 0, policy_version 524610 (0.0007) [2023-12-27 09:16:12,748][20347] Updated weights for policy 0, policy_version 524620 (0.0007) [2023-12-27 09:16:12,810][20347] Updated weights for policy 0, policy_version 524630 (0.0005) [2023-12-27 09:16:12,866][20347] Updated weights for policy 0, policy_version 524640 (0.0005) [2023-12-27 09:16:12,878][20381] Updated weights for policy 1, policy_version 524340 (0.0008) [2023-12-27 09:16:12,940][20381] Updated weights for policy 1, policy_version 524350 (0.0009) [2023-12-27 09:16:12,995][20381] Updated weights for policy 1, policy_version 524360 (0.0008) [2023-12-27 09:16:13,476][20347] Updated weights for policy 0, policy_version 524650 (0.0007) [2023-12-27 09:16:13,533][20347] Updated weights for policy 0, policy_version 524660 (0.0007) [2023-12-27 09:16:13,593][20347] Updated weights for policy 0, policy_version 524670 (0.0007) [2023-12-27 09:16:13,715][20381] Updated weights for policy 1, policy_version 524370 (0.0008) [2023-12-27 09:16:13,767][20381] Updated weights for policy 1, policy_version 524380 (0.0010) [2023-12-27 09:16:13,824][20381] Updated weights for policy 1, policy_version 524390 (0.0011) [2023-12-27 09:16:13,880][20381] Updated weights for policy 1, policy_version 524400 (0.0010) [2023-12-27 09:16:14,292][20347] Updated weights for policy 0, policy_version 524680 (0.0007) [2023-12-27 09:16:14,346][20347] Updated weights for policy 0, policy_version 524690 (0.0007) [2023-12-27 09:16:14,400][20347] Updated weights for policy 0, policy_version 524700 (0.0007) [2023-12-27 09:16:14,573][20381] Updated weights for policy 1, policy_version 524410 (0.0010) [2023-12-27 09:16:14,634][20381] Updated weights for policy 1, policy_version 524420 (0.0010) [2023-12-27 09:16:14,697][20381] Updated weights for policy 1, policy_version 524430 (0.0010) [2023-12-27 09:16:15,109][20347] Updated weights for policy 0, policy_version 524710 (0.0007) [2023-12-27 09:16:15,167][20347] Updated weights for policy 0, policy_version 524720 (0.0007) [2023-12-27 09:16:15,211][20347] Updated weights for policy 0, policy_version 524730 (0.0007) [2023-12-27 09:16:15,382][20381] Updated weights for policy 1, policy_version 524440 (0.0010) [2023-12-27 09:16:15,432][20381] Updated weights for policy 1, policy_version 524450 (0.0010) [2023-12-27 09:16:15,491][20381] Updated weights for policy 1, policy_version 524460 (0.0010) [2023-12-27 09:16:15,903][20347] Updated weights for policy 0, policy_version 524740 (0.0007) [2023-12-27 09:16:15,965][20347] Updated weights for policy 0, policy_version 524750 (0.0005) [2023-12-27 09:16:16,015][20347] Updated weights for policy 0, policy_version 524760 (0.0005) [2023-12-27 09:16:16,116][20381] Updated weights for policy 1, policy_version 524470 (0.0007) [2023-12-27 09:16:16,170][20381] Updated weights for policy 1, policy_version 524480 (0.0006) [2023-12-27 09:16:16,229][20381] Updated weights for policy 1, policy_version 524490 (0.0007) [2023-12-27 09:16:16,642][20347] Updated weights for policy 0, policy_version 524770 (0.0006) [2023-12-27 09:16:16,690][20347] Updated weights for policy 0, policy_version 524780 (0.0008) [2023-12-27 09:16:16,747][20347] Updated weights for policy 0, policy_version 524790 (0.0007) [2023-12-27 09:16:16,803][20347] Updated weights for policy 0, policy_version 524800 (0.0007) [2023-12-27 09:16:16,879][20381] Updated weights for policy 1, policy_version 524500 (0.0008) [2023-12-27 09:16:16,927][20381] Updated weights for policy 1, policy_version 524510 (0.0007) [2023-12-27 09:16:16,988][20381] Updated weights for policy 1, policy_version 524520 (0.0008) [2023-12-27 09:16:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 268664832. Throughput: 0: 10433.3, 1: 10220.7. Samples: 268675828. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '668.660')] [2023-12-27 09:16:17,509][20347] Updated weights for policy 0, policy_version 524810 (0.0007) [2023-12-27 09:16:17,559][20347] Updated weights for policy 0, policy_version 524820 (0.0007) [2023-12-27 09:16:17,610][20347] Updated weights for policy 0, policy_version 524830 (0.0007) [2023-12-27 09:16:17,655][20381] Updated weights for policy 1, policy_version 524530 (0.0010) [2023-12-27 09:16:17,720][20381] Updated weights for policy 1, policy_version 524540 (0.0010) [2023-12-27 09:16:17,772][20381] Updated weights for policy 1, policy_version 524550 (0.0010) [2023-12-27 09:16:17,821][20381] Updated weights for policy 1, policy_version 524560 (0.0010) [2023-12-27 09:16:18,243][20347] Updated weights for policy 0, policy_version 524840 (0.0006) [2023-12-27 09:16:18,308][20347] Updated weights for policy 0, policy_version 524850 (0.0005) [2023-12-27 09:16:18,368][20347] Updated weights for policy 0, policy_version 524860 (0.0008) [2023-12-27 09:16:18,472][20381] Updated weights for policy 1, policy_version 524570 (0.0011) [2023-12-27 09:16:18,534][20381] Updated weights for policy 1, policy_version 524580 (0.0011) [2023-12-27 09:16:18,593][20381] Updated weights for policy 1, policy_version 524590 (0.0011) [2023-12-27 09:16:18,984][20347] Updated weights for policy 0, policy_version 524870 (0.0007) [2023-12-27 09:16:19,042][20347] Updated weights for policy 0, policy_version 524880 (0.0005) [2023-12-27 09:16:19,098][20347] Updated weights for policy 0, policy_version 524890 (0.0005) [2023-12-27 09:16:19,249][20381] Updated weights for policy 1, policy_version 524600 (0.0013) [2023-12-27 09:16:19,306][20381] Updated weights for policy 1, policy_version 524610 (0.0008) [2023-12-27 09:16:19,370][20381] Updated weights for policy 1, policy_version 524620 (0.0007) [2023-12-27 09:16:19,685][20347] Updated weights for policy 0, policy_version 524900 (0.0006) [2023-12-27 09:16:19,747][20347] Updated weights for policy 0, policy_version 524910 (0.0008) [2023-12-27 09:16:19,803][20347] Updated weights for policy 0, policy_version 524920 (0.0007) [2023-12-27 09:16:20,121][20381] Updated weights for policy 1, policy_version 524630 (0.0007) [2023-12-27 09:16:20,178][20381] Updated weights for policy 1, policy_version 524640 (0.0008) [2023-12-27 09:16:20,227][20381] Updated weights for policy 1, policy_version 524650 (0.0007) [2023-12-27 09:16:20,506][20347] Updated weights for policy 0, policy_version 524930 (0.0008) [2023-12-27 09:16:20,558][20347] Updated weights for policy 0, policy_version 524940 (0.0006) [2023-12-27 09:16:20,624][20347] Updated weights for policy 0, policy_version 524950 (0.0008) [2023-12-27 09:16:20,689][20347] Updated weights for policy 0, policy_version 524960 (0.0008) [2023-12-27 09:16:20,915][20381] Updated weights for policy 1, policy_version 524660 (0.0007) [2023-12-27 09:16:20,972][20381] Updated weights for policy 1, policy_version 524670 (0.0008) [2023-12-27 09:16:21,034][20381] Updated weights for policy 1, policy_version 524680 (0.0008) [2023-12-27 09:16:21,389][20347] Updated weights for policy 0, policy_version 524970 (0.0009) [2023-12-27 09:16:21,452][20347] Updated weights for policy 0, policy_version 524980 (0.0007) [2023-12-27 09:16:21,507][20347] Updated weights for policy 0, policy_version 524990 (0.0007) [2023-12-27 09:16:21,756][20381] Updated weights for policy 1, policy_version 524690 (0.0008) [2023-12-27 09:16:21,820][20381] Updated weights for policy 1, policy_version 524700 (0.0008) [2023-12-27 09:16:21,883][20381] Updated weights for policy 1, policy_version 524710 (0.0008) [2023-12-27 09:16:21,939][20381] Updated weights for policy 1, policy_version 524720 (0.0008) [2023-12-27 09:16:22,194][20347] Updated weights for policy 0, policy_version 525000 (0.0006) [2023-12-27 09:16:22,247][20347] Updated weights for policy 0, policy_version 525010 (0.0006) [2023-12-27 09:16:22,311][20347] Updated weights for policy 0, policy_version 525020 (0.0006) [2023-12-27 09:16:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 268771328. Throughput: 0: 10460.6, 1: 10195.9. Samples: 268738012. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '698.070')] [2023-12-27 09:16:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000524720_134348800.pth... [2023-12-27 09:16:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000525024_134422528.pth... [2023-12-27 09:16:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000523536_134045696.pth [2023-12-27 09:16:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000523808_134111232.pth [2023-12-27 09:16:22,688][20381] Updated weights for policy 1, policy_version 524730 (0.0008) [2023-12-27 09:16:22,735][20381] Updated weights for policy 1, policy_version 524740 (0.0007) [2023-12-27 09:16:22,788][20381] Updated weights for policy 1, policy_version 524750 (0.0009) [2023-12-27 09:16:22,902][20347] Updated weights for policy 0, policy_version 525030 (0.0007) [2023-12-27 09:16:22,959][20347] Updated weights for policy 0, policy_version 525040 (0.0005) [2023-12-27 09:16:23,008][20347] Updated weights for policy 0, policy_version 525050 (0.0005) [2023-12-27 09:16:23,468][20381] Updated weights for policy 1, policy_version 524760 (0.0006) [2023-12-27 09:16:23,526][20381] Updated weights for policy 1, policy_version 524770 (0.0005) [2023-12-27 09:16:23,542][20347] Updated weights for policy 0, policy_version 525060 (0.0005) [2023-12-27 09:16:23,584][20381] Updated weights for policy 1, policy_version 524780 (0.0007) [2023-12-27 09:16:23,605][20347] Updated weights for policy 0, policy_version 525070 (0.0005) [2023-12-27 09:16:23,666][20347] Updated weights for policy 0, policy_version 525080 (0.0007) [2023-12-27 09:16:24,226][20381] Updated weights for policy 1, policy_version 524790 (0.0007) [2023-12-27 09:16:24,274][20381] Updated weights for policy 1, policy_version 524800 (0.0008) [2023-12-27 09:16:24,308][20347] Updated weights for policy 0, policy_version 525090 (0.0007) [2023-12-27 09:16:24,332][20381] Updated weights for policy 1, policy_version 524810 (0.0007) [2023-12-27 09:16:24,366][20347] Updated weights for policy 0, policy_version 525100 (0.0007) [2023-12-27 09:16:24,419][20347] Updated weights for policy 0, policy_version 525110 (0.0007) [2023-12-27 09:16:24,466][20347] Updated weights for policy 0, policy_version 525120 (0.0006) [2023-12-27 09:16:24,974][20381] Updated weights for policy 1, policy_version 524820 (0.0007) [2023-12-27 09:16:25,027][20381] Updated weights for policy 1, policy_version 524830 (0.0006) [2023-12-27 09:16:25,083][20381] Updated weights for policy 1, policy_version 524840 (0.0007) [2023-12-27 09:16:25,155][20347] Updated weights for policy 0, policy_version 525130 (0.0008) [2023-12-27 09:16:25,206][20347] Updated weights for policy 0, policy_version 525140 (0.0007) [2023-12-27 09:16:25,267][20347] Updated weights for policy 0, policy_version 525150 (0.0006) [2023-12-27 09:16:25,747][20381] Updated weights for policy 1, policy_version 524850 (0.0006) [2023-12-27 09:16:25,806][20381] Updated weights for policy 1, policy_version 524860 (0.0006) [2023-12-27 09:16:25,866][20381] Updated weights for policy 1, policy_version 524870 (0.0009) [2023-12-27 09:16:25,911][20381] Updated weights for policy 1, policy_version 524880 (0.0005) [2023-12-27 09:16:25,920][20347] Updated weights for policy 0, policy_version 525160 (0.0009) [2023-12-27 09:16:25,981][20347] Updated weights for policy 0, policy_version 525170 (0.0010) [2023-12-27 09:16:26,039][20347] Updated weights for policy 0, policy_version 525180 (0.0007) [2023-12-27 09:16:26,598][20381] Updated weights for policy 1, policy_version 524890 (0.0005) [2023-12-27 09:16:26,650][20347] Updated weights for policy 0, policy_version 525190 (0.0007) [2023-12-27 09:16:26,653][20381] Updated weights for policy 1, policy_version 524900 (0.0005) [2023-12-27 09:16:26,698][20347] Updated weights for policy 0, policy_version 525200 (0.0007) [2023-12-27 09:16:26,707][20381] Updated weights for policy 1, policy_version 524910 (0.0006) [2023-12-27 09:16:26,751][20347] Updated weights for policy 0, policy_version 525210 (0.0007) [2023-12-27 09:16:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 268869632. Throughput: 0: 10543.4, 1: 10197.0. Samples: 268862672. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:27,363][20381] Updated weights for policy 1, policy_version 524920 (0.0007) [2023-12-27 09:16:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '691.850')] [2023-12-27 09:16:27,399][20347] Updated weights for policy 0, policy_version 525220 (0.0005) [2023-12-27 09:16:27,419][20381] Updated weights for policy 1, policy_version 524930 (0.0006) [2023-12-27 09:16:27,457][20347] Updated weights for policy 0, policy_version 525230 (0.0006) [2023-12-27 09:16:27,474][20381] Updated weights for policy 1, policy_version 524940 (0.0005) [2023-12-27 09:16:27,512][20347] Updated weights for policy 0, policy_version 525240 (0.0006) [2023-12-27 09:16:28,113][20347] Updated weights for policy 0, policy_version 525250 (0.0008) [2023-12-27 09:16:28,168][20381] Updated weights for policy 1, policy_version 524950 (0.0005) [2023-12-27 09:16:28,172][20347] Updated weights for policy 0, policy_version 525260 (0.0007) [2023-12-27 09:16:28,220][20381] Updated weights for policy 1, policy_version 524960 (0.0008) [2023-12-27 09:16:28,225][20347] Updated weights for policy 0, policy_version 525270 (0.0007) [2023-12-27 09:16:28,271][20381] Updated weights for policy 1, policy_version 524970 (0.0005) [2023-12-27 09:16:28,282][20347] Updated weights for policy 0, policy_version 525280 (0.0006) [2023-12-27 09:16:28,972][20347] Updated weights for policy 0, policy_version 525290 (0.0006) [2023-12-27 09:16:28,978][20381] Updated weights for policy 1, policy_version 524980 (0.0007) [2023-12-27 09:16:29,024][20347] Updated weights for policy 0, policy_version 525300 (0.0008) [2023-12-27 09:16:29,037][20381] Updated weights for policy 1, policy_version 524990 (0.0005) [2023-12-27 09:16:29,079][20347] Updated weights for policy 0, policy_version 525310 (0.0007) [2023-12-27 09:16:29,091][20381] Updated weights for policy 1, policy_version 525000 (0.0005) [2023-12-27 09:16:29,661][20381] Updated weights for policy 1, policy_version 525010 (0.0005) [2023-12-27 09:16:29,718][20381] Updated weights for policy 1, policy_version 525020 (0.0007) [2023-12-27 09:16:29,775][20381] Updated weights for policy 1, policy_version 525030 (0.0007) [2023-12-27 09:16:29,792][20347] Updated weights for policy 0, policy_version 525320 (0.0006) [2023-12-27 09:16:29,833][20381] Updated weights for policy 1, policy_version 525040 (0.0006) [2023-12-27 09:16:29,852][20347] Updated weights for policy 0, policy_version 525330 (0.0007) [2023-12-27 09:16:29,911][20347] Updated weights for policy 0, policy_version 525340 (0.0008) [2023-12-27 09:16:30,524][20381] Updated weights for policy 1, policy_version 525050 (0.0007) [2023-12-27 09:16:30,568][20347] Updated weights for policy 0, policy_version 525350 (0.0006) [2023-12-27 09:16:30,580][20381] Updated weights for policy 1, policy_version 525060 (0.0006) [2023-12-27 09:16:30,619][20347] Updated weights for policy 0, policy_version 525360 (0.0006) [2023-12-27 09:16:30,628][20381] Updated weights for policy 1, policy_version 525070 (0.0005) [2023-12-27 09:16:30,674][20347] Updated weights for policy 0, policy_version 525370 (0.0007) [2023-12-27 09:16:31,295][20381] Updated weights for policy 1, policy_version 525080 (0.0007) [2023-12-27 09:16:31,356][20381] Updated weights for policy 1, policy_version 525090 (0.0007) [2023-12-27 09:16:31,413][20381] Updated weights for policy 1, policy_version 525100 (0.0008) [2023-12-27 09:16:31,435][20347] Updated weights for policy 0, policy_version 525380 (0.0006) [2023-12-27 09:16:31,489][20347] Updated weights for policy 0, policy_version 525390 (0.0007) [2023-12-27 09:16:31,533][20347] Updated weights for policy 0, policy_version 525400 (0.0007) [2023-12-27 09:16:32,145][20381] Updated weights for policy 1, policy_version 525110 (0.0007) [2023-12-27 09:16:32,181][20347] Updated weights for policy 0, policy_version 525410 (0.0006) [2023-12-27 09:16:32,199][20381] Updated weights for policy 1, policy_version 525120 (0.0006) [2023-12-27 09:16:32,239][20347] Updated weights for policy 0, policy_version 525420 (0.0006) [2023-12-27 09:16:32,253][20381] Updated weights for policy 1, policy_version 525130 (0.0006) [2023-12-27 09:16:32,302][20347] Updated weights for policy 0, policy_version 525430 (0.0006) [2023-12-27 09:16:32,361][20347] Updated weights for policy 0, policy_version 525440 (0.0008) [2023-12-27 09:16:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 268984320. Throughput: 0: 10498.4, 1: 10177.6. Samples: 268987568. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '776.110')] [2023-12-27 09:16:32,957][20381] Updated weights for policy 1, policy_version 525140 (0.0006) [2023-12-27 09:16:33,006][20347] Updated weights for policy 0, policy_version 525450 (0.0005) [2023-12-27 09:16:33,018][20381] Updated weights for policy 1, policy_version 525150 (0.0008) [2023-12-27 09:16:33,066][20347] Updated weights for policy 0, policy_version 525460 (0.0005) [2023-12-27 09:16:33,075][20381] Updated weights for policy 1, policy_version 525160 (0.0009) [2023-12-27 09:16:33,125][20347] Updated weights for policy 0, policy_version 525470 (0.0005) [2023-12-27 09:16:33,661][20347] Updated weights for policy 0, policy_version 525480 (0.0005) [2023-12-27 09:16:33,725][20347] Updated weights for policy 0, policy_version 525490 (0.0005) [2023-12-27 09:16:33,778][20347] Updated weights for policy 0, policy_version 525500 (0.0005) [2023-12-27 09:16:33,804][20381] Updated weights for policy 1, policy_version 525170 (0.0007) [2023-12-27 09:16:33,860][20381] Updated weights for policy 1, policy_version 525180 (0.0009) [2023-12-27 09:16:33,923][20381] Updated weights for policy 1, policy_version 525190 (0.0008) [2023-12-27 09:16:33,972][20381] Updated weights for policy 1, policy_version 525200 (0.0008) [2023-12-27 09:16:34,336][20347] Updated weights for policy 0, policy_version 525510 (0.0007) [2023-12-27 09:16:34,389][20347] Updated weights for policy 0, policy_version 525520 (0.0007) [2023-12-27 09:16:34,446][20347] Updated weights for policy 0, policy_version 525530 (0.0006) [2023-12-27 09:16:34,687][20381] Updated weights for policy 1, policy_version 525210 (0.0008) [2023-12-27 09:16:34,745][20381] Updated weights for policy 1, policy_version 525220 (0.0007) [2023-12-27 09:16:34,800][20381] Updated weights for policy 1, policy_version 525230 (0.0005) [2023-12-27 09:16:35,141][20347] Updated weights for policy 0, policy_version 525540 (0.0007) [2023-12-27 09:16:35,200][20347] Updated weights for policy 0, policy_version 525550 (0.0010) [2023-12-27 09:16:35,253][20347] Updated weights for policy 0, policy_version 525560 (0.0010) [2023-12-27 09:16:35,471][20381] Updated weights for policy 1, policy_version 525240 (0.0010) [2023-12-27 09:16:35,528][20381] Updated weights for policy 1, policy_version 525250 (0.0010) [2023-12-27 09:16:35,588][20381] Updated weights for policy 1, policy_version 525260 (0.0010) [2023-12-27 09:16:35,867][20347] Updated weights for policy 0, policy_version 525570 (0.0008) [2023-12-27 09:16:35,925][20347] Updated weights for policy 0, policy_version 525580 (0.0010) [2023-12-27 09:16:35,980][20347] Updated weights for policy 0, policy_version 525590 (0.0010) [2023-12-27 09:16:36,031][20347] Updated weights for policy 0, policy_version 525600 (0.0010) [2023-12-27 09:16:36,199][20381] Updated weights for policy 1, policy_version 525270 (0.0009) [2023-12-27 09:16:36,249][20381] Updated weights for policy 1, policy_version 525280 (0.0007) [2023-12-27 09:16:36,307][20381] Updated weights for policy 1, policy_version 525290 (0.0007) [2023-12-27 09:16:36,730][20347] Updated weights for policy 0, policy_version 525610 (0.0006) [2023-12-27 09:16:36,787][20347] Updated weights for policy 0, policy_version 525620 (0.0008) [2023-12-27 09:16:36,850][20347] Updated weights for policy 0, policy_version 525630 (0.0008) [2023-12-27 09:16:37,001][20381] Updated weights for policy 1, policy_version 525300 (0.0010) [2023-12-27 09:16:37,059][20381] Updated weights for policy 1, policy_version 525310 (0.0007) [2023-12-27 09:16:37,116][20381] Updated weights for policy 1, policy_version 525320 (0.0007) [2023-12-27 09:16:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 269082624. Throughput: 0: 10506.4, 1: 10183.9. Samples: 269050528. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '731.220')] [2023-12-27 09:16:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000525328_134504448.pth... [2023-12-27 09:16:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000525632_134578176.pth... [2023-12-27 09:16:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000524144_134201344.pth [2023-12-27 09:16:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000524416_134266880.pth [2023-12-27 09:16:37,630][20347] Updated weights for policy 0, policy_version 525640 (0.0009) [2023-12-27 09:16:37,686][20347] Updated weights for policy 0, policy_version 525650 (0.0008) [2023-12-27 09:16:37,738][20347] Updated weights for policy 0, policy_version 525660 (0.0008) [2023-12-27 09:16:37,749][20381] Updated weights for policy 1, policy_version 525330 (0.0006) [2023-12-27 09:16:37,810][20381] Updated weights for policy 1, policy_version 525340 (0.0006) [2023-12-27 09:16:37,873][20381] Updated weights for policy 1, policy_version 525350 (0.0005) [2023-12-27 09:16:37,927][20381] Updated weights for policy 1, policy_version 525360 (0.0010) [2023-12-27 09:16:38,403][20347] Updated weights for policy 0, policy_version 525670 (0.0009) [2023-12-27 09:16:38,463][20347] Updated weights for policy 0, policy_version 525680 (0.0008) [2023-12-27 09:16:38,522][20347] Updated weights for policy 0, policy_version 525690 (0.0007) [2023-12-27 09:16:38,587][20381] Updated weights for policy 1, policy_version 525370 (0.0006) [2023-12-27 09:16:38,649][20381] Updated weights for policy 1, policy_version 525380 (0.0008) [2023-12-27 09:16:38,705][20381] Updated weights for policy 1, policy_version 525390 (0.0007) [2023-12-27 09:16:39,182][20347] Updated weights for policy 0, policy_version 525700 (0.0007) [2023-12-27 09:16:39,250][20347] Updated weights for policy 0, policy_version 525710 (0.0006) [2023-12-27 09:16:39,302][20347] Updated weights for policy 0, policy_version 525720 (0.0006) [2023-12-27 09:16:39,398][20381] Updated weights for policy 1, policy_version 525400 (0.0009) [2023-12-27 09:16:39,466][20381] Updated weights for policy 1, policy_version 525410 (0.0008) [2023-12-27 09:16:39,530][20381] Updated weights for policy 1, policy_version 525420 (0.0008) [2023-12-27 09:16:40,040][20347] Updated weights for policy 0, policy_version 525730 (0.0009) [2023-12-27 09:16:40,105][20347] Updated weights for policy 0, policy_version 525740 (0.0008) [2023-12-27 09:16:40,171][20347] Updated weights for policy 0, policy_version 525750 (0.0008) [2023-12-27 09:16:40,196][20381] Updated weights for policy 1, policy_version 525430 (0.0006) [2023-12-27 09:16:40,230][20347] Updated weights for policy 0, policy_version 525760 (0.0007) [2023-12-27 09:16:40,259][20381] Updated weights for policy 1, policy_version 525440 (0.0006) [2023-12-27 09:16:40,319][20381] Updated weights for policy 1, policy_version 525450 (0.0009) [2023-12-27 09:16:40,796][20347] Updated weights for policy 0, policy_version 525770 (0.0008) [2023-12-27 09:16:40,858][20347] Updated weights for policy 0, policy_version 525780 (0.0010) [2023-12-27 09:16:40,916][20347] Updated weights for policy 0, policy_version 525790 (0.0010) [2023-12-27 09:16:40,955][20381] Updated weights for policy 1, policy_version 525460 (0.0008) [2023-12-27 09:16:41,024][20381] Updated weights for policy 1, policy_version 525470 (0.0007) [2023-12-27 09:16:41,080][20381] Updated weights for policy 1, policy_version 525480 (0.0006) [2023-12-27 09:16:41,588][20347] Updated weights for policy 0, policy_version 525800 (0.0006) [2023-12-27 09:16:41,652][20347] Updated weights for policy 0, policy_version 525810 (0.0009) [2023-12-27 09:16:41,707][20347] Updated weights for policy 0, policy_version 525820 (0.0009) [2023-12-27 09:16:41,841][20381] Updated weights for policy 1, policy_version 525490 (0.0007) [2023-12-27 09:16:41,903][20381] Updated weights for policy 1, policy_version 525500 (0.0006) [2023-12-27 09:16:41,966][20381] Updated weights for policy 1, policy_version 525510 (0.0007) [2023-12-27 09:16:42,028][20381] Updated weights for policy 1, policy_version 525520 (0.0005) [2023-12-27 09:16:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 269180928. Throughput: 0: 10444.9, 1: 10184.4. Samples: 269173684. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '740.920')] [2023-12-27 09:16:42,413][20347] Updated weights for policy 0, policy_version 525830 (0.0007) [2023-12-27 09:16:42,471][20347] Updated weights for policy 0, policy_version 525840 (0.0006) [2023-12-27 09:16:42,539][20347] Updated weights for policy 0, policy_version 525850 (0.0007) [2023-12-27 09:16:42,690][20381] Updated weights for policy 1, policy_version 525530 (0.0008) [2023-12-27 09:16:42,746][20381] Updated weights for policy 1, policy_version 525540 (0.0007) [2023-12-27 09:16:42,810][20381] Updated weights for policy 1, policy_version 525550 (0.0006) [2023-12-27 09:16:43,155][20347] Updated weights for policy 0, policy_version 525860 (0.0007) [2023-12-27 09:16:43,217][20347] Updated weights for policy 0, policy_version 525870 (0.0006) [2023-12-27 09:16:43,270][20347] Updated weights for policy 0, policy_version 525880 (0.0007) [2023-12-27 09:16:43,497][20381] Updated weights for policy 1, policy_version 525560 (0.0008) [2023-12-27 09:16:43,546][20381] Updated weights for policy 1, policy_version 525570 (0.0006) [2023-12-27 09:16:43,607][20381] Updated weights for policy 1, policy_version 525580 (0.0006) [2023-12-27 09:16:43,843][20347] Updated weights for policy 0, policy_version 525890 (0.0008) [2023-12-27 09:16:43,901][20347] Updated weights for policy 0, policy_version 525900 (0.0007) [2023-12-27 09:16:43,961][20347] Updated weights for policy 0, policy_version 525910 (0.0006) [2023-12-27 09:16:44,015][20347] Updated weights for policy 0, policy_version 525920 (0.0005) [2023-12-27 09:16:44,346][20381] Updated weights for policy 1, policy_version 525590 (0.0007) [2023-12-27 09:16:44,398][20381] Updated weights for policy 1, policy_version 525600 (0.0008) [2023-12-27 09:16:44,453][20381] Updated weights for policy 1, policy_version 525610 (0.0008) [2023-12-27 09:16:44,588][20347] Updated weights for policy 0, policy_version 525930 (0.0006) [2023-12-27 09:16:44,649][20347] Updated weights for policy 0, policy_version 525940 (0.0007) [2023-12-27 09:16:44,705][20347] Updated weights for policy 0, policy_version 525950 (0.0007) [2023-12-27 09:16:45,183][20381] Updated weights for policy 1, policy_version 525620 (0.0009) [2023-12-27 09:16:45,232][20381] Updated weights for policy 1, policy_version 525630 (0.0007) [2023-12-27 09:16:45,279][20381] Updated weights for policy 1, policy_version 525640 (0.0008) [2023-12-27 09:16:45,402][20347] Updated weights for policy 0, policy_version 525960 (0.0010) [2023-12-27 09:16:45,461][20347] Updated weights for policy 0, policy_version 525970 (0.0010) [2023-12-27 09:16:45,510][20347] Updated weights for policy 0, policy_version 525980 (0.0010) [2023-12-27 09:16:45,975][20381] Updated weights for policy 1, policy_version 525650 (0.0008) [2023-12-27 09:16:46,030][20381] Updated weights for policy 1, policy_version 525660 (0.0007) [2023-12-27 09:16:46,088][20381] Updated weights for policy 1, policy_version 525670 (0.0007) [2023-12-27 09:16:46,139][20381] Updated weights for policy 1, policy_version 525680 (0.0007) [2023-12-27 09:16:46,194][20347] Updated weights for policy 0, policy_version 525990 (0.0010) [2023-12-27 09:16:46,248][20347] Updated weights for policy 0, policy_version 526000 (0.0010) [2023-12-27 09:16:46,306][20347] Updated weights for policy 0, policy_version 526010 (0.0010) [2023-12-27 09:16:46,866][20381] Updated weights for policy 1, policy_version 525690 (0.0008) [2023-12-27 09:16:46,924][20381] Updated weights for policy 1, policy_version 525700 (0.0007) [2023-12-27 09:16:46,944][20347] Updated weights for policy 0, policy_version 526020 (0.0010) [2023-12-27 09:16:46,982][20381] Updated weights for policy 1, policy_version 525710 (0.0006) [2023-12-27 09:16:47,002][20347] Updated weights for policy 0, policy_version 526030 (0.0007) [2023-12-27 09:16:47,055][20347] Updated weights for policy 0, policy_version 526040 (0.0007) [2023-12-27 09:16:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 269287424. Throughput: 0: 10447.1, 1: 10143.7. Samples: 269296912. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '762.550')] [2023-12-27 09:16:47,635][20381] Updated weights for policy 1, policy_version 525720 (0.0007) [2023-12-27 09:16:47,641][20347] Updated weights for policy 0, policy_version 526050 (0.0007) [2023-12-27 09:16:47,687][20381] Updated weights for policy 1, policy_version 525730 (0.0007) [2023-12-27 09:16:47,692][20347] Updated weights for policy 0, policy_version 526060 (0.0005) [2023-12-27 09:16:47,737][20381] Updated weights for policy 1, policy_version 525740 (0.0010) [2023-12-27 09:16:47,741][20347] Updated weights for policy 0, policy_version 526070 (0.0005) [2023-12-27 09:16:47,788][20347] Updated weights for policy 0, policy_version 526080 (0.0006) [2023-12-27 09:16:48,359][20347] Updated weights for policy 0, policy_version 526090 (0.0008) [2023-12-27 09:16:48,403][20381] Updated weights for policy 1, policy_version 525750 (0.0009) [2023-12-27 09:16:48,414][20347] Updated weights for policy 0, policy_version 526100 (0.0007) [2023-12-27 09:16:48,454][20381] Updated weights for policy 1, policy_version 525760 (0.0008) [2023-12-27 09:16:48,472][20347] Updated weights for policy 0, policy_version 526110 (0.0007) [2023-12-27 09:16:48,503][20381] Updated weights for policy 1, policy_version 525770 (0.0005) [2023-12-27 09:16:49,074][20381] Updated weights for policy 1, policy_version 525780 (0.0005) [2023-12-27 09:16:49,095][20347] Updated weights for policy 0, policy_version 526120 (0.0007) [2023-12-27 09:16:49,129][20381] Updated weights for policy 1, policy_version 525790 (0.0006) [2023-12-27 09:16:49,151][20347] Updated weights for policy 0, policy_version 526130 (0.0006) [2023-12-27 09:16:49,176][20381] Updated weights for policy 1, policy_version 525800 (0.0010) [2023-12-27 09:16:49,198][20347] Updated weights for policy 0, policy_version 526140 (0.0005) [2023-12-27 09:16:49,807][20347] Updated weights for policy 0, policy_version 526150 (0.0007) [2023-12-27 09:16:49,865][20347] Updated weights for policy 0, policy_version 526160 (0.0008) [2023-12-27 09:16:49,896][20381] Updated weights for policy 1, policy_version 525810 (0.0007) [2023-12-27 09:16:49,926][20347] Updated weights for policy 0, policy_version 526170 (0.0008) [2023-12-27 09:16:49,971][20381] Updated weights for policy 1, policy_version 525820 (0.0009) [2023-12-27 09:16:50,033][20381] Updated weights for policy 1, policy_version 525830 (0.0008) [2023-12-27 09:16:50,091][20381] Updated weights for policy 1, policy_version 525840 (0.0007) [2023-12-27 09:16:50,571][20347] Updated weights for policy 0, policy_version 526180 (0.0009) [2023-12-27 09:16:50,633][20347] Updated weights for policy 0, policy_version 526190 (0.0008) [2023-12-27 09:16:50,692][20347] Updated weights for policy 0, policy_version 526200 (0.0009) [2023-12-27 09:16:50,852][20381] Updated weights for policy 1, policy_version 525850 (0.0007) [2023-12-27 09:16:50,914][20381] Updated weights for policy 1, policy_version 525860 (0.0008) [2023-12-27 09:16:50,969][20381] Updated weights for policy 1, policy_version 525870 (0.0008) [2023-12-27 09:16:51,386][20347] Updated weights for policy 0, policy_version 526210 (0.0008) [2023-12-27 09:16:51,445][20347] Updated weights for policy 0, policy_version 526220 (0.0009) [2023-12-27 09:16:51,499][20347] Updated weights for policy 0, policy_version 526230 (0.0008) [2023-12-27 09:16:51,555][20347] Updated weights for policy 0, policy_version 526240 (0.0008) [2023-12-27 09:16:51,659][20381] Updated weights for policy 1, policy_version 525880 (0.0008) [2023-12-27 09:16:51,719][20381] Updated weights for policy 1, policy_version 525890 (0.0008) [2023-12-27 09:16:51,777][20381] Updated weights for policy 1, policy_version 525900 (0.0008) [2023-12-27 09:16:52,272][20347] Updated weights for policy 0, policy_version 526250 (0.0008) [2023-12-27 09:16:52,328][20347] Updated weights for policy 0, policy_version 526260 (0.0008) [2023-12-27 09:16:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 269385728. Throughput: 0: 10531.3, 1: 10146.2. Samples: 269362100. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '683.860')] [2023-12-27 09:16:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000525904_134651904.pth... [2023-12-27 09:16:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000524720_134348800.pth [2023-12-27 09:16:52,391][20347] Updated weights for policy 0, policy_version 526270 (0.0008) [2023-12-27 09:16:52,401][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000526272_134742016.pth... [2023-12-27 09:16:52,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000525024_134422528.pth [2023-12-27 09:16:52,490][20381] Updated weights for policy 1, policy_version 525910 (0.0008) [2023-12-27 09:16:52,551][20381] Updated weights for policy 1, policy_version 525920 (0.0008) [2023-12-27 09:16:52,611][20381] Updated weights for policy 1, policy_version 525930 (0.0007) [2023-12-27 09:16:53,048][20347] Updated weights for policy 0, policy_version 526280 (0.0007) [2023-12-27 09:16:53,102][20347] Updated weights for policy 0, policy_version 526290 (0.0007) [2023-12-27 09:16:53,158][20347] Updated weights for policy 0, policy_version 526300 (0.0005) [2023-12-27 09:16:53,272][20381] Updated weights for policy 1, policy_version 525940 (0.0007) [2023-12-27 09:16:53,320][20381] Updated weights for policy 1, policy_version 525950 (0.0007) [2023-12-27 09:16:53,375][20381] Updated weights for policy 1, policy_version 525960 (0.0008) [2023-12-27 09:16:53,836][20347] Updated weights for policy 0, policy_version 526310 (0.0007) [2023-12-27 09:16:53,892][20347] Updated weights for policy 0, policy_version 526320 (0.0005) [2023-12-27 09:16:53,942][20347] Updated weights for policy 0, policy_version 526330 (0.0005) [2023-12-27 09:16:54,007][20381] Updated weights for policy 1, policy_version 525970 (0.0008) [2023-12-27 09:16:54,062][20381] Updated weights for policy 1, policy_version 525980 (0.0006) [2023-12-27 09:16:54,118][20381] Updated weights for policy 1, policy_version 525990 (0.0008) [2023-12-27 09:16:54,172][20381] Updated weights for policy 1, policy_version 526000 (0.0007) [2023-12-27 09:16:54,496][20347] Updated weights for policy 0, policy_version 526340 (0.0005) [2023-12-27 09:16:54,549][20347] Updated weights for policy 0, policy_version 526350 (0.0005) [2023-12-27 09:16:54,608][20347] Updated weights for policy 0, policy_version 526360 (0.0008) [2023-12-27 09:16:54,777][20381] Updated weights for policy 1, policy_version 526010 (0.0008) [2023-12-27 09:16:54,831][20381] Updated weights for policy 1, policy_version 526020 (0.0007) [2023-12-27 09:16:54,883][20381] Updated weights for policy 1, policy_version 526030 (0.0005) [2023-12-27 09:16:55,212][20347] Updated weights for policy 0, policy_version 526370 (0.0006) [2023-12-27 09:16:55,277][20347] Updated weights for policy 0, policy_version 526380 (0.0006) [2023-12-27 09:16:55,343][20347] Updated weights for policy 0, policy_version 526390 (0.0006) [2023-12-27 09:16:55,401][20347] Updated weights for policy 0, policy_version 526400 (0.0007) [2023-12-27 09:16:55,537][20381] Updated weights for policy 1, policy_version 526040 (0.0009) [2023-12-27 09:16:55,587][20381] Updated weights for policy 1, policy_version 526050 (0.0009) [2023-12-27 09:16:55,636][20381] Updated weights for policy 1, policy_version 526060 (0.0008) [2023-12-27 09:16:55,967][20347] Updated weights for policy 0, policy_version 526410 (0.0010) [2023-12-27 09:16:56,023][20347] Updated weights for policy 0, policy_version 526420 (0.0011) [2023-12-27 09:16:56,076][20347] Updated weights for policy 0, policy_version 526430 (0.0010) [2023-12-27 09:16:56,337][20381] Updated weights for policy 1, policy_version 526070 (0.0007) [2023-12-27 09:16:56,385][20381] Updated weights for policy 1, policy_version 526080 (0.0005) [2023-12-27 09:16:56,436][20381] Updated weights for policy 1, policy_version 526090 (0.0005) [2023-12-27 09:16:56,674][20347] Updated weights for policy 0, policy_version 526440 (0.0010) [2023-12-27 09:16:56,718][20347] Updated weights for policy 0, policy_version 526450 (0.0010) [2023-12-27 09:16:56,766][20347] Updated weights for policy 0, policy_version 526460 (0.0009) [2023-12-27 09:16:57,028][20381] Updated weights for policy 1, policy_version 526100 (0.0005) [2023-12-27 09:16:57,077][20381] Updated weights for policy 1, policy_version 526110 (0.0005) [2023-12-27 09:16:57,123][20381] Updated weights for policy 1, policy_version 526120 (0.0005) [2023-12-27 09:16:57,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 269500416. Throughput: 0: 10587.8, 1: 10160.9. Samples: 269487748. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:16:57,364][19170] Avg episode reward: [(0, '980.700'), (1, '800.680')] [2023-12-27 09:16:57,411][20347] Updated weights for policy 0, policy_version 526470 (0.0008) [2023-12-27 09:16:57,467][20347] Updated weights for policy 0, policy_version 526480 (0.0009) [2023-12-27 09:16:57,531][20347] Updated weights for policy 0, policy_version 526490 (0.0007) [2023-12-27 09:16:57,683][20381] Updated weights for policy 1, policy_version 526130 (0.0005) [2023-12-27 09:16:57,736][20381] Updated weights for policy 1, policy_version 526140 (0.0006) [2023-12-27 09:16:57,780][20381] Updated weights for policy 1, policy_version 526150 (0.0007) [2023-12-27 09:16:57,825][20381] Updated weights for policy 1, policy_version 526160 (0.0006) [2023-12-27 09:16:58,110][20347] Updated weights for policy 0, policy_version 526500 (0.0005) [2023-12-27 09:16:58,167][20347] Updated weights for policy 0, policy_version 526510 (0.0006) [2023-12-27 09:16:58,232][20347] Updated weights for policy 0, policy_version 526520 (0.0009) [2023-12-27 09:16:58,456][20381] Updated weights for policy 1, policy_version 526170 (0.0010) [2023-12-27 09:16:58,520][20381] Updated weights for policy 1, policy_version 526180 (0.0011) [2023-12-27 09:16:58,585][20381] Updated weights for policy 1, policy_version 526190 (0.0008) [2023-12-27 09:16:58,945][20347] Updated weights for policy 0, policy_version 526530 (0.0009) [2023-12-27 09:16:59,007][20347] Updated weights for policy 0, policy_version 526540 (0.0008) [2023-12-27 09:16:59,069][20347] Updated weights for policy 0, policy_version 526550 (0.0005) [2023-12-27 09:16:59,128][20347] Updated weights for policy 0, policy_version 526560 (0.0006) [2023-12-27 09:16:59,359][20381] Updated weights for policy 1, policy_version 526200 (0.0009) [2023-12-27 09:16:59,409][20381] Updated weights for policy 1, policy_version 526210 (0.0006) [2023-12-27 09:16:59,453][20381] Updated weights for policy 1, policy_version 526220 (0.0005) [2023-12-27 09:16:59,811][20347] Updated weights for policy 0, policy_version 526570 (0.0007) [2023-12-27 09:16:59,873][20347] Updated weights for policy 0, policy_version 526580 (0.0007) [2023-12-27 09:16:59,937][20347] Updated weights for policy 0, policy_version 526590 (0.0007) [2023-12-27 09:17:00,105][20381] Updated weights for policy 1, policy_version 526230 (0.0007) [2023-12-27 09:17:00,161][20381] Updated weights for policy 1, policy_version 526240 (0.0008) [2023-12-27 09:17:00,221][20381] Updated weights for policy 1, policy_version 526250 (0.0008) [2023-12-27 09:17:00,618][20347] Updated weights for policy 0, policy_version 526600 (0.0007) [2023-12-27 09:17:00,671][20347] Updated weights for policy 0, policy_version 526610 (0.0008) [2023-12-27 09:17:00,715][20347] Updated weights for policy 0, policy_version 526620 (0.0008) [2023-12-27 09:17:00,854][20381] Updated weights for policy 1, policy_version 526260 (0.0008) [2023-12-27 09:17:00,909][20381] Updated weights for policy 1, policy_version 526270 (0.0007) [2023-12-27 09:17:00,967][20381] Updated weights for policy 1, policy_version 526280 (0.0008) [2023-12-27 09:17:01,441][20347] Updated weights for policy 0, policy_version 526630 (0.0009) [2023-12-27 09:17:01,487][20347] Updated weights for policy 0, policy_version 526640 (0.0007) [2023-12-27 09:17:01,532][20347] Updated weights for policy 0, policy_version 526650 (0.0009) [2023-12-27 09:17:01,597][20381] Updated weights for policy 1, policy_version 526290 (0.0008) [2023-12-27 09:17:01,662][20381] Updated weights for policy 1, policy_version 526300 (0.0009) [2023-12-27 09:17:01,716][20381] Updated weights for policy 1, policy_version 526310 (0.0007) [2023-12-27 09:17:01,776][20381] Updated weights for policy 1, policy_version 526320 (0.0006) [2023-12-27 09:17:02,287][20347] Updated weights for policy 0, policy_version 526660 (0.0009) [2023-12-27 09:17:02,347][20347] Updated weights for policy 0, policy_version 526670 (0.0008) [2023-12-27 09:17:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 269598720. Throughput: 0: 10608.2, 1: 10272.3. Samples: 269615448. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:17:02,363][19170] Avg episode reward: [(0, '990.360'), (1, '815.580')] [2023-12-27 09:17:02,379][20381] Updated weights for policy 1, policy_version 526330 (0.0007) [2023-12-27 09:17:02,408][20347] Updated weights for policy 0, policy_version 526680 (0.0008) [2023-12-27 09:17:02,439][20381] Updated weights for policy 1, policy_version 526340 (0.0007) [2023-12-27 09:17:02,501][20381] Updated weights for policy 1, policy_version 526350 (0.0006) [2023-12-27 09:17:03,082][20347] Updated weights for policy 0, policy_version 526690 (0.0007) [2023-12-27 09:17:03,130][20347] Updated weights for policy 0, policy_version 526700 (0.0008) [2023-12-27 09:17:03,176][20381] Updated weights for policy 1, policy_version 526360 (0.0007) [2023-12-27 09:17:03,178][20347] Updated weights for policy 0, policy_version 526710 (0.0005) [2023-12-27 09:17:03,226][20347] Updated weights for policy 0, policy_version 526720 (0.0006) [2023-12-27 09:17:03,230][20381] Updated weights for policy 1, policy_version 526370 (0.0007) [2023-12-27 09:17:03,280][20381] Updated weights for policy 1, policy_version 526380 (0.0007) [2023-12-27 09:17:03,920][20347] Updated weights for policy 0, policy_version 526730 (0.0007) [2023-12-27 09:17:03,967][20381] Updated weights for policy 1, policy_version 526390 (0.0007) [2023-12-27 09:17:03,977][20347] Updated weights for policy 0, policy_version 526740 (0.0006) [2023-12-27 09:17:04,023][20347] Updated weights for policy 0, policy_version 526750 (0.0006) [2023-12-27 09:17:04,026][20381] Updated weights for policy 1, policy_version 526400 (0.0007) [2023-12-27 09:17:04,080][20381] Updated weights for policy 1, policy_version 526410 (0.0007) [2023-12-27 09:17:04,705][20347] Updated weights for policy 0, policy_version 526760 (0.0007) [2023-12-27 09:17:04,753][20347] Updated weights for policy 0, policy_version 526770 (0.0008) [2023-12-27 09:17:04,782][20381] Updated weights for policy 1, policy_version 526420 (0.0008) [2023-12-27 09:17:04,805][20347] Updated weights for policy 0, policy_version 526780 (0.0007) [2023-12-27 09:17:04,827][20381] Updated weights for policy 1, policy_version 526430 (0.0005) [2023-12-27 09:17:04,884][20381] Updated weights for policy 1, policy_version 526440 (0.0008) [2023-12-27 09:17:05,510][20347] Updated weights for policy 0, policy_version 526790 (0.0007) [2023-12-27 09:17:05,559][20347] Updated weights for policy 0, policy_version 526800 (0.0008) [2023-12-27 09:17:05,587][20381] Updated weights for policy 1, policy_version 526450 (0.0008) [2023-12-27 09:17:05,614][20347] Updated weights for policy 0, policy_version 526810 (0.0008) [2023-12-27 09:17:05,646][20381] Updated weights for policy 1, policy_version 526460 (0.0006) [2023-12-27 09:17:05,701][20381] Updated weights for policy 1, policy_version 526470 (0.0007) [2023-12-27 09:17:05,760][20381] Updated weights for policy 1, policy_version 526480 (0.0008) [2023-12-27 09:17:06,353][20347] Updated weights for policy 0, policy_version 526820 (0.0006) [2023-12-27 09:17:06,410][20347] Updated weights for policy 0, policy_version 526830 (0.0008) [2023-12-27 09:17:06,468][20381] Updated weights for policy 1, policy_version 526490 (0.0008) [2023-12-27 09:17:06,470][20347] Updated weights for policy 0, policy_version 526840 (0.0007) [2023-12-27 09:17:06,525][20381] Updated weights for policy 1, policy_version 526500 (0.0007) [2023-12-27 09:17:06,585][20381] Updated weights for policy 1, policy_version 526510 (0.0007) [2023-12-27 09:17:07,146][20347] Updated weights for policy 0, policy_version 526850 (0.0007) [2023-12-27 09:17:07,199][20347] Updated weights for policy 0, policy_version 526860 (0.0010) [2023-12-27 09:17:07,231][20381] Updated weights for policy 1, policy_version 526520 (0.0006) [2023-12-27 09:17:07,252][20347] Updated weights for policy 0, policy_version 526870 (0.0010) [2023-12-27 09:17:07,286][20381] Updated weights for policy 1, policy_version 526530 (0.0005) [2023-12-27 09:17:07,314][20347] Updated weights for policy 0, policy_version 526880 (0.0010) [2023-12-27 09:17:07,342][20381] Updated weights for policy 1, policy_version 526540 (0.0006) [2023-12-27 09:17:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 269713408. Throughput: 0: 10573.1, 1: 10269.1. Samples: 269675912. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:17:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '825.370')] [2023-12-27 09:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000526544_134815744.pth... [2023-12-27 09:17:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000526880_134897664.pth... [2023-12-27 09:17:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000525328_134504448.pth [2023-12-27 09:17:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000525632_134578176.pth [2023-12-27 09:17:07,971][20347] Updated weights for policy 0, policy_version 526890 (0.0005) [2023-12-27 09:17:08,026][20347] Updated weights for policy 0, policy_version 526900 (0.0006) [2023-12-27 09:17:08,035][20381] Updated weights for policy 1, policy_version 526550 (0.0007) [2023-12-27 09:17:08,079][20347] Updated weights for policy 0, policy_version 526910 (0.0006) [2023-12-27 09:17:08,092][20381] Updated weights for policy 1, policy_version 526560 (0.0006) [2023-12-27 09:17:08,160][20381] Updated weights for policy 1, policy_version 526570 (0.0006) [2023-12-27 09:17:08,680][20347] Updated weights for policy 0, policy_version 526920 (0.0007) [2023-12-27 09:17:08,732][20347] Updated weights for policy 0, policy_version 526930 (0.0007) [2023-12-27 09:17:08,784][20347] Updated weights for policy 0, policy_version 526940 (0.0008) [2023-12-27 09:17:08,850][20381] Updated weights for policy 1, policy_version 526580 (0.0008) [2023-12-27 09:17:08,912][20381] Updated weights for policy 1, policy_version 526590 (0.0007) [2023-12-27 09:17:08,979][20381] Updated weights for policy 1, policy_version 526600 (0.0008) [2023-12-27 09:17:09,499][20347] Updated weights for policy 0, policy_version 526950 (0.0008) [2023-12-27 09:17:09,569][20347] Updated weights for policy 0, policy_version 526960 (0.0006) [2023-12-27 09:17:09,637][20347] Updated weights for policy 0, policy_version 526970 (0.0006) [2023-12-27 09:17:09,644][20381] Updated weights for policy 1, policy_version 526610 (0.0009) [2023-12-27 09:17:09,693][20381] Updated weights for policy 1, policy_version 526620 (0.0007) [2023-12-27 09:17:09,747][20381] Updated weights for policy 1, policy_version 526630 (0.0007) [2023-12-27 09:17:09,800][20381] Updated weights for policy 1, policy_version 526640 (0.0007) [2023-12-27 09:17:10,279][20347] Updated weights for policy 0, policy_version 526980 (0.0006) [2023-12-27 09:17:10,340][20347] Updated weights for policy 0, policy_version 526990 (0.0008) [2023-12-27 09:17:10,403][20347] Updated weights for policy 0, policy_version 527000 (0.0007) [2023-12-27 09:17:10,498][20381] Updated weights for policy 1, policy_version 526650 (0.0008) [2023-12-27 09:17:10,555][20381] Updated weights for policy 1, policy_version 526660 (0.0007) [2023-12-27 09:17:10,611][20381] Updated weights for policy 1, policy_version 526670 (0.0008) [2023-12-27 09:17:11,058][20347] Updated weights for policy 0, policy_version 527010 (0.0008) [2023-12-27 09:17:11,112][20347] Updated weights for policy 0, policy_version 527020 (0.0008) [2023-12-27 09:17:11,173][20347] Updated weights for policy 0, policy_version 527030 (0.0007) [2023-12-27 09:17:11,239][20347] Updated weights for policy 0, policy_version 527040 (0.0007) [2023-12-27 09:17:11,363][20381] Updated weights for policy 1, policy_version 526680 (0.0009) [2023-12-27 09:17:11,423][20381] Updated weights for policy 1, policy_version 526690 (0.0011) [2023-12-27 09:17:11,473][20381] Updated weights for policy 1, policy_version 526700 (0.0011) [2023-12-27 09:17:12,005][20347] Updated weights for policy 0, policy_version 527050 (0.0009) [2023-12-27 09:17:12,066][20347] Updated weights for policy 0, policy_version 527060 (0.0008) [2023-12-27 09:17:12,112][20347] Updated weights for policy 0, policy_version 527070 (0.0008) [2023-12-27 09:17:12,114][20381] Updated weights for policy 1, policy_version 526710 (0.0009) [2023-12-27 09:17:12,168][20381] Updated weights for policy 1, policy_version 526720 (0.0008) [2023-12-27 09:17:12,219][20381] Updated weights for policy 1, policy_version 526730 (0.0009) [2023-12-27 09:17:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 269811712. Throughput: 0: 10536.3, 1: 10246.9. Samples: 269797920. Policy #0 lag: (min: 21.0, avg: 29.3, max: 53.0) [2023-12-27 09:17:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.520')] [2023-12-27 09:17:12,853][20347] Updated weights for policy 0, policy_version 527080 (0.0009) [2023-12-27 09:17:12,898][20381] Updated weights for policy 1, policy_version 526740 (0.0009) [2023-12-27 09:17:12,914][20347] Updated weights for policy 0, policy_version 527090 (0.0010) [2023-12-27 09:17:12,958][20381] Updated weights for policy 1, policy_version 526750 (0.0006) [2023-12-27 09:17:12,973][20347] Updated weights for policy 0, policy_version 527100 (0.0006) [2023-12-27 09:17:13,015][20381] Updated weights for policy 1, policy_version 526760 (0.0007) [2023-12-27 09:17:13,538][20347] Updated weights for policy 0, policy_version 527110 (0.0008) [2023-12-27 09:17:13,591][20347] Updated weights for policy 0, policy_version 527120 (0.0005) [2023-12-27 09:17:13,647][20347] Updated weights for policy 0, policy_version 527130 (0.0010) [2023-12-27 09:17:13,691][20381] Updated weights for policy 1, policy_version 526770 (0.0008) [2023-12-27 09:17:13,738][20381] Updated weights for policy 1, policy_version 526780 (0.0006) [2023-12-27 09:17:13,789][20381] Updated weights for policy 1, policy_version 526790 (0.0005) [2023-12-27 09:17:13,841][20381] Updated weights for policy 1, policy_version 526800 (0.0006) [2023-12-27 09:17:14,219][20347] Updated weights for policy 0, policy_version 527140 (0.0008) [2023-12-27 09:17:14,280][20347] Updated weights for policy 0, policy_version 527150 (0.0005) [2023-12-27 09:17:14,339][20347] Updated weights for policy 0, policy_version 527160 (0.0005) [2023-12-27 09:17:14,548][20381] Updated weights for policy 1, policy_version 526810 (0.0008) [2023-12-27 09:17:14,606][20381] Updated weights for policy 1, policy_version 526820 (0.0008) [2023-12-27 09:17:14,651][20381] Updated weights for policy 1, policy_version 526830 (0.0008) [2023-12-27 09:17:14,960][20347] Updated weights for policy 0, policy_version 527170 (0.0006) [2023-12-27 09:17:15,019][20347] Updated weights for policy 0, policy_version 527180 (0.0008) [2023-12-27 09:17:15,080][20347] Updated weights for policy 0, policy_version 527190 (0.0008) [2023-12-27 09:17:15,142][20347] Updated weights for policy 0, policy_version 527200 (0.0007) [2023-12-27 09:17:15,330][20381] Updated weights for policy 1, policy_version 526840 (0.0008) [2023-12-27 09:17:15,379][20381] Updated weights for policy 1, policy_version 526850 (0.0008) [2023-12-27 09:17:15,430][20381] Updated weights for policy 1, policy_version 526860 (0.0008) [2023-12-27 09:17:15,764][20347] Updated weights for policy 0, policy_version 527210 (0.0005) [2023-12-27 09:17:15,821][20347] Updated weights for policy 0, policy_version 527220 (0.0006) [2023-12-27 09:17:15,875][20347] Updated weights for policy 0, policy_version 527230 (0.0010) [2023-12-27 09:17:16,110][20381] Updated weights for policy 1, policy_version 526870 (0.0009) [2023-12-27 09:17:16,156][20381] Updated weights for policy 1, policy_version 526880 (0.0009) [2023-12-27 09:17:16,204][20381] Updated weights for policy 1, policy_version 526890 (0.0007) [2023-12-27 09:17:16,432][20347] Updated weights for policy 0, policy_version 527240 (0.0008) [2023-12-27 09:17:16,499][20347] Updated weights for policy 0, policy_version 527250 (0.0008) [2023-12-27 09:17:16,560][20347] Updated weights for policy 0, policy_version 527260 (0.0005) [2023-12-27 09:17:16,933][20381] Updated weights for policy 1, policy_version 526900 (0.0007) [2023-12-27 09:17:16,984][20381] Updated weights for policy 1, policy_version 526910 (0.0005) [2023-12-27 09:17:17,038][20381] Updated weights for policy 1, policy_version 526920 (0.0007) [2023-12-27 09:17:17,221][20347] Updated weights for policy 0, policy_version 527270 (0.0006) [2023-12-27 09:17:17,287][20347] Updated weights for policy 0, policy_version 527280 (0.0006) [2023-12-27 09:17:17,353][20347] Updated weights for policy 0, policy_version 527290 (0.0005) [2023-12-27 09:17:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 269910016. Throughput: 0: 10556.7, 1: 10243.1. Samples: 269923560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.630')] [2023-12-27 09:17:17,639][20381] Updated weights for policy 1, policy_version 526930 (0.0008) [2023-12-27 09:17:17,693][20381] Updated weights for policy 1, policy_version 526940 (0.0005) [2023-12-27 09:17:17,748][20381] Updated weights for policy 1, policy_version 526950 (0.0005) [2023-12-27 09:17:17,794][20381] Updated weights for policy 1, policy_version 526960 (0.0005) [2023-12-27 09:17:17,904][20347] Updated weights for policy 0, policy_version 527300 (0.0008) [2023-12-27 09:17:17,960][20347] Updated weights for policy 0, policy_version 527310 (0.0007) [2023-12-27 09:17:18,017][20347] Updated weights for policy 0, policy_version 527320 (0.0005) [2023-12-27 09:17:18,432][20381] Updated weights for policy 1, policy_version 526970 (0.0008) [2023-12-27 09:17:18,492][20381] Updated weights for policy 1, policy_version 526980 (0.0007) [2023-12-27 09:17:18,549][20381] Updated weights for policy 1, policy_version 526990 (0.0008) [2023-12-27 09:17:18,601][20347] Updated weights for policy 0, policy_version 527330 (0.0006) [2023-12-27 09:17:18,654][20347] Updated weights for policy 0, policy_version 527340 (0.0008) [2023-12-27 09:17:18,712][20347] Updated weights for policy 0, policy_version 527350 (0.0008) [2023-12-27 09:17:18,771][20347] Updated weights for policy 0, policy_version 527360 (0.0008) [2023-12-27 09:17:19,270][20381] Updated weights for policy 1, policy_version 527000 (0.0008) [2023-12-27 09:17:19,318][20381] Updated weights for policy 1, policy_version 527010 (0.0007) [2023-12-27 09:17:19,382][20381] Updated weights for policy 1, policy_version 527020 (0.0008) [2023-12-27 09:17:19,383][20347] Updated weights for policy 0, policy_version 527370 (0.0008) [2023-12-27 09:17:19,447][20347] Updated weights for policy 0, policy_version 527380 (0.0007) [2023-12-27 09:17:19,506][20347] Updated weights for policy 0, policy_version 527390 (0.0008) [2023-12-27 09:17:20,084][20381] Updated weights for policy 1, policy_version 527030 (0.0008) [2023-12-27 09:17:20,146][20381] Updated weights for policy 1, policy_version 527040 (0.0008) [2023-12-27 09:17:20,193][20347] Updated weights for policy 0, policy_version 527400 (0.0006) [2023-12-27 09:17:20,216][20381] Updated weights for policy 1, policy_version 527050 (0.0008) [2023-12-27 09:17:20,251][20347] Updated weights for policy 0, policy_version 527410 (0.0005) [2023-12-27 09:17:20,319][20347] Updated weights for policy 0, policy_version 527420 (0.0006) [2023-12-27 09:17:20,921][20381] Updated weights for policy 1, policy_version 527060 (0.0007) [2023-12-27 09:17:20,935][20347] Updated weights for policy 0, policy_version 527430 (0.0007) [2023-12-27 09:17:20,978][20381] Updated weights for policy 1, policy_version 527070 (0.0006) [2023-12-27 09:17:20,995][20347] Updated weights for policy 0, policy_version 527440 (0.0007) [2023-12-27 09:17:21,040][20381] Updated weights for policy 1, policy_version 527080 (0.0007) [2023-12-27 09:17:21,059][20347] Updated weights for policy 0, policy_version 527450 (0.0007) [2023-12-27 09:17:21,763][20347] Updated weights for policy 0, policy_version 527460 (0.0009) [2023-12-27 09:17:21,799][20381] Updated weights for policy 1, policy_version 527090 (0.0010) [2023-12-27 09:17:21,821][20347] Updated weights for policy 0, policy_version 527470 (0.0008) [2023-12-27 09:17:21,861][20381] Updated weights for policy 1, policy_version 527100 (0.0008) [2023-12-27 09:17:21,879][20347] Updated weights for policy 0, policy_version 527480 (0.0006) [2023-12-27 09:17:21,920][20381] Updated weights for policy 1, policy_version 527110 (0.0008) [2023-12-27 09:17:21,976][20381] Updated weights for policy 1, policy_version 527120 (0.0008) [2023-12-27 09:17:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20716.1). Total num frames: 270016512. Throughput: 0: 10548.0, 1: 10277.5. Samples: 269987676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.260')] [2023-12-27 09:17:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000527120_134963200.pth... [2023-12-27 09:17:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000527488_135053312.pth... [2023-12-27 09:17:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000525904_134651904.pth [2023-12-27 09:17:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000526272_134742016.pth [2023-12-27 09:17:22,580][20347] Updated weights for policy 0, policy_version 527490 (0.0008) [2023-12-27 09:17:22,638][20347] Updated weights for policy 0, policy_version 527500 (0.0007) [2023-12-27 09:17:22,701][20347] Updated weights for policy 0, policy_version 527510 (0.0008) [2023-12-27 09:17:22,708][20381] Updated weights for policy 1, policy_version 527130 (0.0008) [2023-12-27 09:17:22,753][20347] Updated weights for policy 0, policy_version 527520 (0.0006) [2023-12-27 09:17:22,759][20381] Updated weights for policy 1, policy_version 527140 (0.0006) [2023-12-27 09:17:22,813][20381] Updated weights for policy 1, policy_version 527150 (0.0008) [2023-12-27 09:17:23,406][20347] Updated weights for policy 0, policy_version 527530 (0.0005) [2023-12-27 09:17:23,455][20347] Updated weights for policy 0, policy_version 527540 (0.0005) [2023-12-27 09:17:23,506][20347] Updated weights for policy 0, policy_version 527550 (0.0005) [2023-12-27 09:17:23,553][20381] Updated weights for policy 1, policy_version 527160 (0.0009) [2023-12-27 09:17:23,601][20381] Updated weights for policy 1, policy_version 527170 (0.0010) [2023-12-27 09:17:23,659][20381] Updated weights for policy 1, policy_version 527180 (0.0006) [2023-12-27 09:17:24,118][20347] Updated weights for policy 0, policy_version 527560 (0.0007) [2023-12-27 09:17:24,175][20347] Updated weights for policy 0, policy_version 527570 (0.0007) [2023-12-27 09:17:24,233][20347] Updated weights for policy 0, policy_version 527580 (0.0007) [2023-12-27 09:17:24,338][20381] Updated weights for policy 1, policy_version 527190 (0.0007) [2023-12-27 09:17:24,386][20381] Updated weights for policy 1, policy_version 527200 (0.0007) [2023-12-27 09:17:24,442][20381] Updated weights for policy 1, policy_version 527210 (0.0007) [2023-12-27 09:17:24,874][20347] Updated weights for policy 0, policy_version 527590 (0.0007) [2023-12-27 09:17:24,934][20347] Updated weights for policy 0, policy_version 527600 (0.0007) [2023-12-27 09:17:24,991][20347] Updated weights for policy 0, policy_version 527610 (0.0007) [2023-12-27 09:17:25,140][20381] Updated weights for policy 1, policy_version 527220 (0.0009) [2023-12-27 09:17:25,185][20381] Updated weights for policy 1, policy_version 527230 (0.0010) [2023-12-27 09:17:25,231][20381] Updated weights for policy 1, policy_version 527240 (0.0010) [2023-12-27 09:17:25,646][20347] Updated weights for policy 0, policy_version 527620 (0.0008) [2023-12-27 09:17:25,697][20347] Updated weights for policy 0, policy_version 527630 (0.0006) [2023-12-27 09:17:25,756][20347] Updated weights for policy 0, policy_version 527640 (0.0005) [2023-12-27 09:17:25,941][20381] Updated weights for policy 1, policy_version 527250 (0.0011) [2023-12-27 09:17:25,998][20381] Updated weights for policy 1, policy_version 527260 (0.0011) [2023-12-27 09:17:26,054][20381] Updated weights for policy 1, policy_version 527270 (0.0011) [2023-12-27 09:17:26,110][20381] Updated weights for policy 1, policy_version 527280 (0.0011) [2023-12-27 09:17:26,404][20347] Updated weights for policy 0, policy_version 527650 (0.0006) [2023-12-27 09:17:26,461][20347] Updated weights for policy 0, policy_version 527660 (0.0005) [2023-12-27 09:17:26,512][20347] Updated weights for policy 0, policy_version 527670 (0.0005) [2023-12-27 09:17:26,557][20347] Updated weights for policy 0, policy_version 527680 (0.0007) [2023-12-27 09:17:26,790][20381] Updated weights for policy 1, policy_version 527290 (0.0005) [2023-12-27 09:17:26,845][20381] Updated weights for policy 1, policy_version 527300 (0.0007) [2023-12-27 09:17:26,902][20381] Updated weights for policy 1, policy_version 527310 (0.0008) [2023-12-27 09:17:27,191][20347] Updated weights for policy 0, policy_version 527690 (0.0008) [2023-12-27 09:17:27,236][20347] Updated weights for policy 0, policy_version 527700 (0.0008) [2023-12-27 09:17:27,282][20347] Updated weights for policy 0, policy_version 527710 (0.0009) [2023-12-27 09:17:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 270123008. Throughput: 0: 10577.8, 1: 10206.9. Samples: 270108992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.910')] [2023-12-27 09:17:27,524][20381] Updated weights for policy 1, policy_version 527320 (0.0007) [2023-12-27 09:17:27,566][20381] Updated weights for policy 1, policy_version 527330 (0.0008) [2023-12-27 09:17:27,608][20381] Updated weights for policy 1, policy_version 527340 (0.0008) [2023-12-27 09:17:28,007][20347] Updated weights for policy 0, policy_version 527720 (0.0008) [2023-12-27 09:17:28,053][20347] Updated weights for policy 0, policy_version 527730 (0.0008) [2023-12-27 09:17:28,098][20347] Updated weights for policy 0, policy_version 527740 (0.0009) [2023-12-27 09:17:28,231][20381] Updated weights for policy 1, policy_version 527350 (0.0007) [2023-12-27 09:17:28,286][20381] Updated weights for policy 1, policy_version 527360 (0.0008) [2023-12-27 09:17:28,340][20381] Updated weights for policy 1, policy_version 527370 (0.0007) [2023-12-27 09:17:28,751][20347] Updated weights for policy 0, policy_version 527750 (0.0007) [2023-12-27 09:17:28,802][20347] Updated weights for policy 0, policy_version 527760 (0.0008) [2023-12-27 09:17:28,854][20347] Updated weights for policy 0, policy_version 527770 (0.0008) [2023-12-27 09:17:29,072][20381] Updated weights for policy 1, policy_version 527380 (0.0007) [2023-12-27 09:17:29,120][20381] Updated weights for policy 1, policy_version 527390 (0.0008) [2023-12-27 09:17:29,170][20381] Updated weights for policy 1, policy_version 527400 (0.0008) [2023-12-27 09:17:29,523][20347] Updated weights for policy 0, policy_version 527780 (0.0007) [2023-12-27 09:17:29,583][20347] Updated weights for policy 0, policy_version 527790 (0.0005) [2023-12-27 09:17:29,652][20347] Updated weights for policy 0, policy_version 527800 (0.0006) [2023-12-27 09:17:29,860][20381] Updated weights for policy 1, policy_version 527410 (0.0007) [2023-12-27 09:17:29,918][20381] Updated weights for policy 1, policy_version 527420 (0.0007) [2023-12-27 09:17:29,970][20381] Updated weights for policy 1, policy_version 527430 (0.0008) [2023-12-27 09:17:30,024][20381] Updated weights for policy 1, policy_version 527440 (0.0007) [2023-12-27 09:17:30,302][20347] Updated weights for policy 0, policy_version 527810 (0.0008) [2023-12-27 09:17:30,359][20347] Updated weights for policy 0, policy_version 527820 (0.0007) [2023-12-27 09:17:30,422][20347] Updated weights for policy 0, policy_version 527830 (0.0008) [2023-12-27 09:17:30,476][20347] Updated weights for policy 0, policy_version 527840 (0.0007) [2023-12-27 09:17:30,712][20381] Updated weights for policy 1, policy_version 527450 (0.0008) [2023-12-27 09:17:30,757][20381] Updated weights for policy 1, policy_version 527460 (0.0007) [2023-12-27 09:17:30,809][20381] Updated weights for policy 1, policy_version 527470 (0.0008) [2023-12-27 09:17:31,173][20347] Updated weights for policy 0, policy_version 527850 (0.0007) [2023-12-27 09:17:31,225][20347] Updated weights for policy 0, policy_version 527860 (0.0007) [2023-12-27 09:17:31,289][20347] Updated weights for policy 0, policy_version 527870 (0.0006) [2023-12-27 09:17:31,475][20381] Updated weights for policy 1, policy_version 527480 (0.0007) [2023-12-27 09:17:31,526][20381] Updated weights for policy 1, policy_version 527490 (0.0005) [2023-12-27 09:17:31,586][20381] Updated weights for policy 1, policy_version 527500 (0.0007) [2023-12-27 09:17:31,957][20347] Updated weights for policy 0, policy_version 527880 (0.0006) [2023-12-27 09:17:32,016][20347] Updated weights for policy 0, policy_version 527890 (0.0005) [2023-12-27 09:17:32,062][20347] Updated weights for policy 0, policy_version 527900 (0.0007) [2023-12-27 09:17:32,290][20381] Updated weights for policy 1, policy_version 527510 (0.0008) [2023-12-27 09:17:32,346][20381] Updated weights for policy 1, policy_version 527520 (0.0008) [2023-12-27 09:17:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 270221312. Throughput: 0: 10561.4, 1: 10272.6. Samples: 270234436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.500')] [2023-12-27 09:17:32,403][20381] Updated weights for policy 1, policy_version 527530 (0.0007) [2023-12-27 09:17:32,688][20347] Updated weights for policy 0, policy_version 527910 (0.0006) [2023-12-27 09:17:32,737][20347] Updated weights for policy 0, policy_version 527920 (0.0006) [2023-12-27 09:17:32,782][20347] Updated weights for policy 0, policy_version 527930 (0.0009) [2023-12-27 09:17:33,068][20381] Updated weights for policy 1, policy_version 527540 (0.0006) [2023-12-27 09:17:33,121][20381] Updated weights for policy 1, policy_version 527550 (0.0007) [2023-12-27 09:17:33,175][20381] Updated weights for policy 1, policy_version 527560 (0.0008) [2023-12-27 09:17:33,424][20347] Updated weights for policy 0, policy_version 527940 (0.0008) [2023-12-27 09:17:33,480][20347] Updated weights for policy 0, policy_version 527950 (0.0006) [2023-12-27 09:17:33,533][20347] Updated weights for policy 0, policy_version 527960 (0.0007) [2023-12-27 09:17:33,823][20381] Updated weights for policy 1, policy_version 527570 (0.0010) [2023-12-27 09:17:33,884][20381] Updated weights for policy 1, policy_version 527580 (0.0008) [2023-12-27 09:17:33,942][20381] Updated weights for policy 1, policy_version 527590 (0.0008) [2023-12-27 09:17:34,019][20381] Updated weights for policy 1, policy_version 527600 (0.0007) [2023-12-27 09:17:34,255][20347] Updated weights for policy 0, policy_version 527970 (0.0008) [2023-12-27 09:17:34,308][20347] Updated weights for policy 0, policy_version 527980 (0.0008) [2023-12-27 09:17:34,367][20347] Updated weights for policy 0, policy_version 527990 (0.0007) [2023-12-27 09:17:34,426][20347] Updated weights for policy 0, policy_version 528000 (0.0007) [2023-12-27 09:17:34,623][20381] Updated weights for policy 1, policy_version 527610 (0.0007) [2023-12-27 09:17:34,680][20381] Updated weights for policy 1, policy_version 527620 (0.0005) [2023-12-27 09:17:34,737][20381] Updated weights for policy 1, policy_version 527630 (0.0005) [2023-12-27 09:17:35,150][20347] Updated weights for policy 0, policy_version 528010 (0.0008) [2023-12-27 09:17:35,201][20347] Updated weights for policy 0, policy_version 528020 (0.0009) [2023-12-27 09:17:35,257][20347] Updated weights for policy 0, policy_version 528030 (0.0006) [2023-12-27 09:17:35,275][20381] Updated weights for policy 1, policy_version 527640 (0.0005) [2023-12-27 09:17:35,326][20381] Updated weights for policy 1, policy_version 527650 (0.0005) [2023-12-27 09:17:35,373][20381] Updated weights for policy 1, policy_version 527660 (0.0005) [2023-12-27 09:17:35,905][20381] Updated weights for policy 1, policy_version 527670 (0.0007) [2023-12-27 09:17:35,945][20347] Updated weights for policy 0, policy_version 528040 (0.0005) [2023-12-27 09:17:35,954][20381] Updated weights for policy 1, policy_version 527680 (0.0008) [2023-12-27 09:17:35,994][20347] Updated weights for policy 0, policy_version 528050 (0.0005) [2023-12-27 09:17:36,002][20381] Updated weights for policy 1, policy_version 527690 (0.0006) [2023-12-27 09:17:36,041][20347] Updated weights for policy 0, policy_version 528060 (0.0007) [2023-12-27 09:17:36,603][20381] Updated weights for policy 1, policy_version 527700 (0.0006) [2023-12-27 09:17:36,663][20381] Updated weights for policy 1, policy_version 527710 (0.0009) [2023-12-27 09:17:36,729][20381] Updated weights for policy 1, policy_version 527720 (0.0011) [2023-12-27 09:17:36,837][20347] Updated weights for policy 0, policy_version 528070 (0.0007) [2023-12-27 09:17:36,902][20347] Updated weights for policy 0, policy_version 528080 (0.0008) [2023-12-27 09:17:36,958][20347] Updated weights for policy 0, policy_version 528090 (0.0008) [2023-12-27 09:17:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 270327808. Throughput: 0: 10491.4, 1: 10308.6. Samples: 270298100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.290')] [2023-12-27 09:17:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000527728_135118848.pth... [2023-12-27 09:17:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000528096_135208960.pth... [2023-12-27 09:17:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000526544_134815744.pth [2023-12-27 09:17:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000526880_134897664.pth [2023-12-27 09:17:37,413][20381] Updated weights for policy 1, policy_version 527730 (0.0011) [2023-12-27 09:17:37,481][20381] Updated weights for policy 1, policy_version 527740 (0.0011) [2023-12-27 09:17:37,548][20381] Updated weights for policy 1, policy_version 527750 (0.0011) [2023-12-27 09:17:37,607][20381] Updated weights for policy 1, policy_version 527760 (0.0011) [2023-12-27 09:17:37,661][20347] Updated weights for policy 0, policy_version 528100 (0.0009) [2023-12-27 09:17:37,721][20347] Updated weights for policy 0, policy_version 528110 (0.0010) [2023-12-27 09:17:37,782][20347] Updated weights for policy 0, policy_version 528120 (0.0010) [2023-12-27 09:17:38,316][20381] Updated weights for policy 1, policy_version 527770 (0.0011) [2023-12-27 09:17:38,379][20381] Updated weights for policy 1, policy_version 527780 (0.0010) [2023-12-27 09:17:38,441][20381] Updated weights for policy 1, policy_version 527790 (0.0011) [2023-12-27 09:17:38,490][20347] Updated weights for policy 0, policy_version 528130 (0.0009) [2023-12-27 09:17:38,548][20347] Updated weights for policy 0, policy_version 528140 (0.0007) [2023-12-27 09:17:38,602][20347] Updated weights for policy 0, policy_version 528150 (0.0007) [2023-12-27 09:17:38,663][20347] Updated weights for policy 0, policy_version 528160 (0.0007) [2023-12-27 09:17:39,133][20381] Updated weights for policy 1, policy_version 527800 (0.0011) [2023-12-27 09:17:39,182][20381] Updated weights for policy 1, policy_version 527810 (0.0010) [2023-12-27 09:17:39,242][20381] Updated weights for policy 1, policy_version 527820 (0.0010) [2023-12-27 09:17:39,382][20347] Updated weights for policy 0, policy_version 528170 (0.0009) [2023-12-27 09:17:39,448][20347] Updated weights for policy 0, policy_version 528180 (0.0006) [2023-12-27 09:17:39,501][20347] Updated weights for policy 0, policy_version 528190 (0.0008) [2023-12-27 09:17:39,926][20381] Updated weights for policy 1, policy_version 527830 (0.0011) [2023-12-27 09:17:39,984][20381] Updated weights for policy 1, policy_version 527840 (0.0009) [2023-12-27 09:17:40,044][20381] Updated weights for policy 1, policy_version 527850 (0.0011) [2023-12-27 09:17:40,204][20347] Updated weights for policy 0, policy_version 528200 (0.0009) [2023-12-27 09:17:40,267][20347] Updated weights for policy 0, policy_version 528210 (0.0005) [2023-12-27 09:17:40,330][20347] Updated weights for policy 0, policy_version 528220 (0.0006) [2023-12-27 09:17:40,714][20381] Updated weights for policy 1, policy_version 527860 (0.0008) [2023-12-27 09:17:40,768][20381] Updated weights for policy 1, policy_version 527870 (0.0007) [2023-12-27 09:17:40,824][20381] Updated weights for policy 1, policy_version 527880 (0.0005) [2023-12-27 09:17:40,991][20347] Updated weights for policy 0, policy_version 528230 (0.0009) [2023-12-27 09:17:41,050][20347] Updated weights for policy 0, policy_version 528240 (0.0007) [2023-12-27 09:17:41,116][20347] Updated weights for policy 0, policy_version 528250 (0.0009) [2023-12-27 09:17:41,513][20381] Updated weights for policy 1, policy_version 527890 (0.0006) [2023-12-27 09:17:41,563][20381] Updated weights for policy 1, policy_version 527900 (0.0008) [2023-12-27 09:17:41,617][20381] Updated weights for policy 1, policy_version 527910 (0.0009) [2023-12-27 09:17:41,675][20381] Updated weights for policy 1, policy_version 527920 (0.0008) [2023-12-27 09:17:41,779][20347] Updated weights for policy 0, policy_version 528260 (0.0007) [2023-12-27 09:17:41,833][20347] Updated weights for policy 0, policy_version 528270 (0.0006) [2023-12-27 09:17:41,889][20347] Updated weights for policy 0, policy_version 528280 (0.0005) [2023-12-27 09:17:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 270426112. Throughput: 0: 10368.7, 1: 10348.7. Samples: 270420024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:42,363][19170] Avg episode reward: [(0, '981.000'), (1, '863.780')] [2023-12-27 09:17:42,428][20381] Updated weights for policy 1, policy_version 527930 (0.0007) [2023-12-27 09:17:42,485][20381] Updated weights for policy 1, policy_version 527940 (0.0009) [2023-12-27 09:17:42,512][20347] Updated weights for policy 0, policy_version 528290 (0.0008) [2023-12-27 09:17:42,540][20381] Updated weights for policy 1, policy_version 527950 (0.0006) [2023-12-27 09:17:42,565][20347] Updated weights for policy 0, policy_version 528300 (0.0008) [2023-12-27 09:17:42,633][20347] Updated weights for policy 0, policy_version 528310 (0.0008) [2023-12-27 09:17:42,683][20347] Updated weights for policy 0, policy_version 528320 (0.0009) [2023-12-27 09:17:43,149][20381] Updated weights for policy 1, policy_version 527960 (0.0007) [2023-12-27 09:17:43,210][20381] Updated weights for policy 1, policy_version 527970 (0.0008) [2023-12-27 09:17:43,265][20381] Updated weights for policy 1, policy_version 527980 (0.0006) [2023-12-27 09:17:43,427][20347] Updated weights for policy 0, policy_version 528330 (0.0009) [2023-12-27 09:17:43,477][20347] Updated weights for policy 0, policy_version 528340 (0.0007) [2023-12-27 09:17:43,526][20347] Updated weights for policy 0, policy_version 528350 (0.0007) [2023-12-27 09:17:43,856][20381] Updated weights for policy 1, policy_version 527990 (0.0005) [2023-12-27 09:17:43,913][20381] Updated weights for policy 1, policy_version 528000 (0.0008) [2023-12-27 09:17:43,964][20381] Updated weights for policy 1, policy_version 528010 (0.0005) [2023-12-27 09:17:44,253][20347] Updated weights for policy 0, policy_version 528360 (0.0008) [2023-12-27 09:17:44,315][20347] Updated weights for policy 0, policy_version 528370 (0.0008) [2023-12-27 09:17:44,375][20347] Updated weights for policy 0, policy_version 528380 (0.0008) [2023-12-27 09:17:44,576][20381] Updated weights for policy 1, policy_version 528020 (0.0006) [2023-12-27 09:17:44,623][20381] Updated weights for policy 1, policy_version 528030 (0.0007) [2023-12-27 09:17:44,669][20381] Updated weights for policy 1, policy_version 528040 (0.0007) [2023-12-27 09:17:44,977][20347] Updated weights for policy 0, policy_version 528390 (0.0006) [2023-12-27 09:17:45,041][20347] Updated weights for policy 0, policy_version 528400 (0.0006) [2023-12-27 09:17:45,105][20347] Updated weights for policy 0, policy_version 528410 (0.0005) [2023-12-27 09:17:45,381][20381] Updated weights for policy 1, policy_version 528050 (0.0008) [2023-12-27 09:17:45,437][20381] Updated weights for policy 1, policy_version 528060 (0.0007) [2023-12-27 09:17:45,484][20381] Updated weights for policy 1, policy_version 528070 (0.0008) [2023-12-27 09:17:45,539][20381] Updated weights for policy 1, policy_version 528080 (0.0007) [2023-12-27 09:17:45,751][20347] Updated weights for policy 0, policy_version 528420 (0.0008) [2023-12-27 09:17:45,804][20347] Updated weights for policy 0, policy_version 528430 (0.0009) [2023-12-27 09:17:45,853][20347] Updated weights for policy 0, policy_version 528440 (0.0010) [2023-12-27 09:17:46,195][20381] Updated weights for policy 1, policy_version 528090 (0.0007) [2023-12-27 09:17:46,248][20381] Updated weights for policy 1, policy_version 528100 (0.0007) [2023-12-27 09:17:46,294][20381] Updated weights for policy 1, policy_version 528110 (0.0008) [2023-12-27 09:17:46,510][20347] Updated weights for policy 0, policy_version 528450 (0.0008) [2023-12-27 09:17:46,572][20347] Updated weights for policy 0, policy_version 528460 (0.0005) [2023-12-27 09:17:46,625][20347] Updated weights for policy 0, policy_version 528470 (0.0005) [2023-12-27 09:17:46,672][20347] Updated weights for policy 0, policy_version 528480 (0.0006) [2023-12-27 09:17:47,024][20381] Updated weights for policy 1, policy_version 528120 (0.0008) [2023-12-27 09:17:47,081][20381] Updated weights for policy 1, policy_version 528130 (0.0005) [2023-12-27 09:17:47,136][20381] Updated weights for policy 1, policy_version 528140 (0.0005) [2023-12-27 09:17:47,328][20347] Updated weights for policy 0, policy_version 528491 (0.0009) [2023-12-27 09:17:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 270532608. Throughput: 0: 10374.7, 1: 10282.9. Samples: 270545044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:47,363][19170] Avg episode reward: [(0, '981.000'), (1, '912.590')] [2023-12-27 09:17:47,372][20347] Updated weights for policy 0, policy_version 528501 (0.0008) [2023-12-27 09:17:47,431][20347] Updated weights for policy 0, policy_version 528511 (0.0007) [2023-12-27 09:17:47,705][20381] Updated weights for policy 1, policy_version 528150 (0.0007) [2023-12-27 09:17:47,751][20381] Updated weights for policy 1, policy_version 528160 (0.0007) [2023-12-27 09:17:47,809][20381] Updated weights for policy 1, policy_version 528170 (0.0009) [2023-12-27 09:17:48,127][20347] Updated weights for policy 0, policy_version 528521 (0.0005) [2023-12-27 09:17:48,176][20347] Updated weights for policy 0, policy_version 528531 (0.0005) [2023-12-27 09:17:48,238][20347] Updated weights for policy 0, policy_version 528541 (0.0006) [2023-12-27 09:17:48,529][20381] Updated weights for policy 1, policy_version 528180 (0.0008) [2023-12-27 09:17:48,585][20381] Updated weights for policy 1, policy_version 528190 (0.0008) [2023-12-27 09:17:48,641][20381] Updated weights for policy 1, policy_version 528200 (0.0007) [2023-12-27 09:17:48,869][20347] Updated weights for policy 0, policy_version 528551 (0.0006) [2023-12-27 09:17:48,913][20347] Updated weights for policy 0, policy_version 528561 (0.0005) [2023-12-27 09:17:48,966][20347] Updated weights for policy 0, policy_version 528571 (0.0005) [2023-12-27 09:17:49,407][20381] Updated weights for policy 1, policy_version 528210 (0.0008) [2023-12-27 09:17:49,460][20381] Updated weights for policy 1, policy_version 528220 (0.0008) [2023-12-27 09:17:49,507][20381] Updated weights for policy 1, policy_version 528230 (0.0008) [2023-12-27 09:17:49,555][20381] Updated weights for policy 1, policy_version 528240 (0.0008) [2023-12-27 09:17:49,608][20347] Updated weights for policy 0, policy_version 528581 (0.0006) [2023-12-27 09:17:49,661][20347] Updated weights for policy 0, policy_version 528591 (0.0006) [2023-12-27 09:17:49,711][20347] Updated weights for policy 0, policy_version 528601 (0.0008) [2023-12-27 09:17:50,274][20381] Updated weights for policy 1, policy_version 528250 (0.0007) [2023-12-27 09:17:50,324][20381] Updated weights for policy 1, policy_version 528260 (0.0008) [2023-12-27 09:17:50,383][20381] Updated weights for policy 1, policy_version 528270 (0.0008) [2023-12-27 09:17:50,401][20347] Updated weights for policy 0, policy_version 528611 (0.0006) [2023-12-27 09:17:50,439][20291] KL-divergence is very high: 171.2019 [2023-12-27 09:17:50,447][20347] Updated weights for policy 0, policy_version 528621 (0.0005) [2023-12-27 09:17:50,506][20347] Updated weights for policy 0, policy_version 528631 (0.0006) [2023-12-27 09:17:51,106][20381] Updated weights for policy 1, policy_version 528280 (0.0008) [2023-12-27 09:17:51,168][20381] Updated weights for policy 1, policy_version 528290 (0.0008) [2023-12-27 09:17:51,193][20347] Updated weights for policy 0, policy_version 528641 (0.0006) [2023-12-27 09:17:51,201][20291] KL-divergence is very high: 441.7101 [2023-12-27 09:17:51,226][20381] Updated weights for policy 1, policy_version 528300 (0.0006) [2023-12-27 09:17:51,252][20291] KL-divergence is very high: 1415.3907 [2023-12-27 09:17:51,257][20347] Updated weights for policy 0, policy_version 528651 (0.0007) [2023-12-27 09:17:51,265][20291] KL-divergence is very high: 143.8548 [2023-12-27 09:17:51,319][20347] Updated weights for policy 0, policy_version 528661 (0.0008) [2023-12-27 09:17:51,345][20291] KL-divergence is very high: 437.1639 [2023-12-27 09:17:51,357][20291] KL-divergence is very high: 259.6101 [2023-12-27 09:17:51,375][20347] Updated weights for policy 0, policy_version 528671 (0.0008) [2023-12-27 09:17:51,963][20381] Updated weights for policy 1, policy_version 528310 (0.0007) [2023-12-27 09:17:52,018][20381] Updated weights for policy 1, policy_version 528320 (0.0008) [2023-12-27 09:17:52,063][20381] Updated weights for policy 1, policy_version 528330 (0.0007) [2023-12-27 09:17:52,121][20347] Updated weights for policy 0, policy_version 528681 (0.0010) [2023-12-27 09:17:52,176][20347] Updated weights for policy 0, policy_version 528691 (0.0010) [2023-12-27 09:17:52,225][20347] Updated weights for policy 0, policy_version 528701 (0.0010) [2023-12-27 09:17:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 270639104. Throughput: 0: 10402.3, 1: 10290.6. Samples: 270607092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:52,363][19170] Avg episode reward: [(0, '971.120'), (1, '902.850')] [2023-12-27 09:17:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000528336_135274496.pth... [2023-12-27 09:17:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000528704_135364608.pth... [2023-12-27 09:17:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000527120_134963200.pth [2023-12-27 09:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000527488_135053312.pth [2023-12-27 09:17:52,778][20381] Updated weights for policy 1, policy_version 528340 (0.0007) [2023-12-27 09:17:52,828][20381] Updated weights for policy 1, policy_version 528350 (0.0008) [2023-12-27 09:17:52,890][20381] Updated weights for policy 1, policy_version 528360 (0.0007) [2023-12-27 09:17:52,950][20347] Updated weights for policy 0, policy_version 528711 (0.0010) [2023-12-27 09:17:53,011][20347] Updated weights for policy 0, policy_version 528721 (0.0010) [2023-12-27 09:17:53,071][20347] Updated weights for policy 0, policy_version 528731 (0.0010) [2023-12-27 09:17:53,596][20381] Updated weights for policy 1, policy_version 528370 (0.0007) [2023-12-27 09:17:53,648][20381] Updated weights for policy 1, policy_version 528380 (0.0008) [2023-12-27 09:17:53,709][20381] Updated weights for policy 1, policy_version 528390 (0.0007) [2023-12-27 09:17:53,732][20347] Updated weights for policy 0, policy_version 528741 (0.0008) [2023-12-27 09:17:53,755][20291] KL-divergence is very high: 180.9629 [2023-12-27 09:17:53,764][20381] Updated weights for policy 1, policy_version 528400 (0.0005) [2023-12-27 09:17:53,785][20291] KL-divergence is very high: 311.3595 [2023-12-27 09:17:53,792][20347] Updated weights for policy 0, policy_version 528751 (0.0006) [2023-12-27 09:17:53,801][20291] KL-divergence is very high: 422.4773 [2023-12-27 09:17:53,828][20291] KL-divergence is very high: 140.8334 [2023-12-27 09:17:53,844][20347] Updated weights for policy 0, policy_version 528761 (0.0008) [2023-12-27 09:17:53,846][20291] KL-divergence is very high: 136.2319 [2023-12-27 09:17:54,436][20347] Updated weights for policy 0, policy_version 528771 (0.0007) [2023-12-27 09:17:54,451][20381] Updated weights for policy 1, policy_version 528410 (0.0008) [2023-12-27 09:17:54,487][20347] Updated weights for policy 0, policy_version 528781 (0.0005) [2023-12-27 09:17:54,507][20381] Updated weights for policy 1, policy_version 528420 (0.0006) [2023-12-27 09:17:54,546][20347] Updated weights for policy 0, policy_version 528791 (0.0006) [2023-12-27 09:17:54,561][20381] Updated weights for policy 1, policy_version 528430 (0.0005) [2023-12-27 09:17:55,175][20347] Updated weights for policy 0, policy_version 528801 (0.0007) [2023-12-27 09:17:55,191][20381] Updated weights for policy 1, policy_version 528440 (0.0006) [2023-12-27 09:17:55,226][20347] Updated weights for policy 0, policy_version 528811 (0.0010) [2023-12-27 09:17:55,246][20381] Updated weights for policy 1, policy_version 528450 (0.0005) [2023-12-27 09:17:55,281][20347] Updated weights for policy 0, policy_version 528821 (0.0010) [2023-12-27 09:17:55,301][20381] Updated weights for policy 1, policy_version 528460 (0.0006) [2023-12-27 09:17:55,329][20347] Updated weights for policy 0, policy_version 528831 (0.0009) [2023-12-27 09:17:55,958][20381] Updated weights for policy 1, policy_version 528470 (0.0006) [2023-12-27 09:17:55,999][20291] KL-divergence is very high: 100.2907 [2023-12-27 09:17:56,005][20347] Updated weights for policy 0, policy_version 528841 (0.0008) [2023-12-27 09:17:56,009][20381] Updated weights for policy 1, policy_version 528480 (0.0007) [2023-12-27 09:17:56,070][20347] Updated weights for policy 0, policy_version 528851 (0.0006) [2023-12-27 09:17:56,070][20381] Updated weights for policy 1, policy_version 528490 (0.0007) [2023-12-27 09:17:56,137][20347] Updated weights for policy 0, policy_version 528861 (0.0010) [2023-12-27 09:17:56,658][20381] Updated weights for policy 1, policy_version 528500 (0.0006) [2023-12-27 09:17:56,717][20381] Updated weights for policy 1, policy_version 528510 (0.0007) [2023-12-27 09:17:56,753][20347] Updated weights for policy 0, policy_version 528871 (0.0007) [2023-12-27 09:17:56,766][20381] Updated weights for policy 1, policy_version 528520 (0.0007) [2023-12-27 09:17:56,814][20347] Updated weights for policy 0, policy_version 528881 (0.0005) [2023-12-27 09:17:56,875][20347] Updated weights for policy 0, policy_version 528891 (0.0005) [2023-12-27 09:17:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 270737408. Throughput: 0: 10403.5, 1: 10298.3. Samples: 270729504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:17:57,363][19170] Avg episode reward: [(0, '922.610'), (1, '883.210')] [2023-12-27 09:17:57,406][20347] Updated weights for policy 0, policy_version 528901 (0.0005) [2023-12-27 09:17:57,454][20347] Updated weights for policy 0, policy_version 528911 (0.0005) [2023-12-27 09:17:57,501][20347] Updated weights for policy 0, policy_version 528921 (0.0005) [2023-12-27 09:17:57,516][20381] Updated weights for policy 1, policy_version 528530 (0.0007) [2023-12-27 09:17:57,565][20381] Updated weights for policy 1, policy_version 528540 (0.0006) [2023-12-27 09:17:57,618][20381] Updated weights for policy 1, policy_version 528550 (0.0007) [2023-12-27 09:17:57,669][20381] Updated weights for policy 1, policy_version 528560 (0.0007) [2023-12-27 09:17:58,117][20347] Updated weights for policy 0, policy_version 528931 (0.0010) [2023-12-27 09:17:58,178][20347] Updated weights for policy 0, policy_version 528941 (0.0010) [2023-12-27 09:17:58,236][20347] Updated weights for policy 0, policy_version 528951 (0.0010) [2023-12-27 09:17:58,399][20381] Updated weights for policy 1, policy_version 528570 (0.0008) [2023-12-27 09:17:58,462][20381] Updated weights for policy 1, policy_version 528580 (0.0008) [2023-12-27 09:17:58,523][20381] Updated weights for policy 1, policy_version 528590 (0.0008) [2023-12-27 09:17:58,971][20347] Updated weights for policy 0, policy_version 528961 (0.0010) [2023-12-27 09:17:59,016][20347] Updated weights for policy 0, policy_version 528971 (0.0005) [2023-12-27 09:17:59,063][20347] Updated weights for policy 0, policy_version 528981 (0.0006) [2023-12-27 09:17:59,126][20347] Updated weights for policy 0, policy_version 528991 (0.0006) [2023-12-27 09:17:59,285][20381] Updated weights for policy 1, policy_version 528600 (0.0008) [2023-12-27 09:17:59,346][20381] Updated weights for policy 1, policy_version 528610 (0.0008) [2023-12-27 09:17:59,407][20381] Updated weights for policy 1, policy_version 528620 (0.0007) [2023-12-27 09:17:59,762][20347] Updated weights for policy 0, policy_version 529001 (0.0007) [2023-12-27 09:17:59,818][20347] Updated weights for policy 0, policy_version 529011 (0.0007) [2023-12-27 09:17:59,880][20347] Updated weights for policy 0, policy_version 529021 (0.0008) [2023-12-27 09:18:00,107][20381] Updated weights for policy 1, policy_version 528630 (0.0006) [2023-12-27 09:18:00,162][20381] Updated weights for policy 1, policy_version 528640 (0.0007) [2023-12-27 09:18:00,212][20381] Updated weights for policy 1, policy_version 528650 (0.0006) [2023-12-27 09:18:00,572][20347] Updated weights for policy 0, policy_version 529031 (0.0007) [2023-12-27 09:18:00,629][20347] Updated weights for policy 0, policy_version 529041 (0.0007) [2023-12-27 09:18:00,688][20347] Updated weights for policy 0, policy_version 529051 (0.0007) [2023-12-27 09:18:00,892][20381] Updated weights for policy 1, policy_version 528660 (0.0009) [2023-12-27 09:18:00,946][20381] Updated weights for policy 1, policy_version 528670 (0.0007) [2023-12-27 09:18:01,004][20381] Updated weights for policy 1, policy_version 528680 (0.0007) [2023-12-27 09:18:01,338][20347] Updated weights for policy 0, policy_version 529061 (0.0007) [2023-12-27 09:18:01,401][20347] Updated weights for policy 0, policy_version 529071 (0.0007) [2023-12-27 09:18:01,458][20347] Updated weights for policy 0, policy_version 529081 (0.0008) [2023-12-27 09:18:01,671][20381] Updated weights for policy 1, policy_version 528690 (0.0007) [2023-12-27 09:18:01,731][20381] Updated weights for policy 1, policy_version 528700 (0.0009) [2023-12-27 09:18:01,785][20381] Updated weights for policy 1, policy_version 528710 (0.0008) [2023-12-27 09:18:01,843][20381] Updated weights for policy 1, policy_version 528720 (0.0008) [2023-12-27 09:18:02,093][20347] Updated weights for policy 0, policy_version 529091 (0.0006) [2023-12-27 09:18:02,152][20347] Updated weights for policy 0, policy_version 529101 (0.0005) [2023-12-27 09:18:02,210][20347] Updated weights for policy 0, policy_version 529111 (0.0006) [2023-12-27 09:18:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 270843904. Throughput: 0: 10397.7, 1: 10259.6. Samples: 270853136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:02,363][19170] Avg episode reward: [(0, '903.030'), (1, '824.970')] [2023-12-27 09:18:02,559][20381] Updated weights for policy 1, policy_version 528730 (0.0008) [2023-12-27 09:18:02,617][20381] Updated weights for policy 1, policy_version 528740 (0.0007) [2023-12-27 09:18:02,669][20381] Updated weights for policy 1, policy_version 528750 (0.0007) [2023-12-27 09:18:02,798][20347] Updated weights for policy 0, policy_version 529121 (0.0006) [2023-12-27 09:18:02,851][20347] Updated weights for policy 0, policy_version 529131 (0.0008) [2023-12-27 09:18:02,896][20347] Updated weights for policy 0, policy_version 529141 (0.0010) [2023-12-27 09:18:02,942][20347] Updated weights for policy 0, policy_version 529151 (0.0006) [2023-12-27 09:18:03,327][20381] Updated weights for policy 1, policy_version 528760 (0.0007) [2023-12-27 09:18:03,380][20381] Updated weights for policy 1, policy_version 528770 (0.0007) [2023-12-27 09:18:03,432][20381] Updated weights for policy 1, policy_version 528780 (0.0007) [2023-12-27 09:18:03,616][20347] Updated weights for policy 0, policy_version 529161 (0.0010) [2023-12-27 09:18:03,682][20347] Updated weights for policy 0, policy_version 529171 (0.0010) [2023-12-27 09:18:03,737][20347] Updated weights for policy 0, policy_version 529181 (0.0010) [2023-12-27 09:18:04,063][20381] Updated weights for policy 1, policy_version 528790 (0.0008) [2023-12-27 09:18:04,124][20381] Updated weights for policy 1, policy_version 528800 (0.0008) [2023-12-27 09:18:04,181][20381] Updated weights for policy 1, policy_version 528810 (0.0010) [2023-12-27 09:18:04,398][20347] Updated weights for policy 0, policy_version 529191 (0.0010) [2023-12-27 09:18:04,457][20347] Updated weights for policy 0, policy_version 529201 (0.0007) [2023-12-27 09:18:04,518][20347] Updated weights for policy 0, policy_version 529211 (0.0008) [2023-12-27 09:18:04,880][20381] Updated weights for policy 1, policy_version 528820 (0.0009) [2023-12-27 09:18:04,938][20381] Updated weights for policy 1, policy_version 528830 (0.0008) [2023-12-27 09:18:04,989][20381] Updated weights for policy 1, policy_version 528840 (0.0008) [2023-12-27 09:18:05,229][20347] Updated weights for policy 0, policy_version 529221 (0.0009) [2023-12-27 09:18:05,274][20347] Updated weights for policy 0, policy_version 529231 (0.0009) [2023-12-27 09:18:05,327][20347] Updated weights for policy 0, policy_version 529241 (0.0009) [2023-12-27 09:18:05,566][20381] Updated weights for policy 1, policy_version 528850 (0.0008) [2023-12-27 09:18:05,620][20381] Updated weights for policy 1, policy_version 528860 (0.0010) [2023-12-27 09:18:05,680][20381] Updated weights for policy 1, policy_version 528870 (0.0010) [2023-12-27 09:18:05,729][20381] Updated weights for policy 1, policy_version 528880 (0.0006) [2023-12-27 09:18:06,050][20347] Updated weights for policy 0, policy_version 529251 (0.0006) [2023-12-27 09:18:06,115][20347] Updated weights for policy 0, policy_version 529261 (0.0008) [2023-12-27 09:18:06,171][20347] Updated weights for policy 0, policy_version 529271 (0.0007) [2023-12-27 09:18:06,340][20381] Updated weights for policy 1, policy_version 528890 (0.0008) [2023-12-27 09:18:06,398][20381] Updated weights for policy 1, policy_version 528900 (0.0008) [2023-12-27 09:18:06,454][20381] Updated weights for policy 1, policy_version 528910 (0.0009) [2023-12-27 09:18:06,908][20347] Updated weights for policy 0, policy_version 529281 (0.0008) [2023-12-27 09:18:06,966][20347] Updated weights for policy 0, policy_version 529291 (0.0007) [2023-12-27 09:18:07,021][20347] Updated weights for policy 0, policy_version 529301 (0.0007) [2023-12-27 09:18:07,078][20347] Updated weights for policy 0, policy_version 529311 (0.0007) [2023-12-27 09:18:07,087][20381] Updated weights for policy 1, policy_version 528920 (0.0006) [2023-12-27 09:18:07,136][20381] Updated weights for policy 1, policy_version 528930 (0.0005) [2023-12-27 09:18:07,182][20381] Updated weights for policy 1, policy_version 528940 (0.0005) [2023-12-27 09:18:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.4, 300 sec: 20743.8). Total num frames: 270950400. Throughput: 0: 10366.8, 1: 10258.3. Samples: 270915808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:07,364][19170] Avg episode reward: [(0, '923.300'), (1, '873.640')] [2023-12-27 09:18:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000528944_135430144.pth... [2023-12-27 09:18:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000529312_135520256.pth... [2023-12-27 09:18:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000528096_135208960.pth [2023-12-27 09:18:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000527728_135118848.pth [2023-12-27 09:18:07,713][20381] Updated weights for policy 1, policy_version 528950 (0.0007) [2023-12-27 09:18:07,758][20381] Updated weights for policy 1, policy_version 528960 (0.0010) [2023-12-27 09:18:07,768][20347] Updated weights for policy 0, policy_version 529321 (0.0009) [2023-12-27 09:18:07,803][20381] Updated weights for policy 1, policy_version 528970 (0.0010) [2023-12-27 09:18:07,822][20347] Updated weights for policy 0, policy_version 529331 (0.0005) [2023-12-27 09:18:07,878][20347] Updated weights for policy 0, policy_version 529341 (0.0006) [2023-12-27 09:18:08,557][20381] Updated weights for policy 1, policy_version 528980 (0.0009) [2023-12-27 09:18:08,566][20347] Updated weights for policy 0, policy_version 529351 (0.0010) [2023-12-27 09:18:08,615][20381] Updated weights for policy 1, policy_version 528990 (0.0007) [2023-12-27 09:18:08,633][20347] Updated weights for policy 0, policy_version 529361 (0.0007) [2023-12-27 09:18:08,668][20381] Updated weights for policy 1, policy_version 529000 (0.0006) [2023-12-27 09:18:08,692][20347] Updated weights for policy 0, policy_version 529371 (0.0011) [2023-12-27 09:18:09,346][20347] Updated weights for policy 0, policy_version 529381 (0.0010) [2023-12-27 09:18:09,389][20381] Updated weights for policy 1, policy_version 529010 (0.0006) [2023-12-27 09:18:09,406][20347] Updated weights for policy 0, policy_version 529391 (0.0011) [2023-12-27 09:18:09,456][20381] Updated weights for policy 1, policy_version 529020 (0.0007) [2023-12-27 09:18:09,469][20347] Updated weights for policy 0, policy_version 529401 (0.0007) [2023-12-27 09:18:09,518][20381] Updated weights for policy 1, policy_version 529030 (0.0007) [2023-12-27 09:18:09,578][20381] Updated weights for policy 1, policy_version 529040 (0.0009) [2023-12-27 09:18:10,168][20347] Updated weights for policy 0, policy_version 529411 (0.0006) [2023-12-27 09:18:10,226][20347] Updated weights for policy 0, policy_version 529421 (0.0008) [2023-12-27 09:18:10,279][20347] Updated weights for policy 0, policy_version 529431 (0.0006) [2023-12-27 09:18:10,288][20381] Updated weights for policy 1, policy_version 529050 (0.0009) [2023-12-27 09:18:10,344][20381] Updated weights for policy 1, policy_version 529060 (0.0006) [2023-12-27 09:18:10,401][20381] Updated weights for policy 1, policy_version 529070 (0.0009) [2023-12-27 09:18:10,996][20347] Updated weights for policy 0, policy_version 529441 (0.0007) [2023-12-27 09:18:11,042][20381] Updated weights for policy 1, policy_version 529080 (0.0008) [2023-12-27 09:18:11,059][20347] Updated weights for policy 0, policy_version 529451 (0.0008) [2023-12-27 09:18:11,099][20381] Updated weights for policy 1, policy_version 529090 (0.0007) [2023-12-27 09:18:11,120][20347] Updated weights for policy 0, policy_version 529461 (0.0007) [2023-12-27 09:18:11,161][20381] Updated weights for policy 1, policy_version 529100 (0.0006) [2023-12-27 09:18:11,188][20347] Updated weights for policy 0, policy_version 529471 (0.0007) [2023-12-27 09:18:11,875][20381] Updated weights for policy 1, policy_version 529110 (0.0007) [2023-12-27 09:18:11,888][20347] Updated weights for policy 0, policy_version 529481 (0.0005) [2023-12-27 09:18:11,932][20381] Updated weights for policy 1, policy_version 529120 (0.0005) [2023-12-27 09:18:11,941][20347] Updated weights for policy 0, policy_version 529491 (0.0006) [2023-12-27 09:18:11,988][20381] Updated weights for policy 1, policy_version 529130 (0.0006) [2023-12-27 09:18:11,997][20347] Updated weights for policy 0, policy_version 529501 (0.0007) [2023-12-27 09:18:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20716.1). Total num frames: 271048704. Throughput: 0: 10297.4, 1: 10370.1. Samples: 271039028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:12,363][19170] Avg episode reward: [(0, '971.860'), (1, '912.690')] [2023-12-27 09:18:12,695][20381] Updated weights for policy 1, policy_version 529140 (0.0006) [2023-12-27 09:18:12,715][20347] Updated weights for policy 0, policy_version 529511 (0.0007) [2023-12-27 09:18:12,747][20381] Updated weights for policy 1, policy_version 529150 (0.0005) [2023-12-27 09:18:12,771][20347] Updated weights for policy 0, policy_version 529521 (0.0007) [2023-12-27 09:18:12,803][20381] Updated weights for policy 1, policy_version 529160 (0.0006) [2023-12-27 09:18:12,820][20347] Updated weights for policy 0, policy_version 529531 (0.0006) [2023-12-27 09:18:13,468][20381] Updated weights for policy 1, policy_version 529170 (0.0006) [2023-12-27 09:18:13,513][20347] Updated weights for policy 0, policy_version 529541 (0.0006) [2023-12-27 09:18:13,527][20381] Updated weights for policy 1, policy_version 529180 (0.0008) [2023-12-27 09:18:13,563][20347] Updated weights for policy 0, policy_version 529551 (0.0005) [2023-12-27 09:18:13,574][20381] Updated weights for policy 1, policy_version 529190 (0.0011) [2023-12-27 09:18:13,618][20347] Updated weights for policy 0, policy_version 529561 (0.0006) [2023-12-27 09:18:13,619][20381] Updated weights for policy 1, policy_version 529200 (0.0008) [2023-12-27 09:18:14,290][20381] Updated weights for policy 1, policy_version 529210 (0.0010) [2023-12-27 09:18:14,306][20347] Updated weights for policy 0, policy_version 529571 (0.0006) [2023-12-27 09:18:14,346][20381] Updated weights for policy 1, policy_version 529220 (0.0010) [2023-12-27 09:18:14,365][20347] Updated weights for policy 0, policy_version 529581 (0.0006) [2023-12-27 09:18:14,399][20381] Updated weights for policy 1, policy_version 529230 (0.0010) [2023-12-27 09:18:14,421][20347] Updated weights for policy 0, policy_version 529591 (0.0008) [2023-12-27 09:18:15,065][20347] Updated weights for policy 0, policy_version 529601 (0.0006) [2023-12-27 09:18:15,093][20381] Updated weights for policy 1, policy_version 529240 (0.0007) [2023-12-27 09:18:15,131][20347] Updated weights for policy 0, policy_version 529611 (0.0010) [2023-12-27 09:18:15,154][20381] Updated weights for policy 1, policy_version 529250 (0.0010) [2023-12-27 09:18:15,192][20347] Updated weights for policy 0, policy_version 529621 (0.0010) [2023-12-27 09:18:15,212][20381] Updated weights for policy 1, policy_version 529260 (0.0006) [2023-12-27 09:18:15,250][20347] Updated weights for policy 0, policy_version 529631 (0.0010) [2023-12-27 09:18:15,863][20347] Updated weights for policy 0, policy_version 529641 (0.0007) [2023-12-27 09:18:15,919][20347] Updated weights for policy 0, policy_version 529651 (0.0007) [2023-12-27 09:18:15,932][20381] Updated weights for policy 1, policy_version 529270 (0.0005) [2023-12-27 09:18:15,973][20347] Updated weights for policy 0, policy_version 529661 (0.0005) [2023-12-27 09:18:15,988][20381] Updated weights for policy 1, policy_version 529280 (0.0008) [2023-12-27 09:18:16,043][20381] Updated weights for policy 1, policy_version 529290 (0.0008) [2023-12-27 09:18:16,602][20381] Updated weights for policy 1, policy_version 529300 (0.0007) [2023-12-27 09:18:16,655][20347] Updated weights for policy 0, policy_version 529671 (0.0006) [2023-12-27 09:18:16,663][20381] Updated weights for policy 1, policy_version 529310 (0.0006) [2023-12-27 09:18:16,710][20347] Updated weights for policy 0, policy_version 529681 (0.0005) [2023-12-27 09:18:16,719][20381] Updated weights for policy 1, policy_version 529320 (0.0006) [2023-12-27 09:18:16,766][20347] Updated weights for policy 0, policy_version 529691 (0.0007) [2023-12-27 09:18:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 271147008. Throughput: 0: 10257.0, 1: 10343.4. Samples: 271161452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:17,363][19170] Avg episode reward: [(0, '980.640'), (1, '912.830')] [2023-12-27 09:18:17,397][20381] Updated weights for policy 1, policy_version 529330 (0.0007) [2023-12-27 09:18:17,453][20381] Updated weights for policy 1, policy_version 529340 (0.0010) [2023-12-27 09:18:17,467][20347] Updated weights for policy 0, policy_version 529701 (0.0007) [2023-12-27 09:18:17,498][20381] Updated weights for policy 1, policy_version 529350 (0.0010) [2023-12-27 09:18:17,525][20347] Updated weights for policy 0, policy_version 529711 (0.0005) [2023-12-27 09:18:17,547][20381] Updated weights for policy 1, policy_version 529360 (0.0010) [2023-12-27 09:18:17,584][20347] Updated weights for policy 0, policy_version 529721 (0.0007) [2023-12-27 09:18:18,153][20381] Updated weights for policy 1, policy_version 529370 (0.0005) [2023-12-27 09:18:18,210][20381] Updated weights for policy 1, policy_version 529380 (0.0010) [2023-12-27 09:18:18,268][20381] Updated weights for policy 1, policy_version 529390 (0.0010) [2023-12-27 09:18:18,310][20347] Updated weights for policy 0, policy_version 529731 (0.0006) [2023-12-27 09:18:18,370][20347] Updated weights for policy 0, policy_version 529741 (0.0007) [2023-12-27 09:18:18,420][20347] Updated weights for policy 0, policy_version 529751 (0.0008) [2023-12-27 09:18:18,866][20381] Updated weights for policy 1, policy_version 529400 (0.0006) [2023-12-27 09:18:18,924][20381] Updated weights for policy 1, policy_version 529410 (0.0005) [2023-12-27 09:18:18,983][20381] Updated weights for policy 1, policy_version 529420 (0.0006) [2023-12-27 09:18:19,140][20347] Updated weights for policy 0, policy_version 529761 (0.0008) [2023-12-27 09:18:19,188][20347] Updated weights for policy 0, policy_version 529771 (0.0010) [2023-12-27 09:18:19,245][20347] Updated weights for policy 0, policy_version 529781 (0.0010) [2023-12-27 09:18:19,304][20347] Updated weights for policy 0, policy_version 529791 (0.0009) [2023-12-27 09:18:19,573][20381] Updated weights for policy 1, policy_version 529430 (0.0010) [2023-12-27 09:18:19,630][20381] Updated weights for policy 1, policy_version 529440 (0.0011) [2023-12-27 09:18:19,691][20381] Updated weights for policy 1, policy_version 529450 (0.0009) [2023-12-27 09:18:19,985][20347] Updated weights for policy 0, policy_version 529801 (0.0008) [2023-12-27 09:18:20,050][20347] Updated weights for policy 0, policy_version 529811 (0.0008) [2023-12-27 09:18:20,118][20347] Updated weights for policy 0, policy_version 529821 (0.0010) [2023-12-27 09:18:20,394][20381] Updated weights for policy 1, policy_version 529460 (0.0009) [2023-12-27 09:18:20,460][20381] Updated weights for policy 1, policy_version 529470 (0.0007) [2023-12-27 09:18:20,524][20381] Updated weights for policy 1, policy_version 529480 (0.0006) [2023-12-27 09:18:20,771][20347] Updated weights for policy 0, policy_version 529831 (0.0007) [2023-12-27 09:18:20,833][20347] Updated weights for policy 0, policy_version 529841 (0.0005) [2023-12-27 09:18:20,892][20347] Updated weights for policy 0, policy_version 529851 (0.0006) [2023-12-27 09:18:21,171][20381] Updated weights for policy 1, policy_version 529490 (0.0009) [2023-12-27 09:18:21,228][20381] Updated weights for policy 1, policy_version 529500 (0.0010) [2023-12-27 09:18:21,294][20381] Updated weights for policy 1, policy_version 529510 (0.0011) [2023-12-27 09:18:21,360][20381] Updated weights for policy 1, policy_version 529520 (0.0011) [2023-12-27 09:18:21,558][20347] Updated weights for policy 0, policy_version 529861 (0.0007) [2023-12-27 09:18:21,615][20347] Updated weights for policy 0, policy_version 529871 (0.0007) [2023-12-27 09:18:21,677][20347] Updated weights for policy 0, policy_version 529881 (0.0008) [2023-12-27 09:18:22,024][20381] Updated weights for policy 1, policy_version 529530 (0.0011) [2023-12-27 09:18:22,083][20381] Updated weights for policy 1, policy_version 529540 (0.0008) [2023-12-27 09:18:22,135][20381] Updated weights for policy 1, policy_version 529550 (0.0008) [2023-12-27 09:18:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 271253504. Throughput: 0: 10232.6, 1: 10345.2. Samples: 271224104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:22,363][19170] Avg episode reward: [(0, '941.870'), (1, '913.040')] [2023-12-27 09:18:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000529552_135585792.pth... [2023-12-27 09:18:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000529888_135667712.pth... [2023-12-27 09:18:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000528336_135274496.pth [2023-12-27 09:18:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000528704_135364608.pth [2023-12-27 09:18:22,419][20347] Updated weights for policy 0, policy_version 529891 (0.0007) [2023-12-27 09:18:22,481][20347] Updated weights for policy 0, policy_version 529901 (0.0006) [2023-12-27 09:18:22,532][20347] Updated weights for policy 0, policy_version 529911 (0.0007) [2023-12-27 09:18:22,839][20381] Updated weights for policy 1, policy_version 529560 (0.0010) [2023-12-27 09:18:22,896][20381] Updated weights for policy 1, policy_version 529570 (0.0011) [2023-12-27 09:18:22,949][20381] Updated weights for policy 1, policy_version 529580 (0.0011) [2023-12-27 09:18:23,198][20347] Updated weights for policy 0, policy_version 529921 (0.0007) [2023-12-27 09:18:23,257][20347] Updated weights for policy 0, policy_version 529931 (0.0005) [2023-12-27 09:18:23,313][20347] Updated weights for policy 0, policy_version 529941 (0.0005) [2023-12-27 09:18:23,371][20347] Updated weights for policy 0, policy_version 529951 (0.0005) [2023-12-27 09:18:23,656][20381] Updated weights for policy 1, policy_version 529590 (0.0008) [2023-12-27 09:18:23,709][20381] Updated weights for policy 1, policy_version 529600 (0.0008) [2023-12-27 09:18:23,768][20381] Updated weights for policy 1, policy_version 529610 (0.0008) [2023-12-27 09:18:23,979][20347] Updated weights for policy 0, policy_version 529961 (0.0007) [2023-12-27 09:18:24,030][20347] Updated weights for policy 0, policy_version 529971 (0.0007) [2023-12-27 09:18:24,077][20347] Updated weights for policy 0, policy_version 529981 (0.0007) [2023-12-27 09:18:24,433][20381] Updated weights for policy 1, policy_version 529620 (0.0007) [2023-12-27 09:18:24,488][20381] Updated weights for policy 1, policy_version 529630 (0.0005) [2023-12-27 09:18:24,540][20381] Updated weights for policy 1, policy_version 529640 (0.0005) [2023-12-27 09:18:24,788][20347] Updated weights for policy 0, policy_version 529991 (0.0007) [2023-12-27 09:18:24,843][20347] Updated weights for policy 0, policy_version 530001 (0.0008) [2023-12-27 09:18:24,892][20347] Updated weights for policy 0, policy_version 530011 (0.0009) [2023-12-27 09:18:25,100][20381] Updated weights for policy 1, policy_version 529650 (0.0007) [2023-12-27 09:18:25,154][20381] Updated weights for policy 1, policy_version 529660 (0.0006) [2023-12-27 09:18:25,210][20381] Updated weights for policy 1, policy_version 529670 (0.0005) [2023-12-27 09:18:25,259][20381] Updated weights for policy 1, policy_version 529680 (0.0006) [2023-12-27 09:18:25,546][20347] Updated weights for policy 0, policy_version 530021 (0.0009) [2023-12-27 09:18:25,600][20347] Updated weights for policy 0, policy_version 530031 (0.0007) [2023-12-27 09:18:25,659][20347] Updated weights for policy 0, policy_version 530041 (0.0008) [2023-12-27 09:18:25,829][20381] Updated weights for policy 1, policy_version 529690 (0.0007) [2023-12-27 09:18:25,884][20381] Updated weights for policy 1, policy_version 529700 (0.0006) [2023-12-27 09:18:25,940][20381] Updated weights for policy 1, policy_version 529710 (0.0005) [2023-12-27 09:18:26,378][20347] Updated weights for policy 0, policy_version 530051 (0.0008) [2023-12-27 09:18:26,430][20347] Updated weights for policy 0, policy_version 530061 (0.0008) [2023-12-27 09:18:26,484][20347] Updated weights for policy 0, policy_version 530071 (0.0007) [2023-12-27 09:18:26,594][20381] Updated weights for policy 1, policy_version 529720 (0.0007) [2023-12-27 09:18:26,639][20381] Updated weights for policy 1, policy_version 529730 (0.0007) [2023-12-27 09:18:26,687][20381] Updated weights for policy 1, policy_version 529740 (0.0007) [2023-12-27 09:18:27,158][20347] Updated weights for policy 0, policy_version 530081 (0.0008) [2023-12-27 09:18:27,202][20347] Updated weights for policy 0, policy_version 530091 (0.0007) [2023-12-27 09:18:27,256][20347] Updated weights for policy 0, policy_version 530101 (0.0007) [2023-12-27 09:18:27,318][20347] Updated weights for policy 0, policy_version 530111 (0.0008) [2023-12-27 09:18:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 271360000. Throughput: 0: 10286.1, 1: 10343.3. Samples: 271348344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:27,363][19170] Avg episode reward: [(0, '903.260'), (1, '941.920')] [2023-12-27 09:18:27,378][20381] Updated weights for policy 1, policy_version 529750 (0.0008) [2023-12-27 09:18:27,423][20381] Updated weights for policy 1, policy_version 529760 (0.0007) [2023-12-27 09:18:27,469][20381] Updated weights for policy 1, policy_version 529770 (0.0007) [2023-12-27 09:18:27,944][20347] Updated weights for policy 0, policy_version 530121 (0.0008) [2023-12-27 09:18:28,001][20347] Updated weights for policy 0, policy_version 530131 (0.0007) [2023-12-27 09:18:28,049][20347] Updated weights for policy 0, policy_version 530141 (0.0008) [2023-12-27 09:18:28,207][20381] Updated weights for policy 1, policy_version 529780 (0.0008) [2023-12-27 09:18:28,256][20381] Updated weights for policy 1, policy_version 529790 (0.0008) [2023-12-27 09:18:28,304][20381] Updated weights for policy 1, policy_version 529800 (0.0008) [2023-12-27 09:18:28,718][20347] Updated weights for policy 0, policy_version 530151 (0.0007) [2023-12-27 09:18:28,769][20347] Updated weights for policy 0, policy_version 530161 (0.0007) [2023-12-27 09:18:28,824][20347] Updated weights for policy 0, policy_version 530171 (0.0008) [2023-12-27 09:18:29,033][20381] Updated weights for policy 1, policy_version 529810 (0.0008) [2023-12-27 09:18:29,081][20381] Updated weights for policy 1, policy_version 529820 (0.0007) [2023-12-27 09:18:29,132][20381] Updated weights for policy 1, policy_version 529830 (0.0007) [2023-12-27 09:18:29,181][20381] Updated weights for policy 1, policy_version 529840 (0.0007) [2023-12-27 09:18:29,508][20347] Updated weights for policy 0, policy_version 530182 (0.0007) [2023-12-27 09:18:29,566][20347] Updated weights for policy 0, policy_version 530192 (0.0005) [2023-12-27 09:18:29,632][20347] Updated weights for policy 0, policy_version 530202 (0.0006) [2023-12-27 09:18:29,918][20381] Updated weights for policy 1, policy_version 529850 (0.0008) [2023-12-27 09:18:29,983][20381] Updated weights for policy 1, policy_version 529860 (0.0007) [2023-12-27 09:18:30,042][20381] Updated weights for policy 1, policy_version 529870 (0.0007) [2023-12-27 09:18:30,275][20347] Updated weights for policy 0, policy_version 530212 (0.0007) [2023-12-27 09:18:30,335][20347] Updated weights for policy 0, policy_version 530222 (0.0011) [2023-12-27 09:18:30,394][20347] Updated weights for policy 0, policy_version 530232 (0.0010) [2023-12-27 09:18:30,701][20381] Updated weights for policy 1, policy_version 529880 (0.0008) [2023-12-27 09:18:30,752][20381] Updated weights for policy 1, policy_version 529890 (0.0007) [2023-12-27 09:18:30,810][20381] Updated weights for policy 1, policy_version 529900 (0.0007) [2023-12-27 09:18:31,051][20347] Updated weights for policy 0, policy_version 530242 (0.0010) [2023-12-27 09:18:31,103][20347] Updated weights for policy 0, policy_version 530252 (0.0006) [2023-12-27 09:18:31,167][20347] Updated weights for policy 0, policy_version 530262 (0.0007) [2023-12-27 09:18:31,227][20347] Updated weights for policy 0, policy_version 530272 (0.0007) [2023-12-27 09:18:31,452][20381] Updated weights for policy 1, policy_version 529910 (0.0007) [2023-12-27 09:18:31,511][20381] Updated weights for policy 1, policy_version 529920 (0.0005) [2023-12-27 09:18:31,563][20381] Updated weights for policy 1, policy_version 529930 (0.0005) [2023-12-27 09:18:31,952][20347] Updated weights for policy 0, policy_version 530282 (0.0007) [2023-12-27 09:18:32,010][20347] Updated weights for policy 0, policy_version 530292 (0.0008) [2023-12-27 09:18:32,070][20347] Updated weights for policy 0, policy_version 530302 (0.0008) [2023-12-27 09:18:32,199][20381] Updated weights for policy 1, policy_version 529940 (0.0007) [2023-12-27 09:18:32,252][20381] Updated weights for policy 1, policy_version 529950 (0.0008) [2023-12-27 09:18:32,301][20381] Updated weights for policy 1, policy_version 529960 (0.0008) [2023-12-27 09:18:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 271466496. Throughput: 0: 10279.7, 1: 10318.8. Samples: 271471972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:32,363][19170] Avg episode reward: [(0, '903.200'), (1, '951.570')] [2023-12-27 09:18:32,654][20291] KL-divergence is very high: 153.7428 [2023-12-27 09:18:32,691][20347] Updated weights for policy 0, policy_version 530312 (0.0006) [2023-12-27 09:18:32,754][20347] Updated weights for policy 0, policy_version 530322 (0.0006) [2023-12-27 09:18:32,806][20347] Updated weights for policy 0, policy_version 530332 (0.0007) [2023-12-27 09:18:33,047][20381] Updated weights for policy 1, policy_version 529970 (0.0008) [2023-12-27 09:18:33,104][20381] Updated weights for policy 1, policy_version 529980 (0.0008) [2023-12-27 09:18:33,162][20381] Updated weights for policy 1, policy_version 529990 (0.0007) [2023-12-27 09:18:33,211][20381] Updated weights for policy 1, policy_version 530000 (0.0009) [2023-12-27 09:18:33,408][20347] Updated weights for policy 0, policy_version 530342 (0.0008) [2023-12-27 09:18:33,462][20347] Updated weights for policy 0, policy_version 530352 (0.0007) [2023-12-27 09:18:33,519][20347] Updated weights for policy 0, policy_version 530362 (0.0006) [2023-12-27 09:18:33,856][20381] Updated weights for policy 1, policy_version 530010 (0.0005) [2023-12-27 09:18:33,907][20381] Updated weights for policy 1, policy_version 530020 (0.0008) [2023-12-27 09:18:33,961][20381] Updated weights for policy 1, policy_version 530030 (0.0006) [2023-12-27 09:18:34,098][20347] Updated weights for policy 0, policy_version 530372 (0.0010) [2023-12-27 09:18:34,162][20347] Updated weights for policy 0, policy_version 530382 (0.0009) [2023-12-27 09:18:34,222][20347] Updated weights for policy 0, policy_version 530392 (0.0008) [2023-12-27 09:18:34,566][20381] Updated weights for policy 1, policy_version 530040 (0.0008) [2023-12-27 09:18:34,628][20381] Updated weights for policy 1, policy_version 530050 (0.0008) [2023-12-27 09:18:34,689][20381] Updated weights for policy 1, policy_version 530060 (0.0010) [2023-12-27 09:18:34,866][20347] Updated weights for policy 0, policy_version 530402 (0.0009) [2023-12-27 09:18:34,929][20347] Updated weights for policy 0, policy_version 530412 (0.0008) [2023-12-27 09:18:34,979][20347] Updated weights for policy 0, policy_version 530422 (0.0007) [2023-12-27 09:18:35,025][20347] Updated weights for policy 0, policy_version 530432 (0.0007) [2023-12-27 09:18:35,349][20381] Updated weights for policy 1, policy_version 530070 (0.0010) [2023-12-27 09:18:35,395][20381] Updated weights for policy 1, policy_version 530080 (0.0010) [2023-12-27 09:18:35,452][20381] Updated weights for policy 1, policy_version 530090 (0.0011) [2023-12-27 09:18:35,766][20347] Updated weights for policy 0, policy_version 530442 (0.0008) [2023-12-27 09:18:35,826][20347] Updated weights for policy 0, policy_version 530452 (0.0008) [2023-12-27 09:18:35,881][20347] Updated weights for policy 0, policy_version 530462 (0.0007) [2023-12-27 09:18:36,119][20381] Updated weights for policy 1, policy_version 530100 (0.0009) [2023-12-27 09:18:36,180][20381] Updated weights for policy 1, policy_version 530110 (0.0006) [2023-12-27 09:18:36,234][20381] Updated weights for policy 1, policy_version 530120 (0.0008) [2023-12-27 09:18:36,521][20347] Updated weights for policy 0, policy_version 530472 (0.0010) [2023-12-27 09:18:36,584][20347] Updated weights for policy 0, policy_version 530482 (0.0011) [2023-12-27 09:18:36,646][20347] Updated weights for policy 0, policy_version 530492 (0.0010) [2023-12-27 09:18:36,923][20381] Updated weights for policy 1, policy_version 530130 (0.0009) [2023-12-27 09:18:36,973][20381] Updated weights for policy 1, policy_version 530140 (0.0007) [2023-12-27 09:18:37,029][20381] Updated weights for policy 1, policy_version 530150 (0.0008) [2023-12-27 09:18:37,079][20381] Updated weights for policy 1, policy_version 530160 (0.0009) [2023-12-27 09:18:37,262][20347] Updated weights for policy 0, policy_version 530502 (0.0010) [2023-12-27 09:18:37,311][20347] Updated weights for policy 0, policy_version 530512 (0.0010) [2023-12-27 09:18:37,359][20347] Updated weights for policy 0, policy_version 530522 (0.0008) [2023-12-27 09:18:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 271564800. Throughput: 0: 10309.0, 1: 10339.1. Samples: 271536256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:37,363][20291] KL-divergence is very high: 142.6703 [2023-12-27 09:18:37,363][19170] Avg episode reward: [(0, '928.320'), (1, '932.110')] [2023-12-27 09:18:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000530160_135741440.pth... [2023-12-27 09:18:37,373][20291] KL-divergence is very high: 260.8595 [2023-12-27 09:18:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000528944_135430144.pth [2023-12-27 09:18:37,386][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000530528_135831552.pth... [2023-12-27 09:18:37,389][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000529312_135520256.pth [2023-12-27 09:18:37,855][20381] Updated weights for policy 1, policy_version 530170 (0.0008) [2023-12-27 09:18:37,914][20381] Updated weights for policy 1, policy_version 530180 (0.0008) [2023-12-27 09:18:37,975][20381] Updated weights for policy 1, policy_version 530190 (0.0006) [2023-12-27 09:18:37,992][20347] Updated weights for policy 0, policy_version 530532 (0.0006) [2023-12-27 09:18:38,043][20291] KL-divergence is very high: 113.2571 [2023-12-27 09:18:38,055][20347] Updated weights for policy 0, policy_version 530542 (0.0005) [2023-12-27 09:18:38,094][20291] KL-divergence is very high: 105.9833 [2023-12-27 09:18:38,117][20347] Updated weights for policy 0, policy_version 530552 (0.0006) [2023-12-27 09:18:38,142][20291] KL-divergence is very high: 117.8608 [2023-12-27 09:18:38,640][20381] Updated weights for policy 1, policy_version 530200 (0.0007) [2023-12-27 09:18:38,694][20347] Updated weights for policy 0, policy_version 530562 (0.0007) [2023-12-27 09:18:38,695][20381] Updated weights for policy 1, policy_version 530210 (0.0007) [2023-12-27 09:18:38,749][20347] Updated weights for policy 0, policy_version 530572 (0.0010) [2023-12-27 09:18:38,752][20381] Updated weights for policy 1, policy_version 530220 (0.0005) [2023-12-27 09:18:38,804][20347] Updated weights for policy 0, policy_version 530582 (0.0010) [2023-12-27 09:18:38,852][20347] Updated weights for policy 0, policy_version 530592 (0.0010) [2023-12-27 09:18:39,471][20381] Updated weights for policy 1, policy_version 530230 (0.0008) [2023-12-27 09:18:39,534][20381] Updated weights for policy 1, policy_version 530240 (0.0007) [2023-12-27 09:18:39,537][20347] Updated weights for policy 0, policy_version 530602 (0.0006) [2023-12-27 09:18:39,593][20381] Updated weights for policy 1, policy_version 530250 (0.0007) [2023-12-27 09:18:39,599][20347] Updated weights for policy 0, policy_version 530612 (0.0007) [2023-12-27 09:18:39,661][20347] Updated weights for policy 0, policy_version 530622 (0.0007) [2023-12-27 09:18:40,304][20381] Updated weights for policy 1, policy_version 530260 (0.0006) [2023-12-27 09:18:40,368][20381] Updated weights for policy 1, policy_version 530270 (0.0006) [2023-12-27 09:18:40,430][20381] Updated weights for policy 1, policy_version 530280 (0.0006) [2023-12-27 09:18:40,432][20347] Updated weights for policy 0, policy_version 530632 (0.0008) [2023-12-27 09:18:40,490][20347] Updated weights for policy 0, policy_version 530642 (0.0007) [2023-12-27 09:18:40,551][20347] Updated weights for policy 0, policy_version 530652 (0.0008) [2023-12-27 09:18:41,081][20381] Updated weights for policy 1, policy_version 530290 (0.0007) [2023-12-27 09:18:41,146][20381] Updated weights for policy 1, policy_version 530300 (0.0007) [2023-12-27 09:18:41,211][20381] Updated weights for policy 1, policy_version 530310 (0.0008) [2023-12-27 09:18:41,263][20381] Updated weights for policy 1, policy_version 530320 (0.0007) [2023-12-27 09:18:41,274][20347] Updated weights for policy 0, policy_version 530662 (0.0007) [2023-12-27 09:18:41,337][20347] Updated weights for policy 0, policy_version 530672 (0.0008) [2023-12-27 09:18:41,401][20347] Updated weights for policy 0, policy_version 530682 (0.0007) [2023-12-27 09:18:42,005][20381] Updated weights for policy 1, policy_version 530330 (0.0008) [2023-12-27 09:18:42,065][20381] Updated weights for policy 1, policy_version 530340 (0.0007) [2023-12-27 09:18:42,098][20347] Updated weights for policy 0, policy_version 530692 (0.0007) [2023-12-27 09:18:42,121][20381] Updated weights for policy 1, policy_version 530350 (0.0006) [2023-12-27 09:18:42,158][20347] Updated weights for policy 0, policy_version 530702 (0.0007) [2023-12-27 09:18:42,220][20347] Updated weights for policy 0, policy_version 530712 (0.0007) [2023-12-27 09:18:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 271671296. Throughput: 0: 10308.5, 1: 10319.0. Samples: 271657740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:18:42,363][19170] Avg episode reward: [(0, '740.870'), (1, '932.140')] [2023-12-27 09:18:42,824][20381] Updated weights for policy 1, policy_version 530360 (0.0007) [2023-12-27 09:18:42,838][20347] Updated weights for policy 0, policy_version 530722 (0.0007) [2023-12-27 09:18:42,873][20381] Updated weights for policy 1, policy_version 530370 (0.0005) [2023-12-27 09:18:42,893][20347] Updated weights for policy 0, policy_version 530732 (0.0007) [2023-12-27 09:18:42,922][20381] Updated weights for policy 1, policy_version 530380 (0.0005) [2023-12-27 09:18:42,950][20347] Updated weights for policy 0, policy_version 530742 (0.0008) [2023-12-27 09:18:42,996][20347] Updated weights for policy 0, policy_version 530752 (0.0008) [2023-12-27 09:18:43,582][20381] Updated weights for policy 1, policy_version 530390 (0.0008) [2023-12-27 09:18:43,632][20381] Updated weights for policy 1, policy_version 530400 (0.0005) [2023-12-27 09:18:43,632][20347] Updated weights for policy 0, policy_version 530762 (0.0008) [2023-12-27 09:18:43,683][20381] Updated weights for policy 1, policy_version 530410 (0.0006) [2023-12-27 09:18:43,688][20347] Updated weights for policy 0, policy_version 530772 (0.0011) [2023-12-27 09:18:43,737][20347] Updated weights for policy 0, policy_version 530782 (0.0008) [2023-12-27 09:18:44,294][20347] Updated weights for policy 0, policy_version 530792 (0.0009) [2023-12-27 09:18:44,352][20347] Updated weights for policy 0, policy_version 530802 (0.0011) [2023-12-27 09:18:44,371][20381] Updated weights for policy 1, policy_version 530420 (0.0011) [2023-12-27 09:18:44,405][20347] Updated weights for policy 0, policy_version 530812 (0.0006) [2023-12-27 09:18:44,417][20381] Updated weights for policy 1, policy_version 530430 (0.0010) [2023-12-27 09:18:44,465][20381] Updated weights for policy 1, policy_version 530440 (0.0010) [2023-12-27 09:18:45,029][20347] Updated weights for policy 0, policy_version 530822 (0.0008) [2023-12-27 09:18:45,087][20347] Updated weights for policy 0, policy_version 530832 (0.0010) [2023-12-27 09:18:45,146][20347] Updated weights for policy 0, policy_version 530842 (0.0011) [2023-12-27 09:18:45,162][20381] Updated weights for policy 1, policy_version 530450 (0.0011) [2023-12-27 09:18:45,220][20381] Updated weights for policy 1, policy_version 530460 (0.0011) [2023-12-27 09:18:45,273][20381] Updated weights for policy 1, policy_version 530470 (0.0011) [2023-12-27 09:18:45,328][20381] Updated weights for policy 1, policy_version 530480 (0.0011) [2023-12-27 09:18:45,829][20347] Updated weights for policy 0, policy_version 530852 (0.0006) [2023-12-27 09:18:45,879][20347] Updated weights for policy 0, policy_version 530862 (0.0006) [2023-12-27 09:18:45,931][20347] Updated weights for policy 0, policy_version 530872 (0.0010) [2023-12-27 09:18:45,976][20381] Updated weights for policy 1, policy_version 530490 (0.0006) [2023-12-27 09:18:46,037][20381] Updated weights for policy 1, policy_version 530500 (0.0006) [2023-12-27 09:18:46,101][20381] Updated weights for policy 1, policy_version 530510 (0.0006) [2023-12-27 09:18:46,512][20347] Updated weights for policy 0, policy_version 530882 (0.0010) [2023-12-27 09:18:46,561][20347] Updated weights for policy 0, policy_version 530892 (0.0007) [2023-12-27 09:18:46,613][20347] Updated weights for policy 0, policy_version 530902 (0.0008) [2023-12-27 09:18:46,665][20347] Updated weights for policy 0, policy_version 530912 (0.0008) [2023-12-27 09:18:46,744][20381] Updated weights for policy 1, policy_version 530520 (0.0007) [2023-12-27 09:18:46,803][20381] Updated weights for policy 1, policy_version 530530 (0.0008) [2023-12-27 09:18:46,856][20381] Updated weights for policy 1, policy_version 530540 (0.0008) [2023-12-27 09:18:47,359][20347] Updated weights for policy 0, policy_version 530922 (0.0006) [2023-12-27 09:18:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 271769600. Throughput: 0: 10322.4, 1: 10347.7. Samples: 271783292. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:18:47,363][19170] Avg episode reward: [(0, '809.220'), (1, '951.550')] [2023-12-27 09:18:47,420][20347] Updated weights for policy 0, policy_version 530932 (0.0006) [2023-12-27 09:18:47,471][20347] Updated weights for policy 0, policy_version 530942 (0.0009) [2023-12-27 09:18:47,568][20381] Updated weights for policy 1, policy_version 530550 (0.0007) [2023-12-27 09:18:47,629][20381] Updated weights for policy 1, policy_version 530560 (0.0007) [2023-12-27 09:18:47,693][20381] Updated weights for policy 1, policy_version 530570 (0.0008) [2023-12-27 09:18:48,102][20347] Updated weights for policy 0, policy_version 530952 (0.0009) [2023-12-27 09:18:48,158][20347] Updated weights for policy 0, policy_version 530962 (0.0010) [2023-12-27 09:18:48,210][20347] Updated weights for policy 0, policy_version 530972 (0.0010) [2023-12-27 09:18:48,382][20381] Updated weights for policy 1, policy_version 530580 (0.0008) [2023-12-27 09:18:48,430][20381] Updated weights for policy 1, policy_version 530590 (0.0007) [2023-12-27 09:18:48,476][20381] Updated weights for policy 1, policy_version 530600 (0.0008) [2023-12-27 09:18:48,835][20347] Updated weights for policy 0, policy_version 530982 (0.0010) [2023-12-27 09:18:48,888][20347] Updated weights for policy 0, policy_version 530992 (0.0010) [2023-12-27 09:18:48,940][20347] Updated weights for policy 0, policy_version 531002 (0.0010) [2023-12-27 09:18:49,229][20381] Updated weights for policy 1, policy_version 530610 (0.0009) [2023-12-27 09:18:49,287][20381] Updated weights for policy 1, policy_version 530620 (0.0008) [2023-12-27 09:18:49,348][20381] Updated weights for policy 1, policy_version 530630 (0.0007) [2023-12-27 09:18:49,402][20381] Updated weights for policy 1, policy_version 530640 (0.0008) [2023-12-27 09:18:49,632][20347] Updated weights for policy 0, policy_version 531012 (0.0009) [2023-12-27 09:18:49,695][20347] Updated weights for policy 0, policy_version 531022 (0.0005) [2023-12-27 09:18:49,756][20347] Updated weights for policy 0, policy_version 531032 (0.0005) [2023-12-27 09:18:50,034][20381] Updated weights for policy 1, policy_version 530650 (0.0006) [2023-12-27 09:18:50,098][20381] Updated weights for policy 1, policy_version 530660 (0.0006) [2023-12-27 09:18:50,157][20381] Updated weights for policy 1, policy_version 530670 (0.0006) [2023-12-27 09:18:50,416][20347] Updated weights for policy 0, policy_version 531042 (0.0008) [2023-12-27 09:18:50,474][20347] Updated weights for policy 0, policy_version 531052 (0.0010) [2023-12-27 09:18:50,528][20347] Updated weights for policy 0, policy_version 531062 (0.0010) [2023-12-27 09:18:50,584][20347] Updated weights for policy 0, policy_version 531072 (0.0010) [2023-12-27 09:18:50,759][20381] Updated weights for policy 1, policy_version 530680 (0.0006) [2023-12-27 09:18:50,807][20381] Updated weights for policy 1, policy_version 530690 (0.0009) [2023-12-27 09:18:50,865][20381] Updated weights for policy 1, policy_version 530700 (0.0010) [2023-12-27 09:18:51,364][20347] Updated weights for policy 0, policy_version 531082 (0.0008) [2023-12-27 09:18:51,426][20347] Updated weights for policy 0, policy_version 531092 (0.0007) [2023-12-27 09:18:51,482][20347] Updated weights for policy 0, policy_version 531102 (0.0008) [2023-12-27 09:18:51,591][20381] Updated weights for policy 1, policy_version 530710 (0.0007) [2023-12-27 09:18:51,652][20381] Updated weights for policy 1, policy_version 530720 (0.0007) [2023-12-27 09:18:51,711][20381] Updated weights for policy 1, policy_version 530730 (0.0008) [2023-12-27 09:18:52,240][20347] Updated weights for policy 0, policy_version 531112 (0.0009) [2023-12-27 09:18:52,284][20381] Updated weights for policy 1, policy_version 530740 (0.0007) [2023-12-27 09:18:52,304][20347] Updated weights for policy 0, policy_version 531122 (0.0009) [2023-12-27 09:18:52,337][20381] Updated weights for policy 1, policy_version 530750 (0.0006) [2023-12-27 09:18:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20660.5). Total num frames: 271867904. Throughput: 0: 10318.8, 1: 10338.4. Samples: 271845380. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:18:52,363][19170] Avg episode reward: [(0, '916.120'), (1, '951.450')] [2023-12-27 09:18:52,369][20347] Updated weights for policy 0, policy_version 531132 (0.0006) [2023-12-27 09:18:52,392][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000531136_135987200.pth... [2023-12-27 09:18:52,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000529888_135667712.pth [2023-12-27 09:18:52,401][20381] Updated weights for policy 1, policy_version 530760 (0.0007) [2023-12-27 09:18:52,455][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000530768_135897088.pth... [2023-12-27 09:18:52,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000529552_135585792.pth [2023-12-27 09:18:52,963][20347] Updated weights for policy 0, policy_version 531142 (0.0007) [2023-12-27 09:18:53,026][20347] Updated weights for policy 0, policy_version 531152 (0.0008) [2023-12-27 09:18:53,086][20381] Updated weights for policy 1, policy_version 530770 (0.0006) [2023-12-27 09:18:53,088][20347] Updated weights for policy 0, policy_version 531162 (0.0007) [2023-12-27 09:18:53,151][20381] Updated weights for policy 1, policy_version 530780 (0.0005) [2023-12-27 09:18:53,216][20381] Updated weights for policy 1, policy_version 530790 (0.0007) [2023-12-27 09:18:53,280][20381] Updated weights for policy 1, policy_version 530800 (0.0007) [2023-12-27 09:18:53,688][20347] Updated weights for policy 0, policy_version 531172 (0.0006) [2023-12-27 09:18:53,746][20347] Updated weights for policy 0, policy_version 531182 (0.0006) [2023-12-27 09:18:53,796][20347] Updated weights for policy 0, policy_version 531192 (0.0006) [2023-12-27 09:18:53,961][20381] Updated weights for policy 1, policy_version 530810 (0.0008) [2023-12-27 09:18:54,019][20381] Updated weights for policy 1, policy_version 530820 (0.0007) [2023-12-27 09:18:54,078][20381] Updated weights for policy 1, policy_version 530830 (0.0008) [2023-12-27 09:18:54,427][20347] Updated weights for policy 0, policy_version 531202 (0.0006) [2023-12-27 09:18:54,479][20347] Updated weights for policy 0, policy_version 531212 (0.0008) [2023-12-27 09:18:54,534][20347] Updated weights for policy 0, policy_version 531222 (0.0006) [2023-12-27 09:18:54,583][20347] Updated weights for policy 0, policy_version 531232 (0.0006) [2023-12-27 09:18:54,672][20381] Updated weights for policy 1, policy_version 530840 (0.0008) [2023-12-27 09:18:54,731][20381] Updated weights for policy 1, policy_version 530850 (0.0007) [2023-12-27 09:18:54,788][20381] Updated weights for policy 1, policy_version 530860 (0.0007) [2023-12-27 09:18:55,274][20347] Updated weights for policy 0, policy_version 531242 (0.0010) [2023-12-27 09:18:55,318][20347] Updated weights for policy 0, policy_version 531252 (0.0010) [2023-12-27 09:18:55,378][20347] Updated weights for policy 0, policy_version 531262 (0.0010) [2023-12-27 09:18:55,473][20381] Updated weights for policy 1, policy_version 530870 (0.0007) [2023-12-27 09:18:55,522][20381] Updated weights for policy 1, policy_version 530880 (0.0008) [2023-12-27 09:18:55,583][20381] Updated weights for policy 1, policy_version 530890 (0.0007) [2023-12-27 09:18:56,095][20347] Updated weights for policy 0, policy_version 531272 (0.0008) [2023-12-27 09:18:56,154][20347] Updated weights for policy 0, policy_version 531282 (0.0007) [2023-12-27 09:18:56,211][20347] Updated weights for policy 0, policy_version 531292 (0.0006) [2023-12-27 09:18:56,232][20381] Updated weights for policy 1, policy_version 530900 (0.0008) [2023-12-27 09:18:56,287][20381] Updated weights for policy 1, policy_version 530910 (0.0007) [2023-12-27 09:18:56,342][20381] Updated weights for policy 1, policy_version 530920 (0.0007) [2023-12-27 09:18:56,836][20347] Updated weights for policy 0, policy_version 531302 (0.0006) [2023-12-27 09:18:56,890][20347] Updated weights for policy 0, policy_version 531312 (0.0007) [2023-12-27 09:18:56,937][20347] Updated weights for policy 0, policy_version 531322 (0.0007) [2023-12-27 09:18:57,045][20381] Updated weights for policy 1, policy_version 530930 (0.0007) [2023-12-27 09:18:57,090][20381] Updated weights for policy 1, policy_version 530940 (0.0008) [2023-12-27 09:18:57,146][20381] Updated weights for policy 1, policy_version 530950 (0.0008) [2023-12-27 09:18:57,207][20381] Updated weights for policy 1, policy_version 530960 (0.0006) [2023-12-27 09:18:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 271982592. Throughput: 0: 10368.5, 1: 10323.2. Samples: 271970156. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:18:57,363][19170] Avg episode reward: [(0, '913.600'), (1, '931.910')] [2023-12-27 09:18:57,552][20347] Updated weights for policy 0, policy_version 531332 (0.0007) [2023-12-27 09:18:57,602][20347] Updated weights for policy 0, policy_version 531342 (0.0005) [2023-12-27 09:18:57,662][20347] Updated weights for policy 0, policy_version 531352 (0.0005) [2023-12-27 09:18:57,966][20381] Updated weights for policy 1, policy_version 530970 (0.0007) [2023-12-27 09:18:58,016][20381] Updated weights for policy 1, policy_version 530980 (0.0007) [2023-12-27 09:18:58,068][20381] Updated weights for policy 1, policy_version 530990 (0.0007) [2023-12-27 09:18:58,227][20347] Updated weights for policy 0, policy_version 531362 (0.0006) [2023-12-27 09:18:58,286][20347] Updated weights for policy 0, policy_version 531372 (0.0006) [2023-12-27 09:18:58,345][20347] Updated weights for policy 0, policy_version 531382 (0.0008) [2023-12-27 09:18:58,409][20347] Updated weights for policy 0, policy_version 531392 (0.0009) [2023-12-27 09:18:58,850][20381] Updated weights for policy 1, policy_version 531000 (0.0008) [2023-12-27 09:18:58,905][20381] Updated weights for policy 1, policy_version 531010 (0.0007) [2023-12-27 09:18:58,965][20381] Updated weights for policy 1, policy_version 531020 (0.0008) [2023-12-27 09:18:59,153][20347] Updated weights for policy 0, policy_version 531402 (0.0007) [2023-12-27 09:18:59,215][20347] Updated weights for policy 0, policy_version 531412 (0.0007) [2023-12-27 09:18:59,277][20347] Updated weights for policy 0, policy_version 531422 (0.0007) [2023-12-27 09:18:59,640][20381] Updated weights for policy 1, policy_version 531030 (0.0007) [2023-12-27 09:18:59,699][20381] Updated weights for policy 1, policy_version 531040 (0.0007) [2023-12-27 09:18:59,755][20381] Updated weights for policy 1, policy_version 531050 (0.0006) [2023-12-27 09:18:59,972][20347] Updated weights for policy 0, policy_version 531432 (0.0008) [2023-12-27 09:19:00,037][20347] Updated weights for policy 0, policy_version 531442 (0.0008) [2023-12-27 09:19:00,093][20347] Updated weights for policy 0, policy_version 531452 (0.0007) [2023-12-27 09:19:00,371][20381] Updated weights for policy 1, policy_version 531060 (0.0007) [2023-12-27 09:19:00,418][20381] Updated weights for policy 1, policy_version 531070 (0.0007) [2023-12-27 09:19:00,470][20381] Updated weights for policy 1, policy_version 531080 (0.0009) [2023-12-27 09:19:00,710][20347] Updated weights for policy 0, policy_version 531462 (0.0007) [2023-12-27 09:19:00,757][20347] Updated weights for policy 0, policy_version 531472 (0.0007) [2023-12-27 09:19:00,812][20347] Updated weights for policy 0, policy_version 531482 (0.0007) [2023-12-27 09:19:01,181][20381] Updated weights for policy 1, policy_version 531090 (0.0008) [2023-12-27 09:19:01,237][20381] Updated weights for policy 1, policy_version 531100 (0.0008) [2023-12-27 09:19:01,286][20381] Updated weights for policy 1, policy_version 531110 (0.0007) [2023-12-27 09:19:01,337][20381] Updated weights for policy 1, policy_version 531120 (0.0007) [2023-12-27 09:19:01,535][20347] Updated weights for policy 0, policy_version 531492 (0.0008) [2023-12-27 09:19:01,584][20347] Updated weights for policy 0, policy_version 531502 (0.0008) [2023-12-27 09:19:01,647][20347] Updated weights for policy 0, policy_version 531512 (0.0008) [2023-12-27 09:19:01,982][20381] Updated weights for policy 1, policy_version 531130 (0.0008) [2023-12-27 09:19:02,034][20381] Updated weights for policy 1, policy_version 531140 (0.0007) [2023-12-27 09:19:02,091][20381] Updated weights for policy 1, policy_version 531150 (0.0007) [2023-12-27 09:19:02,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 272080896. Throughput: 0: 10396.2, 1: 10279.0. Samples: 272091836. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:02,363][19170] Avg episode reward: [(0, '951.700'), (1, '951.400')] [2023-12-27 09:19:02,389][20347] Updated weights for policy 0, policy_version 531522 (0.0008) [2023-12-27 09:19:02,441][20347] Updated weights for policy 0, policy_version 531532 (0.0009) [2023-12-27 09:19:02,492][20347] Updated weights for policy 0, policy_version 531542 (0.0007) [2023-12-27 09:19:02,547][20347] Updated weights for policy 0, policy_version 531552 (0.0007) [2023-12-27 09:19:02,723][20381] Updated weights for policy 1, policy_version 531160 (0.0007) [2023-12-27 09:19:02,779][20381] Updated weights for policy 1, policy_version 531170 (0.0008) [2023-12-27 09:19:02,832][20381] Updated weights for policy 1, policy_version 531180 (0.0008) [2023-12-27 09:19:03,246][20347] Updated weights for policy 0, policy_version 531562 (0.0009) [2023-12-27 09:19:03,299][20347] Updated weights for policy 0, policy_version 531572 (0.0005) [2023-12-27 09:19:03,359][20347] Updated weights for policy 0, policy_version 531582 (0.0005) [2023-12-27 09:19:03,431][20381] Updated weights for policy 1, policy_version 531190 (0.0007) [2023-12-27 09:19:03,482][20381] Updated weights for policy 1, policy_version 531200 (0.0010) [2023-12-27 09:19:03,535][20381] Updated weights for policy 1, policy_version 531210 (0.0010) [2023-12-27 09:19:04,016][20347] Updated weights for policy 0, policy_version 531592 (0.0007) [2023-12-27 09:19:04,076][20347] Updated weights for policy 0, policy_version 531602 (0.0008) [2023-12-27 09:19:04,136][20347] Updated weights for policy 0, policy_version 531612 (0.0007) [2023-12-27 09:19:04,157][20381] Updated weights for policy 1, policy_version 531220 (0.0007) [2023-12-27 09:19:04,208][20381] Updated weights for policy 1, policy_version 531230 (0.0005) [2023-12-27 09:19:04,267][20381] Updated weights for policy 1, policy_version 531240 (0.0005) [2023-12-27 09:19:04,806][20347] Updated weights for policy 0, policy_version 531622 (0.0006) [2023-12-27 09:19:04,867][20347] Updated weights for policy 0, policy_version 531632 (0.0007) [2023-12-27 09:19:04,894][20381] Updated weights for policy 1, policy_version 531250 (0.0006) [2023-12-27 09:19:04,921][20347] Updated weights for policy 0, policy_version 531642 (0.0007) [2023-12-27 09:19:04,942][20381] Updated weights for policy 1, policy_version 531260 (0.0005) [2023-12-27 09:19:04,997][20381] Updated weights for policy 1, policy_version 531270 (0.0007) [2023-12-27 09:19:05,044][20381] Updated weights for policy 1, policy_version 531280 (0.0006) [2023-12-27 09:19:05,487][20347] Updated weights for policy 0, policy_version 531652 (0.0007) [2023-12-27 09:19:05,541][20347] Updated weights for policy 0, policy_version 531662 (0.0005) [2023-12-27 09:19:05,599][20347] Updated weights for policy 0, policy_version 531672 (0.0005) [2023-12-27 09:19:05,682][20381] Updated weights for policy 1, policy_version 531290 (0.0007) [2023-12-27 09:19:05,738][20381] Updated weights for policy 1, policy_version 531300 (0.0005) [2023-12-27 09:19:05,790][20381] Updated weights for policy 1, policy_version 531310 (0.0005) [2023-12-27 09:19:06,159][20347] Updated weights for policy 0, policy_version 531682 (0.0006) [2023-12-27 09:19:06,226][20347] Updated weights for policy 0, policy_version 531692 (0.0007) [2023-12-27 09:19:06,289][20347] Updated weights for policy 0, policy_version 531702 (0.0007) [2023-12-27 09:19:06,348][20347] Updated weights for policy 0, policy_version 531712 (0.0007) [2023-12-27 09:19:06,432][20381] Updated weights for policy 1, policy_version 531320 (0.0005) [2023-12-27 09:19:06,485][20381] Updated weights for policy 1, policy_version 531330 (0.0007) [2023-12-27 09:19:06,540][20381] Updated weights for policy 1, policy_version 531340 (0.0008) [2023-12-27 09:19:07,053][20347] Updated weights for policy 0, policy_version 531722 (0.0008) [2023-12-27 09:19:07,117][20347] Updated weights for policy 0, policy_version 531732 (0.0007) [2023-12-27 09:19:07,164][20381] Updated weights for policy 1, policy_version 531350 (0.0008) [2023-12-27 09:19:07,178][20347] Updated weights for policy 0, policy_version 531742 (0.0008) [2023-12-27 09:19:07,224][20381] Updated weights for policy 1, policy_version 531360 (0.0006) [2023-12-27 09:19:07,287][20381] Updated weights for policy 1, policy_version 531370 (0.0006) [2023-12-27 09:19:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 272195584. Throughput: 0: 10415.1, 1: 10290.6. Samples: 272155860. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:07,363][19170] Avg episode reward: [(0, '980.610'), (1, '942.030')] [2023-12-27 09:19:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000531744_136142848.pth... [2023-12-27 09:19:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000531376_136052736.pth... [2023-12-27 09:19:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000530528_135831552.pth [2023-12-27 09:19:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000530160_135741440.pth [2023-12-27 09:19:07,869][20347] Updated weights for policy 0, policy_version 531752 (0.0007) [2023-12-27 09:19:07,913][20347] Updated weights for policy 0, policy_version 531762 (0.0007) [2023-12-27 09:19:07,934][20381] Updated weights for policy 1, policy_version 531380 (0.0006) [2023-12-27 09:19:07,968][20347] Updated weights for policy 0, policy_version 531772 (0.0006) [2023-12-27 09:19:07,988][20381] Updated weights for policy 1, policy_version 531390 (0.0005) [2023-12-27 09:19:08,044][20381] Updated weights for policy 1, policy_version 531400 (0.0008) [2023-12-27 09:19:08,590][20347] Updated weights for policy 0, policy_version 531782 (0.0005) [2023-12-27 09:19:08,640][20347] Updated weights for policy 0, policy_version 531792 (0.0007) [2023-12-27 09:19:08,695][20347] Updated weights for policy 0, policy_version 531802 (0.0007) [2023-12-27 09:19:08,775][20381] Updated weights for policy 1, policy_version 531410 (0.0009) [2023-12-27 09:19:08,831][20381] Updated weights for policy 1, policy_version 531420 (0.0008) [2023-12-27 09:19:08,880][20381] Updated weights for policy 1, policy_version 531430 (0.0008) [2023-12-27 09:19:08,936][20381] Updated weights for policy 1, policy_version 531440 (0.0007) [2023-12-27 09:19:09,388][20347] Updated weights for policy 0, policy_version 531812 (0.0007) [2023-12-27 09:19:09,453][20347] Updated weights for policy 0, policy_version 531822 (0.0008) [2023-12-27 09:19:09,499][20347] Updated weights for policy 0, policy_version 531832 (0.0008) [2023-12-27 09:19:09,682][20381] Updated weights for policy 1, policy_version 531450 (0.0007) [2023-12-27 09:19:09,743][20381] Updated weights for policy 1, policy_version 531460 (0.0008) [2023-12-27 09:19:09,800][20381] Updated weights for policy 1, policy_version 531470 (0.0008) [2023-12-27 09:19:10,207][20347] Updated weights for policy 0, policy_version 531842 (0.0008) [2023-12-27 09:19:10,270][20347] Updated weights for policy 0, policy_version 531852 (0.0008) [2023-12-27 09:19:10,330][20347] Updated weights for policy 0, policy_version 531862 (0.0008) [2023-12-27 09:19:10,388][20347] Updated weights for policy 0, policy_version 531872 (0.0007) [2023-12-27 09:19:10,520][20381] Updated weights for policy 1, policy_version 531480 (0.0010) [2023-12-27 09:19:10,576][20381] Updated weights for policy 1, policy_version 531490 (0.0010) [2023-12-27 09:19:10,624][20381] Updated weights for policy 1, policy_version 531500 (0.0010) [2023-12-27 09:19:11,092][20347] Updated weights for policy 0, policy_version 531882 (0.0007) [2023-12-27 09:19:11,162][20347] Updated weights for policy 0, policy_version 531892 (0.0007) [2023-12-27 09:19:11,229][20347] Updated weights for policy 0, policy_version 531902 (0.0007) [2023-12-27 09:19:11,421][20381] Updated weights for policy 1, policy_version 531510 (0.0009) [2023-12-27 09:19:11,484][20381] Updated weights for policy 1, policy_version 531520 (0.0006) [2023-12-27 09:19:11,544][20381] Updated weights for policy 1, policy_version 531530 (0.0006) [2023-12-27 09:19:11,967][20347] Updated weights for policy 0, policy_version 531912 (0.0007) [2023-12-27 09:19:12,032][20347] Updated weights for policy 0, policy_version 531922 (0.0008) [2023-12-27 09:19:12,093][20347] Updated weights for policy 0, policy_version 531932 (0.0008) [2023-12-27 09:19:12,235][20381] Updated weights for policy 1, policy_version 531540 (0.0008) [2023-12-27 09:19:12,303][20381] Updated weights for policy 1, policy_version 531550 (0.0008) [2023-12-27 09:19:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 272285696. Throughput: 0: 10451.2, 1: 10239.1. Samples: 272279408. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:12,363][19170] Avg episode reward: [(0, '971.070'), (1, '913.080')] [2023-12-27 09:19:12,375][20381] Updated weights for policy 1, policy_version 531560 (0.0009) [2023-12-27 09:19:12,797][20347] Updated weights for policy 0, policy_version 531942 (0.0008) [2023-12-27 09:19:12,850][20347] Updated weights for policy 0, policy_version 531952 (0.0007) [2023-12-27 09:19:12,896][20347] Updated weights for policy 0, policy_version 531962 (0.0009) [2023-12-27 09:19:13,058][20381] Updated weights for policy 1, policy_version 531570 (0.0008) [2023-12-27 09:19:13,114][20381] Updated weights for policy 1, policy_version 531580 (0.0008) [2023-12-27 09:19:13,182][20381] Updated weights for policy 1, policy_version 531590 (0.0008) [2023-12-27 09:19:13,242][20381] Updated weights for policy 1, policy_version 531600 (0.0008) [2023-12-27 09:19:13,583][20347] Updated weights for policy 0, policy_version 531972 (0.0009) [2023-12-27 09:19:13,641][20347] Updated weights for policy 0, policy_version 531982 (0.0008) [2023-12-27 09:19:13,695][20347] Updated weights for policy 0, policy_version 531992 (0.0007) [2023-12-27 09:19:13,903][20381] Updated weights for policy 1, policy_version 531610 (0.0008) [2023-12-27 09:19:13,963][20381] Updated weights for policy 1, policy_version 531620 (0.0008) [2023-12-27 09:19:14,015][20381] Updated weights for policy 1, policy_version 531630 (0.0007) [2023-12-27 09:19:14,311][20347] Updated weights for policy 0, policy_version 532002 (0.0006) [2023-12-27 09:19:14,364][20347] Updated weights for policy 0, policy_version 532012 (0.0006) [2023-12-27 09:19:14,420][20347] Updated weights for policy 0, policy_version 532022 (0.0008) [2023-12-27 09:19:14,475][20347] Updated weights for policy 0, policy_version 532032 (0.0007) [2023-12-27 09:19:14,753][20381] Updated weights for policy 1, policy_version 531640 (0.0008) [2023-12-27 09:19:14,813][20381] Updated weights for policy 1, policy_version 531650 (0.0007) [2023-12-27 09:19:14,865][20381] Updated weights for policy 1, policy_version 531660 (0.0008) [2023-12-27 09:19:15,087][20347] Updated weights for policy 0, policy_version 532042 (0.0006) [2023-12-27 09:19:15,143][20347] Updated weights for policy 0, policy_version 532052 (0.0007) [2023-12-27 09:19:15,201][20347] Updated weights for policy 0, policy_version 532062 (0.0008) [2023-12-27 09:19:15,598][20381] Updated weights for policy 1, policy_version 531670 (0.0008) [2023-12-27 09:19:15,653][20381] Updated weights for policy 1, policy_version 531680 (0.0008) [2023-12-27 09:19:15,714][20381] Updated weights for policy 1, policy_version 531690 (0.0008) [2023-12-27 09:19:15,845][20347] Updated weights for policy 0, policy_version 532072 (0.0009) [2023-12-27 09:19:15,901][20347] Updated weights for policy 0, policy_version 532082 (0.0008) [2023-12-27 09:19:15,948][20347] Updated weights for policy 0, policy_version 532092 (0.0008) [2023-12-27 09:19:16,401][20381] Updated weights for policy 1, policy_version 531700 (0.0008) [2023-12-27 09:19:16,455][20381] Updated weights for policy 1, policy_version 531710 (0.0009) [2023-12-27 09:19:16,501][20381] Updated weights for policy 1, policy_version 531720 (0.0008) [2023-12-27 09:19:16,611][20347] Updated weights for policy 0, policy_version 532102 (0.0007) [2023-12-27 09:19:16,670][20347] Updated weights for policy 0, policy_version 532112 (0.0007) [2023-12-27 09:19:16,721][20347] Updated weights for policy 0, policy_version 532122 (0.0008) [2023-12-27 09:19:17,252][20381] Updated weights for policy 1, policy_version 531730 (0.0008) [2023-12-27 09:19:17,304][20381] Updated weights for policy 1, policy_version 531740 (0.0005) [2023-12-27 09:19:17,356][20381] Updated weights for policy 1, policy_version 531750 (0.0005) [2023-12-27 09:19:17,362][20347] Updated weights for policy 0, policy_version 532132 (0.0008) [2023-12-27 09:19:17,362][19170] Fps is (10 sec: 18841.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 272384000. Throughput: 0: 10451.1, 1: 10191.7. Samples: 272400900. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:17,363][19170] Avg episode reward: [(0, '980.770'), (1, '942.100')] [2023-12-27 09:19:17,407][20381] Updated weights for policy 1, policy_version 531760 (0.0007) [2023-12-27 09:19:17,409][20347] Updated weights for policy 0, policy_version 532142 (0.0007) [2023-12-27 09:19:17,455][20347] Updated weights for policy 0, policy_version 532152 (0.0005) [2023-12-27 09:19:18,039][20347] Updated weights for policy 0, policy_version 532162 (0.0005) [2023-12-27 09:19:18,094][20347] Updated weights for policy 0, policy_version 532172 (0.0007) [2023-12-27 09:19:18,150][20347] Updated weights for policy 0, policy_version 532182 (0.0006) [2023-12-27 09:19:18,154][20381] Updated weights for policy 1, policy_version 531770 (0.0006) [2023-12-27 09:19:18,205][20347] Updated weights for policy 0, policy_version 532192 (0.0006) [2023-12-27 09:19:18,207][20381] Updated weights for policy 1, policy_version 531780 (0.0006) [2023-12-27 09:19:18,262][20381] Updated weights for policy 1, policy_version 531790 (0.0008) [2023-12-27 09:19:18,895][20381] Updated weights for policy 1, policy_version 531800 (0.0007) [2023-12-27 09:19:18,912][20347] Updated weights for policy 0, policy_version 532202 (0.0006) [2023-12-27 09:19:18,948][20381] Updated weights for policy 1, policy_version 531810 (0.0005) [2023-12-27 09:19:18,964][20347] Updated weights for policy 0, policy_version 532212 (0.0007) [2023-12-27 09:19:19,004][20381] Updated weights for policy 1, policy_version 531820 (0.0005) [2023-12-27 09:19:19,019][20347] Updated weights for policy 0, policy_version 532222 (0.0008) [2023-12-27 09:19:19,613][20381] Updated weights for policy 1, policy_version 531830 (0.0006) [2023-12-27 09:19:19,673][20381] Updated weights for policy 1, policy_version 531840 (0.0007) [2023-12-27 09:19:19,733][20381] Updated weights for policy 1, policy_version 531850 (0.0008) [2023-12-27 09:19:19,747][20347] Updated weights for policy 0, policy_version 532232 (0.0010) [2023-12-27 09:19:19,813][20347] Updated weights for policy 0, policy_version 532242 (0.0010) [2023-12-27 09:19:19,875][20347] Updated weights for policy 0, policy_version 532252 (0.0008) [2023-12-27 09:19:20,432][20381] Updated weights for policy 1, policy_version 531860 (0.0008) [2023-12-27 09:19:20,494][20381] Updated weights for policy 1, policy_version 531870 (0.0007) [2023-12-27 09:19:20,550][20381] Updated weights for policy 1, policy_version 531880 (0.0007) [2023-12-27 09:19:20,590][20347] Updated weights for policy 0, policy_version 532262 (0.0010) [2023-12-27 09:19:20,655][20347] Updated weights for policy 0, policy_version 532272 (0.0007) [2023-12-27 09:19:20,710][20347] Updated weights for policy 0, policy_version 532282 (0.0008) [2023-12-27 09:19:21,269][20381] Updated weights for policy 1, policy_version 531890 (0.0008) [2023-12-27 09:19:21,318][20381] Updated weights for policy 1, policy_version 531900 (0.0008) [2023-12-27 09:19:21,383][20381] Updated weights for policy 1, policy_version 531910 (0.0008) [2023-12-27 09:19:21,444][20381] Updated weights for policy 1, policy_version 531920 (0.0007) [2023-12-27 09:19:21,452][20347] Updated weights for policy 0, policy_version 532292 (0.0008) [2023-12-27 09:19:21,512][20347] Updated weights for policy 0, policy_version 532302 (0.0008) [2023-12-27 09:19:21,566][20347] Updated weights for policy 0, policy_version 532312 (0.0008) [2023-12-27 09:19:22,151][20381] Updated weights for policy 1, policy_version 531930 (0.0007) [2023-12-27 09:19:22,209][20381] Updated weights for policy 1, policy_version 531940 (0.0006) [2023-12-27 09:19:22,265][20381] Updated weights for policy 1, policy_version 531950 (0.0006) [2023-12-27 09:19:22,274][20347] Updated weights for policy 0, policy_version 532322 (0.0007) [2023-12-27 09:19:22,332][20347] Updated weights for policy 0, policy_version 532332 (0.0008) [2023-12-27 09:19:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 272490496. Throughput: 0: 10398.0, 1: 10186.0. Samples: 272462536. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:22,363][19170] Avg episode reward: [(0, '971.390'), (1, '971.140')] [2023-12-27 09:19:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000531952_136200192.pth... [2023-12-27 09:19:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000530768_135897088.pth [2023-12-27 09:19:22,400][20347] Updated weights for policy 0, policy_version 532342 (0.0008) [2023-12-27 09:19:22,462][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000532352_136298496.pth... [2023-12-27 09:19:22,464][20347] Updated weights for policy 0, policy_version 532352 (0.0008) [2023-12-27 09:19:22,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000531136_135987200.pth [2023-12-27 09:19:22,919][20381] Updated weights for policy 1, policy_version 531960 (0.0007) [2023-12-27 09:19:22,978][20381] Updated weights for policy 1, policy_version 531970 (0.0008) [2023-12-27 09:19:23,034][20381] Updated weights for policy 1, policy_version 531980 (0.0007) [2023-12-27 09:19:23,179][20347] Updated weights for policy 0, policy_version 532362 (0.0006) [2023-12-27 09:19:23,231][20347] Updated weights for policy 0, policy_version 532372 (0.0005) [2023-12-27 09:19:23,287][20347] Updated weights for policy 0, policy_version 532382 (0.0005) [2023-12-27 09:19:23,760][20381] Updated weights for policy 1, policy_version 531990 (0.0008) [2023-12-27 09:19:23,818][20381] Updated weights for policy 1, policy_version 532000 (0.0007) [2023-12-27 09:19:23,866][20381] Updated weights for policy 1, policy_version 532010 (0.0007) [2023-12-27 09:19:23,944][20347] Updated weights for policy 0, policy_version 532392 (0.0010) [2023-12-27 09:19:24,001][20347] Updated weights for policy 0, policy_version 532402 (0.0010) [2023-12-27 09:19:24,059][20347] Updated weights for policy 0, policy_version 532412 (0.0010) [2023-12-27 09:19:24,558][20381] Updated weights for policy 1, policy_version 532020 (0.0008) [2023-12-27 09:19:24,617][20381] Updated weights for policy 1, policy_version 532030 (0.0008) [2023-12-27 09:19:24,675][20381] Updated weights for policy 1, policy_version 532040 (0.0007) [2023-12-27 09:19:24,710][20347] Updated weights for policy 0, policy_version 532422 (0.0008) [2023-12-27 09:19:24,765][20347] Updated weights for policy 0, policy_version 532432 (0.0007) [2023-12-27 09:19:24,772][20291] KL-divergence is very high: 611.0034 [2023-12-27 09:19:24,782][20291] KL-divergence is very high: 928.3581 [2023-12-27 09:19:24,787][20291] KL-divergence is very high: 120.6797 [2023-12-27 09:19:24,810][20291] KL-divergence is very high: 247.6584 [2023-12-27 09:19:24,815][20347] Updated weights for policy 0, policy_version 532442 (0.0008) [2023-12-27 09:19:24,821][20291] KL-divergence is very high: 253.2041 [2023-12-27 09:19:24,827][20291] KL-divergence is very high: 309.0362 [2023-12-27 09:19:25,352][20381] Updated weights for policy 1, policy_version 532050 (0.0006) [2023-12-27 09:19:25,411][20381] Updated weights for policy 1, policy_version 532060 (0.0008) [2023-12-27 09:19:25,457][20291] KL-divergence is very high: 172.9618 [2023-12-27 09:19:25,460][20381] Updated weights for policy 1, policy_version 532070 (0.0007) [2023-12-27 09:19:25,469][20347] Updated weights for policy 0, policy_version 532452 (0.0007) [2023-12-27 09:19:25,501][20291] KL-divergence is very high: 117.2219 [2023-12-27 09:19:25,512][20381] Updated weights for policy 1, policy_version 532080 (0.0006) [2023-12-27 09:19:25,523][20347] Updated weights for policy 0, policy_version 532462 (0.0007) [2023-12-27 09:19:25,547][20291] KL-divergence is very high: 223.6604 [2023-12-27 09:19:25,579][20347] Updated weights for policy 0, policy_version 532472 (0.0005) [2023-12-27 09:19:26,192][20381] Updated weights for policy 1, policy_version 532090 (0.0005) [2023-12-27 09:19:26,241][20381] Updated weights for policy 1, policy_version 532100 (0.0005) [2023-12-27 09:19:26,255][20291] KL-divergence is very high: 160.7418 [2023-12-27 09:19:26,257][20347] Updated weights for policy 0, policy_version 532482 (0.0006) [2023-12-27 09:19:26,294][20381] Updated weights for policy 1, policy_version 532110 (0.0008) [2023-12-27 09:19:26,300][20291] KL-divergence is very high: 127.4736 [2023-12-27 09:19:26,312][20347] Updated weights for policy 0, policy_version 532492 (0.0010) [2023-12-27 09:19:26,366][20347] Updated weights for policy 0, policy_version 532502 (0.0007) [2023-12-27 09:19:26,422][20347] Updated weights for policy 0, policy_version 532512 (0.0008) [2023-12-27 09:19:26,965][20381] Updated weights for policy 1, policy_version 532120 (0.0010) [2023-12-27 09:19:27,020][20381] Updated weights for policy 1, policy_version 532130 (0.0010) [2023-12-27 09:19:27,073][20347] Updated weights for policy 0, policy_version 532522 (0.0009) [2023-12-27 09:19:27,074][20381] Updated weights for policy 1, policy_version 532140 (0.0007) [2023-12-27 09:19:27,119][20347] Updated weights for policy 0, policy_version 532532 (0.0009) [2023-12-27 09:19:27,175][20347] Updated weights for policy 0, policy_version 532542 (0.0005) [2023-12-27 09:19:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 272596992. Throughput: 0: 10365.8, 1: 10192.5. Samples: 272582864. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:27,364][19170] Avg episode reward: [(0, '865.460'), (1, '913.320')] [2023-12-27 09:19:27,695][20381] Updated weights for policy 1, policy_version 532150 (0.0005) [2023-12-27 09:19:27,752][20381] Updated weights for policy 1, policy_version 532160 (0.0005) [2023-12-27 09:19:27,798][20381] Updated weights for policy 1, policy_version 532170 (0.0005) [2023-12-27 09:19:27,825][20347] Updated weights for policy 0, policy_version 532552 (0.0009) [2023-12-27 09:19:27,876][20347] Updated weights for policy 0, policy_version 532562 (0.0010) [2023-12-27 09:19:27,924][20347] Updated weights for policy 0, policy_version 532572 (0.0010) [2023-12-27 09:19:28,348][20381] Updated weights for policy 1, policy_version 532180 (0.0006) [2023-12-27 09:19:28,406][20381] Updated weights for policy 1, policy_version 532190 (0.0007) [2023-12-27 09:19:28,456][20381] Updated weights for policy 1, policy_version 532200 (0.0008) [2023-12-27 09:19:28,610][20347] Updated weights for policy 0, policy_version 532582 (0.0007) [2023-12-27 09:19:28,663][20347] Updated weights for policy 0, policy_version 532592 (0.0005) [2023-12-27 09:19:28,715][20347] Updated weights for policy 0, policy_version 532602 (0.0005) [2023-12-27 09:19:29,179][20381] Updated weights for policy 1, policy_version 532210 (0.0007) [2023-12-27 09:19:29,234][20381] Updated weights for policy 1, policy_version 532220 (0.0008) [2023-12-27 09:19:29,237][20347] Updated weights for policy 0, policy_version 532612 (0.0007) [2023-12-27 09:19:29,293][20381] Updated weights for policy 1, policy_version 532230 (0.0010) [2023-12-27 09:19:29,293][20347] Updated weights for policy 0, policy_version 532622 (0.0007) [2023-12-27 09:19:29,354][20347] Updated weights for policy 0, policy_version 532632 (0.0007) [2023-12-27 09:19:29,354][20381] Updated weights for policy 1, policy_version 532240 (0.0009) [2023-12-27 09:19:29,975][20347] Updated weights for policy 0, policy_version 532642 (0.0007) [2023-12-27 09:19:30,029][20347] Updated weights for policy 0, policy_version 532652 (0.0005) [2023-12-27 09:19:30,063][20381] Updated weights for policy 1, policy_version 532250 (0.0008) [2023-12-27 09:19:30,080][20347] Updated weights for policy 0, policy_version 532662 (0.0005) [2023-12-27 09:19:30,124][20381] Updated weights for policy 1, policy_version 532260 (0.0007) [2023-12-27 09:19:30,125][20291] KL-divergence is very high: 117.9072 [2023-12-27 09:19:30,130][20347] Updated weights for policy 0, policy_version 532672 (0.0006) [2023-12-27 09:19:30,179][20381] Updated weights for policy 1, policy_version 532270 (0.0011) [2023-12-27 09:19:30,765][20347] Updated weights for policy 0, policy_version 532682 (0.0007) [2023-12-27 09:19:30,814][20347] Updated weights for policy 0, policy_version 532692 (0.0007) [2023-12-27 09:19:30,846][20381] Updated weights for policy 1, policy_version 532280 (0.0010) [2023-12-27 09:19:30,862][20347] Updated weights for policy 0, policy_version 532702 (0.0006) [2023-12-27 09:19:30,898][20381] Updated weights for policy 1, policy_version 532290 (0.0011) [2023-12-27 09:19:30,949][20381] Updated weights for policy 1, policy_version 532300 (0.0010) [2023-12-27 09:19:31,584][20347] Updated weights for policy 0, policy_version 532712 (0.0007) [2023-12-27 09:19:31,645][20347] Updated weights for policy 0, policy_version 532722 (0.0009) [2023-12-27 09:19:31,654][20381] Updated weights for policy 1, policy_version 532310 (0.0009) [2023-12-27 09:19:31,709][20381] Updated weights for policy 1, policy_version 532320 (0.0007) [2023-12-27 09:19:31,710][20347] Updated weights for policy 0, policy_version 532732 (0.0007) [2023-12-27 09:19:31,778][20381] Updated weights for policy 1, policy_version 532330 (0.0008) [2023-12-27 09:19:32,357][20381] Updated weights for policy 1, policy_version 532340 (0.0006) [2023-12-27 09:19:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 272695296. Throughput: 0: 10364.7, 1: 10240.9. Samples: 272710544. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:32,363][19170] Avg episode reward: [(0, '850.620'), (1, '913.040')] [2023-12-27 09:19:32,386][20347] Updated weights for policy 0, policy_version 532742 (0.0010) [2023-12-27 09:19:32,413][20381] Updated weights for policy 1, policy_version 532350 (0.0007) [2023-12-27 09:19:32,438][20347] Updated weights for policy 0, policy_version 532752 (0.0010) [2023-12-27 09:19:32,465][20381] Updated weights for policy 1, policy_version 532360 (0.0005) [2023-12-27 09:19:32,491][20347] Updated weights for policy 0, policy_version 532762 (0.0010) [2023-12-27 09:19:33,135][20381] Updated weights for policy 1, policy_version 532370 (0.0007) [2023-12-27 09:19:33,153][20347] Updated weights for policy 0, policy_version 532772 (0.0010) [2023-12-27 09:19:33,181][20381] Updated weights for policy 1, policy_version 532380 (0.0005) [2023-12-27 09:19:33,207][20347] Updated weights for policy 0, policy_version 532782 (0.0010) [2023-12-27 09:19:33,227][20381] Updated weights for policy 1, policy_version 532390 (0.0005) [2023-12-27 09:19:33,265][20347] Updated weights for policy 0, policy_version 532792 (0.0010) [2023-12-27 09:19:33,270][20381] Updated weights for policy 1, policy_version 532400 (0.0005) [2023-12-27 09:19:33,919][20381] Updated weights for policy 1, policy_version 532410 (0.0007) [2023-12-27 09:19:33,941][20347] Updated weights for policy 0, policy_version 532802 (0.0010) [2023-12-27 09:19:33,964][20381] Updated weights for policy 1, policy_version 532420 (0.0006) [2023-12-27 09:19:33,991][20347] Updated weights for policy 0, policy_version 532812 (0.0010) [2023-12-27 09:19:34,015][20381] Updated weights for policy 1, policy_version 532430 (0.0005) [2023-12-27 09:19:34,036][20347] Updated weights for policy 0, policy_version 532822 (0.0010) [2023-12-27 09:19:34,081][20347] Updated weights for policy 0, policy_version 532832 (0.0010) [2023-12-27 09:19:34,715][20381] Updated weights for policy 1, policy_version 532440 (0.0009) [2023-12-27 09:19:34,771][20381] Updated weights for policy 1, policy_version 532450 (0.0010) [2023-12-27 09:19:34,794][20347] Updated weights for policy 0, policy_version 532842 (0.0008) [2023-12-27 09:19:34,827][20381] Updated weights for policy 1, policy_version 532460 (0.0010) [2023-12-27 09:19:34,851][20347] Updated weights for policy 0, policy_version 532852 (0.0005) [2023-12-27 09:19:34,909][20347] Updated weights for policy 0, policy_version 532862 (0.0007) [2023-12-27 09:19:35,507][20381] Updated weights for policy 1, policy_version 532470 (0.0008) [2023-12-27 09:19:35,517][20347] Updated weights for policy 0, policy_version 532872 (0.0010) [2023-12-27 09:19:35,562][20381] Updated weights for policy 1, policy_version 532480 (0.0008) [2023-12-27 09:19:35,569][20347] Updated weights for policy 0, policy_version 532882 (0.0010) [2023-12-27 09:19:35,618][20381] Updated weights for policy 1, policy_version 532490 (0.0010) [2023-12-27 09:19:35,624][20347] Updated weights for policy 0, policy_version 532892 (0.0010) [2023-12-27 09:19:36,295][20381] Updated weights for policy 1, policy_version 532500 (0.0010) [2023-12-27 09:19:36,332][20347] Updated weights for policy 0, policy_version 532902 (0.0008) [2023-12-27 09:19:36,350][20381] Updated weights for policy 1, policy_version 532510 (0.0011) [2023-12-27 09:19:36,393][20347] Updated weights for policy 0, policy_version 532912 (0.0009) [2023-12-27 09:19:36,405][20381] Updated weights for policy 1, policy_version 532520 (0.0010) [2023-12-27 09:19:36,453][20347] Updated weights for policy 0, policy_version 532922 (0.0011) [2023-12-27 09:19:37,111][20347] Updated weights for policy 0, policy_version 532932 (0.0009) [2023-12-27 09:19:37,117][20381] Updated weights for policy 1, policy_version 532530 (0.0010) [2023-12-27 09:19:37,161][20347] Updated weights for policy 0, policy_version 532942 (0.0009) [2023-12-27 09:19:37,175][20381] Updated weights for policy 1, policy_version 532540 (0.0010) [2023-12-27 09:19:37,217][20347] Updated weights for policy 0, policy_version 532952 (0.0006) [2023-12-27 09:19:37,234][20381] Updated weights for policy 1, policy_version 532550 (0.0010) [2023-12-27 09:19:37,289][20381] Updated weights for policy 1, policy_version 532560 (0.0010) [2023-12-27 09:19:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 272809984. Throughput: 0: 10354.9, 1: 10261.7. Samples: 272773124. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:37,363][19170] Avg episode reward: [(0, '939.730'), (1, '932.430')] [2023-12-27 09:19:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000532560_136355840.pth... [2023-12-27 09:19:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000532960_136454144.pth... [2023-12-27 09:19:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000531376_136052736.pth [2023-12-27 09:19:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000531744_136142848.pth [2023-12-27 09:19:37,869][20347] Updated weights for policy 0, policy_version 532962 (0.0007) [2023-12-27 09:19:37,922][20347] Updated weights for policy 0, policy_version 532972 (0.0010) [2023-12-27 09:19:37,943][20381] Updated weights for policy 1, policy_version 532570 (0.0006) [2023-12-27 09:19:37,971][20347] Updated weights for policy 0, policy_version 532982 (0.0010) [2023-12-27 09:19:37,996][20381] Updated weights for policy 1, policy_version 532580 (0.0006) [2023-12-27 09:19:38,021][20347] Updated weights for policy 0, policy_version 532992 (0.0010) [2023-12-27 09:19:38,053][20381] Updated weights for policy 1, policy_version 532590 (0.0006) [2023-12-27 09:19:38,720][20381] Updated weights for policy 1, policy_version 532600 (0.0006) [2023-12-27 09:19:38,730][20347] Updated weights for policy 0, policy_version 533002 (0.0008) [2023-12-27 09:19:38,790][20347] Updated weights for policy 0, policy_version 533012 (0.0008) [2023-12-27 09:19:38,791][20381] Updated weights for policy 1, policy_version 532610 (0.0007) [2023-12-27 09:19:38,849][20347] Updated weights for policy 0, policy_version 533022 (0.0009) [2023-12-27 09:19:38,854][20381] Updated weights for policy 1, policy_version 532620 (0.0008) [2023-12-27 09:19:39,499][20347] Updated weights for policy 0, policy_version 533032 (0.0006) [2023-12-27 09:19:39,555][20347] Updated weights for policy 0, policy_version 533042 (0.0008) [2023-12-27 09:19:39,600][20381] Updated weights for policy 1, policy_version 532630 (0.0007) [2023-12-27 09:19:39,622][20347] Updated weights for policy 0, policy_version 533052 (0.0007) [2023-12-27 09:19:39,655][20381] Updated weights for policy 1, policy_version 532640 (0.0008) [2023-12-27 09:19:39,713][20381] Updated weights for policy 1, policy_version 532650 (0.0009) [2023-12-27 09:19:40,286][20347] Updated weights for policy 0, policy_version 533062 (0.0006) [2023-12-27 09:19:40,347][20347] Updated weights for policy 0, policy_version 533072 (0.0006) [2023-12-27 09:19:40,413][20347] Updated weights for policy 0, policy_version 533082 (0.0007) [2023-12-27 09:19:40,417][20381] Updated weights for policy 1, policy_version 532661 (0.0008) [2023-12-27 09:19:40,474][20381] Updated weights for policy 1, policy_version 532671 (0.0006) [2023-12-27 09:19:40,533][20381] Updated weights for policy 1, policy_version 532681 (0.0007) [2023-12-27 09:19:41,061][20347] Updated weights for policy 0, policy_version 533092 (0.0008) [2023-12-27 09:19:41,136][20347] Updated weights for policy 0, policy_version 533102 (0.0006) [2023-12-27 09:19:41,199][20347] Updated weights for policy 0, policy_version 533112 (0.0006) [2023-12-27 09:19:41,248][20381] Updated weights for policy 1, policy_version 532691 (0.0008) [2023-12-27 09:19:41,308][20381] Updated weights for policy 1, policy_version 532701 (0.0011) [2023-12-27 09:19:41,374][20381] Updated weights for policy 1, policy_version 532711 (0.0008) [2023-12-27 09:19:41,827][20347] Updated weights for policy 0, policy_version 533122 (0.0007) [2023-12-27 09:19:41,890][20347] Updated weights for policy 0, policy_version 533132 (0.0008) [2023-12-27 09:19:41,950][20347] Updated weights for policy 0, policy_version 533142 (0.0008) [2023-12-27 09:19:42,009][20347] Updated weights for policy 0, policy_version 533152 (0.0008) [2023-12-27 09:19:42,099][20381] Updated weights for policy 1, policy_version 532721 (0.0009) [2023-12-27 09:19:42,147][20381] Updated weights for policy 1, policy_version 532731 (0.0006) [2023-12-27 09:19:42,200][20381] Updated weights for policy 1, policy_version 532741 (0.0008) [2023-12-27 09:19:42,257][20381] Updated weights for policy 1, policy_version 532751 (0.0008) [2023-12-27 09:19:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 272908288. Throughput: 0: 10371.3, 1: 10195.7. Samples: 272895672. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:42,363][19170] Avg episode reward: [(0, '964.680'), (1, '893.480')] [2023-12-27 09:19:42,692][20347] Updated weights for policy 0, policy_version 533162 (0.0007) [2023-12-27 09:19:42,746][20347] Updated weights for policy 0, policy_version 533172 (0.0006) [2023-12-27 09:19:42,797][20347] Updated weights for policy 0, policy_version 533182 (0.0007) [2023-12-27 09:19:42,969][20381] Updated weights for policy 1, policy_version 532761 (0.0008) [2023-12-27 09:19:43,022][20381] Updated weights for policy 1, policy_version 532771 (0.0008) [2023-12-27 09:19:43,079][20381] Updated weights for policy 1, policy_version 532781 (0.0008) [2023-12-27 09:19:43,495][20347] Updated weights for policy 0, policy_version 533192 (0.0008) [2023-12-27 09:19:43,565][20347] Updated weights for policy 0, policy_version 533202 (0.0008) [2023-12-27 09:19:43,620][20347] Updated weights for policy 0, policy_version 533212 (0.0008) [2023-12-27 09:19:43,777][20381] Updated weights for policy 1, policy_version 532791 (0.0010) [2023-12-27 09:19:43,823][20381] Updated weights for policy 1, policy_version 532801 (0.0010) [2023-12-27 09:19:43,871][20381] Updated weights for policy 1, policy_version 532811 (0.0010) [2023-12-27 09:19:44,277][20347] Updated weights for policy 0, policy_version 533222 (0.0008) [2023-12-27 09:19:44,336][20347] Updated weights for policy 0, policy_version 533232 (0.0006) [2023-12-27 09:19:44,381][20347] Updated weights for policy 0, policy_version 533242 (0.0005) [2023-12-27 09:19:44,535][20381] Updated weights for policy 1, policy_version 532821 (0.0011) [2023-12-27 09:19:44,583][20381] Updated weights for policy 1, policy_version 532831 (0.0011) [2023-12-27 09:19:44,632][20381] Updated weights for policy 1, policy_version 532841 (0.0011) [2023-12-27 09:19:45,026][20347] Updated weights for policy 0, policy_version 533252 (0.0006) [2023-12-27 09:19:45,091][20347] Updated weights for policy 0, policy_version 533262 (0.0006) [2023-12-27 09:19:45,153][20347] Updated weights for policy 0, policy_version 533272 (0.0006) [2023-12-27 09:19:45,313][20381] Updated weights for policy 1, policy_version 532851 (0.0011) [2023-12-27 09:19:45,361][20381] Updated weights for policy 1, policy_version 532861 (0.0010) [2023-12-27 09:19:45,413][20381] Updated weights for policy 1, policy_version 532871 (0.0010) [2023-12-27 09:19:45,808][20347] Updated weights for policy 0, policy_version 533282 (0.0007) [2023-12-27 09:19:45,856][20347] Updated weights for policy 0, policy_version 533292 (0.0005) [2023-12-27 09:19:45,901][20347] Updated weights for policy 0, policy_version 533302 (0.0005) [2023-12-27 09:19:45,954][20347] Updated weights for policy 0, policy_version 533312 (0.0008) [2023-12-27 09:19:46,002][20381] Updated weights for policy 1, policy_version 532881 (0.0010) [2023-12-27 09:19:46,050][20381] Updated weights for policy 1, policy_version 532891 (0.0005) [2023-12-27 09:19:46,108][20381] Updated weights for policy 1, policy_version 532901 (0.0005) [2023-12-27 09:19:46,167][20381] Updated weights for policy 1, policy_version 532911 (0.0005) [2023-12-27 09:19:46,681][20347] Updated weights for policy 0, policy_version 533322 (0.0008) [2023-12-27 09:19:46,740][20347] Updated weights for policy 0, policy_version 533332 (0.0006) [2023-12-27 09:19:46,746][20381] Updated weights for policy 1, policy_version 532921 (0.0006) [2023-12-27 09:19:46,791][20347] Updated weights for policy 0, policy_version 533342 (0.0007) [2023-12-27 09:19:46,802][20381] Updated weights for policy 1, policy_version 532931 (0.0009) [2023-12-27 09:19:46,858][20381] Updated weights for policy 1, policy_version 532941 (0.0005) [2023-12-27 09:19:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 273006592. Throughput: 0: 10361.6, 1: 10267.3. Samples: 273020136. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:47,363][19170] Avg episode reward: [(0, '971.470'), (1, '854.330')] [2023-12-27 09:19:47,406][20347] Updated weights for policy 0, policy_version 533352 (0.0006) [2023-12-27 09:19:47,459][20347] Updated weights for policy 0, policy_version 533362 (0.0005) [2023-12-27 09:19:47,468][20381] Updated weights for policy 1, policy_version 532951 (0.0005) [2023-12-27 09:19:47,510][20347] Updated weights for policy 0, policy_version 533372 (0.0006) [2023-12-27 09:19:47,517][20381] Updated weights for policy 1, policy_version 532961 (0.0005) [2023-12-27 09:19:47,570][20381] Updated weights for policy 1, policy_version 532971 (0.0009) [2023-12-27 09:19:48,151][20381] Updated weights for policy 1, policy_version 532981 (0.0009) [2023-12-27 09:19:48,178][20347] Updated weights for policy 0, policy_version 533382 (0.0006) [2023-12-27 09:19:48,209][20381] Updated weights for policy 1, policy_version 532991 (0.0011) [2023-12-27 09:19:48,240][20347] Updated weights for policy 0, policy_version 533392 (0.0006) [2023-12-27 09:19:48,263][20381] Updated weights for policy 1, policy_version 533001 (0.0010) [2023-12-27 09:19:48,298][20347] Updated weights for policy 0, policy_version 533402 (0.0006) [2023-12-27 09:19:48,948][20381] Updated weights for policy 1, policy_version 533011 (0.0010) [2023-12-27 09:19:49,008][20381] Updated weights for policy 1, policy_version 533021 (0.0011) [2023-12-27 09:19:49,015][20347] Updated weights for policy 0, policy_version 533412 (0.0007) [2023-12-27 09:19:49,067][20347] Updated weights for policy 0, policy_version 533422 (0.0009) [2023-12-27 09:19:49,068][20381] Updated weights for policy 1, policy_version 533031 (0.0010) [2023-12-27 09:19:49,115][20347] Updated weights for policy 0, policy_version 533432 (0.0006) [2023-12-27 09:19:49,731][20381] Updated weights for policy 1, policy_version 533041 (0.0011) [2023-12-27 09:19:49,789][20381] Updated weights for policy 1, policy_version 533051 (0.0010) [2023-12-27 09:19:49,793][20347] Updated weights for policy 0, policy_version 533442 (0.0007) [2023-12-27 09:19:49,848][20381] Updated weights for policy 1, policy_version 533061 (0.0011) [2023-12-27 09:19:49,864][20347] Updated weights for policy 0, policy_version 533453 (0.0009) [2023-12-27 09:19:49,910][20381] Updated weights for policy 1, policy_version 533071 (0.0010) [2023-12-27 09:19:49,920][20347] Updated weights for policy 0, policy_version 533463 (0.0010) [2023-12-27 09:19:50,542][20347] Updated weights for policy 0, policy_version 533473 (0.0009) [2023-12-27 09:19:50,581][20381] Updated weights for policy 1, policy_version 533081 (0.0008) [2023-12-27 09:19:50,595][20291] KL-divergence is very high: 253.3672 [2023-12-27 09:19:50,606][20347] Updated weights for policy 0, policy_version 533483 (0.0011) [2023-12-27 09:19:50,638][20381] Updated weights for policy 1, policy_version 533091 (0.0006) [2023-12-27 09:19:50,645][20291] KL-divergence is very high: 120.7827 [2023-12-27 09:19:50,669][20347] Updated weights for policy 0, policy_version 533493 (0.0011) [2023-12-27 09:19:50,694][20381] Updated weights for policy 1, policy_version 533101 (0.0006) [2023-12-27 09:19:50,729][20347] Updated weights for policy 0, policy_version 533503 (0.0011) [2023-12-27 09:19:51,341][20381] Updated weights for policy 1, policy_version 533111 (0.0009) [2023-12-27 09:19:51,404][20381] Updated weights for policy 1, policy_version 533121 (0.0007) [2023-12-27 09:19:51,421][20347] Updated weights for policy 0, policy_version 533513 (0.0006) [2023-12-27 09:19:51,464][20381] Updated weights for policy 1, policy_version 533131 (0.0007) [2023-12-27 09:19:51,478][20347] Updated weights for policy 0, policy_version 533523 (0.0005) [2023-12-27 09:19:51,526][20347] Updated weights for policy 0, policy_version 533533 (0.0007) [2023-12-27 09:19:52,152][20381] Updated weights for policy 1, policy_version 533141 (0.0007) [2023-12-27 09:19:52,204][20381] Updated weights for policy 1, policy_version 533151 (0.0007) [2023-12-27 09:19:52,259][20347] Updated weights for policy 0, policy_version 533543 (0.0006) [2023-12-27 09:19:52,268][20381] Updated weights for policy 1, policy_version 533161 (0.0007) [2023-12-27 09:19:52,319][20347] Updated weights for policy 0, policy_version 533553 (0.0006) [2023-12-27 09:19:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 273113088. Throughput: 0: 10365.2, 1: 10236.4. Samples: 273082928. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:52,363][19170] Avg episode reward: [(0, '942.760'), (1, '756.660')] [2023-12-27 09:19:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000533168_136511488.pth... [2023-12-27 09:19:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000531952_136200192.pth [2023-12-27 09:19:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000533168_136511488.pth [2023-12-27 09:19:52,381][20347] Updated weights for policy 0, policy_version 533563 (0.0008) [2023-12-27 09:19:52,408][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000533568_136609792.pth... [2023-12-27 09:19:52,412][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000532352_136298496.pth [2023-12-27 09:19:52,413][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000533568_136609792.pth [2023-12-27 09:19:52,939][20381] Updated weights for policy 1, policy_version 533171 (0.0006) [2023-12-27 09:19:53,005][20381] Updated weights for policy 1, policy_version 533181 (0.0005) [2023-12-27 09:19:53,067][20381] Updated weights for policy 1, policy_version 533191 (0.0006) [2023-12-27 09:19:53,095][20347] Updated weights for policy 0, policy_version 533573 (0.0007) [2023-12-27 09:19:53,161][20347] Updated weights for policy 0, policy_version 533583 (0.0006) [2023-12-27 09:19:53,221][20347] Updated weights for policy 0, policy_version 533593 (0.0008) [2023-12-27 09:19:53,621][20381] Updated weights for policy 1, policy_version 533201 (0.0006) [2023-12-27 09:19:53,675][20381] Updated weights for policy 1, policy_version 533211 (0.0010) [2023-12-27 09:19:53,738][20381] Updated weights for policy 1, policy_version 533221 (0.0006) [2023-12-27 09:19:53,807][20381] Updated weights for policy 1, policy_version 533231 (0.0007) [2023-12-27 09:19:53,894][20347] Updated weights for policy 0, policy_version 533603 (0.0007) [2023-12-27 09:19:53,951][20347] Updated weights for policy 0, policy_version 533613 (0.0008) [2023-12-27 09:19:54,005][20347] Updated weights for policy 0, policy_version 533623 (0.0007) [2023-12-27 09:19:54,418][20381] Updated weights for policy 1, policy_version 533241 (0.0008) [2023-12-27 09:19:54,477][20381] Updated weights for policy 1, policy_version 533251 (0.0008) [2023-12-27 09:19:54,538][20381] Updated weights for policy 1, policy_version 533261 (0.0007) [2023-12-27 09:19:54,696][20347] Updated weights for policy 0, policy_version 533633 (0.0008) [2023-12-27 09:19:54,752][20347] Updated weights for policy 0, policy_version 533643 (0.0006) [2023-12-27 09:19:54,804][20347] Updated weights for policy 0, policy_version 533653 (0.0005) [2023-12-27 09:19:54,865][20347] Updated weights for policy 0, policy_version 533663 (0.0005) [2023-12-27 09:19:55,258][20381] Updated weights for policy 1, policy_version 533271 (0.0008) [2023-12-27 09:19:55,309][20381] Updated weights for policy 1, policy_version 533281 (0.0007) [2023-12-27 09:19:55,360][20381] Updated weights for policy 1, policy_version 533291 (0.0008) [2023-12-27 09:19:55,458][20347] Updated weights for policy 0, policy_version 533673 (0.0010) [2023-12-27 09:19:55,518][20347] Updated weights for policy 0, policy_version 533683 (0.0007) [2023-12-27 09:19:55,575][20347] Updated weights for policy 0, policy_version 533693 (0.0007) [2023-12-27 09:19:55,974][20381] Updated weights for policy 1, policy_version 533301 (0.0006) [2023-12-27 09:19:56,023][20381] Updated weights for policy 1, policy_version 533311 (0.0005) [2023-12-27 09:19:56,069][20381] Updated weights for policy 1, policy_version 533321 (0.0005) [2023-12-27 09:19:56,189][20347] Updated weights for policy 0, policy_version 533703 (0.0005) [2023-12-27 09:19:56,245][20347] Updated weights for policy 0, policy_version 533713 (0.0009) [2023-12-27 09:19:56,291][20347] Updated weights for policy 0, policy_version 533723 (0.0010) [2023-12-27 09:19:56,742][20381] Updated weights for policy 1, policy_version 533331 (0.0009) [2023-12-27 09:19:56,801][20381] Updated weights for policy 1, policy_version 533341 (0.0010) [2023-12-27 09:19:56,858][20381] Updated weights for policy 1, policy_version 533351 (0.0010) [2023-12-27 09:19:56,939][20347] Updated weights for policy 0, policy_version 533733 (0.0008) [2023-12-27 09:19:56,993][20347] Updated weights for policy 0, policy_version 533743 (0.0005) [2023-12-27 09:19:57,051][20347] Updated weights for policy 0, policy_version 533753 (0.0006) [2023-12-27 09:19:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 273219584. Throughput: 0: 10346.4, 1: 10297.0. Samples: 273208360. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:19:57,363][19170] Avg episode reward: [(0, '952.400'), (1, '697.920')] [2023-12-27 09:19:57,497][20381] Updated weights for policy 1, policy_version 533361 (0.0010) [2023-12-27 09:19:57,553][20381] Updated weights for policy 1, policy_version 533371 (0.0007) [2023-12-27 09:19:57,608][20381] Updated weights for policy 1, policy_version 533381 (0.0007) [2023-12-27 09:19:57,661][20381] Updated weights for policy 1, policy_version 533391 (0.0006) [2023-12-27 09:19:57,669][20347] Updated weights for policy 0, policy_version 533763 (0.0007) [2023-12-27 09:19:57,718][20347] Updated weights for policy 0, policy_version 533773 (0.0010) [2023-12-27 09:19:57,764][20347] Updated weights for policy 0, policy_version 533783 (0.0010) [2023-12-27 09:19:58,375][20381] Updated weights for policy 1, policy_version 533401 (0.0008) [2023-12-27 09:19:58,410][20347] Updated weights for policy 0, policy_version 533793 (0.0010) [2023-12-27 09:19:58,434][20381] Updated weights for policy 1, policy_version 533411 (0.0008) [2023-12-27 09:19:58,469][20347] Updated weights for policy 0, policy_version 533803 (0.0008) [2023-12-27 09:19:58,493][20381] Updated weights for policy 1, policy_version 533421 (0.0008) [2023-12-27 09:19:58,525][20347] Updated weights for policy 0, policy_version 533813 (0.0007) [2023-12-27 09:19:58,587][20347] Updated weights for policy 0, policy_version 533823 (0.0007) [2023-12-27 09:19:59,315][20347] Updated weights for policy 0, policy_version 533833 (0.0008) [2023-12-27 09:19:59,317][20381] Updated weights for policy 1, policy_version 533431 (0.0008) [2023-12-27 09:19:59,376][20347] Updated weights for policy 0, policy_version 533843 (0.0007) [2023-12-27 09:19:59,379][20381] Updated weights for policy 1, policy_version 533441 (0.0008) [2023-12-27 09:19:59,430][20347] Updated weights for policy 0, policy_version 533853 (0.0007) [2023-12-27 09:19:59,434][20381] Updated weights for policy 1, policy_version 533451 (0.0008) [2023-12-27 09:20:00,063][20347] Updated weights for policy 0, policy_version 533863 (0.0013) [2023-12-27 09:20:00,124][20347] Updated weights for policy 0, policy_version 533873 (0.0010) [2023-12-27 09:20:00,180][20347] Updated weights for policy 0, policy_version 533883 (0.0007) [2023-12-27 09:20:00,203][20381] Updated weights for policy 1, policy_version 533461 (0.0007) [2023-12-27 09:20:00,265][20381] Updated weights for policy 1, policy_version 533471 (0.0006) [2023-12-27 09:20:00,318][20381] Updated weights for policy 1, policy_version 533481 (0.0006) [2023-12-27 09:20:00,809][20347] Updated weights for policy 0, policy_version 533893 (0.0006) [2023-12-27 09:20:00,864][20347] Updated weights for policy 0, policy_version 533903 (0.0005) [2023-12-27 09:20:00,916][20347] Updated weights for policy 0, policy_version 533913 (0.0005) [2023-12-27 09:20:00,979][20381] Updated weights for policy 1, policy_version 533491 (0.0007) [2023-12-27 09:20:01,034][20381] Updated weights for policy 1, policy_version 533501 (0.0007) [2023-12-27 09:20:01,092][20381] Updated weights for policy 1, policy_version 533511 (0.0007) [2023-12-27 09:20:01,552][20347] Updated weights for policy 0, policy_version 533923 (0.0006) [2023-12-27 09:20:01,610][20347] Updated weights for policy 0, policy_version 533933 (0.0007) [2023-12-27 09:20:01,662][20347] Updated weights for policy 0, policy_version 533943 (0.0009) [2023-12-27 09:20:01,727][20381] Updated weights for policy 1, policy_version 533521 (0.0007) [2023-12-27 09:20:01,791][20381] Updated weights for policy 1, policy_version 533531 (0.0006) [2023-12-27 09:20:01,853][20381] Updated weights for policy 1, policy_version 533541 (0.0005) [2023-12-27 09:20:01,916][20381] Updated weights for policy 1, policy_version 533551 (0.0005) [2023-12-27 09:20:02,282][20347] Updated weights for policy 0, policy_version 533953 (0.0007) [2023-12-27 09:20:02,332][20347] Updated weights for policy 0, policy_version 533963 (0.0006) [2023-12-27 09:20:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 273317888. Throughput: 0: 10382.5, 1: 10315.8. Samples: 273332320. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:20:02,363][19170] Avg episode reward: [(0, '971.140'), (1, '511.580')] [2023-12-27 09:20:02,385][20347] Updated weights for policy 0, policy_version 533973 (0.0009) [2023-12-27 09:20:02,433][20347] Updated weights for policy 0, policy_version 533983 (0.0007) [2023-12-27 09:20:02,560][20381] Updated weights for policy 1, policy_version 533561 (0.0006) [2023-12-27 09:20:02,604][20381] Updated weights for policy 1, policy_version 533571 (0.0009) [2023-12-27 09:20:02,647][20381] Updated weights for policy 1, policy_version 533581 (0.0007) [2023-12-27 09:20:03,074][20347] Updated weights for policy 0, policy_version 533993 (0.0006) [2023-12-27 09:20:03,132][20347] Updated weights for policy 0, policy_version 534003 (0.0005) [2023-12-27 09:20:03,192][20347] Updated weights for policy 0, policy_version 534013 (0.0005) [2023-12-27 09:20:03,355][20381] Updated weights for policy 1, policy_version 533591 (0.0007) [2023-12-27 09:20:03,403][20381] Updated weights for policy 1, policy_version 533601 (0.0007) [2023-12-27 09:20:03,462][20381] Updated weights for policy 1, policy_version 533611 (0.0008) [2023-12-27 09:20:03,728][20347] Updated weights for policy 0, policy_version 534023 (0.0005) [2023-12-27 09:20:03,782][20347] Updated weights for policy 0, policy_version 534033 (0.0008) [2023-12-27 09:20:03,830][20347] Updated weights for policy 0, policy_version 534043 (0.0009) [2023-12-27 09:20:04,115][20381] Updated weights for policy 1, policy_version 533621 (0.0008) [2023-12-27 09:20:04,165][20381] Updated weights for policy 1, policy_version 533631 (0.0008) [2023-12-27 09:20:04,230][20381] Updated weights for policy 1, policy_version 533641 (0.0008) [2023-12-27 09:20:04,487][20347] Updated weights for policy 0, policy_version 534053 (0.0007) [2023-12-27 09:20:04,543][20347] Updated weights for policy 0, policy_version 534063 (0.0005) [2023-12-27 09:20:04,596][20347] Updated weights for policy 0, policy_version 534073 (0.0005) [2023-12-27 09:20:04,856][20381] Updated weights for policy 1, policy_version 533651 (0.0007) [2023-12-27 09:20:04,906][20381] Updated weights for policy 1, policy_version 533661 (0.0010) [2023-12-27 09:20:04,951][20381] Updated weights for policy 1, policy_version 533671 (0.0010) [2023-12-27 09:20:05,150][20347] Updated weights for policy 0, policy_version 534083 (0.0007) [2023-12-27 09:20:05,202][20347] Updated weights for policy 0, policy_version 534093 (0.0005) [2023-12-27 09:20:05,253][20347] Updated weights for policy 0, policy_version 534103 (0.0006) [2023-12-27 09:20:05,644][20381] Updated weights for policy 1, policy_version 533681 (0.0010) [2023-12-27 09:20:05,700][20381] Updated weights for policy 1, policy_version 533691 (0.0009) [2023-12-27 09:20:05,754][20381] Updated weights for policy 1, policy_version 533701 (0.0008) [2023-12-27 09:20:05,815][20381] Updated weights for policy 1, policy_version 533711 (0.0010) [2023-12-27 09:20:05,913][20347] Updated weights for policy 0, policy_version 534113 (0.0005) [2023-12-27 09:20:05,973][20347] Updated weights for policy 0, policy_version 534123 (0.0007) [2023-12-27 09:20:06,032][20347] Updated weights for policy 0, policy_version 534133 (0.0008) [2023-12-27 09:20:06,093][20347] Updated weights for policy 0, policy_version 534143 (0.0007) [2023-12-27 09:20:06,499][20381] Updated weights for policy 1, policy_version 533721 (0.0011) [2023-12-27 09:20:06,558][20381] Updated weights for policy 1, policy_version 533731 (0.0011) [2023-12-27 09:20:06,620][20381] Updated weights for policy 1, policy_version 533741 (0.0011) [2023-12-27 09:20:06,785][20347] Updated weights for policy 0, policy_version 534153 (0.0007) [2023-12-27 09:20:06,846][20347] Updated weights for policy 0, policy_version 534163 (0.0008) [2023-12-27 09:20:06,907][20347] Updated weights for policy 0, policy_version 534173 (0.0007) [2023-12-27 09:20:07,271][20381] Updated weights for policy 1, policy_version 533751 (0.0008) [2023-12-27 09:20:07,315][20381] Updated weights for policy 1, policy_version 533761 (0.0007) [2023-12-27 09:20:07,361][20381] Updated weights for policy 1, policy_version 533771 (0.0007) [2023-12-27 09:20:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 273424384. Throughput: 0: 10472.9, 1: 10321.4. Samples: 273398280. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:20:07,363][19170] Avg episode reward: [(0, '990.300'), (1, '471.410')] [2023-12-27 09:20:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000534176_136765440.pth... [2023-12-27 09:20:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000532960_136454144.pth [2023-12-27 09:20:07,386][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000533776_136667136.pth... [2023-12-27 09:20:07,389][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000532560_136355840.pth [2023-12-27 09:20:07,644][20347] Updated weights for policy 0, policy_version 534183 (0.0007) [2023-12-27 09:20:07,690][20347] Updated weights for policy 0, policy_version 534193 (0.0006) [2023-12-27 09:20:07,748][20347] Updated weights for policy 0, policy_version 534203 (0.0010) [2023-12-27 09:20:08,037][20381] Updated weights for policy 1, policy_version 533781 (0.0009) [2023-12-27 09:20:08,092][20381] Updated weights for policy 1, policy_version 533791 (0.0011) [2023-12-27 09:20:08,150][20381] Updated weights for policy 1, policy_version 533801 (0.0009) [2023-12-27 09:20:08,386][20347] Updated weights for policy 0, policy_version 534213 (0.0009) [2023-12-27 09:20:08,445][20347] Updated weights for policy 0, policy_version 534223 (0.0008) [2023-12-27 09:20:08,501][20347] Updated weights for policy 0, policy_version 534233 (0.0008) [2023-12-27 09:20:08,844][20381] Updated weights for policy 1, policy_version 533811 (0.0010) [2023-12-27 09:20:08,895][20381] Updated weights for policy 1, policy_version 533821 (0.0007) [2023-12-27 09:20:08,953][20381] Updated weights for policy 1, policy_version 533831 (0.0009) [2023-12-27 09:20:09,240][20347] Updated weights for policy 0, policy_version 534243 (0.0007) [2023-12-27 09:20:09,292][20347] Updated weights for policy 0, policy_version 534253 (0.0006) [2023-12-27 09:20:09,356][20347] Updated weights for policy 0, policy_version 534263 (0.0008) [2023-12-27 09:20:09,618][20381] Updated weights for policy 1, policy_version 533841 (0.0006) [2023-12-27 09:20:09,684][20381] Updated weights for policy 1, policy_version 533851 (0.0008) [2023-12-27 09:20:09,698][20292] KL-divergence is very high: 211.1927 [2023-12-27 09:20:09,750][20381] Updated weights for policy 1, policy_version 533861 (0.0008) [2023-12-27 09:20:09,804][20381] Updated weights for policy 1, policy_version 533871 (0.0008) [2023-12-27 09:20:09,983][20347] Updated weights for policy 0, policy_version 534273 (0.0007) [2023-12-27 09:20:10,043][20347] Updated weights for policy 0, policy_version 534283 (0.0008) [2023-12-27 09:20:10,100][20347] Updated weights for policy 0, policy_version 534293 (0.0008) [2023-12-27 09:20:10,165][20347] Updated weights for policy 0, policy_version 534303 (0.0008) [2023-12-27 09:20:10,536][20381] Updated weights for policy 1, policy_version 533881 (0.0008) [2023-12-27 09:20:10,585][20381] Updated weights for policy 1, policy_version 533891 (0.0008) [2023-12-27 09:20:10,634][20381] Updated weights for policy 1, policy_version 533901 (0.0007) [2023-12-27 09:20:10,879][20347] Updated weights for policy 0, policy_version 534313 (0.0007) [2023-12-27 09:20:10,934][20347] Updated weights for policy 0, policy_version 534323 (0.0008) [2023-12-27 09:20:10,991][20347] Updated weights for policy 0, policy_version 534333 (0.0007) [2023-12-27 09:20:11,309][20381] Updated weights for policy 1, policy_version 533911 (0.0007) [2023-12-27 09:20:11,372][20381] Updated weights for policy 1, policy_version 533921 (0.0008) [2023-12-27 09:20:11,436][20381] Updated weights for policy 1, policy_version 533931 (0.0007) [2023-12-27 09:20:11,729][20347] Updated weights for policy 0, policy_version 534343 (0.0009) [2023-12-27 09:20:11,796][20347] Updated weights for policy 0, policy_version 534353 (0.0009) [2023-12-27 09:20:11,868][20347] Updated weights for policy 0, policy_version 534363 (0.0008) [2023-12-27 09:20:12,111][20381] Updated weights for policy 1, policy_version 533941 (0.0008) [2023-12-27 09:20:12,166][20381] Updated weights for policy 1, policy_version 533951 (0.0008) [2023-12-27 09:20:12,217][20381] Updated weights for policy 1, policy_version 533961 (0.0008) [2023-12-27 09:20:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 273530880. Throughput: 0: 10485.1, 1: 10352.3. Samples: 273520544. Policy #0 lag: (min: 31.0, avg: 41.6, max: 63.0) [2023-12-27 09:20:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '449.600')] [2023-12-27 09:20:12,563][20347] Updated weights for policy 0, policy_version 534373 (0.0008) [2023-12-27 09:20:12,612][20347] Updated weights for policy 0, policy_version 534383 (0.0007) [2023-12-27 09:20:12,669][20347] Updated weights for policy 0, policy_version 534393 (0.0009) [2023-12-27 09:20:12,944][20381] Updated weights for policy 1, policy_version 533971 (0.0008) [2023-12-27 09:20:13,002][20381] Updated weights for policy 1, policy_version 533981 (0.0008) [2023-12-27 09:20:13,071][20381] Updated weights for policy 1, policy_version 533991 (0.0008) [2023-12-27 09:20:13,388][20347] Updated weights for policy 0, policy_version 534403 (0.0008) [2023-12-27 09:20:13,444][20347] Updated weights for policy 0, policy_version 534413 (0.0007) [2023-12-27 09:20:13,501][20347] Updated weights for policy 0, policy_version 534423 (0.0007) [2023-12-27 09:20:13,716][20381] Updated weights for policy 1, policy_version 534001 (0.0008) [2023-12-27 09:20:13,778][20381] Updated weights for policy 1, policy_version 534011 (0.0006) [2023-12-27 09:20:13,830][20381] Updated weights for policy 1, policy_version 534021 (0.0005) [2023-12-27 09:20:13,876][20381] Updated weights for policy 1, policy_version 534031 (0.0007) [2023-12-27 09:20:14,199][20347] Updated weights for policy 0, policy_version 534433 (0.0007) [2023-12-27 09:20:14,255][20347] Updated weights for policy 0, policy_version 534443 (0.0006) [2023-12-27 09:20:14,316][20347] Updated weights for policy 0, policy_version 534453 (0.0008) [2023-12-27 09:20:14,378][20347] Updated weights for policy 0, policy_version 534463 (0.0008) [2023-12-27 09:20:14,569][20381] Updated weights for policy 1, policy_version 534041 (0.0008) [2023-12-27 09:20:14,624][20381] Updated weights for policy 1, policy_version 534051 (0.0007) [2023-12-27 09:20:14,686][20381] Updated weights for policy 1, policy_version 534061 (0.0007) [2023-12-27 09:20:15,030][20347] Updated weights for policy 0, policy_version 534473 (0.0007) [2023-12-27 09:20:15,094][20347] Updated weights for policy 0, policy_version 534483 (0.0008) [2023-12-27 09:20:15,156][20347] Updated weights for policy 0, policy_version 534493 (0.0008) [2023-12-27 09:20:15,413][20381] Updated weights for policy 1, policy_version 534071 (0.0008) [2023-12-27 09:20:15,465][20381] Updated weights for policy 1, policy_version 534081 (0.0007) [2023-12-27 09:20:15,513][20381] Updated weights for policy 1, policy_version 534091 (0.0007) [2023-12-27 09:20:15,799][20347] Updated weights for policy 0, policy_version 534503 (0.0007) [2023-12-27 09:20:15,844][20347] Updated weights for policy 0, policy_version 534513 (0.0007) [2023-12-27 09:20:15,892][20347] Updated weights for policy 0, policy_version 534523 (0.0007) [2023-12-27 09:20:16,251][20381] Updated weights for policy 1, policy_version 534101 (0.0006) [2023-12-27 09:20:16,306][20381] Updated weights for policy 1, policy_version 534111 (0.0007) [2023-12-27 09:20:16,361][20381] Updated weights for policy 1, policy_version 534121 (0.0007) [2023-12-27 09:20:16,556][20347] Updated weights for policy 0, policy_version 534533 (0.0007) [2023-12-27 09:20:16,608][20347] Updated weights for policy 0, policy_version 534543 (0.0006) [2023-12-27 09:20:16,658][20347] Updated weights for policy 0, policy_version 534553 (0.0005) [2023-12-27 09:20:17,037][20381] Updated weights for policy 1, policy_version 534131 (0.0008) [2023-12-27 09:20:17,085][20381] Updated weights for policy 1, policy_version 534141 (0.0005) [2023-12-27 09:20:17,133][20381] Updated weights for policy 1, policy_version 534151 (0.0005) [2023-12-27 09:20:17,318][20347] Updated weights for policy 0, policy_version 534563 (0.0007) [2023-12-27 09:20:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 273629184. Throughput: 0: 10402.9, 1: 10283.1. Samples: 273641412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '510.950')] [2023-12-27 09:20:17,375][20347] Updated weights for policy 0, policy_version 534573 (0.0010) [2023-12-27 09:20:17,424][20347] Updated weights for policy 0, policy_version 534583 (0.0009) [2023-12-27 09:20:17,708][20381] Updated weights for policy 1, policy_version 534161 (0.0005) [2023-12-27 09:20:17,767][20381] Updated weights for policy 1, policy_version 534171 (0.0007) [2023-12-27 09:20:17,819][20381] Updated weights for policy 1, policy_version 534181 (0.0007) [2023-12-27 09:20:17,881][20381] Updated weights for policy 1, policy_version 534191 (0.0007) [2023-12-27 09:20:18,078][20347] Updated weights for policy 0, policy_version 534593 (0.0006) [2023-12-27 09:20:18,132][20347] Updated weights for policy 0, policy_version 534603 (0.0010) [2023-12-27 09:20:18,189][20347] Updated weights for policy 0, policy_version 534613 (0.0010) [2023-12-27 09:20:18,247][20347] Updated weights for policy 0, policy_version 534623 (0.0010) [2023-12-27 09:20:18,535][20381] Updated weights for policy 1, policy_version 534201 (0.0007) [2023-12-27 09:20:18,582][20381] Updated weights for policy 1, policy_version 534211 (0.0007) [2023-12-27 09:20:18,641][20381] Updated weights for policy 1, policy_version 534221 (0.0007) [2023-12-27 09:20:18,916][20347] Updated weights for policy 0, policy_version 534633 (0.0010) [2023-12-27 09:20:18,973][20347] Updated weights for policy 0, policy_version 534643 (0.0010) [2023-12-27 09:20:19,031][20347] Updated weights for policy 0, policy_version 534653 (0.0010) [2023-12-27 09:20:19,258][20381] Updated weights for policy 1, policy_version 534231 (0.0007) [2023-12-27 09:20:19,318][20381] Updated weights for policy 1, policy_version 534241 (0.0007) [2023-12-27 09:20:19,385][20381] Updated weights for policy 1, policy_version 534251 (0.0007) [2023-12-27 09:20:19,718][20347] Updated weights for policy 0, policy_version 534663 (0.0010) [2023-12-27 09:20:19,780][20347] Updated weights for policy 0, policy_version 534673 (0.0010) [2023-12-27 09:20:19,850][20347] Updated weights for policy 0, policy_version 534683 (0.0009) [2023-12-27 09:20:19,988][20381] Updated weights for policy 1, policy_version 534261 (0.0006) [2023-12-27 09:20:20,047][20381] Updated weights for policy 1, policy_version 534271 (0.0008) [2023-12-27 09:20:20,109][20381] Updated weights for policy 1, policy_version 534281 (0.0007) [2023-12-27 09:20:20,534][20347] Updated weights for policy 0, policy_version 534693 (0.0009) [2023-12-27 09:20:20,593][20347] Updated weights for policy 0, policy_version 534703 (0.0010) [2023-12-27 09:20:20,655][20347] Updated weights for policy 0, policy_version 534713 (0.0008) [2023-12-27 09:20:20,795][20381] Updated weights for policy 1, policy_version 534291 (0.0007) [2023-12-27 09:20:20,858][20381] Updated weights for policy 1, policy_version 534301 (0.0008) [2023-12-27 09:20:20,914][20381] Updated weights for policy 1, policy_version 534311 (0.0008) [2023-12-27 09:20:21,383][20347] Updated weights for policy 0, policy_version 534723 (0.0008) [2023-12-27 09:20:21,440][20347] Updated weights for policy 0, policy_version 534733 (0.0007) [2023-12-27 09:20:21,500][20347] Updated weights for policy 0, policy_version 534743 (0.0007) [2023-12-27 09:20:21,665][20381] Updated weights for policy 1, policy_version 534321 (0.0008) [2023-12-27 09:20:21,725][20381] Updated weights for policy 1, policy_version 534331 (0.0008) [2023-12-27 09:20:21,783][20381] Updated weights for policy 1, policy_version 534341 (0.0007) [2023-12-27 09:20:21,841][20381] Updated weights for policy 1, policy_version 534351 (0.0007) [2023-12-27 09:20:22,166][20347] Updated weights for policy 0, policy_version 534753 (0.0006) [2023-12-27 09:20:22,225][20347] Updated weights for policy 0, policy_version 534763 (0.0010) [2023-12-27 09:20:22,286][20347] Updated weights for policy 0, policy_version 534773 (0.0011) [2023-12-27 09:20:22,349][20347] Updated weights for policy 0, policy_version 534783 (0.0010) [2023-12-27 09:20:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 273735680. Throughput: 0: 10388.4, 1: 10305.4. Samples: 273704344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '126.800')] [2023-12-27 09:20:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000534352_136814592.pth... [2023-12-27 09:20:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000534784_136921088.pth... [2023-12-27 09:20:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000533168_136511488.pth [2023-12-27 09:20:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000533568_136609792.pth [2023-12-27 09:20:22,589][20381] Updated weights for policy 1, policy_version 534361 (0.0006) [2023-12-27 09:20:22,651][20381] Updated weights for policy 1, policy_version 534371 (0.0005) [2023-12-27 09:20:22,715][20381] Updated weights for policy 1, policy_version 534381 (0.0006) [2023-12-27 09:20:23,020][20347] Updated weights for policy 0, policy_version 534793 (0.0006) [2023-12-27 09:20:23,073][20347] Updated weights for policy 0, policy_version 534803 (0.0007) [2023-12-27 09:20:23,141][20347] Updated weights for policy 0, policy_version 534813 (0.0007) [2023-12-27 09:20:23,295][20381] Updated weights for policy 1, policy_version 534391 (0.0010) [2023-12-27 09:20:23,349][20381] Updated weights for policy 1, policy_version 534401 (0.0008) [2023-12-27 09:20:23,409][20381] Updated weights for policy 1, policy_version 534411 (0.0005) [2023-12-27 09:20:23,731][20347] Updated weights for policy 0, policy_version 534823 (0.0007) [2023-12-27 09:20:23,794][20347] Updated weights for policy 0, policy_version 534833 (0.0008) [2023-12-27 09:20:23,856][20347] Updated weights for policy 0, policy_version 534843 (0.0007) [2023-12-27 09:20:23,980][20381] Updated weights for policy 1, policy_version 534421 (0.0005) [2023-12-27 09:20:24,044][20381] Updated weights for policy 1, policy_version 534431 (0.0006) [2023-12-27 09:20:24,111][20381] Updated weights for policy 1, policy_version 534441 (0.0008) [2023-12-27 09:20:24,550][20347] Updated weights for policy 0, policy_version 534853 (0.0008) [2023-12-27 09:20:24,605][20347] Updated weights for policy 0, policy_version 534863 (0.0008) [2023-12-27 09:20:24,662][20347] Updated weights for policy 0, policy_version 534873 (0.0005) [2023-12-27 09:20:24,721][20381] Updated weights for policy 1, policy_version 534451 (0.0008) [2023-12-27 09:20:24,780][20381] Updated weights for policy 1, policy_version 534461 (0.0007) [2023-12-27 09:20:24,833][20381] Updated weights for policy 1, policy_version 534471 (0.0007) [2023-12-27 09:20:25,219][20347] Updated weights for policy 0, policy_version 534883 (0.0006) [2023-12-27 09:20:25,278][20347] Updated weights for policy 0, policy_version 534893 (0.0008) [2023-12-27 09:20:25,323][20347] Updated weights for policy 0, policy_version 534903 (0.0007) [2023-12-27 09:20:25,531][20381] Updated weights for policy 1, policy_version 534481 (0.0008) [2023-12-27 09:20:25,588][20381] Updated weights for policy 1, policy_version 534491 (0.0006) [2023-12-27 09:20:25,639][20381] Updated weights for policy 1, policy_version 534501 (0.0005) [2023-12-27 09:20:25,691][20381] Updated weights for policy 1, policy_version 534511 (0.0006) [2023-12-27 09:20:25,930][20347] Updated weights for policy 0, policy_version 534913 (0.0008) [2023-12-27 09:20:25,975][20347] Updated weights for policy 0, policy_version 534923 (0.0010) [2023-12-27 09:20:26,020][20347] Updated weights for policy 0, policy_version 534933 (0.0010) [2023-12-27 09:20:26,065][20347] Updated weights for policy 0, policy_version 534943 (0.0009) [2023-12-27 09:20:26,331][20381] Updated weights for policy 1, policy_version 534521 (0.0006) [2023-12-27 09:20:26,390][20381] Updated weights for policy 1, policy_version 534531 (0.0008) [2023-12-27 09:20:26,449][20381] Updated weights for policy 1, policy_version 534541 (0.0008) [2023-12-27 09:20:26,731][20347] Updated weights for policy 0, policy_version 534953 (0.0007) [2023-12-27 09:20:26,779][20347] Updated weights for policy 0, policy_version 534963 (0.0009) [2023-12-27 09:20:26,836][20347] Updated weights for policy 0, policy_version 534973 (0.0010) [2023-12-27 09:20:27,064][20381] Updated weights for policy 1, policy_version 534551 (0.0007) [2023-12-27 09:20:27,122][20381] Updated weights for policy 1, policy_version 534561 (0.0007) [2023-12-27 09:20:27,175][20381] Updated weights for policy 1, policy_version 534571 (0.0007) [2023-12-27 09:20:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 273842176. Throughput: 0: 10403.4, 1: 10355.5. Samples: 273829820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:27,363][19170] Avg episode reward: [(0, '991.430'), (1, '236.180')] [2023-12-27 09:20:27,472][20347] Updated weights for policy 0, policy_version 534983 (0.0010) [2023-12-27 09:20:27,520][20347] Updated weights for policy 0, policy_version 534993 (0.0010) [2023-12-27 09:20:27,521][20291] KL-divergence is very high: 203.5779 [2023-12-27 09:20:27,572][20347] Updated weights for policy 0, policy_version 535003 (0.0007) [2023-12-27 09:20:27,735][20381] Updated weights for policy 1, policy_version 534581 (0.0006) [2023-12-27 09:20:27,781][20381] Updated weights for policy 1, policy_version 534591 (0.0005) [2023-12-27 09:20:27,826][20381] Updated weights for policy 1, policy_version 534601 (0.0005) [2023-12-27 09:20:28,226][20347] Updated weights for policy 0, policy_version 535013 (0.0007) [2023-12-27 09:20:28,284][20347] Updated weights for policy 0, policy_version 535023 (0.0005) [2023-12-27 09:20:28,344][20347] Updated weights for policy 0, policy_version 535033 (0.0006) [2023-12-27 09:20:28,476][20381] Updated weights for policy 1, policy_version 534611 (0.0006) [2023-12-27 09:20:28,534][20381] Updated weights for policy 1, policy_version 534621 (0.0010) [2023-12-27 09:20:28,597][20381] Updated weights for policy 1, policy_version 534631 (0.0010) [2023-12-27 09:20:28,890][20347] Updated weights for policy 0, policy_version 535043 (0.0006) [2023-12-27 09:20:28,935][20347] Updated weights for policy 0, policy_version 535053 (0.0008) [2023-12-27 09:20:28,980][20347] Updated weights for policy 0, policy_version 535063 (0.0008) [2023-12-27 09:20:29,318][20381] Updated weights for policy 1, policy_version 534641 (0.0009) [2023-12-27 09:20:29,382][20381] Updated weights for policy 1, policy_version 534651 (0.0007) [2023-12-27 09:20:29,438][20381] Updated weights for policy 1, policy_version 534661 (0.0008) [2023-12-27 09:20:29,490][20381] Updated weights for policy 1, policy_version 534671 (0.0008) [2023-12-27 09:20:29,670][20347] Updated weights for policy 0, policy_version 535073 (0.0007) [2023-12-27 09:20:29,719][20347] Updated weights for policy 0, policy_version 535083 (0.0008) [2023-12-27 09:20:29,772][20347] Updated weights for policy 0, policy_version 535093 (0.0009) [2023-12-27 09:20:29,824][20347] Updated weights for policy 0, policy_version 535103 (0.0008) [2023-12-27 09:20:30,131][20381] Updated weights for policy 1, policy_version 534681 (0.0007) [2023-12-27 09:20:30,186][20381] Updated weights for policy 1, policy_version 534691 (0.0008) [2023-12-27 09:20:30,236][20381] Updated weights for policy 1, policy_version 534701 (0.0007) [2023-12-27 09:20:30,554][20347] Updated weights for policy 0, policy_version 535113 (0.0007) [2023-12-27 09:20:30,601][20347] Updated weights for policy 0, policy_version 535123 (0.0007) [2023-12-27 09:20:30,655][20347] Updated weights for policy 0, policy_version 535133 (0.0007) [2023-12-27 09:20:30,881][20381] Updated weights for policy 1, policy_version 534711 (0.0009) [2023-12-27 09:20:30,944][20381] Updated weights for policy 1, policy_version 534721 (0.0010) [2023-12-27 09:20:31,003][20381] Updated weights for policy 1, policy_version 534731 (0.0010) [2023-12-27 09:20:31,276][20347] Updated weights for policy 0, policy_version 535143 (0.0007) [2023-12-27 09:20:31,332][20347] Updated weights for policy 0, policy_version 535153 (0.0007) [2023-12-27 09:20:31,394][20347] Updated weights for policy 0, policy_version 535163 (0.0007) [2023-12-27 09:20:31,738][20381] Updated weights for policy 1, policy_version 534741 (0.0009) [2023-12-27 09:20:31,790][20381] Updated weights for policy 1, policy_version 534751 (0.0008) [2023-12-27 09:20:31,839][20381] Updated weights for policy 1, policy_version 534761 (0.0009) [2023-12-27 09:20:32,027][20347] Updated weights for policy 0, policy_version 535173 (0.0006) [2023-12-27 09:20:32,083][20347] Updated weights for policy 0, policy_version 535183 (0.0007) [2023-12-27 09:20:32,140][20347] Updated weights for policy 0, policy_version 535193 (0.0006) [2023-12-27 09:20:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.1). Total num frames: 273948672. Throughput: 0: 10470.9, 1: 10366.0. Samples: 273957796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:32,363][19170] Avg episode reward: [(0, '991.430'), (1, '245.560')] [2023-12-27 09:20:32,544][20381] Updated weights for policy 1, policy_version 534771 (0.0007) [2023-12-27 09:20:32,596][20381] Updated weights for policy 1, policy_version 534781 (0.0007) [2023-12-27 09:20:32,646][20381] Updated weights for policy 1, policy_version 534791 (0.0007) [2023-12-27 09:20:32,815][20347] Updated weights for policy 0, policy_version 535203 (0.0005) [2023-12-27 09:20:32,879][20347] Updated weights for policy 0, policy_version 535213 (0.0007) [2023-12-27 09:20:32,944][20347] Updated weights for policy 0, policy_version 535223 (0.0006) [2023-12-27 09:20:33,312][20381] Updated weights for policy 1, policy_version 534801 (0.0007) [2023-12-27 09:20:33,368][20381] Updated weights for policy 1, policy_version 534811 (0.0006) [2023-12-27 09:20:33,426][20381] Updated weights for policy 1, policy_version 534821 (0.0008) [2023-12-27 09:20:33,480][20381] Updated weights for policy 1, policy_version 534831 (0.0007) [2023-12-27 09:20:33,567][20347] Updated weights for policy 0, policy_version 535233 (0.0008) [2023-12-27 09:20:33,618][20347] Updated weights for policy 0, policy_version 535243 (0.0007) [2023-12-27 09:20:33,673][20347] Updated weights for policy 0, policy_version 535253 (0.0008) [2023-12-27 09:20:33,728][20347] Updated weights for policy 0, policy_version 535263 (0.0007) [2023-12-27 09:20:34,166][20381] Updated weights for policy 1, policy_version 534841 (0.0007) [2023-12-27 09:20:34,225][20381] Updated weights for policy 1, policy_version 534851 (0.0006) [2023-12-27 09:20:34,290][20381] Updated weights for policy 1, policy_version 534861 (0.0006) [2023-12-27 09:20:34,323][20347] Updated weights for policy 0, policy_version 535273 (0.0007) [2023-12-27 09:20:34,376][20347] Updated weights for policy 0, policy_version 535283 (0.0008) [2023-12-27 09:20:34,437][20347] Updated weights for policy 0, policy_version 535293 (0.0008) [2023-12-27 09:20:34,812][20381] Updated weights for policy 1, policy_version 534871 (0.0006) [2023-12-27 09:20:34,856][20381] Updated weights for policy 1, policy_version 534881 (0.0005) [2023-12-27 09:20:34,910][20381] Updated weights for policy 1, policy_version 534891 (0.0005) [2023-12-27 09:20:35,094][20347] Updated weights for policy 0, policy_version 535303 (0.0006) [2023-12-27 09:20:35,143][20347] Updated weights for policy 0, policy_version 535313 (0.0005) [2023-12-27 09:20:35,198][20347] Updated weights for policy 0, policy_version 535323 (0.0006) [2023-12-27 09:20:35,493][20381] Updated weights for policy 1, policy_version 534901 (0.0007) [2023-12-27 09:20:35,537][20381] Updated weights for policy 1, policy_version 534911 (0.0008) [2023-12-27 09:20:35,591][20381] Updated weights for policy 1, policy_version 534921 (0.0007) [2023-12-27 09:20:35,774][20347] Updated weights for policy 0, policy_version 535333 (0.0006) [2023-12-27 09:20:35,822][20347] Updated weights for policy 0, policy_version 535343 (0.0008) [2023-12-27 09:20:35,883][20347] Updated weights for policy 0, policy_version 535353 (0.0006) [2023-12-27 09:20:36,241][20381] Updated weights for policy 1, policy_version 534931 (0.0005) [2023-12-27 09:20:36,298][20381] Updated weights for policy 1, policy_version 534941 (0.0007) [2023-12-27 09:20:36,360][20381] Updated weights for policy 1, policy_version 534951 (0.0006) [2023-12-27 09:20:36,545][20347] Updated weights for policy 0, policy_version 535363 (0.0007) [2023-12-27 09:20:36,613][20347] Updated weights for policy 0, policy_version 535373 (0.0008) [2023-12-27 09:20:36,684][20347] Updated weights for policy 0, policy_version 535383 (0.0008) [2023-12-27 09:20:36,988][20381] Updated weights for policy 1, policy_version 534961 (0.0008) [2023-12-27 09:20:37,047][20381] Updated weights for policy 1, policy_version 534971 (0.0007) [2023-12-27 09:20:37,094][20381] Updated weights for policy 1, policy_version 534981 (0.0008) [2023-12-27 09:20:37,149][20381] Updated weights for policy 1, policy_version 534991 (0.0007) [2023-12-27 09:20:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 274055168. Throughput: 0: 10509.1, 1: 10382.4. Samples: 274023044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '249.500')] [2023-12-27 09:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000534992_136978432.pth... [2023-12-27 09:20:37,369][20347] Updated weights for policy 0, policy_version 535393 (0.0007) [2023-12-27 09:20:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000533776_136667136.pth [2023-12-27 09:20:37,423][20347] Updated weights for policy 0, policy_version 535403 (0.0005) [2023-12-27 09:20:37,474][20347] Updated weights for policy 0, policy_version 535413 (0.0006) [2023-12-27 09:20:37,527][20347] Updated weights for policy 0, policy_version 535423 (0.0005) [2023-12-27 09:20:37,529][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000535424_137084928.pth... [2023-12-27 09:20:37,532][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000534176_136765440.pth [2023-12-27 09:20:37,864][20381] Updated weights for policy 1, policy_version 535001 (0.0006) [2023-12-27 09:20:37,921][20381] Updated weights for policy 1, policy_version 535011 (0.0007) [2023-12-27 09:20:37,970][20381] Updated weights for policy 1, policy_version 535021 (0.0007) [2023-12-27 09:20:38,142][20347] Updated weights for policy 0, policy_version 535433 (0.0008) [2023-12-27 09:20:38,193][20347] Updated weights for policy 0, policy_version 535443 (0.0006) [2023-12-27 09:20:38,252][20347] Updated weights for policy 0, policy_version 535453 (0.0007) [2023-12-27 09:20:38,638][20381] Updated weights for policy 1, policy_version 535031 (0.0009) [2023-12-27 09:20:38,702][20381] Updated weights for policy 1, policy_version 535041 (0.0007) [2023-12-27 09:20:38,759][20381] Updated weights for policy 1, policy_version 535051 (0.0007) [2023-12-27 09:20:38,932][20347] Updated weights for policy 0, policy_version 535463 (0.0007) [2023-12-27 09:20:38,982][20347] Updated weights for policy 0, policy_version 535473 (0.0005) [2023-12-27 09:20:39,035][20347] Updated weights for policy 0, policy_version 535483 (0.0006) [2023-12-27 09:20:39,416][20381] Updated weights for policy 1, policy_version 535061 (0.0008) [2023-12-27 09:20:39,476][20381] Updated weights for policy 1, policy_version 535071 (0.0008) [2023-12-27 09:20:39,540][20381] Updated weights for policy 1, policy_version 535081 (0.0007) [2023-12-27 09:20:39,773][20347] Updated weights for policy 0, policy_version 535493 (0.0008) [2023-12-27 09:20:39,836][20347] Updated weights for policy 0, policy_version 535503 (0.0009) [2023-12-27 09:20:39,889][20347] Updated weights for policy 0, policy_version 535513 (0.0008) [2023-12-27 09:20:40,155][20381] Updated weights for policy 1, policy_version 535091 (0.0007) [2023-12-27 09:20:40,217][20381] Updated weights for policy 1, policy_version 535101 (0.0009) [2023-12-27 09:20:40,274][20381] Updated weights for policy 1, policy_version 535111 (0.0008) [2023-12-27 09:20:40,629][20347] Updated weights for policy 0, policy_version 535523 (0.0009) [2023-12-27 09:20:40,690][20347] Updated weights for policy 0, policy_version 535533 (0.0008) [2023-12-27 09:20:40,754][20347] Updated weights for policy 0, policy_version 535543 (0.0007) [2023-12-27 09:20:40,977][20381] Updated weights for policy 1, policy_version 535121 (0.0007) [2023-12-27 09:20:41,044][20381] Updated weights for policy 1, policy_version 535131 (0.0007) [2023-12-27 09:20:41,103][20381] Updated weights for policy 1, policy_version 535141 (0.0010) [2023-12-27 09:20:41,165][20381] Updated weights for policy 1, policy_version 535151 (0.0010) [2023-12-27 09:20:41,450][20347] Updated weights for policy 0, policy_version 535553 (0.0007) [2023-12-27 09:20:41,505][20347] Updated weights for policy 0, policy_version 535563 (0.0008) [2023-12-27 09:20:41,560][20347] Updated weights for policy 0, policy_version 535573 (0.0006) [2023-12-27 09:20:41,618][20347] Updated weights for policy 0, policy_version 535583 (0.0008) [2023-12-27 09:20:41,859][20381] Updated weights for policy 1, policy_version 535161 (0.0006) [2023-12-27 09:20:41,908][20381] Updated weights for policy 1, policy_version 535171 (0.0005) [2023-12-27 09:20:41,955][20381] Updated weights for policy 1, policy_version 535181 (0.0007) [2023-12-27 09:20:42,315][20347] Updated weights for policy 0, policy_version 535593 (0.0008) [2023-12-27 09:20:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 274153472. Throughput: 0: 10496.0, 1: 10361.1. Samples: 274146928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '576.400')] [2023-12-27 09:20:42,382][20347] Updated weights for policy 0, policy_version 535603 (0.0006) [2023-12-27 09:20:42,441][20347] Updated weights for policy 0, policy_version 535613 (0.0005) [2023-12-27 09:20:42,619][20381] Updated weights for policy 1, policy_version 535191 (0.0007) [2023-12-27 09:20:42,666][20381] Updated weights for policy 1, policy_version 535201 (0.0005) [2023-12-27 09:20:42,724][20381] Updated weights for policy 1, policy_version 535211 (0.0005) [2023-12-27 09:20:43,159][20347] Updated weights for policy 0, policy_version 535623 (0.0009) [2023-12-27 09:20:43,226][20347] Updated weights for policy 0, policy_version 535633 (0.0007) [2023-12-27 09:20:43,287][20381] Updated weights for policy 1, policy_version 535221 (0.0008) [2023-12-27 09:20:43,290][20347] Updated weights for policy 0, policy_version 535643 (0.0007) [2023-12-27 09:20:43,346][20381] Updated weights for policy 1, policy_version 535231 (0.0010) [2023-12-27 09:20:43,398][20381] Updated weights for policy 1, policy_version 535241 (0.0010) [2023-12-27 09:20:43,884][20347] Updated weights for policy 0, policy_version 535653 (0.0006) [2023-12-27 09:20:43,937][20347] Updated weights for policy 0, policy_version 535663 (0.0008) [2023-12-27 09:20:43,987][20347] Updated weights for policy 0, policy_version 535673 (0.0007) [2023-12-27 09:20:44,064][20381] Updated weights for policy 1, policy_version 535251 (0.0010) [2023-12-27 09:20:44,112][20381] Updated weights for policy 1, policy_version 535261 (0.0007) [2023-12-27 09:20:44,163][20381] Updated weights for policy 1, policy_version 535271 (0.0008) [2023-12-27 09:20:44,666][20347] Updated weights for policy 0, policy_version 535683 (0.0006) [2023-12-27 09:20:44,726][20347] Updated weights for policy 0, policy_version 535693 (0.0005) [2023-12-27 09:20:44,777][20347] Updated weights for policy 0, policy_version 535703 (0.0008) [2023-12-27 09:20:44,802][20381] Updated weights for policy 1, policy_version 535281 (0.0006) [2023-12-27 09:20:44,860][20381] Updated weights for policy 1, policy_version 535291 (0.0007) [2023-12-27 09:20:44,922][20381] Updated weights for policy 1, policy_version 535301 (0.0007) [2023-12-27 09:20:44,979][20381] Updated weights for policy 1, policy_version 535311 (0.0009) [2023-12-27 09:20:45,419][20347] Updated weights for policy 0, policy_version 535713 (0.0006) [2023-12-27 09:20:45,478][20347] Updated weights for policy 0, policy_version 535723 (0.0006) [2023-12-27 09:20:45,535][20347] Updated weights for policy 0, policy_version 535733 (0.0007) [2023-12-27 09:20:45,594][20347] Updated weights for policy 0, policy_version 535743 (0.0008) [2023-12-27 09:20:45,730][20381] Updated weights for policy 1, policy_version 535321 (0.0008) [2023-12-27 09:20:45,790][20381] Updated weights for policy 1, policy_version 535331 (0.0008) [2023-12-27 09:20:45,850][20381] Updated weights for policy 1, policy_version 535341 (0.0008) [2023-12-27 09:20:46,150][20347] Updated weights for policy 0, policy_version 535753 (0.0006) [2023-12-27 09:20:46,215][20347] Updated weights for policy 0, policy_version 535763 (0.0007) [2023-12-27 09:20:46,273][20347] Updated weights for policy 0, policy_version 535773 (0.0007) [2023-12-27 09:20:46,568][20381] Updated weights for policy 1, policy_version 535351 (0.0007) [2023-12-27 09:20:46,625][20381] Updated weights for policy 1, policy_version 535361 (0.0005) [2023-12-27 09:20:46,683][20381] Updated weights for policy 1, policy_version 535371 (0.0007) [2023-12-27 09:20:46,909][20347] Updated weights for policy 0, policy_version 535783 (0.0009) [2023-12-27 09:20:46,956][20347] Updated weights for policy 0, policy_version 535793 (0.0010) [2023-12-27 09:20:47,019][20347] Updated weights for policy 0, policy_version 535803 (0.0010) [2023-12-27 09:20:47,282][20381] Updated weights for policy 1, policy_version 535381 (0.0007) [2023-12-27 09:20:47,340][20381] Updated weights for policy 1, policy_version 535391 (0.0007) [2023-12-27 09:20:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20716.1). Total num frames: 274259968. Throughput: 0: 10472.6, 1: 10415.8. Samples: 274272296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '371.630')] [2023-12-27 09:20:47,396][20381] Updated weights for policy 1, policy_version 535401 (0.0006) [2023-12-27 09:20:47,662][20347] Updated weights for policy 0, policy_version 535813 (0.0007) [2023-12-27 09:20:47,711][20347] Updated weights for policy 0, policy_version 535823 (0.0008) [2023-12-27 09:20:47,767][20347] Updated weights for policy 0, policy_version 535833 (0.0007) [2023-12-27 09:20:48,050][20381] Updated weights for policy 1, policy_version 535411 (0.0008) [2023-12-27 09:20:48,102][20381] Updated weights for policy 1, policy_version 535421 (0.0007) [2023-12-27 09:20:48,147][20381] Updated weights for policy 1, policy_version 535431 (0.0008) [2023-12-27 09:20:48,467][20347] Updated weights for policy 0, policy_version 535843 (0.0007) [2023-12-27 09:20:48,522][20347] Updated weights for policy 0, policy_version 535853 (0.0007) [2023-12-27 09:20:48,578][20347] Updated weights for policy 0, policy_version 535863 (0.0008) [2023-12-27 09:20:48,881][20381] Updated weights for policy 1, policy_version 535441 (0.0007) [2023-12-27 09:20:48,936][20381] Updated weights for policy 1, policy_version 535451 (0.0006) [2023-12-27 09:20:48,993][20381] Updated weights for policy 1, policy_version 535461 (0.0007) [2023-12-27 09:20:49,044][20381] Updated weights for policy 1, policy_version 535471 (0.0009) [2023-12-27 09:20:49,245][20347] Updated weights for policy 0, policy_version 535873 (0.0008) [2023-12-27 09:20:49,298][20347] Updated weights for policy 0, policy_version 535883 (0.0008) [2023-12-27 09:20:49,368][20347] Updated weights for policy 0, policy_version 535893 (0.0008) [2023-12-27 09:20:49,433][20347] Updated weights for policy 0, policy_version 535903 (0.0008) [2023-12-27 09:20:49,676][20381] Updated weights for policy 1, policy_version 535481 (0.0008) [2023-12-27 09:20:49,722][20381] Updated weights for policy 1, policy_version 535491 (0.0008) [2023-12-27 09:20:49,774][20381] Updated weights for policy 1, policy_version 535501 (0.0007) [2023-12-27 09:20:50,094][20347] Updated weights for policy 0, policy_version 535913 (0.0008) [2023-12-27 09:20:50,153][20347] Updated weights for policy 0, policy_version 535923 (0.0006) [2023-12-27 09:20:50,215][20347] Updated weights for policy 0, policy_version 535933 (0.0007) [2023-12-27 09:20:50,472][20381] Updated weights for policy 1, policy_version 535511 (0.0008) [2023-12-27 09:20:50,538][20381] Updated weights for policy 1, policy_version 535521 (0.0008) [2023-12-27 09:20:50,603][20381] Updated weights for policy 1, policy_version 535531 (0.0007) [2023-12-27 09:20:50,857][20347] Updated weights for policy 0, policy_version 535943 (0.0008) [2023-12-27 09:20:50,912][20347] Updated weights for policy 0, policy_version 535953 (0.0008) [2023-12-27 09:20:50,961][20347] Updated weights for policy 0, policy_version 535963 (0.0007) [2023-12-27 09:20:51,303][20381] Updated weights for policy 1, policy_version 535541 (0.0008) [2023-12-27 09:20:51,368][20381] Updated weights for policy 1, policy_version 535551 (0.0007) [2023-12-27 09:20:51,428][20381] Updated weights for policy 1, policy_version 535561 (0.0008) [2023-12-27 09:20:51,679][20347] Updated weights for policy 0, policy_version 535973 (0.0008) [2023-12-27 09:20:51,743][20347] Updated weights for policy 0, policy_version 535983 (0.0007) [2023-12-27 09:20:51,796][20347] Updated weights for policy 0, policy_version 535993 (0.0007) [2023-12-27 09:20:52,058][20381] Updated weights for policy 1, policy_version 535571 (0.0007) [2023-12-27 09:20:52,119][20381] Updated weights for policy 1, policy_version 535581 (0.0007) [2023-12-27 09:20:52,181][20381] Updated weights for policy 1, policy_version 535591 (0.0008) [2023-12-27 09:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 274366464. Throughput: 0: 10394.5, 1: 10410.3. Samples: 274334496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '458.070')] [2023-12-27 09:20:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000535600_137134080.pth... [2023-12-27 09:20:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000536000_137232384.pth... [2023-12-27 09:20:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000534352_136814592.pth [2023-12-27 09:20:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000534784_136921088.pth [2023-12-27 09:20:52,574][20347] Updated weights for policy 0, policy_version 536003 (0.0009) [2023-12-27 09:20:52,640][20347] Updated weights for policy 0, policy_version 536013 (0.0011) [2023-12-27 09:20:52,697][20347] Updated weights for policy 0, policy_version 536023 (0.0010) [2023-12-27 09:20:52,815][20381] Updated weights for policy 1, policy_version 535601 (0.0008) [2023-12-27 09:20:52,874][20381] Updated weights for policy 1, policy_version 535611 (0.0007) [2023-12-27 09:20:52,932][20381] Updated weights for policy 1, policy_version 535621 (0.0008) [2023-12-27 09:20:52,981][20381] Updated weights for policy 1, policy_version 535631 (0.0007) [2023-12-27 09:20:53,348][20347] Updated weights for policy 0, policy_version 536033 (0.0011) [2023-12-27 09:20:53,399][20347] Updated weights for policy 0, policy_version 536043 (0.0010) [2023-12-27 09:20:53,408][20291] KL-divergence is very high: 111.7637 [2023-12-27 09:20:53,448][20347] Updated weights for policy 0, policy_version 536053 (0.0010) [2023-12-27 09:20:53,490][20291] KL-divergence is very high: 136.0820 [2023-12-27 09:20:53,503][20347] Updated weights for policy 0, policy_version 536063 (0.0010) [2023-12-27 09:20:53,595][20381] Updated weights for policy 1, policy_version 535641 (0.0007) [2023-12-27 09:20:53,653][20381] Updated weights for policy 1, policy_version 535651 (0.0007) [2023-12-27 09:20:53,715][20381] Updated weights for policy 1, policy_version 535661 (0.0010) [2023-12-27 09:20:54,166][20347] Updated weights for policy 0, policy_version 536073 (0.0006) [2023-12-27 09:20:54,221][20347] Updated weights for policy 0, policy_version 536083 (0.0005) [2023-12-27 09:20:54,267][20347] Updated weights for policy 0, policy_version 536093 (0.0005) [2023-12-27 09:20:54,336][20381] Updated weights for policy 1, policy_version 535671 (0.0008) [2023-12-27 09:20:54,395][20381] Updated weights for policy 1, policy_version 535681 (0.0008) [2023-12-27 09:20:54,455][20381] Updated weights for policy 1, policy_version 535691 (0.0007) [2023-12-27 09:20:54,919][20347] Updated weights for policy 0, policy_version 536103 (0.0006) [2023-12-27 09:20:54,979][20347] Updated weights for policy 0, policy_version 536113 (0.0007) [2023-12-27 09:20:55,043][20347] Updated weights for policy 0, policy_version 536123 (0.0008) [2023-12-27 09:20:55,145][20381] Updated weights for policy 1, policy_version 535701 (0.0008) [2023-12-27 09:20:55,200][20381] Updated weights for policy 1, policy_version 535711 (0.0007) [2023-12-27 09:20:55,258][20381] Updated weights for policy 1, policy_version 535721 (0.0008) [2023-12-27 09:20:55,716][20347] Updated weights for policy 0, policy_version 536133 (0.0008) [2023-12-27 09:20:55,772][20347] Updated weights for policy 0, policy_version 536143 (0.0007) [2023-12-27 09:20:55,831][20347] Updated weights for policy 0, policy_version 536153 (0.0007) [2023-12-27 09:20:55,933][20381] Updated weights for policy 1, policy_version 535731 (0.0008) [2023-12-27 09:20:55,981][20381] Updated weights for policy 1, policy_version 535741 (0.0007) [2023-12-27 09:20:56,040][20381] Updated weights for policy 1, policy_version 535751 (0.0008) [2023-12-27 09:20:56,469][20347] Updated weights for policy 0, policy_version 536163 (0.0008) [2023-12-27 09:20:56,532][20347] Updated weights for policy 0, policy_version 536173 (0.0008) [2023-12-27 09:20:56,588][20347] Updated weights for policy 0, policy_version 536183 (0.0008) [2023-12-27 09:20:56,743][20381] Updated weights for policy 1, policy_version 535761 (0.0008) [2023-12-27 09:20:56,788][20381] Updated weights for policy 1, policy_version 535771 (0.0008) [2023-12-27 09:20:56,841][20381] Updated weights for policy 1, policy_version 535781 (0.0007) [2023-12-27 09:20:56,903][20381] Updated weights for policy 1, policy_version 535791 (0.0007) [2023-12-27 09:20:57,233][20347] Updated weights for policy 0, policy_version 536193 (0.0007) [2023-12-27 09:20:57,278][20347] Updated weights for policy 0, policy_version 536203 (0.0008) [2023-12-27 09:20:57,330][20347] Updated weights for policy 0, policy_version 536213 (0.0007) [2023-12-27 09:20:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 274464768. Throughput: 0: 10386.8, 1: 10434.8. Samples: 274457516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:20:57,363][19170] Avg episode reward: [(0, '981.770'), (1, '480.730')] [2023-12-27 09:20:57,386][20347] Updated weights for policy 0, policy_version 536223 (0.0008) [2023-12-27 09:20:57,581][20381] Updated weights for policy 1, policy_version 535801 (0.0008) [2023-12-27 09:20:57,628][20381] Updated weights for policy 1, policy_version 535811 (0.0007) [2023-12-27 09:20:57,674][20381] Updated weights for policy 1, policy_version 535821 (0.0007) [2023-12-27 09:20:58,095][20347] Updated weights for policy 0, policy_version 536233 (0.0008) [2023-12-27 09:20:58,146][20347] Updated weights for policy 0, policy_version 536243 (0.0006) [2023-12-27 09:20:58,207][20347] Updated weights for policy 0, policy_version 536253 (0.0008) [2023-12-27 09:20:58,426][20381] Updated weights for policy 1, policy_version 535831 (0.0008) [2023-12-27 09:20:58,482][20381] Updated weights for policy 1, policy_version 535841 (0.0007) [2023-12-27 09:20:58,542][20381] Updated weights for policy 1, policy_version 535851 (0.0007) [2023-12-27 09:20:59,051][20347] Updated weights for policy 0, policy_version 536263 (0.0008) [2023-12-27 09:20:59,111][20347] Updated weights for policy 0, policy_version 536273 (0.0008) [2023-12-27 09:20:59,175][20347] Updated weights for policy 0, policy_version 536283 (0.0007) [2023-12-27 09:20:59,310][20381] Updated weights for policy 1, policy_version 535861 (0.0007) [2023-12-27 09:20:59,376][20381] Updated weights for policy 1, policy_version 535871 (0.0007) [2023-12-27 09:20:59,424][20381] Updated weights for policy 1, policy_version 535881 (0.0006) [2023-12-27 09:20:59,952][20347] Updated weights for policy 0, policy_version 536293 (0.0007) [2023-12-27 09:21:00,000][20347] Updated weights for policy 0, policy_version 536303 (0.0007) [2023-12-27 09:21:00,035][20381] Updated weights for policy 1, policy_version 535891 (0.0005) [2023-12-27 09:21:00,056][20347] Updated weights for policy 0, policy_version 536313 (0.0006) [2023-12-27 09:21:00,088][20381] Updated weights for policy 1, policy_version 535901 (0.0006) [2023-12-27 09:21:00,140][20381] Updated weights for policy 1, policy_version 535911 (0.0007) [2023-12-27 09:21:00,671][20347] Updated weights for policy 0, policy_version 536323 (0.0007) [2023-12-27 09:21:00,714][20291] KL-divergence is very high: 279.7999 [2023-12-27 09:21:00,731][20347] Updated weights for policy 0, policy_version 536333 (0.0005) [2023-12-27 09:21:00,785][20381] Updated weights for policy 1, policy_version 535921 (0.0007) [2023-12-27 09:21:00,789][20347] Updated weights for policy 0, policy_version 536343 (0.0005) [2023-12-27 09:21:00,841][20381] Updated weights for policy 1, policy_version 535931 (0.0005) [2023-12-27 09:21:00,895][20381] Updated weights for policy 1, policy_version 535941 (0.0005) [2023-12-27 09:21:00,950][20381] Updated weights for policy 1, policy_version 535951 (0.0005) [2023-12-27 09:21:01,469][20347] Updated weights for policy 0, policy_version 536353 (0.0007) [2023-12-27 09:21:01,523][20347] Updated weights for policy 0, policy_version 536363 (0.0007) [2023-12-27 09:21:01,566][20381] Updated weights for policy 1, policy_version 535961 (0.0006) [2023-12-27 09:21:01,581][20347] Updated weights for policy 0, policy_version 536373 (0.0008) [2023-12-27 09:21:01,627][20381] Updated weights for policy 1, policy_version 535971 (0.0010) [2023-12-27 09:21:01,641][20347] Updated weights for policy 0, policy_version 536383 (0.0009) [2023-12-27 09:21:01,684][20381] Updated weights for policy 1, policy_version 535981 (0.0009) [2023-12-27 09:21:02,333][20347] Updated weights for policy 0, policy_version 536393 (0.0006) [2023-12-27 09:21:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 274563072. Throughput: 0: 10375.2, 1: 10464.5. Samples: 274579200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:02,363][19170] Avg episode reward: [(0, '972.580'), (1, '426.410')] [2023-12-27 09:21:02,376][20381] Updated weights for policy 1, policy_version 535991 (0.0008) [2023-12-27 09:21:02,419][20347] Updated weights for policy 0, policy_version 536403 (0.0008) [2023-12-27 09:21:02,421][20381] Updated weights for policy 1, policy_version 536001 (0.0007) [2023-12-27 09:21:02,466][20381] Updated weights for policy 1, policy_version 536011 (0.0005) [2023-12-27 09:21:02,471][20347] Updated weights for policy 0, policy_version 536413 (0.0006) [2023-12-27 09:21:03,037][20381] Updated weights for policy 1, policy_version 536021 (0.0008) [2023-12-27 09:21:03,082][20381] Updated weights for policy 1, policy_version 536031 (0.0010) [2023-12-27 09:21:03,130][20381] Updated weights for policy 1, policy_version 536041 (0.0010) [2023-12-27 09:21:03,176][20347] Updated weights for policy 0, policy_version 536423 (0.0008) [2023-12-27 09:21:03,238][20347] Updated weights for policy 0, policy_version 536433 (0.0009) [2023-12-27 09:21:03,301][20347] Updated weights for policy 0, policy_version 536443 (0.0008) [2023-12-27 09:21:03,730][20381] Updated weights for policy 1, policy_version 536051 (0.0006) [2023-12-27 09:21:03,782][20381] Updated weights for policy 1, policy_version 536061 (0.0010) [2023-12-27 09:21:03,833][20381] Updated weights for policy 1, policy_version 536071 (0.0010) [2023-12-27 09:21:04,027][20347] Updated weights for policy 0, policy_version 536453 (0.0009) [2023-12-27 09:21:04,089][20347] Updated weights for policy 0, policy_version 536463 (0.0008) [2023-12-27 09:21:04,151][20347] Updated weights for policy 0, policy_version 536473 (0.0008) [2023-12-27 09:21:04,449][20381] Updated weights for policy 1, policy_version 536081 (0.0010) [2023-12-27 09:21:04,514][20381] Updated weights for policy 1, policy_version 536091 (0.0011) [2023-12-27 09:21:04,566][20381] Updated weights for policy 1, policy_version 536101 (0.0011) [2023-12-27 09:21:04,625][20381] Updated weights for policy 1, policy_version 536111 (0.0010) [2023-12-27 09:21:04,878][20347] Updated weights for policy 0, policy_version 536483 (0.0007) [2023-12-27 09:21:04,941][20347] Updated weights for policy 0, policy_version 536493 (0.0008) [2023-12-27 09:21:04,997][20347] Updated weights for policy 0, policy_version 536503 (0.0007) [2023-12-27 09:21:05,264][20381] Updated weights for policy 1, policy_version 536121 (0.0010) [2023-12-27 09:21:05,316][20381] Updated weights for policy 1, policy_version 536131 (0.0010) [2023-12-27 09:21:05,361][20381] Updated weights for policy 1, policy_version 536141 (0.0010) [2023-12-27 09:21:05,601][20347] Updated weights for policy 0, policy_version 536513 (0.0007) [2023-12-27 09:21:05,655][20347] Updated weights for policy 0, policy_version 536523 (0.0005) [2023-12-27 09:21:05,709][20347] Updated weights for policy 0, policy_version 536533 (0.0005) [2023-12-27 09:21:05,760][20347] Updated weights for policy 0, policy_version 536543 (0.0005) [2023-12-27 09:21:06,019][20381] Updated weights for policy 1, policy_version 536151 (0.0010) [2023-12-27 09:21:06,064][20381] Updated weights for policy 1, policy_version 536161 (0.0010) [2023-12-27 09:21:06,125][20381] Updated weights for policy 1, policy_version 536171 (0.0011) [2023-12-27 09:21:06,329][20347] Updated weights for policy 0, policy_version 536553 (0.0006) [2023-12-27 09:21:06,390][20347] Updated weights for policy 0, policy_version 536563 (0.0006) [2023-12-27 09:21:06,449][20347] Updated weights for policy 0, policy_version 536573 (0.0009) [2023-12-27 09:21:06,782][20381] Updated weights for policy 1, policy_version 536181 (0.0009) [2023-12-27 09:21:06,839][20381] Updated weights for policy 1, policy_version 536191 (0.0009) [2023-12-27 09:21:06,900][20381] Updated weights for policy 1, policy_version 536201 (0.0005) [2023-12-27 09:21:07,099][20347] Updated weights for policy 0, policy_version 536583 (0.0010) [2023-12-27 09:21:07,154][20347] Updated weights for policy 0, policy_version 536593 (0.0010) [2023-12-27 09:21:07,195][20291] KL-divergence is very high: 579.8469 [2023-12-27 09:21:07,200][20291] KL-divergence is very high: 399.6619 [2023-12-27 09:21:07,207][20347] Updated weights for policy 0, policy_version 536603 (0.0010) [2023-12-27 09:21:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 274677760. Throughput: 0: 10339.1, 1: 10495.9. Samples: 274641920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:07,364][19170] Avg episode reward: [(0, '962.840'), (1, '438.630')] [2023-12-27 09:21:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000536208_137289728.pth... [2023-12-27 09:21:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000536608_137388032.pth... [2023-12-27 09:21:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000534992_136978432.pth [2023-12-27 09:21:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000535424_137084928.pth [2023-12-27 09:21:07,514][20381] Updated weights for policy 1, policy_version 536211 (0.0006) [2023-12-27 09:21:07,568][20381] Updated weights for policy 1, policy_version 536221 (0.0010) [2023-12-27 09:21:07,627][20381] Updated weights for policy 1, policy_version 536231 (0.0010) [2023-12-27 09:21:07,883][20347] Updated weights for policy 0, policy_version 536613 (0.0008) [2023-12-27 09:21:07,935][20347] Updated weights for policy 0, policy_version 536623 (0.0005) [2023-12-27 09:21:07,986][20347] Updated weights for policy 0, policy_version 536633 (0.0005) [2023-12-27 09:21:08,233][20381] Updated weights for policy 1, policy_version 536241 (0.0010) [2023-12-27 09:21:08,284][20381] Updated weights for policy 1, policy_version 536251 (0.0010) [2023-12-27 09:21:08,335][20381] Updated weights for policy 1, policy_version 536261 (0.0010) [2023-12-27 09:21:08,400][20381] Updated weights for policy 1, policy_version 536271 (0.0010) [2023-12-27 09:21:08,695][20347] Updated weights for policy 0, policy_version 536643 (0.0007) [2023-12-27 09:21:08,751][20347] Updated weights for policy 0, policy_version 536653 (0.0007) [2023-12-27 09:21:08,804][20347] Updated weights for policy 0, policy_version 536663 (0.0007) [2023-12-27 09:21:09,069][20381] Updated weights for policy 1, policy_version 536281 (0.0008) [2023-12-27 09:21:09,128][20381] Updated weights for policy 1, policy_version 536291 (0.0008) [2023-12-27 09:21:09,183][20381] Updated weights for policy 1, policy_version 536301 (0.0010) [2023-12-27 09:21:09,457][20347] Updated weights for policy 0, policy_version 536673 (0.0008) [2023-12-27 09:21:09,519][20347] Updated weights for policy 0, policy_version 536683 (0.0006) [2023-12-27 09:21:09,579][20347] Updated weights for policy 0, policy_version 536693 (0.0008) [2023-12-27 09:21:09,642][20347] Updated weights for policy 0, policy_version 536703 (0.0007) [2023-12-27 09:21:09,962][20381] Updated weights for policy 1, policy_version 536311 (0.0008) [2023-12-27 09:21:10,020][20381] Updated weights for policy 1, policy_version 536321 (0.0008) [2023-12-27 09:21:10,084][20381] Updated weights for policy 1, policy_version 536331 (0.0007) [2023-12-27 09:21:10,297][20347] Updated weights for policy 0, policy_version 536713 (0.0007) [2023-12-27 09:21:10,349][20347] Updated weights for policy 0, policy_version 536723 (0.0007) [2023-12-27 09:21:10,402][20347] Updated weights for policy 0, policy_version 536733 (0.0007) [2023-12-27 09:21:10,680][20381] Updated weights for policy 1, policy_version 536341 (0.0008) [2023-12-27 09:21:10,740][20381] Updated weights for policy 1, policy_version 536351 (0.0011) [2023-12-27 09:21:10,806][20381] Updated weights for policy 1, policy_version 536361 (0.0011) [2023-12-27 09:21:11,180][20347] Updated weights for policy 0, policy_version 536743 (0.0008) [2023-12-27 09:21:11,238][20347] Updated weights for policy 0, policy_version 536753 (0.0008) [2023-12-27 09:21:11,294][20347] Updated weights for policy 0, policy_version 536763 (0.0007) [2023-12-27 09:21:11,510][20381] Updated weights for policy 1, policy_version 536371 (0.0011) [2023-12-27 09:21:11,565][20381] Updated weights for policy 1, policy_version 536381 (0.0010) [2023-12-27 09:21:11,619][20381] Updated weights for policy 1, policy_version 536391 (0.0011) [2023-12-27 09:21:12,049][20347] Updated weights for policy 0, policy_version 536773 (0.0008) [2023-12-27 09:21:12,104][20347] Updated weights for policy 0, policy_version 536783 (0.0008) [2023-12-27 09:21:12,129][20291] KL-divergence is very high: 510.4163 [2023-12-27 09:21:12,150][20347] Updated weights for policy 0, policy_version 536793 (0.0009) [2023-12-27 09:21:12,276][20381] Updated weights for policy 1, policy_version 536401 (0.0011) [2023-12-27 09:21:12,341][20381] Updated weights for policy 1, policy_version 536411 (0.0011) [2023-12-27 09:21:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 274776064. Throughput: 0: 10348.5, 1: 10503.8. Samples: 274768168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:12,363][19170] Avg episode reward: [(0, '934.260'), (1, '715.610')] [2023-12-27 09:21:12,398][20381] Updated weights for policy 1, policy_version 536421 (0.0011) [2023-12-27 09:21:12,456][20381] Updated weights for policy 1, policy_version 536431 (0.0011) [2023-12-27 09:21:12,802][20347] Updated weights for policy 0, policy_version 536803 (0.0008) [2023-12-27 09:21:12,865][20347] Updated weights for policy 0, policy_version 536813 (0.0007) [2023-12-27 09:21:12,914][20347] Updated weights for policy 0, policy_version 536823 (0.0008) [2023-12-27 09:21:13,131][20381] Updated weights for policy 1, policy_version 536441 (0.0006) [2023-12-27 09:21:13,191][20381] Updated weights for policy 1, policy_version 536451 (0.0010) [2023-12-27 09:21:13,253][20381] Updated weights for policy 1, policy_version 536461 (0.0011) [2023-12-27 09:21:13,572][20347] Updated weights for policy 0, policy_version 536833 (0.0007) [2023-12-27 09:21:13,628][20347] Updated weights for policy 0, policy_version 536843 (0.0007) [2023-12-27 09:21:13,683][20347] Updated weights for policy 0, policy_version 536853 (0.0007) [2023-12-27 09:21:13,745][20347] Updated weights for policy 0, policy_version 536863 (0.0007) [2023-12-27 09:21:13,885][20381] Updated weights for policy 1, policy_version 536471 (0.0011) [2023-12-27 09:21:13,934][20381] Updated weights for policy 1, policy_version 536481 (0.0010) [2023-12-27 09:21:13,991][20381] Updated weights for policy 1, policy_version 536491 (0.0010) [2023-12-27 09:21:14,434][20347] Updated weights for policy 0, policy_version 536873 (0.0006) [2023-12-27 09:21:14,487][20347] Updated weights for policy 0, policy_version 536883 (0.0005) [2023-12-27 09:21:14,546][20347] Updated weights for policy 0, policy_version 536893 (0.0005) [2023-12-27 09:21:14,610][20381] Updated weights for policy 1, policy_version 536501 (0.0005) [2023-12-27 09:21:14,662][20381] Updated weights for policy 1, policy_version 536511 (0.0005) [2023-12-27 09:21:14,718][20381] Updated weights for policy 1, policy_version 536521 (0.0005) [2023-12-27 09:21:15,086][20347] Updated weights for policy 0, policy_version 536903 (0.0006) [2023-12-27 09:21:15,125][20291] KL-divergence is very high: 105.9101 [2023-12-27 09:21:15,147][20347] Updated weights for policy 0, policy_version 536913 (0.0005) [2023-12-27 09:21:15,207][20347] Updated weights for policy 0, policy_version 536923 (0.0010) [2023-12-27 09:21:15,355][20381] Updated weights for policy 1, policy_version 536531 (0.0005) [2023-12-27 09:21:15,417][20381] Updated weights for policy 1, policy_version 536541 (0.0005) [2023-12-27 09:21:15,475][20381] Updated weights for policy 1, policy_version 536551 (0.0006) [2023-12-27 09:21:15,861][20347] Updated weights for policy 0, policy_version 536933 (0.0009) [2023-12-27 09:21:15,911][20347] Updated weights for policy 0, policy_version 536943 (0.0007) [2023-12-27 09:21:15,959][20347] Updated weights for policy 0, policy_version 536953 (0.0007) [2023-12-27 09:21:15,960][20291] KL-divergence is very high: 205.1816 [2023-12-27 09:21:16,126][20381] Updated weights for policy 1, policy_version 536561 (0.0011) [2023-12-27 09:21:16,175][20381] Updated weights for policy 1, policy_version 536571 (0.0010) [2023-12-27 09:21:16,226][20381] Updated weights for policy 1, policy_version 536581 (0.0011) [2023-12-27 09:21:16,278][20381] Updated weights for policy 1, policy_version 536591 (0.0010) [2023-12-27 09:21:16,634][20347] Updated weights for policy 0, policy_version 536963 (0.0008) [2023-12-27 09:21:16,688][20347] Updated weights for policy 0, policy_version 536973 (0.0007) [2023-12-27 09:21:16,740][20347] Updated weights for policy 0, policy_version 536983 (0.0007) [2023-12-27 09:21:16,954][20381] Updated weights for policy 1, policy_version 536601 (0.0010) [2023-12-27 09:21:16,999][20381] Updated weights for policy 1, policy_version 536611 (0.0010) [2023-12-27 09:21:17,049][20381] Updated weights for policy 1, policy_version 536621 (0.0011) [2023-12-27 09:21:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 274882560. Throughput: 0: 10296.7, 1: 10501.0. Samples: 274893692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:17,363][19170] Avg episode reward: [(0, '895.780'), (1, '843.180')] [2023-12-27 09:21:17,445][20347] Updated weights for policy 0, policy_version 536993 (0.0007) [2023-12-27 09:21:17,503][20347] Updated weights for policy 0, policy_version 537003 (0.0007) [2023-12-27 09:21:17,569][20347] Updated weights for policy 0, policy_version 537013 (0.0006) [2023-12-27 09:21:17,633][20347] Updated weights for policy 0, policy_version 537023 (0.0008) [2023-12-27 09:21:17,727][20381] Updated weights for policy 1, policy_version 536631 (0.0007) [2023-12-27 09:21:17,784][20381] Updated weights for policy 1, policy_version 536641 (0.0006) [2023-12-27 09:21:17,848][20381] Updated weights for policy 1, policy_version 536651 (0.0007) [2023-12-27 09:21:18,334][20347] Updated weights for policy 0, policy_version 537033 (0.0008) [2023-12-27 09:21:18,395][20347] Updated weights for policy 0, policy_version 537043 (0.0007) [2023-12-27 09:21:18,446][20347] Updated weights for policy 0, policy_version 537053 (0.0007) [2023-12-27 09:21:18,511][20381] Updated weights for policy 1, policy_version 536661 (0.0008) [2023-12-27 09:21:18,567][20381] Updated weights for policy 1, policy_version 536671 (0.0008) [2023-12-27 09:21:18,620][20381] Updated weights for policy 1, policy_version 536681 (0.0008) [2023-12-27 09:21:19,140][20347] Updated weights for policy 0, policy_version 537063 (0.0007) [2023-12-27 09:21:19,197][20347] Updated weights for policy 0, policy_version 537073 (0.0007) [2023-12-27 09:21:19,260][20347] Updated weights for policy 0, policy_version 537083 (0.0007) [2023-12-27 09:21:19,348][20381] Updated weights for policy 1, policy_version 536691 (0.0008) [2023-12-27 09:21:19,414][20381] Updated weights for policy 1, policy_version 536701 (0.0008) [2023-12-27 09:21:19,463][20381] Updated weights for policy 1, policy_version 536711 (0.0007) [2023-12-27 09:21:19,885][20347] Updated weights for policy 0, policy_version 537093 (0.0005) [2023-12-27 09:21:19,945][20347] Updated weights for policy 0, policy_version 537103 (0.0007) [2023-12-27 09:21:20,005][20347] Updated weights for policy 0, policy_version 537113 (0.0008) [2023-12-27 09:21:20,214][20381] Updated weights for policy 1, policy_version 536721 (0.0007) [2023-12-27 09:21:20,273][20381] Updated weights for policy 1, policy_version 536731 (0.0008) [2023-12-27 09:21:20,327][20381] Updated weights for policy 1, policy_version 536741 (0.0008) [2023-12-27 09:21:20,388][20381] Updated weights for policy 1, policy_version 536751 (0.0007) [2023-12-27 09:21:20,668][20347] Updated weights for policy 0, policy_version 537123 (0.0006) [2023-12-27 09:21:20,720][20347] Updated weights for policy 0, policy_version 537133 (0.0008) [2023-12-27 09:21:20,726][20291] KL-divergence is very high: 114.3597 [2023-12-27 09:21:20,767][20347] Updated weights for policy 0, policy_version 537143 (0.0009) [2023-12-27 09:21:21,063][20381] Updated weights for policy 1, policy_version 536761 (0.0008) [2023-12-27 09:21:21,130][20381] Updated weights for policy 1, policy_version 536771 (0.0008) [2023-12-27 09:21:21,203][20381] Updated weights for policy 1, policy_version 536781 (0.0008) [2023-12-27 09:21:21,527][20347] Updated weights for policy 0, policy_version 537153 (0.0010) [2023-12-27 09:21:21,546][20291] KL-divergence is very high: 119.1294 [2023-12-27 09:21:21,589][20347] Updated weights for policy 0, policy_version 537163 (0.0008) [2023-12-27 09:21:21,648][20347] Updated weights for policy 0, policy_version 537173 (0.0009) [2023-12-27 09:21:21,711][20347] Updated weights for policy 0, policy_version 537183 (0.0007) [2023-12-27 09:21:21,906][20381] Updated weights for policy 1, policy_version 536791 (0.0008) [2023-12-27 09:21:21,961][20381] Updated weights for policy 1, policy_version 536801 (0.0006) [2023-12-27 09:21:22,020][20381] Updated weights for policy 1, policy_version 536811 (0.0008) [2023-12-27 09:21:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 274980864. Throughput: 0: 10251.9, 1: 10429.9. Samples: 274953724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:22,363][19170] Avg episode reward: [(0, '802.020'), (1, '796.630')] [2023-12-27 09:21:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000536816_137445376.pth... [2023-12-27 09:21:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000537184_137535488.pth... [2023-12-27 09:21:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000535600_137134080.pth [2023-12-27 09:21:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000536000_137232384.pth [2023-12-27 09:21:22,393][20291] KL-divergence is very high: 155.6874 [2023-12-27 09:21:22,441][20347] Updated weights for policy 0, policy_version 537193 (0.0007) [2023-12-27 09:21:22,489][20347] Updated weights for policy 0, policy_version 537203 (0.0007) [2023-12-27 09:21:22,544][20347] Updated weights for policy 0, policy_version 537213 (0.0007) [2023-12-27 09:21:22,663][20381] Updated weights for policy 1, policy_version 536821 (0.0008) [2023-12-27 09:21:22,714][20381] Updated weights for policy 1, policy_version 536831 (0.0011) [2023-12-27 09:21:22,767][20381] Updated weights for policy 1, policy_version 536841 (0.0011) [2023-12-27 09:21:23,248][20347] Updated weights for policy 0, policy_version 537223 (0.0006) [2023-12-27 09:21:23,299][20347] Updated weights for policy 0, policy_version 537233 (0.0007) [2023-12-27 09:21:23,345][20347] Updated weights for policy 0, policy_version 537243 (0.0007) [2023-12-27 09:21:23,485][20381] Updated weights for policy 1, policy_version 536851 (0.0010) [2023-12-27 09:21:23,540][20381] Updated weights for policy 1, policy_version 536861 (0.0010) [2023-12-27 09:21:23,594][20381] Updated weights for policy 1, policy_version 536871 (0.0010) [2023-12-27 09:21:23,986][20347] Updated weights for policy 0, policy_version 537253 (0.0007) [2023-12-27 09:21:24,044][20347] Updated weights for policy 0, policy_version 537263 (0.0006) [2023-12-27 09:21:24,099][20347] Updated weights for policy 0, policy_version 537273 (0.0007) [2023-12-27 09:21:24,285][20381] Updated weights for policy 1, policy_version 536881 (0.0010) [2023-12-27 09:21:24,339][20381] Updated weights for policy 1, policy_version 536891 (0.0007) [2023-12-27 09:21:24,399][20381] Updated weights for policy 1, policy_version 536901 (0.0007) [2023-12-27 09:21:24,448][20381] Updated weights for policy 1, policy_version 536911 (0.0006) [2023-12-27 09:21:24,797][20347] Updated weights for policy 0, policy_version 537283 (0.0008) [2023-12-27 09:21:24,852][20347] Updated weights for policy 0, policy_version 537293 (0.0007) [2023-12-27 09:21:24,904][20347] Updated weights for policy 0, policy_version 537303 (0.0006) [2023-12-27 09:21:25,032][20381] Updated weights for policy 1, policy_version 536921 (0.0010) [2023-12-27 09:21:25,086][20381] Updated weights for policy 1, policy_version 536931 (0.0010) [2023-12-27 09:21:25,138][20381] Updated weights for policy 1, policy_version 536941 (0.0010) [2023-12-27 09:21:25,616][20347] Updated weights for policy 0, policy_version 537313 (0.0008) [2023-12-27 09:21:25,674][20347] Updated weights for policy 0, policy_version 537323 (0.0005) [2023-12-27 09:21:25,738][20347] Updated weights for policy 0, policy_version 537333 (0.0006) [2023-12-27 09:21:25,799][20347] Updated weights for policy 0, policy_version 537343 (0.0005) [2023-12-27 09:21:25,826][20381] Updated weights for policy 1, policy_version 536951 (0.0010) [2023-12-27 09:21:25,879][20381] Updated weights for policy 1, policy_version 536961 (0.0009) [2023-12-27 09:21:25,930][20381] Updated weights for policy 1, policy_version 536971 (0.0005) [2023-12-27 09:21:26,376][20347] Updated weights for policy 0, policy_version 537353 (0.0007) [2023-12-27 09:21:26,429][20347] Updated weights for policy 0, policy_version 537363 (0.0007) [2023-12-27 09:21:26,479][20347] Updated weights for policy 0, policy_version 537373 (0.0007) [2023-12-27 09:21:26,554][20381] Updated weights for policy 1, policy_version 536981 (0.0005) [2023-12-27 09:21:26,613][20381] Updated weights for policy 1, policy_version 536991 (0.0007) [2023-12-27 09:21:26,668][20381] Updated weights for policy 1, policy_version 537001 (0.0007) [2023-12-27 09:21:27,168][20347] Updated weights for policy 0, policy_version 537383 (0.0007) [2023-12-27 09:21:27,220][20347] Updated weights for policy 0, policy_version 537393 (0.0007) [2023-12-27 09:21:27,236][20381] Updated weights for policy 1, policy_version 537011 (0.0006) [2023-12-27 09:21:27,271][20347] Updated weights for policy 0, policy_version 537403 (0.0006) [2023-12-27 09:21:27,281][20291] KL-divergence is very high: 196.6396 [2023-12-27 09:21:27,291][20381] Updated weights for policy 1, policy_version 537021 (0.0007) [2023-12-27 09:21:27,344][20381] Updated weights for policy 1, policy_version 537031 (0.0007) [2023-12-27 09:21:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 275087360. Throughput: 0: 10238.4, 1: 10420.7. Samples: 275076588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:27,363][19170] Avg episode reward: [(0, '831.430'), (1, '776.850')] [2023-12-27 09:21:27,929][20381] Updated weights for policy 1, policy_version 537041 (0.0006) [2023-12-27 09:21:27,986][20347] Updated weights for policy 0, policy_version 537413 (0.0006) [2023-12-27 09:21:27,988][20381] Updated weights for policy 1, policy_version 537051 (0.0006) [2023-12-27 09:21:28,038][20347] Updated weights for policy 0, policy_version 537423 (0.0006) [2023-12-27 09:21:28,043][20381] Updated weights for policy 1, policy_version 537061 (0.0006) [2023-12-27 09:21:28,083][20291] KL-divergence is very high: 115.4149 [2023-12-27 09:21:28,088][20347] Updated weights for policy 0, policy_version 537433 (0.0005) [2023-12-27 09:21:28,097][20381] Updated weights for policy 1, policy_version 537071 (0.0006) [2023-12-27 09:21:28,706][20381] Updated weights for policy 1, policy_version 537081 (0.0005) [2023-12-27 09:21:28,753][20381] Updated weights for policy 1, policy_version 537091 (0.0005) [2023-12-27 09:21:28,807][20381] Updated weights for policy 1, policy_version 537101 (0.0008) [2023-12-27 09:21:28,808][20291] KL-divergence is very high: 188.6662 [2023-12-27 09:21:28,814][20347] Updated weights for policy 0, policy_version 537443 (0.0007) [2023-12-27 09:21:28,867][20347] Updated weights for policy 0, policy_version 537453 (0.0007) [2023-12-27 09:21:28,918][20347] Updated weights for policy 0, policy_version 537463 (0.0007) [2023-12-27 09:21:28,930][20291] KL-divergence is very high: 208.4555 [2023-12-27 09:21:29,507][20381] Updated weights for policy 1, policy_version 537111 (0.0008) [2023-12-27 09:21:29,563][20347] Updated weights for policy 0, policy_version 537473 (0.0008) [2023-12-27 09:21:29,566][20381] Updated weights for policy 1, policy_version 537121 (0.0007) [2023-12-27 09:21:29,622][20381] Updated weights for policy 1, policy_version 537131 (0.0006) [2023-12-27 09:21:29,623][20347] Updated weights for policy 0, policy_version 537483 (0.0006) [2023-12-27 09:21:29,679][20347] Updated weights for policy 0, policy_version 537493 (0.0007) [2023-12-27 09:21:29,737][20347] Updated weights for policy 0, policy_version 537503 (0.0007) [2023-12-27 09:21:30,315][20381] Updated weights for policy 1, policy_version 537141 (0.0008) [2023-12-27 09:21:30,375][20381] Updated weights for policy 1, policy_version 537151 (0.0008) [2023-12-27 09:21:30,426][20347] Updated weights for policy 0, policy_version 537513 (0.0005) [2023-12-27 09:21:30,428][20381] Updated weights for policy 1, policy_version 537161 (0.0007) [2023-12-27 09:21:30,491][20347] Updated weights for policy 0, policy_version 537523 (0.0005) [2023-12-27 09:21:30,537][20347] Updated weights for policy 0, policy_version 537533 (0.0005) [2023-12-27 09:21:31,143][20381] Updated weights for policy 1, policy_version 537171 (0.0008) [2023-12-27 09:21:31,158][20347] Updated weights for policy 0, policy_version 537543 (0.0006) [2023-12-27 09:21:31,199][20381] Updated weights for policy 1, policy_version 537181 (0.0007) [2023-12-27 09:21:31,215][20347] Updated weights for policy 0, policy_version 537553 (0.0005) [2023-12-27 09:21:31,257][20381] Updated weights for policy 1, policy_version 537191 (0.0008) [2023-12-27 09:21:31,273][20347] Updated weights for policy 0, policy_version 537563 (0.0007) [2023-12-27 09:21:31,908][20347] Updated weights for policy 0, policy_version 537573 (0.0006) [2023-12-27 09:21:31,960][20347] Updated weights for policy 0, policy_version 537583 (0.0006) [2023-12-27 09:21:31,989][20381] Updated weights for policy 1, policy_version 537201 (0.0007) [2023-12-27 09:21:32,020][20347] Updated weights for policy 0, policy_version 537593 (0.0005) [2023-12-27 09:21:32,042][20381] Updated weights for policy 1, policy_version 537211 (0.0007) [2023-12-27 09:21:32,093][20381] Updated weights for policy 1, policy_version 537221 (0.0009) [2023-12-27 09:21:32,143][20381] Updated weights for policy 1, policy_version 537231 (0.0007) [2023-12-27 09:21:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 275193856. Throughput: 0: 10224.9, 1: 10428.3. Samples: 275201692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:32,363][19170] Avg episode reward: [(0, '829.720'), (1, '747.100')] [2023-12-27 09:21:32,707][20381] Updated weights for policy 1, policy_version 537241 (0.0007) [2023-12-27 09:21:32,728][20347] Updated weights for policy 0, policy_version 537603 (0.0006) [2023-12-27 09:21:32,749][20381] Updated weights for policy 1, policy_version 537251 (0.0006) [2023-12-27 09:21:32,785][20347] Updated weights for policy 0, policy_version 537613 (0.0006) [2023-12-27 09:21:32,797][20381] Updated weights for policy 1, policy_version 537261 (0.0006) [2023-12-27 09:21:32,838][20347] Updated weights for policy 0, policy_version 537623 (0.0006) [2023-12-27 09:21:33,412][20381] Updated weights for policy 1, policy_version 537271 (0.0007) [2023-12-27 09:21:33,465][20347] Updated weights for policy 0, policy_version 537633 (0.0009) [2023-12-27 09:21:33,467][20381] Updated weights for policy 1, policy_version 537281 (0.0008) [2023-12-27 09:21:33,523][20347] Updated weights for policy 0, policy_version 537643 (0.0005) [2023-12-27 09:21:33,523][20381] Updated weights for policy 1, policy_version 537291 (0.0007) [2023-12-27 09:21:33,578][20347] Updated weights for policy 0, policy_version 537653 (0.0005) [2023-12-27 09:21:33,635][20347] Updated weights for policy 0, policy_version 537663 (0.0005) [2023-12-27 09:21:34,155][20381] Updated weights for policy 1, policy_version 537301 (0.0008) [2023-12-27 09:21:34,211][20381] Updated weights for policy 1, policy_version 537311 (0.0008) [2023-12-27 09:21:34,236][20347] Updated weights for policy 0, policy_version 537673 (0.0005) [2023-12-27 09:21:34,259][20381] Updated weights for policy 1, policy_version 537321 (0.0007) [2023-12-27 09:21:34,292][20347] Updated weights for policy 0, policy_version 537683 (0.0007) [2023-12-27 09:21:34,352][20347] Updated weights for policy 0, policy_version 537693 (0.0006) [2023-12-27 09:21:34,939][20347] Updated weights for policy 0, policy_version 537703 (0.0007) [2023-12-27 09:21:34,987][20347] Updated weights for policy 0, policy_version 537713 (0.0007) [2023-12-27 09:21:34,997][20381] Updated weights for policy 1, policy_version 537331 (0.0007) [2023-12-27 09:21:35,038][20347] Updated weights for policy 0, policy_version 537723 (0.0007) [2023-12-27 09:21:35,039][20381] Updated weights for policy 1, policy_version 537341 (0.0006) [2023-12-27 09:21:35,087][20381] Updated weights for policy 1, policy_version 537351 (0.0009) [2023-12-27 09:21:35,707][20381] Updated weights for policy 1, policy_version 537361 (0.0010) [2023-12-27 09:21:35,755][20347] Updated weights for policy 0, policy_version 537733 (0.0009) [2023-12-27 09:21:35,759][20381] Updated weights for policy 1, policy_version 537371 (0.0011) [2023-12-27 09:21:35,806][20347] Updated weights for policy 0, policy_version 537743 (0.0010) [2023-12-27 09:21:35,821][20381] Updated weights for policy 1, policy_version 537381 (0.0011) [2023-12-27 09:21:35,857][20347] Updated weights for policy 0, policy_version 537753 (0.0010) [2023-12-27 09:21:35,879][20381] Updated weights for policy 1, policy_version 537391 (0.0010) [2023-12-27 09:21:36,500][20347] Updated weights for policy 0, policy_version 537763 (0.0009) [2023-12-27 09:21:36,562][20347] Updated weights for policy 0, policy_version 537773 (0.0010) [2023-12-27 09:21:36,585][20381] Updated weights for policy 1, policy_version 537401 (0.0011) [2023-12-27 09:21:36,622][20347] Updated weights for policy 0, policy_version 537783 (0.0010) [2023-12-27 09:21:36,642][20381] Updated weights for policy 1, policy_version 537411 (0.0011) [2023-12-27 09:21:36,692][20381] Updated weights for policy 1, policy_version 537421 (0.0011) [2023-12-27 09:21:37,271][20347] Updated weights for policy 0, policy_version 537793 (0.0010) [2023-12-27 09:21:37,305][20381] Updated weights for policy 1, policy_version 537431 (0.0011) [2023-12-27 09:21:37,322][20347] Updated weights for policy 0, policy_version 537803 (0.0010) [2023-12-27 09:21:37,361][20381] Updated weights for policy 1, policy_version 537441 (0.0010) [2023-12-27 09:21:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 275292160. Throughput: 0: 10246.4, 1: 10468.1. Samples: 275266648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:21:37,363][19170] Avg episode reward: [(0, '826.520'), (1, '669.450')] [2023-12-27 09:21:37,372][20347] Updated weights for policy 0, policy_version 537813 (0.0010) [2023-12-27 09:21:37,419][20381] Updated weights for policy 1, policy_version 537451 (0.0006) [2023-12-27 09:21:37,422][20347] Updated weights for policy 0, policy_version 537823 (0.0011) [2023-12-27 09:21:37,425][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000537824_137699328.pth... [2023-12-27 09:21:37,429][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000536608_137388032.pth [2023-12-27 09:21:37,446][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000537456_137609216.pth... [2023-12-27 09:21:37,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000536208_137289728.pth [2023-12-27 09:21:38,081][20381] Updated weights for policy 1, policy_version 537461 (0.0008) [2023-12-27 09:21:38,081][20347] Updated weights for policy 0, policy_version 537833 (0.0006) [2023-12-27 09:21:38,141][20347] Updated weights for policy 0, policy_version 537843 (0.0008) [2023-12-27 09:21:38,143][20381] Updated weights for policy 1, policy_version 537471 (0.0010) [2023-12-27 09:21:38,192][20347] Updated weights for policy 0, policy_version 537853 (0.0010) [2023-12-27 09:21:38,207][20381] Updated weights for policy 1, policy_version 537481 (0.0011) [2023-12-27 09:21:38,851][20347] Updated weights for policy 0, policy_version 537863 (0.0010) [2023-12-27 09:21:38,885][20381] Updated weights for policy 1, policy_version 537491 (0.0009) [2023-12-27 09:21:38,906][20347] Updated weights for policy 0, policy_version 537873 (0.0010) [2023-12-27 09:21:38,944][20381] Updated weights for policy 1, policy_version 537501 (0.0008) [2023-12-27 09:21:38,957][20347] Updated weights for policy 0, policy_version 537883 (0.0008) [2023-12-27 09:21:38,999][20381] Updated weights for policy 1, policy_version 537511 (0.0009) [2023-12-27 09:21:39,656][20347] Updated weights for policy 0, policy_version 537893 (0.0009) [2023-12-27 09:21:39,712][20347] Updated weights for policy 0, policy_version 537903 (0.0010) [2023-12-27 09:21:39,720][20381] Updated weights for policy 1, policy_version 537521 (0.0006) [2023-12-27 09:21:39,771][20347] Updated weights for policy 0, policy_version 537913 (0.0011) [2023-12-27 09:21:39,778][20381] Updated weights for policy 1, policy_version 537531 (0.0006) [2023-12-27 09:21:39,835][20381] Updated weights for policy 1, policy_version 537541 (0.0006) [2023-12-27 09:21:39,896][20381] Updated weights for policy 1, policy_version 537551 (0.0007) [2023-12-27 09:21:40,473][20347] Updated weights for policy 0, policy_version 537923 (0.0010) [2023-12-27 09:21:40,526][20347] Updated weights for policy 0, policy_version 537933 (0.0006) [2023-12-27 09:21:40,591][20347] Updated weights for policy 0, policy_version 537943 (0.0005) [2023-12-27 09:21:40,611][20381] Updated weights for policy 1, policy_version 537561 (0.0007) [2023-12-27 09:21:40,665][20381] Updated weights for policy 1, policy_version 537571 (0.0009) [2023-12-27 09:21:40,713][20381] Updated weights for policy 1, policy_version 537581 (0.0007) [2023-12-27 09:21:41,271][20347] Updated weights for policy 0, policy_version 537953 (0.0009) [2023-12-27 09:21:41,327][20347] Updated weights for policy 0, policy_version 537963 (0.0008) [2023-12-27 09:21:41,389][20347] Updated weights for policy 0, policy_version 537973 (0.0008) [2023-12-27 09:21:41,427][20381] Updated weights for policy 1, policy_version 537591 (0.0007) [2023-12-27 09:21:41,441][20347] Updated weights for policy 0, policy_version 537983 (0.0008) [2023-12-27 09:21:41,486][20381] Updated weights for policy 1, policy_version 537601 (0.0007) [2023-12-27 09:21:41,537][20381] Updated weights for policy 1, policy_version 537611 (0.0006) [2023-12-27 09:21:42,205][20347] Updated weights for policy 0, policy_version 537993 (0.0009) [2023-12-27 09:21:42,231][20381] Updated weights for policy 1, policy_version 537621 (0.0005) [2023-12-27 09:21:42,255][20347] Updated weights for policy 0, policy_version 538003 (0.0007) [2023-12-27 09:21:42,290][20381] Updated weights for policy 1, policy_version 537631 (0.0006) [2023-12-27 09:21:42,315][20347] Updated weights for policy 0, policy_version 538013 (0.0009) [2023-12-27 09:21:42,352][20381] Updated weights for policy 1, policy_version 537641 (0.0007) [2023-12-27 09:21:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 275398656. Throughput: 0: 10276.5, 1: 10475.7. Samples: 275391364. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:21:42,363][19170] Avg episode reward: [(0, '816.850'), (1, '741.900')] [2023-12-27 09:21:42,984][20381] Updated weights for policy 1, policy_version 537651 (0.0007) [2023-12-27 09:21:43,041][20381] Updated weights for policy 1, policy_version 537661 (0.0005) [2023-12-27 09:21:43,064][20347] Updated weights for policy 0, policy_version 538023 (0.0007) [2023-12-27 09:21:43,099][20381] Updated weights for policy 1, policy_version 537671 (0.0005) [2023-12-27 09:21:43,114][20347] Updated weights for policy 0, policy_version 538033 (0.0008) [2023-12-27 09:21:43,168][20347] Updated weights for policy 0, policy_version 538043 (0.0008) [2023-12-27 09:21:43,710][20381] Updated weights for policy 1, policy_version 537681 (0.0006) [2023-12-27 09:21:43,765][20381] Updated weights for policy 1, policy_version 537691 (0.0011) [2023-12-27 09:21:43,823][20381] Updated weights for policy 1, policy_version 537701 (0.0010) [2023-12-27 09:21:43,875][20381] Updated weights for policy 1, policy_version 537711 (0.0010) [2023-12-27 09:21:43,899][20347] Updated weights for policy 0, policy_version 538053 (0.0008) [2023-12-27 09:21:43,959][20347] Updated weights for policy 0, policy_version 538063 (0.0007) [2023-12-27 09:21:44,007][20347] Updated weights for policy 0, policy_version 538073 (0.0007) [2023-12-27 09:21:44,531][20381] Updated weights for policy 1, policy_version 537721 (0.0011) [2023-12-27 09:21:44,579][20381] Updated weights for policy 1, policy_version 537731 (0.0010) [2023-12-27 09:21:44,600][20347] Updated weights for policy 0, policy_version 538083 (0.0006) [2023-12-27 09:21:44,628][20381] Updated weights for policy 1, policy_version 537741 (0.0010) [2023-12-27 09:21:44,648][20347] Updated weights for policy 0, policy_version 538093 (0.0005) [2023-12-27 09:21:44,702][20347] Updated weights for policy 0, policy_version 538103 (0.0008) [2023-12-27 09:21:45,361][20347] Updated weights for policy 0, policy_version 538114 (0.0007) [2023-12-27 09:21:45,363][20381] Updated weights for policy 1, policy_version 537751 (0.0008) [2023-12-27 09:21:45,423][20347] Updated weights for policy 0, policy_version 538124 (0.0006) [2023-12-27 09:21:45,425][20381] Updated weights for policy 1, policy_version 537761 (0.0006) [2023-12-27 09:21:45,475][20291] KL-divergence is very high: 123.0122 [2023-12-27 09:21:45,481][20347] Updated weights for policy 0, policy_version 538134 (0.0008) [2023-12-27 09:21:45,483][20381] Updated weights for policy 1, policy_version 537771 (0.0008) [2023-12-27 09:21:45,539][20347] Updated weights for policy 0, policy_version 538144 (0.0007) [2023-12-27 09:21:46,163][20347] Updated weights for policy 0, policy_version 538154 (0.0007) [2023-12-27 09:21:46,197][20381] Updated weights for policy 1, policy_version 537781 (0.0007) [2023-12-27 09:21:46,218][20347] Updated weights for policy 0, policy_version 538164 (0.0009) [2023-12-27 09:21:46,248][20381] Updated weights for policy 1, policy_version 537791 (0.0009) [2023-12-27 09:21:46,266][20347] Updated weights for policy 0, policy_version 538174 (0.0010) [2023-12-27 09:21:46,297][20381] Updated weights for policy 1, policy_version 537801 (0.0009) [2023-12-27 09:21:46,926][20347] Updated weights for policy 0, policy_version 538184 (0.0009) [2023-12-27 09:21:46,972][20347] Updated weights for policy 0, policy_version 538194 (0.0010) [2023-12-27 09:21:46,975][20381] Updated weights for policy 1, policy_version 537811 (0.0009) [2023-12-27 09:21:46,982][20291] KL-divergence is very high: 215.8868 [2023-12-27 09:21:46,997][20291] KL-divergence is very high: 387.7678 [2023-12-27 09:21:47,002][20291] KL-divergence is very high: 248.9787 [2023-12-27 09:21:47,024][20347] Updated weights for policy 0, policy_version 538204 (0.0010) [2023-12-27 09:21:47,034][20381] Updated weights for policy 1, policy_version 537821 (0.0005) [2023-12-27 09:21:47,043][20291] KL-divergence is very high: 172.8156 [2023-12-27 09:21:47,081][20381] Updated weights for policy 1, policy_version 537831 (0.0007) [2023-12-27 09:21:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 275505152. Throughput: 0: 10287.3, 1: 10481.6. Samples: 275513800. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:21:47,363][19170] Avg episode reward: [(0, '725.690'), (1, '727.610')] [2023-12-27 09:21:47,683][20291] KL-divergence is very high: 625.8933 [2023-12-27 09:21:47,689][20291] KL-divergence is very high: 128.8559 [2023-12-27 09:21:47,696][20291] KL-divergence is very high: 212.2441 [2023-12-27 09:21:47,702][20291] KL-divergence is very high: 165.3692 [2023-12-27 09:21:47,709][20291] KL-divergence is very high: 266.7729 [2023-12-27 09:21:47,719][20291] KL-divergence is very high: 259.6048 [2023-12-27 09:21:47,719][20347] Updated weights for policy 0, policy_version 538214 (0.0010) [2023-12-27 09:21:47,729][20291] KL-divergence is very high: 700.8942 [2023-12-27 09:21:47,733][20291] KL-divergence is very high: 164.0359 [2023-12-27 09:21:47,738][20291] KL-divergence is very high: 467.5796 [2023-12-27 09:21:47,743][20291] KL-divergence is very high: 269.2453 [2023-12-27 09:21:47,748][20291] KL-divergence is very high: 430.4375 [2023-12-27 09:21:47,761][20291] KL-divergence is very high: 174.2971 [2023-12-27 09:21:47,771][20381] Updated weights for policy 1, policy_version 537841 (0.0007) [2023-12-27 09:21:47,774][20291] KL-divergence is very high: 479.4416 [2023-12-27 09:21:47,774][20347] Updated weights for policy 0, policy_version 538224 (0.0010) [2023-12-27 09:21:47,779][20291] KL-divergence is very high: 154.4202 [2023-12-27 09:21:47,784][20291] KL-divergence is very high: 443.2045 [2023-12-27 09:21:47,791][20291] KL-divergence is very high: 291.5677 [2023-12-27 09:21:47,798][20291] KL-divergence is very high: 476.7690 [2023-12-27 09:21:47,811][20291] KL-divergence is very high: 222.9783 [2023-12-27 09:21:47,821][20381] Updated weights for policy 1, policy_version 537851 (0.0007) [2023-12-27 09:21:47,823][20291] KL-divergence is very high: 583.0679 [2023-12-27 09:21:47,828][20291] KL-divergence is very high: 170.7805 [2023-12-27 09:21:47,833][20291] KL-divergence is very high: 556.0059 [2023-12-27 09:21:47,833][20347] Updated weights for policy 0, policy_version 538234 (0.0010) [2023-12-27 09:21:47,839][20291] KL-divergence is very high: 385.8753 [2023-12-27 09:21:47,845][20291] KL-divergence is very high: 569.1429 [2023-12-27 09:21:47,856][20291] KL-divergence is very high: 195.0817 [2023-12-27 09:21:47,876][20381] Updated weights for policy 1, policy_version 537861 (0.0007) [2023-12-27 09:21:47,927][20381] Updated weights for policy 1, policy_version 537871 (0.0007) [2023-12-27 09:21:48,510][20291] KL-divergence is very high: 116.2933 [2023-12-27 09:21:48,524][20347] Updated weights for policy 0, policy_version 538244 (0.0010) [2023-12-27 09:21:48,582][20347] Updated weights for policy 0, policy_version 538254 (0.0011) [2023-12-27 09:21:48,605][20291] KL-divergence is very high: 113.0331 [2023-12-27 09:21:48,642][20347] Updated weights for policy 0, policy_version 538264 (0.0010) [2023-12-27 09:21:48,686][20381] Updated weights for policy 1, policy_version 537881 (0.0007) [2023-12-27 09:21:48,742][20381] Updated weights for policy 1, policy_version 537891 (0.0008) [2023-12-27 09:21:48,800][20381] Updated weights for policy 1, policy_version 537901 (0.0007) [2023-12-27 09:21:49,343][20347] Updated weights for policy 0, policy_version 538274 (0.0011) [2023-12-27 09:21:49,408][20347] Updated weights for policy 0, policy_version 538284 (0.0008) [2023-12-27 09:21:49,463][20347] Updated weights for policy 0, policy_version 538294 (0.0007) [2023-12-27 09:21:49,525][20347] Updated weights for policy 0, policy_version 538304 (0.0008) [2023-12-27 09:21:49,531][20381] Updated weights for policy 1, policy_version 537911 (0.0006) [2023-12-27 09:21:49,582][20381] Updated weights for policy 1, policy_version 537921 (0.0007) [2023-12-27 09:21:49,631][20381] Updated weights for policy 1, policy_version 537931 (0.0008) [2023-12-27 09:21:50,237][20347] Updated weights for policy 0, policy_version 538314 (0.0010) [2023-12-27 09:21:50,266][20381] Updated weights for policy 1, policy_version 537941 (0.0007) [2023-12-27 09:21:50,296][20347] Updated weights for policy 0, policy_version 538324 (0.0008) [2023-12-27 09:21:50,325][20381] Updated weights for policy 1, policy_version 537951 (0.0007) [2023-12-27 09:21:50,359][20347] Updated weights for policy 0, policy_version 538334 (0.0010) [2023-12-27 09:21:50,382][20381] Updated weights for policy 1, policy_version 537961 (0.0007) [2023-12-27 09:21:50,981][20347] Updated weights for policy 0, policy_version 538344 (0.0011) [2023-12-27 09:21:51,010][20381] Updated weights for policy 1, policy_version 537971 (0.0006) [2023-12-27 09:21:51,039][20347] Updated weights for policy 0, policy_version 538354 (0.0009) [2023-12-27 09:21:51,070][20381] Updated weights for policy 1, policy_version 537981 (0.0007) [2023-12-27 09:21:51,100][20347] Updated weights for policy 0, policy_version 538364 (0.0008) [2023-12-27 09:21:51,129][20381] Updated weights for policy 1, policy_version 537991 (0.0007) [2023-12-27 09:21:51,768][20347] Updated weights for policy 0, policy_version 538374 (0.0009) [2023-12-27 09:21:51,823][20347] Updated weights for policy 0, policy_version 538384 (0.0007) [2023-12-27 09:21:51,834][20381] Updated weights for policy 1, policy_version 538001 (0.0008) [2023-12-27 09:21:51,871][20347] Updated weights for policy 0, policy_version 538394 (0.0006) [2023-12-27 09:21:51,893][20381] Updated weights for policy 1, policy_version 538011 (0.0008) [2023-12-27 09:21:51,946][20381] Updated weights for policy 1, policy_version 538021 (0.0008) [2023-12-27 09:21:52,002][20381] Updated weights for policy 1, policy_version 538031 (0.0007) [2023-12-27 09:21:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 275603456. Throughput: 0: 10310.0, 1: 10402.5. Samples: 275573980. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:21:52,363][19170] Avg episode reward: [(0, '326.690'), (1, '727.680')] [2023-12-27 09:21:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000538032_137756672.pth... [2023-12-27 09:21:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000538400_137846784.pth... [2023-12-27 09:21:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000537184_137535488.pth [2023-12-27 09:21:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000536816_137445376.pth [2023-12-27 09:21:52,481][20347] Updated weights for policy 0, policy_version 538404 (0.0006) [2023-12-27 09:21:52,527][20347] Updated weights for policy 0, policy_version 538414 (0.0005) [2023-12-27 09:21:52,580][20347] Updated weights for policy 0, policy_version 538424 (0.0005) [2023-12-27 09:21:52,754][20381] Updated weights for policy 1, policy_version 538041 (0.0007) [2023-12-27 09:21:52,807][20381] Updated weights for policy 1, policy_version 538051 (0.0007) [2023-12-27 09:21:52,865][20381] Updated weights for policy 1, policy_version 538061 (0.0007) [2023-12-27 09:21:53,209][20347] Updated weights for policy 0, policy_version 538434 (0.0006) [2023-12-27 09:21:53,266][20347] Updated weights for policy 0, policy_version 538444 (0.0007) [2023-12-27 09:21:53,315][20347] Updated weights for policy 0, policy_version 538454 (0.0008) [2023-12-27 09:21:53,369][20347] Updated weights for policy 0, policy_version 538464 (0.0007) [2023-12-27 09:21:53,559][20381] Updated weights for policy 1, policy_version 538071 (0.0008) [2023-12-27 09:21:53,607][20381] Updated weights for policy 1, policy_version 538081 (0.0006) [2023-12-27 09:21:53,660][20381] Updated weights for policy 1, policy_version 538091 (0.0006) [2023-12-27 09:21:54,064][20347] Updated weights for policy 0, policy_version 538474 (0.0007) [2023-12-27 09:21:54,113][20347] Updated weights for policy 0, policy_version 538484 (0.0005) [2023-12-27 09:21:54,173][20347] Updated weights for policy 0, policy_version 538494 (0.0005) [2023-12-27 09:21:54,243][20381] Updated weights for policy 1, policy_version 538101 (0.0008) [2023-12-27 09:21:54,294][20381] Updated weights for policy 1, policy_version 538111 (0.0006) [2023-12-27 09:21:54,345][20381] Updated weights for policy 1, policy_version 538121 (0.0005) [2023-12-27 09:21:54,767][20347] Updated weights for policy 0, policy_version 538504 (0.0009) [2023-12-27 09:21:54,828][20347] Updated weights for policy 0, policy_version 538514 (0.0007) [2023-12-27 09:21:54,884][20347] Updated weights for policy 0, policy_version 538524 (0.0005) [2023-12-27 09:21:54,944][20381] Updated weights for policy 1, policy_version 538131 (0.0005) [2023-12-27 09:21:55,005][20381] Updated weights for policy 1, policy_version 538141 (0.0007) [2023-12-27 09:21:55,068][20381] Updated weights for policy 1, policy_version 538151 (0.0007) [2023-12-27 09:21:55,536][20347] Updated weights for policy 0, policy_version 538534 (0.0010) [2023-12-27 09:21:55,591][20347] Updated weights for policy 0, policy_version 538544 (0.0007) [2023-12-27 09:21:55,643][20347] Updated weights for policy 0, policy_version 538554 (0.0005) [2023-12-27 09:21:55,736][20381] Updated weights for policy 1, policy_version 538161 (0.0007) [2023-12-27 09:21:55,796][20381] Updated weights for policy 1, policy_version 538171 (0.0008) [2023-12-27 09:21:55,854][20381] Updated weights for policy 1, policy_version 538181 (0.0007) [2023-12-27 09:21:55,913][20381] Updated weights for policy 1, policy_version 538191 (0.0007) [2023-12-27 09:21:56,287][20347] Updated weights for policy 0, policy_version 538564 (0.0007) [2023-12-27 09:21:56,338][20347] Updated weights for policy 0, policy_version 538574 (0.0010) [2023-12-27 09:21:56,387][20347] Updated weights for policy 0, policy_version 538584 (0.0010) [2023-12-27 09:21:56,607][20381] Updated weights for policy 1, policy_version 538201 (0.0007) [2023-12-27 09:21:56,664][20381] Updated weights for policy 1, policy_version 538211 (0.0007) [2023-12-27 09:21:56,724][20381] Updated weights for policy 1, policy_version 538221 (0.0008) [2023-12-27 09:21:57,091][20347] Updated weights for policy 0, policy_version 538594 (0.0010) [2023-12-27 09:21:57,153][20347] Updated weights for policy 0, policy_version 538604 (0.0008) [2023-12-27 09:21:57,208][20347] Updated weights for policy 0, policy_version 538614 (0.0007) [2023-12-27 09:21:57,268][20347] Updated weights for policy 0, policy_version 538624 (0.0008) [2023-12-27 09:21:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 275709952. Throughput: 0: 10330.8, 1: 10391.6. Samples: 275700680. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:21:57,363][19170] Avg episode reward: [(0, '464.210'), (1, '766.730')] [2023-12-27 09:21:57,407][20381] Updated weights for policy 1, policy_version 538231 (0.0007) [2023-12-27 09:21:57,466][20381] Updated weights for policy 1, policy_version 538241 (0.0006) [2023-12-27 09:21:57,520][20381] Updated weights for policy 1, policy_version 538251 (0.0007) [2023-12-27 09:21:57,932][20347] Updated weights for policy 0, policy_version 538634 (0.0006) [2023-12-27 09:21:57,990][20347] Updated weights for policy 0, policy_version 538644 (0.0005) [2023-12-27 09:21:58,049][20347] Updated weights for policy 0, policy_version 538654 (0.0007) [2023-12-27 09:21:58,188][20381] Updated weights for policy 1, policy_version 538261 (0.0006) [2023-12-27 09:21:58,243][20381] Updated weights for policy 1, policy_version 538271 (0.0006) [2023-12-27 09:21:58,291][20381] Updated weights for policy 1, policy_version 538281 (0.0006) [2023-12-27 09:21:58,744][20347] Updated weights for policy 0, policy_version 538664 (0.0009) [2023-12-27 09:21:58,805][20347] Updated weights for policy 0, policy_version 538674 (0.0011) [2023-12-27 09:21:58,873][20347] Updated weights for policy 0, policy_version 538684 (0.0008) [2023-12-27 09:21:58,996][20381] Updated weights for policy 1, policy_version 538291 (0.0006) [2023-12-27 09:21:59,061][20381] Updated weights for policy 1, policy_version 538301 (0.0005) [2023-12-27 09:21:59,125][20381] Updated weights for policy 1, policy_version 538311 (0.0007) [2023-12-27 09:21:59,523][20347] Updated weights for policy 0, policy_version 538694 (0.0010) [2023-12-27 09:21:59,575][20347] Updated weights for policy 0, policy_version 538704 (0.0010) [2023-12-27 09:21:59,630][20347] Updated weights for policy 0, policy_version 538714 (0.0010) [2023-12-27 09:21:59,740][20381] Updated weights for policy 1, policy_version 538321 (0.0006) [2023-12-27 09:21:59,804][20381] Updated weights for policy 1, policy_version 538331 (0.0008) [2023-12-27 09:21:59,861][20381] Updated weights for policy 1, policy_version 538341 (0.0007) [2023-12-27 09:21:59,917][20381] Updated weights for policy 1, policy_version 538351 (0.0007) [2023-12-27 09:22:00,257][20347] Updated weights for policy 0, policy_version 538724 (0.0008) [2023-12-27 09:22:00,307][20347] Updated weights for policy 0, policy_version 538734 (0.0010) [2023-12-27 09:22:00,355][20347] Updated weights for policy 0, policy_version 538744 (0.0010) [2023-12-27 09:22:00,593][20381] Updated weights for policy 1, policy_version 538361 (0.0007) [2023-12-27 09:22:00,648][20381] Updated weights for policy 1, policy_version 538371 (0.0005) [2023-12-27 09:22:00,702][20381] Updated weights for policy 1, policy_version 538381 (0.0008) [2023-12-27 09:22:01,000][20347] Updated weights for policy 0, policy_version 538754 (0.0010) [2023-12-27 09:22:01,059][20347] Updated weights for policy 0, policy_version 538764 (0.0009) [2023-12-27 09:22:01,110][20347] Updated weights for policy 0, policy_version 538774 (0.0006) [2023-12-27 09:22:01,177][20347] Updated weights for policy 0, policy_version 538784 (0.0010) [2023-12-27 09:22:01,381][20381] Updated weights for policy 1, policy_version 538391 (0.0008) [2023-12-27 09:22:01,442][20381] Updated weights for policy 1, policy_version 538401 (0.0008) [2023-12-27 09:22:01,503][20381] Updated weights for policy 1, policy_version 538411 (0.0008) [2023-12-27 09:22:01,913][20347] Updated weights for policy 0, policy_version 538794 (0.0006) [2023-12-27 09:22:01,965][20347] Updated weights for policy 0, policy_version 538804 (0.0008) [2023-12-27 09:22:02,019][20347] Updated weights for policy 0, policy_version 538814 (0.0009) [2023-12-27 09:22:02,072][20381] Updated weights for policy 1, policy_version 538421 (0.0008) [2023-12-27 09:22:02,125][20381] Updated weights for policy 1, policy_version 538431 (0.0006) [2023-12-27 09:22:02,183][20381] Updated weights for policy 1, policy_version 538441 (0.0005) [2023-12-27 09:22:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 275816448. Throughput: 0: 10304.6, 1: 10375.8. Samples: 275824312. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:02,363][19170] Avg episode reward: [(0, '752.310'), (1, '718.610')] [2023-12-27 09:22:02,741][20347] Updated weights for policy 0, policy_version 538824 (0.0008) [2023-12-27 09:22:02,799][20347] Updated weights for policy 0, policy_version 538834 (0.0007) [2023-12-27 09:22:02,811][20381] Updated weights for policy 1, policy_version 538451 (0.0006) [2023-12-27 09:22:02,860][20381] Updated weights for policy 1, policy_version 538461 (0.0006) [2023-12-27 09:22:02,862][20347] Updated weights for policy 0, policy_version 538844 (0.0005) [2023-12-27 09:22:02,904][20381] Updated weights for policy 1, policy_version 538471 (0.0006) [2023-12-27 09:22:03,438][20347] Updated weights for policy 0, policy_version 538854 (0.0005) [2023-12-27 09:22:03,456][20381] Updated weights for policy 1, policy_version 538481 (0.0005) [2023-12-27 09:22:03,484][20347] Updated weights for policy 0, policy_version 538864 (0.0005) [2023-12-27 09:22:03,502][20381] Updated weights for policy 1, policy_version 538491 (0.0005) [2023-12-27 09:22:03,540][20347] Updated weights for policy 0, policy_version 538874 (0.0005) [2023-12-27 09:22:03,549][20381] Updated weights for policy 1, policy_version 538501 (0.0009) [2023-12-27 09:22:03,605][20381] Updated weights for policy 1, policy_version 538511 (0.0009) [2023-12-27 09:22:04,191][20381] Updated weights for policy 1, policy_version 538521 (0.0005) [2023-12-27 09:22:04,240][20347] Updated weights for policy 0, policy_version 538884 (0.0005) [2023-12-27 09:22:04,246][20381] Updated weights for policy 1, policy_version 538531 (0.0006) [2023-12-27 09:22:04,296][20347] Updated weights for policy 0, policy_version 538894 (0.0007) [2023-12-27 09:22:04,315][20381] Updated weights for policy 1, policy_version 538541 (0.0006) [2023-12-27 09:22:04,350][20347] Updated weights for policy 0, policy_version 538904 (0.0007) [2023-12-27 09:22:04,943][20347] Updated weights for policy 0, policy_version 538914 (0.0006) [2023-12-27 09:22:04,963][20381] Updated weights for policy 1, policy_version 538551 (0.0007) [2023-12-27 09:22:04,996][20347] Updated weights for policy 0, policy_version 538924 (0.0005) [2023-12-27 09:22:05,025][20381] Updated weights for policy 1, policy_version 538561 (0.0008) [2023-12-27 09:22:05,058][20347] Updated weights for policy 0, policy_version 538934 (0.0005) [2023-12-27 09:22:05,086][20381] Updated weights for policy 1, policy_version 538571 (0.0005) [2023-12-27 09:22:05,116][20347] Updated weights for policy 0, policy_version 538944 (0.0007) [2023-12-27 09:22:05,709][20347] Updated weights for policy 0, policy_version 538954 (0.0007) [2023-12-27 09:22:05,766][20347] Updated weights for policy 0, policy_version 538964 (0.0007) [2023-12-27 09:22:05,775][20381] Updated weights for policy 1, policy_version 538581 (0.0006) [2023-12-27 09:22:05,823][20347] Updated weights for policy 0, policy_version 538974 (0.0007) [2023-12-27 09:22:05,825][20381] Updated weights for policy 1, policy_version 538591 (0.0005) [2023-12-27 09:22:05,874][20381] Updated weights for policy 1, policy_version 538601 (0.0007) [2023-12-27 09:22:06,516][20347] Updated weights for policy 0, policy_version 538984 (0.0007) [2023-12-27 09:22:06,562][20381] Updated weights for policy 1, policy_version 538611 (0.0007) [2023-12-27 09:22:06,574][20347] Updated weights for policy 0, policy_version 538994 (0.0009) [2023-12-27 09:22:06,618][20381] Updated weights for policy 1, policy_version 538621 (0.0007) [2023-12-27 09:22:06,633][20347] Updated weights for policy 0, policy_version 539004 (0.0007) [2023-12-27 09:22:06,672][20381] Updated weights for policy 1, policy_version 538631 (0.0008) [2023-12-27 09:22:07,303][20347] Updated weights for policy 0, policy_version 539014 (0.0005) [2023-12-27 09:22:07,320][20381] Updated weights for policy 1, policy_version 538641 (0.0008) [2023-12-27 09:22:07,354][20347] Updated weights for policy 0, policy_version 539024 (0.0006) [2023-12-27 09:22:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 275914752. Throughput: 0: 10356.3, 1: 10471.0. Samples: 275890956. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:07,363][19170] Avg episode reward: [(0, '829.330'), (1, '726.790')] [2023-12-27 09:22:07,370][20381] Updated weights for policy 1, policy_version 538651 (0.0011) [2023-12-27 09:22:07,408][20347] Updated weights for policy 0, policy_version 539034 (0.0007) [2023-12-27 09:22:07,425][20381] Updated weights for policy 1, policy_version 538661 (0.0010) [2023-12-27 09:22:07,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000539040_138010624.pth... [2023-12-27 09:22:07,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000537824_137699328.pth [2023-12-27 09:22:07,483][20381] Updated weights for policy 1, policy_version 538671 (0.0011) [2023-12-27 09:22:07,489][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000538672_137920512.pth... [2023-12-27 09:22:07,492][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000537456_137609216.pth [2023-12-27 09:22:08,082][20347] Updated weights for policy 0, policy_version 539044 (0.0006) [2023-12-27 09:22:08,131][20347] Updated weights for policy 0, policy_version 539054 (0.0007) [2023-12-27 09:22:08,149][20381] Updated weights for policy 1, policy_version 538681 (0.0011) [2023-12-27 09:22:08,180][20347] Updated weights for policy 0, policy_version 539064 (0.0005) [2023-12-27 09:22:08,200][20381] Updated weights for policy 1, policy_version 538691 (0.0010) [2023-12-27 09:22:08,256][20381] Updated weights for policy 1, policy_version 538701 (0.0010) [2023-12-27 09:22:08,802][20347] Updated weights for policy 0, policy_version 539074 (0.0006) [2023-12-27 09:22:08,863][20347] Updated weights for policy 0, policy_version 539084 (0.0007) [2023-12-27 09:22:08,892][20381] Updated weights for policy 1, policy_version 538711 (0.0010) [2023-12-27 09:22:08,921][20347] Updated weights for policy 0, policy_version 539094 (0.0009) [2023-12-27 09:22:08,948][20381] Updated weights for policy 1, policy_version 538721 (0.0011) [2023-12-27 09:22:08,983][20347] Updated weights for policy 0, policy_version 539104 (0.0006) [2023-12-27 09:22:09,006][20381] Updated weights for policy 1, policy_version 538731 (0.0010) [2023-12-27 09:22:09,665][20347] Updated weights for policy 0, policy_version 539114 (0.0008) [2023-12-27 09:22:09,680][20381] Updated weights for policy 1, policy_version 538741 (0.0010) [2023-12-27 09:22:09,722][20347] Updated weights for policy 0, policy_version 539124 (0.0006) [2023-12-27 09:22:09,748][20381] Updated weights for policy 1, policy_version 538751 (0.0008) [2023-12-27 09:22:09,779][20347] Updated weights for policy 0, policy_version 539134 (0.0007) [2023-12-27 09:22:09,814][20381] Updated weights for policy 1, policy_version 538761 (0.0008) [2023-12-27 09:22:10,468][20381] Updated weights for policy 1, policy_version 538771 (0.0007) [2023-12-27 09:22:10,527][20347] Updated weights for policy 0, policy_version 539144 (0.0006) [2023-12-27 09:22:10,531][20381] Updated weights for policy 1, policy_version 538781 (0.0006) [2023-12-27 09:22:10,581][20347] Updated weights for policy 0, policy_version 539154 (0.0005) [2023-12-27 09:22:10,592][20381] Updated weights for policy 1, policy_version 538791 (0.0005) [2023-12-27 09:22:10,641][20347] Updated weights for policy 0, policy_version 539164 (0.0009) [2023-12-27 09:22:11,201][20381] Updated weights for policy 1, policy_version 538801 (0.0005) [2023-12-27 09:22:11,218][20347] Updated weights for policy 0, policy_version 539174 (0.0010) [2023-12-27 09:22:11,260][20381] Updated weights for policy 1, policy_version 538811 (0.0009) [2023-12-27 09:22:11,281][20347] Updated weights for policy 0, policy_version 539184 (0.0010) [2023-12-27 09:22:11,317][20381] Updated weights for policy 1, policy_version 538821 (0.0006) [2023-12-27 09:22:11,341][20347] Updated weights for policy 0, policy_version 539194 (0.0010) [2023-12-27 09:22:11,374][20381] Updated weights for policy 1, policy_version 538831 (0.0008) [2023-12-27 09:22:12,037][20347] Updated weights for policy 0, policy_version 539204 (0.0009) [2023-12-27 09:22:12,073][20381] Updated weights for policy 1, policy_version 538841 (0.0006) [2023-12-27 09:22:12,096][20347] Updated weights for policy 0, policy_version 539214 (0.0007) [2023-12-27 09:22:12,129][20381] Updated weights for policy 1, policy_version 538851 (0.0005) [2023-12-27 09:22:12,153][20347] Updated weights for policy 0, policy_version 539224 (0.0008) [2023-12-27 09:22:12,185][20381] Updated weights for policy 1, policy_version 538861 (0.0006) [2023-12-27 09:22:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 276029440. Throughput: 0: 10401.0, 1: 10485.0. Samples: 276016464. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:12,363][19170] Avg episode reward: [(0, '837.920'), (1, '679.500')] [2023-12-27 09:22:12,785][20347] Updated weights for policy 0, policy_version 539234 (0.0007) [2023-12-27 09:22:12,848][20347] Updated weights for policy 0, policy_version 539244 (0.0005) [2023-12-27 09:22:12,911][20347] Updated weights for policy 0, policy_version 539254 (0.0006) [2023-12-27 09:22:12,915][20381] Updated weights for policy 1, policy_version 538871 (0.0008) [2023-12-27 09:22:12,971][20347] Updated weights for policy 0, policy_version 539264 (0.0006) [2023-12-27 09:22:12,979][20381] Updated weights for policy 1, policy_version 538881 (0.0008) [2023-12-27 09:22:13,043][20381] Updated weights for policy 1, policy_version 538891 (0.0006) [2023-12-27 09:22:13,565][20381] Updated weights for policy 1, policy_version 538901 (0.0006) [2023-12-27 09:22:13,586][20347] Updated weights for policy 0, policy_version 539274 (0.0006) [2023-12-27 09:22:13,620][20381] Updated weights for policy 1, policy_version 538911 (0.0006) [2023-12-27 09:22:13,633][20347] Updated weights for policy 0, policy_version 539284 (0.0007) [2023-12-27 09:22:13,675][20347] Updated weights for policy 0, policy_version 539294 (0.0007) [2023-12-27 09:22:13,677][20381] Updated weights for policy 1, policy_version 538921 (0.0007) [2023-12-27 09:22:14,291][20381] Updated weights for policy 1, policy_version 538931 (0.0007) [2023-12-27 09:22:14,334][20347] Updated weights for policy 0, policy_version 539304 (0.0007) [2023-12-27 09:22:14,355][20381] Updated weights for policy 1, policy_version 538941 (0.0006) [2023-12-27 09:22:14,380][20347] Updated weights for policy 0, policy_version 539314 (0.0010) [2023-12-27 09:22:14,420][20381] Updated weights for policy 1, policy_version 538951 (0.0007) [2023-12-27 09:22:14,432][20347] Updated weights for policy 0, policy_version 539324 (0.0010) [2023-12-27 09:22:14,976][20381] Updated weights for policy 1, policy_version 538961 (0.0010) [2023-12-27 09:22:15,036][20381] Updated weights for policy 1, policy_version 538971 (0.0011) [2023-12-27 09:22:15,100][20381] Updated weights for policy 1, policy_version 538981 (0.0011) [2023-12-27 09:22:15,139][20347] Updated weights for policy 0, policy_version 539334 (0.0008) [2023-12-27 09:22:15,166][20381] Updated weights for policy 1, policy_version 538991 (0.0011) [2023-12-27 09:22:15,186][20347] Updated weights for policy 0, policy_version 539344 (0.0010) [2023-12-27 09:22:15,240][20347] Updated weights for policy 0, policy_version 539354 (0.0009) [2023-12-27 09:22:15,849][20381] Updated weights for policy 1, policy_version 539001 (0.0010) [2023-12-27 09:22:15,872][20347] Updated weights for policy 0, policy_version 539364 (0.0007) [2023-12-27 09:22:15,900][20381] Updated weights for policy 1, policy_version 539011 (0.0010) [2023-12-27 09:22:15,930][20347] Updated weights for policy 0, policy_version 539374 (0.0010) [2023-12-27 09:22:15,952][20381] Updated weights for policy 1, policy_version 539021 (0.0010) [2023-12-27 09:22:15,988][20347] Updated weights for policy 0, policy_version 539384 (0.0010) [2023-12-27 09:22:16,631][20381] Updated weights for policy 1, policy_version 539031 (0.0010) [2023-12-27 09:22:16,657][20347] Updated weights for policy 0, policy_version 539394 (0.0009) [2023-12-27 09:22:16,681][20381] Updated weights for policy 1, policy_version 539041 (0.0010) [2023-12-27 09:22:16,713][20347] Updated weights for policy 0, policy_version 539404 (0.0005) [2023-12-27 09:22:16,723][20381] Updated weights for policy 1, policy_version 539051 (0.0007) [2023-12-27 09:22:16,774][20347] Updated weights for policy 0, policy_version 539414 (0.0005) [2023-12-27 09:22:16,834][20347] Updated weights for policy 0, policy_version 539424 (0.0008) [2023-12-27 09:22:17,312][20381] Updated weights for policy 1, policy_version 539061 (0.0006) [2023-12-27 09:22:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 276127744. Throughput: 0: 10420.1, 1: 10514.6. Samples: 276143752. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:17,363][19170] Avg episode reward: [(0, '857.890'), (1, '611.290')] [2023-12-27 09:22:17,367][20381] Updated weights for policy 1, policy_version 539071 (0.0007) [2023-12-27 09:22:17,391][20347] Updated weights for policy 0, policy_version 539434 (0.0007) [2023-12-27 09:22:17,418][20381] Updated weights for policy 1, policy_version 539081 (0.0006) [2023-12-27 09:22:17,439][20347] Updated weights for policy 0, policy_version 539444 (0.0005) [2023-12-27 09:22:17,489][20347] Updated weights for policy 0, policy_version 539454 (0.0007) [2023-12-27 09:22:18,075][20381] Updated weights for policy 1, policy_version 539091 (0.0006) [2023-12-27 09:22:18,140][20381] Updated weights for policy 1, policy_version 539101 (0.0008) [2023-12-27 09:22:18,188][20347] Updated weights for policy 0, policy_version 539464 (0.0006) [2023-12-27 09:22:18,201][20381] Updated weights for policy 1, policy_version 539111 (0.0007) [2023-12-27 09:22:18,243][20347] Updated weights for policy 0, policy_version 539474 (0.0005) [2023-12-27 09:22:18,294][20347] Updated weights for policy 0, policy_version 539484 (0.0005) [2023-12-27 09:22:18,304][20291] KL-divergence is very high: 310.9025 [2023-12-27 09:22:18,309][20291] KL-divergence is very high: 333.7011 [2023-12-27 09:22:18,911][20347] Updated weights for policy 0, policy_version 539494 (0.0007) [2023-12-27 09:22:18,913][20381] Updated weights for policy 1, policy_version 539121 (0.0007) [2023-12-27 09:22:18,964][20347] Updated weights for policy 0, policy_version 539504 (0.0005) [2023-12-27 09:22:18,969][20381] Updated weights for policy 1, policy_version 539131 (0.0007) [2023-12-27 09:22:19,020][20381] Updated weights for policy 1, policy_version 539141 (0.0006) [2023-12-27 09:22:19,021][20347] Updated weights for policy 0, policy_version 539514 (0.0006) [2023-12-27 09:22:19,069][20381] Updated weights for policy 1, policy_version 539151 (0.0006) [2023-12-27 09:22:19,713][20347] Updated weights for policy 0, policy_version 539524 (0.0008) [2023-12-27 09:22:19,772][20347] Updated weights for policy 0, policy_version 539534 (0.0010) [2023-12-27 09:22:19,835][20347] Updated weights for policy 0, policy_version 539544 (0.0010) [2023-12-27 09:22:19,835][20381] Updated weights for policy 1, policy_version 539161 (0.0009) [2023-12-27 09:22:19,893][20381] Updated weights for policy 1, policy_version 539171 (0.0007) [2023-12-27 09:22:19,960][20381] Updated weights for policy 1, policy_version 539181 (0.0009) [2023-12-27 09:22:20,513][20347] Updated weights for policy 0, policy_version 539554 (0.0010) [2023-12-27 09:22:20,566][20347] Updated weights for policy 0, policy_version 539564 (0.0011) [2023-12-27 09:22:20,629][20347] Updated weights for policy 0, policy_version 539574 (0.0011) [2023-12-27 09:22:20,659][20381] Updated weights for policy 1, policy_version 539191 (0.0009) [2023-12-27 09:22:20,686][20347] Updated weights for policy 0, policy_version 539584 (0.0008) [2023-12-27 09:22:20,713][20381] Updated weights for policy 1, policy_version 539201 (0.0011) [2023-12-27 09:22:20,781][20381] Updated weights for policy 1, policy_version 539211 (0.0010) [2023-12-27 09:22:21,360][20347] Updated weights for policy 0, policy_version 539594 (0.0010) [2023-12-27 09:22:21,423][20347] Updated weights for policy 0, policy_version 539604 (0.0007) [2023-12-27 09:22:21,483][20347] Updated weights for policy 0, policy_version 539614 (0.0007) [2023-12-27 09:22:21,484][20381] Updated weights for policy 1, policy_version 539221 (0.0011) [2023-12-27 09:22:21,542][20381] Updated weights for policy 1, policy_version 539231 (0.0010) [2023-12-27 09:22:21,601][20381] Updated weights for policy 1, policy_version 539241 (0.0011) [2023-12-27 09:22:22,203][20347] Updated weights for policy 0, policy_version 539624 (0.0007) [2023-12-27 09:22:22,270][20347] Updated weights for policy 0, policy_version 539634 (0.0007) [2023-12-27 09:22:22,301][20381] Updated weights for policy 1, policy_version 539251 (0.0011) [2023-12-27 09:22:22,324][20347] Updated weights for policy 0, policy_version 539644 (0.0008) [2023-12-27 09:22:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 276234240. Throughput: 0: 10406.4, 1: 10447.0. Samples: 276205052. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:22,363][19170] Avg episode reward: [(0, '595.350'), (1, '651.070')] [2023-12-27 09:22:22,368][20381] Updated weights for policy 1, policy_version 539261 (0.0011) [2023-12-27 09:22:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000539648_138166272.pth... [2023-12-27 09:22:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000538400_137846784.pth [2023-12-27 09:22:22,439][20381] Updated weights for policy 1, policy_version 539271 (0.0011) [2023-12-27 09:22:22,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000539280_138076160.pth... [2023-12-27 09:22:22,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000538032_137756672.pth [2023-12-27 09:22:23,044][20347] Updated weights for policy 0, policy_version 539654 (0.0007) [2023-12-27 09:22:23,101][20347] Updated weights for policy 0, policy_version 539664 (0.0007) [2023-12-27 09:22:23,109][20381] Updated weights for policy 1, policy_version 539281 (0.0011) [2023-12-27 09:22:23,157][20381] Updated weights for policy 1, policy_version 539291 (0.0010) [2023-12-27 09:22:23,162][20347] Updated weights for policy 0, policy_version 539674 (0.0006) [2023-12-27 09:22:23,208][20381] Updated weights for policy 1, policy_version 539301 (0.0010) [2023-12-27 09:22:23,264][20381] Updated weights for policy 1, policy_version 539311 (0.0010) [2023-12-27 09:22:23,779][20347] Updated weights for policy 0, policy_version 539684 (0.0006) [2023-12-27 09:22:23,826][20347] Updated weights for policy 0, policy_version 539694 (0.0005) [2023-12-27 09:22:23,884][20347] Updated weights for policy 0, policy_version 539704 (0.0005) [2023-12-27 09:22:23,960][20381] Updated weights for policy 1, policy_version 539321 (0.0010) [2023-12-27 09:22:24,008][20381] Updated weights for policy 1, policy_version 539331 (0.0006) [2023-12-27 09:22:24,062][20381] Updated weights for policy 1, policy_version 539341 (0.0010) [2023-12-27 09:22:24,470][20347] Updated weights for policy 0, policy_version 539714 (0.0005) [2023-12-27 09:22:24,525][20347] Updated weights for policy 0, policy_version 539724 (0.0007) [2023-12-27 09:22:24,572][20347] Updated weights for policy 0, policy_version 539734 (0.0007) [2023-12-27 09:22:24,627][20347] Updated weights for policy 0, policy_version 539744 (0.0007) [2023-12-27 09:22:24,780][20381] Updated weights for policy 1, policy_version 539351 (0.0007) [2023-12-27 09:22:24,840][20381] Updated weights for policy 1, policy_version 539361 (0.0005) [2023-12-27 09:22:24,893][20381] Updated weights for policy 1, policy_version 539371 (0.0005) [2023-12-27 09:22:25,244][20347] Updated weights for policy 0, policy_version 539754 (0.0006) [2023-12-27 09:22:25,292][20347] Updated weights for policy 0, policy_version 539764 (0.0007) [2023-12-27 09:22:25,336][20347] Updated weights for policy 0, policy_version 539774 (0.0007) [2023-12-27 09:22:25,532][20381] Updated weights for policy 1, policy_version 539381 (0.0008) [2023-12-27 09:22:25,590][20381] Updated weights for policy 1, policy_version 539391 (0.0010) [2023-12-27 09:22:25,649][20381] Updated weights for policy 1, policy_version 539401 (0.0011) [2023-12-27 09:22:26,055][20347] Updated weights for policy 0, policy_version 539784 (0.0007) [2023-12-27 09:22:26,115][20347] Updated weights for policy 0, policy_version 539794 (0.0007) [2023-12-27 09:22:26,178][20347] Updated weights for policy 0, policy_version 539804 (0.0007) [2023-12-27 09:22:26,326][20381] Updated weights for policy 1, policy_version 539411 (0.0010) [2023-12-27 09:22:26,381][20381] Updated weights for policy 1, policy_version 539421 (0.0011) [2023-12-27 09:22:26,441][20381] Updated weights for policy 1, policy_version 539431 (0.0010) [2023-12-27 09:22:26,802][20347] Updated weights for policy 0, policy_version 539814 (0.0007) [2023-12-27 09:22:26,865][20347] Updated weights for policy 0, policy_version 539824 (0.0007) [2023-12-27 09:22:26,925][20347] Updated weights for policy 0, policy_version 539834 (0.0007) [2023-12-27 09:22:27,130][20381] Updated weights for policy 1, policy_version 539441 (0.0010) [2023-12-27 09:22:27,181][20381] Updated weights for policy 1, policy_version 539451 (0.0010) [2023-12-27 09:22:27,232][20381] Updated weights for policy 1, policy_version 539461 (0.0010) [2023-12-27 09:22:27,283][20381] Updated weights for policy 1, policy_version 539471 (0.0010) [2023-12-27 09:22:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 276340736. Throughput: 0: 10420.4, 1: 10400.0. Samples: 276328280. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:27,363][19170] Avg episode reward: [(0, '305.680'), (1, '750.150')] [2023-12-27 09:22:27,549][20347] Updated weights for policy 0, policy_version 539844 (0.0007) [2023-12-27 09:22:27,599][20347] Updated weights for policy 0, policy_version 539854 (0.0007) [2023-12-27 09:22:27,648][20347] Updated weights for policy 0, policy_version 539864 (0.0007) [2023-12-27 09:22:27,971][20381] Updated weights for policy 1, policy_version 539481 (0.0010) [2023-12-27 09:22:28,031][20381] Updated weights for policy 1, policy_version 539491 (0.0010) [2023-12-27 09:22:28,092][20381] Updated weights for policy 1, policy_version 539501 (0.0007) [2023-12-27 09:22:28,313][20347] Updated weights for policy 0, policy_version 539874 (0.0007) [2023-12-27 09:22:28,379][20347] Updated weights for policy 0, policy_version 539884 (0.0008) [2023-12-27 09:22:28,438][20347] Updated weights for policy 0, policy_version 539894 (0.0006) [2023-12-27 09:22:28,493][20347] Updated weights for policy 0, policy_version 539904 (0.0007) [2023-12-27 09:22:28,761][20381] Updated weights for policy 1, policy_version 539511 (0.0008) [2023-12-27 09:22:28,843][20381] Updated weights for policy 1, policy_version 539521 (0.0010) [2023-12-27 09:22:28,903][20381] Updated weights for policy 1, policy_version 539531 (0.0010) [2023-12-27 09:22:29,163][20347] Updated weights for policy 0, policy_version 539914 (0.0010) [2023-12-27 09:22:29,212][20347] Updated weights for policy 0, policy_version 539924 (0.0010) [2023-12-27 09:22:29,274][20347] Updated weights for policy 0, policy_version 539934 (0.0008) [2023-12-27 09:22:29,527][20381] Updated weights for policy 1, policy_version 539541 (0.0009) [2023-12-27 09:22:29,588][20381] Updated weights for policy 1, policy_version 539551 (0.0008) [2023-12-27 09:22:29,642][20381] Updated weights for policy 1, policy_version 539561 (0.0007) [2023-12-27 09:22:29,955][20347] Updated weights for policy 0, policy_version 539944 (0.0010) [2023-12-27 09:22:30,009][20347] Updated weights for policy 0, policy_version 539954 (0.0007) [2023-12-27 09:22:30,065][20347] Updated weights for policy 0, policy_version 539964 (0.0007) [2023-12-27 09:22:30,307][20381] Updated weights for policy 1, policy_version 539571 (0.0009) [2023-12-27 09:22:30,369][20381] Updated weights for policy 1, policy_version 539581 (0.0008) [2023-12-27 09:22:30,417][20381] Updated weights for policy 1, policy_version 539591 (0.0006) [2023-12-27 09:22:30,790][20347] Updated weights for policy 0, policy_version 539974 (0.0009) [2023-12-27 09:22:30,835][20347] Updated weights for policy 0, policy_version 539984 (0.0010) [2023-12-27 09:22:30,886][20347] Updated weights for policy 0, policy_version 539994 (0.0010) [2023-12-27 09:22:31,006][20381] Updated weights for policy 1, policy_version 539601 (0.0005) [2023-12-27 09:22:31,062][20381] Updated weights for policy 1, policy_version 539611 (0.0007) [2023-12-27 09:22:31,123][20381] Updated weights for policy 1, policy_version 539621 (0.0007) [2023-12-27 09:22:31,184][20381] Updated weights for policy 1, policy_version 539631 (0.0007) [2023-12-27 09:22:31,596][20347] Updated weights for policy 0, policy_version 540004 (0.0010) [2023-12-27 09:22:31,648][20347] Updated weights for policy 0, policy_version 540014 (0.0009) [2023-12-27 09:22:31,711][20347] Updated weights for policy 0, policy_version 540024 (0.0010) [2023-12-27 09:22:31,893][20381] Updated weights for policy 1, policy_version 539641 (0.0007) [2023-12-27 09:22:31,937][20381] Updated weights for policy 1, policy_version 539651 (0.0007) [2023-12-27 09:22:31,986][20381] Updated weights for policy 1, policy_version 539661 (0.0008) [2023-12-27 09:22:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 276439040. Throughput: 0: 10437.4, 1: 10406.6. Samples: 276451780. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:32,363][19170] Avg episode reward: [(0, '464.610'), (1, '674.000')] [2023-12-27 09:22:32,405][20347] Updated weights for policy 0, policy_version 540034 (0.0010) [2023-12-27 09:22:32,462][20347] Updated weights for policy 0, policy_version 540044 (0.0010) [2023-12-27 09:22:32,510][20347] Updated weights for policy 0, policy_version 540054 (0.0010) [2023-12-27 09:22:32,562][20347] Updated weights for policy 0, policy_version 540064 (0.0010) [2023-12-27 09:22:32,708][20381] Updated weights for policy 1, policy_version 539671 (0.0006) [2023-12-27 09:22:32,768][20381] Updated weights for policy 1, policy_version 539681 (0.0009) [2023-12-27 09:22:32,827][20381] Updated weights for policy 1, policy_version 539691 (0.0010) [2023-12-27 09:22:33,228][20347] Updated weights for policy 0, policy_version 540074 (0.0005) [2023-12-27 09:22:33,278][20347] Updated weights for policy 0, policy_version 540084 (0.0005) [2023-12-27 09:22:33,323][20347] Updated weights for policy 0, policy_version 540094 (0.0005) [2023-12-27 09:22:33,495][20381] Updated weights for policy 1, policy_version 539701 (0.0010) [2023-12-27 09:22:33,561][20381] Updated weights for policy 1, policy_version 539711 (0.0010) [2023-12-27 09:22:33,624][20381] Updated weights for policy 1, policy_version 539721 (0.0010) [2023-12-27 09:22:33,909][20347] Updated weights for policy 0, policy_version 540104 (0.0005) [2023-12-27 09:22:33,962][20347] Updated weights for policy 0, policy_version 540114 (0.0008) [2023-12-27 09:22:34,016][20347] Updated weights for policy 0, policy_version 540124 (0.0010) [2023-12-27 09:22:34,291][20381] Updated weights for policy 1, policy_version 539731 (0.0010) [2023-12-27 09:22:34,352][20381] Updated weights for policy 1, policy_version 539741 (0.0007) [2023-12-27 09:22:34,416][20381] Updated weights for policy 1, policy_version 539751 (0.0007) [2023-12-27 09:22:34,730][20347] Updated weights for policy 0, policy_version 540134 (0.0010) [2023-12-27 09:22:34,781][20347] Updated weights for policy 0, policy_version 540144 (0.0007) [2023-12-27 09:22:34,841][20347] Updated weights for policy 0, policy_version 540154 (0.0005) [2023-12-27 09:22:35,026][20381] Updated weights for policy 1, policy_version 539761 (0.0007) [2023-12-27 09:22:35,074][20381] Updated weights for policy 1, policy_version 539771 (0.0009) [2023-12-27 09:22:35,125][20381] Updated weights for policy 1, policy_version 539781 (0.0010) [2023-12-27 09:22:35,183][20381] Updated weights for policy 1, policy_version 539791 (0.0007) [2023-12-27 09:22:35,466][20347] Updated weights for policy 0, policy_version 540164 (0.0006) [2023-12-27 09:22:35,522][20347] Updated weights for policy 0, policy_version 540174 (0.0006) [2023-12-27 09:22:35,572][20347] Updated weights for policy 0, policy_version 540184 (0.0010) [2023-12-27 09:22:35,765][20381] Updated weights for policy 1, policy_version 539801 (0.0010) [2023-12-27 09:22:35,825][20381] Updated weights for policy 1, policy_version 539811 (0.0010) [2023-12-27 09:22:35,890][20381] Updated weights for policy 1, policy_version 539821 (0.0008) [2023-12-27 09:22:36,172][20347] Updated weights for policy 0, policy_version 540194 (0.0010) [2023-12-27 09:22:36,222][20347] Updated weights for policy 0, policy_version 540204 (0.0007) [2023-12-27 09:22:36,270][20347] Updated weights for policy 0, policy_version 540214 (0.0005) [2023-12-27 09:22:36,328][20347] Updated weights for policy 0, policy_version 540224 (0.0007) [2023-12-27 09:22:36,562][20381] Updated weights for policy 1, policy_version 539831 (0.0011) [2023-12-27 09:22:36,622][20381] Updated weights for policy 1, policy_version 539841 (0.0011) [2023-12-27 09:22:36,675][20381] Updated weights for policy 1, policy_version 539851 (0.0008) [2023-12-27 09:22:37,035][20347] Updated weights for policy 0, policy_version 540234 (0.0007) [2023-12-27 09:22:37,092][20347] Updated weights for policy 0, policy_version 540244 (0.0007) [2023-12-27 09:22:37,148][20347] Updated weights for policy 0, policy_version 540254 (0.0007) [2023-12-27 09:22:37,284][20381] Updated weights for policy 1, policy_version 539861 (0.0005) [2023-12-27 09:22:37,348][20381] Updated weights for policy 1, policy_version 539871 (0.0008) [2023-12-27 09:22:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.5, 300 sec: 20743.8). Total num frames: 276545536. Throughput: 0: 10490.4, 1: 10425.8. Samples: 276515212. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:37,364][19170] Avg episode reward: [(0, '576.590'), (1, '626.540')] [2023-12-27 09:22:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000540256_138321920.pth... [2023-12-27 09:22:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000539040_138010624.pth [2023-12-27 09:22:37,405][20381] Updated weights for policy 1, policy_version 539881 (0.0011) [2023-12-27 09:22:37,441][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000539888_138231808.pth... [2023-12-27 09:22:37,444][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000538672_137920512.pth [2023-12-27 09:22:37,856][20347] Updated weights for policy 0, policy_version 540264 (0.0006) [2023-12-27 09:22:37,905][20347] Updated weights for policy 0, policy_version 540274 (0.0007) [2023-12-27 09:22:37,953][20347] Updated weights for policy 0, policy_version 540284 (0.0006) [2023-12-27 09:22:38,069][20381] Updated weights for policy 1, policy_version 539891 (0.0011) [2023-12-27 09:22:38,118][20381] Updated weights for policy 1, policy_version 539901 (0.0011) [2023-12-27 09:22:38,180][20381] Updated weights for policy 1, policy_version 539911 (0.0011) [2023-12-27 09:22:38,589][20347] Updated weights for policy 0, policy_version 540294 (0.0008) [2023-12-27 09:22:38,650][20347] Updated weights for policy 0, policy_version 540304 (0.0006) [2023-12-27 09:22:38,711][20347] Updated weights for policy 0, policy_version 540314 (0.0010) [2023-12-27 09:22:38,823][20381] Updated weights for policy 1, policy_version 539921 (0.0010) [2023-12-27 09:22:38,876][20381] Updated weights for policy 1, policy_version 539931 (0.0007) [2023-12-27 09:22:38,927][20381] Updated weights for policy 1, policy_version 539941 (0.0007) [2023-12-27 09:22:38,972][20381] Updated weights for policy 1, policy_version 539951 (0.0007) [2023-12-27 09:22:39,447][20347] Updated weights for policy 0, policy_version 540324 (0.0010) [2023-12-27 09:22:39,500][20347] Updated weights for policy 0, policy_version 540334 (0.0011) [2023-12-27 09:22:39,559][20347] Updated weights for policy 0, policy_version 540344 (0.0010) [2023-12-27 09:22:39,673][20381] Updated weights for policy 1, policy_version 539961 (0.0008) [2023-12-27 09:22:39,731][20381] Updated weights for policy 1, policy_version 539971 (0.0008) [2023-12-27 09:22:39,790][20381] Updated weights for policy 1, policy_version 539981 (0.0007) [2023-12-27 09:22:40,263][20347] Updated weights for policy 0, policy_version 540354 (0.0010) [2023-12-27 09:22:40,322][20347] Updated weights for policy 0, policy_version 540364 (0.0011) [2023-12-27 09:22:40,384][20347] Updated weights for policy 0, policy_version 540374 (0.0009) [2023-12-27 09:22:40,442][20347] Updated weights for policy 0, policy_version 540384 (0.0011) [2023-12-27 09:22:40,477][20381] Updated weights for policy 1, policy_version 539991 (0.0007) [2023-12-27 09:22:40,530][20381] Updated weights for policy 1, policy_version 540001 (0.0008) [2023-12-27 09:22:40,587][20381] Updated weights for policy 1, policy_version 540011 (0.0008) [2023-12-27 09:22:41,127][20347] Updated weights for policy 0, policy_version 540394 (0.0011) [2023-12-27 09:22:41,193][20347] Updated weights for policy 0, policy_version 540404 (0.0008) [2023-12-27 09:22:41,214][20381] Updated weights for policy 1, policy_version 540021 (0.0006) [2023-12-27 09:22:41,252][20347] Updated weights for policy 0, policy_version 540414 (0.0011) [2023-12-27 09:22:41,276][20381] Updated weights for policy 1, policy_version 540031 (0.0006) [2023-12-27 09:22:41,336][20381] Updated weights for policy 1, policy_version 540041 (0.0006) [2023-12-27 09:22:41,971][20347] Updated weights for policy 0, policy_version 540424 (0.0010) [2023-12-27 09:22:42,028][20347] Updated weights for policy 0, policy_version 540434 (0.0010) [2023-12-27 09:22:42,042][20381] Updated weights for policy 1, policy_version 540051 (0.0009) [2023-12-27 09:22:42,080][20347] Updated weights for policy 0, policy_version 540444 (0.0010) [2023-12-27 09:22:42,096][20381] Updated weights for policy 1, policy_version 540061 (0.0011) [2023-12-27 09:22:42,152][20381] Updated weights for policy 1, policy_version 540071 (0.0009) [2023-12-27 09:22:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 276652032. Throughput: 0: 10431.3, 1: 10451.4. Samples: 276640400. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:42,363][19170] Avg episode reward: [(0, '696.180'), (1, '698.600')] [2023-12-27 09:22:42,761][20347] Updated weights for policy 0, policy_version 540454 (0.0010) [2023-12-27 09:22:42,823][20347] Updated weights for policy 0, policy_version 540464 (0.0008) [2023-12-27 09:22:42,838][20381] Updated weights for policy 1, policy_version 540081 (0.0010) [2023-12-27 09:22:42,881][20347] Updated weights for policy 0, policy_version 540474 (0.0010) [2023-12-27 09:22:42,896][20381] Updated weights for policy 1, policy_version 540091 (0.0005) [2023-12-27 09:22:42,949][20381] Updated weights for policy 1, policy_version 540101 (0.0005) [2023-12-27 09:22:43,006][20381] Updated weights for policy 1, policy_version 540111 (0.0006) [2023-12-27 09:22:43,548][20347] Updated weights for policy 0, policy_version 540484 (0.0008) [2023-12-27 09:22:43,606][20347] Updated weights for policy 0, policy_version 540494 (0.0010) [2023-12-27 09:22:43,608][20381] Updated weights for policy 1, policy_version 540121 (0.0009) [2023-12-27 09:22:43,658][20347] Updated weights for policy 0, policy_version 540504 (0.0010) [2023-12-27 09:22:43,663][20381] Updated weights for policy 1, policy_version 540131 (0.0011) [2023-12-27 09:22:43,723][20381] Updated weights for policy 1, policy_version 540141 (0.0010) [2023-12-27 09:22:44,260][20347] Updated weights for policy 0, policy_version 540514 (0.0009) [2023-12-27 09:22:44,303][20381] Updated weights for policy 1, policy_version 540151 (0.0007) [2023-12-27 09:22:44,312][20347] Updated weights for policy 0, policy_version 540524 (0.0005) [2023-12-27 09:22:44,359][20347] Updated weights for policy 0, policy_version 540534 (0.0005) [2023-12-27 09:22:44,362][20381] Updated weights for policy 1, policy_version 540161 (0.0011) [2023-12-27 09:22:44,406][20347] Updated weights for policy 0, policy_version 540544 (0.0008) [2023-12-27 09:22:44,413][20381] Updated weights for policy 1, policy_version 540171 (0.0010) [2023-12-27 09:22:45,016][20347] Updated weights for policy 0, policy_version 540554 (0.0011) [2023-12-27 09:22:45,079][20347] Updated weights for policy 0, policy_version 540564 (0.0010) [2023-12-27 09:22:45,103][20381] Updated weights for policy 1, policy_version 540181 (0.0010) [2023-12-27 09:22:45,139][20347] Updated weights for policy 0, policy_version 540574 (0.0010) [2023-12-27 09:22:45,156][20381] Updated weights for policy 1, policy_version 540191 (0.0009) [2023-12-27 09:22:45,218][20381] Updated weights for policy 1, policy_version 540201 (0.0009) [2023-12-27 09:22:45,802][20381] Updated weights for policy 1, policy_version 540211 (0.0009) [2023-12-27 09:22:45,821][20347] Updated weights for policy 0, policy_version 540584 (0.0010) [2023-12-27 09:22:45,851][20381] Updated weights for policy 1, policy_version 540221 (0.0005) [2023-12-27 09:22:45,873][20347] Updated weights for policy 0, policy_version 540594 (0.0010) [2023-12-27 09:22:45,899][20381] Updated weights for policy 1, policy_version 540231 (0.0005) [2023-12-27 09:22:45,917][20347] Updated weights for policy 0, policy_version 540604 (0.0010) [2023-12-27 09:22:46,547][20381] Updated weights for policy 1, policy_version 540241 (0.0008) [2023-12-27 09:22:46,597][20381] Updated weights for policy 1, policy_version 540251 (0.0005) [2023-12-27 09:22:46,598][20347] Updated weights for policy 0, policy_version 540614 (0.0010) [2023-12-27 09:22:46,645][20381] Updated weights for policy 1, policy_version 540261 (0.0005) [2023-12-27 09:22:46,646][20347] Updated weights for policy 0, policy_version 540624 (0.0010) [2023-12-27 09:22:46,700][20381] Updated weights for policy 1, policy_version 540271 (0.0005) [2023-12-27 09:22:46,700][20347] Updated weights for policy 0, policy_version 540634 (0.0010) [2023-12-27 09:22:47,269][20381] Updated weights for policy 1, policy_version 540281 (0.0006) [2023-12-27 09:22:47,337][20381] Updated weights for policy 1, policy_version 540291 (0.0006) [2023-12-27 09:22:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 276750336. Throughput: 0: 10452.5, 1: 10513.8. Samples: 276767796. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:47,363][19170] Avg episode reward: [(0, '606.540'), (1, '786.340')] [2023-12-27 09:22:47,364][20347] Updated weights for policy 0, policy_version 540644 (0.0010) [2023-12-27 09:22:47,388][20381] Updated weights for policy 1, policy_version 540301 (0.0010) [2023-12-27 09:22:47,416][20347] Updated weights for policy 0, policy_version 540654 (0.0010) [2023-12-27 09:22:47,472][20347] Updated weights for policy 0, policy_version 540664 (0.0010) [2023-12-27 09:22:47,913][20381] Updated weights for policy 1, policy_version 540311 (0.0011) [2023-12-27 09:22:47,971][20381] Updated weights for policy 1, policy_version 540321 (0.0010) [2023-12-27 09:22:48,028][20381] Updated weights for policy 1, policy_version 540331 (0.0010) [2023-12-27 09:22:48,142][20347] Updated weights for policy 0, policy_version 540674 (0.0011) [2023-12-27 09:22:48,200][20347] Updated weights for policy 0, policy_version 540684 (0.0010) [2023-12-27 09:22:48,267][20347] Updated weights for policy 0, policy_version 540694 (0.0010) [2023-12-27 09:22:48,333][20347] Updated weights for policy 0, policy_version 540704 (0.0010) [2023-12-27 09:22:48,692][20381] Updated weights for policy 1, policy_version 540341 (0.0008) [2023-12-27 09:22:48,748][20381] Updated weights for policy 1, policy_version 540351 (0.0011) [2023-12-27 09:22:48,797][20381] Updated weights for policy 1, policy_version 540361 (0.0011) [2023-12-27 09:22:49,032][20347] Updated weights for policy 0, policy_version 540714 (0.0005) [2023-12-27 09:22:49,091][20347] Updated weights for policy 0, policy_version 540724 (0.0005) [2023-12-27 09:22:49,154][20347] Updated weights for policy 0, policy_version 540734 (0.0005) [2023-12-27 09:22:49,500][20381] Updated weights for policy 1, policy_version 540371 (0.0011) [2023-12-27 09:22:49,556][20381] Updated weights for policy 1, policy_version 540381 (0.0011) [2023-12-27 09:22:49,609][20381] Updated weights for policy 1, policy_version 540391 (0.0011) [2023-12-27 09:22:49,764][20347] Updated weights for policy 0, policy_version 540744 (0.0009) [2023-12-27 09:22:49,825][20347] Updated weights for policy 0, policy_version 540754 (0.0010) [2023-12-27 09:22:49,888][20347] Updated weights for policy 0, policy_version 540764 (0.0011) [2023-12-27 09:22:50,346][20381] Updated weights for policy 1, policy_version 540401 (0.0010) [2023-12-27 09:22:50,398][20381] Updated weights for policy 1, policy_version 540411 (0.0007) [2023-12-27 09:22:50,451][20381] Updated weights for policy 1, policy_version 540421 (0.0008) [2023-12-27 09:22:50,501][20381] Updated weights for policy 1, policy_version 540431 (0.0006) [2023-12-27 09:22:50,539][20347] Updated weights for policy 0, policy_version 540774 (0.0008) [2023-12-27 09:22:50,595][20347] Updated weights for policy 0, policy_version 540784 (0.0009) [2023-12-27 09:22:50,653][20347] Updated weights for policy 0, policy_version 540794 (0.0006) [2023-12-27 09:22:51,230][20381] Updated weights for policy 1, policy_version 540441 (0.0008) [2023-12-27 09:22:51,292][20381] Updated weights for policy 1, policy_version 540451 (0.0008) [2023-12-27 09:22:51,322][20347] Updated weights for policy 0, policy_version 540804 (0.0008) [2023-12-27 09:22:51,357][20381] Updated weights for policy 1, policy_version 540461 (0.0007) [2023-12-27 09:22:51,386][20347] Updated weights for policy 0, policy_version 540814 (0.0008) [2023-12-27 09:22:51,450][20347] Updated weights for policy 0, policy_version 540824 (0.0005) [2023-12-27 09:22:52,078][20381] Updated weights for policy 1, policy_version 540471 (0.0010) [2023-12-27 09:22:52,124][20347] Updated weights for policy 0, policy_version 540834 (0.0006) [2023-12-27 09:22:52,134][20381] Updated weights for policy 1, policy_version 540481 (0.0011) [2023-12-27 09:22:52,182][20347] Updated weights for policy 0, policy_version 540844 (0.0007) [2023-12-27 09:22:52,191][20381] Updated weights for policy 1, policy_version 540491 (0.0011) [2023-12-27 09:22:52,244][20347] Updated weights for policy 0, policy_version 540854 (0.0005) [2023-12-27 09:22:52,299][20347] Updated weights for policy 0, policy_version 540864 (0.0010) [2023-12-27 09:22:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 276865024. Throughput: 0: 10409.6, 1: 10462.7. Samples: 276830212. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:52,363][19170] Avg episode reward: [(0, '601.880'), (1, '776.800')] [2023-12-27 09:22:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000540496_138387456.pth... [2023-12-27 09:22:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000540864_138477568.pth... [2023-12-27 09:22:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000539280_138076160.pth [2023-12-27 09:22:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000539648_138166272.pth [2023-12-27 09:22:52,894][20381] Updated weights for policy 1, policy_version 540501 (0.0010) [2023-12-27 09:22:52,948][20381] Updated weights for policy 1, policy_version 540511 (0.0010) [2023-12-27 09:22:53,000][20347] Updated weights for policy 0, policy_version 540874 (0.0005) [2023-12-27 09:22:53,002][20381] Updated weights for policy 1, policy_version 540521 (0.0011) [2023-12-27 09:22:53,053][20347] Updated weights for policy 0, policy_version 540884 (0.0006) [2023-12-27 09:22:53,098][20347] Updated weights for policy 0, policy_version 540894 (0.0007) [2023-12-27 09:22:53,648][20381] Updated weights for policy 1, policy_version 540531 (0.0009) [2023-12-27 09:22:53,707][20381] Updated weights for policy 1, policy_version 540541 (0.0009) [2023-12-27 09:22:53,745][20347] Updated weights for policy 0, policy_version 540904 (0.0009) [2023-12-27 09:22:53,766][20381] Updated weights for policy 1, policy_version 540551 (0.0009) [2023-12-27 09:22:53,800][20347] Updated weights for policy 0, policy_version 540914 (0.0007) [2023-12-27 09:22:53,854][20347] Updated weights for policy 0, policy_version 540924 (0.0005) [2023-12-27 09:22:54,425][20347] Updated weights for policy 0, policy_version 540934 (0.0005) [2023-12-27 09:22:54,439][20381] Updated weights for policy 1, policy_version 540561 (0.0011) [2023-12-27 09:22:54,488][20347] Updated weights for policy 0, policy_version 540944 (0.0008) [2023-12-27 09:22:54,492][20381] Updated weights for policy 1, policy_version 540571 (0.0011) [2023-12-27 09:22:54,537][20347] Updated weights for policy 0, policy_version 540954 (0.0007) [2023-12-27 09:22:54,551][20381] Updated weights for policy 1, policy_version 540581 (0.0011) [2023-12-27 09:22:54,607][20381] Updated weights for policy 1, policy_version 540591 (0.0011) [2023-12-27 09:22:55,125][20347] Updated weights for policy 0, policy_version 540964 (0.0007) [2023-12-27 09:22:55,173][20347] Updated weights for policy 0, policy_version 540974 (0.0005) [2023-12-27 09:22:55,219][20347] Updated weights for policy 0, policy_version 540984 (0.0005) [2023-12-27 09:22:55,278][20381] Updated weights for policy 1, policy_version 540601 (0.0011) [2023-12-27 09:22:55,323][20381] Updated weights for policy 1, policy_version 540611 (0.0010) [2023-12-27 09:22:55,368][20381] Updated weights for policy 1, policy_version 540621 (0.0010) [2023-12-27 09:22:55,886][20347] Updated weights for policy 0, policy_version 540994 (0.0010) [2023-12-27 09:22:55,947][20347] Updated weights for policy 0, policy_version 541004 (0.0010) [2023-12-27 09:22:55,999][20347] Updated weights for policy 0, policy_version 541014 (0.0010) [2023-12-27 09:22:56,050][20347] Updated weights for policy 0, policy_version 541024 (0.0010) [2023-12-27 09:22:56,082][20381] Updated weights for policy 1, policy_version 540631 (0.0010) [2023-12-27 09:22:56,137][20381] Updated weights for policy 1, policy_version 540641 (0.0010) [2023-12-27 09:22:56,188][20381] Updated weights for policy 1, policy_version 540651 (0.0010) [2023-12-27 09:22:56,737][20347] Updated weights for policy 0, policy_version 541034 (0.0007) [2023-12-27 09:22:56,798][20347] Updated weights for policy 0, policy_version 541044 (0.0006) [2023-12-27 09:22:56,861][20347] Updated weights for policy 0, policy_version 541054 (0.0005) [2023-12-27 09:22:56,882][20381] Updated weights for policy 1, policy_version 540661 (0.0010) [2023-12-27 09:22:56,937][20381] Updated weights for policy 1, policy_version 540671 (0.0007) [2023-12-27 09:22:57,003][20381] Updated weights for policy 1, policy_version 540681 (0.0008) [2023-12-27 09:22:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 276963328. Throughput: 0: 10442.9, 1: 10398.3. Samples: 276954320. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:22:57,363][19170] Avg episode reward: [(0, '743.560'), (1, '815.070')] [2023-12-27 09:22:57,379][20347] Updated weights for policy 0, policy_version 541064 (0.0006) [2023-12-27 09:22:57,443][20347] Updated weights for policy 0, policy_version 541074 (0.0005) [2023-12-27 09:22:57,502][20347] Updated weights for policy 0, policy_version 541084 (0.0006) [2023-12-27 09:22:57,640][20381] Updated weights for policy 1, policy_version 540691 (0.0007) [2023-12-27 09:22:57,700][20381] Updated weights for policy 1, policy_version 540701 (0.0005) [2023-12-27 09:22:57,763][20381] Updated weights for policy 1, policy_version 540711 (0.0007) [2023-12-27 09:22:58,101][20347] Updated weights for policy 0, policy_version 541094 (0.0007) [2023-12-27 09:22:58,148][20347] Updated weights for policy 0, policy_version 541104 (0.0010) [2023-12-27 09:22:58,213][20347] Updated weights for policy 0, policy_version 541114 (0.0008) [2023-12-27 09:22:58,381][20381] Updated weights for policy 1, policy_version 540721 (0.0006) [2023-12-27 09:22:58,438][20381] Updated weights for policy 1, policy_version 540731 (0.0007) [2023-12-27 09:22:58,498][20381] Updated weights for policy 1, policy_version 540741 (0.0007) [2023-12-27 09:22:58,560][20381] Updated weights for policy 1, policy_version 540751 (0.0008) [2023-12-27 09:22:58,955][20347] Updated weights for policy 0, policy_version 541124 (0.0008) [2023-12-27 09:22:59,013][20347] Updated weights for policy 0, policy_version 541134 (0.0006) [2023-12-27 09:22:59,074][20347] Updated weights for policy 0, policy_version 541144 (0.0008) [2023-12-27 09:22:59,178][20381] Updated weights for policy 1, policy_version 540761 (0.0006) [2023-12-27 09:22:59,237][20381] Updated weights for policy 1, policy_version 540771 (0.0008) [2023-12-27 09:22:59,299][20381] Updated weights for policy 1, policy_version 540781 (0.0008) [2023-12-27 09:22:59,739][20347] Updated weights for policy 0, policy_version 541154 (0.0009) [2023-12-27 09:22:59,790][20347] Updated weights for policy 0, policy_version 541164 (0.0008) [2023-12-27 09:22:59,841][20347] Updated weights for policy 0, policy_version 541174 (0.0008) [2023-12-27 09:22:59,905][20347] Updated weights for policy 0, policy_version 541184 (0.0008) [2023-12-27 09:23:00,028][20381] Updated weights for policy 1, policy_version 540791 (0.0008) [2023-12-27 09:23:00,082][20381] Updated weights for policy 1, policy_version 540801 (0.0008) [2023-12-27 09:23:00,137][20381] Updated weights for policy 1, policy_version 540811 (0.0009) [2023-12-27 09:23:00,574][20347] Updated weights for policy 0, policy_version 541194 (0.0005) [2023-12-27 09:23:00,621][20347] Updated weights for policy 0, policy_version 541204 (0.0007) [2023-12-27 09:23:00,675][20347] Updated weights for policy 0, policy_version 541214 (0.0007) [2023-12-27 09:23:00,865][20381] Updated weights for policy 1, policy_version 540821 (0.0008) [2023-12-27 09:23:00,919][20381] Updated weights for policy 1, policy_version 540831 (0.0008) [2023-12-27 09:23:00,975][20381] Updated weights for policy 1, policy_version 540841 (0.0007) [2023-12-27 09:23:01,303][20347] Updated weights for policy 0, policy_version 541224 (0.0008) [2023-12-27 09:23:01,382][20347] Updated weights for policy 0, policy_version 541234 (0.0009) [2023-12-27 09:23:01,443][20347] Updated weights for policy 0, policy_version 541244 (0.0006) [2023-12-27 09:23:01,619][20381] Updated weights for policy 1, policy_version 540851 (0.0008) [2023-12-27 09:23:01,678][20381] Updated weights for policy 1, policy_version 540861 (0.0006) [2023-12-27 09:23:01,740][20381] Updated weights for policy 1, policy_version 540871 (0.0007) [2023-12-27 09:23:02,141][20347] Updated weights for policy 0, policy_version 541254 (0.0006) [2023-12-27 09:23:02,192][20347] Updated weights for policy 0, policy_version 541264 (0.0007) [2023-12-27 09:23:02,237][20347] Updated weights for policy 0, policy_version 541274 (0.0007) [2023-12-27 09:23:02,296][20381] Updated weights for policy 1, policy_version 540881 (0.0008) [2023-12-27 09:23:02,357][20381] Updated weights for policy 1, policy_version 540891 (0.0008) [2023-12-27 09:23:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 277069824. Throughput: 0: 10453.1, 1: 10377.4. Samples: 277081128. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:23:02,363][19170] Avg episode reward: [(0, '724.060'), (1, '766.470')] [2023-12-27 09:23:02,413][20381] Updated weights for policy 1, policy_version 540901 (0.0008) [2023-12-27 09:23:02,470][20381] Updated weights for policy 1, policy_version 540911 (0.0007) [2023-12-27 09:23:02,993][20347] Updated weights for policy 0, policy_version 541284 (0.0008) [2023-12-27 09:23:03,048][20347] Updated weights for policy 0, policy_version 541294 (0.0008) [2023-12-27 09:23:03,101][20347] Updated weights for policy 0, policy_version 541304 (0.0007) [2023-12-27 09:23:03,103][20381] Updated weights for policy 1, policy_version 540921 (0.0008) [2023-12-27 09:23:03,155][20381] Updated weights for policy 1, policy_version 540931 (0.0010) [2023-12-27 09:23:03,205][20381] Updated weights for policy 1, policy_version 540941 (0.0007) [2023-12-27 09:23:03,725][20347] Updated weights for policy 0, policy_version 541314 (0.0006) [2023-12-27 09:23:03,770][20347] Updated weights for policy 0, policy_version 541324 (0.0008) [2023-12-27 09:23:03,793][20381] Updated weights for policy 1, policy_version 540951 (0.0005) [2023-12-27 09:23:03,828][20347] Updated weights for policy 0, policy_version 541334 (0.0006) [2023-12-27 09:23:03,845][20381] Updated weights for policy 1, policy_version 540961 (0.0008) [2023-12-27 09:23:03,893][20347] Updated weights for policy 0, policy_version 541344 (0.0007) [2023-12-27 09:23:03,904][20381] Updated weights for policy 1, policy_version 540971 (0.0007) [2023-12-27 09:23:04,534][20381] Updated weights for policy 1, policy_version 540981 (0.0007) [2023-12-27 09:23:04,590][20381] Updated weights for policy 1, policy_version 540991 (0.0008) [2023-12-27 09:23:04,629][20347] Updated weights for policy 0, policy_version 541354 (0.0006) [2023-12-27 09:23:04,646][20381] Updated weights for policy 1, policy_version 541001 (0.0006) [2023-12-27 09:23:04,685][20347] Updated weights for policy 0, policy_version 541364 (0.0006) [2023-12-27 09:23:04,741][20347] Updated weights for policy 0, policy_version 541374 (0.0008) [2023-12-27 09:23:05,264][20381] Updated weights for policy 1, policy_version 541011 (0.0007) [2023-12-27 09:23:05,316][20381] Updated weights for policy 1, policy_version 541021 (0.0010) [2023-12-27 09:23:05,361][20381] Updated weights for policy 1, policy_version 541031 (0.0010) [2023-12-27 09:23:05,458][20347] Updated weights for policy 0, policy_version 541384 (0.0008) [2023-12-27 09:23:05,511][20347] Updated weights for policy 0, policy_version 541394 (0.0007) [2023-12-27 09:23:05,562][20347] Updated weights for policy 0, policy_version 541404 (0.0007) [2023-12-27 09:23:06,013][20381] Updated weights for policy 1, policy_version 541041 (0.0010) [2023-12-27 09:23:06,067][20381] Updated weights for policy 1, policy_version 541051 (0.0009) [2023-12-27 09:23:06,123][20381] Updated weights for policy 1, policy_version 541061 (0.0010) [2023-12-27 09:23:06,183][20381] Updated weights for policy 1, policy_version 541071 (0.0010) [2023-12-27 09:23:06,285][20347] Updated weights for policy 0, policy_version 541414 (0.0007) [2023-12-27 09:23:06,347][20347] Updated weights for policy 0, policy_version 541424 (0.0008) [2023-12-27 09:23:06,409][20347] Updated weights for policy 0, policy_version 541434 (0.0008) [2023-12-27 09:23:06,836][20381] Updated weights for policy 1, policy_version 541081 (0.0006) [2023-12-27 09:23:06,893][20381] Updated weights for policy 1, policy_version 541091 (0.0005) [2023-12-27 09:23:06,949][20381] Updated weights for policy 1, policy_version 541101 (0.0005) [2023-12-27 09:23:07,152][20347] Updated weights for policy 0, policy_version 541444 (0.0010) [2023-12-27 09:23:07,200][20347] Updated weights for policy 0, policy_version 541454 (0.0009) [2023-12-27 09:23:07,246][20347] Updated weights for policy 0, policy_version 541464 (0.0010) [2023-12-27 09:23:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 277176320. Throughput: 0: 10410.6, 1: 10456.6. Samples: 277144076. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:23:07,363][19170] Avg episode reward: [(0, '725.180'), (1, '782.860')] [2023-12-27 09:23:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000541104_138543104.pth... [2023-12-27 09:23:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000541472_138633216.pth... [2023-12-27 09:23:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000539888_138231808.pth [2023-12-27 09:23:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000540256_138321920.pth [2023-12-27 09:23:07,486][20381] Updated weights for policy 1, policy_version 541111 (0.0007) [2023-12-27 09:23:07,538][20381] Updated weights for policy 1, policy_version 541121 (0.0007) [2023-12-27 09:23:07,584][20381] Updated weights for policy 1, policy_version 541131 (0.0007) [2023-12-27 09:23:07,966][20347] Updated weights for policy 0, policy_version 541474 (0.0010) [2023-12-27 09:23:08,014][20347] Updated weights for policy 0, policy_version 541484 (0.0008) [2023-12-27 09:23:08,064][20347] Updated weights for policy 0, policy_version 541494 (0.0007) [2023-12-27 09:23:08,123][20347] Updated weights for policy 0, policy_version 541504 (0.0008) [2023-12-27 09:23:08,135][20381] Updated weights for policy 1, policy_version 541141 (0.0005) [2023-12-27 09:23:08,182][20381] Updated weights for policy 1, policy_version 541151 (0.0005) [2023-12-27 09:23:08,229][20381] Updated weights for policy 1, policy_version 541161 (0.0005) [2023-12-27 09:23:08,856][20381] Updated weights for policy 1, policy_version 541171 (0.0005) [2023-12-27 09:23:08,893][20347] Updated weights for policy 0, policy_version 541514 (0.0007) [2023-12-27 09:23:08,910][20381] Updated weights for policy 1, policy_version 541181 (0.0007) [2023-12-27 09:23:08,912][20292] KL-divergence is very high: 135.0883 [2023-12-27 09:23:08,953][20381] Updated weights for policy 1, policy_version 541191 (0.0005) [2023-12-27 09:23:08,957][20347] Updated weights for policy 0, policy_version 541524 (0.0007) [2023-12-27 09:23:09,012][20347] Updated weights for policy 0, policy_version 541534 (0.0008) [2023-12-27 09:23:09,591][20381] Updated weights for policy 1, policy_version 541201 (0.0005) [2023-12-27 09:23:09,601][20292] KL-divergence is very high: 144.2037 [2023-12-27 09:23:09,655][20381] Updated weights for policy 1, policy_version 541211 (0.0006) [2023-12-27 09:23:09,716][20381] Updated weights for policy 1, policy_version 541221 (0.0008) [2023-12-27 09:23:09,749][20292] KL-divergence is very high: 107.3964 [2023-12-27 09:23:09,752][20347] Updated weights for policy 0, policy_version 541544 (0.0006) [2023-12-27 09:23:09,780][20381] Updated weights for policy 1, policy_version 541231 (0.0007) [2023-12-27 09:23:09,810][20347] Updated weights for policy 0, policy_version 541554 (0.0006) [2023-12-27 09:23:09,876][20347] Updated weights for policy 0, policy_version 541564 (0.0008) [2023-12-27 09:23:10,368][20381] Updated weights for policy 1, policy_version 541241 (0.0007) [2023-12-27 09:23:10,425][20381] Updated weights for policy 1, policy_version 541251 (0.0008) [2023-12-27 09:23:10,473][20381] Updated weights for policy 1, policy_version 541261 (0.0007) [2023-12-27 09:23:10,591][20347] Updated weights for policy 0, policy_version 541574 (0.0008) [2023-12-27 09:23:10,647][20347] Updated weights for policy 0, policy_version 541584 (0.0008) [2023-12-27 09:23:10,700][20347] Updated weights for policy 0, policy_version 541594 (0.0008) [2023-12-27 09:23:11,183][20381] Updated weights for policy 1, policy_version 541271 (0.0008) [2023-12-27 09:23:11,234][20381] Updated weights for policy 1, policy_version 541281 (0.0008) [2023-12-27 09:23:11,293][20381] Updated weights for policy 1, policy_version 541291 (0.0008) [2023-12-27 09:23:11,436][20347] Updated weights for policy 0, policy_version 541604 (0.0007) [2023-12-27 09:23:11,493][20347] Updated weights for policy 0, policy_version 541614 (0.0008) [2023-12-27 09:23:11,553][20347] Updated weights for policy 0, policy_version 541624 (0.0007) [2023-12-27 09:23:12,084][20381] Updated weights for policy 1, policy_version 541301 (0.0008) [2023-12-27 09:23:12,137][20381] Updated weights for policy 1, policy_version 541311 (0.0007) [2023-12-27 09:23:12,199][20381] Updated weights for policy 1, policy_version 541321 (0.0008) [2023-12-27 09:23:12,229][20347] Updated weights for policy 0, policy_version 541634 (0.0007) [2023-12-27 09:23:12,289][20347] Updated weights for policy 0, policy_version 541644 (0.0008) [2023-12-27 09:23:12,349][20347] Updated weights for policy 0, policy_version 541654 (0.0008) [2023-12-27 09:23:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 277274624. Throughput: 0: 10291.3, 1: 10580.7. Samples: 277267520. Policy #0 lag: (min: 28.0, avg: 34.0, max: 60.0) [2023-12-27 09:23:12,363][19170] Avg episode reward: [(0, '809.550'), (1, '810.260')] [2023-12-27 09:23:12,412][20347] Updated weights for policy 0, policy_version 541664 (0.0007) [2023-12-27 09:23:12,845][20381] Updated weights for policy 1, policy_version 541331 (0.0007) [2023-12-27 09:23:12,893][20381] Updated weights for policy 1, policy_version 541341 (0.0007) [2023-12-27 09:23:12,945][20381] Updated weights for policy 1, policy_version 541351 (0.0007) [2023-12-27 09:23:13,132][20347] Updated weights for policy 0, policy_version 541674 (0.0008) [2023-12-27 09:23:13,191][20347] Updated weights for policy 0, policy_version 541684 (0.0008) [2023-12-27 09:23:13,246][20347] Updated weights for policy 0, policy_version 541694 (0.0008) [2023-12-27 09:23:13,585][20381] Updated weights for policy 1, policy_version 541361 (0.0005) [2023-12-27 09:23:13,645][20381] Updated weights for policy 1, policy_version 541371 (0.0007) [2023-12-27 09:23:13,700][20381] Updated weights for policy 1, policy_version 541381 (0.0008) [2023-12-27 09:23:13,757][20381] Updated weights for policy 1, policy_version 541391 (0.0008) [2023-12-27 09:23:13,938][20347] Updated weights for policy 0, policy_version 541704 (0.0007) [2023-12-27 09:23:13,984][20347] Updated weights for policy 0, policy_version 541714 (0.0008) [2023-12-27 09:23:14,037][20347] Updated weights for policy 0, policy_version 541724 (0.0008) [2023-12-27 09:23:14,417][20381] Updated weights for policy 1, policy_version 541401 (0.0007) [2023-12-27 09:23:14,469][20381] Updated weights for policy 1, policy_version 541411 (0.0006) [2023-12-27 09:23:14,522][20381] Updated weights for policy 1, policy_version 541421 (0.0005) [2023-12-27 09:23:14,762][20347] Updated weights for policy 0, policy_version 541734 (0.0007) [2023-12-27 09:23:14,830][20347] Updated weights for policy 0, policy_version 541744 (0.0007) [2023-12-27 09:23:14,903][20347] Updated weights for policy 0, policy_version 541754 (0.0007) [2023-12-27 09:23:15,189][20381] Updated weights for policy 1, policy_version 541431 (0.0005) [2023-12-27 09:23:15,246][20381] Updated weights for policy 1, policy_version 541441 (0.0005) [2023-12-27 09:23:15,308][20381] Updated weights for policy 1, policy_version 541451 (0.0005) [2023-12-27 09:23:15,570][20347] Updated weights for policy 0, policy_version 541764 (0.0008) [2023-12-27 09:23:15,625][20347] Updated weights for policy 0, policy_version 541774 (0.0008) [2023-12-27 09:23:15,681][20347] Updated weights for policy 0, policy_version 541784 (0.0008) [2023-12-27 09:23:15,912][20381] Updated weights for policy 1, policy_version 541461 (0.0007) [2023-12-27 09:23:15,975][20381] Updated weights for policy 1, policy_version 541471 (0.0008) [2023-12-27 09:23:16,035][20381] Updated weights for policy 1, policy_version 541481 (0.0008) [2023-12-27 09:23:16,392][20347] Updated weights for policy 0, policy_version 541794 (0.0008) [2023-12-27 09:23:16,444][20347] Updated weights for policy 0, policy_version 541804 (0.0007) [2023-12-27 09:23:16,495][20347] Updated weights for policy 0, policy_version 541814 (0.0008) [2023-12-27 09:23:16,551][20347] Updated weights for policy 0, policy_version 541824 (0.0008) [2023-12-27 09:23:16,696][20381] Updated weights for policy 1, policy_version 541491 (0.0007) [2023-12-27 09:23:16,754][20381] Updated weights for policy 1, policy_version 541501 (0.0007) [2023-12-27 09:23:16,812][20381] Updated weights for policy 1, policy_version 541511 (0.0006) [2023-12-27 09:23:17,283][20347] Updated weights for policy 0, policy_version 541834 (0.0007) [2023-12-27 09:23:17,334][20347] Updated weights for policy 0, policy_version 541844 (0.0007) [2023-12-27 09:23:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 277372928. Throughput: 0: 10234.7, 1: 10599.1. Samples: 277389300. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:17,363][19170] Avg episode reward: [(0, '860.370'), (1, '762.550')] [2023-12-27 09:23:17,386][20347] Updated weights for policy 0, policy_version 541854 (0.0008) [2023-12-27 09:23:17,411][20381] Updated weights for policy 1, policy_version 541521 (0.0006) [2023-12-27 09:23:17,470][20381] Updated weights for policy 1, policy_version 541531 (0.0008) [2023-12-27 09:23:17,474][20292] KL-divergence is very high: 178.9666 [2023-12-27 09:23:17,485][20292] KL-divergence is very high: 216.5633 [2023-12-27 09:23:17,491][20292] KL-divergence is very high: 194.5601 [2023-12-27 09:23:17,497][20292] KL-divergence is very high: 118.5663 [2023-12-27 09:23:17,524][20381] Updated weights for policy 1, policy_version 541541 (0.0008) [2023-12-27 09:23:17,531][20292] KL-divergence is very high: 136.5355 [2023-12-27 09:23:17,537][20292] KL-divergence is very high: 225.2732 [2023-12-27 09:23:17,543][20292] KL-divergence is very high: 140.4950 [2023-12-27 09:23:17,579][20381] Updated weights for policy 1, policy_version 541551 (0.0007) [2023-12-27 09:23:18,104][20347] Updated weights for policy 0, policy_version 541864 (0.0007) [2023-12-27 09:23:18,169][20347] Updated weights for policy 0, policy_version 541874 (0.0008) [2023-12-27 09:23:18,227][20347] Updated weights for policy 0, policy_version 541884 (0.0007) [2023-12-27 09:23:18,239][20292] KL-divergence is very high: 128.3509 [2023-12-27 09:23:18,245][20292] KL-divergence is very high: 229.6418 [2023-12-27 09:23:18,252][20292] KL-divergence is very high: 113.5697 [2023-12-27 09:23:18,290][20381] Updated weights for policy 1, policy_version 541561 (0.0006) [2023-12-27 09:23:18,298][20292] KL-divergence is very high: 107.3870 [2023-12-27 09:23:18,305][20292] KL-divergence is very high: 170.4160 [2023-12-27 09:23:18,344][20292] KL-divergence is very high: 120.2725 [2023-12-27 09:23:18,350][20292] KL-divergence is very high: 177.8330 [2023-12-27 09:23:18,355][20381] Updated weights for policy 1, policy_version 541571 (0.0008) [2023-12-27 09:23:18,356][20292] KL-divergence is very high: 178.4688 [2023-12-27 09:23:18,395][20292] KL-divergence is very high: 161.3212 [2023-12-27 09:23:18,401][20292] KL-divergence is very high: 224.3363 [2023-12-27 09:23:18,408][20292] KL-divergence is very high: 193.7699 [2023-12-27 09:23:18,420][20381] Updated weights for policy 1, policy_version 541581 (0.0008) [2023-12-27 09:23:18,930][20347] Updated weights for policy 0, policy_version 541894 (0.0006) [2023-12-27 09:23:18,978][20347] Updated weights for policy 0, policy_version 541904 (0.0007) [2023-12-27 09:23:19,025][20347] Updated weights for policy 0, policy_version 541914 (0.0008) [2023-12-27 09:23:19,096][20381] Updated weights for policy 1, policy_version 541591 (0.0008) [2023-12-27 09:23:19,144][20381] Updated weights for policy 1, policy_version 541601 (0.0007) [2023-12-27 09:23:19,200][20381] Updated weights for policy 1, policy_version 541611 (0.0007) [2023-12-27 09:23:19,704][20347] Updated weights for policy 0, policy_version 541924 (0.0007) [2023-12-27 09:23:19,764][20347] Updated weights for policy 0, policy_version 541934 (0.0007) [2023-12-27 09:23:19,828][20347] Updated weights for policy 0, policy_version 541944 (0.0008) [2023-12-27 09:23:19,990][20381] Updated weights for policy 1, policy_version 541621 (0.0008) [2023-12-27 09:23:20,046][20381] Updated weights for policy 1, policy_version 541631 (0.0008) [2023-12-27 09:23:20,104][20381] Updated weights for policy 1, policy_version 541641 (0.0007) [2023-12-27 09:23:20,552][20347] Updated weights for policy 0, policy_version 541954 (0.0008) [2023-12-27 09:23:20,614][20347] Updated weights for policy 0, policy_version 541964 (0.0008) [2023-12-27 09:23:20,658][20347] Updated weights for policy 0, policy_version 541974 (0.0008) [2023-12-27 09:23:20,712][20347] Updated weights for policy 0, policy_version 541984 (0.0008) [2023-12-27 09:23:20,804][20381] Updated weights for policy 1, policy_version 541651 (0.0006) [2023-12-27 09:23:20,855][20381] Updated weights for policy 1, policy_version 541661 (0.0008) [2023-12-27 09:23:20,904][20381] Updated weights for policy 1, policy_version 541671 (0.0008) [2023-12-27 09:23:21,465][20347] Updated weights for policy 0, policy_version 541994 (0.0008) [2023-12-27 09:23:21,515][20347] Updated weights for policy 0, policy_version 542004 (0.0008) [2023-12-27 09:23:21,572][20347] Updated weights for policy 0, policy_version 542014 (0.0008) [2023-12-27 09:23:21,600][20381] Updated weights for policy 1, policy_version 541681 (0.0006) [2023-12-27 09:23:21,664][20381] Updated weights for policy 1, policy_version 541691 (0.0008) [2023-12-27 09:23:21,731][20381] Updated weights for policy 1, policy_version 541701 (0.0007) [2023-12-27 09:23:21,792][20381] Updated weights for policy 1, policy_version 541711 (0.0008) [2023-12-27 09:23:22,331][20347] Updated weights for policy 0, policy_version 542024 (0.0007) [2023-12-27 09:23:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 277471232. Throughput: 0: 10179.8, 1: 10554.3. Samples: 277448240. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:22,363][19170] Avg episode reward: [(0, '806.120'), (1, '614.390')] [2023-12-27 09:23:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000541712_138698752.pth... [2023-12-27 09:23:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000540496_138387456.pth [2023-12-27 09:23:22,404][20347] Updated weights for policy 0, policy_version 542034 (0.0008) [2023-12-27 09:23:22,434][20381] Updated weights for policy 1, policy_version 541721 (0.0006) [2023-12-27 09:23:22,466][20347] Updated weights for policy 0, policy_version 542044 (0.0007) [2023-12-27 09:23:22,485][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000542048_138780672.pth... [2023-12-27 09:23:22,487][20381] Updated weights for policy 1, policy_version 541731 (0.0006) [2023-12-27 09:23:22,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000540864_138477568.pth [2023-12-27 09:23:22,543][20381] Updated weights for policy 1, policy_version 541741 (0.0008) [2023-12-27 09:23:23,168][20381] Updated weights for policy 1, policy_version 541751 (0.0007) [2023-12-27 09:23:23,184][20347] Updated weights for policy 0, policy_version 542054 (0.0007) [2023-12-27 09:23:23,223][20381] Updated weights for policy 1, policy_version 541761 (0.0006) [2023-12-27 09:23:23,230][20347] Updated weights for policy 0, policy_version 542064 (0.0007) [2023-12-27 09:23:23,273][20381] Updated weights for policy 1, policy_version 541771 (0.0008) [2023-12-27 09:23:23,279][20347] Updated weights for policy 0, policy_version 542074 (0.0006) [2023-12-27 09:23:23,959][20381] Updated weights for policy 1, policy_version 541781 (0.0007) [2023-12-27 09:23:23,999][20347] Updated weights for policy 0, policy_version 542084 (0.0008) [2023-12-27 09:23:24,011][20381] Updated weights for policy 1, policy_version 541791 (0.0009) [2023-12-27 09:23:24,058][20347] Updated weights for policy 0, policy_version 542094 (0.0006) [2023-12-27 09:23:24,060][20381] Updated weights for policy 1, policy_version 541801 (0.0005) [2023-12-27 09:23:24,117][20347] Updated weights for policy 0, policy_version 542104 (0.0007) [2023-12-27 09:23:24,738][20381] Updated weights for policy 1, policy_version 541811 (0.0006) [2023-12-27 09:23:24,795][20381] Updated weights for policy 1, policy_version 541821 (0.0005) [2023-12-27 09:23:24,844][20381] Updated weights for policy 1, policy_version 541831 (0.0005) [2023-12-27 09:23:24,845][20347] Updated weights for policy 0, policy_version 542114 (0.0008) [2023-12-27 09:23:24,900][20347] Updated weights for policy 0, policy_version 542124 (0.0007) [2023-12-27 09:23:24,956][20347] Updated weights for policy 0, policy_version 542134 (0.0007) [2023-12-27 09:23:25,012][20347] Updated weights for policy 0, policy_version 542144 (0.0008) [2023-12-27 09:23:25,415][20381] Updated weights for policy 1, policy_version 541841 (0.0005) [2023-12-27 09:23:25,474][20381] Updated weights for policy 1, policy_version 541851 (0.0005) [2023-12-27 09:23:25,526][20381] Updated weights for policy 1, policy_version 541861 (0.0005) [2023-12-27 09:23:25,577][20381] Updated weights for policy 1, policy_version 541871 (0.0007) [2023-12-27 09:23:25,749][20347] Updated weights for policy 0, policy_version 542154 (0.0006) [2023-12-27 09:23:25,809][20347] Updated weights for policy 0, policy_version 542164 (0.0008) [2023-12-27 09:23:25,865][20347] Updated weights for policy 0, policy_version 542174 (0.0007) [2023-12-27 09:23:26,211][20381] Updated weights for policy 1, policy_version 541881 (0.0007) [2023-12-27 09:23:26,259][20381] Updated weights for policy 1, policy_version 541891 (0.0006) [2023-12-27 09:23:26,317][20381] Updated weights for policy 1, policy_version 541901 (0.0006) [2023-12-27 09:23:26,579][20347] Updated weights for policy 0, policy_version 542184 (0.0008) [2023-12-27 09:23:26,624][20347] Updated weights for policy 0, policy_version 542194 (0.0008) [2023-12-27 09:23:26,675][20347] Updated weights for policy 0, policy_version 542204 (0.0007) [2023-12-27 09:23:26,902][20381] Updated weights for policy 1, policy_version 541911 (0.0006) [2023-12-27 09:23:26,950][20381] Updated weights for policy 1, policy_version 541921 (0.0007) [2023-12-27 09:23:27,004][20381] Updated weights for policy 1, policy_version 541931 (0.0007) [2023-12-27 09:23:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 277577728. Throughput: 0: 10070.1, 1: 10563.7. Samples: 277568920. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:27,363][19170] Avg episode reward: [(0, '787.750'), (1, '702.490')] [2023-12-27 09:23:27,390][20347] Updated weights for policy 0, policy_version 542214 (0.0008) [2023-12-27 09:23:27,436][20347] Updated weights for policy 0, policy_version 542224 (0.0008) [2023-12-27 09:23:27,487][20347] Updated weights for policy 0, policy_version 542234 (0.0008) [2023-12-27 09:23:27,667][20381] Updated weights for policy 1, policy_version 541941 (0.0007) [2023-12-27 09:23:27,717][20381] Updated weights for policy 1, policy_version 541951 (0.0006) [2023-12-27 09:23:27,764][20381] Updated weights for policy 1, policy_version 541961 (0.0007) [2023-12-27 09:23:28,206][20347] Updated weights for policy 0, policy_version 542244 (0.0007) [2023-12-27 09:23:28,254][20347] Updated weights for policy 0, policy_version 542254 (0.0007) [2023-12-27 09:23:28,311][20347] Updated weights for policy 0, policy_version 542264 (0.0007) [2023-12-27 09:23:28,401][20381] Updated weights for policy 1, policy_version 541971 (0.0007) [2023-12-27 09:23:28,462][20381] Updated weights for policy 1, policy_version 541981 (0.0006) [2023-12-27 09:23:28,528][20381] Updated weights for policy 1, policy_version 541991 (0.0006) [2023-12-27 09:23:29,042][20347] Updated weights for policy 0, policy_version 542274 (0.0008) [2023-12-27 09:23:29,104][20347] Updated weights for policy 0, policy_version 542284 (0.0008) [2023-12-27 09:23:29,142][20381] Updated weights for policy 1, policy_version 542001 (0.0006) [2023-12-27 09:23:29,157][20347] Updated weights for policy 0, policy_version 542294 (0.0008) [2023-12-27 09:23:29,200][20381] Updated weights for policy 1, policy_version 542011 (0.0007) [2023-12-27 09:23:29,213][20347] Updated weights for policy 0, policy_version 542304 (0.0008) [2023-12-27 09:23:29,260][20381] Updated weights for policy 1, policy_version 542021 (0.0007) [2023-12-27 09:23:29,312][20381] Updated weights for policy 1, policy_version 542031 (0.0005) [2023-12-27 09:23:29,944][20381] Updated weights for policy 1, policy_version 542041 (0.0006) [2023-12-27 09:23:29,954][20347] Updated weights for policy 0, policy_version 542314 (0.0007) [2023-12-27 09:23:30,005][20381] Updated weights for policy 1, policy_version 542051 (0.0007) [2023-12-27 09:23:30,014][20347] Updated weights for policy 0, policy_version 542324 (0.0006) [2023-12-27 09:23:30,067][20381] Updated weights for policy 1, policy_version 542061 (0.0005) [2023-12-27 09:23:30,069][20347] Updated weights for policy 0, policy_version 542334 (0.0006) [2023-12-27 09:23:30,637][20381] Updated weights for policy 1, policy_version 542071 (0.0006) [2023-12-27 09:23:30,687][20381] Updated weights for policy 1, policy_version 542081 (0.0008) [2023-12-27 09:23:30,739][20381] Updated weights for policy 1, policy_version 542091 (0.0008) [2023-12-27 09:23:30,805][20347] Updated weights for policy 0, policy_version 542344 (0.0007) [2023-12-27 09:23:30,860][20347] Updated weights for policy 0, policy_version 542354 (0.0008) [2023-12-27 09:23:30,909][20347] Updated weights for policy 0, policy_version 542364 (0.0007) [2023-12-27 09:23:31,420][20381] Updated weights for policy 1, policy_version 542101 (0.0008) [2023-12-27 09:23:31,469][20381] Updated weights for policy 1, policy_version 542111 (0.0008) [2023-12-27 09:23:31,517][20381] Updated weights for policy 1, policy_version 542121 (0.0006) [2023-12-27 09:23:31,598][20347] Updated weights for policy 0, policy_version 542374 (0.0007) [2023-12-27 09:23:31,659][20347] Updated weights for policy 0, policy_version 542384 (0.0007) [2023-12-27 09:23:31,711][20347] Updated weights for policy 0, policy_version 542394 (0.0008) [2023-12-27 09:23:32,139][20381] Updated weights for policy 1, policy_version 542131 (0.0006) [2023-12-27 09:23:32,196][20381] Updated weights for policy 1, policy_version 542141 (0.0011) [2023-12-27 09:23:32,245][20381] Updated weights for policy 1, policy_version 542151 (0.0010) [2023-12-27 09:23:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 277684224. Throughput: 0: 9986.8, 1: 10578.0. Samples: 277693208. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:32,363][19170] Avg episode reward: [(0, '825.110'), (1, '795.990')] [2023-12-27 09:23:32,446][20347] Updated weights for policy 0, policy_version 542404 (0.0007) [2023-12-27 09:23:32,506][20347] Updated weights for policy 0, policy_version 542414 (0.0005) [2023-12-27 09:23:32,563][20347] Updated weights for policy 0, policy_version 542424 (0.0005) [2023-12-27 09:23:32,979][20381] Updated weights for policy 1, policy_version 542161 (0.0008) [2023-12-27 09:23:33,030][20381] Updated weights for policy 1, policy_version 542171 (0.0008) [2023-12-27 09:23:33,083][20381] Updated weights for policy 1, policy_version 542181 (0.0006) [2023-12-27 09:23:33,148][20381] Updated weights for policy 1, policy_version 542191 (0.0006) [2023-12-27 09:23:33,155][20347] Updated weights for policy 0, policy_version 542434 (0.0005) [2023-12-27 09:23:33,206][20347] Updated weights for policy 0, policy_version 542444 (0.0006) [2023-12-27 09:23:33,261][20347] Updated weights for policy 0, policy_version 542454 (0.0007) [2023-12-27 09:23:33,313][20347] Updated weights for policy 0, policy_version 542464 (0.0008) [2023-12-27 09:23:33,816][20381] Updated weights for policy 1, policy_version 542201 (0.0008) [2023-12-27 09:23:33,860][20381] Updated weights for policy 1, policy_version 542211 (0.0007) [2023-12-27 09:23:33,912][20381] Updated weights for policy 1, policy_version 542221 (0.0007) [2023-12-27 09:23:33,996][20347] Updated weights for policy 0, policy_version 542474 (0.0008) [2023-12-27 09:23:34,053][20347] Updated weights for policy 0, policy_version 542484 (0.0008) [2023-12-27 09:23:34,102][20347] Updated weights for policy 0, policy_version 542494 (0.0008) [2023-12-27 09:23:34,533][20381] Updated weights for policy 1, policy_version 542231 (0.0007) [2023-12-27 09:23:34,589][20381] Updated weights for policy 1, policy_version 542241 (0.0007) [2023-12-27 09:23:34,645][20381] Updated weights for policy 1, policy_version 542251 (0.0008) [2023-12-27 09:23:34,847][20347] Updated weights for policy 0, policy_version 542505 (0.0009) [2023-12-27 09:23:34,899][20347] Updated weights for policy 0, policy_version 542515 (0.0008) [2023-12-27 09:23:34,951][20347] Updated weights for policy 0, policy_version 542525 (0.0007) [2023-12-27 09:23:35,320][20381] Updated weights for policy 1, policy_version 542261 (0.0008) [2023-12-27 09:23:35,375][20381] Updated weights for policy 1, policy_version 542271 (0.0006) [2023-12-27 09:23:35,425][20381] Updated weights for policy 1, policy_version 542281 (0.0006) [2023-12-27 09:23:35,626][20291] KL-divergence is very high: 101.1030 [2023-12-27 09:23:35,631][20347] Updated weights for policy 0, policy_version 542535 (0.0008) [2023-12-27 09:23:35,691][20347] Updated weights for policy 0, policy_version 542545 (0.0008) [2023-12-27 09:23:35,754][20347] Updated weights for policy 0, policy_version 542555 (0.0007) [2023-12-27 09:23:36,041][20381] Updated weights for policy 1, policy_version 542291 (0.0008) [2023-12-27 09:23:36,094][20381] Updated weights for policy 1, policy_version 542301 (0.0008) [2023-12-27 09:23:36,151][20381] Updated weights for policy 1, policy_version 542311 (0.0008) [2023-12-27 09:23:36,452][20347] Updated weights for policy 0, policy_version 542565 (0.0007) [2023-12-27 09:23:36,502][20347] Updated weights for policy 0, policy_version 542575 (0.0007) [2023-12-27 09:23:36,551][20347] Updated weights for policy 0, policy_version 542585 (0.0007) [2023-12-27 09:23:36,844][20381] Updated weights for policy 1, policy_version 542321 (0.0010) [2023-12-27 09:23:36,902][20381] Updated weights for policy 1, policy_version 542331 (0.0011) [2023-12-27 09:23:36,954][20381] Updated weights for policy 1, policy_version 542341 (0.0011) [2023-12-27 09:23:37,006][20381] Updated weights for policy 1, policy_version 542351 (0.0009) [2023-12-27 09:23:37,288][20347] Updated weights for policy 0, policy_version 542595 (0.0008) [2023-12-27 09:23:37,353][20347] Updated weights for policy 0, policy_version 542605 (0.0007) [2023-12-27 09:23:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 277782528. Throughput: 0: 9987.0, 1: 10563.2. Samples: 277754972. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:37,363][19170] Avg episode reward: [(0, '717.350'), (1, '720.410')] [2023-12-27 09:23:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000542352_138862592.pth... [2023-12-27 09:23:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000541104_138543104.pth [2023-12-27 09:23:37,416][20347] Updated weights for policy 0, policy_version 542615 (0.0007) [2023-12-27 09:23:37,470][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000542624_138928128.pth... [2023-12-27 09:23:37,473][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000541472_138633216.pth [2023-12-27 09:23:37,630][20381] Updated weights for policy 1, policy_version 542361 (0.0006) [2023-12-27 09:23:37,679][20381] Updated weights for policy 1, policy_version 542371 (0.0006) [2023-12-27 09:23:37,734][20381] Updated weights for policy 1, policy_version 542381 (0.0011) [2023-12-27 09:23:38,106][20347] Updated weights for policy 0, policy_version 542625 (0.0007) [2023-12-27 09:23:38,160][20347] Updated weights for policy 0, policy_version 542635 (0.0005) [2023-12-27 09:23:38,220][20347] Updated weights for policy 0, policy_version 542645 (0.0007) [2023-12-27 09:23:38,273][20347] Updated weights for policy 0, policy_version 542655 (0.0008) [2023-12-27 09:23:38,418][20381] Updated weights for policy 1, policy_version 542391 (0.0011) [2023-12-27 09:23:38,470][20381] Updated weights for policy 1, policy_version 542401 (0.0010) [2023-12-27 09:23:38,523][20381] Updated weights for policy 1, policy_version 542411 (0.0011) [2023-12-27 09:23:38,962][20347] Updated weights for policy 0, policy_version 542665 (0.0007) [2023-12-27 09:23:39,020][20347] Updated weights for policy 0, policy_version 542675 (0.0007) [2023-12-27 09:23:39,068][20347] Updated weights for policy 0, policy_version 542685 (0.0007) [2023-12-27 09:23:39,221][20381] Updated weights for policy 1, policy_version 542421 (0.0009) [2023-12-27 09:23:39,284][20381] Updated weights for policy 1, policy_version 542431 (0.0011) [2023-12-27 09:23:39,346][20381] Updated weights for policy 1, policy_version 542441 (0.0010) [2023-12-27 09:23:39,733][20347] Updated weights for policy 0, policy_version 542695 (0.0006) [2023-12-27 09:23:39,797][20347] Updated weights for policy 0, policy_version 542705 (0.0006) [2023-12-27 09:23:39,862][20347] Updated weights for policy 0, policy_version 542715 (0.0007) [2023-12-27 09:23:39,992][20381] Updated weights for policy 1, policy_version 542451 (0.0008) [2023-12-27 09:23:40,052][20381] Updated weights for policy 1, policy_version 542461 (0.0006) [2023-12-27 09:23:40,103][20381] Updated weights for policy 1, policy_version 542471 (0.0007) [2023-12-27 09:23:40,528][20347] Updated weights for policy 0, policy_version 542725 (0.0008) [2023-12-27 09:23:40,584][20347] Updated weights for policy 0, policy_version 542735 (0.0007) [2023-12-27 09:23:40,629][20347] Updated weights for policy 0, policy_version 542745 (0.0008) [2023-12-27 09:23:40,753][20381] Updated weights for policy 1, policy_version 542481 (0.0007) [2023-12-27 09:23:40,820][20381] Updated weights for policy 1, policy_version 542491 (0.0008) [2023-12-27 09:23:40,882][20381] Updated weights for policy 1, policy_version 542501 (0.0008) [2023-12-27 09:23:40,942][20381] Updated weights for policy 1, policy_version 542511 (0.0008) [2023-12-27 09:23:41,280][20347] Updated weights for policy 0, policy_version 542755 (0.0007) [2023-12-27 09:23:41,340][20347] Updated weights for policy 0, policy_version 542765 (0.0008) [2023-12-27 09:23:41,400][20347] Updated weights for policy 0, policy_version 542775 (0.0008) [2023-12-27 09:23:41,689][20381] Updated weights for policy 1, policy_version 542521 (0.0007) [2023-12-27 09:23:41,756][20381] Updated weights for policy 1, policy_version 542531 (0.0007) [2023-12-27 09:23:41,815][20381] Updated weights for policy 1, policy_version 542541 (0.0007) [2023-12-27 09:23:42,118][20347] Updated weights for policy 0, policy_version 542785 (0.0008) [2023-12-27 09:23:42,182][20347] Updated weights for policy 0, policy_version 542795 (0.0006) [2023-12-27 09:23:42,236][20347] Updated weights for policy 0, policy_version 542805 (0.0007) [2023-12-27 09:23:42,294][20347] Updated weights for policy 0, policy_version 542815 (0.0008) [2023-12-27 09:23:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 277889024. Throughput: 0: 9908.8, 1: 10612.4. Samples: 277877776. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:42,363][19170] Avg episode reward: [(0, '849.460'), (1, '710.410')] [2023-12-27 09:23:42,507][20381] Updated weights for policy 1, policy_version 542551 (0.0007) [2023-12-27 09:23:42,569][20381] Updated weights for policy 1, policy_version 542561 (0.0008) [2023-12-27 09:23:42,634][20381] Updated weights for policy 1, policy_version 542571 (0.0009) [2023-12-27 09:23:42,898][20347] Updated weights for policy 0, policy_version 542825 (0.0007) [2023-12-27 09:23:42,949][20347] Updated weights for policy 0, policy_version 542835 (0.0008) [2023-12-27 09:23:42,995][20347] Updated weights for policy 0, policy_version 542845 (0.0007) [2023-12-27 09:23:43,299][20381] Updated weights for policy 1, policy_version 542581 (0.0007) [2023-12-27 09:23:43,360][20381] Updated weights for policy 1, policy_version 542591 (0.0005) [2023-12-27 09:23:43,417][20381] Updated weights for policy 1, policy_version 542601 (0.0006) [2023-12-27 09:23:43,661][20347] Updated weights for policy 0, policy_version 542855 (0.0006) [2023-12-27 09:23:43,714][20347] Updated weights for policy 0, policy_version 542865 (0.0005) [2023-12-27 09:23:43,761][20347] Updated weights for policy 0, policy_version 542875 (0.0007) [2023-12-27 09:23:44,039][20381] Updated weights for policy 1, policy_version 542611 (0.0010) [2023-12-27 09:23:44,093][20381] Updated weights for policy 1, policy_version 542621 (0.0010) [2023-12-27 09:23:44,143][20381] Updated weights for policy 1, policy_version 542631 (0.0010) [2023-12-27 09:23:44,416][20347] Updated weights for policy 0, policy_version 542885 (0.0006) [2023-12-27 09:23:44,473][20347] Updated weights for policy 0, policy_version 542895 (0.0005) [2023-12-27 09:23:44,524][20347] Updated weights for policy 0, policy_version 542905 (0.0005) [2023-12-27 09:23:44,852][20381] Updated weights for policy 1, policy_version 542641 (0.0010) [2023-12-27 09:23:44,905][20381] Updated weights for policy 1, policy_version 542651 (0.0009) [2023-12-27 09:23:44,959][20381] Updated weights for policy 1, policy_version 542661 (0.0005) [2023-12-27 09:23:45,015][20381] Updated weights for policy 1, policy_version 542671 (0.0011) [2023-12-27 09:23:45,147][20347] Updated weights for policy 0, policy_version 542915 (0.0007) [2023-12-27 09:23:45,197][20347] Updated weights for policy 0, policy_version 542925 (0.0011) [2023-12-27 09:23:45,247][20347] Updated weights for policy 0, policy_version 542935 (0.0010) [2023-12-27 09:23:45,654][20381] Updated weights for policy 1, policy_version 542681 (0.0011) [2023-12-27 09:23:45,678][20292] KL-divergence is very high: 209.8898 [2023-12-27 09:23:45,718][20381] Updated weights for policy 1, policy_version 542691 (0.0011) [2023-12-27 09:23:45,728][20292] KL-divergence is very high: 224.4501 [2023-12-27 09:23:45,771][20381] Updated weights for policy 1, policy_version 542701 (0.0011) [2023-12-27 09:23:45,864][20347] Updated weights for policy 0, policy_version 542945 (0.0010) [2023-12-27 09:23:45,925][20347] Updated weights for policy 0, policy_version 542955 (0.0010) [2023-12-27 09:23:45,984][20347] Updated weights for policy 0, policy_version 542965 (0.0010) [2023-12-27 09:23:46,036][20347] Updated weights for policy 0, policy_version 542975 (0.0010) [2023-12-27 09:23:46,444][20381] Updated weights for policy 1, policy_version 542711 (0.0009) [2023-12-27 09:23:46,493][20381] Updated weights for policy 1, policy_version 542721 (0.0009) [2023-12-27 09:23:46,547][20381] Updated weights for policy 1, policy_version 542731 (0.0009) [2023-12-27 09:23:46,760][20347] Updated weights for policy 0, policy_version 542985 (0.0006) [2023-12-27 09:23:46,814][20347] Updated weights for policy 0, policy_version 542995 (0.0006) [2023-12-27 09:23:46,869][20347] Updated weights for policy 0, policy_version 543005 (0.0007) [2023-12-27 09:23:47,232][20381] Updated weights for policy 1, policy_version 542741 (0.0007) [2023-12-27 09:23:47,282][20381] Updated weights for policy 1, policy_version 542751 (0.0005) [2023-12-27 09:23:47,346][20381] Updated weights for policy 1, policy_version 542761 (0.0007) [2023-12-27 09:23:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 277987328. Throughput: 0: 9914.8, 1: 10568.3. Samples: 278002864. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:47,363][19170] Avg episode reward: [(0, '990.450'), (1, '767.640')] [2023-12-27 09:23:47,494][20347] Updated weights for policy 0, policy_version 543015 (0.0005) [2023-12-27 09:23:47,553][20347] Updated weights for policy 0, policy_version 543025 (0.0005) [2023-12-27 09:23:47,614][20347] Updated weights for policy 0, policy_version 543035 (0.0005) [2023-12-27 09:23:47,977][20381] Updated weights for policy 1, policy_version 542771 (0.0009) [2023-12-27 09:23:48,038][20381] Updated weights for policy 1, policy_version 542781 (0.0009) [2023-12-27 09:23:48,086][20381] Updated weights for policy 1, policy_version 542791 (0.0010) [2023-12-27 09:23:48,183][20347] Updated weights for policy 0, policy_version 543045 (0.0008) [2023-12-27 09:23:48,243][20347] Updated weights for policy 0, policy_version 543055 (0.0010) [2023-12-27 09:23:48,300][20347] Updated weights for policy 0, policy_version 543065 (0.0010) [2023-12-27 09:23:48,772][20381] Updated weights for policy 1, policy_version 542801 (0.0010) [2023-12-27 09:23:48,839][20381] Updated weights for policy 1, policy_version 542811 (0.0007) [2023-12-27 09:23:48,897][20381] Updated weights for policy 1, policy_version 542821 (0.0008) [2023-12-27 09:23:48,967][20381] Updated weights for policy 1, policy_version 542831 (0.0007) [2023-12-27 09:23:48,980][20347] Updated weights for policy 0, policy_version 543075 (0.0011) [2023-12-27 09:23:49,040][20347] Updated weights for policy 0, policy_version 543085 (0.0009) [2023-12-27 09:23:49,094][20347] Updated weights for policy 0, policy_version 543095 (0.0010) [2023-12-27 09:23:49,656][20381] Updated weights for policy 1, policy_version 542841 (0.0006) [2023-12-27 09:23:49,719][20381] Updated weights for policy 1, policy_version 542851 (0.0006) [2023-12-27 09:23:49,776][20347] Updated weights for policy 0, policy_version 543105 (0.0005) [2023-12-27 09:23:49,786][20381] Updated weights for policy 1, policy_version 542861 (0.0007) [2023-12-27 09:23:49,837][20347] Updated weights for policy 0, policy_version 543115 (0.0007) [2023-12-27 09:23:49,892][20347] Updated weights for policy 0, policy_version 543125 (0.0008) [2023-12-27 09:23:49,946][20347] Updated weights for policy 0, policy_version 543135 (0.0008) [2023-12-27 09:23:50,393][20381] Updated weights for policy 1, policy_version 542871 (0.0006) [2023-12-27 09:23:50,445][20381] Updated weights for policy 1, policy_version 542881 (0.0009) [2023-12-27 09:23:50,503][20381] Updated weights for policy 1, policy_version 542891 (0.0007) [2023-12-27 09:23:50,597][20347] Updated weights for policy 0, policy_version 543145 (0.0007) [2023-12-27 09:23:50,654][20347] Updated weights for policy 0, policy_version 543155 (0.0007) [2023-12-27 09:23:50,703][20347] Updated weights for policy 0, policy_version 543165 (0.0008) [2023-12-27 09:23:51,209][20381] Updated weights for policy 1, policy_version 542901 (0.0008) [2023-12-27 09:23:51,272][20381] Updated weights for policy 1, policy_version 542911 (0.0008) [2023-12-27 09:23:51,331][20381] Updated weights for policy 1, policy_version 542921 (0.0008) [2023-12-27 09:23:51,408][20347] Updated weights for policy 0, policy_version 543175 (0.0008) [2023-12-27 09:23:51,454][20347] Updated weights for policy 0, policy_version 543185 (0.0005) [2023-12-27 09:23:51,501][20347] Updated weights for policy 0, policy_version 543195 (0.0005) [2023-12-27 09:23:52,112][20381] Updated weights for policy 1, policy_version 542931 (0.0009) [2023-12-27 09:23:52,151][20347] Updated weights for policy 0, policy_version 543205 (0.0006) [2023-12-27 09:23:52,172][20381] Updated weights for policy 1, policy_version 542941 (0.0008) [2023-12-27 09:23:52,203][20347] Updated weights for policy 0, policy_version 543215 (0.0005) [2023-12-27 09:23:52,226][20381] Updated weights for policy 1, policy_version 542951 (0.0007) [2023-12-27 09:23:52,262][20347] Updated weights for policy 0, policy_version 543225 (0.0007) [2023-12-27 09:23:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 278102016. Throughput: 0: 9950.6, 1: 10518.8. Samples: 278065196. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:52,363][19170] Avg episode reward: [(0, '990.450'), (1, '806.670')] [2023-12-27 09:23:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000542960_139018240.pth... [2023-12-27 09:23:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000543232_139083776.pth... [2023-12-27 09:23:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000541712_138698752.pth [2023-12-27 09:23:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000542048_138780672.pth [2023-12-27 09:23:52,902][20381] Updated weights for policy 1, policy_version 542961 (0.0007) [2023-12-27 09:23:52,946][20347] Updated weights for policy 0, policy_version 543235 (0.0009) [2023-12-27 09:23:52,947][20381] Updated weights for policy 1, policy_version 542971 (0.0005) [2023-12-27 09:23:52,994][20381] Updated weights for policy 1, policy_version 542981 (0.0006) [2023-12-27 09:23:52,996][20347] Updated weights for policy 0, policy_version 543245 (0.0006) [2023-12-27 09:23:53,042][20381] Updated weights for policy 1, policy_version 542991 (0.0006) [2023-12-27 09:23:53,048][20347] Updated weights for policy 0, policy_version 543255 (0.0008) [2023-12-27 09:23:53,671][20347] Updated weights for policy 0, policy_version 543265 (0.0006) [2023-12-27 09:23:53,677][20381] Updated weights for policy 1, policy_version 543001 (0.0005) [2023-12-27 09:23:53,726][20381] Updated weights for policy 1, policy_version 543011 (0.0005) [2023-12-27 09:23:53,730][20347] Updated weights for policy 0, policy_version 543275 (0.0006) [2023-12-27 09:23:53,774][20381] Updated weights for policy 1, policy_version 543021 (0.0005) [2023-12-27 09:23:53,783][20347] Updated weights for policy 0, policy_version 543285 (0.0008) [2023-12-27 09:23:53,830][20347] Updated weights for policy 0, policy_version 543295 (0.0008) [2023-12-27 09:23:54,428][20381] Updated weights for policy 1, policy_version 543031 (0.0007) [2023-12-27 09:23:54,490][20381] Updated weights for policy 1, policy_version 543041 (0.0008) [2023-12-27 09:23:54,518][20347] Updated weights for policy 0, policy_version 543305 (0.0007) [2023-12-27 09:23:54,553][20381] Updated weights for policy 1, policy_version 543051 (0.0006) [2023-12-27 09:23:54,578][20347] Updated weights for policy 0, policy_version 543315 (0.0008) [2023-12-27 09:23:54,630][20347] Updated weights for policy 0, policy_version 543325 (0.0009) [2023-12-27 09:23:55,090][20381] Updated weights for policy 1, policy_version 543061 (0.0008) [2023-12-27 09:23:55,147][20381] Updated weights for policy 1, policy_version 543071 (0.0010) [2023-12-27 09:23:55,199][20381] Updated weights for policy 1, policy_version 543081 (0.0010) [2023-12-27 09:23:55,314][20347] Updated weights for policy 0, policy_version 543335 (0.0007) [2023-12-27 09:23:55,367][20347] Updated weights for policy 0, policy_version 543345 (0.0008) [2023-12-27 09:23:55,419][20347] Updated weights for policy 0, policy_version 543355 (0.0008) [2023-12-27 09:23:55,835][20381] Updated weights for policy 1, policy_version 543091 (0.0009) [2023-12-27 09:23:55,885][20381] Updated weights for policy 1, policy_version 543101 (0.0005) [2023-12-27 09:23:55,943][20381] Updated weights for policy 1, policy_version 543111 (0.0005) [2023-12-27 09:23:56,202][20347] Updated weights for policy 0, policy_version 543365 (0.0008) [2023-12-27 09:23:56,260][20347] Updated weights for policy 0, policy_version 543375 (0.0008) [2023-12-27 09:23:56,311][20347] Updated weights for policy 0, policy_version 543385 (0.0008) [2023-12-27 09:23:56,532][20381] Updated weights for policy 1, policy_version 543121 (0.0006) [2023-12-27 09:23:56,580][20381] Updated weights for policy 1, policy_version 543131 (0.0009) [2023-12-27 09:23:56,637][20381] Updated weights for policy 1, policy_version 543142 (0.0008) [2023-12-27 09:23:56,687][20381] Updated weights for policy 1, policy_version 543152 (0.0007) [2023-12-27 09:23:56,910][20347] Updated weights for policy 0, policy_version 543395 (0.0007) [2023-12-27 09:23:56,961][20347] Updated weights for policy 0, policy_version 543405 (0.0007) [2023-12-27 09:23:57,024][20347] Updated weights for policy 0, policy_version 543415 (0.0006) [2023-12-27 09:23:57,347][20381] Updated weights for policy 1, policy_version 543162 (0.0006) [2023-12-27 09:23:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 278200320. Throughput: 0: 10053.4, 1: 10463.1. Samples: 278190756. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:23:57,363][19170] Avg episode reward: [(0, '990.600'), (1, '855.350')] [2023-12-27 09:23:57,403][20381] Updated weights for policy 1, policy_version 543172 (0.0007) [2023-12-27 09:23:57,460][20381] Updated weights for policy 1, policy_version 543182 (0.0005) [2023-12-27 09:23:57,627][20347] Updated weights for policy 0, policy_version 543425 (0.0010) [2023-12-27 09:23:57,688][20347] Updated weights for policy 0, policy_version 543435 (0.0005) [2023-12-27 09:23:57,740][20347] Updated weights for policy 0, policy_version 543445 (0.0005) [2023-12-27 09:23:57,791][20347] Updated weights for policy 0, policy_version 543455 (0.0005) [2023-12-27 09:23:57,984][20381] Updated weights for policy 1, policy_version 543192 (0.0005) [2023-12-27 09:23:58,032][20381] Updated weights for policy 1, policy_version 543202 (0.0009) [2023-12-27 09:23:58,083][20381] Updated weights for policy 1, policy_version 543212 (0.0010) [2023-12-27 09:23:58,390][20347] Updated weights for policy 0, policy_version 543465 (0.0007) [2023-12-27 09:23:58,458][20347] Updated weights for policy 0, policy_version 543475 (0.0008) [2023-12-27 09:23:58,527][20347] Updated weights for policy 0, policy_version 543485 (0.0006) [2023-12-27 09:23:58,785][20381] Updated weights for policy 1, policy_version 543222 (0.0008) [2023-12-27 09:23:58,848][20381] Updated weights for policy 1, policy_version 543232 (0.0007) [2023-12-27 09:23:58,907][20381] Updated weights for policy 1, policy_version 543242 (0.0007) [2023-12-27 09:23:59,275][20347] Updated weights for policy 0, policy_version 543495 (0.0008) [2023-12-27 09:23:59,330][20347] Updated weights for policy 0, policy_version 543505 (0.0009) [2023-12-27 09:23:59,392][20347] Updated weights for policy 0, policy_version 543515 (0.0008) [2023-12-27 09:23:59,631][20381] Updated weights for policy 1, policy_version 543252 (0.0007) [2023-12-27 09:23:59,685][20381] Updated weights for policy 1, policy_version 543262 (0.0005) [2023-12-27 09:23:59,745][20381] Updated weights for policy 1, policy_version 543272 (0.0006) [2023-12-27 09:24:00,085][20347] Updated weights for policy 0, policy_version 543525 (0.0008) [2023-12-27 09:24:00,144][20347] Updated weights for policy 0, policy_version 543535 (0.0007) [2023-12-27 09:24:00,195][20347] Updated weights for policy 0, policy_version 543545 (0.0007) [2023-12-27 09:24:00,374][20381] Updated weights for policy 1, policy_version 543282 (0.0008) [2023-12-27 09:24:00,421][20381] Updated weights for policy 1, policy_version 543292 (0.0010) [2023-12-27 09:24:00,476][20381] Updated weights for policy 1, policy_version 543302 (0.0010) [2023-12-27 09:24:00,533][20381] Updated weights for policy 1, policy_version 543312 (0.0010) [2023-12-27 09:24:00,825][20347] Updated weights for policy 0, policy_version 543555 (0.0009) [2023-12-27 09:24:00,874][20347] Updated weights for policy 0, policy_version 543565 (0.0007) [2023-12-27 09:24:00,928][20347] Updated weights for policy 0, policy_version 543575 (0.0005) [2023-12-27 09:24:01,165][20381] Updated weights for policy 1, policy_version 543322 (0.0010) [2023-12-27 09:24:01,214][20381] Updated weights for policy 1, policy_version 543332 (0.0007) [2023-12-27 09:24:01,268][20381] Updated weights for policy 1, policy_version 543342 (0.0009) [2023-12-27 09:24:01,549][20347] Updated weights for policy 0, policy_version 543585 (0.0006) [2023-12-27 09:24:01,601][20347] Updated weights for policy 0, policy_version 543595 (0.0010) [2023-12-27 09:24:01,655][20347] Updated weights for policy 0, policy_version 543605 (0.0009) [2023-12-27 09:24:01,704][20347] Updated weights for policy 0, policy_version 543615 (0.0008) [2023-12-27 09:24:01,948][20381] Updated weights for policy 1, policy_version 543352 (0.0010) [2023-12-27 09:24:02,009][20381] Updated weights for policy 1, policy_version 543362 (0.0011) [2023-12-27 09:24:02,067][20381] Updated weights for policy 1, policy_version 543372 (0.0010) [2023-12-27 09:24:02,342][20347] Updated weights for policy 0, policy_version 543625 (0.0007) [2023-12-27 09:24:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 278306816. Throughput: 0: 10163.7, 1: 10481.0. Samples: 278318312. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:02,363][19170] Avg episode reward: [(0, '990.600'), (1, '788.360')] [2023-12-27 09:24:02,408][20347] Updated weights for policy 0, policy_version 543635 (0.0008) [2023-12-27 09:24:02,464][20347] Updated weights for policy 0, policy_version 543645 (0.0008) [2023-12-27 09:24:02,713][20381] Updated weights for policy 1, policy_version 543382 (0.0008) [2023-12-27 09:24:02,765][20381] Updated weights for policy 1, policy_version 543392 (0.0005) [2023-12-27 09:24:02,824][20381] Updated weights for policy 1, policy_version 543402 (0.0008) [2023-12-27 09:24:03,143][20347] Updated weights for policy 0, policy_version 543655 (0.0007) [2023-12-27 09:24:03,199][20347] Updated weights for policy 0, policy_version 543665 (0.0006) [2023-12-27 09:24:03,257][20347] Updated weights for policy 0, policy_version 543675 (0.0006) [2023-12-27 09:24:03,470][20381] Updated weights for policy 1, policy_version 543412 (0.0009) [2023-12-27 09:24:03,537][20381] Updated weights for policy 1, policy_version 543422 (0.0008) [2023-12-27 09:24:03,593][20381] Updated weights for policy 1, policy_version 543432 (0.0008) [2023-12-27 09:24:03,818][20347] Updated weights for policy 0, policy_version 543685 (0.0008) [2023-12-27 09:24:03,880][20347] Updated weights for policy 0, policy_version 543695 (0.0009) [2023-12-27 09:24:03,932][20347] Updated weights for policy 0, policy_version 543705 (0.0008) [2023-12-27 09:24:04,290][20381] Updated weights for policy 1, policy_version 543442 (0.0008) [2023-12-27 09:24:04,352][20381] Updated weights for policy 1, policy_version 543452 (0.0006) [2023-12-27 09:24:04,409][20381] Updated weights for policy 1, policy_version 543462 (0.0007) [2023-12-27 09:24:04,465][20381] Updated weights for policy 1, policy_version 543472 (0.0007) [2023-12-27 09:24:04,664][20347] Updated weights for policy 0, policy_version 543715 (0.0007) [2023-12-27 09:24:04,734][20347] Updated weights for policy 0, policy_version 543725 (0.0008) [2023-12-27 09:24:04,787][20347] Updated weights for policy 0, policy_version 543735 (0.0008) [2023-12-27 09:24:05,145][20381] Updated weights for policy 1, policy_version 543482 (0.0008) [2023-12-27 09:24:05,201][20381] Updated weights for policy 1, policy_version 543492 (0.0010) [2023-12-27 09:24:05,252][20381] Updated weights for policy 1, policy_version 543502 (0.0007) [2023-12-27 09:24:05,501][20347] Updated weights for policy 0, policy_version 543745 (0.0008) [2023-12-27 09:24:05,556][20347] Updated weights for policy 0, policy_version 543755 (0.0008) [2023-12-27 09:24:05,569][20291] KL-divergence is very high: 140.8705 [2023-12-27 09:24:05,611][20347] Updated weights for policy 0, policy_version 543765 (0.0007) [2023-12-27 09:24:05,669][20347] Updated weights for policy 0, policy_version 543775 (0.0006) [2023-12-27 09:24:05,877][20381] Updated weights for policy 1, policy_version 543512 (0.0009) [2023-12-27 09:24:05,932][20381] Updated weights for policy 1, policy_version 543522 (0.0010) [2023-12-27 09:24:05,977][20381] Updated weights for policy 1, policy_version 543532 (0.0010) [2023-12-27 09:24:06,358][20347] Updated weights for policy 0, policy_version 543785 (0.0007) [2023-12-27 09:24:06,414][20347] Updated weights for policy 0, policy_version 543795 (0.0007) [2023-12-27 09:24:06,473][20347] Updated weights for policy 0, policy_version 543805 (0.0008) [2023-12-27 09:24:06,675][20381] Updated weights for policy 1, policy_version 543542 (0.0010) [2023-12-27 09:24:06,727][20381] Updated weights for policy 1, policy_version 543552 (0.0010) [2023-12-27 09:24:06,776][20381] Updated weights for policy 1, policy_version 543562 (0.0010) [2023-12-27 09:24:06,786][20292] KL-divergence is very high: 124.9171 [2023-12-27 09:24:07,185][20347] Updated weights for policy 0, policy_version 543815 (0.0008) [2023-12-27 09:24:07,235][20347] Updated weights for policy 0, policy_version 543825 (0.0008) [2023-12-27 09:24:07,292][20347] Updated weights for policy 0, policy_version 543835 (0.0007) [2023-12-27 09:24:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 278413312. Throughput: 0: 10198.1, 1: 10526.0. Samples: 278380828. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:07,363][19170] Avg episode reward: [(0, '971.470'), (1, '797.130')] [2023-12-27 09:24:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000543568_139173888.pth... [2023-12-27 09:24:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000543840_139239424.pth... [2023-12-27 09:24:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000542352_138862592.pth [2023-12-27 09:24:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000542624_138928128.pth [2023-12-27 09:24:07,451][20292] KL-divergence is very high: 169.0112 [2023-12-27 09:24:07,469][20381] Updated weights for policy 1, policy_version 543572 (0.0008) [2023-12-27 09:24:07,499][20292] KL-divergence is very high: 148.5195 [2023-12-27 09:24:07,534][20381] Updated weights for policy 1, policy_version 543582 (0.0006) [2023-12-27 09:24:07,595][20381] Updated weights for policy 1, policy_version 543592 (0.0006) [2023-12-27 09:24:08,036][20347] Updated weights for policy 0, policy_version 543845 (0.0008) [2023-12-27 09:24:08,096][20347] Updated weights for policy 0, policy_version 543855 (0.0007) [2023-12-27 09:24:08,149][20347] Updated weights for policy 0, policy_version 543865 (0.0008) [2023-12-27 09:24:08,214][20381] Updated weights for policy 1, policy_version 543602 (0.0008) [2023-12-27 09:24:08,266][20381] Updated weights for policy 1, policy_version 543612 (0.0008) [2023-12-27 09:24:08,326][20381] Updated weights for policy 1, policy_version 543622 (0.0008) [2023-12-27 09:24:08,381][20381] Updated weights for policy 1, policy_version 543632 (0.0008) [2023-12-27 09:24:08,860][20347] Updated weights for policy 0, policy_version 543875 (0.0008) [2023-12-27 09:24:08,916][20347] Updated weights for policy 0, policy_version 543885 (0.0008) [2023-12-27 09:24:08,974][20347] Updated weights for policy 0, policy_version 543895 (0.0007) [2023-12-27 09:24:09,065][20381] Updated weights for policy 1, policy_version 543642 (0.0007) [2023-12-27 09:24:09,124][20381] Updated weights for policy 1, policy_version 543652 (0.0006) [2023-12-27 09:24:09,183][20381] Updated weights for policy 1, policy_version 543662 (0.0006) [2023-12-27 09:24:09,710][20347] Updated weights for policy 0, policy_version 543905 (0.0007) [2023-12-27 09:24:09,774][20347] Updated weights for policy 0, policy_version 543915 (0.0006) [2023-12-27 09:24:09,776][20381] Updated weights for policy 1, policy_version 543672 (0.0006) [2023-12-27 09:24:09,831][20347] Updated weights for policy 0, policy_version 543925 (0.0007) [2023-12-27 09:24:09,836][20381] Updated weights for policy 1, policy_version 543682 (0.0006) [2023-12-27 09:24:09,894][20347] Updated weights for policy 0, policy_version 543935 (0.0007) [2023-12-27 09:24:09,905][20381] Updated weights for policy 1, policy_version 543692 (0.0007) [2023-12-27 09:24:10,585][20381] Updated weights for policy 1, policy_version 543702 (0.0010) [2023-12-27 09:24:10,609][20347] Updated weights for policy 0, policy_version 543945 (0.0006) [2023-12-27 09:24:10,641][20381] Updated weights for policy 1, policy_version 543712 (0.0011) [2023-12-27 09:24:10,662][20347] Updated weights for policy 0, policy_version 543955 (0.0005) [2023-12-27 09:24:10,694][20381] Updated weights for policy 1, policy_version 543722 (0.0011) [2023-12-27 09:24:10,718][20347] Updated weights for policy 0, policy_version 543965 (0.0006) [2023-12-27 09:24:11,406][20347] Updated weights for policy 0, policy_version 543975 (0.0007) [2023-12-27 09:24:11,437][20381] Updated weights for policy 1, policy_version 543732 (0.0011) [2023-12-27 09:24:11,464][20347] Updated weights for policy 0, policy_version 543985 (0.0007) [2023-12-27 09:24:11,496][20381] Updated weights for policy 1, policy_version 543742 (0.0011) [2023-12-27 09:24:11,527][20347] Updated weights for policy 0, policy_version 543995 (0.0005) [2023-12-27 09:24:11,558][20381] Updated weights for policy 1, policy_version 543752 (0.0008) [2023-12-27 09:24:12,136][20381] Updated weights for policy 1, policy_version 543762 (0.0007) [2023-12-27 09:24:12,159][20292] KL-divergence is very high: 137.4950 [2023-12-27 09:24:12,166][20292] KL-divergence is very high: 290.4390 [2023-12-27 09:24:12,183][20292] KL-divergence is very high: 221.1082 [2023-12-27 09:24:12,188][20381] Updated weights for policy 1, policy_version 543772 (0.0005) [2023-12-27 09:24:12,194][20292] KL-divergence is very high: 172.0182 [2023-12-27 09:24:12,204][20292] KL-divergence is very high: 120.6202 [2023-12-27 09:24:12,209][20292] KL-divergence is very high: 161.3320 [2023-12-27 09:24:12,238][20381] Updated weights for policy 1, policy_version 543782 (0.0006) [2023-12-27 09:24:12,244][20292] KL-divergence is very high: 140.3003 [2023-12-27 09:24:12,284][20347] Updated weights for policy 0, policy_version 544005 (0.0008) [2023-12-27 09:24:12,301][20381] Updated weights for policy 1, policy_version 543792 (0.0010) [2023-12-27 09:24:12,346][20347] Updated weights for policy 0, policy_version 544015 (0.0007) [2023-12-27 09:24:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 278511616. Throughput: 0: 10235.7, 1: 10502.6. Samples: 278502148. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:12,363][19170] Avg episode reward: [(0, '961.930'), (1, '710.500')] [2023-12-27 09:24:12,409][20347] Updated weights for policy 0, policy_version 544025 (0.0008) [2023-12-27 09:24:12,996][20381] Updated weights for policy 1, policy_version 543802 (0.0011) [2023-12-27 09:24:13,051][20381] Updated weights for policy 1, policy_version 543812 (0.0009) [2023-12-27 09:24:13,075][20347] Updated weights for policy 0, policy_version 544035 (0.0007) [2023-12-27 09:24:13,099][20381] Updated weights for policy 1, policy_version 543822 (0.0005) [2023-12-27 09:24:13,121][20347] Updated weights for policy 0, policy_version 544045 (0.0008) [2023-12-27 09:24:13,170][20347] Updated weights for policy 0, policy_version 544055 (0.0008) [2023-12-27 09:24:13,706][20381] Updated weights for policy 1, policy_version 543832 (0.0010) [2023-12-27 09:24:13,762][20381] Updated weights for policy 1, policy_version 543842 (0.0011) [2023-12-27 09:24:13,815][20381] Updated weights for policy 1, policy_version 543852 (0.0010) [2023-12-27 09:24:13,829][20347] Updated weights for policy 0, policy_version 544065 (0.0007) [2023-12-27 09:24:13,873][20347] Updated weights for policy 0, policy_version 544075 (0.0008) [2023-12-27 09:24:13,927][20347] Updated weights for policy 0, policy_version 544085 (0.0007) [2023-12-27 09:24:13,972][20347] Updated weights for policy 0, policy_version 544095 (0.0006) [2023-12-27 09:24:14,493][20292] KL-divergence is very high: 124.6047 [2023-12-27 09:24:14,501][20381] Updated weights for policy 1, policy_version 543862 (0.0010) [2023-12-27 09:24:14,549][20381] Updated weights for policy 1, policy_version 543872 (0.0011) [2023-12-27 09:24:14,578][20347] Updated weights for policy 0, policy_version 544105 (0.0006) [2023-12-27 09:24:14,599][20381] Updated weights for policy 1, policy_version 543882 (0.0005) [2023-12-27 09:24:14,623][20347] Updated weights for policy 0, policy_version 544115 (0.0008) [2023-12-27 09:24:14,674][20347] Updated weights for policy 0, policy_version 544125 (0.0007) [2023-12-27 09:24:15,242][20381] Updated weights for policy 1, policy_version 543892 (0.0007) [2023-12-27 09:24:15,300][20381] Updated weights for policy 1, policy_version 543902 (0.0006) [2023-12-27 09:24:15,356][20381] Updated weights for policy 1, policy_version 543912 (0.0005) [2023-12-27 09:24:15,382][20347] Updated weights for policy 0, policy_version 544135 (0.0007) [2023-12-27 09:24:15,429][20347] Updated weights for policy 0, policy_version 544145 (0.0006) [2023-12-27 09:24:15,481][20347] Updated weights for policy 0, policy_version 544155 (0.0007) [2023-12-27 09:24:15,965][20381] Updated weights for policy 1, policy_version 543922 (0.0007) [2023-12-27 09:24:16,017][20381] Updated weights for policy 1, policy_version 543932 (0.0010) [2023-12-27 09:24:16,067][20381] Updated weights for policy 1, policy_version 543942 (0.0006) [2023-12-27 09:24:16,122][20381] Updated weights for policy 1, policy_version 543952 (0.0005) [2023-12-27 09:24:16,157][20347] Updated weights for policy 0, policy_version 544165 (0.0007) [2023-12-27 09:24:16,209][20347] Updated weights for policy 0, policy_version 544175 (0.0010) [2023-12-27 09:24:16,257][20347] Updated weights for policy 0, policy_version 544185 (0.0010) [2023-12-27 09:24:16,784][20381] Updated weights for policy 1, policy_version 543962 (0.0005) [2023-12-27 09:24:16,845][20381] Updated weights for policy 1, policy_version 543972 (0.0006) [2023-12-27 09:24:16,871][20347] Updated weights for policy 0, policy_version 544195 (0.0010) [2023-12-27 09:24:16,902][20381] Updated weights for policy 1, policy_version 543982 (0.0011) [2023-12-27 09:24:16,927][20347] Updated weights for policy 0, policy_version 544205 (0.0010) [2023-12-27 09:24:16,987][20347] Updated weights for policy 0, policy_version 544215 (0.0009) [2023-12-27 09:24:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 278618112. Throughput: 0: 10314.2, 1: 10470.6. Samples: 278628528. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:17,363][19170] Avg episode reward: [(0, '952.370'), (1, '544.780')] [2023-12-27 09:24:17,544][20381] Updated weights for policy 1, policy_version 543992 (0.0011) [2023-12-27 09:24:17,598][20381] Updated weights for policy 1, policy_version 544002 (0.0010) [2023-12-27 09:24:17,621][20347] Updated weights for policy 0, policy_version 544225 (0.0006) [2023-12-27 09:24:17,647][20381] Updated weights for policy 1, policy_version 544012 (0.0010) [2023-12-27 09:24:17,679][20347] Updated weights for policy 0, policy_version 544235 (0.0010) [2023-12-27 09:24:17,734][20347] Updated weights for policy 0, policy_version 544245 (0.0010) [2023-12-27 09:24:17,784][20347] Updated weights for policy 0, policy_version 544255 (0.0010) [2023-12-27 09:24:18,348][20381] Updated weights for policy 1, policy_version 544022 (0.0009) [2023-12-27 09:24:18,410][20381] Updated weights for policy 1, policy_version 544032 (0.0008) [2023-12-27 09:24:18,427][20291] KL-divergence is very high: 174.0280 [2023-12-27 09:24:18,464][20381] Updated weights for policy 1, policy_version 544042 (0.0009) [2023-12-27 09:24:18,470][20347] Updated weights for policy 0, policy_version 544265 (0.0010) [2023-12-27 09:24:18,471][20291] KL-divergence is very high: 186.2455 [2023-12-27 09:24:18,534][20347] Updated weights for policy 0, policy_version 544275 (0.0007) [2023-12-27 09:24:18,595][20347] Updated weights for policy 0, policy_version 544285 (0.0005) [2023-12-27 09:24:19,169][20381] Updated weights for policy 1, policy_version 544052 (0.0006) [2023-12-27 09:24:19,242][20381] Updated weights for policy 1, policy_version 544062 (0.0007) [2023-12-27 09:24:19,251][20347] Updated weights for policy 0, policy_version 544295 (0.0007) [2023-12-27 09:24:19,296][20381] Updated weights for policy 1, policy_version 544072 (0.0006) [2023-12-27 09:24:19,314][20347] Updated weights for policy 0, policy_version 544305 (0.0008) [2023-12-27 09:24:19,380][20347] Updated weights for policy 0, policy_version 544315 (0.0006) [2023-12-27 09:24:20,008][20381] Updated weights for policy 1, policy_version 544082 (0.0007) [2023-12-27 09:24:20,037][20347] Updated weights for policy 0, policy_version 544325 (0.0007) [2023-12-27 09:24:20,062][20381] Updated weights for policy 1, policy_version 544092 (0.0007) [2023-12-27 09:24:20,092][20347] Updated weights for policy 0, policy_version 544335 (0.0006) [2023-12-27 09:24:20,117][20381] Updated weights for policy 1, policy_version 544102 (0.0006) [2023-12-27 09:24:20,150][20347] Updated weights for policy 0, policy_version 544345 (0.0006) [2023-12-27 09:24:20,182][20381] Updated weights for policy 1, policy_version 544112 (0.0007) [2023-12-27 09:24:20,842][20347] Updated weights for policy 0, policy_version 544355 (0.0006) [2023-12-27 09:24:20,904][20347] Updated weights for policy 0, policy_version 544365 (0.0005) [2023-12-27 09:24:20,957][20381] Updated weights for policy 1, policy_version 544122 (0.0006) [2023-12-27 09:24:20,962][20347] Updated weights for policy 0, policy_version 544375 (0.0007) [2023-12-27 09:24:21,017][20381] Updated weights for policy 1, policy_version 544132 (0.0005) [2023-12-27 09:24:21,080][20381] Updated weights for policy 1, policy_version 544142 (0.0008) [2023-12-27 09:24:21,667][20347] Updated weights for policy 0, policy_version 544385 (0.0007) [2023-12-27 09:24:21,733][20347] Updated weights for policy 0, policy_version 544395 (0.0008) [2023-12-27 09:24:21,787][20381] Updated weights for policy 1, policy_version 544152 (0.0008) [2023-12-27 09:24:21,797][20347] Updated weights for policy 0, policy_version 544405 (0.0006) [2023-12-27 09:24:21,839][20381] Updated weights for policy 1, policy_version 544162 (0.0008) [2023-12-27 09:24:21,858][20347] Updated weights for policy 0, policy_version 544415 (0.0006) [2023-12-27 09:24:21,903][20381] Updated weights for policy 1, policy_version 544172 (0.0008) [2023-12-27 09:24:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 278716416. Throughput: 0: 10326.1, 1: 10433.9. Samples: 278689172. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:22,363][19170] Avg episode reward: [(0, '942.610'), (1, '592.780')] [2023-12-27 09:24:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000544176_139329536.pth... [2023-12-27 09:24:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000544416_139386880.pth... [2023-12-27 09:24:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000543232_139083776.pth [2023-12-27 09:24:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000542960_139018240.pth [2023-12-27 09:24:22,589][20347] Updated weights for policy 0, policy_version 544425 (0.0007) [2023-12-27 09:24:22,643][20381] Updated weights for policy 1, policy_version 544182 (0.0008) [2023-12-27 09:24:22,650][20347] Updated weights for policy 0, policy_version 544435 (0.0006) [2023-12-27 09:24:22,695][20381] Updated weights for policy 1, policy_version 544192 (0.0007) [2023-12-27 09:24:22,706][20347] Updated weights for policy 0, policy_version 544445 (0.0008) [2023-12-27 09:24:22,746][20381] Updated weights for policy 1, policy_version 544202 (0.0011) [2023-12-27 09:24:23,371][20347] Updated weights for policy 0, policy_version 544455 (0.0008) [2023-12-27 09:24:23,420][20347] Updated weights for policy 0, policy_version 544465 (0.0010) [2023-12-27 09:24:23,458][20381] Updated weights for policy 1, policy_version 544212 (0.0010) [2023-12-27 09:24:23,473][20347] Updated weights for policy 0, policy_version 544475 (0.0009) [2023-12-27 09:24:23,513][20381] Updated weights for policy 1, policy_version 544222 (0.0005) [2023-12-27 09:24:23,576][20381] Updated weights for policy 1, policy_version 544232 (0.0005) [2023-12-27 09:24:24,098][20347] Updated weights for policy 0, policy_version 544485 (0.0005) [2023-12-27 09:24:24,158][20347] Updated weights for policy 0, policy_version 544495 (0.0008) [2023-12-27 09:24:24,193][20381] Updated weights for policy 1, policy_version 544242 (0.0005) [2023-12-27 09:24:24,217][20347] Updated weights for policy 0, policy_version 544505 (0.0008) [2023-12-27 09:24:24,252][20381] Updated weights for policy 1, policy_version 544252 (0.0005) [2023-12-27 09:24:24,300][20381] Updated weights for policy 1, policy_version 544262 (0.0007) [2023-12-27 09:24:24,360][20381] Updated weights for policy 1, policy_version 544272 (0.0007) [2023-12-27 09:24:24,847][20347] Updated weights for policy 0, policy_version 544515 (0.0006) [2023-12-27 09:24:24,906][20347] Updated weights for policy 0, policy_version 544525 (0.0008) [2023-12-27 09:24:24,955][20347] Updated weights for policy 0, policy_version 544535 (0.0007) [2023-12-27 09:24:25,008][20381] Updated weights for policy 1, policy_version 544282 (0.0005) [2023-12-27 09:24:25,066][20381] Updated weights for policy 1, policy_version 544292 (0.0005) [2023-12-27 09:24:25,118][20381] Updated weights for policy 1, policy_version 544302 (0.0005) [2023-12-27 09:24:25,642][20347] Updated weights for policy 0, policy_version 544545 (0.0007) [2023-12-27 09:24:25,700][20347] Updated weights for policy 0, policy_version 544555 (0.0005) [2023-12-27 09:24:25,701][20381] Updated weights for policy 1, policy_version 544312 (0.0007) [2023-12-27 09:24:25,753][20347] Updated weights for policy 0, policy_version 544565 (0.0006) [2023-12-27 09:24:25,755][20381] Updated weights for policy 1, policy_version 544322 (0.0006) [2023-12-27 09:24:25,807][20347] Updated weights for policy 0, policy_version 544575 (0.0005) [2023-12-27 09:24:25,811][20381] Updated weights for policy 1, policy_version 544332 (0.0008) [2023-12-27 09:24:26,392][20381] Updated weights for policy 1, policy_version 544342 (0.0007) [2023-12-27 09:24:26,449][20381] Updated weights for policy 1, policy_version 544352 (0.0006) [2023-12-27 09:24:26,505][20347] Updated weights for policy 0, policy_version 544585 (0.0006) [2023-12-27 09:24:26,511][20381] Updated weights for policy 1, policy_version 544362 (0.0005) [2023-12-27 09:24:26,556][20347] Updated weights for policy 0, policy_version 544595 (0.0007) [2023-12-27 09:24:26,604][20347] Updated weights for policy 0, policy_version 544605 (0.0007) [2023-12-27 09:24:27,134][20381] Updated weights for policy 1, policy_version 544372 (0.0008) [2023-12-27 09:24:27,192][20381] Updated weights for policy 1, policy_version 544382 (0.0007) [2023-12-27 09:24:27,248][20381] Updated weights for policy 1, policy_version 544392 (0.0008) [2023-12-27 09:24:27,261][20347] Updated weights for policy 0, policy_version 544615 (0.0005) [2023-12-27 09:24:27,316][20347] Updated weights for policy 0, policy_version 544625 (0.0005) [2023-12-27 09:24:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 278822912. Throughput: 0: 10348.0, 1: 10437.2. Samples: 278813108. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:27,363][19170] Avg episode reward: [(0, '943.980'), (1, '688.830')] [2023-12-27 09:24:27,373][20347] Updated weights for policy 0, policy_version 544635 (0.0009) [2023-12-27 09:24:27,806][20381] Updated weights for policy 1, policy_version 544402 (0.0007) [2023-12-27 09:24:27,866][20381] Updated weights for policy 1, policy_version 544412 (0.0008) [2023-12-27 09:24:27,921][20381] Updated weights for policy 1, policy_version 544422 (0.0010) [2023-12-27 09:24:27,974][20381] Updated weights for policy 1, policy_version 544432 (0.0010) [2023-12-27 09:24:27,995][20347] Updated weights for policy 0, policy_version 544645 (0.0010) [2023-12-27 09:24:28,044][20347] Updated weights for policy 0, policy_version 544655 (0.0010) [2023-12-27 09:24:28,098][20347] Updated weights for policy 0, policy_version 544665 (0.0010) [2023-12-27 09:24:28,584][20381] Updated weights for policy 1, policy_version 544442 (0.0009) [2023-12-27 09:24:28,637][20381] Updated weights for policy 1, policy_version 544452 (0.0011) [2023-12-27 09:24:28,686][20381] Updated weights for policy 1, policy_version 544462 (0.0010) [2023-12-27 09:24:28,784][20347] Updated weights for policy 0, policy_version 544675 (0.0009) [2023-12-27 09:24:28,835][20347] Updated weights for policy 0, policy_version 544685 (0.0007) [2023-12-27 09:24:28,890][20347] Updated weights for policy 0, policy_version 544695 (0.0007) [2023-12-27 09:24:29,372][20381] Updated weights for policy 1, policy_version 544472 (0.0008) [2023-12-27 09:24:29,435][20381] Updated weights for policy 1, policy_version 544482 (0.0008) [2023-12-27 09:24:29,490][20381] Updated weights for policy 1, policy_version 544492 (0.0009) [2023-12-27 09:24:29,559][20347] Updated weights for policy 0, policy_version 544705 (0.0007) [2023-12-27 09:24:29,610][20347] Updated weights for policy 0, policy_version 544715 (0.0007) [2023-12-27 09:24:29,660][20347] Updated weights for policy 0, policy_version 544725 (0.0008) [2023-12-27 09:24:29,704][20347] Updated weights for policy 0, policy_version 544735 (0.0008) [2023-12-27 09:24:30,143][20381] Updated weights for policy 1, policy_version 544502 (0.0007) [2023-12-27 09:24:30,200][20381] Updated weights for policy 1, policy_version 544512 (0.0007) [2023-12-27 09:24:30,250][20381] Updated weights for policy 1, policy_version 544522 (0.0009) [2023-12-27 09:24:30,396][20347] Updated weights for policy 0, policy_version 544745 (0.0007) [2023-12-27 09:24:30,441][20347] Updated weights for policy 0, policy_version 544755 (0.0007) [2023-12-27 09:24:30,492][20347] Updated weights for policy 0, policy_version 544765 (0.0007) [2023-12-27 09:24:30,964][20381] Updated weights for policy 1, policy_version 544532 (0.0006) [2023-12-27 09:24:31,023][20381] Updated weights for policy 1, policy_version 544542 (0.0006) [2023-12-27 09:24:31,075][20381] Updated weights for policy 1, policy_version 544552 (0.0008) [2023-12-27 09:24:31,115][20347] Updated weights for policy 0, policy_version 544775 (0.0007) [2023-12-27 09:24:31,178][20347] Updated weights for policy 0, policy_version 544785 (0.0008) [2023-12-27 09:24:31,234][20347] Updated weights for policy 0, policy_version 544795 (0.0008) [2023-12-27 09:24:31,710][20381] Updated weights for policy 1, policy_version 544562 (0.0007) [2023-12-27 09:24:31,778][20381] Updated weights for policy 1, policy_version 544572 (0.0007) [2023-12-27 09:24:31,836][20381] Updated weights for policy 1, policy_version 544582 (0.0008) [2023-12-27 09:24:31,891][20381] Updated weights for policy 1, policy_version 544592 (0.0008) [2023-12-27 09:24:31,925][20347] Updated weights for policy 0, policy_version 544805 (0.0008) [2023-12-27 09:24:31,978][20347] Updated weights for policy 0, policy_version 544815 (0.0008) [2023-12-27 09:24:32,027][20347] Updated weights for policy 0, policy_version 544825 (0.0008) [2023-12-27 09:24:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 278929408. Throughput: 0: 10312.1, 1: 10507.5. Samples: 278939752. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:32,363][19170] Avg episode reward: [(0, '963.110'), (1, '700.030')] [2023-12-27 09:24:32,495][20381] Updated weights for policy 1, policy_version 544602 (0.0007) [2023-12-27 09:24:32,549][20381] Updated weights for policy 1, policy_version 544612 (0.0005) [2023-12-27 09:24:32,608][20381] Updated weights for policy 1, policy_version 544622 (0.0005) [2023-12-27 09:24:32,749][20347] Updated weights for policy 0, policy_version 544835 (0.0008) [2023-12-27 09:24:32,817][20347] Updated weights for policy 0, policy_version 544845 (0.0008) [2023-12-27 09:24:32,881][20347] Updated weights for policy 0, policy_version 544855 (0.0008) [2023-12-27 09:24:33,171][20381] Updated weights for policy 1, policy_version 544632 (0.0007) [2023-12-27 09:24:33,223][20381] Updated weights for policy 1, policy_version 544642 (0.0007) [2023-12-27 09:24:33,270][20381] Updated weights for policy 1, policy_version 544652 (0.0007) [2023-12-27 09:24:33,521][20347] Updated weights for policy 0, policy_version 544865 (0.0006) [2023-12-27 09:24:33,575][20347] Updated weights for policy 0, policy_version 544875 (0.0008) [2023-12-27 09:24:33,622][20347] Updated weights for policy 0, policy_version 544885 (0.0007) [2023-12-27 09:24:33,671][20347] Updated weights for policy 0, policy_version 544895 (0.0008) [2023-12-27 09:24:33,972][20381] Updated weights for policy 1, policy_version 544662 (0.0008) [2023-12-27 09:24:34,026][20381] Updated weights for policy 1, policy_version 544672 (0.0008) [2023-12-27 09:24:34,075][20381] Updated weights for policy 1, policy_version 544682 (0.0008) [2023-12-27 09:24:34,360][20347] Updated weights for policy 0, policy_version 544905 (0.0007) [2023-12-27 09:24:34,418][20347] Updated weights for policy 0, policy_version 544915 (0.0007) [2023-12-27 09:24:34,468][20347] Updated weights for policy 0, policy_version 544925 (0.0007) [2023-12-27 09:24:34,791][20381] Updated weights for policy 1, policy_version 544692 (0.0009) [2023-12-27 09:24:34,843][20381] Updated weights for policy 1, policy_version 544702 (0.0011) [2023-12-27 09:24:34,891][20381] Updated weights for policy 1, policy_version 544712 (0.0009) [2023-12-27 09:24:35,113][20347] Updated weights for policy 0, policy_version 544935 (0.0007) [2023-12-27 09:24:35,175][20347] Updated weights for policy 0, policy_version 544945 (0.0008) [2023-12-27 09:24:35,239][20347] Updated weights for policy 0, policy_version 544955 (0.0007) [2023-12-27 09:24:35,583][20381] Updated weights for policy 1, policy_version 544722 (0.0007) [2023-12-27 09:24:35,634][20381] Updated weights for policy 1, policy_version 544732 (0.0008) [2023-12-27 09:24:35,689][20381] Updated weights for policy 1, policy_version 544742 (0.0008) [2023-12-27 09:24:35,744][20381] Updated weights for policy 1, policy_version 544752 (0.0009) [2023-12-27 09:24:35,862][20347] Updated weights for policy 0, policy_version 544965 (0.0007) [2023-12-27 09:24:35,919][20347] Updated weights for policy 0, policy_version 544975 (0.0007) [2023-12-27 09:24:35,975][20347] Updated weights for policy 0, policy_version 544985 (0.0008) [2023-12-27 09:24:36,505][20381] Updated weights for policy 1, policy_version 544762 (0.0011) [2023-12-27 09:24:36,563][20381] Updated weights for policy 1, policy_version 544772 (0.0010) [2023-12-27 09:24:36,623][20381] Updated weights for policy 1, policy_version 544782 (0.0011) [2023-12-27 09:24:36,697][20347] Updated weights for policy 0, policy_version 544995 (0.0008) [2023-12-27 09:24:36,748][20347] Updated weights for policy 0, policy_version 545005 (0.0008) [2023-12-27 09:24:36,796][20347] Updated weights for policy 0, policy_version 545015 (0.0010) [2023-12-27 09:24:37,182][20381] Updated weights for policy 1, policy_version 544792 (0.0008) [2023-12-27 09:24:37,238][20381] Updated weights for policy 1, policy_version 544802 (0.0006) [2023-12-27 09:24:37,296][20381] Updated weights for policy 1, policy_version 544812 (0.0005) [2023-12-27 09:24:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 279035904. Throughput: 0: 10315.6, 1: 10520.9. Samples: 279002840. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:37,363][19170] Avg episode reward: [(0, '990.630'), (1, '727.830')] [2023-12-27 09:24:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000544816_139493376.pth... [2023-12-27 09:24:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000545024_139542528.pth... [2023-12-27 09:24:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000543840_139239424.pth [2023-12-27 09:24:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000543568_139173888.pth [2023-12-27 09:24:37,529][20347] Updated weights for policy 0, policy_version 545025 (0.0009) [2023-12-27 09:24:37,580][20347] Updated weights for policy 0, policy_version 545035 (0.0006) [2023-12-27 09:24:37,638][20347] Updated weights for policy 0, policy_version 545045 (0.0006) [2023-12-27 09:24:37,694][20347] Updated weights for policy 0, policy_version 545055 (0.0009) [2023-12-27 09:24:37,867][20381] Updated weights for policy 1, policy_version 544822 (0.0006) [2023-12-27 09:24:37,917][20381] Updated weights for policy 1, policy_version 544832 (0.0009) [2023-12-27 09:24:37,984][20381] Updated weights for policy 1, policy_version 544842 (0.0008) [2023-12-27 09:24:38,335][20347] Updated weights for policy 0, policy_version 545065 (0.0007) [2023-12-27 09:24:38,399][20347] Updated weights for policy 0, policy_version 545075 (0.0006) [2023-12-27 09:24:38,464][20347] Updated weights for policy 0, policy_version 545085 (0.0007) [2023-12-27 09:24:38,631][20381] Updated weights for policy 1, policy_version 544852 (0.0008) [2023-12-27 09:24:38,690][20381] Updated weights for policy 1, policy_version 544862 (0.0008) [2023-12-27 09:24:38,746][20381] Updated weights for policy 1, policy_version 544872 (0.0007) [2023-12-27 09:24:39,069][20347] Updated weights for policy 0, policy_version 545095 (0.0008) [2023-12-27 09:24:39,126][20347] Updated weights for policy 0, policy_version 545105 (0.0006) [2023-12-27 09:24:39,186][20347] Updated weights for policy 0, policy_version 545115 (0.0008) [2023-12-27 09:24:39,413][20381] Updated weights for policy 1, policy_version 544882 (0.0008) [2023-12-27 09:24:39,473][20381] Updated weights for policy 1, policy_version 544892 (0.0007) [2023-12-27 09:24:39,526][20381] Updated weights for policy 1, policy_version 544902 (0.0008) [2023-12-27 09:24:39,572][20381] Updated weights for policy 1, policy_version 544912 (0.0009) [2023-12-27 09:24:39,929][20347] Updated weights for policy 0, policy_version 545125 (0.0007) [2023-12-27 09:24:39,980][20347] Updated weights for policy 0, policy_version 545135 (0.0007) [2023-12-27 09:24:40,037][20347] Updated weights for policy 0, policy_version 545145 (0.0008) [2023-12-27 09:24:40,269][20381] Updated weights for policy 1, policy_version 544922 (0.0008) [2023-12-27 09:24:40,334][20381] Updated weights for policy 1, policy_version 544932 (0.0008) [2023-12-27 09:24:40,394][20381] Updated weights for policy 1, policy_version 544942 (0.0008) [2023-12-27 09:24:40,687][20347] Updated weights for policy 0, policy_version 545155 (0.0008) [2023-12-27 09:24:40,745][20347] Updated weights for policy 0, policy_version 545165 (0.0007) [2023-12-27 09:24:40,789][20347] Updated weights for policy 0, policy_version 545175 (0.0007) [2023-12-27 09:24:40,972][20381] Updated weights for policy 1, policy_version 544952 (0.0006) [2023-12-27 09:24:41,023][20381] Updated weights for policy 1, policy_version 544962 (0.0006) [2023-12-27 09:24:41,083][20381] Updated weights for policy 1, policy_version 544972 (0.0009) [2023-12-27 09:24:41,487][20347] Updated weights for policy 0, policy_version 545185 (0.0008) [2023-12-27 09:24:41,540][20347] Updated weights for policy 0, policy_version 545195 (0.0011) [2023-12-27 09:24:41,603][20347] Updated weights for policy 0, policy_version 545205 (0.0010) [2023-12-27 09:24:41,667][20347] Updated weights for policy 0, policy_version 545215 (0.0009) [2023-12-27 09:24:41,788][20381] Updated weights for policy 1, policy_version 544982 (0.0007) [2023-12-27 09:24:41,841][20381] Updated weights for policy 1, policy_version 544992 (0.0007) [2023-12-27 09:24:41,900][20381] Updated weights for policy 1, policy_version 545002 (0.0008) [2023-12-27 09:24:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 279134208. Throughput: 0: 10312.6, 1: 10532.5. Samples: 279128788. Policy #0 lag: (min: 31.0, avg: 32.5, max: 62.0) [2023-12-27 09:24:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '776.260')] [2023-12-27 09:24:42,382][20347] Updated weights for policy 0, policy_version 545225 (0.0010) [2023-12-27 09:24:42,448][20347] Updated weights for policy 0, policy_version 545235 (0.0010) [2023-12-27 09:24:42,513][20347] Updated weights for policy 0, policy_version 545245 (0.0010) [2023-12-27 09:24:42,536][20381] Updated weights for policy 1, policy_version 545012 (0.0007) [2023-12-27 09:24:42,591][20381] Updated weights for policy 1, policy_version 545022 (0.0006) [2023-12-27 09:24:42,650][20381] Updated weights for policy 1, policy_version 545032 (0.0007) [2023-12-27 09:24:43,190][20347] Updated weights for policy 0, policy_version 545255 (0.0010) [2023-12-27 09:24:43,244][20347] Updated weights for policy 0, policy_version 545265 (0.0010) [2023-12-27 09:24:43,277][20381] Updated weights for policy 1, policy_version 545042 (0.0005) [2023-12-27 09:24:43,307][20347] Updated weights for policy 0, policy_version 545275 (0.0010) [2023-12-27 09:24:43,340][20381] Updated weights for policy 1, policy_version 545052 (0.0007) [2023-12-27 09:24:43,388][20381] Updated weights for policy 1, policy_version 545062 (0.0008) [2023-12-27 09:24:43,442][20381] Updated weights for policy 1, policy_version 545072 (0.0006) [2023-12-27 09:24:43,928][20347] Updated weights for policy 0, policy_version 545285 (0.0005) [2023-12-27 09:24:43,986][20347] Updated weights for policy 0, policy_version 545295 (0.0010) [2023-12-27 09:24:44,041][20347] Updated weights for policy 0, policy_version 545305 (0.0010) [2023-12-27 09:24:44,057][20381] Updated weights for policy 1, policy_version 545082 (0.0005) [2023-12-27 09:24:44,120][20381] Updated weights for policy 1, policy_version 545092 (0.0005) [2023-12-27 09:24:44,177][20381] Updated weights for policy 1, policy_version 545102 (0.0006) [2023-12-27 09:24:44,710][20347] Updated weights for policy 0, policy_version 545315 (0.0010) [2023-12-27 09:24:44,768][20347] Updated weights for policy 0, policy_version 545325 (0.0010) [2023-12-27 09:24:44,828][20381] Updated weights for policy 1, policy_version 545112 (0.0007) [2023-12-27 09:24:44,829][20347] Updated weights for policy 0, policy_version 545335 (0.0011) [2023-12-27 09:24:44,891][20381] Updated weights for policy 1, policy_version 545122 (0.0006) [2023-12-27 09:24:44,952][20381] Updated weights for policy 1, policy_version 545132 (0.0009) [2023-12-27 09:24:45,425][20347] Updated weights for policy 0, policy_version 545345 (0.0008) [2023-12-27 09:24:45,485][20347] Updated weights for policy 0, policy_version 545355 (0.0011) [2023-12-27 09:24:45,542][20347] Updated weights for policy 0, policy_version 545365 (0.0010) [2023-12-27 09:24:45,590][20347] Updated weights for policy 0, policy_version 545375 (0.0010) [2023-12-27 09:24:45,672][20381] Updated weights for policy 1, policy_version 545142 (0.0008) [2023-12-27 09:24:45,725][20381] Updated weights for policy 1, policy_version 545152 (0.0007) [2023-12-27 09:24:45,776][20381] Updated weights for policy 1, policy_version 545162 (0.0007) [2023-12-27 09:24:46,263][20347] Updated weights for policy 0, policy_version 545385 (0.0010) [2023-12-27 09:24:46,325][20347] Updated weights for policy 0, policy_version 545395 (0.0010) [2023-12-27 09:24:46,376][20347] Updated weights for policy 0, policy_version 545405 (0.0010) [2023-12-27 09:24:46,418][20381] Updated weights for policy 1, policy_version 545172 (0.0007) [2023-12-27 09:24:46,466][20381] Updated weights for policy 1, policy_version 545182 (0.0007) [2023-12-27 09:24:46,513][20381] Updated weights for policy 1, policy_version 545192 (0.0007) [2023-12-27 09:24:47,044][20347] Updated weights for policy 0, policy_version 545415 (0.0007) [2023-12-27 09:24:47,095][20347] Updated weights for policy 0, policy_version 545425 (0.0008) [2023-12-27 09:24:47,150][20347] Updated weights for policy 0, policy_version 545435 (0.0006) [2023-12-27 09:24:47,188][20381] Updated weights for policy 1, policy_version 545202 (0.0008) [2023-12-27 09:24:47,241][20381] Updated weights for policy 1, policy_version 545212 (0.0007) [2023-12-27 09:24:47,289][20381] Updated weights for policy 1, policy_version 545222 (0.0008) [2023-12-27 09:24:47,339][20381] Updated weights for policy 1, policy_version 545232 (0.0008) [2023-12-27 09:24:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 279248896. Throughput: 0: 10277.9, 1: 10509.8. Samples: 279253756. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:24:47,363][19170] Avg episode reward: [(0, '952.310'), (1, '795.880')] [2023-12-27 09:24:47,789][20347] Updated weights for policy 0, policy_version 545445 (0.0006) [2023-12-27 09:24:47,847][20347] Updated weights for policy 0, policy_version 545455 (0.0005) [2023-12-27 09:24:47,908][20347] Updated weights for policy 0, policy_version 545465 (0.0007) [2023-12-27 09:24:48,058][20381] Updated weights for policy 1, policy_version 545242 (0.0008) [2023-12-27 09:24:48,118][20381] Updated weights for policy 1, policy_version 545252 (0.0008) [2023-12-27 09:24:48,174][20381] Updated weights for policy 1, policy_version 545262 (0.0009) [2023-12-27 09:24:48,527][20347] Updated weights for policy 0, policy_version 545475 (0.0006) [2023-12-27 09:24:48,580][20347] Updated weights for policy 0, policy_version 545485 (0.0008) [2023-12-27 09:24:48,635][20347] Updated weights for policy 0, policy_version 545495 (0.0008) [2023-12-27 09:24:48,849][20381] Updated weights for policy 1, policy_version 545272 (0.0008) [2023-12-27 09:24:48,911][20381] Updated weights for policy 1, policy_version 545282 (0.0007) [2023-12-27 09:24:48,970][20381] Updated weights for policy 1, policy_version 545292 (0.0008) [2023-12-27 09:24:49,282][20347] Updated weights for policy 0, policy_version 545505 (0.0008) [2023-12-27 09:24:49,349][20347] Updated weights for policy 0, policy_version 545515 (0.0006) [2023-12-27 09:24:49,406][20347] Updated weights for policy 0, policy_version 545525 (0.0007) [2023-12-27 09:24:49,464][20347] Updated weights for policy 0, policy_version 545535 (0.0008) [2023-12-27 09:24:49,671][20381] Updated weights for policy 1, policy_version 545302 (0.0005) [2023-12-27 09:24:49,736][20381] Updated weights for policy 1, policy_version 545312 (0.0008) [2023-12-27 09:24:49,803][20381] Updated weights for policy 1, policy_version 545322 (0.0008) [2023-12-27 09:24:50,132][20347] Updated weights for policy 0, policy_version 545545 (0.0010) [2023-12-27 09:24:50,198][20347] Updated weights for policy 0, policy_version 545555 (0.0011) [2023-12-27 09:24:50,263][20347] Updated weights for policy 0, policy_version 545565 (0.0009) [2023-12-27 09:24:50,456][20381] Updated weights for policy 1, policy_version 545332 (0.0008) [2023-12-27 09:24:50,498][20381] Updated weights for policy 1, policy_version 545342 (0.0007) [2023-12-27 09:24:50,549][20381] Updated weights for policy 1, policy_version 545352 (0.0008) [2023-12-27 09:24:50,909][20347] Updated weights for policy 0, policy_version 545575 (0.0007) [2023-12-27 09:24:50,919][20291] KL-divergence is very high: 100.7726 [2023-12-27 09:24:50,967][20347] Updated weights for policy 0, policy_version 545585 (0.0006) [2023-12-27 09:24:51,032][20347] Updated weights for policy 0, policy_version 545595 (0.0006) [2023-12-27 09:24:51,268][20381] Updated weights for policy 1, policy_version 545362 (0.0008) [2023-12-27 09:24:51,328][20381] Updated weights for policy 1, policy_version 545372 (0.0008) [2023-12-27 09:24:51,389][20381] Updated weights for policy 1, policy_version 545382 (0.0008) [2023-12-27 09:24:51,448][20381] Updated weights for policy 1, policy_version 545392 (0.0008) [2023-12-27 09:24:51,699][20347] Updated weights for policy 0, policy_version 545605 (0.0008) [2023-12-27 09:24:51,760][20347] Updated weights for policy 0, policy_version 545615 (0.0008) [2023-12-27 09:24:51,821][20347] Updated weights for policy 0, policy_version 545625 (0.0008) [2023-12-27 09:24:52,182][20381] Updated weights for policy 1, policy_version 545402 (0.0008) [2023-12-27 09:24:52,233][20381] Updated weights for policy 1, policy_version 545412 (0.0009) [2023-12-27 09:24:52,301][20381] Updated weights for policy 1, policy_version 545422 (0.0007) [2023-12-27 09:24:52,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 279347200. Throughput: 0: 10292.5, 1: 10490.5. Samples: 279316064. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:24:52,363][19170] Avg episode reward: [(0, '810.580'), (1, '766.920')] [2023-12-27 09:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000545424_139649024.pth... [2023-12-27 09:24:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000545632_139698176.pth... [2023-12-27 09:24:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000544416_139386880.pth [2023-12-27 09:24:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000544176_139329536.pth [2023-12-27 09:24:52,427][20347] Updated weights for policy 0, policy_version 545635 (0.0007) [2023-12-27 09:24:52,480][20347] Updated weights for policy 0, policy_version 545645 (0.0011) [2023-12-27 09:24:52,530][20347] Updated weights for policy 0, policy_version 545655 (0.0005) [2023-12-27 09:24:53,021][20381] Updated weights for policy 1, policy_version 545432 (0.0008) [2023-12-27 09:24:53,089][20381] Updated weights for policy 1, policy_version 545442 (0.0008) [2023-12-27 09:24:53,151][20381] Updated weights for policy 1, policy_version 545452 (0.0007) [2023-12-27 09:24:53,157][20347] Updated weights for policy 0, policy_version 545665 (0.0009) [2023-12-27 09:24:53,211][20347] Updated weights for policy 0, policy_version 545675 (0.0005) [2023-12-27 09:24:53,268][20347] Updated weights for policy 0, policy_version 545685 (0.0005) [2023-12-27 09:24:53,318][20347] Updated weights for policy 0, policy_version 545695 (0.0005) [2023-12-27 09:24:53,791][20381] Updated weights for policy 1, policy_version 545462 (0.0007) [2023-12-27 09:24:53,853][20381] Updated weights for policy 1, policy_version 545472 (0.0008) [2023-12-27 09:24:53,906][20347] Updated weights for policy 0, policy_version 545705 (0.0007) [2023-12-27 09:24:53,923][20381] Updated weights for policy 1, policy_version 545482 (0.0007) [2023-12-27 09:24:53,960][20347] Updated weights for policy 0, policy_version 545715 (0.0005) [2023-12-27 09:24:54,019][20347] Updated weights for policy 0, policy_version 545725 (0.0010) [2023-12-27 09:24:54,532][20381] Updated weights for policy 1, policy_version 545492 (0.0006) [2023-12-27 09:24:54,591][20381] Updated weights for policy 1, policy_version 545502 (0.0005) [2023-12-27 09:24:54,649][20347] Updated weights for policy 0, policy_version 545735 (0.0007) [2023-12-27 09:24:54,650][20381] Updated weights for policy 1, policy_version 545512 (0.0005) [2023-12-27 09:24:54,677][20291] KL-divergence is very high: 149.4087 [2023-12-27 09:24:54,690][20291] KL-divergence is very high: 103.3497 [2023-12-27 09:24:54,711][20347] Updated weights for policy 0, policy_version 545745 (0.0006) [2023-12-27 09:24:54,764][20347] Updated weights for policy 0, policy_version 545755 (0.0005) [2023-12-27 09:24:55,243][20381] Updated weights for policy 1, policy_version 545522 (0.0006) [2023-12-27 09:24:55,291][20381] Updated weights for policy 1, policy_version 545532 (0.0007) [2023-12-27 09:24:55,341][20381] Updated weights for policy 1, policy_version 545542 (0.0007) [2023-12-27 09:24:55,380][20347] Updated weights for policy 0, policy_version 545765 (0.0006) [2023-12-27 09:24:55,393][20381] Updated weights for policy 1, policy_version 545552 (0.0006) [2023-12-27 09:24:55,437][20347] Updated weights for policy 0, policy_version 545775 (0.0009) [2023-12-27 09:24:55,492][20347] Updated weights for policy 0, policy_version 545785 (0.0010) [2023-12-27 09:24:56,100][20381] Updated weights for policy 1, policy_version 545562 (0.0006) [2023-12-27 09:24:56,162][20347] Updated weights for policy 0, policy_version 545795 (0.0010) [2023-12-27 09:24:56,164][20381] Updated weights for policy 1, policy_version 545572 (0.0005) [2023-12-27 09:24:56,217][20347] Updated weights for policy 0, policy_version 545805 (0.0011) [2023-12-27 09:24:56,221][20381] Updated weights for policy 1, policy_version 545582 (0.0006) [2023-12-27 09:24:56,272][20347] Updated weights for policy 0, policy_version 545815 (0.0010) [2023-12-27 09:24:56,808][20381] Updated weights for policy 1, policy_version 545592 (0.0009) [2023-12-27 09:24:56,860][20381] Updated weights for policy 1, policy_version 545602 (0.0007) [2023-12-27 09:24:56,908][20347] Updated weights for policy 0, policy_version 545825 (0.0010) [2023-12-27 09:24:56,918][20381] Updated weights for policy 1, policy_version 545612 (0.0007) [2023-12-27 09:24:56,959][20347] Updated weights for policy 0, policy_version 545835 (0.0007) [2023-12-27 09:24:57,011][20347] Updated weights for policy 0, policy_version 545845 (0.0007) [2023-12-27 09:24:57,064][20347] Updated weights for policy 0, policy_version 545855 (0.0005) [2023-12-27 09:24:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 279453696. Throughput: 0: 10425.2, 1: 10453.4. Samples: 279441684. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:24:57,363][19170] Avg episode reward: [(0, '679.970'), (1, '805.420')] [2023-12-27 09:24:57,656][20347] Updated weights for policy 0, policy_version 545865 (0.0007) [2023-12-27 09:24:57,660][20381] Updated weights for policy 1, policy_version 545622 (0.0006) [2023-12-27 09:24:57,706][20381] Updated weights for policy 1, policy_version 545632 (0.0005) [2023-12-27 09:24:57,707][20347] Updated weights for policy 0, policy_version 545875 (0.0006) [2023-12-27 09:24:57,762][20381] Updated weights for policy 1, policy_version 545642 (0.0007) [2023-12-27 09:24:57,766][20347] Updated weights for policy 0, policy_version 545885 (0.0005) [2023-12-27 09:24:58,376][20347] Updated weights for policy 0, policy_version 545895 (0.0007) [2023-12-27 09:24:58,446][20347] Updated weights for policy 0, policy_version 545905 (0.0006) [2023-12-27 09:24:58,500][20381] Updated weights for policy 1, policy_version 545652 (0.0008) [2023-12-27 09:24:58,509][20347] Updated weights for policy 0, policy_version 545915 (0.0007) [2023-12-27 09:24:58,559][20381] Updated weights for policy 1, policy_version 545662 (0.0008) [2023-12-27 09:24:58,619][20381] Updated weights for policy 1, policy_version 545672 (0.0009) [2023-12-27 09:24:59,246][20347] Updated weights for policy 0, policy_version 545925 (0.0006) [2023-12-27 09:24:59,320][20347] Updated weights for policy 0, policy_version 545935 (0.0009) [2023-12-27 09:24:59,383][20347] Updated weights for policy 0, policy_version 545945 (0.0007) [2023-12-27 09:24:59,387][20381] Updated weights for policy 1, policy_version 545682 (0.0010) [2023-12-27 09:24:59,437][20381] Updated weights for policy 1, policy_version 545692 (0.0007) [2023-12-27 09:24:59,492][20381] Updated weights for policy 1, policy_version 545702 (0.0006) [2023-12-27 09:24:59,550][20381] Updated weights for policy 1, policy_version 545712 (0.0007) [2023-12-27 09:25:00,035][20347] Updated weights for policy 0, policy_version 545955 (0.0007) [2023-12-27 09:25:00,097][20347] Updated weights for policy 0, policy_version 545965 (0.0010) [2023-12-27 09:25:00,153][20347] Updated weights for policy 0, policy_version 545975 (0.0010) [2023-12-27 09:25:00,197][20381] Updated weights for policy 1, policy_version 545722 (0.0006) [2023-12-27 09:25:00,255][20381] Updated weights for policy 1, policy_version 545732 (0.0007) [2023-12-27 09:25:00,314][20381] Updated weights for policy 1, policy_version 545742 (0.0007) [2023-12-27 09:25:00,779][20347] Updated weights for policy 0, policy_version 545985 (0.0010) [2023-12-27 09:25:00,837][20347] Updated weights for policy 0, policy_version 545995 (0.0005) [2023-12-27 09:25:00,898][20347] Updated weights for policy 0, policy_version 546005 (0.0005) [2023-12-27 09:25:00,948][20347] Updated weights for policy 0, policy_version 546015 (0.0010) [2023-12-27 09:25:00,949][20381] Updated weights for policy 1, policy_version 545752 (0.0006) [2023-12-27 09:25:01,013][20381] Updated weights for policy 1, policy_version 545762 (0.0005) [2023-12-27 09:25:01,075][20381] Updated weights for policy 1, policy_version 545772 (0.0006) [2023-12-27 09:25:01,594][20347] Updated weights for policy 0, policy_version 546025 (0.0010) [2023-12-27 09:25:01,639][20381] Updated weights for policy 1, policy_version 545782 (0.0007) [2023-12-27 09:25:01,655][20347] Updated weights for policy 0, policy_version 546035 (0.0009) [2023-12-27 09:25:01,699][20381] Updated weights for policy 1, policy_version 545792 (0.0009) [2023-12-27 09:25:01,716][20347] Updated weights for policy 0, policy_version 546045 (0.0007) [2023-12-27 09:25:01,757][20381] Updated weights for policy 1, policy_version 545802 (0.0007) [2023-12-27 09:25:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 279552000. Throughput: 0: 10451.7, 1: 10399.2. Samples: 279566820. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:02,363][19170] Avg episode reward: [(0, '557.520'), (1, '795.330')] [2023-12-27 09:25:02,432][20347] Updated weights for policy 0, policy_version 546055 (0.0011) [2023-12-27 09:25:02,438][20381] Updated weights for policy 1, policy_version 545812 (0.0008) [2023-12-27 09:25:02,482][20347] Updated weights for policy 0, policy_version 546065 (0.0010) [2023-12-27 09:25:02,493][20381] Updated weights for policy 1, policy_version 545822 (0.0005) [2023-12-27 09:25:02,529][20347] Updated weights for policy 0, policy_version 546075 (0.0007) [2023-12-27 09:25:02,549][20381] Updated weights for policy 1, policy_version 545832 (0.0005) [2023-12-27 09:25:03,144][20381] Updated weights for policy 1, policy_version 545842 (0.0005) [2023-12-27 09:25:03,198][20381] Updated weights for policy 1, policy_version 545852 (0.0005) [2023-12-27 09:25:03,252][20347] Updated weights for policy 0, policy_version 546085 (0.0007) [2023-12-27 09:25:03,259][20381] Updated weights for policy 1, policy_version 545862 (0.0006) [2023-12-27 09:25:03,311][20347] Updated weights for policy 0, policy_version 546095 (0.0006) [2023-12-27 09:25:03,313][20381] Updated weights for policy 1, policy_version 545872 (0.0010) [2023-12-27 09:25:03,361][20347] Updated weights for policy 0, policy_version 546105 (0.0007) [2023-12-27 09:25:03,945][20381] Updated weights for policy 1, policy_version 545882 (0.0007) [2023-12-27 09:25:04,000][20381] Updated weights for policy 1, policy_version 545892 (0.0010) [2023-12-27 09:25:04,062][20381] Updated weights for policy 1, policy_version 545902 (0.0011) [2023-12-27 09:25:04,072][20347] Updated weights for policy 0, policy_version 546115 (0.0007) [2023-12-27 09:25:04,130][20347] Updated weights for policy 0, policy_version 546125 (0.0007) [2023-12-27 09:25:04,188][20347] Updated weights for policy 0, policy_version 546135 (0.0007) [2023-12-27 09:25:04,743][20381] Updated weights for policy 1, policy_version 545912 (0.0010) [2023-12-27 09:25:04,791][20381] Updated weights for policy 1, policy_version 545922 (0.0010) [2023-12-27 09:25:04,848][20381] Updated weights for policy 1, policy_version 545932 (0.0010) [2023-12-27 09:25:04,882][20347] Updated weights for policy 0, policy_version 546145 (0.0007) [2023-12-27 09:25:04,933][20347] Updated weights for policy 0, policy_version 546155 (0.0007) [2023-12-27 09:25:04,980][20347] Updated weights for policy 0, policy_version 546165 (0.0007) [2023-12-27 09:25:05,028][20347] Updated weights for policy 0, policy_version 546175 (0.0007) [2023-12-27 09:25:05,540][20381] Updated weights for policy 1, policy_version 545942 (0.0011) [2023-12-27 09:25:05,595][20381] Updated weights for policy 1, policy_version 545952 (0.0011) [2023-12-27 09:25:05,657][20381] Updated weights for policy 1, policy_version 545962 (0.0011) [2023-12-27 09:25:05,751][20347] Updated weights for policy 0, policy_version 546185 (0.0007) [2023-12-27 09:25:05,808][20347] Updated weights for policy 0, policy_version 546195 (0.0007) [2023-12-27 09:25:05,868][20347] Updated weights for policy 0, policy_version 546205 (0.0008) [2023-12-27 09:25:06,345][20381] Updated weights for policy 1, policy_version 545972 (0.0011) [2023-12-27 09:25:06,404][20381] Updated weights for policy 1, policy_version 545982 (0.0011) [2023-12-27 09:25:06,457][20381] Updated weights for policy 1, policy_version 545992 (0.0011) [2023-12-27 09:25:06,570][20347] Updated weights for policy 0, policy_version 546215 (0.0008) [2023-12-27 09:25:06,622][20347] Updated weights for policy 0, policy_version 546225 (0.0010) [2023-12-27 09:25:06,668][20347] Updated weights for policy 0, policy_version 546235 (0.0010) [2023-12-27 09:25:07,081][20381] Updated weights for policy 1, policy_version 546002 (0.0008) [2023-12-27 09:25:07,140][20381] Updated weights for policy 1, policy_version 546012 (0.0008) [2023-12-27 09:25:07,198][20381] Updated weights for policy 1, policy_version 546022 (0.0008) [2023-12-27 09:25:07,259][20381] Updated weights for policy 1, policy_version 546032 (0.0009) [2023-12-27 09:25:07,340][20347] Updated weights for policy 0, policy_version 546245 (0.0010) [2023-12-27 09:25:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 279658496. Throughput: 0: 10411.0, 1: 10451.6. Samples: 279627988. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:07,363][19170] Avg episode reward: [(0, '453.120'), (1, '737.250')] [2023-12-27 09:25:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000546032_139804672.pth... [2023-12-27 09:25:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000544816_139493376.pth [2023-12-27 09:25:07,394][20347] Updated weights for policy 0, policy_version 546255 (0.0010) [2023-12-27 09:25:07,446][20347] Updated weights for policy 0, policy_version 546265 (0.0005) [2023-12-27 09:25:07,479][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000546272_139862016.pth... [2023-12-27 09:25:07,482][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000545024_139542528.pth [2023-12-27 09:25:07,939][20381] Updated weights for policy 1, policy_version 546042 (0.0010) [2023-12-27 09:25:07,985][20381] Updated weights for policy 1, policy_version 546052 (0.0010) [2023-12-27 09:25:08,036][20381] Updated weights for policy 1, policy_version 546062 (0.0010) [2023-12-27 09:25:08,057][20347] Updated weights for policy 0, policy_version 546275 (0.0006) [2023-12-27 09:25:08,114][20347] Updated weights for policy 0, policy_version 546285 (0.0005) [2023-12-27 09:25:08,172][20347] Updated weights for policy 0, policy_version 546295 (0.0005) [2023-12-27 09:25:08,732][20381] Updated weights for policy 1, policy_version 546072 (0.0006) [2023-12-27 09:25:08,786][20347] Updated weights for policy 0, policy_version 546305 (0.0006) [2023-12-27 09:25:08,796][20381] Updated weights for policy 1, policy_version 546082 (0.0006) [2023-12-27 09:25:08,838][20347] Updated weights for policy 0, policy_version 546315 (0.0010) [2023-12-27 09:25:08,847][20381] Updated weights for policy 1, policy_version 546092 (0.0006) [2023-12-27 09:25:08,890][20347] Updated weights for policy 0, policy_version 546325 (0.0008) [2023-12-27 09:25:08,945][20347] Updated weights for policy 0, policy_version 546335 (0.0008) [2023-12-27 09:25:09,546][20381] Updated weights for policy 1, policy_version 546102 (0.0007) [2023-12-27 09:25:09,603][20347] Updated weights for policy 0, policy_version 546345 (0.0007) [2023-12-27 09:25:09,604][20381] Updated weights for policy 1, policy_version 546112 (0.0006) [2023-12-27 09:25:09,663][20347] Updated weights for policy 0, policy_version 546355 (0.0005) [2023-12-27 09:25:09,665][20381] Updated weights for policy 1, policy_version 546122 (0.0007) [2023-12-27 09:25:09,719][20347] Updated weights for policy 0, policy_version 546365 (0.0006) [2023-12-27 09:25:10,361][20381] Updated weights for policy 1, policy_version 546132 (0.0008) [2023-12-27 09:25:10,412][20347] Updated weights for policy 0, policy_version 546375 (0.0008) [2023-12-27 09:25:10,418][20381] Updated weights for policy 1, policy_version 546142 (0.0006) [2023-12-27 09:25:10,463][20347] Updated weights for policy 0, policy_version 546385 (0.0008) [2023-12-27 09:25:10,468][20381] Updated weights for policy 1, policy_version 546152 (0.0006) [2023-12-27 09:25:10,521][20347] Updated weights for policy 0, policy_version 546395 (0.0006) [2023-12-27 09:25:11,193][20381] Updated weights for policy 1, policy_version 546162 (0.0007) [2023-12-27 09:25:11,208][20347] Updated weights for policy 0, policy_version 546405 (0.0007) [2023-12-27 09:25:11,247][20381] Updated weights for policy 1, policy_version 546172 (0.0006) [2023-12-27 09:25:11,270][20347] Updated weights for policy 0, policy_version 546415 (0.0008) [2023-12-27 09:25:11,307][20381] Updated weights for policy 1, policy_version 546182 (0.0006) [2023-12-27 09:25:11,327][20347] Updated weights for policy 0, policy_version 546425 (0.0007) [2023-12-27 09:25:11,374][20381] Updated weights for policy 1, policy_version 546192 (0.0008) [2023-12-27 09:25:11,996][20381] Updated weights for policy 1, policy_version 546202 (0.0006) [2023-12-27 09:25:12,055][20381] Updated weights for policy 1, policy_version 546212 (0.0006) [2023-12-27 09:25:12,080][20347] Updated weights for policy 0, policy_version 546435 (0.0009) [2023-12-27 09:25:12,106][20381] Updated weights for policy 1, policy_version 546222 (0.0005) [2023-12-27 09:25:12,131][20347] Updated weights for policy 0, policy_version 546445 (0.0006) [2023-12-27 09:25:12,177][20347] Updated weights for policy 0, policy_version 546455 (0.0008) [2023-12-27 09:25:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 279764992. Throughput: 0: 10438.5, 1: 10406.6. Samples: 279751136. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:12,363][19170] Avg episode reward: [(0, '58.640'), (1, '785.850')] [2023-12-27 09:25:12,698][20381] Updated weights for policy 1, policy_version 546232 (0.0007) [2023-12-27 09:25:12,749][20381] Updated weights for policy 1, policy_version 546242 (0.0007) [2023-12-27 09:25:12,802][20381] Updated weights for policy 1, policy_version 546252 (0.0007) [2023-12-27 09:25:12,939][20347] Updated weights for policy 0, policy_version 546465 (0.0008) [2023-12-27 09:25:12,990][20347] Updated weights for policy 0, policy_version 546475 (0.0010) [2023-12-27 09:25:13,052][20347] Updated weights for policy 0, policy_version 546485 (0.0010) [2023-12-27 09:25:13,103][20347] Updated weights for policy 0, policy_version 546495 (0.0010) [2023-12-27 09:25:13,486][20381] Updated weights for policy 1, policy_version 546262 (0.0010) [2023-12-27 09:25:13,539][20381] Updated weights for policy 1, policy_version 546272 (0.0011) [2023-12-27 09:25:13,581][20381] Updated weights for policy 1, policy_version 546282 (0.0010) [2023-12-27 09:25:13,723][20347] Updated weights for policy 0, policy_version 546505 (0.0006) [2023-12-27 09:25:13,771][20347] Updated weights for policy 0, policy_version 546515 (0.0005) [2023-12-27 09:25:13,821][20347] Updated weights for policy 0, policy_version 546525 (0.0005) [2023-12-27 09:25:14,293][20381] Updated weights for policy 1, policy_version 546292 (0.0010) [2023-12-27 09:25:14,348][20381] Updated weights for policy 1, policy_version 546302 (0.0011) [2023-12-27 09:25:14,349][20292] KL-divergence is very high: 621.5734 [2023-12-27 09:25:14,397][20292] KL-divergence is very high: 145.0137 [2023-12-27 09:25:14,410][20381] Updated weights for policy 1, policy_version 546312 (0.0011) [2023-12-27 09:25:14,452][20347] Updated weights for policy 0, policy_version 546535 (0.0005) [2023-12-27 09:25:14,507][20347] Updated weights for policy 0, policy_version 546545 (0.0007) [2023-12-27 09:25:14,556][20347] Updated weights for policy 0, policy_version 546555 (0.0006) [2023-12-27 09:25:15,103][20381] Updated weights for policy 1, policy_version 546322 (0.0010) [2023-12-27 09:25:15,158][20381] Updated weights for policy 1, policy_version 546332 (0.0009) [2023-12-27 09:25:15,182][20347] Updated weights for policy 0, policy_version 546565 (0.0010) [2023-12-27 09:25:15,223][20381] Updated weights for policy 1, policy_version 546342 (0.0007) [2023-12-27 09:25:15,245][20347] Updated weights for policy 0, policy_version 546575 (0.0010) [2023-12-27 09:25:15,281][20381] Updated weights for policy 1, policy_version 546352 (0.0007) [2023-12-27 09:25:15,304][20347] Updated weights for policy 0, policy_version 546585 (0.0009) [2023-12-27 09:25:15,942][20347] Updated weights for policy 0, policy_version 546595 (0.0008) [2023-12-27 09:25:15,978][20381] Updated weights for policy 1, policy_version 546362 (0.0009) [2023-12-27 09:25:15,998][20347] Updated weights for policy 0, policy_version 546605 (0.0007) [2023-12-27 09:25:16,036][20381] Updated weights for policy 1, policy_version 546372 (0.0008) [2023-12-27 09:25:16,063][20347] Updated weights for policy 0, policy_version 546615 (0.0006) [2023-12-27 09:25:16,098][20381] Updated weights for policy 1, policy_version 546382 (0.0008) [2023-12-27 09:25:16,692][20347] Updated weights for policy 0, policy_version 546625 (0.0006) [2023-12-27 09:25:16,738][20347] Updated weights for policy 0, policy_version 546635 (0.0006) [2023-12-27 09:25:16,743][20381] Updated weights for policy 1, policy_version 546392 (0.0007) [2023-12-27 09:25:16,787][20347] Updated weights for policy 0, policy_version 546645 (0.0006) [2023-12-27 09:25:16,796][20381] Updated weights for policy 1, policy_version 546402 (0.0006) [2023-12-27 09:25:16,836][20347] Updated weights for policy 0, policy_version 546655 (0.0006) [2023-12-27 09:25:16,846][20381] Updated weights for policy 1, policy_version 546412 (0.0006) [2023-12-27 09:25:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 279863296. Throughput: 0: 10446.8, 1: 10343.8. Samples: 279875324. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:17,363][19170] Avg episode reward: [(0, '57.450'), (1, '776.530')] [2023-12-27 09:25:17,508][20347] Updated weights for policy 0, policy_version 546665 (0.0010) [2023-12-27 09:25:17,564][20381] Updated weights for policy 1, policy_version 546422 (0.0006) [2023-12-27 09:25:17,572][20347] Updated weights for policy 0, policy_version 546675 (0.0009) [2023-12-27 09:25:17,622][20381] Updated weights for policy 1, policy_version 546432 (0.0008) [2023-12-27 09:25:17,627][20347] Updated weights for policy 0, policy_version 546685 (0.0006) [2023-12-27 09:25:17,668][20381] Updated weights for policy 1, policy_version 546442 (0.0006) [2023-12-27 09:25:18,251][20347] Updated weights for policy 0, policy_version 546695 (0.0007) [2023-12-27 09:25:18,304][20347] Updated weights for policy 0, policy_version 546705 (0.0008) [2023-12-27 09:25:18,354][20381] Updated weights for policy 1, policy_version 546452 (0.0006) [2023-12-27 09:25:18,362][20347] Updated weights for policy 0, policy_version 546715 (0.0007) [2023-12-27 09:25:18,421][20381] Updated weights for policy 1, policy_version 546462 (0.0007) [2023-12-27 09:25:18,481][20381] Updated weights for policy 1, policy_version 546472 (0.0008) [2023-12-27 09:25:19,087][20347] Updated weights for policy 0, policy_version 546725 (0.0007) [2023-12-27 09:25:19,143][20381] Updated weights for policy 1, policy_version 546482 (0.0008) [2023-12-27 09:25:19,167][20347] Updated weights for policy 0, policy_version 546735 (0.0006) [2023-12-27 09:25:19,188][20381] Updated weights for policy 1, policy_version 546492 (0.0005) [2023-12-27 09:25:19,223][20347] Updated weights for policy 0, policy_version 546745 (0.0007) [2023-12-27 09:25:19,246][20381] Updated weights for policy 1, policy_version 546502 (0.0007) [2023-12-27 09:25:19,298][20381] Updated weights for policy 1, policy_version 546512 (0.0007) [2023-12-27 09:25:19,937][20347] Updated weights for policy 0, policy_version 546755 (0.0009) [2023-12-27 09:25:19,994][20347] Updated weights for policy 0, policy_version 546765 (0.0011) [2023-12-27 09:25:20,052][20347] Updated weights for policy 0, policy_version 546775 (0.0010) [2023-12-27 09:25:20,102][20381] Updated weights for policy 1, policy_version 546522 (0.0007) [2023-12-27 09:25:20,162][20381] Updated weights for policy 1, policy_version 546532 (0.0008) [2023-12-27 09:25:20,219][20381] Updated weights for policy 1, policy_version 546542 (0.0007) [2023-12-27 09:25:20,782][20347] Updated weights for policy 0, policy_version 546785 (0.0011) [2023-12-27 09:25:20,841][20347] Updated weights for policy 0, policy_version 546795 (0.0010) [2023-12-27 09:25:20,903][20347] Updated weights for policy 0, policy_version 546805 (0.0011) [2023-12-27 09:25:20,953][20381] Updated weights for policy 1, policy_version 546552 (0.0006) [2023-12-27 09:25:20,963][20347] Updated weights for policy 0, policy_version 546815 (0.0010) [2023-12-27 09:25:21,016][20381] Updated weights for policy 1, policy_version 546562 (0.0007) [2023-12-27 09:25:21,082][20381] Updated weights for policy 1, policy_version 546572 (0.0008) [2023-12-27 09:25:21,712][20347] Updated weights for policy 0, policy_version 546825 (0.0009) [2023-12-27 09:25:21,772][20347] Updated weights for policy 0, policy_version 546835 (0.0009) [2023-12-27 09:25:21,832][20347] Updated weights for policy 0, policy_version 546845 (0.0010) [2023-12-27 09:25:21,857][20381] Updated weights for policy 1, policy_version 546582 (0.0008) [2023-12-27 09:25:21,906][20381] Updated weights for policy 1, policy_version 546592 (0.0007) [2023-12-27 09:25:21,957][20381] Updated weights for policy 1, policy_version 546602 (0.0007) [2023-12-27 09:25:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 279961600. Throughput: 0: 10416.9, 1: 10298.8. Samples: 279935048. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:22,363][19170] Avg episode reward: [(0, '87.120'), (1, '795.820')] [2023-12-27 09:25:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000546608_139952128.pth... [2023-12-27 09:25:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000546848_140009472.pth... [2023-12-27 09:25:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000545424_139649024.pth [2023-12-27 09:25:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000545632_139698176.pth [2023-12-27 09:25:22,544][20347] Updated weights for policy 0, policy_version 546855 (0.0010) [2023-12-27 09:25:22,606][20347] Updated weights for policy 0, policy_version 546865 (0.0010) [2023-12-27 09:25:22,673][20347] Updated weights for policy 0, policy_version 546875 (0.0011) [2023-12-27 09:25:22,693][20381] Updated weights for policy 1, policy_version 546612 (0.0008) [2023-12-27 09:25:22,760][20381] Updated weights for policy 1, policy_version 546622 (0.0008) [2023-12-27 09:25:22,821][20381] Updated weights for policy 1, policy_version 546632 (0.0008) [2023-12-27 09:25:23,362][20347] Updated weights for policy 0, policy_version 546885 (0.0010) [2023-12-27 09:25:23,409][20347] Updated weights for policy 0, policy_version 546895 (0.0007) [2023-12-27 09:25:23,452][20381] Updated weights for policy 1, policy_version 546642 (0.0007) [2023-12-27 09:25:23,464][20347] Updated weights for policy 0, policy_version 546905 (0.0007) [2023-12-27 09:25:23,515][20381] Updated weights for policy 1, policy_version 546652 (0.0005) [2023-12-27 09:25:23,564][20381] Updated weights for policy 1, policy_version 546662 (0.0008) [2023-12-27 09:25:23,609][20381] Updated weights for policy 1, policy_version 546672 (0.0010) [2023-12-27 09:25:24,127][20347] Updated weights for policy 0, policy_version 546915 (0.0006) [2023-12-27 09:25:24,183][20347] Updated weights for policy 0, policy_version 546925 (0.0007) [2023-12-27 09:25:24,205][20381] Updated weights for policy 1, policy_version 546682 (0.0011) [2023-12-27 09:25:24,243][20347] Updated weights for policy 0, policy_version 546935 (0.0007) [2023-12-27 09:25:24,257][20381] Updated weights for policy 1, policy_version 546692 (0.0011) [2023-12-27 09:25:24,315][20381] Updated weights for policy 1, policy_version 546702 (0.0010) [2023-12-27 09:25:24,815][20347] Updated weights for policy 0, policy_version 546945 (0.0005) [2023-12-27 09:25:24,873][20347] Updated weights for policy 0, policy_version 546955 (0.0007) [2023-12-27 09:25:24,883][20381] Updated weights for policy 1, policy_version 546712 (0.0006) [2023-12-27 09:25:24,931][20347] Updated weights for policy 0, policy_version 546965 (0.0006) [2023-12-27 09:25:24,937][20381] Updated weights for policy 1, policy_version 546722 (0.0010) [2023-12-27 09:25:24,983][20347] Updated weights for policy 0, policy_version 546975 (0.0005) [2023-12-27 09:25:24,989][20381] Updated weights for policy 1, policy_version 546732 (0.0010) [2023-12-27 09:25:25,584][20347] Updated weights for policy 0, policy_version 546985 (0.0006) [2023-12-27 09:25:25,618][20381] Updated weights for policy 1, policy_version 546742 (0.0011) [2023-12-27 09:25:25,657][20347] Updated weights for policy 0, policy_version 546995 (0.0007) [2023-12-27 09:25:25,681][20381] Updated weights for policy 1, policy_version 546752 (0.0008) [2023-12-27 09:25:25,713][20347] Updated weights for policy 0, policy_version 547005 (0.0007) [2023-12-27 09:25:25,734][20381] Updated weights for policy 1, policy_version 546762 (0.0007) [2023-12-27 09:25:26,280][20381] Updated weights for policy 1, policy_version 546772 (0.0005) [2023-12-27 09:25:26,327][20381] Updated weights for policy 1, policy_version 546782 (0.0005) [2023-12-27 09:25:26,348][20347] Updated weights for policy 0, policy_version 547015 (0.0009) [2023-12-27 09:25:26,386][20381] Updated weights for policy 1, policy_version 546792 (0.0005) [2023-12-27 09:25:26,400][20347] Updated weights for policy 0, policy_version 547025 (0.0010) [2023-12-27 09:25:26,457][20347] Updated weights for policy 0, policy_version 547035 (0.0010) [2023-12-27 09:25:27,016][20381] Updated weights for policy 1, policy_version 546802 (0.0006) [2023-12-27 09:25:27,064][20381] Updated weights for policy 1, policy_version 546812 (0.0007) [2023-12-27 09:25:27,116][20381] Updated weights for policy 1, policy_version 546822 (0.0010) [2023-12-27 09:25:27,143][20347] Updated weights for policy 0, policy_version 547045 (0.0008) [2023-12-27 09:25:27,168][20381] Updated weights for policy 1, policy_version 546832 (0.0007) [2023-12-27 09:25:27,197][20347] Updated weights for policy 0, policy_version 547055 (0.0008) [2023-12-27 09:25:27,262][20347] Updated weights for policy 0, policy_version 547065 (0.0007) [2023-12-27 09:25:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 280076288. Throughput: 0: 10418.1, 1: 10288.7. Samples: 280060596. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:27,363][19170] Avg episode reward: [(0, '556.550'), (1, '825.350')] [2023-12-27 09:25:27,795][20381] Updated weights for policy 1, policy_version 546842 (0.0009) [2023-12-27 09:25:27,849][20381] Updated weights for policy 1, policy_version 546852 (0.0010) [2023-12-27 09:25:27,904][20381] Updated weights for policy 1, policy_version 546862 (0.0010) [2023-12-27 09:25:27,912][20347] Updated weights for policy 0, policy_version 547075 (0.0006) [2023-12-27 09:25:27,966][20347] Updated weights for policy 0, policy_version 547085 (0.0006) [2023-12-27 09:25:28,021][20347] Updated weights for policy 0, policy_version 547095 (0.0008) [2023-12-27 09:25:28,579][20381] Updated weights for policy 1, policy_version 546872 (0.0010) [2023-12-27 09:25:28,637][20381] Updated weights for policy 1, policy_version 546882 (0.0010) [2023-12-27 09:25:28,641][20347] Updated weights for policy 0, policy_version 547105 (0.0006) [2023-12-27 09:25:28,690][20347] Updated weights for policy 0, policy_version 547115 (0.0010) [2023-12-27 09:25:28,691][20381] Updated weights for policy 1, policy_version 546892 (0.0010) [2023-12-27 09:25:28,736][20347] Updated weights for policy 0, policy_version 547125 (0.0010) [2023-12-27 09:25:28,784][20347] Updated weights for policy 0, policy_version 547135 (0.0010) [2023-12-27 09:25:29,382][20381] Updated weights for policy 1, policy_version 546902 (0.0010) [2023-12-27 09:25:29,441][20381] Updated weights for policy 1, policy_version 546912 (0.0011) [2023-12-27 09:25:29,499][20381] Updated weights for policy 1, policy_version 546922 (0.0006) [2023-12-27 09:25:29,521][20347] Updated weights for policy 0, policy_version 547145 (0.0010) [2023-12-27 09:25:29,578][20347] Updated weights for policy 0, policy_version 547155 (0.0010) [2023-12-27 09:25:29,627][20347] Updated weights for policy 0, policy_version 547165 (0.0010) [2023-12-27 09:25:30,116][20381] Updated weights for policy 1, policy_version 546932 (0.0008) [2023-12-27 09:25:30,167][20381] Updated weights for policy 1, policy_version 546942 (0.0010) [2023-12-27 09:25:30,221][20381] Updated weights for policy 1, policy_version 546952 (0.0008) [2023-12-27 09:25:30,354][20347] Updated weights for policy 0, policy_version 547175 (0.0010) [2023-12-27 09:25:30,420][20347] Updated weights for policy 0, policy_version 547185 (0.0010) [2023-12-27 09:25:30,491][20347] Updated weights for policy 0, policy_version 547195 (0.0010) [2023-12-27 09:25:30,857][20381] Updated weights for policy 1, policy_version 546962 (0.0007) [2023-12-27 09:25:30,909][20381] Updated weights for policy 1, policy_version 546972 (0.0006) [2023-12-27 09:25:30,961][20381] Updated weights for policy 1, policy_version 546982 (0.0009) [2023-12-27 09:25:31,017][20381] Updated weights for policy 1, policy_version 546992 (0.0010) [2023-12-27 09:25:31,147][20347] Updated weights for policy 0, policy_version 547205 (0.0010) [2023-12-27 09:25:31,217][20347] Updated weights for policy 0, policy_version 547215 (0.0007) [2023-12-27 09:25:31,281][20347] Updated weights for policy 0, policy_version 547225 (0.0007) [2023-12-27 09:25:31,657][20381] Updated weights for policy 1, policy_version 547002 (0.0008) [2023-12-27 09:25:31,720][20381] Updated weights for policy 1, policy_version 547012 (0.0008) [2023-12-27 09:25:31,775][20381] Updated weights for policy 1, policy_version 547022 (0.0007) [2023-12-27 09:25:31,927][20347] Updated weights for policy 0, policy_version 547235 (0.0006) [2023-12-27 09:25:31,979][20347] Updated weights for policy 0, policy_version 547245 (0.0007) [2023-12-27 09:25:32,027][20347] Updated weights for policy 0, policy_version 547255 (0.0007) [2023-12-27 09:25:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 280174592. Throughput: 0: 10401.2, 1: 10321.4. Samples: 280186276. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:32,363][19170] Avg episode reward: [(0, '252.330'), (1, '822.710')] [2023-12-27 09:25:32,410][20381] Updated weights for policy 1, policy_version 547032 (0.0006) [2023-12-27 09:25:32,466][20381] Updated weights for policy 1, policy_version 547042 (0.0007) [2023-12-27 09:25:32,522][20381] Updated weights for policy 1, policy_version 547052 (0.0009) [2023-12-27 09:25:32,743][20347] Updated weights for policy 0, policy_version 547265 (0.0008) [2023-12-27 09:25:32,804][20347] Updated weights for policy 0, policy_version 547275 (0.0010) [2023-12-27 09:25:32,855][20347] Updated weights for policy 0, policy_version 547285 (0.0010) [2023-12-27 09:25:32,911][20347] Updated weights for policy 0, policy_version 547295 (0.0010) [2023-12-27 09:25:33,120][20381] Updated weights for policy 1, policy_version 547062 (0.0008) [2023-12-27 09:25:33,170][20381] Updated weights for policy 1, policy_version 547072 (0.0008) [2023-12-27 09:25:33,218][20381] Updated weights for policy 1, policy_version 547082 (0.0008) [2023-12-27 09:25:33,514][20347] Updated weights for policy 0, policy_version 547305 (0.0008) [2023-12-27 09:25:33,558][20347] Updated weights for policy 0, policy_version 547315 (0.0008) [2023-12-27 09:25:33,612][20347] Updated weights for policy 0, policy_version 547325 (0.0008) [2023-12-27 09:25:33,830][20381] Updated weights for policy 1, policy_version 547092 (0.0008) [2023-12-27 09:25:33,890][20381] Updated weights for policy 1, policy_version 547102 (0.0006) [2023-12-27 09:25:33,945][20381] Updated weights for policy 1, policy_version 547112 (0.0008) [2023-12-27 09:25:34,357][20347] Updated weights for policy 0, policy_version 547335 (0.0006) [2023-12-27 09:25:34,419][20347] Updated weights for policy 0, policy_version 547345 (0.0006) [2023-12-27 09:25:34,477][20347] Updated weights for policy 0, policy_version 547355 (0.0006) [2023-12-27 09:25:34,674][20381] Updated weights for policy 1, policy_version 547122 (0.0007) [2023-12-27 09:25:34,732][20381] Updated weights for policy 1, policy_version 547132 (0.0007) [2023-12-27 09:25:34,794][20381] Updated weights for policy 1, policy_version 547142 (0.0007) [2023-12-27 09:25:34,853][20381] Updated weights for policy 1, policy_version 547152 (0.0008) [2023-12-27 09:25:35,132][20347] Updated weights for policy 0, policy_version 547365 (0.0007) [2023-12-27 09:25:35,188][20347] Updated weights for policy 0, policy_version 547375 (0.0007) [2023-12-27 09:25:35,234][20347] Updated weights for policy 0, policy_version 547385 (0.0007) [2023-12-27 09:25:35,470][20381] Updated weights for policy 1, policy_version 547162 (0.0005) [2023-12-27 09:25:35,529][20381] Updated weights for policy 1, policy_version 547172 (0.0007) [2023-12-27 09:25:35,587][20381] Updated weights for policy 1, policy_version 547182 (0.0011) [2023-12-27 09:25:35,947][20347] Updated weights for policy 0, policy_version 547395 (0.0008) [2023-12-27 09:25:36,000][20347] Updated weights for policy 0, policy_version 547405 (0.0007) [2023-12-27 09:25:36,059][20347] Updated weights for policy 0, policy_version 547415 (0.0007) [2023-12-27 09:25:36,198][20381] Updated weights for policy 1, policy_version 547192 (0.0010) [2023-12-27 09:25:36,208][20292] KL-divergence is very high: 1601.4192 [2023-12-27 09:25:36,243][20292] KL-divergence is very high: 164.0307 [2023-12-27 09:25:36,252][20381] Updated weights for policy 1, policy_version 547202 (0.0008) [2023-12-27 09:25:36,253][20292] KL-divergence is very high: 428.9174 [2023-12-27 09:25:36,296][20292] KL-divergence is very high: 226.5925 [2023-12-27 09:25:36,309][20381] Updated weights for policy 1, policy_version 547212 (0.0008) [2023-12-27 09:25:36,708][20347] Updated weights for policy 0, policy_version 547425 (0.0008) [2023-12-27 09:25:36,769][20347] Updated weights for policy 0, policy_version 547435 (0.0008) [2023-12-27 09:25:36,837][20347] Updated weights for policy 0, policy_version 547445 (0.0008) [2023-12-27 09:25:36,897][20347] Updated weights for policy 0, policy_version 547455 (0.0008) [2023-12-27 09:25:36,979][20381] Updated weights for policy 1, policy_version 547222 (0.0008) [2023-12-27 09:25:37,039][20381] Updated weights for policy 1, policy_version 547232 (0.0009) [2023-12-27 09:25:37,098][20381] Updated weights for policy 1, policy_version 547242 (0.0007) [2023-12-27 09:25:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 280281088. Throughput: 0: 10388.3, 1: 10368.0. Samples: 280250096. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:37,363][19170] Avg episode reward: [(0, '154.370'), (1, '793.830')] [2023-12-27 09:25:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000547248_140115968.pth... [2023-12-27 09:25:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000547456_140165120.pth... [2023-12-27 09:25:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000546032_139804672.pth [2023-12-27 09:25:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000546272_139862016.pth [2023-12-27 09:25:37,556][20347] Updated weights for policy 0, policy_version 547465 (0.0008) [2023-12-27 09:25:37,604][20347] Updated weights for policy 0, policy_version 547475 (0.0007) [2023-12-27 09:25:37,660][20347] Updated weights for policy 0, policy_version 547485 (0.0006) [2023-12-27 09:25:37,781][20381] Updated weights for policy 1, policy_version 547252 (0.0007) [2023-12-27 09:25:37,826][20381] Updated weights for policy 1, policy_version 547262 (0.0007) [2023-12-27 09:25:37,876][20381] Updated weights for policy 1, policy_version 547272 (0.0008) [2023-12-27 09:25:38,295][20347] Updated weights for policy 0, policy_version 547495 (0.0006) [2023-12-27 09:25:38,357][20347] Updated weights for policy 0, policy_version 547505 (0.0007) [2023-12-27 09:25:38,416][20347] Updated weights for policy 0, policy_version 547515 (0.0006) [2023-12-27 09:25:38,537][20381] Updated weights for policy 1, policy_version 547282 (0.0007) [2023-12-27 09:25:38,599][20381] Updated weights for policy 1, policy_version 547292 (0.0006) [2023-12-27 09:25:38,666][20381] Updated weights for policy 1, policy_version 547302 (0.0007) [2023-12-27 09:25:38,727][20381] Updated weights for policy 1, policy_version 547312 (0.0008) [2023-12-27 09:25:39,005][20347] Updated weights for policy 0, policy_version 547525 (0.0005) [2023-12-27 09:25:39,049][20347] Updated weights for policy 0, policy_version 547535 (0.0005) [2023-12-27 09:25:39,095][20347] Updated weights for policy 0, policy_version 547545 (0.0007) [2023-12-27 09:25:39,407][20381] Updated weights for policy 1, policy_version 547322 (0.0008) [2023-12-27 09:25:39,464][20381] Updated weights for policy 1, policy_version 547332 (0.0008) [2023-12-27 09:25:39,511][20381] Updated weights for policy 1, policy_version 547342 (0.0008) [2023-12-27 09:25:39,822][20347] Updated weights for policy 0, policy_version 547555 (0.0008) [2023-12-27 09:25:39,882][20347] Updated weights for policy 0, policy_version 547565 (0.0007) [2023-12-27 09:25:39,940][20347] Updated weights for policy 0, policy_version 547575 (0.0008) [2023-12-27 09:25:40,281][20381] Updated weights for policy 1, policy_version 547352 (0.0008) [2023-12-27 09:25:40,329][20292] KL-divergence is very high: 148.9640 [2023-12-27 09:25:40,342][20381] Updated weights for policy 1, policy_version 547362 (0.0008) [2023-12-27 09:25:40,378][20292] KL-divergence is very high: 144.7257 [2023-12-27 09:25:40,402][20381] Updated weights for policy 1, policy_version 547372 (0.0008) [2023-12-27 09:25:40,578][20347] Updated weights for policy 0, policy_version 547585 (0.0007) [2023-12-27 09:25:40,639][20347] Updated weights for policy 0, policy_version 547595 (0.0008) [2023-12-27 09:25:40,701][20347] Updated weights for policy 0, policy_version 547605 (0.0007) [2023-12-27 09:25:40,759][20347] Updated weights for policy 0, policy_version 547615 (0.0006) [2023-12-27 09:25:41,109][20381] Updated weights for policy 1, policy_version 547382 (0.0009) [2023-12-27 09:25:41,174][20381] Updated weights for policy 1, policy_version 547392 (0.0011) [2023-12-27 09:25:41,233][20381] Updated weights for policy 1, policy_version 547402 (0.0010) [2023-12-27 09:25:41,421][20347] Updated weights for policy 0, policy_version 547625 (0.0008) [2023-12-27 09:25:41,489][20347] Updated weights for policy 0, policy_version 547635 (0.0007) [2023-12-27 09:25:41,552][20347] Updated weights for policy 0, policy_version 547645 (0.0005) [2023-12-27 09:25:41,938][20381] Updated weights for policy 1, policy_version 547412 (0.0009) [2023-12-27 09:25:41,988][20381] Updated weights for policy 1, policy_version 547422 (0.0006) [2023-12-27 09:25:42,049][20381] Updated weights for policy 1, policy_version 547432 (0.0008) [2023-12-27 09:25:42,182][20347] Updated weights for policy 0, policy_version 547655 (0.0006) [2023-12-27 09:25:42,238][20347] Updated weights for policy 0, policy_version 547665 (0.0008) [2023-12-27 09:25:42,296][20347] Updated weights for policy 0, policy_version 547675 (0.0008) [2023-12-27 09:25:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 280387584. Throughput: 0: 10343.7, 1: 10368.7. Samples: 280373744. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:42,363][19170] Avg episode reward: [(0, '167.660'), (1, '796.010')] [2023-12-27 09:25:42,755][20381] Updated weights for policy 1, policy_version 547442 (0.0009) [2023-12-27 09:25:42,819][20381] Updated weights for policy 1, policy_version 547452 (0.0008) [2023-12-27 09:25:42,873][20381] Updated weights for policy 1, policy_version 547462 (0.0009) [2023-12-27 09:25:42,937][20381] Updated weights for policy 1, policy_version 547472 (0.0006) [2023-12-27 09:25:42,956][20347] Updated weights for policy 0, policy_version 547685 (0.0008) [2023-12-27 09:25:43,016][20347] Updated weights for policy 0, policy_version 547695 (0.0010) [2023-12-27 09:25:43,077][20347] Updated weights for policy 0, policy_version 547705 (0.0007) [2023-12-27 09:25:43,587][20381] Updated weights for policy 1, policy_version 547482 (0.0011) [2023-12-27 09:25:43,635][20381] Updated weights for policy 1, policy_version 547492 (0.0010) [2023-12-27 09:25:43,682][20347] Updated weights for policy 0, policy_version 547715 (0.0007) [2023-12-27 09:25:43,690][20381] Updated weights for policy 1, policy_version 547502 (0.0011) [2023-12-27 09:25:43,742][20347] Updated weights for policy 0, policy_version 547725 (0.0008) [2023-12-27 09:25:43,813][20347] Updated weights for policy 0, policy_version 547736 (0.0006) [2023-12-27 09:25:44,357][20347] Updated weights for policy 0, policy_version 547746 (0.0006) [2023-12-27 09:25:44,368][20381] Updated weights for policy 1, policy_version 547512 (0.0011) [2023-12-27 09:25:44,415][20347] Updated weights for policy 0, policy_version 547756 (0.0006) [2023-12-27 09:25:44,429][20381] Updated weights for policy 1, policy_version 547522 (0.0010) [2023-12-27 09:25:44,473][20347] Updated weights for policy 0, policy_version 547766 (0.0008) [2023-12-27 09:25:44,484][20381] Updated weights for policy 1, policy_version 547532 (0.0006) [2023-12-27 09:25:44,530][20347] Updated weights for policy 0, policy_version 547776 (0.0008) [2023-12-27 09:25:45,084][20381] Updated weights for policy 1, policy_version 547542 (0.0009) [2023-12-27 09:25:45,143][20381] Updated weights for policy 1, policy_version 547552 (0.0011) [2023-12-27 09:25:45,199][20381] Updated weights for policy 1, policy_version 547562 (0.0011) [2023-12-27 09:25:45,213][20347] Updated weights for policy 0, policy_version 547786 (0.0011) [2023-12-27 09:25:45,259][20347] Updated weights for policy 0, policy_version 547796 (0.0010) [2023-12-27 09:25:45,311][20347] Updated weights for policy 0, policy_version 547806 (0.0006) [2023-12-27 09:25:45,868][20381] Updated weights for policy 1, policy_version 547572 (0.0011) [2023-12-27 09:25:45,916][20381] Updated weights for policy 1, policy_version 547582 (0.0006) [2023-12-27 09:25:45,958][20347] Updated weights for policy 0, policy_version 547816 (0.0009) [2023-12-27 09:25:45,962][20381] Updated weights for policy 1, policy_version 547592 (0.0005) [2023-12-27 09:25:46,013][20347] Updated weights for policy 0, policy_version 547826 (0.0010) [2023-12-27 09:25:46,061][20347] Updated weights for policy 0, policy_version 547836 (0.0011) [2023-12-27 09:25:46,584][20381] Updated weights for policy 1, policy_version 547602 (0.0005) [2023-12-27 09:25:46,640][20381] Updated weights for policy 1, policy_version 547612 (0.0005) [2023-12-27 09:25:46,693][20381] Updated weights for policy 1, policy_version 547622 (0.0006) [2023-12-27 09:25:46,714][20347] Updated weights for policy 0, policy_version 547846 (0.0009) [2023-12-27 09:25:46,738][20381] Updated weights for policy 1, policy_version 547632 (0.0005) [2023-12-27 09:25:46,764][20347] Updated weights for policy 0, policy_version 547856 (0.0007) [2023-12-27 09:25:46,813][20347] Updated weights for policy 0, policy_version 547866 (0.0007) [2023-12-27 09:25:47,276][20381] Updated weights for policy 1, policy_version 547642 (0.0006) [2023-12-27 09:25:47,329][20381] Updated weights for policy 1, policy_version 547652 (0.0005) [2023-12-27 09:25:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 280485888. Throughput: 0: 10381.0, 1: 10428.0. Samples: 280503228. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:47,364][19170] Avg episode reward: [(0, '259.120'), (1, '815.110')] [2023-12-27 09:25:47,389][20381] Updated weights for policy 1, policy_version 547662 (0.0007) [2023-12-27 09:25:47,457][20347] Updated weights for policy 0, policy_version 547876 (0.0007) [2023-12-27 09:25:47,513][20347] Updated weights for policy 0, policy_version 547886 (0.0007) [2023-12-27 09:25:47,566][20347] Updated weights for policy 0, policy_version 547896 (0.0006) [2023-12-27 09:25:48,019][20381] Updated weights for policy 1, policy_version 547672 (0.0008) [2023-12-27 09:25:48,076][20381] Updated weights for policy 1, policy_version 547682 (0.0005) [2023-12-27 09:25:48,135][20381] Updated weights for policy 1, policy_version 547692 (0.0009) [2023-12-27 09:25:48,265][20347] Updated weights for policy 0, policy_version 547906 (0.0009) [2023-12-27 09:25:48,329][20347] Updated weights for policy 0, policy_version 547916 (0.0010) [2023-12-27 09:25:48,389][20347] Updated weights for policy 0, policy_version 547926 (0.0010) [2023-12-27 09:25:48,444][20347] Updated weights for policy 0, policy_version 547936 (0.0010) [2023-12-27 09:25:48,800][20381] Updated weights for policy 1, policy_version 547702 (0.0011) [2023-12-27 09:25:48,859][20381] Updated weights for policy 1, policy_version 547712 (0.0011) [2023-12-27 09:25:48,914][20381] Updated weights for policy 1, policy_version 547722 (0.0010) [2023-12-27 09:25:49,106][20347] Updated weights for policy 0, policy_version 547946 (0.0010) [2023-12-27 09:25:49,160][20347] Updated weights for policy 0, policy_version 547956 (0.0010) [2023-12-27 09:25:49,231][20347] Updated weights for policy 0, policy_version 547966 (0.0007) [2023-12-27 09:25:49,631][20381] Updated weights for policy 1, policy_version 547732 (0.0009) [2023-12-27 09:25:49,690][20381] Updated weights for policy 1, policy_version 547742 (0.0008) [2023-12-27 09:25:49,747][20381] Updated weights for policy 1, policy_version 547752 (0.0008) [2023-12-27 09:25:49,940][20347] Updated weights for policy 0, policy_version 547976 (0.0007) [2023-12-27 09:25:49,987][20347] Updated weights for policy 0, policy_version 547986 (0.0008) [2023-12-27 09:25:50,042][20347] Updated weights for policy 0, policy_version 547996 (0.0008) [2023-12-27 09:25:50,432][20381] Updated weights for policy 1, policy_version 547762 (0.0006) [2023-12-27 09:25:50,491][20381] Updated weights for policy 1, policy_version 547772 (0.0006) [2023-12-27 09:25:50,546][20381] Updated weights for policy 1, policy_version 547782 (0.0009) [2023-12-27 09:25:50,607][20381] Updated weights for policy 1, policy_version 547792 (0.0009) [2023-12-27 09:25:50,724][20347] Updated weights for policy 0, policy_version 548006 (0.0010) [2023-12-27 09:25:50,786][20347] Updated weights for policy 0, policy_version 548016 (0.0008) [2023-12-27 09:25:50,844][20347] Updated weights for policy 0, policy_version 548026 (0.0007) [2023-12-27 09:25:51,218][20381] Updated weights for policy 1, policy_version 547802 (0.0007) [2023-12-27 09:25:51,281][20381] Updated weights for policy 1, policy_version 547812 (0.0007) [2023-12-27 09:25:51,345][20381] Updated weights for policy 1, policy_version 547822 (0.0007) [2023-12-27 09:25:51,471][20347] Updated weights for policy 0, policy_version 548036 (0.0007) [2023-12-27 09:25:51,522][20347] Updated weights for policy 0, policy_version 548046 (0.0008) [2023-12-27 09:25:51,580][20347] Updated weights for policy 0, policy_version 548056 (0.0010) [2023-12-27 09:25:52,012][20381] Updated weights for policy 1, policy_version 547832 (0.0007) [2023-12-27 09:25:52,074][20381] Updated weights for policy 1, policy_version 547842 (0.0008) [2023-12-27 09:25:52,135][20381] Updated weights for policy 1, policy_version 547852 (0.0006) [2023-12-27 09:25:52,243][20347] Updated weights for policy 0, policy_version 548066 (0.0010) [2023-12-27 09:25:52,307][20347] Updated weights for policy 0, policy_version 548076 (0.0008) [2023-12-27 09:25:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 280592384. Throughput: 0: 10396.2, 1: 10401.8. Samples: 280563896. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:52,363][19170] Avg episode reward: [(0, '313.940'), (1, '795.710')] [2023-12-27 09:25:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000547856_140271616.pth... [2023-12-27 09:25:52,372][20347] Updated weights for policy 0, policy_version 548086 (0.0008) [2023-12-27 09:25:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000546608_139952128.pth [2023-12-27 09:25:52,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000548096_140328960.pth... [2023-12-27 09:25:52,437][20347] Updated weights for policy 0, policy_version 548096 (0.0008) [2023-12-27 09:25:52,439][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000546848_140009472.pth [2023-12-27 09:25:52,777][20381] Updated weights for policy 1, policy_version 547862 (0.0005) [2023-12-27 09:25:52,824][20381] Updated weights for policy 1, policy_version 547872 (0.0007) [2023-12-27 09:25:52,886][20381] Updated weights for policy 1, policy_version 547882 (0.0008) [2023-12-27 09:25:53,107][20347] Updated weights for policy 0, policy_version 548106 (0.0010) [2023-12-27 09:25:53,171][20347] Updated weights for policy 0, policy_version 548116 (0.0007) [2023-12-27 09:25:53,241][20347] Updated weights for policy 0, policy_version 548126 (0.0008) [2023-12-27 09:25:53,545][20381] Updated weights for policy 1, policy_version 547892 (0.0006) [2023-12-27 09:25:53,607][20381] Updated weights for policy 1, policy_version 547902 (0.0006) [2023-12-27 09:25:53,666][20381] Updated weights for policy 1, policy_version 547912 (0.0005) [2023-12-27 09:25:53,890][20347] Updated weights for policy 0, policy_version 548136 (0.0009) [2023-12-27 09:25:53,950][20347] Updated weights for policy 0, policy_version 548146 (0.0007) [2023-12-27 09:25:54,002][20347] Updated weights for policy 0, policy_version 548156 (0.0009) [2023-12-27 09:25:54,237][20381] Updated weights for policy 1, policy_version 547922 (0.0007) [2023-12-27 09:25:54,294][20381] Updated weights for policy 1, policy_version 547932 (0.0006) [2023-12-27 09:25:54,353][20381] Updated weights for policy 1, policy_version 547942 (0.0005) [2023-12-27 09:25:54,409][20381] Updated weights for policy 1, policy_version 547952 (0.0005) [2023-12-27 09:25:54,746][20347] Updated weights for policy 0, policy_version 548166 (0.0007) [2023-12-27 09:25:54,797][20347] Updated weights for policy 0, policy_version 548176 (0.0008) [2023-12-27 09:25:54,853][20347] Updated weights for policy 0, policy_version 548186 (0.0009) [2023-12-27 09:25:54,955][20381] Updated weights for policy 1, policy_version 547962 (0.0006) [2023-12-27 09:25:55,009][20381] Updated weights for policy 1, policy_version 547972 (0.0010) [2023-12-27 09:25:55,058][20381] Updated weights for policy 1, policy_version 547982 (0.0010) [2023-12-27 09:25:55,613][20347] Updated weights for policy 0, policy_version 548196 (0.0007) [2023-12-27 09:25:55,623][20381] Updated weights for policy 1, policy_version 547992 (0.0010) [2023-12-27 09:25:55,667][20347] Updated weights for policy 0, policy_version 548206 (0.0007) [2023-12-27 09:25:55,671][20381] Updated weights for policy 1, policy_version 548002 (0.0010) [2023-12-27 09:25:55,718][20347] Updated weights for policy 0, policy_version 548216 (0.0007) [2023-12-27 09:25:55,723][20381] Updated weights for policy 1, policy_version 548012 (0.0010) [2023-12-27 09:25:56,355][20347] Updated weights for policy 0, policy_version 548226 (0.0006) [2023-12-27 09:25:56,356][20381] Updated weights for policy 1, policy_version 548022 (0.0008) [2023-12-27 09:25:56,406][20381] Updated weights for policy 1, policy_version 548032 (0.0005) [2023-12-27 09:25:56,411][20347] Updated weights for policy 0, policy_version 548236 (0.0008) [2023-12-27 09:25:56,456][20347] Updated weights for policy 0, policy_version 548246 (0.0007) [2023-12-27 09:25:56,462][20381] Updated weights for policy 1, policy_version 548042 (0.0005) [2023-12-27 09:25:56,502][20347] Updated weights for policy 0, policy_version 548256 (0.0007) [2023-12-27 09:25:57,104][20381] Updated weights for policy 1, policy_version 548052 (0.0011) [2023-12-27 09:25:57,117][20347] Updated weights for policy 0, policy_version 548266 (0.0005) [2023-12-27 09:25:57,162][20381] Updated weights for policy 1, policy_version 548062 (0.0011) [2023-12-27 09:25:57,172][20347] Updated weights for policy 0, policy_version 548276 (0.0005) [2023-12-27 09:25:57,214][20381] Updated weights for policy 1, policy_version 548072 (0.0010) [2023-12-27 09:25:57,227][20347] Updated weights for policy 0, policy_version 548286 (0.0006) [2023-12-27 09:25:57,362][19170] Fps is (10 sec: 22118.6, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 280707072. Throughput: 0: 10340.5, 1: 10534.2. Samples: 280690500. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:25:57,363][19170] Avg episode reward: [(0, '383.370'), (1, '708.350')] [2023-12-27 09:25:57,813][20347] Updated weights for policy 0, policy_version 548296 (0.0005) [2023-12-27 09:25:57,863][20347] Updated weights for policy 0, policy_version 548306 (0.0006) [2023-12-27 09:25:57,866][20381] Updated weights for policy 1, policy_version 548082 (0.0011) [2023-12-27 09:25:57,912][20347] Updated weights for policy 0, policy_version 548316 (0.0010) [2023-12-27 09:25:57,919][20381] Updated weights for policy 1, policy_version 548092 (0.0010) [2023-12-27 09:25:57,968][20381] Updated weights for policy 1, policy_version 548102 (0.0010) [2023-12-27 09:25:58,017][20381] Updated weights for policy 1, policy_version 548112 (0.0010) [2023-12-27 09:25:58,602][20347] Updated weights for policy 0, policy_version 548326 (0.0008) [2023-12-27 09:25:58,659][20347] Updated weights for policy 0, policy_version 548336 (0.0007) [2023-12-27 09:25:58,722][20347] Updated weights for policy 0, policy_version 548346 (0.0008) [2023-12-27 09:25:58,849][20381] Updated weights for policy 1, policy_version 548122 (0.0008) [2023-12-27 09:25:58,916][20381] Updated weights for policy 1, policy_version 548132 (0.0006) [2023-12-27 09:25:58,970][20381] Updated weights for policy 1, policy_version 548142 (0.0008) [2023-12-27 09:25:59,476][20347] Updated weights for policy 0, policy_version 548356 (0.0007) [2023-12-27 09:25:59,534][20347] Updated weights for policy 0, policy_version 548366 (0.0006) [2023-12-27 09:25:59,594][20347] Updated weights for policy 0, policy_version 548376 (0.0008) [2023-12-27 09:25:59,657][20381] Updated weights for policy 1, policy_version 548152 (0.0010) [2023-12-27 09:25:59,705][20381] Updated weights for policy 1, policy_version 548162 (0.0010) [2023-12-27 09:25:59,760][20381] Updated weights for policy 1, policy_version 548172 (0.0010) [2023-12-27 09:26:00,260][20347] Updated weights for policy 0, policy_version 548386 (0.0006) [2023-12-27 09:26:00,320][20347] Updated weights for policy 0, policy_version 548396 (0.0006) [2023-12-27 09:26:00,379][20347] Updated weights for policy 0, policy_version 548406 (0.0007) [2023-12-27 09:26:00,431][20347] Updated weights for policy 0, policy_version 548416 (0.0008) [2023-12-27 09:26:00,460][20381] Updated weights for policy 1, policy_version 548182 (0.0008) [2023-12-27 09:26:00,519][20381] Updated weights for policy 1, policy_version 548192 (0.0007) [2023-12-27 09:26:00,570][20381] Updated weights for policy 1, policy_version 548202 (0.0009) [2023-12-27 09:26:01,050][20347] Updated weights for policy 0, policy_version 548426 (0.0008) [2023-12-27 09:26:01,106][20347] Updated weights for policy 0, policy_version 548436 (0.0007) [2023-12-27 09:26:01,164][20347] Updated weights for policy 0, policy_version 548446 (0.0007) [2023-12-27 09:26:01,227][20381] Updated weights for policy 1, policy_version 548212 (0.0007) [2023-12-27 09:26:01,287][20381] Updated weights for policy 1, policy_version 548222 (0.0007) [2023-12-27 09:26:01,342][20381] Updated weights for policy 1, policy_version 548232 (0.0007) [2023-12-27 09:26:01,878][20347] Updated weights for policy 0, policy_version 548456 (0.0006) [2023-12-27 09:26:01,931][20347] Updated weights for policy 0, policy_version 548466 (0.0006) [2023-12-27 09:26:01,986][20347] Updated weights for policy 0, policy_version 548476 (0.0010) [2023-12-27 09:26:01,998][20381] Updated weights for policy 1, policy_version 548242 (0.0008) [2023-12-27 09:26:02,054][20381] Updated weights for policy 1, policy_version 548252 (0.0008) [2023-12-27 09:26:02,107][20381] Updated weights for policy 1, policy_version 548262 (0.0008) [2023-12-27 09:26:02,157][20381] Updated weights for policy 1, policy_version 548272 (0.0007) [2023-12-27 09:26:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 280805376. Throughput: 0: 10348.5, 1: 10513.3. Samples: 280814108. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:26:02,363][19170] Avg episode reward: [(0, '362.360'), (1, '765.920')] [2023-12-27 09:26:02,673][20347] Updated weights for policy 0, policy_version 548486 (0.0009) [2023-12-27 09:26:02,733][20347] Updated weights for policy 0, policy_version 548496 (0.0007) [2023-12-27 09:26:02,767][20381] Updated weights for policy 1, policy_version 548282 (0.0010) [2023-12-27 09:26:02,791][20347] Updated weights for policy 0, policy_version 548506 (0.0006) [2023-12-27 09:26:02,822][20381] Updated weights for policy 1, policy_version 548292 (0.0008) [2023-12-27 09:26:02,876][20381] Updated weights for policy 1, policy_version 548302 (0.0005) [2023-12-27 09:26:03,437][20347] Updated weights for policy 0, policy_version 548516 (0.0007) [2023-12-27 09:26:03,447][20381] Updated weights for policy 1, policy_version 548312 (0.0010) [2023-12-27 09:26:03,487][20347] Updated weights for policy 0, policy_version 548526 (0.0006) [2023-12-27 09:26:03,496][20381] Updated weights for policy 1, policy_version 548322 (0.0010) [2023-12-27 09:26:03,544][20347] Updated weights for policy 0, policy_version 548536 (0.0010) [2023-12-27 09:26:03,553][20381] Updated weights for policy 1, policy_version 548332 (0.0011) [2023-12-27 09:26:04,176][20381] Updated weights for policy 1, policy_version 548342 (0.0011) [2023-12-27 09:26:04,236][20381] Updated weights for policy 1, policy_version 548352 (0.0011) [2023-12-27 09:26:04,272][20347] Updated weights for policy 0, policy_version 548546 (0.0011) [2023-12-27 09:26:04,292][20381] Updated weights for policy 1, policy_version 548362 (0.0011) [2023-12-27 09:26:04,332][20347] Updated weights for policy 0, policy_version 548556 (0.0010) [2023-12-27 09:26:04,392][20347] Updated weights for policy 0, policy_version 548566 (0.0010) [2023-12-27 09:26:04,449][20347] Updated weights for policy 0, policy_version 548576 (0.0012) [2023-12-27 09:26:04,987][20381] Updated weights for policy 1, policy_version 548372 (0.0011) [2023-12-27 09:26:05,042][20381] Updated weights for policy 1, policy_version 548382 (0.0010) [2023-12-27 09:26:05,094][20381] Updated weights for policy 1, policy_version 548392 (0.0010) [2023-12-27 09:26:05,114][20347] Updated weights for policy 0, policy_version 548586 (0.0008) [2023-12-27 09:26:05,169][20347] Updated weights for policy 0, policy_version 548596 (0.0010) [2023-12-27 09:26:05,223][20347] Updated weights for policy 0, policy_version 548606 (0.0010) [2023-12-27 09:26:05,797][20381] Updated weights for policy 1, policy_version 548402 (0.0010) [2023-12-27 09:26:05,813][20347] Updated weights for policy 0, policy_version 548616 (0.0005) [2023-12-27 09:26:05,854][20381] Updated weights for policy 1, policy_version 548412 (0.0010) [2023-12-27 09:26:05,871][20347] Updated weights for policy 0, policy_version 548626 (0.0009) [2023-12-27 09:26:05,915][20381] Updated weights for policy 1, policy_version 548422 (0.0010) [2023-12-27 09:26:05,932][20347] Updated weights for policy 0, policy_version 548636 (0.0010) [2023-12-27 09:26:05,974][20381] Updated weights for policy 1, policy_version 548432 (0.0011) [2023-12-27 09:26:06,584][20347] Updated weights for policy 0, policy_version 548646 (0.0009) [2023-12-27 09:26:06,643][20347] Updated weights for policy 0, policy_version 548656 (0.0008) [2023-12-27 09:26:06,661][20381] Updated weights for policy 1, policy_version 548442 (0.0010) [2023-12-27 09:26:06,701][20347] Updated weights for policy 0, policy_version 548666 (0.0009) [2023-12-27 09:26:06,719][20381] Updated weights for policy 1, policy_version 548452 (0.0010) [2023-12-27 09:26:06,777][20381] Updated weights for policy 1, policy_version 548462 (0.0011) [2023-12-27 09:26:07,331][20347] Updated weights for policy 0, policy_version 548676 (0.0007) [2023-12-27 09:26:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 280903680. Throughput: 0: 10359.2, 1: 10587.1. Samples: 280877640. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:26:07,364][19170] Avg episode reward: [(0, '209.580'), (1, '707.520')] [2023-12-27 09:26:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000548464_140427264.pth... [2023-12-27 09:26:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000547248_140115968.pth [2023-12-27 09:26:07,391][20347] Updated weights for policy 0, policy_version 548686 (0.0006) [2023-12-27 09:26:07,445][20347] Updated weights for policy 0, policy_version 548696 (0.0005) [2023-12-27 09:26:07,452][20381] Updated weights for policy 1, policy_version 548472 (0.0008) [2023-12-27 09:26:07,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000548704_140484608.pth... [2023-12-27 09:26:07,494][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000547456_140165120.pth [2023-12-27 09:26:07,504][20381] Updated weights for policy 1, policy_version 548482 (0.0009) [2023-12-27 09:26:07,555][20381] Updated weights for policy 1, policy_version 548492 (0.0010) [2023-12-27 09:26:08,079][20347] Updated weights for policy 0, policy_version 548706 (0.0011) [2023-12-27 09:26:08,128][20347] Updated weights for policy 0, policy_version 548716 (0.0010) [2023-12-27 09:26:08,182][20381] Updated weights for policy 1, policy_version 548502 (0.0010) [2023-12-27 09:26:08,183][20347] Updated weights for policy 0, policy_version 548726 (0.0010) [2023-12-27 09:26:08,238][20347] Updated weights for policy 0, policy_version 548736 (0.0010) [2023-12-27 09:26:08,244][20381] Updated weights for policy 1, policy_version 548512 (0.0007) [2023-12-27 09:26:08,305][20381] Updated weights for policy 1, policy_version 548522 (0.0007) [2023-12-27 09:26:08,950][20347] Updated weights for policy 0, policy_version 548746 (0.0011) [2023-12-27 09:26:08,977][20381] Updated weights for policy 1, policy_version 548532 (0.0009) [2023-12-27 09:26:09,003][20347] Updated weights for policy 0, policy_version 548756 (0.0011) [2023-12-27 09:26:09,033][20381] Updated weights for policy 1, policy_version 548542 (0.0011) [2023-12-27 09:26:09,058][20347] Updated weights for policy 0, policy_version 548766 (0.0010) [2023-12-27 09:26:09,093][20381] Updated weights for policy 1, policy_version 548552 (0.0010) [2023-12-27 09:26:09,747][20347] Updated weights for policy 0, policy_version 548776 (0.0010) [2023-12-27 09:26:09,809][20347] Updated weights for policy 0, policy_version 548786 (0.0010) [2023-12-27 09:26:09,843][20381] Updated weights for policy 1, policy_version 548562 (0.0010) [2023-12-27 09:26:09,871][20347] Updated weights for policy 0, policy_version 548796 (0.0011) [2023-12-27 09:26:09,903][20381] Updated weights for policy 1, policy_version 548572 (0.0005) [2023-12-27 09:26:09,967][20381] Updated weights for policy 1, policy_version 548582 (0.0007) [2023-12-27 09:26:10,030][20381] Updated weights for policy 1, policy_version 548592 (0.0006) [2023-12-27 09:26:10,570][20347] Updated weights for policy 0, policy_version 548806 (0.0011) [2023-12-27 09:26:10,636][20347] Updated weights for policy 0, policy_version 548816 (0.0010) [2023-12-27 09:26:10,704][20347] Updated weights for policy 0, policy_version 548826 (0.0010) [2023-12-27 09:26:10,709][20381] Updated weights for policy 1, policy_version 548602 (0.0007) [2023-12-27 09:26:10,768][20381] Updated weights for policy 1, policy_version 548612 (0.0005) [2023-12-27 09:26:10,828][20381] Updated weights for policy 1, policy_version 548622 (0.0009) [2023-12-27 09:26:11,390][20347] Updated weights for policy 0, policy_version 548836 (0.0008) [2023-12-27 09:26:11,440][20347] Updated weights for policy 0, policy_version 548846 (0.0007) [2023-12-27 09:26:11,499][20347] Updated weights for policy 0, policy_version 548856 (0.0007) [2023-12-27 09:26:11,535][20381] Updated weights for policy 1, policy_version 548632 (0.0011) [2023-12-27 09:26:11,585][20381] Updated weights for policy 1, policy_version 548642 (0.0010) [2023-12-27 09:26:11,652][20381] Updated weights for policy 1, policy_version 548652 (0.0011) [2023-12-27 09:26:12,169][20347] Updated weights for policy 0, policy_version 548866 (0.0006) [2023-12-27 09:26:12,227][20347] Updated weights for policy 0, policy_version 548876 (0.0007) [2023-12-27 09:26:12,288][20347] Updated weights for policy 0, policy_version 548886 (0.0008) [2023-12-27 09:26:12,346][20381] Updated weights for policy 1, policy_version 548662 (0.0010) [2023-12-27 09:26:12,354][20347] Updated weights for policy 0, policy_version 548896 (0.0008) [2023-12-27 09:26:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 281010176. Throughput: 0: 10369.1, 1: 10513.9. Samples: 281000328. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 09:26:12,363][19170] Avg episode reward: [(0, '245.420'), (1, '756.540')] [2023-12-27 09:26:12,406][20381] Updated weights for policy 1, policy_version 548672 (0.0008) [2023-12-27 09:26:12,461][20381] Updated weights for policy 1, policy_version 548682 (0.0007) [2023-12-27 09:26:12,994][20347] Updated weights for policy 0, policy_version 548906 (0.0007) [2023-12-27 09:26:13,040][20347] Updated weights for policy 0, policy_version 548916 (0.0007) [2023-12-27 09:26:13,084][20347] Updated weights for policy 0, policy_version 548926 (0.0008) [2023-12-27 09:26:13,159][20381] Updated weights for policy 1, policy_version 548692 (0.0008) [2023-12-27 09:26:13,205][20381] Updated weights for policy 1, policy_version 548702 (0.0010) [2023-12-27 09:26:13,249][20381] Updated weights for policy 1, policy_version 548712 (0.0010) [2023-12-27 09:26:13,754][20347] Updated weights for policy 0, policy_version 548936 (0.0009) [2023-12-27 09:26:13,809][20347] Updated weights for policy 0, policy_version 548946 (0.0010) [2023-12-27 09:26:13,864][20347] Updated weights for policy 0, policy_version 548956 (0.0010) [2023-12-27 09:26:13,920][20381] Updated weights for policy 1, policy_version 548722 (0.0009) [2023-12-27 09:26:13,980][20381] Updated weights for policy 1, policy_version 548732 (0.0007) [2023-12-27 09:26:14,037][20381] Updated weights for policy 1, policy_version 548742 (0.0005) [2023-12-27 09:26:14,091][20381] Updated weights for policy 1, policy_version 548752 (0.0007) [2023-12-27 09:26:14,499][20347] Updated weights for policy 0, policy_version 548966 (0.0008) [2023-12-27 09:26:14,566][20347] Updated weights for policy 0, policy_version 548976 (0.0005) [2023-12-27 09:26:14,630][20347] Updated weights for policy 0, policy_version 548986 (0.0005) [2023-12-27 09:26:14,688][20381] Updated weights for policy 1, policy_version 548762 (0.0010) [2023-12-27 09:26:14,740][20381] Updated weights for policy 1, policy_version 548772 (0.0010) [2023-12-27 09:26:14,796][20381] Updated weights for policy 1, policy_version 548782 (0.0011) [2023-12-27 09:26:15,219][20347] Updated weights for policy 0, policy_version 548996 (0.0006) [2023-12-27 09:26:15,276][20347] Updated weights for policy 0, policy_version 549006 (0.0008) [2023-12-27 09:26:15,326][20347] Updated weights for policy 0, policy_version 549016 (0.0010) [2023-12-27 09:26:15,379][20381] Updated weights for policy 1, policy_version 548792 (0.0006) [2023-12-27 09:26:15,442][20381] Updated weights for policy 1, policy_version 548802 (0.0010) [2023-12-27 09:26:15,496][20381] Updated weights for policy 1, policy_version 548812 (0.0010) [2023-12-27 09:26:16,055][20347] Updated weights for policy 0, policy_version 549026 (0.0007) [2023-12-27 09:26:16,111][20381] Updated weights for policy 1, policy_version 548822 (0.0007) [2023-12-27 09:26:16,121][20347] Updated weights for policy 0, policy_version 549036 (0.0007) [2023-12-27 09:26:16,177][20381] Updated weights for policy 1, policy_version 548832 (0.0006) [2023-12-27 09:26:16,184][20347] Updated weights for policy 0, policy_version 549046 (0.0007) [2023-12-27 09:26:16,238][20381] Updated weights for policy 1, policy_version 548842 (0.0006) [2023-12-27 09:26:16,244][20347] Updated weights for policy 0, policy_version 549056 (0.0007) [2023-12-27 09:26:16,853][20381] Updated weights for policy 1, policy_version 548852 (0.0006) [2023-12-27 09:26:16,914][20381] Updated weights for policy 1, policy_version 548862 (0.0006) [2023-12-27 09:26:16,914][20347] Updated weights for policy 0, policy_version 549066 (0.0010) [2023-12-27 09:26:16,973][20347] Updated weights for policy 0, policy_version 549076 (0.0005) [2023-12-27 09:26:16,975][20381] Updated weights for policy 1, policy_version 548872 (0.0006) [2023-12-27 09:26:17,026][20347] Updated weights for policy 0, policy_version 549086 (0.0005) [2023-12-27 09:26:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 281116672. Throughput: 0: 10389.7, 1: 10519.1. Samples: 281127172. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:17,363][19170] Avg episode reward: [(0, '267.540'), (1, '766.390')] [2023-12-27 09:26:17,565][20347] Updated weights for policy 0, policy_version 549096 (0.0005) [2023-12-27 09:26:17,604][20381] Updated weights for policy 1, policy_version 548882 (0.0006) [2023-12-27 09:26:17,616][20347] Updated weights for policy 0, policy_version 549106 (0.0005) [2023-12-27 09:26:17,651][20381] Updated weights for policy 1, policy_version 548892 (0.0007) [2023-12-27 09:26:17,665][20347] Updated weights for policy 0, policy_version 549116 (0.0005) [2023-12-27 09:26:17,697][20381] Updated weights for policy 1, policy_version 548902 (0.0007) [2023-12-27 09:26:17,749][20381] Updated weights for policy 1, policy_version 548912 (0.0008) [2023-12-27 09:26:18,199][20347] Updated weights for policy 0, policy_version 549126 (0.0005) [2023-12-27 09:26:18,254][20347] Updated weights for policy 0, policy_version 549136 (0.0005) [2023-12-27 09:26:18,315][20347] Updated weights for policy 0, policy_version 549146 (0.0005) [2023-12-27 09:26:18,492][20381] Updated weights for policy 1, policy_version 548922 (0.0008) [2023-12-27 09:26:18,542][20381] Updated weights for policy 1, policy_version 548932 (0.0008) [2023-12-27 09:26:18,594][20381] Updated weights for policy 1, policy_version 548942 (0.0008) [2023-12-27 09:26:18,966][20347] Updated weights for policy 0, policy_version 549156 (0.0008) [2023-12-27 09:26:19,023][20347] Updated weights for policy 0, policy_version 549166 (0.0005) [2023-12-27 09:26:19,073][20347] Updated weights for policy 0, policy_version 549176 (0.0005) [2023-12-27 09:26:19,333][20381] Updated weights for policy 1, policy_version 548952 (0.0008) [2023-12-27 09:26:19,398][20381] Updated weights for policy 1, policy_version 548962 (0.0007) [2023-12-27 09:26:19,451][20381] Updated weights for policy 1, policy_version 548972 (0.0007) [2023-12-27 09:26:19,767][20347] Updated weights for policy 0, policy_version 549186 (0.0008) [2023-12-27 09:26:19,834][20347] Updated weights for policy 0, policy_version 549196 (0.0007) [2023-12-27 09:26:19,903][20347] Updated weights for policy 0, policy_version 549206 (0.0010) [2023-12-27 09:26:19,968][20347] Updated weights for policy 0, policy_version 549216 (0.0009) [2023-12-27 09:26:20,207][20381] Updated weights for policy 1, policy_version 548982 (0.0008) [2023-12-27 09:26:20,266][20381] Updated weights for policy 1, policy_version 548992 (0.0007) [2023-12-27 09:26:20,324][20381] Updated weights for policy 1, policy_version 549002 (0.0008) [2023-12-27 09:26:20,556][20347] Updated weights for policy 0, policy_version 549226 (0.0010) [2023-12-27 09:26:20,622][20347] Updated weights for policy 0, policy_version 549236 (0.0010) [2023-12-27 09:26:20,688][20347] Updated weights for policy 0, policy_version 549246 (0.0010) [2023-12-27 09:26:21,062][20381] Updated weights for policy 1, policy_version 549012 (0.0008) [2023-12-27 09:26:21,121][20381] Updated weights for policy 1, policy_version 549022 (0.0008) [2023-12-27 09:26:21,182][20381] Updated weights for policy 1, policy_version 549032 (0.0008) [2023-12-27 09:26:21,401][20347] Updated weights for policy 0, policy_version 549256 (0.0009) [2023-12-27 09:26:21,460][20347] Updated weights for policy 0, policy_version 549266 (0.0005) [2023-12-27 09:26:21,510][20347] Updated weights for policy 0, policy_version 549276 (0.0005) [2023-12-27 09:26:21,963][20381] Updated weights for policy 1, policy_version 549042 (0.0008) [2023-12-27 09:26:22,018][20381] Updated weights for policy 1, policy_version 549052 (0.0007) [2023-12-27 09:26:22,078][20381] Updated weights for policy 1, policy_version 549062 (0.0005) [2023-12-27 09:26:22,120][20347] Updated weights for policy 0, policy_version 549286 (0.0006) [2023-12-27 09:26:22,132][20381] Updated weights for policy 1, policy_version 549072 (0.0007) [2023-12-27 09:26:22,168][20347] Updated weights for policy 0, policy_version 549296 (0.0005) [2023-12-27 09:26:22,214][20347] Updated weights for policy 0, policy_version 549306 (0.0005) [2023-12-27 09:26:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 281223168. Throughput: 0: 10428.4, 1: 10430.2. Samples: 281188732. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:22,363][19170] Avg episode reward: [(0, '281.360'), (1, '746.880')] [2023-12-27 09:26:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000549072_140582912.pth... [2023-12-27 09:26:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000549312_140640256.pth... [2023-12-27 09:26:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000547856_140271616.pth [2023-12-27 09:26:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000548096_140328960.pth [2023-12-27 09:26:22,783][20381] Updated weights for policy 1, policy_version 549082 (0.0006) [2023-12-27 09:26:22,842][20381] Updated weights for policy 1, policy_version 549092 (0.0007) [2023-12-27 09:26:22,864][20347] Updated weights for policy 0, policy_version 549316 (0.0006) [2023-12-27 09:26:22,905][20381] Updated weights for policy 1, policy_version 549103 (0.0006) [2023-12-27 09:26:22,927][20347] Updated weights for policy 0, policy_version 549326 (0.0011) [2023-12-27 09:26:22,986][20347] Updated weights for policy 0, policy_version 549336 (0.0008) [2023-12-27 09:26:23,510][20381] Updated weights for policy 1, policy_version 549113 (0.0007) [2023-12-27 09:26:23,568][20381] Updated weights for policy 1, policy_version 549123 (0.0010) [2023-12-27 09:26:23,622][20381] Updated weights for policy 1, policy_version 549133 (0.0008) [2023-12-27 09:26:23,644][20347] Updated weights for policy 0, policy_version 549346 (0.0008) [2023-12-27 09:26:23,698][20347] Updated weights for policy 0, policy_version 549356 (0.0005) [2023-12-27 09:26:23,757][20347] Updated weights for policy 0, policy_version 549366 (0.0005) [2023-12-27 09:26:23,818][20347] Updated weights for policy 0, policy_version 549376 (0.0005) [2023-12-27 09:26:24,297][20381] Updated weights for policy 1, policy_version 549143 (0.0007) [2023-12-27 09:26:24,336][20347] Updated weights for policy 0, policy_version 549386 (0.0006) [2023-12-27 09:26:24,349][20381] Updated weights for policy 1, policy_version 549153 (0.0008) [2023-12-27 09:26:24,382][20347] Updated weights for policy 0, policy_version 549396 (0.0006) [2023-12-27 09:26:24,401][20381] Updated weights for policy 1, policy_version 549163 (0.0007) [2023-12-27 09:26:24,439][20347] Updated weights for policy 0, policy_version 549406 (0.0006) [2023-12-27 09:26:25,075][20347] Updated weights for policy 0, policy_version 549416 (0.0010) [2023-12-27 09:26:25,114][20381] Updated weights for policy 1, policy_version 549173 (0.0007) [2023-12-27 09:26:25,130][20347] Updated weights for policy 0, policy_version 549426 (0.0010) [2023-12-27 09:26:25,161][20381] Updated weights for policy 1, policy_version 549183 (0.0006) [2023-12-27 09:26:25,185][20347] Updated weights for policy 0, policy_version 549436 (0.0010) [2023-12-27 09:26:25,215][20381] Updated weights for policy 1, policy_version 549193 (0.0005) [2023-12-27 09:26:25,881][20347] Updated weights for policy 0, policy_version 549446 (0.0010) [2023-12-27 09:26:25,905][20381] Updated weights for policy 1, policy_version 549203 (0.0007) [2023-12-27 09:26:25,939][20347] Updated weights for policy 0, policy_version 549456 (0.0010) [2023-12-27 09:26:25,959][20381] Updated weights for policy 1, policy_version 549213 (0.0005) [2023-12-27 09:26:25,988][20347] Updated weights for policy 0, policy_version 549466 (0.0010) [2023-12-27 09:26:26,018][20381] Updated weights for policy 1, policy_version 549223 (0.0008) [2023-12-27 09:26:26,613][20347] Updated weights for policy 0, policy_version 549476 (0.0007) [2023-12-27 09:26:26,662][20347] Updated weights for policy 0, policy_version 549486 (0.0006) [2023-12-27 09:26:26,689][20381] Updated weights for policy 1, policy_version 549233 (0.0007) [2023-12-27 09:26:26,717][20347] Updated weights for policy 0, policy_version 549496 (0.0005) [2023-12-27 09:26:26,745][20381] Updated weights for policy 1, policy_version 549243 (0.0007) [2023-12-27 09:26:26,793][20381] Updated weights for policy 1, policy_version 549253 (0.0008) [2023-12-27 09:26:26,838][20381] Updated weights for policy 1, policy_version 549263 (0.0006) [2023-12-27 09:26:27,328][20347] Updated weights for policy 0, policy_version 549506 (0.0006) [2023-12-27 09:26:27,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 281321472. Throughput: 0: 10477.7, 1: 10412.8. Samples: 281313824. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:27,364][19170] Avg episode reward: [(0, '364.250'), (1, '775.850')] [2023-12-27 09:26:27,386][20347] Updated weights for policy 0, policy_version 549516 (0.0007) [2023-12-27 09:26:27,440][20347] Updated weights for policy 0, policy_version 549526 (0.0007) [2023-12-27 09:26:27,492][20347] Updated weights for policy 0, policy_version 549536 (0.0006) [2023-12-27 09:26:27,547][20381] Updated weights for policy 1, policy_version 549273 (0.0009) [2023-12-27 09:26:27,599][20381] Updated weights for policy 1, policy_version 549283 (0.0009) [2023-12-27 09:26:27,652][20381] Updated weights for policy 1, policy_version 549293 (0.0005) [2023-12-27 09:26:28,172][20347] Updated weights for policy 0, policy_version 549546 (0.0007) [2023-12-27 09:26:28,220][20347] Updated weights for policy 0, policy_version 549556 (0.0005) [2023-12-27 09:26:28,254][20381] Updated weights for policy 1, policy_version 549303 (0.0009) [2023-12-27 09:26:28,273][20347] Updated weights for policy 0, policy_version 549566 (0.0006) [2023-12-27 09:26:28,312][20381] Updated weights for policy 1, policy_version 549313 (0.0010) [2023-12-27 09:26:28,366][20381] Updated weights for policy 1, policy_version 549323 (0.0011) [2023-12-27 09:26:28,975][20347] Updated weights for policy 0, policy_version 549576 (0.0007) [2023-12-27 09:26:29,030][20347] Updated weights for policy 0, policy_version 549586 (0.0007) [2023-12-27 09:26:29,031][20381] Updated weights for policy 1, policy_version 549333 (0.0007) [2023-12-27 09:26:29,085][20347] Updated weights for policy 0, policy_version 549596 (0.0008) [2023-12-27 09:26:29,087][20381] Updated weights for policy 1, policy_version 549343 (0.0006) [2023-12-27 09:26:29,144][20381] Updated weights for policy 1, policy_version 549353 (0.0006) [2023-12-27 09:26:29,774][20347] Updated weights for policy 0, policy_version 549606 (0.0006) [2023-12-27 09:26:29,812][20381] Updated weights for policy 1, policy_version 549363 (0.0006) [2023-12-27 09:26:29,835][20347] Updated weights for policy 0, policy_version 549616 (0.0009) [2023-12-27 09:26:29,872][20381] Updated weights for policy 1, policy_version 549373 (0.0008) [2023-12-27 09:26:29,883][20347] Updated weights for policy 0, policy_version 549626 (0.0006) [2023-12-27 09:26:29,940][20381] Updated weights for policy 1, policy_version 549383 (0.0008) [2023-12-27 09:26:30,558][20381] Updated weights for policy 1, policy_version 549393 (0.0011) [2023-12-27 09:26:30,581][20347] Updated weights for policy 0, policy_version 549636 (0.0007) [2023-12-27 09:26:30,610][20381] Updated weights for policy 1, policy_version 549403 (0.0010) [2023-12-27 09:26:30,629][20347] Updated weights for policy 0, policy_version 549646 (0.0009) [2023-12-27 09:26:30,665][20381] Updated weights for policy 1, policy_version 549413 (0.0011) [2023-12-27 09:26:30,674][20347] Updated weights for policy 0, policy_version 549656 (0.0010) [2023-12-27 09:26:30,720][20381] Updated weights for policy 1, policy_version 549423 (0.0009) [2023-12-27 09:26:31,376][20381] Updated weights for policy 1, policy_version 549433 (0.0010) [2023-12-27 09:26:31,382][20347] Updated weights for policy 0, policy_version 549666 (0.0010) [2023-12-27 09:26:31,431][20381] Updated weights for policy 1, policy_version 549443 (0.0011) [2023-12-27 09:26:31,436][20347] Updated weights for policy 0, policy_version 549676 (0.0006) [2023-12-27 09:26:31,480][20381] Updated weights for policy 1, policy_version 549453 (0.0007) [2023-12-27 09:26:31,498][20347] Updated weights for policy 0, policy_version 549686 (0.0010) [2023-12-27 09:26:31,559][20347] Updated weights for policy 0, policy_version 549696 (0.0010) [2023-12-27 09:26:32,086][20381] Updated weights for policy 1, policy_version 549463 (0.0006) [2023-12-27 09:26:32,147][20381] Updated weights for policy 1, policy_version 549473 (0.0008) [2023-12-27 09:26:32,205][20381] Updated weights for policy 1, policy_version 549483 (0.0010) [2023-12-27 09:26:32,237][20347] Updated weights for policy 0, policy_version 549706 (0.0010) [2023-12-27 09:26:32,293][20347] Updated weights for policy 0, policy_version 549716 (0.0008) [2023-12-27 09:26:32,359][20347] Updated weights for policy 0, policy_version 549726 (0.0007) [2023-12-27 09:26:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 281427968. Throughput: 0: 10414.4, 1: 10406.4. Samples: 281440164. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:32,363][19170] Avg episode reward: [(0, '335.730'), (1, '878.370')] [2023-12-27 09:26:32,830][20381] Updated weights for policy 1, policy_version 549493 (0.0008) [2023-12-27 09:26:32,884][20381] Updated weights for policy 1, policy_version 549503 (0.0007) [2023-12-27 09:26:32,938][20381] Updated weights for policy 1, policy_version 549513 (0.0006) [2023-12-27 09:26:33,038][20347] Updated weights for policy 0, policy_version 549736 (0.0008) [2023-12-27 09:26:33,095][20347] Updated weights for policy 0, policy_version 549746 (0.0007) [2023-12-27 09:26:33,155][20347] Updated weights for policy 0, policy_version 549756 (0.0007) [2023-12-27 09:26:33,605][20381] Updated weights for policy 1, policy_version 549523 (0.0007) [2023-12-27 09:26:33,651][20381] Updated weights for policy 1, policy_version 549533 (0.0009) [2023-12-27 09:26:33,703][20381] Updated weights for policy 1, policy_version 549543 (0.0007) [2023-12-27 09:26:33,707][20347] Updated weights for policy 0, policy_version 549766 (0.0006) [2023-12-27 09:26:33,763][20347] Updated weights for policy 0, policy_version 549776 (0.0006) [2023-12-27 09:26:33,828][20347] Updated weights for policy 0, policy_version 549786 (0.0008) [2023-12-27 09:26:34,330][20381] Updated weights for policy 1, policy_version 549553 (0.0007) [2023-12-27 09:26:34,395][20381] Updated weights for policy 1, policy_version 549563 (0.0007) [2023-12-27 09:26:34,456][20381] Updated weights for policy 1, policy_version 549573 (0.0009) [2023-12-27 09:26:34,459][20347] Updated weights for policy 0, policy_version 549796 (0.0007) [2023-12-27 09:26:34,509][20347] Updated weights for policy 0, policy_version 549806 (0.0010) [2023-12-27 09:26:34,513][20381] Updated weights for policy 1, policy_version 549583 (0.0011) [2023-12-27 09:26:34,558][20347] Updated weights for policy 0, policy_version 549816 (0.0010) [2023-12-27 09:26:35,131][20292] KL-divergence is very high: 175.4901 [2023-12-27 09:26:35,137][20381] Updated weights for policy 1, policy_version 549593 (0.0011) [2023-12-27 09:26:35,192][20381] Updated weights for policy 1, policy_version 549603 (0.0010) [2023-12-27 09:26:35,224][20347] Updated weights for policy 0, policy_version 549826 (0.0010) [2023-12-27 09:26:35,250][20381] Updated weights for policy 1, policy_version 549613 (0.0010) [2023-12-27 09:26:35,284][20347] Updated weights for policy 0, policy_version 549836 (0.0011) [2023-12-27 09:26:35,338][20347] Updated weights for policy 0, policy_version 549846 (0.0010) [2023-12-27 09:26:35,393][20347] Updated weights for policy 0, policy_version 549856 (0.0010) [2023-12-27 09:26:35,874][20381] Updated weights for policy 1, policy_version 549623 (0.0010) [2023-12-27 09:26:35,929][20381] Updated weights for policy 1, policy_version 549633 (0.0011) [2023-12-27 09:26:35,982][20381] Updated weights for policy 1, policy_version 549643 (0.0008) [2023-12-27 09:26:36,012][20347] Updated weights for policy 0, policy_version 549866 (0.0005) [2023-12-27 09:26:36,072][20347] Updated weights for policy 0, policy_version 549876 (0.0005) [2023-12-27 09:26:36,133][20347] Updated weights for policy 0, policy_version 549886 (0.0007) [2023-12-27 09:26:36,634][20381] Updated weights for policy 1, policy_version 549653 (0.0008) [2023-12-27 09:26:36,693][20381] Updated weights for policy 1, policy_version 549663 (0.0011) [2023-12-27 09:26:36,745][20381] Updated weights for policy 1, policy_version 549673 (0.0011) [2023-12-27 09:26:36,784][20347] Updated weights for policy 0, policy_version 549896 (0.0006) [2023-12-27 09:26:36,844][20347] Updated weights for policy 0, policy_version 549906 (0.0008) [2023-12-27 09:26:36,898][20347] Updated weights for policy 0, policy_version 549916 (0.0007) [2023-12-27 09:26:37,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 281534464. Throughput: 0: 10463.9, 1: 10445.5. Samples: 281504820. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:37,363][19170] Avg episode reward: [(0, '317.420'), (1, '799.880')] [2023-12-27 09:26:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000549920_140795904.pth... [2023-12-27 09:26:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000549680_140738560.pth... [2023-12-27 09:26:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000548464_140427264.pth [2023-12-27 09:26:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000548704_140484608.pth [2023-12-27 09:26:37,441][20381] Updated weights for policy 1, policy_version 549683 (0.0010) [2023-12-27 09:26:37,490][20381] Updated weights for policy 1, policy_version 549693 (0.0011) [2023-12-27 09:26:37,539][20381] Updated weights for policy 1, policy_version 549703 (0.0010) [2023-12-27 09:26:37,603][20347] Updated weights for policy 0, policy_version 549926 (0.0009) [2023-12-27 09:26:37,658][20347] Updated weights for policy 0, policy_version 549936 (0.0009) [2023-12-27 09:26:37,708][20347] Updated weights for policy 0, policy_version 549946 (0.0010) [2023-12-27 09:26:38,247][20381] Updated weights for policy 1, policy_version 549713 (0.0010) [2023-12-27 09:26:38,301][20381] Updated weights for policy 1, policy_version 549723 (0.0008) [2023-12-27 09:26:38,367][20381] Updated weights for policy 1, policy_version 549733 (0.0008) [2023-12-27 09:26:38,385][20347] Updated weights for policy 0, policy_version 549956 (0.0011) [2023-12-27 09:26:38,425][20381] Updated weights for policy 1, policy_version 549743 (0.0008) [2023-12-27 09:26:38,440][20347] Updated weights for policy 0, policy_version 549966 (0.0010) [2023-12-27 09:26:38,495][20347] Updated weights for policy 0, policy_version 549976 (0.0010) [2023-12-27 09:26:39,084][20381] Updated weights for policy 1, policy_version 549753 (0.0010) [2023-12-27 09:26:39,129][20381] Updated weights for policy 1, policy_version 549763 (0.0010) [2023-12-27 09:26:39,176][20347] Updated weights for policy 0, policy_version 549986 (0.0010) [2023-12-27 09:26:39,181][20381] Updated weights for policy 1, policy_version 549773 (0.0011) [2023-12-27 09:26:39,230][20347] Updated weights for policy 0, policy_version 549996 (0.0007) [2023-12-27 09:26:39,287][20347] Updated weights for policy 0, policy_version 550006 (0.0008) [2023-12-27 09:26:39,350][20347] Updated weights for policy 0, policy_version 550016 (0.0008) [2023-12-27 09:26:39,912][20381] Updated weights for policy 1, policy_version 549783 (0.0008) [2023-12-27 09:26:39,979][20381] Updated weights for policy 1, policy_version 549793 (0.0008) [2023-12-27 09:26:40,024][20347] Updated weights for policy 0, policy_version 550026 (0.0006) [2023-12-27 09:26:40,041][20381] Updated weights for policy 1, policy_version 549803 (0.0008) [2023-12-27 09:26:40,080][20347] Updated weights for policy 0, policy_version 550036 (0.0005) [2023-12-27 09:26:40,142][20347] Updated weights for policy 0, policy_version 550046 (0.0006) [2023-12-27 09:26:40,732][20381] Updated weights for policy 1, policy_version 549813 (0.0009) [2023-12-27 09:26:40,788][20381] Updated weights for policy 1, policy_version 549823 (0.0011) [2023-12-27 09:26:40,797][20347] Updated weights for policy 0, policy_version 550056 (0.0010) [2023-12-27 09:26:40,841][20381] Updated weights for policy 1, policy_version 549833 (0.0011) [2023-12-27 09:26:40,851][20347] Updated weights for policy 0, policy_version 550066 (0.0010) [2023-12-27 09:26:40,906][20347] Updated weights for policy 0, policy_version 550076 (0.0010) [2023-12-27 09:26:41,563][20381] Updated weights for policy 1, policy_version 549843 (0.0011) [2023-12-27 09:26:41,618][20347] Updated weights for policy 0, policy_version 550086 (0.0008) [2023-12-27 09:26:41,633][20381] Updated weights for policy 1, policy_version 549853 (0.0009) [2023-12-27 09:26:41,677][20347] Updated weights for policy 0, policy_version 550096 (0.0007) [2023-12-27 09:26:41,703][20381] Updated weights for policy 1, policy_version 549863 (0.0007) [2023-12-27 09:26:41,740][20347] Updated weights for policy 0, policy_version 550106 (0.0008) [2023-12-27 09:26:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 281632768. Throughput: 0: 10500.3, 1: 10330.7. Samples: 281627896. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:42,363][19170] Avg episode reward: [(0, '314.780'), (1, '824.620')] [2023-12-27 09:26:42,374][20381] Updated weights for policy 1, policy_version 549873 (0.0007) [2023-12-27 09:26:42,397][20347] Updated weights for policy 0, policy_version 550116 (0.0006) [2023-12-27 09:26:42,425][20381] Updated weights for policy 1, policy_version 549883 (0.0006) [2023-12-27 09:26:42,452][20347] Updated weights for policy 0, policy_version 550126 (0.0007) [2023-12-27 09:26:42,473][20381] Updated weights for policy 1, policy_version 549893 (0.0008) [2023-12-27 09:26:42,501][20347] Updated weights for policy 0, policy_version 550136 (0.0006) [2023-12-27 09:26:42,522][20381] Updated weights for policy 1, policy_version 549903 (0.0005) [2023-12-27 09:26:43,199][20347] Updated weights for policy 0, policy_version 550146 (0.0006) [2023-12-27 09:26:43,246][20347] Updated weights for policy 0, policy_version 550156 (0.0006) [2023-12-27 09:26:43,259][20381] Updated weights for policy 1, policy_version 549913 (0.0007) [2023-12-27 09:26:43,296][20347] Updated weights for policy 0, policy_version 550166 (0.0005) [2023-12-27 09:26:43,309][20381] Updated weights for policy 1, policy_version 549923 (0.0006) [2023-12-27 09:26:43,344][20347] Updated weights for policy 0, policy_version 550176 (0.0005) [2023-12-27 09:26:43,362][20381] Updated weights for policy 1, policy_version 549933 (0.0006) [2023-12-27 09:26:44,019][20381] Updated weights for policy 1, policy_version 549943 (0.0009) [2023-12-27 09:26:44,059][20347] Updated weights for policy 0, policy_version 550186 (0.0005) [2023-12-27 09:26:44,083][20381] Updated weights for policy 1, policy_version 549953 (0.0008) [2023-12-27 09:26:44,104][20347] Updated weights for policy 0, policy_version 550196 (0.0006) [2023-12-27 09:26:44,142][20381] Updated weights for policy 1, policy_version 549963 (0.0007) [2023-12-27 09:26:44,152][20347] Updated weights for policy 0, policy_version 550206 (0.0005) [2023-12-27 09:26:44,716][20381] Updated weights for policy 1, policy_version 549973 (0.0006) [2023-12-27 09:26:44,773][20381] Updated weights for policy 1, policy_version 549983 (0.0006) [2023-12-27 09:26:44,827][20381] Updated weights for policy 1, policy_version 549993 (0.0005) [2023-12-27 09:26:44,862][20347] Updated weights for policy 0, policy_version 550216 (0.0007) [2023-12-27 09:26:44,918][20347] Updated weights for policy 0, policy_version 550226 (0.0008) [2023-12-27 09:26:44,976][20347] Updated weights for policy 0, policy_version 550236 (0.0008) [2023-12-27 09:26:45,473][20381] Updated weights for policy 1, policy_version 550003 (0.0007) [2023-12-27 09:26:45,531][20381] Updated weights for policy 1, policy_version 550013 (0.0011) [2023-12-27 09:26:45,575][20381] Updated weights for policy 1, policy_version 550023 (0.0010) [2023-12-27 09:26:45,694][20347] Updated weights for policy 0, policy_version 550246 (0.0007) [2023-12-27 09:26:45,750][20347] Updated weights for policy 0, policy_version 550256 (0.0007) [2023-12-27 09:26:45,805][20347] Updated weights for policy 0, policy_version 550266 (0.0007) [2023-12-27 09:26:46,230][20381] Updated weights for policy 1, policy_version 550033 (0.0010) [2023-12-27 09:26:46,286][20381] Updated weights for policy 1, policy_version 550043 (0.0010) [2023-12-27 09:26:46,344][20381] Updated weights for policy 1, policy_version 550053 (0.0005) [2023-12-27 09:26:46,400][20381] Updated weights for policy 1, policy_version 550063 (0.0009) [2023-12-27 09:26:46,538][20347] Updated weights for policy 0, policy_version 550276 (0.0007) [2023-12-27 09:26:46,588][20347] Updated weights for policy 0, policy_version 550286 (0.0007) [2023-12-27 09:26:46,639][20347] Updated weights for policy 0, policy_version 550296 (0.0007) [2023-12-27 09:26:47,002][20381] Updated weights for policy 1, policy_version 550073 (0.0006) [2023-12-27 09:26:47,058][20381] Updated weights for policy 1, policy_version 550083 (0.0005) [2023-12-27 09:26:47,116][20381] Updated weights for policy 1, policy_version 550093 (0.0005) [2023-12-27 09:26:47,356][20347] Updated weights for policy 0, policy_version 550306 (0.0007) [2023-12-27 09:26:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20799.4). Total num frames: 281739264. Throughput: 0: 10429.3, 1: 10392.9. Samples: 281751104. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:47,363][19170] Avg episode reward: [(0, '368.840'), (1, '814.970')] [2023-12-27 09:26:47,414][20347] Updated weights for policy 0, policy_version 550316 (0.0007) [2023-12-27 09:26:47,479][20347] Updated weights for policy 0, policy_version 550326 (0.0007) [2023-12-27 09:26:47,543][20347] Updated weights for policy 0, policy_version 550336 (0.0007) [2023-12-27 09:26:47,747][20381] Updated weights for policy 1, policy_version 550103 (0.0007) [2023-12-27 09:26:47,803][20381] Updated weights for policy 1, policy_version 550113 (0.0007) [2023-12-27 09:26:47,863][20381] Updated weights for policy 1, policy_version 550123 (0.0006) [2023-12-27 09:26:48,227][20347] Updated weights for policy 0, policy_version 550346 (0.0008) [2023-12-27 09:26:48,287][20347] Updated weights for policy 0, policy_version 550356 (0.0007) [2023-12-27 09:26:48,350][20347] Updated weights for policy 0, policy_version 550366 (0.0008) [2023-12-27 09:26:48,524][20381] Updated weights for policy 1, policy_version 550133 (0.0006) [2023-12-27 09:26:48,587][20381] Updated weights for policy 1, policy_version 550143 (0.0008) [2023-12-27 09:26:48,649][20381] Updated weights for policy 1, policy_version 550153 (0.0008) [2023-12-27 09:26:49,035][20347] Updated weights for policy 0, policy_version 550376 (0.0006) [2023-12-27 09:26:49,096][20347] Updated weights for policy 0, policy_version 550386 (0.0005) [2023-12-27 09:26:49,145][20347] Updated weights for policy 0, policy_version 550396 (0.0005) [2023-12-27 09:26:49,352][20381] Updated weights for policy 1, policy_version 550163 (0.0008) [2023-12-27 09:26:49,415][20381] Updated weights for policy 1, policy_version 550173 (0.0007) [2023-12-27 09:26:49,469][20381] Updated weights for policy 1, policy_version 550183 (0.0005) [2023-12-27 09:26:49,809][20347] Updated weights for policy 0, policy_version 550406 (0.0007) [2023-12-27 09:26:49,868][20347] Updated weights for policy 0, policy_version 550416 (0.0006) [2023-12-27 09:26:49,929][20347] Updated weights for policy 0, policy_version 550426 (0.0006) [2023-12-27 09:26:50,077][20381] Updated weights for policy 1, policy_version 550193 (0.0010) [2023-12-27 09:26:50,135][20381] Updated weights for policy 1, policy_version 550203 (0.0010) [2023-12-27 09:26:50,196][20381] Updated weights for policy 1, policy_version 550213 (0.0008) [2023-12-27 09:26:50,256][20381] Updated weights for policy 1, policy_version 550223 (0.0010) [2023-12-27 09:26:50,640][20347] Updated weights for policy 0, policy_version 550436 (0.0008) [2023-12-27 09:26:50,703][20347] Updated weights for policy 0, policy_version 550446 (0.0008) [2023-12-27 09:26:50,763][20347] Updated weights for policy 0, policy_version 550456 (0.0007) [2023-12-27 09:26:50,929][20381] Updated weights for policy 1, policy_version 550233 (0.0010) [2023-12-27 09:26:50,990][20381] Updated weights for policy 1, policy_version 550243 (0.0009) [2023-12-27 09:26:51,058][20381] Updated weights for policy 1, policy_version 550253 (0.0008) [2023-12-27 09:26:51,483][20347] Updated weights for policy 0, policy_version 550466 (0.0007) [2023-12-27 09:26:51,538][20347] Updated weights for policy 0, policy_version 550476 (0.0008) [2023-12-27 09:26:51,600][20347] Updated weights for policy 0, policy_version 550486 (0.0008) [2023-12-27 09:26:51,664][20347] Updated weights for policy 0, policy_version 550496 (0.0008) [2023-12-27 09:26:51,729][20381] Updated weights for policy 1, policy_version 550263 (0.0007) [2023-12-27 09:26:51,792][20381] Updated weights for policy 1, policy_version 550273 (0.0006) [2023-12-27 09:26:51,853][20381] Updated weights for policy 1, policy_version 550283 (0.0006) [2023-12-27 09:26:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 281837568. Throughput: 0: 10415.9, 1: 10355.7. Samples: 281812356. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:52,363][19170] Avg episode reward: [(0, '295.610'), (1, '844.380')] [2023-12-27 09:26:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000550288_140894208.pth... [2023-12-27 09:26:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000549072_140582912.pth [2023-12-27 09:26:52,380][20347] Updated weights for policy 0, policy_version 550506 (0.0009) [2023-12-27 09:26:52,439][20347] Updated weights for policy 0, policy_version 550516 (0.0008) [2023-12-27 09:26:52,486][20381] Updated weights for policy 1, policy_version 550293 (0.0008) [2023-12-27 09:26:52,491][20347] Updated weights for policy 0, policy_version 550526 (0.0007) [2023-12-27 09:26:52,504][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000550528_140951552.pth... [2023-12-27 09:26:52,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000549312_140640256.pth [2023-12-27 09:26:52,547][20381] Updated weights for policy 1, policy_version 550303 (0.0006) [2023-12-27 09:26:52,600][20381] Updated weights for policy 1, policy_version 550313 (0.0007) [2023-12-27 09:26:53,159][20347] Updated weights for policy 0, policy_version 550536 (0.0008) [2023-12-27 09:26:53,206][20347] Updated weights for policy 0, policy_version 550546 (0.0010) [2023-12-27 09:26:53,252][20347] Updated weights for policy 0, policy_version 550556 (0.0009) [2023-12-27 09:26:53,319][20381] Updated weights for policy 1, policy_version 550323 (0.0008) [2023-12-27 09:26:53,380][20381] Updated weights for policy 1, policy_version 550333 (0.0007) [2023-12-27 09:26:53,433][20381] Updated weights for policy 1, policy_version 550343 (0.0006) [2023-12-27 09:26:53,909][20347] Updated weights for policy 0, policy_version 550566 (0.0010) [2023-12-27 09:26:53,960][20347] Updated weights for policy 0, policy_version 550576 (0.0010) [2023-12-27 09:26:54,012][20347] Updated weights for policy 0, policy_version 550586 (0.0010) [2023-12-27 09:26:54,106][20381] Updated weights for policy 1, policy_version 550353 (0.0005) [2023-12-27 09:26:54,163][20381] Updated weights for policy 1, policy_version 550363 (0.0007) [2023-12-27 09:26:54,220][20381] Updated weights for policy 1, policy_version 550373 (0.0008) [2023-12-27 09:26:54,278][20381] Updated weights for policy 1, policy_version 550383 (0.0007) [2023-12-27 09:26:54,687][20347] Updated weights for policy 0, policy_version 550596 (0.0010) [2023-12-27 09:26:54,742][20347] Updated weights for policy 0, policy_version 550606 (0.0010) [2023-12-27 09:26:54,797][20347] Updated weights for policy 0, policy_version 550616 (0.0010) [2023-12-27 09:26:54,991][20381] Updated weights for policy 1, policy_version 550393 (0.0007) [2023-12-27 09:26:55,055][20381] Updated weights for policy 1, policy_version 550403 (0.0008) [2023-12-27 09:26:55,111][20381] Updated weights for policy 1, policy_version 550413 (0.0008) [2023-12-27 09:26:55,461][20347] Updated weights for policy 0, policy_version 550626 (0.0009) [2023-12-27 09:26:55,524][20347] Updated weights for policy 0, policy_version 550636 (0.0010) [2023-12-27 09:26:55,585][20347] Updated weights for policy 0, policy_version 550646 (0.0010) [2023-12-27 09:26:55,639][20347] Updated weights for policy 0, policy_version 550656 (0.0010) [2023-12-27 09:26:55,772][20381] Updated weights for policy 1, policy_version 550423 (0.0009) [2023-12-27 09:26:55,820][20381] Updated weights for policy 1, policy_version 550433 (0.0007) [2023-12-27 09:26:55,868][20381] Updated weights for policy 1, policy_version 550443 (0.0007) [2023-12-27 09:26:56,292][20347] Updated weights for policy 0, policy_version 550666 (0.0010) [2023-12-27 09:26:56,346][20347] Updated weights for policy 0, policy_version 550676 (0.0010) [2023-12-27 09:26:56,407][20347] Updated weights for policy 0, policy_version 550686 (0.0007) [2023-12-27 09:26:56,498][20381] Updated weights for policy 1, policy_version 550453 (0.0009) [2023-12-27 09:26:56,561][20381] Updated weights for policy 1, policy_version 550463 (0.0011) [2023-12-27 09:26:56,613][20381] Updated weights for policy 1, policy_version 550473 (0.0010) [2023-12-27 09:26:57,098][20347] Updated weights for policy 0, policy_version 550696 (0.0006) [2023-12-27 09:26:57,154][20347] Updated weights for policy 0, policy_version 550706 (0.0005) [2023-12-27 09:26:57,201][20381] Updated weights for policy 1, policy_version 550483 (0.0010) [2023-12-27 09:26:57,207][20347] Updated weights for policy 0, policy_version 550716 (0.0007) [2023-12-27 09:26:57,254][20381] Updated weights for policy 1, policy_version 550493 (0.0005) [2023-12-27 09:26:57,307][20381] Updated weights for policy 1, policy_version 550503 (0.0005) [2023-12-27 09:26:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 281952256. Throughput: 0: 10381.9, 1: 10379.5. Samples: 281934588. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:26:57,363][19170] Avg episode reward: [(0, '69.970'), (1, '844.450')] [2023-12-27 09:26:57,810][20347] Updated weights for policy 0, policy_version 550726 (0.0008) [2023-12-27 09:26:57,870][20347] Updated weights for policy 0, policy_version 550736 (0.0008) [2023-12-27 09:26:57,920][20381] Updated weights for policy 1, policy_version 550513 (0.0007) [2023-12-27 09:26:57,932][20347] Updated weights for policy 0, policy_version 550746 (0.0008) [2023-12-27 09:26:57,978][20381] Updated weights for policy 1, policy_version 550523 (0.0007) [2023-12-27 09:26:58,036][20381] Updated weights for policy 1, policy_version 550533 (0.0008) [2023-12-27 09:26:58,092][20381] Updated weights for policy 1, policy_version 550543 (0.0007) [2023-12-27 09:26:58,665][20347] Updated weights for policy 0, policy_version 550756 (0.0009) [2023-12-27 09:26:58,728][20347] Updated weights for policy 0, policy_version 550766 (0.0010) [2023-12-27 09:26:58,796][20347] Updated weights for policy 0, policy_version 550776 (0.0008) [2023-12-27 09:26:58,885][20381] Updated weights for policy 1, policy_version 550553 (0.0008) [2023-12-27 09:26:58,950][20381] Updated weights for policy 1, policy_version 550563 (0.0010) [2023-12-27 09:26:59,007][20381] Updated weights for policy 1, policy_version 550573 (0.0011) [2023-12-27 09:26:59,563][20347] Updated weights for policy 0, policy_version 550786 (0.0011) [2023-12-27 09:26:59,614][20347] Updated weights for policy 0, policy_version 550796 (0.0006) [2023-12-27 09:26:59,673][20347] Updated weights for policy 0, policy_version 550806 (0.0010) [2023-12-27 09:26:59,706][20381] Updated weights for policy 1, policy_version 550583 (0.0009) [2023-12-27 09:26:59,728][20347] Updated weights for policy 0, policy_version 550816 (0.0010) [2023-12-27 09:26:59,773][20381] Updated weights for policy 1, policy_version 550593 (0.0008) [2023-12-27 09:26:59,838][20381] Updated weights for policy 1, policy_version 550603 (0.0008) [2023-12-27 09:27:00,358][20347] Updated weights for policy 0, policy_version 550826 (0.0008) [2023-12-27 09:27:00,419][20347] Updated weights for policy 0, policy_version 550836 (0.0008) [2023-12-27 09:27:00,474][20347] Updated weights for policy 0, policy_version 550846 (0.0005) [2023-12-27 09:27:00,540][20381] Updated weights for policy 1, policy_version 550613 (0.0009) [2023-12-27 09:27:00,595][20381] Updated weights for policy 1, policy_version 550623 (0.0010) [2023-12-27 09:27:00,640][20381] Updated weights for policy 1, policy_version 550633 (0.0010) [2023-12-27 09:27:01,090][20347] Updated weights for policy 0, policy_version 550856 (0.0007) [2023-12-27 09:27:01,143][20347] Updated weights for policy 0, policy_version 550866 (0.0007) [2023-12-27 09:27:01,203][20347] Updated weights for policy 0, policy_version 550876 (0.0007) [2023-12-27 09:27:01,404][20381] Updated weights for policy 1, policy_version 550643 (0.0009) [2023-12-27 09:27:01,458][20381] Updated weights for policy 1, policy_version 550653 (0.0006) [2023-12-27 09:27:01,515][20381] Updated weights for policy 1, policy_version 550663 (0.0009) [2023-12-27 09:27:01,564][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-27 09:27:01,919][20347] Updated weights for policy 0, policy_version 550886 (0.0010) [2023-12-27 09:27:01,982][20347] Updated weights for policy 0, policy_version 550896 (0.0010) [2023-12-27 09:27:02,041][20347] Updated weights for policy 0, policy_version 550906 (0.0010) [2023-12-27 09:27:02,182][20381] Updated weights for policy 1, policy_version 550673 (0.0008) [2023-12-27 09:27:02,249][20381] Updated weights for policy 1, policy_version 550683 (0.0007) [2023-12-27 09:27:02,306][20381] Updated weights for policy 1, policy_version 550693 (0.0007) [2023-12-27 09:27:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 282042368. Throughput: 0: 10348.5, 1: 10296.5. Samples: 282056200. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:02,363][19170] Avg episode reward: [(0, '115.740'), (1, '922.260')] [2023-12-27 09:27:02,367][20381] Updated weights for policy 1, policy_version 550703 (0.0007) [2023-12-27 09:27:02,736][20347] Updated weights for policy 0, policy_version 550916 (0.0009) [2023-12-27 09:27:02,784][20347] Updated weights for policy 0, policy_version 550926 (0.0007) [2023-12-27 09:27:02,833][20347] Updated weights for policy 0, policy_version 550936 (0.0007) [2023-12-27 09:27:02,976][20381] Updated weights for policy 1, policy_version 550713 (0.0008) [2023-12-27 09:27:03,021][20381] Updated weights for policy 1, policy_version 550723 (0.0007) [2023-12-27 09:27:03,063][20381] Updated weights for policy 1, policy_version 550733 (0.0007) [2023-12-27 09:27:03,562][20347] Updated weights for policy 0, policy_version 550946 (0.0008) [2023-12-27 09:27:03,618][20347] Updated weights for policy 0, policy_version 550956 (0.0006) [2023-12-27 09:27:03,678][20347] Updated weights for policy 0, policy_version 550966 (0.0005) [2023-12-27 09:27:03,715][20381] Updated weights for policy 1, policy_version 550743 (0.0006) [2023-12-27 09:27:03,738][20347] Updated weights for policy 0, policy_version 550976 (0.0005) [2023-12-27 09:27:03,766][20381] Updated weights for policy 1, policy_version 550753 (0.0010) [2023-12-27 09:27:03,824][20381] Updated weights for policy 1, policy_version 550763 (0.0010) [2023-12-27 09:27:04,376][20347] Updated weights for policy 0, policy_version 550986 (0.0007) [2023-12-27 09:27:04,434][20347] Updated weights for policy 0, policy_version 550996 (0.0007) [2023-12-27 09:27:04,504][20347] Updated weights for policy 0, policy_version 551006 (0.0009) [2023-12-27 09:27:04,521][20381] Updated weights for policy 1, policy_version 550773 (0.0009) [2023-12-27 09:27:04,577][20381] Updated weights for policy 1, policy_version 550783 (0.0008) [2023-12-27 09:27:04,637][20381] Updated weights for policy 1, policy_version 550793 (0.0005) [2023-12-27 09:27:05,189][20381] Updated weights for policy 1, policy_version 550803 (0.0005) [2023-12-27 09:27:05,247][20381] Updated weights for policy 1, policy_version 550813 (0.0007) [2023-12-27 09:27:05,248][20347] Updated weights for policy 0, policy_version 551016 (0.0005) [2023-12-27 09:27:05,295][20347] Updated weights for policy 0, policy_version 551026 (0.0007) [2023-12-27 09:27:05,309][20381] Updated weights for policy 1, policy_version 550823 (0.0007) [2023-12-27 09:27:05,345][20347] Updated weights for policy 0, policy_version 551036 (0.0005) [2023-12-27 09:27:05,963][20347] Updated weights for policy 0, policy_version 551046 (0.0008) [2023-12-27 09:27:05,993][20381] Updated weights for policy 1, policy_version 550833 (0.0009) [2023-12-27 09:27:06,006][20347] Updated weights for policy 0, policy_version 551056 (0.0006) [2023-12-27 09:27:06,048][20381] Updated weights for policy 1, policy_version 550843 (0.0010) [2023-12-27 09:27:06,064][20347] Updated weights for policy 0, policy_version 551066 (0.0006) [2023-12-27 09:27:06,100][20381] Updated weights for policy 1, policy_version 550853 (0.0010) [2023-12-27 09:27:06,165][20381] Updated weights for policy 1, policy_version 550863 (0.0009) [2023-12-27 09:27:06,798][20347] Updated weights for policy 0, policy_version 551076 (0.0008) [2023-12-27 09:27:06,852][20347] Updated weights for policy 0, policy_version 551086 (0.0007) [2023-12-27 09:27:06,889][20381] Updated weights for policy 1, policy_version 550873 (0.0007) [2023-12-27 09:27:06,905][20347] Updated weights for policy 0, policy_version 551096 (0.0009) [2023-12-27 09:27:06,953][20381] Updated weights for policy 1, policy_version 550883 (0.0006) [2023-12-27 09:27:07,010][20381] Updated weights for policy 1, policy_version 550893 (0.0008) [2023-12-27 09:27:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 282148864. Throughput: 0: 10267.6, 1: 10391.5. Samples: 282118388. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:07,363][19170] Avg episode reward: [(0, '182.540'), (1, '951.440')] [2023-12-27 09:27:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000550896_141049856.pth... [2023-12-27 09:27:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000551104_141099008.pth... [2023-12-27 09:27:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000549680_140738560.pth [2023-12-27 09:27:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000549920_140795904.pth [2023-12-27 09:27:07,538][20347] Updated weights for policy 0, policy_version 551106 (0.0006) [2023-12-27 09:27:07,584][20381] Updated weights for policy 1, policy_version 550903 (0.0007) [2023-12-27 09:27:07,600][20347] Updated weights for policy 0, policy_version 551116 (0.0008) [2023-12-27 09:27:07,638][20381] Updated weights for policy 1, policy_version 550913 (0.0006) [2023-12-27 09:27:07,665][20347] Updated weights for policy 0, policy_version 551126 (0.0010) [2023-12-27 09:27:07,695][20381] Updated weights for policy 1, policy_version 550923 (0.0005) [2023-12-27 09:27:07,724][20347] Updated weights for policy 0, policy_version 551136 (0.0010) [2023-12-27 09:27:08,228][20381] Updated weights for policy 1, policy_version 550933 (0.0005) [2023-12-27 09:27:08,286][20381] Updated weights for policy 1, policy_version 550943 (0.0005) [2023-12-27 09:27:08,333][20347] Updated weights for policy 0, policy_version 551146 (0.0010) [2023-12-27 09:27:08,351][20381] Updated weights for policy 1, policy_version 550953 (0.0007) [2023-12-27 09:27:08,396][20347] Updated weights for policy 0, policy_version 551156 (0.0009) [2023-12-27 09:27:08,458][20347] Updated weights for policy 0, policy_version 551166 (0.0007) [2023-12-27 09:27:08,962][20381] Updated weights for policy 1, policy_version 550963 (0.0007) [2023-12-27 09:27:09,023][20381] Updated weights for policy 1, policy_version 550973 (0.0007) [2023-12-27 09:27:09,075][20381] Updated weights for policy 1, policy_version 550983 (0.0007) [2023-12-27 09:27:09,121][20347] Updated weights for policy 0, policy_version 551176 (0.0009) [2023-12-27 09:27:09,178][20347] Updated weights for policy 0, policy_version 551186 (0.0008) [2023-12-27 09:27:09,235][20347] Updated weights for policy 0, policy_version 551196 (0.0008) [2023-12-27 09:27:09,788][20381] Updated weights for policy 1, policy_version 550993 (0.0006) [2023-12-27 09:27:09,851][20381] Updated weights for policy 1, policy_version 551003 (0.0007) [2023-12-27 09:27:09,903][20381] Updated weights for policy 1, policy_version 551013 (0.0006) [2023-12-27 09:27:09,967][20381] Updated weights for policy 1, policy_version 551023 (0.0007) [2023-12-27 09:27:09,982][20347] Updated weights for policy 0, policy_version 551206 (0.0010) [2023-12-27 09:27:10,041][20347] Updated weights for policy 0, policy_version 551216 (0.0010) [2023-12-27 09:27:10,097][20347] Updated weights for policy 0, policy_version 551226 (0.0011) [2023-12-27 09:27:10,682][20381] Updated weights for policy 1, policy_version 551033 (0.0007) [2023-12-27 09:27:10,734][20381] Updated weights for policy 1, policy_version 551043 (0.0007) [2023-12-27 09:27:10,786][20347] Updated weights for policy 0, policy_version 551236 (0.0009) [2023-12-27 09:27:10,792][20381] Updated weights for policy 1, policy_version 551053 (0.0008) [2023-12-27 09:27:10,836][20347] Updated weights for policy 0, policy_version 551246 (0.0006) [2023-12-27 09:27:10,891][20347] Updated weights for policy 0, policy_version 551256 (0.0007) [2023-12-27 09:27:11,521][20381] Updated weights for policy 1, policy_version 551063 (0.0006) [2023-12-27 09:27:11,585][20381] Updated weights for policy 1, policy_version 551073 (0.0007) [2023-12-27 09:27:11,617][20347] Updated weights for policy 0, policy_version 551266 (0.0008) [2023-12-27 09:27:11,648][20381] Updated weights for policy 1, policy_version 551083 (0.0007) [2023-12-27 09:27:11,677][20347] Updated weights for policy 0, policy_version 551276 (0.0007) [2023-12-27 09:27:11,753][20347] Updated weights for policy 0, policy_version 551286 (0.0007) [2023-12-27 09:27:11,813][20347] Updated weights for policy 0, policy_version 551296 (0.0008) [2023-12-27 09:27:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 282247168. Throughput: 0: 10194.2, 1: 10440.3. Samples: 282242372. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:12,363][19170] Avg episode reward: [(0, '317.500'), (1, '922.500')] [2023-12-27 09:27:12,367][20381] Updated weights for policy 1, policy_version 551093 (0.0008) [2023-12-27 09:27:12,422][20381] Updated weights for policy 1, policy_version 551103 (0.0008) [2023-12-27 09:27:12,480][20381] Updated weights for policy 1, policy_version 551113 (0.0008) [2023-12-27 09:27:12,521][20347] Updated weights for policy 0, policy_version 551306 (0.0007) [2023-12-27 09:27:12,583][20347] Updated weights for policy 0, policy_version 551316 (0.0008) [2023-12-27 09:27:12,642][20347] Updated weights for policy 0, policy_version 551326 (0.0008) [2023-12-27 09:27:13,118][20381] Updated weights for policy 1, policy_version 551123 (0.0007) [2023-12-27 09:27:13,172][20381] Updated weights for policy 1, policy_version 551133 (0.0005) [2023-12-27 09:27:13,219][20381] Updated weights for policy 1, policy_version 551143 (0.0005) [2023-12-27 09:27:13,323][20347] Updated weights for policy 0, policy_version 551336 (0.0009) [2023-12-27 09:27:13,389][20347] Updated weights for policy 0, policy_version 551346 (0.0008) [2023-12-27 09:27:13,457][20347] Updated weights for policy 0, policy_version 551356 (0.0005) [2023-12-27 09:27:13,851][20381] Updated weights for policy 1, policy_version 551153 (0.0007) [2023-12-27 09:27:13,909][20381] Updated weights for policy 1, policy_version 551163 (0.0007) [2023-12-27 09:27:13,963][20381] Updated weights for policy 1, policy_version 551173 (0.0008) [2023-12-27 09:27:14,021][20381] Updated weights for policy 1, policy_version 551183 (0.0005) [2023-12-27 09:27:14,056][20347] Updated weights for policy 0, policy_version 551366 (0.0006) [2023-12-27 09:27:14,119][20347] Updated weights for policy 0, policy_version 551376 (0.0006) [2023-12-27 09:27:14,170][20347] Updated weights for policy 0, policy_version 551386 (0.0005) [2023-12-27 09:27:14,675][20381] Updated weights for policy 1, policy_version 551193 (0.0010) [2023-12-27 09:27:14,736][20381] Updated weights for policy 1, policy_version 551203 (0.0010) [2023-12-27 09:27:14,768][20347] Updated weights for policy 0, policy_version 551396 (0.0006) [2023-12-27 09:27:14,800][20381] Updated weights for policy 1, policy_version 551213 (0.0011) [2023-12-27 09:27:14,827][20347] Updated weights for policy 0, policy_version 551406 (0.0008) [2023-12-27 09:27:14,879][20347] Updated weights for policy 0, policy_version 551416 (0.0008) [2023-12-27 09:27:15,470][20381] Updated weights for policy 1, policy_version 551223 (0.0011) [2023-12-27 09:27:15,530][20381] Updated weights for policy 1, policy_version 551233 (0.0008) [2023-12-27 09:27:15,579][20381] Updated weights for policy 1, policy_version 551243 (0.0005) [2023-12-27 09:27:15,596][20347] Updated weights for policy 0, policy_version 551426 (0.0009) [2023-12-27 09:27:15,646][20347] Updated weights for policy 0, policy_version 551436 (0.0010) [2023-12-27 09:27:15,691][20347] Updated weights for policy 0, policy_version 551446 (0.0010) [2023-12-27 09:27:15,748][20347] Updated weights for policy 0, policy_version 551456 (0.0009) [2023-12-27 09:27:16,160][20381] Updated weights for policy 1, policy_version 551253 (0.0008) [2023-12-27 09:27:16,210][20381] Updated weights for policy 1, policy_version 551263 (0.0006) [2023-12-27 09:27:16,269][20381] Updated weights for policy 1, policy_version 551273 (0.0006) [2023-12-27 09:27:16,406][20347] Updated weights for policy 0, policy_version 551466 (0.0010) [2023-12-27 09:27:16,464][20347] Updated weights for policy 0, policy_version 551476 (0.0010) [2023-12-27 09:27:16,526][20347] Updated weights for policy 0, policy_version 551486 (0.0010) [2023-12-27 09:27:16,918][20381] Updated weights for policy 1, policy_version 551283 (0.0009) [2023-12-27 09:27:16,972][20381] Updated weights for policy 1, policy_version 551293 (0.0007) [2023-12-27 09:27:17,035][20381] Updated weights for policy 1, policy_version 551303 (0.0005) [2023-12-27 09:27:17,165][20347] Updated weights for policy 0, policy_version 551496 (0.0006) [2023-12-27 09:27:17,228][20347] Updated weights for policy 0, policy_version 551506 (0.0005) [2023-12-27 09:27:17,292][20347] Updated weights for policy 0, policy_version 551516 (0.0005) [2023-12-27 09:27:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 282361856. Throughput: 0: 10203.3, 1: 10420.6. Samples: 282368240. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:17,363][19170] Avg episode reward: [(0, '276.260'), (1, '903.130')] [2023-12-27 09:27:17,679][20381] Updated weights for policy 1, policy_version 551313 (0.0006) [2023-12-27 09:27:17,727][20381] Updated weights for policy 1, policy_version 551323 (0.0009) [2023-12-27 09:27:17,777][20381] Updated weights for policy 1, policy_version 551333 (0.0011) [2023-12-27 09:27:17,832][20381] Updated weights for policy 1, policy_version 551343 (0.0009) [2023-12-27 09:27:17,893][20347] Updated weights for policy 0, policy_version 551526 (0.0007) [2023-12-27 09:27:17,941][20347] Updated weights for policy 0, policy_version 551536 (0.0007) [2023-12-27 09:27:17,985][20347] Updated weights for policy 0, policy_version 551546 (0.0007) [2023-12-27 09:27:18,482][20381] Updated weights for policy 1, policy_version 551353 (0.0006) [2023-12-27 09:27:18,541][20381] Updated weights for policy 1, policy_version 551363 (0.0006) [2023-12-27 09:27:18,607][20381] Updated weights for policy 1, policy_version 551373 (0.0007) [2023-12-27 09:27:18,763][20347] Updated weights for policy 0, policy_version 551556 (0.0007) [2023-12-27 09:27:18,816][20347] Updated weights for policy 0, policy_version 551566 (0.0008) [2023-12-27 09:27:18,875][20347] Updated weights for policy 0, policy_version 551576 (0.0008) [2023-12-27 09:27:19,249][20381] Updated weights for policy 1, policy_version 551383 (0.0008) [2023-12-27 09:27:19,322][20381] Updated weights for policy 1, policy_version 551393 (0.0007) [2023-12-27 09:27:19,383][20381] Updated weights for policy 1, policy_version 551403 (0.0008) [2023-12-27 09:27:19,573][20347] Updated weights for policy 0, policy_version 551586 (0.0008) [2023-12-27 09:27:19,636][20347] Updated weights for policy 0, policy_version 551596 (0.0010) [2023-12-27 09:27:19,696][20347] Updated weights for policy 0, policy_version 551606 (0.0011) [2023-12-27 09:27:19,752][20347] Updated weights for policy 0, policy_version 551616 (0.0010) [2023-12-27 09:27:20,038][20381] Updated weights for policy 1, policy_version 551413 (0.0007) [2023-12-27 09:27:20,105][20381] Updated weights for policy 1, policy_version 551423 (0.0006) [2023-12-27 09:27:20,168][20381] Updated weights for policy 1, policy_version 551433 (0.0007) [2023-12-27 09:27:20,487][20347] Updated weights for policy 0, policy_version 551626 (0.0008) [2023-12-27 09:27:20,540][20347] Updated weights for policy 0, policy_version 551636 (0.0008) [2023-12-27 09:27:20,603][20347] Updated weights for policy 0, policy_version 551646 (0.0008) [2023-12-27 09:27:20,820][20381] Updated weights for policy 1, policy_version 551443 (0.0008) [2023-12-27 09:27:20,886][20381] Updated weights for policy 1, policy_version 551453 (0.0007) [2023-12-27 09:27:20,954][20381] Updated weights for policy 1, policy_version 551463 (0.0008) [2023-12-27 09:27:21,328][20347] Updated weights for policy 0, policy_version 551656 (0.0010) [2023-12-27 09:27:21,399][20347] Updated weights for policy 0, policy_version 551666 (0.0009) [2023-12-27 09:27:21,458][20347] Updated weights for policy 0, policy_version 551676 (0.0010) [2023-12-27 09:27:21,674][20381] Updated weights for policy 1, policy_version 551473 (0.0008) [2023-12-27 09:27:21,742][20381] Updated weights for policy 1, policy_version 551483 (0.0007) [2023-12-27 09:27:21,800][20381] Updated weights for policy 1, policy_version 551493 (0.0007) [2023-12-27 09:27:21,872][20381] Updated weights for policy 1, policy_version 551503 (0.0007) [2023-12-27 09:27:22,143][20347] Updated weights for policy 0, policy_version 551686 (0.0010) [2023-12-27 09:27:22,198][20347] Updated weights for policy 0, policy_version 551696 (0.0010) [2023-12-27 09:27:22,262][20347] Updated weights for policy 0, policy_version 551706 (0.0009) [2023-12-27 09:27:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 282460160. Throughput: 0: 10155.1, 1: 10397.8. Samples: 282429704. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:22,363][19170] Avg episode reward: [(0, '287.820'), (1, '922.300')] [2023-12-27 09:27:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000551712_141254656.pth... [2023-12-27 09:27:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000551504_141205504.pth... [2023-12-27 09:27:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000550528_140951552.pth [2023-12-27 09:27:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000550288_140894208.pth [2023-12-27 09:27:22,578][20381] Updated weights for policy 1, policy_version 551513 (0.0009) [2023-12-27 09:27:22,644][20381] Updated weights for policy 1, policy_version 551523 (0.0008) [2023-12-27 09:27:22,704][20381] Updated weights for policy 1, policy_version 551533 (0.0007) [2023-12-27 09:27:22,945][20347] Updated weights for policy 0, policy_version 551716 (0.0010) [2023-12-27 09:27:23,006][20347] Updated weights for policy 0, policy_version 551726 (0.0008) [2023-12-27 09:27:23,064][20347] Updated weights for policy 0, policy_version 551736 (0.0008) [2023-12-27 09:27:23,369][20381] Updated weights for policy 1, policy_version 551543 (0.0006) [2023-12-27 09:27:23,423][20381] Updated weights for policy 1, policy_version 551553 (0.0008) [2023-12-27 09:27:23,475][20381] Updated weights for policy 1, policy_version 551563 (0.0009) [2023-12-27 09:27:23,665][20347] Updated weights for policy 0, policy_version 551746 (0.0007) [2023-12-27 09:27:23,727][20347] Updated weights for policy 0, policy_version 551756 (0.0006) [2023-12-27 09:27:23,794][20347] Updated weights for policy 0, policy_version 551766 (0.0008) [2023-12-27 09:27:23,858][20347] Updated weights for policy 0, policy_version 551776 (0.0008) [2023-12-27 09:27:24,061][20381] Updated weights for policy 1, policy_version 551573 (0.0008) [2023-12-27 09:27:24,123][20381] Updated weights for policy 1, policy_version 551583 (0.0008) [2023-12-27 09:27:24,186][20381] Updated weights for policy 1, policy_version 551593 (0.0005) [2023-12-27 09:27:24,491][20347] Updated weights for policy 0, policy_version 551786 (0.0006) [2023-12-27 09:27:24,553][20347] Updated weights for policy 0, policy_version 551796 (0.0007) [2023-12-27 09:27:24,605][20347] Updated weights for policy 0, policy_version 551806 (0.0007) [2023-12-27 09:27:24,717][20381] Updated weights for policy 1, policy_version 551603 (0.0007) [2023-12-27 09:27:24,763][20381] Updated weights for policy 1, policy_version 551613 (0.0009) [2023-12-27 09:27:24,807][20381] Updated weights for policy 1, policy_version 551623 (0.0008) [2023-12-27 09:27:25,171][20347] Updated weights for policy 0, policy_version 551816 (0.0006) [2023-12-27 09:27:25,232][20347] Updated weights for policy 0, policy_version 551826 (0.0008) [2023-12-27 09:27:25,294][20347] Updated weights for policy 0, policy_version 551836 (0.0005) [2023-12-27 09:27:25,574][20381] Updated weights for policy 1, policy_version 551633 (0.0009) [2023-12-27 09:27:25,630][20381] Updated weights for policy 1, policy_version 551643 (0.0009) [2023-12-27 09:27:25,641][20292] KL-divergence is very high: 129.1469 [2023-12-27 09:27:25,685][20381] Updated weights for policy 1, policy_version 551653 (0.0007) [2023-12-27 09:27:25,734][20381] Updated weights for policy 1, policy_version 551663 (0.0007) [2023-12-27 09:27:25,860][20347] Updated weights for policy 0, policy_version 551846 (0.0005) [2023-12-27 09:27:25,910][20347] Updated weights for policy 0, policy_version 551856 (0.0008) [2023-12-27 09:27:25,962][20347] Updated weights for policy 0, policy_version 551866 (0.0008) [2023-12-27 09:27:26,317][20381] Updated weights for policy 1, policy_version 551673 (0.0009) [2023-12-27 09:27:26,371][20381] Updated weights for policy 1, policy_version 551683 (0.0007) [2023-12-27 09:27:26,422][20381] Updated weights for policy 1, policy_version 551693 (0.0008) [2023-12-27 09:27:26,632][20347] Updated weights for policy 0, policy_version 551876 (0.0006) [2023-12-27 09:27:26,700][20347] Updated weights for policy 0, policy_version 551886 (0.0006) [2023-12-27 09:27:26,763][20347] Updated weights for policy 0, policy_version 551896 (0.0005) [2023-12-27 09:27:27,111][20381] Updated weights for policy 1, policy_version 551703 (0.0007) [2023-12-27 09:27:27,163][20381] Updated weights for policy 1, policy_version 551713 (0.0008) [2023-12-27 09:27:27,211][20381] Updated weights for policy 1, policy_version 551723 (0.0007) [2023-12-27 09:27:27,350][20347] Updated weights for policy 0, policy_version 551906 (0.0005) [2023-12-27 09:27:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20771.6). Total num frames: 282566656. Throughput: 0: 10162.1, 1: 10444.2. Samples: 282555176. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:27,363][19170] Avg episode reward: [(0, '299.420'), (1, '932.250')] [2023-12-27 09:27:27,394][20347] Updated weights for policy 0, policy_version 551916 (0.0008) [2023-12-27 09:27:27,439][20347] Updated weights for policy 0, policy_version 551926 (0.0008) [2023-12-27 09:27:27,487][20347] Updated weights for policy 0, policy_version 551936 (0.0007) [2023-12-27 09:27:27,816][20381] Updated weights for policy 1, policy_version 551733 (0.0008) [2023-12-27 09:27:27,870][20381] Updated weights for policy 1, policy_version 551743 (0.0008) [2023-12-27 09:27:27,928][20381] Updated weights for policy 1, policy_version 551753 (0.0008) [2023-12-27 09:27:28,204][20347] Updated weights for policy 0, policy_version 551946 (0.0005) [2023-12-27 09:27:28,257][20347] Updated weights for policy 0, policy_version 551956 (0.0006) [2023-12-27 09:27:28,309][20347] Updated weights for policy 0, policy_version 551966 (0.0007) [2023-12-27 09:27:28,675][20381] Updated weights for policy 1, policy_version 551763 (0.0008) [2023-12-27 09:27:28,731][20381] Updated weights for policy 1, policy_version 551773 (0.0008) [2023-12-27 09:27:28,779][20381] Updated weights for policy 1, policy_version 551783 (0.0008) [2023-12-27 09:27:28,924][20347] Updated weights for policy 0, policy_version 551976 (0.0008) [2023-12-27 09:27:28,967][20347] Updated weights for policy 0, policy_version 551986 (0.0006) [2023-12-27 09:27:29,013][20347] Updated weights for policy 0, policy_version 551996 (0.0005) [2023-12-27 09:27:29,523][20381] Updated weights for policy 1, policy_version 551793 (0.0007) [2023-12-27 09:27:29,572][20381] Updated weights for policy 1, policy_version 551803 (0.0008) [2023-12-27 09:27:29,610][20347] Updated weights for policy 0, policy_version 552006 (0.0007) [2023-12-27 09:27:29,624][20381] Updated weights for policy 1, policy_version 551813 (0.0005) [2023-12-27 09:27:29,659][20347] Updated weights for policy 0, policy_version 552016 (0.0005) [2023-12-27 09:27:29,677][20381] Updated weights for policy 1, policy_version 551823 (0.0005) [2023-12-27 09:27:29,711][20347] Updated weights for policy 0, policy_version 552026 (0.0007) [2023-12-27 09:27:30,358][20347] Updated weights for policy 0, policy_version 552036 (0.0007) [2023-12-27 09:27:30,404][20347] Updated weights for policy 0, policy_version 552046 (0.0005) [2023-12-27 09:27:30,428][20381] Updated weights for policy 1, policy_version 551833 (0.0007) [2023-12-27 09:27:30,447][20347] Updated weights for policy 0, policy_version 552056 (0.0005) [2023-12-27 09:27:30,472][20381] Updated weights for policy 1, policy_version 551843 (0.0006) [2023-12-27 09:27:30,521][20381] Updated weights for policy 1, policy_version 551853 (0.0008) [2023-12-27 09:27:31,105][20347] Updated weights for policy 0, policy_version 552066 (0.0005) [2023-12-27 09:27:31,149][20381] Updated weights for policy 1, policy_version 551863 (0.0008) [2023-12-27 09:27:31,167][20347] Updated weights for policy 0, policy_version 552076 (0.0007) [2023-12-27 09:27:31,206][20381] Updated weights for policy 1, policy_version 551873 (0.0007) [2023-12-27 09:27:31,225][20347] Updated weights for policy 0, policy_version 552086 (0.0007) [2023-12-27 09:27:31,262][20381] Updated weights for policy 1, policy_version 551883 (0.0007) [2023-12-27 09:27:31,289][20347] Updated weights for policy 0, policy_version 552096 (0.0007) [2023-12-27 09:27:31,966][20381] Updated weights for policy 1, policy_version 551893 (0.0006) [2023-12-27 09:27:31,993][20347] Updated weights for policy 0, policy_version 552106 (0.0007) [2023-12-27 09:27:32,020][20381] Updated weights for policy 1, policy_version 551903 (0.0006) [2023-12-27 09:27:32,045][20347] Updated weights for policy 0, policy_version 552116 (0.0006) [2023-12-27 09:27:32,077][20381] Updated weights for policy 1, policy_version 551913 (0.0006) [2023-12-27 09:27:32,097][20347] Updated weights for policy 0, policy_version 552126 (0.0006) [2023-12-27 09:27:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 282673152. Throughput: 0: 10273.2, 1: 10368.4. Samples: 282679980. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:32,363][19170] Avg episode reward: [(0, '308.010'), (1, '951.610')] [2023-12-27 09:27:32,771][20347] Updated weights for policy 0, policy_version 552136 (0.0007) [2023-12-27 09:27:32,798][20381] Updated weights for policy 1, policy_version 551923 (0.0006) [2023-12-27 09:27:32,822][20347] Updated weights for policy 0, policy_version 552146 (0.0005) [2023-12-27 09:27:32,850][20381] Updated weights for policy 1, policy_version 551933 (0.0005) [2023-12-27 09:27:32,870][20347] Updated weights for policy 0, policy_version 552156 (0.0006) [2023-12-27 09:27:32,902][20381] Updated weights for policy 1, policy_version 551943 (0.0006) [2023-12-27 09:27:33,474][20347] Updated weights for policy 0, policy_version 552166 (0.0006) [2023-12-27 09:27:33,531][20347] Updated weights for policy 0, policy_version 552176 (0.0005) [2023-12-27 09:27:33,581][20381] Updated weights for policy 1, policy_version 551953 (0.0007) [2023-12-27 09:27:33,589][20347] Updated weights for policy 0, policy_version 552186 (0.0005) [2023-12-27 09:27:33,631][20381] Updated weights for policy 1, policy_version 551963 (0.0006) [2023-12-27 09:27:33,678][20381] Updated weights for policy 1, policy_version 551973 (0.0006) [2023-12-27 09:27:33,724][20381] Updated weights for policy 1, policy_version 551983 (0.0005) [2023-12-27 09:27:34,095][20347] Updated weights for policy 0, policy_version 552196 (0.0006) [2023-12-27 09:27:34,143][20347] Updated weights for policy 0, policy_version 552206 (0.0007) [2023-12-27 09:27:34,209][20347] Updated weights for policy 0, policy_version 552216 (0.0007) [2023-12-27 09:27:34,405][20381] Updated weights for policy 1, policy_version 551993 (0.0006) [2023-12-27 09:27:34,470][20381] Updated weights for policy 1, policy_version 552003 (0.0008) [2023-12-27 09:27:34,531][20381] Updated weights for policy 1, policy_version 552013 (0.0006) [2023-12-27 09:27:34,927][20347] Updated weights for policy 0, policy_version 552226 (0.0008) [2023-12-27 09:27:34,976][20347] Updated weights for policy 0, policy_version 552236 (0.0007) [2023-12-27 09:27:35,024][20347] Updated weights for policy 0, policy_version 552246 (0.0007) [2023-12-27 09:27:35,079][20347] Updated weights for policy 0, policy_version 552256 (0.0007) [2023-12-27 09:27:35,129][20381] Updated weights for policy 1, policy_version 552023 (0.0009) [2023-12-27 09:27:35,188][20381] Updated weights for policy 1, policy_version 552033 (0.0010) [2023-12-27 09:27:35,243][20381] Updated weights for policy 1, policy_version 552043 (0.0010) [2023-12-27 09:27:35,799][20347] Updated weights for policy 0, policy_version 552266 (0.0007) [2023-12-27 09:27:35,852][20347] Updated weights for policy 0, policy_version 552276 (0.0007) [2023-12-27 09:27:35,910][20347] Updated weights for policy 0, policy_version 552286 (0.0007) [2023-12-27 09:27:35,938][20381] Updated weights for policy 1, policy_version 552053 (0.0011) [2023-12-27 09:27:35,996][20381] Updated weights for policy 1, policy_version 552063 (0.0011) [2023-12-27 09:27:36,042][20381] Updated weights for policy 1, policy_version 552073 (0.0010) [2023-12-27 09:27:36,630][20347] Updated weights for policy 0, policy_version 552296 (0.0007) [2023-12-27 09:27:36,688][20347] Updated weights for policy 0, policy_version 552306 (0.0007) [2023-12-27 09:27:36,733][20381] Updated weights for policy 1, policy_version 552083 (0.0010) [2023-12-27 09:27:36,739][20347] Updated weights for policy 0, policy_version 552316 (0.0007) [2023-12-27 09:27:36,792][20381] Updated weights for policy 1, policy_version 552093 (0.0011) [2023-12-27 09:27:36,851][20381] Updated weights for policy 1, policy_version 552103 (0.0011) [2023-12-27 09:27:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 282771456. Throughput: 0: 10335.9, 1: 10371.0. Samples: 282744168. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:37,363][19170] Avg episode reward: [(0, '340.470'), (1, '922.390')] [2023-12-27 09:27:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000552320_141410304.pth... [2023-12-27 09:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000552112_141361152.pth... [2023-12-27 09:27:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000551104_141099008.pth [2023-12-27 09:27:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000550896_141049856.pth [2023-12-27 09:27:37,449][20347] Updated weights for policy 0, policy_version 552326 (0.0008) [2023-12-27 09:27:37,500][20347] Updated weights for policy 0, policy_version 552336 (0.0007) [2023-12-27 09:27:37,537][20381] Updated weights for policy 1, policy_version 552113 (0.0011) [2023-12-27 09:27:37,550][20347] Updated weights for policy 0, policy_version 552346 (0.0007) [2023-12-27 09:27:37,589][20381] Updated weights for policy 1, policy_version 552123 (0.0011) [2023-12-27 09:27:37,642][20381] Updated weights for policy 1, policy_version 552133 (0.0011) [2023-12-27 09:27:37,703][20381] Updated weights for policy 1, policy_version 552143 (0.0010) [2023-12-27 09:27:38,251][20347] Updated weights for policy 0, policy_version 552356 (0.0006) [2023-12-27 09:27:38,306][20347] Updated weights for policy 0, policy_version 552366 (0.0008) [2023-12-27 09:27:38,369][20347] Updated weights for policy 0, policy_version 552376 (0.0007) [2023-12-27 09:27:38,400][20381] Updated weights for policy 1, policy_version 552153 (0.0011) [2023-12-27 09:27:38,465][20381] Updated weights for policy 1, policy_version 552163 (0.0011) [2023-12-27 09:27:38,524][20381] Updated weights for policy 1, policy_version 552173 (0.0011) [2023-12-27 09:27:39,096][20347] Updated weights for policy 0, policy_version 552386 (0.0006) [2023-12-27 09:27:39,144][20347] Updated weights for policy 0, policy_version 552396 (0.0007) [2023-12-27 09:27:39,191][20347] Updated weights for policy 0, policy_version 552406 (0.0006) [2023-12-27 09:27:39,212][20381] Updated weights for policy 1, policy_version 552183 (0.0011) [2023-12-27 09:27:39,250][20347] Updated weights for policy 0, policy_version 552416 (0.0007) [2023-12-27 09:27:39,271][20381] Updated weights for policy 1, policy_version 552193 (0.0008) [2023-12-27 09:27:39,321][20381] Updated weights for policy 1, policy_version 552203 (0.0009) [2023-12-27 09:27:39,942][20347] Updated weights for policy 0, policy_version 552426 (0.0012) [2023-12-27 09:27:39,998][20347] Updated weights for policy 0, policy_version 552436 (0.0008) [2023-12-27 09:27:40,057][20347] Updated weights for policy 0, policy_version 552446 (0.0007) [2023-12-27 09:27:40,059][20381] Updated weights for policy 1, policy_version 552213 (0.0007) [2023-12-27 09:27:40,108][20381] Updated weights for policy 1, policy_version 552223 (0.0006) [2023-12-27 09:27:40,165][20381] Updated weights for policy 1, policy_version 552233 (0.0006) [2023-12-27 09:27:40,745][20347] Updated weights for policy 0, policy_version 552456 (0.0006) [2023-12-27 09:27:40,805][20347] Updated weights for policy 0, policy_version 552466 (0.0009) [2023-12-27 09:27:40,830][20381] Updated weights for policy 1, policy_version 552243 (0.0005) [2023-12-27 09:27:40,865][20347] Updated weights for policy 0, policy_version 552476 (0.0008) [2023-12-27 09:27:40,893][20381] Updated weights for policy 1, policy_version 552253 (0.0005) [2023-12-27 09:27:40,953][20381] Updated weights for policy 1, policy_version 552263 (0.0006) [2023-12-27 09:27:41,564][20347] Updated weights for policy 0, policy_version 552486 (0.0008) [2023-12-27 09:27:41,576][20381] Updated weights for policy 1, policy_version 552273 (0.0010) [2023-12-27 09:27:41,621][20347] Updated weights for policy 0, policy_version 552496 (0.0009) [2023-12-27 09:27:41,640][20381] Updated weights for policy 1, policy_version 552283 (0.0009) [2023-12-27 09:27:41,679][20347] Updated weights for policy 0, policy_version 552506 (0.0006) [2023-12-27 09:27:41,696][20381] Updated weights for policy 1, policy_version 552293 (0.0011) [2023-12-27 09:27:41,758][20381] Updated weights for policy 1, policy_version 552303 (0.0009) [2023-12-27 09:27:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 282869760. Throughput: 0: 10318.4, 1: 10342.6. Samples: 282864336. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-12-27 09:27:42,363][19170] Avg episode reward: [(0, '387.130'), (1, '873.860')] [2023-12-27 09:27:42,370][20381] Updated weights for policy 1, policy_version 552313 (0.0010) [2023-12-27 09:27:42,402][20347] Updated weights for policy 0, policy_version 552516 (0.0006) [2023-12-27 09:27:42,433][20381] Updated weights for policy 1, policy_version 552323 (0.0008) [2023-12-27 09:27:42,457][20347] Updated weights for policy 0, policy_version 552526 (0.0008) [2023-12-27 09:27:42,489][20381] Updated weights for policy 1, policy_version 552333 (0.0008) [2023-12-27 09:27:42,513][20347] Updated weights for policy 0, policy_version 552536 (0.0008) [2023-12-27 09:27:43,119][20347] Updated weights for policy 0, policy_version 552546 (0.0007) [2023-12-27 09:27:43,180][20347] Updated weights for policy 0, policy_version 552556 (0.0008) [2023-12-27 09:27:43,202][20381] Updated weights for policy 1, policy_version 552343 (0.0008) [2023-12-27 09:27:43,227][20347] Updated weights for policy 0, policy_version 552566 (0.0007) [2023-12-27 09:27:43,254][20381] Updated weights for policy 1, policy_version 552353 (0.0007) [2023-12-27 09:27:43,270][20347] Updated weights for policy 0, policy_version 552576 (0.0008) [2023-12-27 09:27:43,308][20381] Updated weights for policy 1, policy_version 552363 (0.0007) [2023-12-27 09:27:43,893][20347] Updated weights for policy 0, policy_version 552586 (0.0007) [2023-12-27 09:27:43,941][20347] Updated weights for policy 0, policy_version 552596 (0.0008) [2023-12-27 09:27:43,986][20347] Updated weights for policy 0, policy_version 552606 (0.0007) [2023-12-27 09:27:44,028][20381] Updated weights for policy 1, policy_version 552373 (0.0008) [2023-12-27 09:27:44,079][20381] Updated weights for policy 1, policy_version 552383 (0.0007) [2023-12-27 09:27:44,131][20381] Updated weights for policy 1, policy_version 552393 (0.0008) [2023-12-27 09:27:44,633][20347] Updated weights for policy 0, policy_version 552616 (0.0008) [2023-12-27 09:27:44,679][20347] Updated weights for policy 0, policy_version 552626 (0.0008) [2023-12-27 09:27:44,732][20347] Updated weights for policy 0, policy_version 552636 (0.0007) [2023-12-27 09:27:44,782][20381] Updated weights for policy 1, policy_version 552403 (0.0008) [2023-12-27 09:27:44,843][20381] Updated weights for policy 1, policy_version 552413 (0.0009) [2023-12-27 09:27:44,910][20381] Updated weights for policy 1, policy_version 552423 (0.0008) [2023-12-27 09:27:45,464][20347] Updated weights for policy 0, policy_version 552646 (0.0009) [2023-12-27 09:27:45,518][20347] Updated weights for policy 0, policy_version 552656 (0.0010) [2023-12-27 09:27:45,573][20347] Updated weights for policy 0, policy_version 552666 (0.0007) [2023-12-27 09:27:45,573][20381] Updated weights for policy 1, policy_version 552433 (0.0010) [2023-12-27 09:27:45,634][20381] Updated weights for policy 1, policy_version 552443 (0.0006) [2023-12-27 09:27:45,685][20381] Updated weights for policy 1, policy_version 552453 (0.0010) [2023-12-27 09:27:45,734][20381] Updated weights for policy 1, policy_version 552463 (0.0010) [2023-12-27 09:27:46,168][20347] Updated weights for policy 0, policy_version 552676 (0.0007) [2023-12-27 09:27:46,222][20347] Updated weights for policy 0, policy_version 552686 (0.0010) [2023-12-27 09:27:46,303][20347] Updated weights for policy 0, policy_version 552696 (0.0010) [2023-12-27 09:27:46,388][20381] Updated weights for policy 1, policy_version 552473 (0.0011) [2023-12-27 09:27:46,445][20381] Updated weights for policy 1, policy_version 552483 (0.0010) [2023-12-27 09:27:46,499][20381] Updated weights for policy 1, policy_version 552493 (0.0010) [2023-12-27 09:27:46,935][20347] Updated weights for policy 0, policy_version 552706 (0.0009) [2023-12-27 09:27:46,994][20347] Updated weights for policy 0, policy_version 552716 (0.0005) [2023-12-27 09:27:47,041][20347] Updated weights for policy 0, policy_version 552726 (0.0005) [2023-12-27 09:27:47,077][20381] Updated weights for policy 1, policy_version 552503 (0.0006) [2023-12-27 09:27:47,089][20347] Updated weights for policy 0, policy_version 552736 (0.0009) [2023-12-27 09:27:47,132][20381] Updated weights for policy 1, policy_version 552513 (0.0005) [2023-12-27 09:27:47,190][20381] Updated weights for policy 1, policy_version 552523 (0.0005) [2023-12-27 09:27:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 282984448. Throughput: 0: 10375.8, 1: 10398.1. Samples: 282991028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:27:47,363][19170] Avg episode reward: [(0, '400.580'), (1, '873.850')] [2023-12-27 09:27:47,683][20381] Updated weights for policy 1, policy_version 552533 (0.0005) [2023-12-27 09:27:47,737][20347] Updated weights for policy 0, policy_version 552746 (0.0010) [2023-12-27 09:27:47,744][20381] Updated weights for policy 1, policy_version 552543 (0.0006) [2023-12-27 09:27:47,788][20347] Updated weights for policy 0, policy_version 552756 (0.0010) [2023-12-27 09:27:47,793][20381] Updated weights for policy 1, policy_version 552553 (0.0011) [2023-12-27 09:27:47,836][20347] Updated weights for policy 0, policy_version 552766 (0.0010) [2023-12-27 09:27:48,491][20347] Updated weights for policy 0, policy_version 552776 (0.0010) [2023-12-27 09:27:48,500][20381] Updated weights for policy 1, policy_version 552563 (0.0010) [2023-12-27 09:27:48,546][20347] Updated weights for policy 0, policy_version 552786 (0.0007) [2023-12-27 09:27:48,559][20381] Updated weights for policy 1, policy_version 552573 (0.0009) [2023-12-27 09:27:48,594][20347] Updated weights for policy 0, policy_version 552796 (0.0007) [2023-12-27 09:27:48,624][20381] Updated weights for policy 1, policy_version 552583 (0.0008) [2023-12-27 09:27:49,168][20381] Updated weights for policy 1, policy_version 552593 (0.0005) [2023-12-27 09:27:49,223][20381] Updated weights for policy 1, policy_version 552603 (0.0008) [2023-12-27 09:27:49,288][20381] Updated weights for policy 1, policy_version 552613 (0.0008) [2023-12-27 09:27:49,299][20347] Updated weights for policy 0, policy_version 552806 (0.0009) [2023-12-27 09:27:49,350][20381] Updated weights for policy 1, policy_version 552623 (0.0009) [2023-12-27 09:27:49,361][20347] Updated weights for policy 0, policy_version 552816 (0.0010) [2023-12-27 09:27:49,415][20347] Updated weights for policy 0, policy_version 552826 (0.0005) [2023-12-27 09:27:50,051][20347] Updated weights for policy 0, policy_version 552836 (0.0006) [2023-12-27 09:27:50,107][20347] Updated weights for policy 0, policy_version 552846 (0.0008) [2023-12-27 09:27:50,120][20381] Updated weights for policy 1, policy_version 552633 (0.0007) [2023-12-27 09:27:50,162][20347] Updated weights for policy 0, policy_version 552856 (0.0007) [2023-12-27 09:27:50,173][20381] Updated weights for policy 1, policy_version 552643 (0.0007) [2023-12-27 09:27:50,232][20381] Updated weights for policy 1, policy_version 552653 (0.0006) [2023-12-27 09:27:50,863][20347] Updated weights for policy 0, policy_version 552866 (0.0008) [2023-12-27 09:27:50,899][20381] Updated weights for policy 1, policy_version 552663 (0.0006) [2023-12-27 09:27:50,918][20347] Updated weights for policy 0, policy_version 552876 (0.0010) [2023-12-27 09:27:50,957][20381] Updated weights for policy 1, policy_version 552673 (0.0007) [2023-12-27 09:27:50,974][20347] Updated weights for policy 0, policy_version 552886 (0.0010) [2023-12-27 09:27:51,014][20381] Updated weights for policy 1, policy_version 552683 (0.0005) [2023-12-27 09:27:51,033][20347] Updated weights for policy 0, policy_version 552896 (0.0010) [2023-12-27 09:27:51,767][20381] Updated weights for policy 1, policy_version 552693 (0.0007) [2023-12-27 09:27:51,777][20347] Updated weights for policy 0, policy_version 552906 (0.0007) [2023-12-27 09:27:51,828][20381] Updated weights for policy 1, policy_version 552703 (0.0007) [2023-12-27 09:27:51,831][20347] Updated weights for policy 0, policy_version 552916 (0.0005) [2023-12-27 09:27:51,882][20347] Updated weights for policy 0, policy_version 552926 (0.0008) [2023-12-27 09:27:51,887][20381] Updated weights for policy 1, policy_version 552713 (0.0007) [2023-12-27 09:27:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 283082752. Throughput: 0: 10415.7, 1: 10394.3. Samples: 283054840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:27:52,363][19170] Avg episode reward: [(0, '367.990'), (1, '892.980')] [2023-12-27 09:27:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000552928_141565952.pth... [2023-12-27 09:27:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000552720_141516800.pth... [2023-12-27 09:27:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000551504_141205504.pth [2023-12-27 09:27:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000551712_141254656.pth [2023-12-27 09:27:52,567][20347] Updated weights for policy 0, policy_version 552936 (0.0007) [2023-12-27 09:27:52,618][20347] Updated weights for policy 0, policy_version 552946 (0.0010) [2023-12-27 09:27:52,622][20381] Updated weights for policy 1, policy_version 552723 (0.0007) [2023-12-27 09:27:52,669][20347] Updated weights for policy 0, policy_version 552956 (0.0006) [2023-12-27 09:27:52,671][20381] Updated weights for policy 1, policy_version 552733 (0.0007) [2023-12-27 09:27:52,724][20381] Updated weights for policy 1, policy_version 552743 (0.0006) [2023-12-27 09:27:53,316][20381] Updated weights for policy 1, policy_version 552753 (0.0007) [2023-12-27 09:27:53,342][20347] Updated weights for policy 0, policy_version 552966 (0.0007) [2023-12-27 09:27:53,365][20381] Updated weights for policy 1, policy_version 552763 (0.0006) [2023-12-27 09:27:53,389][20347] Updated weights for policy 0, policy_version 552976 (0.0008) [2023-12-27 09:27:53,420][20381] Updated weights for policy 1, policy_version 552773 (0.0005) [2023-12-27 09:27:53,437][20347] Updated weights for policy 0, policy_version 552986 (0.0007) [2023-12-27 09:27:53,472][20381] Updated weights for policy 1, policy_version 552783 (0.0005) [2023-12-27 09:27:53,994][20381] Updated weights for policy 1, policy_version 552793 (0.0008) [2023-12-27 09:27:54,045][20381] Updated weights for policy 1, policy_version 552803 (0.0008) [2023-12-27 09:27:54,062][20347] Updated weights for policy 0, policy_version 552996 (0.0007) [2023-12-27 09:27:54,095][20381] Updated weights for policy 1, policy_version 552813 (0.0006) [2023-12-27 09:27:54,107][20347] Updated weights for policy 0, policy_version 553006 (0.0006) [2023-12-27 09:27:54,156][20347] Updated weights for policy 0, policy_version 553016 (0.0005) [2023-12-27 09:27:54,719][20347] Updated weights for policy 0, policy_version 553026 (0.0006) [2023-12-27 09:27:54,780][20347] Updated weights for policy 0, policy_version 553036 (0.0008) [2023-12-27 09:27:54,821][20381] Updated weights for policy 1, policy_version 552823 (0.0010) [2023-12-27 09:27:54,835][20347] Updated weights for policy 0, policy_version 553046 (0.0006) [2023-12-27 09:27:54,884][20381] Updated weights for policy 1, policy_version 552833 (0.0008) [2023-12-27 09:27:54,892][20347] Updated weights for policy 0, policy_version 553056 (0.0008) [2023-12-27 09:27:54,939][20381] Updated weights for policy 1, policy_version 552843 (0.0005) [2023-12-27 09:27:55,513][20347] Updated weights for policy 0, policy_version 553066 (0.0007) [2023-12-27 09:27:55,567][20347] Updated weights for policy 0, policy_version 553076 (0.0007) [2023-12-27 09:27:55,584][20381] Updated weights for policy 1, policy_version 552853 (0.0008) [2023-12-27 09:27:55,621][20347] Updated weights for policy 0, policy_version 553086 (0.0006) [2023-12-27 09:27:55,639][20381] Updated weights for policy 1, policy_version 552863 (0.0010) [2023-12-27 09:27:55,694][20381] Updated weights for policy 1, policy_version 552873 (0.0010) [2023-12-27 09:27:56,231][20347] Updated weights for policy 0, policy_version 553096 (0.0006) [2023-12-27 09:27:56,290][20347] Updated weights for policy 0, policy_version 553106 (0.0007) [2023-12-27 09:27:56,337][20347] Updated weights for policy 0, policy_version 553116 (0.0010) [2023-12-27 09:27:56,395][20381] Updated weights for policy 1, policy_version 552883 (0.0010) [2023-12-27 09:27:56,443][20381] Updated weights for policy 1, policy_version 552893 (0.0010) [2023-12-27 09:27:56,497][20381] Updated weights for policy 1, policy_version 552903 (0.0010) [2023-12-27 09:27:56,954][20347] Updated weights for policy 0, policy_version 553126 (0.0007) [2023-12-27 09:27:57,005][20347] Updated weights for policy 0, policy_version 553136 (0.0006) [2023-12-27 09:27:57,055][20347] Updated weights for policy 0, policy_version 553146 (0.0007) [2023-12-27 09:27:57,193][20381] Updated weights for policy 1, policy_version 552913 (0.0010) [2023-12-27 09:27:57,241][20381] Updated weights for policy 1, policy_version 552923 (0.0010) [2023-12-27 09:27:57,288][20381] Updated weights for policy 1, policy_version 552933 (0.0010) [2023-12-27 09:27:57,348][20381] Updated weights for policy 1, policy_version 552943 (0.0010) [2023-12-27 09:27:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 283197440. Throughput: 0: 10466.7, 1: 10397.5. Samples: 283181260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:27:57,363][19170] Avg episode reward: [(0, '319.210'), (1, '883.250')] [2023-12-27 09:27:57,672][20347] Updated weights for policy 0, policy_version 553156 (0.0007) [2023-12-27 09:27:57,720][20347] Updated weights for policy 0, policy_version 553166 (0.0006) [2023-12-27 09:27:57,774][20347] Updated weights for policy 0, policy_version 553176 (0.0008) [2023-12-27 09:27:58,052][20381] Updated weights for policy 1, policy_version 552953 (0.0011) [2023-12-27 09:27:58,106][20381] Updated weights for policy 1, policy_version 552963 (0.0008) [2023-12-27 09:27:58,163][20381] Updated weights for policy 1, policy_version 552973 (0.0006) [2023-12-27 09:27:58,507][20347] Updated weights for policy 0, policy_version 553186 (0.0006) [2023-12-27 09:27:58,564][20347] Updated weights for policy 0, policy_version 553196 (0.0008) [2023-12-27 09:27:58,623][20347] Updated weights for policy 0, policy_version 553206 (0.0008) [2023-12-27 09:27:58,682][20347] Updated weights for policy 0, policy_version 553216 (0.0008) [2023-12-27 09:27:58,884][20381] Updated weights for policy 1, policy_version 552983 (0.0010) [2023-12-27 09:27:58,942][20381] Updated weights for policy 1, policy_version 552993 (0.0010) [2023-12-27 09:27:58,996][20381] Updated weights for policy 1, policy_version 553003 (0.0009) [2023-12-27 09:27:59,425][20347] Updated weights for policy 0, policy_version 553226 (0.0009) [2023-12-27 09:27:59,482][20347] Updated weights for policy 0, policy_version 553236 (0.0008) [2023-12-27 09:27:59,548][20347] Updated weights for policy 0, policy_version 553246 (0.0007) [2023-12-27 09:27:59,685][20381] Updated weights for policy 1, policy_version 553013 (0.0010) [2023-12-27 09:27:59,748][20381] Updated weights for policy 1, policy_version 553023 (0.0005) [2023-12-27 09:27:59,803][20381] Updated weights for policy 1, policy_version 553033 (0.0009) [2023-12-27 09:28:00,232][20347] Updated weights for policy 0, policy_version 553256 (0.0007) [2023-12-27 09:28:00,286][20347] Updated weights for policy 0, policy_version 553266 (0.0007) [2023-12-27 09:28:00,344][20347] Updated weights for policy 0, policy_version 553276 (0.0007) [2023-12-27 09:28:00,472][20381] Updated weights for policy 1, policy_version 553043 (0.0010) [2023-12-27 09:28:00,537][20381] Updated weights for policy 1, policy_version 553053 (0.0011) [2023-12-27 09:28:00,592][20381] Updated weights for policy 1, policy_version 553063 (0.0010) [2023-12-27 09:28:01,042][20347] Updated weights for policy 0, policy_version 553286 (0.0007) [2023-12-27 09:28:01,101][20347] Updated weights for policy 0, policy_version 553296 (0.0007) [2023-12-27 09:28:01,167][20347] Updated weights for policy 0, policy_version 553306 (0.0007) [2023-12-27 09:28:01,274][20381] Updated weights for policy 1, policy_version 553073 (0.0010) [2023-12-27 09:28:01,336][20381] Updated weights for policy 1, policy_version 553083 (0.0006) [2023-12-27 09:28:01,396][20381] Updated weights for policy 1, policy_version 553093 (0.0007) [2023-12-27 09:28:01,456][20381] Updated weights for policy 1, policy_version 553103 (0.0006) [2023-12-27 09:28:01,855][20347] Updated weights for policy 0, policy_version 553316 (0.0007) [2023-12-27 09:28:01,914][20347] Updated weights for policy 0, policy_version 553326 (0.0006) [2023-12-27 09:28:01,975][20347] Updated weights for policy 0, policy_version 553336 (0.0006) [2023-12-27 09:28:02,080][20381] Updated weights for policy 1, policy_version 553113 (0.0005) [2023-12-27 09:28:02,132][20381] Updated weights for policy 1, policy_version 553123 (0.0005) [2023-12-27 09:28:02,192][20381] Updated weights for policy 1, policy_version 553133 (0.0005) [2023-12-27 09:28:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 283295744. Throughput: 0: 10456.5, 1: 10342.0. Samples: 283304172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:02,363][19170] Avg episode reward: [(0, '318.740'), (1, '816.110')] [2023-12-27 09:28:02,553][20347] Updated weights for policy 0, policy_version 553346 (0.0005) [2023-12-27 09:28:02,610][20347] Updated weights for policy 0, policy_version 553356 (0.0006) [2023-12-27 09:28:02,667][20347] Updated weights for policy 0, policy_version 553366 (0.0008) [2023-12-27 09:28:02,732][20347] Updated weights for policy 0, policy_version 553376 (0.0008) [2023-12-27 09:28:02,800][20381] Updated weights for policy 1, policy_version 553143 (0.0007) [2023-12-27 09:28:02,848][20381] Updated weights for policy 1, policy_version 553153 (0.0006) [2023-12-27 09:28:02,898][20381] Updated weights for policy 1, policy_version 553163 (0.0007) [2023-12-27 09:28:03,389][20347] Updated weights for policy 0, policy_version 553386 (0.0005) [2023-12-27 09:28:03,444][20347] Updated weights for policy 0, policy_version 553396 (0.0005) [2023-12-27 09:28:03,496][20347] Updated weights for policy 0, policy_version 553406 (0.0005) [2023-12-27 09:28:03,546][20381] Updated weights for policy 1, policy_version 553173 (0.0009) [2023-12-27 09:28:03,600][20381] Updated weights for policy 1, policy_version 553183 (0.0010) [2023-12-27 09:28:03,645][20381] Updated weights for policy 1, policy_version 553193 (0.0010) [2023-12-27 09:28:04,014][20347] Updated weights for policy 0, policy_version 553416 (0.0006) [2023-12-27 09:28:04,079][20347] Updated weights for policy 0, policy_version 553426 (0.0006) [2023-12-27 09:28:04,136][20347] Updated weights for policy 0, policy_version 553436 (0.0007) [2023-12-27 09:28:04,389][20381] Updated weights for policy 1, policy_version 553203 (0.0009) [2023-12-27 09:28:04,449][20381] Updated weights for policy 1, policy_version 553213 (0.0007) [2023-12-27 09:28:04,513][20381] Updated weights for policy 1, policy_version 553223 (0.0007) [2023-12-27 09:28:04,786][20347] Updated weights for policy 0, policy_version 553446 (0.0006) [2023-12-27 09:28:04,842][20347] Updated weights for policy 0, policy_version 553456 (0.0005) [2023-12-27 09:28:04,897][20347] Updated weights for policy 0, policy_version 553466 (0.0008) [2023-12-27 09:28:05,187][20381] Updated weights for policy 1, policy_version 553233 (0.0007) [2023-12-27 09:28:05,234][20381] Updated weights for policy 1, policy_version 553243 (0.0008) [2023-12-27 09:28:05,280][20381] Updated weights for policy 1, policy_version 553253 (0.0008) [2023-12-27 09:28:05,326][20381] Updated weights for policy 1, policy_version 553263 (0.0007) [2023-12-27 09:28:05,553][20347] Updated weights for policy 0, policy_version 553476 (0.0006) [2023-12-27 09:28:05,600][20347] Updated weights for policy 0, policy_version 553486 (0.0007) [2023-12-27 09:28:05,651][20347] Updated weights for policy 0, policy_version 553496 (0.0007) [2023-12-27 09:28:06,038][20381] Updated weights for policy 1, policy_version 553273 (0.0008) [2023-12-27 09:28:06,089][20381] Updated weights for policy 1, policy_version 553283 (0.0005) [2023-12-27 09:28:06,154][20381] Updated weights for policy 1, policy_version 553293 (0.0008) [2023-12-27 09:28:06,351][20347] Updated weights for policy 0, policy_version 553506 (0.0009) [2023-12-27 09:28:06,413][20347] Updated weights for policy 0, policy_version 553516 (0.0009) [2023-12-27 09:28:06,473][20347] Updated weights for policy 0, policy_version 553526 (0.0008) [2023-12-27 09:28:06,532][20347] Updated weights for policy 0, policy_version 553536 (0.0008) [2023-12-27 09:28:06,744][20381] Updated weights for policy 1, policy_version 553303 (0.0007) [2023-12-27 09:28:06,810][20381] Updated weights for policy 1, policy_version 553313 (0.0008) [2023-12-27 09:28:06,872][20381] Updated weights for policy 1, policy_version 553323 (0.0006) [2023-12-27 09:28:07,253][20347] Updated weights for policy 0, policy_version 553546 (0.0010) [2023-12-27 09:28:07,301][20347] Updated weights for policy 0, policy_version 553556 (0.0010) [2023-12-27 09:28:07,352][20347] Updated weights for policy 0, policy_version 553566 (0.0010) [2023-12-27 09:28:07,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 283394048. Throughput: 0: 10533.0, 1: 10332.2. Samples: 283368640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:07,364][19170] Avg episode reward: [(0, '413.920'), (1, '864.800')] [2023-12-27 09:28:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000553328_141672448.pth... [2023-12-27 09:28:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000553568_141729792.pth... [2023-12-27 09:28:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000552112_141361152.pth [2023-12-27 09:28:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000552320_141410304.pth [2023-12-27 09:28:07,513][20381] Updated weights for policy 1, policy_version 553333 (0.0009) [2023-12-27 09:28:07,571][20381] Updated weights for policy 1, policy_version 553343 (0.0011) [2023-12-27 09:28:07,628][20381] Updated weights for policy 1, policy_version 553353 (0.0011) [2023-12-27 09:28:07,978][20347] Updated weights for policy 0, policy_version 553576 (0.0010) [2023-12-27 09:28:08,043][20347] Updated weights for policy 0, policy_version 553586 (0.0008) [2023-12-27 09:28:08,099][20347] Updated weights for policy 0, policy_version 553596 (0.0010) [2023-12-27 09:28:08,316][20381] Updated weights for policy 1, policy_version 553363 (0.0009) [2023-12-27 09:28:08,384][20381] Updated weights for policy 1, policy_version 553373 (0.0007) [2023-12-27 09:28:08,442][20381] Updated weights for policy 1, policy_version 553383 (0.0008) [2023-12-27 09:28:08,796][20347] Updated weights for policy 0, policy_version 553606 (0.0010) [2023-12-27 09:28:08,842][20347] Updated weights for policy 0, policy_version 553616 (0.0010) [2023-12-27 09:28:08,889][20347] Updated weights for policy 0, policy_version 553626 (0.0010) [2023-12-27 09:28:09,103][20381] Updated weights for policy 1, policy_version 553393 (0.0007) [2023-12-27 09:28:09,152][20381] Updated weights for policy 1, policy_version 553403 (0.0007) [2023-12-27 09:28:09,214][20381] Updated weights for policy 1, policy_version 553413 (0.0008) [2023-12-27 09:28:09,277][20381] Updated weights for policy 1, policy_version 553423 (0.0009) [2023-12-27 09:28:09,645][20347] Updated weights for policy 0, policy_version 553636 (0.0009) [2023-12-27 09:28:09,702][20347] Updated weights for policy 0, policy_version 553646 (0.0006) [2023-12-27 09:28:09,760][20347] Updated weights for policy 0, policy_version 553656 (0.0006) [2023-12-27 09:28:10,034][20381] Updated weights for policy 1, policy_version 553433 (0.0007) [2023-12-27 09:28:10,098][20381] Updated weights for policy 1, policy_version 553443 (0.0006) [2023-12-27 09:28:10,160][20381] Updated weights for policy 1, policy_version 553453 (0.0007) [2023-12-27 09:28:10,465][20347] Updated weights for policy 0, policy_version 553666 (0.0006) [2023-12-27 09:28:10,514][20347] Updated weights for policy 0, policy_version 553676 (0.0008) [2023-12-27 09:28:10,563][20347] Updated weights for policy 0, policy_version 553686 (0.0010) [2023-12-27 09:28:10,611][20347] Updated weights for policy 0, policy_version 553696 (0.0006) [2023-12-27 09:28:10,771][20381] Updated weights for policy 1, policy_version 553463 (0.0008) [2023-12-27 09:28:10,816][20381] Updated weights for policy 1, policy_version 553473 (0.0008) [2023-12-27 09:28:10,862][20381] Updated weights for policy 1, policy_version 553483 (0.0008) [2023-12-27 09:28:11,274][20347] Updated weights for policy 0, policy_version 553706 (0.0007) [2023-12-27 09:28:11,338][20347] Updated weights for policy 0, policy_version 553716 (0.0008) [2023-12-27 09:28:11,407][20347] Updated weights for policy 0, policy_version 553726 (0.0008) [2023-12-27 09:28:11,581][20381] Updated weights for policy 1, policy_version 553493 (0.0008) [2023-12-27 09:28:11,636][20381] Updated weights for policy 1, policy_version 553503 (0.0007) [2023-12-27 09:28:11,704][20381] Updated weights for policy 1, policy_version 553513 (0.0009) [2023-12-27 09:28:12,104][20347] Updated weights for policy 0, policy_version 553736 (0.0010) [2023-12-27 09:28:12,159][20347] Updated weights for policy 0, policy_version 553746 (0.0010) [2023-12-27 09:28:12,225][20347] Updated weights for policy 0, policy_version 553756 (0.0009) [2023-12-27 09:28:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 283500544. Throughput: 0: 10497.8, 1: 10306.3. Samples: 283491360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:12,363][19170] Avg episode reward: [(0, '490.550'), (1, '912.370')] [2023-12-27 09:28:12,409][20381] Updated weights for policy 1, policy_version 553523 (0.0007) [2023-12-27 09:28:12,461][20381] Updated weights for policy 1, policy_version 553533 (0.0007) [2023-12-27 09:28:12,519][20381] Updated weights for policy 1, policy_version 553543 (0.0007) [2023-12-27 09:28:12,882][20347] Updated weights for policy 0, policy_version 553766 (0.0010) [2023-12-27 09:28:12,941][20347] Updated weights for policy 0, policy_version 553776 (0.0009) [2023-12-27 09:28:12,989][20347] Updated weights for policy 0, policy_version 553786 (0.0005) [2023-12-27 09:28:13,203][20381] Updated weights for policy 1, policy_version 553553 (0.0007) [2023-12-27 09:28:13,260][20381] Updated weights for policy 1, policy_version 553563 (0.0008) [2023-12-27 09:28:13,310][20381] Updated weights for policy 1, policy_version 553573 (0.0011) [2023-12-27 09:28:13,376][20381] Updated weights for policy 1, policy_version 553583 (0.0008) [2023-12-27 09:28:13,658][20347] Updated weights for policy 0, policy_version 553796 (0.0007) [2023-12-27 09:28:13,704][20347] Updated weights for policy 0, policy_version 553806 (0.0010) [2023-12-27 09:28:13,748][20347] Updated weights for policy 0, policy_version 553816 (0.0010) [2023-12-27 09:28:14,046][20381] Updated weights for policy 1, policy_version 553593 (0.0008) [2023-12-27 09:28:14,108][20381] Updated weights for policy 1, policy_version 553603 (0.0007) [2023-12-27 09:28:14,167][20381] Updated weights for policy 1, policy_version 553613 (0.0008) [2023-12-27 09:28:14,436][20347] Updated weights for policy 0, policy_version 553826 (0.0010) [2023-12-27 09:28:14,496][20347] Updated weights for policy 0, policy_version 553836 (0.0010) [2023-12-27 09:28:14,550][20347] Updated weights for policy 0, policy_version 553846 (0.0010) [2023-12-27 09:28:14,604][20347] Updated weights for policy 0, policy_version 553856 (0.0007) [2023-12-27 09:28:14,890][20381] Updated weights for policy 1, policy_version 553623 (0.0007) [2023-12-27 09:28:14,949][20381] Updated weights for policy 1, policy_version 553633 (0.0006) [2023-12-27 09:28:15,012][20381] Updated weights for policy 1, policy_version 553643 (0.0008) [2023-12-27 09:28:15,232][20347] Updated weights for policy 0, policy_version 553866 (0.0008) [2023-12-27 09:28:15,293][20347] Updated weights for policy 0, policy_version 553876 (0.0010) [2023-12-27 09:28:15,355][20347] Updated weights for policy 0, policy_version 553886 (0.0009) [2023-12-27 09:28:15,713][20381] Updated weights for policy 1, policy_version 553653 (0.0007) [2023-12-27 09:28:15,764][20381] Updated weights for policy 1, policy_version 553663 (0.0005) [2023-12-27 09:28:15,822][20381] Updated weights for policy 1, policy_version 553673 (0.0006) [2023-12-27 09:28:15,886][20347] Updated weights for policy 0, policy_version 553896 (0.0009) [2023-12-27 09:28:15,941][20347] Updated weights for policy 0, policy_version 553906 (0.0010) [2023-12-27 09:28:15,992][20347] Updated weights for policy 0, policy_version 553916 (0.0010) [2023-12-27 09:28:16,488][20381] Updated weights for policy 1, policy_version 553683 (0.0008) [2023-12-27 09:28:16,544][20381] Updated weights for policy 1, policy_version 553693 (0.0008) [2023-12-27 09:28:16,603][20381] Updated weights for policy 1, policy_version 553703 (0.0008) [2023-12-27 09:28:16,650][20347] Updated weights for policy 0, policy_version 553926 (0.0007) [2023-12-27 09:28:16,707][20347] Updated weights for policy 0, policy_version 553936 (0.0007) [2023-12-27 09:28:16,765][20347] Updated weights for policy 0, policy_version 553946 (0.0008) [2023-12-27 09:28:17,274][20381] Updated weights for policy 1, policy_version 553713 (0.0006) [2023-12-27 09:28:17,328][20381] Updated weights for policy 1, policy_version 553723 (0.0007) [2023-12-27 09:28:17,354][20347] Updated weights for policy 0, policy_version 553956 (0.0008) [2023-12-27 09:28:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 283598848. Throughput: 0: 10486.2, 1: 10303.0. Samples: 283615496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:17,363][19170] Avg episode reward: [(0, '558.470'), (1, '844.570')] [2023-12-27 09:28:17,375][20381] Updated weights for policy 1, policy_version 553733 (0.0006) [2023-12-27 09:28:17,404][20347] Updated weights for policy 0, policy_version 553966 (0.0006) [2023-12-27 09:28:17,424][20381] Updated weights for policy 1, policy_version 553743 (0.0007) [2023-12-27 09:28:17,464][20347] Updated weights for policy 0, policy_version 553976 (0.0005) [2023-12-27 09:28:17,987][20347] Updated weights for policy 0, policy_version 553986 (0.0006) [2023-12-27 09:28:18,039][20347] Updated weights for policy 0, policy_version 553996 (0.0010) [2023-12-27 09:28:18,084][20347] Updated weights for policy 0, policy_version 554006 (0.0007) [2023-12-27 09:28:18,097][20381] Updated weights for policy 1, policy_version 553753 (0.0007) [2023-12-27 09:28:18,138][20347] Updated weights for policy 0, policy_version 554016 (0.0007) [2023-12-27 09:28:18,154][20381] Updated weights for policy 1, policy_version 553763 (0.0007) [2023-12-27 09:28:18,210][20381] Updated weights for policy 1, policy_version 553773 (0.0007) [2023-12-27 09:28:18,797][20347] Updated weights for policy 0, policy_version 554026 (0.0005) [2023-12-27 09:28:18,850][20347] Updated weights for policy 0, policy_version 554036 (0.0007) [2023-12-27 09:28:18,899][20347] Updated weights for policy 0, policy_version 554046 (0.0007) [2023-12-27 09:28:18,924][20381] Updated weights for policy 1, policy_version 553783 (0.0007) [2023-12-27 09:28:18,973][20381] Updated weights for policy 1, policy_version 553793 (0.0007) [2023-12-27 09:28:19,018][20381] Updated weights for policy 1, policy_version 553803 (0.0008) [2023-12-27 09:28:19,563][20347] Updated weights for policy 0, policy_version 554056 (0.0007) [2023-12-27 09:28:19,620][20347] Updated weights for policy 0, policy_version 554066 (0.0008) [2023-12-27 09:28:19,680][20347] Updated weights for policy 0, policy_version 554076 (0.0007) [2023-12-27 09:28:19,733][20381] Updated weights for policy 1, policy_version 553813 (0.0008) [2023-12-27 09:28:19,790][20381] Updated weights for policy 1, policy_version 553823 (0.0008) [2023-12-27 09:28:19,850][20381] Updated weights for policy 1, policy_version 553833 (0.0008) [2023-12-27 09:28:20,326][20347] Updated weights for policy 0, policy_version 554086 (0.0008) [2023-12-27 09:28:20,386][20347] Updated weights for policy 0, policy_version 554096 (0.0006) [2023-12-27 09:28:20,442][20347] Updated weights for policy 0, policy_version 554106 (0.0005) [2023-12-27 09:28:20,585][20381] Updated weights for policy 1, policy_version 553843 (0.0008) [2023-12-27 09:28:20,648][20381] Updated weights for policy 1, policy_version 553853 (0.0008) [2023-12-27 09:28:20,709][20381] Updated weights for policy 1, policy_version 553863 (0.0008) [2023-12-27 09:28:21,074][20347] Updated weights for policy 0, policy_version 554116 (0.0007) [2023-12-27 09:28:21,138][20347] Updated weights for policy 0, policy_version 554126 (0.0007) [2023-12-27 09:28:21,199][20347] Updated weights for policy 0, policy_version 554136 (0.0006) [2023-12-27 09:28:21,456][20381] Updated weights for policy 1, policy_version 553873 (0.0008) [2023-12-27 09:28:21,515][20381] Updated weights for policy 1, policy_version 553883 (0.0008) [2023-12-27 09:28:21,568][20381] Updated weights for policy 1, policy_version 553893 (0.0008) [2023-12-27 09:28:21,623][20381] Updated weights for policy 1, policy_version 553903 (0.0007) [2023-12-27 09:28:21,855][20347] Updated weights for policy 0, policy_version 554146 (0.0008) [2023-12-27 09:28:21,914][20347] Updated weights for policy 0, policy_version 554156 (0.0010) [2023-12-27 09:28:21,974][20347] Updated weights for policy 0, policy_version 554166 (0.0007) [2023-12-27 09:28:22,036][20347] Updated weights for policy 0, policy_version 554176 (0.0007) [2023-12-27 09:28:22,283][20381] Updated weights for policy 1, policy_version 553913 (0.0007) [2023-12-27 09:28:22,346][20381] Updated weights for policy 1, policy_version 553923 (0.0009) [2023-12-27 09:28:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 283705344. Throughput: 0: 10482.2, 1: 10269.2. Samples: 283677976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:22,363][19170] Avg episode reward: [(0, '776.510'), (1, '776.820')] [2023-12-27 09:28:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000554176_141885440.pth... [2023-12-27 09:28:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000552928_141565952.pth [2023-12-27 09:28:22,401][20381] Updated weights for policy 1, policy_version 553933 (0.0009) [2023-12-27 09:28:22,416][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000553936_141828096.pth... [2023-12-27 09:28:22,419][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000552720_141516800.pth [2023-12-27 09:28:22,680][20347] Updated weights for policy 0, policy_version 554186 (0.0005) [2023-12-27 09:28:22,732][20347] Updated weights for policy 0, policy_version 554196 (0.0010) [2023-12-27 09:28:22,784][20347] Updated weights for policy 0, policy_version 554206 (0.0008) [2023-12-27 09:28:23,081][20381] Updated weights for policy 1, policy_version 553943 (0.0006) [2023-12-27 09:28:23,135][20381] Updated weights for policy 1, policy_version 553953 (0.0005) [2023-12-27 09:28:23,191][20381] Updated weights for policy 1, policy_version 553963 (0.0006) [2023-12-27 09:28:23,422][20347] Updated weights for policy 0, policy_version 554216 (0.0009) [2023-12-27 09:28:23,477][20347] Updated weights for policy 0, policy_version 554226 (0.0010) [2023-12-27 09:28:23,534][20347] Updated weights for policy 0, policy_version 554236 (0.0010) [2023-12-27 09:28:23,805][20381] Updated weights for policy 1, policy_version 553973 (0.0006) [2023-12-27 09:28:23,863][20381] Updated weights for policy 1, policy_version 553983 (0.0005) [2023-12-27 09:28:23,925][20381] Updated weights for policy 1, policy_version 553993 (0.0008) [2023-12-27 09:28:24,127][20347] Updated weights for policy 0, policy_version 554246 (0.0009) [2023-12-27 09:28:24,192][20347] Updated weights for policy 0, policy_version 554256 (0.0009) [2023-12-27 09:28:24,261][20347] Updated weights for policy 0, policy_version 554266 (0.0008) [2023-12-27 09:28:24,530][20381] Updated weights for policy 1, policy_version 554003 (0.0009) [2023-12-27 09:28:24,583][20381] Updated weights for policy 1, policy_version 554013 (0.0008) [2023-12-27 09:28:24,636][20381] Updated weights for policy 1, policy_version 554023 (0.0007) [2023-12-27 09:28:24,907][20347] Updated weights for policy 0, policy_version 554276 (0.0009) [2023-12-27 09:28:24,958][20347] Updated weights for policy 0, policy_version 554286 (0.0005) [2023-12-27 09:28:25,010][20347] Updated weights for policy 0, policy_version 554296 (0.0008) [2023-12-27 09:28:25,302][20381] Updated weights for policy 1, policy_version 554033 (0.0006) [2023-12-27 09:28:25,357][20381] Updated weights for policy 1, policy_version 554043 (0.0011) [2023-12-27 09:28:25,426][20381] Updated weights for policy 1, policy_version 554053 (0.0009) [2023-12-27 09:28:25,484][20381] Updated weights for policy 1, policy_version 554063 (0.0007) [2023-12-27 09:28:25,686][20347] Updated weights for policy 0, policy_version 554306 (0.0007) [2023-12-27 09:28:25,731][20347] Updated weights for policy 0, policy_version 554316 (0.0009) [2023-12-27 09:28:25,783][20347] Updated weights for policy 0, policy_version 554326 (0.0011) [2023-12-27 09:28:25,838][20347] Updated weights for policy 0, policy_version 554336 (0.0007) [2023-12-27 09:28:26,095][20381] Updated weights for policy 1, policy_version 554073 (0.0008) [2023-12-27 09:28:26,149][20381] Updated weights for policy 1, policy_version 554083 (0.0009) [2023-12-27 09:28:26,201][20381] Updated weights for policy 1, policy_version 554093 (0.0008) [2023-12-27 09:28:26,545][20347] Updated weights for policy 0, policy_version 554346 (0.0008) [2023-12-27 09:28:26,603][20347] Updated weights for policy 0, policy_version 554356 (0.0008) [2023-12-27 09:28:26,656][20347] Updated weights for policy 0, policy_version 554366 (0.0008) [2023-12-27 09:28:26,919][20381] Updated weights for policy 1, policy_version 554103 (0.0007) [2023-12-27 09:28:26,978][20381] Updated weights for policy 1, policy_version 554113 (0.0008) [2023-12-27 09:28:27,033][20381] Updated weights for policy 1, policy_version 554123 (0.0008) [2023-12-27 09:28:27,349][20347] Updated weights for policy 0, policy_version 554376 (0.0006) [2023-12-27 09:28:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 283811840. Throughput: 0: 10565.8, 1: 10303.4. Samples: 283803448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:27,363][19170] Avg episode reward: [(0, '876.570'), (1, '766.280')] [2023-12-27 09:28:27,396][20347] Updated weights for policy 0, policy_version 554386 (0.0007) [2023-12-27 09:28:27,441][20347] Updated weights for policy 0, policy_version 554396 (0.0008) [2023-12-27 09:28:27,705][20381] Updated weights for policy 1, policy_version 554133 (0.0008) [2023-12-27 09:28:27,757][20381] Updated weights for policy 1, policy_version 554143 (0.0007) [2023-12-27 09:28:27,805][20381] Updated weights for policy 1, policy_version 554153 (0.0007) [2023-12-27 09:28:28,049][20347] Updated weights for policy 0, policy_version 554406 (0.0007) [2023-12-27 09:28:28,094][20347] Updated weights for policy 0, policy_version 554416 (0.0005) [2023-12-27 09:28:28,152][20347] Updated weights for policy 0, policy_version 554426 (0.0005) [2023-12-27 09:28:28,467][20381] Updated weights for policy 1, policy_version 554163 (0.0007) [2023-12-27 09:28:28,521][20381] Updated weights for policy 1, policy_version 554173 (0.0008) [2023-12-27 09:28:28,569][20381] Updated weights for policy 1, policy_version 554183 (0.0008) [2023-12-27 09:28:28,870][20347] Updated weights for policy 0, policy_version 554436 (0.0007) [2023-12-27 09:28:28,919][20347] Updated weights for policy 0, policy_version 554446 (0.0008) [2023-12-27 09:28:28,965][20347] Updated weights for policy 0, policy_version 554456 (0.0010) [2023-12-27 09:28:29,164][20381] Updated weights for policy 1, policy_version 554193 (0.0006) [2023-12-27 09:28:29,217][20381] Updated weights for policy 1, policy_version 554203 (0.0005) [2023-12-27 09:28:29,278][20381] Updated weights for policy 1, policy_version 554213 (0.0007) [2023-12-27 09:28:29,333][20381] Updated weights for policy 1, policy_version 554223 (0.0008) [2023-12-27 09:28:29,588][20347] Updated weights for policy 0, policy_version 554466 (0.0008) [2023-12-27 09:28:29,639][20347] Updated weights for policy 0, policy_version 554476 (0.0007) [2023-12-27 09:28:29,692][20347] Updated weights for policy 0, policy_version 554486 (0.0008) [2023-12-27 09:28:29,745][20347] Updated weights for policy 0, policy_version 554496 (0.0008) [2023-12-27 09:28:29,942][20381] Updated weights for policy 1, policy_version 554233 (0.0007) [2023-12-27 09:28:30,004][20381] Updated weights for policy 1, policy_version 554243 (0.0007) [2023-12-27 09:28:30,067][20381] Updated weights for policy 1, policy_version 554253 (0.0008) [2023-12-27 09:28:30,420][20347] Updated weights for policy 0, policy_version 554506 (0.0005) [2023-12-27 09:28:30,474][20347] Updated weights for policy 0, policy_version 554516 (0.0005) [2023-12-27 09:28:30,526][20347] Updated weights for policy 0, policy_version 554526 (0.0006) [2023-12-27 09:28:30,647][20381] Updated weights for policy 1, policy_version 554263 (0.0007) [2023-12-27 09:28:30,699][20381] Updated weights for policy 1, policy_version 554273 (0.0008) [2023-12-27 09:28:30,747][20381] Updated weights for policy 1, policy_version 554283 (0.0009) [2023-12-27 09:28:31,056][20347] Updated weights for policy 0, policy_version 554536 (0.0008) [2023-12-27 09:28:31,113][20347] Updated weights for policy 0, policy_version 554546 (0.0008) [2023-12-27 09:28:31,174][20347] Updated weights for policy 0, policy_version 554556 (0.0008) [2023-12-27 09:28:31,507][20381] Updated weights for policy 1, policy_version 554293 (0.0007) [2023-12-27 09:28:31,562][20381] Updated weights for policy 1, policy_version 554303 (0.0006) [2023-12-27 09:28:31,623][20381] Updated weights for policy 1, policy_version 554313 (0.0007) [2023-12-27 09:28:31,947][20347] Updated weights for policy 0, policy_version 554566 (0.0007) [2023-12-27 09:28:32,009][20347] Updated weights for policy 0, policy_version 554576 (0.0007) [2023-12-27 09:28:32,069][20347] Updated weights for policy 0, policy_version 554586 (0.0007) [2023-12-27 09:28:32,246][20381] Updated weights for policy 1, policy_version 554323 (0.0007) [2023-12-27 09:28:32,306][20381] Updated weights for policy 1, policy_version 554333 (0.0007) [2023-12-27 09:28:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 283918336. Throughput: 0: 10571.5, 1: 10319.4. Samples: 283931120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:32,363][19170] Avg episode reward: [(0, '942.760'), (1, '747.500')] [2023-12-27 09:28:32,370][20381] Updated weights for policy 1, policy_version 554343 (0.0007) [2023-12-27 09:28:32,750][20347] Updated weights for policy 0, policy_version 554596 (0.0007) [2023-12-27 09:28:32,808][20347] Updated weights for policy 0, policy_version 554606 (0.0007) [2023-12-27 09:28:32,862][20347] Updated weights for policy 0, policy_version 554616 (0.0007) [2023-12-27 09:28:32,964][20381] Updated weights for policy 1, policy_version 554353 (0.0006) [2023-12-27 09:28:33,029][20381] Updated weights for policy 1, policy_version 554363 (0.0005) [2023-12-27 09:28:33,041][20292] KL-divergence is very high: 149.7789 [2023-12-27 09:28:33,047][20292] KL-divergence is very high: 311.8171 [2023-12-27 09:28:33,054][20292] KL-divergence is very high: 575.3310 [2023-12-27 09:28:33,084][20381] Updated weights for policy 1, policy_version 554373 (0.0005) [2023-12-27 09:28:33,097][20292] KL-divergence is very high: 179.2240 [2023-12-27 09:28:33,118][20292] KL-divergence is very high: 121.1395 [2023-12-27 09:28:33,123][20292] KL-divergence is very high: 143.1884 [2023-12-27 09:28:33,128][20292] KL-divergence is very high: 199.9322 [2023-12-27 09:28:33,133][20292] KL-divergence is very high: 176.1686 [2023-12-27 09:28:33,139][20292] KL-divergence is very high: 120.5904 [2023-12-27 09:28:33,143][20381] Updated weights for policy 1, policy_version 554383 (0.0005) [2023-12-27 09:28:33,534][20347] Updated weights for policy 0, policy_version 554626 (0.0009) [2023-12-27 09:28:33,581][20292] KL-divergence is very high: 103.7517 [2023-12-27 09:28:33,588][20292] KL-divergence is very high: 336.1445 [2023-12-27 09:28:33,595][20292] KL-divergence is very high: 381.3080 [2023-12-27 09:28:33,600][20347] Updated weights for policy 0, policy_version 554636 (0.0005) [2023-12-27 09:28:33,625][20292] KL-divergence is very high: 120.4619 [2023-12-27 09:28:33,631][20292] KL-divergence is very high: 130.1548 [2023-12-27 09:28:33,635][20381] Updated weights for policy 1, policy_version 554393 (0.0005) [2023-12-27 09:28:33,636][20292] KL-divergence is very high: 212.9588 [2023-12-27 09:28:33,643][20292] KL-divergence is very high: 215.4370 [2023-12-27 09:28:33,659][20347] Updated weights for policy 0, policy_version 554646 (0.0008) [2023-12-27 09:28:33,678][20292] KL-divergence is very high: 125.7355 [2023-12-27 09:28:33,684][20292] KL-divergence is very high: 240.8308 [2023-12-27 09:28:33,691][20292] KL-divergence is very high: 274.9938 [2023-12-27 09:28:33,696][20381] Updated weights for policy 1, policy_version 554403 (0.0005) [2023-12-27 09:28:33,712][20347] Updated weights for policy 0, policy_version 554656 (0.0007) [2023-12-27 09:28:33,721][20292] KL-divergence is very high: 121.7015 [2023-12-27 09:28:33,726][20292] KL-divergence is very high: 136.0359 [2023-12-27 09:28:33,731][20292] KL-divergence is very high: 169.2258 [2023-12-27 09:28:33,735][20292] KL-divergence is very high: 157.8531 [2023-12-27 09:28:33,750][20381] Updated weights for policy 1, policy_version 554413 (0.0007) [2023-12-27 09:28:33,761][20292] KL-divergence is very high: 138.8976 [2023-12-27 09:28:34,342][20347] Updated weights for policy 0, policy_version 554666 (0.0008) [2023-12-27 09:28:34,375][20292] KL-divergence is very high: 162.3370 [2023-12-27 09:28:34,387][20292] KL-divergence is very high: 209.9115 [2023-12-27 09:28:34,402][20347] Updated weights for policy 0, policy_version 554676 (0.0007) [2023-12-27 09:28:34,407][20381] Updated weights for policy 1, policy_version 554423 (0.0008) [2023-12-27 09:28:34,456][20347] Updated weights for policy 0, policy_version 554686 (0.0007) [2023-12-27 09:28:34,473][20381] Updated weights for policy 1, policy_version 554433 (0.0008) [2023-12-27 09:28:34,545][20381] Updated weights for policy 1, policy_version 554443 (0.0008) [2023-12-27 09:28:35,099][20347] Updated weights for policy 0, policy_version 554696 (0.0008) [2023-12-27 09:28:35,121][20381] Updated weights for policy 1, policy_version 554453 (0.0006) [2023-12-27 09:28:35,151][20347] Updated weights for policy 0, policy_version 554706 (0.0007) [2023-12-27 09:28:35,181][20381] Updated weights for policy 1, policy_version 554463 (0.0005) [2023-12-27 09:28:35,208][20347] Updated weights for policy 0, policy_version 554716 (0.0008) [2023-12-27 09:28:35,234][20381] Updated weights for policy 1, policy_version 554473 (0.0007) [2023-12-27 09:28:35,829][20381] Updated weights for policy 1, policy_version 554483 (0.0007) [2023-12-27 09:28:35,877][20347] Updated weights for policy 0, policy_version 554726 (0.0007) [2023-12-27 09:28:35,883][20381] Updated weights for policy 1, policy_version 554493 (0.0006) [2023-12-27 09:28:35,925][20347] Updated weights for policy 0, policy_version 554736 (0.0005) [2023-12-27 09:28:35,938][20381] Updated weights for policy 1, policy_version 554503 (0.0005) [2023-12-27 09:28:35,982][20347] Updated weights for policy 0, policy_version 554746 (0.0007) [2023-12-27 09:28:36,584][20381] Updated weights for policy 1, policy_version 554513 (0.0006) [2023-12-27 09:28:36,648][20381] Updated weights for policy 1, policy_version 554523 (0.0007) [2023-12-27 09:28:36,654][20347] Updated weights for policy 0, policy_version 554756 (0.0008) [2023-12-27 09:28:36,710][20347] Updated weights for policy 0, policy_version 554766 (0.0006) [2023-12-27 09:28:36,711][20381] Updated weights for policy 1, policy_version 554533 (0.0006) [2023-12-27 09:28:36,768][20347] Updated weights for policy 0, policy_version 554776 (0.0007) [2023-12-27 09:28:36,770][20381] Updated weights for policy 1, policy_version 554543 (0.0005) [2023-12-27 09:28:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 284024832. Throughput: 0: 10574.0, 1: 10355.3. Samples: 283996656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:37,363][19170] Avg episode reward: [(0, '971.850'), (1, '457.090')] [2023-12-27 09:28:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000554784_142041088.pth... [2023-12-27 09:28:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000553568_141729792.pth [2023-12-27 09:28:37,392][20381] Updated weights for policy 1, policy_version 554553 (0.0010) [2023-12-27 09:28:37,412][20347] Updated weights for policy 0, policy_version 554786 (0.0006) [2023-12-27 09:28:37,454][20381] Updated weights for policy 1, policy_version 554563 (0.0011) [2023-12-27 09:28:37,470][20347] Updated weights for policy 0, policy_version 554796 (0.0008) [2023-12-27 09:28:37,504][20381] Updated weights for policy 1, policy_version 554573 (0.0010) [2023-12-27 09:28:37,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000554576_141991936.pth... [2023-12-27 09:28:37,522][20347] Updated weights for policy 0, policy_version 554806 (0.0010) [2023-12-27 09:28:37,523][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000553328_141672448.pth [2023-12-27 09:28:37,584][20347] Updated weights for policy 0, policy_version 554816 (0.0010) [2023-12-27 09:28:38,140][20381] Updated weights for policy 1, policy_version 554583 (0.0011) [2023-12-27 09:28:38,185][20381] Updated weights for policy 1, policy_version 554593 (0.0010) [2023-12-27 09:28:38,232][20381] Updated weights for policy 1, policy_version 554603 (0.0010) [2023-12-27 09:28:38,236][20347] Updated weights for policy 0, policy_version 554826 (0.0010) [2023-12-27 09:28:38,295][20347] Updated weights for policy 0, policy_version 554836 (0.0010) [2023-12-27 09:28:38,357][20347] Updated weights for policy 0, policy_version 554846 (0.0010) [2023-12-27 09:28:38,933][20381] Updated weights for policy 1, policy_version 554613 (0.0011) [2023-12-27 09:28:38,991][20381] Updated weights for policy 1, policy_version 554623 (0.0010) [2023-12-27 09:28:39,036][20347] Updated weights for policy 0, policy_version 554856 (0.0010) [2023-12-27 09:28:39,039][20381] Updated weights for policy 1, policy_version 554633 (0.0010) [2023-12-27 09:28:39,094][20347] Updated weights for policy 0, policy_version 554866 (0.0010) [2023-12-27 09:28:39,155][20347] Updated weights for policy 0, policy_version 554876 (0.0010) [2023-12-27 09:28:39,756][20381] Updated weights for policy 1, policy_version 554643 (0.0009) [2023-12-27 09:28:39,805][20381] Updated weights for policy 1, policy_version 554653 (0.0007) [2023-12-27 09:28:39,868][20381] Updated weights for policy 1, policy_version 554663 (0.0007) [2023-12-27 09:28:39,872][20347] Updated weights for policy 0, policy_version 554886 (0.0011) [2023-12-27 09:28:39,934][20347] Updated weights for policy 0, policy_version 554896 (0.0011) [2023-12-27 09:28:40,000][20347] Updated weights for policy 0, policy_version 554906 (0.0008) [2023-12-27 09:28:40,509][20381] Updated weights for policy 1, policy_version 554673 (0.0006) [2023-12-27 09:28:40,565][20381] Updated weights for policy 1, policy_version 554683 (0.0007) [2023-12-27 09:28:40,622][20381] Updated weights for policy 1, policy_version 554693 (0.0008) [2023-12-27 09:28:40,684][20381] Updated weights for policy 1, policy_version 554703 (0.0008) [2023-12-27 09:28:40,705][20347] Updated weights for policy 0, policy_version 554916 (0.0009) [2023-12-27 09:28:40,758][20347] Updated weights for policy 0, policy_version 554926 (0.0009) [2023-12-27 09:28:40,813][20347] Updated weights for policy 0, policy_version 554936 (0.0010) [2023-12-27 09:28:41,377][20381] Updated weights for policy 1, policy_version 554713 (0.0007) [2023-12-27 09:28:41,443][20381] Updated weights for policy 1, policy_version 554723 (0.0008) [2023-12-27 09:28:41,499][20381] Updated weights for policy 1, policy_version 554733 (0.0005) [2023-12-27 09:28:41,530][20347] Updated weights for policy 0, policy_version 554946 (0.0010) [2023-12-27 09:28:41,592][20347] Updated weights for policy 0, policy_version 554956 (0.0008) [2023-12-27 09:28:41,658][20347] Updated weights for policy 0, policy_version 554966 (0.0008) [2023-12-27 09:28:41,726][20347] Updated weights for policy 0, policy_version 554976 (0.0008) [2023-12-27 09:28:42,106][20381] Updated weights for policy 1, policy_version 554743 (0.0007) [2023-12-27 09:28:42,155][20381] Updated weights for policy 1, policy_version 554753 (0.0006) [2023-12-27 09:28:42,217][20381] Updated weights for policy 1, policy_version 554763 (0.0005) [2023-12-27 09:28:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 284131328. Throughput: 0: 10518.1, 1: 10369.0. Samples: 284121180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:42,363][19170] Avg episode reward: [(0, '963.020'), (1, '595.090')] [2023-12-27 09:28:42,425][20347] Updated weights for policy 0, policy_version 554986 (0.0009) [2023-12-27 09:28:42,474][20347] Updated weights for policy 0, policy_version 554996 (0.0010) [2023-12-27 09:28:42,526][20347] Updated weights for policy 0, policy_version 555006 (0.0010) [2023-12-27 09:28:42,865][20381] Updated weights for policy 1, policy_version 554773 (0.0007) [2023-12-27 09:28:42,920][20381] Updated weights for policy 1, policy_version 554783 (0.0007) [2023-12-27 09:28:42,978][20381] Updated weights for policy 1, policy_version 554793 (0.0008) [2023-12-27 09:28:43,205][20347] Updated weights for policy 0, policy_version 555016 (0.0010) [2023-12-27 09:28:43,256][20347] Updated weights for policy 0, policy_version 555026 (0.0010) [2023-12-27 09:28:43,311][20347] Updated weights for policy 0, policy_version 555036 (0.0010) [2023-12-27 09:28:43,640][20381] Updated weights for policy 1, policy_version 554803 (0.0008) [2023-12-27 09:28:43,702][20381] Updated weights for policy 1, policy_version 554813 (0.0007) [2023-12-27 09:28:43,759][20381] Updated weights for policy 1, policy_version 554823 (0.0007) [2023-12-27 09:28:43,999][20347] Updated weights for policy 0, policy_version 555046 (0.0007) [2023-12-27 09:28:44,048][20347] Updated weights for policy 0, policy_version 555056 (0.0005) [2023-12-27 09:28:44,098][20347] Updated weights for policy 0, policy_version 555066 (0.0005) [2023-12-27 09:28:44,473][20381] Updated weights for policy 1, policy_version 554833 (0.0006) [2023-12-27 09:28:44,529][20381] Updated weights for policy 1, policy_version 554843 (0.0007) [2023-12-27 09:28:44,578][20381] Updated weights for policy 1, policy_version 554853 (0.0008) [2023-12-27 09:28:44,624][20381] Updated weights for policy 1, policy_version 554863 (0.0007) [2023-12-27 09:28:44,724][20347] Updated weights for policy 0, policy_version 555076 (0.0011) [2023-12-27 09:28:44,782][20347] Updated weights for policy 0, policy_version 555086 (0.0009) [2023-12-27 09:28:44,839][20347] Updated weights for policy 0, policy_version 555096 (0.0010) [2023-12-27 09:28:45,378][20381] Updated weights for policy 1, policy_version 554873 (0.0007) [2023-12-27 09:28:45,432][20381] Updated weights for policy 1, policy_version 554883 (0.0007) [2023-12-27 09:28:45,487][20381] Updated weights for policy 1, policy_version 554893 (0.0007) [2023-12-27 09:28:45,529][20347] Updated weights for policy 0, policy_version 555106 (0.0009) [2023-12-27 09:28:45,580][20347] Updated weights for policy 0, policy_version 555116 (0.0006) [2023-12-27 09:28:45,634][20347] Updated weights for policy 0, policy_version 555126 (0.0008) [2023-12-27 09:28:45,691][20347] Updated weights for policy 0, policy_version 555136 (0.0008) [2023-12-27 09:28:46,196][20381] Updated weights for policy 1, policy_version 554903 (0.0008) [2023-12-27 09:28:46,251][20381] Updated weights for policy 1, policy_version 554913 (0.0007) [2023-12-27 09:28:46,317][20381] Updated weights for policy 1, policy_version 554923 (0.0008) [2023-12-27 09:28:46,330][20347] Updated weights for policy 0, policy_version 555146 (0.0006) [2023-12-27 09:28:46,381][20347] Updated weights for policy 0, policy_version 555156 (0.0007) [2023-12-27 09:28:46,439][20347] Updated weights for policy 0, policy_version 555166 (0.0008) [2023-12-27 09:28:46,909][20381] Updated weights for policy 1, policy_version 554933 (0.0005) [2023-12-27 09:28:46,963][20381] Updated weights for policy 1, policy_version 554943 (0.0005) [2023-12-27 09:28:47,015][20381] Updated weights for policy 1, policy_version 554953 (0.0005) [2023-12-27 09:28:47,205][20347] Updated weights for policy 0, policy_version 555176 (0.0009) [2023-12-27 09:28:47,265][20347] Updated weights for policy 0, policy_version 555186 (0.0008) [2023-12-27 09:28:47,324][20347] Updated weights for policy 0, policy_version 555196 (0.0008) [2023-12-27 09:28:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 284237824. Throughput: 0: 10498.1, 1: 10412.9. Samples: 284245168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:47,363][19170] Avg episode reward: [(0, '877.170'), (1, '646.150')] [2023-12-27 09:28:47,590][20381] Updated weights for policy 1, policy_version 554963 (0.0006) [2023-12-27 09:28:47,642][20381] Updated weights for policy 1, policy_version 554973 (0.0007) [2023-12-27 09:28:47,694][20381] Updated weights for policy 1, policy_version 554983 (0.0007) [2023-12-27 09:28:47,974][20347] Updated weights for policy 0, policy_version 555206 (0.0006) [2023-12-27 09:28:48,031][20347] Updated weights for policy 0, policy_version 555216 (0.0007) [2023-12-27 09:28:48,076][20347] Updated weights for policy 0, policy_version 555226 (0.0008) [2023-12-27 09:28:48,393][20381] Updated weights for policy 1, policy_version 554993 (0.0008) [2023-12-27 09:28:48,452][20381] Updated weights for policy 1, policy_version 555003 (0.0007) [2023-12-27 09:28:48,503][20381] Updated weights for policy 1, policy_version 555013 (0.0007) [2023-12-27 09:28:48,558][20381] Updated weights for policy 1, policy_version 555023 (0.0008) [2023-12-27 09:28:48,762][20347] Updated weights for policy 0, policy_version 555236 (0.0008) [2023-12-27 09:28:48,808][20347] Updated weights for policy 0, policy_version 555246 (0.0008) [2023-12-27 09:28:48,857][20347] Updated weights for policy 0, policy_version 555256 (0.0007) [2023-12-27 09:28:49,186][20381] Updated weights for policy 1, policy_version 555033 (0.0005) [2023-12-27 09:28:49,247][20381] Updated weights for policy 1, policy_version 555043 (0.0007) [2023-12-27 09:28:49,307][20381] Updated weights for policy 1, policy_version 555053 (0.0007) [2023-12-27 09:28:49,635][20347] Updated weights for policy 0, policy_version 555266 (0.0007) [2023-12-27 09:28:49,685][20347] Updated weights for policy 0, policy_version 555276 (0.0008) [2023-12-27 09:28:49,734][20347] Updated weights for policy 0, policy_version 555286 (0.0008) [2023-12-27 09:28:49,783][20347] Updated weights for policy 0, policy_version 555296 (0.0007) [2023-12-27 09:28:50,000][20381] Updated weights for policy 1, policy_version 555063 (0.0008) [2023-12-27 09:28:50,060][20381] Updated weights for policy 1, policy_version 555073 (0.0009) [2023-12-27 09:28:50,125][20381] Updated weights for policy 1, policy_version 555083 (0.0008) [2023-12-27 09:28:50,469][20347] Updated weights for policy 0, policy_version 555306 (0.0006) [2023-12-27 09:28:50,530][20347] Updated weights for policy 0, policy_version 555316 (0.0005) [2023-12-27 09:28:50,597][20347] Updated weights for policy 0, policy_version 555326 (0.0005) [2023-12-27 09:28:50,841][20381] Updated weights for policy 1, policy_version 555093 (0.0007) [2023-12-27 09:28:50,894][20381] Updated weights for policy 1, policy_version 555103 (0.0008) [2023-12-27 09:28:50,947][20381] Updated weights for policy 1, policy_version 555113 (0.0008) [2023-12-27 09:28:51,196][20347] Updated weights for policy 0, policy_version 555336 (0.0008) [2023-12-27 09:28:51,252][20347] Updated weights for policy 0, policy_version 555346 (0.0007) [2023-12-27 09:28:51,308][20347] Updated weights for policy 0, policy_version 555356 (0.0007) [2023-12-27 09:28:51,676][20381] Updated weights for policy 1, policy_version 555123 (0.0007) [2023-12-27 09:28:51,744][20381] Updated weights for policy 1, policy_version 555133 (0.0008) [2023-12-27 09:28:51,803][20381] Updated weights for policy 1, policy_version 555143 (0.0006) [2023-12-27 09:28:51,962][20347] Updated weights for policy 0, policy_version 555366 (0.0007) [2023-12-27 09:28:52,013][20347] Updated weights for policy 0, policy_version 555376 (0.0007) [2023-12-27 09:28:52,070][20347] Updated weights for policy 0, policy_version 555386 (0.0007) [2023-12-27 09:28:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 284336128. Throughput: 0: 10406.2, 1: 10406.3. Samples: 284305196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:52,363][19170] Avg episode reward: [(0, '933.750'), (1, '528.460')] [2023-12-27 09:28:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000555152_142139392.pth... [2023-12-27 09:28:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000555392_142196736.pth... [2023-12-27 09:28:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000553936_141828096.pth [2023-12-27 09:28:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000554176_141885440.pth [2023-12-27 09:28:52,476][20381] Updated weights for policy 1, policy_version 555153 (0.0006) [2023-12-27 09:28:52,533][20381] Updated weights for policy 1, policy_version 555163 (0.0005) [2023-12-27 09:28:52,595][20381] Updated weights for policy 1, policy_version 555173 (0.0007) [2023-12-27 09:28:52,650][20381] Updated weights for policy 1, policy_version 555183 (0.0005) [2023-12-27 09:28:52,812][20347] Updated weights for policy 0, policy_version 555396 (0.0008) [2023-12-27 09:28:52,873][20347] Updated weights for policy 0, policy_version 555406 (0.0008) [2023-12-27 09:28:52,929][20347] Updated weights for policy 0, policy_version 555416 (0.0007) [2023-12-27 09:28:53,338][20381] Updated weights for policy 1, policy_version 555193 (0.0010) [2023-12-27 09:28:53,387][20381] Updated weights for policy 1, policy_version 555203 (0.0010) [2023-12-27 09:28:53,435][20381] Updated weights for policy 1, policy_version 555213 (0.0010) [2023-12-27 09:28:53,580][20347] Updated weights for policy 0, policy_version 555426 (0.0007) [2023-12-27 09:28:53,634][20347] Updated weights for policy 0, policy_version 555436 (0.0007) [2023-12-27 09:28:53,697][20347] Updated weights for policy 0, policy_version 555446 (0.0008) [2023-12-27 09:28:53,748][20347] Updated weights for policy 0, policy_version 555456 (0.0007) [2023-12-27 09:28:54,146][20381] Updated weights for policy 1, policy_version 555223 (0.0007) [2023-12-27 09:28:54,206][20381] Updated weights for policy 1, policy_version 555233 (0.0006) [2023-12-27 09:28:54,258][20381] Updated weights for policy 1, policy_version 555243 (0.0005) [2023-12-27 09:28:54,473][20347] Updated weights for policy 0, policy_version 555466 (0.0008) [2023-12-27 09:28:54,527][20347] Updated weights for policy 0, policy_version 555476 (0.0007) [2023-12-27 09:28:54,587][20347] Updated weights for policy 0, policy_version 555486 (0.0007) [2023-12-27 09:28:54,855][20381] Updated weights for policy 1, policy_version 555253 (0.0008) [2023-12-27 09:28:54,902][20381] Updated weights for policy 1, policy_version 555263 (0.0010) [2023-12-27 09:28:54,958][20381] Updated weights for policy 1, policy_version 555273 (0.0010) [2023-12-27 09:28:55,292][20347] Updated weights for policy 0, policy_version 555496 (0.0008) [2023-12-27 09:28:55,351][20347] Updated weights for policy 0, policy_version 555506 (0.0007) [2023-12-27 09:28:55,416][20347] Updated weights for policy 0, policy_version 555516 (0.0007) [2023-12-27 09:28:55,584][20381] Updated weights for policy 1, policy_version 555283 (0.0005) [2023-12-27 09:28:55,640][20381] Updated weights for policy 1, policy_version 555293 (0.0005) [2023-12-27 09:28:55,699][20381] Updated weights for policy 1, policy_version 555303 (0.0007) [2023-12-27 09:28:56,092][20347] Updated weights for policy 0, policy_version 555526 (0.0007) [2023-12-27 09:28:56,145][20347] Updated weights for policy 0, policy_version 555536 (0.0005) [2023-12-27 09:28:56,206][20347] Updated weights for policy 0, policy_version 555546 (0.0007) [2023-12-27 09:28:56,266][20381] Updated weights for policy 1, policy_version 555313 (0.0007) [2023-12-27 09:28:56,323][20381] Updated weights for policy 1, policy_version 555323 (0.0005) [2023-12-27 09:28:56,388][20381] Updated weights for policy 1, policy_version 555333 (0.0005) [2023-12-27 09:28:56,453][20381] Updated weights for policy 1, policy_version 555343 (0.0009) [2023-12-27 09:28:56,792][20347] Updated weights for policy 0, policy_version 555556 (0.0006) [2023-12-27 09:28:56,838][20347] Updated weights for policy 0, policy_version 555566 (0.0005) [2023-12-27 09:28:56,881][20347] Updated weights for policy 0, policy_version 555576 (0.0005) [2023-12-27 09:28:57,045][20381] Updated weights for policy 1, policy_version 555353 (0.0009) [2023-12-27 09:28:57,108][20381] Updated weights for policy 1, policy_version 555363 (0.0007) [2023-12-27 09:28:57,176][20381] Updated weights for policy 1, policy_version 555373 (0.0009) [2023-12-27 09:28:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 284442624. Throughput: 0: 10417.0, 1: 10436.7. Samples: 284429776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:28:57,363][19170] Avg episode reward: [(0, '952.380'), (1, '634.610')] [2023-12-27 09:28:57,475][20347] Updated weights for policy 0, policy_version 555586 (0.0006) [2023-12-27 09:28:57,535][20347] Updated weights for policy 0, policy_version 555596 (0.0008) [2023-12-27 09:28:57,585][20347] Updated weights for policy 0, policy_version 555606 (0.0006) [2023-12-27 09:28:57,637][20347] Updated weights for policy 0, policy_version 555616 (0.0005) [2023-12-27 09:28:57,815][20381] Updated weights for policy 1, policy_version 555383 (0.0006) [2023-12-27 09:28:57,860][20381] Updated weights for policy 1, policy_version 555393 (0.0005) [2023-12-27 09:28:57,905][20381] Updated weights for policy 1, policy_version 555403 (0.0005) [2023-12-27 09:28:58,277][20347] Updated weights for policy 0, policy_version 555626 (0.0010) [2023-12-27 09:28:58,336][20347] Updated weights for policy 0, policy_version 555636 (0.0008) [2023-12-27 09:28:58,394][20347] Updated weights for policy 0, policy_version 555646 (0.0008) [2023-12-27 09:28:58,607][20381] Updated weights for policy 1, policy_version 555413 (0.0006) [2023-12-27 09:28:58,662][20381] Updated weights for policy 1, policy_version 555423 (0.0008) [2023-12-27 09:28:58,717][20381] Updated weights for policy 1, policy_version 555433 (0.0007) [2023-12-27 09:28:59,182][20347] Updated weights for policy 0, policy_version 555656 (0.0007) [2023-12-27 09:28:59,250][20347] Updated weights for policy 0, policy_version 555666 (0.0007) [2023-12-27 09:28:59,306][20347] Updated weights for policy 0, policy_version 555676 (0.0007) [2023-12-27 09:28:59,435][20381] Updated weights for policy 1, policy_version 555443 (0.0008) [2023-12-27 09:28:59,493][20381] Updated weights for policy 1, policy_version 555453 (0.0008) [2023-12-27 09:28:59,545][20381] Updated weights for policy 1, policy_version 555463 (0.0008) [2023-12-27 09:28:59,999][20347] Updated weights for policy 0, policy_version 555686 (0.0008) [2023-12-27 09:29:00,062][20347] Updated weights for policy 0, policy_version 555696 (0.0008) [2023-12-27 09:29:00,123][20347] Updated weights for policy 0, policy_version 555706 (0.0007) [2023-12-27 09:29:00,183][20381] Updated weights for policy 1, policy_version 555473 (0.0007) [2023-12-27 09:29:00,244][20381] Updated weights for policy 1, policy_version 555483 (0.0007) [2023-12-27 09:29:00,304][20381] Updated weights for policy 1, policy_version 555493 (0.0007) [2023-12-27 09:29:00,353][20381] Updated weights for policy 1, policy_version 555503 (0.0008) [2023-12-27 09:29:00,743][20347] Updated weights for policy 0, policy_version 555716 (0.0007) [2023-12-27 09:29:00,813][20347] Updated weights for policy 0, policy_version 555726 (0.0009) [2023-12-27 09:29:00,874][20347] Updated weights for policy 0, policy_version 555736 (0.0008) [2023-12-27 09:29:01,073][20381] Updated weights for policy 1, policy_version 555513 (0.0008) [2023-12-27 09:29:01,132][20381] Updated weights for policy 1, policy_version 555523 (0.0008) [2023-12-27 09:29:01,184][20381] Updated weights for policy 1, policy_version 555533 (0.0008) [2023-12-27 09:29:01,549][20347] Updated weights for policy 0, policy_version 555746 (0.0009) [2023-12-27 09:29:01,605][20347] Updated weights for policy 0, policy_version 555756 (0.0008) [2023-12-27 09:29:01,662][20347] Updated weights for policy 0, policy_version 555766 (0.0008) [2023-12-27 09:29:01,712][20347] Updated weights for policy 0, policy_version 555776 (0.0007) [2023-12-27 09:29:01,893][20381] Updated weights for policy 1, policy_version 555543 (0.0008) [2023-12-27 09:29:01,947][20381] Updated weights for policy 1, policy_version 555553 (0.0006) [2023-12-27 09:29:02,008][20381] Updated weights for policy 1, policy_version 555563 (0.0007) [2023-12-27 09:29:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 284540928. Throughput: 0: 10373.9, 1: 10452.4. Samples: 284552676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:29:02,363][19170] Avg episode reward: [(0, '981.000'), (1, '597.190')] [2023-12-27 09:29:02,380][20347] Updated weights for policy 0, policy_version 555786 (0.0010) [2023-12-27 09:29:02,444][20347] Updated weights for policy 0, policy_version 555796 (0.0010) [2023-12-27 09:29:02,504][20347] Updated weights for policy 0, policy_version 555806 (0.0009) [2023-12-27 09:29:02,712][20381] Updated weights for policy 1, policy_version 555573 (0.0008) [2023-12-27 09:29:02,757][20381] Updated weights for policy 1, policy_version 555583 (0.0007) [2023-12-27 09:29:02,816][20381] Updated weights for policy 1, policy_version 555593 (0.0007) [2023-12-27 09:29:03,126][20347] Updated weights for policy 0, policy_version 555816 (0.0007) [2023-12-27 09:29:03,184][20347] Updated weights for policy 0, policy_version 555826 (0.0007) [2023-12-27 09:29:03,240][20347] Updated weights for policy 0, policy_version 555836 (0.0007) [2023-12-27 09:29:03,538][20381] Updated weights for policy 1, policy_version 555603 (0.0007) [2023-12-27 09:29:03,593][20381] Updated weights for policy 1, policy_version 555613 (0.0005) [2023-12-27 09:29:03,648][20381] Updated weights for policy 1, policy_version 555623 (0.0005) [2023-12-27 09:29:03,881][20347] Updated weights for policy 0, policy_version 555846 (0.0007) [2023-12-27 09:29:03,932][20347] Updated weights for policy 0, policy_version 555856 (0.0008) [2023-12-27 09:29:03,986][20347] Updated weights for policy 0, policy_version 555866 (0.0008) [2023-12-27 09:29:04,221][20381] Updated weights for policy 1, policy_version 555633 (0.0005) [2023-12-27 09:29:04,276][20381] Updated weights for policy 1, policy_version 555643 (0.0009) [2023-12-27 09:29:04,335][20381] Updated weights for policy 1, policy_version 555653 (0.0008) [2023-12-27 09:29:04,393][20381] Updated weights for policy 1, policy_version 555663 (0.0008) [2023-12-27 09:29:04,610][20347] Updated weights for policy 0, policy_version 555876 (0.0007) [2023-12-27 09:29:04,667][20347] Updated weights for policy 0, policy_version 555886 (0.0007) [2023-12-27 09:29:04,722][20347] Updated weights for policy 0, policy_version 555896 (0.0005) [2023-12-27 09:29:05,068][20381] Updated weights for policy 1, policy_version 555673 (0.0010) [2023-12-27 09:29:05,122][20381] Updated weights for policy 1, policy_version 555683 (0.0010) [2023-12-27 09:29:05,181][20381] Updated weights for policy 1, policy_version 555693 (0.0006) [2023-12-27 09:29:05,334][20347] Updated weights for policy 0, policy_version 555906 (0.0006) [2023-12-27 09:29:05,382][20347] Updated weights for policy 0, policy_version 555916 (0.0010) [2023-12-27 09:29:05,430][20347] Updated weights for policy 0, policy_version 555926 (0.0010) [2023-12-27 09:29:05,485][20347] Updated weights for policy 0, policy_version 555936 (0.0010) [2023-12-27 09:29:05,723][20381] Updated weights for policy 1, policy_version 555703 (0.0006) [2023-12-27 09:29:05,771][20381] Updated weights for policy 1, policy_version 555713 (0.0005) [2023-12-27 09:29:05,829][20381] Updated weights for policy 1, policy_version 555723 (0.0005) [2023-12-27 09:29:06,077][20347] Updated weights for policy 0, policy_version 555946 (0.0007) [2023-12-27 09:29:06,139][20347] Updated weights for policy 0, policy_version 555956 (0.0009) [2023-12-27 09:29:06,198][20347] Updated weights for policy 0, policy_version 555966 (0.0008) [2023-12-27 09:29:06,393][20381] Updated weights for policy 1, policy_version 555733 (0.0005) [2023-12-27 09:29:06,447][20381] Updated weights for policy 1, policy_version 555743 (0.0008) [2023-12-27 09:29:06,503][20381] Updated weights for policy 1, policy_version 555753 (0.0008) [2023-12-27 09:29:06,829][20347] Updated weights for policy 0, policy_version 555976 (0.0006) [2023-12-27 09:29:06,878][20347] Updated weights for policy 0, policy_version 555986 (0.0005) [2023-12-27 09:29:06,933][20347] Updated weights for policy 0, policy_version 555996 (0.0005) [2023-12-27 09:29:07,136][20381] Updated weights for policy 1, policy_version 555763 (0.0007) [2023-12-27 09:29:07,183][20381] Updated weights for policy 1, policy_version 555773 (0.0008) [2023-12-27 09:29:07,231][20381] Updated weights for policy 1, policy_version 555783 (0.0007) [2023-12-27 09:29:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 284655616. Throughput: 0: 10387.3, 1: 10485.9. Samples: 284617268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:29:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '500.630')] [2023-12-27 09:29:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000555792_142303232.pth... [2023-12-27 09:29:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000556000_142352384.pth... [2023-12-27 09:29:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000554576_141991936.pth [2023-12-27 09:29:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000554784_142041088.pth [2023-12-27 09:29:07,593][20347] Updated weights for policy 0, policy_version 556006 (0.0008) [2023-12-27 09:29:07,649][20347] Updated weights for policy 0, policy_version 556016 (0.0008) [2023-12-27 09:29:07,707][20347] Updated weights for policy 0, policy_version 556026 (0.0006) [2023-12-27 09:29:07,955][20381] Updated weights for policy 1, policy_version 555793 (0.0008) [2023-12-27 09:29:08,014][20381] Updated weights for policy 1, policy_version 555803 (0.0008) [2023-12-27 09:29:08,068][20381] Updated weights for policy 1, policy_version 555813 (0.0008) [2023-12-27 09:29:08,119][20381] Updated weights for policy 1, policy_version 555823 (0.0008) [2023-12-27 09:29:08,355][20347] Updated weights for policy 0, policy_version 556036 (0.0008) [2023-12-27 09:29:08,411][20347] Updated weights for policy 0, policy_version 556046 (0.0007) [2023-12-27 09:29:08,467][20347] Updated weights for policy 0, policy_version 556056 (0.0007) [2023-12-27 09:29:08,838][20381] Updated weights for policy 1, policy_version 555833 (0.0008) [2023-12-27 09:29:08,901][20381] Updated weights for policy 1, policy_version 555843 (0.0008) [2023-12-27 09:29:08,951][20381] Updated weights for policy 1, policy_version 555853 (0.0007) [2023-12-27 09:29:09,148][20347] Updated weights for policy 0, policy_version 556066 (0.0008) [2023-12-27 09:29:09,205][20347] Updated weights for policy 0, policy_version 556076 (0.0007) [2023-12-27 09:29:09,261][20347] Updated weights for policy 0, policy_version 556086 (0.0009) [2023-12-27 09:29:09,307][20347] Updated weights for policy 0, policy_version 556096 (0.0006) [2023-12-27 09:29:09,717][20381] Updated weights for policy 1, policy_version 555863 (0.0007) [2023-12-27 09:29:09,776][20381] Updated weights for policy 1, policy_version 555873 (0.0008) [2023-12-27 09:29:09,832][20381] Updated weights for policy 1, policy_version 555883 (0.0008) [2023-12-27 09:29:10,073][20347] Updated weights for policy 0, policy_version 556106 (0.0008) [2023-12-27 09:29:10,130][20347] Updated weights for policy 0, policy_version 556116 (0.0007) [2023-12-27 09:29:10,197][20347] Updated weights for policy 0, policy_version 556126 (0.0008) [2023-12-27 09:29:10,563][20381] Updated weights for policy 1, policy_version 555893 (0.0007) [2023-12-27 09:29:10,621][20381] Updated weights for policy 1, policy_version 555903 (0.0007) [2023-12-27 09:29:10,670][20381] Updated weights for policy 1, policy_version 555913 (0.0008) [2023-12-27 09:29:10,912][20347] Updated weights for policy 0, policy_version 556136 (0.0008) [2023-12-27 09:29:10,974][20347] Updated weights for policy 0, policy_version 556146 (0.0008) [2023-12-27 09:29:11,031][20347] Updated weights for policy 0, policy_version 556156 (0.0008) [2023-12-27 09:29:11,347][20381] Updated weights for policy 1, policy_version 555923 (0.0008) [2023-12-27 09:29:11,417][20381] Updated weights for policy 1, policy_version 555933 (0.0008) [2023-12-27 09:29:11,485][20381] Updated weights for policy 1, policy_version 555943 (0.0008) [2023-12-27 09:29:11,758][20347] Updated weights for policy 0, policy_version 556166 (0.0008) [2023-12-27 09:29:11,808][20347] Updated weights for policy 0, policy_version 556176 (0.0007) [2023-12-27 09:29:11,863][20347] Updated weights for policy 0, policy_version 556186 (0.0006) [2023-12-27 09:29:12,168][20381] Updated weights for policy 1, policy_version 555953 (0.0008) [2023-12-27 09:29:12,230][20381] Updated weights for policy 1, policy_version 555963 (0.0008) [2023-12-27 09:29:12,286][20381] Updated weights for policy 1, policy_version 555973 (0.0009) [2023-12-27 09:29:12,348][20381] Updated weights for policy 1, policy_version 555983 (0.0008) [2023-12-27 09:29:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 284753920. Throughput: 0: 10343.6, 1: 10498.2. Samples: 284741332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:29:12,364][19170] Avg episode reward: [(0, '1000.000'), (1, '614.480')] [2023-12-27 09:29:12,479][20347] Updated weights for policy 0, policy_version 556196 (0.0006) [2023-12-27 09:29:12,532][20347] Updated weights for policy 0, policy_version 556206 (0.0007) [2023-12-27 09:29:12,590][20347] Updated weights for policy 0, policy_version 556216 (0.0008) [2023-12-27 09:29:13,057][20381] Updated weights for policy 1, policy_version 555993 (0.0010) [2023-12-27 09:29:13,125][20381] Updated weights for policy 1, policy_version 556003 (0.0009) [2023-12-27 09:29:13,186][20347] Updated weights for policy 0, policy_version 556226 (0.0007) [2023-12-27 09:29:13,189][20381] Updated weights for policy 1, policy_version 556013 (0.0007) [2023-12-27 09:29:13,243][20347] Updated weights for policy 0, policy_version 556236 (0.0006) [2023-12-27 09:29:13,296][20347] Updated weights for policy 0, policy_version 556246 (0.0007) [2023-12-27 09:29:13,350][20347] Updated weights for policy 0, policy_version 556256 (0.0006) [2023-12-27 09:29:13,798][20381] Updated weights for policy 1, policy_version 556023 (0.0007) [2023-12-27 09:29:13,864][20381] Updated weights for policy 1, policy_version 556033 (0.0005) [2023-12-27 09:29:13,926][20381] Updated weights for policy 1, policy_version 556043 (0.0006) [2023-12-27 09:29:13,941][20347] Updated weights for policy 0, policy_version 556266 (0.0007) [2023-12-27 09:29:13,987][20347] Updated weights for policy 0, policy_version 556276 (0.0008) [2023-12-27 09:29:14,038][20347] Updated weights for policy 0, policy_version 556286 (0.0008) [2023-12-27 09:29:14,545][20381] Updated weights for policy 1, policy_version 556053 (0.0008) [2023-12-27 09:29:14,603][20381] Updated weights for policy 1, policy_version 556063 (0.0011) [2023-12-27 09:29:14,651][20381] Updated weights for policy 1, policy_version 556073 (0.0010) [2023-12-27 09:29:14,724][20347] Updated weights for policy 0, policy_version 556296 (0.0007) [2023-12-27 09:29:14,783][20347] Updated weights for policy 0, policy_version 556306 (0.0008) [2023-12-27 09:29:14,834][20347] Updated weights for policy 0, policy_version 556316 (0.0007) [2023-12-27 09:29:15,346][20381] Updated weights for policy 1, policy_version 556083 (0.0010) [2023-12-27 09:29:15,401][20381] Updated weights for policy 1, policy_version 556093 (0.0011) [2023-12-27 09:29:15,460][20381] Updated weights for policy 1, policy_version 556103 (0.0011) [2023-12-27 09:29:15,522][20347] Updated weights for policy 0, policy_version 556326 (0.0007) [2023-12-27 09:29:15,576][20347] Updated weights for policy 0, policy_version 556336 (0.0007) [2023-12-27 09:29:15,630][20347] Updated weights for policy 0, policy_version 556346 (0.0008) [2023-12-27 09:29:16,074][20381] Updated weights for policy 1, policy_version 556113 (0.0011) [2023-12-27 09:29:16,129][20381] Updated weights for policy 1, policy_version 556123 (0.0011) [2023-12-27 09:29:16,184][20381] Updated weights for policy 1, policy_version 556133 (0.0011) [2023-12-27 09:29:16,237][20381] Updated weights for policy 1, policy_version 556143 (0.0011) [2023-12-27 09:29:16,302][20347] Updated weights for policy 0, policy_version 556356 (0.0008) [2023-12-27 09:29:16,349][20347] Updated weights for policy 0, policy_version 556366 (0.0007) [2023-12-27 09:29:16,405][20347] Updated weights for policy 0, policy_version 556376 (0.0005) [2023-12-27 09:29:16,943][20381] Updated weights for policy 1, policy_version 556153 (0.0008) [2023-12-27 09:29:16,997][20347] Updated weights for policy 0, policy_version 556386 (0.0005) [2023-12-27 09:29:17,002][20381] Updated weights for policy 1, policy_version 556163 (0.0005) [2023-12-27 09:29:17,046][20347] Updated weights for policy 0, policy_version 556396 (0.0006) [2023-12-27 09:29:17,063][20381] Updated weights for policy 1, policy_version 556173 (0.0008) [2023-12-27 09:29:17,093][20347] Updated weights for policy 0, policy_version 556406 (0.0006) [2023-12-27 09:29:17,141][20347] Updated weights for policy 0, policy_version 556416 (0.0008) [2023-12-27 09:29:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 284860416. Throughput: 0: 10357.8, 1: 10447.7. Samples: 284867364. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:17,363][19170] Avg episode reward: [(0, '980.770'), (1, '784.030')] [2023-12-27 09:29:17,740][20381] Updated weights for policy 1, policy_version 556183 (0.0009) [2023-12-27 09:29:17,757][20347] Updated weights for policy 0, policy_version 556426 (0.0007) [2023-12-27 09:29:17,788][20381] Updated weights for policy 1, policy_version 556193 (0.0010) [2023-12-27 09:29:17,812][20347] Updated weights for policy 0, policy_version 556436 (0.0005) [2023-12-27 09:29:17,849][20381] Updated weights for policy 1, policy_version 556203 (0.0010) [2023-12-27 09:29:17,869][20347] Updated weights for policy 0, policy_version 556446 (0.0006) [2023-12-27 09:29:18,488][20347] Updated weights for policy 0, policy_version 556456 (0.0008) [2023-12-27 09:29:18,531][20381] Updated weights for policy 1, policy_version 556213 (0.0008) [2023-12-27 09:29:18,532][20347] Updated weights for policy 0, policy_version 556466 (0.0008) [2023-12-27 09:29:18,586][20381] Updated weights for policy 1, policy_version 556223 (0.0006) [2023-12-27 09:29:18,588][20347] Updated weights for policy 0, policy_version 556476 (0.0008) [2023-12-27 09:29:18,648][20381] Updated weights for policy 1, policy_version 556233 (0.0006) [2023-12-27 09:29:19,272][20381] Updated weights for policy 1, policy_version 556243 (0.0007) [2023-12-27 09:29:19,320][20347] Updated weights for policy 0, policy_version 556486 (0.0007) [2023-12-27 09:29:19,330][20381] Updated weights for policy 1, policy_version 556253 (0.0008) [2023-12-27 09:29:19,382][20347] Updated weights for policy 0, policy_version 556496 (0.0006) [2023-12-27 09:29:19,392][20381] Updated weights for policy 1, policy_version 556263 (0.0007) [2023-12-27 09:29:19,431][20347] Updated weights for policy 0, policy_version 556506 (0.0005) [2023-12-27 09:29:20,021][20381] Updated weights for policy 1, policy_version 556273 (0.0007) [2023-12-27 09:29:20,078][20381] Updated weights for policy 1, policy_version 556283 (0.0008) [2023-12-27 09:29:20,131][20381] Updated weights for policy 1, policy_version 556293 (0.0007) [2023-12-27 09:29:20,173][20347] Updated weights for policy 0, policy_version 556516 (0.0008) [2023-12-27 09:29:20,187][20381] Updated weights for policy 1, policy_version 556303 (0.0006) [2023-12-27 09:29:20,226][20347] Updated weights for policy 0, policy_version 556526 (0.0007) [2023-12-27 09:29:20,289][20347] Updated weights for policy 0, policy_version 556536 (0.0007) [2023-12-27 09:29:20,903][20381] Updated weights for policy 1, policy_version 556313 (0.0008) [2023-12-27 09:29:20,970][20381] Updated weights for policy 1, policy_version 556323 (0.0007) [2023-12-27 09:29:21,026][20347] Updated weights for policy 0, policy_version 556546 (0.0008) [2023-12-27 09:29:21,038][20381] Updated weights for policy 1, policy_version 556333 (0.0007) [2023-12-27 09:29:21,088][20347] Updated weights for policy 0, policy_version 556556 (0.0007) [2023-12-27 09:29:21,152][20347] Updated weights for policy 0, policy_version 556566 (0.0006) [2023-12-27 09:29:21,219][20347] Updated weights for policy 0, policy_version 556576 (0.0006) [2023-12-27 09:29:21,758][20381] Updated weights for policy 1, policy_version 556343 (0.0008) [2023-12-27 09:29:21,823][20381] Updated weights for policy 1, policy_version 556353 (0.0006) [2023-12-27 09:29:21,886][20381] Updated weights for policy 1, policy_version 556363 (0.0008) [2023-12-27 09:29:21,908][20347] Updated weights for policy 0, policy_version 556586 (0.0006) [2023-12-27 09:29:21,966][20347] Updated weights for policy 0, policy_version 556596 (0.0007) [2023-12-27 09:29:22,028][20347] Updated weights for policy 0, policy_version 556606 (0.0007) [2023-12-27 09:29:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 284958720. Throughput: 0: 10352.1, 1: 10380.6. Samples: 284929628. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:22,363][19170] Avg episode reward: [(0, '980.810'), (1, '748.770')] [2023-12-27 09:29:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000556368_142450688.pth... [2023-12-27 09:29:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000556608_142508032.pth... [2023-12-27 09:29:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000555152_142139392.pth [2023-12-27 09:29:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000555392_142196736.pth [2023-12-27 09:29:22,519][20381] Updated weights for policy 1, policy_version 556373 (0.0007) [2023-12-27 09:29:22,576][20381] Updated weights for policy 1, policy_version 556383 (0.0006) [2023-12-27 09:29:22,632][20381] Updated weights for policy 1, policy_version 556393 (0.0005) [2023-12-27 09:29:22,727][20347] Updated weights for policy 0, policy_version 556616 (0.0009) [2023-12-27 09:29:22,794][20347] Updated weights for policy 0, policy_version 556626 (0.0008) [2023-12-27 09:29:22,855][20347] Updated weights for policy 0, policy_version 556636 (0.0008) [2023-12-27 09:29:23,258][20381] Updated weights for policy 1, policy_version 556403 (0.0008) [2023-12-27 09:29:23,314][20381] Updated weights for policy 1, policy_version 556413 (0.0008) [2023-12-27 09:29:23,370][20381] Updated weights for policy 1, policy_version 556423 (0.0009) [2023-12-27 09:29:23,474][20347] Updated weights for policy 0, policy_version 556646 (0.0006) [2023-12-27 09:29:23,536][20347] Updated weights for policy 0, policy_version 556656 (0.0005) [2023-12-27 09:29:23,583][20347] Updated weights for policy 0, policy_version 556666 (0.0006) [2023-12-27 09:29:24,018][20381] Updated weights for policy 1, policy_version 556433 (0.0007) [2023-12-27 09:29:24,067][20381] Updated weights for policy 1, policy_version 556443 (0.0005) [2023-12-27 09:29:24,117][20381] Updated weights for policy 1, policy_version 556453 (0.0005) [2023-12-27 09:29:24,173][20381] Updated weights for policy 1, policy_version 556463 (0.0005) [2023-12-27 09:29:24,240][20347] Updated weights for policy 0, policy_version 556676 (0.0005) [2023-12-27 09:29:24,295][20347] Updated weights for policy 0, policy_version 556686 (0.0006) [2023-12-27 09:29:24,346][20347] Updated weights for policy 0, policy_version 556696 (0.0007) [2023-12-27 09:29:24,827][20381] Updated weights for policy 1, policy_version 556473 (0.0008) [2023-12-27 09:29:24,887][20381] Updated weights for policy 1, policy_version 556483 (0.0008) [2023-12-27 09:29:24,943][20381] Updated weights for policy 1, policy_version 556493 (0.0008) [2023-12-27 09:29:24,955][20347] Updated weights for policy 0, policy_version 556706 (0.0006) [2023-12-27 09:29:25,013][20347] Updated weights for policy 0, policy_version 556716 (0.0008) [2023-12-27 09:29:25,065][20347] Updated weights for policy 0, policy_version 556726 (0.0007) [2023-12-27 09:29:25,119][20347] Updated weights for policy 0, policy_version 556736 (0.0008) [2023-12-27 09:29:25,651][20381] Updated weights for policy 1, policy_version 556503 (0.0005) [2023-12-27 09:29:25,707][20381] Updated weights for policy 1, policy_version 556513 (0.0007) [2023-12-27 09:29:25,742][20347] Updated weights for policy 0, policy_version 556746 (0.0005) [2023-12-27 09:29:25,765][20381] Updated weights for policy 1, policy_version 556523 (0.0005) [2023-12-27 09:29:25,797][20347] Updated weights for policy 0, policy_version 556756 (0.0006) [2023-12-27 09:29:25,846][20347] Updated weights for policy 0, policy_version 556766 (0.0005) [2023-12-27 09:29:26,385][20381] Updated weights for policy 1, policy_version 556533 (0.0007) [2023-12-27 09:29:26,394][20347] Updated weights for policy 0, policy_version 556776 (0.0006) [2023-12-27 09:29:26,432][20381] Updated weights for policy 1, policy_version 556543 (0.0005) [2023-12-27 09:29:26,438][20347] Updated weights for policy 0, policy_version 556786 (0.0005) [2023-12-27 09:29:26,480][20381] Updated weights for policy 1, policy_version 556553 (0.0006) [2023-12-27 09:29:26,483][20347] Updated weights for policy 0, policy_version 556796 (0.0005) [2023-12-27 09:29:27,102][20347] Updated weights for policy 0, policy_version 556806 (0.0007) [2023-12-27 09:29:27,136][20381] Updated weights for policy 1, policy_version 556563 (0.0005) [2023-12-27 09:29:27,156][20347] Updated weights for policy 0, policy_version 556816 (0.0006) [2023-12-27 09:29:27,199][20381] Updated weights for policy 1, policy_version 556573 (0.0008) [2023-12-27 09:29:27,207][20347] Updated weights for policy 0, policy_version 556826 (0.0007) [2023-12-27 09:29:27,248][20381] Updated weights for policy 1, policy_version 556583 (0.0006) [2023-12-27 09:29:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 285073408. Throughput: 0: 10382.7, 1: 10371.3. Samples: 285055112. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:27,363][19170] Avg episode reward: [(0, '961.560'), (1, '661.550')] [2023-12-27 09:29:27,823][20347] Updated weights for policy 0, policy_version 556836 (0.0006) [2023-12-27 09:29:27,871][20347] Updated weights for policy 0, policy_version 556846 (0.0005) [2023-12-27 09:29:27,916][20381] Updated weights for policy 1, policy_version 556593 (0.0008) [2023-12-27 09:29:27,928][20347] Updated weights for policy 0, policy_version 556856 (0.0005) [2023-12-27 09:29:27,977][20381] Updated weights for policy 1, policy_version 556603 (0.0005) [2023-12-27 09:29:28,040][20381] Updated weights for policy 1, policy_version 556613 (0.0008) [2023-12-27 09:29:28,103][20381] Updated weights for policy 1, policy_version 556623 (0.0007) [2023-12-27 09:29:28,585][20347] Updated weights for policy 0, policy_version 556866 (0.0006) [2023-12-27 09:29:28,640][20347] Updated weights for policy 0, policy_version 556876 (0.0007) [2023-12-27 09:29:28,688][20347] Updated weights for policy 0, policy_version 556886 (0.0007) [2023-12-27 09:29:28,734][20381] Updated weights for policy 1, policy_version 556633 (0.0007) [2023-12-27 09:29:28,744][20347] Updated weights for policy 0, policy_version 556896 (0.0006) [2023-12-27 09:29:28,789][20381] Updated weights for policy 1, policy_version 556643 (0.0007) [2023-12-27 09:29:28,845][20381] Updated weights for policy 1, policy_version 556653 (0.0008) [2023-12-27 09:29:29,457][20347] Updated weights for policy 0, policy_version 556906 (0.0005) [2023-12-27 09:29:29,514][20347] Updated weights for policy 0, policy_version 556916 (0.0008) [2023-12-27 09:29:29,542][20381] Updated weights for policy 1, policy_version 556663 (0.0006) [2023-12-27 09:29:29,569][20347] Updated weights for policy 0, policy_version 556926 (0.0010) [2023-12-27 09:29:29,591][20381] Updated weights for policy 1, policy_version 556673 (0.0007) [2023-12-27 09:29:29,640][20381] Updated weights for policy 1, policy_version 556683 (0.0007) [2023-12-27 09:29:30,165][20347] Updated weights for policy 0, policy_version 556936 (0.0005) [2023-12-27 09:29:30,217][20347] Updated weights for policy 0, policy_version 556946 (0.0010) [2023-12-27 09:29:30,265][20347] Updated weights for policy 0, policy_version 556956 (0.0010) [2023-12-27 09:29:30,376][20381] Updated weights for policy 1, policy_version 556693 (0.0008) [2023-12-27 09:29:30,430][20381] Updated weights for policy 1, policy_version 556703 (0.0007) [2023-12-27 09:29:30,485][20381] Updated weights for policy 1, policy_version 556713 (0.0006) [2023-12-27 09:29:30,902][20347] Updated weights for policy 0, policy_version 556966 (0.0010) [2023-12-27 09:29:30,961][20347] Updated weights for policy 0, policy_version 556976 (0.0010) [2023-12-27 09:29:31,016][20347] Updated weights for policy 0, policy_version 556986 (0.0008) [2023-12-27 09:29:31,188][20381] Updated weights for policy 1, policy_version 556723 (0.0007) [2023-12-27 09:29:31,254][20381] Updated weights for policy 1, policy_version 556733 (0.0008) [2023-12-27 09:29:31,316][20381] Updated weights for policy 1, policy_version 556744 (0.0009) [2023-12-27 09:29:31,706][20347] Updated weights for policy 0, policy_version 556996 (0.0007) [2023-12-27 09:29:31,779][20347] Updated weights for policy 0, policy_version 557006 (0.0009) [2023-12-27 09:29:31,840][20347] Updated weights for policy 0, policy_version 557016 (0.0007) [2023-12-27 09:29:31,988][20381] Updated weights for policy 1, policy_version 556754 (0.0009) [2023-12-27 09:29:32,047][20381] Updated weights for policy 1, policy_version 556764 (0.0007) [2023-12-27 09:29:32,095][20381] Updated weights for policy 1, policy_version 556774 (0.0007) [2023-12-27 09:29:32,150][20381] Updated weights for policy 1, policy_version 556784 (0.0008) [2023-12-27 09:29:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 285171712. Throughput: 0: 10458.3, 1: 10326.0. Samples: 285180464. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:32,363][19170] Avg episode reward: [(0, '961.520'), (1, '747.110')] [2023-12-27 09:29:32,514][20347] Updated weights for policy 0, policy_version 557026 (0.0008) [2023-12-27 09:29:32,573][20347] Updated weights for policy 0, policy_version 557036 (0.0007) [2023-12-27 09:29:32,631][20347] Updated weights for policy 0, policy_version 557046 (0.0008) [2023-12-27 09:29:32,688][20347] Updated weights for policy 0, policy_version 557056 (0.0007) [2023-12-27 09:29:32,816][20381] Updated weights for policy 1, policy_version 556794 (0.0006) [2023-12-27 09:29:32,873][20381] Updated weights for policy 1, policy_version 556804 (0.0007) [2023-12-27 09:29:32,932][20381] Updated weights for policy 1, policy_version 556814 (0.0008) [2023-12-27 09:29:33,324][20347] Updated weights for policy 0, policy_version 557066 (0.0007) [2023-12-27 09:29:33,372][20347] Updated weights for policy 0, policy_version 557076 (0.0007) [2023-12-27 09:29:33,418][20347] Updated weights for policy 0, policy_version 557086 (0.0007) [2023-12-27 09:29:33,624][20381] Updated weights for policy 1, policy_version 556824 (0.0011) [2023-12-27 09:29:33,681][20381] Updated weights for policy 1, policy_version 556834 (0.0010) [2023-12-27 09:29:33,732][20381] Updated weights for policy 1, policy_version 556844 (0.0010) [2023-12-27 09:29:34,122][20347] Updated weights for policy 0, policy_version 557096 (0.0007) [2023-12-27 09:29:34,185][20347] Updated weights for policy 0, policy_version 557106 (0.0007) [2023-12-27 09:29:34,243][20347] Updated weights for policy 0, policy_version 557116 (0.0005) [2023-12-27 09:29:34,415][20381] Updated weights for policy 1, policy_version 556854 (0.0007) [2023-12-27 09:29:34,478][20381] Updated weights for policy 1, policy_version 556864 (0.0008) [2023-12-27 09:29:34,541][20381] Updated weights for policy 1, policy_version 556874 (0.0008) [2023-12-27 09:29:34,889][20347] Updated weights for policy 0, policy_version 557126 (0.0007) [2023-12-27 09:29:34,941][20347] Updated weights for policy 0, policy_version 557136 (0.0008) [2023-12-27 09:29:35,000][20347] Updated weights for policy 0, policy_version 557146 (0.0007) [2023-12-27 09:29:35,208][20381] Updated weights for policy 1, policy_version 556884 (0.0007) [2023-12-27 09:29:35,270][20381] Updated weights for policy 1, policy_version 556894 (0.0006) [2023-12-27 09:29:35,285][20292] KL-divergence is very high: 131.9559 [2023-12-27 09:29:35,301][20292] KL-divergence is very high: 223.1159 [2023-12-27 09:29:35,307][20292] KL-divergence is very high: 204.9115 [2023-12-27 09:29:35,313][20292] KL-divergence is very high: 127.1702 [2023-12-27 09:29:35,326][20381] Updated weights for policy 1, policy_version 556904 (0.0009) [2023-12-27 09:29:35,678][20347] Updated weights for policy 0, policy_version 557156 (0.0007) [2023-12-27 09:29:35,729][20347] Updated weights for policy 0, policy_version 557166 (0.0006) [2023-12-27 09:29:35,785][20347] Updated weights for policy 0, policy_version 557176 (0.0005) [2023-12-27 09:29:35,936][20381] Updated weights for policy 1, policy_version 556914 (0.0009) [2023-12-27 09:29:35,985][20381] Updated weights for policy 1, policy_version 556924 (0.0005) [2023-12-27 09:29:36,043][20381] Updated weights for policy 1, policy_version 556934 (0.0006) [2023-12-27 09:29:36,098][20381] Updated weights for policy 1, policy_version 556944 (0.0010) [2023-12-27 09:29:36,427][20347] Updated weights for policy 0, policy_version 557186 (0.0006) [2023-12-27 09:29:36,483][20347] Updated weights for policy 0, policy_version 557196 (0.0008) [2023-12-27 09:29:36,547][20347] Updated weights for policy 0, policy_version 557206 (0.0007) [2023-12-27 09:29:36,616][20347] Updated weights for policy 0, policy_version 557216 (0.0007) [2023-12-27 09:29:36,762][20381] Updated weights for policy 1, policy_version 556954 (0.0009) [2023-12-27 09:29:36,822][20381] Updated weights for policy 1, policy_version 556964 (0.0008) [2023-12-27 09:29:36,872][20381] Updated weights for policy 1, policy_version 556974 (0.0009) [2023-12-27 09:29:37,158][20347] Updated weights for policy 0, policy_version 557226 (0.0005) [2023-12-27 09:29:37,216][20347] Updated weights for policy 0, policy_version 557236 (0.0005) [2023-12-27 09:29:37,272][20347] Updated weights for policy 0, policy_version 557246 (0.0007) [2023-12-27 09:29:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 285278208. Throughput: 0: 10494.5, 1: 10336.1. Samples: 285242572. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '730.150')] [2023-12-27 09:29:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000556976_142606336.pth... [2023-12-27 09:29:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000557248_142671872.pth... [2023-12-27 09:29:37,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000555792_142303232.pth [2023-12-27 09:29:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000556000_142352384.pth [2023-12-27 09:29:37,600][20381] Updated weights for policy 1, policy_version 556984 (0.0008) [2023-12-27 09:29:37,664][20381] Updated weights for policy 1, policy_version 556994 (0.0008) [2023-12-27 09:29:37,724][20381] Updated weights for policy 1, policy_version 557004 (0.0008) [2023-12-27 09:29:37,826][20347] Updated weights for policy 0, policy_version 557256 (0.0007) [2023-12-27 09:29:37,870][20347] Updated weights for policy 0, policy_version 557266 (0.0007) [2023-12-27 09:29:37,920][20347] Updated weights for policy 0, policy_version 557276 (0.0006) [2023-12-27 09:29:38,340][20381] Updated weights for policy 1, policy_version 557014 (0.0006) [2023-12-27 09:29:38,401][20381] Updated weights for policy 1, policy_version 557024 (0.0008) [2023-12-27 09:29:38,461][20381] Updated weights for policy 1, policy_version 557034 (0.0008) [2023-12-27 09:29:38,663][20347] Updated weights for policy 0, policy_version 557286 (0.0007) [2023-12-27 09:29:38,709][20347] Updated weights for policy 0, policy_version 557296 (0.0007) [2023-12-27 09:29:38,754][20347] Updated weights for policy 0, policy_version 557306 (0.0007) [2023-12-27 09:29:39,096][20381] Updated weights for policy 1, policy_version 557044 (0.0009) [2023-12-27 09:29:39,148][20381] Updated weights for policy 1, policy_version 557054 (0.0010) [2023-12-27 09:29:39,200][20381] Updated weights for policy 1, policy_version 557064 (0.0010) [2023-12-27 09:29:39,462][20347] Updated weights for policy 0, policy_version 557316 (0.0009) [2023-12-27 09:29:39,515][20347] Updated weights for policy 0, policy_version 557326 (0.0010) [2023-12-27 09:29:39,574][20347] Updated weights for policy 0, policy_version 557336 (0.0010) [2023-12-27 09:29:39,950][20381] Updated weights for policy 1, policy_version 557074 (0.0010) [2023-12-27 09:29:40,008][20381] Updated weights for policy 1, policy_version 557084 (0.0009) [2023-12-27 09:29:40,063][20381] Updated weights for policy 1, policy_version 557094 (0.0009) [2023-12-27 09:29:40,111][20381] Updated weights for policy 1, policy_version 557104 (0.0008) [2023-12-27 09:29:40,273][20347] Updated weights for policy 0, policy_version 557346 (0.0006) [2023-12-27 09:29:40,333][20347] Updated weights for policy 0, policy_version 557356 (0.0007) [2023-12-27 09:29:40,388][20291] KL-divergence is very high: 269.5590 [2023-12-27 09:29:40,396][20347] Updated weights for policy 0, policy_version 557366 (0.0008) [2023-12-27 09:29:40,430][20291] KL-divergence is very high: 185.8517 [2023-12-27 09:29:40,448][20347] Updated weights for policy 0, policy_version 557376 (0.0009) [2023-12-27 09:29:40,813][20381] Updated weights for policy 1, policy_version 557114 (0.0008) [2023-12-27 09:29:40,880][20381] Updated weights for policy 1, policy_version 557124 (0.0009) [2023-12-27 09:29:40,939][20381] Updated weights for policy 1, policy_version 557134 (0.0006) [2023-12-27 09:29:41,075][20347] Updated weights for policy 0, policy_version 557386 (0.0007) [2023-12-27 09:29:41,079][20291] KL-divergence is very high: 122.4410 [2023-12-27 09:29:41,131][20347] Updated weights for policy 0, policy_version 557396 (0.0007) [2023-12-27 09:29:41,197][20347] Updated weights for policy 0, policy_version 557406 (0.0007) [2023-12-27 09:29:41,614][20381] Updated weights for policy 1, policy_version 557144 (0.0007) [2023-12-27 09:29:41,683][20381] Updated weights for policy 1, policy_version 557154 (0.0006) [2023-12-27 09:29:41,750][20381] Updated weights for policy 1, policy_version 557164 (0.0008) [2023-12-27 09:29:41,946][20347] Updated weights for policy 0, policy_version 557416 (0.0008) [2023-12-27 09:29:42,014][20347] Updated weights for policy 0, policy_version 557426 (0.0009) [2023-12-27 09:29:42,077][20347] Updated weights for policy 0, policy_version 557436 (0.0008) [2023-12-27 09:29:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 285376512. Throughput: 0: 10559.7, 1: 10301.1. Samples: 285368516. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:42,363][19170] Avg episode reward: [(0, '980.070'), (1, '678.410')] [2023-12-27 09:29:42,388][20381] Updated weights for policy 1, policy_version 557174 (0.0008) [2023-12-27 09:29:42,447][20381] Updated weights for policy 1, policy_version 557184 (0.0008) [2023-12-27 09:29:42,501][20381] Updated weights for policy 1, policy_version 557194 (0.0008) [2023-12-27 09:29:42,792][20347] Updated weights for policy 0, policy_version 557446 (0.0007) [2023-12-27 09:29:42,847][20347] Updated weights for policy 0, policy_version 557456 (0.0008) [2023-12-27 09:29:42,911][20347] Updated weights for policy 0, policy_version 557466 (0.0007) [2023-12-27 09:29:43,207][20381] Updated weights for policy 1, policy_version 557204 (0.0007) [2023-12-27 09:29:43,259][20381] Updated weights for policy 1, policy_version 557214 (0.0006) [2023-12-27 09:29:43,309][20381] Updated weights for policy 1, policy_version 557224 (0.0011) [2023-12-27 09:29:43,509][20347] Updated weights for policy 0, policy_version 557476 (0.0005) [2023-12-27 09:29:43,565][20347] Updated weights for policy 0, policy_version 557486 (0.0005) [2023-12-27 09:29:43,616][20347] Updated weights for policy 0, policy_version 557496 (0.0008) [2023-12-27 09:29:43,918][20381] Updated weights for policy 1, policy_version 557234 (0.0007) [2023-12-27 09:29:43,981][20381] Updated weights for policy 1, policy_version 557244 (0.0008) [2023-12-27 09:29:44,043][20381] Updated weights for policy 1, policy_version 557254 (0.0008) [2023-12-27 09:29:44,097][20381] Updated weights for policy 1, policy_version 557264 (0.0010) [2023-12-27 09:29:44,260][20347] Updated weights for policy 0, policy_version 557506 (0.0006) [2023-12-27 09:29:44,319][20347] Updated weights for policy 0, policy_version 557516 (0.0010) [2023-12-27 09:29:44,374][20347] Updated weights for policy 0, policy_version 557526 (0.0010) [2023-12-27 09:29:44,435][20347] Updated weights for policy 0, policy_version 557536 (0.0010) [2023-12-27 09:29:44,726][20381] Updated weights for policy 1, policy_version 557274 (0.0007) [2023-12-27 09:29:44,801][20381] Updated weights for policy 1, policy_version 557284 (0.0008) [2023-12-27 09:29:44,861][20381] Updated weights for policy 1, policy_version 557294 (0.0008) [2023-12-27 09:29:45,082][20347] Updated weights for policy 0, policy_version 557546 (0.0010) [2023-12-27 09:29:45,145][20347] Updated weights for policy 0, policy_version 557556 (0.0009) [2023-12-27 09:29:45,208][20347] Updated weights for policy 0, policy_version 557566 (0.0008) [2023-12-27 09:29:45,488][20381] Updated weights for policy 1, policy_version 557304 (0.0007) [2023-12-27 09:29:45,546][20381] Updated weights for policy 1, policy_version 557314 (0.0006) [2023-12-27 09:29:45,601][20381] Updated weights for policy 1, policy_version 557324 (0.0006) [2023-12-27 09:29:45,867][20347] Updated weights for policy 0, policy_version 557576 (0.0007) [2023-12-27 09:29:45,924][20347] Updated weights for policy 0, policy_version 557586 (0.0007) [2023-12-27 09:29:45,977][20347] Updated weights for policy 0, policy_version 557596 (0.0007) [2023-12-27 09:29:46,214][20381] Updated weights for policy 1, policy_version 557334 (0.0009) [2023-12-27 09:29:46,278][20381] Updated weights for policy 1, policy_version 557344 (0.0010) [2023-12-27 09:29:46,330][20381] Updated weights for policy 1, policy_version 557354 (0.0010) [2023-12-27 09:29:46,688][20347] Updated weights for policy 0, policy_version 557606 (0.0007) [2023-12-27 09:29:46,749][20347] Updated weights for policy 0, policy_version 557616 (0.0007) [2023-12-27 09:29:46,805][20347] Updated weights for policy 0, policy_version 557626 (0.0008) [2023-12-27 09:29:46,970][20381] Updated weights for policy 1, policy_version 557364 (0.0008) [2023-12-27 09:29:47,026][20381] Updated weights for policy 1, policy_version 557374 (0.0005) [2023-12-27 09:29:47,074][20381] Updated weights for policy 1, policy_version 557384 (0.0007) [2023-12-27 09:29:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 285483008. Throughput: 0: 10523.4, 1: 10363.3. Samples: 285492580. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:47,363][19170] Avg episode reward: [(0, '970.420'), (1, '812.200')] [2023-12-27 09:29:47,428][20347] Updated weights for policy 0, policy_version 557636 (0.0008) [2023-12-27 09:29:47,479][20347] Updated weights for policy 0, policy_version 557646 (0.0007) [2023-12-27 09:29:47,527][20347] Updated weights for policy 0, policy_version 557656 (0.0009) [2023-12-27 09:29:47,721][20381] Updated weights for policy 1, policy_version 557394 (0.0009) [2023-12-27 09:29:47,793][20381] Updated weights for policy 1, policy_version 557404 (0.0010) [2023-12-27 09:29:47,854][20381] Updated weights for policy 1, policy_version 557414 (0.0010) [2023-12-27 09:29:47,910][20381] Updated weights for policy 1, policy_version 557424 (0.0011) [2023-12-27 09:29:48,234][20347] Updated weights for policy 0, policy_version 557666 (0.0009) [2023-12-27 09:29:48,288][20347] Updated weights for policy 0, policy_version 557676 (0.0008) [2023-12-27 09:29:48,341][20347] Updated weights for policy 0, policy_version 557686 (0.0007) [2023-12-27 09:29:48,398][20347] Updated weights for policy 0, policy_version 557696 (0.0007) [2023-12-27 09:29:48,568][20381] Updated weights for policy 1, policy_version 557434 (0.0010) [2023-12-27 09:29:48,625][20381] Updated weights for policy 1, policy_version 557444 (0.0011) [2023-12-27 09:29:48,681][20381] Updated weights for policy 1, policy_version 557454 (0.0010) [2023-12-27 09:29:49,084][20347] Updated weights for policy 0, policy_version 557706 (0.0010) [2023-12-27 09:29:49,138][20347] Updated weights for policy 0, policy_version 557716 (0.0010) [2023-12-27 09:29:49,197][20347] Updated weights for policy 0, policy_version 557726 (0.0008) [2023-12-27 09:29:49,387][20381] Updated weights for policy 1, policy_version 557464 (0.0007) [2023-12-27 09:29:49,453][20381] Updated weights for policy 1, policy_version 557474 (0.0005) [2023-12-27 09:29:49,519][20381] Updated weights for policy 1, policy_version 557484 (0.0006) [2023-12-27 09:29:49,837][20347] Updated weights for policy 0, policy_version 557736 (0.0007) [2023-12-27 09:29:49,892][20347] Updated weights for policy 0, policy_version 557746 (0.0007) [2023-12-27 09:29:49,955][20347] Updated weights for policy 0, policy_version 557756 (0.0008) [2023-12-27 09:29:50,152][20381] Updated weights for policy 1, policy_version 557494 (0.0008) [2023-12-27 09:29:50,220][20381] Updated weights for policy 1, policy_version 557504 (0.0007) [2023-12-27 09:29:50,285][20381] Updated weights for policy 1, policy_version 557514 (0.0006) [2023-12-27 09:29:50,643][20347] Updated weights for policy 0, policy_version 557766 (0.0005) [2023-12-27 09:29:50,710][20347] Updated weights for policy 0, policy_version 557776 (0.0006) [2023-12-27 09:29:50,776][20347] Updated weights for policy 0, policy_version 557786 (0.0007) [2023-12-27 09:29:50,907][20381] Updated weights for policy 1, policy_version 557524 (0.0006) [2023-12-27 09:29:50,959][20381] Updated weights for policy 1, policy_version 557534 (0.0006) [2023-12-27 09:29:51,020][20381] Updated weights for policy 1, policy_version 557544 (0.0007) [2023-12-27 09:29:51,435][20347] Updated weights for policy 0, policy_version 557796 (0.0007) [2023-12-27 09:29:51,497][20347] Updated weights for policy 0, policy_version 557806 (0.0005) [2023-12-27 09:29:51,544][20347] Updated weights for policy 0, policy_version 557816 (0.0006) [2023-12-27 09:29:51,709][20381] Updated weights for policy 1, policy_version 557554 (0.0008) [2023-12-27 09:29:51,772][20381] Updated weights for policy 1, policy_version 557564 (0.0008) [2023-12-27 09:29:51,824][20381] Updated weights for policy 1, policy_version 557574 (0.0007) [2023-12-27 09:29:51,884][20381] Updated weights for policy 1, policy_version 557584 (0.0007) [2023-12-27 09:29:52,135][20347] Updated weights for policy 0, policy_version 557826 (0.0008) [2023-12-27 09:29:52,197][20347] Updated weights for policy 0, policy_version 557836 (0.0008) [2023-12-27 09:29:52,261][20347] Updated weights for policy 0, policy_version 557846 (0.0007) [2023-12-27 09:29:52,315][20347] Updated weights for policy 0, policy_version 557856 (0.0008) [2023-12-27 09:29:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 285589504. Throughput: 0: 10491.7, 1: 10352.4. Samples: 285555256. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:52,363][19170] Avg episode reward: [(0, '980.780'), (1, '815.010')] [2023-12-27 09:29:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000557584_142761984.pth... [2023-12-27 09:29:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000557856_142827520.pth... [2023-12-27 09:29:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000556368_142450688.pth [2023-12-27 09:29:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000556608_142508032.pth [2023-12-27 09:29:52,627][20381] Updated weights for policy 1, policy_version 557594 (0.0008) [2023-12-27 09:29:52,692][20381] Updated weights for policy 1, policy_version 557604 (0.0008) [2023-12-27 09:29:52,754][20381] Updated weights for policy 1, policy_version 557614 (0.0009) [2023-12-27 09:29:52,983][20347] Updated weights for policy 0, policy_version 557866 (0.0010) [2023-12-27 09:29:53,042][20347] Updated weights for policy 0, policy_version 557876 (0.0007) [2023-12-27 09:29:53,106][20347] Updated weights for policy 0, policy_version 557886 (0.0010) [2023-12-27 09:29:53,432][20381] Updated weights for policy 1, policy_version 557624 (0.0008) [2023-12-27 09:29:53,485][20381] Updated weights for policy 1, policy_version 557634 (0.0007) [2023-12-27 09:29:53,540][20381] Updated weights for policy 1, policy_version 557644 (0.0007) [2023-12-27 09:29:53,758][20347] Updated weights for policy 0, policy_version 557896 (0.0010) [2023-12-27 09:29:53,821][20347] Updated weights for policy 0, policy_version 557906 (0.0010) [2023-12-27 09:29:53,875][20347] Updated weights for policy 0, policy_version 557916 (0.0010) [2023-12-27 09:29:54,149][20381] Updated weights for policy 1, policy_version 557654 (0.0009) [2023-12-27 09:29:54,201][20381] Updated weights for policy 1, policy_version 557664 (0.0011) [2023-12-27 09:29:54,256][20381] Updated weights for policy 1, policy_version 557674 (0.0010) [2023-12-27 09:29:54,539][20347] Updated weights for policy 0, policy_version 557926 (0.0007) [2023-12-27 09:29:54,597][20347] Updated weights for policy 0, policy_version 557936 (0.0006) [2023-12-27 09:29:54,649][20347] Updated weights for policy 0, policy_version 557946 (0.0006) [2023-12-27 09:29:54,884][20381] Updated weights for policy 1, policy_version 557684 (0.0010) [2023-12-27 09:29:54,945][20381] Updated weights for policy 1, policy_version 557694 (0.0006) [2023-12-27 09:29:55,009][20381] Updated weights for policy 1, policy_version 557704 (0.0008) [2023-12-27 09:29:55,319][20347] Updated weights for policy 0, policy_version 557956 (0.0006) [2023-12-27 09:29:55,366][20347] Updated weights for policy 0, policy_version 557966 (0.0008) [2023-12-27 09:29:55,410][20347] Updated weights for policy 0, policy_version 557976 (0.0008) [2023-12-27 09:29:55,590][20381] Updated weights for policy 1, policy_version 557714 (0.0006) [2023-12-27 09:29:55,643][20381] Updated weights for policy 1, policy_version 557724 (0.0008) [2023-12-27 09:29:55,692][20381] Updated weights for policy 1, policy_version 557734 (0.0008) [2023-12-27 09:29:55,752][20381] Updated weights for policy 1, policy_version 557744 (0.0008) [2023-12-27 09:29:56,068][20347] Updated weights for policy 0, policy_version 557986 (0.0007) [2023-12-27 09:29:56,132][20347] Updated weights for policy 0, policy_version 557996 (0.0005) [2023-12-27 09:29:56,188][20347] Updated weights for policy 0, policy_version 558006 (0.0007) [2023-12-27 09:29:56,243][20347] Updated weights for policy 0, policy_version 558016 (0.0005) [2023-12-27 09:29:56,387][20381] Updated weights for policy 1, policy_version 557754 (0.0005) [2023-12-27 09:29:56,443][20381] Updated weights for policy 1, policy_version 557764 (0.0005) [2023-12-27 09:29:56,505][20381] Updated weights for policy 1, policy_version 557774 (0.0005) [2023-12-27 09:29:56,837][20347] Updated weights for policy 0, policy_version 558026 (0.0007) [2023-12-27 09:29:56,887][20347] Updated weights for policy 0, policy_version 558036 (0.0008) [2023-12-27 09:29:56,942][20347] Updated weights for policy 0, policy_version 558046 (0.0006) [2023-12-27 09:29:57,086][20381] Updated weights for policy 1, policy_version 557784 (0.0006) [2023-12-27 09:29:57,139][20381] Updated weights for policy 1, policy_version 557794 (0.0006) [2023-12-27 09:29:57,195][20381] Updated weights for policy 1, policy_version 557804 (0.0007) [2023-12-27 09:29:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 285696000. Throughput: 0: 10521.1, 1: 10379.5. Samples: 285681864. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:29:57,363][19170] Avg episode reward: [(0, '990.430'), (1, '766.570')] [2023-12-27 09:29:57,551][20347] Updated weights for policy 0, policy_version 558056 (0.0005) [2023-12-27 09:29:57,603][20347] Updated weights for policy 0, policy_version 558066 (0.0005) [2023-12-27 09:29:57,659][20347] Updated weights for policy 0, policy_version 558076 (0.0005) [2023-12-27 09:29:57,898][20381] Updated weights for policy 1, policy_version 557814 (0.0008) [2023-12-27 09:29:57,954][20381] Updated weights for policy 1, policy_version 557824 (0.0008) [2023-12-27 09:29:58,013][20381] Updated weights for policy 1, policy_version 557834 (0.0006) [2023-12-27 09:29:58,217][20347] Updated weights for policy 0, policy_version 558086 (0.0007) [2023-12-27 09:29:58,269][20347] Updated weights for policy 0, policy_version 558096 (0.0008) [2023-12-27 09:29:58,295][20291] KL-divergence is very high: 175.4332 [2023-12-27 09:29:58,313][20347] Updated weights for policy 0, policy_version 558106 (0.0007) [2023-12-27 09:29:58,799][20381] Updated weights for policy 1, policy_version 557844 (0.0009) [2023-12-27 09:29:58,858][20381] Updated weights for policy 1, policy_version 557854 (0.0007) [2023-12-27 09:29:58,916][20381] Updated weights for policy 1, policy_version 557864 (0.0007) [2023-12-27 09:29:59,164][20347] Updated weights for policy 0, policy_version 558116 (0.0008) [2023-12-27 09:29:59,230][20347] Updated weights for policy 0, policy_version 558126 (0.0008) [2023-12-27 09:29:59,293][20347] Updated weights for policy 0, policy_version 558136 (0.0008) [2023-12-27 09:29:59,640][20381] Updated weights for policy 1, policy_version 557874 (0.0007) [2023-12-27 09:29:59,690][20381] Updated weights for policy 1, policy_version 557884 (0.0007) [2023-12-27 09:29:59,748][20381] Updated weights for policy 1, policy_version 557894 (0.0006) [2023-12-27 09:29:59,807][20381] Updated weights for policy 1, policy_version 557904 (0.0005) [2023-12-27 09:29:59,930][20347] Updated weights for policy 0, policy_version 558146 (0.0007) [2023-12-27 09:29:59,984][20347] Updated weights for policy 0, policy_version 558156 (0.0010) [2023-12-27 09:30:00,044][20347] Updated weights for policy 0, policy_version 558166 (0.0010) [2023-12-27 09:30:00,096][20347] Updated weights for policy 0, policy_version 558176 (0.0005) [2023-12-27 09:30:00,413][20381] Updated weights for policy 1, policy_version 557914 (0.0008) [2023-12-27 09:30:00,470][20381] Updated weights for policy 1, policy_version 557924 (0.0007) [2023-12-27 09:30:00,530][20381] Updated weights for policy 1, policy_version 557934 (0.0010) [2023-12-27 09:30:00,690][20347] Updated weights for policy 0, policy_version 558186 (0.0010) [2023-12-27 09:30:00,738][20347] Updated weights for policy 0, policy_version 558196 (0.0010) [2023-12-27 09:30:00,790][20347] Updated weights for policy 0, policy_version 558206 (0.0010) [2023-12-27 09:30:01,146][20381] Updated weights for policy 1, policy_version 557944 (0.0007) [2023-12-27 09:30:01,211][20381] Updated weights for policy 1, policy_version 557954 (0.0006) [2023-12-27 09:30:01,274][20381] Updated weights for policy 1, policy_version 557964 (0.0007) [2023-12-27 09:30:01,512][20347] Updated weights for policy 0, policy_version 558216 (0.0010) [2023-12-27 09:30:01,576][20347] Updated weights for policy 0, policy_version 558226 (0.0010) [2023-12-27 09:30:01,637][20347] Updated weights for policy 0, policy_version 558236 (0.0009) [2023-12-27 09:30:01,938][20381] Updated weights for policy 1, policy_version 557974 (0.0006) [2023-12-27 09:30:01,998][20381] Updated weights for policy 1, policy_version 557984 (0.0006) [2023-12-27 09:30:02,049][20381] Updated weights for policy 1, policy_version 557994 (0.0008) [2023-12-27 09:30:02,329][20347] Updated weights for policy 0, policy_version 558246 (0.0009) [2023-12-27 09:30:02,363][19170] Fps is (10 sec: 20477.7, 60 sec: 20889.2, 300 sec: 20799.3). Total num frames: 285794304. Throughput: 0: 10492.3, 1: 10387.8. Samples: 285806992. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:02,364][19170] Avg episode reward: [(0, '971.640'), (1, '805.400')] [2023-12-27 09:30:02,388][20347] Updated weights for policy 0, policy_version 558256 (0.0007) [2023-12-27 09:30:02,448][20347] Updated weights for policy 0, policy_version 558266 (0.0007) [2023-12-27 09:30:02,650][20381] Updated weights for policy 1, policy_version 558004 (0.0007) [2023-12-27 09:30:02,704][20381] Updated weights for policy 1, policy_version 558014 (0.0008) [2023-12-27 09:30:02,767][20381] Updated weights for policy 1, policy_version 558024 (0.0008) [2023-12-27 09:30:03,053][20347] Updated weights for policy 0, policy_version 558276 (0.0008) [2023-12-27 09:30:03,105][20347] Updated weights for policy 0, policy_version 558286 (0.0010) [2023-12-27 09:30:03,158][20347] Updated weights for policy 0, policy_version 558296 (0.0010) [2023-12-27 09:30:03,463][20381] Updated weights for policy 1, policy_version 558034 (0.0008) [2023-12-27 09:30:03,508][20381] Updated weights for policy 1, policy_version 558044 (0.0007) [2023-12-27 09:30:03,560][20381] Updated weights for policy 1, policy_version 558054 (0.0007) [2023-12-27 09:30:03,614][20381] Updated weights for policy 1, policy_version 558064 (0.0010) [2023-12-27 09:30:03,807][20347] Updated weights for policy 0, policy_version 558306 (0.0010) [2023-12-27 09:30:03,864][20347] Updated weights for policy 0, policy_version 558316 (0.0009) [2023-12-27 09:30:03,922][20347] Updated weights for policy 0, policy_version 558326 (0.0006) [2023-12-27 09:30:03,986][20347] Updated weights for policy 0, policy_version 558336 (0.0008) [2023-12-27 09:30:04,303][20381] Updated weights for policy 1, policy_version 558074 (0.0006) [2023-12-27 09:30:04,362][20381] Updated weights for policy 1, policy_version 558084 (0.0006) [2023-12-27 09:30:04,427][20381] Updated weights for policy 1, policy_version 558094 (0.0008) [2023-12-27 09:30:04,654][20347] Updated weights for policy 0, policy_version 558346 (0.0010) [2023-12-27 09:30:04,709][20347] Updated weights for policy 0, policy_version 558356 (0.0010) [2023-12-27 09:30:04,764][20347] Updated weights for policy 0, policy_version 558366 (0.0010) [2023-12-27 09:30:04,977][20381] Updated weights for policy 1, policy_version 558104 (0.0006) [2023-12-27 09:30:05,031][20381] Updated weights for policy 1, policy_version 558114 (0.0005) [2023-12-27 09:30:05,092][20381] Updated weights for policy 1, policy_version 558124 (0.0005) [2023-12-27 09:30:05,452][20347] Updated weights for policy 0, policy_version 558376 (0.0010) [2023-12-27 09:30:05,516][20347] Updated weights for policy 0, policy_version 558386 (0.0011) [2023-12-27 09:30:05,562][20347] Updated weights for policy 0, policy_version 558396 (0.0010) [2023-12-27 09:30:05,638][20381] Updated weights for policy 1, policy_version 558134 (0.0006) [2023-12-27 09:30:05,696][20381] Updated weights for policy 1, policy_version 558144 (0.0007) [2023-12-27 09:30:05,758][20381] Updated weights for policy 1, policy_version 558154 (0.0007) [2023-12-27 09:30:06,243][20347] Updated weights for policy 0, policy_version 558406 (0.0010) [2023-12-27 09:30:06,305][20347] Updated weights for policy 0, policy_version 558416 (0.0011) [2023-12-27 09:30:06,365][20347] Updated weights for policy 0, policy_version 558426 (0.0011) [2023-12-27 09:30:06,409][20381] Updated weights for policy 1, policy_version 558164 (0.0008) [2023-12-27 09:30:06,462][20381] Updated weights for policy 1, policy_version 558174 (0.0010) [2023-12-27 09:30:06,514][20381] Updated weights for policy 1, policy_version 558184 (0.0007) [2023-12-27 09:30:07,071][20347] Updated weights for policy 0, policy_version 558436 (0.0010) [2023-12-27 09:30:07,113][20347] Updated weights for policy 0, policy_version 558446 (0.0010) [2023-12-27 09:30:07,162][20347] Updated weights for policy 0, policy_version 558456 (0.0010) [2023-12-27 09:30:07,217][20381] Updated weights for policy 1, policy_version 558194 (0.0007) [2023-12-27 09:30:07,268][20381] Updated weights for policy 1, policy_version 558204 (0.0005) [2023-12-27 09:30:07,318][20381] Updated weights for policy 1, policy_version 558214 (0.0005) [2023-12-27 09:30:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 285900800. Throughput: 0: 10499.0, 1: 10424.6. Samples: 285871196. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:07,364][19170] Avg episode reward: [(0, '962.050'), (1, '776.160')] [2023-12-27 09:30:07,368][20381] Updated weights for policy 1, policy_version 558224 (0.0005) [2023-12-27 09:30:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000558224_142925824.pth... [2023-12-27 09:30:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000558464_142983168.pth... [2023-12-27 09:30:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000556976_142606336.pth [2023-12-27 09:30:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000557248_142671872.pth [2023-12-27 09:30:07,857][20347] Updated weights for policy 0, policy_version 558466 (0.0010) [2023-12-27 09:30:07,915][20347] Updated weights for policy 0, policy_version 558476 (0.0010) [2023-12-27 09:30:07,934][20381] Updated weights for policy 1, policy_version 558234 (0.0006) [2023-12-27 09:30:07,982][20347] Updated weights for policy 0, policy_version 558486 (0.0010) [2023-12-27 09:30:07,994][20381] Updated weights for policy 1, policy_version 558244 (0.0006) [2023-12-27 09:30:08,042][20347] Updated weights for policy 0, policy_version 558496 (0.0006) [2023-12-27 09:30:08,056][20381] Updated weights for policy 1, policy_version 558254 (0.0006) [2023-12-27 09:30:08,636][20347] Updated weights for policy 0, policy_version 558506 (0.0007) [2023-12-27 09:30:08,655][20381] Updated weights for policy 1, policy_version 558264 (0.0005) [2023-12-27 09:30:08,693][20347] Updated weights for policy 0, policy_version 558516 (0.0008) [2023-12-27 09:30:08,721][20381] Updated weights for policy 1, policy_version 558274 (0.0006) [2023-12-27 09:30:08,750][20347] Updated weights for policy 0, policy_version 558526 (0.0006) [2023-12-27 09:30:08,791][20381] Updated weights for policy 1, policy_version 558284 (0.0006) [2023-12-27 09:30:09,414][20347] Updated weights for policy 0, policy_version 558536 (0.0009) [2023-12-27 09:30:09,438][20381] Updated weights for policy 1, policy_version 558294 (0.0007) [2023-12-27 09:30:09,468][20347] Updated weights for policy 0, policy_version 558546 (0.0010) [2023-12-27 09:30:09,488][20381] Updated weights for policy 1, policy_version 558304 (0.0007) [2023-12-27 09:30:09,528][20347] Updated weights for policy 0, policy_version 558556 (0.0010) [2023-12-27 09:30:09,544][20381] Updated weights for policy 1, policy_version 558314 (0.0007) [2023-12-27 09:30:10,170][20347] Updated weights for policy 0, policy_version 558566 (0.0008) [2023-12-27 09:30:10,235][20347] Updated weights for policy 0, policy_version 558576 (0.0005) [2023-12-27 09:30:10,262][20381] Updated weights for policy 1, policy_version 558324 (0.0007) [2023-12-27 09:30:10,303][20347] Updated weights for policy 0, policy_version 558586 (0.0005) [2023-12-27 09:30:10,319][20381] Updated weights for policy 1, policy_version 558334 (0.0008) [2023-12-27 09:30:10,375][20381] Updated weights for policy 1, policy_version 558344 (0.0007) [2023-12-27 09:30:10,890][20347] Updated weights for policy 0, policy_version 558596 (0.0006) [2023-12-27 09:30:10,942][20347] Updated weights for policy 0, policy_version 558606 (0.0006) [2023-12-27 09:30:10,999][20347] Updated weights for policy 0, policy_version 558616 (0.0005) [2023-12-27 09:30:11,054][20381] Updated weights for policy 1, policy_version 558354 (0.0007) [2023-12-27 09:30:11,113][20381] Updated weights for policy 1, policy_version 558364 (0.0006) [2023-12-27 09:30:11,174][20381] Updated weights for policy 1, policy_version 558374 (0.0008) [2023-12-27 09:30:11,233][20381] Updated weights for policy 1, policy_version 558384 (0.0005) [2023-12-27 09:30:11,675][20347] Updated weights for policy 0, policy_version 558626 (0.0010) [2023-12-27 09:30:11,739][20347] Updated weights for policy 0, policy_version 558636 (0.0008) [2023-12-27 09:30:11,799][20347] Updated weights for policy 0, policy_version 558646 (0.0008) [2023-12-27 09:30:11,880][20347] Updated weights for policy 0, policy_version 558656 (0.0008) [2023-12-27 09:30:11,962][20381] Updated weights for policy 1, policy_version 558394 (0.0005) [2023-12-27 09:30:12,012][20381] Updated weights for policy 1, policy_version 558404 (0.0007) [2023-12-27 09:30:12,066][20381] Updated weights for policy 1, policy_version 558414 (0.0011) [2023-12-27 09:30:12,362][19170] Fps is (10 sec: 21301.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 286007296. Throughput: 0: 10507.3, 1: 10436.4. Samples: 285997580. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:12,363][19170] Avg episode reward: [(0, '961.660'), (1, '757.040')] [2023-12-27 09:30:12,550][20347] Updated weights for policy 0, policy_version 558666 (0.0010) [2023-12-27 09:30:12,600][20347] Updated weights for policy 0, policy_version 558676 (0.0011) [2023-12-27 09:30:12,653][20347] Updated weights for policy 0, policy_version 558686 (0.0011) [2023-12-27 09:30:12,776][20381] Updated weights for policy 1, policy_version 558424 (0.0010) [2023-12-27 09:30:12,837][20381] Updated weights for policy 1, policy_version 558434 (0.0011) [2023-12-27 09:30:12,900][20381] Updated weights for policy 1, policy_version 558444 (0.0011) [2023-12-27 09:30:13,359][20347] Updated weights for policy 0, policy_version 558696 (0.0010) [2023-12-27 09:30:13,415][20347] Updated weights for policy 0, policy_version 558706 (0.0010) [2023-12-27 09:30:13,467][20347] Updated weights for policy 0, policy_version 558716 (0.0010) [2023-12-27 09:30:13,543][20381] Updated weights for policy 1, policy_version 558454 (0.0007) [2023-12-27 09:30:13,595][20381] Updated weights for policy 1, policy_version 558464 (0.0009) [2023-12-27 09:30:13,644][20381] Updated weights for policy 1, policy_version 558474 (0.0010) [2023-12-27 09:30:14,060][20347] Updated weights for policy 0, policy_version 558726 (0.0007) [2023-12-27 09:30:14,114][20347] Updated weights for policy 0, policy_version 558736 (0.0005) [2023-12-27 09:30:14,170][20347] Updated weights for policy 0, policy_version 558746 (0.0005) [2023-12-27 09:30:14,315][20381] Updated weights for policy 1, policy_version 558484 (0.0009) [2023-12-27 09:30:14,378][20381] Updated weights for policy 1, policy_version 558494 (0.0006) [2023-12-27 09:30:14,439][20381] Updated weights for policy 1, policy_version 558504 (0.0007) [2023-12-27 09:30:14,753][20347] Updated weights for policy 0, policy_version 558756 (0.0006) [2023-12-27 09:30:14,822][20347] Updated weights for policy 0, policy_version 558767 (0.0008) [2023-12-27 09:30:14,875][20347] Updated weights for policy 0, policy_version 558777 (0.0007) [2023-12-27 09:30:15,130][20381] Updated weights for policy 1, policy_version 558514 (0.0008) [2023-12-27 09:30:15,193][20381] Updated weights for policy 1, policy_version 558524 (0.0007) [2023-12-27 09:30:15,254][20381] Updated weights for policy 1, policy_version 558534 (0.0009) [2023-12-27 09:30:15,311][20381] Updated weights for policy 1, policy_version 558544 (0.0006) [2023-12-27 09:30:15,604][20347] Updated weights for policy 0, policy_version 558787 (0.0008) [2023-12-27 09:30:15,655][20347] Updated weights for policy 0, policy_version 558797 (0.0008) [2023-12-27 09:30:15,695][20347] Updated weights for policy 0, policy_version 558807 (0.0007) [2023-12-27 09:30:15,898][20381] Updated weights for policy 1, policy_version 558554 (0.0008) [2023-12-27 09:30:15,960][20381] Updated weights for policy 1, policy_version 558564 (0.0010) [2023-12-27 09:30:15,990][20292] KL-divergence is very high: 118.5784 [2023-12-27 09:30:16,002][20292] KL-divergence is very high: 121.2149 [2023-12-27 09:30:16,018][20381] Updated weights for policy 1, policy_version 558574 (0.0011) [2023-12-27 09:30:16,408][20347] Updated weights for policy 0, policy_version 558817 (0.0006) [2023-12-27 09:30:16,473][20347] Updated weights for policy 0, policy_version 558827 (0.0007) [2023-12-27 09:30:16,534][20347] Updated weights for policy 0, policy_version 558837 (0.0008) [2023-12-27 09:30:16,580][20381] Updated weights for policy 1, policy_version 558584 (0.0008) [2023-12-27 09:30:16,601][20347] Updated weights for policy 0, policy_version 558847 (0.0008) [2023-12-27 09:30:16,639][20381] Updated weights for policy 1, policy_version 558594 (0.0010) [2023-12-27 09:30:16,703][20381] Updated weights for policy 1, policy_version 558604 (0.0009) [2023-12-27 09:30:17,248][20347] Updated weights for policy 0, policy_version 558857 (0.0008) [2023-12-27 09:30:17,304][20347] Updated weights for policy 0, policy_version 558867 (0.0007) [2023-12-27 09:30:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 286105600. Throughput: 0: 10451.0, 1: 10492.8. Samples: 286122936. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:17,363][19170] Avg episode reward: [(0, '913.330'), (1, '825.370')] [2023-12-27 09:30:17,366][20347] Updated weights for policy 0, policy_version 558877 (0.0008) [2023-12-27 09:30:17,367][20381] Updated weights for policy 1, policy_version 558614 (0.0008) [2023-12-27 09:30:17,422][20381] Updated weights for policy 1, policy_version 558624 (0.0005) [2023-12-27 09:30:17,479][20381] Updated weights for policy 1, policy_version 558634 (0.0010) [2023-12-27 09:30:18,086][20347] Updated weights for policy 0, policy_version 558887 (0.0006) [2023-12-27 09:30:18,141][20381] Updated weights for policy 1, policy_version 558644 (0.0010) [2023-12-27 09:30:18,148][20347] Updated weights for policy 0, policy_version 558897 (0.0006) [2023-12-27 09:30:18,192][20381] Updated weights for policy 1, policy_version 558654 (0.0008) [2023-12-27 09:30:18,197][20347] Updated weights for policy 0, policy_version 558907 (0.0006) [2023-12-27 09:30:18,246][20381] Updated weights for policy 1, policy_version 558664 (0.0008) [2023-12-27 09:30:18,897][20347] Updated weights for policy 0, policy_version 558917 (0.0007) [2023-12-27 09:30:18,956][20347] Updated weights for policy 0, policy_version 558927 (0.0008) [2023-12-27 09:30:18,971][20381] Updated weights for policy 1, policy_version 558674 (0.0010) [2023-12-27 09:30:19,018][20347] Updated weights for policy 0, policy_version 558937 (0.0007) [2023-12-27 09:30:19,035][20381] Updated weights for policy 1, policy_version 558684 (0.0008) [2023-12-27 09:30:19,097][20381] Updated weights for policy 1, policy_version 558694 (0.0009) [2023-12-27 09:30:19,157][20381] Updated weights for policy 1, policy_version 558704 (0.0010) [2023-12-27 09:30:19,707][20347] Updated weights for policy 0, policy_version 558947 (0.0006) [2023-12-27 09:30:19,767][20347] Updated weights for policy 0, policy_version 558957 (0.0008) [2023-12-27 09:30:19,805][20381] Updated weights for policy 1, policy_version 558714 (0.0011) [2023-12-27 09:30:19,822][20347] Updated weights for policy 0, policy_version 558967 (0.0008) [2023-12-27 09:30:19,867][20381] Updated weights for policy 1, policy_version 558724 (0.0009) [2023-12-27 09:30:19,926][20381] Updated weights for policy 1, policy_version 558734 (0.0009) [2023-12-27 09:30:20,524][20347] Updated weights for policy 0, policy_version 558977 (0.0008) [2023-12-27 09:30:20,582][20347] Updated weights for policy 0, policy_version 558987 (0.0007) [2023-12-27 09:30:20,598][20381] Updated weights for policy 1, policy_version 558744 (0.0008) [2023-12-27 09:30:20,640][20347] Updated weights for policy 0, policy_version 558997 (0.0008) [2023-12-27 09:30:20,663][20381] Updated weights for policy 1, policy_version 558754 (0.0009) [2023-12-27 09:30:20,699][20347] Updated weights for policy 0, policy_version 559007 (0.0008) [2023-12-27 09:30:20,724][20381] Updated weights for policy 1, policy_version 558764 (0.0011) [2023-12-27 09:30:21,454][20381] Updated weights for policy 1, policy_version 558774 (0.0011) [2023-12-27 09:30:21,468][20347] Updated weights for policy 0, policy_version 559017 (0.0007) [2023-12-27 09:30:21,519][20381] Updated weights for policy 1, policy_version 558784 (0.0011) [2023-12-27 09:30:21,523][20347] Updated weights for policy 0, policy_version 559027 (0.0007) [2023-12-27 09:30:21,579][20381] Updated weights for policy 1, policy_version 558794 (0.0011) [2023-12-27 09:30:21,588][20347] Updated weights for policy 0, policy_version 559037 (0.0007) [2023-12-27 09:30:22,333][20347] Updated weights for policy 0, policy_version 559047 (0.0008) [2023-12-27 09:30:22,355][20381] Updated weights for policy 1, policy_version 558804 (0.0011) [2023-12-27 09:30:22,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 286203904. Throughput: 0: 10409.1, 1: 10479.0. Samples: 286182532. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:22,363][19170] Avg episode reward: [(0, '913.300'), (1, '844.560')] [2023-12-27 09:30:22,401][20347] Updated weights for policy 0, policy_version 559057 (0.0008) [2023-12-27 09:30:22,424][20381] Updated weights for policy 1, policy_version 558814 (0.0009) [2023-12-27 09:30:22,460][20347] Updated weights for policy 0, policy_version 559067 (0.0007) [2023-12-27 09:30:22,480][20381] Updated weights for policy 1, policy_version 558824 (0.0008) [2023-12-27 09:30:22,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000559072_143138816.pth... [2023-12-27 09:30:22,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000557856_142827520.pth [2023-12-27 09:30:22,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000558832_143081472.pth... [2023-12-27 09:30:22,526][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000557584_142761984.pth [2023-12-27 09:30:23,157][20347] Updated weights for policy 0, policy_version 559077 (0.0007) [2023-12-27 09:30:23,212][20347] Updated weights for policy 0, policy_version 559087 (0.0007) [2023-12-27 09:30:23,233][20381] Updated weights for policy 1, policy_version 558834 (0.0008) [2023-12-27 09:30:23,263][20347] Updated weights for policy 0, policy_version 559097 (0.0007) [2023-12-27 09:30:23,288][20381] Updated weights for policy 1, policy_version 558844 (0.0010) [2023-12-27 09:30:23,333][20381] Updated weights for policy 1, policy_version 558854 (0.0010) [2023-12-27 09:30:23,382][20381] Updated weights for policy 1, policy_version 558864 (0.0010) [2023-12-27 09:30:23,928][20347] Updated weights for policy 0, policy_version 559107 (0.0005) [2023-12-27 09:30:23,982][20347] Updated weights for policy 0, policy_version 559117 (0.0005) [2023-12-27 09:30:24,031][20347] Updated weights for policy 0, policy_version 559127 (0.0005) [2023-12-27 09:30:24,047][20381] Updated weights for policy 1, policy_version 558874 (0.0008) [2023-12-27 09:30:24,112][20381] Updated weights for policy 1, policy_version 558884 (0.0007) [2023-12-27 09:30:24,175][20381] Updated weights for policy 1, policy_version 558894 (0.0009) [2023-12-27 09:30:24,686][20347] Updated weights for policy 0, policy_version 559137 (0.0005) [2023-12-27 09:30:24,748][20347] Updated weights for policy 0, policy_version 559147 (0.0008) [2023-12-27 09:30:24,803][20347] Updated weights for policy 0, policy_version 559157 (0.0006) [2023-12-27 09:30:24,840][20381] Updated weights for policy 1, policy_version 558904 (0.0009) [2023-12-27 09:30:24,850][20347] Updated weights for policy 0, policy_version 559167 (0.0006) [2023-12-27 09:30:24,892][20381] Updated weights for policy 1, policy_version 558914 (0.0009) [2023-12-27 09:30:24,953][20381] Updated weights for policy 1, policy_version 558924 (0.0012) [2023-12-27 09:30:25,594][20347] Updated weights for policy 0, policy_version 559177 (0.0007) [2023-12-27 09:30:25,607][20381] Updated weights for policy 1, policy_version 558934 (0.0007) [2023-12-27 09:30:25,649][20347] Updated weights for policy 0, policy_version 559187 (0.0007) [2023-12-27 09:30:25,668][20381] Updated weights for policy 1, policy_version 558944 (0.0006) [2023-12-27 09:30:25,709][20347] Updated weights for policy 0, policy_version 559197 (0.0008) [2023-12-27 09:30:25,726][20381] Updated weights for policy 1, policy_version 558954 (0.0006) [2023-12-27 09:30:26,370][20381] Updated weights for policy 1, policy_version 558964 (0.0007) [2023-12-27 09:30:26,375][20347] Updated weights for policy 0, policy_version 559207 (0.0007) [2023-12-27 09:30:26,427][20381] Updated weights for policy 1, policy_version 558974 (0.0011) [2023-12-27 09:30:26,428][20347] Updated weights for policy 0, policy_version 559217 (0.0005) [2023-12-27 09:30:26,483][20381] Updated weights for policy 1, policy_version 558984 (0.0011) [2023-12-27 09:30:26,485][20347] Updated weights for policy 0, policy_version 559227 (0.0005) [2023-12-27 09:30:27,117][20347] Updated weights for policy 0, policy_version 559237 (0.0007) [2023-12-27 09:30:27,174][20347] Updated weights for policy 0, policy_version 559247 (0.0008) [2023-12-27 09:30:27,187][20381] Updated weights for policy 1, policy_version 558994 (0.0011) [2023-12-27 09:30:27,235][20347] Updated weights for policy 0, policy_version 559257 (0.0006) [2023-12-27 09:30:27,245][20381] Updated weights for policy 1, policy_version 559004 (0.0011) [2023-12-27 09:30:27,306][20381] Updated weights for policy 1, policy_version 559014 (0.0011) [2023-12-27 09:30:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 286310400. Throughput: 0: 10304.5, 1: 10439.1. Samples: 286301980. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:27,363][19170] Avg episode reward: [(0, '951.680'), (1, '902.910')] [2023-12-27 09:30:27,368][20381] Updated weights for policy 1, policy_version 559024 (0.0011) [2023-12-27 09:30:27,916][20347] Updated weights for policy 0, policy_version 559267 (0.0005) [2023-12-27 09:30:27,968][20347] Updated weights for policy 0, policy_version 559277 (0.0007) [2023-12-27 09:30:28,024][20347] Updated weights for policy 0, policy_version 559287 (0.0007) [2023-12-27 09:30:28,045][20381] Updated weights for policy 1, policy_version 559034 (0.0010) [2023-12-27 09:30:28,100][20381] Updated weights for policy 1, policy_version 559044 (0.0010) [2023-12-27 09:30:28,160][20381] Updated weights for policy 1, policy_version 559054 (0.0010) [2023-12-27 09:30:28,638][20347] Updated weights for policy 0, policy_version 559297 (0.0007) [2023-12-27 09:30:28,698][20347] Updated weights for policy 0, policy_version 559307 (0.0008) [2023-12-27 09:30:28,749][20347] Updated weights for policy 0, policy_version 559317 (0.0007) [2023-12-27 09:30:28,800][20347] Updated weights for policy 0, policy_version 559327 (0.0007) [2023-12-27 09:30:28,846][20381] Updated weights for policy 1, policy_version 559064 (0.0011) [2023-12-27 09:30:28,899][20381] Updated weights for policy 1, policy_version 559074 (0.0011) [2023-12-27 09:30:28,957][20381] Updated weights for policy 1, policy_version 559084 (0.0009) [2023-12-27 09:30:29,472][20347] Updated weights for policy 0, policy_version 559337 (0.0007) [2023-12-27 09:30:29,520][20347] Updated weights for policy 0, policy_version 559347 (0.0007) [2023-12-27 09:30:29,566][20347] Updated weights for policy 0, policy_version 559357 (0.0007) [2023-12-27 09:30:29,632][20381] Updated weights for policy 1, policy_version 559094 (0.0011) [2023-12-27 09:30:29,690][20381] Updated weights for policy 1, policy_version 559104 (0.0011) [2023-12-27 09:30:29,742][20381] Updated weights for policy 1, policy_version 559114 (0.0010) [2023-12-27 09:30:30,332][20347] Updated weights for policy 0, policy_version 559367 (0.0007) [2023-12-27 09:30:30,374][20381] Updated weights for policy 1, policy_version 559124 (0.0011) [2023-12-27 09:30:30,393][20347] Updated weights for policy 0, policy_version 559377 (0.0007) [2023-12-27 09:30:30,430][20381] Updated weights for policy 1, policy_version 559134 (0.0010) [2023-12-27 09:30:30,447][20347] Updated weights for policy 0, policy_version 559387 (0.0009) [2023-12-27 09:30:30,491][20381] Updated weights for policy 1, policy_version 559144 (0.0010) [2023-12-27 09:30:31,065][20381] Updated weights for policy 1, policy_version 559154 (0.0010) [2023-12-27 09:30:31,070][20347] Updated weights for policy 0, policy_version 559397 (0.0007) [2023-12-27 09:30:31,122][20381] Updated weights for policy 1, policy_version 559164 (0.0011) [2023-12-27 09:30:31,136][20347] Updated weights for policy 0, policy_version 559407 (0.0008) [2023-12-27 09:30:31,183][20381] Updated weights for policy 1, policy_version 559174 (0.0011) [2023-12-27 09:30:31,189][20347] Updated weights for policy 0, policy_version 559417 (0.0005) [2023-12-27 09:30:31,243][20381] Updated weights for policy 1, policy_version 559184 (0.0011) [2023-12-27 09:30:31,863][20347] Updated weights for policy 0, policy_version 559427 (0.0006) [2023-12-27 09:30:31,913][20381] Updated weights for policy 1, policy_version 559194 (0.0008) [2023-12-27 09:30:31,920][20347] Updated weights for policy 0, policy_version 559437 (0.0007) [2023-12-27 09:30:31,973][20347] Updated weights for policy 0, policy_version 559447 (0.0007) [2023-12-27 09:30:31,975][20381] Updated weights for policy 1, policy_version 559204 (0.0005) [2023-12-27 09:30:32,031][20381] Updated weights for policy 1, policy_version 559214 (0.0005) [2023-12-27 09:30:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 286416896. Throughput: 0: 10338.0, 1: 10429.5. Samples: 286427116. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:32,363][19170] Avg episode reward: [(0, '962.760'), (1, '912.880')] [2023-12-27 09:30:32,627][20381] Updated weights for policy 1, policy_version 559224 (0.0007) [2023-12-27 09:30:32,691][20381] Updated weights for policy 1, policy_version 559234 (0.0007) [2023-12-27 09:30:32,698][20347] Updated weights for policy 0, policy_version 559457 (0.0007) [2023-12-27 09:30:32,747][20381] Updated weights for policy 1, policy_version 559244 (0.0008) [2023-12-27 09:30:32,754][20347] Updated weights for policy 0, policy_version 559467 (0.0006) [2023-12-27 09:30:32,809][20347] Updated weights for policy 0, policy_version 559477 (0.0007) [2023-12-27 09:30:32,868][20347] Updated weights for policy 0, policy_version 559487 (0.0007) [2023-12-27 09:30:33,345][20381] Updated weights for policy 1, policy_version 559254 (0.0005) [2023-12-27 09:30:33,397][20381] Updated weights for policy 1, policy_version 559264 (0.0008) [2023-12-27 09:30:33,443][20381] Updated weights for policy 1, policy_version 559274 (0.0010) [2023-12-27 09:30:33,559][20347] Updated weights for policy 0, policy_version 559497 (0.0006) [2023-12-27 09:30:33,613][20347] Updated weights for policy 0, policy_version 559507 (0.0005) [2023-12-27 09:30:33,663][20347] Updated weights for policy 0, policy_version 559517 (0.0007) [2023-12-27 09:30:34,095][20381] Updated weights for policy 1, policy_version 559284 (0.0010) [2023-12-27 09:30:34,143][20381] Updated weights for policy 1, policy_version 559294 (0.0010) [2023-12-27 09:30:34,213][20381] Updated weights for policy 1, policy_version 559304 (0.0008) [2023-12-27 09:30:34,302][20347] Updated weights for policy 0, policy_version 559527 (0.0007) [2023-12-27 09:30:34,357][20347] Updated weights for policy 0, policy_version 559537 (0.0007) [2023-12-27 09:30:34,416][20347] Updated weights for policy 0, policy_version 559547 (0.0007) [2023-12-27 09:30:34,803][20381] Updated weights for policy 1, policy_version 559314 (0.0007) [2023-12-27 09:30:34,863][20381] Updated weights for policy 1, policy_version 559324 (0.0010) [2023-12-27 09:30:34,928][20381] Updated weights for policy 1, policy_version 559334 (0.0010) [2023-12-27 09:30:34,989][20381] Updated weights for policy 1, policy_version 559344 (0.0010) [2023-12-27 09:30:35,133][20347] Updated weights for policy 0, policy_version 559557 (0.0008) [2023-12-27 09:30:35,187][20347] Updated weights for policy 0, policy_version 559567 (0.0009) [2023-12-27 09:30:35,242][20347] Updated weights for policy 0, policy_version 559577 (0.0008) [2023-12-27 09:30:35,637][20381] Updated weights for policy 1, policy_version 559354 (0.0006) [2023-12-27 09:30:35,697][20381] Updated weights for policy 1, policy_version 559364 (0.0009) [2023-12-27 09:30:35,751][20381] Updated weights for policy 1, policy_version 559374 (0.0010) [2023-12-27 09:30:35,959][20347] Updated weights for policy 0, policy_version 559587 (0.0008) [2023-12-27 09:30:36,014][20347] Updated weights for policy 0, policy_version 559597 (0.0007) [2023-12-27 09:30:36,068][20347] Updated weights for policy 0, policy_version 559607 (0.0007) [2023-12-27 09:30:36,098][20291] KL-divergence is very high: 101.1426 [2023-12-27 09:30:36,428][20381] Updated weights for policy 1, policy_version 559384 (0.0008) [2023-12-27 09:30:36,492][20381] Updated weights for policy 1, policy_version 559394 (0.0006) [2023-12-27 09:30:36,553][20381] Updated weights for policy 1, policy_version 559404 (0.0006) [2023-12-27 09:30:36,755][20291] KL-divergence is very high: 101.4730 [2023-12-27 09:30:36,755][20347] Updated weights for policy 0, policy_version 559617 (0.0008) [2023-12-27 09:30:36,809][20291] KL-divergence is very high: 146.8107 [2023-12-27 09:30:36,817][20347] Updated weights for policy 0, policy_version 559627 (0.0009) [2023-12-27 09:30:36,876][20347] Updated weights for policy 0, policy_version 559637 (0.0008) [2023-12-27 09:30:36,897][20291] KL-divergence is very high: 125.8305 [2023-12-27 09:30:36,907][20291] KL-divergence is very high: 138.8808 [2023-12-27 09:30:36,925][20347] Updated weights for policy 0, policy_version 559647 (0.0009) [2023-12-27 09:30:37,114][20381] Updated weights for policy 1, policy_version 559414 (0.0006) [2023-12-27 09:30:37,168][20381] Updated weights for policy 1, policy_version 559424 (0.0005) [2023-12-27 09:30:37,222][20381] Updated weights for policy 1, policy_version 559434 (0.0005) [2023-12-27 09:30:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 286523392. Throughput: 0: 10304.9, 1: 10468.7. Samples: 286490068. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:37,363][19170] Avg episode reward: [(0, '856.620'), (1, '893.230')] [2023-12-27 09:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000559440_143237120.pth... [2023-12-27 09:30:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000559648_143286272.pth... [2023-12-27 09:30:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000558224_142925824.pth [2023-12-27 09:30:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000558464_142983168.pth [2023-12-27 09:30:37,678][20347] Updated weights for policy 0, policy_version 559657 (0.0008) [2023-12-27 09:30:37,735][20347] Updated weights for policy 0, policy_version 559667 (0.0008) [2023-12-27 09:30:37,780][20347] Updated weights for policy 0, policy_version 559677 (0.0008) [2023-12-27 09:30:37,825][20381] Updated weights for policy 1, policy_version 559444 (0.0008) [2023-12-27 09:30:37,872][20381] Updated weights for policy 1, policy_version 559454 (0.0005) [2023-12-27 09:30:37,926][20381] Updated weights for policy 1, policy_version 559464 (0.0005) [2023-12-27 09:30:38,537][20347] Updated weights for policy 0, policy_version 559687 (0.0008) [2023-12-27 09:30:38,556][20381] Updated weights for policy 1, policy_version 559474 (0.0008) [2023-12-27 09:30:38,595][20347] Updated weights for policy 0, policy_version 559697 (0.0006) [2023-12-27 09:30:38,612][20381] Updated weights for policy 1, policy_version 559484 (0.0010) [2023-12-27 09:30:38,650][20347] Updated weights for policy 0, policy_version 559707 (0.0005) [2023-12-27 09:30:38,667][20381] Updated weights for policy 1, policy_version 559494 (0.0010) [2023-12-27 09:30:38,720][20381] Updated weights for policy 1, policy_version 559504 (0.0010) [2023-12-27 09:30:39,352][20347] Updated weights for policy 0, policy_version 559717 (0.0007) [2023-12-27 09:30:39,414][20347] Updated weights for policy 0, policy_version 559727 (0.0007) [2023-12-27 09:30:39,450][20381] Updated weights for policy 1, policy_version 559514 (0.0007) [2023-12-27 09:30:39,470][20347] Updated weights for policy 0, policy_version 559737 (0.0007) [2023-12-27 09:30:39,510][20381] Updated weights for policy 1, policy_version 559524 (0.0006) [2023-12-27 09:30:39,572][20381] Updated weights for policy 1, policy_version 559534 (0.0006) [2023-12-27 09:30:40,195][20381] Updated weights for policy 1, policy_version 559544 (0.0008) [2023-12-27 09:30:40,211][20347] Updated weights for policy 0, policy_version 559747 (0.0008) [2023-12-27 09:30:40,251][20381] Updated weights for policy 1, policy_version 559554 (0.0011) [2023-12-27 09:30:40,268][20347] Updated weights for policy 0, policy_version 559757 (0.0006) [2023-12-27 09:30:40,311][20381] Updated weights for policy 1, policy_version 559564 (0.0011) [2023-12-27 09:30:40,328][20347] Updated weights for policy 0, policy_version 559767 (0.0006) [2023-12-27 09:30:40,936][20347] Updated weights for policy 0, policy_version 559777 (0.0008) [2023-12-27 09:30:40,964][20381] Updated weights for policy 1, policy_version 559574 (0.0009) [2023-12-27 09:30:40,993][20347] Updated weights for policy 0, policy_version 559787 (0.0007) [2023-12-27 09:30:41,014][20381] Updated weights for policy 1, policy_version 559584 (0.0011) [2023-12-27 09:30:41,058][20347] Updated weights for policy 0, policy_version 559797 (0.0008) [2023-12-27 09:30:41,077][20381] Updated weights for policy 1, policy_version 559594 (0.0009) [2023-12-27 09:30:41,118][20347] Updated weights for policy 0, policy_version 559807 (0.0007) [2023-12-27 09:30:41,849][20381] Updated weights for policy 1, policy_version 559604 (0.0009) [2023-12-27 09:30:41,881][20347] Updated weights for policy 0, policy_version 559817 (0.0006) [2023-12-27 09:30:41,905][20381] Updated weights for policy 1, policy_version 559614 (0.0009) [2023-12-27 09:30:41,941][20347] Updated weights for policy 0, policy_version 559827 (0.0009) [2023-12-27 09:30:41,963][20381] Updated weights for policy 1, policy_version 559624 (0.0010) [2023-12-27 09:30:41,991][20347] Updated weights for policy 0, policy_version 559837 (0.0008) [2023-12-27 09:30:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 286621696. Throughput: 0: 10218.1, 1: 10467.2. Samples: 286612704. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-12-27 09:30:42,363][19170] Avg episode reward: [(0, '893.860'), (1, '912.680')] [2023-12-27 09:30:42,658][20381] Updated weights for policy 1, policy_version 559634 (0.0010) [2023-12-27 09:30:42,706][20381] Updated weights for policy 1, policy_version 559644 (0.0007) [2023-12-27 09:30:42,734][20347] Updated weights for policy 0, policy_version 559847 (0.0006) [2023-12-27 09:30:42,761][20381] Updated weights for policy 1, policy_version 559654 (0.0008) [2023-12-27 09:30:42,793][20347] Updated weights for policy 0, policy_version 559857 (0.0006) [2023-12-27 09:30:42,815][20381] Updated weights for policy 1, policy_version 559664 (0.0007) [2023-12-27 09:30:42,850][20347] Updated weights for policy 0, policy_version 559867 (0.0005) [2023-12-27 09:30:43,429][20381] Updated weights for policy 1, policy_version 559674 (0.0007) [2023-12-27 09:30:43,475][20381] Updated weights for policy 1, policy_version 559684 (0.0005) [2023-12-27 09:30:43,499][20347] Updated weights for policy 0, policy_version 559877 (0.0007) [2023-12-27 09:30:43,528][20381] Updated weights for policy 1, policy_version 559694 (0.0005) [2023-12-27 09:30:43,557][20347] Updated weights for policy 0, policy_version 559887 (0.0008) [2023-12-27 09:30:43,619][20347] Updated weights for policy 0, policy_version 559897 (0.0008) [2023-12-27 09:30:44,122][20381] Updated weights for policy 1, policy_version 559704 (0.0008) [2023-12-27 09:30:44,184][20381] Updated weights for policy 1, policy_version 559714 (0.0008) [2023-12-27 09:30:44,250][20381] Updated weights for policy 1, policy_version 559724 (0.0008) [2023-12-27 09:30:44,345][20347] Updated weights for policy 0, policy_version 559907 (0.0008) [2023-12-27 09:30:44,407][20347] Updated weights for policy 0, policy_version 559917 (0.0008) [2023-12-27 09:30:44,469][20347] Updated weights for policy 0, policy_version 559927 (0.0007) [2023-12-27 09:30:44,979][20381] Updated weights for policy 1, policy_version 559734 (0.0007) [2023-12-27 09:30:45,043][20381] Updated weights for policy 1, policy_version 559744 (0.0009) [2023-12-27 09:30:45,101][20347] Updated weights for policy 0, policy_version 559937 (0.0008) [2023-12-27 09:30:45,114][20381] Updated weights for policy 1, policy_version 559754 (0.0006) [2023-12-27 09:30:45,158][20347] Updated weights for policy 0, policy_version 559947 (0.0007) [2023-12-27 09:30:45,213][20347] Updated weights for policy 0, policy_version 559957 (0.0007) [2023-12-27 09:30:45,266][20347] Updated weights for policy 0, policy_version 559967 (0.0008) [2023-12-27 09:30:45,763][20381] Updated weights for policy 1, policy_version 559764 (0.0006) [2023-12-27 09:30:45,823][20381] Updated weights for policy 1, policy_version 559774 (0.0005) [2023-12-27 09:30:45,887][20381] Updated weights for policy 1, policy_version 559784 (0.0005) [2023-12-27 09:30:45,966][20347] Updated weights for policy 0, policy_version 559977 (0.0009) [2023-12-27 09:30:46,014][20347] Updated weights for policy 0, policy_version 559987 (0.0006) [2023-12-27 09:30:46,069][20347] Updated weights for policy 0, policy_version 559997 (0.0007) [2023-12-27 09:30:46,484][20381] Updated weights for policy 1, policy_version 559794 (0.0006) [2023-12-27 09:30:46,539][20381] Updated weights for policy 1, policy_version 559804 (0.0008) [2023-12-27 09:30:46,599][20381] Updated weights for policy 1, policy_version 559814 (0.0008) [2023-12-27 09:30:46,654][20381] Updated weights for policy 1, policy_version 559824 (0.0008) [2023-12-27 09:30:46,721][20347] Updated weights for policy 0, policy_version 560007 (0.0006) [2023-12-27 09:30:46,770][20347] Updated weights for policy 0, policy_version 560017 (0.0005) [2023-12-27 09:30:46,820][20347] Updated weights for policy 0, policy_version 560027 (0.0007) [2023-12-27 09:30:47,353][20381] Updated weights for policy 1, policy_version 559834 (0.0008) [2023-12-27 09:30:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 286720000. Throughput: 0: 10170.6, 1: 10470.3. Samples: 286735808. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:30:47,363][19170] Avg episode reward: [(0, '904.860'), (1, '854.430')] [2023-12-27 09:30:47,406][20381] Updated weights for policy 1, policy_version 559844 (0.0007) [2023-12-27 09:30:47,440][20347] Updated weights for policy 0, policy_version 560037 (0.0006) [2023-12-27 09:30:47,451][20381] Updated weights for policy 1, policy_version 559854 (0.0006) [2023-12-27 09:30:47,500][20347] Updated weights for policy 0, policy_version 560047 (0.0005) [2023-12-27 09:30:47,549][20347] Updated weights for policy 0, policy_version 560057 (0.0005) [2023-12-27 09:30:48,139][20347] Updated weights for policy 0, policy_version 560067 (0.0007) [2023-12-27 09:30:48,194][20347] Updated weights for policy 0, policy_version 560077 (0.0009) [2023-12-27 09:30:48,200][20381] Updated weights for policy 1, policy_version 559864 (0.0008) [2023-12-27 09:30:48,248][20347] Updated weights for policy 0, policy_version 560087 (0.0006) [2023-12-27 09:30:48,255][20381] Updated weights for policy 1, policy_version 559874 (0.0005) [2023-12-27 09:30:48,308][20381] Updated weights for policy 1, policy_version 559884 (0.0006) [2023-12-27 09:30:48,962][20347] Updated weights for policy 0, policy_version 560097 (0.0006) [2023-12-27 09:30:48,997][20381] Updated weights for policy 1, policy_version 559894 (0.0006) [2023-12-27 09:30:49,021][20347] Updated weights for policy 0, policy_version 560107 (0.0006) [2023-12-27 09:30:49,053][20381] Updated weights for policy 1, policy_version 559904 (0.0006) [2023-12-27 09:30:49,083][20347] Updated weights for policy 0, policy_version 560117 (0.0008) [2023-12-27 09:30:49,113][20381] Updated weights for policy 1, policy_version 559914 (0.0007) [2023-12-27 09:30:49,136][20347] Updated weights for policy 0, policy_version 560127 (0.0008) [2023-12-27 09:30:49,805][20381] Updated weights for policy 1, policy_version 559924 (0.0006) [2023-12-27 09:30:49,834][20347] Updated weights for policy 0, policy_version 560137 (0.0007) [2023-12-27 09:30:49,867][20381] Updated weights for policy 1, policy_version 559934 (0.0006) [2023-12-27 09:30:49,894][20347] Updated weights for policy 0, policy_version 560147 (0.0007) [2023-12-27 09:30:49,930][20381] Updated weights for policy 1, policy_version 559944 (0.0007) [2023-12-27 09:30:49,960][20347] Updated weights for policy 0, policy_version 560157 (0.0008) [2023-12-27 09:30:50,568][20381] Updated weights for policy 1, policy_version 559954 (0.0007) [2023-12-27 09:30:50,633][20381] Updated weights for policy 1, policy_version 559964 (0.0007) [2023-12-27 09:30:50,649][20347] Updated weights for policy 0, policy_version 560167 (0.0007) [2023-12-27 09:30:50,690][20381] Updated weights for policy 1, policy_version 559974 (0.0006) [2023-12-27 09:30:50,709][20347] Updated weights for policy 0, policy_version 560177 (0.0008) [2023-12-27 09:30:50,746][20381] Updated weights for policy 1, policy_version 559984 (0.0010) [2023-12-27 09:30:50,762][20347] Updated weights for policy 0, policy_version 560187 (0.0009) [2023-12-27 09:30:51,337][20381] Updated weights for policy 1, policy_version 559994 (0.0006) [2023-12-27 09:30:51,400][20381] Updated weights for policy 1, policy_version 560004 (0.0007) [2023-12-27 09:30:51,453][20381] Updated weights for policy 1, policy_version 560014 (0.0006) [2023-12-27 09:30:51,537][20347] Updated weights for policy 0, policy_version 560197 (0.0008) [2023-12-27 09:30:51,597][20347] Updated weights for policy 0, policy_version 560207 (0.0007) [2023-12-27 09:30:51,664][20347] Updated weights for policy 0, policy_version 560217 (0.0008) [2023-12-27 09:30:52,138][20381] Updated weights for policy 1, policy_version 560024 (0.0007) [2023-12-27 09:30:52,189][20381] Updated weights for policy 1, policy_version 560034 (0.0008) [2023-12-27 09:30:52,237][20381] Updated weights for policy 1, policy_version 560044 (0.0007) [2023-12-27 09:30:52,314][20347] Updated weights for policy 0, policy_version 560227 (0.0008) [2023-12-27 09:30:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 286826496. Throughput: 0: 10158.5, 1: 10402.7. Samples: 286796448. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:30:52,363][19170] Avg episode reward: [(0, '933.810'), (1, '766.660')] [2023-12-27 09:30:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000560048_143392768.pth... [2023-12-27 09:30:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000558832_143081472.pth [2023-12-27 09:30:52,378][20347] Updated weights for policy 0, policy_version 560237 (0.0009) [2023-12-27 09:30:52,433][20347] Updated weights for policy 0, policy_version 560247 (0.0007) [2023-12-27 09:30:52,479][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000560256_143441920.pth... [2023-12-27 09:30:52,482][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000559072_143138816.pth [2023-12-27 09:30:52,940][20381] Updated weights for policy 1, policy_version 560054 (0.0006) [2023-12-27 09:30:52,989][20381] Updated weights for policy 1, policy_version 560064 (0.0005) [2023-12-27 09:30:53,048][20381] Updated weights for policy 1, policy_version 560074 (0.0007) [2023-12-27 09:30:53,074][20347] Updated weights for policy 0, policy_version 560257 (0.0009) [2023-12-27 09:30:53,135][20347] Updated weights for policy 0, policy_version 560267 (0.0008) [2023-12-27 09:30:53,196][20347] Updated weights for policy 0, policy_version 560277 (0.0009) [2023-12-27 09:30:53,255][20347] Updated weights for policy 0, policy_version 560287 (0.0008) [2023-12-27 09:30:53,660][20381] Updated weights for policy 1, policy_version 560084 (0.0007) [2023-12-27 09:30:53,722][20381] Updated weights for policy 1, policy_version 560094 (0.0005) [2023-12-27 09:30:53,770][20381] Updated weights for policy 1, policy_version 560104 (0.0005) [2023-12-27 09:30:53,818][20347] Updated weights for policy 0, policy_version 560297 (0.0010) [2023-12-27 09:30:53,870][20347] Updated weights for policy 0, policy_version 560307 (0.0006) [2023-12-27 09:30:53,928][20347] Updated weights for policy 0, policy_version 560317 (0.0006) [2023-12-27 09:30:54,337][20381] Updated weights for policy 1, policy_version 560114 (0.0005) [2023-12-27 09:30:54,392][20381] Updated weights for policy 1, policy_version 560124 (0.0007) [2023-12-27 09:30:54,452][20381] Updated weights for policy 1, policy_version 560134 (0.0009) [2023-12-27 09:30:54,497][20347] Updated weights for policy 0, policy_version 560327 (0.0010) [2023-12-27 09:30:54,510][20381] Updated weights for policy 1, policy_version 560144 (0.0006) [2023-12-27 09:30:54,545][20347] Updated weights for policy 0, policy_version 560337 (0.0010) [2023-12-27 09:30:54,608][20347] Updated weights for policy 0, policy_version 560347 (0.0006) [2023-12-27 09:30:55,177][20347] Updated weights for policy 0, policy_version 560357 (0.0006) [2023-12-27 09:30:55,216][20381] Updated weights for policy 1, policy_version 560154 (0.0007) [2023-12-27 09:30:55,232][20347] Updated weights for policy 0, policy_version 560367 (0.0007) [2023-12-27 09:30:55,273][20381] Updated weights for policy 1, policy_version 560164 (0.0007) [2023-12-27 09:30:55,288][20347] Updated weights for policy 0, policy_version 560377 (0.0008) [2023-12-27 09:30:55,330][20381] Updated weights for policy 1, policy_version 560174 (0.0007) [2023-12-27 09:30:55,886][20347] Updated weights for policy 0, policy_version 560387 (0.0006) [2023-12-27 09:30:55,946][20347] Updated weights for policy 0, policy_version 560397 (0.0005) [2023-12-27 09:30:56,002][20347] Updated weights for policy 0, policy_version 560407 (0.0006) [2023-12-27 09:30:56,053][20381] Updated weights for policy 1, policy_version 560184 (0.0008) [2023-12-27 09:30:56,109][20381] Updated weights for policy 1, policy_version 560194 (0.0008) [2023-12-27 09:30:56,166][20381] Updated weights for policy 1, policy_version 560204 (0.0008) [2023-12-27 09:30:56,670][20347] Updated weights for policy 0, policy_version 560417 (0.0008) [2023-12-27 09:30:56,718][20347] Updated weights for policy 0, policy_version 560427 (0.0007) [2023-12-27 09:30:56,766][20347] Updated weights for policy 0, policy_version 560437 (0.0008) [2023-12-27 09:30:56,816][20347] Updated weights for policy 0, policy_version 560447 (0.0007) [2023-12-27 09:30:56,837][20381] Updated weights for policy 1, policy_version 560214 (0.0007) [2023-12-27 09:30:56,892][20381] Updated weights for policy 1, policy_version 560224 (0.0008) [2023-12-27 09:30:56,947][20381] Updated weights for policy 1, policy_version 560234 (0.0008) [2023-12-27 09:30:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 286932992. Throughput: 0: 10191.3, 1: 10397.9. Samples: 286924092. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:30:57,363][19170] Avg episode reward: [(0, '943.490'), (1, '747.350')] [2023-12-27 09:30:57,516][20347] Updated weights for policy 0, policy_version 560457 (0.0007) [2023-12-27 09:30:57,564][20347] Updated weights for policy 0, policy_version 560467 (0.0008) [2023-12-27 09:30:57,615][20347] Updated weights for policy 0, policy_version 560477 (0.0007) [2023-12-27 09:30:57,628][20381] Updated weights for policy 1, policy_version 560244 (0.0007) [2023-12-27 09:30:57,672][20381] Updated weights for policy 1, policy_version 560254 (0.0007) [2023-12-27 09:30:57,724][20381] Updated weights for policy 1, policy_version 560264 (0.0008) [2023-12-27 09:30:58,300][20347] Updated weights for policy 0, policy_version 560487 (0.0005) [2023-12-27 09:30:58,365][20347] Updated weights for policy 0, policy_version 560497 (0.0008) [2023-12-27 09:30:58,417][20347] Updated weights for policy 0, policy_version 560507 (0.0010) [2023-12-27 09:30:58,476][20381] Updated weights for policy 1, policy_version 560274 (0.0007) [2023-12-27 09:30:58,540][20381] Updated weights for policy 1, policy_version 560284 (0.0008) [2023-12-27 09:30:58,602][20381] Updated weights for policy 1, policy_version 560294 (0.0008) [2023-12-27 09:30:58,664][20381] Updated weights for policy 1, policy_version 560304 (0.0008) [2023-12-27 09:30:59,176][20347] Updated weights for policy 0, policy_version 560517 (0.0009) [2023-12-27 09:30:59,233][20347] Updated weights for policy 0, policy_version 560527 (0.0008) [2023-12-27 09:30:59,296][20347] Updated weights for policy 0, policy_version 560537 (0.0008) [2023-12-27 09:30:59,430][20381] Updated weights for policy 1, policy_version 560314 (0.0008) [2023-12-27 09:30:59,493][20381] Updated weights for policy 1, policy_version 560324 (0.0008) [2023-12-27 09:30:59,554][20381] Updated weights for policy 1, policy_version 560334 (0.0008) [2023-12-27 09:30:59,943][20347] Updated weights for policy 0, policy_version 560547 (0.0006) [2023-12-27 09:31:00,008][20347] Updated weights for policy 0, policy_version 560557 (0.0006) [2023-12-27 09:31:00,073][20347] Updated weights for policy 0, policy_version 560567 (0.0009) [2023-12-27 09:31:00,258][20381] Updated weights for policy 1, policy_version 560344 (0.0009) [2023-12-27 09:31:00,313][20381] Updated weights for policy 1, policy_version 560354 (0.0008) [2023-12-27 09:31:00,366][20381] Updated weights for policy 1, policy_version 560364 (0.0008) [2023-12-27 09:31:00,604][20347] Updated weights for policy 0, policy_version 560577 (0.0005) [2023-12-27 09:31:00,657][20347] Updated weights for policy 0, policy_version 560587 (0.0005) [2023-12-27 09:31:00,720][20347] Updated weights for policy 0, policy_version 560597 (0.0010) [2023-12-27 09:31:00,777][20347] Updated weights for policy 0, policy_version 560607 (0.0010) [2023-12-27 09:31:00,991][20381] Updated weights for policy 1, policy_version 560374 (0.0006) [2023-12-27 09:31:01,046][20381] Updated weights for policy 1, policy_version 560384 (0.0006) [2023-12-27 09:31:01,103][20381] Updated weights for policy 1, policy_version 560394 (0.0006) [2023-12-27 09:31:01,389][20347] Updated weights for policy 0, policy_version 560617 (0.0007) [2023-12-27 09:31:01,446][20347] Updated weights for policy 0, policy_version 560627 (0.0005) [2023-12-27 09:31:01,502][20347] Updated weights for policy 0, policy_version 560637 (0.0006) [2023-12-27 09:31:01,838][20381] Updated weights for policy 1, policy_version 560404 (0.0008) [2023-12-27 09:31:01,889][20381] Updated weights for policy 1, policy_version 560414 (0.0008) [2023-12-27 09:31:01,941][20381] Updated weights for policy 1, policy_version 560424 (0.0008) [2023-12-27 09:31:02,152][20347] Updated weights for policy 0, policy_version 560647 (0.0010) [2023-12-27 09:31:02,211][20347] Updated weights for policy 0, policy_version 560657 (0.0007) [2023-12-27 09:31:02,265][20347] Updated weights for policy 0, policy_version 560667 (0.0007) [2023-12-27 09:31:02,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.5, 300 sec: 20799.4). Total num frames: 287039488. Throughput: 0: 10214.1, 1: 10315.6. Samples: 287046768. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:02,363][19170] Avg episode reward: [(0, '962.680'), (1, '806.170')] [2023-12-27 09:31:02,648][20381] Updated weights for policy 1, policy_version 560434 (0.0008) [2023-12-27 09:31:02,709][20381] Updated weights for policy 1, policy_version 560444 (0.0010) [2023-12-27 09:31:02,776][20381] Updated weights for policy 1, policy_version 560454 (0.0010) [2023-12-27 09:31:02,841][20381] Updated weights for policy 1, policy_version 560464 (0.0007) [2023-12-27 09:31:02,904][20347] Updated weights for policy 0, policy_version 560677 (0.0007) [2023-12-27 09:31:02,951][20347] Updated weights for policy 0, policy_version 560687 (0.0007) [2023-12-27 09:31:03,007][20347] Updated weights for policy 0, policy_version 560697 (0.0008) [2023-12-27 09:31:03,349][20381] Updated weights for policy 1, policy_version 560474 (0.0005) [2023-12-27 09:31:03,395][20381] Updated weights for policy 1, policy_version 560484 (0.0005) [2023-12-27 09:31:03,455][20381] Updated weights for policy 1, policy_version 560494 (0.0008) [2023-12-27 09:31:03,694][20347] Updated weights for policy 0, policy_version 560707 (0.0008) [2023-12-27 09:31:03,749][20347] Updated weights for policy 0, policy_version 560717 (0.0008) [2023-12-27 09:31:03,800][20347] Updated weights for policy 0, policy_version 560727 (0.0007) [2023-12-27 09:31:04,162][20381] Updated weights for policy 1, policy_version 560504 (0.0011) [2023-12-27 09:31:04,224][20381] Updated weights for policy 1, policy_version 560514 (0.0010) [2023-12-27 09:31:04,285][20381] Updated weights for policy 1, policy_version 560524 (0.0011) [2023-12-27 09:31:04,520][20347] Updated weights for policy 0, policy_version 560737 (0.0008) [2023-12-27 09:31:04,571][20347] Updated weights for policy 0, policy_version 560747 (0.0007) [2023-12-27 09:31:04,620][20347] Updated weights for policy 0, policy_version 560757 (0.0008) [2023-12-27 09:31:04,673][20347] Updated weights for policy 0, policy_version 560767 (0.0008) [2023-12-27 09:31:04,951][20381] Updated weights for policy 1, policy_version 560534 (0.0009) [2023-12-27 09:31:05,003][20381] Updated weights for policy 1, policy_version 560544 (0.0011) [2023-12-27 09:31:05,062][20381] Updated weights for policy 1, policy_version 560554 (0.0011) [2023-12-27 09:31:05,407][20347] Updated weights for policy 0, policy_version 560777 (0.0008) [2023-12-27 09:31:05,464][20347] Updated weights for policy 0, policy_version 560787 (0.0007) [2023-12-27 09:31:05,524][20347] Updated weights for policy 0, policy_version 560797 (0.0007) [2023-12-27 09:31:05,736][20381] Updated weights for policy 1, policy_version 560564 (0.0011) [2023-12-27 09:31:05,794][20381] Updated weights for policy 1, policy_version 560574 (0.0010) [2023-12-27 09:31:05,843][20381] Updated weights for policy 1, policy_version 560584 (0.0010) [2023-12-27 09:31:06,187][20347] Updated weights for policy 0, policy_version 560807 (0.0007) [2023-12-27 09:31:06,248][20347] Updated weights for policy 0, policy_version 560817 (0.0007) [2023-12-27 09:31:06,312][20347] Updated weights for policy 0, policy_version 560827 (0.0007) [2023-12-27 09:31:06,524][20381] Updated weights for policy 1, policy_version 560594 (0.0010) [2023-12-27 09:31:06,592][20381] Updated weights for policy 1, policy_version 560604 (0.0009) [2023-12-27 09:31:06,649][20381] Updated weights for policy 1, policy_version 560614 (0.0008) [2023-12-27 09:31:06,715][20381] Updated weights for policy 1, policy_version 560624 (0.0008) [2023-12-27 09:31:06,990][20347] Updated weights for policy 0, policy_version 560837 (0.0008) [2023-12-27 09:31:07,053][20347] Updated weights for policy 0, policy_version 560847 (0.0008) [2023-12-27 09:31:07,110][20347] Updated weights for policy 0, policy_version 560857 (0.0009) [2023-12-27 09:31:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 287137792. Throughput: 0: 10233.6, 1: 10351.8. Samples: 287108872. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '786.780')] [2023-12-27 09:31:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000560864_143597568.pth... [2023-12-27 09:31:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000559648_143286272.pth [2023-12-27 09:31:07,401][20381] Updated weights for policy 1, policy_version 560634 (0.0011) [2023-12-27 09:31:07,450][20381] Updated weights for policy 1, policy_version 560644 (0.0011) [2023-12-27 09:31:07,502][20381] Updated weights for policy 1, policy_version 560654 (0.0010) [2023-12-27 09:31:07,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000560656_143548416.pth... [2023-12-27 09:31:07,514][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000559440_143237120.pth [2023-12-27 09:31:07,751][20347] Updated weights for policy 0, policy_version 560867 (0.0008) [2023-12-27 09:31:07,808][20347] Updated weights for policy 0, policy_version 560877 (0.0007) [2023-12-27 09:31:07,865][20347] Updated weights for policy 0, policy_version 560887 (0.0006) [2023-12-27 09:31:08,212][20381] Updated weights for policy 1, policy_version 560664 (0.0010) [2023-12-27 09:31:08,270][20381] Updated weights for policy 1, policy_version 560674 (0.0011) [2023-12-27 09:31:08,329][20381] Updated weights for policy 1, policy_version 560684 (0.0011) [2023-12-27 09:31:08,535][20347] Updated weights for policy 0, policy_version 560897 (0.0006) [2023-12-27 09:31:08,593][20347] Updated weights for policy 0, policy_version 560907 (0.0010) [2023-12-27 09:31:08,655][20347] Updated weights for policy 0, policy_version 560917 (0.0005) [2023-12-27 09:31:08,715][20347] Updated weights for policy 0, policy_version 560927 (0.0006) [2023-12-27 09:31:09,036][20381] Updated weights for policy 1, policy_version 560694 (0.0011) [2023-12-27 09:31:09,091][20381] Updated weights for policy 1, policy_version 560704 (0.0010) [2023-12-27 09:31:09,149][20381] Updated weights for policy 1, policy_version 560714 (0.0011) [2023-12-27 09:31:09,363][20347] Updated weights for policy 0, policy_version 560937 (0.0010) [2023-12-27 09:31:09,423][20347] Updated weights for policy 0, policy_version 560947 (0.0011) [2023-12-27 09:31:09,489][20347] Updated weights for policy 0, policy_version 560957 (0.0006) [2023-12-27 09:31:09,867][20381] Updated weights for policy 1, policy_version 560724 (0.0010) [2023-12-27 09:31:09,923][20381] Updated weights for policy 1, policy_version 560734 (0.0011) [2023-12-27 09:31:09,984][20381] Updated weights for policy 1, policy_version 560744 (0.0011) [2023-12-27 09:31:10,128][20347] Updated weights for policy 0, policy_version 560967 (0.0009) [2023-12-27 09:31:10,187][20347] Updated weights for policy 0, policy_version 560977 (0.0010) [2023-12-27 09:31:10,240][20347] Updated weights for policy 0, policy_version 560987 (0.0010) [2023-12-27 09:31:10,637][20381] Updated weights for policy 1, policy_version 560754 (0.0010) [2023-12-27 09:31:10,703][20381] Updated weights for policy 1, policy_version 560764 (0.0006) [2023-12-27 09:31:10,768][20381] Updated weights for policy 1, policy_version 560774 (0.0007) [2023-12-27 09:31:10,829][20381] Updated weights for policy 1, policy_version 560784 (0.0008) [2023-12-27 09:31:10,907][20347] Updated weights for policy 0, policy_version 560997 (0.0008) [2023-12-27 09:31:10,969][20347] Updated weights for policy 0, policy_version 561007 (0.0005) [2023-12-27 09:31:11,030][20347] Updated weights for policy 0, policy_version 561017 (0.0008) [2023-12-27 09:31:11,543][20381] Updated weights for policy 1, policy_version 560794 (0.0008) [2023-12-27 09:31:11,602][20381] Updated weights for policy 1, policy_version 560804 (0.0007) [2023-12-27 09:31:11,664][20381] Updated weights for policy 1, policy_version 560814 (0.0007) [2023-12-27 09:31:11,702][20347] Updated weights for policy 0, policy_version 561027 (0.0010) [2023-12-27 09:31:11,768][20347] Updated weights for policy 0, policy_version 561037 (0.0010) [2023-12-27 09:31:11,824][20347] Updated weights for policy 0, policy_version 561047 (0.0010) [2023-12-27 09:31:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 287236096. Throughput: 0: 10291.4, 1: 10352.0. Samples: 287230932. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.300')] [2023-12-27 09:31:12,378][20381] Updated weights for policy 1, policy_version 560824 (0.0007) [2023-12-27 09:31:12,439][20381] Updated weights for policy 1, policy_version 560834 (0.0006) [2023-12-27 09:31:12,501][20381] Updated weights for policy 1, policy_version 560844 (0.0006) [2023-12-27 09:31:12,516][20347] Updated weights for policy 0, policy_version 561057 (0.0010) [2023-12-27 09:31:12,574][20347] Updated weights for policy 0, policy_version 561067 (0.0008) [2023-12-27 09:31:12,629][20347] Updated weights for policy 0, policy_version 561077 (0.0008) [2023-12-27 09:31:12,685][20347] Updated weights for policy 0, policy_version 561087 (0.0006) [2023-12-27 09:31:13,182][20381] Updated weights for policy 1, policy_version 560854 (0.0007) [2023-12-27 09:31:13,228][20381] Updated weights for policy 1, policy_version 560864 (0.0005) [2023-12-27 09:31:13,273][20381] Updated weights for policy 1, policy_version 560874 (0.0005) [2023-12-27 09:31:13,363][20347] Updated weights for policy 0, policy_version 561097 (0.0010) [2023-12-27 09:31:13,412][20347] Updated weights for policy 0, policy_version 561107 (0.0010) [2023-12-27 09:31:13,464][20347] Updated weights for policy 0, policy_version 561117 (0.0010) [2023-12-27 09:31:13,887][20381] Updated weights for policy 1, policy_version 560884 (0.0006) [2023-12-27 09:31:13,941][20381] Updated weights for policy 1, policy_version 560894 (0.0007) [2023-12-27 09:31:13,998][20381] Updated weights for policy 1, policy_version 560904 (0.0007) [2023-12-27 09:31:14,165][20347] Updated weights for policy 0, policy_version 561127 (0.0009) [2023-12-27 09:31:14,215][20347] Updated weights for policy 0, policy_version 561137 (0.0007) [2023-12-27 09:31:14,270][20347] Updated weights for policy 0, policy_version 561147 (0.0007) [2023-12-27 09:31:14,705][20381] Updated weights for policy 1, policy_version 560914 (0.0007) [2023-12-27 09:31:14,767][20381] Updated weights for policy 1, policy_version 560924 (0.0007) [2023-12-27 09:31:14,835][20381] Updated weights for policy 1, policy_version 560934 (0.0008) [2023-12-27 09:31:14,893][20381] Updated weights for policy 1, policy_version 560944 (0.0007) [2023-12-27 09:31:14,952][20347] Updated weights for policy 0, policy_version 561157 (0.0009) [2023-12-27 09:31:15,006][20347] Updated weights for policy 0, policy_version 561167 (0.0010) [2023-12-27 09:31:15,065][20347] Updated weights for policy 0, policy_version 561177 (0.0010) [2023-12-27 09:31:15,568][20381] Updated weights for policy 1, policy_version 560954 (0.0006) [2023-12-27 09:31:15,628][20381] Updated weights for policy 1, policy_version 560964 (0.0007) [2023-12-27 09:31:15,674][20381] Updated weights for policy 1, policy_version 560974 (0.0010) [2023-12-27 09:31:15,731][20347] Updated weights for policy 0, policy_version 561187 (0.0009) [2023-12-27 09:31:15,790][20347] Updated weights for policy 0, policy_version 561197 (0.0005) [2023-12-27 09:31:15,844][20347] Updated weights for policy 0, policy_version 561207 (0.0005) [2023-12-27 09:31:16,331][20381] Updated weights for policy 1, policy_version 560984 (0.0008) [2023-12-27 09:31:16,387][20381] Updated weights for policy 1, policy_version 560994 (0.0009) [2023-12-27 09:31:16,446][20381] Updated weights for policy 1, policy_version 561004 (0.0007) [2023-12-27 09:31:16,451][20347] Updated weights for policy 0, policy_version 561217 (0.0008) [2023-12-27 09:31:16,487][20291] KL-divergence is very high: 639.6699 [2023-12-27 09:31:16,502][20347] Updated weights for policy 0, policy_version 561227 (0.0010) [2023-12-27 09:31:16,557][20347] Updated weights for policy 0, policy_version 561237 (0.0006) [2023-12-27 09:31:16,613][20347] Updated weights for policy 0, policy_version 561247 (0.0005) [2023-12-27 09:31:17,161][20381] Updated weights for policy 1, policy_version 561014 (0.0007) [2023-12-27 09:31:17,214][20347] Updated weights for policy 0, policy_version 561257 (0.0005) [2023-12-27 09:31:17,217][20381] Updated weights for policy 1, policy_version 561024 (0.0007) [2023-12-27 09:31:17,272][20347] Updated weights for policy 0, policy_version 561267 (0.0006) [2023-12-27 09:31:17,276][20381] Updated weights for policy 1, policy_version 561034 (0.0008) [2023-12-27 09:31:17,322][20347] Updated weights for policy 0, policy_version 561277 (0.0006) [2023-12-27 09:31:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 287350784. Throughput: 0: 10293.5, 1: 10292.8. Samples: 287353496. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:17,363][19170] Avg episode reward: [(0, '962.660'), (1, '806.200')] [2023-12-27 09:31:17,842][20347] Updated weights for policy 0, policy_version 561287 (0.0010) [2023-12-27 09:31:17,887][20347] Updated weights for policy 0, policy_version 561297 (0.0010) [2023-12-27 09:31:17,932][20347] Updated weights for policy 0, policy_version 561307 (0.0010) [2023-12-27 09:31:18,018][20381] Updated weights for policy 1, policy_version 561044 (0.0007) [2023-12-27 09:31:18,072][20381] Updated weights for policy 1, policy_version 561054 (0.0007) [2023-12-27 09:31:18,120][20381] Updated weights for policy 1, policy_version 561064 (0.0007) [2023-12-27 09:31:18,630][20347] Updated weights for policy 0, policy_version 561317 (0.0009) [2023-12-27 09:31:18,676][20347] Updated weights for policy 0, policy_version 561327 (0.0008) [2023-12-27 09:31:18,731][20347] Updated weights for policy 0, policy_version 561337 (0.0007) [2023-12-27 09:31:18,824][20381] Updated weights for policy 1, policy_version 561074 (0.0007) [2023-12-27 09:31:18,886][20381] Updated weights for policy 1, policy_version 561084 (0.0008) [2023-12-27 09:31:18,943][20381] Updated weights for policy 1, policy_version 561094 (0.0007) [2023-12-27 09:31:19,004][20381] Updated weights for policy 1, policy_version 561104 (0.0007) [2023-12-27 09:31:19,464][20347] Updated weights for policy 0, policy_version 561347 (0.0007) [2023-12-27 09:31:19,502][20291] KL-divergence is very high: 110.0286 [2023-12-27 09:31:19,528][20347] Updated weights for policy 0, policy_version 561357 (0.0008) [2023-12-27 09:31:19,591][20347] Updated weights for policy 0, policy_version 561367 (0.0007) [2023-12-27 09:31:19,606][20291] KL-divergence is very high: 144.1666 [2023-12-27 09:31:19,613][20291] KL-divergence is very high: 215.2457 [2023-12-27 09:31:19,620][20291] KL-divergence is very high: 176.1140 [2023-12-27 09:31:19,697][20381] Updated weights for policy 1, policy_version 561114 (0.0007) [2023-12-27 09:31:19,765][20381] Updated weights for policy 1, policy_version 561124 (0.0007) [2023-12-27 09:31:19,828][20381] Updated weights for policy 1, policy_version 561134 (0.0008) [2023-12-27 09:31:20,274][20347] Updated weights for policy 0, policy_version 561377 (0.0008) [2023-12-27 09:31:20,333][20291] KL-divergence is very high: 130.4002 [2023-12-27 09:31:20,335][20347] Updated weights for policy 0, policy_version 561387 (0.0007) [2023-12-27 09:31:20,340][20291] KL-divergence is very high: 128.0969 [2023-12-27 09:31:20,392][20347] Updated weights for policy 0, policy_version 561397 (0.0008) [2023-12-27 09:31:20,452][20347] Updated weights for policy 0, policy_version 561407 (0.0008) [2023-12-27 09:31:20,485][20381] Updated weights for policy 1, policy_version 561144 (0.0007) [2023-12-27 09:31:20,543][20381] Updated weights for policy 1, policy_version 561154 (0.0007) [2023-12-27 09:31:20,601][20381] Updated weights for policy 1, policy_version 561164 (0.0008) [2023-12-27 09:31:21,143][20347] Updated weights for policy 0, policy_version 561417 (0.0008) [2023-12-27 09:31:21,203][20347] Updated weights for policy 0, policy_version 561427 (0.0007) [2023-12-27 09:31:21,271][20347] Updated weights for policy 0, policy_version 561437 (0.0007) [2023-12-27 09:31:21,294][20381] Updated weights for policy 1, policy_version 561174 (0.0008) [2023-12-27 09:31:21,362][20381] Updated weights for policy 1, policy_version 561184 (0.0009) [2023-12-27 09:31:21,423][20381] Updated weights for policy 1, policy_version 561194 (0.0007) [2023-12-27 09:31:21,907][20347] Updated weights for policy 0, policy_version 561447 (0.0007) [2023-12-27 09:31:21,961][20347] Updated weights for policy 0, policy_version 561457 (0.0008) [2023-12-27 09:31:22,018][20347] Updated weights for policy 0, policy_version 561467 (0.0008) [2023-12-27 09:31:22,086][20381] Updated weights for policy 1, policy_version 561204 (0.0009) [2023-12-27 09:31:22,149][20381] Updated weights for policy 1, policy_version 561214 (0.0008) [2023-12-27 09:31:22,214][20381] Updated weights for policy 1, policy_version 561224 (0.0008) [2023-12-27 09:31:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 287449088. Throughput: 0: 10323.8, 1: 10227.9. Samples: 287414896. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:22,363][19170] Avg episode reward: [(0, '904.780'), (1, '796.130')] [2023-12-27 09:31:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000561232_143695872.pth... [2023-12-27 09:31:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000561472_143753216.pth... [2023-12-27 09:31:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000560048_143392768.pth [2023-12-27 09:31:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000560256_143441920.pth [2023-12-27 09:31:22,768][20347] Updated weights for policy 0, policy_version 561477 (0.0007) [2023-12-27 09:31:22,825][20347] Updated weights for policy 0, policy_version 561487 (0.0008) [2023-12-27 09:31:22,880][20347] Updated weights for policy 0, policy_version 561497 (0.0007) [2023-12-27 09:31:22,921][20381] Updated weights for policy 1, policy_version 561234 (0.0009) [2023-12-27 09:31:22,978][20381] Updated weights for policy 1, policy_version 561244 (0.0011) [2023-12-27 09:31:23,038][20381] Updated weights for policy 1, policy_version 561254 (0.0011) [2023-12-27 09:31:23,096][20381] Updated weights for policy 1, policy_version 561264 (0.0010) [2023-12-27 09:31:23,609][20347] Updated weights for policy 0, policy_version 561507 (0.0007) [2023-12-27 09:31:23,671][20347] Updated weights for policy 0, policy_version 561517 (0.0005) [2023-12-27 09:31:23,709][20381] Updated weights for policy 1, policy_version 561274 (0.0007) [2023-12-27 09:31:23,722][20347] Updated weights for policy 0, policy_version 561527 (0.0006) [2023-12-27 09:31:23,768][20381] Updated weights for policy 1, policy_version 561284 (0.0010) [2023-12-27 09:31:23,827][20381] Updated weights for policy 1, policy_version 561294 (0.0010) [2023-12-27 09:31:24,374][20381] Updated weights for policy 1, policy_version 561304 (0.0011) [2023-12-27 09:31:24,399][20347] Updated weights for policy 0, policy_version 561537 (0.0005) [2023-12-27 09:31:24,428][20381] Updated weights for policy 1, policy_version 561314 (0.0008) [2023-12-27 09:31:24,452][20347] Updated weights for policy 0, policy_version 561547 (0.0006) [2023-12-27 09:31:24,482][20381] Updated weights for policy 1, policy_version 561324 (0.0011) [2023-12-27 09:31:24,505][20347] Updated weights for policy 0, policy_version 561557 (0.0005) [2023-12-27 09:31:24,554][20347] Updated weights for policy 0, policy_version 561567 (0.0007) [2023-12-27 09:31:25,063][20381] Updated weights for policy 1, policy_version 561334 (0.0009) [2023-12-27 09:31:25,119][20381] Updated weights for policy 1, policy_version 561344 (0.0007) [2023-12-27 09:31:25,175][20381] Updated weights for policy 1, policy_version 561354 (0.0008) [2023-12-27 09:31:25,259][20347] Updated weights for policy 0, policy_version 561577 (0.0008) [2023-12-27 09:31:25,315][20347] Updated weights for policy 0, policy_version 561587 (0.0007) [2023-12-27 09:31:25,373][20347] Updated weights for policy 0, policy_version 561597 (0.0007) [2023-12-27 09:31:25,866][20381] Updated weights for policy 1, policy_version 561364 (0.0009) [2023-12-27 09:31:25,916][20381] Updated weights for policy 1, policy_version 561374 (0.0009) [2023-12-27 09:31:25,961][20381] Updated weights for policy 1, policy_version 561384 (0.0010) [2023-12-27 09:31:26,072][20347] Updated weights for policy 0, policy_version 561607 (0.0007) [2023-12-27 09:31:26,123][20347] Updated weights for policy 0, policy_version 561617 (0.0007) [2023-12-27 09:31:26,177][20347] Updated weights for policy 0, policy_version 561627 (0.0007) [2023-12-27 09:31:26,657][20381] Updated weights for policy 1, policy_version 561394 (0.0011) [2023-12-27 09:31:26,709][20381] Updated weights for policy 1, policy_version 561404 (0.0011) [2023-12-27 09:31:26,759][20381] Updated weights for policy 1, policy_version 561414 (0.0011) [2023-12-27 09:31:26,807][20381] Updated weights for policy 1, policy_version 561424 (0.0010) [2023-12-27 09:31:26,869][20347] Updated weights for policy 0, policy_version 561637 (0.0007) [2023-12-27 09:31:26,922][20347] Updated weights for policy 0, policy_version 561647 (0.0007) [2023-12-27 09:31:26,976][20347] Updated weights for policy 0, policy_version 561657 (0.0008) [2023-12-27 09:31:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 287547392. Throughput: 0: 10344.6, 1: 10214.8. Samples: 287537876. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:27,363][19170] Avg episode reward: [(0, '865.480'), (1, '854.560')] [2023-12-27 09:31:27,483][20381] Updated weights for policy 1, policy_version 561434 (0.0007) [2023-12-27 09:31:27,537][20381] Updated weights for policy 1, policy_version 561444 (0.0010) [2023-12-27 09:31:27,584][20381] Updated weights for policy 1, policy_version 561454 (0.0010) [2023-12-27 09:31:27,695][20347] Updated weights for policy 0, policy_version 561667 (0.0007) [2023-12-27 09:31:27,746][20347] Updated weights for policy 0, policy_version 561677 (0.0007) [2023-12-27 09:31:27,789][20347] Updated weights for policy 0, policy_version 561687 (0.0007) [2023-12-27 09:31:28,268][20381] Updated weights for policy 1, policy_version 561464 (0.0010) [2023-12-27 09:31:28,326][20381] Updated weights for policy 1, policy_version 561474 (0.0010) [2023-12-27 09:31:28,387][20381] Updated weights for policy 1, policy_version 561484 (0.0010) [2023-12-27 09:31:28,474][20347] Updated weights for policy 0, policy_version 561697 (0.0007) [2023-12-27 09:31:28,525][20347] Updated weights for policy 0, policy_version 561707 (0.0007) [2023-12-27 09:31:28,576][20347] Updated weights for policy 0, policy_version 561717 (0.0007) [2023-12-27 09:31:28,630][20347] Updated weights for policy 0, policy_version 561727 (0.0007) [2023-12-27 09:31:29,024][20381] Updated weights for policy 1, policy_version 561494 (0.0009) [2023-12-27 09:31:29,076][20381] Updated weights for policy 1, policy_version 561504 (0.0010) [2023-12-27 09:31:29,124][20381] Updated weights for policy 1, policy_version 561514 (0.0010) [2023-12-27 09:31:29,363][20347] Updated weights for policy 0, policy_version 561737 (0.0007) [2023-12-27 09:31:29,411][20347] Updated weights for policy 0, policy_version 561747 (0.0008) [2023-12-27 09:31:29,462][20347] Updated weights for policy 0, policy_version 561758 (0.0009) [2023-12-27 09:31:29,752][20381] Updated weights for policy 1, policy_version 561524 (0.0008) [2023-12-27 09:31:29,817][20381] Updated weights for policy 1, policy_version 561534 (0.0006) [2023-12-27 09:31:29,871][20381] Updated weights for policy 1, policy_version 561544 (0.0007) [2023-12-27 09:31:30,151][20347] Updated weights for policy 0, policy_version 561768 (0.0010) [2023-12-27 09:31:30,204][20347] Updated weights for policy 0, policy_version 561778 (0.0008) [2023-12-27 09:31:30,257][20347] Updated weights for policy 0, policy_version 561788 (0.0007) [2023-12-27 09:31:30,423][20381] Updated weights for policy 1, policy_version 561554 (0.0007) [2023-12-27 09:31:30,471][20381] Updated weights for policy 1, policy_version 561564 (0.0005) [2023-12-27 09:31:30,522][20381] Updated weights for policy 1, policy_version 561574 (0.0005) [2023-12-27 09:31:30,571][20381] Updated weights for policy 1, policy_version 561584 (0.0005) [2023-12-27 09:31:31,034][20347] Updated weights for policy 0, policy_version 561798 (0.0008) [2023-12-27 09:31:31,094][20347] Updated weights for policy 0, policy_version 561808 (0.0008) [2023-12-27 09:31:31,123][20381] Updated weights for policy 1, policy_version 561594 (0.0006) [2023-12-27 09:31:31,175][20347] Updated weights for policy 0, policy_version 561818 (0.0008) [2023-12-27 09:31:31,191][20381] Updated weights for policy 1, policy_version 561604 (0.0011) [2023-12-27 09:31:31,250][20381] Updated weights for policy 1, policy_version 561614 (0.0007) [2023-12-27 09:31:31,897][20347] Updated weights for policy 0, policy_version 561828 (0.0008) [2023-12-27 09:31:31,948][20381] Updated weights for policy 1, policy_version 561624 (0.0010) [2023-12-27 09:31:31,952][20347] Updated weights for policy 0, policy_version 561838 (0.0010) [2023-12-27 09:31:31,997][20381] Updated weights for policy 1, policy_version 561634 (0.0010) [2023-12-27 09:31:32,011][20347] Updated weights for policy 0, policy_version 561848 (0.0006) [2023-12-27 09:31:32,050][20381] Updated weights for policy 1, policy_version 561644 (0.0011) [2023-12-27 09:31:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 287653888. Throughput: 0: 10302.6, 1: 10282.5. Samples: 287662136. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:32,363][19170] Avg episode reward: [(0, '836.710'), (1, '844.950')] [2023-12-27 09:31:32,661][20347] Updated weights for policy 0, policy_version 561858 (0.0007) [2023-12-27 09:31:32,720][20347] Updated weights for policy 0, policy_version 561868 (0.0007) [2023-12-27 09:31:32,766][20381] Updated weights for policy 1, policy_version 561654 (0.0008) [2023-12-27 09:31:32,780][20347] Updated weights for policy 0, policy_version 561878 (0.0008) [2023-12-27 09:31:32,812][20381] Updated weights for policy 1, policy_version 561664 (0.0005) [2023-12-27 09:31:32,829][20347] Updated weights for policy 0, policy_version 561888 (0.0007) [2023-12-27 09:31:32,859][20381] Updated weights for policy 1, policy_version 561674 (0.0007) [2023-12-27 09:31:33,388][20347] Updated weights for policy 0, policy_version 561898 (0.0005) [2023-12-27 09:31:33,403][20381] Updated weights for policy 1, policy_version 561684 (0.0006) [2023-12-27 09:31:33,441][20347] Updated weights for policy 0, policy_version 561908 (0.0005) [2023-12-27 09:31:33,451][20381] Updated weights for policy 1, policy_version 561694 (0.0006) [2023-12-27 09:31:33,499][20347] Updated weights for policy 0, policy_version 561918 (0.0005) [2023-12-27 09:31:33,503][20381] Updated weights for policy 1, policy_version 561704 (0.0006) [2023-12-27 09:31:34,057][20347] Updated weights for policy 0, policy_version 561928 (0.0005) [2023-12-27 09:31:34,088][20381] Updated weights for policy 1, policy_version 561714 (0.0005) [2023-12-27 09:31:34,112][20347] Updated weights for policy 0, policy_version 561938 (0.0005) [2023-12-27 09:31:34,142][20381] Updated weights for policy 1, policy_version 561724 (0.0008) [2023-12-27 09:31:34,172][20347] Updated weights for policy 0, policy_version 561948 (0.0006) [2023-12-27 09:31:34,200][20381] Updated weights for policy 1, policy_version 561734 (0.0007) [2023-12-27 09:31:34,254][20381] Updated weights for policy 1, policy_version 561744 (0.0006) [2023-12-27 09:31:34,861][20347] Updated weights for policy 0, policy_version 561958 (0.0008) [2023-12-27 09:31:34,908][20347] Updated weights for policy 0, policy_version 561968 (0.0007) [2023-12-27 09:31:34,934][20381] Updated weights for policy 1, policy_version 561754 (0.0010) [2023-12-27 09:31:34,962][20347] Updated weights for policy 0, policy_version 561978 (0.0005) [2023-12-27 09:31:34,990][20381] Updated weights for policy 1, policy_version 561764 (0.0011) [2023-12-27 09:31:35,040][20381] Updated weights for policy 1, policy_version 561774 (0.0008) [2023-12-27 09:31:35,604][20347] Updated weights for policy 0, policy_version 561988 (0.0005) [2023-12-27 09:31:35,656][20347] Updated weights for policy 0, policy_version 561998 (0.0005) [2023-12-27 09:31:35,712][20347] Updated weights for policy 0, policy_version 562008 (0.0006) [2023-12-27 09:31:35,720][20381] Updated weights for policy 1, policy_version 561784 (0.0010) [2023-12-27 09:31:35,782][20381] Updated weights for policy 1, policy_version 561794 (0.0010) [2023-12-27 09:31:35,841][20381] Updated weights for policy 1, policy_version 561804 (0.0011) [2023-12-27 09:31:36,390][20347] Updated weights for policy 0, policy_version 562018 (0.0006) [2023-12-27 09:31:36,448][20347] Updated weights for policy 0, policy_version 562028 (0.0007) [2023-12-27 09:31:36,465][20381] Updated weights for policy 1, policy_version 561814 (0.0008) [2023-12-27 09:31:36,502][20347] Updated weights for policy 0, policy_version 562038 (0.0007) [2023-12-27 09:31:36,523][20381] Updated weights for policy 1, policy_version 561824 (0.0006) [2023-12-27 09:31:36,556][20347] Updated weights for policy 0, policy_version 562048 (0.0007) [2023-12-27 09:31:36,582][20381] Updated weights for policy 1, policy_version 561834 (0.0006) [2023-12-27 09:31:37,228][20381] Updated weights for policy 1, policy_version 561844 (0.0008) [2023-12-27 09:31:37,274][20347] Updated weights for policy 0, policy_version 562058 (0.0007) [2023-12-27 09:31:37,276][20381] Updated weights for policy 1, policy_version 561854 (0.0005) [2023-12-27 09:31:37,323][20347] Updated weights for policy 0, policy_version 562068 (0.0006) [2023-12-27 09:31:37,325][20381] Updated weights for policy 1, policy_version 561864 (0.0005) [2023-12-27 09:31:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 287760384. Throughput: 0: 10346.8, 1: 10357.4. Samples: 287728132. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:37,363][19170] Avg episode reward: [(0, '800.480'), (1, '883.640')] [2023-12-27 09:31:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000561872_143859712.pth... [2023-12-27 09:31:37,371][20347] Updated weights for policy 0, policy_version 562078 (0.0006) [2023-12-27 09:31:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000560656_143548416.pth [2023-12-27 09:31:37,379][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000562080_143908864.pth... [2023-12-27 09:31:37,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000560864_143597568.pth [2023-12-27 09:31:38,054][20347] Updated weights for policy 0, policy_version 562088 (0.0006) [2023-12-27 09:31:38,072][20381] Updated weights for policy 1, policy_version 561874 (0.0006) [2023-12-27 09:31:38,101][20347] Updated weights for policy 0, policy_version 562098 (0.0005) [2023-12-27 09:31:38,122][20381] Updated weights for policy 1, policy_version 561884 (0.0007) [2023-12-27 09:31:38,154][20347] Updated weights for policy 0, policy_version 562108 (0.0005) [2023-12-27 09:31:38,179][20381] Updated weights for policy 1, policy_version 561894 (0.0007) [2023-12-27 09:31:38,229][20381] Updated weights for policy 1, policy_version 561904 (0.0005) [2023-12-27 09:31:38,820][20347] Updated weights for policy 0, policy_version 562118 (0.0009) [2023-12-27 09:31:38,865][20381] Updated weights for policy 1, policy_version 561914 (0.0005) [2023-12-27 09:31:38,873][20347] Updated weights for policy 0, policy_version 562128 (0.0010) [2023-12-27 09:31:38,919][20347] Updated weights for policy 0, policy_version 562138 (0.0010) [2023-12-27 09:31:38,926][20381] Updated weights for policy 1, policy_version 561924 (0.0008) [2023-12-27 09:31:38,975][20381] Updated weights for policy 1, policy_version 561934 (0.0011) [2023-12-27 09:31:39,625][20347] Updated weights for policy 0, policy_version 562148 (0.0010) [2023-12-27 09:31:39,639][20381] Updated weights for policy 1, policy_version 561944 (0.0007) [2023-12-27 09:31:39,682][20347] Updated weights for policy 0, policy_version 562158 (0.0010) [2023-12-27 09:31:39,700][20291] KL-divergence is very high: 115.7833 [2023-12-27 09:31:39,702][20381] Updated weights for policy 1, policy_version 561954 (0.0006) [2023-12-27 09:31:39,741][20347] Updated weights for policy 0, policy_version 562168 (0.0010) [2023-12-27 09:31:39,762][20381] Updated weights for policy 1, policy_version 561964 (0.0006) [2023-12-27 09:31:40,441][20381] Updated weights for policy 1, policy_version 561974 (0.0009) [2023-12-27 09:31:40,443][20347] Updated weights for policy 0, policy_version 562178 (0.0009) [2023-12-27 09:31:40,494][20381] Updated weights for policy 1, policy_version 561984 (0.0011) [2023-12-27 09:31:40,501][20347] Updated weights for policy 0, policy_version 562188 (0.0008) [2023-12-27 09:31:40,549][20381] Updated weights for policy 1, policy_version 561994 (0.0009) [2023-12-27 09:31:40,558][20347] Updated weights for policy 0, policy_version 562198 (0.0006) [2023-12-27 09:31:40,609][20347] Updated weights for policy 0, policy_version 562208 (0.0007) [2023-12-27 09:31:41,249][20381] Updated weights for policy 1, policy_version 562004 (0.0006) [2023-12-27 09:31:41,306][20381] Updated weights for policy 1, policy_version 562014 (0.0007) [2023-12-27 09:31:41,342][20291] KL-divergence is very high: 111.2133 [2023-12-27 09:31:41,356][20347] Updated weights for policy 0, policy_version 562218 (0.0008) [2023-12-27 09:31:41,374][20381] Updated weights for policy 1, policy_version 562024 (0.0007) [2023-12-27 09:31:41,419][20347] Updated weights for policy 0, policy_version 562228 (0.0007) [2023-12-27 09:31:41,468][20347] Updated weights for policy 0, policy_version 562238 (0.0008) [2023-12-27 09:31:41,995][20381] Updated weights for policy 1, policy_version 562034 (0.0007) [2023-12-27 09:31:42,057][20381] Updated weights for policy 1, policy_version 562044 (0.0005) [2023-12-27 09:31:42,116][20381] Updated weights for policy 1, policy_version 562054 (0.0005) [2023-12-27 09:31:42,176][20381] Updated weights for policy 1, policy_version 562064 (0.0005) [2023-12-27 09:31:42,239][20347] Updated weights for policy 0, policy_version 562248 (0.0007) [2023-12-27 09:31:42,302][20347] Updated weights for policy 0, policy_version 562258 (0.0008) [2023-12-27 09:31:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 287858688. Throughput: 0: 10287.6, 1: 10334.9. Samples: 287852104. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:42,363][20347] Updated weights for policy 0, policy_version 562268 (0.0008) [2023-12-27 09:31:42,363][19170] Avg episode reward: [(0, '757.010'), (1, '835.290')] [2023-12-27 09:31:42,737][20381] Updated weights for policy 1, policy_version 562074 (0.0008) [2023-12-27 09:31:42,796][20381] Updated weights for policy 1, policy_version 562084 (0.0008) [2023-12-27 09:31:42,849][20381] Updated weights for policy 1, policy_version 562094 (0.0007) [2023-12-27 09:31:43,090][20347] Updated weights for policy 0, policy_version 562278 (0.0007) [2023-12-27 09:31:43,153][20347] Updated weights for policy 0, policy_version 562288 (0.0006) [2023-12-27 09:31:43,218][20347] Updated weights for policy 0, policy_version 562298 (0.0006) [2023-12-27 09:31:43,484][20381] Updated weights for policy 1, policy_version 562104 (0.0006) [2023-12-27 09:31:43,541][20381] Updated weights for policy 1, policy_version 562114 (0.0005) [2023-12-27 09:31:43,597][20381] Updated weights for policy 1, policy_version 562124 (0.0005) [2023-12-27 09:31:43,778][20347] Updated weights for policy 0, policy_version 562308 (0.0006) [2023-12-27 09:31:43,825][20347] Updated weights for policy 0, policy_version 562318 (0.0005) [2023-12-27 09:31:43,871][20347] Updated weights for policy 0, policy_version 562328 (0.0005) [2023-12-27 09:31:44,122][20381] Updated weights for policy 1, policy_version 562134 (0.0006) [2023-12-27 09:31:44,167][20381] Updated weights for policy 1, policy_version 562144 (0.0006) [2023-12-27 09:31:44,221][20381] Updated weights for policy 1, policy_version 562154 (0.0009) [2023-12-27 09:31:44,455][20347] Updated weights for policy 0, policy_version 562338 (0.0005) [2023-12-27 09:31:44,507][20347] Updated weights for policy 0, policy_version 562348 (0.0005) [2023-12-27 09:31:44,560][20347] Updated weights for policy 0, policy_version 562358 (0.0005) [2023-12-27 09:31:44,610][20347] Updated weights for policy 0, policy_version 562368 (0.0006) [2023-12-27 09:31:44,839][20381] Updated weights for policy 1, policy_version 562164 (0.0008) [2023-12-27 09:31:44,893][20381] Updated weights for policy 1, policy_version 562174 (0.0008) [2023-12-27 09:31:44,953][20381] Updated weights for policy 1, policy_version 562184 (0.0008) [2023-12-27 09:31:45,207][20347] Updated weights for policy 0, policy_version 562378 (0.0007) [2023-12-27 09:31:45,269][20347] Updated weights for policy 0, policy_version 562388 (0.0008) [2023-12-27 09:31:45,328][20347] Updated weights for policy 0, policy_version 562398 (0.0007) [2023-12-27 09:31:45,629][20381] Updated weights for policy 1, policy_version 562194 (0.0009) [2023-12-27 09:31:45,685][20381] Updated weights for policy 1, policy_version 562204 (0.0008) [2023-12-27 09:31:45,735][20381] Updated weights for policy 1, policy_version 562214 (0.0006) [2023-12-27 09:31:45,787][20381] Updated weights for policy 1, policy_version 562224 (0.0008) [2023-12-27 09:31:46,050][20347] Updated weights for policy 0, policy_version 562408 (0.0007) [2023-12-27 09:31:46,098][20347] Updated weights for policy 0, policy_version 562418 (0.0009) [2023-12-27 09:31:46,155][20347] Updated weights for policy 0, policy_version 562428 (0.0007) [2023-12-27 09:31:46,372][20381] Updated weights for policy 1, policy_version 562234 (0.0008) [2023-12-27 09:31:46,424][20381] Updated weights for policy 1, policy_version 562244 (0.0007) [2023-12-27 09:31:46,481][20381] Updated weights for policy 1, policy_version 562254 (0.0005) [2023-12-27 09:31:46,915][20347] Updated weights for policy 0, policy_version 562438 (0.0008) [2023-12-27 09:31:46,963][20347] Updated weights for policy 0, policy_version 562448 (0.0007) [2023-12-27 09:31:47,023][20347] Updated weights for policy 0, policy_version 562458 (0.0007) [2023-12-27 09:31:47,039][20381] Updated weights for policy 1, policy_version 562264 (0.0007) [2023-12-27 09:31:47,095][20381] Updated weights for policy 1, policy_version 562274 (0.0006) [2023-12-27 09:31:47,151][20381] Updated weights for policy 1, policy_version 562284 (0.0005) [2023-12-27 09:31:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 287973376. Throughput: 0: 10241.4, 1: 10510.9. Samples: 287980620. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:47,363][19170] Avg episode reward: [(0, '620.530'), (1, '893.770')] [2023-12-27 09:31:47,754][20381] Updated weights for policy 1, policy_version 562294 (0.0008) [2023-12-27 09:31:47,756][20347] Updated weights for policy 0, policy_version 562468 (0.0007) [2023-12-27 09:31:47,804][20347] Updated weights for policy 0, policy_version 562478 (0.0007) [2023-12-27 09:31:47,810][20381] Updated weights for policy 1, policy_version 562304 (0.0006) [2023-12-27 09:31:47,854][20347] Updated weights for policy 0, policy_version 562488 (0.0007) [2023-12-27 09:31:47,862][20381] Updated weights for policy 1, policy_version 562314 (0.0008) [2023-12-27 09:31:48,442][20381] Updated weights for policy 1, policy_version 562324 (0.0006) [2023-12-27 09:31:48,496][20381] Updated weights for policy 1, policy_version 562334 (0.0005) [2023-12-27 09:31:48,548][20381] Updated weights for policy 1, policy_version 562344 (0.0008) [2023-12-27 09:31:48,613][20347] Updated weights for policy 0, policy_version 562498 (0.0007) [2023-12-27 09:31:48,676][20347] Updated weights for policy 0, policy_version 562508 (0.0009) [2023-12-27 09:31:48,739][20347] Updated weights for policy 0, policy_version 562518 (0.0008) [2023-12-27 09:31:48,800][20347] Updated weights for policy 0, policy_version 562528 (0.0008) [2023-12-27 09:31:49,178][20381] Updated weights for policy 1, policy_version 562354 (0.0008) [2023-12-27 09:31:49,230][20381] Updated weights for policy 1, policy_version 562364 (0.0008) [2023-12-27 09:31:49,284][20381] Updated weights for policy 1, policy_version 562374 (0.0007) [2023-12-27 09:31:49,338][20381] Updated weights for policy 1, policy_version 562384 (0.0007) [2023-12-27 09:31:49,528][20347] Updated weights for policy 0, policy_version 562538 (0.0008) [2023-12-27 09:31:49,577][20347] Updated weights for policy 0, policy_version 562548 (0.0007) [2023-12-27 09:31:49,628][20347] Updated weights for policy 0, policy_version 562558 (0.0007) [2023-12-27 09:31:50,048][20381] Updated weights for policy 1, policy_version 562394 (0.0008) [2023-12-27 09:31:50,095][20381] Updated weights for policy 1, policy_version 562404 (0.0007) [2023-12-27 09:31:50,145][20381] Updated weights for policy 1, policy_version 562414 (0.0007) [2023-12-27 09:31:50,290][20347] Updated weights for policy 0, policy_version 562568 (0.0006) [2023-12-27 09:31:50,354][20347] Updated weights for policy 0, policy_version 562578 (0.0008) [2023-12-27 09:31:50,417][20347] Updated weights for policy 0, policy_version 562588 (0.0008) [2023-12-27 09:31:50,849][20381] Updated weights for policy 1, policy_version 562424 (0.0006) [2023-12-27 09:31:50,906][20381] Updated weights for policy 1, policy_version 562434 (0.0006) [2023-12-27 09:31:50,959][20381] Updated weights for policy 1, policy_version 562444 (0.0007) [2023-12-27 09:31:51,142][20347] Updated weights for policy 0, policy_version 562598 (0.0007) [2023-12-27 09:31:51,195][20347] Updated weights for policy 0, policy_version 562608 (0.0007) [2023-12-27 09:31:51,259][20347] Updated weights for policy 0, policy_version 562618 (0.0007) [2023-12-27 09:31:51,649][20381] Updated weights for policy 1, policy_version 562454 (0.0009) [2023-12-27 09:31:51,709][20381] Updated weights for policy 1, policy_version 562464 (0.0008) [2023-12-27 09:31:51,773][20381] Updated weights for policy 1, policy_version 562474 (0.0009) [2023-12-27 09:31:51,949][20347] Updated weights for policy 0, policy_version 562628 (0.0007) [2023-12-27 09:31:52,014][20347] Updated weights for policy 0, policy_version 562638 (0.0006) [2023-12-27 09:31:52,079][20347] Updated weights for policy 0, policy_version 562648 (0.0006) [2023-12-27 09:31:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 288071680. Throughput: 0: 10225.1, 1: 10521.7. Samples: 288042480. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:52,363][19170] Avg episode reward: [(0, '736.340'), (1, '942.050')] [2023-12-27 09:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000562656_144056320.pth... [2023-12-27 09:31:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000561472_143753216.pth [2023-12-27 09:31:52,387][20381] Updated weights for policy 1, policy_version 562484 (0.0008) [2023-12-27 09:31:52,432][20381] Updated weights for policy 1, policy_version 562494 (0.0006) [2023-12-27 09:31:52,491][20381] Updated weights for policy 1, policy_version 562504 (0.0005) [2023-12-27 09:31:52,527][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000562512_144023552.pth... [2023-12-27 09:31:52,530][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000561232_143695872.pth [2023-12-27 09:31:52,741][20347] Updated weights for policy 0, policy_version 562658 (0.0006) [2023-12-27 09:31:52,789][20347] Updated weights for policy 0, policy_version 562668 (0.0007) [2023-12-27 09:31:52,843][20347] Updated weights for policy 0, policy_version 562678 (0.0007) [2023-12-27 09:31:52,890][20347] Updated weights for policy 0, policy_version 562688 (0.0007) [2023-12-27 09:31:53,161][20381] Updated weights for policy 1, policy_version 562514 (0.0008) [2023-12-27 09:31:53,222][20381] Updated weights for policy 1, policy_version 562524 (0.0011) [2023-12-27 09:31:53,276][20381] Updated weights for policy 1, policy_version 562534 (0.0010) [2023-12-27 09:31:53,325][20381] Updated weights for policy 1, policy_version 562544 (0.0010) [2023-12-27 09:31:53,568][20347] Updated weights for policy 0, policy_version 562698 (0.0006) [2023-12-27 09:31:53,624][20347] Updated weights for policy 0, policy_version 562708 (0.0010) [2023-12-27 09:31:53,682][20347] Updated weights for policy 0, policy_version 562718 (0.0008) [2023-12-27 09:31:54,013][20381] Updated weights for policy 1, policy_version 562554 (0.0005) [2023-12-27 09:31:54,069][20381] Updated weights for policy 1, policy_version 562564 (0.0005) [2023-12-27 09:31:54,123][20381] Updated weights for policy 1, policy_version 562574 (0.0005) [2023-12-27 09:31:54,231][20347] Updated weights for policy 0, policy_version 562728 (0.0005) [2023-12-27 09:31:54,284][20347] Updated weights for policy 0, policy_version 562738 (0.0005) [2023-12-27 09:31:54,336][20347] Updated weights for policy 0, policy_version 562748 (0.0005) [2023-12-27 09:31:54,760][20381] Updated weights for policy 1, policy_version 562584 (0.0009) [2023-12-27 09:31:54,811][20381] Updated weights for policy 1, policy_version 562594 (0.0010) [2023-12-27 09:31:54,860][20381] Updated weights for policy 1, policy_version 562604 (0.0010) [2023-12-27 09:31:54,886][20347] Updated weights for policy 0, policy_version 562758 (0.0009) [2023-12-27 09:31:54,947][20347] Updated weights for policy 0, policy_version 562768 (0.0010) [2023-12-27 09:31:55,006][20347] Updated weights for policy 0, policy_version 562778 (0.0011) [2023-12-27 09:31:55,546][20381] Updated weights for policy 1, policy_version 562614 (0.0007) [2023-12-27 09:31:55,604][20381] Updated weights for policy 1, policy_version 562624 (0.0005) [2023-12-27 09:31:55,612][20347] Updated weights for policy 0, policy_version 562788 (0.0010) [2023-12-27 09:31:55,660][20381] Updated weights for policy 1, policy_version 562634 (0.0006) [2023-12-27 09:31:55,670][20347] Updated weights for policy 0, policy_version 562798 (0.0010) [2023-12-27 09:31:55,728][20347] Updated weights for policy 0, policy_version 562808 (0.0010) [2023-12-27 09:31:56,183][20381] Updated weights for policy 1, policy_version 562644 (0.0011) [2023-12-27 09:31:56,235][20381] Updated weights for policy 1, policy_version 562654 (0.0010) [2023-12-27 09:31:56,283][20381] Updated weights for policy 1, policy_version 562664 (0.0010) [2023-12-27 09:31:56,317][20347] Updated weights for policy 0, policy_version 562818 (0.0010) [2023-12-27 09:31:56,371][20347] Updated weights for policy 0, policy_version 562828 (0.0010) [2023-12-27 09:31:56,425][20347] Updated weights for policy 0, policy_version 562838 (0.0010) [2023-12-27 09:31:56,482][20347] Updated weights for policy 0, policy_version 562848 (0.0010) [2023-12-27 09:31:56,907][20381] Updated weights for policy 1, policy_version 562674 (0.0010) [2023-12-27 09:31:56,965][20381] Updated weights for policy 1, policy_version 562684 (0.0010) [2023-12-27 09:31:57,018][20381] Updated weights for policy 1, policy_version 562694 (0.0010) [2023-12-27 09:31:57,073][20381] Updated weights for policy 1, policy_version 562704 (0.0010) [2023-12-27 09:31:57,173][20347] Updated weights for policy 0, policy_version 562858 (0.0010) [2023-12-27 09:31:57,235][20347] Updated weights for policy 0, policy_version 562868 (0.0010) [2023-12-27 09:31:57,284][20347] Updated weights for policy 0, policy_version 562878 (0.0010) [2023-12-27 09:31:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 288186368. Throughput: 0: 10278.3, 1: 10621.5. Samples: 288171424. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:31:57,363][19170] Avg episode reward: [(0, '830.660'), (1, '951.610')] [2023-12-27 09:31:57,766][20381] Updated weights for policy 1, policy_version 562714 (0.0010) [2023-12-27 09:31:57,814][20381] Updated weights for policy 1, policy_version 562724 (0.0010) [2023-12-27 09:31:57,862][20347] Updated weights for policy 0, policy_version 562888 (0.0006) [2023-12-27 09:31:57,868][20381] Updated weights for policy 1, policy_version 562734 (0.0008) [2023-12-27 09:31:57,907][20347] Updated weights for policy 0, policy_version 562898 (0.0006) [2023-12-27 09:31:57,951][20347] Updated weights for policy 0, policy_version 562908 (0.0008) [2023-12-27 09:31:58,543][20381] Updated weights for policy 1, policy_version 562744 (0.0007) [2023-12-27 09:31:58,596][20381] Updated weights for policy 1, policy_version 562754 (0.0008) [2023-12-27 09:31:58,652][20381] Updated weights for policy 1, policy_version 562764 (0.0008) [2023-12-27 09:31:58,667][20347] Updated weights for policy 0, policy_version 562919 (0.0008) [2023-12-27 09:31:58,730][20347] Updated weights for policy 0, policy_version 562929 (0.0007) [2023-12-27 09:31:58,801][20347] Updated weights for policy 0, policy_version 562939 (0.0008) [2023-12-27 09:31:59,440][20381] Updated weights for policy 1, policy_version 562774 (0.0010) [2023-12-27 09:31:59,460][20347] Updated weights for policy 0, policy_version 562949 (0.0009) [2023-12-27 09:31:59,463][20291] KL-divergence is very high: 152.1328 [2023-12-27 09:31:59,487][20291] KL-divergence is very high: 308.8954 [2023-12-27 09:31:59,494][20381] Updated weights for policy 1, policy_version 562784 (0.0011) [2023-12-27 09:31:59,505][20291] KL-divergence is very high: 249.5383 [2023-12-27 09:31:59,511][20291] KL-divergence is very high: 259.9007 [2023-12-27 09:31:59,515][20347] Updated weights for policy 0, policy_version 562959 (0.0007) [2023-12-27 09:31:59,556][20381] Updated weights for policy 1, policy_version 562794 (0.0011) [2023-12-27 09:31:59,575][20347] Updated weights for policy 0, policy_version 562969 (0.0006) [2023-12-27 09:32:00,145][20381] Updated weights for policy 1, policy_version 562804 (0.0011) [2023-12-27 09:32:00,195][20381] Updated weights for policy 1, policy_version 562814 (0.0010) [2023-12-27 09:32:00,236][20347] Updated weights for policy 0, policy_version 562979 (0.0006) [2023-12-27 09:32:00,256][20381] Updated weights for policy 1, policy_version 562824 (0.0010) [2023-12-27 09:32:00,297][20347] Updated weights for policy 0, policy_version 562989 (0.0006) [2023-12-27 09:32:00,349][20347] Updated weights for policy 0, policy_version 562999 (0.0006) [2023-12-27 09:32:00,947][20381] Updated weights for policy 1, policy_version 562834 (0.0010) [2023-12-27 09:32:00,995][20381] Updated weights for policy 1, policy_version 562844 (0.0010) [2023-12-27 09:32:01,037][20347] Updated weights for policy 0, policy_version 563009 (0.0007) [2023-12-27 09:32:01,052][20381] Updated weights for policy 1, policy_version 562854 (0.0009) [2023-12-27 09:32:01,095][20347] Updated weights for policy 0, policy_version 563019 (0.0008) [2023-12-27 09:32:01,109][20381] Updated weights for policy 1, policy_version 562864 (0.0008) [2023-12-27 09:32:01,154][20347] Updated weights for policy 0, policy_version 563029 (0.0008) [2023-12-27 09:32:01,202][20347] Updated weights for policy 0, policy_version 563039 (0.0007) [2023-12-27 09:32:01,893][20381] Updated weights for policy 1, policy_version 562874 (0.0008) [2023-12-27 09:32:01,911][20347] Updated weights for policy 0, policy_version 563049 (0.0006) [2023-12-27 09:32:01,948][20381] Updated weights for policy 1, policy_version 562884 (0.0008) [2023-12-27 09:32:01,973][20347] Updated weights for policy 0, policy_version 563059 (0.0007) [2023-12-27 09:32:02,007][20381] Updated weights for policy 1, policy_version 562894 (0.0009) [2023-12-27 09:32:02,034][20347] Updated weights for policy 0, policy_version 563069 (0.0009) [2023-12-27 09:32:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 288284672. Throughput: 0: 10281.1, 1: 10619.2. Samples: 288294008. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:32:02,363][19170] Avg episode reward: [(0, '783.260'), (1, '961.210')] [2023-12-27 09:32:02,690][20381] Updated weights for policy 1, policy_version 562904 (0.0007) [2023-12-27 09:32:02,740][20347] Updated weights for policy 0, policy_version 563079 (0.0006) [2023-12-27 09:32:02,753][20381] Updated weights for policy 1, policy_version 562914 (0.0006) [2023-12-27 09:32:02,799][20347] Updated weights for policy 0, policy_version 563089 (0.0006) [2023-12-27 09:32:02,812][20381] Updated weights for policy 1, policy_version 562924 (0.0005) [2023-12-27 09:32:02,861][20347] Updated weights for policy 0, policy_version 563099 (0.0007) [2023-12-27 09:32:03,476][20347] Updated weights for policy 0, policy_version 563109 (0.0007) [2023-12-27 09:32:03,504][20381] Updated weights for policy 1, policy_version 562934 (0.0006) [2023-12-27 09:32:03,527][20347] Updated weights for policy 0, policy_version 563119 (0.0007) [2023-12-27 09:32:03,540][20291] KL-divergence is very high: 122.2912 [2023-12-27 09:32:03,558][20381] Updated weights for policy 1, policy_version 562944 (0.0006) [2023-12-27 09:32:03,579][20347] Updated weights for policy 0, policy_version 563129 (0.0008) [2023-12-27 09:32:03,602][20381] Updated weights for policy 1, policy_version 562954 (0.0005) [2023-12-27 09:32:04,265][20381] Updated weights for policy 1, policy_version 562964 (0.0005) [2023-12-27 09:32:04,287][20347] Updated weights for policy 0, policy_version 563139 (0.0009) [2023-12-27 09:32:04,322][20381] Updated weights for policy 1, policy_version 562974 (0.0007) [2023-12-27 09:32:04,339][20347] Updated weights for policy 0, policy_version 563149 (0.0010) [2023-12-27 09:32:04,383][20381] Updated weights for policy 1, policy_version 562984 (0.0006) [2023-12-27 09:32:04,392][20347] Updated weights for policy 0, policy_version 563159 (0.0010) [2023-12-27 09:32:05,028][20381] Updated weights for policy 1, policy_version 562994 (0.0006) [2023-12-27 09:32:05,085][20381] Updated weights for policy 1, policy_version 563004 (0.0005) [2023-12-27 09:32:05,126][20347] Updated weights for policy 0, policy_version 563169 (0.0010) [2023-12-27 09:32:05,146][20381] Updated weights for policy 1, policy_version 563014 (0.0007) [2023-12-27 09:32:05,183][20347] Updated weights for policy 0, policy_version 563179 (0.0006) [2023-12-27 09:32:05,197][20381] Updated weights for policy 1, policy_version 563024 (0.0006) [2023-12-27 09:32:05,227][20347] Updated weights for policy 0, policy_version 563189 (0.0007) [2023-12-27 09:32:05,272][20347] Updated weights for policy 0, policy_version 563199 (0.0007) [2023-12-27 09:32:05,858][20381] Updated weights for policy 1, policy_version 563034 (0.0007) [2023-12-27 09:32:05,913][20381] Updated weights for policy 1, policy_version 563044 (0.0007) [2023-12-27 09:32:05,920][20291] KL-divergence is very high: 124.4028 [2023-12-27 09:32:05,938][20291] KL-divergence is very high: 153.2169 [2023-12-27 09:32:05,957][20291] KL-divergence is very high: 326.5506 [2023-12-27 09:32:05,963][20347] Updated weights for policy 0, policy_version 563209 (0.0006) [2023-12-27 09:32:05,963][20291] KL-divergence is very high: 346.9482 [2023-12-27 09:32:05,968][20381] Updated weights for policy 1, policy_version 563054 (0.0006) [2023-12-27 09:32:06,021][20347] Updated weights for policy 0, policy_version 563219 (0.0007) [2023-12-27 09:32:06,075][20347] Updated weights for policy 0, policy_version 563229 (0.0008) [2023-12-27 09:32:06,681][20381] Updated weights for policy 1, policy_version 563064 (0.0008) [2023-12-27 09:32:06,733][20381] Updated weights for policy 1, policy_version 563074 (0.0008) [2023-12-27 09:32:06,783][20381] Updated weights for policy 1, policy_version 563084 (0.0007) [2023-12-27 09:32:06,793][20347] Updated weights for policy 0, policy_version 563239 (0.0006) [2023-12-27 09:32:06,850][20347] Updated weights for policy 0, policy_version 563249 (0.0007) [2023-12-27 09:32:06,908][20347] Updated weights for policy 0, policy_version 563259 (0.0008) [2023-12-27 09:32:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 288382976. Throughput: 0: 10248.0, 1: 10651.9. Samples: 288355388. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:32:07,363][19170] Avg episode reward: [(0, '572.910'), (1, '971.040')] [2023-12-27 09:32:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000563088_144171008.pth... [2023-12-27 09:32:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000563264_144211968.pth... [2023-12-27 09:32:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000561872_143859712.pth [2023-12-27 09:32:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000562080_143908864.pth [2023-12-27 09:32:07,451][20381] Updated weights for policy 1, policy_version 563094 (0.0006) [2023-12-27 09:32:07,514][20381] Updated weights for policy 1, policy_version 563104 (0.0006) [2023-12-27 09:32:07,532][20347] Updated weights for policy 0, policy_version 563269 (0.0006) [2023-12-27 09:32:07,567][20381] Updated weights for policy 1, policy_version 563114 (0.0007) [2023-12-27 09:32:07,589][20347] Updated weights for policy 0, policy_version 563279 (0.0007) [2023-12-27 09:32:07,651][20347] Updated weights for policy 0, policy_version 563289 (0.0008) [2023-12-27 09:32:08,124][20381] Updated weights for policy 1, policy_version 563124 (0.0006) [2023-12-27 09:32:08,182][20381] Updated weights for policy 1, policy_version 563134 (0.0008) [2023-12-27 09:32:08,228][20381] Updated weights for policy 1, policy_version 563144 (0.0005) [2023-12-27 09:32:08,314][20347] Updated weights for policy 0, policy_version 563299 (0.0008) [2023-12-27 09:32:08,378][20347] Updated weights for policy 0, policy_version 563309 (0.0010) [2023-12-27 09:32:08,431][20347] Updated weights for policy 0, policy_version 563319 (0.0007) [2023-12-27 09:32:08,922][20381] Updated weights for policy 1, policy_version 563154 (0.0006) [2023-12-27 09:32:08,983][20381] Updated weights for policy 1, policy_version 563164 (0.0007) [2023-12-27 09:32:09,032][20381] Updated weights for policy 1, policy_version 563174 (0.0008) [2023-12-27 09:32:09,052][20347] Updated weights for policy 0, policy_version 563329 (0.0008) [2023-12-27 09:32:09,085][20381] Updated weights for policy 1, policy_version 563184 (0.0007) [2023-12-27 09:32:09,102][20347] Updated weights for policy 0, policy_version 563339 (0.0006) [2023-12-27 09:32:09,161][20347] Updated weights for policy 0, policy_version 563349 (0.0007) [2023-12-27 09:32:09,226][20347] Updated weights for policy 0, policy_version 563359 (0.0008) [2023-12-27 09:32:09,734][20381] Updated weights for policy 1, policy_version 563194 (0.0008) [2023-12-27 09:32:09,798][20381] Updated weights for policy 1, policy_version 563204 (0.0008) [2023-12-27 09:32:09,851][20347] Updated weights for policy 0, policy_version 563369 (0.0007) [2023-12-27 09:32:09,862][20381] Updated weights for policy 1, policy_version 563214 (0.0008) [2023-12-27 09:32:09,909][20347] Updated weights for policy 0, policy_version 563379 (0.0008) [2023-12-27 09:32:09,973][20347] Updated weights for policy 0, policy_version 563389 (0.0007) [2023-12-27 09:32:10,547][20381] Updated weights for policy 1, policy_version 563224 (0.0008) [2023-12-27 09:32:10,598][20381] Updated weights for policy 1, policy_version 563234 (0.0007) [2023-12-27 09:32:10,627][20347] Updated weights for policy 0, policy_version 563399 (0.0007) [2023-12-27 09:32:10,656][20381] Updated weights for policy 1, policy_version 563244 (0.0007) [2023-12-27 09:32:10,677][20347] Updated weights for policy 0, policy_version 563409 (0.0005) [2023-12-27 09:32:10,725][20347] Updated weights for policy 0, policy_version 563419 (0.0007) [2023-12-27 09:32:11,380][20381] Updated weights for policy 1, policy_version 563254 (0.0007) [2023-12-27 09:32:11,440][20381] Updated weights for policy 1, policy_version 563264 (0.0008) [2023-12-27 09:32:11,452][20347] Updated weights for policy 0, policy_version 563429 (0.0008) [2023-12-27 09:32:11,495][20381] Updated weights for policy 1, policy_version 563274 (0.0008) [2023-12-27 09:32:11,518][20347] Updated weights for policy 0, policy_version 563439 (0.0007) [2023-12-27 09:32:11,580][20347] Updated weights for policy 0, policy_version 563449 (0.0008) [2023-12-27 09:32:12,215][20381] Updated weights for policy 1, policy_version 563284 (0.0008) [2023-12-27 09:32:12,235][20347] Updated weights for policy 0, policy_version 563459 (0.0008) [2023-12-27 09:32:12,277][20381] Updated weights for policy 1, policy_version 563294 (0.0009) [2023-12-27 09:32:12,296][20347] Updated weights for policy 0, policy_version 563469 (0.0008) [2023-12-27 09:32:12,337][20381] Updated weights for policy 1, policy_version 563304 (0.0008) [2023-12-27 09:32:12,361][20347] Updated weights for policy 0, policy_version 563479 (0.0007) [2023-12-27 09:32:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 288481280. Throughput: 0: 10300.6, 1: 10636.6. Samples: 288480052. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-12-27 09:32:12,363][19170] Avg episode reward: [(0, '677.960'), (1, '990.400')] [2023-12-27 09:32:13,001][20347] Updated weights for policy 0, policy_version 563489 (0.0006) [2023-12-27 09:32:13,043][20381] Updated weights for policy 1, policy_version 563314 (0.0008) [2023-12-27 09:32:13,057][20347] Updated weights for policy 0, policy_version 563499 (0.0006) [2023-12-27 09:32:13,092][20381] Updated weights for policy 1, policy_version 563324 (0.0006) [2023-12-27 09:32:13,116][20347] Updated weights for policy 0, policy_version 563509 (0.0006) [2023-12-27 09:32:13,148][20381] Updated weights for policy 1, policy_version 563334 (0.0006) [2023-12-27 09:32:13,178][20347] Updated weights for policy 0, policy_version 563519 (0.0007) [2023-12-27 09:32:13,201][20381] Updated weights for policy 1, policy_version 563344 (0.0006) [2023-12-27 09:32:13,825][20381] Updated weights for policy 1, policy_version 563354 (0.0005) [2023-12-27 09:32:13,873][20347] Updated weights for policy 0, policy_version 563529 (0.0007) [2023-12-27 09:32:13,902][20381] Updated weights for policy 1, policy_version 563364 (0.0005) [2023-12-27 09:32:13,920][20347] Updated weights for policy 0, policy_version 563539 (0.0008) [2023-12-27 09:32:13,951][20381] Updated weights for policy 1, policy_version 563374 (0.0005) [2023-12-27 09:32:13,969][20347] Updated weights for policy 0, policy_version 563549 (0.0007) [2023-12-27 09:32:14,505][20381] Updated weights for policy 1, policy_version 563384 (0.0009) [2023-12-27 09:32:14,559][20381] Updated weights for policy 1, policy_version 563394 (0.0011) [2023-12-27 09:32:14,614][20381] Updated weights for policy 1, policy_version 563404 (0.0011) [2023-12-27 09:32:14,718][20347] Updated weights for policy 0, policy_version 563559 (0.0008) [2023-12-27 09:32:14,767][20347] Updated weights for policy 0, policy_version 563569 (0.0007) [2023-12-27 09:32:14,829][20347] Updated weights for policy 0, policy_version 563579 (0.0007) [2023-12-27 09:32:15,247][20381] Updated weights for policy 1, policy_version 563414 (0.0010) [2023-12-27 09:32:15,312][20381] Updated weights for policy 1, policy_version 563424 (0.0011) [2023-12-27 09:32:15,371][20381] Updated weights for policy 1, policy_version 563434 (0.0011) [2023-12-27 09:32:15,552][20347] Updated weights for policy 0, policy_version 563589 (0.0006) [2023-12-27 09:32:15,609][20347] Updated weights for policy 0, policy_version 563599 (0.0005) [2023-12-27 09:32:15,662][20347] Updated weights for policy 0, policy_version 563609 (0.0005) [2023-12-27 09:32:16,015][20381] Updated weights for policy 1, policy_version 563444 (0.0008) [2023-12-27 09:32:16,065][20381] Updated weights for policy 1, policy_version 563454 (0.0005) [2023-12-27 09:32:16,118][20381] Updated weights for policy 1, policy_version 563464 (0.0005) [2023-12-27 09:32:16,328][20347] Updated weights for policy 0, policy_version 563619 (0.0007) [2023-12-27 09:32:16,386][20347] Updated weights for policy 0, policy_version 563629 (0.0008) [2023-12-27 09:32:16,441][20347] Updated weights for policy 0, policy_version 563639 (0.0008) [2023-12-27 09:32:16,732][20381] Updated weights for policy 1, policy_version 563474 (0.0006) [2023-12-27 09:32:16,789][20381] Updated weights for policy 1, policy_version 563484 (0.0007) [2023-12-27 09:32:16,845][20381] Updated weights for policy 1, policy_version 563494 (0.0008) [2023-12-27 09:32:16,898][20381] Updated weights for policy 1, policy_version 563504 (0.0008) [2023-12-27 09:32:17,042][20347] Updated weights for policy 0, policy_version 563649 (0.0007) [2023-12-27 09:32:17,099][20347] Updated weights for policy 0, policy_version 563659 (0.0008) [2023-12-27 09:32:17,147][20347] Updated weights for policy 0, policy_version 563669 (0.0008) [2023-12-27 09:32:17,193][20347] Updated weights for policy 0, policy_version 563679 (0.0007) [2023-12-27 09:32:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 288595968. Throughput: 0: 10353.9, 1: 10591.8. Samples: 288604696. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:17,363][19170] Avg episode reward: [(0, '762.500'), (1, '990.330')] [2023-12-27 09:32:17,604][20381] Updated weights for policy 1, policy_version 563514 (0.0010) [2023-12-27 09:32:17,656][20381] Updated weights for policy 1, policy_version 563524 (0.0010) [2023-12-27 09:32:17,714][20381] Updated weights for policy 1, policy_version 563534 (0.0006) [2023-12-27 09:32:17,914][20347] Updated weights for policy 0, policy_version 563689 (0.0007) [2023-12-27 09:32:17,965][20347] Updated weights for policy 0, policy_version 563699 (0.0008) [2023-12-27 09:32:18,022][20347] Updated weights for policy 0, policy_version 563709 (0.0008) [2023-12-27 09:32:18,328][20381] Updated weights for policy 1, policy_version 563544 (0.0007) [2023-12-27 09:32:18,388][20381] Updated weights for policy 1, policy_version 563554 (0.0007) [2023-12-27 09:32:18,435][20381] Updated weights for policy 1, policy_version 563564 (0.0006) [2023-12-27 09:32:18,694][20347] Updated weights for policy 0, policy_version 563719 (0.0009) [2023-12-27 09:32:18,750][20347] Updated weights for policy 0, policy_version 563729 (0.0008) [2023-12-27 09:32:18,810][20347] Updated weights for policy 0, policy_version 563739 (0.0008) [2023-12-27 09:32:19,093][20381] Updated weights for policy 1, policy_version 563574 (0.0007) [2023-12-27 09:32:19,146][20381] Updated weights for policy 1, policy_version 563584 (0.0008) [2023-12-27 09:32:19,200][20381] Updated weights for policy 1, policy_version 563594 (0.0007) [2023-12-27 09:32:19,552][20347] Updated weights for policy 0, policy_version 563749 (0.0007) [2023-12-27 09:32:19,609][20347] Updated weights for policy 0, policy_version 563759 (0.0005) [2023-12-27 09:32:19,656][20347] Updated weights for policy 0, policy_version 563769 (0.0008) [2023-12-27 09:32:19,884][20381] Updated weights for policy 1, policy_version 563604 (0.0007) [2023-12-27 09:32:19,950][20381] Updated weights for policy 1, policy_version 563614 (0.0009) [2023-12-27 09:32:20,013][20381] Updated weights for policy 1, policy_version 563624 (0.0008) [2023-12-27 09:32:20,333][20347] Updated weights for policy 0, policy_version 563779 (0.0006) [2023-12-27 09:32:20,394][20347] Updated weights for policy 0, policy_version 563789 (0.0007) [2023-12-27 09:32:20,453][20347] Updated weights for policy 0, policy_version 563799 (0.0008) [2023-12-27 09:32:20,651][20381] Updated weights for policy 1, policy_version 563634 (0.0007) [2023-12-27 09:32:20,711][20381] Updated weights for policy 1, policy_version 563644 (0.0007) [2023-12-27 09:32:20,774][20381] Updated weights for policy 1, policy_version 563654 (0.0008) [2023-12-27 09:32:20,834][20381] Updated weights for policy 1, policy_version 563664 (0.0008) [2023-12-27 09:32:21,230][20347] Updated weights for policy 0, policy_version 563809 (0.0008) [2023-12-27 09:32:21,299][20347] Updated weights for policy 0, policy_version 563819 (0.0011) [2023-12-27 09:32:21,365][20347] Updated weights for policy 0, policy_version 563829 (0.0009) [2023-12-27 09:32:21,435][20347] Updated weights for policy 0, policy_version 563839 (0.0009) [2023-12-27 09:32:21,510][20381] Updated weights for policy 1, policy_version 563674 (0.0007) [2023-12-27 09:32:21,575][20381] Updated weights for policy 1, policy_version 563684 (0.0008) [2023-12-27 09:32:21,632][20381] Updated weights for policy 1, policy_version 563694 (0.0009) [2023-12-27 09:32:22,161][20347] Updated weights for policy 0, policy_version 563849 (0.0008) [2023-12-27 09:32:22,227][20347] Updated weights for policy 0, policy_version 563859 (0.0007) [2023-12-27 09:32:22,241][20381] Updated weights for policy 1, policy_version 563704 (0.0008) [2023-12-27 09:32:22,288][20347] Updated weights for policy 0, policy_version 563869 (0.0007) [2023-12-27 09:32:22,302][20381] Updated weights for policy 1, policy_version 563714 (0.0006) [2023-12-27 09:32:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 288694272. Throughput: 0: 10286.5, 1: 10560.6. Samples: 288666252. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:22,363][20381] Updated weights for policy 1, policy_version 563724 (0.0008) [2023-12-27 09:32:22,363][19170] Avg episode reward: [(0, '827.360'), (1, '943.020')] [2023-12-27 09:32:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000563872_144367616.pth... [2023-12-27 09:32:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000562656_144056320.pth [2023-12-27 09:32:22,386][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000563728_144334848.pth... [2023-12-27 09:32:22,394][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000562512_144023552.pth [2023-12-27 09:32:22,967][20347] Updated weights for policy 0, policy_version 563879 (0.0006) [2023-12-27 09:32:23,022][20347] Updated weights for policy 0, policy_version 563889 (0.0005) [2023-12-27 09:32:23,065][20381] Updated weights for policy 1, policy_version 563734 (0.0008) [2023-12-27 09:32:23,073][20347] Updated weights for policy 0, policy_version 563899 (0.0005) [2023-12-27 09:32:23,118][20381] Updated weights for policy 1, policy_version 563744 (0.0008) [2023-12-27 09:32:23,166][20381] Updated weights for policy 1, policy_version 563754 (0.0008) [2023-12-27 09:32:23,696][20347] Updated weights for policy 0, policy_version 563909 (0.0006) [2023-12-27 09:32:23,754][20347] Updated weights for policy 0, policy_version 563919 (0.0005) [2023-12-27 09:32:23,819][20347] Updated weights for policy 0, policy_version 563929 (0.0008) [2023-12-27 09:32:23,880][20381] Updated weights for policy 1, policy_version 563764 (0.0008) [2023-12-27 09:32:23,931][20381] Updated weights for policy 1, policy_version 563774 (0.0007) [2023-12-27 09:32:23,977][20381] Updated weights for policy 1, policy_version 563784 (0.0008) [2023-12-27 09:32:24,504][20347] Updated weights for policy 0, policy_version 563939 (0.0007) [2023-12-27 09:32:24,556][20347] Updated weights for policy 0, policy_version 563949 (0.0008) [2023-12-27 09:32:24,588][20381] Updated weights for policy 1, policy_version 563794 (0.0007) [2023-12-27 09:32:24,602][20347] Updated weights for policy 0, policy_version 563959 (0.0008) [2023-12-27 09:32:24,646][20381] Updated weights for policy 1, policy_version 563804 (0.0006) [2023-12-27 09:32:24,698][20381] Updated weights for policy 1, policy_version 563814 (0.0006) [2023-12-27 09:32:24,757][20381] Updated weights for policy 1, policy_version 563824 (0.0007) [2023-12-27 09:32:25,290][20381] Updated weights for policy 1, policy_version 563834 (0.0005) [2023-12-27 09:32:25,322][20347] Updated weights for policy 0, policy_version 563969 (0.0006) [2023-12-27 09:32:25,347][20381] Updated weights for policy 1, policy_version 563844 (0.0005) [2023-12-27 09:32:25,380][20347] Updated weights for policy 0, policy_version 563979 (0.0008) [2023-12-27 09:32:25,407][20381] Updated weights for policy 1, policy_version 563854 (0.0009) [2023-12-27 09:32:25,434][20347] Updated weights for policy 0, policy_version 563989 (0.0006) [2023-12-27 09:32:25,488][20347] Updated weights for policy 0, policy_version 563999 (0.0007) [2023-12-27 09:32:26,042][20381] Updated weights for policy 1, policy_version 563864 (0.0011) [2023-12-27 09:32:26,066][20347] Updated weights for policy 0, policy_version 564009 (0.0006) [2023-12-27 09:32:26,094][20381] Updated weights for policy 1, policy_version 563874 (0.0008) [2023-12-27 09:32:26,115][20347] Updated weights for policy 0, policy_version 564019 (0.0007) [2023-12-27 09:32:26,150][20381] Updated weights for policy 1, policy_version 563884 (0.0005) [2023-12-27 09:32:26,172][20347] Updated weights for policy 0, policy_version 564029 (0.0007) [2023-12-27 09:32:26,798][20381] Updated weights for policy 1, policy_version 563894 (0.0006) [2023-12-27 09:32:26,853][20381] Updated weights for policy 1, policy_version 563904 (0.0011) [2023-12-27 09:32:26,867][20347] Updated weights for policy 0, policy_version 564039 (0.0008) [2023-12-27 09:32:26,899][20381] Updated weights for policy 1, policy_version 563914 (0.0010) [2023-12-27 09:32:26,919][20347] Updated weights for policy 0, policy_version 564049 (0.0005) [2023-12-27 09:32:26,973][20347] Updated weights for policy 0, policy_version 564059 (0.0006) [2023-12-27 09:32:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 288800768. Throughput: 0: 10258.2, 1: 10613.3. Samples: 288791316. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:27,363][19170] Avg episode reward: [(0, '904.680'), (1, '962.420')] [2023-12-27 09:32:27,549][20347] Updated weights for policy 0, policy_version 564069 (0.0007) [2023-12-27 09:32:27,572][20381] Updated weights for policy 1, policy_version 563924 (0.0010) [2023-12-27 09:32:27,607][20347] Updated weights for policy 0, policy_version 564079 (0.0006) [2023-12-27 09:32:27,628][20381] Updated weights for policy 1, policy_version 563934 (0.0011) [2023-12-27 09:32:27,663][20347] Updated weights for policy 0, policy_version 564089 (0.0005) [2023-12-27 09:32:27,684][20381] Updated weights for policy 1, policy_version 563944 (0.0011) [2023-12-27 09:32:28,264][20347] Updated weights for policy 0, policy_version 564099 (0.0006) [2023-12-27 09:32:28,324][20347] Updated weights for policy 0, policy_version 564109 (0.0007) [2023-12-27 09:32:28,364][20381] Updated weights for policy 1, policy_version 563954 (0.0010) [2023-12-27 09:32:28,382][20347] Updated weights for policy 0, policy_version 564119 (0.0008) [2023-12-27 09:32:28,420][20381] Updated weights for policy 1, policy_version 563964 (0.0007) [2023-12-27 09:32:28,475][20381] Updated weights for policy 1, policy_version 563974 (0.0008) [2023-12-27 09:32:28,520][20381] Updated weights for policy 1, policy_version 563984 (0.0010) [2023-12-27 09:32:29,028][20347] Updated weights for policy 0, policy_version 564129 (0.0009) [2023-12-27 09:32:29,082][20347] Updated weights for policy 0, policy_version 564139 (0.0010) [2023-12-27 09:32:29,130][20347] Updated weights for policy 0, policy_version 564149 (0.0010) [2023-12-27 09:32:29,149][20381] Updated weights for policy 1, policy_version 563994 (0.0005) [2023-12-27 09:32:29,191][20347] Updated weights for policy 0, policy_version 564159 (0.0010) [2023-12-27 09:32:29,197][20381] Updated weights for policy 1, policy_version 564004 (0.0006) [2023-12-27 09:32:29,250][20381] Updated weights for policy 1, policy_version 564014 (0.0008) [2023-12-27 09:32:29,829][20347] Updated weights for policy 0, policy_version 564169 (0.0008) [2023-12-27 09:32:29,837][20381] Updated weights for policy 1, policy_version 564024 (0.0009) [2023-12-27 09:32:29,886][20347] Updated weights for policy 0, policy_version 564179 (0.0010) [2023-12-27 09:32:29,901][20381] Updated weights for policy 1, policy_version 564034 (0.0006) [2023-12-27 09:32:29,944][20347] Updated weights for policy 0, policy_version 564189 (0.0008) [2023-12-27 09:32:29,961][20381] Updated weights for policy 1, policy_version 564044 (0.0007) [2023-12-27 09:32:30,556][20347] Updated weights for policy 0, policy_version 564199 (0.0009) [2023-12-27 09:32:30,611][20347] Updated weights for policy 0, policy_version 564209 (0.0008) [2023-12-27 09:32:30,629][20381] Updated weights for policy 1, policy_version 564054 (0.0006) [2023-12-27 09:32:30,669][20347] Updated weights for policy 0, policy_version 564219 (0.0007) [2023-12-27 09:32:30,688][20381] Updated weights for policy 1, policy_version 564064 (0.0005) [2023-12-27 09:32:30,746][20381] Updated weights for policy 1, policy_version 564074 (0.0005) [2023-12-27 09:32:31,298][20381] Updated weights for policy 1, policy_version 564084 (0.0006) [2023-12-27 09:32:31,358][20381] Updated weights for policy 1, policy_version 564094 (0.0007) [2023-12-27 09:32:31,418][20381] Updated weights for policy 1, policy_version 564104 (0.0007) [2023-12-27 09:32:31,422][20347] Updated weights for policy 0, policy_version 564229 (0.0007) [2023-12-27 09:32:31,475][20347] Updated weights for policy 0, policy_version 564239 (0.0008) [2023-12-27 09:32:31,526][20347] Updated weights for policy 0, policy_version 564249 (0.0008) [2023-12-27 09:32:32,043][20381] Updated weights for policy 1, policy_version 564114 (0.0006) [2023-12-27 09:32:32,100][20381] Updated weights for policy 1, policy_version 564124 (0.0007) [2023-12-27 09:32:32,155][20381] Updated weights for policy 1, policy_version 564134 (0.0007) [2023-12-27 09:32:32,212][20381] Updated weights for policy 1, policy_version 564144 (0.0007) [2023-12-27 09:32:32,273][20347] Updated weights for policy 0, policy_version 564259 (0.0008) [2023-12-27 09:32:32,329][20347] Updated weights for policy 0, policy_version 564269 (0.0008) [2023-12-27 09:32:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 288907264. Throughput: 0: 10302.7, 1: 10548.5. Samples: 288918928. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:32,363][19170] Avg episode reward: [(0, '963.100'), (1, '962.090')] [2023-12-27 09:32:32,389][20347] Updated weights for policy 0, policy_version 564279 (0.0008) [2023-12-27 09:32:32,797][20381] Updated weights for policy 1, policy_version 564154 (0.0011) [2023-12-27 09:32:32,853][20381] Updated weights for policy 1, policy_version 564164 (0.0010) [2023-12-27 09:32:32,913][20381] Updated weights for policy 1, policy_version 564174 (0.0011) [2023-12-27 09:32:33,132][20347] Updated weights for policy 0, policy_version 564289 (0.0007) [2023-12-27 09:32:33,189][20347] Updated weights for policy 0, policy_version 564299 (0.0007) [2023-12-27 09:32:33,240][20347] Updated weights for policy 0, policy_version 564309 (0.0007) [2023-12-27 09:32:33,291][20347] Updated weights for policy 0, policy_version 564319 (0.0008) [2023-12-27 09:32:33,587][20381] Updated weights for policy 1, policy_version 564184 (0.0006) [2023-12-27 09:32:33,640][20381] Updated weights for policy 1, policy_version 564194 (0.0007) [2023-12-27 09:32:33,700][20381] Updated weights for policy 1, policy_version 564204 (0.0005) [2023-12-27 09:32:33,883][20347] Updated weights for policy 0, policy_version 564329 (0.0008) [2023-12-27 09:32:33,926][20347] Updated weights for policy 0, policy_version 564339 (0.0008) [2023-12-27 09:32:33,977][20347] Updated weights for policy 0, policy_version 564349 (0.0007) [2023-12-27 09:32:34,330][20381] Updated weights for policy 1, policy_version 564214 (0.0007) [2023-12-27 09:32:34,389][20381] Updated weights for policy 1, policy_version 564224 (0.0010) [2023-12-27 09:32:34,451][20381] Updated weights for policy 1, policy_version 564234 (0.0011) [2023-12-27 09:32:34,636][20347] Updated weights for policy 0, policy_version 564359 (0.0008) [2023-12-27 09:32:34,698][20347] Updated weights for policy 0, policy_version 564369 (0.0010) [2023-12-27 09:32:34,756][20347] Updated weights for policy 0, policy_version 564379 (0.0010) [2023-12-27 09:32:35,133][20381] Updated weights for policy 1, policy_version 564244 (0.0009) [2023-12-27 09:32:35,180][20381] Updated weights for policy 1, policy_version 564254 (0.0007) [2023-12-27 09:32:35,235][20381] Updated weights for policy 1, policy_version 564264 (0.0006) [2023-12-27 09:32:35,417][20347] Updated weights for policy 0, policy_version 564389 (0.0010) [2023-12-27 09:32:35,468][20347] Updated weights for policy 0, policy_version 564399 (0.0010) [2023-12-27 09:32:35,515][20347] Updated weights for policy 0, policy_version 564409 (0.0010) [2023-12-27 09:32:35,830][20381] Updated weights for policy 1, policy_version 564274 (0.0006) [2023-12-27 09:32:35,893][20381] Updated weights for policy 1, policy_version 564284 (0.0007) [2023-12-27 09:32:35,953][20381] Updated weights for policy 1, policy_version 564294 (0.0008) [2023-12-27 09:32:36,024][20381] Updated weights for policy 1, policy_version 564304 (0.0008) [2023-12-27 09:32:36,130][20347] Updated weights for policy 0, policy_version 564419 (0.0010) [2023-12-27 09:32:36,184][20347] Updated weights for policy 0, policy_version 564429 (0.0009) [2023-12-27 09:32:36,242][20347] Updated weights for policy 0, policy_version 564439 (0.0009) [2023-12-27 09:32:36,744][20381] Updated weights for policy 1, policy_version 564314 (0.0008) [2023-12-27 09:32:36,804][20381] Updated weights for policy 1, policy_version 564324 (0.0007) [2023-12-27 09:32:36,863][20381] Updated weights for policy 1, policy_version 564334 (0.0007) [2023-12-27 09:32:36,937][20347] Updated weights for policy 0, policy_version 564449 (0.0008) [2023-12-27 09:32:37,001][20347] Updated weights for policy 0, policy_version 564459 (0.0007) [2023-12-27 09:32:37,056][20347] Updated weights for policy 0, policy_version 564469 (0.0005) [2023-12-27 09:32:37,111][20347] Updated weights for policy 0, policy_version 564479 (0.0006) [2023-12-27 09:32:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 289013760. Throughput: 0: 10349.5, 1: 10536.3. Samples: 288982336. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:37,363][19170] Avg episode reward: [(0, '972.350'), (1, '980.870')] [2023-12-27 09:32:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000564336_144490496.pth... [2023-12-27 09:32:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000564480_144523264.pth... [2023-12-27 09:32:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000563264_144211968.pth [2023-12-27 09:32:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000563088_144171008.pth [2023-12-27 09:32:37,485][20381] Updated weights for policy 1, policy_version 564344 (0.0007) [2023-12-27 09:32:37,540][20381] Updated weights for policy 1, policy_version 564354 (0.0008) [2023-12-27 09:32:37,600][20381] Updated weights for policy 1, policy_version 564364 (0.0008) [2023-12-27 09:32:37,778][20347] Updated weights for policy 0, policy_version 564489 (0.0008) [2023-12-27 09:32:37,832][20347] Updated weights for policy 0, policy_version 564499 (0.0007) [2023-12-27 09:32:37,890][20347] Updated weights for policy 0, policy_version 564509 (0.0005) [2023-12-27 09:32:38,242][20381] Updated weights for policy 1, policy_version 564374 (0.0007) [2023-12-27 09:32:38,301][20381] Updated weights for policy 1, policy_version 564384 (0.0007) [2023-12-27 09:32:38,362][20381] Updated weights for policy 1, policy_version 564394 (0.0007) [2023-12-27 09:32:38,553][20347] Updated weights for policy 0, policy_version 564519 (0.0009) [2023-12-27 09:32:38,602][20347] Updated weights for policy 0, policy_version 564529 (0.0010) [2023-12-27 09:32:38,647][20347] Updated weights for policy 0, policy_version 564539 (0.0009) [2023-12-27 09:32:39,037][20381] Updated weights for policy 1, policy_version 564404 (0.0008) [2023-12-27 09:32:39,088][20381] Updated weights for policy 1, policy_version 564414 (0.0007) [2023-12-27 09:32:39,140][20381] Updated weights for policy 1, policy_version 564424 (0.0008) [2023-12-27 09:32:39,318][20347] Updated weights for policy 0, policy_version 564549 (0.0007) [2023-12-27 09:32:39,382][20347] Updated weights for policy 0, policy_version 564559 (0.0008) [2023-12-27 09:32:39,443][20347] Updated weights for policy 0, policy_version 564569 (0.0008) [2023-12-27 09:32:39,849][20381] Updated weights for policy 1, policy_version 564434 (0.0006) [2023-12-27 09:32:39,913][20381] Updated weights for policy 1, policy_version 564444 (0.0007) [2023-12-27 09:32:39,981][20381] Updated weights for policy 1, policy_version 564454 (0.0008) [2023-12-27 09:32:40,045][20381] Updated weights for policy 1, policy_version 564464 (0.0008) [2023-12-27 09:32:40,185][20347] Updated weights for policy 0, policy_version 564579 (0.0007) [2023-12-27 09:32:40,246][20347] Updated weights for policy 0, policy_version 564589 (0.0008) [2023-12-27 09:32:40,302][20347] Updated weights for policy 0, policy_version 564599 (0.0008) [2023-12-27 09:32:40,728][20381] Updated weights for policy 1, policy_version 564474 (0.0007) [2023-12-27 09:32:40,781][20381] Updated weights for policy 1, policy_version 564484 (0.0007) [2023-12-27 09:32:40,830][20381] Updated weights for policy 1, policy_version 564494 (0.0008) [2023-12-27 09:32:40,952][20347] Updated weights for policy 0, policy_version 564609 (0.0007) [2023-12-27 09:32:41,014][20347] Updated weights for policy 0, policy_version 564619 (0.0008) [2023-12-27 09:32:41,074][20347] Updated weights for policy 0, policy_version 564629 (0.0009) [2023-12-27 09:32:41,132][20347] Updated weights for policy 0, policy_version 564639 (0.0007) [2023-12-27 09:32:41,529][20381] Updated weights for policy 1, policy_version 564504 (0.0008) [2023-12-27 09:32:41,584][20381] Updated weights for policy 1, policy_version 564514 (0.0008) [2023-12-27 09:32:41,646][20381] Updated weights for policy 1, policy_version 564524 (0.0008) [2023-12-27 09:32:41,900][20347] Updated weights for policy 0, policy_version 564649 (0.0008) [2023-12-27 09:32:41,958][20347] Updated weights for policy 0, policy_version 564659 (0.0008) [2023-12-27 09:32:42,013][20347] Updated weights for policy 0, policy_version 564669 (0.0008) [2023-12-27 09:32:42,329][20381] Updated weights for policy 1, policy_version 564534 (0.0007) [2023-12-27 09:32:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 289112064. Throughput: 0: 10284.7, 1: 10473.7. Samples: 289105552. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:42,363][19170] Avg episode reward: [(0, '990.390'), (1, '971.240')] [2023-12-27 09:32:42,394][20381] Updated weights for policy 1, policy_version 564544 (0.0007) [2023-12-27 09:32:42,456][20381] Updated weights for policy 1, policy_version 564554 (0.0010) [2023-12-27 09:32:42,484][20292] KL-divergence is very high: 204.0476 [2023-12-27 09:32:42,714][20347] Updated weights for policy 0, policy_version 564679 (0.0008) [2023-12-27 09:32:42,767][20347] Updated weights for policy 0, policy_version 564689 (0.0008) [2023-12-27 09:32:42,824][20347] Updated weights for policy 0, policy_version 564699 (0.0008) [2023-12-27 09:32:43,147][20381] Updated weights for policy 1, policy_version 564564 (0.0011) [2023-12-27 09:32:43,196][20381] Updated weights for policy 1, policy_version 564574 (0.0010) [2023-12-27 09:32:43,241][20381] Updated weights for policy 1, policy_version 564584 (0.0010) [2023-12-27 09:32:43,476][20347] Updated weights for policy 0, policy_version 564709 (0.0007) [2023-12-27 09:32:43,527][20347] Updated weights for policy 0, policy_version 564719 (0.0007) [2023-12-27 09:32:43,572][20347] Updated weights for policy 0, policy_version 564729 (0.0007) [2023-12-27 09:32:43,933][20381] Updated weights for policy 1, policy_version 564594 (0.0009) [2023-12-27 09:32:43,979][20381] Updated weights for policy 1, policy_version 564604 (0.0008) [2023-12-27 09:32:44,027][20381] Updated weights for policy 1, policy_version 564614 (0.0008) [2023-12-27 09:32:44,084][20381] Updated weights for policy 1, policy_version 564624 (0.0007) [2023-12-27 09:32:44,278][20347] Updated weights for policy 0, policy_version 564739 (0.0007) [2023-12-27 09:32:44,324][20347] Updated weights for policy 0, policy_version 564749 (0.0009) [2023-12-27 09:32:44,379][20347] Updated weights for policy 0, policy_version 564759 (0.0008) [2023-12-27 09:32:44,732][20381] Updated weights for policy 1, policy_version 564634 (0.0009) [2023-12-27 09:32:44,793][20381] Updated weights for policy 1, policy_version 564644 (0.0009) [2023-12-27 09:32:44,824][20292] KL-divergence is very high: 130.5677 [2023-12-27 09:32:44,847][20381] Updated weights for policy 1, policy_version 564654 (0.0007) [2023-12-27 09:32:45,026][20347] Updated weights for policy 0, policy_version 564769 (0.0008) [2023-12-27 09:32:45,080][20347] Updated weights for policy 0, policy_version 564779 (0.0008) [2023-12-27 09:32:45,137][20347] Updated weights for policy 0, policy_version 564789 (0.0008) [2023-12-27 09:32:45,192][20347] Updated weights for policy 0, policy_version 564799 (0.0007) [2023-12-27 09:32:45,558][20381] Updated weights for policy 1, policy_version 564664 (0.0007) [2023-12-27 09:32:45,611][20381] Updated weights for policy 1, policy_version 564674 (0.0005) [2023-12-27 09:32:45,657][20381] Updated weights for policy 1, policy_version 564684 (0.0005) [2023-12-27 09:32:45,925][20347] Updated weights for policy 0, policy_version 564809 (0.0010) [2023-12-27 09:32:45,987][20347] Updated weights for policy 0, policy_version 564819 (0.0010) [2023-12-27 09:32:46,042][20347] Updated weights for policy 0, policy_version 564829 (0.0010) [2023-12-27 09:32:46,283][20381] Updated weights for policy 1, policy_version 564694 (0.0007) [2023-12-27 09:32:46,337][20381] Updated weights for policy 1, policy_version 564704 (0.0007) [2023-12-27 09:32:46,386][20381] Updated weights for policy 1, policy_version 564714 (0.0007) [2023-12-27 09:32:46,716][20347] Updated weights for policy 0, policy_version 564839 (0.0010) [2023-12-27 09:32:46,768][20347] Updated weights for policy 0, policy_version 564849 (0.0010) [2023-12-27 09:32:46,817][20347] Updated weights for policy 0, policy_version 564859 (0.0008) [2023-12-27 09:32:47,100][20381] Updated weights for policy 1, policy_version 564724 (0.0007) [2023-12-27 09:32:47,156][20381] Updated weights for policy 1, policy_version 564734 (0.0008) [2023-12-27 09:32:47,212][20381] Updated weights for policy 1, policy_version 564744 (0.0007) [2023-12-27 09:32:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 289218560. Throughput: 0: 10236.4, 1: 10499.2. Samples: 289227112. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '894.820')] [2023-12-27 09:32:47,504][20347] Updated weights for policy 0, policy_version 564869 (0.0009) [2023-12-27 09:32:47,559][20347] Updated weights for policy 0, policy_version 564879 (0.0010) [2023-12-27 09:32:47,616][20347] Updated weights for policy 0, policy_version 564889 (0.0010) [2023-12-27 09:32:47,904][20381] Updated weights for policy 1, policy_version 564754 (0.0006) [2023-12-27 09:32:47,955][20381] Updated weights for policy 1, policy_version 564764 (0.0007) [2023-12-27 09:32:48,013][20381] Updated weights for policy 1, policy_version 564774 (0.0007) [2023-12-27 09:32:48,073][20381] Updated weights for policy 1, policy_version 564784 (0.0007) [2023-12-27 09:32:48,272][20347] Updated weights for policy 0, policy_version 564899 (0.0010) [2023-12-27 09:32:48,325][20347] Updated weights for policy 0, policy_version 564909 (0.0010) [2023-12-27 09:32:48,389][20347] Updated weights for policy 0, policy_version 564919 (0.0011) [2023-12-27 09:32:48,792][20381] Updated weights for policy 1, policy_version 564794 (0.0008) [2023-12-27 09:32:48,852][20381] Updated weights for policy 1, policy_version 564804 (0.0007) [2023-12-27 09:32:48,907][20381] Updated weights for policy 1, policy_version 564814 (0.0007) [2023-12-27 09:32:49,039][20347] Updated weights for policy 0, policy_version 564929 (0.0006) [2023-12-27 09:32:49,090][20347] Updated weights for policy 0, policy_version 564939 (0.0007) [2023-12-27 09:32:49,148][20347] Updated weights for policy 0, policy_version 564949 (0.0006) [2023-12-27 09:32:49,200][20347] Updated weights for policy 0, policy_version 564959 (0.0005) [2023-12-27 09:32:49,628][20381] Updated weights for policy 1, policy_version 564824 (0.0007) [2023-12-27 09:32:49,692][20381] Updated weights for policy 1, policy_version 564834 (0.0008) [2023-12-27 09:32:49,749][20381] Updated weights for policy 1, policy_version 564844 (0.0008) [2023-12-27 09:32:49,899][20347] Updated weights for policy 0, policy_version 564969 (0.0008) [2023-12-27 09:32:49,966][20347] Updated weights for policy 0, policy_version 564979 (0.0007) [2023-12-27 09:32:50,026][20347] Updated weights for policy 0, policy_version 564989 (0.0007) [2023-12-27 09:32:50,469][20381] Updated weights for policy 1, policy_version 564854 (0.0008) [2023-12-27 09:32:50,521][20381] Updated weights for policy 1, policy_version 564864 (0.0009) [2023-12-27 09:32:50,578][20381] Updated weights for policy 1, policy_version 564874 (0.0009) [2023-12-27 09:32:50,673][20347] Updated weights for policy 0, policy_version 564999 (0.0009) [2023-12-27 09:32:50,736][20347] Updated weights for policy 0, policy_version 565009 (0.0011) [2023-12-27 09:32:50,799][20347] Updated weights for policy 0, policy_version 565019 (0.0010) [2023-12-27 09:32:51,326][20381] Updated weights for policy 1, policy_version 564884 (0.0008) [2023-12-27 09:32:51,391][20381] Updated weights for policy 1, policy_version 564894 (0.0008) [2023-12-27 09:32:51,444][20347] Updated weights for policy 0, policy_version 565029 (0.0010) [2023-12-27 09:32:51,449][20381] Updated weights for policy 1, policy_version 564904 (0.0007) [2023-12-27 09:32:51,493][20347] Updated weights for policy 0, policy_version 565039 (0.0010) [2023-12-27 09:32:51,546][20347] Updated weights for policy 0, policy_version 565049 (0.0010) [2023-12-27 09:32:52,169][20381] Updated weights for policy 1, policy_version 564914 (0.0008) [2023-12-27 09:32:52,179][20347] Updated weights for policy 0, policy_version 565059 (0.0009) [2023-12-27 09:32:52,219][20381] Updated weights for policy 1, policy_version 564924 (0.0006) [2023-12-27 09:32:52,235][20347] Updated weights for policy 0, policy_version 565069 (0.0006) [2023-12-27 09:32:52,277][20381] Updated weights for policy 1, policy_version 564934 (0.0007) [2023-12-27 09:32:52,293][20347] Updated weights for policy 0, policy_version 565079 (0.0008) [2023-12-27 09:32:52,329][20381] Updated weights for policy 1, policy_version 564944 (0.0006) [2023-12-27 09:32:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 289325056. Throughput: 0: 10253.4, 1: 10451.0. Samples: 289287088. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:52,363][19170] Avg episode reward: [(0, '991.300'), (1, '884.010')] [2023-12-27 09:32:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000565088_144678912.pth... [2023-12-27 09:32:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000564944_144646144.pth... [2023-12-27 09:32:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000563872_144367616.pth [2023-12-27 09:32:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000563728_144334848.pth [2023-12-27 09:32:52,956][20347] Updated weights for policy 0, policy_version 565089 (0.0011) [2023-12-27 09:32:53,003][20347] Updated weights for policy 0, policy_version 565099 (0.0009) [2023-12-27 09:32:53,035][20381] Updated weights for policy 1, policy_version 564954 (0.0007) [2023-12-27 09:32:53,052][20347] Updated weights for policy 0, policy_version 565109 (0.0008) [2023-12-27 09:32:53,081][20381] Updated weights for policy 1, policy_version 564964 (0.0006) [2023-12-27 09:32:53,101][20347] Updated weights for policy 0, policy_version 565119 (0.0006) [2023-12-27 09:32:53,131][20381] Updated weights for policy 1, policy_version 564974 (0.0006) [2023-12-27 09:32:53,800][20347] Updated weights for policy 0, policy_version 565129 (0.0010) [2023-12-27 09:32:53,815][20381] Updated weights for policy 1, policy_version 564984 (0.0007) [2023-12-27 09:32:53,852][20347] Updated weights for policy 0, policy_version 565139 (0.0010) [2023-12-27 09:32:53,873][20381] Updated weights for policy 1, policy_version 564994 (0.0007) [2023-12-27 09:32:53,916][20347] Updated weights for policy 0, policy_version 565149 (0.0010) [2023-12-27 09:32:53,923][20381] Updated weights for policy 1, policy_version 565004 (0.0007) [2023-12-27 09:32:54,478][20347] Updated weights for policy 0, policy_version 565159 (0.0010) [2023-12-27 09:32:54,537][20347] Updated weights for policy 0, policy_version 565169 (0.0010) [2023-12-27 09:32:54,568][20381] Updated weights for policy 1, policy_version 565014 (0.0006) [2023-12-27 09:32:54,589][20347] Updated weights for policy 0, policy_version 565179 (0.0010) [2023-12-27 09:32:54,624][20381] Updated weights for policy 1, policy_version 565024 (0.0006) [2023-12-27 09:32:54,674][20381] Updated weights for policy 1, policy_version 565034 (0.0007) [2023-12-27 09:32:55,249][20347] Updated weights for policy 0, policy_version 565189 (0.0010) [2023-12-27 09:32:55,306][20347] Updated weights for policy 0, policy_version 565199 (0.0010) [2023-12-27 09:32:55,364][20347] Updated weights for policy 0, policy_version 565209 (0.0010) [2023-12-27 09:32:55,373][20381] Updated weights for policy 1, policy_version 565044 (0.0007) [2023-12-27 09:32:55,428][20381] Updated weights for policy 1, policy_version 565054 (0.0006) [2023-12-27 09:32:55,475][20381] Updated weights for policy 1, policy_version 565064 (0.0007) [2023-12-27 09:32:56,031][20347] Updated weights for policy 0, policy_version 565219 (0.0009) [2023-12-27 09:32:56,078][20347] Updated weights for policy 0, policy_version 565229 (0.0005) [2023-12-27 09:32:56,129][20347] Updated weights for policy 0, policy_version 565239 (0.0009) [2023-12-27 09:32:56,140][20381] Updated weights for policy 1, policy_version 565074 (0.0007) [2023-12-27 09:32:56,194][20381] Updated weights for policy 1, policy_version 565084 (0.0006) [2023-12-27 09:32:56,239][20381] Updated weights for policy 1, policy_version 565094 (0.0007) [2023-12-27 09:32:56,289][20381] Updated weights for policy 1, policy_version 565104 (0.0007) [2023-12-27 09:32:56,711][20347] Updated weights for policy 0, policy_version 565249 (0.0010) [2023-12-27 09:32:56,762][20347] Updated weights for policy 0, policy_version 565259 (0.0005) [2023-12-27 09:32:56,805][20347] Updated weights for policy 0, policy_version 565269 (0.0006) [2023-12-27 09:32:56,859][20347] Updated weights for policy 0, policy_version 565279 (0.0010) [2023-12-27 09:32:56,953][20381] Updated weights for policy 1, policy_version 565114 (0.0007) [2023-12-27 09:32:56,999][20381] Updated weights for policy 1, policy_version 565124 (0.0007) [2023-12-27 09:32:57,049][20381] Updated weights for policy 1, policy_version 565134 (0.0008) [2023-12-27 09:32:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 289423360. Throughput: 0: 10289.2, 1: 10403.3. Samples: 289411216. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:32:57,363][19170] Avg episode reward: [(0, '991.300'), (1, '913.740')] [2023-12-27 09:32:57,515][20347] Updated weights for policy 0, policy_version 565289 (0.0010) [2023-12-27 09:32:57,575][20347] Updated weights for policy 0, policy_version 565299 (0.0010) [2023-12-27 09:32:57,634][20347] Updated weights for policy 0, policy_version 565309 (0.0010) [2023-12-27 09:32:57,750][20381] Updated weights for policy 1, policy_version 565144 (0.0007) [2023-12-27 09:32:57,801][20381] Updated weights for policy 1, policy_version 565154 (0.0007) [2023-12-27 09:32:57,852][20381] Updated weights for policy 1, policy_version 565164 (0.0007) [2023-12-27 09:32:58,245][20347] Updated weights for policy 0, policy_version 565319 (0.0008) [2023-12-27 09:32:58,307][20347] Updated weights for policy 0, policy_version 565329 (0.0008) [2023-12-27 09:32:58,369][20347] Updated weights for policy 0, policy_version 565339 (0.0008) [2023-12-27 09:32:58,677][20381] Updated weights for policy 1, policy_version 565174 (0.0007) [2023-12-27 09:32:58,738][20381] Updated weights for policy 1, policy_version 565184 (0.0008) [2023-12-27 09:32:58,807][20381] Updated weights for policy 1, policy_version 565194 (0.0007) [2023-12-27 09:32:59,111][20347] Updated weights for policy 0, policy_version 565349 (0.0008) [2023-12-27 09:32:59,166][20347] Updated weights for policy 0, policy_version 565359 (0.0010) [2023-12-27 09:32:59,218][20347] Updated weights for policy 0, policy_version 565369 (0.0010) [2023-12-27 09:32:59,525][20381] Updated weights for policy 1, policy_version 565204 (0.0008) [2023-12-27 09:32:59,581][20381] Updated weights for policy 1, policy_version 565214 (0.0008) [2023-12-27 09:32:59,651][20381] Updated weights for policy 1, policy_version 565224 (0.0008) [2023-12-27 09:32:59,882][20347] Updated weights for policy 0, policy_version 565379 (0.0009) [2023-12-27 09:32:59,939][20347] Updated weights for policy 0, policy_version 565389 (0.0007) [2023-12-27 09:32:59,993][20347] Updated weights for policy 0, policy_version 565399 (0.0005) [2023-12-27 09:33:00,363][20381] Updated weights for policy 1, policy_version 565234 (0.0008) [2023-12-27 09:33:00,416][20381] Updated weights for policy 1, policy_version 565244 (0.0007) [2023-12-27 09:33:00,466][20381] Updated weights for policy 1, policy_version 565254 (0.0007) [2023-12-27 09:33:00,511][20381] Updated weights for policy 1, policy_version 565264 (0.0008) [2023-12-27 09:33:00,597][20347] Updated weights for policy 0, policy_version 565409 (0.0006) [2023-12-27 09:33:00,653][20347] Updated weights for policy 0, policy_version 565419 (0.0008) [2023-12-27 09:33:00,708][20347] Updated weights for policy 0, policy_version 565429 (0.0006) [2023-12-27 09:33:00,768][20347] Updated weights for policy 0, policy_version 565439 (0.0006) [2023-12-27 09:33:01,260][20381] Updated weights for policy 1, policy_version 565274 (0.0008) [2023-12-27 09:33:01,316][20381] Updated weights for policy 1, policy_version 565284 (0.0008) [2023-12-27 09:33:01,383][20381] Updated weights for policy 1, policy_version 565294 (0.0009) [2023-12-27 09:33:01,399][20347] Updated weights for policy 0, policy_version 565449 (0.0009) [2023-12-27 09:33:01,459][20347] Updated weights for policy 0, policy_version 565459 (0.0006) [2023-12-27 09:33:01,506][20347] Updated weights for policy 0, policy_version 565469 (0.0007) [2023-12-27 09:33:02,150][20347] Updated weights for policy 0, policy_version 565479 (0.0009) [2023-12-27 09:33:02,155][20381] Updated weights for policy 1, policy_version 565304 (0.0005) [2023-12-27 09:33:02,204][20347] Updated weights for policy 0, policy_version 565489 (0.0010) [2023-12-27 09:33:02,206][20381] Updated weights for policy 1, policy_version 565314 (0.0005) [2023-12-27 09:33:02,258][20381] Updated weights for policy 1, policy_version 565324 (0.0008) [2023-12-27 09:33:02,261][20347] Updated weights for policy 0, policy_version 565499 (0.0010) [2023-12-27 09:33:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 289529856. Throughput: 0: 10353.1, 1: 10289.7. Samples: 289533620. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:02,363][19170] Avg episode reward: [(0, '990.490'), (1, '924.050')] [2023-12-27 09:33:02,888][20347] Updated weights for policy 0, policy_version 565509 (0.0008) [2023-12-27 09:33:02,939][20347] Updated weights for policy 0, policy_version 565519 (0.0005) [2023-12-27 09:33:02,975][20381] Updated weights for policy 1, policy_version 565334 (0.0007) [2023-12-27 09:33:02,997][20347] Updated weights for policy 0, policy_version 565529 (0.0008) [2023-12-27 09:33:03,034][20381] Updated weights for policy 1, policy_version 565344 (0.0006) [2023-12-27 09:33:03,088][20381] Updated weights for policy 1, policy_version 565354 (0.0007) [2023-12-27 09:33:03,644][20347] Updated weights for policy 0, policy_version 565539 (0.0010) [2023-12-27 09:33:03,695][20347] Updated weights for policy 0, policy_version 565549 (0.0010) [2023-12-27 09:33:03,739][20347] Updated weights for policy 0, policy_version 565559 (0.0010) [2023-12-27 09:33:03,781][20381] Updated weights for policy 1, policy_version 565364 (0.0006) [2023-12-27 09:33:03,840][20381] Updated weights for policy 1, policy_version 565374 (0.0007) [2023-12-27 09:33:03,896][20381] Updated weights for policy 1, policy_version 565384 (0.0007) [2023-12-27 09:33:04,445][20347] Updated weights for policy 0, policy_version 565569 (0.0010) [2023-12-27 09:33:04,500][20347] Updated weights for policy 0, policy_version 565579 (0.0010) [2023-12-27 09:33:04,564][20347] Updated weights for policy 0, policy_version 565589 (0.0010) [2023-12-27 09:33:04,616][20381] Updated weights for policy 1, policy_version 565394 (0.0007) [2023-12-27 09:33:04,619][20347] Updated weights for policy 0, policy_version 565599 (0.0010) [2023-12-27 09:33:04,673][20381] Updated weights for policy 1, policy_version 565404 (0.0007) [2023-12-27 09:33:04,731][20381] Updated weights for policy 1, policy_version 565414 (0.0008) [2023-12-27 09:33:04,789][20381] Updated weights for policy 1, policy_version 565424 (0.0007) [2023-12-27 09:33:05,254][20347] Updated weights for policy 0, policy_version 565609 (0.0006) [2023-12-27 09:33:05,310][20347] Updated weights for policy 0, policy_version 565619 (0.0005) [2023-12-27 09:33:05,366][20347] Updated weights for policy 0, policy_version 565629 (0.0005) [2023-12-27 09:33:05,498][20381] Updated weights for policy 1, policy_version 565434 (0.0011) [2023-12-27 09:33:05,551][20381] Updated weights for policy 1, policy_version 565444 (0.0010) [2023-12-27 09:33:05,598][20381] Updated weights for policy 1, policy_version 565454 (0.0011) [2023-12-27 09:33:05,961][20347] Updated weights for policy 0, policy_version 565639 (0.0006) [2023-12-27 09:33:06,019][20347] Updated weights for policy 0, policy_version 565649 (0.0005) [2023-12-27 09:33:06,078][20347] Updated weights for policy 0, policy_version 565659 (0.0005) [2023-12-27 09:33:06,286][20381] Updated weights for policy 1, policy_version 565464 (0.0010) [2023-12-27 09:33:06,338][20381] Updated weights for policy 1, policy_version 565474 (0.0006) [2023-12-27 09:33:06,400][20381] Updated weights for policy 1, policy_version 565484 (0.0010) [2023-12-27 09:33:06,735][20347] Updated weights for policy 0, policy_version 565669 (0.0009) [2023-12-27 09:33:06,790][20347] Updated weights for policy 0, policy_version 565679 (0.0010) [2023-12-27 09:33:06,846][20347] Updated weights for policy 0, policy_version 565689 (0.0010) [2023-12-27 09:33:07,101][20381] Updated weights for policy 1, policy_version 565494 (0.0010) [2023-12-27 09:33:07,163][20381] Updated weights for policy 1, policy_version 565504 (0.0010) [2023-12-27 09:33:07,221][20381] Updated weights for policy 1, policy_version 565514 (0.0010) [2023-12-27 09:33:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 289628160. Throughput: 0: 10388.2, 1: 10242.9. Samples: 289594648. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:07,363][19170] Avg episode reward: [(0, '971.260'), (1, '942.200')] [2023-12-27 09:33:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000565696_144834560.pth... [2023-12-27 09:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000565520_144793600.pth... [2023-12-27 09:33:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000564480_144523264.pth [2023-12-27 09:33:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000564336_144490496.pth [2023-12-27 09:33:07,543][20347] Updated weights for policy 0, policy_version 565699 (0.0010) [2023-12-27 09:33:07,591][20347] Updated weights for policy 0, policy_version 565709 (0.0010) [2023-12-27 09:33:07,636][20347] Updated weights for policy 0, policy_version 565719 (0.0010) [2023-12-27 09:33:07,748][20381] Updated weights for policy 1, policy_version 565524 (0.0005) [2023-12-27 09:33:07,809][20381] Updated weights for policy 1, policy_version 565534 (0.0005) [2023-12-27 09:33:07,862][20381] Updated weights for policy 1, policy_version 565544 (0.0005) [2023-12-27 09:33:08,346][20347] Updated weights for policy 0, policy_version 565729 (0.0010) [2023-12-27 09:33:08,409][20347] Updated weights for policy 0, policy_version 565739 (0.0008) [2023-12-27 09:33:08,464][20381] Updated weights for policy 1, policy_version 565554 (0.0006) [2023-12-27 09:33:08,465][20347] Updated weights for policy 0, policy_version 565749 (0.0007) [2023-12-27 09:33:08,517][20347] Updated weights for policy 0, policy_version 565759 (0.0005) [2023-12-27 09:33:08,526][20381] Updated weights for policy 1, policy_version 565564 (0.0007) [2023-12-27 09:33:08,587][20381] Updated weights for policy 1, policy_version 565574 (0.0007) [2023-12-27 09:33:08,648][20381] Updated weights for policy 1, policy_version 565584 (0.0007) [2023-12-27 09:33:09,141][20347] Updated weights for policy 0, policy_version 565769 (0.0006) [2023-12-27 09:33:09,205][20347] Updated weights for policy 0, policy_version 565779 (0.0007) [2023-12-27 09:33:09,272][20381] Updated weights for policy 1, policy_version 565594 (0.0007) [2023-12-27 09:33:09,274][20347] Updated weights for policy 0, policy_version 565789 (0.0008) [2023-12-27 09:33:09,335][20381] Updated weights for policy 1, policy_version 565604 (0.0008) [2023-12-27 09:33:09,395][20381] Updated weights for policy 1, policy_version 565614 (0.0007) [2023-12-27 09:33:09,891][20347] Updated weights for policy 0, policy_version 565799 (0.0006) [2023-12-27 09:33:09,956][20347] Updated weights for policy 0, policy_version 565809 (0.0007) [2023-12-27 09:33:10,019][20347] Updated weights for policy 0, policy_version 565819 (0.0006) [2023-12-27 09:33:10,122][20381] Updated weights for policy 1, policy_version 565624 (0.0006) [2023-12-27 09:33:10,188][20381] Updated weights for policy 1, policy_version 565634 (0.0006) [2023-12-27 09:33:10,248][20381] Updated weights for policy 1, policy_version 565644 (0.0008) [2023-12-27 09:33:10,656][20347] Updated weights for policy 0, policy_version 565829 (0.0008) [2023-12-27 09:33:10,718][20347] Updated weights for policy 0, policy_version 565839 (0.0008) [2023-12-27 09:33:10,776][20347] Updated weights for policy 0, policy_version 565849 (0.0008) [2023-12-27 09:33:10,906][20381] Updated weights for policy 1, policy_version 565654 (0.0008) [2023-12-27 09:33:10,964][20381] Updated weights for policy 1, policy_version 565664 (0.0008) [2023-12-27 09:33:11,025][20381] Updated weights for policy 1, policy_version 565674 (0.0008) [2023-12-27 09:33:11,442][20347] Updated weights for policy 0, policy_version 565859 (0.0008) [2023-12-27 09:33:11,502][20347] Updated weights for policy 0, policy_version 565869 (0.0008) [2023-12-27 09:33:11,563][20347] Updated weights for policy 0, policy_version 565879 (0.0008) [2023-12-27 09:33:11,777][20381] Updated weights for policy 1, policy_version 565684 (0.0008) [2023-12-27 09:33:11,834][20381] Updated weights for policy 1, policy_version 565694 (0.0011) [2023-12-27 09:33:11,886][20381] Updated weights for policy 1, policy_version 565704 (0.0011) [2023-12-27 09:33:12,265][20347] Updated weights for policy 0, policy_version 565889 (0.0007) [2023-12-27 09:33:12,326][20347] Updated weights for policy 0, policy_version 565899 (0.0007) [2023-12-27 09:33:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 289726464. Throughput: 0: 10458.1, 1: 10197.0. Samples: 289720796. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:12,363][19170] Avg episode reward: [(0, '980.780'), (1, '971.310')] [2023-12-27 09:33:12,388][20347] Updated weights for policy 0, policy_version 565909 (0.0006) [2023-12-27 09:33:12,444][20347] Updated weights for policy 0, policy_version 565919 (0.0007) [2023-12-27 09:33:12,600][20381] Updated weights for policy 1, policy_version 565714 (0.0011) [2023-12-27 09:33:12,649][20381] Updated weights for policy 1, policy_version 565724 (0.0008) [2023-12-27 09:33:12,697][20381] Updated weights for policy 1, policy_version 565734 (0.0008) [2023-12-27 09:33:12,752][20381] Updated weights for policy 1, policy_version 565744 (0.0010) [2023-12-27 09:33:13,124][20347] Updated weights for policy 0, policy_version 565929 (0.0005) [2023-12-27 09:33:13,176][20347] Updated weights for policy 0, policy_version 565939 (0.0010) [2023-12-27 09:33:13,221][20347] Updated weights for policy 0, policy_version 565949 (0.0010) [2023-12-27 09:33:13,406][20381] Updated weights for policy 1, policy_version 565754 (0.0010) [2023-12-27 09:33:13,461][20381] Updated weights for policy 1, policy_version 565764 (0.0010) [2023-12-27 09:33:13,519][20381] Updated weights for policy 1, policy_version 565774 (0.0011) [2023-12-27 09:33:13,889][20347] Updated weights for policy 0, policy_version 565959 (0.0010) [2023-12-27 09:33:13,940][20347] Updated weights for policy 0, policy_version 565969 (0.0010) [2023-12-27 09:33:13,988][20347] Updated weights for policy 0, policy_version 565979 (0.0010) [2023-12-27 09:33:14,177][20381] Updated weights for policy 1, policy_version 565784 (0.0011) [2023-12-27 09:33:14,219][20292] KL-divergence is very high: 101.6295 [2023-12-27 09:33:14,230][20292] KL-divergence is very high: 260.2864 [2023-12-27 09:33:14,235][20292] KL-divergence is very high: 161.5150 [2023-12-27 09:33:14,235][20381] Updated weights for policy 1, policy_version 565794 (0.0006) [2023-12-27 09:33:14,283][20381] Updated weights for policy 1, policy_version 565804 (0.0005) [2023-12-27 09:33:14,302][20292] KL-divergence is very high: 412.1388 [2023-12-27 09:33:14,541][20347] Updated weights for policy 0, policy_version 565989 (0.0008) [2023-12-27 09:33:14,595][20347] Updated weights for policy 0, policy_version 565999 (0.0010) [2023-12-27 09:33:14,649][20347] Updated weights for policy 0, policy_version 566009 (0.0010) [2023-12-27 09:33:14,855][20292] KL-divergence is very high: 285.9773 [2023-12-27 09:33:14,868][20292] KL-divergence is very high: 132.6257 [2023-12-27 09:33:14,876][20292] KL-divergence is very high: 212.6026 [2023-12-27 09:33:14,888][20292] KL-divergence is very high: 273.0270 [2023-12-27 09:33:14,893][20381] Updated weights for policy 1, policy_version 565814 (0.0009) [2023-12-27 09:33:14,935][20292] KL-divergence is very high: 334.0631 [2023-12-27 09:33:14,952][20381] Updated weights for policy 1, policy_version 565824 (0.0008) [2023-12-27 09:33:14,972][20292] KL-divergence is very high: 218.4636 [2023-12-27 09:33:14,983][20292] KL-divergence is very high: 308.7315 [2023-12-27 09:33:15,015][20381] Updated weights for policy 1, policy_version 565834 (0.0011) [2023-12-27 09:33:15,019][20292] KL-divergence is very high: 105.0256 [2023-12-27 09:33:15,030][20292] KL-divergence is very high: 257.3678 [2023-12-27 09:33:15,249][20347] Updated weights for policy 0, policy_version 566019 (0.0009) [2023-12-27 09:33:15,313][20347] Updated weights for policy 0, policy_version 566029 (0.0010) [2023-12-27 09:33:15,381][20347] Updated weights for policy 0, policy_version 566039 (0.0010) [2023-12-27 09:33:15,683][20381] Updated weights for policy 1, policy_version 565844 (0.0010) [2023-12-27 09:33:15,736][20381] Updated weights for policy 1, policy_version 565854 (0.0006) [2023-12-27 09:33:15,803][20381] Updated weights for policy 1, policy_version 565864 (0.0010) [2023-12-27 09:33:16,007][20347] Updated weights for policy 0, policy_version 566049 (0.0009) [2023-12-27 09:33:16,061][20347] Updated weights for policy 0, policy_version 566059 (0.0010) [2023-12-27 09:33:16,114][20347] Updated weights for policy 0, policy_version 566069 (0.0011) [2023-12-27 09:33:16,176][20347] Updated weights for policy 0, policy_version 566079 (0.0010) [2023-12-27 09:33:16,481][20381] Updated weights for policy 1, policy_version 565874 (0.0010) [2023-12-27 09:33:16,536][20381] Updated weights for policy 1, policy_version 565884 (0.0010) [2023-12-27 09:33:16,594][20381] Updated weights for policy 1, policy_version 565894 (0.0010) [2023-12-27 09:33:16,639][20381] Updated weights for policy 1, policy_version 565904 (0.0010) [2023-12-27 09:33:16,822][20347] Updated weights for policy 0, policy_version 566089 (0.0008) [2023-12-27 09:33:16,879][20347] Updated weights for policy 0, policy_version 566099 (0.0005) [2023-12-27 09:33:16,925][20347] Updated weights for policy 0, policy_version 566109 (0.0007) [2023-12-27 09:33:17,325][20381] Updated weights for policy 1, policy_version 565914 (0.0010) [2023-12-27 09:33:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 289832960. Throughput: 0: 10481.6, 1: 10144.9. Samples: 289847120. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:17,363][19170] Avg episode reward: [(0, '990.440'), (1, '914.040')] [2023-12-27 09:33:17,381][20381] Updated weights for policy 1, policy_version 565924 (0.0010) [2023-12-27 09:33:17,429][20381] Updated weights for policy 1, policy_version 565934 (0.0010) [2023-12-27 09:33:17,618][20347] Updated weights for policy 0, policy_version 566119 (0.0007) [2023-12-27 09:33:17,670][20347] Updated weights for policy 0, policy_version 566129 (0.0007) [2023-12-27 09:33:17,727][20347] Updated weights for policy 0, policy_version 566139 (0.0006) [2023-12-27 09:33:18,102][20381] Updated weights for policy 1, policy_version 565944 (0.0006) [2023-12-27 09:33:18,155][20381] Updated weights for policy 1, policy_version 565954 (0.0009) [2023-12-27 09:33:18,207][20381] Updated weights for policy 1, policy_version 565964 (0.0010) [2023-12-27 09:33:18,416][20347] Updated weights for policy 0, policy_version 566149 (0.0005) [2023-12-27 09:33:18,481][20347] Updated weights for policy 0, policy_version 566159 (0.0006) [2023-12-27 09:33:18,540][20347] Updated weights for policy 0, policy_version 566169 (0.0005) [2023-12-27 09:33:18,883][20381] Updated weights for policy 1, policy_version 565974 (0.0011) [2023-12-27 09:33:18,943][20381] Updated weights for policy 1, policy_version 565984 (0.0010) [2023-12-27 09:33:19,005][20381] Updated weights for policy 1, policy_version 565994 (0.0010) [2023-12-27 09:33:19,077][20347] Updated weights for policy 0, policy_version 566179 (0.0006) [2023-12-27 09:33:19,132][20347] Updated weights for policy 0, policy_version 566189 (0.0007) [2023-12-27 09:33:19,190][20347] Updated weights for policy 0, policy_version 566199 (0.0007) [2023-12-27 09:33:19,649][20381] Updated weights for policy 1, policy_version 566004 (0.0010) [2023-12-27 09:33:19,700][20381] Updated weights for policy 1, policy_version 566014 (0.0011) [2023-12-27 09:33:19,754][20381] Updated weights for policy 1, policy_version 566024 (0.0011) [2023-12-27 09:33:19,910][20347] Updated weights for policy 0, policy_version 566209 (0.0008) [2023-12-27 09:33:19,977][20347] Updated weights for policy 0, policy_version 566219 (0.0006) [2023-12-27 09:33:20,041][20347] Updated weights for policy 0, policy_version 566229 (0.0008) [2023-12-27 09:33:20,104][20347] Updated weights for policy 0, policy_version 566239 (0.0007) [2023-12-27 09:33:20,462][20381] Updated weights for policy 1, policy_version 566034 (0.0009) [2023-12-27 09:33:20,507][20381] Updated weights for policy 1, policy_version 566044 (0.0010) [2023-12-27 09:33:20,562][20381] Updated weights for policy 1, policy_version 566054 (0.0010) [2023-12-27 09:33:20,625][20381] Updated weights for policy 1, policy_version 566064 (0.0011) [2023-12-27 09:33:20,822][20347] Updated weights for policy 0, policy_version 566249 (0.0007) [2023-12-27 09:33:20,882][20347] Updated weights for policy 0, policy_version 566259 (0.0008) [2023-12-27 09:33:20,943][20347] Updated weights for policy 0, policy_version 566269 (0.0009) [2023-12-27 09:33:21,262][20381] Updated weights for policy 1, policy_version 566074 (0.0009) [2023-12-27 09:33:21,327][20381] Updated weights for policy 1, policy_version 566084 (0.0011) [2023-12-27 09:33:21,397][20381] Updated weights for policy 1, policy_version 566094 (0.0009) [2023-12-27 09:33:21,726][20347] Updated weights for policy 0, policy_version 566279 (0.0008) [2023-12-27 09:33:21,782][20347] Updated weights for policy 0, policy_version 566289 (0.0008) [2023-12-27 09:33:21,842][20347] Updated weights for policy 0, policy_version 566299 (0.0006) [2023-12-27 09:33:22,081][20381] Updated weights for policy 1, policy_version 566104 (0.0011) [2023-12-27 09:33:22,138][20381] Updated weights for policy 1, policy_version 566114 (0.0011) [2023-12-27 09:33:22,191][20381] Updated weights for policy 1, policy_version 566124 (0.0010) [2023-12-27 09:33:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 289939456. Throughput: 0: 10480.0, 1: 10114.6. Samples: 289909092. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:22,363][19170] Avg episode reward: [(0, '990.560'), (1, '893.450')] [2023-12-27 09:33:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000566128_144949248.pth... [2023-12-27 09:33:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000566304_144990208.pth... [2023-12-27 09:33:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000565088_144678912.pth [2023-12-27 09:33:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000564944_144646144.pth [2023-12-27 09:33:22,508][20347] Updated weights for policy 0, policy_version 566309 (0.0007) [2023-12-27 09:33:22,564][20347] Updated weights for policy 0, policy_version 566319 (0.0008) [2023-12-27 09:33:22,623][20347] Updated weights for policy 0, policy_version 566329 (0.0007) [2023-12-27 09:33:22,844][20381] Updated weights for policy 1, policy_version 566134 (0.0009) [2023-12-27 09:33:22,900][20381] Updated weights for policy 1, policy_version 566144 (0.0011) [2023-12-27 09:33:22,957][20381] Updated weights for policy 1, policy_version 566154 (0.0011) [2023-12-27 09:33:23,347][20347] Updated weights for policy 0, policy_version 566339 (0.0007) [2023-12-27 09:33:23,401][20347] Updated weights for policy 0, policy_version 566349 (0.0007) [2023-12-27 09:33:23,456][20347] Updated weights for policy 0, policy_version 566359 (0.0007) [2023-12-27 09:33:23,636][20381] Updated weights for policy 1, policy_version 566164 (0.0011) [2023-12-27 09:33:23,695][20381] Updated weights for policy 1, policy_version 566174 (0.0011) [2023-12-27 09:33:23,747][20381] Updated weights for policy 1, policy_version 566184 (0.0010) [2023-12-27 09:33:24,154][20347] Updated weights for policy 0, policy_version 566369 (0.0007) [2023-12-27 09:33:24,201][20347] Updated weights for policy 0, policy_version 566379 (0.0006) [2023-12-27 09:33:24,258][20347] Updated weights for policy 0, policy_version 566389 (0.0005) [2023-12-27 09:33:24,329][20347] Updated weights for policy 0, policy_version 566399 (0.0006) [2023-12-27 09:33:24,418][20381] Updated weights for policy 1, policy_version 566194 (0.0009) [2023-12-27 09:33:24,472][20381] Updated weights for policy 1, policy_version 566204 (0.0009) [2023-12-27 09:33:24,520][20381] Updated weights for policy 1, policy_version 566214 (0.0010) [2023-12-27 09:33:24,565][20381] Updated weights for policy 1, policy_version 566224 (0.0010) [2023-12-27 09:33:24,974][20347] Updated weights for policy 0, policy_version 566409 (0.0007) [2023-12-27 09:33:25,029][20347] Updated weights for policy 0, policy_version 566419 (0.0007) [2023-12-27 09:33:25,074][20347] Updated weights for policy 0, policy_version 566429 (0.0008) [2023-12-27 09:33:25,172][20381] Updated weights for policy 1, policy_version 566234 (0.0008) [2023-12-27 09:33:25,220][20381] Updated weights for policy 1, policy_version 566244 (0.0007) [2023-12-27 09:33:25,263][20381] Updated weights for policy 1, policy_version 566254 (0.0005) [2023-12-27 09:33:25,755][20347] Updated weights for policy 0, policy_version 566439 (0.0008) [2023-12-27 09:33:25,800][20347] Updated weights for policy 0, policy_version 566449 (0.0008) [2023-12-27 09:33:25,866][20347] Updated weights for policy 0, policy_version 566459 (0.0007) [2023-12-27 09:33:25,898][20381] Updated weights for policy 1, policy_version 566264 (0.0006) [2023-12-27 09:33:25,954][20381] Updated weights for policy 1, policy_version 566274 (0.0007) [2023-12-27 09:33:26,017][20381] Updated weights for policy 1, policy_version 566284 (0.0007) [2023-12-27 09:33:26,558][20347] Updated weights for policy 0, policy_version 566469 (0.0008) [2023-12-27 09:33:26,604][20347] Updated weights for policy 0, policy_version 566479 (0.0010) [2023-12-27 09:33:26,649][20347] Updated weights for policy 0, policy_version 566489 (0.0010) [2023-12-27 09:33:26,650][20381] Updated weights for policy 1, policy_version 566294 (0.0009) [2023-12-27 09:33:26,704][20381] Updated weights for policy 1, policy_version 566304 (0.0010) [2023-12-27 09:33:26,753][20381] Updated weights for policy 1, policy_version 566314 (0.0007) [2023-12-27 09:33:27,252][20347] Updated weights for policy 0, policy_version 566499 (0.0010) [2023-12-27 09:33:27,296][20347] Updated weights for policy 0, policy_version 566509 (0.0010) [2023-12-27 09:33:27,349][20347] Updated weights for policy 0, policy_version 566519 (0.0010) [2023-12-27 09:33:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 290037760. Throughput: 0: 10424.7, 1: 10152.3. Samples: 290031520. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:27,363][19170] Avg episode reward: [(0, '990.560'), (1, '922.710')] [2023-12-27 09:33:27,378][20381] Updated weights for policy 1, policy_version 566324 (0.0008) [2023-12-27 09:33:27,428][20381] Updated weights for policy 1, policy_version 566334 (0.0006) [2023-12-27 09:33:27,479][20381] Updated weights for policy 1, policy_version 566344 (0.0007) [2023-12-27 09:33:28,037][20347] Updated weights for policy 0, policy_version 566529 (0.0010) [2023-12-27 09:33:28,089][20347] Updated weights for policy 0, policy_version 566539 (0.0007) [2023-12-27 09:33:28,115][20381] Updated weights for policy 1, policy_version 566354 (0.0008) [2023-12-27 09:33:28,147][20347] Updated weights for policy 0, policy_version 566549 (0.0008) [2023-12-27 09:33:28,163][20381] Updated weights for policy 1, policy_version 566364 (0.0010) [2023-12-27 09:33:28,204][20347] Updated weights for policy 0, policy_version 566559 (0.0008) [2023-12-27 09:33:28,216][20381] Updated weights for policy 1, policy_version 566374 (0.0009) [2023-12-27 09:33:28,266][20381] Updated weights for policy 1, policy_version 566384 (0.0005) [2023-12-27 09:33:28,869][20347] Updated weights for policy 0, policy_version 566569 (0.0010) [2023-12-27 09:33:28,872][20381] Updated weights for policy 1, policy_version 566394 (0.0010) [2023-12-27 09:33:28,922][20347] Updated weights for policy 0, policy_version 566579 (0.0010) [2023-12-27 09:33:28,924][20381] Updated weights for policy 1, policy_version 566404 (0.0010) [2023-12-27 09:33:28,983][20381] Updated weights for policy 1, policy_version 566414 (0.0011) [2023-12-27 09:33:28,984][20347] Updated weights for policy 0, policy_version 566589 (0.0009) [2023-12-27 09:33:29,663][20347] Updated weights for policy 0, policy_version 566599 (0.0007) [2023-12-27 09:33:29,692][20381] Updated weights for policy 1, policy_version 566424 (0.0010) [2023-12-27 09:33:29,715][20347] Updated weights for policy 0, policy_version 566609 (0.0008) [2023-12-27 09:33:29,744][20381] Updated weights for policy 1, policy_version 566434 (0.0010) [2023-12-27 09:33:29,773][20347] Updated weights for policy 0, policy_version 566619 (0.0008) [2023-12-27 09:33:29,794][20381] Updated weights for policy 1, policy_version 566444 (0.0011) [2023-12-27 09:33:30,436][20347] Updated weights for policy 0, policy_version 566629 (0.0006) [2023-12-27 09:33:30,442][20381] Updated weights for policy 1, policy_version 566454 (0.0008) [2023-12-27 09:33:30,493][20381] Updated weights for policy 1, policy_version 566464 (0.0005) [2023-12-27 09:33:30,495][20347] Updated weights for policy 0, policy_version 566639 (0.0008) [2023-12-27 09:33:30,545][20381] Updated weights for policy 1, policy_version 566474 (0.0007) [2023-12-27 09:33:30,545][20347] Updated weights for policy 0, policy_version 566649 (0.0005) [2023-12-27 09:33:31,173][20347] Updated weights for policy 0, policy_version 566659 (0.0006) [2023-12-27 09:33:31,225][20347] Updated weights for policy 0, policy_version 566669 (0.0006) [2023-12-27 09:33:31,231][20381] Updated weights for policy 1, policy_version 566484 (0.0008) [2023-12-27 09:33:31,282][20347] Updated weights for policy 0, policy_version 566679 (0.0006) [2023-12-27 09:33:31,285][20381] Updated weights for policy 1, policy_version 566494 (0.0006) [2023-12-27 09:33:31,337][20381] Updated weights for policy 1, policy_version 566504 (0.0006) [2023-12-27 09:33:31,932][20347] Updated weights for policy 0, policy_version 566689 (0.0007) [2023-12-27 09:33:31,991][20347] Updated weights for policy 0, policy_version 566699 (0.0007) [2023-12-27 09:33:32,025][20381] Updated weights for policy 1, policy_version 566514 (0.0008) [2023-12-27 09:33:32,051][20347] Updated weights for policy 0, policy_version 566709 (0.0006) [2023-12-27 09:33:32,084][20381] Updated weights for policy 1, policy_version 566524 (0.0006) [2023-12-27 09:33:32,101][20347] Updated weights for policy 0, policy_version 566719 (0.0006) [2023-12-27 09:33:32,133][20381] Updated weights for policy 1, policy_version 566534 (0.0006) [2023-12-27 09:33:32,188][20381] Updated weights for policy 1, policy_version 566544 (0.0008) [2023-12-27 09:33:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 290152448. Throughput: 0: 10503.0, 1: 10198.3. Samples: 290158672. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.930')] [2023-12-27 09:33:32,807][20347] Updated weights for policy 0, policy_version 566729 (0.0006) [2023-12-27 09:33:32,832][20381] Updated weights for policy 1, policy_version 566554 (0.0007) [2023-12-27 09:33:32,856][20347] Updated weights for policy 0, policy_version 566739 (0.0005) [2023-12-27 09:33:32,880][20381] Updated weights for policy 1, policy_version 566564 (0.0006) [2023-12-27 09:33:32,903][20347] Updated weights for policy 0, policy_version 566749 (0.0006) [2023-12-27 09:33:32,929][20381] Updated weights for policy 1, policy_version 566574 (0.0006) [2023-12-27 09:33:33,528][20381] Updated weights for policy 1, policy_version 566584 (0.0006) [2023-12-27 09:33:33,584][20381] Updated weights for policy 1, policy_version 566594 (0.0005) [2023-12-27 09:33:33,585][20347] Updated weights for policy 0, policy_version 566759 (0.0007) [2023-12-27 09:33:33,634][20381] Updated weights for policy 1, policy_version 566604 (0.0005) [2023-12-27 09:33:33,636][20347] Updated weights for policy 0, policy_version 566769 (0.0007) [2023-12-27 09:33:33,686][20347] Updated weights for policy 0, policy_version 566779 (0.0006) [2023-12-27 09:33:34,207][20381] Updated weights for policy 1, policy_version 566614 (0.0006) [2023-12-27 09:33:34,226][20292] KL-divergence is very high: 148.0783 [2023-12-27 09:33:34,243][20292] KL-divergence is very high: 242.0097 [2023-12-27 09:33:34,264][20381] Updated weights for policy 1, policy_version 566624 (0.0008) [2023-12-27 09:33:34,302][20347] Updated weights for policy 0, policy_version 566789 (0.0005) [2023-12-27 09:33:34,319][20381] Updated weights for policy 1, policy_version 566634 (0.0009) [2023-12-27 09:33:34,361][20347] Updated weights for policy 0, policy_version 566799 (0.0006) [2023-12-27 09:33:34,417][20347] Updated weights for policy 0, policy_version 566809 (0.0005) [2023-12-27 09:33:35,045][20347] Updated weights for policy 0, policy_version 566819 (0.0008) [2023-12-27 09:33:35,072][20381] Updated weights for policy 1, policy_version 566644 (0.0009) [2023-12-27 09:33:35,093][20347] Updated weights for policy 0, policy_version 566829 (0.0006) [2023-12-27 09:33:35,138][20381] Updated weights for policy 1, policy_version 566654 (0.0008) [2023-12-27 09:33:35,140][20347] Updated weights for policy 0, policy_version 566839 (0.0006) [2023-12-27 09:33:35,198][20381] Updated weights for policy 1, policy_version 566664 (0.0006) [2023-12-27 09:33:35,782][20347] Updated weights for policy 0, policy_version 566849 (0.0008) [2023-12-27 09:33:35,829][20347] Updated weights for policy 0, policy_version 566859 (0.0007) [2023-12-27 09:33:35,849][20381] Updated weights for policy 1, policy_version 566674 (0.0007) [2023-12-27 09:33:35,881][20347] Updated weights for policy 0, policy_version 566869 (0.0007) [2023-12-27 09:33:35,912][20381] Updated weights for policy 1, policy_version 566684 (0.0007) [2023-12-27 09:33:35,933][20347] Updated weights for policy 0, policy_version 566879 (0.0005) [2023-12-27 09:33:35,969][20381] Updated weights for policy 1, policy_version 566694 (0.0007) [2023-12-27 09:33:36,028][20381] Updated weights for policy 1, policy_version 566704 (0.0007) [2023-12-27 09:33:36,616][20347] Updated weights for policy 0, policy_version 566889 (0.0006) [2023-12-27 09:33:36,670][20347] Updated weights for policy 0, policy_version 566899 (0.0008) [2023-12-27 09:33:36,721][20381] Updated weights for policy 1, policy_version 566714 (0.0006) [2023-12-27 09:33:36,725][20347] Updated weights for policy 0, policy_version 566909 (0.0008) [2023-12-27 09:33:36,779][20381] Updated weights for policy 1, policy_version 566724 (0.0007) [2023-12-27 09:33:36,846][20381] Updated weights for policy 1, policy_version 566734 (0.0010) [2023-12-27 09:33:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 290250752. Throughput: 0: 10523.2, 1: 10277.6. Samples: 290223116. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '867.070')] [2023-12-27 09:33:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000566736_145104896.pth... [2023-12-27 09:33:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000565520_144793600.pth [2023-12-27 09:33:37,427][20347] Updated weights for policy 0, policy_version 566919 (0.0008) [2023-12-27 09:33:37,490][20347] Updated weights for policy 0, policy_version 566929 (0.0007) [2023-12-27 09:33:37,539][20381] Updated weights for policy 1, policy_version 566744 (0.0008) [2023-12-27 09:33:37,544][20347] Updated weights for policy 0, policy_version 566939 (0.0005) [2023-12-27 09:33:37,571][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000566944_145154048.pth... [2023-12-27 09:33:37,576][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000565696_144834560.pth [2023-12-27 09:33:37,585][20381] Updated weights for policy 1, policy_version 566754 (0.0007) [2023-12-27 09:33:37,638][20381] Updated weights for policy 1, policy_version 566764 (0.0009) [2023-12-27 09:33:38,187][20347] Updated weights for policy 0, policy_version 566949 (0.0007) [2023-12-27 09:33:38,232][20347] Updated weights for policy 0, policy_version 566959 (0.0009) [2023-12-27 09:33:38,278][20347] Updated weights for policy 0, policy_version 566969 (0.0009) [2023-12-27 09:33:38,309][20381] Updated weights for policy 1, policy_version 566774 (0.0006) [2023-12-27 09:33:38,363][20381] Updated weights for policy 1, policy_version 566784 (0.0008) [2023-12-27 09:33:38,410][20381] Updated weights for policy 1, policy_version 566794 (0.0007) [2023-12-27 09:33:39,051][20347] Updated weights for policy 0, policy_version 566979 (0.0007) [2023-12-27 09:33:39,068][20381] Updated weights for policy 1, policy_version 566804 (0.0011) [2023-12-27 09:33:39,108][20347] Updated weights for policy 0, policy_version 566989 (0.0006) [2023-12-27 09:33:39,124][20381] Updated weights for policy 1, policy_version 566814 (0.0011) [2023-12-27 09:33:39,160][20347] Updated weights for policy 0, policy_version 566999 (0.0005) [2023-12-27 09:33:39,179][20381] Updated weights for policy 1, policy_version 566824 (0.0011) [2023-12-27 09:33:39,912][20347] Updated weights for policy 0, policy_version 567009 (0.0006) [2023-12-27 09:33:39,944][20381] Updated weights for policy 1, policy_version 566834 (0.0010) [2023-12-27 09:33:39,970][20347] Updated weights for policy 0, policy_version 567019 (0.0007) [2023-12-27 09:33:39,998][20381] Updated weights for policy 1, policy_version 566844 (0.0007) [2023-12-27 09:33:40,028][20347] Updated weights for policy 0, policy_version 567029 (0.0006) [2023-12-27 09:33:40,058][20381] Updated weights for policy 1, policy_version 566854 (0.0007) [2023-12-27 09:33:40,076][20347] Updated weights for policy 0, policy_version 567039 (0.0007) [2023-12-27 09:33:40,123][20381] Updated weights for policy 1, policy_version 566864 (0.0006) [2023-12-27 09:33:40,772][20347] Updated weights for policy 0, policy_version 567049 (0.0006) [2023-12-27 09:33:40,790][20381] Updated weights for policy 1, policy_version 566874 (0.0010) [2023-12-27 09:33:40,824][20347] Updated weights for policy 0, policy_version 567059 (0.0006) [2023-12-27 09:33:40,843][20381] Updated weights for policy 1, policy_version 566884 (0.0007) [2023-12-27 09:33:40,881][20347] Updated weights for policy 0, policy_version 567069 (0.0005) [2023-12-27 09:33:40,894][20381] Updated weights for policy 1, policy_version 566894 (0.0007) [2023-12-27 09:33:41,622][20347] Updated weights for policy 0, policy_version 567079 (0.0006) [2023-12-27 09:33:41,639][20381] Updated weights for policy 1, policy_version 566904 (0.0010) [2023-12-27 09:33:41,676][20347] Updated weights for policy 0, policy_version 567089 (0.0007) [2023-12-27 09:33:41,697][20381] Updated weights for policy 1, policy_version 566914 (0.0011) [2023-12-27 09:33:41,742][20347] Updated weights for policy 0, policy_version 567099 (0.0007) [2023-12-27 09:33:41,763][20381] Updated weights for policy 1, policy_version 566924 (0.0008) [2023-12-27 09:33:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 290349056. Throughput: 0: 10436.0, 1: 10293.6. Samples: 290344048. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:42,363][19170] Avg episode reward: [(0, '980.900'), (1, '884.360')] [2023-12-27 09:33:42,414][20381] Updated weights for policy 1, policy_version 566934 (0.0008) [2023-12-27 09:33:42,463][20381] Updated weights for policy 1, policy_version 566944 (0.0008) [2023-12-27 09:33:42,492][20347] Updated weights for policy 0, policy_version 567109 (0.0008) [2023-12-27 09:33:42,513][20381] Updated weights for policy 1, policy_version 566954 (0.0006) [2023-12-27 09:33:42,556][20347] Updated weights for policy 0, policy_version 567119 (0.0007) [2023-12-27 09:33:42,611][20347] Updated weights for policy 0, policy_version 567129 (0.0008) [2023-12-27 09:33:43,235][20381] Updated weights for policy 1, policy_version 566964 (0.0007) [2023-12-27 09:33:43,251][20347] Updated weights for policy 0, policy_version 567139 (0.0008) [2023-12-27 09:33:43,297][20381] Updated weights for policy 1, policy_version 566974 (0.0006) [2023-12-27 09:33:43,302][20347] Updated weights for policy 0, policy_version 567149 (0.0010) [2023-12-27 09:33:43,351][20381] Updated weights for policy 1, policy_version 566984 (0.0005) [2023-12-27 09:33:43,352][20291] KL-divergence is very high: 126.8543 [2023-12-27 09:33:43,357][20347] Updated weights for policy 0, policy_version 567159 (0.0010) [2023-12-27 09:33:44,019][20347] Updated weights for policy 0, policy_version 567169 (0.0006) [2023-12-27 09:33:44,025][20381] Updated weights for policy 1, policy_version 566994 (0.0006) [2023-12-27 09:33:44,065][20347] Updated weights for policy 0, policy_version 567179 (0.0005) [2023-12-27 09:33:44,069][20381] Updated weights for policy 1, policy_version 567004 (0.0007) [2023-12-27 09:33:44,111][20347] Updated weights for policy 0, policy_version 567189 (0.0005) [2023-12-27 09:33:44,124][20381] Updated weights for policy 1, policy_version 567014 (0.0008) [2023-12-27 09:33:44,170][20347] Updated weights for policy 0, policy_version 567199 (0.0008) [2023-12-27 09:33:44,184][20381] Updated weights for policy 1, policy_version 567024 (0.0007) [2023-12-27 09:33:44,756][20347] Updated weights for policy 0, policy_version 567209 (0.0009) [2023-12-27 09:33:44,822][20347] Updated weights for policy 0, policy_version 567219 (0.0011) [2023-12-27 09:33:44,845][20381] Updated weights for policy 1, policy_version 567034 (0.0008) [2023-12-27 09:33:44,878][20347] Updated weights for policy 0, policy_version 567229 (0.0011) [2023-12-27 09:33:44,899][20381] Updated weights for policy 1, policy_version 567044 (0.0006) [2023-12-27 09:33:44,955][20381] Updated weights for policy 1, policy_version 567054 (0.0008) [2023-12-27 09:33:45,558][20347] Updated weights for policy 0, policy_version 567239 (0.0010) [2023-12-27 09:33:45,615][20347] Updated weights for policy 0, policy_version 567249 (0.0010) [2023-12-27 09:33:45,653][20381] Updated weights for policy 1, policy_version 567064 (0.0006) [2023-12-27 09:33:45,673][20347] Updated weights for policy 0, policy_version 567259 (0.0010) [2023-12-27 09:33:45,706][20381] Updated weights for policy 1, policy_version 567074 (0.0006) [2023-12-27 09:33:45,761][20381] Updated weights for policy 1, policy_version 567084 (0.0007) [2023-12-27 09:33:46,289][20347] Updated weights for policy 0, policy_version 567269 (0.0006) [2023-12-27 09:33:46,350][20347] Updated weights for policy 0, policy_version 567279 (0.0006) [2023-12-27 09:33:46,405][20347] Updated weights for policy 0, policy_version 567289 (0.0009) [2023-12-27 09:33:46,487][20381] Updated weights for policy 1, policy_version 567094 (0.0007) [2023-12-27 09:33:46,544][20381] Updated weights for policy 1, policy_version 567104 (0.0005) [2023-12-27 09:33:46,598][20381] Updated weights for policy 1, policy_version 567114 (0.0005) [2023-12-27 09:33:47,023][20347] Updated weights for policy 0, policy_version 567299 (0.0010) [2023-12-27 09:33:47,084][20347] Updated weights for policy 0, policy_version 567309 (0.0010) [2023-12-27 09:33:47,144][20381] Updated weights for policy 1, policy_version 567124 (0.0005) [2023-12-27 09:33:47,148][20347] Updated weights for policy 0, policy_version 567319 (0.0010) [2023-12-27 09:33:47,195][20381] Updated weights for policy 1, policy_version 567134 (0.0005) [2023-12-27 09:33:47,251][20381] Updated weights for policy 1, policy_version 567144 (0.0005) [2023-12-27 09:33:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 290463744. Throughput: 0: 10404.2, 1: 10379.5. Samples: 290468892. Policy #0 lag: (min: 30.0, avg: 39.0, max: 62.0) [2023-12-27 09:33:47,364][19170] Avg episode reward: [(0, '914.840'), (1, '932.300')] [2023-12-27 09:33:47,773][20347] Updated weights for policy 0, policy_version 567329 (0.0010) [2023-12-27 09:33:47,828][20347] Updated weights for policy 0, policy_version 567339 (0.0009) [2023-12-27 09:33:47,854][20381] Updated weights for policy 1, policy_version 567154 (0.0005) [2023-12-27 09:33:47,883][20347] Updated weights for policy 0, policy_version 567349 (0.0010) [2023-12-27 09:33:47,920][20381] Updated weights for policy 1, policy_version 567164 (0.0005) [2023-12-27 09:33:47,928][20347] Updated weights for policy 0, policy_version 567359 (0.0010) [2023-12-27 09:33:47,983][20381] Updated weights for policy 1, policy_version 567174 (0.0005) [2023-12-27 09:33:48,038][20381] Updated weights for policy 1, policy_version 567184 (0.0005) [2023-12-27 09:33:48,617][20347] Updated weights for policy 0, policy_version 567369 (0.0010) [2023-12-27 09:33:48,642][20381] Updated weights for policy 1, policy_version 567194 (0.0011) [2023-12-27 09:33:48,676][20347] Updated weights for policy 0, policy_version 567379 (0.0010) [2023-12-27 09:33:48,687][20381] Updated weights for policy 1, policy_version 567204 (0.0010) [2023-12-27 09:33:48,733][20381] Updated weights for policy 1, policy_version 567214 (0.0011) [2023-12-27 09:33:48,737][20347] Updated weights for policy 0, policy_version 567389 (0.0010) [2023-12-27 09:33:49,369][20381] Updated weights for policy 1, policy_version 567224 (0.0007) [2023-12-27 09:33:49,419][20347] Updated weights for policy 0, policy_version 567399 (0.0010) [2023-12-27 09:33:49,426][20381] Updated weights for policy 1, policy_version 567234 (0.0009) [2023-12-27 09:33:49,474][20347] Updated weights for policy 0, policy_version 567409 (0.0010) [2023-12-27 09:33:49,476][20381] Updated weights for policy 1, policy_version 567244 (0.0008) [2023-12-27 09:33:49,529][20347] Updated weights for policy 0, policy_version 567419 (0.0010) [2023-12-27 09:33:50,172][20381] Updated weights for policy 1, policy_version 567254 (0.0007) [2023-12-27 09:33:50,222][20381] Updated weights for policy 1, policy_version 567264 (0.0008) [2023-12-27 09:33:50,240][20347] Updated weights for policy 0, policy_version 567429 (0.0008) [2023-12-27 09:33:50,276][20381] Updated weights for policy 1, policy_version 567274 (0.0008) [2023-12-27 09:33:50,297][20347] Updated weights for policy 0, policy_version 567439 (0.0005) [2023-12-27 09:33:50,356][20347] Updated weights for policy 0, policy_version 567449 (0.0006) [2023-12-27 09:33:50,981][20381] Updated weights for policy 1, policy_version 567284 (0.0007) [2023-12-27 09:33:51,031][20347] Updated weights for policy 0, policy_version 567459 (0.0007) [2023-12-27 09:33:51,045][20381] Updated weights for policy 1, policy_version 567294 (0.0008) [2023-12-27 09:33:51,093][20347] Updated weights for policy 0, policy_version 567469 (0.0006) [2023-12-27 09:33:51,099][20381] Updated weights for policy 1, policy_version 567304 (0.0006) [2023-12-27 09:33:51,155][20347] Updated weights for policy 0, policy_version 567479 (0.0008) [2023-12-27 09:33:51,793][20381] Updated weights for policy 1, policy_version 567314 (0.0008) [2023-12-27 09:33:51,855][20381] Updated weights for policy 1, policy_version 567324 (0.0008) [2023-12-27 09:33:51,868][20347] Updated weights for policy 0, policy_version 567489 (0.0007) [2023-12-27 09:33:51,914][20381] Updated weights for policy 1, policy_version 567334 (0.0008) [2023-12-27 09:33:51,924][20347] Updated weights for policy 0, policy_version 567499 (0.0006) [2023-12-27 09:33:51,969][20381] Updated weights for policy 1, policy_version 567344 (0.0008) [2023-12-27 09:33:51,978][20347] Updated weights for policy 0, policy_version 567509 (0.0007) [2023-12-27 09:33:52,027][20347] Updated weights for policy 0, policy_version 567519 (0.0007) [2023-12-27 09:33:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 290562048. Throughput: 0: 10385.5, 1: 10438.7. Samples: 290531740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:33:52,363][19170] Avg episode reward: [(0, '924.940'), (1, '932.720')] [2023-12-27 09:33:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000567344_145260544.pth... [2023-12-27 09:33:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000567520_145301504.pth... [2023-12-27 09:33:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000566304_144990208.pth [2023-12-27 09:33:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000566128_144949248.pth [2023-12-27 09:33:52,675][20381] Updated weights for policy 1, policy_version 567354 (0.0007) [2023-12-27 09:33:52,732][20381] Updated weights for policy 1, policy_version 567364 (0.0007) [2023-12-27 09:33:52,760][20347] Updated weights for policy 0, policy_version 567529 (0.0007) [2023-12-27 09:33:52,784][20381] Updated weights for policy 1, policy_version 567374 (0.0006) [2023-12-27 09:33:52,821][20347] Updated weights for policy 0, policy_version 567539 (0.0007) [2023-12-27 09:33:52,881][20347] Updated weights for policy 0, policy_version 567549 (0.0008) [2023-12-27 09:33:53,404][20381] Updated weights for policy 1, policy_version 567384 (0.0006) [2023-12-27 09:33:53,448][20381] Updated weights for policy 1, policy_version 567394 (0.0005) [2023-12-27 09:33:53,496][20381] Updated weights for policy 1, policy_version 567404 (0.0005) [2023-12-27 09:33:53,581][20347] Updated weights for policy 0, policy_version 567559 (0.0008) [2023-12-27 09:33:53,626][20347] Updated weights for policy 0, policy_version 567569 (0.0008) [2023-12-27 09:33:53,671][20347] Updated weights for policy 0, policy_version 567579 (0.0007) [2023-12-27 09:33:54,149][20381] Updated weights for policy 1, policy_version 567414 (0.0007) [2023-12-27 09:33:54,200][20381] Updated weights for policy 1, policy_version 567424 (0.0006) [2023-12-27 09:33:54,260][20381] Updated weights for policy 1, policy_version 567434 (0.0006) [2023-12-27 09:33:54,321][20347] Updated weights for policy 0, policy_version 567589 (0.0006) [2023-12-27 09:33:54,392][20347] Updated weights for policy 0, policy_version 567599 (0.0007) [2023-12-27 09:33:54,454][20347] Updated weights for policy 0, policy_version 567609 (0.0008) [2023-12-27 09:33:54,896][20381] Updated weights for policy 1, policy_version 567444 (0.0006) [2023-12-27 09:33:54,952][20381] Updated weights for policy 1, policy_version 567454 (0.0008) [2023-12-27 09:33:55,011][20381] Updated weights for policy 1, policy_version 567464 (0.0007) [2023-12-27 09:33:55,113][20347] Updated weights for policy 0, policy_version 567619 (0.0007) [2023-12-27 09:33:55,166][20347] Updated weights for policy 0, policy_version 567629 (0.0007) [2023-12-27 09:33:55,221][20347] Updated weights for policy 0, policy_version 567639 (0.0006) [2023-12-27 09:33:55,704][20381] Updated weights for policy 1, policy_version 567474 (0.0008) [2023-12-27 09:33:55,758][20381] Updated weights for policy 1, policy_version 567484 (0.0005) [2023-12-27 09:33:55,814][20381] Updated weights for policy 1, policy_version 567494 (0.0006) [2023-12-27 09:33:55,856][20347] Updated weights for policy 0, policy_version 567649 (0.0006) [2023-12-27 09:33:55,875][20381] Updated weights for policy 1, policy_version 567504 (0.0005) [2023-12-27 09:33:55,909][20347] Updated weights for policy 0, policy_version 567659 (0.0007) [2023-12-27 09:33:55,964][20347] Updated weights for policy 0, policy_version 567669 (0.0008) [2023-12-27 09:33:56,013][20347] Updated weights for policy 0, policy_version 567679 (0.0009) [2023-12-27 09:33:56,468][20381] Updated weights for policy 1, policy_version 567514 (0.0008) [2023-12-27 09:33:56,518][20381] Updated weights for policy 1, policy_version 567524 (0.0008) [2023-12-27 09:33:56,561][20381] Updated weights for policy 1, policy_version 567534 (0.0007) [2023-12-27 09:33:56,698][20347] Updated weights for policy 0, policy_version 567689 (0.0008) [2023-12-27 09:33:56,756][20347] Updated weights for policy 0, policy_version 567699 (0.0006) [2023-12-27 09:33:56,812][20347] Updated weights for policy 0, policy_version 567709 (0.0006) [2023-12-27 09:33:57,255][20381] Updated weights for policy 1, policy_version 567544 (0.0006) [2023-12-27 09:33:57,317][20381] Updated weights for policy 1, policy_version 567554 (0.0006) [2023-12-27 09:33:57,360][20347] Updated weights for policy 0, policy_version 567719 (0.0006) [2023-12-27 09:33:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 290660352. Throughput: 0: 10316.7, 1: 10454.6. Samples: 290655504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:33:57,363][19170] Avg episode reward: [(0, '980.960'), (1, '903.490')] [2023-12-27 09:33:57,377][20381] Updated weights for policy 1, policy_version 567564 (0.0007) [2023-12-27 09:33:57,422][20347] Updated weights for policy 0, policy_version 567729 (0.0006) [2023-12-27 09:33:57,483][20347] Updated weights for policy 0, policy_version 567739 (0.0005) [2023-12-27 09:33:58,009][20347] Updated weights for policy 0, policy_version 567749 (0.0007) [2023-12-27 09:33:58,060][20381] Updated weights for policy 1, policy_version 567574 (0.0005) [2023-12-27 09:33:58,066][20347] Updated weights for policy 0, policy_version 567759 (0.0009) [2023-12-27 09:33:58,115][20381] Updated weights for policy 1, policy_version 567584 (0.0006) [2023-12-27 09:33:58,127][20347] Updated weights for policy 0, policy_version 567769 (0.0010) [2023-12-27 09:33:58,170][20381] Updated weights for policy 1, policy_version 567594 (0.0007) [2023-12-27 09:33:58,779][20347] Updated weights for policy 0, policy_version 567779 (0.0011) [2023-12-27 09:33:58,840][20347] Updated weights for policy 0, policy_version 567789 (0.0009) [2023-12-27 09:33:58,889][20347] Updated weights for policy 0, policy_version 567799 (0.0009) [2023-12-27 09:33:58,966][20381] Updated weights for policy 1, policy_version 567604 (0.0008) [2023-12-27 09:33:59,024][20381] Updated weights for policy 1, policy_version 567614 (0.0006) [2023-12-27 09:33:59,081][20381] Updated weights for policy 1, policy_version 567624 (0.0006) [2023-12-27 09:33:59,603][20347] Updated weights for policy 0, policy_version 567809 (0.0008) [2023-12-27 09:33:59,659][20347] Updated weights for policy 0, policy_version 567819 (0.0005) [2023-12-27 09:33:59,718][20347] Updated weights for policy 0, policy_version 567829 (0.0009) [2023-12-27 09:33:59,768][20381] Updated weights for policy 1, policy_version 567634 (0.0006) [2023-12-27 09:33:59,775][20347] Updated weights for policy 0, policy_version 567839 (0.0010) [2023-12-27 09:33:59,830][20381] Updated weights for policy 1, policy_version 567644 (0.0006) [2023-12-27 09:33:59,892][20381] Updated weights for policy 1, policy_version 567654 (0.0006) [2023-12-27 09:33:59,953][20381] Updated weights for policy 1, policy_version 567664 (0.0006) [2023-12-27 09:34:00,406][20347] Updated weights for policy 0, policy_version 567849 (0.0010) [2023-12-27 09:34:00,447][20347] Updated weights for policy 0, policy_version 567859 (0.0010) [2023-12-27 09:34:00,495][20347] Updated weights for policy 0, policy_version 567869 (0.0010) [2023-12-27 09:34:00,551][20381] Updated weights for policy 1, policy_version 567674 (0.0005) [2023-12-27 09:34:00,606][20381] Updated weights for policy 1, policy_version 567684 (0.0005) [2023-12-27 09:34:00,662][20381] Updated weights for policy 1, policy_version 567694 (0.0006) [2023-12-27 09:34:01,204][20347] Updated weights for policy 0, policy_version 567879 (0.0010) [2023-12-27 09:34:01,232][20381] Updated weights for policy 1, policy_version 567704 (0.0006) [2023-12-27 09:34:01,260][20347] Updated weights for policy 0, policy_version 567889 (0.0010) [2023-12-27 09:34:01,291][20381] Updated weights for policy 1, policy_version 567714 (0.0005) [2023-12-27 09:34:01,314][20347] Updated weights for policy 0, policy_version 567899 (0.0009) [2023-12-27 09:34:01,350][20381] Updated weights for policy 1, policy_version 567724 (0.0007) [2023-12-27 09:34:01,993][20347] Updated weights for policy 0, policy_version 567909 (0.0006) [2023-12-27 09:34:02,021][20381] Updated weights for policy 1, policy_version 567734 (0.0006) [2023-12-27 09:34:02,056][20347] Updated weights for policy 0, policy_version 567919 (0.0005) [2023-12-27 09:34:02,077][20381] Updated weights for policy 1, policy_version 567744 (0.0005) [2023-12-27 09:34:02,110][20347] Updated weights for policy 0, policy_version 567929 (0.0005) [2023-12-27 09:34:02,133][20381] Updated weights for policy 1, policy_version 567754 (0.0005) [2023-12-27 09:34:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 290775040. Throughput: 0: 10331.5, 1: 10445.6. Samples: 290782088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:02,363][19170] Avg episode reward: [(0, '972.250'), (1, '951.590')] [2023-12-27 09:34:02,701][20347] Updated weights for policy 0, policy_version 567939 (0.0006) [2023-12-27 09:34:02,734][20291] KL-divergence is very high: 152.1017 [2023-12-27 09:34:02,744][20381] Updated weights for policy 1, policy_version 567764 (0.0007) [2023-12-27 09:34:02,761][20347] Updated weights for policy 0, policy_version 567949 (0.0007) [2023-12-27 09:34:02,793][20381] Updated weights for policy 1, policy_version 567774 (0.0010) [2023-12-27 09:34:02,814][20347] Updated weights for policy 0, policy_version 567959 (0.0005) [2023-12-27 09:34:02,823][20291] KL-divergence is very high: 132.3844 [2023-12-27 09:34:02,850][20381] Updated weights for policy 1, policy_version 567784 (0.0010) [2023-12-27 09:34:03,505][20347] Updated weights for policy 0, policy_version 567969 (0.0006) [2023-12-27 09:34:03,551][20381] Updated weights for policy 1, policy_version 567794 (0.0010) [2023-12-27 09:34:03,562][20347] Updated weights for policy 0, policy_version 567979 (0.0006) [2023-12-27 09:34:03,606][20381] Updated weights for policy 1, policy_version 567804 (0.0006) [2023-12-27 09:34:03,612][20347] Updated weights for policy 0, policy_version 567989 (0.0007) [2023-12-27 09:34:03,665][20381] Updated weights for policy 1, policy_version 567814 (0.0008) [2023-12-27 09:34:03,674][20347] Updated weights for policy 0, policy_version 567999 (0.0006) [2023-12-27 09:34:03,708][20381] Updated weights for policy 1, policy_version 567824 (0.0008) [2023-12-27 09:34:04,265][20347] Updated weights for policy 0, policy_version 568009 (0.0006) [2023-12-27 09:34:04,323][20347] Updated weights for policy 0, policy_version 568019 (0.0005) [2023-12-27 09:34:04,355][20381] Updated weights for policy 1, policy_version 567834 (0.0011) [2023-12-27 09:34:04,386][20347] Updated weights for policy 0, policy_version 568029 (0.0005) [2023-12-27 09:34:04,417][20381] Updated weights for policy 1, policy_version 567844 (0.0010) [2023-12-27 09:34:04,476][20381] Updated weights for policy 1, policy_version 567854 (0.0009) [2023-12-27 09:34:05,055][20347] Updated weights for policy 0, policy_version 568039 (0.0008) [2023-12-27 09:34:05,105][20347] Updated weights for policy 0, policy_version 568049 (0.0006) [2023-12-27 09:34:05,111][20381] Updated weights for policy 1, policy_version 567864 (0.0006) [2023-12-27 09:34:05,160][20347] Updated weights for policy 0, policy_version 568059 (0.0007) [2023-12-27 09:34:05,173][20381] Updated weights for policy 1, policy_version 567874 (0.0007) [2023-12-27 09:34:05,227][20381] Updated weights for policy 1, policy_version 567884 (0.0007) [2023-12-27 09:34:05,808][20347] Updated weights for policy 0, policy_version 568069 (0.0006) [2023-12-27 09:34:05,889][20347] Updated weights for policy 0, policy_version 568079 (0.0007) [2023-12-27 09:34:05,938][20381] Updated weights for policy 1, policy_version 567894 (0.0010) [2023-12-27 09:34:05,944][20347] Updated weights for policy 0, policy_version 568089 (0.0006) [2023-12-27 09:34:05,999][20381] Updated weights for policy 1, policy_version 567904 (0.0008) [2023-12-27 09:34:06,066][20381] Updated weights for policy 1, policy_version 567914 (0.0007) [2023-12-27 09:34:06,649][20347] Updated weights for policy 0, policy_version 568099 (0.0006) [2023-12-27 09:34:06,693][20347] Updated weights for policy 0, policy_version 568109 (0.0008) [2023-12-27 09:34:06,700][20381] Updated weights for policy 1, policy_version 567924 (0.0008) [2023-12-27 09:34:06,741][20347] Updated weights for policy 0, policy_version 568119 (0.0008) [2023-12-27 09:34:06,758][20381] Updated weights for policy 1, policy_version 567934 (0.0005) [2023-12-27 09:34:06,804][20381] Updated weights for policy 1, policy_version 567944 (0.0007) [2023-12-27 09:34:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 290873344. Throughput: 0: 10345.8, 1: 10463.0. Samples: 290845488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:07,363][19170] Avg episode reward: [(0, '916.040'), (1, '922.280')] [2023-12-27 09:34:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000568128_145457152.pth... [2023-12-27 09:34:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000567952_145416192.pth... [2023-12-27 09:34:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000566944_145154048.pth [2023-12-27 09:34:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000566736_145104896.pth [2023-12-27 09:34:07,450][20347] Updated weights for policy 0, policy_version 568129 (0.0008) [2023-12-27 09:34:07,487][20381] Updated weights for policy 1, policy_version 567954 (0.0006) [2023-12-27 09:34:07,502][20347] Updated weights for policy 0, policy_version 568139 (0.0008) [2023-12-27 09:34:07,541][20381] Updated weights for policy 1, policy_version 567964 (0.0008) [2023-12-27 09:34:07,557][20347] Updated weights for policy 0, policy_version 568149 (0.0010) [2023-12-27 09:34:07,597][20381] Updated weights for policy 1, policy_version 567974 (0.0010) [2023-12-27 09:34:07,615][20347] Updated weights for policy 0, policy_version 568159 (0.0008) [2023-12-27 09:34:07,652][20381] Updated weights for policy 1, policy_version 567984 (0.0007) [2023-12-27 09:34:08,301][20347] Updated weights for policy 0, policy_version 568169 (0.0007) [2023-12-27 09:34:08,336][20381] Updated weights for policy 1, policy_version 567994 (0.0006) [2023-12-27 09:34:08,366][20347] Updated weights for policy 0, policy_version 568179 (0.0006) [2023-12-27 09:34:08,399][20381] Updated weights for policy 1, policy_version 568004 (0.0008) [2023-12-27 09:34:08,430][20347] Updated weights for policy 0, policy_version 568189 (0.0006) [2023-12-27 09:34:08,462][20381] Updated weights for policy 1, policy_version 568014 (0.0007) [2023-12-27 09:34:09,127][20347] Updated weights for policy 0, policy_version 568199 (0.0009) [2023-12-27 09:34:09,147][20381] Updated weights for policy 1, policy_version 568024 (0.0006) [2023-12-27 09:34:09,185][20347] Updated weights for policy 0, policy_version 568209 (0.0010) [2023-12-27 09:34:09,209][20381] Updated weights for policy 1, policy_version 568034 (0.0005) [2023-12-27 09:34:09,246][20347] Updated weights for policy 0, policy_version 568219 (0.0010) [2023-12-27 09:34:09,270][20381] Updated weights for policy 1, policy_version 568044 (0.0007) [2023-12-27 09:34:09,930][20347] Updated weights for policy 0, policy_version 568229 (0.0009) [2023-12-27 09:34:09,990][20347] Updated weights for policy 0, policy_version 568239 (0.0009) [2023-12-27 09:34:09,995][20381] Updated weights for policy 1, policy_version 568054 (0.0006) [2023-12-27 09:34:10,047][20347] Updated weights for policy 0, policy_version 568249 (0.0010) [2023-12-27 09:34:10,056][20381] Updated weights for policy 1, policy_version 568064 (0.0005) [2023-12-27 09:34:10,116][20381] Updated weights for policy 1, policy_version 568074 (0.0007) [2023-12-27 09:34:10,762][20347] Updated weights for policy 0, policy_version 568259 (0.0007) [2023-12-27 09:34:10,819][20347] Updated weights for policy 0, policy_version 568269 (0.0007) [2023-12-27 09:34:10,836][20381] Updated weights for policy 1, policy_version 568084 (0.0007) [2023-12-27 09:34:10,873][20347] Updated weights for policy 0, policy_version 568279 (0.0005) [2023-12-27 09:34:10,886][20381] Updated weights for policy 1, policy_version 568094 (0.0005) [2023-12-27 09:34:10,938][20381] Updated weights for policy 1, policy_version 568104 (0.0006) [2023-12-27 09:34:11,573][20347] Updated weights for policy 0, policy_version 568289 (0.0007) [2023-12-27 09:34:11,618][20291] KL-divergence is very high: 157.1174 [2023-12-27 09:34:11,639][20347] Updated weights for policy 0, policy_version 568299 (0.0007) [2023-12-27 09:34:11,672][20291] KL-divergence is very high: 102.3122 [2023-12-27 09:34:11,703][20347] Updated weights for policy 0, policy_version 568309 (0.0008) [2023-12-27 09:34:11,746][20381] Updated weights for policy 1, policy_version 568114 (0.0008) [2023-12-27 09:34:11,772][20347] Updated weights for policy 0, policy_version 568319 (0.0009) [2023-12-27 09:34:11,805][20381] Updated weights for policy 1, policy_version 568124 (0.0008) [2023-12-27 09:34:11,856][20381] Updated weights for policy 1, policy_version 568134 (0.0008) [2023-12-27 09:34:11,920][20381] Updated weights for policy 1, policy_version 568144 (0.0008) [2023-12-27 09:34:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 290971648. Throughput: 0: 10368.7, 1: 10372.2. Samples: 290964860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:12,363][19170] Avg episode reward: [(0, '776.670'), (1, '863.680')] [2023-12-27 09:34:12,397][20347] Updated weights for policy 0, policy_version 568329 (0.0009) [2023-12-27 09:34:12,460][20347] Updated weights for policy 0, policy_version 568339 (0.0007) [2023-12-27 09:34:12,513][20347] Updated weights for policy 0, policy_version 568349 (0.0008) [2023-12-27 09:34:12,574][20381] Updated weights for policy 1, policy_version 568154 (0.0008) [2023-12-27 09:34:12,621][20381] Updated weights for policy 1, policy_version 568164 (0.0008) [2023-12-27 09:34:12,678][20381] Updated weights for policy 1, policy_version 568174 (0.0008) [2023-12-27 09:34:13,182][20347] Updated weights for policy 0, policy_version 568359 (0.0008) [2023-12-27 09:34:13,239][20347] Updated weights for policy 0, policy_version 568369 (0.0008) [2023-12-27 09:34:13,288][20347] Updated weights for policy 0, policy_version 568379 (0.0008) [2023-12-27 09:34:13,415][20381] Updated weights for policy 1, policy_version 568184 (0.0007) [2023-12-27 09:34:13,471][20381] Updated weights for policy 1, policy_version 568194 (0.0008) [2023-12-27 09:34:13,520][20381] Updated weights for policy 1, policy_version 568204 (0.0007) [2023-12-27 09:34:13,978][20347] Updated weights for policy 0, policy_version 568389 (0.0006) [2023-12-27 09:34:14,038][20347] Updated weights for policy 0, policy_version 568399 (0.0007) [2023-12-27 09:34:14,092][20347] Updated weights for policy 0, policy_version 568409 (0.0008) [2023-12-27 09:34:14,233][20381] Updated weights for policy 1, policy_version 568214 (0.0007) [2023-12-27 09:34:14,287][20381] Updated weights for policy 1, policy_version 568224 (0.0006) [2023-12-27 09:34:14,333][20381] Updated weights for policy 1, policy_version 568234 (0.0009) [2023-12-27 09:34:14,746][20347] Updated weights for policy 0, policy_version 568419 (0.0009) [2023-12-27 09:34:14,808][20347] Updated weights for policy 0, policy_version 568429 (0.0011) [2023-12-27 09:34:14,865][20347] Updated weights for policy 0, policy_version 568439 (0.0010) [2023-12-27 09:34:15,006][20381] Updated weights for policy 1, policy_version 568244 (0.0007) [2023-12-27 09:34:15,061][20381] Updated weights for policy 1, policy_version 568254 (0.0008) [2023-12-27 09:34:15,123][20381] Updated weights for policy 1, policy_version 568264 (0.0010) [2023-12-27 09:34:15,533][20347] Updated weights for policy 0, policy_version 568449 (0.0010) [2023-12-27 09:34:15,582][20347] Updated weights for policy 0, policy_version 568459 (0.0008) [2023-12-27 09:34:15,632][20347] Updated weights for policy 0, policy_version 568469 (0.0007) [2023-12-27 09:34:15,682][20347] Updated weights for policy 0, policy_version 568479 (0.0008) [2023-12-27 09:34:15,815][20381] Updated weights for policy 1, policy_version 568274 (0.0010) [2023-12-27 09:34:15,872][20381] Updated weights for policy 1, policy_version 568284 (0.0008) [2023-12-27 09:34:15,927][20381] Updated weights for policy 1, policy_version 568294 (0.0008) [2023-12-27 09:34:15,980][20381] Updated weights for policy 1, policy_version 568304 (0.0007) [2023-12-27 09:34:16,387][20347] Updated weights for policy 0, policy_version 568489 (0.0008) [2023-12-27 09:34:16,450][20347] Updated weights for policy 0, policy_version 568499 (0.0008) [2023-12-27 09:34:16,511][20347] Updated weights for policy 0, policy_version 568509 (0.0007) [2023-12-27 09:34:16,641][20381] Updated weights for policy 1, policy_version 568314 (0.0009) [2023-12-27 09:34:16,696][20381] Updated weights for policy 1, policy_version 568324 (0.0010) [2023-12-27 09:34:16,751][20381] Updated weights for policy 1, policy_version 568334 (0.0010) [2023-12-27 09:34:17,201][20347] Updated weights for policy 0, policy_version 568519 (0.0010) [2023-12-27 09:34:17,255][20347] Updated weights for policy 0, policy_version 568529 (0.0010) [2023-12-27 09:34:17,315][20347] Updated weights for policy 0, policy_version 568539 (0.0006) [2023-12-27 09:34:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 291078144. Throughput: 0: 10322.7, 1: 10309.8. Samples: 291087132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:17,363][19170] Avg episode reward: [(0, '735.100'), (1, '853.960')] [2023-12-27 09:34:17,417][20381] Updated weights for policy 1, policy_version 568344 (0.0009) [2023-12-27 09:34:17,472][20381] Updated weights for policy 1, policy_version 568354 (0.0008) [2023-12-27 09:34:17,518][20381] Updated weights for policy 1, policy_version 568364 (0.0008) [2023-12-27 09:34:17,922][20347] Updated weights for policy 0, policy_version 568549 (0.0008) [2023-12-27 09:34:17,973][20347] Updated weights for policy 0, policy_version 568559 (0.0010) [2023-12-27 09:34:18,024][20347] Updated weights for policy 0, policy_version 568569 (0.0010) [2023-12-27 09:34:18,234][20381] Updated weights for policy 1, policy_version 568374 (0.0007) [2023-12-27 09:34:18,290][20381] Updated weights for policy 1, policy_version 568384 (0.0005) [2023-12-27 09:34:18,350][20381] Updated weights for policy 1, policy_version 568394 (0.0006) [2023-12-27 09:34:18,720][20347] Updated weights for policy 0, policy_version 568579 (0.0009) [2023-12-27 09:34:18,781][20347] Updated weights for policy 0, policy_version 568589 (0.0009) [2023-12-27 09:34:18,843][20347] Updated weights for policy 0, policy_version 568599 (0.0010) [2023-12-27 09:34:19,006][20381] Updated weights for policy 1, policy_version 568404 (0.0007) [2023-12-27 09:34:19,057][20381] Updated weights for policy 1, policy_version 568414 (0.0007) [2023-12-27 09:34:19,109][20381] Updated weights for policy 1, policy_version 568424 (0.0007) [2023-12-27 09:34:19,500][20347] Updated weights for policy 0, policy_version 568609 (0.0010) [2023-12-27 09:34:19,562][20347] Updated weights for policy 0, policy_version 568619 (0.0006) [2023-12-27 09:34:19,626][20347] Updated weights for policy 0, policy_version 568629 (0.0006) [2023-12-27 09:34:19,687][20347] Updated weights for policy 0, policy_version 568639 (0.0010) [2023-12-27 09:34:19,865][20381] Updated weights for policy 1, policy_version 568434 (0.0007) [2023-12-27 09:34:19,929][20381] Updated weights for policy 1, policy_version 568444 (0.0006) [2023-12-27 09:34:19,984][20381] Updated weights for policy 1, policy_version 568454 (0.0006) [2023-12-27 09:34:20,034][20381] Updated weights for policy 1, policy_version 568464 (0.0007) [2023-12-27 09:34:20,369][20347] Updated weights for policy 0, policy_version 568649 (0.0007) [2023-12-27 09:34:20,418][20347] Updated weights for policy 0, policy_version 568659 (0.0007) [2023-12-27 09:34:20,467][20347] Updated weights for policy 0, policy_version 568669 (0.0005) [2023-12-27 09:34:20,734][20381] Updated weights for policy 1, policy_version 568474 (0.0007) [2023-12-27 09:34:20,786][20381] Updated weights for policy 1, policy_version 568484 (0.0005) [2023-12-27 09:34:20,844][20381] Updated weights for policy 1, policy_version 568494 (0.0008) [2023-12-27 09:34:21,152][20347] Updated weights for policy 0, policy_version 568679 (0.0008) [2023-12-27 09:34:21,211][20347] Updated weights for policy 0, policy_version 568689 (0.0008) [2023-12-27 09:34:21,271][20347] Updated weights for policy 0, policy_version 568699 (0.0007) [2023-12-27 09:34:21,558][20381] Updated weights for policy 1, policy_version 568504 (0.0008) [2023-12-27 09:34:21,620][20381] Updated weights for policy 1, policy_version 568514 (0.0008) [2023-12-27 09:34:21,681][20381] Updated weights for policy 1, policy_version 568524 (0.0007) [2023-12-27 09:34:21,954][20347] Updated weights for policy 0, policy_version 568709 (0.0007) [2023-12-27 09:34:22,007][20347] Updated weights for policy 0, policy_version 568719 (0.0008) [2023-12-27 09:34:22,060][20347] Updated weights for policy 0, policy_version 568729 (0.0008) [2023-12-27 09:34:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 291176448. Throughput: 0: 10310.3, 1: 10246.1. Samples: 291148156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:22,363][19170] Avg episode reward: [(0, '648.920'), (1, '912.350')] [2023-12-27 09:34:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000568736_145612800.pth... [2023-12-27 09:34:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000567520_145301504.pth [2023-12-27 09:34:22,411][20381] Updated weights for policy 1, policy_version 568534 (0.0009) [2023-12-27 09:34:22,471][20381] Updated weights for policy 1, policy_version 568544 (0.0008) [2023-12-27 09:34:22,533][20381] Updated weights for policy 1, policy_version 568554 (0.0008) [2023-12-27 09:34:22,570][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000568560_145571840.pth... [2023-12-27 09:34:22,574][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000567344_145260544.pth [2023-12-27 09:34:22,820][20347] Updated weights for policy 0, policy_version 568739 (0.0008) [2023-12-27 09:34:22,880][20347] Updated weights for policy 0, policy_version 568749 (0.0007) [2023-12-27 09:34:22,933][20347] Updated weights for policy 0, policy_version 568759 (0.0009) [2023-12-27 09:34:22,977][20291] KL-divergence is very high: 107.8689 [2023-12-27 09:34:23,137][20381] Updated weights for policy 1, policy_version 568564 (0.0007) [2023-12-27 09:34:23,188][20381] Updated weights for policy 1, policy_version 568574 (0.0007) [2023-12-27 09:34:23,235][20381] Updated weights for policy 1, policy_version 568584 (0.0007) [2023-12-27 09:34:23,615][20347] Updated weights for policy 0, policy_version 568769 (0.0007) [2023-12-27 09:34:23,666][20347] Updated weights for policy 0, policy_version 568779 (0.0009) [2023-12-27 09:34:23,718][20347] Updated weights for policy 0, policy_version 568789 (0.0010) [2023-12-27 09:34:23,763][20347] Updated weights for policy 0, policy_version 568799 (0.0009) [2023-12-27 09:34:23,893][20381] Updated weights for policy 1, policy_version 568594 (0.0007) [2023-12-27 09:34:23,949][20381] Updated weights for policy 1, policy_version 568604 (0.0005) [2023-12-27 09:34:24,008][20381] Updated weights for policy 1, policy_version 568614 (0.0007) [2023-12-27 09:34:24,054][20381] Updated weights for policy 1, policy_version 568624 (0.0005) [2023-12-27 09:34:24,522][20347] Updated weights for policy 0, policy_version 568809 (0.0008) [2023-12-27 09:34:24,577][20347] Updated weights for policy 0, policy_version 568819 (0.0008) [2023-12-27 09:34:24,634][20347] Updated weights for policy 0, policy_version 568829 (0.0008) [2023-12-27 09:34:24,672][20381] Updated weights for policy 1, policy_version 568634 (0.0008) [2023-12-27 09:34:24,729][20381] Updated weights for policy 1, policy_version 568644 (0.0007) [2023-12-27 09:34:24,782][20381] Updated weights for policy 1, policy_version 568654 (0.0005) [2023-12-27 09:34:25,351][20347] Updated weights for policy 0, policy_version 568839 (0.0007) [2023-12-27 09:34:25,400][20347] Updated weights for policy 0, policy_version 568849 (0.0007) [2023-12-27 09:34:25,435][20381] Updated weights for policy 1, policy_version 568664 (0.0006) [2023-12-27 09:34:25,460][20347] Updated weights for policy 0, policy_version 568859 (0.0006) [2023-12-27 09:34:25,485][20381] Updated weights for policy 1, policy_version 568674 (0.0006) [2023-12-27 09:34:25,538][20381] Updated weights for policy 1, policy_version 568684 (0.0006) [2023-12-27 09:34:26,133][20347] Updated weights for policy 0, policy_version 568869 (0.0006) [2023-12-27 09:34:26,192][20347] Updated weights for policy 0, policy_version 568879 (0.0007) [2023-12-27 09:34:26,222][20381] Updated weights for policy 1, policy_version 568694 (0.0007) [2023-12-27 09:34:26,252][20347] Updated weights for policy 0, policy_version 568889 (0.0008) [2023-12-27 09:34:26,282][20381] Updated weights for policy 1, policy_version 568704 (0.0006) [2023-12-27 09:34:26,334][20381] Updated weights for policy 1, policy_version 568714 (0.0007) [2023-12-27 09:34:26,857][20347] Updated weights for policy 0, policy_version 568899 (0.0007) [2023-12-27 09:34:26,916][20347] Updated weights for policy 0, policy_version 568909 (0.0007) [2023-12-27 09:34:26,970][20347] Updated weights for policy 0, policy_version 568919 (0.0007) [2023-12-27 09:34:27,057][20381] Updated weights for policy 1, policy_version 568724 (0.0008) [2023-12-27 09:34:27,101][20381] Updated weights for policy 1, policy_version 568734 (0.0009) [2023-12-27 09:34:27,146][20381] Updated weights for policy 1, policy_version 568744 (0.0008) [2023-12-27 09:34:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 291282944. Throughput: 0: 10293.0, 1: 10275.3. Samples: 291269620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:27,363][19170] Avg episode reward: [(0, '706.660'), (1, '883.360')] [2023-12-27 09:34:27,618][20347] Updated weights for policy 0, policy_version 568929 (0.0006) [2023-12-27 09:34:27,666][20347] Updated weights for policy 0, policy_version 568939 (0.0007) [2023-12-27 09:34:27,720][20347] Updated weights for policy 0, policy_version 568949 (0.0007) [2023-12-27 09:34:27,766][20347] Updated weights for policy 0, policy_version 568959 (0.0008) [2023-12-27 09:34:27,834][20381] Updated weights for policy 1, policy_version 568754 (0.0007) [2023-12-27 09:34:27,892][20381] Updated weights for policy 1, policy_version 568764 (0.0008) [2023-12-27 09:34:27,944][20381] Updated weights for policy 1, policy_version 568774 (0.0008) [2023-12-27 09:34:28,000][20381] Updated weights for policy 1, policy_version 568784 (0.0007) [2023-12-27 09:34:28,406][20347] Updated weights for policy 0, policy_version 568969 (0.0008) [2023-12-27 09:34:28,461][20347] Updated weights for policy 0, policy_version 568979 (0.0008) [2023-12-27 09:34:28,519][20347] Updated weights for policy 0, policy_version 568989 (0.0007) [2023-12-27 09:34:28,658][20381] Updated weights for policy 1, policy_version 568794 (0.0008) [2023-12-27 09:34:28,714][20381] Updated weights for policy 1, policy_version 568804 (0.0007) [2023-12-27 09:34:28,774][20381] Updated weights for policy 1, policy_version 568814 (0.0008) [2023-12-27 09:34:29,185][20347] Updated weights for policy 0, policy_version 568999 (0.0007) [2023-12-27 09:34:29,245][20347] Updated weights for policy 0, policy_version 569009 (0.0007) [2023-12-27 09:34:29,300][20347] Updated weights for policy 0, policy_version 569019 (0.0008) [2023-12-27 09:34:29,483][20381] Updated weights for policy 1, policy_version 568824 (0.0008) [2023-12-27 09:34:29,538][20381] Updated weights for policy 1, policy_version 568834 (0.0007) [2023-12-27 09:34:29,590][20381] Updated weights for policy 1, policy_version 568844 (0.0007) [2023-12-27 09:34:30,027][20347] Updated weights for policy 0, policy_version 569029 (0.0008) [2023-12-27 09:34:30,077][20347] Updated weights for policy 0, policy_version 569039 (0.0009) [2023-12-27 09:34:30,126][20347] Updated weights for policy 0, policy_version 569049 (0.0009) [2023-12-27 09:34:30,320][20381] Updated weights for policy 1, policy_version 568854 (0.0007) [2023-12-27 09:34:30,373][20381] Updated weights for policy 1, policy_version 568864 (0.0009) [2023-12-27 09:34:30,430][20381] Updated weights for policy 1, policy_version 568874 (0.0008) [2023-12-27 09:34:30,849][20347] Updated weights for policy 0, policy_version 569059 (0.0008) [2023-12-27 09:34:30,900][20347] Updated weights for policy 0, policy_version 569069 (0.0007) [2023-12-27 09:34:30,952][20347] Updated weights for policy 0, policy_version 569079 (0.0007) [2023-12-27 09:34:31,126][20381] Updated weights for policy 1, policy_version 568884 (0.0008) [2023-12-27 09:34:31,186][20381] Updated weights for policy 1, policy_version 568894 (0.0007) [2023-12-27 09:34:31,235][20381] Updated weights for policy 1, policy_version 568904 (0.0007) [2023-12-27 09:34:31,689][20347] Updated weights for policy 0, policy_version 569089 (0.0007) [2023-12-27 09:34:31,755][20347] Updated weights for policy 0, policy_version 569099 (0.0008) [2023-12-27 09:34:31,800][20347] Updated weights for policy 0, policy_version 569109 (0.0009) [2023-12-27 09:34:31,848][20347] Updated weights for policy 0, policy_version 569119 (0.0008) [2023-12-27 09:34:31,909][20381] Updated weights for policy 1, policy_version 568914 (0.0008) [2023-12-27 09:34:31,970][20381] Updated weights for policy 1, policy_version 568924 (0.0005) [2023-12-27 09:34:32,033][20381] Updated weights for policy 1, policy_version 568934 (0.0006) [2023-12-27 09:34:32,085][20381] Updated weights for policy 1, policy_version 568944 (0.0008) [2023-12-27 09:34:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 291381248. Throughput: 0: 10255.6, 1: 10242.6. Samples: 291391308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:32,363][19170] Avg episode reward: [(0, '828.470'), (1, '912.580')] [2023-12-27 09:34:32,597][20347] Updated weights for policy 0, policy_version 569129 (0.0007) [2023-12-27 09:34:32,646][20347] Updated weights for policy 0, policy_version 569139 (0.0007) [2023-12-27 09:34:32,703][20347] Updated weights for policy 0, policy_version 569149 (0.0007) [2023-12-27 09:34:32,721][20381] Updated weights for policy 1, policy_version 568954 (0.0006) [2023-12-27 09:34:32,770][20381] Updated weights for policy 1, policy_version 568964 (0.0008) [2023-12-27 09:34:32,816][20381] Updated weights for policy 1, policy_version 568974 (0.0008) [2023-12-27 09:34:33,407][20347] Updated weights for policy 0, policy_version 569159 (0.0006) [2023-12-27 09:34:33,466][20347] Updated weights for policy 0, policy_version 569169 (0.0006) [2023-12-27 09:34:33,519][20347] Updated weights for policy 0, policy_version 569179 (0.0006) [2023-12-27 09:34:33,520][20381] Updated weights for policy 1, policy_version 568984 (0.0006) [2023-12-27 09:34:33,580][20381] Updated weights for policy 1, policy_version 568994 (0.0007) [2023-12-27 09:34:33,628][20381] Updated weights for policy 1, policy_version 569004 (0.0008) [2023-12-27 09:34:34,179][20347] Updated weights for policy 0, policy_version 569189 (0.0006) [2023-12-27 09:34:34,238][20347] Updated weights for policy 0, policy_version 569199 (0.0008) [2023-12-27 09:34:34,296][20347] Updated weights for policy 0, policy_version 569209 (0.0008) [2023-12-27 09:34:34,318][20381] Updated weights for policy 1, policy_version 569014 (0.0007) [2023-12-27 09:34:34,376][20381] Updated weights for policy 1, policy_version 569024 (0.0007) [2023-12-27 09:34:34,442][20381] Updated weights for policy 1, policy_version 569034 (0.0008) [2023-12-27 09:34:34,989][20347] Updated weights for policy 0, policy_version 569219 (0.0008) [2023-12-27 09:34:35,050][20347] Updated weights for policy 0, policy_version 569229 (0.0008) [2023-12-27 09:34:35,105][20347] Updated weights for policy 0, policy_version 569239 (0.0008) [2023-12-27 09:34:35,137][20381] Updated weights for policy 1, policy_version 569044 (0.0007) [2023-12-27 09:34:35,197][20381] Updated weights for policy 1, policy_version 569054 (0.0007) [2023-12-27 09:34:35,251][20381] Updated weights for policy 1, policy_version 569064 (0.0008) [2023-12-27 09:34:35,797][20347] Updated weights for policy 0, policy_version 569249 (0.0007) [2023-12-27 09:34:35,853][20347] Updated weights for policy 0, policy_version 569259 (0.0008) [2023-12-27 09:34:35,916][20347] Updated weights for policy 0, policy_version 569269 (0.0008) [2023-12-27 09:34:35,918][20381] Updated weights for policy 1, policy_version 569074 (0.0007) [2023-12-27 09:34:35,974][20347] Updated weights for policy 0, policy_version 569279 (0.0006) [2023-12-27 09:34:35,981][20381] Updated weights for policy 1, policy_version 569084 (0.0008) [2023-12-27 09:34:36,036][20381] Updated weights for policy 1, policy_version 569094 (0.0008) [2023-12-27 09:34:36,089][20381] Updated weights for policy 1, policy_version 569104 (0.0007) [2023-12-27 09:34:36,665][20347] Updated weights for policy 0, policy_version 569289 (0.0008) [2023-12-27 09:34:36,717][20347] Updated weights for policy 0, policy_version 569299 (0.0007) [2023-12-27 09:34:36,746][20381] Updated weights for policy 1, policy_version 569114 (0.0006) [2023-12-27 09:34:36,758][20291] KL-divergence is very high: 149.9136 [2023-12-27 09:34:36,768][20291] KL-divergence is very high: 196.8051 [2023-12-27 09:34:36,768][20347] Updated weights for policy 0, policy_version 569309 (0.0006) [2023-12-27 09:34:36,789][20381] Updated weights for policy 1, policy_version 569124 (0.0006) [2023-12-27 09:34:36,832][20381] Updated weights for policy 1, policy_version 569134 (0.0007) [2023-12-27 09:34:37,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20688.3). Total num frames: 291479552. Throughput: 0: 10246.7, 1: 10201.3. Samples: 291451900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:37,364][19170] Avg episode reward: [(0, '804.110'), (1, '961.170')] [2023-12-27 09:34:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000569136_145719296.pth... [2023-12-27 09:34:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000569312_145760256.pth... [2023-12-27 09:34:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000568128_145457152.pth [2023-12-27 09:34:37,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000567952_145416192.pth [2023-12-27 09:34:37,425][20347] Updated weights for policy 0, policy_version 569319 (0.0007) [2023-12-27 09:34:37,481][20347] Updated weights for policy 0, policy_version 569329 (0.0008) [2023-12-27 09:34:37,532][20347] Updated weights for policy 0, policy_version 569339 (0.0008) [2023-12-27 09:34:37,581][20381] Updated weights for policy 1, policy_version 569144 (0.0007) [2023-12-27 09:34:37,628][20381] Updated weights for policy 1, policy_version 569154 (0.0008) [2023-12-27 09:34:37,674][20381] Updated weights for policy 1, policy_version 569164 (0.0008) [2023-12-27 09:34:38,198][20347] Updated weights for policy 0, policy_version 569349 (0.0009) [2023-12-27 09:34:38,250][20347] Updated weights for policy 0, policy_version 569359 (0.0010) [2023-12-27 09:34:38,310][20347] Updated weights for policy 0, policy_version 569369 (0.0010) [2023-12-27 09:34:38,335][20381] Updated weights for policy 1, policy_version 569174 (0.0008) [2023-12-27 09:34:38,389][20381] Updated weights for policy 1, policy_version 569184 (0.0007) [2023-12-27 09:34:38,452][20381] Updated weights for policy 1, policy_version 569194 (0.0007) [2023-12-27 09:34:38,998][20347] Updated weights for policy 0, policy_version 569379 (0.0009) [2023-12-27 09:34:39,055][20347] Updated weights for policy 0, policy_version 569389 (0.0008) [2023-12-27 09:34:39,107][20347] Updated weights for policy 0, policy_version 569399 (0.0008) [2023-12-27 09:34:39,169][20381] Updated weights for policy 1, policy_version 569204 (0.0007) [2023-12-27 09:34:39,230][20381] Updated weights for policy 1, policy_version 569214 (0.0006) [2023-12-27 09:34:39,299][20381] Updated weights for policy 1, policy_version 569224 (0.0007) [2023-12-27 09:34:39,796][20347] Updated weights for policy 0, policy_version 569409 (0.0008) [2023-12-27 09:34:39,858][20347] Updated weights for policy 0, policy_version 569419 (0.0007) [2023-12-27 09:34:39,924][20347] Updated weights for policy 0, policy_version 569429 (0.0009) [2023-12-27 09:34:39,986][20347] Updated weights for policy 0, policy_version 569439 (0.0007) [2023-12-27 09:34:40,025][20381] Updated weights for policy 1, policy_version 569234 (0.0008) [2023-12-27 09:34:40,087][20381] Updated weights for policy 1, policy_version 569244 (0.0008) [2023-12-27 09:34:40,148][20381] Updated weights for policy 1, policy_version 569254 (0.0008) [2023-12-27 09:34:40,212][20381] Updated weights for policy 1, policy_version 569264 (0.0008) [2023-12-27 09:34:40,583][20347] Updated weights for policy 0, policy_version 569449 (0.0007) [2023-12-27 09:34:40,639][20347] Updated weights for policy 0, policy_version 569459 (0.0008) [2023-12-27 09:34:40,691][20347] Updated weights for policy 0, policy_version 569469 (0.0008) [2023-12-27 09:34:40,854][20381] Updated weights for policy 1, policy_version 569274 (0.0007) [2023-12-27 09:34:40,919][20381] Updated weights for policy 1, policy_version 569284 (0.0009) [2023-12-27 09:34:40,982][20381] Updated weights for policy 1, policy_version 569294 (0.0008) [2023-12-27 09:34:41,387][20347] Updated weights for policy 0, policy_version 569479 (0.0008) [2023-12-27 09:34:41,453][20347] Updated weights for policy 0, policy_version 569489 (0.0008) [2023-12-27 09:34:41,517][20347] Updated weights for policy 0, policy_version 569499 (0.0008) [2023-12-27 09:34:41,724][20381] Updated weights for policy 1, policy_version 569304 (0.0008) [2023-12-27 09:34:41,786][20381] Updated weights for policy 1, policy_version 569314 (0.0009) [2023-12-27 09:34:41,863][20381] Updated weights for policy 1, policy_version 569324 (0.0007) [2023-12-27 09:34:42,211][20347] Updated weights for policy 0, policy_version 569509 (0.0007) [2023-12-27 09:34:42,268][20347] Updated weights for policy 0, policy_version 569519 (0.0005) [2023-12-27 09:34:42,326][20347] Updated weights for policy 0, policy_version 569529 (0.0007) [2023-12-27 09:34:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 291577856. Throughput: 0: 10273.3, 1: 10139.0. Samples: 291574060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:42,363][19170] Avg episode reward: [(0, '772.240'), (1, '922.540')] [2023-12-27 09:34:42,623][20381] Updated weights for policy 1, policy_version 569334 (0.0006) [2023-12-27 09:34:42,687][20381] Updated weights for policy 1, policy_version 569344 (0.0007) [2023-12-27 09:34:42,752][20381] Updated weights for policy 1, policy_version 569354 (0.0006) [2023-12-27 09:34:42,952][20347] Updated weights for policy 0, policy_version 569539 (0.0008) [2023-12-27 09:34:43,001][20347] Updated weights for policy 0, policy_version 569549 (0.0010) [2023-12-27 09:34:43,046][20347] Updated weights for policy 0, policy_version 569559 (0.0010) [2023-12-27 09:34:43,436][20381] Updated weights for policy 1, policy_version 569364 (0.0008) [2023-12-27 09:34:43,492][20381] Updated weights for policy 1, policy_version 569375 (0.0008) [2023-12-27 09:34:43,541][20381] Updated weights for policy 1, policy_version 569385 (0.0009) [2023-12-27 09:34:43,703][20347] Updated weights for policy 0, policy_version 569569 (0.0010) [2023-12-27 09:34:43,751][20347] Updated weights for policy 0, policy_version 569579 (0.0005) [2023-12-27 09:34:43,800][20347] Updated weights for policy 0, policy_version 569589 (0.0005) [2023-12-27 09:34:43,859][20347] Updated weights for policy 0, policy_version 569599 (0.0005) [2023-12-27 09:34:44,316][20381] Updated weights for policy 1, policy_version 569395 (0.0007) [2023-12-27 09:34:44,377][20381] Updated weights for policy 1, policy_version 569405 (0.0008) [2023-12-27 09:34:44,434][20347] Updated weights for policy 0, policy_version 569609 (0.0010) [2023-12-27 09:34:44,439][20381] Updated weights for policy 1, policy_version 569415 (0.0005) [2023-12-27 09:34:44,489][20347] Updated weights for policy 0, policy_version 569619 (0.0007) [2023-12-27 09:34:44,546][20347] Updated weights for policy 0, policy_version 569629 (0.0005) [2023-12-27 09:34:45,159][20347] Updated weights for policy 0, policy_version 569639 (0.0006) [2023-12-27 09:34:45,180][20381] Updated weights for policy 1, policy_version 569425 (0.0007) [2023-12-27 09:34:45,212][20347] Updated weights for policy 0, policy_version 569649 (0.0007) [2023-12-27 09:34:45,233][20381] Updated weights for policy 1, policy_version 569435 (0.0006) [2023-12-27 09:34:45,258][20347] Updated weights for policy 0, policy_version 569659 (0.0005) [2023-12-27 09:34:45,286][20381] Updated weights for policy 1, policy_version 569445 (0.0005) [2023-12-27 09:34:45,342][20381] Updated weights for policy 1, policy_version 569455 (0.0008) [2023-12-27 09:34:45,891][20347] Updated weights for policy 0, policy_version 569669 (0.0007) [2023-12-27 09:34:45,937][20347] Updated weights for policy 0, policy_version 569679 (0.0008) [2023-12-27 09:34:45,990][20347] Updated weights for policy 0, policy_version 569689 (0.0007) [2023-12-27 09:34:46,070][20381] Updated weights for policy 1, policy_version 569465 (0.0008) [2023-12-27 09:34:46,116][20381] Updated weights for policy 1, policy_version 569475 (0.0008) [2023-12-27 09:34:46,164][20381] Updated weights for policy 1, policy_version 569485 (0.0008) [2023-12-27 09:34:46,712][20347] Updated weights for policy 0, policy_version 569699 (0.0007) [2023-12-27 09:34:46,772][20347] Updated weights for policy 0, policy_version 569709 (0.0005) [2023-12-27 09:34:46,785][20381] Updated weights for policy 1, policy_version 569495 (0.0006) [2023-12-27 09:34:46,825][20347] Updated weights for policy 0, policy_version 569719 (0.0007) [2023-12-27 09:34:46,850][20381] Updated weights for policy 1, policy_version 569505 (0.0006) [2023-12-27 09:34:46,910][20381] Updated weights for policy 1, policy_version 569515 (0.0007) [2023-12-27 09:34:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 291684352. Throughput: 0: 10259.6, 1: 10069.5. Samples: 291696896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:47,363][19170] Avg episode reward: [(0, '661.420'), (1, '864.270')] [2023-12-27 09:34:47,485][20347] Updated weights for policy 0, policy_version 569729 (0.0007) [2023-12-27 09:34:47,500][20381] Updated weights for policy 1, policy_version 569525 (0.0007) [2023-12-27 09:34:47,540][20347] Updated weights for policy 0, policy_version 569739 (0.0007) [2023-12-27 09:34:47,555][20381] Updated weights for policy 1, policy_version 569535 (0.0005) [2023-12-27 09:34:47,596][20347] Updated weights for policy 0, policy_version 569749 (0.0007) [2023-12-27 09:34:47,614][20381] Updated weights for policy 1, policy_version 569545 (0.0005) [2023-12-27 09:34:47,655][20347] Updated weights for policy 0, policy_version 569759 (0.0008) [2023-12-27 09:34:48,150][20381] Updated weights for policy 1, policy_version 569555 (0.0006) [2023-12-27 09:34:48,214][20381] Updated weights for policy 1, policy_version 569565 (0.0008) [2023-12-27 09:34:48,269][20381] Updated weights for policy 1, policy_version 569575 (0.0007) [2023-12-27 09:34:48,298][20347] Updated weights for policy 0, policy_version 569769 (0.0005) [2023-12-27 09:34:48,362][20347] Updated weights for policy 0, policy_version 569779 (0.0007) [2023-12-27 09:34:48,419][20347] Updated weights for policy 0, policy_version 569789 (0.0008) [2023-12-27 09:34:48,917][20381] Updated weights for policy 1, policy_version 569585 (0.0006) [2023-12-27 09:34:48,975][20381] Updated weights for policy 1, policy_version 569595 (0.0008) [2023-12-27 09:34:49,031][20381] Updated weights for policy 1, policy_version 569605 (0.0005) [2023-12-27 09:34:49,081][20381] Updated weights for policy 1, policy_version 569615 (0.0005) [2023-12-27 09:34:49,136][20347] Updated weights for policy 0, policy_version 569799 (0.0007) [2023-12-27 09:34:49,196][20347] Updated weights for policy 0, policy_version 569809 (0.0008) [2023-12-27 09:34:49,260][20347] Updated weights for policy 0, policy_version 569819 (0.0009) [2023-12-27 09:34:49,678][20381] Updated weights for policy 1, policy_version 569625 (0.0009) [2023-12-27 09:34:49,724][20381] Updated weights for policy 1, policy_version 569635 (0.0006) [2023-12-27 09:34:49,772][20381] Updated weights for policy 1, policy_version 569645 (0.0007) [2023-12-27 09:34:49,862][20347] Updated weights for policy 0, policy_version 569829 (0.0008) [2023-12-27 09:34:49,913][20347] Updated weights for policy 0, policy_version 569839 (0.0008) [2023-12-27 09:34:49,970][20347] Updated weights for policy 0, policy_version 569849 (0.0006) [2023-12-27 09:34:50,475][20381] Updated weights for policy 1, policy_version 569655 (0.0007) [2023-12-27 09:34:50,534][20381] Updated weights for policy 1, policy_version 569665 (0.0008) [2023-12-27 09:34:50,594][20381] Updated weights for policy 1, policy_version 569675 (0.0008) [2023-12-27 09:34:50,650][20347] Updated weights for policy 0, policy_version 569859 (0.0007) [2023-12-27 09:34:50,713][20347] Updated weights for policy 0, policy_version 569869 (0.0007) [2023-12-27 09:34:50,765][20347] Updated weights for policy 0, policy_version 569879 (0.0007) [2023-12-27 09:34:51,276][20381] Updated weights for policy 1, policy_version 569685 (0.0008) [2023-12-27 09:34:51,322][20381] Updated weights for policy 1, policy_version 569695 (0.0008) [2023-12-27 09:34:51,392][20381] Updated weights for policy 1, policy_version 569705 (0.0007) [2023-12-27 09:34:51,448][20347] Updated weights for policy 0, policy_version 569889 (0.0008) [2023-12-27 09:34:51,504][20347] Updated weights for policy 0, policy_version 569899 (0.0006) [2023-12-27 09:34:51,564][20347] Updated weights for policy 0, policy_version 569909 (0.0007) [2023-12-27 09:34:51,630][20347] Updated weights for policy 0, policy_version 569919 (0.0008) [2023-12-27 09:34:52,124][20381] Updated weights for policy 1, policy_version 569715 (0.0008) [2023-12-27 09:34:52,170][20381] Updated weights for policy 1, policy_version 569725 (0.0007) [2023-12-27 09:34:52,219][20347] Updated weights for policy 0, policy_version 569929 (0.0009) [2023-12-27 09:34:52,222][20381] Updated weights for policy 1, policy_version 569735 (0.0007) [2023-12-27 09:34:52,273][20347] Updated weights for policy 0, policy_version 569939 (0.0008) [2023-12-27 09:34:52,332][20347] Updated weights for policy 0, policy_version 569949 (0.0008) [2023-12-27 09:34:52,363][19170] Fps is (10 sec: 22118.0, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 291799040. Throughput: 0: 10229.6, 1: 10115.3. Samples: 291761012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:52,363][19170] Avg episode reward: [(0, '764.730'), (1, '893.290')] [2023-12-27 09:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000569744_145874944.pth... [2023-12-27 09:34:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000569952_145924096.pth... [2023-12-27 09:34:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000568560_145571840.pth [2023-12-27 09:34:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000568736_145612800.pth [2023-12-27 09:34:52,930][20381] Updated weights for policy 1, policy_version 569745 (0.0007) [2023-12-27 09:34:52,996][20381] Updated weights for policy 1, policy_version 569755 (0.0008) [2023-12-27 09:34:53,047][20381] Updated weights for policy 1, policy_version 569765 (0.0007) [2023-12-27 09:34:53,070][20347] Updated weights for policy 0, policy_version 569959 (0.0008) [2023-12-27 09:34:53,093][20381] Updated weights for policy 1, policy_version 569775 (0.0007) [2023-12-27 09:34:53,125][20347] Updated weights for policy 0, policy_version 569969 (0.0007) [2023-12-27 09:34:53,173][20347] Updated weights for policy 0, policy_version 569979 (0.0007) [2023-12-27 09:34:53,717][20381] Updated weights for policy 1, policy_version 569785 (0.0008) [2023-12-27 09:34:53,762][20381] Updated weights for policy 1, policy_version 569795 (0.0007) [2023-12-27 09:34:53,820][20381] Updated weights for policy 1, policy_version 569805 (0.0008) [2023-12-27 09:34:53,892][20347] Updated weights for policy 0, policy_version 569989 (0.0008) [2023-12-27 09:34:53,936][20347] Updated weights for policy 0, policy_version 569999 (0.0007) [2023-12-27 09:34:53,996][20347] Updated weights for policy 0, policy_version 570009 (0.0008) [2023-12-27 09:34:54,576][20381] Updated weights for policy 1, policy_version 569815 (0.0008) [2023-12-27 09:34:54,578][20347] Updated weights for policy 0, policy_version 570019 (0.0007) [2023-12-27 09:34:54,626][20347] Updated weights for policy 0, policy_version 570029 (0.0005) [2023-12-27 09:34:54,635][20381] Updated weights for policy 1, policy_version 569825 (0.0007) [2023-12-27 09:34:54,677][20347] Updated weights for policy 0, policy_version 570039 (0.0005) [2023-12-27 09:34:54,694][20381] Updated weights for policy 1, policy_version 569835 (0.0006) [2023-12-27 09:34:55,286][20347] Updated weights for policy 0, policy_version 570049 (0.0006) [2023-12-27 09:34:55,320][20381] Updated weights for policy 1, policy_version 569845 (0.0006) [2023-12-27 09:34:55,344][20347] Updated weights for policy 0, policy_version 570059 (0.0010) [2023-12-27 09:34:55,372][20381] Updated weights for policy 1, policy_version 569855 (0.0005) [2023-12-27 09:34:55,399][20347] Updated weights for policy 0, policy_version 570069 (0.0010) [2023-12-27 09:34:55,427][20381] Updated weights for policy 1, policy_version 569865 (0.0006) [2023-12-27 09:34:55,450][20347] Updated weights for policy 0, policy_version 570079 (0.0010) [2023-12-27 09:34:56,127][20347] Updated weights for policy 0, policy_version 570089 (0.0010) [2023-12-27 09:34:56,144][20381] Updated weights for policy 1, policy_version 569875 (0.0008) [2023-12-27 09:34:56,181][20347] Updated weights for policy 0, policy_version 570099 (0.0008) [2023-12-27 09:34:56,193][20381] Updated weights for policy 1, policy_version 569885 (0.0007) [2023-12-27 09:34:56,232][20347] Updated weights for policy 0, policy_version 570109 (0.0005) [2023-12-27 09:34:56,249][20381] Updated weights for policy 1, policy_version 569895 (0.0007) [2023-12-27 09:34:56,850][20347] Updated weights for policy 0, policy_version 570119 (0.0005) [2023-12-27 09:34:56,908][20347] Updated weights for policy 0, policy_version 570129 (0.0005) [2023-12-27 09:34:56,958][20347] Updated weights for policy 0, policy_version 570139 (0.0007) [2023-12-27 09:34:56,994][20381] Updated weights for policy 1, policy_version 569905 (0.0008) [2023-12-27 09:34:57,049][20381] Updated weights for policy 1, policy_version 569915 (0.0008) [2023-12-27 09:34:57,095][20381] Updated weights for policy 1, policy_version 569925 (0.0008) [2023-12-27 09:34:57,148][20381] Updated weights for policy 1, policy_version 569935 (0.0008) [2023-12-27 09:34:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 291897344. Throughput: 0: 10306.9, 1: 10143.0. Samples: 291885108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:34:57,364][19170] Avg episode reward: [(0, '848.680'), (1, '961.190')] [2023-12-27 09:34:57,585][20347] Updated weights for policy 0, policy_version 570149 (0.0009) [2023-12-27 09:34:57,636][20347] Updated weights for policy 0, policy_version 570159 (0.0010) [2023-12-27 09:34:57,693][20347] Updated weights for policy 0, policy_version 570169 (0.0010) [2023-12-27 09:34:57,882][20381] Updated weights for policy 1, policy_version 569945 (0.0008) [2023-12-27 09:34:57,936][20381] Updated weights for policy 1, policy_version 569955 (0.0007) [2023-12-27 09:34:57,983][20381] Updated weights for policy 1, policy_version 569965 (0.0007) [2023-12-27 09:34:58,387][20347] Updated weights for policy 0, policy_version 570179 (0.0009) [2023-12-27 09:34:58,445][20347] Updated weights for policy 0, policy_version 570189 (0.0007) [2023-12-27 09:34:58,509][20347] Updated weights for policy 0, policy_version 570199 (0.0008) [2023-12-27 09:34:58,742][20381] Updated weights for policy 1, policy_version 569975 (0.0007) [2023-12-27 09:34:58,805][20381] Updated weights for policy 1, policy_version 569985 (0.0007) [2023-12-27 09:34:58,871][20381] Updated weights for policy 1, policy_version 569995 (0.0008) [2023-12-27 09:34:59,188][20347] Updated weights for policy 0, policy_version 570209 (0.0010) [2023-12-27 09:34:59,253][20347] Updated weights for policy 0, policy_version 570219 (0.0009) [2023-12-27 09:34:59,314][20347] Updated weights for policy 0, policy_version 570229 (0.0006) [2023-12-27 09:34:59,381][20347] Updated weights for policy 0, policy_version 570239 (0.0008) [2023-12-27 09:34:59,633][20381] Updated weights for policy 1, policy_version 570005 (0.0008) [2023-12-27 09:34:59,690][20381] Updated weights for policy 1, policy_version 570015 (0.0008) [2023-12-27 09:34:59,745][20381] Updated weights for policy 1, policy_version 570025 (0.0007) [2023-12-27 09:35:00,039][20347] Updated weights for policy 0, policy_version 570249 (0.0008) [2023-12-27 09:35:00,089][20347] Updated weights for policy 0, policy_version 570259 (0.0005) [2023-12-27 09:35:00,142][20347] Updated weights for policy 0, policy_version 570269 (0.0008) [2023-12-27 09:35:00,446][20381] Updated weights for policy 1, policy_version 570035 (0.0007) [2023-12-27 09:35:00,505][20381] Updated weights for policy 1, policy_version 570045 (0.0007) [2023-12-27 09:35:00,510][20292] KL-divergence is very high: 109.5784 [2023-12-27 09:35:00,550][20292] KL-divergence is very high: 257.7777 [2023-12-27 09:35:00,555][20381] Updated weights for policy 1, policy_version 570055 (0.0007) [2023-12-27 09:35:00,825][20347] Updated weights for policy 0, policy_version 570279 (0.0007) [2023-12-27 09:35:00,880][20347] Updated weights for policy 0, policy_version 570289 (0.0008) [2023-12-27 09:35:00,928][20347] Updated weights for policy 0, policy_version 570299 (0.0007) [2023-12-27 09:35:01,267][20381] Updated weights for policy 1, policy_version 570065 (0.0007) [2023-12-27 09:35:01,320][20381] Updated weights for policy 1, policy_version 570075 (0.0007) [2023-12-27 09:35:01,379][20381] Updated weights for policy 1, policy_version 570085 (0.0008) [2023-12-27 09:35:01,433][20381] Updated weights for policy 1, policy_version 570095 (0.0007) [2023-12-27 09:35:01,685][20347] Updated weights for policy 0, policy_version 570309 (0.0007) [2023-12-27 09:35:01,741][20347] Updated weights for policy 0, policy_version 570319 (0.0007) [2023-12-27 09:35:01,796][20347] Updated weights for policy 0, policy_version 570329 (0.0007) [2023-12-27 09:35:02,159][20381] Updated weights for policy 1, policy_version 570105 (0.0008) [2023-12-27 09:35:02,212][20381] Updated weights for policy 1, policy_version 570115 (0.0007) [2023-12-27 09:35:02,269][20381] Updated weights for policy 1, policy_version 570125 (0.0008) [2023-12-27 09:35:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 291995648. Throughput: 0: 10294.1, 1: 10065.5. Samples: 292003316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:35:02,363][19170] Avg episode reward: [(0, '839.720'), (1, '912.610')] [2023-12-27 09:35:02,429][20347] Updated weights for policy 0, policy_version 570339 (0.0006) [2023-12-27 09:35:02,488][20347] Updated weights for policy 0, policy_version 570349 (0.0007) [2023-12-27 09:35:02,542][20347] Updated weights for policy 0, policy_version 570359 (0.0008) [2023-12-27 09:35:03,008][20381] Updated weights for policy 1, policy_version 570135 (0.0007) [2023-12-27 09:35:03,064][20381] Updated weights for policy 1, policy_version 570145 (0.0008) [2023-12-27 09:35:03,126][20347] Updated weights for policy 0, policy_version 570369 (0.0007) [2023-12-27 09:35:03,126][20381] Updated weights for policy 1, policy_version 570155 (0.0008) [2023-12-27 09:35:03,177][20347] Updated weights for policy 0, policy_version 570379 (0.0006) [2023-12-27 09:35:03,226][20347] Updated weights for policy 0, policy_version 570389 (0.0008) [2023-12-27 09:35:03,270][20347] Updated weights for policy 0, policy_version 570399 (0.0007) [2023-12-27 09:35:03,833][20381] Updated weights for policy 1, policy_version 570165 (0.0007) [2023-12-27 09:35:03,889][20381] Updated weights for policy 1, policy_version 570175 (0.0007) [2023-12-27 09:35:03,937][20381] Updated weights for policy 1, policy_version 570185 (0.0008) [2023-12-27 09:35:03,964][20347] Updated weights for policy 0, policy_version 570409 (0.0009) [2023-12-27 09:35:04,020][20347] Updated weights for policy 0, policy_version 570419 (0.0008) [2023-12-27 09:35:04,072][20347] Updated weights for policy 0, policy_version 570429 (0.0008) [2023-12-27 09:35:04,672][20381] Updated weights for policy 1, policy_version 570195 (0.0008) [2023-12-27 09:35:04,732][20381] Updated weights for policy 1, policy_version 570205 (0.0006) [2023-12-27 09:35:04,759][20347] Updated weights for policy 0, policy_version 570439 (0.0008) [2023-12-27 09:35:04,790][20381] Updated weights for policy 1, policy_version 570215 (0.0006) [2023-12-27 09:35:04,818][20347] Updated weights for policy 0, policy_version 570449 (0.0006) [2023-12-27 09:35:04,869][20347] Updated weights for policy 0, policy_version 570459 (0.0006) [2023-12-27 09:35:05,464][20381] Updated weights for policy 1, policy_version 570225 (0.0006) [2023-12-27 09:35:05,509][20381] Updated weights for policy 1, policy_version 570235 (0.0007) [2023-12-27 09:35:05,556][20381] Updated weights for policy 1, policy_version 570245 (0.0006) [2023-12-27 09:35:05,558][20347] Updated weights for policy 0, policy_version 570469 (0.0006) [2023-12-27 09:35:05,603][20381] Updated weights for policy 1, policy_version 570255 (0.0006) [2023-12-27 09:35:05,616][20347] Updated weights for policy 0, policy_version 570479 (0.0006) [2023-12-27 09:35:05,668][20347] Updated weights for policy 0, policy_version 570489 (0.0008) [2023-12-27 09:35:06,344][20381] Updated weights for policy 1, policy_version 570265 (0.0007) [2023-12-27 09:35:06,349][20347] Updated weights for policy 0, policy_version 570499 (0.0008) [2023-12-27 09:35:06,396][20381] Updated weights for policy 1, policy_version 570275 (0.0007) [2023-12-27 09:35:06,408][20347] Updated weights for policy 0, policy_version 570509 (0.0006) [2023-12-27 09:35:06,448][20381] Updated weights for policy 1, policy_version 570285 (0.0007) [2023-12-27 09:35:06,461][20347] Updated weights for policy 0, policy_version 570519 (0.0006) [2023-12-27 09:35:07,110][20347] Updated weights for policy 0, policy_version 570529 (0.0007) [2023-12-27 09:35:07,166][20347] Updated weights for policy 0, policy_version 570539 (0.0005) [2023-12-27 09:35:07,192][20381] Updated weights for policy 1, policy_version 570295 (0.0007) [2023-12-27 09:35:07,219][20347] Updated weights for policy 0, policy_version 570549 (0.0005) [2023-12-27 09:35:07,242][20381] Updated weights for policy 1, policy_version 570305 (0.0007) [2023-12-27 09:35:07,280][20347] Updated weights for policy 0, policy_version 570559 (0.0007) [2023-12-27 09:35:07,294][20381] Updated weights for policy 1, policy_version 570315 (0.0006) [2023-12-27 09:35:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20660.5). Total num frames: 292102144. Throughput: 0: 10303.9, 1: 10062.1. Samples: 292064628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:35:07,364][19170] Avg episode reward: [(0, '634.210'), (1, '922.290')] [2023-12-27 09:35:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000570320_146022400.pth... [2023-12-27 09:35:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000570560_146079744.pth... [2023-12-27 09:35:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000569136_145719296.pth [2023-12-27 09:35:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000569312_145760256.pth [2023-12-27 09:35:07,911][20381] Updated weights for policy 1, policy_version 570325 (0.0008) [2023-12-27 09:35:07,958][20347] Updated weights for policy 0, policy_version 570569 (0.0006) [2023-12-27 09:35:07,968][20381] Updated weights for policy 1, policy_version 570335 (0.0006) [2023-12-27 09:35:08,011][20347] Updated weights for policy 0, policy_version 570579 (0.0007) [2023-12-27 09:35:08,017][20381] Updated weights for policy 1, policy_version 570345 (0.0006) [2023-12-27 09:35:08,068][20347] Updated weights for policy 0, policy_version 570589 (0.0006) [2023-12-27 09:35:08,743][20381] Updated weights for policy 1, policy_version 570355 (0.0007) [2023-12-27 09:35:08,781][20292] KL-divergence is very high: 108.0964 [2023-12-27 09:35:08,787][20347] Updated weights for policy 0, policy_version 570599 (0.0006) [2023-12-27 09:35:08,804][20381] Updated weights for policy 1, policy_version 570365 (0.0006) [2023-12-27 09:35:08,840][20347] Updated weights for policy 0, policy_version 570609 (0.0005) [2023-12-27 09:35:08,861][20381] Updated weights for policy 1, policy_version 570375 (0.0005) [2023-12-27 09:35:08,896][20347] Updated weights for policy 0, policy_version 570619 (0.0005) [2023-12-27 09:35:09,588][20381] Updated weights for policy 1, policy_version 570385 (0.0006) [2023-12-27 09:35:09,601][20347] Updated weights for policy 0, policy_version 570629 (0.0006) [2023-12-27 09:35:09,645][20381] Updated weights for policy 1, policy_version 570395 (0.0006) [2023-12-27 09:35:09,655][20347] Updated weights for policy 0, policy_version 570639 (0.0005) [2023-12-27 09:35:09,706][20381] Updated weights for policy 1, policy_version 570405 (0.0005) [2023-12-27 09:35:09,713][20347] Updated weights for policy 0, policy_version 570649 (0.0007) [2023-12-27 09:35:09,766][20381] Updated weights for policy 1, policy_version 570415 (0.0007) [2023-12-27 09:35:10,355][20381] Updated weights for policy 1, policy_version 570425 (0.0006) [2023-12-27 09:35:10,418][20381] Updated weights for policy 1, policy_version 570435 (0.0006) [2023-12-27 09:35:10,479][20381] Updated weights for policy 1, policy_version 570445 (0.0006) [2023-12-27 09:35:10,503][20347] Updated weights for policy 0, policy_version 570659 (0.0008) [2023-12-27 09:35:10,567][20347] Updated weights for policy 0, policy_version 570669 (0.0008) [2023-12-27 09:35:10,628][20347] Updated weights for policy 0, policy_version 570679 (0.0008) [2023-12-27 09:35:11,085][20381] Updated weights for policy 1, policy_version 570455 (0.0007) [2023-12-27 09:35:11,148][20381] Updated weights for policy 1, policy_version 570465 (0.0008) [2023-12-27 09:35:11,198][20381] Updated weights for policy 1, policy_version 570475 (0.0008) [2023-12-27 09:35:11,371][20347] Updated weights for policy 0, policy_version 570689 (0.0008) [2023-12-27 09:35:11,435][20347] Updated weights for policy 0, policy_version 570699 (0.0008) [2023-12-27 09:35:11,495][20347] Updated weights for policy 0, policy_version 570709 (0.0008) [2023-12-27 09:35:11,546][20347] Updated weights for policy 0, policy_version 570719 (0.0007) [2023-12-27 09:35:11,953][20381] Updated weights for policy 1, policy_version 570485 (0.0006) [2023-12-27 09:35:12,005][20381] Updated weights for policy 1, policy_version 570495 (0.0007) [2023-12-27 09:35:12,058][20381] Updated weights for policy 1, policy_version 570505 (0.0007) [2023-12-27 09:35:12,268][20347] Updated weights for policy 0, policy_version 570729 (0.0009) [2023-12-27 09:35:12,329][20347] Updated weights for policy 0, policy_version 570739 (0.0008) [2023-12-27 09:35:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 292192256. Throughput: 0: 10315.7, 1: 10036.6. Samples: 292185472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:35:12,363][19170] Avg episode reward: [(0, '423.090'), (1, '980.730')] [2023-12-27 09:35:12,395][20347] Updated weights for policy 0, policy_version 570749 (0.0008) [2023-12-27 09:35:12,797][20381] Updated weights for policy 1, policy_version 570515 (0.0008) [2023-12-27 09:35:12,852][20381] Updated weights for policy 1, policy_version 570525 (0.0007) [2023-12-27 09:35:12,908][20381] Updated weights for policy 1, policy_version 570535 (0.0007) [2023-12-27 09:35:13,030][20347] Updated weights for policy 0, policy_version 570759 (0.0008) [2023-12-27 09:35:13,077][20347] Updated weights for policy 0, policy_version 570769 (0.0006) [2023-12-27 09:35:13,128][20347] Updated weights for policy 0, policy_version 570779 (0.0005) [2023-12-27 09:35:13,609][20381] Updated weights for policy 1, policy_version 570545 (0.0008) [2023-12-27 09:35:13,657][20381] Updated weights for policy 1, policy_version 570555 (0.0010) [2023-12-27 09:35:13,704][20381] Updated weights for policy 1, policy_version 570565 (0.0010) [2023-12-27 09:35:13,744][20347] Updated weights for policy 0, policy_version 570789 (0.0005) [2023-12-27 09:35:13,756][20381] Updated weights for policy 1, policy_version 570575 (0.0010) [2023-12-27 09:35:13,789][20347] Updated weights for policy 0, policy_version 570799 (0.0006) [2023-12-27 09:35:13,838][20347] Updated weights for policy 0, policy_version 570809 (0.0005) [2023-12-27 09:35:14,447][20381] Updated weights for policy 1, policy_version 570585 (0.0011) [2023-12-27 09:35:14,461][20347] Updated weights for policy 0, policy_version 570819 (0.0006) [2023-12-27 09:35:14,509][20381] Updated weights for policy 1, policy_version 570595 (0.0011) [2023-12-27 09:35:14,519][20347] Updated weights for policy 0, policy_version 570829 (0.0009) [2023-12-27 09:35:14,565][20381] Updated weights for policy 1, policy_version 570605 (0.0011) [2023-12-27 09:35:14,577][20347] Updated weights for policy 0, policy_version 570839 (0.0006) [2023-12-27 09:35:15,220][20347] Updated weights for policy 0, policy_version 570849 (0.0006) [2023-12-27 09:35:15,234][20381] Updated weights for policy 1, policy_version 570615 (0.0011) [2023-12-27 09:35:15,286][20347] Updated weights for policy 0, policy_version 570859 (0.0008) [2023-12-27 09:35:15,294][20381] Updated weights for policy 1, policy_version 570625 (0.0011) [2023-12-27 09:35:15,336][20347] Updated weights for policy 0, policy_version 570869 (0.0009) [2023-12-27 09:35:15,350][20381] Updated weights for policy 1, policy_version 570635 (0.0011) [2023-12-27 09:35:15,388][20347] Updated weights for policy 0, policy_version 570879 (0.0006) [2023-12-27 09:35:16,031][20347] Updated weights for policy 0, policy_version 570889 (0.0006) [2023-12-27 09:35:16,037][20381] Updated weights for policy 1, policy_version 570645 (0.0011) [2023-12-27 09:35:16,085][20347] Updated weights for policy 0, policy_version 570899 (0.0005) [2023-12-27 09:35:16,088][20381] Updated weights for policy 1, policy_version 570655 (0.0011) [2023-12-27 09:35:16,139][20347] Updated weights for policy 0, policy_version 570909 (0.0006) [2023-12-27 09:35:16,150][20381] Updated weights for policy 1, policy_version 570665 (0.0011) [2023-12-27 09:35:16,698][20347] Updated weights for policy 0, policy_version 570919 (0.0007) [2023-12-27 09:35:16,757][20347] Updated weights for policy 0, policy_version 570929 (0.0007) [2023-12-27 09:35:16,822][20347] Updated weights for policy 0, policy_version 570939 (0.0007) [2023-12-27 09:35:16,856][20381] Updated weights for policy 1, policy_version 570675 (0.0011) [2023-12-27 09:35:16,920][20381] Updated weights for policy 1, policy_version 570685 (0.0010) [2023-12-27 09:35:16,967][20381] Updated weights for policy 1, policy_version 570695 (0.0010) [2023-12-27 09:35:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.4, 300 sec: 20660.5). Total num frames: 292298752. Throughput: 0: 10376.5, 1: 10020.0. Samples: 292309152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:35:17,363][19170] Avg episode reward: [(0, '483.830'), (1, '971.740')] [2023-12-27 09:35:17,504][20347] Updated weights for policy 0, policy_version 570949 (0.0009) [2023-12-27 09:35:17,563][20347] Updated weights for policy 0, policy_version 570959 (0.0011) [2023-12-27 09:35:17,616][20347] Updated weights for policy 0, policy_version 570969 (0.0009) [2023-12-27 09:35:17,650][20381] Updated weights for policy 1, policy_version 570705 (0.0010) [2023-12-27 09:35:17,701][20381] Updated weights for policy 1, policy_version 570715 (0.0010) [2023-12-27 09:35:17,750][20381] Updated weights for policy 1, policy_version 570725 (0.0010) [2023-12-27 09:35:17,795][20381] Updated weights for policy 1, policy_version 570735 (0.0010) [2023-12-27 09:35:18,249][20347] Updated weights for policy 0, policy_version 570979 (0.0008) [2023-12-27 09:35:18,313][20347] Updated weights for policy 0, policy_version 570989 (0.0010) [2023-12-27 09:35:18,378][20347] Updated weights for policy 0, policy_version 570999 (0.0008) [2023-12-27 09:35:18,487][20381] Updated weights for policy 1, policy_version 570745 (0.0009) [2023-12-27 09:35:18,545][20381] Updated weights for policy 1, policy_version 570755 (0.0008) [2023-12-27 09:35:18,600][20381] Updated weights for policy 1, policy_version 570765 (0.0008) [2023-12-27 09:35:18,999][20347] Updated weights for policy 0, policy_version 571009 (0.0010) [2023-12-27 09:35:19,059][20347] Updated weights for policy 0, policy_version 571019 (0.0008) [2023-12-27 09:35:19,106][20347] Updated weights for policy 0, policy_version 571029 (0.0010) [2023-12-27 09:35:19,154][20347] Updated weights for policy 0, policy_version 571039 (0.0010) [2023-12-27 09:35:19,314][20381] Updated weights for policy 1, policy_version 570775 (0.0008) [2023-12-27 09:35:19,380][20381] Updated weights for policy 1, policy_version 570785 (0.0007) [2023-12-27 09:35:19,435][20381] Updated weights for policy 1, policy_version 570795 (0.0008) [2023-12-27 09:35:19,900][20347] Updated weights for policy 0, policy_version 571049 (0.0008) [2023-12-27 09:35:19,962][20347] Updated weights for policy 0, policy_version 571059 (0.0007) [2023-12-27 09:35:20,020][20347] Updated weights for policy 0, policy_version 571069 (0.0006) [2023-12-27 09:35:20,074][20381] Updated weights for policy 1, policy_version 570805 (0.0008) [2023-12-27 09:35:20,119][20381] Updated weights for policy 1, policy_version 570815 (0.0008) [2023-12-27 09:35:20,169][20381] Updated weights for policy 1, policy_version 570825 (0.0008) [2023-12-27 09:35:20,641][20347] Updated weights for policy 0, policy_version 571079 (0.0008) [2023-12-27 09:35:20,707][20347] Updated weights for policy 0, policy_version 571089 (0.0008) [2023-12-27 09:35:20,763][20347] Updated weights for policy 0, policy_version 571099 (0.0007) [2023-12-27 09:35:20,914][20381] Updated weights for policy 1, policy_version 570835 (0.0008) [2023-12-27 09:35:20,982][20381] Updated weights for policy 1, policy_version 570845 (0.0007) [2023-12-27 09:35:21,039][20381] Updated weights for policy 1, policy_version 570855 (0.0008) [2023-12-27 09:35:21,537][20347] Updated weights for policy 0, policy_version 571109 (0.0008) [2023-12-27 09:35:21,601][20347] Updated weights for policy 0, policy_version 571119 (0.0008) [2023-12-27 09:35:21,662][20347] Updated weights for policy 0, policy_version 571129 (0.0008) [2023-12-27 09:35:21,832][20381] Updated weights for policy 1, policy_version 570865 (0.0008) [2023-12-27 09:35:21,896][20381] Updated weights for policy 1, policy_version 570875 (0.0009) [2023-12-27 09:35:21,957][20381] Updated weights for policy 1, policy_version 570885 (0.0009) [2023-12-27 09:35:22,015][20381] Updated weights for policy 1, policy_version 570895 (0.0008) [2023-12-27 09:35:22,302][20347] Updated weights for policy 0, policy_version 571139 (0.0008) [2023-12-27 09:35:22,356][20347] Updated weights for policy 0, policy_version 571149 (0.0007) [2023-12-27 09:35:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 292397056. Throughput: 0: 10391.7, 1: 10012.3. Samples: 292370076. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:22,363][19170] Avg episode reward: [(0, '443.850'), (1, '991.010')] [2023-12-27 09:35:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000570896_146169856.pth... [2023-12-27 09:35:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000569744_145874944.pth [2023-12-27 09:35:22,415][20347] Updated weights for policy 0, policy_version 571159 (0.0008) [2023-12-27 09:35:22,462][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000571168_146235392.pth... [2023-12-27 09:35:22,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000569952_145924096.pth [2023-12-27 09:35:22,743][20381] Updated weights for policy 1, policy_version 570905 (0.0008) [2023-12-27 09:35:22,800][20381] Updated weights for policy 1, policy_version 570915 (0.0008) [2023-12-27 09:35:22,860][20381] Updated weights for policy 1, policy_version 570925 (0.0008) [2023-12-27 09:35:23,147][20347] Updated weights for policy 0, policy_version 571169 (0.0008) [2023-12-27 09:35:23,199][20347] Updated weights for policy 0, policy_version 571179 (0.0008) [2023-12-27 09:35:23,246][20347] Updated weights for policy 0, policy_version 571189 (0.0007) [2023-12-27 09:35:23,298][20347] Updated weights for policy 0, policy_version 571199 (0.0007) [2023-12-27 09:35:23,559][20381] Updated weights for policy 1, policy_version 570935 (0.0008) [2023-12-27 09:35:23,611][20381] Updated weights for policy 1, policy_version 570945 (0.0007) [2023-12-27 09:35:23,670][20381] Updated weights for policy 1, policy_version 570955 (0.0008) [2023-12-27 09:35:23,989][20347] Updated weights for policy 0, policy_version 571209 (0.0007) [2023-12-27 09:35:24,044][20347] Updated weights for policy 0, policy_version 571219 (0.0008) [2023-12-27 09:35:24,107][20347] Updated weights for policy 0, policy_version 571229 (0.0008) [2023-12-27 09:35:24,357][20381] Updated weights for policy 1, policy_version 570965 (0.0008) [2023-12-27 09:35:24,410][20381] Updated weights for policy 1, policy_version 570975 (0.0007) [2023-12-27 09:35:24,461][20381] Updated weights for policy 1, policy_version 570985 (0.0007) [2023-12-27 09:35:24,793][20347] Updated weights for policy 0, policy_version 571239 (0.0008) [2023-12-27 09:35:24,840][20347] Updated weights for policy 0, policy_version 571249 (0.0007) [2023-12-27 09:35:24,892][20347] Updated weights for policy 0, policy_version 571259 (0.0008) [2023-12-27 09:35:25,152][20381] Updated weights for policy 1, policy_version 570995 (0.0007) [2023-12-27 09:35:25,198][20381] Updated weights for policy 1, policy_version 571005 (0.0008) [2023-12-27 09:35:25,244][20381] Updated weights for policy 1, policy_version 571015 (0.0007) [2023-12-27 09:35:25,558][20347] Updated weights for policy 0, policy_version 571269 (0.0007) [2023-12-27 09:35:25,616][20347] Updated weights for policy 0, policy_version 571279 (0.0005) [2023-12-27 09:35:25,665][20347] Updated weights for policy 0, policy_version 571289 (0.0005) [2023-12-27 09:35:25,904][20381] Updated weights for policy 1, policy_version 571025 (0.0008) [2023-12-27 09:35:25,959][20381] Updated weights for policy 1, policy_version 571035 (0.0006) [2023-12-27 09:35:26,015][20381] Updated weights for policy 1, policy_version 571045 (0.0007) [2023-12-27 09:35:26,068][20381] Updated weights for policy 1, policy_version 571055 (0.0008) [2023-12-27 09:35:26,220][20347] Updated weights for policy 0, policy_version 571299 (0.0007) [2023-12-27 09:35:26,279][20347] Updated weights for policy 0, policy_version 571309 (0.0007) [2023-12-27 09:35:26,335][20347] Updated weights for policy 0, policy_version 571319 (0.0005) [2023-12-27 09:35:26,681][20381] Updated weights for policy 1, policy_version 571065 (0.0008) [2023-12-27 09:35:26,732][20381] Updated weights for policy 1, policy_version 571075 (0.0007) [2023-12-27 09:35:26,780][20381] Updated weights for policy 1, policy_version 571085 (0.0005) [2023-12-27 09:35:26,905][20347] Updated weights for policy 0, policy_version 571329 (0.0005) [2023-12-27 09:35:26,954][20347] Updated weights for policy 0, policy_version 571339 (0.0005) [2023-12-27 09:35:27,005][20347] Updated weights for policy 0, policy_version 571349 (0.0008) [2023-12-27 09:35:27,056][20347] Updated weights for policy 0, policy_version 571359 (0.0008) [2023-12-27 09:35:27,343][20381] Updated weights for policy 1, policy_version 571095 (0.0007) [2023-12-27 09:35:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 292503552. Throughput: 0: 10401.6, 1: 10000.4. Samples: 292492148. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:27,363][19170] Avg episode reward: [(0, '253.750'), (1, '1000.000')] [2023-12-27 09:35:27,402][20381] Updated weights for policy 1, policy_version 571105 (0.0007) [2023-12-27 09:35:27,466][20381] Updated weights for policy 1, policy_version 571115 (0.0009) [2023-12-27 09:35:27,619][20347] Updated weights for policy 0, policy_version 571369 (0.0007) [2023-12-27 09:35:27,668][20347] Updated weights for policy 0, policy_version 571379 (0.0009) [2023-12-27 09:35:27,725][20347] Updated weights for policy 0, policy_version 571389 (0.0008) [2023-12-27 09:35:28,030][20381] Updated weights for policy 1, policy_version 571125 (0.0007) [2023-12-27 09:35:28,082][20381] Updated weights for policy 1, policy_version 571135 (0.0008) [2023-12-27 09:35:28,130][20381] Updated weights for policy 1, policy_version 571145 (0.0005) [2023-12-27 09:35:28,454][20347] Updated weights for policy 0, policy_version 571399 (0.0009) [2023-12-27 09:35:28,516][20347] Updated weights for policy 0, policy_version 571409 (0.0010) [2023-12-27 09:35:28,574][20347] Updated weights for policy 0, policy_version 571419 (0.0010) [2023-12-27 09:35:28,748][20381] Updated weights for policy 1, policy_version 571155 (0.0006) [2023-12-27 09:35:28,797][20381] Updated weights for policy 1, policy_version 571165 (0.0008) [2023-12-27 09:35:28,848][20381] Updated weights for policy 1, policy_version 571175 (0.0007) [2023-12-27 09:35:29,258][20347] Updated weights for policy 0, policy_version 571429 (0.0009) [2023-12-27 09:35:29,313][20347] Updated weights for policy 0, policy_version 571439 (0.0010) [2023-12-27 09:35:29,373][20347] Updated weights for policy 0, policy_version 571449 (0.0011) [2023-12-27 09:35:29,546][20381] Updated weights for policy 1, policy_version 571185 (0.0007) [2023-12-27 09:35:29,608][20381] Updated weights for policy 1, policy_version 571195 (0.0006) [2023-12-27 09:35:29,657][20381] Updated weights for policy 1, policy_version 571205 (0.0008) [2023-12-27 09:35:29,715][20381] Updated weights for policy 1, policy_version 571215 (0.0008) [2023-12-27 09:35:30,046][20347] Updated weights for policy 0, policy_version 571459 (0.0009) [2023-12-27 09:35:30,099][20347] Updated weights for policy 0, policy_version 571469 (0.0007) [2023-12-27 09:35:30,161][20347] Updated weights for policy 0, policy_version 571479 (0.0008) [2023-12-27 09:35:30,359][20381] Updated weights for policy 1, policy_version 571225 (0.0006) [2023-12-27 09:35:30,417][20381] Updated weights for policy 1, policy_version 571235 (0.0005) [2023-12-27 09:35:30,485][20381] Updated weights for policy 1, policy_version 571245 (0.0005) [2023-12-27 09:35:30,811][20347] Updated weights for policy 0, policy_version 571489 (0.0007) [2023-12-27 09:35:30,864][20347] Updated weights for policy 0, policy_version 571499 (0.0005) [2023-12-27 09:35:30,919][20347] Updated weights for policy 0, policy_version 571509 (0.0005) [2023-12-27 09:35:30,967][20347] Updated weights for policy 0, policy_version 571519 (0.0006) [2023-12-27 09:35:30,973][20381] Updated weights for policy 1, policy_version 571255 (0.0005) [2023-12-27 09:35:31,024][20381] Updated weights for policy 1, policy_version 571265 (0.0005) [2023-12-27 09:35:31,085][20381] Updated weights for policy 1, policy_version 571275 (0.0007) [2023-12-27 09:35:31,617][20347] Updated weights for policy 0, policy_version 571529 (0.0007) [2023-12-27 09:35:31,677][20347] Updated weights for policy 0, policy_version 571539 (0.0007) [2023-12-27 09:35:31,718][20381] Updated weights for policy 1, policy_version 571285 (0.0007) [2023-12-27 09:35:31,744][20347] Updated weights for policy 0, policy_version 571549 (0.0007) [2023-12-27 09:35:31,779][20381] Updated weights for policy 1, policy_version 571295 (0.0007) [2023-12-27 09:35:31,842][20381] Updated weights for policy 1, policy_version 571305 (0.0008) [2023-12-27 09:35:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 292610048. Throughput: 0: 10393.0, 1: 10200.7. Samples: 292623612. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:32,363][19170] Avg episode reward: [(0, '204.000'), (1, '990.560')] [2023-12-27 09:35:32,462][20347] Updated weights for policy 0, policy_version 571559 (0.0008) [2023-12-27 09:35:32,476][20381] Updated weights for policy 1, policy_version 571315 (0.0007) [2023-12-27 09:35:32,508][20347] Updated weights for policy 0, policy_version 571569 (0.0006) [2023-12-27 09:35:32,533][20381] Updated weights for policy 1, policy_version 571325 (0.0006) [2023-12-27 09:35:32,559][20347] Updated weights for policy 0, policy_version 571579 (0.0006) [2023-12-27 09:35:32,592][20381] Updated weights for policy 1, policy_version 571335 (0.0007) [2023-12-27 09:35:33,128][20347] Updated weights for policy 0, policy_version 571589 (0.0008) [2023-12-27 09:35:33,182][20347] Updated weights for policy 0, policy_version 571599 (0.0007) [2023-12-27 09:35:33,232][20381] Updated weights for policy 1, policy_version 571345 (0.0008) [2023-12-27 09:35:33,235][20347] Updated weights for policy 0, policy_version 571609 (0.0007) [2023-12-27 09:35:33,286][20381] Updated weights for policy 1, policy_version 571355 (0.0010) [2023-12-27 09:35:33,337][20381] Updated weights for policy 1, policy_version 571365 (0.0010) [2023-12-27 09:35:33,391][20381] Updated weights for policy 1, policy_version 571375 (0.0010) [2023-12-27 09:35:33,863][20347] Updated weights for policy 0, policy_version 571619 (0.0008) [2023-12-27 09:35:33,908][20347] Updated weights for policy 0, policy_version 571629 (0.0005) [2023-12-27 09:35:33,945][20381] Updated weights for policy 1, policy_version 571385 (0.0009) [2023-12-27 09:35:33,958][20347] Updated weights for policy 0, policy_version 571639 (0.0006) [2023-12-27 09:35:34,002][20381] Updated weights for policy 1, policy_version 571395 (0.0006) [2023-12-27 09:35:34,054][20381] Updated weights for policy 1, policy_version 571405 (0.0008) [2023-12-27 09:35:34,635][20347] Updated weights for policy 0, policy_version 571649 (0.0007) [2023-12-27 09:35:34,700][20347] Updated weights for policy 0, policy_version 571659 (0.0005) [2023-12-27 09:35:34,757][20347] Updated weights for policy 0, policy_version 571669 (0.0007) [2023-12-27 09:35:34,779][20381] Updated weights for policy 1, policy_version 571415 (0.0006) [2023-12-27 09:35:34,809][20347] Updated weights for policy 0, policy_version 571679 (0.0007) [2023-12-27 09:35:34,841][20381] Updated weights for policy 1, policy_version 571425 (0.0007) [2023-12-27 09:35:34,912][20381] Updated weights for policy 1, policy_version 571435 (0.0007) [2023-12-27 09:35:35,458][20381] Updated weights for policy 1, policy_version 571445 (0.0006) [2023-12-27 09:35:35,466][20347] Updated weights for policy 0, policy_version 571689 (0.0008) [2023-12-27 09:35:35,507][20381] Updated weights for policy 1, policy_version 571455 (0.0005) [2023-12-27 09:35:35,515][20347] Updated weights for policy 0, policy_version 571699 (0.0008) [2023-12-27 09:35:35,563][20381] Updated weights for policy 1, policy_version 571465 (0.0005) [2023-12-27 09:35:35,567][20347] Updated weights for policy 0, policy_version 571709 (0.0007) [2023-12-27 09:35:36,148][20381] Updated weights for policy 1, policy_version 571475 (0.0006) [2023-12-27 09:35:36,206][20381] Updated weights for policy 1, policy_version 571485 (0.0006) [2023-12-27 09:35:36,230][20347] Updated weights for policy 0, policy_version 571719 (0.0007) [2023-12-27 09:35:36,266][20381] Updated weights for policy 1, policy_version 571495 (0.0006) [2023-12-27 09:35:36,279][20347] Updated weights for policy 0, policy_version 571729 (0.0007) [2023-12-27 09:35:36,342][20347] Updated weights for policy 0, policy_version 571739 (0.0008) [2023-12-27 09:35:36,927][20381] Updated weights for policy 1, policy_version 571505 (0.0006) [2023-12-27 09:35:36,990][20381] Updated weights for policy 1, policy_version 571515 (0.0008) [2023-12-27 09:35:37,040][20347] Updated weights for policy 0, policy_version 571749 (0.0008) [2023-12-27 09:35:37,050][20381] Updated weights for policy 1, policy_version 571525 (0.0007) [2023-12-27 09:35:37,100][20347] Updated weights for policy 0, policy_version 571759 (0.0007) [2023-12-27 09:35:37,108][20381] Updated weights for policy 1, policy_version 571535 (0.0009) [2023-12-27 09:35:37,157][20347] Updated weights for policy 0, policy_version 571769 (0.0008) [2023-12-27 09:35:37,362][19170] Fps is (10 sec: 22118.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 292724736. Throughput: 0: 10438.2, 1: 10181.9. Samples: 292688916. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:37,363][19170] Avg episode reward: [(0, '287.280'), (1, '981.000')] [2023-12-27 09:35:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000571536_146333696.pth... [2023-12-27 09:35:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000571776_146391040.pth... [2023-12-27 09:35:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000570320_146022400.pth [2023-12-27 09:35:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000570560_146079744.pth [2023-12-27 09:35:37,763][20381] Updated weights for policy 1, policy_version 571545 (0.0007) [2023-12-27 09:35:37,819][20381] Updated weights for policy 1, policy_version 571555 (0.0007) [2023-12-27 09:35:37,828][20347] Updated weights for policy 0, policy_version 571779 (0.0007) [2023-12-27 09:35:37,870][20381] Updated weights for policy 1, policy_version 571565 (0.0010) [2023-12-27 09:35:37,876][20347] Updated weights for policy 0, policy_version 571789 (0.0005) [2023-12-27 09:35:37,927][20347] Updated weights for policy 0, policy_version 571799 (0.0007) [2023-12-27 09:35:38,595][20347] Updated weights for policy 0, policy_version 571809 (0.0008) [2023-12-27 09:35:38,595][20381] Updated weights for policy 1, policy_version 571575 (0.0009) [2023-12-27 09:35:38,648][20347] Updated weights for policy 0, policy_version 571819 (0.0007) [2023-12-27 09:35:38,652][20381] Updated weights for policy 1, policy_version 571585 (0.0008) [2023-12-27 09:35:38,704][20347] Updated weights for policy 0, policy_version 571829 (0.0006) [2023-12-27 09:35:38,711][20381] Updated weights for policy 1, policy_version 571595 (0.0009) [2023-12-27 09:35:38,761][20347] Updated weights for policy 0, policy_version 571839 (0.0007) [2023-12-27 09:35:39,393][20347] Updated weights for policy 0, policy_version 571849 (0.0007) [2023-12-27 09:35:39,462][20347] Updated weights for policy 0, policy_version 571859 (0.0008) [2023-12-27 09:35:39,476][20381] Updated weights for policy 1, policy_version 571605 (0.0006) [2023-12-27 09:35:39,527][20347] Updated weights for policy 0, policy_version 571869 (0.0007) [2023-12-27 09:35:39,536][20381] Updated weights for policy 1, policy_version 571615 (0.0005) [2023-12-27 09:35:39,601][20381] Updated weights for policy 1, policy_version 571625 (0.0006) [2023-12-27 09:35:40,197][20347] Updated weights for policy 0, policy_version 571879 (0.0007) [2023-12-27 09:35:40,259][20381] Updated weights for policy 1, policy_version 571635 (0.0008) [2023-12-27 09:35:40,260][20347] Updated weights for policy 0, policy_version 571889 (0.0006) [2023-12-27 09:35:40,319][20381] Updated weights for policy 1, policy_version 571645 (0.0007) [2023-12-27 09:35:40,324][20347] Updated weights for policy 0, policy_version 571899 (0.0005) [2023-12-27 09:35:40,376][20381] Updated weights for policy 1, policy_version 571655 (0.0008) [2023-12-27 09:35:40,877][20347] Updated weights for policy 0, policy_version 571909 (0.0007) [2023-12-27 09:35:40,942][20347] Updated weights for policy 0, policy_version 571919 (0.0008) [2023-12-27 09:35:41,005][20347] Updated weights for policy 0, policy_version 571929 (0.0008) [2023-12-27 09:35:41,079][20381] Updated weights for policy 1, policy_version 571665 (0.0010) [2023-12-27 09:35:41,141][20381] Updated weights for policy 1, policy_version 571675 (0.0011) [2023-12-27 09:35:41,211][20381] Updated weights for policy 1, policy_version 571685 (0.0007) [2023-12-27 09:35:41,282][20381] Updated weights for policy 1, policy_version 571695 (0.0008) [2023-12-27 09:35:41,677][20347] Updated weights for policy 0, policy_version 571939 (0.0007) [2023-12-27 09:35:41,740][20347] Updated weights for policy 0, policy_version 571949 (0.0007) [2023-12-27 09:35:41,799][20347] Updated weights for policy 0, policy_version 571959 (0.0008) [2023-12-27 09:35:41,979][20381] Updated weights for policy 1, policy_version 571705 (0.0011) [2023-12-27 09:35:42,031][20381] Updated weights for policy 1, policy_version 571715 (0.0011) [2023-12-27 09:35:42,084][20381] Updated weights for policy 1, policy_version 571725 (0.0010) [2023-12-27 09:35:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 292823040. Throughput: 0: 10426.8, 1: 10210.4. Samples: 292813780. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:42,363][19170] Avg episode reward: [(0, '679.500'), (1, '951.740')] [2023-12-27 09:35:42,455][20347] Updated weights for policy 0, policy_version 571969 (0.0007) [2023-12-27 09:35:42,512][20347] Updated weights for policy 0, policy_version 571979 (0.0007) [2023-12-27 09:35:42,567][20347] Updated weights for policy 0, policy_version 571989 (0.0006) [2023-12-27 09:35:42,625][20347] Updated weights for policy 0, policy_version 571999 (0.0007) [2023-12-27 09:35:42,775][20381] Updated weights for policy 1, policy_version 571735 (0.0011) [2023-12-27 09:35:42,837][20381] Updated weights for policy 1, policy_version 571745 (0.0011) [2023-12-27 09:35:42,889][20381] Updated weights for policy 1, policy_version 571755 (0.0011) [2023-12-27 09:35:43,329][20347] Updated weights for policy 0, policy_version 572009 (0.0006) [2023-12-27 09:35:43,393][20347] Updated weights for policy 0, policy_version 572019 (0.0005) [2023-12-27 09:35:43,444][20347] Updated weights for policy 0, policy_version 572029 (0.0005) [2023-12-27 09:35:43,574][20381] Updated weights for policy 1, policy_version 571765 (0.0008) [2023-12-27 09:35:43,637][20381] Updated weights for policy 1, policy_version 571775 (0.0005) [2023-12-27 09:35:43,697][20381] Updated weights for policy 1, policy_version 571785 (0.0006) [2023-12-27 09:35:44,051][20347] Updated weights for policy 0, policy_version 572039 (0.0008) [2023-12-27 09:35:44,106][20347] Updated weights for policy 0, policy_version 572049 (0.0010) [2023-12-27 09:35:44,157][20347] Updated weights for policy 0, policy_version 572059 (0.0010) [2023-12-27 09:35:44,271][20381] Updated weights for policy 1, policy_version 571795 (0.0009) [2023-12-27 09:35:44,325][20381] Updated weights for policy 1, policy_version 571805 (0.0007) [2023-12-27 09:35:44,380][20381] Updated weights for policy 1, policy_version 571815 (0.0007) [2023-12-27 09:35:44,847][20347] Updated weights for policy 0, policy_version 572069 (0.0008) [2023-12-27 09:35:44,917][20347] Updated weights for policy 0, policy_version 572079 (0.0006) [2023-12-27 09:35:44,980][20347] Updated weights for policy 0, policy_version 572089 (0.0009) [2023-12-27 09:35:45,014][20381] Updated weights for policy 1, policy_version 571825 (0.0007) [2023-12-27 09:35:45,066][20381] Updated weights for policy 1, policy_version 571835 (0.0007) [2023-12-27 09:35:45,120][20381] Updated weights for policy 1, policy_version 571845 (0.0007) [2023-12-27 09:35:45,170][20381] Updated weights for policy 1, policy_version 571855 (0.0008) [2023-12-27 09:35:45,619][20347] Updated weights for policy 0, policy_version 572099 (0.0011) [2023-12-27 09:35:45,673][20347] Updated weights for policy 0, policy_version 572109 (0.0010) [2023-12-27 09:35:45,724][20347] Updated weights for policy 0, policy_version 572119 (0.0009) [2023-12-27 09:35:45,890][20381] Updated weights for policy 1, policy_version 571865 (0.0006) [2023-12-27 09:35:45,947][20381] Updated weights for policy 1, policy_version 571875 (0.0005) [2023-12-27 09:35:46,005][20381] Updated weights for policy 1, policy_version 571885 (0.0005) [2023-12-27 09:35:46,383][20347] Updated weights for policy 0, policy_version 572129 (0.0010) [2023-12-27 09:35:46,437][20347] Updated weights for policy 0, policy_version 572139 (0.0008) [2023-12-27 09:35:46,489][20347] Updated weights for policy 0, policy_version 572149 (0.0007) [2023-12-27 09:35:46,534][20347] Updated weights for policy 0, policy_version 572159 (0.0010) [2023-12-27 09:35:46,574][20381] Updated weights for policy 1, policy_version 571895 (0.0006) [2023-12-27 09:35:46,631][20381] Updated weights for policy 1, policy_version 571905 (0.0007) [2023-12-27 09:35:46,687][20381] Updated weights for policy 1, policy_version 571915 (0.0007) [2023-12-27 09:35:47,223][20347] Updated weights for policy 0, policy_version 572169 (0.0007) [2023-12-27 09:35:47,283][20347] Updated weights for policy 0, policy_version 572179 (0.0009) [2023-12-27 09:35:47,335][20347] Updated weights for policy 0, policy_version 572189 (0.0010) [2023-12-27 09:35:47,338][20381] Updated weights for policy 1, policy_version 571925 (0.0007) [2023-12-27 09:35:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 292929536. Throughput: 0: 10461.4, 1: 10339.3. Samples: 292939352. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:47,363][19170] Avg episode reward: [(0, '767.300'), (1, '924.090')] [2023-12-27 09:35:47,401][20381] Updated weights for policy 1, policy_version 571935 (0.0008) [2023-12-27 09:35:47,457][20381] Updated weights for policy 1, policy_version 571945 (0.0007) [2023-12-27 09:35:47,936][20347] Updated weights for policy 0, policy_version 572199 (0.0007) [2023-12-27 09:35:47,992][20347] Updated weights for policy 0, policy_version 572209 (0.0005) [2023-12-27 09:35:48,051][20347] Updated weights for policy 0, policy_version 572219 (0.0005) [2023-12-27 09:35:48,154][20381] Updated weights for policy 1, policy_version 571955 (0.0007) [2023-12-27 09:35:48,213][20381] Updated weights for policy 1, policy_version 571965 (0.0008) [2023-12-27 09:35:48,267][20381] Updated weights for policy 1, policy_version 571975 (0.0009) [2023-12-27 09:35:48,655][20347] Updated weights for policy 0, policy_version 572229 (0.0006) [2023-12-27 09:35:48,721][20347] Updated weights for policy 0, policy_version 572239 (0.0009) [2023-12-27 09:35:48,789][20347] Updated weights for policy 0, policy_version 572249 (0.0008) [2023-12-27 09:35:48,976][20381] Updated weights for policy 1, policy_version 571985 (0.0008) [2023-12-27 09:35:49,035][20381] Updated weights for policy 1, policy_version 571995 (0.0008) [2023-12-27 09:35:49,085][20381] Updated weights for policy 1, policy_version 572005 (0.0010) [2023-12-27 09:35:49,144][20381] Updated weights for policy 1, policy_version 572015 (0.0011) [2023-12-27 09:35:49,411][20347] Updated weights for policy 0, policy_version 572259 (0.0007) [2023-12-27 09:35:49,473][20347] Updated weights for policy 0, policy_version 572269 (0.0008) [2023-12-27 09:35:49,535][20347] Updated weights for policy 0, policy_version 572279 (0.0007) [2023-12-27 09:35:49,844][20381] Updated weights for policy 1, policy_version 572025 (0.0007) [2023-12-27 09:35:49,901][20381] Updated weights for policy 1, policy_version 572035 (0.0006) [2023-12-27 09:35:49,961][20381] Updated weights for policy 1, policy_version 572045 (0.0008) [2023-12-27 09:35:50,187][20347] Updated weights for policy 0, policy_version 572289 (0.0006) [2023-12-27 09:35:50,247][20347] Updated weights for policy 0, policy_version 572299 (0.0006) [2023-12-27 09:35:50,308][20347] Updated weights for policy 0, policy_version 572309 (0.0005) [2023-12-27 09:35:50,359][20347] Updated weights for policy 0, policy_version 572319 (0.0006) [2023-12-27 09:35:50,558][20381] Updated weights for policy 1, policy_version 572055 (0.0005) [2023-12-27 09:35:50,624][20381] Updated weights for policy 1, policy_version 572065 (0.0006) [2023-12-27 09:35:50,685][20381] Updated weights for policy 1, policy_version 572075 (0.0007) [2023-12-27 09:35:51,014][20347] Updated weights for policy 0, policy_version 572329 (0.0007) [2023-12-27 09:35:51,073][20347] Updated weights for policy 0, policy_version 572339 (0.0008) [2023-12-27 09:35:51,131][20347] Updated weights for policy 0, policy_version 572349 (0.0008) [2023-12-27 09:35:51,325][20381] Updated weights for policy 1, policy_version 572085 (0.0007) [2023-12-27 09:35:51,387][20381] Updated weights for policy 1, policy_version 572095 (0.0008) [2023-12-27 09:35:51,448][20381] Updated weights for policy 1, policy_version 572105 (0.0009) [2023-12-27 09:35:51,796][20347] Updated weights for policy 0, policy_version 572359 (0.0007) [2023-12-27 09:35:51,852][20347] Updated weights for policy 0, policy_version 572369 (0.0008) [2023-12-27 09:35:51,905][20347] Updated weights for policy 0, policy_version 572379 (0.0007) [2023-12-27 09:35:52,194][20381] Updated weights for policy 1, policy_version 572115 (0.0009) [2023-12-27 09:35:52,247][20381] Updated weights for policy 1, policy_version 572125 (0.0009) [2023-12-27 09:35:52,299][20381] Updated weights for policy 1, policy_version 572135 (0.0007) [2023-12-27 09:35:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 293036032. Throughput: 0: 10481.4, 1: 10345.3. Samples: 293001824. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:52,363][19170] Avg episode reward: [(0, '831.330'), (1, '943.590')] [2023-12-27 09:35:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000572144_146489344.pth... [2023-12-27 09:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000572384_146546688.pth... [2023-12-27 09:35:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000570896_146169856.pth [2023-12-27 09:35:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000571168_146235392.pth [2023-12-27 09:35:52,526][20347] Updated weights for policy 0, policy_version 572389 (0.0006) [2023-12-27 09:35:52,583][20347] Updated weights for policy 0, policy_version 572399 (0.0005) [2023-12-27 09:35:52,638][20347] Updated weights for policy 0, policy_version 572409 (0.0007) [2023-12-27 09:35:53,033][20381] Updated weights for policy 1, policy_version 572145 (0.0007) [2023-12-27 09:35:53,085][20381] Updated weights for policy 1, policy_version 572155 (0.0010) [2023-12-27 09:35:53,145][20381] Updated weights for policy 1, policy_version 572165 (0.0011) [2023-12-27 09:35:53,191][20347] Updated weights for policy 0, policy_version 572419 (0.0006) [2023-12-27 09:35:53,198][20381] Updated weights for policy 1, policy_version 572175 (0.0010) [2023-12-27 09:35:53,242][20347] Updated weights for policy 0, policy_version 572429 (0.0005) [2023-12-27 09:35:53,288][20347] Updated weights for policy 0, policy_version 572439 (0.0006) [2023-12-27 09:35:53,859][20381] Updated weights for policy 1, policy_version 572185 (0.0007) [2023-12-27 09:35:53,905][20381] Updated weights for policy 1, policy_version 572195 (0.0005) [2023-12-27 09:35:53,932][20347] Updated weights for policy 0, policy_version 572449 (0.0008) [2023-12-27 09:35:53,955][20381] Updated weights for policy 1, policy_version 572205 (0.0006) [2023-12-27 09:35:53,987][20347] Updated weights for policy 0, policy_version 572459 (0.0010) [2023-12-27 09:35:54,041][20347] Updated weights for policy 0, policy_version 572469 (0.0010) [2023-12-27 09:35:54,095][20347] Updated weights for policy 0, policy_version 572479 (0.0010) [2023-12-27 09:35:54,585][20381] Updated weights for policy 1, policy_version 572215 (0.0005) [2023-12-27 09:35:54,641][20381] Updated weights for policy 1, policy_version 572225 (0.0007) [2023-12-27 09:35:54,697][20381] Updated weights for policy 1, policy_version 572235 (0.0007) [2023-12-27 09:35:54,701][20347] Updated weights for policy 0, policy_version 572489 (0.0006) [2023-12-27 09:35:54,754][20347] Updated weights for policy 0, policy_version 572499 (0.0006) [2023-12-27 09:35:54,807][20347] Updated weights for policy 0, policy_version 572509 (0.0007) [2023-12-27 09:35:55,367][20381] Updated weights for policy 1, policy_version 572245 (0.0007) [2023-12-27 09:35:55,398][20347] Updated weights for policy 0, policy_version 572519 (0.0006) [2023-12-27 09:35:55,424][20381] Updated weights for policy 1, policy_version 572255 (0.0006) [2023-12-27 09:35:55,450][20347] Updated weights for policy 0, policy_version 572529 (0.0008) [2023-12-27 09:35:55,481][20381] Updated weights for policy 1, policy_version 572265 (0.0006) [2023-12-27 09:35:55,505][20347] Updated weights for policy 0, policy_version 572539 (0.0005) [2023-12-27 09:35:56,126][20381] Updated weights for policy 1, policy_version 572275 (0.0006) [2023-12-27 09:35:56,173][20381] Updated weights for policy 1, policy_version 572285 (0.0007) [2023-12-27 09:35:56,194][20347] Updated weights for policy 0, policy_version 572549 (0.0006) [2023-12-27 09:35:56,220][20381] Updated weights for policy 1, policy_version 572295 (0.0007) [2023-12-27 09:35:56,245][20347] Updated weights for policy 0, policy_version 572559 (0.0005) [2023-12-27 09:35:56,302][20347] Updated weights for policy 0, policy_version 572569 (0.0007) [2023-12-27 09:35:56,909][20381] Updated weights for policy 1, policy_version 572305 (0.0008) [2023-12-27 09:35:56,959][20381] Updated weights for policy 1, policy_version 572315 (0.0007) [2023-12-27 09:35:56,973][20347] Updated weights for policy 0, policy_version 572579 (0.0008) [2023-12-27 09:35:57,012][20381] Updated weights for policy 1, policy_version 572325 (0.0006) [2023-12-27 09:35:57,025][20347] Updated weights for policy 0, policy_version 572589 (0.0006) [2023-12-27 09:35:57,070][20381] Updated weights for policy 1, policy_version 572335 (0.0008) [2023-12-27 09:35:57,076][20347] Updated weights for policy 0, policy_version 572599 (0.0006) [2023-12-27 09:35:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 293142528. Throughput: 0: 10604.1, 1: 10373.6. Samples: 293129468. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:35:57,363][19170] Avg episode reward: [(0, '855.410'), (1, '971.210')] [2023-12-27 09:35:57,746][20381] Updated weights for policy 1, policy_version 572345 (0.0005) [2023-12-27 09:35:57,767][20347] Updated weights for policy 0, policy_version 572609 (0.0007) [2023-12-27 09:35:57,793][20381] Updated weights for policy 1, policy_version 572355 (0.0007) [2023-12-27 09:35:57,821][20347] Updated weights for policy 0, policy_version 572619 (0.0006) [2023-12-27 09:35:57,846][20381] Updated weights for policy 1, policy_version 572365 (0.0006) [2023-12-27 09:35:57,879][20347] Updated weights for policy 0, policy_version 572629 (0.0007) [2023-12-27 09:35:57,931][20347] Updated weights for policy 0, policy_version 572639 (0.0008) [2023-12-27 09:35:58,440][20381] Updated weights for policy 1, policy_version 572375 (0.0006) [2023-12-27 09:35:58,504][20381] Updated weights for policy 1, policy_version 572385 (0.0007) [2023-12-27 09:35:58,568][20381] Updated weights for policy 1, policy_version 572395 (0.0008) [2023-12-27 09:35:58,659][20347] Updated weights for policy 0, policy_version 572649 (0.0007) [2023-12-27 09:35:58,720][20347] Updated weights for policy 0, policy_version 572659 (0.0008) [2023-12-27 09:35:58,782][20347] Updated weights for policy 0, policy_version 572669 (0.0006) [2023-12-27 09:35:59,270][20381] Updated weights for policy 1, policy_version 572405 (0.0008) [2023-12-27 09:35:59,337][20381] Updated weights for policy 1, policy_version 572415 (0.0008) [2023-12-27 09:35:59,403][20381] Updated weights for policy 1, policy_version 572425 (0.0008) [2023-12-27 09:35:59,463][20347] Updated weights for policy 0, policy_version 572679 (0.0010) [2023-12-27 09:35:59,527][20347] Updated weights for policy 0, policy_version 572689 (0.0011) [2023-12-27 09:35:59,595][20347] Updated weights for policy 0, policy_version 572699 (0.0009) [2023-12-27 09:36:00,077][20381] Updated weights for policy 1, policy_version 572435 (0.0007) [2023-12-27 09:36:00,134][20381] Updated weights for policy 1, policy_version 572445 (0.0006) [2023-12-27 09:36:00,191][20381] Updated weights for policy 1, policy_version 572455 (0.0009) [2023-12-27 09:36:00,266][20347] Updated weights for policy 0, policy_version 572709 (0.0007) [2023-12-27 09:36:00,326][20347] Updated weights for policy 0, policy_version 572719 (0.0005) [2023-12-27 09:36:00,384][20347] Updated weights for policy 0, policy_version 572729 (0.0007) [2023-12-27 09:36:00,781][20381] Updated weights for policy 1, policy_version 572465 (0.0007) [2023-12-27 09:36:00,841][20381] Updated weights for policy 1, policy_version 572475 (0.0010) [2023-12-27 09:36:00,895][20381] Updated weights for policy 1, policy_version 572485 (0.0010) [2023-12-27 09:36:00,940][20381] Updated weights for policy 1, policy_version 572495 (0.0010) [2023-12-27 09:36:00,977][20347] Updated weights for policy 0, policy_version 572739 (0.0007) [2023-12-27 09:36:01,038][20347] Updated weights for policy 0, policy_version 572749 (0.0010) [2023-12-27 09:36:01,099][20347] Updated weights for policy 0, policy_version 572759 (0.0010) [2023-12-27 09:36:01,632][20381] Updated weights for policy 1, policy_version 572505 (0.0011) [2023-12-27 09:36:01,688][20381] Updated weights for policy 1, policy_version 572515 (0.0011) [2023-12-27 09:36:01,755][20381] Updated weights for policy 1, policy_version 572525 (0.0008) [2023-12-27 09:36:01,789][20347] Updated weights for policy 0, policy_version 572769 (0.0010) [2023-12-27 09:36:01,848][20291] KL-divergence is very high: 255.0283 [2023-12-27 09:36:01,852][20347] Updated weights for policy 0, policy_version 572779 (0.0005) [2023-12-27 09:36:01,892][20291] KL-divergence is very high: 352.4301 [2023-12-27 09:36:01,907][20347] Updated weights for policy 0, policy_version 572789 (0.0005) [2023-12-27 09:36:01,929][20291] KL-divergence is very high: 167.2995 [2023-12-27 09:36:01,962][20347] Updated weights for policy 0, policy_version 572799 (0.0007) [2023-12-27 09:36:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 293240832. Throughput: 0: 10547.7, 1: 10442.9. Samples: 293253728. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:02,363][19170] Avg episode reward: [(0, '866.310'), (1, '980.720')] [2023-12-27 09:36:02,420][20381] Updated weights for policy 1, policy_version 572535 (0.0007) [2023-12-27 09:36:02,472][20381] Updated weights for policy 1, policy_version 572545 (0.0008) [2023-12-27 09:36:02,522][20381] Updated weights for policy 1, policy_version 572555 (0.0007) [2023-12-27 09:36:02,575][20347] Updated weights for policy 0, policy_version 572809 (0.0006) [2023-12-27 09:36:02,630][20347] Updated weights for policy 0, policy_version 572819 (0.0007) [2023-12-27 09:36:02,687][20347] Updated weights for policy 0, policy_version 572829 (0.0007) [2023-12-27 09:36:03,225][20381] Updated weights for policy 1, policy_version 572565 (0.0009) [2023-12-27 09:36:03,285][20381] Updated weights for policy 1, policy_version 572575 (0.0010) [2023-12-27 09:36:03,342][20381] Updated weights for policy 1, policy_version 572585 (0.0009) [2023-12-27 09:36:03,355][20347] Updated weights for policy 0, policy_version 572839 (0.0006) [2023-12-27 09:36:03,404][20347] Updated weights for policy 0, policy_version 572849 (0.0006) [2023-12-27 09:36:03,459][20347] Updated weights for policy 0, policy_version 572859 (0.0007) [2023-12-27 09:36:03,964][20381] Updated weights for policy 1, policy_version 572595 (0.0010) [2023-12-27 09:36:04,024][20381] Updated weights for policy 1, policy_version 572605 (0.0011) [2023-12-27 09:36:04,081][20381] Updated weights for policy 1, policy_version 572615 (0.0011) [2023-12-27 09:36:04,187][20347] Updated weights for policy 0, policy_version 572869 (0.0008) [2023-12-27 09:36:04,239][20347] Updated weights for policy 0, policy_version 572879 (0.0008) [2023-12-27 09:36:04,296][20347] Updated weights for policy 0, policy_version 572889 (0.0010) [2023-12-27 09:36:04,741][20381] Updated weights for policy 1, policy_version 572625 (0.0010) [2023-12-27 09:36:04,797][20381] Updated weights for policy 1, policy_version 572635 (0.0011) [2023-12-27 09:36:04,849][20381] Updated weights for policy 1, policy_version 572645 (0.0010) [2023-12-27 09:36:04,909][20381] Updated weights for policy 1, policy_version 572655 (0.0010) [2023-12-27 09:36:04,962][20347] Updated weights for policy 0, policy_version 572899 (0.0009) [2023-12-27 09:36:05,021][20347] Updated weights for policy 0, policy_version 572909 (0.0007) [2023-12-27 09:36:05,078][20347] Updated weights for policy 0, policy_version 572919 (0.0007) [2023-12-27 09:36:05,559][20381] Updated weights for policy 1, policy_version 572665 (0.0005) [2023-12-27 09:36:05,621][20381] Updated weights for policy 1, policy_version 572675 (0.0005) [2023-12-27 09:36:05,683][20381] Updated weights for policy 1, policy_version 572685 (0.0005) [2023-12-27 09:36:05,794][20347] Updated weights for policy 0, policy_version 572929 (0.0008) [2023-12-27 09:36:05,864][20347] Updated weights for policy 0, policy_version 572939 (0.0008) [2023-12-27 09:36:05,926][20347] Updated weights for policy 0, policy_version 572949 (0.0007) [2023-12-27 09:36:05,987][20347] Updated weights for policy 0, policy_version 572959 (0.0008) [2023-12-27 09:36:06,214][20381] Updated weights for policy 1, policy_version 572695 (0.0006) [2023-12-27 09:36:06,272][20381] Updated weights for policy 1, policy_version 572705 (0.0005) [2023-12-27 09:36:06,329][20381] Updated weights for policy 1, policy_version 572715 (0.0006) [2023-12-27 09:36:06,697][20347] Updated weights for policy 0, policy_version 572969 (0.0007) [2023-12-27 09:36:06,749][20347] Updated weights for policy 0, policy_version 572979 (0.0007) [2023-12-27 09:36:06,801][20347] Updated weights for policy 0, policy_version 572989 (0.0007) [2023-12-27 09:36:06,962][20381] Updated weights for policy 1, policy_version 572725 (0.0008) [2023-12-27 09:36:07,015][20381] Updated weights for policy 1, policy_version 572735 (0.0011) [2023-12-27 09:36:07,068][20381] Updated weights for policy 1, policy_version 572745 (0.0011) [2023-12-27 09:36:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 293347328. Throughput: 0: 10541.9, 1: 10472.5. Samples: 293315724. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:07,363][19170] Avg episode reward: [(0, '867.760'), (1, '932.520')] [2023-12-27 09:36:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000572752_146644992.pth... [2023-12-27 09:36:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000572992_146702336.pth... [2023-12-27 09:36:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000571536_146333696.pth [2023-12-27 09:36:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000571776_146391040.pth [2023-12-27 09:36:07,511][20347] Updated weights for policy 0, policy_version 572999 (0.0008) [2023-12-27 09:36:07,573][20347] Updated weights for policy 0, policy_version 573009 (0.0010) [2023-12-27 09:36:07,633][20347] Updated weights for policy 0, policy_version 573019 (0.0010) [2023-12-27 09:36:07,766][20381] Updated weights for policy 1, policy_version 572755 (0.0009) [2023-12-27 09:36:07,824][20381] Updated weights for policy 1, policy_version 572765 (0.0009) [2023-12-27 09:36:07,877][20381] Updated weights for policy 1, policy_version 572775 (0.0010) [2023-12-27 09:36:08,263][20347] Updated weights for policy 0, policy_version 573029 (0.0008) [2023-12-27 09:36:08,315][20347] Updated weights for policy 0, policy_version 573039 (0.0005) [2023-12-27 09:36:08,375][20347] Updated weights for policy 0, policy_version 573049 (0.0008) [2023-12-27 09:36:08,466][20381] Updated weights for policy 1, policy_version 572785 (0.0010) [2023-12-27 09:36:08,521][20381] Updated weights for policy 1, policy_version 572795 (0.0010) [2023-12-27 09:36:08,577][20381] Updated weights for policy 1, policy_version 572805 (0.0011) [2023-12-27 09:36:08,629][20381] Updated weights for policy 1, policy_version 572815 (0.0010) [2023-12-27 09:36:09,060][20347] Updated weights for policy 0, policy_version 573059 (0.0010) [2023-12-27 09:36:09,125][20347] Updated weights for policy 0, policy_version 573069 (0.0010) [2023-12-27 09:36:09,177][20347] Updated weights for policy 0, policy_version 573079 (0.0010) [2023-12-27 09:36:09,316][20381] Updated weights for policy 1, policy_version 572825 (0.0011) [2023-12-27 09:36:09,382][20381] Updated weights for policy 1, policy_version 572835 (0.0010) [2023-12-27 09:36:09,445][20381] Updated weights for policy 1, policy_version 572845 (0.0008) [2023-12-27 09:36:09,902][20347] Updated weights for policy 0, policy_version 573089 (0.0008) [2023-12-27 09:36:09,972][20347] Updated weights for policy 0, policy_version 573099 (0.0007) [2023-12-27 09:36:10,032][20347] Updated weights for policy 0, policy_version 573109 (0.0008) [2023-12-27 09:36:10,093][20347] Updated weights for policy 0, policy_version 573119 (0.0007) [2023-12-27 09:36:10,156][20381] Updated weights for policy 1, policy_version 572855 (0.0009) [2023-12-27 09:36:10,214][20381] Updated weights for policy 1, policy_version 572865 (0.0011) [2023-12-27 09:36:10,274][20381] Updated weights for policy 1, policy_version 572875 (0.0011) [2023-12-27 09:36:10,790][20347] Updated weights for policy 0, policy_version 573129 (0.0008) [2023-12-27 09:36:10,840][20347] Updated weights for policy 0, policy_version 573139 (0.0007) [2023-12-27 09:36:10,888][20347] Updated weights for policy 0, policy_version 573149 (0.0007) [2023-12-27 09:36:10,979][20381] Updated weights for policy 1, policy_version 572885 (0.0011) [2023-12-27 09:36:11,035][20381] Updated weights for policy 1, policy_version 572895 (0.0010) [2023-12-27 09:36:11,099][20381] Updated weights for policy 1, policy_version 572905 (0.0011) [2023-12-27 09:36:11,675][20347] Updated weights for policy 0, policy_version 573159 (0.0008) [2023-12-27 09:36:11,741][20347] Updated weights for policy 0, policy_version 573169 (0.0008) [2023-12-27 09:36:11,782][20381] Updated weights for policy 1, policy_version 572915 (0.0009) [2023-12-27 09:36:11,795][20347] Updated weights for policy 0, policy_version 573179 (0.0007) [2023-12-27 09:36:11,833][20381] Updated weights for policy 1, policy_version 572925 (0.0006) [2023-12-27 09:36:11,879][20381] Updated weights for policy 1, policy_version 572935 (0.0005) [2023-12-27 09:36:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 293445632. Throughput: 0: 10486.0, 1: 10538.0. Samples: 293438232. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:12,363][19170] Avg episode reward: [(0, '837.350'), (1, '980.730')] [2023-12-27 09:36:12,451][20347] Updated weights for policy 0, policy_version 573189 (0.0006) [2023-12-27 09:36:12,509][20347] Updated weights for policy 0, policy_version 573199 (0.0008) [2023-12-27 09:36:12,555][20381] Updated weights for policy 1, policy_version 572945 (0.0005) [2023-12-27 09:36:12,564][20347] Updated weights for policy 0, policy_version 573209 (0.0007) [2023-12-27 09:36:12,614][20381] Updated weights for policy 1, policy_version 572955 (0.0007) [2023-12-27 09:36:12,680][20381] Updated weights for policy 1, policy_version 572966 (0.0008) [2023-12-27 09:36:12,736][20381] Updated weights for policy 1, policy_version 572976 (0.0006) [2023-12-27 09:36:13,249][20347] Updated weights for policy 0, policy_version 573219 (0.0006) [2023-12-27 09:36:13,297][20347] Updated weights for policy 0, policy_version 573229 (0.0005) [2023-12-27 09:36:13,356][20347] Updated weights for policy 0, policy_version 573239 (0.0007) [2023-12-27 09:36:13,395][20381] Updated weights for policy 1, policy_version 572986 (0.0007) [2023-12-27 09:36:13,450][20381] Updated weights for policy 1, policy_version 572996 (0.0006) [2023-12-27 09:36:13,499][20381] Updated weights for policy 1, policy_version 573006 (0.0005) [2023-12-27 09:36:13,915][20347] Updated weights for policy 0, policy_version 573249 (0.0008) [2023-12-27 09:36:13,971][20347] Updated weights for policy 0, policy_version 573259 (0.0010) [2023-12-27 09:36:14,031][20381] Updated weights for policy 1, policy_version 573016 (0.0007) [2023-12-27 09:36:14,038][20347] Updated weights for policy 0, policy_version 573269 (0.0010) [2023-12-27 09:36:14,085][20381] Updated weights for policy 1, policy_version 573026 (0.0007) [2023-12-27 09:36:14,100][20347] Updated weights for policy 0, policy_version 573279 (0.0010) [2023-12-27 09:36:14,147][20381] Updated weights for policy 1, policy_version 573036 (0.0005) [2023-12-27 09:36:14,761][20381] Updated weights for policy 1, policy_version 573046 (0.0006) [2023-12-27 09:36:14,762][20347] Updated weights for policy 0, policy_version 573289 (0.0007) [2023-12-27 09:36:14,820][20381] Updated weights for policy 1, policy_version 573056 (0.0005) [2023-12-27 09:36:14,831][20347] Updated weights for policy 0, policy_version 573299 (0.0009) [2023-12-27 09:36:14,879][20381] Updated weights for policy 1, policy_version 573066 (0.0009) [2023-12-27 09:36:14,890][20347] Updated weights for policy 0, policy_version 573309 (0.0010) [2023-12-27 09:36:15,535][20347] Updated weights for policy 0, policy_version 573319 (0.0007) [2023-12-27 09:36:15,585][20347] Updated weights for policy 0, policy_version 573329 (0.0007) [2023-12-27 09:36:15,586][20381] Updated weights for policy 1, policy_version 573076 (0.0007) [2023-12-27 09:36:15,628][20381] Updated weights for policy 1, policy_version 573086 (0.0006) [2023-12-27 09:36:15,633][20347] Updated weights for policy 0, policy_version 573339 (0.0005) [2023-12-27 09:36:15,678][20381] Updated weights for policy 1, policy_version 573096 (0.0007) [2023-12-27 09:36:16,215][20347] Updated weights for policy 0, policy_version 573349 (0.0005) [2023-12-27 09:36:16,276][20347] Updated weights for policy 0, policy_version 573359 (0.0006) [2023-12-27 09:36:16,335][20347] Updated weights for policy 0, policy_version 573369 (0.0007) [2023-12-27 09:36:16,380][20381] Updated weights for policy 1, policy_version 573106 (0.0009) [2023-12-27 09:36:16,441][20381] Updated weights for policy 1, policy_version 573116 (0.0011) [2023-12-27 09:36:16,496][20381] Updated weights for policy 1, policy_version 573126 (0.0011) [2023-12-27 09:36:16,556][20381] Updated weights for policy 1, policy_version 573136 (0.0011) [2023-12-27 09:36:16,917][20347] Updated weights for policy 0, policy_version 573379 (0.0006) [2023-12-27 09:36:16,966][20347] Updated weights for policy 0, policy_version 573389 (0.0005) [2023-12-27 09:36:17,015][20347] Updated weights for policy 0, policy_version 573399 (0.0006) [2023-12-27 09:36:17,219][20381] Updated weights for policy 1, policy_version 573146 (0.0007) [2023-12-27 09:36:17,269][20381] Updated weights for policy 1, policy_version 573156 (0.0005) [2023-12-27 09:36:17,326][20381] Updated weights for policy 1, policy_version 573166 (0.0009) [2023-12-27 09:36:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.2, 300 sec: 20716.1). Total num frames: 293560320. Throughput: 0: 10505.4, 1: 10440.3. Samples: 293566168. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:17,363][19170] Avg episode reward: [(0, '826.350'), (1, '981.210')] [2023-12-27 09:36:17,646][20347] Updated weights for policy 0, policy_version 573409 (0.0008) [2023-12-27 09:36:17,700][20347] Updated weights for policy 0, policy_version 573419 (0.0007) [2023-12-27 09:36:17,756][20347] Updated weights for policy 0, policy_version 573429 (0.0007) [2023-12-27 09:36:17,810][20347] Updated weights for policy 0, policy_version 573439 (0.0008) [2023-12-27 09:36:17,998][20381] Updated weights for policy 1, policy_version 573176 (0.0006) [2023-12-27 09:36:18,065][20381] Updated weights for policy 1, policy_version 573186 (0.0007) [2023-12-27 09:36:18,125][20381] Updated weights for policy 1, policy_version 573196 (0.0005) [2023-12-27 09:36:18,440][20347] Updated weights for policy 0, policy_version 573449 (0.0010) [2023-12-27 09:36:18,500][20347] Updated weights for policy 0, policy_version 573459 (0.0010) [2023-12-27 09:36:18,558][20347] Updated weights for policy 0, policy_version 573469 (0.0010) [2023-12-27 09:36:18,713][20381] Updated weights for policy 1, policy_version 573206 (0.0006) [2023-12-27 09:36:18,765][20381] Updated weights for policy 1, policy_version 573216 (0.0008) [2023-12-27 09:36:18,820][20381] Updated weights for policy 1, policy_version 573226 (0.0008) [2023-12-27 09:36:19,244][20347] Updated weights for policy 0, policy_version 573479 (0.0010) [2023-12-27 09:36:19,302][20347] Updated weights for policy 0, policy_version 573489 (0.0010) [2023-12-27 09:36:19,359][20347] Updated weights for policy 0, policy_version 573499 (0.0010) [2023-12-27 09:36:19,533][20381] Updated weights for policy 1, policy_version 573236 (0.0007) [2023-12-27 09:36:19,584][20381] Updated weights for policy 1, policy_version 573246 (0.0007) [2023-12-27 09:36:19,640][20381] Updated weights for policy 1, policy_version 573256 (0.0008) [2023-12-27 09:36:20,029][20347] Updated weights for policy 0, policy_version 573509 (0.0007) [2023-12-27 09:36:20,094][20347] Updated weights for policy 0, policy_version 573519 (0.0009) [2023-12-27 09:36:20,156][20347] Updated weights for policy 0, policy_version 573529 (0.0011) [2023-12-27 09:36:20,365][20381] Updated weights for policy 1, policy_version 573266 (0.0007) [2023-12-27 09:36:20,424][20381] Updated weights for policy 1, policy_version 573276 (0.0008) [2023-12-27 09:36:20,481][20381] Updated weights for policy 1, policy_version 573286 (0.0008) [2023-12-27 09:36:20,536][20381] Updated weights for policy 1, policy_version 573296 (0.0008) [2023-12-27 09:36:20,822][20347] Updated weights for policy 0, policy_version 573539 (0.0009) [2023-12-27 09:36:20,882][20347] Updated weights for policy 0, policy_version 573549 (0.0008) [2023-12-27 09:36:20,931][20347] Updated weights for policy 0, policy_version 573559 (0.0007) [2023-12-27 09:36:21,280][20381] Updated weights for policy 1, policy_version 573306 (0.0009) [2023-12-27 09:36:21,340][20381] Updated weights for policy 1, policy_version 573316 (0.0011) [2023-12-27 09:36:21,403][20381] Updated weights for policy 1, policy_version 573326 (0.0011) [2023-12-27 09:36:21,620][20347] Updated weights for policy 0, policy_version 573569 (0.0009) [2023-12-27 09:36:21,685][20347] Updated weights for policy 0, policy_version 573579 (0.0009) [2023-12-27 09:36:21,749][20347] Updated weights for policy 0, policy_version 573589 (0.0007) [2023-12-27 09:36:21,814][20347] Updated weights for policy 0, policy_version 573599 (0.0007) [2023-12-27 09:36:22,107][20381] Updated weights for policy 1, policy_version 573336 (0.0011) [2023-12-27 09:36:22,164][20381] Updated weights for policy 1, policy_version 573346 (0.0011) [2023-12-27 09:36:22,217][20381] Updated weights for policy 1, policy_version 573356 (0.0011) [2023-12-27 09:36:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 293658624. Throughput: 0: 10478.7, 1: 10397.4. Samples: 293628340. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:22,363][19170] Avg episode reward: [(0, '817.350'), (1, '972.020')] [2023-12-27 09:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000573360_146800640.pth... [2023-12-27 09:36:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000573600_146857984.pth... [2023-12-27 09:36:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000572144_146489344.pth [2023-12-27 09:36:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000572384_146546688.pth [2023-12-27 09:36:22,438][20347] Updated weights for policy 0, policy_version 573609 (0.0008) [2023-12-27 09:36:22,493][20347] Updated weights for policy 0, policy_version 573619 (0.0007) [2023-12-27 09:36:22,538][20347] Updated weights for policy 0, policy_version 573629 (0.0007) [2023-12-27 09:36:22,892][20381] Updated weights for policy 1, policy_version 573366 (0.0009) [2023-12-27 09:36:22,940][20381] Updated weights for policy 1, policy_version 573376 (0.0010) [2023-12-27 09:36:22,997][20381] Updated weights for policy 1, policy_version 573386 (0.0007) [2023-12-27 09:36:23,130][20347] Updated weights for policy 0, policy_version 573639 (0.0006) [2023-12-27 09:36:23,187][20347] Updated weights for policy 0, policy_version 573649 (0.0007) [2023-12-27 09:36:23,243][20347] Updated weights for policy 0, policy_version 573659 (0.0008) [2023-12-27 09:36:23,687][20381] Updated weights for policy 1, policy_version 573396 (0.0010) [2023-12-27 09:36:23,740][20381] Updated weights for policy 1, policy_version 573406 (0.0010) [2023-12-27 09:36:23,793][20381] Updated weights for policy 1, policy_version 573416 (0.0010) [2023-12-27 09:36:23,914][20347] Updated weights for policy 0, policy_version 573669 (0.0008) [2023-12-27 09:36:23,962][20347] Updated weights for policy 0, policy_version 573679 (0.0010) [2023-12-27 09:36:24,017][20347] Updated weights for policy 0, policy_version 573689 (0.0010) [2023-12-27 09:36:24,359][20381] Updated weights for policy 1, policy_version 573426 (0.0009) [2023-12-27 09:36:24,415][20381] Updated weights for policy 1, policy_version 573436 (0.0007) [2023-12-27 09:36:24,471][20381] Updated weights for policy 1, policy_version 573446 (0.0009) [2023-12-27 09:36:24,530][20381] Updated weights for policy 1, policy_version 573456 (0.0005) [2023-12-27 09:36:24,693][20347] Updated weights for policy 0, policy_version 573699 (0.0008) [2023-12-27 09:36:24,747][20347] Updated weights for policy 0, policy_version 573709 (0.0010) [2023-12-27 09:36:24,802][20347] Updated weights for policy 0, policy_version 573719 (0.0008) [2023-12-27 09:36:25,156][20381] Updated weights for policy 1, policy_version 573466 (0.0009) [2023-12-27 09:36:25,204][20381] Updated weights for policy 1, policy_version 573476 (0.0005) [2023-12-27 09:36:25,262][20381] Updated weights for policy 1, policy_version 573486 (0.0005) [2023-12-27 09:36:25,367][20347] Updated weights for policy 0, policy_version 573729 (0.0006) [2023-12-27 09:36:25,421][20347] Updated weights for policy 0, policy_version 573739 (0.0010) [2023-12-27 09:36:25,482][20347] Updated weights for policy 0, policy_version 573749 (0.0010) [2023-12-27 09:36:25,534][20347] Updated weights for policy 0, policy_version 573759 (0.0010) [2023-12-27 09:36:25,858][20381] Updated weights for policy 1, policy_version 573496 (0.0007) [2023-12-27 09:36:25,913][20381] Updated weights for policy 1, policy_version 573506 (0.0007) [2023-12-27 09:36:25,966][20381] Updated weights for policy 1, policy_version 573516 (0.0008) [2023-12-27 09:36:26,169][20347] Updated weights for policy 0, policy_version 573769 (0.0010) [2023-12-27 09:36:26,225][20347] Updated weights for policy 0, policy_version 573779 (0.0010) [2023-12-27 09:36:26,280][20347] Updated weights for policy 0, policy_version 573789 (0.0010) [2023-12-27 09:36:26,642][20381] Updated weights for policy 1, policy_version 573526 (0.0006) [2023-12-27 09:36:26,695][20381] Updated weights for policy 1, policy_version 573536 (0.0005) [2023-12-27 09:36:26,740][20381] Updated weights for policy 1, policy_version 573546 (0.0005) [2023-12-27 09:36:26,911][20347] Updated weights for policy 0, policy_version 573799 (0.0010) [2023-12-27 09:36:26,962][20347] Updated weights for policy 0, policy_version 573809 (0.0010) [2023-12-27 09:36:27,014][20347] Updated weights for policy 0, policy_version 573819 (0.0009) [2023-12-27 09:36:27,270][20381] Updated weights for policy 1, policy_version 573556 (0.0005) [2023-12-27 09:36:27,331][20381] Updated weights for policy 1, policy_version 573566 (0.0006) [2023-12-27 09:36:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 293765120. Throughput: 0: 10511.7, 1: 10427.7. Samples: 293756056. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:27,363][19170] Avg episode reward: [(0, '769.640'), (1, '981.210')] [2023-12-27 09:36:27,379][20381] Updated weights for policy 1, policy_version 573576 (0.0005) [2023-12-27 09:36:27,670][20347] Updated weights for policy 0, policy_version 573829 (0.0008) [2023-12-27 09:36:27,721][20347] Updated weights for policy 0, policy_version 573839 (0.0010) [2023-12-27 09:36:27,775][20347] Updated weights for policy 0, policy_version 573849 (0.0010) [2023-12-27 09:36:27,940][20381] Updated weights for policy 1, policy_version 573586 (0.0006) [2023-12-27 09:36:27,985][20381] Updated weights for policy 1, policy_version 573596 (0.0010) [2023-12-27 09:36:28,034][20381] Updated weights for policy 1, policy_version 573606 (0.0010) [2023-12-27 09:36:28,082][20381] Updated weights for policy 1, policy_version 573616 (0.0010) [2023-12-27 09:36:28,425][20347] Updated weights for policy 0, policy_version 573859 (0.0009) [2023-12-27 09:36:28,477][20347] Updated weights for policy 0, policy_version 573869 (0.0010) [2023-12-27 09:36:28,524][20347] Updated weights for policy 0, policy_version 573879 (0.0007) [2023-12-27 09:36:28,709][20381] Updated weights for policy 1, policy_version 573626 (0.0010) [2023-12-27 09:36:28,767][20381] Updated weights for policy 1, policy_version 573636 (0.0010) [2023-12-27 09:36:28,828][20381] Updated weights for policy 1, policy_version 573646 (0.0010) [2023-12-27 09:36:29,198][20347] Updated weights for policy 0, policy_version 573889 (0.0010) [2023-12-27 09:36:29,262][20347] Updated weights for policy 0, policy_version 573899 (0.0008) [2023-12-27 09:36:29,320][20347] Updated weights for policy 0, policy_version 573909 (0.0007) [2023-12-27 09:36:29,386][20347] Updated weights for policy 0, policy_version 573919 (0.0006) [2023-12-27 09:36:29,434][20381] Updated weights for policy 1, policy_version 573656 (0.0005) [2023-12-27 09:36:29,487][20381] Updated weights for policy 1, policy_version 573666 (0.0006) [2023-12-27 09:36:29,548][20381] Updated weights for policy 1, policy_version 573676 (0.0009) [2023-12-27 09:36:30,023][20347] Updated weights for policy 0, policy_version 573929 (0.0007) [2023-12-27 09:36:30,075][20347] Updated weights for policy 0, policy_version 573939 (0.0007) [2023-12-27 09:36:30,130][20347] Updated weights for policy 0, policy_version 573949 (0.0008) [2023-12-27 09:36:30,253][20381] Updated weights for policy 1, policy_version 573686 (0.0007) [2023-12-27 09:36:30,313][20381] Updated weights for policy 1, policy_version 573696 (0.0008) [2023-12-27 09:36:30,358][20381] Updated weights for policy 1, policy_version 573706 (0.0008) [2023-12-27 09:36:30,743][20347] Updated weights for policy 0, policy_version 573959 (0.0008) [2023-12-27 09:36:30,791][20347] Updated weights for policy 0, policy_version 573969 (0.0006) [2023-12-27 09:36:30,837][20347] Updated weights for policy 0, policy_version 573979 (0.0005) [2023-12-27 09:36:31,132][20381] Updated weights for policy 1, policy_version 573716 (0.0008) [2023-12-27 09:36:31,195][20381] Updated weights for policy 1, policy_version 573726 (0.0008) [2023-12-27 09:36:31,251][20381] Updated weights for policy 1, policy_version 573736 (0.0008) [2023-12-27 09:36:31,489][20347] Updated weights for policy 0, policy_version 573989 (0.0005) [2023-12-27 09:36:31,543][20347] Updated weights for policy 0, policy_version 573999 (0.0005) [2023-12-27 09:36:31,605][20347] Updated weights for policy 0, policy_version 574009 (0.0005) [2023-12-27 09:36:31,964][20381] Updated weights for policy 1, policy_version 573746 (0.0007) [2023-12-27 09:36:32,024][20381] Updated weights for policy 1, policy_version 573756 (0.0008) [2023-12-27 09:36:32,077][20381] Updated weights for policy 1, policy_version 573766 (0.0007) [2023-12-27 09:36:32,138][20381] Updated weights for policy 1, policy_version 573776 (0.0007) [2023-12-27 09:36:32,249][20347] Updated weights for policy 0, policy_version 574019 (0.0009) [2023-12-27 09:36:32,307][20347] Updated weights for policy 0, policy_version 574029 (0.0010) [2023-12-27 09:36:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 293871616. Throughput: 0: 10551.2, 1: 10451.0. Samples: 293884448. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:32,363][19170] Avg episode reward: [(0, '807.190'), (1, '980.740')] [2023-12-27 09:36:32,391][20347] Updated weights for policy 0, policy_version 574039 (0.0010) [2023-12-27 09:36:32,789][20381] Updated weights for policy 1, policy_version 573786 (0.0009) [2023-12-27 09:36:32,845][20381] Updated weights for policy 1, policy_version 573796 (0.0008) [2023-12-27 09:36:32,901][20381] Updated weights for policy 1, policy_version 573806 (0.0008) [2023-12-27 09:36:32,946][20347] Updated weights for policy 0, policy_version 574049 (0.0006) [2023-12-27 09:36:33,002][20347] Updated weights for policy 0, policy_version 574059 (0.0008) [2023-12-27 09:36:33,038][20291] KL-divergence is very high: 107.9285 [2023-12-27 09:36:33,067][20347] Updated weights for policy 0, policy_version 574069 (0.0008) [2023-12-27 09:36:33,122][20347] Updated weights for policy 0, policy_version 574079 (0.0007) [2023-12-27 09:36:33,631][20381] Updated weights for policy 1, policy_version 573816 (0.0008) [2023-12-27 09:36:33,672][20381] Updated weights for policy 1, policy_version 573826 (0.0008) [2023-12-27 09:36:33,718][20347] Updated weights for policy 0, policy_version 574089 (0.0006) [2023-12-27 09:36:33,720][20381] Updated weights for policy 1, policy_version 573836 (0.0006) [2023-12-27 09:36:33,767][20347] Updated weights for policy 0, policy_version 574099 (0.0006) [2023-12-27 09:36:33,821][20347] Updated weights for policy 0, policy_version 574109 (0.0007) [2023-12-27 09:36:34,444][20381] Updated weights for policy 1, policy_version 573846 (0.0007) [2023-12-27 09:36:34,473][20347] Updated weights for policy 0, policy_version 574119 (0.0009) [2023-12-27 09:36:34,505][20381] Updated weights for policy 1, policy_version 573856 (0.0006) [2023-12-27 09:36:34,529][20347] Updated weights for policy 0, policy_version 574129 (0.0010) [2023-12-27 09:36:34,561][20381] Updated weights for policy 1, policy_version 573866 (0.0005) [2023-12-27 09:36:34,591][20347] Updated weights for policy 0, policy_version 574139 (0.0010) [2023-12-27 09:36:35,129][20381] Updated weights for policy 1, policy_version 573876 (0.0007) [2023-12-27 09:36:35,183][20381] Updated weights for policy 1, policy_version 573886 (0.0008) [2023-12-27 09:36:35,239][20381] Updated weights for policy 1, policy_version 573896 (0.0007) [2023-12-27 09:36:35,247][20347] Updated weights for policy 0, policy_version 574149 (0.0008) [2023-12-27 09:36:35,307][20347] Updated weights for policy 0, policy_version 574159 (0.0010) [2023-12-27 09:36:35,365][20347] Updated weights for policy 0, policy_version 574169 (0.0010) [2023-12-27 09:36:35,943][20381] Updated weights for policy 1, policy_version 573906 (0.0007) [2023-12-27 09:36:35,950][20347] Updated weights for policy 0, policy_version 574179 (0.0009) [2023-12-27 09:36:35,995][20381] Updated weights for policy 1, policy_version 573916 (0.0007) [2023-12-27 09:36:36,001][20347] Updated weights for policy 0, policy_version 574189 (0.0005) [2023-12-27 09:36:36,051][20381] Updated weights for policy 1, policy_version 573926 (0.0007) [2023-12-27 09:36:36,066][20347] Updated weights for policy 0, policy_version 574199 (0.0005) [2023-12-27 09:36:36,111][20381] Updated weights for policy 1, policy_version 573936 (0.0007) [2023-12-27 09:36:36,702][20347] Updated weights for policy 0, policy_version 574209 (0.0008) [2023-12-27 09:36:36,753][20347] Updated weights for policy 0, policy_version 574219 (0.0005) [2023-12-27 09:36:36,803][20347] Updated weights for policy 0, policy_version 574229 (0.0007) [2023-12-27 09:36:36,861][20381] Updated weights for policy 1, policy_version 573946 (0.0006) [2023-12-27 09:36:36,869][20347] Updated weights for policy 0, policy_version 574239 (0.0010) [2023-12-27 09:36:36,905][20381] Updated weights for policy 1, policy_version 573956 (0.0007) [2023-12-27 09:36:36,956][20381] Updated weights for policy 1, policy_version 573966 (0.0007) [2023-12-27 09:36:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 293978112. Throughput: 0: 10548.2, 1: 10481.5. Samples: 293948160. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:37,363][19170] Avg episode reward: [(0, '818.240'), (1, '961.310')] [2023-12-27 09:36:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000573968_146956288.pth... [2023-12-27 09:36:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000574240_147021824.pth... [2023-12-27 09:36:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000572752_146644992.pth [2023-12-27 09:36:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000572992_146702336.pth [2023-12-27 09:36:37,516][20347] Updated weights for policy 0, policy_version 574249 (0.0011) [2023-12-27 09:36:37,575][20347] Updated weights for policy 0, policy_version 574259 (0.0010) [2023-12-27 09:36:37,626][20347] Updated weights for policy 0, policy_version 574269 (0.0010) [2023-12-27 09:36:37,682][20292] KL-divergence is very high: 139.5914 [2023-12-27 09:36:37,682][20381] Updated weights for policy 1, policy_version 573976 (0.0007) [2023-12-27 09:36:37,739][20381] Updated weights for policy 1, policy_version 573986 (0.0007) [2023-12-27 09:36:37,772][20292] KL-divergence is very high: 104.7490 [2023-12-27 09:36:37,793][20381] Updated weights for policy 1, policy_version 573996 (0.0007) [2023-12-27 09:36:38,306][20347] Updated weights for policy 0, policy_version 574279 (0.0011) [2023-12-27 09:36:38,369][20347] Updated weights for policy 0, policy_version 574289 (0.0011) [2023-12-27 09:36:38,415][20347] Updated weights for policy 0, policy_version 574299 (0.0010) [2023-12-27 09:36:38,519][20381] Updated weights for policy 1, policy_version 574006 (0.0007) [2023-12-27 09:36:38,576][20381] Updated weights for policy 1, policy_version 574016 (0.0008) [2023-12-27 09:36:38,635][20381] Updated weights for policy 1, policy_version 574026 (0.0008) [2023-12-27 09:36:39,121][20347] Updated weights for policy 0, policy_version 574309 (0.0009) [2023-12-27 09:36:39,182][20347] Updated weights for policy 0, policy_version 574319 (0.0007) [2023-12-27 09:36:39,241][20347] Updated weights for policy 0, policy_version 574329 (0.0007) [2023-12-27 09:36:39,340][20381] Updated weights for policy 1, policy_version 574036 (0.0008) [2023-12-27 09:36:39,405][20381] Updated weights for policy 1, policy_version 574046 (0.0007) [2023-12-27 09:36:39,470][20381] Updated weights for policy 1, policy_version 574056 (0.0007) [2023-12-27 09:36:39,964][20347] Updated weights for policy 0, policy_version 574339 (0.0007) [2023-12-27 09:36:40,026][20347] Updated weights for policy 0, policy_version 574349 (0.0008) [2023-12-27 09:36:40,094][20347] Updated weights for policy 0, policy_version 574359 (0.0007) [2023-12-27 09:36:40,201][20381] Updated weights for policy 1, policy_version 574066 (0.0008) [2023-12-27 09:36:40,269][20381] Updated weights for policy 1, policy_version 574076 (0.0009) [2023-12-27 09:36:40,329][20381] Updated weights for policy 1, policy_version 574086 (0.0006) [2023-12-27 09:36:40,396][20381] Updated weights for policy 1, policy_version 574096 (0.0008) [2023-12-27 09:36:40,714][20347] Updated weights for policy 0, policy_version 574369 (0.0006) [2023-12-27 09:36:40,770][20347] Updated weights for policy 0, policy_version 574379 (0.0008) [2023-12-27 09:36:40,822][20347] Updated weights for policy 0, policy_version 574389 (0.0006) [2023-12-27 09:36:40,878][20347] Updated weights for policy 0, policy_version 574399 (0.0005) [2023-12-27 09:36:41,044][20381] Updated weights for policy 1, policy_version 574106 (0.0007) [2023-12-27 09:36:41,106][20381] Updated weights for policy 1, policy_version 574116 (0.0007) [2023-12-27 09:36:41,171][20381] Updated weights for policy 1, policy_version 574126 (0.0008) [2023-12-27 09:36:41,567][20347] Updated weights for policy 0, policy_version 574409 (0.0009) [2023-12-27 09:36:41,635][20347] Updated weights for policy 0, policy_version 574419 (0.0009) [2023-12-27 09:36:41,700][20347] Updated weights for policy 0, policy_version 574429 (0.0008) [2023-12-27 09:36:41,902][20381] Updated weights for policy 1, policy_version 574136 (0.0008) [2023-12-27 09:36:41,966][20381] Updated weights for policy 1, policy_version 574146 (0.0008) [2023-12-27 09:36:42,020][20381] Updated weights for policy 1, policy_version 574156 (0.0008) [2023-12-27 09:36:42,349][20347] Updated weights for policy 0, policy_version 574439 (0.0009) [2023-12-27 09:36:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 294076416. Throughput: 0: 10504.6, 1: 10397.4. Samples: 294070060. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:42,363][19170] Avg episode reward: [(0, '723.540'), (1, '970.970')] [2023-12-27 09:36:42,411][20347] Updated weights for policy 0, policy_version 574449 (0.0006) [2023-12-27 09:36:42,475][20347] Updated weights for policy 0, policy_version 574459 (0.0007) [2023-12-27 09:36:42,680][20381] Updated weights for policy 1, policy_version 574166 (0.0007) [2023-12-27 09:36:42,744][20381] Updated weights for policy 1, policy_version 574176 (0.0007) [2023-12-27 09:36:42,807][20381] Updated weights for policy 1, policy_version 574186 (0.0006) [2023-12-27 09:36:43,097][20347] Updated weights for policy 0, policy_version 574469 (0.0006) [2023-12-27 09:36:43,145][20347] Updated weights for policy 0, policy_version 574479 (0.0005) [2023-12-27 09:36:43,208][20347] Updated weights for policy 0, policy_version 574489 (0.0006) [2023-12-27 09:36:43,399][20381] Updated weights for policy 1, policy_version 574196 (0.0007) [2023-12-27 09:36:43,454][20381] Updated weights for policy 1, policy_version 574206 (0.0005) [2023-12-27 09:36:43,507][20381] Updated weights for policy 1, policy_version 574216 (0.0005) [2023-12-27 09:36:43,750][20347] Updated weights for policy 0, policy_version 574499 (0.0007) [2023-12-27 09:36:43,796][20347] Updated weights for policy 0, policy_version 574509 (0.0009) [2023-12-27 09:36:43,848][20347] Updated weights for policy 0, policy_version 574519 (0.0008) [2023-12-27 09:36:44,056][20381] Updated weights for policy 1, policy_version 574226 (0.0006) [2023-12-27 09:36:44,120][20381] Updated weights for policy 1, policy_version 574236 (0.0009) [2023-12-27 09:36:44,163][20381] Updated weights for policy 1, policy_version 574246 (0.0008) [2023-12-27 09:36:44,212][20381] Updated weights for policy 1, policy_version 574256 (0.0007) [2023-12-27 09:36:44,500][20347] Updated weights for policy 0, policy_version 574529 (0.0008) [2023-12-27 09:36:44,554][20347] Updated weights for policy 0, policy_version 574539 (0.0010) [2023-12-27 09:36:44,610][20347] Updated weights for policy 0, policy_version 574549 (0.0008) [2023-12-27 09:36:44,664][20347] Updated weights for policy 0, policy_version 574559 (0.0005) [2023-12-27 09:36:44,970][20381] Updated weights for policy 1, policy_version 574266 (0.0011) [2023-12-27 09:36:45,035][20381] Updated weights for policy 1, policy_version 574276 (0.0009) [2023-12-27 09:36:45,100][20381] Updated weights for policy 1, policy_version 574286 (0.0005) [2023-12-27 09:36:45,275][20347] Updated weights for policy 0, policy_version 574569 (0.0005) [2023-12-27 09:36:45,334][20347] Updated weights for policy 0, policy_version 574579 (0.0009) [2023-12-27 09:36:45,389][20347] Updated weights for policy 0, policy_version 574589 (0.0010) [2023-12-27 09:36:45,727][20381] Updated weights for policy 1, policy_version 574296 (0.0009) [2023-12-27 09:36:45,779][20381] Updated weights for policy 1, policy_version 574306 (0.0010) [2023-12-27 09:36:45,824][20381] Updated weights for policy 1, policy_version 574316 (0.0010) [2023-12-27 09:36:46,053][20347] Updated weights for policy 0, policy_version 574599 (0.0008) [2023-12-27 09:36:46,107][20347] Updated weights for policy 0, policy_version 574609 (0.0007) [2023-12-27 09:36:46,160][20347] Updated weights for policy 0, policy_version 574619 (0.0007) [2023-12-27 09:36:46,543][20381] Updated weights for policy 1, policy_version 574326 (0.0011) [2023-12-27 09:36:46,595][20381] Updated weights for policy 1, policy_version 574336 (0.0010) [2023-12-27 09:36:46,648][20381] Updated weights for policy 1, policy_version 574346 (0.0006) [2023-12-27 09:36:46,845][20347] Updated weights for policy 0, policy_version 574629 (0.0007) [2023-12-27 09:36:46,901][20347] Updated weights for policy 0, policy_version 574639 (0.0007) [2023-12-27 09:36:46,958][20347] Updated weights for policy 0, policy_version 574649 (0.0007) [2023-12-27 09:36:47,319][20381] Updated weights for policy 1, policy_version 574356 (0.0008) [2023-12-27 09:36:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.7, 300 sec: 20716.0). Total num frames: 294182912. Throughput: 0: 10557.2, 1: 10406.7. Samples: 294197100. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:47,363][19170] Avg episode reward: [(0, '772.210'), (1, '951.830')] [2023-12-27 09:36:47,379][20381] Updated weights for policy 1, policy_version 574366 (0.0010) [2023-12-27 09:36:47,445][20381] Updated weights for policy 1, policy_version 574376 (0.0011) [2023-12-27 09:36:47,639][20347] Updated weights for policy 0, policy_version 574659 (0.0007) [2023-12-27 09:36:47,688][20347] Updated weights for policy 0, policy_version 574669 (0.0008) [2023-12-27 09:36:47,735][20347] Updated weights for policy 0, policy_version 574679 (0.0007) [2023-12-27 09:36:48,017][20381] Updated weights for policy 1, policy_version 574386 (0.0010) [2023-12-27 09:36:48,071][20381] Updated weights for policy 1, policy_version 574396 (0.0005) [2023-12-27 09:36:48,127][20381] Updated weights for policy 1, policy_version 574406 (0.0009) [2023-12-27 09:36:48,184][20381] Updated weights for policy 1, policy_version 574416 (0.0010) [2023-12-27 09:36:48,490][20347] Updated weights for policy 0, policy_version 574689 (0.0009) [2023-12-27 09:36:48,554][20347] Updated weights for policy 0, policy_version 574699 (0.0007) [2023-12-27 09:36:48,611][20347] Updated weights for policy 0, policy_version 574709 (0.0007) [2023-12-27 09:36:48,672][20347] Updated weights for policy 0, policy_version 574719 (0.0007) [2023-12-27 09:36:48,849][20381] Updated weights for policy 1, policy_version 574426 (0.0011) [2023-12-27 09:36:48,855][20292] KL-divergence is very high: 110.9873 [2023-12-27 09:36:48,910][20381] Updated weights for policy 1, policy_version 574436 (0.0010) [2023-12-27 09:36:48,968][20381] Updated weights for policy 1, policy_version 574446 (0.0007) [2023-12-27 09:36:49,392][20347] Updated weights for policy 0, policy_version 574729 (0.0008) [2023-12-27 09:36:49,449][20347] Updated weights for policy 0, policy_version 574739 (0.0006) [2023-12-27 09:36:49,510][20347] Updated weights for policy 0, policy_version 574749 (0.0007) [2023-12-27 09:36:49,559][20292] KL-divergence is very high: 129.0663 [2023-12-27 09:36:49,605][20381] Updated weights for policy 1, policy_version 574456 (0.0008) [2023-12-27 09:36:49,656][20292] KL-divergence is very high: 127.3886 [2023-12-27 09:36:49,667][20381] Updated weights for policy 1, policy_version 574466 (0.0006) [2023-12-27 09:36:49,727][20381] Updated weights for policy 1, policy_version 574476 (0.0006) [2023-12-27 09:36:50,124][20347] Updated weights for policy 0, policy_version 574759 (0.0007) [2023-12-27 09:36:50,180][20347] Updated weights for policy 0, policy_version 574769 (0.0008) [2023-12-27 09:36:50,243][20347] Updated weights for policy 0, policy_version 574779 (0.0008) [2023-12-27 09:36:50,383][20381] Updated weights for policy 1, policy_version 574486 (0.0008) [2023-12-27 09:36:50,438][20381] Updated weights for policy 1, policy_version 574496 (0.0008) [2023-12-27 09:36:50,491][20381] Updated weights for policy 1, policy_version 574506 (0.0007) [2023-12-27 09:36:50,931][20347] Updated weights for policy 0, policy_version 574789 (0.0007) [2023-12-27 09:36:50,991][20347] Updated weights for policy 0, policy_version 574799 (0.0008) [2023-12-27 09:36:51,052][20347] Updated weights for policy 0, policy_version 574809 (0.0007) [2023-12-27 09:36:51,200][20292] KL-divergence is very high: 108.3269 [2023-12-27 09:36:51,201][20381] Updated weights for policy 1, policy_version 574516 (0.0006) [2023-12-27 09:36:51,260][20381] Updated weights for policy 1, policy_version 574526 (0.0006) [2023-12-27 09:36:51,320][20381] Updated weights for policy 1, policy_version 574536 (0.0009) [2023-12-27 09:36:51,714][20347] Updated weights for policy 0, policy_version 574819 (0.0006) [2023-12-27 09:36:51,765][20347] Updated weights for policy 0, policy_version 574829 (0.0006) [2023-12-27 09:36:51,820][20347] Updated weights for policy 0, policy_version 574839 (0.0006) [2023-12-27 09:36:52,036][20381] Updated weights for policy 1, policy_version 574546 (0.0008) [2023-12-27 09:36:52,086][20381] Updated weights for policy 1, policy_version 574556 (0.0008) [2023-12-27 09:36:52,139][20381] Updated weights for policy 1, policy_version 574566 (0.0008) [2023-12-27 09:36:52,195][20381] Updated weights for policy 1, policy_version 574576 (0.0007) [2023-12-27 09:36:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 294289408. Throughput: 0: 10543.9, 1: 10421.5. Samples: 294259168. Policy #0 lag: (min: 14.0, avg: 26.3, max: 46.0) [2023-12-27 09:36:52,363][19170] Avg episode reward: [(0, '756.110'), (1, '741.930')] [2023-12-27 09:36:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000574576_147111936.pth... [2023-12-27 09:36:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000574848_147177472.pth... [2023-12-27 09:36:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000573600_146857984.pth [2023-12-27 09:36:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000573360_146800640.pth [2023-12-27 09:36:52,494][20347] Updated weights for policy 0, policy_version 574849 (0.0007) [2023-12-27 09:36:52,543][20347] Updated weights for policy 0, policy_version 574859 (0.0007) [2023-12-27 09:36:52,605][20347] Updated weights for policy 0, policy_version 574869 (0.0008) [2023-12-27 09:36:52,661][20347] Updated weights for policy 0, policy_version 574879 (0.0008) [2023-12-27 09:36:52,910][20381] Updated weights for policy 1, policy_version 574586 (0.0006) [2023-12-27 09:36:52,947][20292] KL-divergence is very high: 125.6651 [2023-12-27 09:36:52,963][20292] KL-divergence is very high: 214.5769 [2023-12-27 09:36:52,964][20381] Updated weights for policy 1, policy_version 574596 (0.0005) [2023-12-27 09:36:52,973][20292] KL-divergence is very high: 204.6697 [2023-12-27 09:36:52,979][20292] KL-divergence is very high: 152.1651 [2023-12-27 09:36:53,016][20292] KL-divergence is very high: 118.0853 [2023-12-27 09:36:53,018][20381] Updated weights for policy 1, policy_version 574606 (0.0009) [2023-12-27 09:36:53,022][20292] KL-divergence is very high: 114.4279 [2023-12-27 09:36:53,308][20347] Updated weights for policy 0, policy_version 574889 (0.0008) [2023-12-27 09:36:53,369][20347] Updated weights for policy 0, policy_version 574899 (0.0007) [2023-12-27 09:36:53,420][20347] Updated weights for policy 0, policy_version 574909 (0.0007) [2023-12-27 09:36:53,735][20381] Updated weights for policy 1, policy_version 574616 (0.0010) [2023-12-27 09:36:53,785][20381] Updated weights for policy 1, policy_version 574626 (0.0009) [2023-12-27 09:36:53,838][20381] Updated weights for policy 1, policy_version 574636 (0.0005) [2023-12-27 09:36:54,049][20347] Updated weights for policy 0, policy_version 574919 (0.0006) [2023-12-27 09:36:54,102][20347] Updated weights for policy 0, policy_version 574929 (0.0005) [2023-12-27 09:36:54,159][20347] Updated weights for policy 0, policy_version 574939 (0.0005) [2023-12-27 09:36:54,517][20381] Updated weights for policy 1, policy_version 574646 (0.0010) [2023-12-27 09:36:54,568][20381] Updated weights for policy 1, policy_version 574656 (0.0010) [2023-12-27 09:36:54,618][20381] Updated weights for policy 1, policy_version 574666 (0.0010) [2023-12-27 09:36:54,815][20347] Updated weights for policy 0, policy_version 574949 (0.0007) [2023-12-27 09:36:54,871][20347] Updated weights for policy 0, policy_version 574959 (0.0007) [2023-12-27 09:36:54,929][20347] Updated weights for policy 0, policy_version 574969 (0.0008) [2023-12-27 09:36:55,282][20381] Updated weights for policy 1, policy_version 574676 (0.0008) [2023-12-27 09:36:55,327][20381] Updated weights for policy 1, policy_version 574686 (0.0005) [2023-12-27 09:36:55,385][20381] Updated weights for policy 1, policy_version 574696 (0.0005) [2023-12-27 09:36:55,499][20347] Updated weights for policy 0, policy_version 574979 (0.0006) [2023-12-27 09:36:55,567][20347] Updated weights for policy 0, policy_version 574989 (0.0006) [2023-12-27 09:36:55,620][20347] Updated weights for policy 0, policy_version 574999 (0.0005) [2023-12-27 09:36:55,913][20381] Updated weights for policy 1, policy_version 574706 (0.0005) [2023-12-27 09:36:55,958][20381] Updated weights for policy 1, policy_version 574716 (0.0008) [2023-12-27 09:36:56,000][20381] Updated weights for policy 1, policy_version 574726 (0.0006) [2023-12-27 09:36:56,046][20381] Updated weights for policy 1, policy_version 574736 (0.0005) [2023-12-27 09:36:56,211][20347] Updated weights for policy 0, policy_version 575009 (0.0005) [2023-12-27 09:36:56,270][20347] Updated weights for policy 0, policy_version 575019 (0.0005) [2023-12-27 09:36:56,327][20347] Updated weights for policy 0, policy_version 575029 (0.0005) [2023-12-27 09:36:56,371][20347] Updated weights for policy 0, policy_version 575039 (0.0005) [2023-12-27 09:36:56,652][20381] Updated weights for policy 1, policy_version 574746 (0.0008) [2023-12-27 09:36:56,701][20381] Updated weights for policy 1, policy_version 574756 (0.0008) [2023-12-27 09:36:56,750][20381] Updated weights for policy 1, policy_version 574766 (0.0008) [2023-12-27 09:36:56,915][20347] Updated weights for policy 0, policy_version 575049 (0.0005) [2023-12-27 09:36:56,970][20347] Updated weights for policy 0, policy_version 575059 (0.0007) [2023-12-27 09:36:57,021][20347] Updated weights for policy 0, policy_version 575069 (0.0007) [2023-12-27 09:36:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 294395904. Throughput: 0: 10642.2, 1: 10417.6. Samples: 294385920. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:36:57,363][19170] Avg episode reward: [(0, '764.830'), (1, '420.980')] [2023-12-27 09:36:57,442][20381] Updated weights for policy 1, policy_version 574776 (0.0008) [2023-12-27 09:36:57,501][20381] Updated weights for policy 1, policy_version 574786 (0.0008) [2023-12-27 09:36:57,557][20381] Updated weights for policy 1, policy_version 574796 (0.0007) [2023-12-27 09:36:57,682][20347] Updated weights for policy 0, policy_version 575079 (0.0008) [2023-12-27 09:36:57,740][20347] Updated weights for policy 0, policy_version 575089 (0.0008) [2023-12-27 09:36:57,788][20347] Updated weights for policy 0, policy_version 575099 (0.0007) [2023-12-27 09:36:58,197][20381] Updated weights for policy 1, policy_version 574806 (0.0008) [2023-12-27 09:36:58,255][20381] Updated weights for policy 1, policy_version 574816 (0.0007) [2023-12-27 09:36:58,312][20381] Updated weights for policy 1, policy_version 574826 (0.0007) [2023-12-27 09:36:58,556][20347] Updated weights for policy 0, policy_version 575109 (0.0007) [2023-12-27 09:36:58,620][20347] Updated weights for policy 0, policy_version 575119 (0.0008) [2023-12-27 09:36:58,683][20347] Updated weights for policy 0, policy_version 575129 (0.0008) [2023-12-27 09:36:59,015][20381] Updated weights for policy 1, policy_version 574836 (0.0008) [2023-12-27 09:36:59,061][20381] Updated weights for policy 1, policy_version 574846 (0.0007) [2023-12-27 09:36:59,107][20381] Updated weights for policy 1, policy_version 574856 (0.0008) [2023-12-27 09:36:59,394][20347] Updated weights for policy 0, policy_version 575139 (0.0008) [2023-12-27 09:36:59,447][20347] Updated weights for policy 0, policy_version 575149 (0.0007) [2023-12-27 09:36:59,501][20347] Updated weights for policy 0, policy_version 575159 (0.0007) [2023-12-27 09:36:59,849][20381] Updated weights for policy 1, policy_version 574866 (0.0007) [2023-12-27 09:36:59,910][20381] Updated weights for policy 1, policy_version 574876 (0.0008) [2023-12-27 09:36:59,971][20381] Updated weights for policy 1, policy_version 574886 (0.0006) [2023-12-27 09:37:00,025][20381] Updated weights for policy 1, policy_version 574896 (0.0007) [2023-12-27 09:37:00,214][20347] Updated weights for policy 0, policy_version 575169 (0.0008) [2023-12-27 09:37:00,265][20347] Updated weights for policy 0, policy_version 575179 (0.0007) [2023-12-27 09:37:00,310][20347] Updated weights for policy 0, policy_version 575189 (0.0007) [2023-12-27 09:37:00,362][20347] Updated weights for policy 0, policy_version 575199 (0.0007) [2023-12-27 09:37:00,728][20381] Updated weights for policy 1, policy_version 574906 (0.0007) [2023-12-27 09:37:00,776][20381] Updated weights for policy 1, policy_version 574916 (0.0007) [2023-12-27 09:37:00,823][20381] Updated weights for policy 1, policy_version 574926 (0.0008) [2023-12-27 09:37:01,087][20347] Updated weights for policy 0, policy_version 575209 (0.0007) [2023-12-27 09:37:01,147][20347] Updated weights for policy 0, policy_version 575219 (0.0008) [2023-12-27 09:37:01,200][20347] Updated weights for policy 0, policy_version 575229 (0.0008) [2023-12-27 09:37:01,544][20381] Updated weights for policy 1, policy_version 574936 (0.0007) [2023-12-27 09:37:01,601][20381] Updated weights for policy 1, policy_version 574946 (0.0007) [2023-12-27 09:37:01,660][20381] Updated weights for policy 1, policy_version 574956 (0.0006) [2023-12-27 09:37:01,919][20347] Updated weights for policy 0, policy_version 575239 (0.0008) [2023-12-27 09:37:01,979][20347] Updated weights for policy 0, policy_version 575249 (0.0008) [2023-12-27 09:37:02,040][20347] Updated weights for policy 0, policy_version 575259 (0.0008) [2023-12-27 09:37:02,336][20381] Updated weights for policy 1, policy_version 574966 (0.0007) [2023-12-27 09:37:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 294494208. Throughput: 0: 10549.3, 1: 10361.9. Samples: 294507172. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:02,363][19170] Avg episode reward: [(0, '780.270'), (1, '624.360')] [2023-12-27 09:37:02,397][20381] Updated weights for policy 1, policy_version 574976 (0.0007) [2023-12-27 09:37:02,458][20381] Updated weights for policy 1, policy_version 574986 (0.0009) [2023-12-27 09:37:02,712][20347] Updated weights for policy 0, policy_version 575269 (0.0007) [2023-12-27 09:37:02,773][20347] Updated weights for policy 0, policy_version 575279 (0.0008) [2023-12-27 09:37:02,832][20347] Updated weights for policy 0, policy_version 575289 (0.0006) [2023-12-27 09:37:03,160][20381] Updated weights for policy 1, policy_version 574996 (0.0007) [2023-12-27 09:37:03,218][20381] Updated weights for policy 1, policy_version 575006 (0.0005) [2023-12-27 09:37:03,280][20381] Updated weights for policy 1, policy_version 575016 (0.0006) [2023-12-27 09:37:03,549][20347] Updated weights for policy 0, policy_version 575299 (0.0007) [2023-12-27 09:37:03,595][20347] Updated weights for policy 0, policy_version 575309 (0.0008) [2023-12-27 09:37:03,640][20347] Updated weights for policy 0, policy_version 575319 (0.0007) [2023-12-27 09:37:03,868][20381] Updated weights for policy 1, policy_version 575026 (0.0006) [2023-12-27 09:37:03,932][20381] Updated weights for policy 1, policy_version 575036 (0.0006) [2023-12-27 09:37:03,994][20381] Updated weights for policy 1, policy_version 575046 (0.0007) [2023-12-27 09:37:04,051][20381] Updated weights for policy 1, policy_version 575056 (0.0008) [2023-12-27 09:37:04,386][20347] Updated weights for policy 0, policy_version 575329 (0.0008) [2023-12-27 09:37:04,447][20347] Updated weights for policy 0, policy_version 575339 (0.0006) [2023-12-27 09:37:04,501][20347] Updated weights for policy 0, policy_version 575349 (0.0007) [2023-12-27 09:37:04,555][20347] Updated weights for policy 0, policy_version 575359 (0.0008) [2023-12-27 09:37:04,680][20381] Updated weights for policy 1, policy_version 575066 (0.0008) [2023-12-27 09:37:04,732][20381] Updated weights for policy 1, policy_version 575076 (0.0007) [2023-12-27 09:37:04,782][20381] Updated weights for policy 1, policy_version 575086 (0.0009) [2023-12-27 09:37:05,270][20347] Updated weights for policy 0, policy_version 575369 (0.0007) [2023-12-27 09:37:05,319][20347] Updated weights for policy 0, policy_version 575379 (0.0007) [2023-12-27 09:37:05,381][20347] Updated weights for policy 0, policy_version 575389 (0.0008) [2023-12-27 09:37:05,428][20381] Updated weights for policy 1, policy_version 575096 (0.0006) [2023-12-27 09:37:05,485][20381] Updated weights for policy 1, policy_version 575106 (0.0010) [2023-12-27 09:37:05,539][20381] Updated weights for policy 1, policy_version 575116 (0.0010) [2023-12-27 09:37:06,107][20347] Updated weights for policy 0, policy_version 575399 (0.0007) [2023-12-27 09:37:06,126][20381] Updated weights for policy 1, policy_version 575126 (0.0010) [2023-12-27 09:37:06,161][20347] Updated weights for policy 0, policy_version 575409 (0.0005) [2023-12-27 09:37:06,188][20381] Updated weights for policy 1, policy_version 575136 (0.0008) [2023-12-27 09:37:06,215][20347] Updated weights for policy 0, policy_version 575419 (0.0007) [2023-12-27 09:37:06,249][20381] Updated weights for policy 1, policy_version 575146 (0.0008) [2023-12-27 09:37:06,934][20347] Updated weights for policy 0, policy_version 575429 (0.0006) [2023-12-27 09:37:06,946][20381] Updated weights for policy 1, policy_version 575156 (0.0009) [2023-12-27 09:37:06,996][20347] Updated weights for policy 0, policy_version 575439 (0.0005) [2023-12-27 09:37:07,004][20381] Updated weights for policy 1, policy_version 575166 (0.0011) [2023-12-27 09:37:07,054][20347] Updated weights for policy 0, policy_version 575449 (0.0006) [2023-12-27 09:37:07,063][20381] Updated weights for policy 1, policy_version 575176 (0.0011) [2023-12-27 09:37:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 294600704. Throughput: 0: 10503.5, 1: 10387.8. Samples: 294568444. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:07,363][19170] Avg episode reward: [(0, '768.670'), (1, '888.100')] [2023-12-27 09:37:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000575184_147267584.pth... [2023-12-27 09:37:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000575456_147333120.pth... [2023-12-27 09:37:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000574240_147021824.pth [2023-12-27 09:37:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000573968_146956288.pth [2023-12-27 09:37:07,694][20381] Updated weights for policy 1, policy_version 575186 (0.0010) [2023-12-27 09:37:07,746][20381] Updated weights for policy 1, policy_version 575196 (0.0007) [2023-12-27 09:37:07,752][20347] Updated weights for policy 0, policy_version 575459 (0.0006) [2023-12-27 09:37:07,795][20381] Updated weights for policy 1, policy_version 575206 (0.0006) [2023-12-27 09:37:07,808][20347] Updated weights for policy 0, policy_version 575469 (0.0006) [2023-12-27 09:37:07,843][20381] Updated weights for policy 1, policy_version 575216 (0.0007) [2023-12-27 09:37:07,861][20347] Updated weights for policy 0, policy_version 575479 (0.0005) [2023-12-27 09:37:08,498][20347] Updated weights for policy 0, policy_version 575489 (0.0006) [2023-12-27 09:37:08,512][20381] Updated weights for policy 1, policy_version 575226 (0.0006) [2023-12-27 09:37:08,562][20347] Updated weights for policy 0, policy_version 575499 (0.0007) [2023-12-27 09:37:08,572][20381] Updated weights for policy 1, policy_version 575236 (0.0006) [2023-12-27 09:37:08,623][20347] Updated weights for policy 0, policy_version 575509 (0.0007) [2023-12-27 09:37:08,626][20381] Updated weights for policy 1, policy_version 575246 (0.0006) [2023-12-27 09:37:08,684][20347] Updated weights for policy 0, policy_version 575519 (0.0008) [2023-12-27 09:37:09,261][20381] Updated weights for policy 1, policy_version 575256 (0.0008) [2023-12-27 09:37:09,312][20381] Updated weights for policy 1, policy_version 575266 (0.0008) [2023-12-27 09:37:09,371][20347] Updated weights for policy 0, policy_version 575529 (0.0006) [2023-12-27 09:37:09,380][20381] Updated weights for policy 1, policy_version 575276 (0.0008) [2023-12-27 09:37:09,433][20347] Updated weights for policy 0, policy_version 575539 (0.0009) [2023-12-27 09:37:09,486][20347] Updated weights for policy 0, policy_version 575549 (0.0009) [2023-12-27 09:37:10,171][20381] Updated weights for policy 1, policy_version 575286 (0.0007) [2023-12-27 09:37:10,224][20347] Updated weights for policy 0, policy_version 575559 (0.0010) [2023-12-27 09:37:10,233][20381] Updated weights for policy 1, policy_version 575296 (0.0006) [2023-12-27 09:37:10,283][20347] Updated weights for policy 0, policy_version 575569 (0.0010) [2023-12-27 09:37:10,289][20381] Updated weights for policy 1, policy_version 575306 (0.0007) [2023-12-27 09:37:10,345][20347] Updated weights for policy 0, policy_version 575579 (0.0011) [2023-12-27 09:37:10,828][20381] Updated weights for policy 1, policy_version 575316 (0.0006) [2023-12-27 09:37:10,880][20381] Updated weights for policy 1, policy_version 575326 (0.0008) [2023-12-27 09:37:10,933][20381] Updated weights for policy 1, policy_version 575336 (0.0007) [2023-12-27 09:37:11,029][20347] Updated weights for policy 0, policy_version 575589 (0.0009) [2023-12-27 09:37:11,091][20347] Updated weights for policy 0, policy_version 575599 (0.0008) [2023-12-27 09:37:11,152][20347] Updated weights for policy 0, policy_version 575609 (0.0008) [2023-12-27 09:37:11,718][20381] Updated weights for policy 1, policy_version 575346 (0.0008) [2023-12-27 09:37:11,782][20381] Updated weights for policy 1, policy_version 575356 (0.0008) [2023-12-27 09:37:11,838][20347] Updated weights for policy 0, policy_version 575619 (0.0008) [2023-12-27 09:37:11,843][20381] Updated weights for policy 1, policy_version 575366 (0.0007) [2023-12-27 09:37:11,895][20347] Updated weights for policy 0, policy_version 575629 (0.0009) [2023-12-27 09:37:11,900][20381] Updated weights for policy 1, policy_version 575376 (0.0006) [2023-12-27 09:37:11,955][20347] Updated weights for policy 0, policy_version 575639 (0.0009) [2023-12-27 09:37:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 294699008. Throughput: 0: 10384.2, 1: 10386.3. Samples: 294690728. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:12,363][19170] Avg episode reward: [(0, '742.100'), (1, '933.840')] [2023-12-27 09:37:12,538][20381] Updated weights for policy 1, policy_version 575386 (0.0009) [2023-12-27 09:37:12,596][20381] Updated weights for policy 1, policy_version 575396 (0.0009) [2023-12-27 09:37:12,635][20347] Updated weights for policy 0, policy_version 575649 (0.0007) [2023-12-27 09:37:12,655][20381] Updated weights for policy 1, policy_version 575406 (0.0007) [2023-12-27 09:37:12,683][20347] Updated weights for policy 0, policy_version 575659 (0.0010) [2023-12-27 09:37:12,741][20347] Updated weights for policy 0, policy_version 575669 (0.0010) [2023-12-27 09:37:12,804][20347] Updated weights for policy 0, policy_version 575679 (0.0010) [2023-12-27 09:37:13,296][20381] Updated weights for policy 1, policy_version 575416 (0.0008) [2023-12-27 09:37:13,346][20381] Updated weights for policy 1, policy_version 575426 (0.0011) [2023-12-27 09:37:13,399][20381] Updated weights for policy 1, policy_version 575436 (0.0011) [2023-12-27 09:37:13,490][20347] Updated weights for policy 0, policy_version 575689 (0.0010) [2023-12-27 09:37:13,550][20347] Updated weights for policy 0, policy_version 575699 (0.0010) [2023-12-27 09:37:13,602][20347] Updated weights for policy 0, policy_version 575709 (0.0010) [2023-12-27 09:37:14,049][20381] Updated weights for policy 1, policy_version 575446 (0.0011) [2023-12-27 09:37:14,105][20381] Updated weights for policy 1, policy_version 575456 (0.0011) [2023-12-27 09:37:14,155][20381] Updated weights for policy 1, policy_version 575466 (0.0010) [2023-12-27 09:37:14,229][20347] Updated weights for policy 0, policy_version 575719 (0.0007) [2023-12-27 09:37:14,289][20347] Updated weights for policy 0, policy_version 575729 (0.0006) [2023-12-27 09:37:14,342][20347] Updated weights for policy 0, policy_version 575739 (0.0005) [2023-12-27 09:37:14,766][20381] Updated weights for policy 1, policy_version 575476 (0.0010) [2023-12-27 09:37:14,823][20381] Updated weights for policy 1, policy_version 575486 (0.0011) [2023-12-27 09:37:14,873][20381] Updated weights for policy 1, policy_version 575496 (0.0011) [2023-12-27 09:37:14,987][20347] Updated weights for policy 0, policy_version 575749 (0.0006) [2023-12-27 09:37:15,045][20347] Updated weights for policy 0, policy_version 575759 (0.0008) [2023-12-27 09:37:15,103][20347] Updated weights for policy 0, policy_version 575769 (0.0008) [2023-12-27 09:37:15,526][20381] Updated weights for policy 1, policy_version 575506 (0.0010) [2023-12-27 09:37:15,578][20381] Updated weights for policy 1, policy_version 575516 (0.0008) [2023-12-27 09:37:15,633][20381] Updated weights for policy 1, policy_version 575526 (0.0011) [2023-12-27 09:37:15,703][20381] Updated weights for policy 1, policy_version 575536 (0.0011) [2023-12-27 09:37:15,740][20347] Updated weights for policy 0, policy_version 575779 (0.0006) [2023-12-27 09:37:15,791][20347] Updated weights for policy 0, policy_version 575789 (0.0005) [2023-12-27 09:37:15,849][20347] Updated weights for policy 0, policy_version 575799 (0.0005) [2023-12-27 09:37:16,312][20381] Updated weights for policy 1, policy_version 575546 (0.0008) [2023-12-27 09:37:16,371][20381] Updated weights for policy 1, policy_version 575556 (0.0011) [2023-12-27 09:37:16,427][20381] Updated weights for policy 1, policy_version 575566 (0.0011) [2023-12-27 09:37:16,440][20347] Updated weights for policy 0, policy_version 575809 (0.0006) [2023-12-27 09:37:16,498][20347] Updated weights for policy 0, policy_version 575819 (0.0010) [2023-12-27 09:37:16,568][20347] Updated weights for policy 0, policy_version 575829 (0.0010) [2023-12-27 09:37:16,626][20347] Updated weights for policy 0, policy_version 575839 (0.0008) [2023-12-27 09:37:17,053][20381] Updated weights for policy 1, policy_version 575576 (0.0010) [2023-12-27 09:37:17,120][20381] Updated weights for policy 1, policy_version 575586 (0.0011) [2023-12-27 09:37:17,184][20381] Updated weights for policy 1, policy_version 575596 (0.0011) [2023-12-27 09:37:17,253][20347] Updated weights for policy 0, policy_version 575849 (0.0007) [2023-12-27 09:37:17,312][20347] Updated weights for policy 0, policy_version 575859 (0.0007) [2023-12-27 09:37:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 294805504. Throughput: 0: 10365.5, 1: 10376.5. Samples: 294817836. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:17,363][19170] Avg episode reward: [(0, '807.320'), (1, '934.300')] [2023-12-27 09:37:17,376][20347] Updated weights for policy 0, policy_version 575869 (0.0006) [2023-12-27 09:37:17,793][20381] Updated weights for policy 1, policy_version 575606 (0.0010) [2023-12-27 09:37:17,858][20381] Updated weights for policy 1, policy_version 575616 (0.0011) [2023-12-27 09:37:17,921][20381] Updated weights for policy 1, policy_version 575626 (0.0011) [2023-12-27 09:37:17,963][20347] Updated weights for policy 0, policy_version 575879 (0.0006) [2023-12-27 09:37:18,013][20347] Updated weights for policy 0, policy_version 575889 (0.0005) [2023-12-27 09:37:18,064][20347] Updated weights for policy 0, policy_version 575899 (0.0005) [2023-12-27 09:37:18,594][20381] Updated weights for policy 1, policy_version 575636 (0.0009) [2023-12-27 09:37:18,654][20381] Updated weights for policy 1, policy_version 575646 (0.0011) [2023-12-27 09:37:18,713][20381] Updated weights for policy 1, policy_version 575656 (0.0011) [2023-12-27 09:37:18,743][20347] Updated weights for policy 0, policy_version 575909 (0.0007) [2023-12-27 09:37:18,789][20347] Updated weights for policy 0, policy_version 575919 (0.0007) [2023-12-27 09:37:18,838][20347] Updated weights for policy 0, policy_version 575929 (0.0007) [2023-12-27 09:37:19,389][20381] Updated weights for policy 1, policy_version 575666 (0.0010) [2023-12-27 09:37:19,453][20381] Updated weights for policy 1, policy_version 575676 (0.0008) [2023-12-27 09:37:19,519][20381] Updated weights for policy 1, policy_version 575686 (0.0008) [2023-12-27 09:37:19,521][20347] Updated weights for policy 0, policy_version 575939 (0.0007) [2023-12-27 09:37:19,578][20381] Updated weights for policy 1, policy_version 575696 (0.0007) [2023-12-27 09:37:19,580][20347] Updated weights for policy 0, policy_version 575949 (0.0005) [2023-12-27 09:37:19,634][20347] Updated weights for policy 0, policy_version 575959 (0.0007) [2023-12-27 09:37:20,291][20347] Updated weights for policy 0, policy_version 575969 (0.0007) [2023-12-27 09:37:20,296][20381] Updated weights for policy 1, policy_version 575706 (0.0009) [2023-12-27 09:37:20,349][20347] Updated weights for policy 0, policy_version 575979 (0.0005) [2023-12-27 09:37:20,350][20381] Updated weights for policy 1, policy_version 575716 (0.0008) [2023-12-27 09:37:20,403][20347] Updated weights for policy 0, policy_version 575989 (0.0006) [2023-12-27 09:37:20,404][20381] Updated weights for policy 1, policy_version 575726 (0.0007) [2023-12-27 09:37:20,457][20347] Updated weights for policy 0, policy_version 575999 (0.0005) [2023-12-27 09:37:21,116][20381] Updated weights for policy 1, policy_version 575736 (0.0006) [2023-12-27 09:37:21,139][20347] Updated weights for policy 0, policy_version 576009 (0.0008) [2023-12-27 09:37:21,183][20381] Updated weights for policy 1, policy_version 575746 (0.0007) [2023-12-27 09:37:21,196][20347] Updated weights for policy 0, policy_version 576019 (0.0007) [2023-12-27 09:37:21,245][20381] Updated weights for policy 1, policy_version 575756 (0.0008) [2023-12-27 09:37:21,263][20347] Updated weights for policy 0, policy_version 576029 (0.0006) [2023-12-27 09:37:21,914][20381] Updated weights for policy 1, policy_version 575766 (0.0010) [2023-12-27 09:37:21,938][20347] Updated weights for policy 0, policy_version 576039 (0.0006) [2023-12-27 09:37:21,966][20381] Updated weights for policy 1, policy_version 575776 (0.0009) [2023-12-27 09:37:21,998][20347] Updated weights for policy 0, policy_version 576049 (0.0006) [2023-12-27 09:37:22,020][20381] Updated weights for policy 1, policy_version 575786 (0.0011) [2023-12-27 09:37:22,049][20347] Updated weights for policy 0, policy_version 576059 (0.0006) [2023-12-27 09:37:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 294912000. Throughput: 0: 10346.3, 1: 10358.8. Samples: 294879892. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:22,363][19170] Avg episode reward: [(0, '811.930'), (1, '884.780')] [2023-12-27 09:37:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000575792_147423232.pth... [2023-12-27 09:37:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000576064_147488768.pth... [2023-12-27 09:37:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000574576_147111936.pth [2023-12-27 09:37:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000574848_147177472.pth [2023-12-27 09:37:22,766][20347] Updated weights for policy 0, policy_version 576069 (0.0007) [2023-12-27 09:37:22,773][20381] Updated weights for policy 1, policy_version 575796 (0.0011) [2023-12-27 09:37:22,820][20347] Updated weights for policy 0, policy_version 576079 (0.0005) [2023-12-27 09:37:22,833][20381] Updated weights for policy 1, policy_version 575806 (0.0011) [2023-12-27 09:37:22,882][20347] Updated weights for policy 0, policy_version 576089 (0.0005) [2023-12-27 09:37:22,898][20381] Updated weights for policy 1, policy_version 575816 (0.0011) [2023-12-27 09:37:23,494][20381] Updated weights for policy 1, policy_version 575826 (0.0007) [2023-12-27 09:37:23,524][20347] Updated weights for policy 0, policy_version 576099 (0.0006) [2023-12-27 09:37:23,546][20381] Updated weights for policy 1, policy_version 575836 (0.0007) [2023-12-27 09:37:23,588][20347] Updated weights for policy 0, policy_version 576109 (0.0009) [2023-12-27 09:37:23,598][20381] Updated weights for policy 1, policy_version 575846 (0.0006) [2023-12-27 09:37:23,640][20347] Updated weights for policy 0, policy_version 576119 (0.0008) [2023-12-27 09:37:23,648][20381] Updated weights for policy 1, policy_version 575856 (0.0008) [2023-12-27 09:37:24,280][20381] Updated weights for policy 1, policy_version 575866 (0.0007) [2023-12-27 09:37:24,316][20347] Updated weights for policy 0, policy_version 576129 (0.0007) [2023-12-27 09:37:24,326][20381] Updated weights for policy 1, policy_version 575876 (0.0007) [2023-12-27 09:37:24,373][20347] Updated weights for policy 0, policy_version 576139 (0.0010) [2023-12-27 09:37:24,381][20381] Updated weights for policy 1, policy_version 575886 (0.0006) [2023-12-27 09:37:24,432][20347] Updated weights for policy 0, policy_version 576149 (0.0008) [2023-12-27 09:37:24,487][20347] Updated weights for policy 0, policy_version 576159 (0.0010) [2023-12-27 09:37:25,080][20347] Updated weights for policy 0, policy_version 576169 (0.0010) [2023-12-27 09:37:25,094][20381] Updated weights for policy 1, policy_version 575896 (0.0006) [2023-12-27 09:37:25,128][20347] Updated weights for policy 0, policy_version 576179 (0.0010) [2023-12-27 09:37:25,156][20381] Updated weights for policy 1, policy_version 575906 (0.0005) [2023-12-27 09:37:25,183][20347] Updated weights for policy 0, policy_version 576189 (0.0010) [2023-12-27 09:37:25,214][20381] Updated weights for policy 1, policy_version 575916 (0.0006) [2023-12-27 09:37:25,802][20347] Updated weights for policy 0, policy_version 576199 (0.0010) [2023-12-27 09:37:25,831][20381] Updated weights for policy 1, policy_version 575926 (0.0006) [2023-12-27 09:37:25,856][20347] Updated weights for policy 0, policy_version 576209 (0.0010) [2023-12-27 09:37:25,885][20381] Updated weights for policy 1, policy_version 575936 (0.0006) [2023-12-27 09:37:25,920][20347] Updated weights for policy 0, policy_version 576219 (0.0010) [2023-12-27 09:37:25,934][20381] Updated weights for policy 1, policy_version 575946 (0.0006) [2023-12-27 09:37:26,478][20381] Updated weights for policy 1, policy_version 575956 (0.0005) [2023-12-27 09:37:26,538][20381] Updated weights for policy 1, policy_version 575966 (0.0005) [2023-12-27 09:37:26,586][20347] Updated weights for policy 0, policy_version 576229 (0.0008) [2023-12-27 09:37:26,591][20381] Updated weights for policy 1, policy_version 575976 (0.0006) [2023-12-27 09:37:26,651][20347] Updated weights for policy 0, policy_version 576239 (0.0008) [2023-12-27 09:37:26,711][20347] Updated weights for policy 0, policy_version 576249 (0.0008) [2023-12-27 09:37:27,170][20381] Updated weights for policy 1, policy_version 575986 (0.0005) [2023-12-27 09:37:27,222][20381] Updated weights for policy 1, policy_version 575996 (0.0008) [2023-12-27 09:37:27,275][20381] Updated weights for policy 1, policy_version 576006 (0.0006) [2023-12-27 09:37:27,323][20347] Updated weights for policy 0, policy_version 576259 (0.0008) [2023-12-27 09:37:27,332][20381] Updated weights for policy 1, policy_version 576016 (0.0006) [2023-12-27 09:37:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 295018496. Throughput: 0: 10333.6, 1: 10453.3. Samples: 295005472. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:27,363][19170] Avg episode reward: [(0, '783.690'), (1, '894.080')] [2023-12-27 09:37:27,376][20347] Updated weights for policy 0, policy_version 576269 (0.0010) [2023-12-27 09:37:27,435][20347] Updated weights for policy 0, policy_version 576279 (0.0010) [2023-12-27 09:37:28,005][20381] Updated weights for policy 1, policy_version 576026 (0.0007) [2023-12-27 09:37:28,065][20381] Updated weights for policy 1, policy_version 576036 (0.0005) [2023-12-27 09:37:28,127][20381] Updated weights for policy 1, policy_version 576046 (0.0005) [2023-12-27 09:37:28,129][20347] Updated weights for policy 0, policy_version 576289 (0.0010) [2023-12-27 09:37:28,173][20347] Updated weights for policy 0, policy_version 576299 (0.0005) [2023-12-27 09:37:28,221][20347] Updated weights for policy 0, policy_version 576309 (0.0005) [2023-12-27 09:37:28,271][20347] Updated weights for policy 0, policy_version 576319 (0.0005) [2023-12-27 09:37:28,711][20381] Updated weights for policy 1, policy_version 576056 (0.0006) [2023-12-27 09:37:28,759][20381] Updated weights for policy 1, policy_version 576066 (0.0005) [2023-12-27 09:37:28,805][20381] Updated weights for policy 1, policy_version 576076 (0.0005) [2023-12-27 09:37:28,888][20347] Updated weights for policy 0, policy_version 576329 (0.0006) [2023-12-27 09:37:28,937][20347] Updated weights for policy 0, policy_version 576339 (0.0010) [2023-12-27 09:37:28,983][20347] Updated weights for policy 0, policy_version 576349 (0.0010) [2023-12-27 09:37:29,477][20381] Updated weights for policy 1, policy_version 576086 (0.0007) [2023-12-27 09:37:29,535][20381] Updated weights for policy 1, policy_version 576096 (0.0007) [2023-12-27 09:37:29,593][20381] Updated weights for policy 1, policy_version 576106 (0.0007) [2023-12-27 09:37:29,678][20347] Updated weights for policy 0, policy_version 576359 (0.0010) [2023-12-27 09:37:29,737][20347] Updated weights for policy 0, policy_version 576370 (0.0009) [2023-12-27 09:37:29,801][20347] Updated weights for policy 0, policy_version 576380 (0.0008) [2023-12-27 09:37:30,213][20381] Updated weights for policy 1, policy_version 576116 (0.0009) [2023-12-27 09:37:30,262][20381] Updated weights for policy 1, policy_version 576126 (0.0011) [2023-12-27 09:37:30,325][20381] Updated weights for policy 1, policy_version 576136 (0.0010) [2023-12-27 09:37:30,456][20347] Updated weights for policy 0, policy_version 576390 (0.0007) [2023-12-27 09:37:30,505][20347] Updated weights for policy 0, policy_version 576400 (0.0007) [2023-12-27 09:37:30,560][20347] Updated weights for policy 0, policy_version 576410 (0.0007) [2023-12-27 09:37:30,882][20381] Updated weights for policy 1, policy_version 576146 (0.0006) [2023-12-27 09:37:30,927][20381] Updated weights for policy 1, policy_version 576156 (0.0010) [2023-12-27 09:37:30,977][20381] Updated weights for policy 1, policy_version 576166 (0.0010) [2023-12-27 09:37:31,038][20381] Updated weights for policy 1, policy_version 576176 (0.0011) [2023-12-27 09:37:31,199][20347] Updated weights for policy 0, policy_version 576420 (0.0008) [2023-12-27 09:37:31,256][20347] Updated weights for policy 0, policy_version 576430 (0.0007) [2023-12-27 09:37:31,310][20347] Updated weights for policy 0, policy_version 576440 (0.0005) [2023-12-27 09:37:31,672][20381] Updated weights for policy 1, policy_version 576186 (0.0008) [2023-12-27 09:37:31,733][20381] Updated weights for policy 1, policy_version 576196 (0.0007) [2023-12-27 09:37:31,786][20381] Updated weights for policy 1, policy_version 576206 (0.0008) [2023-12-27 09:37:31,980][20347] Updated weights for policy 0, policy_version 576450 (0.0007) [2023-12-27 09:37:32,030][20347] Updated weights for policy 0, policy_version 576460 (0.0008) [2023-12-27 09:37:32,089][20347] Updated weights for policy 0, policy_version 576470 (0.0007) [2023-12-27 09:37:32,143][20347] Updated weights for policy 0, policy_version 576480 (0.0007) [2023-12-27 09:37:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 295124992. Throughput: 0: 10322.6, 1: 10531.4. Samples: 295135532. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:32,363][19170] Avg episode reward: [(0, '786.530'), (1, '903.560')] [2023-12-27 09:37:32,457][20381] Updated weights for policy 1, policy_version 576216 (0.0008) [2023-12-27 09:37:32,507][20381] Updated weights for policy 1, policy_version 576226 (0.0007) [2023-12-27 09:37:32,557][20381] Updated weights for policy 1, policy_version 576236 (0.0005) [2023-12-27 09:37:32,875][20347] Updated weights for policy 0, policy_version 576490 (0.0005) [2023-12-27 09:37:32,929][20347] Updated weights for policy 0, policy_version 576500 (0.0007) [2023-12-27 09:37:32,981][20347] Updated weights for policy 0, policy_version 576510 (0.0010) [2023-12-27 09:37:33,238][20381] Updated weights for policy 1, policy_version 576246 (0.0007) [2023-12-27 09:37:33,306][20381] Updated weights for policy 1, policy_version 576256 (0.0009) [2023-12-27 09:37:33,367][20381] Updated weights for policy 1, policy_version 576266 (0.0007) [2023-12-27 09:37:33,648][20347] Updated weights for policy 0, policy_version 576520 (0.0010) [2023-12-27 09:37:33,696][20347] Updated weights for policy 0, policy_version 576530 (0.0009) [2023-12-27 09:37:33,757][20347] Updated weights for policy 0, policy_version 576540 (0.0008) [2023-12-27 09:37:34,094][20381] Updated weights for policy 1, policy_version 576276 (0.0008) [2023-12-27 09:37:34,149][20381] Updated weights for policy 1, policy_version 576286 (0.0008) [2023-12-27 09:37:34,208][20381] Updated weights for policy 1, policy_version 576296 (0.0008) [2023-12-27 09:37:34,402][20347] Updated weights for policy 0, policy_version 576550 (0.0008) [2023-12-27 09:37:34,455][20347] Updated weights for policy 0, policy_version 576560 (0.0009) [2023-12-27 09:37:34,522][20347] Updated weights for policy 0, policy_version 576570 (0.0006) [2023-12-27 09:37:34,902][20381] Updated weights for policy 1, policy_version 576306 (0.0008) [2023-12-27 09:37:34,959][20381] Updated weights for policy 1, policy_version 576316 (0.0007) [2023-12-27 09:37:35,014][20381] Updated weights for policy 1, policy_version 576326 (0.0006) [2023-12-27 09:37:35,072][20381] Updated weights for policy 1, policy_version 576336 (0.0005) [2023-12-27 09:37:35,166][20347] Updated weights for policy 0, policy_version 576580 (0.0011) [2023-12-27 09:37:35,226][20347] Updated weights for policy 0, policy_version 576590 (0.0010) [2023-12-27 09:37:35,284][20347] Updated weights for policy 0, policy_version 576600 (0.0010) [2023-12-27 09:37:35,626][20381] Updated weights for policy 1, policy_version 576346 (0.0007) [2023-12-27 09:37:35,677][20381] Updated weights for policy 1, policy_version 576356 (0.0007) [2023-12-27 09:37:35,725][20381] Updated weights for policy 1, policy_version 576366 (0.0007) [2023-12-27 09:37:35,950][20347] Updated weights for policy 0, policy_version 576610 (0.0010) [2023-12-27 09:37:36,004][20347] Updated weights for policy 0, policy_version 576620 (0.0010) [2023-12-27 09:37:36,065][20347] Updated weights for policy 0, policy_version 576630 (0.0010) [2023-12-27 09:37:36,130][20347] Updated weights for policy 0, policy_version 576640 (0.0010) [2023-12-27 09:37:36,427][20381] Updated weights for policy 1, policy_version 576376 (0.0007) [2023-12-27 09:37:36,492][20381] Updated weights for policy 1, policy_version 576386 (0.0006) [2023-12-27 09:37:36,553][20381] Updated weights for policy 1, policy_version 576396 (0.0008) [2023-12-27 09:37:36,785][20347] Updated weights for policy 0, policy_version 576650 (0.0005) [2023-12-27 09:37:36,848][20347] Updated weights for policy 0, policy_version 576660 (0.0005) [2023-12-27 09:37:36,915][20347] Updated weights for policy 0, policy_version 576670 (0.0007) [2023-12-27 09:37:37,110][20381] Updated weights for policy 1, policy_version 576406 (0.0007) [2023-12-27 09:37:37,160][20381] Updated weights for policy 1, policy_version 576416 (0.0009) [2023-12-27 09:37:37,209][20381] Updated weights for policy 1, policy_version 576426 (0.0009) [2023-12-27 09:37:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 295231488. Throughput: 0: 10354.5, 1: 10495.5. Samples: 295197416. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:37,363][19170] Avg episode reward: [(0, '831.700'), (1, '971.060')] [2023-12-27 09:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000576432_147587072.pth... [2023-12-27 09:37:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000576672_147644416.pth... [2023-12-27 09:37:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000575184_147267584.pth [2023-12-27 09:37:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000575456_147333120.pth [2023-12-27 09:37:37,491][20347] Updated weights for policy 0, policy_version 576680 (0.0008) [2023-12-27 09:37:37,545][20347] Updated weights for policy 0, policy_version 576690 (0.0008) [2023-12-27 09:37:37,593][20347] Updated weights for policy 0, policy_version 576700 (0.0008) [2023-12-27 09:37:37,860][20381] Updated weights for policy 1, policy_version 576436 (0.0009) [2023-12-27 09:37:37,916][20381] Updated weights for policy 1, policy_version 576446 (0.0007) [2023-12-27 09:37:37,966][20381] Updated weights for policy 1, policy_version 576456 (0.0007) [2023-12-27 09:37:38,229][20347] Updated weights for policy 0, policy_version 576710 (0.0009) [2023-12-27 09:37:38,290][20347] Updated weights for policy 0, policy_version 576720 (0.0010) [2023-12-27 09:37:38,357][20347] Updated weights for policy 0, policy_version 576730 (0.0009) [2023-12-27 09:37:38,676][20381] Updated weights for policy 1, policy_version 576466 (0.0008) [2023-12-27 09:37:38,733][20381] Updated weights for policy 1, policy_version 576476 (0.0010) [2023-12-27 09:37:38,788][20381] Updated weights for policy 1, policy_version 576486 (0.0010) [2023-12-27 09:37:38,841][20381] Updated weights for policy 1, policy_version 576496 (0.0007) [2023-12-27 09:37:39,013][20347] Updated weights for policy 0, policy_version 576740 (0.0010) [2023-12-27 09:37:39,059][20347] Updated weights for policy 0, policy_version 576750 (0.0009) [2023-12-27 09:37:39,106][20347] Updated weights for policy 0, policy_version 576760 (0.0009) [2023-12-27 09:37:39,558][20381] Updated weights for policy 1, policy_version 576506 (0.0007) [2023-12-27 09:37:39,618][20381] Updated weights for policy 1, policy_version 576516 (0.0006) [2023-12-27 09:37:39,683][20381] Updated weights for policy 1, policy_version 576526 (0.0007) [2023-12-27 09:37:39,912][20347] Updated weights for policy 0, policy_version 576770 (0.0010) [2023-12-27 09:37:39,969][20347] Updated weights for policy 0, policy_version 576780 (0.0008) [2023-12-27 09:37:40,037][20347] Updated weights for policy 0, policy_version 576790 (0.0008) [2023-12-27 09:37:40,096][20347] Updated weights for policy 0, policy_version 576800 (0.0008) [2023-12-27 09:37:40,284][20381] Updated weights for policy 1, policy_version 576536 (0.0008) [2023-12-27 09:37:40,335][20381] Updated weights for policy 1, policy_version 576546 (0.0008) [2023-12-27 09:37:40,389][20381] Updated weights for policy 1, policy_version 576556 (0.0008) [2023-12-27 09:37:40,827][20347] Updated weights for policy 0, policy_version 576810 (0.0008) [2023-12-27 09:37:40,888][20347] Updated weights for policy 0, policy_version 576820 (0.0007) [2023-12-27 09:37:40,932][20347] Updated weights for policy 0, policy_version 576830 (0.0008) [2023-12-27 09:37:40,998][20381] Updated weights for policy 1, policy_version 576566 (0.0006) [2023-12-27 09:37:41,064][20381] Updated weights for policy 1, policy_version 576576 (0.0008) [2023-12-27 09:37:41,124][20381] Updated weights for policy 1, policy_version 576586 (0.0006) [2023-12-27 09:37:41,631][20347] Updated weights for policy 0, policy_version 576840 (0.0007) [2023-12-27 09:37:41,686][20347] Updated weights for policy 0, policy_version 576850 (0.0007) [2023-12-27 09:37:41,749][20347] Updated weights for policy 0, policy_version 576860 (0.0007) [2023-12-27 09:37:41,856][20381] Updated weights for policy 1, policy_version 576596 (0.0007) [2023-12-27 09:37:41,918][20381] Updated weights for policy 1, policy_version 576606 (0.0005) [2023-12-27 09:37:41,980][20381] Updated weights for policy 1, policy_version 576616 (0.0006) [2023-12-27 09:37:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 295329792. Throughput: 0: 10284.0, 1: 10535.2. Samples: 295322788. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:42,363][19170] Avg episode reward: [(0, '865.850'), (1, '990.410')] [2023-12-27 09:37:42,384][20347] Updated weights for policy 0, policy_version 576870 (0.0006) [2023-12-27 09:37:42,448][20347] Updated weights for policy 0, policy_version 576880 (0.0006) [2023-12-27 09:37:42,502][20347] Updated weights for policy 0, policy_version 576890 (0.0007) [2023-12-27 09:37:42,652][20381] Updated weights for policy 1, policy_version 576626 (0.0008) [2023-12-27 09:37:42,709][20381] Updated weights for policy 1, policy_version 576636 (0.0008) [2023-12-27 09:37:42,762][20381] Updated weights for policy 1, policy_version 576646 (0.0008) [2023-12-27 09:37:42,815][20381] Updated weights for policy 1, policy_version 576656 (0.0007) [2023-12-27 09:37:43,223][20347] Updated weights for policy 0, policy_version 576900 (0.0009) [2023-12-27 09:37:43,278][20347] Updated weights for policy 0, policy_version 576910 (0.0008) [2023-12-27 09:37:43,332][20347] Updated weights for policy 0, policy_version 576920 (0.0008) [2023-12-27 09:37:43,414][20381] Updated weights for policy 1, policy_version 576666 (0.0007) [2023-12-27 09:37:43,470][20381] Updated weights for policy 1, policy_version 576676 (0.0008) [2023-12-27 09:37:43,530][20381] Updated weights for policy 1, policy_version 576686 (0.0008) [2023-12-27 09:37:44,038][20347] Updated weights for policy 0, policy_version 576930 (0.0007) [2023-12-27 09:37:44,090][20347] Updated weights for policy 0, policy_version 576940 (0.0007) [2023-12-27 09:37:44,142][20347] Updated weights for policy 0, policy_version 576950 (0.0007) [2023-12-27 09:37:44,198][20347] Updated weights for policy 0, policy_version 576960 (0.0008) [2023-12-27 09:37:44,206][20381] Updated weights for policy 1, policy_version 576696 (0.0006) [2023-12-27 09:37:44,258][20381] Updated weights for policy 1, policy_version 576706 (0.0008) [2023-12-27 09:37:44,310][20381] Updated weights for policy 1, policy_version 576716 (0.0008) [2023-12-27 09:37:44,907][20347] Updated weights for policy 0, policy_version 576970 (0.0008) [2023-12-27 09:37:44,966][20347] Updated weights for policy 0, policy_version 576980 (0.0007) [2023-12-27 09:37:44,988][20381] Updated weights for policy 1, policy_version 576726 (0.0005) [2023-12-27 09:37:45,031][20347] Updated weights for policy 0, policy_version 576990 (0.0009) [2023-12-27 09:37:45,042][20381] Updated weights for policy 1, policy_version 576736 (0.0006) [2023-12-27 09:37:45,098][20381] Updated weights for policy 1, policy_version 576746 (0.0008) [2023-12-27 09:37:45,708][20347] Updated weights for policy 0, policy_version 577000 (0.0006) [2023-12-27 09:37:45,775][20381] Updated weights for policy 1, policy_version 576756 (0.0009) [2023-12-27 09:37:45,779][20347] Updated weights for policy 0, policy_version 577010 (0.0005) [2023-12-27 09:37:45,825][20381] Updated weights for policy 1, policy_version 576766 (0.0011) [2023-12-27 09:37:45,847][20347] Updated weights for policy 0, policy_version 577020 (0.0005) [2023-12-27 09:37:45,866][20381] Updated weights for policy 1, policy_version 576776 (0.0006) [2023-12-27 09:37:46,389][20347] Updated weights for policy 0, policy_version 577030 (0.0006) [2023-12-27 09:37:46,443][20347] Updated weights for policy 0, policy_version 577040 (0.0006) [2023-12-27 09:37:46,500][20347] Updated weights for policy 0, policy_version 577050 (0.0005) [2023-12-27 09:37:46,525][20381] Updated weights for policy 1, policy_version 576786 (0.0010) [2023-12-27 09:37:46,588][20381] Updated weights for policy 1, policy_version 576796 (0.0008) [2023-12-27 09:37:46,646][20381] Updated weights for policy 1, policy_version 576806 (0.0006) [2023-12-27 09:37:46,698][20381] Updated weights for policy 1, policy_version 576816 (0.0005) [2023-12-27 09:37:47,157][20347] Updated weights for policy 0, policy_version 577060 (0.0008) [2023-12-27 09:37:47,217][20347] Updated weights for policy 0, policy_version 577070 (0.0009) [2023-12-27 09:37:47,262][20381] Updated weights for policy 1, policy_version 576826 (0.0005) [2023-12-27 09:37:47,270][20347] Updated weights for policy 0, policy_version 577080 (0.0007) [2023-12-27 09:37:47,321][20381] Updated weights for policy 1, policy_version 576836 (0.0005) [2023-12-27 09:37:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20716.1). Total num frames: 295436288. Throughput: 0: 10329.5, 1: 10582.0. Samples: 295448188. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:47,363][19170] Avg episode reward: [(0, '903.310'), (1, '980.760')] [2023-12-27 09:37:47,378][20381] Updated weights for policy 1, policy_version 576846 (0.0005) [2023-12-27 09:37:47,874][20347] Updated weights for policy 0, policy_version 577090 (0.0007) [2023-12-27 09:37:47,929][20347] Updated weights for policy 0, policy_version 577100 (0.0006) [2023-12-27 09:37:47,989][20347] Updated weights for policy 0, policy_version 577110 (0.0006) [2023-12-27 09:37:48,040][20381] Updated weights for policy 1, policy_version 576856 (0.0007) [2023-12-27 09:37:48,050][20347] Updated weights for policy 0, policy_version 577120 (0.0006) [2023-12-27 09:37:48,100][20381] Updated weights for policy 1, policy_version 576866 (0.0007) [2023-12-27 09:37:48,164][20381] Updated weights for policy 1, policy_version 576876 (0.0007) [2023-12-27 09:37:48,706][20347] Updated weights for policy 0, policy_version 577130 (0.0005) [2023-12-27 09:37:48,756][20347] Updated weights for policy 0, policy_version 577140 (0.0005) [2023-12-27 09:37:48,809][20347] Updated weights for policy 0, policy_version 577150 (0.0005) [2023-12-27 09:37:48,870][20381] Updated weights for policy 1, policy_version 576886 (0.0010) [2023-12-27 09:37:48,930][20381] Updated weights for policy 1, policy_version 576896 (0.0011) [2023-12-27 09:37:48,976][20381] Updated weights for policy 1, policy_version 576906 (0.0009) [2023-12-27 09:37:49,381][20347] Updated weights for policy 0, policy_version 577160 (0.0007) [2023-12-27 09:37:49,442][20347] Updated weights for policy 0, policy_version 577170 (0.0006) [2023-12-27 09:37:49,504][20347] Updated weights for policy 0, policy_version 577180 (0.0008) [2023-12-27 09:37:49,663][20381] Updated weights for policy 1, policy_version 576916 (0.0010) [2023-12-27 09:37:49,718][20381] Updated weights for policy 1, policy_version 576926 (0.0011) [2023-12-27 09:37:49,770][20381] Updated weights for policy 1, policy_version 576936 (0.0011) [2023-12-27 09:37:50,117][20347] Updated weights for policy 0, policy_version 577190 (0.0006) [2023-12-27 09:37:50,175][20347] Updated weights for policy 0, policy_version 577200 (0.0006) [2023-12-27 09:37:50,236][20347] Updated weights for policy 0, policy_version 577210 (0.0006) [2023-12-27 09:37:50,445][20381] Updated weights for policy 1, policy_version 576946 (0.0010) [2023-12-27 09:37:50,493][20381] Updated weights for policy 1, policy_version 576956 (0.0010) [2023-12-27 09:37:50,541][20381] Updated weights for policy 1, policy_version 576966 (0.0010) [2023-12-27 09:37:50,597][20381] Updated weights for policy 1, policy_version 576976 (0.0009) [2023-12-27 09:37:50,850][20347] Updated weights for policy 0, policy_version 577220 (0.0007) [2023-12-27 09:37:50,907][20347] Updated weights for policy 0, policy_version 577230 (0.0007) [2023-12-27 09:37:50,970][20347] Updated weights for policy 0, policy_version 577240 (0.0007) [2023-12-27 09:37:51,240][20381] Updated weights for policy 1, policy_version 576986 (0.0008) [2023-12-27 09:37:51,302][20381] Updated weights for policy 1, policy_version 576996 (0.0008) [2023-12-27 09:37:51,368][20381] Updated weights for policy 1, policy_version 577006 (0.0007) [2023-12-27 09:37:51,653][20347] Updated weights for policy 0, policy_version 577250 (0.0007) [2023-12-27 09:37:51,712][20347] Updated weights for policy 0, policy_version 577260 (0.0006) [2023-12-27 09:37:51,776][20347] Updated weights for policy 0, policy_version 577270 (0.0008) [2023-12-27 09:37:51,834][20347] Updated weights for policy 0, policy_version 577280 (0.0010) [2023-12-27 09:37:52,094][20381] Updated weights for policy 1, policy_version 577016 (0.0007) [2023-12-27 09:37:52,151][20381] Updated weights for policy 1, policy_version 577026 (0.0008) [2023-12-27 09:37:52,211][20381] Updated weights for policy 1, policy_version 577036 (0.0006) [2023-12-27 09:37:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 295542784. Throughput: 0: 10410.7, 1: 10545.6. Samples: 295511480. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:52,363][19170] Avg episode reward: [(0, '867.390'), (1, '980.870')] [2023-12-27 09:37:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000577040_147742720.pth... [2023-12-27 09:37:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000577280_147800064.pth... [2023-12-27 09:37:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000575792_147423232.pth [2023-12-27 09:37:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000576064_147488768.pth [2023-12-27 09:37:52,536][20347] Updated weights for policy 0, policy_version 577290 (0.0011) [2023-12-27 09:37:52,596][20347] Updated weights for policy 0, policy_version 577300 (0.0011) [2023-12-27 09:37:52,644][20347] Updated weights for policy 0, policy_version 577310 (0.0010) [2023-12-27 09:37:52,892][20381] Updated weights for policy 1, policy_version 577046 (0.0005) [2023-12-27 09:37:52,946][20381] Updated weights for policy 1, policy_version 577056 (0.0006) [2023-12-27 09:37:53,002][20381] Updated weights for policy 1, policy_version 577066 (0.0005) [2023-12-27 09:37:53,295][20347] Updated weights for policy 0, policy_version 577320 (0.0009) [2023-12-27 09:37:53,355][20347] Updated weights for policy 0, policy_version 577330 (0.0009) [2023-12-27 09:37:53,416][20347] Updated weights for policy 0, policy_version 577340 (0.0006) [2023-12-27 09:37:53,547][20381] Updated weights for policy 1, policy_version 577076 (0.0006) [2023-12-27 09:37:53,612][20381] Updated weights for policy 1, policy_version 577086 (0.0005) [2023-12-27 09:37:53,668][20381] Updated weights for policy 1, policy_version 577096 (0.0005) [2023-12-27 09:37:54,045][20347] Updated weights for policy 0, policy_version 577350 (0.0006) [2023-12-27 09:37:54,090][20347] Updated weights for policy 0, policy_version 577360 (0.0008) [2023-12-27 09:37:54,145][20347] Updated weights for policy 0, policy_version 577370 (0.0007) [2023-12-27 09:37:54,307][20381] Updated weights for policy 1, policy_version 577106 (0.0006) [2023-12-27 09:37:54,356][20381] Updated weights for policy 1, policy_version 577116 (0.0008) [2023-12-27 09:37:54,411][20381] Updated weights for policy 1, policy_version 577126 (0.0008) [2023-12-27 09:37:54,476][20381] Updated weights for policy 1, policy_version 577136 (0.0008) [2023-12-27 09:37:54,868][20347] Updated weights for policy 0, policy_version 577380 (0.0009) [2023-12-27 09:37:54,916][20347] Updated weights for policy 0, policy_version 577390 (0.0009) [2023-12-27 09:37:54,968][20347] Updated weights for policy 0, policy_version 577400 (0.0007) [2023-12-27 09:37:55,114][20381] Updated weights for policy 1, policy_version 577146 (0.0005) [2023-12-27 09:37:55,168][20381] Updated weights for policy 1, policy_version 577156 (0.0005) [2023-12-27 09:37:55,222][20381] Updated weights for policy 1, policy_version 577166 (0.0008) [2023-12-27 09:37:55,625][20347] Updated weights for policy 0, policy_version 577410 (0.0006) [2023-12-27 09:37:55,678][20347] Updated weights for policy 0, policy_version 577420 (0.0007) [2023-12-27 09:37:55,742][20347] Updated weights for policy 0, policy_version 577430 (0.0007) [2023-12-27 09:37:55,792][20347] Updated weights for policy 0, policy_version 577440 (0.0007) [2023-12-27 09:37:55,807][20381] Updated weights for policy 1, policy_version 577176 (0.0007) [2023-12-27 09:37:55,865][20381] Updated weights for policy 1, policy_version 577186 (0.0005) [2023-12-27 09:37:55,929][20381] Updated weights for policy 1, policy_version 577196 (0.0005) [2023-12-27 09:37:56,437][20347] Updated weights for policy 0, policy_version 577450 (0.0010) [2023-12-27 09:37:56,495][20347] Updated weights for policy 0, policy_version 577460 (0.0010) [2023-12-27 09:37:56,502][20381] Updated weights for policy 1, policy_version 577206 (0.0005) [2023-12-27 09:37:56,552][20347] Updated weights for policy 0, policy_version 577470 (0.0010) [2023-12-27 09:37:56,555][20381] Updated weights for policy 1, policy_version 577216 (0.0005) [2023-12-27 09:37:56,605][20381] Updated weights for policy 1, policy_version 577226 (0.0005) [2023-12-27 09:37:57,166][20347] Updated weights for policy 0, policy_version 577480 (0.0010) [2023-12-27 09:37:57,198][20381] Updated weights for policy 1, policy_version 577236 (0.0007) [2023-12-27 09:37:57,211][20347] Updated weights for policy 0, policy_version 577490 (0.0010) [2023-12-27 09:37:57,250][20381] Updated weights for policy 1, policy_version 577246 (0.0005) [2023-12-27 09:37:57,259][20347] Updated weights for policy 0, policy_version 577500 (0.0010) [2023-12-27 09:37:57,302][20381] Updated weights for policy 1, policy_version 577256 (0.0006) [2023-12-27 09:37:57,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 295657472. Throughput: 0: 10487.7, 1: 10588.1. Samples: 295639136. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:37:57,363][19170] Avg episode reward: [(0, '876.210'), (1, '961.530')] [2023-12-27 09:37:57,845][20347] Updated weights for policy 0, policy_version 577510 (0.0007) [2023-12-27 09:37:57,902][20347] Updated weights for policy 0, policy_version 577520 (0.0005) [2023-12-27 09:37:57,953][20347] Updated weights for policy 0, policy_version 577530 (0.0005) [2023-12-27 09:37:57,994][20381] Updated weights for policy 1, policy_version 577266 (0.0007) [2023-12-27 09:37:58,044][20381] Updated weights for policy 1, policy_version 577276 (0.0008) [2023-12-27 09:37:58,093][20381] Updated weights for policy 1, policy_version 577286 (0.0009) [2023-12-27 09:37:58,147][20381] Updated weights for policy 1, policy_version 577296 (0.0007) [2023-12-27 09:37:58,562][20347] Updated weights for policy 0, policy_version 577540 (0.0007) [2023-12-27 09:37:58,624][20347] Updated weights for policy 0, policy_version 577550 (0.0010) [2023-12-27 09:37:58,692][20347] Updated weights for policy 0, policy_version 577560 (0.0011) [2023-12-27 09:37:58,929][20381] Updated weights for policy 1, policy_version 577306 (0.0009) [2023-12-27 09:37:58,986][20381] Updated weights for policy 1, policy_version 577316 (0.0007) [2023-12-27 09:37:59,037][20381] Updated weights for policy 1, policy_version 577326 (0.0006) [2023-12-27 09:37:59,453][20347] Updated weights for policy 0, policy_version 577570 (0.0010) [2023-12-27 09:37:59,505][20347] Updated weights for policy 0, policy_version 577580 (0.0007) [2023-12-27 09:37:59,551][20347] Updated weights for policy 0, policy_version 577590 (0.0007) [2023-12-27 09:37:59,599][20347] Updated weights for policy 0, policy_version 577600 (0.0007) [2023-12-27 09:37:59,776][20381] Updated weights for policy 1, policy_version 577336 (0.0008) [2023-12-27 09:37:59,821][20381] Updated weights for policy 1, policy_version 577346 (0.0008) [2023-12-27 09:37:59,874][20381] Updated weights for policy 1, policy_version 577356 (0.0008) [2023-12-27 09:38:00,284][20347] Updated weights for policy 0, policy_version 577610 (0.0010) [2023-12-27 09:38:00,344][20347] Updated weights for policy 0, policy_version 577620 (0.0010) [2023-12-27 09:38:00,399][20347] Updated weights for policy 0, policy_version 577630 (0.0010) [2023-12-27 09:38:00,630][20381] Updated weights for policy 1, policy_version 577366 (0.0006) [2023-12-27 09:38:00,692][20381] Updated weights for policy 1, policy_version 577376 (0.0006) [2023-12-27 09:38:00,758][20381] Updated weights for policy 1, policy_version 577386 (0.0005) [2023-12-27 09:38:00,963][20347] Updated weights for policy 0, policy_version 577640 (0.0008) [2023-12-27 09:38:01,007][20347] Updated weights for policy 0, policy_version 577650 (0.0008) [2023-12-27 09:38:01,067][20347] Updated weights for policy 0, policy_version 577660 (0.0008) [2023-12-27 09:38:01,307][20381] Updated weights for policy 1, policy_version 577396 (0.0006) [2023-12-27 09:38:01,364][20381] Updated weights for policy 1, policy_version 577406 (0.0007) [2023-12-27 09:38:01,417][20381] Updated weights for policy 1, policy_version 577416 (0.0005) [2023-12-27 09:38:01,820][20347] Updated weights for policy 0, policy_version 577670 (0.0007) [2023-12-27 09:38:01,875][20347] Updated weights for policy 0, policy_version 577680 (0.0006) [2023-12-27 09:38:01,938][20347] Updated weights for policy 0, policy_version 577690 (0.0008) [2023-12-27 09:38:02,029][20381] Updated weights for policy 1, policy_version 577426 (0.0006) [2023-12-27 09:38:02,088][20381] Updated weights for policy 1, policy_version 577436 (0.0008) [2023-12-27 09:38:02,146][20381] Updated weights for policy 1, policy_version 577446 (0.0006) [2023-12-27 09:38:02,205][20381] Updated weights for policy 1, policy_version 577456 (0.0006) [2023-12-27 09:38:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 295755776. Throughput: 0: 10486.8, 1: 10560.4. Samples: 295764956. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:38:02,363][19170] Avg episode reward: [(0, '807.960'), (1, '952.410')] [2023-12-27 09:38:02,628][20347] Updated weights for policy 0, policy_version 577700 (0.0007) [2023-12-27 09:38:02,686][20347] Updated weights for policy 0, policy_version 577710 (0.0009) [2023-12-27 09:38:02,750][20347] Updated weights for policy 0, policy_version 577720 (0.0008) [2023-12-27 09:38:02,801][20381] Updated weights for policy 1, policy_version 577466 (0.0010) [2023-12-27 09:38:02,858][20381] Updated weights for policy 1, policy_version 577476 (0.0007) [2023-12-27 09:38:02,909][20381] Updated weights for policy 1, policy_version 577486 (0.0007) [2023-12-27 09:38:03,373][20347] Updated weights for policy 0, policy_version 577730 (0.0009) [2023-12-27 09:38:03,436][20347] Updated weights for policy 0, policy_version 577740 (0.0007) [2023-12-27 09:38:03,494][20347] Updated weights for policy 0, policy_version 577750 (0.0007) [2023-12-27 09:38:03,527][20381] Updated weights for policy 1, policy_version 577496 (0.0006) [2023-12-27 09:38:03,547][20347] Updated weights for policy 0, policy_version 577760 (0.0007) [2023-12-27 09:38:03,576][20381] Updated weights for policy 1, policy_version 577506 (0.0008) [2023-12-27 09:38:03,621][20381] Updated weights for policy 1, policy_version 577516 (0.0008) [2023-12-27 09:38:04,252][20347] Updated weights for policy 0, policy_version 577770 (0.0008) [2023-12-27 09:38:04,315][20347] Updated weights for policy 0, policy_version 577780 (0.0007) [2023-12-27 09:38:04,325][20381] Updated weights for policy 1, policy_version 577526 (0.0008) [2023-12-27 09:38:04,371][20347] Updated weights for policy 0, policy_version 577790 (0.0006) [2023-12-27 09:38:04,380][20381] Updated weights for policy 1, policy_version 577536 (0.0006) [2023-12-27 09:38:04,432][20381] Updated weights for policy 1, policy_version 577546 (0.0008) [2023-12-27 09:38:05,082][20347] Updated weights for policy 0, policy_version 577800 (0.0007) [2023-12-27 09:38:05,129][20347] Updated weights for policy 0, policy_version 577810 (0.0007) [2023-12-27 09:38:05,146][20381] Updated weights for policy 1, policy_version 577556 (0.0011) [2023-12-27 09:38:05,189][20347] Updated weights for policy 0, policy_version 577820 (0.0005) [2023-12-27 09:38:05,199][20381] Updated weights for policy 1, policy_version 577566 (0.0010) [2023-12-27 09:38:05,251][20381] Updated weights for policy 1, policy_version 577576 (0.0010) [2023-12-27 09:38:05,894][20347] Updated weights for policy 0, policy_version 577830 (0.0006) [2023-12-27 09:38:05,900][20381] Updated weights for policy 1, policy_version 577586 (0.0008) [2023-12-27 09:38:05,958][20381] Updated weights for policy 1, policy_version 577596 (0.0008) [2023-12-27 09:38:05,959][20347] Updated weights for policy 0, policy_version 577840 (0.0008) [2023-12-27 09:38:06,013][20381] Updated weights for policy 1, policy_version 577606 (0.0007) [2023-12-27 09:38:06,026][20347] Updated weights for policy 0, policy_version 577850 (0.0006) [2023-12-27 09:38:06,067][20381] Updated weights for policy 1, policy_version 577616 (0.0008) [2023-12-27 09:38:06,669][20347] Updated weights for policy 0, policy_version 577860 (0.0007) [2023-12-27 09:38:06,726][20347] Updated weights for policy 0, policy_version 577870 (0.0008) [2023-12-27 09:38:06,755][20381] Updated weights for policy 1, policy_version 577626 (0.0011) [2023-12-27 09:38:06,787][20347] Updated weights for policy 0, policy_version 577880 (0.0006) [2023-12-27 09:38:06,814][20381] Updated weights for policy 1, policy_version 577636 (0.0011) [2023-12-27 09:38:06,875][20381] Updated weights for policy 1, policy_version 577646 (0.0011) [2023-12-27 09:38:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 295854080. Throughput: 0: 10465.2, 1: 10582.8. Samples: 295827052. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:38:07,363][19170] Avg episode reward: [(0, '837.300'), (1, '952.520')] [2023-12-27 09:38:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000577888_147955712.pth... [2023-12-27 09:38:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000577648_147898368.pth... [2023-12-27 09:38:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000576672_147644416.pth [2023-12-27 09:38:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000576432_147587072.pth [2023-12-27 09:38:07,472][20347] Updated weights for policy 0, policy_version 577890 (0.0005) [2023-12-27 09:38:07,517][20381] Updated weights for policy 1, policy_version 577656 (0.0010) [2023-12-27 09:38:07,526][20347] Updated weights for policy 0, policy_version 577900 (0.0006) [2023-12-27 09:38:07,572][20381] Updated weights for policy 1, policy_version 577666 (0.0010) [2023-12-27 09:38:07,581][20347] Updated weights for policy 0, policy_version 577910 (0.0005) [2023-12-27 09:38:07,628][20381] Updated weights for policy 1, policy_version 577676 (0.0010) [2023-12-27 09:38:07,636][20347] Updated weights for policy 0, policy_version 577920 (0.0005) [2023-12-27 09:38:08,203][20381] Updated weights for policy 1, policy_version 577686 (0.0007) [2023-12-27 09:38:08,257][20381] Updated weights for policy 1, policy_version 577696 (0.0005) [2023-12-27 09:38:08,283][20347] Updated weights for policy 0, policy_version 577930 (0.0006) [2023-12-27 09:38:08,320][20381] Updated weights for policy 1, policy_version 577706 (0.0006) [2023-12-27 09:38:08,337][20347] Updated weights for policy 0, policy_version 577940 (0.0008) [2023-12-27 09:38:08,397][20347] Updated weights for policy 0, policy_version 577950 (0.0007) [2023-12-27 09:38:08,878][20381] Updated weights for policy 1, policy_version 577716 (0.0008) [2023-12-27 09:38:08,935][20381] Updated weights for policy 1, policy_version 577726 (0.0005) [2023-12-27 09:38:08,991][20381] Updated weights for policy 1, policy_version 577736 (0.0005) [2023-12-27 09:38:09,054][20347] Updated weights for policy 0, policy_version 577960 (0.0006) [2023-12-27 09:38:09,108][20347] Updated weights for policy 0, policy_version 577970 (0.0006) [2023-12-27 09:38:09,153][20347] Updated weights for policy 0, policy_version 577980 (0.0010) [2023-12-27 09:38:09,643][20381] Updated weights for policy 1, policy_version 577746 (0.0008) [2023-12-27 09:38:09,703][20381] Updated weights for policy 1, policy_version 577756 (0.0008) [2023-12-27 09:38:09,764][20381] Updated weights for policy 1, policy_version 577766 (0.0011) [2023-12-27 09:38:09,830][20381] Updated weights for policy 1, policy_version 577776 (0.0011) [2023-12-27 09:38:09,858][20347] Updated weights for policy 0, policy_version 577990 (0.0009) [2023-12-27 09:38:09,918][20347] Updated weights for policy 0, policy_version 578000 (0.0008) [2023-12-27 09:38:09,978][20347] Updated weights for policy 0, policy_version 578010 (0.0008) [2023-12-27 09:38:10,514][20381] Updated weights for policy 1, policy_version 577786 (0.0011) [2023-12-27 09:38:10,566][20381] Updated weights for policy 1, policy_version 577796 (0.0011) [2023-12-27 09:38:10,588][20347] Updated weights for policy 0, policy_version 578020 (0.0009) [2023-12-27 09:38:10,612][20381] Updated weights for policy 1, policy_version 577806 (0.0010) [2023-12-27 09:38:10,633][20347] Updated weights for policy 0, policy_version 578030 (0.0010) [2023-12-27 09:38:10,678][20347] Updated weights for policy 0, policy_version 578040 (0.0010) [2023-12-27 09:38:11,345][20381] Updated weights for policy 1, policy_version 577816 (0.0011) [2023-12-27 09:38:11,407][20381] Updated weights for policy 1, policy_version 577826 (0.0010) [2023-12-27 09:38:11,454][20347] Updated weights for policy 0, policy_version 578050 (0.0010) [2023-12-27 09:38:11,468][20381] Updated weights for policy 1, policy_version 577836 (0.0009) [2023-12-27 09:38:11,515][20347] Updated weights for policy 0, policy_version 578060 (0.0009) [2023-12-27 09:38:11,569][20347] Updated weights for policy 0, policy_version 578070 (0.0007) [2023-12-27 09:38:11,633][20347] Updated weights for policy 0, policy_version 578080 (0.0007) [2023-12-27 09:38:12,204][20381] Updated weights for policy 1, policy_version 577846 (0.0010) [2023-12-27 09:38:12,261][20381] Updated weights for policy 1, policy_version 577856 (0.0010) [2023-12-27 09:38:12,323][20381] Updated weights for policy 1, policy_version 577866 (0.0011) [2023-12-27 09:38:12,361][20347] Updated weights for policy 0, policy_version 578090 (0.0007) [2023-12-27 09:38:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 295960576. Throughput: 0: 10444.9, 1: 10610.7. Samples: 295952972. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:38:12,363][19170] Avg episode reward: [(0, '849.770'), (1, '971.900')] [2023-12-27 09:38:12,413][20347] Updated weights for policy 0, policy_version 578100 (0.0008) [2023-12-27 09:38:12,460][20347] Updated weights for policy 0, policy_version 578110 (0.0007) [2023-12-27 09:38:13,034][20381] Updated weights for policy 1, policy_version 577876 (0.0010) [2023-12-27 09:38:13,086][20381] Updated weights for policy 1, policy_version 577886 (0.0010) [2023-12-27 09:38:13,145][20381] Updated weights for policy 1, policy_version 577896 (0.0010) [2023-12-27 09:38:13,188][20347] Updated weights for policy 0, policy_version 578120 (0.0007) [2023-12-27 09:38:13,242][20347] Updated weights for policy 0, policy_version 578130 (0.0007) [2023-12-27 09:38:13,294][20347] Updated weights for policy 0, policy_version 578140 (0.0008) [2023-12-27 09:38:13,825][20381] Updated weights for policy 1, policy_version 577906 (0.0010) [2023-12-27 09:38:13,880][20381] Updated weights for policy 1, policy_version 577916 (0.0010) [2023-12-27 09:38:13,931][20381] Updated weights for policy 1, policy_version 577926 (0.0010) [2023-12-27 09:38:13,982][20381] Updated weights for policy 1, policy_version 577936 (0.0010) [2023-12-27 09:38:14,001][20347] Updated weights for policy 0, policy_version 578150 (0.0007) [2023-12-27 09:38:14,056][20347] Updated weights for policy 0, policy_version 578160 (0.0007) [2023-12-27 09:38:14,110][20347] Updated weights for policy 0, policy_version 578170 (0.0007) [2023-12-27 09:38:14,657][20381] Updated weights for policy 1, policy_version 577946 (0.0009) [2023-12-27 09:38:14,711][20381] Updated weights for policy 1, policy_version 577956 (0.0005) [2023-12-27 09:38:14,750][20347] Updated weights for policy 0, policy_version 578180 (0.0008) [2023-12-27 09:38:14,769][20381] Updated weights for policy 1, policy_version 577966 (0.0006) [2023-12-27 09:38:14,810][20347] Updated weights for policy 0, policy_version 578190 (0.0007) [2023-12-27 09:38:14,866][20347] Updated weights for policy 0, policy_version 578200 (0.0008) [2023-12-27 09:38:15,432][20381] Updated weights for policy 1, policy_version 577976 (0.0010) [2023-12-27 09:38:15,487][20381] Updated weights for policy 1, policy_version 577986 (0.0010) [2023-12-27 09:38:15,493][20347] Updated weights for policy 0, policy_version 578210 (0.0007) [2023-12-27 09:38:15,535][20381] Updated weights for policy 1, policy_version 577996 (0.0010) [2023-12-27 09:38:15,550][20347] Updated weights for policy 0, policy_version 578220 (0.0005) [2023-12-27 09:38:15,610][20347] Updated weights for policy 0, policy_version 578230 (0.0005) [2023-12-27 09:38:15,659][20347] Updated weights for policy 0, policy_version 578240 (0.0006) [2023-12-27 09:38:16,232][20381] Updated weights for policy 1, policy_version 578006 (0.0011) [2023-12-27 09:38:16,237][20347] Updated weights for policy 0, policy_version 578250 (0.0007) [2023-12-27 09:38:16,284][20381] Updated weights for policy 1, policy_version 578016 (0.0011) [2023-12-27 09:38:16,293][20347] Updated weights for policy 0, policy_version 578260 (0.0005) [2023-12-27 09:38:16,329][20381] Updated weights for policy 1, policy_version 578026 (0.0010) [2023-12-27 09:38:16,346][20347] Updated weights for policy 0, policy_version 578270 (0.0005) [2023-12-27 09:38:17,014][20347] Updated weights for policy 0, policy_version 578280 (0.0009) [2023-12-27 09:38:17,031][20381] Updated weights for policy 1, policy_version 578036 (0.0008) [2023-12-27 09:38:17,071][20347] Updated weights for policy 0, policy_version 578290 (0.0010) [2023-12-27 09:38:17,085][20381] Updated weights for policy 1, policy_version 578046 (0.0005) [2023-12-27 09:38:17,128][20347] Updated weights for policy 0, policy_version 578300 (0.0010) [2023-12-27 09:38:17,144][20381] Updated weights for policy 1, policy_version 578056 (0.0005) [2023-12-27 09:38:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 296067072. Throughput: 0: 10399.1, 1: 10479.5. Samples: 296075068. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:38:17,363][19170] Avg episode reward: [(0, '811.530'), (1, '991.140')] [2023-12-27 09:38:17,763][20347] Updated weights for policy 0, policy_version 578310 (0.0010) [2023-12-27 09:38:17,782][20381] Updated weights for policy 1, policy_version 578066 (0.0006) [2023-12-27 09:38:17,815][20347] Updated weights for policy 0, policy_version 578320 (0.0010) [2023-12-27 09:38:17,830][20381] Updated weights for policy 1, policy_version 578076 (0.0005) [2023-12-27 09:38:17,876][20347] Updated weights for policy 0, policy_version 578330 (0.0010) [2023-12-27 09:38:17,883][20381] Updated weights for policy 1, policy_version 578086 (0.0005) [2023-12-27 09:38:17,942][20381] Updated weights for policy 1, policy_version 578096 (0.0005) [2023-12-27 09:38:18,490][20347] Updated weights for policy 0, policy_version 578340 (0.0009) [2023-12-27 09:38:18,547][20347] Updated weights for policy 0, policy_version 578350 (0.0010) [2023-12-27 09:38:18,603][20381] Updated weights for policy 1, policy_version 578106 (0.0006) [2023-12-27 09:38:18,605][20347] Updated weights for policy 0, policy_version 578360 (0.0007) [2023-12-27 09:38:18,651][20381] Updated weights for policy 1, policy_version 578116 (0.0007) [2023-12-27 09:38:18,706][20381] Updated weights for policy 1, policy_version 578126 (0.0007) [2023-12-27 09:38:19,276][20347] Updated weights for policy 0, policy_version 578370 (0.0008) [2023-12-27 09:38:19,339][20347] Updated weights for policy 0, policy_version 578380 (0.0006) [2023-12-27 09:38:19,398][20347] Updated weights for policy 0, policy_version 578390 (0.0007) [2023-12-27 09:38:19,446][20347] Updated weights for policy 0, policy_version 578400 (0.0008) [2023-12-27 09:38:19,447][20381] Updated weights for policy 1, policy_version 578136 (0.0007) [2023-12-27 09:38:19,510][20381] Updated weights for policy 1, policy_version 578146 (0.0009) [2023-12-27 09:38:19,564][20381] Updated weights for policy 1, policy_version 578156 (0.0008) [2023-12-27 09:38:20,149][20347] Updated weights for policy 0, policy_version 578410 (0.0008) [2023-12-27 09:38:20,201][20347] Updated weights for policy 0, policy_version 578420 (0.0008) [2023-12-27 09:38:20,203][20381] Updated weights for policy 1, policy_version 578166 (0.0007) [2023-12-27 09:38:20,258][20381] Updated weights for policy 1, policy_version 578176 (0.0008) [2023-12-27 09:38:20,264][20347] Updated weights for policy 0, policy_version 578430 (0.0006) [2023-12-27 09:38:20,316][20381] Updated weights for policy 1, policy_version 578186 (0.0008) [2023-12-27 09:38:20,882][20347] Updated weights for policy 0, policy_version 578440 (0.0006) [2023-12-27 09:38:20,933][20347] Updated weights for policy 0, policy_version 578450 (0.0008) [2023-12-27 09:38:20,940][20381] Updated weights for policy 1, policy_version 578196 (0.0007) [2023-12-27 09:38:20,996][20381] Updated weights for policy 1, policy_version 578206 (0.0006) [2023-12-27 09:38:20,999][20347] Updated weights for policy 0, policy_version 578460 (0.0007) [2023-12-27 09:38:21,059][20381] Updated weights for policy 1, policy_version 578216 (0.0008) [2023-12-27 09:38:21,670][20347] Updated weights for policy 0, policy_version 578470 (0.0007) [2023-12-27 09:38:21,733][20347] Updated weights for policy 0, policy_version 578480 (0.0008) [2023-12-27 09:38:21,741][20381] Updated weights for policy 1, policy_version 578226 (0.0008) [2023-12-27 09:38:21,796][20347] Updated weights for policy 0, policy_version 578490 (0.0008) [2023-12-27 09:38:21,806][20381] Updated weights for policy 1, policy_version 578236 (0.0011) [2023-12-27 09:38:21,872][20381] Updated weights for policy 1, policy_version 578246 (0.0011) [2023-12-27 09:38:21,942][20381] Updated weights for policy 1, policy_version 578256 (0.0008) [2023-12-27 09:38:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 296165376. Throughput: 0: 10398.9, 1: 10485.6. Samples: 296137216. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-12-27 09:38:22,363][19170] Avg episode reward: [(0, '876.860'), (1, '1000.000')] [2023-12-27 09:38:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000578256_148054016.pth... [2023-12-27 09:38:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000578496_148111360.pth... [2023-12-27 09:38:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000577040_147742720.pth [2023-12-27 09:38:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000577280_147800064.pth [2023-12-27 09:38:22,429][20347] Updated weights for policy 0, policy_version 578500 (0.0007) [2023-12-27 09:38:22,488][20347] Updated weights for policy 0, policy_version 578510 (0.0007) [2023-12-27 09:38:22,538][20347] Updated weights for policy 0, policy_version 578520 (0.0007) [2023-12-27 09:38:22,617][20381] Updated weights for policy 1, policy_version 578266 (0.0008) [2023-12-27 09:38:22,672][20381] Updated weights for policy 1, policy_version 578276 (0.0007) [2023-12-27 09:38:22,725][20381] Updated weights for policy 1, policy_version 578286 (0.0007) [2023-12-27 09:38:23,255][20347] Updated weights for policy 0, policy_version 578530 (0.0007) [2023-12-27 09:38:23,302][20347] Updated weights for policy 0, policy_version 578540 (0.0008) [2023-12-27 09:38:23,352][20347] Updated weights for policy 0, policy_version 578550 (0.0007) [2023-12-27 09:38:23,413][20347] Updated weights for policy 0, policy_version 578560 (0.0006) [2023-12-27 09:38:23,438][20381] Updated weights for policy 1, policy_version 578296 (0.0008) [2023-12-27 09:38:23,490][20381] Updated weights for policy 1, policy_version 578306 (0.0007) [2023-12-27 09:38:23,541][20381] Updated weights for policy 1, policy_version 578316 (0.0008) [2023-12-27 09:38:24,091][20347] Updated weights for policy 0, policy_version 578570 (0.0008) [2023-12-27 09:38:24,154][20347] Updated weights for policy 0, policy_version 578580 (0.0008) [2023-12-27 09:38:24,171][20381] Updated weights for policy 1, policy_version 578326 (0.0007) [2023-12-27 09:38:24,218][20347] Updated weights for policy 0, policy_version 578590 (0.0006) [2023-12-27 09:38:24,237][20381] Updated weights for policy 1, policy_version 578336 (0.0005) [2023-12-27 09:38:24,305][20381] Updated weights for policy 1, policy_version 578346 (0.0006) [2023-12-27 09:38:24,863][20347] Updated weights for policy 0, policy_version 578600 (0.0005) [2023-12-27 09:38:24,874][20381] Updated weights for policy 1, policy_version 578356 (0.0007) [2023-12-27 09:38:24,919][20347] Updated weights for policy 0, policy_version 578610 (0.0005) [2023-12-27 09:38:24,934][20381] Updated weights for policy 1, policy_version 578366 (0.0008) [2023-12-27 09:38:24,964][20347] Updated weights for policy 0, policy_version 578620 (0.0005) [2023-12-27 09:38:24,980][20381] Updated weights for policy 1, policy_version 578376 (0.0008) [2023-12-27 09:38:25,559][20347] Updated weights for policy 0, policy_version 578630 (0.0007) [2023-12-27 09:38:25,607][20347] Updated weights for policy 0, policy_version 578640 (0.0007) [2023-12-27 09:38:25,619][20381] Updated weights for policy 1, policy_version 578386 (0.0007) [2023-12-27 09:38:25,657][20347] Updated weights for policy 0, policy_version 578650 (0.0007) [2023-12-27 09:38:25,674][20381] Updated weights for policy 1, policy_version 578396 (0.0006) [2023-12-27 09:38:25,721][20381] Updated weights for policy 1, policy_version 578406 (0.0007) [2023-12-27 09:38:25,764][20381] Updated weights for policy 1, policy_version 578416 (0.0007) [2023-12-27 09:38:26,298][20347] Updated weights for policy 0, policy_version 578660 (0.0007) [2023-12-27 09:38:26,342][20347] Updated weights for policy 0, policy_version 578670 (0.0006) [2023-12-27 09:38:26,400][20347] Updated weights for policy 0, policy_version 578680 (0.0005) [2023-12-27 09:38:26,426][20381] Updated weights for policy 1, policy_version 578426 (0.0005) [2023-12-27 09:38:26,478][20381] Updated weights for policy 1, policy_version 578436 (0.0005) [2023-12-27 09:38:26,529][20381] Updated weights for policy 1, policy_version 578446 (0.0005) [2023-12-27 09:38:26,928][20347] Updated weights for policy 0, policy_version 578690 (0.0005) [2023-12-27 09:38:26,978][20347] Updated weights for policy 0, policy_version 578700 (0.0005) [2023-12-27 09:38:27,033][20347] Updated weights for policy 0, policy_version 578710 (0.0005) [2023-12-27 09:38:27,086][20347] Updated weights for policy 0, policy_version 578720 (0.0006) [2023-12-27 09:38:27,166][20381] Updated weights for policy 1, policy_version 578456 (0.0006) [2023-12-27 09:38:27,227][20381] Updated weights for policy 1, policy_version 578466 (0.0005) [2023-12-27 09:38:27,290][20381] Updated weights for policy 1, policy_version 578476 (0.0005) [2023-12-27 09:38:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 296280064. Throughput: 0: 10431.4, 1: 10454.0. Samples: 296262628. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:27,363][19170] Avg episode reward: [(0, '943.030'), (1, '1000.000')] [2023-12-27 09:38:27,652][20347] Updated weights for policy 0, policy_version 578730 (0.0005) [2023-12-27 09:38:27,702][20347] Updated weights for policy 0, policy_version 578740 (0.0005) [2023-12-27 09:38:27,748][20347] Updated weights for policy 0, policy_version 578750 (0.0005) [2023-12-27 09:38:27,848][20381] Updated weights for policy 1, policy_version 578486 (0.0006) [2023-12-27 09:38:27,898][20381] Updated weights for policy 1, policy_version 578496 (0.0005) [2023-12-27 09:38:27,941][20381] Updated weights for policy 1, policy_version 578506 (0.0005) [2023-12-27 09:38:28,372][20347] Updated weights for policy 0, policy_version 578760 (0.0007) [2023-12-27 09:38:28,429][20347] Updated weights for policy 0, policy_version 578770 (0.0007) [2023-12-27 09:38:28,486][20347] Updated weights for policy 0, policy_version 578780 (0.0007) [2023-12-27 09:38:28,511][20381] Updated weights for policy 1, policy_version 578516 (0.0006) [2023-12-27 09:38:28,571][20381] Updated weights for policy 1, policy_version 578526 (0.0009) [2023-12-27 09:38:28,632][20381] Updated weights for policy 1, policy_version 578536 (0.0008) [2023-12-27 09:38:29,038][20347] Updated weights for policy 0, policy_version 578790 (0.0006) [2023-12-27 09:38:29,097][20347] Updated weights for policy 0, policy_version 578800 (0.0005) [2023-12-27 09:38:29,149][20347] Updated weights for policy 0, policy_version 578810 (0.0007) [2023-12-27 09:38:29,378][20381] Updated weights for policy 1, policy_version 578546 (0.0008) [2023-12-27 09:38:29,438][20381] Updated weights for policy 1, policy_version 578556 (0.0008) [2023-12-27 09:38:29,498][20381] Updated weights for policy 1, policy_version 578566 (0.0008) [2023-12-27 09:38:29,555][20381] Updated weights for policy 1, policy_version 578576 (0.0007) [2023-12-27 09:38:29,852][20347] Updated weights for policy 0, policy_version 578820 (0.0007) [2023-12-27 09:38:29,911][20347] Updated weights for policy 0, policy_version 578830 (0.0008) [2023-12-27 09:38:29,973][20347] Updated weights for policy 0, policy_version 578840 (0.0006) [2023-12-27 09:38:30,281][20381] Updated weights for policy 1, policy_version 578586 (0.0007) [2023-12-27 09:38:30,329][20381] Updated weights for policy 1, policy_version 578596 (0.0008) [2023-12-27 09:38:30,382][20381] Updated weights for policy 1, policy_version 578606 (0.0008) [2023-12-27 09:38:30,546][20347] Updated weights for policy 0, policy_version 578850 (0.0006) [2023-12-27 09:38:30,597][20347] Updated weights for policy 0, policy_version 578860 (0.0006) [2023-12-27 09:38:30,653][20347] Updated weights for policy 0, policy_version 578870 (0.0008) [2023-12-27 09:38:30,710][20347] Updated weights for policy 0, policy_version 578880 (0.0007) [2023-12-27 09:38:31,103][20381] Updated weights for policy 1, policy_version 578616 (0.0010) [2023-12-27 09:38:31,170][20381] Updated weights for policy 1, policy_version 578626 (0.0011) [2023-12-27 09:38:31,230][20381] Updated weights for policy 1, policy_version 578636 (0.0011) [2023-12-27 09:38:31,414][20347] Updated weights for policy 0, policy_version 578890 (0.0007) [2023-12-27 09:38:31,461][20347] Updated weights for policy 0, policy_version 578900 (0.0007) [2023-12-27 09:38:31,506][20347] Updated weights for policy 0, policy_version 578910 (0.0009) [2023-12-27 09:38:31,899][20381] Updated weights for policy 1, policy_version 578646 (0.0010) [2023-12-27 09:38:31,961][20381] Updated weights for policy 1, policy_version 578656 (0.0011) [2023-12-27 09:38:32,015][20381] Updated weights for policy 1, policy_version 578666 (0.0010) [2023-12-27 09:38:32,226][20347] Updated weights for policy 0, policy_version 578920 (0.0008) [2023-12-27 09:38:32,282][20347] Updated weights for policy 0, policy_version 578930 (0.0007) [2023-12-27 09:38:32,335][20347] Updated weights for policy 0, policy_version 578940 (0.0007) [2023-12-27 09:38:32,362][19170] Fps is (10 sec: 22118.8, 60 sec: 21026.2, 300 sec: 20799.4). Total num frames: 296386560. Throughput: 0: 10535.0, 1: 10439.8. Samples: 296392052. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:32,363][19170] Avg episode reward: [(0, '933.540'), (1, '1000.000')] [2023-12-27 09:38:32,711][20381] Updated weights for policy 1, policy_version 578676 (0.0010) [2023-12-27 09:38:32,763][20381] Updated weights for policy 1, policy_version 578686 (0.0010) [2023-12-27 09:38:32,814][20381] Updated weights for policy 1, policy_version 578696 (0.0010) [2023-12-27 09:38:32,970][20347] Updated weights for policy 0, policy_version 578950 (0.0008) [2023-12-27 09:38:33,021][20347] Updated weights for policy 0, policy_version 578960 (0.0007) [2023-12-27 09:38:33,077][20347] Updated weights for policy 0, policy_version 578970 (0.0008) [2023-12-27 09:38:33,454][20381] Updated weights for policy 1, policy_version 578706 (0.0010) [2023-12-27 09:38:33,498][20381] Updated weights for policy 1, policy_version 578716 (0.0007) [2023-12-27 09:38:33,547][20381] Updated weights for policy 1, policy_version 578726 (0.0007) [2023-12-27 09:38:33,601][20381] Updated weights for policy 1, policy_version 578736 (0.0006) [2023-12-27 09:38:33,806][20347] Updated weights for policy 0, policy_version 578980 (0.0007) [2023-12-27 09:38:33,875][20347] Updated weights for policy 0, policy_version 578990 (0.0006) [2023-12-27 09:38:33,929][20347] Updated weights for policy 0, policy_version 579000 (0.0007) [2023-12-27 09:38:34,295][20381] Updated weights for policy 1, policy_version 578746 (0.0007) [2023-12-27 09:38:34,349][20381] Updated weights for policy 1, policy_version 578756 (0.0008) [2023-12-27 09:38:34,398][20381] Updated weights for policy 1, policy_version 578766 (0.0010) [2023-12-27 09:38:34,592][20347] Updated weights for policy 0, policy_version 579010 (0.0007) [2023-12-27 09:38:34,662][20347] Updated weights for policy 0, policy_version 579020 (0.0008) [2023-12-27 09:38:34,728][20347] Updated weights for policy 0, policy_version 579030 (0.0008) [2023-12-27 09:38:34,790][20347] Updated weights for policy 0, policy_version 579040 (0.0008) [2023-12-27 09:38:35,093][20381] Updated weights for policy 1, policy_version 578776 (0.0011) [2023-12-27 09:38:35,145][20381] Updated weights for policy 1, policy_version 578786 (0.0010) [2023-12-27 09:38:35,198][20381] Updated weights for policy 1, policy_version 578796 (0.0010) [2023-12-27 09:38:35,395][20347] Updated weights for policy 0, policy_version 579050 (0.0007) [2023-12-27 09:38:35,446][20347] Updated weights for policy 0, policy_version 579060 (0.0008) [2023-12-27 09:38:35,505][20347] Updated weights for policy 0, policy_version 579070 (0.0006) [2023-12-27 09:38:35,909][20381] Updated weights for policy 1, policy_version 578806 (0.0008) [2023-12-27 09:38:35,968][20381] Updated weights for policy 1, policy_version 578816 (0.0008) [2023-12-27 09:38:36,025][20381] Updated weights for policy 1, policy_version 578826 (0.0008) [2023-12-27 09:38:36,140][20347] Updated weights for policy 0, policy_version 579080 (0.0007) [2023-12-27 09:38:36,199][20347] Updated weights for policy 0, policy_version 579090 (0.0007) [2023-12-27 09:38:36,260][20347] Updated weights for policy 0, policy_version 579100 (0.0008) [2023-12-27 09:38:36,704][20381] Updated weights for policy 1, policy_version 578836 (0.0006) [2023-12-27 09:38:36,766][20381] Updated weights for policy 1, policy_version 578846 (0.0005) [2023-12-27 09:38:36,828][20381] Updated weights for policy 1, policy_version 578856 (0.0006) [2023-12-27 09:38:36,969][20347] Updated weights for policy 0, policy_version 579110 (0.0007) [2023-12-27 09:38:37,029][20347] Updated weights for policy 0, policy_version 579120 (0.0008) [2023-12-27 09:38:37,085][20347] Updated weights for policy 0, policy_version 579130 (0.0007) [2023-12-27 09:38:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 296484864. Throughput: 0: 10495.5, 1: 10453.5. Samples: 296454188. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:37,364][19170] Avg episode reward: [(0, '972.020'), (1, '990.430')] [2023-12-27 09:38:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000579136_148275200.pth... [2023-12-27 09:38:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000578864_148209664.pth... [2023-12-27 09:38:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000577888_147955712.pth [2023-12-27 09:38:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000577648_147898368.pth [2023-12-27 09:38:37,493][20381] Updated weights for policy 1, policy_version 578866 (0.0006) [2023-12-27 09:38:37,546][20381] Updated weights for policy 1, policy_version 578876 (0.0008) [2023-12-27 09:38:37,600][20381] Updated weights for policy 1, policy_version 578886 (0.0008) [2023-12-27 09:38:37,654][20381] Updated weights for policy 1, policy_version 578896 (0.0008) [2023-12-27 09:38:37,792][20347] Updated weights for policy 0, policy_version 579140 (0.0008) [2023-12-27 09:38:37,852][20347] Updated weights for policy 0, policy_version 579150 (0.0007) [2023-12-27 09:38:37,908][20347] Updated weights for policy 0, policy_version 579160 (0.0009) [2023-12-27 09:38:38,324][20381] Updated weights for policy 1, policy_version 578906 (0.0008) [2023-12-27 09:38:38,374][20381] Updated weights for policy 1, policy_version 578916 (0.0007) [2023-12-27 09:38:38,422][20381] Updated weights for policy 1, policy_version 578926 (0.0007) [2023-12-27 09:38:38,606][20347] Updated weights for policy 0, policy_version 579170 (0.0008) [2023-12-27 09:38:38,671][20347] Updated weights for policy 0, policy_version 579180 (0.0007) [2023-12-27 09:38:38,726][20347] Updated weights for policy 0, policy_version 579190 (0.0008) [2023-12-27 09:38:38,789][20347] Updated weights for policy 0, policy_version 579200 (0.0008) [2023-12-27 09:38:39,099][20381] Updated weights for policy 1, policy_version 578936 (0.0007) [2023-12-27 09:38:39,149][20381] Updated weights for policy 1, policy_version 578946 (0.0007) [2023-12-27 09:38:39,205][20381] Updated weights for policy 1, policy_version 578956 (0.0008) [2023-12-27 09:38:39,493][20347] Updated weights for policy 0, policy_version 579210 (0.0008) [2023-12-27 09:38:39,556][20347] Updated weights for policy 0, policy_version 579220 (0.0008) [2023-12-27 09:38:39,611][20347] Updated weights for policy 0, policy_version 579230 (0.0007) [2023-12-27 09:38:39,971][20381] Updated weights for policy 1, policy_version 578966 (0.0008) [2023-12-27 09:38:40,027][20381] Updated weights for policy 1, policy_version 578976 (0.0007) [2023-12-27 09:38:40,076][20381] Updated weights for policy 1, policy_version 578986 (0.0009) [2023-12-27 09:38:40,260][20347] Updated weights for policy 0, policy_version 579240 (0.0008) [2023-12-27 09:38:40,319][20347] Updated weights for policy 0, policy_version 579250 (0.0007) [2023-12-27 09:38:40,377][20347] Updated weights for policy 0, policy_version 579260 (0.0006) [2023-12-27 09:38:40,786][20381] Updated weights for policy 1, policy_version 578996 (0.0007) [2023-12-27 09:38:40,835][20381] Updated weights for policy 1, policy_version 579006 (0.0007) [2023-12-27 09:38:40,894][20381] Updated weights for policy 1, policy_version 579016 (0.0008) [2023-12-27 09:38:41,047][20347] Updated weights for policy 0, policy_version 579270 (0.0008) [2023-12-27 09:38:41,106][20347] Updated weights for policy 0, policy_version 579280 (0.0007) [2023-12-27 09:38:41,171][20347] Updated weights for policy 0, policy_version 579290 (0.0009) [2023-12-27 09:38:41,672][20381] Updated weights for policy 1, policy_version 579026 (0.0008) [2023-12-27 09:38:41,742][20381] Updated weights for policy 1, policy_version 579036 (0.0008) [2023-12-27 09:38:41,801][20381] Updated weights for policy 1, policy_version 579046 (0.0007) [2023-12-27 09:38:41,865][20381] Updated weights for policy 1, policy_version 579056 (0.0007) [2023-12-27 09:38:41,869][20347] Updated weights for policy 0, policy_version 579300 (0.0007) [2023-12-27 09:38:41,933][20347] Updated weights for policy 0, policy_version 579310 (0.0005) [2023-12-27 09:38:41,992][20347] Updated weights for policy 0, policy_version 579320 (0.0007) [2023-12-27 09:38:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 296583168. Throughput: 0: 10462.8, 1: 10352.3. Samples: 296575812. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:42,363][19170] Avg episode reward: [(0, '971.090'), (1, '990.430')] [2023-12-27 09:38:42,557][20347] Updated weights for policy 0, policy_version 579330 (0.0006) [2023-12-27 09:38:42,562][20381] Updated weights for policy 1, policy_version 579066 (0.0006) [2023-12-27 09:38:42,626][20347] Updated weights for policy 0, policy_version 579340 (0.0006) [2023-12-27 09:38:42,627][20381] Updated weights for policy 1, policy_version 579076 (0.0006) [2023-12-27 09:38:42,684][20381] Updated weights for policy 1, policy_version 579086 (0.0006) [2023-12-27 09:38:42,691][20347] Updated weights for policy 0, policy_version 579350 (0.0006) [2023-12-27 09:38:42,754][20347] Updated weights for policy 0, policy_version 579360 (0.0007) [2023-12-27 09:38:43,363][20347] Updated weights for policy 0, policy_version 579370 (0.0007) [2023-12-27 09:38:43,383][20381] Updated weights for policy 1, policy_version 579096 (0.0006) [2023-12-27 09:38:43,410][20347] Updated weights for policy 0, policy_version 579380 (0.0005) [2023-12-27 09:38:43,434][20381] Updated weights for policy 1, policy_version 579106 (0.0007) [2023-12-27 09:38:43,461][20347] Updated weights for policy 0, policy_version 579390 (0.0005) [2023-12-27 09:38:43,493][20381] Updated weights for policy 1, policy_version 579116 (0.0008) [2023-12-27 09:38:44,083][20381] Updated weights for policy 1, policy_version 579126 (0.0007) [2023-12-27 09:38:44,135][20381] Updated weights for policy 1, policy_version 579136 (0.0008) [2023-12-27 09:38:44,173][20347] Updated weights for policy 0, policy_version 579400 (0.0006) [2023-12-27 09:38:44,189][20381] Updated weights for policy 1, policy_version 579146 (0.0007) [2023-12-27 09:38:44,228][20347] Updated weights for policy 0, policy_version 579410 (0.0005) [2023-12-27 09:38:44,283][20347] Updated weights for policy 0, policy_version 579420 (0.0005) [2023-12-27 09:38:44,803][20347] Updated weights for policy 0, policy_version 579430 (0.0007) [2023-12-27 09:38:44,854][20347] Updated weights for policy 0, policy_version 579440 (0.0006) [2023-12-27 09:38:44,884][20381] Updated weights for policy 1, policy_version 579156 (0.0008) [2023-12-27 09:38:44,902][20347] Updated weights for policy 0, policy_version 579450 (0.0005) [2023-12-27 09:38:44,942][20381] Updated weights for policy 1, policy_version 579166 (0.0007) [2023-12-27 09:38:45,007][20381] Updated weights for policy 1, policy_version 579176 (0.0007) [2023-12-27 09:38:45,575][20381] Updated weights for policy 1, policy_version 579186 (0.0005) [2023-12-27 09:38:45,605][20347] Updated weights for policy 0, policy_version 579460 (0.0006) [2023-12-27 09:38:45,625][20381] Updated weights for policy 1, policy_version 579196 (0.0006) [2023-12-27 09:38:45,655][20347] Updated weights for policy 0, policy_version 579470 (0.0006) [2023-12-27 09:38:45,682][20381] Updated weights for policy 1, policy_version 579206 (0.0011) [2023-12-27 09:38:45,709][20347] Updated weights for policy 0, policy_version 579480 (0.0005) [2023-12-27 09:38:45,742][20381] Updated weights for policy 1, policy_version 579216 (0.0011) [2023-12-27 09:38:46,311][20347] Updated weights for policy 0, policy_version 579490 (0.0006) [2023-12-27 09:38:46,360][20347] Updated weights for policy 0, policy_version 579500 (0.0008) [2023-12-27 09:38:46,370][20381] Updated weights for policy 1, policy_version 579226 (0.0010) [2023-12-27 09:38:46,415][20347] Updated weights for policy 0, policy_version 579510 (0.0005) [2023-12-27 09:38:46,427][20381] Updated weights for policy 1, policy_version 579236 (0.0010) [2023-12-27 09:38:46,470][20347] Updated weights for policy 0, policy_version 579520 (0.0008) [2023-12-27 09:38:46,485][20381] Updated weights for policy 1, policy_version 579246 (0.0010) [2023-12-27 09:38:47,129][20381] Updated weights for policy 1, policy_version 579256 (0.0008) [2023-12-27 09:38:47,159][20347] Updated weights for policy 0, policy_version 579530 (0.0006) [2023-12-27 09:38:47,178][20381] Updated weights for policy 1, policy_version 579266 (0.0005) [2023-12-27 09:38:47,214][20347] Updated weights for policy 0, policy_version 579540 (0.0008) [2023-12-27 09:38:47,238][20381] Updated weights for policy 1, policy_version 579276 (0.0005) [2023-12-27 09:38:47,264][20347] Updated weights for policy 0, policy_version 579550 (0.0007) [2023-12-27 09:38:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 296697856. Throughput: 0: 10485.3, 1: 10363.5. Samples: 296703156. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:47,363][19170] Avg episode reward: [(0, '980.700'), (1, '1000.000')] [2023-12-27 09:38:47,775][20381] Updated weights for policy 1, policy_version 579286 (0.0006) [2023-12-27 09:38:47,834][20381] Updated weights for policy 1, policy_version 579296 (0.0008) [2023-12-27 09:38:47,889][20381] Updated weights for policy 1, policy_version 579306 (0.0008) [2023-12-27 09:38:47,983][20347] Updated weights for policy 0, policy_version 579560 (0.0007) [2023-12-27 09:38:48,045][20347] Updated weights for policy 0, policy_version 579570 (0.0007) [2023-12-27 09:38:48,096][20347] Updated weights for policy 0, policy_version 579580 (0.0008) [2023-12-27 09:38:48,556][20381] Updated weights for policy 1, policy_version 579316 (0.0008) [2023-12-27 09:38:48,617][20381] Updated weights for policy 1, policy_version 579326 (0.0009) [2023-12-27 09:38:48,675][20381] Updated weights for policy 1, policy_version 579336 (0.0007) [2023-12-27 09:38:48,738][20347] Updated weights for policy 0, policy_version 579590 (0.0006) [2023-12-27 09:38:48,790][20347] Updated weights for policy 0, policy_version 579600 (0.0007) [2023-12-27 09:38:48,846][20347] Updated weights for policy 0, policy_version 579610 (0.0008) [2023-12-27 09:38:49,352][20381] Updated weights for policy 1, policy_version 579346 (0.0007) [2023-12-27 09:38:49,421][20381] Updated weights for policy 1, policy_version 579356 (0.0008) [2023-12-27 09:38:49,481][20381] Updated weights for policy 1, policy_version 579366 (0.0008) [2023-12-27 09:38:49,538][20347] Updated weights for policy 0, policy_version 579620 (0.0007) [2023-12-27 09:38:49,544][20381] Updated weights for policy 1, policy_version 579376 (0.0007) [2023-12-27 09:38:49,600][20347] Updated weights for policy 0, policy_version 579630 (0.0007) [2023-12-27 09:38:49,661][20347] Updated weights for policy 0, policy_version 579640 (0.0008) [2023-12-27 09:38:50,186][20381] Updated weights for policy 1, policy_version 579386 (0.0009) [2023-12-27 09:38:50,241][20381] Updated weights for policy 1, policy_version 579396 (0.0008) [2023-12-27 09:38:50,294][20381] Updated weights for policy 1, policy_version 579406 (0.0008) [2023-12-27 09:38:50,385][20347] Updated weights for policy 0, policy_version 579650 (0.0008) [2023-12-27 09:38:50,440][20347] Updated weights for policy 0, policy_version 579660 (0.0007) [2023-12-27 09:38:50,484][20347] Updated weights for policy 0, policy_version 579670 (0.0007) [2023-12-27 09:38:50,531][20347] Updated weights for policy 0, policy_version 579680 (0.0006) [2023-12-27 09:38:51,027][20381] Updated weights for policy 1, policy_version 579416 (0.0006) [2023-12-27 09:38:51,084][20381] Updated weights for policy 1, policy_version 579426 (0.0008) [2023-12-27 09:38:51,146][20381] Updated weights for policy 1, policy_version 579436 (0.0008) [2023-12-27 09:38:51,185][20347] Updated weights for policy 0, policy_version 579690 (0.0007) [2023-12-27 09:38:51,251][20347] Updated weights for policy 0, policy_version 579700 (0.0008) [2023-12-27 09:38:51,323][20347] Updated weights for policy 0, policy_version 579710 (0.0008) [2023-12-27 09:38:51,774][20381] Updated weights for policy 1, policy_version 579446 (0.0007) [2023-12-27 09:38:51,828][20381] Updated weights for policy 1, policy_version 579456 (0.0008) [2023-12-27 09:38:51,884][20381] Updated weights for policy 1, policy_version 579466 (0.0007) [2023-12-27 09:38:52,061][20347] Updated weights for policy 0, policy_version 579720 (0.0008) [2023-12-27 09:38:52,119][20347] Updated weights for policy 0, policy_version 579730 (0.0007) [2023-12-27 09:38:52,171][20347] Updated weights for policy 0, policy_version 579740 (0.0007) [2023-12-27 09:38:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 296796160. Throughput: 0: 10473.5, 1: 10380.9. Samples: 296765500. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 09:38:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000579472_148365312.pth... [2023-12-27 09:38:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000579744_148430848.pth... [2023-12-27 09:38:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000578256_148054016.pth [2023-12-27 09:38:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000578496_148111360.pth [2023-12-27 09:38:52,591][20381] Updated weights for policy 1, policy_version 579476 (0.0008) [2023-12-27 09:38:52,644][20381] Updated weights for policy 1, policy_version 579486 (0.0007) [2023-12-27 09:38:52,706][20381] Updated weights for policy 1, policy_version 579496 (0.0009) [2023-12-27 09:38:52,877][20347] Updated weights for policy 0, policy_version 579750 (0.0008) [2023-12-27 09:38:52,929][20347] Updated weights for policy 0, policy_version 579760 (0.0007) [2023-12-27 09:38:52,991][20347] Updated weights for policy 0, policy_version 579770 (0.0006) [2023-12-27 09:38:53,379][20381] Updated weights for policy 1, policy_version 579506 (0.0009) [2023-12-27 09:38:53,425][20381] Updated weights for policy 1, policy_version 579516 (0.0009) [2023-12-27 09:38:53,484][20381] Updated weights for policy 1, policy_version 579526 (0.0006) [2023-12-27 09:38:53,546][20381] Updated weights for policy 1, policy_version 579536 (0.0006) [2023-12-27 09:38:53,693][20347] Updated weights for policy 0, policy_version 579780 (0.0008) [2023-12-27 09:38:53,751][20347] Updated weights for policy 0, policy_version 579790 (0.0007) [2023-12-27 09:38:53,812][20347] Updated weights for policy 0, policy_version 579800 (0.0007) [2023-12-27 09:38:54,188][20381] Updated weights for policy 1, policy_version 579546 (0.0006) [2023-12-27 09:38:54,245][20381] Updated weights for policy 1, policy_version 579556 (0.0007) [2023-12-27 09:38:54,301][20381] Updated weights for policy 1, policy_version 579566 (0.0007) [2023-12-27 09:38:54,498][20347] Updated weights for policy 0, policy_version 579810 (0.0007) [2023-12-27 09:38:54,552][20347] Updated weights for policy 0, policy_version 579820 (0.0007) [2023-12-27 09:38:54,599][20347] Updated weights for policy 0, policy_version 579830 (0.0007) [2023-12-27 09:38:54,646][20347] Updated weights for policy 0, policy_version 579840 (0.0007) [2023-12-27 09:38:54,981][20381] Updated weights for policy 1, policy_version 579576 (0.0010) [2023-12-27 09:38:55,037][20381] Updated weights for policy 1, policy_version 579586 (0.0011) [2023-12-27 09:38:55,092][20381] Updated weights for policy 1, policy_version 579596 (0.0011) [2023-12-27 09:38:55,314][20347] Updated weights for policy 0, policy_version 579850 (0.0005) [2023-12-27 09:38:55,357][20347] Updated weights for policy 0, policy_version 579860 (0.0005) [2023-12-27 09:38:55,405][20347] Updated weights for policy 0, policy_version 579870 (0.0009) [2023-12-27 09:38:55,685][20381] Updated weights for policy 1, policy_version 579606 (0.0010) [2023-12-27 09:38:55,739][20381] Updated weights for policy 1, policy_version 579616 (0.0010) [2023-12-27 09:38:55,804][20381] Updated weights for policy 1, policy_version 579626 (0.0011) [2023-12-27 09:38:56,087][20347] Updated weights for policy 0, policy_version 579880 (0.0006) [2023-12-27 09:38:56,148][20347] Updated weights for policy 0, policy_version 579890 (0.0005) [2023-12-27 09:38:56,199][20347] Updated weights for policy 0, policy_version 579900 (0.0005) [2023-12-27 09:38:56,386][20381] Updated weights for policy 1, policy_version 579636 (0.0010) [2023-12-27 09:38:56,440][20381] Updated weights for policy 1, policy_version 579646 (0.0010) [2023-12-27 09:38:56,494][20381] Updated weights for policy 1, policy_version 579656 (0.0010) [2023-12-27 09:38:56,815][20347] Updated weights for policy 0, policy_version 579910 (0.0008) [2023-12-27 09:38:56,874][20347] Updated weights for policy 0, policy_version 579920 (0.0010) [2023-12-27 09:38:56,930][20347] Updated weights for policy 0, policy_version 579930 (0.0009) [2023-12-27 09:38:57,150][20381] Updated weights for policy 1, policy_version 579666 (0.0007) [2023-12-27 09:38:57,215][20381] Updated weights for policy 1, policy_version 579676 (0.0005) [2023-12-27 09:38:57,272][20381] Updated weights for policy 1, policy_version 579686 (0.0005) [2023-12-27 09:38:57,335][20381] Updated weights for policy 1, policy_version 579696 (0.0008) [2023-12-27 09:38:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 296902656. Throughput: 0: 10450.1, 1: 10356.9. Samples: 296889292. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:38:57,364][19170] Avg episode reward: [(0, '971.090'), (1, '990.340')] [2023-12-27 09:38:57,503][20347] Updated weights for policy 0, policy_version 579940 (0.0010) [2023-12-27 09:38:57,555][20347] Updated weights for policy 0, policy_version 579950 (0.0010) [2023-12-27 09:38:57,602][20347] Updated weights for policy 0, policy_version 579960 (0.0010) [2023-12-27 09:38:57,966][20381] Updated weights for policy 1, policy_version 579706 (0.0005) [2023-12-27 09:38:58,015][20381] Updated weights for policy 1, policy_version 579716 (0.0005) [2023-12-27 09:38:58,068][20381] Updated weights for policy 1, policy_version 579726 (0.0006) [2023-12-27 09:38:58,254][20347] Updated weights for policy 0, policy_version 579970 (0.0008) [2023-12-27 09:38:58,316][20347] Updated weights for policy 0, policy_version 579980 (0.0010) [2023-12-27 09:38:58,378][20347] Updated weights for policy 0, policy_version 579990 (0.0009) [2023-12-27 09:38:58,460][20347] Updated weights for policy 0, policy_version 580000 (0.0011) [2023-12-27 09:38:58,786][20381] Updated weights for policy 1, policy_version 579736 (0.0008) [2023-12-27 09:38:58,857][20381] Updated weights for policy 1, policy_version 579746 (0.0008) [2023-12-27 09:38:58,929][20381] Updated weights for policy 1, policy_version 579756 (0.0008) [2023-12-27 09:38:59,226][20347] Updated weights for policy 0, policy_version 580010 (0.0008) [2023-12-27 09:38:59,260][20291] KL-divergence is very high: 168.9082 [2023-12-27 09:38:59,289][20347] Updated weights for policy 0, policy_version 580020 (0.0008) [2023-12-27 09:38:59,351][20347] Updated weights for policy 0, policy_version 580030 (0.0008) [2023-12-27 09:38:59,604][20381] Updated weights for policy 1, policy_version 579766 (0.0007) [2023-12-27 09:38:59,664][20381] Updated weights for policy 1, policy_version 579776 (0.0005) [2023-12-27 09:38:59,724][20381] Updated weights for policy 1, policy_version 579786 (0.0006) [2023-12-27 09:39:00,079][20347] Updated weights for policy 0, policy_version 580040 (0.0009) [2023-12-27 09:39:00,137][20347] Updated weights for policy 0, policy_version 580050 (0.0008) [2023-12-27 09:39:00,192][20347] Updated weights for policy 0, policy_version 580060 (0.0007) [2023-12-27 09:39:00,364][20381] Updated weights for policy 1, policy_version 579796 (0.0007) [2023-12-27 09:39:00,419][20381] Updated weights for policy 1, policy_version 579806 (0.0008) [2023-12-27 09:39:00,476][20381] Updated weights for policy 1, policy_version 579816 (0.0008) [2023-12-27 09:39:00,873][20347] Updated weights for policy 0, policy_version 580070 (0.0008) [2023-12-27 09:39:00,932][20347] Updated weights for policy 0, policy_version 580080 (0.0006) [2023-12-27 09:39:00,991][20347] Updated weights for policy 0, policy_version 580090 (0.0005) [2023-12-27 09:39:01,164][20381] Updated weights for policy 1, policy_version 579826 (0.0007) [2023-12-27 09:39:01,229][20381] Updated weights for policy 1, policy_version 579836 (0.0007) [2023-12-27 09:39:01,288][20381] Updated weights for policy 1, policy_version 579846 (0.0008) [2023-12-27 09:39:01,343][20381] Updated weights for policy 1, policy_version 579856 (0.0009) [2023-12-27 09:39:01,610][20347] Updated weights for policy 0, policy_version 580100 (0.0007) [2023-12-27 09:39:01,665][20347] Updated weights for policy 0, policy_version 580110 (0.0008) [2023-12-27 09:39:01,733][20347] Updated weights for policy 0, policy_version 580120 (0.0007) [2023-12-27 09:39:02,051][20381] Updated weights for policy 1, policy_version 579866 (0.0007) [2023-12-27 09:39:02,108][20381] Updated weights for policy 1, policy_version 579876 (0.0007) [2023-12-27 09:39:02,167][20381] Updated weights for policy 1, policy_version 579886 (0.0008) [2023-12-27 09:39:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 297000960. Throughput: 0: 10453.1, 1: 10362.1. Samples: 297011752. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:02,363][19170] Avg episode reward: [(0, '971.390'), (1, '990.340')] [2023-12-27 09:39:02,438][20347] Updated weights for policy 0, policy_version 580130 (0.0008) [2023-12-27 09:39:02,488][20347] Updated weights for policy 0, policy_version 580140 (0.0007) [2023-12-27 09:39:02,536][20347] Updated weights for policy 0, policy_version 580150 (0.0008) [2023-12-27 09:39:02,592][20347] Updated weights for policy 0, policy_version 580160 (0.0008) [2023-12-27 09:39:02,853][20381] Updated weights for policy 1, policy_version 579896 (0.0010) [2023-12-27 09:39:02,913][20381] Updated weights for policy 1, policy_version 579906 (0.0008) [2023-12-27 09:39:02,974][20381] Updated weights for policy 1, policy_version 579916 (0.0011) [2023-12-27 09:39:03,212][20347] Updated weights for policy 0, policy_version 580170 (0.0007) [2023-12-27 09:39:03,269][20347] Updated weights for policy 0, policy_version 580180 (0.0007) [2023-12-27 09:39:03,318][20347] Updated weights for policy 0, policy_version 580190 (0.0007) [2023-12-27 09:39:03,654][20381] Updated weights for policy 1, policy_version 579926 (0.0011) [2023-12-27 09:39:03,714][20381] Updated weights for policy 1, policy_version 579936 (0.0010) [2023-12-27 09:39:03,775][20381] Updated weights for policy 1, policy_version 579946 (0.0010) [2023-12-27 09:39:04,005][20347] Updated weights for policy 0, policy_version 580200 (0.0007) [2023-12-27 09:39:04,063][20347] Updated weights for policy 0, policy_version 580210 (0.0007) [2023-12-27 09:39:04,126][20347] Updated weights for policy 0, policy_version 580220 (0.0009) [2023-12-27 09:39:04,427][20381] Updated weights for policy 1, policy_version 579956 (0.0008) [2023-12-27 09:39:04,495][20381] Updated weights for policy 1, policy_version 579966 (0.0007) [2023-12-27 09:39:04,564][20381] Updated weights for policy 1, policy_version 579976 (0.0005) [2023-12-27 09:39:04,798][20347] Updated weights for policy 0, policy_version 580230 (0.0006) [2023-12-27 09:39:04,863][20347] Updated weights for policy 0, policy_version 580240 (0.0006) [2023-12-27 09:39:04,929][20347] Updated weights for policy 0, policy_version 580250 (0.0007) [2023-12-27 09:39:05,202][20381] Updated weights for policy 1, policy_version 579986 (0.0006) [2023-12-27 09:39:05,265][20381] Updated weights for policy 1, policy_version 579996 (0.0007) [2023-12-27 09:39:05,325][20381] Updated weights for policy 1, policy_version 580006 (0.0006) [2023-12-27 09:39:05,384][20381] Updated weights for policy 1, policy_version 580016 (0.0005) [2023-12-27 09:39:05,574][20347] Updated weights for policy 0, policy_version 580260 (0.0007) [2023-12-27 09:39:05,631][20347] Updated weights for policy 0, policy_version 580270 (0.0008) [2023-12-27 09:39:05,683][20347] Updated weights for policy 0, policy_version 580280 (0.0007) [2023-12-27 09:39:05,951][20381] Updated weights for policy 1, policy_version 580026 (0.0008) [2023-12-27 09:39:05,998][20381] Updated weights for policy 1, policy_version 580036 (0.0010) [2023-12-27 09:39:06,051][20381] Updated weights for policy 1, policy_version 580046 (0.0008) [2023-12-27 09:39:06,322][20347] Updated weights for policy 0, policy_version 580290 (0.0007) [2023-12-27 09:39:06,375][20347] Updated weights for policy 0, policy_version 580300 (0.0007) [2023-12-27 09:39:06,434][20347] Updated weights for policy 0, policy_version 580310 (0.0008) [2023-12-27 09:39:06,493][20347] Updated weights for policy 0, policy_version 580320 (0.0007) [2023-12-27 09:39:06,833][20381] Updated weights for policy 1, policy_version 580056 (0.0006) [2023-12-27 09:39:06,908][20381] Updated weights for policy 1, policy_version 580066 (0.0006) [2023-12-27 09:39:06,968][20381] Updated weights for policy 1, policy_version 580076 (0.0005) [2023-12-27 09:39:07,143][20347] Updated weights for policy 0, policy_version 580330 (0.0008) [2023-12-27 09:39:07,195][20347] Updated weights for policy 0, policy_version 580340 (0.0007) [2023-12-27 09:39:07,265][20347] Updated weights for policy 0, policy_version 580350 (0.0007) [2023-12-27 09:39:07,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 297107456. Throughput: 0: 10454.3, 1: 10372.8. Samples: 297074436. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:07,363][19170] Avg episode reward: [(0, '971.390'), (1, '990.370')] [2023-12-27 09:39:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000580080_148520960.pth... [2023-12-27 09:39:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000580352_148586496.pth... [2023-12-27 09:39:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000579136_148275200.pth [2023-12-27 09:39:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000578864_148209664.pth [2023-12-27 09:39:07,487][20381] Updated weights for policy 1, policy_version 580086 (0.0005) [2023-12-27 09:39:07,547][20381] Updated weights for policy 1, policy_version 580096 (0.0005) [2023-12-27 09:39:07,612][20381] Updated weights for policy 1, policy_version 580106 (0.0006) [2023-12-27 09:39:07,947][20347] Updated weights for policy 0, policy_version 580360 (0.0009) [2023-12-27 09:39:07,995][20347] Updated weights for policy 0, policy_version 580370 (0.0010) [2023-12-27 09:39:08,049][20347] Updated weights for policy 0, policy_version 580380 (0.0010) [2023-12-27 09:39:08,267][20381] Updated weights for policy 1, policy_version 580116 (0.0007) [2023-12-27 09:39:08,322][20381] Updated weights for policy 1, policy_version 580126 (0.0007) [2023-12-27 09:39:08,387][20381] Updated weights for policy 1, policy_version 580136 (0.0009) [2023-12-27 09:39:08,740][20347] Updated weights for policy 0, policy_version 580390 (0.0010) [2023-12-27 09:39:08,792][20347] Updated weights for policy 0, policy_version 580400 (0.0010) [2023-12-27 09:39:08,844][20347] Updated weights for policy 0, policy_version 580410 (0.0010) [2023-12-27 09:39:09,087][20381] Updated weights for policy 1, policy_version 580146 (0.0008) [2023-12-27 09:39:09,139][20381] Updated weights for policy 1, policy_version 580156 (0.0011) [2023-12-27 09:39:09,198][20381] Updated weights for policy 1, policy_version 580166 (0.0011) [2023-12-27 09:39:09,262][20381] Updated weights for policy 1, policy_version 580176 (0.0010) [2023-12-27 09:39:09,557][20347] Updated weights for policy 0, policy_version 580420 (0.0009) [2023-12-27 09:39:09,621][20347] Updated weights for policy 0, policy_version 580430 (0.0010) [2023-12-27 09:39:09,681][20347] Updated weights for policy 0, policy_version 580440 (0.0010) [2023-12-27 09:39:09,941][20381] Updated weights for policy 1, policy_version 580186 (0.0010) [2023-12-27 09:39:09,998][20381] Updated weights for policy 1, policy_version 580196 (0.0011) [2023-12-27 09:39:10,058][20381] Updated weights for policy 1, policy_version 580206 (0.0011) [2023-12-27 09:39:10,366][20347] Updated weights for policy 0, policy_version 580450 (0.0010) [2023-12-27 09:39:10,423][20347] Updated weights for policy 0, policy_version 580460 (0.0008) [2023-12-27 09:39:10,476][20347] Updated weights for policy 0, policy_version 580470 (0.0007) [2023-12-27 09:39:10,531][20347] Updated weights for policy 0, policy_version 580480 (0.0007) [2023-12-27 09:39:10,746][20381] Updated weights for policy 1, policy_version 580216 (0.0009) [2023-12-27 09:39:10,809][20381] Updated weights for policy 1, policy_version 580226 (0.0008) [2023-12-27 09:39:10,871][20381] Updated weights for policy 1, policy_version 580236 (0.0008) [2023-12-27 09:39:11,222][20347] Updated weights for policy 0, policy_version 580490 (0.0008) [2023-12-27 09:39:11,289][20347] Updated weights for policy 0, policy_version 580500 (0.0008) [2023-12-27 09:39:11,363][20347] Updated weights for policy 0, policy_version 580510 (0.0008) [2023-12-27 09:39:11,563][20381] Updated weights for policy 1, policy_version 580246 (0.0010) [2023-12-27 09:39:11,626][20381] Updated weights for policy 1, policy_version 580256 (0.0010) [2023-12-27 09:39:11,679][20381] Updated weights for policy 1, policy_version 580266 (0.0009) [2023-12-27 09:39:12,064][20347] Updated weights for policy 0, policy_version 580520 (0.0007) [2023-12-27 09:39:12,120][20347] Updated weights for policy 0, policy_version 580530 (0.0008) [2023-12-27 09:39:12,176][20347] Updated weights for policy 0, policy_version 580540 (0.0008) [2023-12-27 09:39:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 297205760. Throughput: 0: 10425.9, 1: 10353.1. Samples: 297197680. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.740')] [2023-12-27 09:39:12,375][20381] Updated weights for policy 1, policy_version 580276 (0.0008) [2023-12-27 09:39:12,430][20381] Updated weights for policy 1, policy_version 580286 (0.0005) [2023-12-27 09:39:12,486][20381] Updated weights for policy 1, policy_version 580296 (0.0006) [2023-12-27 09:39:12,875][20347] Updated weights for policy 0, policy_version 580550 (0.0006) [2023-12-27 09:39:12,924][20347] Updated weights for policy 0, policy_version 580560 (0.0006) [2023-12-27 09:39:12,976][20347] Updated weights for policy 0, policy_version 580570 (0.0007) [2023-12-27 09:39:13,134][20381] Updated weights for policy 1, policy_version 580306 (0.0009) [2023-12-27 09:39:13,191][20381] Updated weights for policy 1, policy_version 580316 (0.0005) [2023-12-27 09:39:13,249][20381] Updated weights for policy 1, policy_version 580326 (0.0005) [2023-12-27 09:39:13,300][20381] Updated weights for policy 1, policy_version 580336 (0.0005) [2023-12-27 09:39:13,700][20347] Updated weights for policy 0, policy_version 580580 (0.0008) [2023-12-27 09:39:13,758][20347] Updated weights for policy 0, policy_version 580590 (0.0008) [2023-12-27 09:39:13,813][20347] Updated weights for policy 0, policy_version 580600 (0.0007) [2023-12-27 09:39:13,932][20381] Updated weights for policy 1, policy_version 580346 (0.0008) [2023-12-27 09:39:13,981][20381] Updated weights for policy 1, policy_version 580356 (0.0008) [2023-12-27 09:39:14,026][20381] Updated weights for policy 1, policy_version 580366 (0.0007) [2023-12-27 09:39:14,468][20347] Updated weights for policy 0, policy_version 580610 (0.0007) [2023-12-27 09:39:14,521][20347] Updated weights for policy 0, policy_version 580620 (0.0007) [2023-12-27 09:39:14,586][20347] Updated weights for policy 0, policy_version 580630 (0.0007) [2023-12-27 09:39:14,647][20347] Updated weights for policy 0, policy_version 580640 (0.0008) [2023-12-27 09:39:14,733][20381] Updated weights for policy 1, policy_version 580376 (0.0008) [2023-12-27 09:39:14,798][20381] Updated weights for policy 1, policy_version 580386 (0.0008) [2023-12-27 09:39:14,858][20381] Updated weights for policy 1, policy_version 580396 (0.0007) [2023-12-27 09:39:15,331][20347] Updated weights for policy 0, policy_version 580650 (0.0008) [2023-12-27 09:39:15,389][20347] Updated weights for policy 0, policy_version 580660 (0.0007) [2023-12-27 09:39:15,448][20347] Updated weights for policy 0, policy_version 580670 (0.0007) [2023-12-27 09:39:15,511][20381] Updated weights for policy 1, policy_version 580406 (0.0007) [2023-12-27 09:39:15,566][20381] Updated weights for policy 1, policy_version 580416 (0.0008) [2023-12-27 09:39:15,621][20381] Updated weights for policy 1, policy_version 580426 (0.0006) [2023-12-27 09:39:16,103][20347] Updated weights for policy 0, policy_version 580680 (0.0007) [2023-12-27 09:39:16,155][20347] Updated weights for policy 0, policy_version 580690 (0.0007) [2023-12-27 09:39:16,211][20347] Updated weights for policy 0, policy_version 580700 (0.0007) [2023-12-27 09:39:16,245][20381] Updated weights for policy 1, policy_version 580436 (0.0006) [2023-12-27 09:39:16,291][20381] Updated weights for policy 1, policy_version 580446 (0.0008) [2023-12-27 09:39:16,337][20381] Updated weights for policy 1, policy_version 580456 (0.0007) [2023-12-27 09:39:16,863][20347] Updated weights for policy 0, policy_version 580710 (0.0006) [2023-12-27 09:39:16,912][20347] Updated weights for policy 0, policy_version 580720 (0.0008) [2023-12-27 09:39:16,958][20347] Updated weights for policy 0, policy_version 580730 (0.0009) [2023-12-27 09:39:17,014][20381] Updated weights for policy 1, policy_version 580466 (0.0007) [2023-12-27 09:39:17,077][20381] Updated weights for policy 1, policy_version 580476 (0.0005) [2023-12-27 09:39:17,132][20381] Updated weights for policy 1, policy_version 580486 (0.0007) [2023-12-27 09:39:17,190][20381] Updated weights for policy 1, policy_version 580496 (0.0008) [2023-12-27 09:39:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 297312256. Throughput: 0: 10267.5, 1: 10360.0. Samples: 297320296. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.370')] [2023-12-27 09:39:17,687][20347] Updated weights for policy 0, policy_version 580740 (0.0007) [2023-12-27 09:39:17,740][20347] Updated weights for policy 0, policy_version 580750 (0.0008) [2023-12-27 09:39:17,795][20347] Updated weights for policy 0, policy_version 580760 (0.0007) [2023-12-27 09:39:17,859][20381] Updated weights for policy 1, policy_version 580506 (0.0008) [2023-12-27 09:39:17,919][20381] Updated weights for policy 1, policy_version 580516 (0.0007) [2023-12-27 09:39:17,965][20381] Updated weights for policy 1, policy_version 580526 (0.0005) [2023-12-27 09:39:18,484][20347] Updated weights for policy 0, policy_version 580770 (0.0006) [2023-12-27 09:39:18,544][20347] Updated weights for policy 0, policy_version 580780 (0.0007) [2023-12-27 09:39:18,546][20381] Updated weights for policy 1, policy_version 580536 (0.0005) [2023-12-27 09:39:18,605][20381] Updated weights for policy 1, policy_version 580546 (0.0005) [2023-12-27 09:39:18,606][20347] Updated weights for policy 0, policy_version 580790 (0.0007) [2023-12-27 09:39:18,658][20381] Updated weights for policy 1, policy_version 580556 (0.0005) [2023-12-27 09:39:18,659][20347] Updated weights for policy 0, policy_version 580800 (0.0006) [2023-12-27 09:39:19,362][20347] Updated weights for policy 0, policy_version 580810 (0.0006) [2023-12-27 09:39:19,368][20381] Updated weights for policy 1, policy_version 580566 (0.0008) [2023-12-27 09:39:19,420][20347] Updated weights for policy 0, policy_version 580820 (0.0006) [2023-12-27 09:39:19,428][20381] Updated weights for policy 1, policy_version 580576 (0.0006) [2023-12-27 09:39:19,482][20347] Updated weights for policy 0, policy_version 580830 (0.0006) [2023-12-27 09:39:19,485][20381] Updated weights for policy 1, policy_version 580586 (0.0007) [2023-12-27 09:39:20,112][20347] Updated weights for policy 0, policy_version 580840 (0.0008) [2023-12-27 09:39:20,162][20347] Updated weights for policy 0, policy_version 580850 (0.0005) [2023-12-27 09:39:20,213][20347] Updated weights for policy 0, policy_version 580860 (0.0005) [2023-12-27 09:39:20,252][20381] Updated weights for policy 1, policy_version 580596 (0.0008) [2023-12-27 09:39:20,305][20381] Updated weights for policy 1, policy_version 580606 (0.0008) [2023-12-27 09:39:20,365][20381] Updated weights for policy 1, policy_version 580616 (0.0008) [2023-12-27 09:39:20,832][20347] Updated weights for policy 0, policy_version 580870 (0.0005) [2023-12-27 09:39:20,890][20347] Updated weights for policy 0, policy_version 580880 (0.0006) [2023-12-27 09:39:20,950][20347] Updated weights for policy 0, policy_version 580890 (0.0008) [2023-12-27 09:39:21,087][20381] Updated weights for policy 1, policy_version 580626 (0.0009) [2023-12-27 09:39:21,155][20381] Updated weights for policy 1, policy_version 580636 (0.0006) [2023-12-27 09:39:21,205][20381] Updated weights for policy 1, policy_version 580646 (0.0006) [2023-12-27 09:39:21,259][20381] Updated weights for policy 1, policy_version 580656 (0.0007) [2023-12-27 09:39:21,597][20347] Updated weights for policy 0, policy_version 580900 (0.0007) [2023-12-27 09:39:21,662][20347] Updated weights for policy 0, policy_version 580910 (0.0008) [2023-12-27 09:39:21,730][20347] Updated weights for policy 0, policy_version 580920 (0.0008) [2023-12-27 09:39:21,963][20381] Updated weights for policy 1, policy_version 580666 (0.0008) [2023-12-27 09:39:22,024][20381] Updated weights for policy 1, policy_version 580676 (0.0008) [2023-12-27 09:39:22,084][20381] Updated weights for policy 1, policy_version 580686 (0.0008) [2023-12-27 09:39:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 297410560. Throughput: 0: 10251.8, 1: 10350.7. Samples: 297381296. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:22,363][19170] Avg episode reward: [(0, '980.910'), (1, '1000.000')] [2023-12-27 09:39:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000580688_148676608.pth... [2023-12-27 09:39:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000580928_148733952.pth... [2023-12-27 09:39:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000579472_148365312.pth [2023-12-27 09:39:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000579744_148430848.pth [2023-12-27 09:39:22,428][20347] Updated weights for policy 0, policy_version 580930 (0.0008) [2023-12-27 09:39:22,488][20347] Updated weights for policy 0, policy_version 580940 (0.0010) [2023-12-27 09:39:22,544][20347] Updated weights for policy 0, policy_version 580950 (0.0011) [2023-12-27 09:39:22,608][20347] Updated weights for policy 0, policy_version 580960 (0.0013) [2023-12-27 09:39:22,779][20381] Updated weights for policy 1, policy_version 580696 (0.0006) [2023-12-27 09:39:22,836][20381] Updated weights for policy 1, policy_version 580706 (0.0005) [2023-12-27 09:39:22,898][20381] Updated weights for policy 1, policy_version 580716 (0.0006) [2023-12-27 09:39:23,263][20347] Updated weights for policy 0, policy_version 580970 (0.0005) [2023-12-27 09:39:23,317][20347] Updated weights for policy 0, policy_version 580980 (0.0005) [2023-12-27 09:39:23,363][20347] Updated weights for policy 0, policy_version 580990 (0.0005) [2023-12-27 09:39:23,602][20381] Updated weights for policy 1, policy_version 580726 (0.0008) [2023-12-27 09:39:23,649][20381] Updated weights for policy 1, policy_version 580736 (0.0007) [2023-12-27 09:39:23,695][20381] Updated weights for policy 1, policy_version 580746 (0.0010) [2023-12-27 09:39:23,914][20347] Updated weights for policy 0, policy_version 581000 (0.0005) [2023-12-27 09:39:23,965][20347] Updated weights for policy 0, policy_version 581010 (0.0005) [2023-12-27 09:39:24,020][20347] Updated weights for policy 0, policy_version 581020 (0.0005) [2023-12-27 09:39:24,378][20381] Updated weights for policy 1, policy_version 580756 (0.0008) [2023-12-27 09:39:24,437][20381] Updated weights for policy 1, policy_version 580766 (0.0008) [2023-12-27 09:39:24,497][20381] Updated weights for policy 1, policy_version 580776 (0.0008) [2023-12-27 09:39:24,610][20347] Updated weights for policy 0, policy_version 581030 (0.0005) [2023-12-27 09:39:24,671][20347] Updated weights for policy 0, policy_version 581040 (0.0010) [2023-12-27 09:39:24,729][20347] Updated weights for policy 0, policy_version 581050 (0.0010) [2023-12-27 09:39:25,085][20381] Updated weights for policy 1, policy_version 580786 (0.0007) [2023-12-27 09:39:25,144][20381] Updated weights for policy 1, policy_version 580796 (0.0010) [2023-12-27 09:39:25,204][20381] Updated weights for policy 1, policy_version 580806 (0.0008) [2023-12-27 09:39:25,261][20381] Updated weights for policy 1, policy_version 580816 (0.0008) [2023-12-27 09:39:25,366][20347] Updated weights for policy 0, policy_version 581060 (0.0010) [2023-12-27 09:39:25,414][20347] Updated weights for policy 0, policy_version 581070 (0.0010) [2023-12-27 09:39:25,463][20347] Updated weights for policy 0, policy_version 581080 (0.0010) [2023-12-27 09:39:25,908][20381] Updated weights for policy 1, policy_version 580826 (0.0011) [2023-12-27 09:39:25,967][20381] Updated weights for policy 1, policy_version 580836 (0.0011) [2023-12-27 09:39:26,025][20381] Updated weights for policy 1, policy_version 580846 (0.0011) [2023-12-27 09:39:26,074][20347] Updated weights for policy 0, policy_version 581090 (0.0010) [2023-12-27 09:39:26,132][20347] Updated weights for policy 0, policy_version 581100 (0.0010) [2023-12-27 09:39:26,200][20347] Updated weights for policy 0, policy_version 581110 (0.0009) [2023-12-27 09:39:26,264][20347] Updated weights for policy 0, policy_version 581120 (0.0005) [2023-12-27 09:39:26,705][20381] Updated weights for policy 1, policy_version 580856 (0.0011) [2023-12-27 09:39:26,759][20381] Updated weights for policy 1, policy_version 580866 (0.0010) [2023-12-27 09:39:26,806][20381] Updated weights for policy 1, policy_version 580876 (0.0009) [2023-12-27 09:39:26,873][20347] Updated weights for policy 0, policy_version 581130 (0.0010) [2023-12-27 09:39:26,921][20347] Updated weights for policy 0, policy_version 581140 (0.0010) [2023-12-27 09:39:26,972][20347] Updated weights for policy 0, policy_version 581150 (0.0010) [2023-12-27 09:39:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 297517056. Throughput: 0: 10349.5, 1: 10363.2. Samples: 297507884. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:27,363][19170] Avg episode reward: [(0, '971.270'), (1, '1000.000')] [2023-12-27 09:39:27,415][20381] Updated weights for policy 1, policy_version 580886 (0.0010) [2023-12-27 09:39:27,482][20381] Updated weights for policy 1, policy_version 580896 (0.0010) [2023-12-27 09:39:27,545][20381] Updated weights for policy 1, policy_version 580906 (0.0010) [2023-12-27 09:39:27,637][20347] Updated weights for policy 0, policy_version 581160 (0.0010) [2023-12-27 09:39:27,688][20347] Updated weights for policy 0, policy_version 581170 (0.0010) [2023-12-27 09:39:27,745][20347] Updated weights for policy 0, policy_version 581180 (0.0010) [2023-12-27 09:39:28,198][20381] Updated weights for policy 1, policy_version 580916 (0.0010) [2023-12-27 09:39:28,252][20381] Updated weights for policy 1, policy_version 580926 (0.0009) [2023-12-27 09:39:28,319][20381] Updated weights for policy 1, policy_version 580936 (0.0010) [2023-12-27 09:39:28,417][20347] Updated weights for policy 0, policy_version 581190 (0.0010) [2023-12-27 09:39:28,478][20347] Updated weights for policy 0, policy_version 581200 (0.0010) [2023-12-27 09:39:28,529][20347] Updated weights for policy 0, policy_version 581210 (0.0010) [2023-12-27 09:39:28,969][20381] Updated weights for policy 1, policy_version 580946 (0.0010) [2023-12-27 09:39:29,024][20381] Updated weights for policy 1, policy_version 580956 (0.0008) [2023-12-27 09:39:29,078][20381] Updated weights for policy 1, policy_version 580966 (0.0007) [2023-12-27 09:39:29,137][20381] Updated weights for policy 1, policy_version 580976 (0.0007) [2023-12-27 09:39:29,212][20347] Updated weights for policy 0, policy_version 581220 (0.0010) [2023-12-27 09:39:29,275][20347] Updated weights for policy 0, policy_version 581230 (0.0009) [2023-12-27 09:39:29,333][20347] Updated weights for policy 0, policy_version 581240 (0.0010) [2023-12-27 09:39:29,854][20381] Updated weights for policy 1, policy_version 580986 (0.0008) [2023-12-27 09:39:29,900][20381] Updated weights for policy 1, policy_version 580996 (0.0007) [2023-12-27 09:39:29,956][20381] Updated weights for policy 1, policy_version 581006 (0.0008) [2023-12-27 09:39:29,994][20347] Updated weights for policy 0, policy_version 581250 (0.0009) [2023-12-27 09:39:30,054][20347] Updated weights for policy 0, policy_version 581260 (0.0005) [2023-12-27 09:39:30,113][20347] Updated weights for policy 0, policy_version 581270 (0.0005) [2023-12-27 09:39:30,173][20347] Updated weights for policy 0, policy_version 581280 (0.0006) [2023-12-27 09:39:30,703][20381] Updated weights for policy 1, policy_version 581016 (0.0009) [2023-12-27 09:39:30,728][20347] Updated weights for policy 0, policy_version 581290 (0.0006) [2023-12-27 09:39:30,751][20291] KL-divergence is very high: 561.1481 [2023-12-27 09:39:30,756][20381] Updated weights for policy 1, policy_version 581026 (0.0007) [2023-12-27 09:39:30,773][20347] Updated weights for policy 0, policy_version 581300 (0.0005) [2023-12-27 09:39:30,811][20381] Updated weights for policy 1, policy_version 581036 (0.0007) [2023-12-27 09:39:30,819][20347] Updated weights for policy 0, policy_version 581310 (0.0005) [2023-12-27 09:39:31,411][20347] Updated weights for policy 0, policy_version 581320 (0.0007) [2023-12-27 09:39:31,469][20347] Updated weights for policy 0, policy_version 581330 (0.0006) [2023-12-27 09:39:31,494][20381] Updated weights for policy 1, policy_version 581046 (0.0006) [2023-12-27 09:39:31,531][20347] Updated weights for policy 0, policy_version 581340 (0.0006) [2023-12-27 09:39:31,550][20381] Updated weights for policy 1, policy_version 581056 (0.0007) [2023-12-27 09:39:31,610][20381] Updated weights for policy 1, policy_version 581066 (0.0008) [2023-12-27 09:39:32,146][20347] Updated weights for policy 0, policy_version 581350 (0.0007) [2023-12-27 09:39:32,211][20347] Updated weights for policy 0, policy_version 581360 (0.0005) [2023-12-27 09:39:32,279][20347] Updated weights for policy 0, policy_version 581370 (0.0007) [2023-12-27 09:39:32,324][20381] Updated weights for policy 1, policy_version 581076 (0.0007) [2023-12-27 09:39:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 297623552. Throughput: 0: 10365.4, 1: 10326.4. Samples: 297634284. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:32,363][19170] Avg episode reward: [(0, '971.130'), (1, '1000.000')] [2023-12-27 09:39:32,384][20381] Updated weights for policy 1, policy_version 581086 (0.0006) [2023-12-27 09:39:32,440][20381] Updated weights for policy 1, policy_version 581096 (0.0005) [2023-12-27 09:39:32,933][20347] Updated weights for policy 0, policy_version 581380 (0.0008) [2023-12-27 09:39:32,980][20347] Updated weights for policy 0, policy_version 581390 (0.0007) [2023-12-27 09:39:33,035][20347] Updated weights for policy 0, policy_version 581400 (0.0008) [2023-12-27 09:39:33,082][20381] Updated weights for policy 1, policy_version 581106 (0.0007) [2023-12-27 09:39:33,138][20381] Updated weights for policy 1, policy_version 581116 (0.0008) [2023-12-27 09:39:33,190][20381] Updated weights for policy 1, policy_version 581126 (0.0007) [2023-12-27 09:39:33,244][20381] Updated weights for policy 1, policy_version 581136 (0.0008) [2023-12-27 09:39:33,694][20347] Updated weights for policy 0, policy_version 581410 (0.0007) [2023-12-27 09:39:33,749][20347] Updated weights for policy 0, policy_version 581420 (0.0006) [2023-12-27 09:39:33,803][20347] Updated weights for policy 0, policy_version 581430 (0.0005) [2023-12-27 09:39:33,851][20347] Updated weights for policy 0, policy_version 581440 (0.0007) [2023-12-27 09:39:33,958][20381] Updated weights for policy 1, policy_version 581146 (0.0008) [2023-12-27 09:39:34,014][20381] Updated weights for policy 1, policy_version 581156 (0.0007) [2023-12-27 09:39:34,070][20381] Updated weights for policy 1, policy_version 581166 (0.0008) [2023-12-27 09:39:34,513][20347] Updated weights for policy 0, policy_version 581450 (0.0007) [2023-12-27 09:39:34,564][20347] Updated weights for policy 0, policy_version 581460 (0.0007) [2023-12-27 09:39:34,611][20347] Updated weights for policy 0, policy_version 581470 (0.0007) [2023-12-27 09:39:34,796][20381] Updated weights for policy 1, policy_version 581176 (0.0007) [2023-12-27 09:39:34,841][20381] Updated weights for policy 1, policy_version 581186 (0.0010) [2023-12-27 09:39:34,898][20381] Updated weights for policy 1, policy_version 581196 (0.0010) [2023-12-27 09:39:35,328][20347] Updated weights for policy 0, policy_version 581480 (0.0007) [2023-12-27 09:39:35,388][20347] Updated weights for policy 0, policy_version 581490 (0.0006) [2023-12-27 09:39:35,435][20347] Updated weights for policy 0, policy_version 581500 (0.0007) [2023-12-27 09:39:35,532][20381] Updated weights for policy 1, policy_version 581206 (0.0010) [2023-12-27 09:39:35,578][20381] Updated weights for policy 1, policy_version 581216 (0.0010) [2023-12-27 09:39:35,627][20381] Updated weights for policy 1, policy_version 581226 (0.0010) [2023-12-27 09:39:36,154][20347] Updated weights for policy 0, policy_version 581510 (0.0009) [2023-12-27 09:39:36,214][20347] Updated weights for policy 0, policy_version 581520 (0.0008) [2023-12-27 09:39:36,272][20347] Updated weights for policy 0, policy_version 581530 (0.0007) [2023-12-27 09:39:36,281][20381] Updated weights for policy 1, policy_version 581236 (0.0008) [2023-12-27 09:39:36,341][20381] Updated weights for policy 1, policy_version 581246 (0.0008) [2023-12-27 09:39:36,424][20381] Updated weights for policy 1, policy_version 581256 (0.0010) [2023-12-27 09:39:36,995][20347] Updated weights for policy 0, policy_version 581540 (0.0008) [2023-12-27 09:39:37,053][20347] Updated weights for policy 0, policy_version 581550 (0.0007) [2023-12-27 09:39:37,057][20381] Updated weights for policy 1, policy_version 581266 (0.0010) [2023-12-27 09:39:37,112][20347] Updated weights for policy 0, policy_version 581560 (0.0007) [2023-12-27 09:39:37,116][20381] Updated weights for policy 1, policy_version 581276 (0.0007) [2023-12-27 09:39:37,175][20381] Updated weights for policy 1, policy_version 581286 (0.0007) [2023-12-27 09:39:37,226][20381] Updated weights for policy 1, policy_version 581296 (0.0008) [2023-12-27 09:39:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20854.9). Total num frames: 297730048. Throughput: 0: 10388.1, 1: 10298.4. Samples: 297696392. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:37,363][19170] Avg episode reward: [(0, '953.420'), (1, '1000.000')] [2023-12-27 09:39:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000581296_148832256.pth... [2023-12-27 09:39:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000581568_148897792.pth... [2023-12-27 09:39:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000580352_148586496.pth [2023-12-27 09:39:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000580080_148520960.pth [2023-12-27 09:39:37,736][20347] Updated weights for policy 0, policy_version 581570 (0.0006) [2023-12-27 09:39:37,784][20347] Updated weights for policy 0, policy_version 581580 (0.0007) [2023-12-27 09:39:37,824][20381] Updated weights for policy 1, policy_version 581306 (0.0006) [2023-12-27 09:39:37,830][20347] Updated weights for policy 0, policy_version 581590 (0.0010) [2023-12-27 09:39:37,875][20347] Updated weights for policy 0, policy_version 581600 (0.0010) [2023-12-27 09:39:37,878][20381] Updated weights for policy 1, policy_version 581316 (0.0006) [2023-12-27 09:39:37,939][20381] Updated weights for policy 1, policy_version 581326 (0.0005) [2023-12-27 09:39:38,538][20381] Updated weights for policy 1, policy_version 581336 (0.0006) [2023-12-27 09:39:38,558][20347] Updated weights for policy 0, policy_version 581610 (0.0010) [2023-12-27 09:39:38,591][20381] Updated weights for policy 1, policy_version 581346 (0.0006) [2023-12-27 09:39:38,613][20347] Updated weights for policy 0, policy_version 581620 (0.0010) [2023-12-27 09:39:38,645][20381] Updated weights for policy 1, policy_version 581356 (0.0006) [2023-12-27 09:39:38,669][20347] Updated weights for policy 0, policy_version 581630 (0.0010) [2023-12-27 09:39:39,345][20347] Updated weights for policy 0, policy_version 581640 (0.0010) [2023-12-27 09:39:39,383][20381] Updated weights for policy 1, policy_version 581366 (0.0006) [2023-12-27 09:39:39,409][20347] Updated weights for policy 0, policy_version 581650 (0.0008) [2023-12-27 09:39:39,445][20381] Updated weights for policy 1, policy_version 581376 (0.0008) [2023-12-27 09:39:39,468][20347] Updated weights for policy 0, policy_version 581660 (0.0010) [2023-12-27 09:39:39,501][20381] Updated weights for policy 1, policy_version 581386 (0.0006) [2023-12-27 09:39:40,184][20347] Updated weights for policy 0, policy_version 581670 (0.0011) [2023-12-27 09:39:40,237][20381] Updated weights for policy 1, policy_version 581396 (0.0008) [2023-12-27 09:39:40,240][20347] Updated weights for policy 0, policy_version 581680 (0.0008) [2023-12-27 09:39:40,290][20381] Updated weights for policy 1, policy_version 581406 (0.0007) [2023-12-27 09:39:40,303][20347] Updated weights for policy 0, policy_version 581690 (0.0008) [2023-12-27 09:39:40,350][20381] Updated weights for policy 1, policy_version 581416 (0.0006) [2023-12-27 09:39:40,970][20347] Updated weights for policy 0, policy_version 581700 (0.0007) [2023-12-27 09:39:41,032][20347] Updated weights for policy 0, policy_version 581710 (0.0010) [2023-12-27 09:39:41,044][20381] Updated weights for policy 1, policy_version 581426 (0.0008) [2023-12-27 09:39:41,097][20347] Updated weights for policy 0, policy_version 581720 (0.0009) [2023-12-27 09:39:41,108][20381] Updated weights for policy 1, policy_version 581436 (0.0007) [2023-12-27 09:39:41,178][20381] Updated weights for policy 1, policy_version 581446 (0.0009) [2023-12-27 09:39:41,247][20381] Updated weights for policy 1, policy_version 581456 (0.0009) [2023-12-27 09:39:41,802][20347] Updated weights for policy 0, policy_version 581730 (0.0010) [2023-12-27 09:39:41,865][20347] Updated weights for policy 0, policy_version 581740 (0.0010) [2023-12-27 09:39:41,924][20347] Updated weights for policy 0, policy_version 581750 (0.0010) [2023-12-27 09:39:41,982][20347] Updated weights for policy 0, policy_version 581760 (0.0010) [2023-12-27 09:39:42,028][20381] Updated weights for policy 1, policy_version 581466 (0.0005) [2023-12-27 09:39:42,091][20381] Updated weights for policy 1, policy_version 581476 (0.0006) [2023-12-27 09:39:42,157][20381] Updated weights for policy 1, policy_version 581486 (0.0007) [2023-12-27 09:39:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 297828352. Throughput: 0: 10379.0, 1: 10264.8. Samples: 297818256. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:42,363][19170] Avg episode reward: [(0, '963.010'), (1, '1000.000')] [2023-12-27 09:39:42,670][20347] Updated weights for policy 0, policy_version 581770 (0.0010) [2023-12-27 09:39:42,721][20347] Updated weights for policy 0, policy_version 581780 (0.0010) [2023-12-27 09:39:42,780][20347] Updated weights for policy 0, policy_version 581790 (0.0007) [2023-12-27 09:39:42,782][20381] Updated weights for policy 1, policy_version 581496 (0.0009) [2023-12-27 09:39:42,842][20381] Updated weights for policy 1, policy_version 581506 (0.0010) [2023-12-27 09:39:42,899][20381] Updated weights for policy 1, policy_version 581516 (0.0006) [2023-12-27 09:39:43,458][20347] Updated weights for policy 0, policy_version 581800 (0.0007) [2023-12-27 09:39:43,501][20381] Updated weights for policy 1, policy_version 581526 (0.0005) [2023-12-27 09:39:43,513][20347] Updated weights for policy 0, policy_version 581810 (0.0008) [2023-12-27 09:39:43,558][20381] Updated weights for policy 1, policy_version 581536 (0.0005) [2023-12-27 09:39:43,571][20347] Updated weights for policy 0, policy_version 581820 (0.0008) [2023-12-27 09:39:43,613][20381] Updated weights for policy 1, policy_version 581546 (0.0007) [2023-12-27 09:39:44,162][20381] Updated weights for policy 1, policy_version 581556 (0.0008) [2023-12-27 09:39:44,167][20347] Updated weights for policy 0, policy_version 581830 (0.0006) [2023-12-27 09:39:44,215][20381] Updated weights for policy 1, policy_version 581566 (0.0007) [2023-12-27 09:39:44,217][20347] Updated weights for policy 0, policy_version 581840 (0.0005) [2023-12-27 09:39:44,270][20347] Updated weights for policy 0, policy_version 581850 (0.0005) [2023-12-27 09:39:44,271][20381] Updated weights for policy 1, policy_version 581576 (0.0007) [2023-12-27 09:39:44,890][20381] Updated weights for policy 1, policy_version 581586 (0.0008) [2023-12-27 09:39:44,921][20347] Updated weights for policy 0, policy_version 581860 (0.0006) [2023-12-27 09:39:44,956][20381] Updated weights for policy 1, policy_version 581596 (0.0008) [2023-12-27 09:39:44,977][20347] Updated weights for policy 0, policy_version 581870 (0.0007) [2023-12-27 09:39:45,020][20381] Updated weights for policy 1, policy_version 581606 (0.0008) [2023-12-27 09:39:45,026][20347] Updated weights for policy 0, policy_version 581880 (0.0006) [2023-12-27 09:39:45,090][20381] Updated weights for policy 1, policy_version 581616 (0.0008) [2023-12-27 09:39:45,730][20347] Updated weights for policy 0, policy_version 581890 (0.0007) [2023-12-27 09:39:45,788][20347] Updated weights for policy 0, policy_version 581900 (0.0010) [2023-12-27 09:39:45,804][20381] Updated weights for policy 1, policy_version 581626 (0.0005) [2023-12-27 09:39:45,849][20347] Updated weights for policy 0, policy_version 581910 (0.0010) [2023-12-27 09:39:45,861][20381] Updated weights for policy 1, policy_version 581636 (0.0005) [2023-12-27 09:39:45,907][20347] Updated weights for policy 0, policy_version 581920 (0.0010) [2023-12-27 09:39:45,912][20381] Updated weights for policy 1, policy_version 581646 (0.0005) [2023-12-27 09:39:46,509][20347] Updated weights for policy 0, policy_version 581930 (0.0010) [2023-12-27 09:39:46,542][20381] Updated weights for policy 1, policy_version 581656 (0.0007) [2023-12-27 09:39:46,561][20347] Updated weights for policy 0, policy_version 581940 (0.0010) [2023-12-27 09:39:46,606][20381] Updated weights for policy 1, policy_version 581666 (0.0008) [2023-12-27 09:39:46,617][20347] Updated weights for policy 0, policy_version 581950 (0.0009) [2023-12-27 09:39:46,671][20381] Updated weights for policy 1, policy_version 581676 (0.0008) [2023-12-27 09:39:47,278][20381] Updated weights for policy 1, policy_version 581686 (0.0007) [2023-12-27 09:39:47,299][20347] Updated weights for policy 0, policy_version 581960 (0.0007) [2023-12-27 09:39:47,333][20381] Updated weights for policy 1, policy_version 581696 (0.0008) [2023-12-27 09:39:47,350][20347] Updated weights for policy 0, policy_version 581970 (0.0006) [2023-12-27 09:39:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 297926656. Throughput: 0: 10394.6, 1: 10318.9. Samples: 297943860. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:47,363][19170] Avg episode reward: [(0, '980.790'), (1, '1000.000')] [2023-12-27 09:39:47,382][20381] Updated weights for policy 1, policy_version 581706 (0.0006) [2023-12-27 09:39:47,403][20347] Updated weights for policy 0, policy_version 581980 (0.0006) [2023-12-27 09:39:48,083][20347] Updated weights for policy 0, policy_version 581990 (0.0006) [2023-12-27 09:39:48,092][20381] Updated weights for policy 1, policy_version 581716 (0.0006) [2023-12-27 09:39:48,142][20347] Updated weights for policy 0, policy_version 582000 (0.0007) [2023-12-27 09:39:48,148][20381] Updated weights for policy 1, policy_version 581726 (0.0006) [2023-12-27 09:39:48,204][20347] Updated weights for policy 0, policy_version 582010 (0.0006) [2023-12-27 09:39:48,206][20381] Updated weights for policy 1, policy_version 581736 (0.0008) [2023-12-27 09:39:48,822][20381] Updated weights for policy 1, policy_version 581746 (0.0006) [2023-12-27 09:39:48,875][20381] Updated weights for policy 1, policy_version 581756 (0.0006) [2023-12-27 09:39:48,901][20347] Updated weights for policy 0, policy_version 582020 (0.0007) [2023-12-27 09:39:48,928][20381] Updated weights for policy 1, policy_version 581766 (0.0005) [2023-12-27 09:39:48,960][20347] Updated weights for policy 0, policy_version 582030 (0.0007) [2023-12-27 09:39:48,980][20381] Updated weights for policy 1, policy_version 581776 (0.0006) [2023-12-27 09:39:49,008][20347] Updated weights for policy 0, policy_version 582040 (0.0008) [2023-12-27 09:39:49,624][20381] Updated weights for policy 1, policy_version 581786 (0.0006) [2023-12-27 09:39:49,654][20292] KL-divergence is very high: 128.0864 [2023-12-27 09:39:49,683][20381] Updated weights for policy 1, policy_version 581796 (0.0006) [2023-12-27 09:39:49,744][20381] Updated weights for policy 1, policy_version 581806 (0.0006) [2023-12-27 09:39:49,753][20347] Updated weights for policy 0, policy_version 582050 (0.0007) [2023-12-27 09:39:49,805][20347] Updated weights for policy 0, policy_version 582060 (0.0007) [2023-12-27 09:39:49,844][20291] KL-divergence is very high: 134.0985 [2023-12-27 09:39:49,867][20347] Updated weights for policy 0, policy_version 582070 (0.0009) [2023-12-27 09:39:49,922][20347] Updated weights for policy 0, policy_version 582080 (0.0009) [2023-12-27 09:39:50,395][20381] Updated weights for policy 1, policy_version 581816 (0.0008) [2023-12-27 09:39:50,448][20381] Updated weights for policy 1, policy_version 581826 (0.0008) [2023-12-27 09:39:50,508][20381] Updated weights for policy 1, policy_version 581836 (0.0007) [2023-12-27 09:39:50,634][20347] Updated weights for policy 0, policy_version 582090 (0.0008) [2023-12-27 09:39:50,685][20347] Updated weights for policy 0, policy_version 582100 (0.0009) [2023-12-27 09:39:50,732][20347] Updated weights for policy 0, policy_version 582110 (0.0009) [2023-12-27 09:39:51,109][20381] Updated weights for policy 1, policy_version 581846 (0.0007) [2023-12-27 09:39:51,166][20381] Updated weights for policy 1, policy_version 581856 (0.0008) [2023-12-27 09:39:51,228][20381] Updated weights for policy 1, policy_version 581866 (0.0008) [2023-12-27 09:39:51,475][20347] Updated weights for policy 0, policy_version 582120 (0.0009) [2023-12-27 09:39:51,524][20347] Updated weights for policy 0, policy_version 582130 (0.0008) [2023-12-27 09:39:51,572][20347] Updated weights for policy 0, policy_version 582140 (0.0008) [2023-12-27 09:39:51,953][20381] Updated weights for policy 1, policy_version 581876 (0.0008) [2023-12-27 09:39:52,010][20381] Updated weights for policy 1, policy_version 581886 (0.0008) [2023-12-27 09:39:52,069][20381] Updated weights for policy 1, policy_version 581896 (0.0007) [2023-12-27 09:39:52,322][20347] Updated weights for policy 0, policy_version 582150 (0.0007) [2023-12-27 09:39:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20799.4). Total num frames: 298033152. Throughput: 0: 10346.4, 1: 10329.3. Samples: 298004844. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:52,363][19170] Avg episode reward: [(0, '961.650'), (1, '990.300')] [2023-12-27 09:39:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000581904_148987904.pth... [2023-12-27 09:39:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000580688_148676608.pth [2023-12-27 09:39:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000581904_148987904.pth [2023-12-27 09:39:52,390][20347] Updated weights for policy 0, policy_version 582160 (0.0007) [2023-12-27 09:39:52,456][20347] Updated weights for policy 0, policy_version 582170 (0.0008) [2023-12-27 09:39:52,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000582176_149053440.pth... [2023-12-27 09:39:52,495][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000580928_148733952.pth [2023-12-27 09:39:52,495][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000582176_149053440.pth [2023-12-27 09:39:52,781][20381] Updated weights for policy 1, policy_version 581906 (0.0008) [2023-12-27 09:39:52,846][20381] Updated weights for policy 1, policy_version 581916 (0.0011) [2023-12-27 09:39:52,907][20381] Updated weights for policy 1, policy_version 581926 (0.0010) [2023-12-27 09:39:52,965][20381] Updated weights for policy 1, policy_version 581936 (0.0010) [2023-12-27 09:39:53,089][20347] Updated weights for policy 0, policy_version 582180 (0.0008) [2023-12-27 09:39:53,145][20347] Updated weights for policy 0, policy_version 582190 (0.0008) [2023-12-27 09:39:53,151][20291] KL-divergence is very high: 154.0989 [2023-12-27 09:39:53,195][20347] Updated weights for policy 0, policy_version 582200 (0.0007) [2023-12-27 09:39:53,616][20381] Updated weights for policy 1, policy_version 581946 (0.0007) [2023-12-27 09:39:53,672][20381] Updated weights for policy 1, policy_version 581956 (0.0007) [2023-12-27 09:39:53,725][20381] Updated weights for policy 1, policy_version 581966 (0.0009) [2023-12-27 09:39:53,862][20347] Updated weights for policy 0, policy_version 582210 (0.0008) [2023-12-27 09:39:53,915][20347] Updated weights for policy 0, policy_version 582220 (0.0010) [2023-12-27 09:39:53,926][20291] KL-divergence is very high: 171.2860 [2023-12-27 09:39:53,969][20347] Updated weights for policy 0, policy_version 582230 (0.0010) [2023-12-27 09:39:54,023][20347] Updated weights for policy 0, policy_version 582240 (0.0008) [2023-12-27 09:39:54,422][20381] Updated weights for policy 1, policy_version 581976 (0.0007) [2023-12-27 09:39:54,475][20381] Updated weights for policy 1, policy_version 581986 (0.0007) [2023-12-27 09:39:54,525][20381] Updated weights for policy 1, policy_version 581996 (0.0007) [2023-12-27 09:39:54,640][20291] KL-divergence is very high: 126.8293 [2023-12-27 09:39:54,649][20347] Updated weights for policy 0, policy_version 582250 (0.0007) [2023-12-27 09:39:54,654][20291] KL-divergence is very high: 107.6973 [2023-12-27 09:39:54,698][20347] Updated weights for policy 0, policy_version 582260 (0.0007) [2023-12-27 09:39:54,746][20347] Updated weights for policy 0, policy_version 582270 (0.0005) [2023-12-27 09:39:55,154][20381] Updated weights for policy 1, policy_version 582006 (0.0006) [2023-12-27 09:39:55,204][20381] Updated weights for policy 1, policy_version 582016 (0.0005) [2023-12-27 09:39:55,256][20381] Updated weights for policy 1, policy_version 582026 (0.0005) [2023-12-27 09:39:55,385][20347] Updated weights for policy 0, policy_version 582280 (0.0008) [2023-12-27 09:39:55,441][20347] Updated weights for policy 0, policy_version 582290 (0.0007) [2023-12-27 09:39:55,495][20347] Updated weights for policy 0, policy_version 582300 (0.0007) [2023-12-27 09:39:55,820][20381] Updated weights for policy 1, policy_version 582036 (0.0008) [2023-12-27 09:39:55,873][20381] Updated weights for policy 1, policy_version 582046 (0.0006) [2023-12-27 09:39:55,926][20381] Updated weights for policy 1, policy_version 582056 (0.0005) [2023-12-27 09:39:56,229][20347] Updated weights for policy 0, policy_version 582310 (0.0007) [2023-12-27 09:39:56,273][20347] Updated weights for policy 0, policy_version 582320 (0.0007) [2023-12-27 09:39:56,324][20347] Updated weights for policy 0, policy_version 582330 (0.0007) [2023-12-27 09:39:56,511][20381] Updated weights for policy 1, policy_version 582066 (0.0006) [2023-12-27 09:39:56,554][20381] Updated weights for policy 1, policy_version 582076 (0.0010) [2023-12-27 09:39:56,602][20381] Updated weights for policy 1, policy_version 582086 (0.0010) [2023-12-27 09:39:56,659][20381] Updated weights for policy 1, policy_version 582096 (0.0010) [2023-12-27 09:39:57,024][20347] Updated weights for policy 0, policy_version 582340 (0.0007) [2023-12-27 09:39:57,074][20347] Updated weights for policy 0, policy_version 582350 (0.0007) [2023-12-27 09:39:57,128][20347] Updated weights for policy 0, policy_version 582360 (0.0007) [2023-12-27 09:39:57,342][20381] Updated weights for policy 1, policy_version 582106 (0.0007) [2023-12-27 09:39:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 298139648. Throughput: 0: 10355.1, 1: 10365.2. Samples: 298130096. Policy #0 lag: (min: 35.0, avg: 47.3, max: 48.0) [2023-12-27 09:39:57,363][19170] Avg episode reward: [(0, '847.880'), (1, '980.640')] [2023-12-27 09:39:57,402][20381] Updated weights for policy 1, policy_version 582116 (0.0010) [2023-12-27 09:39:57,465][20381] Updated weights for policy 1, policy_version 582126 (0.0008) [2023-12-27 09:39:57,707][20347] Updated weights for policy 0, policy_version 582370 (0.0006) [2023-12-27 09:39:57,758][20347] Updated weights for policy 0, policy_version 582380 (0.0005) [2023-12-27 09:39:57,810][20347] Updated weights for policy 0, policy_version 582390 (0.0005) [2023-12-27 09:39:57,865][20347] Updated weights for policy 0, policy_version 582400 (0.0005) [2023-12-27 09:39:58,007][20381] Updated weights for policy 1, policy_version 582136 (0.0009) [2023-12-27 09:39:58,051][20381] Updated weights for policy 1, policy_version 582146 (0.0010) [2023-12-27 09:39:58,102][20381] Updated weights for policy 1, policy_version 582156 (0.0010) [2023-12-27 09:39:58,501][20347] Updated weights for policy 0, policy_version 582410 (0.0007) [2023-12-27 09:39:58,563][20347] Updated weights for policy 0, policy_version 582420 (0.0006) [2023-12-27 09:39:58,625][20347] Updated weights for policy 0, policy_version 582430 (0.0006) [2023-12-27 09:39:58,839][20381] Updated weights for policy 1, policy_version 582166 (0.0009) [2023-12-27 09:39:58,892][20381] Updated weights for policy 1, policy_version 582176 (0.0008) [2023-12-27 09:39:58,939][20381] Updated weights for policy 1, policy_version 582186 (0.0008) [2023-12-27 09:39:59,218][20347] Updated weights for policy 0, policy_version 582440 (0.0006) [2023-12-27 09:39:59,277][20347] Updated weights for policy 0, policy_version 582450 (0.0006) [2023-12-27 09:39:59,339][20347] Updated weights for policy 0, policy_version 582460 (0.0008) [2023-12-27 09:39:59,640][20381] Updated weights for policy 1, policy_version 582196 (0.0007) [2023-12-27 09:39:59,703][20381] Updated weights for policy 1, policy_version 582206 (0.0005) [2023-12-27 09:39:59,766][20381] Updated weights for policy 1, policy_version 582216 (0.0005) [2023-12-27 09:39:59,922][20347] Updated weights for policy 0, policy_version 582470 (0.0008) [2023-12-27 09:39:59,976][20347] Updated weights for policy 0, policy_version 582480 (0.0006) [2023-12-27 09:40:00,037][20347] Updated weights for policy 0, policy_version 582490 (0.0006) [2023-12-27 09:40:00,388][20381] Updated weights for policy 1, policy_version 582226 (0.0006) [2023-12-27 09:40:00,441][20381] Updated weights for policy 1, policy_version 582236 (0.0006) [2023-12-27 09:40:00,487][20381] Updated weights for policy 1, policy_version 582246 (0.0007) [2023-12-27 09:40:00,543][20381] Updated weights for policy 1, policy_version 582256 (0.0008) [2023-12-27 09:40:00,606][20347] Updated weights for policy 0, policy_version 582500 (0.0005) [2023-12-27 09:40:00,657][20347] Updated weights for policy 0, policy_version 582510 (0.0008) [2023-12-27 09:40:00,700][20347] Updated weights for policy 0, policy_version 582520 (0.0007) [2023-12-27 09:40:01,228][20381] Updated weights for policy 1, policy_version 582266 (0.0008) [2023-12-27 09:40:01,290][20381] Updated weights for policy 1, policy_version 582276 (0.0008) [2023-12-27 09:40:01,348][20381] Updated weights for policy 1, policy_version 582286 (0.0008) [2023-12-27 09:40:01,395][20347] Updated weights for policy 0, policy_version 582530 (0.0007) [2023-12-27 09:40:01,453][20347] Updated weights for policy 0, policy_version 582540 (0.0006) [2023-12-27 09:40:01,501][20347] Updated weights for policy 0, policy_version 582550 (0.0006) [2023-12-27 09:40:01,557][20347] Updated weights for policy 0, policy_version 582560 (0.0007) [2023-12-27 09:40:02,050][20381] Updated weights for policy 1, policy_version 582296 (0.0008) [2023-12-27 09:40:02,101][20381] Updated weights for policy 1, policy_version 582306 (0.0007) [2023-12-27 09:40:02,151][20381] Updated weights for policy 1, policy_version 582316 (0.0008) [2023-12-27 09:40:02,247][20347] Updated weights for policy 0, policy_version 582570 (0.0007) [2023-12-27 09:40:02,310][20347] Updated weights for policy 0, policy_version 582580 (0.0006) [2023-12-27 09:40:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 298246144. Throughput: 0: 10483.0, 1: 10353.8. Samples: 298257948. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:02,363][19170] Avg episode reward: [(0, '857.340'), (1, '932.280')] [2023-12-27 09:40:02,370][20347] Updated weights for policy 0, policy_version 582590 (0.0006) [2023-12-27 09:40:02,842][20381] Updated weights for policy 1, policy_version 582326 (0.0008) [2023-12-27 09:40:02,887][20381] Updated weights for policy 1, policy_version 582336 (0.0008) [2023-12-27 09:40:02,943][20381] Updated weights for policy 1, policy_version 582346 (0.0008) [2023-12-27 09:40:03,041][20347] Updated weights for policy 0, policy_version 582600 (0.0009) [2023-12-27 09:40:03,097][20347] Updated weights for policy 0, policy_version 582610 (0.0008) [2023-12-27 09:40:03,144][20347] Updated weights for policy 0, policy_version 582620 (0.0007) [2023-12-27 09:40:03,528][20381] Updated weights for policy 1, policy_version 582356 (0.0009) [2023-12-27 09:40:03,583][20381] Updated weights for policy 1, policy_version 582366 (0.0010) [2023-12-27 09:40:03,634][20381] Updated weights for policy 1, policy_version 582376 (0.0010) [2023-12-27 09:40:03,820][20347] Updated weights for policy 0, policy_version 582630 (0.0006) [2023-12-27 09:40:03,872][20347] Updated weights for policy 0, policy_version 582640 (0.0008) [2023-12-27 09:40:03,926][20347] Updated weights for policy 0, policy_version 582650 (0.0008) [2023-12-27 09:40:04,305][20381] Updated weights for policy 1, policy_version 582386 (0.0010) [2023-12-27 09:40:04,353][20381] Updated weights for policy 1, policy_version 582396 (0.0007) [2023-12-27 09:40:04,414][20381] Updated weights for policy 1, policy_version 582406 (0.0007) [2023-12-27 09:40:04,469][20381] Updated weights for policy 1, policy_version 582416 (0.0006) [2023-12-27 09:40:04,603][20347] Updated weights for policy 0, policy_version 582660 (0.0008) [2023-12-27 09:40:04,668][20347] Updated weights for policy 0, policy_version 582670 (0.0008) [2023-12-27 09:40:04,730][20347] Updated weights for policy 0, policy_version 582680 (0.0007) [2023-12-27 09:40:05,042][20381] Updated weights for policy 1, policy_version 582426 (0.0005) [2023-12-27 09:40:05,091][20381] Updated weights for policy 1, policy_version 582436 (0.0005) [2023-12-27 09:40:05,137][20381] Updated weights for policy 1, policy_version 582446 (0.0005) [2023-12-27 09:40:05,315][20347] Updated weights for policy 0, policy_version 582690 (0.0008) [2023-12-27 09:40:05,369][20347] Updated weights for policy 0, policy_version 582700 (0.0008) [2023-12-27 09:40:05,428][20347] Updated weights for policy 0, policy_version 582710 (0.0006) [2023-12-27 09:40:05,485][20347] Updated weights for policy 0, policy_version 582720 (0.0007) [2023-12-27 09:40:05,777][20381] Updated weights for policy 1, policy_version 582456 (0.0009) [2023-12-27 09:40:05,827][20381] Updated weights for policy 1, policy_version 582466 (0.0010) [2023-12-27 09:40:05,882][20381] Updated weights for policy 1, policy_version 582476 (0.0010) [2023-12-27 09:40:06,182][20347] Updated weights for policy 0, policy_version 582730 (0.0007) [2023-12-27 09:40:06,245][20347] Updated weights for policy 0, policy_version 582740 (0.0008) [2023-12-27 09:40:06,313][20347] Updated weights for policy 0, policy_version 582750 (0.0007) [2023-12-27 09:40:06,580][20381] Updated weights for policy 1, policy_version 582486 (0.0011) [2023-12-27 09:40:06,633][20381] Updated weights for policy 1, policy_version 582496 (0.0011) [2023-12-27 09:40:06,690][20381] Updated weights for policy 1, policy_version 582506 (0.0009) [2023-12-27 09:40:06,982][20347] Updated weights for policy 0, policy_version 582760 (0.0006) [2023-12-27 09:40:07,049][20347] Updated weights for policy 0, policy_version 582770 (0.0007) [2023-12-27 09:40:07,111][20347] Updated weights for policy 0, policy_version 582780 (0.0011) [2023-12-27 09:40:07,311][20381] Updated weights for policy 1, policy_version 582516 (0.0009) [2023-12-27 09:40:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 298352640. Throughput: 0: 10492.7, 1: 10424.3. Samples: 298322560. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:07,363][19170] Avg episode reward: [(0, '904.820'), (1, '922.740')] [2023-12-27 09:40:07,366][20381] Updated weights for policy 1, policy_version 582526 (0.0010) [2023-12-27 09:40:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000582784_149209088.pth... [2023-12-27 09:40:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000581568_148897792.pth [2023-12-27 09:40:07,417][20381] Updated weights for policy 1, policy_version 582536 (0.0010) [2023-12-27 09:40:07,460][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000582544_149151744.pth... [2023-12-27 09:40:07,464][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000581296_148832256.pth [2023-12-27 09:40:07,740][20347] Updated weights for policy 0, policy_version 582790 (0.0010) [2023-12-27 09:40:07,792][20347] Updated weights for policy 0, policy_version 582800 (0.0010) [2023-12-27 09:40:07,841][20347] Updated weights for policy 0, policy_version 582810 (0.0010) [2023-12-27 09:40:08,083][20381] Updated weights for policy 1, policy_version 582546 (0.0010) [2023-12-27 09:40:08,144][20381] Updated weights for policy 1, policy_version 582556 (0.0011) [2023-12-27 09:40:08,206][20381] Updated weights for policy 1, policy_version 582566 (0.0010) [2023-12-27 09:40:08,258][20381] Updated weights for policy 1, policy_version 582576 (0.0010) [2023-12-27 09:40:08,511][20347] Updated weights for policy 0, policy_version 582820 (0.0009) [2023-12-27 09:40:08,563][20347] Updated weights for policy 0, policy_version 582830 (0.0006) [2023-12-27 09:40:08,621][20347] Updated weights for policy 0, policy_version 582840 (0.0007) [2023-12-27 09:40:08,935][20381] Updated weights for policy 1, policy_version 582586 (0.0011) [2023-12-27 09:40:08,980][20381] Updated weights for policy 1, policy_version 582596 (0.0010) [2023-12-27 09:40:09,025][20381] Updated weights for policy 1, policy_version 582606 (0.0010) [2023-12-27 09:40:09,259][20347] Updated weights for policy 0, policy_version 582850 (0.0007) [2023-12-27 09:40:09,317][20347] Updated weights for policy 0, policy_version 582860 (0.0010) [2023-12-27 09:40:09,387][20347] Updated weights for policy 0, policy_version 582870 (0.0009) [2023-12-27 09:40:09,448][20347] Updated weights for policy 0, policy_version 582880 (0.0010) [2023-12-27 09:40:09,770][20381] Updated weights for policy 1, policy_version 582616 (0.0007) [2023-12-27 09:40:09,825][20381] Updated weights for policy 1, policy_version 582626 (0.0006) [2023-12-27 09:40:09,886][20381] Updated weights for policy 1, policy_version 582636 (0.0007) [2023-12-27 09:40:10,157][20347] Updated weights for policy 0, policy_version 582890 (0.0006) [2023-12-27 09:40:10,216][20347] Updated weights for policy 0, policy_version 582900 (0.0005) [2023-12-27 09:40:10,285][20347] Updated weights for policy 0, policy_version 582910 (0.0006) [2023-12-27 09:40:10,482][20381] Updated weights for policy 1, policy_version 582646 (0.0007) [2023-12-27 09:40:10,531][20381] Updated weights for policy 1, policy_version 582656 (0.0008) [2023-12-27 09:40:10,580][20381] Updated weights for policy 1, policy_version 582666 (0.0007) [2023-12-27 09:40:10,919][20347] Updated weights for policy 0, policy_version 582920 (0.0007) [2023-12-27 09:40:10,972][20347] Updated weights for policy 0, policy_version 582930 (0.0010) [2023-12-27 09:40:11,025][20347] Updated weights for policy 0, policy_version 582940 (0.0007) [2023-12-27 09:40:11,239][20381] Updated weights for policy 1, policy_version 582676 (0.0007) [2023-12-27 09:40:11,302][20381] Updated weights for policy 1, policy_version 582686 (0.0008) [2023-12-27 09:40:11,371][20381] Updated weights for policy 1, policy_version 582696 (0.0007) [2023-12-27 09:40:11,736][20347] Updated weights for policy 0, policy_version 582950 (0.0008) [2023-12-27 09:40:11,797][20347] Updated weights for policy 0, policy_version 582960 (0.0007) [2023-12-27 09:40:11,852][20347] Updated weights for policy 0, policy_version 582970 (0.0007) [2023-12-27 09:40:12,076][20381] Updated weights for policy 1, policy_version 582706 (0.0008) [2023-12-27 09:40:12,136][20381] Updated weights for policy 1, policy_version 582716 (0.0008) [2023-12-27 09:40:12,192][20381] Updated weights for policy 1, policy_version 582726 (0.0007) [2023-12-27 09:40:12,250][20381] Updated weights for policy 1, policy_version 582736 (0.0007) [2023-12-27 09:40:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 298459136. Throughput: 0: 10407.3, 1: 10469.4. Samples: 298447336. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:12,363][19170] Avg episode reward: [(0, '933.900'), (1, '884.290')] [2023-12-27 09:40:12,589][20347] Updated weights for policy 0, policy_version 582980 (0.0008) [2023-12-27 09:40:12,648][20347] Updated weights for policy 0, policy_version 582990 (0.0008) [2023-12-27 09:40:12,698][20347] Updated weights for policy 0, policy_version 583000 (0.0006) [2023-12-27 09:40:12,919][20381] Updated weights for policy 1, policy_version 582746 (0.0008) [2023-12-27 09:40:12,976][20381] Updated weights for policy 1, policy_version 582756 (0.0007) [2023-12-27 09:40:13,029][20381] Updated weights for policy 1, policy_version 582766 (0.0008) [2023-12-27 09:40:13,336][20347] Updated weights for policy 0, policy_version 583010 (0.0006) [2023-12-27 09:40:13,386][20347] Updated weights for policy 0, policy_version 583020 (0.0007) [2023-12-27 09:40:13,435][20347] Updated weights for policy 0, policy_version 583030 (0.0005) [2023-12-27 09:40:13,486][20347] Updated weights for policy 0, policy_version 583040 (0.0009) [2023-12-27 09:40:13,743][20381] Updated weights for policy 1, policy_version 582776 (0.0007) [2023-12-27 09:40:13,793][20381] Updated weights for policy 1, policy_version 582786 (0.0007) [2023-12-27 09:40:13,841][20381] Updated weights for policy 1, policy_version 582796 (0.0009) [2023-12-27 09:40:14,123][20347] Updated weights for policy 0, policy_version 583050 (0.0008) [2023-12-27 09:40:14,187][20347] Updated weights for policy 0, policy_version 583060 (0.0009) [2023-12-27 09:40:14,258][20347] Updated weights for policy 0, policy_version 583070 (0.0007) [2023-12-27 09:40:14,570][20381] Updated weights for policy 1, policy_version 582806 (0.0007) [2023-12-27 09:40:14,622][20381] Updated weights for policy 1, policy_version 582816 (0.0007) [2023-12-27 09:40:14,674][20381] Updated weights for policy 1, policy_version 582826 (0.0008) [2023-12-27 09:40:14,886][20347] Updated weights for policy 0, policy_version 583080 (0.0010) [2023-12-27 09:40:14,948][20347] Updated weights for policy 0, policy_version 583090 (0.0010) [2023-12-27 09:40:15,001][20347] Updated weights for policy 0, policy_version 583100 (0.0010) [2023-12-27 09:40:15,320][20381] Updated weights for policy 1, policy_version 582836 (0.0008) [2023-12-27 09:40:15,383][20381] Updated weights for policy 1, policy_version 582846 (0.0007) [2023-12-27 09:40:15,442][20381] Updated weights for policy 1, policy_version 582856 (0.0007) [2023-12-27 09:40:15,702][20347] Updated weights for policy 0, policy_version 583110 (0.0009) [2023-12-27 09:40:15,760][20347] Updated weights for policy 0, policy_version 583120 (0.0006) [2023-12-27 09:40:15,814][20347] Updated weights for policy 0, policy_version 583130 (0.0010) [2023-12-27 09:40:16,147][20381] Updated weights for policy 1, policy_version 582866 (0.0008) [2023-12-27 09:40:16,203][20381] Updated weights for policy 1, policy_version 582876 (0.0006) [2023-12-27 09:40:16,264][20381] Updated weights for policy 1, policy_version 582886 (0.0005) [2023-12-27 09:40:16,319][20381] Updated weights for policy 1, policy_version 582896 (0.0007) [2023-12-27 09:40:16,434][20347] Updated weights for policy 0, policy_version 583140 (0.0008) [2023-12-27 09:40:16,495][20347] Updated weights for policy 0, policy_version 583150 (0.0005) [2023-12-27 09:40:16,553][20347] Updated weights for policy 0, policy_version 583160 (0.0005) [2023-12-27 09:40:16,904][20381] Updated weights for policy 1, policy_version 582906 (0.0007) [2023-12-27 09:40:16,950][20381] Updated weights for policy 1, policy_version 582916 (0.0007) [2023-12-27 09:40:17,008][20381] Updated weights for policy 1, policy_version 582926 (0.0007) [2023-12-27 09:40:17,180][20347] Updated weights for policy 0, policy_version 583170 (0.0008) [2023-12-27 09:40:17,231][20347] Updated weights for policy 0, policy_version 583180 (0.0010) [2023-12-27 09:40:17,279][20347] Updated weights for policy 0, policy_version 583190 (0.0010) [2023-12-27 09:40:17,333][20347] Updated weights for policy 0, policy_version 583200 (0.0010) [2023-12-27 09:40:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20910.4). Total num frames: 298565632. Throughput: 0: 10345.7, 1: 10466.0. Samples: 298570812. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:17,363][19170] Avg episode reward: [(0, '942.650'), (1, '864.390')] [2023-12-27 09:40:17,581][20381] Updated weights for policy 1, policy_version 582936 (0.0006) [2023-12-27 09:40:17,630][20381] Updated weights for policy 1, policy_version 582946 (0.0005) [2023-12-27 09:40:17,675][20381] Updated weights for policy 1, policy_version 582956 (0.0005) [2023-12-27 09:40:18,025][20347] Updated weights for policy 0, policy_version 583210 (0.0010) [2023-12-27 09:40:18,086][20347] Updated weights for policy 0, policy_version 583220 (0.0010) [2023-12-27 09:40:18,139][20347] Updated weights for policy 0, policy_version 583230 (0.0010) [2023-12-27 09:40:18,268][20381] Updated weights for policy 1, policy_version 582966 (0.0005) [2023-12-27 09:40:18,331][20381] Updated weights for policy 1, policy_version 582976 (0.0007) [2023-12-27 09:40:18,388][20381] Updated weights for policy 1, policy_version 582986 (0.0011) [2023-12-27 09:40:18,854][20347] Updated weights for policy 0, policy_version 583240 (0.0008) [2023-12-27 09:40:18,901][20347] Updated weights for policy 0, policy_version 583250 (0.0007) [2023-12-27 09:40:18,952][20347] Updated weights for policy 0, policy_version 583260 (0.0007) [2023-12-27 09:40:19,064][20381] Updated weights for policy 1, policy_version 582996 (0.0011) [2023-12-27 09:40:19,119][20381] Updated weights for policy 1, policy_version 583006 (0.0010) [2023-12-27 09:40:19,175][20381] Updated weights for policy 1, policy_version 583016 (0.0011) [2023-12-27 09:40:19,681][20347] Updated weights for policy 0, policy_version 583270 (0.0007) [2023-12-27 09:40:19,740][20347] Updated weights for policy 0, policy_version 583280 (0.0007) [2023-12-27 09:40:19,807][20347] Updated weights for policy 0, policy_version 583290 (0.0008) [2023-12-27 09:40:19,907][20381] Updated weights for policy 1, policy_version 583026 (0.0011) [2023-12-27 09:40:19,964][20381] Updated weights for policy 1, policy_version 583036 (0.0008) [2023-12-27 09:40:20,029][20381] Updated weights for policy 1, policy_version 583046 (0.0006) [2023-12-27 09:40:20,092][20381] Updated weights for policy 1, policy_version 583056 (0.0007) [2023-12-27 09:40:20,549][20347] Updated weights for policy 0, policy_version 583300 (0.0008) [2023-12-27 09:40:20,609][20347] Updated weights for policy 0, policy_version 583310 (0.0008) [2023-12-27 09:40:20,658][20347] Updated weights for policy 0, policy_version 583320 (0.0008) [2023-12-27 09:40:20,756][20381] Updated weights for policy 1, policy_version 583066 (0.0007) [2023-12-27 09:40:20,818][20381] Updated weights for policy 1, policy_version 583076 (0.0008) [2023-12-27 09:40:20,880][20381] Updated weights for policy 1, policy_version 583086 (0.0007) [2023-12-27 09:40:21,380][20347] Updated weights for policy 0, policy_version 583330 (0.0008) [2023-12-27 09:40:21,437][20347] Updated weights for policy 0, policy_version 583340 (0.0010) [2023-12-27 09:40:21,499][20347] Updated weights for policy 0, policy_version 583350 (0.0010) [2023-12-27 09:40:21,563][20347] Updated weights for policy 0, policy_version 583360 (0.0011) [2023-12-27 09:40:21,566][20381] Updated weights for policy 1, policy_version 583096 (0.0008) [2023-12-27 09:40:21,631][20381] Updated weights for policy 1, policy_version 583106 (0.0007) [2023-12-27 09:40:21,688][20381] Updated weights for policy 1, policy_version 583116 (0.0007) [2023-12-27 09:40:22,289][20347] Updated weights for policy 0, policy_version 583370 (0.0008) [2023-12-27 09:40:22,349][20347] Updated weights for policy 0, policy_version 583380 (0.0007) [2023-12-27 09:40:22,358][20381] Updated weights for policy 1, policy_version 583126 (0.0007) [2023-12-27 09:40:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 298655744. Throughput: 0: 10309.9, 1: 10484.0. Samples: 298632116. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:22,363][19170] Avg episode reward: [(0, '932.930'), (1, '912.940')] [2023-12-27 09:40:22,415][20347] Updated weights for policy 0, policy_version 583390 (0.0006) [2023-12-27 09:40:22,425][20381] Updated weights for policy 1, policy_version 583136 (0.0006) [2023-12-27 09:40:22,425][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000583392_149364736.pth... [2023-12-27 09:40:22,428][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000582176_149053440.pth [2023-12-27 09:40:22,481][20381] Updated weights for policy 1, policy_version 583146 (0.0007) [2023-12-27 09:40:22,516][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000583152_149307392.pth... [2023-12-27 09:40:22,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000581904_148987904.pth [2023-12-27 09:40:23,137][20347] Updated weights for policy 0, policy_version 583400 (0.0006) [2023-12-27 09:40:23,143][20381] Updated weights for policy 1, policy_version 583156 (0.0007) [2023-12-27 09:40:23,192][20381] Updated weights for policy 1, policy_version 583166 (0.0005) [2023-12-27 09:40:23,197][20347] Updated weights for policy 0, policy_version 583410 (0.0007) [2023-12-27 09:40:23,245][20381] Updated weights for policy 1, policy_version 583176 (0.0005) [2023-12-27 09:40:23,247][20347] Updated weights for policy 0, policy_version 583420 (0.0007) [2023-12-27 09:40:23,900][20347] Updated weights for policy 0, policy_version 583430 (0.0008) [2023-12-27 09:40:23,938][20381] Updated weights for policy 1, policy_version 583186 (0.0006) [2023-12-27 09:40:23,952][20347] Updated weights for policy 0, policy_version 583440 (0.0010) [2023-12-27 09:40:23,988][20381] Updated weights for policy 1, policy_version 583196 (0.0005) [2023-12-27 09:40:24,004][20347] Updated weights for policy 0, policy_version 583450 (0.0010) [2023-12-27 09:40:24,048][20381] Updated weights for policy 1, policy_version 583206 (0.0007) [2023-12-27 09:40:24,104][20381] Updated weights for policy 1, policy_version 583216 (0.0008) [2023-12-27 09:40:24,623][20347] Updated weights for policy 0, policy_version 583460 (0.0008) [2023-12-27 09:40:24,684][20347] Updated weights for policy 0, policy_version 583470 (0.0010) [2023-12-27 09:40:24,743][20347] Updated weights for policy 0, policy_version 583480 (0.0010) [2023-12-27 09:40:24,757][20381] Updated weights for policy 1, policy_version 583226 (0.0006) [2023-12-27 09:40:24,813][20381] Updated weights for policy 1, policy_version 583236 (0.0007) [2023-12-27 09:40:24,876][20381] Updated weights for policy 1, policy_version 583246 (0.0008) [2023-12-27 09:40:25,360][20347] Updated weights for policy 0, policy_version 583490 (0.0010) [2023-12-27 09:40:25,416][20347] Updated weights for policy 0, policy_version 583500 (0.0008) [2023-12-27 09:40:25,465][20347] Updated weights for policy 0, policy_version 583510 (0.0008) [2023-12-27 09:40:25,515][20347] Updated weights for policy 0, policy_version 583520 (0.0007) [2023-12-27 09:40:25,581][20381] Updated weights for policy 1, policy_version 583256 (0.0007) [2023-12-27 09:40:25,636][20381] Updated weights for policy 1, policy_version 583266 (0.0006) [2023-12-27 09:40:25,686][20381] Updated weights for policy 1, policy_version 583276 (0.0005) [2023-12-27 09:40:26,171][20347] Updated weights for policy 0, policy_version 583530 (0.0006) [2023-12-27 09:40:26,225][20347] Updated weights for policy 0, policy_version 583540 (0.0007) [2023-12-27 09:40:26,275][20381] Updated weights for policy 1, policy_version 583286 (0.0009) [2023-12-27 09:40:26,281][20347] Updated weights for policy 0, policy_version 583550 (0.0007) [2023-12-27 09:40:26,333][20381] Updated weights for policy 1, policy_version 583296 (0.0011) [2023-12-27 09:40:26,386][20381] Updated weights for policy 1, policy_version 583306 (0.0010) [2023-12-27 09:40:26,956][20347] Updated weights for policy 0, policy_version 583560 (0.0007) [2023-12-27 09:40:27,000][20347] Updated weights for policy 0, policy_version 583570 (0.0005) [2023-12-27 09:40:27,046][20347] Updated weights for policy 0, policy_version 583580 (0.0005) [2023-12-27 09:40:27,059][20381] Updated weights for policy 1, policy_version 583316 (0.0009) [2023-12-27 09:40:27,103][20381] Updated weights for policy 1, policy_version 583326 (0.0006) [2023-12-27 09:40:27,146][20381] Updated weights for policy 1, policy_version 583336 (0.0005) [2023-12-27 09:40:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 298770432. Throughput: 0: 10327.9, 1: 10511.9. Samples: 298756048. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.930')] [2023-12-27 09:40:27,617][20347] Updated weights for policy 0, policy_version 583590 (0.0006) [2023-12-27 09:40:27,665][20347] Updated weights for policy 0, policy_version 583600 (0.0005) [2023-12-27 09:40:27,712][20347] Updated weights for policy 0, policy_version 583610 (0.0006) [2023-12-27 09:40:27,836][20381] Updated weights for policy 1, policy_version 583346 (0.0006) [2023-12-27 09:40:27,885][20381] Updated weights for policy 1, policy_version 583356 (0.0008) [2023-12-27 09:40:27,940][20381] Updated weights for policy 1, policy_version 583366 (0.0008) [2023-12-27 09:40:27,990][20381] Updated weights for policy 1, policy_version 583376 (0.0007) [2023-12-27 09:40:28,350][20347] Updated weights for policy 0, policy_version 583620 (0.0008) [2023-12-27 09:40:28,407][20347] Updated weights for policy 0, policy_version 583630 (0.0007) [2023-12-27 09:40:28,466][20347] Updated weights for policy 0, policy_version 583640 (0.0008) [2023-12-27 09:40:28,710][20381] Updated weights for policy 1, policy_version 583386 (0.0008) [2023-12-27 09:40:28,771][20381] Updated weights for policy 1, policy_version 583396 (0.0008) [2023-12-27 09:40:28,828][20381] Updated weights for policy 1, policy_version 583406 (0.0007) [2023-12-27 09:40:29,121][20347] Updated weights for policy 0, policy_version 583650 (0.0008) [2023-12-27 09:40:29,166][20347] Updated weights for policy 0, policy_version 583660 (0.0007) [2023-12-27 09:40:29,227][20347] Updated weights for policy 0, policy_version 583670 (0.0007) [2023-12-27 09:40:29,285][20347] Updated weights for policy 0, policy_version 583680 (0.0008) [2023-12-27 09:40:29,526][20381] Updated weights for policy 1, policy_version 583416 (0.0008) [2023-12-27 09:40:29,585][20381] Updated weights for policy 1, policy_version 583426 (0.0008) [2023-12-27 09:40:29,634][20381] Updated weights for policy 1, policy_version 583436 (0.0008) [2023-12-27 09:40:29,982][20347] Updated weights for policy 0, policy_version 583690 (0.0007) [2023-12-27 09:40:30,045][20347] Updated weights for policy 0, policy_version 583700 (0.0007) [2023-12-27 09:40:30,108][20347] Updated weights for policy 0, policy_version 583710 (0.0007) [2023-12-27 09:40:30,331][20381] Updated weights for policy 1, policy_version 583446 (0.0008) [2023-12-27 09:40:30,392][20381] Updated weights for policy 1, policy_version 583456 (0.0008) [2023-12-27 09:40:30,443][20381] Updated weights for policy 1, policy_version 583466 (0.0006) [2023-12-27 09:40:30,722][20347] Updated weights for policy 0, policy_version 583720 (0.0008) [2023-12-27 09:40:30,782][20347] Updated weights for policy 0, policy_version 583730 (0.0008) [2023-12-27 09:40:30,833][20347] Updated weights for policy 0, policy_version 583740 (0.0007) [2023-12-27 09:40:31,100][20381] Updated weights for policy 1, policy_version 583476 (0.0008) [2023-12-27 09:40:31,157][20381] Updated weights for policy 1, policy_version 583486 (0.0008) [2023-12-27 09:40:31,207][20381] Updated weights for policy 1, policy_version 583496 (0.0008) [2023-12-27 09:40:31,518][20347] Updated weights for policy 0, policy_version 583750 (0.0006) [2023-12-27 09:40:31,572][20347] Updated weights for policy 0, policy_version 583760 (0.0005) [2023-12-27 09:40:31,636][20347] Updated weights for policy 0, policy_version 583770 (0.0006) [2023-12-27 09:40:31,879][20381] Updated weights for policy 1, policy_version 583506 (0.0007) [2023-12-27 09:40:31,936][20381] Updated weights for policy 1, policy_version 583516 (0.0006) [2023-12-27 09:40:31,994][20381] Updated weights for policy 1, policy_version 583526 (0.0007) [2023-12-27 09:40:32,055][20381] Updated weights for policy 1, policy_version 583536 (0.0007) [2023-12-27 09:40:32,322][20347] Updated weights for policy 0, policy_version 583780 (0.0006) [2023-12-27 09:40:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 298868736. Throughput: 0: 10368.0, 1: 10464.2. Samples: 298881308. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.110')] [2023-12-27 09:40:32,386][20347] Updated weights for policy 0, policy_version 583790 (0.0008) [2023-12-27 09:40:32,444][20347] Updated weights for policy 0, policy_version 583800 (0.0008) [2023-12-27 09:40:32,671][20381] Updated weights for policy 1, policy_version 583546 (0.0008) [2023-12-27 09:40:32,726][20381] Updated weights for policy 1, policy_version 583556 (0.0008) [2023-12-27 09:40:32,784][20381] Updated weights for policy 1, policy_version 583566 (0.0007) [2023-12-27 09:40:33,022][20347] Updated weights for policy 0, policy_version 583810 (0.0007) [2023-12-27 09:40:33,078][20347] Updated weights for policy 0, policy_version 583820 (0.0005) [2023-12-27 09:40:33,129][20347] Updated weights for policy 0, policy_version 583830 (0.0005) [2023-12-27 09:40:33,185][20347] Updated weights for policy 0, policy_version 583840 (0.0007) [2023-12-27 09:40:33,511][20381] Updated weights for policy 1, policy_version 583576 (0.0007) [2023-12-27 09:40:33,564][20381] Updated weights for policy 1, policy_version 583586 (0.0007) [2023-12-27 09:40:33,620][20381] Updated weights for policy 1, policy_version 583596 (0.0007) [2023-12-27 09:40:33,766][20347] Updated weights for policy 0, policy_version 583850 (0.0005) [2023-12-27 09:40:33,812][20347] Updated weights for policy 0, policy_version 583860 (0.0005) [2023-12-27 09:40:33,865][20347] Updated weights for policy 0, policy_version 583870 (0.0005) [2023-12-27 09:40:34,286][20381] Updated weights for policy 1, policy_version 583606 (0.0006) [2023-12-27 09:40:34,338][20381] Updated weights for policy 1, policy_version 583616 (0.0005) [2023-12-27 09:40:34,400][20381] Updated weights for policy 1, policy_version 583626 (0.0009) [2023-12-27 09:40:34,513][20347] Updated weights for policy 0, policy_version 583880 (0.0007) [2023-12-27 09:40:34,565][20347] Updated weights for policy 0, policy_version 583890 (0.0007) [2023-12-27 09:40:34,614][20347] Updated weights for policy 0, policy_version 583900 (0.0008) [2023-12-27 09:40:35,072][20381] Updated weights for policy 1, policy_version 583636 (0.0008) [2023-12-27 09:40:35,123][20381] Updated weights for policy 1, policy_version 583646 (0.0007) [2023-12-27 09:40:35,164][20381] Updated weights for policy 1, policy_version 583656 (0.0008) [2023-12-27 09:40:35,289][20347] Updated weights for policy 0, policy_version 583910 (0.0009) [2023-12-27 09:40:35,340][20347] Updated weights for policy 0, policy_version 583920 (0.0010) [2023-12-27 09:40:35,395][20347] Updated weights for policy 0, policy_version 583930 (0.0010) [2023-12-27 09:40:35,846][20381] Updated weights for policy 1, policy_version 583666 (0.0008) [2023-12-27 09:40:35,900][20381] Updated weights for policy 1, policy_version 583676 (0.0007) [2023-12-27 09:40:35,952][20381] Updated weights for policy 1, policy_version 583686 (0.0006) [2023-12-27 09:40:36,000][20381] Updated weights for policy 1, policy_version 583696 (0.0005) [2023-12-27 09:40:36,015][20347] Updated weights for policy 0, policy_version 583940 (0.0008) [2023-12-27 09:40:36,079][20347] Updated weights for policy 0, policy_version 583950 (0.0010) [2023-12-27 09:40:36,141][20347] Updated weights for policy 0, policy_version 583960 (0.0008) [2023-12-27 09:40:36,636][20381] Updated weights for policy 1, policy_version 583706 (0.0007) [2023-12-27 09:40:36,699][20381] Updated weights for policy 1, policy_version 583716 (0.0008) [2023-12-27 09:40:36,755][20381] Updated weights for policy 1, policy_version 583726 (0.0007) [2023-12-27 09:40:36,841][20347] Updated weights for policy 0, policy_version 583970 (0.0010) [2023-12-27 09:40:36,898][20347] Updated weights for policy 0, policy_version 583980 (0.0011) [2023-12-27 09:40:36,958][20347] Updated weights for policy 0, policy_version 583990 (0.0011) [2023-12-27 09:40:37,018][20347] Updated weights for policy 0, policy_version 584000 (0.0009) [2023-12-27 09:40:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 298975232. Throughput: 0: 10447.9, 1: 10433.7. Samples: 298944516. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.840')] [2023-12-27 09:40:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000583728_149454848.pth... [2023-12-27 09:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000584000_149520384.pth... [2023-12-27 09:40:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000582544_149151744.pth [2023-12-27 09:40:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000582784_149209088.pth [2023-12-27 09:40:37,429][20381] Updated weights for policy 1, policy_version 583736 (0.0007) [2023-12-27 09:40:37,485][20381] Updated weights for policy 1, policy_version 583746 (0.0008) [2023-12-27 09:40:37,545][20381] Updated weights for policy 1, policy_version 583756 (0.0006) [2023-12-27 09:40:37,706][20347] Updated weights for policy 0, policy_version 584010 (0.0008) [2023-12-27 09:40:37,767][20347] Updated weights for policy 0, policy_version 584020 (0.0007) [2023-12-27 09:40:37,823][20347] Updated weights for policy 0, policy_version 584030 (0.0007) [2023-12-27 09:40:38,217][20381] Updated weights for policy 1, policy_version 583766 (0.0007) [2023-12-27 09:40:38,266][20381] Updated weights for policy 1, policy_version 583776 (0.0008) [2023-12-27 09:40:38,316][20381] Updated weights for policy 1, policy_version 583786 (0.0007) [2023-12-27 09:40:38,489][20347] Updated weights for policy 0, policy_version 584040 (0.0007) [2023-12-27 09:40:38,552][20347] Updated weights for policy 0, policy_version 584050 (0.0006) [2023-12-27 09:40:38,615][20347] Updated weights for policy 0, policy_version 584060 (0.0006) [2023-12-27 09:40:39,044][20381] Updated weights for policy 1, policy_version 583796 (0.0008) [2023-12-27 09:40:39,093][20381] Updated weights for policy 1, policy_version 583806 (0.0008) [2023-12-27 09:40:39,152][20381] Updated weights for policy 1, policy_version 583816 (0.0008) [2023-12-27 09:40:39,195][20347] Updated weights for policy 0, policy_version 584070 (0.0006) [2023-12-27 09:40:39,260][20347] Updated weights for policy 0, policy_version 584080 (0.0008) [2023-12-27 09:40:39,310][20347] Updated weights for policy 0, policy_version 584090 (0.0006) [2023-12-27 09:40:39,895][20381] Updated weights for policy 1, policy_version 583826 (0.0008) [2023-12-27 09:40:39,956][20381] Updated weights for policy 1, policy_version 583836 (0.0008) [2023-12-27 09:40:39,979][20347] Updated weights for policy 0, policy_version 584100 (0.0007) [2023-12-27 09:40:40,016][20381] Updated weights for policy 1, policy_version 583846 (0.0008) [2023-12-27 09:40:40,037][20347] Updated weights for policy 0, policy_version 584110 (0.0006) [2023-12-27 09:40:40,066][20381] Updated weights for policy 1, policy_version 583856 (0.0006) [2023-12-27 09:40:40,100][20347] Updated weights for policy 0, policy_version 584120 (0.0006) [2023-12-27 09:40:40,687][20381] Updated weights for policy 1, policy_version 583866 (0.0008) [2023-12-27 09:40:40,740][20381] Updated weights for policy 1, policy_version 583876 (0.0008) [2023-12-27 09:40:40,796][20381] Updated weights for policy 1, policy_version 583886 (0.0008) [2023-12-27 09:40:40,831][20347] Updated weights for policy 0, policy_version 584130 (0.0007) [2023-12-27 09:40:40,885][20347] Updated weights for policy 0, policy_version 584140 (0.0007) [2023-12-27 09:40:40,943][20347] Updated weights for policy 0, policy_version 584150 (0.0006) [2023-12-27 09:40:41,001][20347] Updated weights for policy 0, policy_version 584160 (0.0006) [2023-12-27 09:40:41,523][20381] Updated weights for policy 1, policy_version 583896 (0.0007) [2023-12-27 09:40:41,570][20381] Updated weights for policy 1, policy_version 583906 (0.0010) [2023-12-27 09:40:41,627][20381] Updated weights for policy 1, policy_version 583916 (0.0008) [2023-12-27 09:40:41,703][20347] Updated weights for policy 0, policy_version 584170 (0.0008) [2023-12-27 09:40:41,766][20347] Updated weights for policy 0, policy_version 584180 (0.0008) [2023-12-27 09:40:41,836][20347] Updated weights for policy 0, policy_version 584190 (0.0008) [2023-12-27 09:40:42,319][20381] Updated weights for policy 1, policy_version 583926 (0.0008) [2023-12-27 09:40:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 299073536. Throughput: 0: 10452.9, 1: 10380.8. Samples: 299067612. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.530')] [2023-12-27 09:40:42,388][20381] Updated weights for policy 1, policy_version 583936 (0.0008) [2023-12-27 09:40:42,452][20381] Updated weights for policy 1, policy_version 583946 (0.0008) [2023-12-27 09:40:42,606][20347] Updated weights for policy 0, policy_version 584200 (0.0008) [2023-12-27 09:40:42,665][20347] Updated weights for policy 0, policy_version 584210 (0.0007) [2023-12-27 09:40:42,729][20347] Updated weights for policy 0, policy_version 584220 (0.0008) [2023-12-27 09:40:43,091][20381] Updated weights for policy 1, policy_version 583956 (0.0009) [2023-12-27 09:40:43,151][20381] Updated weights for policy 1, policy_version 583966 (0.0008) [2023-12-27 09:40:43,210][20381] Updated weights for policy 1, policy_version 583976 (0.0008) [2023-12-27 09:40:43,422][20347] Updated weights for policy 0, policy_version 584230 (0.0007) [2023-12-27 09:40:43,474][20347] Updated weights for policy 0, policy_version 584240 (0.0008) [2023-12-27 09:40:43,533][20347] Updated weights for policy 0, policy_version 584250 (0.0007) [2023-12-27 09:40:43,805][20381] Updated weights for policy 1, policy_version 583986 (0.0007) [2023-12-27 09:40:43,858][20381] Updated weights for policy 1, policy_version 583996 (0.0005) [2023-12-27 09:40:43,909][20381] Updated weights for policy 1, policy_version 584006 (0.0005) [2023-12-27 09:40:43,954][20381] Updated weights for policy 1, policy_version 584016 (0.0005) [2023-12-27 09:40:44,220][20347] Updated weights for policy 0, policy_version 584260 (0.0006) [2023-12-27 09:40:44,280][20347] Updated weights for policy 0, policy_version 584270 (0.0007) [2023-12-27 09:40:44,337][20347] Updated weights for policy 0, policy_version 584280 (0.0006) [2023-12-27 09:40:44,523][20381] Updated weights for policy 1, policy_version 584026 (0.0005) [2023-12-27 09:40:44,584][20381] Updated weights for policy 1, policy_version 584036 (0.0005) [2023-12-27 09:40:44,642][20381] Updated weights for policy 1, policy_version 584046 (0.0007) [2023-12-27 09:40:45,027][20347] Updated weights for policy 0, policy_version 584290 (0.0007) [2023-12-27 09:40:45,085][20347] Updated weights for policy 0, policy_version 584300 (0.0008) [2023-12-27 09:40:45,143][20347] Updated weights for policy 0, policy_version 584310 (0.0007) [2023-12-27 09:40:45,201][20347] Updated weights for policy 0, policy_version 584320 (0.0005) [2023-12-27 09:40:45,319][20381] Updated weights for policy 1, policy_version 584056 (0.0008) [2023-12-27 09:40:45,374][20381] Updated weights for policy 1, policy_version 584066 (0.0007) [2023-12-27 09:40:45,426][20381] Updated weights for policy 1, policy_version 584076 (0.0008) [2023-12-27 09:40:45,807][20347] Updated weights for policy 0, policy_version 584330 (0.0008) [2023-12-27 09:40:45,852][20347] Updated weights for policy 0, policy_version 584340 (0.0008) [2023-12-27 09:40:45,897][20347] Updated weights for policy 0, policy_version 584350 (0.0008) [2023-12-27 09:40:46,071][20381] Updated weights for policy 1, policy_version 584086 (0.0007) [2023-12-27 09:40:46,118][20381] Updated weights for policy 1, policy_version 584096 (0.0007) [2023-12-27 09:40:46,173][20381] Updated weights for policy 1, policy_version 584106 (0.0005) [2023-12-27 09:40:46,629][20347] Updated weights for policy 0, policy_version 584360 (0.0008) [2023-12-27 09:40:46,674][20347] Updated weights for policy 0, policy_version 584370 (0.0007) [2023-12-27 09:40:46,728][20347] Updated weights for policy 0, policy_version 584380 (0.0008) [2023-12-27 09:40:46,830][20381] Updated weights for policy 1, policy_version 584116 (0.0007) [2023-12-27 09:40:46,880][20381] Updated weights for policy 1, policy_version 584126 (0.0007) [2023-12-27 09:40:46,936][20381] Updated weights for policy 1, policy_version 584136 (0.0008) [2023-12-27 09:40:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 299180032. Throughput: 0: 10329.5, 1: 10427.5. Samples: 299192016. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:47,363][19170] Avg episode reward: [(0, '990.520'), (1, '912.940')] [2023-12-27 09:40:47,439][20347] Updated weights for policy 0, policy_version 584390 (0.0008) [2023-12-27 09:40:47,495][20347] Updated weights for policy 0, policy_version 584400 (0.0008) [2023-12-27 09:40:47,550][20381] Updated weights for policy 1, policy_version 584146 (0.0007) [2023-12-27 09:40:47,554][20347] Updated weights for policy 0, policy_version 584410 (0.0008) [2023-12-27 09:40:47,598][20381] Updated weights for policy 1, policy_version 584156 (0.0005) [2023-12-27 09:40:47,654][20381] Updated weights for policy 1, policy_version 584166 (0.0005) [2023-12-27 09:40:47,713][20381] Updated weights for policy 1, policy_version 584176 (0.0005) [2023-12-27 09:40:48,171][20347] Updated weights for policy 0, policy_version 584420 (0.0007) [2023-12-27 09:40:48,227][20347] Updated weights for policy 0, policy_version 584430 (0.0007) [2023-12-27 09:40:48,283][20347] Updated weights for policy 0, policy_version 584440 (0.0006) [2023-12-27 09:40:48,310][20381] Updated weights for policy 1, policy_version 584186 (0.0009) [2023-12-27 09:40:48,369][20381] Updated weights for policy 1, policy_version 584196 (0.0007) [2023-12-27 09:40:48,425][20381] Updated weights for policy 1, policy_version 584206 (0.0008) [2023-12-27 09:40:48,982][20347] Updated weights for policy 0, policy_version 584450 (0.0007) [2023-12-27 09:40:49,041][20347] Updated weights for policy 0, policy_version 584460 (0.0008) [2023-12-27 09:40:49,100][20347] Updated weights for policy 0, policy_version 584470 (0.0008) [2023-12-27 09:40:49,111][20381] Updated weights for policy 1, policy_version 584216 (0.0006) [2023-12-27 09:40:49,156][20347] Updated weights for policy 0, policy_version 584480 (0.0007) [2023-12-27 09:40:49,169][20381] Updated weights for policy 1, policy_version 584226 (0.0007) [2023-12-27 09:40:49,236][20381] Updated weights for policy 1, policy_version 584236 (0.0008) [2023-12-27 09:40:49,912][20347] Updated weights for policy 0, policy_version 584490 (0.0008) [2023-12-27 09:40:49,919][20381] Updated weights for policy 1, policy_version 584246 (0.0009) [2023-12-27 09:40:49,975][20381] Updated weights for policy 1, policy_version 584256 (0.0008) [2023-12-27 09:40:49,981][20347] Updated weights for policy 0, policy_version 584500 (0.0006) [2023-12-27 09:40:50,032][20381] Updated weights for policy 1, policy_version 584266 (0.0006) [2023-12-27 09:40:50,051][20347] Updated weights for policy 0, policy_version 584510 (0.0008) [2023-12-27 09:40:50,641][20347] Updated weights for policy 0, policy_version 584520 (0.0008) [2023-12-27 09:40:50,697][20347] Updated weights for policy 0, policy_version 584530 (0.0008) [2023-12-27 09:40:50,757][20381] Updated weights for policy 1, policy_version 584276 (0.0007) [2023-12-27 09:40:50,758][20347] Updated weights for policy 0, policy_version 584541 (0.0006) [2023-12-27 09:40:50,816][20381] Updated weights for policy 1, policy_version 584286 (0.0006) [2023-12-27 09:40:50,872][20381] Updated weights for policy 1, policy_version 584296 (0.0007) [2023-12-27 09:40:51,401][20347] Updated weights for policy 0, policy_version 584551 (0.0007) [2023-12-27 09:40:51,470][20347] Updated weights for policy 0, policy_version 584561 (0.0008) [2023-12-27 09:40:51,514][20381] Updated weights for policy 1, policy_version 584306 (0.0007) [2023-12-27 09:40:51,535][20347] Updated weights for policy 0, policy_version 584571 (0.0007) [2023-12-27 09:40:51,576][20381] Updated weights for policy 1, policy_version 584316 (0.0007) [2023-12-27 09:40:51,634][20381] Updated weights for policy 1, policy_version 584326 (0.0008) [2023-12-27 09:40:51,697][20381] Updated weights for policy 1, policy_version 584336 (0.0006) [2023-12-27 09:40:52,225][20347] Updated weights for policy 0, policy_version 584581 (0.0008) [2023-12-27 09:40:52,280][20347] Updated weights for policy 0, policy_version 584591 (0.0006) [2023-12-27 09:40:52,342][20347] Updated weights for policy 0, policy_version 584601 (0.0006) [2023-12-27 09:40:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 299278336. Throughput: 0: 10308.2, 1: 10391.7. Samples: 299254056. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:52,363][19170] Avg episode reward: [(0, '952.330'), (1, '951.670')] [2023-12-27 09:40:52,365][20381] Updated weights for policy 1, policy_version 584346 (0.0008) [2023-12-27 09:40:52,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000584608_149676032.pth... [2023-12-27 09:40:52,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000583392_149364736.pth [2023-12-27 09:40:52,421][20381] Updated weights for policy 1, policy_version 584356 (0.0007) [2023-12-27 09:40:52,477][20381] Updated weights for policy 1, policy_version 584366 (0.0008) [2023-12-27 09:40:52,487][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000584368_149618688.pth... [2023-12-27 09:40:52,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000583152_149307392.pth [2023-12-27 09:40:52,929][20347] Updated weights for policy 0, policy_version 584611 (0.0007) [2023-12-27 09:40:52,980][20347] Updated weights for policy 0, policy_version 584621 (0.0007) [2023-12-27 09:40:53,033][20347] Updated weights for policy 0, policy_version 584631 (0.0007) [2023-12-27 09:40:53,218][20381] Updated weights for policy 1, policy_version 584376 (0.0008) [2023-12-27 09:40:53,248][20292] KL-divergence is very high: 113.7917 [2023-12-27 09:40:53,278][20381] Updated weights for policy 1, policy_version 584386 (0.0008) [2023-12-27 09:40:53,336][20381] Updated weights for policy 1, policy_version 584396 (0.0008) [2023-12-27 09:40:53,697][20347] Updated weights for policy 0, policy_version 584641 (0.0008) [2023-12-27 09:40:53,761][20347] Updated weights for policy 0, policy_version 584651 (0.0008) [2023-12-27 09:40:53,816][20347] Updated weights for policy 0, policy_version 584661 (0.0007) [2023-12-27 09:40:53,869][20347] Updated weights for policy 0, policy_version 584671 (0.0008) [2023-12-27 09:40:53,979][20381] Updated weights for policy 1, policy_version 584406 (0.0007) [2023-12-27 09:40:53,999][20292] KL-divergence is very high: 245.4823 [2023-12-27 09:40:54,010][20292] KL-divergence is very high: 261.7416 [2023-12-27 09:40:54,031][20381] Updated weights for policy 1, policy_version 584416 (0.0008) [2023-12-27 09:40:54,040][20292] KL-divergence is very high: 257.7378 [2023-12-27 09:40:54,054][20292] KL-divergence is very high: 198.9637 [2023-12-27 09:40:54,088][20381] Updated weights for policy 1, policy_version 584426 (0.0009) [2023-12-27 09:40:54,479][20347] Updated weights for policy 0, policy_version 584681 (0.0005) [2023-12-27 09:40:54,526][20347] Updated weights for policy 0, policy_version 584691 (0.0006) [2023-12-27 09:40:54,574][20347] Updated weights for policy 0, policy_version 584701 (0.0005) [2023-12-27 09:40:54,853][20381] Updated weights for policy 1, policy_version 584436 (0.0008) [2023-12-27 09:40:54,902][20381] Updated weights for policy 1, policy_version 584446 (0.0009) [2023-12-27 09:40:54,947][20381] Updated weights for policy 1, policy_version 584456 (0.0008) [2023-12-27 09:40:55,140][20347] Updated weights for policy 0, policy_version 584711 (0.0007) [2023-12-27 09:40:55,205][20347] Updated weights for policy 0, policy_version 584721 (0.0008) [2023-12-27 09:40:55,268][20347] Updated weights for policy 0, policy_version 584731 (0.0008) [2023-12-27 09:40:55,613][20381] Updated weights for policy 1, policy_version 584466 (0.0006) [2023-12-27 09:40:55,662][20381] Updated weights for policy 1, policy_version 584476 (0.0009) [2023-12-27 09:40:55,709][20381] Updated weights for policy 1, policy_version 584486 (0.0010) [2023-12-27 09:40:55,761][20381] Updated weights for policy 1, policy_version 584496 (0.0008) [2023-12-27 09:40:55,860][20347] Updated weights for policy 0, policy_version 584741 (0.0009) [2023-12-27 09:40:55,918][20347] Updated weights for policy 0, policy_version 584751 (0.0010) [2023-12-27 09:40:55,976][20347] Updated weights for policy 0, policy_version 584761 (0.0010) [2023-12-27 09:40:56,460][20381] Updated weights for policy 1, policy_version 584506 (0.0005) [2023-12-27 09:40:56,505][20381] Updated weights for policy 1, policy_version 584516 (0.0005) [2023-12-27 09:40:56,549][20381] Updated weights for policy 1, policy_version 584526 (0.0007) [2023-12-27 09:40:56,643][20347] Updated weights for policy 0, policy_version 584771 (0.0010) [2023-12-27 09:40:56,692][20347] Updated weights for policy 0, policy_version 584781 (0.0010) [2023-12-27 09:40:56,746][20347] Updated weights for policy 0, policy_version 584791 (0.0009) [2023-12-27 09:40:57,121][20381] Updated weights for policy 1, policy_version 584536 (0.0006) [2023-12-27 09:40:57,166][20381] Updated weights for policy 1, policy_version 584546 (0.0005) [2023-12-27 09:40:57,219][20381] Updated weights for policy 1, policy_version 584556 (0.0005) [2023-12-27 09:40:57,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 299393024. Throughput: 0: 10396.2, 1: 10323.9. Samples: 299379748. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:40:57,364][19170] Avg episode reward: [(0, '980.810'), (1, '965.290')] [2023-12-27 09:40:57,419][20347] Updated weights for policy 0, policy_version 584801 (0.0005) [2023-12-27 09:40:57,478][20347] Updated weights for policy 0, policy_version 584811 (0.0005) [2023-12-27 09:40:57,535][20347] Updated weights for policy 0, policy_version 584821 (0.0007) [2023-12-27 09:40:57,583][20347] Updated weights for policy 0, policy_version 584831 (0.0006) [2023-12-27 09:40:57,840][20381] Updated weights for policy 1, policy_version 584566 (0.0005) [2023-12-27 09:40:57,898][20381] Updated weights for policy 1, policy_version 584576 (0.0008) [2023-12-27 09:40:57,958][20381] Updated weights for policy 1, policy_version 584586 (0.0011) [2023-12-27 09:40:58,171][20347] Updated weights for policy 0, policy_version 584841 (0.0007) [2023-12-27 09:40:58,230][20347] Updated weights for policy 0, policy_version 584851 (0.0007) [2023-12-27 09:40:58,292][20347] Updated weights for policy 0, policy_version 584861 (0.0008) [2023-12-27 09:40:58,663][20381] Updated weights for policy 1, policy_version 584596 (0.0009) [2023-12-27 09:40:58,718][20381] Updated weights for policy 1, policy_version 584606 (0.0008) [2023-12-27 09:40:58,779][20381] Updated weights for policy 1, policy_version 584616 (0.0008) [2023-12-27 09:40:59,050][20347] Updated weights for policy 0, policy_version 584871 (0.0008) [2023-12-27 09:40:59,107][20347] Updated weights for policy 0, policy_version 584881 (0.0007) [2023-12-27 09:40:59,162][20347] Updated weights for policy 0, policy_version 584891 (0.0008) [2023-12-27 09:40:59,558][20381] Updated weights for policy 1, policy_version 584626 (0.0009) [2023-12-27 09:40:59,611][20381] Updated weights for policy 1, policy_version 584636 (0.0011) [2023-12-27 09:40:59,659][20381] Updated weights for policy 1, policy_version 584646 (0.0011) [2023-12-27 09:40:59,711][20381] Updated weights for policy 1, policy_version 584656 (0.0010) [2023-12-27 09:40:59,881][20347] Updated weights for policy 0, policy_version 584901 (0.0007) [2023-12-27 09:40:59,936][20347] Updated weights for policy 0, policy_version 584911 (0.0007) [2023-12-27 09:40:59,995][20347] Updated weights for policy 0, policy_version 584921 (0.0008) [2023-12-27 09:41:00,417][20381] Updated weights for policy 1, policy_version 584666 (0.0011) [2023-12-27 09:41:00,472][20381] Updated weights for policy 1, policy_version 584676 (0.0011) [2023-12-27 09:41:00,527][20381] Updated weights for policy 1, policy_version 584686 (0.0011) [2023-12-27 09:41:00,698][20347] Updated weights for policy 0, policy_version 584931 (0.0007) [2023-12-27 09:41:00,745][20347] Updated weights for policy 0, policy_version 584941 (0.0007) [2023-12-27 09:41:00,801][20347] Updated weights for policy 0, policy_version 584951 (0.0007) [2023-12-27 09:41:01,215][20381] Updated weights for policy 1, policy_version 584696 (0.0011) [2023-12-27 09:41:01,274][20381] Updated weights for policy 1, policy_version 584706 (0.0011) [2023-12-27 09:41:01,333][20381] Updated weights for policy 1, policy_version 584716 (0.0008) [2023-12-27 09:41:01,519][20347] Updated weights for policy 0, policy_version 584961 (0.0007) [2023-12-27 09:41:01,572][20347] Updated weights for policy 0, policy_version 584971 (0.0007) [2023-12-27 09:41:01,633][20347] Updated weights for policy 0, policy_version 584981 (0.0007) [2023-12-27 09:41:01,694][20347] Updated weights for policy 0, policy_version 584991 (0.0007) [2023-12-27 09:41:02,039][20381] Updated weights for policy 1, policy_version 584726 (0.0010) [2023-12-27 09:41:02,073][20292] KL-divergence is very high: 144.7045 [2023-12-27 09:41:02,098][20381] Updated weights for policy 1, policy_version 584736 (0.0011) [2023-12-27 09:41:02,147][20381] Updated weights for policy 1, policy_version 584746 (0.0010) [2023-12-27 09:41:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 299491328. Throughput: 0: 10358.4, 1: 10334.7. Samples: 299502004. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:02,363][19170] Avg episode reward: [(0, '980.880'), (1, '975.220')] [2023-12-27 09:41:02,401][20347] Updated weights for policy 0, policy_version 585001 (0.0008) [2023-12-27 09:41:02,409][20291] KL-divergence is very high: 197.4571 [2023-12-27 09:41:02,452][20347] Updated weights for policy 0, policy_version 585011 (0.0007) [2023-12-27 09:41:02,502][20347] Updated weights for policy 0, policy_version 585021 (0.0007) [2023-12-27 09:41:02,781][20381] Updated weights for policy 1, policy_version 584756 (0.0007) [2023-12-27 09:41:02,840][20381] Updated weights for policy 1, policy_version 584766 (0.0007) [2023-12-27 09:41:02,892][20381] Updated weights for policy 1, policy_version 584776 (0.0006) [2023-12-27 09:41:03,150][20347] Updated weights for policy 0, policy_version 585031 (0.0007) [2023-12-27 09:41:03,199][20347] Updated weights for policy 0, policy_version 585041 (0.0006) [2023-12-27 09:41:03,249][20347] Updated weights for policy 0, policy_version 585051 (0.0006) [2023-12-27 09:41:03,539][20381] Updated weights for policy 1, policy_version 584786 (0.0010) [2023-12-27 09:41:03,582][20381] Updated weights for policy 1, policy_version 584796 (0.0008) [2023-12-27 09:41:03,630][20381] Updated weights for policy 1, policy_version 584806 (0.0007) [2023-12-27 09:41:03,683][20381] Updated weights for policy 1, policy_version 584816 (0.0008) [2023-12-27 09:41:03,942][20347] Updated weights for policy 0, policy_version 585061 (0.0006) [2023-12-27 09:41:04,006][20347] Updated weights for policy 0, policy_version 585071 (0.0007) [2023-12-27 09:41:04,059][20347] Updated weights for policy 0, policy_version 585081 (0.0009) [2023-12-27 09:41:04,409][20381] Updated weights for policy 1, policy_version 584826 (0.0011) [2023-12-27 09:41:04,458][20381] Updated weights for policy 1, policy_version 584836 (0.0010) [2023-12-27 09:41:04,511][20381] Updated weights for policy 1, policy_version 584846 (0.0011) [2023-12-27 09:41:04,747][20347] Updated weights for policy 0, policy_version 585091 (0.0010) [2023-12-27 09:41:04,800][20347] Updated weights for policy 0, policy_version 585101 (0.0010) [2023-12-27 09:41:04,853][20347] Updated weights for policy 0, policy_version 585111 (0.0010) [2023-12-27 09:41:05,216][20381] Updated weights for policy 1, policy_version 584856 (0.0010) [2023-12-27 09:41:05,278][20381] Updated weights for policy 1, policy_version 584866 (0.0011) [2023-12-27 09:41:05,333][20381] Updated weights for policy 1, policy_version 584876 (0.0010) [2023-12-27 09:41:05,554][20347] Updated weights for policy 0, policy_version 585121 (0.0007) [2023-12-27 09:41:05,600][20347] Updated weights for policy 0, policy_version 585131 (0.0007) [2023-12-27 09:41:05,660][20347] Updated weights for policy 0, policy_version 585141 (0.0007) [2023-12-27 09:41:05,718][20347] Updated weights for policy 0, policy_version 585151 (0.0007) [2023-12-27 09:41:06,031][20381] Updated weights for policy 1, policy_version 584886 (0.0010) [2023-12-27 09:41:06,077][20381] Updated weights for policy 1, policy_version 584896 (0.0010) [2023-12-27 09:41:06,134][20381] Updated weights for policy 1, policy_version 584906 (0.0009) [2023-12-27 09:41:06,392][20347] Updated weights for policy 0, policy_version 585161 (0.0009) [2023-12-27 09:41:06,440][20347] Updated weights for policy 0, policy_version 585171 (0.0008) [2023-12-27 09:41:06,488][20347] Updated weights for policy 0, policy_version 585181 (0.0007) [2023-12-27 09:41:06,762][20381] Updated weights for policy 1, policy_version 584916 (0.0008) [2023-12-27 09:41:06,826][20381] Updated weights for policy 1, policy_version 584926 (0.0006) [2023-12-27 09:41:06,882][20381] Updated weights for policy 1, policy_version 584936 (0.0005) [2023-12-27 09:41:07,242][20347] Updated weights for policy 0, policy_version 585191 (0.0007) [2023-12-27 09:41:07,304][20347] Updated weights for policy 0, policy_version 585201 (0.0008) [2023-12-27 09:41:07,353][20347] Updated weights for policy 0, policy_version 585211 (0.0008) [2023-12-27 09:41:07,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 299589632. Throughput: 0: 10381.2, 1: 10320.6. Samples: 299563692. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:07,363][19170] Avg episode reward: [(0, '990.400'), (1, '924.990')] [2023-12-27 09:41:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000584944_149766144.pth... [2023-12-27 09:41:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000583728_149454848.pth [2023-12-27 09:41:07,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000585216_149831680.pth... [2023-12-27 09:41:07,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000584000_149520384.pth [2023-12-27 09:41:07,488][20381] Updated weights for policy 1, policy_version 584946 (0.0006) [2023-12-27 09:41:07,550][20381] Updated weights for policy 1, policy_version 584956 (0.0011) [2023-12-27 09:41:07,608][20381] Updated weights for policy 1, policy_version 584966 (0.0007) [2023-12-27 09:41:07,656][20292] KL-divergence is very high: 215.0634 [2023-12-27 09:41:07,667][20292] KL-divergence is very high: 110.1614 [2023-12-27 09:41:07,671][20381] Updated weights for policy 1, policy_version 584976 (0.0010) [2023-12-27 09:41:08,048][20347] Updated weights for policy 0, policy_version 585221 (0.0009) [2023-12-27 09:41:08,094][20347] Updated weights for policy 0, policy_version 585231 (0.0010) [2023-12-27 09:41:08,146][20347] Updated weights for policy 0, policy_version 585241 (0.0010) [2023-12-27 09:41:08,258][20292] KL-divergence is very high: 130.4995 [2023-12-27 09:41:08,320][20381] Updated weights for policy 1, policy_version 584986 (0.0006) [2023-12-27 09:41:08,384][20381] Updated weights for policy 1, policy_version 584996 (0.0011) [2023-12-27 09:41:08,445][20381] Updated weights for policy 1, policy_version 585006 (0.0007) [2023-12-27 09:41:08,831][20347] Updated weights for policy 0, policy_version 585251 (0.0009) [2023-12-27 09:41:08,895][20347] Updated weights for policy 0, policy_version 585261 (0.0005) [2023-12-27 09:41:08,959][20347] Updated weights for policy 0, policy_version 585271 (0.0005) [2023-12-27 09:41:09,052][20381] Updated weights for policy 1, policy_version 585016 (0.0007) [2023-12-27 09:41:09,109][20381] Updated weights for policy 1, policy_version 585026 (0.0008) [2023-12-27 09:41:09,170][20381] Updated weights for policy 1, policy_version 585036 (0.0008) [2023-12-27 09:41:09,587][20347] Updated weights for policy 0, policy_version 585281 (0.0005) [2023-12-27 09:41:09,642][20347] Updated weights for policy 0, policy_version 585291 (0.0005) [2023-12-27 09:41:09,701][20347] Updated weights for policy 0, policy_version 585301 (0.0005) [2023-12-27 09:41:09,759][20347] Updated weights for policy 0, policy_version 585311 (0.0006) [2023-12-27 09:41:09,909][20381] Updated weights for policy 1, policy_version 585046 (0.0008) [2023-12-27 09:41:09,969][20381] Updated weights for policy 1, policy_version 585056 (0.0007) [2023-12-27 09:41:10,032][20381] Updated weights for policy 1, policy_version 585066 (0.0006) [2023-12-27 09:41:10,427][20347] Updated weights for policy 0, policy_version 585321 (0.0007) [2023-12-27 09:41:10,489][20347] Updated weights for policy 0, policy_version 585331 (0.0006) [2023-12-27 09:41:10,538][20347] Updated weights for policy 0, policy_version 585341 (0.0005) [2023-12-27 09:41:10,705][20381] Updated weights for policy 1, policy_version 585076 (0.0006) [2023-12-27 09:41:10,763][20381] Updated weights for policy 1, policy_version 585086 (0.0007) [2023-12-27 09:41:10,817][20381] Updated weights for policy 1, policy_version 585096 (0.0008) [2023-12-27 09:41:11,197][20347] Updated weights for policy 0, policy_version 585351 (0.0010) [2023-12-27 09:41:11,258][20347] Updated weights for policy 0, policy_version 585361 (0.0011) [2023-12-27 09:41:11,307][20347] Updated weights for policy 0, policy_version 585371 (0.0010) [2023-12-27 09:41:11,525][20381] Updated weights for policy 1, policy_version 585106 (0.0008) [2023-12-27 09:41:11,594][20381] Updated weights for policy 1, policy_version 585116 (0.0009) [2023-12-27 09:41:11,658][20381] Updated weights for policy 1, policy_version 585126 (0.0009) [2023-12-27 09:41:11,726][20381] Updated weights for policy 1, policy_version 585136 (0.0009) [2023-12-27 09:41:11,977][20347] Updated weights for policy 0, policy_version 585381 (0.0007) [2023-12-27 09:41:12,036][20347] Updated weights for policy 0, policy_version 585391 (0.0006) [2023-12-27 09:41:12,097][20347] Updated weights for policy 0, policy_version 585401 (0.0007) [2023-12-27 09:41:12,337][20381] Updated weights for policy 1, policy_version 585146 (0.0008) [2023-12-27 09:41:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 299696128. Throughput: 0: 10389.1, 1: 10302.0. Samples: 299687144. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '934.330')] [2023-12-27 09:41:12,397][20381] Updated weights for policy 1, policy_version 585156 (0.0009) [2023-12-27 09:41:12,455][20381] Updated weights for policy 1, policy_version 585166 (0.0007) [2023-12-27 09:41:12,781][20347] Updated weights for policy 0, policy_version 585411 (0.0007) [2023-12-27 09:41:12,845][20347] Updated weights for policy 0, policy_version 585421 (0.0005) [2023-12-27 09:41:12,906][20347] Updated weights for policy 0, policy_version 585431 (0.0005) [2023-12-27 09:41:13,119][20381] Updated weights for policy 1, policy_version 585176 (0.0007) [2023-12-27 09:41:13,179][20381] Updated weights for policy 1, policy_version 585186 (0.0006) [2023-12-27 09:41:13,230][20381] Updated weights for policy 1, policy_version 585196 (0.0008) [2023-12-27 09:41:13,558][20347] Updated weights for policy 0, policy_version 585441 (0.0005) [2023-12-27 09:41:13,614][20347] Updated weights for policy 0, policy_version 585451 (0.0007) [2023-12-27 09:41:13,667][20347] Updated weights for policy 0, policy_version 585461 (0.0008) [2023-12-27 09:41:13,726][20347] Updated weights for policy 0, policy_version 585471 (0.0008) [2023-12-27 09:41:13,815][20381] Updated weights for policy 1, policy_version 585206 (0.0006) [2023-12-27 09:41:13,868][20381] Updated weights for policy 1, policy_version 585216 (0.0006) [2023-12-27 09:41:13,928][20381] Updated weights for policy 1, policy_version 585226 (0.0007) [2023-12-27 09:41:14,403][20347] Updated weights for policy 0, policy_version 585481 (0.0007) [2023-12-27 09:41:14,458][20347] Updated weights for policy 0, policy_version 585491 (0.0008) [2023-12-27 09:41:14,520][20347] Updated weights for policy 0, policy_version 585501 (0.0007) [2023-12-27 09:41:14,604][20381] Updated weights for policy 1, policy_version 585236 (0.0008) [2023-12-27 09:41:14,651][20381] Updated weights for policy 1, policy_version 585246 (0.0008) [2023-12-27 09:41:14,703][20381] Updated weights for policy 1, policy_version 585256 (0.0008) [2023-12-27 09:41:15,239][20347] Updated weights for policy 0, policy_version 585511 (0.0008) [2023-12-27 09:41:15,295][20347] Updated weights for policy 0, policy_version 585521 (0.0008) [2023-12-27 09:41:15,350][20347] Updated weights for policy 0, policy_version 585531 (0.0008) [2023-12-27 09:41:15,389][20381] Updated weights for policy 1, policy_version 585266 (0.0007) [2023-12-27 09:41:15,437][20381] Updated weights for policy 1, policy_version 585276 (0.0007) [2023-12-27 09:41:15,487][20381] Updated weights for policy 1, policy_version 585286 (0.0008) [2023-12-27 09:41:15,533][20381] Updated weights for policy 1, policy_version 585296 (0.0006) [2023-12-27 09:41:16,047][20347] Updated weights for policy 0, policy_version 585541 (0.0008) [2023-12-27 09:41:16,098][20347] Updated weights for policy 0, policy_version 585551 (0.0007) [2023-12-27 09:41:16,145][20347] Updated weights for policy 0, policy_version 585561 (0.0007) [2023-12-27 09:41:16,208][20381] Updated weights for policy 1, policy_version 585306 (0.0009) [2023-12-27 09:41:16,266][20381] Updated weights for policy 1, policy_version 585316 (0.0009) [2023-12-27 09:41:16,331][20381] Updated weights for policy 1, policy_version 585326 (0.0008) [2023-12-27 09:41:16,726][20347] Updated weights for policy 0, policy_version 585571 (0.0005) [2023-12-27 09:41:16,777][20347] Updated weights for policy 0, policy_version 585581 (0.0005) [2023-12-27 09:41:16,828][20347] Updated weights for policy 0, policy_version 585591 (0.0005) [2023-12-27 09:41:17,079][20381] Updated weights for policy 1, policy_version 585336 (0.0008) [2023-12-27 09:41:17,135][20381] Updated weights for policy 1, policy_version 585346 (0.0007) [2023-12-27 09:41:17,187][20381] Updated weights for policy 1, policy_version 585356 (0.0008) [2023-12-27 09:41:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 299802624. Throughput: 0: 10348.4, 1: 10315.6. Samples: 299811188. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.830')] [2023-12-27 09:41:17,482][20347] Updated weights for policy 0, policy_version 585601 (0.0007) [2023-12-27 09:41:17,536][20347] Updated weights for policy 0, policy_version 585611 (0.0007) [2023-12-27 09:41:17,585][20347] Updated weights for policy 0, policy_version 585621 (0.0008) [2023-12-27 09:41:17,645][20347] Updated weights for policy 0, policy_version 585631 (0.0007) [2023-12-27 09:41:17,900][20381] Updated weights for policy 1, policy_version 585366 (0.0007) [2023-12-27 09:41:17,953][20381] Updated weights for policy 1, policy_version 585376 (0.0007) [2023-12-27 09:41:18,002][20381] Updated weights for policy 1, policy_version 585386 (0.0007) [2023-12-27 09:41:18,276][20347] Updated weights for policy 0, policy_version 585641 (0.0007) [2023-12-27 09:41:18,335][20347] Updated weights for policy 0, policy_version 585651 (0.0008) [2023-12-27 09:41:18,392][20347] Updated weights for policy 0, policy_version 585661 (0.0007) [2023-12-27 09:41:18,689][20381] Updated weights for policy 1, policy_version 585396 (0.0008) [2023-12-27 09:41:18,749][20381] Updated weights for policy 1, policy_version 585406 (0.0008) [2023-12-27 09:41:18,806][20381] Updated weights for policy 1, policy_version 585416 (0.0008) [2023-12-27 09:41:19,089][20347] Updated weights for policy 0, policy_version 585671 (0.0008) [2023-12-27 09:41:19,150][20347] Updated weights for policy 0, policy_version 585681 (0.0007) [2023-12-27 09:41:19,206][20347] Updated weights for policy 0, policy_version 585691 (0.0007) [2023-12-27 09:41:19,494][20381] Updated weights for policy 1, policy_version 585426 (0.0008) [2023-12-27 09:41:19,555][20381] Updated weights for policy 1, policy_version 585436 (0.0008) [2023-12-27 09:41:19,611][20381] Updated weights for policy 1, policy_version 585446 (0.0008) [2023-12-27 09:41:19,678][20381] Updated weights for policy 1, policy_version 585456 (0.0008) [2023-12-27 09:41:19,903][20347] Updated weights for policy 0, policy_version 585701 (0.0008) [2023-12-27 09:41:19,962][20347] Updated weights for policy 0, policy_version 585711 (0.0006) [2023-12-27 09:41:20,019][20347] Updated weights for policy 0, policy_version 585721 (0.0007) [2023-12-27 09:41:20,446][20381] Updated weights for policy 1, policy_version 585466 (0.0008) [2023-12-27 09:41:20,498][20381] Updated weights for policy 1, policy_version 585476 (0.0008) [2023-12-27 09:41:20,548][20381] Updated weights for policy 1, policy_version 585486 (0.0007) [2023-12-27 09:41:20,666][20347] Updated weights for policy 0, policy_version 585731 (0.0007) [2023-12-27 09:41:20,722][20347] Updated weights for policy 0, policy_version 585741 (0.0006) [2023-12-27 09:41:20,781][20347] Updated weights for policy 0, policy_version 585751 (0.0008) [2023-12-27 09:41:21,322][20381] Updated weights for policy 1, policy_version 585496 (0.0006) [2023-12-27 09:41:21,366][20347] Updated weights for policy 0, policy_version 585761 (0.0008) [2023-12-27 09:41:21,397][20381] Updated weights for policy 1, policy_version 585506 (0.0009) [2023-12-27 09:41:21,425][20347] Updated weights for policy 0, policy_version 585771 (0.0006) [2023-12-27 09:41:21,451][20381] Updated weights for policy 1, policy_version 585516 (0.0007) [2023-12-27 09:41:21,481][20347] Updated weights for policy 0, policy_version 585781 (0.0005) [2023-12-27 09:41:21,535][20347] Updated weights for policy 0, policy_version 585791 (0.0006) [2023-12-27 09:41:22,113][20381] Updated weights for policy 1, policy_version 585526 (0.0007) [2023-12-27 09:41:22,171][20381] Updated weights for policy 1, policy_version 585536 (0.0007) [2023-12-27 09:41:22,234][20381] Updated weights for policy 1, policy_version 585546 (0.0007) [2023-12-27 09:41:22,250][20347] Updated weights for policy 0, policy_version 585801 (0.0006) [2023-12-27 09:41:22,308][20347] Updated weights for policy 0, policy_version 585811 (0.0008) [2023-12-27 09:41:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 299900928. Throughput: 0: 10298.7, 1: 10304.2. Samples: 299871648. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.660')] [2023-12-27 09:41:22,367][20347] Updated weights for policy 0, policy_version 585821 (0.0008) [2023-12-27 09:41:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000585552_149921792.pth... [2023-12-27 09:41:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000584368_149618688.pth [2023-12-27 09:41:22,383][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000585824_149987328.pth... [2023-12-27 09:41:22,387][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000584608_149676032.pth [2023-12-27 09:41:22,884][20381] Updated weights for policy 1, policy_version 585556 (0.0009) [2023-12-27 09:41:22,940][20381] Updated weights for policy 1, policy_version 585566 (0.0011) [2023-12-27 09:41:22,996][20381] Updated weights for policy 1, policy_version 585576 (0.0010) [2023-12-27 09:41:23,044][20347] Updated weights for policy 0, policy_version 585831 (0.0007) [2023-12-27 09:41:23,109][20347] Updated weights for policy 0, policy_version 585841 (0.0006) [2023-12-27 09:41:23,167][20347] Updated weights for policy 0, policy_version 585851 (0.0005) [2023-12-27 09:41:23,674][20381] Updated weights for policy 1, policy_version 585586 (0.0006) [2023-12-27 09:41:23,681][20347] Updated weights for policy 0, policy_version 585861 (0.0008) [2023-12-27 09:41:23,727][20381] Updated weights for policy 1, policy_version 585596 (0.0007) [2023-12-27 09:41:23,733][20347] Updated weights for policy 0, policy_version 585871 (0.0010) [2023-12-27 09:41:23,776][20381] Updated weights for policy 1, policy_version 585606 (0.0005) [2023-12-27 09:41:23,788][20347] Updated weights for policy 0, policy_version 585881 (0.0010) [2023-12-27 09:41:23,835][20381] Updated weights for policy 1, policy_version 585616 (0.0005) [2023-12-27 09:41:24,399][20347] Updated weights for policy 0, policy_version 585891 (0.0008) [2023-12-27 09:41:24,459][20347] Updated weights for policy 0, policy_version 585901 (0.0006) [2023-12-27 09:41:24,472][20381] Updated weights for policy 1, policy_version 585626 (0.0007) [2023-12-27 09:41:24,517][20347] Updated weights for policy 0, policy_version 585911 (0.0007) [2023-12-27 09:41:24,532][20381] Updated weights for policy 1, policy_version 585636 (0.0009) [2023-12-27 09:41:24,588][20381] Updated weights for policy 1, policy_version 585646 (0.0008) [2023-12-27 09:41:25,151][20381] Updated weights for policy 1, policy_version 585656 (0.0005) [2023-12-27 09:41:25,178][20347] Updated weights for policy 0, policy_version 585921 (0.0007) [2023-12-27 09:41:25,205][20381] Updated weights for policy 1, policy_version 585666 (0.0007) [2023-12-27 09:41:25,226][20347] Updated weights for policy 0, policy_version 585931 (0.0005) [2023-12-27 09:41:25,259][20381] Updated weights for policy 1, policy_version 585676 (0.0008) [2023-12-27 09:41:25,283][20347] Updated weights for policy 0, policy_version 585941 (0.0006) [2023-12-27 09:41:25,335][20347] Updated weights for policy 0, policy_version 585951 (0.0008) [2023-12-27 09:41:25,874][20381] Updated weights for policy 1, policy_version 585686 (0.0006) [2023-12-27 09:41:25,931][20381] Updated weights for policy 1, policy_version 585696 (0.0008) [2023-12-27 09:41:25,933][20347] Updated weights for policy 0, policy_version 585961 (0.0006) [2023-12-27 09:41:25,985][20381] Updated weights for policy 1, policy_version 585706 (0.0006) [2023-12-27 09:41:26,009][20347] Updated weights for policy 0, policy_version 585971 (0.0005) [2023-12-27 09:41:26,054][20347] Updated weights for policy 0, policy_version 585981 (0.0008) [2023-12-27 09:41:26,578][20381] Updated weights for policy 1, policy_version 585716 (0.0008) [2023-12-27 09:41:26,623][20381] Updated weights for policy 1, policy_version 585726 (0.0007) [2023-12-27 09:41:26,672][20381] Updated weights for policy 1, policy_version 585736 (0.0007) [2023-12-27 09:41:26,725][20347] Updated weights for policy 0, policy_version 585991 (0.0009) [2023-12-27 09:41:26,776][20347] Updated weights for policy 0, policy_version 586001 (0.0008) [2023-12-27 09:41:26,832][20347] Updated weights for policy 0, policy_version 586011 (0.0009) [2023-12-27 09:41:27,230][20381] Updated weights for policy 1, policy_version 585746 (0.0005) [2023-12-27 09:41:27,287][20381] Updated weights for policy 1, policy_version 585756 (0.0005) [2023-12-27 09:41:27,343][20381] Updated weights for policy 1, policy_version 585766 (0.0007) [2023-12-27 09:41:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 300007424. Throughput: 0: 10375.2, 1: 10347.7. Samples: 300000144. Policy #0 lag: (min: 31.0, avg: 32.5, max: 63.0) [2023-12-27 09:41:27,363][19170] Avg episode reward: [(0, '990.680'), (1, '903.010')] [2023-12-27 09:41:27,401][20381] Updated weights for policy 1, policy_version 585776 (0.0010) [2023-12-27 09:41:27,483][20347] Updated weights for policy 0, policy_version 586021 (0.0008) [2023-12-27 09:41:27,535][20347] Updated weights for policy 0, policy_version 586031 (0.0005) [2023-12-27 09:41:27,589][20347] Updated weights for policy 0, policy_version 586041 (0.0007) [2023-12-27 09:41:27,966][20381] Updated weights for policy 1, policy_version 585786 (0.0005) [2023-12-27 09:41:28,026][20381] Updated weights for policy 1, policy_version 585796 (0.0008) [2023-12-27 09:41:28,077][20381] Updated weights for policy 1, policy_version 585806 (0.0010) [2023-12-27 09:41:28,207][20347] Updated weights for policy 0, policy_version 586051 (0.0010) [2023-12-27 09:41:28,264][20347] Updated weights for policy 0, policy_version 586061 (0.0010) [2023-12-27 09:41:28,311][20347] Updated weights for policy 0, policy_version 586071 (0.0010) [2023-12-27 09:41:28,648][20381] Updated weights for policy 1, policy_version 585816 (0.0010) [2023-12-27 09:41:28,698][20381] Updated weights for policy 1, policy_version 585826 (0.0010) [2023-12-27 09:41:28,750][20381] Updated weights for policy 1, policy_version 585836 (0.0011) [2023-12-27 09:41:28,982][20347] Updated weights for policy 0, policy_version 586081 (0.0010) [2023-12-27 09:41:29,034][20347] Updated weights for policy 0, policy_version 586091 (0.0010) [2023-12-27 09:41:29,084][20347] Updated weights for policy 0, policy_version 586101 (0.0010) [2023-12-27 09:41:29,138][20347] Updated weights for policy 0, policy_version 586111 (0.0010) [2023-12-27 09:41:29,444][20381] Updated weights for policy 1, policy_version 585846 (0.0010) [2023-12-27 09:41:29,505][20381] Updated weights for policy 1, policy_version 585856 (0.0009) [2023-12-27 09:41:29,567][20381] Updated weights for policy 1, policy_version 585866 (0.0010) [2023-12-27 09:41:29,835][20347] Updated weights for policy 0, policy_version 586121 (0.0009) [2023-12-27 09:41:29,897][20347] Updated weights for policy 0, policy_version 586131 (0.0009) [2023-12-27 09:41:29,957][20347] Updated weights for policy 0, policy_version 586141 (0.0009) [2023-12-27 09:41:30,266][20381] Updated weights for policy 1, policy_version 585876 (0.0009) [2023-12-27 09:41:30,325][20381] Updated weights for policy 1, policy_version 585886 (0.0006) [2023-12-27 09:41:30,380][20381] Updated weights for policy 1, policy_version 585896 (0.0010) [2023-12-27 09:41:30,653][20347] Updated weights for policy 0, policy_version 586151 (0.0007) [2023-12-27 09:41:30,697][20347] Updated weights for policy 0, policy_version 586161 (0.0007) [2023-12-27 09:41:30,738][20347] Updated weights for policy 0, policy_version 586171 (0.0007) [2023-12-27 09:41:30,990][20381] Updated weights for policy 1, policy_version 585906 (0.0009) [2023-12-27 09:41:31,049][20381] Updated weights for policy 1, policy_version 585916 (0.0006) [2023-12-27 09:41:31,102][20381] Updated weights for policy 1, policy_version 585926 (0.0008) [2023-12-27 09:41:31,163][20381] Updated weights for policy 1, policy_version 585936 (0.0007) [2023-12-27 09:41:31,431][20347] Updated weights for policy 0, policy_version 586181 (0.0008) [2023-12-27 09:41:31,486][20347] Updated weights for policy 0, policy_version 586191 (0.0008) [2023-12-27 09:41:31,540][20347] Updated weights for policy 0, policy_version 586201 (0.0008) [2023-12-27 09:41:31,799][20381] Updated weights for policy 1, policy_version 585946 (0.0008) [2023-12-27 09:41:31,858][20381] Updated weights for policy 1, policy_version 585956 (0.0008) [2023-12-27 09:41:31,918][20381] Updated weights for policy 1, policy_version 585966 (0.0008) [2023-12-27 09:41:32,198][20347] Updated weights for policy 0, policy_version 586211 (0.0007) [2023-12-27 09:41:32,262][20347] Updated weights for policy 0, policy_version 586221 (0.0007) [2023-12-27 09:41:32,319][20347] Updated weights for policy 0, policy_version 586231 (0.0007) [2023-12-27 09:41:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 300113920. Throughput: 0: 10435.7, 1: 10384.2. Samples: 300128912. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:32,363][19170] Avg episode reward: [(0, '981.850'), (1, '835.720')] [2023-12-27 09:41:32,640][20381] Updated weights for policy 1, policy_version 585976 (0.0008) [2023-12-27 09:41:32,695][20381] Updated weights for policy 1, policy_version 585986 (0.0010) [2023-12-27 09:41:32,753][20381] Updated weights for policy 1, policy_version 585996 (0.0010) [2023-12-27 09:41:32,967][20347] Updated weights for policy 0, policy_version 586241 (0.0008) [2023-12-27 09:41:33,023][20347] Updated weights for policy 0, policy_version 586251 (0.0006) [2023-12-27 09:41:33,074][20347] Updated weights for policy 0, policy_version 586261 (0.0008) [2023-12-27 09:41:33,122][20347] Updated weights for policy 0, policy_version 586271 (0.0007) [2023-12-27 09:41:33,432][20381] Updated weights for policy 1, policy_version 586006 (0.0007) [2023-12-27 09:41:33,487][20381] Updated weights for policy 1, policy_version 586016 (0.0005) [2023-12-27 09:41:33,542][20381] Updated weights for policy 1, policy_version 586026 (0.0010) [2023-12-27 09:41:33,774][20347] Updated weights for policy 0, policy_version 586281 (0.0005) [2023-12-27 09:41:33,830][20347] Updated weights for policy 0, policy_version 586291 (0.0005) [2023-12-27 09:41:33,893][20347] Updated weights for policy 0, policy_version 586301 (0.0005) [2023-12-27 09:41:34,130][20381] Updated weights for policy 1, policy_version 586036 (0.0011) [2023-12-27 09:41:34,186][20381] Updated weights for policy 1, policy_version 586046 (0.0007) [2023-12-27 09:41:34,246][20381] Updated weights for policy 1, policy_version 586056 (0.0009) [2023-12-27 09:41:34,502][20347] Updated weights for policy 0, policy_version 586311 (0.0007) [2023-12-27 09:41:34,559][20347] Updated weights for policy 0, policy_version 586321 (0.0008) [2023-12-27 09:41:34,614][20347] Updated weights for policy 0, policy_version 586331 (0.0008) [2023-12-27 09:41:34,899][20381] Updated weights for policy 1, policy_version 586066 (0.0008) [2023-12-27 09:41:34,957][20381] Updated weights for policy 1, policy_version 586076 (0.0007) [2023-12-27 09:41:35,012][20381] Updated weights for policy 1, policy_version 586086 (0.0006) [2023-12-27 09:41:35,067][20381] Updated weights for policy 1, policy_version 586096 (0.0005) [2023-12-27 09:41:35,333][20347] Updated weights for policy 0, policy_version 586341 (0.0007) [2023-12-27 09:41:35,396][20347] Updated weights for policy 0, policy_version 586351 (0.0008) [2023-12-27 09:41:35,455][20347] Updated weights for policy 0, policy_version 586361 (0.0007) [2023-12-27 09:41:35,726][20381] Updated weights for policy 1, policy_version 586106 (0.0007) [2023-12-27 09:41:35,781][20381] Updated weights for policy 1, policy_version 586116 (0.0007) [2023-12-27 09:41:35,841][20381] Updated weights for policy 1, policy_version 586126 (0.0008) [2023-12-27 09:41:36,154][20347] Updated weights for policy 0, policy_version 586371 (0.0008) [2023-12-27 09:41:36,207][20347] Updated weights for policy 0, policy_version 586381 (0.0008) [2023-12-27 09:41:36,263][20347] Updated weights for policy 0, policy_version 586391 (0.0008) [2023-12-27 09:41:36,535][20381] Updated weights for policy 1, policy_version 586136 (0.0008) [2023-12-27 09:41:36,594][20381] Updated weights for policy 1, policy_version 586146 (0.0008) [2023-12-27 09:41:36,657][20381] Updated weights for policy 1, policy_version 586156 (0.0008) [2023-12-27 09:41:36,975][20347] Updated weights for policy 0, policy_version 586401 (0.0008) [2023-12-27 09:41:37,023][20347] Updated weights for policy 0, policy_version 586411 (0.0010) [2023-12-27 09:41:37,071][20347] Updated weights for policy 0, policy_version 586421 (0.0010) [2023-12-27 09:41:37,135][20347] Updated weights for policy 0, policy_version 586431 (0.0010) [2023-12-27 09:41:37,307][20381] Updated weights for policy 1, policy_version 586166 (0.0008) [2023-12-27 09:41:37,360][20381] Updated weights for policy 1, policy_version 586176 (0.0007) [2023-12-27 09:41:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 300220416. Throughput: 0: 10461.5, 1: 10379.0. Samples: 300191880. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:37,363][19170] Avg episode reward: [(0, '991.170'), (1, '797.510')] [2023-12-27 09:41:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000586432_150142976.pth... [2023-12-27 09:41:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000585216_149831680.pth [2023-12-27 09:41:37,415][20381] Updated weights for policy 1, policy_version 586186 (0.0007) [2023-12-27 09:41:37,452][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000586192_150085632.pth... [2023-12-27 09:41:37,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000584944_149766144.pth [2023-12-27 09:41:37,707][20347] Updated weights for policy 0, policy_version 586441 (0.0008) [2023-12-27 09:41:37,759][20347] Updated weights for policy 0, policy_version 586451 (0.0010) [2023-12-27 09:41:37,817][20347] Updated weights for policy 0, policy_version 586461 (0.0008) [2023-12-27 09:41:38,123][20381] Updated weights for policy 1, policy_version 586196 (0.0007) [2023-12-27 09:41:38,176][20381] Updated weights for policy 1, policy_version 586206 (0.0008) [2023-12-27 09:41:38,226][20381] Updated weights for policy 1, policy_version 586217 (0.0008) [2023-12-27 09:41:38,483][20347] Updated weights for policy 0, policy_version 586471 (0.0008) [2023-12-27 09:41:38,534][20347] Updated weights for policy 0, policy_version 586481 (0.0007) [2023-12-27 09:41:38,583][20347] Updated weights for policy 0, policy_version 586491 (0.0006) [2023-12-27 09:41:38,948][20381] Updated weights for policy 1, policy_version 586227 (0.0007) [2023-12-27 09:41:39,002][20381] Updated weights for policy 1, policy_version 586237 (0.0008) [2023-12-27 09:41:39,057][20381] Updated weights for policy 1, policy_version 586247 (0.0008) [2023-12-27 09:41:39,275][20347] Updated weights for policy 0, policy_version 586501 (0.0009) [2023-12-27 09:41:39,339][20347] Updated weights for policy 0, policy_version 586511 (0.0011) [2023-12-27 09:41:39,400][20347] Updated weights for policy 0, policy_version 586521 (0.0011) [2023-12-27 09:41:39,751][20381] Updated weights for policy 1, policy_version 586257 (0.0007) [2023-12-27 09:41:39,812][20381] Updated weights for policy 1, policy_version 586267 (0.0008) [2023-12-27 09:41:39,877][20381] Updated weights for policy 1, policy_version 586277 (0.0008) [2023-12-27 09:41:39,937][20381] Updated weights for policy 1, policy_version 586287 (0.0007) [2023-12-27 09:41:40,075][20347] Updated weights for policy 0, policy_version 586531 (0.0008) [2023-12-27 09:41:40,136][20347] Updated weights for policy 0, policy_version 586541 (0.0010) [2023-12-27 09:41:40,199][20347] Updated weights for policy 0, policy_version 586551 (0.0010) [2023-12-27 09:41:40,640][20381] Updated weights for policy 1, policy_version 586297 (0.0008) [2023-12-27 09:41:40,694][20381] Updated weights for policy 1, policy_version 586307 (0.0009) [2023-12-27 09:41:40,743][20381] Updated weights for policy 1, policy_version 586317 (0.0007) [2023-12-27 09:41:40,796][20347] Updated weights for policy 0, policy_version 586561 (0.0011) [2023-12-27 09:41:40,860][20347] Updated weights for policy 0, policy_version 586571 (0.0010) [2023-12-27 09:41:40,918][20347] Updated weights for policy 0, policy_version 586581 (0.0010) [2023-12-27 09:41:40,969][20347] Updated weights for policy 0, policy_version 586591 (0.0005) [2023-12-27 09:41:41,485][20381] Updated weights for policy 1, policy_version 586327 (0.0008) [2023-12-27 09:41:41,541][20381] Updated weights for policy 1, policy_version 586337 (0.0007) [2023-12-27 09:41:41,596][20381] Updated weights for policy 1, policy_version 586347 (0.0008) [2023-12-27 09:41:41,669][20347] Updated weights for policy 0, policy_version 586601 (0.0010) [2023-12-27 09:41:41,723][20347] Updated weights for policy 0, policy_version 586611 (0.0010) [2023-12-27 09:41:41,784][20347] Updated weights for policy 0, policy_version 586621 (0.0010) [2023-12-27 09:41:42,332][20381] Updated weights for policy 1, policy_version 586357 (0.0008) [2023-12-27 09:41:42,363][19170] Fps is (10 sec: 20478.8, 60 sec: 20752.8, 300 sec: 20799.3). Total num frames: 300318720. Throughput: 0: 10394.5, 1: 10382.5. Samples: 300314720. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:42,364][19170] Avg episode reward: [(0, '1000.000'), (1, '855.570')] [2023-12-27 09:41:42,401][20381] Updated weights for policy 1, policy_version 586367 (0.0009) [2023-12-27 09:41:42,456][20381] Updated weights for policy 1, policy_version 586377 (0.0007) [2023-12-27 09:41:42,502][20347] Updated weights for policy 0, policy_version 586631 (0.0010) [2023-12-27 09:41:42,559][20347] Updated weights for policy 0, policy_version 586641 (0.0010) [2023-12-27 09:41:42,614][20347] Updated weights for policy 0, policy_version 586651 (0.0008) [2023-12-27 09:41:43,163][20381] Updated weights for policy 1, policy_version 586387 (0.0008) [2023-12-27 09:41:43,208][20381] Updated weights for policy 1, policy_version 586397 (0.0007) [2023-12-27 09:41:43,257][20381] Updated weights for policy 1, policy_version 586407 (0.0008) [2023-12-27 09:41:43,316][20347] Updated weights for policy 0, policy_version 586661 (0.0009) [2023-12-27 09:41:43,382][20347] Updated weights for policy 0, policy_version 586671 (0.0009) [2023-12-27 09:41:43,441][20347] Updated weights for policy 0, policy_version 586681 (0.0006) [2023-12-27 09:41:43,950][20381] Updated weights for policy 1, policy_version 586417 (0.0007) [2023-12-27 09:41:44,007][20381] Updated weights for policy 1, policy_version 586427 (0.0006) [2023-12-27 09:41:44,042][20347] Updated weights for policy 0, policy_version 586691 (0.0006) [2023-12-27 09:41:44,066][20381] Updated weights for policy 1, policy_version 586437 (0.0008) [2023-12-27 09:41:44,091][20347] Updated weights for policy 0, policy_version 586701 (0.0005) [2023-12-27 09:41:44,122][20381] Updated weights for policy 1, policy_version 586447 (0.0007) [2023-12-27 09:41:44,138][20347] Updated weights for policy 0, policy_version 586711 (0.0005) [2023-12-27 09:41:44,757][20381] Updated weights for policy 1, policy_version 586457 (0.0008) [2023-12-27 09:41:44,783][20347] Updated weights for policy 0, policy_version 586721 (0.0005) [2023-12-27 09:41:44,822][20381] Updated weights for policy 1, policy_version 586467 (0.0008) [2023-12-27 09:41:44,846][20347] Updated weights for policy 0, policy_version 586731 (0.0009) [2023-12-27 09:41:44,880][20381] Updated weights for policy 1, policy_version 586477 (0.0008) [2023-12-27 09:41:44,906][20347] Updated weights for policy 0, policy_version 586741 (0.0007) [2023-12-27 09:41:44,969][20347] Updated weights for policy 0, policy_version 586751 (0.0007) [2023-12-27 09:41:45,602][20381] Updated weights for policy 1, policy_version 586487 (0.0007) [2023-12-27 09:41:45,649][20347] Updated weights for policy 0, policy_version 586761 (0.0007) [2023-12-27 09:41:45,666][20381] Updated weights for policy 1, policy_version 586497 (0.0006) [2023-12-27 09:41:45,706][20347] Updated weights for policy 0, policy_version 586771 (0.0005) [2023-12-27 09:41:45,722][20381] Updated weights for policy 1, policy_version 586507 (0.0007) [2023-12-27 09:41:45,756][20347] Updated weights for policy 0, policy_version 586781 (0.0007) [2023-12-27 09:41:46,385][20381] Updated weights for policy 1, policy_version 586517 (0.0006) [2023-12-27 09:41:46,449][20381] Updated weights for policy 1, policy_version 586527 (0.0005) [2023-12-27 09:41:46,459][20347] Updated weights for policy 0, policy_version 586791 (0.0007) [2023-12-27 09:41:46,507][20381] Updated weights for policy 1, policy_version 586537 (0.0006) [2023-12-27 09:41:46,509][20347] Updated weights for policy 0, policy_version 586801 (0.0006) [2023-12-27 09:41:46,564][20347] Updated weights for policy 0, policy_version 586811 (0.0007) [2023-12-27 09:41:47,047][20381] Updated weights for policy 1, policy_version 586547 (0.0006) [2023-12-27 09:41:47,102][20381] Updated weights for policy 1, policy_version 586557 (0.0006) [2023-12-27 09:41:47,156][20381] Updated weights for policy 1, policy_version 586567 (0.0005) [2023-12-27 09:41:47,217][20347] Updated weights for policy 0, policy_version 586821 (0.0009) [2023-12-27 09:41:47,275][20347] Updated weights for policy 0, policy_version 586831 (0.0010) [2023-12-27 09:41:47,339][20347] Updated weights for policy 0, policy_version 586841 (0.0010) [2023-12-27 09:41:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 300425216. Throughput: 0: 10411.3, 1: 10380.5. Samples: 300437632. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '884.420')] [2023-12-27 09:41:47,741][20381] Updated weights for policy 1, policy_version 586577 (0.0006) [2023-12-27 09:41:47,796][20381] Updated weights for policy 1, policy_version 586587 (0.0009) [2023-12-27 09:41:47,840][20381] Updated weights for policy 1, policy_version 586597 (0.0005) [2023-12-27 09:41:47,892][20381] Updated weights for policy 1, policy_version 586607 (0.0005) [2023-12-27 09:41:47,990][20347] Updated weights for policy 0, policy_version 586851 (0.0010) [2023-12-27 09:41:48,041][20347] Updated weights for policy 0, policy_version 586861 (0.0008) [2023-12-27 09:41:48,099][20347] Updated weights for policy 0, policy_version 586871 (0.0010) [2023-12-27 09:41:48,569][20381] Updated weights for policy 1, policy_version 586617 (0.0010) [2023-12-27 09:41:48,621][20381] Updated weights for policy 1, policy_version 586627 (0.0010) [2023-12-27 09:41:48,673][20381] Updated weights for policy 1, policy_version 586637 (0.0010) [2023-12-27 09:41:48,795][20347] Updated weights for policy 0, policy_version 586881 (0.0010) [2023-12-27 09:41:48,849][20347] Updated weights for policy 0, policy_version 586891 (0.0010) [2023-12-27 09:41:48,903][20347] Updated weights for policy 0, policy_version 586901 (0.0010) [2023-12-27 09:41:48,955][20347] Updated weights for policy 0, policy_version 586911 (0.0010) [2023-12-27 09:41:49,299][20381] Updated weights for policy 1, policy_version 586647 (0.0011) [2023-12-27 09:41:49,365][20381] Updated weights for policy 1, policy_version 586657 (0.0011) [2023-12-27 09:41:49,422][20381] Updated weights for policy 1, policy_version 586667 (0.0006) [2023-12-27 09:41:49,621][20347] Updated weights for policy 0, policy_version 586921 (0.0007) [2023-12-27 09:41:49,678][20347] Updated weights for policy 0, policy_version 586931 (0.0008) [2023-12-27 09:41:49,734][20347] Updated weights for policy 0, policy_version 586941 (0.0007) [2023-12-27 09:41:50,033][20381] Updated weights for policy 1, policy_version 586677 (0.0011) [2023-12-27 09:41:50,097][20381] Updated weights for policy 1, policy_version 586687 (0.0011) [2023-12-27 09:41:50,162][20381] Updated weights for policy 1, policy_version 586697 (0.0009) [2023-12-27 09:41:50,397][20347] Updated weights for policy 0, policy_version 586951 (0.0006) [2023-12-27 09:41:50,459][20347] Updated weights for policy 0, policy_version 586961 (0.0007) [2023-12-27 09:41:50,514][20347] Updated weights for policy 0, policy_version 586971 (0.0009) [2023-12-27 09:41:50,787][20381] Updated weights for policy 1, policy_version 586707 (0.0008) [2023-12-27 09:41:50,853][20381] Updated weights for policy 1, policy_version 586717 (0.0006) [2023-12-27 09:41:50,910][20381] Updated weights for policy 1, policy_version 586727 (0.0007) [2023-12-27 09:41:51,177][20347] Updated weights for policy 0, policy_version 586981 (0.0006) [2023-12-27 09:41:51,237][20347] Updated weights for policy 0, policy_version 586991 (0.0005) [2023-12-27 09:41:51,299][20347] Updated weights for policy 0, policy_version 587001 (0.0006) [2023-12-27 09:41:51,593][20381] Updated weights for policy 1, policy_version 586737 (0.0007) [2023-12-27 09:41:51,656][20381] Updated weights for policy 1, policy_version 586747 (0.0007) [2023-12-27 09:41:51,711][20381] Updated weights for policy 1, policy_version 586757 (0.0009) [2023-12-27 09:41:51,769][20381] Updated weights for policy 1, policy_version 586767 (0.0008) [2023-12-27 09:41:51,920][20347] Updated weights for policy 0, policy_version 587011 (0.0007) [2023-12-27 09:41:51,977][20347] Updated weights for policy 0, policy_version 587021 (0.0007) [2023-12-27 09:41:52,037][20347] Updated weights for policy 0, policy_version 587031 (0.0008) [2023-12-27 09:41:52,363][19170] Fps is (10 sec: 21300.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 300531712. Throughput: 0: 10421.8, 1: 10417.8. Samples: 300501480. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.450')] [2023-12-27 09:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000586768_150233088.pth... [2023-12-27 09:41:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000587040_150298624.pth... [2023-12-27 09:41:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000585824_149987328.pth [2023-12-27 09:41:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000585552_149921792.pth [2023-12-27 09:41:52,481][20381] Updated weights for policy 1, policy_version 586777 (0.0008) [2023-12-27 09:41:52,545][20381] Updated weights for policy 1, policy_version 586787 (0.0008) [2023-12-27 09:41:52,598][20381] Updated weights for policy 1, policy_version 586797 (0.0009) [2023-12-27 09:41:52,657][20347] Updated weights for policy 0, policy_version 587041 (0.0008) [2023-12-27 09:41:52,717][20347] Updated weights for policy 0, policy_version 587051 (0.0008) [2023-12-27 09:41:52,766][20347] Updated weights for policy 0, policy_version 587061 (0.0005) [2023-12-27 09:41:52,826][20347] Updated weights for policy 0, policy_version 587071 (0.0005) [2023-12-27 09:41:53,345][20381] Updated weights for policy 1, policy_version 586807 (0.0009) [2023-12-27 09:41:53,400][20381] Updated weights for policy 1, policy_version 586817 (0.0005) [2023-12-27 09:41:53,427][20347] Updated weights for policy 0, policy_version 587081 (0.0005) [2023-12-27 09:41:53,463][20381] Updated weights for policy 1, policy_version 586827 (0.0005) [2023-12-27 09:41:53,476][20347] Updated weights for policy 0, policy_version 587091 (0.0007) [2023-12-27 09:41:53,519][20347] Updated weights for policy 0, policy_version 587101 (0.0009) [2023-12-27 09:41:54,145][20381] Updated weights for policy 1, policy_version 586837 (0.0006) [2023-12-27 09:41:54,196][20347] Updated weights for policy 0, policy_version 587111 (0.0009) [2023-12-27 09:41:54,202][20381] Updated weights for policy 1, policy_version 586847 (0.0007) [2023-12-27 09:41:54,253][20347] Updated weights for policy 0, policy_version 587121 (0.0007) [2023-12-27 09:41:54,262][20381] Updated weights for policy 1, policy_version 586857 (0.0006) [2023-12-27 09:41:54,313][20347] Updated weights for policy 0, policy_version 587131 (0.0010) [2023-12-27 09:41:54,949][20381] Updated weights for policy 1, policy_version 586867 (0.0006) [2023-12-27 09:41:54,983][20347] Updated weights for policy 0, policy_version 587141 (0.0008) [2023-12-27 09:41:55,007][20381] Updated weights for policy 1, policy_version 586877 (0.0007) [2023-12-27 09:41:55,035][20347] Updated weights for policy 0, policy_version 587151 (0.0005) [2023-12-27 09:41:55,060][20381] Updated weights for policy 1, policy_version 586887 (0.0007) [2023-12-27 09:41:55,086][20347] Updated weights for policy 0, policy_version 587161 (0.0005) [2023-12-27 09:41:55,717][20347] Updated weights for policy 0, policy_version 587171 (0.0006) [2023-12-27 09:41:55,763][20381] Updated weights for policy 1, policy_version 586897 (0.0008) [2023-12-27 09:41:55,776][20347] Updated weights for policy 0, policy_version 587181 (0.0007) [2023-12-27 09:41:55,822][20381] Updated weights for policy 1, policy_version 586907 (0.0005) [2023-12-27 09:41:55,826][20347] Updated weights for policy 0, policy_version 587191 (0.0008) [2023-12-27 09:41:55,880][20381] Updated weights for policy 1, policy_version 586917 (0.0005) [2023-12-27 09:41:55,936][20381] Updated weights for policy 1, policy_version 586927 (0.0008) [2023-12-27 09:41:56,479][20347] Updated weights for policy 0, policy_version 587201 (0.0007) [2023-12-27 09:41:56,521][20347] Updated weights for policy 0, policy_version 587211 (0.0008) [2023-12-27 09:41:56,526][20381] Updated weights for policy 1, policy_version 586937 (0.0006) [2023-12-27 09:41:56,569][20347] Updated weights for policy 0, policy_version 587221 (0.0007) [2023-12-27 09:41:56,576][20381] Updated weights for policy 1, policy_version 586947 (0.0005) [2023-12-27 09:41:56,620][20347] Updated weights for policy 0, policy_version 587231 (0.0008) [2023-12-27 09:41:56,626][20381] Updated weights for policy 1, policy_version 586957 (0.0005) [2023-12-27 09:41:57,239][20381] Updated weights for policy 1, policy_version 586967 (0.0007) [2023-12-27 09:41:57,296][20381] Updated weights for policy 1, policy_version 586977 (0.0008) [2023-12-27 09:41:57,322][20347] Updated weights for policy 0, policy_version 587241 (0.0006) [2023-12-27 09:41:57,351][20381] Updated weights for policy 1, policy_version 586987 (0.0008) [2023-12-27 09:41:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.7, 300 sec: 20799.3). Total num frames: 300630016. Throughput: 0: 10473.6, 1: 10382.1. Samples: 300625652. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:41:57,363][19170] Avg episode reward: [(0, '990.400'), (1, '768.500')] [2023-12-27 09:41:57,372][20347] Updated weights for policy 0, policy_version 587251 (0.0005) [2023-12-27 09:41:57,419][20347] Updated weights for policy 0, policy_version 587261 (0.0005) [2023-12-27 09:41:58,021][20347] Updated weights for policy 0, policy_version 587271 (0.0007) [2023-12-27 09:41:58,060][20381] Updated weights for policy 1, policy_version 586997 (0.0007) [2023-12-27 09:41:58,069][20347] Updated weights for policy 0, policy_version 587281 (0.0006) [2023-12-27 09:41:58,114][20347] Updated weights for policy 0, policy_version 587291 (0.0006) [2023-12-27 09:41:58,117][20381] Updated weights for policy 1, policy_version 587007 (0.0006) [2023-12-27 09:41:58,167][20381] Updated weights for policy 1, policy_version 587017 (0.0006) [2023-12-27 09:41:58,878][20347] Updated weights for policy 0, policy_version 587301 (0.0008) [2023-12-27 09:41:58,902][20381] Updated weights for policy 1, policy_version 587027 (0.0008) [2023-12-27 09:41:58,930][20347] Updated weights for policy 0, policy_version 587311 (0.0006) [2023-12-27 09:41:58,933][20292] KL-divergence is very high: 151.0012 [2023-12-27 09:41:58,965][20381] Updated weights for policy 1, policy_version 587037 (0.0007) [2023-12-27 09:41:58,989][20347] Updated weights for policy 0, policy_version 587321 (0.0009) [2023-12-27 09:41:59,021][20381] Updated weights for policy 1, policy_version 587047 (0.0006) [2023-12-27 09:41:59,661][20347] Updated weights for policy 0, policy_version 587331 (0.0008) [2023-12-27 09:41:59,707][20347] Updated weights for policy 0, policy_version 587341 (0.0005) [2023-12-27 09:41:59,755][20347] Updated weights for policy 0, policy_version 587351 (0.0010) [2023-12-27 09:41:59,779][20381] Updated weights for policy 1, policy_version 587057 (0.0007) [2023-12-27 09:41:59,845][20381] Updated weights for policy 1, policy_version 587067 (0.0007) [2023-12-27 09:41:59,907][20381] Updated weights for policy 1, policy_version 587077 (0.0008) [2023-12-27 09:41:59,965][20381] Updated weights for policy 1, policy_version 587087 (0.0006) [2023-12-27 09:42:00,431][20347] Updated weights for policy 0, policy_version 587361 (0.0010) [2023-12-27 09:42:00,476][20347] Updated weights for policy 0, policy_version 587371 (0.0008) [2023-12-27 09:42:00,518][20347] Updated weights for policy 0, policy_version 587381 (0.0007) [2023-12-27 09:42:00,564][20347] Updated weights for policy 0, policy_version 587391 (0.0007) [2023-12-27 09:42:00,647][20381] Updated weights for policy 1, policy_version 587097 (0.0008) [2023-12-27 09:42:00,698][20381] Updated weights for policy 1, policy_version 587107 (0.0007) [2023-12-27 09:42:00,747][20381] Updated weights for policy 1, policy_version 587117 (0.0008) [2023-12-27 09:42:01,240][20347] Updated weights for policy 0, policy_version 587401 (0.0007) [2023-12-27 09:42:01,296][20347] Updated weights for policy 0, policy_version 587411 (0.0008) [2023-12-27 09:42:01,354][20347] Updated weights for policy 0, policy_version 587421 (0.0007) [2023-12-27 09:42:01,407][20381] Updated weights for policy 1, policy_version 587127 (0.0008) [2023-12-27 09:42:01,463][20381] Updated weights for policy 1, policy_version 587137 (0.0008) [2023-12-27 09:42:01,515][20381] Updated weights for policy 1, policy_version 587147 (0.0007) [2023-12-27 09:42:02,075][20347] Updated weights for policy 0, policy_version 587431 (0.0009) [2023-12-27 09:42:02,121][20347] Updated weights for policy 0, policy_version 587441 (0.0009) [2023-12-27 09:42:02,162][20347] Updated weights for policy 0, policy_version 587451 (0.0008) [2023-12-27 09:42:02,213][20381] Updated weights for policy 1, policy_version 587157 (0.0007) [2023-12-27 09:42:02,274][20381] Updated weights for policy 1, policy_version 587167 (0.0009) [2023-12-27 09:42:02,331][20381] Updated weights for policy 1, policy_version 587177 (0.0007) [2023-12-27 09:42:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 300736512. Throughput: 0: 10474.1, 1: 10353.6. Samples: 300748436. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:02,363][19170] Avg episode reward: [(0, '990.400'), (1, '750.050')] [2023-12-27 09:42:02,885][20347] Updated weights for policy 0, policy_version 587461 (0.0008) [2023-12-27 09:42:02,931][20347] Updated weights for policy 0, policy_version 587471 (0.0007) [2023-12-27 09:42:02,964][20381] Updated weights for policy 1, policy_version 587187 (0.0008) [2023-12-27 09:42:02,982][20347] Updated weights for policy 0, policy_version 587481 (0.0006) [2023-12-27 09:42:03,014][20381] Updated weights for policy 1, policy_version 587197 (0.0011) [2023-12-27 09:42:03,066][20381] Updated weights for policy 1, policy_version 587207 (0.0010) [2023-12-27 09:42:03,590][20347] Updated weights for policy 0, policy_version 587491 (0.0006) [2023-12-27 09:42:03,639][20347] Updated weights for policy 0, policy_version 587501 (0.0005) [2023-12-27 09:42:03,662][20381] Updated weights for policy 1, policy_version 587217 (0.0010) [2023-12-27 09:42:03,692][20347] Updated weights for policy 0, policy_version 587511 (0.0005) [2023-12-27 09:42:03,721][20381] Updated weights for policy 1, policy_version 587227 (0.0006) [2023-12-27 09:42:03,775][20381] Updated weights for policy 1, policy_version 587237 (0.0009) [2023-12-27 09:42:03,824][20381] Updated weights for policy 1, policy_version 587247 (0.0010) [2023-12-27 09:42:04,288][20347] Updated weights for policy 0, policy_version 587521 (0.0007) [2023-12-27 09:42:04,348][20347] Updated weights for policy 0, policy_version 587531 (0.0007) [2023-12-27 09:42:04,350][20291] KL-divergence is very high: 106.3365 [2023-12-27 09:42:04,408][20347] Updated weights for policy 0, policy_version 587541 (0.0007) [2023-12-27 09:42:04,462][20347] Updated weights for policy 0, policy_version 587551 (0.0007) [2023-12-27 09:42:04,505][20381] Updated weights for policy 1, policy_version 587257 (0.0010) [2023-12-27 09:42:04,568][20381] Updated weights for policy 1, policy_version 587267 (0.0010) [2023-12-27 09:42:04,629][20381] Updated weights for policy 1, policy_version 587277 (0.0011) [2023-12-27 09:42:05,063][20347] Updated weights for policy 0, policy_version 587561 (0.0006) [2023-12-27 09:42:05,118][20347] Updated weights for policy 0, policy_version 587571 (0.0007) [2023-12-27 09:42:05,179][20347] Updated weights for policy 0, policy_version 587581 (0.0007) [2023-12-27 09:42:05,281][20381] Updated weights for policy 1, policy_version 587287 (0.0007) [2023-12-27 09:42:05,341][20381] Updated weights for policy 1, policy_version 587297 (0.0006) [2023-12-27 09:42:05,405][20381] Updated weights for policy 1, policy_version 587307 (0.0006) [2023-12-27 09:42:05,801][20347] Updated weights for policy 0, policy_version 587591 (0.0009) [2023-12-27 09:42:05,853][20347] Updated weights for policy 0, policy_version 587601 (0.0007) [2023-12-27 09:42:05,901][20347] Updated weights for policy 0, policy_version 587611 (0.0007) [2023-12-27 09:42:05,988][20381] Updated weights for policy 1, policy_version 587317 (0.0007) [2023-12-27 09:42:06,041][20381] Updated weights for policy 1, policy_version 587327 (0.0008) [2023-12-27 09:42:06,101][20381] Updated weights for policy 1, policy_version 587337 (0.0008) [2023-12-27 09:42:06,634][20347] Updated weights for policy 0, policy_version 587621 (0.0009) [2023-12-27 09:42:06,693][20347] Updated weights for policy 0, policy_version 587631 (0.0008) [2023-12-27 09:42:06,752][20347] Updated weights for policy 0, policy_version 587641 (0.0005) [2023-12-27 09:42:06,801][20381] Updated weights for policy 1, policy_version 587347 (0.0007) [2023-12-27 09:42:06,820][20292] KL-divergence is very high: 113.5912 [2023-12-27 09:42:06,860][20381] Updated weights for policy 1, policy_version 587357 (0.0008) [2023-12-27 09:42:06,912][20381] Updated weights for policy 1, policy_version 587367 (0.0008) [2023-12-27 09:42:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 300843008. Throughput: 0: 10523.8, 1: 10402.2. Samples: 300813312. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:07,363][19170] Avg episode reward: [(0, '990.440'), (1, '644.900')] [2023-12-27 09:42:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000587376_150388736.pth... [2023-12-27 09:42:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000587648_150454272.pth... [2023-12-27 09:42:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000586192_150085632.pth [2023-12-27 09:42:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000586432_150142976.pth [2023-12-27 09:42:07,409][20347] Updated weights for policy 0, policy_version 587651 (0.0008) [2023-12-27 09:42:07,467][20347] Updated weights for policy 0, policy_version 587661 (0.0010) [2023-12-27 09:42:07,525][20347] Updated weights for policy 0, policy_version 587671 (0.0010) [2023-12-27 09:42:07,614][20381] Updated weights for policy 1, policy_version 587377 (0.0007) [2023-12-27 09:42:07,675][20381] Updated weights for policy 1, policy_version 587387 (0.0007) [2023-12-27 09:42:07,732][20381] Updated weights for policy 1, policy_version 587397 (0.0007) [2023-12-27 09:42:07,792][20381] Updated weights for policy 1, policy_version 587407 (0.0006) [2023-12-27 09:42:08,146][20347] Updated weights for policy 0, policy_version 587681 (0.0010) [2023-12-27 09:42:08,199][20347] Updated weights for policy 0, policy_version 587691 (0.0009) [2023-12-27 09:42:08,260][20347] Updated weights for policy 0, policy_version 587701 (0.0010) [2023-12-27 09:42:08,320][20347] Updated weights for policy 0, policy_version 587712 (0.0006) [2023-12-27 09:42:08,504][20381] Updated weights for policy 1, policy_version 587417 (0.0008) [2023-12-27 09:42:08,560][20381] Updated weights for policy 1, policy_version 587427 (0.0008) [2023-12-27 09:42:08,618][20381] Updated weights for policy 1, policy_version 587437 (0.0009) [2023-12-27 09:42:08,930][20347] Updated weights for policy 0, policy_version 587722 (0.0007) [2023-12-27 09:42:08,982][20347] Updated weights for policy 0, policy_version 587732 (0.0007) [2023-12-27 09:42:09,042][20347] Updated weights for policy 0, policy_version 587742 (0.0008) [2023-12-27 09:42:09,372][20381] Updated weights for policy 1, policy_version 587447 (0.0008) [2023-12-27 09:42:09,439][20381] Updated weights for policy 1, policy_version 587457 (0.0009) [2023-12-27 09:42:09,502][20381] Updated weights for policy 1, policy_version 587467 (0.0008) [2023-12-27 09:42:09,779][20347] Updated weights for policy 0, policy_version 587752 (0.0007) [2023-12-27 09:42:09,844][20347] Updated weights for policy 0, policy_version 587762 (0.0006) [2023-12-27 09:42:09,909][20347] Updated weights for policy 0, policy_version 587772 (0.0006) [2023-12-27 09:42:10,252][20381] Updated weights for policy 1, policy_version 587477 (0.0008) [2023-12-27 09:42:10,310][20381] Updated weights for policy 1, policy_version 587487 (0.0007) [2023-12-27 09:42:10,371][20381] Updated weights for policy 1, policy_version 587497 (0.0008) [2023-12-27 09:42:10,579][20347] Updated weights for policy 0, policy_version 587782 (0.0006) [2023-12-27 09:42:10,629][20347] Updated weights for policy 0, policy_version 587792 (0.0005) [2023-12-27 09:42:10,676][20347] Updated weights for policy 0, policy_version 587802 (0.0005) [2023-12-27 09:42:11,118][20381] Updated weights for policy 1, policy_version 587507 (0.0008) [2023-12-27 09:42:11,185][20381] Updated weights for policy 1, policy_version 587517 (0.0007) [2023-12-27 09:42:11,246][20381] Updated weights for policy 1, policy_version 587527 (0.0008) [2023-12-27 09:42:11,336][20347] Updated weights for policy 0, policy_version 587812 (0.0006) [2023-12-27 09:42:11,400][20347] Updated weights for policy 0, policy_version 587822 (0.0008) [2023-12-27 09:42:11,465][20347] Updated weights for policy 0, policy_version 587832 (0.0008) [2023-12-27 09:42:11,971][20381] Updated weights for policy 1, policy_version 587537 (0.0008) [2023-12-27 09:42:12,036][20381] Updated weights for policy 1, policy_version 587547 (0.0008) [2023-12-27 09:42:12,098][20381] Updated weights for policy 1, policy_version 587557 (0.0008) [2023-12-27 09:42:12,154][20347] Updated weights for policy 0, policy_version 587842 (0.0007) [2023-12-27 09:42:12,160][20381] Updated weights for policy 1, policy_version 587567 (0.0010) [2023-12-27 09:42:12,220][20347] Updated weights for policy 0, policy_version 587852 (0.0007) [2023-12-27 09:42:12,280][20347] Updated weights for policy 0, policy_version 587862 (0.0007) [2023-12-27 09:42:12,341][20347] Updated weights for policy 0, policy_version 587872 (0.0007) [2023-12-27 09:42:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 300949504. Throughput: 0: 10472.3, 1: 10308.2. Samples: 300935272. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:12,363][19170] Avg episode reward: [(0, '980.840'), (1, '465.710')] [2023-12-27 09:42:12,842][20381] Updated weights for policy 1, policy_version 587577 (0.0007) [2023-12-27 09:42:12,906][20381] Updated weights for policy 1, policy_version 587587 (0.0008) [2023-12-27 09:42:12,966][20381] Updated weights for policy 1, policy_version 587597 (0.0008) [2023-12-27 09:42:13,034][20347] Updated weights for policy 0, policy_version 587882 (0.0008) [2023-12-27 09:42:13,088][20347] Updated weights for policy 0, policy_version 587892 (0.0009) [2023-12-27 09:42:13,137][20347] Updated weights for policy 0, policy_version 587902 (0.0008) [2023-12-27 09:42:13,585][20381] Updated weights for policy 1, policy_version 587607 (0.0007) [2023-12-27 09:42:13,634][20381] Updated weights for policy 1, policy_version 587617 (0.0007) [2023-12-27 09:42:13,691][20381] Updated weights for policy 1, policy_version 587627 (0.0008) [2023-12-27 09:42:13,808][20347] Updated weights for policy 0, policy_version 587912 (0.0008) [2023-12-27 09:42:13,867][20347] Updated weights for policy 0, policy_version 587922 (0.0007) [2023-12-27 09:42:13,922][20347] Updated weights for policy 0, policy_version 587932 (0.0008) [2023-12-27 09:42:14,413][20381] Updated weights for policy 1, policy_version 587637 (0.0007) [2023-12-27 09:42:14,470][20381] Updated weights for policy 1, policy_version 587647 (0.0007) [2023-12-27 09:42:14,534][20381] Updated weights for policy 1, policy_version 587657 (0.0008) [2023-12-27 09:42:14,559][20347] Updated weights for policy 0, policy_version 587942 (0.0007) [2023-12-27 09:42:14,622][20347] Updated weights for policy 0, policy_version 587952 (0.0006) [2023-12-27 09:42:14,680][20347] Updated weights for policy 0, policy_version 587962 (0.0007) [2023-12-27 09:42:15,219][20381] Updated weights for policy 1, policy_version 587667 (0.0008) [2023-12-27 09:42:15,277][20381] Updated weights for policy 1, policy_version 587677 (0.0008) [2023-12-27 09:42:15,307][20347] Updated weights for policy 0, policy_version 587972 (0.0007) [2023-12-27 09:42:15,339][20381] Updated weights for policy 1, policy_version 587687 (0.0007) [2023-12-27 09:42:15,361][20347] Updated weights for policy 0, policy_version 587982 (0.0007) [2023-12-27 09:42:15,413][20347] Updated weights for policy 0, policy_version 587992 (0.0006) [2023-12-27 09:42:15,961][20347] Updated weights for policy 0, policy_version 588002 (0.0006) [2023-12-27 09:42:16,017][20347] Updated weights for policy 0, policy_version 588012 (0.0005) [2023-12-27 09:42:16,070][20347] Updated weights for policy 0, policy_version 588022 (0.0005) [2023-12-27 09:42:16,088][20381] Updated weights for policy 1, policy_version 587697 (0.0007) [2023-12-27 09:42:16,121][20347] Updated weights for policy 0, policy_version 588032 (0.0005) [2023-12-27 09:42:16,137][20381] Updated weights for policy 1, policy_version 587707 (0.0008) [2023-12-27 09:42:16,194][20381] Updated weights for policy 1, policy_version 587717 (0.0008) [2023-12-27 09:42:16,248][20381] Updated weights for policy 1, policy_version 587727 (0.0008) [2023-12-27 09:42:16,718][20347] Updated weights for policy 0, policy_version 588042 (0.0005) [2023-12-27 09:42:16,775][20347] Updated weights for policy 0, policy_version 588052 (0.0005) [2023-12-27 09:42:16,836][20347] Updated weights for policy 0, policy_version 588062 (0.0005) [2023-12-27 09:42:16,980][20381] Updated weights for policy 1, policy_version 587737 (0.0006) [2023-12-27 09:42:17,028][20381] Updated weights for policy 1, policy_version 587747 (0.0006) [2023-12-27 09:42:17,040][20292] KL-divergence is very high: 104.3651 [2023-12-27 09:42:17,078][20381] Updated weights for policy 1, policy_version 587757 (0.0006) [2023-12-27 09:42:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 301047808. Throughput: 0: 10519.1, 1: 10163.0. Samples: 301059608. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:17,363][19170] Avg episode reward: [(0, '990.400'), (1, '289.740')] [2023-12-27 09:42:17,439][20347] Updated weights for policy 0, policy_version 588072 (0.0007) [2023-12-27 09:42:17,484][20347] Updated weights for policy 0, policy_version 588082 (0.0007) [2023-12-27 09:42:17,537][20347] Updated weights for policy 0, policy_version 588092 (0.0007) [2023-12-27 09:42:17,707][20292] KL-divergence is very high: 107.8741 [2023-12-27 09:42:17,722][20381] Updated weights for policy 1, policy_version 587767 (0.0008) [2023-12-27 09:42:17,778][20381] Updated weights for policy 1, policy_version 587777 (0.0007) [2023-12-27 09:42:17,779][20292] KL-divergence is very high: 116.9701 [2023-12-27 09:42:17,831][20381] Updated weights for policy 1, policy_version 587787 (0.0005) [2023-12-27 09:42:18,212][20347] Updated weights for policy 0, policy_version 588102 (0.0007) [2023-12-27 09:42:18,266][20347] Updated weights for policy 0, policy_version 588112 (0.0005) [2023-12-27 09:42:18,318][20347] Updated weights for policy 0, policy_version 588122 (0.0006) [2023-12-27 09:42:18,529][20381] Updated weights for policy 1, policy_version 587797 (0.0007) [2023-12-27 09:42:18,596][20381] Updated weights for policy 1, policy_version 587807 (0.0008) [2023-12-27 09:42:18,658][20381] Updated weights for policy 1, policy_version 587817 (0.0009) [2023-12-27 09:42:18,919][20347] Updated weights for policy 0, policy_version 588132 (0.0006) [2023-12-27 09:42:18,971][20347] Updated weights for policy 0, policy_version 588142 (0.0007) [2023-12-27 09:42:19,025][20347] Updated weights for policy 0, policy_version 588152 (0.0008) [2023-12-27 09:42:19,363][20381] Updated weights for policy 1, policy_version 587827 (0.0007) [2023-12-27 09:42:19,422][20381] Updated weights for policy 1, policy_version 587837 (0.0008) [2023-12-27 09:42:19,476][20381] Updated weights for policy 1, policy_version 587847 (0.0007) [2023-12-27 09:42:19,755][20347] Updated weights for policy 0, policy_version 588162 (0.0007) [2023-12-27 09:42:19,808][20347] Updated weights for policy 0, policy_version 588172 (0.0007) [2023-12-27 09:42:19,865][20347] Updated weights for policy 0, policy_version 588182 (0.0008) [2023-12-27 09:42:19,922][20347] Updated weights for policy 0, policy_version 588192 (0.0007) [2023-12-27 09:42:20,205][20381] Updated weights for policy 1, policy_version 587857 (0.0007) [2023-12-27 09:42:20,266][20381] Updated weights for policy 1, policy_version 587867 (0.0008) [2023-12-27 09:42:20,326][20381] Updated weights for policy 1, policy_version 587877 (0.0006) [2023-12-27 09:42:20,386][20381] Updated weights for policy 1, policy_version 587887 (0.0008) [2023-12-27 09:42:20,590][20347] Updated weights for policy 0, policy_version 588202 (0.0007) [2023-12-27 09:42:20,649][20347] Updated weights for policy 0, policy_version 588212 (0.0007) [2023-12-27 09:42:20,705][20347] Updated weights for policy 0, policy_version 588222 (0.0008) [2023-12-27 09:42:21,104][20381] Updated weights for policy 1, policy_version 587897 (0.0008) [2023-12-27 09:42:21,167][20381] Updated weights for policy 1, policy_version 587907 (0.0009) [2023-12-27 09:42:21,231][20381] Updated weights for policy 1, policy_version 587917 (0.0006) [2023-12-27 09:42:21,401][20347] Updated weights for policy 0, policy_version 588232 (0.0008) [2023-12-27 09:42:21,459][20347] Updated weights for policy 0, policy_version 588242 (0.0009) [2023-12-27 09:42:21,517][20347] Updated weights for policy 0, policy_version 588252 (0.0007) [2023-12-27 09:42:21,949][20381] Updated weights for policy 1, policy_version 587927 (0.0007) [2023-12-27 09:42:22,009][20381] Updated weights for policy 1, policy_version 587937 (0.0007) [2023-12-27 09:42:22,070][20381] Updated weights for policy 1, policy_version 587947 (0.0008) [2023-12-27 09:42:22,244][20347] Updated weights for policy 0, policy_version 588262 (0.0008) [2023-12-27 09:42:22,307][20347] Updated weights for policy 0, policy_version 588272 (0.0008) [2023-12-27 09:42:22,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 301146112. Throughput: 0: 10507.2, 1: 10126.8. Samples: 301120408. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:22,363][19170] Avg episode reward: [(0, '990.310'), (1, '392.250')] [2023-12-27 09:42:22,367][20347] Updated weights for policy 0, policy_version 588282 (0.0008) [2023-12-27 09:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000587952_150536192.pth... [2023-12-27 09:42:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000586768_150233088.pth [2023-12-27 09:42:22,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000588288_150618112.pth... [2023-12-27 09:42:22,408][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000587040_150298624.pth [2023-12-27 09:42:22,809][20381] Updated weights for policy 1, policy_version 587957 (0.0008) [2023-12-27 09:42:22,870][20381] Updated weights for policy 1, policy_version 587967 (0.0007) [2023-12-27 09:42:22,921][20381] Updated weights for policy 1, policy_version 587977 (0.0005) [2023-12-27 09:42:23,075][20347] Updated weights for policy 0, policy_version 588292 (0.0008) [2023-12-27 09:42:23,123][20347] Updated weights for policy 0, policy_version 588302 (0.0007) [2023-12-27 09:42:23,178][20347] Updated weights for policy 0, policy_version 588312 (0.0007) [2023-12-27 09:42:23,577][20381] Updated weights for policy 1, policy_version 587987 (0.0006) [2023-12-27 09:42:23,640][20381] Updated weights for policy 1, policy_version 587997 (0.0007) [2023-12-27 09:42:23,693][20381] Updated weights for policy 1, policy_version 588007 (0.0008) [2023-12-27 09:42:23,846][20347] Updated weights for policy 0, policy_version 588322 (0.0008) [2023-12-27 09:42:23,889][20347] Updated weights for policy 0, policy_version 588332 (0.0007) [2023-12-27 09:42:23,940][20347] Updated weights for policy 0, policy_version 588342 (0.0005) [2023-12-27 09:42:23,991][20347] Updated weights for policy 0, policy_version 588352 (0.0006) [2023-12-27 09:42:24,412][20381] Updated weights for policy 1, policy_version 588017 (0.0008) [2023-12-27 09:42:24,466][20381] Updated weights for policy 1, policy_version 588027 (0.0008) [2023-12-27 09:42:24,515][20381] Updated weights for policy 1, policy_version 588037 (0.0008) [2023-12-27 09:42:24,581][20381] Updated weights for policy 1, policy_version 588047 (0.0008) [2023-12-27 09:42:24,603][20347] Updated weights for policy 0, policy_version 588362 (0.0007) [2023-12-27 09:42:24,656][20347] Updated weights for policy 0, policy_version 588372 (0.0007) [2023-12-27 09:42:24,697][20291] KL-divergence is very high: 250.7085 [2023-12-27 09:42:24,708][20347] Updated weights for policy 0, policy_version 588382 (0.0007) [2023-12-27 09:42:25,250][20381] Updated weights for policy 1, policy_version 588057 (0.0007) [2023-12-27 09:42:25,305][20381] Updated weights for policy 1, policy_version 588067 (0.0008) [2023-12-27 09:42:25,353][20381] Updated weights for policy 1, policy_version 588077 (0.0008) [2023-12-27 09:42:25,412][20347] Updated weights for policy 0, policy_version 588392 (0.0007) [2023-12-27 09:42:25,466][20347] Updated weights for policy 0, policy_version 588402 (0.0007) [2023-12-27 09:42:25,519][20347] Updated weights for policy 0, policy_version 588412 (0.0008) [2023-12-27 09:42:26,008][20381] Updated weights for policy 1, policy_version 588087 (0.0007) [2023-12-27 09:42:26,056][20381] Updated weights for policy 1, policy_version 588097 (0.0007) [2023-12-27 09:42:26,104][20381] Updated weights for policy 1, policy_version 588107 (0.0007) [2023-12-27 09:42:26,215][20347] Updated weights for policy 0, policy_version 588422 (0.0008) [2023-12-27 09:42:26,272][20347] Updated weights for policy 0, policy_version 588432 (0.0008) [2023-12-27 09:42:26,327][20347] Updated weights for policy 0, policy_version 588442 (0.0007) [2023-12-27 09:42:26,792][20381] Updated weights for policy 1, policy_version 588117 (0.0007) [2023-12-27 09:42:26,854][20381] Updated weights for policy 1, policy_version 588127 (0.0008) [2023-12-27 09:42:26,899][20381] Updated weights for policy 1, policy_version 588137 (0.0008) [2023-12-27 09:42:27,012][20347] Updated weights for policy 0, policy_version 588452 (0.0007) [2023-12-27 09:42:27,069][20347] Updated weights for policy 0, policy_version 588462 (0.0007) [2023-12-27 09:42:27,134][20347] Updated weights for policy 0, policy_version 588472 (0.0007) [2023-12-27 09:42:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 301252608. Throughput: 0: 10480.0, 1: 10108.6. Samples: 301241192. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:27,363][19170] Avg episode reward: [(0, '961.330'), (1, '213.740')] [2023-12-27 09:42:27,593][20381] Updated weights for policy 1, policy_version 588147 (0.0008) [2023-12-27 09:42:27,635][20381] Updated weights for policy 1, policy_version 588157 (0.0008) [2023-12-27 09:42:27,683][20381] Updated weights for policy 1, policy_version 588167 (0.0007) [2023-12-27 09:42:27,809][20347] Updated weights for policy 0, policy_version 588482 (0.0007) [2023-12-27 09:42:27,867][20347] Updated weights for policy 0, policy_version 588492 (0.0007) [2023-12-27 09:42:27,907][20347] Updated weights for policy 0, policy_version 588502 (0.0007) [2023-12-27 09:42:27,954][20347] Updated weights for policy 0, policy_version 588512 (0.0007) [2023-12-27 09:42:28,390][20381] Updated weights for policy 1, policy_version 588177 (0.0008) [2023-12-27 09:42:28,450][20381] Updated weights for policy 1, policy_version 588187 (0.0008) [2023-12-27 09:42:28,496][20381] Updated weights for policy 1, policy_version 588197 (0.0008) [2023-12-27 09:42:28,545][20381] Updated weights for policy 1, policy_version 588207 (0.0007) [2023-12-27 09:42:28,615][20347] Updated weights for policy 0, policy_version 588522 (0.0008) [2023-12-27 09:42:28,671][20347] Updated weights for policy 0, policy_version 588532 (0.0008) [2023-12-27 09:42:28,717][20347] Updated weights for policy 0, policy_version 588542 (0.0007) [2023-12-27 09:42:29,240][20381] Updated weights for policy 1, policy_version 588217 (0.0008) [2023-12-27 09:42:29,299][20381] Updated weights for policy 1, policy_version 588227 (0.0008) [2023-12-27 09:42:29,359][20381] Updated weights for policy 1, policy_version 588237 (0.0007) [2023-12-27 09:42:29,441][20347] Updated weights for policy 0, policy_version 588552 (0.0008) [2023-12-27 09:42:29,467][20291] KL-divergence is very high: 135.2346 [2023-12-27 09:42:29,486][20347] Updated weights for policy 0, policy_version 588562 (0.0007) [2023-12-27 09:42:29,534][20347] Updated weights for policy 0, policy_version 588572 (0.0008) [2023-12-27 09:42:30,040][20381] Updated weights for policy 1, policy_version 588247 (0.0006) [2023-12-27 09:42:30,095][20381] Updated weights for policy 1, policy_version 588257 (0.0006) [2023-12-27 09:42:30,153][20381] Updated weights for policy 1, policy_version 588267 (0.0008) [2023-12-27 09:42:30,267][20347] Updated weights for policy 0, policy_version 588582 (0.0008) [2023-12-27 09:42:30,323][20347] Updated weights for policy 0, policy_version 588592 (0.0008) [2023-12-27 09:42:30,378][20347] Updated weights for policy 0, policy_version 588602 (0.0008) [2023-12-27 09:42:30,816][20381] Updated weights for policy 1, policy_version 588277 (0.0008) [2023-12-27 09:42:30,873][20381] Updated weights for policy 1, policy_version 588287 (0.0006) [2023-12-27 09:42:30,927][20381] Updated weights for policy 1, policy_version 588297 (0.0008) [2023-12-27 09:42:31,072][20347] Updated weights for policy 0, policy_version 588612 (0.0008) [2023-12-27 09:42:31,120][20347] Updated weights for policy 0, policy_version 588622 (0.0007) [2023-12-27 09:42:31,187][20347] Updated weights for policy 0, policy_version 588632 (0.0008) [2023-12-27 09:42:31,626][20381] Updated weights for policy 1, policy_version 588307 (0.0008) [2023-12-27 09:42:31,690][20381] Updated weights for policy 1, policy_version 588317 (0.0007) [2023-12-27 09:42:31,751][20381] Updated weights for policy 1, policy_version 588327 (0.0007) [2023-12-27 09:42:31,877][20347] Updated weights for policy 0, policy_version 588642 (0.0008) [2023-12-27 09:42:31,934][20347] Updated weights for policy 0, policy_version 588652 (0.0007) [2023-12-27 09:42:31,988][20347] Updated weights for policy 0, policy_version 588662 (0.0006) [2023-12-27 09:42:32,041][20347] Updated weights for policy 0, policy_version 588672 (0.0006) [2023-12-27 09:42:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 301350912. Throughput: 0: 10462.6, 1: 10089.3. Samples: 301362472. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:32,363][19170] Avg episode reward: [(0, '894.300'), (1, '286.450')] [2023-12-27 09:42:32,521][20381] Updated weights for policy 1, policy_version 588337 (0.0010) [2023-12-27 09:42:32,576][20347] Updated weights for policy 0, policy_version 588682 (0.0006) [2023-12-27 09:42:32,577][20381] Updated weights for policy 1, policy_version 588347 (0.0009) [2023-12-27 09:42:32,625][20347] Updated weights for policy 0, policy_version 588692 (0.0005) [2023-12-27 09:42:32,633][20381] Updated weights for policy 1, policy_version 588357 (0.0008) [2023-12-27 09:42:32,681][20347] Updated weights for policy 0, policy_version 588702 (0.0006) [2023-12-27 09:42:32,690][20381] Updated weights for policy 1, policy_version 588367 (0.0007) [2023-12-27 09:42:33,231][20347] Updated weights for policy 0, policy_version 588712 (0.0005) [2023-12-27 09:42:33,291][20347] Updated weights for policy 0, policy_version 588722 (0.0005) [2023-12-27 09:42:33,343][20347] Updated weights for policy 0, policy_version 588732 (0.0005) [2023-12-27 09:42:33,442][20381] Updated weights for policy 1, policy_version 588377 (0.0009) [2023-12-27 09:42:33,484][20381] Updated weights for policy 1, policy_version 588387 (0.0006) [2023-12-27 09:42:33,532][20381] Updated weights for policy 1, policy_version 588397 (0.0007) [2023-12-27 09:42:33,865][20347] Updated weights for policy 0, policy_version 588742 (0.0008) [2023-12-27 09:42:33,919][20347] Updated weights for policy 0, policy_version 588752 (0.0009) [2023-12-27 09:42:33,981][20347] Updated weights for policy 0, policy_version 588762 (0.0007) [2023-12-27 09:42:34,149][20381] Updated weights for policy 1, policy_version 588407 (0.0006) [2023-12-27 09:42:34,212][20381] Updated weights for policy 1, policy_version 588417 (0.0007) [2023-12-27 09:42:34,271][20381] Updated weights for policy 1, policy_version 588427 (0.0006) [2023-12-27 09:42:34,588][20347] Updated weights for policy 0, policy_version 588772 (0.0008) [2023-12-27 09:42:34,640][20347] Updated weights for policy 0, policy_version 588782 (0.0007) [2023-12-27 09:42:34,694][20347] Updated weights for policy 0, policy_version 588792 (0.0011) [2023-12-27 09:42:34,816][20381] Updated weights for policy 1, policy_version 588437 (0.0005) [2023-12-27 09:42:34,873][20381] Updated weights for policy 1, policy_version 588447 (0.0005) [2023-12-27 09:42:34,935][20381] Updated weights for policy 1, policy_version 588457 (0.0008) [2023-12-27 09:42:35,333][20347] Updated weights for policy 0, policy_version 588802 (0.0010) [2023-12-27 09:42:35,379][20347] Updated weights for policy 0, policy_version 588812 (0.0008) [2023-12-27 09:42:35,439][20347] Updated weights for policy 0, policy_version 588822 (0.0006) [2023-12-27 09:42:35,497][20347] Updated weights for policy 0, policy_version 588832 (0.0011) [2023-12-27 09:42:35,581][20381] Updated weights for policy 1, policy_version 588467 (0.0008) [2023-12-27 09:42:35,640][20381] Updated weights for policy 1, policy_version 588477 (0.0008) [2023-12-27 09:42:35,698][20381] Updated weights for policy 1, policy_version 588487 (0.0007) [2023-12-27 09:42:36,166][20347] Updated weights for policy 0, policy_version 588842 (0.0010) [2023-12-27 09:42:36,187][20291] KL-divergence is very high: 159.1931 [2023-12-27 09:42:36,223][20347] Updated weights for policy 0, policy_version 588852 (0.0010) [2023-12-27 09:42:36,235][20291] KL-divergence is very high: 137.9669 [2023-12-27 09:42:36,283][20347] Updated weights for policy 0, policy_version 588862 (0.0011) [2023-12-27 09:42:36,289][20291] KL-divergence is very high: 105.4281 [2023-12-27 09:42:36,343][20381] Updated weights for policy 1, policy_version 588497 (0.0008) [2023-12-27 09:42:36,394][20381] Updated weights for policy 1, policy_version 588507 (0.0008) [2023-12-27 09:42:36,442][20381] Updated weights for policy 1, policy_version 588517 (0.0008) [2023-12-27 09:42:36,494][20381] Updated weights for policy 1, policy_version 588527 (0.0008) [2023-12-27 09:42:36,979][20347] Updated weights for policy 0, policy_version 588872 (0.0008) [2023-12-27 09:42:36,990][20291] KL-divergence is very high: 193.5171 [2023-12-27 09:42:37,027][20347] Updated weights for policy 0, policy_version 588882 (0.0007) [2023-12-27 09:42:37,041][20291] KL-divergence is very high: 121.2911 [2023-12-27 09:42:37,048][20291] KL-divergence is very high: 101.2402 [2023-12-27 09:42:37,086][20347] Updated weights for policy 0, policy_version 588892 (0.0008) [2023-12-27 09:42:37,194][20381] Updated weights for policy 1, policy_version 588537 (0.0008) [2023-12-27 09:42:37,257][20381] Updated weights for policy 1, policy_version 588547 (0.0008) [2023-12-27 09:42:37,319][20381] Updated weights for policy 1, policy_version 588557 (0.0007) [2023-12-27 09:42:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 301465600. Throughput: 0: 10580.1, 1: 10060.9. Samples: 301430324. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:37,363][19170] Avg episode reward: [(0, '830.970'), (1, '371.740')] [2023-12-27 09:42:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000588560_150691840.pth... [2023-12-27 09:42:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000588896_150773760.pth... [2023-12-27 09:42:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000587376_150388736.pth [2023-12-27 09:42:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000587648_150454272.pth [2023-12-27 09:42:37,693][20347] Updated weights for policy 0, policy_version 588902 (0.0006) [2023-12-27 09:42:37,752][20347] Updated weights for policy 0, policy_version 588912 (0.0005) [2023-12-27 09:42:37,803][20291] KL-divergence is very high: 111.8516 [2023-12-27 09:42:37,815][20347] Updated weights for policy 0, policy_version 588922 (0.0005) [2023-12-27 09:42:38,063][20381] Updated weights for policy 1, policy_version 588567 (0.0008) [2023-12-27 09:42:38,121][20381] Updated weights for policy 1, policy_version 588577 (0.0006) [2023-12-27 09:42:38,178][20381] Updated weights for policy 1, policy_version 588587 (0.0007) [2023-12-27 09:42:38,404][20347] Updated weights for policy 0, policy_version 588932 (0.0006) [2023-12-27 09:42:38,456][20347] Updated weights for policy 0, policy_version 588942 (0.0005) [2023-12-27 09:42:38,511][20347] Updated weights for policy 0, policy_version 588952 (0.0005) [2023-12-27 09:42:38,827][20381] Updated weights for policy 1, policy_version 588597 (0.0007) [2023-12-27 09:42:38,880][20381] Updated weights for policy 1, policy_version 588607 (0.0007) [2023-12-27 09:42:38,934][20381] Updated weights for policy 1, policy_version 588617 (0.0008) [2023-12-27 09:42:39,124][20347] Updated weights for policy 0, policy_version 588962 (0.0008) [2023-12-27 09:42:39,144][20291] KL-divergence is very high: 105.3125 [2023-12-27 09:42:39,174][20347] Updated weights for policy 0, policy_version 588972 (0.0010) [2023-12-27 09:42:39,181][20291] KL-divergence is very high: 137.8858 [2023-12-27 09:42:39,223][20347] Updated weights for policy 0, policy_version 588982 (0.0010) [2023-12-27 09:42:39,228][20291] KL-divergence is very high: 137.7421 [2023-12-27 09:42:39,279][20347] Updated weights for policy 0, policy_version 588992 (0.0008) [2023-12-27 09:42:39,607][20381] Updated weights for policy 1, policy_version 588627 (0.0007) [2023-12-27 09:42:39,670][20381] Updated weights for policy 1, policy_version 588637 (0.0008) [2023-12-27 09:42:39,734][20381] Updated weights for policy 1, policy_version 588647 (0.0007) [2023-12-27 09:42:39,928][20291] KL-divergence is very high: 143.8924 [2023-12-27 09:42:39,946][20291] KL-divergence is very high: 166.8956 [2023-12-27 09:42:39,968][20291] KL-divergence is very high: 150.9550 [2023-12-27 09:42:39,981][20347] Updated weights for policy 0, policy_version 589002 (0.0008) [2023-12-27 09:42:40,016][20291] KL-divergence is very high: 178.0650 [2023-12-27 09:42:40,029][20291] KL-divergence is very high: 189.3969 [2023-12-27 09:42:40,040][20347] Updated weights for policy 0, policy_version 589012 (0.0008) [2023-12-27 09:42:40,043][20291] KL-divergence is very high: 149.3548 [2023-12-27 09:42:40,070][20291] KL-divergence is very high: 150.5038 [2023-12-27 09:42:40,077][20291] KL-divergence is very high: 125.8092 [2023-12-27 09:42:40,110][20347] Updated weights for policy 0, policy_version 589022 (0.0008) [2023-12-27 09:42:40,427][20381] Updated weights for policy 1, policy_version 588657 (0.0007) [2023-12-27 09:42:40,480][20381] Updated weights for policy 1, policy_version 588667 (0.0008) [2023-12-27 09:42:40,524][20381] Updated weights for policy 1, policy_version 588677 (0.0007) [2023-12-27 09:42:40,573][20381] Updated weights for policy 1, policy_version 588687 (0.0005) [2023-12-27 09:42:40,790][20347] Updated weights for policy 0, policy_version 589032 (0.0008) [2023-12-27 09:42:40,845][20347] Updated weights for policy 0, policy_version 589042 (0.0007) [2023-12-27 09:42:40,913][20347] Updated weights for policy 0, policy_version 589052 (0.0007) [2023-12-27 09:42:41,214][20381] Updated weights for policy 1, policy_version 588697 (0.0009) [2023-12-27 09:42:41,277][20381] Updated weights for policy 1, policy_version 588707 (0.0008) [2023-12-27 09:42:41,332][20381] Updated weights for policy 1, policy_version 588717 (0.0008) [2023-12-27 09:42:41,599][20347] Updated weights for policy 0, policy_version 589062 (0.0007) [2023-12-27 09:42:41,667][20347] Updated weights for policy 0, policy_version 589072 (0.0006) [2023-12-27 09:42:41,726][20347] Updated weights for policy 0, policy_version 589082 (0.0008) [2023-12-27 09:42:42,048][20381] Updated weights for policy 1, policy_version 588727 (0.0010) [2023-12-27 09:42:42,103][20381] Updated weights for policy 1, policy_version 588737 (0.0006) [2023-12-27 09:42:42,160][20381] Updated weights for policy 1, policy_version 588747 (0.0006) [2023-12-27 09:42:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.3, 300 sec: 20771.6). Total num frames: 301563904. Throughput: 0: 10548.3, 1: 10092.7. Samples: 301554496. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:42,363][19170] Avg episode reward: [(0, '595.660'), (1, '629.820')] [2023-12-27 09:42:42,380][20347] Updated weights for policy 0, policy_version 589092 (0.0008) [2023-12-27 09:42:42,432][20347] Updated weights for policy 0, policy_version 589102 (0.0008) [2023-12-27 09:42:42,480][20347] Updated weights for policy 0, policy_version 589112 (0.0008) [2023-12-27 09:42:42,788][20381] Updated weights for policy 1, policy_version 588757 (0.0009) [2023-12-27 09:42:42,847][20381] Updated weights for policy 1, policy_version 588767 (0.0008) [2023-12-27 09:42:42,898][20381] Updated weights for policy 1, policy_version 588777 (0.0008) [2023-12-27 09:42:43,195][20347] Updated weights for policy 0, policy_version 589122 (0.0008) [2023-12-27 09:42:43,243][20347] Updated weights for policy 0, policy_version 589132 (0.0010) [2023-12-27 09:42:43,298][20347] Updated weights for policy 0, policy_version 589142 (0.0010) [2023-12-27 09:42:43,352][20347] Updated weights for policy 0, policy_version 589152 (0.0010) [2023-12-27 09:42:43,534][20381] Updated weights for policy 1, policy_version 588787 (0.0007) [2023-12-27 09:42:43,586][20381] Updated weights for policy 1, policy_version 588797 (0.0007) [2023-12-27 09:42:43,646][20381] Updated weights for policy 1, policy_version 588807 (0.0007) [2023-12-27 09:42:44,038][20291] KL-divergence is very high: 101.8829 [2023-12-27 09:42:44,039][20347] Updated weights for policy 0, policy_version 589162 (0.0009) [2023-12-27 09:42:44,052][20291] KL-divergence is very high: 149.6162 [2023-12-27 09:42:44,057][20291] KL-divergence is very high: 123.9266 [2023-12-27 09:42:44,085][20347] Updated weights for policy 0, policy_version 589172 (0.0010) [2023-12-27 09:42:44,124][20291] KL-divergence is very high: 130.5768 [2023-12-27 09:42:44,129][20291] KL-divergence is very high: 116.1915 [2023-12-27 09:42:44,130][20347] Updated weights for policy 0, policy_version 589182 (0.0010) [2023-12-27 09:42:44,260][20381] Updated weights for policy 1, policy_version 588817 (0.0006) [2023-12-27 09:42:44,315][20381] Updated weights for policy 1, policy_version 588827 (0.0008) [2023-12-27 09:42:44,370][20381] Updated weights for policy 1, policy_version 588837 (0.0008) [2023-12-27 09:42:44,421][20381] Updated weights for policy 1, policy_version 588847 (0.0009) [2023-12-27 09:42:44,821][20347] Updated weights for policy 0, policy_version 589192 (0.0010) [2023-12-27 09:42:44,889][20347] Updated weights for policy 0, policy_version 589202 (0.0010) [2023-12-27 09:42:44,955][20347] Updated weights for policy 0, policy_version 589212 (0.0010) [2023-12-27 09:42:45,091][20381] Updated weights for policy 1, policy_version 588857 (0.0006) [2023-12-27 09:42:45,157][20381] Updated weights for policy 1, policy_version 588867 (0.0007) [2023-12-27 09:42:45,221][20381] Updated weights for policy 1, policy_version 588877 (0.0008) [2023-12-27 09:42:45,594][20347] Updated weights for policy 0, policy_version 589222 (0.0007) [2023-12-27 09:42:45,659][20347] Updated weights for policy 0, policy_version 589232 (0.0005) [2023-12-27 09:42:45,715][20347] Updated weights for policy 0, policy_version 589242 (0.0005) [2023-12-27 09:42:45,875][20381] Updated weights for policy 1, policy_version 588887 (0.0008) [2023-12-27 09:42:45,937][20381] Updated weights for policy 1, policy_version 588897 (0.0008) [2023-12-27 09:42:45,989][20381] Updated weights for policy 1, policy_version 588907 (0.0008) [2023-12-27 09:42:46,335][20347] Updated weights for policy 0, policy_version 589252 (0.0005) [2023-12-27 09:42:46,396][20347] Updated weights for policy 0, policy_version 589262 (0.0005) [2023-12-27 09:42:46,453][20347] Updated weights for policy 0, policy_version 589272 (0.0008) [2023-12-27 09:42:46,713][20381] Updated weights for policy 1, policy_version 588917 (0.0009) [2023-12-27 09:42:46,762][20381] Updated weights for policy 1, policy_version 588927 (0.0009) [2023-12-27 09:42:46,819][20381] Updated weights for policy 1, policy_version 588937 (0.0007) [2023-12-27 09:42:47,034][20347] Updated weights for policy 0, policy_version 589282 (0.0006) [2023-12-27 09:42:47,078][20347] Updated weights for policy 0, policy_version 589292 (0.0010) [2023-12-27 09:42:47,124][20347] Updated weights for policy 0, policy_version 589302 (0.0010) [2023-12-27 09:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 301670400. Throughput: 0: 10568.9, 1: 10120.6. Samples: 301679460. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:47,363][19170] Avg episode reward: [(0, '667.520'), (1, '385.760')] [2023-12-27 09:42:47,512][20381] Updated weights for policy 1, policy_version 588947 (0.0007) [2023-12-27 09:42:47,561][20381] Updated weights for policy 1, policy_version 588957 (0.0007) [2023-12-27 09:42:47,611][20381] Updated weights for policy 1, policy_version 588967 (0.0006) [2023-12-27 09:42:47,821][20347] Updated weights for policy 0, policy_version 589313 (0.0009) [2023-12-27 09:42:47,887][20347] Updated weights for policy 0, policy_version 589323 (0.0007) [2023-12-27 09:42:47,949][20347] Updated weights for policy 0, policy_version 589333 (0.0008) [2023-12-27 09:42:48,012][20347] Updated weights for policy 0, policy_version 589343 (0.0008) [2023-12-27 09:42:48,216][20381] Updated weights for policy 1, policy_version 588977 (0.0008) [2023-12-27 09:42:48,265][20381] Updated weights for policy 1, policy_version 588987 (0.0007) [2023-12-27 09:42:48,321][20381] Updated weights for policy 1, policy_version 588997 (0.0010) [2023-12-27 09:42:48,382][20381] Updated weights for policy 1, policy_version 589007 (0.0007) [2023-12-27 09:42:48,686][20347] Updated weights for policy 0, policy_version 589353 (0.0007) [2023-12-27 09:42:48,746][20347] Updated weights for policy 0, policy_version 589363 (0.0007) [2023-12-27 09:42:48,805][20347] Updated weights for policy 0, policy_version 589373 (0.0008) [2023-12-27 09:42:49,147][20381] Updated weights for policy 1, policy_version 589017 (0.0009) [2023-12-27 09:42:49,191][20381] Updated weights for policy 1, policy_version 589027 (0.0010) [2023-12-27 09:42:49,250][20381] Updated weights for policy 1, policy_version 589037 (0.0009) [2023-12-27 09:42:49,469][20347] Updated weights for policy 0, policy_version 589383 (0.0006) [2023-12-27 09:42:49,528][20347] Updated weights for policy 0, policy_version 589393 (0.0007) [2023-12-27 09:42:49,590][20347] Updated weights for policy 0, policy_version 589403 (0.0005) [2023-12-27 09:42:49,962][20381] Updated weights for policy 1, policy_version 589047 (0.0007) [2023-12-27 09:42:50,027][20381] Updated weights for policy 1, policy_version 589057 (0.0006) [2023-12-27 09:42:50,088][20381] Updated weights for policy 1, policy_version 589067 (0.0006) [2023-12-27 09:42:50,234][20347] Updated weights for policy 0, policy_version 589413 (0.0008) [2023-12-27 09:42:50,284][20347] Updated weights for policy 0, policy_version 589423 (0.0007) [2023-12-27 09:42:50,341][20347] Updated weights for policy 0, policy_version 589433 (0.0007) [2023-12-27 09:42:50,702][20381] Updated weights for policy 1, policy_version 589077 (0.0009) [2023-12-27 09:42:50,756][20381] Updated weights for policy 1, policy_version 589087 (0.0007) [2023-12-27 09:42:50,811][20381] Updated weights for policy 1, policy_version 589097 (0.0008) [2023-12-27 09:42:51,074][20347] Updated weights for policy 0, policy_version 589443 (0.0007) [2023-12-27 09:42:51,122][20347] Updated weights for policy 0, policy_version 589453 (0.0008) [2023-12-27 09:42:51,189][20347] Updated weights for policy 0, policy_version 589463 (0.0007) [2023-12-27 09:42:51,526][20381] Updated weights for policy 1, policy_version 589107 (0.0008) [2023-12-27 09:42:51,587][20381] Updated weights for policy 1, policy_version 589117 (0.0010) [2023-12-27 09:42:51,651][20381] Updated weights for policy 1, policy_version 589127 (0.0008) [2023-12-27 09:42:51,925][20347] Updated weights for policy 0, policy_version 589473 (0.0007) [2023-12-27 09:42:51,981][20347] Updated weights for policy 0, policy_version 589483 (0.0008) [2023-12-27 09:42:52,036][20347] Updated weights for policy 0, policy_version 589493 (0.0007) [2023-12-27 09:42:52,096][20347] Updated weights for policy 0, policy_version 589503 (0.0007) [2023-12-27 09:42:52,309][20381] Updated weights for policy 1, policy_version 589137 (0.0007) [2023-12-27 09:42:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 301768704. Throughput: 0: 10512.2, 1: 10088.6. Samples: 301740348. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:52,363][19170] Avg episode reward: [(0, '693.520'), (1, '529.720')] [2023-12-27 09:42:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000589504_150929408.pth... [2023-12-27 09:42:52,373][20381] Updated weights for policy 1, policy_version 589147 (0.0011) [2023-12-27 09:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000588288_150618112.pth [2023-12-27 09:42:52,436][20381] Updated weights for policy 1, policy_version 589157 (0.0011) [2023-12-27 09:42:52,496][20381] Updated weights for policy 1, policy_version 589167 (0.0011) [2023-12-27 09:42:52,500][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000589168_150847488.pth... [2023-12-27 09:42:52,504][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000587952_150536192.pth [2023-12-27 09:42:52,800][20347] Updated weights for policy 0, policy_version 589513 (0.0008) [2023-12-27 09:42:52,857][20347] Updated weights for policy 0, policy_version 589523 (0.0007) [2023-12-27 09:42:52,917][20347] Updated weights for policy 0, policy_version 589533 (0.0007) [2023-12-27 09:42:53,195][20381] Updated weights for policy 1, policy_version 589177 (0.0011) [2023-12-27 09:42:53,247][20381] Updated weights for policy 1, policy_version 589187 (0.0010) [2023-12-27 09:42:53,306][20381] Updated weights for policy 1, policy_version 589197 (0.0011) [2023-12-27 09:42:53,562][20347] Updated weights for policy 0, policy_version 589543 (0.0007) [2023-12-27 09:42:53,618][20347] Updated weights for policy 0, policy_version 589553 (0.0005) [2023-12-27 09:42:53,668][20347] Updated weights for policy 0, policy_version 589563 (0.0005) [2023-12-27 09:42:54,003][20381] Updated weights for policy 1, policy_version 589207 (0.0011) [2023-12-27 09:42:54,058][20381] Updated weights for policy 1, policy_version 589217 (0.0011) [2023-12-27 09:42:54,113][20381] Updated weights for policy 1, policy_version 589227 (0.0010) [2023-12-27 09:42:54,222][20347] Updated weights for policy 0, policy_version 589573 (0.0006) [2023-12-27 09:42:54,284][20347] Updated weights for policy 0, policy_version 589583 (0.0007) [2023-12-27 09:42:54,340][20347] Updated weights for policy 0, policy_version 589593 (0.0007) [2023-12-27 09:42:54,806][20381] Updated weights for policy 1, policy_version 589237 (0.0011) [2023-12-27 09:42:54,862][20381] Updated weights for policy 1, policy_version 589247 (0.0011) [2023-12-27 09:42:54,912][20347] Updated weights for policy 0, policy_version 589603 (0.0007) [2023-12-27 09:42:54,921][20381] Updated weights for policy 1, policy_version 589257 (0.0011) [2023-12-27 09:42:54,964][20347] Updated weights for policy 0, policy_version 589613 (0.0005) [2023-12-27 09:42:55,020][20347] Updated weights for policy 0, policy_version 589623 (0.0008) [2023-12-27 09:42:55,604][20381] Updated weights for policy 1, policy_version 589267 (0.0010) [2023-12-27 09:42:55,658][20381] Updated weights for policy 1, policy_version 589277 (0.0010) [2023-12-27 09:42:55,706][20381] Updated weights for policy 1, policy_version 589287 (0.0010) [2023-12-27 09:42:55,720][20347] Updated weights for policy 0, policy_version 589633 (0.0007) [2023-12-27 09:42:55,768][20347] Updated weights for policy 0, policy_version 589643 (0.0006) [2023-12-27 09:42:55,819][20347] Updated weights for policy 0, policy_version 589653 (0.0007) [2023-12-27 09:42:55,876][20347] Updated weights for policy 0, policy_version 589663 (0.0007) [2023-12-27 09:42:56,396][20381] Updated weights for policy 1, policy_version 589297 (0.0010) [2023-12-27 09:42:56,456][20381] Updated weights for policy 1, policy_version 589307 (0.0010) [2023-12-27 09:42:56,508][20381] Updated weights for policy 1, policy_version 589317 (0.0007) [2023-12-27 09:42:56,565][20381] Updated weights for policy 1, policy_version 589327 (0.0005) [2023-12-27 09:42:56,567][20347] Updated weights for policy 0, policy_version 589673 (0.0010) [2023-12-27 09:42:56,624][20347] Updated weights for policy 0, policy_version 589683 (0.0010) [2023-12-27 09:42:56,684][20347] Updated weights for policy 0, policy_version 589693 (0.0011) [2023-12-27 09:42:57,081][20381] Updated weights for policy 1, policy_version 589337 (0.0005) [2023-12-27 09:42:57,135][20381] Updated weights for policy 1, policy_version 589347 (0.0009) [2023-12-27 09:42:57,185][20381] Updated weights for policy 1, policy_version 589357 (0.0010) [2023-12-27 09:42:57,325][20347] Updated weights for policy 0, policy_version 589703 (0.0008) [2023-12-27 09:42:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 301875200. Throughput: 0: 10491.8, 1: 10133.4. Samples: 301863408. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:42:57,364][19170] Avg episode reward: [(0, '745.910'), (1, '258.730')] [2023-12-27 09:42:57,375][20347] Updated weights for policy 0, policy_version 589713 (0.0007) [2023-12-27 09:42:57,429][20347] Updated weights for policy 0, policy_version 589723 (0.0007) [2023-12-27 09:42:57,839][20381] Updated weights for policy 1, policy_version 589367 (0.0010) [2023-12-27 09:42:57,899][20381] Updated weights for policy 1, policy_version 589377 (0.0010) [2023-12-27 09:42:57,956][20381] Updated weights for policy 1, policy_version 589387 (0.0010) [2023-12-27 09:42:58,058][20347] Updated weights for policy 0, policy_version 589733 (0.0007) [2023-12-27 09:42:58,109][20347] Updated weights for policy 0, policy_version 589743 (0.0009) [2023-12-27 09:42:58,167][20347] Updated weights for policy 0, policy_version 589753 (0.0006) [2023-12-27 09:42:58,617][20381] Updated weights for policy 1, policy_version 589397 (0.0009) [2023-12-27 09:42:58,686][20381] Updated weights for policy 1, policy_version 589407 (0.0008) [2023-12-27 09:42:58,749][20381] Updated weights for policy 1, policy_version 589417 (0.0007) [2023-12-27 09:42:58,846][20347] Updated weights for policy 0, policy_version 589763 (0.0007) [2023-12-27 09:42:58,923][20347] Updated weights for policy 0, policy_version 589773 (0.0008) [2023-12-27 09:42:58,980][20347] Updated weights for policy 0, policy_version 589783 (0.0007) [2023-12-27 09:42:59,485][20381] Updated weights for policy 1, policy_version 589427 (0.0008) [2023-12-27 09:42:59,543][20381] Updated weights for policy 1, policy_version 589437 (0.0009) [2023-12-27 09:42:59,611][20381] Updated weights for policy 1, policy_version 589447 (0.0008) [2023-12-27 09:42:59,744][20347] Updated weights for policy 0, policy_version 589793 (0.0008) [2023-12-27 09:42:59,795][20347] Updated weights for policy 0, policy_version 589803 (0.0007) [2023-12-27 09:42:59,852][20347] Updated weights for policy 0, policy_version 589813 (0.0007) [2023-12-27 09:42:59,910][20347] Updated weights for policy 0, policy_version 589823 (0.0007) [2023-12-27 09:43:00,288][20381] Updated weights for policy 1, policy_version 589457 (0.0008) [2023-12-27 09:43:00,346][20381] Updated weights for policy 1, policy_version 589467 (0.0011) [2023-12-27 09:43:00,398][20381] Updated weights for policy 1, policy_version 589477 (0.0010) [2023-12-27 09:43:00,450][20381] Updated weights for policy 1, policy_version 589487 (0.0010) [2023-12-27 09:43:00,494][20347] Updated weights for policy 0, policy_version 589833 (0.0007) [2023-12-27 09:43:00,541][20347] Updated weights for policy 0, policy_version 589843 (0.0006) [2023-12-27 09:43:00,591][20347] Updated weights for policy 0, policy_version 589853 (0.0007) [2023-12-27 09:43:01,134][20381] Updated weights for policy 1, policy_version 589497 (0.0010) [2023-12-27 09:43:01,193][20381] Updated weights for policy 1, policy_version 589507 (0.0011) [2023-12-27 09:43:01,248][20381] Updated weights for policy 1, policy_version 589517 (0.0011) [2023-12-27 09:43:01,266][20347] Updated weights for policy 0, policy_version 589863 (0.0007) [2023-12-27 09:43:01,314][20347] Updated weights for policy 0, policy_version 589873 (0.0007) [2023-12-27 09:43:01,380][20347] Updated weights for policy 0, policy_version 589883 (0.0008) [2023-12-27 09:43:01,954][20381] Updated weights for policy 1, policy_version 589527 (0.0011) [2023-12-27 09:43:02,014][20381] Updated weights for policy 1, policy_version 589537 (0.0007) [2023-12-27 09:43:02,015][20347] Updated weights for policy 0, policy_version 589893 (0.0007) [2023-12-27 09:43:02,070][20347] Updated weights for policy 0, policy_version 589903 (0.0008) [2023-12-27 09:43:02,075][20381] Updated weights for policy 1, policy_version 589547 (0.0008) [2023-12-27 09:43:02,124][20347] Updated weights for policy 0, policy_version 589913 (0.0007) [2023-12-27 09:43:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 301981696. Throughput: 0: 10447.5, 1: 10196.2. Samples: 301988572. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 09:43:02,363][19170] Avg episode reward: [(0, '729.710'), (1, '165.920')] [2023-12-27 09:43:02,746][20381] Updated weights for policy 1, policy_version 589557 (0.0008) [2023-12-27 09:43:02,800][20381] Updated weights for policy 1, policy_version 589567 (0.0005) [2023-12-27 09:43:02,859][20381] Updated weights for policy 1, policy_version 589577 (0.0006) [2023-12-27 09:43:02,859][20347] Updated weights for policy 0, policy_version 589923 (0.0007) [2023-12-27 09:43:02,912][20347] Updated weights for policy 0, policy_version 589933 (0.0006) [2023-12-27 09:43:02,975][20347] Updated weights for policy 0, policy_version 589943 (0.0007) [2023-12-27 09:43:03,422][20381] Updated weights for policy 1, policy_version 589587 (0.0006) [2023-12-27 09:43:03,476][20381] Updated weights for policy 1, policy_version 589597 (0.0005) [2023-12-27 09:43:03,532][20381] Updated weights for policy 1, policy_version 589607 (0.0005) [2023-12-27 09:43:03,635][20347] Updated weights for policy 0, policy_version 589953 (0.0006) [2023-12-27 09:43:03,697][20347] Updated weights for policy 0, policy_version 589963 (0.0005) [2023-12-27 09:43:03,757][20347] Updated weights for policy 0, policy_version 589973 (0.0005) [2023-12-27 09:43:03,830][20347] Updated weights for policy 0, policy_version 589983 (0.0006) [2023-12-27 09:43:04,143][20381] Updated weights for policy 1, policy_version 589617 (0.0005) [2023-12-27 09:43:04,197][20381] Updated weights for policy 1, policy_version 589627 (0.0008) [2023-12-27 09:43:04,255][20381] Updated weights for policy 1, policy_version 589637 (0.0006) [2023-12-27 09:43:04,311][20381] Updated weights for policy 1, policy_version 589647 (0.0006) [2023-12-27 09:43:04,428][20347] Updated weights for policy 0, policy_version 589993 (0.0008) [2023-12-27 09:43:04,480][20347] Updated weights for policy 0, policy_version 590003 (0.0008) [2023-12-27 09:43:04,544][20347] Updated weights for policy 0, policy_version 590013 (0.0008) [2023-12-27 09:43:04,891][20381] Updated weights for policy 1, policy_version 589657 (0.0005) [2023-12-27 09:43:04,950][20381] Updated weights for policy 1, policy_version 589667 (0.0005) [2023-12-27 09:43:05,006][20381] Updated weights for policy 1, policy_version 589677 (0.0005) [2023-12-27 09:43:05,295][20347] Updated weights for policy 0, policy_version 590023 (0.0007) [2023-12-27 09:43:05,344][20347] Updated weights for policy 0, policy_version 590033 (0.0009) [2023-12-27 09:43:05,396][20347] Updated weights for policy 0, policy_version 590043 (0.0007) [2023-12-27 09:43:05,512][20381] Updated weights for policy 1, policy_version 589687 (0.0005) [2023-12-27 09:43:05,559][20381] Updated weights for policy 1, policy_version 589697 (0.0005) [2023-12-27 09:43:05,619][20381] Updated weights for policy 1, policy_version 589707 (0.0007) [2023-12-27 09:43:06,159][20347] Updated weights for policy 0, policy_version 590053 (0.0008) [2023-12-27 09:43:06,217][20347] Updated weights for policy 0, policy_version 590063 (0.0007) [2023-12-27 09:43:06,265][20381] Updated weights for policy 1, policy_version 589717 (0.0007) [2023-12-27 09:43:06,275][20347] Updated weights for policy 0, policy_version 590073 (0.0006) [2023-12-27 09:43:06,327][20381] Updated weights for policy 1, policy_version 589727 (0.0006) [2023-12-27 09:43:06,391][20381] Updated weights for policy 1, policy_version 589737 (0.0006) [2023-12-27 09:43:06,995][20347] Updated weights for policy 0, policy_version 590083 (0.0008) [2023-12-27 09:43:07,011][20381] Updated weights for policy 1, policy_version 589747 (0.0006) [2023-12-27 09:43:07,031][20291] KL-divergence is very high: 126.1158 [2023-12-27 09:43:07,047][20347] Updated weights for policy 0, policy_version 590093 (0.0005) [2023-12-27 09:43:07,070][20381] Updated weights for policy 1, policy_version 589757 (0.0006) [2023-12-27 09:43:07,097][20347] Updated weights for policy 0, policy_version 590103 (0.0007) [2023-12-27 09:43:07,121][20381] Updated weights for policy 1, policy_version 589767 (0.0008) [2023-12-27 09:43:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 302088192. Throughput: 0: 10435.7, 1: 10292.3. Samples: 302053172. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:07,363][19170] Avg episode reward: [(0, '731.880'), (1, '156.830')] [2023-12-27 09:43:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000589776_151003136.pth... [2023-12-27 09:43:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000590112_151085056.pth... [2023-12-27 09:43:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000588560_150691840.pth [2023-12-27 09:43:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000588896_150773760.pth [2023-12-27 09:43:07,772][20347] Updated weights for policy 0, policy_version 590113 (0.0005) [2023-12-27 09:43:07,801][20381] Updated weights for policy 1, policy_version 589777 (0.0007) [2023-12-27 09:43:07,832][20347] Updated weights for policy 0, policy_version 590123 (0.0005) [2023-12-27 09:43:07,857][20381] Updated weights for policy 1, policy_version 589787 (0.0011) [2023-12-27 09:43:07,890][20347] Updated weights for policy 0, policy_version 590133 (0.0007) [2023-12-27 09:43:07,912][20381] Updated weights for policy 1, policy_version 589797 (0.0010) [2023-12-27 09:43:07,946][20347] Updated weights for policy 0, policy_version 590143 (0.0007) [2023-12-27 09:43:07,973][20381] Updated weights for policy 1, policy_version 589807 (0.0010) [2023-12-27 09:43:08,607][20347] Updated weights for policy 0, policy_version 590153 (0.0007) [2023-12-27 09:43:08,645][20381] Updated weights for policy 1, policy_version 589817 (0.0011) [2023-12-27 09:43:08,662][20347] Updated weights for policy 0, policy_version 590163 (0.0007) [2023-12-27 09:43:08,702][20381] Updated weights for policy 1, policy_version 589827 (0.0008) [2023-12-27 09:43:08,719][20347] Updated weights for policy 0, policy_version 590173 (0.0007) [2023-12-27 09:43:08,761][20381] Updated weights for policy 1, policy_version 589837 (0.0008) [2023-12-27 09:43:09,398][20381] Updated weights for policy 1, policy_version 589847 (0.0010) [2023-12-27 09:43:09,458][20381] Updated weights for policy 1, policy_version 589857 (0.0010) [2023-12-27 09:43:09,462][20347] Updated weights for policy 0, policy_version 590183 (0.0006) [2023-12-27 09:43:09,515][20381] Updated weights for policy 1, policy_version 589867 (0.0011) [2023-12-27 09:43:09,521][20347] Updated weights for policy 0, policy_version 590193 (0.0007) [2023-12-27 09:43:09,580][20347] Updated weights for policy 0, policy_version 590203 (0.0008) [2023-12-27 09:43:10,219][20381] Updated weights for policy 1, policy_version 589877 (0.0011) [2023-12-27 09:43:10,275][20381] Updated weights for policy 1, policy_version 589887 (0.0010) [2023-12-27 09:43:10,329][20381] Updated weights for policy 1, policy_version 589897 (0.0008) [2023-12-27 09:43:10,342][20347] Updated weights for policy 0, policy_version 590213 (0.0008) [2023-12-27 09:43:10,397][20347] Updated weights for policy 0, policy_version 590223 (0.0008) [2023-12-27 09:43:10,451][20347] Updated weights for policy 0, policy_version 590233 (0.0008) [2023-12-27 09:43:11,017][20381] Updated weights for policy 1, policy_version 589907 (0.0011) [2023-12-27 09:43:11,081][20381] Updated weights for policy 1, policy_version 589917 (0.0009) [2023-12-27 09:43:11,141][20347] Updated weights for policy 0, policy_version 590243 (0.0009) [2023-12-27 09:43:11,145][20381] Updated weights for policy 1, policy_version 589927 (0.0007) [2023-12-27 09:43:11,199][20347] Updated weights for policy 0, policy_version 590253 (0.0010) [2023-12-27 09:43:11,258][20347] Updated weights for policy 0, policy_version 590263 (0.0010) [2023-12-27 09:43:11,811][20381] Updated weights for policy 1, policy_version 589937 (0.0008) [2023-12-27 09:43:11,871][20381] Updated weights for policy 1, policy_version 589947 (0.0008) [2023-12-27 09:43:11,936][20381] Updated weights for policy 1, policy_version 589957 (0.0007) [2023-12-27 09:43:11,990][20347] Updated weights for policy 0, policy_version 590273 (0.0010) [2023-12-27 09:43:11,997][20381] Updated weights for policy 1, policy_version 589967 (0.0005) [2023-12-27 09:43:12,043][20347] Updated weights for policy 0, policy_version 590283 (0.0010) [2023-12-27 09:43:12,103][20347] Updated weights for policy 0, policy_version 590293 (0.0008) [2023-12-27 09:43:12,165][20347] Updated weights for policy 0, policy_version 590303 (0.0007) [2023-12-27 09:43:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 302186496. Throughput: 0: 10372.4, 1: 10359.2. Samples: 302174116. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:12,363][19170] Avg episode reward: [(0, '731.270'), (1, '137.310')] [2023-12-27 09:43:12,597][20381] Updated weights for policy 1, policy_version 589977 (0.0005) [2023-12-27 09:43:12,651][20381] Updated weights for policy 1, policy_version 589987 (0.0006) [2023-12-27 09:43:12,711][20381] Updated weights for policy 1, policy_version 589997 (0.0007) [2023-12-27 09:43:12,825][20347] Updated weights for policy 0, policy_version 590313 (0.0008) [2023-12-27 09:43:12,875][20347] Updated weights for policy 0, policy_version 590323 (0.0007) [2023-12-27 09:43:12,927][20347] Updated weights for policy 0, policy_version 590333 (0.0007) [2023-12-27 09:43:13,344][20381] Updated weights for policy 1, policy_version 590007 (0.0007) [2023-12-27 09:43:13,406][20381] Updated weights for policy 1, policy_version 590017 (0.0007) [2023-12-27 09:43:13,461][20381] Updated weights for policy 1, policy_version 590027 (0.0008) [2023-12-27 09:43:13,564][20347] Updated weights for policy 0, policy_version 590343 (0.0005) [2023-12-27 09:43:13,611][20347] Updated weights for policy 0, policy_version 590353 (0.0008) [2023-12-27 09:43:13,657][20347] Updated weights for policy 0, policy_version 590363 (0.0010) [2023-12-27 09:43:14,011][20381] Updated weights for policy 1, policy_version 590037 (0.0008) [2023-12-27 09:43:14,063][20381] Updated weights for policy 1, policy_version 590047 (0.0008) [2023-12-27 09:43:14,120][20381] Updated weights for policy 1, policy_version 590057 (0.0007) [2023-12-27 09:43:14,331][20347] Updated weights for policy 0, policy_version 590373 (0.0006) [2023-12-27 09:43:14,393][20347] Updated weights for policy 0, policy_version 590383 (0.0008) [2023-12-27 09:43:14,451][20347] Updated weights for policy 0, policy_version 590393 (0.0007) [2023-12-27 09:43:14,768][20381] Updated weights for policy 1, policy_version 590067 (0.0008) [2023-12-27 09:43:14,828][20381] Updated weights for policy 1, policy_version 590077 (0.0010) [2023-12-27 09:43:14,880][20381] Updated weights for policy 1, policy_version 590087 (0.0010) [2023-12-27 09:43:15,126][20347] Updated weights for policy 0, policy_version 590403 (0.0009) [2023-12-27 09:43:15,185][20347] Updated weights for policy 0, policy_version 590413 (0.0010) [2023-12-27 09:43:15,247][20347] Updated weights for policy 0, policy_version 590423 (0.0010) [2023-12-27 09:43:15,553][20381] Updated weights for policy 1, policy_version 590097 (0.0010) [2023-12-27 09:43:15,605][20381] Updated weights for policy 1, policy_version 590107 (0.0006) [2023-12-27 09:43:15,661][20381] Updated weights for policy 1, policy_version 590117 (0.0010) [2023-12-27 09:43:15,710][20381] Updated weights for policy 1, policy_version 590127 (0.0010) [2023-12-27 09:43:15,923][20347] Updated weights for policy 0, policy_version 590433 (0.0010) [2023-12-27 09:43:15,974][20347] Updated weights for policy 0, policy_version 590443 (0.0010) [2023-12-27 09:43:16,018][20347] Updated weights for policy 0, policy_version 590453 (0.0010) [2023-12-27 09:43:16,072][20347] Updated weights for policy 0, policy_version 590463 (0.0010) [2023-12-27 09:43:16,343][20381] Updated weights for policy 1, policy_version 590137 (0.0011) [2023-12-27 09:43:16,392][20381] Updated weights for policy 1, policy_version 590147 (0.0011) [2023-12-27 09:43:16,441][20381] Updated weights for policy 1, policy_version 590157 (0.0010) [2023-12-27 09:43:16,674][20347] Updated weights for policy 0, policy_version 590473 (0.0007) [2023-12-27 09:43:16,734][20347] Updated weights for policy 0, policy_version 590483 (0.0008) [2023-12-27 09:43:16,797][20347] Updated weights for policy 0, policy_version 590493 (0.0006) [2023-12-27 09:43:17,047][20381] Updated weights for policy 1, policy_version 590167 (0.0009) [2023-12-27 09:43:17,098][20381] Updated weights for policy 1, policy_version 590177 (0.0010) [2023-12-27 09:43:17,155][20381] Updated weights for policy 1, policy_version 590187 (0.0010) [2023-12-27 09:43:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 302292992. Throughput: 0: 10413.9, 1: 10475.2. Samples: 302302480. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:17,363][19170] Avg episode reward: [(0, '672.740'), (1, '230.710')] [2023-12-27 09:43:17,387][20347] Updated weights for policy 0, policy_version 590503 (0.0007) [2023-12-27 09:43:17,439][20347] Updated weights for policy 0, policy_version 590513 (0.0010) [2023-12-27 09:43:17,488][20347] Updated weights for policy 0, policy_version 590523 (0.0010) [2023-12-27 09:43:17,840][20381] Updated weights for policy 1, policy_version 590197 (0.0010) [2023-12-27 09:43:17,894][20381] Updated weights for policy 1, policy_version 590207 (0.0011) [2023-12-27 09:43:17,952][20381] Updated weights for policy 1, policy_version 590217 (0.0010) [2023-12-27 09:43:18,187][20347] Updated weights for policy 0, policy_version 590533 (0.0009) [2023-12-27 09:43:18,241][20347] Updated weights for policy 0, policy_version 590543 (0.0007) [2023-12-27 09:43:18,299][20347] Updated weights for policy 0, policy_version 590553 (0.0007) [2023-12-27 09:43:18,648][20381] Updated weights for policy 1, policy_version 590227 (0.0010) [2023-12-27 09:43:18,700][20381] Updated weights for policy 1, policy_version 590237 (0.0010) [2023-12-27 09:43:18,756][20381] Updated weights for policy 1, policy_version 590247 (0.0010) [2023-12-27 09:43:19,016][20347] Updated weights for policy 0, policy_version 590563 (0.0007) [2023-12-27 09:43:19,073][20347] Updated weights for policy 0, policy_version 590573 (0.0007) [2023-12-27 09:43:19,128][20347] Updated weights for policy 0, policy_version 590583 (0.0007) [2023-12-27 09:43:19,460][20381] Updated weights for policy 1, policy_version 590257 (0.0010) [2023-12-27 09:43:19,521][20381] Updated weights for policy 1, policy_version 590267 (0.0008) [2023-12-27 09:43:19,575][20381] Updated weights for policy 1, policy_version 590277 (0.0006) [2023-12-27 09:43:19,630][20381] Updated weights for policy 1, policy_version 590287 (0.0006) [2023-12-27 09:43:19,908][20347] Updated weights for policy 0, policy_version 590593 (0.0007) [2023-12-27 09:43:19,970][20347] Updated weights for policy 0, policy_version 590603 (0.0009) [2023-12-27 09:43:20,033][20347] Updated weights for policy 0, policy_version 590613 (0.0007) [2023-12-27 09:43:20,099][20347] Updated weights for policy 0, policy_version 590623 (0.0006) [2023-12-27 09:43:20,304][20381] Updated weights for policy 1, policy_version 590297 (0.0007) [2023-12-27 09:43:20,363][20381] Updated weights for policy 1, policy_version 590307 (0.0007) [2023-12-27 09:43:20,422][20381] Updated weights for policy 1, policy_version 590317 (0.0007) [2023-12-27 09:43:20,769][20347] Updated weights for policy 0, policy_version 590633 (0.0008) [2023-12-27 09:43:20,828][20347] Updated weights for policy 0, policy_version 590643 (0.0008) [2023-12-27 09:43:20,902][20347] Updated weights for policy 0, policy_version 590653 (0.0007) [2023-12-27 09:43:21,112][20381] Updated weights for policy 1, policy_version 590327 (0.0008) [2023-12-27 09:43:21,178][20381] Updated weights for policy 1, policy_version 590337 (0.0006) [2023-12-27 09:43:21,240][20381] Updated weights for policy 1, policy_version 590347 (0.0007) [2023-12-27 09:43:21,565][20347] Updated weights for policy 0, policy_version 590663 (0.0006) [2023-12-27 09:43:21,623][20347] Updated weights for policy 0, policy_version 590673 (0.0008) [2023-12-27 09:43:21,680][20347] Updated weights for policy 0, policy_version 590683 (0.0006) [2023-12-27 09:43:21,996][20381] Updated weights for policy 1, policy_version 590357 (0.0008) [2023-12-27 09:43:22,049][20381] Updated weights for policy 1, policy_version 590367 (0.0010) [2023-12-27 09:43:22,109][20381] Updated weights for policy 1, policy_version 590377 (0.0008) [2023-12-27 09:43:22,282][20347] Updated weights for policy 0, policy_version 590693 (0.0008) [2023-12-27 09:43:22,346][20347] Updated weights for policy 0, policy_version 590703 (0.0007) [2023-12-27 09:43:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 302391296. Throughput: 0: 10269.7, 1: 10453.2. Samples: 302362852. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:22,363][19170] Avg episode reward: [(0, '683.100'), (1, '602.950')] [2023-12-27 09:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000590384_151158784.pth... [2023-12-27 09:43:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000589168_150847488.pth [2023-12-27 09:43:22,406][20347] Updated weights for policy 0, policy_version 590713 (0.0007) [2023-12-27 09:43:22,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000590720_151240704.pth... [2023-12-27 09:43:22,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000589504_150929408.pth [2023-12-27 09:43:22,824][20381] Updated weights for policy 1, policy_version 590387 (0.0008) [2023-12-27 09:43:22,886][20381] Updated weights for policy 1, policy_version 590397 (0.0007) [2023-12-27 09:43:22,941][20381] Updated weights for policy 1, policy_version 590407 (0.0005) [2023-12-27 09:43:23,097][20347] Updated weights for policy 0, policy_version 590723 (0.0007) [2023-12-27 09:43:23,164][20347] Updated weights for policy 0, policy_version 590733 (0.0008) [2023-12-27 09:43:23,218][20347] Updated weights for policy 0, policy_version 590743 (0.0007) [2023-12-27 09:43:23,600][20381] Updated weights for policy 1, policy_version 590417 (0.0008) [2023-12-27 09:43:23,646][20381] Updated weights for policy 1, policy_version 590427 (0.0008) [2023-12-27 09:43:23,695][20381] Updated weights for policy 1, policy_version 590437 (0.0007) [2023-12-27 09:43:23,749][20381] Updated weights for policy 1, policy_version 590447 (0.0008) [2023-12-27 09:43:23,871][20347] Updated weights for policy 0, policy_version 590753 (0.0006) [2023-12-27 09:43:23,932][20347] Updated weights for policy 0, policy_version 590763 (0.0006) [2023-12-27 09:43:23,988][20347] Updated weights for policy 0, policy_version 590773 (0.0007) [2023-12-27 09:43:24,042][20347] Updated weights for policy 0, policy_version 590783 (0.0007) [2023-12-27 09:43:24,530][20381] Updated weights for policy 1, policy_version 590457 (0.0008) [2023-12-27 09:43:24,586][20381] Updated weights for policy 1, policy_version 590467 (0.0007) [2023-12-27 09:43:24,632][20347] Updated weights for policy 0, policy_version 590793 (0.0005) [2023-12-27 09:43:24,636][20381] Updated weights for policy 1, policy_version 590477 (0.0008) [2023-12-27 09:43:24,693][20347] Updated weights for policy 0, policy_version 590803 (0.0006) [2023-12-27 09:43:24,754][20347] Updated weights for policy 0, policy_version 590813 (0.0005) [2023-12-27 09:43:25,303][20381] Updated weights for policy 1, policy_version 590487 (0.0006) [2023-12-27 09:43:25,328][20347] Updated weights for policy 0, policy_version 590823 (0.0005) [2023-12-27 09:43:25,353][20381] Updated weights for policy 1, policy_version 590497 (0.0005) [2023-12-27 09:43:25,386][20347] Updated weights for policy 0, policy_version 590833 (0.0011) [2023-12-27 09:43:25,413][20381] Updated weights for policy 1, policy_version 590507 (0.0005) [2023-12-27 09:43:25,445][20347] Updated weights for policy 0, policy_version 590843 (0.0006) [2023-12-27 09:43:25,981][20347] Updated weights for policy 0, policy_version 590853 (0.0006) [2023-12-27 09:43:26,036][20347] Updated weights for policy 0, policy_version 590863 (0.0010) [2023-12-27 09:43:26,089][20381] Updated weights for policy 1, policy_version 590517 (0.0005) [2023-12-27 09:43:26,091][20347] Updated weights for policy 0, policy_version 590873 (0.0010) [2023-12-27 09:43:26,146][20381] Updated weights for policy 1, policy_version 590527 (0.0006) [2023-12-27 09:43:26,201][20381] Updated weights for policy 1, policy_version 590537 (0.0008) [2023-12-27 09:43:26,758][20347] Updated weights for policy 0, policy_version 590883 (0.0011) [2023-12-27 09:43:26,802][20347] Updated weights for policy 0, policy_version 590893 (0.0010) [2023-12-27 09:43:26,806][20381] Updated weights for policy 1, policy_version 590547 (0.0007) [2023-12-27 09:43:26,845][20347] Updated weights for policy 0, policy_version 590903 (0.0010) [2023-12-27 09:43:26,855][20381] Updated weights for policy 1, policy_version 590557 (0.0010) [2023-12-27 09:43:26,900][20381] Updated weights for policy 1, policy_version 590567 (0.0009) [2023-12-27 09:43:27,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 302497792. Throughput: 0: 10334.7, 1: 10394.9. Samples: 302487332. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:27,364][19170] Avg episode reward: [(0, '695.920'), (1, '629.140')] [2023-12-27 09:43:27,497][20381] Updated weights for policy 1, policy_version 590577 (0.0006) [2023-12-27 09:43:27,519][20347] Updated weights for policy 0, policy_version 590913 (0.0010) [2023-12-27 09:43:27,551][20381] Updated weights for policy 1, policy_version 590587 (0.0007) [2023-12-27 09:43:27,574][20347] Updated weights for policy 0, policy_version 590923 (0.0008) [2023-12-27 09:43:27,612][20381] Updated weights for policy 1, policy_version 590597 (0.0006) [2023-12-27 09:43:27,629][20347] Updated weights for policy 0, policy_version 590933 (0.0005) [2023-12-27 09:43:27,661][20381] Updated weights for policy 1, policy_version 590607 (0.0006) [2023-12-27 09:43:27,682][20347] Updated weights for policy 0, policy_version 590943 (0.0007) [2023-12-27 09:43:28,210][20381] Updated weights for policy 1, policy_version 590617 (0.0007) [2023-12-27 09:43:28,264][20381] Updated weights for policy 1, policy_version 590627 (0.0008) [2023-12-27 09:43:28,327][20381] Updated weights for policy 1, policy_version 590637 (0.0007) [2023-12-27 09:43:28,374][20347] Updated weights for policy 0, policy_version 590953 (0.0007) [2023-12-27 09:43:28,429][20347] Updated weights for policy 0, policy_version 590963 (0.0008) [2023-12-27 09:43:28,477][20347] Updated weights for policy 0, policy_version 590973 (0.0008) [2023-12-27 09:43:29,005][20381] Updated weights for policy 1, policy_version 590647 (0.0007) [2023-12-27 09:43:29,055][20381] Updated weights for policy 1, policy_version 590657 (0.0006) [2023-12-27 09:43:29,102][20381] Updated weights for policy 1, policy_version 590667 (0.0006) [2023-12-27 09:43:29,140][20347] Updated weights for policy 0, policy_version 590983 (0.0009) [2023-12-27 09:43:29,188][20347] Updated weights for policy 0, policy_version 590993 (0.0010) [2023-12-27 09:43:29,240][20347] Updated weights for policy 0, policy_version 591003 (0.0008) [2023-12-27 09:43:29,796][20381] Updated weights for policy 1, policy_version 590677 (0.0006) [2023-12-27 09:43:29,855][20381] Updated weights for policy 1, policy_version 590687 (0.0006) [2023-12-27 09:43:29,910][20381] Updated weights for policy 1, policy_version 590697 (0.0007) [2023-12-27 09:43:29,933][20347] Updated weights for policy 0, policy_version 591013 (0.0008) [2023-12-27 09:43:29,993][20347] Updated weights for policy 0, policy_version 591023 (0.0008) [2023-12-27 09:43:30,056][20347] Updated weights for policy 0, policy_version 591033 (0.0006) [2023-12-27 09:43:30,498][20381] Updated weights for policy 1, policy_version 590707 (0.0006) [2023-12-27 09:43:30,554][20381] Updated weights for policy 1, policy_version 590717 (0.0007) [2023-12-27 09:43:30,608][20381] Updated weights for policy 1, policy_version 590727 (0.0005) [2023-12-27 09:43:30,753][20347] Updated weights for policy 0, policy_version 591043 (0.0006) [2023-12-27 09:43:30,807][20347] Updated weights for policy 0, policy_version 591053 (0.0008) [2023-12-27 09:43:30,851][20347] Updated weights for policy 0, policy_version 591063 (0.0008) [2023-12-27 09:43:31,191][20381] Updated weights for policy 1, policy_version 590737 (0.0005) [2023-12-27 09:43:31,246][20381] Updated weights for policy 1, policy_version 590747 (0.0008) [2023-12-27 09:43:31,300][20381] Updated weights for policy 1, policy_version 590757 (0.0007) [2023-12-27 09:43:31,358][20381] Updated weights for policy 1, policy_version 590767 (0.0007) [2023-12-27 09:43:31,574][20347] Updated weights for policy 0, policy_version 591073 (0.0007) [2023-12-27 09:43:31,628][20347] Updated weights for policy 0, policy_version 591083 (0.0010) [2023-12-27 09:43:31,684][20347] Updated weights for policy 0, policy_version 591093 (0.0008) [2023-12-27 09:43:31,745][20347] Updated weights for policy 0, policy_version 591103 (0.0007) [2023-12-27 09:43:32,021][20381] Updated weights for policy 1, policy_version 590777 (0.0007) [2023-12-27 09:43:32,080][20381] Updated weights for policy 1, policy_version 590787 (0.0006) [2023-12-27 09:43:32,142][20381] Updated weights for policy 1, policy_version 590797 (0.0006) [2023-12-27 09:43:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 302604288. Throughput: 0: 10284.8, 1: 10501.5. Samples: 302614844. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:32,363][19170] Avg episode reward: [(0, '680.050'), (1, '728.350')] [2023-12-27 09:43:32,456][20347] Updated weights for policy 0, policy_version 591113 (0.0007) [2023-12-27 09:43:32,503][20347] Updated weights for policy 0, policy_version 591123 (0.0007) [2023-12-27 09:43:32,561][20347] Updated weights for policy 0, policy_version 591133 (0.0005) [2023-12-27 09:43:32,750][20381] Updated weights for policy 1, policy_version 590807 (0.0005) [2023-12-27 09:43:32,796][20381] Updated weights for policy 1, policy_version 590817 (0.0005) [2023-12-27 09:43:32,838][20381] Updated weights for policy 1, policy_version 590827 (0.0005) [2023-12-27 09:43:33,282][20347] Updated weights for policy 0, policy_version 591143 (0.0007) [2023-12-27 09:43:33,329][20347] Updated weights for policy 0, policy_version 591153 (0.0007) [2023-12-27 09:43:33,373][20347] Updated weights for policy 0, policy_version 591163 (0.0008) [2023-12-27 09:43:33,404][20381] Updated weights for policy 1, policy_version 590837 (0.0006) [2023-12-27 09:43:33,459][20381] Updated weights for policy 1, policy_version 590847 (0.0007) [2023-12-27 09:43:33,516][20381] Updated weights for policy 1, policy_version 590857 (0.0006) [2023-12-27 09:43:33,971][20347] Updated weights for policy 0, policy_version 591173 (0.0008) [2023-12-27 09:43:34,021][20347] Updated weights for policy 0, policy_version 591183 (0.0007) [2023-12-27 09:43:34,065][20381] Updated weights for policy 1, policy_version 590867 (0.0008) [2023-12-27 09:43:34,067][20347] Updated weights for policy 0, policy_version 591193 (0.0006) [2023-12-27 09:43:34,119][20381] Updated weights for policy 1, policy_version 590877 (0.0010) [2023-12-27 09:43:34,180][20381] Updated weights for policy 1, policy_version 590887 (0.0010) [2023-12-27 09:43:34,770][20347] Updated weights for policy 0, policy_version 591203 (0.0005) [2023-12-27 09:43:34,801][20381] Updated weights for policy 1, policy_version 590897 (0.0009) [2023-12-27 09:43:34,824][20347] Updated weights for policy 0, policy_version 591213 (0.0005) [2023-12-27 09:43:34,852][20381] Updated weights for policy 1, policy_version 590907 (0.0008) [2023-12-27 09:43:34,886][20347] Updated weights for policy 0, policy_version 591223 (0.0005) [2023-12-27 09:43:34,904][20381] Updated weights for policy 1, policy_version 590917 (0.0007) [2023-12-27 09:43:34,954][20381] Updated weights for policy 1, policy_version 590927 (0.0006) [2023-12-27 09:43:35,466][20347] Updated weights for policy 0, policy_version 591233 (0.0005) [2023-12-27 09:43:35,528][20347] Updated weights for policy 0, policy_version 591243 (0.0005) [2023-12-27 09:43:35,585][20347] Updated weights for policy 0, policy_version 591253 (0.0005) [2023-12-27 09:43:35,634][20381] Updated weights for policy 1, policy_version 590937 (0.0006) [2023-12-27 09:43:35,643][20347] Updated weights for policy 0, policy_version 591263 (0.0005) [2023-12-27 09:43:35,695][20381] Updated weights for policy 1, policy_version 590947 (0.0005) [2023-12-27 09:43:35,748][20381] Updated weights for policy 1, policy_version 590957 (0.0008) [2023-12-27 09:43:36,248][20347] Updated weights for policy 0, policy_version 591273 (0.0006) [2023-12-27 09:43:36,300][20347] Updated weights for policy 0, policy_version 591283 (0.0006) [2023-12-27 09:43:36,350][20347] Updated weights for policy 0, policy_version 591293 (0.0008) [2023-12-27 09:43:36,436][20381] Updated weights for policy 1, policy_version 590967 (0.0007) [2023-12-27 09:43:36,496][20381] Updated weights for policy 1, policy_version 590977 (0.0007) [2023-12-27 09:43:36,552][20381] Updated weights for policy 1, policy_version 590987 (0.0006) [2023-12-27 09:43:37,016][20347] Updated weights for policy 0, policy_version 591303 (0.0007) [2023-12-27 09:43:37,071][20347] Updated weights for policy 0, policy_version 591313 (0.0007) [2023-12-27 09:43:37,122][20347] Updated weights for policy 0, policy_version 591323 (0.0007) [2023-12-27 09:43:37,211][20381] Updated weights for policy 1, policy_version 590997 (0.0007) [2023-12-27 09:43:37,267][20381] Updated weights for policy 1, policy_version 591007 (0.0007) [2023-12-27 09:43:37,324][20381] Updated weights for policy 1, policy_version 591017 (0.0008) [2023-12-27 09:43:37,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 302710784. Throughput: 0: 10315.5, 1: 10578.2. Samples: 302680560. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:37,363][19170] Avg episode reward: [(0, '678.380'), (1, '727.990')] [2023-12-27 09:43:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000591024_151322624.pth... [2023-12-27 09:43:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000591328_151396352.pth... [2023-12-27 09:43:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000589776_151003136.pth [2023-12-27 09:43:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000590112_151085056.pth [2023-12-27 09:43:37,821][20347] Updated weights for policy 0, policy_version 591333 (0.0006) [2023-12-27 09:43:37,878][20347] Updated weights for policy 0, policy_version 591343 (0.0007) [2023-12-27 09:43:37,942][20347] Updated weights for policy 0, policy_version 591353 (0.0006) [2023-12-27 09:43:37,966][20381] Updated weights for policy 1, policy_version 591027 (0.0006) [2023-12-27 09:43:38,014][20381] Updated weights for policy 1, policy_version 591037 (0.0007) [2023-12-27 09:43:38,059][20381] Updated weights for policy 1, policy_version 591047 (0.0009) [2023-12-27 09:43:38,569][20347] Updated weights for policy 0, policy_version 591363 (0.0006) [2023-12-27 09:43:38,633][20347] Updated weights for policy 0, policy_version 591373 (0.0008) [2023-12-27 09:43:38,677][20381] Updated weights for policy 1, policy_version 591057 (0.0008) [2023-12-27 09:43:38,694][20347] Updated weights for policy 0, policy_version 591383 (0.0008) [2023-12-27 09:43:38,735][20381] Updated weights for policy 1, policy_version 591067 (0.0006) [2023-12-27 09:43:38,790][20381] Updated weights for policy 1, policy_version 591077 (0.0007) [2023-12-27 09:43:38,846][20381] Updated weights for policy 1, policy_version 591087 (0.0008) [2023-12-27 09:43:39,372][20347] Updated weights for policy 0, policy_version 591393 (0.0006) [2023-12-27 09:43:39,428][20347] Updated weights for policy 0, policy_version 591403 (0.0008) [2023-12-27 09:43:39,486][20347] Updated weights for policy 0, policy_version 591413 (0.0006) [2023-12-27 09:43:39,549][20347] Updated weights for policy 0, policy_version 591423 (0.0007) [2023-12-27 09:43:39,602][20381] Updated weights for policy 1, policy_version 591097 (0.0010) [2023-12-27 09:43:39,661][20381] Updated weights for policy 1, policy_version 591107 (0.0011) [2023-12-27 09:43:39,720][20381] Updated weights for policy 1, policy_version 591117 (0.0011) [2023-12-27 09:43:40,285][20347] Updated weights for policy 0, policy_version 591433 (0.0007) [2023-12-27 09:43:40,354][20347] Updated weights for policy 0, policy_version 591443 (0.0008) [2023-12-27 09:43:40,419][20347] Updated weights for policy 0, policy_version 591453 (0.0008) [2023-12-27 09:43:40,464][20381] Updated weights for policy 1, policy_version 591127 (0.0009) [2023-12-27 09:43:40,523][20381] Updated weights for policy 1, policy_version 591137 (0.0008) [2023-12-27 09:43:40,572][20381] Updated weights for policy 1, policy_version 591147 (0.0006) [2023-12-27 09:43:41,113][20347] Updated weights for policy 0, policy_version 591463 (0.0008) [2023-12-27 09:43:41,168][20381] Updated weights for policy 1, policy_version 591157 (0.0006) [2023-12-27 09:43:41,177][20347] Updated weights for policy 0, policy_version 591473 (0.0007) [2023-12-27 09:43:41,227][20381] Updated weights for policy 1, policy_version 591167 (0.0006) [2023-12-27 09:43:41,234][20347] Updated weights for policy 0, policy_version 591483 (0.0006) [2023-12-27 09:43:41,286][20381] Updated weights for policy 1, policy_version 591177 (0.0008) [2023-12-27 09:43:41,887][20347] Updated weights for policy 0, policy_version 591493 (0.0006) [2023-12-27 09:43:41,947][20347] Updated weights for policy 0, policy_version 591503 (0.0008) [2023-12-27 09:43:42,011][20347] Updated weights for policy 0, policy_version 591513 (0.0007) [2023-12-27 09:43:42,023][20381] Updated weights for policy 1, policy_version 591187 (0.0007) [2023-12-27 09:43:42,083][20381] Updated weights for policy 1, policy_version 591197 (0.0008) [2023-12-27 09:43:42,146][20381] Updated weights for policy 1, policy_version 591207 (0.0007) [2023-12-27 09:43:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 302817280. Throughput: 0: 10304.5, 1: 10608.2. Samples: 302804476. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:42,363][19170] Avg episode reward: [(0, '655.830'), (1, '835.360')] [2023-12-27 09:43:42,746][20347] Updated weights for policy 0, policy_version 591523 (0.0007) [2023-12-27 09:43:42,793][20381] Updated weights for policy 1, policy_version 591217 (0.0008) [2023-12-27 09:43:42,804][20347] Updated weights for policy 0, policy_version 591533 (0.0007) [2023-12-27 09:43:42,848][20381] Updated weights for policy 1, policy_version 591227 (0.0010) [2023-12-27 09:43:42,854][20347] Updated weights for policy 0, policy_version 591543 (0.0008) [2023-12-27 09:43:42,904][20381] Updated weights for policy 1, policy_version 591237 (0.0010) [2023-12-27 09:43:42,959][20381] Updated weights for policy 1, policy_version 591247 (0.0010) [2023-12-27 09:43:43,490][20347] Updated weights for policy 0, policy_version 591553 (0.0005) [2023-12-27 09:43:43,549][20347] Updated weights for policy 0, policy_version 591563 (0.0008) [2023-12-27 09:43:43,606][20347] Updated weights for policy 0, policy_version 591573 (0.0008) [2023-12-27 09:43:43,617][20381] Updated weights for policy 1, policy_version 591257 (0.0010) [2023-12-27 09:43:43,656][20347] Updated weights for policy 0, policy_version 591583 (0.0007) [2023-12-27 09:43:43,662][20381] Updated weights for policy 1, policy_version 591267 (0.0010) [2023-12-27 09:43:43,707][20381] Updated weights for policy 1, policy_version 591277 (0.0010) [2023-12-27 09:43:44,354][20381] Updated weights for policy 1, policy_version 591287 (0.0010) [2023-12-27 09:43:44,363][20347] Updated weights for policy 0, policy_version 591593 (0.0007) [2023-12-27 09:43:44,413][20381] Updated weights for policy 1, policy_version 591297 (0.0010) [2023-12-27 09:43:44,422][20347] Updated weights for policy 0, policy_version 591603 (0.0006) [2023-12-27 09:43:44,463][20381] Updated weights for policy 1, policy_version 591307 (0.0010) [2023-12-27 09:43:44,476][20347] Updated weights for policy 0, policy_version 591613 (0.0008) [2023-12-27 09:43:45,063][20381] Updated weights for policy 1, policy_version 591317 (0.0009) [2023-12-27 09:43:45,127][20381] Updated weights for policy 1, policy_version 591327 (0.0008) [2023-12-27 09:43:45,169][20347] Updated weights for policy 0, policy_version 591623 (0.0006) [2023-12-27 09:43:45,192][20381] Updated weights for policy 1, policy_version 591337 (0.0007) [2023-12-27 09:43:45,229][20347] Updated weights for policy 0, policy_version 591633 (0.0005) [2023-12-27 09:43:45,279][20347] Updated weights for policy 0, policy_version 591643 (0.0007) [2023-12-27 09:43:45,858][20381] Updated weights for policy 1, policy_version 591347 (0.0008) [2023-12-27 09:43:45,870][20347] Updated weights for policy 0, policy_version 591653 (0.0009) [2023-12-27 09:43:45,913][20381] Updated weights for policy 1, policy_version 591357 (0.0010) [2023-12-27 09:43:45,921][20347] Updated weights for policy 0, policy_version 591663 (0.0007) [2023-12-27 09:43:45,964][20381] Updated weights for policy 1, policy_version 591367 (0.0010) [2023-12-27 09:43:45,970][20347] Updated weights for policy 0, policy_version 591673 (0.0010) [2023-12-27 09:43:46,540][20347] Updated weights for policy 0, policy_version 591683 (0.0010) [2023-12-27 09:43:46,593][20347] Updated weights for policy 0, policy_version 591693 (0.0009) [2023-12-27 09:43:46,606][20381] Updated weights for policy 1, policy_version 591377 (0.0010) [2023-12-27 09:43:46,643][20347] Updated weights for policy 0, policy_version 591703 (0.0009) [2023-12-27 09:43:46,663][20381] Updated weights for policy 1, policy_version 591387 (0.0005) [2023-12-27 09:43:46,710][20381] Updated weights for policy 1, policy_version 591397 (0.0010) [2023-12-27 09:43:46,759][20381] Updated weights for policy 1, policy_version 591407 (0.0010) [2023-12-27 09:43:47,323][20347] Updated weights for policy 0, policy_version 591713 (0.0010) [2023-12-27 09:43:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 302915584. Throughput: 0: 10301.9, 1: 10625.1. Samples: 302930292. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:47,363][19170] Avg episode reward: [(0, '677.230'), (1, '893.320')] [2023-12-27 09:43:47,374][20347] Updated weights for policy 0, policy_version 591723 (0.0010) [2023-12-27 09:43:47,424][20381] Updated weights for policy 1, policy_version 591417 (0.0010) [2023-12-27 09:43:47,429][20347] Updated weights for policy 0, policy_version 591733 (0.0010) [2023-12-27 09:43:47,475][20381] Updated weights for policy 1, policy_version 591427 (0.0010) [2023-12-27 09:43:47,483][20347] Updated weights for policy 0, policy_version 591743 (0.0010) [2023-12-27 09:43:47,526][20381] Updated weights for policy 1, policy_version 591437 (0.0010) [2023-12-27 09:43:48,109][20381] Updated weights for policy 1, policy_version 591447 (0.0010) [2023-12-27 09:43:48,113][20347] Updated weights for policy 0, policy_version 591753 (0.0006) [2023-12-27 09:43:48,163][20381] Updated weights for policy 1, policy_version 591457 (0.0006) [2023-12-27 09:43:48,169][20347] Updated weights for policy 0, policy_version 591763 (0.0005) [2023-12-27 09:43:48,221][20381] Updated weights for policy 1, policy_version 591467 (0.0005) [2023-12-27 09:43:48,231][20347] Updated weights for policy 0, policy_version 591773 (0.0006) [2023-12-27 09:43:48,819][20381] Updated weights for policy 1, policy_version 591477 (0.0006) [2023-12-27 09:43:48,861][20347] Updated weights for policy 0, policy_version 591783 (0.0006) [2023-12-27 09:43:48,865][20381] Updated weights for policy 1, policy_version 591487 (0.0006) [2023-12-27 09:43:48,912][20381] Updated weights for policy 1, policy_version 591497 (0.0005) [2023-12-27 09:43:48,926][20347] Updated weights for policy 0, policy_version 591793 (0.0006) [2023-12-27 09:43:48,995][20347] Updated weights for policy 0, policy_version 591803 (0.0005) [2023-12-27 09:43:49,609][20347] Updated weights for policy 0, policy_version 591813 (0.0006) [2023-12-27 09:43:49,631][20381] Updated weights for policy 1, policy_version 591507 (0.0006) [2023-12-27 09:43:49,675][20347] Updated weights for policy 0, policy_version 591823 (0.0007) [2023-12-27 09:43:49,687][20381] Updated weights for policy 1, policy_version 591517 (0.0008) [2023-12-27 09:43:49,728][20347] Updated weights for policy 0, policy_version 591833 (0.0007) [2023-12-27 09:43:49,738][20381] Updated weights for policy 1, policy_version 591527 (0.0006) [2023-12-27 09:43:50,384][20381] Updated weights for policy 1, policy_version 591537 (0.0006) [2023-12-27 09:43:50,433][20381] Updated weights for policy 1, policy_version 591547 (0.0005) [2023-12-27 09:43:50,458][20347] Updated weights for policy 0, policy_version 591843 (0.0006) [2023-12-27 09:43:50,486][20381] Updated weights for policy 1, policy_version 591557 (0.0006) [2023-12-27 09:43:50,510][20347] Updated weights for policy 0, policy_version 591853 (0.0006) [2023-12-27 09:43:50,542][20381] Updated weights for policy 1, policy_version 591567 (0.0006) [2023-12-27 09:43:50,569][20347] Updated weights for policy 0, policy_version 591863 (0.0008) [2023-12-27 09:43:51,230][20381] Updated weights for policy 1, policy_version 591577 (0.0008) [2023-12-27 09:43:51,291][20381] Updated weights for policy 1, policy_version 591587 (0.0008) [2023-12-27 09:43:51,314][20347] Updated weights for policy 0, policy_version 591873 (0.0007) [2023-12-27 09:43:51,355][20381] Updated weights for policy 1, policy_version 591597 (0.0007) [2023-12-27 09:43:51,387][20347] Updated weights for policy 0, policy_version 591883 (0.0008) [2023-12-27 09:43:51,443][20347] Updated weights for policy 0, policy_version 591893 (0.0008) [2023-12-27 09:43:51,505][20347] Updated weights for policy 0, policy_version 591903 (0.0008) [2023-12-27 09:43:52,058][20381] Updated weights for policy 1, policy_version 591607 (0.0008) [2023-12-27 09:43:52,106][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-27 09:43:52,197][20347] Updated weights for policy 0, policy_version 591913 (0.0007) [2023-12-27 09:43:52,252][20347] Updated weights for policy 0, policy_version 591923 (0.0006) [2023-12-27 09:43:52,335][20347] Updated weights for policy 0, policy_version 591933 (0.0008) [2023-12-27 09:43:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 303030272. Throughput: 0: 10337.3, 1: 10585.2. Samples: 302994684. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:52,363][19170] Avg episode reward: [(0, '684.810'), (1, '883.350')] [2023-12-27 09:43:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000591616_151478272.pth... [2023-12-27 09:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000591936_151552000.pth... [2023-12-27 09:43:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000590384_151158784.pth [2023-12-27 09:43:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000590720_151240704.pth [2023-12-27 09:43:52,796][20381] Updated weights for policy 1, policy_version 591617 (0.0008) [2023-12-27 09:43:52,859][20381] Updated weights for policy 1, policy_version 591627 (0.0008) [2023-12-27 09:43:52,931][20381] Updated weights for policy 1, policy_version 591637 (0.0006) [2023-12-27 09:43:52,994][20381] Updated weights for policy 1, policy_version 591647 (0.0007) [2023-12-27 09:43:52,996][20347] Updated weights for policy 0, policy_version 591943 (0.0005) [2023-12-27 09:43:53,052][20347] Updated weights for policy 0, policy_version 591953 (0.0008) [2023-12-27 09:43:53,107][20347] Updated weights for policy 0, policy_version 591963 (0.0007) [2023-12-27 09:43:53,638][20381] Updated weights for policy 1, policy_version 591657 (0.0007) [2023-12-27 09:43:53,698][20381] Updated weights for policy 1, policy_version 591667 (0.0007) [2023-12-27 09:43:53,756][20381] Updated weights for policy 1, policy_version 591677 (0.0005) [2023-12-27 09:43:53,791][20347] Updated weights for policy 0, policy_version 591973 (0.0006) [2023-12-27 09:43:53,842][20347] Updated weights for policy 0, policy_version 591983 (0.0005) [2023-12-27 09:43:53,901][20347] Updated weights for policy 0, policy_version 591993 (0.0005) [2023-12-27 09:43:54,422][20381] Updated weights for policy 1, policy_version 591687 (0.0006) [2023-12-27 09:43:54,466][20347] Updated weights for policy 0, policy_version 592003 (0.0005) [2023-12-27 09:43:54,468][20381] Updated weights for policy 1, policy_version 591697 (0.0005) [2023-12-27 09:43:54,522][20347] Updated weights for policy 0, policy_version 592013 (0.0006) [2023-12-27 09:43:54,524][20381] Updated weights for policy 1, policy_version 591707 (0.0005) [2023-12-27 09:43:54,587][20347] Updated weights for policy 0, policy_version 592023 (0.0008) [2023-12-27 09:43:55,200][20347] Updated weights for policy 0, policy_version 592033 (0.0007) [2023-12-27 09:43:55,252][20381] Updated weights for policy 1, policy_version 591717 (0.0006) [2023-12-27 09:43:55,257][20347] Updated weights for policy 0, policy_version 592043 (0.0007) [2023-12-27 09:43:55,303][20381] Updated weights for policy 1, policy_version 591727 (0.0005) [2023-12-27 09:43:55,312][20347] Updated weights for policy 0, policy_version 592053 (0.0007) [2023-12-27 09:43:55,356][20381] Updated weights for policy 1, policy_version 591737 (0.0006) [2023-12-27 09:43:55,369][20347] Updated weights for policy 0, policy_version 592063 (0.0006) [2023-12-27 09:43:56,043][20381] Updated weights for policy 1, policy_version 591747 (0.0006) [2023-12-27 09:43:56,043][20347] Updated weights for policy 0, policy_version 592073 (0.0006) [2023-12-27 09:43:56,095][20381] Updated weights for policy 1, policy_version 591757 (0.0007) [2023-12-27 09:43:56,103][20347] Updated weights for policy 0, policy_version 592083 (0.0005) [2023-12-27 09:43:56,144][20381] Updated weights for policy 1, policy_version 591767 (0.0007) [2023-12-27 09:43:56,162][20347] Updated weights for policy 0, policy_version 592093 (0.0005) [2023-12-27 09:43:56,719][20347] Updated weights for policy 0, policy_version 592103 (0.0006) [2023-12-27 09:43:56,768][20347] Updated weights for policy 0, policy_version 592113 (0.0005) [2023-12-27 09:43:56,818][20347] Updated weights for policy 0, policy_version 592123 (0.0006) [2023-12-27 09:43:56,890][20381] Updated weights for policy 1, policy_version 591777 (0.0007) [2023-12-27 09:43:56,942][20381] Updated weights for policy 1, policy_version 591787 (0.0007) [2023-12-27 09:43:56,990][20381] Updated weights for policy 1, policy_version 591797 (0.0008) [2023-12-27 09:43:57,052][20381] Updated weights for policy 1, policy_version 591807 (0.0008) [2023-12-27 09:43:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 303128576. Throughput: 0: 10413.6, 1: 10550.2. Samples: 303117488. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:43:57,363][19170] Avg episode reward: [(0, '680.240'), (1, '854.390')] [2023-12-27 09:43:57,438][20347] Updated weights for policy 0, policy_version 592133 (0.0005) [2023-12-27 09:43:57,484][20347] Updated weights for policy 0, policy_version 592143 (0.0005) [2023-12-27 09:43:57,531][20347] Updated weights for policy 0, policy_version 592153 (0.0006) [2023-12-27 09:43:57,777][20381] Updated weights for policy 1, policy_version 591817 (0.0008) [2023-12-27 09:43:57,830][20381] Updated weights for policy 1, policy_version 591827 (0.0007) [2023-12-27 09:43:57,875][20381] Updated weights for policy 1, policy_version 591837 (0.0005) [2023-12-27 09:43:58,178][20347] Updated weights for policy 0, policy_version 592163 (0.0007) [2023-12-27 09:43:58,237][20347] Updated weights for policy 0, policy_version 592173 (0.0010) [2023-12-27 09:43:58,293][20347] Updated weights for policy 0, policy_version 592183 (0.0010) [2023-12-27 09:43:58,567][20381] Updated weights for policy 1, policy_version 591847 (0.0008) [2023-12-27 09:43:58,632][20381] Updated weights for policy 1, policy_version 591857 (0.0007) [2023-12-27 09:43:58,693][20381] Updated weights for policy 1, policy_version 591867 (0.0011) [2023-12-27 09:43:59,019][20347] Updated weights for policy 0, policy_version 592193 (0.0010) [2023-12-27 09:43:59,070][20347] Updated weights for policy 0, policy_version 592203 (0.0009) [2023-12-27 09:43:59,127][20347] Updated weights for policy 0, policy_version 592213 (0.0007) [2023-12-27 09:43:59,189][20347] Updated weights for policy 0, policy_version 592223 (0.0008) [2023-12-27 09:43:59,442][20381] Updated weights for policy 1, policy_version 591877 (0.0009) [2023-12-27 09:43:59,496][20381] Updated weights for policy 1, policy_version 591887 (0.0006) [2023-12-27 09:43:59,550][20381] Updated weights for policy 1, policy_version 591897 (0.0005) [2023-12-27 09:43:59,912][20347] Updated weights for policy 0, policy_version 592233 (0.0008) [2023-12-27 09:43:59,985][20347] Updated weights for policy 0, policy_version 592243 (0.0008) [2023-12-27 09:44:00,054][20347] Updated weights for policy 0, policy_version 592253 (0.0009) [2023-12-27 09:44:00,168][20381] Updated weights for policy 1, policy_version 591907 (0.0006) [2023-12-27 09:44:00,221][20381] Updated weights for policy 1, policy_version 591917 (0.0008) [2023-12-27 09:44:00,274][20381] Updated weights for policy 1, policy_version 591927 (0.0006) [2023-12-27 09:44:00,724][20347] Updated weights for policy 0, policy_version 592263 (0.0007) [2023-12-27 09:44:00,772][20347] Updated weights for policy 0, policy_version 592273 (0.0007) [2023-12-27 09:44:00,818][20347] Updated weights for policy 0, policy_version 592283 (0.0006) [2023-12-27 09:44:00,883][20381] Updated weights for policy 1, policy_version 591937 (0.0006) [2023-12-27 09:44:00,934][20381] Updated weights for policy 1, policy_version 591947 (0.0005) [2023-12-27 09:44:00,987][20381] Updated weights for policy 1, policy_version 591957 (0.0005) [2023-12-27 09:44:01,049][20381] Updated weights for policy 1, policy_version 591967 (0.0008) [2023-12-27 09:44:01,529][20347] Updated weights for policy 0, policy_version 592293 (0.0008) [2023-12-27 09:44:01,584][20347] Updated weights for policy 0, policy_version 592303 (0.0008) [2023-12-27 09:44:01,648][20347] Updated weights for policy 0, policy_version 592313 (0.0007) [2023-12-27 09:44:01,702][20381] Updated weights for policy 1, policy_version 591977 (0.0006) [2023-12-27 09:44:01,761][20381] Updated weights for policy 1, policy_version 591987 (0.0009) [2023-12-27 09:44:01,805][20381] Updated weights for policy 1, policy_version 591997 (0.0007) [2023-12-27 09:44:02,287][20347] Updated weights for policy 0, policy_version 592323 (0.0006) [2023-12-27 09:44:02,346][20347] Updated weights for policy 0, policy_version 592333 (0.0008) [2023-12-27 09:44:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 303226880. Throughput: 0: 10403.7, 1: 10467.5. Samples: 303241688. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:02,363][19170] Avg episode reward: [(0, '712.950'), (1, '922.520')] [2023-12-27 09:44:02,399][20347] Updated weights for policy 0, policy_version 592343 (0.0008) [2023-12-27 09:44:02,496][20381] Updated weights for policy 1, policy_version 592007 (0.0010) [2023-12-27 09:44:02,549][20381] Updated weights for policy 1, policy_version 592018 (0.0010) [2023-12-27 09:44:02,612][20381] Updated weights for policy 1, policy_version 592028 (0.0008) [2023-12-27 09:44:03,017][20347] Updated weights for policy 0, policy_version 592353 (0.0006) [2023-12-27 09:44:03,062][20347] Updated weights for policy 0, policy_version 592363 (0.0008) [2023-12-27 09:44:03,104][20347] Updated weights for policy 0, policy_version 592373 (0.0008) [2023-12-27 09:44:03,153][20347] Updated weights for policy 0, policy_version 592383 (0.0008) [2023-12-27 09:44:03,328][20381] Updated weights for policy 1, policy_version 592038 (0.0008) [2023-12-27 09:44:03,389][20381] Updated weights for policy 1, policy_version 592048 (0.0010) [2023-12-27 09:44:03,442][20381] Updated weights for policy 1, policy_version 592058 (0.0010) [2023-12-27 09:44:03,787][20347] Updated weights for policy 0, policy_version 592393 (0.0007) [2023-12-27 09:44:03,844][20347] Updated weights for policy 0, policy_version 592403 (0.0008) [2023-12-27 09:44:03,903][20347] Updated weights for policy 0, policy_version 592413 (0.0008) [2023-12-27 09:44:04,078][20381] Updated weights for policy 1, policy_version 592068 (0.0010) [2023-12-27 09:44:04,139][20381] Updated weights for policy 1, policy_version 592078 (0.0006) [2023-12-27 09:44:04,203][20381] Updated weights for policy 1, policy_version 592088 (0.0006) [2023-12-27 09:44:04,628][20347] Updated weights for policy 0, policy_version 592423 (0.0007) [2023-12-27 09:44:04,689][20347] Updated weights for policy 0, policy_version 592433 (0.0007) [2023-12-27 09:44:04,749][20347] Updated weights for policy 0, policy_version 592443 (0.0008) [2023-12-27 09:44:04,867][20381] Updated weights for policy 1, policy_version 592098 (0.0010) [2023-12-27 09:44:04,928][20381] Updated weights for policy 1, policy_version 592108 (0.0009) [2023-12-27 09:44:04,979][20381] Updated weights for policy 1, policy_version 592118 (0.0010) [2023-12-27 09:44:05,035][20381] Updated weights for policy 1, policy_version 592128 (0.0011) [2023-12-27 09:44:05,418][20347] Updated weights for policy 0, policy_version 592453 (0.0009) [2023-12-27 09:44:05,478][20347] Updated weights for policy 0, policy_version 592463 (0.0008) [2023-12-27 09:44:05,532][20347] Updated weights for policy 0, policy_version 592473 (0.0006) [2023-12-27 09:44:05,664][20381] Updated weights for policy 1, policy_version 592138 (0.0006) [2023-12-27 09:44:05,722][20381] Updated weights for policy 1, policy_version 592148 (0.0008) [2023-12-27 09:44:05,782][20381] Updated weights for policy 1, policy_version 592158 (0.0008) [2023-12-27 09:44:06,134][20347] Updated weights for policy 0, policy_version 592483 (0.0006) [2023-12-27 09:44:06,192][20347] Updated weights for policy 0, policy_version 592493 (0.0008) [2023-12-27 09:44:06,266][20347] Updated weights for policy 0, policy_version 592503 (0.0008) [2023-12-27 09:44:06,513][20381] Updated weights for policy 1, policy_version 592168 (0.0009) [2023-12-27 09:44:06,574][20381] Updated weights for policy 1, policy_version 592178 (0.0009) [2023-12-27 09:44:06,630][20381] Updated weights for policy 1, policy_version 592188 (0.0011) [2023-12-27 09:44:06,916][20347] Updated weights for policy 0, policy_version 592513 (0.0005) [2023-12-27 09:44:06,982][20347] Updated weights for policy 0, policy_version 592523 (0.0007) [2023-12-27 09:44:07,045][20347] Updated weights for policy 0, policy_version 592533 (0.0006) [2023-12-27 09:44:07,095][20347] Updated weights for policy 0, policy_version 592543 (0.0005) [2023-12-27 09:44:07,282][20381] Updated weights for policy 1, policy_version 592198 (0.0008) [2023-12-27 09:44:07,334][20381] Updated weights for policy 1, policy_version 592208 (0.0005) [2023-12-27 09:44:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 303333376. Throughput: 0: 10436.5, 1: 10469.0. Samples: 303303604. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:07,364][19170] Avg episode reward: [(0, '825.460'), (1, '961.820')] [2023-12-27 09:44:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000592544_151707648.pth... [2023-12-27 09:44:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000591328_151396352.pth [2023-12-27 09:44:07,383][20381] Updated weights for policy 1, policy_version 592218 (0.0008) [2023-12-27 09:44:07,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000592224_151633920.pth... [2023-12-27 09:44:07,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000591024_151322624.pth [2023-12-27 09:44:07,657][20347] Updated weights for policy 0, policy_version 592553 (0.0008) [2023-12-27 09:44:07,717][20347] Updated weights for policy 0, policy_version 592563 (0.0008) [2023-12-27 09:44:07,767][20347] Updated weights for policy 0, policy_version 592573 (0.0009) [2023-12-27 09:44:07,997][20381] Updated weights for policy 1, policy_version 592228 (0.0009) [2023-12-27 09:44:08,053][20381] Updated weights for policy 1, policy_version 592238 (0.0011) [2023-12-27 09:44:08,109][20381] Updated weights for policy 1, policy_version 592248 (0.0011) [2023-12-27 09:44:08,422][20347] Updated weights for policy 0, policy_version 592583 (0.0010) [2023-12-27 09:44:08,486][20347] Updated weights for policy 0, policy_version 592593 (0.0008) [2023-12-27 09:44:08,542][20347] Updated weights for policy 0, policy_version 592603 (0.0005) [2023-12-27 09:44:08,733][20381] Updated weights for policy 1, policy_version 592258 (0.0010) [2023-12-27 09:44:08,786][20381] Updated weights for policy 1, policy_version 592268 (0.0005) [2023-12-27 09:44:08,839][20381] Updated weights for policy 1, policy_version 592278 (0.0005) [2023-12-27 09:44:08,894][20381] Updated weights for policy 1, policy_version 592288 (0.0007) [2023-12-27 09:44:09,140][20347] Updated weights for policy 0, policy_version 592613 (0.0006) [2023-12-27 09:44:09,202][20347] Updated weights for policy 0, policy_version 592623 (0.0005) [2023-12-27 09:44:09,263][20347] Updated weights for policy 0, policy_version 592634 (0.0008) [2023-12-27 09:44:09,568][20381] Updated weights for policy 1, policy_version 592298 (0.0007) [2023-12-27 09:44:09,623][20381] Updated weights for policy 1, policy_version 592308 (0.0007) [2023-12-27 09:44:09,671][20381] Updated weights for policy 1, policy_version 592318 (0.0007) [2023-12-27 09:44:10,006][20347] Updated weights for policy 0, policy_version 592644 (0.0008) [2023-12-27 09:44:10,065][20347] Updated weights for policy 0, policy_version 592654 (0.0006) [2023-12-27 09:44:10,128][20347] Updated weights for policy 0, policy_version 592664 (0.0007) [2023-12-27 09:44:10,340][20381] Updated weights for policy 1, policy_version 592328 (0.0010) [2023-12-27 09:44:10,396][20381] Updated weights for policy 1, policy_version 592338 (0.0010) [2023-12-27 09:44:10,449][20381] Updated weights for policy 1, policy_version 592348 (0.0006) [2023-12-27 09:44:10,787][20347] Updated weights for policy 0, policy_version 592674 (0.0008) [2023-12-27 09:44:10,845][20347] Updated weights for policy 0, policy_version 592684 (0.0005) [2023-12-27 09:44:10,907][20347] Updated weights for policy 0, policy_version 592694 (0.0006) [2023-12-27 09:44:10,965][20347] Updated weights for policy 0, policy_version 592704 (0.0005) [2023-12-27 09:44:11,048][20381] Updated weights for policy 1, policy_version 592358 (0.0009) [2023-12-27 09:44:11,116][20381] Updated weights for policy 1, policy_version 592368 (0.0007) [2023-12-27 09:44:11,179][20381] Updated weights for policy 1, policy_version 592378 (0.0008) [2023-12-27 09:44:11,628][20347] Updated weights for policy 0, policy_version 592714 (0.0008) [2023-12-27 09:44:11,698][20347] Updated weights for policy 0, policy_version 592724 (0.0008) [2023-12-27 09:44:11,757][20347] Updated weights for policy 0, policy_version 592734 (0.0007) [2023-12-27 09:44:11,968][20381] Updated weights for policy 1, policy_version 592388 (0.0009) [2023-12-27 09:44:12,025][20381] Updated weights for policy 1, policy_version 592398 (0.0010) [2023-12-27 09:44:12,083][20381] Updated weights for policy 1, policy_version 592408 (0.0011) [2023-12-27 09:44:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 303439872. Throughput: 0: 10420.9, 1: 10559.8. Samples: 303431456. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:12,363][19170] Avg episode reward: [(0, '825.700'), (1, '961.210')] [2023-12-27 09:44:12,388][20347] Updated weights for policy 0, policy_version 592744 (0.0010) [2023-12-27 09:44:12,451][20347] Updated weights for policy 0, policy_version 592754 (0.0010) [2023-12-27 09:44:12,515][20347] Updated weights for policy 0, policy_version 592764 (0.0006) [2023-12-27 09:44:12,751][20381] Updated weights for policy 1, policy_version 592418 (0.0010) [2023-12-27 09:44:12,809][20381] Updated weights for policy 1, policy_version 592428 (0.0007) [2023-12-27 09:44:12,864][20381] Updated weights for policy 1, policy_version 592438 (0.0010) [2023-12-27 09:44:12,915][20381] Updated weights for policy 1, policy_version 592448 (0.0010) [2023-12-27 09:44:13,168][20347] Updated weights for policy 0, policy_version 592774 (0.0006) [2023-12-27 09:44:13,228][20347] Updated weights for policy 0, policy_version 592784 (0.0009) [2023-12-27 09:44:13,277][20347] Updated weights for policy 0, policy_version 592794 (0.0010) [2023-12-27 09:44:13,550][20381] Updated weights for policy 1, policy_version 592458 (0.0005) [2023-12-27 09:44:13,608][20381] Updated weights for policy 1, policy_version 592468 (0.0005) [2023-12-27 09:44:13,676][20381] Updated weights for policy 1, policy_version 592478 (0.0006) [2023-12-27 09:44:13,912][20347] Updated weights for policy 0, policy_version 592804 (0.0008) [2023-12-27 09:44:13,961][20347] Updated weights for policy 0, policy_version 592814 (0.0005) [2023-12-27 09:44:14,014][20347] Updated weights for policy 0, policy_version 592824 (0.0008) [2023-12-27 09:44:14,212][20381] Updated weights for policy 1, policy_version 592488 (0.0010) [2023-12-27 09:44:14,270][20381] Updated weights for policy 1, policy_version 592498 (0.0010) [2023-12-27 09:44:14,324][20381] Updated weights for policy 1, policy_version 592508 (0.0010) [2023-12-27 09:44:14,625][20347] Updated weights for policy 0, policy_version 592834 (0.0009) [2023-12-27 09:44:14,678][20347] Updated weights for policy 0, policy_version 592844 (0.0006) [2023-12-27 09:44:14,724][20347] Updated weights for policy 0, policy_version 592854 (0.0005) [2023-12-27 09:44:14,788][20347] Updated weights for policy 0, policy_version 592864 (0.0007) [2023-12-27 09:44:15,004][20381] Updated weights for policy 1, policy_version 592518 (0.0007) [2023-12-27 09:44:15,077][20381] Updated weights for policy 1, policy_version 592528 (0.0007) [2023-12-27 09:44:15,146][20381] Updated weights for policy 1, policy_version 592538 (0.0008) [2023-12-27 09:44:15,502][20347] Updated weights for policy 0, policy_version 592874 (0.0007) [2023-12-27 09:44:15,557][20347] Updated weights for policy 0, policy_version 592884 (0.0005) [2023-12-27 09:44:15,612][20347] Updated weights for policy 0, policy_version 592894 (0.0007) [2023-12-27 09:44:15,805][20381] Updated weights for policy 1, policy_version 592548 (0.0011) [2023-12-27 09:44:15,869][20381] Updated weights for policy 1, policy_version 592558 (0.0006) [2023-12-27 09:44:15,932][20381] Updated weights for policy 1, policy_version 592568 (0.0005) [2023-12-27 09:44:16,266][20347] Updated weights for policy 0, policy_version 592904 (0.0006) [2023-12-27 09:44:16,313][20347] Updated weights for policy 0, policy_version 592914 (0.0006) [2023-12-27 09:44:16,365][20347] Updated weights for policy 0, policy_version 592924 (0.0007) [2023-12-27 09:44:16,529][20381] Updated weights for policy 1, policy_version 592578 (0.0006) [2023-12-27 09:44:16,585][20381] Updated weights for policy 1, policy_version 592588 (0.0008) [2023-12-27 09:44:16,642][20381] Updated weights for policy 1, policy_version 592598 (0.0008) [2023-12-27 09:44:16,699][20381] Updated weights for policy 1, policy_version 592608 (0.0008) [2023-12-27 09:44:17,051][20347] Updated weights for policy 0, policy_version 592934 (0.0006) [2023-12-27 09:44:17,102][20347] Updated weights for policy 0, policy_version 592944 (0.0005) [2023-12-27 09:44:17,162][20347] Updated weights for policy 0, policy_version 592954 (0.0006) [2023-12-27 09:44:17,282][20381] Updated weights for policy 1, policy_version 592618 (0.0007) [2023-12-27 09:44:17,335][20381] Updated weights for policy 1, policy_version 592628 (0.0009) [2023-12-27 09:44:17,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 303546368. Throughput: 0: 10463.6, 1: 10494.1. Samples: 303557940. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:17,363][19170] Avg episode reward: [(0, '786.550'), (1, '970.950')] [2023-12-27 09:44:17,391][20381] Updated weights for policy 1, policy_version 592638 (0.0007) [2023-12-27 09:44:17,714][20347] Updated weights for policy 0, policy_version 592964 (0.0005) [2023-12-27 09:44:17,772][20347] Updated weights for policy 0, policy_version 592974 (0.0005) [2023-12-27 09:44:17,833][20347] Updated weights for policy 0, policy_version 592984 (0.0008) [2023-12-27 09:44:18,150][20381] Updated weights for policy 1, policy_version 592648 (0.0008) [2023-12-27 09:44:18,197][20381] Updated weights for policy 1, policy_version 592658 (0.0008) [2023-12-27 09:44:18,247][20381] Updated weights for policy 1, policy_version 592668 (0.0007) [2023-12-27 09:44:18,411][20347] Updated weights for policy 0, policy_version 592994 (0.0007) [2023-12-27 09:44:18,477][20347] Updated weights for policy 0, policy_version 593004 (0.0008) [2023-12-27 09:44:18,542][20347] Updated weights for policy 0, policy_version 593014 (0.0008) [2023-12-27 09:44:18,604][20347] Updated weights for policy 0, policy_version 593024 (0.0008) [2023-12-27 09:44:19,017][20381] Updated weights for policy 1, policy_version 592678 (0.0007) [2023-12-27 09:44:19,074][20381] Updated weights for policy 1, policy_version 592688 (0.0008) [2023-12-27 09:44:19,122][20381] Updated weights for policy 1, policy_version 592698 (0.0008) [2023-12-27 09:44:19,242][20347] Updated weights for policy 0, policy_version 593034 (0.0007) [2023-12-27 09:44:19,294][20347] Updated weights for policy 0, policy_version 593044 (0.0008) [2023-12-27 09:44:19,352][20347] Updated weights for policy 0, policy_version 593054 (0.0007) [2023-12-27 09:44:19,758][20381] Updated weights for policy 1, policy_version 592708 (0.0007) [2023-12-27 09:44:19,820][20381] Updated weights for policy 1, policy_version 592718 (0.0007) [2023-12-27 09:44:19,878][20381] Updated weights for policy 1, policy_version 592728 (0.0008) [2023-12-27 09:44:20,005][20347] Updated weights for policy 0, policy_version 593064 (0.0008) [2023-12-27 09:44:20,056][20347] Updated weights for policy 0, policy_version 593074 (0.0007) [2023-12-27 09:44:20,104][20347] Updated weights for policy 0, policy_version 593084 (0.0008) [2023-12-27 09:44:20,486][20381] Updated weights for policy 1, policy_version 592738 (0.0008) [2023-12-27 09:44:20,548][20381] Updated weights for policy 1, policy_version 592748 (0.0005) [2023-12-27 09:44:20,604][20381] Updated weights for policy 1, policy_version 592758 (0.0008) [2023-12-27 09:44:20,655][20381] Updated weights for policy 1, policy_version 592768 (0.0007) [2023-12-27 09:44:20,849][20347] Updated weights for policy 0, policy_version 593094 (0.0008) [2023-12-27 09:44:20,915][20347] Updated weights for policy 0, policy_version 593104 (0.0007) [2023-12-27 09:44:20,975][20347] Updated weights for policy 0, policy_version 593114 (0.0007) [2023-12-27 09:44:21,347][20381] Updated weights for policy 1, policy_version 592778 (0.0008) [2023-12-27 09:44:21,419][20381] Updated weights for policy 1, policy_version 592788 (0.0007) [2023-12-27 09:44:21,484][20381] Updated weights for policy 1, policy_version 592798 (0.0008) [2023-12-27 09:44:21,710][20347] Updated weights for policy 0, policy_version 593124 (0.0008) [2023-12-27 09:44:21,771][20347] Updated weights for policy 0, policy_version 593134 (0.0006) [2023-12-27 09:44:21,836][20347] Updated weights for policy 0, policy_version 593144 (0.0008) [2023-12-27 09:44:22,184][20381] Updated weights for policy 1, policy_version 592808 (0.0006) [2023-12-27 09:44:22,238][20381] Updated weights for policy 1, policy_version 592818 (0.0007) [2023-12-27 09:44:22,303][20381] Updated weights for policy 1, policy_version 592828 (0.0008) [2023-12-27 09:44:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 303652864. Throughput: 0: 10488.1, 1: 10398.2. Samples: 303620448. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:22,363][19170] Avg episode reward: [(0, '766.730'), (1, '990.350')] [2023-12-27 09:44:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000592832_151789568.pth... [2023-12-27 09:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000593152_151863296.pth... [2023-12-27 09:44:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000591936_151552000.pth [2023-12-27 09:44:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000591616_151478272.pth [2023-12-27 09:44:22,510][20347] Updated weights for policy 0, policy_version 593154 (0.0008) [2023-12-27 09:44:22,570][20347] Updated weights for policy 0, policy_version 593164 (0.0006) [2023-12-27 09:44:22,635][20347] Updated weights for policy 0, policy_version 593174 (0.0006) [2023-12-27 09:44:22,700][20347] Updated weights for policy 0, policy_version 593184 (0.0008) [2023-12-27 09:44:23,077][20381] Updated weights for policy 1, policy_version 592838 (0.0008) [2023-12-27 09:44:23,131][20381] Updated weights for policy 1, policy_version 592848 (0.0009) [2023-12-27 09:44:23,190][20381] Updated weights for policy 1, policy_version 592858 (0.0007) [2023-12-27 09:44:23,457][20347] Updated weights for policy 0, policy_version 593194 (0.0010) [2023-12-27 09:44:23,512][20347] Updated weights for policy 0, policy_version 593204 (0.0010) [2023-12-27 09:44:23,571][20347] Updated weights for policy 0, policy_version 593214 (0.0009) [2023-12-27 09:44:23,876][20381] Updated weights for policy 1, policy_version 592868 (0.0010) [2023-12-27 09:44:23,920][20381] Updated weights for policy 1, policy_version 592878 (0.0010) [2023-12-27 09:44:23,964][20381] Updated weights for policy 1, policy_version 592888 (0.0006) [2023-12-27 09:44:24,099][20347] Updated weights for policy 0, policy_version 593224 (0.0005) [2023-12-27 09:44:24,160][20347] Updated weights for policy 0, policy_version 593234 (0.0005) [2023-12-27 09:44:24,225][20347] Updated weights for policy 0, policy_version 593244 (0.0005) [2023-12-27 09:44:24,562][20381] Updated weights for policy 1, policy_version 592898 (0.0006) [2023-12-27 09:44:24,617][20381] Updated weights for policy 1, policy_version 592908 (0.0010) [2023-12-27 09:44:24,666][20381] Updated weights for policy 1, policy_version 592918 (0.0011) [2023-12-27 09:44:24,724][20381] Updated weights for policy 1, policy_version 592928 (0.0011) [2023-12-27 09:44:24,790][20347] Updated weights for policy 0, policy_version 593254 (0.0008) [2023-12-27 09:44:24,843][20347] Updated weights for policy 0, policy_version 593264 (0.0007) [2023-12-27 09:44:24,899][20347] Updated weights for policy 0, policy_version 593274 (0.0007) [2023-12-27 09:44:25,423][20381] Updated weights for policy 1, policy_version 592938 (0.0006) [2023-12-27 09:44:25,485][20381] Updated weights for policy 1, policy_version 592948 (0.0005) [2023-12-27 09:44:25,533][20381] Updated weights for policy 1, policy_version 592958 (0.0007) [2023-12-27 09:44:25,565][20347] Updated weights for policy 0, policy_version 593284 (0.0007) [2023-12-27 09:44:25,628][20347] Updated weights for policy 0, policy_version 593294 (0.0008) [2023-12-27 09:44:25,686][20347] Updated weights for policy 0, policy_version 593304 (0.0007) [2023-12-27 09:44:26,163][20381] Updated weights for policy 1, policy_version 592968 (0.0008) [2023-12-27 09:44:26,216][20381] Updated weights for policy 1, policy_version 592978 (0.0008) [2023-12-27 09:44:26,272][20381] Updated weights for policy 1, policy_version 592988 (0.0007) [2023-12-27 09:44:26,356][20347] Updated weights for policy 0, policy_version 593314 (0.0007) [2023-12-27 09:44:26,417][20347] Updated weights for policy 0, policy_version 593324 (0.0008) [2023-12-27 09:44:26,463][20347] Updated weights for policy 0, policy_version 593334 (0.0008) [2023-12-27 09:44:26,525][20347] Updated weights for policy 0, policy_version 593344 (0.0008) [2023-12-27 09:44:26,909][20381] Updated weights for policy 1, policy_version 592998 (0.0008) [2023-12-27 09:44:26,962][20381] Updated weights for policy 1, policy_version 593008 (0.0008) [2023-12-27 09:44:27,017][20381] Updated weights for policy 1, policy_version 593018 (0.0010) [2023-12-27 09:44:27,226][20347] Updated weights for policy 0, policy_version 593354 (0.0007) [2023-12-27 09:44:27,272][20347] Updated weights for policy 0, policy_version 593364 (0.0007) [2023-12-27 09:44:27,325][20347] Updated weights for policy 0, policy_version 593374 (0.0007) [2023-12-27 09:44:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20799.3). Total num frames: 303759360. Throughput: 0: 10497.3, 1: 10380.0. Samples: 303743956. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:27,363][19170] Avg episode reward: [(0, '713.990'), (1, '1000.000')] [2023-12-27 09:44:27,629][20381] Updated weights for policy 1, policy_version 593028 (0.0008) [2023-12-27 09:44:27,689][20381] Updated weights for policy 1, policy_version 593038 (0.0005) [2023-12-27 09:44:27,742][20381] Updated weights for policy 1, policy_version 593048 (0.0005) [2023-12-27 09:44:27,925][20347] Updated weights for policy 0, policy_version 593384 (0.0010) [2023-12-27 09:44:27,973][20347] Updated weights for policy 0, policy_version 593394 (0.0010) [2023-12-27 09:44:28,024][20347] Updated weights for policy 0, policy_version 593404 (0.0010) [2023-12-27 09:44:28,355][20381] Updated weights for policy 1, policy_version 593058 (0.0005) [2023-12-27 09:44:28,415][20381] Updated weights for policy 1, policy_version 593068 (0.0007) [2023-12-27 09:44:28,475][20381] Updated weights for policy 1, policy_version 593078 (0.0007) [2023-12-27 09:44:28,531][20381] Updated weights for policy 1, policy_version 593088 (0.0005) [2023-12-27 09:44:28,660][20347] Updated weights for policy 0, policy_version 593414 (0.0007) [2023-12-27 09:44:28,730][20347] Updated weights for policy 0, policy_version 593424 (0.0005) [2023-12-27 09:44:28,798][20347] Updated weights for policy 0, policy_version 593434 (0.0005) [2023-12-27 09:44:29,119][20381] Updated weights for policy 1, policy_version 593098 (0.0007) [2023-12-27 09:44:29,175][20381] Updated weights for policy 1, policy_version 593108 (0.0008) [2023-12-27 09:44:29,230][20381] Updated weights for policy 1, policy_version 593118 (0.0008) [2023-12-27 09:44:29,348][20347] Updated weights for policy 0, policy_version 593444 (0.0007) [2023-12-27 09:44:29,411][20347] Updated weights for policy 0, policy_version 593454 (0.0009) [2023-12-27 09:44:29,466][20347] Updated weights for policy 0, policy_version 593464 (0.0010) [2023-12-27 09:44:29,886][20381] Updated weights for policy 1, policy_version 593128 (0.0008) [2023-12-27 09:44:29,946][20381] Updated weights for policy 1, policy_version 593138 (0.0008) [2023-12-27 09:44:29,998][20381] Updated weights for policy 1, policy_version 593148 (0.0009) [2023-12-27 09:44:30,115][20347] Updated weights for policy 0, policy_version 593474 (0.0009) [2023-12-27 09:44:30,171][20347] Updated weights for policy 0, policy_version 593484 (0.0010) [2023-12-27 09:44:30,229][20347] Updated weights for policy 0, policy_version 593494 (0.0010) [2023-12-27 09:44:30,286][20347] Updated weights for policy 0, policy_version 593504 (0.0010) [2023-12-27 09:44:30,612][20381] Updated weights for policy 1, policy_version 593158 (0.0007) [2023-12-27 09:44:30,677][20381] Updated weights for policy 1, policy_version 593168 (0.0005) [2023-12-27 09:44:30,772][20381] Updated weights for policy 1, policy_version 593178 (0.0005) [2023-12-27 09:44:30,886][20347] Updated weights for policy 0, policy_version 593514 (0.0005) [2023-12-27 09:44:30,944][20347] Updated weights for policy 0, policy_version 593524 (0.0007) [2023-12-27 09:44:31,000][20347] Updated weights for policy 0, policy_version 593534 (0.0008) [2023-12-27 09:44:31,349][20381] Updated weights for policy 1, policy_version 593188 (0.0006) [2023-12-27 09:44:31,415][20381] Updated weights for policy 1, policy_version 593198 (0.0010) [2023-12-27 09:44:31,473][20381] Updated weights for policy 1, policy_version 593208 (0.0008) [2023-12-27 09:44:31,641][20347] Updated weights for policy 0, policy_version 593544 (0.0008) [2023-12-27 09:44:31,699][20347] Updated weights for policy 0, policy_version 593554 (0.0008) [2023-12-27 09:44:31,764][20347] Updated weights for policy 0, policy_version 593564 (0.0008) [2023-12-27 09:44:32,173][20381] Updated weights for policy 1, policy_version 593218 (0.0006) [2023-12-27 09:44:32,231][20381] Updated weights for policy 1, policy_version 593228 (0.0006) [2023-12-27 09:44:32,290][20381] Updated weights for policy 1, policy_version 593238 (0.0008) [2023-12-27 09:44:32,346][20381] Updated weights for policy 1, policy_version 593248 (0.0006) [2023-12-27 09:44:32,363][19170] Fps is (10 sec: 21297.2, 60 sec: 21025.8, 300 sec: 20799.3). Total num frames: 303865856. Throughput: 0: 10552.4, 1: 10435.4. Samples: 303874764. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:32,364][19170] Avg episode reward: [(0, '540.390'), (1, '990.340')] [2023-12-27 09:44:32,408][20347] Updated weights for policy 0, policy_version 593574 (0.0008) [2023-12-27 09:44:32,470][20347] Updated weights for policy 0, policy_version 593584 (0.0007) [2023-12-27 09:44:32,532][20347] Updated weights for policy 0, policy_version 593594 (0.0008) [2023-12-27 09:44:32,959][20381] Updated weights for policy 1, policy_version 593258 (0.0006) [2023-12-27 09:44:33,016][20381] Updated weights for policy 1, policy_version 593268 (0.0006) [2023-12-27 09:44:33,078][20381] Updated weights for policy 1, policy_version 593278 (0.0005) [2023-12-27 09:44:33,230][20347] Updated weights for policy 0, policy_version 593604 (0.0008) [2023-12-27 09:44:33,282][20347] Updated weights for policy 0, policy_version 593614 (0.0008) [2023-12-27 09:44:33,329][20347] Updated weights for policy 0, policy_version 593624 (0.0005) [2023-12-27 09:44:33,726][20381] Updated weights for policy 1, policy_version 593288 (0.0009) [2023-12-27 09:44:33,773][20381] Updated weights for policy 1, policy_version 593298 (0.0010) [2023-12-27 09:44:33,818][20381] Updated weights for policy 1, policy_version 593308 (0.0010) [2023-12-27 09:44:33,894][20347] Updated weights for policy 0, policy_version 593634 (0.0006) [2023-12-27 09:44:33,948][20347] Updated weights for policy 0, policy_version 593644 (0.0007) [2023-12-27 09:44:34,001][20347] Updated weights for policy 0, policy_version 593654 (0.0007) [2023-12-27 09:44:34,044][20347] Updated weights for policy 0, policy_version 593664 (0.0006) [2023-12-27 09:44:34,539][20381] Updated weights for policy 1, policy_version 593318 (0.0009) [2023-12-27 09:44:34,599][20381] Updated weights for policy 1, policy_version 593328 (0.0008) [2023-12-27 09:44:34,624][20292] KL-divergence is very high: 135.2928 [2023-12-27 09:44:34,662][20381] Updated weights for policy 1, policy_version 593338 (0.0007) [2023-12-27 09:44:34,713][20347] Updated weights for policy 0, policy_version 593674 (0.0008) [2023-12-27 09:44:34,766][20347] Updated weights for policy 0, policy_version 593684 (0.0009) [2023-12-27 09:44:34,820][20347] Updated weights for policy 0, policy_version 593694 (0.0008) [2023-12-27 09:44:35,268][20381] Updated weights for policy 1, policy_version 593348 (0.0007) [2023-12-27 09:44:35,318][20381] Updated weights for policy 1, policy_version 593358 (0.0008) [2023-12-27 09:44:35,366][20381] Updated weights for policy 1, policy_version 593368 (0.0008) [2023-12-27 09:44:35,546][20347] Updated weights for policy 0, policy_version 593704 (0.0007) [2023-12-27 09:44:35,602][20347] Updated weights for policy 0, policy_version 593714 (0.0008) [2023-12-27 09:44:35,654][20347] Updated weights for policy 0, policy_version 593724 (0.0008) [2023-12-27 09:44:35,979][20381] Updated weights for policy 1, policy_version 593378 (0.0008) [2023-12-27 09:44:36,021][20381] Updated weights for policy 1, policy_version 593388 (0.0008) [2023-12-27 09:44:36,080][20381] Updated weights for policy 1, policy_version 593398 (0.0005) [2023-12-27 09:44:36,134][20381] Updated weights for policy 1, policy_version 593408 (0.0007) [2023-12-27 09:44:36,449][20347] Updated weights for policy 0, policy_version 593734 (0.0009) [2023-12-27 09:44:36,506][20347] Updated weights for policy 0, policy_version 593744 (0.0008) [2023-12-27 09:44:36,563][20347] Updated weights for policy 0, policy_version 593754 (0.0008) [2023-12-27 09:44:36,732][20381] Updated weights for policy 1, policy_version 593418 (0.0008) [2023-12-27 09:44:36,796][20381] Updated weights for policy 1, policy_version 593428 (0.0008) [2023-12-27 09:44:36,854][20381] Updated weights for policy 1, policy_version 593438 (0.0007) [2023-12-27 09:44:37,314][20347] Updated weights for policy 0, policy_version 593764 (0.0010) [2023-12-27 09:44:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 303964160. Throughput: 0: 10541.5, 1: 10417.9. Samples: 303937860. Policy #0 lag: (min: 22.0, avg: 24.3, max: 54.0) [2023-12-27 09:44:37,363][19170] Avg episode reward: [(0, '323.650'), (1, '971.250')] [2023-12-27 09:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000593440_151945216.pth... [2023-12-27 09:44:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000592224_151633920.pth [2023-12-27 09:44:37,378][20347] Updated weights for policy 0, policy_version 593774 (0.0006) [2023-12-27 09:44:37,432][20347] Updated weights for policy 0, policy_version 593784 (0.0008) [2023-12-27 09:44:37,487][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000593792_152027136.pth... [2023-12-27 09:44:37,491][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000592544_151707648.pth [2023-12-27 09:44:37,508][20381] Updated weights for policy 1, policy_version 593448 (0.0006) [2023-12-27 09:44:37,540][20292] KL-divergence is very high: 800.0331 [2023-12-27 09:44:37,547][20292] KL-divergence is very high: 518.2620 [2023-12-27 09:44:37,571][20381] Updated weights for policy 1, policy_version 593458 (0.0005) [2023-12-27 09:44:37,637][20381] Updated weights for policy 1, policy_version 593468 (0.0005) [2023-12-27 09:44:38,100][20347] Updated weights for policy 0, policy_version 593794 (0.0008) [2023-12-27 09:44:38,139][20381] Updated weights for policy 1, policy_version 593478 (0.0005) [2023-12-27 09:44:38,152][20347] Updated weights for policy 0, policy_version 593804 (0.0010) [2023-12-27 09:44:38,202][20381] Updated weights for policy 1, policy_version 593488 (0.0006) [2023-12-27 09:44:38,206][20347] Updated weights for policy 0, policy_version 593814 (0.0010) [2023-12-27 09:44:38,258][20347] Updated weights for policy 0, policy_version 593824 (0.0010) [2023-12-27 09:44:38,258][20381] Updated weights for policy 1, policy_version 593498 (0.0006) [2023-12-27 09:44:38,818][20381] Updated weights for policy 1, policy_version 593508 (0.0005) [2023-12-27 09:44:38,879][20381] Updated weights for policy 1, policy_version 593518 (0.0005) [2023-12-27 09:44:38,937][20381] Updated weights for policy 1, policy_version 593528 (0.0006) [2023-12-27 09:44:38,947][20347] Updated weights for policy 0, policy_version 593834 (0.0009) [2023-12-27 09:44:38,996][20347] Updated weights for policy 0, policy_version 593844 (0.0007) [2023-12-27 09:44:39,047][20347] Updated weights for policy 0, policy_version 593854 (0.0007) [2023-12-27 09:44:39,652][20381] Updated weights for policy 1, policy_version 593538 (0.0008) [2023-12-27 09:44:39,709][20381] Updated weights for policy 1, policy_version 593548 (0.0008) [2023-12-27 09:44:39,710][20292] KL-divergence is very high: 157.7338 [2023-12-27 09:44:39,712][20347] Updated weights for policy 0, policy_version 593864 (0.0006) [2023-12-27 09:44:39,765][20381] Updated weights for policy 1, policy_version 593558 (0.0006) [2023-12-27 09:44:39,767][20347] Updated weights for policy 0, policy_version 593874 (0.0006) [2023-12-27 09:44:39,820][20381] Updated weights for policy 1, policy_version 593568 (0.0006) [2023-12-27 09:44:39,825][20347] Updated weights for policy 0, policy_version 593884 (0.0007) [2023-12-27 09:44:40,512][20381] Updated weights for policy 1, policy_version 593578 (0.0008) [2023-12-27 09:44:40,518][20292] KL-divergence is very high: 557.0500 [2023-12-27 09:44:40,542][20292] KL-divergence is very high: 328.8335 [2023-12-27 09:44:40,560][20347] Updated weights for policy 0, policy_version 593894 (0.0007) [2023-12-27 09:44:40,569][20381] Updated weights for policy 1, policy_version 593588 (0.0008) [2023-12-27 09:44:40,616][20347] Updated weights for policy 0, policy_version 593904 (0.0006) [2023-12-27 09:44:40,622][20381] Updated weights for policy 1, policy_version 593598 (0.0006) [2023-12-27 09:44:40,671][20347] Updated weights for policy 0, policy_version 593914 (0.0008) [2023-12-27 09:44:41,352][20347] Updated weights for policy 0, policy_version 593924 (0.0008) [2023-12-27 09:44:41,354][20381] Updated weights for policy 1, policy_version 593608 (0.0007) [2023-12-27 09:44:41,416][20347] Updated weights for policy 0, policy_version 593934 (0.0008) [2023-12-27 09:44:41,417][20381] Updated weights for policy 1, policy_version 593618 (0.0008) [2023-12-27 09:44:41,467][20381] Updated weights for policy 1, policy_version 593628 (0.0008) [2023-12-27 09:44:41,479][20347] Updated weights for policy 0, policy_version 593944 (0.0006) [2023-12-27 09:44:42,132][20347] Updated weights for policy 0, policy_version 593954 (0.0008) [2023-12-27 09:44:42,183][20381] Updated weights for policy 1, policy_version 593638 (0.0007) [2023-12-27 09:44:42,192][20347] Updated weights for policy 0, policy_version 593964 (0.0007) [2023-12-27 09:44:42,242][20381] Updated weights for policy 1, policy_version 593648 (0.0008) [2023-12-27 09:44:42,248][20347] Updated weights for policy 0, policy_version 593974 (0.0008) [2023-12-27 09:44:42,302][20381] Updated weights for policy 1, policy_version 593658 (0.0008) [2023-12-27 09:44:42,311][20347] Updated weights for policy 0, policy_version 593984 (0.0007) [2023-12-27 09:44:42,362][19170] Fps is (10 sec: 21301.4, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 304078848. Throughput: 0: 10483.9, 1: 10518.9. Samples: 304062616. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:44:42,363][19170] Avg episode reward: [(0, '472.170'), (1, '951.620')] [2023-12-27 09:44:42,917][20381] Updated weights for policy 1, policy_version 593668 (0.0007) [2023-12-27 09:44:42,971][20381] Updated weights for policy 1, policy_version 593678 (0.0007) [2023-12-27 09:44:43,004][20347] Updated weights for policy 0, policy_version 593994 (0.0007) [2023-12-27 09:44:43,027][20381] Updated weights for policy 1, policy_version 593688 (0.0005) [2023-12-27 09:44:43,059][20347] Updated weights for policy 0, policy_version 594004 (0.0007) [2023-12-27 09:44:43,115][20347] Updated weights for policy 0, policy_version 594014 (0.0007) [2023-12-27 09:44:43,678][20347] Updated weights for policy 0, policy_version 594024 (0.0008) [2023-12-27 09:44:43,693][20381] Updated weights for policy 1, policy_version 593698 (0.0006) [2023-12-27 09:44:43,724][20347] Updated weights for policy 0, policy_version 594034 (0.0006) [2023-12-27 09:44:43,751][20381] Updated weights for policy 1, policy_version 593708 (0.0010) [2023-12-27 09:44:43,772][20347] Updated weights for policy 0, policy_version 594044 (0.0010) [2023-12-27 09:44:43,805][20381] Updated weights for policy 1, policy_version 593718 (0.0010) [2023-12-27 09:44:43,866][20381] Updated weights for policy 1, policy_version 593728 (0.0010) [2023-12-27 09:44:44,428][20347] Updated weights for policy 0, policy_version 594054 (0.0010) [2023-12-27 09:44:44,481][20347] Updated weights for policy 0, policy_version 594064 (0.0010) [2023-12-27 09:44:44,497][20381] Updated weights for policy 1, policy_version 593738 (0.0010) [2023-12-27 09:44:44,530][20347] Updated weights for policy 0, policy_version 594074 (0.0010) [2023-12-27 09:44:44,554][20381] Updated weights for policy 1, policy_version 593748 (0.0010) [2023-12-27 09:44:44,611][20381] Updated weights for policy 1, policy_version 593758 (0.0007) [2023-12-27 09:44:45,183][20347] Updated weights for policy 0, policy_version 594084 (0.0010) [2023-12-27 09:44:45,238][20347] Updated weights for policy 0, policy_version 594094 (0.0010) [2023-12-27 09:44:45,265][20381] Updated weights for policy 1, policy_version 593768 (0.0010) [2023-12-27 09:44:45,290][20347] Updated weights for policy 0, policy_version 594104 (0.0010) [2023-12-27 09:44:45,314][20381] Updated weights for policy 1, policy_version 593778 (0.0010) [2023-12-27 09:44:45,369][20381] Updated weights for policy 1, policy_version 593788 (0.0010) [2023-12-27 09:44:45,914][20347] Updated weights for policy 0, policy_version 594114 (0.0009) [2023-12-27 09:44:45,979][20347] Updated weights for policy 0, policy_version 594124 (0.0006) [2023-12-27 09:44:46,013][20381] Updated weights for policy 1, policy_version 593798 (0.0007) [2023-12-27 09:44:46,033][20347] Updated weights for policy 0, policy_version 594134 (0.0007) [2023-12-27 09:44:46,082][20381] Updated weights for policy 1, policy_version 593808 (0.0006) [2023-12-27 09:44:46,091][20347] Updated weights for policy 0, policy_version 594144 (0.0005) [2023-12-27 09:44:46,143][20381] Updated weights for policy 1, policy_version 593818 (0.0006) [2023-12-27 09:44:46,689][20381] Updated weights for policy 1, policy_version 593828 (0.0006) [2023-12-27 09:44:46,710][20347] Updated weights for policy 0, policy_version 594154 (0.0010) [2023-12-27 09:44:46,741][20381] Updated weights for policy 1, policy_version 593838 (0.0006) [2023-12-27 09:44:46,761][20347] Updated weights for policy 0, policy_version 594164 (0.0010) [2023-12-27 09:44:46,785][20381] Updated weights for policy 1, policy_version 593848 (0.0005) [2023-12-27 09:44:46,812][20347] Updated weights for policy 0, policy_version 594174 (0.0010) [2023-12-27 09:44:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 304177152. Throughput: 0: 10525.4, 1: 10553.9. Samples: 304190260. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:44:47,363][19170] Avg episode reward: [(0, '452.700'), (1, '941.640')] [2023-12-27 09:44:47,448][20347] Updated weights for policy 0, policy_version 594184 (0.0010) [2023-12-27 09:44:47,495][20381] Updated weights for policy 1, policy_version 593858 (0.0005) [2023-12-27 09:44:47,512][20347] Updated weights for policy 0, policy_version 594194 (0.0012) [2023-12-27 09:44:47,549][20381] Updated weights for policy 1, policy_version 593868 (0.0010) [2023-12-27 09:44:47,564][20347] Updated weights for policy 0, policy_version 594204 (0.0010) [2023-12-27 09:44:47,609][20381] Updated weights for policy 1, policy_version 593878 (0.0010) [2023-12-27 09:44:47,663][20381] Updated weights for policy 1, policy_version 593888 (0.0009) [2023-12-27 09:44:48,136][20347] Updated weights for policy 0, policy_version 594214 (0.0010) [2023-12-27 09:44:48,195][20347] Updated weights for policy 0, policy_version 594224 (0.0007) [2023-12-27 09:44:48,261][20347] Updated weights for policy 0, policy_version 594234 (0.0006) [2023-12-27 09:44:48,318][20381] Updated weights for policy 1, policy_version 593898 (0.0007) [2023-12-27 09:44:48,377][20381] Updated weights for policy 1, policy_version 593908 (0.0007) [2023-12-27 09:44:48,428][20381] Updated weights for policy 1, policy_version 593918 (0.0008) [2023-12-27 09:44:48,863][20347] Updated weights for policy 0, policy_version 594244 (0.0005) [2023-12-27 09:44:48,925][20347] Updated weights for policy 0, policy_version 594254 (0.0006) [2023-12-27 09:44:48,987][20347] Updated weights for policy 0, policy_version 594264 (0.0007) [2023-12-27 09:44:49,004][20381] Updated weights for policy 1, policy_version 593928 (0.0007) [2023-12-27 09:44:49,064][20381] Updated weights for policy 1, policy_version 593938 (0.0006) [2023-12-27 09:44:49,128][20381] Updated weights for policy 1, policy_version 593948 (0.0005) [2023-12-27 09:44:49,605][20347] Updated weights for policy 0, policy_version 594274 (0.0007) [2023-12-27 09:44:49,668][20347] Updated weights for policy 0, policy_version 594284 (0.0008) [2023-12-27 09:44:49,699][20381] Updated weights for policy 1, policy_version 593958 (0.0007) [2023-12-27 09:44:49,730][20347] Updated weights for policy 0, policy_version 594294 (0.0005) [2023-12-27 09:44:49,746][20381] Updated weights for policy 1, policy_version 593968 (0.0008) [2023-12-27 09:44:49,785][20347] Updated weights for policy 0, policy_version 594304 (0.0005) [2023-12-27 09:44:49,799][20381] Updated weights for policy 1, policy_version 593978 (0.0007) [2023-12-27 09:44:50,432][20347] Updated weights for policy 0, policy_version 594314 (0.0005) [2023-12-27 09:44:50,482][20347] Updated weights for policy 0, policy_version 594324 (0.0008) [2023-12-27 09:44:50,528][20381] Updated weights for policy 1, policy_version 593988 (0.0008) [2023-12-27 09:44:50,537][20347] Updated weights for policy 0, policy_version 594334 (0.0007) [2023-12-27 09:44:50,587][20381] Updated weights for policy 1, policy_version 593998 (0.0008) [2023-12-27 09:44:50,645][20381] Updated weights for policy 1, policy_version 594008 (0.0006) [2023-12-27 09:44:51,155][20347] Updated weights for policy 0, policy_version 594344 (0.0006) [2023-12-27 09:44:51,220][20347] Updated weights for policy 0, policy_version 594354 (0.0006) [2023-12-27 09:44:51,289][20347] Updated weights for policy 0, policy_version 594364 (0.0010) [2023-12-27 09:44:51,314][20381] Updated weights for policy 1, policy_version 594018 (0.0006) [2023-12-27 09:44:51,371][20381] Updated weights for policy 1, policy_version 594028 (0.0007) [2023-12-27 09:44:51,432][20381] Updated weights for policy 1, policy_version 594038 (0.0007) [2023-12-27 09:44:51,486][20381] Updated weights for policy 1, policy_version 594048 (0.0010) [2023-12-27 09:44:51,941][20347] Updated weights for policy 0, policy_version 594374 (0.0009) [2023-12-27 09:44:52,006][20347] Updated weights for policy 0, policy_version 594384 (0.0007) [2023-12-27 09:44:52,065][20347] Updated weights for policy 0, policy_version 594394 (0.0008) [2023-12-27 09:44:52,138][20381] Updated weights for policy 1, policy_version 594058 (0.0008) [2023-12-27 09:44:52,198][20381] Updated weights for policy 1, policy_version 594068 (0.0008) [2023-12-27 09:44:52,254][20381] Updated weights for policy 1, policy_version 594078 (0.0008) [2023-12-27 09:44:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 304291840. Throughput: 0: 10573.5, 1: 10588.8. Samples: 304255900. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:44:52,363][19170] Avg episode reward: [(0, '152.910'), (1, '941.840')] [2023-12-27 09:44:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000594080_152109056.pth... [2023-12-27 09:44:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000594400_152182784.pth... [2023-12-27 09:44:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000592832_151789568.pth [2023-12-27 09:44:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000593152_151863296.pth [2023-12-27 09:44:52,817][20347] Updated weights for policy 0, policy_version 594404 (0.0007) [2023-12-27 09:44:52,867][20347] Updated weights for policy 0, policy_version 594414 (0.0009) [2023-12-27 09:44:52,918][20381] Updated weights for policy 1, policy_version 594088 (0.0006) [2023-12-27 09:44:52,924][20347] Updated weights for policy 0, policy_version 594424 (0.0008) [2023-12-27 09:44:52,974][20381] Updated weights for policy 1, policy_version 594098 (0.0005) [2023-12-27 09:44:53,027][20381] Updated weights for policy 1, policy_version 594108 (0.0008) [2023-12-27 09:44:53,594][20347] Updated weights for policy 0, policy_version 594434 (0.0007) [2023-12-27 09:44:53,650][20347] Updated weights for policy 0, policy_version 594444 (0.0010) [2023-12-27 09:44:53,696][20381] Updated weights for policy 1, policy_version 594118 (0.0006) [2023-12-27 09:44:53,712][20347] Updated weights for policy 0, policy_version 594454 (0.0010) [2023-12-27 09:44:53,750][20381] Updated weights for policy 1, policy_version 594128 (0.0006) [2023-12-27 09:44:53,759][20347] Updated weights for policy 0, policy_version 594464 (0.0009) [2023-12-27 09:44:53,800][20381] Updated weights for policy 1, policy_version 594138 (0.0008) [2023-12-27 09:44:54,408][20347] Updated weights for policy 0, policy_version 594474 (0.0009) [2023-12-27 09:44:54,420][20381] Updated weights for policy 1, policy_version 594148 (0.0006) [2023-12-27 09:44:54,466][20347] Updated weights for policy 0, policy_version 594484 (0.0006) [2023-12-27 09:44:54,480][20381] Updated weights for policy 1, policy_version 594158 (0.0009) [2023-12-27 09:44:54,529][20347] Updated weights for policy 0, policy_version 594494 (0.0005) [2023-12-27 09:44:54,532][20381] Updated weights for policy 1, policy_version 594168 (0.0010) [2023-12-27 09:44:55,151][20347] Updated weights for policy 0, policy_version 594504 (0.0007) [2023-12-27 09:44:55,203][20381] Updated weights for policy 1, policy_version 594178 (0.0010) [2023-12-27 09:44:55,208][20347] Updated weights for policy 0, policy_version 594514 (0.0007) [2023-12-27 09:44:55,260][20381] Updated weights for policy 1, policy_version 594188 (0.0010) [2023-12-27 09:44:55,262][20347] Updated weights for policy 0, policy_version 594524 (0.0005) [2023-12-27 09:44:55,314][20381] Updated weights for policy 1, policy_version 594198 (0.0010) [2023-12-27 09:44:55,384][20381] Updated weights for policy 1, policy_version 594208 (0.0010) [2023-12-27 09:44:55,837][20347] Updated weights for policy 0, policy_version 594534 (0.0006) [2023-12-27 09:44:55,888][20347] Updated weights for policy 0, policy_version 594544 (0.0005) [2023-12-27 09:44:55,944][20347] Updated weights for policy 0, policy_version 594554 (0.0005) [2023-12-27 09:44:56,054][20381] Updated weights for policy 1, policy_version 594218 (0.0011) [2023-12-27 09:44:56,111][20381] Updated weights for policy 1, policy_version 594228 (0.0008) [2023-12-27 09:44:56,161][20381] Updated weights for policy 1, policy_version 594238 (0.0008) [2023-12-27 09:44:56,586][20347] Updated weights for policy 0, policy_version 594564 (0.0007) [2023-12-27 09:44:56,634][20347] Updated weights for policy 0, policy_version 594574 (0.0008) [2023-12-27 09:44:56,689][20347] Updated weights for policy 0, policy_version 594584 (0.0006) [2023-12-27 09:44:56,833][20381] Updated weights for policy 1, policy_version 594248 (0.0010) [2023-12-27 09:44:56,879][20381] Updated weights for policy 1, policy_version 594258 (0.0009) [2023-12-27 09:44:56,923][20381] Updated weights for policy 1, policy_version 594268 (0.0010) [2023-12-27 09:44:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 304390144. Throughput: 0: 10546.0, 1: 10578.5. Samples: 304382060. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:44:57,363][19170] Avg episode reward: [(0, '254.910'), (1, '961.280')] [2023-12-27 09:44:57,376][20347] Updated weights for policy 0, policy_version 594594 (0.0006) [2023-12-27 09:44:57,431][20347] Updated weights for policy 0, policy_version 594604 (0.0008) [2023-12-27 09:44:57,487][20347] Updated weights for policy 0, policy_version 594614 (0.0008) [2023-12-27 09:44:57,545][20347] Updated weights for policy 0, policy_version 594624 (0.0007) [2023-12-27 09:44:57,576][20381] Updated weights for policy 1, policy_version 594278 (0.0009) [2023-12-27 09:44:57,639][20381] Updated weights for policy 1, policy_version 594288 (0.0008) [2023-12-27 09:44:57,700][20381] Updated weights for policy 1, policy_version 594298 (0.0008) [2023-12-27 09:44:58,133][20347] Updated weights for policy 0, policy_version 594634 (0.0006) [2023-12-27 09:44:58,195][20347] Updated weights for policy 0, policy_version 594644 (0.0006) [2023-12-27 09:44:58,251][20347] Updated weights for policy 0, policy_version 594654 (0.0008) [2023-12-27 09:44:58,441][20381] Updated weights for policy 1, policy_version 594308 (0.0007) [2023-12-27 09:44:58,500][20381] Updated weights for policy 1, policy_version 594318 (0.0006) [2023-12-27 09:44:58,563][20381] Updated weights for policy 1, policy_version 594328 (0.0006) [2023-12-27 09:44:58,993][20347] Updated weights for policy 0, policy_version 594664 (0.0007) [2023-12-27 09:44:59,049][20347] Updated weights for policy 0, policy_version 594674 (0.0006) [2023-12-27 09:44:59,099][20347] Updated weights for policy 0, policy_version 594684 (0.0005) [2023-12-27 09:44:59,219][20381] Updated weights for policy 1, policy_version 594338 (0.0006) [2023-12-27 09:44:59,275][20381] Updated weights for policy 1, policy_version 594348 (0.0009) [2023-12-27 09:44:59,333][20381] Updated weights for policy 1, policy_version 594358 (0.0008) [2023-12-27 09:44:59,398][20381] Updated weights for policy 1, policy_version 594368 (0.0008) [2023-12-27 09:44:59,750][20347] Updated weights for policy 0, policy_version 594694 (0.0008) [2023-12-27 09:44:59,811][20347] Updated weights for policy 0, policy_version 594704 (0.0010) [2023-12-27 09:44:59,869][20347] Updated weights for policy 0, policy_version 594714 (0.0011) [2023-12-27 09:45:00,092][20381] Updated weights for policy 1, policy_version 594378 (0.0006) [2023-12-27 09:45:00,143][20381] Updated weights for policy 1, policy_version 594388 (0.0006) [2023-12-27 09:45:00,203][20381] Updated weights for policy 1, policy_version 594398 (0.0007) [2023-12-27 09:45:00,564][20347] Updated weights for policy 0, policy_version 594724 (0.0009) [2023-12-27 09:45:00,611][20347] Updated weights for policy 0, policy_version 594734 (0.0006) [2023-12-27 09:45:00,661][20347] Updated weights for policy 0, policy_version 594744 (0.0005) [2023-12-27 09:45:00,835][20381] Updated weights for policy 1, policy_version 594408 (0.0006) [2023-12-27 09:45:00,887][20381] Updated weights for policy 1, policy_version 594418 (0.0005) [2023-12-27 09:45:00,937][20381] Updated weights for policy 1, policy_version 594428 (0.0005) [2023-12-27 09:45:01,296][20347] Updated weights for policy 0, policy_version 594754 (0.0006) [2023-12-27 09:45:01,361][20347] Updated weights for policy 0, policy_version 594764 (0.0007) [2023-12-27 09:45:01,421][20347] Updated weights for policy 0, policy_version 594774 (0.0009) [2023-12-27 09:45:01,465][20347] Updated weights for policy 0, policy_version 594784 (0.0008) [2023-12-27 09:45:01,565][20381] Updated weights for policy 1, policy_version 594438 (0.0007) [2023-12-27 09:45:01,626][20381] Updated weights for policy 1, policy_version 594448 (0.0008) [2023-12-27 09:45:01,687][20381] Updated weights for policy 1, policy_version 594458 (0.0008) [2023-12-27 09:45:02,121][20347] Updated weights for policy 0, policy_version 594794 (0.0006) [2023-12-27 09:45:02,170][20347] Updated weights for policy 0, policy_version 594804 (0.0005) [2023-12-27 09:45:02,219][20347] Updated weights for policy 0, policy_version 594814 (0.0008) [2023-12-27 09:45:02,286][20381] Updated weights for policy 1, policy_version 594468 (0.0008) [2023-12-27 09:45:02,345][20381] Updated weights for policy 1, policy_version 594478 (0.0008) [2023-12-27 09:45:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 21162.7, 300 sec: 20827.1). Total num frames: 304496640. Throughput: 0: 10545.0, 1: 10563.2. Samples: 304507808. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:02,363][19170] Avg episode reward: [(0, '227.740'), (1, '990.330')] [2023-12-27 09:45:02,399][20381] Updated weights for policy 1, policy_version 594488 (0.0010) [2023-12-27 09:45:02,809][20347] Updated weights for policy 0, policy_version 594824 (0.0006) [2023-12-27 09:45:02,872][20347] Updated weights for policy 0, policy_version 594834 (0.0007) [2023-12-27 09:45:02,934][20347] Updated weights for policy 0, policy_version 594844 (0.0007) [2023-12-27 09:45:02,965][20381] Updated weights for policy 1, policy_version 594498 (0.0010) [2023-12-27 09:45:03,022][20381] Updated weights for policy 1, policy_version 594508 (0.0010) [2023-12-27 09:45:03,079][20381] Updated weights for policy 1, policy_version 594518 (0.0010) [2023-12-27 09:45:03,132][20381] Updated weights for policy 1, policy_version 594528 (0.0010) [2023-12-27 09:45:03,491][20347] Updated weights for policy 0, policy_version 594854 (0.0006) [2023-12-27 09:45:03,534][20347] Updated weights for policy 0, policy_version 594864 (0.0005) [2023-12-27 09:45:03,586][20347] Updated weights for policy 0, policy_version 594874 (0.0005) [2023-12-27 09:45:03,773][20381] Updated weights for policy 1, policy_version 594538 (0.0008) [2023-12-27 09:45:03,829][20381] Updated weights for policy 1, policy_version 594548 (0.0010) [2023-12-27 09:45:03,899][20381] Updated weights for policy 1, policy_version 594558 (0.0009) [2023-12-27 09:45:04,131][20347] Updated weights for policy 0, policy_version 594884 (0.0007) [2023-12-27 09:45:04,190][20347] Updated weights for policy 0, policy_version 594894 (0.0010) [2023-12-27 09:45:04,250][20347] Updated weights for policy 0, policy_version 594904 (0.0010) [2023-12-27 09:45:04,580][20381] Updated weights for policy 1, policy_version 594568 (0.0006) [2023-12-27 09:45:04,632][20381] Updated weights for policy 1, policy_version 594578 (0.0006) [2023-12-27 09:45:04,680][20381] Updated weights for policy 1, policy_version 594588 (0.0008) [2023-12-27 09:45:04,872][20347] Updated weights for policy 0, policy_version 594914 (0.0006) [2023-12-27 09:45:04,919][20347] Updated weights for policy 0, policy_version 594924 (0.0007) [2023-12-27 09:45:04,971][20347] Updated weights for policy 0, policy_version 594934 (0.0007) [2023-12-27 09:45:05,030][20347] Updated weights for policy 0, policy_version 594944 (0.0007) [2023-12-27 09:45:05,361][20381] Updated weights for policy 1, policy_version 594598 (0.0007) [2023-12-27 09:45:05,418][20381] Updated weights for policy 1, policy_version 594608 (0.0007) [2023-12-27 09:45:05,472][20381] Updated weights for policy 1, policy_version 594618 (0.0010) [2023-12-27 09:45:05,684][20347] Updated weights for policy 0, policy_version 594954 (0.0010) [2023-12-27 09:45:05,731][20347] Updated weights for policy 0, policy_version 594964 (0.0007) [2023-12-27 09:45:05,781][20347] Updated weights for policy 0, policy_version 594974 (0.0005) [2023-12-27 09:45:06,066][20381] Updated weights for policy 1, policy_version 594628 (0.0009) [2023-12-27 09:45:06,134][20381] Updated weights for policy 1, policy_version 594638 (0.0008) [2023-12-27 09:45:06,191][20381] Updated weights for policy 1, policy_version 594648 (0.0011) [2023-12-27 09:45:06,452][20347] Updated weights for policy 0, policy_version 594984 (0.0010) [2023-12-27 09:45:06,502][20347] Updated weights for policy 0, policy_version 594994 (0.0010) [2023-12-27 09:45:06,549][20347] Updated weights for policy 0, policy_version 595004 (0.0009) [2023-12-27 09:45:06,887][20381] Updated weights for policy 1, policy_version 594658 (0.0008) [2023-12-27 09:45:06,941][20381] Updated weights for policy 1, policy_version 594668 (0.0008) [2023-12-27 09:45:06,994][20381] Updated weights for policy 1, policy_version 594678 (0.0009) [2023-12-27 09:45:07,053][20381] Updated weights for policy 1, policy_version 594688 (0.0007) [2023-12-27 09:45:07,162][20347] Updated weights for policy 0, policy_version 595014 (0.0008) [2023-12-27 09:45:07,214][20347] Updated weights for policy 0, policy_version 595024 (0.0010) [2023-12-27 09:45:07,280][20347] Updated weights for policy 0, policy_version 595034 (0.0009) [2023-12-27 09:45:07,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21299.3, 300 sec: 20854.9). Total num frames: 304611328. Throughput: 0: 10580.7, 1: 10624.6. Samples: 304574684. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:07,363][19170] Avg episode reward: [(0, '303.560'), (1, '1000.000')] [2023-12-27 09:45:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000594688_152264704.pth... [2023-12-27 09:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000595040_152346624.pth... [2023-12-27 09:45:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000593440_151945216.pth [2023-12-27 09:45:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000593792_152027136.pth [2023-12-27 09:45:07,761][20381] Updated weights for policy 1, policy_version 594698 (0.0006) [2023-12-27 09:45:07,814][20381] Updated weights for policy 1, policy_version 594708 (0.0006) [2023-12-27 09:45:07,871][20381] Updated weights for policy 1, policy_version 594718 (0.0005) [2023-12-27 09:45:07,952][20347] Updated weights for policy 0, policy_version 595044 (0.0010) [2023-12-27 09:45:08,013][20347] Updated weights for policy 0, policy_version 595054 (0.0010) [2023-12-27 09:45:08,068][20347] Updated weights for policy 0, policy_version 595064 (0.0010) [2023-12-27 09:45:08,442][20381] Updated weights for policy 1, policy_version 594728 (0.0008) [2023-12-27 09:45:08,487][20381] Updated weights for policy 1, policy_version 594738 (0.0008) [2023-12-27 09:45:08,544][20381] Updated weights for policy 1, policy_version 594748 (0.0008) [2023-12-27 09:45:08,732][20347] Updated weights for policy 0, policy_version 595074 (0.0010) [2023-12-27 09:45:08,795][20347] Updated weights for policy 0, policy_version 595084 (0.0008) [2023-12-27 09:45:08,865][20347] Updated weights for policy 0, policy_version 595094 (0.0007) [2023-12-27 09:45:08,929][20347] Updated weights for policy 0, policy_version 595104 (0.0008) [2023-12-27 09:45:09,242][20381] Updated weights for policy 1, policy_version 594758 (0.0009) [2023-12-27 09:45:09,292][20381] Updated weights for policy 1, policy_version 594768 (0.0010) [2023-12-27 09:45:09,343][20381] Updated weights for policy 1, policy_version 594778 (0.0011) [2023-12-27 09:45:09,631][20347] Updated weights for policy 0, policy_version 595114 (0.0007) [2023-12-27 09:45:09,695][20347] Updated weights for policy 0, policy_version 595124 (0.0007) [2023-12-27 09:45:09,752][20347] Updated weights for policy 0, policy_version 595134 (0.0007) [2023-12-27 09:45:10,057][20381] Updated weights for policy 1, policy_version 594788 (0.0007) [2023-12-27 09:45:10,114][20381] Updated weights for policy 1, policy_version 594798 (0.0007) [2023-12-27 09:45:10,164][20381] Updated weights for policy 1, policy_version 594808 (0.0008) [2023-12-27 09:45:10,436][20347] Updated weights for policy 0, policy_version 595144 (0.0008) [2023-12-27 09:45:10,488][20347] Updated weights for policy 0, policy_version 595154 (0.0010) [2023-12-27 09:45:10,543][20347] Updated weights for policy 0, policy_version 595164 (0.0010) [2023-12-27 09:45:10,843][20381] Updated weights for policy 1, policy_version 594818 (0.0006) [2023-12-27 09:45:10,903][20381] Updated weights for policy 1, policy_version 594828 (0.0005) [2023-12-27 09:45:10,958][20381] Updated weights for policy 1, policy_version 594838 (0.0006) [2023-12-27 09:45:11,008][20381] Updated weights for policy 1, policy_version 594848 (0.0008) [2023-12-27 09:45:11,221][20347] Updated weights for policy 0, policy_version 595174 (0.0011) [2023-12-27 09:45:11,287][20347] Updated weights for policy 0, policy_version 595184 (0.0010) [2023-12-27 09:45:11,351][20347] Updated weights for policy 0, policy_version 595194 (0.0010) [2023-12-27 09:45:11,786][20381] Updated weights for policy 1, policy_version 594858 (0.0009) [2023-12-27 09:45:11,843][20381] Updated weights for policy 1, policy_version 594868 (0.0009) [2023-12-27 09:45:11,902][20381] Updated weights for policy 1, policy_version 594878 (0.0008) [2023-12-27 09:45:12,153][20347] Updated weights for policy 0, policy_version 595204 (0.0010) [2023-12-27 09:45:12,218][20347] Updated weights for policy 0, policy_version 595214 (0.0011) [2023-12-27 09:45:12,272][20347] Updated weights for policy 0, policy_version 595224 (0.0011) [2023-12-27 09:45:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21162.6, 300 sec: 20827.1). Total num frames: 304709632. Throughput: 0: 10599.3, 1: 10633.8. Samples: 304699444. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:12,363][19170] Avg episode reward: [(0, '277.400'), (1, '990.400')] [2023-12-27 09:45:12,573][20381] Updated weights for policy 1, policy_version 594888 (0.0008) [2023-12-27 09:45:12,618][20381] Updated weights for policy 1, policy_version 594898 (0.0005) [2023-12-27 09:45:12,670][20381] Updated weights for policy 1, policy_version 594908 (0.0007) [2023-12-27 09:45:12,974][20347] Updated weights for policy 0, policy_version 595234 (0.0010) [2023-12-27 09:45:13,036][20347] Updated weights for policy 0, policy_version 595244 (0.0010) [2023-12-27 09:45:13,091][20347] Updated weights for policy 0, policy_version 595254 (0.0010) [2023-12-27 09:45:13,150][20347] Updated weights for policy 0, policy_version 595264 (0.0011) [2023-12-27 09:45:13,295][20381] Updated weights for policy 1, policy_version 594918 (0.0006) [2023-12-27 09:45:13,353][20381] Updated weights for policy 1, policy_version 594928 (0.0006) [2023-12-27 09:45:13,410][20381] Updated weights for policy 1, policy_version 594938 (0.0007) [2023-12-27 09:45:13,821][20347] Updated weights for policy 0, policy_version 595274 (0.0010) [2023-12-27 09:45:13,877][20347] Updated weights for policy 0, policy_version 595284 (0.0010) [2023-12-27 09:45:13,938][20347] Updated weights for policy 0, policy_version 595294 (0.0010) [2023-12-27 09:45:14,034][20381] Updated weights for policy 1, policy_version 594948 (0.0007) [2023-12-27 09:45:14,087][20381] Updated weights for policy 1, policy_version 594958 (0.0005) [2023-12-27 09:45:14,136][20381] Updated weights for policy 1, policy_version 594968 (0.0007) [2023-12-27 09:45:14,585][20347] Updated weights for policy 0, policy_version 595304 (0.0008) [2023-12-27 09:45:14,638][20347] Updated weights for policy 0, policy_version 595314 (0.0008) [2023-12-27 09:45:14,688][20347] Updated weights for policy 0, policy_version 595324 (0.0009) [2023-12-27 09:45:14,790][20381] Updated weights for policy 1, policy_version 594978 (0.0008) [2023-12-27 09:45:14,846][20381] Updated weights for policy 1, policy_version 594988 (0.0008) [2023-12-27 09:45:14,903][20381] Updated weights for policy 1, policy_version 594998 (0.0007) [2023-12-27 09:45:14,958][20381] Updated weights for policy 1, policy_version 595008 (0.0008) [2023-12-27 09:45:15,405][20347] Updated weights for policy 0, policy_version 595334 (0.0007) [2023-12-27 09:45:15,460][20347] Updated weights for policy 0, policy_version 595344 (0.0005) [2023-12-27 09:45:15,519][20347] Updated weights for policy 0, policy_version 595354 (0.0005) [2023-12-27 09:45:15,711][20381] Updated weights for policy 1, policy_version 595018 (0.0008) [2023-12-27 09:45:15,769][20381] Updated weights for policy 1, policy_version 595028 (0.0009) [2023-12-27 09:45:15,825][20381] Updated weights for policy 1, policy_version 595038 (0.0006) [2023-12-27 09:45:16,093][20347] Updated weights for policy 0, policy_version 595364 (0.0006) [2023-12-27 09:45:16,139][20347] Updated weights for policy 0, policy_version 595374 (0.0008) [2023-12-27 09:45:16,186][20347] Updated weights for policy 0, policy_version 595384 (0.0007) [2023-12-27 09:45:16,459][20381] Updated weights for policy 1, policy_version 595048 (0.0005) [2023-12-27 09:45:16,514][20381] Updated weights for policy 1, policy_version 595058 (0.0005) [2023-12-27 09:45:16,569][20381] Updated weights for policy 1, policy_version 595068 (0.0008) [2023-12-27 09:45:16,891][20347] Updated weights for policy 0, policy_version 595394 (0.0007) [2023-12-27 09:45:16,951][20347] Updated weights for policy 0, policy_version 595404 (0.0006) [2023-12-27 09:45:17,010][20347] Updated weights for policy 0, policy_version 595414 (0.0006) [2023-12-27 09:45:17,071][20347] Updated weights for policy 0, policy_version 595424 (0.0008) [2023-12-27 09:45:17,268][20381] Updated weights for policy 1, policy_version 595078 (0.0008) [2023-12-27 09:45:17,331][20381] Updated weights for policy 1, policy_version 595088 (0.0008) [2023-12-27 09:45:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 304807936. Throughput: 0: 10487.5, 1: 10543.8. Samples: 304821152. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:17,363][19170] Avg episode reward: [(0, '283.520'), (1, '1000.000')] [2023-12-27 09:45:17,391][20381] Updated weights for policy 1, policy_version 595098 (0.0008) [2023-12-27 09:45:17,746][20347] Updated weights for policy 0, policy_version 595434 (0.0007) [2023-12-27 09:45:17,803][20347] Updated weights for policy 0, policy_version 595444 (0.0007) [2023-12-27 09:45:17,866][20347] Updated weights for policy 0, policy_version 595454 (0.0008) [2023-12-27 09:45:18,082][20381] Updated weights for policy 1, policy_version 595108 (0.0008) [2023-12-27 09:45:18,128][20381] Updated weights for policy 1, policy_version 595118 (0.0008) [2023-12-27 09:45:18,175][20381] Updated weights for policy 1, policy_version 595128 (0.0007) [2023-12-27 09:45:18,532][20347] Updated weights for policy 0, policy_version 595464 (0.0006) [2023-12-27 09:45:18,596][20347] Updated weights for policy 0, policy_version 595474 (0.0006) [2023-12-27 09:45:18,658][20347] Updated weights for policy 0, policy_version 595484 (0.0006) [2023-12-27 09:45:18,891][20381] Updated weights for policy 1, policy_version 595138 (0.0007) [2023-12-27 09:45:18,952][20381] Updated weights for policy 1, policy_version 595148 (0.0006) [2023-12-27 09:45:19,013][20381] Updated weights for policy 1, policy_version 595158 (0.0010) [2023-12-27 09:45:19,075][20381] Updated weights for policy 1, policy_version 595168 (0.0011) [2023-12-27 09:45:19,325][20347] Updated weights for policy 0, policy_version 595494 (0.0009) [2023-12-27 09:45:19,389][20347] Updated weights for policy 0, policy_version 595504 (0.0009) [2023-12-27 09:45:19,448][20347] Updated weights for policy 0, policy_version 595514 (0.0008) [2023-12-27 09:45:19,734][20381] Updated weights for policy 1, policy_version 595178 (0.0009) [2023-12-27 09:45:19,791][20381] Updated weights for policy 1, policy_version 595188 (0.0007) [2023-12-27 09:45:19,849][20381] Updated weights for policy 1, policy_version 595198 (0.0007) [2023-12-27 09:45:20,186][20347] Updated weights for policy 0, policy_version 595524 (0.0008) [2023-12-27 09:45:20,251][20347] Updated weights for policy 0, policy_version 595534 (0.0008) [2023-12-27 09:45:20,312][20347] Updated weights for policy 0, policy_version 595544 (0.0008) [2023-12-27 09:45:20,575][20381] Updated weights for policy 1, policy_version 595208 (0.0008) [2023-12-27 09:45:20,642][20381] Updated weights for policy 1, policy_version 595218 (0.0008) [2023-12-27 09:45:20,710][20381] Updated weights for policy 1, policy_version 595228 (0.0008) [2023-12-27 09:45:21,049][20347] Updated weights for policy 0, policy_version 595554 (0.0007) [2023-12-27 09:45:21,111][20347] Updated weights for policy 0, policy_version 595564 (0.0006) [2023-12-27 09:45:21,176][20347] Updated weights for policy 0, policy_version 595574 (0.0008) [2023-12-27 09:45:21,237][20347] Updated weights for policy 0, policy_version 595584 (0.0006) [2023-12-27 09:45:21,423][20381] Updated weights for policy 1, policy_version 595238 (0.0009) [2023-12-27 09:45:21,481][20381] Updated weights for policy 1, policy_version 595248 (0.0011) [2023-12-27 09:45:21,536][20381] Updated weights for policy 1, policy_version 595258 (0.0010) [2023-12-27 09:45:22,017][20347] Updated weights for policy 0, policy_version 595594 (0.0006) [2023-12-27 09:45:22,072][20347] Updated weights for policy 0, policy_version 595604 (0.0006) [2023-12-27 09:45:22,133][20347] Updated weights for policy 0, policy_version 595614 (0.0007) [2023-12-27 09:45:22,311][20381] Updated weights for policy 1, policy_version 595268 (0.0010) [2023-12-27 09:45:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 304906240. Throughput: 0: 10452.1, 1: 10497.2. Samples: 304880576. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:22,363][19170] Avg episode reward: [(0, '233.160'), (1, '1000.000')] [2023-12-27 09:45:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000595616_152494080.pth... [2023-12-27 09:45:22,371][20381] Updated weights for policy 1, policy_version 595278 (0.0009) [2023-12-27 09:45:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000594400_152182784.pth [2023-12-27 09:45:22,431][20381] Updated weights for policy 1, policy_version 595288 (0.0008) [2023-12-27 09:45:22,473][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000595296_152420352.pth... [2023-12-27 09:45:22,477][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000594080_152109056.pth [2023-12-27 09:45:22,835][20347] Updated weights for policy 0, policy_version 595624 (0.0007) [2023-12-27 09:45:22,893][20347] Updated weights for policy 0, policy_version 595634 (0.0006) [2023-12-27 09:45:22,946][20347] Updated weights for policy 0, policy_version 595644 (0.0006) [2023-12-27 09:45:23,121][20381] Updated weights for policy 1, policy_version 595298 (0.0008) [2023-12-27 09:45:23,174][20381] Updated weights for policy 1, policy_version 595308 (0.0008) [2023-12-27 09:45:23,230][20381] Updated weights for policy 1, policy_version 595318 (0.0007) [2023-12-27 09:45:23,286][20381] Updated weights for policy 1, policy_version 595328 (0.0007) [2023-12-27 09:45:23,607][20347] Updated weights for policy 0, policy_version 595654 (0.0007) [2023-12-27 09:45:23,658][20347] Updated weights for policy 0, policy_version 595664 (0.0008) [2023-12-27 09:45:23,704][20347] Updated weights for policy 0, policy_version 595674 (0.0007) [2023-12-27 09:45:23,969][20381] Updated weights for policy 1, policy_version 595338 (0.0008) [2023-12-27 09:45:24,026][20381] Updated weights for policy 1, policy_version 595348 (0.0008) [2023-12-27 09:45:24,072][20381] Updated weights for policy 1, policy_version 595358 (0.0007) [2023-12-27 09:45:24,436][20347] Updated weights for policy 0, policy_version 595684 (0.0007) [2023-12-27 09:45:24,491][20347] Updated weights for policy 0, policy_version 595694 (0.0005) [2023-12-27 09:45:24,545][20347] Updated weights for policy 0, policy_version 595704 (0.0006) [2023-12-27 09:45:24,725][20381] Updated weights for policy 1, policy_version 595368 (0.0006) [2023-12-27 09:45:24,760][20292] KL-divergence is very high: 140.9194 [2023-12-27 09:45:24,784][20381] Updated weights for policy 1, policy_version 595378 (0.0006) [2023-12-27 09:45:24,840][20381] Updated weights for policy 1, policy_version 595388 (0.0005) [2023-12-27 09:45:25,177][20347] Updated weights for policy 0, policy_version 595714 (0.0007) [2023-12-27 09:45:25,227][20347] Updated weights for policy 0, policy_version 595724 (0.0007) [2023-12-27 09:45:25,279][20347] Updated weights for policy 0, policy_version 595734 (0.0010) [2023-12-27 09:45:25,328][20347] Updated weights for policy 0, policy_version 595744 (0.0010) [2023-12-27 09:45:25,352][20381] Updated weights for policy 1, policy_version 595398 (0.0005) [2023-12-27 09:45:25,413][20381] Updated weights for policy 1, policy_version 595408 (0.0005) [2023-12-27 09:45:25,475][20381] Updated weights for policy 1, policy_version 595418 (0.0005) [2023-12-27 09:45:25,983][20347] Updated weights for policy 0, policy_version 595754 (0.0006) [2023-12-27 09:45:26,027][20381] Updated weights for policy 1, policy_version 595428 (0.0007) [2023-12-27 09:45:26,043][20347] Updated weights for policy 0, policy_version 595764 (0.0009) [2023-12-27 09:45:26,085][20381] Updated weights for policy 1, policy_version 595438 (0.0006) [2023-12-27 09:45:26,090][20347] Updated weights for policy 0, policy_version 595774 (0.0005) [2023-12-27 09:45:26,145][20381] Updated weights for policy 1, policy_version 595448 (0.0006) [2023-12-27 09:45:26,718][20347] Updated weights for policy 0, policy_version 595784 (0.0009) [2023-12-27 09:45:26,759][20347] Updated weights for policy 0, policy_version 595794 (0.0010) [2023-12-27 09:45:26,768][20381] Updated weights for policy 1, policy_version 595458 (0.0007) [2023-12-27 09:45:26,804][20347] Updated weights for policy 0, policy_version 595804 (0.0010) [2023-12-27 09:45:26,820][20381] Updated weights for policy 1, policy_version 595468 (0.0006) [2023-12-27 09:45:26,874][20381] Updated weights for policy 1, policy_version 595478 (0.0007) [2023-12-27 09:45:26,925][20381] Updated weights for policy 1, policy_version 595488 (0.0007) [2023-12-27 09:45:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 305012736. Throughput: 0: 10460.5, 1: 10441.7. Samples: 305003216. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:27,363][19170] Avg episode reward: [(0, '294.310'), (1, '990.420')] [2023-12-27 09:45:27,506][20347] Updated weights for policy 0, policy_version 595814 (0.0010) [2023-12-27 09:45:27,541][20381] Updated weights for policy 1, policy_version 595498 (0.0005) [2023-12-27 09:45:27,558][20347] Updated weights for policy 0, policy_version 595824 (0.0010) [2023-12-27 09:45:27,595][20381] Updated weights for policy 1, policy_version 595508 (0.0005) [2023-12-27 09:45:27,609][20347] Updated weights for policy 0, policy_version 595834 (0.0010) [2023-12-27 09:45:27,646][20381] Updated weights for policy 1, policy_version 595518 (0.0005) [2023-12-27 09:45:28,251][20347] Updated weights for policy 0, policy_version 595844 (0.0008) [2023-12-27 09:45:28,262][20381] Updated weights for policy 1, policy_version 595528 (0.0008) [2023-12-27 09:45:28,311][20347] Updated weights for policy 0, policy_version 595854 (0.0005) [2023-12-27 09:45:28,318][20381] Updated weights for policy 1, policy_version 595538 (0.0010) [2023-12-27 09:45:28,376][20381] Updated weights for policy 1, policy_version 595548 (0.0010) [2023-12-27 09:45:28,376][20347] Updated weights for policy 0, policy_version 595864 (0.0008) [2023-12-27 09:45:29,004][20347] Updated weights for policy 0, policy_version 595874 (0.0008) [2023-12-27 09:45:29,053][20381] Updated weights for policy 1, policy_version 595558 (0.0010) [2023-12-27 09:45:29,064][20347] Updated weights for policy 0, policy_version 595884 (0.0010) [2023-12-27 09:45:29,104][20381] Updated weights for policy 1, policy_version 595568 (0.0010) [2023-12-27 09:45:29,118][20347] Updated weights for policy 0, policy_version 595894 (0.0010) [2023-12-27 09:45:29,159][20381] Updated weights for policy 1, policy_version 595578 (0.0010) [2023-12-27 09:45:29,172][20347] Updated weights for policy 0, policy_version 595904 (0.0010) [2023-12-27 09:45:29,782][20347] Updated weights for policy 0, policy_version 595914 (0.0010) [2023-12-27 09:45:29,845][20347] Updated weights for policy 0, policy_version 595924 (0.0009) [2023-12-27 09:45:29,846][20381] Updated weights for policy 1, policy_version 595588 (0.0009) [2023-12-27 09:45:29,892][20347] Updated weights for policy 0, policy_version 595934 (0.0009) [2023-12-27 09:45:29,907][20381] Updated weights for policy 1, policy_version 595598 (0.0007) [2023-12-27 09:45:29,965][20381] Updated weights for policy 1, policy_version 595608 (0.0009) [2023-12-27 09:45:30,585][20347] Updated weights for policy 0, policy_version 595944 (0.0006) [2023-12-27 09:45:30,591][20381] Updated weights for policy 1, policy_version 595618 (0.0010) [2023-12-27 09:45:30,634][20347] Updated weights for policy 0, policy_version 595954 (0.0005) [2023-12-27 09:45:30,646][20381] Updated weights for policy 1, policy_version 595628 (0.0010) [2023-12-27 09:45:30,683][20347] Updated weights for policy 0, policy_version 595964 (0.0009) [2023-12-27 09:45:30,702][20381] Updated weights for policy 1, policy_version 595638 (0.0011) [2023-12-27 09:45:30,756][20381] Updated weights for policy 1, policy_version 595648 (0.0010) [2023-12-27 09:45:31,279][20347] Updated weights for policy 0, policy_version 595974 (0.0008) [2023-12-27 09:45:31,333][20347] Updated weights for policy 0, policy_version 595984 (0.0008) [2023-12-27 09:45:31,397][20347] Updated weights for policy 0, policy_version 595994 (0.0007) [2023-12-27 09:45:31,412][20381] Updated weights for policy 1, policy_version 595658 (0.0007) [2023-12-27 09:45:31,467][20381] Updated weights for policy 1, policy_version 595668 (0.0005) [2023-12-27 09:45:31,518][20381] Updated weights for policy 1, policy_version 595678 (0.0005) [2023-12-27 09:45:32,086][20347] Updated weights for policy 0, policy_version 596004 (0.0008) [2023-12-27 09:45:32,138][20381] Updated weights for policy 1, policy_version 595688 (0.0007) [2023-12-27 09:45:32,141][20347] Updated weights for policy 0, policy_version 596014 (0.0010) [2023-12-27 09:45:32,196][20381] Updated weights for policy 1, policy_version 595698 (0.0009) [2023-12-27 09:45:32,199][20347] Updated weights for policy 0, policy_version 596024 (0.0010) [2023-12-27 09:45:32,250][20381] Updated weights for policy 1, policy_version 595708 (0.0007) [2023-12-27 09:45:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.5, 300 sec: 20854.9). Total num frames: 305127424. Throughput: 0: 10468.8, 1: 10459.5. Samples: 305132024. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:32,363][19170] Avg episode reward: [(0, '235.000'), (1, '990.420')] [2023-12-27 09:45:32,882][20381] Updated weights for policy 1, policy_version 595718 (0.0009) [2023-12-27 09:45:32,888][20347] Updated weights for policy 0, policy_version 596034 (0.0009) [2023-12-27 09:45:32,931][20381] Updated weights for policy 1, policy_version 595728 (0.0011) [2023-12-27 09:45:32,944][20347] Updated weights for policy 0, policy_version 596044 (0.0006) [2023-12-27 09:45:32,980][20381] Updated weights for policy 1, policy_version 595738 (0.0010) [2023-12-27 09:45:32,993][20347] Updated weights for policy 0, policy_version 596054 (0.0006) [2023-12-27 09:45:33,042][20347] Updated weights for policy 0, policy_version 596064 (0.0006) [2023-12-27 09:45:33,684][20347] Updated weights for policy 0, policy_version 596074 (0.0007) [2023-12-27 09:45:33,704][20381] Updated weights for policy 1, policy_version 595748 (0.0010) [2023-12-27 09:45:33,738][20347] Updated weights for policy 0, policy_version 596084 (0.0005) [2023-12-27 09:45:33,758][20381] Updated weights for policy 1, policy_version 595758 (0.0010) [2023-12-27 09:45:33,793][20347] Updated weights for policy 0, policy_version 596094 (0.0007) [2023-12-27 09:45:33,815][20381] Updated weights for policy 1, policy_version 595768 (0.0006) [2023-12-27 09:45:34,390][20347] Updated weights for policy 0, policy_version 596104 (0.0006) [2023-12-27 09:45:34,447][20347] Updated weights for policy 0, policy_version 596114 (0.0006) [2023-12-27 09:45:34,467][20381] Updated weights for policy 1, policy_version 595778 (0.0007) [2023-12-27 09:45:34,511][20347] Updated weights for policy 0, policy_version 596124 (0.0006) [2023-12-27 09:45:34,529][20381] Updated weights for policy 1, policy_version 595788 (0.0007) [2023-12-27 09:45:34,582][20381] Updated weights for policy 1, policy_version 595798 (0.0009) [2023-12-27 09:45:34,639][20381] Updated weights for policy 1, policy_version 595808 (0.0007) [2023-12-27 09:45:35,078][20347] Updated weights for policy 0, policy_version 596134 (0.0007) [2023-12-27 09:45:35,139][20347] Updated weights for policy 0, policy_version 596144 (0.0007) [2023-12-27 09:45:35,193][20347] Updated weights for policy 0, policy_version 596154 (0.0005) [2023-12-27 09:45:35,210][20381] Updated weights for policy 1, policy_version 595818 (0.0006) [2023-12-27 09:45:35,260][20381] Updated weights for policy 1, policy_version 595828 (0.0007) [2023-12-27 09:45:35,311][20381] Updated weights for policy 1, policy_version 595838 (0.0008) [2023-12-27 09:45:35,817][20347] Updated weights for policy 0, policy_version 596164 (0.0005) [2023-12-27 09:45:35,872][20347] Updated weights for policy 0, policy_version 596174 (0.0008) [2023-12-27 09:45:35,929][20347] Updated weights for policy 0, policy_version 596184 (0.0008) [2023-12-27 09:45:36,011][20381] Updated weights for policy 1, policy_version 595848 (0.0008) [2023-12-27 09:45:36,066][20381] Updated weights for policy 1, policy_version 595858 (0.0008) [2023-12-27 09:45:36,123][20381] Updated weights for policy 1, policy_version 595868 (0.0007) [2023-12-27 09:45:36,612][20347] Updated weights for policy 0, policy_version 596194 (0.0007) [2023-12-27 09:45:36,665][20347] Updated weights for policy 0, policy_version 596204 (0.0008) [2023-12-27 09:45:36,710][20347] Updated weights for policy 0, policy_version 596214 (0.0007) [2023-12-27 09:45:36,765][20347] Updated weights for policy 0, policy_version 596224 (0.0008) [2023-12-27 09:45:36,836][20381] Updated weights for policy 1, policy_version 595878 (0.0007) [2023-12-27 09:45:36,905][20381] Updated weights for policy 1, policy_version 595888 (0.0007) [2023-12-27 09:45:36,966][20381] Updated weights for policy 1, policy_version 595898 (0.0008) [2023-12-27 09:45:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 305225728. Throughput: 0: 10455.8, 1: 10459.9. Samples: 305197104. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:37,363][19170] Avg episode reward: [(0, '321.390'), (1, '1000.000')] [2023-12-27 09:45:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000595904_152576000.pth... [2023-12-27 09:45:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000596224_152649728.pth... [2023-12-27 09:45:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000594688_152264704.pth [2023-12-27 09:45:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000595040_152346624.pth [2023-12-27 09:45:37,469][20347] Updated weights for policy 0, policy_version 596234 (0.0007) [2023-12-27 09:45:37,530][20347] Updated weights for policy 0, policy_version 596244 (0.0005) [2023-12-27 09:45:37,597][20347] Updated weights for policy 0, policy_version 596254 (0.0008) [2023-12-27 09:45:37,655][20381] Updated weights for policy 1, policy_version 595908 (0.0008) [2023-12-27 09:45:37,721][20381] Updated weights for policy 1, policy_version 595918 (0.0006) [2023-12-27 09:45:37,776][20381] Updated weights for policy 1, policy_version 595928 (0.0007) [2023-12-27 09:45:38,143][20347] Updated weights for policy 0, policy_version 596264 (0.0006) [2023-12-27 09:45:38,200][20347] Updated weights for policy 0, policy_version 596274 (0.0005) [2023-12-27 09:45:38,254][20347] Updated weights for policy 0, policy_version 596284 (0.0005) [2023-12-27 09:45:38,321][20381] Updated weights for policy 1, policy_version 595938 (0.0007) [2023-12-27 09:45:38,381][20381] Updated weights for policy 1, policy_version 595948 (0.0008) [2023-12-27 09:45:38,441][20381] Updated weights for policy 1, policy_version 595958 (0.0006) [2023-12-27 09:45:38,499][20381] Updated weights for policy 1, policy_version 595968 (0.0008) [2023-12-27 09:45:38,843][20347] Updated weights for policy 0, policy_version 596294 (0.0008) [2023-12-27 09:45:38,899][20347] Updated weights for policy 0, policy_version 596304 (0.0007) [2023-12-27 09:45:38,957][20347] Updated weights for policy 0, policy_version 596314 (0.0010) [2023-12-27 09:45:39,199][20381] Updated weights for policy 1, policy_version 595978 (0.0007) [2023-12-27 09:45:39,262][20381] Updated weights for policy 1, policy_version 595988 (0.0008) [2023-12-27 09:45:39,323][20381] Updated weights for policy 1, policy_version 595998 (0.0006) [2023-12-27 09:45:39,635][20347] Updated weights for policy 0, policy_version 596324 (0.0010) [2023-12-27 09:45:39,702][20347] Updated weights for policy 0, policy_version 596334 (0.0011) [2023-12-27 09:45:39,761][20347] Updated weights for policy 0, policy_version 596344 (0.0010) [2023-12-27 09:45:40,047][20381] Updated weights for policy 1, policy_version 596008 (0.0006) [2023-12-27 09:45:40,097][20381] Updated weights for policy 1, policy_version 596018 (0.0005) [2023-12-27 09:45:40,151][20381] Updated weights for policy 1, policy_version 596028 (0.0005) [2023-12-27 09:45:40,489][20347] Updated weights for policy 0, policy_version 596354 (0.0011) [2023-12-27 09:45:40,546][20347] Updated weights for policy 0, policy_version 596364 (0.0007) [2023-12-27 09:45:40,598][20347] Updated weights for policy 0, policy_version 596374 (0.0007) [2023-12-27 09:45:40,659][20347] Updated weights for policy 0, policy_version 596384 (0.0009) [2023-12-27 09:45:40,849][20381] Updated weights for policy 1, policy_version 596038 (0.0008) [2023-12-27 09:45:40,897][20381] Updated weights for policy 1, policy_version 596048 (0.0007) [2023-12-27 09:45:40,952][20381] Updated weights for policy 1, policy_version 596058 (0.0007) [2023-12-27 09:45:41,308][20347] Updated weights for policy 0, policy_version 596394 (0.0011) [2023-12-27 09:45:41,368][20347] Updated weights for policy 0, policy_version 596404 (0.0011) [2023-12-27 09:45:41,429][20347] Updated weights for policy 0, policy_version 596414 (0.0009) [2023-12-27 09:45:41,744][20381] Updated weights for policy 1, policy_version 596068 (0.0008) [2023-12-27 09:45:41,793][20381] Updated weights for policy 1, policy_version 596078 (0.0009) [2023-12-27 09:45:41,839][20381] Updated weights for policy 1, policy_version 596088 (0.0007) [2023-12-27 09:45:42,095][20347] Updated weights for policy 0, policy_version 596424 (0.0006) [2023-12-27 09:45:42,144][20347] Updated weights for policy 0, policy_version 596434 (0.0006) [2023-12-27 09:45:42,194][20347] Updated weights for policy 0, policy_version 596444 (0.0010) [2023-12-27 09:45:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 305332224. Throughput: 0: 10450.8, 1: 10417.6. Samples: 305321140. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:42,363][19170] Avg episode reward: [(0, '343.650'), (1, '980.830')] [2023-12-27 09:45:42,572][20381] Updated weights for policy 1, policy_version 596098 (0.0006) [2023-12-27 09:45:42,617][20381] Updated weights for policy 1, policy_version 596108 (0.0008) [2023-12-27 09:45:42,665][20381] Updated weights for policy 1, policy_version 596118 (0.0007) [2023-12-27 09:45:42,710][20381] Updated weights for policy 1, policy_version 596128 (0.0008) [2023-12-27 09:45:42,896][20347] Updated weights for policy 0, policy_version 596454 (0.0010) [2023-12-27 09:45:42,955][20347] Updated weights for policy 0, policy_version 596464 (0.0010) [2023-12-27 09:45:43,009][20347] Updated weights for policy 0, policy_version 596474 (0.0010) [2023-12-27 09:45:43,442][20381] Updated weights for policy 1, policy_version 596138 (0.0007) [2023-12-27 09:45:43,500][20381] Updated weights for policy 1, policy_version 596148 (0.0007) [2023-12-27 09:45:43,557][20381] Updated weights for policy 1, policy_version 596158 (0.0007) [2023-12-27 09:45:43,685][20347] Updated weights for policy 0, policy_version 596484 (0.0010) [2023-12-27 09:45:43,736][20347] Updated weights for policy 0, policy_version 596494 (0.0010) [2023-12-27 09:45:43,786][20347] Updated weights for policy 0, policy_version 596504 (0.0010) [2023-12-27 09:45:44,269][20381] Updated weights for policy 1, policy_version 596168 (0.0010) [2023-12-27 09:45:44,331][20381] Updated weights for policy 1, policy_version 596178 (0.0011) [2023-12-27 09:45:44,389][20381] Updated weights for policy 1, policy_version 596188 (0.0011) [2023-12-27 09:45:44,456][20347] Updated weights for policy 0, policy_version 596514 (0.0009) [2023-12-27 09:45:44,517][20347] Updated weights for policy 0, policy_version 596524 (0.0005) [2023-12-27 09:45:44,580][20347] Updated weights for policy 0, policy_version 596534 (0.0006) [2023-12-27 09:45:44,633][20347] Updated weights for policy 0, policy_version 596544 (0.0005) [2023-12-27 09:45:45,009][20381] Updated weights for policy 1, policy_version 596198 (0.0011) [2023-12-27 09:45:45,065][20381] Updated weights for policy 1, policy_version 596208 (0.0011) [2023-12-27 09:45:45,122][20381] Updated weights for policy 1, policy_version 596218 (0.0011) [2023-12-27 09:45:45,219][20347] Updated weights for policy 0, policy_version 596554 (0.0010) [2023-12-27 09:45:45,278][20347] Updated weights for policy 0, policy_version 596564 (0.0010) [2023-12-27 09:45:45,336][20347] Updated weights for policy 0, policy_version 596574 (0.0010) [2023-12-27 09:45:45,798][20381] Updated weights for policy 1, policy_version 596228 (0.0011) [2023-12-27 09:45:45,855][20381] Updated weights for policy 1, policy_version 596238 (0.0010) [2023-12-27 09:45:45,906][20381] Updated weights for policy 1, policy_version 596248 (0.0010) [2023-12-27 09:45:46,003][20347] Updated weights for policy 0, policy_version 596584 (0.0007) [2023-12-27 09:45:46,058][20347] Updated weights for policy 0, policy_version 596594 (0.0005) [2023-12-27 09:45:46,116][20347] Updated weights for policy 0, policy_version 596604 (0.0007) [2023-12-27 09:45:46,599][20381] Updated weights for policy 1, policy_version 596258 (0.0010) [2023-12-27 09:45:46,659][20381] Updated weights for policy 1, policy_version 596268 (0.0006) [2023-12-27 09:45:46,717][20381] Updated weights for policy 1, policy_version 596278 (0.0005) [2023-12-27 09:45:46,767][20347] Updated weights for policy 0, policy_version 596614 (0.0010) [2023-12-27 09:45:46,772][20381] Updated weights for policy 1, policy_version 596288 (0.0005) [2023-12-27 09:45:46,812][20347] Updated weights for policy 0, policy_version 596624 (0.0009) [2023-12-27 09:45:46,858][20347] Updated weights for policy 0, policy_version 596634 (0.0008) [2023-12-27 09:45:47,353][20381] Updated weights for policy 1, policy_version 596298 (0.0008) [2023-12-27 09:45:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 305430528. Throughput: 0: 10448.8, 1: 10381.8. Samples: 305445184. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:47,363][19170] Avg episode reward: [(0, '233.460'), (1, '980.830')] [2023-12-27 09:45:47,408][20381] Updated weights for policy 1, policy_version 596308 (0.0008) [2023-12-27 09:45:47,460][20381] Updated weights for policy 1, policy_version 596318 (0.0008) [2023-12-27 09:45:47,494][20347] Updated weights for policy 0, policy_version 596644 (0.0007) [2023-12-27 09:45:47,556][20347] Updated weights for policy 0, policy_version 596654 (0.0007) [2023-12-27 09:45:47,607][20347] Updated weights for policy 0, policy_version 596664 (0.0005) [2023-12-27 09:45:48,116][20381] Updated weights for policy 1, policy_version 596328 (0.0005) [2023-12-27 09:45:48,173][20381] Updated weights for policy 1, policy_version 596338 (0.0007) [2023-12-27 09:45:48,229][20381] Updated weights for policy 1, policy_version 596348 (0.0008) [2023-12-27 09:45:48,263][20347] Updated weights for policy 0, policy_version 596674 (0.0006) [2023-12-27 09:45:48,316][20347] Updated weights for policy 0, policy_version 596684 (0.0005) [2023-12-27 09:45:48,381][20347] Updated weights for policy 0, policy_version 596694 (0.0007) [2023-12-27 09:45:48,439][20347] Updated weights for policy 0, policy_version 596704 (0.0005) [2023-12-27 09:45:48,841][20381] Updated weights for policy 1, policy_version 596358 (0.0008) [2023-12-27 09:45:48,895][20381] Updated weights for policy 1, policy_version 596368 (0.0009) [2023-12-27 09:45:48,959][20381] Updated weights for policy 1, policy_version 596378 (0.0008) [2023-12-27 09:45:48,999][20347] Updated weights for policy 0, policy_version 596714 (0.0005) [2023-12-27 09:45:49,060][20347] Updated weights for policy 0, policy_version 596724 (0.0007) [2023-12-27 09:45:49,123][20347] Updated weights for policy 0, policy_version 596734 (0.0009) [2023-12-27 09:45:49,626][20381] Updated weights for policy 1, policy_version 596388 (0.0007) [2023-12-27 09:45:49,684][20381] Updated weights for policy 1, policy_version 596398 (0.0008) [2023-12-27 09:45:49,747][20381] Updated weights for policy 1, policy_version 596408 (0.0008) [2023-12-27 09:45:49,807][20347] Updated weights for policy 0, policy_version 596744 (0.0007) [2023-12-27 09:45:49,866][20347] Updated weights for policy 0, policy_version 596754 (0.0008) [2023-12-27 09:45:49,930][20347] Updated weights for policy 0, policy_version 596764 (0.0008) [2023-12-27 09:45:50,458][20381] Updated weights for policy 1, policy_version 596418 (0.0007) [2023-12-27 09:45:50,517][20381] Updated weights for policy 1, policy_version 596428 (0.0008) [2023-12-27 09:45:50,581][20381] Updated weights for policy 1, policy_version 596438 (0.0009) [2023-12-27 09:45:50,605][20347] Updated weights for policy 0, policy_version 596774 (0.0008) [2023-12-27 09:45:50,639][20381] Updated weights for policy 1, policy_version 596448 (0.0007) [2023-12-27 09:45:50,667][20347] Updated weights for policy 0, policy_version 596784 (0.0008) [2023-12-27 09:45:50,732][20347] Updated weights for policy 0, policy_version 596794 (0.0008) [2023-12-27 09:45:51,307][20381] Updated weights for policy 1, policy_version 596458 (0.0007) [2023-12-27 09:45:51,372][20381] Updated weights for policy 1, policy_version 596468 (0.0008) [2023-12-27 09:45:51,429][20381] Updated weights for policy 1, policy_version 596478 (0.0009) [2023-12-27 09:45:51,500][20347] Updated weights for policy 0, policy_version 596804 (0.0007) [2023-12-27 09:45:51,567][20347] Updated weights for policy 0, policy_version 596814 (0.0007) [2023-12-27 09:45:51,627][20347] Updated weights for policy 0, policy_version 596824 (0.0008) [2023-12-27 09:45:52,185][20381] Updated weights for policy 1, policy_version 596488 (0.0011) [2023-12-27 09:45:52,243][20381] Updated weights for policy 1, policy_version 596498 (0.0010) [2023-12-27 09:45:52,255][20347] Updated weights for policy 0, policy_version 596834 (0.0008) [2023-12-27 09:45:52,295][20381] Updated weights for policy 1, policy_version 596508 (0.0008) [2023-12-27 09:45:52,317][20347] Updated weights for policy 0, policy_version 596844 (0.0010) [2023-12-27 09:45:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 305537024. Throughput: 0: 10389.8, 1: 10348.3. Samples: 305507904. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:52,363][19170] Avg episode reward: [(0, '197.380'), (1, '990.370')] [2023-12-27 09:45:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000596512_152731648.pth... [2023-12-27 09:45:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000595296_152420352.pth [2023-12-27 09:45:52,383][20347] Updated weights for policy 0, policy_version 596854 (0.0007) [2023-12-27 09:45:52,434][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000596864_152813568.pth... [2023-12-27 09:45:52,436][20347] Updated weights for policy 0, policy_version 596864 (0.0008) [2023-12-27 09:45:52,436][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000595616_152494080.pth [2023-12-27 09:45:52,980][20381] Updated weights for policy 1, policy_version 596518 (0.0006) [2023-12-27 09:45:53,043][20381] Updated weights for policy 1, policy_version 596528 (0.0006) [2023-12-27 09:45:53,068][20347] Updated weights for policy 0, policy_version 596874 (0.0007) [2023-12-27 09:45:53,089][20381] Updated weights for policy 1, policy_version 596538 (0.0007) [2023-12-27 09:45:53,120][20347] Updated weights for policy 0, policy_version 596884 (0.0006) [2023-12-27 09:45:53,183][20347] Updated weights for policy 0, policy_version 596894 (0.0007) [2023-12-27 09:45:53,727][20381] Updated weights for policy 1, policy_version 596548 (0.0006) [2023-12-27 09:45:53,771][20381] Updated weights for policy 1, policy_version 596558 (0.0007) [2023-12-27 09:45:53,816][20381] Updated weights for policy 1, policy_version 596568 (0.0007) [2023-12-27 09:45:53,876][20347] Updated weights for policy 0, policy_version 596904 (0.0006) [2023-12-27 09:45:53,925][20347] Updated weights for policy 0, policy_version 596914 (0.0006) [2023-12-27 09:45:53,978][20347] Updated weights for policy 0, policy_version 596924 (0.0006) [2023-12-27 09:45:54,411][20381] Updated weights for policy 1, policy_version 596578 (0.0007) [2023-12-27 09:45:54,458][20381] Updated weights for policy 1, policy_version 596588 (0.0005) [2023-12-27 09:45:54,504][20381] Updated weights for policy 1, policy_version 596598 (0.0005) [2023-12-27 09:45:54,569][20381] Updated weights for policy 1, policy_version 596608 (0.0006) [2023-12-27 09:45:54,703][20347] Updated weights for policy 0, policy_version 596934 (0.0008) [2023-12-27 09:45:54,759][20347] Updated weights for policy 0, policy_version 596944 (0.0007) [2023-12-27 09:45:54,810][20347] Updated weights for policy 0, policy_version 596954 (0.0007) [2023-12-27 09:45:55,195][20381] Updated weights for policy 1, policy_version 596618 (0.0006) [2023-12-27 09:45:55,242][20381] Updated weights for policy 1, policy_version 596628 (0.0008) [2023-12-27 09:45:55,302][20381] Updated weights for policy 1, policy_version 596638 (0.0008) [2023-12-27 09:45:55,499][20347] Updated weights for policy 0, policy_version 596964 (0.0008) [2023-12-27 09:45:55,548][20347] Updated weights for policy 0, policy_version 596974 (0.0007) [2023-12-27 09:45:55,592][20347] Updated weights for policy 0, policy_version 596984 (0.0009) [2023-12-27 09:45:55,981][20381] Updated weights for policy 1, policy_version 596648 (0.0008) [2023-12-27 09:45:56,034][20381] Updated weights for policy 1, policy_version 596658 (0.0007) [2023-12-27 09:45:56,092][20381] Updated weights for policy 1, policy_version 596668 (0.0008) [2023-12-27 09:45:56,220][20347] Updated weights for policy 0, policy_version 596994 (0.0006) [2023-12-27 09:45:56,269][20347] Updated weights for policy 0, policy_version 597004 (0.0007) [2023-12-27 09:45:56,313][20347] Updated weights for policy 0, policy_version 597014 (0.0008) [2023-12-27 09:45:56,362][20347] Updated weights for policy 0, policy_version 597024 (0.0008) [2023-12-27 09:45:56,796][20381] Updated weights for policy 1, policy_version 596678 (0.0006) [2023-12-27 09:45:56,849][20381] Updated weights for policy 1, policy_version 596688 (0.0005) [2023-12-27 09:45:56,896][20381] Updated weights for policy 1, policy_version 596698 (0.0005) [2023-12-27 09:45:57,000][20347] Updated weights for policy 0, policy_version 597034 (0.0007) [2023-12-27 09:45:57,058][20347] Updated weights for policy 0, policy_version 597044 (0.0005) [2023-12-27 09:45:57,115][20347] Updated weights for policy 0, policy_version 597054 (0.0005) [2023-12-27 09:45:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 305643520. Throughput: 0: 10363.5, 1: 10346.1. Samples: 305631380. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:45:57,363][19170] Avg episode reward: [(0, '274.050'), (1, '980.770')] [2023-12-27 09:45:57,446][20381] Updated weights for policy 1, policy_version 596708 (0.0005) [2023-12-27 09:45:57,511][20381] Updated weights for policy 1, policy_version 596718 (0.0005) [2023-12-27 09:45:57,576][20381] Updated weights for policy 1, policy_version 596728 (0.0005) [2023-12-27 09:45:57,682][20347] Updated weights for policy 0, policy_version 597064 (0.0007) [2023-12-27 09:45:57,727][20347] Updated weights for policy 0, policy_version 597074 (0.0008) [2023-12-27 09:45:57,773][20347] Updated weights for policy 0, policy_version 597084 (0.0009) [2023-12-27 09:45:58,089][20381] Updated weights for policy 1, policy_version 596738 (0.0006) [2023-12-27 09:45:58,145][20381] Updated weights for policy 1, policy_version 596748 (0.0005) [2023-12-27 09:45:58,206][20381] Updated weights for policy 1, policy_version 596758 (0.0008) [2023-12-27 09:45:58,266][20381] Updated weights for policy 1, policy_version 596768 (0.0007) [2023-12-27 09:45:58,459][20347] Updated weights for policy 0, policy_version 597094 (0.0010) [2023-12-27 09:45:58,520][20347] Updated weights for policy 0, policy_version 597104 (0.0008) [2023-12-27 09:45:58,577][20347] Updated weights for policy 0, policy_version 597114 (0.0009) [2023-12-27 09:45:59,059][20381] Updated weights for policy 1, policy_version 596778 (0.0011) [2023-12-27 09:45:59,122][20381] Updated weights for policy 1, policy_version 596788 (0.0008) [2023-12-27 09:45:59,185][20381] Updated weights for policy 1, policy_version 596798 (0.0006) [2023-12-27 09:45:59,309][20347] Updated weights for policy 0, policy_version 597124 (0.0009) [2023-12-27 09:45:59,373][20347] Updated weights for policy 0, policy_version 597134 (0.0007) [2023-12-27 09:45:59,430][20347] Updated weights for policy 0, policy_version 597144 (0.0007) [2023-12-27 09:45:59,875][20381] Updated weights for policy 1, policy_version 596808 (0.0006) [2023-12-27 09:45:59,933][20381] Updated weights for policy 1, policy_version 596818 (0.0009) [2023-12-27 09:45:59,996][20381] Updated weights for policy 1, policy_version 596828 (0.0007) [2023-12-27 09:46:00,110][20347] Updated weights for policy 0, policy_version 597154 (0.0007) [2023-12-27 09:46:00,165][20347] Updated weights for policy 0, policy_version 597164 (0.0009) [2023-12-27 09:46:00,223][20347] Updated weights for policy 0, policy_version 597174 (0.0007) [2023-12-27 09:46:00,278][20347] Updated weights for policy 0, policy_version 597184 (0.0007) [2023-12-27 09:46:00,662][20381] Updated weights for policy 1, policy_version 596838 (0.0010) [2023-12-27 09:46:00,711][20381] Updated weights for policy 1, policy_version 596848 (0.0010) [2023-12-27 09:46:00,759][20381] Updated weights for policy 1, policy_version 596858 (0.0010) [2023-12-27 09:46:00,937][20347] Updated weights for policy 0, policy_version 597194 (0.0006) [2023-12-27 09:46:00,993][20347] Updated weights for policy 0, policy_version 597204 (0.0005) [2023-12-27 09:46:01,046][20347] Updated weights for policy 0, policy_version 597214 (0.0007) [2023-12-27 09:46:01,459][20381] Updated weights for policy 1, policy_version 596868 (0.0009) [2023-12-27 09:46:01,517][20381] Updated weights for policy 1, policy_version 596878 (0.0007) [2023-12-27 09:46:01,573][20381] Updated weights for policy 1, policy_version 596888 (0.0008) [2023-12-27 09:46:01,752][20347] Updated weights for policy 0, policy_version 597224 (0.0008) [2023-12-27 09:46:01,808][20347] Updated weights for policy 0, policy_version 597234 (0.0007) [2023-12-27 09:46:01,858][20347] Updated weights for policy 0, policy_version 597244 (0.0007) [2023-12-27 09:46:02,232][20381] Updated weights for policy 1, policy_version 596898 (0.0007) [2023-12-27 09:46:02,291][20381] Updated weights for policy 1, policy_version 596908 (0.0008) [2023-12-27 09:46:02,349][20381] Updated weights for policy 1, policy_version 596918 (0.0008) [2023-12-27 09:46:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 305741824. Throughput: 0: 10410.5, 1: 10391.3. Samples: 305757232. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:46:02,363][19170] Avg episode reward: [(0, '272.600'), (1, '980.690')] [2023-12-27 09:46:02,407][20381] Updated weights for policy 1, policy_version 596928 (0.0008) [2023-12-27 09:46:02,562][20347] Updated weights for policy 0, policy_version 597254 (0.0007) [2023-12-27 09:46:02,610][20347] Updated weights for policy 0, policy_version 597264 (0.0008) [2023-12-27 09:46:02,666][20347] Updated weights for policy 0, policy_version 597274 (0.0008) [2023-12-27 09:46:03,115][20381] Updated weights for policy 1, policy_version 596938 (0.0006) [2023-12-27 09:46:03,126][20292] KL-divergence is very high: 101.3496 [2023-12-27 09:46:03,175][20381] Updated weights for policy 1, policy_version 596948 (0.0005) [2023-12-27 09:46:03,239][20381] Updated weights for policy 1, policy_version 596958 (0.0007) [2023-12-27 09:46:03,312][20347] Updated weights for policy 0, policy_version 597284 (0.0007) [2023-12-27 09:46:03,359][20347] Updated weights for policy 0, policy_version 597294 (0.0005) [2023-12-27 09:46:03,412][20347] Updated weights for policy 0, policy_version 597304 (0.0005) [2023-12-27 09:46:03,902][20381] Updated weights for policy 1, policy_version 596968 (0.0007) [2023-12-27 09:46:03,959][20381] Updated weights for policy 1, policy_version 596978 (0.0008) [2023-12-27 09:46:04,012][20381] Updated weights for policy 1, policy_version 596988 (0.0008) [2023-12-27 09:46:04,077][20347] Updated weights for policy 0, policy_version 597314 (0.0009) [2023-12-27 09:46:04,141][20347] Updated weights for policy 0, policy_version 597324 (0.0007) [2023-12-27 09:46:04,203][20347] Updated weights for policy 0, policy_version 597334 (0.0006) [2023-12-27 09:46:04,260][20347] Updated weights for policy 0, policy_version 597344 (0.0008) [2023-12-27 09:46:04,740][20381] Updated weights for policy 1, policy_version 596998 (0.0007) [2023-12-27 09:46:04,807][20381] Updated weights for policy 1, policy_version 597008 (0.0006) [2023-12-27 09:46:04,868][20381] Updated weights for policy 1, policy_version 597018 (0.0006) [2023-12-27 09:46:04,907][20347] Updated weights for policy 0, policy_version 597354 (0.0006) [2023-12-27 09:46:04,958][20347] Updated weights for policy 0, policy_version 597364 (0.0006) [2023-12-27 09:46:05,011][20347] Updated weights for policy 0, policy_version 597374 (0.0007) [2023-12-27 09:46:05,532][20381] Updated weights for policy 1, policy_version 597028 (0.0010) [2023-12-27 09:46:05,594][20381] Updated weights for policy 1, policy_version 597038 (0.0010) [2023-12-27 09:46:05,623][20347] Updated weights for policy 0, policy_version 597384 (0.0005) [2023-12-27 09:46:05,646][20381] Updated weights for policy 1, policy_version 597048 (0.0010) [2023-12-27 09:46:05,679][20347] Updated weights for policy 0, policy_version 597394 (0.0005) [2023-12-27 09:46:05,732][20347] Updated weights for policy 0, policy_version 597404 (0.0006) [2023-12-27 09:46:06,303][20381] Updated weights for policy 1, policy_version 597058 (0.0008) [2023-12-27 09:46:06,360][20347] Updated weights for policy 0, policy_version 597414 (0.0008) [2023-12-27 09:46:06,364][20381] Updated weights for policy 1, policy_version 597068 (0.0006) [2023-12-27 09:46:06,408][20347] Updated weights for policy 0, policy_version 597424 (0.0006) [2023-12-27 09:46:06,421][20381] Updated weights for policy 1, policy_version 597078 (0.0007) [2023-12-27 09:46:06,458][20347] Updated weights for policy 0, policy_version 597434 (0.0009) [2023-12-27 09:46:06,472][20381] Updated weights for policy 1, policy_version 597088 (0.0006) [2023-12-27 09:46:07,131][20347] Updated weights for policy 0, policy_version 597444 (0.0011) [2023-12-27 09:46:07,177][20381] Updated weights for policy 1, policy_version 597098 (0.0005) [2023-12-27 09:46:07,187][20347] Updated weights for policy 0, policy_version 597454 (0.0010) [2023-12-27 09:46:07,237][20381] Updated weights for policy 1, policy_version 597108 (0.0007) [2023-12-27 09:46:07,247][20347] Updated weights for policy 0, policy_version 597464 (0.0010) [2023-12-27 09:46:07,298][20381] Updated weights for policy 1, policy_version 597118 (0.0006) [2023-12-27 09:46:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 305856512. Throughput: 0: 10452.5, 1: 10392.6. Samples: 305818608. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 09:46:07,363][19170] Avg episode reward: [(0, '239.890'), (1, '990.290')] [2023-12-27 09:46:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000597120_152887296.pth... [2023-12-27 09:46:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000597472_152969216.pth... [2023-12-27 09:46:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000596224_152649728.pth [2023-12-27 09:46:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000595904_152576000.pth [2023-12-27 09:46:07,919][20347] Updated weights for policy 0, policy_version 597474 (0.0011) [2023-12-27 09:46:07,971][20347] Updated weights for policy 0, policy_version 597484 (0.0010) [2023-12-27 09:46:07,993][20381] Updated weights for policy 1, policy_version 597128 (0.0006) [2023-12-27 09:46:08,030][20347] Updated weights for policy 0, policy_version 597494 (0.0010) [2023-12-27 09:46:08,046][20381] Updated weights for policy 1, policy_version 597138 (0.0006) [2023-12-27 09:46:08,078][20347] Updated weights for policy 0, policy_version 597504 (0.0010) [2023-12-27 09:46:08,092][20381] Updated weights for policy 1, policy_version 597148 (0.0006) [2023-12-27 09:46:08,764][20347] Updated weights for policy 0, policy_version 597514 (0.0010) [2023-12-27 09:46:08,815][20381] Updated weights for policy 1, policy_version 597158 (0.0006) [2023-12-27 09:46:08,820][20347] Updated weights for policy 0, policy_version 597524 (0.0010) [2023-12-27 09:46:08,870][20381] Updated weights for policy 1, policy_version 597168 (0.0007) [2023-12-27 09:46:08,882][20347] Updated weights for policy 0, policy_version 597534 (0.0008) [2023-12-27 09:46:08,926][20381] Updated weights for policy 1, policy_version 597178 (0.0008) [2023-12-27 09:46:09,601][20347] Updated weights for policy 0, policy_version 597544 (0.0008) [2023-12-27 09:46:09,617][20381] Updated weights for policy 1, policy_version 597188 (0.0005) [2023-12-27 09:46:09,656][20347] Updated weights for policy 0, policy_version 597554 (0.0007) [2023-12-27 09:46:09,668][20381] Updated weights for policy 1, policy_version 597198 (0.0005) [2023-12-27 09:46:09,714][20347] Updated weights for policy 0, policy_version 597564 (0.0007) [2023-12-27 09:46:09,723][20381] Updated weights for policy 1, policy_version 597208 (0.0005) [2023-12-27 09:46:10,386][20347] Updated weights for policy 0, policy_version 597574 (0.0009) [2023-12-27 09:46:10,414][20381] Updated weights for policy 1, policy_version 597218 (0.0006) [2023-12-27 09:46:10,447][20347] Updated weights for policy 0, policy_version 597584 (0.0010) [2023-12-27 09:46:10,470][20381] Updated weights for policy 1, policy_version 597228 (0.0008) [2023-12-27 09:46:10,507][20347] Updated weights for policy 0, policy_version 597594 (0.0009) [2023-12-27 09:46:10,515][20381] Updated weights for policy 1, policy_version 597238 (0.0008) [2023-12-27 09:46:10,559][20381] Updated weights for policy 1, policy_version 597248 (0.0007) [2023-12-27 09:46:11,141][20347] Updated weights for policy 0, policy_version 597604 (0.0010) [2023-12-27 09:46:11,202][20347] Updated weights for policy 0, policy_version 597614 (0.0010) [2023-12-27 09:46:11,220][20381] Updated weights for policy 1, policy_version 597258 (0.0007) [2023-12-27 09:46:11,265][20347] Updated weights for policy 0, policy_version 597624 (0.0010) [2023-12-27 09:46:11,278][20381] Updated weights for policy 1, policy_version 597268 (0.0008) [2023-12-27 09:46:11,339][20381] Updated weights for policy 1, policy_version 597278 (0.0008) [2023-12-27 09:46:11,957][20347] Updated weights for policy 0, policy_version 597634 (0.0009) [2023-12-27 09:46:12,014][20347] Updated weights for policy 0, policy_version 597644 (0.0007) [2023-12-27 09:46:12,068][20381] Updated weights for policy 1, policy_version 597288 (0.0006) [2023-12-27 09:46:12,075][20347] Updated weights for policy 0, policy_version 597654 (0.0006) [2023-12-27 09:46:12,128][20381] Updated weights for policy 1, policy_version 597298 (0.0006) [2023-12-27 09:46:12,129][20347] Updated weights for policy 0, policy_version 597664 (0.0008) [2023-12-27 09:46:12,180][20381] Updated weights for policy 1, policy_version 597308 (0.0007) [2023-12-27 09:46:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 305954816. Throughput: 0: 10521.8, 1: 10352.7. Samples: 305942568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:12,363][19170] Avg episode reward: [(0, '304.830'), (1, '990.290')] [2023-12-27 09:46:12,774][20347] Updated weights for policy 0, policy_version 597674 (0.0007) [2023-12-27 09:46:12,833][20347] Updated weights for policy 0, policy_version 597684 (0.0007) [2023-12-27 09:46:12,888][20347] Updated weights for policy 0, policy_version 597694 (0.0005) [2023-12-27 09:46:12,895][20381] Updated weights for policy 1, policy_version 597318 (0.0007) [2023-12-27 09:46:12,951][20381] Updated weights for policy 1, policy_version 597328 (0.0007) [2023-12-27 09:46:12,999][20381] Updated weights for policy 1, policy_version 597338 (0.0008) [2023-12-27 09:46:13,513][20347] Updated weights for policy 0, policy_version 597704 (0.0007) [2023-12-27 09:46:13,571][20347] Updated weights for policy 0, policy_version 597714 (0.0007) [2023-12-27 09:46:13,629][20347] Updated weights for policy 0, policy_version 597724 (0.0008) [2023-12-27 09:46:13,738][20381] Updated weights for policy 1, policy_version 597348 (0.0007) [2023-12-27 09:46:13,787][20381] Updated weights for policy 1, policy_version 597358 (0.0007) [2023-12-27 09:46:13,831][20381] Updated weights for policy 1, policy_version 597368 (0.0006) [2023-12-27 09:46:14,258][20347] Updated weights for policy 0, policy_version 597734 (0.0007) [2023-12-27 09:46:14,310][20347] Updated weights for policy 0, policy_version 597744 (0.0006) [2023-12-27 09:46:14,371][20347] Updated weights for policy 0, policy_version 597754 (0.0007) [2023-12-27 09:46:14,533][20381] Updated weights for policy 1, policy_version 597378 (0.0007) [2023-12-27 09:46:14,588][20381] Updated weights for policy 1, policy_version 597388 (0.0007) [2023-12-27 09:46:14,643][20381] Updated weights for policy 1, policy_version 597398 (0.0007) [2023-12-27 09:46:14,704][20381] Updated weights for policy 1, policy_version 597408 (0.0008) [2023-12-27 09:46:15,069][20347] Updated weights for policy 0, policy_version 597764 (0.0007) [2023-12-27 09:46:15,127][20347] Updated weights for policy 0, policy_version 597774 (0.0006) [2023-12-27 09:46:15,176][20347] Updated weights for policy 0, policy_version 597784 (0.0008) [2023-12-27 09:46:15,377][20381] Updated weights for policy 1, policy_version 597418 (0.0008) [2023-12-27 09:46:15,424][20381] Updated weights for policy 1, policy_version 597428 (0.0007) [2023-12-27 09:46:15,480][20381] Updated weights for policy 1, policy_version 597438 (0.0008) [2023-12-27 09:46:15,822][20347] Updated weights for policy 0, policy_version 597794 (0.0007) [2023-12-27 09:46:15,882][20347] Updated weights for policy 0, policy_version 597804 (0.0005) [2023-12-27 09:46:15,942][20347] Updated weights for policy 0, policy_version 597814 (0.0010) [2023-12-27 09:46:15,999][20347] Updated weights for policy 0, policy_version 597824 (0.0010) [2023-12-27 09:46:16,211][20381] Updated weights for policy 1, policy_version 597448 (0.0007) [2023-12-27 09:46:16,267][20381] Updated weights for policy 1, policy_version 597458 (0.0008) [2023-12-27 09:46:16,322][20381] Updated weights for policy 1, policy_version 597468 (0.0008) [2023-12-27 09:46:16,626][20347] Updated weights for policy 0, policy_version 597834 (0.0010) [2023-12-27 09:46:16,677][20347] Updated weights for policy 0, policy_version 597844 (0.0010) [2023-12-27 09:46:16,737][20347] Updated weights for policy 0, policy_version 597854 (0.0010) [2023-12-27 09:46:17,007][20381] Updated weights for policy 1, policy_version 597478 (0.0009) [2023-12-27 09:46:17,071][20381] Updated weights for policy 1, policy_version 597488 (0.0011) [2023-12-27 09:46:17,126][20381] Updated weights for policy 1, policy_version 597498 (0.0010) [2023-12-27 09:46:17,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 306053120. Throughput: 0: 10488.4, 1: 10228.5. Samples: 306064292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:17,364][19170] Avg episode reward: [(0, '363.860'), (1, '1000.000')] [2023-12-27 09:46:17,418][20347] Updated weights for policy 0, policy_version 597864 (0.0008) [2023-12-27 09:46:17,470][20347] Updated weights for policy 0, policy_version 597874 (0.0007) [2023-12-27 09:46:17,528][20347] Updated weights for policy 0, policy_version 597884 (0.0005) [2023-12-27 09:46:17,726][20381] Updated weights for policy 1, policy_version 597508 (0.0010) [2023-12-27 09:46:17,781][20381] Updated weights for policy 1, policy_version 597518 (0.0007) [2023-12-27 09:46:17,837][20381] Updated weights for policy 1, policy_version 597528 (0.0007) [2023-12-27 09:46:18,100][20347] Updated weights for policy 0, policy_version 597894 (0.0005) [2023-12-27 09:46:18,148][20347] Updated weights for policy 0, policy_version 597904 (0.0005) [2023-12-27 09:46:18,200][20347] Updated weights for policy 0, policy_version 597914 (0.0007) [2023-12-27 09:46:18,447][20381] Updated weights for policy 1, policy_version 597538 (0.0007) [2023-12-27 09:46:18,510][20381] Updated weights for policy 1, policy_version 597548 (0.0005) [2023-12-27 09:46:18,564][20381] Updated weights for policy 1, policy_version 597558 (0.0007) [2023-12-27 09:46:18,621][20381] Updated weights for policy 1, policy_version 597568 (0.0008) [2023-12-27 09:46:18,857][20347] Updated weights for policy 0, policy_version 597924 (0.0008) [2023-12-27 09:46:18,916][20347] Updated weights for policy 0, policy_version 597934 (0.0008) [2023-12-27 09:46:18,977][20347] Updated weights for policy 0, policy_version 597944 (0.0007) [2023-12-27 09:46:19,291][20381] Updated weights for policy 1, policy_version 597578 (0.0008) [2023-12-27 09:46:19,351][20381] Updated weights for policy 1, policy_version 597588 (0.0008) [2023-12-27 09:46:19,412][20381] Updated weights for policy 1, policy_version 597598 (0.0007) [2023-12-27 09:46:19,742][20347] Updated weights for policy 0, policy_version 597954 (0.0008) [2023-12-27 09:46:19,795][20347] Updated weights for policy 0, policy_version 597964 (0.0007) [2023-12-27 09:46:19,864][20347] Updated weights for policy 0, policy_version 597974 (0.0008) [2023-12-27 09:46:19,930][20347] Updated weights for policy 0, policy_version 597984 (0.0008) [2023-12-27 09:46:20,092][20381] Updated weights for policy 1, policy_version 597608 (0.0007) [2023-12-27 09:46:20,147][20381] Updated weights for policy 1, policy_version 597618 (0.0008) [2023-12-27 09:46:20,213][20381] Updated weights for policy 1, policy_version 597628 (0.0006) [2023-12-27 09:46:20,664][20347] Updated weights for policy 0, policy_version 597994 (0.0008) [2023-12-27 09:46:20,720][20347] Updated weights for policy 0, policy_version 598004 (0.0008) [2023-12-27 09:46:20,775][20347] Updated weights for policy 0, policy_version 598014 (0.0007) [2023-12-27 09:46:20,843][20381] Updated weights for policy 1, policy_version 597638 (0.0009) [2023-12-27 09:46:20,901][20381] Updated weights for policy 1, policy_version 597648 (0.0010) [2023-12-27 09:46:20,953][20381] Updated weights for policy 1, policy_version 597658 (0.0010) [2023-12-27 09:46:21,513][20347] Updated weights for policy 0, policy_version 598024 (0.0008) [2023-12-27 09:46:21,577][20347] Updated weights for policy 0, policy_version 598034 (0.0008) [2023-12-27 09:46:21,642][20347] Updated weights for policy 0, policy_version 598044 (0.0007) [2023-12-27 09:46:21,718][20381] Updated weights for policy 1, policy_version 597668 (0.0010) [2023-12-27 09:46:21,779][20381] Updated weights for policy 1, policy_version 597678 (0.0008) [2023-12-27 09:46:21,838][20381] Updated weights for policy 1, policy_version 597688 (0.0007) [2023-12-27 09:46:22,355][20347] Updated weights for policy 0, policy_version 598054 (0.0007) [2023-12-27 09:46:22,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 306151424. Throughput: 0: 10438.9, 1: 10223.6. Samples: 306126920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:22,363][19170] Avg episode reward: [(0, '257.820'), (1, '980.560')] [2023-12-27 09:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000597696_153034752.pth... [2023-12-27 09:46:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000596512_152731648.pth [2023-12-27 09:46:22,419][20347] Updated weights for policy 0, policy_version 598064 (0.0007) [2023-12-27 09:46:22,481][20347] Updated weights for policy 0, policy_version 598074 (0.0008) [2023-12-27 09:46:22,517][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000598080_153124864.pth... [2023-12-27 09:46:22,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000596864_152813568.pth [2023-12-27 09:46:22,545][20381] Updated weights for policy 1, policy_version 597698 (0.0008) [2023-12-27 09:46:22,605][20381] Updated weights for policy 1, policy_version 597708 (0.0008) [2023-12-27 09:46:22,623][20292] KL-divergence is very high: 401.9409 [2023-12-27 09:46:22,663][20381] Updated weights for policy 1, policy_version 597718 (0.0006) [2023-12-27 09:46:22,719][20381] Updated weights for policy 1, policy_version 597728 (0.0008) [2023-12-27 09:46:23,219][20347] Updated weights for policy 0, policy_version 598084 (0.0008) [2023-12-27 09:46:23,274][20347] Updated weights for policy 0, policy_version 598094 (0.0005) [2023-12-27 09:46:23,328][20347] Updated weights for policy 0, policy_version 598104 (0.0007) [2023-12-27 09:46:23,348][20381] Updated weights for policy 1, policy_version 597738 (0.0007) [2023-12-27 09:46:23,398][20381] Updated weights for policy 1, policy_version 597748 (0.0007) [2023-12-27 09:46:23,451][20381] Updated weights for policy 1, policy_version 597758 (0.0008) [2023-12-27 09:46:23,958][20347] Updated weights for policy 0, policy_version 598114 (0.0006) [2023-12-27 09:46:24,022][20347] Updated weights for policy 0, policy_version 598124 (0.0008) [2023-12-27 09:46:24,071][20347] Updated weights for policy 0, policy_version 598134 (0.0008) [2023-12-27 09:46:24,127][20347] Updated weights for policy 0, policy_version 598144 (0.0008) [2023-12-27 09:46:24,132][20381] Updated weights for policy 1, policy_version 597768 (0.0006) [2023-12-27 09:46:24,182][20381] Updated weights for policy 1, policy_version 597778 (0.0006) [2023-12-27 09:46:24,238][20381] Updated weights for policy 1, policy_version 597788 (0.0007) [2023-12-27 09:46:24,830][20347] Updated weights for policy 0, policy_version 598154 (0.0008) [2023-12-27 09:46:24,880][20347] Updated weights for policy 0, policy_version 598164 (0.0006) [2023-12-27 09:46:24,882][20381] Updated weights for policy 1, policy_version 597798 (0.0006) [2023-12-27 09:46:24,932][20347] Updated weights for policy 0, policy_version 598174 (0.0005) [2023-12-27 09:46:24,937][20381] Updated weights for policy 1, policy_version 597808 (0.0006) [2023-12-27 09:46:24,992][20381] Updated weights for policy 1, policy_version 597818 (0.0007) [2023-12-27 09:46:25,603][20347] Updated weights for policy 0, policy_version 598184 (0.0008) [2023-12-27 09:46:25,641][20381] Updated weights for policy 1, policy_version 597828 (0.0008) [2023-12-27 09:46:25,655][20347] Updated weights for policy 0, policy_version 598194 (0.0006) [2023-12-27 09:46:25,658][20292] KL-divergence is very high: 154.5722 [2023-12-27 09:46:25,668][20292] KL-divergence is very high: 117.9134 [2023-12-27 09:46:25,690][20381] Updated weights for policy 1, policy_version 597838 (0.0011) [2023-12-27 09:46:25,704][20347] Updated weights for policy 0, policy_version 598204 (0.0005) [2023-12-27 09:46:25,746][20381] Updated weights for policy 1, policy_version 597848 (0.0011) [2023-12-27 09:46:26,334][20347] Updated weights for policy 0, policy_version 598214 (0.0005) [2023-12-27 09:46:26,392][20347] Updated weights for policy 0, policy_version 598224 (0.0005) [2023-12-27 09:46:26,419][20381] Updated weights for policy 1, policy_version 597858 (0.0010) [2023-12-27 09:46:26,445][20347] Updated weights for policy 0, policy_version 598234 (0.0005) [2023-12-27 09:46:26,471][20381] Updated weights for policy 1, policy_version 597868 (0.0010) [2023-12-27 09:46:26,501][20292] KL-divergence is very high: 253.5877 [2023-12-27 09:46:26,526][20381] Updated weights for policy 1, policy_version 597878 (0.0010) [2023-12-27 09:46:26,552][20292] KL-divergence is very high: 338.6607 [2023-12-27 09:46:26,590][20381] Updated weights for policy 1, policy_version 597888 (0.0011) [2023-12-27 09:46:26,961][20347] Updated weights for policy 0, policy_version 598244 (0.0006) [2023-12-27 09:46:27,011][20347] Updated weights for policy 0, policy_version 598254 (0.0007) [2023-12-27 09:46:27,071][20347] Updated weights for policy 0, policy_version 598264 (0.0007) [2023-12-27 09:46:27,226][20381] Updated weights for policy 1, policy_version 597898 (0.0011) [2023-12-27 09:46:27,279][20381] Updated weights for policy 1, policy_version 597908 (0.0005) [2023-12-27 09:46:27,336][20381] Updated weights for policy 1, policy_version 597918 (0.0006) [2023-12-27 09:46:27,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 306266112. Throughput: 0: 10348.6, 1: 10253.4. Samples: 306248232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:27,363][19170] Avg episode reward: [(0, '311.660'), (1, '883.350')] [2023-12-27 09:46:27,731][20347] Updated weights for policy 0, policy_version 598274 (0.0007) [2023-12-27 09:46:27,788][20347] Updated weights for policy 0, policy_version 598284 (0.0006) [2023-12-27 09:46:27,842][20347] Updated weights for policy 0, policy_version 598294 (0.0008) [2023-12-27 09:46:27,898][20347] Updated weights for policy 0, policy_version 598304 (0.0008) [2023-12-27 09:46:27,911][20381] Updated weights for policy 1, policy_version 597928 (0.0007) [2023-12-27 09:46:27,965][20381] Updated weights for policy 1, policy_version 597938 (0.0007) [2023-12-27 09:46:28,010][20381] Updated weights for policy 1, policy_version 597948 (0.0008) [2023-12-27 09:46:28,616][20347] Updated weights for policy 0, policy_version 598314 (0.0007) [2023-12-27 09:46:28,668][20347] Updated weights for policy 0, policy_version 598324 (0.0007) [2023-12-27 09:46:28,681][20381] Updated weights for policy 1, policy_version 597958 (0.0007) [2023-12-27 09:46:28,725][20347] Updated weights for policy 0, policy_version 598334 (0.0006) [2023-12-27 09:46:28,735][20381] Updated weights for policy 1, policy_version 597968 (0.0005) [2023-12-27 09:46:28,787][20381] Updated weights for policy 1, policy_version 597978 (0.0008) [2023-12-27 09:46:29,450][20347] Updated weights for policy 0, policy_version 598344 (0.0007) [2023-12-27 09:46:29,468][20381] Updated weights for policy 1, policy_version 597988 (0.0007) [2023-12-27 09:46:29,503][20347] Updated weights for policy 0, policy_version 598354 (0.0008) [2023-12-27 09:46:29,520][20381] Updated weights for policy 1, policy_version 597998 (0.0006) [2023-12-27 09:46:29,558][20347] Updated weights for policy 0, policy_version 598364 (0.0007) [2023-12-27 09:46:29,575][20381] Updated weights for policy 1, policy_version 598008 (0.0006) [2023-12-27 09:46:30,245][20347] Updated weights for policy 0, policy_version 598374 (0.0005) [2023-12-27 09:46:30,287][20381] Updated weights for policy 1, policy_version 598018 (0.0007) [2023-12-27 09:46:30,304][20347] Updated weights for policy 0, policy_version 598384 (0.0007) [2023-12-27 09:46:30,342][20381] Updated weights for policy 1, policy_version 598028 (0.0009) [2023-12-27 09:46:30,364][20347] Updated weights for policy 0, policy_version 598394 (0.0009) [2023-12-27 09:46:30,407][20381] Updated weights for policy 1, policy_version 598038 (0.0008) [2023-12-27 09:46:30,467][20381] Updated weights for policy 1, policy_version 598048 (0.0008) [2023-12-27 09:46:30,905][20347] Updated weights for policy 0, policy_version 598404 (0.0006) [2023-12-27 09:46:30,951][20347] Updated weights for policy 0, policy_version 598414 (0.0005) [2023-12-27 09:46:30,998][20347] Updated weights for policy 0, policy_version 598424 (0.0005) [2023-12-27 09:46:31,015][20381] Updated weights for policy 1, policy_version 598058 (0.0007) [2023-12-27 09:46:31,078][20381] Updated weights for policy 1, policy_version 598068 (0.0007) [2023-12-27 09:46:31,130][20381] Updated weights for policy 1, policy_version 598078 (0.0008) [2023-12-27 09:46:31,644][20347] Updated weights for policy 0, policy_version 598434 (0.0009) [2023-12-27 09:46:31,696][20347] Updated weights for policy 0, policy_version 598444 (0.0008) [2023-12-27 09:46:31,760][20347] Updated weights for policy 0, policy_version 598454 (0.0008) [2023-12-27 09:46:31,817][20347] Updated weights for policy 0, policy_version 598464 (0.0006) [2023-12-27 09:46:31,818][20381] Updated weights for policy 1, policy_version 598088 (0.0007) [2023-12-27 09:46:31,870][20381] Updated weights for policy 1, policy_version 598098 (0.0008) [2023-12-27 09:46:31,928][20381] Updated weights for policy 1, policy_version 598108 (0.0007) [2023-12-27 09:46:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 306364416. Throughput: 0: 10391.9, 1: 10308.7. Samples: 306376712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:32,363][19170] Avg episode reward: [(0, '271.710'), (1, '834.950')] [2023-12-27 09:46:32,507][20381] Updated weights for policy 1, policy_version 598118 (0.0005) [2023-12-27 09:46:32,560][20381] Updated weights for policy 1, policy_version 598128 (0.0005) [2023-12-27 09:46:32,570][20347] Updated weights for policy 0, policy_version 598474 (0.0008) [2023-12-27 09:46:32,613][20381] Updated weights for policy 1, policy_version 598138 (0.0006) [2023-12-27 09:46:32,622][20347] Updated weights for policy 0, policy_version 598484 (0.0006) [2023-12-27 09:46:32,683][20347] Updated weights for policy 0, policy_version 598494 (0.0007) [2023-12-27 09:46:33,306][20381] Updated weights for policy 1, policy_version 598148 (0.0006) [2023-12-27 09:46:33,312][20347] Updated weights for policy 0, policy_version 598504 (0.0006) [2023-12-27 09:46:33,349][20381] Updated weights for policy 1, policy_version 598158 (0.0006) [2023-12-27 09:46:33,358][20347] Updated weights for policy 0, policy_version 598514 (0.0005) [2023-12-27 09:46:33,404][20381] Updated weights for policy 1, policy_version 598168 (0.0006) [2023-12-27 09:46:33,409][20347] Updated weights for policy 0, policy_version 598524 (0.0005) [2023-12-27 09:46:33,990][20347] Updated weights for policy 0, policy_version 598534 (0.0008) [2023-12-27 09:46:34,047][20347] Updated weights for policy 0, policy_version 598544 (0.0010) [2023-12-27 09:46:34,104][20347] Updated weights for policy 0, policy_version 598554 (0.0010) [2023-12-27 09:46:34,112][20381] Updated weights for policy 1, policy_version 598178 (0.0007) [2023-12-27 09:46:34,173][20381] Updated weights for policy 1, policy_version 598188 (0.0008) [2023-12-27 09:46:34,225][20381] Updated weights for policy 1, policy_version 598198 (0.0007) [2023-12-27 09:46:34,280][20381] Updated weights for policy 1, policy_version 598208 (0.0007) [2023-12-27 09:46:34,811][20347] Updated weights for policy 0, policy_version 598564 (0.0010) [2023-12-27 09:46:34,869][20347] Updated weights for policy 0, policy_version 598574 (0.0010) [2023-12-27 09:46:34,926][20347] Updated weights for policy 0, policy_version 598584 (0.0010) [2023-12-27 09:46:34,973][20381] Updated weights for policy 1, policy_version 598218 (0.0006) [2023-12-27 09:46:35,026][20381] Updated weights for policy 1, policy_version 598228 (0.0007) [2023-12-27 09:46:35,073][20381] Updated weights for policy 1, policy_version 598238 (0.0005) [2023-12-27 09:46:35,579][20347] Updated weights for policy 0, policy_version 598594 (0.0010) [2023-12-27 09:46:35,637][20347] Updated weights for policy 0, policy_version 598604 (0.0010) [2023-12-27 09:46:35,697][20347] Updated weights for policy 0, policy_version 598614 (0.0010) [2023-12-27 09:46:35,754][20347] Updated weights for policy 0, policy_version 598624 (0.0009) [2023-12-27 09:46:35,766][20381] Updated weights for policy 1, policy_version 598248 (0.0006) [2023-12-27 09:46:35,819][20381] Updated weights for policy 1, policy_version 598258 (0.0008) [2023-12-27 09:46:35,876][20381] Updated weights for policy 1, policy_version 598268 (0.0007) [2023-12-27 09:46:36,411][20347] Updated weights for policy 0, policy_version 598634 (0.0007) [2023-12-27 09:46:36,479][20347] Updated weights for policy 0, policy_version 598644 (0.0008) [2023-12-27 09:46:36,537][20347] Updated weights for policy 0, policy_version 598654 (0.0007) [2023-12-27 09:46:36,554][20381] Updated weights for policy 1, policy_version 598278 (0.0008) [2023-12-27 09:46:36,572][20292] KL-divergence is very high: 214.3714 [2023-12-27 09:46:36,616][20381] Updated weights for policy 1, policy_version 598288 (0.0008) [2023-12-27 09:46:36,624][20292] KL-divergence is very high: 293.5271 [2023-12-27 09:46:36,673][20292] KL-divergence is very high: 239.5832 [2023-12-27 09:46:36,679][20381] Updated weights for policy 1, policy_version 598298 (0.0007) [2023-12-27 09:46:37,176][20347] Updated weights for policy 0, policy_version 598664 (0.0009) [2023-12-27 09:46:37,232][20347] Updated weights for policy 0, policy_version 598674 (0.0010) [2023-12-27 09:46:37,290][20347] Updated weights for policy 0, policy_version 598684 (0.0006) [2023-12-27 09:46:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 306470912. Throughput: 0: 10390.3, 1: 10306.3. Samples: 306439252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:37,363][19170] Avg episode reward: [(0, '283.380'), (1, '835.250')] [2023-12-27 09:46:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000598688_153280512.pth... [2023-12-27 09:46:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000597472_152969216.pth [2023-12-27 09:46:37,388][20381] Updated weights for policy 1, policy_version 598308 (0.0007) [2023-12-27 09:46:37,455][20381] Updated weights for policy 1, policy_version 598318 (0.0007) [2023-12-27 09:46:37,522][20381] Updated weights for policy 1, policy_version 598328 (0.0008) [2023-12-27 09:46:37,572][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000598336_153198592.pth... [2023-12-27 09:46:37,576][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000597120_152887296.pth [2023-12-27 09:46:37,970][20347] Updated weights for policy 0, policy_version 598694 (0.0007) [2023-12-27 09:46:38,032][20347] Updated weights for policy 0, policy_version 598704 (0.0008) [2023-12-27 09:46:38,089][20347] Updated weights for policy 0, policy_version 598714 (0.0007) [2023-12-27 09:46:38,169][20381] Updated weights for policy 1, policy_version 598338 (0.0006) [2023-12-27 09:46:38,227][20381] Updated weights for policy 1, policy_version 598348 (0.0008) [2023-12-27 09:46:38,291][20381] Updated weights for policy 1, policy_version 598358 (0.0007) [2023-12-27 09:46:38,355][20381] Updated weights for policy 1, policy_version 598368 (0.0008) [2023-12-27 09:46:38,785][20347] Updated weights for policy 0, policy_version 598724 (0.0008) [2023-12-27 09:46:38,846][20347] Updated weights for policy 0, policy_version 598734 (0.0008) [2023-12-27 09:46:38,905][20347] Updated weights for policy 0, policy_version 598744 (0.0007) [2023-12-27 09:46:39,053][20381] Updated weights for policy 1, policy_version 598378 (0.0008) [2023-12-27 09:46:39,064][20292] KL-divergence is very high: 153.0963 [2023-12-27 09:46:39,070][20292] KL-divergence is very high: 128.4320 [2023-12-27 09:46:39,113][20381] Updated weights for policy 1, policy_version 598388 (0.0006) [2023-12-27 09:46:39,166][20381] Updated weights for policy 1, policy_version 598398 (0.0005) [2023-12-27 09:46:39,546][20347] Updated weights for policy 0, policy_version 598754 (0.0007) [2023-12-27 09:46:39,605][20347] Updated weights for policy 0, policy_version 598764 (0.0006) [2023-12-27 09:46:39,666][20347] Updated weights for policy 0, policy_version 598774 (0.0008) [2023-12-27 09:46:39,722][20347] Updated weights for policy 0, policy_version 598784 (0.0008) [2023-12-27 09:46:39,861][20381] Updated weights for policy 1, policy_version 598408 (0.0007) [2023-12-27 09:46:39,922][20381] Updated weights for policy 1, policy_version 598418 (0.0007) [2023-12-27 09:46:39,984][20381] Updated weights for policy 1, policy_version 598428 (0.0008) [2023-12-27 09:46:40,351][20347] Updated weights for policy 0, policy_version 598794 (0.0009) [2023-12-27 09:46:40,401][20347] Updated weights for policy 0, policy_version 598804 (0.0009) [2023-12-27 09:46:40,450][20347] Updated weights for policy 0, policy_version 598814 (0.0007) [2023-12-27 09:46:40,664][20381] Updated weights for policy 1, policy_version 598438 (0.0010) [2023-12-27 09:46:40,710][20381] Updated weights for policy 1, policy_version 598448 (0.0010) [2023-12-27 09:46:40,764][20381] Updated weights for policy 1, policy_version 598458 (0.0010) [2023-12-27 09:46:41,105][20347] Updated weights for policy 0, policy_version 598824 (0.0007) [2023-12-27 09:46:41,174][20347] Updated weights for policy 0, policy_version 598834 (0.0007) [2023-12-27 09:46:41,227][20347] Updated weights for policy 0, policy_version 598844 (0.0007) [2023-12-27 09:46:41,516][20381] Updated weights for policy 1, policy_version 598468 (0.0010) [2023-12-27 09:46:41,577][20381] Updated weights for policy 1, policy_version 598478 (0.0011) [2023-12-27 09:46:41,639][20381] Updated weights for policy 1, policy_version 598488 (0.0012) [2023-12-27 09:46:42,002][20347] Updated weights for policy 0, policy_version 598854 (0.0008) [2023-12-27 09:46:42,056][20347] Updated weights for policy 0, policy_version 598864 (0.0005) [2023-12-27 09:46:42,115][20347] Updated weights for policy 0, policy_version 598874 (0.0005) [2023-12-27 09:46:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 306569216. Throughput: 0: 10412.6, 1: 10264.8. Samples: 306561864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:42,363][19170] Avg episode reward: [(0, '260.650'), (1, '845.720')] [2023-12-27 09:46:42,368][20381] Updated weights for policy 1, policy_version 598498 (0.0010) [2023-12-27 09:46:42,424][20381] Updated weights for policy 1, policy_version 598508 (0.0008) [2023-12-27 09:46:42,473][20381] Updated weights for policy 1, policy_version 598518 (0.0007) [2023-12-27 09:46:42,526][20381] Updated weights for policy 1, policy_version 598528 (0.0009) [2023-12-27 09:46:42,752][20347] Updated weights for policy 0, policy_version 598884 (0.0006) [2023-12-27 09:46:42,806][20347] Updated weights for policy 0, policy_version 598894 (0.0007) [2023-12-27 09:46:42,864][20347] Updated weights for policy 0, policy_version 598904 (0.0007) [2023-12-27 09:46:43,180][20381] Updated weights for policy 1, policy_version 598538 (0.0005) [2023-12-27 09:46:43,244][20381] Updated weights for policy 1, policy_version 598548 (0.0007) [2023-12-27 09:46:43,307][20381] Updated weights for policy 1, policy_version 598558 (0.0007) [2023-12-27 09:46:43,486][20347] Updated weights for policy 0, policy_version 598914 (0.0010) [2023-12-27 09:46:43,544][20347] Updated weights for policy 0, policy_version 598924 (0.0010) [2023-12-27 09:46:43,600][20347] Updated weights for policy 0, policy_version 598934 (0.0010) [2023-12-27 09:46:43,658][20347] Updated weights for policy 0, policy_version 598944 (0.0010) [2023-12-27 09:46:43,938][20381] Updated weights for policy 1, policy_version 598568 (0.0006) [2023-12-27 09:46:43,986][20381] Updated weights for policy 1, policy_version 598578 (0.0005) [2023-12-27 09:46:44,035][20381] Updated weights for policy 1, policy_version 598588 (0.0005) [2023-12-27 09:46:44,279][20347] Updated weights for policy 0, policy_version 598954 (0.0010) [2023-12-27 09:46:44,331][20347] Updated weights for policy 0, policy_version 598964 (0.0010) [2023-12-27 09:46:44,392][20347] Updated weights for policy 0, policy_version 598974 (0.0010) [2023-12-27 09:46:44,603][20381] Updated weights for policy 1, policy_version 598598 (0.0008) [2023-12-27 09:46:44,655][20381] Updated weights for policy 1, policy_version 598608 (0.0011) [2023-12-27 09:46:44,714][20381] Updated weights for policy 1, policy_version 598618 (0.0008) [2023-12-27 09:46:45,098][20347] Updated weights for policy 0, policy_version 598984 (0.0007) [2023-12-27 09:46:45,158][20347] Updated weights for policy 0, policy_version 598994 (0.0008) [2023-12-27 09:46:45,217][20347] Updated weights for policy 0, policy_version 599004 (0.0007) [2023-12-27 09:46:45,361][20381] Updated weights for policy 1, policy_version 598628 (0.0006) [2023-12-27 09:46:45,409][20381] Updated weights for policy 1, policy_version 598638 (0.0010) [2023-12-27 09:46:45,461][20381] Updated weights for policy 1, policy_version 598648 (0.0010) [2023-12-27 09:46:45,935][20347] Updated weights for policy 0, policy_version 599014 (0.0008) [2023-12-27 09:46:46,000][20347] Updated weights for policy 0, policy_version 599024 (0.0007) [2023-12-27 09:46:46,052][20347] Updated weights for policy 0, policy_version 599034 (0.0007) [2023-12-27 09:46:46,103][20381] Updated weights for policy 1, policy_version 598658 (0.0008) [2023-12-27 09:46:46,152][20381] Updated weights for policy 1, policy_version 598668 (0.0008) [2023-12-27 09:46:46,203][20381] Updated weights for policy 1, policy_version 598678 (0.0008) [2023-12-27 09:46:46,251][20381] Updated weights for policy 1, policy_version 598688 (0.0008) [2023-12-27 09:46:46,708][20347] Updated weights for policy 0, policy_version 599044 (0.0009) [2023-12-27 09:46:46,753][20347] Updated weights for policy 0, policy_version 599054 (0.0010) [2023-12-27 09:46:46,802][20347] Updated weights for policy 0, policy_version 599064 (0.0010) [2023-12-27 09:46:46,957][20381] Updated weights for policy 1, policy_version 598698 (0.0007) [2023-12-27 09:46:47,004][20381] Updated weights for policy 1, policy_version 598708 (0.0008) [2023-12-27 09:46:47,053][20381] Updated weights for policy 1, policy_version 598718 (0.0008) [2023-12-27 09:46:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 306675712. Throughput: 0: 10383.6, 1: 10257.5. Samples: 306686080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:47,363][19170] Avg episode reward: [(0, '229.270'), (1, '894.000')] [2023-12-27 09:46:47,498][20347] Updated weights for policy 0, policy_version 599074 (0.0010) [2023-12-27 09:46:47,549][20347] Updated weights for policy 0, policy_version 599084 (0.0010) [2023-12-27 09:46:47,598][20347] Updated weights for policy 0, policy_version 599094 (0.0010) [2023-12-27 09:46:47,646][20347] Updated weights for policy 0, policy_version 599104 (0.0010) [2023-12-27 09:46:47,739][20381] Updated weights for policy 1, policy_version 598728 (0.0010) [2023-12-27 09:46:47,790][20381] Updated weights for policy 1, policy_version 598738 (0.0010) [2023-12-27 09:46:47,839][20381] Updated weights for policy 1, policy_version 598748 (0.0008) [2023-12-27 09:46:48,335][20347] Updated weights for policy 0, policy_version 599114 (0.0009) [2023-12-27 09:46:48,397][20347] Updated weights for policy 0, policy_version 599124 (0.0010) [2023-12-27 09:46:48,459][20347] Updated weights for policy 0, policy_version 599134 (0.0010) [2023-12-27 09:46:48,490][20381] Updated weights for policy 1, policy_version 598758 (0.0007) [2023-12-27 09:46:48,547][20381] Updated weights for policy 1, policy_version 598768 (0.0007) [2023-12-27 09:46:48,609][20381] Updated weights for policy 1, policy_version 598778 (0.0005) [2023-12-27 09:46:49,128][20347] Updated weights for policy 0, policy_version 599144 (0.0010) [2023-12-27 09:46:49,184][20347] Updated weights for policy 0, policy_version 599154 (0.0011) [2023-12-27 09:46:49,249][20381] Updated weights for policy 1, policy_version 598788 (0.0009) [2023-12-27 09:46:49,255][20347] Updated weights for policy 0, policy_version 599164 (0.0010) [2023-12-27 09:46:49,310][20381] Updated weights for policy 1, policy_version 598798 (0.0008) [2023-12-27 09:46:49,377][20381] Updated weights for policy 1, policy_version 598808 (0.0008) [2023-12-27 09:46:49,975][20347] Updated weights for policy 0, policy_version 599174 (0.0007) [2023-12-27 09:46:50,030][20347] Updated weights for policy 0, policy_version 599184 (0.0007) [2023-12-27 09:46:50,044][20381] Updated weights for policy 1, policy_version 598818 (0.0006) [2023-12-27 09:46:50,087][20347] Updated weights for policy 0, policy_version 599194 (0.0006) [2023-12-27 09:46:50,093][20381] Updated weights for policy 1, policy_version 598828 (0.0008) [2023-12-27 09:46:50,138][20381] Updated weights for policy 1, policy_version 598838 (0.0008) [2023-12-27 09:46:50,187][20381] Updated weights for policy 1, policy_version 598848 (0.0008) [2023-12-27 09:46:50,775][20347] Updated weights for policy 0, policy_version 599204 (0.0007) [2023-12-27 09:46:50,840][20347] Updated weights for policy 0, policy_version 599214 (0.0008) [2023-12-27 09:46:50,908][20347] Updated weights for policy 0, policy_version 599224 (0.0007) [2023-12-27 09:46:50,972][20381] Updated weights for policy 1, policy_version 598858 (0.0011) [2023-12-27 09:46:51,027][20381] Updated weights for policy 1, policy_version 598868 (0.0011) [2023-12-27 09:46:51,082][20381] Updated weights for policy 1, policy_version 598878 (0.0008) [2023-12-27 09:46:51,574][20347] Updated weights for policy 0, policy_version 599234 (0.0008) [2023-12-27 09:46:51,634][20347] Updated weights for policy 0, policy_version 599244 (0.0007) [2023-12-27 09:46:51,695][20347] Updated weights for policy 0, policy_version 599254 (0.0005) [2023-12-27 09:46:51,761][20347] Updated weights for policy 0, policy_version 599264 (0.0007) [2023-12-27 09:46:51,875][20381] Updated weights for policy 1, policy_version 598888 (0.0007) [2023-12-27 09:46:51,904][20292] KL-divergence is very high: 136.5787 [2023-12-27 09:46:51,920][20292] KL-divergence is very high: 101.8291 [2023-12-27 09:46:51,923][20381] Updated weights for policy 1, policy_version 598898 (0.0008) [2023-12-27 09:46:51,982][20381] Updated weights for policy 1, policy_version 598908 (0.0008) [2023-12-27 09:46:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 306774016. Throughput: 0: 10353.2, 1: 10288.5. Samples: 306747484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:52,363][19170] Avg episode reward: [(0, '352.040'), (1, '922.360')] [2023-12-27 09:46:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000598912_153346048.pth... [2023-12-27 09:46:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000597696_153034752.pth [2023-12-27 09:46:52,398][20347] Updated weights for policy 0, policy_version 599274 (0.0008) [2023-12-27 09:46:52,463][20347] Updated weights for policy 0, policy_version 599284 (0.0006) [2023-12-27 09:46:52,515][20347] Updated weights for policy 0, policy_version 599294 (0.0006) [2023-12-27 09:46:52,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000599296_153436160.pth... [2023-12-27 09:46:52,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000598080_153124864.pth [2023-12-27 09:46:52,718][20381] Updated weights for policy 1, policy_version 598918 (0.0008) [2023-12-27 09:46:52,775][20381] Updated weights for policy 1, policy_version 598928 (0.0008) [2023-12-27 09:46:52,833][20381] Updated weights for policy 1, policy_version 598938 (0.0008) [2023-12-27 09:46:53,131][20347] Updated weights for policy 0, policy_version 599304 (0.0006) [2023-12-27 09:46:53,185][20347] Updated weights for policy 0, policy_version 599314 (0.0007) [2023-12-27 09:46:53,229][20347] Updated weights for policy 0, policy_version 599324 (0.0009) [2023-12-27 09:46:53,470][20381] Updated weights for policy 1, policy_version 598948 (0.0006) [2023-12-27 09:46:53,521][20381] Updated weights for policy 1, policy_version 598958 (0.0007) [2023-12-27 09:46:53,565][20381] Updated weights for policy 1, policy_version 598968 (0.0007) [2023-12-27 09:46:53,962][20347] Updated weights for policy 0, policy_version 599334 (0.0008) [2023-12-27 09:46:54,018][20347] Updated weights for policy 0, policy_version 599344 (0.0007) [2023-12-27 09:46:54,075][20347] Updated weights for policy 0, policy_version 599354 (0.0007) [2023-12-27 09:46:54,172][20381] Updated weights for policy 1, policy_version 598978 (0.0007) [2023-12-27 09:46:54,228][20381] Updated weights for policy 1, policy_version 598988 (0.0008) [2023-12-27 09:46:54,281][20381] Updated weights for policy 1, policy_version 598998 (0.0007) [2023-12-27 09:46:54,336][20381] Updated weights for policy 1, policy_version 599008 (0.0005) [2023-12-27 09:46:54,723][20347] Updated weights for policy 0, policy_version 599364 (0.0007) [2023-12-27 09:46:54,782][20347] Updated weights for policy 0, policy_version 599374 (0.0008) [2023-12-27 09:46:54,845][20347] Updated weights for policy 0, policy_version 599384 (0.0008) [2023-12-27 09:46:54,928][20381] Updated weights for policy 1, policy_version 599018 (0.0005) [2023-12-27 09:46:54,982][20381] Updated weights for policy 1, policy_version 599028 (0.0006) [2023-12-27 09:46:55,038][20381] Updated weights for policy 1, policy_version 599038 (0.0005) [2023-12-27 09:46:55,515][20347] Updated weights for policy 0, policy_version 599394 (0.0008) [2023-12-27 09:46:55,570][20347] Updated weights for policy 0, policy_version 599404 (0.0008) [2023-12-27 09:46:55,618][20347] Updated weights for policy 0, policy_version 599414 (0.0007) [2023-12-27 09:46:55,664][20347] Updated weights for policy 0, policy_version 599424 (0.0006) [2023-12-27 09:46:55,677][20381] Updated weights for policy 1, policy_version 599048 (0.0006) [2023-12-27 09:46:55,739][20381] Updated weights for policy 1, policy_version 599058 (0.0008) [2023-12-27 09:46:55,780][20381] Updated weights for policy 1, policy_version 599068 (0.0007) [2023-12-27 09:46:56,362][20347] Updated weights for policy 0, policy_version 599434 (0.0005) [2023-12-27 09:46:56,418][20347] Updated weights for policy 0, policy_version 599444 (0.0007) [2023-12-27 09:46:56,476][20347] Updated weights for policy 0, policy_version 599454 (0.0007) [2023-12-27 09:46:56,485][20381] Updated weights for policy 1, policy_version 599078 (0.0007) [2023-12-27 09:46:56,533][20381] Updated weights for policy 1, policy_version 599088 (0.0008) [2023-12-27 09:46:56,577][20381] Updated weights for policy 1, policy_version 599098 (0.0007) [2023-12-27 09:46:57,122][20347] Updated weights for policy 0, policy_version 599464 (0.0007) [2023-12-27 09:46:57,171][20347] Updated weights for policy 0, policy_version 599474 (0.0008) [2023-12-27 09:46:57,232][20347] Updated weights for policy 0, policy_version 599484 (0.0007) [2023-12-27 09:46:57,268][20381] Updated weights for policy 1, policy_version 599108 (0.0007) [2023-12-27 09:46:57,327][20381] Updated weights for policy 1, policy_version 599118 (0.0008) [2023-12-27 09:46:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 306880512. Throughput: 0: 10333.0, 1: 10308.2. Samples: 306871424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:46:57,363][19170] Avg episode reward: [(0, '285.110'), (1, '893.050')] [2023-12-27 09:46:57,386][20381] Updated weights for policy 1, policy_version 599128 (0.0008) [2023-12-27 09:46:57,917][20347] Updated weights for policy 0, policy_version 599494 (0.0007) [2023-12-27 09:46:57,971][20347] Updated weights for policy 0, policy_version 599504 (0.0007) [2023-12-27 09:46:58,034][20347] Updated weights for policy 0, policy_version 599514 (0.0007) [2023-12-27 09:46:58,061][20381] Updated weights for policy 1, policy_version 599138 (0.0007) [2023-12-27 09:46:58,111][20381] Updated weights for policy 1, policy_version 599148 (0.0007) [2023-12-27 09:46:58,159][20381] Updated weights for policy 1, policy_version 599158 (0.0007) [2023-12-27 09:46:58,221][20381] Updated weights for policy 1, policy_version 599168 (0.0008) [2023-12-27 09:46:58,737][20347] Updated weights for policy 0, policy_version 599524 (0.0008) [2023-12-27 09:46:58,805][20347] Updated weights for policy 0, policy_version 599534 (0.0008) [2023-12-27 09:46:58,873][20347] Updated weights for policy 0, policy_version 599544 (0.0009) [2023-12-27 09:46:58,976][20381] Updated weights for policy 1, policy_version 599178 (0.0008) [2023-12-27 09:46:59,035][20381] Updated weights for policy 1, policy_version 599188 (0.0009) [2023-12-27 09:46:59,092][20381] Updated weights for policy 1, policy_version 599198 (0.0007) [2023-12-27 09:46:59,639][20347] Updated weights for policy 0, policy_version 599554 (0.0008) [2023-12-27 09:46:59,697][20347] Updated weights for policy 0, policy_version 599564 (0.0008) [2023-12-27 09:46:59,748][20347] Updated weights for policy 0, policy_version 599574 (0.0007) [2023-12-27 09:46:59,765][20381] Updated weights for policy 1, policy_version 599208 (0.0006) [2023-12-27 09:46:59,804][20347] Updated weights for policy 0, policy_version 599584 (0.0006) [2023-12-27 09:46:59,826][20381] Updated weights for policy 1, policy_version 599218 (0.0011) [2023-12-27 09:46:59,883][20381] Updated weights for policy 1, policy_version 599228 (0.0011) [2023-12-27 09:47:00,533][20381] Updated weights for policy 1, policy_version 599238 (0.0009) [2023-12-27 09:47:00,538][20347] Updated weights for policy 0, policy_version 599594 (0.0006) [2023-12-27 09:47:00,583][20381] Updated weights for policy 1, policy_version 599248 (0.0005) [2023-12-27 09:47:00,595][20347] Updated weights for policy 0, policy_version 599604 (0.0007) [2023-12-27 09:47:00,634][20381] Updated weights for policy 1, policy_version 599258 (0.0005) [2023-12-27 09:47:00,651][20347] Updated weights for policy 0, policy_version 599614 (0.0008) [2023-12-27 09:47:01,190][20381] Updated weights for policy 1, policy_version 599268 (0.0007) [2023-12-27 09:47:01,246][20381] Updated weights for policy 1, policy_version 599278 (0.0008) [2023-12-27 09:47:01,306][20381] Updated weights for policy 1, policy_version 599288 (0.0009) [2023-12-27 09:47:01,346][20347] Updated weights for policy 0, policy_version 599624 (0.0010) [2023-12-27 09:47:01,410][20347] Updated weights for policy 0, policy_version 599634 (0.0009) [2023-12-27 09:47:01,472][20347] Updated weights for policy 0, policy_version 599644 (0.0006) [2023-12-27 09:47:01,944][20381] Updated weights for policy 1, policy_version 599298 (0.0010) [2023-12-27 09:47:01,998][20381] Updated weights for policy 1, policy_version 599308 (0.0007) [2023-12-27 09:47:02,055][20381] Updated weights for policy 1, policy_version 599318 (0.0007) [2023-12-27 09:47:02,107][20381] Updated weights for policy 1, policy_version 599328 (0.0007) [2023-12-27 09:47:02,162][20347] Updated weights for policy 0, policy_version 599654 (0.0010) [2023-12-27 09:47:02,210][20347] Updated weights for policy 0, policy_version 599664 (0.0007) [2023-12-27 09:47:02,265][20347] Updated weights for policy 0, policy_version 599674 (0.0006) [2023-12-27 09:47:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 306987008. Throughput: 0: 10256.3, 1: 10382.0. Samples: 306993008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:02,363][19170] Avg episode reward: [(0, '299.700'), (1, '863.770')] [2023-12-27 09:47:02,771][20381] Updated weights for policy 1, policy_version 599338 (0.0007) [2023-12-27 09:47:02,814][20381] Updated weights for policy 1, policy_version 599348 (0.0007) [2023-12-27 09:47:02,868][20381] Updated weights for policy 1, policy_version 599358 (0.0007) [2023-12-27 09:47:02,934][20347] Updated weights for policy 0, policy_version 599684 (0.0006) [2023-12-27 09:47:02,982][20347] Updated weights for policy 0, policy_version 599694 (0.0005) [2023-12-27 09:47:03,025][20347] Updated weights for policy 0, policy_version 599704 (0.0005) [2023-12-27 09:47:03,570][20381] Updated weights for policy 1, policy_version 599368 (0.0006) [2023-12-27 09:47:03,583][20347] Updated weights for policy 0, policy_version 599714 (0.0005) [2023-12-27 09:47:03,618][20381] Updated weights for policy 1, policy_version 599378 (0.0006) [2023-12-27 09:47:03,634][20347] Updated weights for policy 0, policy_version 599724 (0.0006) [2023-12-27 09:47:03,671][20381] Updated weights for policy 1, policy_version 599388 (0.0007) [2023-12-27 09:47:03,686][20347] Updated weights for policy 0, policy_version 599734 (0.0006) [2023-12-27 09:47:03,732][20347] Updated weights for policy 0, policy_version 599744 (0.0010) [2023-12-27 09:47:04,303][20381] Updated weights for policy 1, policy_version 599398 (0.0007) [2023-12-27 09:47:04,365][20381] Updated weights for policy 1, policy_version 599408 (0.0010) [2023-12-27 09:47:04,396][20347] Updated weights for policy 0, policy_version 599754 (0.0010) [2023-12-27 09:47:04,419][20381] Updated weights for policy 1, policy_version 599418 (0.0005) [2023-12-27 09:47:04,450][20347] Updated weights for policy 0, policy_version 599764 (0.0009) [2023-12-27 09:47:04,505][20347] Updated weights for policy 0, policy_version 599774 (0.0009) [2023-12-27 09:47:05,056][20381] Updated weights for policy 1, policy_version 599428 (0.0007) [2023-12-27 09:47:05,107][20381] Updated weights for policy 1, policy_version 599438 (0.0010) [2023-12-27 09:47:05,156][20381] Updated weights for policy 1, policy_version 599448 (0.0010) [2023-12-27 09:47:05,198][20347] Updated weights for policy 0, policy_version 599784 (0.0007) [2023-12-27 09:47:05,252][20347] Updated weights for policy 0, policy_version 599794 (0.0006) [2023-12-27 09:47:05,306][20347] Updated weights for policy 0, policy_version 599804 (0.0007) [2023-12-27 09:47:05,851][20381] Updated weights for policy 1, policy_version 599458 (0.0010) [2023-12-27 09:47:05,871][20347] Updated weights for policy 0, policy_version 599814 (0.0006) [2023-12-27 09:47:05,910][20381] Updated weights for policy 1, policy_version 599468 (0.0011) [2023-12-27 09:47:05,923][20347] Updated weights for policy 0, policy_version 599824 (0.0005) [2023-12-27 09:47:05,972][20381] Updated weights for policy 1, policy_version 599478 (0.0010) [2023-12-27 09:47:05,983][20347] Updated weights for policy 0, policy_version 599834 (0.0006) [2023-12-27 09:47:06,026][20381] Updated weights for policy 1, policy_version 599488 (0.0010) [2023-12-27 09:47:06,592][20347] Updated weights for policy 0, policy_version 599844 (0.0006) [2023-12-27 09:47:06,641][20347] Updated weights for policy 0, policy_version 599854 (0.0007) [2023-12-27 09:47:06,692][20347] Updated weights for policy 0, policy_version 599864 (0.0007) [2023-12-27 09:47:06,719][20381] Updated weights for policy 1, policy_version 599498 (0.0011) [2023-12-27 09:47:06,779][20381] Updated weights for policy 1, policy_version 599508 (0.0011) [2023-12-27 09:47:06,844][20381] Updated weights for policy 1, policy_version 599518 (0.0011) [2023-12-27 09:47:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 307085312. Throughput: 0: 10297.4, 1: 10374.3. Samples: 307057144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:07,363][19170] Avg episode reward: [(0, '362.040'), (1, '863.900')] [2023-12-27 09:47:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000599520_153501696.pth... [2023-12-27 09:47:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000598336_153198592.pth [2023-12-27 09:47:07,391][20347] Updated weights for policy 0, policy_version 599874 (0.0006) [2023-12-27 09:47:07,443][20347] Updated weights for policy 0, policy_version 599884 (0.0007) [2023-12-27 09:47:07,498][20347] Updated weights for policy 0, policy_version 599894 (0.0006) [2023-12-27 09:47:07,507][20381] Updated weights for policy 1, policy_version 599528 (0.0011) [2023-12-27 09:47:07,552][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000599904_153591808.pth... [2023-12-27 09:47:07,553][20347] Updated weights for policy 0, policy_version 599904 (0.0005) [2023-12-27 09:47:07,556][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000598688_153280512.pth [2023-12-27 09:47:07,562][20381] Updated weights for policy 1, policy_version 599538 (0.0010) [2023-12-27 09:47:07,620][20381] Updated weights for policy 1, policy_version 599548 (0.0011) [2023-12-27 09:47:08,237][20347] Updated weights for policy 0, policy_version 599914 (0.0007) [2023-12-27 09:47:08,277][20381] Updated weights for policy 1, policy_version 599558 (0.0007) [2023-12-27 09:47:08,293][20347] Updated weights for policy 0, policy_version 599924 (0.0007) [2023-12-27 09:47:08,333][20381] Updated weights for policy 1, policy_version 599568 (0.0006) [2023-12-27 09:47:08,350][20347] Updated weights for policy 0, policy_version 599934 (0.0008) [2023-12-27 09:47:08,391][20381] Updated weights for policy 1, policy_version 599578 (0.0009) [2023-12-27 09:47:09,054][20381] Updated weights for policy 1, policy_version 599588 (0.0008) [2023-12-27 09:47:09,062][20347] Updated weights for policy 0, policy_version 599944 (0.0008) [2023-12-27 09:47:09,112][20381] Updated weights for policy 1, policy_version 599598 (0.0006) [2023-12-27 09:47:09,121][20347] Updated weights for policy 0, policy_version 599954 (0.0008) [2023-12-27 09:47:09,170][20347] Updated weights for policy 0, policy_version 599964 (0.0006) [2023-12-27 09:47:09,171][20381] Updated weights for policy 1, policy_version 599608 (0.0006) [2023-12-27 09:47:09,858][20347] Updated weights for policy 0, policy_version 599974 (0.0007) [2023-12-27 09:47:09,928][20347] Updated weights for policy 0, policy_version 599984 (0.0008) [2023-12-27 09:47:09,930][20381] Updated weights for policy 1, policy_version 599618 (0.0008) [2023-12-27 09:47:09,990][20347] Updated weights for policy 0, policy_version 599994 (0.0010) [2023-12-27 09:47:09,992][20381] Updated weights for policy 1, policy_version 599628 (0.0007) [2023-12-27 09:47:10,050][20381] Updated weights for policy 1, policy_version 599638 (0.0007) [2023-12-27 09:47:10,107][20381] Updated weights for policy 1, policy_version 599648 (0.0008) [2023-12-27 09:47:10,683][20347] Updated weights for policy 0, policy_version 600004 (0.0006) [2023-12-27 09:47:10,740][20347] Updated weights for policy 0, policy_version 600014 (0.0005) [2023-12-27 09:47:10,772][20381] Updated weights for policy 1, policy_version 599658 (0.0007) [2023-12-27 09:47:10,790][20347] Updated weights for policy 0, policy_version 600024 (0.0005) [2023-12-27 09:47:10,816][20381] Updated weights for policy 1, policy_version 599668 (0.0006) [2023-12-27 09:47:10,876][20381] Updated weights for policy 1, policy_version 599678 (0.0006) [2023-12-27 09:47:11,445][20347] Updated weights for policy 0, policy_version 600034 (0.0007) [2023-12-27 09:47:11,501][20347] Updated weights for policy 0, policy_version 600044 (0.0010) [2023-12-27 09:47:11,561][20347] Updated weights for policy 0, policy_version 600054 (0.0010) [2023-12-27 09:47:11,627][20381] Updated weights for policy 1, policy_version 599688 (0.0007) [2023-12-27 09:47:11,628][20347] Updated weights for policy 0, policy_version 600064 (0.0011) [2023-12-27 09:47:11,698][20381] Updated weights for policy 1, policy_version 599698 (0.0008) [2023-12-27 09:47:11,759][20381] Updated weights for policy 1, policy_version 599708 (0.0008) [2023-12-27 09:47:12,245][20347] Updated weights for policy 0, policy_version 600074 (0.0010) [2023-12-27 09:47:12,306][20347] Updated weights for policy 0, policy_version 600084 (0.0007) [2023-12-27 09:47:12,361][20347] Updated weights for policy 0, policy_version 600094 (0.0008) [2023-12-27 09:47:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20799.4). Total num frames: 307183616. Throughput: 0: 10377.3, 1: 10339.3. Samples: 307180480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:12,363][19170] Avg episode reward: [(0, '454.780'), (1, '902.850')] [2023-12-27 09:47:12,483][20381] Updated weights for policy 1, policy_version 599718 (0.0007) [2023-12-27 09:47:12,547][20381] Updated weights for policy 1, policy_version 599728 (0.0006) [2023-12-27 09:47:12,607][20381] Updated weights for policy 1, policy_version 599738 (0.0005) [2023-12-27 09:47:12,982][20347] Updated weights for policy 0, policy_version 600104 (0.0010) [2023-12-27 09:47:13,028][20347] Updated weights for policy 0, policy_version 600114 (0.0008) [2023-12-27 09:47:13,076][20347] Updated weights for policy 0, policy_version 600124 (0.0005) [2023-12-27 09:47:13,188][20381] Updated weights for policy 1, policy_version 599748 (0.0006) [2023-12-27 09:47:13,243][20381] Updated weights for policy 1, policy_version 599758 (0.0005) [2023-12-27 09:47:13,307][20381] Updated weights for policy 1, policy_version 599768 (0.0007) [2023-12-27 09:47:13,692][20347] Updated weights for policy 0, policy_version 600134 (0.0006) [2023-12-27 09:47:13,758][20347] Updated weights for policy 0, policy_version 600144 (0.0005) [2023-12-27 09:47:13,816][20347] Updated weights for policy 0, policy_version 600154 (0.0005) [2023-12-27 09:47:13,852][20381] Updated weights for policy 1, policy_version 599778 (0.0007) [2023-12-27 09:47:13,901][20381] Updated weights for policy 1, policy_version 599788 (0.0009) [2023-12-27 09:47:13,947][20381] Updated weights for policy 1, policy_version 599798 (0.0010) [2023-12-27 09:47:14,002][20381] Updated weights for policy 1, policy_version 599808 (0.0008) [2023-12-27 09:47:14,395][20347] Updated weights for policy 0, policy_version 600164 (0.0007) [2023-12-27 09:47:14,446][20347] Updated weights for policy 0, policy_version 600174 (0.0008) [2023-12-27 09:47:14,510][20347] Updated weights for policy 0, policy_version 600184 (0.0008) [2023-12-27 09:47:14,661][20381] Updated weights for policy 1, policy_version 599818 (0.0005) [2023-12-27 09:47:14,722][20381] Updated weights for policy 1, policy_version 599828 (0.0007) [2023-12-27 09:47:14,779][20381] Updated weights for policy 1, policy_version 599838 (0.0009) [2023-12-27 09:47:15,246][20347] Updated weights for policy 0, policy_version 600194 (0.0009) [2023-12-27 09:47:15,304][20347] Updated weights for policy 0, policy_version 600204 (0.0008) [2023-12-27 09:47:15,360][20347] Updated weights for policy 0, policy_version 600214 (0.0007) [2023-12-27 09:47:15,412][20347] Updated weights for policy 0, policy_version 600224 (0.0007) [2023-12-27 09:47:15,442][20381] Updated weights for policy 1, policy_version 599848 (0.0007) [2023-12-27 09:47:15,495][20381] Updated weights for policy 1, policy_version 599858 (0.0005) [2023-12-27 09:47:15,543][20381] Updated weights for policy 1, policy_version 599868 (0.0005) [2023-12-27 09:47:16,100][20347] Updated weights for policy 0, policy_version 600234 (0.0006) [2023-12-27 09:47:16,147][20347] Updated weights for policy 0, policy_version 600244 (0.0005) [2023-12-27 09:47:16,196][20347] Updated weights for policy 0, policy_version 600254 (0.0005) [2023-12-27 09:47:16,213][20381] Updated weights for policy 1, policy_version 599878 (0.0007) [2023-12-27 09:47:16,270][20381] Updated weights for policy 1, policy_version 599888 (0.0009) [2023-12-27 09:47:16,326][20381] Updated weights for policy 1, policy_version 599898 (0.0008) [2023-12-27 09:47:16,793][20347] Updated weights for policy 0, policy_version 600264 (0.0007) [2023-12-27 09:47:16,853][20347] Updated weights for policy 0, policy_version 600274 (0.0008) [2023-12-27 09:47:16,911][20347] Updated weights for policy 0, policy_version 600284 (0.0006) [2023-12-27 09:47:16,972][20381] Updated weights for policy 1, policy_version 599908 (0.0009) [2023-12-27 09:47:17,035][20381] Updated weights for policy 1, policy_version 599918 (0.0006) [2023-12-27 09:47:17,097][20381] Updated weights for policy 1, policy_version 599928 (0.0005) [2023-12-27 09:47:17,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 307298304. Throughput: 0: 10369.7, 1: 10336.9. Samples: 307308512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:17,364][19170] Avg episode reward: [(0, '509.390'), (1, '883.440')] [2023-12-27 09:47:17,481][20347] Updated weights for policy 0, policy_version 600294 (0.0005) [2023-12-27 09:47:17,543][20347] Updated weights for policy 0, policy_version 600304 (0.0005) [2023-12-27 09:47:17,611][20347] Updated weights for policy 0, policy_version 600314 (0.0007) [2023-12-27 09:47:17,620][20381] Updated weights for policy 1, policy_version 599938 (0.0005) [2023-12-27 09:47:17,672][20381] Updated weights for policy 1, policy_version 599948 (0.0006) [2023-12-27 09:47:17,729][20381] Updated weights for policy 1, policy_version 599958 (0.0005) [2023-12-27 09:47:17,787][20381] Updated weights for policy 1, policy_version 599968 (0.0007) [2023-12-27 09:47:18,188][20347] Updated weights for policy 0, policy_version 600324 (0.0006) [2023-12-27 09:47:18,255][20347] Updated weights for policy 0, policy_version 600334 (0.0006) [2023-12-27 09:47:18,319][20347] Updated weights for policy 0, policy_version 600344 (0.0005) [2023-12-27 09:47:18,410][20381] Updated weights for policy 1, policy_version 599978 (0.0011) [2023-12-27 09:47:18,465][20381] Updated weights for policy 1, policy_version 599988 (0.0010) [2023-12-27 09:47:18,523][20381] Updated weights for policy 1, policy_version 599998 (0.0011) [2023-12-27 09:47:18,963][20347] Updated weights for policy 0, policy_version 600354 (0.0008) [2023-12-27 09:47:19,023][20347] Updated weights for policy 0, policy_version 600364 (0.0008) [2023-12-27 09:47:19,083][20347] Updated weights for policy 0, policy_version 600374 (0.0007) [2023-12-27 09:47:19,140][20347] Updated weights for policy 0, policy_version 600384 (0.0007) [2023-12-27 09:47:19,221][20381] Updated weights for policy 1, policy_version 600008 (0.0010) [2023-12-27 09:47:19,280][20381] Updated weights for policy 1, policy_version 600018 (0.0011) [2023-12-27 09:47:19,332][20381] Updated weights for policy 1, policy_version 600028 (0.0010) [2023-12-27 09:47:19,852][20347] Updated weights for policy 0, policy_version 600394 (0.0008) [2023-12-27 09:47:19,917][20347] Updated weights for policy 0, policy_version 600404 (0.0007) [2023-12-27 09:47:19,989][20347] Updated weights for policy 0, policy_version 600414 (0.0008) [2023-12-27 09:47:20,051][20381] Updated weights for policy 1, policy_version 600038 (0.0011) [2023-12-27 09:47:20,107][20381] Updated weights for policy 1, policy_version 600048 (0.0010) [2023-12-27 09:47:20,163][20381] Updated weights for policy 1, policy_version 600058 (0.0011) [2023-12-27 09:47:20,659][20347] Updated weights for policy 0, policy_version 600424 (0.0006) [2023-12-27 09:47:20,715][20347] Updated weights for policy 0, policy_version 600434 (0.0005) [2023-12-27 09:47:20,770][20347] Updated weights for policy 0, policy_version 600444 (0.0006) [2023-12-27 09:47:20,779][20381] Updated weights for policy 1, policy_version 600068 (0.0011) [2023-12-27 09:47:20,842][20381] Updated weights for policy 1, policy_version 600078 (0.0011) [2023-12-27 09:47:20,901][20381] Updated weights for policy 1, policy_version 600088 (0.0011) [2023-12-27 09:47:21,402][20347] Updated weights for policy 0, policy_version 600454 (0.0007) [2023-12-27 09:47:21,467][20347] Updated weights for policy 0, policy_version 600464 (0.0006) [2023-12-27 09:47:21,527][20347] Updated weights for policy 0, policy_version 600474 (0.0008) [2023-12-27 09:47:21,619][20381] Updated weights for policy 1, policy_version 600098 (0.0010) [2023-12-27 09:47:21,683][20381] Updated weights for policy 1, policy_version 600108 (0.0007) [2023-12-27 09:47:21,752][20381] Updated weights for policy 1, policy_version 600118 (0.0010) [2023-12-27 09:47:21,808][20381] Updated weights for policy 1, policy_version 600128 (0.0011) [2023-12-27 09:47:22,272][20347] Updated weights for policy 0, policy_version 600484 (0.0010) [2023-12-27 09:47:22,340][20347] Updated weights for policy 0, policy_version 600494 (0.0011) [2023-12-27 09:47:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 307396608. Throughput: 0: 10374.9, 1: 10353.3. Samples: 307372024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:22,363][19170] Avg episode reward: [(0, '462.320'), (1, '893.910')] [2023-12-27 09:47:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000600128_153657344.pth... [2023-12-27 09:47:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000598912_153346048.pth [2023-12-27 09:47:22,406][20347] Updated weights for policy 0, policy_version 600504 (0.0011) [2023-12-27 09:47:22,455][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000600512_153747456.pth... [2023-12-27 09:47:22,456][20381] Updated weights for policy 1, policy_version 600138 (0.0007) [2023-12-27 09:47:22,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000599296_153436160.pth [2023-12-27 09:47:22,511][20381] Updated weights for policy 1, policy_version 600148 (0.0009) [2023-12-27 09:47:22,570][20381] Updated weights for policy 1, policy_version 600158 (0.0011) [2023-12-27 09:47:23,143][20347] Updated weights for policy 0, policy_version 600514 (0.0011) [2023-12-27 09:47:23,206][20347] Updated weights for policy 0, policy_version 600524 (0.0010) [2023-12-27 09:47:23,258][20347] Updated weights for policy 0, policy_version 600534 (0.0010) [2023-12-27 09:47:23,260][20381] Updated weights for policy 1, policy_version 600168 (0.0006) [2023-12-27 09:47:23,317][20347] Updated weights for policy 0, policy_version 600544 (0.0010) [2023-12-27 09:47:23,317][20381] Updated weights for policy 1, policy_version 600178 (0.0009) [2023-12-27 09:47:23,375][20381] Updated weights for policy 1, policy_version 600188 (0.0010) [2023-12-27 09:47:23,892][20347] Updated weights for policy 0, policy_version 600554 (0.0007) [2023-12-27 09:47:23,942][20347] Updated weights for policy 0, policy_version 600564 (0.0007) [2023-12-27 09:47:23,996][20347] Updated weights for policy 0, policy_version 600574 (0.0007) [2023-12-27 09:47:24,043][20381] Updated weights for policy 1, policy_version 600198 (0.0011) [2023-12-27 09:47:24,091][20381] Updated weights for policy 1, policy_version 600208 (0.0010) [2023-12-27 09:47:24,146][20381] Updated weights for policy 1, policy_version 600218 (0.0010) [2023-12-27 09:47:24,688][20347] Updated weights for policy 0, policy_version 600584 (0.0006) [2023-12-27 09:47:24,739][20347] Updated weights for policy 0, policy_version 600594 (0.0009) [2023-12-27 09:47:24,801][20347] Updated weights for policy 0, policy_version 600604 (0.0008) [2023-12-27 09:47:24,809][20381] Updated weights for policy 1, policy_version 600228 (0.0008) [2023-12-27 09:47:24,863][20381] Updated weights for policy 1, policy_version 600238 (0.0005) [2023-12-27 09:47:24,910][20381] Updated weights for policy 1, policy_version 600248 (0.0005) [2023-12-27 09:47:25,485][20381] Updated weights for policy 1, policy_version 600258 (0.0006) [2023-12-27 09:47:25,537][20347] Updated weights for policy 0, policy_version 600614 (0.0006) [2023-12-27 09:47:25,542][20381] Updated weights for policy 1, policy_version 600268 (0.0010) [2023-12-27 09:47:25,588][20347] Updated weights for policy 0, policy_version 600624 (0.0005) [2023-12-27 09:47:25,597][20381] Updated weights for policy 1, policy_version 600278 (0.0010) [2023-12-27 09:47:25,634][20347] Updated weights for policy 0, policy_version 600634 (0.0006) [2023-12-27 09:47:25,655][20381] Updated weights for policy 1, policy_version 600288 (0.0010) [2023-12-27 09:47:26,277][20381] Updated weights for policy 1, policy_version 600298 (0.0011) [2023-12-27 09:47:26,325][20381] Updated weights for policy 1, policy_version 600308 (0.0010) [2023-12-27 09:47:26,327][20347] Updated weights for policy 0, policy_version 600644 (0.0005) [2023-12-27 09:47:26,381][20381] Updated weights for policy 1, policy_version 600318 (0.0010) [2023-12-27 09:47:26,383][20347] Updated weights for policy 0, policy_version 600654 (0.0006) [2023-12-27 09:47:26,440][20347] Updated weights for policy 0, policy_version 600664 (0.0007) [2023-12-27 09:47:27,064][20381] Updated weights for policy 1, policy_version 600328 (0.0010) [2023-12-27 09:47:27,073][20347] Updated weights for policy 0, policy_version 600674 (0.0008) [2023-12-27 09:47:27,109][20381] Updated weights for policy 1, policy_version 600338 (0.0010) [2023-12-27 09:47:27,121][20347] Updated weights for policy 0, policy_version 600684 (0.0006) [2023-12-27 09:47:27,163][20381] Updated weights for policy 1, policy_version 600348 (0.0010) [2023-12-27 09:47:27,169][20347] Updated weights for policy 0, policy_version 600694 (0.0005) [2023-12-27 09:47:27,217][20347] Updated weights for policy 0, policy_version 600704 (0.0006) [2023-12-27 09:47:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20882.7). Total num frames: 307511296. Throughput: 0: 10328.0, 1: 10431.4. Samples: 307496036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:27,363][19170] Avg episode reward: [(0, '508.120'), (1, '883.970')] [2023-12-27 09:47:27,780][20381] Updated weights for policy 1, policy_version 600358 (0.0010) [2023-12-27 09:47:27,838][20381] Updated weights for policy 1, policy_version 600368 (0.0011) [2023-12-27 09:47:27,893][20381] Updated weights for policy 1, policy_version 600378 (0.0010) [2023-12-27 09:47:27,942][20347] Updated weights for policy 0, policy_version 600714 (0.0010) [2023-12-27 09:47:27,993][20347] Updated weights for policy 0, policy_version 600724 (0.0010) [2023-12-27 09:47:28,046][20347] Updated weights for policy 0, policy_version 600734 (0.0009) [2023-12-27 09:47:28,590][20381] Updated weights for policy 1, policy_version 600388 (0.0009) [2023-12-27 09:47:28,647][20381] Updated weights for policy 1, policy_version 600398 (0.0007) [2023-12-27 09:47:28,674][20347] Updated weights for policy 0, policy_version 600744 (0.0008) [2023-12-27 09:47:28,702][20381] Updated weights for policy 1, policy_version 600408 (0.0005) [2023-12-27 09:47:28,731][20347] Updated weights for policy 0, policy_version 600754 (0.0007) [2023-12-27 09:47:28,789][20347] Updated weights for policy 0, policy_version 600764 (0.0008) [2023-12-27 09:47:29,306][20381] Updated weights for policy 1, policy_version 600418 (0.0007) [2023-12-27 09:47:29,368][20381] Updated weights for policy 1, policy_version 600428 (0.0007) [2023-12-27 09:47:29,433][20381] Updated weights for policy 1, policy_version 600438 (0.0008) [2023-12-27 09:47:29,492][20381] Updated weights for policy 1, policy_version 600448 (0.0008) [2023-12-27 09:47:29,494][20347] Updated weights for policy 0, policy_version 600774 (0.0009) [2023-12-27 09:47:29,548][20347] Updated weights for policy 0, policy_version 600784 (0.0010) [2023-12-27 09:47:29,596][20347] Updated weights for policy 0, policy_version 600794 (0.0010) [2023-12-27 09:47:30,202][20381] Updated weights for policy 1, policy_version 600458 (0.0007) [2023-12-27 09:47:30,263][20381] Updated weights for policy 1, policy_version 600468 (0.0007) [2023-12-27 09:47:30,304][20347] Updated weights for policy 0, policy_version 600804 (0.0010) [2023-12-27 09:47:30,319][20381] Updated weights for policy 1, policy_version 600478 (0.0007) [2023-12-27 09:47:30,355][20347] Updated weights for policy 0, policy_version 600814 (0.0010) [2023-12-27 09:47:30,409][20347] Updated weights for policy 0, policy_version 600824 (0.0010) [2023-12-27 09:47:31,013][20381] Updated weights for policy 1, policy_version 600488 (0.0007) [2023-12-27 09:47:31,084][20381] Updated weights for policy 1, policy_version 600498 (0.0008) [2023-12-27 09:47:31,089][20347] Updated weights for policy 0, policy_version 600834 (0.0009) [2023-12-27 09:47:31,147][20347] Updated weights for policy 0, policy_version 600844 (0.0009) [2023-12-27 09:47:31,147][20381] Updated weights for policy 1, policy_version 600508 (0.0007) [2023-12-27 09:47:31,213][20347] Updated weights for policy 0, policy_version 600854 (0.0008) [2023-12-27 09:47:31,276][20347] Updated weights for policy 0, policy_version 600864 (0.0009) [2023-12-27 09:47:31,892][20381] Updated weights for policy 1, policy_version 600518 (0.0009) [2023-12-27 09:47:31,944][20381] Updated weights for policy 1, policy_version 600528 (0.0009) [2023-12-27 09:47:31,984][20347] Updated weights for policy 0, policy_version 600874 (0.0010) [2023-12-27 09:47:31,991][20381] Updated weights for policy 1, policy_version 600538 (0.0011) [2023-12-27 09:47:32,037][20347] Updated weights for policy 0, policy_version 600884 (0.0010) [2023-12-27 09:47:32,093][20347] Updated weights for policy 0, policy_version 600894 (0.0010) [2023-12-27 09:47:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 307609600. Throughput: 0: 10332.6, 1: 10380.1. Samples: 307618152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:32,363][19170] Avg episode reward: [(0, '518.750'), (1, '902.820')] [2023-12-27 09:47:32,697][20381] Updated weights for policy 1, policy_version 600548 (0.0009) [2023-12-27 09:47:32,749][20381] Updated weights for policy 1, policy_version 600558 (0.0007) [2023-12-27 09:47:32,769][20347] Updated weights for policy 0, policy_version 600904 (0.0010) [2023-12-27 09:47:32,800][20381] Updated weights for policy 1, policy_version 600568 (0.0005) [2023-12-27 09:47:32,827][20347] Updated weights for policy 0, policy_version 600914 (0.0010) [2023-12-27 09:47:32,876][20347] Updated weights for policy 0, policy_version 600924 (0.0010) [2023-12-27 09:47:33,414][20381] Updated weights for policy 1, policy_version 600578 (0.0007) [2023-12-27 09:47:33,457][20381] Updated weights for policy 1, policy_version 600588 (0.0007) [2023-12-27 09:47:33,510][20381] Updated weights for policy 1, policy_version 600598 (0.0007) [2023-12-27 09:47:33,558][20347] Updated weights for policy 0, policy_version 600934 (0.0010) [2023-12-27 09:47:33,568][20381] Updated weights for policy 1, policy_version 600608 (0.0007) [2023-12-27 09:47:33,606][20347] Updated weights for policy 0, policy_version 600944 (0.0010) [2023-12-27 09:47:33,661][20347] Updated weights for policy 0, policy_version 600954 (0.0009) [2023-12-27 09:47:34,241][20381] Updated weights for policy 1, policy_version 600618 (0.0008) [2023-12-27 09:47:34,299][20381] Updated weights for policy 1, policy_version 600628 (0.0008) [2023-12-27 09:47:34,325][20347] Updated weights for policy 0, policy_version 600964 (0.0007) [2023-12-27 09:47:34,351][20381] Updated weights for policy 1, policy_version 600638 (0.0008) [2023-12-27 09:47:34,385][20347] Updated weights for policy 0, policy_version 600974 (0.0007) [2023-12-27 09:47:34,447][20347] Updated weights for policy 0, policy_version 600984 (0.0008) [2023-12-27 09:47:35,017][20381] Updated weights for policy 1, policy_version 600648 (0.0007) [2023-12-27 09:47:35,066][20381] Updated weights for policy 1, policy_version 600658 (0.0008) [2023-12-27 09:47:35,115][20381] Updated weights for policy 1, policy_version 600668 (0.0007) [2023-12-27 09:47:35,136][20347] Updated weights for policy 0, policy_version 600994 (0.0007) [2023-12-27 09:47:35,184][20347] Updated weights for policy 0, policy_version 601004 (0.0008) [2023-12-27 09:47:35,235][20347] Updated weights for policy 0, policy_version 601014 (0.0008) [2023-12-27 09:47:35,290][20347] Updated weights for policy 0, policy_version 601024 (0.0007) [2023-12-27 09:47:35,841][20381] Updated weights for policy 1, policy_version 600678 (0.0008) [2023-12-27 09:47:35,899][20381] Updated weights for policy 1, policy_version 600688 (0.0008) [2023-12-27 09:47:35,936][20347] Updated weights for policy 0, policy_version 601034 (0.0006) [2023-12-27 09:47:35,955][20381] Updated weights for policy 1, policy_version 600698 (0.0008) [2023-12-27 09:47:35,992][20347] Updated weights for policy 0, policy_version 601044 (0.0008) [2023-12-27 09:47:36,044][20347] Updated weights for policy 0, policy_version 601054 (0.0005) [2023-12-27 09:47:36,594][20381] Updated weights for policy 1, policy_version 600708 (0.0006) [2023-12-27 09:47:36,657][20381] Updated weights for policy 1, policy_version 600718 (0.0008) [2023-12-27 09:47:36,700][20347] Updated weights for policy 0, policy_version 601064 (0.0008) [2023-12-27 09:47:36,716][20381] Updated weights for policy 1, policy_version 600728 (0.0008) [2023-12-27 09:47:36,745][20347] Updated weights for policy 0, policy_version 601074 (0.0005) [2023-12-27 09:47:36,795][20347] Updated weights for policy 0, policy_version 601084 (0.0007) [2023-12-27 09:47:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 307707904. Throughput: 0: 10343.7, 1: 10393.3. Samples: 307680652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:37,363][19170] Avg episode reward: [(0, '387.500'), (1, '883.560')] [2023-12-27 09:47:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000601088_153894912.pth... [2023-12-27 09:47:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000600736_153812992.pth... [2023-12-27 09:47:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000599520_153501696.pth [2023-12-27 09:47:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000599904_153591808.pth [2023-12-27 09:47:37,409][20381] Updated weights for policy 1, policy_version 600738 (0.0008) [2023-12-27 09:47:37,469][20381] Updated weights for policy 1, policy_version 600748 (0.0011) [2023-12-27 09:47:37,509][20347] Updated weights for policy 0, policy_version 601094 (0.0007) [2023-12-27 09:47:37,519][20381] Updated weights for policy 1, policy_version 600758 (0.0011) [2023-12-27 09:47:37,555][20347] Updated weights for policy 0, policy_version 601104 (0.0005) [2023-12-27 09:47:37,569][20381] Updated weights for policy 1, policy_version 600768 (0.0011) [2023-12-27 09:47:37,608][20347] Updated weights for policy 0, policy_version 601114 (0.0007) [2023-12-27 09:47:38,202][20381] Updated weights for policy 1, policy_version 600778 (0.0011) [2023-12-27 09:47:38,261][20381] Updated weights for policy 1, policy_version 600788 (0.0010) [2023-12-27 09:47:38,325][20381] Updated weights for policy 1, policy_version 600798 (0.0010) [2023-12-27 09:47:38,329][20347] Updated weights for policy 0, policy_version 601124 (0.0009) [2023-12-27 09:47:38,393][20347] Updated weights for policy 0, policy_version 601134 (0.0011) [2023-12-27 09:47:38,452][20347] Updated weights for policy 0, policy_version 601144 (0.0010) [2023-12-27 09:47:39,016][20381] Updated weights for policy 1, policy_version 600808 (0.0011) [2023-12-27 09:47:39,070][20347] Updated weights for policy 0, policy_version 601154 (0.0009) [2023-12-27 09:47:39,077][20381] Updated weights for policy 1, policy_version 600818 (0.0011) [2023-12-27 09:47:39,125][20381] Updated weights for policy 1, policy_version 600828 (0.0010) [2023-12-27 09:47:39,126][20347] Updated weights for policy 0, policy_version 601164 (0.0005) [2023-12-27 09:47:39,186][20347] Updated weights for policy 0, policy_version 601174 (0.0007) [2023-12-27 09:47:39,245][20347] Updated weights for policy 0, policy_version 601184 (0.0008) [2023-12-27 09:47:39,809][20381] Updated weights for policy 1, policy_version 600838 (0.0009) [2023-12-27 09:47:39,873][20381] Updated weights for policy 1, policy_version 600848 (0.0009) [2023-12-27 09:47:39,938][20381] Updated weights for policy 1, policy_version 600858 (0.0007) [2023-12-27 09:47:39,941][20347] Updated weights for policy 0, policy_version 601194 (0.0006) [2023-12-27 09:47:40,003][20347] Updated weights for policy 0, policy_version 601204 (0.0007) [2023-12-27 09:47:40,059][20347] Updated weights for policy 0, policy_version 601214 (0.0008) [2023-12-27 09:47:40,594][20381] Updated weights for policy 1, policy_version 600868 (0.0007) [2023-12-27 09:47:40,643][20381] Updated weights for policy 1, policy_version 600878 (0.0007) [2023-12-27 09:47:40,693][20381] Updated weights for policy 1, policy_version 600888 (0.0006) [2023-12-27 09:47:40,710][20347] Updated weights for policy 0, policy_version 601224 (0.0007) [2023-12-27 09:47:40,770][20347] Updated weights for policy 0, policy_version 601234 (0.0006) [2023-12-27 09:47:40,830][20347] Updated weights for policy 0, policy_version 601244 (0.0006) [2023-12-27 09:47:41,429][20381] Updated weights for policy 1, policy_version 600898 (0.0006) [2023-12-27 09:47:41,491][20347] Updated weights for policy 0, policy_version 601254 (0.0009) [2023-12-27 09:47:41,498][20381] Updated weights for policy 1, policy_version 600908 (0.0007) [2023-12-27 09:47:41,541][20347] Updated weights for policy 0, policy_version 601264 (0.0008) [2023-12-27 09:47:41,551][20381] Updated weights for policy 1, policy_version 600918 (0.0007) [2023-12-27 09:47:41,592][20347] Updated weights for policy 0, policy_version 601274 (0.0005) [2023-12-27 09:47:41,608][20381] Updated weights for policy 1, policy_version 600928 (0.0007) [2023-12-27 09:47:42,282][20347] Updated weights for policy 0, policy_version 601284 (0.0008) [2023-12-27 09:47:42,331][20292] KL-divergence is very high: 151.7595 [2023-12-27 09:47:42,342][20347] Updated weights for policy 0, policy_version 601294 (0.0007) [2023-12-27 09:47:42,345][20381] Updated weights for policy 1, policy_version 600938 (0.0007) [2023-12-27 09:47:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 307806208. Throughput: 0: 10343.2, 1: 10375.0. Samples: 307803744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:42,363][19170] Avg episode reward: [(0, '367.160'), (1, '903.110')] [2023-12-27 09:47:42,405][20347] Updated weights for policy 0, policy_version 601304 (0.0008) [2023-12-27 09:47:42,406][20381] Updated weights for policy 1, policy_version 600948 (0.0006) [2023-12-27 09:47:42,463][20381] Updated weights for policy 1, policy_version 600958 (0.0006) [2023-12-27 09:47:43,040][20347] Updated weights for policy 0, policy_version 601314 (0.0008) [2023-12-27 09:47:43,101][20347] Updated weights for policy 0, policy_version 601324 (0.0005) [2023-12-27 09:47:43,157][20347] Updated weights for policy 0, policy_version 601334 (0.0005) [2023-12-27 09:47:43,158][20381] Updated weights for policy 1, policy_version 600968 (0.0008) [2023-12-27 09:47:43,211][20381] Updated weights for policy 1, policy_version 600978 (0.0008) [2023-12-27 09:47:43,216][20347] Updated weights for policy 0, policy_version 601344 (0.0005) [2023-12-27 09:47:43,267][20381] Updated weights for policy 1, policy_version 600988 (0.0008) [2023-12-27 09:47:43,800][20347] Updated weights for policy 0, policy_version 601354 (0.0006) [2023-12-27 09:47:43,853][20347] Updated weights for policy 0, policy_version 601364 (0.0008) [2023-12-27 09:47:43,881][20381] Updated weights for policy 1, policy_version 600998 (0.0010) [2023-12-27 09:47:43,902][20347] Updated weights for policy 0, policy_version 601374 (0.0005) [2023-12-27 09:47:43,937][20381] Updated weights for policy 1, policy_version 601008 (0.0010) [2023-12-27 09:47:43,996][20381] Updated weights for policy 1, policy_version 601018 (0.0010) [2023-12-27 09:47:44,506][20347] Updated weights for policy 0, policy_version 601384 (0.0008) [2023-12-27 09:47:44,553][20347] Updated weights for policy 0, policy_version 601394 (0.0007) [2023-12-27 09:47:44,606][20347] Updated weights for policy 0, policy_version 601404 (0.0007) [2023-12-27 09:47:44,639][20381] Updated weights for policy 1, policy_version 601028 (0.0009) [2023-12-27 09:47:44,694][20381] Updated weights for policy 1, policy_version 601038 (0.0005) [2023-12-27 09:47:44,751][20381] Updated weights for policy 1, policy_version 601048 (0.0007) [2023-12-27 09:47:45,350][20347] Updated weights for policy 0, policy_version 601414 (0.0007) [2023-12-27 09:47:45,394][20347] Updated weights for policy 0, policy_version 601424 (0.0007) [2023-12-27 09:47:45,454][20347] Updated weights for policy 0, policy_version 601434 (0.0007) [2023-12-27 09:47:45,456][20381] Updated weights for policy 1, policy_version 601058 (0.0008) [2023-12-27 09:47:45,510][20381] Updated weights for policy 1, policy_version 601068 (0.0007) [2023-12-27 09:47:45,570][20381] Updated weights for policy 1, policy_version 601078 (0.0005) [2023-12-27 09:47:45,628][20381] Updated weights for policy 1, policy_version 601088 (0.0010) [2023-12-27 09:47:46,195][20347] Updated weights for policy 0, policy_version 601444 (0.0006) [2023-12-27 09:47:46,222][20381] Updated weights for policy 1, policy_version 601098 (0.0005) [2023-12-27 09:47:46,253][20347] Updated weights for policy 0, policy_version 601454 (0.0007) [2023-12-27 09:47:46,276][20381] Updated weights for policy 1, policy_version 601108 (0.0005) [2023-12-27 09:47:46,304][20347] Updated weights for policy 0, policy_version 601464 (0.0007) [2023-12-27 09:47:46,322][20381] Updated weights for policy 1, policy_version 601118 (0.0005) [2023-12-27 09:47:46,896][20381] Updated weights for policy 1, policy_version 601128 (0.0006) [2023-12-27 09:47:46,953][20381] Updated weights for policy 1, policy_version 601138 (0.0005) [2023-12-27 09:47:47,017][20381] Updated weights for policy 1, policy_version 601148 (0.0005) [2023-12-27 09:47:47,036][20347] Updated weights for policy 0, policy_version 601474 (0.0007) [2023-12-27 09:47:47,095][20347] Updated weights for policy 0, policy_version 601484 (0.0007) [2023-12-27 09:47:47,149][20347] Updated weights for policy 0, policy_version 601494 (0.0007) [2023-12-27 09:47:47,208][20347] Updated weights for policy 0, policy_version 601504 (0.0007) [2023-12-27 09:47:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 307920896. Throughput: 0: 10391.7, 1: 10409.9. Samples: 307929084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:47,363][19170] Avg episode reward: [(0, '457.680'), (1, '884.180')] [2023-12-27 09:47:47,557][20381] Updated weights for policy 1, policy_version 601158 (0.0007) [2023-12-27 09:47:47,606][20381] Updated weights for policy 1, policy_version 601168 (0.0008) [2023-12-27 09:47:47,651][20381] Updated weights for policy 1, policy_version 601178 (0.0007) [2023-12-27 09:47:47,943][20347] Updated weights for policy 0, policy_version 601514 (0.0007) [2023-12-27 09:47:47,998][20347] Updated weights for policy 0, policy_version 601524 (0.0007) [2023-12-27 09:47:48,051][20347] Updated weights for policy 0, policy_version 601534 (0.0008) [2023-12-27 09:47:48,364][20381] Updated weights for policy 1, policy_version 601188 (0.0008) [2023-12-27 09:47:48,422][20381] Updated weights for policy 1, policy_version 601198 (0.0007) [2023-12-27 09:47:48,476][20381] Updated weights for policy 1, policy_version 601208 (0.0005) [2023-12-27 09:47:48,763][20347] Updated weights for policy 0, policy_version 601544 (0.0008) [2023-12-27 09:47:48,828][20347] Updated weights for policy 0, policy_version 601554 (0.0008) [2023-12-27 09:47:48,887][20347] Updated weights for policy 0, policy_version 601564 (0.0007) [2023-12-27 09:47:49,022][20381] Updated weights for policy 1, policy_version 601218 (0.0005) [2023-12-27 09:47:49,082][20381] Updated weights for policy 1, policy_version 601228 (0.0007) [2023-12-27 09:47:49,148][20381] Updated weights for policy 1, policy_version 601238 (0.0008) [2023-12-27 09:47:49,210][20381] Updated weights for policy 1, policy_version 601248 (0.0008) [2023-12-27 09:47:49,609][20347] Updated weights for policy 0, policy_version 601574 (0.0008) [2023-12-27 09:47:49,665][20347] Updated weights for policy 0, policy_version 601584 (0.0008) [2023-12-27 09:47:49,724][20347] Updated weights for policy 0, policy_version 601594 (0.0007) [2023-12-27 09:47:49,902][20381] Updated weights for policy 1, policy_version 601258 (0.0007) [2023-12-27 09:47:49,960][20381] Updated weights for policy 1, policy_version 601268 (0.0007) [2023-12-27 09:47:50,020][20381] Updated weights for policy 1, policy_version 601278 (0.0007) [2023-12-27 09:47:50,488][20347] Updated weights for policy 0, policy_version 601604 (0.0006) [2023-12-27 09:47:50,540][20347] Updated weights for policy 0, policy_version 601614 (0.0005) [2023-12-27 09:47:50,605][20347] Updated weights for policy 0, policy_version 601624 (0.0007) [2023-12-27 09:47:50,620][20381] Updated weights for policy 1, policy_version 601288 (0.0007) [2023-12-27 09:47:50,674][20381] Updated weights for policy 1, policy_version 601298 (0.0006) [2023-12-27 09:47:50,730][20381] Updated weights for policy 1, policy_version 601308 (0.0007) [2023-12-27 09:47:51,276][20347] Updated weights for policy 0, policy_version 601634 (0.0008) [2023-12-27 09:47:51,323][20347] Updated weights for policy 0, policy_version 601644 (0.0008) [2023-12-27 09:47:51,389][20347] Updated weights for policy 0, policy_version 601654 (0.0009) [2023-12-27 09:47:51,438][20381] Updated weights for policy 1, policy_version 601318 (0.0006) [2023-12-27 09:47:51,445][20347] Updated weights for policy 0, policy_version 601664 (0.0007) [2023-12-27 09:47:51,500][20381] Updated weights for policy 1, policy_version 601328 (0.0005) [2023-12-27 09:47:51,561][20381] Updated weights for policy 1, policy_version 601338 (0.0007) [2023-12-27 09:47:52,216][20347] Updated weights for policy 0, policy_version 601674 (0.0008) [2023-12-27 09:47:52,221][20381] Updated weights for policy 1, policy_version 601348 (0.0009) [2023-12-27 09:47:52,273][20347] Updated weights for policy 0, policy_version 601684 (0.0006) [2023-12-27 09:47:52,278][20381] Updated weights for policy 1, policy_version 601358 (0.0008) [2023-12-27 09:47:52,331][20347] Updated weights for policy 0, policy_version 601694 (0.0006) [2023-12-27 09:47:52,337][20381] Updated weights for policy 1, policy_version 601368 (0.0006) [2023-12-27 09:47:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 308019200. Throughput: 0: 10310.3, 1: 10433.4. Samples: 307990612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:52,363][19170] Avg episode reward: [(0, '563.180'), (1, '913.680')] [2023-12-27 09:47:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000601696_154050560.pth... [2023-12-27 09:47:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000600512_153747456.pth [2023-12-27 09:47:52,388][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000601376_153976832.pth... [2023-12-27 09:47:52,392][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000600128_153657344.pth [2023-12-27 09:47:52,973][20347] Updated weights for policy 0, policy_version 601704 (0.0010) [2023-12-27 09:47:53,030][20347] Updated weights for policy 0, policy_version 601714 (0.0010) [2023-12-27 09:47:53,065][20381] Updated weights for policy 1, policy_version 601378 (0.0008) [2023-12-27 09:47:53,090][20347] Updated weights for policy 0, policy_version 601724 (0.0010) [2023-12-27 09:47:53,130][20381] Updated weights for policy 1, policy_version 601388 (0.0007) [2023-12-27 09:47:53,183][20381] Updated weights for policy 1, policy_version 601398 (0.0007) [2023-12-27 09:47:53,254][20381] Updated weights for policy 1, policy_version 601408 (0.0008) [2023-12-27 09:47:53,700][20347] Updated weights for policy 0, policy_version 601734 (0.0010) [2023-12-27 09:47:53,745][20347] Updated weights for policy 0, policy_version 601744 (0.0010) [2023-12-27 09:47:53,794][20347] Updated weights for policy 0, policy_version 601754 (0.0010) [2023-12-27 09:47:53,919][20381] Updated weights for policy 1, policy_version 601418 (0.0008) [2023-12-27 09:47:53,980][20381] Updated weights for policy 1, policy_version 601428 (0.0007) [2023-12-27 09:47:54,034][20381] Updated weights for policy 1, policy_version 601438 (0.0007) [2023-12-27 09:47:54,414][20347] Updated weights for policy 0, policy_version 601764 (0.0009) [2023-12-27 09:47:54,477][20347] Updated weights for policy 0, policy_version 601774 (0.0009) [2023-12-27 09:47:54,537][20347] Updated weights for policy 0, policy_version 601784 (0.0006) [2023-12-27 09:47:54,667][20381] Updated weights for policy 1, policy_version 601448 (0.0006) [2023-12-27 09:47:54,727][20381] Updated weights for policy 1, policy_version 601458 (0.0005) [2023-12-27 09:47:54,788][20381] Updated weights for policy 1, policy_version 601468 (0.0006) [2023-12-27 09:47:55,084][20347] Updated weights for policy 0, policy_version 601794 (0.0007) [2023-12-27 09:47:55,136][20347] Updated weights for policy 0, policy_version 601804 (0.0005) [2023-12-27 09:47:55,189][20347] Updated weights for policy 0, policy_version 601814 (0.0005) [2023-12-27 09:47:55,244][20347] Updated weights for policy 0, policy_version 601824 (0.0005) [2023-12-27 09:47:55,374][20381] Updated weights for policy 1, policy_version 601478 (0.0006) [2023-12-27 09:47:55,419][20381] Updated weights for policy 1, policy_version 601488 (0.0008) [2023-12-27 09:47:55,469][20381] Updated weights for policy 1, policy_version 601498 (0.0008) [2023-12-27 09:47:55,835][20347] Updated weights for policy 0, policy_version 601834 (0.0007) [2023-12-27 09:47:55,888][20347] Updated weights for policy 0, policy_version 601844 (0.0006) [2023-12-27 09:47:55,941][20347] Updated weights for policy 0, policy_version 601854 (0.0008) [2023-12-27 09:47:56,167][20381] Updated weights for policy 1, policy_version 601508 (0.0007) [2023-12-27 09:47:56,229][20381] Updated weights for policy 1, policy_version 601518 (0.0005) [2023-12-27 09:47:56,283][20381] Updated weights for policy 1, policy_version 601528 (0.0005) [2023-12-27 09:47:56,582][20347] Updated weights for policy 0, policy_version 601864 (0.0006) [2023-12-27 09:47:56,637][20347] Updated weights for policy 0, policy_version 601874 (0.0005) [2023-12-27 09:47:56,685][20347] Updated weights for policy 0, policy_version 601884 (0.0005) [2023-12-27 09:47:56,943][20381] Updated weights for policy 1, policy_version 601538 (0.0007) [2023-12-27 09:47:56,993][20381] Updated weights for policy 1, policy_version 601548 (0.0008) [2023-12-27 09:47:57,048][20381] Updated weights for policy 1, policy_version 601558 (0.0006) [2023-12-27 09:47:57,102][20381] Updated weights for policy 1, policy_version 601568 (0.0007) [2023-12-27 09:47:57,310][20347] Updated weights for policy 0, policy_version 601894 (0.0006) [2023-12-27 09:47:57,362][20347] Updated weights for policy 0, policy_version 601904 (0.0007) [2023-12-27 09:47:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 308125696. Throughput: 0: 10327.5, 1: 10470.7. Samples: 308116396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:47:57,363][19170] Avg episode reward: [(0, '522.510'), (1, '884.040')] [2023-12-27 09:47:57,415][20347] Updated weights for policy 0, policy_version 601914 (0.0009) [2023-12-27 09:47:57,728][20381] Updated weights for policy 1, policy_version 601578 (0.0008) [2023-12-27 09:47:57,774][20381] Updated weights for policy 1, policy_version 601588 (0.0007) [2023-12-27 09:47:57,825][20381] Updated weights for policy 1, policy_version 601598 (0.0008) [2023-12-27 09:47:58,118][20347] Updated weights for policy 0, policy_version 601924 (0.0007) [2023-12-27 09:47:58,171][20347] Updated weights for policy 0, policy_version 601934 (0.0009) [2023-12-27 09:47:58,231][20347] Updated weights for policy 0, policy_version 601944 (0.0008) [2023-12-27 09:47:58,570][20381] Updated weights for policy 1, policy_version 601608 (0.0010) [2023-12-27 09:47:58,632][20381] Updated weights for policy 1, policy_version 601618 (0.0011) [2023-12-27 09:47:58,694][20381] Updated weights for policy 1, policy_version 601628 (0.0011) [2023-12-27 09:47:58,978][20347] Updated weights for policy 0, policy_version 601954 (0.0008) [2023-12-27 09:47:59,042][20347] Updated weights for policy 0, policy_version 601964 (0.0007) [2023-12-27 09:47:59,106][20347] Updated weights for policy 0, policy_version 601974 (0.0007) [2023-12-27 09:47:59,154][20347] Updated weights for policy 0, policy_version 601984 (0.0007) [2023-12-27 09:47:59,368][20381] Updated weights for policy 1, policy_version 601638 (0.0010) [2023-12-27 09:47:59,422][20381] Updated weights for policy 1, policy_version 601648 (0.0008) [2023-12-27 09:47:59,476][20381] Updated weights for policy 1, policy_version 601658 (0.0006) [2023-12-27 09:47:59,850][20347] Updated weights for policy 0, policy_version 601994 (0.0007) [2023-12-27 09:47:59,899][20347] Updated weights for policy 0, policy_version 602004 (0.0008) [2023-12-27 09:47:59,959][20347] Updated weights for policy 0, policy_version 602014 (0.0008) [2023-12-27 09:48:00,129][20381] Updated weights for policy 1, policy_version 601668 (0.0005) [2023-12-27 09:48:00,184][20381] Updated weights for policy 1, policy_version 601678 (0.0008) [2023-12-27 09:48:00,248][20381] Updated weights for policy 1, policy_version 601688 (0.0006) [2023-12-27 09:48:00,634][20347] Updated weights for policy 0, policy_version 602024 (0.0007) [2023-12-27 09:48:00,685][20347] Updated weights for policy 0, policy_version 602034 (0.0007) [2023-12-27 09:48:00,735][20347] Updated weights for policy 0, policy_version 602044 (0.0008) [2023-12-27 09:48:00,932][20381] Updated weights for policy 1, policy_version 601698 (0.0008) [2023-12-27 09:48:00,983][20381] Updated weights for policy 1, policy_version 601708 (0.0008) [2023-12-27 09:48:01,039][20381] Updated weights for policy 1, policy_version 601718 (0.0007) [2023-12-27 09:48:01,092][20381] Updated weights for policy 1, policy_version 601728 (0.0008) [2023-12-27 09:48:01,409][20347] Updated weights for policy 0, policy_version 602054 (0.0008) [2023-12-27 09:48:01,467][20347] Updated weights for policy 0, policy_version 602064 (0.0007) [2023-12-27 09:48:01,527][20347] Updated weights for policy 0, policy_version 602074 (0.0008) [2023-12-27 09:48:01,782][20381] Updated weights for policy 1, policy_version 601738 (0.0007) [2023-12-27 09:48:01,809][20292] KL-divergence is very high: 133.6125 [2023-12-27 09:48:01,835][20381] Updated weights for policy 1, policy_version 601748 (0.0005) [2023-12-27 09:48:01,855][20292] KL-divergence is very high: 101.0777 [2023-12-27 09:48:01,891][20381] Updated weights for policy 1, policy_version 601758 (0.0007) [2023-12-27 09:48:02,177][20347] Updated weights for policy 0, policy_version 602084 (0.0007) [2023-12-27 09:48:02,240][20347] Updated weights for policy 0, policy_version 602094 (0.0005) [2023-12-27 09:48:02,301][20347] Updated weights for policy 0, policy_version 602104 (0.0007) [2023-12-27 09:48:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 308232192. Throughput: 0: 10276.0, 1: 10419.9. Samples: 308239828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:02,363][19170] Avg episode reward: [(0, '488.810'), (1, '834.670')] [2023-12-27 09:48:02,549][20381] Updated weights for policy 1, policy_version 601768 (0.0007) [2023-12-27 09:48:02,602][20381] Updated weights for policy 1, policy_version 601778 (0.0008) [2023-12-27 09:48:02,667][20381] Updated weights for policy 1, policy_version 601788 (0.0005) [2023-12-27 09:48:02,994][20347] Updated weights for policy 0, policy_version 602114 (0.0008) [2023-12-27 09:48:03,052][20347] Updated weights for policy 0, policy_version 602124 (0.0006) [2023-12-27 09:48:03,101][20347] Updated weights for policy 0, policy_version 602134 (0.0008) [2023-12-27 09:48:03,145][20347] Updated weights for policy 0, policy_version 602144 (0.0007) [2023-12-27 09:48:03,284][20381] Updated weights for policy 1, policy_version 601798 (0.0007) [2023-12-27 09:48:03,336][20381] Updated weights for policy 1, policy_version 601808 (0.0009) [2023-12-27 09:48:03,389][20381] Updated weights for policy 1, policy_version 601818 (0.0007) [2023-12-27 09:48:03,814][20347] Updated weights for policy 0, policy_version 602154 (0.0005) [2023-12-27 09:48:03,871][20347] Updated weights for policy 0, policy_version 602164 (0.0007) [2023-12-27 09:48:03,926][20347] Updated weights for policy 0, policy_version 602174 (0.0008) [2023-12-27 09:48:04,083][20381] Updated weights for policy 1, policy_version 601828 (0.0008) [2023-12-27 09:48:04,143][20381] Updated weights for policy 1, policy_version 601838 (0.0009) [2023-12-27 09:48:04,203][20381] Updated weights for policy 1, policy_version 601848 (0.0007) [2023-12-27 09:48:04,590][20347] Updated weights for policy 0, policy_version 602184 (0.0007) [2023-12-27 09:48:04,641][20347] Updated weights for policy 0, policy_version 602194 (0.0010) [2023-12-27 09:48:04,687][20347] Updated weights for policy 0, policy_version 602204 (0.0010) [2023-12-27 09:48:04,828][20381] Updated weights for policy 1, policy_version 601858 (0.0008) [2023-12-27 09:48:04,878][20381] Updated weights for policy 1, policy_version 601868 (0.0008) [2023-12-27 09:48:04,927][20381] Updated weights for policy 1, policy_version 601878 (0.0007) [2023-12-27 09:48:04,977][20381] Updated weights for policy 1, policy_version 601888 (0.0009) [2023-12-27 09:48:05,323][20347] Updated weights for policy 0, policy_version 602214 (0.0007) [2023-12-27 09:48:05,375][20347] Updated weights for policy 0, policy_version 602224 (0.0010) [2023-12-27 09:48:05,430][20347] Updated weights for policy 0, policy_version 602234 (0.0010) [2023-12-27 09:48:05,680][20381] Updated weights for policy 1, policy_version 601898 (0.0005) [2023-12-27 09:48:05,738][20381] Updated weights for policy 1, policy_version 601908 (0.0006) [2023-12-27 09:48:05,792][20381] Updated weights for policy 1, policy_version 601918 (0.0007) [2023-12-27 09:48:06,053][20347] Updated weights for policy 0, policy_version 602244 (0.0009) [2023-12-27 09:48:06,102][20347] Updated weights for policy 0, policy_version 602254 (0.0008) [2023-12-27 09:48:06,164][20347] Updated weights for policy 0, policy_version 602264 (0.0008) [2023-12-27 09:48:06,493][20381] Updated weights for policy 1, policy_version 601928 (0.0007) [2023-12-27 09:48:06,563][20381] Updated weights for policy 1, policy_version 601938 (0.0008) [2023-12-27 09:48:06,622][20381] Updated weights for policy 1, policy_version 601948 (0.0008) [2023-12-27 09:48:06,858][20347] Updated weights for policy 0, policy_version 602274 (0.0008) [2023-12-27 09:48:06,911][20347] Updated weights for policy 0, policy_version 602284 (0.0008) [2023-12-27 09:48:06,964][20347] Updated weights for policy 0, policy_version 602294 (0.0009) [2023-12-27 09:48:07,021][20347] Updated weights for policy 0, policy_version 602304 (0.0007) [2023-12-27 09:48:07,182][20381] Updated weights for policy 1, policy_version 601958 (0.0008) [2023-12-27 09:48:07,237][20381] Updated weights for policy 1, policy_version 601968 (0.0010) [2023-12-27 09:48:07,285][20381] Updated weights for policy 1, policy_version 601978 (0.0008) [2023-12-27 09:48:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 308338688. Throughput: 0: 10255.0, 1: 10406.1. Samples: 308301776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:07,364][19170] Avg episode reward: [(0, '676.320'), (1, '834.610')] [2023-12-27 09:48:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000601984_154132480.pth... [2023-12-27 09:48:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000602304_154206208.pth... [2023-12-27 09:48:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000601088_153894912.pth [2023-12-27 09:48:07,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000600736_153812992.pth [2023-12-27 09:48:07,746][20347] Updated weights for policy 0, policy_version 602314 (0.0007) [2023-12-27 09:48:07,790][20347] Updated weights for policy 0, policy_version 602324 (0.0007) [2023-12-27 09:48:07,834][20347] Updated weights for policy 0, policy_version 602334 (0.0007) [2023-12-27 09:48:07,943][20381] Updated weights for policy 1, policy_version 601988 (0.0007) [2023-12-27 09:48:07,995][20381] Updated weights for policy 1, policy_version 601998 (0.0010) [2023-12-27 09:48:08,055][20381] Updated weights for policy 1, policy_version 602008 (0.0010) [2023-12-27 09:48:08,542][20347] Updated weights for policy 0, policy_version 602344 (0.0007) [2023-12-27 09:48:08,602][20347] Updated weights for policy 0, policy_version 602354 (0.0007) [2023-12-27 09:48:08,659][20347] Updated weights for policy 0, policy_version 602364 (0.0008) [2023-12-27 09:48:08,791][20381] Updated weights for policy 1, policy_version 602018 (0.0010) [2023-12-27 09:48:08,844][20381] Updated weights for policy 1, policy_version 602028 (0.0009) [2023-12-27 09:48:08,894][20381] Updated weights for policy 1, policy_version 602038 (0.0011) [2023-12-27 09:48:08,939][20381] Updated weights for policy 1, policy_version 602048 (0.0006) [2023-12-27 09:48:09,383][20347] Updated weights for policy 0, policy_version 602374 (0.0008) [2023-12-27 09:48:09,445][20347] Updated weights for policy 0, policy_version 602384 (0.0008) [2023-12-27 09:48:09,498][20347] Updated weights for policy 0, policy_version 602394 (0.0007) [2023-12-27 09:48:09,595][20381] Updated weights for policy 1, policy_version 602058 (0.0011) [2023-12-27 09:48:09,647][20381] Updated weights for policy 1, policy_version 602068 (0.0011) [2023-12-27 09:48:09,697][20381] Updated weights for policy 1, policy_version 602078 (0.0011) [2023-12-27 09:48:10,223][20347] Updated weights for policy 0, policy_version 602404 (0.0008) [2023-12-27 09:48:10,280][20347] Updated weights for policy 0, policy_version 602414 (0.0007) [2023-12-27 09:48:10,337][20347] Updated weights for policy 0, policy_version 602424 (0.0009) [2023-12-27 09:48:10,413][20381] Updated weights for policy 1, policy_version 602088 (0.0009) [2023-12-27 09:48:10,468][20381] Updated weights for policy 1, policy_version 602098 (0.0008) [2023-12-27 09:48:10,519][20381] Updated weights for policy 1, policy_version 602108 (0.0008) [2023-12-27 09:48:10,949][20347] Updated weights for policy 0, policy_version 602434 (0.0008) [2023-12-27 09:48:11,005][20347] Updated weights for policy 0, policy_version 602444 (0.0009) [2023-12-27 09:48:11,066][20347] Updated weights for policy 0, policy_version 602454 (0.0008) [2023-12-27 09:48:11,120][20347] Updated weights for policy 0, policy_version 602464 (0.0005) [2023-12-27 09:48:11,136][20381] Updated weights for policy 1, policy_version 602118 (0.0009) [2023-12-27 09:48:11,204][20381] Updated weights for policy 1, policy_version 602128 (0.0008) [2023-12-27 09:48:11,272][20381] Updated weights for policy 1, policy_version 602138 (0.0009) [2023-12-27 09:48:11,836][20347] Updated weights for policy 0, policy_version 602474 (0.0005) [2023-12-27 09:48:11,890][20347] Updated weights for policy 0, policy_version 602484 (0.0006) [2023-12-27 09:48:11,948][20347] Updated weights for policy 0, policy_version 602494 (0.0006) [2023-12-27 09:48:12,015][20381] Updated weights for policy 1, policy_version 602148 (0.0009) [2023-12-27 09:48:12,065][20381] Updated weights for policy 1, policy_version 602158 (0.0010) [2023-12-27 09:48:12,114][20381] Updated weights for policy 1, policy_version 602168 (0.0011) [2023-12-27 09:48:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 308436992. Throughput: 0: 10269.6, 1: 10401.4. Samples: 308426232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:12,363][19170] Avg episode reward: [(0, '786.000'), (1, '854.270')] [2023-12-27 09:48:12,525][20347] Updated weights for policy 0, policy_version 602504 (0.0007) [2023-12-27 09:48:12,577][20347] Updated weights for policy 0, policy_version 602514 (0.0007) [2023-12-27 09:48:12,622][20347] Updated weights for policy 0, policy_version 602524 (0.0008) [2023-12-27 09:48:12,814][20381] Updated weights for policy 1, policy_version 602178 (0.0010) [2023-12-27 09:48:12,867][20381] Updated weights for policy 1, policy_version 602188 (0.0008) [2023-12-27 09:48:12,919][20381] Updated weights for policy 1, policy_version 602198 (0.0006) [2023-12-27 09:48:12,970][20381] Updated weights for policy 1, policy_version 602208 (0.0005) [2023-12-27 09:48:13,356][20347] Updated weights for policy 0, policy_version 602534 (0.0006) [2023-12-27 09:48:13,406][20347] Updated weights for policy 0, policy_version 602544 (0.0005) [2023-12-27 09:48:13,458][20347] Updated weights for policy 0, policy_version 602554 (0.0005) [2023-12-27 09:48:13,604][20381] Updated weights for policy 1, policy_version 602218 (0.0009) [2023-12-27 09:48:13,687][20381] Updated weights for policy 1, policy_version 602228 (0.0008) [2023-12-27 09:48:13,746][20381] Updated weights for policy 1, policy_version 602238 (0.0008) [2023-12-27 09:48:14,079][20347] Updated weights for policy 0, policy_version 602564 (0.0006) [2023-12-27 09:48:14,133][20347] Updated weights for policy 0, policy_version 602574 (0.0006) [2023-12-27 09:48:14,183][20347] Updated weights for policy 0, policy_version 602584 (0.0006) [2023-12-27 09:48:14,422][20381] Updated weights for policy 1, policy_version 602248 (0.0006) [2023-12-27 09:48:14,478][20381] Updated weights for policy 1, policy_version 602258 (0.0005) [2023-12-27 09:48:14,527][20381] Updated weights for policy 1, policy_version 602268 (0.0005) [2023-12-27 09:48:14,837][20347] Updated weights for policy 0, policy_version 602594 (0.0007) [2023-12-27 09:48:14,894][20347] Updated weights for policy 0, policy_version 602604 (0.0008) [2023-12-27 09:48:14,954][20347] Updated weights for policy 0, policy_version 602614 (0.0008) [2023-12-27 09:48:15,012][20347] Updated weights for policy 0, policy_version 602624 (0.0008) [2023-12-27 09:48:15,155][20381] Updated weights for policy 1, policy_version 602278 (0.0007) [2023-12-27 09:48:15,212][20381] Updated weights for policy 1, policy_version 602288 (0.0008) [2023-12-27 09:48:15,266][20381] Updated weights for policy 1, policy_version 602298 (0.0008) [2023-12-27 09:48:15,704][20347] Updated weights for policy 0, policy_version 602634 (0.0007) [2023-12-27 09:48:15,753][20347] Updated weights for policy 0, policy_version 602644 (0.0008) [2023-12-27 09:48:15,806][20347] Updated weights for policy 0, policy_version 602654 (0.0008) [2023-12-27 09:48:15,966][20381] Updated weights for policy 1, policy_version 602308 (0.0008) [2023-12-27 09:48:16,016][20381] Updated weights for policy 1, policy_version 602318 (0.0007) [2023-12-27 09:48:16,064][20381] Updated weights for policy 1, policy_version 602328 (0.0008) [2023-12-27 09:48:16,535][20347] Updated weights for policy 0, policy_version 602664 (0.0009) [2023-12-27 09:48:16,587][20347] Updated weights for policy 0, policy_version 602674 (0.0007) [2023-12-27 09:48:16,635][20347] Updated weights for policy 0, policy_version 602684 (0.0008) [2023-12-27 09:48:16,673][20381] Updated weights for policy 1, policy_version 602338 (0.0007) [2023-12-27 09:48:16,719][20381] Updated weights for policy 1, policy_version 602348 (0.0007) [2023-12-27 09:48:16,776][20381] Updated weights for policy 1, policy_version 602358 (0.0006) [2023-12-27 09:48:16,831][20381] Updated weights for policy 1, policy_version 602368 (0.0006) [2023-12-27 09:48:17,347][20347] Updated weights for policy 0, policy_version 602694 (0.0010) [2023-12-27 09:48:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 308535296. Throughput: 0: 10279.1, 1: 10462.3. Samples: 308551516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:17,363][19170] Avg episode reward: [(0, '815.460'), (1, '844.760')] [2023-12-27 09:48:17,386][20381] Updated weights for policy 1, policy_version 602378 (0.0009) [2023-12-27 09:48:17,402][20347] Updated weights for policy 0, policy_version 602704 (0.0010) [2023-12-27 09:48:17,447][20381] Updated weights for policy 1, policy_version 602388 (0.0005) [2023-12-27 09:48:17,465][20347] Updated weights for policy 0, policy_version 602714 (0.0010) [2023-12-27 09:48:17,498][20381] Updated weights for policy 1, policy_version 602398 (0.0005) [2023-12-27 09:48:18,085][20347] Updated weights for policy 0, policy_version 602724 (0.0007) [2023-12-27 09:48:18,121][20381] Updated weights for policy 1, policy_version 602408 (0.0009) [2023-12-27 09:48:18,140][20347] Updated weights for policy 0, policy_version 602734 (0.0005) [2023-12-27 09:48:18,175][20381] Updated weights for policy 1, policy_version 602418 (0.0010) [2023-12-27 09:48:18,197][20347] Updated weights for policy 0, policy_version 602744 (0.0006) [2023-12-27 09:48:18,224][20381] Updated weights for policy 1, policy_version 602428 (0.0010) [2023-12-27 09:48:18,746][20347] Updated weights for policy 0, policy_version 602754 (0.0005) [2023-12-27 09:48:18,800][20347] Updated weights for policy 0, policy_version 602764 (0.0008) [2023-12-27 09:48:18,851][20347] Updated weights for policy 0, policy_version 602774 (0.0007) [2023-12-27 09:48:18,904][20347] Updated weights for policy 0, policy_version 602784 (0.0008) [2023-12-27 09:48:18,937][20381] Updated weights for policy 1, policy_version 602438 (0.0008) [2023-12-27 09:48:18,998][20381] Updated weights for policy 1, policy_version 602448 (0.0005) [2023-12-27 09:48:19,055][20381] Updated weights for policy 1, policy_version 602458 (0.0008) [2023-12-27 09:48:19,653][20347] Updated weights for policy 0, policy_version 602794 (0.0007) [2023-12-27 09:48:19,678][20381] Updated weights for policy 1, policy_version 602468 (0.0007) [2023-12-27 09:48:19,711][20347] Updated weights for policy 0, policy_version 602804 (0.0008) [2023-12-27 09:48:19,745][20381] Updated weights for policy 1, policy_version 602478 (0.0006) [2023-12-27 09:48:19,767][20347] Updated weights for policy 0, policy_version 602814 (0.0008) [2023-12-27 09:48:19,800][20381] Updated weights for policy 1, policy_version 602488 (0.0005) [2023-12-27 09:48:20,465][20381] Updated weights for policy 1, policy_version 602498 (0.0007) [2023-12-27 09:48:20,489][20347] Updated weights for policy 0, policy_version 602824 (0.0006) [2023-12-27 09:48:20,522][20381] Updated weights for policy 1, policy_version 602508 (0.0008) [2023-12-27 09:48:20,543][20347] Updated weights for policy 0, policy_version 602834 (0.0006) [2023-12-27 09:48:20,578][20381] Updated weights for policy 1, policy_version 602518 (0.0008) [2023-12-27 09:48:20,602][20347] Updated weights for policy 0, policy_version 602844 (0.0007) [2023-12-27 09:48:20,640][20381] Updated weights for policy 1, policy_version 602528 (0.0011) [2023-12-27 09:48:21,334][20381] Updated weights for policy 1, policy_version 602538 (0.0007) [2023-12-27 09:48:21,373][20347] Updated weights for policy 0, policy_version 602854 (0.0007) [2023-12-27 09:48:21,401][20381] Updated weights for policy 1, policy_version 602548 (0.0009) [2023-12-27 09:48:21,441][20347] Updated weights for policy 0, policy_version 602864 (0.0007) [2023-12-27 09:48:21,455][20381] Updated weights for policy 1, policy_version 602558 (0.0006) [2023-12-27 09:48:21,500][20347] Updated weights for policy 0, policy_version 602874 (0.0007) [2023-12-27 09:48:22,164][20381] Updated weights for policy 1, policy_version 602568 (0.0007) [2023-12-27 09:48:22,188][20347] Updated weights for policy 0, policy_version 602884 (0.0007) [2023-12-27 09:48:22,213][20381] Updated weights for policy 1, policy_version 602578 (0.0006) [2023-12-27 09:48:22,233][20347] Updated weights for policy 0, policy_version 602894 (0.0008) [2023-12-27 09:48:22,264][20381] Updated weights for policy 1, policy_version 602588 (0.0006) [2023-12-27 09:48:22,286][20347] Updated weights for policy 0, policy_version 602904 (0.0008) [2023-12-27 09:48:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 308649984. Throughput: 0: 10304.1, 1: 10456.3. Samples: 308614868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:22,364][19170] Avg episode reward: [(0, '844.470'), (1, '873.700')] [2023-12-27 09:48:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000602592_154288128.pth... [2023-12-27 09:48:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000602912_154361856.pth... [2023-12-27 09:48:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000601696_154050560.pth [2023-12-27 09:48:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000601376_153976832.pth [2023-12-27 09:48:22,870][20381] Updated weights for policy 1, policy_version 602598 (0.0007) [2023-12-27 09:48:22,922][20381] Updated weights for policy 1, policy_version 602608 (0.0008) [2023-12-27 09:48:22,975][20381] Updated weights for policy 1, policy_version 602618 (0.0008) [2023-12-27 09:48:23,026][20347] Updated weights for policy 0, policy_version 602914 (0.0008) [2023-12-27 09:48:23,090][20347] Updated weights for policy 0, policy_version 602924 (0.0008) [2023-12-27 09:48:23,147][20347] Updated weights for policy 0, policy_version 602934 (0.0007) [2023-12-27 09:48:23,196][20347] Updated weights for policy 0, policy_version 602944 (0.0008) [2023-12-27 09:48:23,653][20381] Updated weights for policy 1, policy_version 602628 (0.0008) [2023-12-27 09:48:23,708][20381] Updated weights for policy 1, policy_version 602638 (0.0008) [2023-12-27 09:48:23,768][20381] Updated weights for policy 1, policy_version 602648 (0.0007) [2023-12-27 09:48:23,913][20347] Updated weights for policy 0, policy_version 602954 (0.0006) [2023-12-27 09:48:23,970][20347] Updated weights for policy 0, policy_version 602964 (0.0006) [2023-12-27 09:48:24,029][20347] Updated weights for policy 0, policy_version 602974 (0.0006) [2023-12-27 09:48:24,444][20381] Updated weights for policy 1, policy_version 602658 (0.0007) [2023-12-27 09:48:24,503][20381] Updated weights for policy 1, policy_version 602668 (0.0008) [2023-12-27 09:48:24,566][20381] Updated weights for policy 1, policy_version 602678 (0.0007) [2023-12-27 09:48:24,584][20292] KL-divergence is very high: 160.5914 [2023-12-27 09:48:24,623][20381] Updated weights for policy 1, policy_version 602688 (0.0006) [2023-12-27 09:48:24,644][20347] Updated weights for policy 0, policy_version 602984 (0.0007) [2023-12-27 09:48:24,693][20347] Updated weights for policy 0, policy_version 602994 (0.0008) [2023-12-27 09:48:24,750][20347] Updated weights for policy 0, policy_version 603004 (0.0005) [2023-12-27 09:48:25,292][20381] Updated weights for policy 1, policy_version 602698 (0.0010) [2023-12-27 09:48:25,340][20381] Updated weights for policy 1, policy_version 602708 (0.0010) [2023-12-27 09:48:25,388][20381] Updated weights for policy 1, policy_version 602718 (0.0010) [2023-12-27 09:48:25,407][20347] Updated weights for policy 0, policy_version 603014 (0.0007) [2023-12-27 09:48:25,461][20347] Updated weights for policy 0, policy_version 603024 (0.0005) [2023-12-27 09:48:25,514][20347] Updated weights for policy 0, policy_version 603034 (0.0005) [2023-12-27 09:48:25,998][20381] Updated weights for policy 1, policy_version 602728 (0.0010) [2023-12-27 09:48:26,053][20381] Updated weights for policy 1, policy_version 602738 (0.0010) [2023-12-27 09:48:26,108][20381] Updated weights for policy 1, policy_version 602748 (0.0010) [2023-12-27 09:48:26,128][20347] Updated weights for policy 0, policy_version 603044 (0.0007) [2023-12-27 09:48:26,182][20347] Updated weights for policy 0, policy_version 603054 (0.0007) [2023-12-27 09:48:26,236][20347] Updated weights for policy 0, policy_version 603064 (0.0006) [2023-12-27 09:48:26,788][20381] Updated weights for policy 1, policy_version 602758 (0.0011) [2023-12-27 09:48:26,846][20381] Updated weights for policy 1, policy_version 602768 (0.0010) [2023-12-27 09:48:26,892][20347] Updated weights for policy 0, policy_version 603074 (0.0007) [2023-12-27 09:48:26,897][20381] Updated weights for policy 1, policy_version 602778 (0.0010) [2023-12-27 09:48:26,942][20347] Updated weights for policy 0, policy_version 603084 (0.0007) [2023-12-27 09:48:26,990][20347] Updated weights for policy 0, policy_version 603094 (0.0007) [2023-12-27 09:48:27,040][20347] Updated weights for policy 0, policy_version 603104 (0.0008) [2023-12-27 09:48:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 308748288. Throughput: 0: 10274.2, 1: 10488.3. Samples: 308738060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:27,363][19170] Avg episode reward: [(0, '902.660'), (1, '864.870')] [2023-12-27 09:48:27,501][20381] Updated weights for policy 1, policy_version 602788 (0.0008) [2023-12-27 09:48:27,564][20381] Updated weights for policy 1, policy_version 602798 (0.0005) [2023-12-27 09:48:27,625][20381] Updated weights for policy 1, policy_version 602808 (0.0008) [2023-12-27 09:48:27,729][20347] Updated weights for policy 0, policy_version 603114 (0.0007) [2023-12-27 09:48:27,778][20347] Updated weights for policy 0, policy_version 603124 (0.0006) [2023-12-27 09:48:27,827][20347] Updated weights for policy 0, policy_version 603134 (0.0005) [2023-12-27 09:48:28,212][20381] Updated weights for policy 1, policy_version 602818 (0.0010) [2023-12-27 09:48:28,269][20381] Updated weights for policy 1, policy_version 602828 (0.0010) [2023-12-27 09:48:28,326][20381] Updated weights for policy 1, policy_version 602838 (0.0010) [2023-12-27 09:48:28,381][20381] Updated weights for policy 1, policy_version 602848 (0.0011) [2023-12-27 09:48:28,465][20347] Updated weights for policy 0, policy_version 603144 (0.0007) [2023-12-27 09:48:28,522][20347] Updated weights for policy 0, policy_version 603154 (0.0007) [2023-12-27 09:48:28,582][20347] Updated weights for policy 0, policy_version 603164 (0.0008) [2023-12-27 09:48:28,966][20381] Updated weights for policy 1, policy_version 602858 (0.0005) [2023-12-27 09:48:29,012][20381] Updated weights for policy 1, policy_version 602868 (0.0005) [2023-12-27 09:48:29,066][20381] Updated weights for policy 1, policy_version 602878 (0.0007) [2023-12-27 09:48:29,326][20347] Updated weights for policy 0, policy_version 603174 (0.0007) [2023-12-27 09:48:29,391][20347] Updated weights for policy 0, policy_version 603184 (0.0009) [2023-12-27 09:48:29,457][20347] Updated weights for policy 0, policy_version 603194 (0.0011) [2023-12-27 09:48:29,777][20381] Updated weights for policy 1, policy_version 602888 (0.0008) [2023-12-27 09:48:29,832][20381] Updated weights for policy 1, policy_version 602898 (0.0008) [2023-12-27 09:48:29,880][20381] Updated weights for policy 1, policy_version 602908 (0.0008) [2023-12-27 09:48:30,048][20347] Updated weights for policy 0, policy_version 603204 (0.0009) [2023-12-27 09:48:30,111][20347] Updated weights for policy 0, policy_version 603214 (0.0005) [2023-12-27 09:48:30,175][20347] Updated weights for policy 0, policy_version 603224 (0.0006) [2023-12-27 09:48:30,604][20381] Updated weights for policy 1, policy_version 602918 (0.0007) [2023-12-27 09:48:30,664][20381] Updated weights for policy 1, policy_version 602928 (0.0007) [2023-12-27 09:48:30,723][20381] Updated weights for policy 1, policy_version 602938 (0.0007) [2023-12-27 09:48:30,826][20347] Updated weights for policy 0, policy_version 603234 (0.0006) [2023-12-27 09:48:30,877][20347] Updated weights for policy 0, policy_version 603244 (0.0007) [2023-12-27 09:48:30,931][20347] Updated weights for policy 0, policy_version 603254 (0.0007) [2023-12-27 09:48:30,980][20347] Updated weights for policy 0, policy_version 603264 (0.0008) [2023-12-27 09:48:31,375][20381] Updated weights for policy 1, policy_version 602948 (0.0007) [2023-12-27 09:48:31,437][20381] Updated weights for policy 1, policy_version 602958 (0.0007) [2023-12-27 09:48:31,498][20381] Updated weights for policy 1, policy_version 602968 (0.0008) [2023-12-27 09:48:31,664][20347] Updated weights for policy 0, policy_version 603274 (0.0007) [2023-12-27 09:48:31,723][20347] Updated weights for policy 0, policy_version 603284 (0.0007) [2023-12-27 09:48:31,782][20347] Updated weights for policy 0, policy_version 603294 (0.0008) [2023-12-27 09:48:32,123][20381] Updated weights for policy 1, policy_version 602978 (0.0008) [2023-12-27 09:48:32,182][20381] Updated weights for policy 1, policy_version 602988 (0.0005) [2023-12-27 09:48:32,237][20381] Updated weights for policy 1, policy_version 602998 (0.0008) [2023-12-27 09:48:32,293][20381] Updated weights for policy 1, policy_version 603008 (0.0008) [2023-12-27 09:48:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 308854784. Throughput: 0: 10297.3, 1: 10494.0. Samples: 308864692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:32,363][19170] Avg episode reward: [(0, '951.610'), (1, '913.160')] [2023-12-27 09:48:32,396][20347] Updated weights for policy 0, policy_version 603304 (0.0010) [2023-12-27 09:48:32,452][20347] Updated weights for policy 0, policy_version 603314 (0.0010) [2023-12-27 09:48:32,513][20347] Updated weights for policy 0, policy_version 603324 (0.0008) [2023-12-27 09:48:33,013][20381] Updated weights for policy 1, policy_version 603018 (0.0007) [2023-12-27 09:48:33,060][20381] Updated weights for policy 1, policy_version 603028 (0.0007) [2023-12-27 09:48:33,085][20347] Updated weights for policy 0, policy_version 603334 (0.0010) [2023-12-27 09:48:33,114][20381] Updated weights for policy 1, policy_version 603038 (0.0009) [2023-12-27 09:48:33,132][20347] Updated weights for policy 0, policy_version 603344 (0.0010) [2023-12-27 09:48:33,179][20347] Updated weights for policy 0, policy_version 603354 (0.0010) [2023-12-27 09:48:33,820][20381] Updated weights for policy 1, policy_version 603048 (0.0008) [2023-12-27 09:48:33,854][20347] Updated weights for policy 0, policy_version 603364 (0.0010) [2023-12-27 09:48:33,881][20381] Updated weights for policy 1, policy_version 603058 (0.0005) [2023-12-27 09:48:33,914][20347] Updated weights for policy 0, policy_version 603374 (0.0010) [2023-12-27 09:48:33,934][20381] Updated weights for policy 1, policy_version 603068 (0.0005) [2023-12-27 09:48:33,972][20347] Updated weights for policy 0, policy_version 603384 (0.0010) [2023-12-27 09:48:34,654][20381] Updated weights for policy 1, policy_version 603078 (0.0007) [2023-12-27 09:48:34,666][20347] Updated weights for policy 0, policy_version 603394 (0.0010) [2023-12-27 09:48:34,713][20381] Updated weights for policy 1, policy_version 603088 (0.0006) [2023-12-27 09:48:34,722][20347] Updated weights for policy 0, policy_version 603404 (0.0010) [2023-12-27 09:48:34,775][20381] Updated weights for policy 1, policy_version 603098 (0.0005) [2023-12-27 09:48:34,780][20347] Updated weights for policy 0, policy_version 603414 (0.0009) [2023-12-27 09:48:34,835][20347] Updated weights for policy 0, policy_version 603424 (0.0010) [2023-12-27 09:48:35,441][20381] Updated weights for policy 1, policy_version 603108 (0.0006) [2023-12-27 09:48:35,500][20381] Updated weights for policy 1, policy_version 603118 (0.0006) [2023-12-27 09:48:35,514][20347] Updated weights for policy 0, policy_version 603434 (0.0010) [2023-12-27 09:48:35,556][20381] Updated weights for policy 1, policy_version 603128 (0.0005) [2023-12-27 09:48:35,562][20347] Updated weights for policy 0, policy_version 603444 (0.0010) [2023-12-27 09:48:35,616][20347] Updated weights for policy 0, policy_version 603454 (0.0010) [2023-12-27 09:48:36,270][20381] Updated weights for policy 1, policy_version 603138 (0.0006) [2023-12-27 09:48:36,313][20347] Updated weights for policy 0, policy_version 603464 (0.0010) [2023-12-27 09:48:36,323][20381] Updated weights for policy 1, policy_version 603148 (0.0006) [2023-12-27 09:48:36,369][20347] Updated weights for policy 0, policy_version 603474 (0.0010) [2023-12-27 09:48:36,374][20381] Updated weights for policy 1, policy_version 603158 (0.0006) [2023-12-27 09:48:36,419][20347] Updated weights for policy 0, policy_version 603484 (0.0010) [2023-12-27 09:48:36,431][20381] Updated weights for policy 1, policy_version 603168 (0.0006) [2023-12-27 09:48:37,048][20347] Updated weights for policy 0, policy_version 603494 (0.0010) [2023-12-27 09:48:37,106][20347] Updated weights for policy 0, policy_version 603504 (0.0010) [2023-12-27 09:48:37,115][20381] Updated weights for policy 1, policy_version 603178 (0.0005) [2023-12-27 09:48:37,165][20347] Updated weights for policy 0, policy_version 603514 (0.0009) [2023-12-27 09:48:37,166][20381] Updated weights for policy 1, policy_version 603188 (0.0006) [2023-12-27 09:48:37,228][20381] Updated weights for policy 1, policy_version 603198 (0.0005) [2023-12-27 09:48:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 308961280. Throughput: 0: 10373.7, 1: 10423.1. Samples: 308926464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:37,363][19170] Avg episode reward: [(0, '971.060'), (1, '893.430')] [2023-12-27 09:48:37,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000603520_154517504.pth... [2023-12-27 09:48:37,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000603200_154443776.pth... [2023-12-27 09:48:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000602304_154206208.pth [2023-12-27 09:48:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000601984_154132480.pth [2023-12-27 09:48:37,792][20381] Updated weights for policy 1, policy_version 603208 (0.0007) [2023-12-27 09:48:37,845][20381] Updated weights for policy 1, policy_version 603218 (0.0006) [2023-12-27 09:48:37,853][20347] Updated weights for policy 0, policy_version 603524 (0.0011) [2023-12-27 09:48:37,895][20381] Updated weights for policy 1, policy_version 603228 (0.0008) [2023-12-27 09:48:37,915][20347] Updated weights for policy 0, policy_version 603534 (0.0010) [2023-12-27 09:48:37,964][20347] Updated weights for policy 0, policy_version 603544 (0.0010) [2023-12-27 09:48:38,580][20347] Updated weights for policy 0, policy_version 603554 (0.0010) [2023-12-27 09:48:38,598][20381] Updated weights for policy 1, policy_version 603238 (0.0008) [2023-12-27 09:48:38,638][20347] Updated weights for policy 0, policy_version 603564 (0.0010) [2023-12-27 09:48:38,658][20381] Updated weights for policy 1, policy_version 603248 (0.0008) [2023-12-27 09:48:38,690][20347] Updated weights for policy 0, policy_version 603574 (0.0005) [2023-12-27 09:48:38,715][20381] Updated weights for policy 1, policy_version 603258 (0.0010) [2023-12-27 09:48:38,752][20347] Updated weights for policy 0, policy_version 603584 (0.0006) [2023-12-27 09:48:39,341][20381] Updated weights for policy 1, policy_version 603268 (0.0009) [2023-12-27 09:48:39,382][20347] Updated weights for policy 0, policy_version 603594 (0.0007) [2023-12-27 09:48:39,398][20381] Updated weights for policy 1, policy_version 603278 (0.0008) [2023-12-27 09:48:39,452][20347] Updated weights for policy 0, policy_version 603604 (0.0007) [2023-12-27 09:48:39,462][20381] Updated weights for policy 1, policy_version 603288 (0.0006) [2023-12-27 09:48:39,511][20347] Updated weights for policy 0, policy_version 603614 (0.0007) [2023-12-27 09:48:40,173][20381] Updated weights for policy 1, policy_version 603298 (0.0006) [2023-12-27 09:48:40,220][20347] Updated weights for policy 0, policy_version 603624 (0.0010) [2023-12-27 09:48:40,227][20381] Updated weights for policy 1, policy_version 603308 (0.0006) [2023-12-27 09:48:40,270][20347] Updated weights for policy 0, policy_version 603634 (0.0010) [2023-12-27 09:48:40,284][20381] Updated weights for policy 1, policy_version 603318 (0.0006) [2023-12-27 09:48:40,320][20347] Updated weights for policy 0, policy_version 603644 (0.0010) [2023-12-27 09:48:40,343][20381] Updated weights for policy 1, policy_version 603328 (0.0006) [2023-12-27 09:48:40,974][20347] Updated weights for policy 0, policy_version 603654 (0.0007) [2023-12-27 09:48:41,032][20347] Updated weights for policy 0, policy_version 603664 (0.0007) [2023-12-27 09:48:41,070][20381] Updated weights for policy 1, policy_version 603338 (0.0008) [2023-12-27 09:48:41,092][20347] Updated weights for policy 0, policy_version 603674 (0.0008) [2023-12-27 09:48:41,133][20381] Updated weights for policy 1, policy_version 603348 (0.0008) [2023-12-27 09:48:41,196][20381] Updated weights for policy 1, policy_version 603358 (0.0008) [2023-12-27 09:48:41,809][20347] Updated weights for policy 0, policy_version 603684 (0.0010) [2023-12-27 09:48:41,865][20347] Updated weights for policy 0, policy_version 603694 (0.0005) [2023-12-27 09:48:41,900][20291] KL-divergence is very high: 273.6188 [2023-12-27 09:48:41,906][20291] KL-divergence is very high: 175.2969 [2023-12-27 09:48:41,918][20291] KL-divergence is very high: 188.1393 [2023-12-27 09:48:41,924][20347] Updated weights for policy 0, policy_version 603704 (0.0005) [2023-12-27 09:48:41,925][20381] Updated weights for policy 1, policy_version 603368 (0.0007) [2023-12-27 09:48:41,949][20291] KL-divergence is very high: 189.1903 [2023-12-27 09:48:41,979][20381] Updated weights for policy 1, policy_version 603378 (0.0008) [2023-12-27 09:48:42,034][20381] Updated weights for policy 1, policy_version 603388 (0.0006) [2023-12-27 09:48:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 309059584. Throughput: 0: 10350.7, 1: 10395.6. Samples: 309049980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:42,363][19170] Avg episode reward: [(0, '941.760'), (1, '912.910')] [2023-12-27 09:48:42,519][20347] Updated weights for policy 0, policy_version 603714 (0.0005) [2023-12-27 09:48:42,579][20347] Updated weights for policy 0, policy_version 603724 (0.0008) [2023-12-27 09:48:42,642][20347] Updated weights for policy 0, policy_version 603734 (0.0008) [2023-12-27 09:48:42,699][20347] Updated weights for policy 0, policy_version 603744 (0.0008) [2023-12-27 09:48:42,713][20381] Updated weights for policy 1, policy_version 603398 (0.0005) [2023-12-27 09:48:42,776][20381] Updated weights for policy 1, policy_version 603408 (0.0006) [2023-12-27 09:48:42,828][20381] Updated weights for policy 1, policy_version 603418 (0.0009) [2023-12-27 09:48:43,404][20347] Updated weights for policy 0, policy_version 603754 (0.0008) [2023-12-27 09:48:43,431][20381] Updated weights for policy 1, policy_version 603428 (0.0008) [2023-12-27 09:48:43,456][20347] Updated weights for policy 0, policy_version 603764 (0.0007) [2023-12-27 09:48:43,488][20381] Updated weights for policy 1, policy_version 603438 (0.0005) [2023-12-27 09:48:43,508][20347] Updated weights for policy 0, policy_version 603774 (0.0009) [2023-12-27 09:48:43,545][20381] Updated weights for policy 1, policy_version 603448 (0.0010) [2023-12-27 09:48:44,140][20381] Updated weights for policy 1, policy_version 603458 (0.0009) [2023-12-27 09:48:44,191][20381] Updated weights for policy 1, policy_version 603468 (0.0007) [2023-12-27 09:48:44,193][20347] Updated weights for policy 0, policy_version 603784 (0.0006) [2023-12-27 09:48:44,247][20381] Updated weights for policy 1, policy_version 603478 (0.0006) [2023-12-27 09:48:44,258][20347] Updated weights for policy 0, policy_version 603794 (0.0006) [2023-12-27 09:48:44,304][20381] Updated weights for policy 1, policy_version 603488 (0.0006) [2023-12-27 09:48:44,320][20347] Updated weights for policy 0, policy_version 603804 (0.0006) [2023-12-27 09:48:44,908][20381] Updated weights for policy 1, policy_version 603498 (0.0006) [2023-12-27 09:48:44,942][20347] Updated weights for policy 0, policy_version 603814 (0.0007) [2023-12-27 09:48:44,967][20381] Updated weights for policy 1, policy_version 603508 (0.0007) [2023-12-27 09:48:44,995][20347] Updated weights for policy 0, policy_version 603824 (0.0006) [2023-12-27 09:48:45,028][20381] Updated weights for policy 1, policy_version 603518 (0.0006) [2023-12-27 09:48:45,055][20347] Updated weights for policy 0, policy_version 603834 (0.0007) [2023-12-27 09:48:45,678][20381] Updated weights for policy 1, policy_version 603528 (0.0007) [2023-12-27 09:48:45,733][20381] Updated weights for policy 1, policy_version 603538 (0.0008) [2023-12-27 09:48:45,763][20347] Updated weights for policy 0, policy_version 603844 (0.0008) [2023-12-27 09:48:45,785][20381] Updated weights for policy 1, policy_version 603548 (0.0008) [2023-12-27 09:48:45,811][20347] Updated weights for policy 0, policy_version 603854 (0.0006) [2023-12-27 09:48:45,884][20347] Updated weights for policy 0, policy_version 603864 (0.0007) [2023-12-27 09:48:46,476][20381] Updated weights for policy 1, policy_version 603558 (0.0007) [2023-12-27 09:48:46,534][20381] Updated weights for policy 1, policy_version 603568 (0.0007) [2023-12-27 09:48:46,558][20347] Updated weights for policy 0, policy_version 603874 (0.0006) [2023-12-27 09:48:46,593][20381] Updated weights for policy 1, policy_version 603578 (0.0007) [2023-12-27 09:48:46,613][20347] Updated weights for policy 0, policy_version 603884 (0.0006) [2023-12-27 09:48:46,664][20347] Updated weights for policy 0, policy_version 603894 (0.0007) [2023-12-27 09:48:46,708][20347] Updated weights for policy 0, policy_version 603904 (0.0007) [2023-12-27 09:48:47,281][20381] Updated weights for policy 1, policy_version 603588 (0.0006) [2023-12-27 09:48:47,327][20381] Updated weights for policy 1, policy_version 603598 (0.0006) [2023-12-27 09:48:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 309157888. Throughput: 0: 10329.6, 1: 10437.2. Samples: 309174332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:47,363][19170] Avg episode reward: [(0, '899.220'), (1, '932.230')] [2023-12-27 09:48:47,379][20381] Updated weights for policy 1, policy_version 603608 (0.0005) [2023-12-27 09:48:47,427][20347] Updated weights for policy 0, policy_version 603914 (0.0006) [2023-12-27 09:48:47,483][20347] Updated weights for policy 0, policy_version 603924 (0.0007) [2023-12-27 09:48:47,538][20347] Updated weights for policy 0, policy_version 603934 (0.0007) [2023-12-27 09:48:47,966][20381] Updated weights for policy 1, policy_version 603618 (0.0005) [2023-12-27 09:48:48,025][20381] Updated weights for policy 1, policy_version 603628 (0.0006) [2023-12-27 09:48:48,077][20381] Updated weights for policy 1, policy_version 603638 (0.0007) [2023-12-27 09:48:48,126][20381] Updated weights for policy 1, policy_version 603648 (0.0008) [2023-12-27 09:48:48,248][20347] Updated weights for policy 0, policy_version 603944 (0.0006) [2023-12-27 09:48:48,307][20347] Updated weights for policy 0, policy_version 603954 (0.0007) [2023-12-27 09:48:48,371][20347] Updated weights for policy 0, policy_version 603964 (0.0008) [2023-12-27 09:48:48,789][20381] Updated weights for policy 1, policy_version 603658 (0.0007) [2023-12-27 09:48:48,838][20381] Updated weights for policy 1, policy_version 603668 (0.0008) [2023-12-27 09:48:48,900][20381] Updated weights for policy 1, policy_version 603678 (0.0008) [2023-12-27 09:48:49,069][20347] Updated weights for policy 0, policy_version 603974 (0.0008) [2023-12-27 09:48:49,123][20347] Updated weights for policy 0, policy_version 603984 (0.0008) [2023-12-27 09:48:49,179][20347] Updated weights for policy 0, policy_version 603994 (0.0008) [2023-12-27 09:48:49,614][20381] Updated weights for policy 1, policy_version 603688 (0.0006) [2023-12-27 09:48:49,659][20381] Updated weights for policy 1, policy_version 603698 (0.0008) [2023-12-27 09:48:49,707][20381] Updated weights for policy 1, policy_version 603708 (0.0008) [2023-12-27 09:48:49,850][20347] Updated weights for policy 0, policy_version 604004 (0.0007) [2023-12-27 09:48:49,913][20347] Updated weights for policy 0, policy_version 604014 (0.0007) [2023-12-27 09:48:49,980][20347] Updated weights for policy 0, policy_version 604024 (0.0007) [2023-12-27 09:48:50,419][20381] Updated weights for policy 1, policy_version 603718 (0.0007) [2023-12-27 09:48:50,478][20381] Updated weights for policy 1, policy_version 603728 (0.0007) [2023-12-27 09:48:50,540][20381] Updated weights for policy 1, policy_version 603738 (0.0007) [2023-12-27 09:48:50,663][20347] Updated weights for policy 0, policy_version 604034 (0.0007) [2023-12-27 09:48:50,728][20347] Updated weights for policy 0, policy_version 604044 (0.0009) [2023-12-27 09:48:50,790][20347] Updated weights for policy 0, policy_version 604054 (0.0010) [2023-12-27 09:48:50,851][20347] Updated weights for policy 0, policy_version 604064 (0.0010) [2023-12-27 09:48:51,146][20381] Updated weights for policy 1, policy_version 603748 (0.0008) [2023-12-27 09:48:51,200][20381] Updated weights for policy 1, policy_version 603758 (0.0005) [2023-12-27 09:48:51,202][20292] KL-divergence is very high: 316.1412 [2023-12-27 09:48:51,262][20381] Updated weights for policy 1, policy_version 603768 (0.0007) [2023-12-27 09:48:51,517][20347] Updated weights for policy 0, policy_version 604074 (0.0010) [2023-12-27 09:48:51,585][20347] Updated weights for policy 0, policy_version 604084 (0.0005) [2023-12-27 09:48:51,650][20347] Updated weights for policy 0, policy_version 604094 (0.0007) [2023-12-27 09:48:51,934][20381] Updated weights for policy 1, policy_version 603778 (0.0009) [2023-12-27 09:48:51,995][20381] Updated weights for policy 1, policy_version 603788 (0.0006) [2023-12-27 09:48:52,051][20381] Updated weights for policy 1, policy_version 603798 (0.0007) [2023-12-27 09:48:52,107][20381] Updated weights for policy 1, policy_version 603808 (0.0007) [2023-12-27 09:48:52,273][20347] Updated weights for policy 0, policy_version 604104 (0.0006) [2023-12-27 09:48:52,334][20347] Updated weights for policy 0, policy_version 604114 (0.0008) [2023-12-27 09:48:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 309264384. Throughput: 0: 10303.8, 1: 10451.3. Samples: 309235752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:52,363][19170] Avg episode reward: [(0, '957.540'), (1, '961.190')] [2023-12-27 09:48:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000603808_154599424.pth... [2023-12-27 09:48:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000602592_154288128.pth [2023-12-27 09:48:52,393][20347] Updated weights for policy 0, policy_version 604124 (0.0010) [2023-12-27 09:48:52,413][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000604128_154673152.pth... [2023-12-27 09:48:52,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000602912_154361856.pth [2023-12-27 09:48:52,804][20381] Updated weights for policy 1, policy_version 603818 (0.0006) [2023-12-27 09:48:52,859][20381] Updated weights for policy 1, policy_version 603828 (0.0006) [2023-12-27 09:48:52,914][20381] Updated weights for policy 1, policy_version 603838 (0.0007) [2023-12-27 09:48:53,011][20347] Updated weights for policy 0, policy_version 604134 (0.0006) [2023-12-27 09:48:53,071][20347] Updated weights for policy 0, policy_version 604144 (0.0006) [2023-12-27 09:48:53,116][20347] Updated weights for policy 0, policy_version 604154 (0.0005) [2023-12-27 09:48:53,535][20381] Updated weights for policy 1, policy_version 603848 (0.0009) [2023-12-27 09:48:53,583][20381] Updated weights for policy 1, policy_version 603858 (0.0010) [2023-12-27 09:48:53,643][20381] Updated weights for policy 1, policy_version 603868 (0.0011) [2023-12-27 09:48:53,787][20347] Updated weights for policy 0, policy_version 604164 (0.0008) [2023-12-27 09:48:53,850][20347] Updated weights for policy 0, policy_version 604174 (0.0011) [2023-12-27 09:48:53,913][20347] Updated weights for policy 0, policy_version 604184 (0.0011) [2023-12-27 09:48:54,337][20381] Updated weights for policy 1, policy_version 603878 (0.0009) [2023-12-27 09:48:54,392][20381] Updated weights for policy 1, policy_version 603888 (0.0008) [2023-12-27 09:48:54,448][20381] Updated weights for policy 1, policy_version 603898 (0.0008) [2023-12-27 09:48:54,571][20347] Updated weights for policy 0, policy_version 604194 (0.0010) [2023-12-27 09:48:54,632][20347] Updated weights for policy 0, policy_version 604204 (0.0008) [2023-12-27 09:48:54,696][20347] Updated weights for policy 0, policy_version 604214 (0.0007) [2023-12-27 09:48:54,761][20347] Updated weights for policy 0, policy_version 604224 (0.0007) [2023-12-27 09:48:55,107][20381] Updated weights for policy 1, policy_version 603908 (0.0007) [2023-12-27 09:48:55,160][20381] Updated weights for policy 1, policy_version 603918 (0.0006) [2023-12-27 09:48:55,214][20381] Updated weights for policy 1, policy_version 603928 (0.0008) [2023-12-27 09:48:55,456][20347] Updated weights for policy 0, policy_version 604234 (0.0008) [2023-12-27 09:48:55,502][20347] Updated weights for policy 0, policy_version 604244 (0.0008) [2023-12-27 09:48:55,545][20347] Updated weights for policy 0, policy_version 604254 (0.0008) [2023-12-27 09:48:55,858][20381] Updated weights for policy 1, policy_version 603938 (0.0005) [2023-12-27 09:48:55,910][20381] Updated weights for policy 1, policy_version 603948 (0.0005) [2023-12-27 09:48:55,960][20381] Updated weights for policy 1, policy_version 603958 (0.0006) [2023-12-27 09:48:56,006][20381] Updated weights for policy 1, policy_version 603968 (0.0005) [2023-12-27 09:48:56,300][20347] Updated weights for policy 0, policy_version 604264 (0.0007) [2023-12-27 09:48:56,359][20347] Updated weights for policy 0, policy_version 604274 (0.0007) [2023-12-27 09:48:56,412][20347] Updated weights for policy 0, policy_version 604284 (0.0007) [2023-12-27 09:48:56,608][20381] Updated weights for policy 1, policy_version 603978 (0.0005) [2023-12-27 09:48:56,662][20381] Updated weights for policy 1, policy_version 603988 (0.0005) [2023-12-27 09:48:56,717][20381] Updated weights for policy 1, policy_version 603998 (0.0006) [2023-12-27 09:48:57,024][20347] Updated weights for policy 0, policy_version 604294 (0.0008) [2023-12-27 09:48:57,075][20347] Updated weights for policy 0, policy_version 604304 (0.0008) [2023-12-27 09:48:57,119][20347] Updated weights for policy 0, policy_version 604314 (0.0007) [2023-12-27 09:48:57,302][20381] Updated weights for policy 1, policy_version 604008 (0.0008) [2023-12-27 09:48:57,356][20381] Updated weights for policy 1, policy_version 604018 (0.0008) [2023-12-27 09:48:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 309370880. Throughput: 0: 10315.9, 1: 10449.2. Samples: 309360660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:48:57,363][19170] Avg episode reward: [(0, '961.180'), (1, '980.640')] [2023-12-27 09:48:57,412][20381] Updated weights for policy 1, policy_version 604028 (0.0008) [2023-12-27 09:48:57,819][20347] Updated weights for policy 0, policy_version 604324 (0.0007) [2023-12-27 09:48:57,870][20347] Updated weights for policy 0, policy_version 604334 (0.0008) [2023-12-27 09:48:57,919][20347] Updated weights for policy 0, policy_version 604344 (0.0008) [2023-12-27 09:48:58,061][20381] Updated weights for policy 1, policy_version 604038 (0.0008) [2023-12-27 09:48:58,117][20381] Updated weights for policy 1, policy_version 604048 (0.0007) [2023-12-27 09:48:58,172][20381] Updated weights for policy 1, policy_version 604058 (0.0007) [2023-12-27 09:48:58,689][20347] Updated weights for policy 0, policy_version 604354 (0.0008) [2023-12-27 09:48:58,750][20347] Updated weights for policy 0, policy_version 604364 (0.0008) [2023-12-27 09:48:58,815][20347] Updated weights for policy 0, policy_version 604374 (0.0008) [2023-12-27 09:48:58,877][20347] Updated weights for policy 0, policy_version 604384 (0.0008) [2023-12-27 09:48:58,884][20381] Updated weights for policy 1, policy_version 604068 (0.0007) [2023-12-27 09:48:58,939][20381] Updated weights for policy 1, policy_version 604078 (0.0007) [2023-12-27 09:48:58,998][20381] Updated weights for policy 1, policy_version 604088 (0.0008) [2023-12-27 09:48:59,533][20347] Updated weights for policy 0, policy_version 604394 (0.0007) [2023-12-27 09:48:59,594][20347] Updated weights for policy 0, policy_version 604405 (0.0010) [2023-12-27 09:48:59,645][20347] Updated weights for policy 0, policy_version 604415 (0.0009) [2023-12-27 09:48:59,726][20381] Updated weights for policy 1, policy_version 604098 (0.0007) [2023-12-27 09:48:59,780][20381] Updated weights for policy 1, policy_version 604108 (0.0008) [2023-12-27 09:48:59,847][20381] Updated weights for policy 1, policy_version 604118 (0.0007) [2023-12-27 09:48:59,910][20381] Updated weights for policy 1, policy_version 604128 (0.0008) [2023-12-27 09:49:00,276][20347] Updated weights for policy 0, policy_version 604425 (0.0007) [2023-12-27 09:49:00,325][20347] Updated weights for policy 0, policy_version 604435 (0.0010) [2023-12-27 09:49:00,384][20347] Updated weights for policy 0, policy_version 604445 (0.0010) [2023-12-27 09:49:00,550][20381] Updated weights for policy 1, policy_version 604138 (0.0005) [2023-12-27 09:49:00,608][20381] Updated weights for policy 1, policy_version 604148 (0.0005) [2023-12-27 09:49:00,663][20381] Updated weights for policy 1, policy_version 604158 (0.0005) [2023-12-27 09:49:01,069][20347] Updated weights for policy 0, policy_version 604455 (0.0009) [2023-12-27 09:49:01,130][20347] Updated weights for policy 0, policy_version 604465 (0.0008) [2023-12-27 09:49:01,189][20347] Updated weights for policy 0, policy_version 604475 (0.0007) [2023-12-27 09:49:01,239][20381] Updated weights for policy 1, policy_version 604168 (0.0007) [2023-12-27 09:49:01,298][20381] Updated weights for policy 1, policy_version 604178 (0.0006) [2023-12-27 09:49:01,355][20381] Updated weights for policy 1, policy_version 604188 (0.0007) [2023-12-27 09:49:01,863][20347] Updated weights for policy 0, policy_version 604485 (0.0007) [2023-12-27 09:49:01,914][20347] Updated weights for policy 0, policy_version 604495 (0.0010) [2023-12-27 09:49:01,978][20347] Updated weights for policy 0, policy_version 604505 (0.0006) [2023-12-27 09:49:01,988][20381] Updated weights for policy 1, policy_version 604198 (0.0007) [2023-12-27 09:49:02,037][20381] Updated weights for policy 1, policy_version 604208 (0.0007) [2023-12-27 09:49:02,094][20381] Updated weights for policy 1, policy_version 604218 (0.0006) [2023-12-27 09:49:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 309477376. Throughput: 0: 10312.6, 1: 10463.0. Samples: 309486416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:49:02,363][19170] Avg episode reward: [(0, '932.380'), (1, '961.220')] [2023-12-27 09:49:02,649][20347] Updated weights for policy 0, policy_version 604515 (0.0008) [2023-12-27 09:49:02,706][20347] Updated weights for policy 0, policy_version 604525 (0.0007) [2023-12-27 09:49:02,730][20381] Updated weights for policy 1, policy_version 604228 (0.0006) [2023-12-27 09:49:02,761][20347] Updated weights for policy 0, policy_version 604535 (0.0007) [2023-12-27 09:49:02,785][20381] Updated weights for policy 1, policy_version 604238 (0.0005) [2023-12-27 09:49:02,837][20381] Updated weights for policy 1, policy_version 604248 (0.0006) [2023-12-27 09:49:03,395][20347] Updated weights for policy 0, policy_version 604545 (0.0006) [2023-12-27 09:49:03,443][20347] Updated weights for policy 0, policy_version 604555 (0.0010) [2023-12-27 09:49:03,492][20347] Updated weights for policy 0, policy_version 604565 (0.0008) [2023-12-27 09:49:03,533][20381] Updated weights for policy 1, policy_version 604258 (0.0007) [2023-12-27 09:49:03,547][20347] Updated weights for policy 0, policy_version 604575 (0.0010) [2023-12-27 09:49:03,584][20381] Updated weights for policy 1, policy_version 604268 (0.0005) [2023-12-27 09:49:03,638][20381] Updated weights for policy 1, policy_version 604278 (0.0005) [2023-12-27 09:49:03,694][20381] Updated weights for policy 1, policy_version 604288 (0.0005) [2023-12-27 09:49:04,240][20347] Updated weights for policy 0, policy_version 604585 (0.0006) [2023-12-27 09:49:04,281][20381] Updated weights for policy 1, policy_version 604298 (0.0008) [2023-12-27 09:49:04,298][20347] Updated weights for policy 0, policy_version 604595 (0.0007) [2023-12-27 09:49:04,348][20381] Updated weights for policy 1, policy_version 604308 (0.0008) [2023-12-27 09:49:04,350][20347] Updated weights for policy 0, policy_version 604605 (0.0008) [2023-12-27 09:49:04,410][20381] Updated weights for policy 1, policy_version 604318 (0.0008) [2023-12-27 09:49:05,049][20381] Updated weights for policy 1, policy_version 604328 (0.0007) [2023-12-27 09:49:05,055][20347] Updated weights for policy 0, policy_version 604615 (0.0006) [2023-12-27 09:49:05,104][20381] Updated weights for policy 1, policy_version 604338 (0.0005) [2023-12-27 09:49:05,113][20347] Updated weights for policy 0, policy_version 604625 (0.0007) [2023-12-27 09:49:05,153][20381] Updated weights for policy 1, policy_version 604348 (0.0008) [2023-12-27 09:49:05,175][20347] Updated weights for policy 0, policy_version 604635 (0.0007) [2023-12-27 09:49:05,774][20347] Updated weights for policy 0, policy_version 604645 (0.0010) [2023-12-27 09:49:05,829][20347] Updated weights for policy 0, policy_version 604655 (0.0010) [2023-12-27 09:49:05,887][20347] Updated weights for policy 0, policy_version 604665 (0.0010) [2023-12-27 09:49:05,897][20381] Updated weights for policy 1, policy_version 604358 (0.0006) [2023-12-27 09:49:05,949][20381] Updated weights for policy 1, policy_version 604368 (0.0006) [2023-12-27 09:49:05,997][20381] Updated weights for policy 1, policy_version 604378 (0.0008) [2023-12-27 09:49:06,592][20347] Updated weights for policy 0, policy_version 604675 (0.0010) [2023-12-27 09:49:06,658][20347] Updated weights for policy 0, policy_version 604685 (0.0010) [2023-12-27 09:49:06,713][20347] Updated weights for policy 0, policy_version 604695 (0.0010) [2023-12-27 09:49:06,730][20381] Updated weights for policy 1, policy_version 604388 (0.0006) [2023-12-27 09:49:06,786][20381] Updated weights for policy 1, policy_version 604398 (0.0007) [2023-12-27 09:49:06,834][20381] Updated weights for policy 1, policy_version 604408 (0.0007) [2023-12-27 09:49:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 309575680. Throughput: 0: 10290.7, 1: 10475.3. Samples: 309549336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:49:07,363][19170] Avg episode reward: [(0, '922.820'), (1, '932.080')] [2023-12-27 09:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000604416_154755072.pth... [2023-12-27 09:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000604704_154820608.pth... [2023-12-27 09:49:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000603520_154517504.pth [2023-12-27 09:49:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000603200_154443776.pth [2023-12-27 09:49:07,393][20347] Updated weights for policy 0, policy_version 604705 (0.0010) [2023-12-27 09:49:07,439][20347] Updated weights for policy 0, policy_version 604715 (0.0010) [2023-12-27 09:49:07,496][20347] Updated weights for policy 0, policy_version 604725 (0.0010) [2023-12-27 09:49:07,534][20381] Updated weights for policy 1, policy_version 604418 (0.0007) [2023-12-27 09:49:07,548][20347] Updated weights for policy 0, policy_version 604735 (0.0010) [2023-12-27 09:49:07,587][20381] Updated weights for policy 1, policy_version 604428 (0.0007) [2023-12-27 09:49:07,641][20381] Updated weights for policy 1, policy_version 604438 (0.0008) [2023-12-27 09:49:07,695][20381] Updated weights for policy 1, policy_version 604448 (0.0007) [2023-12-27 09:49:08,245][20347] Updated weights for policy 0, policy_version 604745 (0.0010) [2023-12-27 09:49:08,297][20347] Updated weights for policy 0, policy_version 604755 (0.0010) [2023-12-27 09:49:08,360][20347] Updated weights for policy 0, policy_version 604765 (0.0008) [2023-12-27 09:49:08,404][20381] Updated weights for policy 1, policy_version 604458 (0.0008) [2023-12-27 09:49:08,467][20381] Updated weights for policy 1, policy_version 604468 (0.0008) [2023-12-27 09:49:08,528][20381] Updated weights for policy 1, policy_version 604478 (0.0008) [2023-12-27 09:49:09,010][20347] Updated weights for policy 0, policy_version 604775 (0.0007) [2023-12-27 09:49:09,070][20347] Updated weights for policy 0, policy_version 604785 (0.0008) [2023-12-27 09:49:09,123][20347] Updated weights for policy 0, policy_version 604795 (0.0008) [2023-12-27 09:49:09,217][20381] Updated weights for policy 1, policy_version 604488 (0.0008) [2023-12-27 09:49:09,280][20381] Updated weights for policy 1, policy_version 604498 (0.0006) [2023-12-27 09:49:09,341][20381] Updated weights for policy 1, policy_version 604508 (0.0007) [2023-12-27 09:49:09,860][20347] Updated weights for policy 0, policy_version 604805 (0.0007) [2023-12-27 09:49:09,921][20347] Updated weights for policy 0, policy_version 604815 (0.0008) [2023-12-27 09:49:09,985][20347] Updated weights for policy 0, policy_version 604825 (0.0008) [2023-12-27 09:49:10,036][20381] Updated weights for policy 1, policy_version 604518 (0.0006) [2023-12-27 09:49:10,099][20381] Updated weights for policy 1, policy_version 604528 (0.0008) [2023-12-27 09:49:10,159][20381] Updated weights for policy 1, policy_version 604538 (0.0008) [2023-12-27 09:49:10,664][20347] Updated weights for policy 0, policy_version 604835 (0.0008) [2023-12-27 09:49:10,725][20347] Updated weights for policy 0, policy_version 604845 (0.0005) [2023-12-27 09:49:10,789][20347] Updated weights for policy 0, policy_version 604855 (0.0005) [2023-12-27 09:49:10,815][20381] Updated weights for policy 1, policy_version 604548 (0.0008) [2023-12-27 09:49:10,868][20381] Updated weights for policy 1, policy_version 604558 (0.0006) [2023-12-27 09:49:10,920][20381] Updated weights for policy 1, policy_version 604568 (0.0007) [2023-12-27 09:49:11,438][20347] Updated weights for policy 0, policy_version 604865 (0.0007) [2023-12-27 09:49:11,494][20347] Updated weights for policy 0, policy_version 604875 (0.0007) [2023-12-27 09:49:11,550][20347] Updated weights for policy 0, policy_version 604885 (0.0008) [2023-12-27 09:49:11,608][20347] Updated weights for policy 0, policy_version 604895 (0.0008) [2023-12-27 09:49:11,644][20381] Updated weights for policy 1, policy_version 604578 (0.0009) [2023-12-27 09:49:11,705][20381] Updated weights for policy 1, policy_version 604588 (0.0011) [2023-12-27 09:49:11,774][20381] Updated weights for policy 1, policy_version 604598 (0.0010) [2023-12-27 09:49:11,833][20381] Updated weights for policy 1, policy_version 604608 (0.0007) [2023-12-27 09:49:12,306][20347] Updated weights for policy 0, policy_version 604905 (0.0006) [2023-12-27 09:49:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 309673984. Throughput: 0: 10307.5, 1: 10408.8. Samples: 309670292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:49:12,363][19170] Avg episode reward: [(0, '961.260'), (1, '980.580')] [2023-12-27 09:49:12,365][20347] Updated weights for policy 0, policy_version 604915 (0.0008) [2023-12-27 09:49:12,427][20347] Updated weights for policy 0, policy_version 604925 (0.0008) [2023-12-27 09:49:12,543][20381] Updated weights for policy 1, policy_version 604618 (0.0010) [2023-12-27 09:49:12,598][20381] Updated weights for policy 1, policy_version 604628 (0.0010) [2023-12-27 09:49:12,656][20381] Updated weights for policy 1, policy_version 604638 (0.0010) [2023-12-27 09:49:13,098][20347] Updated weights for policy 0, policy_version 604935 (0.0009) [2023-12-27 09:49:13,161][20347] Updated weights for policy 0, policy_version 604945 (0.0010) [2023-12-27 09:49:13,224][20347] Updated weights for policy 0, policy_version 604955 (0.0007) [2023-12-27 09:49:13,313][20381] Updated weights for policy 1, policy_version 604648 (0.0009) [2023-12-27 09:49:13,360][20381] Updated weights for policy 1, policy_version 604658 (0.0006) [2023-12-27 09:49:13,416][20381] Updated weights for policy 1, policy_version 604668 (0.0005) [2023-12-27 09:49:13,817][20347] Updated weights for policy 0, policy_version 604965 (0.0005) [2023-12-27 09:49:13,863][20347] Updated weights for policy 0, policy_version 604975 (0.0008) [2023-12-27 09:49:13,907][20347] Updated weights for policy 0, policy_version 604985 (0.0010) [2023-12-27 09:49:14,007][20381] Updated weights for policy 1, policy_version 604678 (0.0006) [2023-12-27 09:49:14,069][20381] Updated weights for policy 1, policy_version 604688 (0.0006) [2023-12-27 09:49:14,127][20381] Updated weights for policy 1, policy_version 604698 (0.0008) [2023-12-27 09:49:14,576][20347] Updated weights for policy 0, policy_version 604995 (0.0009) [2023-12-27 09:49:14,629][20347] Updated weights for policy 0, policy_version 605005 (0.0007) [2023-12-27 09:49:14,683][20347] Updated weights for policy 0, policy_version 605015 (0.0010) [2023-12-27 09:49:14,727][20381] Updated weights for policy 1, policy_version 604708 (0.0008) [2023-12-27 09:49:14,783][20381] Updated weights for policy 1, policy_version 604718 (0.0011) [2023-12-27 09:49:14,839][20381] Updated weights for policy 1, policy_version 604728 (0.0011) [2023-12-27 09:49:15,368][20347] Updated weights for policy 0, policy_version 605025 (0.0010) [2023-12-27 09:49:15,434][20347] Updated weights for policy 0, policy_version 605035 (0.0010) [2023-12-27 09:49:15,491][20347] Updated weights for policy 0, policy_version 605045 (0.0009) [2023-12-27 09:49:15,512][20381] Updated weights for policy 1, policy_version 604738 (0.0011) [2023-12-27 09:49:15,555][20347] Updated weights for policy 0, policy_version 605055 (0.0010) [2023-12-27 09:49:15,568][20381] Updated weights for policy 1, policy_version 604748 (0.0011) [2023-12-27 09:49:15,629][20381] Updated weights for policy 1, policy_version 604758 (0.0010) [2023-12-27 09:49:15,689][20381] Updated weights for policy 1, policy_version 604768 (0.0010) [2023-12-27 09:49:16,228][20347] Updated weights for policy 0, policy_version 605065 (0.0010) [2023-12-27 09:49:16,286][20347] Updated weights for policy 0, policy_version 605075 (0.0008) [2023-12-27 09:49:16,333][20381] Updated weights for policy 1, policy_version 604778 (0.0011) [2023-12-27 09:49:16,342][20347] Updated weights for policy 0, policy_version 605085 (0.0005) [2023-12-27 09:49:16,382][20381] Updated weights for policy 1, policy_version 604788 (0.0010) [2023-12-27 09:49:16,430][20381] Updated weights for policy 1, policy_version 604798 (0.0010) [2023-12-27 09:49:16,974][20347] Updated weights for policy 0, policy_version 605095 (0.0006) [2023-12-27 09:49:17,021][20347] Updated weights for policy 0, policy_version 605105 (0.0010) [2023-12-27 09:49:17,070][20347] Updated weights for policy 0, policy_version 605115 (0.0010) [2023-12-27 09:49:17,126][20381] Updated weights for policy 1, policy_version 604808 (0.0011) [2023-12-27 09:49:17,187][20381] Updated weights for policy 1, policy_version 604818 (0.0011) [2023-12-27 09:49:17,245][20381] Updated weights for policy 1, policy_version 604828 (0.0010) [2023-12-27 09:49:17,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 309788672. Throughput: 0: 10300.3, 1: 10380.1. Samples: 309795312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 09:49:17,363][19170] Avg episode reward: [(0, '990.320'), (1, '951.640')] [2023-12-27 09:49:17,732][20347] Updated weights for policy 0, policy_version 605125 (0.0010) [2023-12-27 09:49:17,786][20347] Updated weights for policy 0, policy_version 605135 (0.0010) [2023-12-27 09:49:17,836][20347] Updated weights for policy 0, policy_version 605145 (0.0010) [2023-12-27 09:49:17,933][20381] Updated weights for policy 1, policy_version 604838 (0.0010) [2023-12-27 09:49:17,990][20381] Updated weights for policy 1, policy_version 604848 (0.0005) [2023-12-27 09:49:18,046][20381] Updated weights for policy 1, policy_version 604858 (0.0006) [2023-12-27 09:49:18,520][20347] Updated weights for policy 0, policy_version 605155 (0.0009) [2023-12-27 09:49:18,578][20347] Updated weights for policy 0, policy_version 605165 (0.0009) [2023-12-27 09:49:18,636][20347] Updated weights for policy 0, policy_version 605175 (0.0010) [2023-12-27 09:49:18,644][20381] Updated weights for policy 1, policy_version 604868 (0.0008) [2023-12-27 09:49:18,697][20381] Updated weights for policy 1, policy_version 604878 (0.0010) [2023-12-27 09:49:18,749][20381] Updated weights for policy 1, policy_version 604888 (0.0011) [2023-12-27 09:49:19,257][20347] Updated weights for policy 0, policy_version 605185 (0.0010) [2023-12-27 09:49:19,309][20347] Updated weights for policy 0, policy_version 605195 (0.0007) [2023-12-27 09:49:19,356][20381] Updated weights for policy 1, policy_version 604898 (0.0010) [2023-12-27 09:49:19,368][20347] Updated weights for policy 0, policy_version 605205 (0.0008) [2023-12-27 09:49:19,417][20381] Updated weights for policy 1, policy_version 604908 (0.0010) [2023-12-27 09:49:19,429][20347] Updated weights for policy 0, policy_version 605215 (0.0008) [2023-12-27 09:49:19,474][20381] Updated weights for policy 1, policy_version 604918 (0.0011) [2023-12-27 09:49:19,533][20381] Updated weights for policy 1, policy_version 604928 (0.0007) [2023-12-27 09:49:20,150][20347] Updated weights for policy 0, policy_version 605225 (0.0007) [2023-12-27 09:49:20,208][20347] Updated weights for policy 0, policy_version 605235 (0.0008) [2023-12-27 09:49:20,245][20381] Updated weights for policy 1, policy_version 604938 (0.0008) [2023-12-27 09:49:20,255][20347] Updated weights for policy 0, policy_version 605245 (0.0008) [2023-12-27 09:49:20,308][20381] Updated weights for policy 1, policy_version 604948 (0.0008) [2023-12-27 09:49:20,368][20381] Updated weights for policy 1, policy_version 604958 (0.0008) [2023-12-27 09:49:20,989][20347] Updated weights for policy 0, policy_version 605255 (0.0010) [2023-12-27 09:49:21,036][20381] Updated weights for policy 1, policy_version 604968 (0.0007) [2023-12-27 09:49:21,053][20347] Updated weights for policy 0, policy_version 605265 (0.0008) [2023-12-27 09:49:21,101][20381] Updated weights for policy 1, policy_version 604978 (0.0007) [2023-12-27 09:49:21,121][20347] Updated weights for policy 0, policy_version 605275 (0.0007) [2023-12-27 09:49:21,166][20381] Updated weights for policy 1, policy_version 604988 (0.0007) [2023-12-27 09:49:21,823][20347] Updated weights for policy 0, policy_version 605285 (0.0009) [2023-12-27 09:49:21,835][20381] Updated weights for policy 1, policy_version 604998 (0.0006) [2023-12-27 09:49:21,882][20347] Updated weights for policy 0, policy_version 605295 (0.0010) [2023-12-27 09:49:21,893][20381] Updated weights for policy 1, policy_version 605008 (0.0006) [2023-12-27 09:49:21,941][20347] Updated weights for policy 0, policy_version 605305 (0.0010) [2023-12-27 09:49:21,943][20381] Updated weights for policy 1, policy_version 605018 (0.0007) [2023-12-27 09:49:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 309886976. Throughput: 0: 10272.2, 1: 10433.7. Samples: 309858236. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:22,363][19170] Avg episode reward: [(0, '971.020'), (1, '922.730')] [2023-12-27 09:49:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000605024_154910720.pth... [2023-12-27 09:49:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000605312_154976256.pth... [2023-12-27 09:49:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000604128_154673152.pth [2023-12-27 09:49:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000603808_154599424.pth [2023-12-27 09:49:22,610][20381] Updated weights for policy 1, policy_version 605028 (0.0007) [2023-12-27 09:49:22,652][20347] Updated weights for policy 0, policy_version 605315 (0.0011) [2023-12-27 09:49:22,670][20381] Updated weights for policy 1, policy_version 605038 (0.0009) [2023-12-27 09:49:22,708][20347] Updated weights for policy 0, policy_version 605325 (0.0010) [2023-12-27 09:49:22,727][20381] Updated weights for policy 1, policy_version 605048 (0.0009) [2023-12-27 09:49:22,760][20347] Updated weights for policy 0, policy_version 605335 (0.0010) [2023-12-27 09:49:23,405][20381] Updated weights for policy 1, policy_version 605058 (0.0006) [2023-12-27 09:49:23,425][20347] Updated weights for policy 0, policy_version 605345 (0.0010) [2023-12-27 09:49:23,461][20381] Updated weights for policy 1, policy_version 605068 (0.0007) [2023-12-27 09:49:23,474][20347] Updated weights for policy 0, policy_version 605355 (0.0006) [2023-12-27 09:49:23,518][20381] Updated weights for policy 1, policy_version 605078 (0.0008) [2023-12-27 09:49:23,520][20347] Updated weights for policy 0, policy_version 605365 (0.0007) [2023-12-27 09:49:23,576][20381] Updated weights for policy 1, policy_version 605088 (0.0007) [2023-12-27 09:49:23,579][20347] Updated weights for policy 0, policy_version 605375 (0.0006) [2023-12-27 09:49:24,196][20347] Updated weights for policy 0, policy_version 605385 (0.0008) [2023-12-27 09:49:24,251][20347] Updated weights for policy 0, policy_version 605395 (0.0007) [2023-12-27 09:49:24,300][20381] Updated weights for policy 1, policy_version 605098 (0.0005) [2023-12-27 09:49:24,313][20347] Updated weights for policy 0, policy_version 605405 (0.0008) [2023-12-27 09:49:24,356][20381] Updated weights for policy 1, policy_version 605108 (0.0006) [2023-12-27 09:49:24,404][20381] Updated weights for policy 1, policy_version 605118 (0.0005) [2023-12-27 09:49:24,960][20347] Updated weights for policy 0, policy_version 605415 (0.0006) [2023-12-27 09:49:25,016][20347] Updated weights for policy 0, policy_version 605425 (0.0006) [2023-12-27 09:49:25,066][20347] Updated weights for policy 0, policy_version 605435 (0.0005) [2023-12-27 09:49:25,098][20381] Updated weights for policy 1, policy_version 605128 (0.0008) [2023-12-27 09:49:25,163][20381] Updated weights for policy 1, policy_version 605138 (0.0008) [2023-12-27 09:49:25,228][20381] Updated weights for policy 1, policy_version 605148 (0.0007) [2023-12-27 09:49:25,648][20347] Updated weights for policy 0, policy_version 605445 (0.0006) [2023-12-27 09:49:25,700][20347] Updated weights for policy 0, policy_version 605455 (0.0008) [2023-12-27 09:49:25,745][20347] Updated weights for policy 0, policy_version 605465 (0.0008) [2023-12-27 09:49:25,920][20381] Updated weights for policy 1, policy_version 605158 (0.0007) [2023-12-27 09:49:25,976][20381] Updated weights for policy 1, policy_version 605168 (0.0005) [2023-12-27 09:49:26,030][20381] Updated weights for policy 1, policy_version 605178 (0.0005) [2023-12-27 09:49:26,506][20347] Updated weights for policy 0, policy_version 605475 (0.0007) [2023-12-27 09:49:26,562][20347] Updated weights for policy 0, policy_version 605485 (0.0008) [2023-12-27 09:49:26,616][20347] Updated weights for policy 0, policy_version 605495 (0.0007) [2023-12-27 09:49:26,633][20381] Updated weights for policy 1, policy_version 605188 (0.0006) [2023-12-27 09:49:26,681][20381] Updated weights for policy 1, policy_version 605198 (0.0006) [2023-12-27 09:49:26,729][20381] Updated weights for policy 1, policy_version 605208 (0.0007) [2023-12-27 09:49:27,273][20347] Updated weights for policy 0, policy_version 605505 (0.0006) [2023-12-27 09:49:27,324][20347] Updated weights for policy 0, policy_version 605515 (0.0010) [2023-12-27 09:49:27,349][20381] Updated weights for policy 1, policy_version 605218 (0.0007) [2023-12-27 09:49:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20743.9). Total num frames: 309985280. Throughput: 0: 10251.9, 1: 10425.6. Samples: 309980468. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:27,363][19170] Avg episode reward: [(0, '932.880'), (1, '913.320')] [2023-12-27 09:49:27,376][20347] Updated weights for policy 0, policy_version 605525 (0.0010) [2023-12-27 09:49:27,401][20381] Updated weights for policy 1, policy_version 605228 (0.0006) [2023-12-27 09:49:27,429][20347] Updated weights for policy 0, policy_version 605535 (0.0009) [2023-12-27 09:49:27,451][20381] Updated weights for policy 1, policy_version 605238 (0.0008) [2023-12-27 09:49:27,502][20381] Updated weights for policy 1, policy_version 605248 (0.0007) [2023-12-27 09:49:28,015][20347] Updated weights for policy 0, policy_version 605545 (0.0005) [2023-12-27 09:49:28,065][20347] Updated weights for policy 0, policy_version 605555 (0.0005) [2023-12-27 09:49:28,122][20347] Updated weights for policy 0, policy_version 605565 (0.0005) [2023-12-27 09:49:28,211][20381] Updated weights for policy 1, policy_version 605258 (0.0007) [2023-12-27 09:49:28,258][20381] Updated weights for policy 1, policy_version 605268 (0.0007) [2023-12-27 09:49:28,305][20381] Updated weights for policy 1, policy_version 605278 (0.0007) [2023-12-27 09:49:28,748][20347] Updated weights for policy 0, policy_version 605575 (0.0009) [2023-12-27 09:49:28,793][20347] Updated weights for policy 0, policy_version 605585 (0.0006) [2023-12-27 09:49:28,848][20347] Updated weights for policy 0, policy_version 605595 (0.0007) [2023-12-27 09:49:29,022][20381] Updated weights for policy 1, policy_version 605288 (0.0007) [2023-12-27 09:49:29,076][20381] Updated weights for policy 1, policy_version 605298 (0.0007) [2023-12-27 09:49:29,131][20381] Updated weights for policy 1, policy_version 605308 (0.0008) [2023-12-27 09:49:29,525][20347] Updated weights for policy 0, policy_version 605605 (0.0010) [2023-12-27 09:49:29,571][20347] Updated weights for policy 0, policy_version 605615 (0.0010) [2023-12-27 09:49:29,622][20347] Updated weights for policy 0, policy_version 605625 (0.0010) [2023-12-27 09:49:29,814][20381] Updated weights for policy 1, policy_version 605318 (0.0008) [2023-12-27 09:49:29,875][20381] Updated weights for policy 1, policy_version 605328 (0.0007) [2023-12-27 09:49:29,936][20381] Updated weights for policy 1, policy_version 605338 (0.0008) [2023-12-27 09:49:30,342][20347] Updated weights for policy 0, policy_version 605635 (0.0010) [2023-12-27 09:49:30,405][20347] Updated weights for policy 0, policy_version 605645 (0.0010) [2023-12-27 09:49:30,453][20347] Updated weights for policy 0, policy_version 605655 (0.0010) [2023-12-27 09:49:30,600][20381] Updated weights for policy 1, policy_version 605348 (0.0008) [2023-12-27 09:49:30,648][20381] Updated weights for policy 1, policy_version 605358 (0.0006) [2023-12-27 09:49:30,704][20381] Updated weights for policy 1, policy_version 605368 (0.0007) [2023-12-27 09:49:31,076][20347] Updated weights for policy 0, policy_version 605665 (0.0010) [2023-12-27 09:49:31,134][20347] Updated weights for policy 0, policy_version 605675 (0.0007) [2023-12-27 09:49:31,192][20347] Updated weights for policy 0, policy_version 605685 (0.0006) [2023-12-27 09:49:31,248][20347] Updated weights for policy 0, policy_version 605695 (0.0007) [2023-12-27 09:49:31,440][20381] Updated weights for policy 1, policy_version 605378 (0.0007) [2023-12-27 09:49:31,486][20381] Updated weights for policy 1, policy_version 605388 (0.0006) [2023-12-27 09:49:31,537][20381] Updated weights for policy 1, policy_version 605398 (0.0005) [2023-12-27 09:49:31,595][20381] Updated weights for policy 1, policy_version 605408 (0.0007) [2023-12-27 09:49:32,010][20347] Updated weights for policy 0, policy_version 605705 (0.0006) [2023-12-27 09:49:32,062][20347] Updated weights for policy 0, policy_version 605715 (0.0008) [2023-12-27 09:49:32,111][20347] Updated weights for policy 0, policy_version 605725 (0.0007) [2023-12-27 09:49:32,276][20381] Updated weights for policy 1, policy_version 605418 (0.0008) [2023-12-27 09:49:32,323][20381] Updated weights for policy 1, policy_version 605428 (0.0007) [2023-12-27 09:49:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 310091776. Throughput: 0: 10288.0, 1: 10391.9. Samples: 310104928. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:32,363][19170] Avg episode reward: [(0, '952.030'), (1, '971.330')] [2023-12-27 09:49:32,379][20381] Updated weights for policy 1, policy_version 605438 (0.0008) [2023-12-27 09:49:32,743][20347] Updated weights for policy 0, policy_version 605735 (0.0008) [2023-12-27 09:49:32,802][20347] Updated weights for policy 0, policy_version 605745 (0.0008) [2023-12-27 09:49:32,857][20347] Updated weights for policy 0, policy_version 605755 (0.0008) [2023-12-27 09:49:33,112][20381] Updated weights for policy 1, policy_version 605448 (0.0010) [2023-12-27 09:49:33,173][20381] Updated weights for policy 1, policy_version 605458 (0.0010) [2023-12-27 09:49:33,237][20381] Updated weights for policy 1, policy_version 605468 (0.0009) [2023-12-27 09:49:33,494][20347] Updated weights for policy 0, policy_version 605765 (0.0008) [2023-12-27 09:49:33,552][20347] Updated weights for policy 0, policy_version 605775 (0.0007) [2023-12-27 09:49:33,595][20347] Updated weights for policy 0, policy_version 605785 (0.0008) [2023-12-27 09:49:33,820][20381] Updated weights for policy 1, policy_version 605478 (0.0007) [2023-12-27 09:49:33,882][20381] Updated weights for policy 1, policy_version 605488 (0.0005) [2023-12-27 09:49:33,940][20381] Updated weights for policy 1, policy_version 605498 (0.0007) [2023-12-27 09:49:34,342][20347] Updated weights for policy 0, policy_version 605795 (0.0008) [2023-12-27 09:49:34,408][20347] Updated weights for policy 0, policy_version 605805 (0.0008) [2023-12-27 09:49:34,472][20347] Updated weights for policy 0, policy_version 605815 (0.0007) [2023-12-27 09:49:34,557][20381] Updated weights for policy 1, policy_version 605508 (0.0008) [2023-12-27 09:49:34,621][20381] Updated weights for policy 1, policy_version 605518 (0.0006) [2023-12-27 09:49:34,683][20381] Updated weights for policy 1, policy_version 605528 (0.0005) [2023-12-27 09:49:35,135][20347] Updated weights for policy 0, policy_version 605825 (0.0008) [2023-12-27 09:49:35,180][20347] Updated weights for policy 0, policy_version 605835 (0.0007) [2023-12-27 09:49:35,236][20347] Updated weights for policy 0, policy_version 605845 (0.0005) [2023-12-27 09:49:35,238][20381] Updated weights for policy 1, policy_version 605538 (0.0005) [2023-12-27 09:49:35,285][20347] Updated weights for policy 0, policy_version 605855 (0.0005) [2023-12-27 09:49:35,294][20381] Updated weights for policy 1, policy_version 605548 (0.0005) [2023-12-27 09:49:35,298][20292] KL-divergence is very high: 149.5524 [2023-12-27 09:49:35,302][20292] KL-divergence is very high: 144.0120 [2023-12-27 09:49:35,347][20381] Updated weights for policy 1, policy_version 605558 (0.0005) [2023-12-27 09:49:35,389][20292] KL-divergence is very high: 188.7825 [2023-12-27 09:49:35,395][20292] KL-divergence is very high: 181.0582 [2023-12-27 09:49:35,407][20381] Updated weights for policy 1, policy_version 605568 (0.0005) [2023-12-27 09:49:35,914][20347] Updated weights for policy 0, policy_version 605865 (0.0005) [2023-12-27 09:49:35,969][20381] Updated weights for policy 1, policy_version 605578 (0.0008) [2023-12-27 09:49:35,976][20347] Updated weights for policy 0, policy_version 605875 (0.0007) [2023-12-27 09:49:36,015][20381] Updated weights for policy 1, policy_version 605588 (0.0007) [2023-12-27 09:49:36,032][20347] Updated weights for policy 0, policy_version 605885 (0.0006) [2023-12-27 09:49:36,061][20381] Updated weights for policy 1, policy_version 605598 (0.0010) [2023-12-27 09:49:36,639][20347] Updated weights for policy 0, policy_version 605895 (0.0007) [2023-12-27 09:49:36,704][20347] Updated weights for policy 0, policy_version 605905 (0.0008) [2023-12-27 09:49:36,767][20347] Updated weights for policy 0, policy_version 605915 (0.0008) [2023-12-27 09:49:36,805][20381] Updated weights for policy 1, policy_version 605608 (0.0011) [2023-12-27 09:49:36,862][20381] Updated weights for policy 1, policy_version 605618 (0.0011) [2023-12-27 09:49:36,922][20381] Updated weights for policy 1, policy_version 605628 (0.0011) [2023-12-27 09:49:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 310198272. Throughput: 0: 10310.9, 1: 10420.3. Samples: 310168656. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:37,363][19170] Avg episode reward: [(0, '961.220'), (1, '922.550')] [2023-12-27 09:49:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000605632_155066368.pth... [2023-12-27 09:49:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000605920_155131904.pth... [2023-12-27 09:49:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000604704_154820608.pth [2023-12-27 09:49:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000604416_154755072.pth [2023-12-27 09:49:37,468][20347] Updated weights for policy 0, policy_version 605925 (0.0007) [2023-12-27 09:49:37,534][20347] Updated weights for policy 0, policy_version 605935 (0.0008) [2023-12-27 09:49:37,580][20381] Updated weights for policy 1, policy_version 605638 (0.0011) [2023-12-27 09:49:37,586][20347] Updated weights for policy 0, policy_version 605945 (0.0007) [2023-12-27 09:49:37,630][20381] Updated weights for policy 1, policy_version 605648 (0.0011) [2023-12-27 09:49:37,686][20381] Updated weights for policy 1, policy_version 605658 (0.0010) [2023-12-27 09:49:38,292][20347] Updated weights for policy 0, policy_version 605955 (0.0006) [2023-12-27 09:49:38,346][20347] Updated weights for policy 0, policy_version 605965 (0.0007) [2023-12-27 09:49:38,384][20381] Updated weights for policy 1, policy_version 605668 (0.0011) [2023-12-27 09:49:38,412][20347] Updated weights for policy 0, policy_version 605975 (0.0008) [2023-12-27 09:49:38,441][20381] Updated weights for policy 1, policy_version 605678 (0.0009) [2023-12-27 09:49:38,499][20381] Updated weights for policy 1, policy_version 605688 (0.0010) [2023-12-27 09:49:39,148][20347] Updated weights for policy 0, policy_version 605985 (0.0006) [2023-12-27 09:49:39,202][20381] Updated weights for policy 1, policy_version 605698 (0.0010) [2023-12-27 09:49:39,211][20347] Updated weights for policy 0, policy_version 605995 (0.0007) [2023-12-27 09:49:39,268][20381] Updated weights for policy 1, policy_version 605708 (0.0008) [2023-12-27 09:49:39,271][20347] Updated weights for policy 0, policy_version 606005 (0.0006) [2023-12-27 09:49:39,320][20347] Updated weights for policy 0, policy_version 606015 (0.0006) [2023-12-27 09:49:39,322][20381] Updated weights for policy 1, policy_version 605718 (0.0010) [2023-12-27 09:49:39,390][20381] Updated weights for policy 1, policy_version 605728 (0.0011) [2023-12-27 09:49:39,985][20347] Updated weights for policy 0, policy_version 606025 (0.0006) [2023-12-27 09:49:40,053][20347] Updated weights for policy 0, policy_version 606035 (0.0006) [2023-12-27 09:49:40,116][20347] Updated weights for policy 0, policy_version 606045 (0.0006) [2023-12-27 09:49:40,118][20381] Updated weights for policy 1, policy_version 605738 (0.0007) [2023-12-27 09:49:40,179][20381] Updated weights for policy 1, policy_version 605748 (0.0007) [2023-12-27 09:49:40,237][20381] Updated weights for policy 1, policy_version 605758 (0.0005) [2023-12-27 09:49:40,767][20347] Updated weights for policy 0, policy_version 606055 (0.0008) [2023-12-27 09:49:40,822][20347] Updated weights for policy 0, policy_version 606065 (0.0008) [2023-12-27 09:49:40,847][20381] Updated weights for policy 1, policy_version 605768 (0.0005) [2023-12-27 09:49:40,884][20347] Updated weights for policy 0, policy_version 606075 (0.0006) [2023-12-27 09:49:40,901][20381] Updated weights for policy 1, policy_version 605778 (0.0006) [2023-12-27 09:49:40,950][20381] Updated weights for policy 1, policy_version 605788 (0.0008) [2023-12-27 09:49:41,614][20347] Updated weights for policy 0, policy_version 606085 (0.0007) [2023-12-27 09:49:41,627][20381] Updated weights for policy 1, policy_version 605798 (0.0007) [2023-12-27 09:49:41,681][20347] Updated weights for policy 0, policy_version 606095 (0.0008) [2023-12-27 09:49:41,686][20381] Updated weights for policy 1, policy_version 605808 (0.0008) [2023-12-27 09:49:41,744][20347] Updated weights for policy 0, policy_version 606105 (0.0008) [2023-12-27 09:49:41,750][20381] Updated weights for policy 1, policy_version 605818 (0.0007) [2023-12-27 09:49:42,351][20381] Updated weights for policy 1, policy_version 605828 (0.0006) [2023-12-27 09:49:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 310296576. Throughput: 0: 10303.7, 1: 10383.8. Samples: 310291600. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:42,363][19170] Avg episode reward: [(0, '941.960'), (1, '912.990')] [2023-12-27 09:49:42,412][20381] Updated weights for policy 1, policy_version 605838 (0.0007) [2023-12-27 09:49:42,474][20381] Updated weights for policy 1, policy_version 605848 (0.0006) [2023-12-27 09:49:42,475][20347] Updated weights for policy 0, policy_version 606115 (0.0008) [2023-12-27 09:49:42,525][20347] Updated weights for policy 0, policy_version 606125 (0.0006) [2023-12-27 09:49:42,580][20347] Updated weights for policy 0, policy_version 606135 (0.0008) [2023-12-27 09:49:43,099][20381] Updated weights for policy 1, policy_version 605858 (0.0008) [2023-12-27 09:49:43,148][20381] Updated weights for policy 1, policy_version 605868 (0.0007) [2023-12-27 09:49:43,201][20381] Updated weights for policy 1, policy_version 605878 (0.0007) [2023-12-27 09:49:43,257][20381] Updated weights for policy 1, policy_version 605888 (0.0008) [2023-12-27 09:49:43,312][20347] Updated weights for policy 0, policy_version 606145 (0.0007) [2023-12-27 09:49:43,365][20347] Updated weights for policy 0, policy_version 606155 (0.0008) [2023-12-27 09:49:43,417][20347] Updated weights for policy 0, policy_version 606165 (0.0008) [2023-12-27 09:49:43,466][20347] Updated weights for policy 0, policy_version 606175 (0.0007) [2023-12-27 09:49:43,956][20381] Updated weights for policy 1, policy_version 605898 (0.0008) [2023-12-27 09:49:44,012][20381] Updated weights for policy 1, policy_version 605908 (0.0005) [2023-12-27 09:49:44,061][20381] Updated weights for policy 1, policy_version 605918 (0.0007) [2023-12-27 09:49:44,166][20347] Updated weights for policy 0, policy_version 606185 (0.0007) [2023-12-27 09:49:44,222][20347] Updated weights for policy 0, policy_version 606195 (0.0008) [2023-12-27 09:49:44,268][20347] Updated weights for policy 0, policy_version 606205 (0.0007) [2023-12-27 09:49:44,722][20381] Updated weights for policy 1, policy_version 605928 (0.0008) [2023-12-27 09:49:44,776][20381] Updated weights for policy 1, policy_version 605938 (0.0008) [2023-12-27 09:49:44,832][20381] Updated weights for policy 1, policy_version 605948 (0.0007) [2023-12-27 09:49:44,988][20347] Updated weights for policy 0, policy_version 606215 (0.0008) [2023-12-27 09:49:45,050][20347] Updated weights for policy 0, policy_version 606225 (0.0008) [2023-12-27 09:49:45,110][20347] Updated weights for policy 0, policy_version 606235 (0.0008) [2023-12-27 09:49:45,532][20381] Updated weights for policy 1, policy_version 605958 (0.0008) [2023-12-27 09:49:45,594][20381] Updated weights for policy 1, policy_version 605968 (0.0007) [2023-12-27 09:49:45,648][20381] Updated weights for policy 1, policy_version 605978 (0.0007) [2023-12-27 09:49:45,810][20347] Updated weights for policy 0, policy_version 606245 (0.0008) [2023-12-27 09:49:45,866][20347] Updated weights for policy 0, policy_version 606255 (0.0008) [2023-12-27 09:49:45,926][20347] Updated weights for policy 0, policy_version 606265 (0.0008) [2023-12-27 09:49:46,328][20381] Updated weights for policy 1, policy_version 605988 (0.0007) [2023-12-27 09:49:46,392][20381] Updated weights for policy 1, policy_version 605998 (0.0007) [2023-12-27 09:49:46,452][20381] Updated weights for policy 1, policy_version 606008 (0.0006) [2023-12-27 09:49:46,587][20347] Updated weights for policy 0, policy_version 606275 (0.0007) [2023-12-27 09:49:46,647][20347] Updated weights for policy 0, policy_version 606285 (0.0006) [2023-12-27 09:49:46,705][20347] Updated weights for policy 0, policy_version 606295 (0.0005) [2023-12-27 09:49:47,113][20381] Updated weights for policy 1, policy_version 606018 (0.0008) [2023-12-27 09:49:47,165][20381] Updated weights for policy 1, policy_version 606028 (0.0009) [2023-12-27 09:49:47,211][20381] Updated weights for policy 1, policy_version 606038 (0.0009) [2023-12-27 09:49:47,243][20347] Updated weights for policy 0, policy_version 606305 (0.0005) [2023-12-27 09:49:47,260][20381] Updated weights for policy 1, policy_version 606048 (0.0007) [2023-12-27 09:49:47,299][20347] Updated weights for policy 0, policy_version 606315 (0.0005) [2023-12-27 09:49:47,354][20347] Updated weights for policy 0, policy_version 606325 (0.0006) [2023-12-27 09:49:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 310403072. Throughput: 0: 10269.3, 1: 10336.6. Samples: 310413684. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:47,363][19170] Avg episode reward: [(0, '961.270'), (1, '893.590')] [2023-12-27 09:49:47,411][20347] Updated weights for policy 0, policy_version 606335 (0.0007) [2023-12-27 09:49:48,010][20381] Updated weights for policy 1, policy_version 606058 (0.0007) [2023-12-27 09:49:48,029][20347] Updated weights for policy 0, policy_version 606345 (0.0009) [2023-12-27 09:49:48,069][20381] Updated weights for policy 1, policy_version 606068 (0.0010) [2023-12-27 09:49:48,090][20347] Updated weights for policy 0, policy_version 606355 (0.0008) [2023-12-27 09:49:48,119][20381] Updated weights for policy 1, policy_version 606078 (0.0006) [2023-12-27 09:49:48,153][20347] Updated weights for policy 0, policy_version 606365 (0.0007) [2023-12-27 09:49:48,780][20347] Updated weights for policy 0, policy_version 606375 (0.0007) [2023-12-27 09:49:48,838][20347] Updated weights for policy 0, policy_version 606385 (0.0006) [2023-12-27 09:49:48,838][20381] Updated weights for policy 1, policy_version 606088 (0.0007) [2023-12-27 09:49:48,887][20381] Updated weights for policy 1, policy_version 606098 (0.0006) [2023-12-27 09:49:48,895][20347] Updated weights for policy 0, policy_version 606395 (0.0007) [2023-12-27 09:49:48,942][20381] Updated weights for policy 1, policy_version 606108 (0.0006) [2023-12-27 09:49:49,554][20347] Updated weights for policy 0, policy_version 606405 (0.0007) [2023-12-27 09:49:49,610][20347] Updated weights for policy 0, policy_version 606415 (0.0008) [2023-12-27 09:49:49,667][20381] Updated weights for policy 1, policy_version 606118 (0.0007) [2023-12-27 09:49:49,669][20347] Updated weights for policy 0, policy_version 606425 (0.0006) [2023-12-27 09:49:49,720][20381] Updated weights for policy 1, policy_version 606128 (0.0006) [2023-12-27 09:49:49,772][20381] Updated weights for policy 1, policy_version 606138 (0.0007) [2023-12-27 09:49:50,415][20347] Updated weights for policy 0, policy_version 606435 (0.0008) [2023-12-27 09:49:50,469][20347] Updated weights for policy 0, policy_version 606445 (0.0006) [2023-12-27 09:49:50,509][20381] Updated weights for policy 1, policy_version 606148 (0.0008) [2023-12-27 09:49:50,524][20347] Updated weights for policy 0, policy_version 606455 (0.0005) [2023-12-27 09:49:50,560][20381] Updated weights for policy 1, policy_version 606158 (0.0007) [2023-12-27 09:49:50,622][20381] Updated weights for policy 1, policy_version 606168 (0.0007) [2023-12-27 09:49:51,194][20347] Updated weights for policy 0, policy_version 606465 (0.0006) [2023-12-27 09:49:51,251][20347] Updated weights for policy 0, policy_version 606475 (0.0010) [2023-12-27 09:49:51,314][20347] Updated weights for policy 0, policy_version 606485 (0.0010) [2023-12-27 09:49:51,351][20381] Updated weights for policy 1, policy_version 606178 (0.0007) [2023-12-27 09:49:51,376][20347] Updated weights for policy 0, policy_version 606495 (0.0009) [2023-12-27 09:49:51,417][20381] Updated weights for policy 1, policy_version 606188 (0.0006) [2023-12-27 09:49:51,470][20381] Updated weights for policy 1, policy_version 606198 (0.0007) [2023-12-27 09:49:51,525][20381] Updated weights for policy 1, policy_version 606208 (0.0007) [2023-12-27 09:49:52,049][20347] Updated weights for policy 0, policy_version 606505 (0.0008) [2023-12-27 09:49:52,110][20347] Updated weights for policy 0, policy_version 606515 (0.0006) [2023-12-27 09:49:52,159][20347] Updated weights for policy 0, policy_version 606525 (0.0008) [2023-12-27 09:49:52,232][20381] Updated weights for policy 1, policy_version 606218 (0.0008) [2023-12-27 09:49:52,289][20381] Updated weights for policy 1, policy_version 606228 (0.0008) [2023-12-27 09:49:52,352][20381] Updated weights for policy 1, policy_version 606238 (0.0008) [2023-12-27 09:49:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 310509568. Throughput: 0: 10288.8, 1: 10276.2. Samples: 310474760. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:52,363][19170] Avg episode reward: [(0, '961.350'), (1, '903.040')] [2023-12-27 09:49:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000606240_155222016.pth... [2023-12-27 09:49:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000606528_155287552.pth... [2023-12-27 09:49:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000605024_154910720.pth [2023-12-27 09:49:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000605312_154976256.pth [2023-12-27 09:49:52,809][20347] Updated weights for policy 0, policy_version 606535 (0.0007) [2023-12-27 09:49:52,862][20347] Updated weights for policy 0, policy_version 606545 (0.0007) [2023-12-27 09:49:52,914][20347] Updated weights for policy 0, policy_version 606555 (0.0007) [2023-12-27 09:49:53,059][20381] Updated weights for policy 1, policy_version 606248 (0.0007) [2023-12-27 09:49:53,118][20381] Updated weights for policy 1, policy_version 606258 (0.0008) [2023-12-27 09:49:53,172][20381] Updated weights for policy 1, policy_version 606268 (0.0007) [2023-12-27 09:49:53,608][20347] Updated weights for policy 0, policy_version 606565 (0.0008) [2023-12-27 09:49:53,661][20347] Updated weights for policy 0, policy_version 606575 (0.0007) [2023-12-27 09:49:53,718][20347] Updated weights for policy 0, policy_version 606585 (0.0008) [2023-12-27 09:49:53,863][20381] Updated weights for policy 1, policy_version 606278 (0.0008) [2023-12-27 09:49:53,917][20381] Updated weights for policy 1, policy_version 606288 (0.0008) [2023-12-27 09:49:53,958][20381] Updated weights for policy 1, policy_version 606298 (0.0007) [2023-12-27 09:49:54,381][20347] Updated weights for policy 0, policy_version 606595 (0.0007) [2023-12-27 09:49:54,434][20347] Updated weights for policy 0, policy_version 606605 (0.0007) [2023-12-27 09:49:54,492][20347] Updated weights for policy 0, policy_version 606615 (0.0006) [2023-12-27 09:49:54,632][20381] Updated weights for policy 1, policy_version 606308 (0.0009) [2023-12-27 09:49:54,692][20381] Updated weights for policy 1, policy_version 606318 (0.0007) [2023-12-27 09:49:54,755][20381] Updated weights for policy 1, policy_version 606328 (0.0011) [2023-12-27 09:49:55,076][20347] Updated weights for policy 0, policy_version 606625 (0.0006) [2023-12-27 09:49:55,138][20347] Updated weights for policy 0, policy_version 606635 (0.0006) [2023-12-27 09:49:55,196][20347] Updated weights for policy 0, policy_version 606645 (0.0006) [2023-12-27 09:49:55,257][20347] Updated weights for policy 0, policy_version 606655 (0.0006) [2023-12-27 09:49:55,362][20381] Updated weights for policy 1, policy_version 606338 (0.0011) [2023-12-27 09:49:55,412][20381] Updated weights for policy 1, policy_version 606348 (0.0010) [2023-12-27 09:49:55,461][20381] Updated weights for policy 1, policy_version 606358 (0.0011) [2023-12-27 09:49:55,520][20381] Updated weights for policy 1, policy_version 606368 (0.0011) [2023-12-27 09:49:55,811][20347] Updated weights for policy 0, policy_version 606665 (0.0006) [2023-12-27 09:49:55,860][20347] Updated weights for policy 0, policy_version 606675 (0.0005) [2023-12-27 09:49:55,909][20347] Updated weights for policy 0, policy_version 606685 (0.0005) [2023-12-27 09:49:56,210][20381] Updated weights for policy 1, policy_version 606378 (0.0011) [2023-12-27 09:49:56,271][20381] Updated weights for policy 1, policy_version 606388 (0.0006) [2023-12-27 09:49:56,318][20381] Updated weights for policy 1, policy_version 606398 (0.0005) [2023-12-27 09:49:56,517][20347] Updated weights for policy 0, policy_version 606695 (0.0008) [2023-12-27 09:49:56,559][20347] Updated weights for policy 0, policy_version 606705 (0.0010) [2023-12-27 09:49:56,607][20347] Updated weights for policy 0, policy_version 606715 (0.0010) [2023-12-27 09:49:56,866][20381] Updated weights for policy 1, policy_version 606408 (0.0005) [2023-12-27 09:49:56,925][20381] Updated weights for policy 1, policy_version 606418 (0.0008) [2023-12-27 09:49:56,984][20381] Updated weights for policy 1, policy_version 606428 (0.0010) [2023-12-27 09:49:57,315][20347] Updated weights for policy 0, policy_version 606725 (0.0009) [2023-12-27 09:49:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 310607872. Throughput: 0: 10343.7, 1: 10304.3. Samples: 310599452. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:49:57,363][19170] Avg episode reward: [(0, '971.040'), (1, '961.360')] [2023-12-27 09:49:57,371][20347] Updated weights for policy 0, policy_version 606735 (0.0007) [2023-12-27 09:49:57,428][20347] Updated weights for policy 0, policy_version 606745 (0.0007) [2023-12-27 09:49:57,633][20381] Updated weights for policy 1, policy_version 606438 (0.0010) [2023-12-27 09:49:57,678][20381] Updated weights for policy 1, policy_version 606448 (0.0010) [2023-12-27 09:49:57,725][20381] Updated weights for policy 1, policy_version 606458 (0.0010) [2023-12-27 09:49:58,044][20347] Updated weights for policy 0, policy_version 606755 (0.0007) [2023-12-27 09:49:58,093][20347] Updated weights for policy 0, policy_version 606765 (0.0007) [2023-12-27 09:49:58,147][20347] Updated weights for policy 0, policy_version 606775 (0.0007) [2023-12-27 09:49:58,416][20381] Updated weights for policy 1, policy_version 606468 (0.0010) [2023-12-27 09:49:58,473][20381] Updated weights for policy 1, policy_version 606478 (0.0008) [2023-12-27 09:49:58,526][20381] Updated weights for policy 1, policy_version 606488 (0.0008) [2023-12-27 09:49:58,820][20347] Updated weights for policy 0, policy_version 606785 (0.0006) [2023-12-27 09:49:58,886][20347] Updated weights for policy 0, policy_version 606795 (0.0008) [2023-12-27 09:49:58,954][20347] Updated weights for policy 0, policy_version 606805 (0.0008) [2023-12-27 09:49:59,012][20347] Updated weights for policy 0, policy_version 606815 (0.0008) [2023-12-27 09:49:59,248][20381] Updated weights for policy 1, policy_version 606498 (0.0008) [2023-12-27 09:49:59,307][20381] Updated weights for policy 1, policy_version 606508 (0.0007) [2023-12-27 09:49:59,373][20381] Updated weights for policy 1, policy_version 606518 (0.0007) [2023-12-27 09:49:59,435][20381] Updated weights for policy 1, policy_version 606528 (0.0008) [2023-12-27 09:49:59,725][20347] Updated weights for policy 0, policy_version 606825 (0.0008) [2023-12-27 09:49:59,784][20347] Updated weights for policy 0, policy_version 606835 (0.0007) [2023-12-27 09:49:59,843][20347] Updated weights for policy 0, policy_version 606845 (0.0008) [2023-12-27 09:50:00,110][20381] Updated weights for policy 1, policy_version 606538 (0.0007) [2023-12-27 09:50:00,166][20381] Updated weights for policy 1, policy_version 606548 (0.0010) [2023-12-27 09:50:00,225][20381] Updated weights for policy 1, policy_version 606558 (0.0010) [2023-12-27 09:50:00,506][20347] Updated weights for policy 0, policy_version 606855 (0.0006) [2023-12-27 09:50:00,558][20347] Updated weights for policy 0, policy_version 606865 (0.0005) [2023-12-27 09:50:00,614][20347] Updated weights for policy 0, policy_version 606875 (0.0005) [2023-12-27 09:50:00,890][20381] Updated weights for policy 1, policy_version 606568 (0.0009) [2023-12-27 09:50:00,950][20381] Updated weights for policy 1, policy_version 606578 (0.0008) [2023-12-27 09:50:01,009][20381] Updated weights for policy 1, policy_version 606588 (0.0008) [2023-12-27 09:50:01,189][20347] Updated weights for policy 0, policy_version 606885 (0.0007) [2023-12-27 09:50:01,256][20347] Updated weights for policy 0, policy_version 606895 (0.0006) [2023-12-27 09:50:01,323][20347] Updated weights for policy 0, policy_version 606905 (0.0006) [2023-12-27 09:50:01,657][20381] Updated weights for policy 1, policy_version 606598 (0.0006) [2023-12-27 09:50:01,706][20381] Updated weights for policy 1, policy_version 606608 (0.0005) [2023-12-27 09:50:01,768][20381] Updated weights for policy 1, policy_version 606618 (0.0008) [2023-12-27 09:50:02,012][20347] Updated weights for policy 0, policy_version 606915 (0.0009) [2023-12-27 09:50:02,072][20347] Updated weights for policy 0, policy_version 606925 (0.0010) [2023-12-27 09:50:02,131][20347] Updated weights for policy 0, policy_version 606935 (0.0011) [2023-12-27 09:50:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 310714368. Throughput: 0: 10356.9, 1: 10298.8. Samples: 310724816. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:02,363][19170] Avg episode reward: [(0, '860.500'), (1, '971.110')] [2023-12-27 09:50:02,408][20381] Updated weights for policy 1, policy_version 606628 (0.0007) [2023-12-27 09:50:02,468][20381] Updated weights for policy 1, policy_version 606638 (0.0005) [2023-12-27 09:50:02,523][20381] Updated weights for policy 1, policy_version 606648 (0.0005) [2023-12-27 09:50:02,762][20347] Updated weights for policy 0, policy_version 606945 (0.0010) [2023-12-27 09:50:02,821][20347] Updated weights for policy 0, policy_version 606955 (0.0009) [2023-12-27 09:50:02,871][20347] Updated weights for policy 0, policy_version 606965 (0.0010) [2023-12-27 09:50:02,921][20347] Updated weights for policy 0, policy_version 606975 (0.0010) [2023-12-27 09:50:03,149][20381] Updated weights for policy 1, policy_version 606658 (0.0006) [2023-12-27 09:50:03,213][20381] Updated weights for policy 1, policy_version 606668 (0.0006) [2023-12-27 09:50:03,272][20381] Updated weights for policy 1, policy_version 606678 (0.0006) [2023-12-27 09:50:03,330][20381] Updated weights for policy 1, policy_version 606688 (0.0006) [2023-12-27 09:50:03,536][20347] Updated weights for policy 0, policy_version 606985 (0.0010) [2023-12-27 09:50:03,583][20347] Updated weights for policy 0, policy_version 606995 (0.0007) [2023-12-27 09:50:03,629][20347] Updated weights for policy 0, policy_version 607005 (0.0010) [2023-12-27 09:50:03,967][20381] Updated weights for policy 1, policy_version 606698 (0.0006) [2023-12-27 09:50:04,027][20381] Updated weights for policy 1, policy_version 606708 (0.0008) [2023-12-27 09:50:04,092][20381] Updated weights for policy 1, policy_version 606718 (0.0008) [2023-12-27 09:50:04,328][20347] Updated weights for policy 0, policy_version 607015 (0.0009) [2023-12-27 09:50:04,386][20347] Updated weights for policy 0, policy_version 607025 (0.0011) [2023-12-27 09:50:04,445][20347] Updated weights for policy 0, policy_version 607035 (0.0010) [2023-12-27 09:50:04,827][20381] Updated weights for policy 1, policy_version 606728 (0.0008) [2023-12-27 09:50:04,888][20381] Updated weights for policy 1, policy_version 606738 (0.0006) [2023-12-27 09:50:04,944][20381] Updated weights for policy 1, policy_version 606748 (0.0009) [2023-12-27 09:50:05,059][20347] Updated weights for policy 0, policy_version 607045 (0.0009) [2023-12-27 09:50:05,107][20347] Updated weights for policy 0, policy_version 607055 (0.0010) [2023-12-27 09:50:05,159][20347] Updated weights for policy 0, policy_version 607065 (0.0010) [2023-12-27 09:50:05,547][20381] Updated weights for policy 1, policy_version 606758 (0.0008) [2023-12-27 09:50:05,604][20381] Updated weights for policy 1, policy_version 606768 (0.0007) [2023-12-27 09:50:05,660][20381] Updated weights for policy 1, policy_version 606778 (0.0007) [2023-12-27 09:50:05,848][20347] Updated weights for policy 0, policy_version 607075 (0.0010) [2023-12-27 09:50:05,893][20347] Updated weights for policy 0, policy_version 607085 (0.0010) [2023-12-27 09:50:05,942][20347] Updated weights for policy 0, policy_version 607095 (0.0010) [2023-12-27 09:50:06,287][20381] Updated weights for policy 1, policy_version 606788 (0.0007) [2023-12-27 09:50:06,336][20381] Updated weights for policy 1, policy_version 606798 (0.0007) [2023-12-27 09:50:06,401][20381] Updated weights for policy 1, policy_version 606808 (0.0007) [2023-12-27 09:50:06,603][20347] Updated weights for policy 0, policy_version 607105 (0.0009) [2023-12-27 09:50:06,669][20347] Updated weights for policy 0, policy_version 607115 (0.0008) [2023-12-27 09:50:06,728][20347] Updated weights for policy 0, policy_version 607125 (0.0008) [2023-12-27 09:50:06,772][20347] Updated weights for policy 0, policy_version 607135 (0.0008) [2023-12-27 09:50:07,043][20381] Updated weights for policy 1, policy_version 606818 (0.0008) [2023-12-27 09:50:07,099][20381] Updated weights for policy 1, policy_version 606828 (0.0007) [2023-12-27 09:50:07,157][20381] Updated weights for policy 1, policy_version 606838 (0.0008) [2023-12-27 09:50:07,206][20381] Updated weights for policy 1, policy_version 606848 (0.0007) [2023-12-27 09:50:07,363][19170] Fps is (10 sec: 21298.2, 60 sec: 20752.9, 300 sec: 20716.0). Total num frames: 310820864. Throughput: 0: 10390.9, 1: 10283.4. Samples: 310788584. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:07,364][19170] Avg episode reward: [(0, '782.250'), (1, '951.810')] [2023-12-27 09:50:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000606848_155377664.pth... [2023-12-27 09:50:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000605632_155066368.pth [2023-12-27 09:50:07,413][20347] Updated weights for policy 0, policy_version 607145 (0.0008) [2023-12-27 09:50:07,468][20347] Updated weights for policy 0, policy_version 607155 (0.0008) [2023-12-27 09:50:07,521][20347] Updated weights for policy 0, policy_version 607165 (0.0008) [2023-12-27 09:50:07,533][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000607168_155451392.pth... [2023-12-27 09:50:07,536][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000605920_155131904.pth [2023-12-27 09:50:07,892][20381] Updated weights for policy 1, policy_version 606858 (0.0008) [2023-12-27 09:50:07,949][20381] Updated weights for policy 1, policy_version 606868 (0.0008) [2023-12-27 09:50:07,992][20292] KL-divergence is very high: 156.9563 [2023-12-27 09:50:08,004][20381] Updated weights for policy 1, policy_version 606878 (0.0007) [2023-12-27 09:50:08,206][20347] Updated weights for policy 0, policy_version 607175 (0.0007) [2023-12-27 09:50:08,265][20347] Updated weights for policy 0, policy_version 607185 (0.0007) [2023-12-27 09:50:08,321][20347] Updated weights for policy 0, policy_version 607195 (0.0008) [2023-12-27 09:50:08,652][20381] Updated weights for policy 1, policy_version 606888 (0.0009) [2023-12-27 09:50:08,707][20381] Updated weights for policy 1, policy_version 606898 (0.0008) [2023-12-27 09:50:08,758][20381] Updated weights for policy 1, policy_version 606908 (0.0008) [2023-12-27 09:50:09,023][20347] Updated weights for policy 0, policy_version 607205 (0.0008) [2023-12-27 09:50:09,079][20347] Updated weights for policy 0, policy_version 607215 (0.0007) [2023-12-27 09:50:09,125][20347] Updated weights for policy 0, policy_version 607225 (0.0008) [2023-12-27 09:50:09,498][20381] Updated weights for policy 1, policy_version 606918 (0.0008) [2023-12-27 09:50:09,557][20381] Updated weights for policy 1, policy_version 606928 (0.0008) [2023-12-27 09:50:09,618][20381] Updated weights for policy 1, policy_version 606938 (0.0008) [2023-12-27 09:50:09,883][20347] Updated weights for policy 0, policy_version 607235 (0.0007) [2023-12-27 09:50:09,949][20347] Updated weights for policy 0, policy_version 607245 (0.0007) [2023-12-27 09:50:10,012][20347] Updated weights for policy 0, policy_version 607255 (0.0006) [2023-12-27 09:50:10,344][20381] Updated weights for policy 1, policy_version 606948 (0.0007) [2023-12-27 09:50:10,402][20381] Updated weights for policy 1, policy_version 606958 (0.0008) [2023-12-27 09:50:10,468][20381] Updated weights for policy 1, policy_version 606968 (0.0008) [2023-12-27 09:50:10,610][20347] Updated weights for policy 0, policy_version 607265 (0.0006) [2023-12-27 09:50:10,673][20347] Updated weights for policy 0, policy_version 607275 (0.0005) [2023-12-27 09:50:10,732][20347] Updated weights for policy 0, policy_version 607285 (0.0007) [2023-12-27 09:50:10,795][20347] Updated weights for policy 0, policy_version 607295 (0.0005) [2023-12-27 09:50:11,143][20381] Updated weights for policy 1, policy_version 606978 (0.0007) [2023-12-27 09:50:11,205][20381] Updated weights for policy 1, policy_version 606988 (0.0006) [2023-12-27 09:50:11,263][20381] Updated weights for policy 1, policy_version 606998 (0.0007) [2023-12-27 09:50:11,321][20381] Updated weights for policy 1, policy_version 607008 (0.0007) [2023-12-27 09:50:11,442][20347] Updated weights for policy 0, policy_version 607305 (0.0008) [2023-12-27 09:50:11,495][20347] Updated weights for policy 0, policy_version 607315 (0.0007) [2023-12-27 09:50:11,545][20347] Updated weights for policy 0, policy_version 607325 (0.0008) [2023-12-27 09:50:12,042][20381] Updated weights for policy 1, policy_version 607018 (0.0011) [2023-12-27 09:50:12,109][20381] Updated weights for policy 1, policy_version 607028 (0.0011) [2023-12-27 09:50:12,168][20381] Updated weights for policy 1, policy_version 607038 (0.0011) [2023-12-27 09:50:12,276][20347] Updated weights for policy 0, policy_version 607335 (0.0007) [2023-12-27 09:50:12,336][20347] Updated weights for policy 0, policy_version 607345 (0.0008) [2023-12-27 09:50:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 310919168. Throughput: 0: 10402.5, 1: 10317.8. Samples: 310912880. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:12,363][19170] Avg episode reward: [(0, '864.170'), (1, '961.470')] [2023-12-27 09:50:12,397][20347] Updated weights for policy 0, policy_version 607355 (0.0008) [2023-12-27 09:50:12,869][20381] Updated weights for policy 1, policy_version 607048 (0.0009) [2023-12-27 09:50:12,924][20381] Updated weights for policy 1, policy_version 607058 (0.0010) [2023-12-27 09:50:12,982][20381] Updated weights for policy 1, policy_version 607068 (0.0010) [2023-12-27 09:50:13,105][20347] Updated weights for policy 0, policy_version 607365 (0.0007) [2023-12-27 09:50:13,158][20347] Updated weights for policy 0, policy_version 607375 (0.0007) [2023-12-27 09:50:13,222][20347] Updated weights for policy 0, policy_version 607385 (0.0008) [2023-12-27 09:50:13,570][20381] Updated weights for policy 1, policy_version 607078 (0.0007) [2023-12-27 09:50:13,627][20381] Updated weights for policy 1, policy_version 607088 (0.0008) [2023-12-27 09:50:13,679][20381] Updated weights for policy 1, policy_version 607098 (0.0010) [2023-12-27 09:50:13,857][20347] Updated weights for policy 0, policy_version 607395 (0.0006) [2023-12-27 09:50:13,920][20347] Updated weights for policy 0, policy_version 607405 (0.0005) [2023-12-27 09:50:13,980][20347] Updated weights for policy 0, policy_version 607415 (0.0005) [2023-12-27 09:50:14,355][20381] Updated weights for policy 1, policy_version 607108 (0.0010) [2023-12-27 09:50:14,411][20381] Updated weights for policy 1, policy_version 607118 (0.0011) [2023-12-27 09:50:14,463][20381] Updated weights for policy 1, policy_version 607128 (0.0010) [2023-12-27 09:50:14,504][20347] Updated weights for policy 0, policy_version 607425 (0.0005) [2023-12-27 09:50:14,551][20347] Updated weights for policy 0, policy_version 607435 (0.0005) [2023-12-27 09:50:14,601][20347] Updated weights for policy 0, policy_version 607445 (0.0006) [2023-12-27 09:50:14,651][20347] Updated weights for policy 0, policy_version 607455 (0.0006) [2023-12-27 09:50:15,144][20381] Updated weights for policy 1, policy_version 607138 (0.0010) [2023-12-27 09:50:15,193][20381] Updated weights for policy 1, policy_version 607148 (0.0011) [2023-12-27 09:50:15,241][20381] Updated weights for policy 1, policy_version 607158 (0.0010) [2023-12-27 09:50:15,294][20381] Updated weights for policy 1, policy_version 607168 (0.0011) [2023-12-27 09:50:15,335][20347] Updated weights for policy 0, policy_version 607465 (0.0007) [2023-12-27 09:50:15,386][20347] Updated weights for policy 0, policy_version 607475 (0.0007) [2023-12-27 09:50:15,450][20347] Updated weights for policy 0, policy_version 607485 (0.0007) [2023-12-27 09:50:15,947][20381] Updated weights for policy 1, policy_version 607178 (0.0010) [2023-12-27 09:50:15,999][20381] Updated weights for policy 1, policy_version 607188 (0.0010) [2023-12-27 09:50:16,053][20381] Updated weights for policy 1, policy_version 607198 (0.0010) [2023-12-27 09:50:16,171][20347] Updated weights for policy 0, policy_version 607495 (0.0007) [2023-12-27 09:50:16,229][20347] Updated weights for policy 0, policy_version 607505 (0.0005) [2023-12-27 09:50:16,277][20347] Updated weights for policy 0, policy_version 607515 (0.0005) [2023-12-27 09:50:16,659][20381] Updated weights for policy 1, policy_version 607208 (0.0011) [2023-12-27 09:50:16,703][20381] Updated weights for policy 1, policy_version 607218 (0.0008) [2023-12-27 09:50:16,751][20381] Updated weights for policy 1, policy_version 607228 (0.0010) [2023-12-27 09:50:16,878][20347] Updated weights for policy 0, policy_version 607525 (0.0006) [2023-12-27 09:50:16,922][20347] Updated weights for policy 0, policy_version 607535 (0.0007) [2023-12-27 09:50:16,968][20347] Updated weights for policy 0, policy_version 607545 (0.0007) [2023-12-27 09:50:17,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 311025664. Throughput: 0: 10393.9, 1: 10341.2. Samples: 311038004. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:17,363][19170] Avg episode reward: [(0, '902.980'), (1, '990.280')] [2023-12-27 09:50:17,388][20381] Updated weights for policy 1, policy_version 607238 (0.0007) [2023-12-27 09:50:17,433][20381] Updated weights for policy 1, policy_version 607248 (0.0005) [2023-12-27 09:50:17,492][20381] Updated weights for policy 1, policy_version 607258 (0.0005) [2023-12-27 09:50:17,588][20347] Updated weights for policy 0, policy_version 607555 (0.0007) [2023-12-27 09:50:17,641][20347] Updated weights for policy 0, policy_version 607565 (0.0005) [2023-12-27 09:50:17,697][20347] Updated weights for policy 0, policy_version 607575 (0.0005) [2023-12-27 09:50:18,072][20381] Updated weights for policy 1, policy_version 607268 (0.0007) [2023-12-27 09:50:18,128][20381] Updated weights for policy 1, policy_version 607278 (0.0008) [2023-12-27 09:50:18,180][20381] Updated weights for policy 1, policy_version 607288 (0.0008) [2023-12-27 09:50:18,261][20347] Updated weights for policy 0, policy_version 607585 (0.0005) [2023-12-27 09:50:18,319][20347] Updated weights for policy 0, policy_version 607595 (0.0007) [2023-12-27 09:50:18,389][20347] Updated weights for policy 0, policy_version 607605 (0.0007) [2023-12-27 09:50:18,448][20347] Updated weights for policy 0, policy_version 607615 (0.0008) [2023-12-27 09:50:18,880][20381] Updated weights for policy 1, policy_version 607298 (0.0007) [2023-12-27 09:50:18,932][20381] Updated weights for policy 1, policy_version 607308 (0.0007) [2023-12-27 09:50:18,982][20381] Updated weights for policy 1, policy_version 607318 (0.0010) [2023-12-27 09:50:19,027][20381] Updated weights for policy 1, policy_version 607328 (0.0010) [2023-12-27 09:50:19,125][20347] Updated weights for policy 0, policy_version 607625 (0.0007) [2023-12-27 09:50:19,171][20347] Updated weights for policy 0, policy_version 607635 (0.0007) [2023-12-27 09:50:19,225][20347] Updated weights for policy 0, policy_version 607645 (0.0007) [2023-12-27 09:50:19,755][20381] Updated weights for policy 1, policy_version 607338 (0.0011) [2023-12-27 09:50:19,815][20381] Updated weights for policy 1, policy_version 607348 (0.0010) [2023-12-27 09:50:19,877][20381] Updated weights for policy 1, policy_version 607358 (0.0010) [2023-12-27 09:50:19,955][20347] Updated weights for policy 0, policy_version 607655 (0.0008) [2023-12-27 09:50:20,013][20347] Updated weights for policy 0, policy_version 607665 (0.0007) [2023-12-27 09:50:20,071][20347] Updated weights for policy 0, policy_version 607675 (0.0007) [2023-12-27 09:50:20,600][20381] Updated weights for policy 1, policy_version 607368 (0.0008) [2023-12-27 09:50:20,664][20381] Updated weights for policy 1, policy_version 607378 (0.0010) [2023-12-27 09:50:20,719][20381] Updated weights for policy 1, policy_version 607388 (0.0011) [2023-12-27 09:50:20,750][20347] Updated weights for policy 0, policy_version 607685 (0.0008) [2023-12-27 09:50:20,805][20347] Updated weights for policy 0, policy_version 607695 (0.0007) [2023-12-27 09:50:20,864][20347] Updated weights for policy 0, policy_version 607705 (0.0007) [2023-12-27 09:50:21,455][20381] Updated weights for policy 1, policy_version 607398 (0.0011) [2023-12-27 09:50:21,515][20381] Updated weights for policy 1, policy_version 607408 (0.0011) [2023-12-27 09:50:21,532][20347] Updated weights for policy 0, policy_version 607715 (0.0006) [2023-12-27 09:50:21,573][20381] Updated weights for policy 1, policy_version 607418 (0.0011) [2023-12-27 09:50:21,586][20347] Updated weights for policy 0, policy_version 607725 (0.0006) [2023-12-27 09:50:21,647][20347] Updated weights for policy 0, policy_version 607735 (0.0007) [2023-12-27 09:50:22,294][20381] Updated weights for policy 1, policy_version 607428 (0.0009) [2023-12-27 09:50:22,318][20347] Updated weights for policy 0, policy_version 607745 (0.0006) [2023-12-27 09:50:22,357][20381] Updated weights for policy 1, policy_version 607438 (0.0008) [2023-12-27 09:50:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 311123968. Throughput: 0: 10414.1, 1: 10292.2. Samples: 311100436. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:22,363][19170] Avg episode reward: [(0, '757.220'), (1, '990.280')] [2023-12-27 09:50:22,377][20347] Updated weights for policy 0, policy_version 607755 (0.0008) [2023-12-27 09:50:22,419][20381] Updated weights for policy 1, policy_version 607448 (0.0009) [2023-12-27 09:50:22,436][20347] Updated weights for policy 0, policy_version 607765 (0.0007) [2023-12-27 09:50:22,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000607456_155533312.pth... [2023-12-27 09:50:22,470][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000606240_155222016.pth [2023-12-27 09:50:22,500][20347] Updated weights for policy 0, policy_version 607775 (0.0006) [2023-12-27 09:50:22,504][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000607776_155607040.pth... [2023-12-27 09:50:22,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000606528_155287552.pth [2023-12-27 09:50:23,076][20381] Updated weights for policy 1, policy_version 607458 (0.0008) [2023-12-27 09:50:23,128][20381] Updated weights for policy 1, policy_version 607468 (0.0008) [2023-12-27 09:50:23,175][20347] Updated weights for policy 0, policy_version 607785 (0.0006) [2023-12-27 09:50:23,180][20381] Updated weights for policy 1, policy_version 607478 (0.0006) [2023-12-27 09:50:23,226][20347] Updated weights for policy 0, policy_version 607795 (0.0006) [2023-12-27 09:50:23,240][20381] Updated weights for policy 1, policy_version 607488 (0.0008) [2023-12-27 09:50:23,293][20347] Updated weights for policy 0, policy_version 607805 (0.0008) [2023-12-27 09:50:23,801][20381] Updated weights for policy 1, policy_version 607498 (0.0011) [2023-12-27 09:50:23,853][20381] Updated weights for policy 1, policy_version 607508 (0.0010) [2023-12-27 09:50:23,908][20381] Updated weights for policy 1, policy_version 607518 (0.0010) [2023-12-27 09:50:24,001][20347] Updated weights for policy 0, policy_version 607815 (0.0008) [2023-12-27 09:50:24,052][20347] Updated weights for policy 0, policy_version 607825 (0.0007) [2023-12-27 09:50:24,110][20347] Updated weights for policy 0, policy_version 607835 (0.0008) [2023-12-27 09:50:24,590][20381] Updated weights for policy 1, policy_version 607528 (0.0008) [2023-12-27 09:50:24,644][20381] Updated weights for policy 1, policy_version 607538 (0.0006) [2023-12-27 09:50:24,696][20381] Updated weights for policy 1, policy_version 607548 (0.0010) [2023-12-27 09:50:24,834][20347] Updated weights for policy 0, policy_version 607845 (0.0008) [2023-12-27 09:50:24,894][20347] Updated weights for policy 0, policy_version 607855 (0.0008) [2023-12-27 09:50:24,946][20347] Updated weights for policy 0, policy_version 607865 (0.0008) [2023-12-27 09:50:25,280][20381] Updated weights for policy 1, policy_version 607558 (0.0010) [2023-12-27 09:50:25,343][20381] Updated weights for policy 1, policy_version 607568 (0.0008) [2023-12-27 09:50:25,403][20381] Updated weights for policy 1, policy_version 607578 (0.0007) [2023-12-27 09:50:25,616][20347] Updated weights for policy 0, policy_version 607875 (0.0007) [2023-12-27 09:50:25,674][20347] Updated weights for policy 0, policy_version 607885 (0.0006) [2023-12-27 09:50:25,723][20347] Updated weights for policy 0, policy_version 607895 (0.0008) [2023-12-27 09:50:26,081][20381] Updated weights for policy 1, policy_version 607588 (0.0008) [2023-12-27 09:50:26,137][20381] Updated weights for policy 1, policy_version 607598 (0.0007) [2023-12-27 09:50:26,199][20381] Updated weights for policy 1, policy_version 607608 (0.0007) [2023-12-27 09:50:26,319][20347] Updated weights for policy 0, policy_version 607905 (0.0006) [2023-12-27 09:50:26,371][20347] Updated weights for policy 0, policy_version 607915 (0.0005) [2023-12-27 09:50:26,420][20347] Updated weights for policy 0, policy_version 607925 (0.0009) [2023-12-27 09:50:26,472][20347] Updated weights for policy 0, policy_version 607935 (0.0010) [2023-12-27 09:50:26,847][20381] Updated weights for policy 1, policy_version 607618 (0.0006) [2023-12-27 09:50:26,897][20381] Updated weights for policy 1, policy_version 607628 (0.0005) [2023-12-27 09:50:26,942][20381] Updated weights for policy 1, policy_version 607638 (0.0005) [2023-12-27 09:50:26,990][20381] Updated weights for policy 1, policy_version 607648 (0.0005) [2023-12-27 09:50:27,026][20347] Updated weights for policy 0, policy_version 607945 (0.0008) [2023-12-27 09:50:27,077][20347] Updated weights for policy 0, policy_version 607955 (0.0008) [2023-12-27 09:50:27,122][20347] Updated weights for policy 0, policy_version 607965 (0.0009) [2023-12-27 09:50:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 311238656. Throughput: 0: 10408.6, 1: 10306.4. Samples: 311223780. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:27,364][19170] Avg episode reward: [(0, '805.450'), (1, '990.290')] [2023-12-27 09:50:27,535][20381] Updated weights for policy 1, policy_version 607658 (0.0010) [2023-12-27 09:50:27,596][20381] Updated weights for policy 1, policy_version 607668 (0.0011) [2023-12-27 09:50:27,658][20381] Updated weights for policy 1, policy_version 607678 (0.0011) [2023-12-27 09:50:27,744][20347] Updated weights for policy 0, policy_version 607975 (0.0007) [2023-12-27 09:50:27,803][20347] Updated weights for policy 0, policy_version 607985 (0.0008) [2023-12-27 09:50:27,859][20347] Updated weights for policy 0, policy_version 607995 (0.0007) [2023-12-27 09:50:28,237][20381] Updated weights for policy 1, policy_version 607688 (0.0006) [2023-12-27 09:50:28,283][20381] Updated weights for policy 1, policy_version 607698 (0.0005) [2023-12-27 09:50:28,346][20381] Updated weights for policy 1, policy_version 607708 (0.0006) [2023-12-27 09:50:28,490][20347] Updated weights for policy 0, policy_version 608005 (0.0009) [2023-12-27 09:50:28,546][20347] Updated weights for policy 0, policy_version 608015 (0.0009) [2023-12-27 09:50:28,600][20347] Updated weights for policy 0, policy_version 608025 (0.0010) [2023-12-27 09:50:29,003][20381] Updated weights for policy 1, policy_version 607718 (0.0009) [2023-12-27 09:50:29,059][20381] Updated weights for policy 1, policy_version 607728 (0.0009) [2023-12-27 09:50:29,113][20381] Updated weights for policy 1, policy_version 607738 (0.0010) [2023-12-27 09:50:29,231][20347] Updated weights for policy 0, policy_version 608035 (0.0010) [2023-12-27 09:50:29,287][20347] Updated weights for policy 0, policy_version 608045 (0.0007) [2023-12-27 09:50:29,342][20347] Updated weights for policy 0, policy_version 608055 (0.0008) [2023-12-27 09:50:29,794][20381] Updated weights for policy 1, policy_version 607748 (0.0010) [2023-12-27 09:50:29,856][20381] Updated weights for policy 1, policy_version 607758 (0.0010) [2023-12-27 09:50:29,915][20381] Updated weights for policy 1, policy_version 607768 (0.0011) [2023-12-27 09:50:29,944][20347] Updated weights for policy 0, policy_version 608065 (0.0006) [2023-12-27 09:50:30,002][20347] Updated weights for policy 0, policy_version 608075 (0.0007) [2023-12-27 09:50:30,063][20347] Updated weights for policy 0, policy_version 608085 (0.0006) [2023-12-27 09:50:30,120][20347] Updated weights for policy 0, policy_version 608095 (0.0007) [2023-12-27 09:50:30,530][20381] Updated weights for policy 1, policy_version 607778 (0.0010) [2023-12-27 09:50:30,589][20381] Updated weights for policy 1, policy_version 607788 (0.0010) [2023-12-27 09:50:30,641][20381] Updated weights for policy 1, policy_version 607798 (0.0010) [2023-12-27 09:50:30,689][20381] Updated weights for policy 1, policy_version 607808 (0.0010) [2023-12-27 09:50:30,780][20347] Updated weights for policy 0, policy_version 608105 (0.0007) [2023-12-27 09:50:30,836][20347] Updated weights for policy 0, policy_version 608115 (0.0007) [2023-12-27 09:50:30,892][20347] Updated weights for policy 0, policy_version 608125 (0.0007) [2023-12-27 09:50:31,394][20381] Updated weights for policy 1, policy_version 607818 (0.0009) [2023-12-27 09:50:31,453][20381] Updated weights for policy 1, policy_version 607828 (0.0007) [2023-12-27 09:50:31,515][20381] Updated weights for policy 1, policy_version 607838 (0.0005) [2023-12-27 09:50:31,554][20347] Updated weights for policy 0, policy_version 608135 (0.0007) [2023-12-27 09:50:31,605][20347] Updated weights for policy 0, policy_version 608145 (0.0008) [2023-12-27 09:50:31,665][20347] Updated weights for policy 0, policy_version 608155 (0.0007) [2023-12-27 09:50:32,171][20381] Updated weights for policy 1, policy_version 607848 (0.0007) [2023-12-27 09:50:32,221][20381] Updated weights for policy 1, policy_version 607858 (0.0007) [2023-12-27 09:50:32,275][20381] Updated weights for policy 1, policy_version 607868 (0.0007) [2023-12-27 09:50:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 311345152. Throughput: 0: 10540.3, 1: 10364.1. Samples: 311354380. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:32,363][19170] Avg episode reward: [(0, '834.790'), (1, '990.290')] [2023-12-27 09:50:32,368][20347] Updated weights for policy 0, policy_version 608165 (0.0007) [2023-12-27 09:50:32,423][20347] Updated weights for policy 0, policy_version 608175 (0.0007) [2023-12-27 09:50:32,471][20347] Updated weights for policy 0, policy_version 608185 (0.0008) [2023-12-27 09:50:32,952][20381] Updated weights for policy 1, policy_version 607878 (0.0006) [2023-12-27 09:50:33,014][20381] Updated weights for policy 1, policy_version 607888 (0.0005) [2023-12-27 09:50:33,071][20381] Updated weights for policy 1, policy_version 607898 (0.0005) [2023-12-27 09:50:33,075][20347] Updated weights for policy 0, policy_version 608195 (0.0008) [2023-12-27 09:50:33,136][20347] Updated weights for policy 0, policy_version 608205 (0.0008) [2023-12-27 09:50:33,188][20347] Updated weights for policy 0, policy_version 608215 (0.0007) [2023-12-27 09:50:33,650][20381] Updated weights for policy 1, policy_version 607908 (0.0006) [2023-12-27 09:50:33,697][20381] Updated weights for policy 1, policy_version 607918 (0.0007) [2023-12-27 09:50:33,745][20381] Updated weights for policy 1, policy_version 607928 (0.0007) [2023-12-27 09:50:33,803][20347] Updated weights for policy 0, policy_version 608225 (0.0008) [2023-12-27 09:50:33,856][20347] Updated weights for policy 0, policy_version 608235 (0.0007) [2023-12-27 09:50:33,911][20347] Updated weights for policy 0, policy_version 608245 (0.0007) [2023-12-27 09:50:33,958][20347] Updated weights for policy 0, policy_version 608255 (0.0007) [2023-12-27 09:50:34,419][20381] Updated weights for policy 1, policy_version 607938 (0.0007) [2023-12-27 09:50:34,474][20381] Updated weights for policy 1, policy_version 607948 (0.0007) [2023-12-27 09:50:34,535][20381] Updated weights for policy 1, policy_version 607958 (0.0006) [2023-12-27 09:50:34,597][20381] Updated weights for policy 1, policy_version 607968 (0.0006) [2023-12-27 09:50:34,686][20347] Updated weights for policy 0, policy_version 608265 (0.0008) [2023-12-27 09:50:34,739][20347] Updated weights for policy 0, policy_version 608275 (0.0009) [2023-12-27 09:50:34,785][20347] Updated weights for policy 0, policy_version 608285 (0.0010) [2023-12-27 09:50:35,130][20381] Updated weights for policy 1, policy_version 607978 (0.0005) [2023-12-27 09:50:35,182][20381] Updated weights for policy 1, policy_version 607988 (0.0005) [2023-12-27 09:50:35,226][20381] Updated weights for policy 1, policy_version 607998 (0.0005) [2023-12-27 09:50:35,529][20347] Updated weights for policy 0, policy_version 608295 (0.0007) [2023-12-27 09:50:35,579][20347] Updated weights for policy 0, policy_version 608305 (0.0008) [2023-12-27 09:50:35,641][20347] Updated weights for policy 0, policy_version 608315 (0.0008) [2023-12-27 09:50:35,850][20381] Updated weights for policy 1, policy_version 608008 (0.0007) [2023-12-27 09:50:35,908][20381] Updated weights for policy 1, policy_version 608018 (0.0007) [2023-12-27 09:50:35,960][20381] Updated weights for policy 1, policy_version 608028 (0.0007) [2023-12-27 09:50:36,351][20347] Updated weights for policy 0, policy_version 608325 (0.0008) [2023-12-27 09:50:36,407][20347] Updated weights for policy 0, policy_version 608335 (0.0008) [2023-12-27 09:50:36,464][20347] Updated weights for policy 0, policy_version 608345 (0.0008) [2023-12-27 09:50:36,693][20381] Updated weights for policy 1, policy_version 608038 (0.0008) [2023-12-27 09:50:36,749][20381] Updated weights for policy 1, policy_version 608048 (0.0009) [2023-12-27 09:50:36,797][20381] Updated weights for policy 1, policy_version 608058 (0.0009) [2023-12-27 09:50:37,217][20347] Updated weights for policy 0, policy_version 608355 (0.0009) [2023-12-27 09:50:37,273][20347] Updated weights for policy 0, policy_version 608365 (0.0009) [2023-12-27 09:50:37,325][20347] Updated weights for policy 0, policy_version 608375 (0.0009) [2023-12-27 09:50:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 311443456. Throughput: 0: 10532.0, 1: 10464.3. Samples: 311419596. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:37,364][19170] Avg episode reward: [(0, '815.570'), (1, '1000.000')] [2023-12-27 09:50:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000608064_155688960.pth... [2023-12-27 09:50:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000608384_155762688.pth... [2023-12-27 09:50:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000607168_155451392.pth [2023-12-27 09:50:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000606848_155377664.pth [2023-12-27 09:50:37,488][20381] Updated weights for policy 1, policy_version 608068 (0.0007) [2023-12-27 09:50:37,549][20381] Updated weights for policy 1, policy_version 608078 (0.0005) [2023-12-27 09:50:37,610][20381] Updated weights for policy 1, policy_version 608088 (0.0006) [2023-12-27 09:50:38,000][20347] Updated weights for policy 0, policy_version 608385 (0.0007) [2023-12-27 09:50:38,055][20347] Updated weights for policy 0, policy_version 608395 (0.0006) [2023-12-27 09:50:38,115][20347] Updated weights for policy 0, policy_version 608405 (0.0009) [2023-12-27 09:50:38,170][20347] Updated weights for policy 0, policy_version 608415 (0.0010) [2023-12-27 09:50:38,258][20381] Updated weights for policy 1, policy_version 608098 (0.0007) [2023-12-27 09:50:38,318][20381] Updated weights for policy 1, policy_version 608108 (0.0007) [2023-12-27 09:50:38,376][20381] Updated weights for policy 1, policy_version 608118 (0.0007) [2023-12-27 09:50:38,433][20381] Updated weights for policy 1, policy_version 608128 (0.0009) [2023-12-27 09:50:38,816][20347] Updated weights for policy 0, policy_version 608425 (0.0008) [2023-12-27 09:50:38,866][20347] Updated weights for policy 0, policy_version 608435 (0.0008) [2023-12-27 09:50:38,928][20347] Updated weights for policy 0, policy_version 608445 (0.0007) [2023-12-27 09:50:39,140][20381] Updated weights for policy 1, policy_version 608138 (0.0007) [2023-12-27 09:50:39,193][20381] Updated weights for policy 1, policy_version 608148 (0.0008) [2023-12-27 09:50:39,256][20381] Updated weights for policy 1, policy_version 608158 (0.0007) [2023-12-27 09:50:39,647][20347] Updated weights for policy 0, policy_version 608455 (0.0007) [2023-12-27 09:50:39,706][20347] Updated weights for policy 0, policy_version 608465 (0.0008) [2023-12-27 09:50:39,756][20347] Updated weights for policy 0, policy_version 608475 (0.0008) [2023-12-27 09:50:39,987][20381] Updated weights for policy 1, policy_version 608168 (0.0008) [2023-12-27 09:50:40,045][20381] Updated weights for policy 1, policy_version 608178 (0.0008) [2023-12-27 09:50:40,121][20381] Updated weights for policy 1, policy_version 608188 (0.0008) [2023-12-27 09:50:40,488][20347] Updated weights for policy 0, policy_version 608485 (0.0008) [2023-12-27 09:50:40,540][20291] KL-divergence is very high: 112.9938 [2023-12-27 09:50:40,545][20291] KL-divergence is very high: 107.3386 [2023-12-27 09:50:40,545][20347] Updated weights for policy 0, policy_version 608495 (0.0007) [2023-12-27 09:50:40,550][20291] KL-divergence is very high: 179.6704 [2023-12-27 09:50:40,601][20347] Updated weights for policy 0, policy_version 608505 (0.0007) [2023-12-27 09:50:40,772][20381] Updated weights for policy 1, policy_version 608198 (0.0008) [2023-12-27 09:50:40,829][20381] Updated weights for policy 1, policy_version 608208 (0.0007) [2023-12-27 09:50:40,892][20381] Updated weights for policy 1, policy_version 608218 (0.0005) [2023-12-27 09:50:41,356][20347] Updated weights for policy 0, policy_version 608515 (0.0008) [2023-12-27 09:50:41,422][20347] Updated weights for policy 0, policy_version 608525 (0.0008) [2023-12-27 09:50:41,482][20347] Updated weights for policy 0, policy_version 608535 (0.0008) [2023-12-27 09:50:41,575][20381] Updated weights for policy 1, policy_version 608228 (0.0005) [2023-12-27 09:50:41,637][20381] Updated weights for policy 1, policy_version 608238 (0.0007) [2023-12-27 09:50:41,692][20381] Updated weights for policy 1, policy_version 608248 (0.0006) [2023-12-27 09:50:42,169][20347] Updated weights for policy 0, policy_version 608545 (0.0008) [2023-12-27 09:50:42,221][20347] Updated weights for policy 0, policy_version 608555 (0.0007) [2023-12-27 09:50:42,282][20347] Updated weights for policy 0, policy_version 608565 (0.0006) [2023-12-27 09:50:42,328][20381] Updated weights for policy 1, policy_version 608258 (0.0010) [2023-12-27 09:50:42,341][20347] Updated weights for policy 0, policy_version 608575 (0.0007) [2023-12-27 09:50:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 311549952. Throughput: 0: 10433.0, 1: 10461.8. Samples: 311539716. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:42,363][19170] Avg episode reward: [(0, '844.590'), (1, '990.380')] [2023-12-27 09:50:42,390][20381] Updated weights for policy 1, policy_version 608268 (0.0010) [2023-12-27 09:50:42,447][20381] Updated weights for policy 1, policy_version 608278 (0.0009) [2023-12-27 09:50:42,504][20381] Updated weights for policy 1, policy_version 608288 (0.0005) [2023-12-27 09:50:43,040][20347] Updated weights for policy 0, policy_version 608585 (0.0006) [2023-12-27 09:50:43,076][20381] Updated weights for policy 1, policy_version 608298 (0.0005) [2023-12-27 09:50:43,088][20347] Updated weights for policy 0, policy_version 608595 (0.0006) [2023-12-27 09:50:43,139][20381] Updated weights for policy 1, policy_version 608308 (0.0006) [2023-12-27 09:50:43,144][20347] Updated weights for policy 0, policy_version 608605 (0.0006) [2023-12-27 09:50:43,197][20381] Updated weights for policy 1, policy_version 608318 (0.0010) [2023-12-27 09:50:43,777][20347] Updated weights for policy 0, policy_version 608615 (0.0007) [2023-12-27 09:50:43,783][20381] Updated weights for policy 1, policy_version 608328 (0.0008) [2023-12-27 09:50:43,826][20347] Updated weights for policy 0, policy_version 608625 (0.0010) [2023-12-27 09:50:43,832][20381] Updated weights for policy 1, policy_version 608338 (0.0005) [2023-12-27 09:50:43,880][20347] Updated weights for policy 0, policy_version 608635 (0.0010) [2023-12-27 09:50:43,889][20381] Updated weights for policy 1, policy_version 608348 (0.0007) [2023-12-27 09:50:44,510][20381] Updated weights for policy 1, policy_version 608358 (0.0010) [2023-12-27 09:50:44,521][20347] Updated weights for policy 0, policy_version 608645 (0.0008) [2023-12-27 09:50:44,567][20381] Updated weights for policy 1, policy_version 608368 (0.0010) [2023-12-27 09:50:44,579][20347] Updated weights for policy 0, policy_version 608655 (0.0009) [2023-12-27 09:50:44,622][20381] Updated weights for policy 1, policy_version 608378 (0.0010) [2023-12-27 09:50:44,631][20347] Updated weights for policy 0, policy_version 608665 (0.0010) [2023-12-27 09:50:45,274][20347] Updated weights for policy 0, policy_version 608675 (0.0010) [2023-12-27 09:50:45,304][20381] Updated weights for policy 1, policy_version 608388 (0.0010) [2023-12-27 09:50:45,327][20347] Updated weights for policy 0, policy_version 608685 (0.0006) [2023-12-27 09:50:45,359][20381] Updated weights for policy 1, policy_version 608398 (0.0010) [2023-12-27 09:50:45,378][20347] Updated weights for policy 0, policy_version 608695 (0.0005) [2023-12-27 09:50:45,411][20381] Updated weights for policy 1, policy_version 608408 (0.0010) [2023-12-27 09:50:45,929][20347] Updated weights for policy 0, policy_version 608705 (0.0009) [2023-12-27 09:50:45,984][20347] Updated weights for policy 0, policy_version 608715 (0.0005) [2023-12-27 09:50:46,035][20347] Updated weights for policy 0, policy_version 608725 (0.0005) [2023-12-27 09:50:46,086][20347] Updated weights for policy 0, policy_version 608735 (0.0008) [2023-12-27 09:50:46,100][20381] Updated weights for policy 1, policy_version 608418 (0.0011) [2023-12-27 09:50:46,153][20381] Updated weights for policy 1, policy_version 608428 (0.0007) [2023-12-27 09:50:46,201][20381] Updated weights for policy 1, policy_version 608438 (0.0007) [2023-12-27 09:50:46,256][20381] Updated weights for policy 1, policy_version 608448 (0.0010) [2023-12-27 09:50:46,677][20347] Updated weights for policy 0, policy_version 608745 (0.0007) [2023-12-27 09:50:46,741][20347] Updated weights for policy 0, policy_version 608755 (0.0007) [2023-12-27 09:50:46,806][20347] Updated weights for policy 0, policy_version 608765 (0.0007) [2023-12-27 09:50:46,872][20381] Updated weights for policy 1, policy_version 608458 (0.0008) [2023-12-27 09:50:46,928][20381] Updated weights for policy 1, policy_version 608468 (0.0010) [2023-12-27 09:50:46,980][20381] Updated weights for policy 1, policy_version 608478 (0.0010) [2023-12-27 09:50:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 311656448. Throughput: 0: 10459.9, 1: 10524.3. Samples: 311669104. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:47,363][19170] Avg episode reward: [(0, '883.220'), (1, '980.790')] [2023-12-27 09:50:47,515][20347] Updated weights for policy 0, policy_version 608775 (0.0007) [2023-12-27 09:50:47,523][20381] Updated weights for policy 1, policy_version 608488 (0.0006) [2023-12-27 09:50:47,567][20347] Updated weights for policy 0, policy_version 608785 (0.0008) [2023-12-27 09:50:47,573][20381] Updated weights for policy 1, policy_version 608498 (0.0005) [2023-12-27 09:50:47,624][20347] Updated weights for policy 0, policy_version 608795 (0.0007) [2023-12-27 09:50:47,630][20381] Updated weights for policy 1, policy_version 608508 (0.0010) [2023-12-27 09:50:48,252][20347] Updated weights for policy 0, policy_version 608805 (0.0005) [2023-12-27 09:50:48,255][20381] Updated weights for policy 1, policy_version 608518 (0.0011) [2023-12-27 09:50:48,307][20381] Updated weights for policy 1, policy_version 608528 (0.0010) [2023-12-27 09:50:48,309][20347] Updated weights for policy 0, policy_version 608815 (0.0005) [2023-12-27 09:50:48,370][20381] Updated weights for policy 1, policy_version 608538 (0.0011) [2023-12-27 09:50:48,379][20347] Updated weights for policy 0, policy_version 608825 (0.0006) [2023-12-27 09:50:49,030][20347] Updated weights for policy 0, policy_version 608835 (0.0007) [2023-12-27 09:50:49,081][20347] Updated weights for policy 0, policy_version 608845 (0.0008) [2023-12-27 09:50:49,090][20381] Updated weights for policy 1, policy_version 608548 (0.0010) [2023-12-27 09:50:49,130][20347] Updated weights for policy 0, policy_version 608855 (0.0006) [2023-12-27 09:50:49,146][20381] Updated weights for policy 1, policy_version 608558 (0.0006) [2023-12-27 09:50:49,204][20381] Updated weights for policy 1, policy_version 608568 (0.0006) [2023-12-27 09:50:49,864][20381] Updated weights for policy 1, policy_version 608578 (0.0008) [2023-12-27 09:50:49,896][20347] Updated weights for policy 0, policy_version 608865 (0.0006) [2023-12-27 09:50:49,923][20381] Updated weights for policy 1, policy_version 608588 (0.0006) [2023-12-27 09:50:49,955][20347] Updated weights for policy 0, policy_version 608875 (0.0006) [2023-12-27 09:50:49,973][20381] Updated weights for policy 1, policy_version 608598 (0.0006) [2023-12-27 09:50:50,009][20347] Updated weights for policy 0, policy_version 608885 (0.0006) [2023-12-27 09:50:50,025][20381] Updated weights for policy 1, policy_version 608608 (0.0005) [2023-12-27 09:50:50,064][20347] Updated weights for policy 0, policy_version 608895 (0.0007) [2023-12-27 09:50:50,670][20381] Updated weights for policy 1, policy_version 608618 (0.0008) [2023-12-27 09:50:50,727][20347] Updated weights for policy 0, policy_version 608905 (0.0010) [2023-12-27 09:50:50,729][20381] Updated weights for policy 1, policy_version 608628 (0.0006) [2023-12-27 09:50:50,778][20381] Updated weights for policy 1, policy_version 608638 (0.0005) [2023-12-27 09:50:50,783][20347] Updated weights for policy 0, policy_version 608915 (0.0008) [2023-12-27 09:50:50,832][20347] Updated weights for policy 0, policy_version 608925 (0.0010) [2023-12-27 09:50:51,424][20381] Updated weights for policy 1, policy_version 608648 (0.0006) [2023-12-27 09:50:51,488][20381] Updated weights for policy 1, policy_version 608658 (0.0011) [2023-12-27 09:50:51,515][20347] Updated weights for policy 0, policy_version 608935 (0.0008) [2023-12-27 09:50:51,552][20381] Updated weights for policy 1, policy_version 608668 (0.0011) [2023-12-27 09:50:51,574][20347] Updated weights for policy 0, policy_version 608945 (0.0008) [2023-12-27 09:50:51,633][20347] Updated weights for policy 0, policy_version 608955 (0.0009) [2023-12-27 09:50:52,162][20381] Updated weights for policy 1, policy_version 608678 (0.0006) [2023-12-27 09:50:52,221][20381] Updated weights for policy 1, policy_version 608688 (0.0005) [2023-12-27 09:50:52,284][20381] Updated weights for policy 1, policy_version 608698 (0.0007) [2023-12-27 09:50:52,352][20347] Updated weights for policy 0, policy_version 608965 (0.0007) [2023-12-27 09:50:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 311762944. Throughput: 0: 10403.9, 1: 10532.0. Samples: 311730692. Policy #0 lag: (min: 15.0, avg: 39.0, max: 47.0) [2023-12-27 09:50:52,363][19170] Avg episode reward: [(0, '902.950'), (1, '990.410')] [2023-12-27 09:50:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000608704_155852800.pth... [2023-12-27 09:50:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000607456_155533312.pth [2023-12-27 09:50:52,405][20347] Updated weights for policy 0, policy_version 608975 (0.0006) [2023-12-27 09:50:52,458][20347] Updated weights for policy 0, policy_version 608985 (0.0006) [2023-12-27 09:50:52,493][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000608992_155918336.pth... [2023-12-27 09:50:52,497][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000607776_155607040.pth [2023-12-27 09:50:52,827][20381] Updated weights for policy 1, policy_version 608708 (0.0005) [2023-12-27 09:50:52,877][20381] Updated weights for policy 1, policy_version 608718 (0.0006) [2023-12-27 09:50:52,929][20381] Updated weights for policy 1, policy_version 608728 (0.0005) [2023-12-27 09:50:53,184][20347] Updated weights for policy 0, policy_version 608995 (0.0007) [2023-12-27 09:50:53,237][20347] Updated weights for policy 0, policy_version 609005 (0.0007) [2023-12-27 09:50:53,289][20347] Updated weights for policy 0, policy_version 609015 (0.0006) [2023-12-27 09:50:53,572][20381] Updated weights for policy 1, policy_version 608738 (0.0007) [2023-12-27 09:50:53,632][20381] Updated weights for policy 1, policy_version 608748 (0.0008) [2023-12-27 09:50:53,689][20381] Updated weights for policy 1, policy_version 608758 (0.0008) [2023-12-27 09:50:53,739][20381] Updated weights for policy 1, policy_version 608768 (0.0008) [2023-12-27 09:50:53,959][20347] Updated weights for policy 0, policy_version 609025 (0.0008) [2023-12-27 09:50:54,017][20347] Updated weights for policy 0, policy_version 609035 (0.0008) [2023-12-27 09:50:54,069][20347] Updated weights for policy 0, policy_version 609045 (0.0007) [2023-12-27 09:50:54,121][20347] Updated weights for policy 0, policy_version 609055 (0.0008) [2023-12-27 09:50:54,470][20381] Updated weights for policy 1, policy_version 608778 (0.0008) [2023-12-27 09:50:54,525][20381] Updated weights for policy 1, policy_version 608788 (0.0008) [2023-12-27 09:50:54,576][20381] Updated weights for policy 1, policy_version 608798 (0.0008) [2023-12-27 09:50:54,760][20347] Updated weights for policy 0, policy_version 609065 (0.0007) [2023-12-27 09:50:54,823][20347] Updated weights for policy 0, policy_version 609075 (0.0006) [2023-12-27 09:50:54,880][20347] Updated weights for policy 0, policy_version 609085 (0.0006) [2023-12-27 09:50:55,302][20381] Updated weights for policy 1, policy_version 608808 (0.0007) [2023-12-27 09:50:55,356][20381] Updated weights for policy 1, policy_version 608818 (0.0007) [2023-12-27 09:50:55,403][20381] Updated weights for policy 1, policy_version 608828 (0.0007) [2023-12-27 09:50:55,528][20347] Updated weights for policy 0, policy_version 609095 (0.0007) [2023-12-27 09:50:55,585][20347] Updated weights for policy 0, policy_version 609105 (0.0007) [2023-12-27 09:50:55,632][20347] Updated weights for policy 0, policy_version 609115 (0.0008) [2023-12-27 09:50:56,060][20381] Updated weights for policy 1, policy_version 608838 (0.0007) [2023-12-27 09:50:56,106][20381] Updated weights for policy 1, policy_version 608848 (0.0008) [2023-12-27 09:50:56,156][20381] Updated weights for policy 1, policy_version 608858 (0.0009) [2023-12-27 09:50:56,284][20347] Updated weights for policy 0, policy_version 609125 (0.0007) [2023-12-27 09:50:56,328][20347] Updated weights for policy 0, policy_version 609135 (0.0005) [2023-12-27 09:50:56,376][20347] Updated weights for policy 0, policy_version 609145 (0.0005) [2023-12-27 09:50:56,820][20381] Updated weights for policy 1, policy_version 608868 (0.0009) [2023-12-27 09:50:56,878][20381] Updated weights for policy 1, policy_version 608878 (0.0011) [2023-12-27 09:50:56,939][20381] Updated weights for policy 1, policy_version 608888 (0.0011) [2023-12-27 09:50:57,024][20347] Updated weights for policy 0, policy_version 609155 (0.0006) [2023-12-27 09:50:57,077][20347] Updated weights for policy 0, policy_version 609165 (0.0008) [2023-12-27 09:50:57,123][20347] Updated weights for policy 0, policy_version 609175 (0.0007) [2023-12-27 09:50:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 311869440. Throughput: 0: 10385.3, 1: 10562.4. Samples: 311855528. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:50:57,363][19170] Avg episode reward: [(0, '893.290'), (1, '980.780')] [2023-12-27 09:50:57,509][20381] Updated weights for policy 1, policy_version 608898 (0.0005) [2023-12-27 09:50:57,557][20381] Updated weights for policy 1, policy_version 608908 (0.0008) [2023-12-27 09:50:57,626][20381] Updated weights for policy 1, policy_version 608918 (0.0006) [2023-12-27 09:50:57,680][20381] Updated weights for policy 1, policy_version 608928 (0.0007) [2023-12-27 09:50:57,766][20347] Updated weights for policy 0, policy_version 609185 (0.0008) [2023-12-27 09:50:57,817][20347] Updated weights for policy 0, policy_version 609195 (0.0007) [2023-12-27 09:50:57,861][20347] Updated weights for policy 0, policy_version 609205 (0.0007) [2023-12-27 09:50:57,907][20347] Updated weights for policy 0, policy_version 609215 (0.0005) [2023-12-27 09:50:58,320][20381] Updated weights for policy 1, policy_version 608938 (0.0008) [2023-12-27 09:50:58,385][20381] Updated weights for policy 1, policy_version 608948 (0.0008) [2023-12-27 09:50:58,445][20381] Updated weights for policy 1, policy_version 608958 (0.0008) [2023-12-27 09:50:58,648][20347] Updated weights for policy 0, policy_version 609225 (0.0011) [2023-12-27 09:50:58,712][20347] Updated weights for policy 0, policy_version 609235 (0.0010) [2023-12-27 09:50:58,777][20347] Updated weights for policy 0, policy_version 609245 (0.0010) [2023-12-27 09:50:59,233][20381] Updated weights for policy 1, policy_version 608968 (0.0008) [2023-12-27 09:50:59,293][20381] Updated weights for policy 1, policy_version 608978 (0.0007) [2023-12-27 09:50:59,355][20381] Updated weights for policy 1, policy_version 608988 (0.0007) [2023-12-27 09:50:59,524][20347] Updated weights for policy 0, policy_version 609255 (0.0008) [2023-12-27 09:50:59,585][20347] Updated weights for policy 0, policy_version 609265 (0.0009) [2023-12-27 09:50:59,645][20347] Updated weights for policy 0, policy_version 609275 (0.0005) [2023-12-27 09:51:00,075][20381] Updated weights for policy 1, policy_version 608998 (0.0008) [2023-12-27 09:51:00,126][20381] Updated weights for policy 1, policy_version 609008 (0.0009) [2023-12-27 09:51:00,179][20381] Updated weights for policy 1, policy_version 609018 (0.0008) [2023-12-27 09:51:00,265][20347] Updated weights for policy 0, policy_version 609285 (0.0006) [2023-12-27 09:51:00,313][20347] Updated weights for policy 0, policy_version 609295 (0.0007) [2023-12-27 09:51:00,368][20347] Updated weights for policy 0, policy_version 609305 (0.0006) [2023-12-27 09:51:00,906][20381] Updated weights for policy 1, policy_version 609028 (0.0009) [2023-12-27 09:51:00,956][20381] Updated weights for policy 1, policy_version 609038 (0.0010) [2023-12-27 09:51:01,007][20381] Updated weights for policy 1, policy_version 609048 (0.0010) [2023-12-27 09:51:01,045][20347] Updated weights for policy 0, policy_version 609315 (0.0005) [2023-12-27 09:51:01,102][20347] Updated weights for policy 0, policy_version 609325 (0.0008) [2023-12-27 09:51:01,162][20347] Updated weights for policy 0, policy_version 609335 (0.0006) [2023-12-27 09:51:01,699][20381] Updated weights for policy 1, policy_version 609058 (0.0010) [2023-12-27 09:51:01,761][20381] Updated weights for policy 1, policy_version 609068 (0.0008) [2023-12-27 09:51:01,812][20381] Updated weights for policy 1, policy_version 609078 (0.0008) [2023-12-27 09:51:01,817][20347] Updated weights for policy 0, policy_version 609345 (0.0008) [2023-12-27 09:51:01,864][20347] Updated weights for policy 0, policy_version 609355 (0.0007) [2023-12-27 09:51:01,866][20381] Updated weights for policy 1, policy_version 609088 (0.0008) [2023-12-27 09:51:01,913][20347] Updated weights for policy 0, policy_version 609365 (0.0007) [2023-12-27 09:51:01,964][20347] Updated weights for policy 0, policy_version 609375 (0.0007) [2023-12-27 09:51:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 311967744. Throughput: 0: 10384.4, 1: 10514.1. Samples: 311978440. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:02,363][19170] Avg episode reward: [(0, '903.020'), (1, '980.780')] [2023-12-27 09:51:02,599][20381] Updated weights for policy 1, policy_version 609098 (0.0010) [2023-12-27 09:51:02,654][20381] Updated weights for policy 1, policy_version 609108 (0.0011) [2023-12-27 09:51:02,676][20347] Updated weights for policy 0, policy_version 609385 (0.0005) [2023-12-27 09:51:02,705][20381] Updated weights for policy 1, policy_version 609118 (0.0006) [2023-12-27 09:51:02,733][20347] Updated weights for policy 0, policy_version 609395 (0.0007) [2023-12-27 09:51:02,794][20347] Updated weights for policy 0, policy_version 609405 (0.0007) [2023-12-27 09:51:03,340][20381] Updated weights for policy 1, policy_version 609128 (0.0007) [2023-12-27 09:51:03,395][20381] Updated weights for policy 1, policy_version 609138 (0.0007) [2023-12-27 09:51:03,448][20381] Updated weights for policy 1, policy_version 609148 (0.0006) [2023-12-27 09:51:03,450][20347] Updated weights for policy 0, policy_version 609415 (0.0007) [2023-12-27 09:51:03,500][20347] Updated weights for policy 0, policy_version 609425 (0.0007) [2023-12-27 09:51:03,555][20347] Updated weights for policy 0, policy_version 609435 (0.0008) [2023-12-27 09:51:04,147][20381] Updated weights for policy 1, policy_version 609158 (0.0007) [2023-12-27 09:51:04,207][20381] Updated weights for policy 1, policy_version 609168 (0.0007) [2023-12-27 09:51:04,241][20347] Updated weights for policy 0, policy_version 609445 (0.0008) [2023-12-27 09:51:04,270][20381] Updated weights for policy 1, policy_version 609178 (0.0007) [2023-12-27 09:51:04,298][20347] Updated weights for policy 0, policy_version 609455 (0.0007) [2023-12-27 09:51:04,360][20347] Updated weights for policy 0, policy_version 609465 (0.0007) [2023-12-27 09:51:04,930][20381] Updated weights for policy 1, policy_version 609188 (0.0006) [2023-12-27 09:51:04,991][20381] Updated weights for policy 1, policy_version 609198 (0.0007) [2023-12-27 09:51:05,042][20381] Updated weights for policy 1, policy_version 609208 (0.0007) [2023-12-27 09:51:05,048][20347] Updated weights for policy 0, policy_version 609475 (0.0008) [2023-12-27 09:51:05,110][20347] Updated weights for policy 0, policy_version 609485 (0.0008) [2023-12-27 09:51:05,177][20347] Updated weights for policy 0, policy_version 609495 (0.0008) [2023-12-27 09:51:05,733][20347] Updated weights for policy 0, policy_version 609505 (0.0009) [2023-12-27 09:51:05,783][20381] Updated weights for policy 1, policy_version 609218 (0.0008) [2023-12-27 09:51:05,788][20347] Updated weights for policy 0, policy_version 609515 (0.0005) [2023-12-27 09:51:05,842][20381] Updated weights for policy 1, policy_version 609228 (0.0008) [2023-12-27 09:51:05,844][20347] Updated weights for policy 0, policy_version 609525 (0.0005) [2023-12-27 09:51:05,902][20381] Updated weights for policy 1, policy_version 609238 (0.0006) [2023-12-27 09:51:05,904][20347] Updated weights for policy 0, policy_version 609535 (0.0008) [2023-12-27 09:51:05,956][20381] Updated weights for policy 1, policy_version 609248 (0.0007) [2023-12-27 09:51:06,519][20347] Updated weights for policy 0, policy_version 609545 (0.0010) [2023-12-27 09:51:06,576][20347] Updated weights for policy 0, policy_version 609555 (0.0008) [2023-12-27 09:51:06,633][20347] Updated weights for policy 0, policy_version 609565 (0.0009) [2023-12-27 09:51:06,667][20381] Updated weights for policy 1, policy_version 609258 (0.0009) [2023-12-27 09:51:06,716][20381] Updated weights for policy 1, policy_version 609268 (0.0008) [2023-12-27 09:51:06,768][20381] Updated weights for policy 1, policy_version 609278 (0.0007) [2023-12-27 09:51:07,315][20347] Updated weights for policy 0, policy_version 609575 (0.0009) [2023-12-27 09:51:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 312066048. Throughput: 0: 10357.9, 1: 10519.1. Samples: 312039904. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:07,363][19170] Avg episode reward: [(0, '893.350'), (1, '970.990')] [2023-12-27 09:51:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000609280_156000256.pth... [2023-12-27 09:51:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000608064_155688960.pth [2023-12-27 09:51:07,377][20347] Updated weights for policy 0, policy_version 609585 (0.0007) [2023-12-27 09:51:07,434][20347] Updated weights for policy 0, policy_version 609595 (0.0006) [2023-12-27 09:51:07,447][20381] Updated weights for policy 1, policy_version 609288 (0.0007) [2023-12-27 09:51:07,459][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000609600_156073984.pth... [2023-12-27 09:51:07,462][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000608384_155762688.pth [2023-12-27 09:51:07,494][20381] Updated weights for policy 1, policy_version 609298 (0.0007) [2023-12-27 09:51:07,546][20381] Updated weights for policy 1, policy_version 609308 (0.0008) [2023-12-27 09:51:08,101][20347] Updated weights for policy 0, policy_version 609605 (0.0005) [2023-12-27 09:51:08,157][20347] Updated weights for policy 0, policy_version 609615 (0.0005) [2023-12-27 09:51:08,211][20347] Updated weights for policy 0, policy_version 609625 (0.0005) [2023-12-27 09:51:08,277][20381] Updated weights for policy 1, policy_version 609318 (0.0008) [2023-12-27 09:51:08,321][20381] Updated weights for policy 1, policy_version 609328 (0.0008) [2023-12-27 09:51:08,384][20381] Updated weights for policy 1, policy_version 609338 (0.0008) [2023-12-27 09:51:08,783][20347] Updated weights for policy 0, policy_version 609635 (0.0006) [2023-12-27 09:51:08,831][20347] Updated weights for policy 0, policy_version 609645 (0.0007) [2023-12-27 09:51:08,885][20347] Updated weights for policy 0, policy_version 609655 (0.0007) [2023-12-27 09:51:09,097][20381] Updated weights for policy 1, policy_version 609348 (0.0008) [2023-12-27 09:51:09,156][20381] Updated weights for policy 1, policy_version 609358 (0.0007) [2023-12-27 09:51:09,209][20381] Updated weights for policy 1, policy_version 609368 (0.0007) [2023-12-27 09:51:09,626][20347] Updated weights for policy 0, policy_version 609665 (0.0008) [2023-12-27 09:51:09,683][20347] Updated weights for policy 0, policy_version 609675 (0.0008) [2023-12-27 09:51:09,742][20347] Updated weights for policy 0, policy_version 609685 (0.0007) [2023-12-27 09:51:09,805][20347] Updated weights for policy 0, policy_version 609695 (0.0008) [2023-12-27 09:51:09,944][20381] Updated weights for policy 1, policy_version 609378 (0.0008) [2023-12-27 09:51:10,005][20381] Updated weights for policy 1, policy_version 609388 (0.0011) [2023-12-27 09:51:10,059][20381] Updated weights for policy 1, policy_version 609398 (0.0011) [2023-12-27 09:51:10,111][20381] Updated weights for policy 1, policy_version 609408 (0.0011) [2023-12-27 09:51:10,527][20347] Updated weights for policy 0, policy_version 609705 (0.0008) [2023-12-27 09:51:10,581][20347] Updated weights for policy 0, policy_version 609715 (0.0007) [2023-12-27 09:51:10,638][20347] Updated weights for policy 0, policy_version 609725 (0.0006) [2023-12-27 09:51:10,802][20381] Updated weights for policy 1, policy_version 609418 (0.0010) [2023-12-27 09:51:10,851][20381] Updated weights for policy 1, policy_version 609428 (0.0010) [2023-12-27 09:51:10,907][20381] Updated weights for policy 1, policy_version 609438 (0.0008) [2023-12-27 09:51:11,318][20347] Updated weights for policy 0, policy_version 609735 (0.0009) [2023-12-27 09:51:11,386][20347] Updated weights for policy 0, policy_version 609745 (0.0009) [2023-12-27 09:51:11,444][20347] Updated weights for policy 0, policy_version 609755 (0.0008) [2023-12-27 09:51:11,636][20381] Updated weights for policy 1, policy_version 609448 (0.0008) [2023-12-27 09:51:11,696][20381] Updated weights for policy 1, policy_version 609458 (0.0008) [2023-12-27 09:51:11,768][20381] Updated weights for policy 1, policy_version 609468 (0.0010) [2023-12-27 09:51:12,192][20347] Updated weights for policy 0, policy_version 609765 (0.0008) [2023-12-27 09:51:12,256][20347] Updated weights for policy 0, policy_version 609775 (0.0008) [2023-12-27 09:51:12,321][20347] Updated weights for policy 0, policy_version 609785 (0.0010) [2023-12-27 09:51:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20716.1). Total num frames: 312164352. Throughput: 0: 10397.8, 1: 10455.4. Samples: 312162172. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:12,363][19170] Avg episode reward: [(0, '893.820'), (1, '970.940')] [2023-12-27 09:51:12,481][20381] Updated weights for policy 1, policy_version 609478 (0.0010) [2023-12-27 09:51:12,541][20381] Updated weights for policy 1, policy_version 609488 (0.0011) [2023-12-27 09:51:12,593][20381] Updated weights for policy 1, policy_version 609498 (0.0011) [2023-12-27 09:51:13,032][20347] Updated weights for policy 0, policy_version 609795 (0.0008) [2023-12-27 09:51:13,088][20347] Updated weights for policy 0, policy_version 609805 (0.0007) [2023-12-27 09:51:13,148][20347] Updated weights for policy 0, policy_version 609815 (0.0007) [2023-12-27 09:51:13,210][20381] Updated weights for policy 1, policy_version 609508 (0.0011) [2023-12-27 09:51:13,261][20381] Updated weights for policy 1, policy_version 609518 (0.0010) [2023-12-27 09:51:13,315][20381] Updated weights for policy 1, policy_version 609528 (0.0010) [2023-12-27 09:51:13,826][20347] Updated weights for policy 0, policy_version 609825 (0.0007) [2023-12-27 09:51:13,879][20347] Updated weights for policy 0, policy_version 609835 (0.0010) [2023-12-27 09:51:13,881][20381] Updated weights for policy 1, policy_version 609538 (0.0009) [2023-12-27 09:51:13,933][20347] Updated weights for policy 0, policy_version 609845 (0.0010) [2023-12-27 09:51:13,944][20381] Updated weights for policy 1, policy_version 609548 (0.0008) [2023-12-27 09:51:13,988][20347] Updated weights for policy 0, policy_version 609855 (0.0010) [2023-12-27 09:51:14,002][20381] Updated weights for policy 1, policy_version 609558 (0.0011) [2023-12-27 09:51:14,067][20381] Updated weights for policy 1, policy_version 609568 (0.0009) [2023-12-27 09:51:14,667][20381] Updated weights for policy 1, policy_version 609578 (0.0005) [2023-12-27 09:51:14,677][20347] Updated weights for policy 0, policy_version 609865 (0.0010) [2023-12-27 09:51:14,725][20381] Updated weights for policy 1, policy_version 609588 (0.0005) [2023-12-27 09:51:14,725][20347] Updated weights for policy 0, policy_version 609875 (0.0010) [2023-12-27 09:51:14,785][20347] Updated weights for policy 0, policy_version 609885 (0.0010) [2023-12-27 09:51:14,786][20381] Updated weights for policy 1, policy_version 609598 (0.0009) [2023-12-27 09:51:15,403][20381] Updated weights for policy 1, policy_version 609608 (0.0008) [2023-12-27 09:51:15,464][20381] Updated weights for policy 1, policy_version 609618 (0.0005) [2023-12-27 09:51:15,475][20347] Updated weights for policy 0, policy_version 609895 (0.0007) [2023-12-27 09:51:15,520][20381] Updated weights for policy 1, policy_version 609628 (0.0008) [2023-12-27 09:51:15,530][20347] Updated weights for policy 0, policy_version 609905 (0.0005) [2023-12-27 09:51:15,583][20347] Updated weights for policy 0, policy_version 609915 (0.0005) [2023-12-27 09:51:16,115][20381] Updated weights for policy 1, policy_version 609638 (0.0010) [2023-12-27 09:51:16,165][20381] Updated weights for policy 1, policy_version 609648 (0.0008) [2023-12-27 09:51:16,170][20347] Updated weights for policy 0, policy_version 609925 (0.0008) [2023-12-27 09:51:16,214][20381] Updated weights for policy 1, policy_version 609658 (0.0005) [2023-12-27 09:51:16,235][20347] Updated weights for policy 0, policy_version 609935 (0.0006) [2023-12-27 09:51:16,301][20347] Updated weights for policy 0, policy_version 609945 (0.0005) [2023-12-27 09:51:16,791][20347] Updated weights for policy 0, policy_version 609955 (0.0005) [2023-12-27 09:51:16,840][20347] Updated weights for policy 0, policy_version 609965 (0.0006) [2023-12-27 09:51:16,882][20381] Updated weights for policy 1, policy_version 609668 (0.0005) [2023-12-27 09:51:16,893][20347] Updated weights for policy 0, policy_version 609975 (0.0007) [2023-12-27 09:51:16,927][20381] Updated weights for policy 1, policy_version 609678 (0.0006) [2023-12-27 09:51:16,988][20381] Updated weights for policy 1, policy_version 609688 (0.0005) [2023-12-27 09:51:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 312279040. Throughput: 0: 10333.7, 1: 10457.1. Samples: 312289968. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:17,363][19170] Avg episode reward: [(0, '951.860'), (1, '990.340')] [2023-12-27 09:51:17,422][20347] Updated weights for policy 0, policy_version 609985 (0.0006) [2023-12-27 09:51:17,471][20347] Updated weights for policy 0, policy_version 609995 (0.0010) [2023-12-27 09:51:17,522][20347] Updated weights for policy 0, policy_version 610005 (0.0010) [2023-12-27 09:51:17,566][20347] Updated weights for policy 0, policy_version 610015 (0.0006) [2023-12-27 09:51:17,597][20381] Updated weights for policy 1, policy_version 609698 (0.0005) [2023-12-27 09:51:17,650][20381] Updated weights for policy 1, policy_version 609708 (0.0005) [2023-12-27 09:51:17,701][20381] Updated weights for policy 1, policy_version 609718 (0.0005) [2023-12-27 09:51:17,757][20381] Updated weights for policy 1, policy_version 609728 (0.0007) [2023-12-27 09:51:18,159][20347] Updated weights for policy 0, policy_version 610025 (0.0005) [2023-12-27 09:51:18,229][20347] Updated weights for policy 0, policy_version 610035 (0.0006) [2023-12-27 09:51:18,278][20347] Updated weights for policy 0, policy_version 610045 (0.0006) [2023-12-27 09:51:18,344][20381] Updated weights for policy 1, policy_version 609738 (0.0009) [2023-12-27 09:51:18,402][20381] Updated weights for policy 1, policy_version 609748 (0.0008) [2023-12-27 09:51:18,458][20381] Updated weights for policy 1, policy_version 609758 (0.0007) [2023-12-27 09:51:18,895][20347] Updated weights for policy 0, policy_version 610055 (0.0010) [2023-12-27 09:51:18,952][20347] Updated weights for policy 0, policy_version 610065 (0.0009) [2023-12-27 09:51:19,002][20347] Updated weights for policy 0, policy_version 610075 (0.0008) [2023-12-27 09:51:19,176][20381] Updated weights for policy 1, policy_version 609768 (0.0007) [2023-12-27 09:51:19,239][20381] Updated weights for policy 1, policy_version 609778 (0.0007) [2023-12-27 09:51:19,298][20381] Updated weights for policy 1, policy_version 609788 (0.0009) [2023-12-27 09:51:19,647][20347] Updated weights for policy 0, policy_version 610085 (0.0006) [2023-12-27 09:51:19,711][20347] Updated weights for policy 0, policy_version 610095 (0.0006) [2023-12-27 09:51:19,771][20347] Updated weights for policy 0, policy_version 610105 (0.0006) [2023-12-27 09:51:20,005][20381] Updated weights for policy 1, policy_version 609798 (0.0007) [2023-12-27 09:51:20,062][20381] Updated weights for policy 1, policy_version 609808 (0.0008) [2023-12-27 09:51:20,122][20381] Updated weights for policy 1, policy_version 609818 (0.0007) [2023-12-27 09:51:20,414][20347] Updated weights for policy 0, policy_version 610115 (0.0006) [2023-12-27 09:51:20,475][20347] Updated weights for policy 0, policy_version 610125 (0.0005) [2023-12-27 09:51:20,527][20347] Updated weights for policy 0, policy_version 610135 (0.0005) [2023-12-27 09:51:20,804][20381] Updated weights for policy 1, policy_version 609828 (0.0008) [2023-12-27 09:51:20,853][20381] Updated weights for policy 1, policy_version 609838 (0.0009) [2023-12-27 09:51:20,903][20381] Updated weights for policy 1, policy_version 609848 (0.0010) [2023-12-27 09:51:21,142][20347] Updated weights for policy 0, policy_version 610145 (0.0007) [2023-12-27 09:51:21,198][20347] Updated weights for policy 0, policy_version 610155 (0.0008) [2023-12-27 09:51:21,254][20347] Updated weights for policy 0, policy_version 610165 (0.0008) [2023-12-27 09:51:21,319][20347] Updated weights for policy 0, policy_version 610175 (0.0008) [2023-12-27 09:51:21,617][20381] Updated weights for policy 1, policy_version 609858 (0.0008) [2023-12-27 09:51:21,676][20381] Updated weights for policy 1, policy_version 609868 (0.0008) [2023-12-27 09:51:21,742][20381] Updated weights for policy 1, policy_version 609878 (0.0008) [2023-12-27 09:51:21,805][20381] Updated weights for policy 1, policy_version 609888 (0.0006) [2023-12-27 09:51:22,051][20347] Updated weights for policy 0, policy_version 610185 (0.0008) [2023-12-27 09:51:22,110][20347] Updated weights for policy 0, policy_version 610195 (0.0006) [2023-12-27 09:51:22,176][20347] Updated weights for policy 0, policy_version 610205 (0.0006) [2023-12-27 09:51:22,362][19170] Fps is (10 sec: 22118.3, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 312385536. Throughput: 0: 10388.9, 1: 10392.5. Samples: 312354752. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:22,363][19170] Avg episode reward: [(0, '942.150'), (1, '1000.000')] [2023-12-27 09:51:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000609888_156155904.pth... [2023-12-27 09:51:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000610208_156229632.pth... [2023-12-27 09:51:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000608992_155918336.pth [2023-12-27 09:51:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000608704_155852800.pth [2023-12-27 09:51:22,488][20381] Updated weights for policy 1, policy_version 609898 (0.0007) [2023-12-27 09:51:22,554][20381] Updated weights for policy 1, policy_version 609908 (0.0006) [2023-12-27 09:51:22,606][20381] Updated weights for policy 1, policy_version 609918 (0.0006) [2023-12-27 09:51:22,855][20347] Updated weights for policy 0, policy_version 610215 (0.0009) [2023-12-27 09:51:22,918][20347] Updated weights for policy 0, policy_version 610225 (0.0010) [2023-12-27 09:51:22,970][20347] Updated weights for policy 0, policy_version 610235 (0.0010) [2023-12-27 09:51:23,253][20381] Updated weights for policy 1, policy_version 609928 (0.0007) [2023-12-27 09:51:23,307][20381] Updated weights for policy 1, policy_version 609938 (0.0007) [2023-12-27 09:51:23,366][20381] Updated weights for policy 1, policy_version 609948 (0.0008) [2023-12-27 09:51:23,607][20347] Updated weights for policy 0, policy_version 610245 (0.0010) [2023-12-27 09:51:23,659][20347] Updated weights for policy 0, policy_version 610255 (0.0007) [2023-12-27 09:51:23,707][20347] Updated weights for policy 0, policy_version 610265 (0.0005) [2023-12-27 09:51:24,076][20381] Updated weights for policy 1, policy_version 609958 (0.0008) [2023-12-27 09:51:24,120][20381] Updated weights for policy 1, policy_version 609968 (0.0009) [2023-12-27 09:51:24,170][20381] Updated weights for policy 1, policy_version 609978 (0.0008) [2023-12-27 09:51:24,306][20347] Updated weights for policy 0, policy_version 610275 (0.0009) [2023-12-27 09:51:24,357][20347] Updated weights for policy 0, policy_version 610285 (0.0010) [2023-12-27 09:51:24,412][20347] Updated weights for policy 0, policy_version 610295 (0.0010) [2023-12-27 09:51:24,891][20381] Updated weights for policy 1, policy_version 609988 (0.0008) [2023-12-27 09:51:24,939][20381] Updated weights for policy 1, policy_version 609998 (0.0007) [2023-12-27 09:51:24,998][20381] Updated weights for policy 1, policy_version 610008 (0.0007) [2023-12-27 09:51:25,080][20347] Updated weights for policy 0, policy_version 610305 (0.0010) [2023-12-27 09:51:25,137][20347] Updated weights for policy 0, policy_version 610315 (0.0011) [2023-12-27 09:51:25,190][20347] Updated weights for policy 0, policy_version 610325 (0.0009) [2023-12-27 09:51:25,242][20347] Updated weights for policy 0, policy_version 610335 (0.0008) [2023-12-27 09:51:25,718][20381] Updated weights for policy 1, policy_version 610018 (0.0007) [2023-12-27 09:51:25,764][20381] Updated weights for policy 1, policy_version 610028 (0.0005) [2023-12-27 09:51:25,824][20381] Updated weights for policy 1, policy_version 610038 (0.0005) [2023-12-27 09:51:25,852][20347] Updated weights for policy 0, policy_version 610345 (0.0007) [2023-12-27 09:51:25,879][20381] Updated weights for policy 1, policy_version 610048 (0.0005) [2023-12-27 09:51:25,910][20347] Updated weights for policy 0, policy_version 610355 (0.0010) [2023-12-27 09:51:25,963][20347] Updated weights for policy 0, policy_version 610365 (0.0008) [2023-12-27 09:51:26,429][20381] Updated weights for policy 1, policy_version 610058 (0.0008) [2023-12-27 09:51:26,483][20381] Updated weights for policy 1, policy_version 610068 (0.0010) [2023-12-27 09:51:26,538][20381] Updated weights for policy 1, policy_version 610078 (0.0010) [2023-12-27 09:51:26,617][20347] Updated weights for policy 0, policy_version 610375 (0.0010) [2023-12-27 09:51:26,661][20347] Updated weights for policy 0, policy_version 610385 (0.0010) [2023-12-27 09:51:26,709][20347] Updated weights for policy 0, policy_version 610395 (0.0010) [2023-12-27 09:51:27,200][20381] Updated weights for policy 1, policy_version 610088 (0.0011) [2023-12-27 09:51:27,260][20381] Updated weights for policy 1, policy_version 610098 (0.0010) [2023-12-27 09:51:27,317][20381] Updated weights for policy 1, policy_version 610108 (0.0009) [2023-12-27 09:51:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 312492032. Throughput: 0: 10479.7, 1: 10404.6. Samples: 312479508. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:27,363][19170] Avg episode reward: [(0, '942.080'), (1, '990.370')] [2023-12-27 09:51:27,394][20347] Updated weights for policy 0, policy_version 610405 (0.0008) [2023-12-27 09:51:27,453][20347] Updated weights for policy 0, policy_version 610415 (0.0005) [2023-12-27 09:51:27,517][20347] Updated weights for policy 0, policy_version 610425 (0.0005) [2023-12-27 09:51:27,948][20381] Updated weights for policy 1, policy_version 610118 (0.0007) [2023-12-27 09:51:28,004][20381] Updated weights for policy 1, policy_version 610128 (0.0005) [2023-12-27 09:51:28,052][20347] Updated weights for policy 0, policy_version 610435 (0.0005) [2023-12-27 09:51:28,058][20381] Updated weights for policy 1, policy_version 610138 (0.0005) [2023-12-27 09:51:28,106][20347] Updated weights for policy 0, policy_version 610445 (0.0005) [2023-12-27 09:51:28,156][20347] Updated weights for policy 0, policy_version 610455 (0.0005) [2023-12-27 09:51:28,636][20381] Updated weights for policy 1, policy_version 610148 (0.0009) [2023-12-27 09:51:28,691][20381] Updated weights for policy 1, policy_version 610158 (0.0011) [2023-12-27 09:51:28,739][20381] Updated weights for policy 1, policy_version 610168 (0.0009) [2023-12-27 09:51:28,787][20347] Updated weights for policy 0, policy_version 610465 (0.0006) [2023-12-27 09:51:28,850][20347] Updated weights for policy 0, policy_version 610475 (0.0010) [2023-12-27 09:51:28,907][20347] Updated weights for policy 0, policy_version 610485 (0.0008) [2023-12-27 09:51:28,960][20347] Updated weights for policy 0, policy_version 610495 (0.0005) [2023-12-27 09:51:29,414][20381] Updated weights for policy 1, policy_version 610178 (0.0006) [2023-12-27 09:51:29,475][20381] Updated weights for policy 1, policy_version 610188 (0.0010) [2023-12-27 09:51:29,531][20381] Updated weights for policy 1, policy_version 610198 (0.0010) [2023-12-27 09:51:29,567][20347] Updated weights for policy 0, policy_version 610505 (0.0008) [2023-12-27 09:51:29,583][20381] Updated weights for policy 1, policy_version 610208 (0.0010) [2023-12-27 09:51:29,617][20347] Updated weights for policy 0, policy_version 610515 (0.0007) [2023-12-27 09:51:29,675][20347] Updated weights for policy 0, policy_version 610525 (0.0008) [2023-12-27 09:51:30,263][20381] Updated weights for policy 1, policy_version 610218 (0.0007) [2023-12-27 09:51:30,320][20381] Updated weights for policy 1, policy_version 610228 (0.0007) [2023-12-27 09:51:30,332][20347] Updated weights for policy 0, policy_version 610535 (0.0008) [2023-12-27 09:51:30,382][20381] Updated weights for policy 1, policy_version 610238 (0.0007) [2023-12-27 09:51:30,387][20347] Updated weights for policy 0, policy_version 610545 (0.0007) [2023-12-27 09:51:30,446][20347] Updated weights for policy 0, policy_version 610555 (0.0007) [2023-12-27 09:51:31,032][20347] Updated weights for policy 0, policy_version 610565 (0.0007) [2023-12-27 09:51:31,052][20381] Updated weights for policy 1, policy_version 610248 (0.0009) [2023-12-27 09:51:31,094][20347] Updated weights for policy 0, policy_version 610575 (0.0008) [2023-12-27 09:51:31,113][20381] Updated weights for policy 1, policy_version 610258 (0.0010) [2023-12-27 09:51:31,159][20347] Updated weights for policy 0, policy_version 610585 (0.0008) [2023-12-27 09:51:31,177][20381] Updated weights for policy 1, policy_version 610268 (0.0009) [2023-12-27 09:51:31,799][20347] Updated weights for policy 0, policy_version 610595 (0.0007) [2023-12-27 09:51:31,854][20347] Updated weights for policy 0, policy_version 610605 (0.0008) [2023-12-27 09:51:31,879][20381] Updated weights for policy 1, policy_version 610278 (0.0006) [2023-12-27 09:51:31,907][20347] Updated weights for policy 0, policy_version 610615 (0.0007) [2023-12-27 09:51:31,942][20381] Updated weights for policy 1, policy_version 610288 (0.0006) [2023-12-27 09:51:32,003][20381] Updated weights for policy 1, policy_version 610298 (0.0008) [2023-12-27 09:51:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 312598528. Throughput: 0: 10507.0, 1: 10353.0. Samples: 312607804. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:32,363][19170] Avg episode reward: [(0, '951.600'), (1, '961.630')] [2023-12-27 09:51:32,551][20347] Updated weights for policy 0, policy_version 610625 (0.0006) [2023-12-27 09:51:32,599][20347] Updated weights for policy 0, policy_version 610635 (0.0008) [2023-12-27 09:51:32,648][20347] Updated weights for policy 0, policy_version 610645 (0.0007) [2023-12-27 09:51:32,670][20381] Updated weights for policy 1, policy_version 610308 (0.0009) [2023-12-27 09:51:32,704][20347] Updated weights for policy 0, policy_version 610655 (0.0007) [2023-12-27 09:51:32,729][20381] Updated weights for policy 1, policy_version 610318 (0.0008) [2023-12-27 09:51:32,784][20381] Updated weights for policy 1, policy_version 610328 (0.0007) [2023-12-27 09:51:33,389][20347] Updated weights for policy 0, policy_version 610665 (0.0008) [2023-12-27 09:51:33,438][20381] Updated weights for policy 1, policy_version 610338 (0.0007) [2023-12-27 09:51:33,440][20347] Updated weights for policy 0, policy_version 610675 (0.0008) [2023-12-27 09:51:33,489][20381] Updated weights for policy 1, policy_version 610348 (0.0007) [2023-12-27 09:51:33,496][20347] Updated weights for policy 0, policy_version 610685 (0.0008) [2023-12-27 09:51:33,534][20381] Updated weights for policy 1, policy_version 610358 (0.0006) [2023-12-27 09:51:33,582][20381] Updated weights for policy 1, policy_version 610368 (0.0008) [2023-12-27 09:51:34,176][20347] Updated weights for policy 0, policy_version 610695 (0.0006) [2023-12-27 09:51:34,238][20347] Updated weights for policy 0, policy_version 610705 (0.0007) [2023-12-27 09:51:34,287][20381] Updated weights for policy 1, policy_version 610378 (0.0008) [2023-12-27 09:51:34,294][20347] Updated weights for policy 0, policy_version 610715 (0.0006) [2023-12-27 09:51:34,348][20381] Updated weights for policy 1, policy_version 610388 (0.0008) [2023-12-27 09:51:34,407][20381] Updated weights for policy 1, policy_version 610398 (0.0007) [2023-12-27 09:51:34,986][20347] Updated weights for policy 0, policy_version 610725 (0.0007) [2023-12-27 09:51:35,030][20381] Updated weights for policy 1, policy_version 610408 (0.0006) [2023-12-27 09:51:35,042][20347] Updated weights for policy 0, policy_version 610735 (0.0008) [2023-12-27 09:51:35,079][20381] Updated weights for policy 1, policy_version 610418 (0.0005) [2023-12-27 09:51:35,102][20347] Updated weights for policy 0, policy_version 610745 (0.0007) [2023-12-27 09:51:35,130][20381] Updated weights for policy 1, policy_version 610428 (0.0006) [2023-12-27 09:51:35,761][20347] Updated weights for policy 0, policy_version 610755 (0.0007) [2023-12-27 09:51:35,783][20381] Updated weights for policy 1, policy_version 610438 (0.0006) [2023-12-27 09:51:35,818][20347] Updated weights for policy 0, policy_version 610765 (0.0007) [2023-12-27 09:51:35,827][20381] Updated weights for policy 1, policy_version 610448 (0.0005) [2023-12-27 09:51:35,871][20347] Updated weights for policy 0, policy_version 610775 (0.0006) [2023-12-27 09:51:35,876][20381] Updated weights for policy 1, policy_version 610458 (0.0010) [2023-12-27 09:51:36,468][20381] Updated weights for policy 1, policy_version 610468 (0.0008) [2023-12-27 09:51:36,531][20381] Updated weights for policy 1, policy_version 610478 (0.0008) [2023-12-27 09:51:36,594][20381] Updated weights for policy 1, policy_version 610488 (0.0011) [2023-12-27 09:51:36,600][20347] Updated weights for policy 0, policy_version 610785 (0.0006) [2023-12-27 09:51:36,656][20347] Updated weights for policy 0, policy_version 610795 (0.0010) [2023-12-27 09:51:36,712][20347] Updated weights for policy 0, policy_version 610805 (0.0010) [2023-12-27 09:51:36,763][20347] Updated weights for policy 0, policy_version 610815 (0.0010) [2023-12-27 09:51:37,233][20381] Updated weights for policy 1, policy_version 610498 (0.0011) [2023-12-27 09:51:37,288][20381] Updated weights for policy 1, policy_version 610508 (0.0011) [2023-12-27 09:51:37,347][20381] Updated weights for policy 1, policy_version 610518 (0.0011) [2023-12-27 09:51:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 312696832. Throughput: 0: 10530.9, 1: 10348.4. Samples: 312670256. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:37,363][19170] Avg episode reward: [(0, '980.560'), (1, '942.380')] [2023-12-27 09:51:37,404][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000610528_156319744.pth... [2023-12-27 09:51:37,407][20381] Updated weights for policy 1, policy_version 610528 (0.0011) [2023-12-27 09:51:37,408][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000609280_156000256.pth [2023-12-27 09:51:37,416][20347] Updated weights for policy 0, policy_version 610825 (0.0006) [2023-12-27 09:51:37,475][20347] Updated weights for policy 0, policy_version 610835 (0.0007) [2023-12-27 09:51:37,530][20347] Updated weights for policy 0, policy_version 610845 (0.0007) [2023-12-27 09:51:37,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000610848_156393472.pth... [2023-12-27 09:51:37,547][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000609600_156073984.pth [2023-12-27 09:51:38,051][20381] Updated weights for policy 1, policy_version 610538 (0.0010) [2023-12-27 09:51:38,109][20381] Updated weights for policy 1, policy_version 610548 (0.0007) [2023-12-27 09:51:38,141][20347] Updated weights for policy 0, policy_version 610855 (0.0006) [2023-12-27 09:51:38,164][20381] Updated weights for policy 1, policy_version 610558 (0.0010) [2023-12-27 09:51:38,198][20347] Updated weights for policy 0, policy_version 610865 (0.0005) [2023-12-27 09:51:38,261][20347] Updated weights for policy 0, policy_version 610875 (0.0005) [2023-12-27 09:51:38,783][20381] Updated weights for policy 1, policy_version 610568 (0.0008) [2023-12-27 09:51:38,835][20381] Updated weights for policy 1, policy_version 610578 (0.0008) [2023-12-27 09:51:38,885][20381] Updated weights for policy 1, policy_version 610588 (0.0006) [2023-12-27 09:51:38,947][20347] Updated weights for policy 0, policy_version 610885 (0.0007) [2023-12-27 09:51:39,002][20347] Updated weights for policy 0, policy_version 610895 (0.0007) [2023-12-27 09:51:39,054][20347] Updated weights for policy 0, policy_version 610905 (0.0007) [2023-12-27 09:51:39,578][20381] Updated weights for policy 1, policy_version 610598 (0.0007) [2023-12-27 09:51:39,632][20381] Updated weights for policy 1, policy_version 610608 (0.0007) [2023-12-27 09:51:39,689][20381] Updated weights for policy 1, policy_version 610618 (0.0008) [2023-12-27 09:51:39,753][20347] Updated weights for policy 0, policy_version 610915 (0.0008) [2023-12-27 09:51:39,813][20347] Updated weights for policy 0, policy_version 610925 (0.0008) [2023-12-27 09:51:39,880][20347] Updated weights for policy 0, policy_version 610935 (0.0006) [2023-12-27 09:51:40,397][20381] Updated weights for policy 1, policy_version 610628 (0.0008) [2023-12-27 09:51:40,433][20292] KL-divergence is very high: 766.8347 [2023-12-27 09:51:40,457][20381] Updated weights for policy 1, policy_version 610638 (0.0010) [2023-12-27 09:51:40,509][20381] Updated weights for policy 1, policy_version 610648 (0.0010) [2023-12-27 09:51:40,556][20347] Updated weights for policy 0, policy_version 610945 (0.0007) [2023-12-27 09:51:40,612][20347] Updated weights for policy 0, policy_version 610955 (0.0009) [2023-12-27 09:51:40,668][20347] Updated weights for policy 0, policy_version 610965 (0.0007) [2023-12-27 09:51:40,727][20347] Updated weights for policy 0, policy_version 610975 (0.0005) [2023-12-27 09:51:41,195][20381] Updated weights for policy 1, policy_version 610658 (0.0010) [2023-12-27 09:51:41,256][20381] Updated weights for policy 1, policy_version 610668 (0.0010) [2023-12-27 09:51:41,315][20381] Updated weights for policy 1, policy_version 610678 (0.0011) [2023-12-27 09:51:41,328][20347] Updated weights for policy 0, policy_version 610985 (0.0007) [2023-12-27 09:51:41,384][20381] Updated weights for policy 1, policy_version 610688 (0.0010) [2023-12-27 09:51:41,393][20347] Updated weights for policy 0, policy_version 610995 (0.0008) [2023-12-27 09:51:41,452][20347] Updated weights for policy 0, policy_version 611005 (0.0007) [2023-12-27 09:51:42,127][20381] Updated weights for policy 1, policy_version 610698 (0.0008) [2023-12-27 09:51:42,145][20347] Updated weights for policy 0, policy_version 611015 (0.0008) [2023-12-27 09:51:42,188][20381] Updated weights for policy 1, policy_version 610708 (0.0008) [2023-12-27 09:51:42,208][20347] Updated weights for policy 0, policy_version 611025 (0.0008) [2023-12-27 09:51:42,241][20381] Updated weights for policy 1, policy_version 610718 (0.0007) [2023-12-27 09:51:42,278][20347] Updated weights for policy 0, policy_version 611035 (0.0008) [2023-12-27 09:51:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 312811520. Throughput: 0: 10557.1, 1: 10347.1. Samples: 312796216. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.160')] [2023-12-27 09:51:42,928][20381] Updated weights for policy 1, policy_version 610728 (0.0006) [2023-12-27 09:51:42,940][20347] Updated weights for policy 0, policy_version 611045 (0.0007) [2023-12-27 09:51:42,982][20381] Updated weights for policy 1, policy_version 610738 (0.0005) [2023-12-27 09:51:43,002][20347] Updated weights for policy 0, policy_version 611055 (0.0005) [2023-12-27 09:51:43,045][20381] Updated weights for policy 1, policy_version 610748 (0.0005) [2023-12-27 09:51:43,063][20347] Updated weights for policy 0, policy_version 611065 (0.0007) [2023-12-27 09:51:43,561][20381] Updated weights for policy 1, policy_version 610758 (0.0007) [2023-12-27 09:51:43,620][20381] Updated weights for policy 1, policy_version 610768 (0.0008) [2023-12-27 09:51:43,672][20381] Updated weights for policy 1, policy_version 610778 (0.0006) [2023-12-27 09:51:43,727][20347] Updated weights for policy 0, policy_version 611075 (0.0007) [2023-12-27 09:51:43,775][20347] Updated weights for policy 0, policy_version 611085 (0.0008) [2023-12-27 09:51:43,819][20347] Updated weights for policy 0, policy_version 611095 (0.0007) [2023-12-27 09:51:44,316][20381] Updated weights for policy 1, policy_version 610788 (0.0005) [2023-12-27 09:51:44,376][20381] Updated weights for policy 1, policy_version 610798 (0.0006) [2023-12-27 09:51:44,428][20381] Updated weights for policy 1, policy_version 610808 (0.0007) [2023-12-27 09:51:44,505][20347] Updated weights for policy 0, policy_version 611105 (0.0008) [2023-12-27 09:51:44,565][20347] Updated weights for policy 0, policy_version 611115 (0.0011) [2023-12-27 09:51:44,621][20347] Updated weights for policy 0, policy_version 611125 (0.0010) [2023-12-27 09:51:44,676][20347] Updated weights for policy 0, policy_version 611135 (0.0010) [2023-12-27 09:51:45,119][20381] Updated weights for policy 1, policy_version 610818 (0.0007) [2023-12-27 09:51:45,177][20381] Updated weights for policy 1, policy_version 610828 (0.0007) [2023-12-27 09:51:45,238][20381] Updated weights for policy 1, policy_version 610838 (0.0008) [2023-12-27 09:51:45,299][20381] Updated weights for policy 1, policy_version 610848 (0.0008) [2023-12-27 09:51:45,324][20347] Updated weights for policy 0, policy_version 611145 (0.0008) [2023-12-27 09:51:45,372][20347] Updated weights for policy 0, policy_version 611155 (0.0005) [2023-12-27 09:51:45,419][20347] Updated weights for policy 0, policy_version 611165 (0.0007) [2023-12-27 09:51:45,992][20381] Updated weights for policy 1, policy_version 610858 (0.0007) [2023-12-27 09:51:46,046][20381] Updated weights for policy 1, policy_version 610868 (0.0008) [2023-12-27 09:51:46,094][20381] Updated weights for policy 1, policy_version 610878 (0.0006) [2023-12-27 09:51:46,100][20347] Updated weights for policy 0, policy_version 611175 (0.0010) [2023-12-27 09:51:46,154][20347] Updated weights for policy 0, policy_version 611185 (0.0010) [2023-12-27 09:51:46,215][20347] Updated weights for policy 0, policy_version 611195 (0.0008) [2023-12-27 09:51:46,806][20381] Updated weights for policy 1, policy_version 610888 (0.0007) [2023-12-27 09:51:46,856][20381] Updated weights for policy 1, policy_version 610898 (0.0007) [2023-12-27 09:51:46,877][20347] Updated weights for policy 0, policy_version 611205 (0.0008) [2023-12-27 09:51:46,911][20381] Updated weights for policy 1, policy_version 610908 (0.0005) [2023-12-27 09:51:46,937][20347] Updated weights for policy 0, policy_version 611215 (0.0009) [2023-12-27 09:51:46,981][20347] Updated weights for policy 0, policy_version 611225 (0.0010) [2023-12-27 09:51:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 312909824. Throughput: 0: 10552.6, 1: 10371.5. Samples: 312920024. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.550')] [2023-12-27 09:51:47,486][20381] Updated weights for policy 1, policy_version 610918 (0.0006) [2023-12-27 09:51:47,537][20381] Updated weights for policy 1, policy_version 610928 (0.0006) [2023-12-27 09:51:47,588][20381] Updated weights for policy 1, policy_version 610938 (0.0005) [2023-12-27 09:51:47,670][20347] Updated weights for policy 0, policy_version 611235 (0.0010) [2023-12-27 09:51:47,719][20347] Updated weights for policy 0, policy_version 611245 (0.0007) [2023-12-27 09:51:47,772][20347] Updated weights for policy 0, policy_version 611255 (0.0009) [2023-12-27 09:51:48,211][20381] Updated weights for policy 1, policy_version 610948 (0.0006) [2023-12-27 09:51:48,275][20381] Updated weights for policy 1, policy_version 610958 (0.0007) [2023-12-27 09:51:48,327][20381] Updated weights for policy 1, policy_version 610968 (0.0007) [2023-12-27 09:51:48,401][20347] Updated weights for policy 0, policy_version 611265 (0.0006) [2023-12-27 09:51:48,456][20347] Updated weights for policy 0, policy_version 611275 (0.0008) [2023-12-27 09:51:48,513][20347] Updated weights for policy 0, policy_version 611285 (0.0007) [2023-12-27 09:51:48,568][20347] Updated weights for policy 0, policy_version 611295 (0.0007) [2023-12-27 09:51:49,009][20381] Updated weights for policy 1, policy_version 610978 (0.0008) [2023-12-27 09:51:49,061][20381] Updated weights for policy 1, policy_version 610988 (0.0006) [2023-12-27 09:51:49,110][20381] Updated weights for policy 1, policy_version 610998 (0.0006) [2023-12-27 09:51:49,169][20381] Updated weights for policy 1, policy_version 611008 (0.0005) [2023-12-27 09:51:49,206][20347] Updated weights for policy 0, policy_version 611305 (0.0008) [2023-12-27 09:51:49,267][20347] Updated weights for policy 0, policy_version 611315 (0.0006) [2023-12-27 09:51:49,330][20347] Updated weights for policy 0, policy_version 611325 (0.0007) [2023-12-27 09:51:49,839][20381] Updated weights for policy 1, policy_version 611018 (0.0008) [2023-12-27 09:51:49,894][20381] Updated weights for policy 1, policy_version 611028 (0.0007) [2023-12-27 09:51:49,959][20381] Updated weights for policy 1, policy_version 611038 (0.0008) [2023-12-27 09:51:50,037][20347] Updated weights for policy 0, policy_version 611335 (0.0008) [2023-12-27 09:51:50,095][20347] Updated weights for policy 0, policy_version 611345 (0.0008) [2023-12-27 09:51:50,148][20347] Updated weights for policy 0, policy_version 611355 (0.0007) [2023-12-27 09:51:50,676][20381] Updated weights for policy 1, policy_version 611048 (0.0010) [2023-12-27 09:51:50,738][20381] Updated weights for policy 1, policy_version 611058 (0.0011) [2023-12-27 09:51:50,794][20381] Updated weights for policy 1, policy_version 611068 (0.0007) [2023-12-27 09:51:50,848][20347] Updated weights for policy 0, policy_version 611365 (0.0007) [2023-12-27 09:51:50,914][20347] Updated weights for policy 0, policy_version 611375 (0.0005) [2023-12-27 09:51:50,973][20347] Updated weights for policy 0, policy_version 611385 (0.0006) [2023-12-27 09:51:51,467][20381] Updated weights for policy 1, policy_version 611078 (0.0010) [2023-12-27 09:51:51,525][20381] Updated weights for policy 1, policy_version 611088 (0.0010) [2023-12-27 09:51:51,585][20381] Updated weights for policy 1, policy_version 611098 (0.0010) [2023-12-27 09:51:51,644][20347] Updated weights for policy 0, policy_version 611395 (0.0007) [2023-12-27 09:51:51,706][20347] Updated weights for policy 0, policy_version 611405 (0.0008) [2023-12-27 09:51:51,769][20347] Updated weights for policy 0, policy_version 611415 (0.0008) [2023-12-27 09:51:52,281][20381] Updated weights for policy 1, policy_version 611108 (0.0011) [2023-12-27 09:51:52,341][20381] Updated weights for policy 1, policy_version 611118 (0.0011) [2023-12-27 09:51:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 313008128. Throughput: 0: 10569.8, 1: 10392.5. Samples: 312983212. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.680')] [2023-12-27 09:51:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000611424_156540928.pth... [2023-12-27 09:51:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000610208_156229632.pth [2023-12-27 09:51:52,398][20381] Updated weights for policy 1, policy_version 611128 (0.0011) [2023-12-27 09:51:52,441][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000611136_156475392.pth... [2023-12-27 09:51:52,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000609888_156155904.pth [2023-12-27 09:51:52,476][20347] Updated weights for policy 0, policy_version 611425 (0.0007) [2023-12-27 09:51:52,533][20347] Updated weights for policy 0, policy_version 611435 (0.0007) [2023-12-27 09:51:52,591][20347] Updated weights for policy 0, policy_version 611445 (0.0007) [2023-12-27 09:51:52,635][20347] Updated weights for policy 0, policy_version 611455 (0.0007) [2023-12-27 09:51:53,029][20381] Updated weights for policy 1, policy_version 611138 (0.0011) [2023-12-27 09:51:53,081][20381] Updated weights for policy 1, policy_version 611148 (0.0010) [2023-12-27 09:51:53,133][20381] Updated weights for policy 1, policy_version 611158 (0.0005) [2023-12-27 09:51:53,184][20381] Updated weights for policy 1, policy_version 611168 (0.0005) [2023-12-27 09:51:53,316][20347] Updated weights for policy 0, policy_version 611465 (0.0005) [2023-12-27 09:51:53,370][20347] Updated weights for policy 0, policy_version 611475 (0.0005) [2023-12-27 09:51:53,425][20347] Updated weights for policy 0, policy_version 611485 (0.0005) [2023-12-27 09:51:53,818][20381] Updated weights for policy 1, policy_version 611178 (0.0005) [2023-12-27 09:51:53,874][20381] Updated weights for policy 1, policy_version 611188 (0.0005) [2023-12-27 09:51:53,933][20381] Updated weights for policy 1, policy_version 611198 (0.0005) [2023-12-27 09:51:54,061][20347] Updated weights for policy 0, policy_version 611495 (0.0005) [2023-12-27 09:51:54,114][20347] Updated weights for policy 0, policy_version 611505 (0.0005) [2023-12-27 09:51:54,165][20347] Updated weights for policy 0, policy_version 611515 (0.0007) [2023-12-27 09:51:54,468][20381] Updated weights for policy 1, policy_version 611208 (0.0007) [2023-12-27 09:51:54,516][20381] Updated weights for policy 1, policy_version 611218 (0.0009) [2023-12-27 09:51:54,570][20381] Updated weights for policy 1, policy_version 611228 (0.0010) [2023-12-27 09:51:54,839][20347] Updated weights for policy 0, policy_version 611525 (0.0006) [2023-12-27 09:51:54,889][20347] Updated weights for policy 0, policy_version 611535 (0.0005) [2023-12-27 09:51:54,939][20347] Updated weights for policy 0, policy_version 611545 (0.0006) [2023-12-27 09:51:55,230][20381] Updated weights for policy 1, policy_version 611238 (0.0010) [2023-12-27 09:51:55,290][20381] Updated weights for policy 1, policy_version 611248 (0.0010) [2023-12-27 09:51:55,349][20381] Updated weights for policy 1, policy_version 611258 (0.0010) [2023-12-27 09:51:55,564][20347] Updated weights for policy 0, policy_version 611555 (0.0007) [2023-12-27 09:51:55,615][20347] Updated weights for policy 0, policy_version 611565 (0.0008) [2023-12-27 09:51:55,665][20347] Updated weights for policy 0, policy_version 611575 (0.0008) [2023-12-27 09:51:55,992][20381] Updated weights for policy 1, policy_version 611268 (0.0009) [2023-12-27 09:51:56,066][20381] Updated weights for policy 1, policy_version 611278 (0.0006) [2023-12-27 09:51:56,134][20381] Updated weights for policy 1, policy_version 611288 (0.0007) [2023-12-27 09:51:56,280][20347] Updated weights for policy 0, policy_version 611585 (0.0007) [2023-12-27 09:51:56,335][20347] Updated weights for policy 0, policy_version 611595 (0.0007) [2023-12-27 09:51:56,395][20347] Updated weights for policy 0, policy_version 611605 (0.0007) [2023-12-27 09:51:56,450][20347] Updated weights for policy 0, policy_version 611615 (0.0007) [2023-12-27 09:51:56,784][20381] Updated weights for policy 1, policy_version 611298 (0.0009) [2023-12-27 09:51:56,835][20381] Updated weights for policy 1, policy_version 611308 (0.0011) [2023-12-27 09:51:56,886][20381] Updated weights for policy 1, policy_version 611318 (0.0010) [2023-12-27 09:51:56,930][20381] Updated weights for policy 1, policy_version 611328 (0.0010) [2023-12-27 09:51:57,051][20347] Updated weights for policy 0, policy_version 611625 (0.0007) [2023-12-27 09:51:57,098][20347] Updated weights for policy 0, policy_version 611635 (0.0007) [2023-12-27 09:51:57,146][20347] Updated weights for policy 0, policy_version 611645 (0.0007) [2023-12-27 09:51:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 313122816. Throughput: 0: 10556.7, 1: 10490.6. Samples: 313109300. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:51:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.840')] [2023-12-27 09:51:57,602][20381] Updated weights for policy 1, policy_version 611338 (0.0010) [2023-12-27 09:51:57,653][20381] Updated weights for policy 1, policy_version 611348 (0.0010) [2023-12-27 09:51:57,706][20381] Updated weights for policy 1, policy_version 611358 (0.0010) [2023-12-27 09:51:57,837][20347] Updated weights for policy 0, policy_version 611655 (0.0008) [2023-12-27 09:51:57,899][20347] Updated weights for policy 0, policy_version 611665 (0.0007) [2023-12-27 09:51:57,958][20347] Updated weights for policy 0, policy_version 611675 (0.0008) [2023-12-27 09:51:58,306][20381] Updated weights for policy 1, policy_version 611368 (0.0009) [2023-12-27 09:51:58,372][20381] Updated weights for policy 1, policy_version 611378 (0.0009) [2023-12-27 09:51:58,432][20381] Updated weights for policy 1, policy_version 611388 (0.0009) [2023-12-27 09:51:58,799][20347] Updated weights for policy 0, policy_version 611685 (0.0008) [2023-12-27 09:51:58,856][20347] Updated weights for policy 0, policy_version 611695 (0.0009) [2023-12-27 09:51:58,918][20347] Updated weights for policy 0, policy_version 611705 (0.0009) [2023-12-27 09:51:59,256][20381] Updated weights for policy 1, policy_version 611398 (0.0007) [2023-12-27 09:51:59,315][20381] Updated weights for policy 1, policy_version 611408 (0.0005) [2023-12-27 09:51:59,370][20381] Updated weights for policy 1, policy_version 611418 (0.0007) [2023-12-27 09:51:59,597][20347] Updated weights for policy 0, policy_version 611715 (0.0008) [2023-12-27 09:51:59,649][20347] Updated weights for policy 0, policy_version 611725 (0.0005) [2023-12-27 09:51:59,708][20347] Updated weights for policy 0, policy_version 611735 (0.0007) [2023-12-27 09:52:00,040][20381] Updated weights for policy 1, policy_version 611428 (0.0007) [2023-12-27 09:52:00,090][20381] Updated weights for policy 1, policy_version 611438 (0.0008) [2023-12-27 09:52:00,139][20381] Updated weights for policy 1, policy_version 611448 (0.0008) [2023-12-27 09:52:00,388][20347] Updated weights for policy 0, policy_version 611745 (0.0006) [2023-12-27 09:52:00,446][20347] Updated weights for policy 0, policy_version 611755 (0.0008) [2023-12-27 09:52:00,503][20347] Updated weights for policy 0, policy_version 611765 (0.0007) [2023-12-27 09:52:00,552][20347] Updated weights for policy 0, policy_version 611775 (0.0007) [2023-12-27 09:52:00,794][20381] Updated weights for policy 1, policy_version 611458 (0.0007) [2023-12-27 09:52:00,845][20381] Updated weights for policy 1, policy_version 611468 (0.0005) [2023-12-27 09:52:00,897][20381] Updated weights for policy 1, policy_version 611478 (0.0009) [2023-12-27 09:52:00,948][20381] Updated weights for policy 1, policy_version 611488 (0.0010) [2023-12-27 09:52:01,279][20347] Updated weights for policy 0, policy_version 611785 (0.0008) [2023-12-27 09:52:01,326][20347] Updated weights for policy 0, policy_version 611795 (0.0006) [2023-12-27 09:52:01,386][20347] Updated weights for policy 0, policy_version 611805 (0.0008) [2023-12-27 09:52:01,669][20381] Updated weights for policy 1, policy_version 611498 (0.0007) [2023-12-27 09:52:01,730][20381] Updated weights for policy 1, policy_version 611508 (0.0008) [2023-12-27 09:52:01,783][20381] Updated weights for policy 1, policy_version 611518 (0.0008) [2023-12-27 09:52:02,073][20347] Updated weights for policy 0, policy_version 611815 (0.0008) [2023-12-27 09:52:02,135][20347] Updated weights for policy 0, policy_version 611825 (0.0008) [2023-12-27 09:52:02,197][20347] Updated weights for policy 0, policy_version 611835 (0.0007) [2023-12-27 09:52:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 313221120. Throughput: 0: 10491.3, 1: 10407.3. Samples: 313230404. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.770')] [2023-12-27 09:52:02,499][20381] Updated weights for policy 1, policy_version 611528 (0.0008) [2023-12-27 09:52:02,545][20381] Updated weights for policy 1, policy_version 611538 (0.0007) [2023-12-27 09:52:02,590][20381] Updated weights for policy 1, policy_version 611548 (0.0007) [2023-12-27 09:52:02,894][20347] Updated weights for policy 0, policy_version 611845 (0.0007) [2023-12-27 09:52:02,942][20347] Updated weights for policy 0, policy_version 611855 (0.0009) [2023-12-27 09:52:02,986][20347] Updated weights for policy 0, policy_version 611865 (0.0010) [2023-12-27 09:52:03,236][20381] Updated weights for policy 1, policy_version 611558 (0.0009) [2023-12-27 09:52:03,283][20381] Updated weights for policy 1, policy_version 611568 (0.0010) [2023-12-27 09:52:03,335][20381] Updated weights for policy 1, policy_version 611578 (0.0010) [2023-12-27 09:52:03,571][20347] Updated weights for policy 0, policy_version 611875 (0.0009) [2023-12-27 09:52:03,627][20347] Updated weights for policy 0, policy_version 611885 (0.0006) [2023-12-27 09:52:03,678][20347] Updated weights for policy 0, policy_version 611895 (0.0007) [2023-12-27 09:52:03,986][20381] Updated weights for policy 1, policy_version 611588 (0.0008) [2023-12-27 09:52:04,047][20381] Updated weights for policy 1, policy_version 611598 (0.0006) [2023-12-27 09:52:04,101][20381] Updated weights for policy 1, policy_version 611608 (0.0006) [2023-12-27 09:52:04,333][20347] Updated weights for policy 0, policy_version 611905 (0.0006) [2023-12-27 09:52:04,392][20347] Updated weights for policy 0, policy_version 611915 (0.0010) [2023-12-27 09:52:04,445][20347] Updated weights for policy 0, policy_version 611925 (0.0010) [2023-12-27 09:52:04,496][20347] Updated weights for policy 0, policy_version 611935 (0.0010) [2023-12-27 09:52:04,762][20381] Updated weights for policy 1, policy_version 611618 (0.0008) [2023-12-27 09:52:04,813][20381] Updated weights for policy 1, policy_version 611628 (0.0007) [2023-12-27 09:52:04,818][20292] KL-divergence is very high: 185.6427 [2023-12-27 09:52:04,848][20292] KL-divergence is very high: 529.9238 [2023-12-27 09:52:04,861][20292] KL-divergence is very high: 378.3022 [2023-12-27 09:52:04,874][20292] KL-divergence is very high: 152.2066 [2023-12-27 09:52:04,880][20381] Updated weights for policy 1, policy_version 611638 (0.0007) [2023-12-27 09:52:04,910][20292] KL-divergence is very high: 153.5382 [2023-12-27 09:52:04,923][20292] KL-divergence is very high: 153.6570 [2023-12-27 09:52:04,941][20381] Updated weights for policy 1, policy_version 611648 (0.0007) [2023-12-27 09:52:05,160][20347] Updated weights for policy 0, policy_version 611945 (0.0010) [2023-12-27 09:52:05,211][20347] Updated weights for policy 0, policy_version 611955 (0.0010) [2023-12-27 09:52:05,276][20347] Updated weights for policy 0, policy_version 611965 (0.0010) [2023-12-27 09:52:05,575][20381] Updated weights for policy 1, policy_version 611658 (0.0006) [2023-12-27 09:52:05,638][20381] Updated weights for policy 1, policy_version 611668 (0.0006) [2023-12-27 09:52:05,687][20381] Updated weights for policy 1, policy_version 611678 (0.0005) [2023-12-27 09:52:05,956][20347] Updated weights for policy 0, policy_version 611975 (0.0010) [2023-12-27 09:52:06,004][20347] Updated weights for policy 0, policy_version 611985 (0.0010) [2023-12-27 09:52:06,048][20347] Updated weights for policy 0, policy_version 611995 (0.0010) [2023-12-27 09:52:06,351][20381] Updated weights for policy 1, policy_version 611688 (0.0008) [2023-12-27 09:52:06,409][20381] Updated weights for policy 1, policy_version 611698 (0.0008) [2023-12-27 09:52:06,475][20381] Updated weights for policy 1, policy_version 611708 (0.0008) [2023-12-27 09:52:06,764][20347] Updated weights for policy 0, policy_version 612005 (0.0010) [2023-12-27 09:52:06,812][20347] Updated weights for policy 0, policy_version 612015 (0.0010) [2023-12-27 09:52:06,873][20347] Updated weights for policy 0, policy_version 612025 (0.0010) [2023-12-27 09:52:07,164][20381] Updated weights for policy 1, policy_version 611718 (0.0008) [2023-12-27 09:52:07,217][20381] Updated weights for policy 1, policy_version 611728 (0.0007) [2023-12-27 09:52:07,267][20381] Updated weights for policy 1, policy_version 611738 (0.0007) [2023-12-27 09:52:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 313327616. Throughput: 0: 10448.9, 1: 10417.2. Samples: 313293728. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:07,363][19170] Avg episode reward: [(0, '990.490'), (1, '903.250')] [2023-12-27 09:52:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000611744_156631040.pth... [2023-12-27 09:52:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000612032_156696576.pth... [2023-12-27 09:52:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000610528_156319744.pth [2023-12-27 09:52:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000610848_156393472.pth [2023-12-27 09:52:07,513][20347] Updated weights for policy 0, policy_version 612035 (0.0010) [2023-12-27 09:52:07,574][20347] Updated weights for policy 0, policy_version 612045 (0.0010) [2023-12-27 09:52:07,630][20347] Updated weights for policy 0, policy_version 612055 (0.0010) [2023-12-27 09:52:07,956][20381] Updated weights for policy 1, policy_version 611748 (0.0007) [2023-12-27 09:52:08,017][20381] Updated weights for policy 1, policy_version 611758 (0.0008) [2023-12-27 09:52:08,081][20381] Updated weights for policy 1, policy_version 611768 (0.0007) [2023-12-27 09:52:08,262][20347] Updated weights for policy 0, policy_version 612065 (0.0010) [2023-12-27 09:52:08,326][20347] Updated weights for policy 0, policy_version 612075 (0.0005) [2023-12-27 09:52:08,388][20347] Updated weights for policy 0, policy_version 612085 (0.0007) [2023-12-27 09:52:08,451][20347] Updated weights for policy 0, policy_version 612095 (0.0008) [2023-12-27 09:52:08,724][20381] Updated weights for policy 1, policy_version 611778 (0.0006) [2023-12-27 09:52:08,769][20381] Updated weights for policy 1, policy_version 611788 (0.0010) [2023-12-27 09:52:08,830][20381] Updated weights for policy 1, policy_version 611798 (0.0010) [2023-12-27 09:52:08,881][20381] Updated weights for policy 1, policy_version 611808 (0.0010) [2023-12-27 09:52:09,034][20347] Updated weights for policy 0, policy_version 612105 (0.0006) [2023-12-27 09:52:09,081][20347] Updated weights for policy 0, policy_version 612115 (0.0008) [2023-12-27 09:52:09,129][20347] Updated weights for policy 0, policy_version 612125 (0.0010) [2023-12-27 09:52:09,599][20381] Updated weights for policy 1, policy_version 611818 (0.0009) [2023-12-27 09:52:09,650][20381] Updated weights for policy 1, policy_version 611828 (0.0010) [2023-12-27 09:52:09,706][20381] Updated weights for policy 1, policy_version 611838 (0.0011) [2023-12-27 09:52:09,847][20347] Updated weights for policy 0, policy_version 612135 (0.0009) [2023-12-27 09:52:09,909][20347] Updated weights for policy 0, policy_version 612145 (0.0007) [2023-12-27 09:52:09,971][20347] Updated weights for policy 0, policy_version 612155 (0.0009) [2023-12-27 09:52:10,434][20381] Updated weights for policy 1, policy_version 611848 (0.0008) [2023-12-27 09:52:10,467][20292] KL-divergence is very high: 604.8801 [2023-12-27 09:52:10,496][20381] Updated weights for policy 1, policy_version 611858 (0.0009) [2023-12-27 09:52:10,512][20292] KL-divergence is very high: 133.6214 [2023-12-27 09:52:10,552][20381] Updated weights for policy 1, policy_version 611868 (0.0008) [2023-12-27 09:52:10,659][20347] Updated weights for policy 0, policy_version 612165 (0.0006) [2023-12-27 09:52:10,720][20347] Updated weights for policy 0, policy_version 612175 (0.0008) [2023-12-27 09:52:10,777][20347] Updated weights for policy 0, policy_version 612185 (0.0008) [2023-12-27 09:52:11,259][20381] Updated weights for policy 1, policy_version 611878 (0.0008) [2023-12-27 09:52:11,309][20381] Updated weights for policy 1, policy_version 611888 (0.0007) [2023-12-27 09:52:11,377][20381] Updated weights for policy 1, policy_version 611898 (0.0009) [2023-12-27 09:52:11,479][20347] Updated weights for policy 0, policy_version 612195 (0.0008) [2023-12-27 09:52:11,537][20347] Updated weights for policy 0, policy_version 612205 (0.0008) [2023-12-27 09:52:11,598][20347] Updated weights for policy 0, policy_version 612215 (0.0008) [2023-12-27 09:52:12,104][20381] Updated weights for policy 1, policy_version 611908 (0.0009) [2023-12-27 09:52:12,155][20381] Updated weights for policy 1, policy_version 611918 (0.0007) [2023-12-27 09:52:12,212][20381] Updated weights for policy 1, policy_version 611928 (0.0008) [2023-12-27 09:52:12,308][20347] Updated weights for policy 0, policy_version 612225 (0.0009) [2023-12-27 09:52:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 313425920. Throughput: 0: 10424.3, 1: 10415.5. Samples: 313417304. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:12,363][19170] Avg episode reward: [(0, '990.490'), (1, '951.770')] [2023-12-27 09:52:12,372][20347] Updated weights for policy 0, policy_version 612235 (0.0010) [2023-12-27 09:52:12,427][20347] Updated weights for policy 0, policy_version 612245 (0.0010) [2023-12-27 09:52:12,485][20347] Updated weights for policy 0, policy_version 612255 (0.0010) [2023-12-27 09:52:12,902][20381] Updated weights for policy 1, policy_version 611938 (0.0007) [2023-12-27 09:52:12,945][20381] Updated weights for policy 1, policy_version 611948 (0.0009) [2023-12-27 09:52:12,995][20381] Updated weights for policy 1, policy_version 611958 (0.0008) [2023-12-27 09:52:13,049][20381] Updated weights for policy 1, policy_version 611968 (0.0009) [2023-12-27 09:52:13,151][20347] Updated weights for policy 0, policy_version 612265 (0.0010) [2023-12-27 09:52:13,208][20347] Updated weights for policy 0, policy_version 612275 (0.0010) [2023-12-27 09:52:13,275][20347] Updated weights for policy 0, policy_version 612285 (0.0010) [2023-12-27 09:52:13,701][20381] Updated weights for policy 1, policy_version 611978 (0.0009) [2023-12-27 09:52:13,754][20381] Updated weights for policy 1, policy_version 611988 (0.0007) [2023-12-27 09:52:13,798][20381] Updated weights for policy 1, policy_version 611998 (0.0008) [2023-12-27 09:52:13,836][20347] Updated weights for policy 0, policy_version 612295 (0.0007) [2023-12-27 09:52:13,886][20347] Updated weights for policy 0, policy_version 612305 (0.0007) [2023-12-27 09:52:13,944][20347] Updated weights for policy 0, policy_version 612315 (0.0005) [2023-12-27 09:52:14,445][20381] Updated weights for policy 1, policy_version 612008 (0.0006) [2023-12-27 09:52:14,510][20381] Updated weights for policy 1, policy_version 612018 (0.0007) [2023-12-27 09:52:14,576][20381] Updated weights for policy 1, policy_version 612028 (0.0006) [2023-12-27 09:52:14,598][20347] Updated weights for policy 0, policy_version 612325 (0.0008) [2023-12-27 09:52:14,647][20347] Updated weights for policy 0, policy_version 612335 (0.0007) [2023-12-27 09:52:14,702][20347] Updated weights for policy 0, policy_version 612345 (0.0007) [2023-12-27 09:52:15,232][20381] Updated weights for policy 1, policy_version 612038 (0.0010) [2023-12-27 09:52:15,291][20381] Updated weights for policy 1, policy_version 612048 (0.0007) [2023-12-27 09:52:15,347][20381] Updated weights for policy 1, policy_version 612058 (0.0010) [2023-12-27 09:52:15,385][20347] Updated weights for policy 0, policy_version 612355 (0.0007) [2023-12-27 09:52:15,448][20347] Updated weights for policy 0, policy_version 612365 (0.0007) [2023-12-27 09:52:15,505][20347] Updated weights for policy 0, policy_version 612375 (0.0007) [2023-12-27 09:52:15,934][20381] Updated weights for policy 1, policy_version 612068 (0.0008) [2023-12-27 09:52:15,991][20381] Updated weights for policy 1, policy_version 612078 (0.0008) [2023-12-27 09:52:16,038][20381] Updated weights for policy 1, policy_version 612088 (0.0011) [2023-12-27 09:52:16,171][20347] Updated weights for policy 0, policy_version 612385 (0.0008) [2023-12-27 09:52:16,236][20347] Updated weights for policy 0, policy_version 612395 (0.0008) [2023-12-27 09:52:16,298][20347] Updated weights for policy 0, policy_version 612405 (0.0008) [2023-12-27 09:52:16,362][20347] Updated weights for policy 0, policy_version 612415 (0.0010) [2023-12-27 09:52:16,663][20381] Updated weights for policy 1, policy_version 612098 (0.0010) [2023-12-27 09:52:16,717][20381] Updated weights for policy 1, policy_version 612108 (0.0008) [2023-12-27 09:52:16,764][20381] Updated weights for policy 1, policy_version 612118 (0.0010) [2023-12-27 09:52:16,818][20381] Updated weights for policy 1, policy_version 612128 (0.0010) [2023-12-27 09:52:17,028][20347] Updated weights for policy 0, policy_version 612425 (0.0006) [2023-12-27 09:52:17,084][20347] Updated weights for policy 0, policy_version 612435 (0.0005) [2023-12-27 09:52:17,142][20347] Updated weights for policy 0, policy_version 612445 (0.0008) [2023-12-27 09:52:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 313532416. Throughput: 0: 10359.5, 1: 10422.2. Samples: 313542980. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:17,363][19170] Avg episode reward: [(0, '980.680'), (1, '971.040')] [2023-12-27 09:52:17,475][20381] Updated weights for policy 1, policy_version 612138 (0.0008) [2023-12-27 09:52:17,529][20381] Updated weights for policy 1, policy_version 612148 (0.0007) [2023-12-27 09:52:17,587][20381] Updated weights for policy 1, policy_version 612158 (0.0007) [2023-12-27 09:52:17,772][20347] Updated weights for policy 0, policy_version 612455 (0.0008) [2023-12-27 09:52:17,822][20347] Updated weights for policy 0, policy_version 612465 (0.0007) [2023-12-27 09:52:17,881][20347] Updated weights for policy 0, policy_version 612475 (0.0008) [2023-12-27 09:52:18,199][20381] Updated weights for policy 1, policy_version 612168 (0.0007) [2023-12-27 09:52:18,248][20381] Updated weights for policy 1, policy_version 612178 (0.0007) [2023-12-27 09:52:18,306][20381] Updated weights for policy 1, policy_version 612188 (0.0008) [2023-12-27 09:52:18,538][20347] Updated weights for policy 0, policy_version 612485 (0.0007) [2023-12-27 09:52:18,601][20347] Updated weights for policy 0, policy_version 612495 (0.0005) [2023-12-27 09:52:18,656][20347] Updated weights for policy 0, policy_version 612505 (0.0009) [2023-12-27 09:52:19,000][20381] Updated weights for policy 1, policy_version 612198 (0.0010) [2023-12-27 09:52:19,060][20381] Updated weights for policy 1, policy_version 612208 (0.0011) [2023-12-27 09:52:19,123][20381] Updated weights for policy 1, policy_version 612218 (0.0011) [2023-12-27 09:52:19,336][20347] Updated weights for policy 0, policy_version 612515 (0.0010) [2023-12-27 09:52:19,393][20347] Updated weights for policy 0, policy_version 612525 (0.0008) [2023-12-27 09:52:19,455][20347] Updated weights for policy 0, policy_version 612535 (0.0006) [2023-12-27 09:52:19,821][20381] Updated weights for policy 1, policy_version 612228 (0.0009) [2023-12-27 09:52:19,885][20381] Updated weights for policy 1, policy_version 612238 (0.0011) [2023-12-27 09:52:19,949][20381] Updated weights for policy 1, policy_version 612248 (0.0011) [2023-12-27 09:52:20,153][20347] Updated weights for policy 0, policy_version 612545 (0.0007) [2023-12-27 09:52:20,213][20347] Updated weights for policy 0, policy_version 612555 (0.0007) [2023-12-27 09:52:20,274][20347] Updated weights for policy 0, policy_version 612565 (0.0007) [2023-12-27 09:52:20,333][20347] Updated weights for policy 0, policy_version 612575 (0.0007) [2023-12-27 09:52:20,662][20381] Updated weights for policy 1, policy_version 612258 (0.0011) [2023-12-27 09:52:20,725][20381] Updated weights for policy 1, policy_version 612268 (0.0011) [2023-12-27 09:52:20,788][20381] Updated weights for policy 1, policy_version 612278 (0.0010) [2023-12-27 09:52:20,854][20381] Updated weights for policy 1, policy_version 612288 (0.0011) [2023-12-27 09:52:21,047][20347] Updated weights for policy 0, policy_version 612585 (0.0007) [2023-12-27 09:52:21,108][20347] Updated weights for policy 0, policy_version 612595 (0.0009) [2023-12-27 09:52:21,176][20347] Updated weights for policy 0, policy_version 612605 (0.0008) [2023-12-27 09:52:21,516][20381] Updated weights for policy 1, policy_version 612298 (0.0007) [2023-12-27 09:52:21,583][20381] Updated weights for policy 1, policy_version 612308 (0.0008) [2023-12-27 09:52:21,650][20381] Updated weights for policy 1, policy_version 612318 (0.0009) [2023-12-27 09:52:21,875][20347] Updated weights for policy 0, policy_version 612615 (0.0008) [2023-12-27 09:52:21,934][20347] Updated weights for policy 0, policy_version 612625 (0.0007) [2023-12-27 09:52:21,997][20347] Updated weights for policy 0, policy_version 612635 (0.0007) [2023-12-27 09:52:22,336][20381] Updated weights for policy 1, policy_version 612328 (0.0008) [2023-12-27 09:52:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 313630720. Throughput: 0: 10353.1, 1: 10401.7. Samples: 313604224. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:22,363][19170] Avg episode reward: [(0, '961.410'), (1, '990.320')] [2023-12-27 09:52:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000612640_156852224.pth... [2023-12-27 09:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000611424_156540928.pth [2023-12-27 09:52:22,398][20381] Updated weights for policy 1, policy_version 612338 (0.0007) [2023-12-27 09:52:22,448][20381] Updated weights for policy 1, policy_version 612348 (0.0008) [2023-12-27 09:52:22,473][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000612352_156786688.pth... [2023-12-27 09:52:22,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000611136_156475392.pth [2023-12-27 09:52:22,661][20347] Updated weights for policy 0, policy_version 612645 (0.0007) [2023-12-27 09:52:22,717][20347] Updated weights for policy 0, policy_version 612655 (0.0008) [2023-12-27 09:52:22,769][20347] Updated weights for policy 0, policy_version 612665 (0.0007) [2023-12-27 09:52:23,177][20381] Updated weights for policy 1, policy_version 612358 (0.0008) [2023-12-27 09:52:23,233][20381] Updated weights for policy 1, policy_version 612368 (0.0008) [2023-12-27 09:52:23,293][20381] Updated weights for policy 1, policy_version 612378 (0.0008) [2023-12-27 09:52:23,458][20347] Updated weights for policy 0, policy_version 612675 (0.0008) [2023-12-27 09:52:23,520][20347] Updated weights for policy 0, policy_version 612685 (0.0008) [2023-12-27 09:52:23,566][20347] Updated weights for policy 0, policy_version 612695 (0.0008) [2023-12-27 09:52:23,986][20381] Updated weights for policy 1, policy_version 612388 (0.0008) [2023-12-27 09:52:24,035][20381] Updated weights for policy 1, policy_version 612398 (0.0008) [2023-12-27 09:52:24,088][20381] Updated weights for policy 1, policy_version 612408 (0.0008) [2023-12-27 09:52:24,262][20347] Updated weights for policy 0, policy_version 612705 (0.0007) [2023-12-27 09:52:24,311][20347] Updated weights for policy 0, policy_version 612715 (0.0008) [2023-12-27 09:52:24,360][20347] Updated weights for policy 0, policy_version 612725 (0.0007) [2023-12-27 09:52:24,411][20347] Updated weights for policy 0, policy_version 612735 (0.0007) [2023-12-27 09:52:24,789][20381] Updated weights for policy 1, policy_version 612418 (0.0008) [2023-12-27 09:52:24,837][20381] Updated weights for policy 1, policy_version 612428 (0.0008) [2023-12-27 09:52:24,886][20381] Updated weights for policy 1, policy_version 612438 (0.0008) [2023-12-27 09:52:24,931][20381] Updated weights for policy 1, policy_version 612448 (0.0007) [2023-12-27 09:52:25,103][20347] Updated weights for policy 0, policy_version 612745 (0.0007) [2023-12-27 09:52:25,147][20347] Updated weights for policy 0, policy_version 612755 (0.0007) [2023-12-27 09:52:25,192][20347] Updated weights for policy 0, policy_version 612765 (0.0007) [2023-12-27 09:52:25,652][20381] Updated weights for policy 1, policy_version 612458 (0.0008) [2023-12-27 09:52:25,711][20381] Updated weights for policy 1, policy_version 612468 (0.0007) [2023-12-27 09:52:25,757][20381] Updated weights for policy 1, policy_version 612478 (0.0008) [2023-12-27 09:52:25,900][20347] Updated weights for policy 0, policy_version 612775 (0.0008) [2023-12-27 09:52:25,942][20291] KL-divergence is very high: 132.4793 [2023-12-27 09:52:25,948][20291] KL-divergence is very high: 109.9332 [2023-12-27 09:52:25,952][20347] Updated weights for policy 0, policy_version 612785 (0.0007) [2023-12-27 09:52:26,001][20347] Updated weights for policy 0, policy_version 612795 (0.0007) [2023-12-27 09:52:26,445][20381] Updated weights for policy 1, policy_version 612488 (0.0006) [2023-12-27 09:52:26,511][20381] Updated weights for policy 1, policy_version 612498 (0.0007) [2023-12-27 09:52:26,560][20381] Updated weights for policy 1, policy_version 612508 (0.0010) [2023-12-27 09:52:26,701][20347] Updated weights for policy 0, policy_version 612805 (0.0008) [2023-12-27 09:52:26,759][20347] Updated weights for policy 0, policy_version 612815 (0.0008) [2023-12-27 09:52:26,815][20347] Updated weights for policy 0, policy_version 612825 (0.0007) [2023-12-27 09:52:26,817][20291] KL-divergence is very high: 656.4594 [2023-12-27 09:52:27,198][20381] Updated weights for policy 1, policy_version 612518 (0.0011) [2023-12-27 09:52:27,255][20381] Updated weights for policy 1, policy_version 612528 (0.0010) [2023-12-27 09:52:27,305][20381] Updated weights for policy 1, policy_version 612538 (0.0010) [2023-12-27 09:52:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 313737216. Throughput: 0: 10282.0, 1: 10322.8. Samples: 313723428. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 09:52:27,363][19170] Avg episode reward: [(0, '865.980'), (1, '1000.000')] [2023-12-27 09:52:27,449][20291] KL-divergence is very high: 336.9489 [2023-12-27 09:52:27,455][20347] Updated weights for policy 0, policy_version 612835 (0.0007) [2023-12-27 09:52:27,510][20291] KL-divergence is very high: 125.3921 [2023-12-27 09:52:27,513][20347] Updated weights for policy 0, policy_version 612845 (0.0005) [2023-12-27 09:52:27,515][20291] KL-divergence is very high: 198.1493 [2023-12-27 09:52:27,542][20291] KL-divergence is very high: 104.7566 [2023-12-27 09:52:27,561][20291] KL-divergence is very high: 123.2630 [2023-12-27 09:52:27,573][20347] Updated weights for policy 0, policy_version 612855 (0.0005) [2023-12-27 09:52:27,592][20291] KL-divergence is very high: 149.0728 [2023-12-27 09:52:27,985][20381] Updated weights for policy 1, policy_version 612548 (0.0011) [2023-12-27 09:52:28,043][20381] Updated weights for policy 1, policy_version 612558 (0.0010) [2023-12-27 09:52:28,094][20381] Updated weights for policy 1, policy_version 612568 (0.0010) [2023-12-27 09:52:28,111][20347] Updated weights for policy 0, policy_version 612865 (0.0005) [2023-12-27 09:52:28,166][20347] Updated weights for policy 0, policy_version 612875 (0.0006) [2023-12-27 09:52:28,217][20347] Updated weights for policy 0, policy_version 612885 (0.0006) [2023-12-27 09:52:28,266][20347] Updated weights for policy 0, policy_version 612895 (0.0005) [2023-12-27 09:52:28,759][20381] Updated weights for policy 1, policy_version 612578 (0.0010) [2023-12-27 09:52:28,817][20381] Updated weights for policy 1, policy_version 612588 (0.0011) [2023-12-27 09:52:28,876][20381] Updated weights for policy 1, policy_version 612598 (0.0010) [2023-12-27 09:52:28,903][20347] Updated weights for policy 0, policy_version 612905 (0.0007) [2023-12-27 09:52:28,924][20381] Updated weights for policy 1, policy_version 612608 (0.0010) [2023-12-27 09:52:28,964][20347] Updated weights for policy 0, policy_version 612915 (0.0008) [2023-12-27 09:52:29,016][20347] Updated weights for policy 0, policy_version 612925 (0.0006) [2023-12-27 09:52:29,619][20381] Updated weights for policy 1, policy_version 612618 (0.0006) [2023-12-27 09:52:29,665][20381] Updated weights for policy 1, policy_version 612628 (0.0005) [2023-12-27 09:52:29,685][20347] Updated weights for policy 0, policy_version 612935 (0.0009) [2023-12-27 09:52:29,716][20381] Updated weights for policy 1, policy_version 612638 (0.0006) [2023-12-27 09:52:29,746][20347] Updated weights for policy 0, policy_version 612945 (0.0010) [2023-12-27 09:52:29,801][20347] Updated weights for policy 0, policy_version 612955 (0.0010) [2023-12-27 09:52:30,388][20381] Updated weights for policy 1, policy_version 612648 (0.0006) [2023-12-27 09:52:30,449][20381] Updated weights for policy 1, policy_version 612658 (0.0009) [2023-12-27 09:52:30,468][20347] Updated weights for policy 0, policy_version 612965 (0.0010) [2023-12-27 09:52:30,504][20381] Updated weights for policy 1, policy_version 612668 (0.0010) [2023-12-27 09:52:30,522][20347] Updated weights for policy 0, policy_version 612975 (0.0010) [2023-12-27 09:52:30,577][20347] Updated weights for policy 0, policy_version 612985 (0.0009) [2023-12-27 09:52:31,089][20381] Updated weights for policy 1, policy_version 612678 (0.0008) [2023-12-27 09:52:31,149][20381] Updated weights for policy 1, policy_version 612688 (0.0009) [2023-12-27 09:52:31,205][20381] Updated weights for policy 1, policy_version 612698 (0.0007) [2023-12-27 09:52:31,223][20347] Updated weights for policy 0, policy_version 612995 (0.0010) [2023-12-27 09:52:31,279][20347] Updated weights for policy 0, policy_version 613005 (0.0011) [2023-12-27 09:52:31,337][20347] Updated weights for policy 0, policy_version 613015 (0.0010) [2023-12-27 09:52:31,863][20381] Updated weights for policy 1, policy_version 612708 (0.0007) [2023-12-27 09:52:31,922][20381] Updated weights for policy 1, policy_version 612718 (0.0006) [2023-12-27 09:52:31,981][20381] Updated weights for policy 1, policy_version 612728 (0.0008) [2023-12-27 09:52:32,036][20347] Updated weights for policy 0, policy_version 613025 (0.0010) [2023-12-27 09:52:32,094][20347] Updated weights for policy 0, policy_version 613035 (0.0010) [2023-12-27 09:52:32,147][20347] Updated weights for policy 0, policy_version 613045 (0.0007) [2023-12-27 09:52:32,203][20347] Updated weights for policy 0, policy_version 613055 (0.0005) [2023-12-27 09:52:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 313843712. Throughput: 0: 10329.9, 1: 10344.4. Samples: 313850372. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:32,363][19170] Avg episode reward: [(0, '856.020'), (1, '990.340')] [2023-12-27 09:52:32,709][20381] Updated weights for policy 1, policy_version 612738 (0.0008) [2023-12-27 09:52:32,767][20381] Updated weights for policy 1, policy_version 612748 (0.0010) [2023-12-27 09:52:32,783][20347] Updated weights for policy 0, policy_version 613065 (0.0007) [2023-12-27 09:52:32,825][20381] Updated weights for policy 1, policy_version 612758 (0.0010) [2023-12-27 09:52:32,835][20347] Updated weights for policy 0, policy_version 613075 (0.0005) [2023-12-27 09:52:32,879][20381] Updated weights for policy 1, policy_version 612768 (0.0007) [2023-12-27 09:52:32,884][20347] Updated weights for policy 0, policy_version 613085 (0.0005) [2023-12-27 09:52:33,470][20381] Updated weights for policy 1, policy_version 612778 (0.0010) [2023-12-27 09:52:33,523][20347] Updated weights for policy 0, policy_version 613095 (0.0009) [2023-12-27 09:52:33,525][20381] Updated weights for policy 1, policy_version 612788 (0.0006) [2023-12-27 09:52:33,581][20381] Updated weights for policy 1, policy_version 612798 (0.0006) [2023-12-27 09:52:33,586][20347] Updated weights for policy 0, policy_version 613105 (0.0010) [2023-12-27 09:52:33,646][20347] Updated weights for policy 0, policy_version 613115 (0.0010) [2023-12-27 09:52:34,190][20381] Updated weights for policy 1, policy_version 612808 (0.0007) [2023-12-27 09:52:34,248][20381] Updated weights for policy 1, policy_version 612818 (0.0006) [2023-12-27 09:52:34,305][20347] Updated weights for policy 0, policy_version 613125 (0.0010) [2023-12-27 09:52:34,312][20381] Updated weights for policy 1, policy_version 612828 (0.0005) [2023-12-27 09:52:34,365][20347] Updated weights for policy 0, policy_version 613135 (0.0010) [2023-12-27 09:52:34,427][20347] Updated weights for policy 0, policy_version 613145 (0.0010) [2023-12-27 09:52:34,996][20381] Updated weights for policy 1, policy_version 612838 (0.0006) [2023-12-27 09:52:35,043][20381] Updated weights for policy 1, policy_version 612848 (0.0007) [2023-12-27 09:52:35,097][20381] Updated weights for policy 1, policy_version 612858 (0.0007) [2023-12-27 09:52:35,116][20347] Updated weights for policy 0, policy_version 613155 (0.0010) [2023-12-27 09:52:35,179][20347] Updated weights for policy 0, policy_version 613165 (0.0010) [2023-12-27 09:52:35,231][20347] Updated weights for policy 0, policy_version 613175 (0.0010) [2023-12-27 09:52:35,793][20381] Updated weights for policy 1, policy_version 612868 (0.0008) [2023-12-27 09:52:35,850][20381] Updated weights for policy 1, policy_version 612878 (0.0007) [2023-12-27 09:52:35,897][20347] Updated weights for policy 0, policy_version 613185 (0.0010) [2023-12-27 09:52:35,916][20381] Updated weights for policy 1, policy_version 612888 (0.0007) [2023-12-27 09:52:35,955][20347] Updated weights for policy 0, policy_version 613195 (0.0007) [2023-12-27 09:52:36,003][20347] Updated weights for policy 0, policy_version 613205 (0.0005) [2023-12-27 09:52:36,049][20347] Updated weights for policy 0, policy_version 613215 (0.0005) [2023-12-27 09:52:36,604][20381] Updated weights for policy 1, policy_version 612898 (0.0008) [2023-12-27 09:52:36,651][20347] Updated weights for policy 0, policy_version 613225 (0.0006) [2023-12-27 09:52:36,671][20381] Updated weights for policy 1, policy_version 612908 (0.0005) [2023-12-27 09:52:36,709][20347] Updated weights for policy 0, policy_version 613235 (0.0006) [2023-12-27 09:52:36,738][20381] Updated weights for policy 1, policy_version 612918 (0.0006) [2023-12-27 09:52:36,760][20347] Updated weights for policy 0, policy_version 613245 (0.0007) [2023-12-27 09:52:36,806][20381] Updated weights for policy 1, policy_version 612928 (0.0009) [2023-12-27 09:52:37,337][20291] KL-divergence is very high: 115.2515 [2023-12-27 09:52:37,353][20347] Updated weights for policy 0, policy_version 613255 (0.0006) [2023-12-27 09:52:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 313942016. Throughput: 0: 10342.4, 1: 10349.6. Samples: 313914352. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:37,363][19170] Avg episode reward: [(0, '932.060'), (1, '991.210')] [2023-12-27 09:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000612928_156934144.pth... [2023-12-27 09:52:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000611744_156631040.pth [2023-12-27 09:52:37,409][20347] Updated weights for policy 0, policy_version 613265 (0.0007) [2023-12-27 09:52:37,459][20347] Updated weights for policy 0, policy_version 613275 (0.0006) [2023-12-27 09:52:37,480][20381] Updated weights for policy 1, policy_version 612938 (0.0011) [2023-12-27 09:52:37,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000613280_157016064.pth... [2023-12-27 09:52:37,487][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000612032_156696576.pth [2023-12-27 09:52:37,532][20381] Updated weights for policy 1, policy_version 612948 (0.0011) [2023-12-27 09:52:37,592][20381] Updated weights for policy 1, policy_version 612958 (0.0011) [2023-12-27 09:52:38,105][20347] Updated weights for policy 0, policy_version 613285 (0.0006) [2023-12-27 09:52:38,159][20347] Updated weights for policy 0, policy_version 613295 (0.0007) [2023-12-27 09:52:38,214][20347] Updated weights for policy 0, policy_version 613305 (0.0007) [2023-12-27 09:52:38,234][20381] Updated weights for policy 1, policy_version 612968 (0.0010) [2023-12-27 09:52:38,292][20381] Updated weights for policy 1, policy_version 612978 (0.0010) [2023-12-27 09:52:38,353][20381] Updated weights for policy 1, policy_version 612988 (0.0009) [2023-12-27 09:52:38,930][20347] Updated weights for policy 0, policy_version 613315 (0.0006) [2023-12-27 09:52:38,981][20347] Updated weights for policy 0, policy_version 613325 (0.0007) [2023-12-27 09:52:38,998][20381] Updated weights for policy 1, policy_version 612998 (0.0008) [2023-12-27 09:52:39,033][20347] Updated weights for policy 0, policy_version 613335 (0.0006) [2023-12-27 09:52:39,056][20381] Updated weights for policy 1, policy_version 613008 (0.0010) [2023-12-27 09:52:39,112][20381] Updated weights for policy 1, policy_version 613018 (0.0010) [2023-12-27 09:52:39,777][20347] Updated weights for policy 0, policy_version 613345 (0.0007) [2023-12-27 09:52:39,783][20381] Updated weights for policy 1, policy_version 613028 (0.0010) [2023-12-27 09:52:39,837][20347] Updated weights for policy 0, policy_version 613355 (0.0007) [2023-12-27 09:52:39,839][20381] Updated weights for policy 1, policy_version 613038 (0.0011) [2023-12-27 09:52:39,897][20347] Updated weights for policy 0, policy_version 613365 (0.0006) [2023-12-27 09:52:39,899][20381] Updated weights for policy 1, policy_version 613048 (0.0011) [2023-12-27 09:52:39,958][20347] Updated weights for policy 0, policy_version 613375 (0.0006) [2023-12-27 09:52:40,613][20381] Updated weights for policy 1, policy_version 613058 (0.0011) [2023-12-27 09:52:40,658][20347] Updated weights for policy 0, policy_version 613385 (0.0007) [2023-12-27 09:52:40,679][20381] Updated weights for policy 1, policy_version 613068 (0.0011) [2023-12-27 09:52:40,721][20347] Updated weights for policy 0, policy_version 613395 (0.0007) [2023-12-27 09:52:40,736][20381] Updated weights for policy 1, policy_version 613078 (0.0011) [2023-12-27 09:52:40,782][20347] Updated weights for policy 0, policy_version 613405 (0.0007) [2023-12-27 09:52:40,792][20381] Updated weights for policy 1, policy_version 613088 (0.0009) [2023-12-27 09:52:41,479][20347] Updated weights for policy 0, policy_version 613415 (0.0006) [2023-12-27 09:52:41,524][20381] Updated weights for policy 1, policy_version 613098 (0.0008) [2023-12-27 09:52:41,539][20347] Updated weights for policy 0, policy_version 613425 (0.0007) [2023-12-27 09:52:41,579][20381] Updated weights for policy 1, policy_version 613108 (0.0008) [2023-12-27 09:52:41,597][20347] Updated weights for policy 0, policy_version 613435 (0.0007) [2023-12-27 09:52:41,642][20381] Updated weights for policy 1, policy_version 613118 (0.0008) [2023-12-27 09:52:42,287][20381] Updated weights for policy 1, policy_version 613128 (0.0007) [2023-12-27 09:52:42,333][20347] Updated weights for policy 0, policy_version 613445 (0.0008) [2023-12-27 09:52:42,352][20381] Updated weights for policy 1, policy_version 613138 (0.0007) [2023-12-27 09:52:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 314040320. Throughput: 0: 10338.7, 1: 10282.6. Samples: 314037256. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:42,363][19170] Avg episode reward: [(0, '942.150'), (1, '981.620')] [2023-12-27 09:52:42,403][20347] Updated weights for policy 0, policy_version 613455 (0.0009) [2023-12-27 09:52:42,413][20381] Updated weights for policy 1, policy_version 613148 (0.0008) [2023-12-27 09:52:42,467][20347] Updated weights for policy 0, policy_version 613465 (0.0008) [2023-12-27 09:52:43,022][20381] Updated weights for policy 1, policy_version 613158 (0.0006) [2023-12-27 09:52:43,090][20381] Updated weights for policy 1, policy_version 613168 (0.0006) [2023-12-27 09:52:43,152][20381] Updated weights for policy 1, policy_version 613178 (0.0008) [2023-12-27 09:52:43,191][20347] Updated weights for policy 0, policy_version 613475 (0.0008) [2023-12-27 09:52:43,247][20347] Updated weights for policy 0, policy_version 613485 (0.0006) [2023-12-27 09:52:43,306][20347] Updated weights for policy 0, policy_version 613495 (0.0005) [2023-12-27 09:52:43,818][20381] Updated weights for policy 1, policy_version 613188 (0.0008) [2023-12-27 09:52:43,863][20381] Updated weights for policy 1, policy_version 613198 (0.0008) [2023-12-27 09:52:43,907][20381] Updated weights for policy 1, policy_version 613208 (0.0007) [2023-12-27 09:52:43,965][20347] Updated weights for policy 0, policy_version 613505 (0.0008) [2023-12-27 09:52:44,026][20347] Updated weights for policy 0, policy_version 613515 (0.0006) [2023-12-27 09:52:44,086][20347] Updated weights for policy 0, policy_version 613525 (0.0005) [2023-12-27 09:52:44,141][20347] Updated weights for policy 0, policy_version 613535 (0.0005) [2023-12-27 09:52:44,599][20381] Updated weights for policy 1, policy_version 613218 (0.0006) [2023-12-27 09:52:44,652][20381] Updated weights for policy 1, policy_version 613228 (0.0005) [2023-12-27 09:52:44,705][20381] Updated weights for policy 1, policy_version 613238 (0.0005) [2023-12-27 09:52:44,757][20381] Updated weights for policy 1, policy_version 613248 (0.0006) [2023-12-27 09:52:44,765][20347] Updated weights for policy 0, policy_version 613545 (0.0008) [2023-12-27 09:52:44,820][20347] Updated weights for policy 0, policy_version 613555 (0.0008) [2023-12-27 09:52:44,875][20347] Updated weights for policy 0, policy_version 613565 (0.0007) [2023-12-27 09:52:45,409][20381] Updated weights for policy 1, policy_version 613258 (0.0007) [2023-12-27 09:52:45,464][20381] Updated weights for policy 1, policy_version 613268 (0.0007) [2023-12-27 09:52:45,495][20347] Updated weights for policy 0, policy_version 613575 (0.0008) [2023-12-27 09:52:45,521][20381] Updated weights for policy 1, policy_version 613278 (0.0009) [2023-12-27 09:52:45,549][20347] Updated weights for policy 0, policy_version 613585 (0.0009) [2023-12-27 09:52:45,605][20347] Updated weights for policy 0, policy_version 613595 (0.0005) [2023-12-27 09:52:46,191][20347] Updated weights for policy 0, policy_version 613605 (0.0007) [2023-12-27 09:52:46,234][20381] Updated weights for policy 1, policy_version 613288 (0.0008) [2023-12-27 09:52:46,239][20291] KL-divergence is very high: 118.7904 [2023-12-27 09:52:46,251][20347] Updated weights for policy 0, policy_version 613615 (0.0006) [2023-12-27 09:52:46,298][20381] Updated weights for policy 1, policy_version 613298 (0.0008) [2023-12-27 09:52:46,304][20347] Updated weights for policy 0, policy_version 613625 (0.0006) [2023-12-27 09:52:46,356][20381] Updated weights for policy 1, policy_version 613308 (0.0006) [2023-12-27 09:52:46,932][20347] Updated weights for policy 0, policy_version 613635 (0.0010) [2023-12-27 09:52:46,980][20347] Updated weights for policy 0, policy_version 613645 (0.0007) [2023-12-27 09:52:47,015][20381] Updated weights for policy 1, policy_version 613318 (0.0007) [2023-12-27 09:52:47,033][20347] Updated weights for policy 0, policy_version 613655 (0.0008) [2023-12-27 09:52:47,064][20381] Updated weights for policy 1, policy_version 613328 (0.0006) [2023-12-27 09:52:47,114][20381] Updated weights for policy 1, policy_version 613338 (0.0007) [2023-12-27 09:52:47,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 314155008. Throughput: 0: 10398.9, 1: 10287.3. Samples: 314161288. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:47,364][19170] Avg episode reward: [(0, '942.300'), (1, '981.040')] [2023-12-27 09:52:47,725][20347] Updated weights for policy 0, policy_version 613665 (0.0007) [2023-12-27 09:52:47,783][20347] Updated weights for policy 0, policy_version 613675 (0.0010) [2023-12-27 09:52:47,835][20381] Updated weights for policy 1, policy_version 613348 (0.0007) [2023-12-27 09:52:47,844][20347] Updated weights for policy 0, policy_version 613685 (0.0010) [2023-12-27 09:52:47,889][20381] Updated weights for policy 1, policy_version 613358 (0.0005) [2023-12-27 09:52:47,898][20347] Updated weights for policy 0, policy_version 613695 (0.0010) [2023-12-27 09:52:47,935][20381] Updated weights for policy 1, policy_version 613368 (0.0007) [2023-12-27 09:52:48,483][20347] Updated weights for policy 0, policy_version 613705 (0.0010) [2023-12-27 09:52:48,539][20347] Updated weights for policy 0, policy_version 613715 (0.0010) [2023-12-27 09:52:48,598][20347] Updated weights for policy 0, policy_version 613725 (0.0010) [2023-12-27 09:52:48,607][20381] Updated weights for policy 1, policy_version 613378 (0.0009) [2023-12-27 09:52:48,659][20292] KL-divergence is very high: 252.9535 [2023-12-27 09:52:48,666][20381] Updated weights for policy 1, policy_version 613388 (0.0008) [2023-12-27 09:52:48,717][20381] Updated weights for policy 1, policy_version 613398 (0.0007) [2023-12-27 09:52:48,778][20381] Updated weights for policy 1, policy_version 613408 (0.0007) [2023-12-27 09:52:49,235][20347] Updated weights for policy 0, policy_version 613735 (0.0010) [2023-12-27 09:52:49,291][20347] Updated weights for policy 0, policy_version 613745 (0.0010) [2023-12-27 09:52:49,354][20347] Updated weights for policy 0, policy_version 613755 (0.0010) [2023-12-27 09:52:49,480][20381] Updated weights for policy 1, policy_version 613418 (0.0011) [2023-12-27 09:52:49,535][20381] Updated weights for policy 1, policy_version 613428 (0.0011) [2023-12-27 09:52:49,586][20381] Updated weights for policy 1, policy_version 613438 (0.0010) [2023-12-27 09:52:50,017][20347] Updated weights for policy 0, policy_version 613765 (0.0007) [2023-12-27 09:52:50,074][20347] Updated weights for policy 0, policy_version 613775 (0.0010) [2023-12-27 09:52:50,138][20347] Updated weights for policy 0, policy_version 613785 (0.0011) [2023-12-27 09:52:50,248][20381] Updated weights for policy 1, policy_version 613448 (0.0008) [2023-12-27 09:52:50,308][20381] Updated weights for policy 1, policy_version 613458 (0.0005) [2023-12-27 09:52:50,373][20381] Updated weights for policy 1, policy_version 613468 (0.0007) [2023-12-27 09:52:50,823][20347] Updated weights for policy 0, policy_version 613795 (0.0008) [2023-12-27 09:52:50,878][20347] Updated weights for policy 0, policy_version 613805 (0.0007) [2023-12-27 09:52:50,939][20347] Updated weights for policy 0, policy_version 613815 (0.0007) [2023-12-27 09:52:50,980][20381] Updated weights for policy 1, policy_version 613478 (0.0006) [2023-12-27 09:52:51,037][20381] Updated weights for policy 1, policy_version 613488 (0.0007) [2023-12-27 09:52:51,098][20381] Updated weights for policy 1, policy_version 613498 (0.0008) [2023-12-27 09:52:51,640][20347] Updated weights for policy 0, policy_version 613825 (0.0007) [2023-12-27 09:52:51,708][20347] Updated weights for policy 0, policy_version 613835 (0.0008) [2023-12-27 09:52:51,763][20347] Updated weights for policy 0, policy_version 613845 (0.0008) [2023-12-27 09:52:51,815][20347] Updated weights for policy 0, policy_version 613855 (0.0007) [2023-12-27 09:52:51,839][20381] Updated weights for policy 1, policy_version 613508 (0.0008) [2023-12-27 09:52:51,890][20381] Updated weights for policy 1, policy_version 613518 (0.0008) [2023-12-27 09:52:51,946][20381] Updated weights for policy 1, policy_version 613528 (0.0007) [2023-12-27 09:52:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 314253312. Throughput: 0: 10400.3, 1: 10273.2. Samples: 314224040. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:52,363][19170] Avg episode reward: [(0, '961.570'), (1, '961.750')] [2023-12-27 09:52:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000613536_157089792.pth... [2023-12-27 09:52:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000613856_157163520.pth... [2023-12-27 09:52:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000612352_156786688.pth [2023-12-27 09:52:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000612640_156852224.pth [2023-12-27 09:52:52,498][20347] Updated weights for policy 0, policy_version 613865 (0.0006) [2023-12-27 09:52:52,551][20347] Updated weights for policy 0, policy_version 613875 (0.0005) [2023-12-27 09:52:52,613][20347] Updated weights for policy 0, policy_version 613885 (0.0006) [2023-12-27 09:52:52,620][20381] Updated weights for policy 1, policy_version 613538 (0.0007) [2023-12-27 09:52:52,678][20381] Updated weights for policy 1, policy_version 613548 (0.0005) [2023-12-27 09:52:52,735][20381] Updated weights for policy 1, policy_version 613558 (0.0005) [2023-12-27 09:52:52,791][20381] Updated weights for policy 1, policy_version 613568 (0.0005) [2023-12-27 09:52:53,251][20347] Updated weights for policy 0, policy_version 613895 (0.0005) [2023-12-27 09:52:53,318][20347] Updated weights for policy 0, policy_version 613905 (0.0008) [2023-12-27 09:52:53,370][20381] Updated weights for policy 1, policy_version 613578 (0.0005) [2023-12-27 09:52:53,371][20347] Updated weights for policy 0, policy_version 613915 (0.0007) [2023-12-27 09:52:53,419][20381] Updated weights for policy 1, policy_version 613588 (0.0005) [2023-12-27 09:52:53,479][20381] Updated weights for policy 1, policy_version 613598 (0.0005) [2023-12-27 09:52:54,007][20347] Updated weights for policy 0, policy_version 613925 (0.0006) [2023-12-27 09:52:54,061][20347] Updated weights for policy 0, policy_version 613935 (0.0005) [2023-12-27 09:52:54,119][20347] Updated weights for policy 0, policy_version 613945 (0.0005) [2023-12-27 09:52:54,145][20381] Updated weights for policy 1, policy_version 613608 (0.0007) [2023-12-27 09:52:54,204][20381] Updated weights for policy 1, policy_version 613618 (0.0008) [2023-12-27 09:52:54,258][20381] Updated weights for policy 1, policy_version 613628 (0.0008) [2023-12-27 09:52:54,648][20347] Updated weights for policy 0, policy_version 613955 (0.0005) [2023-12-27 09:52:54,707][20347] Updated weights for policy 0, policy_version 613965 (0.0005) [2023-12-27 09:52:54,760][20347] Updated weights for policy 0, policy_version 613975 (0.0005) [2023-12-27 09:52:54,937][20381] Updated weights for policy 1, policy_version 613638 (0.0006) [2023-12-27 09:52:54,995][20381] Updated weights for policy 1, policy_version 613648 (0.0006) [2023-12-27 09:52:55,058][20381] Updated weights for policy 1, policy_version 613658 (0.0009) [2023-12-27 09:52:55,372][20347] Updated weights for policy 0, policy_version 613985 (0.0005) [2023-12-27 09:52:55,415][20347] Updated weights for policy 0, policy_version 613995 (0.0005) [2023-12-27 09:52:55,471][20347] Updated weights for policy 0, policy_version 614005 (0.0005) [2023-12-27 09:52:55,525][20347] Updated weights for policy 0, policy_version 614015 (0.0005) [2023-12-27 09:52:55,677][20381] Updated weights for policy 1, policy_version 613668 (0.0007) [2023-12-27 09:52:55,736][20381] Updated weights for policy 1, policy_version 613678 (0.0008) [2023-12-27 09:52:55,787][20381] Updated weights for policy 1, policy_version 613688 (0.0008) [2023-12-27 09:52:56,123][20347] Updated weights for policy 0, policy_version 614025 (0.0006) [2023-12-27 09:52:56,181][20347] Updated weights for policy 0, policy_version 614035 (0.0007) [2023-12-27 09:52:56,235][20347] Updated weights for policy 0, policy_version 614045 (0.0006) [2023-12-27 09:52:56,499][20381] Updated weights for policy 1, policy_version 613698 (0.0007) [2023-12-27 09:52:56,544][20381] Updated weights for policy 1, policy_version 613708 (0.0008) [2023-12-27 09:52:56,589][20381] Updated weights for policy 1, policy_version 613718 (0.0008) [2023-12-27 09:52:56,635][20381] Updated weights for policy 1, policy_version 613728 (0.0007) [2023-12-27 09:52:56,879][20347] Updated weights for policy 0, policy_version 614055 (0.0006) [2023-12-27 09:52:56,928][20347] Updated weights for policy 0, policy_version 614065 (0.0009) [2023-12-27 09:52:56,979][20347] Updated weights for policy 0, policy_version 614075 (0.0005) [2023-12-27 09:52:57,241][20381] Updated weights for policy 1, policy_version 613738 (0.0005) [2023-12-27 09:52:57,293][20381] Updated weights for policy 1, policy_version 613748 (0.0006) [2023-12-27 09:52:57,354][20381] Updated weights for policy 1, policy_version 613758 (0.0007) [2023-12-27 09:52:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 314359808. Throughput: 0: 10448.3, 1: 10303.4. Samples: 314351128. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:52:57,363][19170] Avg episode reward: [(0, '980.730'), (1, '971.060')] [2023-12-27 09:52:57,691][20347] Updated weights for policy 0, policy_version 614085 (0.0006) [2023-12-27 09:52:57,741][20347] Updated weights for policy 0, policy_version 614095 (0.0006) [2023-12-27 09:52:57,786][20347] Updated weights for policy 0, policy_version 614105 (0.0005) [2023-12-27 09:52:57,945][20381] Updated weights for policy 1, policy_version 613768 (0.0009) [2023-12-27 09:52:57,996][20381] Updated weights for policy 1, policy_version 613778 (0.0009) [2023-12-27 09:52:58,059][20381] Updated weights for policy 1, policy_version 613788 (0.0006) [2023-12-27 09:52:58,403][20347] Updated weights for policy 0, policy_version 614115 (0.0006) [2023-12-27 09:52:58,466][20347] Updated weights for policy 0, policy_version 614125 (0.0010) [2023-12-27 09:52:58,530][20347] Updated weights for policy 0, policy_version 614135 (0.0011) [2023-12-27 09:52:58,783][20381] Updated weights for policy 1, policy_version 613799 (0.0008) [2023-12-27 09:52:58,848][20381] Updated weights for policy 1, policy_version 613809 (0.0010) [2023-12-27 09:52:58,913][20381] Updated weights for policy 1, policy_version 613819 (0.0009) [2023-12-27 09:52:59,283][20347] Updated weights for policy 0, policy_version 614145 (0.0007) [2023-12-27 09:52:59,350][20347] Updated weights for policy 0, policy_version 614155 (0.0009) [2023-12-27 09:52:59,409][20347] Updated weights for policy 0, policy_version 614165 (0.0010) [2023-12-27 09:52:59,460][20347] Updated weights for policy 0, policy_version 614175 (0.0010) [2023-12-27 09:52:59,581][20381] Updated weights for policy 1, policy_version 613829 (0.0007) [2023-12-27 09:52:59,641][20381] Updated weights for policy 1, policy_version 613839 (0.0006) [2023-12-27 09:52:59,705][20381] Updated weights for policy 1, policy_version 613849 (0.0006) [2023-12-27 09:53:00,155][20347] Updated weights for policy 0, policy_version 614185 (0.0007) [2023-12-27 09:53:00,216][20347] Updated weights for policy 0, policy_version 614195 (0.0008) [2023-12-27 09:53:00,272][20347] Updated weights for policy 0, policy_version 614205 (0.0007) [2023-12-27 09:53:00,299][20381] Updated weights for policy 1, policy_version 613859 (0.0006) [2023-12-27 09:53:00,359][20381] Updated weights for policy 1, policy_version 613869 (0.0005) [2023-12-27 09:53:00,409][20381] Updated weights for policy 1, policy_version 613879 (0.0008) [2023-12-27 09:53:00,925][20347] Updated weights for policy 0, policy_version 614215 (0.0006) [2023-12-27 09:53:00,973][20347] Updated weights for policy 0, policy_version 614225 (0.0005) [2023-12-27 09:53:00,981][20381] Updated weights for policy 1, policy_version 613889 (0.0006) [2023-12-27 09:53:01,017][20347] Updated weights for policy 0, policy_version 614235 (0.0005) [2023-12-27 09:53:01,038][20381] Updated weights for policy 1, policy_version 613899 (0.0005) [2023-12-27 09:53:01,095][20381] Updated weights for policy 1, policy_version 613909 (0.0008) [2023-12-27 09:53:01,148][20381] Updated weights for policy 1, policy_version 613919 (0.0007) [2023-12-27 09:53:01,733][20347] Updated weights for policy 0, policy_version 614245 (0.0008) [2023-12-27 09:53:01,790][20347] Updated weights for policy 0, policy_version 614255 (0.0011) [2023-12-27 09:53:01,821][20381] Updated weights for policy 1, policy_version 613929 (0.0007) [2023-12-27 09:53:01,842][20347] Updated weights for policy 0, policy_version 614265 (0.0010) [2023-12-27 09:53:01,877][20381] Updated weights for policy 1, policy_version 613939 (0.0005) [2023-12-27 09:53:01,928][20381] Updated weights for policy 1, policy_version 613949 (0.0005) [2023-12-27 09:53:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 314466304. Throughput: 0: 10425.0, 1: 10349.3. Samples: 314477828. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:02,363][19170] Avg episode reward: [(0, '951.640'), (1, '961.380')] [2023-12-27 09:53:02,521][20381] Updated weights for policy 1, policy_version 613959 (0.0006) [2023-12-27 09:53:02,540][20347] Updated weights for policy 0, policy_version 614275 (0.0010) [2023-12-27 09:53:02,572][20381] Updated weights for policy 1, policy_version 613969 (0.0007) [2023-12-27 09:53:02,599][20347] Updated weights for policy 0, policy_version 614285 (0.0010) [2023-12-27 09:53:02,628][20381] Updated weights for policy 1, policy_version 613979 (0.0010) [2023-12-27 09:53:02,657][20347] Updated weights for policy 0, policy_version 614295 (0.0010) [2023-12-27 09:53:03,242][20381] Updated weights for policy 1, policy_version 613989 (0.0006) [2023-12-27 09:53:03,285][20381] Updated weights for policy 1, policy_version 613999 (0.0005) [2023-12-27 09:53:03,323][20347] Updated weights for policy 0, policy_version 614305 (0.0010) [2023-12-27 09:53:03,333][20381] Updated weights for policy 1, policy_version 614009 (0.0005) [2023-12-27 09:53:03,380][20347] Updated weights for policy 0, policy_version 614315 (0.0010) [2023-12-27 09:53:03,435][20347] Updated weights for policy 0, policy_version 614325 (0.0010) [2023-12-27 09:53:03,494][20347] Updated weights for policy 0, policy_version 614335 (0.0010) [2023-12-27 09:53:03,933][20381] Updated weights for policy 1, policy_version 614019 (0.0006) [2023-12-27 09:53:03,990][20381] Updated weights for policy 1, policy_version 614029 (0.0005) [2023-12-27 09:53:04,049][20381] Updated weights for policy 1, policy_version 614039 (0.0005) [2023-12-27 09:53:04,146][20347] Updated weights for policy 0, policy_version 614345 (0.0008) [2023-12-27 09:53:04,207][20347] Updated weights for policy 0, policy_version 614355 (0.0006) [2023-12-27 09:53:04,264][20347] Updated weights for policy 0, policy_version 614365 (0.0007) [2023-12-27 09:53:04,619][20381] Updated weights for policy 1, policy_version 614049 (0.0006) [2023-12-27 09:53:04,674][20381] Updated weights for policy 1, policy_version 614059 (0.0008) [2023-12-27 09:53:04,733][20381] Updated weights for policy 1, policy_version 614069 (0.0008) [2023-12-27 09:53:04,791][20381] Updated weights for policy 1, policy_version 614079 (0.0008) [2023-12-27 09:53:04,925][20347] Updated weights for policy 0, policy_version 614375 (0.0006) [2023-12-27 09:53:04,973][20347] Updated weights for policy 0, policy_version 614385 (0.0005) [2023-12-27 09:53:05,024][20347] Updated weights for policy 0, policy_version 614395 (0.0005) [2023-12-27 09:53:05,474][20381] Updated weights for policy 1, policy_version 614089 (0.0007) [2023-12-27 09:53:05,521][20381] Updated weights for policy 1, policy_version 614099 (0.0006) [2023-12-27 09:53:05,582][20381] Updated weights for policy 1, policy_version 614109 (0.0008) [2023-12-27 09:53:05,588][20347] Updated weights for policy 0, policy_version 614405 (0.0007) [2023-12-27 09:53:05,649][20347] Updated weights for policy 0, policy_version 614415 (0.0007) [2023-12-27 09:53:05,709][20347] Updated weights for policy 0, policy_version 614425 (0.0008) [2023-12-27 09:53:06,266][20381] Updated weights for policy 1, policy_version 614119 (0.0006) [2023-12-27 09:53:06,319][20381] Updated weights for policy 1, policy_version 614129 (0.0008) [2023-12-27 09:53:06,352][20347] Updated weights for policy 0, policy_version 614435 (0.0006) [2023-12-27 09:53:06,378][20381] Updated weights for policy 1, policy_version 614139 (0.0006) [2023-12-27 09:53:06,414][20347] Updated weights for policy 0, policy_version 614445 (0.0008) [2023-12-27 09:53:06,469][20347] Updated weights for policy 0, policy_version 614455 (0.0008) [2023-12-27 09:53:07,039][20381] Updated weights for policy 1, policy_version 614149 (0.0007) [2023-12-27 09:53:07,087][20381] Updated weights for policy 1, policy_version 614159 (0.0010) [2023-12-27 09:53:07,127][20347] Updated weights for policy 0, policy_version 614465 (0.0008) [2023-12-27 09:53:07,146][20381] Updated weights for policy 1, policy_version 614169 (0.0008) [2023-12-27 09:53:07,193][20347] Updated weights for policy 0, policy_version 614475 (0.0008) [2023-12-27 09:53:07,255][20347] Updated weights for policy 0, policy_version 614485 (0.0008) [2023-12-27 09:53:07,323][20347] Updated weights for policy 0, policy_version 614495 (0.0008) [2023-12-27 09:53:07,363][19170] Fps is (10 sec: 22117.5, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 314580992. Throughput: 0: 10442.2, 1: 10431.1. Samples: 314543528. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:07,364][19170] Avg episode reward: [(0, '958.530'), (1, '980.570')] [2023-12-27 09:53:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000614496_157327360.pth... [2023-12-27 09:53:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000614176_157253632.pth... [2023-12-27 09:53:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000612928_156934144.pth [2023-12-27 09:53:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000613280_157016064.pth [2023-12-27 09:53:07,671][20381] Updated weights for policy 1, policy_version 614179 (0.0005) [2023-12-27 09:53:07,721][20381] Updated weights for policy 1, policy_version 614189 (0.0005) [2023-12-27 09:53:07,771][20381] Updated weights for policy 1, policy_version 614199 (0.0009) [2023-12-27 09:53:07,937][20347] Updated weights for policy 0, policy_version 614505 (0.0006) [2023-12-27 09:53:07,993][20347] Updated weights for policy 0, policy_version 614515 (0.0007) [2023-12-27 09:53:08,046][20347] Updated weights for policy 0, policy_version 614525 (0.0006) [2023-12-27 09:53:08,372][20381] Updated weights for policy 1, policy_version 614209 (0.0010) [2023-12-27 09:53:08,431][20381] Updated weights for policy 1, policy_version 614219 (0.0007) [2023-12-27 09:53:08,485][20381] Updated weights for policy 1, policy_version 614229 (0.0008) [2023-12-27 09:53:08,545][20381] Updated weights for policy 1, policy_version 614239 (0.0007) [2023-12-27 09:53:08,673][20291] KL-divergence is very high: 106.1722 [2023-12-27 09:53:08,693][20347] Updated weights for policy 0, policy_version 614535 (0.0005) [2023-12-27 09:53:08,753][20347] Updated weights for policy 0, policy_version 614545 (0.0008) [2023-12-27 09:53:08,819][20347] Updated weights for policy 0, policy_version 614555 (0.0008) [2023-12-27 09:53:09,211][20381] Updated weights for policy 1, policy_version 614249 (0.0006) [2023-12-27 09:53:09,271][20381] Updated weights for policy 1, policy_version 614259 (0.0008) [2023-12-27 09:53:09,325][20381] Updated weights for policy 1, policy_version 614269 (0.0008) [2023-12-27 09:53:09,487][20347] Updated weights for policy 0, policy_version 614565 (0.0008) [2023-12-27 09:53:09,547][20347] Updated weights for policy 0, policy_version 614575 (0.0007) [2023-12-27 09:53:09,610][20347] Updated weights for policy 0, policy_version 614585 (0.0006) [2023-12-27 09:53:10,045][20381] Updated weights for policy 1, policy_version 614279 (0.0007) [2023-12-27 09:53:10,107][20381] Updated weights for policy 1, policy_version 614289 (0.0007) [2023-12-27 09:53:10,165][20381] Updated weights for policy 1, policy_version 614299 (0.0007) [2023-12-27 09:53:10,261][20347] Updated weights for policy 0, policy_version 614595 (0.0006) [2023-12-27 09:53:10,327][20347] Updated weights for policy 0, policy_version 614605 (0.0005) [2023-12-27 09:53:10,388][20347] Updated weights for policy 0, policy_version 614615 (0.0006) [2023-12-27 09:53:10,752][20381] Updated weights for policy 1, policy_version 614309 (0.0008) [2023-12-27 09:53:10,803][20381] Updated weights for policy 1, policy_version 614319 (0.0007) [2023-12-27 09:53:10,854][20381] Updated weights for policy 1, policy_version 614329 (0.0007) [2023-12-27 09:53:11,069][20347] Updated weights for policy 0, policy_version 614625 (0.0006) [2023-12-27 09:53:11,135][20347] Updated weights for policy 0, policy_version 614635 (0.0007) [2023-12-27 09:53:11,197][20347] Updated weights for policy 0, policy_version 614645 (0.0007) [2023-12-27 09:53:11,257][20347] Updated weights for policy 0, policy_version 614655 (0.0008) [2023-12-27 09:53:11,517][20381] Updated weights for policy 1, policy_version 614339 (0.0008) [2023-12-27 09:53:11,575][20381] Updated weights for policy 1, policy_version 614349 (0.0007) [2023-12-27 09:53:11,635][20381] Updated weights for policy 1, policy_version 614359 (0.0008) [2023-12-27 09:53:11,923][20347] Updated weights for policy 0, policy_version 614665 (0.0008) [2023-12-27 09:53:11,987][20347] Updated weights for policy 0, policy_version 614675 (0.0009) [2023-12-27 09:53:12,054][20347] Updated weights for policy 0, policy_version 614685 (0.0007) [2023-12-27 09:53:12,274][20381] Updated weights for policy 1, policy_version 614369 (0.0010) [2023-12-27 09:53:12,337][20381] Updated weights for policy 1, policy_version 614379 (0.0008) [2023-12-27 09:53:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 314679296. Throughput: 0: 10541.4, 1: 10537.1. Samples: 314671960. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:12,363][19170] Avg episode reward: [(0, '983.090'), (1, '971.020')] [2023-12-27 09:53:12,396][20381] Updated weights for policy 1, policy_version 614389 (0.0010) [2023-12-27 09:53:12,454][20381] Updated weights for policy 1, policy_version 614399 (0.0011) [2023-12-27 09:53:12,753][20347] Updated weights for policy 0, policy_version 614695 (0.0009) [2023-12-27 09:53:12,807][20347] Updated weights for policy 0, policy_version 614705 (0.0009) [2023-12-27 09:53:12,856][20347] Updated weights for policy 0, policy_version 614715 (0.0010) [2023-12-27 09:53:13,085][20381] Updated weights for policy 1, policy_version 614409 (0.0010) [2023-12-27 09:53:13,137][20381] Updated weights for policy 1, policy_version 614419 (0.0009) [2023-12-27 09:53:13,185][20381] Updated weights for policy 1, policy_version 614429 (0.0006) [2023-12-27 09:53:13,555][20347] Updated weights for policy 0, policy_version 614725 (0.0010) [2023-12-27 09:53:13,604][20347] Updated weights for policy 0, policy_version 614735 (0.0010) [2023-12-27 09:53:13,653][20347] Updated weights for policy 0, policy_version 614745 (0.0010) [2023-12-27 09:53:13,768][20381] Updated weights for policy 1, policy_version 614439 (0.0008) [2023-12-27 09:53:13,826][20381] Updated weights for policy 1, policy_version 614449 (0.0007) [2023-12-27 09:53:13,890][20381] Updated weights for policy 1, policy_version 614459 (0.0007) [2023-12-27 09:53:14,342][20347] Updated weights for policy 0, policy_version 614755 (0.0010) [2023-12-27 09:53:14,402][20347] Updated weights for policy 0, policy_version 614765 (0.0010) [2023-12-27 09:53:14,455][20347] Updated weights for policy 0, policy_version 614775 (0.0006) [2023-12-27 09:53:14,495][20381] Updated weights for policy 1, policy_version 614469 (0.0008) [2023-12-27 09:53:14,554][20381] Updated weights for policy 1, policy_version 614479 (0.0009) [2023-12-27 09:53:14,608][20381] Updated weights for policy 1, policy_version 614489 (0.0007) [2023-12-27 09:53:15,113][20347] Updated weights for policy 0, policy_version 614785 (0.0009) [2023-12-27 09:53:15,171][20347] Updated weights for policy 0, policy_version 614795 (0.0010) [2023-12-27 09:53:15,223][20347] Updated weights for policy 0, policy_version 614805 (0.0010) [2023-12-27 09:53:15,275][20347] Updated weights for policy 0, policy_version 614815 (0.0010) [2023-12-27 09:53:15,332][20381] Updated weights for policy 1, policy_version 614499 (0.0008) [2023-12-27 09:53:15,384][20381] Updated weights for policy 1, policy_version 614509 (0.0011) [2023-12-27 09:53:15,430][20381] Updated weights for policy 1, policy_version 614519 (0.0010) [2023-12-27 09:53:15,908][20347] Updated weights for policy 0, policy_version 614825 (0.0006) [2023-12-27 09:53:15,963][20347] Updated weights for policy 0, policy_version 614835 (0.0008) [2023-12-27 09:53:16,015][20347] Updated weights for policy 0, policy_version 614845 (0.0010) [2023-12-27 09:53:16,017][20381] Updated weights for policy 1, policy_version 614529 (0.0010) [2023-12-27 09:53:16,074][20381] Updated weights for policy 1, policy_version 614539 (0.0007) [2023-12-27 09:53:16,129][20381] Updated weights for policy 1, policy_version 614549 (0.0011) [2023-12-27 09:53:16,188][20381] Updated weights for policy 1, policy_version 614559 (0.0010) [2023-12-27 09:53:16,690][20347] Updated weights for policy 0, policy_version 614855 (0.0008) [2023-12-27 09:53:16,733][20347] Updated weights for policy 0, policy_version 614865 (0.0007) [2023-12-27 09:53:16,779][20347] Updated weights for policy 0, policy_version 614875 (0.0007) [2023-12-27 09:53:16,874][20381] Updated weights for policy 1, policy_version 614569 (0.0011) [2023-12-27 09:53:16,933][20381] Updated weights for policy 1, policy_version 614579 (0.0011) [2023-12-27 09:53:16,982][20381] Updated weights for policy 1, policy_version 614589 (0.0010) [2023-12-27 09:53:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 314785792. Throughput: 0: 10477.0, 1: 10566.8. Samples: 314797340. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:17,363][19170] Avg episode reward: [(0, '948.070'), (1, '954.840')] [2023-12-27 09:53:17,487][20347] Updated weights for policy 0, policy_version 614885 (0.0007) [2023-12-27 09:53:17,534][20347] Updated weights for policy 0, policy_version 614895 (0.0007) [2023-12-27 09:53:17,585][20347] Updated weights for policy 0, policy_version 614905 (0.0007) [2023-12-27 09:53:17,683][20381] Updated weights for policy 1, policy_version 614599 (0.0010) [2023-12-27 09:53:17,712][20292] KL-divergence is very high: 153.2284 [2023-12-27 09:53:17,719][20292] KL-divergence is very high: 151.7494 [2023-12-27 09:53:17,739][20381] Updated weights for policy 1, policy_version 614609 (0.0011) [2023-12-27 09:53:17,792][20381] Updated weights for policy 1, policy_version 614619 (0.0011) [2023-12-27 09:53:18,300][20347] Updated weights for policy 0, policy_version 614915 (0.0007) [2023-12-27 09:53:18,362][20347] Updated weights for policy 0, policy_version 614925 (0.0008) [2023-12-27 09:53:18,427][20347] Updated weights for policy 0, policy_version 614935 (0.0007) [2023-12-27 09:53:18,465][20292] KL-divergence is very high: 105.2346 [2023-12-27 09:53:18,492][20381] Updated weights for policy 1, policy_version 614629 (0.0009) [2023-12-27 09:53:18,551][20381] Updated weights for policy 1, policy_version 614639 (0.0007) [2023-12-27 09:53:18,609][20292] KL-divergence is very high: 144.6955 [2023-12-27 09:53:18,614][20381] Updated weights for policy 1, policy_version 614649 (0.0008) [2023-12-27 09:53:19,120][20347] Updated weights for policy 0, policy_version 614945 (0.0007) [2023-12-27 09:53:19,171][20347] Updated weights for policy 0, policy_version 614955 (0.0007) [2023-12-27 09:53:19,220][20347] Updated weights for policy 0, policy_version 614965 (0.0008) [2023-12-27 09:53:19,281][20347] Updated weights for policy 0, policy_version 614975 (0.0007) [2023-12-27 09:53:19,311][20381] Updated weights for policy 1, policy_version 614659 (0.0008) [2023-12-27 09:53:19,374][20381] Updated weights for policy 1, policy_version 614669 (0.0011) [2023-12-27 09:53:19,427][20381] Updated weights for policy 1, policy_version 614679 (0.0011) [2023-12-27 09:53:19,921][20347] Updated weights for policy 0, policy_version 614985 (0.0008) [2023-12-27 09:53:19,977][20347] Updated weights for policy 0, policy_version 614995 (0.0006) [2023-12-27 09:53:20,036][20347] Updated weights for policy 0, policy_version 615005 (0.0007) [2023-12-27 09:53:20,120][20381] Updated weights for policy 1, policy_version 614689 (0.0010) [2023-12-27 09:53:20,185][20381] Updated weights for policy 1, policy_version 614699 (0.0006) [2023-12-27 09:53:20,241][20381] Updated weights for policy 1, policy_version 614709 (0.0009) [2023-12-27 09:53:20,301][20381] Updated weights for policy 1, policy_version 614719 (0.0009) [2023-12-27 09:53:20,632][20347] Updated weights for policy 0, policy_version 615015 (0.0007) [2023-12-27 09:53:20,662][20291] KL-divergence is very high: 250.5678 [2023-12-27 09:53:20,691][20347] Updated weights for policy 0, policy_version 615025 (0.0007) [2023-12-27 09:53:20,711][20291] KL-divergence is very high: 120.4169 [2023-12-27 09:53:20,752][20347] Updated weights for policy 0, policy_version 615035 (0.0006) [2023-12-27 09:53:20,988][20381] Updated weights for policy 1, policy_version 614729 (0.0011) [2023-12-27 09:53:21,050][20381] Updated weights for policy 1, policy_version 614739 (0.0011) [2023-12-27 09:53:21,114][20381] Updated weights for policy 1, policy_version 614749 (0.0007) [2023-12-27 09:53:21,378][20347] Updated weights for policy 0, policy_version 615045 (0.0008) [2023-12-27 09:53:21,446][20347] Updated weights for policy 0, policy_version 615055 (0.0007) [2023-12-27 09:53:21,515][20347] Updated weights for policy 0, policy_version 615065 (0.0008) [2023-12-27 09:53:21,778][20381] Updated weights for policy 1, policy_version 614759 (0.0007) [2023-12-27 09:53:21,842][20381] Updated weights for policy 1, policy_version 614769 (0.0007) [2023-12-27 09:53:21,905][20381] Updated weights for policy 1, policy_version 614779 (0.0007) [2023-12-27 09:53:22,243][20347] Updated weights for policy 0, policy_version 615075 (0.0009) [2023-12-27 09:53:22,305][20347] Updated weights for policy 0, policy_version 615085 (0.0008) [2023-12-27 09:53:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 314884096. Throughput: 0: 10433.9, 1: 10520.5. Samples: 314857300. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:22,363][19170] Avg episode reward: [(0, '962.150'), (1, '885.790')] [2023-12-27 09:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000614784_157409280.pth... [2023-12-27 09:53:22,371][20347] Updated weights for policy 0, policy_version 615095 (0.0009) [2023-12-27 09:53:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000613536_157089792.pth [2023-12-27 09:53:22,418][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000615104_157483008.pth... [2023-12-27 09:53:22,421][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000613856_157163520.pth [2023-12-27 09:53:22,611][20381] Updated weights for policy 1, policy_version 614789 (0.0007) [2023-12-27 09:53:22,668][20381] Updated weights for policy 1, policy_version 614799 (0.0010) [2023-12-27 09:53:22,725][20381] Updated weights for policy 1, policy_version 614809 (0.0011) [2023-12-27 09:53:23,048][20347] Updated weights for policy 0, policy_version 615105 (0.0007) [2023-12-27 09:53:23,105][20347] Updated weights for policy 0, policy_version 615115 (0.0005) [2023-12-27 09:53:23,154][20347] Updated weights for policy 0, policy_version 615125 (0.0005) [2023-12-27 09:53:23,212][20347] Updated weights for policy 0, policy_version 615135 (0.0005) [2023-12-27 09:53:23,358][20381] Updated weights for policy 1, policy_version 614819 (0.0009) [2023-12-27 09:53:23,407][20381] Updated weights for policy 1, policy_version 614829 (0.0005) [2023-12-27 09:53:23,467][20381] Updated weights for policy 1, policy_version 614839 (0.0006) [2023-12-27 09:53:23,838][20347] Updated weights for policy 0, policy_version 615145 (0.0009) [2023-12-27 09:53:23,890][20347] Updated weights for policy 0, policy_version 615155 (0.0007) [2023-12-27 09:53:23,945][20347] Updated weights for policy 0, policy_version 615165 (0.0008) [2023-12-27 09:53:24,061][20381] Updated weights for policy 1, policy_version 614849 (0.0006) [2023-12-27 09:53:24,084][20292] KL-divergence is very high: 111.3436 [2023-12-27 09:53:24,119][20381] Updated weights for policy 1, policy_version 614859 (0.0006) [2023-12-27 09:53:24,183][20381] Updated weights for policy 1, policy_version 614869 (0.0005) [2023-12-27 09:53:24,239][20381] Updated weights for policy 1, policy_version 614879 (0.0005) [2023-12-27 09:53:24,646][20347] Updated weights for policy 0, policy_version 615175 (0.0010) [2023-12-27 09:53:24,707][20347] Updated weights for policy 0, policy_version 615185 (0.0010) [2023-12-27 09:53:24,760][20347] Updated weights for policy 0, policy_version 615195 (0.0011) [2023-12-27 09:53:24,852][20381] Updated weights for policy 1, policy_version 614889 (0.0007) [2023-12-27 09:53:24,910][20381] Updated weights for policy 1, policy_version 614899 (0.0007) [2023-12-27 09:53:24,965][20381] Updated weights for policy 1, policy_version 614909 (0.0007) [2023-12-27 09:53:25,444][20347] Updated weights for policy 0, policy_version 615205 (0.0010) [2023-12-27 09:53:25,498][20347] Updated weights for policy 0, policy_version 615215 (0.0010) [2023-12-27 09:53:25,553][20347] Updated weights for policy 0, policy_version 615225 (0.0010) [2023-12-27 09:53:25,662][20381] Updated weights for policy 1, policy_version 614919 (0.0010) [2023-12-27 09:53:25,724][20381] Updated weights for policy 1, policy_version 614929 (0.0010) [2023-12-27 09:53:25,773][20381] Updated weights for policy 1, policy_version 614939 (0.0011) [2023-12-27 09:53:26,224][20347] Updated weights for policy 0, policy_version 615235 (0.0010) [2023-12-27 09:53:26,277][20347] Updated weights for policy 0, policy_version 615245 (0.0010) [2023-12-27 09:53:26,331][20347] Updated weights for policy 0, policy_version 615255 (0.0010) [2023-12-27 09:53:26,391][20381] Updated weights for policy 1, policy_version 614949 (0.0008) [2023-12-27 09:53:26,449][20381] Updated weights for policy 1, policy_version 614959 (0.0007) [2023-12-27 09:53:26,506][20381] Updated weights for policy 1, policy_version 614969 (0.0007) [2023-12-27 09:53:27,006][20347] Updated weights for policy 0, policy_version 615265 (0.0010) [2023-12-27 09:53:27,063][20347] Updated weights for policy 0, policy_version 615275 (0.0010) [2023-12-27 09:53:27,112][20347] Updated weights for policy 0, policy_version 615285 (0.0007) [2023-12-27 09:53:27,131][20381] Updated weights for policy 1, policy_version 614979 (0.0007) [2023-12-27 09:53:27,166][20347] Updated weights for policy 0, policy_version 615295 (0.0005) [2023-12-27 09:53:27,184][20381] Updated weights for policy 1, policy_version 614989 (0.0008) [2023-12-27 09:53:27,236][20381] Updated weights for policy 1, policy_version 614999 (0.0008) [2023-12-27 09:53:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20827.1). Total num frames: 314998784. Throughput: 0: 10434.2, 1: 10563.4. Samples: 314982148. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:27,363][19170] Avg episode reward: [(0, '980.840'), (1, '835.590')] [2023-12-27 09:53:27,730][20347] Updated weights for policy 0, policy_version 615305 (0.0010) [2023-12-27 09:53:27,784][20347] Updated weights for policy 0, policy_version 615315 (0.0010) [2023-12-27 09:53:27,833][20347] Updated weights for policy 0, policy_version 615325 (0.0010) [2023-12-27 09:53:27,921][20381] Updated weights for policy 1, policy_version 615009 (0.0008) [2023-12-27 09:53:27,976][20381] Updated weights for policy 1, policy_version 615019 (0.0008) [2023-12-27 09:53:28,023][20381] Updated weights for policy 1, policy_version 615029 (0.0007) [2023-12-27 09:53:28,077][20381] Updated weights for policy 1, policy_version 615039 (0.0007) [2023-12-27 09:53:28,493][20347] Updated weights for policy 0, policy_version 615335 (0.0010) [2023-12-27 09:53:28,554][20347] Updated weights for policy 0, policy_version 615345 (0.0010) [2023-12-27 09:53:28,612][20347] Updated weights for policy 0, policy_version 615355 (0.0010) [2023-12-27 09:53:28,750][20381] Updated weights for policy 1, policy_version 615049 (0.0007) [2023-12-27 09:53:28,806][20381] Updated weights for policy 1, policy_version 615059 (0.0006) [2023-12-27 09:53:28,867][20381] Updated weights for policy 1, policy_version 615069 (0.0005) [2023-12-27 09:53:29,188][20347] Updated weights for policy 0, policy_version 615365 (0.0006) [2023-12-27 09:53:29,253][20347] Updated weights for policy 0, policy_version 615375 (0.0006) [2023-12-27 09:53:29,322][20347] Updated weights for policy 0, policy_version 615385 (0.0007) [2023-12-27 09:53:29,469][20381] Updated weights for policy 1, policy_version 615079 (0.0008) [2023-12-27 09:53:29,514][20381] Updated weights for policy 1, policy_version 615089 (0.0009) [2023-12-27 09:53:29,559][20381] Updated weights for policy 1, policy_version 615099 (0.0008) [2023-12-27 09:53:29,880][20347] Updated weights for policy 0, policy_version 615395 (0.0008) [2023-12-27 09:53:29,950][20347] Updated weights for policy 0, policy_version 615405 (0.0009) [2023-12-27 09:53:30,009][20347] Updated weights for policy 0, policy_version 615415 (0.0008) [2023-12-27 09:53:30,262][20381] Updated weights for policy 1, policy_version 615109 (0.0007) [2023-12-27 09:53:30,322][20381] Updated weights for policy 1, policy_version 615119 (0.0005) [2023-12-27 09:53:30,384][20381] Updated weights for policy 1, policy_version 615129 (0.0005) [2023-12-27 09:53:30,603][20347] Updated weights for policy 0, policy_version 615425 (0.0010) [2023-12-27 09:53:30,654][20347] Updated weights for policy 0, policy_version 615435 (0.0010) [2023-12-27 09:53:30,699][20347] Updated weights for policy 0, policy_version 615445 (0.0010) [2023-12-27 09:53:30,751][20347] Updated weights for policy 0, policy_version 615455 (0.0010) [2023-12-27 09:53:30,969][20381] Updated weights for policy 1, policy_version 615139 (0.0006) [2023-12-27 09:53:31,027][20381] Updated weights for policy 1, policy_version 615149 (0.0006) [2023-12-27 09:53:31,079][20381] Updated weights for policy 1, policy_version 615159 (0.0008) [2023-12-27 09:53:31,460][20347] Updated weights for policy 0, policy_version 615465 (0.0006) [2023-12-27 09:53:31,519][20347] Updated weights for policy 0, policy_version 615475 (0.0008) [2023-12-27 09:53:31,573][20347] Updated weights for policy 0, policy_version 615485 (0.0007) [2023-12-27 09:53:31,829][20381] Updated weights for policy 1, policy_version 615169 (0.0008) [2023-12-27 09:53:31,884][20381] Updated weights for policy 1, policy_version 615179 (0.0007) [2023-12-27 09:53:31,937][20381] Updated weights for policy 1, policy_version 615189 (0.0007) [2023-12-27 09:53:31,993][20381] Updated weights for policy 1, policy_version 615199 (0.0007) [2023-12-27 09:53:32,236][20347] Updated weights for policy 0, policy_version 615495 (0.0010) [2023-12-27 09:53:32,307][20347] Updated weights for policy 0, policy_version 615505 (0.0011) [2023-12-27 09:53:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 315097088. Throughput: 0: 10499.4, 1: 10591.6. Samples: 315110380. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:32,363][19170] Avg episode reward: [(0, '981.330'), (1, '865.360')] [2023-12-27 09:53:32,369][20347] Updated weights for policy 0, policy_version 615515 (0.0009) [2023-12-27 09:53:32,693][20381] Updated weights for policy 1, policy_version 615209 (0.0010) [2023-12-27 09:53:32,749][20381] Updated weights for policy 1, policy_version 615219 (0.0010) [2023-12-27 09:53:32,810][20381] Updated weights for policy 1, policy_version 615229 (0.0009) [2023-12-27 09:53:33,030][20347] Updated weights for policy 0, policy_version 615525 (0.0010) [2023-12-27 09:53:33,078][20347] Updated weights for policy 0, policy_version 615535 (0.0010) [2023-12-27 09:53:33,119][20347] Updated weights for policy 0, policy_version 615545 (0.0010) [2023-12-27 09:53:33,475][20381] Updated weights for policy 1, policy_version 615239 (0.0007) [2023-12-27 09:53:33,529][20381] Updated weights for policy 1, policy_version 615249 (0.0010) [2023-12-27 09:53:33,583][20381] Updated weights for policy 1, policy_version 615259 (0.0010) [2023-12-27 09:53:33,804][20347] Updated weights for policy 0, policy_version 615555 (0.0010) [2023-12-27 09:53:33,865][20347] Updated weights for policy 0, policy_version 615565 (0.0010) [2023-12-27 09:53:33,912][20347] Updated weights for policy 0, policy_version 615575 (0.0010) [2023-12-27 09:53:34,239][20381] Updated weights for policy 1, policy_version 615269 (0.0009) [2023-12-27 09:53:34,295][20381] Updated weights for policy 1, policy_version 615279 (0.0007) [2023-12-27 09:53:34,348][20381] Updated weights for policy 1, policy_version 615289 (0.0007) [2023-12-27 09:53:34,582][20347] Updated weights for policy 0, policy_version 615585 (0.0010) [2023-12-27 09:53:34,635][20347] Updated weights for policy 0, policy_version 615595 (0.0010) [2023-12-27 09:53:34,697][20347] Updated weights for policy 0, policy_version 615605 (0.0010) [2023-12-27 09:53:34,751][20347] Updated weights for policy 0, policy_version 615615 (0.0008) [2023-12-27 09:53:35,004][20381] Updated weights for policy 1, policy_version 615299 (0.0006) [2023-12-27 09:53:35,054][20292] KL-divergence is very high: 131.4308 [2023-12-27 09:53:35,059][20381] Updated weights for policy 1, policy_version 615309 (0.0007) [2023-12-27 09:53:35,060][20292] KL-divergence is very high: 127.5393 [2023-12-27 09:53:35,115][20381] Updated weights for policy 1, policy_version 615319 (0.0005) [2023-12-27 09:53:35,364][20347] Updated weights for policy 0, policy_version 615625 (0.0010) [2023-12-27 09:53:35,421][20347] Updated weights for policy 0, policy_version 615635 (0.0005) [2023-12-27 09:53:35,477][20347] Updated weights for policy 0, policy_version 615645 (0.0005) [2023-12-27 09:53:35,778][20381] Updated weights for policy 1, policy_version 615329 (0.0006) [2023-12-27 09:53:35,830][20381] Updated weights for policy 1, policy_version 615339 (0.0007) [2023-12-27 09:53:35,882][20381] Updated weights for policy 1, policy_version 615349 (0.0008) [2023-12-27 09:53:35,944][20381] Updated weights for policy 1, policy_version 615359 (0.0007) [2023-12-27 09:53:36,118][20347] Updated weights for policy 0, policy_version 615655 (0.0008) [2023-12-27 09:53:36,173][20347] Updated weights for policy 0, policy_version 615665 (0.0008) [2023-12-27 09:53:36,228][20347] Updated weights for policy 0, policy_version 615675 (0.0007) [2023-12-27 09:53:36,628][20381] Updated weights for policy 1, policy_version 615369 (0.0008) [2023-12-27 09:53:36,664][20292] KL-divergence is very high: 129.4599 [2023-12-27 09:53:36,691][20381] Updated weights for policy 1, policy_version 615379 (0.0008) [2023-12-27 09:53:36,742][20292] KL-divergence is very high: 150.0326 [2023-12-27 09:53:36,747][20381] Updated weights for policy 1, policy_version 615389 (0.0008) [2023-12-27 09:53:36,754][20292] KL-divergence is very high: 170.7136 [2023-12-27 09:53:36,760][20292] KL-divergence is very high: 159.6593 [2023-12-27 09:53:36,888][20347] Updated weights for policy 0, policy_version 615685 (0.0008) [2023-12-27 09:53:36,944][20347] Updated weights for policy 0, policy_version 615695 (0.0008) [2023-12-27 09:53:37,004][20347] Updated weights for policy 0, policy_version 615705 (0.0006) [2023-12-27 09:53:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 315203584. Throughput: 0: 10490.1, 1: 10605.8. Samples: 315173352. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:37,363][19170] Avg episode reward: [(0, '943.740'), (1, '745.220')] [2023-12-27 09:53:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000615712_157638656.pth... [2023-12-27 09:53:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000614496_157327360.pth [2023-12-27 09:53:37,409][20381] Updated weights for policy 1, policy_version 615399 (0.0008) [2023-12-27 09:53:37,410][20292] KL-divergence is very high: 289.0206 [2023-12-27 09:53:37,450][20292] KL-divergence is very high: 252.4005 [2023-12-27 09:53:37,459][20381] Updated weights for policy 1, policy_version 615409 (0.0006) [2023-12-27 09:53:37,492][20292] KL-divergence is very high: 113.9197 [2023-12-27 09:53:37,511][20381] Updated weights for policy 1, policy_version 615419 (0.0005) [2023-12-27 09:53:37,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000615424_157573120.pth... [2023-12-27 09:53:37,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000614176_157253632.pth [2023-12-27 09:53:37,702][20347] Updated weights for policy 0, policy_version 615715 (0.0007) [2023-12-27 09:53:37,751][20347] Updated weights for policy 0, policy_version 615725 (0.0007) [2023-12-27 09:53:37,800][20347] Updated weights for policy 0, policy_version 615735 (0.0008) [2023-12-27 09:53:38,064][20292] KL-divergence is very high: 104.4805 [2023-12-27 09:53:38,087][20381] Updated weights for policy 1, policy_version 615429 (0.0005) [2023-12-27 09:53:38,102][20292] KL-divergence is very high: 153.9661 [2023-12-27 09:53:38,138][20381] Updated weights for policy 1, policy_version 615439 (0.0006) [2023-12-27 09:53:38,144][20292] KL-divergence is very high: 148.6910 [2023-12-27 09:53:38,187][20292] KL-divergence is very high: 135.0263 [2023-12-27 09:53:38,192][20381] Updated weights for policy 1, policy_version 615449 (0.0007) [2023-12-27 09:53:38,480][20347] Updated weights for policy 0, policy_version 615745 (0.0008) [2023-12-27 09:53:38,540][20347] Updated weights for policy 0, policy_version 615755 (0.0005) [2023-12-27 09:53:38,600][20347] Updated weights for policy 0, policy_version 615765 (0.0006) [2023-12-27 09:53:38,662][20347] Updated weights for policy 0, policy_version 615775 (0.0007) [2023-12-27 09:53:38,871][20381] Updated weights for policy 1, policy_version 615459 (0.0008) [2023-12-27 09:53:38,930][20381] Updated weights for policy 1, policy_version 615469 (0.0005) [2023-12-27 09:53:38,986][20381] Updated weights for policy 1, policy_version 615479 (0.0005) [2023-12-27 09:53:39,278][20347] Updated weights for policy 0, policy_version 615785 (0.0009) [2023-12-27 09:53:39,370][20347] Updated weights for policy 0, policy_version 615795 (0.0008) [2023-12-27 09:53:39,434][20347] Updated weights for policy 0, policy_version 615805 (0.0011) [2023-12-27 09:53:39,677][20381] Updated weights for policy 1, policy_version 615489 (0.0006) [2023-12-27 09:53:39,736][20381] Updated weights for policy 1, policy_version 615499 (0.0010) [2023-12-27 09:53:39,792][20381] Updated weights for policy 1, policy_version 615509 (0.0010) [2023-12-27 09:53:39,856][20381] Updated weights for policy 1, policy_version 615519 (0.0011) [2023-12-27 09:53:40,087][20347] Updated weights for policy 0, policy_version 615815 (0.0010) [2023-12-27 09:53:40,145][20347] Updated weights for policy 0, policy_version 615825 (0.0011) [2023-12-27 09:53:40,203][20347] Updated weights for policy 0, policy_version 615835 (0.0011) [2023-12-27 09:53:40,629][20381] Updated weights for policy 1, policy_version 615529 (0.0011) [2023-12-27 09:53:40,688][20381] Updated weights for policy 1, policy_version 615539 (0.0011) [2023-12-27 09:53:40,741][20381] Updated weights for policy 1, policy_version 615549 (0.0008) [2023-12-27 09:53:40,879][20347] Updated weights for policy 0, policy_version 615845 (0.0009) [2023-12-27 09:53:40,942][20347] Updated weights for policy 0, policy_version 615855 (0.0007) [2023-12-27 09:53:40,995][20347] Updated weights for policy 0, policy_version 615865 (0.0010) [2023-12-27 09:53:41,466][20381] Updated weights for policy 1, policy_version 615559 (0.0006) [2023-12-27 09:53:41,518][20381] Updated weights for policy 1, policy_version 615569 (0.0011) [2023-12-27 09:53:41,572][20381] Updated weights for policy 1, policy_version 615579 (0.0008) [2023-12-27 09:53:41,712][20347] Updated weights for policy 0, policy_version 615875 (0.0009) [2023-12-27 09:53:41,778][20347] Updated weights for policy 0, policy_version 615885 (0.0009) [2023-12-27 09:53:41,834][20347] Updated weights for policy 0, policy_version 615895 (0.0007) [2023-12-27 09:53:42,228][20381] Updated weights for policy 1, policy_version 615589 (0.0005) [2023-12-27 09:53:42,285][20381] Updated weights for policy 1, policy_version 615599 (0.0010) [2023-12-27 09:53:42,349][20381] Updated weights for policy 1, policy_version 615609 (0.0008) [2023-12-27 09:53:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 315301888. Throughput: 0: 10440.1, 1: 10585.5. Samples: 315297280. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:42,363][19170] Avg episode reward: [(0, '911.810'), (1, '774.870')] [2023-12-27 09:53:42,490][20347] Updated weights for policy 0, policy_version 615905 (0.0007) [2023-12-27 09:53:42,547][20347] Updated weights for policy 0, policy_version 615915 (0.0008) [2023-12-27 09:53:42,599][20347] Updated weights for policy 0, policy_version 615925 (0.0008) [2023-12-27 09:53:42,659][20347] Updated weights for policy 0, policy_version 615935 (0.0010) [2023-12-27 09:53:43,057][20381] Updated weights for policy 1, policy_version 615619 (0.0008) [2023-12-27 09:53:43,110][20381] Updated weights for policy 1, policy_version 615629 (0.0008) [2023-12-27 09:53:43,159][20381] Updated weights for policy 1, policy_version 615639 (0.0009) [2023-12-27 09:53:43,309][20347] Updated weights for policy 0, policy_version 615945 (0.0010) [2023-12-27 09:53:43,354][20347] Updated weights for policy 0, policy_version 615955 (0.0010) [2023-12-27 09:53:43,399][20347] Updated weights for policy 0, policy_version 615965 (0.0010) [2023-12-27 09:53:43,876][20381] Updated weights for policy 1, policy_version 615649 (0.0007) [2023-12-27 09:53:43,938][20381] Updated weights for policy 1, policy_version 615659 (0.0011) [2023-12-27 09:53:44,002][20381] Updated weights for policy 1, policy_version 615669 (0.0008) [2023-12-27 09:53:44,058][20347] Updated weights for policy 0, policy_version 615975 (0.0010) [2023-12-27 09:53:44,065][20381] Updated weights for policy 1, policy_version 615679 (0.0005) [2023-12-27 09:53:44,120][20347] Updated weights for policy 0, policy_version 615985 (0.0010) [2023-12-27 09:53:44,184][20347] Updated weights for policy 0, policy_version 615995 (0.0010) [2023-12-27 09:53:44,673][20381] Updated weights for policy 1, policy_version 615689 (0.0010) [2023-12-27 09:53:44,726][20381] Updated weights for policy 1, policy_version 615699 (0.0011) [2023-12-27 09:53:44,783][20381] Updated weights for policy 1, policy_version 615709 (0.0010) [2023-12-27 09:53:44,799][20347] Updated weights for policy 0, policy_version 616005 (0.0010) [2023-12-27 09:53:44,852][20347] Updated weights for policy 0, policy_version 616015 (0.0011) [2023-12-27 09:53:44,905][20347] Updated weights for policy 0, policy_version 616025 (0.0008) [2023-12-27 09:53:45,483][20381] Updated weights for policy 1, policy_version 615719 (0.0007) [2023-12-27 09:53:45,545][20381] Updated weights for policy 1, policy_version 615729 (0.0005) [2023-12-27 09:53:45,608][20381] Updated weights for policy 1, policy_version 615739 (0.0008) [2023-12-27 09:53:45,640][20347] Updated weights for policy 0, policy_version 616035 (0.0008) [2023-12-27 09:53:45,686][20347] Updated weights for policy 0, policy_version 616045 (0.0007) [2023-12-27 09:53:45,739][20347] Updated weights for policy 0, policy_version 616055 (0.0007) [2023-12-27 09:53:46,272][20381] Updated weights for policy 1, policy_version 615749 (0.0006) [2023-12-27 09:53:46,333][20381] Updated weights for policy 1, policy_version 615759 (0.0005) [2023-12-27 09:53:46,366][20347] Updated weights for policy 0, policy_version 616065 (0.0008) [2023-12-27 09:53:46,396][20381] Updated weights for policy 1, policy_version 615769 (0.0005) [2023-12-27 09:53:46,418][20347] Updated weights for policy 0, policy_version 616075 (0.0005) [2023-12-27 09:53:46,475][20347] Updated weights for policy 0, policy_version 616085 (0.0010) [2023-12-27 09:53:46,538][20347] Updated weights for policy 0, policy_version 616095 (0.0010) [2023-12-27 09:53:46,956][20381] Updated weights for policy 1, policy_version 615779 (0.0007) [2023-12-27 09:53:47,017][20381] Updated weights for policy 1, policy_version 615789 (0.0007) [2023-12-27 09:53:47,076][20381] Updated weights for policy 1, policy_version 615799 (0.0006) [2023-12-27 09:53:47,188][20347] Updated weights for policy 0, policy_version 616105 (0.0007) [2023-12-27 09:53:47,248][20347] Updated weights for policy 0, policy_version 616115 (0.0006) [2023-12-27 09:53:47,311][20347] Updated weights for policy 0, policy_version 616125 (0.0007) [2023-12-27 09:53:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 315416576. Throughput: 0: 10471.6, 1: 10510.6. Samples: 315422024. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:47,363][19170] Avg episode reward: [(0, '929.140'), (1, '756.660')] [2023-12-27 09:53:47,672][20381] Updated weights for policy 1, policy_version 615809 (0.0005) [2023-12-27 09:53:47,730][20381] Updated weights for policy 1, policy_version 615819 (0.0007) [2023-12-27 09:53:47,778][20381] Updated weights for policy 1, policy_version 615829 (0.0005) [2023-12-27 09:53:47,834][20381] Updated weights for policy 1, policy_version 615839 (0.0005) [2023-12-27 09:53:47,870][20347] Updated weights for policy 0, policy_version 616135 (0.0005) [2023-12-27 09:53:47,930][20347] Updated weights for policy 0, policy_version 616145 (0.0006) [2023-12-27 09:53:47,978][20347] Updated weights for policy 0, policy_version 616155 (0.0008) [2023-12-27 09:53:48,383][20381] Updated weights for policy 1, policy_version 615849 (0.0007) [2023-12-27 09:53:48,435][20381] Updated weights for policy 1, policy_version 615859 (0.0007) [2023-12-27 09:53:48,494][20381] Updated weights for policy 1, policy_version 615869 (0.0007) [2023-12-27 09:53:48,587][20347] Updated weights for policy 0, policy_version 616165 (0.0007) [2023-12-27 09:53:48,645][20347] Updated weights for policy 0, policy_version 616175 (0.0007) [2023-12-27 09:53:48,706][20347] Updated weights for policy 0, policy_version 616185 (0.0008) [2023-12-27 09:53:49,073][20381] Updated weights for policy 1, policy_version 615879 (0.0006) [2023-12-27 09:53:49,129][20381] Updated weights for policy 1, policy_version 615889 (0.0005) [2023-12-27 09:53:49,181][20381] Updated weights for policy 1, policy_version 615899 (0.0005) [2023-12-27 09:53:49,480][20347] Updated weights for policy 0, policy_version 616195 (0.0008) [2023-12-27 09:53:49,538][20347] Updated weights for policy 0, policy_version 616205 (0.0007) [2023-12-27 09:53:49,609][20347] Updated weights for policy 0, policy_version 616215 (0.0006) [2023-12-27 09:53:49,880][20381] Updated weights for policy 1, policy_version 615909 (0.0006) [2023-12-27 09:53:49,939][20381] Updated weights for policy 1, policy_version 615919 (0.0007) [2023-12-27 09:53:49,993][20381] Updated weights for policy 1, policy_version 615929 (0.0006) [2023-12-27 09:53:50,276][20347] Updated weights for policy 0, policy_version 616225 (0.0005) [2023-12-27 09:53:50,332][20347] Updated weights for policy 0, policy_version 616235 (0.0008) [2023-12-27 09:53:50,385][20347] Updated weights for policy 0, policy_version 616245 (0.0008) [2023-12-27 09:53:50,442][20347] Updated weights for policy 0, policy_version 616255 (0.0005) [2023-12-27 09:53:50,674][20381] Updated weights for policy 1, policy_version 615939 (0.0006) [2023-12-27 09:53:50,727][20381] Updated weights for policy 1, policy_version 615949 (0.0005) [2023-12-27 09:53:50,784][20381] Updated weights for policy 1, policy_version 615959 (0.0005) [2023-12-27 09:53:51,044][20347] Updated weights for policy 0, policy_version 616265 (0.0008) [2023-12-27 09:53:51,102][20347] Updated weights for policy 0, policy_version 616275 (0.0007) [2023-12-27 09:53:51,165][20347] Updated weights for policy 0, policy_version 616285 (0.0008) [2023-12-27 09:53:51,461][20381] Updated weights for policy 1, policy_version 615969 (0.0005) [2023-12-27 09:53:51,518][20381] Updated weights for policy 1, policy_version 615979 (0.0008) [2023-12-27 09:53:51,573][20381] Updated weights for policy 1, policy_version 615989 (0.0007) [2023-12-27 09:53:51,635][20381] Updated weights for policy 1, policy_version 615999 (0.0007) [2023-12-27 09:53:51,866][20347] Updated weights for policy 0, policy_version 616295 (0.0008) [2023-12-27 09:53:51,930][20347] Updated weights for policy 0, policy_version 616305 (0.0008) [2023-12-27 09:53:52,001][20347] Updated weights for policy 0, policy_version 616315 (0.0007) [2023-12-27 09:53:52,341][20381] Updated weights for policy 1, policy_version 616009 (0.0007) [2023-12-27 09:53:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 315514880. Throughput: 0: 10465.7, 1: 10498.5. Samples: 315486912. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:52,363][19170] Avg episode reward: [(0, '990.410'), (1, '766.420')] [2023-12-27 09:53:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000616320_157794304.pth... [2023-12-27 09:53:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000615104_157483008.pth [2023-12-27 09:53:52,402][20381] Updated weights for policy 1, policy_version 616019 (0.0009) [2023-12-27 09:53:52,470][20381] Updated weights for policy 1, policy_version 616029 (0.0008) [2023-12-27 09:53:52,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000616032_157728768.pth... [2023-12-27 09:53:52,492][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000614784_157409280.pth [2023-12-27 09:53:52,657][20347] Updated weights for policy 0, policy_version 616325 (0.0006) [2023-12-27 09:53:52,716][20347] Updated weights for policy 0, policy_version 616335 (0.0007) [2023-12-27 09:53:52,771][20347] Updated weights for policy 0, policy_version 616345 (0.0008) [2023-12-27 09:53:53,133][20381] Updated weights for policy 1, policy_version 616039 (0.0008) [2023-12-27 09:53:53,185][20381] Updated weights for policy 1, policy_version 616049 (0.0008) [2023-12-27 09:53:53,241][20381] Updated weights for policy 1, policy_version 616059 (0.0007) [2023-12-27 09:53:53,485][20347] Updated weights for policy 0, policy_version 616355 (0.0009) [2023-12-27 09:53:53,534][20347] Updated weights for policy 0, policy_version 616365 (0.0006) [2023-12-27 09:53:53,595][20347] Updated weights for policy 0, policy_version 616375 (0.0005) [2023-12-27 09:53:53,840][20381] Updated weights for policy 1, policy_version 616069 (0.0009) [2023-12-27 09:53:53,893][20381] Updated weights for policy 1, policy_version 616079 (0.0008) [2023-12-27 09:53:53,948][20381] Updated weights for policy 1, policy_version 616089 (0.0009) [2023-12-27 09:53:54,259][20347] Updated weights for policy 0, policy_version 616385 (0.0007) [2023-12-27 09:53:54,310][20347] Updated weights for policy 0, policy_version 616395 (0.0010) [2023-12-27 09:53:54,359][20347] Updated weights for policy 0, policy_version 616405 (0.0010) [2023-12-27 09:53:54,419][20347] Updated weights for policy 0, policy_version 616415 (0.0009) [2023-12-27 09:53:54,541][20381] Updated weights for policy 1, policy_version 616099 (0.0006) [2023-12-27 09:53:54,590][20381] Updated weights for policy 1, policy_version 616109 (0.0005) [2023-12-27 09:53:54,640][20381] Updated weights for policy 1, policy_version 616119 (0.0005) [2023-12-27 09:53:55,095][20347] Updated weights for policy 0, policy_version 616425 (0.0010) [2023-12-27 09:53:55,150][20347] Updated weights for policy 0, policy_version 616435 (0.0010) [2023-12-27 09:53:55,204][20347] Updated weights for policy 0, policy_version 616445 (0.0010) [2023-12-27 09:53:55,246][20381] Updated weights for policy 1, policy_version 616129 (0.0006) [2023-12-27 09:53:55,298][20381] Updated weights for policy 1, policy_version 616139 (0.0006) [2023-12-27 09:53:55,354][20381] Updated weights for policy 1, policy_version 616149 (0.0006) [2023-12-27 09:53:55,402][20381] Updated weights for policy 1, policy_version 616159 (0.0007) [2023-12-27 09:53:55,893][20347] Updated weights for policy 0, policy_version 616455 (0.0010) [2023-12-27 09:53:55,954][20347] Updated weights for policy 0, policy_version 616465 (0.0008) [2023-12-27 09:53:56,008][20347] Updated weights for policy 0, policy_version 616475 (0.0010) [2023-12-27 09:53:56,032][20381] Updated weights for policy 1, policy_version 616169 (0.0006) [2023-12-27 09:53:56,080][20381] Updated weights for policy 1, policy_version 616179 (0.0006) [2023-12-27 09:53:56,139][20381] Updated weights for policy 1, policy_version 616189 (0.0005) [2023-12-27 09:53:56,670][20347] Updated weights for policy 0, policy_version 616485 (0.0007) [2023-12-27 09:53:56,727][20347] Updated weights for policy 0, policy_version 616495 (0.0005) [2023-12-27 09:53:56,731][20381] Updated weights for policy 1, policy_version 616199 (0.0009) [2023-12-27 09:53:56,782][20347] Updated weights for policy 0, policy_version 616505 (0.0007) [2023-12-27 09:53:56,785][20381] Updated weights for policy 1, policy_version 616209 (0.0010) [2023-12-27 09:53:56,844][20381] Updated weights for policy 1, policy_version 616219 (0.0010) [2023-12-27 09:53:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 315621376. Throughput: 0: 10409.3, 1: 10497.1. Samples: 315612752. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:53:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '734.600')] [2023-12-27 09:53:57,395][20347] Updated weights for policy 0, policy_version 616515 (0.0009) [2023-12-27 09:53:57,441][20381] Updated weights for policy 1, policy_version 616229 (0.0011) [2023-12-27 09:53:57,447][20347] Updated weights for policy 0, policy_version 616525 (0.0010) [2023-12-27 09:53:57,494][20347] Updated weights for policy 0, policy_version 616535 (0.0010) [2023-12-27 09:53:57,496][20381] Updated weights for policy 1, policy_version 616239 (0.0010) [2023-12-27 09:53:57,546][20381] Updated weights for policy 1, policy_version 616249 (0.0009) [2023-12-27 09:53:58,100][20381] Updated weights for policy 1, policy_version 616259 (0.0007) [2023-12-27 09:53:58,133][20347] Updated weights for policy 0, policy_version 616545 (0.0010) [2023-12-27 09:53:58,157][20381] Updated weights for policy 1, policy_version 616269 (0.0005) [2023-12-27 09:53:58,191][20347] Updated weights for policy 0, policy_version 616555 (0.0009) [2023-12-27 09:53:58,220][20381] Updated weights for policy 1, policy_version 616279 (0.0008) [2023-12-27 09:53:58,250][20347] Updated weights for policy 0, policy_version 616565 (0.0009) [2023-12-27 09:53:58,303][20347] Updated weights for policy 0, policy_version 616575 (0.0008) [2023-12-27 09:53:58,927][20381] Updated weights for policy 1, policy_version 616289 (0.0007) [2023-12-27 09:53:58,986][20381] Updated weights for policy 1, policy_version 616299 (0.0007) [2023-12-27 09:53:59,046][20381] Updated weights for policy 1, policy_version 616309 (0.0007) [2023-12-27 09:53:59,073][20347] Updated weights for policy 0, policy_version 616585 (0.0006) [2023-12-27 09:53:59,104][20381] Updated weights for policy 1, policy_version 616319 (0.0008) [2023-12-27 09:53:59,132][20347] Updated weights for policy 0, policy_version 616595 (0.0008) [2023-12-27 09:53:59,184][20347] Updated weights for policy 0, policy_version 616605 (0.0008) [2023-12-27 09:53:59,834][20381] Updated weights for policy 1, policy_version 616329 (0.0008) [2023-12-27 09:53:59,852][20347] Updated weights for policy 0, policy_version 616615 (0.0009) [2023-12-27 09:53:59,894][20381] Updated weights for policy 1, policy_version 616339 (0.0007) [2023-12-27 09:53:59,907][20347] Updated weights for policy 0, policy_version 616625 (0.0008) [2023-12-27 09:53:59,957][20381] Updated weights for policy 1, policy_version 616349 (0.0007) [2023-12-27 09:53:59,966][20347] Updated weights for policy 0, policy_version 616635 (0.0008) [2023-12-27 09:54:00,572][20347] Updated weights for policy 0, policy_version 616645 (0.0008) [2023-12-27 09:54:00,624][20347] Updated weights for policy 0, policy_version 616655 (0.0007) [2023-12-27 09:54:00,640][20381] Updated weights for policy 1, policy_version 616359 (0.0006) [2023-12-27 09:54:00,676][20347] Updated weights for policy 0, policy_version 616665 (0.0005) [2023-12-27 09:54:00,695][20381] Updated weights for policy 1, policy_version 616369 (0.0007) [2023-12-27 09:54:00,744][20381] Updated weights for policy 1, policy_version 616379 (0.0008) [2023-12-27 09:54:01,330][20347] Updated weights for policy 0, policy_version 616675 (0.0009) [2023-12-27 09:54:01,392][20347] Updated weights for policy 0, policy_version 616685 (0.0008) [2023-12-27 09:54:01,434][20347] Updated weights for policy 0, policy_version 616695 (0.0008) [2023-12-27 09:54:01,461][20381] Updated weights for policy 1, policy_version 616389 (0.0008) [2023-12-27 09:54:01,509][20381] Updated weights for policy 1, policy_version 616399 (0.0008) [2023-12-27 09:54:01,565][20381] Updated weights for policy 1, policy_version 616409 (0.0008) [2023-12-27 09:54:02,087][20347] Updated weights for policy 0, policy_version 616706 (0.0008) [2023-12-27 09:54:02,147][20347] Updated weights for policy 0, policy_version 616716 (0.0010) [2023-12-27 09:54:02,207][20347] Updated weights for policy 0, policy_version 616726 (0.0008) [2023-12-27 09:54:02,226][20381] Updated weights for policy 1, policy_version 616419 (0.0008) [2023-12-27 09:54:02,265][20347] Updated weights for policy 0, policy_version 616736 (0.0007) [2023-12-27 09:54:02,287][20381] Updated weights for policy 1, policy_version 616429 (0.0006) [2023-12-27 09:54:02,351][20381] Updated weights for policy 1, policy_version 616439 (0.0006) [2023-12-27 09:54:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 20854.9). Total num frames: 315727872. Throughput: 0: 10447.2, 1: 10486.4. Samples: 315739352. Policy #0 lag: (min: 10.0, avg: 22.9, max: 42.0) [2023-12-27 09:54:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '748.980')] [2023-12-27 09:54:02,894][20347] Updated weights for policy 0, policy_version 616746 (0.0010) [2023-12-27 09:54:02,948][20347] Updated weights for policy 0, policy_version 616756 (0.0010) [2023-12-27 09:54:02,994][20347] Updated weights for policy 0, policy_version 616766 (0.0009) [2023-12-27 09:54:03,060][20381] Updated weights for policy 1, policy_version 616449 (0.0007) [2023-12-27 09:54:03,115][20381] Updated weights for policy 1, policy_version 616459 (0.0008) [2023-12-27 09:54:03,170][20381] Updated weights for policy 1, policy_version 616469 (0.0007) [2023-12-27 09:54:03,221][20381] Updated weights for policy 1, policy_version 616479 (0.0005) [2023-12-27 09:54:03,739][20347] Updated weights for policy 0, policy_version 616776 (0.0007) [2023-12-27 09:54:03,752][20381] Updated weights for policy 1, policy_version 616489 (0.0006) [2023-12-27 09:54:03,792][20347] Updated weights for policy 0, policy_version 616786 (0.0005) [2023-12-27 09:54:03,807][20381] Updated weights for policy 1, policy_version 616499 (0.0005) [2023-12-27 09:54:03,843][20347] Updated weights for policy 0, policy_version 616796 (0.0006) [2023-12-27 09:54:03,867][20381] Updated weights for policy 1, policy_version 616509 (0.0007) [2023-12-27 09:54:04,464][20381] Updated weights for policy 1, policy_version 616519 (0.0006) [2023-12-27 09:54:04,521][20381] Updated weights for policy 1, policy_version 616529 (0.0006) [2023-12-27 09:54:04,522][20347] Updated weights for policy 0, policy_version 616806 (0.0007) [2023-12-27 09:54:04,576][20381] Updated weights for policy 1, policy_version 616539 (0.0006) [2023-12-27 09:54:04,582][20347] Updated weights for policy 0, policy_version 616816 (0.0009) [2023-12-27 09:54:04,641][20347] Updated weights for policy 0, policy_version 616826 (0.0011) [2023-12-27 09:54:05,227][20381] Updated weights for policy 1, policy_version 616549 (0.0006) [2023-12-27 09:54:05,259][20347] Updated weights for policy 0, policy_version 616836 (0.0010) [2023-12-27 09:54:05,279][20381] Updated weights for policy 1, policy_version 616559 (0.0005) [2023-12-27 09:54:05,308][20347] Updated weights for policy 0, policy_version 616846 (0.0010) [2023-12-27 09:54:05,335][20381] Updated weights for policy 1, policy_version 616569 (0.0006) [2023-12-27 09:54:05,359][20347] Updated weights for policy 0, policy_version 616856 (0.0010) [2023-12-27 09:54:05,984][20381] Updated weights for policy 1, policy_version 616579 (0.0009) [2023-12-27 09:54:05,994][20347] Updated weights for policy 0, policy_version 616866 (0.0009) [2023-12-27 09:54:06,032][20381] Updated weights for policy 1, policy_version 616589 (0.0006) [2023-12-27 09:54:06,041][20347] Updated weights for policy 0, policy_version 616876 (0.0008) [2023-12-27 09:54:06,078][20381] Updated weights for policy 1, policy_version 616599 (0.0006) [2023-12-27 09:54:06,087][20347] Updated weights for policy 0, policy_version 616886 (0.0010) [2023-12-27 09:54:06,147][20347] Updated weights for policy 0, policy_version 616896 (0.0009) [2023-12-27 09:54:06,709][20381] Updated weights for policy 1, policy_version 616609 (0.0006) [2023-12-27 09:54:06,770][20381] Updated weights for policy 1, policy_version 616619 (0.0005) [2023-12-27 09:54:06,829][20347] Updated weights for policy 0, policy_version 616906 (0.0010) [2023-12-27 09:54:06,832][20381] Updated weights for policy 1, policy_version 616629 (0.0006) [2023-12-27 09:54:06,882][20381] Updated weights for policy 1, policy_version 616639 (0.0005) [2023-12-27 09:54:06,886][20347] Updated weights for policy 0, policy_version 616916 (0.0009) [2023-12-27 09:54:06,952][20347] Updated weights for policy 0, policy_version 616926 (0.0008) [2023-12-27 09:54:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.7, 300 sec: 20882.7). Total num frames: 315834368. Throughput: 0: 10467.0, 1: 10556.9. Samples: 315803376. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '790.930')] [2023-12-27 09:54:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000616928_157949952.pth... [2023-12-27 09:54:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000616640_157884416.pth... [2023-12-27 09:54:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000615712_157638656.pth [2023-12-27 09:54:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000615424_157573120.pth [2023-12-27 09:54:07,506][20381] Updated weights for policy 1, policy_version 616649 (0.0007) [2023-12-27 09:54:07,556][20381] Updated weights for policy 1, policy_version 616659 (0.0007) [2023-12-27 09:54:07,606][20381] Updated weights for policy 1, policy_version 616669 (0.0008) [2023-12-27 09:54:07,617][20347] Updated weights for policy 0, policy_version 616936 (0.0010) [2023-12-27 09:54:07,665][20347] Updated weights for policy 0, policy_version 616946 (0.0010) [2023-12-27 09:54:07,719][20347] Updated weights for policy 0, policy_version 616956 (0.0009) [2023-12-27 09:54:08,271][20381] Updated weights for policy 1, policy_version 616679 (0.0007) [2023-12-27 09:54:08,323][20347] Updated weights for policy 0, policy_version 616966 (0.0008) [2023-12-27 09:54:08,325][20381] Updated weights for policy 1, policy_version 616689 (0.0006) [2023-12-27 09:54:08,371][20292] KL-divergence is very high: 168.4523 [2023-12-27 09:54:08,378][20292] KL-divergence is very high: 187.7618 [2023-12-27 09:54:08,383][20347] Updated weights for policy 0, policy_version 616976 (0.0010) [2023-12-27 09:54:08,385][20292] KL-divergence is very high: 305.4923 [2023-12-27 09:54:08,391][20381] Updated weights for policy 1, policy_version 616699 (0.0008) [2023-12-27 09:54:08,397][20292] KL-divergence is very high: 134.4971 [2023-12-27 09:54:08,429][20347] Updated weights for policy 0, policy_version 616986 (0.0010) [2023-12-27 09:54:09,048][20347] Updated weights for policy 0, policy_version 616996 (0.0009) [2023-12-27 09:54:09,110][20381] Updated weights for policy 1, policy_version 616709 (0.0009) [2023-12-27 09:54:09,111][20347] Updated weights for policy 0, policy_version 617006 (0.0005) [2023-12-27 09:54:09,171][20347] Updated weights for policy 0, policy_version 617016 (0.0006) [2023-12-27 09:54:09,174][20381] Updated weights for policy 1, policy_version 616719 (0.0008) [2023-12-27 09:54:09,244][20381] Updated weights for policy 1, policy_version 616729 (0.0007) [2023-12-27 09:54:09,817][20347] Updated weights for policy 0, policy_version 617026 (0.0009) [2023-12-27 09:54:09,875][20347] Updated weights for policy 0, policy_version 617036 (0.0011) [2023-12-27 09:54:09,932][20347] Updated weights for policy 0, policy_version 617046 (0.0010) [2023-12-27 09:54:09,949][20381] Updated weights for policy 1, policy_version 616739 (0.0008) [2023-12-27 09:54:09,996][20347] Updated weights for policy 0, policy_version 617056 (0.0009) [2023-12-27 09:54:10,008][20381] Updated weights for policy 1, policy_version 616749 (0.0006) [2023-12-27 09:54:10,063][20381] Updated weights for policy 1, policy_version 616759 (0.0005) [2023-12-27 09:54:10,674][20347] Updated weights for policy 0, policy_version 617066 (0.0009) [2023-12-27 09:54:10,731][20347] Updated weights for policy 0, policy_version 617076 (0.0008) [2023-12-27 09:54:10,733][20381] Updated weights for policy 1, policy_version 616769 (0.0005) [2023-12-27 09:54:10,781][20381] Updated weights for policy 1, policy_version 616779 (0.0005) [2023-12-27 09:54:10,782][20347] Updated weights for policy 0, policy_version 617086 (0.0010) [2023-12-27 09:54:10,834][20381] Updated weights for policy 1, policy_version 616789 (0.0007) [2023-12-27 09:54:10,881][20381] Updated weights for policy 1, policy_version 616799 (0.0007) [2023-12-27 09:54:11,501][20347] Updated weights for policy 0, policy_version 617096 (0.0009) [2023-12-27 09:54:11,563][20347] Updated weights for policy 0, policy_version 617106 (0.0006) [2023-12-27 09:54:11,623][20347] Updated weights for policy 0, policy_version 617116 (0.0006) [2023-12-27 09:54:11,645][20381] Updated weights for policy 1, policy_version 616809 (0.0008) [2023-12-27 09:54:11,704][20381] Updated weights for policy 1, policy_version 616819 (0.0009) [2023-12-27 09:54:11,754][20381] Updated weights for policy 1, policy_version 616829 (0.0006) [2023-12-27 09:54:12,333][20347] Updated weights for policy 0, policy_version 617126 (0.0009) [2023-12-27 09:54:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 315932672. Throughput: 0: 10503.5, 1: 10528.7. Samples: 315928600. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '798.940')] [2023-12-27 09:54:12,395][20347] Updated weights for policy 0, policy_version 617136 (0.0008) [2023-12-27 09:54:12,403][20381] Updated weights for policy 1, policy_version 616839 (0.0006) [2023-12-27 09:54:12,451][20347] Updated weights for policy 0, policy_version 617146 (0.0008) [2023-12-27 09:54:12,453][20381] Updated weights for policy 1, policy_version 616849 (0.0006) [2023-12-27 09:54:12,506][20381] Updated weights for policy 1, policy_version 616859 (0.0005) [2023-12-27 09:54:13,177][20381] Updated weights for policy 1, policy_version 616869 (0.0007) [2023-12-27 09:54:13,193][20347] Updated weights for policy 0, policy_version 617156 (0.0008) [2023-12-27 09:54:13,235][20381] Updated weights for policy 1, policy_version 616879 (0.0006) [2023-12-27 09:54:13,244][20347] Updated weights for policy 0, policy_version 617166 (0.0007) [2023-12-27 09:54:13,294][20347] Updated weights for policy 0, policy_version 617176 (0.0007) [2023-12-27 09:54:13,297][20381] Updated weights for policy 1, policy_version 616889 (0.0005) [2023-12-27 09:54:13,953][20381] Updated weights for policy 1, policy_version 616899 (0.0007) [2023-12-27 09:54:13,969][20347] Updated weights for policy 0, policy_version 617186 (0.0006) [2023-12-27 09:54:14,008][20381] Updated weights for policy 1, policy_version 616909 (0.0006) [2023-12-27 09:54:14,028][20347] Updated weights for policy 0, policy_version 617196 (0.0006) [2023-12-27 09:54:14,062][20381] Updated weights for policy 1, policy_version 616919 (0.0007) [2023-12-27 09:54:14,089][20347] Updated weights for policy 0, policy_version 617206 (0.0005) [2023-12-27 09:54:14,145][20347] Updated weights for policy 0, policy_version 617216 (0.0005) [2023-12-27 09:54:14,709][20381] Updated weights for policy 1, policy_version 616929 (0.0008) [2023-12-27 09:54:14,754][20381] Updated weights for policy 1, policy_version 616939 (0.0008) [2023-12-27 09:54:14,811][20347] Updated weights for policy 0, policy_version 617226 (0.0008) [2023-12-27 09:54:14,820][20381] Updated weights for policy 1, policy_version 616949 (0.0007) [2023-12-27 09:54:14,871][20347] Updated weights for policy 0, policy_version 617236 (0.0008) [2023-12-27 09:54:14,877][20381] Updated weights for policy 1, policy_version 616959 (0.0007) [2023-12-27 09:54:14,915][20347] Updated weights for policy 0, policy_version 617246 (0.0006) [2023-12-27 09:54:15,570][20381] Updated weights for policy 1, policy_version 616969 (0.0007) [2023-12-27 09:54:15,608][20347] Updated weights for policy 0, policy_version 617256 (0.0008) [2023-12-27 09:54:15,626][20381] Updated weights for policy 1, policy_version 616979 (0.0005) [2023-12-27 09:54:15,672][20347] Updated weights for policy 0, policy_version 617266 (0.0007) [2023-12-27 09:54:15,681][20381] Updated weights for policy 1, policy_version 616989 (0.0005) [2023-12-27 09:54:15,725][20347] Updated weights for policy 0, policy_version 617276 (0.0006) [2023-12-27 09:54:16,355][20347] Updated weights for policy 0, policy_version 617286 (0.0006) [2023-12-27 09:54:16,367][20381] Updated weights for policy 1, policy_version 616999 (0.0009) [2023-12-27 09:54:16,405][20347] Updated weights for policy 0, policy_version 617296 (0.0007) [2023-12-27 09:54:16,423][20381] Updated weights for policy 1, policy_version 617009 (0.0010) [2023-12-27 09:54:16,463][20347] Updated weights for policy 0, policy_version 617306 (0.0007) [2023-12-27 09:54:16,482][20381] Updated weights for policy 1, policy_version 617019 (0.0011) [2023-12-27 09:54:17,148][20347] Updated weights for policy 0, policy_version 617316 (0.0009) [2023-12-27 09:54:17,165][20381] Updated weights for policy 1, policy_version 617029 (0.0011) [2023-12-27 09:54:17,199][20347] Updated weights for policy 0, policy_version 617326 (0.0005) [2023-12-27 09:54:17,221][20381] Updated weights for policy 1, policy_version 617039 (0.0010) [2023-12-27 09:54:17,248][20347] Updated weights for policy 0, policy_version 617336 (0.0006) [2023-12-27 09:54:17,276][20381] Updated weights for policy 1, policy_version 617049 (0.0011) [2023-12-27 09:54:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 316047360. Throughput: 0: 10394.1, 1: 10506.0. Samples: 316050880. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '567.170')] [2023-12-27 09:54:17,829][20381] Updated weights for policy 1, policy_version 617059 (0.0011) [2023-12-27 09:54:17,880][20381] Updated weights for policy 1, policy_version 617069 (0.0011) [2023-12-27 09:54:17,926][20381] Updated weights for policy 1, policy_version 617079 (0.0010) [2023-12-27 09:54:17,989][20347] Updated weights for policy 0, policy_version 617346 (0.0006) [2023-12-27 09:54:18,039][20347] Updated weights for policy 0, policy_version 617356 (0.0007) [2023-12-27 09:54:18,100][20347] Updated weights for policy 0, policy_version 617366 (0.0007) [2023-12-27 09:54:18,147][20347] Updated weights for policy 0, policy_version 617376 (0.0007) [2023-12-27 09:54:18,647][20381] Updated weights for policy 1, policy_version 617089 (0.0010) [2023-12-27 09:54:18,709][20381] Updated weights for policy 1, policy_version 617099 (0.0011) [2023-12-27 09:54:18,762][20381] Updated weights for policy 1, policy_version 617109 (0.0011) [2023-12-27 09:54:18,818][20381] Updated weights for policy 1, policy_version 617119 (0.0011) [2023-12-27 09:54:18,843][20347] Updated weights for policy 0, policy_version 617386 (0.0008) [2023-12-27 09:54:18,897][20347] Updated weights for policy 0, policy_version 617396 (0.0008) [2023-12-27 09:54:18,953][20347] Updated weights for policy 0, policy_version 617406 (0.0007) [2023-12-27 09:54:19,537][20381] Updated weights for policy 1, policy_version 617129 (0.0008) [2023-12-27 09:54:19,589][20381] Updated weights for policy 1, policy_version 617139 (0.0008) [2023-12-27 09:54:19,635][20347] Updated weights for policy 0, policy_version 617416 (0.0006) [2023-12-27 09:54:19,648][20381] Updated weights for policy 1, policy_version 617149 (0.0007) [2023-12-27 09:54:19,693][20347] Updated weights for policy 0, policy_version 617426 (0.0006) [2023-12-27 09:54:19,752][20347] Updated weights for policy 0, policy_version 617436 (0.0008) [2023-12-27 09:54:20,452][20381] Updated weights for policy 1, policy_version 617159 (0.0007) [2023-12-27 09:54:20,483][20347] Updated weights for policy 0, policy_version 617446 (0.0009) [2023-12-27 09:54:20,513][20381] Updated weights for policy 1, policy_version 617169 (0.0007) [2023-12-27 09:54:20,541][20347] Updated weights for policy 0, policy_version 617456 (0.0007) [2023-12-27 09:54:20,581][20381] Updated weights for policy 1, policy_version 617179 (0.0007) [2023-12-27 09:54:20,601][20347] Updated weights for policy 0, policy_version 617466 (0.0007) [2023-12-27 09:54:21,249][20347] Updated weights for policy 0, policy_version 617476 (0.0008) [2023-12-27 09:54:21,314][20347] Updated weights for policy 0, policy_version 617486 (0.0007) [2023-12-27 09:54:21,321][20381] Updated weights for policy 1, policy_version 617189 (0.0007) [2023-12-27 09:54:21,381][20347] Updated weights for policy 0, policy_version 617496 (0.0007) [2023-12-27 09:54:21,388][20381] Updated weights for policy 1, policy_version 617199 (0.0008) [2023-12-27 09:54:21,439][20381] Updated weights for policy 1, policy_version 617209 (0.0008) [2023-12-27 09:54:22,078][20347] Updated weights for policy 0, policy_version 617506 (0.0006) [2023-12-27 09:54:22,133][20347] Updated weights for policy 0, policy_version 617516 (0.0006) [2023-12-27 09:54:22,189][20347] Updated weights for policy 0, policy_version 617526 (0.0008) [2023-12-27 09:54:22,198][20381] Updated weights for policy 1, policy_version 617219 (0.0008) [2023-12-27 09:54:22,239][20347] Updated weights for policy 0, policy_version 617536 (0.0005) [2023-12-27 09:54:22,256][20381] Updated weights for policy 1, policy_version 617229 (0.0007) [2023-12-27 09:54:22,317][20381] Updated weights for policy 1, policy_version 617239 (0.0007) [2023-12-27 09:54:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 316137472. Throughput: 0: 10343.4, 1: 10483.3. Samples: 316110552. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '320.370')] [2023-12-27 09:54:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000617536_158105600.pth... [2023-12-27 09:54:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000616320_157794304.pth [2023-12-27 09:54:22,376][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000617248_158040064.pth... [2023-12-27 09:54:22,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000616032_157728768.pth [2023-12-27 09:54:22,966][20381] Updated weights for policy 1, policy_version 617249 (0.0007) [2023-12-27 09:54:22,972][20347] Updated weights for policy 0, policy_version 617546 (0.0008) [2023-12-27 09:54:23,020][20381] Updated weights for policy 1, policy_version 617259 (0.0007) [2023-12-27 09:54:23,027][20347] Updated weights for policy 0, policy_version 617556 (0.0007) [2023-12-27 09:54:23,069][20381] Updated weights for policy 1, policy_version 617269 (0.0006) [2023-12-27 09:54:23,084][20347] Updated weights for policy 0, policy_version 617566 (0.0007) [2023-12-27 09:54:23,116][20381] Updated weights for policy 1, policy_version 617279 (0.0008) [2023-12-27 09:54:23,720][20347] Updated weights for policy 0, policy_version 617576 (0.0006) [2023-12-27 09:54:23,780][20347] Updated weights for policy 0, policy_version 617586 (0.0005) [2023-12-27 09:54:23,839][20347] Updated weights for policy 0, policy_version 617596 (0.0007) [2023-12-27 09:54:23,848][20381] Updated weights for policy 1, policy_version 617289 (0.0006) [2023-12-27 09:54:23,903][20381] Updated weights for policy 1, policy_version 617299 (0.0007) [2023-12-27 09:54:23,957][20381] Updated weights for policy 1, policy_version 617309 (0.0007) [2023-12-27 09:54:24,486][20347] Updated weights for policy 0, policy_version 617606 (0.0007) [2023-12-27 09:54:24,548][20347] Updated weights for policy 0, policy_version 617616 (0.0008) [2023-12-27 09:54:24,607][20347] Updated weights for policy 0, policy_version 617626 (0.0007) [2023-12-27 09:54:24,623][20381] Updated weights for policy 1, policy_version 617319 (0.0007) [2023-12-27 09:54:24,683][20381] Updated weights for policy 1, policy_version 617329 (0.0008) [2023-12-27 09:54:24,745][20381] Updated weights for policy 1, policy_version 617339 (0.0008) [2023-12-27 09:54:25,271][20347] Updated weights for policy 0, policy_version 617636 (0.0006) [2023-12-27 09:54:25,327][20347] Updated weights for policy 0, policy_version 617646 (0.0005) [2023-12-27 09:54:25,380][20347] Updated weights for policy 0, policy_version 617656 (0.0006) [2023-12-27 09:54:25,437][20381] Updated weights for policy 1, policy_version 617349 (0.0008) [2023-12-27 09:54:25,492][20381] Updated weights for policy 1, policy_version 617359 (0.0007) [2023-12-27 09:54:25,539][20381] Updated weights for policy 1, policy_version 617369 (0.0010) [2023-12-27 09:54:25,912][20347] Updated weights for policy 0, policy_version 617666 (0.0005) [2023-12-27 09:54:25,964][20347] Updated weights for policy 0, policy_version 617676 (0.0007) [2023-12-27 09:54:26,027][20347] Updated weights for policy 0, policy_version 617686 (0.0008) [2023-12-27 09:54:26,083][20347] Updated weights for policy 0, policy_version 617696 (0.0009) [2023-12-27 09:54:26,197][20381] Updated weights for policy 1, policy_version 617379 (0.0009) [2023-12-27 09:54:26,242][20381] Updated weights for policy 1, policy_version 617389 (0.0008) [2023-12-27 09:54:26,287][20381] Updated weights for policy 1, policy_version 617399 (0.0008) [2023-12-27 09:54:26,741][20347] Updated weights for policy 0, policy_version 617706 (0.0007) [2023-12-27 09:54:26,791][20347] Updated weights for policy 0, policy_version 617716 (0.0006) [2023-12-27 09:54:26,841][20347] Updated weights for policy 0, policy_version 617726 (0.0006) [2023-12-27 09:54:26,942][20381] Updated weights for policy 1, policy_version 617409 (0.0008) [2023-12-27 09:54:27,004][20381] Updated weights for policy 1, policy_version 617419 (0.0007) [2023-12-27 09:54:27,064][20381] Updated weights for policy 1, policy_version 617429 (0.0008) [2023-12-27 09:54:27,115][20381] Updated weights for policy 1, policy_version 617439 (0.0007) [2023-12-27 09:54:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 316243968. Throughput: 0: 10358.9, 1: 10435.3. Samples: 316233016. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '421.690')] [2023-12-27 09:54:27,540][20347] Updated weights for policy 0, policy_version 617737 (0.0008) [2023-12-27 09:54:27,598][20347] Updated weights for policy 0, policy_version 617747 (0.0006) [2023-12-27 09:54:27,642][20347] Updated weights for policy 0, policy_version 617757 (0.0007) [2023-12-27 09:54:27,748][20381] Updated weights for policy 1, policy_version 617449 (0.0008) [2023-12-27 09:54:27,796][20381] Updated weights for policy 1, policy_version 617459 (0.0007) [2023-12-27 09:54:27,844][20381] Updated weights for policy 1, policy_version 617469 (0.0006) [2023-12-27 09:54:28,307][20347] Updated weights for policy 0, policy_version 617767 (0.0005) [2023-12-27 09:54:28,368][20347] Updated weights for policy 0, policy_version 617777 (0.0008) [2023-12-27 09:54:28,428][20347] Updated weights for policy 0, policy_version 617787 (0.0006) [2023-12-27 09:54:28,452][20381] Updated weights for policy 1, policy_version 617479 (0.0007) [2023-12-27 09:54:28,515][20381] Updated weights for policy 1, policy_version 617489 (0.0008) [2023-12-27 09:54:28,569][20381] Updated weights for policy 1, policy_version 617499 (0.0008) [2023-12-27 09:54:29,101][20347] Updated weights for policy 0, policy_version 617797 (0.0007) [2023-12-27 09:54:29,159][20347] Updated weights for policy 0, policy_version 617807 (0.0006) [2023-12-27 09:54:29,181][20381] Updated weights for policy 1, policy_version 617509 (0.0007) [2023-12-27 09:54:29,215][20347] Updated weights for policy 0, policy_version 617817 (0.0006) [2023-12-27 09:54:29,243][20381] Updated weights for policy 1, policy_version 617519 (0.0007) [2023-12-27 09:54:29,298][20381] Updated weights for policy 1, policy_version 617529 (0.0005) [2023-12-27 09:54:29,901][20381] Updated weights for policy 1, policy_version 617539 (0.0007) [2023-12-27 09:54:29,927][20347] Updated weights for policy 0, policy_version 617827 (0.0007) [2023-12-27 09:54:29,976][20381] Updated weights for policy 1, policy_version 617549 (0.0008) [2023-12-27 09:54:29,983][20347] Updated weights for policy 0, policy_version 617837 (0.0008) [2023-12-27 09:54:30,037][20347] Updated weights for policy 0, policy_version 617847 (0.0006) [2023-12-27 09:54:30,040][20381] Updated weights for policy 1, policy_version 617559 (0.0008) [2023-12-27 09:54:30,704][20381] Updated weights for policy 1, policy_version 617569 (0.0008) [2023-12-27 09:54:30,713][20347] Updated weights for policy 0, policy_version 617857 (0.0006) [2023-12-27 09:54:30,758][20381] Updated weights for policy 1, policy_version 617579 (0.0007) [2023-12-27 09:54:30,761][20347] Updated weights for policy 0, policy_version 617867 (0.0006) [2023-12-27 09:54:30,806][20381] Updated weights for policy 1, policy_version 617589 (0.0006) [2023-12-27 09:54:30,815][20347] Updated weights for policy 0, policy_version 617877 (0.0005) [2023-12-27 09:54:30,851][20381] Updated weights for policy 1, policy_version 617599 (0.0005) [2023-12-27 09:54:30,868][20347] Updated weights for policy 0, policy_version 617887 (0.0006) [2023-12-27 09:54:31,530][20347] Updated weights for policy 0, policy_version 617897 (0.0006) [2023-12-27 09:54:31,570][20381] Updated weights for policy 1, policy_version 617609 (0.0006) [2023-12-27 09:54:31,592][20347] Updated weights for policy 0, policy_version 617907 (0.0007) [2023-12-27 09:54:31,632][20381] Updated weights for policy 1, policy_version 617619 (0.0008) [2023-12-27 09:54:31,656][20347] Updated weights for policy 0, policy_version 617917 (0.0008) [2023-12-27 09:54:31,689][20381] Updated weights for policy 1, policy_version 617629 (0.0007) [2023-12-27 09:54:32,285][20347] Updated weights for policy 0, policy_version 617927 (0.0006) [2023-12-27 09:54:32,333][20381] Updated weights for policy 1, policy_version 617639 (0.0007) [2023-12-27 09:54:32,344][20347] Updated weights for policy 0, policy_version 617937 (0.0007) [2023-12-27 09:54:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 316342272. Throughput: 0: 10341.3, 1: 10489.8. Samples: 316359428. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '252.660')] [2023-12-27 09:54:32,387][20381] Updated weights for policy 1, policy_version 617649 (0.0006) [2023-12-27 09:54:32,404][20347] Updated weights for policy 0, policy_version 617947 (0.0008) [2023-12-27 09:54:32,447][20381] Updated weights for policy 1, policy_version 617659 (0.0006) [2023-12-27 09:54:33,077][20347] Updated weights for policy 0, policy_version 617957 (0.0006) [2023-12-27 09:54:33,081][20381] Updated weights for policy 1, policy_version 617669 (0.0006) [2023-12-27 09:54:33,133][20381] Updated weights for policy 1, policy_version 617679 (0.0005) [2023-12-27 09:54:33,134][20347] Updated weights for policy 0, policy_version 617967 (0.0006) [2023-12-27 09:54:33,191][20347] Updated weights for policy 0, policy_version 617977 (0.0005) [2023-12-27 09:54:33,192][20381] Updated weights for policy 1, policy_version 617689 (0.0006) [2023-12-27 09:54:33,786][20347] Updated weights for policy 0, policy_version 617987 (0.0006) [2023-12-27 09:54:33,834][20381] Updated weights for policy 1, policy_version 617699 (0.0007) [2023-12-27 09:54:33,843][20347] Updated weights for policy 0, policy_version 617997 (0.0007) [2023-12-27 09:54:33,890][20381] Updated weights for policy 1, policy_version 617709 (0.0006) [2023-12-27 09:54:33,895][20347] Updated weights for policy 0, policy_version 618007 (0.0008) [2023-12-27 09:54:33,939][20381] Updated weights for policy 1, policy_version 617719 (0.0005) [2023-12-27 09:54:34,460][20347] Updated weights for policy 0, policy_version 618017 (0.0006) [2023-12-27 09:54:34,523][20347] Updated weights for policy 0, policy_version 618027 (0.0006) [2023-12-27 09:54:34,535][20381] Updated weights for policy 1, policy_version 617729 (0.0005) [2023-12-27 09:54:34,583][20347] Updated weights for policy 0, policy_version 618037 (0.0008) [2023-12-27 09:54:34,590][20381] Updated weights for policy 1, policy_version 617739 (0.0005) [2023-12-27 09:54:34,642][20381] Updated weights for policy 1, policy_version 617749 (0.0005) [2023-12-27 09:54:34,644][20347] Updated weights for policy 0, policy_version 618047 (0.0007) [2023-12-27 09:54:34,701][20381] Updated weights for policy 1, policy_version 617759 (0.0005) [2023-12-27 09:54:35,249][20381] Updated weights for policy 1, policy_version 617769 (0.0009) [2023-12-27 09:54:35,296][20381] Updated weights for policy 1, policy_version 617779 (0.0005) [2023-12-27 09:54:35,303][20347] Updated weights for policy 0, policy_version 618057 (0.0008) [2023-12-27 09:54:35,345][20381] Updated weights for policy 1, policy_version 617789 (0.0005) [2023-12-27 09:54:35,351][20347] Updated weights for policy 0, policy_version 618067 (0.0007) [2023-12-27 09:54:35,410][20347] Updated weights for policy 0, policy_version 618077 (0.0007) [2023-12-27 09:54:35,892][20381] Updated weights for policy 1, policy_version 617799 (0.0007) [2023-12-27 09:54:35,945][20381] Updated weights for policy 1, policy_version 617809 (0.0007) [2023-12-27 09:54:35,995][20381] Updated weights for policy 1, policy_version 617819 (0.0010) [2023-12-27 09:54:36,074][20347] Updated weights for policy 0, policy_version 618087 (0.0007) [2023-12-27 09:54:36,137][20347] Updated weights for policy 0, policy_version 618097 (0.0007) [2023-12-27 09:54:36,194][20347] Updated weights for policy 0, policy_version 618107 (0.0007) [2023-12-27 09:54:36,696][20381] Updated weights for policy 1, policy_version 617829 (0.0007) [2023-12-27 09:54:36,756][20381] Updated weights for policy 1, policy_version 617839 (0.0008) [2023-12-27 09:54:36,818][20381] Updated weights for policy 1, policy_version 617849 (0.0008) [2023-12-27 09:54:36,904][20347] Updated weights for policy 0, policy_version 618117 (0.0006) [2023-12-27 09:54:36,962][20347] Updated weights for policy 0, policy_version 618127 (0.0005) [2023-12-27 09:54:37,014][20347] Updated weights for policy 0, policy_version 618137 (0.0005) [2023-12-27 09:54:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20889.5, 300 sec: 20882.6). Total num frames: 316456960. Throughput: 0: 10362.5, 1: 10492.9. Samples: 316425408. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '238.930')] [2023-12-27 09:54:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000617856_158195712.pth... [2023-12-27 09:54:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000618144_158261248.pth... [2023-12-27 09:54:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000616928_157949952.pth [2023-12-27 09:54:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000616640_157884416.pth [2023-12-27 09:54:37,546][20381] Updated weights for policy 1, policy_version 617859 (0.0009) [2023-12-27 09:54:37,607][20381] Updated weights for policy 1, policy_version 617869 (0.0008) [2023-12-27 09:54:37,638][20347] Updated weights for policy 0, policy_version 618147 (0.0007) [2023-12-27 09:54:37,663][20381] Updated weights for policy 1, policy_version 617879 (0.0006) [2023-12-27 09:54:37,695][20347] Updated weights for policy 0, policy_version 618157 (0.0006) [2023-12-27 09:54:37,761][20347] Updated weights for policy 0, policy_version 618167 (0.0008) [2023-12-27 09:54:38,368][20381] Updated weights for policy 1, policy_version 617889 (0.0006) [2023-12-27 09:54:38,410][20347] Updated weights for policy 0, policy_version 618177 (0.0008) [2023-12-27 09:54:38,427][20381] Updated weights for policy 1, policy_version 617899 (0.0007) [2023-12-27 09:54:38,470][20347] Updated weights for policy 0, policy_version 618187 (0.0007) [2023-12-27 09:54:38,480][20381] Updated weights for policy 1, policy_version 617909 (0.0006) [2023-12-27 09:54:38,526][20347] Updated weights for policy 0, policy_version 618197 (0.0008) [2023-12-27 09:54:38,535][20381] Updated weights for policy 1, policy_version 617919 (0.0008) [2023-12-27 09:54:38,587][20347] Updated weights for policy 0, policy_version 618207 (0.0006) [2023-12-27 09:54:39,195][20347] Updated weights for policy 0, policy_version 618217 (0.0010) [2023-12-27 09:54:39,263][20347] Updated weights for policy 0, policy_version 618227 (0.0010) [2023-12-27 09:54:39,286][20381] Updated weights for policy 1, policy_version 617929 (0.0008) [2023-12-27 09:54:39,322][20347] Updated weights for policy 0, policy_version 618237 (0.0009) [2023-12-27 09:54:39,361][20381] Updated weights for policy 1, policy_version 617939 (0.0008) [2023-12-27 09:54:39,432][20381] Updated weights for policy 1, policy_version 617949 (0.0008) [2023-12-27 09:54:40,036][20347] Updated weights for policy 0, policy_version 618247 (0.0009) [2023-12-27 09:54:40,087][20347] Updated weights for policy 0, policy_version 618257 (0.0009) [2023-12-27 09:54:40,142][20347] Updated weights for policy 0, policy_version 618267 (0.0011) [2023-12-27 09:54:40,169][20381] Updated weights for policy 1, policy_version 617959 (0.0009) [2023-12-27 09:54:40,228][20381] Updated weights for policy 1, policy_version 617969 (0.0009) [2023-12-27 09:54:40,283][20381] Updated weights for policy 1, policy_version 617979 (0.0005) [2023-12-27 09:54:40,864][20347] Updated weights for policy 0, policy_version 618277 (0.0008) [2023-12-27 09:54:40,919][20347] Updated weights for policy 0, policy_version 618287 (0.0007) [2023-12-27 09:54:40,939][20381] Updated weights for policy 1, policy_version 617989 (0.0008) [2023-12-27 09:54:40,966][20347] Updated weights for policy 0, policy_version 618297 (0.0007) [2023-12-27 09:54:40,990][20381] Updated weights for policy 1, policy_version 617999 (0.0005) [2023-12-27 09:54:41,052][20381] Updated weights for policy 1, policy_version 618009 (0.0007) [2023-12-27 09:54:41,724][20347] Updated weights for policy 0, policy_version 618307 (0.0007) [2023-12-27 09:54:41,759][20381] Updated weights for policy 1, policy_version 618019 (0.0006) [2023-12-27 09:54:41,784][20347] Updated weights for policy 0, policy_version 618317 (0.0007) [2023-12-27 09:54:41,813][20381] Updated weights for policy 1, policy_version 618029 (0.0006) [2023-12-27 09:54:41,838][20347] Updated weights for policy 0, policy_version 618327 (0.0006) [2023-12-27 09:54:41,867][20381] Updated weights for policy 1, policy_version 618039 (0.0006) [2023-12-27 09:54:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 316555264. Throughput: 0: 10375.9, 1: 10393.1. Samples: 316547352. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '217.210')] [2023-12-27 09:54:42,544][20347] Updated weights for policy 0, policy_version 618337 (0.0007) [2023-12-27 09:54:42,549][20381] Updated weights for policy 1, policy_version 618049 (0.0008) [2023-12-27 09:54:42,597][20347] Updated weights for policy 0, policy_version 618347 (0.0006) [2023-12-27 09:54:42,606][20381] Updated weights for policy 1, policy_version 618059 (0.0006) [2023-12-27 09:54:42,655][20347] Updated weights for policy 0, policy_version 618357 (0.0005) [2023-12-27 09:54:42,659][20381] Updated weights for policy 1, policy_version 618069 (0.0008) [2023-12-27 09:54:42,707][20347] Updated weights for policy 0, policy_version 618367 (0.0006) [2023-12-27 09:54:42,713][20381] Updated weights for policy 1, policy_version 618079 (0.0006) [2023-12-27 09:54:43,327][20381] Updated weights for policy 1, policy_version 618089 (0.0006) [2023-12-27 09:54:43,385][20381] Updated weights for policy 1, policy_version 618099 (0.0006) [2023-12-27 09:54:43,424][20347] Updated weights for policy 0, policy_version 618377 (0.0007) [2023-12-27 09:54:43,444][20381] Updated weights for policy 1, policy_version 618109 (0.0006) [2023-12-27 09:54:43,480][20347] Updated weights for policy 0, policy_version 618387 (0.0006) [2023-12-27 09:54:43,531][20347] Updated weights for policy 0, policy_version 618397 (0.0007) [2023-12-27 09:54:44,090][20381] Updated weights for policy 1, policy_version 618119 (0.0005) [2023-12-27 09:54:44,140][20381] Updated weights for policy 1, policy_version 618129 (0.0006) [2023-12-27 09:54:44,173][20347] Updated weights for policy 0, policy_version 618407 (0.0007) [2023-12-27 09:54:44,194][20381] Updated weights for policy 1, policy_version 618139 (0.0007) [2023-12-27 09:54:44,239][20347] Updated weights for policy 0, policy_version 618417 (0.0007) [2023-12-27 09:54:44,305][20347] Updated weights for policy 0, policy_version 618427 (0.0009) [2023-12-27 09:54:44,797][20381] Updated weights for policy 1, policy_version 618149 (0.0007) [2023-12-27 09:54:44,853][20381] Updated weights for policy 1, policy_version 618159 (0.0007) [2023-12-27 09:54:44,913][20381] Updated weights for policy 1, policy_version 618169 (0.0008) [2023-12-27 09:54:44,984][20347] Updated weights for policy 0, policy_version 618437 (0.0008) [2023-12-27 09:54:45,037][20347] Updated weights for policy 0, policy_version 618447 (0.0008) [2023-12-27 09:54:45,095][20347] Updated weights for policy 0, policy_version 618457 (0.0007) [2023-12-27 09:54:45,561][20381] Updated weights for policy 1, policy_version 618179 (0.0009) [2023-12-27 09:54:45,617][20381] Updated weights for policy 1, policy_version 618189 (0.0007) [2023-12-27 09:54:45,678][20381] Updated weights for policy 1, policy_version 618199 (0.0008) [2023-12-27 09:54:45,772][20347] Updated weights for policy 0, policy_version 618467 (0.0007) [2023-12-27 09:54:45,828][20347] Updated weights for policy 0, policy_version 618477 (0.0005) [2023-12-27 09:54:45,881][20347] Updated weights for policy 0, policy_version 618487 (0.0006) [2023-12-27 09:54:46,382][20381] Updated weights for policy 1, policy_version 618209 (0.0008) [2023-12-27 09:54:46,440][20381] Updated weights for policy 1, policy_version 618219 (0.0005) [2023-12-27 09:54:46,485][20381] Updated weights for policy 1, policy_version 618229 (0.0006) [2023-12-27 09:54:46,518][20347] Updated weights for policy 0, policy_version 618497 (0.0005) [2023-12-27 09:54:46,531][20381] Updated weights for policy 1, policy_version 618239 (0.0006) [2023-12-27 09:54:46,578][20347] Updated weights for policy 0, policy_version 618507 (0.0008) [2023-12-27 09:54:46,633][20347] Updated weights for policy 0, policy_version 618517 (0.0008) [2023-12-27 09:54:46,678][20347] Updated weights for policy 0, policy_version 618527 (0.0008) [2023-12-27 09:54:47,182][20381] Updated weights for policy 1, policy_version 618249 (0.0005) [2023-12-27 09:54:47,228][20381] Updated weights for policy 1, policy_version 618259 (0.0005) [2023-12-27 09:54:47,279][20381] Updated weights for policy 1, policy_version 618269 (0.0007) [2023-12-27 09:54:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 316661760. Throughput: 0: 10334.1, 1: 10396.6. Samples: 316672232. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '247.020')] [2023-12-27 09:54:47,385][20347] Updated weights for policy 0, policy_version 618537 (0.0006) [2023-12-27 09:54:47,433][20347] Updated weights for policy 0, policy_version 618547 (0.0005) [2023-12-27 09:54:47,480][20347] Updated weights for policy 0, policy_version 618557 (0.0007) [2023-12-27 09:54:47,963][20381] Updated weights for policy 1, policy_version 618279 (0.0008) [2023-12-27 09:54:48,024][20381] Updated weights for policy 1, policy_version 618289 (0.0008) [2023-12-27 09:54:48,087][20381] Updated weights for policy 1, policy_version 618299 (0.0005) [2023-12-27 09:54:48,145][20347] Updated weights for policy 0, policy_version 618567 (0.0007) [2023-12-27 09:54:48,203][20347] Updated weights for policy 0, policy_version 618577 (0.0008) [2023-12-27 09:54:48,259][20347] Updated weights for policy 0, policy_version 618587 (0.0007) [2023-12-27 09:54:48,723][20381] Updated weights for policy 1, policy_version 618309 (0.0006) [2023-12-27 09:54:48,779][20381] Updated weights for policy 1, policy_version 618319 (0.0008) [2023-12-27 09:54:48,836][20381] Updated weights for policy 1, policy_version 618329 (0.0008) [2023-12-27 09:54:48,972][20347] Updated weights for policy 0, policy_version 618597 (0.0008) [2023-12-27 09:54:49,035][20347] Updated weights for policy 0, policy_version 618607 (0.0007) [2023-12-27 09:54:49,095][20347] Updated weights for policy 0, policy_version 618617 (0.0008) [2023-12-27 09:54:49,497][20381] Updated weights for policy 1, policy_version 618339 (0.0005) [2023-12-27 09:54:49,560][20381] Updated weights for policy 1, policy_version 618349 (0.0008) [2023-12-27 09:54:49,624][20381] Updated weights for policy 1, policy_version 618359 (0.0006) [2023-12-27 09:54:49,778][20347] Updated weights for policy 0, policy_version 618627 (0.0008) [2023-12-27 09:54:49,847][20347] Updated weights for policy 0, policy_version 618637 (0.0011) [2023-12-27 09:54:49,913][20347] Updated weights for policy 0, policy_version 618647 (0.0011) [2023-12-27 09:54:50,304][20381] Updated weights for policy 1, policy_version 618369 (0.0006) [2023-12-27 09:54:50,368][20381] Updated weights for policy 1, policy_version 618379 (0.0007) [2023-12-27 09:54:50,429][20381] Updated weights for policy 1, policy_version 618389 (0.0005) [2023-12-27 09:54:50,488][20381] Updated weights for policy 1, policy_version 618399 (0.0005) [2023-12-27 09:54:50,541][20347] Updated weights for policy 0, policy_version 618657 (0.0010) [2023-12-27 09:54:50,591][20347] Updated weights for policy 0, policy_version 618667 (0.0010) [2023-12-27 09:54:50,647][20347] Updated weights for policy 0, policy_version 618677 (0.0009) [2023-12-27 09:54:50,706][20347] Updated weights for policy 0, policy_version 618687 (0.0009) [2023-12-27 09:54:51,113][20381] Updated weights for policy 1, policy_version 618409 (0.0010) [2023-12-27 09:54:51,169][20381] Updated weights for policy 1, policy_version 618419 (0.0011) [2023-12-27 09:54:51,228][20381] Updated weights for policy 1, policy_version 618429 (0.0011) [2023-12-27 09:54:51,410][20347] Updated weights for policy 0, policy_version 618697 (0.0010) [2023-12-27 09:54:51,467][20347] Updated weights for policy 0, policy_version 618707 (0.0010) [2023-12-27 09:54:51,526][20347] Updated weights for policy 0, policy_version 618717 (0.0010) [2023-12-27 09:54:51,939][20381] Updated weights for policy 1, policy_version 618439 (0.0009) [2023-12-27 09:54:51,996][20381] Updated weights for policy 1, policy_version 618449 (0.0005) [2023-12-27 09:54:52,058][20381] Updated weights for policy 1, policy_version 618459 (0.0005) [2023-12-27 09:54:52,239][20347] Updated weights for policy 0, policy_version 618727 (0.0010) [2023-12-27 09:54:52,299][20347] Updated weights for policy 0, policy_version 618737 (0.0008) [2023-12-27 09:54:52,362][20347] Updated weights for policy 0, policy_version 618747 (0.0010) [2023-12-27 09:54:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 316760064. Throughput: 0: 10312.2, 1: 10347.2. Samples: 316733052. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '283.520')] [2023-12-27 09:54:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000618464_158351360.pth... [2023-12-27 09:54:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000617248_158040064.pth [2023-12-27 09:54:52,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000618752_158416896.pth... [2023-12-27 09:54:52,396][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000617536_158105600.pth [2023-12-27 09:54:52,705][20381] Updated weights for policy 1, policy_version 618469 (0.0008) [2023-12-27 09:54:52,765][20381] Updated weights for policy 1, policy_version 618479 (0.0008) [2023-12-27 09:54:52,822][20381] Updated weights for policy 1, policy_version 618489 (0.0011) [2023-12-27 09:54:53,026][20347] Updated weights for policy 0, policy_version 618757 (0.0008) [2023-12-27 09:54:53,082][20347] Updated weights for policy 0, policy_version 618767 (0.0005) [2023-12-27 09:54:53,137][20347] Updated weights for policy 0, policy_version 618777 (0.0005) [2023-12-27 09:54:53,451][20381] Updated weights for policy 1, policy_version 618499 (0.0011) [2023-12-27 09:54:53,498][20381] Updated weights for policy 1, policy_version 618509 (0.0010) [2023-12-27 09:54:53,543][20381] Updated weights for policy 1, policy_version 618519 (0.0010) [2023-12-27 09:54:53,700][20347] Updated weights for policy 0, policy_version 618787 (0.0007) [2023-12-27 09:54:53,745][20347] Updated weights for policy 0, policy_version 618797 (0.0010) [2023-12-27 09:54:53,792][20347] Updated weights for policy 0, policy_version 618807 (0.0010) [2023-12-27 09:54:54,158][20381] Updated weights for policy 1, policy_version 618529 (0.0010) [2023-12-27 09:54:54,215][20381] Updated weights for policy 1, policy_version 618539 (0.0007) [2023-12-27 09:54:54,274][20381] Updated weights for policy 1, policy_version 618549 (0.0007) [2023-12-27 09:54:54,337][20381] Updated weights for policy 1, policy_version 618559 (0.0008) [2023-12-27 09:54:54,491][20347] Updated weights for policy 0, policy_version 618817 (0.0010) [2023-12-27 09:54:54,553][20347] Updated weights for policy 0, policy_version 618827 (0.0009) [2023-12-27 09:54:54,611][20347] Updated weights for policy 0, policy_version 618837 (0.0008) [2023-12-27 09:54:54,666][20347] Updated weights for policy 0, policy_version 618847 (0.0009) [2023-12-27 09:54:54,998][20381] Updated weights for policy 1, policy_version 618569 (0.0007) [2023-12-27 09:54:55,056][20381] Updated weights for policy 1, policy_version 618579 (0.0006) [2023-12-27 09:54:55,116][20381] Updated weights for policy 1, policy_version 618589 (0.0005) [2023-12-27 09:54:55,322][20347] Updated weights for policy 0, policy_version 618857 (0.0010) [2023-12-27 09:54:55,379][20347] Updated weights for policy 0, policy_version 618867 (0.0008) [2023-12-27 09:54:55,437][20347] Updated weights for policy 0, policy_version 618877 (0.0010) [2023-12-27 09:54:55,740][20381] Updated weights for policy 1, policy_version 618599 (0.0006) [2023-12-27 09:54:55,805][20381] Updated weights for policy 1, policy_version 618609 (0.0005) [2023-12-27 09:54:55,874][20381] Updated weights for policy 1, policy_version 618619 (0.0005) [2023-12-27 09:54:55,978][20347] Updated weights for policy 0, policy_version 618887 (0.0007) [2023-12-27 09:54:56,031][20347] Updated weights for policy 0, policy_version 618897 (0.0006) [2023-12-27 09:54:56,085][20347] Updated weights for policy 0, policy_version 618907 (0.0007) [2023-12-27 09:54:56,511][20381] Updated weights for policy 1, policy_version 618629 (0.0007) [2023-12-27 09:54:56,559][20381] Updated weights for policy 1, policy_version 618639 (0.0007) [2023-12-27 09:54:56,611][20381] Updated weights for policy 1, policy_version 618649 (0.0007) [2023-12-27 09:54:56,670][20347] Updated weights for policy 0, policy_version 618917 (0.0008) [2023-12-27 09:54:56,715][20347] Updated weights for policy 0, policy_version 618927 (0.0005) [2023-12-27 09:54:56,772][20347] Updated weights for policy 0, policy_version 618937 (0.0005) [2023-12-27 09:54:57,318][20381] Updated weights for policy 1, policy_version 618659 (0.0008) [2023-12-27 09:54:57,340][20347] Updated weights for policy 0, policy_version 618947 (0.0007) [2023-12-27 09:54:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 316866560. Throughput: 0: 10317.2, 1: 10401.4. Samples: 316860932. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:54:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '295.680')] [2023-12-27 09:54:57,376][20381] Updated weights for policy 1, policy_version 618669 (0.0008) [2023-12-27 09:54:57,402][20347] Updated weights for policy 0, policy_version 618957 (0.0007) [2023-12-27 09:54:57,433][20381] Updated weights for policy 1, policy_version 618679 (0.0008) [2023-12-27 09:54:57,455][20347] Updated weights for policy 0, policy_version 618967 (0.0006) [2023-12-27 09:54:58,060][20347] Updated weights for policy 0, policy_version 618977 (0.0007) [2023-12-27 09:54:58,103][20381] Updated weights for policy 1, policy_version 618689 (0.0007) [2023-12-27 09:54:58,104][20347] Updated weights for policy 0, policy_version 618987 (0.0007) [2023-12-27 09:54:58,156][20381] Updated weights for policy 1, policy_version 618699 (0.0006) [2023-12-27 09:54:58,161][20347] Updated weights for policy 0, policy_version 618997 (0.0007) [2023-12-27 09:54:58,221][20381] Updated weights for policy 1, policy_version 618709 (0.0007) [2023-12-27 09:54:58,227][20347] Updated weights for policy 0, policy_version 619007 (0.0007) [2023-12-27 09:54:58,276][20381] Updated weights for policy 1, policy_version 618719 (0.0006) [2023-12-27 09:54:58,981][20347] Updated weights for policy 0, policy_version 619017 (0.0007) [2023-12-27 09:54:59,015][20381] Updated weights for policy 1, policy_version 618729 (0.0006) [2023-12-27 09:54:59,040][20347] Updated weights for policy 0, policy_version 619027 (0.0006) [2023-12-27 09:54:59,071][20381] Updated weights for policy 1, policy_version 618739 (0.0006) [2023-12-27 09:54:59,095][20347] Updated weights for policy 0, policy_version 619037 (0.0008) [2023-12-27 09:54:59,128][20381] Updated weights for policy 1, policy_version 618749 (0.0006) [2023-12-27 09:54:59,777][20381] Updated weights for policy 1, policy_version 618759 (0.0007) [2023-12-27 09:54:59,812][20347] Updated weights for policy 0, policy_version 619047 (0.0007) [2023-12-27 09:54:59,832][20381] Updated weights for policy 1, policy_version 618769 (0.0006) [2023-12-27 09:54:59,874][20347] Updated weights for policy 0, policy_version 619057 (0.0008) [2023-12-27 09:54:59,895][20381] Updated weights for policy 1, policy_version 618779 (0.0008) [2023-12-27 09:54:59,935][20347] Updated weights for policy 0, policy_version 619067 (0.0006) [2023-12-27 09:55:00,544][20347] Updated weights for policy 0, policy_version 619077 (0.0005) [2023-12-27 09:55:00,573][20381] Updated weights for policy 1, policy_version 618789 (0.0008) [2023-12-27 09:55:00,606][20347] Updated weights for policy 0, policy_version 619087 (0.0006) [2023-12-27 09:55:00,639][20381] Updated weights for policy 1, policy_version 618799 (0.0008) [2023-12-27 09:55:00,654][20347] Updated weights for policy 0, policy_version 619097 (0.0008) [2023-12-27 09:55:00,703][20381] Updated weights for policy 1, policy_version 618809 (0.0008) [2023-12-27 09:55:01,306][20347] Updated weights for policy 0, policy_version 619107 (0.0009) [2023-12-27 09:55:01,363][20381] Updated weights for policy 1, policy_version 618819 (0.0008) [2023-12-27 09:55:01,371][20347] Updated weights for policy 0, policy_version 619117 (0.0012) [2023-12-27 09:55:01,428][20347] Updated weights for policy 0, policy_version 619127 (0.0007) [2023-12-27 09:55:01,429][20381] Updated weights for policy 1, policy_version 618829 (0.0007) [2023-12-27 09:55:01,484][20381] Updated weights for policy 1, policy_version 618839 (0.0008) [2023-12-27 09:55:02,080][20347] Updated weights for policy 0, policy_version 619137 (0.0005) [2023-12-27 09:55:02,135][20347] Updated weights for policy 0, policy_version 619147 (0.0010) [2023-12-27 09:55:02,170][20381] Updated weights for policy 1, policy_version 618849 (0.0009) [2023-12-27 09:55:02,190][20347] Updated weights for policy 0, policy_version 619157 (0.0009) [2023-12-27 09:55:02,235][20381] Updated weights for policy 1, policy_version 618859 (0.0009) [2023-12-27 09:55:02,243][20347] Updated weights for policy 0, policy_version 619167 (0.0005) [2023-12-27 09:55:02,291][20381] Updated weights for policy 1, policy_version 618869 (0.0008) [2023-12-27 09:55:02,358][20381] Updated weights for policy 1, policy_version 618879 (0.0008) [2023-12-27 09:55:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 316973056. Throughput: 0: 10387.2, 1: 10383.1. Samples: 316985544. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:02,366][19170] Avg episode reward: [(0, '1000.000'), (1, '319.210')] [2023-12-27 09:55:02,928][20347] Updated weights for policy 0, policy_version 619177 (0.0010) [2023-12-27 09:55:02,980][20347] Updated weights for policy 0, policy_version 619187 (0.0010) [2023-12-27 09:55:02,989][20381] Updated weights for policy 1, policy_version 618889 (0.0007) [2023-12-27 09:55:03,029][20347] Updated weights for policy 0, policy_version 619197 (0.0010) [2023-12-27 09:55:03,037][20381] Updated weights for policy 1, policy_version 618899 (0.0005) [2023-12-27 09:55:03,085][20381] Updated weights for policy 1, policy_version 618909 (0.0005) [2023-12-27 09:55:03,612][20347] Updated weights for policy 0, policy_version 619207 (0.0007) [2023-12-27 09:55:03,665][20347] Updated weights for policy 0, policy_version 619217 (0.0006) [2023-12-27 09:55:03,714][20347] Updated weights for policy 0, policy_version 619227 (0.0005) [2023-12-27 09:55:03,732][20381] Updated weights for policy 1, policy_version 618919 (0.0006) [2023-12-27 09:55:03,776][20381] Updated weights for policy 1, policy_version 618929 (0.0008) [2023-12-27 09:55:03,821][20381] Updated weights for policy 1, policy_version 618939 (0.0009) [2023-12-27 09:55:04,316][20347] Updated weights for policy 0, policy_version 619237 (0.0006) [2023-12-27 09:55:04,381][20347] Updated weights for policy 0, policy_version 619247 (0.0011) [2023-12-27 09:55:04,438][20347] Updated weights for policy 0, policy_version 619257 (0.0011) [2023-12-27 09:55:04,588][20381] Updated weights for policy 1, policy_version 618949 (0.0008) [2023-12-27 09:55:04,643][20381] Updated weights for policy 1, policy_version 618959 (0.0008) [2023-12-27 09:55:04,694][20381] Updated weights for policy 1, policy_version 618969 (0.0007) [2023-12-27 09:55:05,094][20347] Updated weights for policy 0, policy_version 619267 (0.0009) [2023-12-27 09:55:05,154][20347] Updated weights for policy 0, policy_version 619277 (0.0005) [2023-12-27 09:55:05,215][20347] Updated weights for policy 0, policy_version 619287 (0.0005) [2023-12-27 09:55:05,442][20381] Updated weights for policy 1, policy_version 618979 (0.0007) [2023-12-27 09:55:05,490][20381] Updated weights for policy 1, policy_version 618989 (0.0007) [2023-12-27 09:55:05,541][20381] Updated weights for policy 1, policy_version 618999 (0.0008) [2023-12-27 09:55:05,779][20347] Updated weights for policy 0, policy_version 619297 (0.0005) [2023-12-27 09:55:05,834][20347] Updated weights for policy 0, policy_version 619307 (0.0005) [2023-12-27 09:55:05,889][20347] Updated weights for policy 0, policy_version 619317 (0.0005) [2023-12-27 09:55:05,947][20347] Updated weights for policy 0, policy_version 619327 (0.0009) [2023-12-27 09:55:06,268][20381] Updated weights for policy 1, policy_version 619009 (0.0008) [2023-12-27 09:55:06,322][20381] Updated weights for policy 1, policy_version 619019 (0.0006) [2023-12-27 09:55:06,378][20381] Updated weights for policy 1, policy_version 619029 (0.0007) [2023-12-27 09:55:06,443][20381] Updated weights for policy 1, policy_version 619039 (0.0007) [2023-12-27 09:55:06,554][20347] Updated weights for policy 0, policy_version 619337 (0.0010) [2023-12-27 09:55:06,606][20347] Updated weights for policy 0, policy_version 619347 (0.0011) [2023-12-27 09:55:06,658][20347] Updated weights for policy 0, policy_version 619357 (0.0006) [2023-12-27 09:55:07,145][20381] Updated weights for policy 1, policy_version 619049 (0.0006) [2023-12-27 09:55:07,211][20381] Updated weights for policy 1, policy_version 619059 (0.0008) [2023-12-27 09:55:07,267][20381] Updated weights for policy 1, policy_version 619069 (0.0007) [2023-12-27 09:55:07,333][20347] Updated weights for policy 0, policy_version 619367 (0.0010) [2023-12-27 09:55:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20882.7). Total num frames: 317079552. Throughput: 0: 10466.5, 1: 10380.5. Samples: 317048664. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '366.910')] [2023-12-27 09:55:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000619072_158507008.pth... [2023-12-27 09:55:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000617856_158195712.pth [2023-12-27 09:55:07,384][20347] Updated weights for policy 0, policy_version 619377 (0.0010) [2023-12-27 09:55:07,436][20347] Updated weights for policy 0, policy_version 619387 (0.0010) [2023-12-27 09:55:07,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000619392_158580736.pth... [2023-12-27 09:55:07,464][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000618144_158261248.pth [2023-12-27 09:55:07,945][20381] Updated weights for policy 1, policy_version 619079 (0.0008) [2023-12-27 09:55:08,004][20381] Updated weights for policy 1, policy_version 619089 (0.0007) [2023-12-27 09:55:08,067][20381] Updated weights for policy 1, policy_version 619099 (0.0008) [2023-12-27 09:55:08,091][20347] Updated weights for policy 0, policy_version 619397 (0.0008) [2023-12-27 09:55:08,148][20347] Updated weights for policy 0, policy_version 619407 (0.0005) [2023-12-27 09:55:08,205][20347] Updated weights for policy 0, policy_version 619417 (0.0005) [2023-12-27 09:55:08,744][20347] Updated weights for policy 0, policy_version 619427 (0.0006) [2023-12-27 09:55:08,794][20347] Updated weights for policy 0, policy_version 619437 (0.0007) [2023-12-27 09:55:08,838][20347] Updated weights for policy 0, policy_version 619447 (0.0008) [2023-12-27 09:55:08,842][20381] Updated weights for policy 1, policy_version 619109 (0.0008) [2023-12-27 09:55:08,891][20381] Updated weights for policy 1, policy_version 619119 (0.0006) [2023-12-27 09:55:08,937][20381] Updated weights for policy 1, policy_version 619129 (0.0006) [2023-12-27 09:55:09,495][20347] Updated weights for policy 0, policy_version 619457 (0.0006) [2023-12-27 09:55:09,551][20347] Updated weights for policy 0, policy_version 619467 (0.0006) [2023-12-27 09:55:09,616][20347] Updated weights for policy 0, policy_version 619477 (0.0007) [2023-12-27 09:55:09,640][20381] Updated weights for policy 1, policy_version 619139 (0.0006) [2023-12-27 09:55:09,673][20347] Updated weights for policy 0, policy_version 619487 (0.0006) [2023-12-27 09:55:09,703][20381] Updated weights for policy 1, policy_version 619149 (0.0008) [2023-12-27 09:55:09,764][20381] Updated weights for policy 1, policy_version 619159 (0.0007) [2023-12-27 09:55:10,348][20347] Updated weights for policy 0, policy_version 619497 (0.0008) [2023-12-27 09:55:10,405][20347] Updated weights for policy 0, policy_version 619507 (0.0008) [2023-12-27 09:55:10,458][20381] Updated weights for policy 1, policy_version 619169 (0.0010) [2023-12-27 09:55:10,468][20347] Updated weights for policy 0, policy_version 619517 (0.0007) [2023-12-27 09:55:10,520][20381] Updated weights for policy 1, policy_version 619179 (0.0007) [2023-12-27 09:55:10,580][20381] Updated weights for policy 1, policy_version 619189 (0.0005) [2023-12-27 09:55:10,636][20381] Updated weights for policy 1, policy_version 619199 (0.0005) [2023-12-27 09:55:11,174][20347] Updated weights for policy 0, policy_version 619527 (0.0007) [2023-12-27 09:55:11,237][20347] Updated weights for policy 0, policy_version 619537 (0.0008) [2023-12-27 09:55:11,266][20381] Updated weights for policy 1, policy_version 619209 (0.0007) [2023-12-27 09:55:11,301][20347] Updated weights for policy 0, policy_version 619547 (0.0008) [2023-12-27 09:55:11,327][20381] Updated weights for policy 1, policy_version 619219 (0.0007) [2023-12-27 09:55:11,385][20381] Updated weights for policy 1, policy_version 619229 (0.0008) [2023-12-27 09:55:12,029][20347] Updated weights for policy 0, policy_version 619557 (0.0007) [2023-12-27 09:55:12,056][20381] Updated weights for policy 1, policy_version 619239 (0.0009) [2023-12-27 09:55:12,086][20347] Updated weights for policy 0, policy_version 619567 (0.0007) [2023-12-27 09:55:12,118][20381] Updated weights for policy 1, policy_version 619249 (0.0010) [2023-12-27 09:55:12,142][20347] Updated weights for policy 0, policy_version 619577 (0.0005) [2023-12-27 09:55:12,181][20381] Updated weights for policy 1, policy_version 619259 (0.0010) [2023-12-27 09:55:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 317186048. Throughput: 0: 10503.3, 1: 10382.0. Samples: 317172852. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '489.880')] [2023-12-27 09:55:12,827][20381] Updated weights for policy 1, policy_version 619269 (0.0007) [2023-12-27 09:55:12,830][20347] Updated weights for policy 0, policy_version 619587 (0.0007) [2023-12-27 09:55:12,880][20347] Updated weights for policy 0, policy_version 619597 (0.0006) [2023-12-27 09:55:12,886][20381] Updated weights for policy 1, policy_version 619279 (0.0006) [2023-12-27 09:55:12,927][20347] Updated weights for policy 0, policy_version 619607 (0.0006) [2023-12-27 09:55:12,928][20381] Updated weights for policy 1, policy_version 619289 (0.0005) [2023-12-27 09:55:13,579][20381] Updated weights for policy 1, policy_version 619299 (0.0006) [2023-12-27 09:55:13,623][20347] Updated weights for policy 0, policy_version 619617 (0.0006) [2023-12-27 09:55:13,632][20381] Updated weights for policy 1, policy_version 619309 (0.0008) [2023-12-27 09:55:13,679][20347] Updated weights for policy 0, policy_version 619627 (0.0006) [2023-12-27 09:55:13,682][20381] Updated weights for policy 1, policy_version 619319 (0.0006) [2023-12-27 09:55:13,736][20347] Updated weights for policy 0, policy_version 619637 (0.0008) [2023-12-27 09:55:13,790][20347] Updated weights for policy 0, policy_version 619647 (0.0008) [2023-12-27 09:55:14,330][20381] Updated weights for policy 1, policy_version 619329 (0.0006) [2023-12-27 09:55:14,376][20381] Updated weights for policy 1, policy_version 619339 (0.0008) [2023-12-27 09:55:14,432][20381] Updated weights for policy 1, policy_version 619349 (0.0008) [2023-12-27 09:55:14,483][20347] Updated weights for policy 0, policy_version 619657 (0.0006) [2023-12-27 09:55:14,492][20381] Updated weights for policy 1, policy_version 619359 (0.0007) [2023-12-27 09:55:14,543][20347] Updated weights for policy 0, policy_version 619667 (0.0005) [2023-12-27 09:55:14,604][20347] Updated weights for policy 0, policy_version 619677 (0.0007) [2023-12-27 09:55:15,185][20381] Updated weights for policy 1, policy_version 619369 (0.0007) [2023-12-27 09:55:15,237][20381] Updated weights for policy 1, policy_version 619379 (0.0008) [2023-12-27 09:55:15,271][20347] Updated weights for policy 0, policy_version 619687 (0.0008) [2023-12-27 09:55:15,290][20381] Updated weights for policy 1, policy_version 619389 (0.0006) [2023-12-27 09:55:15,323][20347] Updated weights for policy 0, policy_version 619697 (0.0008) [2023-12-27 09:55:15,374][20347] Updated weights for policy 0, policy_version 619707 (0.0008) [2023-12-27 09:55:15,901][20381] Updated weights for policy 1, policy_version 619399 (0.0006) [2023-12-27 09:55:15,961][20381] Updated weights for policy 1, policy_version 619409 (0.0006) [2023-12-27 09:55:16,008][20381] Updated weights for policy 1, policy_version 619419 (0.0007) [2023-12-27 09:55:16,010][20347] Updated weights for policy 0, policy_version 619717 (0.0007) [2023-12-27 09:55:16,060][20347] Updated weights for policy 0, policy_version 619727 (0.0007) [2023-12-27 09:55:16,113][20347] Updated weights for policy 0, policy_version 619737 (0.0006) [2023-12-27 09:55:16,637][20381] Updated weights for policy 1, policy_version 619429 (0.0006) [2023-12-27 09:55:16,693][20381] Updated weights for policy 1, policy_version 619439 (0.0008) [2023-12-27 09:55:16,731][20347] Updated weights for policy 0, policy_version 619747 (0.0006) [2023-12-27 09:55:16,748][20381] Updated weights for policy 1, policy_version 619449 (0.0007) [2023-12-27 09:55:16,792][20347] Updated weights for policy 0, policy_version 619757 (0.0008) [2023-12-27 09:55:16,861][20347] Updated weights for policy 0, policy_version 619767 (0.0008) [2023-12-27 09:55:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20882.6). Total num frames: 317284352. Throughput: 0: 10500.5, 1: 10353.8. Samples: 317297876. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '438.330')] [2023-12-27 09:55:17,418][20381] Updated weights for policy 1, policy_version 619459 (0.0007) [2023-12-27 09:55:17,419][20347] Updated weights for policy 0, policy_version 619777 (0.0007) [2023-12-27 09:55:17,475][20381] Updated weights for policy 1, policy_version 619469 (0.0007) [2023-12-27 09:55:17,475][20347] Updated weights for policy 0, policy_version 619787 (0.0007) [2023-12-27 09:55:17,528][20347] Updated weights for policy 0, policy_version 619797 (0.0008) [2023-12-27 09:55:17,536][20381] Updated weights for policy 1, policy_version 619479 (0.0006) [2023-12-27 09:55:17,580][20347] Updated weights for policy 0, policy_version 619807 (0.0007) [2023-12-27 09:55:18,218][20347] Updated weights for policy 0, policy_version 619817 (0.0005) [2023-12-27 09:55:18,261][20381] Updated weights for policy 1, policy_version 619489 (0.0006) [2023-12-27 09:55:18,275][20347] Updated weights for policy 0, policy_version 619827 (0.0007) [2023-12-27 09:55:18,318][20381] Updated weights for policy 1, policy_version 619499 (0.0007) [2023-12-27 09:55:18,327][20347] Updated weights for policy 0, policy_version 619837 (0.0010) [2023-12-27 09:55:18,375][20381] Updated weights for policy 1, policy_version 619509 (0.0006) [2023-12-27 09:55:18,430][20381] Updated weights for policy 1, policy_version 619519 (0.0007) [2023-12-27 09:55:19,028][20347] Updated weights for policy 0, policy_version 619847 (0.0009) [2023-12-27 09:55:19,088][20347] Updated weights for policy 0, policy_version 619857 (0.0007) [2023-12-27 09:55:19,137][20381] Updated weights for policy 1, policy_version 619529 (0.0008) [2023-12-27 09:55:19,147][20347] Updated weights for policy 0, policy_version 619867 (0.0006) [2023-12-27 09:55:19,192][20381] Updated weights for policy 1, policy_version 619539 (0.0007) [2023-12-27 09:55:19,254][20381] Updated weights for policy 1, policy_version 619549 (0.0008) [2023-12-27 09:55:19,819][20347] Updated weights for policy 0, policy_version 619877 (0.0007) [2023-12-27 09:55:19,880][20347] Updated weights for policy 0, policy_version 619887 (0.0007) [2023-12-27 09:55:19,940][20347] Updated weights for policy 0, policy_version 619897 (0.0008) [2023-12-27 09:55:19,999][20381] Updated weights for policy 1, policy_version 619559 (0.0007) [2023-12-27 09:55:20,055][20381] Updated weights for policy 1, policy_version 619569 (0.0008) [2023-12-27 09:55:20,113][20381] Updated weights for policy 1, policy_version 619579 (0.0007) [2023-12-27 09:55:20,649][20347] Updated weights for policy 0, policy_version 619907 (0.0007) [2023-12-27 09:55:20,709][20347] Updated weights for policy 0, policy_version 619917 (0.0010) [2023-12-27 09:55:20,775][20347] Updated weights for policy 0, policy_version 619927 (0.0010) [2023-12-27 09:55:20,880][20381] Updated weights for policy 1, policy_version 619589 (0.0008) [2023-12-27 09:55:20,937][20381] Updated weights for policy 1, policy_version 619599 (0.0008) [2023-12-27 09:55:20,986][20381] Updated weights for policy 1, policy_version 619609 (0.0008) [2023-12-27 09:55:21,487][20347] Updated weights for policy 0, policy_version 619937 (0.0011) [2023-12-27 09:55:21,544][20347] Updated weights for policy 0, policy_version 619947 (0.0007) [2023-12-27 09:55:21,602][20347] Updated weights for policy 0, policy_version 619957 (0.0007) [2023-12-27 09:55:21,663][20347] Updated weights for policy 0, policy_version 619967 (0.0009) [2023-12-27 09:55:21,724][20381] Updated weights for policy 1, policy_version 619619 (0.0008) [2023-12-27 09:55:21,787][20381] Updated weights for policy 1, policy_version 619629 (0.0009) [2023-12-27 09:55:21,852][20381] Updated weights for policy 1, policy_version 619639 (0.0008) [2023-12-27 09:55:22,362][20347] Updated weights for policy 0, policy_version 619977 (0.0010) [2023-12-27 09:55:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 317382656. Throughput: 0: 10499.5, 1: 10240.7. Samples: 317358712. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '309.990')] [2023-12-27 09:55:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000619648_158654464.pth... [2023-12-27 09:55:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000618464_158351360.pth [2023-12-27 09:55:22,421][20347] Updated weights for policy 0, policy_version 619987 (0.0011) [2023-12-27 09:55:22,473][20347] Updated weights for policy 0, policy_version 619997 (0.0010) [2023-12-27 09:55:22,487][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000620000_158736384.pth... [2023-12-27 09:55:22,490][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000618752_158416896.pth [2023-12-27 09:55:22,557][20381] Updated weights for policy 1, policy_version 619649 (0.0008) [2023-12-27 09:55:22,613][20381] Updated weights for policy 1, policy_version 619659 (0.0008) [2023-12-27 09:55:22,673][20381] Updated weights for policy 1, policy_version 619669 (0.0007) [2023-12-27 09:55:22,734][20381] Updated weights for policy 1, policy_version 619679 (0.0007) [2023-12-27 09:55:23,159][20347] Updated weights for policy 0, policy_version 620007 (0.0010) [2023-12-27 09:55:23,218][20347] Updated weights for policy 0, policy_version 620017 (0.0007) [2023-12-27 09:55:23,273][20347] Updated weights for policy 0, policy_version 620027 (0.0005) [2023-12-27 09:55:23,457][20381] Updated weights for policy 1, policy_version 619689 (0.0009) [2023-12-27 09:55:23,503][20381] Updated weights for policy 1, policy_version 619699 (0.0010) [2023-12-27 09:55:23,560][20381] Updated weights for policy 1, policy_version 619709 (0.0008) [2023-12-27 09:55:23,838][20347] Updated weights for policy 0, policy_version 620037 (0.0005) [2023-12-27 09:55:23,889][20347] Updated weights for policy 0, policy_version 620047 (0.0005) [2023-12-27 09:55:23,941][20347] Updated weights for policy 0, policy_version 620057 (0.0006) [2023-12-27 09:55:24,311][20381] Updated weights for policy 1, policy_version 619719 (0.0008) [2023-12-27 09:55:24,363][20381] Updated weights for policy 1, policy_version 619729 (0.0007) [2023-12-27 09:55:24,418][20381] Updated weights for policy 1, policy_version 619739 (0.0007) [2023-12-27 09:55:24,602][20347] Updated weights for policy 0, policy_version 620067 (0.0007) [2023-12-27 09:55:24,658][20347] Updated weights for policy 0, policy_version 620077 (0.0005) [2023-12-27 09:55:24,714][20347] Updated weights for policy 0, policy_version 620087 (0.0007) [2023-12-27 09:55:25,094][20381] Updated weights for policy 1, policy_version 619749 (0.0008) [2023-12-27 09:55:25,146][20381] Updated weights for policy 1, policy_version 619759 (0.0007) [2023-12-27 09:55:25,198][20381] Updated weights for policy 1, policy_version 619769 (0.0008) [2023-12-27 09:55:25,306][20347] Updated weights for policy 0, policy_version 620097 (0.0008) [2023-12-27 09:55:25,360][20347] Updated weights for policy 0, policy_version 620107 (0.0010) [2023-12-27 09:55:25,407][20347] Updated weights for policy 0, policy_version 620117 (0.0010) [2023-12-27 09:55:25,458][20347] Updated weights for policy 0, policy_version 620127 (0.0005) [2023-12-27 09:55:25,944][20381] Updated weights for policy 1, policy_version 619779 (0.0007) [2023-12-27 09:55:26,001][20381] Updated weights for policy 1, policy_version 619789 (0.0009) [2023-12-27 09:55:26,043][20347] Updated weights for policy 0, policy_version 620137 (0.0009) [2023-12-27 09:55:26,056][20381] Updated weights for policy 1, policy_version 619799 (0.0006) [2023-12-27 09:55:26,087][20347] Updated weights for policy 0, policy_version 620147 (0.0010) [2023-12-27 09:55:26,133][20347] Updated weights for policy 0, policy_version 620157 (0.0010) [2023-12-27 09:55:26,758][20347] Updated weights for policy 0, policy_version 620167 (0.0010) [2023-12-27 09:55:26,767][20381] Updated weights for policy 1, policy_version 619809 (0.0006) [2023-12-27 09:55:26,812][20347] Updated weights for policy 0, policy_version 620177 (0.0006) [2023-12-27 09:55:26,813][20381] Updated weights for policy 1, policy_version 619819 (0.0008) [2023-12-27 09:55:26,859][20347] Updated weights for policy 0, policy_version 620187 (0.0010) [2023-12-27 09:55:26,861][20381] Updated weights for policy 1, policy_version 619829 (0.0005) [2023-12-27 09:55:26,908][20381] Updated weights for policy 1, policy_version 619839 (0.0007) [2023-12-27 09:55:27,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 317489152. Throughput: 0: 10541.7, 1: 10196.9. Samples: 317480588. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '332.240')] [2023-12-27 09:55:27,558][20347] Updated weights for policy 0, policy_version 620197 (0.0009) [2023-12-27 09:55:27,620][20381] Updated weights for policy 1, policy_version 619849 (0.0006) [2023-12-27 09:55:27,620][20347] Updated weights for policy 0, policy_version 620207 (0.0006) [2023-12-27 09:55:27,671][20381] Updated weights for policy 1, policy_version 619859 (0.0005) [2023-12-27 09:55:27,685][20347] Updated weights for policy 0, policy_version 620217 (0.0006) [2023-12-27 09:55:27,729][20381] Updated weights for policy 1, policy_version 619869 (0.0006) [2023-12-27 09:55:28,266][20347] Updated weights for policy 0, policy_version 620227 (0.0006) [2023-12-27 09:55:28,322][20347] Updated weights for policy 0, policy_version 620237 (0.0006) [2023-12-27 09:55:28,381][20347] Updated weights for policy 0, policy_version 620247 (0.0007) [2023-12-27 09:55:28,399][20381] Updated weights for policy 1, policy_version 619879 (0.0007) [2023-12-27 09:55:28,457][20381] Updated weights for policy 1, policy_version 619889 (0.0007) [2023-12-27 09:55:28,513][20381] Updated weights for policy 1, policy_version 619899 (0.0007) [2023-12-27 09:55:28,964][20347] Updated weights for policy 0, policy_version 620257 (0.0007) [2023-12-27 09:55:29,018][20347] Updated weights for policy 0, policy_version 620267 (0.0005) [2023-12-27 09:55:29,067][20347] Updated weights for policy 0, policy_version 620277 (0.0005) [2023-12-27 09:55:29,127][20347] Updated weights for policy 0, policy_version 620287 (0.0005) [2023-12-27 09:55:29,271][20381] Updated weights for policy 1, policy_version 619909 (0.0007) [2023-12-27 09:55:29,327][20381] Updated weights for policy 1, policy_version 619919 (0.0008) [2023-12-27 09:55:29,381][20381] Updated weights for policy 1, policy_version 619929 (0.0007) [2023-12-27 09:55:29,721][20347] Updated weights for policy 0, policy_version 620297 (0.0010) [2023-12-27 09:55:29,778][20347] Updated weights for policy 0, policy_version 620307 (0.0008) [2023-12-27 09:55:29,833][20347] Updated weights for policy 0, policy_version 620317 (0.0010) [2023-12-27 09:55:30,093][20381] Updated weights for policy 1, policy_version 619939 (0.0009) [2023-12-27 09:55:30,151][20381] Updated weights for policy 1, policy_version 619949 (0.0007) [2023-12-27 09:55:30,206][20381] Updated weights for policy 1, policy_version 619959 (0.0007) [2023-12-27 09:55:30,535][20347] Updated weights for policy 0, policy_version 620327 (0.0010) [2023-12-27 09:55:30,592][20347] Updated weights for policy 0, policy_version 620337 (0.0010) [2023-12-27 09:55:30,640][20347] Updated weights for policy 0, policy_version 620347 (0.0010) [2023-12-27 09:55:30,902][20381] Updated weights for policy 1, policy_version 619969 (0.0007) [2023-12-27 09:55:30,952][20381] Updated weights for policy 1, policy_version 619979 (0.0007) [2023-12-27 09:55:31,004][20381] Updated weights for policy 1, policy_version 619989 (0.0007) [2023-12-27 09:55:31,068][20381] Updated weights for policy 1, policy_version 619999 (0.0008) [2023-12-27 09:55:31,336][20347] Updated weights for policy 0, policy_version 620357 (0.0009) [2023-12-27 09:55:31,397][20347] Updated weights for policy 0, policy_version 620367 (0.0008) [2023-12-27 09:55:31,451][20347] Updated weights for policy 0, policy_version 620377 (0.0007) [2023-12-27 09:55:31,705][20381] Updated weights for policy 1, policy_version 620009 (0.0008) [2023-12-27 09:55:31,773][20381] Updated weights for policy 1, policy_version 620019 (0.0007) [2023-12-27 09:55:31,834][20381] Updated weights for policy 1, policy_version 620029 (0.0006) [2023-12-27 09:55:32,155][20347] Updated weights for policy 0, policy_version 620387 (0.0008) [2023-12-27 09:55:32,210][20347] Updated weights for policy 0, policy_version 620397 (0.0007) [2023-12-27 09:55:32,272][20347] Updated weights for policy 0, policy_version 620407 (0.0008) [2023-12-27 09:55:32,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 317595648. Throughput: 0: 10615.9, 1: 10130.9. Samples: 317605844. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '316.560')] [2023-12-27 09:55:32,434][20381] Updated weights for policy 1, policy_version 620039 (0.0007) [2023-12-27 09:55:32,496][20381] Updated weights for policy 1, policy_version 620049 (0.0008) [2023-12-27 09:55:32,557][20381] Updated weights for policy 1, policy_version 620059 (0.0007) [2023-12-27 09:55:32,929][20347] Updated weights for policy 0, policy_version 620417 (0.0007) [2023-12-27 09:55:32,983][20347] Updated weights for policy 0, policy_version 620427 (0.0005) [2023-12-27 09:55:33,039][20347] Updated weights for policy 0, policy_version 620437 (0.0005) [2023-12-27 09:55:33,093][20347] Updated weights for policy 0, policy_version 620447 (0.0008) [2023-12-27 09:55:33,136][20381] Updated weights for policy 1, policy_version 620069 (0.0005) [2023-12-27 09:55:33,191][20381] Updated weights for policy 1, policy_version 620079 (0.0006) [2023-12-27 09:55:33,244][20381] Updated weights for policy 1, policy_version 620089 (0.0006) [2023-12-27 09:55:33,760][20381] Updated weights for policy 1, policy_version 620099 (0.0005) [2023-12-27 09:55:33,805][20347] Updated weights for policy 0, policy_version 620457 (0.0007) [2023-12-27 09:55:33,837][20381] Updated weights for policy 1, policy_version 620109 (0.0006) [2023-12-27 09:55:33,864][20347] Updated weights for policy 0, policy_version 620467 (0.0006) [2023-12-27 09:55:33,885][20381] Updated weights for policy 1, policy_version 620119 (0.0005) [2023-12-27 09:55:33,917][20347] Updated weights for policy 0, policy_version 620477 (0.0006) [2023-12-27 09:55:34,551][20381] Updated weights for policy 1, policy_version 620129 (0.0005) [2023-12-27 09:55:34,593][20347] Updated weights for policy 0, policy_version 620487 (0.0009) [2023-12-27 09:55:34,614][20381] Updated weights for policy 1, policy_version 620139 (0.0009) [2023-12-27 09:55:34,650][20347] Updated weights for policy 0, policy_version 620497 (0.0006) [2023-12-27 09:55:34,675][20381] Updated weights for policy 1, policy_version 620149 (0.0007) [2023-12-27 09:55:34,706][20347] Updated weights for policy 0, policy_version 620507 (0.0009) [2023-12-27 09:55:34,734][20381] Updated weights for policy 1, policy_version 620159 (0.0007) [2023-12-27 09:55:35,382][20381] Updated weights for policy 1, policy_version 620169 (0.0007) [2023-12-27 09:55:35,401][20347] Updated weights for policy 0, policy_version 620517 (0.0006) [2023-12-27 09:55:35,432][20381] Updated weights for policy 1, policy_version 620179 (0.0007) [2023-12-27 09:55:35,466][20347] Updated weights for policy 0, policy_version 620527 (0.0008) [2023-12-27 09:55:35,481][20381] Updated weights for policy 1, policy_version 620189 (0.0009) [2023-12-27 09:55:35,516][20347] Updated weights for policy 0, policy_version 620537 (0.0008) [2023-12-27 09:55:36,110][20381] Updated weights for policy 1, policy_version 620199 (0.0007) [2023-12-27 09:55:36,165][20381] Updated weights for policy 1, policy_version 620209 (0.0008) [2023-12-27 09:55:36,220][20381] Updated weights for policy 1, policy_version 620219 (0.0008) [2023-12-27 09:55:36,225][20347] Updated weights for policy 0, policy_version 620547 (0.0007) [2023-12-27 09:55:36,288][20347] Updated weights for policy 0, policy_version 620557 (0.0007) [2023-12-27 09:55:36,348][20347] Updated weights for policy 0, policy_version 620567 (0.0007) [2023-12-27 09:55:36,949][20381] Updated weights for policy 1, policy_version 620229 (0.0008) [2023-12-27 09:55:37,006][20381] Updated weights for policy 1, policy_version 620239 (0.0008) [2023-12-27 09:55:37,014][20347] Updated weights for policy 0, policy_version 620577 (0.0008) [2023-12-27 09:55:37,065][20347] Updated weights for policy 0, policy_version 620587 (0.0007) [2023-12-27 09:55:37,066][20381] Updated weights for policy 1, policy_version 620249 (0.0006) [2023-12-27 09:55:37,113][20347] Updated weights for policy 0, policy_version 620597 (0.0006) [2023-12-27 09:55:37,160][20347] Updated weights for policy 0, policy_version 620607 (0.0007) [2023-12-27 09:55:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20854.9). Total num frames: 317702144. Throughput: 0: 10623.0, 1: 10187.6. Samples: 317669524. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) [2023-12-27 09:55:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '190.820')] [2023-12-27 09:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000620256_158810112.pth... [2023-12-27 09:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000620608_158892032.pth... [2023-12-27 09:55:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000619392_158580736.pth [2023-12-27 09:55:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000619072_158507008.pth [2023-12-27 09:55:37,777][20381] Updated weights for policy 1, policy_version 620259 (0.0008) [2023-12-27 09:55:37,830][20381] Updated weights for policy 1, policy_version 620269 (0.0008) [2023-12-27 09:55:37,877][20347] Updated weights for policy 0, policy_version 620617 (0.0010) [2023-12-27 09:55:37,883][20381] Updated weights for policy 1, policy_version 620279 (0.0007) [2023-12-27 09:55:37,934][20347] Updated weights for policy 0, policy_version 620627 (0.0010) [2023-12-27 09:55:37,991][20347] Updated weights for policy 0, policy_version 620637 (0.0010) [2023-12-27 09:55:38,532][20381] Updated weights for policy 1, policy_version 620289 (0.0008) [2023-12-27 09:55:38,585][20381] Updated weights for policy 1, policy_version 620299 (0.0007) [2023-12-27 09:55:38,643][20381] Updated weights for policy 1, policy_version 620309 (0.0007) [2023-12-27 09:55:38,675][20347] Updated weights for policy 0, policy_version 620647 (0.0010) [2023-12-27 09:55:38,696][20381] Updated weights for policy 1, policy_version 620319 (0.0010) [2023-12-27 09:55:38,727][20347] Updated weights for policy 0, policy_version 620657 (0.0010) [2023-12-27 09:55:38,774][20347] Updated weights for policy 0, policy_version 620667 (0.0010) [2023-12-27 09:55:39,346][20381] Updated weights for policy 1, policy_version 620329 (0.0010) [2023-12-27 09:55:39,413][20381] Updated weights for policy 1, policy_version 620339 (0.0008) [2023-12-27 09:55:39,466][20347] Updated weights for policy 0, policy_version 620677 (0.0010) [2023-12-27 09:55:39,469][20381] Updated weights for policy 1, policy_version 620349 (0.0006) [2023-12-27 09:55:39,522][20347] Updated weights for policy 0, policy_version 620687 (0.0010) [2023-12-27 09:55:39,571][20347] Updated weights for policy 0, policy_version 620697 (0.0010) [2023-12-27 09:55:40,084][20381] Updated weights for policy 1, policy_version 620359 (0.0008) [2023-12-27 09:55:40,136][20381] Updated weights for policy 1, policy_version 620369 (0.0006) [2023-12-27 09:55:40,192][20381] Updated weights for policy 1, policy_version 620379 (0.0007) [2023-12-27 09:55:40,274][20347] Updated weights for policy 0, policy_version 620707 (0.0010) [2023-12-27 09:55:40,338][20347] Updated weights for policy 0, policy_version 620717 (0.0009) [2023-12-27 09:55:40,404][20347] Updated weights for policy 0, policy_version 620727 (0.0008) [2023-12-27 09:55:40,806][20381] Updated weights for policy 1, policy_version 620389 (0.0007) [2023-12-27 09:55:40,854][20381] Updated weights for policy 1, policy_version 620399 (0.0007) [2023-12-27 09:55:40,908][20381] Updated weights for policy 1, policy_version 620409 (0.0008) [2023-12-27 09:55:40,981][20347] Updated weights for policy 0, policy_version 620737 (0.0008) [2023-12-27 09:55:41,035][20347] Updated weights for policy 0, policy_version 620747 (0.0009) [2023-12-27 09:55:41,094][20347] Updated weights for policy 0, policy_version 620757 (0.0010) [2023-12-27 09:55:41,155][20347] Updated weights for policy 0, policy_version 620767 (0.0009) [2023-12-27 09:55:41,678][20381] Updated weights for policy 1, policy_version 620419 (0.0009) [2023-12-27 09:55:41,744][20381] Updated weights for policy 1, policy_version 620429 (0.0007) [2023-12-27 09:55:41,805][20381] Updated weights for policy 1, policy_version 620439 (0.0008) [2023-12-27 09:55:41,880][20347] Updated weights for policy 0, policy_version 620777 (0.0006) [2023-12-27 09:55:41,947][20347] Updated weights for policy 0, policy_version 620787 (0.0007) [2023-12-27 09:55:41,999][20347] Updated weights for policy 0, policy_version 620797 (0.0010) [2023-12-27 09:55:42,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 317800448. Throughput: 0: 10554.9, 1: 10154.2. Samples: 317792844. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:55:42,363][19170] Avg episode reward: [(0, '990.470'), (1, '232.860')] [2023-12-27 09:55:42,563][20381] Updated weights for policy 1, policy_version 620449 (0.0008) [2023-12-27 09:55:42,621][20381] Updated weights for policy 1, policy_version 620459 (0.0006) [2023-12-27 09:55:42,677][20381] Updated weights for policy 1, policy_version 620469 (0.0007) [2023-12-27 09:55:42,721][20381] Updated weights for policy 1, policy_version 620479 (0.0008) [2023-12-27 09:55:42,748][20347] Updated weights for policy 0, policy_version 620807 (0.0007) [2023-12-27 09:55:42,810][20347] Updated weights for policy 0, policy_version 620817 (0.0008) [2023-12-27 09:55:42,870][20347] Updated weights for policy 0, policy_version 620827 (0.0008) [2023-12-27 09:55:43,407][20381] Updated weights for policy 1, policy_version 620489 (0.0008) [2023-12-27 09:55:43,460][20381] Updated weights for policy 1, policy_version 620499 (0.0009) [2023-12-27 09:55:43,506][20381] Updated weights for policy 1, policy_version 620509 (0.0007) [2023-12-27 09:55:43,513][20347] Updated weights for policy 0, policy_version 620837 (0.0006) [2023-12-27 09:55:43,556][20347] Updated weights for policy 0, policy_version 620847 (0.0007) [2023-12-27 09:55:43,602][20347] Updated weights for policy 0, policy_version 620857 (0.0006) [2023-12-27 09:55:44,215][20381] Updated weights for policy 1, policy_version 620519 (0.0007) [2023-12-27 09:55:44,268][20347] Updated weights for policy 0, policy_version 620867 (0.0006) [2023-12-27 09:55:44,272][20381] Updated weights for policy 1, policy_version 620529 (0.0008) [2023-12-27 09:55:44,293][20291] KL-divergence is very high: 279.5337 [2023-12-27 09:55:44,304][20291] KL-divergence is very high: 161.0226 [2023-12-27 09:55:44,320][20347] Updated weights for policy 0, policy_version 620877 (0.0007) [2023-12-27 09:55:44,332][20381] Updated weights for policy 1, policy_version 620539 (0.0007) [2023-12-27 09:55:44,338][20291] KL-divergence is very high: 140.0069 [2023-12-27 09:55:44,379][20347] Updated weights for policy 0, policy_version 620887 (0.0007) [2023-12-27 09:55:44,980][20381] Updated weights for policy 1, policy_version 620549 (0.0006) [2023-12-27 09:55:45,040][20381] Updated weights for policy 1, policy_version 620559 (0.0008) [2023-12-27 09:55:45,087][20347] Updated weights for policy 0, policy_version 620897 (0.0007) [2023-12-27 09:55:45,089][20381] Updated weights for policy 1, policy_version 620569 (0.0007) [2023-12-27 09:55:45,095][20292] KL-divergence is very high: 159.1092 [2023-12-27 09:55:45,142][20347] Updated weights for policy 0, policy_version 620907 (0.0006) [2023-12-27 09:55:45,198][20347] Updated weights for policy 0, policy_version 620917 (0.0008) [2023-12-27 09:55:45,258][20347] Updated weights for policy 0, policy_version 620927 (0.0007) [2023-12-27 09:55:45,747][20381] Updated weights for policy 1, policy_version 620579 (0.0007) [2023-12-27 09:55:45,793][20381] Updated weights for policy 1, policy_version 620589 (0.0005) [2023-12-27 09:55:45,839][20381] Updated weights for policy 1, policy_version 620599 (0.0005) [2023-12-27 09:55:45,973][20347] Updated weights for policy 0, policy_version 620937 (0.0005) [2023-12-27 09:55:46,024][20347] Updated weights for policy 0, policy_version 620947 (0.0005) [2023-12-27 09:55:46,073][20347] Updated weights for policy 0, policy_version 620957 (0.0005) [2023-12-27 09:55:46,434][20381] Updated weights for policy 1, policy_version 620609 (0.0005) [2023-12-27 09:55:46,480][20381] Updated weights for policy 1, policy_version 620619 (0.0007) [2023-12-27 09:55:46,532][20381] Updated weights for policy 1, policy_version 620629 (0.0007) [2023-12-27 09:55:46,593][20381] Updated weights for policy 1, policy_version 620639 (0.0007) [2023-12-27 09:55:46,698][20347] Updated weights for policy 0, policy_version 620967 (0.0007) [2023-12-27 09:55:46,758][20347] Updated weights for policy 0, policy_version 620977 (0.0008) [2023-12-27 09:55:46,810][20347] Updated weights for policy 0, policy_version 620987 (0.0007) [2023-12-27 09:55:47,239][20381] Updated weights for policy 1, policy_version 620649 (0.0006) [2023-12-27 09:55:47,290][20381] Updated weights for policy 1, policy_version 620659 (0.0005) [2023-12-27 09:55:47,345][20381] Updated weights for policy 1, policy_version 620669 (0.0005) [2023-12-27 09:55:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 317906944. Throughput: 0: 10480.9, 1: 10183.4. Samples: 317915436. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:55:47,363][19170] Avg episode reward: [(0, '961.970'), (1, '257.380')] [2023-12-27 09:55:47,378][20347] Updated weights for policy 0, policy_version 620997 (0.0005) [2023-12-27 09:55:47,421][20291] KL-divergence is very high: 1151.7177 [2023-12-27 09:55:47,431][20347] Updated weights for policy 0, policy_version 621007 (0.0005) [2023-12-27 09:55:47,490][20347] Updated weights for policy 0, policy_version 621017 (0.0005) [2023-12-27 09:55:47,866][20381] Updated weights for policy 1, policy_version 620679 (0.0007) [2023-12-27 09:55:47,925][20381] Updated weights for policy 1, policy_version 620689 (0.0007) [2023-12-27 09:55:47,979][20381] Updated weights for policy 1, policy_version 620699 (0.0007) [2023-12-27 09:55:48,035][20347] Updated weights for policy 0, policy_version 621027 (0.0006) [2023-12-27 09:55:48,093][20347] Updated weights for policy 0, policy_version 621037 (0.0007) [2023-12-27 09:55:48,145][20347] Updated weights for policy 0, policy_version 621047 (0.0007) [2023-12-27 09:55:48,626][20381] Updated weights for policy 1, policy_version 620709 (0.0006) [2023-12-27 09:55:48,694][20381] Updated weights for policy 1, policy_version 620719 (0.0005) [2023-12-27 09:55:48,763][20381] Updated weights for policy 1, policy_version 620729 (0.0006) [2023-12-27 09:55:48,865][20347] Updated weights for policy 0, policy_version 621057 (0.0008) [2023-12-27 09:55:48,922][20347] Updated weights for policy 0, policy_version 621067 (0.0008) [2023-12-27 09:55:48,978][20347] Updated weights for policy 0, policy_version 621077 (0.0008) [2023-12-27 09:55:49,034][20347] Updated weights for policy 0, policy_version 621087 (0.0008) [2023-12-27 09:55:49,344][20381] Updated weights for policy 1, policy_version 620739 (0.0008) [2023-12-27 09:55:49,408][20381] Updated weights for policy 1, policy_version 620749 (0.0008) [2023-12-27 09:55:49,472][20381] Updated weights for policy 1, policy_version 620759 (0.0005) [2023-12-27 09:55:49,746][20347] Updated weights for policy 0, policy_version 621097 (0.0007) [2023-12-27 09:55:49,792][20347] Updated weights for policy 0, policy_version 621107 (0.0008) [2023-12-27 09:55:49,853][20347] Updated weights for policy 0, policy_version 621117 (0.0007) [2023-12-27 09:55:49,859][20291] KL-divergence is very high: 175.0705 [2023-12-27 09:55:50,134][20381] Updated weights for policy 1, policy_version 620769 (0.0007) [2023-12-27 09:55:50,189][20381] Updated weights for policy 1, policy_version 620779 (0.0006) [2023-12-27 09:55:50,241][20381] Updated weights for policy 1, policy_version 620789 (0.0006) [2023-12-27 09:55:50,292][20381] Updated weights for policy 1, policy_version 620799 (0.0005) [2023-12-27 09:55:50,552][20291] KL-divergence is very high: 107.4879 [2023-12-27 09:55:50,584][20347] Updated weights for policy 0, policy_version 621127 (0.0008) [2023-12-27 09:55:50,639][20347] Updated weights for policy 0, policy_version 621137 (0.0006) [2023-12-27 09:55:50,699][20347] Updated weights for policy 0, policy_version 621147 (0.0008) [2023-12-27 09:55:50,915][20381] Updated weights for policy 1, policy_version 620809 (0.0010) [2023-12-27 09:55:50,967][20381] Updated weights for policy 1, policy_version 620819 (0.0011) [2023-12-27 09:55:51,023][20381] Updated weights for policy 1, policy_version 620829 (0.0010) [2023-12-27 09:55:51,421][20347] Updated weights for policy 0, policy_version 621157 (0.0007) [2023-12-27 09:55:51,472][20347] Updated weights for policy 0, policy_version 621167 (0.0008) [2023-12-27 09:55:51,518][20347] Updated weights for policy 0, policy_version 621177 (0.0008) [2023-12-27 09:55:51,689][20381] Updated weights for policy 1, policy_version 620839 (0.0011) [2023-12-27 09:55:51,756][20381] Updated weights for policy 1, policy_version 620849 (0.0011) [2023-12-27 09:55:51,808][20381] Updated weights for policy 1, policy_version 620859 (0.0010) [2023-12-27 09:55:52,163][20347] Updated weights for policy 0, policy_version 621187 (0.0007) [2023-12-27 09:55:52,226][20347] Updated weights for policy 0, policy_version 621197 (0.0008) [2023-12-27 09:55:52,288][20347] Updated weights for policy 0, policy_version 621207 (0.0008) [2023-12-27 09:55:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 318013440. Throughput: 0: 10455.2, 1: 10253.5. Samples: 317980560. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:55:52,363][19170] Avg episode reward: [(0, '932.710'), (1, '188.040')] [2023-12-27 09:55:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000620864_158965760.pth... [2023-12-27 09:55:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000621216_159047680.pth... [2023-12-27 09:55:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000619648_158654464.pth [2023-12-27 09:55:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000620000_158736384.pth [2023-12-27 09:55:52,512][20381] Updated weights for policy 1, policy_version 620869 (0.0011) [2023-12-27 09:55:52,563][20381] Updated weights for policy 1, policy_version 620879 (0.0010) [2023-12-27 09:55:52,612][20381] Updated weights for policy 1, policy_version 620889 (0.0010) [2023-12-27 09:55:52,997][20347] Updated weights for policy 0, policy_version 621217 (0.0007) [2023-12-27 09:55:53,047][20347] Updated weights for policy 0, policy_version 621227 (0.0006) [2023-12-27 09:55:53,098][20347] Updated weights for policy 0, policy_version 621237 (0.0007) [2023-12-27 09:55:53,155][20347] Updated weights for policy 0, policy_version 621247 (0.0008) [2023-12-27 09:55:53,295][20381] Updated weights for policy 1, policy_version 620899 (0.0009) [2023-12-27 09:55:53,352][20381] Updated weights for policy 1, policy_version 620909 (0.0005) [2023-12-27 09:55:53,407][20381] Updated weights for policy 1, policy_version 620919 (0.0005) [2023-12-27 09:55:53,878][20347] Updated weights for policy 0, policy_version 621257 (0.0007) [2023-12-27 09:55:53,927][20347] Updated weights for policy 0, policy_version 621267 (0.0008) [2023-12-27 09:55:53,975][20347] Updated weights for policy 0, policy_version 621277 (0.0009) [2023-12-27 09:55:53,979][20381] Updated weights for policy 1, policy_version 620929 (0.0005) [2023-12-27 09:55:54,027][20381] Updated weights for policy 1, policy_version 620939 (0.0005) [2023-12-27 09:55:54,080][20381] Updated weights for policy 1, policy_version 620949 (0.0007) [2023-12-27 09:55:54,136][20381] Updated weights for policy 1, policy_version 620959 (0.0009) [2023-12-27 09:55:54,695][20347] Updated weights for policy 0, policy_version 621287 (0.0007) [2023-12-27 09:55:54,736][20291] KL-divergence is very high: 347.2067 [2023-12-27 09:55:54,747][20347] Updated weights for policy 0, policy_version 621297 (0.0008) [2023-12-27 09:55:54,810][20347] Updated weights for policy 0, policy_version 621307 (0.0008) [2023-12-27 09:55:54,829][20381] Updated weights for policy 1, policy_version 620969 (0.0006) [2023-12-27 09:55:54,880][20381] Updated weights for policy 1, policy_version 620979 (0.0005) [2023-12-27 09:55:54,935][20381] Updated weights for policy 1, policy_version 620989 (0.0008) [2023-12-27 09:55:55,513][20347] Updated weights for policy 0, policy_version 621317 (0.0007) [2023-12-27 09:55:55,562][20347] Updated weights for policy 0, policy_version 621327 (0.0007) [2023-12-27 09:55:55,592][20381] Updated weights for policy 1, policy_version 620999 (0.0011) [2023-12-27 09:55:55,611][20347] Updated weights for policy 0, policy_version 621337 (0.0008) [2023-12-27 09:55:55,637][20381] Updated weights for policy 1, policy_version 621009 (0.0010) [2023-12-27 09:55:55,692][20381] Updated weights for policy 1, policy_version 621019 (0.0010) [2023-12-27 09:55:56,326][20347] Updated weights for policy 0, policy_version 621347 (0.0007) [2023-12-27 09:55:56,360][20381] Updated weights for policy 1, policy_version 621029 (0.0011) [2023-12-27 09:55:56,377][20347] Updated weights for policy 0, policy_version 621357 (0.0006) [2023-12-27 09:55:56,413][20381] Updated weights for policy 1, policy_version 621039 (0.0010) [2023-12-27 09:55:56,428][20347] Updated weights for policy 0, policy_version 621367 (0.0007) [2023-12-27 09:55:56,467][20381] Updated weights for policy 1, policy_version 621049 (0.0011) [2023-12-27 09:55:57,067][20381] Updated weights for policy 1, policy_version 621059 (0.0010) [2023-12-27 09:55:57,124][20381] Updated weights for policy 1, policy_version 621069 (0.0007) [2023-12-27 09:55:57,147][20347] Updated weights for policy 0, policy_version 621377 (0.0008) [2023-12-27 09:55:57,174][20381] Updated weights for policy 1, policy_version 621079 (0.0006) [2023-12-27 09:55:57,201][20347] Updated weights for policy 0, policy_version 621387 (0.0006) [2023-12-27 09:55:57,252][20347] Updated weights for policy 0, policy_version 621397 (0.0008) [2023-12-27 09:55:57,298][20347] Updated weights for policy 0, policy_version 621407 (0.0009) [2023-12-27 09:55:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 318119936. Throughput: 0: 10343.4, 1: 10337.2. Samples: 318103476. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:55:57,363][19170] Avg episode reward: [(0, '903.950'), (1, '229.140')] [2023-12-27 09:55:57,803][20381] Updated weights for policy 1, policy_version 621089 (0.0007) [2023-12-27 09:55:57,860][20381] Updated weights for policy 1, policy_version 621099 (0.0009) [2023-12-27 09:55:57,912][20381] Updated weights for policy 1, policy_version 621109 (0.0008) [2023-12-27 09:55:57,963][20381] Updated weights for policy 1, policy_version 621119 (0.0008) [2023-12-27 09:55:57,997][20347] Updated weights for policy 0, policy_version 621417 (0.0006) [2023-12-27 09:55:58,049][20347] Updated weights for policy 0, policy_version 621427 (0.0007) [2023-12-27 09:55:58,099][20347] Updated weights for policy 0, policy_version 621437 (0.0008) [2023-12-27 09:55:58,676][20381] Updated weights for policy 1, policy_version 621129 (0.0009) [2023-12-27 09:55:58,737][20381] Updated weights for policy 1, policy_version 621139 (0.0007) [2023-12-27 09:55:58,797][20381] Updated weights for policy 1, policy_version 621149 (0.0007) [2023-12-27 09:55:58,840][20347] Updated weights for policy 0, policy_version 621447 (0.0008) [2023-12-27 09:55:58,902][20347] Updated weights for policy 0, policy_version 621457 (0.0007) [2023-12-27 09:55:58,970][20347] Updated weights for policy 0, policy_version 621467 (0.0009) [2023-12-27 09:55:59,609][20381] Updated weights for policy 1, policy_version 621159 (0.0007) [2023-12-27 09:55:59,661][20381] Updated weights for policy 1, policy_version 621169 (0.0006) [2023-12-27 09:55:59,685][20347] Updated weights for policy 0, policy_version 621477 (0.0007) [2023-12-27 09:55:59,709][20291] KL-divergence is very high: 170.4966 [2023-12-27 09:55:59,718][20381] Updated weights for policy 1, policy_version 621179 (0.0005) [2023-12-27 09:55:59,720][20291] KL-divergence is very high: 420.2346 [2023-12-27 09:55:59,727][20291] KL-divergence is very high: 393.3001 [2023-12-27 09:55:59,744][20347] Updated weights for policy 0, policy_version 621487 (0.0007) [2023-12-27 09:55:59,763][20291] KL-divergence is very high: 121.2611 [2023-12-27 09:55:59,790][20347] Updated weights for policy 0, policy_version 621497 (0.0006) [2023-12-27 09:56:00,312][20381] Updated weights for policy 1, policy_version 621189 (0.0005) [2023-12-27 09:56:00,361][20381] Updated weights for policy 1, policy_version 621199 (0.0005) [2023-12-27 09:56:00,406][20291] KL-divergence is very high: 413.4227 [2023-12-27 09:56:00,421][20381] Updated weights for policy 1, policy_version 621209 (0.0005) [2023-12-27 09:56:00,424][20347] Updated weights for policy 0, policy_version 621507 (0.0008) [2023-12-27 09:56:00,426][20291] KL-divergence is very high: 542.2545 [2023-12-27 09:56:00,455][20291] KL-divergence is very high: 273.7782 [2023-12-27 09:56:00,470][20291] KL-divergence is very high: 231.3649 [2023-12-27 09:56:00,476][20291] KL-divergence is very high: 106.7454 [2023-12-27 09:56:00,481][20347] Updated weights for policy 0, policy_version 621517 (0.0007) [2023-12-27 09:56:00,488][20291] KL-divergence is very high: 149.5072 [2023-12-27 09:56:00,517][20291] KL-divergence is very high: 211.2973 [2023-12-27 09:56:00,539][20347] Updated weights for policy 0, policy_version 621527 (0.0007) [2023-12-27 09:56:00,542][20291] KL-divergence is very high: 230.8208 [2023-12-27 09:56:00,556][20291] KL-divergence is very high: 258.6494 [2023-12-27 09:56:01,009][20381] Updated weights for policy 1, policy_version 621219 (0.0005) [2023-12-27 09:56:01,068][20381] Updated weights for policy 1, policy_version 621229 (0.0006) [2023-12-27 09:56:01,122][20381] Updated weights for policy 1, policy_version 621239 (0.0006) [2023-12-27 09:56:01,292][20347] Updated weights for policy 0, policy_version 621537 (0.0007) [2023-12-27 09:56:01,349][20347] Updated weights for policy 0, policy_version 621547 (0.0008) [2023-12-27 09:56:01,410][20347] Updated weights for policy 0, policy_version 621557 (0.0007) [2023-12-27 09:56:01,464][20347] Updated weights for policy 0, policy_version 621567 (0.0007) [2023-12-27 09:56:01,800][20381] Updated weights for policy 1, policy_version 621249 (0.0006) [2023-12-27 09:56:01,850][20381] Updated weights for policy 1, policy_version 621259 (0.0007) [2023-12-27 09:56:01,898][20381] Updated weights for policy 1, policy_version 621269 (0.0007) [2023-12-27 09:56:01,948][20381] Updated weights for policy 1, policy_version 621279 (0.0007) [2023-12-27 09:56:02,046][20347] Updated weights for policy 0, policy_version 621577 (0.0008) [2023-12-27 09:56:02,059][20291] KL-divergence is very high: 109.6990 [2023-12-27 09:56:02,100][20347] Updated weights for policy 0, policy_version 621587 (0.0007) [2023-12-27 09:56:02,152][20347] Updated weights for policy 0, policy_version 621597 (0.0005) [2023-12-27 09:56:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 318218240. Throughput: 0: 10315.7, 1: 10331.3. Samples: 318226988. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:02,363][19170] Avg episode reward: [(0, '873.040'), (1, '210.840')] [2023-12-27 09:56:02,584][20381] Updated weights for policy 1, policy_version 621289 (0.0005) [2023-12-27 09:56:02,644][20381] Updated weights for policy 1, policy_version 621299 (0.0005) [2023-12-27 09:56:02,689][20381] Updated weights for policy 1, policy_version 621309 (0.0008) [2023-12-27 09:56:02,837][20347] Updated weights for policy 0, policy_version 621607 (0.0007) [2023-12-27 09:56:02,893][20347] Updated weights for policy 0, policy_version 621617 (0.0007) [2023-12-27 09:56:02,945][20347] Updated weights for policy 0, policy_version 621627 (0.0007) [2023-12-27 09:56:03,357][20381] Updated weights for policy 1, policy_version 621319 (0.0010) [2023-12-27 09:56:03,405][20381] Updated weights for policy 1, policy_version 621329 (0.0010) [2023-12-27 09:56:03,450][20381] Updated weights for policy 1, policy_version 621339 (0.0010) [2023-12-27 09:56:03,634][20347] Updated weights for policy 0, policy_version 621637 (0.0007) [2023-12-27 09:56:03,688][20347] Updated weights for policy 0, policy_version 621647 (0.0007) [2023-12-27 09:56:03,740][20347] Updated weights for policy 0, policy_version 621657 (0.0007) [2023-12-27 09:56:04,152][20381] Updated weights for policy 1, policy_version 621349 (0.0009) [2023-12-27 09:56:04,219][20381] Updated weights for policy 1, policy_version 621359 (0.0009) [2023-12-27 09:56:04,280][20381] Updated weights for policy 1, policy_version 621369 (0.0007) [2023-12-27 09:56:04,489][20347] Updated weights for policy 0, policy_version 621667 (0.0008) [2023-12-27 09:56:04,551][20347] Updated weights for policy 0, policy_version 621677 (0.0010) [2023-12-27 09:56:04,613][20347] Updated weights for policy 0, policy_version 621687 (0.0010) [2023-12-27 09:56:04,982][20381] Updated weights for policy 1, policy_version 621379 (0.0008) [2023-12-27 09:56:05,035][20381] Updated weights for policy 1, policy_version 621389 (0.0007) [2023-12-27 09:56:05,096][20381] Updated weights for policy 1, policy_version 621399 (0.0008) [2023-12-27 09:56:05,246][20347] Updated weights for policy 0, policy_version 621697 (0.0010) [2023-12-27 09:56:05,304][20347] Updated weights for policy 0, policy_version 621707 (0.0010) [2023-12-27 09:56:05,367][20347] Updated weights for policy 0, policy_version 621717 (0.0010) [2023-12-27 09:56:05,421][20347] Updated weights for policy 0, policy_version 621727 (0.0010) [2023-12-27 09:56:05,793][20381] Updated weights for policy 1, policy_version 621409 (0.0007) [2023-12-27 09:56:05,858][20381] Updated weights for policy 1, policy_version 621419 (0.0005) [2023-12-27 09:56:05,918][20381] Updated weights for policy 1, policy_version 621429 (0.0005) [2023-12-27 09:56:05,983][20381] Updated weights for policy 1, policy_version 621439 (0.0006) [2023-12-27 09:56:06,025][20347] Updated weights for policy 0, policy_version 621737 (0.0007) [2023-12-27 09:56:06,070][20347] Updated weights for policy 0, policy_version 621747 (0.0005) [2023-12-27 09:56:06,144][20347] Updated weights for policy 0, policy_version 621757 (0.0006) [2023-12-27 09:56:06,650][20381] Updated weights for policy 1, policy_version 621449 (0.0007) [2023-12-27 09:56:06,706][20381] Updated weights for policy 1, policy_version 621459 (0.0008) [2023-12-27 09:56:06,764][20381] Updated weights for policy 1, policy_version 621469 (0.0008) [2023-12-27 09:56:06,786][20347] Updated weights for policy 0, policy_version 621767 (0.0006) [2023-12-27 09:56:06,840][20347] Updated weights for policy 0, policy_version 621777 (0.0006) [2023-12-27 09:56:06,886][20347] Updated weights for policy 0, policy_version 621787 (0.0005) [2023-12-27 09:56:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20854.9). Total num frames: 318316544. Throughput: 0: 10270.0, 1: 10379.5. Samples: 318287936. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:07,363][19170] Avg episode reward: [(0, '694.340'), (1, '167.580')] [2023-12-27 09:56:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000621472_159121408.pth... [2023-12-27 09:56:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000621792_159195136.pth... [2023-12-27 09:56:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000620608_158892032.pth [2023-12-27 09:56:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000620256_158810112.pth [2023-12-27 09:56:07,465][20381] Updated weights for policy 1, policy_version 621479 (0.0008) [2023-12-27 09:56:07,529][20381] Updated weights for policy 1, policy_version 621489 (0.0007) [2023-12-27 09:56:07,574][20347] Updated weights for policy 0, policy_version 621797 (0.0007) [2023-12-27 09:56:07,587][20381] Updated weights for policy 1, policy_version 621499 (0.0007) [2023-12-27 09:56:07,624][20347] Updated weights for policy 0, policy_version 621807 (0.0006) [2023-12-27 09:56:07,669][20347] Updated weights for policy 0, policy_version 621817 (0.0008) [2023-12-27 09:56:08,246][20381] Updated weights for policy 1, policy_version 621509 (0.0007) [2023-12-27 09:56:08,293][20381] Updated weights for policy 1, policy_version 621519 (0.0007) [2023-12-27 09:56:08,352][20381] Updated weights for policy 1, policy_version 621529 (0.0008) [2023-12-27 09:56:08,380][20347] Updated weights for policy 0, policy_version 621827 (0.0007) [2023-12-27 09:56:08,438][20347] Updated weights for policy 0, policy_version 621837 (0.0008) [2023-12-27 09:56:08,494][20347] Updated weights for policy 0, policy_version 621847 (0.0008) [2023-12-27 09:56:08,991][20381] Updated weights for policy 1, policy_version 621539 (0.0006) [2023-12-27 09:56:09,055][20381] Updated weights for policy 1, policy_version 621549 (0.0007) [2023-12-27 09:56:09,110][20381] Updated weights for policy 1, policy_version 621559 (0.0006) [2023-12-27 09:56:09,207][20347] Updated weights for policy 0, policy_version 621857 (0.0006) [2023-12-27 09:56:09,266][20347] Updated weights for policy 0, policy_version 621867 (0.0010) [2023-12-27 09:56:09,330][20347] Updated weights for policy 0, policy_version 621877 (0.0007) [2023-12-27 09:56:09,394][20347] Updated weights for policy 0, policy_version 621887 (0.0008) [2023-12-27 09:56:09,768][20381] Updated weights for policy 1, policy_version 621569 (0.0008) [2023-12-27 09:56:09,832][20381] Updated weights for policy 1, policy_version 621579 (0.0008) [2023-12-27 09:56:09,899][20381] Updated weights for policy 1, policy_version 621589 (0.0006) [2023-12-27 09:56:09,961][20381] Updated weights for policy 1, policy_version 621599 (0.0006) [2023-12-27 09:56:10,113][20347] Updated weights for policy 0, policy_version 621897 (0.0008) [2023-12-27 09:56:10,174][20347] Updated weights for policy 0, policy_version 621907 (0.0008) [2023-12-27 09:56:10,237][20347] Updated weights for policy 0, policy_version 621917 (0.0008) [2023-12-27 09:56:10,554][20381] Updated weights for policy 1, policy_version 621609 (0.0007) [2023-12-27 09:56:10,613][20381] Updated weights for policy 1, policy_version 621619 (0.0008) [2023-12-27 09:56:10,674][20381] Updated weights for policy 1, policy_version 621629 (0.0009) [2023-12-27 09:56:10,818][20347] Updated weights for policy 0, policy_version 621927 (0.0007) [2023-12-27 09:56:10,874][20347] Updated weights for policy 0, policy_version 621937 (0.0006) [2023-12-27 09:56:10,925][20347] Updated weights for policy 0, policy_version 621947 (0.0008) [2023-12-27 09:56:11,370][20381] Updated weights for policy 1, policy_version 621639 (0.0011) [2023-12-27 09:56:11,433][20381] Updated weights for policy 1, policy_version 621649 (0.0011) [2023-12-27 09:56:11,490][20381] Updated weights for policy 1, policy_version 621659 (0.0011) [2023-12-27 09:56:11,621][20347] Updated weights for policy 0, policy_version 621957 (0.0008) [2023-12-27 09:56:11,679][20347] Updated weights for policy 0, policy_version 621967 (0.0008) [2023-12-27 09:56:11,738][20347] Updated weights for policy 0, policy_version 621977 (0.0008) [2023-12-27 09:56:12,225][20381] Updated weights for policy 1, policy_version 621669 (0.0011) [2023-12-27 09:56:12,285][20381] Updated weights for policy 1, policy_version 621679 (0.0011) [2023-12-27 09:56:12,346][20381] Updated weights for policy 1, policy_version 621689 (0.0009) [2023-12-27 09:56:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20799.4). Total num frames: 318414848. Throughput: 0: 10229.9, 1: 10473.2. Samples: 318412228. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:12,363][19170] Avg episode reward: [(0, '703.420'), (1, '172.310')] [2023-12-27 09:56:12,437][20347] Updated weights for policy 0, policy_version 621987 (0.0007) [2023-12-27 09:56:12,504][20347] Updated weights for policy 0, policy_version 621997 (0.0007) [2023-12-27 09:56:12,566][20347] Updated weights for policy 0, policy_version 622007 (0.0007) [2023-12-27 09:56:13,014][20381] Updated weights for policy 1, policy_version 621699 (0.0007) [2023-12-27 09:56:13,071][20381] Updated weights for policy 1, policy_version 621709 (0.0005) [2023-12-27 09:56:13,132][20381] Updated weights for policy 1, policy_version 621719 (0.0006) [2023-12-27 09:56:13,219][20347] Updated weights for policy 0, policy_version 622017 (0.0007) [2023-12-27 09:56:13,273][20347] Updated weights for policy 0, policy_version 622027 (0.0005) [2023-12-27 09:56:13,328][20347] Updated weights for policy 0, policy_version 622037 (0.0007) [2023-12-27 09:56:13,377][20347] Updated weights for policy 0, policy_version 622047 (0.0007) [2023-12-27 09:56:13,751][20381] Updated weights for policy 1, policy_version 621729 (0.0006) [2023-12-27 09:56:13,810][20381] Updated weights for policy 1, policy_version 621739 (0.0007) [2023-12-27 09:56:13,869][20381] Updated weights for policy 1, policy_version 621749 (0.0007) [2023-12-27 09:56:13,919][20381] Updated weights for policy 1, policy_version 621759 (0.0007) [2023-12-27 09:56:13,965][20347] Updated weights for policy 0, policy_version 622057 (0.0008) [2023-12-27 09:56:14,017][20347] Updated weights for policy 0, policy_version 622067 (0.0008) [2023-12-27 09:56:14,067][20347] Updated weights for policy 0, policy_version 622077 (0.0007) [2023-12-27 09:56:14,588][20381] Updated weights for policy 1, policy_version 621769 (0.0008) [2023-12-27 09:56:14,645][20381] Updated weights for policy 1, policy_version 621779 (0.0007) [2023-12-27 09:56:14,702][20381] Updated weights for policy 1, policy_version 621789 (0.0007) [2023-12-27 09:56:14,758][20347] Updated weights for policy 0, policy_version 622087 (0.0007) [2023-12-27 09:56:14,822][20347] Updated weights for policy 0, policy_version 622097 (0.0007) [2023-12-27 09:56:14,881][20347] Updated weights for policy 0, policy_version 622107 (0.0007) [2023-12-27 09:56:15,348][20381] Updated weights for policy 1, policy_version 621799 (0.0008) [2023-12-27 09:56:15,414][20381] Updated weights for policy 1, policy_version 621809 (0.0007) [2023-12-27 09:56:15,475][20381] Updated weights for policy 1, policy_version 621819 (0.0005) [2023-12-27 09:56:15,570][20347] Updated weights for policy 0, policy_version 622117 (0.0008) [2023-12-27 09:56:15,625][20347] Updated weights for policy 0, policy_version 622127 (0.0009) [2023-12-27 09:56:15,680][20347] Updated weights for policy 0, policy_version 622137 (0.0010) [2023-12-27 09:56:15,696][20291] KL-divergence is very high: 105.8434 [2023-12-27 09:56:15,708][20291] KL-divergence is very high: 106.9640 [2023-12-27 09:56:16,047][20381] Updated weights for policy 1, policy_version 621829 (0.0005) [2023-12-27 09:56:16,098][20381] Updated weights for policy 1, policy_version 621839 (0.0005) [2023-12-27 09:56:16,153][20381] Updated weights for policy 1, policy_version 621849 (0.0005) [2023-12-27 09:56:16,336][20347] Updated weights for policy 0, policy_version 622147 (0.0010) [2023-12-27 09:56:16,402][20347] Updated weights for policy 0, policy_version 622157 (0.0007) [2023-12-27 09:56:16,467][20347] Updated weights for policy 0, policy_version 622167 (0.0006) [2023-12-27 09:56:16,662][20381] Updated weights for policy 1, policy_version 621859 (0.0005) [2023-12-27 09:56:16,712][20381] Updated weights for policy 1, policy_version 621869 (0.0006) [2023-12-27 09:56:16,760][20381] Updated weights for policy 1, policy_version 621879 (0.0008) [2023-12-27 09:56:17,013][20347] Updated weights for policy 0, policy_version 622177 (0.0005) [2023-12-27 09:56:17,061][20347] Updated weights for policy 0, policy_version 622187 (0.0006) [2023-12-27 09:56:17,113][20347] Updated weights for policy 0, policy_version 622197 (0.0007) [2023-12-27 09:56:17,181][20347] Updated weights for policy 0, policy_version 622207 (0.0007) [2023-12-27 09:56:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 318529536. Throughput: 0: 10201.6, 1: 10540.8. Samples: 318539248. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:17,363][19170] Avg episode reward: [(0, '773.250'), (1, '210.610')] [2023-12-27 09:56:17,459][20381] Updated weights for policy 1, policy_version 621889 (0.0008) [2023-12-27 09:56:17,520][20381] Updated weights for policy 1, policy_version 621899 (0.0010) [2023-12-27 09:56:17,567][20381] Updated weights for policy 1, policy_version 621909 (0.0010) [2023-12-27 09:56:17,615][20381] Updated weights for policy 1, policy_version 621919 (0.0010) [2023-12-27 09:56:17,764][20347] Updated weights for policy 0, policy_version 622217 (0.0005) [2023-12-27 09:56:17,819][20347] Updated weights for policy 0, policy_version 622227 (0.0005) [2023-12-27 09:56:17,875][20347] Updated weights for policy 0, policy_version 622237 (0.0005) [2023-12-27 09:56:18,303][20381] Updated weights for policy 1, policy_version 621929 (0.0007) [2023-12-27 09:56:18,361][20381] Updated weights for policy 1, policy_version 621939 (0.0007) [2023-12-27 09:56:18,421][20347] Updated weights for policy 0, policy_version 622247 (0.0006) [2023-12-27 09:56:18,423][20381] Updated weights for policy 1, policy_version 621949 (0.0007) [2023-12-27 09:56:18,480][20347] Updated weights for policy 0, policy_version 622257 (0.0007) [2023-12-27 09:56:18,545][20347] Updated weights for policy 0, policy_version 622267 (0.0008) [2023-12-27 09:56:19,144][20381] Updated weights for policy 1, policy_version 621959 (0.0005) [2023-12-27 09:56:19,146][20347] Updated weights for policy 0, policy_version 622277 (0.0007) [2023-12-27 09:56:19,198][20381] Updated weights for policy 1, policy_version 621969 (0.0006) [2023-12-27 09:56:19,200][20347] Updated weights for policy 0, policy_version 622287 (0.0006) [2023-12-27 09:56:19,260][20347] Updated weights for policy 0, policy_version 622297 (0.0007) [2023-12-27 09:56:19,261][20381] Updated weights for policy 1, policy_version 621979 (0.0007) [2023-12-27 09:56:19,913][20381] Updated weights for policy 1, policy_version 621989 (0.0005) [2023-12-27 09:56:19,975][20381] Updated weights for policy 1, policy_version 621999 (0.0008) [2023-12-27 09:56:20,037][20381] Updated weights for policy 1, policy_version 622009 (0.0007) [2023-12-27 09:56:20,074][20347] Updated weights for policy 0, policy_version 622307 (0.0007) [2023-12-27 09:56:20,127][20347] Updated weights for policy 0, policy_version 622317 (0.0008) [2023-12-27 09:56:20,186][20347] Updated weights for policy 0, policy_version 622327 (0.0008) [2023-12-27 09:56:20,665][20381] Updated weights for policy 1, policy_version 622019 (0.0006) [2023-12-27 09:56:20,716][20381] Updated weights for policy 1, policy_version 622029 (0.0008) [2023-12-27 09:56:20,766][20381] Updated weights for policy 1, policy_version 622039 (0.0008) [2023-12-27 09:56:20,928][20347] Updated weights for policy 0, policy_version 622337 (0.0008) [2023-12-27 09:56:20,981][20347] Updated weights for policy 0, policy_version 622347 (0.0007) [2023-12-27 09:56:21,043][20347] Updated weights for policy 0, policy_version 622357 (0.0007) [2023-12-27 09:56:21,114][20347] Updated weights for policy 0, policy_version 622367 (0.0008) [2023-12-27 09:56:21,468][20381] Updated weights for policy 1, policy_version 622049 (0.0008) [2023-12-27 09:56:21,521][20381] Updated weights for policy 1, policy_version 622059 (0.0007) [2023-12-27 09:56:21,578][20381] Updated weights for policy 1, policy_version 622069 (0.0008) [2023-12-27 09:56:21,642][20381] Updated weights for policy 1, policy_version 622079 (0.0007) [2023-12-27 09:56:21,849][20347] Updated weights for policy 0, policy_version 622377 (0.0008) [2023-12-27 09:56:21,906][20347] Updated weights for policy 0, policy_version 622387 (0.0008) [2023-12-27 09:56:21,959][20347] Updated weights for policy 0, policy_version 622397 (0.0007) [2023-12-27 09:56:22,249][20381] Updated weights for policy 1, policy_version 622089 (0.0006) [2023-12-27 09:56:22,307][20381] Updated weights for policy 1, policy_version 622099 (0.0009) [2023-12-27 09:56:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 318627840. Throughput: 0: 10245.9, 1: 10485.9. Samples: 318602452. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:22,363][19170] Avg episode reward: [(0, '633.350'), (1, '209.980')] [2023-12-27 09:56:22,368][20381] Updated weights for policy 1, policy_version 622109 (0.0011) [2023-12-27 09:56:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000622400_159350784.pth... [2023-12-27 09:56:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000621216_159047680.pth [2023-12-27 09:56:22,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000622112_159285248.pth... [2023-12-27 09:56:22,390][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000620864_158965760.pth [2023-12-27 09:56:22,635][20347] Updated weights for policy 0, policy_version 622407 (0.0007) [2023-12-27 09:56:22,684][20347] Updated weights for policy 0, policy_version 622417 (0.0007) [2023-12-27 09:56:22,742][20347] Updated weights for policy 0, policy_version 622427 (0.0007) [2023-12-27 09:56:23,029][20381] Updated weights for policy 1, policy_version 622119 (0.0007) [2023-12-27 09:56:23,090][20381] Updated weights for policy 1, policy_version 622129 (0.0005) [2023-12-27 09:56:23,149][20381] Updated weights for policy 1, policy_version 622139 (0.0010) [2023-12-27 09:56:23,359][20347] Updated weights for policy 0, policy_version 622437 (0.0008) [2023-12-27 09:56:23,418][20347] Updated weights for policy 0, policy_version 622447 (0.0008) [2023-12-27 09:56:23,480][20347] Updated weights for policy 0, policy_version 622457 (0.0007) [2023-12-27 09:56:23,683][20381] Updated weights for policy 1, policy_version 622149 (0.0008) [2023-12-27 09:56:23,739][20381] Updated weights for policy 1, policy_version 622159 (0.0010) [2023-12-27 09:56:23,797][20381] Updated weights for policy 1, policy_version 622169 (0.0011) [2023-12-27 09:56:24,177][20347] Updated weights for policy 0, policy_version 622467 (0.0008) [2023-12-27 09:56:24,228][20347] Updated weights for policy 0, policy_version 622477 (0.0007) [2023-12-27 09:56:24,285][20347] Updated weights for policy 0, policy_version 622487 (0.0007) [2023-12-27 09:56:24,500][20381] Updated weights for policy 1, policy_version 622179 (0.0011) [2023-12-27 09:56:24,556][20381] Updated weights for policy 1, policy_version 622189 (0.0011) [2023-12-27 09:56:24,612][20381] Updated weights for policy 1, policy_version 622199 (0.0011) [2023-12-27 09:56:24,968][20347] Updated weights for policy 0, policy_version 622497 (0.0006) [2023-12-27 09:56:25,040][20347] Updated weights for policy 0, policy_version 622507 (0.0009) [2023-12-27 09:56:25,095][20347] Updated weights for policy 0, policy_version 622517 (0.0010) [2023-12-27 09:56:25,140][20347] Updated weights for policy 0, policy_version 622527 (0.0010) [2023-12-27 09:56:25,191][20381] Updated weights for policy 1, policy_version 622209 (0.0010) [2023-12-27 09:56:25,246][20381] Updated weights for policy 1, policy_version 622219 (0.0006) [2023-12-27 09:56:25,298][20381] Updated weights for policy 1, policy_version 622229 (0.0010) [2023-12-27 09:56:25,343][20381] Updated weights for policy 1, policy_version 622239 (0.0010) [2023-12-27 09:56:25,683][20347] Updated weights for policy 0, policy_version 622537 (0.0006) [2023-12-27 09:56:25,741][20347] Updated weights for policy 0, policy_version 622547 (0.0006) [2023-12-27 09:56:25,795][20347] Updated weights for policy 0, policy_version 622557 (0.0010) [2023-12-27 09:56:26,022][20381] Updated weights for policy 1, policy_version 622249 (0.0011) [2023-12-27 09:56:26,080][20381] Updated weights for policy 1, policy_version 622259 (0.0011) [2023-12-27 09:56:26,140][20381] Updated weights for policy 1, policy_version 622269 (0.0011) [2023-12-27 09:56:26,410][20347] Updated weights for policy 0, policy_version 622567 (0.0010) [2023-12-27 09:56:26,455][20347] Updated weights for policy 0, policy_version 622577 (0.0010) [2023-12-27 09:56:26,506][20347] Updated weights for policy 0, policy_version 622587 (0.0010) [2023-12-27 09:56:26,774][20381] Updated weights for policy 1, policy_version 622279 (0.0011) [2023-12-27 09:56:26,831][20381] Updated weights for policy 1, policy_version 622289 (0.0010) [2023-12-27 09:56:26,880][20381] Updated weights for policy 1, policy_version 622299 (0.0010) [2023-12-27 09:56:27,189][20347] Updated weights for policy 0, policy_version 622597 (0.0010) [2023-12-27 09:56:27,243][20347] Updated weights for policy 0, policy_version 622607 (0.0010) [2023-12-27 09:56:27,297][20347] Updated weights for policy 0, policy_version 622617 (0.0010) [2023-12-27 09:56:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 318742528. Throughput: 0: 10281.2, 1: 10520.5. Samples: 318728920. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:27,363][19170] Avg episode reward: [(0, '714.860'), (1, '267.820')] [2023-12-27 09:56:27,559][20381] Updated weights for policy 1, policy_version 622309 (0.0010) [2023-12-27 09:56:27,614][20381] Updated weights for policy 1, policy_version 622319 (0.0009) [2023-12-27 09:56:27,675][20381] Updated weights for policy 1, policy_version 622329 (0.0007) [2023-12-27 09:56:27,995][20347] Updated weights for policy 0, policy_version 622627 (0.0010) [2023-12-27 09:56:28,043][20347] Updated weights for policy 0, policy_version 622637 (0.0010) [2023-12-27 09:56:28,090][20347] Updated weights for policy 0, policy_version 622647 (0.0008) [2023-12-27 09:56:28,375][20381] Updated weights for policy 1, policy_version 622339 (0.0009) [2023-12-27 09:56:28,425][20381] Updated weights for policy 1, policy_version 622349 (0.0008) [2023-12-27 09:56:28,485][20381] Updated weights for policy 1, policy_version 622359 (0.0010) [2023-12-27 09:56:28,739][20347] Updated weights for policy 0, policy_version 622657 (0.0006) [2023-12-27 09:56:28,793][20347] Updated weights for policy 0, policy_version 622667 (0.0009) [2023-12-27 09:56:28,851][20347] Updated weights for policy 0, policy_version 622677 (0.0010) [2023-12-27 09:56:28,910][20347] Updated weights for policy 0, policy_version 622687 (0.0011) [2023-12-27 09:56:29,147][20381] Updated weights for policy 1, policy_version 622369 (0.0011) [2023-12-27 09:56:29,204][20381] Updated weights for policy 1, policy_version 622379 (0.0011) [2023-12-27 09:56:29,265][20381] Updated weights for policy 1, policy_version 622389 (0.0008) [2023-12-27 09:56:29,321][20381] Updated weights for policy 1, policy_version 622399 (0.0008) [2023-12-27 09:56:29,558][20347] Updated weights for policy 0, policy_version 622697 (0.0010) [2023-12-27 09:56:29,614][20347] Updated weights for policy 0, policy_version 622707 (0.0010) [2023-12-27 09:56:29,664][20347] Updated weights for policy 0, policy_version 622717 (0.0007) [2023-12-27 09:56:30,048][20381] Updated weights for policy 1, policy_version 622409 (0.0007) [2023-12-27 09:56:30,104][20381] Updated weights for policy 1, policy_version 622419 (0.0006) [2023-12-27 09:56:30,164][20381] Updated weights for policy 1, policy_version 622429 (0.0005) [2023-12-27 09:56:30,321][20347] Updated weights for policy 0, policy_version 622727 (0.0005) [2023-12-27 09:56:30,371][20347] Updated weights for policy 0, policy_version 622737 (0.0005) [2023-12-27 09:56:30,418][20347] Updated weights for policy 0, policy_version 622747 (0.0005) [2023-12-27 09:56:30,710][20381] Updated weights for policy 1, policy_version 622439 (0.0007) [2023-12-27 09:56:30,763][20381] Updated weights for policy 1, policy_version 622449 (0.0007) [2023-12-27 09:56:30,819][20381] Updated weights for policy 1, policy_version 622459 (0.0007) [2023-12-27 09:56:31,036][20347] Updated weights for policy 0, policy_version 622757 (0.0007) [2023-12-27 09:56:31,095][20347] Updated weights for policy 0, policy_version 622767 (0.0008) [2023-12-27 09:56:31,161][20347] Updated weights for policy 0, policy_version 622777 (0.0006) [2023-12-27 09:56:31,536][20381] Updated weights for policy 1, policy_version 622469 (0.0007) [2023-12-27 09:56:31,598][20381] Updated weights for policy 1, policy_version 622479 (0.0009) [2023-12-27 09:56:31,657][20381] Updated weights for policy 1, policy_version 622489 (0.0007) [2023-12-27 09:56:31,817][20347] Updated weights for policy 0, policy_version 622787 (0.0007) [2023-12-27 09:56:31,876][20347] Updated weights for policy 0, policy_version 622797 (0.0005) [2023-12-27 09:56:31,934][20347] Updated weights for policy 0, policy_version 622807 (0.0006) [2023-12-27 09:56:32,362][20381] Updated weights for policy 1, policy_version 622499 (0.0006) [2023-12-27 09:56:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 318840832. Throughput: 0: 10349.9, 1: 10521.3. Samples: 318854640. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:32,363][19170] Avg episode reward: [(0, '760.600'), (1, '241.700')] [2023-12-27 09:56:32,414][20381] Updated weights for policy 1, policy_version 622509 (0.0008) [2023-12-27 09:56:32,475][20381] Updated weights for policy 1, policy_version 622519 (0.0006) [2023-12-27 09:56:32,565][20347] Updated weights for policy 0, policy_version 622817 (0.0005) [2023-12-27 09:56:32,614][20347] Updated weights for policy 0, policy_version 622827 (0.0008) [2023-12-27 09:56:32,664][20347] Updated weights for policy 0, policy_version 622837 (0.0008) [2023-12-27 09:56:32,725][20347] Updated weights for policy 0, policy_version 622847 (0.0008) [2023-12-27 09:56:33,087][20381] Updated weights for policy 1, policy_version 622529 (0.0006) [2023-12-27 09:56:33,141][20381] Updated weights for policy 1, policy_version 622539 (0.0008) [2023-12-27 09:56:33,196][20381] Updated weights for policy 1, policy_version 622549 (0.0008) [2023-12-27 09:56:33,248][20381] Updated weights for policy 1, policy_version 622559 (0.0008) [2023-12-27 09:56:33,380][20347] Updated weights for policy 0, policy_version 622857 (0.0007) [2023-12-27 09:56:33,434][20347] Updated weights for policy 0, policy_version 622867 (0.0007) [2023-12-27 09:56:33,481][20347] Updated weights for policy 0, policy_version 622877 (0.0007) [2023-12-27 09:56:33,871][20381] Updated weights for policy 1, policy_version 622569 (0.0006) [2023-12-27 09:56:33,916][20381] Updated weights for policy 1, policy_version 622579 (0.0005) [2023-12-27 09:56:33,971][20381] Updated weights for policy 1, policy_version 622589 (0.0005) [2023-12-27 09:56:34,230][20347] Updated weights for policy 0, policy_version 622887 (0.0006) [2023-12-27 09:56:34,297][20347] Updated weights for policy 0, policy_version 622897 (0.0006) [2023-12-27 09:56:34,362][20347] Updated weights for policy 0, policy_version 622907 (0.0006) [2023-12-27 09:56:34,602][20381] Updated weights for policy 1, policy_version 622599 (0.0005) [2023-12-27 09:56:34,661][20381] Updated weights for policy 1, policy_version 622609 (0.0006) [2023-12-27 09:56:34,724][20381] Updated weights for policy 1, policy_version 622619 (0.0005) [2023-12-27 09:56:35,024][20347] Updated weights for policy 0, policy_version 622917 (0.0007) [2023-12-27 09:56:35,082][20347] Updated weights for policy 0, policy_version 622927 (0.0007) [2023-12-27 09:56:35,131][20347] Updated weights for policy 0, policy_version 622937 (0.0007) [2023-12-27 09:56:35,326][20381] Updated weights for policy 1, policy_version 622629 (0.0007) [2023-12-27 09:56:35,378][20381] Updated weights for policy 1, policy_version 622639 (0.0005) [2023-12-27 09:56:35,427][20381] Updated weights for policy 1, policy_version 622649 (0.0005) [2023-12-27 09:56:35,761][20347] Updated weights for policy 0, policy_version 622947 (0.0007) [2023-12-27 09:56:35,815][20347] Updated weights for policy 0, policy_version 622957 (0.0005) [2023-12-27 09:56:35,869][20347] Updated weights for policy 0, policy_version 622967 (0.0007) [2023-12-27 09:56:36,044][20381] Updated weights for policy 1, policy_version 622659 (0.0006) [2023-12-27 09:56:36,089][20381] Updated weights for policy 1, policy_version 622669 (0.0009) [2023-12-27 09:56:36,148][20381] Updated weights for policy 1, policy_version 622679 (0.0008) [2023-12-27 09:56:36,476][20347] Updated weights for policy 0, policy_version 622977 (0.0006) [2023-12-27 09:56:36,531][20347] Updated weights for policy 0, policy_version 622987 (0.0009) [2023-12-27 09:56:36,594][20347] Updated weights for policy 0, policy_version 622997 (0.0006) [2023-12-27 09:56:36,659][20347] Updated weights for policy 0, policy_version 623007 (0.0008) [2023-12-27 09:56:36,898][20381] Updated weights for policy 1, policy_version 622689 (0.0008) [2023-12-27 09:56:36,946][20381] Updated weights for policy 1, policy_version 622699 (0.0008) [2023-12-27 09:56:37,001][20381] Updated weights for policy 1, policy_version 622709 (0.0008) [2023-12-27 09:56:37,056][20381] Updated weights for policy 1, policy_version 622720 (0.0008) [2023-12-27 09:56:37,279][20347] Updated weights for policy 0, policy_version 623017 (0.0010) [2023-12-27 09:56:37,339][20347] Updated weights for policy 0, policy_version 623027 (0.0010) [2023-12-27 09:56:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 318947328. Throughput: 0: 10321.0, 1: 10516.7. Samples: 318918256. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:37,363][19170] Avg episode reward: [(0, '817.270'), (1, '264.020')] [2023-12-27 09:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000622720_159440896.pth... [2023-12-27 09:56:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000621472_159121408.pth [2023-12-27 09:56:37,398][20347] Updated weights for policy 0, policy_version 623037 (0.0011) [2023-12-27 09:56:37,415][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000623040_159514624.pth... [2023-12-27 09:56:37,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000621792_159195136.pth [2023-12-27 09:56:37,783][20381] Updated weights for policy 1, policy_version 622731 (0.0010) [2023-12-27 09:56:37,839][20381] Updated weights for policy 1, policy_version 622741 (0.0005) [2023-12-27 09:56:37,896][20381] Updated weights for policy 1, policy_version 622751 (0.0005) [2023-12-27 09:56:38,060][20347] Updated weights for policy 0, policy_version 623047 (0.0007) [2023-12-27 09:56:38,109][20347] Updated weights for policy 0, policy_version 623057 (0.0005) [2023-12-27 09:56:38,164][20347] Updated weights for policy 0, policy_version 623067 (0.0005) [2023-12-27 09:56:38,454][20381] Updated weights for policy 1, policy_version 622761 (0.0009) [2023-12-27 09:56:38,503][20381] Updated weights for policy 1, policy_version 622771 (0.0010) [2023-12-27 09:56:38,555][20381] Updated weights for policy 1, policy_version 622781 (0.0010) [2023-12-27 09:56:38,766][20347] Updated weights for policy 0, policy_version 623077 (0.0005) [2023-12-27 09:56:38,820][20347] Updated weights for policy 0, policy_version 623087 (0.0005) [2023-12-27 09:56:38,871][20347] Updated weights for policy 0, policy_version 623097 (0.0005) [2023-12-27 09:56:39,166][20381] Updated weights for policy 1, policy_version 622791 (0.0007) [2023-12-27 09:56:39,221][20381] Updated weights for policy 1, policy_version 622801 (0.0006) [2023-12-27 09:56:39,279][20381] Updated weights for policy 1, policy_version 622811 (0.0009) [2023-12-27 09:56:39,435][20347] Updated weights for policy 0, policy_version 623107 (0.0006) [2023-12-27 09:56:39,500][20347] Updated weights for policy 0, policy_version 623117 (0.0009) [2023-12-27 09:56:39,562][20347] Updated weights for policy 0, policy_version 623127 (0.0010) [2023-12-27 09:56:40,052][20381] Updated weights for policy 1, policy_version 622821 (0.0009) [2023-12-27 09:56:40,112][20381] Updated weights for policy 1, policy_version 622831 (0.0008) [2023-12-27 09:56:40,173][20381] Updated weights for policy 1, policy_version 622841 (0.0008) [2023-12-27 09:56:40,241][20347] Updated weights for policy 0, policy_version 623137 (0.0008) [2023-12-27 09:56:40,297][20347] Updated weights for policy 0, policy_version 623147 (0.0009) [2023-12-27 09:56:40,349][20347] Updated weights for policy 0, policy_version 623157 (0.0006) [2023-12-27 09:56:40,413][20347] Updated weights for policy 0, policy_version 623167 (0.0010) [2023-12-27 09:56:40,773][20381] Updated weights for policy 1, policy_version 622851 (0.0007) [2023-12-27 09:56:40,825][20381] Updated weights for policy 1, policy_version 622861 (0.0005) [2023-12-27 09:56:40,882][20381] Updated weights for policy 1, policy_version 622871 (0.0006) [2023-12-27 09:56:41,053][20347] Updated weights for policy 0, policy_version 623177 (0.0011) [2023-12-27 09:56:41,116][20347] Updated weights for policy 0, policy_version 623187 (0.0011) [2023-12-27 09:56:41,177][20347] Updated weights for policy 0, policy_version 623197 (0.0011) [2023-12-27 09:56:41,591][20381] Updated weights for policy 1, policy_version 622881 (0.0007) [2023-12-27 09:56:41,655][20381] Updated weights for policy 1, policy_version 622891 (0.0008) [2023-12-27 09:56:41,728][20381] Updated weights for policy 1, policy_version 622901 (0.0008) [2023-12-27 09:56:41,792][20381] Updated weights for policy 1, policy_version 622911 (0.0008) [2023-12-27 09:56:41,916][20347] Updated weights for policy 0, policy_version 623207 (0.0008) [2023-12-27 09:56:41,968][20347] Updated weights for policy 0, policy_version 623217 (0.0007) [2023-12-27 09:56:42,014][20347] Updated weights for policy 0, policy_version 623227 (0.0007) [2023-12-27 09:56:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 319053824. Throughput: 0: 10441.2, 1: 10515.5. Samples: 319046532. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:42,363][19170] Avg episode reward: [(0, '857.620'), (1, '298.990')] [2023-12-27 09:56:42,461][20381] Updated weights for policy 1, policy_version 622921 (0.0008) [2023-12-27 09:56:42,507][20381] Updated weights for policy 1, policy_version 622931 (0.0007) [2023-12-27 09:56:42,554][20381] Updated weights for policy 1, policy_version 622941 (0.0009) [2023-12-27 09:56:42,715][20347] Updated weights for policy 0, policy_version 623237 (0.0009) [2023-12-27 09:56:42,774][20347] Updated weights for policy 0, policy_version 623247 (0.0007) [2023-12-27 09:56:42,828][20347] Updated weights for policy 0, policy_version 623257 (0.0007) [2023-12-27 09:56:43,277][20381] Updated weights for policy 1, policy_version 622951 (0.0009) [2023-12-27 09:56:43,332][20381] Updated weights for policy 1, policy_version 622961 (0.0010) [2023-12-27 09:56:43,384][20381] Updated weights for policy 1, policy_version 622971 (0.0007) [2023-12-27 09:56:43,474][20347] Updated weights for policy 0, policy_version 623267 (0.0006) [2023-12-27 09:56:43,528][20347] Updated weights for policy 0, policy_version 623277 (0.0005) [2023-12-27 09:56:43,584][20347] Updated weights for policy 0, policy_version 623287 (0.0006) [2023-12-27 09:56:44,010][20381] Updated weights for policy 1, policy_version 622981 (0.0010) [2023-12-27 09:56:44,055][20381] Updated weights for policy 1, policy_version 622991 (0.0010) [2023-12-27 09:56:44,100][20381] Updated weights for policy 1, policy_version 623001 (0.0010) [2023-12-27 09:56:44,126][20347] Updated weights for policy 0, policy_version 623297 (0.0005) [2023-12-27 09:56:44,189][20347] Updated weights for policy 0, policy_version 623307 (0.0006) [2023-12-27 09:56:44,252][20347] Updated weights for policy 0, policy_version 623317 (0.0005) [2023-12-27 09:56:44,301][20347] Updated weights for policy 0, policy_version 623327 (0.0007) [2023-12-27 09:56:44,795][20381] Updated weights for policy 1, policy_version 623011 (0.0010) [2023-12-27 09:56:44,854][20381] Updated weights for policy 1, policy_version 623021 (0.0007) [2023-12-27 09:56:44,913][20381] Updated weights for policy 1, policy_version 623031 (0.0008) [2023-12-27 09:56:44,965][20347] Updated weights for policy 0, policy_version 623337 (0.0007) [2023-12-27 09:56:45,024][20347] Updated weights for policy 0, policy_version 623347 (0.0007) [2023-12-27 09:56:45,087][20347] Updated weights for policy 0, policy_version 623357 (0.0007) [2023-12-27 09:56:45,598][20381] Updated weights for policy 1, policy_version 623041 (0.0010) [2023-12-27 09:56:45,650][20381] Updated weights for policy 1, policy_version 623051 (0.0011) [2023-12-27 09:56:45,674][20347] Updated weights for policy 0, policy_version 623367 (0.0006) [2023-12-27 09:56:45,701][20381] Updated weights for policy 1, policy_version 623061 (0.0010) [2023-12-27 09:56:45,724][20347] Updated weights for policy 0, policy_version 623377 (0.0005) [2023-12-27 09:56:45,757][20381] Updated weights for policy 1, policy_version 623071 (0.0011) [2023-12-27 09:56:45,783][20347] Updated weights for policy 0, policy_version 623387 (0.0006) [2023-12-27 09:56:46,352][20347] Updated weights for policy 0, policy_version 623397 (0.0007) [2023-12-27 09:56:46,402][20347] Updated weights for policy 0, policy_version 623407 (0.0007) [2023-12-27 09:56:46,451][20347] Updated weights for policy 0, policy_version 623417 (0.0006) [2023-12-27 09:56:46,453][20381] Updated weights for policy 1, policy_version 623081 (0.0011) [2023-12-27 09:56:46,508][20381] Updated weights for policy 1, policy_version 623091 (0.0010) [2023-12-27 09:56:46,563][20381] Updated weights for policy 1, policy_version 623101 (0.0010) [2023-12-27 09:56:47,032][20347] Updated weights for policy 0, policy_version 623427 (0.0005) [2023-12-27 09:56:47,091][20347] Updated weights for policy 0, policy_version 623437 (0.0007) [2023-12-27 09:56:47,152][20347] Updated weights for policy 0, policy_version 623447 (0.0009) [2023-12-27 09:56:47,232][20381] Updated weights for policy 1, policy_version 623111 (0.0008) [2023-12-27 09:56:47,287][20381] Updated weights for policy 1, policy_version 623121 (0.0007) [2023-12-27 09:56:47,340][20381] Updated weights for policy 1, policy_version 623131 (0.0008) [2023-12-27 09:56:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 319160320. Throughput: 0: 10551.9, 1: 10479.4. Samples: 319173400. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:47,363][19170] Avg episode reward: [(0, '866.130'), (1, '220.520')] [2023-12-27 09:56:47,778][20347] Updated weights for policy 0, policy_version 623457 (0.0007) [2023-12-27 09:56:47,839][20347] Updated weights for policy 0, policy_version 623467 (0.0005) [2023-12-27 09:56:47,898][20347] Updated weights for policy 0, policy_version 623477 (0.0006) [2023-12-27 09:56:47,942][20381] Updated weights for policy 1, policy_version 623141 (0.0005) [2023-12-27 09:56:47,956][20347] Updated weights for policy 0, policy_version 623487 (0.0008) [2023-12-27 09:56:47,991][20381] Updated weights for policy 1, policy_version 623151 (0.0006) [2023-12-27 09:56:48,049][20381] Updated weights for policy 1, policy_version 623161 (0.0006) [2023-12-27 09:56:48,595][20347] Updated weights for policy 0, policy_version 623497 (0.0006) [2023-12-27 09:56:48,658][20347] Updated weights for policy 0, policy_version 623507 (0.0005) [2023-12-27 09:56:48,717][20347] Updated weights for policy 0, policy_version 623517 (0.0005) [2023-12-27 09:56:48,724][20381] Updated weights for policy 1, policy_version 623171 (0.0006) [2023-12-27 09:56:48,776][20381] Updated weights for policy 1, policy_version 623181 (0.0010) [2023-12-27 09:56:48,825][20381] Updated weights for policy 1, policy_version 623191 (0.0010) [2023-12-27 09:56:49,279][20347] Updated weights for policy 0, policy_version 623527 (0.0006) [2023-12-27 09:56:49,335][20347] Updated weights for policy 0, policy_version 623537 (0.0006) [2023-12-27 09:56:49,395][20347] Updated weights for policy 0, policy_version 623547 (0.0007) [2023-12-27 09:56:49,536][20381] Updated weights for policy 1, policy_version 623201 (0.0007) [2023-12-27 09:56:49,599][20381] Updated weights for policy 1, policy_version 623211 (0.0005) [2023-12-27 09:56:49,659][20381] Updated weights for policy 1, policy_version 623221 (0.0007) [2023-12-27 09:56:49,719][20381] Updated weights for policy 1, policy_version 623231 (0.0005) [2023-12-27 09:56:50,150][20347] Updated weights for policy 0, policy_version 623557 (0.0007) [2023-12-27 09:56:50,211][20347] Updated weights for policy 0, policy_version 623567 (0.0007) [2023-12-27 09:56:50,273][20347] Updated weights for policy 0, policy_version 623577 (0.0006) [2023-12-27 09:56:50,381][20381] Updated weights for policy 1, policy_version 623241 (0.0008) [2023-12-27 09:56:50,441][20381] Updated weights for policy 1, policy_version 623251 (0.0008) [2023-12-27 09:56:50,502][20381] Updated weights for policy 1, policy_version 623261 (0.0007) [2023-12-27 09:56:50,964][20347] Updated weights for policy 0, policy_version 623587 (0.0007) [2023-12-27 09:56:51,030][20347] Updated weights for policy 0, policy_version 623597 (0.0008) [2023-12-27 09:56:51,095][20347] Updated weights for policy 0, policy_version 623607 (0.0007) [2023-12-27 09:56:51,197][20381] Updated weights for policy 1, policy_version 623271 (0.0006) [2023-12-27 09:56:51,255][20381] Updated weights for policy 1, policy_version 623281 (0.0008) [2023-12-27 09:56:51,306][20381] Updated weights for policy 1, policy_version 623291 (0.0008) [2023-12-27 09:56:51,810][20347] Updated weights for policy 0, policy_version 623617 (0.0008) [2023-12-27 09:56:51,872][20347] Updated weights for policy 0, policy_version 623627 (0.0008) [2023-12-27 09:56:51,933][20347] Updated weights for policy 0, policy_version 623637 (0.0007) [2023-12-27 09:56:51,992][20347] Updated weights for policy 0, policy_version 623647 (0.0011) [2023-12-27 09:56:52,031][20381] Updated weights for policy 1, policy_version 623301 (0.0007) [2023-12-27 09:56:52,087][20381] Updated weights for policy 1, policy_version 623311 (0.0005) [2023-12-27 09:56:52,143][20381] Updated weights for policy 1, policy_version 623321 (0.0005) [2023-12-27 09:56:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.7, 300 sec: 20827.1). Total num frames: 319266816. Throughput: 0: 10585.9, 1: 10491.8. Samples: 319236436. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:52,363][19170] Avg episode reward: [(0, '895.410'), (1, '260.270')] [2023-12-27 09:56:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000623648_159670272.pth... [2023-12-27 09:56:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000623328_159596544.pth... [2023-12-27 09:56:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000622400_159350784.pth [2023-12-27 09:56:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000622112_159285248.pth [2023-12-27 09:56:52,690][20347] Updated weights for policy 0, policy_version 623657 (0.0010) [2023-12-27 09:56:52,742][20347] Updated weights for policy 0, policy_version 623667 (0.0010) [2023-12-27 09:56:52,747][20381] Updated weights for policy 1, policy_version 623331 (0.0007) [2023-12-27 09:56:52,799][20347] Updated weights for policy 0, policy_version 623677 (0.0010) [2023-12-27 09:56:52,799][20381] Updated weights for policy 1, policy_version 623341 (0.0010) [2023-12-27 09:56:52,851][20381] Updated weights for policy 1, policy_version 623351 (0.0010) [2023-12-27 09:56:53,458][20381] Updated weights for policy 1, policy_version 623361 (0.0006) [2023-12-27 09:56:53,486][20347] Updated weights for policy 0, policy_version 623687 (0.0010) [2023-12-27 09:56:53,503][20381] Updated weights for policy 1, policy_version 623371 (0.0006) [2023-12-27 09:56:53,543][20347] Updated weights for policy 0, policy_version 623697 (0.0006) [2023-12-27 09:56:53,556][20381] Updated weights for policy 1, policy_version 623381 (0.0006) [2023-12-27 09:56:53,591][20347] Updated weights for policy 0, policy_version 623707 (0.0010) [2023-12-27 09:56:53,612][20381] Updated weights for policy 1, policy_version 623391 (0.0005) [2023-12-27 09:56:54,143][20381] Updated weights for policy 1, policy_version 623401 (0.0005) [2023-12-27 09:56:54,197][20381] Updated weights for policy 1, policy_version 623411 (0.0006) [2023-12-27 09:56:54,253][20381] Updated weights for policy 1, policy_version 623421 (0.0005) [2023-12-27 09:56:54,270][20347] Updated weights for policy 0, policy_version 623717 (0.0009) [2023-12-27 09:56:54,317][20347] Updated weights for policy 0, policy_version 623727 (0.0007) [2023-12-27 09:56:54,368][20347] Updated weights for policy 0, policy_version 623737 (0.0007) [2023-12-27 09:56:54,821][20381] Updated weights for policy 1, policy_version 623431 (0.0005) [2023-12-27 09:56:54,879][20381] Updated weights for policy 1, policy_version 623441 (0.0007) [2023-12-27 09:56:54,933][20381] Updated weights for policy 1, policy_version 623451 (0.0007) [2023-12-27 09:56:55,076][20347] Updated weights for policy 0, policy_version 623747 (0.0007) [2023-12-27 09:56:55,126][20347] Updated weights for policy 0, policy_version 623757 (0.0005) [2023-12-27 09:56:55,194][20347] Updated weights for policy 0, policy_version 623767 (0.0006) [2023-12-27 09:56:55,590][20381] Updated weights for policy 1, policy_version 623461 (0.0007) [2023-12-27 09:56:55,640][20381] Updated weights for policy 1, policy_version 623471 (0.0006) [2023-12-27 09:56:55,695][20381] Updated weights for policy 1, policy_version 623481 (0.0005) [2023-12-27 09:56:55,772][20347] Updated weights for policy 0, policy_version 623777 (0.0010) [2023-12-27 09:56:55,817][20347] Updated weights for policy 0, policy_version 623787 (0.0010) [2023-12-27 09:56:55,865][20347] Updated weights for policy 0, policy_version 623797 (0.0010) [2023-12-27 09:56:55,918][20347] Updated weights for policy 0, policy_version 623807 (0.0005) [2023-12-27 09:56:56,321][20381] Updated weights for policy 1, policy_version 623491 (0.0005) [2023-12-27 09:56:56,382][20381] Updated weights for policy 1, policy_version 623501 (0.0005) [2023-12-27 09:56:56,442][20381] Updated weights for policy 1, policy_version 623511 (0.0006) [2023-12-27 09:56:56,598][20347] Updated weights for policy 0, policy_version 623817 (0.0010) [2023-12-27 09:56:56,658][20347] Updated weights for policy 0, policy_version 623827 (0.0010) [2023-12-27 09:56:56,715][20347] Updated weights for policy 0, policy_version 623837 (0.0010) [2023-12-27 09:56:56,936][20381] Updated weights for policy 1, policy_version 623521 (0.0005) [2023-12-27 09:56:56,983][20381] Updated weights for policy 1, policy_version 623531 (0.0006) [2023-12-27 09:56:57,031][20381] Updated weights for policy 1, policy_version 623541 (0.0007) [2023-12-27 09:56:57,078][20381] Updated weights for policy 1, policy_version 623551 (0.0007) [2023-12-27 09:56:57,347][20347] Updated weights for policy 0, policy_version 623847 (0.0008) [2023-12-27 09:56:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 319373312. Throughput: 0: 10560.3, 1: 10563.7. Samples: 319362808. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:56:57,363][19170] Avg episode reward: [(0, '942.420'), (1, '255.540')] [2023-12-27 09:56:57,402][20347] Updated weights for policy 0, policy_version 623857 (0.0007) [2023-12-27 09:56:57,464][20347] Updated weights for policy 0, policy_version 623867 (0.0008) [2023-12-27 09:56:57,721][20381] Updated weights for policy 1, policy_version 623561 (0.0007) [2023-12-27 09:56:57,776][20381] Updated weights for policy 1, policy_version 623571 (0.0007) [2023-12-27 09:56:57,834][20381] Updated weights for policy 1, policy_version 623581 (0.0008) [2023-12-27 09:56:58,105][20347] Updated weights for policy 0, policy_version 623877 (0.0007) [2023-12-27 09:56:58,167][20347] Updated weights for policy 0, policy_version 623887 (0.0008) [2023-12-27 09:56:58,234][20347] Updated weights for policy 0, policy_version 623897 (0.0008) [2023-12-27 09:56:58,613][20381] Updated weights for policy 1, policy_version 623591 (0.0008) [2023-12-27 09:56:58,677][20381] Updated weights for policy 1, policy_version 623601 (0.0008) [2023-12-27 09:56:58,739][20381] Updated weights for policy 1, policy_version 623611 (0.0008) [2023-12-27 09:56:59,013][20347] Updated weights for policy 0, policy_version 623907 (0.0009) [2023-12-27 09:56:59,071][20347] Updated weights for policy 0, policy_version 623917 (0.0010) [2023-12-27 09:56:59,133][20347] Updated weights for policy 0, policy_version 623927 (0.0010) [2023-12-27 09:56:59,492][20381] Updated weights for policy 1, policy_version 623621 (0.0009) [2023-12-27 09:56:59,547][20381] Updated weights for policy 1, policy_version 623631 (0.0011) [2023-12-27 09:56:59,606][20381] Updated weights for policy 1, policy_version 623641 (0.0011) [2023-12-27 09:56:59,801][20347] Updated weights for policy 0, policy_version 623937 (0.0010) [2023-12-27 09:56:59,860][20347] Updated weights for policy 0, policy_version 623947 (0.0008) [2023-12-27 09:56:59,927][20347] Updated weights for policy 0, policy_version 623957 (0.0008) [2023-12-27 09:56:59,979][20347] Updated weights for policy 0, policy_version 623967 (0.0010) [2023-12-27 09:57:00,268][20381] Updated weights for policy 1, policy_version 623651 (0.0011) [2023-12-27 09:57:00,320][20381] Updated weights for policy 1, policy_version 623661 (0.0010) [2023-12-27 09:57:00,372][20381] Updated weights for policy 1, policy_version 623671 (0.0010) [2023-12-27 09:57:00,611][20347] Updated weights for policy 0, policy_version 623977 (0.0006) [2023-12-27 09:57:00,661][20347] Updated weights for policy 0, policy_version 623987 (0.0005) [2023-12-27 09:57:00,716][20347] Updated weights for policy 0, policy_version 623997 (0.0006) [2023-12-27 09:57:01,045][20381] Updated weights for policy 1, policy_version 623681 (0.0010) [2023-12-27 09:57:01,110][20381] Updated weights for policy 1, policy_version 623691 (0.0010) [2023-12-27 09:57:01,167][20381] Updated weights for policy 1, policy_version 623701 (0.0007) [2023-12-27 09:57:01,223][20381] Updated weights for policy 1, policy_version 623711 (0.0007) [2023-12-27 09:57:01,375][20347] Updated weights for policy 0, policy_version 624007 (0.0009) [2023-12-27 09:57:01,431][20347] Updated weights for policy 0, policy_version 624017 (0.0009) [2023-12-27 09:57:01,493][20347] Updated weights for policy 0, policy_version 624027 (0.0008) [2023-12-27 09:57:01,887][20381] Updated weights for policy 1, policy_version 623721 (0.0010) [2023-12-27 09:57:01,947][20381] Updated weights for policy 1, policy_version 623731 (0.0009) [2023-12-27 09:57:02,005][20381] Updated weights for policy 1, policy_version 623741 (0.0008) [2023-12-27 09:57:02,183][20347] Updated weights for policy 0, policy_version 624037 (0.0007) [2023-12-27 09:57:02,240][20347] Updated weights for policy 0, policy_version 624047 (0.0007) [2023-12-27 09:57:02,298][20347] Updated weights for policy 0, policy_version 624057 (0.0008) [2023-12-27 09:57:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 319479808. Throughput: 0: 10524.6, 1: 10529.7. Samples: 319486696. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:57:02,363][19170] Avg episode reward: [(0, '923.010'), (1, '287.010')] [2023-12-27 09:57:02,642][20381] Updated weights for policy 1, policy_version 623751 (0.0009) [2023-12-27 09:57:02,700][20381] Updated weights for policy 1, policy_version 623761 (0.0011) [2023-12-27 09:57:02,762][20381] Updated weights for policy 1, policy_version 623771 (0.0011) [2023-12-27 09:57:02,968][20347] Updated weights for policy 0, policy_version 624067 (0.0007) [2023-12-27 09:57:03,023][20347] Updated weights for policy 0, policy_version 624077 (0.0006) [2023-12-27 09:57:03,069][20347] Updated weights for policy 0, policy_version 624087 (0.0007) [2023-12-27 09:57:03,405][20381] Updated weights for policy 1, policy_version 623781 (0.0011) [2023-12-27 09:57:03,460][20381] Updated weights for policy 1, policy_version 623791 (0.0010) [2023-12-27 09:57:03,516][20381] Updated weights for policy 1, policy_version 623801 (0.0008) [2023-12-27 09:57:03,761][20347] Updated weights for policy 0, policy_version 624097 (0.0007) [2023-12-27 09:57:03,806][20347] Updated weights for policy 0, policy_version 624107 (0.0007) [2023-12-27 09:57:03,864][20347] Updated weights for policy 0, policy_version 624117 (0.0008) [2023-12-27 09:57:03,929][20347] Updated weights for policy 0, policy_version 624127 (0.0011) [2023-12-27 09:57:04,157][20381] Updated weights for policy 1, policy_version 623811 (0.0007) [2023-12-27 09:57:04,212][20381] Updated weights for policy 1, policy_version 623821 (0.0007) [2023-12-27 09:57:04,270][20381] Updated weights for policy 1, policy_version 623831 (0.0007) [2023-12-27 09:57:04,647][20347] Updated weights for policy 0, policy_version 624137 (0.0008) [2023-12-27 09:57:04,705][20347] Updated weights for policy 0, policy_version 624147 (0.0010) [2023-12-27 09:57:04,763][20347] Updated weights for policy 0, policy_version 624157 (0.0010) [2023-12-27 09:57:04,928][20381] Updated weights for policy 1, policy_version 623841 (0.0006) [2023-12-27 09:57:04,986][20381] Updated weights for policy 1, policy_version 623851 (0.0007) [2023-12-27 09:57:05,045][20381] Updated weights for policy 1, policy_version 623861 (0.0007) [2023-12-27 09:57:05,102][20381] Updated weights for policy 1, policy_version 623871 (0.0008) [2023-12-27 09:57:05,344][20347] Updated weights for policy 0, policy_version 624167 (0.0007) [2023-12-27 09:57:05,392][20347] Updated weights for policy 0, policy_version 624177 (0.0005) [2023-12-27 09:57:05,442][20347] Updated weights for policy 0, policy_version 624187 (0.0005) [2023-12-27 09:57:05,831][20381] Updated weights for policy 1, policy_version 623881 (0.0007) [2023-12-27 09:57:05,877][20381] Updated weights for policy 1, policy_version 623891 (0.0005) [2023-12-27 09:57:05,923][20381] Updated weights for policy 1, policy_version 623901 (0.0005) [2023-12-27 09:57:05,981][20347] Updated weights for policy 0, policy_version 624197 (0.0007) [2023-12-27 09:57:06,034][20347] Updated weights for policy 0, policy_version 624207 (0.0009) [2023-12-27 09:57:06,091][20347] Updated weights for policy 0, policy_version 624217 (0.0005) [2023-12-27 09:57:06,609][20381] Updated weights for policy 1, policy_version 623911 (0.0009) [2023-12-27 09:57:06,668][20381] Updated weights for policy 1, policy_version 623921 (0.0010) [2023-12-27 09:57:06,727][20381] Updated weights for policy 1, policy_version 623931 (0.0010) [2023-12-27 09:57:06,852][20347] Updated weights for policy 0, policy_version 624227 (0.0008) [2023-12-27 09:57:06,904][20347] Updated weights for policy 0, policy_version 624237 (0.0008) [2023-12-27 09:57:06,953][20347] Updated weights for policy 0, policy_version 624247 (0.0007) [2023-12-27 09:57:07,339][20381] Updated weights for policy 1, policy_version 623941 (0.0010) [2023-12-27 09:57:07,362][19170] Fps is (10 sec: 20479.6, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 319578112. Throughput: 0: 10490.8, 1: 10537.0. Samples: 319548708. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:57:07,363][19170] Avg episode reward: [(0, '912.940'), (1, '271.250')] [2023-12-27 09:57:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000624256_159825920.pth... [2023-12-27 09:57:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000623040_159514624.pth [2023-12-27 09:57:07,394][20381] Updated weights for policy 1, policy_version 623951 (0.0010) [2023-12-27 09:57:07,452][20381] Updated weights for policy 1, policy_version 623961 (0.0010) [2023-12-27 09:57:07,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000623968_159760384.pth... [2023-12-27 09:57:07,485][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000622720_159440896.pth [2023-12-27 09:57:07,637][20347] Updated weights for policy 0, policy_version 624257 (0.0007) [2023-12-27 09:57:07,699][20347] Updated weights for policy 0, policy_version 624267 (0.0007) [2023-12-27 09:57:07,753][20347] Updated weights for policy 0, policy_version 624277 (0.0007) [2023-12-27 09:57:07,814][20347] Updated weights for policy 0, policy_version 624287 (0.0007) [2023-12-27 09:57:08,152][20381] Updated weights for policy 1, policy_version 623971 (0.0010) [2023-12-27 09:57:08,211][20381] Updated weights for policy 1, policy_version 623981 (0.0010) [2023-12-27 09:57:08,256][20381] Updated weights for policy 1, policy_version 623991 (0.0010) [2023-12-27 09:57:08,451][20347] Updated weights for policy 0, policy_version 624297 (0.0008) [2023-12-27 09:57:08,498][20347] Updated weights for policy 0, policy_version 624307 (0.0007) [2023-12-27 09:57:08,543][20347] Updated weights for policy 0, policy_version 624317 (0.0007) [2023-12-27 09:57:08,913][20381] Updated weights for policy 1, policy_version 624001 (0.0008) [2023-12-27 09:57:08,974][20381] Updated weights for policy 1, policy_version 624011 (0.0005) [2023-12-27 09:57:09,027][20381] Updated weights for policy 1, policy_version 624021 (0.0005) [2023-12-27 09:57:09,090][20381] Updated weights for policy 1, policy_version 624031 (0.0005) [2023-12-27 09:57:09,268][20347] Updated weights for policy 0, policy_version 624327 (0.0007) [2023-12-27 09:57:09,326][20347] Updated weights for policy 0, policy_version 624337 (0.0008) [2023-12-27 09:57:09,388][20347] Updated weights for policy 0, policy_version 624347 (0.0007) [2023-12-27 09:57:09,715][20381] Updated weights for policy 1, policy_version 624041 (0.0010) [2023-12-27 09:57:09,773][20381] Updated weights for policy 1, policy_version 624051 (0.0011) [2023-12-27 09:57:09,827][20381] Updated weights for policy 1, policy_version 624061 (0.0010) [2023-12-27 09:57:10,069][20291] KL-divergence is very high: 369.1628 [2023-12-27 09:57:10,080][20347] Updated weights for policy 0, policy_version 624357 (0.0009) [2023-12-27 09:57:10,109][20291] KL-divergence is very high: 299.7413 [2023-12-27 09:57:10,129][20347] Updated weights for policy 0, policy_version 624367 (0.0009) [2023-12-27 09:57:10,184][20347] Updated weights for policy 0, policy_version 624377 (0.0006) [2023-12-27 09:57:10,562][20381] Updated weights for policy 1, policy_version 624071 (0.0008) [2023-12-27 09:57:10,625][20381] Updated weights for policy 1, policy_version 624081 (0.0007) [2023-12-27 09:57:10,687][20381] Updated weights for policy 1, policy_version 624091 (0.0007) [2023-12-27 09:57:10,824][20347] Updated weights for policy 0, policy_version 624387 (0.0009) [2023-12-27 09:57:10,877][20347] Updated weights for policy 0, policy_version 624397 (0.0009) [2023-12-27 09:57:10,932][20347] Updated weights for policy 0, policy_version 624407 (0.0007) [2023-12-27 09:57:11,365][20381] Updated weights for policy 1, policy_version 624101 (0.0007) [2023-12-27 09:57:11,429][20381] Updated weights for policy 1, policy_version 624111 (0.0008) [2023-12-27 09:57:11,488][20381] Updated weights for policy 1, policy_version 624121 (0.0008) [2023-12-27 09:57:11,687][20347] Updated weights for policy 0, policy_version 624417 (0.0006) [2023-12-27 09:57:11,753][20347] Updated weights for policy 0, policy_version 624427 (0.0007) [2023-12-27 09:57:11,815][20347] Updated weights for policy 0, policy_version 624437 (0.0008) [2023-12-27 09:57:11,871][20347] Updated weights for policy 0, policy_version 624447 (0.0007) [2023-12-27 09:57:12,207][20381] Updated weights for policy 1, policy_version 624131 (0.0010) [2023-12-27 09:57:12,261][20381] Updated weights for policy 1, policy_version 624141 (0.0011) [2023-12-27 09:57:12,324][20381] Updated weights for policy 1, policy_version 624151 (0.0008) [2023-12-27 09:57:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 319676416. Throughput: 0: 10498.1, 1: 10487.9. Samples: 319673292. Policy #0 lag: (min: 5.0, avg: 30.8, max: 32.0) [2023-12-27 09:57:12,363][19170] Avg episode reward: [(0, '883.760'), (1, '256.120')] [2023-12-27 09:57:12,515][20347] Updated weights for policy 0, policy_version 624457 (0.0007) [2023-12-27 09:57:12,574][20347] Updated weights for policy 0, policy_version 624467 (0.0008) [2023-12-27 09:57:12,633][20347] Updated weights for policy 0, policy_version 624477 (0.0008) [2023-12-27 09:57:13,045][20381] Updated weights for policy 1, policy_version 624161 (0.0008) [2023-12-27 09:57:13,100][20381] Updated weights for policy 1, policy_version 624171 (0.0007) [2023-12-27 09:57:13,152][20381] Updated weights for policy 1, policy_version 624181 (0.0009) [2023-12-27 09:57:13,204][20381] Updated weights for policy 1, policy_version 624191 (0.0008) [2023-12-27 09:57:13,268][20347] Updated weights for policy 0, policy_version 624487 (0.0006) [2023-12-27 09:57:13,322][20347] Updated weights for policy 0, policy_version 624497 (0.0010) [2023-12-27 09:57:13,375][20347] Updated weights for policy 0, policy_version 624507 (0.0010) [2023-12-27 09:57:13,956][20381] Updated weights for policy 1, policy_version 624201 (0.0007) [2023-12-27 09:57:13,968][20347] Updated weights for policy 0, policy_version 624517 (0.0010) [2023-12-27 09:57:14,009][20381] Updated weights for policy 1, policy_version 624211 (0.0009) [2023-12-27 09:57:14,027][20347] Updated weights for policy 0, policy_version 624527 (0.0010) [2023-12-27 09:57:14,055][20381] Updated weights for policy 1, policy_version 624221 (0.0006) [2023-12-27 09:57:14,079][20347] Updated weights for policy 0, policy_version 624537 (0.0008) [2023-12-27 09:57:14,695][20347] Updated weights for policy 0, policy_version 624547 (0.0007) [2023-12-27 09:57:14,743][20347] Updated weights for policy 0, policy_version 624557 (0.0010) [2023-12-27 09:57:14,801][20381] Updated weights for policy 1, policy_version 624231 (0.0009) [2023-12-27 09:57:14,801][20347] Updated weights for policy 0, policy_version 624567 (0.0012) [2023-12-27 09:57:14,857][20381] Updated weights for policy 1, policy_version 624241 (0.0010) [2023-12-27 09:57:14,914][20381] Updated weights for policy 1, policy_version 624251 (0.0011) [2023-12-27 09:57:15,501][20347] Updated weights for policy 0, policy_version 624577 (0.0010) [2023-12-27 09:57:15,558][20347] Updated weights for policy 0, policy_version 624587 (0.0006) [2023-12-27 09:57:15,568][20381] Updated weights for policy 1, policy_version 624261 (0.0008) [2023-12-27 09:57:15,614][20347] Updated weights for policy 0, policy_version 624597 (0.0005) [2023-12-27 09:57:15,635][20381] Updated weights for policy 1, policy_version 624271 (0.0008) [2023-12-27 09:57:15,663][20347] Updated weights for policy 0, policy_version 624607 (0.0006) [2023-12-27 09:57:15,684][20381] Updated weights for policy 1, policy_version 624281 (0.0006) [2023-12-27 09:57:16,242][20347] Updated weights for policy 0, policy_version 624617 (0.0006) [2023-12-27 09:57:16,298][20347] Updated weights for policy 0, policy_version 624627 (0.0006) [2023-12-27 09:57:16,356][20347] Updated weights for policy 0, policy_version 624637 (0.0006) [2023-12-27 09:57:16,415][20381] Updated weights for policy 1, policy_version 624291 (0.0008) [2023-12-27 09:57:16,470][20381] Updated weights for policy 1, policy_version 624301 (0.0007) [2023-12-27 09:57:16,529][20381] Updated weights for policy 1, policy_version 624311 (0.0008) [2023-12-27 09:57:17,015][20347] Updated weights for policy 0, policy_version 624647 (0.0009) [2023-12-27 09:57:17,070][20347] Updated weights for policy 0, policy_version 624657 (0.0010) [2023-12-27 09:57:17,118][20347] Updated weights for policy 0, policy_version 624667 (0.0010) [2023-12-27 09:57:17,241][20381] Updated weights for policy 1, policy_version 624321 (0.0008) [2023-12-27 09:57:17,289][20381] Updated weights for policy 1, policy_version 624331 (0.0010) [2023-12-27 09:57:17,338][20381] Updated weights for policy 1, policy_version 624341 (0.0010) [2023-12-27 09:57:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 319782912. Throughput: 0: 10508.0, 1: 10401.5. Samples: 319795568. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:17,363][19170] Avg episode reward: [(0, '874.350'), (1, '247.590')] [2023-12-27 09:57:17,387][20381] Updated weights for policy 1, policy_version 624351 (0.0011) [2023-12-27 09:57:17,752][20347] Updated weights for policy 0, policy_version 624677 (0.0008) [2023-12-27 09:57:17,799][20347] Updated weights for policy 0, policy_version 624687 (0.0006) [2023-12-27 09:57:17,857][20347] Updated weights for policy 0, policy_version 624697 (0.0006) [2023-12-27 09:57:18,082][20381] Updated weights for policy 1, policy_version 624361 (0.0011) [2023-12-27 09:57:18,127][20381] Updated weights for policy 1, policy_version 624371 (0.0010) [2023-12-27 09:57:18,169][20381] Updated weights for policy 1, policy_version 624381 (0.0010) [2023-12-27 09:57:18,499][20347] Updated weights for policy 0, policy_version 624707 (0.0007) [2023-12-27 09:57:18,557][20347] Updated weights for policy 0, policy_version 624717 (0.0010) [2023-12-27 09:57:18,611][20347] Updated weights for policy 0, policy_version 624727 (0.0010) [2023-12-27 09:57:18,872][20381] Updated weights for policy 1, policy_version 624391 (0.0009) [2023-12-27 09:57:18,916][20381] Updated weights for policy 1, policy_version 624401 (0.0006) [2023-12-27 09:57:18,962][20381] Updated weights for policy 1, policy_version 624411 (0.0005) [2023-12-27 09:57:19,286][20347] Updated weights for policy 0, policy_version 624737 (0.0010) [2023-12-27 09:57:19,347][20347] Updated weights for policy 0, policy_version 624747 (0.0011) [2023-12-27 09:57:19,404][20347] Updated weights for policy 0, policy_version 624757 (0.0007) [2023-12-27 09:57:19,451][20347] Updated weights for policy 0, policy_version 624767 (0.0009) [2023-12-27 09:57:19,632][20381] Updated weights for policy 1, policy_version 624421 (0.0008) [2023-12-27 09:57:19,683][20381] Updated weights for policy 1, policy_version 624431 (0.0010) [2023-12-27 09:57:19,743][20381] Updated weights for policy 1, policy_version 624441 (0.0011) [2023-12-27 09:57:20,201][20347] Updated weights for policy 0, policy_version 624777 (0.0008) [2023-12-27 09:57:20,250][20347] Updated weights for policy 0, policy_version 624787 (0.0006) [2023-12-27 09:57:20,299][20347] Updated weights for policy 0, policy_version 624797 (0.0005) [2023-12-27 09:57:20,388][20381] Updated weights for policy 1, policy_version 624451 (0.0009) [2023-12-27 09:57:20,454][20381] Updated weights for policy 1, policy_version 624461 (0.0006) [2023-12-27 09:57:20,515][20381] Updated weights for policy 1, policy_version 624471 (0.0005) [2023-12-27 09:57:21,058][20347] Updated weights for policy 0, policy_version 624807 (0.0008) [2023-12-27 09:57:21,075][20381] Updated weights for policy 1, policy_version 624481 (0.0009) [2023-12-27 09:57:21,115][20347] Updated weights for policy 0, policy_version 624817 (0.0008) [2023-12-27 09:57:21,137][20381] Updated weights for policy 1, policy_version 624491 (0.0010) [2023-12-27 09:57:21,180][20347] Updated weights for policy 0, policy_version 624827 (0.0007) [2023-12-27 09:57:21,203][20381] Updated weights for policy 1, policy_version 624501 (0.0007) [2023-12-27 09:57:21,267][20381] Updated weights for policy 1, policy_version 624511 (0.0006) [2023-12-27 09:57:21,932][20347] Updated weights for policy 0, policy_version 624837 (0.0009) [2023-12-27 09:57:21,968][20381] Updated weights for policy 1, policy_version 624521 (0.0010) [2023-12-27 09:57:21,999][20347] Updated weights for policy 0, policy_version 624847 (0.0011) [2023-12-27 09:57:22,028][20381] Updated weights for policy 1, policy_version 624531 (0.0011) [2023-12-27 09:57:22,035][20291] KL-divergence is very high: 146.2312 [2023-12-27 09:57:22,042][20291] KL-divergence is very high: 105.8150 [2023-12-27 09:57:22,064][20347] Updated weights for policy 0, policy_version 624857 (0.0010) [2023-12-27 09:57:22,085][20381] Updated weights for policy 1, policy_version 624541 (0.0011) [2023-12-27 09:57:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 319889408. Throughput: 0: 10522.0, 1: 10360.1. Samples: 319857948. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:22,363][19170] Avg episode reward: [(0, '874.280'), (1, '286.000')] [2023-12-27 09:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000624864_159981568.pth... [2023-12-27 09:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000624544_159907840.pth... [2023-12-27 09:57:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000623648_159670272.pth [2023-12-27 09:57:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000623328_159596544.pth [2023-12-27 09:57:22,721][20381] Updated weights for policy 1, policy_version 624551 (0.0010) [2023-12-27 09:57:22,757][20347] Updated weights for policy 0, policy_version 624867 (0.0011) [2023-12-27 09:57:22,780][20381] Updated weights for policy 1, policy_version 624561 (0.0010) [2023-12-27 09:57:22,806][20347] Updated weights for policy 0, policy_version 624877 (0.0010) [2023-12-27 09:57:22,834][20381] Updated weights for policy 1, policy_version 624571 (0.0010) [2023-12-27 09:57:22,858][20347] Updated weights for policy 0, policy_version 624887 (0.0010) [2023-12-27 09:57:23,493][20381] Updated weights for policy 1, policy_version 624581 (0.0008) [2023-12-27 09:57:23,519][20347] Updated weights for policy 0, policy_version 624897 (0.0010) [2023-12-27 09:57:23,545][20381] Updated weights for policy 1, policy_version 624591 (0.0010) [2023-12-27 09:57:23,580][20347] Updated weights for policy 0, policy_version 624907 (0.0008) [2023-12-27 09:57:23,597][20381] Updated weights for policy 1, policy_version 624601 (0.0010) [2023-12-27 09:57:23,634][20347] Updated weights for policy 0, policy_version 624917 (0.0007) [2023-12-27 09:57:23,681][20347] Updated weights for policy 0, policy_version 624927 (0.0008) [2023-12-27 09:57:24,265][20381] Updated weights for policy 1, policy_version 624611 (0.0009) [2023-12-27 09:57:24,323][20381] Updated weights for policy 1, policy_version 624621 (0.0006) [2023-12-27 09:57:24,376][20347] Updated weights for policy 0, policy_version 624937 (0.0010) [2023-12-27 09:57:24,384][20381] Updated weights for policy 1, policy_version 624631 (0.0006) [2023-12-27 09:57:24,424][20347] Updated weights for policy 0, policy_version 624947 (0.0008) [2023-12-27 09:57:24,490][20347] Updated weights for policy 0, policy_version 624957 (0.0005) [2023-12-27 09:57:24,974][20381] Updated weights for policy 1, policy_version 624641 (0.0005) [2023-12-27 09:57:25,034][20381] Updated weights for policy 1, policy_version 624651 (0.0007) [2023-12-27 09:57:25,055][20347] Updated weights for policy 0, policy_version 624967 (0.0005) [2023-12-27 09:57:25,089][20381] Updated weights for policy 1, policy_version 624661 (0.0007) [2023-12-27 09:57:25,114][20347] Updated weights for policy 0, policy_version 624977 (0.0005) [2023-12-27 09:57:25,142][20381] Updated weights for policy 1, policy_version 624671 (0.0005) [2023-12-27 09:57:25,174][20347] Updated weights for policy 0, policy_version 624987 (0.0005) [2023-12-27 09:57:25,704][20381] Updated weights for policy 1, policy_version 624681 (0.0007) [2023-12-27 09:57:25,747][20347] Updated weights for policy 0, policy_version 624997 (0.0008) [2023-12-27 09:57:25,760][20381] Updated weights for policy 1, policy_version 624691 (0.0006) [2023-12-27 09:57:25,804][20347] Updated weights for policy 0, policy_version 625007 (0.0010) [2023-12-27 09:57:25,805][20381] Updated weights for policy 1, policy_version 624701 (0.0007) [2023-12-27 09:57:25,852][20347] Updated weights for policy 0, policy_version 625017 (0.0010) [2023-12-27 09:57:26,433][20347] Updated weights for policy 0, policy_version 625027 (0.0010) [2023-12-27 09:57:26,489][20347] Updated weights for policy 0, policy_version 625037 (0.0010) [2023-12-27 09:57:26,490][20381] Updated weights for policy 1, policy_version 624711 (0.0006) [2023-12-27 09:57:26,539][20381] Updated weights for policy 1, policy_version 624721 (0.0005) [2023-12-27 09:57:26,544][20347] Updated weights for policy 0, policy_version 625047 (0.0010) [2023-12-27 09:57:26,586][20381] Updated weights for policy 1, policy_version 624731 (0.0005) [2023-12-27 09:57:27,131][20347] Updated weights for policy 0, policy_version 625057 (0.0010) [2023-12-27 09:57:27,195][20347] Updated weights for policy 0, policy_version 625067 (0.0006) [2023-12-27 09:57:27,250][20347] Updated weights for policy 0, policy_version 625077 (0.0011) [2023-12-27 09:57:27,310][20347] Updated weights for policy 0, policy_version 625087 (0.0009) [2023-12-27 09:57:27,320][20381] Updated weights for policy 1, policy_version 624741 (0.0007) [2023-12-27 09:57:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 319995904. Throughput: 0: 10451.8, 1: 10394.4. Samples: 319984612. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:27,363][19170] Avg episode reward: [(0, '903.100'), (1, '263.180')] [2023-12-27 09:57:27,371][20381] Updated weights for policy 1, policy_version 624751 (0.0005) [2023-12-27 09:57:27,422][20381] Updated weights for policy 1, policy_version 624761 (0.0005) [2023-12-27 09:57:27,888][20347] Updated weights for policy 0, policy_version 625097 (0.0005) [2023-12-27 09:57:27,946][20347] Updated weights for policy 0, policy_version 625107 (0.0006) [2023-12-27 09:57:28,000][20347] Updated weights for policy 0, policy_version 625117 (0.0005) [2023-12-27 09:57:28,119][20381] Updated weights for policy 1, policy_version 624771 (0.0008) [2023-12-27 09:57:28,188][20381] Updated weights for policy 1, policy_version 624781 (0.0009) [2023-12-27 09:57:28,244][20381] Updated weights for policy 1, policy_version 624791 (0.0009) [2023-12-27 09:57:28,565][20347] Updated weights for policy 0, policy_version 625127 (0.0007) [2023-12-27 09:57:28,622][20347] Updated weights for policy 0, policy_version 625137 (0.0008) [2023-12-27 09:57:28,681][20347] Updated weights for policy 0, policy_version 625147 (0.0008) [2023-12-27 09:57:28,948][20381] Updated weights for policy 1, policy_version 624801 (0.0008) [2023-12-27 09:57:28,999][20381] Updated weights for policy 1, policy_version 624811 (0.0007) [2023-12-27 09:57:29,057][20381] Updated weights for policy 1, policy_version 624821 (0.0007) [2023-12-27 09:57:29,116][20381] Updated weights for policy 1, policy_version 624831 (0.0007) [2023-12-27 09:57:29,346][20347] Updated weights for policy 0, policy_version 625157 (0.0009) [2023-12-27 09:57:29,407][20347] Updated weights for policy 0, policy_version 625167 (0.0010) [2023-12-27 09:57:29,461][20347] Updated weights for policy 0, policy_version 625177 (0.0010) [2023-12-27 09:57:29,840][20381] Updated weights for policy 1, policy_version 624841 (0.0008) [2023-12-27 09:57:29,906][20381] Updated weights for policy 1, policy_version 624851 (0.0008) [2023-12-27 09:57:29,958][20381] Updated weights for policy 1, policy_version 624861 (0.0008) [2023-12-27 09:57:30,162][20347] Updated weights for policy 0, policy_version 625187 (0.0010) [2023-12-27 09:57:30,214][20347] Updated weights for policy 0, policy_version 625197 (0.0010) [2023-12-27 09:57:30,276][20347] Updated weights for policy 0, policy_version 625207 (0.0010) [2023-12-27 09:57:30,657][20381] Updated weights for policy 1, policy_version 624871 (0.0008) [2023-12-27 09:57:30,703][20381] Updated weights for policy 1, policy_version 624881 (0.0007) [2023-12-27 09:57:30,751][20381] Updated weights for policy 1, policy_version 624891 (0.0009) [2023-12-27 09:57:30,915][20347] Updated weights for policy 0, policy_version 625217 (0.0010) [2023-12-27 09:57:30,966][20347] Updated weights for policy 0, policy_version 625227 (0.0007) [2023-12-27 09:57:31,018][20347] Updated weights for policy 0, policy_version 625237 (0.0008) [2023-12-27 09:57:31,075][20347] Updated weights for policy 0, policy_version 625247 (0.0006) [2023-12-27 09:57:31,413][20381] Updated weights for policy 1, policy_version 624901 (0.0008) [2023-12-27 09:57:31,468][20381] Updated weights for policy 1, policy_version 624911 (0.0005) [2023-12-27 09:57:31,517][20381] Updated weights for policy 1, policy_version 624921 (0.0005) [2023-12-27 09:57:31,798][20347] Updated weights for policy 0, policy_version 625257 (0.0009) [2023-12-27 09:57:31,847][20347] Updated weights for policy 0, policy_version 625267 (0.0005) [2023-12-27 09:57:31,901][20347] Updated weights for policy 0, policy_version 625277 (0.0005) [2023-12-27 09:57:32,159][20381] Updated weights for policy 1, policy_version 624931 (0.0006) [2023-12-27 09:57:32,218][20381] Updated weights for policy 1, policy_version 624941 (0.0008) [2023-12-27 09:57:32,274][20381] Updated weights for policy 1, policy_version 624951 (0.0006) [2023-12-27 09:57:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20882.7). Total num frames: 320102400. Throughput: 0: 10441.1, 1: 10371.5. Samples: 320109968. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:32,363][19170] Avg episode reward: [(0, '942.160'), (1, '247.560')] [2023-12-27 09:57:32,576][20347] Updated weights for policy 0, policy_version 625287 (0.0006) [2023-12-27 09:57:32,632][20347] Updated weights for policy 0, policy_version 625297 (0.0008) [2023-12-27 09:57:32,686][20347] Updated weights for policy 0, policy_version 625307 (0.0008) [2023-12-27 09:57:32,968][20381] Updated weights for policy 1, policy_version 624961 (0.0007) [2023-12-27 09:57:33,016][20381] Updated weights for policy 1, policy_version 624971 (0.0008) [2023-12-27 09:57:33,068][20381] Updated weights for policy 1, policy_version 624981 (0.0009) [2023-12-27 09:57:33,113][20381] Updated weights for policy 1, policy_version 624991 (0.0009) [2023-12-27 09:57:33,355][20347] Updated weights for policy 0, policy_version 625317 (0.0008) [2023-12-27 09:57:33,407][20347] Updated weights for policy 0, policy_version 625327 (0.0010) [2023-12-27 09:57:33,461][20347] Updated weights for policy 0, policy_version 625337 (0.0010) [2023-12-27 09:57:33,715][20381] Updated weights for policy 1, policy_version 625001 (0.0006) [2023-12-27 09:57:33,762][20381] Updated weights for policy 1, policy_version 625011 (0.0005) [2023-12-27 09:57:33,814][20381] Updated weights for policy 1, policy_version 625021 (0.0006) [2023-12-27 09:57:34,101][20347] Updated weights for policy 0, policy_version 625347 (0.0009) [2023-12-27 09:57:34,163][20347] Updated weights for policy 0, policy_version 625357 (0.0007) [2023-12-27 09:57:34,227][20347] Updated weights for policy 0, policy_version 625367 (0.0008) [2023-12-27 09:57:34,387][20381] Updated weights for policy 1, policy_version 625031 (0.0006) [2023-12-27 09:57:34,445][20381] Updated weights for policy 1, policy_version 625041 (0.0006) [2023-12-27 09:57:34,498][20381] Updated weights for policy 1, policy_version 625051 (0.0008) [2023-12-27 09:57:34,960][20347] Updated weights for policy 0, policy_version 625377 (0.0007) [2023-12-27 09:57:35,020][20347] Updated weights for policy 0, policy_version 625387 (0.0007) [2023-12-27 09:57:35,072][20347] Updated weights for policy 0, policy_version 625397 (0.0008) [2023-12-27 09:57:35,084][20381] Updated weights for policy 1, policy_version 625061 (0.0008) [2023-12-27 09:57:35,123][20347] Updated weights for policy 0, policy_version 625407 (0.0006) [2023-12-27 09:57:35,133][20381] Updated weights for policy 1, policy_version 625071 (0.0005) [2023-12-27 09:57:35,183][20381] Updated weights for policy 1, policy_version 625081 (0.0007) [2023-12-27 09:57:35,833][20381] Updated weights for policy 1, policy_version 625091 (0.0008) [2023-12-27 09:57:35,838][20347] Updated weights for policy 0, policy_version 625417 (0.0007) [2023-12-27 09:57:35,887][20381] Updated weights for policy 1, policy_version 625101 (0.0007) [2023-12-27 09:57:35,903][20347] Updated weights for policy 0, policy_version 625427 (0.0008) [2023-12-27 09:57:35,944][20381] Updated weights for policy 1, policy_version 625111 (0.0005) [2023-12-27 09:57:35,962][20347] Updated weights for policy 0, policy_version 625437 (0.0008) [2023-12-27 09:57:36,598][20347] Updated weights for policy 0, policy_version 625447 (0.0007) [2023-12-27 09:57:36,655][20347] Updated weights for policy 0, policy_version 625457 (0.0008) [2023-12-27 09:57:36,667][20381] Updated weights for policy 1, policy_version 625121 (0.0006) [2023-12-27 09:57:36,713][20347] Updated weights for policy 0, policy_version 625467 (0.0006) [2023-12-27 09:57:36,734][20381] Updated weights for policy 1, policy_version 625131 (0.0007) [2023-12-27 09:57:36,792][20381] Updated weights for policy 1, policy_version 625141 (0.0008) [2023-12-27 09:57:36,854][20381] Updated weights for policy 1, policy_version 625151 (0.0006) [2023-12-27 09:57:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20889.6, 300 sec: 20882.6). Total num frames: 320200704. Throughput: 0: 10421.0, 1: 10419.6. Samples: 320174268. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:37,364][19170] Avg episode reward: [(0, '942.320'), (1, '260.230')] [2023-12-27 09:57:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000625152_160063488.pth... [2023-12-27 09:57:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000625472_160137216.pth... [2023-12-27 09:57:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000623968_159760384.pth [2023-12-27 09:57:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000624256_159825920.pth [2023-12-27 09:57:37,442][20347] Updated weights for policy 0, policy_version 625477 (0.0006) [2023-12-27 09:57:37,463][20381] Updated weights for policy 1, policy_version 625161 (0.0005) [2023-12-27 09:57:37,506][20347] Updated weights for policy 0, policy_version 625487 (0.0007) [2023-12-27 09:57:37,519][20381] Updated weights for policy 1, policy_version 625171 (0.0005) [2023-12-27 09:57:37,573][20347] Updated weights for policy 0, policy_version 625497 (0.0008) [2023-12-27 09:57:37,575][20381] Updated weights for policy 1, policy_version 625181 (0.0006) [2023-12-27 09:57:38,202][20381] Updated weights for policy 1, policy_version 625191 (0.0007) [2023-12-27 09:57:38,255][20381] Updated weights for policy 1, policy_version 625201 (0.0010) [2023-12-27 09:57:38,290][20347] Updated weights for policy 0, policy_version 625507 (0.0007) [2023-12-27 09:57:38,320][20381] Updated weights for policy 1, policy_version 625211 (0.0007) [2023-12-27 09:57:38,355][20347] Updated weights for policy 0, policy_version 625517 (0.0008) [2023-12-27 09:57:38,414][20347] Updated weights for policy 0, policy_version 625527 (0.0007) [2023-12-27 09:57:39,020][20381] Updated weights for policy 1, policy_version 625221 (0.0007) [2023-12-27 09:57:39,073][20381] Updated weights for policy 1, policy_version 625231 (0.0007) [2023-12-27 09:57:39,112][20347] Updated weights for policy 0, policy_version 625537 (0.0008) [2023-12-27 09:57:39,122][20381] Updated weights for policy 1, policy_version 625241 (0.0006) [2023-12-27 09:57:39,165][20347] Updated weights for policy 0, policy_version 625547 (0.0006) [2023-12-27 09:57:39,222][20347] Updated weights for policy 0, policy_version 625557 (0.0007) [2023-12-27 09:57:39,282][20347] Updated weights for policy 0, policy_version 625567 (0.0007) [2023-12-27 09:57:39,883][20381] Updated weights for policy 1, policy_version 625251 (0.0005) [2023-12-27 09:57:39,946][20381] Updated weights for policy 1, policy_version 625261 (0.0010) [2023-12-27 09:57:39,991][20347] Updated weights for policy 0, policy_version 625577 (0.0008) [2023-12-27 09:57:39,999][20291] KL-divergence is very high: 204.4547 [2023-12-27 09:57:40,013][20381] Updated weights for policy 1, policy_version 625271 (0.0011) [2023-12-27 09:57:40,028][20291] KL-divergence is very high: 187.7155 [2023-12-27 09:57:40,045][20347] Updated weights for policy 0, policy_version 625587 (0.0006) [2023-12-27 09:57:40,046][20291] KL-divergence is very high: 184.8254 [2023-12-27 09:57:40,106][20347] Updated weights for policy 0, policy_version 625597 (0.0006) [2023-12-27 09:57:40,702][20381] Updated weights for policy 1, policy_version 625281 (0.0011) [2023-12-27 09:57:40,758][20381] Updated weights for policy 1, policy_version 625291 (0.0011) [2023-12-27 09:57:40,804][20381] Updated weights for policy 1, policy_version 625301 (0.0010) [2023-12-27 09:57:40,812][20347] Updated weights for policy 0, policy_version 625607 (0.0007) [2023-12-27 09:57:40,852][20381] Updated weights for policy 1, policy_version 625311 (0.0007) [2023-12-27 09:57:40,875][20347] Updated weights for policy 0, policy_version 625617 (0.0008) [2023-12-27 09:57:40,935][20347] Updated weights for policy 0, policy_version 625627 (0.0009) [2023-12-27 09:57:41,591][20381] Updated weights for policy 1, policy_version 625321 (0.0008) [2023-12-27 09:57:41,648][20347] Updated weights for policy 0, policy_version 625637 (0.0008) [2023-12-27 09:57:41,656][20381] Updated weights for policy 1, policy_version 625331 (0.0007) [2023-12-27 09:57:41,708][20347] Updated weights for policy 0, policy_version 625647 (0.0007) [2023-12-27 09:57:41,722][20381] Updated weights for policy 1, policy_version 625341 (0.0007) [2023-12-27 09:57:41,775][20347] Updated weights for policy 0, policy_version 625657 (0.0007) [2023-12-27 09:57:42,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 320299008. Throughput: 0: 10370.0, 1: 10326.9. Samples: 320294172. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:42,363][19170] Avg episode reward: [(0, '961.610'), (1, '305.720')] [2023-12-27 09:57:42,421][20347] Updated weights for policy 0, policy_version 625667 (0.0007) [2023-12-27 09:57:42,456][20381] Updated weights for policy 1, policy_version 625351 (0.0008) [2023-12-27 09:57:42,482][20347] Updated weights for policy 0, policy_version 625677 (0.0006) [2023-12-27 09:57:42,507][20381] Updated weights for policy 1, policy_version 625361 (0.0007) [2023-12-27 09:57:42,545][20347] Updated weights for policy 0, policy_version 625687 (0.0005) [2023-12-27 09:57:42,569][20381] Updated weights for policy 1, policy_version 625371 (0.0008) [2023-12-27 09:57:43,126][20347] Updated weights for policy 0, policy_version 625697 (0.0006) [2023-12-27 09:57:43,185][20347] Updated weights for policy 0, policy_version 625707 (0.0007) [2023-12-27 09:57:43,241][20347] Updated weights for policy 0, policy_version 625717 (0.0008) [2023-12-27 09:57:43,308][20347] Updated weights for policy 0, policy_version 625727 (0.0006) [2023-12-27 09:57:43,314][20381] Updated weights for policy 1, policy_version 625381 (0.0007) [2023-12-27 09:57:43,371][20381] Updated weights for policy 1, policy_version 625391 (0.0008) [2023-12-27 09:57:43,433][20381] Updated weights for policy 1, policy_version 625401 (0.0008) [2023-12-27 09:57:43,966][20347] Updated weights for policy 0, policy_version 625737 (0.0005) [2023-12-27 09:57:44,018][20347] Updated weights for policy 0, policy_version 625747 (0.0007) [2023-12-27 09:57:44,067][20347] Updated weights for policy 0, policy_version 625757 (0.0007) [2023-12-27 09:57:44,106][20381] Updated weights for policy 1, policy_version 625411 (0.0008) [2023-12-27 09:57:44,157][20381] Updated weights for policy 1, policy_version 625421 (0.0008) [2023-12-27 09:57:44,203][20381] Updated weights for policy 1, policy_version 625431 (0.0008) [2023-12-27 09:57:44,753][20347] Updated weights for policy 0, policy_version 625767 (0.0007) [2023-12-27 09:57:44,815][20347] Updated weights for policy 0, policy_version 625777 (0.0007) [2023-12-27 09:57:44,866][20347] Updated weights for policy 0, policy_version 625787 (0.0008) [2023-12-27 09:57:44,904][20381] Updated weights for policy 1, policy_version 625441 (0.0008) [2023-12-27 09:57:44,960][20381] Updated weights for policy 1, policy_version 625451 (0.0006) [2023-12-27 09:57:45,022][20381] Updated weights for policy 1, policy_version 625461 (0.0007) [2023-12-27 09:57:45,088][20381] Updated weights for policy 1, policy_version 625471 (0.0006) [2023-12-27 09:57:45,609][20347] Updated weights for policy 0, policy_version 625797 (0.0009) [2023-12-27 09:57:45,644][20381] Updated weights for policy 1, policy_version 625481 (0.0006) [2023-12-27 09:57:45,657][20347] Updated weights for policy 0, policy_version 625807 (0.0010) [2023-12-27 09:57:45,700][20381] Updated weights for policy 1, policy_version 625491 (0.0005) [2023-12-27 09:57:45,703][20347] Updated weights for policy 0, policy_version 625817 (0.0010) [2023-12-27 09:57:45,758][20381] Updated weights for policy 1, policy_version 625501 (0.0005) [2023-12-27 09:57:46,321][20381] Updated weights for policy 1, policy_version 625511 (0.0007) [2023-12-27 09:57:46,380][20381] Updated weights for policy 1, policy_version 625521 (0.0007) [2023-12-27 09:57:46,396][20347] Updated weights for policy 0, policy_version 625827 (0.0010) [2023-12-27 09:57:46,437][20381] Updated weights for policy 1, policy_version 625531 (0.0005) [2023-12-27 09:57:46,445][20347] Updated weights for policy 0, policy_version 625837 (0.0010) [2023-12-27 09:57:46,498][20347] Updated weights for policy 0, policy_version 625847 (0.0010) [2023-12-27 09:57:47,052][20381] Updated weights for policy 1, policy_version 625541 (0.0006) [2023-12-27 09:57:47,110][20381] Updated weights for policy 1, policy_version 625551 (0.0007) [2023-12-27 09:57:47,167][20381] Updated weights for policy 1, policy_version 625561 (0.0006) [2023-12-27 09:57:47,167][20347] Updated weights for policy 0, policy_version 625857 (0.0010) [2023-12-27 09:57:47,218][20347] Updated weights for policy 0, policy_version 625867 (0.0005) [2023-12-27 09:57:47,266][20347] Updated weights for policy 0, policy_version 625877 (0.0010) [2023-12-27 09:57:47,318][20347] Updated weights for policy 0, policy_version 625887 (0.0010) [2023-12-27 09:57:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20889.6, 300 sec: 20882.7). Total num frames: 320413696. Throughput: 0: 10370.3, 1: 10320.5. Samples: 320417780. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:47,363][19170] Avg episode reward: [(0, '990.490'), (1, '308.560')] [2023-12-27 09:57:47,803][20381] Updated weights for policy 1, policy_version 625571 (0.0007) [2023-12-27 09:57:47,856][20381] Updated weights for policy 1, policy_version 625581 (0.0009) [2023-12-27 09:57:47,918][20381] Updated weights for policy 1, policy_version 625591 (0.0008) [2023-12-27 09:57:47,944][20347] Updated weights for policy 0, policy_version 625897 (0.0010) [2023-12-27 09:57:47,988][20347] Updated weights for policy 0, policy_version 625907 (0.0010) [2023-12-27 09:57:48,041][20347] Updated weights for policy 0, policy_version 625917 (0.0010) [2023-12-27 09:57:48,524][20381] Updated weights for policy 1, policy_version 625601 (0.0006) [2023-12-27 09:57:48,583][20381] Updated weights for policy 1, policy_version 625611 (0.0005) [2023-12-27 09:57:48,644][20381] Updated weights for policy 1, policy_version 625621 (0.0005) [2023-12-27 09:57:48,708][20381] Updated weights for policy 1, policy_version 625631 (0.0005) [2023-12-27 09:57:48,741][20347] Updated weights for policy 0, policy_version 625927 (0.0010) [2023-12-27 09:57:48,790][20347] Updated weights for policy 0, policy_version 625937 (0.0010) [2023-12-27 09:57:48,791][20291] KL-divergence is very high: 108.1769 [2023-12-27 09:57:48,854][20347] Updated weights for policy 0, policy_version 625947 (0.0010) [2023-12-27 09:57:49,274][20381] Updated weights for policy 1, policy_version 625641 (0.0010) [2023-12-27 09:57:49,333][20381] Updated weights for policy 1, policy_version 625651 (0.0011) [2023-12-27 09:57:49,392][20381] Updated weights for policy 1, policy_version 625661 (0.0011) [2023-12-27 09:57:49,551][20347] Updated weights for policy 0, policy_version 625957 (0.0010) [2023-12-27 09:57:49,605][20347] Updated weights for policy 0, policy_version 625967 (0.0010) [2023-12-27 09:57:49,663][20347] Updated weights for policy 0, policy_version 625977 (0.0010) [2023-12-27 09:57:50,080][20381] Updated weights for policy 1, policy_version 625671 (0.0011) [2023-12-27 09:57:50,134][20381] Updated weights for policy 1, policy_version 625681 (0.0011) [2023-12-27 09:57:50,190][20381] Updated weights for policy 1, policy_version 625691 (0.0011) [2023-12-27 09:57:50,357][20347] Updated weights for policy 0, policy_version 625987 (0.0010) [2023-12-27 09:57:50,410][20347] Updated weights for policy 0, policy_version 625997 (0.0010) [2023-12-27 09:57:50,469][20347] Updated weights for policy 0, policy_version 626007 (0.0010) [2023-12-27 09:57:50,839][20381] Updated weights for policy 1, policy_version 625701 (0.0007) [2023-12-27 09:57:50,899][20381] Updated weights for policy 1, policy_version 625711 (0.0005) [2023-12-27 09:57:50,962][20381] Updated weights for policy 1, policy_version 625721 (0.0007) [2023-12-27 09:57:51,146][20347] Updated weights for policy 0, policy_version 626017 (0.0010) [2023-12-27 09:57:51,203][20347] Updated weights for policy 0, policy_version 626027 (0.0008) [2023-12-27 09:57:51,266][20347] Updated weights for policy 0, policy_version 626037 (0.0006) [2023-12-27 09:57:51,327][20347] Updated weights for policy 0, policy_version 626047 (0.0005) [2023-12-27 09:57:51,594][20381] Updated weights for policy 1, policy_version 625731 (0.0007) [2023-12-27 09:57:51,654][20381] Updated weights for policy 1, policy_version 625741 (0.0007) [2023-12-27 09:57:51,701][20381] Updated weights for policy 1, policy_version 625751 (0.0005) [2023-12-27 09:57:52,035][20347] Updated weights for policy 0, policy_version 626057 (0.0008) [2023-12-27 09:57:52,093][20347] Updated weights for policy 0, policy_version 626067 (0.0008) [2023-12-27 09:57:52,153][20347] Updated weights for policy 0, policy_version 626077 (0.0008) [2023-12-27 09:57:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 320512000. Throughput: 0: 10370.3, 1: 10341.2. Samples: 320480724. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:52,363][19170] Avg episode reward: [(0, '981.580'), (1, '318.690')] [2023-12-27 09:57:52,369][20381] Updated weights for policy 1, policy_version 625761 (0.0007) [2023-12-27 09:57:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000626080_160292864.pth... [2023-12-27 09:57:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000624864_159981568.pth [2023-12-27 09:57:52,433][20381] Updated weights for policy 1, policy_version 625771 (0.0007) [2023-12-27 09:57:52,489][20381] Updated weights for policy 1, policy_version 625781 (0.0008) [2023-12-27 09:57:52,548][20381] Updated weights for policy 1, policy_version 625791 (0.0008) [2023-12-27 09:57:52,552][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000625792_160227328.pth... [2023-12-27 09:57:52,556][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000624544_159907840.pth [2023-12-27 09:57:52,801][20347] Updated weights for policy 0, policy_version 626087 (0.0008) [2023-12-27 09:57:52,863][20347] Updated weights for policy 0, policy_version 626097 (0.0008) [2023-12-27 09:57:52,918][20347] Updated weights for policy 0, policy_version 626107 (0.0007) [2023-12-27 09:57:53,252][20381] Updated weights for policy 1, policy_version 625801 (0.0007) [2023-12-27 09:57:53,321][20381] Updated weights for policy 1, policy_version 625811 (0.0008) [2023-12-27 09:57:53,376][20381] Updated weights for policy 1, policy_version 625821 (0.0008) [2023-12-27 09:57:53,545][20347] Updated weights for policy 0, policy_version 626117 (0.0007) [2023-12-27 09:57:53,602][20347] Updated weights for policy 0, policy_version 626127 (0.0008) [2023-12-27 09:57:53,655][20347] Updated weights for policy 0, policy_version 626137 (0.0007) [2023-12-27 09:57:54,039][20381] Updated weights for policy 1, policy_version 625831 (0.0008) [2023-12-27 09:57:54,098][20381] Updated weights for policy 1, policy_version 625841 (0.0008) [2023-12-27 09:57:54,145][20381] Updated weights for policy 1, policy_version 625851 (0.0007) [2023-12-27 09:57:54,272][20347] Updated weights for policy 0, policy_version 626147 (0.0008) [2023-12-27 09:57:54,328][20347] Updated weights for policy 0, policy_version 626157 (0.0009) [2023-12-27 09:57:54,372][20347] Updated weights for policy 0, policy_version 626167 (0.0007) [2023-12-27 09:57:54,778][20381] Updated weights for policy 1, policy_version 625861 (0.0007) [2023-12-27 09:57:54,834][20381] Updated weights for policy 1, policy_version 625871 (0.0007) [2023-12-27 09:57:54,893][20381] Updated weights for policy 1, policy_version 625881 (0.0006) [2023-12-27 09:57:55,037][20347] Updated weights for policy 0, policy_version 626177 (0.0009) [2023-12-27 09:57:55,090][20347] Updated weights for policy 0, policy_version 626187 (0.0007) [2023-12-27 09:57:55,144][20347] Updated weights for policy 0, policy_version 626197 (0.0007) [2023-12-27 09:57:55,190][20347] Updated weights for policy 0, policy_version 626207 (0.0008) [2023-12-27 09:57:55,469][20381] Updated weights for policy 1, policy_version 625891 (0.0006) [2023-12-27 09:57:55,522][20381] Updated weights for policy 1, policy_version 625901 (0.0008) [2023-12-27 09:57:55,582][20381] Updated weights for policy 1, policy_version 625911 (0.0007) [2023-12-27 09:57:55,925][20347] Updated weights for policy 0, policy_version 626217 (0.0010) [2023-12-27 09:57:55,978][20347] Updated weights for policy 0, policy_version 626227 (0.0006) [2023-12-27 09:57:56,037][20347] Updated weights for policy 0, policy_version 626237 (0.0006) [2023-12-27 09:57:56,228][20381] Updated weights for policy 1, policy_version 625921 (0.0006) [2023-12-27 09:57:56,290][20381] Updated weights for policy 1, policy_version 625931 (0.0007) [2023-12-27 09:57:56,336][20381] Updated weights for policy 1, policy_version 625941 (0.0005) [2023-12-27 09:57:56,393][20381] Updated weights for policy 1, policy_version 625951 (0.0005) [2023-12-27 09:57:56,588][20347] Updated weights for policy 0, policy_version 626247 (0.0006) [2023-12-27 09:57:56,642][20347] Updated weights for policy 0, policy_version 626257 (0.0005) [2023-12-27 09:57:56,696][20347] Updated weights for policy 0, policy_version 626267 (0.0006) [2023-12-27 09:57:57,029][20381] Updated weights for policy 1, policy_version 625961 (0.0007) [2023-12-27 09:57:57,090][20381] Updated weights for policy 1, policy_version 625971 (0.0007) [2023-12-27 09:57:57,133][20381] Updated weights for policy 1, policy_version 625981 (0.0007) [2023-12-27 09:57:57,233][20347] Updated weights for policy 0, policy_version 626277 (0.0005) [2023-12-27 09:57:57,288][20347] Updated weights for policy 0, policy_version 626287 (0.0005) [2023-12-27 09:57:57,345][20347] Updated weights for policy 0, policy_version 626297 (0.0009) [2023-12-27 09:57:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 320618496. Throughput: 0: 10351.2, 1: 10390.7. Samples: 320606680. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:57:57,363][19170] Avg episode reward: [(0, '972.130'), (1, '289.160')] [2023-12-27 09:57:57,822][20381] Updated weights for policy 1, policy_version 625991 (0.0009) [2023-12-27 09:57:57,872][20381] Updated weights for policy 1, policy_version 626001 (0.0007) [2023-12-27 09:57:57,909][20347] Updated weights for policy 0, policy_version 626307 (0.0010) [2023-12-27 09:57:57,922][20381] Updated weights for policy 1, policy_version 626011 (0.0007) [2023-12-27 09:57:57,967][20347] Updated weights for policy 0, policy_version 626317 (0.0010) [2023-12-27 09:57:58,026][20347] Updated weights for policy 0, policy_version 626327 (0.0009) [2023-12-27 09:57:58,656][20381] Updated weights for policy 1, policy_version 626021 (0.0008) [2023-12-27 09:57:58,706][20347] Updated weights for policy 0, policy_version 626337 (0.0006) [2023-12-27 09:57:58,719][20381] Updated weights for policy 1, policy_version 626031 (0.0009) [2023-12-27 09:57:58,766][20347] Updated weights for policy 0, policy_version 626347 (0.0008) [2023-12-27 09:57:58,776][20381] Updated weights for policy 1, policy_version 626041 (0.0010) [2023-12-27 09:57:58,827][20347] Updated weights for policy 0, policy_version 626357 (0.0008) [2023-12-27 09:57:58,901][20347] Updated weights for policy 0, policy_version 626367 (0.0009) [2023-12-27 09:57:59,568][20381] Updated weights for policy 1, policy_version 626051 (0.0008) [2023-12-27 09:57:59,623][20381] Updated weights for policy 1, policy_version 626061 (0.0010) [2023-12-27 09:57:59,666][20347] Updated weights for policy 0, policy_version 626377 (0.0006) [2023-12-27 09:57:59,709][20381] Updated weights for policy 1, policy_version 626071 (0.0011) [2023-12-27 09:57:59,722][20347] Updated weights for policy 0, policy_version 626387 (0.0006) [2023-12-27 09:57:59,782][20347] Updated weights for policy 0, policy_version 626397 (0.0005) [2023-12-27 09:58:00,361][20347] Updated weights for policy 0, policy_version 626407 (0.0009) [2023-12-27 09:58:00,392][20381] Updated weights for policy 1, policy_version 626081 (0.0010) [2023-12-27 09:58:00,416][20347] Updated weights for policy 0, policy_version 626417 (0.0010) [2023-12-27 09:58:00,439][20381] Updated weights for policy 1, policy_version 626091 (0.0005) [2023-12-27 09:58:00,467][20347] Updated weights for policy 0, policy_version 626427 (0.0010) [2023-12-27 09:58:00,490][20381] Updated weights for policy 1, policy_version 626101 (0.0007) [2023-12-27 09:58:00,541][20381] Updated weights for policy 1, policy_version 626111 (0.0010) [2023-12-27 09:58:01,143][20347] Updated weights for policy 0, policy_version 626437 (0.0012) [2023-12-27 09:58:01,210][20347] Updated weights for policy 0, policy_version 626447 (0.0009) [2023-12-27 09:58:01,244][20381] Updated weights for policy 1, policy_version 626121 (0.0007) [2023-12-27 09:58:01,272][20347] Updated weights for policy 0, policy_version 626457 (0.0010) [2023-12-27 09:58:01,308][20381] Updated weights for policy 1, policy_version 626131 (0.0006) [2023-12-27 09:58:01,369][20381] Updated weights for policy 1, policy_version 626141 (0.0007) [2023-12-27 09:58:01,969][20347] Updated weights for policy 0, policy_version 626467 (0.0010) [2023-12-27 09:58:02,018][20347] Updated weights for policy 0, policy_version 626477 (0.0010) [2023-12-27 09:58:02,067][20347] Updated weights for policy 0, policy_version 626487 (0.0010) [2023-12-27 09:58:02,098][20381] Updated weights for policy 1, policy_version 626151 (0.0006) [2023-12-27 09:58:02,148][20381] Updated weights for policy 1, policy_version 626161 (0.0006) [2023-12-27 09:58:02,195][20381] Updated weights for policy 1, policy_version 626171 (0.0007) [2023-12-27 09:58:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 320724992. Throughput: 0: 10358.5, 1: 10419.4. Samples: 320730572. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:02,363][19170] Avg episode reward: [(0, '961.980'), (1, '257.410')] [2023-12-27 09:58:02,704][20347] Updated weights for policy 0, policy_version 626497 (0.0010) [2023-12-27 09:58:02,750][20347] Updated weights for policy 0, policy_version 626507 (0.0005) [2023-12-27 09:58:02,802][20347] Updated weights for policy 0, policy_version 626517 (0.0009) [2023-12-27 09:58:02,856][20347] Updated weights for policy 0, policy_version 626527 (0.0010) [2023-12-27 09:58:02,883][20381] Updated weights for policy 1, policy_version 626181 (0.0007) [2023-12-27 09:58:02,939][20381] Updated weights for policy 1, policy_version 626191 (0.0005) [2023-12-27 09:58:02,993][20381] Updated weights for policy 1, policy_version 626201 (0.0005) [2023-12-27 09:58:03,409][20347] Updated weights for policy 0, policy_version 626537 (0.0006) [2023-12-27 09:58:03,454][20347] Updated weights for policy 0, policy_version 626547 (0.0005) [2023-12-27 09:58:03,517][20347] Updated weights for policy 0, policy_version 626557 (0.0005) [2023-12-27 09:58:03,569][20381] Updated weights for policy 1, policy_version 626211 (0.0007) [2023-12-27 09:58:03,626][20381] Updated weights for policy 1, policy_version 626221 (0.0008) [2023-12-27 09:58:03,683][20381] Updated weights for policy 1, policy_version 626231 (0.0009) [2023-12-27 09:58:04,075][20347] Updated weights for policy 0, policy_version 626567 (0.0009) [2023-12-27 09:58:04,139][20347] Updated weights for policy 0, policy_version 626577 (0.0010) [2023-12-27 09:58:04,199][20347] Updated weights for policy 0, policy_version 626587 (0.0011) [2023-12-27 09:58:04,353][20381] Updated weights for policy 1, policy_version 626241 (0.0006) [2023-12-27 09:58:04,410][20381] Updated weights for policy 1, policy_version 626251 (0.0006) [2023-12-27 09:58:04,464][20381] Updated weights for policy 1, policy_version 626261 (0.0005) [2023-12-27 09:58:04,518][20381] Updated weights for policy 1, policy_version 626271 (0.0007) [2023-12-27 09:58:04,887][20347] Updated weights for policy 0, policy_version 626597 (0.0010) [2023-12-27 09:58:04,946][20347] Updated weights for policy 0, policy_version 626607 (0.0010) [2023-12-27 09:58:05,007][20347] Updated weights for policy 0, policy_version 626617 (0.0010) [2023-12-27 09:58:05,126][20381] Updated weights for policy 1, policy_version 626281 (0.0006) [2023-12-27 09:58:05,173][20292] KL-divergence is very high: 107.1252 [2023-12-27 09:58:05,181][20381] Updated weights for policy 1, policy_version 626291 (0.0007) [2023-12-27 09:58:05,182][20292] KL-divergence is very high: 107.9682 [2023-12-27 09:58:05,190][20292] KL-divergence is very high: 131.1951 [2023-12-27 09:58:05,201][20292] KL-divergence is very high: 124.1062 [2023-12-27 09:58:05,211][20292] KL-divergence is very high: 115.0893 [2023-12-27 09:58:05,216][20292] KL-divergence is very high: 120.5529 [2023-12-27 09:58:05,221][20292] KL-divergence is very high: 126.9117 [2023-12-27 09:58:05,226][20292] KL-divergence is very high: 109.7906 [2023-12-27 09:58:05,231][20292] KL-divergence is very high: 122.6279 [2023-12-27 09:58:05,237][20381] Updated weights for policy 1, policy_version 626301 (0.0008) [2023-12-27 09:58:05,240][20292] KL-divergence is very high: 120.0549 [2023-12-27 09:58:05,649][20347] Updated weights for policy 0, policy_version 626627 (0.0010) [2023-12-27 09:58:05,704][20347] Updated weights for policy 0, policy_version 626637 (0.0010) [2023-12-27 09:58:05,758][20347] Updated weights for policy 0, policy_version 626647 (0.0010) [2023-12-27 09:58:05,871][20292] KL-divergence is very high: 124.9285 [2023-12-27 09:58:05,908][20381] Updated weights for policy 1, policy_version 626311 (0.0007) [2023-12-27 09:58:05,956][20381] Updated weights for policy 1, policy_version 626321 (0.0007) [2023-12-27 09:58:06,000][20381] Updated weights for policy 1, policy_version 626331 (0.0007) [2023-12-27 09:58:06,371][20347] Updated weights for policy 0, policy_version 626657 (0.0010) [2023-12-27 09:58:06,427][20347] Updated weights for policy 0, policy_version 626667 (0.0010) [2023-12-27 09:58:06,487][20347] Updated weights for policy 0, policy_version 626677 (0.0009) [2023-12-27 09:58:06,546][20347] Updated weights for policy 0, policy_version 626687 (0.0009) [2023-12-27 09:58:06,762][20381] Updated weights for policy 1, policy_version 626341 (0.0007) [2023-12-27 09:58:06,828][20381] Updated weights for policy 1, policy_version 626351 (0.0006) [2023-12-27 09:58:06,894][20381] Updated weights for policy 1, policy_version 626361 (0.0008) [2023-12-27 09:58:07,211][20347] Updated weights for policy 0, policy_version 626697 (0.0010) [2023-12-27 09:58:07,259][20347] Updated weights for policy 0, policy_version 626707 (0.0010) [2023-12-27 09:58:07,309][20347] Updated weights for policy 0, policy_version 626717 (0.0010) [2023-12-27 09:58:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 320831488. Throughput: 0: 10413.6, 1: 10444.4. Samples: 320796564. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:07,364][19170] Avg episode reward: [(0, '971.430'), (1, '239.700')] [2023-12-27 09:58:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000626368_160374784.pth... [2023-12-27 09:58:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000626720_160456704.pth... [2023-12-27 09:58:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000625152_160063488.pth [2023-12-27 09:58:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000625472_160137216.pth [2023-12-27 09:58:07,513][20381] Updated weights for policy 1, policy_version 626371 (0.0007) [2023-12-27 09:58:07,564][20381] Updated weights for policy 1, policy_version 626381 (0.0005) [2023-12-27 09:58:07,612][20381] Updated weights for policy 1, policy_version 626391 (0.0005) [2023-12-27 09:58:07,968][20347] Updated weights for policy 0, policy_version 626727 (0.0010) [2023-12-27 09:58:08,019][20347] Updated weights for policy 0, policy_version 626737 (0.0009) [2023-12-27 09:58:08,074][20347] Updated weights for policy 0, policy_version 626747 (0.0010) [2023-12-27 09:58:08,235][20381] Updated weights for policy 1, policy_version 626401 (0.0006) [2023-12-27 09:58:08,286][20381] Updated weights for policy 1, policy_version 626411 (0.0009) [2023-12-27 09:58:08,347][20381] Updated weights for policy 1, policy_version 626421 (0.0007) [2023-12-27 09:58:08,399][20381] Updated weights for policy 1, policy_version 626431 (0.0008) [2023-12-27 09:58:08,748][20347] Updated weights for policy 0, policy_version 626757 (0.0009) [2023-12-27 09:58:08,797][20347] Updated weights for policy 0, policy_version 626767 (0.0010) [2023-12-27 09:58:08,845][20347] Updated weights for policy 0, policy_version 626777 (0.0007) [2023-12-27 09:58:09,079][20381] Updated weights for policy 1, policy_version 626441 (0.0006) [2023-12-27 09:58:09,138][20381] Updated weights for policy 1, policy_version 626451 (0.0009) [2023-12-27 09:58:09,188][20381] Updated weights for policy 1, policy_version 626461 (0.0011) [2023-12-27 09:58:09,612][20347] Updated weights for policy 0, policy_version 626787 (0.0008) [2023-12-27 09:58:09,671][20347] Updated weights for policy 0, policy_version 626797 (0.0006) [2023-12-27 09:58:09,729][20347] Updated weights for policy 0, policy_version 626807 (0.0007) [2023-12-27 09:58:09,830][20381] Updated weights for policy 1, policy_version 626471 (0.0008) [2023-12-27 09:58:09,900][20381] Updated weights for policy 1, policy_version 626481 (0.0008) [2023-12-27 09:58:09,966][20381] Updated weights for policy 1, policy_version 626491 (0.0008) [2023-12-27 09:58:10,429][20347] Updated weights for policy 0, policy_version 626817 (0.0008) [2023-12-27 09:58:10,482][20347] Updated weights for policy 0, policy_version 626827 (0.0007) [2023-12-27 09:58:10,538][20347] Updated weights for policy 0, policy_version 626837 (0.0008) [2023-12-27 09:58:10,586][20347] Updated weights for policy 0, policy_version 626847 (0.0008) [2023-12-27 09:58:10,669][20381] Updated weights for policy 1, policy_version 626501 (0.0008) [2023-12-27 09:58:10,731][20381] Updated weights for policy 1, policy_version 626511 (0.0008) [2023-12-27 09:58:10,788][20381] Updated weights for policy 1, policy_version 626521 (0.0006) [2023-12-27 09:58:11,352][20347] Updated weights for policy 0, policy_version 626857 (0.0008) [2023-12-27 09:58:11,360][20291] KL-divergence is very high: 201.8353 [2023-12-27 09:58:11,380][20291] KL-divergence is very high: 250.2209 [2023-12-27 09:58:11,417][20347] Updated weights for policy 0, policy_version 626867 (0.0008) [2023-12-27 09:58:11,451][20381] Updated weights for policy 1, policy_version 626531 (0.0007) [2023-12-27 09:58:11,472][20347] Updated weights for policy 0, policy_version 626877 (0.0006) [2023-12-27 09:58:11,502][20381] Updated weights for policy 1, policy_version 626541 (0.0006) [2023-12-27 09:58:11,558][20381] Updated weights for policy 1, policy_version 626551 (0.0008) [2023-12-27 09:58:12,176][20347] Updated weights for policy 0, policy_version 626887 (0.0007) [2023-12-27 09:58:12,226][20347] Updated weights for policy 0, policy_version 626897 (0.0006) [2023-12-27 09:58:12,277][20347] Updated weights for policy 0, policy_version 626907 (0.0007) [2023-12-27 09:58:12,310][20381] Updated weights for policy 1, policy_version 626561 (0.0008) [2023-12-27 09:58:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 320929792. Throughput: 0: 10415.5, 1: 10387.8. Samples: 320920760. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:12,363][19170] Avg episode reward: [(0, '990.320'), (1, '260.860')] [2023-12-27 09:58:12,373][20381] Updated weights for policy 1, policy_version 626571 (0.0007) [2023-12-27 09:58:12,423][20381] Updated weights for policy 1, policy_version 626581 (0.0007) [2023-12-27 09:58:12,480][20381] Updated weights for policy 1, policy_version 626591 (0.0005) [2023-12-27 09:58:12,941][20347] Updated weights for policy 0, policy_version 626917 (0.0007) [2023-12-27 09:58:12,996][20347] Updated weights for policy 0, policy_version 626927 (0.0009) [2023-12-27 09:58:13,051][20347] Updated weights for policy 0, policy_version 626937 (0.0010) [2023-12-27 09:58:13,076][20291] KL-divergence is very high: 517.8572 [2023-12-27 09:58:13,135][20381] Updated weights for policy 1, policy_version 626601 (0.0008) [2023-12-27 09:58:13,193][20381] Updated weights for policy 1, policy_version 626611 (0.0007) [2023-12-27 09:58:13,244][20381] Updated weights for policy 1, policy_version 626621 (0.0008) [2023-12-27 09:58:13,709][20347] Updated weights for policy 0, policy_version 626947 (0.0010) [2023-12-27 09:58:13,762][20347] Updated weights for policy 0, policy_version 626957 (0.0010) [2023-12-27 09:58:13,818][20347] Updated weights for policy 0, policy_version 626967 (0.0010) [2023-12-27 09:58:13,953][20381] Updated weights for policy 1, policy_version 626631 (0.0008) [2023-12-27 09:58:14,010][20381] Updated weights for policy 1, policy_version 626641 (0.0007) [2023-12-27 09:58:14,061][20381] Updated weights for policy 1, policy_version 626651 (0.0007) [2023-12-27 09:58:14,455][20347] Updated weights for policy 0, policy_version 626977 (0.0010) [2023-12-27 09:58:14,505][20347] Updated weights for policy 0, policy_version 626987 (0.0007) [2023-12-27 09:58:14,550][20347] Updated weights for policy 0, policy_version 626997 (0.0006) [2023-12-27 09:58:14,596][20347] Updated weights for policy 0, policy_version 627007 (0.0007) [2023-12-27 09:58:14,786][20381] Updated weights for policy 1, policy_version 626661 (0.0007) [2023-12-27 09:58:14,842][20381] Updated weights for policy 1, policy_version 626671 (0.0007) [2023-12-27 09:58:14,889][20381] Updated weights for policy 1, policy_version 626681 (0.0008) [2023-12-27 09:58:15,348][20347] Updated weights for policy 0, policy_version 627017 (0.0010) [2023-12-27 09:58:15,403][20347] Updated weights for policy 0, policy_version 627027 (0.0010) [2023-12-27 09:58:15,452][20347] Updated weights for policy 0, policy_version 627037 (0.0010) [2023-12-27 09:58:15,541][20381] Updated weights for policy 1, policy_version 626691 (0.0008) [2023-12-27 09:58:15,607][20381] Updated weights for policy 1, policy_version 626701 (0.0006) [2023-12-27 09:58:15,649][20381] Updated weights for policy 1, policy_version 626711 (0.0005) [2023-12-27 09:58:16,136][20347] Updated weights for policy 0, policy_version 627047 (0.0008) [2023-12-27 09:58:16,189][20347] Updated weights for policy 0, policy_version 627057 (0.0008) [2023-12-27 09:58:16,246][20347] Updated weights for policy 0, policy_version 627067 (0.0008) [2023-12-27 09:58:16,301][20381] Updated weights for policy 1, policy_version 626721 (0.0005) [2023-12-27 09:58:16,357][20381] Updated weights for policy 1, policy_version 626731 (0.0008) [2023-12-27 09:58:16,410][20381] Updated weights for policy 1, policy_version 626741 (0.0009) [2023-12-27 09:58:16,466][20381] Updated weights for policy 1, policy_version 626751 (0.0007) [2023-12-27 09:58:16,890][20347] Updated weights for policy 0, policy_version 627077 (0.0007) [2023-12-27 09:58:16,941][20347] Updated weights for policy 0, policy_version 627087 (0.0006) [2023-12-27 09:58:16,994][20347] Updated weights for policy 0, policy_version 627097 (0.0005) [2023-12-27 09:58:17,112][20381] Updated weights for policy 1, policy_version 626761 (0.0007) [2023-12-27 09:58:17,170][20381] Updated weights for policy 1, policy_version 626771 (0.0006) [2023-12-27 09:58:17,217][20381] Updated weights for policy 1, policy_version 626781 (0.0007) [2023-12-27 09:58:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 321036288. Throughput: 0: 10337.4, 1: 10402.1. Samples: 321043248. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:17,364][19170] Avg episode reward: [(0, '980.830'), (1, '277.530')] [2023-12-27 09:58:17,649][20347] Updated weights for policy 0, policy_version 627107 (0.0006) [2023-12-27 09:58:17,702][20347] Updated weights for policy 0, policy_version 627117 (0.0009) [2023-12-27 09:58:17,754][20347] Updated weights for policy 0, policy_version 627127 (0.0009) [2023-12-27 09:58:17,831][20381] Updated weights for policy 1, policy_version 626791 (0.0006) [2023-12-27 09:58:17,891][20381] Updated weights for policy 1, policy_version 626801 (0.0005) [2023-12-27 09:58:17,956][20381] Updated weights for policy 1, policy_version 626811 (0.0005) [2023-12-27 09:58:18,427][20347] Updated weights for policy 0, policy_version 627137 (0.0008) [2023-12-27 09:58:18,485][20347] Updated weights for policy 0, policy_version 627147 (0.0008) [2023-12-27 09:58:18,543][20347] Updated weights for policy 0, policy_version 627157 (0.0007) [2023-12-27 09:58:18,587][20381] Updated weights for policy 1, policy_version 626821 (0.0007) [2023-12-27 09:58:18,600][20347] Updated weights for policy 0, policy_version 627167 (0.0007) [2023-12-27 09:58:18,645][20381] Updated weights for policy 1, policy_version 626831 (0.0007) [2023-12-27 09:58:18,703][20381] Updated weights for policy 1, policy_version 626841 (0.0008) [2023-12-27 09:58:19,252][20347] Updated weights for policy 0, policy_version 627177 (0.0007) [2023-12-27 09:58:19,301][20347] Updated weights for policy 0, policy_version 627187 (0.0007) [2023-12-27 09:58:19,357][20347] Updated weights for policy 0, policy_version 627197 (0.0008) [2023-12-27 09:58:19,417][20381] Updated weights for policy 1, policy_version 626851 (0.0007) [2023-12-27 09:58:19,481][20381] Updated weights for policy 1, policy_version 626861 (0.0008) [2023-12-27 09:58:19,538][20381] Updated weights for policy 1, policy_version 626871 (0.0008) [2023-12-27 09:58:20,115][20347] Updated weights for policy 0, policy_version 627207 (0.0008) [2023-12-27 09:58:20,167][20347] Updated weights for policy 0, policy_version 627217 (0.0007) [2023-12-27 09:58:20,223][20347] Updated weights for policy 0, policy_version 627227 (0.0008) [2023-12-27 09:58:20,235][20381] Updated weights for policy 1, policy_version 626881 (0.0007) [2023-12-27 09:58:20,296][20381] Updated weights for policy 1, policy_version 626891 (0.0009) [2023-12-27 09:58:20,357][20381] Updated weights for policy 1, policy_version 626901 (0.0009) [2023-12-27 09:58:20,418][20381] Updated weights for policy 1, policy_version 626911 (0.0008) [2023-12-27 09:58:20,917][20347] Updated weights for policy 0, policy_version 627237 (0.0007) [2023-12-27 09:58:20,976][20347] Updated weights for policy 0, policy_version 627247 (0.0006) [2023-12-27 09:58:21,039][20347] Updated weights for policy 0, policy_version 627257 (0.0008) [2023-12-27 09:58:21,138][20381] Updated weights for policy 1, policy_version 626921 (0.0007) [2023-12-27 09:58:21,204][20381] Updated weights for policy 1, policy_version 626931 (0.0007) [2023-12-27 09:58:21,269][20381] Updated weights for policy 1, policy_version 626941 (0.0008) [2023-12-27 09:58:21,811][20347] Updated weights for policy 0, policy_version 627267 (0.0009) [2023-12-27 09:58:21,867][20347] Updated weights for policy 0, policy_version 627277 (0.0010) [2023-12-27 09:58:21,920][20347] Updated weights for policy 0, policy_version 627287 (0.0010) [2023-12-27 09:58:22,016][20381] Updated weights for policy 1, policy_version 626951 (0.0008) [2023-12-27 09:58:22,082][20381] Updated weights for policy 1, policy_version 626961 (0.0010) [2023-12-27 09:58:22,145][20381] Updated weights for policy 1, policy_version 626971 (0.0006) [2023-12-27 09:58:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 321134592. Throughput: 0: 10336.1, 1: 10336.2. Samples: 321104520. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:22,363][19170] Avg episode reward: [(0, '990.510'), (1, '299.080')] [2023-12-27 09:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000626976_160530432.pth... [2023-12-27 09:58:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000627296_160604160.pth... [2023-12-27 09:58:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000626080_160292864.pth [2023-12-27 09:58:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000625792_160227328.pth [2023-12-27 09:58:22,592][20347] Updated weights for policy 0, policy_version 627297 (0.0008) [2023-12-27 09:58:22,651][20347] Updated weights for policy 0, policy_version 627307 (0.0010) [2023-12-27 09:58:22,710][20347] Updated weights for policy 0, policy_version 627317 (0.0011) [2023-12-27 09:58:22,773][20347] Updated weights for policy 0, policy_version 627327 (0.0011) [2023-12-27 09:58:22,808][20381] Updated weights for policy 1, policy_version 626981 (0.0007) [2023-12-27 09:58:22,874][20381] Updated weights for policy 1, policy_version 626991 (0.0008) [2023-12-27 09:58:22,925][20381] Updated weights for policy 1, policy_version 627001 (0.0007) [2023-12-27 09:58:23,411][20347] Updated weights for policy 0, policy_version 627337 (0.0009) [2023-12-27 09:58:23,469][20347] Updated weights for policy 0, policy_version 627347 (0.0010) [2023-12-27 09:58:23,530][20347] Updated weights for policy 0, policy_version 627357 (0.0007) [2023-12-27 09:58:23,581][20381] Updated weights for policy 1, policy_version 627011 (0.0007) [2023-12-27 09:58:23,638][20381] Updated weights for policy 1, policy_version 627021 (0.0008) [2023-12-27 09:58:23,691][20381] Updated weights for policy 1, policy_version 627031 (0.0009) [2023-12-27 09:58:24,066][20347] Updated weights for policy 0, policy_version 627367 (0.0005) [2023-12-27 09:58:24,132][20347] Updated weights for policy 0, policy_version 627377 (0.0008) [2023-12-27 09:58:24,187][20347] Updated weights for policy 0, policy_version 627387 (0.0009) [2023-12-27 09:58:24,366][20381] Updated weights for policy 1, policy_version 627041 (0.0008) [2023-12-27 09:58:24,424][20381] Updated weights for policy 1, policy_version 627051 (0.0008) [2023-12-27 09:58:24,483][20381] Updated weights for policy 1, policy_version 627061 (0.0006) [2023-12-27 09:58:24,537][20381] Updated weights for policy 1, policy_version 627071 (0.0006) [2023-12-27 09:58:24,805][20347] Updated weights for policy 0, policy_version 627397 (0.0010) [2023-12-27 09:58:24,853][20347] Updated weights for policy 0, policy_version 627407 (0.0010) [2023-12-27 09:58:24,901][20347] Updated weights for policy 0, policy_version 627417 (0.0010) [2023-12-27 09:58:25,178][20381] Updated weights for policy 1, policy_version 627081 (0.0008) [2023-12-27 09:58:25,231][20381] Updated weights for policy 1, policy_version 627091 (0.0009) [2023-12-27 09:58:25,279][20381] Updated weights for policy 1, policy_version 627101 (0.0010) [2023-12-27 09:58:25,477][20347] Updated weights for policy 0, policy_version 627427 (0.0009) [2023-12-27 09:58:25,533][20347] Updated weights for policy 0, policy_version 627437 (0.0005) [2023-12-27 09:58:25,593][20347] Updated weights for policy 0, policy_version 627447 (0.0005) [2023-12-27 09:58:26,062][20381] Updated weights for policy 1, policy_version 627111 (0.0008) [2023-12-27 09:58:26,114][20381] Updated weights for policy 1, policy_version 627121 (0.0008) [2023-12-27 09:58:26,159][20347] Updated weights for policy 0, policy_version 627457 (0.0006) [2023-12-27 09:58:26,162][20381] Updated weights for policy 1, policy_version 627131 (0.0008) [2023-12-27 09:58:26,209][20347] Updated weights for policy 0, policy_version 627467 (0.0007) [2023-12-27 09:58:26,254][20347] Updated weights for policy 0, policy_version 627477 (0.0007) [2023-12-27 09:58:26,300][20347] Updated weights for policy 0, policy_version 627487 (0.0006) [2023-12-27 09:58:26,883][20381] Updated weights for policy 1, policy_version 627141 (0.0008) [2023-12-27 09:58:26,922][20347] Updated weights for policy 0, policy_version 627497 (0.0007) [2023-12-27 09:58:26,936][20381] Updated weights for policy 1, policy_version 627151 (0.0009) [2023-12-27 09:58:26,972][20347] Updated weights for policy 0, policy_version 627507 (0.0006) [2023-12-27 09:58:26,989][20381] Updated weights for policy 1, policy_version 627161 (0.0008) [2023-12-27 09:58:27,017][20347] Updated weights for policy 0, policy_version 627517 (0.0007) [2023-12-27 09:58:27,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 321241088. Throughput: 0: 10490.0, 1: 10290.3. Samples: 321229284. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '271.590')] [2023-12-27 09:58:27,599][20381] Updated weights for policy 1, policy_version 627171 (0.0006) [2023-12-27 09:58:27,646][20381] Updated weights for policy 1, policy_version 627181 (0.0005) [2023-12-27 09:58:27,648][20347] Updated weights for policy 0, policy_version 627527 (0.0006) [2023-12-27 09:58:27,701][20381] Updated weights for policy 1, policy_version 627191 (0.0005) [2023-12-27 09:58:27,703][20347] Updated weights for policy 0, policy_version 627537 (0.0007) [2023-12-27 09:58:27,756][20347] Updated weights for policy 0, policy_version 627547 (0.0007) [2023-12-27 09:58:28,290][20381] Updated weights for policy 1, policy_version 627201 (0.0008) [2023-12-27 09:58:28,320][20347] Updated weights for policy 0, policy_version 627557 (0.0006) [2023-12-27 09:58:28,359][20381] Updated weights for policy 1, policy_version 627211 (0.0010) [2023-12-27 09:58:28,373][20347] Updated weights for policy 0, policy_version 627567 (0.0007) [2023-12-27 09:58:28,416][20381] Updated weights for policy 1, policy_version 627221 (0.0005) [2023-12-27 09:58:28,421][20347] Updated weights for policy 0, policy_version 627577 (0.0005) [2023-12-27 09:58:28,476][20381] Updated weights for policy 1, policy_version 627231 (0.0008) [2023-12-27 09:58:29,017][20347] Updated weights for policy 0, policy_version 627587 (0.0006) [2023-12-27 09:58:29,066][20347] Updated weights for policy 0, policy_version 627597 (0.0006) [2023-12-27 09:58:29,120][20347] Updated weights for policy 0, policy_version 627607 (0.0008) [2023-12-27 09:58:29,121][20381] Updated weights for policy 1, policy_version 627241 (0.0005) [2023-12-27 09:58:29,175][20381] Updated weights for policy 1, policy_version 627251 (0.0005) [2023-12-27 09:58:29,221][20381] Updated weights for policy 1, policy_version 627261 (0.0006) [2023-12-27 09:58:29,795][20381] Updated weights for policy 1, policy_version 627271 (0.0007) [2023-12-27 09:58:29,814][20347] Updated weights for policy 0, policy_version 627617 (0.0008) [2023-12-27 09:58:29,856][20381] Updated weights for policy 1, policy_version 627281 (0.0007) [2023-12-27 09:58:29,874][20347] Updated weights for policy 0, policy_version 627627 (0.0006) [2023-12-27 09:58:29,915][20381] Updated weights for policy 1, policy_version 627291 (0.0005) [2023-12-27 09:58:29,937][20347] Updated weights for policy 0, policy_version 627637 (0.0010) [2023-12-27 09:58:29,996][20347] Updated weights for policy 0, policy_version 627647 (0.0010) [2023-12-27 09:58:30,599][20381] Updated weights for policy 1, policy_version 627301 (0.0007) [2023-12-27 09:58:30,649][20381] Updated weights for policy 1, policy_version 627311 (0.0009) [2023-12-27 09:58:30,667][20347] Updated weights for policy 0, policy_version 627657 (0.0010) [2023-12-27 09:58:30,706][20381] Updated weights for policy 1, policy_version 627321 (0.0006) [2023-12-27 09:58:30,717][20347] Updated weights for policy 0, policy_version 627667 (0.0009) [2023-12-27 09:58:30,767][20347] Updated weights for policy 0, policy_version 627677 (0.0005) [2023-12-27 09:58:31,411][20381] Updated weights for policy 1, policy_version 627331 (0.0007) [2023-12-27 09:58:31,465][20347] Updated weights for policy 0, policy_version 627687 (0.0010) [2023-12-27 09:58:31,474][20381] Updated weights for policy 1, policy_version 627341 (0.0006) [2023-12-27 09:58:31,520][20347] Updated weights for policy 0, policy_version 627697 (0.0010) [2023-12-27 09:58:31,533][20381] Updated weights for policy 1, policy_version 627351 (0.0005) [2023-12-27 09:58:31,584][20347] Updated weights for policy 0, policy_version 627707 (0.0010) [2023-12-27 09:58:32,112][20381] Updated weights for policy 1, policy_version 627361 (0.0006) [2023-12-27 09:58:32,164][20381] Updated weights for policy 1, policy_version 627371 (0.0005) [2023-12-27 09:58:32,192][20347] Updated weights for policy 0, policy_version 627717 (0.0010) [2023-12-27 09:58:32,217][20381] Updated weights for policy 1, policy_version 627381 (0.0005) [2023-12-27 09:58:32,252][20347] Updated weights for policy 0, policy_version 627727 (0.0010) [2023-12-27 09:58:32,278][20381] Updated weights for policy 1, policy_version 627391 (0.0006) [2023-12-27 09:58:32,312][20347] Updated weights for policy 0, policy_version 627737 (0.0007) [2023-12-27 09:58:32,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 321355776. Throughput: 0: 10578.5, 1: 10351.7. Samples: 321359640. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:32,363][19170] Avg episode reward: [(0, '990.790'), (1, '257.540')] [2023-12-27 09:58:32,956][20381] Updated weights for policy 1, policy_version 627401 (0.0005) [2023-12-27 09:58:32,983][20347] Updated weights for policy 0, policy_version 627747 (0.0006) [2023-12-27 09:58:33,005][20381] Updated weights for policy 1, policy_version 627411 (0.0005) [2023-12-27 09:58:33,039][20347] Updated weights for policy 0, policy_version 627757 (0.0005) [2023-12-27 09:58:33,051][20381] Updated weights for policy 1, policy_version 627421 (0.0005) [2023-12-27 09:58:33,096][20347] Updated weights for policy 0, policy_version 627767 (0.0009) [2023-12-27 09:58:33,673][20347] Updated weights for policy 0, policy_version 627777 (0.0010) [2023-12-27 09:58:33,676][20381] Updated weights for policy 1, policy_version 627431 (0.0007) [2023-12-27 09:58:33,723][20347] Updated weights for policy 0, policy_version 627787 (0.0007) [2023-12-27 09:58:33,724][20381] Updated weights for policy 1, policy_version 627441 (0.0005) [2023-12-27 09:58:33,777][20347] Updated weights for policy 0, policy_version 627797 (0.0010) [2023-12-27 09:58:33,779][20381] Updated weights for policy 1, policy_version 627451 (0.0005) [2023-12-27 09:58:33,832][20347] Updated weights for policy 0, policy_version 627807 (0.0010) [2023-12-27 09:58:34,329][20381] Updated weights for policy 1, policy_version 627461 (0.0005) [2023-12-27 09:58:34,388][20381] Updated weights for policy 1, policy_version 627471 (0.0006) [2023-12-27 09:58:34,443][20381] Updated weights for policy 1, policy_version 627481 (0.0007) [2023-12-27 09:58:34,555][20347] Updated weights for policy 0, policy_version 627817 (0.0009) [2023-12-27 09:58:34,610][20347] Updated weights for policy 0, policy_version 627827 (0.0008) [2023-12-27 09:58:34,670][20347] Updated weights for policy 0, policy_version 627837 (0.0008) [2023-12-27 09:58:35,043][20381] Updated weights for policy 1, policy_version 627491 (0.0006) [2023-12-27 09:58:35,093][20381] Updated weights for policy 1, policy_version 627501 (0.0008) [2023-12-27 09:58:35,147][20381] Updated weights for policy 1, policy_version 627511 (0.0008) [2023-12-27 09:58:35,388][20347] Updated weights for policy 0, policy_version 627847 (0.0008) [2023-12-27 09:58:35,439][20347] Updated weights for policy 0, policy_version 627857 (0.0010) [2023-12-27 09:58:35,494][20347] Updated weights for policy 0, policy_version 627867 (0.0009) [2023-12-27 09:58:35,763][20381] Updated weights for policy 1, policy_version 627521 (0.0008) [2023-12-27 09:58:35,823][20381] Updated weights for policy 1, policy_version 627531 (0.0008) [2023-12-27 09:58:35,881][20381] Updated weights for policy 1, policy_version 627541 (0.0006) [2023-12-27 09:58:35,938][20381] Updated weights for policy 1, policy_version 627551 (0.0005) [2023-12-27 09:58:36,067][20347] Updated weights for policy 0, policy_version 627877 (0.0008) [2023-12-27 09:58:36,127][20347] Updated weights for policy 0, policy_version 627887 (0.0007) [2023-12-27 09:58:36,184][20347] Updated weights for policy 0, policy_version 627897 (0.0007) [2023-12-27 09:58:36,592][20381] Updated weights for policy 1, policy_version 627561 (0.0008) [2023-12-27 09:58:36,652][20381] Updated weights for policy 1, policy_version 627571 (0.0007) [2023-12-27 09:58:36,710][20381] Updated weights for policy 1, policy_version 627581 (0.0008) [2023-12-27 09:58:36,805][20347] Updated weights for policy 0, policy_version 627907 (0.0006) [2023-12-27 09:58:36,856][20347] Updated weights for policy 0, policy_version 627917 (0.0006) [2023-12-27 09:58:36,911][20347] Updated weights for policy 0, policy_version 627927 (0.0008) [2023-12-27 09:58:37,363][19170] Fps is (10 sec: 21297.8, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 321454080. Throughput: 0: 10579.4, 1: 10374.3. Samples: 321423648. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:37,364][19170] Avg episode reward: [(0, '990.330'), (1, '289.420')] [2023-12-27 09:58:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000627584_160686080.pth... [2023-12-27 09:58:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000627936_160768000.pth... [2023-12-27 09:58:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000626368_160374784.pth [2023-12-27 09:58:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000626720_160456704.pth [2023-12-27 09:58:37,472][20381] Updated weights for policy 1, policy_version 627591 (0.0007) [2023-12-27 09:58:37,498][20347] Updated weights for policy 0, policy_version 627937 (0.0007) [2023-12-27 09:58:37,529][20381] Updated weights for policy 1, policy_version 627601 (0.0008) [2023-12-27 09:58:37,553][20347] Updated weights for policy 0, policy_version 627947 (0.0007) [2023-12-27 09:58:37,578][20381] Updated weights for policy 1, policy_version 627611 (0.0006) [2023-12-27 09:58:37,606][20347] Updated weights for policy 0, policy_version 627957 (0.0005) [2023-12-27 09:58:37,663][20347] Updated weights for policy 0, policy_version 627967 (0.0007) [2023-12-27 09:58:38,283][20381] Updated weights for policy 1, policy_version 627621 (0.0006) [2023-12-27 09:58:38,335][20381] Updated weights for policy 1, policy_version 627631 (0.0007) [2023-12-27 09:58:38,367][20347] Updated weights for policy 0, policy_version 627977 (0.0007) [2023-12-27 09:58:38,399][20381] Updated weights for policy 1, policy_version 627641 (0.0007) [2023-12-27 09:58:38,417][20347] Updated weights for policy 0, policy_version 627987 (0.0006) [2023-12-27 09:58:38,465][20347] Updated weights for policy 0, policy_version 627997 (0.0008) [2023-12-27 09:58:39,125][20381] Updated weights for policy 1, policy_version 627651 (0.0009) [2023-12-27 09:58:39,180][20381] Updated weights for policy 1, policy_version 627661 (0.0010) [2023-12-27 09:58:39,182][20347] Updated weights for policy 0, policy_version 628007 (0.0007) [2023-12-27 09:58:39,234][20381] Updated weights for policy 1, policy_version 627671 (0.0010) [2023-12-27 09:58:39,246][20347] Updated weights for policy 0, policy_version 628017 (0.0007) [2023-12-27 09:58:39,306][20347] Updated weights for policy 0, policy_version 628027 (0.0006) [2023-12-27 09:58:39,982][20381] Updated weights for policy 1, policy_version 627681 (0.0008) [2023-12-27 09:58:40,024][20347] Updated weights for policy 0, policy_version 628037 (0.0007) [2023-12-27 09:58:40,042][20381] Updated weights for policy 1, policy_version 627691 (0.0006) [2023-12-27 09:58:40,080][20347] Updated weights for policy 0, policy_version 628047 (0.0006) [2023-12-27 09:58:40,097][20381] Updated weights for policy 1, policy_version 627701 (0.0007) [2023-12-27 09:58:40,136][20347] Updated weights for policy 0, policy_version 628057 (0.0007) [2023-12-27 09:58:40,146][20381] Updated weights for policy 1, policy_version 627711 (0.0007) [2023-12-27 09:58:40,795][20381] Updated weights for policy 1, policy_version 627721 (0.0008) [2023-12-27 09:58:40,851][20381] Updated weights for policy 1, policy_version 627731 (0.0005) [2023-12-27 09:58:40,868][20347] Updated weights for policy 0, policy_version 628067 (0.0007) [2023-12-27 09:58:40,905][20381] Updated weights for policy 1, policy_version 627741 (0.0005) [2023-12-27 09:58:40,919][20347] Updated weights for policy 0, policy_version 628077 (0.0007) [2023-12-27 09:58:40,967][20347] Updated weights for policy 0, policy_version 628087 (0.0005) [2023-12-27 09:58:41,583][20381] Updated weights for policy 1, policy_version 627751 (0.0007) [2023-12-27 09:58:41,652][20381] Updated weights for policy 1, policy_version 627761 (0.0009) [2023-12-27 09:58:41,656][20347] Updated weights for policy 0, policy_version 628097 (0.0006) [2023-12-27 09:58:41,719][20381] Updated weights for policy 1, policy_version 627771 (0.0008) [2023-12-27 09:58:41,721][20291] KL-divergence is very high: 274.7630 [2023-12-27 09:58:41,722][20347] Updated weights for policy 0, policy_version 628107 (0.0006) [2023-12-27 09:58:41,781][20347] Updated weights for policy 0, policy_version 628117 (0.0007) [2023-12-27 09:58:41,841][20347] Updated weights for policy 0, policy_version 628127 (0.0007) [2023-12-27 09:58:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 321552384. Throughput: 0: 10585.4, 1: 10308.6. Samples: 321546908. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:42,363][19170] Avg episode reward: [(0, '981.090'), (1, '284.540')] [2023-12-27 09:58:42,430][20381] Updated weights for policy 1, policy_version 627781 (0.0008) [2023-12-27 09:58:42,482][20381] Updated weights for policy 1, policy_version 627791 (0.0007) [2023-12-27 09:58:42,511][20347] Updated weights for policy 0, policy_version 628137 (0.0007) [2023-12-27 09:58:42,536][20381] Updated weights for policy 1, policy_version 627801 (0.0007) [2023-12-27 09:58:42,561][20347] Updated weights for policy 0, policy_version 628147 (0.0007) [2023-12-27 09:58:42,622][20347] Updated weights for policy 0, policy_version 628157 (0.0007) [2023-12-27 09:58:43,180][20381] Updated weights for policy 1, policy_version 627811 (0.0006) [2023-12-27 09:58:43,227][20381] Updated weights for policy 1, policy_version 627821 (0.0006) [2023-12-27 09:58:43,276][20381] Updated weights for policy 1, policy_version 627831 (0.0007) [2023-12-27 09:58:43,300][20347] Updated weights for policy 0, policy_version 628167 (0.0007) [2023-12-27 09:58:43,361][20347] Updated weights for policy 0, policy_version 628177 (0.0007) [2023-12-27 09:58:43,417][20347] Updated weights for policy 0, policy_version 628187 (0.0008) [2023-12-27 09:58:43,916][20381] Updated weights for policy 1, policy_version 627841 (0.0006) [2023-12-27 09:58:43,966][20381] Updated weights for policy 1, policy_version 627851 (0.0007) [2023-12-27 09:58:44,022][20381] Updated weights for policy 1, policy_version 627861 (0.0007) [2023-12-27 09:58:44,028][20347] Updated weights for policy 0, policy_version 628197 (0.0007) [2023-12-27 09:58:44,081][20347] Updated weights for policy 0, policy_version 628207 (0.0006) [2023-12-27 09:58:44,091][20381] Updated weights for policy 1, policy_version 627871 (0.0006) [2023-12-27 09:58:44,133][20347] Updated weights for policy 0, policy_version 628217 (0.0005) [2023-12-27 09:58:44,718][20347] Updated weights for policy 0, policy_version 628227 (0.0006) [2023-12-27 09:58:44,776][20347] Updated weights for policy 0, policy_version 628237 (0.0007) [2023-12-27 09:58:44,778][20381] Updated weights for policy 1, policy_version 627881 (0.0007) [2023-12-27 09:58:44,838][20347] Updated weights for policy 0, policy_version 628247 (0.0007) [2023-12-27 09:58:44,842][20381] Updated weights for policy 1, policy_version 627891 (0.0008) [2023-12-27 09:58:44,912][20381] Updated weights for policy 1, policy_version 627901 (0.0008) [2023-12-27 09:58:45,462][20347] Updated weights for policy 0, policy_version 628257 (0.0006) [2023-12-27 09:58:45,521][20347] Updated weights for policy 0, policy_version 628267 (0.0007) [2023-12-27 09:58:45,576][20347] Updated weights for policy 0, policy_version 628277 (0.0005) [2023-12-27 09:58:45,611][20381] Updated weights for policy 1, policy_version 627911 (0.0008) [2023-12-27 09:58:45,624][20347] Updated weights for policy 0, policy_version 628287 (0.0005) [2023-12-27 09:58:45,669][20381] Updated weights for policy 1, policy_version 627921 (0.0006) [2023-12-27 09:58:45,722][20381] Updated weights for policy 1, policy_version 627931 (0.0009) [2023-12-27 09:58:46,224][20347] Updated weights for policy 0, policy_version 628297 (0.0010) [2023-12-27 09:58:46,284][20347] Updated weights for policy 0, policy_version 628307 (0.0010) [2023-12-27 09:58:46,339][20347] Updated weights for policy 0, policy_version 628317 (0.0010) [2023-12-27 09:58:46,426][20381] Updated weights for policy 1, policy_version 627941 (0.0007) [2023-12-27 09:58:46,478][20381] Updated weights for policy 1, policy_version 627951 (0.0008) [2023-12-27 09:58:46,532][20381] Updated weights for policy 1, policy_version 627961 (0.0007) [2023-12-27 09:58:47,022][20347] Updated weights for policy 0, policy_version 628327 (0.0007) [2023-12-27 09:58:47,091][20347] Updated weights for policy 0, policy_version 628337 (0.0005) [2023-12-27 09:58:47,147][20347] Updated weights for policy 0, policy_version 628347 (0.0007) [2023-12-27 09:58:47,217][20381] Updated weights for policy 1, policy_version 627971 (0.0008) [2023-12-27 09:58:47,263][20381] Updated weights for policy 1, policy_version 627981 (0.0010) [2023-12-27 09:58:47,308][20381] Updated weights for policy 1, policy_version 627991 (0.0008) [2023-12-27 09:58:47,363][19170] Fps is (10 sec: 21300.0, 60 sec: 20889.5, 300 sec: 20854.9). Total num frames: 321667072. Throughput: 0: 10574.6, 1: 10329.8. Samples: 321671268. Policy #0 lag: (min: 19.0, avg: 31.6, max: 32.0) [2023-12-27 09:58:47,364][19170] Avg episode reward: [(0, '981.150'), (1, '292.310')] [2023-12-27 09:58:47,685][20347] Updated weights for policy 0, policy_version 628357 (0.0007) [2023-12-27 09:58:47,739][20347] Updated weights for policy 0, policy_version 628367 (0.0007) [2023-12-27 09:58:47,789][20347] Updated weights for policy 0, policy_version 628377 (0.0005) [2023-12-27 09:58:48,062][20381] Updated weights for policy 1, policy_version 628001 (0.0007) [2023-12-27 09:58:48,109][20381] Updated weights for policy 1, policy_version 628011 (0.0006) [2023-12-27 09:58:48,153][20381] Updated weights for policy 1, policy_version 628021 (0.0007) [2023-12-27 09:58:48,197][20381] Updated weights for policy 1, policy_version 628031 (0.0006) [2023-12-27 09:58:48,443][20347] Updated weights for policy 0, policy_version 628387 (0.0006) [2023-12-27 09:58:48,490][20347] Updated weights for policy 0, policy_version 628397 (0.0005) [2023-12-27 09:58:48,543][20347] Updated weights for policy 0, policy_version 628407 (0.0005) [2023-12-27 09:58:48,919][20381] Updated weights for policy 1, policy_version 628041 (0.0009) [2023-12-27 09:58:48,969][20381] Updated weights for policy 1, policy_version 628051 (0.0007) [2023-12-27 09:58:49,026][20381] Updated weights for policy 1, policy_version 628061 (0.0008) [2023-12-27 09:58:49,162][20347] Updated weights for policy 0, policy_version 628417 (0.0006) [2023-12-27 09:58:49,212][20347] Updated weights for policy 0, policy_version 628427 (0.0006) [2023-12-27 09:58:49,279][20347] Updated weights for policy 0, policy_version 628437 (0.0008) [2023-12-27 09:58:49,338][20347] Updated weights for policy 0, policy_version 628447 (0.0008) [2023-12-27 09:58:49,788][20381] Updated weights for policy 1, policy_version 628071 (0.0009) [2023-12-27 09:58:49,845][20381] Updated weights for policy 1, policy_version 628081 (0.0007) [2023-12-27 09:58:49,895][20381] Updated weights for policy 1, policy_version 628091 (0.0008) [2023-12-27 09:58:50,014][20347] Updated weights for policy 0, policy_version 628457 (0.0007) [2023-12-27 09:58:50,073][20347] Updated weights for policy 0, policy_version 628467 (0.0007) [2023-12-27 09:58:50,132][20347] Updated weights for policy 0, policy_version 628477 (0.0006) [2023-12-27 09:58:50,634][20381] Updated weights for policy 1, policy_version 628101 (0.0007) [2023-12-27 09:58:50,693][20381] Updated weights for policy 1, policy_version 628111 (0.0008) [2023-12-27 09:58:50,751][20381] Updated weights for policy 1, policy_version 628121 (0.0007) [2023-12-27 09:58:50,798][20347] Updated weights for policy 0, policy_version 628487 (0.0009) [2023-12-27 09:58:50,846][20347] Updated weights for policy 0, policy_version 628497 (0.0009) [2023-12-27 09:58:50,892][20347] Updated weights for policy 0, policy_version 628507 (0.0007) [2023-12-27 09:58:51,435][20381] Updated weights for policy 1, policy_version 628131 (0.0007) [2023-12-27 09:58:51,492][20381] Updated weights for policy 1, policy_version 628141 (0.0009) [2023-12-27 09:58:51,545][20381] Updated weights for policy 1, policy_version 628151 (0.0007) [2023-12-27 09:58:51,573][20347] Updated weights for policy 0, policy_version 628517 (0.0007) [2023-12-27 09:58:51,639][20347] Updated weights for policy 0, policy_version 628527 (0.0007) [2023-12-27 09:58:51,699][20347] Updated weights for policy 0, policy_version 628537 (0.0007) [2023-12-27 09:58:52,299][20381] Updated weights for policy 1, policy_version 628161 (0.0007) [2023-12-27 09:58:52,339][20347] Updated weights for policy 0, policy_version 628547 (0.0009) [2023-12-27 09:58:52,353][20381] Updated weights for policy 1, policy_version 628171 (0.0006) [2023-12-27 09:58:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 321757184. Throughput: 0: 10554.5, 1: 10247.4. Samples: 321732648. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:58:52,363][19170] Avg episode reward: [(0, '980.830'), (1, '304.030')] [2023-12-27 09:58:52,397][20347] Updated weights for policy 0, policy_version 628557 (0.0007) [2023-12-27 09:58:52,414][20381] Updated weights for policy 1, policy_version 628181 (0.0006) [2023-12-27 09:58:52,450][20347] Updated weights for policy 0, policy_version 628567 (0.0006) [2023-12-27 09:58:52,474][20381] Updated weights for policy 1, policy_version 628191 (0.0005) [2023-12-27 09:58:52,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000628192_160841728.pth... [2023-12-27 09:58:52,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000626976_160530432.pth [2023-12-27 09:58:52,498][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000628576_160931840.pth... [2023-12-27 09:58:52,501][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000627296_160604160.pth [2023-12-27 09:58:53,027][20347] Updated weights for policy 0, policy_version 628577 (0.0006) [2023-12-27 09:58:53,080][20347] Updated weights for policy 0, policy_version 628587 (0.0006) [2023-12-27 09:58:53,124][20347] Updated weights for policy 0, policy_version 628597 (0.0006) [2023-12-27 09:58:53,171][20347] Updated weights for policy 0, policy_version 628607 (0.0006) [2023-12-27 09:58:53,199][20381] Updated weights for policy 1, policy_version 628201 (0.0007) [2023-12-27 09:58:53,252][20381] Updated weights for policy 1, policy_version 628211 (0.0009) [2023-12-27 09:58:53,301][20381] Updated weights for policy 1, policy_version 628221 (0.0009) [2023-12-27 09:58:53,688][20291] KL-divergence is very high: 152.0495 [2023-12-27 09:58:53,737][20291] KL-divergence is very high: 113.3979 [2023-12-27 09:58:53,737][20347] Updated weights for policy 0, policy_version 628617 (0.0008) [2023-12-27 09:58:53,794][20347] Updated weights for policy 0, policy_version 628627 (0.0007) [2023-12-27 09:58:53,850][20347] Updated weights for policy 0, policy_version 628637 (0.0006) [2023-12-27 09:58:54,023][20381] Updated weights for policy 1, policy_version 628231 (0.0007) [2023-12-27 09:58:54,081][20381] Updated weights for policy 1, policy_version 628241 (0.0005) [2023-12-27 09:58:54,133][20381] Updated weights for policy 1, policy_version 628251 (0.0007) [2023-12-27 09:58:54,430][20347] Updated weights for policy 0, policy_version 628647 (0.0006) [2023-12-27 09:58:54,480][20347] Updated weights for policy 0, policy_version 628657 (0.0005) [2023-12-27 09:58:54,532][20347] Updated weights for policy 0, policy_version 628667 (0.0005) [2023-12-27 09:58:54,819][20381] Updated weights for policy 1, policy_version 628261 (0.0007) [2023-12-27 09:58:54,871][20381] Updated weights for policy 1, policy_version 628271 (0.0008) [2023-12-27 09:58:54,917][20381] Updated weights for policy 1, policy_version 628281 (0.0008) [2023-12-27 09:58:55,082][20347] Updated weights for policy 0, policy_version 628677 (0.0007) [2023-12-27 09:58:55,135][20347] Updated weights for policy 0, policy_version 628687 (0.0006) [2023-12-27 09:58:55,189][20347] Updated weights for policy 0, policy_version 628697 (0.0006) [2023-12-27 09:58:55,613][20381] Updated weights for policy 1, policy_version 628291 (0.0007) [2023-12-27 09:58:55,680][20381] Updated weights for policy 1, policy_version 628301 (0.0007) [2023-12-27 09:58:55,733][20381] Updated weights for policy 1, policy_version 628311 (0.0008) [2023-12-27 09:58:55,876][20347] Updated weights for policy 0, policy_version 628707 (0.0007) [2023-12-27 09:58:55,942][20347] Updated weights for policy 0, policy_version 628717 (0.0005) [2023-12-27 09:58:56,003][20347] Updated weights for policy 0, policy_version 628727 (0.0005) [2023-12-27 09:58:56,358][20381] Updated weights for policy 1, policy_version 628321 (0.0005) [2023-12-27 09:58:56,411][20381] Updated weights for policy 1, policy_version 628331 (0.0005) [2023-12-27 09:58:56,459][20381] Updated weights for policy 1, policy_version 628341 (0.0005) [2023-12-27 09:58:56,488][20347] Updated weights for policy 0, policy_version 628737 (0.0005) [2023-12-27 09:58:56,510][20381] Updated weights for policy 1, policy_version 628351 (0.0005) [2023-12-27 09:58:56,539][20347] Updated weights for policy 0, policy_version 628747 (0.0007) [2023-12-27 09:58:56,587][20347] Updated weights for policy 0, policy_version 628757 (0.0008) [2023-12-27 09:58:56,634][20347] Updated weights for policy 0, policy_version 628767 (0.0005) [2023-12-27 09:58:57,129][20381] Updated weights for policy 1, policy_version 628361 (0.0010) [2023-12-27 09:58:57,187][20381] Updated weights for policy 1, policy_version 628371 (0.0010) [2023-12-27 09:58:57,245][20381] Updated weights for policy 1, policy_version 628381 (0.0008) [2023-12-27 09:58:57,266][20347] Updated weights for policy 0, policy_version 628777 (0.0005) [2023-12-27 09:58:57,317][20347] Updated weights for policy 0, policy_version 628787 (0.0005) [2023-12-27 09:58:57,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 321871872. Throughput: 0: 10659.4, 1: 10190.7. Samples: 321859020. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:58:57,364][19170] Avg episode reward: [(0, '924.100'), (1, '314.510')] [2023-12-27 09:58:57,370][20347] Updated weights for policy 0, policy_version 628797 (0.0005) [2023-12-27 09:58:57,823][20381] Updated weights for policy 1, policy_version 628391 (0.0008) [2023-12-27 09:58:57,879][20381] Updated weights for policy 1, policy_version 628401 (0.0006) [2023-12-27 09:58:57,882][20347] Updated weights for policy 0, policy_version 628807 (0.0005) [2023-12-27 09:58:57,932][20381] Updated weights for policy 1, policy_version 628411 (0.0005) [2023-12-27 09:58:57,934][20347] Updated weights for policy 0, policy_version 628817 (0.0007) [2023-12-27 09:58:57,987][20347] Updated weights for policy 0, policy_version 628827 (0.0006) [2023-12-27 09:58:58,599][20381] Updated weights for policy 1, policy_version 628421 (0.0007) [2023-12-27 09:58:58,660][20381] Updated weights for policy 1, policy_version 628431 (0.0009) [2023-12-27 09:58:58,662][20347] Updated weights for policy 0, policy_version 628837 (0.0007) [2023-12-27 09:58:58,718][20381] Updated weights for policy 1, policy_version 628441 (0.0009) [2023-12-27 09:58:58,722][20347] Updated weights for policy 0, policy_version 628847 (0.0009) [2023-12-27 09:58:58,778][20347] Updated weights for policy 0, policy_version 628857 (0.0009) [2023-12-27 09:58:59,428][20381] Updated weights for policy 1, policy_version 628451 (0.0008) [2023-12-27 09:58:59,437][20347] Updated weights for policy 0, policy_version 628867 (0.0008) [2023-12-27 09:58:59,486][20381] Updated weights for policy 1, policy_version 628461 (0.0006) [2023-12-27 09:58:59,489][20347] Updated weights for policy 0, policy_version 628877 (0.0010) [2023-12-27 09:58:59,539][20347] Updated weights for policy 0, policy_version 628887 (0.0008) [2023-12-27 09:58:59,547][20381] Updated weights for policy 1, policy_version 628471 (0.0005) [2023-12-27 09:59:00,145][20381] Updated weights for policy 1, policy_version 628481 (0.0008) [2023-12-27 09:59:00,206][20381] Updated weights for policy 1, policy_version 628491 (0.0006) [2023-12-27 09:59:00,267][20381] Updated weights for policy 1, policy_version 628501 (0.0007) [2023-12-27 09:59:00,305][20347] Updated weights for policy 0, policy_version 628897 (0.0007) [2023-12-27 09:59:00,324][20381] Updated weights for policy 1, policy_version 628511 (0.0008) [2023-12-27 09:59:00,361][20347] Updated weights for policy 0, policy_version 628907 (0.0008) [2023-12-27 09:59:00,416][20347] Updated weights for policy 0, policy_version 628917 (0.0008) [2023-12-27 09:59:00,470][20347] Updated weights for policy 0, policy_version 628927 (0.0007) [2023-12-27 09:59:00,922][20381] Updated weights for policy 1, policy_version 628521 (0.0008) [2023-12-27 09:59:00,983][20381] Updated weights for policy 1, policy_version 628531 (0.0006) [2023-12-27 09:59:01,042][20381] Updated weights for policy 1, policy_version 628541 (0.0007) [2023-12-27 09:59:01,157][20347] Updated weights for policy 0, policy_version 628937 (0.0008) [2023-12-27 09:59:01,213][20347] Updated weights for policy 0, policy_version 628947 (0.0006) [2023-12-27 09:59:01,275][20347] Updated weights for policy 0, policy_version 628957 (0.0006) [2023-12-27 09:59:01,768][20381] Updated weights for policy 1, policy_version 628551 (0.0007) [2023-12-27 09:59:01,824][20381] Updated weights for policy 1, policy_version 628561 (0.0007) [2023-12-27 09:59:01,877][20381] Updated weights for policy 1, policy_version 628571 (0.0008) [2023-12-27 09:59:01,958][20347] Updated weights for policy 0, policy_version 628967 (0.0008) [2023-12-27 09:59:02,011][20347] Updated weights for policy 0, policy_version 628977 (0.0008) [2023-12-27 09:59:02,059][20347] Updated weights for policy 0, policy_version 628987 (0.0008) [2023-12-27 09:59:02,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 321978368. Throughput: 0: 10721.1, 1: 10263.0. Samples: 321987528. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:02,363][19170] Avg episode reward: [(0, '943.320'), (1, '262.360')] [2023-12-27 09:59:02,537][20381] Updated weights for policy 1, policy_version 628581 (0.0008) [2023-12-27 09:59:02,589][20381] Updated weights for policy 1, policy_version 628591 (0.0006) [2023-12-27 09:59:02,640][20381] Updated weights for policy 1, policy_version 628601 (0.0007) [2023-12-27 09:59:02,779][20347] Updated weights for policy 0, policy_version 628997 (0.0010) [2023-12-27 09:59:02,823][20347] Updated weights for policy 0, policy_version 629007 (0.0010) [2023-12-27 09:59:02,884][20347] Updated weights for policy 0, policy_version 629017 (0.0010) [2023-12-27 09:59:03,330][20381] Updated weights for policy 1, policy_version 628611 (0.0007) [2023-12-27 09:59:03,378][20381] Updated weights for policy 1, policy_version 628621 (0.0007) [2023-12-27 09:59:03,425][20381] Updated weights for policy 1, policy_version 628631 (0.0007) [2023-12-27 09:59:03,564][20347] Updated weights for policy 0, policy_version 629027 (0.0010) [2023-12-27 09:59:03,628][20347] Updated weights for policy 0, policy_version 629037 (0.0010) [2023-12-27 09:59:03,676][20347] Updated weights for policy 0, policy_version 629047 (0.0010) [2023-12-27 09:59:04,166][20381] Updated weights for policy 1, policy_version 628641 (0.0008) [2023-12-27 09:59:04,223][20381] Updated weights for policy 1, policy_version 628651 (0.0007) [2023-12-27 09:59:04,278][20381] Updated weights for policy 1, policy_version 628661 (0.0008) [2023-12-27 09:59:04,338][20381] Updated weights for policy 1, policy_version 628671 (0.0007) [2023-12-27 09:59:04,395][20347] Updated weights for policy 0, policy_version 629057 (0.0010) [2023-12-27 09:59:04,451][20347] Updated weights for policy 0, policy_version 629067 (0.0007) [2023-12-27 09:59:04,509][20347] Updated weights for policy 0, policy_version 629077 (0.0007) [2023-12-27 09:59:04,573][20347] Updated weights for policy 0, policy_version 629087 (0.0006) [2023-12-27 09:59:05,060][20381] Updated weights for policy 1, policy_version 628681 (0.0010) [2023-12-27 09:59:05,122][20381] Updated weights for policy 1, policy_version 628691 (0.0009) [2023-12-27 09:59:05,180][20381] Updated weights for policy 1, policy_version 628701 (0.0006) [2023-12-27 09:59:05,194][20347] Updated weights for policy 0, policy_version 629097 (0.0007) [2023-12-27 09:59:05,246][20347] Updated weights for policy 0, policy_version 629107 (0.0006) [2023-12-27 09:59:05,302][20347] Updated weights for policy 0, policy_version 629117 (0.0007) [2023-12-27 09:59:05,818][20381] Updated weights for policy 1, policy_version 628711 (0.0009) [2023-12-27 09:59:05,871][20381] Updated weights for policy 1, policy_version 628721 (0.0010) [2023-12-27 09:59:05,912][20347] Updated weights for policy 0, policy_version 629127 (0.0010) [2023-12-27 09:59:05,926][20381] Updated weights for policy 1, policy_version 628731 (0.0008) [2023-12-27 09:59:05,964][20347] Updated weights for policy 0, policy_version 629137 (0.0010) [2023-12-27 09:59:06,016][20347] Updated weights for policy 0, policy_version 629147 (0.0007) [2023-12-27 09:59:06,575][20381] Updated weights for policy 1, policy_version 628741 (0.0008) [2023-12-27 09:59:06,626][20381] Updated weights for policy 1, policy_version 628751 (0.0011) [2023-12-27 09:59:06,648][20347] Updated weights for policy 0, policy_version 629157 (0.0007) [2023-12-27 09:59:06,673][20381] Updated weights for policy 1, policy_version 628761 (0.0011) [2023-12-27 09:59:06,706][20347] Updated weights for policy 0, policy_version 629167 (0.0006) [2023-12-27 09:59:06,763][20347] Updated weights for policy 0, policy_version 629177 (0.0007) [2023-12-27 09:59:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 322076672. Throughput: 0: 10720.8, 1: 10244.2. Samples: 322047944. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:07,363][20381] Updated weights for policy 1, policy_version 628771 (0.0011) [2023-12-27 09:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '293.110')] [2023-12-27 09:59:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000629184_161087488.pth... [2023-12-27 09:59:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000627936_160768000.pth [2023-12-27 09:59:07,407][20347] Updated weights for policy 0, policy_version 629187 (0.0007) [2023-12-27 09:59:07,420][20381] Updated weights for policy 1, policy_version 628781 (0.0011) [2023-12-27 09:59:07,465][20347] Updated weights for policy 0, policy_version 629197 (0.0006) [2023-12-27 09:59:07,478][20381] Updated weights for policy 1, policy_version 628791 (0.0011) [2023-12-27 09:59:07,522][20347] Updated weights for policy 0, policy_version 629207 (0.0005) [2023-12-27 09:59:07,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000628800_160997376.pth... [2023-12-27 09:59:07,533][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000627584_160686080.pth [2023-12-27 09:59:08,139][20381] Updated weights for policy 1, policy_version 628801 (0.0011) [2023-12-27 09:59:08,203][20381] Updated weights for policy 1, policy_version 628811 (0.0008) [2023-12-27 09:59:08,212][20347] Updated weights for policy 0, policy_version 629217 (0.0007) [2023-12-27 09:59:08,256][20381] Updated weights for policy 1, policy_version 628821 (0.0010) [2023-12-27 09:59:08,265][20347] Updated weights for policy 0, policy_version 629227 (0.0007) [2023-12-27 09:59:08,301][20381] Updated weights for policy 1, policy_version 628831 (0.0010) [2023-12-27 09:59:08,315][20347] Updated weights for policy 0, policy_version 629237 (0.0007) [2023-12-27 09:59:08,377][20347] Updated weights for policy 0, policy_version 629247 (0.0008) [2023-12-27 09:59:09,008][20347] Updated weights for policy 0, policy_version 629257 (0.0007) [2023-12-27 09:59:09,018][20381] Updated weights for policy 1, policy_version 628841 (0.0008) [2023-12-27 09:59:09,064][20347] Updated weights for policy 0, policy_version 629267 (0.0005) [2023-12-27 09:59:09,072][20381] Updated weights for policy 1, policy_version 628851 (0.0008) [2023-12-27 09:59:09,114][20347] Updated weights for policy 0, policy_version 629277 (0.0007) [2023-12-27 09:59:09,124][20381] Updated weights for policy 1, policy_version 628861 (0.0006) [2023-12-27 09:59:09,767][20347] Updated weights for policy 0, policy_version 629287 (0.0007) [2023-12-27 09:59:09,827][20347] Updated weights for policy 0, policy_version 629297 (0.0008) [2023-12-27 09:59:09,848][20381] Updated weights for policy 1, policy_version 628871 (0.0007) [2023-12-27 09:59:09,891][20347] Updated weights for policy 0, policy_version 629307 (0.0008) [2023-12-27 09:59:09,904][20381] Updated weights for policy 1, policy_version 628881 (0.0007) [2023-12-27 09:59:09,971][20381] Updated weights for policy 1, policy_version 628891 (0.0008) [2023-12-27 09:59:10,595][20347] Updated weights for policy 0, policy_version 629317 (0.0006) [2023-12-27 09:59:10,628][20381] Updated weights for policy 1, policy_version 628901 (0.0006) [2023-12-27 09:59:10,657][20347] Updated weights for policy 0, policy_version 629327 (0.0006) [2023-12-27 09:59:10,679][20381] Updated weights for policy 1, policy_version 628911 (0.0006) [2023-12-27 09:59:10,716][20347] Updated weights for policy 0, policy_version 629337 (0.0007) [2023-12-27 09:59:10,730][20381] Updated weights for policy 1, policy_version 628921 (0.0007) [2023-12-27 09:59:11,438][20347] Updated weights for policy 0, policy_version 629347 (0.0006) [2023-12-27 09:59:11,456][20381] Updated weights for policy 1, policy_version 628931 (0.0007) [2023-12-27 09:59:11,499][20347] Updated weights for policy 0, policy_version 629357 (0.0007) [2023-12-27 09:59:11,514][20381] Updated weights for policy 1, policy_version 628941 (0.0008) [2023-12-27 09:59:11,559][20347] Updated weights for policy 0, policy_version 629367 (0.0006) [2023-12-27 09:59:11,577][20381] Updated weights for policy 1, policy_version 628951 (0.0005) [2023-12-27 09:59:12,235][20347] Updated weights for policy 0, policy_version 629377 (0.0007) [2023-12-27 09:59:12,258][20381] Updated weights for policy 1, policy_version 628961 (0.0007) [2023-12-27 09:59:12,296][20347] Updated weights for policy 0, policy_version 629387 (0.0010) [2023-12-27 09:59:12,317][20381] Updated weights for policy 1, policy_version 628971 (0.0010) [2023-12-27 09:59:12,355][20347] Updated weights for policy 0, policy_version 629397 (0.0010) [2023-12-27 09:59:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 322174976. Throughput: 0: 10671.9, 1: 10287.4. Samples: 322172452. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '236.870')] [2023-12-27 09:59:12,376][20381] Updated weights for policy 1, policy_version 628981 (0.0009) [2023-12-27 09:59:12,413][20347] Updated weights for policy 0, policy_version 629407 (0.0007) [2023-12-27 09:59:12,433][20381] Updated weights for policy 1, policy_version 628991 (0.0008) [2023-12-27 09:59:13,098][20381] Updated weights for policy 1, policy_version 629001 (0.0006) [2023-12-27 09:59:13,105][20347] Updated weights for policy 0, policy_version 629417 (0.0010) [2023-12-27 09:59:13,163][20381] Updated weights for policy 1, policy_version 629011 (0.0006) [2023-12-27 09:59:13,171][20347] Updated weights for policy 0, policy_version 629427 (0.0010) [2023-12-27 09:59:13,224][20381] Updated weights for policy 1, policy_version 629021 (0.0010) [2023-12-27 09:59:13,233][20347] Updated weights for policy 0, policy_version 629437 (0.0010) [2023-12-27 09:59:13,830][20347] Updated weights for policy 0, policy_version 629447 (0.0007) [2023-12-27 09:59:13,889][20347] Updated weights for policy 0, policy_version 629457 (0.0008) [2023-12-27 09:59:13,894][20381] Updated weights for policy 1, policy_version 629031 (0.0008) [2023-12-27 09:59:13,948][20381] Updated weights for policy 1, policy_version 629041 (0.0005) [2023-12-27 09:59:13,950][20347] Updated weights for policy 0, policy_version 629467 (0.0010) [2023-12-27 09:59:13,997][20381] Updated weights for policy 1, policy_version 629051 (0.0007) [2023-12-27 09:59:14,599][20347] Updated weights for policy 0, policy_version 629477 (0.0010) [2023-12-27 09:59:14,655][20347] Updated weights for policy 0, policy_version 629487 (0.0010) [2023-12-27 09:59:14,687][20381] Updated weights for policy 1, policy_version 629061 (0.0006) [2023-12-27 09:59:14,722][20347] Updated weights for policy 0, policy_version 629497 (0.0010) [2023-12-27 09:59:14,739][20381] Updated weights for policy 1, policy_version 629071 (0.0005) [2023-12-27 09:59:14,795][20381] Updated weights for policy 1, policy_version 629081 (0.0008) [2023-12-27 09:59:15,423][20381] Updated weights for policy 1, policy_version 629091 (0.0007) [2023-12-27 09:59:15,424][20347] Updated weights for policy 0, policy_version 629507 (0.0009) [2023-12-27 09:59:15,485][20347] Updated weights for policy 0, policy_version 629517 (0.0006) [2023-12-27 09:59:15,492][20381] Updated weights for policy 1, policy_version 629101 (0.0008) [2023-12-27 09:59:15,539][20347] Updated weights for policy 0, policy_version 629527 (0.0007) [2023-12-27 09:59:15,557][20381] Updated weights for policy 1, policy_version 629111 (0.0008) [2023-12-27 09:59:16,173][20381] Updated weights for policy 1, policy_version 629121 (0.0009) [2023-12-27 09:59:16,231][20381] Updated weights for policy 1, policy_version 629131 (0.0006) [2023-12-27 09:59:16,248][20347] Updated weights for policy 0, policy_version 629537 (0.0007) [2023-12-27 09:59:16,283][20381] Updated weights for policy 1, policy_version 629141 (0.0010) [2023-12-27 09:59:16,300][20347] Updated weights for policy 0, policy_version 629547 (0.0005) [2023-12-27 09:59:16,331][20381] Updated weights for policy 1, policy_version 629151 (0.0010) [2023-12-27 09:59:16,344][20347] Updated weights for policy 0, policy_version 629557 (0.0005) [2023-12-27 09:59:16,398][20347] Updated weights for policy 0, policy_version 629567 (0.0008) [2023-12-27 09:59:16,994][20381] Updated weights for policy 1, policy_version 629161 (0.0010) [2023-12-27 09:59:17,045][20381] Updated weights for policy 1, policy_version 629171 (0.0010) [2023-12-27 09:59:17,087][20347] Updated weights for policy 0, policy_version 629577 (0.0006) [2023-12-27 09:59:17,093][20381] Updated weights for policy 1, policy_version 629181 (0.0010) [2023-12-27 09:59:17,147][20347] Updated weights for policy 0, policy_version 629587 (0.0007) [2023-12-27 09:59:17,167][20291] KL-divergence is very high: 2022.4661 [2023-12-27 09:59:17,211][20347] Updated weights for policy 0, policy_version 629597 (0.0007) [2023-12-27 09:59:17,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 322289664. Throughput: 0: 10573.2, 1: 10227.8. Samples: 322295688. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:17,364][19170] Avg episode reward: [(0, '980.920'), (1, '249.650')] [2023-12-27 09:59:17,744][20381] Updated weights for policy 1, policy_version 629191 (0.0008) [2023-12-27 09:59:17,798][20381] Updated weights for policy 1, policy_version 629201 (0.0008) [2023-12-27 09:59:17,855][20381] Updated weights for policy 1, policy_version 629211 (0.0007) [2023-12-27 09:59:17,877][20347] Updated weights for policy 0, policy_version 629607 (0.0006) [2023-12-27 09:59:17,933][20347] Updated weights for policy 0, policy_version 629617 (0.0005) [2023-12-27 09:59:17,999][20347] Updated weights for policy 0, policy_version 629627 (0.0005) [2023-12-27 09:59:18,522][20381] Updated weights for policy 1, policy_version 629221 (0.0008) [2023-12-27 09:59:18,580][20381] Updated weights for policy 1, policy_version 629231 (0.0010) [2023-12-27 09:59:18,595][20347] Updated weights for policy 0, policy_version 629637 (0.0008) [2023-12-27 09:59:18,638][20381] Updated weights for policy 1, policy_version 629241 (0.0010) [2023-12-27 09:59:18,651][20347] Updated weights for policy 0, policy_version 629647 (0.0010) [2023-12-27 09:59:18,698][20347] Updated weights for policy 0, policy_version 629657 (0.0007) [2023-12-27 09:59:19,317][20381] Updated weights for policy 1, policy_version 629251 (0.0009) [2023-12-27 09:59:19,380][20381] Updated weights for policy 1, policy_version 629261 (0.0010) [2023-12-27 09:59:19,402][20347] Updated weights for policy 0, policy_version 629667 (0.0008) [2023-12-27 09:59:19,433][20381] Updated weights for policy 1, policy_version 629271 (0.0011) [2023-12-27 09:59:19,459][20347] Updated weights for policy 0, policy_version 629677 (0.0010) [2023-12-27 09:59:19,521][20347] Updated weights for policy 0, policy_version 629687 (0.0010) [2023-12-27 09:59:20,131][20381] Updated weights for policy 1, policy_version 629281 (0.0011) [2023-12-27 09:59:20,197][20381] Updated weights for policy 1, policy_version 629291 (0.0011) [2023-12-27 09:59:20,236][20347] Updated weights for policy 0, policy_version 629697 (0.0010) [2023-12-27 09:59:20,256][20381] Updated weights for policy 1, policy_version 629301 (0.0010) [2023-12-27 09:59:20,296][20347] Updated weights for policy 0, policy_version 629707 (0.0010) [2023-12-27 09:59:20,316][20381] Updated weights for policy 1, policy_version 629311 (0.0011) [2023-12-27 09:59:20,356][20347] Updated weights for policy 0, policy_version 629717 (0.0011) [2023-12-27 09:59:20,412][20347] Updated weights for policy 0, policy_version 629727 (0.0010) [2023-12-27 09:59:20,984][20381] Updated weights for policy 1, policy_version 629321 (0.0009) [2023-12-27 09:59:21,042][20381] Updated weights for policy 1, policy_version 629331 (0.0010) [2023-12-27 09:59:21,098][20381] Updated weights for policy 1, policy_version 629341 (0.0009) [2023-12-27 09:59:21,113][20347] Updated weights for policy 0, policy_version 629737 (0.0011) [2023-12-27 09:59:21,178][20347] Updated weights for policy 0, policy_version 629747 (0.0009) [2023-12-27 09:59:21,236][20347] Updated weights for policy 0, policy_version 629757 (0.0009) [2023-12-27 09:59:21,790][20381] Updated weights for policy 1, policy_version 629351 (0.0009) [2023-12-27 09:59:21,838][20381] Updated weights for policy 1, policy_version 629361 (0.0008) [2023-12-27 09:59:21,894][20381] Updated weights for policy 1, policy_version 629371 (0.0008) [2023-12-27 09:59:21,939][20347] Updated weights for policy 0, policy_version 629767 (0.0010) [2023-12-27 09:59:22,002][20347] Updated weights for policy 0, policy_version 629777 (0.0011) [2023-12-27 09:59:22,059][20347] Updated weights for policy 0, policy_version 629787 (0.0011) [2023-12-27 09:59:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 322387968. Throughput: 0: 10574.1, 1: 10179.5. Samples: 322357548. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:22,363][19170] Avg episode reward: [(0, '980.920'), (1, '280.960')] [2023-12-27 09:59:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000629376_161144832.pth... [2023-12-27 09:59:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000629792_161243136.pth... [2023-12-27 09:59:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000628192_160841728.pth [2023-12-27 09:59:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000628576_160931840.pth [2023-12-27 09:59:22,652][20381] Updated weights for policy 1, policy_version 629381 (0.0009) [2023-12-27 09:59:22,706][20381] Updated weights for policy 1, policy_version 629391 (0.0011) [2023-12-27 09:59:22,746][20347] Updated weights for policy 0, policy_version 629797 (0.0008) [2023-12-27 09:59:22,757][20381] Updated weights for policy 1, policy_version 629401 (0.0011) [2023-12-27 09:59:22,806][20347] Updated weights for policy 0, policy_version 629807 (0.0006) [2023-12-27 09:59:22,860][20347] Updated weights for policy 0, policy_version 629817 (0.0007) [2023-12-27 09:59:23,353][20381] Updated weights for policy 1, policy_version 629411 (0.0009) [2023-12-27 09:59:23,414][20381] Updated weights for policy 1, policy_version 629421 (0.0010) [2023-12-27 09:59:23,473][20381] Updated weights for policy 1, policy_version 629431 (0.0008) [2023-12-27 09:59:23,540][20347] Updated weights for policy 0, policy_version 629827 (0.0008) [2023-12-27 09:59:23,593][20347] Updated weights for policy 0, policy_version 629837 (0.0007) [2023-12-27 09:59:23,638][20347] Updated weights for policy 0, policy_version 629847 (0.0010) [2023-12-27 09:59:23,997][20381] Updated weights for policy 1, policy_version 629441 (0.0005) [2023-12-27 09:59:24,043][20381] Updated weights for policy 1, policy_version 629451 (0.0005) [2023-12-27 09:59:24,088][20381] Updated weights for policy 1, policy_version 629461 (0.0005) [2023-12-27 09:59:24,154][20381] Updated weights for policy 1, policy_version 629471 (0.0005) [2023-12-27 09:59:24,323][20347] Updated weights for policy 0, policy_version 629857 (0.0007) [2023-12-27 09:59:24,384][20347] Updated weights for policy 0, policy_version 629867 (0.0008) [2023-12-27 09:59:24,448][20347] Updated weights for policy 0, policy_version 629877 (0.0008) [2023-12-27 09:59:24,512][20347] Updated weights for policy 0, policy_version 629887 (0.0008) [2023-12-27 09:59:24,676][20381] Updated weights for policy 1, policy_version 629481 (0.0006) [2023-12-27 09:59:24,734][20381] Updated weights for policy 1, policy_version 629491 (0.0006) [2023-12-27 09:59:24,793][20381] Updated weights for policy 1, policy_version 629501 (0.0010) [2023-12-27 09:59:25,177][20347] Updated weights for policy 0, policy_version 629897 (0.0009) [2023-12-27 09:59:25,226][20347] Updated weights for policy 0, policy_version 629907 (0.0010) [2023-12-27 09:59:25,273][20347] Updated weights for policy 0, policy_version 629917 (0.0008) [2023-12-27 09:59:25,410][20381] Updated weights for policy 1, policy_version 629511 (0.0010) [2023-12-27 09:59:25,465][20381] Updated weights for policy 1, policy_version 629521 (0.0011) [2023-12-27 09:59:25,510][20381] Updated weights for policy 1, policy_version 629531 (0.0010) [2023-12-27 09:59:25,874][20347] Updated weights for policy 0, policy_version 629927 (0.0005) [2023-12-27 09:59:25,937][20347] Updated weights for policy 0, policy_version 629937 (0.0006) [2023-12-27 09:59:25,991][20347] Updated weights for policy 0, policy_version 629947 (0.0010) [2023-12-27 09:59:26,211][20381] Updated weights for policy 1, policy_version 629541 (0.0009) [2023-12-27 09:59:26,262][20381] Updated weights for policy 1, policy_version 629551 (0.0009) [2023-12-27 09:59:26,311][20381] Updated weights for policy 1, policy_version 629561 (0.0007) [2023-12-27 09:59:26,563][20347] Updated weights for policy 0, policy_version 629957 (0.0007) [2023-12-27 09:59:26,618][20347] Updated weights for policy 0, policy_version 629967 (0.0007) [2023-12-27 09:59:26,665][20347] Updated weights for policy 0, policy_version 629977 (0.0010) [2023-12-27 09:59:26,994][20381] Updated weights for policy 1, policy_version 629571 (0.0009) [2023-12-27 09:59:27,038][20381] Updated weights for policy 1, policy_version 629581 (0.0006) [2023-12-27 09:59:27,081][20381] Updated weights for policy 1, policy_version 629591 (0.0005) [2023-12-27 09:59:27,204][20347] Updated weights for policy 0, policy_version 629987 (0.0009) [2023-12-27 09:59:27,265][20347] Updated weights for policy 0, policy_version 629997 (0.0006) [2023-12-27 09:59:27,318][20347] Updated weights for policy 0, policy_version 630007 (0.0009) [2023-12-27 09:59:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 322494464. Throughput: 0: 10550.8, 1: 10271.0. Samples: 322483892. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:27,363][19170] Avg episode reward: [(0, '952.000'), (1, '276.030')] [2023-12-27 09:59:27,686][20381] Updated weights for policy 1, policy_version 629601 (0.0006) [2023-12-27 09:59:27,734][20381] Updated weights for policy 1, policy_version 629611 (0.0010) [2023-12-27 09:59:27,779][20381] Updated weights for policy 1, policy_version 629621 (0.0010) [2023-12-27 09:59:27,826][20381] Updated weights for policy 1, policy_version 629631 (0.0010) [2023-12-27 09:59:27,949][20347] Updated weights for policy 0, policy_version 630017 (0.0006) [2023-12-27 09:59:28,003][20347] Updated weights for policy 0, policy_version 630027 (0.0007) [2023-12-27 09:59:28,051][20347] Updated weights for policy 0, policy_version 630037 (0.0008) [2023-12-27 09:59:28,107][20347] Updated weights for policy 0, policy_version 630047 (0.0008) [2023-12-27 09:59:28,503][20381] Updated weights for policy 1, policy_version 629641 (0.0010) [2023-12-27 09:59:28,558][20381] Updated weights for policy 1, policy_version 629651 (0.0007) [2023-12-27 09:59:28,613][20381] Updated weights for policy 1, policy_version 629661 (0.0008) [2023-12-27 09:59:28,786][20347] Updated weights for policy 0, policy_version 630057 (0.0006) [2023-12-27 09:59:28,843][20347] Updated weights for policy 0, policy_version 630067 (0.0006) [2023-12-27 09:59:28,897][20347] Updated weights for policy 0, policy_version 630077 (0.0006) [2023-12-27 09:59:29,320][20381] Updated weights for policy 1, policy_version 629671 (0.0008) [2023-12-27 09:59:29,379][20381] Updated weights for policy 1, policy_version 629681 (0.0007) [2023-12-27 09:59:29,438][20381] Updated weights for policy 1, policy_version 629691 (0.0007) [2023-12-27 09:59:29,527][20347] Updated weights for policy 0, policy_version 630087 (0.0006) [2023-12-27 09:59:29,591][20347] Updated weights for policy 0, policy_version 630097 (0.0006) [2023-12-27 09:59:29,646][20347] Updated weights for policy 0, policy_version 630107 (0.0007) [2023-12-27 09:59:30,067][20381] Updated weights for policy 1, policy_version 629701 (0.0006) [2023-12-27 09:59:30,129][20381] Updated weights for policy 1, policy_version 629711 (0.0007) [2023-12-27 09:59:30,178][20381] Updated weights for policy 1, policy_version 629721 (0.0007) [2023-12-27 09:59:30,298][20347] Updated weights for policy 0, policy_version 630117 (0.0008) [2023-12-27 09:59:30,347][20347] Updated weights for policy 0, policy_version 630127 (0.0008) [2023-12-27 09:59:30,399][20347] Updated weights for policy 0, policy_version 630137 (0.0008) [2023-12-27 09:59:30,878][20381] Updated weights for policy 1, policy_version 629731 (0.0007) [2023-12-27 09:59:30,935][20381] Updated weights for policy 1, policy_version 629741 (0.0007) [2023-12-27 09:59:30,988][20381] Updated weights for policy 1, policy_version 629751 (0.0007) [2023-12-27 09:59:31,082][20347] Updated weights for policy 0, policy_version 630147 (0.0007) [2023-12-27 09:59:31,144][20347] Updated weights for policy 0, policy_version 630157 (0.0008) [2023-12-27 09:59:31,206][20347] Updated weights for policy 0, policy_version 630167 (0.0008) [2023-12-27 09:59:31,658][20381] Updated weights for policy 1, policy_version 629761 (0.0007) [2023-12-27 09:59:31,716][20381] Updated weights for policy 1, policy_version 629771 (0.0008) [2023-12-27 09:59:31,773][20381] Updated weights for policy 1, policy_version 629781 (0.0008) [2023-12-27 09:59:31,826][20381] Updated weights for policy 1, policy_version 629791 (0.0007) [2023-12-27 09:59:31,883][20347] Updated weights for policy 0, policy_version 630177 (0.0008) [2023-12-27 09:59:31,938][20347] Updated weights for policy 0, policy_version 630187 (0.0007) [2023-12-27 09:59:31,994][20347] Updated weights for policy 0, policy_version 630197 (0.0005) [2023-12-27 09:59:32,047][20347] Updated weights for policy 0, policy_version 630207 (0.0007) [2023-12-27 09:59:32,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 322600960. Throughput: 0: 10570.0, 1: 10332.9. Samples: 322611896. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:32,363][19170] Avg episode reward: [(0, '845.470'), (1, '266.750')] [2023-12-27 09:59:32,473][20381] Updated weights for policy 1, policy_version 629801 (0.0006) [2023-12-27 09:59:32,526][20381] Updated weights for policy 1, policy_version 629811 (0.0006) [2023-12-27 09:59:32,575][20381] Updated weights for policy 1, policy_version 629821 (0.0008) [2023-12-27 09:59:32,675][20347] Updated weights for policy 0, policy_version 630217 (0.0007) [2023-12-27 09:59:32,731][20347] Updated weights for policy 0, policy_version 630227 (0.0008) [2023-12-27 09:59:32,786][20347] Updated weights for policy 0, policy_version 630237 (0.0008) [2023-12-27 09:59:33,213][20381] Updated weights for policy 1, policy_version 629831 (0.0008) [2023-12-27 09:59:33,274][20381] Updated weights for policy 1, policy_version 629841 (0.0007) [2023-12-27 09:59:33,322][20381] Updated weights for policy 1, policy_version 629851 (0.0008) [2023-12-27 09:59:33,467][20347] Updated weights for policy 0, policy_version 630247 (0.0007) [2023-12-27 09:59:33,521][20347] Updated weights for policy 0, policy_version 630257 (0.0005) [2023-12-27 09:59:33,577][20347] Updated weights for policy 0, policy_version 630267 (0.0005) [2023-12-27 09:59:34,050][20381] Updated weights for policy 1, policy_version 629861 (0.0009) [2023-12-27 09:59:34,100][20381] Updated weights for policy 1, policy_version 629871 (0.0008) [2023-12-27 09:59:34,102][20347] Updated weights for policy 0, policy_version 630277 (0.0005) [2023-12-27 09:59:34,155][20381] Updated weights for policy 1, policy_version 629881 (0.0006) [2023-12-27 09:59:34,158][20347] Updated weights for policy 0, policy_version 630287 (0.0009) [2023-12-27 09:59:34,215][20347] Updated weights for policy 0, policy_version 630297 (0.0008) [2023-12-27 09:59:34,818][20347] Updated weights for policy 0, policy_version 630307 (0.0006) [2023-12-27 09:59:34,875][20347] Updated weights for policy 0, policy_version 630317 (0.0005) [2023-12-27 09:59:34,918][20381] Updated weights for policy 1, policy_version 629891 (0.0007) [2023-12-27 09:59:34,936][20347] Updated weights for policy 0, policy_version 630327 (0.0006) [2023-12-27 09:59:34,978][20381] Updated weights for policy 1, policy_version 629901 (0.0007) [2023-12-27 09:59:35,039][20381] Updated weights for policy 1, policy_version 629911 (0.0007) [2023-12-27 09:59:35,527][20347] Updated weights for policy 0, policy_version 630337 (0.0006) [2023-12-27 09:59:35,575][20347] Updated weights for policy 0, policy_version 630347 (0.0009) [2023-12-27 09:59:35,628][20347] Updated weights for policy 0, policy_version 630357 (0.0005) [2023-12-27 09:59:35,687][20347] Updated weights for policy 0, policy_version 630367 (0.0005) [2023-12-27 09:59:35,753][20381] Updated weights for policy 1, policy_version 629921 (0.0008) [2023-12-27 09:59:35,810][20381] Updated weights for policy 1, policy_version 629931 (0.0007) [2023-12-27 09:59:35,861][20381] Updated weights for policy 1, policy_version 629941 (0.0007) [2023-12-27 09:59:35,912][20381] Updated weights for policy 1, policy_version 629951 (0.0007) [2023-12-27 09:59:36,360][20347] Updated weights for policy 0, policy_version 630377 (0.0010) [2023-12-27 09:59:36,417][20347] Updated weights for policy 0, policy_version 630387 (0.0011) [2023-12-27 09:59:36,477][20347] Updated weights for policy 0, policy_version 630397 (0.0010) [2023-12-27 09:59:36,607][20381] Updated weights for policy 1, policy_version 629961 (0.0010) [2023-12-27 09:59:36,656][20381] Updated weights for policy 1, policy_version 629971 (0.0010) [2023-12-27 09:59:36,717][20381] Updated weights for policy 1, policy_version 629981 (0.0011) [2023-12-27 09:59:37,167][20347] Updated weights for policy 0, policy_version 630407 (0.0010) [2023-12-27 09:59:37,229][20347] Updated weights for policy 0, policy_version 630417 (0.0010) [2023-12-27 09:59:37,288][20347] Updated weights for policy 0, policy_version 630427 (0.0010) [2023-12-27 09:59:37,333][20381] Updated weights for policy 1, policy_version 629991 (0.0007) [2023-12-27 09:59:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.8, 300 sec: 20854.9). Total num frames: 322707456. Throughput: 0: 10593.1, 1: 10349.2. Samples: 322675048. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:37,363][19170] Avg episode reward: [(0, '874.480'), (1, '310.860')] [2023-12-27 09:59:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000630432_161406976.pth... [2023-12-27 09:59:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000629184_161087488.pth [2023-12-27 09:59:37,394][20381] Updated weights for policy 1, policy_version 630001 (0.0009) [2023-12-27 09:59:37,453][20381] Updated weights for policy 1, policy_version 630011 (0.0011) [2023-12-27 09:59:37,479][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000630016_161308672.pth... [2023-12-27 09:59:37,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000628800_160997376.pth [2023-12-27 09:59:37,968][20347] Updated weights for policy 0, policy_version 630437 (0.0010) [2023-12-27 09:59:38,022][20347] Updated weights for policy 0, policy_version 630447 (0.0010) [2023-12-27 09:59:38,078][20347] Updated weights for policy 0, policy_version 630457 (0.0010) [2023-12-27 09:59:38,085][20381] Updated weights for policy 1, policy_version 630021 (0.0011) [2023-12-27 09:59:38,133][20381] Updated weights for policy 1, policy_version 630031 (0.0010) [2023-12-27 09:59:38,190][20381] Updated weights for policy 1, policy_version 630041 (0.0009) [2023-12-27 09:59:38,772][20347] Updated weights for policy 0, policy_version 630467 (0.0010) [2023-12-27 09:59:38,807][20381] Updated weights for policy 1, policy_version 630051 (0.0005) [2023-12-27 09:59:38,825][20347] Updated weights for policy 0, policy_version 630477 (0.0011) [2023-12-27 09:59:38,864][20381] Updated weights for policy 1, policy_version 630061 (0.0010) [2023-12-27 09:59:38,881][20347] Updated weights for policy 0, policy_version 630487 (0.0010) [2023-12-27 09:59:38,922][20381] Updated weights for policy 1, policy_version 630071 (0.0008) [2023-12-27 09:59:39,537][20347] Updated weights for policy 0, policy_version 630497 (0.0010) [2023-12-27 09:59:39,594][20347] Updated weights for policy 0, policy_version 630507 (0.0007) [2023-12-27 09:59:39,650][20381] Updated weights for policy 1, policy_version 630081 (0.0007) [2023-12-27 09:59:39,654][20347] Updated weights for policy 0, policy_version 630517 (0.0007) [2023-12-27 09:59:39,710][20381] Updated weights for policy 1, policy_version 630091 (0.0011) [2023-12-27 09:59:39,718][20347] Updated weights for policy 0, policy_version 630527 (0.0007) [2023-12-27 09:59:39,769][20381] Updated weights for policy 1, policy_version 630101 (0.0011) [2023-12-27 09:59:39,826][20381] Updated weights for policy 1, policy_version 630111 (0.0011) [2023-12-27 09:59:40,369][20347] Updated weights for policy 0, policy_version 630537 (0.0007) [2023-12-27 09:59:40,421][20347] Updated weights for policy 0, policy_version 630547 (0.0007) [2023-12-27 09:59:40,471][20347] Updated weights for policy 0, policy_version 630557 (0.0007) [2023-12-27 09:59:40,504][20381] Updated weights for policy 1, policy_version 630121 (0.0011) [2023-12-27 09:59:40,559][20381] Updated weights for policy 1, policy_version 630131 (0.0011) [2023-12-27 09:59:40,611][20381] Updated weights for policy 1, policy_version 630141 (0.0011) [2023-12-27 09:59:41,090][20347] Updated weights for policy 0, policy_version 630567 (0.0006) [2023-12-27 09:59:41,152][20347] Updated weights for policy 0, policy_version 630577 (0.0007) [2023-12-27 09:59:41,208][20347] Updated weights for policy 0, policy_version 630587 (0.0007) [2023-12-27 09:59:41,334][20381] Updated weights for policy 1, policy_version 630151 (0.0007) [2023-12-27 09:59:41,392][20381] Updated weights for policy 1, policy_version 630161 (0.0009) [2023-12-27 09:59:41,444][20381] Updated weights for policy 1, policy_version 630171 (0.0008) [2023-12-27 09:59:41,932][20347] Updated weights for policy 0, policy_version 630597 (0.0008) [2023-12-27 09:59:41,983][20347] Updated weights for policy 0, policy_version 630607 (0.0010) [2023-12-27 09:59:42,032][20347] Updated weights for policy 0, policy_version 630617 (0.0010) [2023-12-27 09:59:42,207][20381] Updated weights for policy 1, policy_version 630181 (0.0008) [2023-12-27 09:59:42,263][20381] Updated weights for policy 1, policy_version 630191 (0.0008) [2023-12-27 09:59:42,319][20381] Updated weights for policy 1, policy_version 630201 (0.0009) [2023-12-27 09:59:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 322805760. Throughput: 0: 10505.7, 1: 10402.1. Samples: 322799872. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:42,363][19170] Avg episode reward: [(0, '836.170'), (1, '293.610')] [2023-12-27 09:59:42,686][20347] Updated weights for policy 0, policy_version 630627 (0.0010) [2023-12-27 09:59:42,738][20347] Updated weights for policy 0, policy_version 630637 (0.0010) [2023-12-27 09:59:42,800][20347] Updated weights for policy 0, policy_version 630647 (0.0010) [2023-12-27 09:59:43,020][20381] Updated weights for policy 1, policy_version 630211 (0.0008) [2023-12-27 09:59:43,076][20381] Updated weights for policy 1, policy_version 630221 (0.0011) [2023-12-27 09:59:43,127][20381] Updated weights for policy 1, policy_version 630231 (0.0007) [2023-12-27 09:59:43,391][20347] Updated weights for policy 0, policy_version 630657 (0.0010) [2023-12-27 09:59:43,453][20347] Updated weights for policy 0, policy_version 630667 (0.0005) [2023-12-27 09:59:43,512][20347] Updated weights for policy 0, policy_version 630677 (0.0009) [2023-12-27 09:59:43,565][20347] Updated weights for policy 0, policy_version 630687 (0.0010) [2023-12-27 09:59:43,801][20381] Updated weights for policy 1, policy_version 630241 (0.0006) [2023-12-27 09:59:43,858][20381] Updated weights for policy 1, policy_version 630251 (0.0009) [2023-12-27 09:59:43,912][20381] Updated weights for policy 1, policy_version 630261 (0.0008) [2023-12-27 09:59:43,969][20381] Updated weights for policy 1, policy_version 630271 (0.0005) [2023-12-27 09:59:44,138][20347] Updated weights for policy 0, policy_version 630697 (0.0010) [2023-12-27 09:59:44,195][20347] Updated weights for policy 0, policy_version 630707 (0.0010) [2023-12-27 09:59:44,247][20347] Updated weights for policy 0, policy_version 630717 (0.0007) [2023-12-27 09:59:44,639][20381] Updated weights for policy 1, policy_version 630281 (0.0007) [2023-12-27 09:59:44,699][20381] Updated weights for policy 1, policy_version 630291 (0.0007) [2023-12-27 09:59:44,758][20381] Updated weights for policy 1, policy_version 630301 (0.0007) [2023-12-27 09:59:44,920][20347] Updated weights for policy 0, policy_version 630727 (0.0007) [2023-12-27 09:59:44,975][20347] Updated weights for policy 0, policy_version 630737 (0.0008) [2023-12-27 09:59:45,033][20347] Updated weights for policy 0, policy_version 630747 (0.0008) [2023-12-27 09:59:45,385][20381] Updated weights for policy 1, policy_version 630311 (0.0008) [2023-12-27 09:59:45,444][20381] Updated weights for policy 1, policy_version 630321 (0.0007) [2023-12-27 09:59:45,504][20381] Updated weights for policy 1, policy_version 630331 (0.0008) [2023-12-27 09:59:45,717][20347] Updated weights for policy 0, policy_version 630757 (0.0007) [2023-12-27 09:59:45,773][20347] Updated weights for policy 0, policy_version 630767 (0.0005) [2023-12-27 09:59:45,829][20347] Updated weights for policy 0, policy_version 630777 (0.0005) [2023-12-27 09:59:46,201][20381] Updated weights for policy 1, policy_version 630341 (0.0007) [2023-12-27 09:59:46,262][20381] Updated weights for policy 1, policy_version 630351 (0.0005) [2023-12-27 09:59:46,326][20381] Updated weights for policy 1, policy_version 630361 (0.0005) [2023-12-27 09:59:46,460][20347] Updated weights for policy 0, policy_version 630787 (0.0007) [2023-12-27 09:59:46,521][20347] Updated weights for policy 0, policy_version 630797 (0.0010) [2023-12-27 09:59:46,582][20347] Updated weights for policy 0, policy_version 630807 (0.0009) [2023-12-27 09:59:46,847][20381] Updated weights for policy 1, policy_version 630371 (0.0006) [2023-12-27 09:59:46,899][20381] Updated weights for policy 1, policy_version 630381 (0.0008) [2023-12-27 09:59:46,952][20381] Updated weights for policy 1, policy_version 630391 (0.0007) [2023-12-27 09:59:47,207][20347] Updated weights for policy 0, policy_version 630817 (0.0008) [2023-12-27 09:59:47,264][20347] Updated weights for policy 0, policy_version 630827 (0.0010) [2023-12-27 09:59:47,321][20347] Updated weights for policy 0, policy_version 630837 (0.0010) [2023-12-27 09:59:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 322912256. Throughput: 0: 10495.4, 1: 10341.0. Samples: 322925164. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:47,363][19170] Avg episode reward: [(0, '830.140'), (1, '260.020')] [2023-12-27 09:59:47,375][20347] Updated weights for policy 0, policy_version 630847 (0.0010) [2023-12-27 09:59:47,649][20381] Updated weights for policy 1, policy_version 630401 (0.0008) [2023-12-27 09:59:47,693][20381] Updated weights for policy 1, policy_version 630411 (0.0006) [2023-12-27 09:59:47,758][20381] Updated weights for policy 1, policy_version 630421 (0.0005) [2023-12-27 09:59:47,813][20381] Updated weights for policy 1, policy_version 630431 (0.0005) [2023-12-27 09:59:48,050][20347] Updated weights for policy 0, policy_version 630857 (0.0006) [2023-12-27 09:59:48,102][20347] Updated weights for policy 0, policy_version 630867 (0.0009) [2023-12-27 09:59:48,162][20347] Updated weights for policy 0, policy_version 630877 (0.0010) [2023-12-27 09:59:48,485][20381] Updated weights for policy 1, policy_version 630441 (0.0009) [2023-12-27 09:59:48,535][20381] Updated weights for policy 1, policy_version 630451 (0.0009) [2023-12-27 09:59:48,583][20381] Updated weights for policy 1, policy_version 630462 (0.0009) [2023-12-27 09:59:48,857][20347] Updated weights for policy 0, policy_version 630887 (0.0010) [2023-12-27 09:59:48,918][20347] Updated weights for policy 0, policy_version 630897 (0.0009) [2023-12-27 09:59:48,967][20347] Updated weights for policy 0, policy_version 630907 (0.0010) [2023-12-27 09:59:49,373][20381] Updated weights for policy 1, policy_version 630472 (0.0008) [2023-12-27 09:59:49,433][20381] Updated weights for policy 1, policy_version 630482 (0.0007) [2023-12-27 09:59:49,485][20381] Updated weights for policy 1, policy_version 630492 (0.0006) [2023-12-27 09:59:49,643][20347] Updated weights for policy 0, policy_version 630917 (0.0010) [2023-12-27 09:59:49,705][20347] Updated weights for policy 0, policy_version 630927 (0.0009) [2023-12-27 09:59:49,725][20291] KL-divergence is very high: 106.3334 [2023-12-27 09:59:49,768][20347] Updated weights for policy 0, policy_version 630937 (0.0010) [2023-12-27 09:59:49,776][20291] KL-divergence is very high: 137.6264 [2023-12-27 09:59:49,793][20291] KL-divergence is very high: 106.9530 [2023-12-27 09:59:50,190][20381] Updated weights for policy 1, policy_version 630502 (0.0007) [2023-12-27 09:59:50,245][20381] Updated weights for policy 1, policy_version 630512 (0.0009) [2023-12-27 09:59:50,294][20381] Updated weights for policy 1, policy_version 630522 (0.0010) [2023-12-27 09:59:50,487][20347] Updated weights for policy 0, policy_version 630947 (0.0009) [2023-12-27 09:59:50,551][20347] Updated weights for policy 0, policy_version 630957 (0.0009) [2023-12-27 09:59:50,614][20347] Updated weights for policy 0, policy_version 630967 (0.0006) [2023-12-27 09:59:50,916][20381] Updated weights for policy 1, policy_version 630532 (0.0008) [2023-12-27 09:59:50,968][20381] Updated weights for policy 1, policy_version 630542 (0.0006) [2023-12-27 09:59:51,023][20381] Updated weights for policy 1, policy_version 630552 (0.0006) [2023-12-27 09:59:51,232][20347] Updated weights for policy 0, policy_version 630977 (0.0006) [2023-12-27 09:59:51,296][20347] Updated weights for policy 0, policy_version 630987 (0.0007) [2023-12-27 09:59:51,356][20347] Updated weights for policy 0, policy_version 630997 (0.0007) [2023-12-27 09:59:51,405][20291] KL-divergence is very high: 152.2440 [2023-12-27 09:59:51,423][20347] Updated weights for policy 0, policy_version 631007 (0.0007) [2023-12-27 09:59:51,636][20381] Updated weights for policy 1, policy_version 630562 (0.0007) [2023-12-27 09:59:51,699][20381] Updated weights for policy 1, policy_version 630572 (0.0008) [2023-12-27 09:59:51,761][20381] Updated weights for policy 1, policy_version 630582 (0.0006) [2023-12-27 09:59:51,823][20381] Updated weights for policy 1, policy_version 630592 (0.0009) [2023-12-27 09:59:52,143][20347] Updated weights for policy 0, policy_version 631017 (0.0008) [2023-12-27 09:59:52,196][20347] Updated weights for policy 0, policy_version 631027 (0.0008) [2023-12-27 09:59:52,255][20347] Updated weights for policy 0, policy_version 631037 (0.0007) [2023-12-27 09:59:52,363][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 323018752. Throughput: 0: 10483.5, 1: 10349.9. Samples: 322985448. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:52,363][19170] Avg episode reward: [(0, '700.750'), (1, '222.830')] [2023-12-27 09:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000630592_161456128.pth... [2023-12-27 09:59:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000631040_161562624.pth... [2023-12-27 09:59:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000629376_161144832.pth [2023-12-27 09:59:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000630592_161456128.pth [2023-12-27 09:59:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000629792_161243136.pth [2023-12-27 09:59:52,376][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000631040_161562624.pth [2023-12-27 09:59:52,447][20381] Updated weights for policy 1, policy_version 630602 (0.0008) [2023-12-27 09:59:52,500][20381] Updated weights for policy 1, policy_version 630612 (0.0007) [2023-12-27 09:59:52,556][20381] Updated weights for policy 1, policy_version 630622 (0.0008) [2023-12-27 09:59:52,958][20347] Updated weights for policy 0, policy_version 631047 (0.0007) [2023-12-27 09:59:53,012][20347] Updated weights for policy 0, policy_version 631057 (0.0006) [2023-12-27 09:59:53,066][20347] Updated weights for policy 0, policy_version 631067 (0.0007) [2023-12-27 09:59:53,263][20381] Updated weights for policy 1, policy_version 630632 (0.0009) [2023-12-27 09:59:53,310][20381] Updated weights for policy 1, policy_version 630642 (0.0009) [2023-12-27 09:59:53,364][20381] Updated weights for policy 1, policy_version 630652 (0.0010) [2023-12-27 09:59:53,709][20347] Updated weights for policy 0, policy_version 631077 (0.0008) [2023-12-27 09:59:53,761][20347] Updated weights for policy 0, policy_version 631087 (0.0007) [2023-12-27 09:59:53,815][20347] Updated weights for policy 0, policy_version 631097 (0.0007) [2023-12-27 09:59:54,055][20381] Updated weights for policy 1, policy_version 630662 (0.0010) [2023-12-27 09:59:54,103][20381] Updated weights for policy 1, policy_version 630672 (0.0010) [2023-12-27 09:59:54,160][20381] Updated weights for policy 1, policy_version 630682 (0.0010) [2023-12-27 09:59:54,499][20347] Updated weights for policy 0, policy_version 631107 (0.0007) [2023-12-27 09:59:54,555][20347] Updated weights for policy 0, policy_version 631117 (0.0006) [2023-12-27 09:59:54,614][20347] Updated weights for policy 0, policy_version 631127 (0.0011) [2023-12-27 09:59:54,771][20381] Updated weights for policy 1, policy_version 630692 (0.0010) [2023-12-27 09:59:54,816][20381] Updated weights for policy 1, policy_version 630702 (0.0010) [2023-12-27 09:59:54,868][20381] Updated weights for policy 1, policy_version 630712 (0.0010) [2023-12-27 09:59:55,265][20347] Updated weights for policy 0, policy_version 631137 (0.0007) [2023-12-27 09:59:55,321][20347] Updated weights for policy 0, policy_version 631147 (0.0005) [2023-12-27 09:59:55,374][20347] Updated weights for policy 0, policy_version 631158 (0.0010) [2023-12-27 09:59:55,421][20347] Updated weights for policy 0, policy_version 631168 (0.0010) [2023-12-27 09:59:55,516][20381] Updated weights for policy 1, policy_version 630722 (0.0009) [2023-12-27 09:59:55,582][20381] Updated weights for policy 1, policy_version 630732 (0.0006) [2023-12-27 09:59:55,635][20381] Updated weights for policy 1, policy_version 630742 (0.0007) [2023-12-27 09:59:55,689][20381] Updated weights for policy 1, policy_version 630752 (0.0006) [2023-12-27 09:59:56,005][20347] Updated weights for policy 0, policy_version 631178 (0.0010) [2023-12-27 09:59:56,064][20347] Updated weights for policy 0, policy_version 631188 (0.0010) [2023-12-27 09:59:56,127][20347] Updated weights for policy 0, policy_version 631198 (0.0009) [2023-12-27 09:59:56,317][20381] Updated weights for policy 1, policy_version 630762 (0.0008) [2023-12-27 09:59:56,366][20381] Updated weights for policy 1, policy_version 630772 (0.0010) [2023-12-27 09:59:56,421][20381] Updated weights for policy 1, policy_version 630782 (0.0008) [2023-12-27 09:59:56,723][20347] Updated weights for policy 0, policy_version 631208 (0.0009) [2023-12-27 09:59:56,774][20347] Updated weights for policy 0, policy_version 631218 (0.0010) [2023-12-27 09:59:56,828][20347] Updated weights for policy 0, policy_version 631228 (0.0010) [2023-12-27 09:59:57,118][20381] Updated weights for policy 1, policy_version 630792 (0.0007) [2023-12-27 09:59:57,173][20381] Updated weights for policy 1, policy_version 630802 (0.0007) [2023-12-27 09:59:57,224][20381] Updated weights for policy 1, policy_version 630812 (0.0007) [2023-12-27 09:59:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.7, 300 sec: 20854.9). Total num frames: 323125248. Throughput: 0: 10476.7, 1: 10411.5. Samples: 323112420. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 09:59:57,363][19170] Avg episode reward: [(0, '760.190'), (1, '258.640')] [2023-12-27 09:59:57,493][20347] Updated weights for policy 0, policy_version 631238 (0.0010) [2023-12-27 09:59:57,541][20347] Updated weights for policy 0, policy_version 631248 (0.0010) [2023-12-27 09:59:57,588][20347] Updated weights for policy 0, policy_version 631258 (0.0010) [2023-12-27 09:59:57,843][20381] Updated weights for policy 1, policy_version 630822 (0.0005) [2023-12-27 09:59:57,896][20381] Updated weights for policy 1, policy_version 630832 (0.0007) [2023-12-27 09:59:57,949][20381] Updated weights for policy 1, policy_version 630842 (0.0008) [2023-12-27 09:59:58,240][20347] Updated weights for policy 0, policy_version 631268 (0.0009) [2023-12-27 09:59:58,299][20347] Updated weights for policy 0, policy_version 631278 (0.0007) [2023-12-27 09:59:58,361][20347] Updated weights for policy 0, policy_version 631288 (0.0009) [2023-12-27 09:59:58,606][20381] Updated weights for policy 1, policy_version 630852 (0.0009) [2023-12-27 09:59:58,669][20381] Updated weights for policy 1, policy_version 630862 (0.0009) [2023-12-27 09:59:58,722][20381] Updated weights for policy 1, policy_version 630872 (0.0007) [2023-12-27 09:59:59,145][20347] Updated weights for policy 0, policy_version 631298 (0.0008) [2023-12-27 09:59:59,201][20347] Updated weights for policy 0, policy_version 631308 (0.0008) [2023-12-27 09:59:59,265][20347] Updated weights for policy 0, policy_version 631318 (0.0008) [2023-12-27 09:59:59,330][20347] Updated weights for policy 0, policy_version 631328 (0.0006) [2023-12-27 09:59:59,461][20381] Updated weights for policy 1, policy_version 630882 (0.0007) [2023-12-27 09:59:59,510][20381] Updated weights for policy 1, policy_version 630892 (0.0005) [2023-12-27 09:59:59,566][20381] Updated weights for policy 1, policy_version 630902 (0.0008) [2023-12-27 09:59:59,621][20381] Updated weights for policy 1, policy_version 630912 (0.0008) [2023-12-27 10:00:00,073][20347] Updated weights for policy 0, policy_version 631338 (0.0007) [2023-12-27 10:00:00,129][20347] Updated weights for policy 0, policy_version 631348 (0.0008) [2023-12-27 10:00:00,184][20347] Updated weights for policy 0, policy_version 631358 (0.0007) [2023-12-27 10:00:00,288][20381] Updated weights for policy 1, policy_version 630922 (0.0005) [2023-12-27 10:00:00,350][20381] Updated weights for policy 1, policy_version 630932 (0.0005) [2023-12-27 10:00:00,409][20381] Updated weights for policy 1, policy_version 630942 (0.0007) [2023-12-27 10:00:00,880][20347] Updated weights for policy 0, policy_version 631368 (0.0006) [2023-12-27 10:00:00,930][20381] Updated weights for policy 1, policy_version 630952 (0.0006) [2023-12-27 10:00:00,938][20347] Updated weights for policy 0, policy_version 631378 (0.0006) [2023-12-27 10:00:00,985][20381] Updated weights for policy 1, policy_version 630962 (0.0006) [2023-12-27 10:00:00,989][20347] Updated weights for policy 0, policy_version 631388 (0.0005) [2023-12-27 10:00:01,042][20381] Updated weights for policy 1, policy_version 630972 (0.0007) [2023-12-27 10:00:01,651][20347] Updated weights for policy 0, policy_version 631398 (0.0007) [2023-12-27 10:00:01,704][20347] Updated weights for policy 0, policy_version 631408 (0.0006) [2023-12-27 10:00:01,725][20381] Updated weights for policy 1, policy_version 630982 (0.0006) [2023-12-27 10:00:01,769][20347] Updated weights for policy 0, policy_version 631418 (0.0007) [2023-12-27 10:00:01,775][20381] Updated weights for policy 1, policy_version 630992 (0.0007) [2023-12-27 10:00:01,827][20381] Updated weights for policy 1, policy_version 631002 (0.0010) [2023-12-27 10:00:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 323223552. Throughput: 0: 10474.2, 1: 10440.7. Samples: 323236856. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 10:00:02,363][19170] Avg episode reward: [(0, '800.670'), (1, '320.610')] [2023-12-27 10:00:02,426][20347] Updated weights for policy 0, policy_version 631428 (0.0005) [2023-12-27 10:00:02,457][20381] Updated weights for policy 1, policy_version 631012 (0.0007) [2023-12-27 10:00:02,476][20347] Updated weights for policy 0, policy_version 631438 (0.0005) [2023-12-27 10:00:02,504][20381] Updated weights for policy 1, policy_version 631022 (0.0007) [2023-12-27 10:00:02,530][20347] Updated weights for policy 0, policy_version 631448 (0.0005) [2023-12-27 10:00:02,551][20381] Updated weights for policy 1, policy_version 631032 (0.0007) [2023-12-27 10:00:03,176][20347] Updated weights for policy 0, policy_version 631458 (0.0006) [2023-12-27 10:00:03,207][20381] Updated weights for policy 1, policy_version 631042 (0.0006) [2023-12-27 10:00:03,235][20347] Updated weights for policy 0, policy_version 631468 (0.0006) [2023-12-27 10:00:03,262][20381] Updated weights for policy 1, policy_version 631052 (0.0008) [2023-12-27 10:00:03,290][20347] Updated weights for policy 0, policy_version 631478 (0.0006) [2023-12-27 10:00:03,321][20381] Updated weights for policy 1, policy_version 631062 (0.0007) [2023-12-27 10:00:03,340][20347] Updated weights for policy 0, policy_version 631488 (0.0006) [2023-12-27 10:00:03,374][20381] Updated weights for policy 1, policy_version 631072 (0.0007) [2023-12-27 10:00:03,926][20347] Updated weights for policy 0, policy_version 631498 (0.0008) [2023-12-27 10:00:03,974][20347] Updated weights for policy 0, policy_version 631508 (0.0008) [2023-12-27 10:00:04,029][20347] Updated weights for policy 0, policy_version 631518 (0.0008) [2023-12-27 10:00:04,084][20381] Updated weights for policy 1, policy_version 631082 (0.0011) [2023-12-27 10:00:04,143][20381] Updated weights for policy 1, policy_version 631092 (0.0010) [2023-12-27 10:00:04,195][20381] Updated weights for policy 1, policy_version 631102 (0.0010) [2023-12-27 10:00:04,727][20347] Updated weights for policy 0, policy_version 631528 (0.0008) [2023-12-27 10:00:04,780][20347] Updated weights for policy 0, policy_version 631538 (0.0007) [2023-12-27 10:00:04,845][20347] Updated weights for policy 0, policy_version 631548 (0.0008) [2023-12-27 10:00:04,873][20381] Updated weights for policy 1, policy_version 631112 (0.0007) [2023-12-27 10:00:04,933][20381] Updated weights for policy 1, policy_version 631122 (0.0008) [2023-12-27 10:00:04,991][20381] Updated weights for policy 1, policy_version 631132 (0.0007) [2023-12-27 10:00:05,533][20347] Updated weights for policy 0, policy_version 631558 (0.0009) [2023-12-27 10:00:05,546][20381] Updated weights for policy 1, policy_version 631142 (0.0006) [2023-12-27 10:00:05,588][20347] Updated weights for policy 0, policy_version 631568 (0.0010) [2023-12-27 10:00:05,605][20381] Updated weights for policy 1, policy_version 631152 (0.0005) [2023-12-27 10:00:05,643][20347] Updated weights for policy 0, policy_version 631578 (0.0010) [2023-12-27 10:00:05,659][20381] Updated weights for policy 1, policy_version 631162 (0.0005) [2023-12-27 10:00:06,317][20347] Updated weights for policy 0, policy_version 631588 (0.0011) [2023-12-27 10:00:06,357][20381] Updated weights for policy 1, policy_version 631172 (0.0006) [2023-12-27 10:00:06,374][20347] Updated weights for policy 0, policy_version 631598 (0.0010) [2023-12-27 10:00:06,417][20381] Updated weights for policy 1, policy_version 631182 (0.0005) [2023-12-27 10:00:06,434][20347] Updated weights for policy 0, policy_version 631608 (0.0011) [2023-12-27 10:00:06,477][20381] Updated weights for policy 1, policy_version 631192 (0.0006) [2023-12-27 10:00:07,148][20347] Updated weights for policy 0, policy_version 631618 (0.0011) [2023-12-27 10:00:07,196][20381] Updated weights for policy 1, policy_version 631202 (0.0007) [2023-12-27 10:00:07,213][20291] KL-divergence is very high: 112.9919 [2023-12-27 10:00:07,213][20347] Updated weights for policy 0, policy_version 631628 (0.0010) [2023-12-27 10:00:07,225][20291] KL-divergence is very high: 106.0675 [2023-12-27 10:00:07,252][20381] Updated weights for policy 1, policy_version 631212 (0.0005) [2023-12-27 10:00:07,272][20347] Updated weights for policy 0, policy_version 631638 (0.0010) [2023-12-27 10:00:07,307][20381] Updated weights for policy 1, policy_version 631222 (0.0006) [2023-12-27 10:00:07,320][20347] Updated weights for policy 0, policy_version 631648 (0.0010) [2023-12-27 10:00:07,360][20381] Updated weights for policy 1, policy_version 631232 (0.0006) [2023-12-27 10:00:07,366][19170] Fps is (10 sec: 21291.7, 60 sec: 21025.0, 300 sec: 20854.6). Total num frames: 323338240. Throughput: 0: 10499.7, 1: 10442.6. Samples: 323300020. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 10:00:07,367][19170] Avg episode reward: [(0, '667.660'), (1, '322.980')] [2023-12-27 10:00:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000631232_161619968.pth... [2023-12-27 10:00:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000631648_161718272.pth... [2023-12-27 10:00:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000630432_161406976.pth [2023-12-27 10:00:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000630016_161308672.pth [2023-12-27 10:00:07,967][20291] KL-divergence is very high: 173.6566 [2023-12-27 10:00:07,973][20347] Updated weights for policy 0, policy_version 631658 (0.0007) [2023-12-27 10:00:08,018][20291] KL-divergence is very high: 151.7789 [2023-12-27 10:00:08,036][20347] Updated weights for policy 0, policy_version 631668 (0.0006) [2023-12-27 10:00:08,069][20381] Updated weights for policy 1, policy_version 631242 (0.0007) [2023-12-27 10:00:08,085][20347] Updated weights for policy 0, policy_version 631678 (0.0007) [2023-12-27 10:00:08,127][20381] Updated weights for policy 1, policy_version 631252 (0.0007) [2023-12-27 10:00:08,177][20381] Updated weights for policy 1, policy_version 631262 (0.0007) [2023-12-27 10:00:08,767][20347] Updated weights for policy 0, policy_version 631688 (0.0007) [2023-12-27 10:00:08,820][20347] Updated weights for policy 0, policy_version 631698 (0.0007) [2023-12-27 10:00:08,869][20347] Updated weights for policy 0, policy_version 631708 (0.0008) [2023-12-27 10:00:08,886][20381] Updated weights for policy 1, policy_version 631272 (0.0007) [2023-12-27 10:00:08,942][20381] Updated weights for policy 1, policy_version 631282 (0.0008) [2023-12-27 10:00:09,003][20381] Updated weights for policy 1, policy_version 631292 (0.0008) [2023-12-27 10:00:09,631][20347] Updated weights for policy 0, policy_version 631718 (0.0007) [2023-12-27 10:00:09,691][20347] Updated weights for policy 0, policy_version 631728 (0.0008) [2023-12-27 10:00:09,719][20381] Updated weights for policy 1, policy_version 631302 (0.0008) [2023-12-27 10:00:09,745][20347] Updated weights for policy 0, policy_version 631738 (0.0007) [2023-12-27 10:00:09,782][20381] Updated weights for policy 1, policy_version 631312 (0.0007) [2023-12-27 10:00:09,847][20381] Updated weights for policy 1, policy_version 631322 (0.0007) [2023-12-27 10:00:10,459][20347] Updated weights for policy 0, policy_version 631748 (0.0008) [2023-12-27 10:00:10,515][20347] Updated weights for policy 0, policy_version 631758 (0.0008) [2023-12-27 10:00:10,551][20381] Updated weights for policy 1, policy_version 631332 (0.0008) [2023-12-27 10:00:10,567][20347] Updated weights for policy 0, policy_version 631768 (0.0007) [2023-12-27 10:00:10,596][20381] Updated weights for policy 1, policy_version 631342 (0.0006) [2023-12-27 10:00:10,644][20381] Updated weights for policy 1, policy_version 631352 (0.0007) [2023-12-27 10:00:11,335][20347] Updated weights for policy 0, policy_version 631778 (0.0006) [2023-12-27 10:00:11,385][20381] Updated weights for policy 1, policy_version 631362 (0.0007) [2023-12-27 10:00:11,400][20347] Updated weights for policy 0, policy_version 631788 (0.0008) [2023-12-27 10:00:11,434][20381] Updated weights for policy 1, policy_version 631372 (0.0006) [2023-12-27 10:00:11,457][20347] Updated weights for policy 0, policy_version 631798 (0.0006) [2023-12-27 10:00:11,486][20381] Updated weights for policy 1, policy_version 631382 (0.0008) [2023-12-27 10:00:11,508][20347] Updated weights for policy 0, policy_version 631808 (0.0006) [2023-12-27 10:00:11,542][20381] Updated weights for policy 1, policy_version 631392 (0.0007) [2023-12-27 10:00:12,207][20347] Updated weights for policy 0, policy_version 631818 (0.0007) [2023-12-27 10:00:12,267][20381] Updated weights for policy 1, policy_version 631402 (0.0011) [2023-12-27 10:00:12,272][20347] Updated weights for policy 0, policy_version 631828 (0.0006) [2023-12-27 10:00:12,327][20381] Updated weights for policy 1, policy_version 631412 (0.0010) [2023-12-27 10:00:12,333][20347] Updated weights for policy 0, policy_version 631838 (0.0007) [2023-12-27 10:00:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 323428352. Throughput: 0: 10472.1, 1: 10323.9. Samples: 323419712. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 10:00:12,363][19170] Avg episode reward: [(0, '711.800'), (1, '315.850')] [2023-12-27 10:00:12,396][20381] Updated weights for policy 1, policy_version 631422 (0.0009) [2023-12-27 10:00:12,995][20381] Updated weights for policy 1, policy_version 631432 (0.0006) [2023-12-27 10:00:13,046][20381] Updated weights for policy 1, policy_version 631442 (0.0005) [2023-12-27 10:00:13,075][20347] Updated weights for policy 0, policy_version 631848 (0.0007) [2023-12-27 10:00:13,102][20381] Updated weights for policy 1, policy_version 631452 (0.0007) [2023-12-27 10:00:13,135][20347] Updated weights for policy 0, policy_version 631858 (0.0006) [2023-12-27 10:00:13,196][20347] Updated weights for policy 0, policy_version 631868 (0.0008) [2023-12-27 10:00:13,753][20381] Updated weights for policy 1, policy_version 631462 (0.0007) [2023-12-27 10:00:13,803][20381] Updated weights for policy 1, policy_version 631472 (0.0007) [2023-12-27 10:00:13,858][20381] Updated weights for policy 1, policy_version 631482 (0.0005) [2023-12-27 10:00:13,882][20347] Updated weights for policy 0, policy_version 631878 (0.0008) [2023-12-27 10:00:13,930][20347] Updated weights for policy 0, policy_version 631888 (0.0009) [2023-12-27 10:00:13,993][20347] Updated weights for policy 0, policy_version 631898 (0.0007) [2023-12-27 10:00:14,460][20381] Updated weights for policy 1, policy_version 631492 (0.0006) [2023-12-27 10:00:14,523][20381] Updated weights for policy 1, policy_version 631502 (0.0008) [2023-12-27 10:00:14,582][20381] Updated weights for policy 1, policy_version 631512 (0.0008) [2023-12-27 10:00:14,748][20347] Updated weights for policy 0, policy_version 631908 (0.0006) [2023-12-27 10:00:14,810][20347] Updated weights for policy 0, policy_version 631918 (0.0008) [2023-12-27 10:00:14,869][20347] Updated weights for policy 0, policy_version 631928 (0.0008) [2023-12-27 10:00:15,279][20381] Updated weights for policy 1, policy_version 631522 (0.0008) [2023-12-27 10:00:15,342][20381] Updated weights for policy 1, policy_version 631532 (0.0008) [2023-12-27 10:00:15,402][20381] Updated weights for policy 1, policy_version 631542 (0.0008) [2023-12-27 10:00:15,467][20381] Updated weights for policy 1, policy_version 631552 (0.0007) [2023-12-27 10:00:15,553][20347] Updated weights for policy 0, policy_version 631938 (0.0007) [2023-12-27 10:00:15,614][20347] Updated weights for policy 0, policy_version 631948 (0.0007) [2023-12-27 10:00:15,666][20347] Updated weights for policy 0, policy_version 631958 (0.0005) [2023-12-27 10:00:15,725][20347] Updated weights for policy 0, policy_version 631968 (0.0005) [2023-12-27 10:00:16,138][20381] Updated weights for policy 1, policy_version 631562 (0.0007) [2023-12-27 10:00:16,201][20381] Updated weights for policy 1, policy_version 631572 (0.0011) [2023-12-27 10:00:16,261][20381] Updated weights for policy 1, policy_version 631582 (0.0011) [2023-12-27 10:00:16,338][20347] Updated weights for policy 0, policy_version 631978 (0.0008) [2023-12-27 10:00:16,392][20347] Updated weights for policy 0, policy_version 631988 (0.0009) [2023-12-27 10:00:16,449][20347] Updated weights for policy 0, policy_version 631998 (0.0007) [2023-12-27 10:00:16,920][20381] Updated weights for policy 1, policy_version 631592 (0.0008) [2023-12-27 10:00:16,975][20381] Updated weights for policy 1, policy_version 631602 (0.0008) [2023-12-27 10:00:17,027][20381] Updated weights for policy 1, policy_version 631612 (0.0007) [2023-12-27 10:00:17,168][20347] Updated weights for policy 0, policy_version 632008 (0.0007) [2023-12-27 10:00:17,218][20347] Updated weights for policy 0, policy_version 632018 (0.0007) [2023-12-27 10:00:17,265][20347] Updated weights for policy 0, policy_version 632028 (0.0007) [2023-12-27 10:00:17,362][19170] Fps is (10 sec: 19667.6, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 323534848. Throughput: 0: 10344.3, 1: 10303.7. Samples: 323541052. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 10:00:17,363][19170] Avg episode reward: [(0, '779.490'), (1, '305.540')] [2023-12-27 10:00:17,729][20381] Updated weights for policy 1, policy_version 631622 (0.0007) [2023-12-27 10:00:17,783][20381] Updated weights for policy 1, policy_version 631632 (0.0007) [2023-12-27 10:00:17,832][20381] Updated weights for policy 1, policy_version 631642 (0.0009) [2023-12-27 10:00:17,924][20347] Updated weights for policy 0, policy_version 632038 (0.0006) [2023-12-27 10:00:17,983][20347] Updated weights for policy 0, policy_version 632048 (0.0005) [2023-12-27 10:00:18,035][20347] Updated weights for policy 0, policy_version 632058 (0.0005) [2023-12-27 10:00:18,554][20381] Updated weights for policy 1, policy_version 631652 (0.0008) [2023-12-27 10:00:18,610][20381] Updated weights for policy 1, policy_version 631662 (0.0008) [2023-12-27 10:00:18,666][20347] Updated weights for policy 0, policy_version 632068 (0.0008) [2023-12-27 10:00:18,679][20381] Updated weights for policy 1, policy_version 631672 (0.0006) [2023-12-27 10:00:18,724][20347] Updated weights for policy 0, policy_version 632078 (0.0008) [2023-12-27 10:00:18,785][20347] Updated weights for policy 0, policy_version 632088 (0.0006) [2023-12-27 10:00:19,300][20381] Updated weights for policy 1, policy_version 631682 (0.0006) [2023-12-27 10:00:19,369][20381] Updated weights for policy 1, policy_version 631692 (0.0008) [2023-12-27 10:00:19,434][20381] Updated weights for policy 1, policy_version 631702 (0.0007) [2023-12-27 10:00:19,479][20347] Updated weights for policy 0, policy_version 632098 (0.0011) [2023-12-27 10:00:19,500][20381] Updated weights for policy 1, policy_version 631712 (0.0008) [2023-12-27 10:00:19,545][20347] Updated weights for policy 0, policy_version 632108 (0.0011) [2023-12-27 10:00:19,607][20347] Updated weights for policy 0, policy_version 632118 (0.0010) [2023-12-27 10:00:19,671][20347] Updated weights for policy 0, policy_version 632128 (0.0010) [2023-12-27 10:00:20,228][20381] Updated weights for policy 1, policy_version 631722 (0.0008) [2023-12-27 10:00:20,285][20381] Updated weights for policy 1, policy_version 631732 (0.0008) [2023-12-27 10:00:20,302][20347] Updated weights for policy 0, policy_version 632138 (0.0007) [2023-12-27 10:00:20,347][20381] Updated weights for policy 1, policy_version 631742 (0.0008) [2023-12-27 10:00:20,365][20347] Updated weights for policy 0, policy_version 632148 (0.0007) [2023-12-27 10:00:20,426][20347] Updated weights for policy 0, policy_version 632158 (0.0007) [2023-12-27 10:00:21,085][20347] Updated weights for policy 0, policy_version 632168 (0.0007) [2023-12-27 10:00:21,154][20347] Updated weights for policy 0, policy_version 632178 (0.0007) [2023-12-27 10:00:21,167][20381] Updated weights for policy 1, policy_version 631752 (0.0007) [2023-12-27 10:00:21,215][20347] Updated weights for policy 0, policy_version 632188 (0.0007) [2023-12-27 10:00:21,229][20381] Updated weights for policy 1, policy_version 631762 (0.0006) [2023-12-27 10:00:21,292][20381] Updated weights for policy 1, policy_version 631772 (0.0008) [2023-12-27 10:00:21,906][20347] Updated weights for policy 0, policy_version 632198 (0.0007) [2023-12-27 10:00:21,967][20347] Updated weights for policy 0, policy_version 632208 (0.0008) [2023-12-27 10:00:22,023][20347] Updated weights for policy 0, policy_version 632218 (0.0007) [2023-12-27 10:00:22,067][20381] Updated weights for policy 1, policy_version 631782 (0.0008) [2023-12-27 10:00:22,131][20381] Updated weights for policy 1, policy_version 631792 (0.0008) [2023-12-27 10:00:22,185][20381] Updated weights for policy 1, policy_version 631802 (0.0008) [2023-12-27 10:00:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 323633152. Throughput: 0: 10292.1, 1: 10311.8. Samples: 323602224. Policy #0 lag: (min: 15.0, avg: 18.3, max: 47.0) [2023-12-27 10:00:22,363][19170] Avg episode reward: [(0, '856.360'), (1, '319.830')] [2023-12-27 10:00:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000631808_161767424.pth... [2023-12-27 10:00:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000632224_161865728.pth... [2023-12-27 10:00:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000630592_161456128.pth [2023-12-27 10:00:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000631040_161562624.pth [2023-12-27 10:00:22,751][20347] Updated weights for policy 0, policy_version 632228 (0.0007) [2023-12-27 10:00:22,805][20347] Updated weights for policy 0, policy_version 632238 (0.0007) [2023-12-27 10:00:22,869][20347] Updated weights for policy 0, policy_version 632248 (0.0008) [2023-12-27 10:00:22,894][20381] Updated weights for policy 1, policy_version 631812 (0.0007) [2023-12-27 10:00:22,958][20381] Updated weights for policy 1, policy_version 631822 (0.0009) [2023-12-27 10:00:23,018][20381] Updated weights for policy 1, policy_version 631832 (0.0008) [2023-12-27 10:00:23,487][20347] Updated weights for policy 0, policy_version 632258 (0.0008) [2023-12-27 10:00:23,542][20347] Updated weights for policy 0, policy_version 632268 (0.0005) [2023-12-27 10:00:23,600][20347] Updated weights for policy 0, policy_version 632278 (0.0005) [2023-12-27 10:00:23,657][20347] Updated weights for policy 0, policy_version 632288 (0.0008) [2023-12-27 10:00:23,741][20381] Updated weights for policy 1, policy_version 631842 (0.0008) [2023-12-27 10:00:23,802][20381] Updated weights for policy 1, policy_version 631852 (0.0007) [2023-12-27 10:00:23,857][20381] Updated weights for policy 1, policy_version 631862 (0.0007) [2023-12-27 10:00:23,909][20381] Updated weights for policy 1, policy_version 631872 (0.0007) [2023-12-27 10:00:24,215][20347] Updated weights for policy 0, policy_version 632298 (0.0007) [2023-12-27 10:00:24,277][20347] Updated weights for policy 0, policy_version 632308 (0.0010) [2023-12-27 10:00:24,338][20347] Updated weights for policy 0, policy_version 632318 (0.0010) [2023-12-27 10:00:24,639][20381] Updated weights for policy 1, policy_version 631882 (0.0008) [2023-12-27 10:00:24,705][20381] Updated weights for policy 1, policy_version 631892 (0.0008) [2023-12-27 10:00:24,763][20381] Updated weights for policy 1, policy_version 631902 (0.0007) [2023-12-27 10:00:24,992][20347] Updated weights for policy 0, policy_version 632328 (0.0010) [2023-12-27 10:00:25,058][20347] Updated weights for policy 0, policy_version 632338 (0.0010) [2023-12-27 10:00:25,116][20347] Updated weights for policy 0, policy_version 632348 (0.0010) [2023-12-27 10:00:25,441][20381] Updated weights for policy 1, policy_version 631912 (0.0007) [2023-12-27 10:00:25,495][20381] Updated weights for policy 1, policy_version 631922 (0.0007) [2023-12-27 10:00:25,543][20381] Updated weights for policy 1, policy_version 631932 (0.0007) [2023-12-27 10:00:25,774][20347] Updated weights for policy 0, policy_version 632358 (0.0010) [2023-12-27 10:00:25,836][20347] Updated weights for policy 0, policy_version 632368 (0.0010) [2023-12-27 10:00:25,893][20347] Updated weights for policy 0, policy_version 632378 (0.0010) [2023-12-27 10:00:26,217][20381] Updated weights for policy 1, policy_version 631942 (0.0006) [2023-12-27 10:00:26,269][20381] Updated weights for policy 1, policy_version 631952 (0.0005) [2023-12-27 10:00:26,325][20381] Updated weights for policy 1, policy_version 631962 (0.0005) [2023-12-27 10:00:26,575][20347] Updated weights for policy 0, policy_version 632388 (0.0010) [2023-12-27 10:00:26,633][20347] Updated weights for policy 0, policy_version 632398 (0.0010) [2023-12-27 10:00:26,691][20347] Updated weights for policy 0, policy_version 632408 (0.0010) [2023-12-27 10:00:26,941][20381] Updated weights for policy 1, policy_version 631972 (0.0006) [2023-12-27 10:00:27,001][20381] Updated weights for policy 1, policy_version 631982 (0.0006) [2023-12-27 10:00:27,049][20381] Updated weights for policy 1, policy_version 631992 (0.0007) [2023-12-27 10:00:27,334][20347] Updated weights for policy 0, policy_version 632418 (0.0009) [2023-12-27 10:00:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 323731456. Throughput: 0: 10277.5, 1: 10210.8. Samples: 323721840. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:27,363][19170] Avg episode reward: [(0, '865.880'), (1, '282.240')] [2023-12-27 10:00:27,391][20347] Updated weights for policy 0, policy_version 632428 (0.0009) [2023-12-27 10:00:27,450][20347] Updated weights for policy 0, policy_version 632438 (0.0010) [2023-12-27 10:00:27,510][20347] Updated weights for policy 0, policy_version 632448 (0.0010) [2023-12-27 10:00:27,708][20381] Updated weights for policy 1, policy_version 632002 (0.0007) [2023-12-27 10:00:27,755][20381] Updated weights for policy 1, policy_version 632012 (0.0008) [2023-12-27 10:00:27,808][20381] Updated weights for policy 1, policy_version 632022 (0.0007) [2023-12-27 10:00:27,857][20381] Updated weights for policy 1, policy_version 632032 (0.0007) [2023-12-27 10:00:28,184][20347] Updated weights for policy 0, policy_version 632458 (0.0010) [2023-12-27 10:00:28,235][20347] Updated weights for policy 0, policy_version 632468 (0.0010) [2023-12-27 10:00:28,279][20347] Updated weights for policy 0, policy_version 632478 (0.0010) [2023-12-27 10:00:28,537][20381] Updated weights for policy 1, policy_version 632042 (0.0007) [2023-12-27 10:00:28,584][20381] Updated weights for policy 1, policy_version 632052 (0.0007) [2023-12-27 10:00:28,635][20381] Updated weights for policy 1, policy_version 632062 (0.0007) [2023-12-27 10:00:28,938][20347] Updated weights for policy 0, policy_version 632488 (0.0010) [2023-12-27 10:00:28,995][20347] Updated weights for policy 0, policy_version 632498 (0.0010) [2023-12-27 10:00:29,043][20347] Updated weights for policy 0, policy_version 632508 (0.0010) [2023-12-27 10:00:29,347][20381] Updated weights for policy 1, policy_version 632072 (0.0007) [2023-12-27 10:00:29,410][20381] Updated weights for policy 1, policy_version 632082 (0.0007) [2023-12-27 10:00:29,472][20381] Updated weights for policy 1, policy_version 632092 (0.0006) [2023-12-27 10:00:29,725][20347] Updated weights for policy 0, policy_version 632518 (0.0010) [2023-12-27 10:00:29,770][20347] Updated weights for policy 0, policy_version 632528 (0.0010) [2023-12-27 10:00:29,819][20347] Updated weights for policy 0, policy_version 632538 (0.0010) [2023-12-27 10:00:30,098][20381] Updated weights for policy 1, policy_version 632102 (0.0006) [2023-12-27 10:00:30,158][20381] Updated weights for policy 1, policy_version 632112 (0.0006) [2023-12-27 10:00:30,227][20381] Updated weights for policy 1, policy_version 632122 (0.0005) [2023-12-27 10:00:30,446][20347] Updated weights for policy 0, policy_version 632548 (0.0010) [2023-12-27 10:00:30,494][20347] Updated weights for policy 0, policy_version 632558 (0.0010) [2023-12-27 10:00:30,541][20347] Updated weights for policy 0, policy_version 632568 (0.0010) [2023-12-27 10:00:30,839][20381] Updated weights for policy 1, policy_version 632132 (0.0007) [2023-12-27 10:00:30,897][20381] Updated weights for policy 1, policy_version 632142 (0.0007) [2023-12-27 10:00:30,957][20381] Updated weights for policy 1, policy_version 632152 (0.0007) [2023-12-27 10:00:31,237][20347] Updated weights for policy 0, policy_version 632578 (0.0010) [2023-12-27 10:00:31,291][20347] Updated weights for policy 0, policy_version 632588 (0.0010) [2023-12-27 10:00:31,351][20347] Updated weights for policy 0, policy_version 632598 (0.0007) [2023-12-27 10:00:31,416][20347] Updated weights for policy 0, policy_version 632608 (0.0008) [2023-12-27 10:00:31,663][20381] Updated weights for policy 1, policy_version 632162 (0.0008) [2023-12-27 10:00:31,726][20381] Updated weights for policy 1, policy_version 632172 (0.0008) [2023-12-27 10:00:31,783][20381] Updated weights for policy 1, policy_version 632182 (0.0007) [2023-12-27 10:00:31,842][20381] Updated weights for policy 1, policy_version 632192 (0.0007) [2023-12-27 10:00:32,086][20347] Updated weights for policy 0, policy_version 632618 (0.0007) [2023-12-27 10:00:32,145][20347] Updated weights for policy 0, policy_version 632628 (0.0007) [2023-12-27 10:00:32,207][20347] Updated weights for policy 0, policy_version 632638 (0.0006) [2023-12-27 10:00:32,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20799.3). Total num frames: 323837952. Throughput: 0: 10253.5, 1: 10229.2. Samples: 323846888. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:32,363][19170] Avg episode reward: [(0, '817.670'), (1, '247.150')] [2023-12-27 10:00:32,536][20381] Updated weights for policy 1, policy_version 632202 (0.0008) [2023-12-27 10:00:32,596][20381] Updated weights for policy 1, policy_version 632212 (0.0009) [2023-12-27 10:00:32,652][20381] Updated weights for policy 1, policy_version 632222 (0.0008) [2023-12-27 10:00:32,840][20347] Updated weights for policy 0, policy_version 632648 (0.0007) [2023-12-27 10:00:32,893][20347] Updated weights for policy 0, policy_version 632658 (0.0007) [2023-12-27 10:00:32,939][20347] Updated weights for policy 0, policy_version 632668 (0.0008) [2023-12-27 10:00:33,370][20381] Updated weights for policy 1, policy_version 632232 (0.0009) [2023-12-27 10:00:33,424][20381] Updated weights for policy 1, policy_version 632242 (0.0007) [2023-12-27 10:00:33,472][20381] Updated weights for policy 1, policy_version 632252 (0.0008) [2023-12-27 10:00:33,579][20347] Updated weights for policy 0, policy_version 632678 (0.0006) [2023-12-27 10:00:33,633][20347] Updated weights for policy 0, policy_version 632688 (0.0007) [2023-12-27 10:00:33,688][20291] KL-divergence is very high: 130.3520 [2023-12-27 10:00:33,689][20347] Updated weights for policy 0, policy_version 632698 (0.0008) [2023-12-27 10:00:34,191][20381] Updated weights for policy 1, policy_version 632262 (0.0007) [2023-12-27 10:00:34,250][20381] Updated weights for policy 1, policy_version 632272 (0.0007) [2023-12-27 10:00:34,309][20381] Updated weights for policy 1, policy_version 632282 (0.0008) [2023-12-27 10:00:34,379][20347] Updated weights for policy 0, policy_version 632708 (0.0008) [2023-12-27 10:00:34,443][20347] Updated weights for policy 0, policy_version 632718 (0.0008) [2023-12-27 10:00:34,496][20347] Updated weights for policy 0, policy_version 632728 (0.0008) [2023-12-27 10:00:35,014][20381] Updated weights for policy 1, policy_version 632292 (0.0008) [2023-12-27 10:00:35,065][20381] Updated weights for policy 1, policy_version 632302 (0.0007) [2023-12-27 10:00:35,123][20381] Updated weights for policy 1, policy_version 632312 (0.0007) [2023-12-27 10:00:35,220][20347] Updated weights for policy 0, policy_version 632738 (0.0008) [2023-12-27 10:00:35,269][20347] Updated weights for policy 0, policy_version 632748 (0.0008) [2023-12-27 10:00:35,324][20347] Updated weights for policy 0, policy_version 632758 (0.0007) [2023-12-27 10:00:35,388][20347] Updated weights for policy 0, policy_version 632768 (0.0007) [2023-12-27 10:00:35,845][20381] Updated weights for policy 1, policy_version 632322 (0.0008) [2023-12-27 10:00:35,905][20381] Updated weights for policy 1, policy_version 632332 (0.0008) [2023-12-27 10:00:35,960][20381] Updated weights for policy 1, policy_version 632342 (0.0007) [2023-12-27 10:00:35,999][20347] Updated weights for policy 0, policy_version 632778 (0.0010) [2023-12-27 10:00:36,012][20381] Updated weights for policy 1, policy_version 632352 (0.0006) [2023-12-27 10:00:36,054][20347] Updated weights for policy 0, policy_version 632788 (0.0010) [2023-12-27 10:00:36,113][20347] Updated weights for policy 0, policy_version 632798 (0.0010) [2023-12-27 10:00:36,747][20381] Updated weights for policy 1, policy_version 632362 (0.0007) [2023-12-27 10:00:36,773][20347] Updated weights for policy 0, policy_version 632808 (0.0010) [2023-12-27 10:00:36,796][20381] Updated weights for policy 1, policy_version 632372 (0.0007) [2023-12-27 10:00:36,835][20347] Updated weights for policy 0, policy_version 632818 (0.0010) [2023-12-27 10:00:36,851][20381] Updated weights for policy 1, policy_version 632382 (0.0008) [2023-12-27 10:00:36,890][20347] Updated weights for policy 0, policy_version 632828 (0.0010) [2023-12-27 10:00:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 323936256. Throughput: 0: 10269.0, 1: 10212.0. Samples: 323907092. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:37,363][19170] Avg episode reward: [(0, '845.620'), (1, '312.830')] [2023-12-27 10:00:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000632384_161914880.pth... [2023-12-27 10:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000632832_162021376.pth... [2023-12-27 10:00:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000631232_161619968.pth [2023-12-27 10:00:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000631648_161718272.pth [2023-12-27 10:00:37,577][20347] Updated weights for policy 0, policy_version 632838 (0.0010) [2023-12-27 10:00:37,581][20381] Updated weights for policy 1, policy_version 632392 (0.0007) [2023-12-27 10:00:37,636][20347] Updated weights for policy 0, policy_version 632848 (0.0010) [2023-12-27 10:00:37,639][20381] Updated weights for policy 1, policy_version 632402 (0.0006) [2023-12-27 10:00:37,690][20347] Updated weights for policy 0, policy_version 632858 (0.0011) [2023-12-27 10:00:37,695][20381] Updated weights for policy 1, policy_version 632412 (0.0005) [2023-12-27 10:00:38,291][20381] Updated weights for policy 1, policy_version 632422 (0.0007) [2023-12-27 10:00:38,355][20381] Updated weights for policy 1, policy_version 632432 (0.0008) [2023-12-27 10:00:38,412][20347] Updated weights for policy 0, policy_version 632868 (0.0009) [2023-12-27 10:00:38,418][20381] Updated weights for policy 1, policy_version 632442 (0.0007) [2023-12-27 10:00:38,467][20347] Updated weights for policy 0, policy_version 632878 (0.0006) [2023-12-27 10:00:38,527][20347] Updated weights for policy 0, policy_version 632888 (0.0007) [2023-12-27 10:00:39,135][20381] Updated weights for policy 1, policy_version 632452 (0.0006) [2023-12-27 10:00:39,164][20347] Updated weights for policy 0, policy_version 632898 (0.0006) [2023-12-27 10:00:39,188][20381] Updated weights for policy 1, policy_version 632462 (0.0006) [2023-12-27 10:00:39,227][20347] Updated weights for policy 0, policy_version 632908 (0.0008) [2023-12-27 10:00:39,247][20381] Updated weights for policy 1, policy_version 632472 (0.0008) [2023-12-27 10:00:39,287][20347] Updated weights for policy 0, policy_version 632918 (0.0008) [2023-12-27 10:00:39,351][20347] Updated weights for policy 0, policy_version 632928 (0.0007) [2023-12-27 10:00:39,982][20381] Updated weights for policy 1, policy_version 632482 (0.0008) [2023-12-27 10:00:40,040][20381] Updated weights for policy 1, policy_version 632492 (0.0007) [2023-12-27 10:00:40,081][20347] Updated weights for policy 0, policy_version 632938 (0.0007) [2023-12-27 10:00:40,100][20381] Updated weights for policy 1, policy_version 632502 (0.0009) [2023-12-27 10:00:40,144][20347] Updated weights for policy 0, policy_version 632948 (0.0006) [2023-12-27 10:00:40,163][20381] Updated weights for policy 1, policy_version 632512 (0.0007) [2023-12-27 10:00:40,213][20347] Updated weights for policy 0, policy_version 632958 (0.0006) [2023-12-27 10:00:40,769][20381] Updated weights for policy 1, policy_version 632522 (0.0006) [2023-12-27 10:00:40,821][20381] Updated weights for policy 1, policy_version 632532 (0.0007) [2023-12-27 10:00:40,881][20381] Updated weights for policy 1, policy_version 632542 (0.0006) [2023-12-27 10:00:40,909][20347] Updated weights for policy 0, policy_version 632968 (0.0008) [2023-12-27 10:00:40,959][20347] Updated weights for policy 0, policy_version 632978 (0.0006) [2023-12-27 10:00:41,015][20347] Updated weights for policy 0, policy_version 632988 (0.0008) [2023-12-27 10:00:41,523][20381] Updated weights for policy 1, policy_version 632552 (0.0006) [2023-12-27 10:00:41,583][20381] Updated weights for policy 1, policy_version 632562 (0.0005) [2023-12-27 10:00:41,644][20381] Updated weights for policy 1, policy_version 632572 (0.0008) [2023-12-27 10:00:41,723][20347] Updated weights for policy 0, policy_version 632998 (0.0008) [2023-12-27 10:00:41,788][20347] Updated weights for policy 0, policy_version 633008 (0.0007) [2023-12-27 10:00:41,851][20347] Updated weights for policy 0, policy_version 633018 (0.0007) [2023-12-27 10:00:42,302][20381] Updated weights for policy 1, policy_version 632582 (0.0008) [2023-12-27 10:00:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 324034560. Throughput: 0: 10214.0, 1: 10149.0. Samples: 324028756. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:42,363][19170] Avg episode reward: [(0, '883.690'), (1, '302.480')] [2023-12-27 10:00:42,364][20381] Updated weights for policy 1, policy_version 632592 (0.0008) [2023-12-27 10:00:42,423][20381] Updated weights for policy 1, policy_version 632602 (0.0008) [2023-12-27 10:00:42,545][20347] Updated weights for policy 0, policy_version 633028 (0.0009) [2023-12-27 10:00:42,604][20347] Updated weights for policy 0, policy_version 633038 (0.0010) [2023-12-27 10:00:42,653][20347] Updated weights for policy 0, policy_version 633048 (0.0010) [2023-12-27 10:00:43,157][20381] Updated weights for policy 1, policy_version 632612 (0.0009) [2023-12-27 10:00:43,209][20381] Updated weights for policy 1, policy_version 632622 (0.0010) [2023-12-27 10:00:43,266][20381] Updated weights for policy 1, policy_version 632632 (0.0010) [2023-12-27 10:00:43,352][20347] Updated weights for policy 0, policy_version 633058 (0.0010) [2023-12-27 10:00:43,410][20347] Updated weights for policy 0, policy_version 633068 (0.0010) [2023-12-27 10:00:43,471][20347] Updated weights for policy 0, policy_version 633078 (0.0010) [2023-12-27 10:00:43,520][20347] Updated weights for policy 0, policy_version 633088 (0.0010) [2023-12-27 10:00:43,955][20381] Updated weights for policy 1, policy_version 632642 (0.0010) [2023-12-27 10:00:44,019][20381] Updated weights for policy 1, policy_version 632652 (0.0009) [2023-12-27 10:00:44,076][20381] Updated weights for policy 1, policy_version 632662 (0.0009) [2023-12-27 10:00:44,114][20347] Updated weights for policy 0, policy_version 633098 (0.0010) [2023-12-27 10:00:44,131][20381] Updated weights for policy 1, policy_version 632672 (0.0010) [2023-12-27 10:00:44,172][20347] Updated weights for policy 0, policy_version 633108 (0.0006) [2023-12-27 10:00:44,233][20347] Updated weights for policy 0, policy_version 633118 (0.0006) [2023-12-27 10:00:44,672][20381] Updated weights for policy 1, policy_version 632682 (0.0009) [2023-12-27 10:00:44,735][20381] Updated weights for policy 1, policy_version 632692 (0.0011) [2023-12-27 10:00:44,792][20381] Updated weights for policy 1, policy_version 632702 (0.0011) [2023-12-27 10:00:44,846][20347] Updated weights for policy 0, policy_version 633128 (0.0007) [2023-12-27 10:00:44,917][20347] Updated weights for policy 0, policy_version 633138 (0.0008) [2023-12-27 10:00:44,987][20347] Updated weights for policy 0, policy_version 633148 (0.0008) [2023-12-27 10:00:45,496][20381] Updated weights for policy 1, policy_version 632712 (0.0010) [2023-12-27 10:00:45,543][20347] Updated weights for policy 0, policy_version 633158 (0.0006) [2023-12-27 10:00:45,560][20381] Updated weights for policy 1, policy_version 632722 (0.0011) [2023-12-27 10:00:45,596][20347] Updated weights for policy 0, policy_version 633168 (0.0006) [2023-12-27 10:00:45,618][20381] Updated weights for policy 1, policy_version 632732 (0.0007) [2023-12-27 10:00:45,643][20347] Updated weights for policy 0, policy_version 633178 (0.0008) [2023-12-27 10:00:46,201][20381] Updated weights for policy 1, policy_version 632742 (0.0009) [2023-12-27 10:00:46,249][20381] Updated weights for policy 1, policy_version 632752 (0.0010) [2023-12-27 10:00:46,301][20381] Updated weights for policy 1, policy_version 632762 (0.0010) [2023-12-27 10:00:46,336][20347] Updated weights for policy 0, policy_version 633188 (0.0008) [2023-12-27 10:00:46,393][20347] Updated weights for policy 0, policy_version 633198 (0.0007) [2023-12-27 10:00:46,452][20347] Updated weights for policy 0, policy_version 633208 (0.0007) [2023-12-27 10:00:46,469][20291] KL-divergence is very high: 116.2020 [2023-12-27 10:00:47,011][20381] Updated weights for policy 1, policy_version 632772 (0.0011) [2023-12-27 10:00:47,068][20381] Updated weights for policy 1, policy_version 632782 (0.0010) [2023-12-27 10:00:47,119][20381] Updated weights for policy 1, policy_version 632792 (0.0010) [2023-12-27 10:00:47,139][20347] Updated weights for policy 0, policy_version 633218 (0.0007) [2023-12-27 10:00:47,183][20347] Updated weights for policy 0, policy_version 633228 (0.0006) [2023-12-27 10:00:47,229][20347] Updated weights for policy 0, policy_version 633238 (0.0007) [2023-12-27 10:00:47,276][20347] Updated weights for policy 0, policy_version 633248 (0.0007) [2023-12-27 10:00:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 324149248. Throughput: 0: 10256.1, 1: 10121.1. Samples: 324153828. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:47,363][19170] Avg episode reward: [(0, '892.170'), (1, '308.250')] [2023-12-27 10:00:47,794][20381] Updated weights for policy 1, policy_version 632802 (0.0010) [2023-12-27 10:00:47,840][20381] Updated weights for policy 1, policy_version 632812 (0.0010) [2023-12-27 10:00:47,890][20381] Updated weights for policy 1, policy_version 632822 (0.0011) [2023-12-27 10:00:47,939][20347] Updated weights for policy 0, policy_version 633258 (0.0005) [2023-12-27 10:00:47,947][20381] Updated weights for policy 1, policy_version 632832 (0.0010) [2023-12-27 10:00:47,995][20347] Updated weights for policy 0, policy_version 633268 (0.0007) [2023-12-27 10:00:48,055][20347] Updated weights for policy 0, policy_version 633278 (0.0008) [2023-12-27 10:00:48,630][20381] Updated weights for policy 1, policy_version 632842 (0.0011) [2023-12-27 10:00:48,686][20381] Updated weights for policy 1, policy_version 632852 (0.0009) [2023-12-27 10:00:48,747][20381] Updated weights for policy 1, policy_version 632862 (0.0006) [2023-12-27 10:00:48,765][20347] Updated weights for policy 0, policy_version 633288 (0.0007) [2023-12-27 10:00:48,826][20347] Updated weights for policy 0, policy_version 633298 (0.0007) [2023-12-27 10:00:48,889][20347] Updated weights for policy 0, policy_version 633308 (0.0007) [2023-12-27 10:00:49,463][20381] Updated weights for policy 1, policy_version 632872 (0.0007) [2023-12-27 10:00:49,528][20381] Updated weights for policy 1, policy_version 632882 (0.0011) [2023-12-27 10:00:49,566][20347] Updated weights for policy 0, policy_version 633318 (0.0008) [2023-12-27 10:00:49,587][20381] Updated weights for policy 1, policy_version 632892 (0.0011) [2023-12-27 10:00:49,622][20347] Updated weights for policy 0, policy_version 633328 (0.0006) [2023-12-27 10:00:49,687][20347] Updated weights for policy 0, policy_version 633338 (0.0006) [2023-12-27 10:00:50,254][20381] Updated weights for policy 1, policy_version 632902 (0.0011) [2023-12-27 10:00:50,304][20381] Updated weights for policy 1, policy_version 632912 (0.0011) [2023-12-27 10:00:50,357][20381] Updated weights for policy 1, policy_version 632922 (0.0011) [2023-12-27 10:00:50,391][20347] Updated weights for policy 0, policy_version 633348 (0.0007) [2023-12-27 10:00:50,450][20347] Updated weights for policy 0, policy_version 633358 (0.0007) [2023-12-27 10:00:50,513][20347] Updated weights for policy 0, policy_version 633368 (0.0007) [2023-12-27 10:00:51,033][20381] Updated weights for policy 1, policy_version 632932 (0.0011) [2023-12-27 10:00:51,093][20381] Updated weights for policy 1, policy_version 632942 (0.0011) [2023-12-27 10:00:51,114][20347] Updated weights for policy 0, policy_version 633378 (0.0007) [2023-12-27 10:00:51,155][20381] Updated weights for policy 1, policy_version 632952 (0.0010) [2023-12-27 10:00:51,174][20347] Updated weights for policy 0, policy_version 633388 (0.0007) [2023-12-27 10:00:51,226][20347] Updated weights for policy 0, policy_version 633398 (0.0007) [2023-12-27 10:00:51,286][20347] Updated weights for policy 0, policy_version 633408 (0.0009) [2023-12-27 10:00:51,891][20381] Updated weights for policy 1, policy_version 632962 (0.0011) [2023-12-27 10:00:51,949][20381] Updated weights for policy 1, policy_version 632972 (0.0011) [2023-12-27 10:00:52,005][20381] Updated weights for policy 1, policy_version 632982 (0.0011) [2023-12-27 10:00:52,013][20347] Updated weights for policy 0, policy_version 633418 (0.0005) [2023-12-27 10:00:52,057][20381] Updated weights for policy 1, policy_version 632992 (0.0011) [2023-12-27 10:00:52,070][20347] Updated weights for policy 0, policy_version 633428 (0.0006) [2023-12-27 10:00:52,120][20347] Updated weights for policy 0, policy_version 633438 (0.0007) [2023-12-27 10:00:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 324247552. Throughput: 0: 10224.2, 1: 10113.6. Samples: 324215148. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:52,363][19170] Avg episode reward: [(0, '851.740'), (1, '310.360')] [2023-12-27 10:00:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000632992_162070528.pth... [2023-12-27 10:00:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000633440_162177024.pth... [2023-12-27 10:00:52,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000631808_161767424.pth [2023-12-27 10:00:52,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000632224_161865728.pth [2023-12-27 10:00:52,758][20347] Updated weights for policy 0, policy_version 633448 (0.0006) [2023-12-27 10:00:52,761][20381] Updated weights for policy 1, policy_version 633002 (0.0007) [2023-12-27 10:00:52,818][20347] Updated weights for policy 0, policy_version 633458 (0.0005) [2023-12-27 10:00:52,822][20381] Updated weights for policy 1, policy_version 633012 (0.0009) [2023-12-27 10:00:52,878][20347] Updated weights for policy 0, policy_version 633468 (0.0006) [2023-12-27 10:00:52,880][20381] Updated weights for policy 1, policy_version 633022 (0.0007) [2023-12-27 10:00:53,450][20347] Updated weights for policy 0, policy_version 633478 (0.0008) [2023-12-27 10:00:53,502][20347] Updated weights for policy 0, policy_version 633488 (0.0010) [2023-12-27 10:00:53,560][20347] Updated weights for policy 0, policy_version 633498 (0.0010) [2023-12-27 10:00:53,570][20381] Updated weights for policy 1, policy_version 633032 (0.0008) [2023-12-27 10:00:53,624][20381] Updated weights for policy 1, policy_version 633042 (0.0010) [2023-12-27 10:00:53,668][20381] Updated weights for policy 1, policy_version 633052 (0.0008) [2023-12-27 10:00:54,165][20347] Updated weights for policy 0, policy_version 633508 (0.0008) [2023-12-27 10:00:54,188][20291] KL-divergence is very high: 113.0260 [2023-12-27 10:00:54,194][20291] KL-divergence is very high: 213.4107 [2023-12-27 10:00:54,200][20291] KL-divergence is very high: 150.7816 [2023-12-27 10:00:54,206][20291] KL-divergence is very high: 153.5393 [2023-12-27 10:00:54,221][20347] Updated weights for policy 0, policy_version 633518 (0.0005) [2023-12-27 10:00:54,277][20347] Updated weights for policy 0, policy_version 633528 (0.0005) [2023-12-27 10:00:54,306][20381] Updated weights for policy 1, policy_version 633062 (0.0008) [2023-12-27 10:00:54,361][20381] Updated weights for policy 1, policy_version 633072 (0.0010) [2023-12-27 10:00:54,416][20381] Updated weights for policy 1, policy_version 633082 (0.0011) [2023-12-27 10:00:54,814][20347] Updated weights for policy 0, policy_version 633538 (0.0006) [2023-12-27 10:00:54,877][20347] Updated weights for policy 0, policy_version 633548 (0.0010) [2023-12-27 10:00:54,932][20347] Updated weights for policy 0, policy_version 633558 (0.0010) [2023-12-27 10:00:54,981][20347] Updated weights for policy 0, policy_version 633568 (0.0009) [2023-12-27 10:00:55,087][20381] Updated weights for policy 1, policy_version 633092 (0.0008) [2023-12-27 10:00:55,143][20381] Updated weights for policy 1, policy_version 633102 (0.0005) [2023-12-27 10:00:55,202][20381] Updated weights for policy 1, policy_version 633112 (0.0005) [2023-12-27 10:00:55,563][20347] Updated weights for policy 0, policy_version 633578 (0.0010) [2023-12-27 10:00:55,615][20347] Updated weights for policy 0, policy_version 633588 (0.0010) [2023-12-27 10:00:55,660][20347] Updated weights for policy 0, policy_version 633598 (0.0010) [2023-12-27 10:00:55,815][20381] Updated weights for policy 1, policy_version 633122 (0.0006) [2023-12-27 10:00:55,866][20381] Updated weights for policy 1, policy_version 633132 (0.0010) [2023-12-27 10:00:55,921][20381] Updated weights for policy 1, policy_version 633142 (0.0010) [2023-12-27 10:00:55,975][20381] Updated weights for policy 1, policy_version 633152 (0.0010) [2023-12-27 10:00:56,327][20347] Updated weights for policy 0, policy_version 633608 (0.0008) [2023-12-27 10:00:56,384][20347] Updated weights for policy 0, policy_version 633618 (0.0008) [2023-12-27 10:00:56,438][20347] Updated weights for policy 0, policy_version 633628 (0.0007) [2023-12-27 10:00:56,652][20381] Updated weights for policy 1, policy_version 633162 (0.0010) [2023-12-27 10:00:56,707][20381] Updated weights for policy 1, policy_version 633172 (0.0010) [2023-12-27 10:00:56,761][20381] Updated weights for policy 1, policy_version 633182 (0.0010) [2023-12-27 10:00:57,116][20347] Updated weights for policy 0, policy_version 633638 (0.0006) [2023-12-27 10:00:57,166][20347] Updated weights for policy 0, policy_version 633648 (0.0005) [2023-12-27 10:00:57,214][20347] Updated weights for policy 0, policy_version 633658 (0.0005) [2023-12-27 10:00:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 324354048. Throughput: 0: 10358.4, 1: 10150.9. Samples: 324342628. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:00:57,363][19170] Avg episode reward: [(0, '809.250'), (1, '258.880')] [2023-12-27 10:00:57,455][20381] Updated weights for policy 1, policy_version 633192 (0.0010) [2023-12-27 10:00:57,506][20381] Updated weights for policy 1, policy_version 633202 (0.0010) [2023-12-27 10:00:57,563][20381] Updated weights for policy 1, policy_version 633212 (0.0010) [2023-12-27 10:00:57,843][20347] Updated weights for policy 0, policy_version 633668 (0.0006) [2023-12-27 10:00:57,908][20347] Updated weights for policy 0, policy_version 633678 (0.0007) [2023-12-27 10:00:57,956][20347] Updated weights for policy 0, policy_version 633688 (0.0008) [2023-12-27 10:00:58,193][20381] Updated weights for policy 1, policy_version 633222 (0.0009) [2023-12-27 10:00:58,255][20381] Updated weights for policy 1, policy_version 633232 (0.0009) [2023-12-27 10:00:58,316][20381] Updated weights for policy 1, policy_version 633242 (0.0008) [2023-12-27 10:00:58,667][20347] Updated weights for policy 0, policy_version 633698 (0.0007) [2023-12-27 10:00:58,730][20347] Updated weights for policy 0, policy_version 633708 (0.0008) [2023-12-27 10:00:58,796][20347] Updated weights for policy 0, policy_version 633718 (0.0008) [2023-12-27 10:00:58,862][20347] Updated weights for policy 0, policy_version 633728 (0.0007) [2023-12-27 10:00:59,120][20381] Updated weights for policy 1, policy_version 633252 (0.0009) [2023-12-27 10:00:59,174][20381] Updated weights for policy 1, policy_version 633262 (0.0008) [2023-12-27 10:00:59,241][20381] Updated weights for policy 1, policy_version 633272 (0.0008) [2023-12-27 10:00:59,619][20347] Updated weights for policy 0, policy_version 633738 (0.0008) [2023-12-27 10:00:59,670][20347] Updated weights for policy 0, policy_version 633748 (0.0008) [2023-12-27 10:00:59,724][20347] Updated weights for policy 0, policy_version 633758 (0.0005) [2023-12-27 10:00:59,869][20381] Updated weights for policy 1, policy_version 633282 (0.0009) [2023-12-27 10:00:59,937][20381] Updated weights for policy 1, policy_version 633292 (0.0008) [2023-12-27 10:00:59,992][20381] Updated weights for policy 1, policy_version 633302 (0.0007) [2023-12-27 10:01:00,049][20381] Updated weights for policy 1, policy_version 633312 (0.0007) [2023-12-27 10:01:00,401][20347] Updated weights for policy 0, policy_version 633768 (0.0007) [2023-12-27 10:01:00,451][20347] Updated weights for policy 0, policy_version 633778 (0.0005) [2023-12-27 10:01:00,497][20347] Updated weights for policy 0, policy_version 633788 (0.0006) [2023-12-27 10:01:00,693][20381] Updated weights for policy 1, policy_version 633322 (0.0005) [2023-12-27 10:01:00,742][20381] Updated weights for policy 1, policy_version 633332 (0.0008) [2023-12-27 10:01:00,791][20381] Updated weights for policy 1, policy_version 633342 (0.0007) [2023-12-27 10:01:01,131][20347] Updated weights for policy 0, policy_version 633798 (0.0007) [2023-12-27 10:01:01,187][20347] Updated weights for policy 0, policy_version 633808 (0.0006) [2023-12-27 10:01:01,246][20347] Updated weights for policy 0, policy_version 633818 (0.0007) [2023-12-27 10:01:01,523][20381] Updated weights for policy 1, policy_version 633352 (0.0007) [2023-12-27 10:01:01,578][20381] Updated weights for policy 1, policy_version 633362 (0.0007) [2023-12-27 10:01:01,637][20381] Updated weights for policy 1, policy_version 633372 (0.0008) [2023-12-27 10:01:01,936][20347] Updated weights for policy 0, policy_version 633828 (0.0006) [2023-12-27 10:01:01,997][20347] Updated weights for policy 0, policy_version 633838 (0.0006) [2023-12-27 10:01:02,057][20347] Updated weights for policy 0, policy_version 633848 (0.0008) [2023-12-27 10:01:02,230][20381] Updated weights for policy 1, policy_version 633382 (0.0006) [2023-12-27 10:01:02,292][20381] Updated weights for policy 1, policy_version 633392 (0.0008) [2023-12-27 10:01:02,352][20381] Updated weights for policy 1, policy_version 633402 (0.0008) [2023-12-27 10:01:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20799.3). Total num frames: 324452352. Throughput: 0: 10406.0, 1: 10131.8. Samples: 324465252. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:02,363][19170] Avg episode reward: [(0, '369.940'), (1, '270.160')] [2023-12-27 10:01:02,744][20347] Updated weights for policy 0, policy_version 633858 (0.0007) [2023-12-27 10:01:02,796][20347] Updated weights for policy 0, policy_version 633868 (0.0007) [2023-12-27 10:01:02,845][20347] Updated weights for policy 0, policy_version 633878 (0.0007) [2023-12-27 10:01:02,904][20347] Updated weights for policy 0, policy_version 633888 (0.0006) [2023-12-27 10:01:02,971][20381] Updated weights for policy 1, policy_version 633412 (0.0006) [2023-12-27 10:01:03,027][20381] Updated weights for policy 1, policy_version 633422 (0.0005) [2023-12-27 10:01:03,079][20381] Updated weights for policy 1, policy_version 633432 (0.0007) [2023-12-27 10:01:03,570][20347] Updated weights for policy 0, policy_version 633898 (0.0008) [2023-12-27 10:01:03,624][20347] Updated weights for policy 0, policy_version 633908 (0.0008) [2023-12-27 10:01:03,678][20347] Updated weights for policy 0, policy_version 633918 (0.0008) [2023-12-27 10:01:03,716][20381] Updated weights for policy 1, policy_version 633442 (0.0008) [2023-12-27 10:01:03,765][20381] Updated weights for policy 1, policy_version 633452 (0.0010) [2023-12-27 10:01:03,819][20381] Updated weights for policy 1, policy_version 633462 (0.0009) [2023-12-27 10:01:03,880][20381] Updated weights for policy 1, policy_version 633472 (0.0010) [2023-12-27 10:01:04,400][20347] Updated weights for policy 0, policy_version 633928 (0.0007) [2023-12-27 10:01:04,459][20347] Updated weights for policy 0, policy_version 633938 (0.0008) [2023-12-27 10:01:04,517][20347] Updated weights for policy 0, policy_version 633948 (0.0007) [2023-12-27 10:01:04,601][20381] Updated weights for policy 1, policy_version 633482 (0.0011) [2023-12-27 10:01:04,662][20381] Updated weights for policy 1, policy_version 633492 (0.0011) [2023-12-27 10:01:04,719][20381] Updated weights for policy 1, policy_version 633502 (0.0011) [2023-12-27 10:01:05,197][20347] Updated weights for policy 0, policy_version 633958 (0.0006) [2023-12-27 10:01:05,256][20347] Updated weights for policy 0, policy_version 633968 (0.0007) [2023-12-27 10:01:05,321][20347] Updated weights for policy 0, policy_version 633978 (0.0007) [2023-12-27 10:01:05,353][20381] Updated weights for policy 1, policy_version 633512 (0.0007) [2023-12-27 10:01:05,406][20381] Updated weights for policy 1, policy_version 633522 (0.0005) [2023-12-27 10:01:05,465][20381] Updated weights for policy 1, policy_version 633532 (0.0006) [2023-12-27 10:01:05,993][20347] Updated weights for policy 0, policy_version 633988 (0.0008) [2023-12-27 10:01:06,040][20347] Updated weights for policy 0, policy_version 633998 (0.0009) [2023-12-27 10:01:06,069][20381] Updated weights for policy 1, policy_version 633542 (0.0006) [2023-12-27 10:01:06,089][20347] Updated weights for policy 0, policy_version 634008 (0.0007) [2023-12-27 10:01:06,127][20381] Updated weights for policy 1, policy_version 633552 (0.0008) [2023-12-27 10:01:06,183][20381] Updated weights for policy 1, policy_version 633562 (0.0007) [2023-12-27 10:01:06,839][20381] Updated weights for policy 1, policy_version 633572 (0.0007) [2023-12-27 10:01:06,840][20347] Updated weights for policy 0, policy_version 634018 (0.0008) [2023-12-27 10:01:06,904][20381] Updated weights for policy 1, policy_version 633582 (0.0009) [2023-12-27 10:01:06,904][20347] Updated weights for policy 0, policy_version 634028 (0.0009) [2023-12-27 10:01:06,958][20381] Updated weights for policy 1, policy_version 633592 (0.0010) [2023-12-27 10:01:06,959][20347] Updated weights for policy 0, policy_version 634038 (0.0006) [2023-12-27 10:01:07,016][20347] Updated weights for policy 0, policy_version 634048 (0.0007) [2023-12-27 10:01:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20344.6, 300 sec: 20827.1). Total num frames: 324558848. Throughput: 0: 10382.6, 1: 10174.9. Samples: 324527312. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:07,363][19170] Avg episode reward: [(0, '475.310'), (1, '217.740')] [2023-12-27 10:01:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000633600_162226176.pth... [2023-12-27 10:01:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000634048_162332672.pth... [2023-12-27 10:01:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000632384_161914880.pth [2023-12-27 10:01:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000632832_162021376.pth [2023-12-27 10:01:07,666][20381] Updated weights for policy 1, policy_version 633602 (0.0010) [2023-12-27 10:01:07,674][20347] Updated weights for policy 0, policy_version 634058 (0.0007) [2023-12-27 10:01:07,723][20381] Updated weights for policy 1, policy_version 633612 (0.0007) [2023-12-27 10:01:07,728][20347] Updated weights for policy 0, policy_version 634068 (0.0005) [2023-12-27 10:01:07,783][20381] Updated weights for policy 1, policy_version 633622 (0.0006) [2023-12-27 10:01:07,785][20347] Updated weights for policy 0, policy_version 634078 (0.0005) [2023-12-27 10:01:07,831][20381] Updated weights for policy 1, policy_version 633632 (0.0007) [2023-12-27 10:01:08,418][20347] Updated weights for policy 0, policy_version 634088 (0.0008) [2023-12-27 10:01:08,466][20381] Updated weights for policy 1, policy_version 633642 (0.0007) [2023-12-27 10:01:08,475][20347] Updated weights for policy 0, policy_version 634098 (0.0006) [2023-12-27 10:01:08,518][20381] Updated weights for policy 1, policy_version 633652 (0.0006) [2023-12-27 10:01:08,528][20347] Updated weights for policy 0, policy_version 634108 (0.0006) [2023-12-27 10:01:08,577][20381] Updated weights for policy 1, policy_version 633662 (0.0007) [2023-12-27 10:01:09,251][20347] Updated weights for policy 0, policy_version 634118 (0.0007) [2023-12-27 10:01:09,290][20381] Updated weights for policy 1, policy_version 633672 (0.0008) [2023-12-27 10:01:09,308][20347] Updated weights for policy 0, policy_version 634128 (0.0006) [2023-12-27 10:01:09,345][20381] Updated weights for policy 1, policy_version 633682 (0.0006) [2023-12-27 10:01:09,367][20347] Updated weights for policy 0, policy_version 634138 (0.0007) [2023-12-27 10:01:09,409][20381] Updated weights for policy 1, policy_version 633692 (0.0007) [2023-12-27 10:01:10,036][20347] Updated weights for policy 0, policy_version 634148 (0.0007) [2023-12-27 10:01:10,093][20347] Updated weights for policy 0, policy_version 634158 (0.0006) [2023-12-27 10:01:10,153][20347] Updated weights for policy 0, policy_version 634168 (0.0011) [2023-12-27 10:01:10,191][20381] Updated weights for policy 1, policy_version 633702 (0.0009) [2023-12-27 10:01:10,253][20381] Updated weights for policy 1, policy_version 633712 (0.0011) [2023-12-27 10:01:10,318][20381] Updated weights for policy 1, policy_version 633722 (0.0011) [2023-12-27 10:01:10,815][20347] Updated weights for policy 0, policy_version 634178 (0.0010) [2023-12-27 10:01:10,866][20347] Updated weights for policy 0, policy_version 634188 (0.0006) [2023-12-27 10:01:10,925][20347] Updated weights for policy 0, policy_version 634198 (0.0008) [2023-12-27 10:01:10,981][20381] Updated weights for policy 1, policy_version 633732 (0.0009) [2023-12-27 10:01:10,982][20347] Updated weights for policy 0, policy_version 634208 (0.0007) [2023-12-27 10:01:11,040][20381] Updated weights for policy 1, policy_version 633742 (0.0008) [2023-12-27 10:01:11,091][20381] Updated weights for policy 1, policy_version 633752 (0.0009) [2023-12-27 10:01:11,615][20347] Updated weights for policy 0, policy_version 634218 (0.0009) [2023-12-27 10:01:11,675][20347] Updated weights for policy 0, policy_version 634228 (0.0011) [2023-12-27 10:01:11,739][20347] Updated weights for policy 0, policy_version 634238 (0.0010) [2023-12-27 10:01:11,840][20381] Updated weights for policy 1, policy_version 633762 (0.0009) [2023-12-27 10:01:11,890][20381] Updated weights for policy 1, policy_version 633772 (0.0008) [2023-12-27 10:01:11,946][20381] Updated weights for policy 1, policy_version 633782 (0.0008) [2023-12-27 10:01:12,008][20381] Updated weights for policy 1, policy_version 633792 (0.0007) [2023-12-27 10:01:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 324657152. Throughput: 0: 10357.1, 1: 10246.0. Samples: 324648980. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:12,363][19170] Avg episode reward: [(0, '460.060'), (1, '174.990')] [2023-12-27 10:01:12,437][20347] Updated weights for policy 0, policy_version 634248 (0.0006) [2023-12-27 10:01:12,495][20347] Updated weights for policy 0, policy_version 634258 (0.0007) [2023-12-27 10:01:12,546][20347] Updated weights for policy 0, policy_version 634268 (0.0008) [2023-12-27 10:01:12,677][20381] Updated weights for policy 1, policy_version 633802 (0.0008) [2023-12-27 10:01:12,737][20381] Updated weights for policy 1, policy_version 633812 (0.0007) [2023-12-27 10:01:12,794][20381] Updated weights for policy 1, policy_version 633822 (0.0008) [2023-12-27 10:01:13,180][20347] Updated weights for policy 0, policy_version 634278 (0.0006) [2023-12-27 10:01:13,231][20347] Updated weights for policy 0, policy_version 634288 (0.0005) [2023-12-27 10:01:13,280][20347] Updated weights for policy 0, policy_version 634298 (0.0009) [2023-12-27 10:01:13,437][20381] Updated weights for policy 1, policy_version 633832 (0.0006) [2023-12-27 10:01:13,497][20381] Updated weights for policy 1, policy_version 633842 (0.0007) [2023-12-27 10:01:13,561][20381] Updated weights for policy 1, policy_version 633852 (0.0008) [2023-12-27 10:01:13,828][20347] Updated weights for policy 0, policy_version 634308 (0.0005) [2023-12-27 10:01:13,885][20347] Updated weights for policy 0, policy_version 634318 (0.0006) [2023-12-27 10:01:13,945][20347] Updated weights for policy 0, policy_version 634328 (0.0006) [2023-12-27 10:01:14,127][20381] Updated weights for policy 1, policy_version 633862 (0.0009) [2023-12-27 10:01:14,183][20381] Updated weights for policy 1, policy_version 633872 (0.0006) [2023-12-27 10:01:14,242][20381] Updated weights for policy 1, policy_version 633882 (0.0005) [2023-12-27 10:01:14,551][20347] Updated weights for policy 0, policy_version 634338 (0.0008) [2023-12-27 10:01:14,599][20347] Updated weights for policy 0, policy_version 634348 (0.0006) [2023-12-27 10:01:14,662][20347] Updated weights for policy 0, policy_version 634358 (0.0005) [2023-12-27 10:01:14,711][20347] Updated weights for policy 0, policy_version 634368 (0.0005) [2023-12-27 10:01:14,794][20381] Updated weights for policy 1, policy_version 633892 (0.0005) [2023-12-27 10:01:14,860][20381] Updated weights for policy 1, policy_version 633902 (0.0010) [2023-12-27 10:01:14,920][20381] Updated weights for policy 1, policy_version 633912 (0.0010) [2023-12-27 10:01:15,308][20347] Updated weights for policy 0, policy_version 634378 (0.0008) [2023-12-27 10:01:15,363][20347] Updated weights for policy 0, policy_version 634388 (0.0010) [2023-12-27 10:01:15,410][20347] Updated weights for policy 0, policy_version 634398 (0.0008) [2023-12-27 10:01:15,577][20381] Updated weights for policy 1, policy_version 633922 (0.0010) [2023-12-27 10:01:15,632][20381] Updated weights for policy 1, policy_version 633932 (0.0010) [2023-12-27 10:01:15,681][20381] Updated weights for policy 1, policy_version 633942 (0.0010) [2023-12-27 10:01:15,730][20381] Updated weights for policy 1, policy_version 633952 (0.0009) [2023-12-27 10:01:15,978][20347] Updated weights for policy 0, policy_version 634408 (0.0005) [2023-12-27 10:01:16,033][20347] Updated weights for policy 0, policy_version 634418 (0.0006) [2023-12-27 10:01:16,084][20347] Updated weights for policy 0, policy_version 634428 (0.0005) [2023-12-27 10:01:16,398][20381] Updated weights for policy 1, policy_version 633962 (0.0010) [2023-12-27 10:01:16,448][20381] Updated weights for policy 1, policy_version 633972 (0.0007) [2023-12-27 10:01:16,498][20381] Updated weights for policy 1, policy_version 633982 (0.0005) [2023-12-27 10:01:16,714][20347] Updated weights for policy 0, policy_version 634438 (0.0008) [2023-12-27 10:01:16,765][20347] Updated weights for policy 0, policy_version 634448 (0.0010) [2023-12-27 10:01:16,826][20347] Updated weights for policy 0, policy_version 634458 (0.0010) [2023-12-27 10:01:17,057][20381] Updated weights for policy 1, policy_version 633992 (0.0007) [2023-12-27 10:01:17,118][20381] Updated weights for policy 1, policy_version 634002 (0.0007) [2023-12-27 10:01:17,171][20381] Updated weights for policy 1, policy_version 634012 (0.0007) [2023-12-27 10:01:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 324771840. Throughput: 0: 10444.3, 1: 10300.8. Samples: 324780412. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:17,363][19170] Avg episode reward: [(0, '646.860'), (1, '176.530')] [2023-12-27 10:01:17,477][20347] Updated weights for policy 0, policy_version 634468 (0.0008) [2023-12-27 10:01:17,545][20347] Updated weights for policy 0, policy_version 634478 (0.0005) [2023-12-27 10:01:17,606][20347] Updated weights for policy 0, policy_version 634488 (0.0005) [2023-12-27 10:01:17,817][20381] Updated weights for policy 1, policy_version 634022 (0.0006) [2023-12-27 10:01:17,870][20381] Updated weights for policy 1, policy_version 634032 (0.0005) [2023-12-27 10:01:17,921][20381] Updated weights for policy 1, policy_version 634042 (0.0005) [2023-12-27 10:01:18,239][20347] Updated weights for policy 0, policy_version 634498 (0.0007) [2023-12-27 10:01:18,286][20347] Updated weights for policy 0, policy_version 634508 (0.0010) [2023-12-27 10:01:18,339][20347] Updated weights for policy 0, policy_version 634518 (0.0010) [2023-12-27 10:01:18,398][20347] Updated weights for policy 0, policy_version 634528 (0.0009) [2023-12-27 10:01:18,597][20381] Updated weights for policy 1, policy_version 634052 (0.0008) [2023-12-27 10:01:18,652][20381] Updated weights for policy 1, policy_version 634062 (0.0008) [2023-12-27 10:01:18,710][20381] Updated weights for policy 1, policy_version 634072 (0.0005) [2023-12-27 10:01:19,042][20347] Updated weights for policy 0, policy_version 634538 (0.0007) [2023-12-27 10:01:19,104][20347] Updated weights for policy 0, policy_version 634548 (0.0008) [2023-12-27 10:01:19,163][20347] Updated weights for policy 0, policy_version 634558 (0.0007) [2023-12-27 10:01:19,354][20381] Updated weights for policy 1, policy_version 634082 (0.0007) [2023-12-27 10:01:19,425][20381] Updated weights for policy 1, policy_version 634092 (0.0010) [2023-12-27 10:01:19,493][20381] Updated weights for policy 1, policy_version 634102 (0.0010) [2023-12-27 10:01:19,555][20381] Updated weights for policy 1, policy_version 634112 (0.0011) [2023-12-27 10:01:19,827][20347] Updated weights for policy 0, policy_version 634568 (0.0008) [2023-12-27 10:01:19,889][20347] Updated weights for policy 0, policy_version 634578 (0.0007) [2023-12-27 10:01:19,946][20347] Updated weights for policy 0, policy_version 634588 (0.0008) [2023-12-27 10:01:20,249][20381] Updated weights for policy 1, policy_version 634122 (0.0010) [2023-12-27 10:01:20,305][20381] Updated weights for policy 1, policy_version 634132 (0.0011) [2023-12-27 10:01:20,367][20381] Updated weights for policy 1, policy_version 634142 (0.0011) [2023-12-27 10:01:20,659][20347] Updated weights for policy 0, policy_version 634598 (0.0007) [2023-12-27 10:01:20,708][20347] Updated weights for policy 0, policy_version 634608 (0.0007) [2023-12-27 10:01:20,757][20347] Updated weights for policy 0, policy_version 634618 (0.0008) [2023-12-27 10:01:21,063][20381] Updated weights for policy 1, policy_version 634152 (0.0011) [2023-12-27 10:01:21,124][20381] Updated weights for policy 1, policy_version 634162 (0.0011) [2023-12-27 10:01:21,192][20381] Updated weights for policy 1, policy_version 634172 (0.0011) [2023-12-27 10:01:21,487][20347] Updated weights for policy 0, policy_version 634628 (0.0007) [2023-12-27 10:01:21,551][20347] Updated weights for policy 0, policy_version 634638 (0.0006) [2023-12-27 10:01:21,609][20347] Updated weights for policy 0, policy_version 634648 (0.0006) [2023-12-27 10:01:21,888][20381] Updated weights for policy 1, policy_version 634182 (0.0007) [2023-12-27 10:01:21,953][20381] Updated weights for policy 1, policy_version 634192 (0.0008) [2023-12-27 10:01:22,019][20381] Updated weights for policy 1, policy_version 634202 (0.0008) [2023-12-27 10:01:22,268][20347] Updated weights for policy 0, policy_version 634658 (0.0008) [2023-12-27 10:01:22,327][20347] Updated weights for policy 0, policy_version 634668 (0.0007) [2023-12-27 10:01:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 324870144. Throughput: 0: 10452.6, 1: 10340.4. Samples: 324842780. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:22,363][19170] Avg episode reward: [(0, '662.940'), (1, '179.500')] [2023-12-27 10:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000634208_162381824.pth... [2023-12-27 10:01:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000632992_162070528.pth [2023-12-27 10:01:22,390][20347] Updated weights for policy 0, policy_version 634678 (0.0007) [2023-12-27 10:01:22,452][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000634688_162496512.pth... [2023-12-27 10:01:22,454][20347] Updated weights for policy 0, policy_version 634688 (0.0007) [2023-12-27 10:01:22,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000633440_162177024.pth [2023-12-27 10:01:22,683][20381] Updated weights for policy 1, policy_version 634212 (0.0006) [2023-12-27 10:01:22,748][20381] Updated weights for policy 1, policy_version 634222 (0.0010) [2023-12-27 10:01:22,805][20381] Updated weights for policy 1, policy_version 634232 (0.0011) [2023-12-27 10:01:23,152][20347] Updated weights for policy 0, policy_version 634698 (0.0007) [2023-12-27 10:01:23,203][20347] Updated weights for policy 0, policy_version 634708 (0.0007) [2023-12-27 10:01:23,250][20347] Updated weights for policy 0, policy_version 634718 (0.0007) [2023-12-27 10:01:23,456][20381] Updated weights for policy 1, policy_version 634242 (0.0011) [2023-12-27 10:01:23,508][20381] Updated weights for policy 1, policy_version 634252 (0.0010) [2023-12-27 10:01:23,557][20381] Updated weights for policy 1, policy_version 634262 (0.0010) [2023-12-27 10:01:23,615][20381] Updated weights for policy 1, policy_version 634272 (0.0010) [2023-12-27 10:01:23,929][20347] Updated weights for policy 0, policy_version 634728 (0.0008) [2023-12-27 10:01:23,979][20347] Updated weights for policy 0, policy_version 634738 (0.0010) [2023-12-27 10:01:24,033][20347] Updated weights for policy 0, policy_version 634748 (0.0009) [2023-12-27 10:01:24,218][20381] Updated weights for policy 1, policy_version 634282 (0.0011) [2023-12-27 10:01:24,283][20381] Updated weights for policy 1, policy_version 634292 (0.0011) [2023-12-27 10:01:24,341][20381] Updated weights for policy 1, policy_version 634302 (0.0011) [2023-12-27 10:01:24,695][20347] Updated weights for policy 0, policy_version 634758 (0.0009) [2023-12-27 10:01:24,746][20347] Updated weights for policy 0, policy_version 634768 (0.0007) [2023-12-27 10:01:24,793][20347] Updated weights for policy 0, policy_version 634778 (0.0007) [2023-12-27 10:01:25,018][20381] Updated weights for policy 1, policy_version 634312 (0.0009) [2023-12-27 10:01:25,069][20381] Updated weights for policy 1, policy_version 634322 (0.0007) [2023-12-27 10:01:25,123][20381] Updated weights for policy 1, policy_version 634332 (0.0007) [2023-12-27 10:01:25,498][20347] Updated weights for policy 0, policy_version 634788 (0.0009) [2023-12-27 10:01:25,559][20347] Updated weights for policy 0, policy_version 634798 (0.0010) [2023-12-27 10:01:25,610][20347] Updated weights for policy 0, policy_version 634808 (0.0010) [2023-12-27 10:01:25,773][20381] Updated weights for policy 1, policy_version 634342 (0.0008) [2023-12-27 10:01:25,818][20381] Updated weights for policy 1, policy_version 634352 (0.0009) [2023-12-27 10:01:25,864][20381] Updated weights for policy 1, policy_version 634362 (0.0008) [2023-12-27 10:01:26,195][20347] Updated weights for policy 0, policy_version 634818 (0.0008) [2023-12-27 10:01:26,247][20347] Updated weights for policy 0, policy_version 634828 (0.0006) [2023-12-27 10:01:26,310][20347] Updated weights for policy 0, policy_version 634838 (0.0006) [2023-12-27 10:01:26,367][20347] Updated weights for policy 0, policy_version 634848 (0.0005) [2023-12-27 10:01:26,513][20381] Updated weights for policy 1, policy_version 634372 (0.0009) [2023-12-27 10:01:26,567][20381] Updated weights for policy 1, policy_version 634382 (0.0007) [2023-12-27 10:01:26,614][20381] Updated weights for policy 1, policy_version 634392 (0.0008) [2023-12-27 10:01:26,978][20347] Updated weights for policy 0, policy_version 634858 (0.0005) [2023-12-27 10:01:27,037][20347] Updated weights for policy 0, policy_version 634868 (0.0005) [2023-12-27 10:01:27,092][20347] Updated weights for policy 0, policy_version 634878 (0.0005) [2023-12-27 10:01:27,190][20381] Updated weights for policy 1, policy_version 634402 (0.0006) [2023-12-27 10:01:27,243][20381] Updated weights for policy 1, policy_version 634412 (0.0005) [2023-12-27 10:01:27,301][20381] Updated weights for policy 1, policy_version 634422 (0.0006) [2023-12-27 10:01:27,361][20381] Updated weights for policy 1, policy_version 634432 (0.0007) [2023-12-27 10:01:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 324984832. Throughput: 0: 10474.2, 1: 10365.2. Samples: 324966528. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:27,363][19170] Avg episode reward: [(0, '511.440'), (1, '245.680')] [2023-12-27 10:01:27,603][20347] Updated weights for policy 0, policy_version 634888 (0.0007) [2023-12-27 10:01:27,653][20347] Updated weights for policy 0, policy_version 634898 (0.0007) [2023-12-27 10:01:27,704][20347] Updated weights for policy 0, policy_version 634908 (0.0007) [2023-12-27 10:01:27,938][20381] Updated weights for policy 1, policy_version 634442 (0.0007) [2023-12-27 10:01:27,987][20381] Updated weights for policy 1, policy_version 634452 (0.0008) [2023-12-27 10:01:28,038][20381] Updated weights for policy 1, policy_version 634462 (0.0008) [2023-12-27 10:01:28,368][20347] Updated weights for policy 0, policy_version 634918 (0.0008) [2023-12-27 10:01:28,418][20347] Updated weights for policy 0, policy_version 634928 (0.0007) [2023-12-27 10:01:28,464][20347] Updated weights for policy 0, policy_version 634938 (0.0008) [2023-12-27 10:01:28,723][20381] Updated weights for policy 1, policy_version 634472 (0.0010) [2023-12-27 10:01:28,783][20381] Updated weights for policy 1, policy_version 634482 (0.0010) [2023-12-27 10:01:28,837][20381] Updated weights for policy 1, policy_version 634492 (0.0010) [2023-12-27 10:01:29,141][20347] Updated weights for policy 0, policy_version 634948 (0.0008) [2023-12-27 10:01:29,199][20347] Updated weights for policy 0, policy_version 634958 (0.0006) [2023-12-27 10:01:29,257][20347] Updated weights for policy 0, policy_version 634968 (0.0008) [2023-12-27 10:01:29,510][20381] Updated weights for policy 1, policy_version 634502 (0.0010) [2023-12-27 10:01:29,573][20381] Updated weights for policy 1, policy_version 634512 (0.0010) [2023-12-27 10:01:29,636][20381] Updated weights for policy 1, policy_version 634522 (0.0010) [2023-12-27 10:01:29,887][20347] Updated weights for policy 0, policy_version 634978 (0.0008) [2023-12-27 10:01:29,947][20347] Updated weights for policy 0, policy_version 634988 (0.0011) [2023-12-27 10:01:30,021][20347] Updated weights for policy 0, policy_version 634999 (0.0009) [2023-12-27 10:01:30,248][20381] Updated weights for policy 1, policy_version 634532 (0.0010) [2023-12-27 10:01:30,299][20381] Updated weights for policy 1, policy_version 634542 (0.0007) [2023-12-27 10:01:30,348][20381] Updated weights for policy 1, policy_version 634552 (0.0008) [2023-12-27 10:01:30,669][20347] Updated weights for policy 0, policy_version 635009 (0.0007) [2023-12-27 10:01:30,725][20347] Updated weights for policy 0, policy_version 635019 (0.0007) [2023-12-27 10:01:30,782][20347] Updated weights for policy 0, policy_version 635029 (0.0007) [2023-12-27 10:01:30,843][20347] Updated weights for policy 0, policy_version 635039 (0.0008) [2023-12-27 10:01:31,041][20381] Updated weights for policy 1, policy_version 634562 (0.0008) [2023-12-27 10:01:31,089][20381] Updated weights for policy 1, policy_version 634572 (0.0007) [2023-12-27 10:01:31,147][20381] Updated weights for policy 1, policy_version 634582 (0.0006) [2023-12-27 10:01:31,206][20381] Updated weights for policy 1, policy_version 634592 (0.0009) [2023-12-27 10:01:31,463][20347] Updated weights for policy 0, policy_version 635049 (0.0008) [2023-12-27 10:01:31,512][20347] Updated weights for policy 0, policy_version 635059 (0.0007) [2023-12-27 10:01:31,571][20347] Updated weights for policy 0, policy_version 635069 (0.0007) [2023-12-27 10:01:31,877][20381] Updated weights for policy 1, policy_version 634602 (0.0008) [2023-12-27 10:01:31,932][20381] Updated weights for policy 1, policy_version 634612 (0.0008) [2023-12-27 10:01:31,986][20381] Updated weights for policy 1, policy_version 634622 (0.0006) [2023-12-27 10:01:32,278][20347] Updated weights for policy 0, policy_version 635079 (0.0010) [2023-12-27 10:01:32,338][20347] Updated weights for policy 0, policy_version 635089 (0.0010) [2023-12-27 10:01:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 325083136. Throughput: 0: 10527.4, 1: 10410.4. Samples: 325096028. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:32,363][19170] Avg episode reward: [(0, '403.990'), (1, '231.670')] [2023-12-27 10:01:32,396][20347] Updated weights for policy 0, policy_version 635099 (0.0010) [2023-12-27 10:01:32,593][20381] Updated weights for policy 1, policy_version 634632 (0.0008) [2023-12-27 10:01:32,651][20381] Updated weights for policy 1, policy_version 634642 (0.0006) [2023-12-27 10:01:32,710][20381] Updated weights for policy 1, policy_version 634652 (0.0009) [2023-12-27 10:01:33,005][20347] Updated weights for policy 0, policy_version 635109 (0.0009) [2023-12-27 10:01:33,055][20347] Updated weights for policy 0, policy_version 635119 (0.0010) [2023-12-27 10:01:33,109][20347] Updated weights for policy 0, policy_version 635129 (0.0006) [2023-12-27 10:01:33,418][20381] Updated weights for policy 1, policy_version 634662 (0.0009) [2023-12-27 10:01:33,477][20381] Updated weights for policy 1, policy_version 634672 (0.0009) [2023-12-27 10:01:33,529][20381] Updated weights for policy 1, policy_version 634682 (0.0010) [2023-12-27 10:01:33,635][20347] Updated weights for policy 0, policy_version 635139 (0.0005) [2023-12-27 10:01:33,684][20347] Updated weights for policy 0, policy_version 635149 (0.0005) [2023-12-27 10:01:33,734][20347] Updated weights for policy 0, policy_version 635159 (0.0005) [2023-12-27 10:01:34,200][20381] Updated weights for policy 1, policy_version 634692 (0.0008) [2023-12-27 10:01:34,252][20381] Updated weights for policy 1, policy_version 634702 (0.0010) [2023-12-27 10:01:34,308][20381] Updated weights for policy 1, policy_version 634712 (0.0010) [2023-12-27 10:01:34,381][20347] Updated weights for policy 0, policy_version 635169 (0.0006) [2023-12-27 10:01:34,444][20347] Updated weights for policy 0, policy_version 635179 (0.0008) [2023-12-27 10:01:34,504][20347] Updated weights for policy 0, policy_version 635189 (0.0007) [2023-12-27 10:01:34,566][20347] Updated weights for policy 0, policy_version 635199 (0.0008) [2023-12-27 10:01:35,000][20381] Updated weights for policy 1, policy_version 634722 (0.0010) [2023-12-27 10:01:35,052][20381] Updated weights for policy 1, policy_version 634732 (0.0010) [2023-12-27 10:01:35,106][20381] Updated weights for policy 1, policy_version 634742 (0.0010) [2023-12-27 10:01:35,157][20381] Updated weights for policy 1, policy_version 634752 (0.0006) [2023-12-27 10:01:35,263][20347] Updated weights for policy 0, policy_version 635209 (0.0008) [2023-12-27 10:01:35,314][20347] Updated weights for policy 0, policy_version 635219 (0.0007) [2023-12-27 10:01:35,370][20347] Updated weights for policy 0, policy_version 635229 (0.0008) [2023-12-27 10:01:35,757][20381] Updated weights for policy 1, policy_version 634762 (0.0006) [2023-12-27 10:01:35,817][20381] Updated weights for policy 1, policy_version 634772 (0.0008) [2023-12-27 10:01:35,874][20381] Updated weights for policy 1, policy_version 634782 (0.0008) [2023-12-27 10:01:36,100][20347] Updated weights for policy 0, policy_version 635239 (0.0007) [2023-12-27 10:01:36,166][20347] Updated weights for policy 0, policy_version 635249 (0.0008) [2023-12-27 10:01:36,232][20347] Updated weights for policy 0, policy_version 635259 (0.0006) [2023-12-27 10:01:36,573][20381] Updated weights for policy 1, policy_version 634792 (0.0007) [2023-12-27 10:01:36,622][20381] Updated weights for policy 1, policy_version 634802 (0.0008) [2023-12-27 10:01:36,673][20381] Updated weights for policy 1, policy_version 634812 (0.0007) [2023-12-27 10:01:36,923][20347] Updated weights for policy 0, policy_version 635269 (0.0008) [2023-12-27 10:01:36,974][20347] Updated weights for policy 0, policy_version 635279 (0.0010) [2023-12-27 10:01:37,027][20347] Updated weights for policy 0, policy_version 635289 (0.0010) [2023-12-27 10:01:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 325189632. Throughput: 0: 10579.3, 1: 10411.3. Samples: 325159724. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:37,363][19170] Avg episode reward: [(0, '426.510'), (1, '213.700')] [2023-12-27 10:01:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000635296_162652160.pth... [2023-12-27 10:01:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000634048_162332672.pth [2023-12-27 10:01:37,386][20381] Updated weights for policy 1, policy_version 634822 (0.0010) [2023-12-27 10:01:37,449][20381] Updated weights for policy 1, policy_version 634832 (0.0011) [2023-12-27 10:01:37,508][20381] Updated weights for policy 1, policy_version 634842 (0.0010) [2023-12-27 10:01:37,545][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000634848_162545664.pth... [2023-12-27 10:01:37,548][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000633600_162226176.pth [2023-12-27 10:01:37,690][20347] Updated weights for policy 0, policy_version 635299 (0.0010) [2023-12-27 10:01:37,738][20347] Updated weights for policy 0, policy_version 635309 (0.0010) [2023-12-27 10:01:37,790][20347] Updated weights for policy 0, policy_version 635319 (0.0010) [2023-12-27 10:01:38,094][20381] Updated weights for policy 1, policy_version 634852 (0.0009) [2023-12-27 10:01:38,141][20381] Updated weights for policy 1, policy_version 634862 (0.0007) [2023-12-27 10:01:38,193][20381] Updated weights for policy 1, policy_version 634872 (0.0007) [2023-12-27 10:01:38,464][20347] Updated weights for policy 0, policy_version 635329 (0.0010) [2023-12-27 10:01:38,525][20347] Updated weights for policy 0, policy_version 635339 (0.0005) [2023-12-27 10:01:38,587][20347] Updated weights for policy 0, policy_version 635349 (0.0007) [2023-12-27 10:01:38,646][20347] Updated weights for policy 0, policy_version 635359 (0.0008) [2023-12-27 10:01:38,946][20381] Updated weights for policy 1, policy_version 634882 (0.0007) [2023-12-27 10:01:38,997][20381] Updated weights for policy 1, policy_version 634892 (0.0009) [2023-12-27 10:01:39,048][20381] Updated weights for policy 1, policy_version 634902 (0.0007) [2023-12-27 10:01:39,107][20381] Updated weights for policy 1, policy_version 634912 (0.0008) [2023-12-27 10:01:39,223][20347] Updated weights for policy 0, policy_version 635369 (0.0008) [2023-12-27 10:01:39,286][20347] Updated weights for policy 0, policy_version 635379 (0.0006) [2023-12-27 10:01:39,352][20347] Updated weights for policy 0, policy_version 635389 (0.0007) [2023-12-27 10:01:39,776][20381] Updated weights for policy 1, policy_version 634922 (0.0007) [2023-12-27 10:01:39,835][20381] Updated weights for policy 1, policy_version 634932 (0.0008) [2023-12-27 10:01:39,900][20381] Updated weights for policy 1, policy_version 634942 (0.0008) [2023-12-27 10:01:39,967][20347] Updated weights for policy 0, policy_version 635399 (0.0007) [2023-12-27 10:01:40,025][20347] Updated weights for policy 0, policy_version 635409 (0.0008) [2023-12-27 10:01:40,081][20347] Updated weights for policy 0, policy_version 635419 (0.0007) [2023-12-27 10:01:40,554][20381] Updated weights for policy 1, policy_version 634952 (0.0007) [2023-12-27 10:01:40,607][20381] Updated weights for policy 1, policy_version 634962 (0.0008) [2023-12-27 10:01:40,661][20381] Updated weights for policy 1, policy_version 634972 (0.0008) [2023-12-27 10:01:40,789][20347] Updated weights for policy 0, policy_version 635429 (0.0007) [2023-12-27 10:01:40,838][20347] Updated weights for policy 0, policy_version 635439 (0.0008) [2023-12-27 10:01:40,888][20347] Updated weights for policy 0, policy_version 635449 (0.0008) [2023-12-27 10:01:41,421][20381] Updated weights for policy 1, policy_version 634982 (0.0008) [2023-12-27 10:01:41,482][20381] Updated weights for policy 1, policy_version 634992 (0.0008) [2023-12-27 10:01:41,528][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-27 10:01:41,719][20347] Updated weights for policy 0, policy_version 635459 (0.0008) [2023-12-27 10:01:41,786][20347] Updated weights for policy 0, policy_version 635469 (0.0008) [2023-12-27 10:01:41,846][20347] Updated weights for policy 0, policy_version 635479 (0.0008) [2023-12-27 10:01:42,264][20381] Updated weights for policy 1, policy_version 635002 (0.0008) [2023-12-27 10:01:42,322][20381] Updated weights for policy 1, policy_version 635012 (0.0006) [2023-12-27 10:01:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 325287936. Throughput: 0: 10486.1, 1: 10418.0. Samples: 325283316. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:42,363][19170] Avg episode reward: [(0, '684.770'), (1, '234.810')] [2023-12-27 10:01:42,383][20381] Updated weights for policy 1, policy_version 635022 (0.0007) [2023-12-27 10:01:42,444][20381] Updated weights for policy 1, policy_version 635032 (0.0008) [2023-12-27 10:01:42,550][20347] Updated weights for policy 0, policy_version 635489 (0.0008) [2023-12-27 10:01:42,607][20347] Updated weights for policy 0, policy_version 635499 (0.0008) [2023-12-27 10:01:42,657][20347] Updated weights for policy 0, policy_version 635509 (0.0008) [2023-12-27 10:01:42,705][20347] Updated weights for policy 0, policy_version 635519 (0.0007) [2023-12-27 10:01:43,114][20381] Updated weights for policy 1, policy_version 635042 (0.0008) [2023-12-27 10:01:43,170][20381] Updated weights for policy 1, policy_version 635052 (0.0008) [2023-12-27 10:01:43,233][20381] Updated weights for policy 1, policy_version 635062 (0.0008) [2023-12-27 10:01:43,389][20347] Updated weights for policy 0, policy_version 635529 (0.0007) [2023-12-27 10:01:43,438][20347] Updated weights for policy 0, policy_version 635539 (0.0005) [2023-12-27 10:01:43,489][20347] Updated weights for policy 0, policy_version 635549 (0.0005) [2023-12-27 10:01:43,986][20381] Updated weights for policy 1, policy_version 635072 (0.0008) [2023-12-27 10:01:44,036][20381] Updated weights for policy 1, policy_version 635082 (0.0007) [2023-12-27 10:01:44,038][20347] Updated weights for policy 0, policy_version 635559 (0.0005) [2023-12-27 10:01:44,083][20347] Updated weights for policy 0, policy_version 635569 (0.0005) [2023-12-27 10:01:44,085][20381] Updated weights for policy 1, policy_version 635092 (0.0007) [2023-12-27 10:01:44,126][20347] Updated weights for policy 0, policy_version 635579 (0.0006) [2023-12-27 10:01:44,808][20347] Updated weights for policy 0, policy_version 635589 (0.0009) [2023-12-27 10:01:44,819][20381] Updated weights for policy 1, policy_version 635102 (0.0007) [2023-12-27 10:01:44,871][20347] Updated weights for policy 0, policy_version 635599 (0.0010) [2023-12-27 10:01:44,881][20381] Updated weights for policy 1, policy_version 635112 (0.0009) [2023-12-27 10:01:44,930][20347] Updated weights for policy 0, policy_version 635609 (0.0010) [2023-12-27 10:01:44,940][20381] Updated weights for policy 1, policy_version 635122 (0.0008) [2023-12-27 10:01:45,605][20347] Updated weights for policy 0, policy_version 635619 (0.0009) [2023-12-27 10:01:45,646][20381] Updated weights for policy 1, policy_version 635132 (0.0006) [2023-12-27 10:01:45,662][20347] Updated weights for policy 0, policy_version 635629 (0.0008) [2023-12-27 10:01:45,699][20381] Updated weights for policy 1, policy_version 635142 (0.0007) [2023-12-27 10:01:45,721][20347] Updated weights for policy 0, policy_version 635639 (0.0010) [2023-12-27 10:01:45,755][20381] Updated weights for policy 1, policy_version 635152 (0.0006) [2023-12-27 10:01:46,304][20347] Updated weights for policy 0, policy_version 635649 (0.0009) [2023-12-27 10:01:46,358][20347] Updated weights for policy 0, policy_version 635659 (0.0007) [2023-12-27 10:01:46,395][20381] Updated weights for policy 1, policy_version 635162 (0.0006) [2023-12-27 10:01:46,415][20347] Updated weights for policy 0, policy_version 635669 (0.0007) [2023-12-27 10:01:46,442][20381] Updated weights for policy 1, policy_version 635172 (0.0009) [2023-12-27 10:01:46,464][20347] Updated weights for policy 0, policy_version 635679 (0.0006) [2023-12-27 10:01:46,523][20381] Updated weights for policy 1, policy_version 635182 (0.0011) [2023-12-27 10:01:46,582][20381] Updated weights for policy 1, policy_version 635192 (0.0011) [2023-12-27 10:01:47,069][20347] Updated weights for policy 0, policy_version 635689 (0.0007) [2023-12-27 10:01:47,114][20347] Updated weights for policy 0, policy_version 635699 (0.0009) [2023-12-27 10:01:47,159][20347] Updated weights for policy 0, policy_version 635709 (0.0007) [2023-12-27 10:01:47,187][20381] Updated weights for policy 1, policy_version 635202 (0.0006) [2023-12-27 10:01:47,247][20381] Updated weights for policy 1, policy_version 635212 (0.0005) [2023-12-27 10:01:47,295][20381] Updated weights for policy 1, policy_version 635222 (0.0006) [2023-12-27 10:01:47,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 325402624. Throughput: 0: 10539.5, 1: 10382.9. Samples: 325406768. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:47,364][19170] Avg episode reward: [(0, '750.520'), (1, '172.500')] [2023-12-27 10:01:47,758][20347] Updated weights for policy 0, policy_version 635719 (0.0006) [2023-12-27 10:01:47,824][20347] Updated weights for policy 0, policy_version 635729 (0.0008) [2023-12-27 10:01:47,884][20347] Updated weights for policy 0, policy_version 635739 (0.0008) [2023-12-27 10:01:47,909][20381] Updated weights for policy 1, policy_version 635232 (0.0006) [2023-12-27 10:01:47,965][20381] Updated weights for policy 1, policy_version 635242 (0.0005) [2023-12-27 10:01:48,028][20381] Updated weights for policy 1, policy_version 635252 (0.0006) [2023-12-27 10:01:48,520][20347] Updated weights for policy 0, policy_version 635749 (0.0007) [2023-12-27 10:01:48,571][20347] Updated weights for policy 0, policy_version 635759 (0.0007) [2023-12-27 10:01:48,621][20347] Updated weights for policy 0, policy_version 635769 (0.0007) [2023-12-27 10:01:48,696][20381] Updated weights for policy 1, policy_version 635262 (0.0011) [2023-12-27 10:01:48,755][20381] Updated weights for policy 1, policy_version 635272 (0.0009) [2023-12-27 10:01:48,814][20381] Updated weights for policy 1, policy_version 635282 (0.0011) [2023-12-27 10:01:49,300][20347] Updated weights for policy 0, policy_version 635779 (0.0006) [2023-12-27 10:01:49,365][20347] Updated weights for policy 0, policy_version 635789 (0.0007) [2023-12-27 10:01:49,422][20347] Updated weights for policy 0, policy_version 635799 (0.0008) [2023-12-27 10:01:49,487][20381] Updated weights for policy 1, policy_version 635292 (0.0009) [2023-12-27 10:01:49,546][20381] Updated weights for policy 1, policy_version 635302 (0.0009) [2023-12-27 10:01:49,602][20381] Updated weights for policy 1, policy_version 635312 (0.0007) [2023-12-27 10:01:50,173][20347] Updated weights for policy 0, policy_version 635809 (0.0007) [2023-12-27 10:01:50,227][20347] Updated weights for policy 0, policy_version 635819 (0.0008) [2023-12-27 10:01:50,256][20381] Updated weights for policy 1, policy_version 635322 (0.0007) [2023-12-27 10:01:50,286][20347] Updated weights for policy 0, policy_version 635829 (0.0008) [2023-12-27 10:01:50,317][20381] Updated weights for policy 1, policy_version 635332 (0.0007) [2023-12-27 10:01:50,346][20347] Updated weights for policy 0, policy_version 635839 (0.0008) [2023-12-27 10:01:50,377][20381] Updated weights for policy 1, policy_version 635342 (0.0007) [2023-12-27 10:01:50,431][20381] Updated weights for policy 1, policy_version 635352 (0.0008) [2023-12-27 10:01:51,056][20347] Updated weights for policy 0, policy_version 635849 (0.0007) [2023-12-27 10:01:51,074][20381] Updated weights for policy 1, policy_version 635362 (0.0007) [2023-12-27 10:01:51,114][20347] Updated weights for policy 0, policy_version 635859 (0.0007) [2023-12-27 10:01:51,134][20381] Updated weights for policy 1, policy_version 635372 (0.0007) [2023-12-27 10:01:51,177][20347] Updated weights for policy 0, policy_version 635869 (0.0006) [2023-12-27 10:01:51,190][20381] Updated weights for policy 1, policy_version 635382 (0.0008) [2023-12-27 10:01:51,854][20347] Updated weights for policy 0, policy_version 635879 (0.0007) [2023-12-27 10:01:51,900][20381] Updated weights for policy 1, policy_version 635392 (0.0008) [2023-12-27 10:01:51,913][20347] Updated weights for policy 0, policy_version 635889 (0.0005) [2023-12-27 10:01:51,957][20381] Updated weights for policy 1, policy_version 635402 (0.0007) [2023-12-27 10:01:51,976][20347] Updated weights for policy 0, policy_version 635899 (0.0007) [2023-12-27 10:01:52,017][20381] Updated weights for policy 1, policy_version 635412 (0.0007) [2023-12-27 10:01:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 325500928. Throughput: 0: 10572.8, 1: 10381.3. Samples: 325470248. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:52,363][19170] Avg episode reward: [(0, '808.880'), (1, '146.800')] [2023-12-27 10:01:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000635416_162693120.pth... [2023-12-27 10:01:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000635904_162807808.pth... [2023-12-27 10:01:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000634688_162496512.pth [2023-12-27 10:01:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000634208_162381824.pth [2023-12-27 10:01:52,613][20381] Updated weights for policy 1, policy_version 635422 (0.0006) [2023-12-27 10:01:52,639][20347] Updated weights for policy 0, policy_version 635909 (0.0007) [2023-12-27 10:01:52,675][20381] Updated weights for policy 1, policy_version 635432 (0.0005) [2023-12-27 10:01:52,696][20347] Updated weights for policy 0, policy_version 635919 (0.0007) [2023-12-27 10:01:52,734][20381] Updated weights for policy 1, policy_version 635442 (0.0005) [2023-12-27 10:01:52,753][20347] Updated weights for policy 0, policy_version 635929 (0.0008) [2023-12-27 10:01:53,282][20381] Updated weights for policy 1, policy_version 635452 (0.0006) [2023-12-27 10:01:53,342][20381] Updated weights for policy 1, policy_version 635462 (0.0008) [2023-12-27 10:01:53,393][20381] Updated weights for policy 1, policy_version 635472 (0.0007) [2023-12-27 10:01:53,447][20347] Updated weights for policy 0, policy_version 635939 (0.0006) [2023-12-27 10:01:53,510][20347] Updated weights for policy 0, policy_version 635949 (0.0005) [2023-12-27 10:01:53,571][20347] Updated weights for policy 0, policy_version 635959 (0.0006) [2023-12-27 10:01:54,015][20381] Updated weights for policy 1, policy_version 635482 (0.0006) [2023-12-27 10:01:54,069][20381] Updated weights for policy 1, policy_version 635492 (0.0009) [2023-12-27 10:01:54,121][20381] Updated weights for policy 1, policy_version 635502 (0.0011) [2023-12-27 10:01:54,179][20381] Updated weights for policy 1, policy_version 635512 (0.0010) [2023-12-27 10:01:54,223][20347] Updated weights for policy 0, policy_version 635969 (0.0007) [2023-12-27 10:01:54,277][20347] Updated weights for policy 0, policy_version 635979 (0.0007) [2023-12-27 10:01:54,332][20347] Updated weights for policy 0, policy_version 635989 (0.0008) [2023-12-27 10:01:54,396][20347] Updated weights for policy 0, policy_version 635999 (0.0005) [2023-12-27 10:01:54,908][20381] Updated weights for policy 1, policy_version 635522 (0.0007) [2023-12-27 10:01:54,954][20381] Updated weights for policy 1, policy_version 635532 (0.0006) [2023-12-27 10:01:54,983][20347] Updated weights for policy 0, policy_version 636009 (0.0010) [2023-12-27 10:01:55,015][20381] Updated weights for policy 1, policy_version 635542 (0.0006) [2023-12-27 10:01:55,036][20347] Updated weights for policy 0, policy_version 636019 (0.0010) [2023-12-27 10:01:55,093][20347] Updated weights for policy 0, policy_version 636029 (0.0009) [2023-12-27 10:01:55,646][20347] Updated weights for policy 0, policy_version 636039 (0.0006) [2023-12-27 10:01:55,700][20347] Updated weights for policy 0, policy_version 636049 (0.0007) [2023-12-27 10:01:55,746][20381] Updated weights for policy 1, policy_version 635552 (0.0009) [2023-12-27 10:01:55,755][20347] Updated weights for policy 0, policy_version 636059 (0.0006) [2023-12-27 10:01:55,797][20381] Updated weights for policy 1, policy_version 635562 (0.0010) [2023-12-27 10:01:55,848][20381] Updated weights for policy 1, policy_version 635572 (0.0010) [2023-12-27 10:01:56,388][20347] Updated weights for policy 0, policy_version 636069 (0.0006) [2023-12-27 10:01:56,441][20347] Updated weights for policy 0, policy_version 636079 (0.0007) [2023-12-27 10:01:56,499][20347] Updated weights for policy 0, policy_version 636089 (0.0007) [2023-12-27 10:01:56,564][20381] Updated weights for policy 1, policy_version 635582 (0.0008) [2023-12-27 10:01:56,632][20381] Updated weights for policy 1, policy_version 635592 (0.0005) [2023-12-27 10:01:56,695][20381] Updated weights for policy 1, policy_version 635602 (0.0005) [2023-12-27 10:01:57,132][20347] Updated weights for policy 0, policy_version 636099 (0.0006) [2023-12-27 10:01:57,194][20347] Updated weights for policy 0, policy_version 636109 (0.0005) [2023-12-27 10:01:57,243][20347] Updated weights for policy 0, policy_version 636119 (0.0007) [2023-12-27 10:01:57,304][20381] Updated weights for policy 1, policy_version 635612 (0.0005) [2023-12-27 10:01:57,359][20381] Updated weights for policy 1, policy_version 635622 (0.0006) [2023-12-27 10:01:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 325607424. Throughput: 0: 10604.2, 1: 10424.9. Samples: 325595292. Policy #0 lag: (min: 31.0, avg: 43.4, max: 63.0) [2023-12-27 10:01:57,363][19170] Avg episode reward: [(0, '799.470'), (1, '198.690')] [2023-12-27 10:01:57,420][20381] Updated weights for policy 1, policy_version 635632 (0.0005) [2023-12-27 10:01:57,933][20347] Updated weights for policy 0, policy_version 636129 (0.0007) [2023-12-27 10:01:57,966][20381] Updated weights for policy 1, policy_version 635642 (0.0006) [2023-12-27 10:01:57,989][20347] Updated weights for policy 0, policy_version 636139 (0.0007) [2023-12-27 10:01:58,017][20381] Updated weights for policy 1, policy_version 635652 (0.0006) [2023-12-27 10:01:58,039][20347] Updated weights for policy 0, policy_version 636149 (0.0006) [2023-12-27 10:01:58,074][20381] Updated weights for policy 1, policy_version 635662 (0.0005) [2023-12-27 10:01:58,097][20347] Updated weights for policy 0, policy_version 636159 (0.0008) [2023-12-27 10:01:58,123][20381] Updated weights for policy 1, policy_version 635672 (0.0006) [2023-12-27 10:01:58,846][20381] Updated weights for policy 1, policy_version 635682 (0.0007) [2023-12-27 10:01:58,850][20347] Updated weights for policy 0, policy_version 636169 (0.0008) [2023-12-27 10:01:58,910][20381] Updated weights for policy 1, policy_version 635692 (0.0007) [2023-12-27 10:01:58,911][20347] Updated weights for policy 0, policy_version 636179 (0.0008) [2023-12-27 10:01:58,967][20347] Updated weights for policy 0, policy_version 636189 (0.0005) [2023-12-27 10:01:58,968][20381] Updated weights for policy 1, policy_version 635702 (0.0006) [2023-12-27 10:01:59,627][20381] Updated weights for policy 1, policy_version 635712 (0.0007) [2023-12-27 10:01:59,658][20347] Updated weights for policy 0, policy_version 636199 (0.0007) [2023-12-27 10:01:59,679][20381] Updated weights for policy 1, policy_version 635722 (0.0006) [2023-12-27 10:01:59,707][20347] Updated weights for policy 0, policy_version 636209 (0.0005) [2023-12-27 10:01:59,738][20381] Updated weights for policy 1, policy_version 635732 (0.0007) [2023-12-27 10:01:59,755][20347] Updated weights for policy 0, policy_version 636219 (0.0005) [2023-12-27 10:02:00,411][20347] Updated weights for policy 0, policy_version 636229 (0.0007) [2023-12-27 10:02:00,423][20381] Updated weights for policy 1, policy_version 635742 (0.0006) [2023-12-27 10:02:00,467][20347] Updated weights for policy 0, policy_version 636239 (0.0008) [2023-12-27 10:02:00,483][20381] Updated weights for policy 1, policy_version 635752 (0.0005) [2023-12-27 10:02:00,518][20347] Updated weights for policy 0, policy_version 636249 (0.0007) [2023-12-27 10:02:00,538][20381] Updated weights for policy 1, policy_version 635762 (0.0007) [2023-12-27 10:02:01,165][20347] Updated weights for policy 0, policy_version 636259 (0.0006) [2023-12-27 10:02:01,200][20381] Updated weights for policy 1, policy_version 635772 (0.0008) [2023-12-27 10:02:01,222][20347] Updated weights for policy 0, policy_version 636269 (0.0008) [2023-12-27 10:02:01,255][20381] Updated weights for policy 1, policy_version 635782 (0.0011) [2023-12-27 10:02:01,272][20347] Updated weights for policy 0, policy_version 636279 (0.0006) [2023-12-27 10:02:01,315][20381] Updated weights for policy 1, policy_version 635792 (0.0011) [2023-12-27 10:02:01,966][20381] Updated weights for policy 1, policy_version 635802 (0.0009) [2023-12-27 10:02:02,028][20381] Updated weights for policy 1, policy_version 635812 (0.0007) [2023-12-27 10:02:02,035][20347] Updated weights for policy 0, policy_version 636289 (0.0005) [2023-12-27 10:02:02,077][20381] Updated weights for policy 1, policy_version 635822 (0.0007) [2023-12-27 10:02:02,086][20347] Updated weights for policy 0, policy_version 636299 (0.0006) [2023-12-27 10:02:02,134][20381] Updated weights for policy 1, policy_version 635832 (0.0005) [2023-12-27 10:02:02,140][20347] Updated weights for policy 0, policy_version 636309 (0.0007) [2023-12-27 10:02:02,198][20347] Updated weights for policy 0, policy_version 636319 (0.0005) [2023-12-27 10:02:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 325713920. Throughput: 0: 10486.6, 1: 10394.4. Samples: 325720056. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:02,363][19170] Avg episode reward: [(0, '818.650'), (1, '179.990')] [2023-12-27 10:02:02,800][20381] Updated weights for policy 1, policy_version 635842 (0.0008) [2023-12-27 10:02:02,854][20381] Updated weights for policy 1, policy_version 635852 (0.0006) [2023-12-27 10:02:02,858][20347] Updated weights for policy 0, policy_version 636329 (0.0006) [2023-12-27 10:02:02,910][20381] Updated weights for policy 1, policy_version 635862 (0.0008) [2023-12-27 10:02:02,917][20347] Updated weights for policy 0, policy_version 636339 (0.0005) [2023-12-27 10:02:02,983][20347] Updated weights for policy 0, policy_version 636349 (0.0005) [2023-12-27 10:02:03,502][20347] Updated weights for policy 0, policy_version 636359 (0.0008) [2023-12-27 10:02:03,563][20347] Updated weights for policy 0, policy_version 636369 (0.0007) [2023-12-27 10:02:03,616][20347] Updated weights for policy 0, policy_version 636379 (0.0006) [2023-12-27 10:02:03,625][20381] Updated weights for policy 1, policy_version 635872 (0.0006) [2023-12-27 10:02:03,675][20381] Updated weights for policy 1, policy_version 635882 (0.0008) [2023-12-27 10:02:03,733][20381] Updated weights for policy 1, policy_version 635892 (0.0007) [2023-12-27 10:02:04,317][20347] Updated weights for policy 0, policy_version 636389 (0.0006) [2023-12-27 10:02:04,371][20347] Updated weights for policy 0, policy_version 636399 (0.0005) [2023-12-27 10:02:04,421][20347] Updated weights for policy 0, policy_version 636409 (0.0007) [2023-12-27 10:02:04,438][20381] Updated weights for policy 1, policy_version 635902 (0.0008) [2023-12-27 10:02:04,496][20381] Updated weights for policy 1, policy_version 635912 (0.0006) [2023-12-27 10:02:04,551][20381] Updated weights for policy 1, policy_version 635922 (0.0008) [2023-12-27 10:02:05,117][20347] Updated weights for policy 0, policy_version 636419 (0.0006) [2023-12-27 10:02:05,118][20381] Updated weights for policy 1, policy_version 635932 (0.0008) [2023-12-27 10:02:05,162][20347] Updated weights for policy 0, policy_version 636429 (0.0005) [2023-12-27 10:02:05,173][20381] Updated weights for policy 1, policy_version 635942 (0.0007) [2023-12-27 10:02:05,214][20347] Updated weights for policy 0, policy_version 636439 (0.0006) [2023-12-27 10:02:05,231][20381] Updated weights for policy 1, policy_version 635952 (0.0006) [2023-12-27 10:02:05,853][20381] Updated weights for policy 1, policy_version 635962 (0.0008) [2023-12-27 10:02:05,869][20347] Updated weights for policy 0, policy_version 636449 (0.0006) [2023-12-27 10:02:05,905][20381] Updated weights for policy 1, policy_version 635972 (0.0006) [2023-12-27 10:02:05,931][20347] Updated weights for policy 0, policy_version 636459 (0.0006) [2023-12-27 10:02:05,953][20381] Updated weights for policy 1, policy_version 635982 (0.0007) [2023-12-27 10:02:05,989][20347] Updated weights for policy 0, policy_version 636469 (0.0006) [2023-12-27 10:02:06,010][20381] Updated weights for policy 1, policy_version 635992 (0.0006) [2023-12-27 10:02:06,049][20347] Updated weights for policy 0, policy_version 636479 (0.0007) [2023-12-27 10:02:06,716][20347] Updated weights for policy 0, policy_version 636489 (0.0007) [2023-12-27 10:02:06,742][20381] Updated weights for policy 1, policy_version 636002 (0.0006) [2023-12-27 10:02:06,763][20347] Updated weights for policy 0, policy_version 636499 (0.0006) [2023-12-27 10:02:06,800][20381] Updated weights for policy 1, policy_version 636012 (0.0005) [2023-12-27 10:02:06,817][20347] Updated weights for policy 0, policy_version 636509 (0.0008) [2023-12-27 10:02:06,861][20381] Updated weights for policy 1, policy_version 636022 (0.0005) [2023-12-27 10:02:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 325812224. Throughput: 0: 10497.8, 1: 10393.7. Samples: 325782896. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:07,363][19170] Avg episode reward: [(0, '894.010'), (1, '190.910')] [2023-12-27 10:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000636024_162848768.pth... [2023-12-27 10:02:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000636512_162963456.pth... [2023-12-27 10:02:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000634848_162545664.pth [2023-12-27 10:02:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000635296_162652160.pth [2023-12-27 10:02:07,470][20381] Updated weights for policy 1, policy_version 636032 (0.0005) [2023-12-27 10:02:07,508][20347] Updated weights for policy 0, policy_version 636519 (0.0008) [2023-12-27 10:02:07,519][20381] Updated weights for policy 1, policy_version 636042 (0.0006) [2023-12-27 10:02:07,566][20347] Updated weights for policy 0, policy_version 636529 (0.0008) [2023-12-27 10:02:07,575][20381] Updated weights for policy 1, policy_version 636052 (0.0006) [2023-12-27 10:02:07,617][20347] Updated weights for policy 0, policy_version 636539 (0.0006) [2023-12-27 10:02:08,196][20381] Updated weights for policy 1, policy_version 636062 (0.0007) [2023-12-27 10:02:08,239][20381] Updated weights for policy 1, policy_version 636072 (0.0008) [2023-12-27 10:02:08,283][20381] Updated weights for policy 1, policy_version 636082 (0.0007) [2023-12-27 10:02:08,333][20347] Updated weights for policy 0, policy_version 636549 (0.0008) [2023-12-27 10:02:08,399][20347] Updated weights for policy 0, policy_version 636559 (0.0008) [2023-12-27 10:02:08,464][20347] Updated weights for policy 0, policy_version 636569 (0.0008) [2023-12-27 10:02:08,998][20381] Updated weights for policy 1, policy_version 636092 (0.0007) [2023-12-27 10:02:09,048][20381] Updated weights for policy 1, policy_version 636102 (0.0008) [2023-12-27 10:02:09,097][20381] Updated weights for policy 1, policy_version 636112 (0.0008) [2023-12-27 10:02:09,175][20347] Updated weights for policy 0, policy_version 636579 (0.0007) [2023-12-27 10:02:09,232][20347] Updated weights for policy 0, policy_version 636589 (0.0008) [2023-12-27 10:02:09,288][20347] Updated weights for policy 0, policy_version 636599 (0.0007) [2023-12-27 10:02:09,824][20381] Updated weights for policy 1, policy_version 636122 (0.0008) [2023-12-27 10:02:09,881][20381] Updated weights for policy 1, policy_version 636132 (0.0007) [2023-12-27 10:02:09,940][20381] Updated weights for policy 1, policy_version 636142 (0.0007) [2023-12-27 10:02:10,000][20381] Updated weights for policy 1, policy_version 636152 (0.0005) [2023-12-27 10:02:10,048][20347] Updated weights for policy 0, policy_version 636609 (0.0008) [2023-12-27 10:02:10,115][20347] Updated weights for policy 0, policy_version 636619 (0.0007) [2023-12-27 10:02:10,180][20347] Updated weights for policy 0, policy_version 636629 (0.0008) [2023-12-27 10:02:10,237][20347] Updated weights for policy 0, policy_version 636639 (0.0008) [2023-12-27 10:02:10,632][20381] Updated weights for policy 1, policy_version 636162 (0.0007) [2023-12-27 10:02:10,681][20381] Updated weights for policy 1, policy_version 636172 (0.0007) [2023-12-27 10:02:10,735][20381] Updated weights for policy 1, policy_version 636182 (0.0008) [2023-12-27 10:02:10,882][20347] Updated weights for policy 0, policy_version 636649 (0.0008) [2023-12-27 10:02:10,938][20347] Updated weights for policy 0, policy_version 636659 (0.0008) [2023-12-27 10:02:10,988][20347] Updated weights for policy 0, policy_version 636669 (0.0008) [2023-12-27 10:02:11,428][20381] Updated weights for policy 1, policy_version 636192 (0.0008) [2023-12-27 10:02:11,488][20381] Updated weights for policy 1, policy_version 636202 (0.0008) [2023-12-27 10:02:11,547][20381] Updated weights for policy 1, policy_version 636212 (0.0008) [2023-12-27 10:02:11,713][20347] Updated weights for policy 0, policy_version 636679 (0.0006) [2023-12-27 10:02:11,780][20347] Updated weights for policy 0, policy_version 636689 (0.0008) [2023-12-27 10:02:11,835][20347] Updated weights for policy 0, policy_version 636699 (0.0008) [2023-12-27 10:02:12,219][20381] Updated weights for policy 1, policy_version 636222 (0.0007) [2023-12-27 10:02:12,283][20381] Updated weights for policy 1, policy_version 636232 (0.0007) [2023-12-27 10:02:12,344][20381] Updated weights for policy 1, policy_version 636242 (0.0007) [2023-12-27 10:02:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 325910528. Throughput: 0: 10470.3, 1: 10402.0. Samples: 325905784. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:12,363][19170] Avg episode reward: [(0, '932.470'), (1, '186.080')] [2023-12-27 10:02:12,536][20347] Updated weights for policy 0, policy_version 636709 (0.0007) [2023-12-27 10:02:12,592][20347] Updated weights for policy 0, policy_version 636719 (0.0006) [2023-12-27 10:02:12,652][20347] Updated weights for policy 0, policy_version 636729 (0.0007) [2023-12-27 10:02:13,070][20381] Updated weights for policy 1, policy_version 636252 (0.0007) [2023-12-27 10:02:13,125][20381] Updated weights for policy 1, policy_version 636262 (0.0005) [2023-12-27 10:02:13,187][20381] Updated weights for policy 1, policy_version 636272 (0.0005) [2023-12-27 10:02:13,288][20347] Updated weights for policy 0, policy_version 636739 (0.0007) [2023-12-27 10:02:13,343][20347] Updated weights for policy 0, policy_version 636749 (0.0006) [2023-12-27 10:02:13,398][20347] Updated weights for policy 0, policy_version 636759 (0.0005) [2023-12-27 10:02:13,742][20381] Updated weights for policy 1, policy_version 636282 (0.0006) [2023-12-27 10:02:13,800][20381] Updated weights for policy 1, policy_version 636292 (0.0005) [2023-12-27 10:02:13,864][20381] Updated weights for policy 1, policy_version 636302 (0.0005) [2023-12-27 10:02:13,922][20381] Updated weights for policy 1, policy_version 636312 (0.0007) [2023-12-27 10:02:14,012][20347] Updated weights for policy 0, policy_version 636769 (0.0005) [2023-12-27 10:02:14,073][20347] Updated weights for policy 0, policy_version 636779 (0.0005) [2023-12-27 10:02:14,135][20347] Updated weights for policy 0, policy_version 636789 (0.0005) [2023-12-27 10:02:14,194][20347] Updated weights for policy 0, policy_version 636799 (0.0008) [2023-12-27 10:02:14,610][20381] Updated weights for policy 1, policy_version 636322 (0.0007) [2023-12-27 10:02:14,663][20381] Updated weights for policy 1, policy_version 636332 (0.0008) [2023-12-27 10:02:14,715][20381] Updated weights for policy 1, policy_version 636342 (0.0007) [2023-12-27 10:02:14,815][20347] Updated weights for policy 0, policy_version 636809 (0.0008) [2023-12-27 10:02:14,880][20347] Updated weights for policy 0, policy_version 636819 (0.0008) [2023-12-27 10:02:14,940][20347] Updated weights for policy 0, policy_version 636829 (0.0007) [2023-12-27 10:02:15,471][20381] Updated weights for policy 1, policy_version 636352 (0.0007) [2023-12-27 10:02:15,534][20381] Updated weights for policy 1, policy_version 636362 (0.0008) [2023-12-27 10:02:15,560][20347] Updated weights for policy 0, policy_version 636839 (0.0007) [2023-12-27 10:02:15,589][20381] Updated weights for policy 1, policy_version 636372 (0.0006) [2023-12-27 10:02:15,621][20347] Updated weights for policy 0, policy_version 636849 (0.0006) [2023-12-27 10:02:15,688][20347] Updated weights for policy 0, policy_version 636859 (0.0006) [2023-12-27 10:02:16,242][20347] Updated weights for policy 0, policy_version 636869 (0.0008) [2023-12-27 10:02:16,269][20381] Updated weights for policy 1, policy_version 636382 (0.0007) [2023-12-27 10:02:16,308][20347] Updated weights for policy 0, policy_version 636879 (0.0008) [2023-12-27 10:02:16,317][20381] Updated weights for policy 1, policy_version 636392 (0.0006) [2023-12-27 10:02:16,366][20381] Updated weights for policy 1, policy_version 636402 (0.0008) [2023-12-27 10:02:16,370][20347] Updated weights for policy 0, policy_version 636889 (0.0008) [2023-12-27 10:02:16,935][20347] Updated weights for policy 0, policy_version 636899 (0.0007) [2023-12-27 10:02:16,992][20347] Updated weights for policy 0, policy_version 636909 (0.0006) [2023-12-27 10:02:17,049][20347] Updated weights for policy 0, policy_version 636919 (0.0008) [2023-12-27 10:02:17,104][20381] Updated weights for policy 1, policy_version 636412 (0.0006) [2023-12-27 10:02:17,165][20381] Updated weights for policy 1, policy_version 636422 (0.0008) [2023-12-27 10:02:17,230][20381] Updated weights for policy 1, policy_version 636432 (0.0006) [2023-12-27 10:02:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 326025216. Throughput: 0: 10458.6, 1: 10315.8. Samples: 326030876. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:17,363][19170] Avg episode reward: [(0, '932.210'), (1, '232.840')] [2023-12-27 10:02:17,623][20347] Updated weights for policy 0, policy_version 636929 (0.0008) [2023-12-27 10:02:17,682][20347] Updated weights for policy 0, policy_version 636939 (0.0007) [2023-12-27 10:02:17,740][20347] Updated weights for policy 0, policy_version 636949 (0.0007) [2023-12-27 10:02:17,807][20347] Updated weights for policy 0, policy_version 636959 (0.0005) [2023-12-27 10:02:17,904][20381] Updated weights for policy 1, policy_version 636442 (0.0006) [2023-12-27 10:02:17,953][20381] Updated weights for policy 1, policy_version 636452 (0.0008) [2023-12-27 10:02:18,002][20381] Updated weights for policy 1, policy_version 636462 (0.0009) [2023-12-27 10:02:18,055][20381] Updated weights for policy 1, policy_version 636472 (0.0009) [2023-12-27 10:02:18,354][20347] Updated weights for policy 0, policy_version 636969 (0.0008) [2023-12-27 10:02:18,406][20347] Updated weights for policy 0, policy_version 636979 (0.0005) [2023-12-27 10:02:18,460][20347] Updated weights for policy 0, policy_version 636989 (0.0008) [2023-12-27 10:02:18,762][20381] Updated weights for policy 1, policy_version 636482 (0.0008) [2023-12-27 10:02:18,814][20381] Updated weights for policy 1, policy_version 636492 (0.0009) [2023-12-27 10:02:18,876][20381] Updated weights for policy 1, policy_version 636502 (0.0007) [2023-12-27 10:02:19,087][20347] Updated weights for policy 0, policy_version 636999 (0.0006) [2023-12-27 10:02:19,138][20347] Updated weights for policy 0, policy_version 637009 (0.0006) [2023-12-27 10:02:19,199][20347] Updated weights for policy 0, policy_version 637019 (0.0005) [2023-12-27 10:02:19,640][20381] Updated weights for policy 1, policy_version 636512 (0.0007) [2023-12-27 10:02:19,693][20381] Updated weights for policy 1, policy_version 636522 (0.0008) [2023-12-27 10:02:19,756][20381] Updated weights for policy 1, policy_version 636532 (0.0008) [2023-12-27 10:02:19,843][20347] Updated weights for policy 0, policy_version 637029 (0.0008) [2023-12-27 10:02:19,900][20347] Updated weights for policy 0, policy_version 637039 (0.0008) [2023-12-27 10:02:19,959][20347] Updated weights for policy 0, policy_version 637049 (0.0007) [2023-12-27 10:02:20,415][20381] Updated weights for policy 1, policy_version 636542 (0.0006) [2023-12-27 10:02:20,468][20381] Updated weights for policy 1, policy_version 636552 (0.0007) [2023-12-27 10:02:20,527][20381] Updated weights for policy 1, policy_version 636562 (0.0008) [2023-12-27 10:02:20,621][20347] Updated weights for policy 0, policy_version 637059 (0.0008) [2023-12-27 10:02:20,676][20347] Updated weights for policy 0, policy_version 637069 (0.0008) [2023-12-27 10:02:20,725][20347] Updated weights for policy 0, policy_version 637079 (0.0008) [2023-12-27 10:02:21,248][20381] Updated weights for policy 1, policy_version 636572 (0.0008) [2023-12-27 10:02:21,306][20381] Updated weights for policy 1, policy_version 636582 (0.0008) [2023-12-27 10:02:21,380][20381] Updated weights for policy 1, policy_version 636592 (0.0008) [2023-12-27 10:02:21,481][20347] Updated weights for policy 0, policy_version 637089 (0.0008) [2023-12-27 10:02:21,538][20347] Updated weights for policy 0, policy_version 637099 (0.0008) [2023-12-27 10:02:21,594][20347] Updated weights for policy 0, policy_version 637109 (0.0007) [2023-12-27 10:02:21,657][20347] Updated weights for policy 0, policy_version 637119 (0.0008) [2023-12-27 10:02:22,092][20381] Updated weights for policy 1, policy_version 636602 (0.0008) [2023-12-27 10:02:22,152][20381] Updated weights for policy 1, policy_version 636612 (0.0007) [2023-12-27 10:02:22,212][20381] Updated weights for policy 1, policy_version 636622 (0.0007) [2023-12-27 10:02:22,269][20381] Updated weights for policy 1, policy_version 636632 (0.0006) [2023-12-27 10:02:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20771.6). Total num frames: 326123520. Throughput: 0: 10465.4, 1: 10297.8. Samples: 326094068. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:22,363][19170] Avg episode reward: [(0, '922.620'), (1, '253.770')] [2023-12-27 10:02:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000636632_163004416.pth... [2023-12-27 10:02:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000635416_162693120.pth [2023-12-27 10:02:22,417][20347] Updated weights for policy 0, policy_version 637129 (0.0006) [2023-12-27 10:02:22,477][20347] Updated weights for policy 0, policy_version 637139 (0.0007) [2023-12-27 10:02:22,534][20347] Updated weights for policy 0, policy_version 637149 (0.0007) [2023-12-27 10:02:22,553][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000637152_163127296.pth... [2023-12-27 10:02:22,558][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000635904_162807808.pth [2023-12-27 10:02:22,964][20381] Updated weights for policy 1, policy_version 636642 (0.0007) [2023-12-27 10:02:23,028][20381] Updated weights for policy 1, policy_version 636652 (0.0008) [2023-12-27 10:02:23,086][20381] Updated weights for policy 1, policy_version 636662 (0.0007) [2023-12-27 10:02:23,220][20347] Updated weights for policy 0, policy_version 637159 (0.0007) [2023-12-27 10:02:23,280][20347] Updated weights for policy 0, policy_version 637169 (0.0010) [2023-12-27 10:02:23,340][20347] Updated weights for policy 0, policy_version 637179 (0.0010) [2023-12-27 10:02:23,707][20381] Updated weights for policy 1, policy_version 636672 (0.0007) [2023-12-27 10:02:23,764][20381] Updated weights for policy 1, policy_version 636682 (0.0007) [2023-12-27 10:02:23,816][20381] Updated weights for policy 1, policy_version 636692 (0.0007) [2023-12-27 10:02:23,928][20347] Updated weights for policy 0, policy_version 637189 (0.0010) [2023-12-27 10:02:23,989][20347] Updated weights for policy 0, policy_version 637199 (0.0010) [2023-12-27 10:02:24,050][20347] Updated weights for policy 0, policy_version 637209 (0.0010) [2023-12-27 10:02:24,508][20381] Updated weights for policy 1, policy_version 636702 (0.0007) [2023-12-27 10:02:24,566][20381] Updated weights for policy 1, policy_version 636712 (0.0007) [2023-12-27 10:02:24,624][20381] Updated weights for policy 1, policy_version 636722 (0.0007) [2023-12-27 10:02:24,723][20347] Updated weights for policy 0, policy_version 637219 (0.0010) [2023-12-27 10:02:24,771][20347] Updated weights for policy 0, policy_version 637229 (0.0010) [2023-12-27 10:02:24,822][20347] Updated weights for policy 0, policy_version 637239 (0.0010) [2023-12-27 10:02:25,214][20381] Updated weights for policy 1, policy_version 636732 (0.0009) [2023-12-27 10:02:25,263][20381] Updated weights for policy 1, policy_version 636742 (0.0010) [2023-12-27 10:02:25,319][20381] Updated weights for policy 1, policy_version 636752 (0.0006) [2023-12-27 10:02:25,410][20347] Updated weights for policy 0, policy_version 637249 (0.0010) [2023-12-27 10:02:25,470][20347] Updated weights for policy 0, policy_version 637259 (0.0008) [2023-12-27 10:02:25,527][20347] Updated weights for policy 0, policy_version 637269 (0.0010) [2023-12-27 10:02:25,586][20347] Updated weights for policy 0, policy_version 637279 (0.0010) [2023-12-27 10:02:25,894][20381] Updated weights for policy 1, policy_version 636762 (0.0006) [2023-12-27 10:02:25,942][20381] Updated weights for policy 1, policy_version 636772 (0.0005) [2023-12-27 10:02:25,995][20381] Updated weights for policy 1, policy_version 636782 (0.0006) [2023-12-27 10:02:26,053][20381] Updated weights for policy 1, policy_version 636792 (0.0006) [2023-12-27 10:02:26,186][20347] Updated weights for policy 0, policy_version 637289 (0.0009) [2023-12-27 10:02:26,232][20347] Updated weights for policy 0, policy_version 637299 (0.0005) [2023-12-27 10:02:26,289][20347] Updated weights for policy 0, policy_version 637309 (0.0005) [2023-12-27 10:02:26,591][20381] Updated weights for policy 1, policy_version 636802 (0.0007) [2023-12-27 10:02:26,634][20381] Updated weights for policy 1, policy_version 636812 (0.0008) [2023-12-27 10:02:26,677][20381] Updated weights for policy 1, policy_version 636822 (0.0008) [2023-12-27 10:02:26,854][20347] Updated weights for policy 0, policy_version 637319 (0.0008) [2023-12-27 10:02:26,914][20347] Updated weights for policy 0, policy_version 637329 (0.0008) [2023-12-27 10:02:26,968][20347] Updated weights for policy 0, policy_version 637339 (0.0007) [2023-12-27 10:02:27,327][20381] Updated weights for policy 1, policy_version 636832 (0.0007) [2023-12-27 10:02:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 326230016. Throughput: 0: 10484.9, 1: 10337.4. Samples: 326220316. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:27,363][19170] Avg episode reward: [(0, '932.780'), (1, '247.570')] [2023-12-27 10:02:27,380][20381] Updated weights for policy 1, policy_version 636842 (0.0007) [2023-12-27 10:02:27,435][20381] Updated weights for policy 1, policy_version 636852 (0.0008) [2023-12-27 10:02:27,648][20347] Updated weights for policy 0, policy_version 637349 (0.0007) [2023-12-27 10:02:27,695][20347] Updated weights for policy 0, policy_version 637359 (0.0009) [2023-12-27 10:02:27,743][20347] Updated weights for policy 0, policy_version 637369 (0.0010) [2023-12-27 10:02:28,081][20381] Updated weights for policy 1, policy_version 636862 (0.0009) [2023-12-27 10:02:28,147][20381] Updated weights for policy 1, policy_version 636872 (0.0008) [2023-12-27 10:02:28,212][20381] Updated weights for policy 1, policy_version 636882 (0.0009) [2023-12-27 10:02:28,445][20347] Updated weights for policy 0, policy_version 637379 (0.0008) [2023-12-27 10:02:28,495][20347] Updated weights for policy 0, policy_version 637389 (0.0005) [2023-12-27 10:02:28,545][20347] Updated weights for policy 0, policy_version 637399 (0.0006) [2023-12-27 10:02:28,900][20381] Updated weights for policy 1, policy_version 636892 (0.0010) [2023-12-27 10:02:28,958][20381] Updated weights for policy 1, policy_version 636902 (0.0005) [2023-12-27 10:02:29,022][20381] Updated weights for policy 1, policy_version 636912 (0.0006) [2023-12-27 10:02:29,089][20347] Updated weights for policy 0, policy_version 637409 (0.0005) [2023-12-27 10:02:29,138][20347] Updated weights for policy 0, policy_version 637419 (0.0005) [2023-12-27 10:02:29,182][20347] Updated weights for policy 0, policy_version 637429 (0.0005) [2023-12-27 10:02:29,243][20347] Updated weights for policy 0, policy_version 637439 (0.0007) [2023-12-27 10:02:29,641][20381] Updated weights for policy 1, policy_version 636922 (0.0006) [2023-12-27 10:02:29,700][20381] Updated weights for policy 1, policy_version 636932 (0.0006) [2023-12-27 10:02:29,750][20381] Updated weights for policy 1, policy_version 636942 (0.0008) [2023-12-27 10:02:29,802][20381] Updated weights for policy 1, policy_version 636952 (0.0010) [2023-12-27 10:02:29,847][20347] Updated weights for policy 0, policy_version 637449 (0.0007) [2023-12-27 10:02:29,903][20347] Updated weights for policy 0, policy_version 637459 (0.0008) [2023-12-27 10:02:29,965][20347] Updated weights for policy 0, policy_version 637469 (0.0008) [2023-12-27 10:02:30,404][20381] Updated weights for policy 1, policy_version 636962 (0.0008) [2023-12-27 10:02:30,450][20381] Updated weights for policy 1, policy_version 636972 (0.0009) [2023-12-27 10:02:30,502][20381] Updated weights for policy 1, policy_version 636982 (0.0006) [2023-12-27 10:02:30,615][20347] Updated weights for policy 0, policy_version 637479 (0.0007) [2023-12-27 10:02:30,674][20347] Updated weights for policy 0, policy_version 637489 (0.0008) [2023-12-27 10:02:30,727][20347] Updated weights for policy 0, policy_version 637499 (0.0007) [2023-12-27 10:02:31,209][20381] Updated weights for policy 1, policy_version 636992 (0.0008) [2023-12-27 10:02:31,275][20381] Updated weights for policy 1, policy_version 637002 (0.0008) [2023-12-27 10:02:31,325][20381] Updated weights for policy 1, policy_version 637012 (0.0009) [2023-12-27 10:02:31,387][20347] Updated weights for policy 0, policy_version 637509 (0.0007) [2023-12-27 10:02:31,447][20347] Updated weights for policy 0, policy_version 637519 (0.0008) [2023-12-27 10:02:31,514][20347] Updated weights for policy 0, policy_version 637529 (0.0008) [2023-12-27 10:02:31,989][20381] Updated weights for policy 1, policy_version 637022 (0.0008) [2023-12-27 10:02:32,045][20381] Updated weights for policy 1, policy_version 637032 (0.0008) [2023-12-27 10:02:32,101][20381] Updated weights for policy 1, policy_version 637042 (0.0008) [2023-12-27 10:02:32,216][20347] Updated weights for policy 0, policy_version 637539 (0.0009) [2023-12-27 10:02:32,279][20347] Updated weights for policy 0, policy_version 637549 (0.0007) [2023-12-27 10:02:32,342][20347] Updated weights for policy 0, policy_version 637559 (0.0008) [2023-12-27 10:02:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 326336512. Throughput: 0: 10509.2, 1: 10421.5. Samples: 326348644. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:32,363][19170] Avg episode reward: [(0, '903.730'), (1, '272.170')] [2023-12-27 10:02:32,773][20381] Updated weights for policy 1, policy_version 637052 (0.0007) [2023-12-27 10:02:32,826][20381] Updated weights for policy 1, policy_version 637062 (0.0005) [2023-12-27 10:02:32,877][20381] Updated weights for policy 1, policy_version 637072 (0.0005) [2023-12-27 10:02:33,037][20347] Updated weights for policy 0, policy_version 637569 (0.0007) [2023-12-27 10:02:33,083][20347] Updated weights for policy 0, policy_version 637579 (0.0006) [2023-12-27 10:02:33,134][20347] Updated weights for policy 0, policy_version 637589 (0.0005) [2023-12-27 10:02:33,191][20347] Updated weights for policy 0, policy_version 637599 (0.0007) [2023-12-27 10:02:33,494][20381] Updated weights for policy 1, policy_version 637082 (0.0005) [2023-12-27 10:02:33,553][20381] Updated weights for policy 1, policy_version 637092 (0.0008) [2023-12-27 10:02:33,605][20381] Updated weights for policy 1, policy_version 637102 (0.0006) [2023-12-27 10:02:33,657][20381] Updated weights for policy 1, policy_version 637112 (0.0008) [2023-12-27 10:02:33,854][20347] Updated weights for policy 0, policy_version 637609 (0.0008) [2023-12-27 10:02:33,898][20347] Updated weights for policy 0, policy_version 637619 (0.0008) [2023-12-27 10:02:33,950][20347] Updated weights for policy 0, policy_version 637629 (0.0007) [2023-12-27 10:02:34,223][20381] Updated weights for policy 1, policy_version 637122 (0.0006) [2023-12-27 10:02:34,279][20381] Updated weights for policy 1, policy_version 637132 (0.0008) [2023-12-27 10:02:34,331][20381] Updated weights for policy 1, policy_version 637142 (0.0008) [2023-12-27 10:02:34,653][20347] Updated weights for policy 0, policy_version 637639 (0.0007) [2023-12-27 10:02:34,708][20347] Updated weights for policy 0, policy_version 637649 (0.0005) [2023-12-27 10:02:34,768][20347] Updated weights for policy 0, policy_version 637659 (0.0005) [2023-12-27 10:02:35,069][20381] Updated weights for policy 1, policy_version 637152 (0.0007) [2023-12-27 10:02:35,118][20381] Updated weights for policy 1, policy_version 637162 (0.0006) [2023-12-27 10:02:35,172][20381] Updated weights for policy 1, policy_version 637172 (0.0007) [2023-12-27 10:02:35,351][20347] Updated weights for policy 0, policy_version 637669 (0.0005) [2023-12-27 10:02:35,400][20291] KL-divergence is very high: 138.8192 [2023-12-27 10:02:35,409][20347] Updated weights for policy 0, policy_version 637679 (0.0005) [2023-12-27 10:02:35,472][20347] Updated weights for policy 0, policy_version 637689 (0.0005) [2023-12-27 10:02:35,496][20291] KL-divergence is very high: 105.2875 [2023-12-27 10:02:35,855][20381] Updated weights for policy 1, policy_version 637182 (0.0007) [2023-12-27 10:02:35,913][20381] Updated weights for policy 1, policy_version 637192 (0.0006) [2023-12-27 10:02:35,971][20381] Updated weights for policy 1, policy_version 637202 (0.0005) [2023-12-27 10:02:36,009][20347] Updated weights for policy 0, policy_version 637699 (0.0006) [2023-12-27 10:02:36,054][20347] Updated weights for policy 0, policy_version 637709 (0.0009) [2023-12-27 10:02:36,101][20347] Updated weights for policy 0, policy_version 637719 (0.0008) [2023-12-27 10:02:36,630][20381] Updated weights for policy 1, policy_version 637212 (0.0007) [2023-12-27 10:02:36,693][20381] Updated weights for policy 1, policy_version 637222 (0.0008) [2023-12-27 10:02:36,751][20381] Updated weights for policy 1, policy_version 637232 (0.0007) [2023-12-27 10:02:36,809][20347] Updated weights for policy 0, policy_version 637729 (0.0008) [2023-12-27 10:02:36,865][20347] Updated weights for policy 0, policy_version 637739 (0.0011) [2023-12-27 10:02:36,919][20347] Updated weights for policy 0, policy_version 637749 (0.0011) [2023-12-27 10:02:36,970][20347] Updated weights for policy 0, policy_version 637759 (0.0010) [2023-12-27 10:02:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 326443008. Throughput: 0: 10500.2, 1: 10431.5. Samples: 326412172. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:37,363][19170] Avg episode reward: [(0, '827.130'), (1, '300.400')] [2023-12-27 10:02:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000637240_163160064.pth... [2023-12-27 10:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000637760_163282944.pth... [2023-12-27 10:02:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000636024_162848768.pth [2023-12-27 10:02:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000636512_162963456.pth [2023-12-27 10:02:37,457][20381] Updated weights for policy 1, policy_version 637242 (0.0007) [2023-12-27 10:02:37,510][20381] Updated weights for policy 1, policy_version 637252 (0.0006) [2023-12-27 10:02:37,559][20381] Updated weights for policy 1, policy_version 637262 (0.0008) [2023-12-27 10:02:37,620][20381] Updated weights for policy 1, policy_version 637272 (0.0008) [2023-12-27 10:02:37,678][20347] Updated weights for policy 0, policy_version 637769 (0.0010) [2023-12-27 10:02:37,735][20347] Updated weights for policy 0, policy_version 637779 (0.0010) [2023-12-27 10:02:37,786][20347] Updated weights for policy 0, policy_version 637789 (0.0010) [2023-12-27 10:02:38,236][20381] Updated weights for policy 1, policy_version 637282 (0.0010) [2023-12-27 10:02:38,290][20381] Updated weights for policy 1, policy_version 637292 (0.0010) [2023-12-27 10:02:38,346][20381] Updated weights for policy 1, policy_version 637302 (0.0011) [2023-12-27 10:02:38,459][20347] Updated weights for policy 0, policy_version 637799 (0.0010) [2023-12-27 10:02:38,524][20347] Updated weights for policy 0, policy_version 637809 (0.0010) [2023-12-27 10:02:38,591][20347] Updated weights for policy 0, policy_version 637819 (0.0010) [2023-12-27 10:02:38,938][20381] Updated weights for policy 1, policy_version 637312 (0.0010) [2023-12-27 10:02:38,984][20381] Updated weights for policy 1, policy_version 637322 (0.0011) [2023-12-27 10:02:39,037][20381] Updated weights for policy 1, policy_version 637332 (0.0011) [2023-12-27 10:02:39,242][20347] Updated weights for policy 0, policy_version 637829 (0.0010) [2023-12-27 10:02:39,297][20347] Updated weights for policy 0, policy_version 637839 (0.0006) [2023-12-27 10:02:39,361][20347] Updated weights for policy 0, policy_version 637849 (0.0010) [2023-12-27 10:02:39,695][20381] Updated weights for policy 1, policy_version 637342 (0.0008) [2023-12-27 10:02:39,756][20381] Updated weights for policy 1, policy_version 637352 (0.0006) [2023-12-27 10:02:39,814][20381] Updated weights for policy 1, policy_version 637362 (0.0008) [2023-12-27 10:02:40,083][20347] Updated weights for policy 0, policy_version 637859 (0.0009) [2023-12-27 10:02:40,116][20291] KL-divergence is very high: 108.1108 [2023-12-27 10:02:40,124][20291] KL-divergence is very high: 102.2044 [2023-12-27 10:02:40,141][20347] Updated weights for policy 0, policy_version 637869 (0.0006) [2023-12-27 10:02:40,205][20347] Updated weights for policy 0, policy_version 637879 (0.0006) [2023-12-27 10:02:40,546][20381] Updated weights for policy 1, policy_version 637372 (0.0008) [2023-12-27 10:02:40,610][20381] Updated weights for policy 1, policy_version 637382 (0.0007) [2023-12-27 10:02:40,673][20381] Updated weights for policy 1, policy_version 637392 (0.0007) [2023-12-27 10:02:40,823][20347] Updated weights for policy 0, policy_version 637889 (0.0007) [2023-12-27 10:02:40,871][20347] Updated weights for policy 0, policy_version 637899 (0.0005) [2023-12-27 10:02:40,923][20347] Updated weights for policy 0, policy_version 637909 (0.0009) [2023-12-27 10:02:40,974][20347] Updated weights for policy 0, policy_version 637919 (0.0010) [2023-12-27 10:02:41,316][20381] Updated weights for policy 1, policy_version 637402 (0.0008) [2023-12-27 10:02:41,380][20381] Updated weights for policy 1, policy_version 637412 (0.0008) [2023-12-27 10:02:41,430][20381] Updated weights for policy 1, policy_version 637422 (0.0008) [2023-12-27 10:02:41,487][20381] Updated weights for policy 1, policy_version 637432 (0.0008) [2023-12-27 10:02:41,711][20347] Updated weights for policy 0, policy_version 637929 (0.0008) [2023-12-27 10:02:41,773][20347] Updated weights for policy 0, policy_version 637939 (0.0009) [2023-12-27 10:02:41,832][20347] Updated weights for policy 0, policy_version 637949 (0.0009) [2023-12-27 10:02:42,241][20381] Updated weights for policy 1, policy_version 637442 (0.0008) [2023-12-27 10:02:42,303][20381] Updated weights for policy 1, policy_version 637452 (0.0008) [2023-12-27 10:02:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 326541312. Throughput: 0: 10512.1, 1: 10431.0. Samples: 326537732. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:42,363][19170] Avg episode reward: [(0, '807.810'), (1, '346.540')] [2023-12-27 10:02:42,364][20381] Updated weights for policy 1, policy_version 637462 (0.0007) [2023-12-27 10:02:42,478][20347] Updated weights for policy 0, policy_version 637959 (0.0009) [2023-12-27 10:02:42,539][20347] Updated weights for policy 0, policy_version 637969 (0.0008) [2023-12-27 10:02:42,605][20347] Updated weights for policy 0, policy_version 637979 (0.0006) [2023-12-27 10:02:43,089][20381] Updated weights for policy 1, policy_version 637472 (0.0008) [2023-12-27 10:02:43,138][20381] Updated weights for policy 1, policy_version 637482 (0.0008) [2023-12-27 10:02:43,179][20347] Updated weights for policy 0, policy_version 637989 (0.0008) [2023-12-27 10:02:43,185][20381] Updated weights for policy 1, policy_version 637492 (0.0005) [2023-12-27 10:02:43,227][20347] Updated weights for policy 0, policy_version 637999 (0.0010) [2023-12-27 10:02:43,288][20347] Updated weights for policy 0, policy_version 638009 (0.0009) [2023-12-27 10:02:43,905][20381] Updated weights for policy 1, policy_version 637502 (0.0008) [2023-12-27 10:02:43,919][20347] Updated weights for policy 0, policy_version 638019 (0.0010) [2023-12-27 10:02:43,961][20381] Updated weights for policy 1, policy_version 637512 (0.0007) [2023-12-27 10:02:43,983][20347] Updated weights for policy 0, policy_version 638029 (0.0008) [2023-12-27 10:02:44,024][20381] Updated weights for policy 1, policy_version 637522 (0.0008) [2023-12-27 10:02:44,046][20347] Updated weights for policy 0, policy_version 638039 (0.0010) [2023-12-27 10:02:44,703][20381] Updated weights for policy 1, policy_version 637532 (0.0011) [2023-12-27 10:02:44,713][20347] Updated weights for policy 0, policy_version 638049 (0.0008) [2023-12-27 10:02:44,759][20347] Updated weights for policy 0, policy_version 638059 (0.0007) [2023-12-27 10:02:44,763][20381] Updated weights for policy 1, policy_version 637542 (0.0010) [2023-12-27 10:02:44,824][20347] Updated weights for policy 0, policy_version 638069 (0.0008) [2023-12-27 10:02:44,825][20381] Updated weights for policy 1, policy_version 637552 (0.0009) [2023-12-27 10:02:44,888][20347] Updated weights for policy 0, policy_version 638079 (0.0008) [2023-12-27 10:02:45,483][20381] Updated weights for policy 1, policy_version 637562 (0.0011) [2023-12-27 10:02:45,532][20381] Updated weights for policy 1, policy_version 637572 (0.0008) [2023-12-27 10:02:45,573][20347] Updated weights for policy 0, policy_version 638089 (0.0006) [2023-12-27 10:02:45,586][20381] Updated weights for policy 1, policy_version 637582 (0.0005) [2023-12-27 10:02:45,630][20347] Updated weights for policy 0, policy_version 638099 (0.0008) [2023-12-27 10:02:45,635][20381] Updated weights for policy 1, policy_version 637592 (0.0005) [2023-12-27 10:02:45,674][20347] Updated weights for policy 0, policy_version 638109 (0.0008) [2023-12-27 10:02:46,200][20381] Updated weights for policy 1, policy_version 637602 (0.0009) [2023-12-27 10:02:46,259][20381] Updated weights for policy 1, policy_version 637612 (0.0006) [2023-12-27 10:02:46,313][20381] Updated weights for policy 1, policy_version 637622 (0.0006) [2023-12-27 10:02:46,444][20347] Updated weights for policy 0, policy_version 638119 (0.0007) [2023-12-27 10:02:46,494][20347] Updated weights for policy 0, policy_version 638129 (0.0007) [2023-12-27 10:02:46,543][20291] KL-divergence is very high: 104.7764 [2023-12-27 10:02:46,549][20347] Updated weights for policy 0, policy_version 638139 (0.0007) [2023-12-27 10:02:46,916][20381] Updated weights for policy 1, policy_version 637632 (0.0008) [2023-12-27 10:02:46,964][20381] Updated weights for policy 1, policy_version 637642 (0.0007) [2023-12-27 10:02:47,025][20381] Updated weights for policy 1, policy_version 637652 (0.0007) [2023-12-27 10:02:47,256][20347] Updated weights for policy 0, policy_version 638149 (0.0007) [2023-12-27 10:02:47,302][20347] Updated weights for policy 0, policy_version 638159 (0.0005) [2023-12-27 10:02:47,347][20347] Updated weights for policy 0, policy_version 638169 (0.0007) [2023-12-27 10:02:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.2, 300 sec: 20799.4). Total num frames: 326647808. Throughput: 0: 10506.0, 1: 10396.9. Samples: 326660684. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:47,363][19170] Avg episode reward: [(0, '875.000'), (1, '294.580')] [2023-12-27 10:02:47,721][20381] Updated weights for policy 1, policy_version 637662 (0.0006) [2023-12-27 10:02:47,776][20381] Updated weights for policy 1, policy_version 637672 (0.0008) [2023-12-27 10:02:47,833][20381] Updated weights for policy 1, policy_version 637682 (0.0006) [2023-12-27 10:02:48,042][20347] Updated weights for policy 0, policy_version 638179 (0.0010) [2023-12-27 10:02:48,090][20347] Updated weights for policy 0, policy_version 638189 (0.0010) [2023-12-27 10:02:48,136][20347] Updated weights for policy 0, policy_version 638199 (0.0006) [2023-12-27 10:02:48,458][20381] Updated weights for policy 1, policy_version 637692 (0.0006) [2023-12-27 10:02:48,510][20381] Updated weights for policy 1, policy_version 637702 (0.0008) [2023-12-27 10:02:48,563][20381] Updated weights for policy 1, policy_version 637712 (0.0010) [2023-12-27 10:02:48,773][20347] Updated weights for policy 0, policy_version 638209 (0.0006) [2023-12-27 10:02:48,824][20347] Updated weights for policy 0, policy_version 638219 (0.0010) [2023-12-27 10:02:48,878][20347] Updated weights for policy 0, policy_version 638229 (0.0010) [2023-12-27 10:02:48,936][20347] Updated weights for policy 0, policy_version 638239 (0.0010) [2023-12-27 10:02:49,228][20381] Updated weights for policy 1, policy_version 637722 (0.0011) [2023-12-27 10:02:49,289][20381] Updated weights for policy 1, policy_version 637732 (0.0008) [2023-12-27 10:02:49,356][20381] Updated weights for policy 1, policy_version 637742 (0.0010) [2023-12-27 10:02:49,413][20381] Updated weights for policy 1, policy_version 637752 (0.0011) [2023-12-27 10:02:49,578][20347] Updated weights for policy 0, policy_version 638249 (0.0009) [2023-12-27 10:02:49,639][20347] Updated weights for policy 0, policy_version 638259 (0.0007) [2023-12-27 10:02:49,692][20347] Updated weights for policy 0, policy_version 638269 (0.0009) [2023-12-27 10:02:50,108][20381] Updated weights for policy 1, policy_version 637762 (0.0007) [2023-12-27 10:02:50,171][20381] Updated weights for policy 1, policy_version 637772 (0.0011) [2023-12-27 10:02:50,232][20381] Updated weights for policy 1, policy_version 637782 (0.0010) [2023-12-27 10:02:50,355][20347] Updated weights for policy 0, policy_version 638279 (0.0009) [2023-12-27 10:02:50,408][20347] Updated weights for policy 0, policy_version 638289 (0.0010) [2023-12-27 10:02:50,462][20347] Updated weights for policy 0, policy_version 638299 (0.0010) [2023-12-27 10:02:50,898][20381] Updated weights for policy 1, policy_version 637792 (0.0011) [2023-12-27 10:02:50,960][20381] Updated weights for policy 1, policy_version 637802 (0.0011) [2023-12-27 10:02:51,016][20381] Updated weights for policy 1, policy_version 637812 (0.0011) [2023-12-27 10:02:51,174][20347] Updated weights for policy 0, policy_version 638309 (0.0009) [2023-12-27 10:02:51,232][20347] Updated weights for policy 0, policy_version 638319 (0.0007) [2023-12-27 10:02:51,294][20347] Updated weights for policy 0, policy_version 638329 (0.0008) [2023-12-27 10:02:51,693][20381] Updated weights for policy 1, policy_version 637822 (0.0009) [2023-12-27 10:02:51,754][20381] Updated weights for policy 1, policy_version 637832 (0.0009) [2023-12-27 10:02:51,811][20381] Updated weights for policy 1, policy_version 637843 (0.0009) [2023-12-27 10:02:52,092][20347] Updated weights for policy 0, policy_version 638339 (0.0009) [2023-12-27 10:02:52,141][20347] Updated weights for policy 0, policy_version 638349 (0.0010) [2023-12-27 10:02:52,197][20347] Updated weights for policy 0, policy_version 638359 (0.0010) [2023-12-27 10:02:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 326754304. Throughput: 0: 10501.9, 1: 10397.5. Samples: 326723372. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:52,363][19170] Avg episode reward: [(0, '903.990'), (1, '335.510')] [2023-12-27 10:02:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000637848_163315712.pth... [2023-12-27 10:02:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000638368_163438592.pth... [2023-12-27 10:02:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000636632_163004416.pth [2023-12-27 10:02:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000637152_163127296.pth [2023-12-27 10:02:52,570][20381] Updated weights for policy 1, policy_version 637853 (0.0008) [2023-12-27 10:02:52,627][20381] Updated weights for policy 1, policy_version 637863 (0.0008) [2023-12-27 10:02:52,685][20381] Updated weights for policy 1, policy_version 637873 (0.0008) [2023-12-27 10:02:52,902][20347] Updated weights for policy 0, policy_version 638369 (0.0010) [2023-12-27 10:02:52,955][20347] Updated weights for policy 0, policy_version 638379 (0.0009) [2023-12-27 10:02:53,012][20347] Updated weights for policy 0, policy_version 638389 (0.0008) [2023-12-27 10:02:53,069][20347] Updated weights for policy 0, policy_version 638399 (0.0006) [2023-12-27 10:02:53,321][20381] Updated weights for policy 1, policy_version 637883 (0.0007) [2023-12-27 10:02:53,370][20381] Updated weights for policy 1, policy_version 637893 (0.0006) [2023-12-27 10:02:53,427][20381] Updated weights for policy 1, policy_version 637903 (0.0006) [2023-12-27 10:02:53,710][20347] Updated weights for policy 0, policy_version 638409 (0.0009) [2023-12-27 10:02:53,752][20347] Updated weights for policy 0, policy_version 638419 (0.0010) [2023-12-27 10:02:53,801][20347] Updated weights for policy 0, policy_version 638429 (0.0010) [2023-12-27 10:02:54,098][20381] Updated weights for policy 1, policy_version 637913 (0.0007) [2023-12-27 10:02:54,153][20381] Updated weights for policy 1, policy_version 637923 (0.0006) [2023-12-27 10:02:54,215][20381] Updated weights for policy 1, policy_version 637933 (0.0007) [2023-12-27 10:02:54,275][20381] Updated weights for policy 1, policy_version 637943 (0.0007) [2023-12-27 10:02:54,459][20347] Updated weights for policy 0, policy_version 638439 (0.0009) [2023-12-27 10:02:54,518][20347] Updated weights for policy 0, policy_version 638449 (0.0007) [2023-12-27 10:02:54,574][20347] Updated weights for policy 0, policy_version 638459 (0.0010) [2023-12-27 10:02:54,966][20381] Updated weights for policy 1, policy_version 637953 (0.0007) [2023-12-27 10:02:55,030][20381] Updated weights for policy 1, policy_version 637963 (0.0007) [2023-12-27 10:02:55,096][20381] Updated weights for policy 1, policy_version 637973 (0.0005) [2023-12-27 10:02:55,233][20347] Updated weights for policy 0, policy_version 638469 (0.0009) [2023-12-27 10:02:55,287][20347] Updated weights for policy 0, policy_version 638479 (0.0010) [2023-12-27 10:02:55,347][20347] Updated weights for policy 0, policy_version 638489 (0.0010) [2023-12-27 10:02:55,638][20381] Updated weights for policy 1, policy_version 637983 (0.0006) [2023-12-27 10:02:55,697][20381] Updated weights for policy 1, policy_version 637993 (0.0006) [2023-12-27 10:02:55,752][20381] Updated weights for policy 1, policy_version 638003 (0.0005) [2023-12-27 10:02:55,967][20347] Updated weights for policy 0, policy_version 638499 (0.0010) [2023-12-27 10:02:56,014][20347] Updated weights for policy 0, policy_version 638509 (0.0010) [2023-12-27 10:02:56,085][20347] Updated weights for policy 0, policy_version 638519 (0.0010) [2023-12-27 10:02:56,328][20381] Updated weights for policy 1, policy_version 638013 (0.0005) [2023-12-27 10:02:56,376][20381] Updated weights for policy 1, policy_version 638023 (0.0005) [2023-12-27 10:02:56,426][20381] Updated weights for policy 1, policy_version 638033 (0.0005) [2023-12-27 10:02:56,735][20347] Updated weights for policy 0, policy_version 638529 (0.0010) [2023-12-27 10:02:56,785][20347] Updated weights for policy 0, policy_version 638539 (0.0010) [2023-12-27 10:02:56,830][20347] Updated weights for policy 0, policy_version 638549 (0.0010) [2023-12-27 10:02:56,874][20347] Updated weights for policy 0, policy_version 638559 (0.0010) [2023-12-27 10:02:56,954][20381] Updated weights for policy 1, policy_version 638043 (0.0006) [2023-12-27 10:02:57,001][20381] Updated weights for policy 1, policy_version 638053 (0.0007) [2023-12-27 10:02:57,063][20381] Updated weights for policy 1, policy_version 638063 (0.0007) [2023-12-27 10:02:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 326860800. Throughput: 0: 10525.5, 1: 10401.3. Samples: 326847492. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:02:57,363][19170] Avg episode reward: [(0, '818.860'), (1, '398.670')] [2023-12-27 10:02:57,568][20347] Updated weights for policy 0, policy_version 638569 (0.0010) [2023-12-27 10:02:57,619][20347] Updated weights for policy 0, policy_version 638579 (0.0009) [2023-12-27 10:02:57,674][20347] Updated weights for policy 0, policy_version 638589 (0.0005) [2023-12-27 10:02:57,684][20381] Updated weights for policy 1, policy_version 638073 (0.0007) [2023-12-27 10:02:57,735][20381] Updated weights for policy 1, policy_version 638083 (0.0008) [2023-12-27 10:02:57,787][20381] Updated weights for policy 1, policy_version 638093 (0.0008) [2023-12-27 10:02:57,835][20381] Updated weights for policy 1, policy_version 638103 (0.0009) [2023-12-27 10:02:58,214][20347] Updated weights for policy 0, policy_version 638599 (0.0008) [2023-12-27 10:02:58,272][20347] Updated weights for policy 0, policy_version 638609 (0.0010) [2023-12-27 10:02:58,338][20347] Updated weights for policy 0, policy_version 638619 (0.0008) [2023-12-27 10:02:58,595][20381] Updated weights for policy 1, policy_version 638113 (0.0008) [2023-12-27 10:02:58,651][20381] Updated weights for policy 1, policy_version 638123 (0.0007) [2023-12-27 10:02:58,711][20381] Updated weights for policy 1, policy_version 638133 (0.0008) [2023-12-27 10:02:59,076][20347] Updated weights for policy 0, policy_version 638629 (0.0008) [2023-12-27 10:02:59,142][20347] Updated weights for policy 0, policy_version 638639 (0.0006) [2023-12-27 10:02:59,212][20347] Updated weights for policy 0, policy_version 638649 (0.0007) [2023-12-27 10:02:59,530][20381] Updated weights for policy 1, policy_version 638143 (0.0007) [2023-12-27 10:02:59,595][20381] Updated weights for policy 1, policy_version 638153 (0.0008) [2023-12-27 10:02:59,654][20381] Updated weights for policy 1, policy_version 638163 (0.0010) [2023-12-27 10:02:59,778][20347] Updated weights for policy 0, policy_version 638659 (0.0006) [2023-12-27 10:02:59,834][20347] Updated weights for policy 0, policy_version 638669 (0.0006) [2023-12-27 10:02:59,894][20347] Updated weights for policy 0, policy_version 638679 (0.0005) [2023-12-27 10:03:00,338][20381] Updated weights for policy 1, policy_version 638173 (0.0010) [2023-12-27 10:03:00,399][20381] Updated weights for policy 1, policy_version 638183 (0.0010) [2023-12-27 10:03:00,453][20381] Updated weights for policy 1, policy_version 638193 (0.0010) [2023-12-27 10:03:00,491][20347] Updated weights for policy 0, policy_version 638689 (0.0007) [2023-12-27 10:03:00,547][20347] Updated weights for policy 0, policy_version 638699 (0.0008) [2023-12-27 10:03:00,601][20347] Updated weights for policy 0, policy_version 638709 (0.0007) [2023-12-27 10:03:00,647][20347] Updated weights for policy 0, policy_version 638719 (0.0005) [2023-12-27 10:03:01,065][20381] Updated weights for policy 1, policy_version 638203 (0.0010) [2023-12-27 10:03:01,120][20381] Updated weights for policy 1, policy_version 638213 (0.0008) [2023-12-27 10:03:01,177][20381] Updated weights for policy 1, policy_version 638223 (0.0007) [2023-12-27 10:03:01,299][20347] Updated weights for policy 0, policy_version 638729 (0.0010) [2023-12-27 10:03:01,360][20347] Updated weights for policy 0, policy_version 638739 (0.0010) [2023-12-27 10:03:01,420][20347] Updated weights for policy 0, policy_version 638749 (0.0010) [2023-12-27 10:03:01,839][20381] Updated weights for policy 1, policy_version 638233 (0.0008) [2023-12-27 10:03:01,884][20381] Updated weights for policy 1, policy_version 638243 (0.0009) [2023-12-27 10:03:01,932][20381] Updated weights for policy 1, policy_version 638253 (0.0007) [2023-12-27 10:03:01,986][20381] Updated weights for policy 1, policy_version 638263 (0.0007) [2023-12-27 10:03:02,077][20347] Updated weights for policy 0, policy_version 638759 (0.0009) [2023-12-27 10:03:02,127][20347] Updated weights for policy 0, policy_version 638769 (0.0007) [2023-12-27 10:03:02,177][20347] Updated weights for policy 0, policy_version 638779 (0.0006) [2023-12-27 10:03:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 326967296. Throughput: 0: 10523.6, 1: 10437.5. Samples: 326974128. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:02,363][19170] Avg episode reward: [(0, '743.870'), (1, '359.810')] [2023-12-27 10:03:02,640][20381] Updated weights for policy 1, policy_version 638273 (0.0005) [2023-12-27 10:03:02,697][20381] Updated weights for policy 1, policy_version 638283 (0.0006) [2023-12-27 10:03:02,742][20381] Updated weights for policy 1, policy_version 638293 (0.0008) [2023-12-27 10:03:02,809][20347] Updated weights for policy 0, policy_version 638789 (0.0006) [2023-12-27 10:03:02,865][20347] Updated weights for policy 0, policy_version 638799 (0.0005) [2023-12-27 10:03:02,924][20347] Updated weights for policy 0, policy_version 638809 (0.0005) [2023-12-27 10:03:03,422][20381] Updated weights for policy 1, policy_version 638303 (0.0008) [2023-12-27 10:03:03,473][20381] Updated weights for policy 1, policy_version 638313 (0.0007) [2023-12-27 10:03:03,527][20381] Updated weights for policy 1, policy_version 638323 (0.0007) [2023-12-27 10:03:03,547][20347] Updated weights for policy 0, policy_version 638819 (0.0005) [2023-12-27 10:03:03,603][20347] Updated weights for policy 0, policy_version 638829 (0.0005) [2023-12-27 10:03:03,654][20347] Updated weights for policy 0, policy_version 638839 (0.0005) [2023-12-27 10:03:04,211][20347] Updated weights for policy 0, policy_version 638849 (0.0006) [2023-12-27 10:03:04,232][20381] Updated weights for policy 1, policy_version 638333 (0.0007) [2023-12-27 10:03:04,276][20347] Updated weights for policy 0, policy_version 638859 (0.0011) [2023-12-27 10:03:04,289][20381] Updated weights for policy 1, policy_version 638343 (0.0005) [2023-12-27 10:03:04,341][20347] Updated weights for policy 0, policy_version 638869 (0.0011) [2023-12-27 10:03:04,347][20381] Updated weights for policy 1, policy_version 638353 (0.0005) [2023-12-27 10:03:04,403][20347] Updated weights for policy 0, policy_version 638879 (0.0010) [2023-12-27 10:03:04,949][20347] Updated weights for policy 0, policy_version 638889 (0.0006) [2023-12-27 10:03:04,996][20291] KL-divergence is very high: 114.7544 [2023-12-27 10:03:05,002][20291] KL-divergence is very high: 139.3902 [2023-12-27 10:03:05,009][20347] Updated weights for policy 0, policy_version 638899 (0.0010) [2023-12-27 10:03:05,048][20381] Updated weights for policy 1, policy_version 638363 (0.0008) [2023-12-27 10:03:05,053][20291] KL-divergence is very high: 126.0040 [2023-12-27 10:03:05,071][20347] Updated weights for policy 0, policy_version 638909 (0.0005) [2023-12-27 10:03:05,105][20381] Updated weights for policy 1, policy_version 638373 (0.0008) [2023-12-27 10:03:05,158][20381] Updated weights for policy 1, policy_version 638383 (0.0009) [2023-12-27 10:03:05,679][20347] Updated weights for policy 0, policy_version 638919 (0.0008) [2023-12-27 10:03:05,686][20291] KL-divergence is very high: 104.0033 [2023-12-27 10:03:05,696][20291] KL-divergence is very high: 121.8533 [2023-12-27 10:03:05,725][20291] KL-divergence is very high: 202.7310 [2023-12-27 10:03:05,724][20347] Updated weights for policy 0, policy_version 638929 (0.0008) [2023-12-27 10:03:05,732][20291] KL-divergence is very high: 145.6405 [2023-12-27 10:03:05,772][20291] KL-divergence is very high: 114.3653 [2023-12-27 10:03:05,782][20347] Updated weights for policy 0, policy_version 638939 (0.0008) [2023-12-27 10:03:05,783][20291] KL-divergence is very high: 174.1225 [2023-12-27 10:03:05,794][20291] KL-divergence is very high: 132.8838 [2023-12-27 10:03:05,806][20381] Updated weights for policy 1, policy_version 638393 (0.0007) [2023-12-27 10:03:05,863][20381] Updated weights for policy 1, policy_version 638403 (0.0005) [2023-12-27 10:03:05,916][20381] Updated weights for policy 1, policy_version 638413 (0.0005) [2023-12-27 10:03:05,969][20381] Updated weights for policy 1, policy_version 638423 (0.0005) [2023-12-27 10:03:06,400][20347] Updated weights for policy 0, policy_version 638949 (0.0008) [2023-12-27 10:03:06,455][20347] Updated weights for policy 0, policy_version 638959 (0.0008) [2023-12-27 10:03:06,508][20347] Updated weights for policy 0, policy_version 638969 (0.0008) [2023-12-27 10:03:06,624][20381] Updated weights for policy 1, policy_version 638433 (0.0007) [2023-12-27 10:03:06,681][20381] Updated weights for policy 1, policy_version 638443 (0.0008) [2023-12-27 10:03:06,734][20381] Updated weights for policy 1, policy_version 638453 (0.0008) [2023-12-27 10:03:07,197][20347] Updated weights for policy 0, policy_version 638979 (0.0007) [2023-12-27 10:03:07,252][20291] KL-divergence is very high: 219.1087 [2023-12-27 10:03:07,256][20347] Updated weights for policy 0, policy_version 638989 (0.0007) [2023-12-27 10:03:07,258][20291] KL-divergence is very high: 116.8991 [2023-12-27 10:03:07,270][20291] KL-divergence is very high: 118.1047 [2023-12-27 10:03:07,315][20347] Updated weights for policy 0, policy_version 638999 (0.0007) [2023-12-27 10:03:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 327065600. Throughput: 0: 10542.1, 1: 10446.9. Samples: 327038576. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:07,363][19170] Avg episode reward: [(0, '477.620'), (1, '368.700')] [2023-12-27 10:03:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000638456_163471360.pth... [2023-12-27 10:03:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000639008_163602432.pth... [2023-12-27 10:03:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000637240_163160064.pth [2023-12-27 10:03:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000637760_163282944.pth [2023-12-27 10:03:07,449][20381] Updated weights for policy 1, policy_version 638463 (0.0007) [2023-12-27 10:03:07,505][20381] Updated weights for policy 1, policy_version 638473 (0.0008) [2023-12-27 10:03:07,568][20381] Updated weights for policy 1, policy_version 638483 (0.0008) [2023-12-27 10:03:07,997][20347] Updated weights for policy 0, policy_version 639009 (0.0007) [2023-12-27 10:03:08,050][20347] Updated weights for policy 0, policy_version 639019 (0.0008) [2023-12-27 10:03:08,112][20347] Updated weights for policy 0, policy_version 639029 (0.0008) [2023-12-27 10:03:08,158][20347] Updated weights for policy 0, policy_version 639039 (0.0009) [2023-12-27 10:03:08,248][20381] Updated weights for policy 1, policy_version 638493 (0.0006) [2023-12-27 10:03:08,305][20381] Updated weights for policy 1, policy_version 638503 (0.0006) [2023-12-27 10:03:08,367][20381] Updated weights for policy 1, policy_version 638513 (0.0007) [2023-12-27 10:03:08,880][20347] Updated weights for policy 0, policy_version 639049 (0.0008) [2023-12-27 10:03:08,943][20347] Updated weights for policy 0, policy_version 639059 (0.0008) [2023-12-27 10:03:09,001][20347] Updated weights for policy 0, policy_version 639069 (0.0007) [2023-12-27 10:03:09,068][20381] Updated weights for policy 1, policy_version 638523 (0.0008) [2023-12-27 10:03:09,121][20381] Updated weights for policy 1, policy_version 638533 (0.0008) [2023-12-27 10:03:09,172][20381] Updated weights for policy 1, policy_version 638543 (0.0007) [2023-12-27 10:03:09,701][20347] Updated weights for policy 0, policy_version 639079 (0.0008) [2023-12-27 10:03:09,758][20347] Updated weights for policy 0, policy_version 639089 (0.0007) [2023-12-27 10:03:09,814][20347] Updated weights for policy 0, policy_version 639099 (0.0007) [2023-12-27 10:03:09,866][20381] Updated weights for policy 1, policy_version 638553 (0.0006) [2023-12-27 10:03:09,927][20381] Updated weights for policy 1, policy_version 638563 (0.0007) [2023-12-27 10:03:09,988][20381] Updated weights for policy 1, policy_version 638573 (0.0008) [2023-12-27 10:03:10,046][20381] Updated weights for policy 1, policy_version 638583 (0.0007) [2023-12-27 10:03:10,564][20347] Updated weights for policy 0, policy_version 639109 (0.0007) [2023-12-27 10:03:10,619][20291] KL-divergence is very high: 101.1002 [2023-12-27 10:03:10,624][20347] Updated weights for policy 0, policy_version 639119 (0.0011) [2023-12-27 10:03:10,683][20347] Updated weights for policy 0, policy_version 639129 (0.0010) [2023-12-27 10:03:10,714][20381] Updated weights for policy 1, policy_version 638593 (0.0008) [2023-12-27 10:03:10,778][20381] Updated weights for policy 1, policy_version 638603 (0.0008) [2023-12-27 10:03:10,838][20381] Updated weights for policy 1, policy_version 638613 (0.0008) [2023-12-27 10:03:11,407][20347] Updated weights for policy 0, policy_version 639139 (0.0009) [2023-12-27 10:03:11,467][20347] Updated weights for policy 0, policy_version 639149 (0.0007) [2023-12-27 10:03:11,524][20347] Updated weights for policy 0, policy_version 639159 (0.0007) [2023-12-27 10:03:11,561][20381] Updated weights for policy 1, policy_version 638623 (0.0009) [2023-12-27 10:03:11,626][20381] Updated weights for policy 1, policy_version 638633 (0.0008) [2023-12-27 10:03:11,691][20381] Updated weights for policy 1, policy_version 638643 (0.0008) [2023-12-27 10:03:12,177][20347] Updated weights for policy 0, policy_version 639169 (0.0007) [2023-12-27 10:03:12,232][20347] Updated weights for policy 0, policy_version 639179 (0.0007) [2023-12-27 10:03:12,290][20347] Updated weights for policy 0, policy_version 639189 (0.0007) [2023-12-27 10:03:12,338][20381] Updated weights for policy 1, policy_version 638653 (0.0008) [2023-12-27 10:03:12,354][20347] Updated weights for policy 0, policy_version 639199 (0.0008) [2023-12-27 10:03:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.2, 300 sec: 20799.4). Total num frames: 327172096. Throughput: 0: 10500.7, 1: 10395.8. Samples: 327160660. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:12,363][19170] Avg episode reward: [(0, '580.980'), (1, '377.180')] [2023-12-27 10:03:12,399][20381] Updated weights for policy 1, policy_version 638663 (0.0006) [2023-12-27 10:03:12,456][20381] Updated weights for policy 1, policy_version 638673 (0.0007) [2023-12-27 10:03:12,996][20347] Updated weights for policy 0, policy_version 639209 (0.0006) [2023-12-27 10:03:13,060][20347] Updated weights for policy 0, policy_version 639219 (0.0008) [2023-12-27 10:03:13,064][20381] Updated weights for policy 1, policy_version 638683 (0.0007) [2023-12-27 10:03:13,116][20347] Updated weights for policy 0, policy_version 639229 (0.0009) [2023-12-27 10:03:13,119][20381] Updated weights for policy 1, policy_version 638693 (0.0005) [2023-12-27 10:03:13,179][20381] Updated weights for policy 1, policy_version 638703 (0.0005) [2023-12-27 10:03:13,685][20347] Updated weights for policy 0, policy_version 639239 (0.0008) [2023-12-27 10:03:13,733][20347] Updated weights for policy 0, policy_version 639249 (0.0008) [2023-12-27 10:03:13,777][20347] Updated weights for policy 0, policy_version 639259 (0.0007) [2023-12-27 10:03:13,786][20291] KL-divergence is very high: 129.9604 [2023-12-27 10:03:13,788][20381] Updated weights for policy 1, policy_version 638713 (0.0006) [2023-12-27 10:03:13,792][20291] KL-divergence is very high: 106.4642 [2023-12-27 10:03:13,797][20291] KL-divergence is very high: 117.1591 [2023-12-27 10:03:13,836][20381] Updated weights for policy 1, policy_version 638723 (0.0011) [2023-12-27 10:03:13,888][20381] Updated weights for policy 1, policy_version 638733 (0.0010) [2023-12-27 10:03:13,936][20381] Updated weights for policy 1, policy_version 638743 (0.0010) [2023-12-27 10:03:14,485][20347] Updated weights for policy 0, policy_version 639269 (0.0007) [2023-12-27 10:03:14,536][20347] Updated weights for policy 0, policy_version 639279 (0.0005) [2023-12-27 10:03:14,585][20347] Updated weights for policy 0, policy_version 639289 (0.0007) [2023-12-27 10:03:14,621][20381] Updated weights for policy 1, policy_version 638753 (0.0007) [2023-12-27 10:03:14,678][20381] Updated weights for policy 1, policy_version 638763 (0.0006) [2023-12-27 10:03:14,727][20381] Updated weights for policy 1, policy_version 638773 (0.0007) [2023-12-27 10:03:15,297][20347] Updated weights for policy 0, policy_version 639299 (0.0008) [2023-12-27 10:03:15,352][20347] Updated weights for policy 0, policy_version 639309 (0.0007) [2023-12-27 10:03:15,409][20347] Updated weights for policy 0, policy_version 639319 (0.0008) [2023-12-27 10:03:15,416][20381] Updated weights for policy 1, policy_version 638783 (0.0006) [2023-12-27 10:03:15,470][20381] Updated weights for policy 1, policy_version 638793 (0.0006) [2023-12-27 10:03:15,531][20381] Updated weights for policy 1, policy_version 638803 (0.0008) [2023-12-27 10:03:16,064][20347] Updated weights for policy 0, policy_version 639329 (0.0007) [2023-12-27 10:03:16,130][20347] Updated weights for policy 0, policy_version 639339 (0.0008) [2023-12-27 10:03:16,189][20381] Updated weights for policy 1, policy_version 638813 (0.0006) [2023-12-27 10:03:16,191][20347] Updated weights for policy 0, policy_version 639349 (0.0007) [2023-12-27 10:03:16,246][20381] Updated weights for policy 1, policy_version 638823 (0.0006) [2023-12-27 10:03:16,248][20347] Updated weights for policy 0, policy_version 639359 (0.0006) [2023-12-27 10:03:16,306][20381] Updated weights for policy 1, policy_version 638833 (0.0005) [2023-12-27 10:03:16,925][20381] Updated weights for policy 1, policy_version 638843 (0.0005) [2023-12-27 10:03:16,959][20347] Updated weights for policy 0, policy_version 639369 (0.0007) [2023-12-27 10:03:16,983][20381] Updated weights for policy 1, policy_version 638853 (0.0006) [2023-12-27 10:03:17,011][20347] Updated weights for policy 0, policy_version 639379 (0.0005) [2023-12-27 10:03:17,035][20381] Updated weights for policy 1, policy_version 638863 (0.0006) [2023-12-27 10:03:17,062][20347] Updated weights for policy 0, policy_version 639389 (0.0007) [2023-12-27 10:03:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 327278592. Throughput: 0: 10430.8, 1: 10393.6. Samples: 327285740. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:17,363][19170] Avg episode reward: [(0, '89.360'), (1, '447.450')] [2023-12-27 10:03:17,715][20381] Updated weights for policy 1, policy_version 638873 (0.0005) [2023-12-27 10:03:17,745][20347] Updated weights for policy 0, policy_version 639399 (0.0007) [2023-12-27 10:03:17,771][20381] Updated weights for policy 1, policy_version 638883 (0.0005) [2023-12-27 10:03:17,803][20347] Updated weights for policy 0, policy_version 639409 (0.0008) [2023-12-27 10:03:17,827][20381] Updated weights for policy 1, policy_version 638893 (0.0005) [2023-12-27 10:03:17,859][20347] Updated weights for policy 0, policy_version 639419 (0.0007) [2023-12-27 10:03:17,888][20381] Updated weights for policy 1, policy_version 638903 (0.0005) [2023-12-27 10:03:18,503][20381] Updated weights for policy 1, policy_version 638913 (0.0009) [2023-12-27 10:03:18,546][20347] Updated weights for policy 0, policy_version 639429 (0.0009) [2023-12-27 10:03:18,565][20381] Updated weights for policy 1, policy_version 638923 (0.0011) [2023-12-27 10:03:18,601][20347] Updated weights for policy 0, policy_version 639439 (0.0008) [2023-12-27 10:03:18,619][20381] Updated weights for policy 1, policy_version 638933 (0.0011) [2023-12-27 10:03:18,660][20347] Updated weights for policy 0, policy_version 639449 (0.0008) [2023-12-27 10:03:19,324][20347] Updated weights for policy 0, policy_version 639459 (0.0008) [2023-12-27 10:03:19,331][20381] Updated weights for policy 1, policy_version 638943 (0.0011) [2023-12-27 10:03:19,387][20347] Updated weights for policy 0, policy_version 639469 (0.0007) [2023-12-27 10:03:19,395][20381] Updated weights for policy 1, policy_version 638953 (0.0011) [2023-12-27 10:03:19,441][20347] Updated weights for policy 0, policy_version 639479 (0.0008) [2023-12-27 10:03:19,451][20381] Updated weights for policy 1, policy_version 638963 (0.0011) [2023-12-27 10:03:20,185][20381] Updated weights for policy 1, policy_version 638973 (0.0010) [2023-12-27 10:03:20,205][20347] Updated weights for policy 0, policy_version 639489 (0.0006) [2023-12-27 10:03:20,244][20381] Updated weights for policy 1, policy_version 638983 (0.0009) [2023-12-27 10:03:20,265][20347] Updated weights for policy 0, policy_version 639499 (0.0006) [2023-12-27 10:03:20,301][20381] Updated weights for policy 1, policy_version 638993 (0.0008) [2023-12-27 10:03:20,326][20347] Updated weights for policy 0, policy_version 639509 (0.0006) [2023-12-27 10:03:20,389][20347] Updated weights for policy 0, policy_version 639519 (0.0007) [2023-12-27 10:03:20,943][20381] Updated weights for policy 1, policy_version 639003 (0.0008) [2023-12-27 10:03:21,006][20381] Updated weights for policy 1, policy_version 639013 (0.0010) [2023-12-27 10:03:21,070][20381] Updated weights for policy 1, policy_version 639023 (0.0010) [2023-12-27 10:03:21,098][20347] Updated weights for policy 0, policy_version 639529 (0.0006) [2023-12-27 10:03:21,161][20347] Updated weights for policy 0, policy_version 639539 (0.0007) [2023-12-27 10:03:21,220][20347] Updated weights for policy 0, policy_version 639549 (0.0007) [2023-12-27 10:03:21,763][20381] Updated weights for policy 1, policy_version 639033 (0.0009) [2023-12-27 10:03:21,821][20381] Updated weights for policy 1, policy_version 639043 (0.0008) [2023-12-27 10:03:21,877][20381] Updated weights for policy 1, policy_version 639053 (0.0010) [2023-12-27 10:03:21,933][20381] Updated weights for policy 1, policy_version 639063 (0.0011) [2023-12-27 10:03:21,936][20347] Updated weights for policy 0, policy_version 639559 (0.0008) [2023-12-27 10:03:21,998][20347] Updated weights for policy 0, policy_version 639569 (0.0010) [2023-12-27 10:03:22,056][20347] Updated weights for policy 0, policy_version 639579 (0.0008) [2023-12-27 10:03:22,375][19170] Fps is (10 sec: 20454.2, 60 sec: 20885.2, 300 sec: 20798.5). Total num frames: 327376896. Throughput: 0: 10402.2, 1: 10349.7. Samples: 327346268. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:22,376][19170] Avg episode reward: [(0, '63.410'), (1, '417.940')] [2023-12-27 10:03:22,383][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000639064_163627008.pth... [2023-12-27 10:03:22,383][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000639584_163749888.pth... [2023-12-27 10:03:22,388][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000637848_163315712.pth [2023-12-27 10:03:22,388][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000638368_163438592.pth [2023-12-27 10:03:22,625][20347] Updated weights for policy 0, policy_version 639589 (0.0008) [2023-12-27 10:03:22,627][20381] Updated weights for policy 1, policy_version 639073 (0.0010) [2023-12-27 10:03:22,676][20347] Updated weights for policy 0, policy_version 639599 (0.0006) [2023-12-27 10:03:22,689][20381] Updated weights for policy 1, policy_version 639083 (0.0008) [2023-12-27 10:03:22,734][20347] Updated weights for policy 0, policy_version 639609 (0.0007) [2023-12-27 10:03:22,749][20381] Updated weights for policy 1, policy_version 639093 (0.0011) [2023-12-27 10:03:23,366][20381] Updated weights for policy 1, policy_version 639103 (0.0011) [2023-12-27 10:03:23,408][20347] Updated weights for policy 0, policy_version 639619 (0.0007) [2023-12-27 10:03:23,418][20381] Updated weights for policy 1, policy_version 639113 (0.0011) [2023-12-27 10:03:23,465][20347] Updated weights for policy 0, policy_version 639629 (0.0006) [2023-12-27 10:03:23,467][20381] Updated weights for policy 1, policy_version 639123 (0.0011) [2023-12-27 10:03:23,523][20347] Updated weights for policy 0, policy_version 639639 (0.0009) [2023-12-27 10:03:24,136][20381] Updated weights for policy 1, policy_version 639133 (0.0011) [2023-12-27 10:03:24,138][20347] Updated weights for policy 0, policy_version 639649 (0.0010) [2023-12-27 10:03:24,182][20381] Updated weights for policy 1, policy_version 639143 (0.0011) [2023-12-27 10:03:24,200][20347] Updated weights for policy 0, policy_version 639659 (0.0009) [2023-12-27 10:03:24,227][20381] Updated weights for policy 1, policy_version 639153 (0.0010) [2023-12-27 10:03:24,256][20347] Updated weights for policy 0, policy_version 639669 (0.0005) [2023-12-27 10:03:24,303][20347] Updated weights for policy 0, policy_version 639679 (0.0007) [2023-12-27 10:03:24,920][20381] Updated weights for policy 1, policy_version 639163 (0.0010) [2023-12-27 10:03:24,972][20381] Updated weights for policy 1, policy_version 639173 (0.0011) [2023-12-27 10:03:24,994][20347] Updated weights for policy 0, policy_version 639689 (0.0010) [2023-12-27 10:03:25,020][20381] Updated weights for policy 1, policy_version 639183 (0.0010) [2023-12-27 10:03:25,039][20347] Updated weights for policy 0, policy_version 639699 (0.0007) [2023-12-27 10:03:25,087][20347] Updated weights for policy 0, policy_version 639709 (0.0005) [2023-12-27 10:03:25,682][20347] Updated weights for policy 0, policy_version 639719 (0.0007) [2023-12-27 10:03:25,711][20381] Updated weights for policy 1, policy_version 639193 (0.0010) [2023-12-27 10:03:25,738][20347] Updated weights for policy 0, policy_version 639729 (0.0006) [2023-12-27 10:03:25,763][20381] Updated weights for policy 1, policy_version 639203 (0.0011) [2023-12-27 10:03:25,787][20347] Updated weights for policy 0, policy_version 639739 (0.0005) [2023-12-27 10:03:25,821][20381] Updated weights for policy 1, policy_version 639213 (0.0011) [2023-12-27 10:03:25,876][20381] Updated weights for policy 1, policy_version 639223 (0.0010) [2023-12-27 10:03:26,462][20381] Updated weights for policy 1, policy_version 639233 (0.0010) [2023-12-27 10:03:26,496][20347] Updated weights for policy 0, policy_version 639749 (0.0006) [2023-12-27 10:03:26,506][20381] Updated weights for policy 1, policy_version 639243 (0.0010) [2023-12-27 10:03:26,547][20347] Updated weights for policy 0, policy_version 639759 (0.0005) [2023-12-27 10:03:26,551][20381] Updated weights for policy 1, policy_version 639253 (0.0010) [2023-12-27 10:03:26,599][20347] Updated weights for policy 0, policy_version 639769 (0.0008) [2023-12-27 10:03:27,263][20381] Updated weights for policy 1, policy_version 639263 (0.0010) [2023-12-27 10:03:27,286][20347] Updated weights for policy 0, policy_version 639779 (0.0007) [2023-12-27 10:03:27,318][20381] Updated weights for policy 1, policy_version 639273 (0.0010) [2023-12-27 10:03:27,346][20347] Updated weights for policy 0, policy_version 639789 (0.0005) [2023-12-27 10:03:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 327475200. Throughput: 0: 10399.6, 1: 10339.1. Samples: 327470976. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:27,363][19170] Avg episode reward: [(0, '83.120'), (1, '404.860')] [2023-12-27 10:03:27,376][20381] Updated weights for policy 1, policy_version 639283 (0.0011) [2023-12-27 10:03:27,396][20347] Updated weights for policy 0, policy_version 639799 (0.0005) [2023-12-27 10:03:28,042][20381] Updated weights for policy 1, policy_version 639293 (0.0010) [2023-12-27 10:03:28,074][20347] Updated weights for policy 0, policy_version 639809 (0.0007) [2023-12-27 10:03:28,087][20381] Updated weights for policy 1, policy_version 639303 (0.0010) [2023-12-27 10:03:28,129][20347] Updated weights for policy 0, policy_version 639819 (0.0005) [2023-12-27 10:03:28,138][20381] Updated weights for policy 1, policy_version 639313 (0.0010) [2023-12-27 10:03:28,183][20347] Updated weights for policy 0, policy_version 639829 (0.0005) [2023-12-27 10:03:28,240][20347] Updated weights for policy 0, policy_version 639839 (0.0005) [2023-12-27 10:03:28,850][20381] Updated weights for policy 1, policy_version 639323 (0.0010) [2023-12-27 10:03:28,905][20381] Updated weights for policy 1, policy_version 639333 (0.0010) [2023-12-27 10:03:28,906][20347] Updated weights for policy 0, policy_version 639849 (0.0005) [2023-12-27 10:03:28,959][20347] Updated weights for policy 0, policy_version 639859 (0.0006) [2023-12-27 10:03:28,961][20381] Updated weights for policy 1, policy_version 639343 (0.0011) [2023-12-27 10:03:29,017][20347] Updated weights for policy 0, policy_version 639869 (0.0005) [2023-12-27 10:03:29,606][20381] Updated weights for policy 1, policy_version 639353 (0.0010) [2023-12-27 10:03:29,662][20381] Updated weights for policy 1, policy_version 639363 (0.0005) [2023-12-27 10:03:29,697][20347] Updated weights for policy 0, policy_version 639879 (0.0007) [2023-12-27 10:03:29,713][20381] Updated weights for policy 1, policy_version 639373 (0.0005) [2023-12-27 10:03:29,745][20347] Updated weights for policy 0, policy_version 639889 (0.0008) [2023-12-27 10:03:29,775][20381] Updated weights for policy 1, policy_version 639383 (0.0005) [2023-12-27 10:03:29,799][20347] Updated weights for policy 0, policy_version 639899 (0.0007) [2023-12-27 10:03:30,423][20381] Updated weights for policy 1, policy_version 639393 (0.0011) [2023-12-27 10:03:30,432][20347] Updated weights for policy 0, policy_version 639909 (0.0008) [2023-12-27 10:03:30,477][20381] Updated weights for policy 1, policy_version 639403 (0.0010) [2023-12-27 10:03:30,483][20347] Updated weights for policy 0, policy_version 639919 (0.0005) [2023-12-27 10:03:30,533][20347] Updated weights for policy 0, policy_version 639929 (0.0006) [2023-12-27 10:03:30,535][20381] Updated weights for policy 1, policy_version 639413 (0.0010) [2023-12-27 10:03:31,192][20347] Updated weights for policy 0, policy_version 639939 (0.0006) [2023-12-27 10:03:31,240][20381] Updated weights for policy 1, policy_version 639423 (0.0010) [2023-12-27 10:03:31,250][20347] Updated weights for policy 0, policy_version 639949 (0.0008) [2023-12-27 10:03:31,296][20381] Updated weights for policy 1, policy_version 639433 (0.0011) [2023-12-27 10:03:31,299][20347] Updated weights for policy 0, policy_version 639959 (0.0008) [2023-12-27 10:03:31,353][20381] Updated weights for policy 1, policy_version 639443 (0.0010) [2023-12-27 10:03:31,975][20347] Updated weights for policy 0, policy_version 639969 (0.0007) [2023-12-27 10:03:32,025][20347] Updated weights for policy 0, policy_version 639979 (0.0007) [2023-12-27 10:03:32,073][20381] Updated weights for policy 1, policy_version 639453 (0.0009) [2023-12-27 10:03:32,082][20347] Updated weights for policy 0, policy_version 639989 (0.0006) [2023-12-27 10:03:32,125][20381] Updated weights for policy 1, policy_version 639463 (0.0010) [2023-12-27 10:03:32,134][20347] Updated weights for policy 0, policy_version 639999 (0.0005) [2023-12-27 10:03:32,179][20381] Updated weights for policy 1, policy_version 639473 (0.0010) [2023-12-27 10:03:32,362][19170] Fps is (10 sec: 21325.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 327589888. Throughput: 0: 10426.4, 1: 10332.8. Samples: 327594852. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:32,363][19170] Avg episode reward: [(0, '562.670'), (1, '409.580')] [2023-12-27 10:03:32,803][20347] Updated weights for policy 0, policy_version 640009 (0.0007) [2023-12-27 10:03:32,835][20381] Updated weights for policy 1, policy_version 639483 (0.0007) [2023-12-27 10:03:32,856][20347] Updated weights for policy 0, policy_version 640019 (0.0008) [2023-12-27 10:03:32,895][20381] Updated weights for policy 1, policy_version 639493 (0.0005) [2023-12-27 10:03:32,913][20347] Updated weights for policy 0, policy_version 640029 (0.0007) [2023-12-27 10:03:32,953][20381] Updated weights for policy 1, policy_version 639503 (0.0010) [2023-12-27 10:03:33,555][20347] Updated weights for policy 0, policy_version 640039 (0.0005) [2023-12-27 10:03:33,585][20381] Updated weights for policy 1, policy_version 639513 (0.0011) [2023-12-27 10:03:33,613][20347] Updated weights for policy 0, policy_version 640049 (0.0005) [2023-12-27 10:03:33,635][20381] Updated weights for policy 1, policy_version 639523 (0.0007) [2023-12-27 10:03:33,668][20347] Updated weights for policy 0, policy_version 640059 (0.0006) [2023-12-27 10:03:33,686][20381] Updated weights for policy 1, policy_version 639533 (0.0005) [2023-12-27 10:03:33,731][20381] Updated weights for policy 1, policy_version 639543 (0.0005) [2023-12-27 10:03:34,234][20347] Updated weights for policy 0, policy_version 640069 (0.0008) [2023-12-27 10:03:34,288][20347] Updated weights for policy 0, policy_version 640079 (0.0010) [2023-12-27 10:03:34,316][20381] Updated weights for policy 1, policy_version 639553 (0.0006) [2023-12-27 10:03:34,343][20347] Updated weights for policy 0, policy_version 640089 (0.0010) [2023-12-27 10:03:34,375][20381] Updated weights for policy 1, policy_version 639563 (0.0006) [2023-12-27 10:03:34,434][20381] Updated weights for policy 1, policy_version 639573 (0.0007) [2023-12-27 10:03:35,072][20347] Updated weights for policy 0, policy_version 640099 (0.0009) [2023-12-27 10:03:35,092][20381] Updated weights for policy 1, policy_version 639583 (0.0006) [2023-12-27 10:03:35,121][20347] Updated weights for policy 0, policy_version 640109 (0.0005) [2023-12-27 10:03:35,141][20381] Updated weights for policy 1, policy_version 639593 (0.0006) [2023-12-27 10:03:35,173][20347] Updated weights for policy 0, policy_version 640119 (0.0005) [2023-12-27 10:03:35,196][20381] Updated weights for policy 1, policy_version 639603 (0.0008) [2023-12-27 10:03:35,842][20347] Updated weights for policy 0, policy_version 640129 (0.0006) [2023-12-27 10:03:35,844][20381] Updated weights for policy 1, policy_version 639613 (0.0006) [2023-12-27 10:03:35,894][20347] Updated weights for policy 0, policy_version 640139 (0.0007) [2023-12-27 10:03:35,910][20381] Updated weights for policy 1, policy_version 639623 (0.0005) [2023-12-27 10:03:35,950][20347] Updated weights for policy 0, policy_version 640149 (0.0006) [2023-12-27 10:03:35,968][20381] Updated weights for policy 1, policy_version 639633 (0.0007) [2023-12-27 10:03:36,009][20347] Updated weights for policy 0, policy_version 640159 (0.0006) [2023-12-27 10:03:36,558][20381] Updated weights for policy 1, policy_version 639643 (0.0008) [2023-12-27 10:03:36,616][20381] Updated weights for policy 1, policy_version 639653 (0.0008) [2023-12-27 10:03:36,679][20381] Updated weights for policy 1, policy_version 639663 (0.0007) [2023-12-27 10:03:36,730][20347] Updated weights for policy 0, policy_version 640169 (0.0008) [2023-12-27 10:03:36,786][20347] Updated weights for policy 0, policy_version 640179 (0.0007) [2023-12-27 10:03:36,839][20347] Updated weights for policy 0, policy_version 640189 (0.0008) [2023-12-27 10:03:37,314][20381] Updated weights for policy 1, policy_version 639673 (0.0006) [2023-12-27 10:03:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 327688192. Throughput: 0: 10434.8, 1: 10363.6. Samples: 327659304. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 10:03:37,363][19170] Avg episode reward: [(0, '692.000'), (1, '309.670')] [2023-12-27 10:03:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000640192_163905536.pth... [2023-12-27 10:03:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000639008_163602432.pth [2023-12-27 10:03:37,377][20381] Updated weights for policy 1, policy_version 639683 (0.0007) [2023-12-27 10:03:37,424][20381] Updated weights for policy 1, policy_version 639693 (0.0005) [2023-12-27 10:03:37,485][20381] Updated weights for policy 1, policy_version 639703 (0.0006) [2023-12-27 10:03:37,491][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000639704_163790848.pth... [2023-12-27 10:03:37,495][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000638456_163471360.pth [2023-12-27 10:03:37,574][20347] Updated weights for policy 0, policy_version 640199 (0.0009) [2023-12-27 10:03:37,619][20347] Updated weights for policy 0, policy_version 640209 (0.0007) [2023-12-27 10:03:37,674][20347] Updated weights for policy 0, policy_version 640219 (0.0008) [2023-12-27 10:03:38,032][20381] Updated weights for policy 1, policy_version 639713 (0.0007) [2023-12-27 10:03:38,092][20381] Updated weights for policy 1, policy_version 639723 (0.0008) [2023-12-27 10:03:38,153][20381] Updated weights for policy 1, policy_version 639733 (0.0008) [2023-12-27 10:03:38,298][20347] Updated weights for policy 0, policy_version 640229 (0.0005) [2023-12-27 10:03:38,357][20347] Updated weights for policy 0, policy_version 640239 (0.0008) [2023-12-27 10:03:38,410][20347] Updated weights for policy 0, policy_version 640249 (0.0007) [2023-12-27 10:03:38,886][20381] Updated weights for policy 1, policy_version 639743 (0.0008) [2023-12-27 10:03:38,933][20381] Updated weights for policy 1, policy_version 639753 (0.0008) [2023-12-27 10:03:38,979][20381] Updated weights for policy 1, policy_version 639763 (0.0008) [2023-12-27 10:03:39,103][20347] Updated weights for policy 0, policy_version 640259 (0.0007) [2023-12-27 10:03:39,154][20347] Updated weights for policy 0, policy_version 640269 (0.0008) [2023-12-27 10:03:39,215][20347] Updated weights for policy 0, policy_version 640279 (0.0008) [2023-12-27 10:03:39,744][20381] Updated weights for policy 1, policy_version 639773 (0.0007) [2023-12-27 10:03:39,800][20381] Updated weights for policy 1, policy_version 639783 (0.0008) [2023-12-27 10:03:39,865][20381] Updated weights for policy 1, policy_version 639793 (0.0007) [2023-12-27 10:03:39,923][20347] Updated weights for policy 0, policy_version 640289 (0.0008) [2023-12-27 10:03:39,976][20347] Updated weights for policy 0, policy_version 640299 (0.0007) [2023-12-27 10:03:40,037][20347] Updated weights for policy 0, policy_version 640309 (0.0008) [2023-12-27 10:03:40,093][20347] Updated weights for policy 0, policy_version 640319 (0.0008) [2023-12-27 10:03:40,556][20381] Updated weights for policy 1, policy_version 639803 (0.0008) [2023-12-27 10:03:40,611][20381] Updated weights for policy 1, policy_version 639813 (0.0007) [2023-12-27 10:03:40,669][20381] Updated weights for policy 1, policy_version 639823 (0.0007) [2023-12-27 10:03:40,816][20347] Updated weights for policy 0, policy_version 640329 (0.0009) [2023-12-27 10:03:40,876][20347] Updated weights for policy 0, policy_version 640339 (0.0007) [2023-12-27 10:03:40,939][20347] Updated weights for policy 0, policy_version 640349 (0.0007) [2023-12-27 10:03:41,329][20381] Updated weights for policy 1, policy_version 639833 (0.0006) [2023-12-27 10:03:41,396][20381] Updated weights for policy 1, policy_version 639843 (0.0007) [2023-12-27 10:03:41,449][20381] Updated weights for policy 1, policy_version 639853 (0.0009) [2023-12-27 10:03:41,502][20381] Updated weights for policy 1, policy_version 639863 (0.0010) [2023-12-27 10:03:41,701][20347] Updated weights for policy 0, policy_version 640359 (0.0008) [2023-12-27 10:03:41,764][20347] Updated weights for policy 0, policy_version 640369 (0.0008) [2023-12-27 10:03:41,815][20347] Updated weights for policy 0, policy_version 640379 (0.0007) [2023-12-27 10:03:42,195][20381] Updated weights for policy 1, policy_version 639873 (0.0011) [2023-12-27 10:03:42,250][20381] Updated weights for policy 1, policy_version 639883 (0.0010) [2023-12-27 10:03:42,321][20381] Updated weights for policy 1, policy_version 639893 (0.0010) [2023-12-27 10:03:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 327794688. Throughput: 0: 10395.4, 1: 10375.8. Samples: 327782196. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:03:42,363][19170] Avg episode reward: [(0, '691.190'), (1, '265.560')] [2023-12-27 10:03:42,558][20347] Updated weights for policy 0, policy_version 640389 (0.0007) [2023-12-27 10:03:42,616][20347] Updated weights for policy 0, policy_version 640399 (0.0007) [2023-12-27 10:03:42,667][20347] Updated weights for policy 0, policy_version 640409 (0.0006) [2023-12-27 10:03:43,030][20381] Updated weights for policy 1, policy_version 639903 (0.0011) [2023-12-27 10:03:43,076][20381] Updated weights for policy 1, policy_version 639913 (0.0008) [2023-12-27 10:03:43,127][20381] Updated weights for policy 1, policy_version 639923 (0.0005) [2023-12-27 10:03:43,385][20347] Updated weights for policy 0, policy_version 640419 (0.0009) [2023-12-27 10:03:43,443][20347] Updated weights for policy 0, policy_version 640429 (0.0007) [2023-12-27 10:03:43,492][20347] Updated weights for policy 0, policy_version 640439 (0.0008) [2023-12-27 10:03:43,807][20381] Updated weights for policy 1, policy_version 639933 (0.0007) [2023-12-27 10:03:43,862][20381] Updated weights for policy 1, policy_version 639943 (0.0008) [2023-12-27 10:03:43,917][20381] Updated weights for policy 1, policy_version 639953 (0.0008) [2023-12-27 10:03:44,187][20347] Updated weights for policy 0, policy_version 640449 (0.0007) [2023-12-27 10:03:44,245][20347] Updated weights for policy 0, policy_version 640459 (0.0007) [2023-12-27 10:03:44,302][20347] Updated weights for policy 0, policy_version 640469 (0.0007) [2023-12-27 10:03:44,348][20347] Updated weights for policy 0, policy_version 640479 (0.0008) [2023-12-27 10:03:44,604][20381] Updated weights for policy 1, policy_version 639963 (0.0008) [2023-12-27 10:03:44,652][20381] Updated weights for policy 1, policy_version 639973 (0.0008) [2023-12-27 10:03:44,698][20381] Updated weights for policy 1, policy_version 639983 (0.0007) [2023-12-27 10:03:45,045][20347] Updated weights for policy 0, policy_version 640489 (0.0008) [2023-12-27 10:03:45,100][20347] Updated weights for policy 0, policy_version 640499 (0.0007) [2023-12-27 10:03:45,159][20347] Updated weights for policy 0, policy_version 640509 (0.0007) [2023-12-27 10:03:45,428][20381] Updated weights for policy 1, policy_version 639993 (0.0008) [2023-12-27 10:03:45,486][20381] Updated weights for policy 1, policy_version 640003 (0.0011) [2023-12-27 10:03:45,539][20381] Updated weights for policy 1, policy_version 640013 (0.0010) [2023-12-27 10:03:45,593][20381] Updated weights for policy 1, policy_version 640023 (0.0010) [2023-12-27 10:03:45,839][20347] Updated weights for policy 0, policy_version 640519 (0.0007) [2023-12-27 10:03:45,884][20347] Updated weights for policy 0, policy_version 640529 (0.0007) [2023-12-27 10:03:45,940][20347] Updated weights for policy 0, policy_version 640539 (0.0008) [2023-12-27 10:03:46,274][20381] Updated weights for policy 1, policy_version 640033 (0.0009) [2023-12-27 10:03:46,329][20381] Updated weights for policy 1, policy_version 640043 (0.0008) [2023-12-27 10:03:46,387][20381] Updated weights for policy 1, policy_version 640053 (0.0007) [2023-12-27 10:03:46,658][20347] Updated weights for policy 0, policy_version 640549 (0.0008) [2023-12-27 10:03:46,717][20347] Updated weights for policy 0, policy_version 640559 (0.0008) [2023-12-27 10:03:46,772][20347] Updated weights for policy 0, policy_version 640569 (0.0007) [2023-12-27 10:03:47,087][20381] Updated weights for policy 1, policy_version 640063 (0.0007) [2023-12-27 10:03:47,145][20381] Updated weights for policy 1, policy_version 640073 (0.0005) [2023-12-27 10:03:47,205][20381] Updated weights for policy 1, policy_version 640083 (0.0008) [2023-12-27 10:03:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 327892992. Throughput: 0: 10268.4, 1: 10329.6. Samples: 327901036. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:03:47,363][19170] Avg episode reward: [(0, '854.960'), (1, '270.300')] [2023-12-27 10:03:47,483][20347] Updated weights for policy 0, policy_version 640579 (0.0008) [2023-12-27 10:03:47,538][20347] Updated weights for policy 0, policy_version 640589 (0.0007) [2023-12-27 10:03:47,596][20347] Updated weights for policy 0, policy_version 640599 (0.0008) [2023-12-27 10:03:47,886][20381] Updated weights for policy 1, policy_version 640093 (0.0008) [2023-12-27 10:03:47,943][20381] Updated weights for policy 1, policy_version 640103 (0.0008) [2023-12-27 10:03:48,004][20381] Updated weights for policy 1, policy_version 640113 (0.0008) [2023-12-27 10:03:48,212][20347] Updated weights for policy 0, policy_version 640609 (0.0007) [2023-12-27 10:03:48,270][20347] Updated weights for policy 0, policy_version 640619 (0.0006) [2023-12-27 10:03:48,324][20347] Updated weights for policy 0, policy_version 640629 (0.0008) [2023-12-27 10:03:48,386][20347] Updated weights for policy 0, policy_version 640639 (0.0008) [2023-12-27 10:03:48,717][20381] Updated weights for policy 1, policy_version 640123 (0.0008) [2023-12-27 10:03:48,769][20381] Updated weights for policy 1, policy_version 640133 (0.0008) [2023-12-27 10:03:48,824][20381] Updated weights for policy 1, policy_version 640143 (0.0008) [2023-12-27 10:03:49,098][20347] Updated weights for policy 0, policy_version 640649 (0.0007) [2023-12-27 10:03:49,149][20347] Updated weights for policy 0, policy_version 640659 (0.0007) [2023-12-27 10:03:49,203][20347] Updated weights for policy 0, policy_version 640669 (0.0008) [2023-12-27 10:03:49,498][20381] Updated weights for policy 1, policy_version 640153 (0.0007) [2023-12-27 10:03:49,557][20381] Updated weights for policy 1, policy_version 640163 (0.0006) [2023-12-27 10:03:49,611][20381] Updated weights for policy 1, policy_version 640173 (0.0006) [2023-12-27 10:03:49,666][20381] Updated weights for policy 1, policy_version 640183 (0.0006) [2023-12-27 10:03:49,934][20347] Updated weights for policy 0, policy_version 640679 (0.0008) [2023-12-27 10:03:49,997][20347] Updated weights for policy 0, policy_version 640689 (0.0007) [2023-12-27 10:03:50,057][20347] Updated weights for policy 0, policy_version 640699 (0.0007) [2023-12-27 10:03:50,341][20381] Updated weights for policy 1, policy_version 640193 (0.0006) [2023-12-27 10:03:50,398][20381] Updated weights for policy 1, policy_version 640203 (0.0008) [2023-12-27 10:03:50,460][20381] Updated weights for policy 1, policy_version 640213 (0.0007) [2023-12-27 10:03:50,683][20347] Updated weights for policy 0, policy_version 640709 (0.0006) [2023-12-27 10:03:50,732][20347] Updated weights for policy 0, policy_version 640719 (0.0008) [2023-12-27 10:03:50,784][20347] Updated weights for policy 0, policy_version 640729 (0.0007) [2023-12-27 10:03:51,118][20381] Updated weights for policy 1, policy_version 640223 (0.0007) [2023-12-27 10:03:51,174][20381] Updated weights for policy 1, policy_version 640233 (0.0006) [2023-12-27 10:03:51,231][20381] Updated weights for policy 1, policy_version 640243 (0.0008) [2023-12-27 10:03:51,488][20347] Updated weights for policy 0, policy_version 640739 (0.0007) [2023-12-27 10:03:51,545][20347] Updated weights for policy 0, policy_version 640749 (0.0008) [2023-12-27 10:03:51,603][20347] Updated weights for policy 0, policy_version 640759 (0.0008) [2023-12-27 10:03:51,963][20381] Updated weights for policy 1, policy_version 640253 (0.0009) [2023-12-27 10:03:52,015][20381] Updated weights for policy 1, policy_version 640263 (0.0008) [2023-12-27 10:03:52,070][20381] Updated weights for policy 1, policy_version 640273 (0.0008) [2023-12-27 10:03:52,215][20347] Updated weights for policy 0, policy_version 640769 (0.0007) [2023-12-27 10:03:52,274][20347] Updated weights for policy 0, policy_version 640779 (0.0008) [2023-12-27 10:03:52,327][20347] Updated weights for policy 0, policy_version 640789 (0.0008) [2023-12-27 10:03:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 327991296. Throughput: 0: 10186.0, 1: 10331.3. Samples: 327961856. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:03:52,363][19170] Avg episode reward: [(0, '815.510'), (1, '286.500')] [2023-12-27 10:03:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000640280_163938304.pth... [2023-12-27 10:03:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000639064_163627008.pth [2023-12-27 10:03:52,397][20347] Updated weights for policy 0, policy_version 640799 (0.0008) [2023-12-27 10:03:52,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000640800_164061184.pth... [2023-12-27 10:03:52,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000639584_163749888.pth [2023-12-27 10:03:52,784][20381] Updated weights for policy 1, policy_version 640283 (0.0007) [2023-12-27 10:03:52,840][20381] Updated weights for policy 1, policy_version 640293 (0.0008) [2023-12-27 10:03:52,889][20381] Updated weights for policy 1, policy_version 640303 (0.0007) [2023-12-27 10:03:53,067][20347] Updated weights for policy 0, policy_version 640809 (0.0007) [2023-12-27 10:03:53,122][20347] Updated weights for policy 0, policy_version 640819 (0.0007) [2023-12-27 10:03:53,180][20347] Updated weights for policy 0, policy_version 640829 (0.0007) [2023-12-27 10:03:53,599][20381] Updated weights for policy 1, policy_version 640313 (0.0009) [2023-12-27 10:03:53,654][20381] Updated weights for policy 1, policy_version 640323 (0.0008) [2023-12-27 10:03:53,711][20381] Updated weights for policy 1, policy_version 640333 (0.0008) [2023-12-27 10:03:53,763][20381] Updated weights for policy 1, policy_version 640343 (0.0008) [2023-12-27 10:03:53,837][20347] Updated weights for policy 0, policy_version 640839 (0.0008) [2023-12-27 10:03:53,883][20347] Updated weights for policy 0, policy_version 640849 (0.0008) [2023-12-27 10:03:53,928][20347] Updated weights for policy 0, policy_version 640859 (0.0007) [2023-12-27 10:03:54,462][20381] Updated weights for policy 1, policy_version 640353 (0.0008) [2023-12-27 10:03:54,524][20381] Updated weights for policy 1, policy_version 640363 (0.0007) [2023-12-27 10:03:54,587][20381] Updated weights for policy 1, policy_version 640373 (0.0007) [2023-12-27 10:03:54,668][20347] Updated weights for policy 0, policy_version 640869 (0.0009) [2023-12-27 10:03:54,725][20347] Updated weights for policy 0, policy_version 640879 (0.0010) [2023-12-27 10:03:54,782][20347] Updated weights for policy 0, policy_version 640889 (0.0010) [2023-12-27 10:03:55,267][20381] Updated weights for policy 1, policy_version 640383 (0.0008) [2023-12-27 10:03:55,311][20381] Updated weights for policy 1, policy_version 640393 (0.0007) [2023-12-27 10:03:55,356][20381] Updated weights for policy 1, policy_version 640403 (0.0007) [2023-12-27 10:03:55,459][20347] Updated weights for policy 0, policy_version 640899 (0.0010) [2023-12-27 10:03:55,513][20347] Updated weights for policy 0, policy_version 640909 (0.0010) [2023-12-27 10:03:55,562][20347] Updated weights for policy 0, policy_version 640919 (0.0010) [2023-12-27 10:03:56,105][20381] Updated weights for policy 1, policy_version 640413 (0.0008) [2023-12-27 10:03:56,151][20381] Updated weights for policy 1, policy_version 640423 (0.0007) [2023-12-27 10:03:56,160][20347] Updated weights for policy 0, policy_version 640929 (0.0008) [2023-12-27 10:03:56,196][20381] Updated weights for policy 1, policy_version 640433 (0.0008) [2023-12-27 10:03:56,206][20347] Updated weights for policy 0, policy_version 640939 (0.0005) [2023-12-27 10:03:56,258][20347] Updated weights for policy 0, policy_version 640949 (0.0007) [2023-12-27 10:03:56,316][20347] Updated weights for policy 0, policy_version 640959 (0.0010) [2023-12-27 10:03:56,861][20347] Updated weights for policy 0, policy_version 640969 (0.0006) [2023-12-27 10:03:56,914][20347] Updated weights for policy 0, policy_version 640979 (0.0005) [2023-12-27 10:03:56,942][20381] Updated weights for policy 1, policy_version 640443 (0.0008) [2023-12-27 10:03:56,957][20347] Updated weights for policy 0, policy_version 640989 (0.0009) [2023-12-27 10:03:56,988][20381] Updated weights for policy 1, policy_version 640453 (0.0007) [2023-12-27 10:03:57,044][20381] Updated weights for policy 1, policy_version 640463 (0.0007) [2023-12-27 10:03:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 328097792. Throughput: 0: 10229.9, 1: 10288.6. Samples: 328083992. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:03:57,363][19170] Avg episode reward: [(0, '718.980'), (1, '345.560')] [2023-12-27 10:03:57,633][20347] Updated weights for policy 0, policy_version 640999 (0.0010) [2023-12-27 10:03:57,692][20347] Updated weights for policy 0, policy_version 641009 (0.0010) [2023-12-27 10:03:57,730][20381] Updated weights for policy 1, policy_version 640473 (0.0007) [2023-12-27 10:03:57,745][20347] Updated weights for policy 0, policy_version 641019 (0.0009) [2023-12-27 10:03:57,779][20381] Updated weights for policy 1, policy_version 640483 (0.0007) [2023-12-27 10:03:57,824][20381] Updated weights for policy 1, policy_version 640493 (0.0009) [2023-12-27 10:03:57,869][20381] Updated weights for policy 1, policy_version 640503 (0.0008) [2023-12-27 10:03:58,349][20347] Updated weights for policy 0, policy_version 641029 (0.0007) [2023-12-27 10:03:58,412][20347] Updated weights for policy 0, policy_version 641039 (0.0009) [2023-12-27 10:03:58,472][20347] Updated weights for policy 0, policy_version 641049 (0.0007) [2023-12-27 10:03:58,620][20381] Updated weights for policy 1, policy_version 640513 (0.0010) [2023-12-27 10:03:58,680][20381] Updated weights for policy 1, policy_version 640523 (0.0011) [2023-12-27 10:03:58,743][20381] Updated weights for policy 1, policy_version 640533 (0.0011) [2023-12-27 10:03:59,205][20347] Updated weights for policy 0, policy_version 641059 (0.0008) [2023-12-27 10:03:59,270][20347] Updated weights for policy 0, policy_version 641069 (0.0009) [2023-12-27 10:03:59,330][20347] Updated weights for policy 0, policy_version 641079 (0.0010) [2023-12-27 10:03:59,471][20381] Updated weights for policy 1, policy_version 640543 (0.0008) [2023-12-27 10:03:59,525][20381] Updated weights for policy 1, policy_version 640553 (0.0007) [2023-12-27 10:03:59,587][20381] Updated weights for policy 1, policy_version 640563 (0.0006) [2023-12-27 10:04:00,027][20347] Updated weights for policy 0, policy_version 641089 (0.0008) [2023-12-27 10:04:00,082][20347] Updated weights for policy 0, policy_version 641099 (0.0008) [2023-12-27 10:04:00,134][20347] Updated weights for policy 0, policy_version 641109 (0.0008) [2023-12-27 10:04:00,184][20347] Updated weights for policy 0, policy_version 641119 (0.0006) [2023-12-27 10:04:00,310][20381] Updated weights for policy 1, policy_version 640573 (0.0008) [2023-12-27 10:04:00,366][20381] Updated weights for policy 1, policy_version 640583 (0.0008) [2023-12-27 10:04:00,420][20381] Updated weights for policy 1, policy_version 640593 (0.0007) [2023-12-27 10:04:00,853][20347] Updated weights for policy 0, policy_version 641129 (0.0007) [2023-12-27 10:04:00,901][20347] Updated weights for policy 0, policy_version 641139 (0.0007) [2023-12-27 10:04:00,953][20347] Updated weights for policy 0, policy_version 641149 (0.0008) [2023-12-27 10:04:01,102][20381] Updated weights for policy 1, policy_version 640603 (0.0008) [2023-12-27 10:04:01,162][20381] Updated weights for policy 1, policy_version 640613 (0.0007) [2023-12-27 10:04:01,219][20381] Updated weights for policy 1, policy_version 640623 (0.0007) [2023-12-27 10:04:01,682][20347] Updated weights for policy 0, policy_version 641159 (0.0008) [2023-12-27 10:04:01,745][20347] Updated weights for policy 0, policy_version 641169 (0.0007) [2023-12-27 10:04:01,806][20347] Updated weights for policy 0, policy_version 641179 (0.0009) [2023-12-27 10:04:01,892][20381] Updated weights for policy 1, policy_version 640633 (0.0007) [2023-12-27 10:04:01,942][20381] Updated weights for policy 1, policy_version 640643 (0.0006) [2023-12-27 10:04:01,996][20381] Updated weights for policy 1, policy_version 640653 (0.0006) [2023-12-27 10:04:02,042][20381] Updated weights for policy 1, policy_version 640663 (0.0005) [2023-12-27 10:04:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 328196096. Throughput: 0: 10249.9, 1: 10212.1. Samples: 328206528. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:02,363][19170] Avg episode reward: [(0, '738.540'), (1, '430.980')] [2023-12-27 10:04:02,527][20347] Updated weights for policy 0, policy_version 641189 (0.0008) [2023-12-27 10:04:02,581][20347] Updated weights for policy 0, policy_version 641199 (0.0008) [2023-12-27 10:04:02,612][20381] Updated weights for policy 1, policy_version 640673 (0.0006) [2023-12-27 10:04:02,641][20347] Updated weights for policy 0, policy_version 641209 (0.0007) [2023-12-27 10:04:02,664][20381] Updated weights for policy 1, policy_version 640683 (0.0005) [2023-12-27 10:04:02,721][20381] Updated weights for policy 1, policy_version 640693 (0.0006) [2023-12-27 10:04:03,295][20347] Updated weights for policy 0, policy_version 641219 (0.0007) [2023-12-27 10:04:03,351][20347] Updated weights for policy 0, policy_version 641229 (0.0006) [2023-12-27 10:04:03,352][20381] Updated weights for policy 1, policy_version 640703 (0.0007) [2023-12-27 10:04:03,409][20347] Updated weights for policy 0, policy_version 641239 (0.0007) [2023-12-27 10:04:03,409][20381] Updated weights for policy 1, policy_version 640713 (0.0006) [2023-12-27 10:04:03,458][20381] Updated weights for policy 1, policy_version 640723 (0.0005) [2023-12-27 10:04:04,092][20347] Updated weights for policy 0, policy_version 641249 (0.0008) [2023-12-27 10:04:04,113][20381] Updated weights for policy 1, policy_version 640733 (0.0006) [2023-12-27 10:04:04,152][20347] Updated weights for policy 0, policy_version 641259 (0.0009) [2023-12-27 10:04:04,173][20381] Updated weights for policy 1, policy_version 640743 (0.0007) [2023-12-27 10:04:04,209][20347] Updated weights for policy 0, policy_version 641269 (0.0006) [2023-12-27 10:04:04,226][20381] Updated weights for policy 1, policy_version 640753 (0.0006) [2023-12-27 10:04:04,263][20347] Updated weights for policy 0, policy_version 641279 (0.0006) [2023-12-27 10:04:04,940][20381] Updated weights for policy 1, policy_version 640763 (0.0007) [2023-12-27 10:04:04,947][20347] Updated weights for policy 0, policy_version 641289 (0.0007) [2023-12-27 10:04:04,993][20381] Updated weights for policy 1, policy_version 640773 (0.0006) [2023-12-27 10:04:04,995][20347] Updated weights for policy 0, policy_version 641299 (0.0007) [2023-12-27 10:04:05,042][20347] Updated weights for policy 0, policy_version 641309 (0.0007) [2023-12-27 10:04:05,052][20381] Updated weights for policy 1, policy_version 640783 (0.0007) [2023-12-27 10:04:05,610][20347] Updated weights for policy 0, policy_version 641319 (0.0005) [2023-12-27 10:04:05,665][20347] Updated weights for policy 0, policy_version 641329 (0.0009) [2023-12-27 10:04:05,720][20347] Updated weights for policy 0, policy_version 641339 (0.0010) [2023-12-27 10:04:05,746][20381] Updated weights for policy 1, policy_version 640793 (0.0010) [2023-12-27 10:04:05,796][20381] Updated weights for policy 1, policy_version 640803 (0.0005) [2023-12-27 10:04:05,844][20381] Updated weights for policy 1, policy_version 640813 (0.0006) [2023-12-27 10:04:05,900][20381] Updated weights for policy 1, policy_version 640823 (0.0006) [2023-12-27 10:04:06,324][20347] Updated weights for policy 0, policy_version 641349 (0.0010) [2023-12-27 10:04:06,380][20347] Updated weights for policy 0, policy_version 641359 (0.0007) [2023-12-27 10:04:06,442][20347] Updated weights for policy 0, policy_version 641369 (0.0008) [2023-12-27 10:04:06,473][20381] Updated weights for policy 1, policy_version 640833 (0.0007) [2023-12-27 10:04:06,532][20381] Updated weights for policy 1, policy_version 640843 (0.0007) [2023-12-27 10:04:06,591][20381] Updated weights for policy 1, policy_version 640853 (0.0007) [2023-12-27 10:04:07,135][20347] Updated weights for policy 0, policy_version 641379 (0.0006) [2023-12-27 10:04:07,195][20347] Updated weights for policy 0, policy_version 641389 (0.0008) [2023-12-27 10:04:07,249][20347] Updated weights for policy 0, policy_version 641399 (0.0008) [2023-12-27 10:04:07,282][20381] Updated weights for policy 1, policy_version 640863 (0.0007) [2023-12-27 10:04:07,340][20381] Updated weights for policy 1, policy_version 640873 (0.0008) [2023-12-27 10:04:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 328302592. Throughput: 0: 10251.1, 1: 10245.4. Samples: 328268352. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:07,363][19170] Avg episode reward: [(0, '795.910'), (1, '512.850')] [2023-12-27 10:04:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000641408_164216832.pth... [2023-12-27 10:04:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000640192_163905536.pth [2023-12-27 10:04:07,398][20381] Updated weights for policy 1, policy_version 640883 (0.0006) [2023-12-27 10:04:07,424][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000640888_164093952.pth... [2023-12-27 10:04:07,428][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000639704_163790848.pth [2023-12-27 10:04:07,949][20347] Updated weights for policy 0, policy_version 641409 (0.0006) [2023-12-27 10:04:07,997][20347] Updated weights for policy 0, policy_version 641419 (0.0008) [2023-12-27 10:04:08,026][20381] Updated weights for policy 1, policy_version 640893 (0.0007) [2023-12-27 10:04:08,048][20347] Updated weights for policy 0, policy_version 641429 (0.0006) [2023-12-27 10:04:08,084][20381] Updated weights for policy 1, policy_version 640903 (0.0007) [2023-12-27 10:04:08,097][20347] Updated weights for policy 0, policy_version 641439 (0.0006) [2023-12-27 10:04:08,137][20381] Updated weights for policy 1, policy_version 640913 (0.0008) [2023-12-27 10:04:08,811][20347] Updated weights for policy 0, policy_version 641449 (0.0007) [2023-12-27 10:04:08,844][20381] Updated weights for policy 1, policy_version 640923 (0.0008) [2023-12-27 10:04:08,869][20347] Updated weights for policy 0, policy_version 641459 (0.0007) [2023-12-27 10:04:08,901][20381] Updated weights for policy 1, policy_version 640933 (0.0006) [2023-12-27 10:04:08,927][20347] Updated weights for policy 0, policy_version 641469 (0.0006) [2023-12-27 10:04:08,961][20381] Updated weights for policy 1, policy_version 640943 (0.0008) [2023-12-27 10:04:09,556][20347] Updated weights for policy 0, policy_version 641479 (0.0006) [2023-12-27 10:04:09,614][20347] Updated weights for policy 0, policy_version 641489 (0.0007) [2023-12-27 10:04:09,675][20347] Updated weights for policy 0, policy_version 641499 (0.0006) [2023-12-27 10:04:09,700][20381] Updated weights for policy 1, policy_version 640953 (0.0009) [2023-12-27 10:04:09,766][20381] Updated weights for policy 1, policy_version 640963 (0.0007) [2023-12-27 10:04:09,823][20381] Updated weights for policy 1, policy_version 640973 (0.0008) [2023-12-27 10:04:09,885][20381] Updated weights for policy 1, policy_version 640983 (0.0008) [2023-12-27 10:04:10,342][20347] Updated weights for policy 0, policy_version 641509 (0.0008) [2023-12-27 10:04:10,395][20347] Updated weights for policy 0, policy_version 641519 (0.0008) [2023-12-27 10:04:10,443][20347] Updated weights for policy 0, policy_version 641529 (0.0008) [2023-12-27 10:04:10,633][20381] Updated weights for policy 1, policy_version 640993 (0.0007) [2023-12-27 10:04:10,684][20381] Updated weights for policy 1, policy_version 641003 (0.0008) [2023-12-27 10:04:10,734][20381] Updated weights for policy 1, policy_version 641013 (0.0008) [2023-12-27 10:04:11,087][20347] Updated weights for policy 0, policy_version 641539 (0.0008) [2023-12-27 10:04:11,149][20347] Updated weights for policy 0, policy_version 641549 (0.0008) [2023-12-27 10:04:11,203][20347] Updated weights for policy 0, policy_version 641559 (0.0005) [2023-12-27 10:04:11,471][20381] Updated weights for policy 1, policy_version 641023 (0.0007) [2023-12-27 10:04:11,528][20381] Updated weights for policy 1, policy_version 641033 (0.0007) [2023-12-27 10:04:11,591][20381] Updated weights for policy 1, policy_version 641043 (0.0008) [2023-12-27 10:04:11,913][20347] Updated weights for policy 0, policy_version 641569 (0.0008) [2023-12-27 10:04:11,979][20347] Updated weights for policy 0, policy_version 641579 (0.0009) [2023-12-27 10:04:12,039][20347] Updated weights for policy 0, policy_version 641589 (0.0007) [2023-12-27 10:04:12,099][20347] Updated weights for policy 0, policy_version 641599 (0.0006) [2023-12-27 10:04:12,311][20381] Updated weights for policy 1, policy_version 641053 (0.0008) [2023-12-27 10:04:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 328400896. Throughput: 0: 10273.4, 1: 10211.1. Samples: 328392780. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:12,363][19170] Avg episode reward: [(0, '700.810'), (1, '456.580')] [2023-12-27 10:04:12,375][20381] Updated weights for policy 1, policy_version 641063 (0.0008) [2023-12-27 10:04:12,428][20381] Updated weights for policy 1, policy_version 641073 (0.0010) [2023-12-27 10:04:12,736][20347] Updated weights for policy 0, policy_version 641609 (0.0007) [2023-12-27 10:04:12,785][20347] Updated weights for policy 0, policy_version 641619 (0.0005) [2023-12-27 10:04:12,852][20347] Updated weights for policy 0, policy_version 641629 (0.0008) [2023-12-27 10:04:13,167][20381] Updated weights for policy 1, policy_version 641083 (0.0008) [2023-12-27 10:04:13,221][20381] Updated weights for policy 1, policy_version 641093 (0.0007) [2023-12-27 10:04:13,274][20381] Updated weights for policy 1, policy_version 641103 (0.0007) [2023-12-27 10:04:13,493][20347] Updated weights for policy 0, policy_version 641639 (0.0009) [2023-12-27 10:04:13,552][20347] Updated weights for policy 0, policy_version 641649 (0.0008) [2023-12-27 10:04:13,609][20347] Updated weights for policy 0, policy_version 641659 (0.0008) [2023-12-27 10:04:13,998][20381] Updated weights for policy 1, policy_version 641113 (0.0007) [2023-12-27 10:04:14,062][20381] Updated weights for policy 1, policy_version 641123 (0.0008) [2023-12-27 10:04:14,107][20381] Updated weights for policy 1, policy_version 641133 (0.0007) [2023-12-27 10:04:14,186][20381] Updated weights for policy 1, policy_version 641143 (0.0006) [2023-12-27 10:04:14,223][20347] Updated weights for policy 0, policy_version 641669 (0.0008) [2023-12-27 10:04:14,277][20347] Updated weights for policy 0, policy_version 641679 (0.0008) [2023-12-27 10:04:14,334][20347] Updated weights for policy 0, policy_version 641689 (0.0008) [2023-12-27 10:04:14,848][20381] Updated weights for policy 1, policy_version 641153 (0.0007) [2023-12-27 10:04:14,912][20381] Updated weights for policy 1, policy_version 641163 (0.0007) [2023-12-27 10:04:14,972][20381] Updated weights for policy 1, policy_version 641173 (0.0008) [2023-12-27 10:04:15,037][20347] Updated weights for policy 0, policy_version 641699 (0.0008) [2023-12-27 10:04:15,102][20347] Updated weights for policy 0, policy_version 641709 (0.0011) [2023-12-27 10:04:15,167][20347] Updated weights for policy 0, policy_version 641719 (0.0011) [2023-12-27 10:04:15,692][20381] Updated weights for policy 1, policy_version 641183 (0.0009) [2023-12-27 10:04:15,719][20347] Updated weights for policy 0, policy_version 641729 (0.0010) [2023-12-27 10:04:15,751][20381] Updated weights for policy 1, policy_version 641193 (0.0008) [2023-12-27 10:04:15,770][20347] Updated weights for policy 0, policy_version 641739 (0.0010) [2023-12-27 10:04:15,800][20381] Updated weights for policy 1, policy_version 641203 (0.0007) [2023-12-27 10:04:15,825][20347] Updated weights for policy 0, policy_version 641749 (0.0010) [2023-12-27 10:04:15,880][20347] Updated weights for policy 0, policy_version 641759 (0.0008) [2023-12-27 10:04:16,487][20347] Updated weights for policy 0, policy_version 641769 (0.0007) [2023-12-27 10:04:16,516][20381] Updated weights for policy 1, policy_version 641213 (0.0007) [2023-12-27 10:04:16,546][20347] Updated weights for policy 0, policy_version 641779 (0.0007) [2023-12-27 10:04:16,580][20381] Updated weights for policy 1, policy_version 641223 (0.0010) [2023-12-27 10:04:16,608][20347] Updated weights for policy 0, policy_version 641789 (0.0007) [2023-12-27 10:04:16,644][20381] Updated weights for policy 1, policy_version 641233 (0.0008) [2023-12-27 10:04:17,279][20381] Updated weights for policy 1, policy_version 641243 (0.0007) [2023-12-27 10:04:17,300][20347] Updated weights for policy 0, policy_version 641799 (0.0007) [2023-12-27 10:04:17,336][20381] Updated weights for policy 1, policy_version 641253 (0.0006) [2023-12-27 10:04:17,349][20347] Updated weights for policy 0, policy_version 641809 (0.0010) [2023-12-27 10:04:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20716.1). Total num frames: 328499200. Throughput: 0: 10304.8, 1: 10160.5. Samples: 328515788. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:17,363][19170] Avg episode reward: [(0, '546.030'), (1, '424.100')] [2023-12-27 10:04:17,387][20381] Updated weights for policy 1, policy_version 641263 (0.0007) [2023-12-27 10:04:17,394][20347] Updated weights for policy 0, policy_version 641819 (0.0010) [2023-12-27 10:04:18,078][20381] Updated weights for policy 1, policy_version 641273 (0.0006) [2023-12-27 10:04:18,091][20347] Updated weights for policy 0, policy_version 641829 (0.0010) [2023-12-27 10:04:18,130][20381] Updated weights for policy 1, policy_version 641283 (0.0006) [2023-12-27 10:04:18,143][20347] Updated weights for policy 0, policy_version 641839 (0.0010) [2023-12-27 10:04:18,189][20381] Updated weights for policy 1, policy_version 641293 (0.0006) [2023-12-27 10:04:18,198][20347] Updated weights for policy 0, policy_version 641849 (0.0010) [2023-12-27 10:04:18,245][20381] Updated weights for policy 1, policy_version 641303 (0.0006) [2023-12-27 10:04:18,897][20381] Updated weights for policy 1, policy_version 641313 (0.0008) [2023-12-27 10:04:18,902][20347] Updated weights for policy 0, policy_version 641859 (0.0010) [2023-12-27 10:04:18,956][20381] Updated weights for policy 1, policy_version 641323 (0.0006) [2023-12-27 10:04:18,962][20347] Updated weights for policy 0, policy_version 641869 (0.0011) [2023-12-27 10:04:19,009][20381] Updated weights for policy 1, policy_version 641333 (0.0006) [2023-12-27 10:04:19,021][20347] Updated weights for policy 0, policy_version 641879 (0.0010) [2023-12-27 10:04:19,699][20347] Updated weights for policy 0, policy_version 641889 (0.0010) [2023-12-27 10:04:19,723][20381] Updated weights for policy 1, policy_version 641343 (0.0006) [2023-12-27 10:04:19,757][20347] Updated weights for policy 0, policy_version 641899 (0.0006) [2023-12-27 10:04:19,782][20381] Updated weights for policy 1, policy_version 641353 (0.0006) [2023-12-27 10:04:19,814][20347] Updated weights for policy 0, policy_version 641909 (0.0006) [2023-12-27 10:04:19,836][20381] Updated weights for policy 1, policy_version 641363 (0.0006) [2023-12-27 10:04:19,878][20347] Updated weights for policy 0, policy_version 641919 (0.0007) [2023-12-27 10:04:20,549][20381] Updated weights for policy 1, policy_version 641373 (0.0007) [2023-12-27 10:04:20,578][20347] Updated weights for policy 0, policy_version 641929 (0.0006) [2023-12-27 10:04:20,611][20381] Updated weights for policy 1, policy_version 641383 (0.0008) [2023-12-27 10:04:20,645][20347] Updated weights for policy 0, policy_version 641939 (0.0007) [2023-12-27 10:04:20,671][20381] Updated weights for policy 1, policy_version 641393 (0.0008) [2023-12-27 10:04:20,706][20347] Updated weights for policy 0, policy_version 641949 (0.0006) [2023-12-27 10:04:21,397][20381] Updated weights for policy 1, policy_version 641403 (0.0008) [2023-12-27 10:04:21,432][20347] Updated weights for policy 0, policy_version 641959 (0.0007) [2023-12-27 10:04:21,454][20381] Updated weights for policy 1, policy_version 641413 (0.0006) [2023-12-27 10:04:21,486][20347] Updated weights for policy 0, policy_version 641969 (0.0006) [2023-12-27 10:04:21,510][20381] Updated weights for policy 1, policy_version 641423 (0.0005) [2023-12-27 10:04:21,545][20347] Updated weights for policy 0, policy_version 641979 (0.0007) [2023-12-27 10:04:22,199][20381] Updated weights for policy 1, policy_version 641433 (0.0006) [2023-12-27 10:04:22,243][20347] Updated weights for policy 0, policy_version 641989 (0.0007) [2023-12-27 10:04:22,253][20381] Updated weights for policy 1, policy_version 641443 (0.0006) [2023-12-27 10:04:22,304][20347] Updated weights for policy 0, policy_version 641999 (0.0006) [2023-12-27 10:04:22,313][20381] Updated weights for policy 1, policy_version 641453 (0.0006) [2023-12-27 10:04:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20347.7, 300 sec: 20688.3). Total num frames: 328597504. Throughput: 0: 10256.8, 1: 10109.8. Samples: 328575804. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:22,363][19170] Avg episode reward: [(0, '613.470'), (1, '410.830')] [2023-12-27 10:04:22,367][20347] Updated weights for policy 0, policy_version 642009 (0.0006) [2023-12-27 10:04:22,380][20381] Updated weights for policy 1, policy_version 641463 (0.0007) [2023-12-27 10:04:22,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000641464_164241408.pth... [2023-12-27 10:04:22,388][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000640280_163938304.pth [2023-12-27 10:04:22,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000642016_164372480.pth... [2023-12-27 10:04:22,409][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000640800_164061184.pth [2023-12-27 10:04:23,046][20347] Updated weights for policy 0, policy_version 642019 (0.0010) [2023-12-27 10:04:23,053][20381] Updated weights for policy 1, policy_version 641473 (0.0010) [2023-12-27 10:04:23,102][20347] Updated weights for policy 0, policy_version 642029 (0.0011) [2023-12-27 10:04:23,109][20381] Updated weights for policy 1, policy_version 641483 (0.0008) [2023-12-27 10:04:23,151][20347] Updated weights for policy 0, policy_version 642039 (0.0010) [2023-12-27 10:04:23,163][20381] Updated weights for policy 1, policy_version 641493 (0.0008) [2023-12-27 10:04:23,740][20347] Updated weights for policy 0, policy_version 642049 (0.0010) [2023-12-27 10:04:23,795][20347] Updated weights for policy 0, policy_version 642059 (0.0007) [2023-12-27 10:04:23,846][20347] Updated weights for policy 0, policy_version 642069 (0.0010) [2023-12-27 10:04:23,873][20381] Updated weights for policy 1, policy_version 641503 (0.0006) [2023-12-27 10:04:23,900][20347] Updated weights for policy 0, policy_version 642079 (0.0009) [2023-12-27 10:04:23,927][20381] Updated weights for policy 1, policy_version 641513 (0.0008) [2023-12-27 10:04:23,981][20381] Updated weights for policy 1, policy_version 641523 (0.0009) [2023-12-27 10:04:24,520][20347] Updated weights for policy 0, policy_version 642089 (0.0010) [2023-12-27 10:04:24,575][20347] Updated weights for policy 0, policy_version 642099 (0.0010) [2023-12-27 10:04:24,628][20347] Updated weights for policy 0, policy_version 642109 (0.0007) [2023-12-27 10:04:24,677][20381] Updated weights for policy 1, policy_version 641533 (0.0008) [2023-12-27 10:04:24,733][20381] Updated weights for policy 1, policy_version 641543 (0.0008) [2023-12-27 10:04:24,798][20381] Updated weights for policy 1, policy_version 641553 (0.0008) [2023-12-27 10:04:25,262][20347] Updated weights for policy 0, policy_version 642119 (0.0010) [2023-12-27 10:04:25,307][20347] Updated weights for policy 0, policy_version 642129 (0.0010) [2023-12-27 10:04:25,356][20347] Updated weights for policy 0, policy_version 642139 (0.0010) [2023-12-27 10:04:25,451][20381] Updated weights for policy 1, policy_version 641563 (0.0007) [2023-12-27 10:04:25,508][20381] Updated weights for policy 1, policy_version 641573 (0.0007) [2023-12-27 10:04:25,563][20381] Updated weights for policy 1, policy_version 641583 (0.0007) [2023-12-27 10:04:26,039][20347] Updated weights for policy 0, policy_version 642149 (0.0008) [2023-12-27 10:04:26,103][20347] Updated weights for policy 0, policy_version 642159 (0.0008) [2023-12-27 10:04:26,154][20347] Updated weights for policy 0, policy_version 642169 (0.0009) [2023-12-27 10:04:26,219][20381] Updated weights for policy 1, policy_version 641593 (0.0008) [2023-12-27 10:04:26,265][20381] Updated weights for policy 1, policy_version 641603 (0.0005) [2023-12-27 10:04:26,310][20381] Updated weights for policy 1, policy_version 641613 (0.0005) [2023-12-27 10:04:26,360][20381] Updated weights for policy 1, policy_version 641623 (0.0005) [2023-12-27 10:04:26,852][20347] Updated weights for policy 0, policy_version 642179 (0.0007) [2023-12-27 10:04:26,909][20347] Updated weights for policy 0, policy_version 642189 (0.0007) [2023-12-27 10:04:26,963][20347] Updated weights for policy 0, policy_version 642199 (0.0007) [2023-12-27 10:04:27,012][20381] Updated weights for policy 1, policy_version 641633 (0.0006) [2023-12-27 10:04:27,072][20381] Updated weights for policy 1, policy_version 641643 (0.0007) [2023-12-27 10:04:27,123][20381] Updated weights for policy 1, policy_version 641653 (0.0010) [2023-12-27 10:04:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 328712192. Throughput: 0: 10334.9, 1: 10044.6. Samples: 328699276. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:27,364][19170] Avg episode reward: [(0, '708.560'), (1, '504.150')] [2023-12-27 10:04:27,644][20347] Updated weights for policy 0, policy_version 642209 (0.0007) [2023-12-27 10:04:27,701][20347] Updated weights for policy 0, policy_version 642219 (0.0007) [2023-12-27 10:04:27,762][20347] Updated weights for policy 0, policy_version 642229 (0.0008) [2023-12-27 10:04:27,779][20381] Updated weights for policy 1, policy_version 641663 (0.0007) [2023-12-27 10:04:27,817][20347] Updated weights for policy 0, policy_version 642239 (0.0008) [2023-12-27 10:04:27,834][20381] Updated weights for policy 1, policy_version 641673 (0.0006) [2023-12-27 10:04:27,886][20381] Updated weights for policy 1, policy_version 641683 (0.0005) [2023-12-27 10:04:28,461][20347] Updated weights for policy 0, policy_version 642249 (0.0007) [2023-12-27 10:04:28,505][20347] Updated weights for policy 0, policy_version 642259 (0.0007) [2023-12-27 10:04:28,559][20381] Updated weights for policy 1, policy_version 641693 (0.0006) [2023-12-27 10:04:28,560][20347] Updated weights for policy 0, policy_version 642269 (0.0008) [2023-12-27 10:04:28,610][20381] Updated weights for policy 1, policy_version 641703 (0.0008) [2023-12-27 10:04:28,666][20381] Updated weights for policy 1, policy_version 641713 (0.0008) [2023-12-27 10:04:29,237][20347] Updated weights for policy 0, policy_version 642279 (0.0007) [2023-12-27 10:04:29,290][20347] Updated weights for policy 0, policy_version 642289 (0.0005) [2023-12-27 10:04:29,360][20347] Updated weights for policy 0, policy_version 642299 (0.0008) [2023-12-27 10:04:29,407][20381] Updated weights for policy 1, policy_version 641723 (0.0008) [2023-12-27 10:04:29,463][20381] Updated weights for policy 1, policy_version 641733 (0.0008) [2023-12-27 10:04:29,518][20381] Updated weights for policy 1, policy_version 641743 (0.0007) [2023-12-27 10:04:29,991][20347] Updated weights for policy 0, policy_version 642309 (0.0006) [2023-12-27 10:04:30,050][20347] Updated weights for policy 0, policy_version 642319 (0.0005) [2023-12-27 10:04:30,107][20347] Updated weights for policy 0, policy_version 642329 (0.0008) [2023-12-27 10:04:30,229][20381] Updated weights for policy 1, policy_version 641753 (0.0007) [2023-12-27 10:04:30,283][20381] Updated weights for policy 1, policy_version 641763 (0.0006) [2023-12-27 10:04:30,343][20381] Updated weights for policy 1, policy_version 641773 (0.0008) [2023-12-27 10:04:30,396][20381] Updated weights for policy 1, policy_version 641783 (0.0008) [2023-12-27 10:04:30,748][20347] Updated weights for policy 0, policy_version 642339 (0.0008) [2023-12-27 10:04:30,792][20347] Updated weights for policy 0, policy_version 642349 (0.0007) [2023-12-27 10:04:30,839][20347] Updated weights for policy 0, policy_version 642359 (0.0007) [2023-12-27 10:04:31,091][20381] Updated weights for policy 1, policy_version 641793 (0.0010) [2023-12-27 10:04:31,145][20381] Updated weights for policy 1, policy_version 641803 (0.0009) [2023-12-27 10:04:31,194][20381] Updated weights for policy 1, policy_version 641814 (0.0009) [2023-12-27 10:04:31,580][20347] Updated weights for policy 0, policy_version 642369 (0.0007) [2023-12-27 10:04:31,629][20347] Updated weights for policy 0, policy_version 642379 (0.0007) [2023-12-27 10:04:31,693][20347] Updated weights for policy 0, policy_version 642389 (0.0008) [2023-12-27 10:04:31,751][20347] Updated weights for policy 0, policy_version 642399 (0.0007) [2023-12-27 10:04:31,912][20381] Updated weights for policy 1, policy_version 641824 (0.0008) [2023-12-27 10:04:31,965][20381] Updated weights for policy 1, policy_version 641834 (0.0007) [2023-12-27 10:04:32,014][20381] Updated weights for policy 1, policy_version 641844 (0.0007) [2023-12-27 10:04:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20343.5, 300 sec: 20688.3). Total num frames: 328810496. Throughput: 0: 10392.4, 1: 10064.9. Samples: 328821612. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:32,363][19170] Avg episode reward: [(0, '718.610'), (1, '451.070')] [2023-12-27 10:04:32,467][20347] Updated weights for policy 0, policy_version 642409 (0.0007) [2023-12-27 10:04:32,526][20347] Updated weights for policy 0, policy_version 642419 (0.0008) [2023-12-27 10:04:32,579][20347] Updated weights for policy 0, policy_version 642429 (0.0007) [2023-12-27 10:04:32,682][20381] Updated weights for policy 1, policy_version 641854 (0.0007) [2023-12-27 10:04:32,728][20381] Updated weights for policy 1, policy_version 641864 (0.0008) [2023-12-27 10:04:32,777][20381] Updated weights for policy 1, policy_version 641874 (0.0008) [2023-12-27 10:04:33,243][20347] Updated weights for policy 0, policy_version 642439 (0.0007) [2023-12-27 10:04:33,286][20347] Updated weights for policy 0, policy_version 642449 (0.0009) [2023-12-27 10:04:33,336][20347] Updated weights for policy 0, policy_version 642459 (0.0007) [2023-12-27 10:04:33,466][20381] Updated weights for policy 1, policy_version 641884 (0.0008) [2023-12-27 10:04:33,518][20381] Updated weights for policy 1, policy_version 641894 (0.0008) [2023-12-27 10:04:33,570][20381] Updated weights for policy 1, policy_version 641904 (0.0007) [2023-12-27 10:04:34,042][20347] Updated weights for policy 0, policy_version 642469 (0.0007) [2023-12-27 10:04:34,090][20347] Updated weights for policy 0, policy_version 642479 (0.0008) [2023-12-27 10:04:34,152][20347] Updated weights for policy 0, policy_version 642489 (0.0008) [2023-12-27 10:04:34,277][20381] Updated weights for policy 1, policy_version 641914 (0.0008) [2023-12-27 10:04:34,335][20381] Updated weights for policy 1, policy_version 641924 (0.0008) [2023-12-27 10:04:34,398][20381] Updated weights for policy 1, policy_version 641934 (0.0008) [2023-12-27 10:04:34,452][20381] Updated weights for policy 1, policy_version 641944 (0.0007) [2023-12-27 10:04:34,817][20347] Updated weights for policy 0, policy_version 642499 (0.0006) [2023-12-27 10:04:34,864][20347] Updated weights for policy 0, policy_version 642509 (0.0008) [2023-12-27 10:04:34,909][20347] Updated weights for policy 0, policy_version 642519 (0.0007) [2023-12-27 10:04:35,126][20381] Updated weights for policy 1, policy_version 641954 (0.0006) [2023-12-27 10:04:35,178][20381] Updated weights for policy 1, policy_version 641964 (0.0009) [2023-12-27 10:04:35,237][20381] Updated weights for policy 1, policy_version 641974 (0.0010) [2023-12-27 10:04:35,644][20347] Updated weights for policy 0, policy_version 642529 (0.0007) [2023-12-27 10:04:35,679][20291] KL-divergence is very high: 107.8748 [2023-12-27 10:04:35,696][20347] Updated weights for policy 0, policy_version 642539 (0.0006) [2023-12-27 10:04:35,749][20347] Updated weights for policy 0, policy_version 642549 (0.0007) [2023-12-27 10:04:35,795][20347] Updated weights for policy 0, policy_version 642559 (0.0008) [2023-12-27 10:04:35,825][20381] Updated weights for policy 1, policy_version 641984 (0.0011) [2023-12-27 10:04:35,882][20381] Updated weights for policy 1, policy_version 641994 (0.0010) [2023-12-27 10:04:35,941][20381] Updated weights for policy 1, policy_version 642004 (0.0010) [2023-12-27 10:04:36,556][20347] Updated weights for policy 0, policy_version 642569 (0.0007) [2023-12-27 10:04:36,560][20381] Updated weights for policy 1, policy_version 642014 (0.0008) [2023-12-27 10:04:36,614][20381] Updated weights for policy 1, policy_version 642024 (0.0005) [2023-12-27 10:04:36,617][20347] Updated weights for policy 0, policy_version 642579 (0.0006) [2023-12-27 10:04:36,671][20381] Updated weights for policy 1, policy_version 642034 (0.0005) [2023-12-27 10:04:36,678][20347] Updated weights for policy 0, policy_version 642589 (0.0008) [2023-12-27 10:04:37,281][20381] Updated weights for policy 1, policy_version 642044 (0.0008) [2023-12-27 10:04:37,344][20381] Updated weights for policy 1, policy_version 642054 (0.0011) [2023-12-27 10:04:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20688.3). Total num frames: 328908800. Throughput: 0: 10397.3, 1: 10067.3. Samples: 328882760. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:37,363][19170] Avg episode reward: [(0, '805.860'), (1, '455.360')] [2023-12-27 10:04:37,395][20347] Updated weights for policy 0, policy_version 642599 (0.0006) [2023-12-27 10:04:37,400][20381] Updated weights for policy 1, policy_version 642064 (0.0011) [2023-12-27 10:04:37,443][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000642072_164397056.pth... [2023-12-27 10:04:37,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000640888_164093952.pth [2023-12-27 10:04:37,451][20347] Updated weights for policy 0, policy_version 642609 (0.0005) [2023-12-27 10:04:37,507][20347] Updated weights for policy 0, policy_version 642619 (0.0007) [2023-12-27 10:04:37,532][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000642624_164528128.pth... [2023-12-27 10:04:37,536][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000641408_164216832.pth [2023-12-27 10:04:38,081][20381] Updated weights for policy 1, policy_version 642074 (0.0010) [2023-12-27 10:04:38,144][20381] Updated weights for policy 1, policy_version 642084 (0.0008) [2023-12-27 10:04:38,180][20347] Updated weights for policy 0, policy_version 642629 (0.0007) [2023-12-27 10:04:38,199][20381] Updated weights for policy 1, policy_version 642094 (0.0005) [2023-12-27 10:04:38,230][20347] Updated weights for policy 0, policy_version 642639 (0.0007) [2023-12-27 10:04:38,261][20381] Updated weights for policy 1, policy_version 642104 (0.0009) [2023-12-27 10:04:38,285][20347] Updated weights for policy 0, policy_version 642649 (0.0006) [2023-12-27 10:04:38,912][20381] Updated weights for policy 1, policy_version 642114 (0.0011) [2023-12-27 10:04:38,967][20381] Updated weights for policy 1, policy_version 642124 (0.0010) [2023-12-27 10:04:38,983][20347] Updated weights for policy 0, policy_version 642659 (0.0006) [2023-12-27 10:04:39,015][20381] Updated weights for policy 1, policy_version 642134 (0.0010) [2023-12-27 10:04:39,032][20347] Updated weights for policy 0, policy_version 642669 (0.0005) [2023-12-27 10:04:39,085][20347] Updated weights for policy 0, policy_version 642679 (0.0008) [2023-12-27 10:04:39,713][20381] Updated weights for policy 1, policy_version 642144 (0.0011) [2023-12-27 10:04:39,773][20381] Updated weights for policy 1, policy_version 642154 (0.0011) [2023-12-27 10:04:39,839][20381] Updated weights for policy 1, policy_version 642164 (0.0011) [2023-12-27 10:04:39,896][20347] Updated weights for policy 0, policy_version 642689 (0.0007) [2023-12-27 10:04:39,964][20347] Updated weights for policy 0, policy_version 642699 (0.0007) [2023-12-27 10:04:40,030][20347] Updated weights for policy 0, policy_version 642709 (0.0008) [2023-12-27 10:04:40,089][20347] Updated weights for policy 0, policy_version 642719 (0.0007) [2023-12-27 10:04:40,571][20381] Updated weights for policy 1, policy_version 642174 (0.0008) [2023-12-27 10:04:40,626][20381] Updated weights for policy 1, policy_version 642184 (0.0008) [2023-12-27 10:04:40,686][20381] Updated weights for policy 1, policy_version 642194 (0.0007) [2023-12-27 10:04:40,739][20347] Updated weights for policy 0, policy_version 642729 (0.0007) [2023-12-27 10:04:40,789][20347] Updated weights for policy 0, policy_version 642739 (0.0007) [2023-12-27 10:04:40,844][20347] Updated weights for policy 0, policy_version 642749 (0.0008) [2023-12-27 10:04:41,326][20381] Updated weights for policy 1, policy_version 642204 (0.0006) [2023-12-27 10:04:41,391][20381] Updated weights for policy 1, policy_version 642214 (0.0007) [2023-12-27 10:04:41,453][20381] Updated weights for policy 1, policy_version 642224 (0.0007) [2023-12-27 10:04:41,559][20347] Updated weights for policy 0, policy_version 642759 (0.0010) [2023-12-27 10:04:41,616][20347] Updated weights for policy 0, policy_version 642769 (0.0010) [2023-12-27 10:04:41,677][20347] Updated weights for policy 0, policy_version 642779 (0.0008) [2023-12-27 10:04:42,103][20381] Updated weights for policy 1, policy_version 642234 (0.0007) [2023-12-27 10:04:42,158][20381] Updated weights for policy 1, policy_version 642244 (0.0007) [2023-12-27 10:04:42,216][20381] Updated weights for policy 1, policy_version 642254 (0.0007) [2023-12-27 10:04:42,274][20381] Updated weights for policy 1, policy_version 642264 (0.0008) [2023-12-27 10:04:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20688.3). Total num frames: 329015296. Throughput: 0: 10301.6, 1: 10149.9. Samples: 329004308. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:42,363][19170] Avg episode reward: [(0, '803.350'), (1, '355.000')] [2023-12-27 10:04:42,389][20347] Updated weights for policy 0, policy_version 642789 (0.0008) [2023-12-27 10:04:42,409][20291] KL-divergence is very high: 109.4448 [2023-12-27 10:04:42,450][20347] Updated weights for policy 0, policy_version 642799 (0.0010) [2023-12-27 10:04:42,507][20347] Updated weights for policy 0, policy_version 642809 (0.0010) [2023-12-27 10:04:43,001][20381] Updated weights for policy 1, policy_version 642274 (0.0007) [2023-12-27 10:04:43,057][20381] Updated weights for policy 1, policy_version 642284 (0.0008) [2023-12-27 10:04:43,107][20381] Updated weights for policy 1, policy_version 642294 (0.0010) [2023-12-27 10:04:43,161][20347] Updated weights for policy 0, policy_version 642819 (0.0008) [2023-12-27 10:04:43,218][20347] Updated weights for policy 0, policy_version 642829 (0.0006) [2023-12-27 10:04:43,278][20347] Updated weights for policy 0, policy_version 642839 (0.0005) [2023-12-27 10:04:43,781][20381] Updated weights for policy 1, policy_version 642304 (0.0011) [2023-12-27 10:04:43,824][20347] Updated weights for policy 0, policy_version 642849 (0.0008) [2023-12-27 10:04:43,836][20381] Updated weights for policy 1, policy_version 642314 (0.0011) [2023-12-27 10:04:43,880][20347] Updated weights for policy 0, policy_version 642859 (0.0005) [2023-12-27 10:04:43,891][20381] Updated weights for policy 1, policy_version 642324 (0.0010) [2023-12-27 10:04:43,928][20347] Updated weights for policy 0, policy_version 642869 (0.0006) [2023-12-27 10:04:43,982][20347] Updated weights for policy 0, policy_version 642879 (0.0007) [2023-12-27 10:04:44,594][20381] Updated weights for policy 1, policy_version 642334 (0.0011) [2023-12-27 10:04:44,628][20347] Updated weights for policy 0, policy_version 642889 (0.0008) [2023-12-27 10:04:44,647][20381] Updated weights for policy 1, policy_version 642344 (0.0008) [2023-12-27 10:04:44,683][20347] Updated weights for policy 0, policy_version 642899 (0.0008) [2023-12-27 10:04:44,701][20381] Updated weights for policy 1, policy_version 642354 (0.0007) [2023-12-27 10:04:44,739][20347] Updated weights for policy 0, policy_version 642909 (0.0007) [2023-12-27 10:04:45,330][20381] Updated weights for policy 1, policy_version 642364 (0.0011) [2023-12-27 10:04:45,387][20381] Updated weights for policy 1, policy_version 642374 (0.0011) [2023-12-27 10:04:45,442][20381] Updated weights for policy 1, policy_version 642384 (0.0010) [2023-12-27 10:04:45,523][20347] Updated weights for policy 0, policy_version 642919 (0.0007) [2023-12-27 10:04:45,579][20347] Updated weights for policy 0, policy_version 642929 (0.0008) [2023-12-27 10:04:45,629][20347] Updated weights for policy 0, policy_version 642939 (0.0009) [2023-12-27 10:04:46,031][20381] Updated weights for policy 1, policy_version 642394 (0.0010) [2023-12-27 10:04:46,079][20381] Updated weights for policy 1, policy_version 642404 (0.0010) [2023-12-27 10:04:46,128][20381] Updated weights for policy 1, policy_version 642414 (0.0010) [2023-12-27 10:04:46,173][20381] Updated weights for policy 1, policy_version 642424 (0.0010) [2023-12-27 10:04:46,369][20347] Updated weights for policy 0, policy_version 642949 (0.0008) [2023-12-27 10:04:46,426][20347] Updated weights for policy 0, policy_version 642959 (0.0007) [2023-12-27 10:04:46,473][20347] Updated weights for policy 0, policy_version 642969 (0.0007) [2023-12-27 10:04:46,814][20381] Updated weights for policy 1, policy_version 642434 (0.0009) [2023-12-27 10:04:46,866][20381] Updated weights for policy 1, policy_version 642444 (0.0010) [2023-12-27 10:04:46,923][20381] Updated weights for policy 1, policy_version 642454 (0.0007) [2023-12-27 10:04:47,166][20347] Updated weights for policy 0, policy_version 642979 (0.0008) [2023-12-27 10:04:47,215][20347] Updated weights for policy 0, policy_version 642989 (0.0007) [2023-12-27 10:04:47,268][20347] Updated weights for policy 0, policy_version 642999 (0.0007) [2023-12-27 10:04:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 329121792. Throughput: 0: 10270.1, 1: 10213.9. Samples: 329128308. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:47,363][19170] Avg episode reward: [(0, '644.160'), (1, '404.030')] [2023-12-27 10:04:47,576][20381] Updated weights for policy 1, policy_version 642464 (0.0009) [2023-12-27 10:04:47,628][20381] Updated weights for policy 1, policy_version 642474 (0.0010) [2023-12-27 10:04:47,681][20381] Updated weights for policy 1, policy_version 642484 (0.0010) [2023-12-27 10:04:47,979][20347] Updated weights for policy 0, policy_version 643009 (0.0008) [2023-12-27 10:04:48,034][20347] Updated weights for policy 0, policy_version 643019 (0.0007) [2023-12-27 10:04:48,084][20347] Updated weights for policy 0, policy_version 643029 (0.0007) [2023-12-27 10:04:48,131][20347] Updated weights for policy 0, policy_version 643039 (0.0007) [2023-12-27 10:04:48,342][20381] Updated weights for policy 1, policy_version 642494 (0.0008) [2023-12-27 10:04:48,401][20381] Updated weights for policy 1, policy_version 642504 (0.0010) [2023-12-27 10:04:48,457][20381] Updated weights for policy 1, policy_version 642514 (0.0011) [2023-12-27 10:04:48,782][20347] Updated weights for policy 0, policy_version 643049 (0.0007) [2023-12-27 10:04:48,833][20347] Updated weights for policy 0, policy_version 643059 (0.0007) [2023-12-27 10:04:48,881][20347] Updated weights for policy 0, policy_version 643069 (0.0008) [2023-12-27 10:04:49,150][20381] Updated weights for policy 1, policy_version 642524 (0.0011) [2023-12-27 10:04:49,204][20381] Updated weights for policy 1, policy_version 642534 (0.0010) [2023-12-27 10:04:49,265][20381] Updated weights for policy 1, policy_version 642544 (0.0010) [2023-12-27 10:04:49,587][20347] Updated weights for policy 0, policy_version 643079 (0.0009) [2023-12-27 10:04:49,655][20347] Updated weights for policy 0, policy_version 643089 (0.0010) [2023-12-27 10:04:49,722][20347] Updated weights for policy 0, policy_version 643099 (0.0006) [2023-12-27 10:04:49,914][20381] Updated weights for policy 1, policy_version 642554 (0.0009) [2023-12-27 10:04:49,977][20381] Updated weights for policy 1, policy_version 642564 (0.0007) [2023-12-27 10:04:50,036][20381] Updated weights for policy 1, policy_version 642574 (0.0006) [2023-12-27 10:04:50,088][20381] Updated weights for policy 1, policy_version 642584 (0.0009) [2023-12-27 10:04:50,413][20347] Updated weights for policy 0, policy_version 643109 (0.0006) [2023-12-27 10:04:50,469][20347] Updated weights for policy 0, policy_version 643119 (0.0007) [2023-12-27 10:04:50,532][20347] Updated weights for policy 0, policy_version 643129 (0.0008) [2023-12-27 10:04:50,767][20381] Updated weights for policy 1, policy_version 642594 (0.0011) [2023-12-27 10:04:50,820][20381] Updated weights for policy 1, policy_version 642604 (0.0011) [2023-12-27 10:04:50,873][20381] Updated weights for policy 1, policy_version 642614 (0.0011) [2023-12-27 10:04:51,171][20347] Updated weights for policy 0, policy_version 643139 (0.0007) [2023-12-27 10:04:51,232][20347] Updated weights for policy 0, policy_version 643149 (0.0006) [2023-12-27 10:04:51,287][20347] Updated weights for policy 0, policy_version 643159 (0.0008) [2023-12-27 10:04:51,580][20381] Updated weights for policy 1, policy_version 642624 (0.0011) [2023-12-27 10:04:51,633][20381] Updated weights for policy 1, policy_version 642634 (0.0009) [2023-12-27 10:04:51,691][20381] Updated weights for policy 1, policy_version 642644 (0.0010) [2023-12-27 10:04:51,989][20347] Updated weights for policy 0, policy_version 643169 (0.0007) [2023-12-27 10:04:52,052][20347] Updated weights for policy 0, policy_version 643179 (0.0005) [2023-12-27 10:04:52,117][20347] Updated weights for policy 0, policy_version 643189 (0.0006) [2023-12-27 10:04:52,181][20347] Updated weights for policy 0, policy_version 643199 (0.0005) [2023-12-27 10:04:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 329220096. Throughput: 0: 10274.8, 1: 10198.1. Samples: 329189636. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:52,363][19170] Avg episode reward: [(0, '674.760'), (1, '380.850')] [2023-12-27 10:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000643200_164675584.pth... [2023-12-27 10:04:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000642016_164372480.pth [2023-12-27 10:04:52,377][20381] Updated weights for policy 1, policy_version 642654 (0.0010) [2023-12-27 10:04:52,432][20381] Updated weights for policy 1, policy_version 642664 (0.0010) [2023-12-27 10:04:52,487][20381] Updated weights for policy 1, policy_version 642674 (0.0010) [2023-12-27 10:04:52,524][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000642680_164552704.pth... [2023-12-27 10:04:52,528][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000641464_164241408.pth [2023-12-27 10:04:52,776][20347] Updated weights for policy 0, policy_version 643209 (0.0005) [2023-12-27 10:04:52,834][20347] Updated weights for policy 0, policy_version 643219 (0.0006) [2023-12-27 10:04:52,894][20347] Updated weights for policy 0, policy_version 643229 (0.0006) [2023-12-27 10:04:53,125][20381] Updated weights for policy 1, policy_version 642684 (0.0008) [2023-12-27 10:04:53,192][20381] Updated weights for policy 1, policy_version 642694 (0.0005) [2023-12-27 10:04:53,256][20381] Updated weights for policy 1, policy_version 642704 (0.0007) [2023-12-27 10:04:53,484][20347] Updated weights for policy 0, policy_version 643239 (0.0007) [2023-12-27 10:04:53,546][20347] Updated weights for policy 0, policy_version 643249 (0.0008) [2023-12-27 10:04:53,600][20347] Updated weights for policy 0, policy_version 643259 (0.0008) [2023-12-27 10:04:53,909][20381] Updated weights for policy 1, policy_version 642714 (0.0008) [2023-12-27 10:04:53,959][20381] Updated weights for policy 1, policy_version 642724 (0.0007) [2023-12-27 10:04:54,013][20381] Updated weights for policy 1, policy_version 642734 (0.0007) [2023-12-27 10:04:54,069][20381] Updated weights for policy 1, policy_version 642744 (0.0008) [2023-12-27 10:04:54,209][20347] Updated weights for policy 0, policy_version 643269 (0.0005) [2023-12-27 10:04:54,263][20347] Updated weights for policy 0, policy_version 643279 (0.0005) [2023-12-27 10:04:54,317][20347] Updated weights for policy 0, policy_version 643289 (0.0005) [2023-12-27 10:04:54,697][20381] Updated weights for policy 1, policy_version 642754 (0.0007) [2023-12-27 10:04:54,749][20381] Updated weights for policy 1, policy_version 642764 (0.0008) [2023-12-27 10:04:54,817][20381] Updated weights for policy 1, policy_version 642774 (0.0009) [2023-12-27 10:04:54,857][20347] Updated weights for policy 0, policy_version 643299 (0.0006) [2023-12-27 10:04:54,919][20347] Updated weights for policy 0, policy_version 643309 (0.0008) [2023-12-27 10:04:54,972][20347] Updated weights for policy 0, policy_version 643319 (0.0010) [2023-12-27 10:04:55,413][20381] Updated weights for policy 1, policy_version 642784 (0.0007) [2023-12-27 10:04:55,463][20381] Updated weights for policy 1, policy_version 642794 (0.0007) [2023-12-27 10:04:55,524][20381] Updated weights for policy 1, policy_version 642804 (0.0007) [2023-12-27 10:04:55,676][20347] Updated weights for policy 0, policy_version 643329 (0.0010) [2023-12-27 10:04:55,729][20347] Updated weights for policy 0, policy_version 643339 (0.0011) [2023-12-27 10:04:55,781][20347] Updated weights for policy 0, policy_version 643349 (0.0011) [2023-12-27 10:04:55,831][20347] Updated weights for policy 0, policy_version 643359 (0.0010) [2023-12-27 10:04:56,200][20381] Updated weights for policy 1, policy_version 642814 (0.0007) [2023-12-27 10:04:56,255][20381] Updated weights for policy 1, policy_version 642824 (0.0007) [2023-12-27 10:04:56,309][20381] Updated weights for policy 1, policy_version 642834 (0.0008) [2023-12-27 10:04:56,522][20347] Updated weights for policy 0, policy_version 643369 (0.0010) [2023-12-27 10:04:56,576][20347] Updated weights for policy 0, policy_version 643379 (0.0010) [2023-12-27 10:04:56,630][20347] Updated weights for policy 0, policy_version 643389 (0.0010) [2023-12-27 10:04:57,005][20381] Updated weights for policy 1, policy_version 642844 (0.0006) [2023-12-27 10:04:57,052][20381] Updated weights for policy 1, policy_version 642854 (0.0005) [2023-12-27 10:04:57,099][20381] Updated weights for policy 1, policy_version 642864 (0.0005) [2023-12-27 10:04:57,279][20347] Updated weights for policy 0, policy_version 643399 (0.0008) [2023-12-27 10:04:57,331][20347] Updated weights for policy 0, policy_version 643409 (0.0009) [2023-12-27 10:04:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 329326592. Throughput: 0: 10294.4, 1: 10244.2. Samples: 329317016. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:04:57,364][19170] Avg episode reward: [(0, '816.630'), (1, '423.380')] [2023-12-27 10:04:57,385][20347] Updated weights for policy 0, policy_version 643419 (0.0010) [2023-12-27 10:04:57,727][20381] Updated weights for policy 1, policy_version 642874 (0.0008) [2023-12-27 10:04:57,775][20381] Updated weights for policy 1, policy_version 642884 (0.0008) [2023-12-27 10:04:57,820][20381] Updated weights for policy 1, policy_version 642894 (0.0008) [2023-12-27 10:04:57,882][20381] Updated weights for policy 1, policy_version 642904 (0.0008) [2023-12-27 10:04:57,964][20347] Updated weights for policy 0, policy_version 643429 (0.0008) [2023-12-27 10:04:58,014][20347] Updated weights for policy 0, policy_version 643439 (0.0006) [2023-12-27 10:04:58,067][20347] Updated weights for policy 0, policy_version 643449 (0.0005) [2023-12-27 10:04:58,647][20381] Updated weights for policy 1, policy_version 642914 (0.0007) [2023-12-27 10:04:58,702][20381] Updated weights for policy 1, policy_version 642924 (0.0007) [2023-12-27 10:04:58,747][20347] Updated weights for policy 0, policy_version 643459 (0.0006) [2023-12-27 10:04:58,763][20381] Updated weights for policy 1, policy_version 642934 (0.0008) [2023-12-27 10:04:58,807][20347] Updated weights for policy 0, policy_version 643469 (0.0007) [2023-12-27 10:04:58,868][20347] Updated weights for policy 0, policy_version 643479 (0.0008) [2023-12-27 10:04:59,453][20381] Updated weights for policy 1, policy_version 642944 (0.0010) [2023-12-27 10:04:59,504][20381] Updated weights for policy 1, policy_version 642954 (0.0010) [2023-12-27 10:04:59,548][20347] Updated weights for policy 0, policy_version 643489 (0.0008) [2023-12-27 10:04:59,549][20381] Updated weights for policy 1, policy_version 642964 (0.0010) [2023-12-27 10:04:59,609][20347] Updated weights for policy 0, policy_version 643499 (0.0008) [2023-12-27 10:04:59,670][20347] Updated weights for policy 0, policy_version 643509 (0.0007) [2023-12-27 10:04:59,727][20347] Updated weights for policy 0, policy_version 643519 (0.0007) [2023-12-27 10:05:00,265][20381] Updated weights for policy 1, policy_version 642974 (0.0010) [2023-12-27 10:05:00,310][20381] Updated weights for policy 1, policy_version 642984 (0.0010) [2023-12-27 10:05:00,359][20381] Updated weights for policy 1, policy_version 642994 (0.0010) [2023-12-27 10:05:00,442][20347] Updated weights for policy 0, policy_version 643529 (0.0007) [2023-12-27 10:05:00,504][20347] Updated weights for policy 0, policy_version 643539 (0.0007) [2023-12-27 10:05:00,555][20347] Updated weights for policy 0, policy_version 643549 (0.0007) [2023-12-27 10:05:01,056][20381] Updated weights for policy 1, policy_version 643004 (0.0010) [2023-12-27 10:05:01,112][20381] Updated weights for policy 1, policy_version 643014 (0.0010) [2023-12-27 10:05:01,176][20381] Updated weights for policy 1, policy_version 643024 (0.0011) [2023-12-27 10:05:01,254][20347] Updated weights for policy 0, policy_version 643559 (0.0007) [2023-12-27 10:05:01,315][20347] Updated weights for policy 0, policy_version 643569 (0.0008) [2023-12-27 10:05:01,374][20347] Updated weights for policy 0, policy_version 643579 (0.0008) [2023-12-27 10:05:01,837][20381] Updated weights for policy 1, policy_version 643034 (0.0010) [2023-12-27 10:05:01,889][20381] Updated weights for policy 1, policy_version 643044 (0.0010) [2023-12-27 10:05:01,943][20381] Updated weights for policy 1, policy_version 643054 (0.0010) [2023-12-27 10:05:02,000][20381] Updated weights for policy 1, policy_version 643064 (0.0005) [2023-12-27 10:05:02,127][20347] Updated weights for policy 0, policy_version 643589 (0.0006) [2023-12-27 10:05:02,194][20347] Updated weights for policy 0, policy_version 643599 (0.0005) [2023-12-27 10:05:02,259][20347] Updated weights for policy 0, policy_version 643609 (0.0005) [2023-12-27 10:05:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20660.7). Total num frames: 329433088. Throughput: 0: 10239.7, 1: 10280.9. Samples: 329439220. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:05:02,363][19170] Avg episode reward: [(0, '796.840'), (1, '452.310')] [2023-12-27 10:05:02,587][20381] Updated weights for policy 1, policy_version 643074 (0.0010) [2023-12-27 10:05:02,651][20381] Updated weights for policy 1, policy_version 643084 (0.0009) [2023-12-27 10:05:02,714][20381] Updated weights for policy 1, policy_version 643094 (0.0010) [2023-12-27 10:05:02,833][20347] Updated weights for policy 0, policy_version 643619 (0.0006) [2023-12-27 10:05:02,879][20347] Updated weights for policy 0, policy_version 643629 (0.0006) [2023-12-27 10:05:02,927][20347] Updated weights for policy 0, policy_version 643639 (0.0005) [2023-12-27 10:05:03,319][20381] Updated weights for policy 1, policy_version 643104 (0.0006) [2023-12-27 10:05:03,380][20381] Updated weights for policy 1, policy_version 643114 (0.0005) [2023-12-27 10:05:03,436][20381] Updated weights for policy 1, policy_version 643124 (0.0006) [2023-12-27 10:05:03,621][20347] Updated weights for policy 0, policy_version 643649 (0.0005) [2023-12-27 10:05:03,669][20347] Updated weights for policy 0, policy_version 643659 (0.0007) [2023-12-27 10:05:03,711][20347] Updated weights for policy 0, policy_version 643669 (0.0007) [2023-12-27 10:05:03,756][20347] Updated weights for policy 0, policy_version 643679 (0.0007) [2023-12-27 10:05:03,965][20381] Updated weights for policy 1, policy_version 643134 (0.0007) [2023-12-27 10:05:04,020][20381] Updated weights for policy 1, policy_version 643144 (0.0008) [2023-12-27 10:05:04,080][20381] Updated weights for policy 1, policy_version 643154 (0.0006) [2023-12-27 10:05:04,509][20347] Updated weights for policy 0, policy_version 643689 (0.0007) [2023-12-27 10:05:04,569][20347] Updated weights for policy 0, policy_version 643699 (0.0008) [2023-12-27 10:05:04,628][20347] Updated weights for policy 0, policy_version 643709 (0.0007) [2023-12-27 10:05:04,756][20381] Updated weights for policy 1, policy_version 643164 (0.0007) [2023-12-27 10:05:04,817][20381] Updated weights for policy 1, policy_version 643174 (0.0007) [2023-12-27 10:05:04,880][20381] Updated weights for policy 1, policy_version 643184 (0.0008) [2023-12-27 10:05:05,312][20347] Updated weights for policy 0, policy_version 643719 (0.0007) [2023-12-27 10:05:05,371][20347] Updated weights for policy 0, policy_version 643729 (0.0007) [2023-12-27 10:05:05,422][20347] Updated weights for policy 0, policy_version 643739 (0.0007) [2023-12-27 10:05:05,528][20381] Updated weights for policy 1, policy_version 643194 (0.0008) [2023-12-27 10:05:05,587][20381] Updated weights for policy 1, policy_version 643204 (0.0008) [2023-12-27 10:05:05,640][20381] Updated weights for policy 1, policy_version 643214 (0.0008) [2023-12-27 10:05:05,698][20381] Updated weights for policy 1, policy_version 643224 (0.0007) [2023-12-27 10:05:06,068][20347] Updated weights for policy 0, policy_version 643749 (0.0007) [2023-12-27 10:05:06,120][20347] Updated weights for policy 0, policy_version 643759 (0.0008) [2023-12-27 10:05:06,173][20347] Updated weights for policy 0, policy_version 643769 (0.0008) [2023-12-27 10:05:06,422][20381] Updated weights for policy 1, policy_version 643234 (0.0008) [2023-12-27 10:05:06,484][20381] Updated weights for policy 1, policy_version 643244 (0.0007) [2023-12-27 10:05:06,545][20381] Updated weights for policy 1, policy_version 643254 (0.0008) [2023-12-27 10:05:06,888][20347] Updated weights for policy 0, policy_version 643779 (0.0008) [2023-12-27 10:05:06,949][20347] Updated weights for policy 0, policy_version 643789 (0.0008) [2023-12-27 10:05:07,008][20347] Updated weights for policy 0, policy_version 643799 (0.0007) [2023-12-27 10:05:07,264][20381] Updated weights for policy 1, policy_version 643264 (0.0008) [2023-12-27 10:05:07,329][20381] Updated weights for policy 1, policy_version 643274 (0.0008) [2023-12-27 10:05:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 329531392. Throughput: 0: 10258.7, 1: 10355.5. Samples: 329503440. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:05:07,363][19170] Avg episode reward: [(0, '854.660'), (1, '383.890')] [2023-12-27 10:05:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000643808_164831232.pth... [2023-12-27 10:05:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000642624_164528128.pth [2023-12-27 10:05:07,383][20381] Updated weights for policy 1, policy_version 643284 (0.0008) [2023-12-27 10:05:07,403][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000643288_164708352.pth... [2023-12-27 10:05:07,407][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000642072_164397056.pth [2023-12-27 10:05:07,590][20347] Updated weights for policy 0, policy_version 643809 (0.0007) [2023-12-27 10:05:07,644][20347] Updated weights for policy 0, policy_version 643819 (0.0005) [2023-12-27 10:05:07,702][20347] Updated weights for policy 0, policy_version 643829 (0.0005) [2023-12-27 10:05:07,764][20347] Updated weights for policy 0, policy_version 643839 (0.0005) [2023-12-27 10:05:08,119][20381] Updated weights for policy 1, policy_version 643294 (0.0008) [2023-12-27 10:05:08,173][20381] Updated weights for policy 1, policy_version 643304 (0.0008) [2023-12-27 10:05:08,228][20381] Updated weights for policy 1, policy_version 643314 (0.0008) [2023-12-27 10:05:08,379][20347] Updated weights for policy 0, policy_version 643849 (0.0008) [2023-12-27 10:05:08,426][20347] Updated weights for policy 0, policy_version 643859 (0.0007) [2023-12-27 10:05:08,475][20347] Updated weights for policy 0, policy_version 643869 (0.0007) [2023-12-27 10:05:08,919][20381] Updated weights for policy 1, policy_version 643324 (0.0008) [2023-12-27 10:05:08,967][20381] Updated weights for policy 1, policy_version 643334 (0.0007) [2023-12-27 10:05:09,022][20381] Updated weights for policy 1, policy_version 643344 (0.0008) [2023-12-27 10:05:09,141][20347] Updated weights for policy 0, policy_version 643879 (0.0006) [2023-12-27 10:05:09,199][20347] Updated weights for policy 0, policy_version 643889 (0.0008) [2023-12-27 10:05:09,265][20347] Updated weights for policy 0, policy_version 643899 (0.0011) [2023-12-27 10:05:09,777][20381] Updated weights for policy 1, policy_version 643354 (0.0007) [2023-12-27 10:05:09,849][20381] Updated weights for policy 1, policy_version 643364 (0.0007) [2023-12-27 10:05:09,925][20381] Updated weights for policy 1, policy_version 643374 (0.0008) [2023-12-27 10:05:09,961][20347] Updated weights for policy 0, policy_version 643909 (0.0007) [2023-12-27 10:05:09,987][20381] Updated weights for policy 1, policy_version 643384 (0.0007) [2023-12-27 10:05:10,018][20347] Updated weights for policy 0, policy_version 643919 (0.0008) [2023-12-27 10:05:10,076][20347] Updated weights for policy 0, policy_version 643929 (0.0007) [2023-12-27 10:05:10,703][20347] Updated weights for policy 0, policy_version 643939 (0.0007) [2023-12-27 10:05:10,706][20381] Updated weights for policy 1, policy_version 643394 (0.0009) [2023-12-27 10:05:10,756][20381] Updated weights for policy 1, policy_version 643404 (0.0008) [2023-12-27 10:05:10,760][20347] Updated weights for policy 0, policy_version 643949 (0.0005) [2023-12-27 10:05:10,817][20347] Updated weights for policy 0, policy_version 643959 (0.0005) [2023-12-27 10:05:10,818][20381] Updated weights for policy 1, policy_version 643414 (0.0008) [2023-12-27 10:05:11,448][20347] Updated weights for policy 0, policy_version 643969 (0.0006) [2023-12-27 10:05:11,506][20347] Updated weights for policy 0, policy_version 643979 (0.0006) [2023-12-27 10:05:11,510][20381] Updated weights for policy 1, policy_version 643424 (0.0007) [2023-12-27 10:05:11,563][20347] Updated weights for policy 0, policy_version 643989 (0.0009) [2023-12-27 10:05:11,577][20381] Updated weights for policy 1, policy_version 643434 (0.0008) [2023-12-27 10:05:11,626][20347] Updated weights for policy 0, policy_version 643999 (0.0008) [2023-12-27 10:05:11,642][20381] Updated weights for policy 1, policy_version 643444 (0.0007) [2023-12-27 10:05:12,301][20381] Updated weights for policy 1, policy_version 643454 (0.0008) [2023-12-27 10:05:12,352][20347] Updated weights for policy 0, policy_version 644009 (0.0007) [2023-12-27 10:05:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 329629696. Throughput: 0: 10268.0, 1: 10307.7. Samples: 329625180. Policy #0 lag: (min: 12.0, avg: 16.0, max: 44.0) [2023-12-27 10:05:12,363][19170] Avg episode reward: [(0, '932.220'), (1, '465.680')] [2023-12-27 10:05:12,364][20381] Updated weights for policy 1, policy_version 643464 (0.0007) [2023-12-27 10:05:12,409][20347] Updated weights for policy 0, policy_version 644019 (0.0008) [2023-12-27 10:05:12,426][20381] Updated weights for policy 1, policy_version 643474 (0.0007) [2023-12-27 10:05:12,469][20347] Updated weights for policy 0, policy_version 644029 (0.0006) [2023-12-27 10:05:13,056][20381] Updated weights for policy 1, policy_version 643484 (0.0006) [2023-12-27 10:05:13,119][20381] Updated weights for policy 1, policy_version 643494 (0.0005) [2023-12-27 10:05:13,170][20381] Updated weights for policy 1, policy_version 643504 (0.0005) [2023-12-27 10:05:13,193][20347] Updated weights for policy 0, policy_version 644039 (0.0008) [2023-12-27 10:05:13,246][20347] Updated weights for policy 0, policy_version 644049 (0.0005) [2023-12-27 10:05:13,295][20347] Updated weights for policy 0, policy_version 644059 (0.0007) [2023-12-27 10:05:13,762][20381] Updated weights for policy 1, policy_version 643514 (0.0006) [2023-12-27 10:05:13,809][20381] Updated weights for policy 1, policy_version 643524 (0.0008) [2023-12-27 10:05:13,861][20381] Updated weights for policy 1, policy_version 643534 (0.0007) [2023-12-27 10:05:13,909][20381] Updated weights for policy 1, policy_version 643544 (0.0008) [2023-12-27 10:05:13,990][20347] Updated weights for policy 0, policy_version 644069 (0.0007) [2023-12-27 10:05:14,048][20347] Updated weights for policy 0, policy_version 644079 (0.0008) [2023-12-27 10:05:14,096][20347] Updated weights for policy 0, policy_version 644089 (0.0007) [2023-12-27 10:05:14,550][20381] Updated weights for policy 1, policy_version 643554 (0.0006) [2023-12-27 10:05:14,601][20381] Updated weights for policy 1, policy_version 643564 (0.0007) [2023-12-27 10:05:14,650][20381] Updated weights for policy 1, policy_version 643574 (0.0008) [2023-12-27 10:05:14,856][20347] Updated weights for policy 0, policy_version 644099 (0.0008) [2023-12-27 10:05:14,911][20347] Updated weights for policy 0, policy_version 644109 (0.0007) [2023-12-27 10:05:14,968][20347] Updated weights for policy 0, policy_version 644119 (0.0008) [2023-12-27 10:05:15,297][20381] Updated weights for policy 1, policy_version 643584 (0.0007) [2023-12-27 10:05:15,357][20381] Updated weights for policy 1, policy_version 643594 (0.0008) [2023-12-27 10:05:15,413][20381] Updated weights for policy 1, policy_version 643604 (0.0008) [2023-12-27 10:05:15,714][20347] Updated weights for policy 0, policy_version 644129 (0.0008) [2023-12-27 10:05:15,770][20347] Updated weights for policy 0, policy_version 644139 (0.0008) [2023-12-27 10:05:15,826][20347] Updated weights for policy 0, policy_version 644149 (0.0007) [2023-12-27 10:05:15,879][20347] Updated weights for policy 0, policy_version 644159 (0.0008) [2023-12-27 10:05:16,036][20381] Updated weights for policy 1, policy_version 643614 (0.0008) [2023-12-27 10:05:16,084][20381] Updated weights for policy 1, policy_version 643624 (0.0007) [2023-12-27 10:05:16,138][20381] Updated weights for policy 1, policy_version 643634 (0.0007) [2023-12-27 10:05:16,590][20347] Updated weights for policy 0, policy_version 644169 (0.0008) [2023-12-27 10:05:16,659][20347] Updated weights for policy 0, policy_version 644179 (0.0008) [2023-12-27 10:05:16,715][20347] Updated weights for policy 0, policy_version 644189 (0.0008) [2023-12-27 10:05:16,831][20381] Updated weights for policy 1, policy_version 643644 (0.0008) [2023-12-27 10:05:16,879][20381] Updated weights for policy 1, policy_version 643654 (0.0010) [2023-12-27 10:05:16,936][20381] Updated weights for policy 1, policy_version 643664 (0.0011) [2023-12-27 10:05:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 329736192. Throughput: 0: 10203.2, 1: 10365.9. Samples: 329747220. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:17,363][19170] Avg episode reward: [(0, '923.090'), (1, '710.260')] [2023-12-27 10:05:17,419][20347] Updated weights for policy 0, policy_version 644199 (0.0008) [2023-12-27 10:05:17,479][20347] Updated weights for policy 0, policy_version 644209 (0.0007) [2023-12-27 10:05:17,536][20347] Updated weights for policy 0, policy_version 644219 (0.0007) [2023-12-27 10:05:17,624][20381] Updated weights for policy 1, policy_version 643674 (0.0007) [2023-12-27 10:05:17,681][20381] Updated weights for policy 1, policy_version 643684 (0.0007) [2023-12-27 10:05:17,735][20381] Updated weights for policy 1, policy_version 643694 (0.0008) [2023-12-27 10:05:17,789][20381] Updated weights for policy 1, policy_version 643704 (0.0007) [2023-12-27 10:05:18,208][20347] Updated weights for policy 0, policy_version 644229 (0.0008) [2023-12-27 10:05:18,265][20347] Updated weights for policy 0, policy_version 644239 (0.0010) [2023-12-27 10:05:18,330][20347] Updated weights for policy 0, policy_version 644249 (0.0010) [2023-12-27 10:05:18,368][20381] Updated weights for policy 1, policy_version 643714 (0.0007) [2023-12-27 10:05:18,420][20381] Updated weights for policy 1, policy_version 643724 (0.0006) [2023-12-27 10:05:18,480][20381] Updated weights for policy 1, policy_version 643734 (0.0007) [2023-12-27 10:05:18,938][20347] Updated weights for policy 0, policy_version 644259 (0.0009) [2023-12-27 10:05:18,986][20347] Updated weights for policy 0, policy_version 644269 (0.0006) [2023-12-27 10:05:19,038][20347] Updated weights for policy 0, policy_version 644279 (0.0008) [2023-12-27 10:05:19,067][20381] Updated weights for policy 1, policy_version 643744 (0.0006) [2023-12-27 10:05:19,118][20381] Updated weights for policy 1, policy_version 643754 (0.0007) [2023-12-27 10:05:19,164][20381] Updated weights for policy 1, policy_version 643764 (0.0007) [2023-12-27 10:05:19,726][20347] Updated weights for policy 0, policy_version 644289 (0.0007) [2023-12-27 10:05:19,770][20291] KL-divergence is very high: 143.2876 [2023-12-27 10:05:19,788][20291] KL-divergence is very high: 152.3735 [2023-12-27 10:05:19,790][20347] Updated weights for policy 0, policy_version 644299 (0.0008) [2023-12-27 10:05:19,849][20347] Updated weights for policy 0, policy_version 644309 (0.0008) [2023-12-27 10:05:19,899][20381] Updated weights for policy 1, policy_version 643774 (0.0006) [2023-12-27 10:05:19,900][20347] Updated weights for policy 0, policy_version 644319 (0.0008) [2023-12-27 10:05:19,961][20381] Updated weights for policy 1, policy_version 643784 (0.0008) [2023-12-27 10:05:20,016][20381] Updated weights for policy 1, policy_version 643794 (0.0009) [2023-12-27 10:05:20,534][20347] Updated weights for policy 0, policy_version 644329 (0.0006) [2023-12-27 10:05:20,600][20347] Updated weights for policy 0, policy_version 644339 (0.0008) [2023-12-27 10:05:20,656][20347] Updated weights for policy 0, policy_version 644349 (0.0006) [2023-12-27 10:05:20,742][20381] Updated weights for policy 1, policy_version 643804 (0.0008) [2023-12-27 10:05:20,804][20381] Updated weights for policy 1, policy_version 643814 (0.0009) [2023-12-27 10:05:20,869][20381] Updated weights for policy 1, policy_version 643824 (0.0008) [2023-12-27 10:05:21,279][20347] Updated weights for policy 0, policy_version 644359 (0.0008) [2023-12-27 10:05:21,345][20347] Updated weights for policy 0, policy_version 644369 (0.0006) [2023-12-27 10:05:21,406][20347] Updated weights for policy 0, policy_version 644379 (0.0008) [2023-12-27 10:05:21,638][20381] Updated weights for policy 1, policy_version 643834 (0.0008) [2023-12-27 10:05:21,702][20381] Updated weights for policy 1, policy_version 643844 (0.0008) [2023-12-27 10:05:21,771][20381] Updated weights for policy 1, policy_version 643854 (0.0008) [2023-12-27 10:05:21,838][20381] Updated weights for policy 1, policy_version 643864 (0.0008) [2023-12-27 10:05:22,107][20347] Updated weights for policy 0, policy_version 644389 (0.0007) [2023-12-27 10:05:22,169][20347] Updated weights for policy 0, policy_version 644399 (0.0006) [2023-12-27 10:05:22,233][20347] Updated weights for policy 0, policy_version 644409 (0.0006) [2023-12-27 10:05:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 329842688. Throughput: 0: 10225.6, 1: 10395.7. Samples: 329810716. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:22,363][19170] Avg episode reward: [(0, '961.870'), (1, '867.720')] [2023-12-27 10:05:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000643864_164855808.pth... [2023-12-27 10:05:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000644416_164986880.pth... [2023-12-27 10:05:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000642680_164552704.pth [2023-12-27 10:05:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000643200_164675584.pth [2023-12-27 10:05:22,517][20381] Updated weights for policy 1, policy_version 643874 (0.0008) [2023-12-27 10:05:22,582][20381] Updated weights for policy 1, policy_version 643884 (0.0008) [2023-12-27 10:05:22,642][20381] Updated weights for policy 1, policy_version 643894 (0.0007) [2023-12-27 10:05:22,942][20347] Updated weights for policy 0, policy_version 644419 (0.0008) [2023-12-27 10:05:22,991][20347] Updated weights for policy 0, policy_version 644429 (0.0008) [2023-12-27 10:05:23,037][20347] Updated weights for policy 0, policy_version 644439 (0.0007) [2023-12-27 10:05:23,333][20381] Updated weights for policy 1, policy_version 643904 (0.0008) [2023-12-27 10:05:23,383][20381] Updated weights for policy 1, policy_version 643914 (0.0008) [2023-12-27 10:05:23,438][20381] Updated weights for policy 1, policy_version 643924 (0.0007) [2023-12-27 10:05:23,751][20347] Updated weights for policy 0, policy_version 644449 (0.0008) [2023-12-27 10:05:23,808][20347] Updated weights for policy 0, policy_version 644459 (0.0007) [2023-12-27 10:05:23,869][20347] Updated weights for policy 0, policy_version 644469 (0.0008) [2023-12-27 10:05:23,950][20347] Updated weights for policy 0, policy_version 644479 (0.0007) [2023-12-27 10:05:24,108][20381] Updated weights for policy 1, policy_version 643934 (0.0009) [2023-12-27 10:05:24,154][20381] Updated weights for policy 1, policy_version 643944 (0.0011) [2023-12-27 10:05:24,200][20381] Updated weights for policy 1, policy_version 643954 (0.0011) [2023-12-27 10:05:24,624][20347] Updated weights for policy 0, policy_version 644489 (0.0007) [2023-12-27 10:05:24,673][20347] Updated weights for policy 0, policy_version 644499 (0.0007) [2023-12-27 10:05:24,720][20347] Updated weights for policy 0, policy_version 644509 (0.0007) [2023-12-27 10:05:24,813][20381] Updated weights for policy 1, policy_version 643964 (0.0008) [2023-12-27 10:05:24,862][20381] Updated weights for policy 1, policy_version 643974 (0.0005) [2023-12-27 10:05:24,906][20381] Updated weights for policy 1, policy_version 643984 (0.0005) [2023-12-27 10:05:25,369][20347] Updated weights for policy 0, policy_version 644519 (0.0007) [2023-12-27 10:05:25,412][20347] Updated weights for policy 0, policy_version 644529 (0.0005) [2023-12-27 10:05:25,458][20347] Updated weights for policy 0, policy_version 644539 (0.0005) [2023-12-27 10:05:25,525][20381] Updated weights for policy 1, policy_version 643994 (0.0006) [2023-12-27 10:05:25,580][20381] Updated weights for policy 1, policy_version 644004 (0.0008) [2023-12-27 10:05:25,634][20381] Updated weights for policy 1, policy_version 644014 (0.0006) [2023-12-27 10:05:25,690][20381] Updated weights for policy 1, policy_version 644024 (0.0007) [2023-12-27 10:05:26,063][20347] Updated weights for policy 0, policy_version 644549 (0.0006) [2023-12-27 10:05:26,116][20347] Updated weights for policy 0, policy_version 644559 (0.0007) [2023-12-27 10:05:26,169][20347] Updated weights for policy 0, policy_version 644569 (0.0007) [2023-12-27 10:05:26,332][20381] Updated weights for policy 1, policy_version 644034 (0.0010) [2023-12-27 10:05:26,389][20381] Updated weights for policy 1, policy_version 644044 (0.0010) [2023-12-27 10:05:26,452][20381] Updated weights for policy 1, policy_version 644054 (0.0010) [2023-12-27 10:05:26,808][20347] Updated weights for policy 0, policy_version 644579 (0.0007) [2023-12-27 10:05:26,857][20347] Updated weights for policy 0, policy_version 644589 (0.0008) [2023-12-27 10:05:26,911][20347] Updated weights for policy 0, policy_version 644599 (0.0006) [2023-12-27 10:05:27,063][20381] Updated weights for policy 1, policy_version 644064 (0.0010) [2023-12-27 10:05:27,108][20381] Updated weights for policy 1, policy_version 644074 (0.0007) [2023-12-27 10:05:27,165][20381] Updated weights for policy 1, policy_version 644084 (0.0008) [2023-12-27 10:05:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 329949184. Throughput: 0: 10315.4, 1: 10368.5. Samples: 329935088. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:27,364][19170] Avg episode reward: [(0, '922.820'), (1, '760.190')] [2023-12-27 10:05:27,465][20347] Updated weights for policy 0, policy_version 644609 (0.0008) [2023-12-27 10:05:27,522][20347] Updated weights for policy 0, policy_version 644619 (0.0005) [2023-12-27 10:05:27,572][20347] Updated weights for policy 0, policy_version 644629 (0.0006) [2023-12-27 10:05:27,634][20347] Updated weights for policy 0, policy_version 644639 (0.0005) [2023-12-27 10:05:27,918][20381] Updated weights for policy 1, policy_version 644094 (0.0008) [2023-12-27 10:05:27,971][20381] Updated weights for policy 1, policy_version 644104 (0.0007) [2023-12-27 10:05:28,022][20381] Updated weights for policy 1, policy_version 644114 (0.0007) [2023-12-27 10:05:28,267][20347] Updated weights for policy 0, policy_version 644649 (0.0006) [2023-12-27 10:05:28,313][20347] Updated weights for policy 0, policy_version 644659 (0.0005) [2023-12-27 10:05:28,367][20347] Updated weights for policy 0, policy_version 644669 (0.0008) [2023-12-27 10:05:28,715][20381] Updated weights for policy 1, policy_version 644124 (0.0008) [2023-12-27 10:05:28,773][20381] Updated weights for policy 1, policy_version 644134 (0.0008) [2023-12-27 10:05:28,829][20381] Updated weights for policy 1, policy_version 644144 (0.0008) [2023-12-27 10:05:29,004][20347] Updated weights for policy 0, policy_version 644679 (0.0007) [2023-12-27 10:05:29,057][20347] Updated weights for policy 0, policy_version 644689 (0.0007) [2023-12-27 10:05:29,113][20347] Updated weights for policy 0, policy_version 644699 (0.0008) [2023-12-27 10:05:29,468][20381] Updated weights for policy 1, policy_version 644154 (0.0006) [2023-12-27 10:05:29,527][20381] Updated weights for policy 1, policy_version 644164 (0.0009) [2023-12-27 10:05:29,581][20381] Updated weights for policy 1, policy_version 644174 (0.0006) [2023-12-27 10:05:29,642][20381] Updated weights for policy 1, policy_version 644184 (0.0005) [2023-12-27 10:05:29,777][20347] Updated weights for policy 0, policy_version 644709 (0.0007) [2023-12-27 10:05:29,833][20347] Updated weights for policy 0, policy_version 644719 (0.0007) [2023-12-27 10:05:29,891][20347] Updated weights for policy 0, policy_version 644729 (0.0008) [2023-12-27 10:05:30,257][20381] Updated weights for policy 1, policy_version 644194 (0.0006) [2023-12-27 10:05:30,305][20381] Updated weights for policy 1, policy_version 644204 (0.0006) [2023-12-27 10:05:30,356][20381] Updated weights for policy 1, policy_version 644214 (0.0007) [2023-12-27 10:05:30,576][20347] Updated weights for policy 0, policy_version 644739 (0.0009) [2023-12-27 10:05:30,633][20347] Updated weights for policy 0, policy_version 644749 (0.0010) [2023-12-27 10:05:30,696][20347] Updated weights for policy 0, policy_version 644759 (0.0010) [2023-12-27 10:05:31,072][20381] Updated weights for policy 1, policy_version 644224 (0.0008) [2023-12-27 10:05:31,135][20381] Updated weights for policy 1, policy_version 644234 (0.0007) [2023-12-27 10:05:31,192][20381] Updated weights for policy 1, policy_version 644244 (0.0008) [2023-12-27 10:05:31,269][20347] Updated weights for policy 0, policy_version 644769 (0.0008) [2023-12-27 10:05:31,320][20347] Updated weights for policy 0, policy_version 644779 (0.0005) [2023-12-27 10:05:31,387][20347] Updated weights for policy 0, policy_version 644789 (0.0010) [2023-12-27 10:05:31,455][20347] Updated weights for policy 0, policy_version 644799 (0.0010) [2023-12-27 10:05:31,886][20381] Updated weights for policy 1, policy_version 644254 (0.0007) [2023-12-27 10:05:31,943][20381] Updated weights for policy 1, policy_version 644264 (0.0007) [2023-12-27 10:05:32,003][20381] Updated weights for policy 1, policy_version 644274 (0.0007) [2023-12-27 10:05:32,098][20347] Updated weights for policy 0, policy_version 644809 (0.0010) [2023-12-27 10:05:32,159][20347] Updated weights for policy 0, policy_version 644819 (0.0010) [2023-12-27 10:05:32,208][20347] Updated weights for policy 0, policy_version 644829 (0.0010) [2023-12-27 10:05:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 330055680. Throughput: 0: 10411.3, 1: 10323.7. Samples: 330061380. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:32,363][19170] Avg episode reward: [(0, '904.480'), (1, '730.110')] [2023-12-27 10:05:32,654][20381] Updated weights for policy 1, policy_version 644284 (0.0007) [2023-12-27 10:05:32,708][20381] Updated weights for policy 1, policy_version 644294 (0.0007) [2023-12-27 10:05:32,768][20381] Updated weights for policy 1, policy_version 644304 (0.0006) [2023-12-27 10:05:32,874][20347] Updated weights for policy 0, policy_version 644839 (0.0008) [2023-12-27 10:05:32,933][20347] Updated weights for policy 0, policy_version 644849 (0.0008) [2023-12-27 10:05:32,979][20347] Updated weights for policy 0, policy_version 644859 (0.0007) [2023-12-27 10:05:33,362][20381] Updated weights for policy 1, policy_version 644314 (0.0006) [2023-12-27 10:05:33,411][20381] Updated weights for policy 1, policy_version 644324 (0.0007) [2023-12-27 10:05:33,459][20381] Updated weights for policy 1, policy_version 644334 (0.0010) [2023-12-27 10:05:33,510][20381] Updated weights for policy 1, policy_version 644344 (0.0010) [2023-12-27 10:05:33,588][20347] Updated weights for policy 0, policy_version 644869 (0.0008) [2023-12-27 10:05:33,641][20347] Updated weights for policy 0, policy_version 644879 (0.0007) [2023-12-27 10:05:33,696][20347] Updated weights for policy 0, policy_version 644889 (0.0007) [2023-12-27 10:05:34,179][20381] Updated weights for policy 1, policy_version 644354 (0.0009) [2023-12-27 10:05:34,230][20381] Updated weights for policy 1, policy_version 644364 (0.0009) [2023-12-27 10:05:34,290][20381] Updated weights for policy 1, policy_version 644374 (0.0010) [2023-12-27 10:05:34,336][20347] Updated weights for policy 0, policy_version 644899 (0.0007) [2023-12-27 10:05:34,388][20347] Updated weights for policy 0, policy_version 644909 (0.0007) [2023-12-27 10:05:34,450][20347] Updated weights for policy 0, policy_version 644919 (0.0007) [2023-12-27 10:05:34,908][20381] Updated weights for policy 1, policy_version 644384 (0.0008) [2023-12-27 10:05:34,956][20381] Updated weights for policy 1, policy_version 644394 (0.0007) [2023-12-27 10:05:35,004][20381] Updated weights for policy 1, policy_version 644404 (0.0007) [2023-12-27 10:05:35,065][20347] Updated weights for policy 0, policy_version 644929 (0.0006) [2023-12-27 10:05:35,122][20347] Updated weights for policy 0, policy_version 644939 (0.0008) [2023-12-27 10:05:35,172][20347] Updated weights for policy 0, policy_version 644949 (0.0010) [2023-12-27 10:05:35,220][20347] Updated weights for policy 0, policy_version 644959 (0.0010) [2023-12-27 10:05:35,735][20381] Updated weights for policy 1, policy_version 644414 (0.0007) [2023-12-27 10:05:35,786][20381] Updated weights for policy 1, policy_version 644424 (0.0008) [2023-12-27 10:05:35,843][20381] Updated weights for policy 1, policy_version 644434 (0.0011) [2023-12-27 10:05:35,854][20347] Updated weights for policy 0, policy_version 644969 (0.0009) [2023-12-27 10:05:35,912][20347] Updated weights for policy 0, policy_version 644979 (0.0007) [2023-12-27 10:05:35,972][20347] Updated weights for policy 0, policy_version 644989 (0.0005) [2023-12-27 10:05:36,520][20381] Updated weights for policy 1, policy_version 644444 (0.0010) [2023-12-27 10:05:36,576][20381] Updated weights for policy 1, policy_version 644454 (0.0010) [2023-12-27 10:05:36,621][20347] Updated weights for policy 0, policy_version 644999 (0.0008) [2023-12-27 10:05:36,635][20381] Updated weights for policy 1, policy_version 644464 (0.0010) [2023-12-27 10:05:36,679][20347] Updated weights for policy 0, policy_version 645009 (0.0007) [2023-12-27 10:05:36,736][20347] Updated weights for policy 0, policy_version 645019 (0.0006) [2023-12-27 10:05:37,320][20381] Updated weights for policy 1, policy_version 644474 (0.0011) [2023-12-27 10:05:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20752.9, 300 sec: 20743.8). Total num frames: 330153984. Throughput: 0: 10468.2, 1: 10361.9. Samples: 330127000. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:37,364][19170] Avg episode reward: [(0, '904.340'), (1, '814.700')] [2023-12-27 10:05:37,375][20381] Updated weights for policy 1, policy_version 644484 (0.0010) [2023-12-27 10:05:37,393][20347] Updated weights for policy 0, policy_version 645029 (0.0007) [2023-12-27 10:05:37,430][20381] Updated weights for policy 1, policy_version 644494 (0.0010) [2023-12-27 10:05:37,450][20347] Updated weights for policy 0, policy_version 645039 (0.0006) [2023-12-27 10:05:37,487][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000644504_165019648.pth... [2023-12-27 10:05:37,489][20381] Updated weights for policy 1, policy_version 644504 (0.0010) [2023-12-27 10:05:37,490][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000643288_164708352.pth [2023-12-27 10:05:37,502][20347] Updated weights for policy 0, policy_version 645049 (0.0006) [2023-12-27 10:05:37,533][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000645056_165150720.pth... [2023-12-27 10:05:37,536][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000643808_164831232.pth [2023-12-27 10:05:38,134][20381] Updated weights for policy 1, policy_version 644514 (0.0008) [2023-12-27 10:05:38,187][20381] Updated weights for policy 1, policy_version 644524 (0.0007) [2023-12-27 10:05:38,225][20347] Updated weights for policy 0, policy_version 645059 (0.0006) [2023-12-27 10:05:38,246][20381] Updated weights for policy 1, policy_version 644534 (0.0006) [2023-12-27 10:05:38,285][20347] Updated weights for policy 0, policy_version 645069 (0.0007) [2023-12-27 10:05:38,344][20347] Updated weights for policy 0, policy_version 645079 (0.0007) [2023-12-27 10:05:38,900][20381] Updated weights for policy 1, policy_version 644544 (0.0007) [2023-12-27 10:05:38,947][20381] Updated weights for policy 1, policy_version 644554 (0.0007) [2023-12-27 10:05:38,991][20381] Updated weights for policy 1, policy_version 644564 (0.0007) [2023-12-27 10:05:39,035][20347] Updated weights for policy 0, policy_version 645089 (0.0009) [2023-12-27 10:05:39,086][20347] Updated weights for policy 0, policy_version 645099 (0.0008) [2023-12-27 10:05:39,144][20347] Updated weights for policy 0, policy_version 645109 (0.0008) [2023-12-27 10:05:39,196][20347] Updated weights for policy 0, policy_version 645119 (0.0008) [2023-12-27 10:05:39,739][20381] Updated weights for policy 1, policy_version 644574 (0.0009) [2023-12-27 10:05:39,788][20381] Updated weights for policy 1, policy_version 644584 (0.0010) [2023-12-27 10:05:39,851][20381] Updated weights for policy 1, policy_version 644594 (0.0010) [2023-12-27 10:05:39,868][20347] Updated weights for policy 0, policy_version 645129 (0.0008) [2023-12-27 10:05:39,929][20347] Updated weights for policy 0, policy_version 645139 (0.0008) [2023-12-27 10:05:39,982][20347] Updated weights for policy 0, policy_version 645149 (0.0008) [2023-12-27 10:05:40,564][20381] Updated weights for policy 1, policy_version 644604 (0.0009) [2023-12-27 10:05:40,614][20381] Updated weights for policy 1, policy_version 644614 (0.0010) [2023-12-27 10:05:40,662][20381] Updated weights for policy 1, policy_version 644624 (0.0010) [2023-12-27 10:05:40,727][20347] Updated weights for policy 0, policy_version 645159 (0.0007) [2023-12-27 10:05:40,781][20347] Updated weights for policy 0, policy_version 645169 (0.0007) [2023-12-27 10:05:40,836][20347] Updated weights for policy 0, policy_version 645179 (0.0006) [2023-12-27 10:05:41,371][20381] Updated weights for policy 1, policy_version 644634 (0.0010) [2023-12-27 10:05:41,434][20381] Updated weights for policy 1, policy_version 644644 (0.0010) [2023-12-27 10:05:41,493][20381] Updated weights for policy 1, policy_version 644654 (0.0008) [2023-12-27 10:05:41,552][20381] Updated weights for policy 1, policy_version 644664 (0.0007) [2023-12-27 10:05:41,601][20347] Updated weights for policy 0, policy_version 645189 (0.0006) [2023-12-27 10:05:41,666][20347] Updated weights for policy 0, policy_version 645199 (0.0008) [2023-12-27 10:05:41,736][20347] Updated weights for policy 0, policy_version 645209 (0.0006) [2023-12-27 10:05:42,249][20381] Updated weights for policy 1, policy_version 644674 (0.0006) [2023-12-27 10:05:42,317][20381] Updated weights for policy 1, policy_version 644684 (0.0009) [2023-12-27 10:05:42,349][20347] Updated weights for policy 0, policy_version 645219 (0.0007) [2023-12-27 10:05:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 330252288. Throughput: 0: 10382.9, 1: 10325.3. Samples: 330248884. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:42,363][19170] Avg episode reward: [(0, '894.130'), (1, '807.030')] [2023-12-27 10:05:42,378][20381] Updated weights for policy 1, policy_version 644694 (0.0009) [2023-12-27 10:05:42,415][20347] Updated weights for policy 0, policy_version 645229 (0.0007) [2023-12-27 10:05:42,476][20347] Updated weights for policy 0, policy_version 645239 (0.0006) [2023-12-27 10:05:42,987][20381] Updated weights for policy 1, policy_version 644704 (0.0009) [2023-12-27 10:05:43,042][20381] Updated weights for policy 1, policy_version 644714 (0.0009) [2023-12-27 10:05:43,043][20347] Updated weights for policy 0, policy_version 645249 (0.0007) [2023-12-27 10:05:43,100][20381] Updated weights for policy 1, policy_version 644724 (0.0007) [2023-12-27 10:05:43,106][20347] Updated weights for policy 0, policy_version 645259 (0.0007) [2023-12-27 10:05:43,160][20347] Updated weights for policy 0, policy_version 645269 (0.0007) [2023-12-27 10:05:43,217][20347] Updated weights for policy 0, policy_version 645279 (0.0007) [2023-12-27 10:05:43,831][20381] Updated weights for policy 1, policy_version 644734 (0.0007) [2023-12-27 10:05:43,847][20347] Updated weights for policy 0, policy_version 645289 (0.0005) [2023-12-27 10:05:43,884][20381] Updated weights for policy 1, policy_version 644744 (0.0008) [2023-12-27 10:05:43,907][20347] Updated weights for policy 0, policy_version 645299 (0.0006) [2023-12-27 10:05:43,937][20381] Updated weights for policy 1, policy_version 644754 (0.0007) [2023-12-27 10:05:43,962][20347] Updated weights for policy 0, policy_version 645309 (0.0005) [2023-12-27 10:05:44,469][20347] Updated weights for policy 0, policy_version 645319 (0.0005) [2023-12-27 10:05:44,525][20347] Updated weights for policy 0, policy_version 645329 (0.0007) [2023-12-27 10:05:44,577][20347] Updated weights for policy 0, policy_version 645339 (0.0008) [2023-12-27 10:05:44,655][20381] Updated weights for policy 1, policy_version 644764 (0.0008) [2023-12-27 10:05:44,710][20381] Updated weights for policy 1, policy_version 644774 (0.0008) [2023-12-27 10:05:44,778][20381] Updated weights for policy 1, policy_version 644784 (0.0007) [2023-12-27 10:05:45,267][20347] Updated weights for policy 0, policy_version 645349 (0.0009) [2023-12-27 10:05:45,323][20347] Updated weights for policy 0, policy_version 645359 (0.0007) [2023-12-27 10:05:45,369][20347] Updated weights for policy 0, policy_version 645369 (0.0005) [2023-12-27 10:05:45,496][20381] Updated weights for policy 1, policy_version 644794 (0.0007) [2023-12-27 10:05:45,549][20381] Updated weights for policy 1, policy_version 644804 (0.0008) [2023-12-27 10:05:45,595][20381] Updated weights for policy 1, policy_version 644814 (0.0007) [2023-12-27 10:05:45,644][20381] Updated weights for policy 1, policy_version 644824 (0.0008) [2023-12-27 10:05:46,007][20347] Updated weights for policy 0, policy_version 645379 (0.0006) [2023-12-27 10:05:46,057][20347] Updated weights for policy 0, policy_version 645389 (0.0008) [2023-12-27 10:05:46,106][20347] Updated weights for policy 0, policy_version 645399 (0.0008) [2023-12-27 10:05:46,379][20381] Updated weights for policy 1, policy_version 644834 (0.0007) [2023-12-27 10:05:46,441][20381] Updated weights for policy 1, policy_version 644844 (0.0008) [2023-12-27 10:05:46,493][20381] Updated weights for policy 1, policy_version 644854 (0.0008) [2023-12-27 10:05:46,807][20347] Updated weights for policy 0, policy_version 645409 (0.0008) [2023-12-27 10:05:46,859][20347] Updated weights for policy 0, policy_version 645419 (0.0008) [2023-12-27 10:05:46,914][20347] Updated weights for policy 0, policy_version 645429 (0.0007) [2023-12-27 10:05:46,964][20347] Updated weights for policy 0, policy_version 645439 (0.0005) [2023-12-27 10:05:47,192][20381] Updated weights for policy 1, policy_version 644864 (0.0008) [2023-12-27 10:05:47,248][20381] Updated weights for policy 1, policy_version 644874 (0.0007) [2023-12-27 10:05:47,306][20381] Updated weights for policy 1, policy_version 644884 (0.0009) [2023-12-27 10:05:47,362][19170] Fps is (10 sec: 21300.5, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 330366976. Throughput: 0: 10449.7, 1: 10287.7. Samples: 330372400. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:47,363][19170] Avg episode reward: [(0, '874.270'), (1, '830.540')] [2023-12-27 10:05:47,620][20347] Updated weights for policy 0, policy_version 645449 (0.0006) [2023-12-27 10:05:47,678][20347] Updated weights for policy 0, policy_version 645459 (0.0005) [2023-12-27 10:05:47,732][20347] Updated weights for policy 0, policy_version 645469 (0.0005) [2023-12-27 10:05:48,073][20381] Updated weights for policy 1, policy_version 644894 (0.0008) [2023-12-27 10:05:48,140][20381] Updated weights for policy 1, policy_version 644904 (0.0008) [2023-12-27 10:05:48,193][20381] Updated weights for policy 1, policy_version 644914 (0.0007) [2023-12-27 10:05:48,282][20347] Updated weights for policy 0, policy_version 645479 (0.0006) [2023-12-27 10:05:48,343][20347] Updated weights for policy 0, policy_version 645489 (0.0007) [2023-12-27 10:05:48,402][20347] Updated weights for policy 0, policy_version 645499 (0.0006) [2023-12-27 10:05:48,919][20381] Updated weights for policy 1, policy_version 644924 (0.0007) [2023-12-27 10:05:48,968][20381] Updated weights for policy 1, policy_version 644934 (0.0008) [2023-12-27 10:05:49,011][20347] Updated weights for policy 0, policy_version 645509 (0.0008) [2023-12-27 10:05:49,028][20381] Updated weights for policy 1, policy_version 644944 (0.0006) [2023-12-27 10:05:49,064][20347] Updated weights for policy 0, policy_version 645519 (0.0010) [2023-12-27 10:05:49,116][20347] Updated weights for policy 0, policy_version 645529 (0.0010) [2023-12-27 10:05:49,690][20381] Updated weights for policy 1, policy_version 644954 (0.0006) [2023-12-27 10:05:49,736][20381] Updated weights for policy 1, policy_version 644964 (0.0007) [2023-12-27 10:05:49,783][20381] Updated weights for policy 1, policy_version 644974 (0.0008) [2023-12-27 10:05:49,839][20381] Updated weights for policy 1, policy_version 644984 (0.0007) [2023-12-27 10:05:49,841][20347] Updated weights for policy 0, policy_version 645539 (0.0009) [2023-12-27 10:05:49,903][20347] Updated weights for policy 0, policy_version 645549 (0.0007) [2023-12-27 10:05:49,966][20347] Updated weights for policy 0, policy_version 645559 (0.0007) [2023-12-27 10:05:50,537][20381] Updated weights for policy 1, policy_version 644994 (0.0005) [2023-12-27 10:05:50,600][20381] Updated weights for policy 1, policy_version 645004 (0.0007) [2023-12-27 10:05:50,647][20347] Updated weights for policy 0, policy_version 645569 (0.0006) [2023-12-27 10:05:50,662][20381] Updated weights for policy 1, policy_version 645014 (0.0007) [2023-12-27 10:05:50,704][20347] Updated weights for policy 0, policy_version 645579 (0.0008) [2023-12-27 10:05:50,762][20347] Updated weights for policy 0, policy_version 645589 (0.0008) [2023-12-27 10:05:50,812][20347] Updated weights for policy 0, policy_version 645599 (0.0008) [2023-12-27 10:05:51,286][20381] Updated weights for policy 1, policy_version 645024 (0.0008) [2023-12-27 10:05:51,341][20381] Updated weights for policy 1, policy_version 645034 (0.0007) [2023-12-27 10:05:51,406][20381] Updated weights for policy 1, policy_version 645044 (0.0008) [2023-12-27 10:05:51,495][20347] Updated weights for policy 0, policy_version 645609 (0.0008) [2023-12-27 10:05:51,557][20347] Updated weights for policy 0, policy_version 645619 (0.0007) [2023-12-27 10:05:51,614][20347] Updated weights for policy 0, policy_version 645629 (0.0008) [2023-12-27 10:05:52,076][20381] Updated weights for policy 1, policy_version 645054 (0.0005) [2023-12-27 10:05:52,135][20381] Updated weights for policy 1, policy_version 645064 (0.0005) [2023-12-27 10:05:52,188][20381] Updated weights for policy 1, policy_version 645074 (0.0005) [2023-12-27 10:05:52,362][20347] Updated weights for policy 0, policy_version 645639 (0.0007) [2023-12-27 10:05:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 330465280. Throughput: 0: 10492.1, 1: 10196.6. Samples: 330434432. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:52,363][19170] Avg episode reward: [(0, '902.760'), (1, '839.150')] [2023-12-27 10:05:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000645080_165167104.pth... [2023-12-27 10:05:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000643864_164855808.pth [2023-12-27 10:05:52,422][20347] Updated weights for policy 0, policy_version 645649 (0.0009) [2023-12-27 10:05:52,479][20347] Updated weights for policy 0, policy_version 645659 (0.0008) [2023-12-27 10:05:52,508][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000645664_165306368.pth... [2023-12-27 10:05:52,512][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000644416_164986880.pth [2023-12-27 10:05:52,819][20381] Updated weights for policy 1, policy_version 645084 (0.0007) [2023-12-27 10:05:52,877][20381] Updated weights for policy 1, policy_version 645094 (0.0008) [2023-12-27 10:05:52,930][20381] Updated weights for policy 1, policy_version 645104 (0.0008) [2023-12-27 10:05:53,178][20347] Updated weights for policy 0, policy_version 645669 (0.0008) [2023-12-27 10:05:53,193][20291] KL-divergence is very high: 113.9917 [2023-12-27 10:05:53,200][20291] KL-divergence is very high: 101.5507 [2023-12-27 10:05:53,243][20347] Updated weights for policy 0, policy_version 645679 (0.0007) [2023-12-27 10:05:53,306][20347] Updated weights for policy 0, policy_version 645689 (0.0008) [2023-12-27 10:05:53,668][20381] Updated weights for policy 1, policy_version 645114 (0.0007) [2023-12-27 10:05:53,730][20381] Updated weights for policy 1, policy_version 645124 (0.0009) [2023-12-27 10:05:53,785][20381] Updated weights for policy 1, policy_version 645134 (0.0007) [2023-12-27 10:05:53,840][20381] Updated weights for policy 1, policy_version 645144 (0.0008) [2023-12-27 10:05:53,924][20347] Updated weights for policy 0, policy_version 645699 (0.0008) [2023-12-27 10:05:53,984][20347] Updated weights for policy 0, policy_version 645709 (0.0007) [2023-12-27 10:05:54,035][20347] Updated weights for policy 0, policy_version 645719 (0.0007) [2023-12-27 10:05:54,520][20381] Updated weights for policy 1, policy_version 645154 (0.0008) [2023-12-27 10:05:54,572][20381] Updated weights for policy 1, policy_version 645164 (0.0009) [2023-12-27 10:05:54,628][20381] Updated weights for policy 1, policy_version 645174 (0.0008) [2023-12-27 10:05:54,702][20347] Updated weights for policy 0, policy_version 645729 (0.0008) [2023-12-27 10:05:54,755][20347] Updated weights for policy 0, policy_version 645739 (0.0005) [2023-12-27 10:05:54,814][20347] Updated weights for policy 0, policy_version 645749 (0.0007) [2023-12-27 10:05:54,862][20347] Updated weights for policy 0, policy_version 645759 (0.0009) [2023-12-27 10:05:55,334][20381] Updated weights for policy 1, policy_version 645184 (0.0008) [2023-12-27 10:05:55,384][20381] Updated weights for policy 1, policy_version 645194 (0.0008) [2023-12-27 10:05:55,426][20381] Updated weights for policy 1, policy_version 645204 (0.0006) [2023-12-27 10:05:55,536][20347] Updated weights for policy 0, policy_version 645769 (0.0006) [2023-12-27 10:05:55,591][20347] Updated weights for policy 0, policy_version 645779 (0.0006) [2023-12-27 10:05:55,650][20347] Updated weights for policy 0, policy_version 645789 (0.0008) [2023-12-27 10:05:56,144][20381] Updated weights for policy 1, policy_version 645214 (0.0007) [2023-12-27 10:05:56,200][20381] Updated weights for policy 1, policy_version 645224 (0.0007) [2023-12-27 10:05:56,255][20381] Updated weights for policy 1, policy_version 645234 (0.0007) [2023-12-27 10:05:56,292][20347] Updated weights for policy 0, policy_version 645799 (0.0006) [2023-12-27 10:05:56,333][20347] Updated weights for policy 0, policy_version 645809 (0.0007) [2023-12-27 10:05:56,383][20347] Updated weights for policy 0, policy_version 645819 (0.0007) [2023-12-27 10:05:56,941][20381] Updated weights for policy 1, policy_version 645244 (0.0007) [2023-12-27 10:05:56,999][20381] Updated weights for policy 1, policy_version 645254 (0.0006) [2023-12-27 10:05:57,061][20381] Updated weights for policy 1, policy_version 645264 (0.0005) [2023-12-27 10:05:57,077][20347] Updated weights for policy 0, policy_version 645829 (0.0007) [2023-12-27 10:05:57,123][20347] Updated weights for policy 0, policy_version 645839 (0.0007) [2023-12-27 10:05:57,174][20347] Updated weights for policy 0, policy_version 645849 (0.0009) [2023-12-27 10:05:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 330571776. Throughput: 0: 10440.3, 1: 10264.6. Samples: 330556900. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:05:57,363][19170] Avg episode reward: [(0, '825.900'), (1, '852.930')] [2023-12-27 10:05:57,642][20381] Updated weights for policy 1, policy_version 645274 (0.0006) [2023-12-27 10:05:57,696][20381] Updated weights for policy 1, policy_version 645284 (0.0007) [2023-12-27 10:05:57,755][20381] Updated weights for policy 1, policy_version 645294 (0.0007) [2023-12-27 10:05:57,813][20381] Updated weights for policy 1, policy_version 645304 (0.0007) [2023-12-27 10:05:57,891][20347] Updated weights for policy 0, policy_version 645859 (0.0008) [2023-12-27 10:05:57,948][20347] Updated weights for policy 0, policy_version 645869 (0.0008) [2023-12-27 10:05:57,996][20347] Updated weights for policy 0, policy_version 645879 (0.0008) [2023-12-27 10:05:58,546][20381] Updated weights for policy 1, policy_version 645314 (0.0008) [2023-12-27 10:05:58,606][20381] Updated weights for policy 1, policy_version 645324 (0.0006) [2023-12-27 10:05:58,665][20381] Updated weights for policy 1, policy_version 645334 (0.0008) [2023-12-27 10:05:58,751][20347] Updated weights for policy 0, policy_version 645889 (0.0008) [2023-12-27 10:05:58,822][20347] Updated weights for policy 0, policy_version 645899 (0.0008) [2023-12-27 10:05:58,882][20347] Updated weights for policy 0, policy_version 645909 (0.0008) [2023-12-27 10:05:58,943][20347] Updated weights for policy 0, policy_version 645919 (0.0008) [2023-12-27 10:05:59,398][20381] Updated weights for policy 1, policy_version 645344 (0.0009) [2023-12-27 10:05:59,453][20381] Updated weights for policy 1, policy_version 645354 (0.0005) [2023-12-27 10:05:59,509][20381] Updated weights for policy 1, policy_version 645364 (0.0005) [2023-12-27 10:05:59,638][20347] Updated weights for policy 0, policy_version 645929 (0.0010) [2023-12-27 10:05:59,691][20347] Updated weights for policy 0, policy_version 645939 (0.0010) [2023-12-27 10:05:59,748][20347] Updated weights for policy 0, policy_version 645949 (0.0010) [2023-12-27 10:06:00,169][20381] Updated weights for policy 1, policy_version 645374 (0.0006) [2023-12-27 10:06:00,231][20381] Updated weights for policy 1, policy_version 645384 (0.0007) [2023-12-27 10:06:00,297][20381] Updated weights for policy 1, policy_version 645394 (0.0008) [2023-12-27 10:06:00,423][20347] Updated weights for policy 0, policy_version 645959 (0.0008) [2023-12-27 10:06:00,467][20347] Updated weights for policy 0, policy_version 645969 (0.0007) [2023-12-27 10:06:00,523][20347] Updated weights for policy 0, policy_version 645979 (0.0007) [2023-12-27 10:06:00,849][20381] Updated weights for policy 1, policy_version 645404 (0.0008) [2023-12-27 10:06:00,897][20381] Updated weights for policy 1, policy_version 645414 (0.0009) [2023-12-27 10:06:00,951][20381] Updated weights for policy 1, policy_version 645424 (0.0009) [2023-12-27 10:06:01,148][20347] Updated weights for policy 0, policy_version 645989 (0.0009) [2023-12-27 10:06:01,213][20347] Updated weights for policy 0, policy_version 645999 (0.0010) [2023-12-27 10:06:01,266][20347] Updated weights for policy 0, policy_version 646009 (0.0010) [2023-12-27 10:06:01,617][20381] Updated weights for policy 1, policy_version 645434 (0.0008) [2023-12-27 10:06:01,681][20381] Updated weights for policy 1, policy_version 645444 (0.0007) [2023-12-27 10:06:01,746][20381] Updated weights for policy 1, policy_version 645454 (0.0008) [2023-12-27 10:06:01,804][20381] Updated weights for policy 1, policy_version 645464 (0.0008) [2023-12-27 10:06:01,986][20347] Updated weights for policy 0, policy_version 646019 (0.0009) [2023-12-27 10:06:02,046][20347] Updated weights for policy 0, policy_version 646029 (0.0008) [2023-12-27 10:06:02,106][20347] Updated weights for policy 0, policy_version 646039 (0.0008) [2023-12-27 10:06:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 330670080. Throughput: 0: 10482.7, 1: 10246.5. Samples: 330680032. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:02,363][19170] Avg episode reward: [(0, '817.200'), (1, '892.620')] [2023-12-27 10:06:02,458][20381] Updated weights for policy 1, policy_version 645474 (0.0008) [2023-12-27 10:06:02,511][20381] Updated weights for policy 1, policy_version 645484 (0.0008) [2023-12-27 10:06:02,560][20381] Updated weights for policy 1, policy_version 645494 (0.0007) [2023-12-27 10:06:02,813][20347] Updated weights for policy 0, policy_version 646049 (0.0008) [2023-12-27 10:06:02,863][20347] Updated weights for policy 0, policy_version 646059 (0.0007) [2023-12-27 10:06:02,918][20347] Updated weights for policy 0, policy_version 646069 (0.0007) [2023-12-27 10:06:02,975][20347] Updated weights for policy 0, policy_version 646079 (0.0009) [2023-12-27 10:06:03,209][20381] Updated weights for policy 1, policy_version 645504 (0.0008) [2023-12-27 10:06:03,262][20381] Updated weights for policy 1, policy_version 645514 (0.0009) [2023-12-27 10:06:03,306][20381] Updated weights for policy 1, policy_version 645524 (0.0007) [2023-12-27 10:06:03,647][20347] Updated weights for policy 0, policy_version 646089 (0.0006) [2023-12-27 10:06:03,696][20347] Updated weights for policy 0, policy_version 646099 (0.0005) [2023-12-27 10:06:03,749][20347] Updated weights for policy 0, policy_version 646109 (0.0005) [2023-12-27 10:06:03,942][20381] Updated weights for policy 1, policy_version 645534 (0.0006) [2023-12-27 10:06:03,998][20381] Updated weights for policy 1, policy_version 645544 (0.0008) [2023-12-27 10:06:04,055][20381] Updated weights for policy 1, policy_version 645554 (0.0008) [2023-12-27 10:06:04,438][20347] Updated weights for policy 0, policy_version 646119 (0.0007) [2023-12-27 10:06:04,497][20347] Updated weights for policy 0, policy_version 646129 (0.0008) [2023-12-27 10:06:04,548][20347] Updated weights for policy 0, policy_version 646139 (0.0007) [2023-12-27 10:06:04,704][20381] Updated weights for policy 1, policy_version 645564 (0.0007) [2023-12-27 10:06:04,762][20381] Updated weights for policy 1, policy_version 645574 (0.0007) [2023-12-27 10:06:04,817][20381] Updated weights for policy 1, policy_version 645584 (0.0009) [2023-12-27 10:06:05,224][20347] Updated weights for policy 0, policy_version 646149 (0.0007) [2023-12-27 10:06:05,272][20347] Updated weights for policy 0, policy_version 646159 (0.0007) [2023-12-27 10:06:05,327][20347] Updated weights for policy 0, policy_version 646169 (0.0007) [2023-12-27 10:06:05,515][20381] Updated weights for policy 1, policy_version 645596 (0.0008) [2023-12-27 10:06:05,569][20381] Updated weights for policy 1, policy_version 645606 (0.0007) [2023-12-27 10:06:05,621][20381] Updated weights for policy 1, policy_version 645616 (0.0008) [2023-12-27 10:06:05,976][20347] Updated weights for policy 0, policy_version 646179 (0.0007) [2023-12-27 10:06:06,050][20347] Updated weights for policy 0, policy_version 646189 (0.0006) [2023-12-27 10:06:06,114][20347] Updated weights for policy 0, policy_version 646199 (0.0008) [2023-12-27 10:06:06,329][20381] Updated weights for policy 1, policy_version 645626 (0.0008) [2023-12-27 10:06:06,392][20381] Updated weights for policy 1, policy_version 645636 (0.0008) [2023-12-27 10:06:06,449][20381] Updated weights for policy 1, policy_version 645646 (0.0008) [2023-12-27 10:06:06,505][20381] Updated weights for policy 1, policy_version 645656 (0.0007) [2023-12-27 10:06:06,785][20347] Updated weights for policy 0, policy_version 646209 (0.0009) [2023-12-27 10:06:06,844][20347] Updated weights for policy 0, policy_version 646219 (0.0008) [2023-12-27 10:06:06,891][20347] Updated weights for policy 0, policy_version 646229 (0.0007) [2023-12-27 10:06:06,942][20347] Updated weights for policy 0, policy_version 646239 (0.0007) [2023-12-27 10:06:07,235][20381] Updated weights for policy 1, policy_version 645666 (0.0007) [2023-12-27 10:06:07,287][20381] Updated weights for policy 1, policy_version 645676 (0.0008) [2023-12-27 10:06:07,340][20381] Updated weights for policy 1, policy_version 645686 (0.0008) [2023-12-27 10:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 330776576. Throughput: 0: 10461.4, 1: 10246.9. Samples: 330742592. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:07,363][19170] Avg episode reward: [(0, '875.000'), (1, '892.520')] [2023-12-27 10:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000645688_165322752.pth... [2023-12-27 10:06:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000646240_165453824.pth... [2023-12-27 10:06:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000644504_165019648.pth [2023-12-27 10:06:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000645056_165150720.pth [2023-12-27 10:06:07,609][20347] Updated weights for policy 0, policy_version 646249 (0.0008) [2023-12-27 10:06:07,664][20347] Updated weights for policy 0, policy_version 646259 (0.0007) [2023-12-27 10:06:07,725][20347] Updated weights for policy 0, policy_version 646269 (0.0008) [2023-12-27 10:06:08,051][20381] Updated weights for policy 1, policy_version 645696 (0.0007) [2023-12-27 10:06:08,111][20381] Updated weights for policy 1, policy_version 645706 (0.0008) [2023-12-27 10:06:08,163][20381] Updated weights for policy 1, policy_version 645716 (0.0006) [2023-12-27 10:06:08,377][20347] Updated weights for policy 0, policy_version 646279 (0.0008) [2023-12-27 10:06:08,444][20347] Updated weights for policy 0, policy_version 646289 (0.0008) [2023-12-27 10:06:08,507][20347] Updated weights for policy 0, policy_version 646299 (0.0008) [2023-12-27 10:06:08,797][20381] Updated weights for policy 1, policy_version 645726 (0.0007) [2023-12-27 10:06:08,842][20381] Updated weights for policy 1, policy_version 645736 (0.0008) [2023-12-27 10:06:08,888][20381] Updated weights for policy 1, policy_version 645746 (0.0008) [2023-12-27 10:06:09,181][20347] Updated weights for policy 0, policy_version 646309 (0.0008) [2023-12-27 10:06:09,242][20347] Updated weights for policy 0, policy_version 646319 (0.0007) [2023-12-27 10:06:09,306][20347] Updated weights for policy 0, policy_version 646329 (0.0007) [2023-12-27 10:06:09,671][20381] Updated weights for policy 1, policy_version 645756 (0.0007) [2023-12-27 10:06:09,731][20381] Updated weights for policy 1, policy_version 645766 (0.0008) [2023-12-27 10:06:09,791][20381] Updated weights for policy 1, policy_version 645776 (0.0008) [2023-12-27 10:06:10,059][20347] Updated weights for policy 0, policy_version 646339 (0.0007) [2023-12-27 10:06:10,113][20347] Updated weights for policy 0, policy_version 646349 (0.0008) [2023-12-27 10:06:10,165][20347] Updated weights for policy 0, policy_version 646359 (0.0006) [2023-12-27 10:06:10,471][20381] Updated weights for policy 1, policy_version 645786 (0.0007) [2023-12-27 10:06:10,529][20381] Updated weights for policy 1, policy_version 645796 (0.0005) [2023-12-27 10:06:10,585][20381] Updated weights for policy 1, policy_version 645806 (0.0006) [2023-12-27 10:06:10,640][20381] Updated weights for policy 1, policy_version 645816 (0.0007) [2023-12-27 10:06:10,880][20347] Updated weights for policy 0, policy_version 646369 (0.0007) [2023-12-27 10:06:10,939][20347] Updated weights for policy 0, policy_version 646379 (0.0007) [2023-12-27 10:06:10,992][20347] Updated weights for policy 0, policy_version 646389 (0.0007) [2023-12-27 10:06:11,049][20347] Updated weights for policy 0, policy_version 646399 (0.0007) [2023-12-27 10:06:11,306][20381] Updated weights for policy 1, policy_version 645826 (0.0007) [2023-12-27 10:06:11,373][20381] Updated weights for policy 1, policy_version 645836 (0.0007) [2023-12-27 10:06:11,437][20381] Updated weights for policy 1, policy_version 645846 (0.0007) [2023-12-27 10:06:11,848][20347] Updated weights for policy 0, policy_version 646409 (0.0009) [2023-12-27 10:06:11,900][20347] Updated weights for policy 0, policy_version 646419 (0.0008) [2023-12-27 10:06:11,961][20347] Updated weights for policy 0, policy_version 646429 (0.0007) [2023-12-27 10:06:12,149][20381] Updated weights for policy 1, policy_version 645856 (0.0009) [2023-12-27 10:06:12,201][20381] Updated weights for policy 1, policy_version 645866 (0.0007) [2023-12-27 10:06:12,251][20381] Updated weights for policy 1, policy_version 645876 (0.0007) [2023-12-27 10:06:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 330874880. Throughput: 0: 10413.3, 1: 10205.7. Samples: 330862940. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:12,363][19170] Avg episode reward: [(0, '932.150'), (1, '931.500')] [2023-12-27 10:06:12,658][20347] Updated weights for policy 0, policy_version 646439 (0.0008) [2023-12-27 10:06:12,713][20347] Updated weights for policy 0, policy_version 646449 (0.0007) [2023-12-27 10:06:12,771][20347] Updated weights for policy 0, policy_version 646459 (0.0008) [2023-12-27 10:06:12,955][20381] Updated weights for policy 1, policy_version 645886 (0.0010) [2023-12-27 10:06:13,017][20381] Updated weights for policy 1, policy_version 645896 (0.0011) [2023-12-27 10:06:13,080][20381] Updated weights for policy 1, policy_version 645906 (0.0011) [2023-12-27 10:06:13,459][20347] Updated weights for policy 0, policy_version 646469 (0.0006) [2023-12-27 10:06:13,516][20347] Updated weights for policy 0, policy_version 646479 (0.0006) [2023-12-27 10:06:13,570][20347] Updated weights for policy 0, policy_version 646489 (0.0010) [2023-12-27 10:06:13,765][20381] Updated weights for policy 1, policy_version 645916 (0.0011) [2023-12-27 10:06:13,819][20381] Updated weights for policy 1, policy_version 645926 (0.0010) [2023-12-27 10:06:13,871][20381] Updated weights for policy 1, policy_version 645936 (0.0010) [2023-12-27 10:06:14,190][20347] Updated weights for policy 0, policy_version 646499 (0.0009) [2023-12-27 10:06:14,251][20347] Updated weights for policy 0, policy_version 646509 (0.0007) [2023-12-27 10:06:14,311][20347] Updated weights for policy 0, policy_version 646519 (0.0008) [2023-12-27 10:06:14,576][20381] Updated weights for policy 1, policy_version 645946 (0.0010) [2023-12-27 10:06:14,627][20381] Updated weights for policy 1, policy_version 645956 (0.0007) [2023-12-27 10:06:14,678][20381] Updated weights for policy 1, policy_version 645966 (0.0007) [2023-12-27 10:06:14,728][20381] Updated weights for policy 1, policy_version 645976 (0.0007) [2023-12-27 10:06:14,974][20347] Updated weights for policy 0, policy_version 646529 (0.0010) [2023-12-27 10:06:15,041][20347] Updated weights for policy 0, policy_version 646539 (0.0008) [2023-12-27 10:06:15,098][20347] Updated weights for policy 0, policy_version 646549 (0.0010) [2023-12-27 10:06:15,161][20347] Updated weights for policy 0, policy_version 646559 (0.0011) [2023-12-27 10:06:15,420][20381] Updated weights for policy 1, policy_version 645986 (0.0008) [2023-12-27 10:06:15,474][20381] Updated weights for policy 1, policy_version 645996 (0.0005) [2023-12-27 10:06:15,519][20381] Updated weights for policy 1, policy_version 646006 (0.0005) [2023-12-27 10:06:15,820][20347] Updated weights for policy 0, policy_version 646569 (0.0009) [2023-12-27 10:06:15,876][20347] Updated weights for policy 0, policy_version 646579 (0.0010) [2023-12-27 10:06:15,925][20347] Updated weights for policy 0, policy_version 646589 (0.0010) [2023-12-27 10:06:16,193][20381] Updated weights for policy 1, policy_version 646016 (0.0007) [2023-12-27 10:06:16,248][20381] Updated weights for policy 1, policy_version 646026 (0.0008) [2023-12-27 10:06:16,311][20381] Updated weights for policy 1, policy_version 646036 (0.0007) [2023-12-27 10:06:16,576][20347] Updated weights for policy 0, policy_version 646599 (0.0010) [2023-12-27 10:06:16,631][20347] Updated weights for policy 0, policy_version 646609 (0.0007) [2023-12-27 10:06:16,688][20347] Updated weights for policy 0, policy_version 646619 (0.0008) [2023-12-27 10:06:16,979][20381] Updated weights for policy 1, policy_version 646046 (0.0006) [2023-12-27 10:06:17,029][20381] Updated weights for policy 1, policy_version 646056 (0.0006) [2023-12-27 10:06:17,081][20381] Updated weights for policy 1, policy_version 646066 (0.0008) [2023-12-27 10:06:17,348][20347] Updated weights for policy 0, policy_version 646629 (0.0009) [2023-12-27 10:06:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 330973184. Throughput: 0: 10320.0, 1: 10198.6. Samples: 330984720. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:17,363][19170] Avg episode reward: [(0, '922.430'), (1, '941.370')] [2023-12-27 10:06:17,400][20347] Updated weights for policy 0, policy_version 646639 (0.0010) [2023-12-27 10:06:17,463][20347] Updated weights for policy 0, policy_version 646649 (0.0011) [2023-12-27 10:06:17,730][20381] Updated weights for policy 1, policy_version 646076 (0.0007) [2023-12-27 10:06:17,777][20381] Updated weights for policy 1, policy_version 646086 (0.0009) [2023-12-27 10:06:17,830][20381] Updated weights for policy 1, policy_version 646096 (0.0010) [2023-12-27 10:06:18,113][20347] Updated weights for policy 0, policy_version 646659 (0.0009) [2023-12-27 10:06:18,169][20347] Updated weights for policy 0, policy_version 646669 (0.0007) [2023-12-27 10:06:18,228][20347] Updated weights for policy 0, policy_version 646679 (0.0008) [2023-12-27 10:06:18,561][20381] Updated weights for policy 1, policy_version 646106 (0.0009) [2023-12-27 10:06:18,625][20381] Updated weights for policy 1, policy_version 646116 (0.0005) [2023-12-27 10:06:18,688][20381] Updated weights for policy 1, policy_version 646126 (0.0007) [2023-12-27 10:06:18,743][20381] Updated weights for policy 1, policy_version 646136 (0.0009) [2023-12-27 10:06:18,897][20347] Updated weights for policy 0, policy_version 646689 (0.0007) [2023-12-27 10:06:18,957][20347] Updated weights for policy 0, policy_version 646699 (0.0008) [2023-12-27 10:06:19,012][20347] Updated weights for policy 0, policy_version 646709 (0.0008) [2023-12-27 10:06:19,064][20347] Updated weights for policy 0, policy_version 646719 (0.0008) [2023-12-27 10:06:19,394][20381] Updated weights for policy 1, policy_version 646146 (0.0007) [2023-12-27 10:06:19,454][20381] Updated weights for policy 1, policy_version 646156 (0.0007) [2023-12-27 10:06:19,520][20381] Updated weights for policy 1, policy_version 646166 (0.0008) [2023-12-27 10:06:19,756][20347] Updated weights for policy 0, policy_version 646729 (0.0006) [2023-12-27 10:06:19,811][20347] Updated weights for policy 0, policy_version 646739 (0.0005) [2023-12-27 10:06:19,879][20347] Updated weights for policy 0, policy_version 646749 (0.0007) [2023-12-27 10:06:20,264][20381] Updated weights for policy 1, policy_version 646176 (0.0008) [2023-12-27 10:06:20,317][20381] Updated weights for policy 1, policy_version 646186 (0.0007) [2023-12-27 10:06:20,373][20381] Updated weights for policy 1, policy_version 646196 (0.0008) [2023-12-27 10:06:20,532][20347] Updated weights for policy 0, policy_version 646759 (0.0007) [2023-12-27 10:06:20,595][20347] Updated weights for policy 0, policy_version 646769 (0.0008) [2023-12-27 10:06:20,646][20347] Updated weights for policy 0, policy_version 646779 (0.0006) [2023-12-27 10:06:21,106][20381] Updated weights for policy 1, policy_version 646206 (0.0007) [2023-12-27 10:06:21,169][20381] Updated weights for policy 1, policy_version 646216 (0.0008) [2023-12-27 10:06:21,231][20381] Updated weights for policy 1, policy_version 646226 (0.0007) [2023-12-27 10:06:21,373][20347] Updated weights for policy 0, policy_version 646789 (0.0009) [2023-12-27 10:06:21,441][20347] Updated weights for policy 0, policy_version 646799 (0.0008) [2023-12-27 10:06:21,496][20347] Updated weights for policy 0, policy_version 646809 (0.0008) [2023-12-27 10:06:21,998][20381] Updated weights for policy 1, policy_version 646236 (0.0008) [2023-12-27 10:06:22,058][20381] Updated weights for policy 1, policy_version 646246 (0.0008) [2023-12-27 10:06:22,124][20381] Updated weights for policy 1, policy_version 646256 (0.0008) [2023-12-27 10:06:22,198][20347] Updated weights for policy 0, policy_version 646819 (0.0008) [2023-12-27 10:06:22,250][20347] Updated weights for policy 0, policy_version 646829 (0.0007) [2023-12-27 10:06:22,312][20347] Updated weights for policy 0, policy_version 646839 (0.0009) [2023-12-27 10:06:22,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20479.9, 300 sec: 20632.7). Total num frames: 331071488. Throughput: 0: 10286.1, 1: 10140.0. Samples: 331046168. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:22,363][19170] Avg episode reward: [(0, '971.480'), (1, '951.190')] [2023-12-27 10:06:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000646264_165470208.pth... [2023-12-27 10:06:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000645080_165167104.pth [2023-12-27 10:06:22,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000646848_165609472.pth... [2023-12-27 10:06:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000645664_165306368.pth [2023-12-27 10:06:22,907][20381] Updated weights for policy 1, policy_version 646266 (0.0008) [2023-12-27 10:06:22,965][20381] Updated weights for policy 1, policy_version 646276 (0.0008) [2023-12-27 10:06:22,980][20347] Updated weights for policy 0, policy_version 646849 (0.0009) [2023-12-27 10:06:23,019][20381] Updated weights for policy 1, policy_version 646286 (0.0006) [2023-12-27 10:06:23,037][20347] Updated weights for policy 0, policy_version 646860 (0.0009) [2023-12-27 10:06:23,071][20381] Updated weights for policy 1, policy_version 646296 (0.0007) [2023-12-27 10:06:23,086][20347] Updated weights for policy 0, policy_version 646870 (0.0009) [2023-12-27 10:06:23,140][20347] Updated weights for policy 0, policy_version 646880 (0.0006) [2023-12-27 10:06:23,725][20381] Updated weights for policy 1, policy_version 646306 (0.0007) [2023-12-27 10:06:23,780][20347] Updated weights for policy 0, policy_version 646890 (0.0006) [2023-12-27 10:06:23,782][20381] Updated weights for policy 1, policy_version 646316 (0.0006) [2023-12-27 10:06:23,834][20347] Updated weights for policy 0, policy_version 646900 (0.0006) [2023-12-27 10:06:23,840][20381] Updated weights for policy 1, policy_version 646326 (0.0005) [2023-12-27 10:06:23,880][20347] Updated weights for policy 0, policy_version 646910 (0.0007) [2023-12-27 10:06:24,512][20347] Updated weights for policy 0, policy_version 646920 (0.0006) [2023-12-27 10:06:24,556][20381] Updated weights for policy 1, policy_version 646336 (0.0007) [2023-12-27 10:06:24,570][20347] Updated weights for policy 0, policy_version 646930 (0.0005) [2023-12-27 10:06:24,602][20381] Updated weights for policy 1, policy_version 646346 (0.0007) [2023-12-27 10:06:24,630][20347] Updated weights for policy 0, policy_version 646940 (0.0007) [2023-12-27 10:06:24,651][20381] Updated weights for policy 1, policy_version 646356 (0.0006) [2023-12-27 10:06:25,193][20347] Updated weights for policy 0, policy_version 646950 (0.0007) [2023-12-27 10:06:25,252][20347] Updated weights for policy 0, policy_version 646960 (0.0008) [2023-12-27 10:06:25,312][20347] Updated weights for policy 0, policy_version 646970 (0.0005) [2023-12-27 10:06:25,357][20381] Updated weights for policy 1, policy_version 646366 (0.0007) [2023-12-27 10:06:25,414][20381] Updated weights for policy 1, policy_version 646376 (0.0009) [2023-12-27 10:06:25,477][20381] Updated weights for policy 1, policy_version 646386 (0.0007) [2023-12-27 10:06:25,934][20347] Updated weights for policy 0, policy_version 646980 (0.0006) [2023-12-27 10:06:25,992][20347] Updated weights for policy 0, policy_version 646990 (0.0007) [2023-12-27 10:06:26,041][20347] Updated weights for policy 0, policy_version 647000 (0.0007) [2023-12-27 10:06:26,184][20381] Updated weights for policy 1, policy_version 646396 (0.0008) [2023-12-27 10:06:26,230][20381] Updated weights for policy 1, policy_version 646406 (0.0008) [2023-12-27 10:06:26,287][20381] Updated weights for policy 1, policy_version 646416 (0.0007) [2023-12-27 10:06:26,647][20347] Updated weights for policy 0, policy_version 647010 (0.0007) [2023-12-27 10:06:26,706][20347] Updated weights for policy 0, policy_version 647020 (0.0007) [2023-12-27 10:06:26,766][20347] Updated weights for policy 0, policy_version 647030 (0.0007) [2023-12-27 10:06:26,816][20347] Updated weights for policy 0, policy_version 647040 (0.0006) [2023-12-27 10:06:27,008][20381] Updated weights for policy 1, policy_version 646426 (0.0010) [2023-12-27 10:06:27,059][20381] Updated weights for policy 1, policy_version 646436 (0.0010) [2023-12-27 10:06:27,113][20381] Updated weights for policy 1, policy_version 646446 (0.0010) [2023-12-27 10:06:27,170][20381] Updated weights for policy 1, policy_version 646456 (0.0010) [2023-12-27 10:06:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.1, 300 sec: 20660.5). Total num frames: 331177984. Throughput: 0: 10345.0, 1: 10077.1. Samples: 331167876. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:27,363][19170] Avg episode reward: [(0, '952.050'), (1, '941.440')] [2023-12-27 10:06:27,471][20347] Updated weights for policy 0, policy_version 647050 (0.0010) [2023-12-27 10:06:27,526][20347] Updated weights for policy 0, policy_version 647060 (0.0008) [2023-12-27 10:06:27,589][20347] Updated weights for policy 0, policy_version 647070 (0.0005) [2023-12-27 10:06:27,842][20381] Updated weights for policy 1, policy_version 646466 (0.0010) [2023-12-27 10:06:27,892][20381] Updated weights for policy 1, policy_version 646476 (0.0010) [2023-12-27 10:06:27,936][20381] Updated weights for policy 1, policy_version 646486 (0.0010) [2023-12-27 10:06:28,163][20347] Updated weights for policy 0, policy_version 647080 (0.0005) [2023-12-27 10:06:28,223][20347] Updated weights for policy 0, policy_version 647090 (0.0005) [2023-12-27 10:06:28,287][20347] Updated weights for policy 0, policy_version 647100 (0.0005) [2023-12-27 10:06:28,613][20381] Updated weights for policy 1, policy_version 646496 (0.0006) [2023-12-27 10:06:28,672][20381] Updated weights for policy 1, policy_version 646506 (0.0010) [2023-12-27 10:06:28,726][20381] Updated weights for policy 1, policy_version 646516 (0.0010) [2023-12-27 10:06:28,897][20347] Updated weights for policy 0, policy_version 647110 (0.0009) [2023-12-27 10:06:28,955][20347] Updated weights for policy 0, policy_version 647120 (0.0010) [2023-12-27 10:06:29,021][20347] Updated weights for policy 0, policy_version 647130 (0.0010) [2023-12-27 10:06:29,383][20381] Updated weights for policy 1, policy_version 646526 (0.0009) [2023-12-27 10:06:29,436][20381] Updated weights for policy 1, policy_version 646536 (0.0006) [2023-12-27 10:06:29,492][20381] Updated weights for policy 1, policy_version 646546 (0.0010) [2023-12-27 10:06:29,701][20347] Updated weights for policy 0, policy_version 647140 (0.0010) [2023-12-27 10:06:29,756][20347] Updated weights for policy 0, policy_version 647150 (0.0010) [2023-12-27 10:06:29,814][20347] Updated weights for policy 0, policy_version 647160 (0.0010) [2023-12-27 10:06:30,147][20381] Updated weights for policy 1, policy_version 646556 (0.0010) [2023-12-27 10:06:30,199][20381] Updated weights for policy 1, policy_version 646566 (0.0010) [2023-12-27 10:06:30,251][20381] Updated weights for policy 1, policy_version 646576 (0.0010) [2023-12-27 10:06:30,387][20347] Updated weights for policy 0, policy_version 647170 (0.0008) [2023-12-27 10:06:30,431][20347] Updated weights for policy 0, policy_version 647180 (0.0005) [2023-12-27 10:06:30,477][20347] Updated weights for policy 0, policy_version 647190 (0.0005) [2023-12-27 10:06:30,529][20347] Updated weights for policy 0, policy_version 647200 (0.0008) [2023-12-27 10:06:30,952][20381] Updated weights for policy 1, policy_version 646586 (0.0011) [2023-12-27 10:06:31,002][20381] Updated weights for policy 1, policy_version 646596 (0.0010) [2023-12-27 10:06:31,064][20381] Updated weights for policy 1, policy_version 646606 (0.0009) [2023-12-27 10:06:31,113][20381] Updated weights for policy 1, policy_version 646616 (0.0010) [2023-12-27 10:06:31,144][20347] Updated weights for policy 0, policy_version 647210 (0.0006) [2023-12-27 10:06:31,200][20347] Updated weights for policy 0, policy_version 647220 (0.0007) [2023-12-27 10:06:31,256][20347] Updated weights for policy 0, policy_version 647230 (0.0007) [2023-12-27 10:06:31,833][20381] Updated weights for policy 1, policy_version 646626 (0.0009) [2023-12-27 10:06:31,890][20381] Updated weights for policy 1, policy_version 646636 (0.0008) [2023-12-27 10:06:31,929][20347] Updated weights for policy 0, policy_version 647240 (0.0007) [2023-12-27 10:06:31,939][20381] Updated weights for policy 1, policy_version 646646 (0.0005) [2023-12-27 10:06:31,984][20347] Updated weights for policy 0, policy_version 647250 (0.0006) [2023-12-27 10:06:32,035][20347] Updated weights for policy 0, policy_version 647260 (0.0005) [2023-12-27 10:06:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 331284480. Throughput: 0: 10386.9, 1: 10136.1. Samples: 331295936. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:32,363][19170] Avg episode reward: [(0, '951.470'), (1, '922.070')] [2023-12-27 10:06:32,593][20381] Updated weights for policy 1, policy_version 646656 (0.0007) [2023-12-27 10:06:32,627][20347] Updated weights for policy 0, policy_version 647270 (0.0006) [2023-12-27 10:06:32,652][20381] Updated weights for policy 1, policy_version 646666 (0.0007) [2023-12-27 10:06:32,683][20347] Updated weights for policy 0, policy_version 647280 (0.0005) [2023-12-27 10:06:32,709][20381] Updated weights for policy 1, policy_version 646676 (0.0007) [2023-12-27 10:06:32,736][20347] Updated weights for policy 0, policy_version 647290 (0.0005) [2023-12-27 10:06:33,317][20381] Updated weights for policy 1, policy_version 646686 (0.0006) [2023-12-27 10:06:33,330][20347] Updated weights for policy 0, policy_version 647300 (0.0007) [2023-12-27 10:06:33,378][20381] Updated weights for policy 1, policy_version 646696 (0.0005) [2023-12-27 10:06:33,386][20347] Updated weights for policy 0, policy_version 647310 (0.0008) [2023-12-27 10:06:33,436][20381] Updated weights for policy 1, policy_version 646706 (0.0005) [2023-12-27 10:06:33,444][20347] Updated weights for policy 0, policy_version 647320 (0.0006) [2023-12-27 10:06:33,957][20381] Updated weights for policy 1, policy_version 646716 (0.0006) [2023-12-27 10:06:34,015][20381] Updated weights for policy 1, policy_version 646726 (0.0008) [2023-12-27 10:06:34,074][20381] Updated weights for policy 1, policy_version 646736 (0.0008) [2023-12-27 10:06:34,129][20347] Updated weights for policy 0, policy_version 647330 (0.0008) [2023-12-27 10:06:34,192][20347] Updated weights for policy 0, policy_version 647340 (0.0008) [2023-12-27 10:06:34,247][20347] Updated weights for policy 0, policy_version 647350 (0.0007) [2023-12-27 10:06:34,309][20347] Updated weights for policy 0, policy_version 647360 (0.0007) [2023-12-27 10:06:34,770][20381] Updated weights for policy 1, policy_version 646746 (0.0007) [2023-12-27 10:06:34,822][20381] Updated weights for policy 1, policy_version 646756 (0.0005) [2023-12-27 10:06:34,878][20381] Updated weights for policy 1, policy_version 646766 (0.0005) [2023-12-27 10:06:34,931][20381] Updated weights for policy 1, policy_version 646776 (0.0005) [2023-12-27 10:06:35,053][20347] Updated weights for policy 0, policy_version 647370 (0.0006) [2023-12-27 10:06:35,113][20347] Updated weights for policy 0, policy_version 647380 (0.0007) [2023-12-27 10:06:35,166][20347] Updated weights for policy 0, policy_version 647390 (0.0007) [2023-12-27 10:06:35,545][20381] Updated weights for policy 1, policy_version 646786 (0.0007) [2023-12-27 10:06:35,606][20381] Updated weights for policy 1, policy_version 646797 (0.0008) [2023-12-27 10:06:35,660][20381] Updated weights for policy 1, policy_version 646807 (0.0007) [2023-12-27 10:06:35,822][20347] Updated weights for policy 0, policy_version 647400 (0.0007) [2023-12-27 10:06:35,868][20347] Updated weights for policy 0, policy_version 647410 (0.0007) [2023-12-27 10:06:35,927][20347] Updated weights for policy 0, policy_version 647420 (0.0008) [2023-12-27 10:06:36,354][20381] Updated weights for policy 1, policy_version 646817 (0.0007) [2023-12-27 10:06:36,414][20381] Updated weights for policy 1, policy_version 646827 (0.0008) [2023-12-27 10:06:36,472][20381] Updated weights for policy 1, policy_version 646837 (0.0007) [2023-12-27 10:06:36,628][20347] Updated weights for policy 0, policy_version 647430 (0.0008) [2023-12-27 10:06:36,688][20347] Updated weights for policy 0, policy_version 647440 (0.0008) [2023-12-27 10:06:36,737][20347] Updated weights for policy 0, policy_version 647450 (0.0007) [2023-12-27 10:06:37,065][20381] Updated weights for policy 1, policy_version 646847 (0.0005) [2023-12-27 10:06:37,121][20381] Updated weights for policy 1, policy_version 646857 (0.0007) [2023-12-27 10:06:37,182][20381] Updated weights for policy 1, policy_version 646867 (0.0008) [2023-12-27 10:06:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.7, 300 sec: 20688.3). Total num frames: 331390976. Throughput: 0: 10370.1, 1: 10223.4. Samples: 331361136. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:37,363][19170] Avg episode reward: [(0, '951.550'), (1, '912.620')] [2023-12-27 10:06:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000646872_165625856.pth... [2023-12-27 10:06:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000647456_165765120.pth... [2023-12-27 10:06:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000645688_165322752.pth [2023-12-27 10:06:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000646240_165453824.pth [2023-12-27 10:06:37,467][20347] Updated weights for policy 0, policy_version 647460 (0.0008) [2023-12-27 10:06:37,528][20347] Updated weights for policy 0, policy_version 647470 (0.0007) [2023-12-27 10:06:37,582][20347] Updated weights for policy 0, policy_version 647480 (0.0007) [2023-12-27 10:06:37,859][20381] Updated weights for policy 1, policy_version 646877 (0.0007) [2023-12-27 10:06:37,911][20381] Updated weights for policy 1, policy_version 646887 (0.0008) [2023-12-27 10:06:37,963][20381] Updated weights for policy 1, policy_version 646897 (0.0007) [2023-12-27 10:06:38,266][20347] Updated weights for policy 0, policy_version 647490 (0.0007) [2023-12-27 10:06:38,323][20347] Updated weights for policy 0, policy_version 647500 (0.0008) [2023-12-27 10:06:38,379][20347] Updated weights for policy 0, policy_version 647510 (0.0008) [2023-12-27 10:06:38,438][20347] Updated weights for policy 0, policy_version 647520 (0.0006) [2023-12-27 10:06:38,652][20381] Updated weights for policy 1, policy_version 646907 (0.0008) [2023-12-27 10:06:38,716][20381] Updated weights for policy 1, policy_version 646917 (0.0010) [2023-12-27 10:06:38,781][20381] Updated weights for policy 1, policy_version 646927 (0.0010) [2023-12-27 10:06:39,065][20347] Updated weights for policy 0, policy_version 647530 (0.0006) [2023-12-27 10:06:39,113][20347] Updated weights for policy 0, policy_version 647540 (0.0007) [2023-12-27 10:06:39,167][20347] Updated weights for policy 0, policy_version 647550 (0.0007) [2023-12-27 10:06:39,459][20381] Updated weights for policy 1, policy_version 646937 (0.0010) [2023-12-27 10:06:39,517][20381] Updated weights for policy 1, policy_version 646947 (0.0011) [2023-12-27 10:06:39,574][20381] Updated weights for policy 1, policy_version 646957 (0.0010) [2023-12-27 10:06:39,633][20381] Updated weights for policy 1, policy_version 646967 (0.0011) [2023-12-27 10:06:39,961][20347] Updated weights for policy 0, policy_version 647560 (0.0008) [2023-12-27 10:06:40,021][20347] Updated weights for policy 0, policy_version 647570 (0.0008) [2023-12-27 10:06:40,084][20347] Updated weights for policy 0, policy_version 647580 (0.0008) [2023-12-27 10:06:40,422][20381] Updated weights for policy 1, policy_version 646977 (0.0009) [2023-12-27 10:06:40,481][20381] Updated weights for policy 1, policy_version 646987 (0.0009) [2023-12-27 10:06:40,538][20381] Updated weights for policy 1, policy_version 646997 (0.0009) [2023-12-27 10:06:40,783][20347] Updated weights for policy 0, policy_version 647590 (0.0008) [2023-12-27 10:06:40,842][20347] Updated weights for policy 0, policy_version 647600 (0.0008) [2023-12-27 10:06:40,893][20347] Updated weights for policy 0, policy_version 647610 (0.0009) [2023-12-27 10:06:41,197][20381] Updated weights for policy 1, policy_version 647007 (0.0007) [2023-12-27 10:06:41,255][20381] Updated weights for policy 1, policy_version 647017 (0.0008) [2023-12-27 10:06:41,317][20381] Updated weights for policy 1, policy_version 647027 (0.0008) [2023-12-27 10:06:41,610][20347] Updated weights for policy 0, policy_version 647620 (0.0008) [2023-12-27 10:06:41,676][20347] Updated weights for policy 0, policy_version 647630 (0.0009) [2023-12-27 10:06:41,735][20347] Updated weights for policy 0, policy_version 647640 (0.0008) [2023-12-27 10:06:42,001][20381] Updated weights for policy 1, policy_version 647037 (0.0007) [2023-12-27 10:06:42,061][20381] Updated weights for policy 1, policy_version 647047 (0.0008) [2023-12-27 10:06:42,120][20381] Updated weights for policy 1, policy_version 647057 (0.0008) [2023-12-27 10:06:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.8). Total num frames: 331489280. Throughput: 0: 10336.6, 1: 10209.8. Samples: 331481488. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:42,363][19170] Avg episode reward: [(0, '875.190'), (1, '815.550')] [2023-12-27 10:06:42,448][20347] Updated weights for policy 0, policy_version 647650 (0.0009) [2023-12-27 10:06:42,504][20347] Updated weights for policy 0, policy_version 647660 (0.0011) [2023-12-27 10:06:42,561][20347] Updated weights for policy 0, policy_version 647670 (0.0010) [2023-12-27 10:06:42,619][20347] Updated weights for policy 0, policy_version 647680 (0.0010) [2023-12-27 10:06:42,819][20381] Updated weights for policy 1, policy_version 647067 (0.0007) [2023-12-27 10:06:42,870][20381] Updated weights for policy 1, policy_version 647077 (0.0008) [2023-12-27 10:06:42,914][20381] Updated weights for policy 1, policy_version 647087 (0.0007) [2023-12-27 10:06:43,306][20347] Updated weights for policy 0, policy_version 647690 (0.0007) [2023-12-27 10:06:43,306][20291] KL-divergence is very high: 124.1091 [2023-12-27 10:06:43,358][20347] Updated weights for policy 0, policy_version 647700 (0.0010) [2023-12-27 10:06:43,410][20347] Updated weights for policy 0, policy_version 647710 (0.0010) [2023-12-27 10:06:43,589][20381] Updated weights for policy 1, policy_version 647097 (0.0008) [2023-12-27 10:06:43,670][20381] Updated weights for policy 1, policy_version 647107 (0.0010) [2023-12-27 10:06:43,728][20381] Updated weights for policy 1, policy_version 647117 (0.0010) [2023-12-27 10:06:43,782][20381] Updated weights for policy 1, policy_version 647127 (0.0010) [2023-12-27 10:06:44,057][20347] Updated weights for policy 0, policy_version 647720 (0.0010) [2023-12-27 10:06:44,108][20347] Updated weights for policy 0, policy_version 647730 (0.0010) [2023-12-27 10:06:44,162][20347] Updated weights for policy 0, policy_version 647740 (0.0010) [2023-12-27 10:06:44,424][20381] Updated weights for policy 1, policy_version 647137 (0.0010) [2023-12-27 10:06:44,488][20381] Updated weights for policy 1, policy_version 647147 (0.0010) [2023-12-27 10:06:44,539][20381] Updated weights for policy 1, policy_version 647157 (0.0010) [2023-12-27 10:06:44,866][20347] Updated weights for policy 0, policy_version 647750 (0.0011) [2023-12-27 10:06:44,929][20347] Updated weights for policy 0, policy_version 647760 (0.0009) [2023-12-27 10:06:44,999][20347] Updated weights for policy 0, policy_version 647770 (0.0005) [2023-12-27 10:06:45,239][20381] Updated weights for policy 1, policy_version 647167 (0.0008) [2023-12-27 10:06:45,295][20381] Updated weights for policy 1, policy_version 647177 (0.0008) [2023-12-27 10:06:45,347][20381] Updated weights for policy 1, policy_version 647187 (0.0008) [2023-12-27 10:06:45,590][20347] Updated weights for policy 0, policy_version 647780 (0.0007) [2023-12-27 10:06:45,647][20347] Updated weights for policy 0, policy_version 647790 (0.0010) [2023-12-27 10:06:45,701][20347] Updated weights for policy 0, policy_version 647800 (0.0010) [2023-12-27 10:06:45,985][20381] Updated weights for policy 1, policy_version 647197 (0.0009) [2023-12-27 10:06:46,048][20381] Updated weights for policy 1, policy_version 647207 (0.0007) [2023-12-27 10:06:46,106][20381] Updated weights for policy 1, policy_version 647217 (0.0007) [2023-12-27 10:06:46,362][20347] Updated weights for policy 0, policy_version 647810 (0.0009) [2023-12-27 10:06:46,426][20347] Updated weights for policy 0, policy_version 647820 (0.0007) [2023-12-27 10:06:46,484][20347] Updated weights for policy 0, policy_version 647830 (0.0010) [2023-12-27 10:06:46,533][20347] Updated weights for policy 0, policy_version 647840 (0.0010) [2023-12-27 10:06:46,795][20381] Updated weights for policy 1, policy_version 647227 (0.0009) [2023-12-27 10:06:46,850][20381] Updated weights for policy 1, policy_version 647237 (0.0010) [2023-12-27 10:06:46,905][20381] Updated weights for policy 1, policy_version 647247 (0.0010) [2023-12-27 10:06:47,188][20347] Updated weights for policy 0, policy_version 647850 (0.0010) [2023-12-27 10:06:47,232][20347] Updated weights for policy 0, policy_version 647860 (0.0010) [2023-12-27 10:06:47,289][20347] Updated weights for policy 0, policy_version 647870 (0.0010) [2023-12-27 10:06:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 331595776. Throughput: 0: 10358.7, 1: 10177.5. Samples: 331604160. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 10:06:47,363][19170] Avg episode reward: [(0, '838.060'), (1, '912.450')] [2023-12-27 10:06:47,594][20381] Updated weights for policy 1, policy_version 647257 (0.0010) [2023-12-27 10:06:47,656][20381] Updated weights for policy 1, policy_version 647267 (0.0011) [2023-12-27 10:06:47,711][20381] Updated weights for policy 1, policy_version 647277 (0.0010) [2023-12-27 10:06:47,775][20381] Updated weights for policy 1, policy_version 647287 (0.0006) [2023-12-27 10:06:47,969][20347] Updated weights for policy 0, policy_version 647880 (0.0009) [2023-12-27 10:06:48,031][20347] Updated weights for policy 0, policy_version 647890 (0.0010) [2023-12-27 10:06:48,090][20347] Updated weights for policy 0, policy_version 647900 (0.0008) [2023-12-27 10:06:48,461][20381] Updated weights for policy 1, policy_version 647297 (0.0007) [2023-12-27 10:06:48,516][20381] Updated weights for policy 1, policy_version 647307 (0.0008) [2023-12-27 10:06:48,577][20381] Updated weights for policy 1, policy_version 647317 (0.0006) [2023-12-27 10:06:48,744][20347] Updated weights for policy 0, policy_version 647910 (0.0009) [2023-12-27 10:06:48,797][20347] Updated weights for policy 0, policy_version 647920 (0.0005) [2023-12-27 10:06:48,856][20347] Updated weights for policy 0, policy_version 647930 (0.0008) [2023-12-27 10:06:49,263][20381] Updated weights for policy 1, policy_version 647327 (0.0007) [2023-12-27 10:06:49,323][20381] Updated weights for policy 1, policy_version 647337 (0.0008) [2023-12-27 10:06:49,385][20381] Updated weights for policy 1, policy_version 647347 (0.0008) [2023-12-27 10:06:49,500][20347] Updated weights for policy 0, policy_version 647940 (0.0008) [2023-12-27 10:06:49,550][20347] Updated weights for policy 0, policy_version 647950 (0.0005) [2023-12-27 10:06:49,610][20347] Updated weights for policy 0, policy_version 647960 (0.0006) [2023-12-27 10:06:50,027][20381] Updated weights for policy 1, policy_version 647357 (0.0007) [2023-12-27 10:06:50,090][20381] Updated weights for policy 1, policy_version 647367 (0.0008) [2023-12-27 10:06:50,149][20381] Updated weights for policy 1, policy_version 647377 (0.0009) [2023-12-27 10:06:50,214][20347] Updated weights for policy 0, policy_version 647970 (0.0006) [2023-12-27 10:06:50,276][20347] Updated weights for policy 0, policy_version 647980 (0.0007) [2023-12-27 10:06:50,331][20347] Updated weights for policy 0, policy_version 647990 (0.0007) [2023-12-27 10:06:50,386][20347] Updated weights for policy 0, policy_version 648000 (0.0007) [2023-12-27 10:06:50,769][20381] Updated weights for policy 1, policy_version 647387 (0.0007) [2023-12-27 10:06:50,829][20381] Updated weights for policy 1, policy_version 647397 (0.0008) [2023-12-27 10:06:50,886][20381] Updated weights for policy 1, policy_version 647407 (0.0005) [2023-12-27 10:06:51,112][20347] Updated weights for policy 0, policy_version 648010 (0.0007) [2023-12-27 10:06:51,173][20347] Updated weights for policy 0, policy_version 648020 (0.0007) [2023-12-27 10:06:51,241][20347] Updated weights for policy 0, policy_version 648030 (0.0007) [2023-12-27 10:06:51,514][20381] Updated weights for policy 1, policy_version 647417 (0.0008) [2023-12-27 10:06:51,572][20381] Updated weights for policy 1, policy_version 647427 (0.0011) [2023-12-27 10:06:51,631][20381] Updated weights for policy 1, policy_version 647437 (0.0011) [2023-12-27 10:06:51,683][20381] Updated weights for policy 1, policy_version 647447 (0.0010) [2023-12-27 10:06:52,000][20347] Updated weights for policy 0, policy_version 648040 (0.0008) [2023-12-27 10:06:52,056][20347] Updated weights for policy 0, policy_version 648050 (0.0008) [2023-12-27 10:06:52,107][20347] Updated weights for policy 0, policy_version 648060 (0.0007) [2023-12-27 10:06:52,290][20381] Updated weights for policy 1, policy_version 647457 (0.0008) [2023-12-27 10:06:52,349][20381] Updated weights for policy 1, policy_version 647467 (0.0009) [2023-12-27 10:06:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 331694080. Throughput: 0: 10386.2, 1: 10151.3. Samples: 331666780. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:06:52,363][19170] Avg episode reward: [(0, '876.420'), (1, '912.230')] [2023-12-27 10:06:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000648064_165920768.pth... [2023-12-27 10:06:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000646848_165609472.pth [2023-12-27 10:06:52,410][20381] Updated weights for policy 1, policy_version 647477 (0.0011) [2023-12-27 10:06:52,424][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000647480_165781504.pth... [2023-12-27 10:06:52,428][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000646264_165470208.pth [2023-12-27 10:06:52,768][20347] Updated weights for policy 0, policy_version 648070 (0.0008) [2023-12-27 10:06:52,829][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 10:06:52,831][20347] Updated weights for policy 0, policy_version 648080 (0.0007) [2023-12-27 10:06:53,037][20381] Updated weights for policy 1, policy_version 647487 (0.0007) [2023-12-27 10:06:53,086][20381] Updated weights for policy 1, policy_version 647497 (0.0005) [2023-12-27 10:06:53,132][20381] Updated weights for policy 1, policy_version 647507 (0.0008) [2023-12-27 10:06:53,588][20347] Updated weights for policy 0, policy_version 648090 (0.0005) [2023-12-27 10:06:53,646][20347] Updated weights for policy 0, policy_version 648100 (0.0005) [2023-12-27 10:06:53,707][20347] Updated weights for policy 0, policy_version 648110 (0.0005) [2023-12-27 10:06:53,774][20381] Updated weights for policy 1, policy_version 647517 (0.0010) [2023-12-27 10:06:53,832][20381] Updated weights for policy 1, policy_version 647527 (0.0011) [2023-12-27 10:06:53,880][20381] Updated weights for policy 1, policy_version 647537 (0.0010) [2023-12-27 10:06:54,257][20347] Updated weights for policy 0, policy_version 648120 (0.0009) [2023-12-27 10:06:54,314][20347] Updated weights for policy 0, policy_version 648130 (0.0010) [2023-12-27 10:06:54,379][20347] Updated weights for policy 0, policy_version 648140 (0.0010) [2023-12-27 10:06:54,550][20381] Updated weights for policy 1, policy_version 647547 (0.0010) [2023-12-27 10:06:54,601][20381] Updated weights for policy 1, policy_version 647557 (0.0008) [2023-12-27 10:06:54,657][20381] Updated weights for policy 1, policy_version 647567 (0.0008) [2023-12-27 10:06:55,019][20347] Updated weights for policy 0, policy_version 648150 (0.0008) [2023-12-27 10:06:55,065][20347] Updated weights for policy 0, policy_version 648160 (0.0008) [2023-12-27 10:06:55,111][20347] Updated weights for policy 0, policy_version 648170 (0.0007) [2023-12-27 10:06:55,256][20381] Updated weights for policy 1, policy_version 647577 (0.0006) [2023-12-27 10:06:55,314][20381] Updated weights for policy 1, policy_version 647587 (0.0008) [2023-12-27 10:06:55,368][20381] Updated weights for policy 1, policy_version 647597 (0.0009) [2023-12-27 10:06:55,422][20381] Updated weights for policy 1, policy_version 647607 (0.0007) [2023-12-27 10:06:55,788][20347] Updated weights for policy 0, policy_version 648180 (0.0008) [2023-12-27 10:06:55,845][20347] Updated weights for policy 0, policy_version 648190 (0.0008) [2023-12-27 10:06:55,908][20347] Updated weights for policy 0, policy_version 648200 (0.0006) [2023-12-27 10:06:56,141][20381] Updated weights for policy 1, policy_version 647617 (0.0008) [2023-12-27 10:06:56,207][20381] Updated weights for policy 1, policy_version 647627 (0.0008) [2023-12-27 10:06:56,278][20381] Updated weights for policy 1, policy_version 647637 (0.0008) [2023-12-27 10:06:56,508][20347] Updated weights for policy 0, policy_version 648210 (0.0006) [2023-12-27 10:06:56,562][20347] Updated weights for policy 0, policy_version 648220 (0.0007) [2023-12-27 10:06:56,607][20347] Updated weights for policy 0, policy_version 648230 (0.0007) [2023-12-27 10:06:56,654][20347] Updated weights for policy 0, policy_version 648240 (0.0008) [2023-12-27 10:06:56,969][20381] Updated weights for policy 1, policy_version 647647 (0.0009) [2023-12-27 10:06:57,018][20381] Updated weights for policy 1, policy_version 647657 (0.0009) [2023-12-27 10:06:57,077][20381] Updated weights for policy 1, policy_version 647667 (0.0009) [2023-12-27 10:06:57,297][20347] Updated weights for policy 0, policy_version 648250 (0.0007) [2023-12-27 10:06:57,352][20347] Updated weights for policy 0, policy_version 648260 (0.0008) [2023-12-27 10:06:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 331800576. Throughput: 0: 10415.1, 1: 10250.8. Samples: 331792900. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:06:57,363][19170] Avg episode reward: [(0, '942.880'), (1, '902.650')] [2023-12-27 10:06:57,397][20347] Updated weights for policy 0, policy_version 648270 (0.0007) [2023-12-27 10:06:57,779][20381] Updated weights for policy 1, policy_version 647677 (0.0008) [2023-12-27 10:06:57,835][20381] Updated weights for policy 1, policy_version 647687 (0.0008) [2023-12-27 10:06:57,892][20381] Updated weights for policy 1, policy_version 647697 (0.0006) [2023-12-27 10:06:58,036][20347] Updated weights for policy 0, policy_version 648280 (0.0008) [2023-12-27 10:06:58,084][20347] Updated weights for policy 0, policy_version 648290 (0.0007) [2023-12-27 10:06:58,133][20347] Updated weights for policy 0, policy_version 648300 (0.0009) [2023-12-27 10:06:58,539][20381] Updated weights for policy 1, policy_version 647707 (0.0008) [2023-12-27 10:06:58,601][20381] Updated weights for policy 1, policy_version 647717 (0.0008) [2023-12-27 10:06:58,660][20381] Updated weights for policy 1, policy_version 647727 (0.0008) [2023-12-27 10:06:58,888][20347] Updated weights for policy 0, policy_version 648310 (0.0006) [2023-12-27 10:06:58,942][20347] Updated weights for policy 0, policy_version 648320 (0.0006) [2023-12-27 10:06:59,003][20347] Updated weights for policy 0, policy_version 648330 (0.0007) [2023-12-27 10:06:59,414][20381] Updated weights for policy 1, policy_version 647737 (0.0008) [2023-12-27 10:06:59,472][20381] Updated weights for policy 1, policy_version 647747 (0.0005) [2023-12-27 10:06:59,525][20381] Updated weights for policy 1, policy_version 647757 (0.0005) [2023-12-27 10:06:59,578][20381] Updated weights for policy 1, policy_version 647767 (0.0006) [2023-12-27 10:06:59,711][20347] Updated weights for policy 0, policy_version 648340 (0.0007) [2023-12-27 10:06:59,760][20347] Updated weights for policy 0, policy_version 648350 (0.0005) [2023-12-27 10:06:59,816][20347] Updated weights for policy 0, policy_version 648360 (0.0006) [2023-12-27 10:07:00,194][20381] Updated weights for policy 1, policy_version 647777 (0.0007) [2023-12-27 10:07:00,253][20381] Updated weights for policy 1, policy_version 647787 (0.0008) [2023-12-27 10:07:00,312][20381] Updated weights for policy 1, policy_version 647797 (0.0007) [2023-12-27 10:07:00,497][20347] Updated weights for policy 0, policy_version 648370 (0.0007) [2023-12-27 10:07:00,552][20347] Updated weights for policy 0, policy_version 648380 (0.0007) [2023-12-27 10:07:00,604][20347] Updated weights for policy 0, policy_version 648390 (0.0008) [2023-12-27 10:07:00,658][20347] Updated weights for policy 0, policy_version 648400 (0.0006) [2023-12-27 10:07:01,007][20381] Updated weights for policy 1, policy_version 647807 (0.0007) [2023-12-27 10:07:01,064][20381] Updated weights for policy 1, policy_version 647817 (0.0008) [2023-12-27 10:07:01,126][20381] Updated weights for policy 1, policy_version 647827 (0.0008) [2023-12-27 10:07:01,335][20347] Updated weights for policy 0, policy_version 648410 (0.0010) [2023-12-27 10:07:01,392][20347] Updated weights for policy 0, policy_version 648420 (0.0006) [2023-12-27 10:07:01,453][20347] Updated weights for policy 0, policy_version 648430 (0.0007) [2023-12-27 10:07:01,769][20381] Updated weights for policy 1, policy_version 647837 (0.0008) [2023-12-27 10:07:01,826][20381] Updated weights for policy 1, policy_version 647847 (0.0007) [2023-12-27 10:07:01,884][20381] Updated weights for policy 1, policy_version 647857 (0.0008) [2023-12-27 10:07:02,093][20347] Updated weights for policy 0, policy_version 648440 (0.0010) [2023-12-27 10:07:02,147][20347] Updated weights for policy 0, policy_version 648450 (0.0005) [2023-12-27 10:07:02,209][20347] Updated weights for policy 0, policy_version 648460 (0.0005) [2023-12-27 10:07:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 331907072. Throughput: 0: 10444.6, 1: 10247.7. Samples: 331915868. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:02,363][19170] Avg episode reward: [(0, '932.950'), (1, '941.650')] [2023-12-27 10:07:02,590][20381] Updated weights for policy 1, policy_version 647867 (0.0008) [2023-12-27 10:07:02,634][20381] Updated weights for policy 1, policy_version 647877 (0.0008) [2023-12-27 10:07:02,679][20381] Updated weights for policy 1, policy_version 647887 (0.0007) [2023-12-27 10:07:02,716][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-27 10:07:02,801][20347] Updated weights for policy 0, policy_version 648470 (0.0005) [2023-12-27 10:07:02,858][20347] Updated weights for policy 0, policy_version 648480 (0.0009) [2023-12-27 10:07:02,916][20347] Updated weights for policy 0, policy_version 648490 (0.0007) [2023-12-27 10:07:03,401][20381] Updated weights for policy 1, policy_version 647897 (0.0007) [2023-12-27 10:07:03,463][20381] Updated weights for policy 1, policy_version 647907 (0.0007) [2023-12-27 10:07:03,513][20381] Updated weights for policy 1, policy_version 647917 (0.0007) [2023-12-27 10:07:03,536][20292] KL-divergence is very high: 178.6366 [2023-12-27 10:07:03,543][20292] KL-divergence is very high: 151.7354 [2023-12-27 10:07:03,559][20292] KL-divergence is very high: 100.0751 [2023-12-27 10:07:03,564][20381] Updated weights for policy 1, policy_version 647927 (0.0009) [2023-12-27 10:07:03,606][20347] Updated weights for policy 0, policy_version 648500 (0.0007) [2023-12-27 10:07:03,651][20347] Updated weights for policy 0, policy_version 648510 (0.0005) [2023-12-27 10:07:03,696][20347] Updated weights for policy 0, policy_version 648520 (0.0005) [2023-12-27 10:07:04,148][20292] KL-divergence is very high: 304.5161 [2023-12-27 10:07:04,170][20381] Updated weights for policy 1, policy_version 647937 (0.0007) [2023-12-27 10:07:04,224][20381] Updated weights for policy 1, policy_version 647947 (0.0007) [2023-12-27 10:07:04,237][20292] KL-divergence is very high: 127.7510 [2023-12-27 10:07:04,288][20381] Updated weights for policy 1, policy_version 647957 (0.0007) [2023-12-27 10:07:04,324][20347] Updated weights for policy 0, policy_version 648530 (0.0006) [2023-12-27 10:07:04,379][20347] Updated weights for policy 0, policy_version 648540 (0.0010) [2023-12-27 10:07:04,440][20347] Updated weights for policy 0, policy_version 648550 (0.0010) [2023-12-27 10:07:04,493][20347] Updated weights for policy 0, policy_version 648560 (0.0010) [2023-12-27 10:07:04,901][20292] KL-divergence is very high: 160.6700 [2023-12-27 10:07:04,931][20381] Updated weights for policy 1, policy_version 647967 (0.0007) [2023-12-27 10:07:04,988][20381] Updated weights for policy 1, policy_version 647977 (0.0008) [2023-12-27 10:07:05,040][20381] Updated weights for policy 1, policy_version 647987 (0.0007) [2023-12-27 10:07:05,183][20347] Updated weights for policy 0, policy_version 648570 (0.0007) [2023-12-27 10:07:05,244][20347] Updated weights for policy 0, policy_version 648580 (0.0008) [2023-12-27 10:07:05,306][20347] Updated weights for policy 0, policy_version 648590 (0.0007) [2023-12-27 10:07:05,701][20381] Updated weights for policy 1, policy_version 647997 (0.0007) [2023-12-27 10:07:05,754][20381] Updated weights for policy 1, policy_version 648007 (0.0009) [2023-12-27 10:07:05,805][20381] Updated weights for policy 1, policy_version 648017 (0.0007) [2023-12-27 10:07:05,977][20347] Updated weights for policy 0, policy_version 648600 (0.0008) [2023-12-27 10:07:06,029][20347] Updated weights for policy 0, policy_version 648610 (0.0008) [2023-12-27 10:07:06,091][20347] Updated weights for policy 0, policy_version 648620 (0.0008) [2023-12-27 10:07:06,478][20381] Updated weights for policy 1, policy_version 648027 (0.0008) [2023-12-27 10:07:06,545][20381] Updated weights for policy 1, policy_version 648037 (0.0008) [2023-12-27 10:07:06,608][20381] Updated weights for policy 1, policy_version 648047 (0.0008) [2023-12-27 10:07:06,793][20347] Updated weights for policy 0, policy_version 648630 (0.0007) [2023-12-27 10:07:06,842][20347] Updated weights for policy 0, policy_version 648640 (0.0008) [2023-12-27 10:07:06,890][20347] Updated weights for policy 0, policy_version 648650 (0.0007) [2023-12-27 10:07:07,306][20381] Updated weights for policy 1, policy_version 648057 (0.0008) [2023-12-27 10:07:07,355][20381] Updated weights for policy 1, policy_version 648067 (0.0008) [2023-12-27 10:07:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20660.5). Total num frames: 332005376. Throughput: 0: 10459.5, 1: 10278.0. Samples: 331979356. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:07,363][19170] Avg episode reward: [(0, '941.940'), (1, '863.800')] [2023-12-27 10:07:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000648656_166076416.pth... [2023-12-27 10:07:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000647456_165765120.pth [2023-12-27 10:07:07,410][20381] Updated weights for policy 1, policy_version 648077 (0.0008) [2023-12-27 10:07:07,474][20381] Updated weights for policy 1, policy_version 648087 (0.0007) [2023-12-27 10:07:07,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000648088_165937152.pth... [2023-12-27 10:07:07,481][20347] Updated weights for policy 0, policy_version 648660 (0.0008) [2023-12-27 10:07:07,482][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000646872_165625856.pth [2023-12-27 10:07:07,539][20347] Updated weights for policy 0, policy_version 648670 (0.0007) [2023-12-27 10:07:07,594][20347] Updated weights for policy 0, policy_version 648680 (0.0008) [2023-12-27 10:07:08,194][20347] Updated weights for policy 0, policy_version 648690 (0.0007) [2023-12-27 10:07:08,230][20381] Updated weights for policy 1, policy_version 648097 (0.0008) [2023-12-27 10:07:08,248][20347] Updated weights for policy 0, policy_version 648700 (0.0006) [2023-12-27 10:07:08,286][20381] Updated weights for policy 1, policy_version 648107 (0.0008) [2023-12-27 10:07:08,300][20347] Updated weights for policy 0, policy_version 648710 (0.0008) [2023-12-27 10:07:08,348][20381] Updated weights for policy 1, policy_version 648117 (0.0007) [2023-12-27 10:07:08,360][20347] Updated weights for policy 0, policy_version 648720 (0.0008) [2023-12-27 10:07:08,976][20347] Updated weights for policy 0, policy_version 648730 (0.0009) [2023-12-27 10:07:09,035][20347] Updated weights for policy 0, policy_version 648740 (0.0010) [2023-12-27 10:07:09,064][20381] Updated weights for policy 1, policy_version 648127 (0.0006) [2023-12-27 10:07:09,085][20347] Updated weights for policy 0, policy_version 648750 (0.0009) [2023-12-27 10:07:09,117][20381] Updated weights for policy 1, policy_version 648137 (0.0007) [2023-12-27 10:07:09,169][20381] Updated weights for policy 1, policy_version 648147 (0.0008) [2023-12-27 10:07:09,826][20347] Updated weights for policy 0, policy_version 648760 (0.0008) [2023-12-27 10:07:09,887][20347] Updated weights for policy 0, policy_version 648770 (0.0007) [2023-12-27 10:07:09,895][20381] Updated weights for policy 1, policy_version 648157 (0.0007) [2023-12-27 10:07:09,947][20347] Updated weights for policy 0, policy_version 648780 (0.0007) [2023-12-27 10:07:09,958][20381] Updated weights for policy 1, policy_version 648167 (0.0007) [2023-12-27 10:07:10,024][20381] Updated weights for policy 1, policy_version 648177 (0.0007) [2023-12-27 10:07:10,682][20347] Updated weights for policy 0, policy_version 648790 (0.0007) [2023-12-27 10:07:10,695][20381] Updated weights for policy 1, policy_version 648187 (0.0006) [2023-12-27 10:07:10,708][20291] KL-divergence is very high: 129.6628 [2023-12-27 10:07:10,742][20347] Updated weights for policy 0, policy_version 648800 (0.0006) [2023-12-27 10:07:10,748][20381] Updated weights for policy 1, policy_version 648197 (0.0005) [2023-12-27 10:07:10,800][20291] KL-divergence is very high: 196.2208 [2023-12-27 10:07:10,801][20347] Updated weights for policy 0, policy_version 648810 (0.0006) [2023-12-27 10:07:10,806][20381] Updated weights for policy 1, policy_version 648207 (0.0006) [2023-12-27 10:07:10,828][20291] KL-divergence is very high: 168.8405 [2023-12-27 10:07:11,476][20291] KL-divergence is very high: 108.7205 [2023-12-27 10:07:11,492][20347] Updated weights for policy 0, policy_version 648820 (0.0006) [2023-12-27 10:07:11,494][20291] KL-divergence is very high: 207.6381 [2023-12-27 10:07:11,506][20291] KL-divergence is very high: 157.5923 [2023-12-27 10:07:11,559][20347] Updated weights for policy 0, policy_version 648830 (0.0007) [2023-12-27 10:07:11,569][20381] Updated weights for policy 1, policy_version 648217 (0.0006) [2023-12-27 10:07:11,628][20347] Updated weights for policy 0, policy_version 648840 (0.0008) [2023-12-27 10:07:11,629][20381] Updated weights for policy 1, policy_version 648227 (0.0008) [2023-12-27 10:07:11,695][20381] Updated weights for policy 1, policy_version 648237 (0.0007) [2023-12-27 10:07:11,759][20381] Updated weights for policy 1, policy_version 648247 (0.0008) [2023-12-27 10:07:12,308][20347] Updated weights for policy 0, policy_version 648850 (0.0007) [2023-12-27 10:07:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 332103680. Throughput: 0: 10434.5, 1: 10308.1. Samples: 332101292. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:12,363][19170] Avg episode reward: [(0, '883.800'), (1, '835.250')] [2023-12-27 10:07:12,371][20347] Updated weights for policy 0, policy_version 648860 (0.0009) [2023-12-27 10:07:12,431][20347] Updated weights for policy 0, policy_version 648870 (0.0007) [2023-12-27 10:07:12,486][20347] Updated weights for policy 0, policy_version 648880 (0.0006) [2023-12-27 10:07:12,503][20381] Updated weights for policy 1, policy_version 648257 (0.0007) [2023-12-27 10:07:12,553][20381] Updated weights for policy 1, policy_version 648267 (0.0007) [2023-12-27 10:07:12,610][20381] Updated weights for policy 1, policy_version 648277 (0.0008) [2023-12-27 10:07:13,127][20347] Updated weights for policy 0, policy_version 648890 (0.0007) [2023-12-27 10:07:13,173][20291] KL-divergence is very high: 161.6842 [2023-12-27 10:07:13,175][20347] Updated weights for policy 0, policy_version 648900 (0.0007) [2023-12-27 10:07:13,223][20347] Updated weights for policy 0, policy_version 648910 (0.0006) [2023-12-27 10:07:13,328][20381] Updated weights for policy 1, policy_version 648287 (0.0007) [2023-12-27 10:07:13,381][20381] Updated weights for policy 1, policy_version 648297 (0.0008) [2023-12-27 10:07:13,441][20381] Updated weights for policy 1, policy_version 648307 (0.0007) [2023-12-27 10:07:13,945][20347] Updated weights for policy 0, policy_version 648920 (0.0007) [2023-12-27 10:07:14,006][20347] Updated weights for policy 0, policy_version 648930 (0.0007) [2023-12-27 10:07:14,064][20347] Updated weights for policy 0, policy_version 648940 (0.0007) [2023-12-27 10:07:14,086][20381] Updated weights for policy 1, policy_version 648317 (0.0007) [2023-12-27 10:07:14,137][20381] Updated weights for policy 1, policy_version 648327 (0.0008) [2023-12-27 10:07:14,186][20381] Updated weights for policy 1, policy_version 648337 (0.0007) [2023-12-27 10:07:14,762][20291] KL-divergence is very high: 130.5442 [2023-12-27 10:07:14,780][20347] Updated weights for policy 0, policy_version 648950 (0.0007) [2023-12-27 10:07:14,799][20381] Updated weights for policy 1, policy_version 648347 (0.0007) [2023-12-27 10:07:14,808][20291] KL-divergence is very high: 139.8963 [2023-12-27 10:07:14,839][20347] Updated weights for policy 0, policy_version 648960 (0.0008) [2023-12-27 10:07:14,849][20381] Updated weights for policy 1, policy_version 648357 (0.0007) [2023-12-27 10:07:14,901][20347] Updated weights for policy 0, policy_version 648970 (0.0006) [2023-12-27 10:07:14,903][20381] Updated weights for policy 1, policy_version 648367 (0.0007) [2023-12-27 10:07:15,590][20347] Updated weights for policy 0, policy_version 648980 (0.0008) [2023-12-27 10:07:15,611][20381] Updated weights for policy 1, policy_version 648377 (0.0007) [2023-12-27 10:07:15,643][20347] Updated weights for policy 0, policy_version 648990 (0.0006) [2023-12-27 10:07:15,667][20381] Updated weights for policy 1, policy_version 648387 (0.0006) [2023-12-27 10:07:15,694][20347] Updated weights for policy 0, policy_version 649000 (0.0006) [2023-12-27 10:07:15,719][20381] Updated weights for policy 1, policy_version 648397 (0.0005) [2023-12-27 10:07:15,773][20381] Updated weights for policy 1, policy_version 648407 (0.0007) [2023-12-27 10:07:16,382][20347] Updated weights for policy 0, policy_version 649010 (0.0006) [2023-12-27 10:07:16,441][20347] Updated weights for policy 0, policy_version 649020 (0.0005) [2023-12-27 10:07:16,495][20381] Updated weights for policy 1, policy_version 648417 (0.0008) [2023-12-27 10:07:16,495][20347] Updated weights for policy 0, policy_version 649030 (0.0005) [2023-12-27 10:07:16,546][20347] Updated weights for policy 0, policy_version 649040 (0.0005) [2023-12-27 10:07:16,551][20381] Updated weights for policy 1, policy_version 648427 (0.0007) [2023-12-27 10:07:16,604][20381] Updated weights for policy 1, policy_version 648437 (0.0009) [2023-12-27 10:07:17,087][20347] Updated weights for policy 0, policy_version 649050 (0.0005) [2023-12-27 10:07:17,145][20347] Updated weights for policy 0, policy_version 649060 (0.0005) [2023-12-27 10:07:17,207][20347] Updated weights for policy 0, policy_version 649070 (0.0006) [2023-12-27 10:07:17,355][20381] Updated weights for policy 1, policy_version 648447 (0.0009) [2023-12-27 10:07:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 332210176. Throughput: 0: 10332.3, 1: 10260.5. Samples: 332222608. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:17,363][19170] Avg episode reward: [(0, '778.460'), (1, '826.930')] [2023-12-27 10:07:17,411][20381] Updated weights for policy 1, policy_version 648457 (0.0008) [2023-12-27 10:07:17,462][20381] Updated weights for policy 1, policy_version 648467 (0.0009) [2023-12-27 10:07:17,773][20347] Updated weights for policy 0, policy_version 649080 (0.0006) [2023-12-27 10:07:17,827][20347] Updated weights for policy 0, policy_version 649090 (0.0005) [2023-12-27 10:07:17,889][20347] Updated weights for policy 0, policy_version 649100 (0.0008) [2023-12-27 10:07:18,159][20381] Updated weights for policy 1, policy_version 648477 (0.0007) [2023-12-27 10:07:18,222][20381] Updated weights for policy 1, policy_version 648487 (0.0008) [2023-12-27 10:07:18,289][20381] Updated weights for policy 1, policy_version 648497 (0.0008) [2023-12-27 10:07:18,642][20347] Updated weights for policy 0, policy_version 649110 (0.0009) [2023-12-27 10:07:18,705][20347] Updated weights for policy 0, policy_version 649120 (0.0006) [2023-12-27 10:07:18,765][20347] Updated weights for policy 0, policy_version 649130 (0.0008) [2023-12-27 10:07:18,897][20381] Updated weights for policy 1, policy_version 648507 (0.0006) [2023-12-27 10:07:18,962][20381] Updated weights for policy 1, policy_version 648517 (0.0006) [2023-12-27 10:07:19,022][20381] Updated weights for policy 1, policy_version 648527 (0.0007) [2023-12-27 10:07:19,490][20347] Updated weights for policy 0, policy_version 649140 (0.0008) [2023-12-27 10:07:19,552][20347] Updated weights for policy 0, policy_version 649150 (0.0008) [2023-12-27 10:07:19,609][20347] Updated weights for policy 0, policy_version 649160 (0.0007) [2023-12-27 10:07:19,672][20381] Updated weights for policy 1, policy_version 648537 (0.0006) [2023-12-27 10:07:19,738][20381] Updated weights for policy 1, policy_version 648547 (0.0008) [2023-12-27 10:07:19,798][20381] Updated weights for policy 1, policy_version 648557 (0.0008) [2023-12-27 10:07:19,862][20381] Updated weights for policy 1, policy_version 648567 (0.0008) [2023-12-27 10:07:20,285][20347] Updated weights for policy 0, policy_version 649170 (0.0007) [2023-12-27 10:07:20,338][20347] Updated weights for policy 0, policy_version 649180 (0.0008) [2023-12-27 10:07:20,401][20347] Updated weights for policy 0, policy_version 649190 (0.0007) [2023-12-27 10:07:20,457][20347] Updated weights for policy 0, policy_version 649200 (0.0007) [2023-12-27 10:07:20,525][20381] Updated weights for policy 1, policy_version 648577 (0.0008) [2023-12-27 10:07:20,586][20381] Updated weights for policy 1, policy_version 648587 (0.0007) [2023-12-27 10:07:20,643][20381] Updated weights for policy 1, policy_version 648597 (0.0007) [2023-12-27 10:07:21,138][20347] Updated weights for policy 0, policy_version 649210 (0.0009) [2023-12-27 10:07:21,198][20347] Updated weights for policy 0, policy_version 649220 (0.0008) [2023-12-27 10:07:21,260][20347] Updated weights for policy 0, policy_version 649230 (0.0009) [2023-12-27 10:07:21,292][20381] Updated weights for policy 1, policy_version 648607 (0.0008) [2023-12-27 10:07:21,350][20381] Updated weights for policy 1, policy_version 648617 (0.0006) [2023-12-27 10:07:21,420][20381] Updated weights for policy 1, policy_version 648627 (0.0007) [2023-12-27 10:07:21,966][20347] Updated weights for policy 0, policy_version 649240 (0.0007) [2023-12-27 10:07:22,024][20347] Updated weights for policy 0, policy_version 649250 (0.0005) [2023-12-27 10:07:22,085][20347] Updated weights for policy 0, policy_version 649260 (0.0006) [2023-12-27 10:07:22,107][20381] Updated weights for policy 1, policy_version 648637 (0.0006) [2023-12-27 10:07:22,171][20381] Updated weights for policy 1, policy_version 648647 (0.0007) [2023-12-27 10:07:22,231][20381] Updated weights for policy 1, policy_version 648657 (0.0008) [2023-12-27 10:07:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 332316672. Throughput: 0: 10307.4, 1: 10208.9. Samples: 332284368. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:22,363][19170] Avg episode reward: [(0, '645.900'), (1, '815.650')] [2023-12-27 10:07:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000648664_166084608.pth... [2023-12-27 10:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000649264_166232064.pth... [2023-12-27 10:07:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000647480_165781504.pth [2023-12-27 10:07:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000648064_165920768.pth [2023-12-27 10:07:22,753][20347] Updated weights for policy 0, policy_version 649270 (0.0007) [2023-12-27 10:07:22,799][20347] Updated weights for policy 0, policy_version 649280 (0.0007) [2023-12-27 10:07:22,848][20347] Updated weights for policy 0, policy_version 649290 (0.0008) [2023-12-27 10:07:22,944][20381] Updated weights for policy 1, policy_version 648667 (0.0008) [2023-12-27 10:07:23,000][20381] Updated weights for policy 1, policy_version 648677 (0.0007) [2023-12-27 10:07:23,052][20381] Updated weights for policy 1, policy_version 648687 (0.0008) [2023-12-27 10:07:23,528][20347] Updated weights for policy 0, policy_version 649300 (0.0007) [2023-12-27 10:07:23,586][20347] Updated weights for policy 0, policy_version 649310 (0.0005) [2023-12-27 10:07:23,647][20347] Updated weights for policy 0, policy_version 649320 (0.0008) [2023-12-27 10:07:23,675][20381] Updated weights for policy 1, policy_version 648697 (0.0007) [2023-12-27 10:07:23,723][20381] Updated weights for policy 1, policy_version 648707 (0.0007) [2023-12-27 10:07:23,769][20381] Updated weights for policy 1, policy_version 648717 (0.0008) [2023-12-27 10:07:23,818][20381] Updated weights for policy 1, policy_version 648727 (0.0008) [2023-12-27 10:07:24,265][20347] Updated weights for policy 0, policy_version 649330 (0.0007) [2023-12-27 10:07:24,320][20347] Updated weights for policy 0, policy_version 649340 (0.0008) [2023-12-27 10:07:24,371][20347] Updated weights for policy 0, policy_version 649350 (0.0007) [2023-12-27 10:07:24,419][20347] Updated weights for policy 0, policy_version 649360 (0.0007) [2023-12-27 10:07:24,563][20381] Updated weights for policy 1, policy_version 648737 (0.0008) [2023-12-27 10:07:24,627][20381] Updated weights for policy 1, policy_version 648747 (0.0009) [2023-12-27 10:07:24,686][20381] Updated weights for policy 1, policy_version 648757 (0.0008) [2023-12-27 10:07:25,022][20347] Updated weights for policy 0, policy_version 649370 (0.0005) [2023-12-27 10:07:25,079][20347] Updated weights for policy 0, policy_version 649380 (0.0006) [2023-12-27 10:07:25,138][20347] Updated weights for policy 0, policy_version 649390 (0.0007) [2023-12-27 10:07:25,383][20381] Updated weights for policy 1, policy_version 648767 (0.0007) [2023-12-27 10:07:25,432][20381] Updated weights for policy 1, policy_version 648777 (0.0010) [2023-12-27 10:07:25,483][20381] Updated weights for policy 1, policy_version 648787 (0.0006) [2023-12-27 10:07:25,734][20347] Updated weights for policy 0, policy_version 649400 (0.0006) [2023-12-27 10:07:25,790][20347] Updated weights for policy 0, policy_version 649410 (0.0005) [2023-12-27 10:07:25,845][20347] Updated weights for policy 0, policy_version 649420 (0.0005) [2023-12-27 10:07:26,100][20381] Updated weights for policy 1, policy_version 648797 (0.0006) [2023-12-27 10:07:26,151][20381] Updated weights for policy 1, policy_version 648807 (0.0007) [2023-12-27 10:07:26,202][20381] Updated weights for policy 1, policy_version 648817 (0.0008) [2023-12-27 10:07:26,364][20347] Updated weights for policy 0, policy_version 649430 (0.0005) [2023-12-27 10:07:26,416][20347] Updated weights for policy 0, policy_version 649440 (0.0005) [2023-12-27 10:07:26,460][20347] Updated weights for policy 0, policy_version 649450 (0.0005) [2023-12-27 10:07:26,955][20381] Updated weights for policy 1, policy_version 648827 (0.0009) [2023-12-27 10:07:27,010][20381] Updated weights for policy 1, policy_version 648837 (0.0007) [2023-12-27 10:07:27,020][20347] Updated weights for policy 0, policy_version 649460 (0.0005) [2023-12-27 10:07:27,070][20381] Updated weights for policy 1, policy_version 648847 (0.0005) [2023-12-27 10:07:27,075][20347] Updated weights for policy 0, policy_version 649470 (0.0007) [2023-12-27 10:07:27,130][20347] Updated weights for policy 0, policy_version 649480 (0.0008) [2023-12-27 10:07:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 332423168. Throughput: 0: 10403.1, 1: 10217.7. Samples: 332409420. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:27,363][19170] Avg episode reward: [(0, '546.300'), (1, '883.800')] [2023-12-27 10:07:27,602][20381] Updated weights for policy 1, policy_version 648857 (0.0005) [2023-12-27 10:07:27,658][20381] Updated weights for policy 1, policy_version 648867 (0.0006) [2023-12-27 10:07:27,699][20347] Updated weights for policy 0, policy_version 649490 (0.0008) [2023-12-27 10:07:27,702][20381] Updated weights for policy 1, policy_version 648877 (0.0007) [2023-12-27 10:07:27,747][20381] Updated weights for policy 1, policy_version 648887 (0.0005) [2023-12-27 10:07:27,754][20347] Updated weights for policy 0, policy_version 649500 (0.0008) [2023-12-27 10:07:27,819][20347] Updated weights for policy 0, policy_version 649510 (0.0007) [2023-12-27 10:07:27,880][20347] Updated weights for policy 0, policy_version 649520 (0.0008) [2023-12-27 10:07:28,368][20381] Updated weights for policy 1, policy_version 648897 (0.0007) [2023-12-27 10:07:28,427][20381] Updated weights for policy 1, policy_version 648907 (0.0007) [2023-12-27 10:07:28,484][20381] Updated weights for policy 1, policy_version 648917 (0.0007) [2023-12-27 10:07:28,574][20347] Updated weights for policy 0, policy_version 649530 (0.0010) [2023-12-27 10:07:28,625][20347] Updated weights for policy 0, policy_version 649540 (0.0010) [2023-12-27 10:07:28,674][20347] Updated weights for policy 0, policy_version 649550 (0.0010) [2023-12-27 10:07:29,107][20381] Updated weights for policy 1, policy_version 648927 (0.0009) [2023-12-27 10:07:29,152][20381] Updated weights for policy 1, policy_version 648937 (0.0010) [2023-12-27 10:07:29,206][20381] Updated weights for policy 1, policy_version 648947 (0.0006) [2023-12-27 10:07:29,356][20347] Updated weights for policy 0, policy_version 649560 (0.0010) [2023-12-27 10:07:29,419][20347] Updated weights for policy 0, policy_version 649570 (0.0010) [2023-12-27 10:07:29,423][20291] KL-divergence is very high: 123.2104 [2023-12-27 10:07:29,467][20347] Updated weights for policy 0, policy_version 649580 (0.0010) [2023-12-27 10:07:29,800][20381] Updated weights for policy 1, policy_version 648957 (0.0007) [2023-12-27 10:07:29,858][20381] Updated weights for policy 1, policy_version 648967 (0.0008) [2023-12-27 10:07:29,906][20381] Updated weights for policy 1, policy_version 648977 (0.0005) [2023-12-27 10:07:30,148][20347] Updated weights for policy 0, policy_version 649590 (0.0010) [2023-12-27 10:07:30,212][20347] Updated weights for policy 0, policy_version 649600 (0.0010) [2023-12-27 10:07:30,279][20347] Updated weights for policy 0, policy_version 649610 (0.0010) [2023-12-27 10:07:30,522][20381] Updated weights for policy 1, policy_version 648987 (0.0008) [2023-12-27 10:07:30,576][20381] Updated weights for policy 1, policy_version 648997 (0.0008) [2023-12-27 10:07:30,624][20381] Updated weights for policy 1, policy_version 649007 (0.0007) [2023-12-27 10:07:30,915][20347] Updated weights for policy 0, policy_version 649620 (0.0010) [2023-12-27 10:07:30,962][20347] Updated weights for policy 0, policy_version 649630 (0.0010) [2023-12-27 10:07:31,009][20347] Updated weights for policy 0, policy_version 649640 (0.0010) [2023-12-27 10:07:31,326][20381] Updated weights for policy 1, policy_version 649017 (0.0007) [2023-12-27 10:07:31,388][20381] Updated weights for policy 1, policy_version 649027 (0.0007) [2023-12-27 10:07:31,449][20381] Updated weights for policy 1, policy_version 649037 (0.0008) [2023-12-27 10:07:31,498][20381] Updated weights for policy 1, policy_version 649047 (0.0006) [2023-12-27 10:07:31,734][20347] Updated weights for policy 0, policy_version 649650 (0.0012) [2023-12-27 10:07:31,787][20347] Updated weights for policy 0, policy_version 649660 (0.0010) [2023-12-27 10:07:31,842][20347] Updated weights for policy 0, policy_version 649670 (0.0007) [2023-12-27 10:07:31,905][20347] Updated weights for policy 0, policy_version 649680 (0.0008) [2023-12-27 10:07:32,195][20381] Updated weights for policy 1, policy_version 649057 (0.0008) [2023-12-27 10:07:32,247][20381] Updated weights for policy 1, policy_version 649067 (0.0008) [2023-12-27 10:07:32,302][20381] Updated weights for policy 1, policy_version 649077 (0.0008) [2023-12-27 10:07:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 332529664. Throughput: 0: 10463.3, 1: 10302.5. Samples: 332538620. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:32,363][19170] Avg episode reward: [(0, '226.100'), (1, '814.780')] [2023-12-27 10:07:32,499][20347] Updated weights for policy 0, policy_version 649690 (0.0006) [2023-12-27 10:07:32,552][20347] Updated weights for policy 0, policy_version 649700 (0.0006) [2023-12-27 10:07:32,607][20347] Updated weights for policy 0, policy_version 649710 (0.0008) [2023-12-27 10:07:32,913][20381] Updated weights for policy 1, policy_version 649087 (0.0006) [2023-12-27 10:07:32,964][20381] Updated weights for policy 1, policy_version 649097 (0.0005) [2023-12-27 10:07:33,024][20381] Updated weights for policy 1, policy_version 649107 (0.0005) [2023-12-27 10:07:33,322][20347] Updated weights for policy 0, policy_version 649720 (0.0009) [2023-12-27 10:07:33,374][20347] Updated weights for policy 0, policy_version 649730 (0.0010) [2023-12-27 10:07:33,431][20347] Updated weights for policy 0, policy_version 649740 (0.0010) [2023-12-27 10:07:33,608][20381] Updated weights for policy 1, policy_version 649117 (0.0006) [2023-12-27 10:07:33,664][20381] Updated weights for policy 1, policy_version 649127 (0.0007) [2023-12-27 10:07:33,729][20381] Updated weights for policy 1, policy_version 649137 (0.0008) [2023-12-27 10:07:34,012][20347] Updated weights for policy 0, policy_version 649750 (0.0007) [2023-12-27 10:07:34,065][20347] Updated weights for policy 0, policy_version 649760 (0.0008) [2023-12-27 10:07:34,119][20347] Updated weights for policy 0, policy_version 649770 (0.0010) [2023-12-27 10:07:34,404][20381] Updated weights for policy 1, policy_version 649147 (0.0008) [2023-12-27 10:07:34,455][20381] Updated weights for policy 1, policy_version 649157 (0.0008) [2023-12-27 10:07:34,504][20381] Updated weights for policy 1, policy_version 649167 (0.0008) [2023-12-27 10:07:34,797][20347] Updated weights for policy 0, policy_version 649780 (0.0009) [2023-12-27 10:07:34,855][20347] Updated weights for policy 0, policy_version 649790 (0.0010) [2023-12-27 10:07:34,909][20347] Updated weights for policy 0, policy_version 649800 (0.0010) [2023-12-27 10:07:35,145][20381] Updated weights for policy 1, policy_version 649177 (0.0007) [2023-12-27 10:07:35,197][20381] Updated weights for policy 1, policy_version 649187 (0.0009) [2023-12-27 10:07:35,248][20381] Updated weights for policy 1, policy_version 649197 (0.0010) [2023-12-27 10:07:35,303][20381] Updated weights for policy 1, policy_version 649207 (0.0008) [2023-12-27 10:07:35,488][20347] Updated weights for policy 0, policy_version 649810 (0.0010) [2023-12-27 10:07:35,547][20347] Updated weights for policy 0, policy_version 649820 (0.0010) [2023-12-27 10:07:35,599][20347] Updated weights for policy 0, policy_version 649830 (0.0010) [2023-12-27 10:07:35,648][20347] Updated weights for policy 0, policy_version 649840 (0.0007) [2023-12-27 10:07:36,048][20381] Updated weights for policy 1, policy_version 649217 (0.0007) [2023-12-27 10:07:36,111][20381] Updated weights for policy 1, policy_version 649227 (0.0008) [2023-12-27 10:07:36,170][20381] Updated weights for policy 1, policy_version 649237 (0.0007) [2023-12-27 10:07:36,252][20347] Updated weights for policy 0, policy_version 649850 (0.0010) [2023-12-27 10:07:36,305][20347] Updated weights for policy 0, policy_version 649860 (0.0010) [2023-12-27 10:07:36,363][20347] Updated weights for policy 0, policy_version 649870 (0.0010) [2023-12-27 10:07:36,892][20381] Updated weights for policy 1, policy_version 649247 (0.0007) [2023-12-27 10:07:36,947][20381] Updated weights for policy 1, policy_version 649257 (0.0007) [2023-12-27 10:07:37,009][20381] Updated weights for policy 1, policy_version 649267 (0.0007) [2023-12-27 10:07:37,077][20347] Updated weights for policy 0, policy_version 649880 (0.0008) [2023-12-27 10:07:37,136][20347] Updated weights for policy 0, policy_version 649890 (0.0006) [2023-12-27 10:07:37,190][20347] Updated weights for policy 0, policy_version 649900 (0.0006) [2023-12-27 10:07:37,362][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 332636160. Throughput: 0: 10470.8, 1: 10344.2. Samples: 332603460. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:37,363][19170] Avg episode reward: [(0, '91.240'), (1, '863.500')] [2023-12-27 10:07:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000649272_166240256.pth... [2023-12-27 10:07:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000649904_166395904.pth... [2023-12-27 10:07:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000648088_165937152.pth [2023-12-27 10:07:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000648656_166076416.pth [2023-12-27 10:07:37,703][20381] Updated weights for policy 1, policy_version 649277 (0.0009) [2023-12-27 10:07:37,757][20381] Updated weights for policy 1, policy_version 649287 (0.0010) [2023-12-27 10:07:37,775][20347] Updated weights for policy 0, policy_version 649910 (0.0007) [2023-12-27 10:07:37,805][20381] Updated weights for policy 1, policy_version 649297 (0.0010) [2023-12-27 10:07:37,842][20347] Updated weights for policy 0, policy_version 649920 (0.0007) [2023-12-27 10:07:37,890][20347] Updated weights for policy 0, policy_version 649930 (0.0005) [2023-12-27 10:07:38,436][20381] Updated weights for policy 1, policy_version 649307 (0.0005) [2023-12-27 10:07:38,452][20347] Updated weights for policy 0, policy_version 649940 (0.0006) [2023-12-27 10:07:38,493][20381] Updated weights for policy 1, policy_version 649317 (0.0010) [2023-12-27 10:07:38,510][20347] Updated weights for policy 0, policy_version 649950 (0.0009) [2023-12-27 10:07:38,547][20381] Updated weights for policy 1, policy_version 649327 (0.0010) [2023-12-27 10:07:38,569][20347] Updated weights for policy 0, policy_version 649960 (0.0010) [2023-12-27 10:07:39,230][20381] Updated weights for policy 1, policy_version 649337 (0.0011) [2023-12-27 10:07:39,251][20347] Updated weights for policy 0, policy_version 649970 (0.0009) [2023-12-27 10:07:39,291][20381] Updated weights for policy 1, policy_version 649347 (0.0008) [2023-12-27 10:07:39,312][20347] Updated weights for policy 0, policy_version 649980 (0.0008) [2023-12-27 10:07:39,360][20381] Updated weights for policy 1, policy_version 649357 (0.0010) [2023-12-27 10:07:39,376][20347] Updated weights for policy 0, policy_version 649990 (0.0009) [2023-12-27 10:07:39,418][20381] Updated weights for policy 1, policy_version 649367 (0.0009) [2023-12-27 10:07:39,438][20347] Updated weights for policy 0, policy_version 650000 (0.0007) [2023-12-27 10:07:40,051][20381] Updated weights for policy 1, policy_version 649377 (0.0006) [2023-12-27 10:07:40,100][20347] Updated weights for policy 0, policy_version 650010 (0.0007) [2023-12-27 10:07:40,109][20381] Updated weights for policy 1, policy_version 649387 (0.0006) [2023-12-27 10:07:40,159][20347] Updated weights for policy 0, policy_version 650020 (0.0007) [2023-12-27 10:07:40,166][20381] Updated weights for policy 1, policy_version 649397 (0.0006) [2023-12-27 10:07:40,214][20347] Updated weights for policy 0, policy_version 650030 (0.0008) [2023-12-27 10:07:40,740][20381] Updated weights for policy 1, policy_version 649407 (0.0006) [2023-12-27 10:07:40,786][20381] Updated weights for policy 1, policy_version 649417 (0.0010) [2023-12-27 10:07:40,839][20381] Updated weights for policy 1, policy_version 649427 (0.0007) [2023-12-27 10:07:40,902][20347] Updated weights for policy 0, policy_version 650040 (0.0006) [2023-12-27 10:07:40,968][20347] Updated weights for policy 0, policy_version 650050 (0.0007) [2023-12-27 10:07:41,019][20347] Updated weights for policy 0, policy_version 650060 (0.0007) [2023-12-27 10:07:41,509][20381] Updated weights for policy 1, policy_version 649437 (0.0008) [2023-12-27 10:07:41,565][20381] Updated weights for policy 1, policy_version 649447 (0.0010) [2023-12-27 10:07:41,632][20381] Updated weights for policy 1, policy_version 649457 (0.0011) [2023-12-27 10:07:41,777][20347] Updated weights for policy 0, policy_version 650070 (0.0010) [2023-12-27 10:07:41,839][20347] Updated weights for policy 0, policy_version 650080 (0.0010) [2023-12-27 10:07:41,900][20347] Updated weights for policy 0, policy_version 650090 (0.0010) [2023-12-27 10:07:42,357][20381] Updated weights for policy 1, policy_version 649467 (0.0010) [2023-12-27 10:07:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 332734464. Throughput: 0: 10534.6, 1: 10296.4. Samples: 332730300. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:42,363][19170] Avg episode reward: [(0, '201.680'), (1, '864.520')] [2023-12-27 10:07:42,422][20381] Updated weights for policy 1, policy_version 649477 (0.0009) [2023-12-27 10:07:42,481][20381] Updated weights for policy 1, policy_version 649487 (0.0010) [2023-12-27 10:07:42,555][20347] Updated weights for policy 0, policy_version 650100 (0.0008) [2023-12-27 10:07:42,614][20347] Updated weights for policy 0, policy_version 650110 (0.0007) [2023-12-27 10:07:42,664][20347] Updated weights for policy 0, policy_version 650120 (0.0005) [2023-12-27 10:07:43,103][20381] Updated weights for policy 1, policy_version 649497 (0.0011) [2023-12-27 10:07:43,167][20381] Updated weights for policy 1, policy_version 649507 (0.0011) [2023-12-27 10:07:43,229][20381] Updated weights for policy 1, policy_version 649517 (0.0011) [2023-12-27 10:07:43,285][20381] Updated weights for policy 1, policy_version 649527 (0.0010) [2023-12-27 10:07:43,306][20347] Updated weights for policy 0, policy_version 650130 (0.0007) [2023-12-27 10:07:43,356][20347] Updated weights for policy 0, policy_version 650140 (0.0008) [2023-12-27 10:07:43,408][20347] Updated weights for policy 0, policy_version 650150 (0.0006) [2023-12-27 10:07:43,464][20347] Updated weights for policy 0, policy_version 650160 (0.0005) [2023-12-27 10:07:43,946][20381] Updated weights for policy 1, policy_version 649537 (0.0010) [2023-12-27 10:07:44,004][20381] Updated weights for policy 1, policy_version 649547 (0.0010) [2023-12-27 10:07:44,046][20347] Updated weights for policy 0, policy_version 650170 (0.0005) [2023-12-27 10:07:44,059][20381] Updated weights for policy 1, policy_version 649557 (0.0008) [2023-12-27 10:07:44,100][20347] Updated weights for policy 0, policy_version 650180 (0.0006) [2023-12-27 10:07:44,155][20347] Updated weights for policy 0, policy_version 650190 (0.0005) [2023-12-27 10:07:44,698][20381] Updated weights for policy 1, policy_version 649567 (0.0009) [2023-12-27 10:07:44,742][20347] Updated weights for policy 0, policy_version 650200 (0.0007) [2023-12-27 10:07:44,755][20381] Updated weights for policy 1, policy_version 649577 (0.0008) [2023-12-27 10:07:44,803][20347] Updated weights for policy 0, policy_version 650210 (0.0008) [2023-12-27 10:07:44,816][20381] Updated weights for policy 1, policy_version 649587 (0.0009) [2023-12-27 10:07:44,860][20347] Updated weights for policy 0, policy_version 650220 (0.0007) [2023-12-27 10:07:45,443][20381] Updated weights for policy 1, policy_version 649597 (0.0007) [2023-12-27 10:07:45,499][20381] Updated weights for policy 1, policy_version 649607 (0.0008) [2023-12-27 10:07:45,550][20381] Updated weights for policy 1, policy_version 649617 (0.0007) [2023-12-27 10:07:45,585][20347] Updated weights for policy 0, policy_version 650230 (0.0007) [2023-12-27 10:07:45,640][20347] Updated weights for policy 0, policy_version 650240 (0.0008) [2023-12-27 10:07:45,696][20347] Updated weights for policy 0, policy_version 650250 (0.0008) [2023-12-27 10:07:46,238][20381] Updated weights for policy 1, policy_version 649627 (0.0007) [2023-12-27 10:07:46,292][20381] Updated weights for policy 1, policy_version 649637 (0.0007) [2023-12-27 10:07:46,346][20381] Updated weights for policy 1, policy_version 649647 (0.0007) [2023-12-27 10:07:46,355][20347] Updated weights for policy 0, policy_version 650260 (0.0009) [2023-12-27 10:07:46,408][20347] Updated weights for policy 0, policy_version 650270 (0.0009) [2023-12-27 10:07:46,463][20347] Updated weights for policy 0, policy_version 650280 (0.0005) [2023-12-27 10:07:46,943][20381] Updated weights for policy 1, policy_version 649657 (0.0006) [2023-12-27 10:07:47,000][20381] Updated weights for policy 1, policy_version 649667 (0.0008) [2023-12-27 10:07:47,045][20381] Updated weights for policy 1, policy_version 649677 (0.0007) [2023-12-27 10:07:47,095][20381] Updated weights for policy 1, policy_version 649687 (0.0007) [2023-12-27 10:07:47,097][20347] Updated weights for policy 0, policy_version 650290 (0.0006) [2023-12-27 10:07:47,152][20347] Updated weights for policy 0, policy_version 650300 (0.0007) [2023-12-27 10:07:47,207][20347] Updated weights for policy 0, policy_version 650310 (0.0005) [2023-12-27 10:07:47,260][20347] Updated weights for policy 0, policy_version 650320 (0.0005) [2023-12-27 10:07:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 332849152. Throughput: 0: 10553.8, 1: 10345.6. Samples: 332856340. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:47,363][19170] Avg episode reward: [(0, '131.110'), (1, '874.180')] [2023-12-27 10:07:47,781][20347] Updated weights for policy 0, policy_version 650330 (0.0005) [2023-12-27 10:07:47,835][20347] Updated weights for policy 0, policy_version 650340 (0.0005) [2023-12-27 10:07:47,849][20381] Updated weights for policy 1, policy_version 649697 (0.0008) [2023-12-27 10:07:47,891][20347] Updated weights for policy 0, policy_version 650350 (0.0006) [2023-12-27 10:07:47,908][20381] Updated weights for policy 1, policy_version 649707 (0.0009) [2023-12-27 10:07:47,966][20381] Updated weights for policy 1, policy_version 649717 (0.0009) [2023-12-27 10:07:48,462][20347] Updated weights for policy 0, policy_version 650360 (0.0007) [2023-12-27 10:07:48,520][20347] Updated weights for policy 0, policy_version 650370 (0.0008) [2023-12-27 10:07:48,584][20347] Updated weights for policy 0, policy_version 650380 (0.0008) [2023-12-27 10:07:48,701][20381] Updated weights for policy 1, policy_version 649727 (0.0009) [2023-12-27 10:07:48,758][20381] Updated weights for policy 1, policy_version 649737 (0.0007) [2023-12-27 10:07:48,803][20381] Updated weights for policy 1, policy_version 649747 (0.0008) [2023-12-27 10:07:49,282][20347] Updated weights for policy 0, policy_version 650390 (0.0008) [2023-12-27 10:07:49,344][20347] Updated weights for policy 0, policy_version 650400 (0.0007) [2023-12-27 10:07:49,408][20347] Updated weights for policy 0, policy_version 650410 (0.0008) [2023-12-27 10:07:49,533][20381] Updated weights for policy 1, policy_version 649757 (0.0007) [2023-12-27 10:07:49,590][20381] Updated weights for policy 1, policy_version 649767 (0.0007) [2023-12-27 10:07:49,647][20381] Updated weights for policy 1, policy_version 649777 (0.0007) [2023-12-27 10:07:50,115][20347] Updated weights for policy 0, policy_version 650420 (0.0007) [2023-12-27 10:07:50,177][20347] Updated weights for policy 0, policy_version 650430 (0.0007) [2023-12-27 10:07:50,247][20347] Updated weights for policy 0, policy_version 650440 (0.0006) [2023-12-27 10:07:50,382][20381] Updated weights for policy 1, policy_version 649787 (0.0008) [2023-12-27 10:07:50,441][20381] Updated weights for policy 1, policy_version 649797 (0.0009) [2023-12-27 10:07:50,506][20381] Updated weights for policy 1, policy_version 649807 (0.0011) [2023-12-27 10:07:50,896][20347] Updated weights for policy 0, policy_version 650450 (0.0007) [2023-12-27 10:07:50,952][20347] Updated weights for policy 0, policy_version 650460 (0.0009) [2023-12-27 10:07:51,006][20347] Updated weights for policy 0, policy_version 650470 (0.0008) [2023-12-27 10:07:51,060][20347] Updated weights for policy 0, policy_version 650480 (0.0008) [2023-12-27 10:07:51,147][20381] Updated weights for policy 1, policy_version 649817 (0.0010) [2023-12-27 10:07:51,215][20381] Updated weights for policy 1, policy_version 649827 (0.0008) [2023-12-27 10:07:51,274][20381] Updated weights for policy 1, policy_version 649837 (0.0010) [2023-12-27 10:07:51,330][20381] Updated weights for policy 1, policy_version 649847 (0.0011) [2023-12-27 10:07:51,836][20347] Updated weights for policy 0, policy_version 650490 (0.0008) [2023-12-27 10:07:51,892][20347] Updated weights for policy 0, policy_version 650500 (0.0007) [2023-12-27 10:07:51,953][20347] Updated weights for policy 0, policy_version 650510 (0.0006) [2023-12-27 10:07:52,046][20381] Updated weights for policy 1, policy_version 649857 (0.0011) [2023-12-27 10:07:52,105][20381] Updated weights for policy 1, policy_version 649867 (0.0007) [2023-12-27 10:07:52,171][20381] Updated weights for policy 1, policy_version 649877 (0.0006) [2023-12-27 10:07:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 332947456. Throughput: 0: 10566.3, 1: 10290.8. Samples: 332917924. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:52,363][19170] Avg episode reward: [(0, '230.040'), (1, '892.700')] [2023-12-27 10:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000649880_166395904.pth... [2023-12-27 10:07:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000650512_166551552.pth... [2023-12-27 10:07:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000648664_166084608.pth [2023-12-27 10:07:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000649264_166232064.pth [2023-12-27 10:07:52,635][20347] Updated weights for policy 0, policy_version 650520 (0.0007) [2023-12-27 10:07:52,701][20347] Updated weights for policy 0, policy_version 650530 (0.0008) [2023-12-27 10:07:52,764][20347] Updated weights for policy 0, policy_version 650540 (0.0010) [2023-12-27 10:07:52,844][20381] Updated weights for policy 1, policy_version 649887 (0.0008) [2023-12-27 10:07:52,903][20381] Updated weights for policy 1, policy_version 649897 (0.0007) [2023-12-27 10:07:52,962][20381] Updated weights for policy 1, policy_version 649907 (0.0008) [2023-12-27 10:07:53,380][20347] Updated weights for policy 0, policy_version 650550 (0.0010) [2023-12-27 10:07:53,439][20347] Updated weights for policy 0, policy_version 650560 (0.0008) [2023-12-27 10:07:53,506][20347] Updated weights for policy 0, policy_version 650570 (0.0010) [2023-12-27 10:07:53,643][20381] Updated weights for policy 1, policy_version 649917 (0.0006) [2023-12-27 10:07:53,694][20381] Updated weights for policy 1, policy_version 649927 (0.0005) [2023-12-27 10:07:53,756][20381] Updated weights for policy 1, policy_version 649937 (0.0007) [2023-12-27 10:07:54,180][20347] Updated weights for policy 0, policy_version 650580 (0.0008) [2023-12-27 10:07:54,236][20347] Updated weights for policy 0, policy_version 650590 (0.0005) [2023-12-27 10:07:54,289][20347] Updated weights for policy 0, policy_version 650600 (0.0006) [2023-12-27 10:07:54,308][20381] Updated weights for policy 1, policy_version 649947 (0.0009) [2023-12-27 10:07:54,373][20381] Updated weights for policy 1, policy_version 649957 (0.0011) [2023-12-27 10:07:54,437][20381] Updated weights for policy 1, policy_version 649967 (0.0011) [2023-12-27 10:07:54,954][20347] Updated weights for policy 0, policy_version 650610 (0.0007) [2023-12-27 10:07:55,018][20347] Updated weights for policy 0, policy_version 650620 (0.0008) [2023-12-27 10:07:55,069][20381] Updated weights for policy 1, policy_version 649977 (0.0007) [2023-12-27 10:07:55,081][20347] Updated weights for policy 0, policy_version 650630 (0.0008) [2023-12-27 10:07:55,118][20381] Updated weights for policy 1, policy_version 649987 (0.0005) [2023-12-27 10:07:55,129][20347] Updated weights for policy 0, policy_version 650640 (0.0007) [2023-12-27 10:07:55,176][20381] Updated weights for policy 1, policy_version 649997 (0.0006) [2023-12-27 10:07:55,229][20381] Updated weights for policy 1, policy_version 650007 (0.0008) [2023-12-27 10:07:55,814][20347] Updated weights for policy 0, policy_version 650650 (0.0005) [2023-12-27 10:07:55,867][20347] Updated weights for policy 0, policy_version 650660 (0.0005) [2023-12-27 10:07:55,896][20381] Updated weights for policy 1, policy_version 650017 (0.0010) [2023-12-27 10:07:55,913][20347] Updated weights for policy 0, policy_version 650670 (0.0005) [2023-12-27 10:07:55,947][20381] Updated weights for policy 1, policy_version 650027 (0.0010) [2023-12-27 10:07:55,998][20381] Updated weights for policy 1, policy_version 650037 (0.0010) [2023-12-27 10:07:56,457][20347] Updated weights for policy 0, policy_version 650680 (0.0007) [2023-12-27 10:07:56,504][20347] Updated weights for policy 0, policy_version 650690 (0.0007) [2023-12-27 10:07:56,564][20347] Updated weights for policy 0, policy_version 650700 (0.0007) [2023-12-27 10:07:56,654][20381] Updated weights for policy 1, policy_version 650047 (0.0010) [2023-12-27 10:07:56,702][20381] Updated weights for policy 1, policy_version 650057 (0.0010) [2023-12-27 10:07:56,747][20381] Updated weights for policy 1, policy_version 650067 (0.0010) [2023-12-27 10:07:57,239][20347] Updated weights for policy 0, policy_version 650710 (0.0007) [2023-12-27 10:07:57,296][20347] Updated weights for policy 0, policy_version 650720 (0.0007) [2023-12-27 10:07:57,352][20347] Updated weights for policy 0, policy_version 650730 (0.0007) [2023-12-27 10:07:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 333045760. Throughput: 0: 10548.3, 1: 10352.1. Samples: 333041808. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:07:57,363][19170] Avg episode reward: [(0, '220.500'), (1, '856.520')] [2023-12-27 10:07:57,371][20381] Updated weights for policy 1, policy_version 650077 (0.0010) [2023-12-27 10:07:57,422][20381] Updated weights for policy 1, policy_version 650087 (0.0010) [2023-12-27 10:07:57,466][20381] Updated weights for policy 1, policy_version 650097 (0.0010) [2023-12-27 10:07:57,997][20347] Updated weights for policy 0, policy_version 650740 (0.0007) [2023-12-27 10:07:58,047][20347] Updated weights for policy 0, policy_version 650750 (0.0009) [2023-12-27 10:07:58,092][20381] Updated weights for policy 1, policy_version 650107 (0.0009) [2023-12-27 10:07:58,100][20347] Updated weights for policy 0, policy_version 650760 (0.0008) [2023-12-27 10:07:58,147][20381] Updated weights for policy 1, policy_version 650117 (0.0011) [2023-12-27 10:07:58,202][20381] Updated weights for policy 1, policy_version 650127 (0.0008) [2023-12-27 10:07:58,870][20347] Updated weights for policy 0, policy_version 650770 (0.0006) [2023-12-27 10:07:58,935][20347] Updated weights for policy 0, policy_version 650780 (0.0008) [2023-12-27 10:07:58,945][20381] Updated weights for policy 1, policy_version 650137 (0.0007) [2023-12-27 10:07:58,995][20347] Updated weights for policy 0, policy_version 650790 (0.0006) [2023-12-27 10:07:59,011][20381] Updated weights for policy 1, policy_version 650147 (0.0006) [2023-12-27 10:07:59,057][20347] Updated weights for policy 0, policy_version 650800 (0.0007) [2023-12-27 10:07:59,077][20381] Updated weights for policy 1, policy_version 650157 (0.0007) [2023-12-27 10:07:59,142][20381] Updated weights for policy 1, policy_version 650167 (0.0008) [2023-12-27 10:07:59,761][20347] Updated weights for policy 0, policy_version 650810 (0.0007) [2023-12-27 10:07:59,767][20381] Updated weights for policy 1, policy_version 650177 (0.0006) [2023-12-27 10:07:59,812][20347] Updated weights for policy 0, policy_version 650820 (0.0007) [2023-12-27 10:07:59,830][20381] Updated weights for policy 1, policy_version 650187 (0.0006) [2023-12-27 10:07:59,872][20347] Updated weights for policy 0, policy_version 650830 (0.0008) [2023-12-27 10:07:59,886][20381] Updated weights for policy 1, policy_version 650197 (0.0006) [2023-12-27 10:08:00,524][20347] Updated weights for policy 0, policy_version 650840 (0.0007) [2023-12-27 10:08:00,562][20381] Updated weights for policy 1, policy_version 650207 (0.0006) [2023-12-27 10:08:00,567][20347] Updated weights for policy 0, policy_version 650850 (0.0006) [2023-12-27 10:08:00,568][20292] KL-divergence is very high: 106.9826 [2023-12-27 10:08:00,614][20347] Updated weights for policy 0, policy_version 650860 (0.0007) [2023-12-27 10:08:00,620][20381] Updated weights for policy 1, policy_version 650217 (0.0006) [2023-12-27 10:08:00,668][20381] Updated weights for policy 1, policy_version 650227 (0.0005) [2023-12-27 10:08:01,272][20381] Updated weights for policy 1, policy_version 650237 (0.0007) [2023-12-27 10:08:01,327][20381] Updated weights for policy 1, policy_version 650247 (0.0008) [2023-12-27 10:08:01,358][20347] Updated weights for policy 0, policy_version 650870 (0.0007) [2023-12-27 10:08:01,386][20381] Updated weights for policy 1, policy_version 650257 (0.0008) [2023-12-27 10:08:01,422][20347] Updated weights for policy 0, policy_version 650880 (0.0006) [2023-12-27 10:08:01,482][20347] Updated weights for policy 0, policy_version 650890 (0.0008) [2023-12-27 10:08:02,034][20381] Updated weights for policy 1, policy_version 650267 (0.0006) [2023-12-27 10:08:02,094][20381] Updated weights for policy 1, policy_version 650277 (0.0006) [2023-12-27 10:08:02,144][20347] Updated weights for policy 0, policy_version 650900 (0.0006) [2023-12-27 10:08:02,154][20381] Updated weights for policy 1, policy_version 650287 (0.0006) [2023-12-27 10:08:02,203][20347] Updated weights for policy 0, policy_version 650910 (0.0008) [2023-12-27 10:08:02,262][20347] Updated weights for policy 0, policy_version 650920 (0.0008) [2023-12-27 10:08:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 333160448. Throughput: 0: 10530.6, 1: 10435.6. Samples: 333166088. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:02,363][19170] Avg episode reward: [(0, '265.600'), (1, '826.340')] [2023-12-27 10:08:02,811][20381] Updated weights for policy 1, policy_version 650297 (0.0007) [2023-12-27 10:08:02,856][20381] Updated weights for policy 1, policy_version 650307 (0.0010) [2023-12-27 10:08:02,893][20347] Updated weights for policy 0, policy_version 650930 (0.0006) [2023-12-27 10:08:02,906][20381] Updated weights for policy 1, policy_version 650317 (0.0010) [2023-12-27 10:08:02,948][20347] Updated weights for policy 0, policy_version 650940 (0.0006) [2023-12-27 10:08:02,963][20381] Updated weights for policy 1, policy_version 650327 (0.0010) [2023-12-27 10:08:02,996][20347] Updated weights for policy 0, policy_version 650950 (0.0007) [2023-12-27 10:08:03,046][20347] Updated weights for policy 0, policy_version 650960 (0.0007) [2023-12-27 10:08:03,661][20381] Updated weights for policy 1, policy_version 650337 (0.0006) [2023-12-27 10:08:03,710][20381] Updated weights for policy 1, policy_version 650347 (0.0006) [2023-12-27 10:08:03,750][20347] Updated weights for policy 0, policy_version 650970 (0.0007) [2023-12-27 10:08:03,757][20381] Updated weights for policy 1, policy_version 650357 (0.0005) [2023-12-27 10:08:03,805][20347] Updated weights for policy 0, policy_version 650980 (0.0007) [2023-12-27 10:08:03,866][20347] Updated weights for policy 0, policy_version 650990 (0.0008) [2023-12-27 10:08:04,395][20381] Updated weights for policy 1, policy_version 650367 (0.0005) [2023-12-27 10:08:04,450][20381] Updated weights for policy 1, policy_version 650377 (0.0006) [2023-12-27 10:08:04,497][20347] Updated weights for policy 0, policy_version 651000 (0.0007) [2023-12-27 10:08:04,501][20381] Updated weights for policy 1, policy_version 650387 (0.0008) [2023-12-27 10:08:04,551][20347] Updated weights for policy 0, policy_version 651010 (0.0007) [2023-12-27 10:08:04,606][20347] Updated weights for policy 0, policy_version 651020 (0.0008) [2023-12-27 10:08:05,088][20381] Updated weights for policy 1, policy_version 650397 (0.0006) [2023-12-27 10:08:05,139][20381] Updated weights for policy 1, policy_version 650407 (0.0006) [2023-12-27 10:08:05,191][20381] Updated weights for policy 1, policy_version 650417 (0.0008) [2023-12-27 10:08:05,191][20292] KL-divergence is very high: 106.7889 [2023-12-27 10:08:05,312][20347] Updated weights for policy 0, policy_version 651030 (0.0009) [2023-12-27 10:08:05,367][20347] Updated weights for policy 0, policy_version 651040 (0.0008) [2023-12-27 10:08:05,416][20347] Updated weights for policy 0, policy_version 651050 (0.0008) [2023-12-27 10:08:05,874][20381] Updated weights for policy 1, policy_version 650427 (0.0008) [2023-12-27 10:08:05,925][20381] Updated weights for policy 1, policy_version 650437 (0.0007) [2023-12-27 10:08:05,980][20381] Updated weights for policy 1, policy_version 650447 (0.0008) [2023-12-27 10:08:06,127][20347] Updated weights for policy 0, policy_version 651060 (0.0008) [2023-12-27 10:08:06,191][20347] Updated weights for policy 0, policy_version 651070 (0.0008) [2023-12-27 10:08:06,248][20347] Updated weights for policy 0, policy_version 651080 (0.0008) [2023-12-27 10:08:06,628][20381] Updated weights for policy 1, policy_version 650457 (0.0007) [2023-12-27 10:08:06,690][20381] Updated weights for policy 1, policy_version 650467 (0.0005) [2023-12-27 10:08:06,754][20381] Updated weights for policy 1, policy_version 650477 (0.0007) [2023-12-27 10:08:06,814][20381] Updated weights for policy 1, policy_version 650487 (0.0007) [2023-12-27 10:08:06,953][20347] Updated weights for policy 0, policy_version 651090 (0.0009) [2023-12-27 10:08:07,004][20347] Updated weights for policy 0, policy_version 651100 (0.0008) [2023-12-27 10:08:07,054][20347] Updated weights for policy 0, policy_version 651110 (0.0007) [2023-12-27 10:08:07,111][20347] Updated weights for policy 0, policy_version 651120 (0.0008) [2023-12-27 10:08:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 333258752. Throughput: 0: 10535.8, 1: 10459.6. Samples: 333229164. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:07,363][19170] Avg episode reward: [(0, '221.670'), (1, '833.490')] [2023-12-27 10:08:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000650488_166551552.pth... [2023-12-27 10:08:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000651120_166707200.pth... [2023-12-27 10:08:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000649272_166240256.pth [2023-12-27 10:08:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000649904_166395904.pth [2023-12-27 10:08:07,464][20381] Updated weights for policy 1, policy_version 650497 (0.0008) [2023-12-27 10:08:07,506][20381] Updated weights for policy 1, policy_version 650507 (0.0008) [2023-12-27 10:08:07,560][20381] Updated weights for policy 1, policy_version 650517 (0.0007) [2023-12-27 10:08:07,810][20347] Updated weights for policy 0, policy_version 651130 (0.0008) [2023-12-27 10:08:07,859][20347] Updated weights for policy 0, policy_version 651140 (0.0008) [2023-12-27 10:08:07,908][20347] Updated weights for policy 0, policy_version 651150 (0.0008) [2023-12-27 10:08:08,279][20381] Updated weights for policy 1, policy_version 650527 (0.0008) [2023-12-27 10:08:08,339][20381] Updated weights for policy 1, policy_version 650537 (0.0007) [2023-12-27 10:08:08,397][20381] Updated weights for policy 1, policy_version 650547 (0.0008) [2023-12-27 10:08:08,623][20347] Updated weights for policy 0, policy_version 651160 (0.0008) [2023-12-27 10:08:08,687][20347] Updated weights for policy 0, policy_version 651170 (0.0007) [2023-12-27 10:08:08,742][20347] Updated weights for policy 0, policy_version 651180 (0.0008) [2023-12-27 10:08:09,066][20381] Updated weights for policy 1, policy_version 650557 (0.0008) [2023-12-27 10:08:09,124][20381] Updated weights for policy 1, policy_version 650567 (0.0008) [2023-12-27 10:08:09,187][20381] Updated weights for policy 1, policy_version 650577 (0.0007) [2023-12-27 10:08:09,479][20347] Updated weights for policy 0, policy_version 651190 (0.0007) [2023-12-27 10:08:09,539][20347] Updated weights for policy 0, policy_version 651200 (0.0006) [2023-12-27 10:08:09,601][20347] Updated weights for policy 0, policy_version 651210 (0.0007) [2023-12-27 10:08:09,899][20381] Updated weights for policy 1, policy_version 650587 (0.0008) [2023-12-27 10:08:09,959][20381] Updated weights for policy 1, policy_version 650597 (0.0008) [2023-12-27 10:08:10,014][20381] Updated weights for policy 1, policy_version 650607 (0.0009) [2023-12-27 10:08:10,255][20347] Updated weights for policy 0, policy_version 651220 (0.0008) [2023-12-27 10:08:10,313][20347] Updated weights for policy 0, policy_version 651230 (0.0007) [2023-12-27 10:08:10,376][20347] Updated weights for policy 0, policy_version 651240 (0.0006) [2023-12-27 10:08:10,754][20381] Updated weights for policy 1, policy_version 650617 (0.0008) [2023-12-27 10:08:10,802][20381] Updated weights for policy 1, policy_version 650627 (0.0007) [2023-12-27 10:08:10,853][20381] Updated weights for policy 1, policy_version 650637 (0.0007) [2023-12-27 10:08:10,916][20381] Updated weights for policy 1, policy_version 650647 (0.0008) [2023-12-27 10:08:10,983][20347] Updated weights for policy 0, policy_version 651250 (0.0006) [2023-12-27 10:08:11,040][20347] Updated weights for policy 0, policy_version 651260 (0.0007) [2023-12-27 10:08:11,107][20347] Updated weights for policy 0, policy_version 651270 (0.0008) [2023-12-27 10:08:11,168][20347] Updated weights for policy 0, policy_version 651280 (0.0008) [2023-12-27 10:08:11,631][20381] Updated weights for policy 1, policy_version 650657 (0.0007) [2023-12-27 10:08:11,697][20381] Updated weights for policy 1, policy_version 650667 (0.0006) [2023-12-27 10:08:11,764][20381] Updated weights for policy 1, policy_version 650677 (0.0007) [2023-12-27 10:08:11,957][20347] Updated weights for policy 0, policy_version 651290 (0.0008) [2023-12-27 10:08:12,019][20347] Updated weights for policy 0, policy_version 651300 (0.0006) [2023-12-27 10:08:12,083][20347] Updated weights for policy 0, policy_version 651310 (0.0008) [2023-12-27 10:08:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 333357056. Throughput: 0: 10452.3, 1: 10442.3. Samples: 333349680. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:12,363][19170] Avg episode reward: [(0, '189.800'), (1, '902.420')] [2023-12-27 10:08:12,428][20381] Updated weights for policy 1, policy_version 650687 (0.0008) [2023-12-27 10:08:12,488][20381] Updated weights for policy 1, policy_version 650697 (0.0009) [2023-12-27 10:08:12,541][20381] Updated weights for policy 1, policy_version 650707 (0.0007) [2023-12-27 10:08:12,752][20347] Updated weights for policy 0, policy_version 651320 (0.0007) [2023-12-27 10:08:12,803][20347] Updated weights for policy 0, policy_version 651330 (0.0006) [2023-12-27 10:08:12,866][20347] Updated weights for policy 0, policy_version 651340 (0.0005) [2023-12-27 10:08:13,223][20381] Updated weights for policy 1, policy_version 650717 (0.0007) [2023-12-27 10:08:13,275][20381] Updated weights for policy 1, policy_version 650727 (0.0007) [2023-12-27 10:08:13,324][20381] Updated weights for policy 1, policy_version 650737 (0.0007) [2023-12-27 10:08:13,494][20347] Updated weights for policy 0, policy_version 651350 (0.0007) [2023-12-27 10:08:13,546][20347] Updated weights for policy 0, policy_version 651360 (0.0008) [2023-12-27 10:08:13,594][20347] Updated weights for policy 0, policy_version 651370 (0.0006) [2023-12-27 10:08:14,035][20381] Updated weights for policy 1, policy_version 650747 (0.0009) [2023-12-27 10:08:14,088][20381] Updated weights for policy 1, policy_version 650757 (0.0010) [2023-12-27 10:08:14,143][20381] Updated weights for policy 1, policy_version 650767 (0.0010) [2023-12-27 10:08:14,282][20347] Updated weights for policy 0, policy_version 651380 (0.0007) [2023-12-27 10:08:14,341][20347] Updated weights for policy 0, policy_version 651390 (0.0007) [2023-12-27 10:08:14,405][20347] Updated weights for policy 0, policy_version 651400 (0.0007) [2023-12-27 10:08:14,749][20381] Updated weights for policy 1, policy_version 650777 (0.0010) [2023-12-27 10:08:14,812][20381] Updated weights for policy 1, policy_version 650787 (0.0007) [2023-12-27 10:08:14,867][20381] Updated weights for policy 1, policy_version 650797 (0.0005) [2023-12-27 10:08:14,929][20381] Updated weights for policy 1, policy_version 650807 (0.0005) [2023-12-27 10:08:15,162][20347] Updated weights for policy 0, policy_version 651410 (0.0008) [2023-12-27 10:08:15,218][20347] Updated weights for policy 0, policy_version 651420 (0.0007) [2023-12-27 10:08:15,283][20347] Updated weights for policy 0, policy_version 651430 (0.0008) [2023-12-27 10:08:15,347][20347] Updated weights for policy 0, policy_version 651440 (0.0009) [2023-12-27 10:08:15,536][20381] Updated weights for policy 1, policy_version 650817 (0.0008) [2023-12-27 10:08:15,584][20381] Updated weights for policy 1, policy_version 650827 (0.0007) [2023-12-27 10:08:15,639][20381] Updated weights for policy 1, policy_version 650837 (0.0007) [2023-12-27 10:08:16,055][20347] Updated weights for policy 0, policy_version 651450 (0.0009) [2023-12-27 10:08:16,104][20347] Updated weights for policy 0, policy_version 651460 (0.0009) [2023-12-27 10:08:16,154][20347] Updated weights for policy 0, policy_version 651470 (0.0006) [2023-12-27 10:08:16,261][20381] Updated weights for policy 1, policy_version 650847 (0.0008) [2023-12-27 10:08:16,318][20381] Updated weights for policy 1, policy_version 650857 (0.0008) [2023-12-27 10:08:16,379][20381] Updated weights for policy 1, policy_version 650867 (0.0006) [2023-12-27 10:08:16,915][20347] Updated weights for policy 0, policy_version 651480 (0.0008) [2023-12-27 10:08:16,974][20347] Updated weights for policy 0, policy_version 651490 (0.0007) [2023-12-27 10:08:16,983][20381] Updated weights for policy 1, policy_version 650877 (0.0006) [2023-12-27 10:08:17,030][20347] Updated weights for policy 0, policy_version 651500 (0.0006) [2023-12-27 10:08:17,035][20381] Updated weights for policy 1, policy_version 650887 (0.0006) [2023-12-27 10:08:17,080][20381] Updated weights for policy 1, policy_version 650897 (0.0007) [2023-12-27 10:08:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20633.6). Total num frames: 333463552. Throughput: 0: 10324.9, 1: 10404.1. Samples: 333471428. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:17,364][19170] Avg episode reward: [(0, '281.850'), (1, '893.370')] [2023-12-27 10:08:17,736][20347] Updated weights for policy 0, policy_version 651510 (0.0007) [2023-12-27 10:08:17,750][20381] Updated weights for policy 1, policy_version 650907 (0.0008) [2023-12-27 10:08:17,788][20347] Updated weights for policy 0, policy_version 651520 (0.0007) [2023-12-27 10:08:17,808][20381] Updated weights for policy 1, policy_version 650917 (0.0006) [2023-12-27 10:08:17,851][20347] Updated weights for policy 0, policy_version 651530 (0.0006) [2023-12-27 10:08:17,871][20381] Updated weights for policy 1, policy_version 650927 (0.0006) [2023-12-27 10:08:18,537][20347] Updated weights for policy 0, policy_version 651540 (0.0007) [2023-12-27 10:08:18,578][20381] Updated weights for policy 1, policy_version 650937 (0.0007) [2023-12-27 10:08:18,592][20347] Updated weights for policy 0, policy_version 651550 (0.0007) [2023-12-27 10:08:18,636][20381] Updated weights for policy 1, policy_version 650947 (0.0006) [2023-12-27 10:08:18,649][20347] Updated weights for policy 0, policy_version 651560 (0.0006) [2023-12-27 10:08:18,700][20381] Updated weights for policy 1, policy_version 650957 (0.0006) [2023-12-27 10:08:18,755][20381] Updated weights for policy 1, policy_version 650967 (0.0008) [2023-12-27 10:08:19,400][20347] Updated weights for policy 0, policy_version 651570 (0.0005) [2023-12-27 10:08:19,406][20381] Updated weights for policy 1, policy_version 650977 (0.0008) [2023-12-27 10:08:19,449][20347] Updated weights for policy 0, policy_version 651580 (0.0006) [2023-12-27 10:08:19,462][20381] Updated weights for policy 1, policy_version 650987 (0.0006) [2023-12-27 10:08:19,500][20347] Updated weights for policy 0, policy_version 651590 (0.0006) [2023-12-27 10:08:19,522][20381] Updated weights for policy 1, policy_version 650997 (0.0007) [2023-12-27 10:08:19,555][20347] Updated weights for policy 0, policy_version 651600 (0.0005) [2023-12-27 10:08:20,223][20347] Updated weights for policy 0, policy_version 651610 (0.0007) [2023-12-27 10:08:20,273][20347] Updated weights for policy 0, policy_version 651620 (0.0006) [2023-12-27 10:08:20,319][20381] Updated weights for policy 1, policy_version 651007 (0.0008) [2023-12-27 10:08:20,331][20347] Updated weights for policy 0, policy_version 651630 (0.0005) [2023-12-27 10:08:20,371][20381] Updated weights for policy 1, policy_version 651017 (0.0009) [2023-12-27 10:08:20,427][20381] Updated weights for policy 1, policy_version 651027 (0.0009) [2023-12-27 10:08:21,014][20347] Updated weights for policy 0, policy_version 651640 (0.0010) [2023-12-27 10:08:21,079][20347] Updated weights for policy 0, policy_version 651650 (0.0010) [2023-12-27 10:08:21,098][20381] Updated weights for policy 1, policy_version 651037 (0.0009) [2023-12-27 10:08:21,143][20347] Updated weights for policy 0, policy_version 651660 (0.0011) [2023-12-27 10:08:21,157][20381] Updated weights for policy 1, policy_version 651047 (0.0006) [2023-12-27 10:08:21,209][20381] Updated weights for policy 1, policy_version 651057 (0.0006) [2023-12-27 10:08:21,886][20381] Updated weights for policy 1, policy_version 651067 (0.0006) [2023-12-27 10:08:21,903][20347] Updated weights for policy 0, policy_version 651670 (0.0008) [2023-12-27 10:08:21,935][20381] Updated weights for policy 1, policy_version 651077 (0.0006) [2023-12-27 10:08:21,956][20347] Updated weights for policy 0, policy_version 651680 (0.0008) [2023-12-27 10:08:21,991][20381] Updated weights for policy 1, policy_version 651087 (0.0008) [2023-12-27 10:08:22,005][20347] Updated weights for policy 0, policy_version 651690 (0.0007) [2023-12-27 10:08:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 333561856. Throughput: 0: 10275.6, 1: 10336.9. Samples: 333531020. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:22,363][19170] Avg episode reward: [(0, '357.400'), (1, '874.900')] [2023-12-27 10:08:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000651096_166707200.pth... [2023-12-27 10:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000651696_166854656.pth... [2023-12-27 10:08:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000649880_166395904.pth [2023-12-27 10:08:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000650512_166551552.pth [2023-12-27 10:08:22,658][20381] Updated weights for policy 1, policy_version 651097 (0.0008) [2023-12-27 10:08:22,714][20381] Updated weights for policy 1, policy_version 651107 (0.0008) [2023-12-27 10:08:22,720][20347] Updated weights for policy 0, policy_version 651700 (0.0008) [2023-12-27 10:08:22,768][20347] Updated weights for policy 0, policy_version 651710 (0.0006) [2023-12-27 10:08:22,769][20381] Updated weights for policy 1, policy_version 651117 (0.0006) [2023-12-27 10:08:22,817][20347] Updated weights for policy 0, policy_version 651720 (0.0011) [2023-12-27 10:08:22,820][20381] Updated weights for policy 1, policy_version 651127 (0.0006) [2023-12-27 10:08:23,487][20347] Updated weights for policy 0, policy_version 651730 (0.0009) [2023-12-27 10:08:23,520][20381] Updated weights for policy 1, policy_version 651137 (0.0006) [2023-12-27 10:08:23,548][20347] Updated weights for policy 0, policy_version 651740 (0.0010) [2023-12-27 10:08:23,578][20381] Updated weights for policy 1, policy_version 651147 (0.0005) [2023-12-27 10:08:23,600][20347] Updated weights for policy 0, policy_version 651750 (0.0010) [2023-12-27 10:08:23,632][20381] Updated weights for policy 1, policy_version 651157 (0.0005) [2023-12-27 10:08:23,648][20347] Updated weights for policy 0, policy_version 651760 (0.0010) [2023-12-27 10:08:24,219][20381] Updated weights for policy 1, policy_version 651167 (0.0005) [2023-12-27 10:08:24,273][20381] Updated weights for policy 1, policy_version 651177 (0.0005) [2023-12-27 10:08:24,312][20347] Updated weights for policy 0, policy_version 651770 (0.0009) [2023-12-27 10:08:24,329][20381] Updated weights for policy 1, policy_version 651187 (0.0006) [2023-12-27 10:08:24,372][20347] Updated weights for policy 0, policy_version 651780 (0.0009) [2023-12-27 10:08:24,419][20347] Updated weights for policy 0, policy_version 651790 (0.0010) [2023-12-27 10:08:24,992][20381] Updated weights for policy 1, policy_version 651197 (0.0006) [2023-12-27 10:08:25,050][20381] Updated weights for policy 1, policy_version 651207 (0.0007) [2023-12-27 10:08:25,072][20347] Updated weights for policy 0, policy_version 651800 (0.0010) [2023-12-27 10:08:25,095][20381] Updated weights for policy 1, policy_version 651217 (0.0007) [2023-12-27 10:08:25,122][20347] Updated weights for policy 0, policy_version 651810 (0.0010) [2023-12-27 10:08:25,173][20347] Updated weights for policy 0, policy_version 651820 (0.0010) [2023-12-27 10:08:25,779][20381] Updated weights for policy 1, policy_version 651227 (0.0008) [2023-12-27 10:08:25,813][20347] Updated weights for policy 0, policy_version 651830 (0.0010) [2023-12-27 10:08:25,837][20381] Updated weights for policy 1, policy_version 651237 (0.0009) [2023-12-27 10:08:25,867][20347] Updated weights for policy 0, policy_version 651840 (0.0010) [2023-12-27 10:08:25,893][20381] Updated weights for policy 1, policy_version 651247 (0.0010) [2023-12-27 10:08:25,922][20347] Updated weights for policy 0, policy_version 651850 (0.0010) [2023-12-27 10:08:26,558][20381] Updated weights for policy 1, policy_version 651257 (0.0010) [2023-12-27 10:08:26,563][20347] Updated weights for policy 0, policy_version 651860 (0.0009) [2023-12-27 10:08:26,609][20381] Updated weights for policy 1, policy_version 651267 (0.0010) [2023-12-27 10:08:26,614][20347] Updated weights for policy 0, policy_version 651870 (0.0005) [2023-12-27 10:08:26,661][20347] Updated weights for policy 0, policy_version 651880 (0.0005) [2023-12-27 10:08:26,663][20381] Updated weights for policy 1, policy_version 651277 (0.0010) [2023-12-27 10:08:26,708][20381] Updated weights for policy 1, policy_version 651287 (0.0010) [2023-12-27 10:08:27,325][20347] Updated weights for policy 0, policy_version 651890 (0.0007) [2023-12-27 10:08:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 333660160. Throughput: 0: 10205.9, 1: 10337.1. Samples: 333654736. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-12-27 10:08:27,363][19170] Avg episode reward: [(0, '410.220'), (1, '855.260')] [2023-12-27 10:08:27,379][20347] Updated weights for policy 0, policy_version 651900 (0.0010) [2023-12-27 10:08:27,403][20381] Updated weights for policy 1, policy_version 651297 (0.0010) [2023-12-27 10:08:27,429][20347] Updated weights for policy 0, policy_version 651910 (0.0010) [2023-12-27 10:08:27,451][20381] Updated weights for policy 1, policy_version 651307 (0.0010) [2023-12-27 10:08:27,487][20347] Updated weights for policy 0, policy_version 651920 (0.0010) [2023-12-27 10:08:27,502][20381] Updated weights for policy 1, policy_version 651317 (0.0010) [2023-12-27 10:08:28,125][20381] Updated weights for policy 1, policy_version 651327 (0.0008) [2023-12-27 10:08:28,167][20347] Updated weights for policy 0, policy_version 651930 (0.0010) [2023-12-27 10:08:28,180][20381] Updated weights for policy 1, policy_version 651337 (0.0005) [2023-12-27 10:08:28,214][20347] Updated weights for policy 0, policy_version 651940 (0.0010) [2023-12-27 10:08:28,232][20381] Updated weights for policy 1, policy_version 651347 (0.0007) [2023-12-27 10:08:28,274][20347] Updated weights for policy 0, policy_version 651950 (0.0010) [2023-12-27 10:08:28,924][20381] Updated weights for policy 1, policy_version 651357 (0.0007) [2023-12-27 10:08:28,951][20347] Updated weights for policy 0, policy_version 651960 (0.0010) [2023-12-27 10:08:28,982][20381] Updated weights for policy 1, policy_version 651367 (0.0006) [2023-12-27 10:08:29,003][20347] Updated weights for policy 0, policy_version 651970 (0.0010) [2023-12-27 10:08:29,036][20381] Updated weights for policy 1, policy_version 651377 (0.0009) [2023-12-27 10:08:29,051][20347] Updated weights for policy 0, policy_version 651980 (0.0010) [2023-12-27 10:08:29,690][20381] Updated weights for policy 1, policy_version 651387 (0.0008) [2023-12-27 10:08:29,747][20381] Updated weights for policy 1, policy_version 651397 (0.0005) [2023-12-27 10:08:29,750][20347] Updated weights for policy 0, policy_version 651990 (0.0010) [2023-12-27 10:08:29,806][20347] Updated weights for policy 0, policy_version 652000 (0.0010) [2023-12-27 10:08:29,806][20381] Updated weights for policy 1, policy_version 651407 (0.0006) [2023-12-27 10:08:29,864][20347] Updated weights for policy 0, policy_version 652010 (0.0006) [2023-12-27 10:08:30,467][20381] Updated weights for policy 1, policy_version 651417 (0.0006) [2023-12-27 10:08:30,522][20381] Updated weights for policy 1, policy_version 651427 (0.0006) [2023-12-27 10:08:30,533][20347] Updated weights for policy 0, policy_version 652020 (0.0008) [2023-12-27 10:08:30,573][20381] Updated weights for policy 1, policy_version 651437 (0.0005) [2023-12-27 10:08:30,594][20347] Updated weights for policy 0, policy_version 652030 (0.0010) [2023-12-27 10:08:30,632][20381] Updated weights for policy 1, policy_version 651447 (0.0005) [2023-12-27 10:08:30,652][20347] Updated weights for policy 0, policy_version 652040 (0.0010) [2023-12-27 10:08:31,251][20381] Updated weights for policy 1, policy_version 651457 (0.0008) [2023-12-27 10:08:31,309][20381] Updated weights for policy 1, policy_version 651467 (0.0008) [2023-12-27 10:08:31,344][20347] Updated weights for policy 0, policy_version 652050 (0.0010) [2023-12-27 10:08:31,374][20381] Updated weights for policy 1, policy_version 651477 (0.0009) [2023-12-27 10:08:31,408][20347] Updated weights for policy 0, policy_version 652060 (0.0007) [2023-12-27 10:08:31,472][20347] Updated weights for policy 0, policy_version 652070 (0.0007) [2023-12-27 10:08:31,524][20347] Updated weights for policy 0, policy_version 652080 (0.0008) [2023-12-27 10:08:32,059][20381] Updated weights for policy 1, policy_version 651487 (0.0007) [2023-12-27 10:08:32,114][20381] Updated weights for policy 1, policy_version 651497 (0.0007) [2023-12-27 10:08:32,167][20347] Updated weights for policy 0, policy_version 652090 (0.0010) [2023-12-27 10:08:32,172][20381] Updated weights for policy 1, policy_version 651507 (0.0007) [2023-12-27 10:08:32,227][20347] Updated weights for policy 0, policy_version 652100 (0.0010) [2023-12-27 10:08:32,289][20347] Updated weights for policy 0, policy_version 652110 (0.0010) [2023-12-27 10:08:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 333774848. Throughput: 0: 10183.1, 1: 10337.3. Samples: 333779760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:32,363][19170] Avg episode reward: [(0, '670.390'), (1, '932.040')] [2023-12-27 10:08:32,801][20381] Updated weights for policy 1, policy_version 651517 (0.0006) [2023-12-27 10:08:32,850][20381] Updated weights for policy 1, policy_version 651527 (0.0007) [2023-12-27 10:08:32,897][20381] Updated weights for policy 1, policy_version 651537 (0.0005) [2023-12-27 10:08:32,990][20347] Updated weights for policy 0, policy_version 652120 (0.0008) [2023-12-27 10:08:33,043][20347] Updated weights for policy 0, policy_version 652130 (0.0007) [2023-12-27 10:08:33,096][20347] Updated weights for policy 0, policy_version 652140 (0.0006) [2023-12-27 10:08:33,569][20381] Updated weights for policy 1, policy_version 651547 (0.0007) [2023-12-27 10:08:33,628][20381] Updated weights for policy 1, policy_version 651557 (0.0008) [2023-12-27 10:08:33,679][20347] Updated weights for policy 0, policy_version 652150 (0.0007) [2023-12-27 10:08:33,690][20381] Updated weights for policy 1, policy_version 651567 (0.0006) [2023-12-27 10:08:33,728][20347] Updated weights for policy 0, policy_version 652160 (0.0008) [2023-12-27 10:08:33,778][20347] Updated weights for policy 0, policy_version 652170 (0.0008) [2023-12-27 10:08:34,364][20381] Updated weights for policy 1, policy_version 651577 (0.0006) [2023-12-27 10:08:34,425][20381] Updated weights for policy 1, policy_version 651587 (0.0008) [2023-12-27 10:08:34,482][20381] Updated weights for policy 1, policy_version 651597 (0.0008) [2023-12-27 10:08:34,501][20347] Updated weights for policy 0, policy_version 652180 (0.0007) [2023-12-27 10:08:34,536][20381] Updated weights for policy 1, policy_version 651607 (0.0007) [2023-12-27 10:08:34,551][20347] Updated weights for policy 0, policy_version 652190 (0.0005) [2023-12-27 10:08:34,609][20347] Updated weights for policy 0, policy_version 652200 (0.0005) [2023-12-27 10:08:35,195][20381] Updated weights for policy 1, policy_version 651617 (0.0010) [2023-12-27 10:08:35,254][20381] Updated weights for policy 1, policy_version 651627 (0.0010) [2023-12-27 10:08:35,294][20347] Updated weights for policy 0, policy_version 652210 (0.0007) [2023-12-27 10:08:35,310][20381] Updated weights for policy 1, policy_version 651637 (0.0011) [2023-12-27 10:08:35,349][20347] Updated weights for policy 0, policy_version 652220 (0.0006) [2023-12-27 10:08:35,408][20347] Updated weights for policy 0, policy_version 652230 (0.0010) [2023-12-27 10:08:35,469][20347] Updated weights for policy 0, policy_version 652240 (0.0010) [2023-12-27 10:08:35,995][20381] Updated weights for policy 1, policy_version 651647 (0.0009) [2023-12-27 10:08:36,050][20381] Updated weights for policy 1, policy_version 651657 (0.0010) [2023-12-27 10:08:36,074][20347] Updated weights for policy 0, policy_version 652250 (0.0008) [2023-12-27 10:08:36,104][20381] Updated weights for policy 1, policy_version 651667 (0.0010) [2023-12-27 10:08:36,142][20347] Updated weights for policy 0, policy_version 652260 (0.0008) [2023-12-27 10:08:36,199][20347] Updated weights for policy 0, policy_version 652270 (0.0007) [2023-12-27 10:08:36,838][20347] Updated weights for policy 0, policy_version 652280 (0.0010) [2023-12-27 10:08:36,847][20381] Updated weights for policy 1, policy_version 651677 (0.0009) [2023-12-27 10:08:36,892][20347] Updated weights for policy 0, policy_version 652290 (0.0011) [2023-12-27 10:08:36,905][20381] Updated weights for policy 1, policy_version 651687 (0.0005) [2023-12-27 10:08:36,944][20347] Updated weights for policy 0, policy_version 652300 (0.0010) [2023-12-27 10:08:36,954][20381] Updated weights for policy 1, policy_version 651697 (0.0006) [2023-12-27 10:08:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 333873152. Throughput: 0: 10139.1, 1: 10396.7. Samples: 333842036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:37,363][19170] Avg episode reward: [(0, '727.770'), (1, '932.160')] [2023-12-27 10:08:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000652304_167010304.pth... [2023-12-27 10:08:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000651704_166862848.pth... [2023-12-27 10:08:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000651120_166707200.pth [2023-12-27 10:08:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000650488_166551552.pth [2023-12-27 10:08:37,639][20347] Updated weights for policy 0, policy_version 652310 (0.0010) [2023-12-27 10:08:37,670][20381] Updated weights for policy 1, policy_version 651707 (0.0007) [2023-12-27 10:08:37,684][20347] Updated weights for policy 0, policy_version 652320 (0.0007) [2023-12-27 10:08:37,723][20381] Updated weights for policy 1, policy_version 651717 (0.0008) [2023-12-27 10:08:37,741][20347] Updated weights for policy 0, policy_version 652330 (0.0005) [2023-12-27 10:08:37,782][20381] Updated weights for policy 1, policy_version 651727 (0.0007) [2023-12-27 10:08:38,316][20347] Updated weights for policy 0, policy_version 652340 (0.0005) [2023-12-27 10:08:38,374][20347] Updated weights for policy 0, policy_version 652350 (0.0007) [2023-12-27 10:08:38,435][20347] Updated weights for policy 0, policy_version 652360 (0.0006) [2023-12-27 10:08:38,491][20381] Updated weights for policy 1, policy_version 651737 (0.0008) [2023-12-27 10:08:38,549][20381] Updated weights for policy 1, policy_version 651747 (0.0008) [2023-12-27 10:08:38,615][20381] Updated weights for policy 1, policy_version 651757 (0.0009) [2023-12-27 10:08:38,674][20381] Updated weights for policy 1, policy_version 651767 (0.0011) [2023-12-27 10:08:39,104][20347] Updated weights for policy 0, policy_version 652370 (0.0008) [2023-12-27 10:08:39,164][20347] Updated weights for policy 0, policy_version 652380 (0.0007) [2023-12-27 10:08:39,229][20347] Updated weights for policy 0, policy_version 652390 (0.0006) [2023-12-27 10:08:39,280][20381] Updated weights for policy 1, policy_version 651777 (0.0007) [2023-12-27 10:08:39,284][20347] Updated weights for policy 0, policy_version 652400 (0.0007) [2023-12-27 10:08:39,333][20381] Updated weights for policy 1, policy_version 651787 (0.0007) [2023-12-27 10:08:39,397][20381] Updated weights for policy 1, policy_version 651797 (0.0008) [2023-12-27 10:08:39,930][20347] Updated weights for policy 0, policy_version 652410 (0.0008) [2023-12-27 10:08:39,987][20347] Updated weights for policy 0, policy_version 652420 (0.0008) [2023-12-27 10:08:40,017][20381] Updated weights for policy 1, policy_version 651807 (0.0007) [2023-12-27 10:08:40,047][20347] Updated weights for policy 0, policy_version 652430 (0.0006) [2023-12-27 10:08:40,078][20381] Updated weights for policy 1, policy_version 651817 (0.0007) [2023-12-27 10:08:40,143][20381] Updated weights for policy 1, policy_version 651827 (0.0006) [2023-12-27 10:08:40,735][20381] Updated weights for policy 1, policy_version 651837 (0.0006) [2023-12-27 10:08:40,741][20347] Updated weights for policy 0, policy_version 652440 (0.0010) [2023-12-27 10:08:40,787][20381] Updated weights for policy 1, policy_version 651847 (0.0007) [2023-12-27 10:08:40,797][20347] Updated weights for policy 0, policy_version 652450 (0.0006) [2023-12-27 10:08:40,836][20381] Updated weights for policy 1, policy_version 651857 (0.0007) [2023-12-27 10:08:40,849][20347] Updated weights for policy 0, policy_version 652460 (0.0006) [2023-12-27 10:08:41,512][20381] Updated weights for policy 1, policy_version 651867 (0.0006) [2023-12-27 10:08:41,548][20347] Updated weights for policy 0, policy_version 652470 (0.0007) [2023-12-27 10:08:41,571][20381] Updated weights for policy 1, policy_version 651877 (0.0007) [2023-12-27 10:08:41,596][20347] Updated weights for policy 0, policy_version 652480 (0.0006) [2023-12-27 10:08:41,641][20381] Updated weights for policy 1, policy_version 651887 (0.0006) [2023-12-27 10:08:41,664][20347] Updated weights for policy 0, policy_version 652490 (0.0007) [2023-12-27 10:08:42,251][20381] Updated weights for policy 1, policy_version 651897 (0.0008) [2023-12-27 10:08:42,316][20381] Updated weights for policy 1, policy_version 651907 (0.0008) [2023-12-27 10:08:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 333971456. Throughput: 0: 10176.2, 1: 10396.2. Samples: 333967564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:42,363][19170] Avg episode reward: [(0, '659.280'), (1, '932.250')] [2023-12-27 10:08:42,375][20347] Updated weights for policy 0, policy_version 652500 (0.0008) [2023-12-27 10:08:42,381][20381] Updated weights for policy 1, policy_version 651917 (0.0007) [2023-12-27 10:08:42,435][20347] Updated weights for policy 0, policy_version 652510 (0.0007) [2023-12-27 10:08:42,439][20381] Updated weights for policy 1, policy_version 651927 (0.0007) [2023-12-27 10:08:42,499][20347] Updated weights for policy 0, policy_version 652520 (0.0008) [2023-12-27 10:08:43,044][20381] Updated weights for policy 1, policy_version 651937 (0.0005) [2023-12-27 10:08:43,096][20381] Updated weights for policy 1, policy_version 651947 (0.0006) [2023-12-27 10:08:43,142][20381] Updated weights for policy 1, policy_version 651957 (0.0006) [2023-12-27 10:08:43,248][20347] Updated weights for policy 0, policy_version 652530 (0.0008) [2023-12-27 10:08:43,301][20347] Updated weights for policy 0, policy_version 652540 (0.0007) [2023-12-27 10:08:43,355][20347] Updated weights for policy 0, policy_version 652550 (0.0005) [2023-12-27 10:08:43,412][20347] Updated weights for policy 0, policy_version 652560 (0.0006) [2023-12-27 10:08:43,756][20381] Updated weights for policy 1, policy_version 651967 (0.0006) [2023-12-27 10:08:43,815][20381] Updated weights for policy 1, policy_version 651977 (0.0008) [2023-12-27 10:08:43,871][20381] Updated weights for policy 1, policy_version 651987 (0.0007) [2023-12-27 10:08:43,993][20347] Updated weights for policy 0, policy_version 652570 (0.0008) [2023-12-27 10:08:44,048][20347] Updated weights for policy 0, policy_version 652580 (0.0008) [2023-12-27 10:08:44,109][20347] Updated weights for policy 0, policy_version 652590 (0.0007) [2023-12-27 10:08:44,531][20381] Updated weights for policy 1, policy_version 651997 (0.0008) [2023-12-27 10:08:44,586][20381] Updated weights for policy 1, policy_version 652007 (0.0008) [2023-12-27 10:08:44,648][20381] Updated weights for policy 1, policy_version 652017 (0.0008) [2023-12-27 10:08:44,791][20347] Updated weights for policy 0, policy_version 652600 (0.0007) [2023-12-27 10:08:44,850][20347] Updated weights for policy 0, policy_version 652610 (0.0008) [2023-12-27 10:08:44,914][20347] Updated weights for policy 0, policy_version 652620 (0.0008) [2023-12-27 10:08:45,357][20381] Updated weights for policy 1, policy_version 652027 (0.0007) [2023-12-27 10:08:45,406][20381] Updated weights for policy 1, policy_version 652037 (0.0008) [2023-12-27 10:08:45,460][20381] Updated weights for policy 1, policy_version 652047 (0.0008) [2023-12-27 10:08:45,583][20347] Updated weights for policy 0, policy_version 652630 (0.0009) [2023-12-27 10:08:45,628][20347] Updated weights for policy 0, policy_version 652640 (0.0010) [2023-12-27 10:08:45,680][20347] Updated weights for policy 0, policy_version 652650 (0.0010) [2023-12-27 10:08:46,122][20381] Updated weights for policy 1, policy_version 652057 (0.0008) [2023-12-27 10:08:46,176][20381] Updated weights for policy 1, policy_version 652067 (0.0010) [2023-12-27 10:08:46,230][20381] Updated weights for policy 1, policy_version 652077 (0.0010) [2023-12-27 10:08:46,279][20381] Updated weights for policy 1, policy_version 652087 (0.0010) [2023-12-27 10:08:46,383][20347] Updated weights for policy 0, policy_version 652660 (0.0010) [2023-12-27 10:08:46,435][20347] Updated weights for policy 0, policy_version 652670 (0.0010) [2023-12-27 10:08:46,490][20347] Updated weights for policy 0, policy_version 652680 (0.0010) [2023-12-27 10:08:46,954][20381] Updated weights for policy 1, policy_version 652097 (0.0011) [2023-12-27 10:08:47,015][20381] Updated weights for policy 1, policy_version 652107 (0.0011) [2023-12-27 10:08:47,079][20381] Updated weights for policy 1, policy_version 652117 (0.0011) [2023-12-27 10:08:47,163][20347] Updated weights for policy 0, policy_version 652690 (0.0010) [2023-12-27 10:08:47,209][20347] Updated weights for policy 0, policy_version 652700 (0.0007) [2023-12-27 10:08:47,256][20347] Updated weights for policy 0, policy_version 652710 (0.0007) [2023-12-27 10:08:47,303][20347] Updated weights for policy 0, policy_version 652720 (0.0007) [2023-12-27 10:08:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 334086144. Throughput: 0: 10173.2, 1: 10388.8. Samples: 334091376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:47,363][19170] Avg episode reward: [(0, '766.550'), (1, '893.340')] [2023-12-27 10:08:47,743][20381] Updated weights for policy 1, policy_version 652127 (0.0011) [2023-12-27 10:08:47,804][20381] Updated weights for policy 1, policy_version 652137 (0.0010) [2023-12-27 10:08:47,861][20381] Updated weights for policy 1, policy_version 652147 (0.0011) [2023-12-27 10:08:48,007][20347] Updated weights for policy 0, policy_version 652730 (0.0009) [2023-12-27 10:08:48,059][20347] Updated weights for policy 0, policy_version 652740 (0.0008) [2023-12-27 10:08:48,109][20347] Updated weights for policy 0, policy_version 652750 (0.0007) [2023-12-27 10:08:48,502][20381] Updated weights for policy 1, policy_version 652157 (0.0009) [2023-12-27 10:08:48,557][20381] Updated weights for policy 1, policy_version 652167 (0.0010) [2023-12-27 10:08:48,606][20381] Updated weights for policy 1, policy_version 652177 (0.0010) [2023-12-27 10:08:48,811][20347] Updated weights for policy 0, policy_version 652760 (0.0007) [2023-12-27 10:08:48,867][20347] Updated weights for policy 0, policy_version 652770 (0.0005) [2023-12-27 10:08:48,919][20347] Updated weights for policy 0, policy_version 652780 (0.0006) [2023-12-27 10:08:49,255][20381] Updated weights for policy 1, policy_version 652187 (0.0007) [2023-12-27 10:08:49,314][20381] Updated weights for policy 1, policy_version 652197 (0.0008) [2023-12-27 10:08:49,384][20381] Updated weights for policy 1, policy_version 652207 (0.0009) [2023-12-27 10:08:49,620][20347] Updated weights for policy 0, policy_version 652790 (0.0008) [2023-12-27 10:08:49,667][20347] Updated weights for policy 0, policy_version 652800 (0.0007) [2023-12-27 10:08:49,715][20347] Updated weights for policy 0, policy_version 652810 (0.0007) [2023-12-27 10:08:50,121][20381] Updated weights for policy 1, policy_version 652217 (0.0010) [2023-12-27 10:08:50,166][20381] Updated weights for policy 1, policy_version 652227 (0.0008) [2023-12-27 10:08:50,214][20381] Updated weights for policy 1, policy_version 652237 (0.0008) [2023-12-27 10:08:50,262][20381] Updated weights for policy 1, policy_version 652247 (0.0008) [2023-12-27 10:08:50,429][20347] Updated weights for policy 0, policy_version 652820 (0.0007) [2023-12-27 10:08:50,490][20347] Updated weights for policy 0, policy_version 652830 (0.0005) [2023-12-27 10:08:50,543][20347] Updated weights for policy 0, policy_version 652840 (0.0005) [2023-12-27 10:08:51,045][20381] Updated weights for policy 1, policy_version 652257 (0.0009) [2023-12-27 10:08:51,106][20381] Updated weights for policy 1, policy_version 652267 (0.0008) [2023-12-27 10:08:51,163][20381] Updated weights for policy 1, policy_version 652277 (0.0008) [2023-12-27 10:08:51,172][20347] Updated weights for policy 0, policy_version 652850 (0.0007) [2023-12-27 10:08:51,229][20347] Updated weights for policy 0, policy_version 652860 (0.0006) [2023-12-27 10:08:51,288][20347] Updated weights for policy 0, policy_version 652870 (0.0007) [2023-12-27 10:08:51,342][20347] Updated weights for policy 0, policy_version 652880 (0.0006) [2023-12-27 10:08:51,884][20381] Updated weights for policy 1, policy_version 652287 (0.0008) [2023-12-27 10:08:51,946][20381] Updated weights for policy 1, policy_version 652297 (0.0006) [2023-12-27 10:08:52,006][20381] Updated weights for policy 1, policy_version 652307 (0.0008) [2023-12-27 10:08:52,081][20347] Updated weights for policy 0, policy_version 652890 (0.0005) [2023-12-27 10:08:52,128][20347] Updated weights for policy 0, policy_version 652900 (0.0005) [2023-12-27 10:08:52,170][20347] Updated weights for policy 0, policy_version 652910 (0.0005) [2023-12-27 10:08:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 334184448. Throughput: 0: 10154.1, 1: 10355.9. Samples: 334152116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:52,363][19170] Avg episode reward: [(0, '756.500'), (1, '903.170')] [2023-12-27 10:08:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000652312_167018496.pth... [2023-12-27 10:08:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000652912_167165952.pth... [2023-12-27 10:08:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000651696_166854656.pth [2023-12-27 10:08:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000651096_166707200.pth [2023-12-27 10:08:52,636][20381] Updated weights for policy 1, policy_version 652317 (0.0008) [2023-12-27 10:08:52,701][20381] Updated weights for policy 1, policy_version 652327 (0.0008) [2023-12-27 10:08:52,758][20381] Updated weights for policy 1, policy_version 652337 (0.0008) [2023-12-27 10:08:52,831][20347] Updated weights for policy 0, policy_version 652920 (0.0006) [2023-12-27 10:08:52,883][20347] Updated weights for policy 0, policy_version 652930 (0.0008) [2023-12-27 10:08:52,938][20347] Updated weights for policy 0, policy_version 652940 (0.0007) [2023-12-27 10:08:53,514][20381] Updated weights for policy 1, policy_version 652347 (0.0008) [2023-12-27 10:08:53,560][20347] Updated weights for policy 0, policy_version 652950 (0.0007) [2023-12-27 10:08:53,565][20381] Updated weights for policy 1, policy_version 652357 (0.0008) [2023-12-27 10:08:53,615][20381] Updated weights for policy 1, policy_version 652367 (0.0008) [2023-12-27 10:08:53,616][20347] Updated weights for policy 0, policy_version 652960 (0.0005) [2023-12-27 10:08:53,668][20347] Updated weights for policy 0, policy_version 652970 (0.0005) [2023-12-27 10:08:54,266][20347] Updated weights for policy 0, policy_version 652980 (0.0005) [2023-12-27 10:08:54,321][20347] Updated weights for policy 0, policy_version 652990 (0.0005) [2023-12-27 10:08:54,354][20381] Updated weights for policy 1, policy_version 652377 (0.0007) [2023-12-27 10:08:54,371][20347] Updated weights for policy 0, policy_version 653000 (0.0005) [2023-12-27 10:08:54,410][20381] Updated weights for policy 1, policy_version 652387 (0.0008) [2023-12-27 10:08:54,460][20381] Updated weights for policy 1, policy_version 652397 (0.0008) [2023-12-27 10:08:54,509][20381] Updated weights for policy 1, policy_version 652407 (0.0007) [2023-12-27 10:08:54,972][20347] Updated weights for policy 0, policy_version 653010 (0.0006) [2023-12-27 10:08:55,034][20347] Updated weights for policy 0, policy_version 653020 (0.0008) [2023-12-27 10:08:55,088][20347] Updated weights for policy 0, policy_version 653030 (0.0005) [2023-12-27 10:08:55,145][20347] Updated weights for policy 0, policy_version 653040 (0.0005) [2023-12-27 10:08:55,158][20381] Updated weights for policy 1, policy_version 652417 (0.0006) [2023-12-27 10:08:55,201][20381] Updated weights for policy 1, policy_version 652427 (0.0005) [2023-12-27 10:08:55,252][20381] Updated weights for policy 1, policy_version 652437 (0.0005) [2023-12-27 10:08:55,805][20347] Updated weights for policy 0, policy_version 653050 (0.0007) [2023-12-27 10:08:55,846][20381] Updated weights for policy 1, policy_version 652447 (0.0009) [2023-12-27 10:08:55,860][20347] Updated weights for policy 0, policy_version 653060 (0.0010) [2023-12-27 10:08:55,893][20381] Updated weights for policy 1, policy_version 652457 (0.0006) [2023-12-27 10:08:55,913][20347] Updated weights for policy 0, policy_version 653070 (0.0010) [2023-12-27 10:08:55,936][20381] Updated weights for policy 1, policy_version 652467 (0.0005) [2023-12-27 10:08:56,577][20347] Updated weights for policy 0, policy_version 653080 (0.0010) [2023-12-27 10:08:56,616][20381] Updated weights for policy 1, policy_version 652477 (0.0006) [2023-12-27 10:08:56,636][20347] Updated weights for policy 0, policy_version 653090 (0.0005) [2023-12-27 10:08:56,666][20381] Updated weights for policy 1, policy_version 652487 (0.0008) [2023-12-27 10:08:56,691][20347] Updated weights for policy 0, policy_version 653100 (0.0010) [2023-12-27 10:08:56,718][20381] Updated weights for policy 1, policy_version 652497 (0.0006) [2023-12-27 10:08:57,325][20381] Updated weights for policy 1, policy_version 652507 (0.0007) [2023-12-27 10:08:57,335][20347] Updated weights for policy 0, policy_version 653110 (0.0010) [2023-12-27 10:08:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 334282752. Throughput: 0: 10247.1, 1: 10368.9. Samples: 334277400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:08:57,363][19170] Avg episode reward: [(0, '698.310'), (1, '951.690')] [2023-12-27 10:08:57,380][20381] Updated weights for policy 1, policy_version 652517 (0.0005) [2023-12-27 10:08:57,389][20347] Updated weights for policy 0, policy_version 653120 (0.0010) [2023-12-27 10:08:57,433][20381] Updated weights for policy 1, policy_version 652527 (0.0007) [2023-12-27 10:08:57,449][20347] Updated weights for policy 0, policy_version 653130 (0.0010) [2023-12-27 10:08:58,049][20347] Updated weights for policy 0, policy_version 653140 (0.0010) [2023-12-27 10:08:58,084][20381] Updated weights for policy 1, policy_version 652537 (0.0009) [2023-12-27 10:08:58,093][20347] Updated weights for policy 0, policy_version 653150 (0.0010) [2023-12-27 10:08:58,132][20381] Updated weights for policy 1, policy_version 652547 (0.0005) [2023-12-27 10:08:58,138][20347] Updated weights for policy 0, policy_version 653160 (0.0010) [2023-12-27 10:08:58,201][20381] Updated weights for policy 1, policy_version 652557 (0.0006) [2023-12-27 10:08:58,265][20381] Updated weights for policy 1, policy_version 652567 (0.0007) [2023-12-27 10:08:58,873][20347] Updated weights for policy 0, policy_version 653170 (0.0007) [2023-12-27 10:08:58,935][20347] Updated weights for policy 0, policy_version 653180 (0.0006) [2023-12-27 10:08:58,967][20381] Updated weights for policy 1, policy_version 652577 (0.0007) [2023-12-27 10:08:58,993][20347] Updated weights for policy 0, policy_version 653190 (0.0010) [2023-12-27 10:08:59,016][20381] Updated weights for policy 1, policy_version 652587 (0.0008) [2023-12-27 10:08:59,051][20347] Updated weights for policy 0, policy_version 653200 (0.0006) [2023-12-27 10:08:59,063][20381] Updated weights for policy 1, policy_version 652597 (0.0006) [2023-12-27 10:08:59,653][20347] Updated weights for policy 0, policy_version 653210 (0.0009) [2023-12-27 10:08:59,705][20381] Updated weights for policy 1, policy_version 652607 (0.0007) [2023-12-27 10:08:59,710][20347] Updated weights for policy 0, policy_version 653220 (0.0008) [2023-12-27 10:08:59,763][20381] Updated weights for policy 1, policy_version 652617 (0.0007) [2023-12-27 10:08:59,766][20347] Updated weights for policy 0, policy_version 653230 (0.0006) [2023-12-27 10:08:59,823][20381] Updated weights for policy 1, policy_version 652627 (0.0008) [2023-12-27 10:09:00,523][20347] Updated weights for policy 0, policy_version 653240 (0.0006) [2023-12-27 10:09:00,572][20347] Updated weights for policy 0, policy_version 653250 (0.0005) [2023-12-27 10:09:00,582][20381] Updated weights for policy 1, policy_version 652637 (0.0009) [2023-12-27 10:09:00,625][20347] Updated weights for policy 0, policy_version 653260 (0.0005) [2023-12-27 10:09:00,638][20381] Updated weights for policy 1, policy_version 652647 (0.0011) [2023-12-27 10:09:00,690][20381] Updated weights for policy 1, policy_version 652657 (0.0005) [2023-12-27 10:09:01,200][20347] Updated weights for policy 0, policy_version 653270 (0.0007) [2023-12-27 10:09:01,256][20347] Updated weights for policy 0, policy_version 653280 (0.0006) [2023-12-27 10:09:01,316][20347] Updated weights for policy 0, policy_version 653290 (0.0006) [2023-12-27 10:09:01,366][20381] Updated weights for policy 1, policy_version 652667 (0.0010) [2023-12-27 10:09:01,424][20381] Updated weights for policy 1, policy_version 652677 (0.0008) [2023-12-27 10:09:01,473][20381] Updated weights for policy 1, policy_version 652687 (0.0005) [2023-12-27 10:09:01,953][20347] Updated weights for policy 0, policy_version 653300 (0.0006) [2023-12-27 10:09:02,008][20347] Updated weights for policy 0, policy_version 653310 (0.0007) [2023-12-27 10:09:02,064][20347] Updated weights for policy 0, policy_version 653320 (0.0008) [2023-12-27 10:09:02,147][20381] Updated weights for policy 1, policy_version 652697 (0.0006) [2023-12-27 10:09:02,203][20381] Updated weights for policy 1, policy_version 652707 (0.0011) [2023-12-27 10:09:02,264][20381] Updated weights for policy 1, policy_version 652717 (0.0010) [2023-12-27 10:09:02,324][20381] Updated weights for policy 1, policy_version 652727 (0.0011) [2023-12-27 10:09:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 334397440. Throughput: 0: 10354.1, 1: 10346.4. Samples: 334402944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:02,363][19170] Avg episode reward: [(0, '707.870'), (1, '970.970')] [2023-12-27 10:09:02,786][20347] Updated weights for policy 0, policy_version 653330 (0.0007) [2023-12-27 10:09:02,840][20347] Updated weights for policy 0, policy_version 653340 (0.0006) [2023-12-27 10:09:02,893][20347] Updated weights for policy 0, policy_version 653350 (0.0007) [2023-12-27 10:09:02,951][20347] Updated weights for policy 0, policy_version 653360 (0.0008) [2023-12-27 10:09:02,951][20381] Updated weights for policy 1, policy_version 652737 (0.0010) [2023-12-27 10:09:03,007][20381] Updated weights for policy 1, policy_version 652747 (0.0010) [2023-12-27 10:09:03,068][20381] Updated weights for policy 1, policy_version 652757 (0.0011) [2023-12-27 10:09:03,528][20347] Updated weights for policy 0, policy_version 653370 (0.0007) [2023-12-27 10:09:03,581][20347] Updated weights for policy 0, policy_version 653380 (0.0008) [2023-12-27 10:09:03,637][20347] Updated weights for policy 0, policy_version 653390 (0.0007) [2023-12-27 10:09:03,748][20381] Updated weights for policy 1, policy_version 652767 (0.0010) [2023-12-27 10:09:03,804][20381] Updated weights for policy 1, policy_version 652777 (0.0010) [2023-12-27 10:09:03,868][20381] Updated weights for policy 1, policy_version 652787 (0.0009) [2023-12-27 10:09:04,301][20347] Updated weights for policy 0, policy_version 653400 (0.0007) [2023-12-27 10:09:04,365][20347] Updated weights for policy 0, policy_version 653410 (0.0007) [2023-12-27 10:09:04,423][20347] Updated weights for policy 0, policy_version 653420 (0.0006) [2023-12-27 10:09:04,589][20381] Updated weights for policy 1, policy_version 652797 (0.0009) [2023-12-27 10:09:04,641][20381] Updated weights for policy 1, policy_version 652807 (0.0010) [2023-12-27 10:09:04,697][20381] Updated weights for policy 1, policy_version 652817 (0.0010) [2023-12-27 10:09:04,973][20347] Updated weights for policy 0, policy_version 653430 (0.0005) [2023-12-27 10:09:05,020][20347] Updated weights for policy 0, policy_version 653440 (0.0005) [2023-12-27 10:09:05,067][20347] Updated weights for policy 0, policy_version 653450 (0.0005) [2023-12-27 10:09:05,333][20381] Updated weights for policy 1, policy_version 652827 (0.0009) [2023-12-27 10:09:05,387][20381] Updated weights for policy 1, policy_version 652837 (0.0005) [2023-12-27 10:09:05,433][20381] Updated weights for policy 1, policy_version 652847 (0.0005) [2023-12-27 10:09:05,723][20347] Updated weights for policy 0, policy_version 653460 (0.0007) [2023-12-27 10:09:05,781][20347] Updated weights for policy 0, policy_version 653470 (0.0010) [2023-12-27 10:09:05,835][20347] Updated weights for policy 0, policy_version 653480 (0.0010) [2023-12-27 10:09:05,973][20381] Updated weights for policy 1, policy_version 652857 (0.0005) [2023-12-27 10:09:06,023][20381] Updated weights for policy 1, policy_version 652867 (0.0005) [2023-12-27 10:09:06,069][20381] Updated weights for policy 1, policy_version 652877 (0.0005) [2023-12-27 10:09:06,126][20381] Updated weights for policy 1, policy_version 652887 (0.0006) [2023-12-27 10:09:06,486][20347] Updated weights for policy 0, policy_version 653490 (0.0010) [2023-12-27 10:09:06,549][20347] Updated weights for policy 0, policy_version 653500 (0.0011) [2023-12-27 10:09:06,606][20347] Updated weights for policy 0, policy_version 653510 (0.0008) [2023-12-27 10:09:06,666][20347] Updated weights for policy 0, policy_version 653520 (0.0010) [2023-12-27 10:09:06,817][20381] Updated weights for policy 1, policy_version 652897 (0.0009) [2023-12-27 10:09:06,875][20381] Updated weights for policy 1, policy_version 652907 (0.0008) [2023-12-27 10:09:06,940][20381] Updated weights for policy 1, policy_version 652917 (0.0008) [2023-12-27 10:09:07,249][20347] Updated weights for policy 0, policy_version 653530 (0.0010) [2023-12-27 10:09:07,294][20347] Updated weights for policy 0, policy_version 653540 (0.0010) [2023-12-27 10:09:07,340][20347] Updated weights for policy 0, policy_version 653550 (0.0010) [2023-12-27 10:09:07,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 334503936. Throughput: 0: 10417.3, 1: 10374.1. Samples: 334466636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:07,363][19170] Avg episode reward: [(0, '815.260'), (1, '990.310')] [2023-12-27 10:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000652920_167174144.pth... [2023-12-27 10:09:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000653552_167329792.pth... [2023-12-27 10:09:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000651704_166862848.pth [2023-12-27 10:09:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000652304_167010304.pth [2023-12-27 10:09:07,604][20381] Updated weights for policy 1, policy_version 652927 (0.0006) [2023-12-27 10:09:07,664][20381] Updated weights for policy 1, policy_version 652937 (0.0005) [2023-12-27 10:09:07,725][20381] Updated weights for policy 1, policy_version 652947 (0.0005) [2023-12-27 10:09:07,991][20347] Updated weights for policy 0, policy_version 653560 (0.0010) [2023-12-27 10:09:08,050][20347] Updated weights for policy 0, policy_version 653570 (0.0010) [2023-12-27 10:09:08,104][20347] Updated weights for policy 0, policy_version 653580 (0.0006) [2023-12-27 10:09:08,327][20381] Updated weights for policy 1, policy_version 652957 (0.0007) [2023-12-27 10:09:08,380][20381] Updated weights for policy 1, policy_version 652967 (0.0006) [2023-12-27 10:09:08,432][20381] Updated weights for policy 1, policy_version 652977 (0.0008) [2023-12-27 10:09:08,706][20347] Updated weights for policy 0, policy_version 653590 (0.0008) [2023-12-27 10:09:08,762][20347] Updated weights for policy 0, policy_version 653600 (0.0011) [2023-12-27 10:09:08,821][20347] Updated weights for policy 0, policy_version 653610 (0.0010) [2023-12-27 10:09:09,132][20381] Updated weights for policy 1, policy_version 652987 (0.0008) [2023-12-27 10:09:09,189][20381] Updated weights for policy 1, policy_version 652997 (0.0006) [2023-12-27 10:09:09,254][20381] Updated weights for policy 1, policy_version 653007 (0.0007) [2023-12-27 10:09:09,501][20347] Updated weights for policy 0, policy_version 653620 (0.0009) [2023-12-27 10:09:09,547][20347] Updated weights for policy 0, policy_version 653630 (0.0008) [2023-12-27 10:09:09,600][20347] Updated weights for policy 0, policy_version 653640 (0.0008) [2023-12-27 10:09:09,942][20381] Updated weights for policy 1, policy_version 653017 (0.0007) [2023-12-27 10:09:09,996][20381] Updated weights for policy 1, policy_version 653027 (0.0008) [2023-12-27 10:09:10,049][20381] Updated weights for policy 1, policy_version 653037 (0.0008) [2023-12-27 10:09:10,103][20381] Updated weights for policy 1, policy_version 653047 (0.0008) [2023-12-27 10:09:10,330][20347] Updated weights for policy 0, policy_version 653650 (0.0008) [2023-12-27 10:09:10,386][20347] Updated weights for policy 0, policy_version 653660 (0.0006) [2023-12-27 10:09:10,445][20347] Updated weights for policy 0, policy_version 653670 (0.0006) [2023-12-27 10:09:10,495][20347] Updated weights for policy 0, policy_version 653680 (0.0006) [2023-12-27 10:09:10,831][20381] Updated weights for policy 1, policy_version 653057 (0.0007) [2023-12-27 10:09:10,915][20381] Updated weights for policy 1, policy_version 653067 (0.0008) [2023-12-27 10:09:10,964][20381] Updated weights for policy 1, policy_version 653077 (0.0008) [2023-12-27 10:09:11,206][20347] Updated weights for policy 0, policy_version 653690 (0.0008) [2023-12-27 10:09:11,276][20347] Updated weights for policy 0, policy_version 653700 (0.0008) [2023-12-27 10:09:11,342][20347] Updated weights for policy 0, policy_version 653710 (0.0008) [2023-12-27 10:09:11,682][20381] Updated weights for policy 1, policy_version 653087 (0.0007) [2023-12-27 10:09:11,744][20381] Updated weights for policy 1, policy_version 653097 (0.0007) [2023-12-27 10:09:11,805][20381] Updated weights for policy 1, policy_version 653107 (0.0006) [2023-12-27 10:09:12,035][20347] Updated weights for policy 0, policy_version 653720 (0.0007) [2023-12-27 10:09:12,101][20347] Updated weights for policy 0, policy_version 653730 (0.0008) [2023-12-27 10:09:12,161][20347] Updated weights for policy 0, policy_version 653740 (0.0009) [2023-12-27 10:09:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 334602240. Throughput: 0: 10467.8, 1: 10377.4. Samples: 334592772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:12,363][19170] Avg episode reward: [(0, '737.050'), (1, '1000.000')] [2023-12-27 10:09:12,376][20381] Updated weights for policy 1, policy_version 653117 (0.0008) [2023-12-27 10:09:12,439][20381] Updated weights for policy 1, policy_version 653127 (0.0007) [2023-12-27 10:09:12,492][20381] Updated weights for policy 1, policy_version 653137 (0.0009) [2023-12-27 10:09:12,844][20347] Updated weights for policy 0, policy_version 653750 (0.0007) [2023-12-27 10:09:12,889][20347] Updated weights for policy 0, policy_version 653760 (0.0005) [2023-12-27 10:09:12,946][20347] Updated weights for policy 0, policy_version 653770 (0.0007) [2023-12-27 10:09:13,172][20381] Updated weights for policy 1, policy_version 653147 (0.0008) [2023-12-27 10:09:13,228][20381] Updated weights for policy 1, policy_version 653157 (0.0008) [2023-12-27 10:09:13,287][20381] Updated weights for policy 1, policy_version 653167 (0.0007) [2023-12-27 10:09:13,599][20347] Updated weights for policy 0, policy_version 653780 (0.0007) [2023-12-27 10:09:13,650][20347] Updated weights for policy 0, policy_version 653790 (0.0005) [2023-12-27 10:09:13,708][20347] Updated weights for policy 0, policy_version 653800 (0.0005) [2023-12-27 10:09:13,920][20381] Updated weights for policy 1, policy_version 653177 (0.0007) [2023-12-27 10:09:13,982][20381] Updated weights for policy 1, policy_version 653187 (0.0007) [2023-12-27 10:09:14,043][20381] Updated weights for policy 1, policy_version 653197 (0.0008) [2023-12-27 10:09:14,103][20381] Updated weights for policy 1, policy_version 653207 (0.0007) [2023-12-27 10:09:14,318][20347] Updated weights for policy 0, policy_version 653810 (0.0008) [2023-12-27 10:09:14,371][20347] Updated weights for policy 0, policy_version 653820 (0.0007) [2023-12-27 10:09:14,422][20347] Updated weights for policy 0, policy_version 653830 (0.0008) [2023-12-27 10:09:14,469][20347] Updated weights for policy 0, policy_version 653840 (0.0007) [2023-12-27 10:09:14,808][20381] Updated weights for policy 1, policy_version 653217 (0.0008) [2023-12-27 10:09:14,862][20381] Updated weights for policy 1, policy_version 653227 (0.0006) [2023-12-27 10:09:14,910][20381] Updated weights for policy 1, policy_version 653237 (0.0005) [2023-12-27 10:09:15,139][20347] Updated weights for policy 0, policy_version 653850 (0.0007) [2023-12-27 10:09:15,200][20347] Updated weights for policy 0, policy_version 653860 (0.0008) [2023-12-27 10:09:15,257][20347] Updated weights for policy 0, policy_version 653870 (0.0008) [2023-12-27 10:09:15,660][20381] Updated weights for policy 1, policy_version 653247 (0.0008) [2023-12-27 10:09:15,705][20381] Updated weights for policy 1, policy_version 653257 (0.0009) [2023-12-27 10:09:15,756][20381] Updated weights for policy 1, policy_version 653267 (0.0005) [2023-12-27 10:09:15,843][20347] Updated weights for policy 0, policy_version 653880 (0.0006) [2023-12-27 10:09:15,895][20347] Updated weights for policy 0, policy_version 653890 (0.0005) [2023-12-27 10:09:15,954][20347] Updated weights for policy 0, policy_version 653900 (0.0006) [2023-12-27 10:09:16,458][20381] Updated weights for policy 1, policy_version 653277 (0.0005) [2023-12-27 10:09:16,519][20381] Updated weights for policy 1, policy_version 653287 (0.0005) [2023-12-27 10:09:16,565][20347] Updated weights for policy 0, policy_version 653910 (0.0008) [2023-12-27 10:09:16,575][20381] Updated weights for policy 1, policy_version 653297 (0.0006) [2023-12-27 10:09:16,623][20347] Updated weights for policy 0, policy_version 653920 (0.0007) [2023-12-27 10:09:16,677][20347] Updated weights for policy 0, policy_version 653930 (0.0007) [2023-12-27 10:09:17,243][20381] Updated weights for policy 1, policy_version 653307 (0.0006) [2023-12-27 10:09:17,297][20381] Updated weights for policy 1, policy_version 653317 (0.0008) [2023-12-27 10:09:17,304][20347] Updated weights for policy 0, policy_version 653940 (0.0010) [2023-12-27 10:09:17,354][20381] Updated weights for policy 1, policy_version 653327 (0.0007) [2023-12-27 10:09:17,359][20347] Updated weights for policy 0, policy_version 653950 (0.0010) [2023-12-27 10:09:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.6, 300 sec: 20688.3). Total num frames: 334700544. Throughput: 0: 10494.3, 1: 10349.2. Samples: 334717720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:17,363][19170] Avg episode reward: [(0, '767.400'), (1, '990.360')] [2023-12-27 10:09:17,417][20347] Updated weights for policy 0, policy_version 653960 (0.0010) [2023-12-27 10:09:18,059][20381] Updated weights for policy 1, policy_version 653337 (0.0006) [2023-12-27 10:09:18,087][20347] Updated weights for policy 0, policy_version 653970 (0.0007) [2023-12-27 10:09:18,118][20381] Updated weights for policy 1, policy_version 653347 (0.0006) [2023-12-27 10:09:18,142][20347] Updated weights for policy 0, policy_version 653980 (0.0010) [2023-12-27 10:09:18,173][20381] Updated weights for policy 1, policy_version 653357 (0.0005) [2023-12-27 10:09:18,193][20347] Updated weights for policy 0, policy_version 653990 (0.0010) [2023-12-27 10:09:18,232][20381] Updated weights for policy 1, policy_version 653367 (0.0005) [2023-12-27 10:09:18,247][20347] Updated weights for policy 0, policy_version 654000 (0.0010) [2023-12-27 10:09:18,895][20347] Updated weights for policy 0, policy_version 654010 (0.0005) [2023-12-27 10:09:18,931][20381] Updated weights for policy 1, policy_version 653377 (0.0007) [2023-12-27 10:09:18,948][20347] Updated weights for policy 0, policy_version 654020 (0.0006) [2023-12-27 10:09:18,989][20381] Updated weights for policy 1, policy_version 653387 (0.0008) [2023-12-27 10:09:19,013][20347] Updated weights for policy 0, policy_version 654030 (0.0007) [2023-12-27 10:09:19,048][20381] Updated weights for policy 1, policy_version 653397 (0.0007) [2023-12-27 10:09:19,725][20347] Updated weights for policy 0, policy_version 654040 (0.0007) [2023-12-27 10:09:19,752][20381] Updated weights for policy 1, policy_version 653407 (0.0007) [2023-12-27 10:09:19,775][20347] Updated weights for policy 0, policy_version 654050 (0.0008) [2023-12-27 10:09:19,813][20381] Updated weights for policy 1, policy_version 653417 (0.0006) [2023-12-27 10:09:19,834][20347] Updated weights for policy 0, policy_version 654060 (0.0007) [2023-12-27 10:09:19,875][20381] Updated weights for policy 1, policy_version 653427 (0.0007) [2023-12-27 10:09:20,586][20347] Updated weights for policy 0, policy_version 654070 (0.0007) [2023-12-27 10:09:20,621][20381] Updated weights for policy 1, policy_version 653437 (0.0008) [2023-12-27 10:09:20,645][20347] Updated weights for policy 0, policy_version 654080 (0.0008) [2023-12-27 10:09:20,678][20381] Updated weights for policy 1, policy_version 653447 (0.0007) [2023-12-27 10:09:20,707][20347] Updated weights for policy 0, policy_version 654090 (0.0007) [2023-12-27 10:09:20,741][20381] Updated weights for policy 1, policy_version 653457 (0.0006) [2023-12-27 10:09:21,378][20347] Updated weights for policy 0, policy_version 654101 (0.0008) [2023-12-27 10:09:21,436][20347] Updated weights for policy 0, policy_version 654111 (0.0010) [2023-12-27 10:09:21,471][20381] Updated weights for policy 1, policy_version 653467 (0.0008) [2023-12-27 10:09:21,494][20347] Updated weights for policy 0, policy_version 654121 (0.0010) [2023-12-27 10:09:21,531][20381] Updated weights for policy 1, policy_version 653477 (0.0011) [2023-12-27 10:09:21,590][20381] Updated weights for policy 1, policy_version 653487 (0.0011) [2023-12-27 10:09:22,190][20347] Updated weights for policy 0, policy_version 654131 (0.0009) [2023-12-27 10:09:22,244][20381] Updated weights for policy 1, policy_version 653497 (0.0009) [2023-12-27 10:09:22,245][20347] Updated weights for policy 0, policy_version 654141 (0.0006) [2023-12-27 10:09:22,303][20381] Updated weights for policy 1, policy_version 653507 (0.0006) [2023-12-27 10:09:22,305][20347] Updated weights for policy 0, policy_version 654151 (0.0008) [2023-12-27 10:09:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 334807040. Throughput: 0: 10490.7, 1: 10296.2. Samples: 334777448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:22,363][20381] Updated weights for policy 1, policy_version 653517 (0.0007) [2023-12-27 10:09:22,363][19170] Avg episode reward: [(0, '708.750'), (1, '980.770')] [2023-12-27 10:09:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000654160_167485440.pth... [2023-12-27 10:09:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000652912_167165952.pth [2023-12-27 10:09:22,428][20381] Updated weights for policy 1, policy_version 653527 (0.0009) [2023-12-27 10:09:22,432][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000653528_167329792.pth... [2023-12-27 10:09:22,437][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000652312_167018496.pth [2023-12-27 10:09:22,937][20347] Updated weights for policy 0, policy_version 654161 (0.0008) [2023-12-27 10:09:22,993][20347] Updated weights for policy 0, policy_version 654171 (0.0010) [2023-12-27 10:09:23,046][20347] Updated weights for policy 0, policy_version 654181 (0.0010) [2023-12-27 10:09:23,096][20347] Updated weights for policy 0, policy_version 654191 (0.0010) [2023-12-27 10:09:23,154][20381] Updated weights for policy 1, policy_version 653537 (0.0007) [2023-12-27 10:09:23,214][20381] Updated weights for policy 1, policy_version 653547 (0.0008) [2023-12-27 10:09:23,262][20381] Updated weights for policy 1, policy_version 653557 (0.0007) [2023-12-27 10:09:23,791][20347] Updated weights for policy 0, policy_version 654201 (0.0010) [2023-12-27 10:09:23,849][20347] Updated weights for policy 0, policy_version 654211 (0.0007) [2023-12-27 10:09:23,907][20347] Updated weights for policy 0, policy_version 654221 (0.0005) [2023-12-27 10:09:23,910][20381] Updated weights for policy 1, policy_version 653567 (0.0010) [2023-12-27 10:09:23,964][20381] Updated weights for policy 1, policy_version 653577 (0.0008) [2023-12-27 10:09:24,017][20381] Updated weights for policy 1, policy_version 653587 (0.0009) [2023-12-27 10:09:24,570][20347] Updated weights for policy 0, policy_version 654231 (0.0007) [2023-12-27 10:09:24,624][20347] Updated weights for policy 0, policy_version 654241 (0.0007) [2023-12-27 10:09:24,677][20347] Updated weights for policy 0, policy_version 654251 (0.0008) [2023-12-27 10:09:24,694][20381] Updated weights for policy 1, policy_version 653597 (0.0010) [2023-12-27 10:09:24,748][20381] Updated weights for policy 1, policy_version 653607 (0.0010) [2023-12-27 10:09:24,806][20381] Updated weights for policy 1, policy_version 653617 (0.0010) [2023-12-27 10:09:25,333][20347] Updated weights for policy 0, policy_version 654261 (0.0006) [2023-12-27 10:09:25,388][20347] Updated weights for policy 0, policy_version 654271 (0.0005) [2023-12-27 10:09:25,444][20347] Updated weights for policy 0, policy_version 654281 (0.0007) [2023-12-27 10:09:25,486][20381] Updated weights for policy 1, policy_version 653627 (0.0010) [2023-12-27 10:09:25,538][20381] Updated weights for policy 1, policy_version 653637 (0.0010) [2023-12-27 10:09:25,589][20381] Updated weights for policy 1, policy_version 653647 (0.0010) [2023-12-27 10:09:26,111][20347] Updated weights for policy 0, policy_version 654291 (0.0008) [2023-12-27 10:09:26,161][20347] Updated weights for policy 0, policy_version 654301 (0.0008) [2023-12-27 10:09:26,195][20381] Updated weights for policy 1, policy_version 653657 (0.0010) [2023-12-27 10:09:26,209][20347] Updated weights for policy 0, policy_version 654311 (0.0007) [2023-12-27 10:09:26,246][20381] Updated weights for policy 1, policy_version 653667 (0.0007) [2023-12-27 10:09:26,299][20381] Updated weights for policy 1, policy_version 653677 (0.0005) [2023-12-27 10:09:26,356][20381] Updated weights for policy 1, policy_version 653687 (0.0007) [2023-12-27 10:09:26,926][20347] Updated weights for policy 0, policy_version 654321 (0.0007) [2023-12-27 10:09:26,979][20347] Updated weights for policy 0, policy_version 654331 (0.0008) [2023-12-27 10:09:26,987][20381] Updated weights for policy 1, policy_version 653697 (0.0005) [2023-12-27 10:09:27,038][20381] Updated weights for policy 1, policy_version 653707 (0.0005) [2023-12-27 10:09:27,041][20347] Updated weights for policy 0, policy_version 654341 (0.0007) [2023-12-27 10:09:27,087][20381] Updated weights for policy 1, policy_version 653717 (0.0006) [2023-12-27 10:09:27,089][20347] Updated weights for policy 0, policy_version 654351 (0.0008) [2023-12-27 10:09:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 334913536. Throughput: 0: 10459.6, 1: 10274.4. Samples: 334900592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:27,363][19170] Avg episode reward: [(0, '707.220'), (1, '990.410')] [2023-12-27 10:09:27,673][20381] Updated weights for policy 1, policy_version 653727 (0.0006) [2023-12-27 10:09:27,725][20381] Updated weights for policy 1, policy_version 653737 (0.0006) [2023-12-27 10:09:27,774][20381] Updated weights for policy 1, policy_version 653747 (0.0005) [2023-12-27 10:09:27,830][20347] Updated weights for policy 0, policy_version 654361 (0.0007) [2023-12-27 10:09:27,884][20347] Updated weights for policy 0, policy_version 654371 (0.0008) [2023-12-27 10:09:27,941][20347] Updated weights for policy 0, policy_version 654381 (0.0007) [2023-12-27 10:09:28,321][20381] Updated weights for policy 1, policy_version 653757 (0.0005) [2023-12-27 10:09:28,382][20381] Updated weights for policy 1, policy_version 653767 (0.0007) [2023-12-27 10:09:28,441][20381] Updated weights for policy 1, policy_version 653777 (0.0008) [2023-12-27 10:09:28,638][20347] Updated weights for policy 0, policy_version 654391 (0.0008) [2023-12-27 10:09:28,689][20347] Updated weights for policy 0, policy_version 654401 (0.0008) [2023-12-27 10:09:28,748][20347] Updated weights for policy 0, policy_version 654411 (0.0007) [2023-12-27 10:09:29,040][20381] Updated weights for policy 1, policy_version 653787 (0.0007) [2023-12-27 10:09:29,090][20381] Updated weights for policy 1, policy_version 653797 (0.0006) [2023-12-27 10:09:29,144][20381] Updated weights for policy 1, policy_version 653807 (0.0007) [2023-12-27 10:09:29,474][20347] Updated weights for policy 0, policy_version 654421 (0.0008) [2023-12-27 10:09:29,520][20347] Updated weights for policy 0, policy_version 654431 (0.0008) [2023-12-27 10:09:29,579][20347] Updated weights for policy 0, policy_version 654441 (0.0007) [2023-12-27 10:09:29,803][20381] Updated weights for policy 1, policy_version 653817 (0.0007) [2023-12-27 10:09:29,870][20381] Updated weights for policy 1, policy_version 653827 (0.0008) [2023-12-27 10:09:29,931][20381] Updated weights for policy 1, policy_version 653837 (0.0009) [2023-12-27 10:09:29,990][20381] Updated weights for policy 1, policy_version 653847 (0.0008) [2023-12-27 10:09:30,325][20347] Updated weights for policy 0, policy_version 654451 (0.0008) [2023-12-27 10:09:30,377][20347] Updated weights for policy 0, policy_version 654461 (0.0008) [2023-12-27 10:09:30,426][20347] Updated weights for policy 0, policy_version 654471 (0.0008) [2023-12-27 10:09:30,664][20381] Updated weights for policy 1, policy_version 653857 (0.0010) [2023-12-27 10:09:30,713][20381] Updated weights for policy 1, policy_version 653867 (0.0010) [2023-12-27 10:09:30,768][20381] Updated weights for policy 1, policy_version 653877 (0.0010) [2023-12-27 10:09:31,149][20347] Updated weights for policy 0, policy_version 654481 (0.0007) [2023-12-27 10:09:31,205][20347] Updated weights for policy 0, policy_version 654491 (0.0008) [2023-12-27 10:09:31,263][20347] Updated weights for policy 0, policy_version 654501 (0.0007) [2023-12-27 10:09:31,321][20347] Updated weights for policy 0, policy_version 654511 (0.0007) [2023-12-27 10:09:31,463][20381] Updated weights for policy 1, policy_version 653887 (0.0010) [2023-12-27 10:09:31,516][20381] Updated weights for policy 1, policy_version 653897 (0.0011) [2023-12-27 10:09:31,576][20381] Updated weights for policy 1, policy_version 653907 (0.0010) [2023-12-27 10:09:32,044][20347] Updated weights for policy 0, policy_version 654521 (0.0007) [2023-12-27 10:09:32,098][20347] Updated weights for policy 0, policy_version 654531 (0.0007) [2023-12-27 10:09:32,149][20347] Updated weights for policy 0, policy_version 654541 (0.0008) [2023-12-27 10:09:32,257][20381] Updated weights for policy 1, policy_version 653917 (0.0010) [2023-12-27 10:09:32,315][20381] Updated weights for policy 1, policy_version 653927 (0.0010) [2023-12-27 10:09:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 335011840. Throughput: 0: 10412.2, 1: 10314.6. Samples: 335024080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:32,363][19170] Avg episode reward: [(0, '675.040'), (1, '1000.000')] [2023-12-27 10:09:32,374][20381] Updated weights for policy 1, policy_version 653937 (0.0011) [2023-12-27 10:09:32,865][20347] Updated weights for policy 0, policy_version 654551 (0.0007) [2023-12-27 10:09:32,910][20347] Updated weights for policy 0, policy_version 654561 (0.0008) [2023-12-27 10:09:32,957][20347] Updated weights for policy 0, policy_version 654571 (0.0007) [2023-12-27 10:09:33,047][20381] Updated weights for policy 1, policy_version 653947 (0.0009) [2023-12-27 10:09:33,101][20381] Updated weights for policy 1, policy_version 653957 (0.0005) [2023-12-27 10:09:33,155][20381] Updated weights for policy 1, policy_version 653967 (0.0007) [2023-12-27 10:09:33,541][20347] Updated weights for policy 0, policy_version 654581 (0.0006) [2023-12-27 10:09:33,598][20347] Updated weights for policy 0, policy_version 654591 (0.0005) [2023-12-27 10:09:33,666][20347] Updated weights for policy 0, policy_version 654601 (0.0005) [2023-12-27 10:09:33,778][20381] Updated weights for policy 1, policy_version 653977 (0.0007) [2023-12-27 10:09:33,833][20381] Updated weights for policy 1, policy_version 653987 (0.0005) [2023-12-27 10:09:33,894][20381] Updated weights for policy 1, policy_version 653997 (0.0005) [2023-12-27 10:09:33,958][20381] Updated weights for policy 1, policy_version 654007 (0.0005) [2023-12-27 10:09:34,251][20347] Updated weights for policy 0, policy_version 654611 (0.0006) [2023-12-27 10:09:34,313][20347] Updated weights for policy 0, policy_version 654621 (0.0008) [2023-12-27 10:09:34,373][20347] Updated weights for policy 0, policy_version 654631 (0.0008) [2023-12-27 10:09:34,518][20381] Updated weights for policy 1, policy_version 654017 (0.0007) [2023-12-27 10:09:34,567][20381] Updated weights for policy 1, policy_version 654027 (0.0008) [2023-12-27 10:09:34,618][20381] Updated weights for policy 1, policy_version 654037 (0.0007) [2023-12-27 10:09:35,088][20347] Updated weights for policy 0, policy_version 654641 (0.0008) [2023-12-27 10:09:35,136][20347] Updated weights for policy 0, policy_version 654651 (0.0008) [2023-12-27 10:09:35,185][20347] Updated weights for policy 0, policy_version 654661 (0.0008) [2023-12-27 10:09:35,237][20347] Updated weights for policy 0, policy_version 654671 (0.0009) [2023-12-27 10:09:35,280][20381] Updated weights for policy 1, policy_version 654047 (0.0005) [2023-12-27 10:09:35,344][20381] Updated weights for policy 1, policy_version 654057 (0.0006) [2023-12-27 10:09:35,403][20381] Updated weights for policy 1, policy_version 654067 (0.0007) [2023-12-27 10:09:35,894][20347] Updated weights for policy 0, policy_version 654681 (0.0008) [2023-12-27 10:09:35,949][20347] Updated weights for policy 0, policy_version 654691 (0.0008) [2023-12-27 10:09:36,004][20347] Updated weights for policy 0, policy_version 654701 (0.0007) [2023-12-27 10:09:36,051][20381] Updated weights for policy 1, policy_version 654077 (0.0006) [2023-12-27 10:09:36,108][20381] Updated weights for policy 1, policy_version 654087 (0.0008) [2023-12-27 10:09:36,168][20381] Updated weights for policy 1, policy_version 654097 (0.0007) [2023-12-27 10:09:36,745][20347] Updated weights for policy 0, policy_version 654711 (0.0007) [2023-12-27 10:09:36,775][20381] Updated weights for policy 1, policy_version 654107 (0.0006) [2023-12-27 10:09:36,805][20347] Updated weights for policy 0, policy_version 654721 (0.0008) [2023-12-27 10:09:36,840][20381] Updated weights for policy 1, policy_version 654117 (0.0005) [2023-12-27 10:09:36,869][20347] Updated weights for policy 0, policy_version 654731 (0.0008) [2023-12-27 10:09:36,901][20381] Updated weights for policy 1, policy_version 654127 (0.0005) [2023-12-27 10:09:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20688.3). Total num frames: 335118336. Throughput: 0: 10453.4, 1: 10358.7. Samples: 335088660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:37,363][19170] Avg episode reward: [(0, '284.400'), (1, '1000.000')] [2023-12-27 10:09:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000654136_167485440.pth... [2023-12-27 10:09:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000654736_167632896.pth... [2023-12-27 10:09:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000652920_167174144.pth [2023-12-27 10:09:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000653552_167329792.pth [2023-12-27 10:09:37,548][20381] Updated weights for policy 1, policy_version 654137 (0.0008) [2023-12-27 10:09:37,579][20347] Updated weights for policy 0, policy_version 654741 (0.0007) [2023-12-27 10:09:37,611][20381] Updated weights for policy 1, policy_version 654147 (0.0006) [2023-12-27 10:09:37,633][20347] Updated weights for policy 0, policy_version 654751 (0.0006) [2023-12-27 10:09:37,666][20381] Updated weights for policy 1, policy_version 654157 (0.0006) [2023-12-27 10:09:37,690][20347] Updated weights for policy 0, policy_version 654761 (0.0008) [2023-12-27 10:09:37,725][20381] Updated weights for policy 1, policy_version 654167 (0.0006) [2023-12-27 10:09:38,391][20347] Updated weights for policy 0, policy_version 654771 (0.0006) [2023-12-27 10:09:38,415][20381] Updated weights for policy 1, policy_version 654177 (0.0006) [2023-12-27 10:09:38,448][20347] Updated weights for policy 0, policy_version 654781 (0.0007) [2023-12-27 10:09:38,474][20381] Updated weights for policy 1, policy_version 654187 (0.0006) [2023-12-27 10:09:38,498][20347] Updated weights for policy 0, policy_version 654791 (0.0006) [2023-12-27 10:09:38,530][20381] Updated weights for policy 1, policy_version 654197 (0.0006) [2023-12-27 10:09:39,142][20347] Updated weights for policy 0, policy_version 654801 (0.0005) [2023-12-27 10:09:39,205][20347] Updated weights for policy 0, policy_version 654811 (0.0005) [2023-12-27 10:09:39,243][20381] Updated weights for policy 1, policy_version 654207 (0.0007) [2023-12-27 10:09:39,263][20347] Updated weights for policy 0, policy_version 654821 (0.0007) [2023-12-27 10:09:39,297][20381] Updated weights for policy 1, policy_version 654217 (0.0006) [2023-12-27 10:09:39,322][20347] Updated weights for policy 0, policy_version 654831 (0.0007) [2023-12-27 10:09:39,360][20381] Updated weights for policy 1, policy_version 654227 (0.0007) [2023-12-27 10:09:39,990][20347] Updated weights for policy 0, policy_version 654841 (0.0010) [2023-12-27 10:09:40,000][20381] Updated weights for policy 1, policy_version 654237 (0.0007) [2023-12-27 10:09:40,051][20347] Updated weights for policy 0, policy_version 654851 (0.0010) [2023-12-27 10:09:40,056][20381] Updated weights for policy 1, policy_version 654247 (0.0009) [2023-12-27 10:09:40,111][20347] Updated weights for policy 0, policy_version 654861 (0.0010) [2023-12-27 10:09:40,113][20381] Updated weights for policy 1, policy_version 654257 (0.0009) [2023-12-27 10:09:40,768][20347] Updated weights for policy 0, policy_version 654871 (0.0010) [2023-12-27 10:09:40,829][20381] Updated weights for policy 1, policy_version 654267 (0.0009) [2023-12-27 10:09:40,829][20347] Updated weights for policy 0, policy_version 654881 (0.0009) [2023-12-27 10:09:40,881][20381] Updated weights for policy 1, policy_version 654277 (0.0008) [2023-12-27 10:09:40,895][20347] Updated weights for policy 0, policy_version 654891 (0.0010) [2023-12-27 10:09:40,940][20381] Updated weights for policy 1, policy_version 654287 (0.0011) [2023-12-27 10:09:41,630][20381] Updated weights for policy 1, policy_version 654297 (0.0009) [2023-12-27 10:09:41,696][20347] Updated weights for policy 0, policy_version 654901 (0.0010) [2023-12-27 10:09:41,698][20381] Updated weights for policy 1, policy_version 654307 (0.0007) [2023-12-27 10:09:41,754][20347] Updated weights for policy 0, policy_version 654911 (0.0011) [2023-12-27 10:09:41,757][20381] Updated weights for policy 1, policy_version 654317 (0.0008) [2023-12-27 10:09:41,808][20381] Updated weights for policy 1, policy_version 654327 (0.0008) [2023-12-27 10:09:41,814][20347] Updated weights for policy 0, policy_version 654921 (0.0011) [2023-12-27 10:09:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 335216640. Throughput: 0: 10372.8, 1: 10369.8. Samples: 335210820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:42,363][19170] Avg episode reward: [(0, '166.980'), (1, '1000.000')] [2023-12-27 10:09:42,490][20381] Updated weights for policy 1, policy_version 654337 (0.0008) [2023-12-27 10:09:42,503][20347] Updated weights for policy 0, policy_version 654931 (0.0010) [2023-12-27 10:09:42,553][20381] Updated weights for policy 1, policy_version 654347 (0.0011) [2023-12-27 10:09:42,559][20347] Updated weights for policy 0, policy_version 654941 (0.0010) [2023-12-27 10:09:42,606][20381] Updated weights for policy 1, policy_version 654357 (0.0011) [2023-12-27 10:09:42,611][20347] Updated weights for policy 0, policy_version 654951 (0.0009) [2023-12-27 10:09:43,210][20347] Updated weights for policy 0, policy_version 654961 (0.0005) [2023-12-27 10:09:43,223][20381] Updated weights for policy 1, policy_version 654367 (0.0009) [2023-12-27 10:09:43,269][20347] Updated weights for policy 0, policy_version 654971 (0.0006) [2023-12-27 10:09:43,280][20381] Updated weights for policy 1, policy_version 654377 (0.0007) [2023-12-27 10:09:43,328][20347] Updated weights for policy 0, policy_version 654981 (0.0006) [2023-12-27 10:09:43,330][20381] Updated weights for policy 1, policy_version 654387 (0.0005) [2023-12-27 10:09:43,377][20347] Updated weights for policy 0, policy_version 654991 (0.0007) [2023-12-27 10:09:43,978][20381] Updated weights for policy 1, policy_version 654397 (0.0006) [2023-12-27 10:09:44,034][20381] Updated weights for policy 1, policy_version 654407 (0.0005) [2023-12-27 10:09:44,052][20347] Updated weights for policy 0, policy_version 655001 (0.0005) [2023-12-27 10:09:44,079][20381] Updated weights for policy 1, policy_version 654417 (0.0006) [2023-12-27 10:09:44,108][20347] Updated weights for policy 0, policy_version 655011 (0.0005) [2023-12-27 10:09:44,164][20347] Updated weights for policy 0, policy_version 655021 (0.0007) [2023-12-27 10:09:44,750][20381] Updated weights for policy 1, policy_version 654427 (0.0006) [2023-12-27 10:09:44,764][20347] Updated weights for policy 0, policy_version 655031 (0.0007) [2023-12-27 10:09:44,816][20381] Updated weights for policy 1, policy_version 654437 (0.0007) [2023-12-27 10:09:44,826][20347] Updated weights for policy 0, policy_version 655041 (0.0007) [2023-12-27 10:09:44,872][20381] Updated weights for policy 1, policy_version 654447 (0.0006) [2023-12-27 10:09:44,883][20347] Updated weights for policy 0, policy_version 655051 (0.0005) [2023-12-27 10:09:45,575][20347] Updated weights for policy 0, policy_version 655061 (0.0006) [2023-12-27 10:09:45,577][20381] Updated weights for policy 1, policy_version 654457 (0.0008) [2023-12-27 10:09:45,626][20347] Updated weights for policy 0, policy_version 655071 (0.0010) [2023-12-27 10:09:45,632][20381] Updated weights for policy 1, policy_version 654467 (0.0006) [2023-12-27 10:09:45,687][20347] Updated weights for policy 0, policy_version 655081 (0.0010) [2023-12-27 10:09:45,690][20381] Updated weights for policy 1, policy_version 654477 (0.0005) [2023-12-27 10:09:45,742][20381] Updated weights for policy 1, policy_version 654487 (0.0006) [2023-12-27 10:09:46,366][20347] Updated weights for policy 0, policy_version 655091 (0.0010) [2023-12-27 10:09:46,428][20347] Updated weights for policy 0, policy_version 655101 (0.0009) [2023-12-27 10:09:46,437][20381] Updated weights for policy 1, policy_version 654497 (0.0006) [2023-12-27 10:09:46,474][20347] Updated weights for policy 0, policy_version 655111 (0.0010) [2023-12-27 10:09:46,497][20381] Updated weights for policy 1, policy_version 654507 (0.0007) [2023-12-27 10:09:46,558][20381] Updated weights for policy 1, policy_version 654517 (0.0009) [2023-12-27 10:09:47,114][20347] Updated weights for policy 0, policy_version 655121 (0.0010) [2023-12-27 10:09:47,173][20347] Updated weights for policy 0, policy_version 655131 (0.0005) [2023-12-27 10:09:47,229][20347] Updated weights for policy 0, policy_version 655141 (0.0006) [2023-12-27 10:09:47,233][20381] Updated weights for policy 1, policy_version 654527 (0.0007) [2023-12-27 10:09:47,279][20347] Updated weights for policy 0, policy_version 655151 (0.0005) [2023-12-27 10:09:47,294][20381] Updated weights for policy 1, policy_version 654537 (0.0007) [2023-12-27 10:09:47,350][20381] Updated weights for policy 1, policy_version 654547 (0.0008) [2023-12-27 10:09:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 335323136. Throughput: 0: 10343.5, 1: 10365.2. Samples: 335334832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:47,363][19170] Avg episode reward: [(0, '207.100'), (1, '980.650')] [2023-12-27 10:09:47,899][20347] Updated weights for policy 0, policy_version 655161 (0.0008) [2023-12-27 10:09:47,940][20381] Updated weights for policy 1, policy_version 654557 (0.0006) [2023-12-27 10:09:47,959][20347] Updated weights for policy 0, policy_version 655171 (0.0009) [2023-12-27 10:09:47,990][20381] Updated weights for policy 1, policy_version 654567 (0.0007) [2023-12-27 10:09:48,006][20347] Updated weights for policy 0, policy_version 655181 (0.0006) [2023-12-27 10:09:48,043][20381] Updated weights for policy 1, policy_version 654577 (0.0011) [2023-12-27 10:09:48,657][20381] Updated weights for policy 1, policy_version 654587 (0.0009) [2023-12-27 10:09:48,719][20381] Updated weights for policy 1, policy_version 654597 (0.0005) [2023-12-27 10:09:48,761][20347] Updated weights for policy 0, policy_version 655191 (0.0007) [2023-12-27 10:09:48,781][20381] Updated weights for policy 1, policy_version 654607 (0.0007) [2023-12-27 10:09:48,818][20347] Updated weights for policy 0, policy_version 655201 (0.0007) [2023-12-27 10:09:48,868][20347] Updated weights for policy 0, policy_version 655211 (0.0008) [2023-12-27 10:09:49,467][20381] Updated weights for policy 1, policy_version 654617 (0.0009) [2023-12-27 10:09:49,519][20381] Updated weights for policy 1, policy_version 654627 (0.0007) [2023-12-27 10:09:49,574][20381] Updated weights for policy 1, policy_version 654637 (0.0009) [2023-12-27 10:09:49,610][20347] Updated weights for policy 0, policy_version 655221 (0.0006) [2023-12-27 10:09:49,630][20381] Updated weights for policy 1, policy_version 654647 (0.0011) [2023-12-27 10:09:49,652][20347] Updated weights for policy 0, policy_version 655231 (0.0006) [2023-12-27 10:09:49,699][20347] Updated weights for policy 0, policy_version 655241 (0.0007) [2023-12-27 10:09:50,332][20381] Updated weights for policy 1, policy_version 654657 (0.0011) [2023-12-27 10:09:50,393][20381] Updated weights for policy 1, policy_version 654667 (0.0007) [2023-12-27 10:09:50,436][20347] Updated weights for policy 0, policy_version 655251 (0.0007) [2023-12-27 10:09:50,443][20381] Updated weights for policy 1, policy_version 654677 (0.0009) [2023-12-27 10:09:50,490][20347] Updated weights for policy 0, policy_version 655261 (0.0007) [2023-12-27 10:09:50,544][20347] Updated weights for policy 0, policy_version 655271 (0.0007) [2023-12-27 10:09:51,061][20381] Updated weights for policy 1, policy_version 654687 (0.0009) [2023-12-27 10:09:51,133][20381] Updated weights for policy 1, policy_version 654697 (0.0009) [2023-12-27 10:09:51,203][20381] Updated weights for policy 1, policy_version 654707 (0.0011) [2023-12-27 10:09:51,257][20347] Updated weights for policy 0, policy_version 655281 (0.0006) [2023-12-27 10:09:51,320][20347] Updated weights for policy 0, policy_version 655291 (0.0007) [2023-12-27 10:09:51,382][20347] Updated weights for policy 0, policy_version 655301 (0.0008) [2023-12-27 10:09:51,449][20347] Updated weights for policy 0, policy_version 655311 (0.0008) [2023-12-27 10:09:51,875][20381] Updated weights for policy 1, policy_version 654717 (0.0007) [2023-12-27 10:09:51,925][20381] Updated weights for policy 1, policy_version 654727 (0.0009) [2023-12-27 10:09:51,979][20381] Updated weights for policy 1, policy_version 654737 (0.0009) [2023-12-27 10:09:52,092][20347] Updated weights for policy 0, policy_version 655321 (0.0006) [2023-12-27 10:09:52,150][20347] Updated weights for policy 0, policy_version 655331 (0.0005) [2023-12-27 10:09:52,206][20347] Updated weights for policy 0, policy_version 655341 (0.0006) [2023-12-27 10:09:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 335429632. Throughput: 0: 10277.8, 1: 10382.5. Samples: 335396348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:52,363][19170] Avg episode reward: [(0, '129.320'), (1, '979.310')] [2023-12-27 10:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000654744_167641088.pth... [2023-12-27 10:09:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000655344_167788544.pth... [2023-12-27 10:09:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000653528_167329792.pth [2023-12-27 10:09:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000654160_167485440.pth [2023-12-27 10:09:52,657][20381] Updated weights for policy 1, policy_version 654747 (0.0009) [2023-12-27 10:09:52,722][20381] Updated weights for policy 1, policy_version 654757 (0.0005) [2023-12-27 10:09:52,780][20381] Updated weights for policy 1, policy_version 654767 (0.0007) [2023-12-27 10:09:52,810][20347] Updated weights for policy 0, policy_version 655351 (0.0006) [2023-12-27 10:09:52,866][20347] Updated weights for policy 0, policy_version 655361 (0.0007) [2023-12-27 10:09:52,911][20347] Updated weights for policy 0, policy_version 655371 (0.0007) [2023-12-27 10:09:53,357][20381] Updated weights for policy 1, policy_version 654777 (0.0008) [2023-12-27 10:09:53,409][20381] Updated weights for policy 1, policy_version 654787 (0.0008) [2023-12-27 10:09:53,461][20381] Updated weights for policy 1, policy_version 654797 (0.0009) [2023-12-27 10:09:53,513][20381] Updated weights for policy 1, policy_version 654807 (0.0008) [2023-12-27 10:09:53,646][20347] Updated weights for policy 0, policy_version 655381 (0.0010) [2023-12-27 10:09:53,711][20347] Updated weights for policy 0, policy_version 655391 (0.0010) [2023-12-27 10:09:53,767][20347] Updated weights for policy 0, policy_version 655401 (0.0010) [2023-12-27 10:09:54,095][20381] Updated weights for policy 1, policy_version 654817 (0.0005) [2023-12-27 10:09:54,152][20381] Updated weights for policy 1, policy_version 654827 (0.0006) [2023-12-27 10:09:54,199][20381] Updated weights for policy 1, policy_version 654837 (0.0006) [2023-12-27 10:09:54,399][20347] Updated weights for policy 0, policy_version 655411 (0.0009) [2023-12-27 10:09:54,462][20347] Updated weights for policy 0, policy_version 655421 (0.0005) [2023-12-27 10:09:54,518][20347] Updated weights for policy 0, policy_version 655431 (0.0005) [2023-12-27 10:09:54,797][20381] Updated weights for policy 1, policy_version 654847 (0.0006) [2023-12-27 10:09:54,853][20381] Updated weights for policy 1, policy_version 654857 (0.0005) [2023-12-27 10:09:54,901][20381] Updated weights for policy 1, policy_version 654867 (0.0005) [2023-12-27 10:09:55,070][20347] Updated weights for policy 0, policy_version 655441 (0.0006) [2023-12-27 10:09:55,125][20347] Updated weights for policy 0, policy_version 655451 (0.0006) [2023-12-27 10:09:55,182][20347] Updated weights for policy 0, policy_version 655461 (0.0007) [2023-12-27 10:09:55,230][20347] Updated weights for policy 0, policy_version 655471 (0.0007) [2023-12-27 10:09:55,523][20381] Updated weights for policy 1, policy_version 654877 (0.0007) [2023-12-27 10:09:55,565][20381] Updated weights for policy 1, policy_version 654887 (0.0005) [2023-12-27 10:09:55,611][20381] Updated weights for policy 1, policy_version 654897 (0.0005) [2023-12-27 10:09:55,858][20347] Updated weights for policy 0, policy_version 655481 (0.0008) [2023-12-27 10:09:55,910][20347] Updated weights for policy 0, policy_version 655491 (0.0008) [2023-12-27 10:09:55,963][20347] Updated weights for policy 0, policy_version 655501 (0.0008) [2023-12-27 10:09:56,189][20381] Updated weights for policy 1, policy_version 654907 (0.0006) [2023-12-27 10:09:56,233][20381] Updated weights for policy 1, policy_version 654917 (0.0005) [2023-12-27 10:09:56,281][20381] Updated weights for policy 1, policy_version 654927 (0.0005) [2023-12-27 10:09:56,739][20347] Updated weights for policy 0, policy_version 655511 (0.0007) [2023-12-27 10:09:56,800][20347] Updated weights for policy 0, policy_version 655521 (0.0006) [2023-12-27 10:09:56,835][20381] Updated weights for policy 1, policy_version 654937 (0.0005) [2023-12-27 10:09:56,857][20347] Updated weights for policy 0, policy_version 655531 (0.0007) [2023-12-27 10:09:56,891][20381] Updated weights for policy 1, policy_version 654947 (0.0006) [2023-12-27 10:09:56,946][20381] Updated weights for policy 1, policy_version 654957 (0.0007) [2023-12-27 10:09:56,993][20381] Updated weights for policy 1, policy_version 654967 (0.0009) [2023-12-27 10:09:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 335536128. Throughput: 0: 10258.1, 1: 10480.9. Samples: 335526024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:09:57,363][19170] Avg episode reward: [(0, '73.620'), (1, '1000.000')] [2023-12-27 10:09:57,420][20347] Updated weights for policy 0, policy_version 655541 (0.0005) [2023-12-27 10:09:57,479][20347] Updated weights for policy 0, policy_version 655551 (0.0009) [2023-12-27 10:09:57,533][20347] Updated weights for policy 0, policy_version 655561 (0.0010) [2023-12-27 10:09:57,636][20381] Updated weights for policy 1, policy_version 654977 (0.0006) [2023-12-27 10:09:57,691][20381] Updated weights for policy 1, policy_version 654987 (0.0005) [2023-12-27 10:09:57,748][20381] Updated weights for policy 1, policy_version 654997 (0.0005) [2023-12-27 10:09:58,134][20347] Updated weights for policy 0, policy_version 655571 (0.0009) [2023-12-27 10:09:58,194][20347] Updated weights for policy 0, policy_version 655581 (0.0010) [2023-12-27 10:09:58,252][20347] Updated weights for policy 0, policy_version 655591 (0.0010) [2023-12-27 10:09:58,288][20381] Updated weights for policy 1, policy_version 655007 (0.0005) [2023-12-27 10:09:58,348][20381] Updated weights for policy 1, policy_version 655017 (0.0008) [2023-12-27 10:09:58,410][20381] Updated weights for policy 1, policy_version 655027 (0.0009) [2023-12-27 10:09:59,001][20347] Updated weights for policy 0, policy_version 655601 (0.0010) [2023-12-27 10:09:59,063][20347] Updated weights for policy 0, policy_version 655611 (0.0006) [2023-12-27 10:09:59,125][20347] Updated weights for policy 0, policy_version 655621 (0.0005) [2023-12-27 10:09:59,184][20347] Updated weights for policy 0, policy_version 655631 (0.0005) [2023-12-27 10:09:59,214][20381] Updated weights for policy 1, policy_version 655037 (0.0010) [2023-12-27 10:09:59,280][20381] Updated weights for policy 1, policy_version 655047 (0.0010) [2023-12-27 10:09:59,344][20381] Updated weights for policy 1, policy_version 655057 (0.0012) [2023-12-27 10:09:59,796][20347] Updated weights for policy 0, policy_version 655641 (0.0005) [2023-12-27 10:09:59,857][20347] Updated weights for policy 0, policy_version 655651 (0.0011) [2023-12-27 10:09:59,915][20347] Updated weights for policy 0, policy_version 655661 (0.0007) [2023-12-27 10:10:00,040][20381] Updated weights for policy 1, policy_version 655067 (0.0009) [2023-12-27 10:10:00,095][20381] Updated weights for policy 1, policy_version 655077 (0.0010) [2023-12-27 10:10:00,149][20381] Updated weights for policy 1, policy_version 655087 (0.0010) [2023-12-27 10:10:00,521][20347] Updated weights for policy 0, policy_version 655671 (0.0007) [2023-12-27 10:10:00,568][20347] Updated weights for policy 0, policy_version 655681 (0.0007) [2023-12-27 10:10:00,627][20347] Updated weights for policy 0, policy_version 655691 (0.0008) [2023-12-27 10:10:00,752][20381] Updated weights for policy 1, policy_version 655097 (0.0007) [2023-12-27 10:10:00,803][20381] Updated weights for policy 1, policy_version 655107 (0.0010) [2023-12-27 10:10:00,854][20381] Updated weights for policy 1, policy_version 655117 (0.0008) [2023-12-27 10:10:00,909][20381] Updated weights for policy 1, policy_version 655127 (0.0005) [2023-12-27 10:10:01,293][20347] Updated weights for policy 0, policy_version 655701 (0.0009) [2023-12-27 10:10:01,348][20347] Updated weights for policy 0, policy_version 655711 (0.0010) [2023-12-27 10:10:01,414][20347] Updated weights for policy 0, policy_version 655721 (0.0010) [2023-12-27 10:10:01,560][20381] Updated weights for policy 1, policy_version 655137 (0.0010) [2023-12-27 10:10:01,626][20381] Updated weights for policy 1, policy_version 655147 (0.0010) [2023-12-27 10:10:01,683][20381] Updated weights for policy 1, policy_version 655157 (0.0010) [2023-12-27 10:10:02,124][20347] Updated weights for policy 0, policy_version 655731 (0.0007) [2023-12-27 10:10:02,184][20347] Updated weights for policy 0, policy_version 655741 (0.0006) [2023-12-27 10:10:02,232][20347] Updated weights for policy 0, policy_version 655751 (0.0007) [2023-12-27 10:10:02,362][20381] Updated weights for policy 1, policy_version 655167 (0.0008) [2023-12-27 10:10:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 335642624. Throughput: 0: 10246.7, 1: 10546.1. Samples: 335653392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:02,363][19170] Avg episode reward: [(0, '74.830'), (1, '1000.000')] [2023-12-27 10:10:02,417][20381] Updated weights for policy 1, policy_version 655177 (0.0011) [2023-12-27 10:10:02,473][20381] Updated weights for policy 1, policy_version 655187 (0.0011) [2023-12-27 10:10:02,819][20347] Updated weights for policy 0, policy_version 655761 (0.0006) [2023-12-27 10:10:02,880][20347] Updated weights for policy 0, policy_version 655771 (0.0010) [2023-12-27 10:10:02,940][20347] Updated weights for policy 0, policy_version 655781 (0.0010) [2023-12-27 10:10:02,991][20347] Updated weights for policy 0, policy_version 655791 (0.0010) [2023-12-27 10:10:03,094][20381] Updated weights for policy 1, policy_version 655197 (0.0010) [2023-12-27 10:10:03,145][20381] Updated weights for policy 1, policy_version 655207 (0.0010) [2023-12-27 10:10:03,202][20381] Updated weights for policy 1, policy_version 655217 (0.0010) [2023-12-27 10:10:03,651][20347] Updated weights for policy 0, policy_version 655801 (0.0010) [2023-12-27 10:10:03,701][20347] Updated weights for policy 0, policy_version 655811 (0.0010) [2023-12-27 10:10:03,752][20347] Updated weights for policy 0, policy_version 655821 (0.0010) [2023-12-27 10:10:03,774][20381] Updated weights for policy 1, policy_version 655227 (0.0010) [2023-12-27 10:10:03,838][20381] Updated weights for policy 1, policy_version 655237 (0.0007) [2023-12-27 10:10:03,895][20381] Updated weights for policy 1, policy_version 655247 (0.0009) [2023-12-27 10:10:04,422][20347] Updated weights for policy 0, policy_version 655831 (0.0009) [2023-12-27 10:10:04,477][20347] Updated weights for policy 0, policy_version 655841 (0.0008) [2023-12-27 10:10:04,532][20347] Updated weights for policy 0, policy_version 655851 (0.0008) [2023-12-27 10:10:04,593][20381] Updated weights for policy 1, policy_version 655257 (0.0007) [2023-12-27 10:10:04,652][20381] Updated weights for policy 1, policy_version 655267 (0.0008) [2023-12-27 10:10:04,708][20381] Updated weights for policy 1, policy_version 655277 (0.0007) [2023-12-27 10:10:04,770][20381] Updated weights for policy 1, policy_version 655287 (0.0007) [2023-12-27 10:10:05,273][20347] Updated weights for policy 0, policy_version 655861 (0.0007) [2023-12-27 10:10:05,333][20347] Updated weights for policy 0, policy_version 655871 (0.0007) [2023-12-27 10:10:05,374][20381] Updated weights for policy 1, policy_version 655297 (0.0006) [2023-12-27 10:10:05,386][20347] Updated weights for policy 0, policy_version 655881 (0.0008) [2023-12-27 10:10:05,427][20381] Updated weights for policy 1, policy_version 655307 (0.0007) [2023-12-27 10:10:05,484][20381] Updated weights for policy 1, policy_version 655317 (0.0008) [2023-12-27 10:10:06,036][20347] Updated weights for policy 0, policy_version 655891 (0.0007) [2023-12-27 10:10:06,087][20347] Updated weights for policy 0, policy_version 655901 (0.0008) [2023-12-27 10:10:06,145][20347] Updated weights for policy 0, policy_version 655911 (0.0008) [2023-12-27 10:10:06,217][20381] Updated weights for policy 1, policy_version 655327 (0.0007) [2023-12-27 10:10:06,271][20381] Updated weights for policy 1, policy_version 655337 (0.0008) [2023-12-27 10:10:06,319][20381] Updated weights for policy 1, policy_version 655347 (0.0007) [2023-12-27 10:10:06,863][20347] Updated weights for policy 0, policy_version 655921 (0.0007) [2023-12-27 10:10:06,919][20347] Updated weights for policy 0, policy_version 655931 (0.0008) [2023-12-27 10:10:06,970][20347] Updated weights for policy 0, policy_version 655941 (0.0008) [2023-12-27 10:10:07,023][20381] Updated weights for policy 1, policy_version 655357 (0.0007) [2023-12-27 10:10:07,028][20347] Updated weights for policy 0, policy_version 655951 (0.0007) [2023-12-27 10:10:07,080][20381] Updated weights for policy 1, policy_version 655367 (0.0007) [2023-12-27 10:10:07,135][20381] Updated weights for policy 1, policy_version 655377 (0.0006) [2023-12-27 10:10:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 335749120. Throughput: 0: 10268.2, 1: 10615.0. Samples: 335717192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:07,363][19170] Avg episode reward: [(0, '118.650'), (1, '990.330')] [2023-12-27 10:10:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000655952_167944192.pth... [2023-12-27 10:10:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000655384_167804928.pth... [2023-12-27 10:10:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000654736_167632896.pth [2023-12-27 10:10:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000654136_167485440.pth [2023-12-27 10:10:07,735][20381] Updated weights for policy 1, policy_version 655387 (0.0006) [2023-12-27 10:10:07,771][20347] Updated weights for policy 0, policy_version 655961 (0.0007) [2023-12-27 10:10:07,784][20381] Updated weights for policy 1, policy_version 655397 (0.0006) [2023-12-27 10:10:07,820][20347] Updated weights for policy 0, policy_version 655971 (0.0005) [2023-12-27 10:10:07,836][20381] Updated weights for policy 1, policy_version 655407 (0.0005) [2023-12-27 10:10:07,878][20347] Updated weights for policy 0, policy_version 655981 (0.0007) [2023-12-27 10:10:08,524][20381] Updated weights for policy 1, policy_version 655417 (0.0007) [2023-12-27 10:10:08,576][20381] Updated weights for policy 1, policy_version 655427 (0.0010) [2023-12-27 10:10:08,595][20347] Updated weights for policy 0, policy_version 655991 (0.0007) [2023-12-27 10:10:08,638][20381] Updated weights for policy 1, policy_version 655437 (0.0010) [2023-12-27 10:10:08,651][20347] Updated weights for policy 0, policy_version 656001 (0.0005) [2023-12-27 10:10:08,683][20381] Updated weights for policy 1, policy_version 655447 (0.0008) [2023-12-27 10:10:08,701][20347] Updated weights for policy 0, policy_version 656011 (0.0006) [2023-12-27 10:10:09,349][20381] Updated weights for policy 1, policy_version 655457 (0.0011) [2023-12-27 10:10:09,368][20347] Updated weights for policy 0, policy_version 656021 (0.0008) [2023-12-27 10:10:09,411][20381] Updated weights for policy 1, policy_version 655467 (0.0009) [2023-12-27 10:10:09,429][20347] Updated weights for policy 0, policy_version 656031 (0.0009) [2023-12-27 10:10:09,475][20381] Updated weights for policy 1, policy_version 655477 (0.0007) [2023-12-27 10:10:09,485][20347] Updated weights for policy 0, policy_version 656041 (0.0008) [2023-12-27 10:10:10,100][20347] Updated weights for policy 0, policy_version 656051 (0.0007) [2023-12-27 10:10:10,150][20381] Updated weights for policy 1, policy_version 655487 (0.0008) [2023-12-27 10:10:10,156][20347] Updated weights for policy 0, policy_version 656061 (0.0010) [2023-12-27 10:10:10,209][20381] Updated weights for policy 1, policy_version 655497 (0.0008) [2023-12-27 10:10:10,213][20347] Updated weights for policy 0, policy_version 656071 (0.0011) [2023-12-27 10:10:10,265][20381] Updated weights for policy 1, policy_version 655507 (0.0008) [2023-12-27 10:10:10,880][20347] Updated weights for policy 0, policy_version 656081 (0.0010) [2023-12-27 10:10:10,931][20381] Updated weights for policy 1, policy_version 655517 (0.0007) [2023-12-27 10:10:10,939][20347] Updated weights for policy 0, policy_version 656091 (0.0010) [2023-12-27 10:10:10,986][20381] Updated weights for policy 1, policy_version 655527 (0.0008) [2023-12-27 10:10:10,994][20347] Updated weights for policy 0, policy_version 656101 (0.0010) [2023-12-27 10:10:11,046][20381] Updated weights for policy 1, policy_version 655537 (0.0008) [2023-12-27 10:10:11,058][20347] Updated weights for policy 0, policy_version 656111 (0.0009) [2023-12-27 10:10:11,738][20347] Updated weights for policy 0, policy_version 656121 (0.0010) [2023-12-27 10:10:11,800][20347] Updated weights for policy 0, policy_version 656131 (0.0010) [2023-12-27 10:10:11,801][20381] Updated weights for policy 1, policy_version 655547 (0.0008) [2023-12-27 10:10:11,852][20347] Updated weights for policy 0, policy_version 656141 (0.0010) [2023-12-27 10:10:11,859][20381] Updated weights for policy 1, policy_version 655557 (0.0007) [2023-12-27 10:10:11,917][20381] Updated weights for policy 1, policy_version 655567 (0.0008) [2023-12-27 10:10:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 335847424. Throughput: 0: 10245.2, 1: 10609.5. Samples: 335839056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:12,363][19170] Avg episode reward: [(0, '163.930'), (1, '990.330')] [2023-12-27 10:10:12,578][20347] Updated weights for policy 0, policy_version 656151 (0.0010) [2023-12-27 10:10:12,595][20381] Updated weights for policy 1, policy_version 655577 (0.0007) [2023-12-27 10:10:12,629][20347] Updated weights for policy 0, policy_version 656161 (0.0011) [2023-12-27 10:10:12,651][20381] Updated weights for policy 1, policy_version 655587 (0.0008) [2023-12-27 10:10:12,681][20347] Updated weights for policy 0, policy_version 656171 (0.0010) [2023-12-27 10:10:12,712][20381] Updated weights for policy 1, policy_version 655597 (0.0006) [2023-12-27 10:10:12,773][20381] Updated weights for policy 1, policy_version 655607 (0.0005) [2023-12-27 10:10:13,353][20381] Updated weights for policy 1, policy_version 655617 (0.0007) [2023-12-27 10:10:13,365][20347] Updated weights for policy 0, policy_version 656181 (0.0010) [2023-12-27 10:10:13,401][20381] Updated weights for policy 1, policy_version 655627 (0.0008) [2023-12-27 10:10:13,416][20347] Updated weights for policy 0, policy_version 656191 (0.0010) [2023-12-27 10:10:13,447][20381] Updated weights for policy 1, policy_version 655637 (0.0006) [2023-12-27 10:10:13,470][20347] Updated weights for policy 0, policy_version 656201 (0.0010) [2023-12-27 10:10:14,020][20347] Updated weights for policy 0, policy_version 656211 (0.0006) [2023-12-27 10:10:14,073][20347] Updated weights for policy 0, policy_version 656221 (0.0007) [2023-12-27 10:10:14,122][20347] Updated weights for policy 0, policy_version 656231 (0.0007) [2023-12-27 10:10:14,207][20381] Updated weights for policy 1, policy_version 655647 (0.0007) [2023-12-27 10:10:14,267][20381] Updated weights for policy 1, policy_version 655657 (0.0008) [2023-12-27 10:10:14,328][20381] Updated weights for policy 1, policy_version 655667 (0.0008) [2023-12-27 10:10:14,841][20347] Updated weights for policy 0, policy_version 656241 (0.0007) [2023-12-27 10:10:14,906][20347] Updated weights for policy 0, policy_version 656251 (0.0008) [2023-12-27 10:10:14,962][20347] Updated weights for policy 0, policy_version 656261 (0.0008) [2023-12-27 10:10:14,980][20381] Updated weights for policy 1, policy_version 655677 (0.0007) [2023-12-27 10:10:15,016][20347] Updated weights for policy 0, policy_version 656271 (0.0006) [2023-12-27 10:10:15,042][20381] Updated weights for policy 1, policy_version 655687 (0.0007) [2023-12-27 10:10:15,107][20381] Updated weights for policy 1, policy_version 655697 (0.0008) [2023-12-27 10:10:15,738][20381] Updated weights for policy 1, policy_version 655707 (0.0007) [2023-12-27 10:10:15,755][20347] Updated weights for policy 0, policy_version 656281 (0.0007) [2023-12-27 10:10:15,798][20381] Updated weights for policy 1, policy_version 655717 (0.0006) [2023-12-27 10:10:15,810][20347] Updated weights for policy 0, policy_version 656291 (0.0005) [2023-12-27 10:10:15,855][20381] Updated weights for policy 1, policy_version 655727 (0.0008) [2023-12-27 10:10:15,864][20347] Updated weights for policy 0, policy_version 656301 (0.0005) [2023-12-27 10:10:16,456][20347] Updated weights for policy 0, policy_version 656311 (0.0007) [2023-12-27 10:10:16,508][20347] Updated weights for policy 0, policy_version 656321 (0.0007) [2023-12-27 10:10:16,564][20347] Updated weights for policy 0, policy_version 656331 (0.0008) [2023-12-27 10:10:16,569][20381] Updated weights for policy 1, policy_version 655737 (0.0008) [2023-12-27 10:10:16,624][20381] Updated weights for policy 1, policy_version 655747 (0.0007) [2023-12-27 10:10:16,677][20381] Updated weights for policy 1, policy_version 655757 (0.0008) [2023-12-27 10:10:16,735][20381] Updated weights for policy 1, policy_version 655767 (0.0007) [2023-12-27 10:10:17,260][20347] Updated weights for policy 0, policy_version 656341 (0.0007) [2023-12-27 10:10:17,309][20347] Updated weights for policy 0, policy_version 656351 (0.0008) [2023-12-27 10:10:17,358][20347] Updated weights for policy 0, policy_version 656361 (0.0008) [2023-12-27 10:10:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 335945728. Throughput: 0: 10324.1, 1: 10532.1. Samples: 335962608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:17,364][19170] Avg episode reward: [(0, '163.500'), (1, '980.660')] [2023-12-27 10:10:17,418][20381] Updated weights for policy 1, policy_version 655777 (0.0007) [2023-12-27 10:10:17,477][20381] Updated weights for policy 1, policy_version 655787 (0.0008) [2023-12-27 10:10:17,536][20381] Updated weights for policy 1, policy_version 655797 (0.0008) [2023-12-27 10:10:18,115][20347] Updated weights for policy 0, policy_version 656371 (0.0007) [2023-12-27 10:10:18,154][20381] Updated weights for policy 1, policy_version 655807 (0.0007) [2023-12-27 10:10:18,178][20347] Updated weights for policy 0, policy_version 656381 (0.0008) [2023-12-27 10:10:18,205][20381] Updated weights for policy 1, policy_version 655817 (0.0006) [2023-12-27 10:10:18,234][20347] Updated weights for policy 0, policy_version 656391 (0.0008) [2023-12-27 10:10:18,251][20381] Updated weights for policy 1, policy_version 655827 (0.0005) [2023-12-27 10:10:18,833][20381] Updated weights for policy 1, policy_version 655837 (0.0007) [2023-12-27 10:10:18,892][20381] Updated weights for policy 1, policy_version 655847 (0.0008) [2023-12-27 10:10:18,953][20381] Updated weights for policy 1, policy_version 655857 (0.0008) [2023-12-27 10:10:18,966][20347] Updated weights for policy 0, policy_version 656401 (0.0008) [2023-12-27 10:10:19,048][20347] Updated weights for policy 0, policy_version 656411 (0.0008) [2023-12-27 10:10:19,105][20347] Updated weights for policy 0, policy_version 656421 (0.0007) [2023-12-27 10:10:19,160][20347] Updated weights for policy 0, policy_version 656431 (0.0008) [2023-12-27 10:10:19,654][20381] Updated weights for policy 1, policy_version 655867 (0.0007) [2023-12-27 10:10:19,716][20381] Updated weights for policy 1, policy_version 655877 (0.0006) [2023-12-27 10:10:19,781][20381] Updated weights for policy 1, policy_version 655887 (0.0006) [2023-12-27 10:10:19,876][20347] Updated weights for policy 0, policy_version 656441 (0.0008) [2023-12-27 10:10:19,942][20347] Updated weights for policy 0, policy_version 656451 (0.0009) [2023-12-27 10:10:20,005][20347] Updated weights for policy 0, policy_version 656461 (0.0007) [2023-12-27 10:10:20,431][20381] Updated weights for policy 1, policy_version 655897 (0.0007) [2023-12-27 10:10:20,484][20381] Updated weights for policy 1, policy_version 655907 (0.0008) [2023-12-27 10:10:20,544][20381] Updated weights for policy 1, policy_version 655917 (0.0007) [2023-12-27 10:10:20,605][20381] Updated weights for policy 1, policy_version 655927 (0.0008) [2023-12-27 10:10:20,693][20347] Updated weights for policy 0, policy_version 656471 (0.0008) [2023-12-27 10:10:20,760][20347] Updated weights for policy 0, policy_version 656481 (0.0008) [2023-12-27 10:10:20,826][20347] Updated weights for policy 0, policy_version 656491 (0.0009) [2023-12-27 10:10:21,283][20381] Updated weights for policy 1, policy_version 655937 (0.0009) [2023-12-27 10:10:21,340][20381] Updated weights for policy 1, policy_version 655947 (0.0009) [2023-12-27 10:10:21,413][20381] Updated weights for policy 1, policy_version 655957 (0.0010) [2023-12-27 10:10:21,470][20347] Updated weights for policy 0, policy_version 656501 (0.0007) [2023-12-27 10:10:21,516][20347] Updated weights for policy 0, policy_version 656511 (0.0009) [2023-12-27 10:10:21,562][20347] Updated weights for policy 0, policy_version 656521 (0.0009) [2023-12-27 10:10:22,063][20381] Updated weights for policy 1, policy_version 655967 (0.0006) [2023-12-27 10:10:22,115][20381] Updated weights for policy 1, policy_version 655977 (0.0006) [2023-12-27 10:10:22,163][20381] Updated weights for policy 1, policy_version 655987 (0.0006) [2023-12-27 10:10:22,333][20347] Updated weights for policy 0, policy_version 656531 (0.0008) [2023-12-27 10:10:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 336052224. Throughput: 0: 10247.1, 1: 10520.8. Samples: 336023216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:22,363][19170] Avg episode reward: [(0, '223.800'), (1, '990.370')] [2023-12-27 10:10:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000655992_167960576.pth... [2023-12-27 10:10:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000654744_167641088.pth [2023-12-27 10:10:22,404][20347] Updated weights for policy 0, policy_version 656541 (0.0008) [2023-12-27 10:10:22,461][20347] Updated weights for policy 0, policy_version 656551 (0.0008) [2023-12-27 10:10:22,512][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000656560_168099840.pth... [2023-12-27 10:10:22,516][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000655344_167788544.pth [2023-12-27 10:10:22,877][20381] Updated weights for policy 1, policy_version 655997 (0.0006) [2023-12-27 10:10:22,939][20381] Updated weights for policy 1, policy_version 656007 (0.0006) [2023-12-27 10:10:22,997][20381] Updated weights for policy 1, policy_version 656017 (0.0008) [2023-12-27 10:10:23,176][20347] Updated weights for policy 0, policy_version 656561 (0.0007) [2023-12-27 10:10:23,223][20347] Updated weights for policy 0, policy_version 656571 (0.0009) [2023-12-27 10:10:23,267][20347] Updated weights for policy 0, policy_version 656581 (0.0008) [2023-12-27 10:10:23,323][20347] Updated weights for policy 0, policy_version 656591 (0.0009) [2023-12-27 10:10:23,571][20381] Updated weights for policy 1, policy_version 656027 (0.0008) [2023-12-27 10:10:23,627][20381] Updated weights for policy 1, policy_version 656037 (0.0008) [2023-12-27 10:10:23,677][20381] Updated weights for policy 1, policy_version 656047 (0.0008) [2023-12-27 10:10:24,003][20347] Updated weights for policy 0, policy_version 656601 (0.0006) [2023-12-27 10:10:24,051][20347] Updated weights for policy 0, policy_version 656611 (0.0005) [2023-12-27 10:10:24,095][20347] Updated weights for policy 0, policy_version 656621 (0.0005) [2023-12-27 10:10:24,307][20381] Updated weights for policy 1, policy_version 656057 (0.0007) [2023-12-27 10:10:24,371][20381] Updated weights for policy 1, policy_version 656067 (0.0006) [2023-12-27 10:10:24,433][20381] Updated weights for policy 1, policy_version 656077 (0.0007) [2023-12-27 10:10:24,488][20381] Updated weights for policy 1, policy_version 656087 (0.0006) [2023-12-27 10:10:24,674][20347] Updated weights for policy 0, policy_version 656631 (0.0009) [2023-12-27 10:10:24,722][20347] Updated weights for policy 0, policy_version 656641 (0.0010) [2023-12-27 10:10:24,771][20347] Updated weights for policy 0, policy_version 656651 (0.0010) [2023-12-27 10:10:25,137][20381] Updated weights for policy 1, policy_version 656097 (0.0007) [2023-12-27 10:10:25,193][20381] Updated weights for policy 1, policy_version 656107 (0.0008) [2023-12-27 10:10:25,248][20381] Updated weights for policy 1, policy_version 656117 (0.0005) [2023-12-27 10:10:25,390][20347] Updated weights for policy 0, policy_version 656661 (0.0009) [2023-12-27 10:10:25,443][20347] Updated weights for policy 0, policy_version 656671 (0.0008) [2023-12-27 10:10:25,488][20347] Updated weights for policy 0, policy_version 656681 (0.0007) [2023-12-27 10:10:25,784][20381] Updated weights for policy 1, policy_version 656127 (0.0007) [2023-12-27 10:10:25,830][20381] Updated weights for policy 1, policy_version 656137 (0.0008) [2023-12-27 10:10:25,878][20381] Updated weights for policy 1, policy_version 656147 (0.0008) [2023-12-27 10:10:26,192][20347] Updated weights for policy 0, policy_version 656691 (0.0007) [2023-12-27 10:10:26,243][20347] Updated weights for policy 0, policy_version 656701 (0.0005) [2023-12-27 10:10:26,286][20347] Updated weights for policy 0, policy_version 656711 (0.0005) [2023-12-27 10:10:26,553][20381] Updated weights for policy 1, policy_version 656157 (0.0008) [2023-12-27 10:10:26,618][20381] Updated weights for policy 1, policy_version 656167 (0.0009) [2023-12-27 10:10:26,671][20381] Updated weights for policy 1, policy_version 656177 (0.0007) [2023-12-27 10:10:26,892][20347] Updated weights for policy 0, policy_version 656721 (0.0006) [2023-12-27 10:10:26,956][20347] Updated weights for policy 0, policy_version 656731 (0.0006) [2023-12-27 10:10:27,021][20347] Updated weights for policy 0, policy_version 656741 (0.0005) [2023-12-27 10:10:27,082][20347] Updated weights for policy 0, policy_version 656751 (0.0005) [2023-12-27 10:10:27,299][20381] Updated weights for policy 1, policy_version 656187 (0.0007) [2023-12-27 10:10:27,345][20381] Updated weights for policy 1, policy_version 656197 (0.0007) [2023-12-27 10:10:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 336158720. Throughput: 0: 10279.5, 1: 10589.5. Samples: 336149924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:27,363][19170] Avg episode reward: [(0, '157.130'), (1, '970.970')] [2023-12-27 10:10:27,403][20381] Updated weights for policy 1, policy_version 656207 (0.0006) [2023-12-27 10:10:27,648][20347] Updated weights for policy 0, policy_version 656761 (0.0005) [2023-12-27 10:10:27,694][20347] Updated weights for policy 0, policy_version 656771 (0.0005) [2023-12-27 10:10:27,745][20347] Updated weights for policy 0, policy_version 656781 (0.0005) [2023-12-27 10:10:28,040][20381] Updated weights for policy 1, policy_version 656217 (0.0005) [2023-12-27 10:10:28,099][20381] Updated weights for policy 1, policy_version 656227 (0.0005) [2023-12-27 10:10:28,162][20381] Updated weights for policy 1, policy_version 656237 (0.0005) [2023-12-27 10:10:28,221][20381] Updated weights for policy 1, policy_version 656247 (0.0007) [2023-12-27 10:10:28,296][20347] Updated weights for policy 0, policy_version 656791 (0.0006) [2023-12-27 10:10:28,357][20347] Updated weights for policy 0, policy_version 656801 (0.0008) [2023-12-27 10:10:28,417][20347] Updated weights for policy 0, policy_version 656811 (0.0008) [2023-12-27 10:10:28,847][20381] Updated weights for policy 1, policy_version 656257 (0.0006) [2023-12-27 10:10:28,908][20381] Updated weights for policy 1, policy_version 656267 (0.0007) [2023-12-27 10:10:28,963][20381] Updated weights for policy 1, policy_version 656277 (0.0008) [2023-12-27 10:10:29,046][20347] Updated weights for policy 0, policy_version 656821 (0.0007) [2023-12-27 10:10:29,094][20347] Updated weights for policy 0, policy_version 656831 (0.0007) [2023-12-27 10:10:29,146][20347] Updated weights for policy 0, policy_version 656841 (0.0007) [2023-12-27 10:10:29,592][20381] Updated weights for policy 1, policy_version 656287 (0.0006) [2023-12-27 10:10:29,649][20381] Updated weights for policy 1, policy_version 656297 (0.0005) [2023-12-27 10:10:29,715][20381] Updated weights for policy 1, policy_version 656307 (0.0006) [2023-12-27 10:10:29,895][20347] Updated weights for policy 0, policy_version 656851 (0.0008) [2023-12-27 10:10:29,957][20347] Updated weights for policy 0, policy_version 656861 (0.0007) [2023-12-27 10:10:30,012][20347] Updated weights for policy 0, policy_version 656871 (0.0009) [2023-12-27 10:10:30,258][20381] Updated weights for policy 1, policy_version 656317 (0.0006) [2023-12-27 10:10:30,311][20381] Updated weights for policy 1, policy_version 656327 (0.0005) [2023-12-27 10:10:30,366][20381] Updated weights for policy 1, policy_version 656337 (0.0005) [2023-12-27 10:10:30,773][20347] Updated weights for policy 0, policy_version 656881 (0.0008) [2023-12-27 10:10:30,820][20347] Updated weights for policy 0, policy_version 656891 (0.0007) [2023-12-27 10:10:30,874][20347] Updated weights for policy 0, policy_version 656901 (0.0007) [2023-12-27 10:10:30,891][20381] Updated weights for policy 1, policy_version 656347 (0.0005) [2023-12-27 10:10:30,928][20347] Updated weights for policy 0, policy_version 656911 (0.0007) [2023-12-27 10:10:30,939][20381] Updated weights for policy 1, policy_version 656357 (0.0005) [2023-12-27 10:10:30,995][20381] Updated weights for policy 1, policy_version 656367 (0.0005) [2023-12-27 10:10:31,612][20381] Updated weights for policy 1, policy_version 656377 (0.0006) [2023-12-27 10:10:31,671][20347] Updated weights for policy 0, policy_version 656921 (0.0007) [2023-12-27 10:10:31,673][20381] Updated weights for policy 1, policy_version 656387 (0.0006) [2023-12-27 10:10:31,730][20381] Updated weights for policy 1, policy_version 656397 (0.0007) [2023-12-27 10:10:31,734][20347] Updated weights for policy 0, policy_version 656931 (0.0009) [2023-12-27 10:10:31,794][20381] Updated weights for policy 1, policy_version 656407 (0.0006) [2023-12-27 10:10:31,799][20347] Updated weights for policy 0, policy_version 656941 (0.0010) [2023-12-27 10:10:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20716.1). Total num frames: 336265216. Throughput: 0: 10309.2, 1: 10695.3. Samples: 336280036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:32,363][19170] Avg episode reward: [(0, '184.270'), (1, '961.240')] [2023-12-27 10:10:32,385][20347] Updated weights for policy 0, policy_version 656951 (0.0010) [2023-12-27 10:10:32,426][20381] Updated weights for policy 1, policy_version 656417 (0.0007) [2023-12-27 10:10:32,443][20347] Updated weights for policy 0, policy_version 656961 (0.0010) [2023-12-27 10:10:32,484][20381] Updated weights for policy 1, policy_version 656427 (0.0005) [2023-12-27 10:10:32,498][20347] Updated weights for policy 0, policy_version 656971 (0.0010) [2023-12-27 10:10:32,542][20381] Updated weights for policy 1, policy_version 656437 (0.0007) [2023-12-27 10:10:33,149][20347] Updated weights for policy 0, policy_version 656981 (0.0007) [2023-12-27 10:10:33,203][20347] Updated weights for policy 0, policy_version 656991 (0.0005) [2023-12-27 10:10:33,239][20381] Updated weights for policy 1, policy_version 656447 (0.0006) [2023-12-27 10:10:33,259][20347] Updated weights for policy 0, policy_version 657001 (0.0008) [2023-12-27 10:10:33,291][20381] Updated weights for policy 1, policy_version 656457 (0.0008) [2023-12-27 10:10:33,351][20381] Updated weights for policy 1, policy_version 656467 (0.0006) [2023-12-27 10:10:33,897][20347] Updated weights for policy 0, policy_version 657011 (0.0008) [2023-12-27 10:10:33,923][20381] Updated weights for policy 1, policy_version 656477 (0.0007) [2023-12-27 10:10:33,951][20347] Updated weights for policy 0, policy_version 657021 (0.0010) [2023-12-27 10:10:33,971][20381] Updated weights for policy 1, policy_version 656487 (0.0005) [2023-12-27 10:10:34,005][20347] Updated weights for policy 0, policy_version 657031 (0.0010) [2023-12-27 10:10:34,021][20381] Updated weights for policy 1, policy_version 656497 (0.0007) [2023-12-27 10:10:34,680][20347] Updated weights for policy 0, policy_version 657041 (0.0010) [2023-12-27 10:10:34,703][20381] Updated weights for policy 1, policy_version 656507 (0.0005) [2023-12-27 10:10:34,739][20347] Updated weights for policy 0, policy_version 657051 (0.0010) [2023-12-27 10:10:34,757][20381] Updated weights for policy 1, policy_version 656517 (0.0005) [2023-12-27 10:10:34,792][20347] Updated weights for policy 0, policy_version 657061 (0.0010) [2023-12-27 10:10:34,809][20381] Updated weights for policy 1, policy_version 656527 (0.0005) [2023-12-27 10:10:34,847][20347] Updated weights for policy 0, policy_version 657071 (0.0010) [2023-12-27 10:10:35,526][20381] Updated weights for policy 1, policy_version 656537 (0.0005) [2023-12-27 10:10:35,535][20347] Updated weights for policy 0, policy_version 657081 (0.0010) [2023-12-27 10:10:35,584][20381] Updated weights for policy 1, policy_version 656547 (0.0005) [2023-12-27 10:10:35,590][20347] Updated weights for policy 0, policy_version 657091 (0.0010) [2023-12-27 10:10:35,632][20381] Updated weights for policy 1, policy_version 656557 (0.0005) [2023-12-27 10:10:35,641][20347] Updated weights for policy 0, policy_version 657101 (0.0009) [2023-12-27 10:10:35,685][20381] Updated weights for policy 1, policy_version 656567 (0.0008) [2023-12-27 10:10:36,319][20347] Updated weights for policy 0, policy_version 657111 (0.0009) [2023-12-27 10:10:36,372][20347] Updated weights for policy 0, policy_version 657121 (0.0011) [2023-12-27 10:10:36,403][20381] Updated weights for policy 1, policy_version 656577 (0.0008) [2023-12-27 10:10:36,421][20347] Updated weights for policy 0, policy_version 657131 (0.0010) [2023-12-27 10:10:36,461][20381] Updated weights for policy 1, policy_version 656587 (0.0006) [2023-12-27 10:10:36,522][20381] Updated weights for policy 1, policy_version 656597 (0.0010) [2023-12-27 10:10:37,072][20347] Updated weights for policy 0, policy_version 657141 (0.0008) [2023-12-27 10:10:37,133][20347] Updated weights for policy 0, policy_version 657151 (0.0010) [2023-12-27 10:10:37,186][20347] Updated weights for policy 0, policy_version 657161 (0.0010) [2023-12-27 10:10:37,203][20381] Updated weights for policy 1, policy_version 656607 (0.0007) [2023-12-27 10:10:37,261][20381] Updated weights for policy 1, policy_version 656617 (0.0005) [2023-12-27 10:10:37,321][20381] Updated weights for policy 1, policy_version 656627 (0.0006) [2023-12-27 10:10:37,363][19170] Fps is (10 sec: 22117.9, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 336379904. Throughput: 0: 10354.8, 1: 10685.5. Samples: 336343164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:37,364][19170] Avg episode reward: [(0, '216.220'), (1, '985.650')] [2023-12-27 10:10:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000656632_168124416.pth... [2023-12-27 10:10:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000657168_168255488.pth... [2023-12-27 10:10:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000655384_167804928.pth [2023-12-27 10:10:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000655952_167944192.pth [2023-12-27 10:10:37,798][20347] Updated weights for policy 0, policy_version 657171 (0.0006) [2023-12-27 10:10:37,856][20347] Updated weights for policy 0, policy_version 657181 (0.0007) [2023-12-27 10:10:37,917][20347] Updated weights for policy 0, policy_version 657191 (0.0008) [2023-12-27 10:10:37,971][20381] Updated weights for policy 1, policy_version 656637 (0.0007) [2023-12-27 10:10:38,034][20381] Updated weights for policy 1, policy_version 656647 (0.0008) [2023-12-27 10:10:38,093][20381] Updated weights for policy 1, policy_version 656657 (0.0008) [2023-12-27 10:10:38,600][20347] Updated weights for policy 0, policy_version 657201 (0.0007) [2023-12-27 10:10:38,667][20347] Updated weights for policy 0, policy_version 657211 (0.0006) [2023-12-27 10:10:38,734][20347] Updated weights for policy 0, policy_version 657221 (0.0006) [2023-12-27 10:10:38,773][20381] Updated weights for policy 1, policy_version 656667 (0.0007) [2023-12-27 10:10:38,795][20347] Updated weights for policy 0, policy_version 657231 (0.0009) [2023-12-27 10:10:38,833][20381] Updated weights for policy 1, policy_version 656677 (0.0007) [2023-12-27 10:10:38,897][20381] Updated weights for policy 1, policy_version 656687 (0.0006) [2023-12-27 10:10:39,364][20347] Updated weights for policy 0, policy_version 657241 (0.0008) [2023-12-27 10:10:39,429][20347] Updated weights for policy 0, policy_version 657251 (0.0009) [2023-12-27 10:10:39,487][20347] Updated weights for policy 0, policy_version 657261 (0.0010) [2023-12-27 10:10:39,608][20381] Updated weights for policy 1, policy_version 656697 (0.0009) [2023-12-27 10:10:39,668][20381] Updated weights for policy 1, policy_version 656707 (0.0011) [2023-12-27 10:10:39,728][20381] Updated weights for policy 1, policy_version 656717 (0.0010) [2023-12-27 10:10:39,794][20381] Updated weights for policy 1, policy_version 656727 (0.0008) [2023-12-27 10:10:40,175][20347] Updated weights for policy 0, policy_version 657271 (0.0007) [2023-12-27 10:10:40,232][20347] Updated weights for policy 0, policy_version 657281 (0.0010) [2023-12-27 10:10:40,293][20347] Updated weights for policy 0, policy_version 657291 (0.0010) [2023-12-27 10:10:40,460][20381] Updated weights for policy 1, policy_version 656737 (0.0006) [2023-12-27 10:10:40,522][20381] Updated weights for policy 1, policy_version 656747 (0.0007) [2023-12-27 10:10:40,581][20381] Updated weights for policy 1, policy_version 656757 (0.0010) [2023-12-27 10:10:40,926][20347] Updated weights for policy 0, policy_version 657301 (0.0009) [2023-12-27 10:10:40,991][20347] Updated weights for policy 0, policy_version 657311 (0.0008) [2023-12-27 10:10:41,058][20347] Updated weights for policy 0, policy_version 657321 (0.0008) [2023-12-27 10:10:41,217][20381] Updated weights for policy 1, policy_version 656767 (0.0010) [2023-12-27 10:10:41,281][20381] Updated weights for policy 1, policy_version 656777 (0.0010) [2023-12-27 10:10:41,345][20381] Updated weights for policy 1, policy_version 656787 (0.0009) [2023-12-27 10:10:41,787][20347] Updated weights for policy 0, policy_version 657331 (0.0007) [2023-12-27 10:10:41,845][20347] Updated weights for policy 0, policy_version 657341 (0.0006) [2023-12-27 10:10:41,898][20347] Updated weights for policy 0, policy_version 657351 (0.0008) [2023-12-27 10:10:41,970][20381] Updated weights for policy 1, policy_version 656797 (0.0010) [2023-12-27 10:10:42,028][20381] Updated weights for policy 1, policy_version 656807 (0.0011) [2023-12-27 10:10:42,081][20381] Updated weights for policy 1, policy_version 656817 (0.0011) [2023-12-27 10:10:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 336478208. Throughput: 0: 10373.7, 1: 10558.8. Samples: 336467992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:42,363][19170] Avg episode reward: [(0, '225.350'), (1, '912.060')] [2023-12-27 10:10:42,580][20347] Updated weights for policy 0, policy_version 657361 (0.0008) [2023-12-27 10:10:42,643][20347] Updated weights for policy 0, policy_version 657371 (0.0008) [2023-12-27 10:10:42,690][20347] Updated weights for policy 0, policy_version 657381 (0.0006) [2023-12-27 10:10:42,750][20347] Updated weights for policy 0, policy_version 657391 (0.0005) [2023-12-27 10:10:42,760][20381] Updated weights for policy 1, policy_version 656827 (0.0009) [2023-12-27 10:10:42,825][20381] Updated weights for policy 1, policy_version 656837 (0.0006) [2023-12-27 10:10:42,890][20381] Updated weights for policy 1, policy_version 656847 (0.0009) [2023-12-27 10:10:43,391][20347] Updated weights for policy 0, policy_version 657401 (0.0006) [2023-12-27 10:10:43,440][20347] Updated weights for policy 0, policy_version 657411 (0.0007) [2023-12-27 10:10:43,489][20381] Updated weights for policy 1, policy_version 656857 (0.0006) [2023-12-27 10:10:43,491][20347] Updated weights for policy 0, policy_version 657421 (0.0008) [2023-12-27 10:10:43,537][20381] Updated weights for policy 1, policy_version 656867 (0.0007) [2023-12-27 10:10:43,596][20381] Updated weights for policy 1, policy_version 656877 (0.0008) [2023-12-27 10:10:43,648][20381] Updated weights for policy 1, policy_version 656887 (0.0007) [2023-12-27 10:10:44,183][20347] Updated weights for policy 0, policy_version 657431 (0.0006) [2023-12-27 10:10:44,234][20347] Updated weights for policy 0, policy_version 657441 (0.0007) [2023-12-27 10:10:44,235][20381] Updated weights for policy 1, policy_version 656897 (0.0006) [2023-12-27 10:10:44,288][20381] Updated weights for policy 1, policy_version 656907 (0.0007) [2023-12-27 10:10:44,291][20347] Updated weights for policy 0, policy_version 657451 (0.0007) [2023-12-27 10:10:44,341][20381] Updated weights for policy 1, policy_version 656917 (0.0007) [2023-12-27 10:10:44,896][20347] Updated weights for policy 0, policy_version 657461 (0.0007) [2023-12-27 10:10:44,962][20347] Updated weights for policy 0, policy_version 657471 (0.0006) [2023-12-27 10:10:45,022][20347] Updated weights for policy 0, policy_version 657481 (0.0008) [2023-12-27 10:10:45,063][20381] Updated weights for policy 1, policy_version 656927 (0.0008) [2023-12-27 10:10:45,127][20381] Updated weights for policy 1, policy_version 656937 (0.0008) [2023-12-27 10:10:45,180][20381] Updated weights for policy 1, policy_version 656947 (0.0008) [2023-12-27 10:10:45,709][20347] Updated weights for policy 0, policy_version 657491 (0.0007) [2023-12-27 10:10:45,767][20347] Updated weights for policy 0, policy_version 657501 (0.0008) [2023-12-27 10:10:45,816][20347] Updated weights for policy 0, policy_version 657511 (0.0008) [2023-12-27 10:10:45,862][20381] Updated weights for policy 1, policy_version 656957 (0.0009) [2023-12-27 10:10:45,910][20381] Updated weights for policy 1, policy_version 656967 (0.0008) [2023-12-27 10:10:45,957][20381] Updated weights for policy 1, policy_version 656977 (0.0010) [2023-12-27 10:10:46,424][20347] Updated weights for policy 0, policy_version 657521 (0.0007) [2023-12-27 10:10:46,484][20347] Updated weights for policy 0, policy_version 657531 (0.0009) [2023-12-27 10:10:46,543][20347] Updated weights for policy 0, policy_version 657541 (0.0007) [2023-12-27 10:10:46,593][20347] Updated weights for policy 0, policy_version 657551 (0.0009) [2023-12-27 10:10:46,614][20381] Updated weights for policy 1, policy_version 656987 (0.0010) [2023-12-27 10:10:46,665][20381] Updated weights for policy 1, policy_version 656997 (0.0008) [2023-12-27 10:10:46,712][20381] Updated weights for policy 1, policy_version 657007 (0.0007) [2023-12-27 10:10:47,233][20347] Updated weights for policy 0, policy_version 657561 (0.0006) [2023-12-27 10:10:47,282][20347] Updated weights for policy 0, policy_version 657571 (0.0007) [2023-12-27 10:10:47,332][20347] Updated weights for policy 0, policy_version 657581 (0.0010) [2023-12-27 10:10:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 336584704. Throughput: 0: 10371.4, 1: 10535.4. Samples: 336594196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:47,363][19170] Avg episode reward: [(0, '210.020'), (1, '795.130')] [2023-12-27 10:10:47,364][20381] Updated weights for policy 1, policy_version 657017 (0.0007) [2023-12-27 10:10:47,423][20381] Updated weights for policy 1, policy_version 657027 (0.0006) [2023-12-27 10:10:47,484][20381] Updated weights for policy 1, policy_version 657037 (0.0009) [2023-12-27 10:10:47,537][20381] Updated weights for policy 1, policy_version 657047 (0.0009) [2023-12-27 10:10:47,899][20347] Updated weights for policy 0, policy_version 657591 (0.0010) [2023-12-27 10:10:47,956][20347] Updated weights for policy 0, policy_version 657601 (0.0010) [2023-12-27 10:10:48,018][20347] Updated weights for policy 0, policy_version 657611 (0.0010) [2023-12-27 10:10:48,204][20381] Updated weights for policy 1, policy_version 657057 (0.0006) [2023-12-27 10:10:48,257][20381] Updated weights for policy 1, policy_version 657067 (0.0006) [2023-12-27 10:10:48,315][20381] Updated weights for policy 1, policy_version 657077 (0.0006) [2023-12-27 10:10:48,705][20347] Updated weights for policy 0, policy_version 657621 (0.0010) [2023-12-27 10:10:48,764][20347] Updated weights for policy 0, policy_version 657631 (0.0005) [2023-12-27 10:10:48,832][20347] Updated weights for policy 0, policy_version 657641 (0.0007) [2023-12-27 10:10:49,010][20381] Updated weights for policy 1, policy_version 657087 (0.0007) [2023-12-27 10:10:49,064][20381] Updated weights for policy 1, policy_version 657097 (0.0008) [2023-12-27 10:10:49,113][20381] Updated weights for policy 1, policy_version 657107 (0.0011) [2023-12-27 10:10:49,477][20347] Updated weights for policy 0, policy_version 657651 (0.0010) [2023-12-27 10:10:49,541][20347] Updated weights for policy 0, policy_version 657661 (0.0010) [2023-12-27 10:10:49,602][20347] Updated weights for policy 0, policy_version 657671 (0.0010) [2023-12-27 10:10:49,730][20381] Updated weights for policy 1, policy_version 657117 (0.0008) [2023-12-27 10:10:49,797][20381] Updated weights for policy 1, policy_version 657127 (0.0010) [2023-12-27 10:10:49,861][20381] Updated weights for policy 1, policy_version 657137 (0.0008) [2023-12-27 10:10:50,222][20347] Updated weights for policy 0, policy_version 657681 (0.0008) [2023-12-27 10:10:50,281][20347] Updated weights for policy 0, policy_version 657691 (0.0006) [2023-12-27 10:10:50,343][20347] Updated weights for policy 0, policy_version 657701 (0.0005) [2023-12-27 10:10:50,404][20347] Updated weights for policy 0, policy_version 657711 (0.0007) [2023-12-27 10:10:50,531][20381] Updated weights for policy 1, policy_version 657147 (0.0010) [2023-12-27 10:10:50,591][20381] Updated weights for policy 1, policy_version 657157 (0.0010) [2023-12-27 10:10:50,643][20381] Updated weights for policy 1, policy_version 657167 (0.0010) [2023-12-27 10:10:51,045][20347] Updated weights for policy 0, policy_version 657721 (0.0008) [2023-12-27 10:10:51,105][20347] Updated weights for policy 0, policy_version 657731 (0.0007) [2023-12-27 10:10:51,167][20347] Updated weights for policy 0, policy_version 657741 (0.0006) [2023-12-27 10:10:51,360][20381] Updated weights for policy 1, policy_version 657177 (0.0007) [2023-12-27 10:10:51,422][20381] Updated weights for policy 1, policy_version 657187 (0.0007) [2023-12-27 10:10:51,481][20381] Updated weights for policy 1, policy_version 657197 (0.0008) [2023-12-27 10:10:51,537][20381] Updated weights for policy 1, policy_version 657207 (0.0007) [2023-12-27 10:10:51,819][20347] Updated weights for policy 0, policy_version 657751 (0.0007) [2023-12-27 10:10:51,886][20347] Updated weights for policy 0, policy_version 657761 (0.0005) [2023-12-27 10:10:51,948][20347] Updated weights for policy 0, policy_version 657771 (0.0007) [2023-12-27 10:10:52,243][20381] Updated weights for policy 1, policy_version 657217 (0.0008) [2023-12-27 10:10:52,273][20292] KL-divergence is very high: 101.3276 [2023-12-27 10:10:52,312][20381] Updated weights for policy 1, policy_version 657227 (0.0006) [2023-12-27 10:10:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 336683008. Throughput: 0: 10378.5, 1: 10509.7. Samples: 336657160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:52,363][19170] Avg episode reward: [(0, '219.280'), (1, '867.960')] [2023-12-27 10:10:52,369][20381] Updated weights for policy 1, policy_version 657237 (0.0008) [2023-12-27 10:10:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000657776_168411136.pth... [2023-12-27 10:10:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000656560_168099840.pth [2023-12-27 10:10:52,391][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000657240_168280064.pth... [2023-12-27 10:10:52,395][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000655992_167960576.pth [2023-12-27 10:10:52,646][20347] Updated weights for policy 0, policy_version 657781 (0.0009) [2023-12-27 10:10:52,705][20347] Updated weights for policy 0, policy_version 657791 (0.0009) [2023-12-27 10:10:52,770][20347] Updated weights for policy 0, policy_version 657801 (0.0005) [2023-12-27 10:10:53,017][20381] Updated weights for policy 1, policy_version 657247 (0.0006) [2023-12-27 10:10:53,075][20381] Updated weights for policy 1, policy_version 657257 (0.0007) [2023-12-27 10:10:53,134][20381] Updated weights for policy 1, policy_version 657267 (0.0007) [2023-12-27 10:10:53,423][20347] Updated weights for policy 0, policy_version 657811 (0.0008) [2023-12-27 10:10:53,480][20347] Updated weights for policy 0, policy_version 657821 (0.0010) [2023-12-27 10:10:53,532][20347] Updated weights for policy 0, policy_version 657831 (0.0010) [2023-12-27 10:10:53,786][20381] Updated weights for policy 1, policy_version 657277 (0.0007) [2023-12-27 10:10:53,835][20381] Updated weights for policy 1, policy_version 657287 (0.0008) [2023-12-27 10:10:53,880][20381] Updated weights for policy 1, policy_version 657297 (0.0008) [2023-12-27 10:10:54,089][20347] Updated weights for policy 0, policy_version 657841 (0.0010) [2023-12-27 10:10:54,145][20347] Updated weights for policy 0, policy_version 657851 (0.0005) [2023-12-27 10:10:54,205][20347] Updated weights for policy 0, policy_version 657861 (0.0005) [2023-12-27 10:10:54,259][20347] Updated weights for policy 0, policy_version 657871 (0.0005) [2023-12-27 10:10:54,612][20381] Updated weights for policy 1, policy_version 657307 (0.0009) [2023-12-27 10:10:54,661][20381] Updated weights for policy 1, policy_version 657317 (0.0008) [2023-12-27 10:10:54,712][20381] Updated weights for policy 1, policy_version 657327 (0.0007) [2023-12-27 10:10:54,873][20347] Updated weights for policy 0, policy_version 657881 (0.0010) [2023-12-27 10:10:54,919][20347] Updated weights for policy 0, policy_version 657891 (0.0010) [2023-12-27 10:10:54,965][20347] Updated weights for policy 0, policy_version 657901 (0.0010) [2023-12-27 10:10:55,377][20381] Updated weights for policy 1, policy_version 657337 (0.0007) [2023-12-27 10:10:55,422][20381] Updated weights for policy 1, policy_version 657347 (0.0007) [2023-12-27 10:10:55,468][20381] Updated weights for policy 1, policy_version 657357 (0.0007) [2023-12-27 10:10:55,520][20381] Updated weights for policy 1, policy_version 657367 (0.0005) [2023-12-27 10:10:55,576][20347] Updated weights for policy 0, policy_version 657911 (0.0007) [2023-12-27 10:10:55,627][20347] Updated weights for policy 0, policy_version 657921 (0.0005) [2023-12-27 10:10:55,690][20347] Updated weights for policy 0, policy_version 657931 (0.0005) [2023-12-27 10:10:56,113][20381] Updated weights for policy 1, policy_version 657377 (0.0005) [2023-12-27 10:10:56,169][20381] Updated weights for policy 1, policy_version 657387 (0.0005) [2023-12-27 10:10:56,223][20381] Updated weights for policy 1, policy_version 657397 (0.0005) [2023-12-27 10:10:56,303][20347] Updated weights for policy 0, policy_version 657941 (0.0009) [2023-12-27 10:10:56,358][20347] Updated weights for policy 0, policy_version 657951 (0.0010) [2023-12-27 10:10:56,406][20347] Updated weights for policy 0, policy_version 657961 (0.0010) [2023-12-27 10:10:56,755][20381] Updated weights for policy 1, policy_version 657407 (0.0007) [2023-12-27 10:10:56,809][20381] Updated weights for policy 1, policy_version 657417 (0.0007) [2023-12-27 10:10:56,867][20381] Updated weights for policy 1, policy_version 657427 (0.0009) [2023-12-27 10:10:57,083][20347] Updated weights for policy 0, policy_version 657971 (0.0010) [2023-12-27 10:10:57,130][20347] Updated weights for policy 0, policy_version 657981 (0.0010) [2023-12-27 10:10:57,175][20347] Updated weights for policy 0, policy_version 657991 (0.0010) [2023-12-27 10:10:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 21026.0, 300 sec: 20771.6). Total num frames: 336797696. Throughput: 0: 10480.7, 1: 10527.1. Samples: 336784412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:10:57,363][19170] Avg episode reward: [(0, '232.510'), (1, '816.020')] [2023-12-27 10:10:57,500][20381] Updated weights for policy 1, policy_version 657437 (0.0010) [2023-12-27 10:10:57,553][20381] Updated weights for policy 1, policy_version 657447 (0.0007) [2023-12-27 10:10:57,611][20381] Updated weights for policy 1, policy_version 657457 (0.0005) [2023-12-27 10:10:57,856][20347] Updated weights for policy 0, policy_version 658001 (0.0010) [2023-12-27 10:10:57,909][20347] Updated weights for policy 0, policy_version 658011 (0.0010) [2023-12-27 10:10:57,963][20347] Updated weights for policy 0, policy_version 658021 (0.0010) [2023-12-27 10:10:58,015][20347] Updated weights for policy 0, policy_version 658031 (0.0010) [2023-12-27 10:10:58,246][20381] Updated weights for policy 1, policy_version 657467 (0.0006) [2023-12-27 10:10:58,307][20381] Updated weights for policy 1, policy_version 657477 (0.0008) [2023-12-27 10:10:58,372][20381] Updated weights for policy 1, policy_version 657487 (0.0008) [2023-12-27 10:10:58,741][20347] Updated weights for policy 0, policy_version 658041 (0.0007) [2023-12-27 10:10:58,812][20347] Updated weights for policy 0, policy_version 658051 (0.0008) [2023-12-27 10:10:58,878][20347] Updated weights for policy 0, policy_version 658061 (0.0007) [2023-12-27 10:10:59,188][20381] Updated weights for policy 1, policy_version 657497 (0.0008) [2023-12-27 10:10:59,258][20381] Updated weights for policy 1, policy_version 657507 (0.0012) [2023-12-27 10:10:59,314][20381] Updated weights for policy 1, policy_version 657517 (0.0008) [2023-12-27 10:10:59,379][20381] Updated weights for policy 1, policy_version 657527 (0.0007) [2023-12-27 10:10:59,619][20347] Updated weights for policy 0, policy_version 658071 (0.0007) [2023-12-27 10:10:59,684][20347] Updated weights for policy 0, policy_version 658081 (0.0010) [2023-12-27 10:10:59,742][20347] Updated weights for policy 0, policy_version 658091 (0.0008) [2023-12-27 10:11:00,046][20381] Updated weights for policy 1, policy_version 657537 (0.0007) [2023-12-27 10:11:00,103][20381] Updated weights for policy 1, policy_version 657547 (0.0006) [2023-12-27 10:11:00,117][20292] KL-divergence is very high: 112.3139 [2023-12-27 10:11:00,162][20381] Updated weights for policy 1, policy_version 657557 (0.0006) [2023-12-27 10:11:00,401][20347] Updated weights for policy 0, policy_version 658101 (0.0010) [2023-12-27 10:11:00,460][20347] Updated weights for policy 0, policy_version 658111 (0.0010) [2023-12-27 10:11:00,516][20347] Updated weights for policy 0, policy_version 658121 (0.0008) [2023-12-27 10:11:00,788][20381] Updated weights for policy 1, policy_version 657567 (0.0005) [2023-12-27 10:11:00,848][20381] Updated weights for policy 1, policy_version 657577 (0.0005) [2023-12-27 10:11:00,895][20381] Updated weights for policy 1, policy_version 657587 (0.0008) [2023-12-27 10:11:01,109][20347] Updated weights for policy 0, policy_version 658131 (0.0009) [2023-12-27 10:11:01,169][20347] Updated weights for policy 0, policy_version 658141 (0.0008) [2023-12-27 10:11:01,235][20347] Updated weights for policy 0, policy_version 658151 (0.0006) [2023-12-27 10:11:01,579][20381] Updated weights for policy 1, policy_version 657597 (0.0008) [2023-12-27 10:11:01,642][20381] Updated weights for policy 1, policy_version 657607 (0.0008) [2023-12-27 10:11:01,702][20381] Updated weights for policy 1, policy_version 657617 (0.0008) [2023-12-27 10:11:01,870][20347] Updated weights for policy 0, policy_version 658161 (0.0007) [2023-12-27 10:11:01,928][20347] Updated weights for policy 0, policy_version 658171 (0.0005) [2023-12-27 10:11:01,978][20347] Updated weights for policy 0, policy_version 658181 (0.0005) [2023-12-27 10:11:02,026][20347] Updated weights for policy 0, policy_version 658191 (0.0006) [2023-12-27 10:11:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 336896000. Throughput: 0: 10477.6, 1: 10542.4. Samples: 336908500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:02,363][19170] Avg episode reward: [(0, '192.260'), (1, '836.150')] [2023-12-27 10:11:02,431][20381] Updated weights for policy 1, policy_version 657627 (0.0007) [2023-12-27 10:11:02,490][20381] Updated weights for policy 1, policy_version 657637 (0.0007) [2023-12-27 10:11:02,545][20381] Updated weights for policy 1, policy_version 657647 (0.0008) [2023-12-27 10:11:02,689][20347] Updated weights for policy 0, policy_version 658201 (0.0007) [2023-12-27 10:11:02,744][20347] Updated weights for policy 0, policy_version 658211 (0.0007) [2023-12-27 10:11:02,800][20347] Updated weights for policy 0, policy_version 658221 (0.0007) [2023-12-27 10:11:03,242][20381] Updated weights for policy 1, policy_version 657657 (0.0008) [2023-12-27 10:11:03,277][20292] KL-divergence is very high: 134.7371 [2023-12-27 10:11:03,282][20292] KL-divergence is very high: 163.5441 [2023-12-27 10:11:03,293][20381] Updated weights for policy 1, policy_version 657667 (0.0008) [2023-12-27 10:11:03,298][20292] KL-divergence is very high: 283.7479 [2023-12-27 10:11:03,303][20292] KL-divergence is very high: 183.8687 [2023-12-27 10:11:03,336][20292] KL-divergence is very high: 100.1467 [2023-12-27 10:11:03,341][20381] Updated weights for policy 1, policy_version 657677 (0.0007) [2023-12-27 10:11:03,390][20381] Updated weights for policy 1, policy_version 657687 (0.0007) [2023-12-27 10:11:03,411][20347] Updated weights for policy 0, policy_version 658231 (0.0006) [2023-12-27 10:11:03,466][20347] Updated weights for policy 0, policy_version 658241 (0.0007) [2023-12-27 10:11:03,516][20347] Updated weights for policy 0, policy_version 658251 (0.0008) [2023-12-27 10:11:03,986][20292] KL-divergence is very high: 125.1935 [2023-12-27 10:11:04,020][20381] Updated weights for policy 1, policy_version 657697 (0.0008) [2023-12-27 10:11:04,025][20292] KL-divergence is very high: 152.0413 [2023-12-27 10:11:04,067][20292] KL-divergence is very high: 113.5073 [2023-12-27 10:11:04,074][20381] Updated weights for policy 1, policy_version 657707 (0.0009) [2023-12-27 10:11:04,129][20381] Updated weights for policy 1, policy_version 657717 (0.0008) [2023-12-27 10:11:04,242][20347] Updated weights for policy 0, policy_version 658261 (0.0007) [2023-12-27 10:11:04,300][20347] Updated weights for policy 0, policy_version 658271 (0.0007) [2023-12-27 10:11:04,360][20347] Updated weights for policy 0, policy_version 658281 (0.0008) [2023-12-27 10:11:04,840][20381] Updated weights for policy 1, policy_version 657727 (0.0007) [2023-12-27 10:11:04,893][20381] Updated weights for policy 1, policy_version 657737 (0.0007) [2023-12-27 10:11:04,952][20381] Updated weights for policy 1, policy_version 657747 (0.0007) [2023-12-27 10:11:05,080][20347] Updated weights for policy 0, policy_version 658291 (0.0008) [2023-12-27 10:11:05,145][20347] Updated weights for policy 0, policy_version 658301 (0.0010) [2023-12-27 10:11:05,208][20347] Updated weights for policy 0, policy_version 658311 (0.0011) [2023-12-27 10:11:05,584][20381] Updated weights for policy 1, policy_version 657757 (0.0007) [2023-12-27 10:11:05,635][20381] Updated weights for policy 1, policy_version 657767 (0.0005) [2023-12-27 10:11:05,684][20381] Updated weights for policy 1, policy_version 657777 (0.0005) [2023-12-27 10:11:05,865][20347] Updated weights for policy 0, policy_version 658321 (0.0010) [2023-12-27 10:11:05,922][20347] Updated weights for policy 0, policy_version 658331 (0.0005) [2023-12-27 10:11:05,972][20347] Updated weights for policy 0, policy_version 658341 (0.0006) [2023-12-27 10:11:06,021][20347] Updated weights for policy 0, policy_version 658351 (0.0007) [2023-12-27 10:11:06,268][20381] Updated weights for policy 1, policy_version 657787 (0.0006) [2023-12-27 10:11:06,324][20381] Updated weights for policy 1, policy_version 657797 (0.0006) [2023-12-27 10:11:06,392][20381] Updated weights for policy 1, policy_version 657807 (0.0008) [2023-12-27 10:11:06,633][20347] Updated weights for policy 0, policy_version 658361 (0.0008) [2023-12-27 10:11:06,688][20347] Updated weights for policy 0, policy_version 658371 (0.0008) [2023-12-27 10:11:06,748][20347] Updated weights for policy 0, policy_version 658381 (0.0008) [2023-12-27 10:11:07,082][20381] Updated weights for policy 1, policy_version 657817 (0.0009) [2023-12-27 10:11:07,137][20381] Updated weights for policy 1, policy_version 657827 (0.0007) [2023-12-27 10:11:07,196][20381] Updated weights for policy 1, policy_version 657837 (0.0007) [2023-12-27 10:11:07,241][20381] Updated weights for policy 1, policy_version 657847 (0.0008) [2023-12-27 10:11:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 337002496. Throughput: 0: 10522.1, 1: 10502.6. Samples: 336969332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:07,364][19170] Avg episode reward: [(0, '216.710'), (1, '679.530')] [2023-12-27 10:11:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000657848_168435712.pth... [2023-12-27 10:11:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000658384_168566784.pth... [2023-12-27 10:11:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000657168_168255488.pth [2023-12-27 10:11:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000656632_168124416.pth [2023-12-27 10:11:07,458][20347] Updated weights for policy 0, policy_version 658391 (0.0006) [2023-12-27 10:11:07,517][20347] Updated weights for policy 0, policy_version 658401 (0.0006) [2023-12-27 10:11:07,572][20347] Updated weights for policy 0, policy_version 658411 (0.0006) [2023-12-27 10:11:07,959][20381] Updated weights for policy 1, policy_version 657857 (0.0010) [2023-12-27 10:11:08,018][20381] Updated weights for policy 1, policy_version 657867 (0.0011) [2023-12-27 10:11:08,074][20381] Updated weights for policy 1, policy_version 657877 (0.0011) [2023-12-27 10:11:08,157][20347] Updated weights for policy 0, policy_version 658421 (0.0006) [2023-12-27 10:11:08,204][20347] Updated weights for policy 0, policy_version 658431 (0.0006) [2023-12-27 10:11:08,249][20347] Updated weights for policy 0, policy_version 658441 (0.0005) [2023-12-27 10:11:08,786][20381] Updated weights for policy 1, policy_version 657887 (0.0009) [2023-12-27 10:11:08,850][20381] Updated weights for policy 1, policy_version 657897 (0.0008) [2023-12-27 10:11:08,859][20347] Updated weights for policy 0, policy_version 658451 (0.0006) [2023-12-27 10:11:08,910][20381] Updated weights for policy 1, policy_version 657907 (0.0005) [2023-12-27 10:11:08,916][20347] Updated weights for policy 0, policy_version 658461 (0.0008) [2023-12-27 10:11:08,970][20347] Updated weights for policy 0, policy_version 658471 (0.0006) [2023-12-27 10:11:09,548][20381] Updated weights for policy 1, policy_version 657917 (0.0006) [2023-12-27 10:11:09,600][20347] Updated weights for policy 0, policy_version 658481 (0.0006) [2023-12-27 10:11:09,610][20381] Updated weights for policy 1, policy_version 657927 (0.0006) [2023-12-27 10:11:09,658][20347] Updated weights for policy 0, policy_version 658491 (0.0008) [2023-12-27 10:11:09,669][20381] Updated weights for policy 1, policy_version 657937 (0.0006) [2023-12-27 10:11:09,712][20347] Updated weights for policy 0, policy_version 658501 (0.0008) [2023-12-27 10:11:09,768][20347] Updated weights for policy 0, policy_version 658511 (0.0011) [2023-12-27 10:11:10,310][20381] Updated weights for policy 1, policy_version 657947 (0.0007) [2023-12-27 10:11:10,372][20381] Updated weights for policy 1, policy_version 657957 (0.0006) [2023-12-27 10:11:10,440][20381] Updated weights for policy 1, policy_version 657967 (0.0006) [2023-12-27 10:11:10,473][20347] Updated weights for policy 0, policy_version 658521 (0.0009) [2023-12-27 10:11:10,540][20347] Updated weights for policy 0, policy_version 658531 (0.0008) [2023-12-27 10:11:10,602][20347] Updated weights for policy 0, policy_version 658541 (0.0005) [2023-12-27 10:11:11,102][20381] Updated weights for policy 1, policy_version 657977 (0.0010) [2023-12-27 10:11:11,161][20381] Updated weights for policy 1, policy_version 657987 (0.0008) [2023-12-27 10:11:11,221][20381] Updated weights for policy 1, policy_version 657997 (0.0007) [2023-12-27 10:11:11,234][20347] Updated weights for policy 0, policy_version 658551 (0.0009) [2023-12-27 10:11:11,282][20381] Updated weights for policy 1, policy_version 658007 (0.0006) [2023-12-27 10:11:11,291][20347] Updated weights for policy 0, policy_version 658561 (0.0010) [2023-12-27 10:11:11,351][20347] Updated weights for policy 0, policy_version 658571 (0.0011) [2023-12-27 10:11:11,997][20381] Updated weights for policy 1, policy_version 658017 (0.0007) [2023-12-27 10:11:12,049][20381] Updated weights for policy 1, policy_version 658027 (0.0008) [2023-12-27 10:11:12,056][20347] Updated weights for policy 0, policy_version 658581 (0.0009) [2023-12-27 10:11:12,103][20381] Updated weights for policy 1, policy_version 658037 (0.0009) [2023-12-27 10:11:12,106][20347] Updated weights for policy 0, policy_version 658591 (0.0007) [2023-12-27 10:11:12,159][20347] Updated weights for policy 0, policy_version 658601 (0.0007) [2023-12-27 10:11:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 337108992. Throughput: 0: 10570.1, 1: 10457.0. Samples: 337096144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:12,363][19170] Avg episode reward: [(0, '217.830'), (1, '605.880')] [2023-12-27 10:11:12,814][20381] Updated weights for policy 1, policy_version 658047 (0.0009) [2023-12-27 10:11:12,869][20381] Updated weights for policy 1, policy_version 658057 (0.0010) [2023-12-27 10:11:12,887][20347] Updated weights for policy 0, policy_version 658611 (0.0008) [2023-12-27 10:11:12,921][20381] Updated weights for policy 1, policy_version 658067 (0.0010) [2023-12-27 10:11:12,942][20347] Updated weights for policy 0, policy_version 658621 (0.0010) [2023-12-27 10:11:12,990][20347] Updated weights for policy 0, policy_version 658631 (0.0010) [2023-12-27 10:11:13,604][20347] Updated weights for policy 0, policy_version 658641 (0.0007) [2023-12-27 10:11:13,617][20381] Updated weights for policy 1, policy_version 658077 (0.0011) [2023-12-27 10:11:13,662][20381] Updated weights for policy 1, policy_version 658087 (0.0010) [2023-12-27 10:11:13,663][20347] Updated weights for policy 0, policy_version 658651 (0.0010) [2023-12-27 10:11:13,709][20381] Updated weights for policy 1, policy_version 658097 (0.0010) [2023-12-27 10:11:13,715][20347] Updated weights for policy 0, policy_version 658661 (0.0010) [2023-12-27 10:11:13,767][20347] Updated weights for policy 0, policy_version 658671 (0.0010) [2023-12-27 10:11:14,346][20347] Updated weights for policy 0, policy_version 658681 (0.0006) [2023-12-27 10:11:14,400][20347] Updated weights for policy 0, policy_version 658691 (0.0006) [2023-12-27 10:11:14,420][20381] Updated weights for policy 1, policy_version 658107 (0.0010) [2023-12-27 10:11:14,454][20347] Updated weights for policy 0, policy_version 658701 (0.0006) [2023-12-27 10:11:14,471][20381] Updated weights for policy 1, policy_version 658117 (0.0010) [2023-12-27 10:11:14,526][20381] Updated weights for policy 1, policy_version 658127 (0.0010) [2023-12-27 10:11:15,091][20347] Updated weights for policy 0, policy_version 658711 (0.0005) [2023-12-27 10:11:15,152][20347] Updated weights for policy 0, policy_version 658721 (0.0005) [2023-12-27 10:11:15,212][20347] Updated weights for policy 0, policy_version 658731 (0.0007) [2023-12-27 10:11:15,257][20381] Updated weights for policy 1, policy_version 658137 (0.0010) [2023-12-27 10:11:15,312][20381] Updated weights for policy 1, policy_version 658147 (0.0008) [2023-12-27 10:11:15,374][20381] Updated weights for policy 1, policy_version 658157 (0.0007) [2023-12-27 10:11:15,434][20381] Updated weights for policy 1, policy_version 658167 (0.0008) [2023-12-27 10:11:15,769][20347] Updated weights for policy 0, policy_version 658741 (0.0010) [2023-12-27 10:11:15,829][20347] Updated weights for policy 0, policy_version 658751 (0.0011) [2023-12-27 10:11:15,889][20347] Updated weights for policy 0, policy_version 658761 (0.0011) [2023-12-27 10:11:16,125][20381] Updated weights for policy 1, policy_version 658177 (0.0007) [2023-12-27 10:11:16,180][20381] Updated weights for policy 1, policy_version 658187 (0.0010) [2023-12-27 10:11:16,235][20381] Updated weights for policy 1, policy_version 658197 (0.0010) [2023-12-27 10:11:16,492][20347] Updated weights for policy 0, policy_version 658771 (0.0009) [2023-12-27 10:11:16,540][20347] Updated weights for policy 0, policy_version 658781 (0.0005) [2023-12-27 10:11:16,591][20347] Updated weights for policy 0, policy_version 658791 (0.0005) [2023-12-27 10:11:16,924][20381] Updated weights for policy 1, policy_version 658207 (0.0010) [2023-12-27 10:11:16,976][20381] Updated weights for policy 1, policy_version 658217 (0.0010) [2023-12-27 10:11:17,027][20381] Updated weights for policy 1, policy_version 658227 (0.0010) [2023-12-27 10:11:17,237][20347] Updated weights for policy 0, policy_version 658801 (0.0007) [2023-12-27 10:11:17,292][20347] Updated weights for policy 0, policy_version 658811 (0.0009) [2023-12-27 10:11:17,352][20347] Updated weights for policy 0, policy_version 658821 (0.0005) [2023-12-27 10:11:17,363][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 337207296. Throughput: 0: 10607.9, 1: 10293.6. Samples: 337220604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:17,364][19170] Avg episode reward: [(0, '207.250'), (1, '807.710')] [2023-12-27 10:11:17,405][20347] Updated weights for policy 0, policy_version 658831 (0.0010) [2023-12-27 10:11:17,719][20381] Updated weights for policy 1, policy_version 658237 (0.0011) [2023-12-27 10:11:17,777][20381] Updated weights for policy 1, policy_version 658247 (0.0008) [2023-12-27 10:11:17,832][20381] Updated weights for policy 1, policy_version 658257 (0.0007) [2023-12-27 10:11:18,060][20347] Updated weights for policy 0, policy_version 658841 (0.0010) [2023-12-27 10:11:18,119][20347] Updated weights for policy 0, policy_version 658851 (0.0010) [2023-12-27 10:11:18,181][20347] Updated weights for policy 0, policy_version 658861 (0.0010) [2023-12-27 10:11:18,499][20381] Updated weights for policy 1, policy_version 658267 (0.0010) [2023-12-27 10:11:18,552][20381] Updated weights for policy 1, policy_version 658277 (0.0010) [2023-12-27 10:11:18,602][20381] Updated weights for policy 1, policy_version 658287 (0.0010) [2023-12-27 10:11:18,856][20347] Updated weights for policy 0, policy_version 658871 (0.0008) [2023-12-27 10:11:18,900][20347] Updated weights for policy 0, policy_version 658881 (0.0007) [2023-12-27 10:11:18,951][20347] Updated weights for policy 0, policy_version 658891 (0.0007) [2023-12-27 10:11:19,296][20381] Updated weights for policy 1, policy_version 658297 (0.0009) [2023-12-27 10:11:19,360][20381] Updated weights for policy 1, policy_version 658307 (0.0010) [2023-12-27 10:11:19,413][20381] Updated weights for policy 1, policy_version 658317 (0.0007) [2023-12-27 10:11:19,470][20381] Updated weights for policy 1, policy_version 658327 (0.0010) [2023-12-27 10:11:19,675][20347] Updated weights for policy 0, policy_version 658901 (0.0008) [2023-12-27 10:11:19,728][20347] Updated weights for policy 0, policy_version 658911 (0.0009) [2023-12-27 10:11:19,783][20347] Updated weights for policy 0, policy_version 658921 (0.0007) [2023-12-27 10:11:20,113][20381] Updated weights for policy 1, policy_version 658337 (0.0009) [2023-12-27 10:11:20,172][20381] Updated weights for policy 1, policy_version 658347 (0.0011) [2023-12-27 10:11:20,233][20381] Updated weights for policy 1, policy_version 658357 (0.0011) [2023-12-27 10:11:20,526][20347] Updated weights for policy 0, policy_version 658931 (0.0006) [2023-12-27 10:11:20,586][20347] Updated weights for policy 0, policy_version 658941 (0.0006) [2023-12-27 10:11:20,652][20347] Updated weights for policy 0, policy_version 658951 (0.0008) [2023-12-27 10:11:20,895][20381] Updated weights for policy 1, policy_version 658367 (0.0008) [2023-12-27 10:11:20,955][20381] Updated weights for policy 1, policy_version 658377 (0.0008) [2023-12-27 10:11:21,019][20381] Updated weights for policy 1, policy_version 658387 (0.0008) [2023-12-27 10:11:21,281][20347] Updated weights for policy 0, policy_version 658961 (0.0007) [2023-12-27 10:11:21,344][20347] Updated weights for policy 0, policy_version 658971 (0.0008) [2023-12-27 10:11:21,407][20347] Updated weights for policy 0, policy_version 658981 (0.0007) [2023-12-27 10:11:21,468][20347] Updated weights for policy 0, policy_version 658991 (0.0008) [2023-12-27 10:11:21,729][20381] Updated weights for policy 1, policy_version 658397 (0.0008) [2023-12-27 10:11:21,783][20381] Updated weights for policy 1, policy_version 658407 (0.0009) [2023-12-27 10:11:21,838][20381] Updated weights for policy 1, policy_version 658417 (0.0007) [2023-12-27 10:11:22,138][20347] Updated weights for policy 0, policy_version 659001 (0.0005) [2023-12-27 10:11:22,198][20347] Updated weights for policy 0, policy_version 659011 (0.0007) [2023-12-27 10:11:22,254][20347] Updated weights for policy 0, policy_version 659021 (0.0009) [2023-12-27 10:11:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 21026.1, 300 sec: 20799.3). Total num frames: 337313792. Throughput: 0: 10561.5, 1: 10289.2. Samples: 337281444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:22,363][19170] Avg episode reward: [(0, '204.930'), (1, '906.120')] [2023-12-27 10:11:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000658424_168583168.pth... [2023-12-27 10:11:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000659024_168730624.pth... [2023-12-27 10:11:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000657776_168411136.pth [2023-12-27 10:11:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000657240_168280064.pth [2023-12-27 10:11:22,524][20381] Updated weights for policy 1, policy_version 658427 (0.0008) [2023-12-27 10:11:22,583][20381] Updated weights for policy 1, policy_version 658437 (0.0007) [2023-12-27 10:11:22,641][20381] Updated weights for policy 1, policy_version 658447 (0.0007) [2023-12-27 10:11:22,903][20347] Updated weights for policy 0, policy_version 659031 (0.0007) [2023-12-27 10:11:22,958][20347] Updated weights for policy 0, policy_version 659041 (0.0007) [2023-12-27 10:11:23,013][20347] Updated weights for policy 0, policy_version 659051 (0.0008) [2023-12-27 10:11:23,360][20381] Updated weights for policy 1, policy_version 658457 (0.0007) [2023-12-27 10:11:23,423][20381] Updated weights for policy 1, policy_version 658467 (0.0008) [2023-12-27 10:11:23,478][20381] Updated weights for policy 1, policy_version 658477 (0.0009) [2023-12-27 10:11:23,541][20381] Updated weights for policy 1, policy_version 658487 (0.0008) [2023-12-27 10:11:23,657][20347] Updated weights for policy 0, policy_version 659061 (0.0007) [2023-12-27 10:11:23,714][20347] Updated weights for policy 0, policy_version 659071 (0.0006) [2023-12-27 10:11:23,772][20347] Updated weights for policy 0, policy_version 659081 (0.0008) [2023-12-27 10:11:24,213][20381] Updated weights for policy 1, policy_version 658497 (0.0008) [2023-12-27 10:11:24,284][20381] Updated weights for policy 1, policy_version 658507 (0.0008) [2023-12-27 10:11:24,339][20381] Updated weights for policy 1, policy_version 658517 (0.0008) [2023-12-27 10:11:24,368][20347] Updated weights for policy 0, policy_version 659091 (0.0007) [2023-12-27 10:11:24,425][20347] Updated weights for policy 0, policy_version 659101 (0.0006) [2023-12-27 10:11:24,488][20347] Updated weights for policy 0, policy_version 659111 (0.0006) [2023-12-27 10:11:25,058][20347] Updated weights for policy 0, policy_version 659121 (0.0007) [2023-12-27 10:11:25,060][20381] Updated weights for policy 1, policy_version 658527 (0.0007) [2023-12-27 10:11:25,109][20347] Updated weights for policy 0, policy_version 659131 (0.0008) [2023-12-27 10:11:25,116][20381] Updated weights for policy 1, policy_version 658537 (0.0007) [2023-12-27 10:11:25,167][20347] Updated weights for policy 0, policy_version 659141 (0.0006) [2023-12-27 10:11:25,170][20381] Updated weights for policy 1, policy_version 658547 (0.0007) [2023-12-27 10:11:25,227][20347] Updated weights for policy 0, policy_version 659151 (0.0005) [2023-12-27 10:11:25,758][20347] Updated weights for policy 0, policy_version 659161 (0.0010) [2023-12-27 10:11:25,817][20347] Updated weights for policy 0, policy_version 659171 (0.0007) [2023-12-27 10:11:25,878][20347] Updated weights for policy 0, policy_version 659181 (0.0007) [2023-12-27 10:11:25,921][20381] Updated weights for policy 1, policy_version 658557 (0.0006) [2023-12-27 10:11:25,986][20381] Updated weights for policy 1, policy_version 658567 (0.0005) [2023-12-27 10:11:26,045][20381] Updated weights for policy 1, policy_version 658577 (0.0007) [2023-12-27 10:11:26,512][20347] Updated weights for policy 0, policy_version 659191 (0.0006) [2023-12-27 10:11:26,572][20347] Updated weights for policy 0, policy_version 659201 (0.0007) [2023-12-27 10:11:26,629][20347] Updated weights for policy 0, policy_version 659211 (0.0005) [2023-12-27 10:11:26,708][20381] Updated weights for policy 1, policy_version 658587 (0.0008) [2023-12-27 10:11:26,756][20381] Updated weights for policy 1, policy_version 658597 (0.0008) [2023-12-27 10:11:26,807][20381] Updated weights for policy 1, policy_version 658607 (0.0008) [2023-12-27 10:11:27,187][20347] Updated weights for policy 0, policy_version 659221 (0.0008) [2023-12-27 10:11:27,235][20347] Updated weights for policy 0, policy_version 659231 (0.0010) [2023-12-27 10:11:27,282][20347] Updated weights for policy 0, policy_version 659241 (0.0010) [2023-12-27 10:11:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.1, 300 sec: 20799.4). Total num frames: 337420288. Throughput: 0: 10598.6, 1: 10250.4. Samples: 337406196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:27,363][19170] Avg episode reward: [(0, '163.520'), (1, '895.250')] [2023-12-27 10:11:27,536][20381] Updated weights for policy 1, policy_version 658617 (0.0007) [2023-12-27 10:11:27,595][20381] Updated weights for policy 1, policy_version 658627 (0.0007) [2023-12-27 10:11:27,645][20381] Updated weights for policy 1, policy_version 658637 (0.0007) [2023-12-27 10:11:27,695][20381] Updated weights for policy 1, policy_version 658647 (0.0007) [2023-12-27 10:11:27,954][20347] Updated weights for policy 0, policy_version 659251 (0.0010) [2023-12-27 10:11:27,995][20347] Updated weights for policy 0, policy_version 659261 (0.0010) [2023-12-27 10:11:28,046][20347] Updated weights for policy 0, policy_version 659271 (0.0010) [2023-12-27 10:11:28,366][20381] Updated weights for policy 1, policy_version 658657 (0.0008) [2023-12-27 10:11:28,428][20381] Updated weights for policy 1, policy_version 658667 (0.0008) [2023-12-27 10:11:28,482][20381] Updated weights for policy 1, policy_version 658677 (0.0007) [2023-12-27 10:11:28,749][20347] Updated weights for policy 0, policy_version 659281 (0.0010) [2023-12-27 10:11:28,811][20347] Updated weights for policy 0, policy_version 659291 (0.0010) [2023-12-27 10:11:28,876][20347] Updated weights for policy 0, policy_version 659301 (0.0010) [2023-12-27 10:11:28,938][20347] Updated weights for policy 0, policy_version 659311 (0.0010) [2023-12-27 10:11:29,170][20381] Updated weights for policy 1, policy_version 658687 (0.0008) [2023-12-27 10:11:29,230][20381] Updated weights for policy 1, policy_version 658697 (0.0008) [2023-12-27 10:11:29,288][20381] Updated weights for policy 1, policy_version 658707 (0.0008) [2023-12-27 10:11:29,558][20347] Updated weights for policy 0, policy_version 659321 (0.0007) [2023-12-27 10:11:29,609][20347] Updated weights for policy 0, policy_version 659331 (0.0005) [2023-12-27 10:11:29,670][20347] Updated weights for policy 0, policy_version 659341 (0.0005) [2023-12-27 10:11:29,973][20381] Updated weights for policy 1, policy_version 658717 (0.0008) [2023-12-27 10:11:30,031][20381] Updated weights for policy 1, policy_version 658727 (0.0008) [2023-12-27 10:11:30,090][20381] Updated weights for policy 1, policy_version 658737 (0.0008) [2023-12-27 10:11:30,331][20347] Updated weights for policy 0, policy_version 659351 (0.0007) [2023-12-27 10:11:30,388][20347] Updated weights for policy 0, policy_version 659361 (0.0008) [2023-12-27 10:11:30,437][20347] Updated weights for policy 0, policy_version 659371 (0.0007) [2023-12-27 10:11:30,677][20381] Updated weights for policy 1, policy_version 658747 (0.0008) [2023-12-27 10:11:30,734][20381] Updated weights for policy 1, policy_version 658757 (0.0010) [2023-12-27 10:11:30,785][20381] Updated weights for policy 1, policy_version 658767 (0.0010) [2023-12-27 10:11:31,154][20347] Updated weights for policy 0, policy_version 659381 (0.0008) [2023-12-27 10:11:31,201][20347] Updated weights for policy 0, policy_version 659391 (0.0007) [2023-12-27 10:11:31,254][20347] Updated weights for policy 0, policy_version 659401 (0.0007) [2023-12-27 10:11:31,483][20381] Updated weights for policy 1, policy_version 658777 (0.0010) [2023-12-27 10:11:31,534][20381] Updated weights for policy 1, policy_version 658787 (0.0010) [2023-12-27 10:11:31,585][20381] Updated weights for policy 1, policy_version 658797 (0.0007) [2023-12-27 10:11:31,641][20381] Updated weights for policy 1, policy_version 658807 (0.0011) [2023-12-27 10:11:31,975][20347] Updated weights for policy 0, policy_version 659411 (0.0007) [2023-12-27 10:11:32,022][20347] Updated weights for policy 0, policy_version 659421 (0.0007) [2023-12-27 10:11:32,074][20347] Updated weights for policy 0, policy_version 659431 (0.0007) [2023-12-27 10:11:32,339][20381] Updated weights for policy 1, policy_version 658817 (0.0011) [2023-12-27 10:11:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 337518592. Throughput: 0: 10602.7, 1: 10209.1. Samples: 337530724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:32,363][19170] Avg episode reward: [(0, '198.110'), (1, '903.110')] [2023-12-27 10:11:32,402][20381] Updated weights for policy 1, policy_version 658827 (0.0011) [2023-12-27 10:11:32,460][20381] Updated weights for policy 1, policy_version 658837 (0.0010) [2023-12-27 10:11:32,800][20347] Updated weights for policy 0, policy_version 659441 (0.0007) [2023-12-27 10:11:32,841][20347] Updated weights for policy 0, policy_version 659451 (0.0008) [2023-12-27 10:11:32,889][20347] Updated weights for policy 0, policy_version 659461 (0.0007) [2023-12-27 10:11:32,950][20347] Updated weights for policy 0, policy_version 659471 (0.0007) [2023-12-27 10:11:33,116][20381] Updated weights for policy 1, policy_version 658847 (0.0010) [2023-12-27 10:11:33,160][20381] Updated weights for policy 1, policy_version 658857 (0.0010) [2023-12-27 10:11:33,214][20381] Updated weights for policy 1, policy_version 658867 (0.0010) [2023-12-27 10:11:33,560][20347] Updated weights for policy 0, policy_version 659481 (0.0005) [2023-12-27 10:11:33,618][20347] Updated weights for policy 0, policy_version 659491 (0.0005) [2023-12-27 10:11:33,674][20347] Updated weights for policy 0, policy_version 659501 (0.0005) [2023-12-27 10:11:33,903][20381] Updated weights for policy 1, policy_version 658877 (0.0010) [2023-12-27 10:11:33,951][20381] Updated weights for policy 1, policy_version 658887 (0.0010) [2023-12-27 10:11:33,997][20381] Updated weights for policy 1, policy_version 658897 (0.0010) [2023-12-27 10:11:34,217][20347] Updated weights for policy 0, policy_version 659511 (0.0007) [2023-12-27 10:11:34,268][20347] Updated weights for policy 0, policy_version 659521 (0.0007) [2023-12-27 10:11:34,317][20347] Updated weights for policy 0, policy_version 659531 (0.0008) [2023-12-27 10:11:34,683][20381] Updated weights for policy 1, policy_version 658907 (0.0010) [2023-12-27 10:11:34,738][20381] Updated weights for policy 1, policy_version 658917 (0.0011) [2023-12-27 10:11:34,797][20381] Updated weights for policy 1, policy_version 658927 (0.0009) [2023-12-27 10:11:35,033][20347] Updated weights for policy 0, policy_version 659541 (0.0007) [2023-12-27 10:11:35,088][20347] Updated weights for policy 0, policy_version 659551 (0.0007) [2023-12-27 10:11:35,147][20347] Updated weights for policy 0, policy_version 659561 (0.0007) [2023-12-27 10:11:35,484][20381] Updated weights for policy 1, policy_version 658937 (0.0010) [2023-12-27 10:11:35,539][20381] Updated weights for policy 1, policy_version 658947 (0.0010) [2023-12-27 10:11:35,590][20381] Updated weights for policy 1, policy_version 658957 (0.0010) [2023-12-27 10:11:35,639][20381] Updated weights for policy 1, policy_version 658967 (0.0010) [2023-12-27 10:11:35,861][20347] Updated weights for policy 0, policy_version 659571 (0.0008) [2023-12-27 10:11:35,911][20347] Updated weights for policy 0, policy_version 659581 (0.0007) [2023-12-27 10:11:35,955][20347] Updated weights for policy 0, policy_version 659591 (0.0007) [2023-12-27 10:11:36,342][20381] Updated weights for policy 1, policy_version 658977 (0.0010) [2023-12-27 10:11:36,404][20381] Updated weights for policy 1, policy_version 658987 (0.0010) [2023-12-27 10:11:36,453][20381] Updated weights for policy 1, policy_version 658997 (0.0010) [2023-12-27 10:11:36,679][20347] Updated weights for policy 0, policy_version 659601 (0.0007) [2023-12-27 10:11:36,732][20347] Updated weights for policy 0, policy_version 659611 (0.0008) [2023-12-27 10:11:36,789][20347] Updated weights for policy 0, policy_version 659621 (0.0007) [2023-12-27 10:11:36,855][20347] Updated weights for policy 0, policy_version 659631 (0.0008) [2023-12-27 10:11:37,138][20381] Updated weights for policy 1, policy_version 659007 (0.0008) [2023-12-27 10:11:37,190][20381] Updated weights for policy 1, policy_version 659017 (0.0010) [2023-12-27 10:11:37,269][20381] Updated weights for policy 1, policy_version 659027 (0.0010) [2023-12-27 10:11:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 337625088. Throughput: 0: 10613.9, 1: 10196.3. Samples: 337593624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:37,363][19170] Avg episode reward: [(0, '227.290'), (1, '866.130')] [2023-12-27 10:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000659032_168738816.pth... [2023-12-27 10:11:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000659632_168886272.pth... [2023-12-27 10:11:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000657848_168435712.pth [2023-12-27 10:11:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000658384_168566784.pth [2023-12-27 10:11:37,555][20347] Updated weights for policy 0, policy_version 659641 (0.0005) [2023-12-27 10:11:37,604][20347] Updated weights for policy 0, policy_version 659651 (0.0006) [2023-12-27 10:11:37,652][20347] Updated weights for policy 0, policy_version 659661 (0.0007) [2023-12-27 10:11:37,843][20381] Updated weights for policy 1, policy_version 659037 (0.0008) [2023-12-27 10:11:37,894][20381] Updated weights for policy 1, policy_version 659047 (0.0010) [2023-12-27 10:11:37,955][20381] Updated weights for policy 1, policy_version 659057 (0.0010) [2023-12-27 10:11:38,274][20347] Updated weights for policy 0, policy_version 659671 (0.0007) [2023-12-27 10:11:38,324][20347] Updated weights for policy 0, policy_version 659681 (0.0005) [2023-12-27 10:11:38,386][20347] Updated weights for policy 0, policy_version 659691 (0.0007) [2023-12-27 10:11:38,554][20381] Updated weights for policy 1, policy_version 659067 (0.0010) [2023-12-27 10:11:38,603][20381] Updated weights for policy 1, policy_version 659077 (0.0009) [2023-12-27 10:11:38,648][20381] Updated weights for policy 1, policy_version 659087 (0.0010) [2023-12-27 10:11:38,965][20347] Updated weights for policy 0, policy_version 659701 (0.0010) [2023-12-27 10:11:39,033][20347] Updated weights for policy 0, policy_version 659711 (0.0009) [2023-12-27 10:11:39,098][20347] Updated weights for policy 0, policy_version 659721 (0.0010) [2023-12-27 10:11:39,301][20381] Updated weights for policy 1, policy_version 659097 (0.0010) [2023-12-27 10:11:39,359][20381] Updated weights for policy 1, policy_version 659107 (0.0011) [2023-12-27 10:11:39,428][20381] Updated weights for policy 1, policy_version 659117 (0.0010) [2023-12-27 10:11:39,490][20381] Updated weights for policy 1, policy_version 659127 (0.0011) [2023-12-27 10:11:39,761][20347] Updated weights for policy 0, policy_version 659731 (0.0009) [2023-12-27 10:11:39,828][20347] Updated weights for policy 0, policy_version 659741 (0.0008) [2023-12-27 10:11:39,891][20347] Updated weights for policy 0, policy_version 659751 (0.0007) [2023-12-27 10:11:40,220][20381] Updated weights for policy 1, policy_version 659137 (0.0011) [2023-12-27 10:11:40,281][20381] Updated weights for policy 1, policy_version 659147 (0.0011) [2023-12-27 10:11:40,337][20381] Updated weights for policy 1, policy_version 659157 (0.0010) [2023-12-27 10:11:40,556][20347] Updated weights for policy 0, policy_version 659761 (0.0008) [2023-12-27 10:11:40,610][20347] Updated weights for policy 0, policy_version 659771 (0.0007) [2023-12-27 10:11:40,665][20347] Updated weights for policy 0, policy_version 659781 (0.0007) [2023-12-27 10:11:40,720][20347] Updated weights for policy 0, policy_version 659791 (0.0008) [2023-12-27 10:11:41,040][20381] Updated weights for policy 1, policy_version 659167 (0.0009) [2023-12-27 10:11:41,097][20381] Updated weights for policy 1, policy_version 659177 (0.0009) [2023-12-27 10:11:41,158][20381] Updated weights for policy 1, policy_version 659187 (0.0009) [2023-12-27 10:11:41,398][20347] Updated weights for policy 0, policy_version 659801 (0.0007) [2023-12-27 10:11:41,464][20347] Updated weights for policy 0, policy_version 659811 (0.0005) [2023-12-27 10:11:41,529][20347] Updated weights for policy 0, policy_version 659821 (0.0006) [2023-12-27 10:11:41,916][20381] Updated weights for policy 1, policy_version 659197 (0.0010) [2023-12-27 10:11:41,980][20381] Updated weights for policy 1, policy_version 659207 (0.0007) [2023-12-27 10:11:42,040][20381] Updated weights for policy 1, policy_version 659217 (0.0008) [2023-12-27 10:11:42,198][20347] Updated weights for policy 0, policy_version 659831 (0.0007) [2023-12-27 10:11:42,259][20347] Updated weights for policy 0, policy_version 659841 (0.0008) [2023-12-27 10:11:42,317][20347] Updated weights for policy 0, policy_version 659851 (0.0008) [2023-12-27 10:11:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 337731584. Throughput: 0: 10531.5, 1: 10201.1. Samples: 337717376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:11:42,363][19170] Avg episode reward: [(0, '242.920'), (1, '847.550')] [2023-12-27 10:11:42,777][20381] Updated weights for policy 1, policy_version 659227 (0.0009) [2023-12-27 10:11:42,835][20381] Updated weights for policy 1, policy_version 659237 (0.0008) [2023-12-27 10:11:42,885][20381] Updated weights for policy 1, policy_version 659247 (0.0008) [2023-12-27 10:11:42,966][20347] Updated weights for policy 0, policy_version 659861 (0.0007) [2023-12-27 10:11:43,014][20347] Updated weights for policy 0, policy_version 659871 (0.0006) [2023-12-27 10:11:43,076][20347] Updated weights for policy 0, policy_version 659881 (0.0007) [2023-12-27 10:11:43,565][20381] Updated weights for policy 1, policy_version 659257 (0.0010) [2023-12-27 10:11:43,623][20381] Updated weights for policy 1, policy_version 659267 (0.0008) [2023-12-27 10:11:43,675][20381] Updated weights for policy 1, policy_version 659277 (0.0007) [2023-12-27 10:11:43,702][20347] Updated weights for policy 0, policy_version 659891 (0.0009) [2023-12-27 10:11:43,734][20381] Updated weights for policy 1, policy_version 659287 (0.0006) [2023-12-27 10:11:43,749][20347] Updated weights for policy 0, policy_version 659901 (0.0006) [2023-12-27 10:11:43,797][20347] Updated weights for policy 0, policy_version 659911 (0.0008) [2023-12-27 10:11:44,352][20381] Updated weights for policy 1, policy_version 659297 (0.0005) [2023-12-27 10:11:44,410][20381] Updated weights for policy 1, policy_version 659307 (0.0005) [2023-12-27 10:11:44,465][20381] Updated weights for policy 1, policy_version 659317 (0.0005) [2023-12-27 10:11:44,519][20347] Updated weights for policy 0, policy_version 659921 (0.0008) [2023-12-27 10:11:44,566][20347] Updated weights for policy 0, policy_version 659931 (0.0008) [2023-12-27 10:11:44,616][20347] Updated weights for policy 0, policy_version 659941 (0.0010) [2023-12-27 10:11:44,664][20347] Updated weights for policy 0, policy_version 659951 (0.0007) [2023-12-27 10:11:45,048][20381] Updated weights for policy 1, policy_version 659327 (0.0009) [2023-12-27 10:11:45,101][20381] Updated weights for policy 1, policy_version 659337 (0.0011) [2023-12-27 10:11:45,167][20381] Updated weights for policy 1, policy_version 659347 (0.0011) [2023-12-27 10:11:45,373][20347] Updated weights for policy 0, policy_version 659961 (0.0009) [2023-12-27 10:11:45,441][20347] Updated weights for policy 0, policy_version 659971 (0.0007) [2023-12-27 10:11:45,502][20347] Updated weights for policy 0, policy_version 659981 (0.0008) [2023-12-27 10:11:45,769][20381] Updated weights for policy 1, policy_version 659357 (0.0009) [2023-12-27 10:11:45,816][20381] Updated weights for policy 1, policy_version 659367 (0.0010) [2023-12-27 10:11:45,876][20381] Updated weights for policy 1, policy_version 659377 (0.0010) [2023-12-27 10:11:46,125][20347] Updated weights for policy 0, policy_version 659991 (0.0006) [2023-12-27 10:11:46,182][20347] Updated weights for policy 0, policy_version 660001 (0.0007) [2023-12-27 10:11:46,236][20347] Updated weights for policy 0, policy_version 660011 (0.0005) [2023-12-27 10:11:46,559][20381] Updated weights for policy 1, policy_version 659387 (0.0008) [2023-12-27 10:11:46,618][20381] Updated weights for policy 1, policy_version 659397 (0.0010) [2023-12-27 10:11:46,676][20381] Updated weights for policy 1, policy_version 659407 (0.0006) [2023-12-27 10:11:46,839][20347] Updated weights for policy 0, policy_version 660021 (0.0008) [2023-12-27 10:11:46,890][20347] Updated weights for policy 0, policy_version 660031 (0.0009) [2023-12-27 10:11:46,939][20347] Updated weights for policy 0, policy_version 660041 (0.0008) [2023-12-27 10:11:47,227][20381] Updated weights for policy 1, policy_version 659417 (0.0005) [2023-12-27 10:11:47,283][20381] Updated weights for policy 1, policy_version 659427 (0.0006) [2023-12-27 10:11:47,332][20381] Updated weights for policy 1, policy_version 659437 (0.0007) [2023-12-27 10:11:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 337829888. Throughput: 0: 10549.5, 1: 10238.8. Samples: 337843972. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:11:47,363][19170] Avg episode reward: [(0, '265.250'), (1, '919.650')] [2023-12-27 10:11:47,383][20381] Updated weights for policy 1, policy_version 659447 (0.0007) [2023-12-27 10:11:47,633][20347] Updated weights for policy 0, policy_version 660051 (0.0008) [2023-12-27 10:11:47,688][20347] Updated weights for policy 0, policy_version 660061 (0.0008) [2023-12-27 10:11:47,750][20347] Updated weights for policy 0, policy_version 660071 (0.0008) [2023-12-27 10:11:48,040][20381] Updated weights for policy 1, policy_version 659457 (0.0008) [2023-12-27 10:11:48,093][20381] Updated weights for policy 1, policy_version 659467 (0.0008) [2023-12-27 10:11:48,145][20381] Updated weights for policy 1, policy_version 659477 (0.0007) [2023-12-27 10:11:48,458][20347] Updated weights for policy 0, policy_version 660081 (0.0008) [2023-12-27 10:11:48,510][20347] Updated weights for policy 0, policy_version 660091 (0.0007) [2023-12-27 10:11:48,564][20347] Updated weights for policy 0, policy_version 660101 (0.0007) [2023-12-27 10:11:48,622][20347] Updated weights for policy 0, policy_version 660111 (0.0009) [2023-12-27 10:11:48,732][20381] Updated weights for policy 1, policy_version 659487 (0.0006) [2023-12-27 10:11:48,792][20381] Updated weights for policy 1, policy_version 659497 (0.0005) [2023-12-27 10:11:48,858][20381] Updated weights for policy 1, policy_version 659507 (0.0007) [2023-12-27 10:11:49,380][20347] Updated weights for policy 0, policy_version 660121 (0.0007) [2023-12-27 10:11:49,437][20347] Updated weights for policy 0, policy_version 660131 (0.0008) [2023-12-27 10:11:49,488][20381] Updated weights for policy 1, policy_version 659517 (0.0007) [2023-12-27 10:11:49,498][20347] Updated weights for policy 0, policy_version 660141 (0.0006) [2023-12-27 10:11:49,539][20381] Updated weights for policy 1, policy_version 659527 (0.0006) [2023-12-27 10:11:49,588][20381] Updated weights for policy 1, policy_version 659537 (0.0007) [2023-12-27 10:11:50,227][20347] Updated weights for policy 0, policy_version 660151 (0.0007) [2023-12-27 10:11:50,284][20347] Updated weights for policy 0, policy_version 660161 (0.0008) [2023-12-27 10:11:50,314][20381] Updated weights for policy 1, policy_version 659547 (0.0008) [2023-12-27 10:11:50,337][20347] Updated weights for policy 0, policy_version 660171 (0.0006) [2023-12-27 10:11:50,380][20381] Updated weights for policy 1, policy_version 659557 (0.0009) [2023-12-27 10:11:50,444][20381] Updated weights for policy 1, policy_version 659567 (0.0007) [2023-12-27 10:11:51,007][20347] Updated weights for policy 0, policy_version 660181 (0.0007) [2023-12-27 10:11:51,063][20347] Updated weights for policy 0, policy_version 660191 (0.0006) [2023-12-27 10:11:51,115][20347] Updated weights for policy 0, policy_version 660201 (0.0005) [2023-12-27 10:11:51,132][20381] Updated weights for policy 1, policy_version 659577 (0.0008) [2023-12-27 10:11:51,201][20381] Updated weights for policy 1, policy_version 659587 (0.0006) [2023-12-27 10:11:51,266][20381] Updated weights for policy 1, policy_version 659597 (0.0008) [2023-12-27 10:11:51,329][20381] Updated weights for policy 1, policy_version 659607 (0.0007) [2023-12-27 10:11:51,798][20347] Updated weights for policy 0, policy_version 660211 (0.0007) [2023-12-27 10:11:51,861][20347] Updated weights for policy 0, policy_version 660221 (0.0009) [2023-12-27 10:11:51,926][20347] Updated weights for policy 0, policy_version 660231 (0.0008) [2023-12-27 10:11:51,973][20381] Updated weights for policy 1, policy_version 659617 (0.0007) [2023-12-27 10:11:52,028][20381] Updated weights for policy 1, policy_version 659627 (0.0008) [2023-12-27 10:11:52,090][20381] Updated weights for policy 1, policy_version 659637 (0.0005) [2023-12-27 10:11:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 337936384. Throughput: 0: 10526.2, 1: 10274.5. Samples: 337905360. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:11:52,363][19170] Avg episode reward: [(0, '255.390'), (1, '869.490')] [2023-12-27 10:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000659640_168894464.pth... [2023-12-27 10:11:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000660240_169041920.pth... [2023-12-27 10:11:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000659024_168730624.pth [2023-12-27 10:11:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000658424_168583168.pth [2023-12-27 10:11:52,594][20347] Updated weights for policy 0, policy_version 660241 (0.0006) [2023-12-27 10:11:52,657][20347] Updated weights for policy 0, policy_version 660251 (0.0006) [2023-12-27 10:11:52,719][20347] Updated weights for policy 0, policy_version 660261 (0.0007) [2023-12-27 10:11:52,775][20381] Updated weights for policy 1, policy_version 659647 (0.0007) [2023-12-27 10:11:52,785][20347] Updated weights for policy 0, policy_version 660271 (0.0005) [2023-12-27 10:11:52,835][20381] Updated weights for policy 1, policy_version 659657 (0.0008) [2023-12-27 10:11:52,898][20381] Updated weights for policy 1, policy_version 659667 (0.0008) [2023-12-27 10:11:53,424][20347] Updated weights for policy 0, policy_version 660281 (0.0008) [2023-12-27 10:11:53,476][20347] Updated weights for policy 0, policy_version 660291 (0.0009) [2023-12-27 10:11:53,520][20381] Updated weights for policy 1, policy_version 659677 (0.0008) [2023-12-27 10:11:53,534][20347] Updated weights for policy 0, policy_version 660301 (0.0007) [2023-12-27 10:11:53,578][20381] Updated weights for policy 1, policy_version 659687 (0.0007) [2023-12-27 10:11:53,640][20381] Updated weights for policy 1, policy_version 659697 (0.0008) [2023-12-27 10:11:54,260][20347] Updated weights for policy 0, policy_version 660311 (0.0007) [2023-12-27 10:11:54,262][20381] Updated weights for policy 1, policy_version 659707 (0.0007) [2023-12-27 10:11:54,313][20347] Updated weights for policy 0, policy_version 660321 (0.0005) [2023-12-27 10:11:54,315][20381] Updated weights for policy 1, policy_version 659717 (0.0006) [2023-12-27 10:11:54,369][20347] Updated weights for policy 0, policy_version 660331 (0.0006) [2023-12-27 10:11:54,375][20381] Updated weights for policy 1, policy_version 659727 (0.0006) [2023-12-27 10:11:55,002][20347] Updated weights for policy 0, policy_version 660341 (0.0008) [2023-12-27 10:11:55,022][20381] Updated weights for policy 1, policy_version 659737 (0.0008) [2023-12-27 10:11:55,056][20347] Updated weights for policy 0, policy_version 660351 (0.0009) [2023-12-27 10:11:55,076][20381] Updated weights for policy 1, policy_version 659747 (0.0008) [2023-12-27 10:11:55,111][20347] Updated weights for policy 0, policy_version 660361 (0.0006) [2023-12-27 10:11:55,132][20381] Updated weights for policy 1, policy_version 659757 (0.0006) [2023-12-27 10:11:55,191][20381] Updated weights for policy 1, policy_version 659767 (0.0005) [2023-12-27 10:11:55,777][20347] Updated weights for policy 0, policy_version 660371 (0.0007) [2023-12-27 10:11:55,834][20347] Updated weights for policy 0, policy_version 660381 (0.0008) [2023-12-27 10:11:55,859][20381] Updated weights for policy 1, policy_version 659777 (0.0005) [2023-12-27 10:11:55,891][20347] Updated weights for policy 0, policy_version 660391 (0.0006) [2023-12-27 10:11:55,915][20381] Updated weights for policy 1, policy_version 659787 (0.0006) [2023-12-27 10:11:55,969][20381] Updated weights for policy 1, policy_version 659797 (0.0007) [2023-12-27 10:11:56,524][20347] Updated weights for policy 0, policy_version 660401 (0.0006) [2023-12-27 10:11:56,575][20347] Updated weights for policy 0, policy_version 660411 (0.0010) [2023-12-27 10:11:56,576][20381] Updated weights for policy 1, policy_version 659807 (0.0006) [2023-12-27 10:11:56,620][20347] Updated weights for policy 0, policy_version 660421 (0.0010) [2023-12-27 10:11:56,634][20381] Updated weights for policy 1, policy_version 659817 (0.0006) [2023-12-27 10:11:56,668][20347] Updated weights for policy 0, policy_version 660431 (0.0010) [2023-12-27 10:11:56,688][20381] Updated weights for policy 1, policy_version 659827 (0.0006) [2023-12-27 10:11:57,244][20347] Updated weights for policy 0, policy_version 660441 (0.0007) [2023-12-27 10:11:57,298][20347] Updated weights for policy 0, policy_version 660451 (0.0010) [2023-12-27 10:11:57,320][20381] Updated weights for policy 1, policy_version 659837 (0.0007) [2023-12-27 10:11:57,357][20347] Updated weights for policy 0, policy_version 660461 (0.0010) [2023-12-27 10:11:57,366][19170] Fps is (10 sec: 20472.9, 60 sec: 20615.4, 300 sec: 20771.3). Total num frames: 338034688. Throughput: 0: 10455.1, 1: 10264.9. Samples: 338028616. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:11:57,367][19170] Avg episode reward: [(0, '338.840'), (1, '815.410')] [2023-12-27 10:11:57,387][20381] Updated weights for policy 1, policy_version 659847 (0.0008) [2023-12-27 10:11:57,436][20381] Updated weights for policy 1, policy_version 659857 (0.0005) [2023-12-27 10:11:58,013][20347] Updated weights for policy 0, policy_version 660472 (0.0008) [2023-12-27 10:11:58,061][20347] Updated weights for policy 0, policy_version 660482 (0.0005) [2023-12-27 10:11:58,099][20381] Updated weights for policy 1, policy_version 659867 (0.0008) [2023-12-27 10:11:58,110][20347] Updated weights for policy 0, policy_version 660492 (0.0005) [2023-12-27 10:11:58,156][20381] Updated weights for policy 1, policy_version 659877 (0.0007) [2023-12-27 10:11:58,217][20381] Updated weights for policy 1, policy_version 659887 (0.0008) [2023-12-27 10:11:58,788][20347] Updated weights for policy 0, policy_version 660502 (0.0006) [2023-12-27 10:11:58,855][20347] Updated weights for policy 0, policy_version 660512 (0.0008) [2023-12-27 10:11:58,919][20347] Updated weights for policy 0, policy_version 660522 (0.0006) [2023-12-27 10:11:58,977][20381] Updated weights for policy 1, policy_version 659897 (0.0008) [2023-12-27 10:11:59,037][20381] Updated weights for policy 1, policy_version 659907 (0.0008) [2023-12-27 10:11:59,101][20381] Updated weights for policy 1, policy_version 659917 (0.0007) [2023-12-27 10:11:59,153][20381] Updated weights for policy 1, policy_version 659927 (0.0008) [2023-12-27 10:11:59,558][20347] Updated weights for policy 0, policy_version 660532 (0.0006) [2023-12-27 10:11:59,610][20347] Updated weights for policy 0, policy_version 660542 (0.0007) [2023-12-27 10:11:59,659][20347] Updated weights for policy 0, policy_version 660552 (0.0007) [2023-12-27 10:11:59,905][20381] Updated weights for policy 1, policy_version 659937 (0.0008) [2023-12-27 10:11:59,958][20381] Updated weights for policy 1, policy_version 659947 (0.0007) [2023-12-27 10:12:00,009][20381] Updated weights for policy 1, policy_version 659957 (0.0008) [2023-12-27 10:12:00,332][20347] Updated weights for policy 0, policy_version 660562 (0.0007) [2023-12-27 10:12:00,389][20347] Updated weights for policy 0, policy_version 660572 (0.0008) [2023-12-27 10:12:00,446][20347] Updated weights for policy 0, policy_version 660582 (0.0007) [2023-12-27 10:12:00,499][20347] Updated weights for policy 0, policy_version 660592 (0.0010) [2023-12-27 10:12:00,734][20381] Updated weights for policy 1, policy_version 659967 (0.0007) [2023-12-27 10:12:00,796][20381] Updated weights for policy 1, policy_version 659977 (0.0005) [2023-12-27 10:12:00,857][20381] Updated weights for policy 1, policy_version 659987 (0.0006) [2023-12-27 10:12:01,059][20347] Updated weights for policy 0, policy_version 660602 (0.0011) [2023-12-27 10:12:01,113][20347] Updated weights for policy 0, policy_version 660612 (0.0010) [2023-12-27 10:12:01,175][20347] Updated weights for policy 0, policy_version 660622 (0.0008) [2023-12-27 10:12:01,456][20381] Updated weights for policy 1, policy_version 659997 (0.0006) [2023-12-27 10:12:01,509][20381] Updated weights for policy 1, policy_version 660007 (0.0007) [2023-12-27 10:12:01,555][20381] Updated weights for policy 1, policy_version 660017 (0.0008) [2023-12-27 10:12:01,895][20347] Updated weights for policy 0, policy_version 660632 (0.0008) [2023-12-27 10:12:01,941][20347] Updated weights for policy 0, policy_version 660642 (0.0010) [2023-12-27 10:12:01,996][20347] Updated weights for policy 0, policy_version 660652 (0.0010) [2023-12-27 10:12:02,261][20381] Updated weights for policy 1, policy_version 660027 (0.0007) [2023-12-27 10:12:02,323][20381] Updated weights for policy 1, policy_version 660037 (0.0007) [2023-12-27 10:12:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20799.4). Total num frames: 338141184. Throughput: 0: 10448.7, 1: 10300.8. Samples: 338154332. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:02,363][19170] Avg episode reward: [(0, '326.910'), (1, '810.850')] [2023-12-27 10:12:02,385][20381] Updated weights for policy 1, policy_version 660047 (0.0007) [2023-12-27 10:12:02,674][20347] Updated weights for policy 0, policy_version 660662 (0.0008) [2023-12-27 10:12:02,722][20347] Updated weights for policy 0, policy_version 660672 (0.0007) [2023-12-27 10:12:02,777][20347] Updated weights for policy 0, policy_version 660682 (0.0008) [2023-12-27 10:12:03,074][20381] Updated weights for policy 1, policy_version 660057 (0.0008) [2023-12-27 10:12:03,120][20381] Updated weights for policy 1, policy_version 660067 (0.0008) [2023-12-27 10:12:03,165][20381] Updated weights for policy 1, policy_version 660077 (0.0008) [2023-12-27 10:12:03,211][20381] Updated weights for policy 1, policy_version 660087 (0.0007) [2023-12-27 10:12:03,465][20347] Updated weights for policy 0, policy_version 660692 (0.0008) [2023-12-27 10:12:03,512][20347] Updated weights for policy 0, policy_version 660702 (0.0008) [2023-12-27 10:12:03,559][20347] Updated weights for policy 0, policy_version 660712 (0.0008) [2023-12-27 10:12:03,863][20381] Updated weights for policy 1, policy_version 660097 (0.0008) [2023-12-27 10:12:03,914][20381] Updated weights for policy 1, policy_version 660107 (0.0007) [2023-12-27 10:12:03,964][20381] Updated weights for policy 1, policy_version 660117 (0.0007) [2023-12-27 10:12:04,313][20347] Updated weights for policy 0, policy_version 660722 (0.0008) [2023-12-27 10:12:04,374][20347] Updated weights for policy 0, policy_version 660732 (0.0007) [2023-12-27 10:12:04,423][20347] Updated weights for policy 0, policy_version 660742 (0.0007) [2023-12-27 10:12:04,477][20347] Updated weights for policy 0, policy_version 660752 (0.0008) [2023-12-27 10:12:04,554][20381] Updated weights for policy 1, policy_version 660127 (0.0008) [2023-12-27 10:12:04,603][20381] Updated weights for policy 1, policy_version 660137 (0.0008) [2023-12-27 10:12:04,652][20381] Updated weights for policy 1, policy_version 660147 (0.0008) [2023-12-27 10:12:05,206][20347] Updated weights for policy 0, policy_version 660762 (0.0007) [2023-12-27 10:12:05,254][20347] Updated weights for policy 0, policy_version 660772 (0.0007) [2023-12-27 10:12:05,263][20381] Updated weights for policy 1, policy_version 660157 (0.0008) [2023-12-27 10:12:05,303][20347] Updated weights for policy 0, policy_version 660782 (0.0006) [2023-12-27 10:12:05,317][20381] Updated weights for policy 1, policy_version 660167 (0.0005) [2023-12-27 10:12:05,368][20381] Updated weights for policy 1, policy_version 660177 (0.0005) [2023-12-27 10:12:05,989][20381] Updated weights for policy 1, policy_version 660187 (0.0006) [2023-12-27 10:12:06,002][20347] Updated weights for policy 0, policy_version 660792 (0.0009) [2023-12-27 10:12:06,054][20381] Updated weights for policy 1, policy_version 660197 (0.0006) [2023-12-27 10:12:06,060][20347] Updated weights for policy 0, policy_version 660802 (0.0006) [2023-12-27 10:12:06,110][20381] Updated weights for policy 1, policy_version 660207 (0.0006) [2023-12-27 10:12:06,127][20347] Updated weights for policy 0, policy_version 660812 (0.0008) [2023-12-27 10:12:06,771][20381] Updated weights for policy 1, policy_version 660217 (0.0008) [2023-12-27 10:12:06,793][20347] Updated weights for policy 0, policy_version 660822 (0.0009) [2023-12-27 10:12:06,822][20381] Updated weights for policy 1, policy_version 660227 (0.0006) [2023-12-27 10:12:06,846][20347] Updated weights for policy 0, policy_version 660832 (0.0010) [2023-12-27 10:12:06,874][20381] Updated weights for policy 1, policy_version 660237 (0.0006) [2023-12-27 10:12:06,899][20347] Updated weights for policy 0, policy_version 660842 (0.0011) [2023-12-27 10:12:06,928][20381] Updated weights for policy 1, policy_version 660247 (0.0006) [2023-12-27 10:12:07,362][19170] Fps is (10 sec: 21306.4, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 338247680. Throughput: 0: 10450.1, 1: 10343.6. Samples: 338217164. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:07,363][19170] Avg episode reward: [(0, '364.310'), (1, '826.720')] [2023-12-27 10:12:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000660248_169050112.pth... [2023-12-27 10:12:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000660848_169197568.pth... [2023-12-27 10:12:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000659032_168738816.pth [2023-12-27 10:12:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000659632_168886272.pth [2023-12-27 10:12:07,605][20347] Updated weights for policy 0, policy_version 660852 (0.0011) [2023-12-27 10:12:07,645][20381] Updated weights for policy 1, policy_version 660257 (0.0007) [2023-12-27 10:12:07,655][20347] Updated weights for policy 0, policy_version 660862 (0.0010) [2023-12-27 10:12:07,702][20347] Updated weights for policy 0, policy_version 660872 (0.0005) [2023-12-27 10:12:07,704][20381] Updated weights for policy 1, policy_version 660267 (0.0007) [2023-12-27 10:12:07,759][20381] Updated weights for policy 1, policy_version 660277 (0.0007) [2023-12-27 10:12:08,307][20347] Updated weights for policy 0, policy_version 660882 (0.0005) [2023-12-27 10:12:08,375][20347] Updated weights for policy 0, policy_version 660892 (0.0006) [2023-12-27 10:12:08,407][20381] Updated weights for policy 1, policy_version 660287 (0.0007) [2023-12-27 10:12:08,435][20347] Updated weights for policy 0, policy_version 660902 (0.0008) [2023-12-27 10:12:08,458][20381] Updated weights for policy 1, policy_version 660297 (0.0008) [2023-12-27 10:12:08,490][20347] Updated weights for policy 0, policy_version 660912 (0.0006) [2023-12-27 10:12:08,517][20381] Updated weights for policy 1, policy_version 660307 (0.0007) [2023-12-27 10:12:09,139][20347] Updated weights for policy 0, policy_version 660922 (0.0007) [2023-12-27 10:12:09,144][20381] Updated weights for policy 1, policy_version 660317 (0.0008) [2023-12-27 10:12:09,197][20347] Updated weights for policy 0, policy_version 660932 (0.0008) [2023-12-27 10:12:09,207][20381] Updated weights for policy 1, policy_version 660327 (0.0005) [2023-12-27 10:12:09,261][20347] Updated weights for policy 0, policy_version 660942 (0.0007) [2023-12-27 10:12:09,275][20381] Updated weights for policy 1, policy_version 660337 (0.0008) [2023-12-27 10:12:09,926][20381] Updated weights for policy 1, policy_version 660347 (0.0008) [2023-12-27 10:12:09,972][20347] Updated weights for policy 0, policy_version 660952 (0.0007) [2023-12-27 10:12:09,982][20381] Updated weights for policy 1, policy_version 660357 (0.0006) [2023-12-27 10:12:10,026][20347] Updated weights for policy 0, policy_version 660962 (0.0007) [2023-12-27 10:12:10,036][20381] Updated weights for policy 1, policy_version 660367 (0.0006) [2023-12-27 10:12:10,073][20347] Updated weights for policy 0, policy_version 660972 (0.0006) [2023-12-27 10:12:10,697][20381] Updated weights for policy 1, policy_version 660377 (0.0005) [2023-12-27 10:12:10,759][20381] Updated weights for policy 1, policy_version 660387 (0.0006) [2023-12-27 10:12:10,814][20381] Updated weights for policy 1, policy_version 660397 (0.0006) [2023-12-27 10:12:10,826][20347] Updated weights for policy 0, policy_version 660982 (0.0006) [2023-12-27 10:12:10,874][20381] Updated weights for policy 1, policy_version 660407 (0.0006) [2023-12-27 10:12:10,883][20347] Updated weights for policy 0, policy_version 660992 (0.0010) [2023-12-27 10:12:10,942][20347] Updated weights for policy 0, policy_version 661002 (0.0008) [2023-12-27 10:12:11,573][20381] Updated weights for policy 1, policy_version 660417 (0.0006) [2023-12-27 10:12:11,636][20381] Updated weights for policy 1, policy_version 660427 (0.0007) [2023-12-27 10:12:11,644][20347] Updated weights for policy 0, policy_version 661012 (0.0010) [2023-12-27 10:12:11,700][20381] Updated weights for policy 1, policy_version 660437 (0.0008) [2023-12-27 10:12:11,710][20347] Updated weights for policy 0, policy_version 661022 (0.0010) [2023-12-27 10:12:11,773][20347] Updated weights for policy 0, policy_version 661032 (0.0010) [2023-12-27 10:12:12,281][20381] Updated weights for policy 1, policy_version 660447 (0.0008) [2023-12-27 10:12:12,338][20381] Updated weights for policy 1, policy_version 660457 (0.0008) [2023-12-27 10:12:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 338345984. Throughput: 0: 10350.0, 1: 10414.5. Samples: 338340596. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:12,363][19170] Avg episode reward: [(0, '471.050'), (1, '791.600')] [2023-12-27 10:12:12,402][20381] Updated weights for policy 1, policy_version 660467 (0.0008) [2023-12-27 10:12:12,496][20347] Updated weights for policy 0, policy_version 661042 (0.0009) [2023-12-27 10:12:12,558][20347] Updated weights for policy 0, policy_version 661052 (0.0010) [2023-12-27 10:12:12,626][20347] Updated weights for policy 0, policy_version 661062 (0.0011) [2023-12-27 10:12:12,689][20347] Updated weights for policy 0, policy_version 661072 (0.0010) [2023-12-27 10:12:13,045][20381] Updated weights for policy 1, policy_version 660477 (0.0008) [2023-12-27 10:12:13,111][20381] Updated weights for policy 1, policy_version 660487 (0.0005) [2023-12-27 10:12:13,175][20381] Updated weights for policy 1, policy_version 660497 (0.0007) [2023-12-27 10:12:13,313][20347] Updated weights for policy 0, policy_version 661082 (0.0006) [2023-12-27 10:12:13,363][20347] Updated weights for policy 0, policy_version 661092 (0.0007) [2023-12-27 10:12:13,416][20347] Updated weights for policy 0, policy_version 661102 (0.0006) [2023-12-27 10:12:13,750][20381] Updated weights for policy 1, policy_version 660507 (0.0007) [2023-12-27 10:12:13,816][20381] Updated weights for policy 1, policy_version 660517 (0.0008) [2023-12-27 10:12:13,870][20381] Updated weights for policy 1, policy_version 660527 (0.0007) [2023-12-27 10:12:14,045][20347] Updated weights for policy 0, policy_version 661112 (0.0009) [2023-12-27 10:12:14,091][20347] Updated weights for policy 0, policy_version 661122 (0.0010) [2023-12-27 10:12:14,136][20347] Updated weights for policy 0, policy_version 661132 (0.0010) [2023-12-27 10:12:14,414][20381] Updated weights for policy 1, policy_version 660537 (0.0007) [2023-12-27 10:12:14,477][20381] Updated weights for policy 1, policy_version 660547 (0.0006) [2023-12-27 10:12:14,536][20381] Updated weights for policy 1, policy_version 660557 (0.0007) [2023-12-27 10:12:14,594][20381] Updated weights for policy 1, policy_version 660567 (0.0008) [2023-12-27 10:12:14,783][20347] Updated weights for policy 0, policy_version 661142 (0.0009) [2023-12-27 10:12:14,844][20347] Updated weights for policy 0, policy_version 661152 (0.0006) [2023-12-27 10:12:14,907][20347] Updated weights for policy 0, policy_version 661162 (0.0010) [2023-12-27 10:12:15,230][20381] Updated weights for policy 1, policy_version 660577 (0.0008) [2023-12-27 10:12:15,294][20381] Updated weights for policy 1, policy_version 660587 (0.0007) [2023-12-27 10:12:15,352][20381] Updated weights for policy 1, policy_version 660597 (0.0007) [2023-12-27 10:12:15,588][20347] Updated weights for policy 0, policy_version 661172 (0.0011) [2023-12-27 10:12:15,640][20347] Updated weights for policy 0, policy_version 661182 (0.0010) [2023-12-27 10:12:15,686][20347] Updated weights for policy 0, policy_version 661192 (0.0010) [2023-12-27 10:12:15,897][20381] Updated weights for policy 1, policy_version 660607 (0.0006) [2023-12-27 10:12:15,947][20381] Updated weights for policy 1, policy_version 660617 (0.0007) [2023-12-27 10:12:16,003][20381] Updated weights for policy 1, policy_version 660627 (0.0006) [2023-12-27 10:12:16,379][20347] Updated weights for policy 0, policy_version 661202 (0.0010) [2023-12-27 10:12:16,427][20347] Updated weights for policy 0, policy_version 661212 (0.0010) [2023-12-27 10:12:16,479][20347] Updated weights for policy 0, policy_version 661222 (0.0010) [2023-12-27 10:12:16,530][20347] Updated weights for policy 0, policy_version 661232 (0.0010) [2023-12-27 10:12:16,625][20381] Updated weights for policy 1, policy_version 660637 (0.0006) [2023-12-27 10:12:16,678][20381] Updated weights for policy 1, policy_version 660647 (0.0005) [2023-12-27 10:12:16,731][20381] Updated weights for policy 1, policy_version 660657 (0.0005) [2023-12-27 10:12:17,233][20381] Updated weights for policy 1, policy_version 660667 (0.0005) [2023-12-27 10:12:17,234][20347] Updated weights for policy 0, policy_version 661242 (0.0005) [2023-12-27 10:12:17,291][20381] Updated weights for policy 1, policy_version 660677 (0.0005) [2023-12-27 10:12:17,292][20347] Updated weights for policy 0, policy_version 661252 (0.0005) [2023-12-27 10:12:17,343][20347] Updated weights for policy 0, policy_version 661262 (0.0007) [2023-12-27 10:12:17,345][20381] Updated weights for policy 1, policy_version 660687 (0.0005) [2023-12-27 10:12:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 338460672. Throughput: 0: 10317.0, 1: 10572.0. Samples: 338470728. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:17,363][19170] Avg episode reward: [(0, '476.350'), (1, '933.630')] [2023-12-27 10:12:17,941][20381] Updated weights for policy 1, policy_version 660697 (0.0007) [2023-12-27 10:12:17,986][20381] Updated weights for policy 1, policy_version 660707 (0.0005) [2023-12-27 10:12:18,035][20381] Updated weights for policy 1, policy_version 660717 (0.0010) [2023-12-27 10:12:18,052][20347] Updated weights for policy 0, policy_version 661272 (0.0008) [2023-12-27 10:12:18,083][20381] Updated weights for policy 1, policy_version 660727 (0.0010) [2023-12-27 10:12:18,111][20347] Updated weights for policy 0, policy_version 661282 (0.0007) [2023-12-27 10:12:18,175][20347] Updated weights for policy 0, policy_version 661292 (0.0006) [2023-12-27 10:12:18,761][20347] Updated weights for policy 0, policy_version 661302 (0.0006) [2023-12-27 10:12:18,777][20381] Updated weights for policy 1, policy_version 660737 (0.0007) [2023-12-27 10:12:18,819][20347] Updated weights for policy 0, policy_version 661312 (0.0008) [2023-12-27 10:12:18,834][20381] Updated weights for policy 1, policy_version 660747 (0.0010) [2023-12-27 10:12:18,877][20347] Updated weights for policy 0, policy_version 661322 (0.0007) [2023-12-27 10:12:18,893][20381] Updated weights for policy 1, policy_version 660757 (0.0010) [2023-12-27 10:12:19,561][20347] Updated weights for policy 0, policy_version 661332 (0.0006) [2023-12-27 10:12:19,591][20381] Updated weights for policy 1, policy_version 660767 (0.0011) [2023-12-27 10:12:19,615][20347] Updated weights for policy 0, policy_version 661342 (0.0006) [2023-12-27 10:12:19,657][20381] Updated weights for policy 1, policy_version 660777 (0.0008) [2023-12-27 10:12:19,677][20347] Updated weights for policy 0, policy_version 661352 (0.0008) [2023-12-27 10:12:19,716][20381] Updated weights for policy 1, policy_version 660787 (0.0006) [2023-12-27 10:12:20,349][20347] Updated weights for policy 0, policy_version 661362 (0.0007) [2023-12-27 10:12:20,371][20381] Updated weights for policy 1, policy_version 660797 (0.0008) [2023-12-27 10:12:20,410][20347] Updated weights for policy 0, policy_version 661372 (0.0007) [2023-12-27 10:12:20,427][20381] Updated weights for policy 1, policy_version 660807 (0.0011) [2023-12-27 10:12:20,470][20347] Updated weights for policy 0, policy_version 661382 (0.0006) [2023-12-27 10:12:20,478][20381] Updated weights for policy 1, policy_version 660817 (0.0011) [2023-12-27 10:12:20,527][20347] Updated weights for policy 0, policy_version 661392 (0.0009) [2023-12-27 10:12:21,100][20381] Updated weights for policy 1, policy_version 660827 (0.0010) [2023-12-27 10:12:21,160][20381] Updated weights for policy 1, policy_version 660837 (0.0007) [2023-12-27 10:12:21,223][20381] Updated weights for policy 1, policy_version 660847 (0.0008) [2023-12-27 10:12:21,244][20347] Updated weights for policy 0, policy_version 661402 (0.0007) [2023-12-27 10:12:21,307][20347] Updated weights for policy 0, policy_version 661412 (0.0008) [2023-12-27 10:12:21,373][20347] Updated weights for policy 0, policy_version 661422 (0.0008) [2023-12-27 10:12:21,944][20381] Updated weights for policy 1, policy_version 660857 (0.0007) [2023-12-27 10:12:22,014][20381] Updated weights for policy 1, policy_version 660867 (0.0008) [2023-12-27 10:12:22,077][20381] Updated weights for policy 1, policy_version 660877 (0.0008) [2023-12-27 10:12:22,103][20347] Updated weights for policy 0, policy_version 661432 (0.0010) [2023-12-27 10:12:22,137][20381] Updated weights for policy 1, policy_version 660887 (0.0007) [2023-12-27 10:12:22,161][20347] Updated weights for policy 0, policy_version 661442 (0.0010) [2023-12-27 10:12:22,221][20347] Updated weights for policy 0, policy_version 661452 (0.0005) [2023-12-27 10:12:22,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 338567168. Throughput: 0: 10278.9, 1: 10589.2. Samples: 338532688. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:22,363][19170] Avg episode reward: [(0, '609.290'), (1, '942.200')] [2023-12-27 10:12:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000660888_169213952.pth... [2023-12-27 10:12:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000661456_169353216.pth... [2023-12-27 10:12:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000659640_168894464.pth [2023-12-27 10:12:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000660240_169041920.pth [2023-12-27 10:12:22,807][20381] Updated weights for policy 1, policy_version 660897 (0.0007) [2023-12-27 10:12:22,865][20381] Updated weights for policy 1, policy_version 660907 (0.0008) [2023-12-27 10:12:22,885][20347] Updated weights for policy 0, policy_version 661462 (0.0006) [2023-12-27 10:12:22,923][20381] Updated weights for policy 1, policy_version 660917 (0.0007) [2023-12-27 10:12:22,940][20347] Updated weights for policy 0, policy_version 661472 (0.0008) [2023-12-27 10:12:22,986][20347] Updated weights for policy 0, policy_version 661482 (0.0010) [2023-12-27 10:12:23,610][20381] Updated weights for policy 1, policy_version 660927 (0.0005) [2023-12-27 10:12:23,666][20381] Updated weights for policy 1, policy_version 660937 (0.0005) [2023-12-27 10:12:23,668][20347] Updated weights for policy 0, policy_version 661492 (0.0008) [2023-12-27 10:12:23,720][20381] Updated weights for policy 1, policy_version 660947 (0.0005) [2023-12-27 10:12:23,727][20347] Updated weights for policy 0, policy_version 661502 (0.0009) [2023-12-27 10:12:23,786][20347] Updated weights for policy 0, policy_version 661512 (0.0005) [2023-12-27 10:12:24,338][20381] Updated weights for policy 1, policy_version 660957 (0.0007) [2023-12-27 10:12:24,394][20381] Updated weights for policy 1, policy_version 660967 (0.0008) [2023-12-27 10:12:24,422][20347] Updated weights for policy 0, policy_version 661522 (0.0010) [2023-12-27 10:12:24,451][20381] Updated weights for policy 1, policy_version 660977 (0.0006) [2023-12-27 10:12:24,478][20347] Updated weights for policy 0, policy_version 661532 (0.0010) [2023-12-27 10:12:24,535][20347] Updated weights for policy 0, policy_version 661542 (0.0010) [2023-12-27 10:12:24,590][20347] Updated weights for policy 0, policy_version 661552 (0.0011) [2023-12-27 10:12:25,100][20381] Updated weights for policy 1, policy_version 660987 (0.0006) [2023-12-27 10:12:25,150][20381] Updated weights for policy 1, policy_version 660997 (0.0006) [2023-12-27 10:12:25,197][20347] Updated weights for policy 0, policy_version 661562 (0.0005) [2023-12-27 10:12:25,200][20381] Updated weights for policy 1, policy_version 661007 (0.0005) [2023-12-27 10:12:25,258][20347] Updated weights for policy 0, policy_version 661572 (0.0005) [2023-12-27 10:12:25,311][20347] Updated weights for policy 0, policy_version 661582 (0.0005) [2023-12-27 10:12:25,815][20381] Updated weights for policy 1, policy_version 661017 (0.0005) [2023-12-27 10:12:25,840][20347] Updated weights for policy 0, policy_version 661592 (0.0008) [2023-12-27 10:12:25,874][20381] Updated weights for policy 1, policy_version 661027 (0.0005) [2023-12-27 10:12:25,897][20347] Updated weights for policy 0, policy_version 661602 (0.0006) [2023-12-27 10:12:25,935][20381] Updated weights for policy 1, policy_version 661037 (0.0005) [2023-12-27 10:12:25,945][20347] Updated weights for policy 0, policy_version 661612 (0.0007) [2023-12-27 10:12:25,983][20381] Updated weights for policy 1, policy_version 661047 (0.0005) [2023-12-27 10:12:26,550][20347] Updated weights for policy 0, policy_version 661622 (0.0007) [2023-12-27 10:12:26,603][20347] Updated weights for policy 0, policy_version 661632 (0.0007) [2023-12-27 10:12:26,645][20381] Updated weights for policy 1, policy_version 661057 (0.0009) [2023-12-27 10:12:26,655][20347] Updated weights for policy 0, policy_version 661642 (0.0007) [2023-12-27 10:12:26,709][20381] Updated weights for policy 1, policy_version 661067 (0.0007) [2023-12-27 10:12:26,774][20381] Updated weights for policy 1, policy_version 661077 (0.0008) [2023-12-27 10:12:27,290][20347] Updated weights for policy 0, policy_version 661652 (0.0007) [2023-12-27 10:12:27,339][20347] Updated weights for policy 0, policy_version 661662 (0.0006) [2023-12-27 10:12:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 338665472. Throughput: 0: 10308.6, 1: 10616.7. Samples: 338659016. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:27,363][19170] Avg episode reward: [(0, '686.600'), (1, '932.700')] [2023-12-27 10:12:27,401][20347] Updated weights for policy 0, policy_version 661672 (0.0005) [2023-12-27 10:12:27,456][20381] Updated weights for policy 1, policy_version 661087 (0.0007) [2023-12-27 10:12:27,510][20381] Updated weights for policy 1, policy_version 661097 (0.0007) [2023-12-27 10:12:27,573][20381] Updated weights for policy 1, policy_version 661107 (0.0009) [2023-12-27 10:12:28,004][20347] Updated weights for policy 0, policy_version 661682 (0.0007) [2023-12-27 10:12:28,049][20347] Updated weights for policy 0, policy_version 661692 (0.0010) [2023-12-27 10:12:28,102][20347] Updated weights for policy 0, policy_version 661702 (0.0010) [2023-12-27 10:12:28,153][20347] Updated weights for policy 0, policy_version 661712 (0.0010) [2023-12-27 10:12:28,236][20381] Updated weights for policy 1, policy_version 661117 (0.0008) [2023-12-27 10:12:28,282][20381] Updated weights for policy 1, policy_version 661127 (0.0007) [2023-12-27 10:12:28,328][20381] Updated weights for policy 1, policy_version 661137 (0.0007) [2023-12-27 10:12:28,847][20347] Updated weights for policy 0, policy_version 661722 (0.0010) [2023-12-27 10:12:28,904][20347] Updated weights for policy 0, policy_version 661732 (0.0010) [2023-12-27 10:12:28,956][20347] Updated weights for policy 0, policy_version 661742 (0.0010) [2023-12-27 10:12:29,035][20381] Updated weights for policy 1, policy_version 661147 (0.0008) [2023-12-27 10:12:29,089][20381] Updated weights for policy 1, policy_version 661157 (0.0007) [2023-12-27 10:12:29,136][20381] Updated weights for policy 1, policy_version 661167 (0.0007) [2023-12-27 10:12:29,614][20347] Updated weights for policy 0, policy_version 661752 (0.0008) [2023-12-27 10:12:29,663][20347] Updated weights for policy 0, policy_version 661762 (0.0008) [2023-12-27 10:12:29,711][20347] Updated weights for policy 0, policy_version 661772 (0.0007) [2023-12-27 10:12:29,848][20381] Updated weights for policy 1, policy_version 661177 (0.0008) [2023-12-27 10:12:29,899][20381] Updated weights for policy 1, policy_version 661187 (0.0008) [2023-12-27 10:12:29,960][20381] Updated weights for policy 1, policy_version 661197 (0.0008) [2023-12-27 10:12:30,019][20381] Updated weights for policy 1, policy_version 661207 (0.0008) [2023-12-27 10:12:30,377][20347] Updated weights for policy 0, policy_version 661782 (0.0008) [2023-12-27 10:12:30,426][20347] Updated weights for policy 0, policy_version 661792 (0.0010) [2023-12-27 10:12:30,470][20347] Updated weights for policy 0, policy_version 661802 (0.0008) [2023-12-27 10:12:30,724][20381] Updated weights for policy 1, policy_version 661217 (0.0006) [2023-12-27 10:12:30,775][20381] Updated weights for policy 1, policy_version 661227 (0.0006) [2023-12-27 10:12:30,823][20381] Updated weights for policy 1, policy_version 661237 (0.0008) [2023-12-27 10:12:31,024][20347] Updated weights for policy 0, policy_version 661812 (0.0006) [2023-12-27 10:12:31,087][20347] Updated weights for policy 0, policy_version 661822 (0.0008) [2023-12-27 10:12:31,140][20347] Updated weights for policy 0, policy_version 661832 (0.0007) [2023-12-27 10:12:31,490][20381] Updated weights for policy 1, policy_version 661247 (0.0008) [2023-12-27 10:12:31,540][20381] Updated weights for policy 1, policy_version 661257 (0.0007) [2023-12-27 10:12:31,590][20381] Updated weights for policy 1, policy_version 661267 (0.0008) [2023-12-27 10:12:31,828][20347] Updated weights for policy 0, policy_version 661842 (0.0007) [2023-12-27 10:12:31,882][20347] Updated weights for policy 0, policy_version 661852 (0.0005) [2023-12-27 10:12:31,929][20347] Updated weights for policy 0, policy_version 661862 (0.0006) [2023-12-27 10:12:31,975][20347] Updated weights for policy 0, policy_version 661872 (0.0010) [2023-12-27 10:12:32,275][20381] Updated weights for policy 1, policy_version 661277 (0.0008) [2023-12-27 10:12:32,334][20381] Updated weights for policy 1, policy_version 661287 (0.0007) [2023-12-27 10:12:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 338771968. Throughput: 0: 10369.4, 1: 10545.4. Samples: 338785140. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:32,363][19170] Avg episode reward: [(0, '735.450'), (1, '953.020')] [2023-12-27 10:12:32,393][20381] Updated weights for policy 1, policy_version 661297 (0.0008) [2023-12-27 10:12:32,664][20347] Updated weights for policy 0, policy_version 661882 (0.0007) [2023-12-27 10:12:32,720][20347] Updated weights for policy 0, policy_version 661892 (0.0007) [2023-12-27 10:12:32,777][20347] Updated weights for policy 0, policy_version 661902 (0.0006) [2023-12-27 10:12:33,103][20381] Updated weights for policy 1, policy_version 661307 (0.0007) [2023-12-27 10:12:33,157][20381] Updated weights for policy 1, policy_version 661317 (0.0006) [2023-12-27 10:12:33,212][20381] Updated weights for policy 1, policy_version 661327 (0.0007) [2023-12-27 10:12:33,359][20347] Updated weights for policy 0, policy_version 661912 (0.0005) [2023-12-27 10:12:33,412][20347] Updated weights for policy 0, policy_version 661922 (0.0005) [2023-12-27 10:12:33,460][20347] Updated weights for policy 0, policy_version 661932 (0.0005) [2023-12-27 10:12:33,834][20381] Updated weights for policy 1, policy_version 661337 (0.0006) [2023-12-27 10:12:33,893][20381] Updated weights for policy 1, policy_version 661347 (0.0007) [2023-12-27 10:12:33,951][20381] Updated weights for policy 1, policy_version 661357 (0.0008) [2023-12-27 10:12:34,002][20347] Updated weights for policy 0, policy_version 661942 (0.0006) [2023-12-27 10:12:34,005][20381] Updated weights for policy 1, policy_version 661367 (0.0008) [2023-12-27 10:12:34,061][20347] Updated weights for policy 0, policy_version 661952 (0.0008) [2023-12-27 10:12:34,122][20347] Updated weights for policy 0, policy_version 661962 (0.0007) [2023-12-27 10:12:34,716][20381] Updated weights for policy 1, policy_version 661377 (0.0008) [2023-12-27 10:12:34,766][20381] Updated weights for policy 1, policy_version 661387 (0.0007) [2023-12-27 10:12:34,769][20347] Updated weights for policy 0, policy_version 661972 (0.0008) [2023-12-27 10:12:34,819][20347] Updated weights for policy 0, policy_version 661982 (0.0006) [2023-12-27 10:12:34,825][20381] Updated weights for policy 1, policy_version 661397 (0.0008) [2023-12-27 10:12:34,875][20347] Updated weights for policy 0, policy_version 661992 (0.0007) [2023-12-27 10:12:35,417][20381] Updated weights for policy 1, policy_version 661407 (0.0009) [2023-12-27 10:12:35,481][20381] Updated weights for policy 1, policy_version 661417 (0.0010) [2023-12-27 10:12:35,535][20381] Updated weights for policy 1, policy_version 661427 (0.0010) [2023-12-27 10:12:35,536][20347] Updated weights for policy 0, policy_version 662002 (0.0008) [2023-12-27 10:12:35,592][20347] Updated weights for policy 0, policy_version 662012 (0.0008) [2023-12-27 10:12:35,649][20347] Updated weights for policy 0, policy_version 662022 (0.0005) [2023-12-27 10:12:35,704][20347] Updated weights for policy 0, policy_version 662032 (0.0009) [2023-12-27 10:12:36,194][20381] Updated weights for policy 1, policy_version 661437 (0.0011) [2023-12-27 10:12:36,247][20381] Updated weights for policy 1, policy_version 661447 (0.0009) [2023-12-27 10:12:36,296][20381] Updated weights for policy 1, policy_version 661457 (0.0009) [2023-12-27 10:12:36,437][20347] Updated weights for policy 0, policy_version 662042 (0.0007) [2023-12-27 10:12:36,499][20347] Updated weights for policy 0, policy_version 662052 (0.0007) [2023-12-27 10:12:36,560][20347] Updated weights for policy 0, policy_version 662062 (0.0008) [2023-12-27 10:12:37,013][20381] Updated weights for policy 1, policy_version 661467 (0.0008) [2023-12-27 10:12:37,069][20381] Updated weights for policy 1, policy_version 661477 (0.0010) [2023-12-27 10:12:37,127][20381] Updated weights for policy 1, policy_version 661487 (0.0010) [2023-12-27 10:12:37,141][20347] Updated weights for policy 0, policy_version 662072 (0.0008) [2023-12-27 10:12:37,203][20347] Updated weights for policy 0, policy_version 662082 (0.0007) [2023-12-27 10:12:37,268][20347] Updated weights for policy 0, policy_version 662092 (0.0006) [2023-12-27 10:12:37,362][19170] Fps is (10 sec: 22118.4, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 338886656. Throughput: 0: 10451.5, 1: 10519.3. Samples: 338849044. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:37,363][19170] Avg episode reward: [(0, '707.360'), (1, '981.900')] [2023-12-27 10:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000661496_169369600.pth... [2023-12-27 10:12:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000662096_169517056.pth... [2023-12-27 10:12:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000660248_169050112.pth [2023-12-27 10:12:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000660848_169197568.pth [2023-12-27 10:12:37,807][20347] Updated weights for policy 0, policy_version 662102 (0.0005) [2023-12-27 10:12:37,808][20381] Updated weights for policy 1, policy_version 661497 (0.0011) [2023-12-27 10:12:37,860][20381] Updated weights for policy 1, policy_version 661507 (0.0010) [2023-12-27 10:12:37,864][20347] Updated weights for policy 0, policy_version 662112 (0.0006) [2023-12-27 10:12:37,916][20347] Updated weights for policy 0, policy_version 662122 (0.0005) [2023-12-27 10:12:37,926][20381] Updated weights for policy 1, policy_version 661517 (0.0011) [2023-12-27 10:12:37,978][20381] Updated weights for policy 1, policy_version 661527 (0.0010) [2023-12-27 10:12:38,551][20347] Updated weights for policy 0, policy_version 662132 (0.0008) [2023-12-27 10:12:38,610][20347] Updated weights for policy 0, policy_version 662142 (0.0011) [2023-12-27 10:12:38,626][20381] Updated weights for policy 1, policy_version 661537 (0.0010) [2023-12-27 10:12:38,671][20347] Updated weights for policy 0, policy_version 662152 (0.0010) [2023-12-27 10:12:38,675][20381] Updated weights for policy 1, policy_version 661547 (0.0010) [2023-12-27 10:12:38,727][20381] Updated weights for policy 1, policy_version 661557 (0.0011) [2023-12-27 10:12:39,342][20347] Updated weights for policy 0, policy_version 662162 (0.0011) [2023-12-27 10:12:39,410][20347] Updated weights for policy 0, policy_version 662172 (0.0011) [2023-12-27 10:12:39,458][20381] Updated weights for policy 1, policy_version 661567 (0.0011) [2023-12-27 10:12:39,469][20347] Updated weights for policy 0, policy_version 662182 (0.0009) [2023-12-27 10:12:39,515][20381] Updated weights for policy 1, policy_version 661577 (0.0011) [2023-12-27 10:12:39,528][20347] Updated weights for policy 0, policy_version 662192 (0.0008) [2023-12-27 10:12:39,575][20381] Updated weights for policy 1, policy_version 661587 (0.0011) [2023-12-27 10:12:40,274][20347] Updated weights for policy 0, policy_version 662202 (0.0008) [2023-12-27 10:12:40,286][20381] Updated weights for policy 1, policy_version 661597 (0.0011) [2023-12-27 10:12:40,332][20347] Updated weights for policy 0, policy_version 662212 (0.0009) [2023-12-27 10:12:40,346][20381] Updated weights for policy 1, policy_version 661607 (0.0009) [2023-12-27 10:12:40,389][20347] Updated weights for policy 0, policy_version 662222 (0.0008) [2023-12-27 10:12:40,414][20381] Updated weights for policy 1, policy_version 661617 (0.0008) [2023-12-27 10:12:41,045][20381] Updated weights for policy 1, policy_version 661627 (0.0010) [2023-12-27 10:12:41,073][20347] Updated weights for policy 0, policy_version 662232 (0.0007) [2023-12-27 10:12:41,108][20381] Updated weights for policy 1, policy_version 661637 (0.0008) [2023-12-27 10:12:41,136][20347] Updated weights for policy 0, policy_version 662242 (0.0008) [2023-12-27 10:12:41,170][20381] Updated weights for policy 1, policy_version 661647 (0.0008) [2023-12-27 10:12:41,192][20347] Updated weights for policy 0, policy_version 662252 (0.0006) [2023-12-27 10:12:41,839][20381] Updated weights for policy 1, policy_version 661657 (0.0008) [2023-12-27 10:12:41,895][20381] Updated weights for policy 1, policy_version 661667 (0.0011) [2023-12-27 10:12:41,936][20347] Updated weights for policy 0, policy_version 662262 (0.0007) [2023-12-27 10:12:41,946][20381] Updated weights for policy 1, policy_version 661677 (0.0005) [2023-12-27 10:12:41,997][20347] Updated weights for policy 0, policy_version 662272 (0.0006) [2023-12-27 10:12:41,999][20381] Updated weights for policy 1, policy_version 661687 (0.0011) [2023-12-27 10:12:42,016][20291] KL-divergence is very high: 117.8699 [2023-12-27 10:12:42,022][20291] KL-divergence is very high: 157.0343 [2023-12-27 10:12:42,044][20291] KL-divergence is very high: 199.3753 [2023-12-27 10:12:42,050][20291] KL-divergence is very high: 286.6932 [2023-12-27 10:12:42,057][20347] Updated weights for policy 0, policy_version 662282 (0.0007) [2023-12-27 10:12:42,062][20291] KL-divergence is very high: 228.5003 [2023-12-27 10:12:42,068][20291] KL-divergence is very high: 157.3278 [2023-12-27 10:12:42,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 338984960. Throughput: 0: 10505.9, 1: 10499.5. Samples: 338973784. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:42,363][19170] Avg episode reward: [(0, '609.630'), (1, '1000.000')] [2023-12-27 10:12:42,718][20381] Updated weights for policy 1, policy_version 661697 (0.0006) [2023-12-27 10:12:42,752][20291] KL-divergence is very high: 115.3765 [2023-12-27 10:12:42,762][20347] Updated weights for policy 0, policy_version 662292 (0.0007) [2023-12-27 10:12:42,778][20381] Updated weights for policy 1, policy_version 661707 (0.0005) [2023-12-27 10:12:42,794][20291] KL-divergence is very high: 145.5357 [2023-12-27 10:12:42,816][20347] Updated weights for policy 0, policy_version 662302 (0.0006) [2023-12-27 10:12:42,829][20381] Updated weights for policy 1, policy_version 661717 (0.0005) [2023-12-27 10:12:42,878][20347] Updated weights for policy 0, policy_version 662312 (0.0007) [2023-12-27 10:12:43,414][20381] Updated weights for policy 1, policy_version 661727 (0.0009) [2023-12-27 10:12:43,459][20381] Updated weights for policy 1, policy_version 661737 (0.0010) [2023-12-27 10:12:43,510][20381] Updated weights for policy 1, policy_version 661747 (0.0010) [2023-12-27 10:12:43,585][20347] Updated weights for policy 0, policy_version 662322 (0.0007) [2023-12-27 10:12:43,639][20347] Updated weights for policy 0, policy_version 662332 (0.0007) [2023-12-27 10:12:43,690][20347] Updated weights for policy 0, policy_version 662342 (0.0007) [2023-12-27 10:12:43,740][20347] Updated weights for policy 0, policy_version 662352 (0.0007) [2023-12-27 10:12:44,163][20381] Updated weights for policy 1, policy_version 661757 (0.0010) [2023-12-27 10:12:44,217][20381] Updated weights for policy 1, policy_version 661767 (0.0008) [2023-12-27 10:12:44,268][20381] Updated weights for policy 1, policy_version 661777 (0.0007) [2023-12-27 10:12:44,428][20347] Updated weights for policy 0, policy_version 662362 (0.0008) [2023-12-27 10:12:44,505][20347] Updated weights for policy 0, policy_version 662372 (0.0008) [2023-12-27 10:12:44,553][20347] Updated weights for policy 0, policy_version 662382 (0.0007) [2023-12-27 10:12:44,955][20381] Updated weights for policy 1, policy_version 661787 (0.0005) [2023-12-27 10:12:45,024][20381] Updated weights for policy 1, policy_version 661797 (0.0006) [2023-12-27 10:12:45,088][20381] Updated weights for policy 1, policy_version 661807 (0.0006) [2023-12-27 10:12:45,243][20347] Updated weights for policy 0, policy_version 662392 (0.0010) [2023-12-27 10:12:45,300][20347] Updated weights for policy 0, policy_version 662402 (0.0010) [2023-12-27 10:12:45,358][20347] Updated weights for policy 0, policy_version 662412 (0.0006) [2023-12-27 10:12:45,646][20381] Updated weights for policy 1, policy_version 661817 (0.0006) [2023-12-27 10:12:45,711][20381] Updated weights for policy 1, policy_version 661827 (0.0005) [2023-12-27 10:12:45,771][20381] Updated weights for policy 1, policy_version 661837 (0.0008) [2023-12-27 10:12:45,819][20381] Updated weights for policy 1, policy_version 661847 (0.0007) [2023-12-27 10:12:45,979][20347] Updated weights for policy 0, policy_version 662422 (0.0007) [2023-12-27 10:12:46,043][20347] Updated weights for policy 0, policy_version 662432 (0.0009) [2023-12-27 10:12:46,090][20347] Updated weights for policy 0, policy_version 662442 (0.0008) [2023-12-27 10:12:46,450][20381] Updated weights for policy 1, policy_version 661857 (0.0008) [2023-12-27 10:12:46,499][20381] Updated weights for policy 1, policy_version 661867 (0.0008) [2023-12-27 10:12:46,550][20381] Updated weights for policy 1, policy_version 661877 (0.0007) [2023-12-27 10:12:46,734][20347] Updated weights for policy 0, policy_version 662452 (0.0005) [2023-12-27 10:12:46,781][20347] Updated weights for policy 0, policy_version 662462 (0.0007) [2023-12-27 10:12:46,831][20347] Updated weights for policy 0, policy_version 662472 (0.0008) [2023-12-27 10:12:47,245][20381] Updated weights for policy 1, policy_version 661887 (0.0008) [2023-12-27 10:12:47,308][20381] Updated weights for policy 1, policy_version 661897 (0.0008) [2023-12-27 10:12:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 339083264. Throughput: 0: 10420.2, 1: 10564.9. Samples: 339098660. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:47,363][19170] Avg episode reward: [(0, '726.820'), (1, '990.480')] [2023-12-27 10:12:47,371][20381] Updated weights for policy 1, policy_version 661907 (0.0008) [2023-12-27 10:12:47,472][20347] Updated weights for policy 0, policy_version 662482 (0.0006) [2023-12-27 10:12:47,527][20347] Updated weights for policy 0, policy_version 662492 (0.0005) [2023-12-27 10:12:47,586][20347] Updated weights for policy 0, policy_version 662502 (0.0005) [2023-12-27 10:12:47,646][20347] Updated weights for policy 0, policy_version 662512 (0.0005) [2023-12-27 10:12:47,942][20381] Updated weights for policy 1, policy_version 661917 (0.0009) [2023-12-27 10:12:48,003][20381] Updated weights for policy 1, policy_version 661927 (0.0010) [2023-12-27 10:12:48,067][20381] Updated weights for policy 1, policy_version 661937 (0.0009) [2023-12-27 10:12:48,207][20347] Updated weights for policy 0, policy_version 662522 (0.0010) [2023-12-27 10:12:48,259][20347] Updated weights for policy 0, policy_version 662532 (0.0010) [2023-12-27 10:12:48,304][20347] Updated weights for policy 0, policy_version 662542 (0.0010) [2023-12-27 10:12:48,673][20381] Updated weights for policy 1, policy_version 661947 (0.0011) [2023-12-27 10:12:48,739][20381] Updated weights for policy 1, policy_version 661957 (0.0011) [2023-12-27 10:12:48,798][20381] Updated weights for policy 1, policy_version 661967 (0.0010) [2023-12-27 10:12:48,939][20347] Updated weights for policy 0, policy_version 662552 (0.0010) [2023-12-27 10:12:49,003][20347] Updated weights for policy 0, policy_version 662562 (0.0010) [2023-12-27 10:12:49,058][20347] Updated weights for policy 0, policy_version 662572 (0.0010) [2023-12-27 10:12:49,498][20381] Updated weights for policy 1, policy_version 661977 (0.0011) [2023-12-27 10:12:49,555][20381] Updated weights for policy 1, policy_version 661987 (0.0007) [2023-12-27 10:12:49,620][20381] Updated weights for policy 1, policy_version 661997 (0.0011) [2023-12-27 10:12:49,678][20381] Updated weights for policy 1, policy_version 662007 (0.0011) [2023-12-27 10:12:49,781][20347] Updated weights for policy 0, policy_version 662582 (0.0010) [2023-12-27 10:12:49,845][20347] Updated weights for policy 0, policy_version 662592 (0.0011) [2023-12-27 10:12:49,905][20347] Updated weights for policy 0, policy_version 662602 (0.0010) [2023-12-27 10:12:50,376][20381] Updated weights for policy 1, policy_version 662017 (0.0010) [2023-12-27 10:12:50,432][20381] Updated weights for policy 1, policy_version 662027 (0.0011) [2023-12-27 10:12:50,488][20381] Updated weights for policy 1, policy_version 662037 (0.0011) [2023-12-27 10:12:50,548][20347] Updated weights for policy 0, policy_version 662612 (0.0010) [2023-12-27 10:12:50,606][20347] Updated weights for policy 0, policy_version 662622 (0.0010) [2023-12-27 10:12:50,662][20347] Updated weights for policy 0, policy_version 662632 (0.0010) [2023-12-27 10:12:51,185][20381] Updated weights for policy 1, policy_version 662047 (0.0009) [2023-12-27 10:12:51,244][20381] Updated weights for policy 1, policy_version 662057 (0.0011) [2023-12-27 10:12:51,299][20381] Updated weights for policy 1, policy_version 662067 (0.0011) [2023-12-27 10:12:51,336][20347] Updated weights for policy 0, policy_version 662642 (0.0010) [2023-12-27 10:12:51,398][20347] Updated weights for policy 0, policy_version 662652 (0.0010) [2023-12-27 10:12:51,450][20347] Updated weights for policy 0, policy_version 662662 (0.0008) [2023-12-27 10:12:51,513][20347] Updated weights for policy 0, policy_version 662672 (0.0009) [2023-12-27 10:12:52,021][20381] Updated weights for policy 1, policy_version 662077 (0.0009) [2023-12-27 10:12:52,079][20381] Updated weights for policy 1, policy_version 662087 (0.0008) [2023-12-27 10:12:52,145][20381] Updated weights for policy 1, policy_version 662097 (0.0007) [2023-12-27 10:12:52,215][20347] Updated weights for policy 0, policy_version 662682 (0.0005) [2023-12-27 10:12:52,275][20347] Updated weights for policy 0, policy_version 662692 (0.0007) [2023-12-27 10:12:52,327][20347] Updated weights for policy 0, policy_version 662702 (0.0006) [2023-12-27 10:12:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 339197952. Throughput: 0: 10460.2, 1: 10536.3. Samples: 339162012. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:52,364][19170] Avg episode reward: [(0, '795.080'), (1, '990.310')] [2023-12-27 10:12:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000662104_169525248.pth... [2023-12-27 10:12:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000662704_169672704.pth... [2023-12-27 10:12:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000661456_169353216.pth [2023-12-27 10:12:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000660888_169213952.pth [2023-12-27 10:12:52,885][20381] Updated weights for policy 1, policy_version 662107 (0.0008) [2023-12-27 10:12:52,898][20347] Updated weights for policy 0, policy_version 662712 (0.0006) [2023-12-27 10:12:52,943][20381] Updated weights for policy 1, policy_version 662117 (0.0007) [2023-12-27 10:12:52,950][20347] Updated weights for policy 0, policy_version 662722 (0.0006) [2023-12-27 10:12:52,998][20347] Updated weights for policy 0, policy_version 662732 (0.0007) [2023-12-27 10:12:53,000][20381] Updated weights for policy 1, policy_version 662127 (0.0006) [2023-12-27 10:12:53,607][20381] Updated weights for policy 1, policy_version 662137 (0.0007) [2023-12-27 10:12:53,662][20381] Updated weights for policy 1, policy_version 662147 (0.0007) [2023-12-27 10:12:53,703][20347] Updated weights for policy 0, policy_version 662742 (0.0009) [2023-12-27 10:12:53,717][20381] Updated weights for policy 1, policy_version 662157 (0.0008) [2023-12-27 10:12:53,758][20347] Updated weights for policy 0, policy_version 662752 (0.0008) [2023-12-27 10:12:53,764][20381] Updated weights for policy 1, policy_version 662167 (0.0007) [2023-12-27 10:12:53,808][20347] Updated weights for policy 0, policy_version 662762 (0.0008) [2023-12-27 10:12:54,416][20381] Updated weights for policy 1, policy_version 662177 (0.0010) [2023-12-27 10:12:54,478][20381] Updated weights for policy 1, policy_version 662187 (0.0010) [2023-12-27 10:12:54,520][20347] Updated weights for policy 0, policy_version 662772 (0.0009) [2023-12-27 10:12:54,534][20381] Updated weights for policy 1, policy_version 662197 (0.0007) [2023-12-27 10:12:54,575][20347] Updated weights for policy 0, policy_version 662782 (0.0008) [2023-12-27 10:12:54,633][20347] Updated weights for policy 0, policy_version 662792 (0.0008) [2023-12-27 10:12:55,190][20381] Updated weights for policy 1, policy_version 662207 (0.0008) [2023-12-27 10:12:55,242][20381] Updated weights for policy 1, policy_version 662217 (0.0007) [2023-12-27 10:12:55,295][20381] Updated weights for policy 1, policy_version 662227 (0.0008) [2023-12-27 10:12:55,321][20347] Updated weights for policy 0, policy_version 662802 (0.0008) [2023-12-27 10:12:55,369][20347] Updated weights for policy 0, policy_version 662812 (0.0008) [2023-12-27 10:12:55,430][20347] Updated weights for policy 0, policy_version 662822 (0.0007) [2023-12-27 10:12:55,481][20347] Updated weights for policy 0, policy_version 662832 (0.0007) [2023-12-27 10:12:55,985][20381] Updated weights for policy 1, policy_version 662237 (0.0007) [2023-12-27 10:12:56,037][20381] Updated weights for policy 1, policy_version 662247 (0.0008) [2023-12-27 10:12:56,086][20381] Updated weights for policy 1, policy_version 662257 (0.0008) [2023-12-27 10:12:56,186][20347] Updated weights for policy 0, policy_version 662842 (0.0010) [2023-12-27 10:12:56,240][20347] Updated weights for policy 0, policy_version 662852 (0.0010) [2023-12-27 10:12:56,297][20347] Updated weights for policy 0, policy_version 662862 (0.0010) [2023-12-27 10:12:56,785][20381] Updated weights for policy 1, policy_version 662267 (0.0009) [2023-12-27 10:12:56,843][20381] Updated weights for policy 1, policy_version 662277 (0.0010) [2023-12-27 10:12:56,845][20347] Updated weights for policy 0, policy_version 662872 (0.0009) [2023-12-27 10:12:56,887][20381] Updated weights for policy 1, policy_version 662287 (0.0010) [2023-12-27 10:12:56,889][20347] Updated weights for policy 0, policy_version 662882 (0.0007) [2023-12-27 10:12:56,937][20347] Updated weights for policy 0, policy_version 662892 (0.0010) [2023-12-27 10:12:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21027.3, 300 sec: 20799.4). Total num frames: 339296256. Throughput: 0: 10489.0, 1: 10493.8. Samples: 339284820. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:12:57,363][19170] Avg episode reward: [(0, '853.460'), (1, '980.590')] [2023-12-27 10:12:57,571][20381] Updated weights for policy 1, policy_version 662297 (0.0010) [2023-12-27 10:12:57,599][20347] Updated weights for policy 0, policy_version 662902 (0.0008) [2023-12-27 10:12:57,631][20381] Updated weights for policy 1, policy_version 662307 (0.0010) [2023-12-27 10:12:57,651][20347] Updated weights for policy 0, policy_version 662912 (0.0006) [2023-12-27 10:12:57,685][20381] Updated weights for policy 1, policy_version 662317 (0.0010) [2023-12-27 10:12:57,703][20347] Updated weights for policy 0, policy_version 662922 (0.0008) [2023-12-27 10:12:57,733][20381] Updated weights for policy 1, policy_version 662327 (0.0010) [2023-12-27 10:12:58,312][20347] Updated weights for policy 0, policy_version 662932 (0.0008) [2023-12-27 10:12:58,369][20347] Updated weights for policy 0, policy_version 662942 (0.0008) [2023-12-27 10:12:58,431][20347] Updated weights for policy 0, policy_version 662952 (0.0009) [2023-12-27 10:12:58,442][20381] Updated weights for policy 1, policy_version 662337 (0.0008) [2023-12-27 10:12:58,502][20381] Updated weights for policy 1, policy_version 662347 (0.0008) [2023-12-27 10:12:58,566][20381] Updated weights for policy 1, policy_version 662357 (0.0007) [2023-12-27 10:12:59,231][20347] Updated weights for policy 0, policy_version 662962 (0.0007) [2023-12-27 10:12:59,290][20347] Updated weights for policy 0, policy_version 662972 (0.0007) [2023-12-27 10:12:59,300][20381] Updated weights for policy 1, policy_version 662367 (0.0009) [2023-12-27 10:12:59,344][20347] Updated weights for policy 0, policy_version 662982 (0.0007) [2023-12-27 10:12:59,362][20381] Updated weights for policy 1, policy_version 662377 (0.0009) [2023-12-27 10:12:59,405][20347] Updated weights for policy 0, policy_version 662992 (0.0007) [2023-12-27 10:12:59,422][20381] Updated weights for policy 1, policy_version 662387 (0.0010) [2023-12-27 10:13:00,059][20381] Updated weights for policy 1, policy_version 662397 (0.0006) [2023-12-27 10:13:00,068][20347] Updated weights for policy 0, policy_version 663002 (0.0007) [2023-12-27 10:13:00,114][20381] Updated weights for policy 1, policy_version 662407 (0.0006) [2023-12-27 10:13:00,120][20347] Updated weights for policy 0, policy_version 663012 (0.0008) [2023-12-27 10:13:00,168][20381] Updated weights for policy 1, policy_version 662417 (0.0006) [2023-12-27 10:13:00,170][20347] Updated weights for policy 0, policy_version 663022 (0.0007) [2023-12-27 10:13:00,782][20381] Updated weights for policy 1, policy_version 662427 (0.0006) [2023-12-27 10:13:00,836][20347] Updated weights for policy 0, policy_version 663032 (0.0006) [2023-12-27 10:13:00,842][20381] Updated weights for policy 1, policy_version 662437 (0.0007) [2023-12-27 10:13:00,879][20347] Updated weights for policy 0, policy_version 663042 (0.0005) [2023-12-27 10:13:00,899][20381] Updated weights for policy 1, policy_version 662447 (0.0007) [2023-12-27 10:13:00,937][20347] Updated weights for policy 0, policy_version 663052 (0.0006) [2023-12-27 10:13:01,601][20347] Updated weights for policy 0, policy_version 663062 (0.0008) [2023-12-27 10:13:01,620][20381] Updated weights for policy 1, policy_version 662457 (0.0007) [2023-12-27 10:13:01,660][20347] Updated weights for policy 0, policy_version 663072 (0.0008) [2023-12-27 10:13:01,681][20381] Updated weights for policy 1, policy_version 662467 (0.0006) [2023-12-27 10:13:01,723][20347] Updated weights for policy 0, policy_version 663082 (0.0006) [2023-12-27 10:13:01,747][20381] Updated weights for policy 1, policy_version 662477 (0.0008) [2023-12-27 10:13:01,803][20381] Updated weights for policy 1, policy_version 662487 (0.0008) [2023-12-27 10:13:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 339394560. Throughput: 0: 10519.8, 1: 10321.8. Samples: 339408600. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:13:02,363][19170] Avg episode reward: [(0, '892.390'), (1, '990.280')] [2023-12-27 10:13:02,405][20347] Updated weights for policy 0, policy_version 663092 (0.0008) [2023-12-27 10:13:02,453][20347] Updated weights for policy 0, policy_version 663102 (0.0008) [2023-12-27 10:13:02,509][20381] Updated weights for policy 1, policy_version 662497 (0.0006) [2023-12-27 10:13:02,511][20347] Updated weights for policy 0, policy_version 663112 (0.0007) [2023-12-27 10:13:02,565][20381] Updated weights for policy 1, policy_version 662507 (0.0006) [2023-12-27 10:13:02,620][20381] Updated weights for policy 1, policy_version 662517 (0.0007) [2023-12-27 10:13:03,109][20347] Updated weights for policy 0, policy_version 663122 (0.0005) [2023-12-27 10:13:03,154][20347] Updated weights for policy 0, policy_version 663132 (0.0006) [2023-12-27 10:13:03,204][20347] Updated weights for policy 0, policy_version 663142 (0.0008) [2023-12-27 10:13:03,254][20347] Updated weights for policy 0, policy_version 663152 (0.0007) [2023-12-27 10:13:03,317][20381] Updated weights for policy 1, policy_version 662527 (0.0006) [2023-12-27 10:13:03,377][20381] Updated weights for policy 1, policy_version 662537 (0.0005) [2023-12-27 10:13:03,429][20381] Updated weights for policy 1, policy_version 662547 (0.0007) [2023-12-27 10:13:03,820][20347] Updated weights for policy 0, policy_version 663162 (0.0008) [2023-12-27 10:13:03,882][20347] Updated weights for policy 0, policy_version 663173 (0.0008) [2023-12-27 10:13:03,938][20347] Updated weights for policy 0, policy_version 663183 (0.0009) [2023-12-27 10:13:04,064][20381] Updated weights for policy 1, policy_version 662557 (0.0007) [2023-12-27 10:13:04,115][20381] Updated weights for policy 1, policy_version 662567 (0.0006) [2023-12-27 10:13:04,182][20381] Updated weights for policy 1, policy_version 662577 (0.0007) [2023-12-27 10:13:04,609][20347] Updated weights for policy 0, policy_version 663193 (0.0008) [2023-12-27 10:13:04,652][20347] Updated weights for policy 0, policy_version 663203 (0.0008) [2023-12-27 10:13:04,704][20347] Updated weights for policy 0, policy_version 663213 (0.0007) [2023-12-27 10:13:04,811][20381] Updated weights for policy 1, policy_version 662587 (0.0007) [2023-12-27 10:13:04,878][20381] Updated weights for policy 1, policy_version 662597 (0.0006) [2023-12-27 10:13:04,939][20381] Updated weights for policy 1, policy_version 662607 (0.0007) [2023-12-27 10:13:05,340][20347] Updated weights for policy 0, policy_version 663223 (0.0006) [2023-12-27 10:13:05,389][20347] Updated weights for policy 0, policy_version 663233 (0.0005) [2023-12-27 10:13:05,439][20347] Updated weights for policy 0, policy_version 663243 (0.0005) [2023-12-27 10:13:05,533][20381] Updated weights for policy 1, policy_version 662617 (0.0006) [2023-12-27 10:13:05,595][20381] Updated weights for policy 1, policy_version 662627 (0.0006) [2023-12-27 10:13:05,651][20381] Updated weights for policy 1, policy_version 662637 (0.0006) [2023-12-27 10:13:05,707][20381] Updated weights for policy 1, policy_version 662647 (0.0005) [2023-12-27 10:13:06,024][20347] Updated weights for policy 0, policy_version 663253 (0.0008) [2023-12-27 10:13:06,071][20347] Updated weights for policy 0, policy_version 663263 (0.0010) [2023-12-27 10:13:06,128][20347] Updated weights for policy 0, policy_version 663273 (0.0010) [2023-12-27 10:13:06,333][20381] Updated weights for policy 1, policy_version 662657 (0.0006) [2023-12-27 10:13:06,387][20381] Updated weights for policy 1, policy_version 662667 (0.0007) [2023-12-27 10:13:06,440][20381] Updated weights for policy 1, policy_version 662677 (0.0007) [2023-12-27 10:13:06,812][20347] Updated weights for policy 0, policy_version 663283 (0.0008) [2023-12-27 10:13:06,859][20347] Updated weights for policy 0, policy_version 663293 (0.0010) [2023-12-27 10:13:06,904][20347] Updated weights for policy 0, policy_version 663303 (0.0007) [2023-12-27 10:13:07,113][20381] Updated weights for policy 1, policy_version 662687 (0.0007) [2023-12-27 10:13:07,167][20381] Updated weights for policy 1, policy_version 662697 (0.0005) [2023-12-27 10:13:07,222][20381] Updated weights for policy 1, policy_version 662707 (0.0006) [2023-12-27 10:13:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 339509248. Throughput: 0: 10568.3, 1: 10325.6. Samples: 339472916. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:13:07,363][19170] Avg episode reward: [(0, '882.690'), (1, '1000.000')] [2023-12-27 10:13:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000662712_169680896.pth... [2023-12-27 10:13:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000663312_169828352.pth... [2023-12-27 10:13:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000661496_169369600.pth [2023-12-27 10:13:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000662096_169517056.pth [2023-12-27 10:13:07,654][20347] Updated weights for policy 0, policy_version 663313 (0.0007) [2023-12-27 10:13:07,715][20347] Updated weights for policy 0, policy_version 663323 (0.0007) [2023-12-27 10:13:07,784][20347] Updated weights for policy 0, policy_version 663333 (0.0005) [2023-12-27 10:13:07,847][20347] Updated weights for policy 0, policy_version 663343 (0.0008) [2023-12-27 10:13:07,866][20381] Updated weights for policy 1, policy_version 662717 (0.0009) [2023-12-27 10:13:07,923][20381] Updated weights for policy 1, policy_version 662727 (0.0008) [2023-12-27 10:13:07,973][20381] Updated weights for policy 1, policy_version 662737 (0.0005) [2023-12-27 10:13:08,514][20347] Updated weights for policy 0, policy_version 663353 (0.0006) [2023-12-27 10:13:08,566][20347] Updated weights for policy 0, policy_version 663363 (0.0005) [2023-12-27 10:13:08,617][20347] Updated weights for policy 0, policy_version 663373 (0.0005) [2023-12-27 10:13:08,633][20381] Updated weights for policy 1, policy_version 662747 (0.0007) [2023-12-27 10:13:08,688][20381] Updated weights for policy 1, policy_version 662757 (0.0010) [2023-12-27 10:13:08,755][20381] Updated weights for policy 1, policy_version 662767 (0.0008) [2023-12-27 10:13:09,187][20347] Updated weights for policy 0, policy_version 663383 (0.0007) [2023-12-27 10:13:09,244][20347] Updated weights for policy 0, policy_version 663393 (0.0008) [2023-12-27 10:13:09,306][20347] Updated weights for policy 0, policy_version 663403 (0.0008) [2023-12-27 10:13:09,426][20381] Updated weights for policy 1, policy_version 662777 (0.0006) [2023-12-27 10:13:09,486][20381] Updated weights for policy 1, policy_version 662787 (0.0008) [2023-12-27 10:13:09,543][20381] Updated weights for policy 1, policy_version 662797 (0.0007) [2023-12-27 10:13:09,600][20381] Updated weights for policy 1, policy_version 662807 (0.0007) [2023-12-27 10:13:10,037][20347] Updated weights for policy 0, policy_version 663413 (0.0008) [2023-12-27 10:13:10,092][20347] Updated weights for policy 0, policy_version 663423 (0.0007) [2023-12-27 10:13:10,145][20347] Updated weights for policy 0, policy_version 663433 (0.0008) [2023-12-27 10:13:10,301][20381] Updated weights for policy 1, policy_version 662817 (0.0008) [2023-12-27 10:13:10,359][20381] Updated weights for policy 1, policy_version 662827 (0.0008) [2023-12-27 10:13:10,413][20381] Updated weights for policy 1, policy_version 662837 (0.0008) [2023-12-27 10:13:10,786][20347] Updated weights for policy 0, policy_version 663443 (0.0007) [2023-12-27 10:13:10,843][20347] Updated weights for policy 0, policy_version 663453 (0.0005) [2023-12-27 10:13:10,897][20347] Updated weights for policy 0, policy_version 663463 (0.0005) [2023-12-27 10:13:11,144][20381] Updated weights for policy 1, policy_version 662847 (0.0009) [2023-12-27 10:13:11,211][20381] Updated weights for policy 1, policy_version 662857 (0.0008) [2023-12-27 10:13:11,276][20381] Updated weights for policy 1, policy_version 662867 (0.0008) [2023-12-27 10:13:11,531][20347] Updated weights for policy 0, policy_version 663473 (0.0006) [2023-12-27 10:13:11,595][20347] Updated weights for policy 0, policy_version 663483 (0.0007) [2023-12-27 10:13:11,665][20347] Updated weights for policy 0, policy_version 663493 (0.0008) [2023-12-27 10:13:11,732][20347] Updated weights for policy 0, policy_version 663503 (0.0008) [2023-12-27 10:13:11,993][20381] Updated weights for policy 1, policy_version 662877 (0.0007) [2023-12-27 10:13:12,052][20381] Updated weights for policy 1, policy_version 662887 (0.0008) [2023-12-27 10:13:12,107][20381] Updated weights for policy 1, policy_version 662897 (0.0008) [2023-12-27 10:13:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 339607552. Throughput: 0: 10595.8, 1: 10293.8. Samples: 339599044. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:13:12,363][19170] Avg episode reward: [(0, '810.690'), (1, '1000.000')] [2023-12-27 10:13:12,427][20347] Updated weights for policy 0, policy_version 663513 (0.0007) [2023-12-27 10:13:12,486][20347] Updated weights for policy 0, policy_version 663523 (0.0007) [2023-12-27 10:13:12,542][20347] Updated weights for policy 0, policy_version 663533 (0.0008) [2023-12-27 10:13:12,798][20381] Updated weights for policy 1, policy_version 662907 (0.0007) [2023-12-27 10:13:12,855][20381] Updated weights for policy 1, policy_version 662917 (0.0006) [2023-12-27 10:13:12,908][20381] Updated weights for policy 1, policy_version 662927 (0.0005) [2023-12-27 10:13:13,246][20347] Updated weights for policy 0, policy_version 663543 (0.0007) [2023-12-27 10:13:13,296][20347] Updated weights for policy 0, policy_version 663553 (0.0007) [2023-12-27 10:13:13,362][20347] Updated weights for policy 0, policy_version 663563 (0.0010) [2023-12-27 10:13:13,504][20381] Updated weights for policy 1, policy_version 662937 (0.0006) [2023-12-27 10:13:13,565][20381] Updated weights for policy 1, policy_version 662947 (0.0009) [2023-12-27 10:13:13,632][20381] Updated weights for policy 1, policy_version 662957 (0.0005) [2023-12-27 10:13:13,686][20381] Updated weights for policy 1, policy_version 662967 (0.0005) [2023-12-27 10:13:14,032][20347] Updated weights for policy 0, policy_version 663573 (0.0008) [2023-12-27 10:13:14,097][20347] Updated weights for policy 0, policy_version 663583 (0.0006) [2023-12-27 10:13:14,152][20347] Updated weights for policy 0, policy_version 663593 (0.0006) [2023-12-27 10:13:14,288][20381] Updated weights for policy 1, policy_version 662977 (0.0007) [2023-12-27 10:13:14,344][20381] Updated weights for policy 1, policy_version 662987 (0.0009) [2023-12-27 10:13:14,397][20381] Updated weights for policy 1, policy_version 662997 (0.0010) [2023-12-27 10:13:14,858][20347] Updated weights for policy 0, policy_version 663603 (0.0007) [2023-12-27 10:13:14,920][20347] Updated weights for policy 0, policy_version 663613 (0.0008) [2023-12-27 10:13:14,982][20347] Updated weights for policy 0, policy_version 663623 (0.0007) [2023-12-27 10:13:15,125][20381] Updated weights for policy 1, policy_version 663007 (0.0008) [2023-12-27 10:13:15,191][20381] Updated weights for policy 1, policy_version 663017 (0.0009) [2023-12-27 10:13:15,246][20381] Updated weights for policy 1, policy_version 663027 (0.0008) [2023-12-27 10:13:15,700][20347] Updated weights for policy 0, policy_version 663633 (0.0007) [2023-12-27 10:13:15,761][20347] Updated weights for policy 0, policy_version 663643 (0.0008) [2023-12-27 10:13:15,814][20347] Updated weights for policy 0, policy_version 663653 (0.0007) [2023-12-27 10:13:15,869][20347] Updated weights for policy 0, policy_version 663663 (0.0007) [2023-12-27 10:13:15,947][20381] Updated weights for policy 1, policy_version 663037 (0.0009) [2023-12-27 10:13:15,992][20381] Updated weights for policy 1, policy_version 663047 (0.0010) [2023-12-27 10:13:16,041][20381] Updated weights for policy 1, policy_version 663057 (0.0010) [2023-12-27 10:13:16,571][20347] Updated weights for policy 0, policy_version 663673 (0.0008) [2023-12-27 10:13:16,625][20347] Updated weights for policy 0, policy_version 663683 (0.0007) [2023-12-27 10:13:16,675][20347] Updated weights for policy 0, policy_version 663693 (0.0007) [2023-12-27 10:13:16,744][20381] Updated weights for policy 1, policy_version 663067 (0.0010) [2023-12-27 10:13:16,793][20381] Updated weights for policy 1, policy_version 663077 (0.0005) [2023-12-27 10:13:16,850][20381] Updated weights for policy 1, policy_version 663087 (0.0005) [2023-12-27 10:13:17,347][20347] Updated weights for policy 0, policy_version 663703 (0.0006) [2023-12-27 10:13:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 339705856. Throughput: 0: 10465.7, 1: 10345.1. Samples: 339721624. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:13:17,363][19170] Avg episode reward: [(0, '784.930'), (1, '1000.000')] [2023-12-27 10:13:17,372][20381] Updated weights for policy 1, policy_version 663097 (0.0005) [2023-12-27 10:13:17,399][20347] Updated weights for policy 0, policy_version 663713 (0.0005) [2023-12-27 10:13:17,434][20381] Updated weights for policy 1, policy_version 663107 (0.0010) [2023-12-27 10:13:17,447][20347] Updated weights for policy 0, policy_version 663723 (0.0005) [2023-12-27 10:13:17,495][20381] Updated weights for policy 1, policy_version 663117 (0.0011) [2023-12-27 10:13:17,559][20381] Updated weights for policy 1, policy_version 663127 (0.0010) [2023-12-27 10:13:18,024][20347] Updated weights for policy 0, policy_version 663733 (0.0006) [2023-12-27 10:13:18,080][20347] Updated weights for policy 0, policy_version 663743 (0.0005) [2023-12-27 10:13:18,127][20347] Updated weights for policy 0, policy_version 663753 (0.0005) [2023-12-27 10:13:18,188][20381] Updated weights for policy 1, policy_version 663137 (0.0007) [2023-12-27 10:13:18,253][20381] Updated weights for policy 1, policy_version 663147 (0.0011) [2023-12-27 10:13:18,310][20381] Updated weights for policy 1, policy_version 663157 (0.0010) [2023-12-27 10:13:18,796][20347] Updated weights for policy 0, policy_version 663763 (0.0006) [2023-12-27 10:13:18,851][20347] Updated weights for policy 0, policy_version 663773 (0.0008) [2023-12-27 10:13:18,909][20347] Updated weights for policy 0, policy_version 663783 (0.0007) [2023-12-27 10:13:18,966][20381] Updated weights for policy 1, policy_version 663167 (0.0010) [2023-12-27 10:13:19,027][20381] Updated weights for policy 1, policy_version 663177 (0.0010) [2023-12-27 10:13:19,093][20381] Updated weights for policy 1, policy_version 663187 (0.0010) [2023-12-27 10:13:19,588][20347] Updated weights for policy 0, policy_version 663793 (0.0007) [2023-12-27 10:13:19,642][20347] Updated weights for policy 0, policy_version 663803 (0.0006) [2023-12-27 10:13:19,702][20347] Updated weights for policy 0, policy_version 663813 (0.0008) [2023-12-27 10:13:19,729][20381] Updated weights for policy 1, policy_version 663197 (0.0010) [2023-12-27 10:13:19,758][20347] Updated weights for policy 0, policy_version 663823 (0.0007) [2023-12-27 10:13:19,791][20381] Updated weights for policy 1, policy_version 663207 (0.0010) [2023-12-27 10:13:19,859][20381] Updated weights for policy 1, policy_version 663217 (0.0009) [2023-12-27 10:13:20,428][20347] Updated weights for policy 0, policy_version 663833 (0.0007) [2023-12-27 10:13:20,475][20347] Updated weights for policy 0, policy_version 663843 (0.0006) [2023-12-27 10:13:20,532][20347] Updated weights for policy 0, policy_version 663853 (0.0005) [2023-12-27 10:13:20,547][20381] Updated weights for policy 1, policy_version 663227 (0.0008) [2023-12-27 10:13:20,607][20381] Updated weights for policy 1, policy_version 663237 (0.0009) [2023-12-27 10:13:20,675][20381] Updated weights for policy 1, policy_version 663247 (0.0010) [2023-12-27 10:13:21,108][20347] Updated weights for policy 0, policy_version 663863 (0.0008) [2023-12-27 10:13:21,174][20347] Updated weights for policy 0, policy_version 663873 (0.0008) [2023-12-27 10:13:21,234][20347] Updated weights for policy 0, policy_version 663883 (0.0007) [2023-12-27 10:13:21,415][20381] Updated weights for policy 1, policy_version 663257 (0.0011) [2023-12-27 10:13:21,477][20381] Updated weights for policy 1, policy_version 663267 (0.0010) [2023-12-27 10:13:21,545][20381] Updated weights for policy 1, policy_version 663277 (0.0009) [2023-12-27 10:13:21,603][20381] Updated weights for policy 1, policy_version 663287 (0.0010) [2023-12-27 10:13:21,891][20347] Updated weights for policy 0, policy_version 663893 (0.0008) [2023-12-27 10:13:21,956][20347] Updated weights for policy 0, policy_version 663903 (0.0008) [2023-12-27 10:13:22,007][20347] Updated weights for policy 0, policy_version 663913 (0.0009) [2023-12-27 10:13:22,252][20381] Updated weights for policy 1, policy_version 663297 (0.0008) [2023-12-27 10:13:22,309][20381] Updated weights for policy 1, policy_version 663307 (0.0007) [2023-12-27 10:13:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 339812352. Throughput: 0: 10441.3, 1: 10353.5. Samples: 339784808. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-12-27 10:13:22,363][19170] Avg episode reward: [(0, '833.820'), (1, '1000.000')] [2023-12-27 10:13:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000663920_169984000.pth... [2023-12-27 10:13:22,371][20381] Updated weights for policy 1, policy_version 663317 (0.0008) [2023-12-27 10:13:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000662704_169672704.pth [2023-12-27 10:13:22,388][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000663320_169836544.pth... [2023-12-27 10:13:22,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000662104_169525248.pth [2023-12-27 10:13:22,752][20347] Updated weights for policy 0, policy_version 663923 (0.0008) [2023-12-27 10:13:22,801][20347] Updated weights for policy 0, policy_version 663933 (0.0007) [2023-12-27 10:13:22,852][20347] Updated weights for policy 0, policy_version 663943 (0.0008) [2023-12-27 10:13:23,063][20381] Updated weights for policy 1, policy_version 663327 (0.0007) [2023-12-27 10:13:23,120][20381] Updated weights for policy 1, policy_version 663337 (0.0008) [2023-12-27 10:13:23,164][20381] Updated weights for policy 1, policy_version 663347 (0.0008) [2023-12-27 10:13:23,500][20347] Updated weights for policy 0, policy_version 663953 (0.0007) [2023-12-27 10:13:23,555][20347] Updated weights for policy 0, policy_version 663963 (0.0010) [2023-12-27 10:13:23,611][20347] Updated weights for policy 0, policy_version 663973 (0.0010) [2023-12-27 10:13:23,664][20347] Updated weights for policy 0, policy_version 663983 (0.0009) [2023-12-27 10:13:23,827][20381] Updated weights for policy 1, policy_version 663357 (0.0008) [2023-12-27 10:13:23,877][20381] Updated weights for policy 1, policy_version 663367 (0.0005) [2023-12-27 10:13:23,934][20381] Updated weights for policy 1, policy_version 663377 (0.0006) [2023-12-27 10:13:24,237][20347] Updated weights for policy 0, policy_version 663993 (0.0005) [2023-12-27 10:13:24,290][20347] Updated weights for policy 0, policy_version 664003 (0.0005) [2023-12-27 10:13:24,350][20347] Updated weights for policy 0, policy_version 664013 (0.0005) [2023-12-27 10:13:24,529][20381] Updated weights for policy 1, policy_version 663387 (0.0006) [2023-12-27 10:13:24,588][20381] Updated weights for policy 1, policy_version 663397 (0.0005) [2023-12-27 10:13:24,650][20381] Updated weights for policy 1, policy_version 663407 (0.0010) [2023-12-27 10:13:24,875][20347] Updated weights for policy 0, policy_version 664023 (0.0009) [2023-12-27 10:13:24,933][20347] Updated weights for policy 0, policy_version 664033 (0.0010) [2023-12-27 10:13:24,998][20347] Updated weights for policy 0, policy_version 664043 (0.0010) [2023-12-27 10:13:25,203][20381] Updated weights for policy 1, policy_version 663417 (0.0008) [2023-12-27 10:13:25,261][20381] Updated weights for policy 1, policy_version 663427 (0.0005) [2023-12-27 10:13:25,315][20381] Updated weights for policy 1, policy_version 663437 (0.0005) [2023-12-27 10:13:25,380][20381] Updated weights for policy 1, policy_version 663447 (0.0009) [2023-12-27 10:13:25,673][20347] Updated weights for policy 0, policy_version 664053 (0.0010) [2023-12-27 10:13:25,722][20347] Updated weights for policy 0, policy_version 664063 (0.0010) [2023-12-27 10:13:25,767][20347] Updated weights for policy 0, policy_version 664073 (0.0010) [2023-12-27 10:13:26,027][20381] Updated weights for policy 1, policy_version 663457 (0.0009) [2023-12-27 10:13:26,086][20381] Updated weights for policy 1, policy_version 663467 (0.0007) [2023-12-27 10:13:26,142][20381] Updated weights for policy 1, policy_version 663477 (0.0008) [2023-12-27 10:13:26,474][20347] Updated weights for policy 0, policy_version 664083 (0.0010) [2023-12-27 10:13:26,524][20347] Updated weights for policy 0, policy_version 664093 (0.0010) [2023-12-27 10:13:26,577][20347] Updated weights for policy 0, policy_version 664103 (0.0010) [2023-12-27 10:13:26,821][20381] Updated weights for policy 1, policy_version 663487 (0.0007) [2023-12-27 10:13:26,880][20381] Updated weights for policy 1, policy_version 663497 (0.0008) [2023-12-27 10:13:26,942][20381] Updated weights for policy 1, policy_version 663507 (0.0008) [2023-12-27 10:13:27,203][20347] Updated weights for policy 0, policy_version 664113 (0.0009) [2023-12-27 10:13:27,265][20347] Updated weights for policy 0, policy_version 664123 (0.0005) [2023-12-27 10:13:27,328][20347] Updated weights for policy 0, policy_version 664133 (0.0007) [2023-12-27 10:13:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 339918848. Throughput: 0: 10470.0, 1: 10394.6. Samples: 339912692. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:27,363][19170] Avg episode reward: [(0, '892.790'), (1, '1000.000')] [2023-12-27 10:13:27,392][20347] Updated weights for policy 0, policy_version 664143 (0.0006) [2023-12-27 10:13:27,643][20381] Updated weights for policy 1, policy_version 663517 (0.0006) [2023-12-27 10:13:27,698][20381] Updated weights for policy 1, policy_version 663527 (0.0005) [2023-12-27 10:13:27,753][20381] Updated weights for policy 1, policy_version 663537 (0.0008) [2023-12-27 10:13:27,924][20347] Updated weights for policy 0, policy_version 664153 (0.0007) [2023-12-27 10:13:27,978][20347] Updated weights for policy 0, policy_version 664163 (0.0010) [2023-12-27 10:13:28,032][20347] Updated weights for policy 0, policy_version 664173 (0.0010) [2023-12-27 10:13:28,439][20381] Updated weights for policy 1, policy_version 663547 (0.0008) [2023-12-27 10:13:28,502][20381] Updated weights for policy 1, policy_version 663557 (0.0005) [2023-12-27 10:13:28,562][20381] Updated weights for policy 1, policy_version 663567 (0.0008) [2023-12-27 10:13:28,717][20347] Updated weights for policy 0, policy_version 664183 (0.0010) [2023-12-27 10:13:28,778][20347] Updated weights for policy 0, policy_version 664193 (0.0010) [2023-12-27 10:13:28,838][20347] Updated weights for policy 0, policy_version 664203 (0.0010) [2023-12-27 10:13:29,229][20381] Updated weights for policy 1, policy_version 663577 (0.0008) [2023-12-27 10:13:29,281][20381] Updated weights for policy 1, policy_version 663587 (0.0008) [2023-12-27 10:13:29,340][20381] Updated weights for policy 1, policy_version 663597 (0.0008) [2023-12-27 10:13:29,391][20381] Updated weights for policy 1, policy_version 663607 (0.0008) [2023-12-27 10:13:29,482][20347] Updated weights for policy 0, policy_version 664213 (0.0010) [2023-12-27 10:13:29,530][20347] Updated weights for policy 0, policy_version 664223 (0.0010) [2023-12-27 10:13:29,581][20347] Updated weights for policy 0, policy_version 664233 (0.0010) [2023-12-27 10:13:30,095][20381] Updated weights for policy 1, policy_version 663617 (0.0008) [2023-12-27 10:13:30,143][20381] Updated weights for policy 1, policy_version 663627 (0.0007) [2023-12-27 10:13:30,190][20381] Updated weights for policy 1, policy_version 663637 (0.0008) [2023-12-27 10:13:30,294][20347] Updated weights for policy 0, policy_version 664243 (0.0010) [2023-12-27 10:13:30,351][20347] Updated weights for policy 0, policy_version 664253 (0.0008) [2023-12-27 10:13:30,407][20347] Updated weights for policy 0, policy_version 664263 (0.0005) [2023-12-27 10:13:30,903][20381] Updated weights for policy 1, policy_version 663647 (0.0008) [2023-12-27 10:13:30,946][20381] Updated weights for policy 1, policy_version 663657 (0.0008) [2023-12-27 10:13:30,984][20347] Updated weights for policy 0, policy_version 664273 (0.0005) [2023-12-27 10:13:31,007][20381] Updated weights for policy 1, policy_version 663667 (0.0006) [2023-12-27 10:13:31,047][20347] Updated weights for policy 0, policy_version 664283 (0.0006) [2023-12-27 10:13:31,102][20347] Updated weights for policy 0, policy_version 664293 (0.0008) [2023-12-27 10:13:31,162][20347] Updated weights for policy 0, policy_version 664303 (0.0007) [2023-12-27 10:13:31,683][20381] Updated weights for policy 1, policy_version 663677 (0.0008) [2023-12-27 10:13:31,738][20381] Updated weights for policy 1, policy_version 663687 (0.0008) [2023-12-27 10:13:31,790][20381] Updated weights for policy 1, policy_version 663697 (0.0008) [2023-12-27 10:13:31,833][20347] Updated weights for policy 0, policy_version 664313 (0.0005) [2023-12-27 10:13:31,886][20347] Updated weights for policy 0, policy_version 664323 (0.0008) [2023-12-27 10:13:31,943][20347] Updated weights for policy 0, policy_version 664333 (0.0007) [2023-12-27 10:13:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 340025344. Throughput: 0: 10545.9, 1: 10314.1. Samples: 340037356. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:32,363][19170] Avg episode reward: [(0, '883.200'), (1, '1000.000')] [2023-12-27 10:13:32,557][20347] Updated weights for policy 0, policy_version 664343 (0.0006) [2023-12-27 10:13:32,559][20381] Updated weights for policy 1, policy_version 663707 (0.0006) [2023-12-27 10:13:32,614][20347] Updated weights for policy 0, policy_version 664353 (0.0006) [2023-12-27 10:13:32,620][20381] Updated weights for policy 1, policy_version 663717 (0.0006) [2023-12-27 10:13:32,676][20347] Updated weights for policy 0, policy_version 664363 (0.0006) [2023-12-27 10:13:32,680][20381] Updated weights for policy 1, policy_version 663727 (0.0008) [2023-12-27 10:13:33,277][20347] Updated weights for policy 0, policy_version 664373 (0.0006) [2023-12-27 10:13:33,323][20347] Updated weights for policy 0, policy_version 664383 (0.0006) [2023-12-27 10:13:33,373][20347] Updated weights for policy 0, policy_version 664393 (0.0007) [2023-12-27 10:13:33,386][20381] Updated weights for policy 1, policy_version 663737 (0.0008) [2023-12-27 10:13:33,435][20381] Updated weights for policy 1, policy_version 663747 (0.0008) [2023-12-27 10:13:33,483][20381] Updated weights for policy 1, policy_version 663757 (0.0009) [2023-12-27 10:13:33,534][20381] Updated weights for policy 1, policy_version 663767 (0.0009) [2023-12-27 10:13:33,959][20347] Updated weights for policy 0, policy_version 664403 (0.0005) [2023-12-27 10:13:34,021][20347] Updated weights for policy 0, policy_version 664413 (0.0007) [2023-12-27 10:13:34,069][20347] Updated weights for policy 0, policy_version 664423 (0.0007) [2023-12-27 10:13:34,278][20381] Updated weights for policy 1, policy_version 663777 (0.0008) [2023-12-27 10:13:34,333][20381] Updated weights for policy 1, policy_version 663787 (0.0008) [2023-12-27 10:13:34,391][20381] Updated weights for policy 1, policy_version 663797 (0.0007) [2023-12-27 10:13:34,740][20347] Updated weights for policy 0, policy_version 664433 (0.0007) [2023-12-27 10:13:34,790][20347] Updated weights for policy 0, policy_version 664443 (0.0008) [2023-12-27 10:13:34,842][20347] Updated weights for policy 0, policy_version 664453 (0.0007) [2023-12-27 10:13:34,896][20347] Updated weights for policy 0, policy_version 664463 (0.0007) [2023-12-27 10:13:35,095][20381] Updated weights for policy 1, policy_version 663807 (0.0008) [2023-12-27 10:13:35,150][20381] Updated weights for policy 1, policy_version 663817 (0.0007) [2023-12-27 10:13:35,207][20381] Updated weights for policy 1, policy_version 663827 (0.0008) [2023-12-27 10:13:35,571][20347] Updated weights for policy 0, policy_version 664473 (0.0008) [2023-12-27 10:13:35,624][20347] Updated weights for policy 0, policy_version 664483 (0.0008) [2023-12-27 10:13:35,673][20347] Updated weights for policy 0, policy_version 664493 (0.0007) [2023-12-27 10:13:35,848][20381] Updated weights for policy 1, policy_version 663837 (0.0007) [2023-12-27 10:13:35,897][20381] Updated weights for policy 1, policy_version 663847 (0.0007) [2023-12-27 10:13:35,958][20381] Updated weights for policy 1, policy_version 663857 (0.0008) [2023-12-27 10:13:36,408][20347] Updated weights for policy 0, policy_version 664503 (0.0007) [2023-12-27 10:13:36,462][20347] Updated weights for policy 0, policy_version 664513 (0.0008) [2023-12-27 10:13:36,523][20347] Updated weights for policy 0, policy_version 664523 (0.0007) [2023-12-27 10:13:36,599][20381] Updated weights for policy 1, policy_version 663867 (0.0007) [2023-12-27 10:13:36,653][20381] Updated weights for policy 1, policy_version 663877 (0.0008) [2023-12-27 10:13:36,707][20381] Updated weights for policy 1, policy_version 663887 (0.0008) [2023-12-27 10:13:37,278][20347] Updated weights for policy 0, policy_version 664533 (0.0008) [2023-12-27 10:13:37,299][20381] Updated weights for policy 1, policy_version 663897 (0.0008) [2023-12-27 10:13:37,342][20347] Updated weights for policy 0, policy_version 664543 (0.0007) [2023-12-27 10:13:37,360][20381] Updated weights for policy 1, policy_version 663907 (0.0007) [2023-12-27 10:13:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 340123648. Throughput: 0: 10578.5, 1: 10251.4. Samples: 340099356. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:37,364][19170] Avg episode reward: [(0, '922.120'), (1, '1000.000')] [2023-12-27 10:13:37,406][20347] Updated weights for policy 0, policy_version 664553 (0.0007) [2023-12-27 10:13:37,418][20381] Updated weights for policy 1, policy_version 663917 (0.0007) [2023-12-27 10:13:37,448][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000664560_170147840.pth... [2023-12-27 10:13:37,453][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000663312_169828352.pth [2023-12-27 10:13:37,479][20381] Updated weights for policy 1, policy_version 663927 (0.0007) [2023-12-27 10:13:37,483][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000663928_169992192.pth... [2023-12-27 10:13:37,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000662712_169680896.pth [2023-12-27 10:13:38,086][20347] Updated weights for policy 0, policy_version 664563 (0.0006) [2023-12-27 10:13:38,127][20381] Updated weights for policy 1, policy_version 663937 (0.0005) [2023-12-27 10:13:38,141][20347] Updated weights for policy 0, policy_version 664573 (0.0008) [2023-12-27 10:13:38,177][20381] Updated weights for policy 1, policy_version 663947 (0.0005) [2023-12-27 10:13:38,197][20347] Updated weights for policy 0, policy_version 664583 (0.0008) [2023-12-27 10:13:38,233][20381] Updated weights for policy 1, policy_version 663957 (0.0005) [2023-12-27 10:13:38,859][20381] Updated weights for policy 1, policy_version 663967 (0.0005) [2023-12-27 10:13:38,918][20381] Updated weights for policy 1, policy_version 663977 (0.0006) [2023-12-27 10:13:38,924][20347] Updated weights for policy 0, policy_version 664593 (0.0007) [2023-12-27 10:13:38,974][20381] Updated weights for policy 1, policy_version 663987 (0.0007) [2023-12-27 10:13:38,984][20347] Updated weights for policy 0, policy_version 664603 (0.0007) [2023-12-27 10:13:39,047][20347] Updated weights for policy 0, policy_version 664613 (0.0007) [2023-12-27 10:13:39,108][20347] Updated weights for policy 0, policy_version 664623 (0.0008) [2023-12-27 10:13:39,616][20381] Updated weights for policy 1, policy_version 663997 (0.0008) [2023-12-27 10:13:39,677][20381] Updated weights for policy 1, policy_version 664007 (0.0007) [2023-12-27 10:13:39,739][20381] Updated weights for policy 1, policy_version 664017 (0.0011) [2023-12-27 10:13:39,811][20347] Updated weights for policy 0, policy_version 664633 (0.0006) [2023-12-27 10:13:39,874][20347] Updated weights for policy 0, policy_version 664643 (0.0007) [2023-12-27 10:13:39,935][20347] Updated weights for policy 0, policy_version 664653 (0.0007) [2023-12-27 10:13:40,443][20381] Updated weights for policy 1, policy_version 664027 (0.0011) [2023-12-27 10:13:40,501][20381] Updated weights for policy 1, policy_version 664037 (0.0010) [2023-12-27 10:13:40,556][20381] Updated weights for policy 1, policy_version 664047 (0.0010) [2023-12-27 10:13:40,598][20347] Updated weights for policy 0, policy_version 664663 (0.0005) [2023-12-27 10:13:40,645][20347] Updated weights for policy 0, policy_version 664673 (0.0007) [2023-12-27 10:13:40,695][20347] Updated weights for policy 0, policy_version 664683 (0.0007) [2023-12-27 10:13:41,177][20381] Updated weights for policy 1, policy_version 664057 (0.0010) [2023-12-27 10:13:41,232][20381] Updated weights for policy 1, policy_version 664067 (0.0010) [2023-12-27 10:13:41,279][20381] Updated weights for policy 1, policy_version 664077 (0.0011) [2023-12-27 10:13:41,333][20381] Updated weights for policy 1, policy_version 664087 (0.0010) [2023-12-27 10:13:41,481][20347] Updated weights for policy 0, policy_version 664693 (0.0007) [2023-12-27 10:13:41,543][20347] Updated weights for policy 0, policy_version 664703 (0.0007) [2023-12-27 10:13:41,605][20347] Updated weights for policy 0, policy_version 664713 (0.0007) [2023-12-27 10:13:42,030][20381] Updated weights for policy 1, policy_version 664097 (0.0008) [2023-12-27 10:13:42,088][20381] Updated weights for policy 1, policy_version 664107 (0.0007) [2023-12-27 10:13:42,134][20381] Updated weights for policy 1, policy_version 664117 (0.0005) [2023-12-27 10:13:42,314][20347] Updated weights for policy 0, policy_version 664723 (0.0008) [2023-12-27 10:13:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 340230144. Throughput: 0: 10497.7, 1: 10337.3. Samples: 340222396. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:42,363][19170] Avg episode reward: [(0, '805.630'), (1, '1000.000')] [2023-12-27 10:13:42,385][20347] Updated weights for policy 0, policy_version 664733 (0.0008) [2023-12-27 10:13:42,450][20347] Updated weights for policy 0, policy_version 664743 (0.0008) [2023-12-27 10:13:42,801][20381] Updated weights for policy 1, policy_version 664127 (0.0006) [2023-12-27 10:13:42,864][20381] Updated weights for policy 1, policy_version 664137 (0.0009) [2023-12-27 10:13:42,921][20381] Updated weights for policy 1, policy_version 664147 (0.0008) [2023-12-27 10:13:43,082][20347] Updated weights for policy 0, policy_version 664753 (0.0008) [2023-12-27 10:13:43,139][20347] Updated weights for policy 0, policy_version 664763 (0.0007) [2023-12-27 10:13:43,194][20347] Updated weights for policy 0, policy_version 664773 (0.0008) [2023-12-27 10:13:43,256][20347] Updated weights for policy 0, policy_version 664783 (0.0007) [2023-12-27 10:13:43,603][20381] Updated weights for policy 1, policy_version 664157 (0.0007) [2023-12-27 10:13:43,660][20381] Updated weights for policy 1, policy_version 664167 (0.0007) [2023-12-27 10:13:43,723][20381] Updated weights for policy 1, policy_version 664177 (0.0005) [2023-12-27 10:13:43,886][20347] Updated weights for policy 0, policy_version 664793 (0.0010) [2023-12-27 10:13:43,938][20347] Updated weights for policy 0, policy_version 664803 (0.0010) [2023-12-27 10:13:43,994][20347] Updated weights for policy 0, policy_version 664813 (0.0010) [2023-12-27 10:13:44,410][20381] Updated weights for policy 1, policy_version 664187 (0.0006) [2023-12-27 10:13:44,460][20381] Updated weights for policy 1, policy_version 664197 (0.0006) [2023-12-27 10:13:44,522][20381] Updated weights for policy 1, policy_version 664207 (0.0007) [2023-12-27 10:13:44,566][20347] Updated weights for policy 0, policy_version 664823 (0.0007) [2023-12-27 10:13:44,625][20347] Updated weights for policy 0, policy_version 664833 (0.0010) [2023-12-27 10:13:44,679][20347] Updated weights for policy 0, policy_version 664843 (0.0010) [2023-12-27 10:13:45,265][20381] Updated weights for policy 1, policy_version 664217 (0.0007) [2023-12-27 10:13:45,320][20381] Updated weights for policy 1, policy_version 664227 (0.0007) [2023-12-27 10:13:45,382][20381] Updated weights for policy 1, policy_version 664237 (0.0008) [2023-12-27 10:13:45,384][20347] Updated weights for policy 0, policy_version 664853 (0.0010) [2023-12-27 10:13:45,438][20381] Updated weights for policy 1, policy_version 664247 (0.0006) [2023-12-27 10:13:45,444][20347] Updated weights for policy 0, policy_version 664863 (0.0011) [2023-12-27 10:13:45,494][20347] Updated weights for policy 0, policy_version 664873 (0.0010) [2023-12-27 10:13:46,131][20381] Updated weights for policy 1, policy_version 664257 (0.0008) [2023-12-27 10:13:46,151][20347] Updated weights for policy 0, policy_version 664883 (0.0010) [2023-12-27 10:13:46,182][20381] Updated weights for policy 1, policy_version 664267 (0.0005) [2023-12-27 10:13:46,210][20347] Updated weights for policy 0, policy_version 664893 (0.0010) [2023-12-27 10:13:46,241][20381] Updated weights for policy 1, policy_version 664277 (0.0006) [2023-12-27 10:13:46,261][20347] Updated weights for policy 0, policy_version 664903 (0.0010) [2023-12-27 10:13:46,938][20347] Updated weights for policy 0, policy_version 664913 (0.0010) [2023-12-27 10:13:46,941][20381] Updated weights for policy 1, policy_version 664287 (0.0007) [2023-12-27 10:13:46,992][20381] Updated weights for policy 1, policy_version 664297 (0.0007) [2023-12-27 10:13:46,994][20347] Updated weights for policy 0, policy_version 664923 (0.0007) [2023-12-27 10:13:47,044][20381] Updated weights for policy 1, policy_version 664307 (0.0005) [2023-12-27 10:13:47,046][20347] Updated weights for policy 0, policy_version 664933 (0.0010) [2023-12-27 10:13:47,100][20347] Updated weights for policy 0, policy_version 664943 (0.0010) [2023-12-27 10:13:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20854.9). Total num frames: 340336640. Throughput: 0: 10483.9, 1: 10334.9. Samples: 340345444. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:47,363][19170] Avg episode reward: [(0, '805.260'), (1, '990.400')] [2023-12-27 10:13:47,745][20381] Updated weights for policy 1, policy_version 664317 (0.0006) [2023-12-27 10:13:47,781][20347] Updated weights for policy 0, policy_version 664953 (0.0010) [2023-12-27 10:13:47,790][20381] Updated weights for policy 1, policy_version 664327 (0.0006) [2023-12-27 10:13:47,830][20347] Updated weights for policy 0, policy_version 664963 (0.0010) [2023-12-27 10:13:47,836][20381] Updated weights for policy 1, policy_version 664337 (0.0005) [2023-12-27 10:13:47,876][20347] Updated weights for policy 0, policy_version 664973 (0.0010) [2023-12-27 10:13:48,541][20347] Updated weights for policy 0, policy_version 664983 (0.0007) [2023-12-27 10:13:48,541][20381] Updated weights for policy 1, policy_version 664347 (0.0007) [2023-12-27 10:13:48,599][20381] Updated weights for policy 1, policy_version 664357 (0.0008) [2023-12-27 10:13:48,602][20347] Updated weights for policy 0, policy_version 664993 (0.0006) [2023-12-27 10:13:48,659][20381] Updated weights for policy 1, policy_version 664367 (0.0008) [2023-12-27 10:13:48,664][20347] Updated weights for policy 0, policy_version 665003 (0.0006) [2023-12-27 10:13:49,346][20347] Updated weights for policy 0, policy_version 665013 (0.0008) [2023-12-27 10:13:49,361][20381] Updated weights for policy 1, policy_version 664377 (0.0007) [2023-12-27 10:13:49,412][20347] Updated weights for policy 0, policy_version 665023 (0.0007) [2023-12-27 10:13:49,419][20381] Updated weights for policy 1, policy_version 664387 (0.0006) [2023-12-27 10:13:49,471][20347] Updated weights for policy 0, policy_version 665033 (0.0006) [2023-12-27 10:13:49,476][20381] Updated weights for policy 1, policy_version 664397 (0.0006) [2023-12-27 10:13:49,530][20381] Updated weights for policy 1, policy_version 664407 (0.0006) [2023-12-27 10:13:50,088][20347] Updated weights for policy 0, policy_version 665043 (0.0005) [2023-12-27 10:13:50,150][20347] Updated weights for policy 0, policy_version 665053 (0.0006) [2023-12-27 10:13:50,210][20347] Updated weights for policy 0, policy_version 665063 (0.0006) [2023-12-27 10:13:50,241][20381] Updated weights for policy 1, policy_version 664417 (0.0009) [2023-12-27 10:13:50,295][20381] Updated weights for policy 1, policy_version 664427 (0.0007) [2023-12-27 10:13:50,346][20381] Updated weights for policy 1, policy_version 664437 (0.0007) [2023-12-27 10:13:50,833][20347] Updated weights for policy 0, policy_version 665073 (0.0008) [2023-12-27 10:13:50,885][20347] Updated weights for policy 0, policy_version 665083 (0.0007) [2023-12-27 10:13:50,938][20347] Updated weights for policy 0, policy_version 665093 (0.0008) [2023-12-27 10:13:50,995][20347] Updated weights for policy 0, policy_version 665103 (0.0008) [2023-12-27 10:13:51,058][20381] Updated weights for policy 1, policy_version 664447 (0.0008) [2023-12-27 10:13:51,111][20381] Updated weights for policy 1, policy_version 664457 (0.0008) [2023-12-27 10:13:51,165][20381] Updated weights for policy 1, policy_version 664467 (0.0007) [2023-12-27 10:13:51,701][20347] Updated weights for policy 0, policy_version 665113 (0.0006) [2023-12-27 10:13:51,762][20347] Updated weights for policy 0, policy_version 665123 (0.0008) [2023-12-27 10:13:51,806][20347] Updated weights for policy 0, policy_version 665133 (0.0008) [2023-12-27 10:13:51,902][20381] Updated weights for policy 1, policy_version 664477 (0.0007) [2023-12-27 10:13:51,963][20381] Updated weights for policy 1, policy_version 664487 (0.0008) [2023-12-27 10:13:52,022][20381] Updated weights for policy 1, policy_version 664497 (0.0008) [2023-12-27 10:13:52,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.5, 300 sec: 20854.9). Total num frames: 340434944. Throughput: 0: 10453.4, 1: 10296.4. Samples: 340406664. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:52,363][19170] Avg episode reward: [(0, '834.390'), (1, '990.400')] [2023-12-27 10:13:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000664504_170139648.pth... [2023-12-27 10:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000665136_170295296.pth... [2023-12-27 10:13:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000663320_169836544.pth [2023-12-27 10:13:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000663920_169984000.pth [2023-12-27 10:13:52,521][20347] Updated weights for policy 0, policy_version 665143 (0.0008) [2023-12-27 10:13:52,580][20347] Updated weights for policy 0, policy_version 665153 (0.0008) [2023-12-27 10:13:52,640][20347] Updated weights for policy 0, policy_version 665163 (0.0008) [2023-12-27 10:13:52,724][20381] Updated weights for policy 1, policy_version 664507 (0.0008) [2023-12-27 10:13:52,772][20381] Updated weights for policy 1, policy_version 664517 (0.0007) [2023-12-27 10:13:52,820][20381] Updated weights for policy 1, policy_version 664527 (0.0008) [2023-12-27 10:13:53,339][20347] Updated weights for policy 0, policy_version 665173 (0.0008) [2023-12-27 10:13:53,393][20347] Updated weights for policy 0, policy_version 665183 (0.0008) [2023-12-27 10:13:53,451][20347] Updated weights for policy 0, policy_version 665193 (0.0008) [2023-12-27 10:13:53,518][20381] Updated weights for policy 1, policy_version 664537 (0.0008) [2023-12-27 10:13:53,566][20381] Updated weights for policy 1, policy_version 664547 (0.0008) [2023-12-27 10:13:53,633][20381] Updated weights for policy 1, policy_version 664557 (0.0007) [2023-12-27 10:13:53,691][20381] Updated weights for policy 1, policy_version 664567 (0.0008) [2023-12-27 10:13:54,114][20347] Updated weights for policy 0, policy_version 665203 (0.0008) [2023-12-27 10:13:54,171][20347] Updated weights for policy 0, policy_version 665213 (0.0007) [2023-12-27 10:13:54,229][20347] Updated weights for policy 0, policy_version 665223 (0.0007) [2023-12-27 10:13:54,399][20381] Updated weights for policy 1, policy_version 664577 (0.0007) [2023-12-27 10:13:54,449][20381] Updated weights for policy 1, policy_version 664587 (0.0008) [2023-12-27 10:13:54,498][20381] Updated weights for policy 1, policy_version 664597 (0.0007) [2023-12-27 10:13:54,925][20347] Updated weights for policy 0, policy_version 665233 (0.0008) [2023-12-27 10:13:54,986][20347] Updated weights for policy 0, policy_version 665243 (0.0006) [2023-12-27 10:13:55,037][20347] Updated weights for policy 0, policy_version 665253 (0.0005) [2023-12-27 10:13:55,088][20347] Updated weights for policy 0, policy_version 665263 (0.0007) [2023-12-27 10:13:55,177][20381] Updated weights for policy 1, policy_version 664607 (0.0007) [2023-12-27 10:13:55,221][20381] Updated weights for policy 1, policy_version 664617 (0.0005) [2023-12-27 10:13:55,268][20381] Updated weights for policy 1, policy_version 664627 (0.0005) [2023-12-27 10:13:55,790][20347] Updated weights for policy 0, policy_version 665273 (0.0007) [2023-12-27 10:13:55,836][20347] Updated weights for policy 0, policy_version 665283 (0.0007) [2023-12-27 10:13:55,849][20381] Updated weights for policy 1, policy_version 664637 (0.0008) [2023-12-27 10:13:55,884][20347] Updated weights for policy 0, policy_version 665293 (0.0005) [2023-12-27 10:13:55,894][20381] Updated weights for policy 1, policy_version 664647 (0.0009) [2023-12-27 10:13:55,942][20381] Updated weights for policy 1, policy_version 664657 (0.0010) [2023-12-27 10:13:56,519][20347] Updated weights for policy 0, policy_version 665303 (0.0007) [2023-12-27 10:13:56,564][20347] Updated weights for policy 0, policy_version 665313 (0.0006) [2023-12-27 10:13:56,611][20347] Updated weights for policy 0, policy_version 665323 (0.0006) [2023-12-27 10:13:56,631][20381] Updated weights for policy 1, policy_version 664667 (0.0011) [2023-12-27 10:13:56,688][20381] Updated weights for policy 1, policy_version 664677 (0.0011) [2023-12-27 10:13:56,739][20381] Updated weights for policy 1, policy_version 664687 (0.0010) [2023-12-27 10:13:57,261][20347] Updated weights for policy 0, policy_version 665333 (0.0006) [2023-12-27 10:13:57,322][20347] Updated weights for policy 0, policy_version 665343 (0.0006) [2023-12-27 10:13:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20799.4). Total num frames: 340533248. Throughput: 0: 10384.1, 1: 10276.8. Samples: 340528784. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:13:57,363][20381] Updated weights for policy 1, policy_version 664697 (0.0009) [2023-12-27 10:13:57,363][19170] Avg episode reward: [(0, '785.540'), (1, '1000.000')] [2023-12-27 10:13:57,381][20347] Updated weights for policy 0, policy_version 665353 (0.0005) [2023-12-27 10:13:57,414][20381] Updated weights for policy 1, policy_version 664707 (0.0009) [2023-12-27 10:13:57,473][20381] Updated weights for policy 1, policy_version 664717 (0.0009) [2023-12-27 10:13:57,528][20381] Updated weights for policy 1, policy_version 664727 (0.0008) [2023-12-27 10:13:57,908][20347] Updated weights for policy 0, policy_version 665363 (0.0006) [2023-12-27 10:13:57,956][20347] Updated weights for policy 0, policy_version 665373 (0.0008) [2023-12-27 10:13:58,006][20347] Updated weights for policy 0, policy_version 665383 (0.0008) [2023-12-27 10:13:58,202][20381] Updated weights for policy 1, policy_version 664737 (0.0008) [2023-12-27 10:13:58,258][20381] Updated weights for policy 1, policy_version 664747 (0.0008) [2023-12-27 10:13:58,318][20381] Updated weights for policy 1, policy_version 664757 (0.0010) [2023-12-27 10:13:58,760][20347] Updated weights for policy 0, policy_version 665393 (0.0007) [2023-12-27 10:13:58,828][20347] Updated weights for policy 0, policy_version 665403 (0.0009) [2023-12-27 10:13:58,894][20347] Updated weights for policy 0, policy_version 665413 (0.0009) [2023-12-27 10:13:58,958][20347] Updated weights for policy 0, policy_version 665423 (0.0008) [2023-12-27 10:13:59,106][20381] Updated weights for policy 1, policy_version 664767 (0.0010) [2023-12-27 10:13:59,162][20381] Updated weights for policy 1, policy_version 664777 (0.0009) [2023-12-27 10:13:59,223][20381] Updated weights for policy 1, policy_version 664787 (0.0007) [2023-12-27 10:13:59,684][20347] Updated weights for policy 0, policy_version 665433 (0.0007) [2023-12-27 10:13:59,736][20347] Updated weights for policy 0, policy_version 665443 (0.0005) [2023-12-27 10:13:59,792][20347] Updated weights for policy 0, policy_version 665453 (0.0005) [2023-12-27 10:13:59,888][20381] Updated weights for policy 1, policy_version 664797 (0.0009) [2023-12-27 10:13:59,951][20381] Updated weights for policy 1, policy_version 664807 (0.0011) [2023-12-27 10:14:00,010][20381] Updated weights for policy 1, policy_version 664817 (0.0006) [2023-12-27 10:14:00,522][20347] Updated weights for policy 0, policy_version 665463 (0.0007) [2023-12-27 10:14:00,551][20381] Updated weights for policy 1, policy_version 664827 (0.0006) [2023-12-27 10:14:00,574][20347] Updated weights for policy 0, policy_version 665473 (0.0007) [2023-12-27 10:14:00,607][20381] Updated weights for policy 1, policy_version 664837 (0.0005) [2023-12-27 10:14:00,620][20347] Updated weights for policy 0, policy_version 665483 (0.0008) [2023-12-27 10:14:00,663][20381] Updated weights for policy 1, policy_version 664847 (0.0005) [2023-12-27 10:14:01,211][20381] Updated weights for policy 1, policy_version 664857 (0.0006) [2023-12-27 10:14:01,270][20381] Updated weights for policy 1, policy_version 664867 (0.0010) [2023-12-27 10:14:01,328][20381] Updated weights for policy 1, policy_version 664877 (0.0010) [2023-12-27 10:14:01,346][20347] Updated weights for policy 0, policy_version 665493 (0.0007) [2023-12-27 10:14:01,392][20381] Updated weights for policy 1, policy_version 664887 (0.0008) [2023-12-27 10:14:01,408][20347] Updated weights for policy 0, policy_version 665503 (0.0008) [2023-12-27 10:14:01,465][20347] Updated weights for policy 0, policy_version 665513 (0.0008) [2023-12-27 10:14:02,054][20381] Updated weights for policy 1, policy_version 664897 (0.0011) [2023-12-27 10:14:02,108][20381] Updated weights for policy 1, policy_version 664907 (0.0010) [2023-12-27 10:14:02,141][20347] Updated weights for policy 0, policy_version 665523 (0.0008) [2023-12-27 10:14:02,170][20381] Updated weights for policy 1, policy_version 664917 (0.0010) [2023-12-27 10:14:02,197][20347] Updated weights for policy 0, policy_version 665533 (0.0006) [2023-12-27 10:14:02,254][20347] Updated weights for policy 0, policy_version 665543 (0.0007) [2023-12-27 10:14:02,362][19170] Fps is (10 sec: 21300.1, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 340647936. Throughput: 0: 10421.6, 1: 10280.8. Samples: 340653232. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:02,363][19170] Avg episode reward: [(0, '814.720'), (1, '1000.000')] [2023-12-27 10:14:02,854][20381] Updated weights for policy 1, policy_version 664927 (0.0008) [2023-12-27 10:14:02,915][20381] Updated weights for policy 1, policy_version 664937 (0.0009) [2023-12-27 10:14:02,949][20347] Updated weights for policy 0, policy_version 665553 (0.0008) [2023-12-27 10:14:02,980][20381] Updated weights for policy 1, policy_version 664947 (0.0009) [2023-12-27 10:14:03,002][20347] Updated weights for policy 0, policy_version 665563 (0.0007) [2023-12-27 10:14:03,057][20347] Updated weights for policy 0, policy_version 665573 (0.0005) [2023-12-27 10:14:03,116][20347] Updated weights for policy 0, policy_version 665583 (0.0007) [2023-12-27 10:14:03,615][20381] Updated weights for policy 1, policy_version 664957 (0.0008) [2023-12-27 10:14:03,668][20381] Updated weights for policy 1, policy_version 664967 (0.0010) [2023-12-27 10:14:03,722][20381] Updated weights for policy 1, policy_version 664977 (0.0010) [2023-12-27 10:14:03,778][20347] Updated weights for policy 0, policy_version 665593 (0.0006) [2023-12-27 10:14:03,827][20347] Updated weights for policy 0, policy_version 665603 (0.0006) [2023-12-27 10:14:03,883][20347] Updated weights for policy 0, policy_version 665613 (0.0007) [2023-12-27 10:14:04,401][20381] Updated weights for policy 1, policy_version 664987 (0.0010) [2023-12-27 10:14:04,457][20381] Updated weights for policy 1, policy_version 664997 (0.0007) [2023-12-27 10:14:04,508][20381] Updated weights for policy 1, policy_version 665007 (0.0009) [2023-12-27 10:14:04,579][20347] Updated weights for policy 0, policy_version 665623 (0.0005) [2023-12-27 10:14:04,638][20347] Updated weights for policy 0, policy_version 665633 (0.0008) [2023-12-27 10:14:04,694][20347] Updated weights for policy 0, policy_version 665643 (0.0008) [2023-12-27 10:14:05,144][20381] Updated weights for policy 1, policy_version 665017 (0.0007) [2023-12-27 10:14:05,202][20381] Updated weights for policy 1, policy_version 665027 (0.0010) [2023-12-27 10:14:05,245][20347] Updated weights for policy 0, policy_version 665653 (0.0007) [2023-12-27 10:14:05,258][20381] Updated weights for policy 1, policy_version 665037 (0.0010) [2023-12-27 10:14:05,301][20347] Updated weights for policy 0, policy_version 665663 (0.0009) [2023-12-27 10:14:05,312][20381] Updated weights for policy 1, policy_version 665047 (0.0011) [2023-12-27 10:14:05,367][20347] Updated weights for policy 0, policy_version 665673 (0.0007) [2023-12-27 10:14:05,934][20381] Updated weights for policy 1, policy_version 665057 (0.0006) [2023-12-27 10:14:05,980][20381] Updated weights for policy 1, policy_version 665067 (0.0005) [2023-12-27 10:14:06,025][20381] Updated weights for policy 1, policy_version 665077 (0.0005) [2023-12-27 10:14:06,077][20347] Updated weights for policy 0, policy_version 665683 (0.0007) [2023-12-27 10:14:06,145][20347] Updated weights for policy 0, policy_version 665693 (0.0007) [2023-12-27 10:14:06,213][20347] Updated weights for policy 0, policy_version 665703 (0.0006) [2023-12-27 10:14:06,686][20381] Updated weights for policy 1, policy_version 665087 (0.0007) [2023-12-27 10:14:06,741][20381] Updated weights for policy 1, policy_version 665097 (0.0007) [2023-12-27 10:14:06,793][20381] Updated weights for policy 1, policy_version 665107 (0.0008) [2023-12-27 10:14:06,825][20347] Updated weights for policy 0, policy_version 665713 (0.0006) [2023-12-27 10:14:06,882][20347] Updated weights for policy 0, policy_version 665723 (0.0009) [2023-12-27 10:14:06,940][20347] Updated weights for policy 0, policy_version 665733 (0.0008) [2023-12-27 10:14:07,007][20347] Updated weights for policy 0, policy_version 665743 (0.0007) [2023-12-27 10:14:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 340746240. Throughput: 0: 10403.5, 1: 10285.3. Samples: 340715808. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:07,363][19170] Avg episode reward: [(0, '892.890'), (1, '1000.000')] [2023-12-27 10:14:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000665112_170295296.pth... [2023-12-27 10:14:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000665744_170450944.pth... [2023-12-27 10:14:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000664560_170147840.pth [2023-12-27 10:14:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000663928_169992192.pth [2023-12-27 10:14:07,469][20381] Updated weights for policy 1, policy_version 665117 (0.0007) [2023-12-27 10:14:07,523][20381] Updated weights for policy 1, policy_version 665127 (0.0008) [2023-12-27 10:14:07,583][20381] Updated weights for policy 1, policy_version 665137 (0.0007) [2023-12-27 10:14:07,694][20347] Updated weights for policy 0, policy_version 665753 (0.0009) [2023-12-27 10:14:07,755][20347] Updated weights for policy 0, policy_version 665763 (0.0010) [2023-12-27 10:14:07,811][20347] Updated weights for policy 0, policy_version 665773 (0.0010) [2023-12-27 10:14:08,261][20381] Updated weights for policy 1, policy_version 665147 (0.0007) [2023-12-27 10:14:08,312][20381] Updated weights for policy 1, policy_version 665157 (0.0007) [2023-12-27 10:14:08,374][20381] Updated weights for policy 1, policy_version 665167 (0.0008) [2023-12-27 10:14:08,503][20347] Updated weights for policy 0, policy_version 665783 (0.0007) [2023-12-27 10:14:08,567][20347] Updated weights for policy 0, policy_version 665793 (0.0005) [2023-12-27 10:14:08,633][20347] Updated weights for policy 0, policy_version 665803 (0.0006) [2023-12-27 10:14:09,049][20381] Updated weights for policy 1, policy_version 665177 (0.0008) [2023-12-27 10:14:09,105][20381] Updated weights for policy 1, policy_version 665187 (0.0005) [2023-12-27 10:14:09,163][20381] Updated weights for policy 1, policy_version 665197 (0.0009) [2023-12-27 10:14:09,217][20381] Updated weights for policy 1, policy_version 665207 (0.0009) [2023-12-27 10:14:09,240][20347] Updated weights for policy 0, policy_version 665813 (0.0011) [2023-12-27 10:14:09,288][20291] KL-divergence is very high: 108.3971 [2023-12-27 10:14:09,299][20347] Updated weights for policy 0, policy_version 665823 (0.0010) [2023-12-27 10:14:09,358][20347] Updated weights for policy 0, policy_version 665833 (0.0011) [2023-12-27 10:14:09,916][20381] Updated weights for policy 1, policy_version 665217 (0.0010) [2023-12-27 10:14:09,977][20381] Updated weights for policy 1, policy_version 665227 (0.0010) [2023-12-27 10:14:10,043][20381] Updated weights for policy 1, policy_version 665237 (0.0010) [2023-12-27 10:14:10,112][20347] Updated weights for policy 0, policy_version 665843 (0.0007) [2023-12-27 10:14:10,171][20347] Updated weights for policy 0, policy_version 665853 (0.0006) [2023-12-27 10:14:10,227][20347] Updated weights for policy 0, policy_version 665863 (0.0007) [2023-12-27 10:14:10,693][20381] Updated weights for policy 1, policy_version 665247 (0.0011) [2023-12-27 10:14:10,748][20381] Updated weights for policy 1, policy_version 665257 (0.0011) [2023-12-27 10:14:10,804][20381] Updated weights for policy 1, policy_version 665267 (0.0010) [2023-12-27 10:14:10,912][20347] Updated weights for policy 0, policy_version 665873 (0.0008) [2023-12-27 10:14:10,970][20347] Updated weights for policy 0, policy_version 665883 (0.0008) [2023-12-27 10:14:11,042][20347] Updated weights for policy 0, policy_version 665893 (0.0008) [2023-12-27 10:14:11,097][20347] Updated weights for policy 0, policy_version 665903 (0.0007) [2023-12-27 10:14:11,506][20381] Updated weights for policy 1, policy_version 665277 (0.0010) [2023-12-27 10:14:11,565][20381] Updated weights for policy 1, policy_version 665287 (0.0010) [2023-12-27 10:14:11,626][20381] Updated weights for policy 1, policy_version 665297 (0.0010) [2023-12-27 10:14:11,786][20347] Updated weights for policy 0, policy_version 665913 (0.0008) [2023-12-27 10:14:11,842][20347] Updated weights for policy 0, policy_version 665923 (0.0007) [2023-12-27 10:14:11,891][20347] Updated weights for policy 0, policy_version 665933 (0.0006) [2023-12-27 10:14:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 340844544. Throughput: 0: 10314.4, 1: 10256.8. Samples: 340838396. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:12,363][19170] Avg episode reward: [(0, '844.220'), (1, '1000.000')] [2023-12-27 10:14:12,366][20381] Updated weights for policy 1, policy_version 665307 (0.0008) [2023-12-27 10:14:12,426][20381] Updated weights for policy 1, policy_version 665317 (0.0008) [2023-12-27 10:14:12,476][20381] Updated weights for policy 1, policy_version 665327 (0.0006) [2023-12-27 10:14:12,497][20347] Updated weights for policy 0, policy_version 665943 (0.0007) [2023-12-27 10:14:12,563][20347] Updated weights for policy 0, policy_version 665953 (0.0006) [2023-12-27 10:14:12,620][20347] Updated weights for policy 0, policy_version 665963 (0.0007) [2023-12-27 10:14:13,133][20381] Updated weights for policy 1, policy_version 665337 (0.0006) [2023-12-27 10:14:13,184][20381] Updated weights for policy 1, policy_version 665347 (0.0007) [2023-12-27 10:14:13,243][20381] Updated weights for policy 1, policy_version 665357 (0.0006) [2023-12-27 10:14:13,299][20381] Updated weights for policy 1, policy_version 665367 (0.0007) [2023-12-27 10:14:13,309][20347] Updated weights for policy 0, policy_version 665973 (0.0008) [2023-12-27 10:14:13,367][20347] Updated weights for policy 0, policy_version 665983 (0.0010) [2023-12-27 10:14:13,424][20347] Updated weights for policy 0, policy_version 665993 (0.0010) [2023-12-27 10:14:13,871][20381] Updated weights for policy 1, policy_version 665377 (0.0005) [2023-12-27 10:14:13,917][20381] Updated weights for policy 1, policy_version 665387 (0.0007) [2023-12-27 10:14:13,961][20381] Updated weights for policy 1, policy_version 665397 (0.0008) [2023-12-27 10:14:14,096][20347] Updated weights for policy 0, policy_version 666003 (0.0010) [2023-12-27 10:14:14,153][20347] Updated weights for policy 0, policy_version 666013 (0.0010) [2023-12-27 10:14:14,212][20347] Updated weights for policy 0, policy_version 666023 (0.0011) [2023-12-27 10:14:14,634][20381] Updated weights for policy 1, policy_version 665407 (0.0008) [2023-12-27 10:14:14,680][20381] Updated weights for policy 1, policy_version 665417 (0.0007) [2023-12-27 10:14:14,741][20381] Updated weights for policy 1, policy_version 665427 (0.0005) [2023-12-27 10:14:14,863][20347] Updated weights for policy 0, policy_version 666033 (0.0009) [2023-12-27 10:14:14,934][20347] Updated weights for policy 0, policy_version 666043 (0.0009) [2023-12-27 10:14:14,988][20347] Updated weights for policy 0, policy_version 666053 (0.0007) [2023-12-27 10:14:15,047][20347] Updated weights for policy 0, policy_version 666063 (0.0007) [2023-12-27 10:14:15,474][20381] Updated weights for policy 1, policy_version 665437 (0.0006) [2023-12-27 10:14:15,531][20381] Updated weights for policy 1, policy_version 665447 (0.0008) [2023-12-27 10:14:15,594][20381] Updated weights for policy 1, policy_version 665457 (0.0005) [2023-12-27 10:14:15,718][20347] Updated weights for policy 0, policy_version 666073 (0.0010) [2023-12-27 10:14:15,763][20347] Updated weights for policy 0, policy_version 666083 (0.0010) [2023-12-27 10:14:15,812][20347] Updated weights for policy 0, policy_version 666093 (0.0010) [2023-12-27 10:14:16,130][20381] Updated weights for policy 1, policy_version 665467 (0.0006) [2023-12-27 10:14:16,185][20381] Updated weights for policy 1, policy_version 665477 (0.0006) [2023-12-27 10:14:16,232][20381] Updated weights for policy 1, policy_version 665487 (0.0007) [2023-12-27 10:14:16,527][20347] Updated weights for policy 0, policy_version 666103 (0.0009) [2023-12-27 10:14:16,577][20347] Updated weights for policy 0, policy_version 666113 (0.0007) [2023-12-27 10:14:16,631][20347] Updated weights for policy 0, policy_version 666123 (0.0006) [2023-12-27 10:14:16,839][20381] Updated weights for policy 1, policy_version 665497 (0.0007) [2023-12-27 10:14:16,896][20381] Updated weights for policy 1, policy_version 665507 (0.0006) [2023-12-27 10:14:16,946][20381] Updated weights for policy 1, policy_version 665517 (0.0011) [2023-12-27 10:14:16,990][20381] Updated weights for policy 1, policy_version 665527 (0.0009) [2023-12-27 10:14:17,284][20347] Updated weights for policy 0, policy_version 666133 (0.0006) [2023-12-27 10:14:17,343][20347] Updated weights for policy 0, policy_version 666143 (0.0005) [2023-12-27 10:14:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 340951040. Throughput: 0: 10267.6, 1: 10344.1. Samples: 340964888. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:17,364][19170] Avg episode reward: [(0, '795.220'), (1, '1000.000')] [2023-12-27 10:14:17,398][20347] Updated weights for policy 0, policy_version 666153 (0.0006) [2023-12-27 10:14:17,600][20381] Updated weights for policy 1, policy_version 665537 (0.0009) [2023-12-27 10:14:17,663][20381] Updated weights for policy 1, policy_version 665547 (0.0010) [2023-12-27 10:14:17,720][20381] Updated weights for policy 1, policy_version 665557 (0.0010) [2023-12-27 10:14:17,978][20347] Updated weights for policy 0, policy_version 666163 (0.0008) [2023-12-27 10:14:18,032][20347] Updated weights for policy 0, policy_version 666173 (0.0010) [2023-12-27 10:14:18,077][20347] Updated weights for policy 0, policy_version 666183 (0.0010) [2023-12-27 10:14:18,318][20381] Updated weights for policy 1, policy_version 665567 (0.0006) [2023-12-27 10:14:18,378][20381] Updated weights for policy 1, policy_version 665577 (0.0007) [2023-12-27 10:14:18,427][20381] Updated weights for policy 1, policy_version 665587 (0.0008) [2023-12-27 10:14:18,761][20347] Updated weights for policy 0, policy_version 666193 (0.0010) [2023-12-27 10:14:18,826][20347] Updated weights for policy 0, policy_version 666203 (0.0010) [2023-12-27 10:14:18,891][20347] Updated weights for policy 0, policy_version 666213 (0.0011) [2023-12-27 10:14:18,952][20347] Updated weights for policy 0, policy_version 666223 (0.0010) [2023-12-27 10:14:19,125][20381] Updated weights for policy 1, policy_version 665597 (0.0009) [2023-12-27 10:14:19,186][20381] Updated weights for policy 1, policy_version 665607 (0.0008) [2023-12-27 10:14:19,242][20381] Updated weights for policy 1, policy_version 665617 (0.0010) [2023-12-27 10:14:19,667][20347] Updated weights for policy 0, policy_version 666233 (0.0008) [2023-12-27 10:14:19,728][20347] Updated weights for policy 0, policy_version 666243 (0.0007) [2023-12-27 10:14:19,783][20347] Updated weights for policy 0, policy_version 666253 (0.0007) [2023-12-27 10:14:19,984][20381] Updated weights for policy 1, policy_version 665627 (0.0008) [2023-12-27 10:14:20,042][20381] Updated weights for policy 1, policy_version 665637 (0.0008) [2023-12-27 10:14:20,092][20381] Updated weights for policy 1, policy_version 665647 (0.0007) [2023-12-27 10:14:20,495][20347] Updated weights for policy 0, policy_version 666263 (0.0007) [2023-12-27 10:14:20,543][20347] Updated weights for policy 0, policy_version 666273 (0.0006) [2023-12-27 10:14:20,602][20347] Updated weights for policy 0, policy_version 666283 (0.0007) [2023-12-27 10:14:20,830][20381] Updated weights for policy 1, policy_version 665657 (0.0008) [2023-12-27 10:14:20,894][20381] Updated weights for policy 1, policy_version 665667 (0.0008) [2023-12-27 10:14:20,948][20381] Updated weights for policy 1, policy_version 665677 (0.0007) [2023-12-27 10:14:21,000][20381] Updated weights for policy 1, policy_version 665687 (0.0008) [2023-12-27 10:14:21,331][20347] Updated weights for policy 0, policy_version 666293 (0.0008) [2023-12-27 10:14:21,396][20347] Updated weights for policy 0, policy_version 666303 (0.0007) [2023-12-27 10:14:21,456][20347] Updated weights for policy 0, policy_version 666313 (0.0007) [2023-12-27 10:14:21,728][20381] Updated weights for policy 1, policy_version 665697 (0.0008) [2023-12-27 10:14:21,781][20381] Updated weights for policy 1, policy_version 665707 (0.0008) [2023-12-27 10:14:21,847][20381] Updated weights for policy 1, policy_version 665717 (0.0008) [2023-12-27 10:14:22,144][20347] Updated weights for policy 0, policy_version 666323 (0.0008) [2023-12-27 10:14:22,196][20347] Updated weights for policy 0, policy_version 666333 (0.0010) [2023-12-27 10:14:22,245][20347] Updated weights for policy 0, policy_version 666343 (0.0010) [2023-12-27 10:14:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 341057536. Throughput: 0: 10218.1, 1: 10392.7. Samples: 341026836. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:22,363][19170] Avg episode reward: [(0, '882.890'), (1, '1000.000')] [2023-12-27 10:14:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000665720_170450944.pth... [2023-12-27 10:14:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000666352_170606592.pth... [2023-12-27 10:14:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000664504_170139648.pth [2023-12-27 10:14:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000665136_170295296.pth [2023-12-27 10:14:22,593][20381] Updated weights for policy 1, policy_version 665727 (0.0008) [2023-12-27 10:14:22,649][20381] Updated weights for policy 1, policy_version 665737 (0.0007) [2023-12-27 10:14:22,712][20381] Updated weights for policy 1, policy_version 665747 (0.0008) [2023-12-27 10:14:22,897][20347] Updated weights for policy 0, policy_version 666353 (0.0006) [2023-12-27 10:14:22,956][20347] Updated weights for policy 0, policy_version 666363 (0.0007) [2023-12-27 10:14:23,017][20347] Updated weights for policy 0, policy_version 666373 (0.0009) [2023-12-27 10:14:23,083][20347] Updated weights for policy 0, policy_version 666383 (0.0010) [2023-12-27 10:14:23,337][20381] Updated weights for policy 1, policy_version 665757 (0.0007) [2023-12-27 10:14:23,390][20381] Updated weights for policy 1, policy_version 665767 (0.0006) [2023-12-27 10:14:23,448][20381] Updated weights for policy 1, policy_version 665777 (0.0006) [2023-12-27 10:14:23,690][20347] Updated weights for policy 0, policy_version 666393 (0.0006) [2023-12-27 10:14:23,745][20347] Updated weights for policy 0, policy_version 666403 (0.0005) [2023-12-27 10:14:23,801][20347] Updated weights for policy 0, policy_version 666413 (0.0010) [2023-12-27 10:14:24,080][20381] Updated weights for policy 1, policy_version 665787 (0.0006) [2023-12-27 10:14:24,134][20381] Updated weights for policy 1, policy_version 665797 (0.0007) [2023-12-27 10:14:24,181][20381] Updated weights for policy 1, policy_version 665807 (0.0007) [2023-12-27 10:14:24,422][20347] Updated weights for policy 0, policy_version 666423 (0.0010) [2023-12-27 10:14:24,479][20347] Updated weights for policy 0, policy_version 666433 (0.0010) [2023-12-27 10:14:24,535][20347] Updated weights for policy 0, policy_version 666443 (0.0010) [2023-12-27 10:14:24,857][20381] Updated weights for policy 1, policy_version 665817 (0.0008) [2023-12-27 10:14:24,911][20381] Updated weights for policy 1, policy_version 665827 (0.0007) [2023-12-27 10:14:24,968][20381] Updated weights for policy 1, policy_version 665837 (0.0007) [2023-12-27 10:14:25,023][20381] Updated weights for policy 1, policy_version 665847 (0.0007) [2023-12-27 10:14:25,198][20347] Updated weights for policy 0, policy_version 666453 (0.0010) [2023-12-27 10:14:25,246][20347] Updated weights for policy 0, policy_version 666463 (0.0010) [2023-12-27 10:14:25,308][20347] Updated weights for policy 0, policy_version 666473 (0.0011) [2023-12-27 10:14:25,714][20381] Updated weights for policy 1, policy_version 665857 (0.0007) [2023-12-27 10:14:25,771][20381] Updated weights for policy 1, policy_version 665867 (0.0007) [2023-12-27 10:14:25,831][20381] Updated weights for policy 1, policy_version 665877 (0.0007) [2023-12-27 10:14:25,991][20347] Updated weights for policy 0, policy_version 666483 (0.0010) [2023-12-27 10:14:26,042][20347] Updated weights for policy 0, policy_version 666493 (0.0010) [2023-12-27 10:14:26,094][20347] Updated weights for policy 0, policy_version 666503 (0.0010) [2023-12-27 10:14:26,519][20381] Updated weights for policy 1, policy_version 665887 (0.0007) [2023-12-27 10:14:26,570][20381] Updated weights for policy 1, policy_version 665897 (0.0008) [2023-12-27 10:14:26,621][20381] Updated weights for policy 1, policy_version 665907 (0.0009) [2023-12-27 10:14:26,723][20347] Updated weights for policy 0, policy_version 666513 (0.0010) [2023-12-27 10:14:26,788][20347] Updated weights for policy 0, policy_version 666523 (0.0005) [2023-12-27 10:14:26,843][20347] Updated weights for policy 0, policy_version 666533 (0.0005) [2023-12-27 10:14:26,889][20347] Updated weights for policy 0, policy_version 666543 (0.0005) [2023-12-27 10:14:27,293][20381] Updated weights for policy 1, policy_version 665917 (0.0007) [2023-12-27 10:14:27,360][20381] Updated weights for policy 1, policy_version 665927 (0.0008) [2023-12-27 10:14:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20827.1). Total num frames: 341155840. Throughput: 0: 10302.1, 1: 10310.2. Samples: 341149948. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:27,363][19170] Avg episode reward: [(0, '902.390'), (1, '990.320')] [2023-12-27 10:14:27,415][20381] Updated weights for policy 1, policy_version 665937 (0.0007) [2023-12-27 10:14:27,424][20347] Updated weights for policy 0, policy_version 666553 (0.0005) [2023-12-27 10:14:27,470][20347] Updated weights for policy 0, policy_version 666563 (0.0007) [2023-12-27 10:14:27,518][20347] Updated weights for policy 0, policy_version 666573 (0.0007) [2023-12-27 10:14:28,082][20381] Updated weights for policy 1, policy_version 665947 (0.0007) [2023-12-27 10:14:28,130][20381] Updated weights for policy 1, policy_version 665957 (0.0007) [2023-12-27 10:14:28,173][20381] Updated weights for policy 1, policy_version 665967 (0.0007) [2023-12-27 10:14:28,193][20347] Updated weights for policy 0, policy_version 666583 (0.0006) [2023-12-27 10:14:28,239][20347] Updated weights for policy 0, policy_version 666593 (0.0006) [2023-12-27 10:14:28,282][20347] Updated weights for policy 0, policy_version 666603 (0.0007) [2023-12-27 10:14:28,894][20381] Updated weights for policy 1, policy_version 665977 (0.0005) [2023-12-27 10:14:28,922][20347] Updated weights for policy 0, policy_version 666613 (0.0006) [2023-12-27 10:14:28,945][20381] Updated weights for policy 1, policy_version 665987 (0.0007) [2023-12-27 10:14:28,970][20347] Updated weights for policy 0, policy_version 666623 (0.0005) [2023-12-27 10:14:29,003][20381] Updated weights for policy 1, policy_version 665997 (0.0005) [2023-12-27 10:14:29,019][20347] Updated weights for policy 0, policy_version 666633 (0.0005) [2023-12-27 10:14:29,058][20381] Updated weights for policy 1, policy_version 666007 (0.0006) [2023-12-27 10:14:29,705][20347] Updated weights for policy 0, policy_version 666643 (0.0006) [2023-12-27 10:14:29,708][20381] Updated weights for policy 1, policy_version 666017 (0.0010) [2023-12-27 10:14:29,756][20381] Updated weights for policy 1, policy_version 666027 (0.0010) [2023-12-27 10:14:29,761][20347] Updated weights for policy 0, policy_version 666653 (0.0007) [2023-12-27 10:14:29,815][20381] Updated weights for policy 1, policy_version 666037 (0.0008) [2023-12-27 10:14:29,820][20347] Updated weights for policy 0, policy_version 666663 (0.0007) [2023-12-27 10:14:30,477][20381] Updated weights for policy 1, policy_version 666047 (0.0009) [2023-12-27 10:14:30,502][20347] Updated weights for policy 0, policy_version 666673 (0.0008) [2023-12-27 10:14:30,535][20381] Updated weights for policy 1, policy_version 666057 (0.0008) [2023-12-27 10:14:30,560][20347] Updated weights for policy 0, policy_version 666683 (0.0006) [2023-12-27 10:14:30,594][20381] Updated weights for policy 1, policy_version 666067 (0.0006) [2023-12-27 10:14:30,618][20347] Updated weights for policy 0, policy_version 666693 (0.0006) [2023-12-27 10:14:30,671][20347] Updated weights for policy 0, policy_version 666703 (0.0007) [2023-12-27 10:14:31,280][20381] Updated weights for policy 1, policy_version 666077 (0.0006) [2023-12-27 10:14:31,303][20347] Updated weights for policy 0, policy_version 666713 (0.0007) [2023-12-27 10:14:31,335][20381] Updated weights for policy 1, policy_version 666087 (0.0007) [2023-12-27 10:14:31,362][20347] Updated weights for policy 0, policy_version 666723 (0.0008) [2023-12-27 10:14:31,396][20381] Updated weights for policy 1, policy_version 666097 (0.0008) [2023-12-27 10:14:31,425][20347] Updated weights for policy 0, policy_version 666733 (0.0008) [2023-12-27 10:14:32,017][20381] Updated weights for policy 1, policy_version 666107 (0.0006) [2023-12-27 10:14:32,079][20381] Updated weights for policy 1, policy_version 666117 (0.0005) [2023-12-27 10:14:32,106][20347] Updated weights for policy 0, policy_version 666743 (0.0007) [2023-12-27 10:14:32,145][20381] Updated weights for policy 1, policy_version 666127 (0.0006) [2023-12-27 10:14:32,158][20347] Updated weights for policy 0, policy_version 666753 (0.0006) [2023-12-27 10:14:32,207][20347] Updated weights for policy 0, policy_version 666763 (0.0006) [2023-12-27 10:14:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 341270528. Throughput: 0: 10348.4, 1: 10335.7. Samples: 341276232. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:32,363][19170] Avg episode reward: [(0, '844.430'), (1, '1000.000')] [2023-12-27 10:14:32,764][20381] Updated weights for policy 1, policy_version 666137 (0.0006) [2023-12-27 10:14:32,820][20381] Updated weights for policy 1, policy_version 666147 (0.0005) [2023-12-27 10:14:32,868][20381] Updated weights for policy 1, policy_version 666157 (0.0005) [2023-12-27 10:14:32,901][20347] Updated weights for policy 0, policy_version 666773 (0.0008) [2023-12-27 10:14:32,926][20381] Updated weights for policy 1, policy_version 666167 (0.0005) [2023-12-27 10:14:32,956][20347] Updated weights for policy 0, policy_version 666783 (0.0008) [2023-12-27 10:14:33,003][20347] Updated weights for policy 0, policy_version 666793 (0.0007) [2023-12-27 10:14:33,521][20381] Updated weights for policy 1, policy_version 666177 (0.0010) [2023-12-27 10:14:33,581][20381] Updated weights for policy 1, policy_version 666187 (0.0010) [2023-12-27 10:14:33,638][20381] Updated weights for policy 1, policy_version 666197 (0.0010) [2023-12-27 10:14:33,714][20347] Updated weights for policy 0, policy_version 666803 (0.0007) [2023-12-27 10:14:33,769][20347] Updated weights for policy 0, policy_version 666813 (0.0005) [2023-12-27 10:14:33,821][20347] Updated weights for policy 0, policy_version 666823 (0.0008) [2023-12-27 10:14:34,278][20381] Updated weights for policy 1, policy_version 666207 (0.0010) [2023-12-27 10:14:34,342][20381] Updated weights for policy 1, policy_version 666217 (0.0010) [2023-12-27 10:14:34,391][20381] Updated weights for policy 1, policy_version 666227 (0.0010) [2023-12-27 10:14:34,507][20347] Updated weights for policy 0, policy_version 666833 (0.0008) [2023-12-27 10:14:34,568][20347] Updated weights for policy 0, policy_version 666843 (0.0007) [2023-12-27 10:14:34,628][20347] Updated weights for policy 0, policy_version 666853 (0.0007) [2023-12-27 10:14:34,676][20347] Updated weights for policy 0, policy_version 666863 (0.0007) [2023-12-27 10:14:35,094][20381] Updated weights for policy 1, policy_version 666237 (0.0010) [2023-12-27 10:14:35,138][20381] Updated weights for policy 1, policy_version 666247 (0.0010) [2023-12-27 10:14:35,185][20381] Updated weights for policy 1, policy_version 666257 (0.0010) [2023-12-27 10:14:35,324][20347] Updated weights for policy 0, policy_version 666873 (0.0005) [2023-12-27 10:14:35,371][20347] Updated weights for policy 0, policy_version 666883 (0.0005) [2023-12-27 10:14:35,422][20347] Updated weights for policy 0, policy_version 666893 (0.0005) [2023-12-27 10:14:35,876][20381] Updated weights for policy 1, policy_version 666267 (0.0010) [2023-12-27 10:14:35,931][20381] Updated weights for policy 1, policy_version 666277 (0.0010) [2023-12-27 10:14:35,975][20347] Updated weights for policy 0, policy_version 666903 (0.0005) [2023-12-27 10:14:35,984][20381] Updated weights for policy 1, policy_version 666287 (0.0010) [2023-12-27 10:14:36,029][20347] Updated weights for policy 0, policy_version 666913 (0.0005) [2023-12-27 10:14:36,076][20347] Updated weights for policy 0, policy_version 666923 (0.0010) [2023-12-27 10:14:36,621][20381] Updated weights for policy 1, policy_version 666297 (0.0010) [2023-12-27 10:14:36,688][20381] Updated weights for policy 1, policy_version 666307 (0.0008) [2023-12-27 10:14:36,751][20381] Updated weights for policy 1, policy_version 666317 (0.0011) [2023-12-27 10:14:36,762][20347] Updated weights for policy 0, policy_version 666933 (0.0009) [2023-12-27 10:14:36,809][20381] Updated weights for policy 1, policy_version 666327 (0.0008) [2023-12-27 10:14:36,818][20347] Updated weights for policy 0, policy_version 666943 (0.0010) [2023-12-27 10:14:36,874][20347] Updated weights for policy 0, policy_version 666953 (0.0007) [2023-12-27 10:14:37,363][19170] Fps is (10 sec: 21298.2, 60 sec: 20753.0, 300 sec: 20854.9). Total num frames: 341368832. Throughput: 0: 10318.9, 1: 10384.6. Samples: 341338324. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:37,364][19170] Avg episode reward: [(0, '844.780'), (1, '990.300')] [2023-12-27 10:14:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000666328_170606592.pth... [2023-12-27 10:14:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000666960_170762240.pth... [2023-12-27 10:14:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000665744_170450944.pth [2023-12-27 10:14:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000665112_170295296.pth [2023-12-27 10:14:37,470][20381] Updated weights for policy 1, policy_version 666337 (0.0010) [2023-12-27 10:14:37,526][20381] Updated weights for policy 1, policy_version 666347 (0.0011) [2023-12-27 10:14:37,556][20347] Updated weights for policy 0, policy_version 666963 (0.0007) [2023-12-27 10:14:37,581][20381] Updated weights for policy 1, policy_version 666357 (0.0009) [2023-12-27 10:14:37,606][20347] Updated weights for policy 0, policy_version 666973 (0.0009) [2023-12-27 10:14:37,650][20347] Updated weights for policy 0, policy_version 666983 (0.0010) [2023-12-27 10:14:38,276][20347] Updated weights for policy 0, policy_version 666993 (0.0009) [2023-12-27 10:14:38,276][20381] Updated weights for policy 1, policy_version 666367 (0.0006) [2023-12-27 10:14:38,285][20292] KL-divergence is very high: 148.7622 [2023-12-27 10:14:38,337][20381] Updated weights for policy 1, policy_version 666377 (0.0007) [2023-12-27 10:14:38,343][20347] Updated weights for policy 0, policy_version 667003 (0.0006) [2023-12-27 10:14:38,394][20381] Updated weights for policy 1, policy_version 666387 (0.0007) [2023-12-27 10:14:38,411][20347] Updated weights for policy 0, policy_version 667013 (0.0006) [2023-12-27 10:14:38,467][20347] Updated weights for policy 0, policy_version 667023 (0.0010) [2023-12-27 10:14:38,944][20381] Updated weights for policy 1, policy_version 666397 (0.0006) [2023-12-27 10:14:39,003][20381] Updated weights for policy 1, policy_version 666407 (0.0006) [2023-12-27 10:14:39,064][20381] Updated weights for policy 1, policy_version 666417 (0.0007) [2023-12-27 10:14:39,119][20347] Updated weights for policy 0, policy_version 667033 (0.0011) [2023-12-27 10:14:39,174][20347] Updated weights for policy 0, policy_version 667043 (0.0010) [2023-12-27 10:14:39,229][20347] Updated weights for policy 0, policy_version 667053 (0.0010) [2023-12-27 10:14:39,721][20381] Updated weights for policy 1, policy_version 666427 (0.0008) [2023-12-27 10:14:39,782][20381] Updated weights for policy 1, policy_version 666437 (0.0007) [2023-12-27 10:14:39,850][20381] Updated weights for policy 1, policy_version 666447 (0.0008) [2023-12-27 10:14:39,983][20347] Updated weights for policy 0, policy_version 667063 (0.0010) [2023-12-27 10:14:40,033][20347] Updated weights for policy 0, policy_version 667073 (0.0010) [2023-12-27 10:14:40,082][20347] Updated weights for policy 0, policy_version 667083 (0.0010) [2023-12-27 10:14:40,516][20381] Updated weights for policy 1, policy_version 666457 (0.0008) [2023-12-27 10:14:40,571][20381] Updated weights for policy 1, policy_version 666467 (0.0007) [2023-12-27 10:14:40,631][20381] Updated weights for policy 1, policy_version 666477 (0.0008) [2023-12-27 10:14:40,688][20381] Updated weights for policy 1, policy_version 666487 (0.0005) [2023-12-27 10:14:40,790][20347] Updated weights for policy 0, policy_version 667093 (0.0011) [2023-12-27 10:14:40,847][20347] Updated weights for policy 0, policy_version 667103 (0.0011) [2023-12-27 10:14:40,899][20347] Updated weights for policy 0, policy_version 667113 (0.0010) [2023-12-27 10:14:41,324][20381] Updated weights for policy 1, policy_version 666497 (0.0007) [2023-12-27 10:14:41,392][20381] Updated weights for policy 1, policy_version 666507 (0.0008) [2023-12-27 10:14:41,447][20381] Updated weights for policy 1, policy_version 666517 (0.0007) [2023-12-27 10:14:41,668][20347] Updated weights for policy 0, policy_version 667123 (0.0010) [2023-12-27 10:14:41,735][20347] Updated weights for policy 0, policy_version 667133 (0.0008) [2023-12-27 10:14:41,787][20347] Updated weights for policy 0, policy_version 667143 (0.0009) [2023-12-27 10:14:42,184][20381] Updated weights for policy 1, policy_version 666527 (0.0008) [2023-12-27 10:14:42,238][20381] Updated weights for policy 1, policy_version 666537 (0.0007) [2023-12-27 10:14:42,294][20381] Updated weights for policy 1, policy_version 666547 (0.0007) [2023-12-27 10:14:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20854.9). Total num frames: 341475328. Throughput: 0: 10352.8, 1: 10423.2. Samples: 341463704. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:42,363][19170] Avg episode reward: [(0, '864.290'), (1, '972.550')] [2023-12-27 10:14:42,512][20347] Updated weights for policy 0, policy_version 667153 (0.0009) [2023-12-27 10:14:42,564][20347] Updated weights for policy 0, policy_version 667163 (0.0010) [2023-12-27 10:14:42,614][20347] Updated weights for policy 0, policy_version 667173 (0.0010) [2023-12-27 10:14:42,667][20347] Updated weights for policy 0, policy_version 667183 (0.0010) [2023-12-27 10:14:43,003][20381] Updated weights for policy 1, policy_version 666557 (0.0008) [2023-12-27 10:14:43,059][20381] Updated weights for policy 1, policy_version 666567 (0.0006) [2023-12-27 10:14:43,117][20381] Updated weights for policy 1, policy_version 666577 (0.0005) [2023-12-27 10:14:43,350][20347] Updated weights for policy 0, policy_version 667193 (0.0006) [2023-12-27 10:14:43,405][20347] Updated weights for policy 0, policy_version 667203 (0.0005) [2023-12-27 10:14:43,458][20347] Updated weights for policy 0, policy_version 667213 (0.0007) [2023-12-27 10:14:43,734][20381] Updated weights for policy 1, policy_version 666587 (0.0005) [2023-12-27 10:14:43,786][20381] Updated weights for policy 1, policy_version 666597 (0.0005) [2023-12-27 10:14:43,830][20381] Updated weights for policy 1, policy_version 666607 (0.0005) [2023-12-27 10:14:44,117][20347] Updated weights for policy 0, policy_version 667223 (0.0006) [2023-12-27 10:14:44,178][20347] Updated weights for policy 0, policy_version 667233 (0.0005) [2023-12-27 10:14:44,239][20347] Updated weights for policy 0, policy_version 667243 (0.0009) [2023-12-27 10:14:44,366][20381] Updated weights for policy 1, policy_version 666617 (0.0005) [2023-12-27 10:14:44,413][20381] Updated weights for policy 1, policy_version 666627 (0.0006) [2023-12-27 10:14:44,464][20381] Updated weights for policy 1, policy_version 666637 (0.0007) [2023-12-27 10:14:44,508][20381] Updated weights for policy 1, policy_version 666647 (0.0005) [2023-12-27 10:14:44,837][20347] Updated weights for policy 0, policy_version 667253 (0.0010) [2023-12-27 10:14:44,889][20347] Updated weights for policy 0, policy_version 667263 (0.0010) [2023-12-27 10:14:44,948][20347] Updated weights for policy 0, policy_version 667273 (0.0010) [2023-12-27 10:14:45,185][20381] Updated weights for policy 1, policy_version 666657 (0.0007) [2023-12-27 10:14:45,249][20381] Updated weights for policy 1, policy_version 666667 (0.0008) [2023-12-27 10:14:45,310][20381] Updated weights for policy 1, policy_version 666677 (0.0008) [2023-12-27 10:14:45,592][20347] Updated weights for policy 0, policy_version 667283 (0.0009) [2023-12-27 10:14:45,650][20347] Updated weights for policy 0, policy_version 667293 (0.0007) [2023-12-27 10:14:45,701][20347] Updated weights for policy 0, policy_version 667303 (0.0008) [2023-12-27 10:14:46,006][20381] Updated weights for policy 1, policy_version 666687 (0.0008) [2023-12-27 10:14:46,049][20381] Updated weights for policy 1, policy_version 666697 (0.0008) [2023-12-27 10:14:46,101][20381] Updated weights for policy 1, policy_version 666707 (0.0005) [2023-12-27 10:14:46,390][20347] Updated weights for policy 0, policy_version 667313 (0.0007) [2023-12-27 10:14:46,444][20347] Updated weights for policy 0, policy_version 667323 (0.0009) [2023-12-27 10:14:46,495][20347] Updated weights for policy 0, policy_version 667333 (0.0008) [2023-12-27 10:14:46,550][20347] Updated weights for policy 0, policy_version 667343 (0.0008) [2023-12-27 10:14:46,768][20381] Updated weights for policy 1, policy_version 666717 (0.0008) [2023-12-27 10:14:46,816][20381] Updated weights for policy 1, policy_version 666727 (0.0007) [2023-12-27 10:14:46,862][20381] Updated weights for policy 1, policy_version 666737 (0.0008) [2023-12-27 10:14:47,264][20347] Updated weights for policy 0, policy_version 667353 (0.0007) [2023-12-27 10:14:47,322][20347] Updated weights for policy 0, policy_version 667363 (0.0007) [2023-12-27 10:14:47,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20827.1). Total num frames: 341573632. Throughput: 0: 10352.2, 1: 10407.7. Samples: 341587428. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:47,363][19170] Avg episode reward: [(0, '931.940'), (1, '953.360')] [2023-12-27 10:14:47,377][20347] Updated weights for policy 0, policy_version 667373 (0.0007) [2023-12-27 10:14:47,583][20381] Updated weights for policy 1, policy_version 666747 (0.0007) [2023-12-27 10:14:47,639][20381] Updated weights for policy 1, policy_version 666757 (0.0005) [2023-12-27 10:14:47,689][20381] Updated weights for policy 1, policy_version 666767 (0.0007) [2023-12-27 10:14:48,068][20347] Updated weights for policy 0, policy_version 667383 (0.0008) [2023-12-27 10:14:48,122][20347] Updated weights for policy 0, policy_version 667393 (0.0007) [2023-12-27 10:14:48,179][20347] Updated weights for policy 0, policy_version 667403 (0.0008) [2023-12-27 10:14:48,335][20381] Updated weights for policy 1, policy_version 666777 (0.0006) [2023-12-27 10:14:48,400][20381] Updated weights for policy 1, policy_version 666787 (0.0005) [2023-12-27 10:14:48,445][20381] Updated weights for policy 1, policy_version 666797 (0.0006) [2023-12-27 10:14:48,498][20381] Updated weights for policy 1, policy_version 666807 (0.0008) [2023-12-27 10:14:48,914][20347] Updated weights for policy 0, policy_version 667413 (0.0008) [2023-12-27 10:14:48,965][20347] Updated weights for policy 0, policy_version 667423 (0.0007) [2023-12-27 10:14:49,020][20347] Updated weights for policy 0, policy_version 667433 (0.0007) [2023-12-27 10:14:49,141][20381] Updated weights for policy 1, policy_version 666817 (0.0006) [2023-12-27 10:14:49,202][20381] Updated weights for policy 1, policy_version 666827 (0.0009) [2023-12-27 10:14:49,266][20381] Updated weights for policy 1, policy_version 666837 (0.0008) [2023-12-27 10:14:49,743][20347] Updated weights for policy 0, policy_version 667443 (0.0008) [2023-12-27 10:14:49,799][20347] Updated weights for policy 0, policy_version 667453 (0.0008) [2023-12-27 10:14:49,862][20347] Updated weights for policy 0, policy_version 667463 (0.0008) [2023-12-27 10:14:49,909][20381] Updated weights for policy 1, policy_version 666847 (0.0006) [2023-12-27 10:14:49,966][20381] Updated weights for policy 1, policy_version 666857 (0.0007) [2023-12-27 10:14:50,030][20381] Updated weights for policy 1, policy_version 666867 (0.0007) [2023-12-27 10:14:50,489][20347] Updated weights for policy 0, policy_version 667473 (0.0007) [2023-12-27 10:14:50,541][20347] Updated weights for policy 0, policy_version 667483 (0.0007) [2023-12-27 10:14:50,602][20347] Updated weights for policy 0, policy_version 667493 (0.0008) [2023-12-27 10:14:50,655][20347] Updated weights for policy 0, policy_version 667503 (0.0008) [2023-12-27 10:14:50,731][20381] Updated weights for policy 1, policy_version 666877 (0.0007) [2023-12-27 10:14:50,790][20381] Updated weights for policy 1, policy_version 666887 (0.0008) [2023-12-27 10:14:50,843][20381] Updated weights for policy 1, policy_version 666897 (0.0007) [2023-12-27 10:14:51,353][20347] Updated weights for policy 0, policy_version 667513 (0.0008) [2023-12-27 10:14:51,415][20347] Updated weights for policy 0, policy_version 667523 (0.0007) [2023-12-27 10:14:51,480][20347] Updated weights for policy 0, policy_version 667533 (0.0007) [2023-12-27 10:14:51,520][20381] Updated weights for policy 1, policy_version 666907 (0.0007) [2023-12-27 10:14:51,578][20381] Updated weights for policy 1, policy_version 666917 (0.0008) [2023-12-27 10:14:51,643][20381] Updated weights for policy 1, policy_version 666927 (0.0008) [2023-12-27 10:14:52,202][20347] Updated weights for policy 0, policy_version 667543 (0.0007) [2023-12-27 10:14:52,263][20347] Updated weights for policy 0, policy_version 667553 (0.0006) [2023-12-27 10:14:52,318][20347] Updated weights for policy 0, policy_version 667563 (0.0007) [2023-12-27 10:14:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20827.1). Total num frames: 341680128. Throughput: 0: 10300.3, 1: 10405.8. Samples: 341647580. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:52,363][19170] Avg episode reward: [(0, '883.610'), (1, '961.430')] [2023-12-27 10:14:52,364][20381] Updated weights for policy 1, policy_version 666937 (0.0007) [2023-12-27 10:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000667568_170917888.pth... [2023-12-27 10:14:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000666352_170606592.pth [2023-12-27 10:14:52,426][20381] Updated weights for policy 1, policy_version 666947 (0.0008) [2023-12-27 10:14:52,483][20381] Updated weights for policy 1, policy_version 666957 (0.0008) [2023-12-27 10:14:52,546][20381] Updated weights for policy 1, policy_version 666967 (0.0007) [2023-12-27 10:14:52,549][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000666968_170770432.pth... [2023-12-27 10:14:52,552][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000665720_170450944.pth [2023-12-27 10:14:53,025][20347] Updated weights for policy 0, policy_version 667573 (0.0007) [2023-12-27 10:14:53,080][20347] Updated weights for policy 0, policy_version 667583 (0.0005) [2023-12-27 10:14:53,140][20347] Updated weights for policy 0, policy_version 667593 (0.0005) [2023-12-27 10:14:53,201][20381] Updated weights for policy 1, policy_version 666977 (0.0007) [2023-12-27 10:14:53,256][20381] Updated weights for policy 1, policy_version 666987 (0.0005) [2023-12-27 10:14:53,311][20381] Updated weights for policy 1, policy_version 666997 (0.0005) [2023-12-27 10:14:53,826][20347] Updated weights for policy 0, policy_version 667603 (0.0007) [2023-12-27 10:14:53,872][20347] Updated weights for policy 0, policy_version 667613 (0.0009) [2023-12-27 10:14:53,920][20381] Updated weights for policy 1, policy_version 667007 (0.0006) [2023-12-27 10:14:53,922][20347] Updated weights for policy 0, policy_version 667623 (0.0007) [2023-12-27 10:14:53,973][20381] Updated weights for policy 1, policy_version 667017 (0.0006) [2023-12-27 10:14:54,025][20381] Updated weights for policy 1, policy_version 667027 (0.0006) [2023-12-27 10:14:54,643][20381] Updated weights for policy 1, policy_version 667037 (0.0006) [2023-12-27 10:14:54,650][20347] Updated weights for policy 0, policy_version 667633 (0.0008) [2023-12-27 10:14:54,689][20381] Updated weights for policy 1, policy_version 667047 (0.0005) [2023-12-27 10:14:54,695][20347] Updated weights for policy 0, policy_version 667643 (0.0007) [2023-12-27 10:14:54,743][20347] Updated weights for policy 0, policy_version 667653 (0.0007) [2023-12-27 10:14:54,748][20381] Updated weights for policy 1, policy_version 667057 (0.0006) [2023-12-27 10:14:54,792][20347] Updated weights for policy 0, policy_version 667663 (0.0006) [2023-12-27 10:14:55,359][20381] Updated weights for policy 1, policy_version 667067 (0.0005) [2023-12-27 10:14:55,409][20381] Updated weights for policy 1, policy_version 667077 (0.0005) [2023-12-27 10:14:55,460][20381] Updated weights for policy 1, policy_version 667087 (0.0005) [2023-12-27 10:14:55,543][20347] Updated weights for policy 0, policy_version 667673 (0.0007) [2023-12-27 10:14:55,607][20347] Updated weights for policy 0, policy_version 667683 (0.0005) [2023-12-27 10:14:55,674][20347] Updated weights for policy 0, policy_version 667693 (0.0005) [2023-12-27 10:14:56,030][20381] Updated weights for policy 1, policy_version 667097 (0.0007) [2023-12-27 10:14:56,094][20381] Updated weights for policy 1, policy_version 667107 (0.0006) [2023-12-27 10:14:56,145][20381] Updated weights for policy 1, policy_version 667117 (0.0005) [2023-12-27 10:14:56,198][20381] Updated weights for policy 1, policy_version 667127 (0.0006) [2023-12-27 10:14:56,292][20347] Updated weights for policy 0, policy_version 667703 (0.0006) [2023-12-27 10:14:56,341][20347] Updated weights for policy 0, policy_version 667713 (0.0007) [2023-12-27 10:14:56,389][20347] Updated weights for policy 0, policy_version 667723 (0.0008) [2023-12-27 10:14:56,797][20381] Updated weights for policy 1, policy_version 667137 (0.0007) [2023-12-27 10:14:56,848][20381] Updated weights for policy 1, policy_version 667147 (0.0006) [2023-12-27 10:14:56,894][20381] Updated weights for policy 1, policy_version 667157 (0.0008) [2023-12-27 10:14:57,110][20347] Updated weights for policy 0, policy_version 667733 (0.0007) [2023-12-27 10:14:57,166][20347] Updated weights for policy 0, policy_version 667743 (0.0008) [2023-12-27 10:14:57,219][20347] Updated weights for policy 0, policy_version 667753 (0.0008) [2023-12-27 10:14:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 341786624. Throughput: 0: 10298.0, 1: 10470.7. Samples: 341772992. Policy #0 lag: (min: 31.0, avg: 32.2, max: 62.0) [2023-12-27 10:14:57,363][19170] Avg episode reward: [(0, '853.780'), (1, '971.080')] [2023-12-27 10:14:57,519][20381] Updated weights for policy 1, policy_version 667167 (0.0008) [2023-12-27 10:14:57,570][20381] Updated weights for policy 1, policy_version 667177 (0.0007) [2023-12-27 10:14:57,625][20381] Updated weights for policy 1, policy_version 667187 (0.0008) [2023-12-27 10:14:57,943][20347] Updated weights for policy 0, policy_version 667763 (0.0008) [2023-12-27 10:14:58,004][20347] Updated weights for policy 0, policy_version 667773 (0.0008) [2023-12-27 10:14:58,060][20347] Updated weights for policy 0, policy_version 667783 (0.0008) [2023-12-27 10:14:58,321][20381] Updated weights for policy 1, policy_version 667197 (0.0008) [2023-12-27 10:14:58,382][20381] Updated weights for policy 1, policy_version 667207 (0.0008) [2023-12-27 10:14:58,448][20381] Updated weights for policy 1, policy_version 667217 (0.0008) [2023-12-27 10:14:58,834][20347] Updated weights for policy 0, policy_version 667793 (0.0008) [2023-12-27 10:14:58,892][20347] Updated weights for policy 0, policy_version 667803 (0.0009) [2023-12-27 10:14:58,954][20347] Updated weights for policy 0, policy_version 667813 (0.0008) [2023-12-27 10:14:59,022][20347] Updated weights for policy 0, policy_version 667823 (0.0007) [2023-12-27 10:14:59,202][20381] Updated weights for policy 1, policy_version 667227 (0.0009) [2023-12-27 10:14:59,261][20381] Updated weights for policy 1, policy_version 667237 (0.0010) [2023-12-27 10:14:59,319][20381] Updated weights for policy 1, policy_version 667247 (0.0006) [2023-12-27 10:14:59,782][20347] Updated weights for policy 0, policy_version 667833 (0.0006) [2023-12-27 10:14:59,849][20347] Updated weights for policy 0, policy_version 667843 (0.0008) [2023-12-27 10:14:59,910][20347] Updated weights for policy 0, policy_version 667853 (0.0008) [2023-12-27 10:14:59,977][20381] Updated weights for policy 1, policy_version 667257 (0.0010) [2023-12-27 10:15:00,043][20381] Updated weights for policy 1, policy_version 667267 (0.0011) [2023-12-27 10:15:00,107][20381] Updated weights for policy 1, policy_version 667277 (0.0009) [2023-12-27 10:15:00,167][20381] Updated weights for policy 1, policy_version 667287 (0.0010) [2023-12-27 10:15:00,585][20347] Updated weights for policy 0, policy_version 667863 (0.0008) [2023-12-27 10:15:00,642][20347] Updated weights for policy 0, policy_version 667873 (0.0007) [2023-12-27 10:15:00,697][20347] Updated weights for policy 0, policy_version 667883 (0.0008) [2023-12-27 10:15:00,822][20381] Updated weights for policy 1, policy_version 667297 (0.0010) [2023-12-27 10:15:00,870][20381] Updated weights for policy 1, policy_version 667307 (0.0010) [2023-12-27 10:15:00,917][20381] Updated weights for policy 1, policy_version 667317 (0.0009) [2023-12-27 10:15:01,379][20347] Updated weights for policy 0, policy_version 667893 (0.0009) [2023-12-27 10:15:01,443][20347] Updated weights for policy 0, policy_version 667903 (0.0008) [2023-12-27 10:15:01,506][20347] Updated weights for policy 0, policy_version 667913 (0.0008) [2023-12-27 10:15:01,632][20381] Updated weights for policy 1, policy_version 667327 (0.0011) [2023-12-27 10:15:01,694][20381] Updated weights for policy 1, policy_version 667337 (0.0010) [2023-12-27 10:15:01,758][20381] Updated weights for policy 1, policy_version 667347 (0.0008) [2023-12-27 10:15:02,162][20347] Updated weights for policy 0, policy_version 667923 (0.0008) [2023-12-27 10:15:02,226][20347] Updated weights for policy 0, policy_version 667933 (0.0008) [2023-12-27 10:15:02,284][20347] Updated weights for policy 0, policy_version 667943 (0.0007) [2023-12-27 10:15:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 341884928. Throughput: 0: 10225.4, 1: 10418.5. Samples: 341893860. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:02,363][19170] Avg episode reward: [(0, '814.660'), (1, '952.240')] [2023-12-27 10:15:02,441][20381] Updated weights for policy 1, policy_version 667357 (0.0008) [2023-12-27 10:15:02,501][20381] Updated weights for policy 1, policy_version 667367 (0.0009) [2023-12-27 10:15:02,539][20292] KL-divergence is very high: 107.6139 [2023-12-27 10:15:02,550][20381] Updated weights for policy 1, policy_version 667377 (0.0010) [2023-12-27 10:15:02,982][20347] Updated weights for policy 0, policy_version 667953 (0.0007) [2023-12-27 10:15:03,033][20347] Updated weights for policy 0, policy_version 667963 (0.0008) [2023-12-27 10:15:03,091][20347] Updated weights for policy 0, policy_version 667973 (0.0007) [2023-12-27 10:15:03,147][20347] Updated weights for policy 0, policy_version 667983 (0.0007) [2023-12-27 10:15:03,229][20381] Updated weights for policy 1, policy_version 667387 (0.0010) [2023-12-27 10:15:03,283][20381] Updated weights for policy 1, policy_version 667397 (0.0010) [2023-12-27 10:15:03,340][20381] Updated weights for policy 1, policy_version 667407 (0.0010) [2023-12-27 10:15:03,831][20347] Updated weights for policy 0, policy_version 667993 (0.0006) [2023-12-27 10:15:03,886][20347] Updated weights for policy 0, policy_version 668003 (0.0010) [2023-12-27 10:15:03,918][20381] Updated weights for policy 1, policy_version 667417 (0.0008) [2023-12-27 10:15:03,948][20347] Updated weights for policy 0, policy_version 668013 (0.0010) [2023-12-27 10:15:03,975][20381] Updated weights for policy 1, policy_version 667427 (0.0006) [2023-12-27 10:15:04,037][20381] Updated weights for policy 1, policy_version 667437 (0.0009) [2023-12-27 10:15:04,091][20381] Updated weights for policy 1, policy_version 667447 (0.0009) [2023-12-27 10:15:04,665][20347] Updated weights for policy 0, policy_version 668023 (0.0008) [2023-12-27 10:15:04,720][20347] Updated weights for policy 0, policy_version 668033 (0.0007) [2023-12-27 10:15:04,767][20347] Updated weights for policy 0, policy_version 668043 (0.0006) [2023-12-27 10:15:04,769][20381] Updated weights for policy 1, policy_version 667457 (0.0006) [2023-12-27 10:15:04,819][20381] Updated weights for policy 1, policy_version 667467 (0.0006) [2023-12-27 10:15:04,867][20381] Updated weights for policy 1, policy_version 667477 (0.0005) [2023-12-27 10:15:05,428][20347] Updated weights for policy 0, policy_version 668053 (0.0006) [2023-12-27 10:15:05,454][20381] Updated weights for policy 1, policy_version 667487 (0.0009) [2023-12-27 10:15:05,478][20347] Updated weights for policy 0, policy_version 668063 (0.0005) [2023-12-27 10:15:05,516][20381] Updated weights for policy 1, policy_version 667497 (0.0008) [2023-12-27 10:15:05,529][20347] Updated weights for policy 0, policy_version 668073 (0.0006) [2023-12-27 10:15:05,571][20381] Updated weights for policy 1, policy_version 667507 (0.0006) [2023-12-27 10:15:06,214][20381] Updated weights for policy 1, policy_version 667517 (0.0008) [2023-12-27 10:15:06,239][20347] Updated weights for policy 0, policy_version 668083 (0.0006) [2023-12-27 10:15:06,270][20381] Updated weights for policy 1, policy_version 667527 (0.0006) [2023-12-27 10:15:06,291][20347] Updated weights for policy 0, policy_version 668093 (0.0005) [2023-12-27 10:15:06,327][20381] Updated weights for policy 1, policy_version 667537 (0.0007) [2023-12-27 10:15:06,344][20347] Updated weights for policy 0, policy_version 668103 (0.0006) [2023-12-27 10:15:06,991][20347] Updated weights for policy 0, policy_version 668113 (0.0007) [2023-12-27 10:15:07,044][20347] Updated weights for policy 0, policy_version 668123 (0.0007) [2023-12-27 10:15:07,077][20381] Updated weights for policy 1, policy_version 667547 (0.0006) [2023-12-27 10:15:07,106][20347] Updated weights for policy 0, policy_version 668133 (0.0008) [2023-12-27 10:15:07,134][20381] Updated weights for policy 1, policy_version 667557 (0.0007) [2023-12-27 10:15:07,161][20347] Updated weights for policy 0, policy_version 668143 (0.0008) [2023-12-27 10:15:07,197][20381] Updated weights for policy 1, policy_version 667567 (0.0007) [2023-12-27 10:15:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 341991424. Throughput: 0: 10195.6, 1: 10441.6. Samples: 341955508. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:07,363][19170] Avg episode reward: [(0, '785.990'), (1, '942.480')] [2023-12-27 10:15:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000667576_170926080.pth... [2023-12-27 10:15:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000668144_171065344.pth... [2023-12-27 10:15:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000666328_170606592.pth [2023-12-27 10:15:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000666960_170762240.pth [2023-12-27 10:15:07,800][20347] Updated weights for policy 0, policy_version 668153 (0.0007) [2023-12-27 10:15:07,859][20347] Updated weights for policy 0, policy_version 668163 (0.0008) [2023-12-27 10:15:07,911][20381] Updated weights for policy 1, policy_version 667577 (0.0007) [2023-12-27 10:15:07,919][20347] Updated weights for policy 0, policy_version 668173 (0.0008) [2023-12-27 10:15:07,971][20381] Updated weights for policy 1, policy_version 667587 (0.0007) [2023-12-27 10:15:08,030][20381] Updated weights for policy 1, policy_version 667597 (0.0008) [2023-12-27 10:15:08,088][20381] Updated weights for policy 1, policy_version 667607 (0.0008) [2023-12-27 10:15:08,628][20347] Updated weights for policy 0, policy_version 668183 (0.0007) [2023-12-27 10:15:08,686][20347] Updated weights for policy 0, policy_version 668193 (0.0007) [2023-12-27 10:15:08,718][20381] Updated weights for policy 1, policy_version 667617 (0.0006) [2023-12-27 10:15:08,743][20347] Updated weights for policy 0, policy_version 668203 (0.0007) [2023-12-27 10:15:08,770][20381] Updated weights for policy 1, policy_version 667627 (0.0007) [2023-12-27 10:15:08,827][20381] Updated weights for policy 1, policy_version 667637 (0.0005) [2023-12-27 10:15:09,491][20381] Updated weights for policy 1, policy_version 667647 (0.0006) [2023-12-27 10:15:09,491][20347] Updated weights for policy 0, policy_version 668213 (0.0008) [2023-12-27 10:15:09,547][20381] Updated weights for policy 1, policy_version 667657 (0.0008) [2023-12-27 10:15:09,551][20347] Updated weights for policy 0, policy_version 668223 (0.0008) [2023-12-27 10:15:09,556][20291] KL-divergence is very high: 117.9554 [2023-12-27 10:15:09,603][20347] Updated weights for policy 0, policy_version 668233 (0.0008) [2023-12-27 10:15:09,606][20381] Updated weights for policy 1, policy_version 667667 (0.0008) [2023-12-27 10:15:10,267][20347] Updated weights for policy 0, policy_version 668243 (0.0007) [2023-12-27 10:15:10,324][20347] Updated weights for policy 0, policy_version 668253 (0.0007) [2023-12-27 10:15:10,338][20381] Updated weights for policy 1, policy_version 667677 (0.0007) [2023-12-27 10:15:10,375][20347] Updated weights for policy 0, policy_version 668263 (0.0008) [2023-12-27 10:15:10,398][20381] Updated weights for policy 1, policy_version 667687 (0.0008) [2023-12-27 10:15:10,457][20381] Updated weights for policy 1, policy_version 667697 (0.0006) [2023-12-27 10:15:11,035][20347] Updated weights for policy 0, policy_version 668273 (0.0006) [2023-12-27 10:15:11,061][20381] Updated weights for policy 1, policy_version 667707 (0.0006) [2023-12-27 10:15:11,107][20347] Updated weights for policy 0, policy_version 668283 (0.0008) [2023-12-27 10:15:11,116][20381] Updated weights for policy 1, policy_version 667717 (0.0007) [2023-12-27 10:15:11,172][20347] Updated weights for policy 0, policy_version 668293 (0.0007) [2023-12-27 10:15:11,177][20381] Updated weights for policy 1, policy_version 667727 (0.0008) [2023-12-27 10:15:11,238][20347] Updated weights for policy 0, policy_version 668303 (0.0007) [2023-12-27 10:15:11,924][20347] Updated weights for policy 0, policy_version 668313 (0.0007) [2023-12-27 10:15:11,926][20381] Updated weights for policy 1, policy_version 667737 (0.0007) [2023-12-27 10:15:11,984][20347] Updated weights for policy 0, policy_version 668323 (0.0009) [2023-12-27 10:15:11,985][20381] Updated weights for policy 1, policy_version 667747 (0.0006) [2023-12-27 10:15:12,045][20381] Updated weights for policy 1, policy_version 667757 (0.0006) [2023-12-27 10:15:12,067][20347] Updated weights for policy 0, policy_version 668333 (0.0008) [2023-12-27 10:15:12,104][20381] Updated weights for policy 1, policy_version 667767 (0.0006) [2023-12-27 10:15:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20827.1). Total num frames: 342089728. Throughput: 0: 10184.3, 1: 10448.8. Samples: 342078436. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:12,363][19170] Avg episode reward: [(0, '766.630'), (1, '961.150')] [2023-12-27 10:15:12,743][20347] Updated weights for policy 0, policy_version 668343 (0.0007) [2023-12-27 10:15:12,786][20381] Updated weights for policy 1, policy_version 667777 (0.0006) [2023-12-27 10:15:12,808][20347] Updated weights for policy 0, policy_version 668353 (0.0008) [2023-12-27 10:15:12,844][20381] Updated weights for policy 1, policy_version 667787 (0.0006) [2023-12-27 10:15:12,872][20347] Updated weights for policy 0, policy_version 668363 (0.0007) [2023-12-27 10:15:12,902][20381] Updated weights for policy 1, policy_version 667797 (0.0006) [2023-12-27 10:15:13,485][20347] Updated weights for policy 0, policy_version 668373 (0.0007) [2023-12-27 10:15:13,533][20347] Updated weights for policy 0, policy_version 668383 (0.0008) [2023-12-27 10:15:13,585][20347] Updated weights for policy 0, policy_version 668393 (0.0010) [2023-12-27 10:15:13,598][20381] Updated weights for policy 1, policy_version 667807 (0.0008) [2023-12-27 10:15:13,620][20292] KL-divergence is very high: 164.5843 [2023-12-27 10:15:13,654][20381] Updated weights for policy 1, policy_version 667817 (0.0006) [2023-12-27 10:15:13,708][20381] Updated weights for policy 1, policy_version 667827 (0.0007) [2023-12-27 10:15:14,296][20347] Updated weights for policy 0, policy_version 668403 (0.0010) [2023-12-27 10:15:14,355][20347] Updated weights for policy 0, policy_version 668413 (0.0008) [2023-12-27 10:15:14,363][20381] Updated weights for policy 1, policy_version 667837 (0.0007) [2023-12-27 10:15:14,413][20347] Updated weights for policy 0, policy_version 668423 (0.0007) [2023-12-27 10:15:14,420][20381] Updated weights for policy 1, policy_version 667847 (0.0007) [2023-12-27 10:15:14,476][20381] Updated weights for policy 1, policy_version 667857 (0.0007) [2023-12-27 10:15:15,073][20347] Updated weights for policy 0, policy_version 668433 (0.0005) [2023-12-27 10:15:15,134][20347] Updated weights for policy 0, policy_version 668443 (0.0006) [2023-12-27 10:15:15,196][20381] Updated weights for policy 1, policy_version 667867 (0.0007) [2023-12-27 10:15:15,199][20347] Updated weights for policy 0, policy_version 668453 (0.0006) [2023-12-27 10:15:15,252][20381] Updated weights for policy 1, policy_version 667877 (0.0006) [2023-12-27 10:15:15,264][20347] Updated weights for policy 0, policy_version 668463 (0.0007) [2023-12-27 10:15:15,305][20381] Updated weights for policy 1, policy_version 667887 (0.0008) [2023-12-27 10:15:15,815][20347] Updated weights for policy 0, policy_version 668473 (0.0005) [2023-12-27 10:15:15,863][20347] Updated weights for policy 0, policy_version 668483 (0.0005) [2023-12-27 10:15:15,875][20381] Updated weights for policy 1, policy_version 667897 (0.0006) [2023-12-27 10:15:15,911][20347] Updated weights for policy 0, policy_version 668493 (0.0005) [2023-12-27 10:15:15,934][20381] Updated weights for policy 1, policy_version 667907 (0.0007) [2023-12-27 10:15:15,987][20381] Updated weights for policy 1, policy_version 667917 (0.0008) [2023-12-27 10:15:16,047][20381] Updated weights for policy 1, policy_version 667927 (0.0007) [2023-12-27 10:15:16,551][20347] Updated weights for policy 0, policy_version 668503 (0.0007) [2023-12-27 10:15:16,600][20347] Updated weights for policy 0, policy_version 668513 (0.0008) [2023-12-27 10:15:16,651][20347] Updated weights for policy 0, policy_version 668523 (0.0007) [2023-12-27 10:15:16,682][20381] Updated weights for policy 1, policy_version 667937 (0.0009) [2023-12-27 10:15:16,737][20381] Updated weights for policy 1, policy_version 667947 (0.0007) [2023-12-27 10:15:16,794][20381] Updated weights for policy 1, policy_version 667957 (0.0008) [2023-12-27 10:15:17,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 342188032. Throughput: 0: 10133.4, 1: 10467.4. Samples: 342203272. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:17,364][19170] Avg episode reward: [(0, '785.830'), (1, '941.670')] [2023-12-27 10:15:17,395][20347] Updated weights for policy 0, policy_version 668533 (0.0006) [2023-12-27 10:15:17,409][20381] Updated weights for policy 1, policy_version 667967 (0.0010) [2023-12-27 10:15:17,458][20381] Updated weights for policy 1, policy_version 667977 (0.0010) [2023-12-27 10:15:17,460][20347] Updated weights for policy 0, policy_version 668543 (0.0007) [2023-12-27 10:15:17,511][20381] Updated weights for policy 1, policy_version 667987 (0.0006) [2023-12-27 10:15:17,521][20347] Updated weights for policy 0, policy_version 668553 (0.0008) [2023-12-27 10:15:18,069][20381] Updated weights for policy 1, policy_version 667997 (0.0005) [2023-12-27 10:15:18,127][20381] Updated weights for policy 1, policy_version 668007 (0.0006) [2023-12-27 10:15:18,190][20381] Updated weights for policy 1, policy_version 668017 (0.0011) [2023-12-27 10:15:18,247][20347] Updated weights for policy 0, policy_version 668563 (0.0007) [2023-12-27 10:15:18,298][20347] Updated weights for policy 0, policy_version 668573 (0.0007) [2023-12-27 10:15:18,360][20347] Updated weights for policy 0, policy_version 668583 (0.0007) [2023-12-27 10:15:18,863][20381] Updated weights for policy 1, policy_version 668027 (0.0011) [2023-12-27 10:15:18,926][20381] Updated weights for policy 1, policy_version 668037 (0.0011) [2023-12-27 10:15:18,984][20381] Updated weights for policy 1, policy_version 668047 (0.0009) [2023-12-27 10:15:19,093][20347] Updated weights for policy 0, policy_version 668593 (0.0008) [2023-12-27 10:15:19,139][20347] Updated weights for policy 0, policy_version 668603 (0.0008) [2023-12-27 10:15:19,193][20347] Updated weights for policy 0, policy_version 668613 (0.0008) [2023-12-27 10:15:19,257][20347] Updated weights for policy 0, policy_version 668623 (0.0008) [2023-12-27 10:15:19,680][20381] Updated weights for policy 1, policy_version 668057 (0.0006) [2023-12-27 10:15:19,743][20381] Updated weights for policy 1, policy_version 668067 (0.0008) [2023-12-27 10:15:19,811][20381] Updated weights for policy 1, policy_version 668077 (0.0011) [2023-12-27 10:15:19,882][20381] Updated weights for policy 1, policy_version 668087 (0.0008) [2023-12-27 10:15:19,983][20347] Updated weights for policy 0, policy_version 668633 (0.0006) [2023-12-27 10:15:20,049][20347] Updated weights for policy 0, policy_version 668643 (0.0008) [2023-12-27 10:15:20,106][20347] Updated weights for policy 0, policy_version 668653 (0.0006) [2023-12-27 10:15:20,569][20381] Updated weights for policy 1, policy_version 668097 (0.0010) [2023-12-27 10:15:20,581][20292] KL-divergence is very high: 115.7595 [2023-12-27 10:15:20,601][20292] KL-divergence is very high: 997.7441 [2023-12-27 10:15:20,626][20292] KL-divergence is very high: 247.1786 [2023-12-27 10:15:20,634][20381] Updated weights for policy 1, policy_version 668107 (0.0008) [2023-12-27 10:15:20,634][20292] KL-divergence is very high: 213.1322 [2023-12-27 10:15:20,653][20292] KL-divergence is very high: 334.7605 [2023-12-27 10:15:20,701][20381] Updated weights for policy 1, policy_version 668117 (0.0008) [2023-12-27 10:15:20,707][20292] KL-divergence is very high: 258.8307 [2023-12-27 10:15:20,768][20347] Updated weights for policy 0, policy_version 668663 (0.0007) [2023-12-27 10:15:20,828][20347] Updated weights for policy 0, policy_version 668673 (0.0007) [2023-12-27 10:15:20,894][20347] Updated weights for policy 0, policy_version 668683 (0.0006) [2023-12-27 10:15:21,468][20381] Updated weights for policy 1, policy_version 668127 (0.0008) [2023-12-27 10:15:21,524][20381] Updated weights for policy 1, policy_version 668137 (0.0008) [2023-12-27 10:15:21,581][20381] Updated weights for policy 1, policy_version 668147 (0.0007) [2023-12-27 10:15:21,636][20347] Updated weights for policy 0, policy_version 668693 (0.0009) [2023-12-27 10:15:21,693][20347] Updated weights for policy 0, policy_version 668703 (0.0007) [2023-12-27 10:15:21,757][20347] Updated weights for policy 0, policy_version 668713 (0.0008) [2023-12-27 10:15:22,306][20381] Updated weights for policy 1, policy_version 668157 (0.0008) [2023-12-27 10:15:22,337][20292] KL-divergence is very high: 189.4784 [2023-12-27 10:15:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20771.6). Total num frames: 342286336. Throughput: 0: 10097.7, 1: 10466.5. Samples: 342263704. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:22,363][19170] Avg episode reward: [(0, '814.940'), (1, '854.350')] [2023-12-27 10:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000668720_171212800.pth... [2023-12-27 10:15:22,370][20381] Updated weights for policy 1, policy_version 668167 (0.0008) [2023-12-27 10:15:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000667568_170917888.pth [2023-12-27 10:15:22,434][20381] Updated weights for policy 1, policy_version 668177 (0.0008) [2023-12-27 10:15:22,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000668184_171081728.pth... [2023-12-27 10:15:22,475][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000666968_170770432.pth [2023-12-27 10:15:22,522][20347] Updated weights for policy 0, policy_version 668723 (0.0008) [2023-12-27 10:15:22,585][20347] Updated weights for policy 0, policy_version 668733 (0.0008) [2023-12-27 10:15:22,644][20347] Updated weights for policy 0, policy_version 668743 (0.0008) [2023-12-27 10:15:23,132][20381] Updated weights for policy 1, policy_version 668187 (0.0008) [2023-12-27 10:15:23,188][20381] Updated weights for policy 1, policy_version 668197 (0.0010) [2023-12-27 10:15:23,245][20381] Updated weights for policy 1, policy_version 668207 (0.0010) [2023-12-27 10:15:23,369][20347] Updated weights for policy 0, policy_version 668753 (0.0008) [2023-12-27 10:15:23,425][20347] Updated weights for policy 0, policy_version 668763 (0.0007) [2023-12-27 10:15:23,490][20347] Updated weights for policy 0, policy_version 668773 (0.0006) [2023-12-27 10:15:23,546][20347] Updated weights for policy 0, policy_version 668783 (0.0008) [2023-12-27 10:15:23,919][20381] Updated weights for policy 1, policy_version 668217 (0.0010) [2023-12-27 10:15:23,965][20292] KL-divergence is very high: 253.9733 [2023-12-27 10:15:23,976][20381] Updated weights for policy 1, policy_version 668227 (0.0008) [2023-12-27 10:15:24,010][20292] KL-divergence is very high: 117.1584 [2023-12-27 10:15:24,034][20381] Updated weights for policy 1, policy_version 668237 (0.0008) [2023-12-27 10:15:24,085][20381] Updated weights for policy 1, policy_version 668247 (0.0008) [2023-12-27 10:15:24,191][20347] Updated weights for policy 0, policy_version 668793 (0.0005) [2023-12-27 10:15:24,248][20347] Updated weights for policy 0, policy_version 668803 (0.0005) [2023-12-27 10:15:24,309][20347] Updated weights for policy 0, policy_version 668813 (0.0006) [2023-12-27 10:15:24,805][20292] KL-divergence is very high: 193.4491 [2023-12-27 10:15:24,847][20347] Updated weights for policy 0, policy_version 668823 (0.0007) [2023-12-27 10:15:24,852][20292] KL-divergence is very high: 123.3735 [2023-12-27 10:15:24,853][20381] Updated weights for policy 1, policy_version 668257 (0.0011) [2023-12-27 10:15:24,898][20292] KL-divergence is very high: 145.5228 [2023-12-27 10:15:24,908][20347] Updated weights for policy 0, policy_version 668833 (0.0007) [2023-12-27 10:15:24,911][20381] Updated weights for policy 1, policy_version 668267 (0.0010) [2023-12-27 10:15:24,945][20292] KL-divergence is very high: 141.7204 [2023-12-27 10:15:24,966][20381] Updated weights for policy 1, policy_version 668277 (0.0010) [2023-12-27 10:15:24,967][20347] Updated weights for policy 0, policy_version 668843 (0.0006) [2023-12-27 10:15:25,563][20347] Updated weights for policy 0, policy_version 668853 (0.0006) [2023-12-27 10:15:25,611][20347] Updated weights for policy 0, policy_version 668863 (0.0007) [2023-12-27 10:15:25,660][20347] Updated weights for policy 0, policy_version 668873 (0.0007) [2023-12-27 10:15:25,688][20381] Updated weights for policy 1, policy_version 668287 (0.0007) [2023-12-27 10:15:25,745][20381] Updated weights for policy 1, policy_version 668297 (0.0006) [2023-12-27 10:15:25,792][20381] Updated weights for policy 1, policy_version 668307 (0.0006) [2023-12-27 10:15:26,367][20347] Updated weights for policy 0, policy_version 668883 (0.0007) [2023-12-27 10:15:26,427][20347] Updated weights for policy 0, policy_version 668893 (0.0010) [2023-12-27 10:15:26,451][20381] Updated weights for policy 1, policy_version 668317 (0.0007) [2023-12-27 10:15:26,477][20347] Updated weights for policy 0, policy_version 668903 (0.0010) [2023-12-27 10:15:26,499][20381] Updated weights for policy 1, policy_version 668327 (0.0005) [2023-12-27 10:15:26,552][20381] Updated weights for policy 1, policy_version 668337 (0.0005) [2023-12-27 10:15:27,076][20347] Updated weights for policy 0, policy_version 668913 (0.0010) [2023-12-27 10:15:27,126][20347] Updated weights for policy 0, policy_version 668923 (0.0008) [2023-12-27 10:15:27,181][20347] Updated weights for policy 0, policy_version 668933 (0.0008) [2023-12-27 10:15:27,217][20381] Updated weights for policy 1, policy_version 668347 (0.0005) [2023-12-27 10:15:27,231][20347] Updated weights for policy 0, policy_version 668943 (0.0010) [2023-12-27 10:15:27,277][20381] Updated weights for policy 1, policy_version 668357 (0.0006) [2023-12-27 10:15:27,337][20381] Updated weights for policy 1, policy_version 668367 (0.0008) [2023-12-27 10:15:27,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20771.6). Total num frames: 342392832. Throughput: 0: 10070.1, 1: 10351.9. Samples: 342382696. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:27,363][19170] Avg episode reward: [(0, '824.560'), (1, '767.110')] [2023-12-27 10:15:27,847][20347] Updated weights for policy 0, policy_version 668953 (0.0006) [2023-12-27 10:15:27,903][20347] Updated weights for policy 0, policy_version 668963 (0.0005) [2023-12-27 10:15:27,961][20347] Updated weights for policy 0, policy_version 668973 (0.0005) [2023-12-27 10:15:28,060][20381] Updated weights for policy 1, policy_version 668377 (0.0007) [2023-12-27 10:15:28,112][20381] Updated weights for policy 1, policy_version 668387 (0.0007) [2023-12-27 10:15:28,173][20381] Updated weights for policy 1, policy_version 668397 (0.0005) [2023-12-27 10:15:28,229][20381] Updated weights for policy 1, policy_version 668407 (0.0005) [2023-12-27 10:15:28,589][20347] Updated weights for policy 0, policy_version 668983 (0.0010) [2023-12-27 10:15:28,647][20347] Updated weights for policy 0, policy_version 668993 (0.0006) [2023-12-27 10:15:28,703][20347] Updated weights for policy 0, policy_version 669003 (0.0006) [2023-12-27 10:15:28,832][20381] Updated weights for policy 1, policy_version 668417 (0.0006) [2023-12-27 10:15:28,889][20381] Updated weights for policy 1, policy_version 668427 (0.0006) [2023-12-27 10:15:28,946][20381] Updated weights for policy 1, policy_version 668437 (0.0006) [2023-12-27 10:15:29,256][20347] Updated weights for policy 0, policy_version 669013 (0.0008) [2023-12-27 10:15:29,316][20347] Updated weights for policy 0, policy_version 669023 (0.0011) [2023-12-27 10:15:29,381][20347] Updated weights for policy 0, policy_version 669033 (0.0010) [2023-12-27 10:15:29,517][20381] Updated weights for policy 1, policy_version 668447 (0.0008) [2023-12-27 10:15:29,564][20381] Updated weights for policy 1, policy_version 668457 (0.0007) [2023-12-27 10:15:29,611][20381] Updated weights for policy 1, policy_version 668467 (0.0007) [2023-12-27 10:15:30,086][20347] Updated weights for policy 0, policy_version 669043 (0.0009) [2023-12-27 10:15:30,145][20347] Updated weights for policy 0, policy_version 669053 (0.0007) [2023-12-27 10:15:30,205][20347] Updated weights for policy 0, policy_version 669063 (0.0006) [2023-12-27 10:15:30,288][20381] Updated weights for policy 1, policy_version 668477 (0.0008) [2023-12-27 10:15:30,344][20381] Updated weights for policy 1, policy_version 668487 (0.0008) [2023-12-27 10:15:30,406][20381] Updated weights for policy 1, policy_version 668497 (0.0007) [2023-12-27 10:15:30,759][20347] Updated weights for policy 0, policy_version 669073 (0.0006) [2023-12-27 10:15:30,819][20347] Updated weights for policy 0, policy_version 669083 (0.0005) [2023-12-27 10:15:30,869][20347] Updated weights for policy 0, policy_version 669093 (0.0005) [2023-12-27 10:15:30,916][20347] Updated weights for policy 0, policy_version 669103 (0.0008) [2023-12-27 10:15:31,017][20381] Updated weights for policy 1, policy_version 668507 (0.0007) [2023-12-27 10:15:31,084][20381] Updated weights for policy 1, policy_version 668517 (0.0008) [2023-12-27 10:15:31,141][20381] Updated weights for policy 1, policy_version 668527 (0.0008) [2023-12-27 10:15:31,535][20347] Updated weights for policy 0, policy_version 669113 (0.0009) [2023-12-27 10:15:31,583][20347] Updated weights for policy 0, policy_version 669123 (0.0008) [2023-12-27 10:15:31,593][20291] KL-divergence is very high: 274.3230 [2023-12-27 10:15:31,638][20291] KL-divergence is very high: 125.2175 [2023-12-27 10:15:31,639][20347] Updated weights for policy 0, policy_version 669133 (0.0008) [2023-12-27 10:15:31,820][20381] Updated weights for policy 1, policy_version 668537 (0.0005) [2023-12-27 10:15:31,872][20381] Updated weights for policy 1, policy_version 668547 (0.0005) [2023-12-27 10:15:31,923][20381] Updated weights for policy 1, policy_version 668557 (0.0005) [2023-12-27 10:15:31,974][20381] Updated weights for policy 1, policy_version 668567 (0.0005) [2023-12-27 10:15:32,349][20291] KL-divergence is very high: 119.8983 [2023-12-27 10:15:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 342499328. Throughput: 0: 10172.0, 1: 10387.4. Samples: 342512600. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:32,363][19170] Avg episode reward: [(0, '835.120'), (1, '806.960')] [2023-12-27 10:15:32,370][20291] KL-divergence is very high: 123.0938 [2023-12-27 10:15:32,389][20347] Updated weights for policy 0, policy_version 669143 (0.0008) [2023-12-27 10:15:32,401][20291] KL-divergence is very high: 121.5840 [2023-12-27 10:15:32,407][20291] KL-divergence is very high: 110.7972 [2023-12-27 10:15:32,447][20347] Updated weights for policy 0, policy_version 669153 (0.0007) [2023-12-27 10:15:32,466][20291] KL-divergence is very high: 110.5884 [2023-12-27 10:15:32,497][20291] KL-divergence is very high: 163.9301 [2023-12-27 10:15:32,509][20347] Updated weights for policy 0, policy_version 669163 (0.0007) [2023-12-27 10:15:32,633][20381] Updated weights for policy 1, policy_version 668577 (0.0006) [2023-12-27 10:15:32,687][20381] Updated weights for policy 1, policy_version 668587 (0.0006) [2023-12-27 10:15:32,742][20381] Updated weights for policy 1, policy_version 668597 (0.0007) [2023-12-27 10:15:33,192][20347] Updated weights for policy 0, policy_version 669173 (0.0007) [2023-12-27 10:15:33,241][20347] Updated weights for policy 0, policy_version 669183 (0.0005) [2023-12-27 10:15:33,291][20347] Updated weights for policy 0, policy_version 669193 (0.0005) [2023-12-27 10:15:33,436][20381] Updated weights for policy 1, policy_version 668607 (0.0007) [2023-12-27 10:15:33,496][20381] Updated weights for policy 1, policy_version 668617 (0.0008) [2023-12-27 10:15:33,552][20381] Updated weights for policy 1, policy_version 668627 (0.0009) [2023-12-27 10:15:33,828][20347] Updated weights for policy 0, policy_version 669203 (0.0005) [2023-12-27 10:15:33,885][20347] Updated weights for policy 0, policy_version 669213 (0.0005) [2023-12-27 10:15:33,944][20347] Updated weights for policy 0, policy_version 669223 (0.0006) [2023-12-27 10:15:34,168][20381] Updated weights for policy 1, policy_version 668637 (0.0008) [2023-12-27 10:15:34,225][20381] Updated weights for policy 1, policy_version 668647 (0.0006) [2023-12-27 10:15:34,277][20381] Updated weights for policy 1, policy_version 668657 (0.0008) [2023-12-27 10:15:34,602][20347] Updated weights for policy 0, policy_version 669233 (0.0008) [2023-12-27 10:15:34,666][20347] Updated weights for policy 0, policy_version 669243 (0.0008) [2023-12-27 10:15:34,735][20347] Updated weights for policy 0, policy_version 669253 (0.0007) [2023-12-27 10:15:34,793][20347] Updated weights for policy 0, policy_version 669263 (0.0008) [2023-12-27 10:15:34,964][20381] Updated weights for policy 1, policy_version 668667 (0.0008) [2023-12-27 10:15:35,017][20381] Updated weights for policy 1, policy_version 668677 (0.0008) [2023-12-27 10:15:35,072][20381] Updated weights for policy 1, policy_version 668687 (0.0008) [2023-12-27 10:15:35,452][20347] Updated weights for policy 0, policy_version 669273 (0.0008) [2023-12-27 10:15:35,510][20347] Updated weights for policy 0, policy_version 669283 (0.0008) [2023-12-27 10:15:35,562][20347] Updated weights for policy 0, policy_version 669293 (0.0007) [2023-12-27 10:15:35,771][20381] Updated weights for policy 1, policy_version 668697 (0.0007) [2023-12-27 10:15:35,820][20381] Updated weights for policy 1, policy_version 668707 (0.0007) [2023-12-27 10:15:35,874][20381] Updated weights for policy 1, policy_version 668717 (0.0007) [2023-12-27 10:15:35,920][20381] Updated weights for policy 1, policy_version 668727 (0.0007) [2023-12-27 10:15:36,301][20347] Updated weights for policy 0, policy_version 669303 (0.0007) [2023-12-27 10:15:36,366][20347] Updated weights for policy 0, policy_version 669313 (0.0007) [2023-12-27 10:15:36,426][20347] Updated weights for policy 0, policy_version 669323 (0.0008) [2023-12-27 10:15:36,579][20381] Updated weights for policy 1, policy_version 668737 (0.0007) [2023-12-27 10:15:36,637][20381] Updated weights for policy 1, policy_version 668747 (0.0007) [2023-12-27 10:15:36,692][20381] Updated weights for policy 1, policy_version 668757 (0.0007) [2023-12-27 10:15:37,137][20347] Updated weights for policy 0, policy_version 669333 (0.0007) [2023-12-27 10:15:37,194][20347] Updated weights for policy 0, policy_version 669343 (0.0008) [2023-12-27 10:15:37,247][20347] Updated weights for policy 0, policy_version 669353 (0.0008) [2023-12-27 10:15:37,340][20381] Updated weights for policy 1, policy_version 668767 (0.0008) [2023-12-27 10:15:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.7, 300 sec: 20771.6). Total num frames: 342605824. Throughput: 0: 10238.1, 1: 10374.9. Samples: 342575160. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:37,363][19170] Avg episode reward: [(0, '601.150'), (1, '816.410')] [2023-12-27 10:15:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000669360_171376640.pth... [2023-12-27 10:15:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000668144_171065344.pth [2023-12-27 10:15:37,394][20381] Updated weights for policy 1, policy_version 668777 (0.0008) [2023-12-27 10:15:37,446][20381] Updated weights for policy 1, policy_version 668787 (0.0008) [2023-12-27 10:15:37,471][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000668792_171237376.pth... [2023-12-27 10:15:37,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000667576_170926080.pth [2023-12-27 10:15:37,954][20347] Updated weights for policy 0, policy_version 669363 (0.0008) [2023-12-27 10:15:38,008][20347] Updated weights for policy 0, policy_version 669373 (0.0007) [2023-12-27 10:15:38,053][20347] Updated weights for policy 0, policy_version 669383 (0.0007) [2023-12-27 10:15:38,101][20291] KL-divergence is very high: 110.4490 [2023-12-27 10:15:38,187][20381] Updated weights for policy 1, policy_version 668797 (0.0007) [2023-12-27 10:15:38,239][20381] Updated weights for policy 1, policy_version 668807 (0.0010) [2023-12-27 10:15:38,289][20381] Updated weights for policy 1, policy_version 668817 (0.0010) [2023-12-27 10:15:38,738][20347] Updated weights for policy 0, policy_version 669393 (0.0007) [2023-12-27 10:15:38,797][20347] Updated weights for policy 0, policy_version 669403 (0.0007) [2023-12-27 10:15:38,854][20347] Updated weights for policy 0, policy_version 669413 (0.0007) [2023-12-27 10:15:38,904][20347] Updated weights for policy 0, policy_version 669423 (0.0007) [2023-12-27 10:15:38,938][20381] Updated weights for policy 1, policy_version 668827 (0.0011) [2023-12-27 10:15:38,984][20381] Updated weights for policy 1, policy_version 668837 (0.0010) [2023-12-27 10:15:39,032][20381] Updated weights for policy 1, policy_version 668847 (0.0010) [2023-12-27 10:15:39,598][20347] Updated weights for policy 0, policy_version 669433 (0.0010) [2023-12-27 10:15:39,654][20347] Updated weights for policy 0, policy_version 669443 (0.0010) [2023-12-27 10:15:39,721][20347] Updated weights for policy 0, policy_version 669453 (0.0010) [2023-12-27 10:15:39,816][20381] Updated weights for policy 1, policy_version 668857 (0.0010) [2023-12-27 10:15:39,883][20381] Updated weights for policy 1, policy_version 668867 (0.0009) [2023-12-27 10:15:39,946][20381] Updated weights for policy 1, policy_version 668877 (0.0008) [2023-12-27 10:15:39,999][20381] Updated weights for policy 1, policy_version 668887 (0.0008) [2023-12-27 10:15:40,391][20347] Updated weights for policy 0, policy_version 669463 (0.0010) [2023-12-27 10:15:40,445][20347] Updated weights for policy 0, policy_version 669473 (0.0011) [2023-12-27 10:15:40,494][20347] Updated weights for policy 0, policy_version 669483 (0.0010) [2023-12-27 10:15:40,617][20381] Updated weights for policy 1, policy_version 668897 (0.0007) [2023-12-27 10:15:40,677][20381] Updated weights for policy 1, policy_version 668907 (0.0007) [2023-12-27 10:15:40,728][20381] Updated weights for policy 1, policy_version 668917 (0.0006) [2023-12-27 10:15:41,220][20347] Updated weights for policy 0, policy_version 669493 (0.0008) [2023-12-27 10:15:41,290][20347] Updated weights for policy 0, policy_version 669503 (0.0008) [2023-12-27 10:15:41,347][20381] Updated weights for policy 1, policy_version 668927 (0.0007) [2023-12-27 10:15:41,354][20347] Updated weights for policy 0, policy_version 669513 (0.0006) [2023-12-27 10:15:41,419][20381] Updated weights for policy 1, policy_version 668937 (0.0009) [2023-12-27 10:15:41,485][20381] Updated weights for policy 1, policy_version 668947 (0.0008) [2023-12-27 10:15:41,973][20347] Updated weights for policy 0, policy_version 669523 (0.0007) [2023-12-27 10:15:42,034][20347] Updated weights for policy 0, policy_version 669533 (0.0006) [2023-12-27 10:15:42,097][20347] Updated weights for policy 0, policy_version 669543 (0.0008) [2023-12-27 10:15:42,160][20381] Updated weights for policy 1, policy_version 668957 (0.0008) [2023-12-27 10:15:42,207][20381] Updated weights for policy 1, policy_version 668967 (0.0008) [2023-12-27 10:15:42,256][20381] Updated weights for policy 1, policy_version 668977 (0.0009) [2023-12-27 10:15:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 342712320. Throughput: 0: 10220.4, 1: 10327.0. Samples: 342697620. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:42,363][19170] Avg episode reward: [(0, '248.230'), (1, '912.690')] [2023-12-27 10:15:42,743][20347] Updated weights for policy 0, policy_version 669553 (0.0007) [2023-12-27 10:15:42,800][20347] Updated weights for policy 0, policy_version 669563 (0.0008) [2023-12-27 10:15:42,849][20347] Updated weights for policy 0, policy_version 669573 (0.0010) [2023-12-27 10:15:42,894][20347] Updated weights for policy 0, policy_version 669583 (0.0007) [2023-12-27 10:15:42,938][20381] Updated weights for policy 1, policy_version 668987 (0.0009) [2023-12-27 10:15:42,992][20381] Updated weights for policy 1, policy_version 668997 (0.0005) [2023-12-27 10:15:43,043][20381] Updated weights for policy 1, policy_version 669007 (0.0005) [2023-12-27 10:15:43,544][20347] Updated weights for policy 0, policy_version 669593 (0.0009) [2023-12-27 10:15:43,598][20347] Updated weights for policy 0, policy_version 669603 (0.0005) [2023-12-27 10:15:43,651][20347] Updated weights for policy 0, policy_version 669613 (0.0005) [2023-12-27 10:15:43,681][20381] Updated weights for policy 1, policy_version 669017 (0.0006) [2023-12-27 10:15:43,745][20381] Updated weights for policy 1, policy_version 669027 (0.0005) [2023-12-27 10:15:43,796][20381] Updated weights for policy 1, policy_version 669037 (0.0008) [2023-12-27 10:15:43,841][20381] Updated weights for policy 1, policy_version 669047 (0.0010) [2023-12-27 10:15:44,271][20347] Updated weights for policy 0, policy_version 669623 (0.0007) [2023-12-27 10:15:44,317][20347] Updated weights for policy 0, policy_version 669633 (0.0007) [2023-12-27 10:15:44,365][20347] Updated weights for policy 0, policy_version 669643 (0.0007) [2023-12-27 10:15:44,514][20381] Updated weights for policy 1, policy_version 669057 (0.0011) [2023-12-27 10:15:44,572][20381] Updated weights for policy 1, policy_version 669067 (0.0011) [2023-12-27 10:15:44,625][20381] Updated weights for policy 1, policy_version 669077 (0.0011) [2023-12-27 10:15:45,079][20347] Updated weights for policy 0, policy_version 669653 (0.0006) [2023-12-27 10:15:45,134][20347] Updated weights for policy 0, policy_version 669663 (0.0008) [2023-12-27 10:15:45,197][20347] Updated weights for policy 0, policy_version 669673 (0.0005) [2023-12-27 10:15:45,335][20381] Updated weights for policy 1, policy_version 669087 (0.0009) [2023-12-27 10:15:45,387][20381] Updated weights for policy 1, policy_version 669097 (0.0006) [2023-12-27 10:15:45,437][20381] Updated weights for policy 1, policy_version 669107 (0.0005) [2023-12-27 10:15:45,784][20347] Updated weights for policy 0, policy_version 669683 (0.0005) [2023-12-27 10:15:45,836][20347] Updated weights for policy 0, policy_version 669693 (0.0005) [2023-12-27 10:15:45,886][20347] Updated weights for policy 0, policy_version 669703 (0.0010) [2023-12-27 10:15:46,065][20381] Updated weights for policy 1, policy_version 669117 (0.0007) [2023-12-27 10:15:46,118][20381] Updated weights for policy 1, policy_version 669127 (0.0005) [2023-12-27 10:15:46,163][20381] Updated weights for policy 1, policy_version 669137 (0.0005) [2023-12-27 10:15:46,543][20347] Updated weights for policy 0, policy_version 669713 (0.0009) [2023-12-27 10:15:46,606][20347] Updated weights for policy 0, policy_version 669723 (0.0008) [2023-12-27 10:15:46,654][20347] Updated weights for policy 0, policy_version 669733 (0.0006) [2023-12-27 10:15:46,706][20347] Updated weights for policy 0, policy_version 669743 (0.0005) [2023-12-27 10:15:46,758][20381] Updated weights for policy 1, policy_version 669147 (0.0007) [2023-12-27 10:15:46,802][20381] Updated weights for policy 1, policy_version 669157 (0.0008) [2023-12-27 10:15:46,850][20381] Updated weights for policy 1, policy_version 669167 (0.0007) [2023-12-27 10:15:47,291][20347] Updated weights for policy 0, policy_version 669753 (0.0005) [2023-12-27 10:15:47,344][20347] Updated weights for policy 0, policy_version 669763 (0.0005) [2023-12-27 10:15:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 342810624. Throughput: 0: 10354.9, 1: 10338.3. Samples: 342825052. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:47,363][19170] Avg episode reward: [(0, '299.900'), (1, '893.710')] [2023-12-27 10:15:47,404][20347] Updated weights for policy 0, policy_version 669773 (0.0009) [2023-12-27 10:15:47,569][20381] Updated weights for policy 1, policy_version 669177 (0.0008) [2023-12-27 10:15:47,628][20381] Updated weights for policy 1, policy_version 669187 (0.0008) [2023-12-27 10:15:47,687][20381] Updated weights for policy 1, policy_version 669197 (0.0007) [2023-12-27 10:15:47,740][20381] Updated weights for policy 1, policy_version 669207 (0.0008) [2023-12-27 10:15:48,010][20347] Updated weights for policy 0, policy_version 669783 (0.0008) [2023-12-27 10:15:48,067][20347] Updated weights for policy 0, policy_version 669793 (0.0008) [2023-12-27 10:15:48,131][20347] Updated weights for policy 0, policy_version 669803 (0.0007) [2023-12-27 10:15:48,488][20381] Updated weights for policy 1, policy_version 669217 (0.0008) [2023-12-27 10:15:48,546][20381] Updated weights for policy 1, policy_version 669227 (0.0008) [2023-12-27 10:15:48,603][20381] Updated weights for policy 1, policy_version 669237 (0.0009) [2023-12-27 10:15:48,761][20347] Updated weights for policy 0, policy_version 669813 (0.0007) [2023-12-27 10:15:48,818][20347] Updated weights for policy 0, policy_version 669823 (0.0005) [2023-12-27 10:15:48,878][20347] Updated weights for policy 0, policy_version 669833 (0.0008) [2023-12-27 10:15:49,334][20381] Updated weights for policy 1, policy_version 669247 (0.0007) [2023-12-27 10:15:49,397][20381] Updated weights for policy 1, policy_version 669257 (0.0009) [2023-12-27 10:15:49,456][20381] Updated weights for policy 1, policy_version 669267 (0.0008) [2023-12-27 10:15:49,515][20347] Updated weights for policy 0, policy_version 669843 (0.0008) [2023-12-27 10:15:49,572][20347] Updated weights for policy 0, policy_version 669853 (0.0008) [2023-12-27 10:15:49,632][20347] Updated weights for policy 0, policy_version 669863 (0.0008) [2023-12-27 10:15:50,143][20381] Updated weights for policy 1, policy_version 669277 (0.0007) [2023-12-27 10:15:50,198][20381] Updated weights for policy 1, policy_version 669287 (0.0006) [2023-12-27 10:15:50,250][20381] Updated weights for policy 1, policy_version 669297 (0.0006) [2023-12-27 10:15:50,391][20347] Updated weights for policy 0, policy_version 669873 (0.0006) [2023-12-27 10:15:50,446][20347] Updated weights for policy 0, policy_version 669883 (0.0008) [2023-12-27 10:15:50,502][20347] Updated weights for policy 0, policy_version 669893 (0.0008) [2023-12-27 10:15:50,550][20347] Updated weights for policy 0, policy_version 669903 (0.0007) [2023-12-27 10:15:50,852][20381] Updated weights for policy 1, policy_version 669307 (0.0006) [2023-12-27 10:15:50,910][20381] Updated weights for policy 1, policy_version 669317 (0.0005) [2023-12-27 10:15:50,962][20381] Updated weights for policy 1, policy_version 669327 (0.0005) [2023-12-27 10:15:51,303][20347] Updated weights for policy 0, policy_version 669913 (0.0008) [2023-12-27 10:15:51,371][20347] Updated weights for policy 0, policy_version 669923 (0.0007) [2023-12-27 10:15:51,430][20347] Updated weights for policy 0, policy_version 669933 (0.0006) [2023-12-27 10:15:51,655][20381] Updated weights for policy 1, policy_version 669337 (0.0007) [2023-12-27 10:15:51,720][20381] Updated weights for policy 1, policy_version 669347 (0.0008) [2023-12-27 10:15:51,780][20381] Updated weights for policy 1, policy_version 669357 (0.0008) [2023-12-27 10:15:51,837][20381] Updated weights for policy 1, policy_version 669367 (0.0008) [2023-12-27 10:15:52,082][20347] Updated weights for policy 0, policy_version 669943 (0.0006) [2023-12-27 10:15:52,128][20347] Updated weights for policy 0, policy_version 669953 (0.0005) [2023-12-27 10:15:52,173][20347] Updated weights for policy 0, policy_version 669963 (0.0006) [2023-12-27 10:15:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 342917120. Throughput: 0: 10399.0, 1: 10282.5. Samples: 342886176. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:52,363][19170] Avg episode reward: [(0, '162.520'), (1, '902.680')] [2023-12-27 10:15:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000669368_171384832.pth... [2023-12-27 10:15:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000669968_171532288.pth... [2023-12-27 10:15:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000668720_171212800.pth [2023-12-27 10:15:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000668184_171081728.pth [2023-12-27 10:15:52,536][20381] Updated weights for policy 1, policy_version 669377 (0.0007) [2023-12-27 10:15:52,596][20381] Updated weights for policy 1, policy_version 669387 (0.0006) [2023-12-27 10:15:52,652][20381] Updated weights for policy 1, policy_version 669397 (0.0008) [2023-12-27 10:15:52,863][20347] Updated weights for policy 0, policy_version 669973 (0.0006) [2023-12-27 10:15:52,920][20347] Updated weights for policy 0, policy_version 669983 (0.0006) [2023-12-27 10:15:52,967][20347] Updated weights for policy 0, policy_version 669993 (0.0007) [2023-12-27 10:15:53,360][20381] Updated weights for policy 1, policy_version 669407 (0.0006) [2023-12-27 10:15:53,421][20381] Updated weights for policy 1, policy_version 669417 (0.0005) [2023-12-27 10:15:53,476][20381] Updated weights for policy 1, policy_version 669427 (0.0005) [2023-12-27 10:15:53,656][20347] Updated weights for policy 0, policy_version 670003 (0.0008) [2023-12-27 10:15:53,722][20347] Updated weights for policy 0, policy_version 670013 (0.0007) [2023-12-27 10:15:53,788][20347] Updated weights for policy 0, policy_version 670023 (0.0008) [2023-12-27 10:15:54,015][20381] Updated weights for policy 1, policy_version 669437 (0.0005) [2023-12-27 10:15:54,070][20381] Updated weights for policy 1, policy_version 669447 (0.0007) [2023-12-27 10:15:54,123][20381] Updated weights for policy 1, policy_version 669457 (0.0008) [2023-12-27 10:15:54,469][20347] Updated weights for policy 0, policy_version 670033 (0.0007) [2023-12-27 10:15:54,525][20347] Updated weights for policy 0, policy_version 670043 (0.0008) [2023-12-27 10:15:54,584][20347] Updated weights for policy 0, policy_version 670053 (0.0007) [2023-12-27 10:15:54,638][20347] Updated weights for policy 0, policy_version 670063 (0.0007) [2023-12-27 10:15:54,752][20381] Updated weights for policy 1, policy_version 669467 (0.0006) [2023-12-27 10:15:54,809][20381] Updated weights for policy 1, policy_version 669477 (0.0008) [2023-12-27 10:15:54,869][20381] Updated weights for policy 1, policy_version 669487 (0.0008) [2023-12-27 10:15:55,361][20347] Updated weights for policy 0, policy_version 670073 (0.0007) [2023-12-27 10:15:55,409][20347] Updated weights for policy 0, policy_version 670083 (0.0009) [2023-12-27 10:15:55,451][20381] Updated weights for policy 1, policy_version 669497 (0.0006) [2023-12-27 10:15:55,472][20347] Updated weights for policy 0, policy_version 670093 (0.0007) [2023-12-27 10:15:55,504][20381] Updated weights for policy 1, policy_version 669507 (0.0005) [2023-12-27 10:15:55,553][20381] Updated weights for policy 1, policy_version 669517 (0.0006) [2023-12-27 10:15:55,601][20381] Updated weights for policy 1, policy_version 669527 (0.0006) [2023-12-27 10:15:56,198][20347] Updated weights for policy 0, policy_version 670103 (0.0008) [2023-12-27 10:15:56,248][20347] Updated weights for policy 0, policy_version 670113 (0.0007) [2023-12-27 10:15:56,250][20381] Updated weights for policy 1, policy_version 669537 (0.0006) [2023-12-27 10:15:56,256][20292] KL-divergence is very high: 279.9510 [2023-12-27 10:15:56,298][20347] Updated weights for policy 0, policy_version 670123 (0.0007) [2023-12-27 10:15:56,307][20292] KL-divergence is very high: 168.2939 [2023-12-27 10:15:56,312][20381] Updated weights for policy 1, policy_version 669547 (0.0005) [2023-12-27 10:15:56,363][20381] Updated weights for policy 1, policy_version 669557 (0.0006) [2023-12-27 10:15:56,888][20381] Updated weights for policy 1, policy_version 669567 (0.0007) [2023-12-27 10:15:56,934][20381] Updated weights for policy 1, policy_version 669577 (0.0008) [2023-12-27 10:15:56,982][20381] Updated weights for policy 1, policy_version 669587 (0.0008) [2023-12-27 10:15:57,040][20347] Updated weights for policy 0, policy_version 670133 (0.0007) [2023-12-27 10:15:57,092][20347] Updated weights for policy 0, policy_version 670143 (0.0007) [2023-12-27 10:15:57,139][20347] Updated weights for policy 0, policy_version 670153 (0.0007) [2023-12-27 10:15:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 343023616. Throughput: 0: 10348.2, 1: 10360.6. Samples: 343010332. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:15:57,363][19170] Avg episode reward: [(0, '227.450'), (1, '951.400')] [2023-12-27 10:15:57,630][20381] Updated weights for policy 1, policy_version 669597 (0.0006) [2023-12-27 10:15:57,682][20381] Updated weights for policy 1, policy_version 669607 (0.0007) [2023-12-27 10:15:57,728][20381] Updated weights for policy 1, policy_version 669617 (0.0007) [2023-12-27 10:15:57,841][20347] Updated weights for policy 0, policy_version 670163 (0.0008) [2023-12-27 10:15:57,899][20347] Updated weights for policy 0, policy_version 670173 (0.0008) [2023-12-27 10:15:57,950][20347] Updated weights for policy 0, policy_version 670183 (0.0007) [2023-12-27 10:15:58,518][20381] Updated weights for policy 1, policy_version 669627 (0.0008) [2023-12-27 10:15:58,579][20381] Updated weights for policy 1, policy_version 669637 (0.0008) [2023-12-27 10:15:58,647][20381] Updated weights for policy 1, policy_version 669647 (0.0008) [2023-12-27 10:15:58,674][20347] Updated weights for policy 0, policy_version 670193 (0.0007) [2023-12-27 10:15:58,723][20347] Updated weights for policy 0, policy_version 670203 (0.0009) [2023-12-27 10:15:58,789][20347] Updated weights for policy 0, policy_version 670213 (0.0008) [2023-12-27 10:15:58,857][20347] Updated weights for policy 0, policy_version 670223 (0.0010) [2023-12-27 10:15:59,374][20381] Updated weights for policy 1, policy_version 669657 (0.0009) [2023-12-27 10:15:59,425][20381] Updated weights for policy 1, policy_version 669667 (0.0005) [2023-12-27 10:15:59,485][20381] Updated weights for policy 1, policy_version 669677 (0.0006) [2023-12-27 10:15:59,541][20381] Updated weights for policy 1, policy_version 669687 (0.0006) [2023-12-27 10:15:59,606][20347] Updated weights for policy 0, policy_version 670233 (0.0006) [2023-12-27 10:15:59,667][20347] Updated weights for policy 0, policy_version 670243 (0.0008) [2023-12-27 10:15:59,721][20347] Updated weights for policy 0, policy_version 670253 (0.0008) [2023-12-27 10:16:00,143][20381] Updated weights for policy 1, policy_version 669697 (0.0005) [2023-12-27 10:16:00,192][20381] Updated weights for policy 1, policy_version 669707 (0.0005) [2023-12-27 10:16:00,241][20381] Updated weights for policy 1, policy_version 669717 (0.0005) [2023-12-27 10:16:00,468][20347] Updated weights for policy 0, policy_version 670263 (0.0008) [2023-12-27 10:16:00,519][20347] Updated weights for policy 0, policy_version 670273 (0.0007) [2023-12-27 10:16:00,575][20347] Updated weights for policy 0, policy_version 670283 (0.0008) [2023-12-27 10:16:00,858][20381] Updated weights for policy 1, policy_version 669727 (0.0009) [2023-12-27 10:16:00,915][20381] Updated weights for policy 1, policy_version 669737 (0.0010) [2023-12-27 10:16:00,972][20381] Updated weights for policy 1, policy_version 669747 (0.0010) [2023-12-27 10:16:01,252][20347] Updated weights for policy 0, policy_version 670293 (0.0007) [2023-12-27 10:16:01,310][20347] Updated weights for policy 0, policy_version 670303 (0.0007) [2023-12-27 10:16:01,376][20347] Updated weights for policy 0, policy_version 670313 (0.0007) [2023-12-27 10:16:01,660][20381] Updated weights for policy 1, policy_version 669757 (0.0011) [2023-12-27 10:16:01,720][20381] Updated weights for policy 1, policy_version 669767 (0.0010) [2023-12-27 10:16:01,780][20381] Updated weights for policy 1, policy_version 669777 (0.0011) [2023-12-27 10:16:02,099][20347] Updated weights for policy 0, policy_version 670323 (0.0008) [2023-12-27 10:16:02,154][20347] Updated weights for policy 0, policy_version 670333 (0.0005) [2023-12-27 10:16:02,207][20347] Updated weights for policy 0, policy_version 670343 (0.0005) [2023-12-27 10:16:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 343121920. Throughput: 0: 10244.5, 1: 10373.9. Samples: 343131092. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:02,363][19170] Avg episode reward: [(0, '152.460'), (1, '951.370')] [2023-12-27 10:16:02,482][20381] Updated weights for policy 1, policy_version 669787 (0.0009) [2023-12-27 10:16:02,536][20381] Updated weights for policy 1, policy_version 669797 (0.0009) [2023-12-27 10:16:02,590][20381] Updated weights for policy 1, policy_version 669807 (0.0008) [2023-12-27 10:16:02,755][20347] Updated weights for policy 0, policy_version 670353 (0.0006) [2023-12-27 10:16:02,811][20347] Updated weights for policy 0, policy_version 670363 (0.0007) [2023-12-27 10:16:02,869][20347] Updated weights for policy 0, policy_version 670373 (0.0008) [2023-12-27 10:16:02,918][20347] Updated weights for policy 0, policy_version 670383 (0.0008) [2023-12-27 10:16:03,313][20381] Updated weights for policy 1, policy_version 669817 (0.0010) [2023-12-27 10:16:03,364][20381] Updated weights for policy 1, policy_version 669827 (0.0008) [2023-12-27 10:16:03,412][20381] Updated weights for policy 1, policy_version 669837 (0.0008) [2023-12-27 10:16:03,461][20381] Updated weights for policy 1, policy_version 669847 (0.0008) [2023-12-27 10:16:03,601][20347] Updated weights for policy 0, policy_version 670393 (0.0007) [2023-12-27 10:16:03,657][20347] Updated weights for policy 0, policy_version 670403 (0.0008) [2023-12-27 10:16:03,710][20347] Updated weights for policy 0, policy_version 670413 (0.0008) [2023-12-27 10:16:04,170][20381] Updated weights for policy 1, policy_version 669857 (0.0008) [2023-12-27 10:16:04,234][20381] Updated weights for policy 1, policy_version 669867 (0.0008) [2023-12-27 10:16:04,295][20381] Updated weights for policy 1, policy_version 669877 (0.0008) [2023-12-27 10:16:04,394][20347] Updated weights for policy 0, policy_version 670423 (0.0008) [2023-12-27 10:16:04,458][20347] Updated weights for policy 0, policy_version 670433 (0.0008) [2023-12-27 10:16:04,518][20347] Updated weights for policy 0, policy_version 670443 (0.0008) [2023-12-27 10:16:05,037][20381] Updated weights for policy 1, policy_version 669887 (0.0007) [2023-12-27 10:16:05,096][20381] Updated weights for policy 1, policy_version 669897 (0.0005) [2023-12-27 10:16:05,141][20347] Updated weights for policy 0, policy_version 670453 (0.0007) [2023-12-27 10:16:05,148][20381] Updated weights for policy 1, policy_version 669907 (0.0007) [2023-12-27 10:16:05,199][20347] Updated weights for policy 0, policy_version 670463 (0.0006) [2023-12-27 10:16:05,252][20347] Updated weights for policy 0, policy_version 670473 (0.0008) [2023-12-27 10:16:05,737][20381] Updated weights for policy 1, policy_version 669917 (0.0007) [2023-12-27 10:16:05,784][20381] Updated weights for policy 1, policy_version 669927 (0.0008) [2023-12-27 10:16:05,831][20381] Updated weights for policy 1, policy_version 669937 (0.0007) [2023-12-27 10:16:05,849][20347] Updated weights for policy 0, policy_version 670483 (0.0007) [2023-12-27 10:16:05,905][20347] Updated weights for policy 0, policy_version 670493 (0.0005) [2023-12-27 10:16:05,960][20347] Updated weights for policy 0, policy_version 670503 (0.0007) [2023-12-27 10:16:06,554][20347] Updated weights for policy 0, policy_version 670513 (0.0007) [2023-12-27 10:16:06,587][20381] Updated weights for policy 1, policy_version 669947 (0.0007) [2023-12-27 10:16:06,618][20347] Updated weights for policy 0, policy_version 670523 (0.0006) [2023-12-27 10:16:06,647][20381] Updated weights for policy 1, policy_version 669957 (0.0005) [2023-12-27 10:16:06,686][20347] Updated weights for policy 0, policy_version 670533 (0.0007) [2023-12-27 10:16:06,702][20381] Updated weights for policy 1, policy_version 669967 (0.0005) [2023-12-27 10:16:06,739][20347] Updated weights for policy 0, policy_version 670543 (0.0008) [2023-12-27 10:16:07,323][20347] Updated weights for policy 0, policy_version 670553 (0.0007) [2023-12-27 10:16:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20716.0). Total num frames: 343220224. Throughput: 0: 10328.9, 1: 10319.4. Samples: 343192884. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:07,364][19170] Avg episode reward: [(0, '185.110'), (1, '951.330')] [2023-12-27 10:16:07,381][20381] Updated weights for policy 1, policy_version 669977 (0.0006) [2023-12-27 10:16:07,382][20347] Updated weights for policy 0, policy_version 670563 (0.0007) [2023-12-27 10:16:07,445][20381] Updated weights for policy 1, policy_version 669987 (0.0008) [2023-12-27 10:16:07,446][20347] Updated weights for policy 0, policy_version 670573 (0.0007) [2023-12-27 10:16:07,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000670576_171687936.pth... [2023-12-27 10:16:07,473][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000669360_171376640.pth [2023-12-27 10:16:07,503][20381] Updated weights for policy 1, policy_version 669997 (0.0007) [2023-12-27 10:16:07,559][20381] Updated weights for policy 1, policy_version 670007 (0.0008) [2023-12-27 10:16:07,562][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000670008_171548672.pth... [2023-12-27 10:16:07,566][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000668792_171237376.pth [2023-12-27 10:16:08,060][20347] Updated weights for policy 0, policy_version 670583 (0.0006) [2023-12-27 10:16:08,113][20347] Updated weights for policy 0, policy_version 670593 (0.0006) [2023-12-27 10:16:08,162][20347] Updated weights for policy 0, policy_version 670603 (0.0005) [2023-12-27 10:16:08,287][20381] Updated weights for policy 1, policy_version 670017 (0.0008) [2023-12-27 10:16:08,347][20381] Updated weights for policy 1, policy_version 670027 (0.0008) [2023-12-27 10:16:08,405][20381] Updated weights for policy 1, policy_version 670037 (0.0007) [2023-12-27 10:16:08,771][20347] Updated weights for policy 0, policy_version 670613 (0.0008) [2023-12-27 10:16:08,827][20347] Updated weights for policy 0, policy_version 670623 (0.0010) [2023-12-27 10:16:08,887][20347] Updated weights for policy 0, policy_version 670633 (0.0010) [2023-12-27 10:16:09,044][20381] Updated weights for policy 1, policy_version 670047 (0.0008) [2023-12-27 10:16:09,111][20381] Updated weights for policy 1, policy_version 670057 (0.0008) [2023-12-27 10:16:09,164][20381] Updated weights for policy 1, policy_version 670067 (0.0007) [2023-12-27 10:16:09,639][20347] Updated weights for policy 0, policy_version 670643 (0.0010) [2023-12-27 10:16:09,695][20347] Updated weights for policy 0, policy_version 670653 (0.0008) [2023-12-27 10:16:09,754][20347] Updated weights for policy 0, policy_version 670663 (0.0008) [2023-12-27 10:16:09,831][20381] Updated weights for policy 1, policy_version 670077 (0.0006) [2023-12-27 10:16:09,886][20381] Updated weights for policy 1, policy_version 670087 (0.0006) [2023-12-27 10:16:09,951][20381] Updated weights for policy 1, policy_version 670097 (0.0007) [2023-12-27 10:16:10,540][20381] Updated weights for policy 1, policy_version 670107 (0.0007) [2023-12-27 10:16:10,560][20347] Updated weights for policy 0, policy_version 670673 (0.0008) [2023-12-27 10:16:10,597][20381] Updated weights for policy 1, policy_version 670117 (0.0007) [2023-12-27 10:16:10,610][20347] Updated weights for policy 0, policy_version 670683 (0.0006) [2023-12-27 10:16:10,653][20381] Updated weights for policy 1, policy_version 670127 (0.0005) [2023-12-27 10:16:10,663][20347] Updated weights for policy 0, policy_version 670693 (0.0006) [2023-12-27 10:16:10,718][20347] Updated weights for policy 0, policy_version 670703 (0.0006) [2023-12-27 10:16:11,421][20347] Updated weights for policy 0, policy_version 670713 (0.0008) [2023-12-27 10:16:11,428][20381] Updated weights for policy 1, policy_version 670137 (0.0006) [2023-12-27 10:16:11,478][20347] Updated weights for policy 0, policy_version 670723 (0.0006) [2023-12-27 10:16:11,487][20381] Updated weights for policy 1, policy_version 670147 (0.0006) [2023-12-27 10:16:11,540][20347] Updated weights for policy 0, policy_version 670733 (0.0008) [2023-12-27 10:16:11,544][20381] Updated weights for policy 1, policy_version 670157 (0.0006) [2023-12-27 10:16:11,604][20381] Updated weights for policy 1, policy_version 670167 (0.0006) [2023-12-27 10:16:12,193][20347] Updated weights for policy 0, policy_version 670743 (0.0008) [2023-12-27 10:16:12,244][20347] Updated weights for policy 0, policy_version 670753 (0.0007) [2023-12-27 10:16:12,288][20381] Updated weights for policy 1, policy_version 670177 (0.0006) [2023-12-27 10:16:12,304][20347] Updated weights for policy 0, policy_version 670763 (0.0008) [2023-12-27 10:16:12,350][20381] Updated weights for policy 1, policy_version 670187 (0.0007) [2023-12-27 10:16:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 343326720. Throughput: 0: 10372.6, 1: 10405.0. Samples: 343317684. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:12,363][19170] Avg episode reward: [(0, '196.290'), (1, '990.280')] [2023-12-27 10:16:12,411][20381] Updated weights for policy 1, policy_version 670197 (0.0008) [2023-12-27 10:16:13,004][20347] Updated weights for policy 0, policy_version 670773 (0.0007) [2023-12-27 10:16:13,065][20347] Updated weights for policy 0, policy_version 670783 (0.0008) [2023-12-27 10:16:13,116][20381] Updated weights for policy 1, policy_version 670207 (0.0007) [2023-12-27 10:16:13,123][20347] Updated weights for policy 0, policy_version 670793 (0.0006) [2023-12-27 10:16:13,166][20381] Updated weights for policy 1, policy_version 670217 (0.0008) [2023-12-27 10:16:13,224][20381] Updated weights for policy 1, policy_version 670227 (0.0007) [2023-12-27 10:16:13,709][20347] Updated weights for policy 0, policy_version 670803 (0.0005) [2023-12-27 10:16:13,757][20347] Updated weights for policy 0, policy_version 670813 (0.0008) [2023-12-27 10:16:13,809][20347] Updated weights for policy 0, policy_version 670823 (0.0010) [2023-12-27 10:16:13,932][20381] Updated weights for policy 1, policy_version 670237 (0.0007) [2023-12-27 10:16:13,987][20381] Updated weights for policy 1, policy_version 670247 (0.0008) [2023-12-27 10:16:14,038][20381] Updated weights for policy 1, policy_version 670257 (0.0007) [2023-12-27 10:16:14,484][20347] Updated weights for policy 0, policy_version 670833 (0.0010) [2023-12-27 10:16:14,539][20347] Updated weights for policy 0, policy_version 670843 (0.0010) [2023-12-27 10:16:14,600][20347] Updated weights for policy 0, policy_version 670853 (0.0010) [2023-12-27 10:16:14,655][20347] Updated weights for policy 0, policy_version 670863 (0.0010) [2023-12-27 10:16:14,700][20381] Updated weights for policy 1, policy_version 670267 (0.0008) [2023-12-27 10:16:14,745][20381] Updated weights for policy 1, policy_version 670277 (0.0011) [2023-12-27 10:16:14,805][20381] Updated weights for policy 1, policy_version 670287 (0.0011) [2023-12-27 10:16:15,329][20347] Updated weights for policy 0, policy_version 670873 (0.0010) [2023-12-27 10:16:15,378][20347] Updated weights for policy 0, policy_version 670883 (0.0010) [2023-12-27 10:16:15,430][20347] Updated weights for policy 0, policy_version 670893 (0.0010) [2023-12-27 10:16:15,461][20381] Updated weights for policy 1, policy_version 670297 (0.0006) [2023-12-27 10:16:15,513][20381] Updated weights for policy 1, policy_version 670307 (0.0010) [2023-12-27 10:16:15,566][20381] Updated weights for policy 1, policy_version 670317 (0.0010) [2023-12-27 10:16:15,617][20381] Updated weights for policy 1, policy_version 670327 (0.0010) [2023-12-27 10:16:16,053][20347] Updated weights for policy 0, policy_version 670903 (0.0007) [2023-12-27 10:16:16,108][20347] Updated weights for policy 0, policy_version 670913 (0.0006) [2023-12-27 10:16:16,161][20347] Updated weights for policy 0, policy_version 670923 (0.0010) [2023-12-27 10:16:16,318][20381] Updated weights for policy 1, policy_version 670337 (0.0010) [2023-12-27 10:16:16,371][20381] Updated weights for policy 1, policy_version 670347 (0.0010) [2023-12-27 10:16:16,422][20381] Updated weights for policy 1, policy_version 670357 (0.0010) [2023-12-27 10:16:16,788][20347] Updated weights for policy 0, policy_version 670933 (0.0008) [2023-12-27 10:16:16,849][20347] Updated weights for policy 0, policy_version 670943 (0.0008) [2023-12-27 10:16:16,902][20347] Updated weights for policy 0, policy_version 670953 (0.0005) [2023-12-27 10:16:17,092][20381] Updated weights for policy 1, policy_version 670367 (0.0007) [2023-12-27 10:16:17,142][20381] Updated weights for policy 1, policy_version 670377 (0.0007) [2023-12-27 10:16:17,193][20381] Updated weights for policy 1, policy_version 670387 (0.0006) [2023-12-27 10:16:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.2, 300 sec: 20743.8). Total num frames: 343433216. Throughput: 0: 10322.1, 1: 10337.3. Samples: 343442276. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:17,363][19170] Avg episode reward: [(0, '209.330'), (1, '1000.000')] [2023-12-27 10:16:17,437][20347] Updated weights for policy 0, policy_version 670963 (0.0005) [2023-12-27 10:16:17,497][20347] Updated weights for policy 0, policy_version 670973 (0.0005) [2023-12-27 10:16:17,563][20347] Updated weights for policy 0, policy_version 670983 (0.0008) [2023-12-27 10:16:17,798][20381] Updated weights for policy 1, policy_version 670397 (0.0006) [2023-12-27 10:16:17,855][20381] Updated weights for policy 1, policy_version 670407 (0.0005) [2023-12-27 10:16:17,917][20381] Updated weights for policy 1, policy_version 670417 (0.0005) [2023-12-27 10:16:18,203][20347] Updated weights for policy 0, policy_version 670993 (0.0010) [2023-12-27 10:16:18,262][20347] Updated weights for policy 0, policy_version 671003 (0.0009) [2023-12-27 10:16:18,321][20347] Updated weights for policy 0, policy_version 671013 (0.0006) [2023-12-27 10:16:18,382][20347] Updated weights for policy 0, policy_version 671023 (0.0007) [2023-12-27 10:16:18,509][20381] Updated weights for policy 1, policy_version 670427 (0.0006) [2023-12-27 10:16:18,570][20381] Updated weights for policy 1, policy_version 670437 (0.0008) [2023-12-27 10:16:18,633][20381] Updated weights for policy 1, policy_version 670447 (0.0008) [2023-12-27 10:16:19,001][20347] Updated weights for policy 0, policy_version 671033 (0.0007) [2023-12-27 10:16:19,073][20347] Updated weights for policy 0, policy_version 671043 (0.0006) [2023-12-27 10:16:19,127][20347] Updated weights for policy 0, policy_version 671053 (0.0007) [2023-12-27 10:16:19,280][20381] Updated weights for policy 1, policy_version 670457 (0.0008) [2023-12-27 10:16:19,345][20381] Updated weights for policy 1, policy_version 670467 (0.0006) [2023-12-27 10:16:19,407][20381] Updated weights for policy 1, policy_version 670477 (0.0007) [2023-12-27 10:16:19,470][20381] Updated weights for policy 1, policy_version 670487 (0.0007) [2023-12-27 10:16:19,816][20347] Updated weights for policy 0, policy_version 671063 (0.0008) [2023-12-27 10:16:19,877][20347] Updated weights for policy 0, policy_version 671073 (0.0008) [2023-12-27 10:16:19,937][20347] Updated weights for policy 0, policy_version 671083 (0.0008) [2023-12-27 10:16:20,113][20381] Updated weights for policy 1, policy_version 670497 (0.0005) [2023-12-27 10:16:20,171][20381] Updated weights for policy 1, policy_version 670507 (0.0007) [2023-12-27 10:16:20,235][20381] Updated weights for policy 1, policy_version 670517 (0.0007) [2023-12-27 10:16:20,634][20347] Updated weights for policy 0, policy_version 671093 (0.0008) [2023-12-27 10:16:20,681][20347] Updated weights for policy 0, policy_version 671103 (0.0007) [2023-12-27 10:16:20,731][20347] Updated weights for policy 0, policy_version 671113 (0.0008) [2023-12-27 10:16:20,878][20381] Updated weights for policy 1, policy_version 670527 (0.0008) [2023-12-27 10:16:20,939][20381] Updated weights for policy 1, policy_version 670537 (0.0007) [2023-12-27 10:16:20,988][20381] Updated weights for policy 1, policy_version 670547 (0.0007) [2023-12-27 10:16:21,513][20347] Updated weights for policy 0, policy_version 671123 (0.0007) [2023-12-27 10:16:21,577][20347] Updated weights for policy 0, policy_version 671133 (0.0007) [2023-12-27 10:16:21,646][20347] Updated weights for policy 0, policy_version 671143 (0.0008) [2023-12-27 10:16:21,699][20381] Updated weights for policy 1, policy_version 670557 (0.0007) [2023-12-27 10:16:21,766][20381] Updated weights for policy 1, policy_version 670567 (0.0007) [2023-12-27 10:16:21,821][20381] Updated weights for policy 1, policy_version 670577 (0.0007) [2023-12-27 10:16:22,278][20347] Updated weights for policy 0, policy_version 671153 (0.0006) [2023-12-27 10:16:22,341][20347] Updated weights for policy 0, policy_version 671163 (0.0008) [2023-12-27 10:16:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 343531520. Throughput: 0: 10317.6, 1: 10374.5. Samples: 343506304. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:22,363][19170] Avg episode reward: [(0, '207.340'), (1, '1000.000')] [2023-12-27 10:16:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000670584_171696128.pth... [2023-12-27 10:16:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000669368_171384832.pth [2023-12-27 10:16:22,403][20347] Updated weights for policy 0, policy_version 671173 (0.0008) [2023-12-27 10:16:22,466][20347] Updated weights for policy 0, policy_version 671183 (0.0008) [2023-12-27 10:16:22,469][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000671184_171843584.pth... [2023-12-27 10:16:22,473][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000669968_171532288.pth [2023-12-27 10:16:22,565][20381] Updated weights for policy 1, policy_version 670587 (0.0007) [2023-12-27 10:16:22,621][20381] Updated weights for policy 1, policy_version 670597 (0.0008) [2023-12-27 10:16:22,684][20381] Updated weights for policy 1, policy_version 670607 (0.0008) [2023-12-27 10:16:23,180][20347] Updated weights for policy 0, policy_version 671193 (0.0008) [2023-12-27 10:16:23,235][20347] Updated weights for policy 0, policy_version 671203 (0.0007) [2023-12-27 10:16:23,293][20347] Updated weights for policy 0, policy_version 671213 (0.0007) [2023-12-27 10:16:23,383][20381] Updated weights for policy 1, policy_version 670617 (0.0008) [2023-12-27 10:16:23,438][20381] Updated weights for policy 1, policy_version 670627 (0.0008) [2023-12-27 10:16:23,489][20381] Updated weights for policy 1, policy_version 670637 (0.0008) [2023-12-27 10:16:23,544][20381] Updated weights for policy 1, policy_version 670647 (0.0007) [2023-12-27 10:16:23,946][20347] Updated weights for policy 0, policy_version 671223 (0.0007) [2023-12-27 10:16:24,003][20347] Updated weights for policy 0, policy_version 671233 (0.0007) [2023-12-27 10:16:24,065][20347] Updated weights for policy 0, policy_version 671243 (0.0008) [2023-12-27 10:16:24,233][20381] Updated weights for policy 1, policy_version 670657 (0.0006) [2023-12-27 10:16:24,288][20381] Updated weights for policy 1, policy_version 670667 (0.0010) [2023-12-27 10:16:24,338][20381] Updated weights for policy 1, policy_version 670677 (0.0010) [2023-12-27 10:16:24,753][20347] Updated weights for policy 0, policy_version 671253 (0.0007) [2023-12-27 10:16:24,812][20347] Updated weights for policy 0, policy_version 671263 (0.0008) [2023-12-27 10:16:24,875][20347] Updated weights for policy 0, policy_version 671273 (0.0008) [2023-12-27 10:16:24,914][20381] Updated weights for policy 1, policy_version 670687 (0.0006) [2023-12-27 10:16:24,965][20381] Updated weights for policy 1, policy_version 670697 (0.0005) [2023-12-27 10:16:25,017][20381] Updated weights for policy 1, policy_version 670707 (0.0005) [2023-12-27 10:16:25,462][20347] Updated weights for policy 0, policy_version 671283 (0.0007) [2023-12-27 10:16:25,511][20347] Updated weights for policy 0, policy_version 671293 (0.0005) [2023-12-27 10:16:25,566][20347] Updated weights for policy 0, policy_version 671303 (0.0005) [2023-12-27 10:16:25,715][20381] Updated weights for policy 1, policy_version 670717 (0.0005) [2023-12-27 10:16:25,762][20381] Updated weights for policy 1, policy_version 670727 (0.0005) [2023-12-27 10:16:25,805][20381] Updated weights for policy 1, policy_version 670737 (0.0005) [2023-12-27 10:16:26,152][20347] Updated weights for policy 0, policy_version 671313 (0.0006) [2023-12-27 10:16:26,203][20347] Updated weights for policy 0, policy_version 671323 (0.0010) [2023-12-27 10:16:26,251][20347] Updated weights for policy 0, policy_version 671333 (0.0010) [2023-12-27 10:16:26,300][20347] Updated weights for policy 0, policy_version 671343 (0.0010) [2023-12-27 10:16:26,403][20381] Updated weights for policy 1, policy_version 670747 (0.0006) [2023-12-27 10:16:26,453][20381] Updated weights for policy 1, policy_version 670757 (0.0007) [2023-12-27 10:16:26,507][20381] Updated weights for policy 1, policy_version 670767 (0.0007) [2023-12-27 10:16:26,982][20347] Updated weights for policy 0, policy_version 671353 (0.0010) [2023-12-27 10:16:27,041][20347] Updated weights for policy 0, policy_version 671363 (0.0010) [2023-12-27 10:16:27,095][20347] Updated weights for policy 0, policy_version 671373 (0.0010) [2023-12-27 10:16:27,206][20381] Updated weights for policy 1, policy_version 670777 (0.0007) [2023-12-27 10:16:27,253][20381] Updated weights for policy 1, policy_version 670787 (0.0007) [2023-12-27 10:16:27,305][20381] Updated weights for policy 1, policy_version 670797 (0.0007) [2023-12-27 10:16:27,361][20381] Updated weights for policy 1, policy_version 670807 (0.0008) [2023-12-27 10:16:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 343638016. Throughput: 0: 10368.7, 1: 10362.1. Samples: 343630504. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:27,363][19170] Avg episode reward: [(0, '264.830'), (1, '990.310')] [2023-12-27 10:16:27,764][20347] Updated weights for policy 0, policy_version 671383 (0.0007) [2023-12-27 10:16:27,817][20347] Updated weights for policy 0, policy_version 671393 (0.0005) [2023-12-27 10:16:27,868][20347] Updated weights for policy 0, policy_version 671403 (0.0006) [2023-12-27 10:16:27,959][20381] Updated weights for policy 1, policy_version 670817 (0.0005) [2023-12-27 10:16:28,012][20381] Updated weights for policy 1, policy_version 670827 (0.0006) [2023-12-27 10:16:28,060][20381] Updated weights for policy 1, policy_version 670837 (0.0006) [2023-12-27 10:16:28,505][20347] Updated weights for policy 0, policy_version 671413 (0.0010) [2023-12-27 10:16:28,567][20347] Updated weights for policy 0, policy_version 671423 (0.0010) [2023-12-27 10:16:28,627][20347] Updated weights for policy 0, policy_version 671433 (0.0010) [2023-12-27 10:16:28,716][20381] Updated weights for policy 1, policy_version 670847 (0.0007) [2023-12-27 10:16:28,771][20381] Updated weights for policy 1, policy_version 670857 (0.0008) [2023-12-27 10:16:28,828][20381] Updated weights for policy 1, policy_version 670867 (0.0008) [2023-12-27 10:16:29,245][20347] Updated weights for policy 0, policy_version 671443 (0.0010) [2023-12-27 10:16:29,304][20347] Updated weights for policy 0, policy_version 671453 (0.0008) [2023-12-27 10:16:29,369][20347] Updated weights for policy 0, policy_version 671463 (0.0010) [2023-12-27 10:16:29,501][20381] Updated weights for policy 1, policy_version 670877 (0.0008) [2023-12-27 10:16:29,562][20381] Updated weights for policy 1, policy_version 670887 (0.0008) [2023-12-27 10:16:29,630][20381] Updated weights for policy 1, policy_version 670897 (0.0008) [2023-12-27 10:16:30,034][20347] Updated weights for policy 0, policy_version 671473 (0.0010) [2023-12-27 10:16:30,088][20347] Updated weights for policy 0, policy_version 671483 (0.0010) [2023-12-27 10:16:30,145][20347] Updated weights for policy 0, policy_version 671493 (0.0010) [2023-12-27 10:16:30,218][20347] Updated weights for policy 0, policy_version 671503 (0.0007) [2023-12-27 10:16:30,321][20381] Updated weights for policy 1, policy_version 670907 (0.0008) [2023-12-27 10:16:30,375][20381] Updated weights for policy 1, policy_version 670917 (0.0007) [2023-12-27 10:16:30,424][20381] Updated weights for policy 1, policy_version 670927 (0.0007) [2023-12-27 10:16:30,832][20347] Updated weights for policy 0, policy_version 671513 (0.0006) [2023-12-27 10:16:30,884][20347] Updated weights for policy 0, policy_version 671523 (0.0005) [2023-12-27 10:16:30,932][20347] Updated weights for policy 0, policy_version 671533 (0.0007) [2023-12-27 10:16:31,098][20381] Updated weights for policy 1, policy_version 670937 (0.0006) [2023-12-27 10:16:31,159][20381] Updated weights for policy 1, policy_version 670947 (0.0008) [2023-12-27 10:16:31,244][20381] Updated weights for policy 1, policy_version 670957 (0.0008) [2023-12-27 10:16:31,298][20381] Updated weights for policy 1, policy_version 670967 (0.0008) [2023-12-27 10:16:31,619][20347] Updated weights for policy 0, policy_version 671543 (0.0007) [2023-12-27 10:16:31,679][20347] Updated weights for policy 0, policy_version 671553 (0.0007) [2023-12-27 10:16:31,735][20347] Updated weights for policy 0, policy_version 671563 (0.0007) [2023-12-27 10:16:31,991][20381] Updated weights for policy 1, policy_version 670977 (0.0010) [2023-12-27 10:16:32,054][20381] Updated weights for policy 1, policy_version 670987 (0.0011) [2023-12-27 10:16:32,113][20381] Updated weights for policy 1, policy_version 670997 (0.0011) [2023-12-27 10:16:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 343744512. Throughput: 0: 10324.2, 1: 10338.8. Samples: 343754884. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:32,363][19170] Avg episode reward: [(0, '282.440'), (1, '990.310')] [2023-12-27 10:16:32,450][20347] Updated weights for policy 0, policy_version 671573 (0.0008) [2023-12-27 10:16:32,497][20347] Updated weights for policy 0, policy_version 671583 (0.0007) [2023-12-27 10:16:32,548][20347] Updated weights for policy 0, policy_version 671593 (0.0005) [2023-12-27 10:16:32,786][20381] Updated weights for policy 1, policy_version 671007 (0.0007) [2023-12-27 10:16:32,846][20381] Updated weights for policy 1, policy_version 671017 (0.0006) [2023-12-27 10:16:32,911][20381] Updated weights for policy 1, policy_version 671027 (0.0005) [2023-12-27 10:16:33,155][20347] Updated weights for policy 0, policy_version 671603 (0.0010) [2023-12-27 10:16:33,198][20347] Updated weights for policy 0, policy_version 671613 (0.0009) [2023-12-27 10:16:33,244][20347] Updated weights for policy 0, policy_version 671623 (0.0007) [2023-12-27 10:16:33,466][20381] Updated weights for policy 1, policy_version 671037 (0.0008) [2023-12-27 10:16:33,523][20381] Updated weights for policy 1, policy_version 671047 (0.0010) [2023-12-27 10:16:33,577][20381] Updated weights for policy 1, policy_version 671057 (0.0010) [2023-12-27 10:16:33,907][20347] Updated weights for policy 0, policy_version 671633 (0.0006) [2023-12-27 10:16:33,967][20347] Updated weights for policy 0, policy_version 671643 (0.0010) [2023-12-27 10:16:34,011][20347] Updated weights for policy 0, policy_version 671653 (0.0009) [2023-12-27 10:16:34,070][20347] Updated weights for policy 0, policy_version 671663 (0.0010) [2023-12-27 10:16:34,142][20381] Updated weights for policy 1, policy_version 671067 (0.0009) [2023-12-27 10:16:34,208][20381] Updated weights for policy 1, policy_version 671077 (0.0009) [2023-12-27 10:16:34,268][20381] Updated weights for policy 1, policy_version 671087 (0.0011) [2023-12-27 10:16:34,733][20347] Updated weights for policy 0, policy_version 671673 (0.0008) [2023-12-27 10:16:34,793][20347] Updated weights for policy 0, policy_version 671683 (0.0007) [2023-12-27 10:16:34,849][20347] Updated weights for policy 0, policy_version 671693 (0.0007) [2023-12-27 10:16:34,961][20381] Updated weights for policy 1, policy_version 671097 (0.0011) [2023-12-27 10:16:35,013][20381] Updated weights for policy 1, policy_version 671107 (0.0011) [2023-12-27 10:16:35,061][20381] Updated weights for policy 1, policy_version 671117 (0.0010) [2023-12-27 10:16:35,119][20381] Updated weights for policy 1, policy_version 671127 (0.0011) [2023-12-27 10:16:35,440][20347] Updated weights for policy 0, policy_version 671703 (0.0007) [2023-12-27 10:16:35,500][20347] Updated weights for policy 0, policy_version 671713 (0.0007) [2023-12-27 10:16:35,555][20347] Updated weights for policy 0, policy_version 671723 (0.0007) [2023-12-27 10:16:35,813][20381] Updated weights for policy 1, policy_version 671137 (0.0011) [2023-12-27 10:16:35,867][20381] Updated weights for policy 1, policy_version 671147 (0.0010) [2023-12-27 10:16:35,918][20381] Updated weights for policy 1, policy_version 671157 (0.0010) [2023-12-27 10:16:36,258][20347] Updated weights for policy 0, policy_version 671733 (0.0008) [2023-12-27 10:16:36,319][20347] Updated weights for policy 0, policy_version 671743 (0.0007) [2023-12-27 10:16:36,384][20347] Updated weights for policy 0, policy_version 671753 (0.0008) [2023-12-27 10:16:36,634][20381] Updated weights for policy 1, policy_version 671167 (0.0011) [2023-12-27 10:16:36,697][20381] Updated weights for policy 1, policy_version 671177 (0.0010) [2023-12-27 10:16:36,752][20381] Updated weights for policy 1, policy_version 671187 (0.0010) [2023-12-27 10:16:37,052][20347] Updated weights for policy 0, policy_version 671763 (0.0009) [2023-12-27 10:16:37,101][20347] Updated weights for policy 0, policy_version 671773 (0.0010) [2023-12-27 10:16:37,153][20347] Updated weights for policy 0, policy_version 671783 (0.0010) [2023-12-27 10:16:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 343851008. Throughput: 0: 10333.2, 1: 10408.7. Samples: 343819564. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) [2023-12-27 10:16:37,363][19170] Avg episode reward: [(0, '240.990'), (1, '1000.000')] [2023-12-27 10:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000671792_171999232.pth... [2023-12-27 10:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000671192_171851776.pth... [2023-12-27 10:16:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000670008_171548672.pth [2023-12-27 10:16:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000670576_171687936.pth [2023-12-27 10:16:37,421][20381] Updated weights for policy 1, policy_version 671197 (0.0011) [2023-12-27 10:16:37,477][20381] Updated weights for policy 1, policy_version 671207 (0.0009) [2023-12-27 10:16:37,538][20381] Updated weights for policy 1, policy_version 671217 (0.0005) [2023-12-27 10:16:37,852][20347] Updated weights for policy 0, policy_version 671793 (0.0010) [2023-12-27 10:16:37,899][20347] Updated weights for policy 0, policy_version 671803 (0.0010) [2023-12-27 10:16:37,956][20347] Updated weights for policy 0, policy_version 671813 (0.0011) [2023-12-27 10:16:38,013][20347] Updated weights for policy 0, policy_version 671823 (0.0009) [2023-12-27 10:16:38,151][20381] Updated weights for policy 1, policy_version 671227 (0.0007) [2023-12-27 10:16:38,210][20381] Updated weights for policy 1, policy_version 671237 (0.0010) [2023-12-27 10:16:38,266][20381] Updated weights for policy 1, policy_version 671247 (0.0011) [2023-12-27 10:16:38,709][20347] Updated weights for policy 0, policy_version 671833 (0.0010) [2023-12-27 10:16:38,762][20347] Updated weights for policy 0, policy_version 671843 (0.0010) [2023-12-27 10:16:38,811][20347] Updated weights for policy 0, policy_version 671853 (0.0010) [2023-12-27 10:16:38,941][20381] Updated weights for policy 1, policy_version 671257 (0.0010) [2023-12-27 10:16:38,993][20381] Updated weights for policy 1, policy_version 671267 (0.0011) [2023-12-27 10:16:39,045][20381] Updated weights for policy 1, policy_version 671277 (0.0011) [2023-12-27 10:16:39,100][20381] Updated weights for policy 1, policy_version 671287 (0.0011) [2023-12-27 10:16:39,513][20347] Updated weights for policy 0, policy_version 671863 (0.0009) [2023-12-27 10:16:39,571][20347] Updated weights for policy 0, policy_version 671873 (0.0007) [2023-12-27 10:16:39,629][20347] Updated weights for policy 0, policy_version 671883 (0.0007) [2023-12-27 10:16:39,810][20381] Updated weights for policy 1, policy_version 671297 (0.0011) [2023-12-27 10:16:39,870][20381] Updated weights for policy 1, policy_version 671307 (0.0011) [2023-12-27 10:16:39,929][20381] Updated weights for policy 1, policy_version 671317 (0.0011) [2023-12-27 10:16:40,284][20347] Updated weights for policy 0, policy_version 671893 (0.0008) [2023-12-27 10:16:40,341][20347] Updated weights for policy 0, policy_version 671903 (0.0009) [2023-12-27 10:16:40,402][20347] Updated weights for policy 0, policy_version 671913 (0.0007) [2023-12-27 10:16:40,603][20381] Updated weights for policy 1, policy_version 671327 (0.0011) [2023-12-27 10:16:40,660][20381] Updated weights for policy 1, policy_version 671337 (0.0011) [2023-12-27 10:16:40,722][20381] Updated weights for policy 1, policy_version 671347 (0.0011) [2023-12-27 10:16:41,098][20347] Updated weights for policy 0, policy_version 671923 (0.0007) [2023-12-27 10:16:41,165][20347] Updated weights for policy 0, policy_version 671933 (0.0008) [2023-12-27 10:16:41,221][20347] Updated weights for policy 0, policy_version 671943 (0.0008) [2023-12-27 10:16:41,468][20381] Updated weights for policy 1, policy_version 671357 (0.0009) [2023-12-27 10:16:41,515][20381] Updated weights for policy 1, policy_version 671367 (0.0008) [2023-12-27 10:16:41,574][20381] Updated weights for policy 1, policy_version 671377 (0.0008) [2023-12-27 10:16:42,005][20347] Updated weights for policy 0, policy_version 671953 (0.0008) [2023-12-27 10:16:42,068][20347] Updated weights for policy 0, policy_version 671963 (0.0007) [2023-12-27 10:16:42,127][20347] Updated weights for policy 0, policy_version 671973 (0.0008) [2023-12-27 10:16:42,197][20347] Updated weights for policy 0, policy_version 671983 (0.0008) [2023-12-27 10:16:42,242][20381] Updated weights for policy 1, policy_version 671387 (0.0007) [2023-12-27 10:16:42,297][20381] Updated weights for policy 1, policy_version 671397 (0.0010) [2023-12-27 10:16:42,351][20381] Updated weights for policy 1, policy_version 671407 (0.0011) [2023-12-27 10:16:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 343949312. Throughput: 0: 10379.7, 1: 10327.0. Samples: 343942136. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:16:42,363][19170] Avg episode reward: [(0, '274.110'), (1, '1000.000')] [2023-12-27 10:16:42,899][20347] Updated weights for policy 0, policy_version 671993 (0.0007) [2023-12-27 10:16:42,959][20347] Updated weights for policy 0, policy_version 672003 (0.0007) [2023-12-27 10:16:43,016][20347] Updated weights for policy 0, policy_version 672013 (0.0007) [2023-12-27 10:16:43,066][20381] Updated weights for policy 1, policy_version 671417 (0.0009) [2023-12-27 10:16:43,121][20381] Updated weights for policy 1, policy_version 671427 (0.0010) [2023-12-27 10:16:43,169][20381] Updated weights for policy 1, policy_version 671437 (0.0010) [2023-12-27 10:16:43,218][20381] Updated weights for policy 1, policy_version 671447 (0.0010) [2023-12-27 10:16:43,574][20347] Updated weights for policy 0, policy_version 672023 (0.0006) [2023-12-27 10:16:43,631][20347] Updated weights for policy 0, policy_version 672033 (0.0006) [2023-12-27 10:16:43,681][20347] Updated weights for policy 0, policy_version 672043 (0.0007) [2023-12-27 10:16:43,907][20381] Updated weights for policy 1, policy_version 671457 (0.0009) [2023-12-27 10:16:43,956][20381] Updated weights for policy 1, policy_version 671467 (0.0008) [2023-12-27 10:16:44,008][20381] Updated weights for policy 1, policy_version 671477 (0.0008) [2023-12-27 10:16:44,291][20347] Updated weights for policy 0, policy_version 672053 (0.0006) [2023-12-27 10:16:44,339][20347] Updated weights for policy 0, policy_version 672063 (0.0006) [2023-12-27 10:16:44,391][20347] Updated weights for policy 0, policy_version 672073 (0.0007) [2023-12-27 10:16:44,694][20381] Updated weights for policy 1, policy_version 671487 (0.0008) [2023-12-27 10:16:44,743][20381] Updated weights for policy 1, policy_version 671497 (0.0008) [2023-12-27 10:16:44,807][20381] Updated weights for policy 1, policy_version 671507 (0.0008) [2023-12-27 10:16:45,056][20347] Updated weights for policy 0, policy_version 672083 (0.0007) [2023-12-27 10:16:45,115][20347] Updated weights for policy 0, policy_version 672093 (0.0007) [2023-12-27 10:16:45,174][20347] Updated weights for policy 0, policy_version 672103 (0.0008) [2023-12-27 10:16:45,498][20381] Updated weights for policy 1, policy_version 671517 (0.0008) [2023-12-27 10:16:45,557][20381] Updated weights for policy 1, policy_version 671527 (0.0011) [2023-12-27 10:16:45,618][20381] Updated weights for policy 1, policy_version 671537 (0.0011) [2023-12-27 10:16:45,877][20347] Updated weights for policy 0, policy_version 672113 (0.0008) [2023-12-27 10:16:45,927][20347] Updated weights for policy 0, policy_version 672123 (0.0007) [2023-12-27 10:16:45,987][20347] Updated weights for policy 0, policy_version 672133 (0.0007) [2023-12-27 10:16:46,051][20347] Updated weights for policy 0, policy_version 672143 (0.0008) [2023-12-27 10:16:46,226][20381] Updated weights for policy 1, policy_version 671547 (0.0011) [2023-12-27 10:16:46,285][20381] Updated weights for policy 1, policy_version 671557 (0.0010) [2023-12-27 10:16:46,336][20381] Updated weights for policy 1, policy_version 671567 (0.0010) [2023-12-27 10:16:46,755][20347] Updated weights for policy 0, policy_version 672153 (0.0008) [2023-12-27 10:16:46,806][20347] Updated weights for policy 0, policy_version 672163 (0.0008) [2023-12-27 10:16:46,863][20347] Updated weights for policy 0, policy_version 672173 (0.0007) [2023-12-27 10:16:47,020][20381] Updated weights for policy 1, policy_version 671577 (0.0011) [2023-12-27 10:16:47,081][20381] Updated weights for policy 1, policy_version 671587 (0.0011) [2023-12-27 10:16:47,135][20381] Updated weights for policy 1, policy_version 671597 (0.0010) [2023-12-27 10:16:47,196][20381] Updated weights for policy 1, policy_version 671607 (0.0010) [2023-12-27 10:16:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 344055808. Throughput: 0: 10457.2, 1: 10282.9. Samples: 344064396. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:16:47,363][19170] Avg episode reward: [(0, '317.100'), (1, '1000.000')] [2023-12-27 10:16:47,556][20347] Updated weights for policy 0, policy_version 672183 (0.0007) [2023-12-27 10:16:47,609][20347] Updated weights for policy 0, policy_version 672193 (0.0007) [2023-12-27 10:16:47,663][20347] Updated weights for policy 0, policy_version 672203 (0.0007) [2023-12-27 10:16:47,884][20381] Updated weights for policy 1, policy_version 671617 (0.0008) [2023-12-27 10:16:47,939][20381] Updated weights for policy 1, policy_version 671627 (0.0007) [2023-12-27 10:16:47,995][20381] Updated weights for policy 1, policy_version 671637 (0.0008) [2023-12-27 10:16:48,324][20347] Updated weights for policy 0, policy_version 672213 (0.0007) [2023-12-27 10:16:48,382][20347] Updated weights for policy 0, policy_version 672223 (0.0007) [2023-12-27 10:16:48,439][20347] Updated weights for policy 0, policy_version 672233 (0.0008) [2023-12-27 10:16:48,704][20381] Updated weights for policy 1, policy_version 671647 (0.0008) [2023-12-27 10:16:48,767][20381] Updated weights for policy 1, policy_version 671657 (0.0008) [2023-12-27 10:16:48,827][20381] Updated weights for policy 1, policy_version 671667 (0.0007) [2023-12-27 10:16:49,130][20347] Updated weights for policy 0, policy_version 672243 (0.0008) [2023-12-27 10:16:49,182][20347] Updated weights for policy 0, policy_version 672253 (0.0008) [2023-12-27 10:16:49,232][20347] Updated weights for policy 0, policy_version 672263 (0.0008) [2023-12-27 10:16:49,509][20381] Updated weights for policy 1, policy_version 671677 (0.0007) [2023-12-27 10:16:49,566][20381] Updated weights for policy 1, policy_version 671687 (0.0008) [2023-12-27 10:16:49,616][20381] Updated weights for policy 1, policy_version 671697 (0.0006) [2023-12-27 10:16:49,984][20347] Updated weights for policy 0, policy_version 672273 (0.0008) [2023-12-27 10:16:50,038][20347] Updated weights for policy 0, policy_version 672283 (0.0008) [2023-12-27 10:16:50,096][20347] Updated weights for policy 0, policy_version 672293 (0.0008) [2023-12-27 10:16:50,155][20347] Updated weights for policy 0, policy_version 672303 (0.0008) [2023-12-27 10:16:50,266][20381] Updated weights for policy 1, policy_version 671707 (0.0006) [2023-12-27 10:16:50,328][20381] Updated weights for policy 1, policy_version 671717 (0.0008) [2023-12-27 10:16:50,397][20381] Updated weights for policy 1, policy_version 671727 (0.0008) [2023-12-27 10:16:50,834][20347] Updated weights for policy 0, policy_version 672313 (0.0007) [2023-12-27 10:16:50,882][20347] Updated weights for policy 0, policy_version 672323 (0.0007) [2023-12-27 10:16:50,933][20347] Updated weights for policy 0, policy_version 672333 (0.0008) [2023-12-27 10:16:51,032][20381] Updated weights for policy 1, policy_version 671737 (0.0008) [2023-12-27 10:16:51,096][20381] Updated weights for policy 1, policy_version 671747 (0.0008) [2023-12-27 10:16:51,157][20381] Updated weights for policy 1, policy_version 671757 (0.0009) [2023-12-27 10:16:51,210][20381] Updated weights for policy 1, policy_version 671767 (0.0008) [2023-12-27 10:16:51,707][20347] Updated weights for policy 0, policy_version 672343 (0.0007) [2023-12-27 10:16:51,776][20347] Updated weights for policy 0, policy_version 672353 (0.0008) [2023-12-27 10:16:51,833][20347] Updated weights for policy 0, policy_version 672363 (0.0008) [2023-12-27 10:16:51,895][20381] Updated weights for policy 1, policy_version 671777 (0.0007) [2023-12-27 10:16:51,952][20381] Updated weights for policy 1, policy_version 671787 (0.0008) [2023-12-27 10:16:52,002][20381] Updated weights for policy 1, policy_version 671797 (0.0008) [2023-12-27 10:16:52,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20744.1). Total num frames: 344154112. Throughput: 0: 10406.9, 1: 10305.3. Samples: 344124928. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:16:52,363][19170] Avg episode reward: [(0, '385.840'), (1, '1000.000')] [2023-12-27 10:16:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000672368_172146688.pth... [2023-12-27 10:16:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000671800_172007424.pth... [2023-12-27 10:16:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000670584_171696128.pth [2023-12-27 10:16:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000671184_171843584.pth [2023-12-27 10:16:52,495][20347] Updated weights for policy 0, policy_version 672373 (0.0007) [2023-12-27 10:16:52,553][20347] Updated weights for policy 0, policy_version 672383 (0.0007) [2023-12-27 10:16:52,612][20347] Updated weights for policy 0, policy_version 672393 (0.0008) [2023-12-27 10:16:52,695][20381] Updated weights for policy 1, policy_version 671807 (0.0008) [2023-12-27 10:16:52,745][20381] Updated weights for policy 1, policy_version 671817 (0.0008) [2023-12-27 10:16:52,798][20381] Updated weights for policy 1, policy_version 671827 (0.0008) [2023-12-27 10:16:53,326][20347] Updated weights for policy 0, policy_version 672403 (0.0008) [2023-12-27 10:16:53,378][20347] Updated weights for policy 0, policy_version 672413 (0.0007) [2023-12-27 10:16:53,429][20347] Updated weights for policy 0, policy_version 672423 (0.0005) [2023-12-27 10:16:53,536][20381] Updated weights for policy 1, policy_version 671837 (0.0008) [2023-12-27 10:16:53,585][20381] Updated weights for policy 1, policy_version 671847 (0.0008) [2023-12-27 10:16:53,637][20381] Updated weights for policy 1, policy_version 671857 (0.0007) [2023-12-27 10:16:54,094][20347] Updated weights for policy 0, policy_version 672433 (0.0006) [2023-12-27 10:16:54,154][20347] Updated weights for policy 0, policy_version 672443 (0.0007) [2023-12-27 10:16:54,206][20347] Updated weights for policy 0, policy_version 672453 (0.0007) [2023-12-27 10:16:54,265][20347] Updated weights for policy 0, policy_version 672463 (0.0005) [2023-12-27 10:16:54,293][20381] Updated weights for policy 1, policy_version 671867 (0.0007) [2023-12-27 10:16:54,350][20381] Updated weights for policy 1, policy_version 671877 (0.0007) [2023-12-27 10:16:54,402][20381] Updated weights for policy 1, policy_version 671887 (0.0008) [2023-12-27 10:16:54,883][20347] Updated weights for policy 0, policy_version 672473 (0.0005) [2023-12-27 10:16:54,935][20347] Updated weights for policy 0, policy_version 672483 (0.0005) [2023-12-27 10:16:54,993][20347] Updated weights for policy 0, policy_version 672493 (0.0006) [2023-12-27 10:16:55,050][20381] Updated weights for policy 1, policy_version 671897 (0.0008) [2023-12-27 10:16:55,104][20381] Updated weights for policy 1, policy_version 671907 (0.0008) [2023-12-27 10:16:55,165][20381] Updated weights for policy 1, policy_version 671917 (0.0007) [2023-12-27 10:16:55,218][20381] Updated weights for policy 1, policy_version 671927 (0.0005) [2023-12-27 10:16:55,670][20347] Updated weights for policy 0, policy_version 672503 (0.0007) [2023-12-27 10:16:55,726][20347] Updated weights for policy 0, policy_version 672513 (0.0008) [2023-12-27 10:16:55,780][20347] Updated weights for policy 0, policy_version 672523 (0.0007) [2023-12-27 10:16:55,816][20381] Updated weights for policy 1, policy_version 671937 (0.0006) [2023-12-27 10:16:55,870][20381] Updated weights for policy 1, policy_version 671947 (0.0007) [2023-12-27 10:16:55,918][20381] Updated weights for policy 1, policy_version 671957 (0.0007) [2023-12-27 10:16:56,444][20347] Updated weights for policy 0, policy_version 672533 (0.0006) [2023-12-27 10:16:56,498][20347] Updated weights for policy 0, policy_version 672543 (0.0008) [2023-12-27 10:16:56,545][20347] Updated weights for policy 0, policy_version 672553 (0.0007) [2023-12-27 10:16:56,612][20381] Updated weights for policy 1, policy_version 671967 (0.0007) [2023-12-27 10:16:56,666][20381] Updated weights for policy 1, policy_version 671977 (0.0008) [2023-12-27 10:16:56,711][20381] Updated weights for policy 1, policy_version 671987 (0.0008) [2023-12-27 10:16:57,288][20347] Updated weights for policy 0, policy_version 672563 (0.0007) [2023-12-27 10:16:57,301][20381] Updated weights for policy 1, policy_version 671997 (0.0007) [2023-12-27 10:16:57,341][20347] Updated weights for policy 0, policy_version 672573 (0.0006) [2023-12-27 10:16:57,357][20381] Updated weights for policy 1, policy_version 672007 (0.0006) [2023-12-27 10:16:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 344252416. Throughput: 0: 10349.7, 1: 10314.9. Samples: 344247592. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:16:57,363][19170] Avg episode reward: [(0, '300.880'), (1, '1000.000')] [2023-12-27 10:16:57,400][20347] Updated weights for policy 0, policy_version 672583 (0.0007) [2023-12-27 10:16:57,403][20381] Updated weights for policy 1, policy_version 672017 (0.0006) [2023-12-27 10:16:57,999][20381] Updated weights for policy 1, policy_version 672027 (0.0006) [2023-12-27 10:16:58,049][20381] Updated weights for policy 1, policy_version 672037 (0.0005) [2023-12-27 10:16:58,100][20381] Updated weights for policy 1, policy_version 672047 (0.0005) [2023-12-27 10:16:58,107][20347] Updated weights for policy 0, policy_version 672593 (0.0006) [2023-12-27 10:16:58,161][20347] Updated weights for policy 0, policy_version 672603 (0.0010) [2023-12-27 10:16:58,224][20347] Updated weights for policy 0, policy_version 672613 (0.0010) [2023-12-27 10:16:58,290][20347] Updated weights for policy 0, policy_version 672623 (0.0010) [2023-12-27 10:16:58,760][20381] Updated weights for policy 1, policy_version 672057 (0.0006) [2023-12-27 10:16:58,819][20381] Updated weights for policy 1, policy_version 672067 (0.0009) [2023-12-27 10:16:58,884][20381] Updated weights for policy 1, policy_version 672077 (0.0008) [2023-12-27 10:16:58,945][20381] Updated weights for policy 1, policy_version 672087 (0.0007) [2023-12-27 10:16:59,040][20347] Updated weights for policy 0, policy_version 672633 (0.0007) [2023-12-27 10:16:59,099][20347] Updated weights for policy 0, policy_version 672643 (0.0006) [2023-12-27 10:16:59,161][20347] Updated weights for policy 0, policy_version 672653 (0.0008) [2023-12-27 10:16:59,661][20381] Updated weights for policy 1, policy_version 672097 (0.0009) [2023-12-27 10:16:59,721][20381] Updated weights for policy 1, policy_version 672107 (0.0008) [2023-12-27 10:16:59,779][20381] Updated weights for policy 1, policy_version 672117 (0.0007) [2023-12-27 10:16:59,798][20347] Updated weights for policy 0, policy_version 672663 (0.0006) [2023-12-27 10:16:59,862][20347] Updated weights for policy 0, policy_version 672673 (0.0008) [2023-12-27 10:16:59,922][20347] Updated weights for policy 0, policy_version 672683 (0.0007) [2023-12-27 10:17:00,543][20347] Updated weights for policy 0, policy_version 672693 (0.0006) [2023-12-27 10:17:00,547][20381] Updated weights for policy 1, policy_version 672127 (0.0008) [2023-12-27 10:17:00,595][20347] Updated weights for policy 0, policy_version 672703 (0.0006) [2023-12-27 10:17:00,601][20381] Updated weights for policy 1, policy_version 672137 (0.0006) [2023-12-27 10:17:00,653][20347] Updated weights for policy 0, policy_version 672713 (0.0006) [2023-12-27 10:17:00,664][20381] Updated weights for policy 1, policy_version 672147 (0.0007) [2023-12-27 10:17:01,341][20347] Updated weights for policy 0, policy_version 672723 (0.0007) [2023-12-27 10:17:01,382][20381] Updated weights for policy 1, policy_version 672157 (0.0006) [2023-12-27 10:17:01,401][20347] Updated weights for policy 0, policy_version 672733 (0.0006) [2023-12-27 10:17:01,433][20381] Updated weights for policy 1, policy_version 672167 (0.0008) [2023-12-27 10:17:01,452][20347] Updated weights for policy 0, policy_version 672743 (0.0005) [2023-12-27 10:17:01,482][20381] Updated weights for policy 1, policy_version 672177 (0.0008) [2023-12-27 10:17:02,118][20347] Updated weights for policy 0, policy_version 672753 (0.0006) [2023-12-27 10:17:02,178][20347] Updated weights for policy 0, policy_version 672763 (0.0007) [2023-12-27 10:17:02,186][20381] Updated weights for policy 1, policy_version 672187 (0.0008) [2023-12-27 10:17:02,230][20347] Updated weights for policy 0, policy_version 672773 (0.0007) [2023-12-27 10:17:02,241][20381] Updated weights for policy 1, policy_version 672197 (0.0007) [2023-12-27 10:17:02,286][20347] Updated weights for policy 0, policy_version 672783 (0.0008) [2023-12-27 10:17:02,295][20381] Updated weights for policy 1, policy_version 672207 (0.0006) [2023-12-27 10:17:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 344367104. Throughput: 0: 10290.0, 1: 10337.8. Samples: 344370524. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:02,363][19170] Avg episode reward: [(0, '353.630'), (1, '1000.000')] [2023-12-27 10:17:02,945][20381] Updated weights for policy 1, policy_version 672217 (0.0007) [2023-12-27 10:17:02,999][20381] Updated weights for policy 1, policy_version 672227 (0.0005) [2023-12-27 10:17:03,008][20347] Updated weights for policy 0, policy_version 672793 (0.0008) [2023-12-27 10:17:03,047][20381] Updated weights for policy 1, policy_version 672237 (0.0005) [2023-12-27 10:17:03,063][20347] Updated weights for policy 0, policy_version 672803 (0.0007) [2023-12-27 10:17:03,101][20381] Updated weights for policy 1, policy_version 672247 (0.0005) [2023-12-27 10:17:03,123][20347] Updated weights for policy 0, policy_version 672813 (0.0007) [2023-12-27 10:17:03,650][20381] Updated weights for policy 1, policy_version 672257 (0.0007) [2023-12-27 10:17:03,699][20381] Updated weights for policy 1, policy_version 672267 (0.0007) [2023-12-27 10:17:03,755][20381] Updated weights for policy 1, policy_version 672277 (0.0008) [2023-12-27 10:17:03,816][20347] Updated weights for policy 0, policy_version 672823 (0.0007) [2023-12-27 10:17:03,881][20347] Updated weights for policy 0, policy_version 672833 (0.0006) [2023-12-27 10:17:03,945][20347] Updated weights for policy 0, policy_version 672843 (0.0005) [2023-12-27 10:17:04,506][20381] Updated weights for policy 1, policy_version 672287 (0.0007) [2023-12-27 10:17:04,551][20347] Updated weights for policy 0, policy_version 672853 (0.0006) [2023-12-27 10:17:04,568][20381] Updated weights for policy 1, policy_version 672297 (0.0007) [2023-12-27 10:17:04,599][20347] Updated weights for policy 0, policy_version 672863 (0.0005) [2023-12-27 10:17:04,626][20381] Updated weights for policy 1, policy_version 672307 (0.0006) [2023-12-27 10:17:04,650][20347] Updated weights for policy 0, policy_version 672873 (0.0007) [2023-12-27 10:17:05,202][20381] Updated weights for policy 1, policy_version 672317 (0.0006) [2023-12-27 10:17:05,252][20381] Updated weights for policy 1, policy_version 672327 (0.0005) [2023-12-27 10:17:05,301][20381] Updated weights for policy 1, policy_version 672337 (0.0005) [2023-12-27 10:17:05,349][20347] Updated weights for policy 0, policy_version 672883 (0.0008) [2023-12-27 10:17:05,409][20347] Updated weights for policy 0, policy_version 672893 (0.0006) [2023-12-27 10:17:05,462][20347] Updated weights for policy 0, policy_version 672903 (0.0008) [2023-12-27 10:17:05,839][20381] Updated weights for policy 1, policy_version 672347 (0.0005) [2023-12-27 10:17:05,901][20381] Updated weights for policy 1, policy_version 672357 (0.0005) [2023-12-27 10:17:05,959][20381] Updated weights for policy 1, policy_version 672367 (0.0005) [2023-12-27 10:17:06,010][20347] Updated weights for policy 0, policy_version 672913 (0.0006) [2023-12-27 10:17:06,070][20347] Updated weights for policy 0, policy_version 672923 (0.0010) [2023-12-27 10:17:06,134][20347] Updated weights for policy 0, policy_version 672933 (0.0010) [2023-12-27 10:17:06,195][20347] Updated weights for policy 0, policy_version 672943 (0.0010) [2023-12-27 10:17:06,667][20381] Updated weights for policy 1, policy_version 672377 (0.0008) [2023-12-27 10:17:06,729][20381] Updated weights for policy 1, policy_version 672387 (0.0008) [2023-12-27 10:17:06,790][20381] Updated weights for policy 1, policy_version 672397 (0.0007) [2023-12-27 10:17:06,836][20347] Updated weights for policy 0, policy_version 672953 (0.0010) [2023-12-27 10:17:06,849][20381] Updated weights for policy 1, policy_version 672407 (0.0006) [2023-12-27 10:17:06,887][20347] Updated weights for policy 0, policy_version 672963 (0.0011) [2023-12-27 10:17:06,933][20347] Updated weights for policy 0, policy_version 672973 (0.0010) [2023-12-27 10:17:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20743.8). Total num frames: 344465408. Throughput: 0: 10253.9, 1: 10345.2. Samples: 344433264. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:07,363][19170] Avg episode reward: [(0, '446.220'), (1, '1000.000')] [2023-12-27 10:17:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000672976_172302336.pth... [2023-12-27 10:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000672408_172163072.pth... [2023-12-27 10:17:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000671192_171851776.pth [2023-12-27 10:17:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000671792_171999232.pth [2023-12-27 10:17:07,540][20381] Updated weights for policy 1, policy_version 672417 (0.0008) [2023-12-27 10:17:07,598][20381] Updated weights for policy 1, policy_version 672427 (0.0008) [2023-12-27 10:17:07,631][20347] Updated weights for policy 0, policy_version 672983 (0.0010) [2023-12-27 10:17:07,648][20381] Updated weights for policy 1, policy_version 672437 (0.0006) [2023-12-27 10:17:07,683][20347] Updated weights for policy 0, policy_version 672993 (0.0010) [2023-12-27 10:17:07,745][20347] Updated weights for policy 0, policy_version 673003 (0.0009) [2023-12-27 10:17:08,383][20381] Updated weights for policy 1, policy_version 672447 (0.0007) [2023-12-27 10:17:08,392][20347] Updated weights for policy 0, policy_version 673013 (0.0006) [2023-12-27 10:17:08,436][20381] Updated weights for policy 1, policy_version 672457 (0.0008) [2023-12-27 10:17:08,445][20347] Updated weights for policy 0, policy_version 673023 (0.0006) [2023-12-27 10:17:08,489][20381] Updated weights for policy 1, policy_version 672467 (0.0008) [2023-12-27 10:17:08,497][20347] Updated weights for policy 0, policy_version 673033 (0.0005) [2023-12-27 10:17:09,114][20347] Updated weights for policy 0, policy_version 673043 (0.0010) [2023-12-27 10:17:09,172][20347] Updated weights for policy 0, policy_version 673053 (0.0010) [2023-12-27 10:17:09,190][20381] Updated weights for policy 1, policy_version 672477 (0.0007) [2023-12-27 10:17:09,235][20347] Updated weights for policy 0, policy_version 673063 (0.0009) [2023-12-27 10:17:09,253][20381] Updated weights for policy 1, policy_version 672487 (0.0008) [2023-12-27 10:17:09,304][20381] Updated weights for policy 1, policy_version 672497 (0.0006) [2023-12-27 10:17:09,968][20347] Updated weights for policy 0, policy_version 673073 (0.0008) [2023-12-27 10:17:09,990][20381] Updated weights for policy 1, policy_version 672507 (0.0007) [2023-12-27 10:17:10,019][20347] Updated weights for policy 0, policy_version 673083 (0.0006) [2023-12-27 10:17:10,047][20381] Updated weights for policy 1, policy_version 672517 (0.0011) [2023-12-27 10:17:10,070][20347] Updated weights for policy 0, policy_version 673093 (0.0005) [2023-12-27 10:17:10,106][20381] Updated weights for policy 1, policy_version 672527 (0.0011) [2023-12-27 10:17:10,126][20347] Updated weights for policy 0, policy_version 673103 (0.0007) [2023-12-27 10:17:10,726][20347] Updated weights for policy 0, policy_version 673113 (0.0008) [2023-12-27 10:17:10,768][20381] Updated weights for policy 1, policy_version 672537 (0.0010) [2023-12-27 10:17:10,784][20347] Updated weights for policy 0, policy_version 673123 (0.0010) [2023-12-27 10:17:10,827][20381] Updated weights for policy 1, policy_version 672547 (0.0008) [2023-12-27 10:17:10,841][20347] Updated weights for policy 0, policy_version 673133 (0.0007) [2023-12-27 10:17:10,879][20381] Updated weights for policy 1, policy_version 672557 (0.0010) [2023-12-27 10:17:10,925][20381] Updated weights for policy 1, policy_version 672567 (0.0010) [2023-12-27 10:17:11,481][20347] Updated weights for policy 0, policy_version 673143 (0.0009) [2023-12-27 10:17:11,544][20347] Updated weights for policy 0, policy_version 673153 (0.0010) [2023-12-27 10:17:11,604][20347] Updated weights for policy 0, policy_version 673163 (0.0010) [2023-12-27 10:17:11,669][20381] Updated weights for policy 1, policy_version 672577 (0.0007) [2023-12-27 10:17:11,725][20381] Updated weights for policy 1, policy_version 672587 (0.0008) [2023-12-27 10:17:11,779][20381] Updated weights for policy 1, policy_version 672597 (0.0007) [2023-12-27 10:17:12,275][20347] Updated weights for policy 0, policy_version 673173 (0.0009) [2023-12-27 10:17:12,340][20347] Updated weights for policy 0, policy_version 673183 (0.0008) [2023-12-27 10:17:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 344563712. Throughput: 0: 10321.0, 1: 10328.1. Samples: 344559716. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:12,363][19170] Avg episode reward: [(0, '408.210'), (1, '1000.000')] [2023-12-27 10:17:12,409][20347] Updated weights for policy 0, policy_version 673193 (0.0010) [2023-12-27 10:17:12,447][20381] Updated weights for policy 1, policy_version 672607 (0.0007) [2023-12-27 10:17:12,512][20381] Updated weights for policy 1, policy_version 672617 (0.0008) [2023-12-27 10:17:12,571][20381] Updated weights for policy 1, policy_version 672627 (0.0007) [2023-12-27 10:17:13,076][20347] Updated weights for policy 0, policy_version 673203 (0.0006) [2023-12-27 10:17:13,131][20347] Updated weights for policy 0, policy_version 673213 (0.0008) [2023-12-27 10:17:13,188][20347] Updated weights for policy 0, policy_version 673223 (0.0006) [2023-12-27 10:17:13,274][20381] Updated weights for policy 1, policy_version 672637 (0.0007) [2023-12-27 10:17:13,331][20381] Updated weights for policy 1, policy_version 672647 (0.0007) [2023-12-27 10:17:13,385][20381] Updated weights for policy 1, policy_version 672657 (0.0005) [2023-12-27 10:17:13,858][20347] Updated weights for policy 0, policy_version 673233 (0.0008) [2023-12-27 10:17:13,908][20347] Updated weights for policy 0, policy_version 673243 (0.0006) [2023-12-27 10:17:13,970][20347] Updated weights for policy 0, policy_version 673253 (0.0009) [2023-12-27 10:17:13,993][20381] Updated weights for policy 1, policy_version 672667 (0.0005) [2023-12-27 10:17:14,019][20347] Updated weights for policy 0, policy_version 673263 (0.0010) [2023-12-27 10:17:14,047][20381] Updated weights for policy 1, policy_version 672677 (0.0006) [2023-12-27 10:17:14,106][20381] Updated weights for policy 1, policy_version 672687 (0.0007) [2023-12-27 10:17:14,652][20347] Updated weights for policy 0, policy_version 673273 (0.0008) [2023-12-27 10:17:14,697][20347] Updated weights for policy 0, policy_version 673283 (0.0005) [2023-12-27 10:17:14,732][20381] Updated weights for policy 1, policy_version 672697 (0.0006) [2023-12-27 10:17:14,748][20347] Updated weights for policy 0, policy_version 673293 (0.0005) [2023-12-27 10:17:14,790][20381] Updated weights for policy 1, policy_version 672707 (0.0011) [2023-12-27 10:17:14,837][20381] Updated weights for policy 1, policy_version 672717 (0.0009) [2023-12-27 10:17:14,895][20381] Updated weights for policy 1, policy_version 672727 (0.0005) [2023-12-27 10:17:15,416][20347] Updated weights for policy 0, policy_version 673303 (0.0010) [2023-12-27 10:17:15,463][20347] Updated weights for policy 0, policy_version 673313 (0.0010) [2023-12-27 10:17:15,504][20381] Updated weights for policy 1, policy_version 672737 (0.0010) [2023-12-27 10:17:15,521][20347] Updated weights for policy 0, policy_version 673323 (0.0010) [2023-12-27 10:17:15,559][20381] Updated weights for policy 1, policy_version 672747 (0.0011) [2023-12-27 10:17:15,613][20381] Updated weights for policy 1, policy_version 672757 (0.0010) [2023-12-27 10:17:16,215][20347] Updated weights for policy 0, policy_version 673333 (0.0010) [2023-12-27 10:17:16,269][20347] Updated weights for policy 0, policy_version 673343 (0.0010) [2023-12-27 10:17:16,303][20381] Updated weights for policy 1, policy_version 672767 (0.0011) [2023-12-27 10:17:16,324][20347] Updated weights for policy 0, policy_version 673353 (0.0010) [2023-12-27 10:17:16,359][20381] Updated weights for policy 1, policy_version 672777 (0.0010) [2023-12-27 10:17:16,413][20381] Updated weights for policy 1, policy_version 672787 (0.0010) [2023-12-27 10:17:16,990][20347] Updated weights for policy 0, policy_version 673363 (0.0007) [2023-12-27 10:17:17,052][20347] Updated weights for policy 0, policy_version 673373 (0.0010) [2023-12-27 10:17:17,088][20381] Updated weights for policy 1, policy_version 672797 (0.0010) [2023-12-27 10:17:17,106][20347] Updated weights for policy 0, policy_version 673383 (0.0010) [2023-12-27 10:17:17,137][20381] Updated weights for policy 1, policy_version 672807 (0.0010) [2023-12-27 10:17:17,184][20381] Updated weights for policy 1, policy_version 672817 (0.0008) [2023-12-27 10:17:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 344678400. Throughput: 0: 10302.8, 1: 10350.4. Samples: 344684276. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:17,363][19170] Avg episode reward: [(0, '447.180'), (1, '980.840')] [2023-12-27 10:17:17,749][20347] Updated weights for policy 0, policy_version 673393 (0.0010) [2023-12-27 10:17:17,761][20381] Updated weights for policy 1, policy_version 672827 (0.0007) [2023-12-27 10:17:17,798][20347] Updated weights for policy 0, policy_version 673403 (0.0005) [2023-12-27 10:17:17,816][20381] Updated weights for policy 1, policy_version 672837 (0.0011) [2023-12-27 10:17:17,856][20347] Updated weights for policy 0, policy_version 673413 (0.0006) [2023-12-27 10:17:17,872][20381] Updated weights for policy 1, policy_version 672847 (0.0011) [2023-12-27 10:17:17,914][20347] Updated weights for policy 0, policy_version 673423 (0.0005) [2023-12-27 10:17:18,508][20347] Updated weights for policy 0, policy_version 673433 (0.0010) [2023-12-27 10:17:18,537][20381] Updated weights for policy 1, policy_version 672857 (0.0010) [2023-12-27 10:17:18,567][20347] Updated weights for policy 0, policy_version 673443 (0.0010) [2023-12-27 10:17:18,593][20381] Updated weights for policy 1, policy_version 672867 (0.0011) [2023-12-27 10:17:18,622][20347] Updated weights for policy 0, policy_version 673453 (0.0010) [2023-12-27 10:17:18,651][20381] Updated weights for policy 1, policy_version 672877 (0.0011) [2023-12-27 10:17:18,713][20381] Updated weights for policy 1, policy_version 672887 (0.0011) [2023-12-27 10:17:19,262][20347] Updated weights for policy 0, policy_version 673463 (0.0013) [2023-12-27 10:17:19,315][20347] Updated weights for policy 0, policy_version 673473 (0.0009) [2023-12-27 10:17:19,376][20347] Updated weights for policy 0, policy_version 673483 (0.0009) [2023-12-27 10:17:19,436][20381] Updated weights for policy 1, policy_version 672897 (0.0009) [2023-12-27 10:17:19,490][20381] Updated weights for policy 1, policy_version 672907 (0.0009) [2023-12-27 10:17:19,543][20381] Updated weights for policy 1, policy_version 672917 (0.0008) [2023-12-27 10:17:20,133][20347] Updated weights for policy 0, policy_version 673493 (0.0007) [2023-12-27 10:17:20,188][20347] Updated weights for policy 0, policy_version 673503 (0.0006) [2023-12-27 10:17:20,236][20381] Updated weights for policy 1, policy_version 672927 (0.0006) [2023-12-27 10:17:20,242][20347] Updated weights for policy 0, policy_version 673513 (0.0006) [2023-12-27 10:17:20,302][20381] Updated weights for policy 1, policy_version 672937 (0.0006) [2023-12-27 10:17:20,362][20381] Updated weights for policy 1, policy_version 672947 (0.0008) [2023-12-27 10:17:20,854][20347] Updated weights for policy 0, policy_version 673523 (0.0006) [2023-12-27 10:17:20,908][20347] Updated weights for policy 0, policy_version 673533 (0.0009) [2023-12-27 10:17:20,969][20381] Updated weights for policy 1, policy_version 672957 (0.0007) [2023-12-27 10:17:20,971][20347] Updated weights for policy 0, policy_version 673543 (0.0011) [2023-12-27 10:17:21,030][20381] Updated weights for policy 1, policy_version 672967 (0.0007) [2023-12-27 10:17:21,096][20381] Updated weights for policy 1, policy_version 672977 (0.0008) [2023-12-27 10:17:21,705][20347] Updated weights for policy 0, policy_version 673553 (0.0010) [2023-12-27 10:17:21,770][20347] Updated weights for policy 0, policy_version 673563 (0.0007) [2023-12-27 10:17:21,810][20381] Updated weights for policy 1, policy_version 672987 (0.0008) [2023-12-27 10:17:21,833][20347] Updated weights for policy 0, policy_version 673573 (0.0007) [2023-12-27 10:17:21,879][20381] Updated weights for policy 1, policy_version 672997 (0.0007) [2023-12-27 10:17:21,898][20347] Updated weights for policy 0, policy_version 673583 (0.0007) [2023-12-27 10:17:21,952][20381] Updated weights for policy 1, policy_version 673007 (0.0009) [2023-12-27 10:17:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 344776704. Throughput: 0: 10290.2, 1: 10318.7. Samples: 344746960. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:22,363][19170] Avg episode reward: [(0, '505.120'), (1, '990.480')] [2023-12-27 10:17:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000673016_172318720.pth... [2023-12-27 10:17:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000673584_172457984.pth... [2023-12-27 10:17:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000672368_172146688.pth [2023-12-27 10:17:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000671800_172007424.pth [2023-12-27 10:17:22,512][20347] Updated weights for policy 0, policy_version 673593 (0.0008) [2023-12-27 10:17:22,561][20347] Updated weights for policy 0, policy_version 673603 (0.0010) [2023-12-27 10:17:22,612][20347] Updated weights for policy 0, policy_version 673613 (0.0010) [2023-12-27 10:17:22,643][20381] Updated weights for policy 1, policy_version 673017 (0.0008) [2023-12-27 10:17:22,708][20381] Updated weights for policy 1, policy_version 673027 (0.0007) [2023-12-27 10:17:22,772][20381] Updated weights for policy 1, policy_version 673037 (0.0008) [2023-12-27 10:17:22,835][20381] Updated weights for policy 1, policy_version 673047 (0.0007) [2023-12-27 10:17:23,300][20347] Updated weights for policy 0, policy_version 673623 (0.0009) [2023-12-27 10:17:23,352][20347] Updated weights for policy 0, policy_version 673633 (0.0008) [2023-12-27 10:17:23,408][20347] Updated weights for policy 0, policy_version 673643 (0.0008) [2023-12-27 10:17:23,530][20381] Updated weights for policy 1, policy_version 673057 (0.0008) [2023-12-27 10:17:23,592][20381] Updated weights for policy 1, policy_version 673067 (0.0008) [2023-12-27 10:17:23,649][20381] Updated weights for policy 1, policy_version 673077 (0.0008) [2023-12-27 10:17:24,069][20347] Updated weights for policy 0, policy_version 673653 (0.0006) [2023-12-27 10:17:24,123][20347] Updated weights for policy 0, policy_version 673663 (0.0005) [2023-12-27 10:17:24,183][20347] Updated weights for policy 0, policy_version 673673 (0.0005) [2023-12-27 10:17:24,360][20381] Updated weights for policy 1, policy_version 673087 (0.0009) [2023-12-27 10:17:24,410][20381] Updated weights for policy 1, policy_version 673097 (0.0011) [2023-12-27 10:17:24,463][20381] Updated weights for policy 1, policy_version 673107 (0.0011) [2023-12-27 10:17:24,800][20347] Updated weights for policy 0, policy_version 673683 (0.0006) [2023-12-27 10:17:24,855][20347] Updated weights for policy 0, policy_version 673693 (0.0007) [2023-12-27 10:17:24,902][20347] Updated weights for policy 0, policy_version 673703 (0.0007) [2023-12-27 10:17:25,152][20381] Updated weights for policy 1, policy_version 673117 (0.0011) [2023-12-27 10:17:25,204][20381] Updated weights for policy 1, policy_version 673127 (0.0010) [2023-12-27 10:17:25,262][20381] Updated weights for policy 1, policy_version 673137 (0.0011) [2023-12-27 10:17:25,522][20347] Updated weights for policy 0, policy_version 673713 (0.0008) [2023-12-27 10:17:25,570][20347] Updated weights for policy 0, policy_version 673723 (0.0010) [2023-12-27 10:17:25,619][20347] Updated weights for policy 0, policy_version 673733 (0.0009) [2023-12-27 10:17:25,670][20347] Updated weights for policy 0, policy_version 673743 (0.0009) [2023-12-27 10:17:25,969][20381] Updated weights for policy 1, policy_version 673147 (0.0011) [2023-12-27 10:17:26,038][20381] Updated weights for policy 1, policy_version 673157 (0.0011) [2023-12-27 10:17:26,100][20381] Updated weights for policy 1, policy_version 673167 (0.0011) [2023-12-27 10:17:26,321][20347] Updated weights for policy 0, policy_version 673753 (0.0010) [2023-12-27 10:17:26,382][20347] Updated weights for policy 0, policy_version 673763 (0.0010) [2023-12-27 10:17:26,430][20347] Updated weights for policy 0, policy_version 673773 (0.0010) [2023-12-27 10:17:26,780][20381] Updated weights for policy 1, policy_version 673177 (0.0011) [2023-12-27 10:17:26,831][20381] Updated weights for policy 1, policy_version 673187 (0.0011) [2023-12-27 10:17:26,888][20381] Updated weights for policy 1, policy_version 673197 (0.0010) [2023-12-27 10:17:26,943][20381] Updated weights for policy 1, policy_version 673207 (0.0010) [2023-12-27 10:17:27,057][20347] Updated weights for policy 0, policy_version 673783 (0.0010) [2023-12-27 10:17:27,116][20347] Updated weights for policy 0, policy_version 673793 (0.0010) [2023-12-27 10:17:27,173][20347] Updated weights for policy 0, policy_version 673803 (0.0010) [2023-12-27 10:17:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 344883200. Throughput: 0: 10346.2, 1: 10291.2. Samples: 344870820. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:27,363][19170] Avg episode reward: [(0, '523.040'), (1, '1000.000')] [2023-12-27 10:17:27,628][20381] Updated weights for policy 1, policy_version 673217 (0.0010) [2023-12-27 10:17:27,672][20381] Updated weights for policy 1, policy_version 673227 (0.0010) [2023-12-27 10:17:27,724][20381] Updated weights for policy 1, policy_version 673237 (0.0011) [2023-12-27 10:17:27,746][20347] Updated weights for policy 0, policy_version 673813 (0.0008) [2023-12-27 10:17:27,802][20347] Updated weights for policy 0, policy_version 673823 (0.0005) [2023-12-27 10:17:27,862][20347] Updated weights for policy 0, policy_version 673833 (0.0005) [2023-12-27 10:17:28,417][20347] Updated weights for policy 0, policy_version 673843 (0.0007) [2023-12-27 10:17:28,443][20381] Updated weights for policy 1, policy_version 673247 (0.0010) [2023-12-27 10:17:28,475][20347] Updated weights for policy 0, policy_version 673853 (0.0010) [2023-12-27 10:17:28,496][20381] Updated weights for policy 1, policy_version 673257 (0.0010) [2023-12-27 10:17:28,529][20347] Updated weights for policy 0, policy_version 673863 (0.0010) [2023-12-27 10:17:28,548][20381] Updated weights for policy 1, policy_version 673267 (0.0010) [2023-12-27 10:17:29,158][20347] Updated weights for policy 0, policy_version 673873 (0.0010) [2023-12-27 10:17:29,215][20347] Updated weights for policy 0, policy_version 673883 (0.0010) [2023-12-27 10:17:29,241][20381] Updated weights for policy 1, policy_version 673277 (0.0009) [2023-12-27 10:17:29,273][20347] Updated weights for policy 0, policy_version 673893 (0.0010) [2023-12-27 10:17:29,297][20381] Updated weights for policy 1, policy_version 673287 (0.0007) [2023-12-27 10:17:29,330][20347] Updated weights for policy 0, policy_version 673903 (0.0007) [2023-12-27 10:17:29,358][20381] Updated weights for policy 1, policy_version 673297 (0.0010) [2023-12-27 10:17:29,953][20347] Updated weights for policy 0, policy_version 673913 (0.0010) [2023-12-27 10:17:30,011][20347] Updated weights for policy 0, policy_version 673923 (0.0009) [2023-12-27 10:17:30,043][20381] Updated weights for policy 1, policy_version 673307 (0.0010) [2023-12-27 10:17:30,066][20347] Updated weights for policy 0, policy_version 673933 (0.0008) [2023-12-27 10:17:30,104][20381] Updated weights for policy 1, policy_version 673317 (0.0009) [2023-12-27 10:17:30,162][20381] Updated weights for policy 1, policy_version 673327 (0.0008) [2023-12-27 10:17:30,684][20347] Updated weights for policy 0, policy_version 673943 (0.0007) [2023-12-27 10:17:30,736][20347] Updated weights for policy 0, policy_version 673953 (0.0005) [2023-12-27 10:17:30,782][20347] Updated weights for policy 0, policy_version 673963 (0.0005) [2023-12-27 10:17:30,904][20381] Updated weights for policy 1, policy_version 673337 (0.0008) [2023-12-27 10:17:30,966][20381] Updated weights for policy 1, policy_version 673347 (0.0007) [2023-12-27 10:17:31,013][20381] Updated weights for policy 1, policy_version 673357 (0.0008) [2023-12-27 10:17:31,076][20381] Updated weights for policy 1, policy_version 673367 (0.0009) [2023-12-27 10:17:31,393][20347] Updated weights for policy 0, policy_version 673973 (0.0007) [2023-12-27 10:17:31,453][20347] Updated weights for policy 0, policy_version 673983 (0.0010) [2023-12-27 10:17:31,512][20347] Updated weights for policy 0, policy_version 673993 (0.0008) [2023-12-27 10:17:31,766][20381] Updated weights for policy 1, policy_version 673377 (0.0008) [2023-12-27 10:17:31,827][20381] Updated weights for policy 1, policy_version 673387 (0.0008) [2023-12-27 10:17:31,880][20381] Updated weights for policy 1, policy_version 673397 (0.0008) [2023-12-27 10:17:32,127][20347] Updated weights for policy 0, policy_version 674003 (0.0007) [2023-12-27 10:17:32,175][20347] Updated weights for policy 0, policy_version 674013 (0.0008) [2023-12-27 10:17:32,227][20347] Updated weights for policy 0, policy_version 674023 (0.0007) [2023-12-27 10:17:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 344989696. Throughput: 0: 10489.9, 1: 10258.0. Samples: 344998048. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:32,363][19170] Avg episode reward: [(0, '645.010'), (1, '1000.000')] [2023-12-27 10:17:32,586][20381] Updated weights for policy 1, policy_version 673407 (0.0007) [2023-12-27 10:17:32,639][20381] Updated weights for policy 1, policy_version 673417 (0.0007) [2023-12-27 10:17:32,689][20381] Updated weights for policy 1, policy_version 673427 (0.0007) [2023-12-27 10:17:32,909][20347] Updated weights for policy 0, policy_version 674033 (0.0008) [2023-12-27 10:17:32,959][20347] Updated weights for policy 0, policy_version 674043 (0.0009) [2023-12-27 10:17:33,005][20347] Updated weights for policy 0, policy_version 674053 (0.0005) [2023-12-27 10:17:33,050][20347] Updated weights for policy 0, policy_version 674063 (0.0007) [2023-12-27 10:17:33,387][20381] Updated weights for policy 1, policy_version 673437 (0.0007) [2023-12-27 10:17:33,439][20381] Updated weights for policy 1, policy_version 673447 (0.0007) [2023-12-27 10:17:33,484][20381] Updated weights for policy 1, policy_version 673457 (0.0007) [2023-12-27 10:17:33,732][20347] Updated weights for policy 0, policy_version 674073 (0.0006) [2023-12-27 10:17:33,788][20347] Updated weights for policy 0, policy_version 674083 (0.0005) [2023-12-27 10:17:33,845][20347] Updated weights for policy 0, policy_version 674093 (0.0005) [2023-12-27 10:17:34,218][20381] Updated weights for policy 1, policy_version 673467 (0.0007) [2023-12-27 10:17:34,278][20381] Updated weights for policy 1, policy_version 673477 (0.0008) [2023-12-27 10:17:34,334][20381] Updated weights for policy 1, policy_version 673487 (0.0007) [2023-12-27 10:17:34,436][20347] Updated weights for policy 0, policy_version 674103 (0.0010) [2023-12-27 10:17:34,495][20347] Updated weights for policy 0, policy_version 674113 (0.0010) [2023-12-27 10:17:34,554][20347] Updated weights for policy 0, policy_version 674123 (0.0010) [2023-12-27 10:17:35,014][20381] Updated weights for policy 1, policy_version 673497 (0.0009) [2023-12-27 10:17:35,066][20381] Updated weights for policy 1, policy_version 673507 (0.0008) [2023-12-27 10:17:35,119][20381] Updated weights for policy 1, policy_version 673517 (0.0007) [2023-12-27 10:17:35,169][20381] Updated weights for policy 1, policy_version 673527 (0.0007) [2023-12-27 10:17:35,211][20347] Updated weights for policy 0, policy_version 674133 (0.0009) [2023-12-27 10:17:35,266][20347] Updated weights for policy 0, policy_version 674143 (0.0010) [2023-12-27 10:17:35,324][20347] Updated weights for policy 0, policy_version 674153 (0.0010) [2023-12-27 10:17:35,891][20381] Updated weights for policy 1, policy_version 673537 (0.0005) [2023-12-27 10:17:35,920][20347] Updated weights for policy 0, policy_version 674163 (0.0008) [2023-12-27 10:17:35,942][20381] Updated weights for policy 1, policy_version 673547 (0.0006) [2023-12-27 10:17:35,969][20347] Updated weights for policy 0, policy_version 674173 (0.0005) [2023-12-27 10:17:35,988][20381] Updated weights for policy 1, policy_version 673557 (0.0007) [2023-12-27 10:17:36,023][20347] Updated weights for policy 0, policy_version 674183 (0.0005) [2023-12-27 10:17:36,614][20347] Updated weights for policy 0, policy_version 674193 (0.0006) [2023-12-27 10:17:36,673][20347] Updated weights for policy 0, policy_version 674203 (0.0008) [2023-12-27 10:17:36,691][20381] Updated weights for policy 1, policy_version 673567 (0.0007) [2023-12-27 10:17:36,726][20347] Updated weights for policy 0, policy_version 674213 (0.0007) [2023-12-27 10:17:36,737][20381] Updated weights for policy 1, policy_version 673577 (0.0005) [2023-12-27 10:17:36,781][20347] Updated weights for policy 0, policy_version 674223 (0.0007) [2023-12-27 10:17:36,790][20381] Updated weights for policy 1, policy_version 673587 (0.0007) [2023-12-27 10:17:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20688.2). Total num frames: 345088000. Throughput: 0: 10533.4, 1: 10238.9. Samples: 345059688. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:37,364][19170] Avg episode reward: [(0, '745.690'), (1, '1000.000')] [2023-12-27 10:17:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000673592_172466176.pth... [2023-12-27 10:17:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000674224_172621824.pth... [2023-12-27 10:17:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000672408_172163072.pth [2023-12-27 10:17:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000672976_172302336.pth [2023-12-27 10:17:37,481][20347] Updated weights for policy 0, policy_version 674233 (0.0010) [2023-12-27 10:17:37,492][20381] Updated weights for policy 1, policy_version 673597 (0.0008) [2023-12-27 10:17:37,533][20347] Updated weights for policy 0, policy_version 674243 (0.0010) [2023-12-27 10:17:37,550][20381] Updated weights for policy 1, policy_version 673607 (0.0009) [2023-12-27 10:17:37,579][20347] Updated weights for policy 0, policy_version 674253 (0.0010) [2023-12-27 10:17:37,608][20381] Updated weights for policy 1, policy_version 673617 (0.0006) [2023-12-27 10:17:38,288][20347] Updated weights for policy 0, policy_version 674263 (0.0010) [2023-12-27 10:17:38,290][20381] Updated weights for policy 1, policy_version 673627 (0.0007) [2023-12-27 10:17:38,352][20381] Updated weights for policy 1, policy_version 673637 (0.0007) [2023-12-27 10:17:38,354][20347] Updated weights for policy 0, policy_version 674273 (0.0010) [2023-12-27 10:17:38,410][20381] Updated weights for policy 1, policy_version 673647 (0.0006) [2023-12-27 10:17:38,415][20347] Updated weights for policy 0, policy_version 674283 (0.0011) [2023-12-27 10:17:39,093][20347] Updated weights for policy 0, policy_version 674293 (0.0009) [2023-12-27 10:17:39,105][20381] Updated weights for policy 1, policy_version 673657 (0.0007) [2023-12-27 10:17:39,141][20347] Updated weights for policy 0, policy_version 674303 (0.0008) [2023-12-27 10:17:39,162][20381] Updated weights for policy 1, policy_version 673667 (0.0006) [2023-12-27 10:17:39,189][20347] Updated weights for policy 0, policy_version 674313 (0.0007) [2023-12-27 10:17:39,213][20381] Updated weights for policy 1, policy_version 673677 (0.0005) [2023-12-27 10:17:39,282][20381] Updated weights for policy 1, policy_version 673687 (0.0007) [2023-12-27 10:17:39,833][20347] Updated weights for policy 0, policy_version 674323 (0.0008) [2023-12-27 10:17:39,901][20347] Updated weights for policy 0, policy_version 674333 (0.0008) [2023-12-27 10:17:39,964][20347] Updated weights for policy 0, policy_version 674343 (0.0008) [2023-12-27 10:17:39,966][20381] Updated weights for policy 1, policy_version 673697 (0.0006) [2023-12-27 10:17:40,021][20381] Updated weights for policy 1, policy_version 673707 (0.0007) [2023-12-27 10:17:40,071][20381] Updated weights for policy 1, policy_version 673717 (0.0008) [2023-12-27 10:17:40,609][20347] Updated weights for policy 0, policy_version 674353 (0.0006) [2023-12-27 10:17:40,673][20347] Updated weights for policy 0, policy_version 674363 (0.0008) [2023-12-27 10:17:40,738][20347] Updated weights for policy 0, policy_version 674373 (0.0007) [2023-12-27 10:17:40,780][20381] Updated weights for policy 1, policy_version 673727 (0.0009) [2023-12-27 10:17:40,799][20347] Updated weights for policy 0, policy_version 674383 (0.0006) [2023-12-27 10:17:40,825][20381] Updated weights for policy 1, policy_version 673737 (0.0010) [2023-12-27 10:17:40,872][20381] Updated weights for policy 1, policy_version 673747 (0.0010) [2023-12-27 10:17:41,473][20347] Updated weights for policy 0, policy_version 674393 (0.0006) [2023-12-27 10:17:41,539][20347] Updated weights for policy 0, policy_version 674403 (0.0009) [2023-12-27 10:17:41,570][20381] Updated weights for policy 1, policy_version 673757 (0.0008) [2023-12-27 10:17:41,593][20347] Updated weights for policy 0, policy_version 674413 (0.0008) [2023-12-27 10:17:41,634][20381] Updated weights for policy 1, policy_version 673767 (0.0008) [2023-12-27 10:17:41,703][20381] Updated weights for policy 1, policy_version 673777 (0.0010) [2023-12-27 10:17:42,186][20347] Updated weights for policy 0, policy_version 674423 (0.0006) [2023-12-27 10:17:42,236][20347] Updated weights for policy 0, policy_version 674433 (0.0007) [2023-12-27 10:17:42,299][20347] Updated weights for policy 0, policy_version 674443 (0.0007) [2023-12-27 10:17:42,308][20381] Updated weights for policy 1, policy_version 673787 (0.0009) [2023-12-27 10:17:42,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 345194496. Throughput: 0: 10576.1, 1: 10202.1. Samples: 345182620. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:42,366][19170] Avg episode reward: [(0, '804.400'), (1, '1000.000')] [2023-12-27 10:17:42,370][20381] Updated weights for policy 1, policy_version 673797 (0.0011) [2023-12-27 10:17:42,434][20381] Updated weights for policy 1, policy_version 673807 (0.0009) [2023-12-27 10:17:42,984][20347] Updated weights for policy 0, policy_version 674453 (0.0007) [2023-12-27 10:17:43,039][20347] Updated weights for policy 0, policy_version 674463 (0.0007) [2023-12-27 10:17:43,042][20381] Updated weights for policy 1, policy_version 673817 (0.0008) [2023-12-27 10:17:43,098][20347] Updated weights for policy 0, policy_version 674473 (0.0006) [2023-12-27 10:17:43,100][20381] Updated weights for policy 1, policy_version 673827 (0.0006) [2023-12-27 10:17:43,160][20381] Updated weights for policy 1, policy_version 673837 (0.0006) [2023-12-27 10:17:43,208][20381] Updated weights for policy 1, policy_version 673847 (0.0007) [2023-12-27 10:17:43,773][20347] Updated weights for policy 0, policy_version 674483 (0.0006) [2023-12-27 10:17:43,829][20347] Updated weights for policy 0, policy_version 674493 (0.0007) [2023-12-27 10:17:43,864][20381] Updated weights for policy 1, policy_version 673857 (0.0007) [2023-12-27 10:17:43,882][20347] Updated weights for policy 0, policy_version 674503 (0.0006) [2023-12-27 10:17:43,913][20381] Updated weights for policy 1, policy_version 673867 (0.0006) [2023-12-27 10:17:43,963][20381] Updated weights for policy 1, policy_version 673877 (0.0007) [2023-12-27 10:17:44,515][20347] Updated weights for policy 0, policy_version 674513 (0.0006) [2023-12-27 10:17:44,578][20347] Updated weights for policy 0, policy_version 674523 (0.0005) [2023-12-27 10:17:44,618][20381] Updated weights for policy 1, policy_version 673887 (0.0006) [2023-12-27 10:17:44,629][20347] Updated weights for policy 0, policy_version 674533 (0.0005) [2023-12-27 10:17:44,675][20381] Updated weights for policy 1, policy_version 673897 (0.0006) [2023-12-27 10:17:44,676][20347] Updated weights for policy 0, policy_version 674543 (0.0007) [2023-12-27 10:17:44,725][20381] Updated weights for policy 1, policy_version 673907 (0.0005) [2023-12-27 10:17:45,274][20381] Updated weights for policy 1, policy_version 673917 (0.0005) [2023-12-27 10:17:45,325][20381] Updated weights for policy 1, policy_version 673927 (0.0005) [2023-12-27 10:17:45,358][20347] Updated weights for policy 0, policy_version 674553 (0.0005) [2023-12-27 10:17:45,380][20381] Updated weights for policy 1, policy_version 673937 (0.0007) [2023-12-27 10:17:45,401][20347] Updated weights for policy 0, policy_version 674563 (0.0005) [2023-12-27 10:17:45,447][20347] Updated weights for policy 0, policy_version 674573 (0.0005) [2023-12-27 10:17:46,022][20347] Updated weights for policy 0, policy_version 674583 (0.0005) [2023-12-27 10:17:46,052][20381] Updated weights for policy 1, policy_version 673947 (0.0009) [2023-12-27 10:17:46,076][20347] Updated weights for policy 0, policy_version 674593 (0.0006) [2023-12-27 10:17:46,112][20381] Updated weights for policy 1, policy_version 673957 (0.0007) [2023-12-27 10:17:46,129][20347] Updated weights for policy 0, policy_version 674603 (0.0006) [2023-12-27 10:17:46,172][20381] Updated weights for policy 1, policy_version 673967 (0.0008) [2023-12-27 10:17:46,710][20347] Updated weights for policy 0, policy_version 674613 (0.0005) [2023-12-27 10:17:46,769][20381] Updated weights for policy 1, policy_version 673977 (0.0008) [2023-12-27 10:17:46,771][20347] Updated weights for policy 0, policy_version 674623 (0.0005) [2023-12-27 10:17:46,830][20381] Updated weights for policy 1, policy_version 673987 (0.0010) [2023-12-27 10:17:46,831][20347] Updated weights for policy 0, policy_version 674633 (0.0005) [2023-12-27 10:17:46,895][20381] Updated weights for policy 1, policy_version 673997 (0.0011) [2023-12-27 10:17:46,960][20381] Updated weights for policy 1, policy_version 674007 (0.0010) [2023-12-27 10:17:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 345300992. Throughput: 0: 10662.9, 1: 10274.8. Samples: 345312724. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:47,363][19170] Avg episode reward: [(0, '843.570'), (1, '1000.000')] [2023-12-27 10:17:47,443][20347] Updated weights for policy 0, policy_version 674643 (0.0006) [2023-12-27 10:17:47,499][20347] Updated weights for policy 0, policy_version 674653 (0.0006) [2023-12-27 10:17:47,549][20381] Updated weights for policy 1, policy_version 674017 (0.0006) [2023-12-27 10:17:47,553][20347] Updated weights for policy 0, policy_version 674663 (0.0010) [2023-12-27 10:17:47,610][20381] Updated weights for policy 1, policy_version 674027 (0.0006) [2023-12-27 10:17:47,660][20381] Updated weights for policy 1, policy_version 674037 (0.0008) [2023-12-27 10:17:48,074][20347] Updated weights for policy 0, policy_version 674673 (0.0005) [2023-12-27 10:17:48,130][20347] Updated weights for policy 0, policy_version 674683 (0.0005) [2023-12-27 10:17:48,190][20347] Updated weights for policy 0, policy_version 674693 (0.0008) [2023-12-27 10:17:48,230][20381] Updated weights for policy 1, policy_version 674047 (0.0008) [2023-12-27 10:17:48,243][20347] Updated weights for policy 0, policy_version 674703 (0.0011) [2023-12-27 10:17:48,279][20381] Updated weights for policy 1, policy_version 674057 (0.0006) [2023-12-27 10:17:48,336][20381] Updated weights for policy 1, policy_version 674067 (0.0008) [2023-12-27 10:17:48,895][20347] Updated weights for policy 0, policy_version 674713 (0.0010) [2023-12-27 10:17:48,953][20347] Updated weights for policy 0, policy_version 674723 (0.0010) [2023-12-27 10:17:49,011][20347] Updated weights for policy 0, policy_version 674733 (0.0010) [2023-12-27 10:17:49,018][20381] Updated weights for policy 1, policy_version 674077 (0.0011) [2023-12-27 10:17:49,082][20381] Updated weights for policy 1, policy_version 674087 (0.0010) [2023-12-27 10:17:49,135][20381] Updated weights for policy 1, policy_version 674097 (0.0006) [2023-12-27 10:17:49,714][20381] Updated weights for policy 1, policy_version 674107 (0.0006) [2023-12-27 10:17:49,746][20347] Updated weights for policy 0, policy_version 674743 (0.0007) [2023-12-27 10:17:49,774][20381] Updated weights for policy 1, policy_version 674117 (0.0007) [2023-12-27 10:17:49,811][20347] Updated weights for policy 0, policy_version 674753 (0.0007) [2023-12-27 10:17:49,834][20381] Updated weights for policy 1, policy_version 674127 (0.0008) [2023-12-27 10:17:49,868][20347] Updated weights for policy 0, policy_version 674763 (0.0007) [2023-12-27 10:17:50,444][20381] Updated weights for policy 1, policy_version 674137 (0.0006) [2023-12-27 10:17:50,509][20381] Updated weights for policy 1, policy_version 674147 (0.0008) [2023-12-27 10:17:50,572][20381] Updated weights for policy 1, policy_version 674157 (0.0008) [2023-12-27 10:17:50,581][20347] Updated weights for policy 0, policy_version 674773 (0.0008) [2023-12-27 10:17:50,632][20347] Updated weights for policy 0, policy_version 674783 (0.0006) [2023-12-27 10:17:50,638][20381] Updated weights for policy 1, policy_version 674167 (0.0007) [2023-12-27 10:17:50,681][20347] Updated weights for policy 0, policy_version 674793 (0.0007) [2023-12-27 10:17:51,321][20381] Updated weights for policy 1, policy_version 674177 (0.0008) [2023-12-27 10:17:51,388][20381] Updated weights for policy 1, policy_version 674187 (0.0009) [2023-12-27 10:17:51,407][20347] Updated weights for policy 0, policy_version 674803 (0.0008) [2023-12-27 10:17:51,448][20381] Updated weights for policy 1, policy_version 674197 (0.0011) [2023-12-27 10:17:51,461][20347] Updated weights for policy 0, policy_version 674813 (0.0006) [2023-12-27 10:17:51,516][20347] Updated weights for policy 0, policy_version 674823 (0.0007) [2023-12-27 10:17:52,085][20381] Updated weights for policy 1, policy_version 674207 (0.0011) [2023-12-27 10:17:52,142][20381] Updated weights for policy 1, policy_version 674217 (0.0009) [2023-12-27 10:17:52,201][20381] Updated weights for policy 1, policy_version 674227 (0.0008) [2023-12-27 10:17:52,253][20347] Updated weights for policy 0, policy_version 674833 (0.0008) [2023-12-27 10:17:52,307][20347] Updated weights for policy 0, policy_version 674843 (0.0006) [2023-12-27 10:17:52,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.5, 300 sec: 20716.0). Total num frames: 345407488. Throughput: 0: 10704.3, 1: 10305.7. Samples: 345378716. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:52,363][19170] Avg episode reward: [(0, '863.290'), (1, '1000.000')] [2023-12-27 10:17:52,366][20347] Updated weights for policy 0, policy_version 674853 (0.0007) [2023-12-27 10:17:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000674232_172630016.pth... [2023-12-27 10:17:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000673016_172318720.pth [2023-12-27 10:17:52,423][20347] Updated weights for policy 0, policy_version 674863 (0.0007) [2023-12-27 10:17:52,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000674864_172785664.pth... [2023-12-27 10:17:52,433][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000673584_172457984.pth [2023-12-27 10:17:52,918][20381] Updated weights for policy 1, policy_version 674237 (0.0009) [2023-12-27 10:17:52,977][20381] Updated weights for policy 1, policy_version 674247 (0.0008) [2023-12-27 10:17:53,034][20381] Updated weights for policy 1, policy_version 674257 (0.0006) [2023-12-27 10:17:53,087][20347] Updated weights for policy 0, policy_version 674873 (0.0008) [2023-12-27 10:17:53,142][20347] Updated weights for policy 0, policy_version 674883 (0.0008) [2023-12-27 10:17:53,195][20347] Updated weights for policy 0, policy_version 674893 (0.0009) [2023-12-27 10:17:53,653][20381] Updated weights for policy 1, policy_version 674267 (0.0006) [2023-12-27 10:17:53,712][20381] Updated weights for policy 1, policy_version 674277 (0.0008) [2023-12-27 10:17:53,768][20381] Updated weights for policy 1, policy_version 674287 (0.0008) [2023-12-27 10:17:53,885][20347] Updated weights for policy 0, policy_version 674903 (0.0008) [2023-12-27 10:17:53,934][20347] Updated weights for policy 0, policy_version 674913 (0.0008) [2023-12-27 10:17:53,990][20347] Updated weights for policy 0, policy_version 674923 (0.0008) [2023-12-27 10:17:54,452][20381] Updated weights for policy 1, policy_version 674297 (0.0007) [2023-12-27 10:17:54,501][20381] Updated weights for policy 1, policy_version 674307 (0.0005) [2023-12-27 10:17:54,550][20381] Updated weights for policy 1, policy_version 674317 (0.0005) [2023-12-27 10:17:54,597][20381] Updated weights for policy 1, policy_version 674327 (0.0005) [2023-12-27 10:17:54,700][20347] Updated weights for policy 0, policy_version 674933 (0.0007) [2023-12-27 10:17:54,761][20347] Updated weights for policy 0, policy_version 674943 (0.0007) [2023-12-27 10:17:54,813][20347] Updated weights for policy 0, policy_version 674953 (0.0008) [2023-12-27 10:17:55,185][20381] Updated weights for policy 1, policy_version 674337 (0.0008) [2023-12-27 10:17:55,242][20381] Updated weights for policy 1, policy_version 674347 (0.0008) [2023-12-27 10:17:55,305][20381] Updated weights for policy 1, policy_version 674358 (0.0008) [2023-12-27 10:17:55,440][20347] Updated weights for policy 0, policy_version 674963 (0.0008) [2023-12-27 10:17:55,487][20347] Updated weights for policy 0, policy_version 674973 (0.0008) [2023-12-27 10:17:55,533][20347] Updated weights for policy 0, policy_version 674983 (0.0005) [2023-12-27 10:17:55,927][20381] Updated weights for policy 1, policy_version 674368 (0.0007) [2023-12-27 10:17:55,981][20381] Updated weights for policy 1, policy_version 674378 (0.0007) [2023-12-27 10:17:56,037][20381] Updated weights for policy 1, policy_version 674388 (0.0006) [2023-12-27 10:17:56,168][20347] Updated weights for policy 0, policy_version 674993 (0.0006) [2023-12-27 10:17:56,223][20347] Updated weights for policy 0, policy_version 675003 (0.0009) [2023-12-27 10:17:56,281][20347] Updated weights for policy 0, policy_version 675013 (0.0007) [2023-12-27 10:17:56,339][20347] Updated weights for policy 0, policy_version 675023 (0.0007) [2023-12-27 10:17:56,698][20381] Updated weights for policy 1, policy_version 674398 (0.0006) [2023-12-27 10:17:56,756][20381] Updated weights for policy 1, policy_version 674408 (0.0007) [2023-12-27 10:17:56,814][20381] Updated weights for policy 1, policy_version 674418 (0.0010) [2023-12-27 10:17:56,945][20347] Updated weights for policy 0, policy_version 675033 (0.0007) [2023-12-27 10:17:56,996][20347] Updated weights for policy 0, policy_version 675043 (0.0010) [2023-12-27 10:17:57,051][20347] Updated weights for policy 0, policy_version 675053 (0.0010) [2023-12-27 10:17:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 345513984. Throughput: 0: 10607.2, 1: 10352.6. Samples: 345502908. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:17:57,363][19170] Avg episode reward: [(0, '873.390'), (1, '990.420')] [2023-12-27 10:17:57,464][20381] Updated weights for policy 1, policy_version 674428 (0.0008) [2023-12-27 10:17:57,524][20381] Updated weights for policy 1, policy_version 674438 (0.0006) [2023-12-27 10:17:57,570][20381] Updated weights for policy 1, policy_version 674448 (0.0007) [2023-12-27 10:17:57,655][20347] Updated weights for policy 0, policy_version 675063 (0.0008) [2023-12-27 10:17:57,705][20347] Updated weights for policy 0, policy_version 675073 (0.0007) [2023-12-27 10:17:57,756][20347] Updated weights for policy 0, policy_version 675083 (0.0007) [2023-12-27 10:17:58,135][20381] Updated weights for policy 1, policy_version 674458 (0.0006) [2023-12-27 10:17:58,190][20381] Updated weights for policy 1, policy_version 674468 (0.0007) [2023-12-27 10:17:58,249][20381] Updated weights for policy 1, policy_version 674478 (0.0007) [2023-12-27 10:17:58,305][20381] Updated weights for policy 1, policy_version 674488 (0.0008) [2023-12-27 10:17:58,514][20347] Updated weights for policy 0, policy_version 675093 (0.0007) [2023-12-27 10:17:58,585][20347] Updated weights for policy 0, policy_version 675103 (0.0007) [2023-12-27 10:17:58,649][20347] Updated weights for policy 0, policy_version 675113 (0.0007) [2023-12-27 10:17:59,028][20381] Updated weights for policy 1, policy_version 674498 (0.0007) [2023-12-27 10:17:59,079][20381] Updated weights for policy 1, policy_version 674508 (0.0010) [2023-12-27 10:17:59,130][20381] Updated weights for policy 1, policy_version 674518 (0.0010) [2023-12-27 10:17:59,330][20347] Updated weights for policy 0, policy_version 675123 (0.0007) [2023-12-27 10:17:59,392][20347] Updated weights for policy 0, policy_version 675133 (0.0008) [2023-12-27 10:17:59,445][20347] Updated weights for policy 0, policy_version 675143 (0.0007) [2023-12-27 10:17:59,836][20381] Updated weights for policy 1, policy_version 674528 (0.0007) [2023-12-27 10:17:59,887][20381] Updated weights for policy 1, policy_version 674538 (0.0008) [2023-12-27 10:17:59,946][20381] Updated weights for policy 1, policy_version 674548 (0.0007) [2023-12-27 10:18:00,122][20347] Updated weights for policy 0, policy_version 675153 (0.0007) [2023-12-27 10:18:00,178][20347] Updated weights for policy 0, policy_version 675163 (0.0007) [2023-12-27 10:18:00,233][20347] Updated weights for policy 0, policy_version 675173 (0.0008) [2023-12-27 10:18:00,292][20347] Updated weights for policy 0, policy_version 675183 (0.0007) [2023-12-27 10:18:00,542][20381] Updated weights for policy 1, policy_version 674558 (0.0005) [2023-12-27 10:18:00,600][20381] Updated weights for policy 1, policy_version 674568 (0.0005) [2023-12-27 10:18:00,660][20381] Updated weights for policy 1, policy_version 674578 (0.0005) [2023-12-27 10:18:01,026][20347] Updated weights for policy 0, policy_version 675193 (0.0009) [2023-12-27 10:18:01,087][20347] Updated weights for policy 0, policy_version 675203 (0.0008) [2023-12-27 10:18:01,146][20347] Updated weights for policy 0, policy_version 675213 (0.0007) [2023-12-27 10:18:01,243][20381] Updated weights for policy 1, policy_version 674588 (0.0006) [2023-12-27 10:18:01,306][20381] Updated weights for policy 1, policy_version 674598 (0.0006) [2023-12-27 10:18:01,370][20381] Updated weights for policy 1, policy_version 674608 (0.0009) [2023-12-27 10:18:01,903][20347] Updated weights for policy 0, policy_version 675223 (0.0008) [2023-12-27 10:18:01,954][20347] Updated weights for policy 0, policy_version 675233 (0.0009) [2023-12-27 10:18:02,011][20347] Updated weights for policy 0, policy_version 675243 (0.0006) [2023-12-27 10:18:02,052][20381] Updated weights for policy 1, policy_version 674618 (0.0008) [2023-12-27 10:18:02,119][20381] Updated weights for policy 1, policy_version 674628 (0.0008) [2023-12-27 10:18:02,186][20381] Updated weights for policy 1, policy_version 674638 (0.0008) [2023-12-27 10:18:02,246][20381] Updated weights for policy 1, policy_version 674648 (0.0007) [2023-12-27 10:18:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 345620480. Throughput: 0: 10571.4, 1: 10382.5. Samples: 345627200. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:18:02,363][19170] Avg episode reward: [(0, '892.600'), (1, '990.420')] [2023-12-27 10:18:02,719][20347] Updated weights for policy 0, policy_version 675253 (0.0006) [2023-12-27 10:18:02,775][20347] Updated weights for policy 0, policy_version 675263 (0.0005) [2023-12-27 10:18:02,830][20347] Updated weights for policy 0, policy_version 675273 (0.0006) [2023-12-27 10:18:02,881][20381] Updated weights for policy 1, policy_version 674658 (0.0005) [2023-12-27 10:18:02,933][20381] Updated weights for policy 1, policy_version 674668 (0.0005) [2023-12-27 10:18:02,990][20381] Updated weights for policy 1, policy_version 674678 (0.0008) [2023-12-27 10:18:03,455][20347] Updated weights for policy 0, policy_version 675283 (0.0007) [2023-12-27 10:18:03,511][20347] Updated weights for policy 0, policy_version 675293 (0.0006) [2023-12-27 10:18:03,536][20381] Updated weights for policy 1, policy_version 674688 (0.0006) [2023-12-27 10:18:03,559][20347] Updated weights for policy 0, policy_version 675303 (0.0007) [2023-12-27 10:18:03,593][20381] Updated weights for policy 1, policy_version 674698 (0.0005) [2023-12-27 10:18:03,649][20381] Updated weights for policy 1, policy_version 674708 (0.0005) [2023-12-27 10:18:04,188][20381] Updated weights for policy 1, policy_version 674718 (0.0007) [2023-12-27 10:18:04,220][20347] Updated weights for policy 0, policy_version 675313 (0.0007) [2023-12-27 10:18:04,244][20381] Updated weights for policy 1, policy_version 674728 (0.0007) [2023-12-27 10:18:04,279][20347] Updated weights for policy 0, policy_version 675323 (0.0007) [2023-12-27 10:18:04,309][20381] Updated weights for policy 1, policy_version 674738 (0.0008) [2023-12-27 10:18:04,337][20347] Updated weights for policy 0, policy_version 675333 (0.0005) [2023-12-27 10:18:04,400][20347] Updated weights for policy 0, policy_version 675343 (0.0005) [2023-12-27 10:18:04,939][20347] Updated weights for policy 0, policy_version 675353 (0.0005) [2023-12-27 10:18:04,983][20381] Updated weights for policy 1, policy_version 674748 (0.0008) [2023-12-27 10:18:04,994][20347] Updated weights for policy 0, policy_version 675363 (0.0007) [2023-12-27 10:18:05,042][20381] Updated weights for policy 1, policy_version 674758 (0.0007) [2023-12-27 10:18:05,049][20347] Updated weights for policy 0, policy_version 675373 (0.0008) [2023-12-27 10:18:05,101][20381] Updated weights for policy 1, policy_version 674768 (0.0006) [2023-12-27 10:18:05,619][20347] Updated weights for policy 0, policy_version 675383 (0.0009) [2023-12-27 10:18:05,674][20347] Updated weights for policy 0, policy_version 675393 (0.0007) [2023-12-27 10:18:05,725][20347] Updated weights for policy 0, policy_version 675403 (0.0008) [2023-12-27 10:18:05,756][20381] Updated weights for policy 1, policy_version 674778 (0.0007) [2023-12-27 10:18:05,807][20381] Updated weights for policy 1, policy_version 674788 (0.0009) [2023-12-27 10:18:05,853][20381] Updated weights for policy 1, policy_version 674798 (0.0007) [2023-12-27 10:18:05,904][20381] Updated weights for policy 1, policy_version 674808 (0.0008) [2023-12-27 10:18:06,383][20347] Updated weights for policy 0, policy_version 675413 (0.0009) [2023-12-27 10:18:06,443][20347] Updated weights for policy 0, policy_version 675423 (0.0007) [2023-12-27 10:18:06,502][20347] Updated weights for policy 0, policy_version 675433 (0.0008) [2023-12-27 10:18:06,630][20381] Updated weights for policy 1, policy_version 674818 (0.0008) [2023-12-27 10:18:06,690][20381] Updated weights for policy 1, policy_version 674828 (0.0008) [2023-12-27 10:18:06,747][20381] Updated weights for policy 1, policy_version 674838 (0.0007) [2023-12-27 10:18:07,193][20347] Updated weights for policy 0, policy_version 675443 (0.0008) [2023-12-27 10:18:07,250][20347] Updated weights for policy 0, policy_version 675453 (0.0007) [2023-12-27 10:18:07,312][20347] Updated weights for policy 0, policy_version 675463 (0.0006) [2023-12-27 10:18:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 345718784. Throughput: 0: 10601.7, 1: 10431.5. Samples: 345693452. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:18:07,363][19170] Avg episode reward: [(0, '911.950'), (1, '990.610')] [2023-12-27 10:18:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000675472_172941312.pth... [2023-12-27 10:18:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000674224_172621824.pth [2023-12-27 10:18:07,372][20381] Updated weights for policy 1, policy_version 674848 (0.0007) [2023-12-27 10:18:07,432][20381] Updated weights for policy 1, policy_version 674858 (0.0008) [2023-12-27 10:18:07,494][20381] Updated weights for policy 1, policy_version 674868 (0.0007) [2023-12-27 10:18:07,514][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000674872_172793856.pth... [2023-12-27 10:18:07,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000673592_172466176.pth [2023-12-27 10:18:07,974][20347] Updated weights for policy 0, policy_version 675473 (0.0006) [2023-12-27 10:18:08,038][20347] Updated weights for policy 0, policy_version 675483 (0.0007) [2023-12-27 10:18:08,097][20347] Updated weights for policy 0, policy_version 675493 (0.0008) [2023-12-27 10:18:08,132][20381] Updated weights for policy 1, policy_version 674878 (0.0007) [2023-12-27 10:18:08,156][20347] Updated weights for policy 0, policy_version 675503 (0.0005) [2023-12-27 10:18:08,169][20292] KL-divergence is very high: 101.8924 [2023-12-27 10:18:08,183][20381] Updated weights for policy 1, policy_version 674888 (0.0006) [2023-12-27 10:18:08,226][20381] Updated weights for policy 1, policy_version 674898 (0.0008) [2023-12-27 10:18:08,860][20347] Updated weights for policy 0, policy_version 675513 (0.0007) [2023-12-27 10:18:08,910][20347] Updated weights for policy 0, policy_version 675523 (0.0008) [2023-12-27 10:18:08,948][20381] Updated weights for policy 1, policy_version 674908 (0.0006) [2023-12-27 10:18:08,961][20347] Updated weights for policy 0, policy_version 675533 (0.0007) [2023-12-27 10:18:08,996][20381] Updated weights for policy 1, policy_version 674918 (0.0006) [2023-12-27 10:18:09,046][20381] Updated weights for policy 1, policy_version 674928 (0.0008) [2023-12-27 10:18:09,663][20347] Updated weights for policy 0, policy_version 675543 (0.0009) [2023-12-27 10:18:09,715][20347] Updated weights for policy 0, policy_version 675553 (0.0010) [2023-12-27 10:18:09,758][20381] Updated weights for policy 1, policy_version 674938 (0.0007) [2023-12-27 10:18:09,772][20347] Updated weights for policy 0, policy_version 675563 (0.0010) [2023-12-27 10:18:09,823][20381] Updated weights for policy 1, policy_version 674948 (0.0006) [2023-12-27 10:18:09,883][20381] Updated weights for policy 1, policy_version 674958 (0.0007) [2023-12-27 10:18:09,943][20381] Updated weights for policy 1, policy_version 674968 (0.0006) [2023-12-27 10:18:10,494][20347] Updated weights for policy 0, policy_version 675573 (0.0011) [2023-12-27 10:18:10,543][20347] Updated weights for policy 0, policy_version 675583 (0.0010) [2023-12-27 10:18:10,592][20347] Updated weights for policy 0, policy_version 675593 (0.0010) [2023-12-27 10:18:10,628][20381] Updated weights for policy 1, policy_version 674978 (0.0005) [2023-12-27 10:18:10,676][20381] Updated weights for policy 1, policy_version 674988 (0.0007) [2023-12-27 10:18:10,729][20381] Updated weights for policy 1, policy_version 674998 (0.0007) [2023-12-27 10:18:11,313][20347] Updated weights for policy 0, policy_version 675603 (0.0011) [2023-12-27 10:18:11,389][20347] Updated weights for policy 0, policy_version 675613 (0.0009) [2023-12-27 10:18:11,428][20381] Updated weights for policy 1, policy_version 675008 (0.0007) [2023-12-27 10:18:11,447][20347] Updated weights for policy 0, policy_version 675623 (0.0007) [2023-12-27 10:18:11,485][20381] Updated weights for policy 1, policy_version 675018 (0.0007) [2023-12-27 10:18:11,538][20381] Updated weights for policy 1, policy_version 675028 (0.0007) [2023-12-27 10:18:12,130][20347] Updated weights for policy 0, policy_version 675633 (0.0006) [2023-12-27 10:18:12,185][20347] Updated weights for policy 0, policy_version 675643 (0.0008) [2023-12-27 10:18:12,240][20347] Updated weights for policy 0, policy_version 675653 (0.0007) [2023-12-27 10:18:12,254][20381] Updated weights for policy 1, policy_version 675038 (0.0010) [2023-12-27 10:18:12,301][20347] Updated weights for policy 0, policy_version 675663 (0.0008) [2023-12-27 10:18:12,319][20381] Updated weights for policy 1, policy_version 675048 (0.0011) [2023-12-27 10:18:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 345825280. Throughput: 0: 10547.0, 1: 10473.0. Samples: 345816716. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:18:12,363][19170] Avg episode reward: [(0, '911.920'), (1, '980.970')] [2023-12-27 10:18:12,381][20381] Updated weights for policy 1, policy_version 675058 (0.0011) [2023-12-27 10:18:12,955][20347] Updated weights for policy 0, policy_version 675673 (0.0006) [2023-12-27 10:18:13,015][20347] Updated weights for policy 0, policy_version 675683 (0.0008) [2023-12-27 10:18:13,065][20347] Updated weights for policy 0, policy_version 675693 (0.0006) [2023-12-27 10:18:13,081][20381] Updated weights for policy 1, policy_version 675068 (0.0009) [2023-12-27 10:18:13,134][20381] Updated weights for policy 1, policy_version 675078 (0.0008) [2023-12-27 10:18:13,189][20381] Updated weights for policy 1, policy_version 675088 (0.0008) [2023-12-27 10:18:13,647][20347] Updated weights for policy 0, policy_version 675703 (0.0009) [2023-12-27 10:18:13,696][20347] Updated weights for policy 0, policy_version 675713 (0.0008) [2023-12-27 10:18:13,748][20347] Updated weights for policy 0, policy_version 675723 (0.0005) [2023-12-27 10:18:13,850][20381] Updated weights for policy 1, policy_version 675098 (0.0008) [2023-12-27 10:18:13,899][20381] Updated weights for policy 1, policy_version 675108 (0.0010) [2023-12-27 10:18:13,957][20381] Updated weights for policy 1, policy_version 675118 (0.0010) [2023-12-27 10:18:14,008][20381] Updated weights for policy 1, policy_version 675128 (0.0010) [2023-12-27 10:18:14,395][20347] Updated weights for policy 0, policy_version 675733 (0.0005) [2023-12-27 10:18:14,455][20347] Updated weights for policy 0, policy_version 675743 (0.0007) [2023-12-27 10:18:14,508][20347] Updated weights for policy 0, policy_version 675753 (0.0007) [2023-12-27 10:18:14,690][20381] Updated weights for policy 1, policy_version 675138 (0.0010) [2023-12-27 10:18:14,738][20381] Updated weights for policy 1, policy_version 675148 (0.0011) [2023-12-27 10:18:14,799][20381] Updated weights for policy 1, policy_version 675158 (0.0011) [2023-12-27 10:18:15,184][20347] Updated weights for policy 0, policy_version 675763 (0.0007) [2023-12-27 10:18:15,240][20347] Updated weights for policy 0, policy_version 675773 (0.0007) [2023-12-27 10:18:15,298][20347] Updated weights for policy 0, policy_version 675783 (0.0007) [2023-12-27 10:18:15,535][20381] Updated weights for policy 1, policy_version 675168 (0.0007) [2023-12-27 10:18:15,594][20381] Updated weights for policy 1, policy_version 675178 (0.0008) [2023-12-27 10:18:15,658][20381] Updated weights for policy 1, policy_version 675188 (0.0008) [2023-12-27 10:18:15,957][20347] Updated weights for policy 0, policy_version 675793 (0.0008) [2023-12-27 10:18:16,012][20347] Updated weights for policy 0, policy_version 675803 (0.0008) [2023-12-27 10:18:16,060][20347] Updated weights for policy 0, policy_version 675813 (0.0007) [2023-12-27 10:18:16,109][20347] Updated weights for policy 0, policy_version 675823 (0.0007) [2023-12-27 10:18:16,257][20381] Updated weights for policy 1, policy_version 675198 (0.0006) [2023-12-27 10:18:16,315][20381] Updated weights for policy 1, policy_version 675208 (0.0005) [2023-12-27 10:18:16,371][20381] Updated weights for policy 1, policy_version 675218 (0.0005) [2023-12-27 10:18:16,810][20347] Updated weights for policy 0, policy_version 675833 (0.0006) [2023-12-27 10:18:16,865][20347] Updated weights for policy 0, policy_version 675843 (0.0005) [2023-12-27 10:18:16,923][20347] Updated weights for policy 0, policy_version 675853 (0.0007) [2023-12-27 10:18:17,013][20381] Updated weights for policy 1, policy_version 675228 (0.0006) [2023-12-27 10:18:17,073][20381] Updated weights for policy 1, policy_version 675238 (0.0005) [2023-12-27 10:18:17,120][20381] Updated weights for policy 1, policy_version 675248 (0.0005) [2023-12-27 10:18:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 345931776. Throughput: 0: 10440.0, 1: 10512.8. Samples: 345940924. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 10:18:17,363][19170] Avg episode reward: [(0, '921.790'), (1, '990.360')] [2023-12-27 10:18:17,538][20347] Updated weights for policy 0, policy_version 675863 (0.0007) [2023-12-27 10:18:17,605][20347] Updated weights for policy 0, policy_version 675873 (0.0006) [2023-12-27 10:18:17,653][20347] Updated weights for policy 0, policy_version 675883 (0.0008) [2023-12-27 10:18:17,692][20381] Updated weights for policy 1, policy_version 675258 (0.0006) [2023-12-27 10:18:17,745][20381] Updated weights for policy 1, policy_version 675268 (0.0008) [2023-12-27 10:18:17,791][20381] Updated weights for policy 1, policy_version 675278 (0.0007) [2023-12-27 10:18:17,837][20381] Updated weights for policy 1, policy_version 675288 (0.0007) [2023-12-27 10:18:18,231][20347] Updated weights for policy 0, policy_version 675893 (0.0007) [2023-12-27 10:18:18,280][20347] Updated weights for policy 0, policy_version 675903 (0.0006) [2023-12-27 10:18:18,328][20347] Updated weights for policy 0, policy_version 675913 (0.0006) [2023-12-27 10:18:18,577][20381] Updated weights for policy 1, policy_version 675298 (0.0007) [2023-12-27 10:18:18,642][20381] Updated weights for policy 1, policy_version 675308 (0.0006) [2023-12-27 10:18:18,707][20381] Updated weights for policy 1, policy_version 675318 (0.0005) [2023-12-27 10:18:19,016][20347] Updated weights for policy 0, policy_version 675923 (0.0008) [2023-12-27 10:18:19,067][20347] Updated weights for policy 0, policy_version 675933 (0.0005) [2023-12-27 10:18:19,125][20347] Updated weights for policy 0, policy_version 675943 (0.0005) [2023-12-27 10:18:19,329][20381] Updated weights for policy 1, policy_version 675328 (0.0009) [2023-12-27 10:18:19,392][20381] Updated weights for policy 1, policy_version 675338 (0.0010) [2023-12-27 10:18:19,452][20381] Updated weights for policy 1, policy_version 675348 (0.0010) [2023-12-27 10:18:19,810][20347] Updated weights for policy 0, policy_version 675953 (0.0006) [2023-12-27 10:18:19,869][20347] Updated weights for policy 0, policy_version 675963 (0.0007) [2023-12-27 10:18:19,930][20347] Updated weights for policy 0, policy_version 675973 (0.0006) [2023-12-27 10:18:19,996][20347] Updated weights for policy 0, policy_version 675983 (0.0007) [2023-12-27 10:18:20,155][20381] Updated weights for policy 1, policy_version 675358 (0.0010) [2023-12-27 10:18:20,214][20381] Updated weights for policy 1, policy_version 675368 (0.0007) [2023-12-27 10:18:20,279][20381] Updated weights for policy 1, policy_version 675378 (0.0006) [2023-12-27 10:18:20,680][20347] Updated weights for policy 0, policy_version 675993 (0.0007) [2023-12-27 10:18:20,735][20347] Updated weights for policy 0, policy_version 676003 (0.0007) [2023-12-27 10:18:20,791][20347] Updated weights for policy 0, policy_version 676013 (0.0007) [2023-12-27 10:18:20,973][20381] Updated weights for policy 1, policy_version 675388 (0.0006) [2023-12-27 10:18:21,034][20381] Updated weights for policy 1, policy_version 675398 (0.0007) [2023-12-27 10:18:21,098][20381] Updated weights for policy 1, policy_version 675408 (0.0006) [2023-12-27 10:18:21,520][20347] Updated weights for policy 0, policy_version 676023 (0.0007) [2023-12-27 10:18:21,579][20347] Updated weights for policy 0, policy_version 676033 (0.0008) [2023-12-27 10:18:21,642][20347] Updated weights for policy 0, policy_version 676043 (0.0008) [2023-12-27 10:18:21,767][20381] Updated weights for policy 1, policy_version 675418 (0.0007) [2023-12-27 10:18:21,817][20381] Updated weights for policy 1, policy_version 675428 (0.0009) [2023-12-27 10:18:21,868][20381] Updated weights for policy 1, policy_version 675438 (0.0008) [2023-12-27 10:18:21,933][20381] Updated weights for policy 1, policy_version 675448 (0.0008) [2023-12-27 10:18:22,317][20347] Updated weights for policy 0, policy_version 676053 (0.0008) [2023-12-27 10:18:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 346030080. Throughput: 0: 10441.2, 1: 10535.5. Samples: 346003636. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:22,363][19170] Avg episode reward: [(0, '921.850'), (1, '1000.000')] [2023-12-27 10:18:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000675448_172941312.pth... [2023-12-27 10:18:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000674232_172630016.pth [2023-12-27 10:18:22,378][20347] Updated weights for policy 0, policy_version 676063 (0.0008) [2023-12-27 10:18:22,428][20347] Updated weights for policy 0, policy_version 676073 (0.0009) [2023-12-27 10:18:22,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000676080_173096960.pth... [2023-12-27 10:18:22,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000674864_172785664.pth [2023-12-27 10:18:22,641][20381] Updated weights for policy 1, policy_version 675458 (0.0005) [2023-12-27 10:18:22,699][20381] Updated weights for policy 1, policy_version 675468 (0.0006) [2023-12-27 10:18:22,755][20381] Updated weights for policy 1, policy_version 675478 (0.0007) [2023-12-27 10:18:23,224][20347] Updated weights for policy 0, policy_version 676083 (0.0008) [2023-12-27 10:18:23,277][20347] Updated weights for policy 0, policy_version 676093 (0.0008) [2023-12-27 10:18:23,321][20381] Updated weights for policy 1, policy_version 675488 (0.0006) [2023-12-27 10:18:23,334][20347] Updated weights for policy 0, policy_version 676103 (0.0006) [2023-12-27 10:18:23,370][20381] Updated weights for policy 1, policy_version 675498 (0.0006) [2023-12-27 10:18:23,431][20381] Updated weights for policy 1, policy_version 675508 (0.0007) [2023-12-27 10:18:24,030][20381] Updated weights for policy 1, policy_version 675518 (0.0008) [2023-12-27 10:18:24,048][20347] Updated weights for policy 0, policy_version 676113 (0.0006) [2023-12-27 10:18:24,095][20381] Updated weights for policy 1, policy_version 675528 (0.0009) [2023-12-27 10:18:24,105][20347] Updated weights for policy 0, policy_version 676123 (0.0008) [2023-12-27 10:18:24,145][20381] Updated weights for policy 1, policy_version 675538 (0.0008) [2023-12-27 10:18:24,168][20347] Updated weights for policy 0, policy_version 676133 (0.0006) [2023-12-27 10:18:24,223][20347] Updated weights for policy 0, policy_version 676143 (0.0007) [2023-12-27 10:18:24,789][20381] Updated weights for policy 1, policy_version 675548 (0.0007) [2023-12-27 10:18:24,844][20381] Updated weights for policy 1, policy_version 675558 (0.0005) [2023-12-27 10:18:24,893][20381] Updated weights for policy 1, policy_version 675568 (0.0005) [2023-12-27 10:18:24,910][20347] Updated weights for policy 0, policy_version 676153 (0.0007) [2023-12-27 10:18:24,966][20347] Updated weights for policy 0, policy_version 676163 (0.0006) [2023-12-27 10:18:25,022][20347] Updated weights for policy 0, policy_version 676173 (0.0008) [2023-12-27 10:18:25,486][20381] Updated weights for policy 1, policy_version 675578 (0.0009) [2023-12-27 10:18:25,548][20381] Updated weights for policy 1, policy_version 675588 (0.0010) [2023-12-27 10:18:25,612][20381] Updated weights for policy 1, policy_version 675598 (0.0008) [2023-12-27 10:18:25,630][20347] Updated weights for policy 0, policy_version 676183 (0.0006) [2023-12-27 10:18:25,665][20381] Updated weights for policy 1, policy_version 675608 (0.0006) [2023-12-27 10:18:25,690][20347] Updated weights for policy 0, policy_version 676193 (0.0007) [2023-12-27 10:18:25,747][20347] Updated weights for policy 0, policy_version 676203 (0.0008) [2023-12-27 10:18:26,330][20381] Updated weights for policy 1, policy_version 675618 (0.0007) [2023-12-27 10:18:26,378][20381] Updated weights for policy 1, policy_version 675628 (0.0008) [2023-12-27 10:18:26,385][20347] Updated weights for policy 0, policy_version 676213 (0.0008) [2023-12-27 10:18:26,423][20381] Updated weights for policy 1, policy_version 675638 (0.0007) [2023-12-27 10:18:26,437][20347] Updated weights for policy 0, policy_version 676223 (0.0007) [2023-12-27 10:18:26,486][20347] Updated weights for policy 0, policy_version 676233 (0.0009) [2023-12-27 10:18:27,052][20381] Updated weights for policy 1, policy_version 675648 (0.0007) [2023-12-27 10:18:27,101][20381] Updated weights for policy 1, policy_version 675658 (0.0008) [2023-12-27 10:18:27,140][20347] Updated weights for policy 0, policy_version 676243 (0.0008) [2023-12-27 10:18:27,156][20381] Updated weights for policy 1, policy_version 675668 (0.0007) [2023-12-27 10:18:27,188][20347] Updated weights for policy 0, policy_version 676253 (0.0005) [2023-12-27 10:18:27,234][20347] Updated weights for policy 0, policy_version 676263 (0.0005) [2023-12-27 10:18:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 346144768. Throughput: 0: 10368.8, 1: 10627.4. Samples: 346127440. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:27,363][19170] Avg episode reward: [(0, '960.890'), (1, '1000.000')] [2023-12-27 10:18:27,728][20381] Updated weights for policy 1, policy_version 675678 (0.0008) [2023-12-27 10:18:27,775][20381] Updated weights for policy 1, policy_version 675688 (0.0007) [2023-12-27 10:18:27,830][20381] Updated weights for policy 1, policy_version 675698 (0.0007) [2023-12-27 10:18:27,914][20347] Updated weights for policy 0, policy_version 676273 (0.0007) [2023-12-27 10:18:27,963][20347] Updated weights for policy 0, policy_version 676283 (0.0008) [2023-12-27 10:18:28,017][20347] Updated weights for policy 0, policy_version 676293 (0.0007) [2023-12-27 10:18:28,065][20347] Updated weights for policy 0, policy_version 676303 (0.0007) [2023-12-27 10:18:28,485][20381] Updated weights for policy 1, policy_version 675708 (0.0009) [2023-12-27 10:18:28,545][20381] Updated weights for policy 1, policy_version 675718 (0.0010) [2023-12-27 10:18:28,607][20381] Updated weights for policy 1, policy_version 675728 (0.0010) [2023-12-27 10:18:28,785][20347] Updated weights for policy 0, policy_version 676313 (0.0007) [2023-12-27 10:18:28,834][20347] Updated weights for policy 0, policy_version 676323 (0.0008) [2023-12-27 10:18:28,886][20347] Updated weights for policy 0, policy_version 676333 (0.0007) [2023-12-27 10:18:29,247][20381] Updated weights for policy 1, policy_version 675738 (0.0011) [2023-12-27 10:18:29,306][20381] Updated weights for policy 1, policy_version 675748 (0.0011) [2023-12-27 10:18:29,365][20381] Updated weights for policy 1, policy_version 675758 (0.0008) [2023-12-27 10:18:29,423][20381] Updated weights for policy 1, policy_version 675768 (0.0005) [2023-12-27 10:18:29,644][20347] Updated weights for policy 0, policy_version 676343 (0.0007) [2023-12-27 10:18:29,700][20347] Updated weights for policy 0, policy_version 676353 (0.0005) [2023-12-27 10:18:29,762][20347] Updated weights for policy 0, policy_version 676363 (0.0006) [2023-12-27 10:18:30,122][20381] Updated weights for policy 1, policy_version 675778 (0.0008) [2023-12-27 10:18:30,181][20381] Updated weights for policy 1, policy_version 675788 (0.0009) [2023-12-27 10:18:30,229][20381] Updated weights for policy 1, policy_version 675798 (0.0006) [2023-12-27 10:18:30,344][20347] Updated weights for policy 0, policy_version 676373 (0.0006) [2023-12-27 10:18:30,400][20347] Updated weights for policy 0, policy_version 676383 (0.0009) [2023-12-27 10:18:30,455][20347] Updated weights for policy 0, policy_version 676393 (0.0006) [2023-12-27 10:18:30,872][20381] Updated weights for policy 1, policy_version 675808 (0.0006) [2023-12-27 10:18:30,935][20381] Updated weights for policy 1, policy_version 675818 (0.0005) [2023-12-27 10:18:30,985][20381] Updated weights for policy 1, policy_version 675828 (0.0005) [2023-12-27 10:18:31,097][20347] Updated weights for policy 0, policy_version 676403 (0.0006) [2023-12-27 10:18:31,160][20347] Updated weights for policy 0, policy_version 676413 (0.0008) [2023-12-27 10:18:31,211][20347] Updated weights for policy 0, policy_version 676423 (0.0007) [2023-12-27 10:18:31,573][20381] Updated weights for policy 1, policy_version 675838 (0.0007) [2023-12-27 10:18:31,629][20381] Updated weights for policy 1, policy_version 675848 (0.0008) [2023-12-27 10:18:31,687][20381] Updated weights for policy 1, policy_version 675858 (0.0007) [2023-12-27 10:18:31,895][20347] Updated weights for policy 0, policy_version 676433 (0.0007) [2023-12-27 10:18:31,951][20347] Updated weights for policy 0, policy_version 676443 (0.0008) [2023-12-27 10:18:32,010][20347] Updated weights for policy 0, policy_version 676453 (0.0008) [2023-12-27 10:18:32,067][20347] Updated weights for policy 0, policy_version 676463 (0.0008) [2023-12-27 10:18:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 346243072. Throughput: 0: 10305.2, 1: 10603.1. Samples: 346253596. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:32,363][19170] Avg episode reward: [(0, '941.330'), (1, '1000.000')] [2023-12-27 10:18:32,421][20381] Updated weights for policy 1, policy_version 675868 (0.0008) [2023-12-27 10:18:32,474][20381] Updated weights for policy 1, policy_version 675878 (0.0008) [2023-12-27 10:18:32,532][20381] Updated weights for policy 1, policy_version 675888 (0.0006) [2023-12-27 10:18:32,779][20347] Updated weights for policy 0, policy_version 676473 (0.0008) [2023-12-27 10:18:32,832][20347] Updated weights for policy 0, policy_version 676483 (0.0007) [2023-12-27 10:18:32,888][20347] Updated weights for policy 0, policy_version 676493 (0.0009) [2023-12-27 10:18:33,083][20381] Updated weights for policy 1, policy_version 675898 (0.0005) [2023-12-27 10:18:33,143][20381] Updated weights for policy 1, policy_version 675908 (0.0006) [2023-12-27 10:18:33,207][20381] Updated weights for policy 1, policy_version 675918 (0.0008) [2023-12-27 10:18:33,269][20381] Updated weights for policy 1, policy_version 675928 (0.0008) [2023-12-27 10:18:33,592][20347] Updated weights for policy 0, policy_version 676503 (0.0008) [2023-12-27 10:18:33,639][20347] Updated weights for policy 0, policy_version 676513 (0.0008) [2023-12-27 10:18:33,689][20347] Updated weights for policy 0, policy_version 676523 (0.0010) [2023-12-27 10:18:33,870][20381] Updated weights for policy 1, policy_version 675938 (0.0005) [2023-12-27 10:18:33,922][20381] Updated weights for policy 1, policy_version 675948 (0.0005) [2023-12-27 10:18:33,968][20381] Updated weights for policy 1, policy_version 675958 (0.0005) [2023-12-27 10:18:34,415][20347] Updated weights for policy 0, policy_version 676533 (0.0009) [2023-12-27 10:18:34,495][20347] Updated weights for policy 0, policy_version 676543 (0.0007) [2023-12-27 10:18:34,555][20347] Updated weights for policy 0, policy_version 676553 (0.0007) [2023-12-27 10:18:34,620][20381] Updated weights for policy 1, policy_version 675968 (0.0007) [2023-12-27 10:18:34,670][20381] Updated weights for policy 1, policy_version 675978 (0.0007) [2023-12-27 10:18:34,717][20381] Updated weights for policy 1, policy_version 675988 (0.0008) [2023-12-27 10:18:35,197][20347] Updated weights for policy 0, policy_version 676563 (0.0006) [2023-12-27 10:18:35,246][20347] Updated weights for policy 0, policy_version 676573 (0.0006) [2023-12-27 10:18:35,292][20347] Updated weights for policy 0, policy_version 676583 (0.0007) [2023-12-27 10:18:35,414][20381] Updated weights for policy 1, policy_version 675998 (0.0008) [2023-12-27 10:18:35,466][20381] Updated weights for policy 1, policy_version 676008 (0.0008) [2023-12-27 10:18:35,514][20381] Updated weights for policy 1, policy_version 676018 (0.0008) [2023-12-27 10:18:35,923][20347] Updated weights for policy 0, policy_version 676593 (0.0008) [2023-12-27 10:18:35,975][20347] Updated weights for policy 0, policy_version 676603 (0.0005) [2023-12-27 10:18:36,030][20347] Updated weights for policy 0, policy_version 676613 (0.0005) [2023-12-27 10:18:36,089][20347] Updated weights for policy 0, policy_version 676623 (0.0007) [2023-12-27 10:18:36,246][20381] Updated weights for policy 1, policy_version 676028 (0.0008) [2023-12-27 10:18:36,296][20381] Updated weights for policy 1, policy_version 676038 (0.0009) [2023-12-27 10:18:36,348][20381] Updated weights for policy 1, policy_version 676048 (0.0009) [2023-12-27 10:18:36,724][20347] Updated weights for policy 0, policy_version 676633 (0.0006) [2023-12-27 10:18:36,785][20347] Updated weights for policy 0, policy_version 676643 (0.0005) [2023-12-27 10:18:36,840][20347] Updated weights for policy 0, policy_version 676653 (0.0006) [2023-12-27 10:18:37,112][20381] Updated weights for policy 1, policy_version 676058 (0.0008) [2023-12-27 10:18:37,171][20381] Updated weights for policy 1, policy_version 676068 (0.0008) [2023-12-27 10:18:37,232][20381] Updated weights for policy 1, policy_version 676078 (0.0009) [2023-12-27 10:18:37,299][20381] Updated weights for policy 1, policy_version 676088 (0.0009) [2023-12-27 10:18:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 346349568. Throughput: 0: 10270.9, 1: 10573.7. Samples: 346316724. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:37,363][19170] Avg episode reward: [(0, '951.050'), (1, '980.560')] [2023-12-27 10:18:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000676088_173105152.pth... [2023-12-27 10:18:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000676656_173244416.pth... [2023-12-27 10:18:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000674872_172793856.pth [2023-12-27 10:18:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000675472_172941312.pth [2023-12-27 10:18:37,437][20347] Updated weights for policy 0, policy_version 676663 (0.0006) [2023-12-27 10:18:37,491][20347] Updated weights for policy 0, policy_version 676673 (0.0006) [2023-12-27 10:18:37,542][20347] Updated weights for policy 0, policy_version 676683 (0.0008) [2023-12-27 10:18:37,981][20381] Updated weights for policy 1, policy_version 676098 (0.0006) [2023-12-27 10:18:38,041][20381] Updated weights for policy 1, policy_version 676108 (0.0005) [2023-12-27 10:18:38,094][20381] Updated weights for policy 1, policy_version 676118 (0.0007) [2023-12-27 10:18:38,264][20347] Updated weights for policy 0, policy_version 676693 (0.0008) [2023-12-27 10:18:38,328][20347] Updated weights for policy 0, policy_version 676703 (0.0006) [2023-12-27 10:18:38,388][20347] Updated weights for policy 0, policy_version 676713 (0.0010) [2023-12-27 10:18:38,756][20381] Updated weights for policy 1, policy_version 676128 (0.0010) [2023-12-27 10:18:38,819][20381] Updated weights for policy 1, policy_version 676138 (0.0010) [2023-12-27 10:18:38,881][20381] Updated weights for policy 1, policy_version 676148 (0.0007) [2023-12-27 10:18:38,999][20347] Updated weights for policy 0, policy_version 676723 (0.0008) [2023-12-27 10:18:39,058][20347] Updated weights for policy 0, policy_version 676733 (0.0010) [2023-12-27 10:18:39,117][20347] Updated weights for policy 0, policy_version 676743 (0.0010) [2023-12-27 10:18:39,557][20381] Updated weights for policy 1, policy_version 676158 (0.0008) [2023-12-27 10:18:39,621][20381] Updated weights for policy 1, policy_version 676168 (0.0008) [2023-12-27 10:18:39,684][20381] Updated weights for policy 1, policy_version 676178 (0.0005) [2023-12-27 10:18:39,798][20347] Updated weights for policy 0, policy_version 676753 (0.0010) [2023-12-27 10:18:39,863][20347] Updated weights for policy 0, policy_version 676763 (0.0010) [2023-12-27 10:18:39,920][20347] Updated weights for policy 0, policy_version 676773 (0.0008) [2023-12-27 10:18:39,976][20347] Updated weights for policy 0, policy_version 676783 (0.0007) [2023-12-27 10:18:40,381][20381] Updated weights for policy 1, policy_version 676188 (0.0006) [2023-12-27 10:18:40,440][20381] Updated weights for policy 1, policy_version 676198 (0.0007) [2023-12-27 10:18:40,491][20381] Updated weights for policy 1, policy_version 676208 (0.0010) [2023-12-27 10:18:40,560][20347] Updated weights for policy 0, policy_version 676793 (0.0007) [2023-12-27 10:18:40,608][20347] Updated weights for policy 0, policy_version 676803 (0.0008) [2023-12-27 10:18:40,653][20347] Updated weights for policy 0, policy_version 676813 (0.0008) [2023-12-27 10:18:41,155][20381] Updated weights for policy 1, policy_version 676218 (0.0007) [2023-12-27 10:18:41,219][20381] Updated weights for policy 1, policy_version 676228 (0.0008) [2023-12-27 10:18:41,278][20381] Updated weights for policy 1, policy_version 676238 (0.0008) [2023-12-27 10:18:41,340][20381] Updated weights for policy 1, policy_version 676248 (0.0007) [2023-12-27 10:18:41,365][20347] Updated weights for policy 0, policy_version 676823 (0.0008) [2023-12-27 10:18:41,430][20347] Updated weights for policy 0, policy_version 676833 (0.0008) [2023-12-27 10:18:41,483][20347] Updated weights for policy 0, policy_version 676843 (0.0008) [2023-12-27 10:18:42,008][20381] Updated weights for policy 1, policy_version 676258 (0.0007) [2023-12-27 10:18:42,073][20381] Updated weights for policy 1, policy_version 676268 (0.0007) [2023-12-27 10:18:42,134][20381] Updated weights for policy 1, policy_version 676278 (0.0007) [2023-12-27 10:18:42,169][20347] Updated weights for policy 0, policy_version 676853 (0.0008) [2023-12-27 10:18:42,231][20347] Updated weights for policy 0, policy_version 676863 (0.0008) [2023-12-27 10:18:42,289][20347] Updated weights for policy 0, policy_version 676873 (0.0006) [2023-12-27 10:18:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 346456064. Throughput: 0: 10359.1, 1: 10493.2. Samples: 346441260. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:42,363][19170] Avg episode reward: [(0, '960.970'), (1, '980.560')] [2023-12-27 10:18:42,834][20381] Updated weights for policy 1, policy_version 676288 (0.0009) [2023-12-27 10:18:42,887][20381] Updated weights for policy 1, policy_version 676298 (0.0008) [2023-12-27 10:18:42,925][20347] Updated weights for policy 0, policy_version 676883 (0.0007) [2023-12-27 10:18:42,933][20381] Updated weights for policy 1, policy_version 676308 (0.0007) [2023-12-27 10:18:42,975][20347] Updated weights for policy 0, policy_version 676893 (0.0007) [2023-12-27 10:18:43,027][20347] Updated weights for policy 0, policy_version 676903 (0.0005) [2023-12-27 10:18:43,563][20381] Updated weights for policy 1, policy_version 676318 (0.0006) [2023-12-27 10:18:43,611][20381] Updated weights for policy 1, policy_version 676328 (0.0006) [2023-12-27 10:18:43,630][20347] Updated weights for policy 0, policy_version 676913 (0.0006) [2023-12-27 10:18:43,660][20381] Updated weights for policy 1, policy_version 676338 (0.0005) [2023-12-27 10:18:43,681][20347] Updated weights for policy 0, policy_version 676923 (0.0008) [2023-12-27 10:18:43,734][20347] Updated weights for policy 0, policy_version 676933 (0.0008) [2023-12-27 10:18:43,780][20347] Updated weights for policy 0, policy_version 676943 (0.0009) [2023-12-27 10:18:44,253][20381] Updated weights for policy 1, policy_version 676348 (0.0006) [2023-12-27 10:18:44,304][20381] Updated weights for policy 1, policy_version 676358 (0.0005) [2023-12-27 10:18:44,355][20381] Updated weights for policy 1, policy_version 676368 (0.0005) [2023-12-27 10:18:44,559][20347] Updated weights for policy 0, policy_version 676953 (0.0007) [2023-12-27 10:18:44,604][20347] Updated weights for policy 0, policy_version 676963 (0.0006) [2023-12-27 10:18:44,652][20347] Updated weights for policy 0, policy_version 676973 (0.0006) [2023-12-27 10:18:44,948][20381] Updated weights for policy 1, policy_version 676378 (0.0006) [2023-12-27 10:18:45,014][20381] Updated weights for policy 1, policy_version 676388 (0.0007) [2023-12-27 10:18:45,075][20381] Updated weights for policy 1, policy_version 676398 (0.0006) [2023-12-27 10:18:45,134][20381] Updated weights for policy 1, policy_version 676408 (0.0007) [2023-12-27 10:18:45,334][20347] Updated weights for policy 0, policy_version 676983 (0.0008) [2023-12-27 10:18:45,395][20347] Updated weights for policy 0, policy_version 676993 (0.0008) [2023-12-27 10:18:45,449][20347] Updated weights for policy 0, policy_version 677003 (0.0006) [2023-12-27 10:18:45,822][20381] Updated weights for policy 1, policy_version 676418 (0.0009) [2023-12-27 10:18:45,878][20381] Updated weights for policy 1, policy_version 676428 (0.0007) [2023-12-27 10:18:45,927][20381] Updated weights for policy 1, policy_version 676438 (0.0007) [2023-12-27 10:18:46,024][20347] Updated weights for policy 0, policy_version 677013 (0.0005) [2023-12-27 10:18:46,081][20347] Updated weights for policy 0, policy_version 677023 (0.0007) [2023-12-27 10:18:46,138][20347] Updated weights for policy 0, policy_version 677033 (0.0007) [2023-12-27 10:18:46,639][20381] Updated weights for policy 1, policy_version 676448 (0.0009) [2023-12-27 10:18:46,697][20381] Updated weights for policy 1, policy_version 676458 (0.0007) [2023-12-27 10:18:46,748][20347] Updated weights for policy 0, policy_version 677043 (0.0008) [2023-12-27 10:18:46,755][20381] Updated weights for policy 1, policy_version 676468 (0.0010) [2023-12-27 10:18:46,796][20347] Updated weights for policy 0, policy_version 677053 (0.0010) [2023-12-27 10:18:46,853][20347] Updated weights for policy 0, policy_version 677063 (0.0010) [2023-12-27 10:18:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 346554368. Throughput: 0: 10422.5, 1: 10481.7. Samples: 346567888. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:47,363][19170] Avg episode reward: [(0, '960.930'), (1, '1000.000')] [2023-12-27 10:18:47,379][20381] Updated weights for policy 1, policy_version 676478 (0.0009) [2023-12-27 10:18:47,427][20381] Updated weights for policy 1, policy_version 676488 (0.0005) [2023-12-27 10:18:47,442][20347] Updated weights for policy 0, policy_version 677073 (0.0010) [2023-12-27 10:18:47,479][20381] Updated weights for policy 1, policy_version 676498 (0.0007) [2023-12-27 10:18:47,495][20347] Updated weights for policy 0, policy_version 677083 (0.0010) [2023-12-27 10:18:47,546][20347] Updated weights for policy 0, policy_version 677093 (0.0010) [2023-12-27 10:18:47,598][20347] Updated weights for policy 0, policy_version 677103 (0.0010) [2023-12-27 10:18:48,066][20381] Updated weights for policy 1, policy_version 676508 (0.0006) [2023-12-27 10:18:48,114][20381] Updated weights for policy 1, policy_version 676518 (0.0005) [2023-12-27 10:18:48,163][20381] Updated weights for policy 1, policy_version 676528 (0.0005) [2023-12-27 10:18:48,248][20347] Updated weights for policy 0, policy_version 677113 (0.0006) [2023-12-27 10:18:48,296][20347] Updated weights for policy 0, policy_version 677123 (0.0005) [2023-12-27 10:18:48,364][20347] Updated weights for policy 0, policy_version 677133 (0.0009) [2023-12-27 10:18:48,723][20381] Updated weights for policy 1, policy_version 676538 (0.0006) [2023-12-27 10:18:48,777][20381] Updated weights for policy 1, policy_version 676548 (0.0008) [2023-12-27 10:18:48,842][20381] Updated weights for policy 1, policy_version 676558 (0.0007) [2023-12-27 10:18:48,907][20381] Updated weights for policy 1, policy_version 676568 (0.0005) [2023-12-27 10:18:49,051][20347] Updated weights for policy 0, policy_version 677143 (0.0010) [2023-12-27 10:18:49,114][20347] Updated weights for policy 0, policy_version 677153 (0.0007) [2023-12-27 10:18:49,166][20347] Updated weights for policy 0, policy_version 677163 (0.0008) [2023-12-27 10:18:49,566][20381] Updated weights for policy 1, policy_version 676578 (0.0007) [2023-12-27 10:18:49,635][20381] Updated weights for policy 1, policy_version 676588 (0.0006) [2023-12-27 10:18:49,692][20381] Updated weights for policy 1, policy_version 676598 (0.0006) [2023-12-27 10:18:49,851][20347] Updated weights for policy 0, policy_version 677173 (0.0008) [2023-12-27 10:18:49,915][20347] Updated weights for policy 0, policy_version 677183 (0.0007) [2023-12-27 10:18:49,984][20347] Updated weights for policy 0, policy_version 677193 (0.0008) [2023-12-27 10:18:50,351][20381] Updated weights for policy 1, policy_version 676608 (0.0008) [2023-12-27 10:18:50,416][20381] Updated weights for policy 1, policy_version 676618 (0.0008) [2023-12-27 10:18:50,470][20381] Updated weights for policy 1, policy_version 676628 (0.0007) [2023-12-27 10:18:50,673][20347] Updated weights for policy 0, policy_version 677203 (0.0008) [2023-12-27 10:18:50,735][20347] Updated weights for policy 0, policy_version 677213 (0.0011) [2023-12-27 10:18:50,794][20347] Updated weights for policy 0, policy_version 677223 (0.0011) [2023-12-27 10:18:51,144][20381] Updated weights for policy 1, policy_version 676638 (0.0007) [2023-12-27 10:18:51,205][20381] Updated weights for policy 1, policy_version 676648 (0.0006) [2023-12-27 10:18:51,267][20381] Updated weights for policy 1, policy_version 676658 (0.0006) [2023-12-27 10:18:51,471][20347] Updated weights for policy 0, policy_version 677233 (0.0010) [2023-12-27 10:18:51,535][20347] Updated weights for policy 0, policy_version 677243 (0.0010) [2023-12-27 10:18:51,597][20347] Updated weights for policy 0, policy_version 677253 (0.0005) [2023-12-27 10:18:51,656][20347] Updated weights for policy 0, policy_version 677263 (0.0010) [2023-12-27 10:18:51,872][20381] Updated weights for policy 1, policy_version 676668 (0.0007) [2023-12-27 10:18:51,934][20381] Updated weights for policy 1, policy_version 676678 (0.0005) [2023-12-27 10:18:51,991][20381] Updated weights for policy 1, policy_version 676688 (0.0005) [2023-12-27 10:18:52,322][20347] Updated weights for policy 0, policy_version 677273 (0.0011) [2023-12-27 10:18:52,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.5, 300 sec: 20771.6). Total num frames: 346660864. Throughput: 0: 10392.6, 1: 10490.0. Samples: 346633172. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:52,363][19170] Avg episode reward: [(0, '892.600'), (1, '1000.000')] [2023-12-27 10:18:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000676696_173260800.pth... [2023-12-27 10:18:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000675448_172941312.pth [2023-12-27 10:18:52,386][20347] Updated weights for policy 0, policy_version 677283 (0.0009) [2023-12-27 10:18:52,438][20347] Updated weights for policy 0, policy_version 677293 (0.0010) [2023-12-27 10:18:52,452][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000677296_173408256.pth... [2023-12-27 10:18:52,455][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000676080_173096960.pth [2023-12-27 10:18:52,640][20381] Updated weights for policy 1, policy_version 676698 (0.0007) [2023-12-27 10:18:52,689][20381] Updated weights for policy 1, policy_version 676708 (0.0011) [2023-12-27 10:18:52,738][20381] Updated weights for policy 1, policy_version 676718 (0.0010) [2023-12-27 10:18:52,786][20381] Updated weights for policy 1, policy_version 676728 (0.0010) [2023-12-27 10:18:53,130][20347] Updated weights for policy 0, policy_version 677303 (0.0010) [2023-12-27 10:18:53,187][20347] Updated weights for policy 0, policy_version 677313 (0.0010) [2023-12-27 10:18:53,246][20347] Updated weights for policy 0, policy_version 677323 (0.0010) [2023-12-27 10:18:53,490][20381] Updated weights for policy 1, policy_version 676738 (0.0010) [2023-12-27 10:18:53,544][20381] Updated weights for policy 1, policy_version 676748 (0.0010) [2023-12-27 10:18:53,608][20381] Updated weights for policy 1, policy_version 676758 (0.0006) [2023-12-27 10:18:53,887][20347] Updated weights for policy 0, policy_version 677333 (0.0008) [2023-12-27 10:18:53,947][20347] Updated weights for policy 0, policy_version 677343 (0.0008) [2023-12-27 10:18:54,007][20347] Updated weights for policy 0, policy_version 677353 (0.0007) [2023-12-27 10:18:54,297][20381] Updated weights for policy 1, policy_version 676768 (0.0008) [2023-12-27 10:18:54,362][20381] Updated weights for policy 1, policy_version 676778 (0.0007) [2023-12-27 10:18:54,422][20381] Updated weights for policy 1, policy_version 676788 (0.0008) [2023-12-27 10:18:54,663][20347] Updated weights for policy 0, policy_version 677363 (0.0006) [2023-12-27 10:18:54,709][20347] Updated weights for policy 0, policy_version 677373 (0.0005) [2023-12-27 10:18:54,761][20347] Updated weights for policy 0, policy_version 677383 (0.0008) [2023-12-27 10:18:55,046][20381] Updated weights for policy 1, policy_version 676798 (0.0008) [2023-12-27 10:18:55,094][20381] Updated weights for policy 1, policy_version 676808 (0.0007) [2023-12-27 10:18:55,144][20381] Updated weights for policy 1, policy_version 676818 (0.0007) [2023-12-27 10:18:55,379][20347] Updated weights for policy 0, policy_version 677393 (0.0009) [2023-12-27 10:18:55,442][20347] Updated weights for policy 0, policy_version 677403 (0.0009) [2023-12-27 10:18:55,497][20347] Updated weights for policy 0, policy_version 677413 (0.0010) [2023-12-27 10:18:55,556][20347] Updated weights for policy 0, policy_version 677423 (0.0006) [2023-12-27 10:18:55,886][20381] Updated weights for policy 1, policy_version 676828 (0.0009) [2023-12-27 10:18:55,935][20381] Updated weights for policy 1, policy_version 676838 (0.0009) [2023-12-27 10:18:55,987][20381] Updated weights for policy 1, policy_version 676848 (0.0008) [2023-12-27 10:18:56,121][20347] Updated weights for policy 0, policy_version 677433 (0.0007) [2023-12-27 10:18:56,179][20347] Updated weights for policy 0, policy_version 677443 (0.0005) [2023-12-27 10:18:56,234][20347] Updated weights for policy 0, policy_version 677453 (0.0007) [2023-12-27 10:18:56,681][20381] Updated weights for policy 1, policy_version 676858 (0.0008) [2023-12-27 10:18:56,743][20381] Updated weights for policy 1, policy_version 676868 (0.0008) [2023-12-27 10:18:56,792][20381] Updated weights for policy 1, policy_version 676878 (0.0007) [2023-12-27 10:18:56,839][20381] Updated weights for policy 1, policy_version 676888 (0.0007) [2023-12-27 10:18:56,894][20347] Updated weights for policy 0, policy_version 677463 (0.0008) [2023-12-27 10:18:56,941][20347] Updated weights for policy 0, policy_version 677473 (0.0007) [2023-12-27 10:18:56,991][20347] Updated weights for policy 0, policy_version 677483 (0.0007) [2023-12-27 10:18:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 346767360. Throughput: 0: 10427.1, 1: 10479.7. Samples: 346757524. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:18:57,363][19170] Avg episode reward: [(0, '879.270'), (1, '990.360')] [2023-12-27 10:18:57,521][20381] Updated weights for policy 1, policy_version 676898 (0.0008) [2023-12-27 10:18:57,570][20381] Updated weights for policy 1, policy_version 676908 (0.0008) [2023-12-27 10:18:57,574][20292] KL-divergence is very high: 118.6975 [2023-12-27 10:18:57,649][20381] Updated weights for policy 1, policy_version 676918 (0.0008) [2023-12-27 10:18:57,704][20347] Updated weights for policy 0, policy_version 677493 (0.0007) [2023-12-27 10:18:57,767][20347] Updated weights for policy 0, policy_version 677503 (0.0008) [2023-12-27 10:18:57,831][20347] Updated weights for policy 0, policy_version 677513 (0.0008) [2023-12-27 10:18:58,281][20381] Updated weights for policy 1, policy_version 676928 (0.0008) [2023-12-27 10:18:58,343][20381] Updated weights for policy 1, policy_version 676938 (0.0008) [2023-12-27 10:18:58,404][20381] Updated weights for policy 1, policy_version 676948 (0.0007) [2023-12-27 10:18:58,567][20347] Updated weights for policy 0, policy_version 677523 (0.0008) [2023-12-27 10:18:58,630][20347] Updated weights for policy 0, policy_version 677533 (0.0008) [2023-12-27 10:18:58,686][20347] Updated weights for policy 0, policy_version 677543 (0.0008) [2023-12-27 10:18:59,165][20381] Updated weights for policy 1, policy_version 676958 (0.0010) [2023-12-27 10:18:59,224][20381] Updated weights for policy 1, policy_version 676968 (0.0010) [2023-12-27 10:18:59,285][20381] Updated weights for policy 1, policy_version 676978 (0.0009) [2023-12-27 10:18:59,453][20347] Updated weights for policy 0, policy_version 677553 (0.0008) [2023-12-27 10:18:59,513][20347] Updated weights for policy 0, policy_version 677563 (0.0008) [2023-12-27 10:18:59,574][20347] Updated weights for policy 0, policy_version 677573 (0.0008) [2023-12-27 10:18:59,630][20347] Updated weights for policy 0, policy_version 677583 (0.0007) [2023-12-27 10:18:59,972][20381] Updated weights for policy 1, policy_version 676988 (0.0007) [2023-12-27 10:19:00,028][20381] Updated weights for policy 1, policy_version 676998 (0.0008) [2023-12-27 10:19:00,080][20381] Updated weights for policy 1, policy_version 677008 (0.0007) [2023-12-27 10:19:00,360][20347] Updated weights for policy 0, policy_version 677593 (0.0008) [2023-12-27 10:19:00,412][20347] Updated weights for policy 0, policy_version 677603 (0.0007) [2023-12-27 10:19:00,466][20347] Updated weights for policy 0, policy_version 677613 (0.0008) [2023-12-27 10:19:00,774][20381] Updated weights for policy 1, policy_version 677018 (0.0008) [2023-12-27 10:19:00,834][20381] Updated weights for policy 1, policy_version 677028 (0.0010) [2023-12-27 10:19:00,884][20381] Updated weights for policy 1, policy_version 677038 (0.0009) [2023-12-27 10:19:00,938][20381] Updated weights for policy 1, policy_version 677048 (0.0009) [2023-12-27 10:19:01,158][20347] Updated weights for policy 0, policy_version 677623 (0.0007) [2023-12-27 10:19:01,216][20347] Updated weights for policy 0, policy_version 677633 (0.0007) [2023-12-27 10:19:01,278][20347] Updated weights for policy 0, policy_version 677643 (0.0007) [2023-12-27 10:19:01,577][20381] Updated weights for policy 1, policy_version 677058 (0.0011) [2023-12-27 10:19:01,630][20381] Updated weights for policy 1, policy_version 677068 (0.0011) [2023-12-27 10:19:01,682][20381] Updated weights for policy 1, policy_version 677078 (0.0007) [2023-12-27 10:19:01,945][20347] Updated weights for policy 0, policy_version 677653 (0.0007) [2023-12-27 10:19:02,009][20347] Updated weights for policy 0, policy_version 677663 (0.0007) [2023-12-27 10:19:02,064][20347] Updated weights for policy 0, policy_version 677673 (0.0007) [2023-12-27 10:19:02,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 346865664. Throughput: 0: 10344.8, 1: 10462.3. Samples: 346877244. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:02,363][19170] Avg episode reward: [(0, '918.330'), (1, '990.360')] [2023-12-27 10:19:02,391][20381] Updated weights for policy 1, policy_version 677088 (0.0010) [2023-12-27 10:19:02,443][20381] Updated weights for policy 1, policy_version 677098 (0.0010) [2023-12-27 10:19:02,498][20381] Updated weights for policy 1, policy_version 677108 (0.0011) [2023-12-27 10:19:02,749][20347] Updated weights for policy 0, policy_version 677683 (0.0007) [2023-12-27 10:19:02,802][20347] Updated weights for policy 0, policy_version 677693 (0.0007) [2023-12-27 10:19:02,856][20347] Updated weights for policy 0, policy_version 677703 (0.0007) [2023-12-27 10:19:03,182][20381] Updated weights for policy 1, policy_version 677118 (0.0011) [2023-12-27 10:19:03,240][20381] Updated weights for policy 1, policy_version 677128 (0.0011) [2023-12-27 10:19:03,305][20381] Updated weights for policy 1, policy_version 677138 (0.0011) [2023-12-27 10:19:03,546][20347] Updated weights for policy 0, policy_version 677713 (0.0008) [2023-12-27 10:19:03,611][20347] Updated weights for policy 0, policy_version 677723 (0.0007) [2023-12-27 10:19:03,666][20347] Updated weights for policy 0, policy_version 677733 (0.0005) [2023-12-27 10:19:03,719][20347] Updated weights for policy 0, policy_version 677743 (0.0005) [2023-12-27 10:19:03,991][20381] Updated weights for policy 1, policy_version 677148 (0.0011) [2023-12-27 10:19:04,054][20381] Updated weights for policy 1, policy_version 677158 (0.0008) [2023-12-27 10:19:04,106][20381] Updated weights for policy 1, policy_version 677168 (0.0005) [2023-12-27 10:19:04,309][20347] Updated weights for policy 0, policy_version 677753 (0.0007) [2023-12-27 10:19:04,371][20347] Updated weights for policy 0, policy_version 677763 (0.0007) [2023-12-27 10:19:04,431][20347] Updated weights for policy 0, policy_version 677773 (0.0008) [2023-12-27 10:19:04,743][20381] Updated weights for policy 1, policy_version 677178 (0.0006) [2023-12-27 10:19:04,799][20381] Updated weights for policy 1, policy_version 677188 (0.0011) [2023-12-27 10:19:04,855][20381] Updated weights for policy 1, policy_version 677198 (0.0011) [2023-12-27 10:19:04,908][20381] Updated weights for policy 1, policy_version 677208 (0.0011) [2023-12-27 10:19:05,135][20347] Updated weights for policy 0, policy_version 677783 (0.0007) [2023-12-27 10:19:05,189][20347] Updated weights for policy 0, policy_version 677793 (0.0008) [2023-12-27 10:19:05,248][20347] Updated weights for policy 0, policy_version 677803 (0.0006) [2023-12-27 10:19:05,551][20381] Updated weights for policy 1, policy_version 677218 (0.0011) [2023-12-27 10:19:05,609][20381] Updated weights for policy 1, policy_version 677228 (0.0010) [2023-12-27 10:19:05,668][20381] Updated weights for policy 1, policy_version 677238 (0.0006) [2023-12-27 10:19:05,887][20347] Updated weights for policy 0, policy_version 677813 (0.0006) [2023-12-27 10:19:05,940][20347] Updated weights for policy 0, policy_version 677823 (0.0007) [2023-12-27 10:19:05,998][20347] Updated weights for policy 0, policy_version 677833 (0.0007) [2023-12-27 10:19:06,414][20381] Updated weights for policy 1, policy_version 677248 (0.0011) [2023-12-27 10:19:06,473][20381] Updated weights for policy 1, policy_version 677258 (0.0011) [2023-12-27 10:19:06,526][20381] Updated weights for policy 1, policy_version 677268 (0.0009) [2023-12-27 10:19:06,711][20347] Updated weights for policy 0, policy_version 677843 (0.0007) [2023-12-27 10:19:06,764][20347] Updated weights for policy 0, policy_version 677853 (0.0008) [2023-12-27 10:19:06,808][20347] Updated weights for policy 0, policy_version 677863 (0.0007) [2023-12-27 10:19:07,130][20381] Updated weights for policy 1, policy_version 677278 (0.0008) [2023-12-27 10:19:07,183][20381] Updated weights for policy 1, policy_version 677288 (0.0005) [2023-12-27 10:19:07,235][20381] Updated weights for policy 1, policy_version 677298 (0.0005) [2023-12-27 10:19:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 346972160. Throughput: 0: 10322.3, 1: 10465.4. Samples: 346939084. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:07,363][19170] Avg episode reward: [(0, '970.720'), (1, '990.360')] [2023-12-27 10:19:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000677304_173416448.pth... [2023-12-27 10:19:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000677872_173555712.pth... [2023-12-27 10:19:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000676088_173105152.pth [2023-12-27 10:19:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000676656_173244416.pth [2023-12-27 10:19:07,558][20347] Updated weights for policy 0, policy_version 677873 (0.0008) [2023-12-27 10:19:07,612][20347] Updated weights for policy 0, policy_version 677883 (0.0008) [2023-12-27 10:19:07,664][20347] Updated weights for policy 0, policy_version 677893 (0.0008) [2023-12-27 10:19:07,712][20347] Updated weights for policy 0, policy_version 677903 (0.0007) [2023-12-27 10:19:07,782][20381] Updated weights for policy 1, policy_version 677308 (0.0006) [2023-12-27 10:19:07,825][20381] Updated weights for policy 1, policy_version 677318 (0.0008) [2023-12-27 10:19:07,882][20381] Updated weights for policy 1, policy_version 677328 (0.0008) [2023-12-27 10:19:08,402][20347] Updated weights for policy 0, policy_version 677913 (0.0007) [2023-12-27 10:19:08,458][20347] Updated weights for policy 0, policy_version 677923 (0.0008) [2023-12-27 10:19:08,478][20381] Updated weights for policy 1, policy_version 677338 (0.0007) [2023-12-27 10:19:08,514][20347] Updated weights for policy 0, policy_version 677933 (0.0007) [2023-12-27 10:19:08,528][20381] Updated weights for policy 1, policy_version 677348 (0.0007) [2023-12-27 10:19:08,595][20381] Updated weights for policy 1, policy_version 677358 (0.0008) [2023-12-27 10:19:08,647][20381] Updated weights for policy 1, policy_version 677368 (0.0008) [2023-12-27 10:19:09,252][20347] Updated weights for policy 0, policy_version 677943 (0.0007) [2023-12-27 10:19:09,309][20347] Updated weights for policy 0, policy_version 677953 (0.0008) [2023-12-27 10:19:09,320][20381] Updated weights for policy 1, policy_version 677378 (0.0006) [2023-12-27 10:19:09,373][20347] Updated weights for policy 0, policy_version 677963 (0.0007) [2023-12-27 10:19:09,383][20381] Updated weights for policy 1, policy_version 677388 (0.0007) [2023-12-27 10:19:09,447][20381] Updated weights for policy 1, policy_version 677398 (0.0008) [2023-12-27 10:19:10,093][20381] Updated weights for policy 1, policy_version 677408 (0.0007) [2023-12-27 10:19:10,110][20347] Updated weights for policy 0, policy_version 677973 (0.0007) [2023-12-27 10:19:10,151][20381] Updated weights for policy 1, policy_version 677418 (0.0005) [2023-12-27 10:19:10,168][20347] Updated weights for policy 0, policy_version 677983 (0.0007) [2023-12-27 10:19:10,209][20381] Updated weights for policy 1, policy_version 677428 (0.0006) [2023-12-27 10:19:10,228][20347] Updated weights for policy 0, policy_version 677993 (0.0007) [2023-12-27 10:19:10,837][20381] Updated weights for policy 1, policy_version 677438 (0.0008) [2023-12-27 10:19:10,885][20381] Updated weights for policy 1, policy_version 677448 (0.0007) [2023-12-27 10:19:10,931][20347] Updated weights for policy 0, policy_version 678003 (0.0007) [2023-12-27 10:19:10,933][20381] Updated weights for policy 1, policy_version 677458 (0.0007) [2023-12-27 10:19:10,983][20347] Updated weights for policy 0, policy_version 678013 (0.0006) [2023-12-27 10:19:11,043][20347] Updated weights for policy 0, policy_version 678023 (0.0007) [2023-12-27 10:19:11,687][20381] Updated weights for policy 1, policy_version 677468 (0.0007) [2023-12-27 10:19:11,747][20381] Updated weights for policy 1, policy_version 677478 (0.0007) [2023-12-27 10:19:11,749][20347] Updated weights for policy 0, policy_version 678033 (0.0008) [2023-12-27 10:19:11,807][20347] Updated weights for policy 0, policy_version 678043 (0.0006) [2023-12-27 10:19:11,809][20381] Updated weights for policy 1, policy_version 677488 (0.0007) [2023-12-27 10:19:11,865][20347] Updated weights for policy 0, policy_version 678053 (0.0006) [2023-12-27 10:19:11,930][20347] Updated weights for policy 0, policy_version 678063 (0.0007) [2023-12-27 10:19:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 347070464. Throughput: 0: 10305.9, 1: 10477.3. Samples: 347062688. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:12,363][19170] Avg episode reward: [(0, '980.500'), (1, '991.030')] [2023-12-27 10:19:12,496][20381] Updated weights for policy 1, policy_version 677498 (0.0007) [2023-12-27 10:19:12,548][20381] Updated weights for policy 1, policy_version 677508 (0.0009) [2023-12-27 10:19:12,590][20347] Updated weights for policy 0, policy_version 678073 (0.0005) [2023-12-27 10:19:12,603][20381] Updated weights for policy 1, policy_version 677518 (0.0005) [2023-12-27 10:19:12,651][20347] Updated weights for policy 0, policy_version 678083 (0.0006) [2023-12-27 10:19:12,659][20381] Updated weights for policy 1, policy_version 677528 (0.0006) [2023-12-27 10:19:12,713][20347] Updated weights for policy 0, policy_version 678093 (0.0006) [2023-12-27 10:19:13,305][20381] Updated weights for policy 1, policy_version 677538 (0.0007) [2023-12-27 10:19:13,350][20347] Updated weights for policy 0, policy_version 678103 (0.0006) [2023-12-27 10:19:13,362][20381] Updated weights for policy 1, policy_version 677548 (0.0007) [2023-12-27 10:19:13,406][20347] Updated weights for policy 0, policy_version 678113 (0.0006) [2023-12-27 10:19:13,415][20381] Updated weights for policy 1, policy_version 677558 (0.0006) [2023-12-27 10:19:13,462][20347] Updated weights for policy 0, policy_version 678123 (0.0007) [2023-12-27 10:19:14,058][20347] Updated weights for policy 0, policy_version 678133 (0.0007) [2023-12-27 10:19:14,071][20381] Updated weights for policy 1, policy_version 677568 (0.0005) [2023-12-27 10:19:14,122][20347] Updated weights for policy 0, policy_version 678143 (0.0006) [2023-12-27 10:19:14,140][20381] Updated weights for policy 1, policy_version 677578 (0.0007) [2023-12-27 10:19:14,179][20347] Updated weights for policy 0, policy_version 678153 (0.0005) [2023-12-27 10:19:14,195][20381] Updated weights for policy 1, policy_version 677588 (0.0007) [2023-12-27 10:19:14,849][20347] Updated weights for policy 0, policy_version 678163 (0.0008) [2023-12-27 10:19:14,883][20381] Updated weights for policy 1, policy_version 677598 (0.0009) [2023-12-27 10:19:14,896][20347] Updated weights for policy 0, policy_version 678173 (0.0008) [2023-12-27 10:19:14,941][20381] Updated weights for policy 1, policy_version 677608 (0.0011) [2023-12-27 10:19:14,953][20347] Updated weights for policy 0, policy_version 678183 (0.0006) [2023-12-27 10:19:14,999][20381] Updated weights for policy 1, policy_version 677618 (0.0011) [2023-12-27 10:19:15,589][20347] Updated weights for policy 0, policy_version 678193 (0.0006) [2023-12-27 10:19:15,640][20347] Updated weights for policy 0, policy_version 678203 (0.0005) [2023-12-27 10:19:15,672][20381] Updated weights for policy 1, policy_version 677628 (0.0009) [2023-12-27 10:19:15,692][20347] Updated weights for policy 0, policy_version 678213 (0.0005) [2023-12-27 10:19:15,729][20381] Updated weights for policy 1, policy_version 677638 (0.0008) [2023-12-27 10:19:15,742][20347] Updated weights for policy 0, policy_version 678223 (0.0006) [2023-12-27 10:19:15,791][20381] Updated weights for policy 1, policy_version 677648 (0.0009) [2023-12-27 10:19:16,403][20347] Updated weights for policy 0, policy_version 678233 (0.0008) [2023-12-27 10:19:16,435][20381] Updated weights for policy 1, policy_version 677658 (0.0007) [2023-12-27 10:19:16,454][20347] Updated weights for policy 0, policy_version 678243 (0.0007) [2023-12-27 10:19:16,481][20381] Updated weights for policy 1, policy_version 677668 (0.0005) [2023-12-27 10:19:16,487][20292] KL-divergence is very high: 269.8696 [2023-12-27 10:19:16,501][20347] Updated weights for policy 0, policy_version 678253 (0.0005) [2023-12-27 10:19:16,534][20381] Updated weights for policy 1, policy_version 677678 (0.0006) [2023-12-27 10:19:16,597][20381] Updated weights for policy 1, policy_version 677688 (0.0008) [2023-12-27 10:19:17,157][20347] Updated weights for policy 0, policy_version 678263 (0.0007) [2023-12-27 10:19:17,212][20347] Updated weights for policy 0, policy_version 678273 (0.0006) [2023-12-27 10:19:17,243][20381] Updated weights for policy 1, policy_version 677698 (0.0006) [2023-12-27 10:19:17,264][20347] Updated weights for policy 0, policy_version 678283 (0.0006) [2023-12-27 10:19:17,295][20381] Updated weights for policy 1, policy_version 677708 (0.0006) [2023-12-27 10:19:17,354][20381] Updated weights for policy 1, policy_version 677718 (0.0009) [2023-12-27 10:19:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 347185152. Throughput: 0: 10346.4, 1: 10426.1. Samples: 347188356. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:17,363][19170] Avg episode reward: [(0, '990.250'), (1, '971.930')] [2023-12-27 10:19:17,844][20347] Updated weights for policy 0, policy_version 678293 (0.0006) [2023-12-27 10:19:17,902][20347] Updated weights for policy 0, policy_version 678303 (0.0005) [2023-12-27 10:19:17,953][20347] Updated weights for policy 0, policy_version 678313 (0.0005) [2023-12-27 10:19:18,069][20381] Updated weights for policy 1, policy_version 677728 (0.0008) [2023-12-27 10:19:18,114][20381] Updated weights for policy 1, policy_version 677738 (0.0007) [2023-12-27 10:19:18,162][20381] Updated weights for policy 1, policy_version 677748 (0.0008) [2023-12-27 10:19:18,597][20347] Updated weights for policy 0, policy_version 678323 (0.0007) [2023-12-27 10:19:18,655][20347] Updated weights for policy 0, policy_version 678333 (0.0005) [2023-12-27 10:19:18,706][20347] Updated weights for policy 0, policy_version 678343 (0.0007) [2023-12-27 10:19:18,898][20381] Updated weights for policy 1, policy_version 677758 (0.0009) [2023-12-27 10:19:18,949][20381] Updated weights for policy 1, policy_version 677768 (0.0008) [2023-12-27 10:19:18,993][20381] Updated weights for policy 1, policy_version 677778 (0.0007) [2023-12-27 10:19:19,346][20347] Updated weights for policy 0, policy_version 678353 (0.0007) [2023-12-27 10:19:19,404][20347] Updated weights for policy 0, policy_version 678363 (0.0007) [2023-12-27 10:19:19,455][20347] Updated weights for policy 0, policy_version 678373 (0.0007) [2023-12-27 10:19:19,513][20347] Updated weights for policy 0, policy_version 678383 (0.0007) [2023-12-27 10:19:19,736][20381] Updated weights for policy 1, policy_version 677788 (0.0008) [2023-12-27 10:19:19,787][20381] Updated weights for policy 1, policy_version 677798 (0.0008) [2023-12-27 10:19:19,847][20381] Updated weights for policy 1, policy_version 677808 (0.0007) [2023-12-27 10:19:20,174][20347] Updated weights for policy 0, policy_version 678393 (0.0007) [2023-12-27 10:19:20,237][20347] Updated weights for policy 0, policy_version 678403 (0.0008) [2023-12-27 10:19:20,297][20347] Updated weights for policy 0, policy_version 678413 (0.0008) [2023-12-27 10:19:20,556][20381] Updated weights for policy 1, policy_version 677818 (0.0008) [2023-12-27 10:19:20,617][20381] Updated weights for policy 1, policy_version 677828 (0.0007) [2023-12-27 10:19:20,681][20381] Updated weights for policy 1, policy_version 677838 (0.0007) [2023-12-27 10:19:20,744][20381] Updated weights for policy 1, policy_version 677848 (0.0007) [2023-12-27 10:19:20,968][20347] Updated weights for policy 0, policy_version 678423 (0.0006) [2023-12-27 10:19:21,023][20347] Updated weights for policy 0, policy_version 678433 (0.0008) [2023-12-27 10:19:21,087][20347] Updated weights for policy 0, policy_version 678443 (0.0008) [2023-12-27 10:19:21,422][20381] Updated weights for policy 1, policy_version 677858 (0.0008) [2023-12-27 10:19:21,493][20381] Updated weights for policy 1, policy_version 677868 (0.0008) [2023-12-27 10:19:21,554][20381] Updated weights for policy 1, policy_version 677878 (0.0008) [2023-12-27 10:19:21,763][20347] Updated weights for policy 0, policy_version 678453 (0.0008) [2023-12-27 10:19:21,818][20347] Updated weights for policy 0, policy_version 678463 (0.0008) [2023-12-27 10:19:21,880][20347] Updated weights for policy 0, policy_version 678473 (0.0008) [2023-12-27 10:19:22,297][20381] Updated weights for policy 1, policy_version 677888 (0.0008) [2023-12-27 10:19:22,345][20381] Updated weights for policy 1, policy_version 677898 (0.0009) [2023-12-27 10:19:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 347275264. Throughput: 0: 10393.3, 1: 10333.8. Samples: 347249444. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:22,363][19170] Avg episode reward: [(0, '951.120'), (1, '971.250')] [2023-12-27 10:19:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000678480_173711360.pth... [2023-12-27 10:19:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000677296_173408256.pth [2023-12-27 10:19:22,410][20381] Updated weights for policy 1, policy_version 677908 (0.0011) [2023-12-27 10:19:22,431][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000677912_173572096.pth... [2023-12-27 10:19:22,434][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000676696_173260800.pth [2023-12-27 10:19:22,624][20347] Updated weights for policy 0, policy_version 678483 (0.0008) [2023-12-27 10:19:22,683][20347] Updated weights for policy 0, policy_version 678493 (0.0008) [2023-12-27 10:19:22,739][20347] Updated weights for policy 0, policy_version 678503 (0.0007) [2023-12-27 10:19:23,074][20381] Updated weights for policy 1, policy_version 677918 (0.0008) [2023-12-27 10:19:23,136][20381] Updated weights for policy 1, policy_version 677928 (0.0011) [2023-12-27 10:19:23,190][20381] Updated weights for policy 1, policy_version 677938 (0.0010) [2023-12-27 10:19:23,450][20347] Updated weights for policy 0, policy_version 678513 (0.0008) [2023-12-27 10:19:23,498][20347] Updated weights for policy 0, policy_version 678523 (0.0007) [2023-12-27 10:19:23,547][20347] Updated weights for policy 0, policy_version 678533 (0.0007) [2023-12-27 10:19:23,605][20347] Updated weights for policy 0, policy_version 678543 (0.0007) [2023-12-27 10:19:23,857][20381] Updated weights for policy 1, policy_version 677948 (0.0009) [2023-12-27 10:19:23,907][20381] Updated weights for policy 1, policy_version 677958 (0.0006) [2023-12-27 10:19:23,965][20381] Updated weights for policy 1, policy_version 677968 (0.0005) [2023-12-27 10:19:24,321][20347] Updated weights for policy 0, policy_version 678553 (0.0008) [2023-12-27 10:19:24,379][20347] Updated weights for policy 0, policy_version 678563 (0.0008) [2023-12-27 10:19:24,433][20347] Updated weights for policy 0, policy_version 678573 (0.0006) [2023-12-27 10:19:24,590][20381] Updated weights for policy 1, policy_version 677978 (0.0005) [2023-12-27 10:19:24,645][20381] Updated weights for policy 1, policy_version 677988 (0.0005) [2023-12-27 10:19:24,707][20381] Updated weights for policy 1, policy_version 677998 (0.0005) [2023-12-27 10:19:24,768][20381] Updated weights for policy 1, policy_version 678008 (0.0005) [2023-12-27 10:19:25,146][20347] Updated weights for policy 0, policy_version 678583 (0.0007) [2023-12-27 10:19:25,206][20347] Updated weights for policy 0, policy_version 678593 (0.0007) [2023-12-27 10:19:25,262][20347] Updated weights for policy 0, policy_version 678603 (0.0007) [2023-12-27 10:19:25,379][20381] Updated weights for policy 1, policy_version 678018 (0.0011) [2023-12-27 10:19:25,433][20381] Updated weights for policy 1, policy_version 678028 (0.0009) [2023-12-27 10:19:25,490][20381] Updated weights for policy 1, policy_version 678038 (0.0006) [2023-12-27 10:19:25,914][20347] Updated weights for policy 0, policy_version 678613 (0.0007) [2023-12-27 10:19:25,963][20347] Updated weights for policy 0, policy_version 678623 (0.0005) [2023-12-27 10:19:26,024][20347] Updated weights for policy 0, policy_version 678633 (0.0006) [2023-12-27 10:19:26,083][20381] Updated weights for policy 1, policy_version 678048 (0.0007) [2023-12-27 10:19:26,139][20381] Updated weights for policy 1, policy_version 678058 (0.0008) [2023-12-27 10:19:26,198][20381] Updated weights for policy 1, policy_version 678068 (0.0008) [2023-12-27 10:19:26,689][20347] Updated weights for policy 0, policy_version 678643 (0.0006) [2023-12-27 10:19:26,745][20347] Updated weights for policy 0, policy_version 678653 (0.0007) [2023-12-27 10:19:26,796][20347] Updated weights for policy 0, policy_version 678663 (0.0007) [2023-12-27 10:19:26,871][20381] Updated weights for policy 1, policy_version 678078 (0.0008) [2023-12-27 10:19:26,929][20381] Updated weights for policy 1, policy_version 678088 (0.0008) [2023-12-27 10:19:26,986][20381] Updated weights for policy 1, policy_version 678098 (0.0007) [2023-12-27 10:19:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 347381760. Throughput: 0: 10277.5, 1: 10398.5. Samples: 347371680. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:27,363][19170] Avg episode reward: [(0, '951.100'), (1, '980.650')] [2023-12-27 10:19:27,483][20347] Updated weights for policy 0, policy_version 678673 (0.0007) [2023-12-27 10:19:27,538][20347] Updated weights for policy 0, policy_version 678683 (0.0007) [2023-12-27 10:19:27,586][20347] Updated weights for policy 0, policy_version 678693 (0.0007) [2023-12-27 10:19:27,635][20347] Updated weights for policy 0, policy_version 678703 (0.0007) [2023-12-27 10:19:27,656][20381] Updated weights for policy 1, policy_version 678108 (0.0008) [2023-12-27 10:19:27,705][20381] Updated weights for policy 1, policy_version 678118 (0.0008) [2023-12-27 10:19:27,758][20381] Updated weights for policy 1, policy_version 678128 (0.0008) [2023-12-27 10:19:28,328][20347] Updated weights for policy 0, policy_version 678713 (0.0008) [2023-12-27 10:19:28,388][20347] Updated weights for policy 0, policy_version 678723 (0.0007) [2023-12-27 10:19:28,448][20347] Updated weights for policy 0, policy_version 678733 (0.0008) [2023-12-27 10:19:28,485][20381] Updated weights for policy 1, policy_version 678138 (0.0008) [2023-12-27 10:19:28,542][20381] Updated weights for policy 1, policy_version 678148 (0.0008) [2023-12-27 10:19:28,597][20381] Updated weights for policy 1, policy_version 678158 (0.0008) [2023-12-27 10:19:28,657][20381] Updated weights for policy 1, policy_version 678168 (0.0008) [2023-12-27 10:19:29,109][20347] Updated weights for policy 0, policy_version 678743 (0.0006) [2023-12-27 10:19:29,165][20347] Updated weights for policy 0, policy_version 678753 (0.0005) [2023-12-27 10:19:29,221][20347] Updated weights for policy 0, policy_version 678763 (0.0006) [2023-12-27 10:19:29,361][20381] Updated weights for policy 1, policy_version 678178 (0.0008) [2023-12-27 10:19:29,417][20381] Updated weights for policy 1, policy_version 678188 (0.0007) [2023-12-27 10:19:29,471][20381] Updated weights for policy 1, policy_version 678198 (0.0007) [2023-12-27 10:19:29,850][20347] Updated weights for policy 0, policy_version 678773 (0.0009) [2023-12-27 10:19:29,909][20347] Updated weights for policy 0, policy_version 678783 (0.0010) [2023-12-27 10:19:29,969][20347] Updated weights for policy 0, policy_version 678793 (0.0009) [2023-12-27 10:19:30,156][20381] Updated weights for policy 1, policy_version 678208 (0.0005) [2023-12-27 10:19:30,203][20381] Updated weights for policy 1, policy_version 678218 (0.0005) [2023-12-27 10:19:30,258][20381] Updated weights for policy 1, policy_version 678228 (0.0006) [2023-12-27 10:19:30,640][20347] Updated weights for policy 0, policy_version 678803 (0.0007) [2023-12-27 10:19:30,699][20347] Updated weights for policy 0, policy_version 678813 (0.0005) [2023-12-27 10:19:30,765][20347] Updated weights for policy 0, policy_version 678823 (0.0005) [2023-12-27 10:19:30,920][20381] Updated weights for policy 1, policy_version 678238 (0.0005) [2023-12-27 10:19:30,973][20381] Updated weights for policy 1, policy_version 678248 (0.0007) [2023-12-27 10:19:31,032][20381] Updated weights for policy 1, policy_version 678258 (0.0007) [2023-12-27 10:19:31,374][20347] Updated weights for policy 0, policy_version 678833 (0.0005) [2023-12-27 10:19:31,430][20347] Updated weights for policy 0, policy_version 678843 (0.0008) [2023-12-27 10:19:31,489][20347] Updated weights for policy 0, policy_version 678853 (0.0008) [2023-12-27 10:19:31,543][20347] Updated weights for policy 0, policy_version 678863 (0.0009) [2023-12-27 10:19:31,725][20381] Updated weights for policy 1, policy_version 678268 (0.0008) [2023-12-27 10:19:31,785][20381] Updated weights for policy 1, policy_version 678278 (0.0007) [2023-12-27 10:19:31,841][20381] Updated weights for policy 1, policy_version 678288 (0.0008) [2023-12-27 10:19:32,224][20347] Updated weights for policy 0, policy_version 678873 (0.0006) [2023-12-27 10:19:32,283][20347] Updated weights for policy 0, policy_version 678883 (0.0005) [2023-12-27 10:19:32,342][20347] Updated weights for policy 0, policy_version 678893 (0.0007) [2023-12-27 10:19:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 347488256. Throughput: 0: 10256.0, 1: 10336.7. Samples: 347494556. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:32,363][19170] Avg episode reward: [(0, '902.470'), (1, '990.300')] [2023-12-27 10:19:32,560][20381] Updated weights for policy 1, policy_version 678298 (0.0008) [2023-12-27 10:19:32,614][20381] Updated weights for policy 1, policy_version 678308 (0.0008) [2023-12-27 10:19:32,671][20381] Updated weights for policy 1, policy_version 678318 (0.0007) [2023-12-27 10:19:32,724][20381] Updated weights for policy 1, policy_version 678328 (0.0007) [2023-12-27 10:19:32,997][20347] Updated weights for policy 0, policy_version 678903 (0.0006) [2023-12-27 10:19:33,048][20347] Updated weights for policy 0, policy_version 678913 (0.0005) [2023-12-27 10:19:33,094][20347] Updated weights for policy 0, policy_version 678923 (0.0005) [2023-12-27 10:19:33,331][20381] Updated weights for policy 1, policy_version 678338 (0.0005) [2023-12-27 10:19:33,385][20381] Updated weights for policy 1, policy_version 678348 (0.0010) [2023-12-27 10:19:33,432][20381] Updated weights for policy 1, policy_version 678358 (0.0010) [2023-12-27 10:19:33,683][20347] Updated weights for policy 0, policy_version 678933 (0.0005) [2023-12-27 10:19:33,732][20347] Updated weights for policy 0, policy_version 678943 (0.0005) [2023-12-27 10:19:33,783][20347] Updated weights for policy 0, policy_version 678953 (0.0005) [2023-12-27 10:19:34,077][20381] Updated weights for policy 1, policy_version 678368 (0.0008) [2023-12-27 10:19:34,127][20381] Updated weights for policy 1, policy_version 678378 (0.0007) [2023-12-27 10:19:34,184][20381] Updated weights for policy 1, policy_version 678388 (0.0007) [2023-12-27 10:19:34,427][20347] Updated weights for policy 0, policy_version 678963 (0.0006) [2023-12-27 10:19:34,487][20347] Updated weights for policy 0, policy_version 678973 (0.0009) [2023-12-27 10:19:34,540][20347] Updated weights for policy 0, policy_version 678983 (0.0009) [2023-12-27 10:19:34,823][20381] Updated weights for policy 1, policy_version 678398 (0.0007) [2023-12-27 10:19:34,878][20381] Updated weights for policy 1, policy_version 678408 (0.0007) [2023-12-27 10:19:34,934][20381] Updated weights for policy 1, policy_version 678418 (0.0010) [2023-12-27 10:19:35,207][20347] Updated weights for policy 0, policy_version 678993 (0.0008) [2023-12-27 10:19:35,258][20347] Updated weights for policy 0, policy_version 679003 (0.0006) [2023-12-27 10:19:35,307][20347] Updated weights for policy 0, policy_version 679013 (0.0008) [2023-12-27 10:19:35,363][20347] Updated weights for policy 0, policy_version 679023 (0.0008) [2023-12-27 10:19:35,546][20381] Updated weights for policy 1, policy_version 678428 (0.0009) [2023-12-27 10:19:35,608][20381] Updated weights for policy 1, policy_version 678438 (0.0006) [2023-12-27 10:19:35,664][20381] Updated weights for policy 1, policy_version 678448 (0.0006) [2023-12-27 10:19:36,030][20347] Updated weights for policy 0, policy_version 679033 (0.0005) [2023-12-27 10:19:36,085][20347] Updated weights for policy 0, policy_version 679043 (0.0005) [2023-12-27 10:19:36,102][20291] KL-divergence is very high: 253.2632 [2023-12-27 10:19:36,152][20347] Updated weights for policy 0, policy_version 679053 (0.0008) [2023-12-27 10:19:36,340][20381] Updated weights for policy 1, policy_version 678458 (0.0007) [2023-12-27 10:19:36,400][20381] Updated weights for policy 1, policy_version 678468 (0.0008) [2023-12-27 10:19:36,464][20381] Updated weights for policy 1, policy_version 678478 (0.0008) [2023-12-27 10:19:36,528][20381] Updated weights for policy 1, policy_version 678488 (0.0007) [2023-12-27 10:19:36,770][20347] Updated weights for policy 0, policy_version 679063 (0.0009) [2023-12-27 10:19:36,830][20347] Updated weights for policy 0, policy_version 679073 (0.0008) [2023-12-27 10:19:36,887][20347] Updated weights for policy 0, policy_version 679083 (0.0008) [2023-12-27 10:19:37,193][20381] Updated weights for policy 1, policy_version 678498 (0.0008) [2023-12-27 10:19:37,252][20381] Updated weights for policy 1, policy_version 678508 (0.0007) [2023-12-27 10:19:37,308][20381] Updated weights for policy 1, policy_version 678518 (0.0008) [2023-12-27 10:19:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 347594752. Throughput: 0: 10283.2, 1: 10303.3. Samples: 347559556. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:37,363][19170] Avg episode reward: [(0, '883.010'), (1, '1000.000')] [2023-12-27 10:19:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000678520_173727744.pth... [2023-12-27 10:19:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000679088_173867008.pth... [2023-12-27 10:19:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000677304_173416448.pth [2023-12-27 10:19:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000677872_173555712.pth [2023-12-27 10:19:37,546][20347] Updated weights for policy 0, policy_version 679093 (0.0007) [2023-12-27 10:19:37,605][20347] Updated weights for policy 0, policy_version 679103 (0.0005) [2023-12-27 10:19:37,662][20347] Updated weights for policy 0, policy_version 679113 (0.0005) [2023-12-27 10:19:38,030][20381] Updated weights for policy 1, policy_version 678528 (0.0008) [2023-12-27 10:19:38,090][20381] Updated weights for policy 1, policy_version 678538 (0.0007) [2023-12-27 10:19:38,151][20381] Updated weights for policy 1, policy_version 678548 (0.0007) [2023-12-27 10:19:38,262][20347] Updated weights for policy 0, policy_version 679123 (0.0006) [2023-12-27 10:19:38,323][20347] Updated weights for policy 0, policy_version 679133 (0.0008) [2023-12-27 10:19:38,382][20347] Updated weights for policy 0, policy_version 679143 (0.0008) [2023-12-27 10:19:38,809][20381] Updated weights for policy 1, policy_version 678558 (0.0006) [2023-12-27 10:19:38,863][20381] Updated weights for policy 1, policy_version 678568 (0.0005) [2023-12-27 10:19:38,924][20381] Updated weights for policy 1, policy_version 678578 (0.0007) [2023-12-27 10:19:39,063][20347] Updated weights for policy 0, policy_version 679153 (0.0008) [2023-12-27 10:19:39,115][20347] Updated weights for policy 0, policy_version 679163 (0.0008) [2023-12-27 10:19:39,166][20347] Updated weights for policy 0, policy_version 679173 (0.0008) [2023-12-27 10:19:39,229][20347] Updated weights for policy 0, policy_version 679183 (0.0008) [2023-12-27 10:19:39,579][20381] Updated weights for policy 1, policy_version 678588 (0.0007) [2023-12-27 10:19:39,635][20381] Updated weights for policy 1, policy_version 678598 (0.0007) [2023-12-27 10:19:39,688][20381] Updated weights for policy 1, policy_version 678608 (0.0008) [2023-12-27 10:19:39,962][20347] Updated weights for policy 0, policy_version 679193 (0.0008) [2023-12-27 10:19:40,014][20347] Updated weights for policy 0, policy_version 679203 (0.0008) [2023-12-27 10:19:40,073][20347] Updated weights for policy 0, policy_version 679213 (0.0008) [2023-12-27 10:19:40,470][20381] Updated weights for policy 1, policy_version 678618 (0.0008) [2023-12-27 10:19:40,524][20381] Updated weights for policy 1, policy_version 678628 (0.0009) [2023-12-27 10:19:40,578][20381] Updated weights for policy 1, policy_version 678638 (0.0009) [2023-12-27 10:19:40,628][20381] Updated weights for policy 1, policy_version 678648 (0.0010) [2023-12-27 10:19:40,747][20347] Updated weights for policy 0, policy_version 679223 (0.0007) [2023-12-27 10:19:40,805][20347] Updated weights for policy 0, policy_version 679233 (0.0007) [2023-12-27 10:19:40,856][20347] Updated weights for policy 0, policy_version 679243 (0.0007) [2023-12-27 10:19:41,407][20381] Updated weights for policy 1, policy_version 678658 (0.0008) [2023-12-27 10:19:41,466][20381] Updated weights for policy 1, policy_version 678668 (0.0007) [2023-12-27 10:19:41,525][20381] Updated weights for policy 1, policy_version 678678 (0.0008) [2023-12-27 10:19:41,628][20347] Updated weights for policy 0, policy_version 679253 (0.0008) [2023-12-27 10:19:41,692][20347] Updated weights for policy 0, policy_version 679263 (0.0007) [2023-12-27 10:19:41,758][20347] Updated weights for policy 0, policy_version 679273 (0.0008) [2023-12-27 10:19:42,231][20381] Updated weights for policy 1, policy_version 678688 (0.0008) [2023-12-27 10:19:42,290][20381] Updated weights for policy 1, policy_version 678698 (0.0008) [2023-12-27 10:19:42,354][20381] Updated weights for policy 1, policy_version 678708 (0.0009) [2023-12-27 10:19:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 347684864. Throughput: 0: 10252.7, 1: 10282.3. Samples: 347681600. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:42,363][19170] Avg episode reward: [(0, '931.660'), (1, '1000.000')] [2023-12-27 10:19:42,488][20347] Updated weights for policy 0, policy_version 679283 (0.0007) [2023-12-27 10:19:42,545][20347] Updated weights for policy 0, policy_version 679293 (0.0007) [2023-12-27 10:19:42,597][20347] Updated weights for policy 0, policy_version 679303 (0.0007) [2023-12-27 10:19:43,040][20381] Updated weights for policy 1, policy_version 678718 (0.0010) [2023-12-27 10:19:43,095][20381] Updated weights for policy 1, policy_version 678728 (0.0010) [2023-12-27 10:19:43,156][20381] Updated weights for policy 1, policy_version 678738 (0.0010) [2023-12-27 10:19:43,300][20347] Updated weights for policy 0, policy_version 679313 (0.0008) [2023-12-27 10:19:43,347][20347] Updated weights for policy 0, policy_version 679323 (0.0007) [2023-12-27 10:19:43,398][20347] Updated weights for policy 0, policy_version 679333 (0.0007) [2023-12-27 10:19:43,452][20347] Updated weights for policy 0, policy_version 679343 (0.0008) [2023-12-27 10:19:43,841][20381] Updated weights for policy 1, policy_version 678748 (0.0010) [2023-12-27 10:19:43,885][20381] Updated weights for policy 1, policy_version 678758 (0.0010) [2023-12-27 10:19:43,929][20381] Updated weights for policy 1, policy_version 678768 (0.0010) [2023-12-27 10:19:44,157][20347] Updated weights for policy 0, policy_version 679353 (0.0007) [2023-12-27 10:19:44,206][20347] Updated weights for policy 0, policy_version 679363 (0.0008) [2023-12-27 10:19:44,258][20347] Updated weights for policy 0, policy_version 679373 (0.0007) [2023-12-27 10:19:44,628][20381] Updated weights for policy 1, policy_version 678778 (0.0010) [2023-12-27 10:19:44,685][20381] Updated weights for policy 1, policy_version 678788 (0.0010) [2023-12-27 10:19:44,735][20381] Updated weights for policy 1, policy_version 678798 (0.0010) [2023-12-27 10:19:44,804][20381] Updated weights for policy 1, policy_version 678808 (0.0011) [2023-12-27 10:19:44,992][20347] Updated weights for policy 0, policy_version 679383 (0.0007) [2023-12-27 10:19:45,041][20347] Updated weights for policy 0, policy_version 679393 (0.0007) [2023-12-27 10:19:45,089][20347] Updated weights for policy 0, policy_version 679403 (0.0007) [2023-12-27 10:19:45,487][20381] Updated weights for policy 1, policy_version 678818 (0.0010) [2023-12-27 10:19:45,544][20381] Updated weights for policy 1, policy_version 678828 (0.0010) [2023-12-27 10:19:45,589][20381] Updated weights for policy 1, policy_version 678838 (0.0010) [2023-12-27 10:19:45,809][20347] Updated weights for policy 0, policy_version 679413 (0.0008) [2023-12-27 10:19:45,856][20347] Updated weights for policy 0, policy_version 679423 (0.0007) [2023-12-27 10:19:45,903][20347] Updated weights for policy 0, policy_version 679433 (0.0008) [2023-12-27 10:19:46,263][20381] Updated weights for policy 1, policy_version 678848 (0.0006) [2023-12-27 10:19:46,326][20381] Updated weights for policy 1, policy_version 678858 (0.0005) [2023-12-27 10:19:46,386][20381] Updated weights for policy 1, policy_version 678868 (0.0005) [2023-12-27 10:19:46,566][20347] Updated weights for policy 0, policy_version 679443 (0.0008) [2023-12-27 10:19:46,626][20347] Updated weights for policy 0, policy_version 679453 (0.0007) [2023-12-27 10:19:46,682][20347] Updated weights for policy 0, policy_version 679463 (0.0006) [2023-12-27 10:19:46,949][20381] Updated weights for policy 1, policy_version 678878 (0.0005) [2023-12-27 10:19:47,010][20381] Updated weights for policy 1, policy_version 678888 (0.0006) [2023-12-27 10:19:47,066][20381] Updated weights for policy 1, policy_version 678898 (0.0005) [2023-12-27 10:19:47,275][20347] Updated weights for policy 0, policy_version 679473 (0.0005) [2023-12-27 10:19:47,337][20347] Updated weights for policy 0, policy_version 679483 (0.0005) [2023-12-27 10:19:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 347791360. Throughput: 0: 10286.4, 1: 10298.9. Samples: 347803584. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:47,363][19170] Avg episode reward: [(0, '951.040'), (1, '1000.000')] [2023-12-27 10:19:47,390][20347] Updated weights for policy 0, policy_version 679493 (0.0005) [2023-12-27 10:19:47,447][20347] Updated weights for policy 0, policy_version 679503 (0.0005) [2023-12-27 10:19:47,690][20381] Updated weights for policy 1, policy_version 678908 (0.0008) [2023-12-27 10:19:47,748][20381] Updated weights for policy 1, policy_version 678918 (0.0009) [2023-12-27 10:19:47,797][20381] Updated weights for policy 1, policy_version 678928 (0.0010) [2023-12-27 10:19:48,021][20347] Updated weights for policy 0, policy_version 679513 (0.0008) [2023-12-27 10:19:48,073][20347] Updated weights for policy 0, policy_version 679523 (0.0007) [2023-12-27 10:19:48,126][20347] Updated weights for policy 0, policy_version 679533 (0.0007) [2023-12-27 10:19:48,472][20381] Updated weights for policy 1, policy_version 678938 (0.0010) [2023-12-27 10:19:48,534][20381] Updated weights for policy 1, policy_version 678948 (0.0007) [2023-12-27 10:19:48,590][20381] Updated weights for policy 1, policy_version 678958 (0.0008) [2023-12-27 10:19:48,646][20381] Updated weights for policy 1, policy_version 678968 (0.0008) [2023-12-27 10:19:48,859][20347] Updated weights for policy 0, policy_version 679543 (0.0007) [2023-12-27 10:19:48,917][20347] Updated weights for policy 0, policy_version 679553 (0.0008) [2023-12-27 10:19:48,977][20347] Updated weights for policy 0, policy_version 679563 (0.0007) [2023-12-27 10:19:49,275][20381] Updated weights for policy 1, policy_version 678978 (0.0008) [2023-12-27 10:19:49,331][20381] Updated weights for policy 1, policy_version 678988 (0.0009) [2023-12-27 10:19:49,397][20381] Updated weights for policy 1, policy_version 678998 (0.0008) [2023-12-27 10:19:49,648][20347] Updated weights for policy 0, policy_version 679573 (0.0008) [2023-12-27 10:19:49,709][20347] Updated weights for policy 0, policy_version 679583 (0.0008) [2023-12-27 10:19:49,768][20347] Updated weights for policy 0, policy_version 679593 (0.0008) [2023-12-27 10:19:50,043][20381] Updated weights for policy 1, policy_version 679008 (0.0006) [2023-12-27 10:19:50,095][20381] Updated weights for policy 1, policy_version 679018 (0.0009) [2023-12-27 10:19:50,142][20381] Updated weights for policy 1, policy_version 679028 (0.0007) [2023-12-27 10:19:50,447][20347] Updated weights for policy 0, policy_version 679603 (0.0007) [2023-12-27 10:19:50,502][20347] Updated weights for policy 0, policy_version 679613 (0.0005) [2023-12-27 10:19:50,571][20347] Updated weights for policy 0, policy_version 679623 (0.0006) [2023-12-27 10:19:50,868][20381] Updated weights for policy 1, policy_version 679038 (0.0009) [2023-12-27 10:19:50,919][20381] Updated weights for policy 1, policy_version 679048 (0.0010) [2023-12-27 10:19:50,966][20381] Updated weights for policy 1, policy_version 679058 (0.0010) [2023-12-27 10:19:51,173][20347] Updated weights for policy 0, policy_version 679633 (0.0007) [2023-12-27 10:19:51,226][20347] Updated weights for policy 0, policy_version 679643 (0.0008) [2023-12-27 10:19:51,284][20347] Updated weights for policy 0, policy_version 679653 (0.0007) [2023-12-27 10:19:51,342][20347] Updated weights for policy 0, policy_version 679663 (0.0007) [2023-12-27 10:19:51,737][20381] Updated weights for policy 1, policy_version 679068 (0.0010) [2023-12-27 10:19:51,793][20381] Updated weights for policy 1, policy_version 679078 (0.0010) [2023-12-27 10:19:51,853][20381] Updated weights for policy 1, policy_version 679088 (0.0011) [2023-12-27 10:19:52,059][20347] Updated weights for policy 0, policy_version 679673 (0.0006) [2023-12-27 10:19:52,111][20347] Updated weights for policy 0, policy_version 679683 (0.0005) [2023-12-27 10:19:52,171][20347] Updated weights for policy 0, policy_version 679693 (0.0007) [2023-12-27 10:19:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 347897856. Throughput: 0: 10293.8, 1: 10311.5. Samples: 347866328. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:52,363][19170] Avg episode reward: [(0, '931.440'), (1, '1000.000')] [2023-12-27 10:19:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000679096_173875200.pth... [2023-12-27 10:19:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000679696_174022656.pth... [2023-12-27 10:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000677912_173572096.pth [2023-12-27 10:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000679096_173875200.pth [2023-12-27 10:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000678480_173711360.pth [2023-12-27 10:19:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000679696_174022656.pth [2023-12-27 10:19:52,578][20381] Updated weights for policy 1, policy_version 679098 (0.0010) [2023-12-27 10:19:52,634][20381] Updated weights for policy 1, policy_version 679108 (0.0008) [2023-12-27 10:19:52,693][20381] Updated weights for policy 1, policy_version 679118 (0.0007) [2023-12-27 10:19:52,752][20381] Updated weights for policy 1, policy_version 679128 (0.0006) [2023-12-27 10:19:52,849][20347] Updated weights for policy 0, policy_version 679703 (0.0007) [2023-12-27 10:19:52,906][20347] Updated weights for policy 0, policy_version 679713 (0.0008) [2023-12-27 10:19:52,967][20347] Updated weights for policy 0, policy_version 679723 (0.0007) [2023-12-27 10:19:53,383][20381] Updated weights for policy 1, policy_version 679138 (0.0007) [2023-12-27 10:19:53,435][20381] Updated weights for policy 1, policy_version 679148 (0.0006) [2023-12-27 10:19:53,480][20381] Updated weights for policy 1, policy_version 679158 (0.0005) [2023-12-27 10:19:53,670][20347] Updated weights for policy 0, policy_version 679733 (0.0008) [2023-12-27 10:19:53,725][20347] Updated weights for policy 0, policy_version 679743 (0.0008) [2023-12-27 10:19:53,787][20347] Updated weights for policy 0, policy_version 679753 (0.0007) [2023-12-27 10:19:54,144][20381] Updated weights for policy 1, policy_version 679168 (0.0007) [2023-12-27 10:19:54,209][20381] Updated weights for policy 1, policy_version 679178 (0.0008) [2023-12-27 10:19:54,271][20381] Updated weights for policy 1, policy_version 679188 (0.0008) [2023-12-27 10:19:54,498][20347] Updated weights for policy 0, policy_version 679763 (0.0007) [2023-12-27 10:19:54,561][20347] Updated weights for policy 0, policy_version 679773 (0.0007) [2023-12-27 10:19:54,627][20347] Updated weights for policy 0, policy_version 679783 (0.0007) [2023-12-27 10:19:54,938][20381] Updated weights for policy 1, policy_version 679198 (0.0008) [2023-12-27 10:19:55,001][20381] Updated weights for policy 1, policy_version 679208 (0.0008) [2023-12-27 10:19:55,068][20381] Updated weights for policy 1, policy_version 679218 (0.0008) [2023-12-27 10:19:55,292][20347] Updated weights for policy 0, policy_version 679793 (0.0007) [2023-12-27 10:19:55,341][20347] Updated weights for policy 0, policy_version 679803 (0.0008) [2023-12-27 10:19:55,392][20347] Updated weights for policy 0, policy_version 679813 (0.0008) [2023-12-27 10:19:55,444][20347] Updated weights for policy 0, policy_version 679823 (0.0007) [2023-12-27 10:19:55,733][20381] Updated weights for policy 1, policy_version 679228 (0.0009) [2023-12-27 10:19:55,781][20381] Updated weights for policy 1, policy_version 679238 (0.0010) [2023-12-27 10:19:55,828][20381] Updated weights for policy 1, policy_version 679248 (0.0010) [2023-12-27 10:19:56,181][20347] Updated weights for policy 0, policy_version 679833 (0.0008) [2023-12-27 10:19:56,238][20347] Updated weights for policy 0, policy_version 679843 (0.0007) [2023-12-27 10:19:56,294][20347] Updated weights for policy 0, policy_version 679853 (0.0006) [2023-12-27 10:19:56,502][20381] Updated weights for policy 1, policy_version 679258 (0.0009) [2023-12-27 10:19:56,548][20381] Updated weights for policy 1, policy_version 679268 (0.0005) [2023-12-27 10:19:56,600][20381] Updated weights for policy 1, policy_version 679278 (0.0005) [2023-12-27 10:19:56,659][20381] Updated weights for policy 1, policy_version 679288 (0.0006) [2023-12-27 10:19:56,991][20347] Updated weights for policy 0, policy_version 679863 (0.0008) [2023-12-27 10:19:57,039][20347] Updated weights for policy 0, policy_version 679873 (0.0009) [2023-12-27 10:19:57,088][20347] Updated weights for policy 0, policy_version 679883 (0.0008) [2023-12-27 10:19:57,239][20381] Updated weights for policy 1, policy_version 679298 (0.0006) [2023-12-27 10:19:57,296][20381] Updated weights for policy 1, policy_version 679308 (0.0005) [2023-12-27 10:19:57,357][20381] Updated weights for policy 1, policy_version 679318 (0.0008) [2023-12-27 10:19:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 347996160. Throughput: 0: 10335.9, 1: 10216.7. Samples: 347987556. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-12-27 10:19:57,364][19170] Avg episode reward: [(0, '931.700'), (1, '991.260')] [2023-12-27 10:19:57,809][20347] Updated weights for policy 0, policy_version 679893 (0.0007) [2023-12-27 10:19:57,857][20347] Updated weights for policy 0, policy_version 679903 (0.0007) [2023-12-27 10:19:57,915][20347] Updated weights for policy 0, policy_version 679913 (0.0007) [2023-12-27 10:19:58,003][20381] Updated weights for policy 1, policy_version 679328 (0.0008) [2023-12-27 10:19:58,064][20381] Updated weights for policy 1, policy_version 679338 (0.0007) [2023-12-27 10:19:58,125][20381] Updated weights for policy 1, policy_version 679348 (0.0007) [2023-12-27 10:19:58,620][20347] Updated weights for policy 0, policy_version 679923 (0.0007) [2023-12-27 10:19:58,682][20347] Updated weights for policy 0, policy_version 679933 (0.0008) [2023-12-27 10:19:58,747][20347] Updated weights for policy 0, policy_version 679943 (0.0007) [2023-12-27 10:19:58,926][20381] Updated weights for policy 1, policy_version 679358 (0.0009) [2023-12-27 10:19:58,987][20381] Updated weights for policy 1, policy_version 679368 (0.0011) [2023-12-27 10:19:59,046][20381] Updated weights for policy 1, policy_version 679378 (0.0011) [2023-12-27 10:19:59,537][20347] Updated weights for policy 0, policy_version 679953 (0.0008) [2023-12-27 10:19:59,596][20347] Updated weights for policy 0, policy_version 679963 (0.0007) [2023-12-27 10:19:59,644][20347] Updated weights for policy 0, policy_version 679973 (0.0006) [2023-12-27 10:19:59,699][20347] Updated weights for policy 0, policy_version 679983 (0.0008) [2023-12-27 10:19:59,741][20381] Updated weights for policy 1, policy_version 679388 (0.0011) [2023-12-27 10:19:59,802][20381] Updated weights for policy 1, policy_version 679398 (0.0010) [2023-12-27 10:19:59,875][20381] Updated weights for policy 1, policy_version 679408 (0.0011) [2023-12-27 10:20:00,352][20347] Updated weights for policy 0, policy_version 679993 (0.0007) [2023-12-27 10:20:00,407][20347] Updated weights for policy 0, policy_version 680003 (0.0007) [2023-12-27 10:20:00,455][20347] Updated weights for policy 0, policy_version 680013 (0.0007) [2023-12-27 10:20:00,556][20381] Updated weights for policy 1, policy_version 679418 (0.0010) [2023-12-27 10:20:00,622][20381] Updated weights for policy 1, policy_version 679428 (0.0007) [2023-12-27 10:20:00,683][20381] Updated weights for policy 1, policy_version 679438 (0.0010) [2023-12-27 10:20:00,748][20381] Updated weights for policy 1, policy_version 679448 (0.0007) [2023-12-27 10:20:01,162][20347] Updated weights for policy 0, policy_version 680023 (0.0008) [2023-12-27 10:20:01,219][20347] Updated weights for policy 0, policy_version 680033 (0.0008) [2023-12-27 10:20:01,282][20347] Updated weights for policy 0, policy_version 680043 (0.0007) [2023-12-27 10:20:01,309][20381] Updated weights for policy 1, policy_version 679458 (0.0005) [2023-12-27 10:20:01,367][20381] Updated weights for policy 1, policy_version 679468 (0.0007) [2023-12-27 10:20:01,418][20381] Updated weights for policy 1, policy_version 679478 (0.0008) [2023-12-27 10:20:02,020][20347] Updated weights for policy 0, policy_version 680053 (0.0007) [2023-12-27 10:20:02,030][20381] Updated weights for policy 1, policy_version 679488 (0.0006) [2023-12-27 10:20:02,070][20347] Updated weights for policy 0, policy_version 680063 (0.0005) [2023-12-27 10:20:02,086][20381] Updated weights for policy 1, policy_version 679498 (0.0006) [2023-12-27 10:20:02,123][20347] Updated weights for policy 0, policy_version 680073 (0.0006) [2023-12-27 10:20:02,140][20381] Updated weights for policy 1, policy_version 679508 (0.0006) [2023-12-27 10:20:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 348102656. Throughput: 0: 10222.5, 1: 10228.9. Samples: 348108668. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:02,363][19170] Avg episode reward: [(0, '892.450'), (1, '972.610')] [2023-12-27 10:20:02,812][20381] Updated weights for policy 1, policy_version 679518 (0.0005) [2023-12-27 10:20:02,856][20347] Updated weights for policy 0, policy_version 680083 (0.0007) [2023-12-27 10:20:02,873][20381] Updated weights for policy 1, policy_version 679528 (0.0006) [2023-12-27 10:20:02,920][20347] Updated weights for policy 0, policy_version 680093 (0.0008) [2023-12-27 10:20:02,928][20381] Updated weights for policy 1, policy_version 679538 (0.0007) [2023-12-27 10:20:02,979][20347] Updated weights for policy 0, policy_version 680103 (0.0007) [2023-12-27 10:20:03,539][20381] Updated weights for policy 1, policy_version 679548 (0.0007) [2023-12-27 10:20:03,595][20381] Updated weights for policy 1, policy_version 679558 (0.0011) [2023-12-27 10:20:03,650][20381] Updated weights for policy 1, policy_version 679568 (0.0010) [2023-12-27 10:20:03,685][20347] Updated weights for policy 0, policy_version 680113 (0.0008) [2023-12-27 10:20:03,738][20347] Updated weights for policy 0, policy_version 680123 (0.0006) [2023-12-27 10:20:03,783][20347] Updated weights for policy 0, policy_version 680133 (0.0005) [2023-12-27 10:20:03,828][20347] Updated weights for policy 0, policy_version 680143 (0.0005) [2023-12-27 10:20:04,355][20381] Updated weights for policy 1, policy_version 679578 (0.0011) [2023-12-27 10:20:04,408][20381] Updated weights for policy 1, policy_version 679588 (0.0010) [2023-12-27 10:20:04,468][20381] Updated weights for policy 1, policy_version 679598 (0.0008) [2023-12-27 10:20:04,519][20347] Updated weights for policy 0, policy_version 680153 (0.0006) [2023-12-27 10:20:04,525][20381] Updated weights for policy 1, policy_version 679608 (0.0008) [2023-12-27 10:20:04,572][20347] Updated weights for policy 0, policy_version 680163 (0.0007) [2023-12-27 10:20:04,620][20347] Updated weights for policy 0, policy_version 680173 (0.0007) [2023-12-27 10:20:05,209][20381] Updated weights for policy 1, policy_version 679618 (0.0005) [2023-12-27 10:20:05,260][20381] Updated weights for policy 1, policy_version 679628 (0.0005) [2023-12-27 10:20:05,311][20381] Updated weights for policy 1, policy_version 679638 (0.0005) [2023-12-27 10:20:05,368][20347] Updated weights for policy 0, policy_version 680183 (0.0007) [2023-12-27 10:20:05,422][20347] Updated weights for policy 0, policy_version 680193 (0.0007) [2023-12-27 10:20:05,478][20347] Updated weights for policy 0, policy_version 680203 (0.0008) [2023-12-27 10:20:05,839][20381] Updated weights for policy 1, policy_version 679648 (0.0005) [2023-12-27 10:20:05,897][20381] Updated weights for policy 1, policy_version 679658 (0.0005) [2023-12-27 10:20:05,943][20381] Updated weights for policy 1, policy_version 679668 (0.0005) [2023-12-27 10:20:06,231][20347] Updated weights for policy 0, policy_version 680213 (0.0007) [2023-12-27 10:20:06,290][20347] Updated weights for policy 0, policy_version 680223 (0.0008) [2023-12-27 10:20:06,344][20347] Updated weights for policy 0, policy_version 680233 (0.0008) [2023-12-27 10:20:06,560][20381] Updated weights for policy 1, policy_version 679678 (0.0008) [2023-12-27 10:20:06,614][20381] Updated weights for policy 1, policy_version 679688 (0.0008) [2023-12-27 10:20:06,667][20381] Updated weights for policy 1, policy_version 679698 (0.0008) [2023-12-27 10:20:07,012][20347] Updated weights for policy 0, policy_version 680243 (0.0008) [2023-12-27 10:20:07,062][20347] Updated weights for policy 0, policy_version 680253 (0.0008) [2023-12-27 10:20:07,113][20347] Updated weights for policy 0, policy_version 680263 (0.0007) [2023-12-27 10:20:07,347][20381] Updated weights for policy 1, policy_version 679708 (0.0007) [2023-12-27 10:20:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 348200960. Throughput: 0: 10162.7, 1: 10296.3. Samples: 348170096. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:07,363][19170] Avg episode reward: [(0, '921.700'), (1, '991.120')] [2023-12-27 10:20:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000680272_174170112.pth... [2023-12-27 10:20:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000679088_173867008.pth [2023-12-27 10:20:07,398][20381] Updated weights for policy 1, policy_version 679718 (0.0005) [2023-12-27 10:20:07,452][20381] Updated weights for policy 1, policy_version 679728 (0.0007) [2023-12-27 10:20:07,486][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000679736_174039040.pth... [2023-12-27 10:20:07,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000678520_173727744.pth [2023-12-27 10:20:07,836][20347] Updated weights for policy 0, policy_version 680273 (0.0008) [2023-12-27 10:20:07,880][20347] Updated weights for policy 0, policy_version 680283 (0.0007) [2023-12-27 10:20:07,926][20347] Updated weights for policy 0, policy_version 680293 (0.0007) [2023-12-27 10:20:07,984][20347] Updated weights for policy 0, policy_version 680303 (0.0007) [2023-12-27 10:20:08,073][20381] Updated weights for policy 1, policy_version 679738 (0.0007) [2023-12-27 10:20:08,125][20381] Updated weights for policy 1, policy_version 679748 (0.0007) [2023-12-27 10:20:08,180][20381] Updated weights for policy 1, policy_version 679758 (0.0006) [2023-12-27 10:20:08,240][20381] Updated weights for policy 1, policy_version 679768 (0.0008) [2023-12-27 10:20:08,736][20347] Updated weights for policy 0, policy_version 680313 (0.0008) [2023-12-27 10:20:08,794][20347] Updated weights for policy 0, policy_version 680323 (0.0007) [2023-12-27 10:20:08,852][20347] Updated weights for policy 0, policy_version 680333 (0.0008) [2023-12-27 10:20:08,873][20381] Updated weights for policy 1, policy_version 679778 (0.0006) [2023-12-27 10:20:08,919][20381] Updated weights for policy 1, policy_version 679788 (0.0006) [2023-12-27 10:20:08,974][20381] Updated weights for policy 1, policy_version 679798 (0.0006) [2023-12-27 10:20:09,606][20347] Updated weights for policy 0, policy_version 680343 (0.0007) [2023-12-27 10:20:09,612][20381] Updated weights for policy 1, policy_version 679808 (0.0005) [2023-12-27 10:20:09,668][20347] Updated weights for policy 0, policy_version 680353 (0.0006) [2023-12-27 10:20:09,673][20381] Updated weights for policy 1, policy_version 679818 (0.0006) [2023-12-27 10:20:09,729][20347] Updated weights for policy 0, policy_version 680363 (0.0008) [2023-12-27 10:20:09,741][20381] Updated weights for policy 1, policy_version 679828 (0.0005) [2023-12-27 10:20:10,404][20381] Updated weights for policy 1, policy_version 679838 (0.0007) [2023-12-27 10:20:10,459][20381] Updated weights for policy 1, policy_version 679848 (0.0006) [2023-12-27 10:20:10,466][20347] Updated weights for policy 0, policy_version 680373 (0.0008) [2023-12-27 10:20:10,513][20381] Updated weights for policy 1, policy_version 679858 (0.0008) [2023-12-27 10:20:10,525][20347] Updated weights for policy 0, policy_version 680383 (0.0009) [2023-12-27 10:20:10,582][20347] Updated weights for policy 0, policy_version 680393 (0.0007) [2023-12-27 10:20:11,198][20381] Updated weights for policy 1, policy_version 679868 (0.0007) [2023-12-27 10:20:11,261][20381] Updated weights for policy 1, policy_version 679878 (0.0008) [2023-12-27 10:20:11,289][20347] Updated weights for policy 0, policy_version 680403 (0.0007) [2023-12-27 10:20:11,315][20381] Updated weights for policy 1, policy_version 679888 (0.0006) [2023-12-27 10:20:11,359][20347] Updated weights for policy 0, policy_version 680413 (0.0009) [2023-12-27 10:20:11,423][20347] Updated weights for policy 0, policy_version 680423 (0.0007) [2023-12-27 10:20:12,018][20381] Updated weights for policy 1, policy_version 679898 (0.0007) [2023-12-27 10:20:12,081][20381] Updated weights for policy 1, policy_version 679908 (0.0008) [2023-12-27 10:20:12,132][20347] Updated weights for policy 0, policy_version 680433 (0.0008) [2023-12-27 10:20:12,142][20381] Updated weights for policy 1, policy_version 679918 (0.0007) [2023-12-27 10:20:12,192][20347] Updated weights for policy 0, policy_version 680443 (0.0007) [2023-12-27 10:20:12,194][20381] Updated weights for policy 1, policy_version 679928 (0.0006) [2023-12-27 10:20:12,247][20347] Updated weights for policy 0, policy_version 680453 (0.0007) [2023-12-27 10:20:12,307][20347] Updated weights for policy 0, policy_version 680463 (0.0007) [2023-12-27 10:20:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 348307456. Throughput: 0: 10124.2, 1: 10353.9. Samples: 348293192. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:12,363][19170] Avg episode reward: [(0, '941.490'), (1, '982.370')] [2023-12-27 10:20:12,918][20381] Updated weights for policy 1, policy_version 679938 (0.0009) [2023-12-27 10:20:12,966][20381] Updated weights for policy 1, policy_version 679948 (0.0008) [2023-12-27 10:20:12,978][20347] Updated weights for policy 0, policy_version 680473 (0.0008) [2023-12-27 10:20:13,023][20381] Updated weights for policy 1, policy_version 679958 (0.0005) [2023-12-27 10:20:13,032][20347] Updated weights for policy 0, policy_version 680483 (0.0008) [2023-12-27 10:20:13,085][20347] Updated weights for policy 0, policy_version 680493 (0.0008) [2023-12-27 10:20:13,650][20381] Updated weights for policy 1, policy_version 679968 (0.0007) [2023-12-27 10:20:13,703][20381] Updated weights for policy 1, policy_version 679978 (0.0007) [2023-12-27 10:20:13,756][20381] Updated weights for policy 1, policy_version 679988 (0.0007) [2023-12-27 10:20:13,815][20347] Updated weights for policy 0, policy_version 680503 (0.0010) [2023-12-27 10:20:13,870][20347] Updated weights for policy 0, policy_version 680513 (0.0010) [2023-12-27 10:20:13,926][20347] Updated weights for policy 0, policy_version 680523 (0.0010) [2023-12-27 10:20:14,395][20381] Updated weights for policy 1, policy_version 679998 (0.0007) [2023-12-27 10:20:14,446][20381] Updated weights for policy 1, policy_version 680008 (0.0005) [2023-12-27 10:20:14,491][20381] Updated weights for policy 1, policy_version 680018 (0.0005) [2023-12-27 10:20:14,590][20347] Updated weights for policy 0, policy_version 680533 (0.0009) [2023-12-27 10:20:14,640][20347] Updated weights for policy 0, policy_version 680543 (0.0009) [2023-12-27 10:20:14,685][20347] Updated weights for policy 0, policy_version 680553 (0.0009) [2023-12-27 10:20:15,069][20381] Updated weights for policy 1, policy_version 680028 (0.0006) [2023-12-27 10:20:15,125][20381] Updated weights for policy 1, policy_version 680038 (0.0007) [2023-12-27 10:20:15,180][20381] Updated weights for policy 1, policy_version 680048 (0.0007) [2023-12-27 10:20:15,406][20347] Updated weights for policy 0, policy_version 680563 (0.0007) [2023-12-27 10:20:15,456][20347] Updated weights for policy 0, policy_version 680573 (0.0007) [2023-12-27 10:20:15,503][20347] Updated weights for policy 0, policy_version 680583 (0.0010) [2023-12-27 10:20:15,892][20381] Updated weights for policy 1, policy_version 680058 (0.0007) [2023-12-27 10:20:15,950][20381] Updated weights for policy 1, policy_version 680069 (0.0008) [2023-12-27 10:20:16,002][20381] Updated weights for policy 1, policy_version 680079 (0.0007) [2023-12-27 10:20:16,162][20347] Updated weights for policy 0, policy_version 680594 (0.0007) [2023-12-27 10:20:16,211][20347] Updated weights for policy 0, policy_version 680604 (0.0008) [2023-12-27 10:20:16,261][20347] Updated weights for policy 0, policy_version 680614 (0.0009) [2023-12-27 10:20:16,309][20347] Updated weights for policy 0, policy_version 680624 (0.0008) [2023-12-27 10:20:16,577][20381] Updated weights for policy 1, policy_version 680089 (0.0008) [2023-12-27 10:20:16,625][20381] Updated weights for policy 1, policy_version 680099 (0.0008) [2023-12-27 10:20:16,674][20381] Updated weights for policy 1, policy_version 680109 (0.0007) [2023-12-27 10:20:16,724][20381] Updated weights for policy 1, policy_version 680119 (0.0008) [2023-12-27 10:20:17,052][20347] Updated weights for policy 0, policy_version 680634 (0.0007) [2023-12-27 10:20:17,108][20347] Updated weights for policy 0, policy_version 680644 (0.0008) [2023-12-27 10:20:17,155][20347] Updated weights for policy 0, policy_version 680654 (0.0007) [2023-12-27 10:20:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20743.8). Total num frames: 348405760. Throughput: 0: 10072.3, 1: 10415.4. Samples: 348416508. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:17,363][19170] Avg episode reward: [(0, '951.100'), (1, '982.300')] [2023-12-27 10:20:17,454][20381] Updated weights for policy 1, policy_version 680129 (0.0010) [2023-12-27 10:20:17,508][20381] Updated weights for policy 1, policy_version 680139 (0.0006) [2023-12-27 10:20:17,567][20381] Updated weights for policy 1, policy_version 680149 (0.0006) [2023-12-27 10:20:17,875][20347] Updated weights for policy 0, policy_version 680664 (0.0007) [2023-12-27 10:20:17,926][20347] Updated weights for policy 0, policy_version 680674 (0.0007) [2023-12-27 10:20:17,983][20347] Updated weights for policy 0, policy_version 680684 (0.0007) [2023-12-27 10:20:18,198][20381] Updated weights for policy 1, policy_version 680159 (0.0010) [2023-12-27 10:20:18,245][20381] Updated weights for policy 1, policy_version 680169 (0.0010) [2023-12-27 10:20:18,293][20381] Updated weights for policy 1, policy_version 680179 (0.0010) [2023-12-27 10:20:18,722][20347] Updated weights for policy 0, policy_version 680694 (0.0008) [2023-12-27 10:20:18,778][20347] Updated weights for policy 0, policy_version 680704 (0.0008) [2023-12-27 10:20:18,834][20347] Updated weights for policy 0, policy_version 680714 (0.0007) [2023-12-27 10:20:18,976][20381] Updated weights for policy 1, policy_version 680189 (0.0007) [2023-12-27 10:20:19,031][20381] Updated weights for policy 1, policy_version 680199 (0.0007) [2023-12-27 10:20:19,085][20381] Updated weights for policy 1, policy_version 680209 (0.0006) [2023-12-27 10:20:19,558][20347] Updated weights for policy 0, policy_version 680724 (0.0009) [2023-12-27 10:20:19,622][20347] Updated weights for policy 0, policy_version 680734 (0.0008) [2023-12-27 10:20:19,688][20347] Updated weights for policy 0, policy_version 680744 (0.0008) [2023-12-27 10:20:19,762][20381] Updated weights for policy 1, policy_version 680219 (0.0006) [2023-12-27 10:20:19,816][20381] Updated weights for policy 1, policy_version 680229 (0.0008) [2023-12-27 10:20:19,875][20381] Updated weights for policy 1, policy_version 680239 (0.0008) [2023-12-27 10:20:20,392][20347] Updated weights for policy 0, policy_version 680754 (0.0007) [2023-12-27 10:20:20,450][20347] Updated weights for policy 0, policy_version 680764 (0.0009) [2023-12-27 10:20:20,503][20347] Updated weights for policy 0, policy_version 680774 (0.0008) [2023-12-27 10:20:20,532][20381] Updated weights for policy 1, policy_version 680249 (0.0008) [2023-12-27 10:20:20,557][20347] Updated weights for policy 0, policy_version 680784 (0.0008) [2023-12-27 10:20:20,587][20381] Updated weights for policy 1, policy_version 680259 (0.0006) [2023-12-27 10:20:20,641][20381] Updated weights for policy 1, policy_version 680269 (0.0006) [2023-12-27 10:20:20,698][20381] Updated weights for policy 1, policy_version 680279 (0.0008) [2023-12-27 10:20:21,287][20381] Updated weights for policy 1, policy_version 680289 (0.0008) [2023-12-27 10:20:21,352][20381] Updated weights for policy 1, policy_version 680299 (0.0007) [2023-12-27 10:20:21,357][20347] Updated weights for policy 0, policy_version 680794 (0.0008) [2023-12-27 10:20:21,417][20381] Updated weights for policy 1, policy_version 680309 (0.0007) [2023-12-27 10:20:21,423][20347] Updated weights for policy 0, policy_version 680804 (0.0007) [2023-12-27 10:20:21,480][20347] Updated weights for policy 0, policy_version 680814 (0.0006) [2023-12-27 10:20:22,130][20381] Updated weights for policy 1, policy_version 680319 (0.0007) [2023-12-27 10:20:22,165][20347] Updated weights for policy 0, policy_version 680824 (0.0006) [2023-12-27 10:20:22,188][20381] Updated weights for policy 1, policy_version 680329 (0.0008) [2023-12-27 10:20:22,216][20347] Updated weights for policy 0, policy_version 680834 (0.0006) [2023-12-27 10:20:22,247][20381] Updated weights for policy 1, policy_version 680339 (0.0008) [2023-12-27 10:20:22,274][20347] Updated weights for policy 0, policy_version 680844 (0.0007) [2023-12-27 10:20:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 348512256. Throughput: 0: 9987.7, 1: 10401.4. Samples: 348477068. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:22,363][19170] Avg episode reward: [(0, '926.240'), (1, '982.320')] [2023-12-27 10:20:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000680344_174194688.pth... [2023-12-27 10:20:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000680848_174317568.pth... [2023-12-27 10:20:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000679096_173875200.pth [2023-12-27 10:20:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000679696_174022656.pth [2023-12-27 10:20:22,911][20381] Updated weights for policy 1, policy_version 680349 (0.0008) [2023-12-27 10:20:22,972][20381] Updated weights for policy 1, policy_version 680359 (0.0007) [2023-12-27 10:20:22,994][20347] Updated weights for policy 0, policy_version 680854 (0.0006) [2023-12-27 10:20:23,030][20381] Updated weights for policy 1, policy_version 680369 (0.0006) [2023-12-27 10:20:23,052][20347] Updated weights for policy 0, policy_version 680864 (0.0007) [2023-12-27 10:20:23,098][20347] Updated weights for policy 0, policy_version 680874 (0.0006) [2023-12-27 10:20:23,599][20381] Updated weights for policy 1, policy_version 680379 (0.0008) [2023-12-27 10:20:23,662][20381] Updated weights for policy 1, policy_version 680389 (0.0005) [2023-12-27 10:20:23,689][20347] Updated weights for policy 0, policy_version 680884 (0.0007) [2023-12-27 10:20:23,724][20381] Updated weights for policy 1, policy_version 680399 (0.0006) [2023-12-27 10:20:23,742][20347] Updated weights for policy 0, policy_version 680894 (0.0006) [2023-12-27 10:20:23,796][20347] Updated weights for policy 0, policy_version 680904 (0.0007) [2023-12-27 10:20:24,317][20381] Updated weights for policy 1, policy_version 680409 (0.0007) [2023-12-27 10:20:24,374][20381] Updated weights for policy 1, policy_version 680419 (0.0008) [2023-12-27 10:20:24,426][20381] Updated weights for policy 1, policy_version 680429 (0.0008) [2023-12-27 10:20:24,466][20347] Updated weights for policy 0, policy_version 680914 (0.0007) [2023-12-27 10:20:24,477][20381] Updated weights for policy 1, policy_version 680439 (0.0007) [2023-12-27 10:20:24,516][20347] Updated weights for policy 0, policy_version 680924 (0.0005) [2023-12-27 10:20:24,569][20347] Updated weights for policy 0, policy_version 680934 (0.0008) [2023-12-27 10:20:24,627][20347] Updated weights for policy 0, policy_version 680944 (0.0008) [2023-12-27 10:20:25,158][20381] Updated weights for policy 1, policy_version 680449 (0.0010) [2023-12-27 10:20:25,210][20381] Updated weights for policy 1, policy_version 680459 (0.0010) [2023-12-27 10:20:25,261][20381] Updated weights for policy 1, policy_version 680469 (0.0010) [2023-12-27 10:20:25,327][20347] Updated weights for policy 0, policy_version 680954 (0.0007) [2023-12-27 10:20:25,366][20291] KL-divergence is very high: 103.7053 [2023-12-27 10:20:25,371][20347] Updated weights for policy 0, policy_version 680964 (0.0007) [2023-12-27 10:20:25,383][20291] KL-divergence is very high: 141.6203 [2023-12-27 10:20:25,431][20347] Updated weights for policy 0, policy_version 680974 (0.0007) [2023-12-27 10:20:25,887][20381] Updated weights for policy 1, policy_version 680479 (0.0008) [2023-12-27 10:20:25,945][20381] Updated weights for policy 1, policy_version 680489 (0.0006) [2023-12-27 10:20:25,996][20381] Updated weights for policy 1, policy_version 680499 (0.0009) [2023-12-27 10:20:26,039][20347] Updated weights for policy 0, policy_version 680984 (0.0006) [2023-12-27 10:20:26,095][20347] Updated weights for policy 0, policy_version 680994 (0.0005) [2023-12-27 10:20:26,152][20347] Updated weights for policy 0, policy_version 681004 (0.0005) [2023-12-27 10:20:26,635][20381] Updated weights for policy 1, policy_version 680509 (0.0010) [2023-12-27 10:20:26,689][20381] Updated weights for policy 1, policy_version 680519 (0.0010) [2023-12-27 10:20:26,711][20347] Updated weights for policy 0, policy_version 681014 (0.0006) [2023-12-27 10:20:26,743][20381] Updated weights for policy 1, policy_version 680529 (0.0010) [2023-12-27 10:20:26,763][20347] Updated weights for policy 0, policy_version 681024 (0.0005) [2023-12-27 10:20:26,808][20347] Updated weights for policy 0, policy_version 681034 (0.0006) [2023-12-27 10:20:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 348610560. Throughput: 0: 9982.2, 1: 10493.9. Samples: 348603024. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:27,363][19170] Avg episode reward: [(0, '765.400'), (1, '991.270')] [2023-12-27 10:20:27,410][20347] Updated weights for policy 0, policy_version 681044 (0.0006) [2023-12-27 10:20:27,415][20381] Updated weights for policy 1, policy_version 680539 (0.0010) [2023-12-27 10:20:27,471][20381] Updated weights for policy 1, policy_version 680549 (0.0011) [2023-12-27 10:20:27,474][20347] Updated weights for policy 0, policy_version 681054 (0.0005) [2023-12-27 10:20:27,527][20381] Updated weights for policy 1, policy_version 680559 (0.0010) [2023-12-27 10:20:27,538][20347] Updated weights for policy 0, policy_version 681064 (0.0005) [2023-12-27 10:20:28,154][20347] Updated weights for policy 0, policy_version 681074 (0.0006) [2023-12-27 10:20:28,207][20347] Updated weights for policy 0, policy_version 681084 (0.0007) [2023-12-27 10:20:28,212][20381] Updated weights for policy 1, policy_version 680569 (0.0010) [2023-12-27 10:20:28,259][20347] Updated weights for policy 0, policy_version 681094 (0.0009) [2023-12-27 10:20:28,266][20381] Updated weights for policy 1, policy_version 680579 (0.0010) [2023-12-27 10:20:28,304][20347] Updated weights for policy 0, policy_version 681104 (0.0006) [2023-12-27 10:20:28,317][20381] Updated weights for policy 1, policy_version 680589 (0.0010) [2023-12-27 10:20:28,377][20381] Updated weights for policy 1, policy_version 680599 (0.0011) [2023-12-27 10:20:29,020][20347] Updated weights for policy 0, policy_version 681114 (0.0006) [2023-12-27 10:20:29,029][20381] Updated weights for policy 1, policy_version 680609 (0.0010) [2023-12-27 10:20:29,069][20347] Updated weights for policy 0, policy_version 681124 (0.0006) [2023-12-27 10:20:29,082][20381] Updated weights for policy 1, policy_version 680619 (0.0010) [2023-12-27 10:20:29,104][20292] KL-divergence is very high: 133.3080 [2023-12-27 10:20:29,128][20347] Updated weights for policy 0, policy_version 681134 (0.0006) [2023-12-27 10:20:29,131][20381] Updated weights for policy 1, policy_version 680629 (0.0010) [2023-12-27 10:20:29,753][20347] Updated weights for policy 0, policy_version 681144 (0.0006) [2023-12-27 10:20:29,779][20381] Updated weights for policy 1, policy_version 680639 (0.0007) [2023-12-27 10:20:29,807][20347] Updated weights for policy 0, policy_version 681154 (0.0006) [2023-12-27 10:20:29,837][20381] Updated weights for policy 1, policy_version 680649 (0.0007) [2023-12-27 10:20:29,864][20347] Updated weights for policy 0, policy_version 681164 (0.0008) [2023-12-27 10:20:29,882][20381] Updated weights for policy 1, policy_version 680659 (0.0008) [2023-12-27 10:20:30,470][20347] Updated weights for policy 0, policy_version 681174 (0.0005) [2023-12-27 10:20:30,483][20381] Updated weights for policy 1, policy_version 680669 (0.0009) [2023-12-27 10:20:30,523][20347] Updated weights for policy 0, policy_version 681184 (0.0005) [2023-12-27 10:20:30,536][20381] Updated weights for policy 1, policy_version 680679 (0.0008) [2023-12-27 10:20:30,573][20347] Updated weights for policy 0, policy_version 681194 (0.0005) [2023-12-27 10:20:30,595][20381] Updated weights for policy 1, policy_version 680689 (0.0009) [2023-12-27 10:20:31,179][20381] Updated weights for policy 1, policy_version 680699 (0.0010) [2023-12-27 10:20:31,189][20347] Updated weights for policy 0, policy_version 681204 (0.0006) [2023-12-27 10:20:31,220][20292] KL-divergence is very high: 119.3585 [2023-12-27 10:20:31,244][20381] Updated weights for policy 1, policy_version 680709 (0.0008) [2023-12-27 10:20:31,250][20347] Updated weights for policy 0, policy_version 681214 (0.0007) [2023-12-27 10:20:31,295][20347] Updated weights for policy 0, policy_version 681224 (0.0007) [2023-12-27 10:20:31,302][20381] Updated weights for policy 1, policy_version 680719 (0.0008) [2023-12-27 10:20:31,940][20347] Updated weights for policy 0, policy_version 681234 (0.0006) [2023-12-27 10:20:32,005][20347] Updated weights for policy 0, policy_version 681244 (0.0005) [2023-12-27 10:20:32,015][20381] Updated weights for policy 1, policy_version 680729 (0.0008) [2023-12-27 10:20:32,061][20347] Updated weights for policy 0, policy_version 681254 (0.0007) [2023-12-27 10:20:32,071][20381] Updated weights for policy 1, policy_version 680739 (0.0006) [2023-12-27 10:20:32,118][20347] Updated weights for policy 0, policy_version 681264 (0.0006) [2023-12-27 10:20:32,130][20381] Updated weights for policy 1, policy_version 680749 (0.0005) [2023-12-27 10:20:32,186][20381] Updated weights for policy 1, policy_version 680759 (0.0008) [2023-12-27 10:20:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 348725248. Throughput: 0: 10115.8, 1: 10528.3. Samples: 348732568. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:32,363][19170] Avg episode reward: [(0, '757.150'), (1, '972.190')] [2023-12-27 10:20:32,732][20347] Updated weights for policy 0, policy_version 681274 (0.0006) [2023-12-27 10:20:32,791][20347] Updated weights for policy 0, policy_version 681284 (0.0010) [2023-12-27 10:20:32,842][20347] Updated weights for policy 0, policy_version 681294 (0.0010) [2023-12-27 10:20:32,884][20381] Updated weights for policy 1, policy_version 680769 (0.0005) [2023-12-27 10:20:32,938][20381] Updated weights for policy 1, policy_version 680779 (0.0005) [2023-12-27 10:20:32,992][20381] Updated weights for policy 1, policy_version 680789 (0.0005) [2023-12-27 10:20:33,460][20347] Updated weights for policy 0, policy_version 681304 (0.0010) [2023-12-27 10:20:33,512][20347] Updated weights for policy 0, policy_version 681314 (0.0010) [2023-12-27 10:20:33,513][20291] KL-divergence is very high: 112.6042 [2023-12-27 10:20:33,558][20381] Updated weights for policy 1, policy_version 680799 (0.0006) [2023-12-27 10:20:33,560][20347] Updated weights for policy 0, policy_version 681324 (0.0009) [2023-12-27 10:20:33,610][20381] Updated weights for policy 1, policy_version 680809 (0.0006) [2023-12-27 10:20:33,667][20381] Updated weights for policy 1, policy_version 680819 (0.0007) [2023-12-27 10:20:34,201][20347] Updated weights for policy 0, policy_version 681334 (0.0008) [2023-12-27 10:20:34,271][20347] Updated weights for policy 0, policy_version 681344 (0.0006) [2023-12-27 10:20:34,329][20347] Updated weights for policy 0, policy_version 681354 (0.0008) [2023-12-27 10:20:34,341][20381] Updated weights for policy 1, policy_version 680829 (0.0007) [2023-12-27 10:20:34,407][20381] Updated weights for policy 1, policy_version 680839 (0.0007) [2023-12-27 10:20:34,469][20381] Updated weights for policy 1, policy_version 680849 (0.0006) [2023-12-27 10:20:34,992][20347] Updated weights for policy 0, policy_version 681364 (0.0007) [2023-12-27 10:20:35,053][20347] Updated weights for policy 0, policy_version 681374 (0.0007) [2023-12-27 10:20:35,110][20347] Updated weights for policy 0, policy_version 681384 (0.0008) [2023-12-27 10:20:35,112][20381] Updated weights for policy 1, policy_version 680859 (0.0008) [2023-12-27 10:20:35,171][20381] Updated weights for policy 1, policy_version 680869 (0.0006) [2023-12-27 10:20:35,230][20381] Updated weights for policy 1, policy_version 680879 (0.0006) [2023-12-27 10:20:35,789][20347] Updated weights for policy 0, policy_version 681394 (0.0006) [2023-12-27 10:20:35,818][20381] Updated weights for policy 1, policy_version 680889 (0.0007) [2023-12-27 10:20:35,838][20347] Updated weights for policy 0, policy_version 681404 (0.0007) [2023-12-27 10:20:35,866][20381] Updated weights for policy 1, policy_version 680899 (0.0006) [2023-12-27 10:20:35,899][20347] Updated weights for policy 0, policy_version 681414 (0.0007) [2023-12-27 10:20:35,923][20381] Updated weights for policy 1, policy_version 680909 (0.0006) [2023-12-27 10:20:35,951][20347] Updated weights for policy 0, policy_version 681424 (0.0007) [2023-12-27 10:20:35,987][20381] Updated weights for policy 1, policy_version 680919 (0.0007) [2023-12-27 10:20:36,615][20347] Updated weights for policy 0, policy_version 681434 (0.0005) [2023-12-27 10:20:36,674][20347] Updated weights for policy 0, policy_version 681444 (0.0005) [2023-12-27 10:20:36,723][20381] Updated weights for policy 1, policy_version 680929 (0.0006) [2023-12-27 10:20:36,728][20347] Updated weights for policy 0, policy_version 681454 (0.0005) [2023-12-27 10:20:36,782][20381] Updated weights for policy 1, policy_version 680939 (0.0005) [2023-12-27 10:20:36,850][20381] Updated weights for policy 1, policy_version 680949 (0.0007) [2023-12-27 10:20:37,323][20347] Updated weights for policy 0, policy_version 681464 (0.0007) [2023-12-27 10:20:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 348823552. Throughput: 0: 10134.2, 1: 10552.9. Samples: 348797240. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:37,363][19170] Avg episode reward: [(0, '433.260'), (1, '962.450')] [2023-12-27 10:20:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000680952_174350336.pth... [2023-12-27 10:20:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000679736_174039040.pth [2023-12-27 10:20:37,390][20347] Updated weights for policy 0, policy_version 681474 (0.0008) [2023-12-27 10:20:37,450][20347] Updated weights for policy 0, policy_version 681484 (0.0007) [2023-12-27 10:20:37,472][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000681488_174481408.pth... [2023-12-27 10:20:37,476][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000680272_174170112.pth [2023-12-27 10:20:37,510][20381] Updated weights for policy 1, policy_version 680959 (0.0009) [2023-12-27 10:20:37,569][20381] Updated weights for policy 1, policy_version 680969 (0.0011) [2023-12-27 10:20:37,629][20381] Updated weights for policy 1, policy_version 680979 (0.0010) [2023-12-27 10:20:38,138][20347] Updated weights for policy 0, policy_version 681494 (0.0006) [2023-12-27 10:20:38,184][20347] Updated weights for policy 0, policy_version 681504 (0.0007) [2023-12-27 10:20:38,244][20347] Updated weights for policy 0, policy_version 681514 (0.0007) [2023-12-27 10:20:38,322][20381] Updated weights for policy 1, policy_version 680989 (0.0011) [2023-12-27 10:20:38,382][20381] Updated weights for policy 1, policy_version 680999 (0.0011) [2023-12-27 10:20:38,438][20381] Updated weights for policy 1, policy_version 681009 (0.0010) [2023-12-27 10:20:38,964][20347] Updated weights for policy 0, policy_version 681524 (0.0007) [2023-12-27 10:20:39,006][20381] Updated weights for policy 1, policy_version 681019 (0.0010) [2023-12-27 10:20:39,019][20347] Updated weights for policy 0, policy_version 681534 (0.0006) [2023-12-27 10:20:39,056][20381] Updated weights for policy 1, policy_version 681029 (0.0007) [2023-12-27 10:20:39,073][20347] Updated weights for policy 0, policy_version 681544 (0.0007) [2023-12-27 10:20:39,118][20381] Updated weights for policy 1, policy_version 681039 (0.0006) [2023-12-27 10:20:39,820][20381] Updated weights for policy 1, policy_version 681049 (0.0005) [2023-12-27 10:20:39,823][20347] Updated weights for policy 0, policy_version 681554 (0.0007) [2023-12-27 10:20:39,884][20381] Updated weights for policy 1, policy_version 681059 (0.0007) [2023-12-27 10:20:39,886][20347] Updated weights for policy 0, policy_version 681564 (0.0007) [2023-12-27 10:20:39,945][20381] Updated weights for policy 1, policy_version 681069 (0.0007) [2023-12-27 10:20:39,953][20347] Updated weights for policy 0, policy_version 681574 (0.0007) [2023-12-27 10:20:40,002][20381] Updated weights for policy 1, policy_version 681079 (0.0007) [2023-12-27 10:20:40,005][20347] Updated weights for policy 0, policy_version 681584 (0.0008) [2023-12-27 10:20:40,668][20381] Updated weights for policy 1, policy_version 681089 (0.0006) [2023-12-27 10:20:40,735][20381] Updated weights for policy 1, policy_version 681099 (0.0008) [2023-12-27 10:20:40,765][20347] Updated weights for policy 0, policy_version 681594 (0.0007) [2023-12-27 10:20:40,809][20381] Updated weights for policy 1, policy_version 681109 (0.0007) [2023-12-27 10:20:40,814][20347] Updated weights for policy 0, policy_version 681604 (0.0006) [2023-12-27 10:20:40,867][20347] Updated weights for policy 0, policy_version 681614 (0.0006) [2023-12-27 10:20:41,453][20381] Updated weights for policy 1, policy_version 681119 (0.0006) [2023-12-27 10:20:41,511][20381] Updated weights for policy 1, policy_version 681129 (0.0005) [2023-12-27 10:20:41,568][20381] Updated weights for policy 1, policy_version 681139 (0.0009) [2023-12-27 10:20:41,578][20347] Updated weights for policy 0, policy_version 681624 (0.0006) [2023-12-27 10:20:41,639][20347] Updated weights for policy 0, policy_version 681634 (0.0008) [2023-12-27 10:20:41,703][20347] Updated weights for policy 0, policy_version 681644 (0.0008) [2023-12-27 10:20:42,242][20381] Updated weights for policy 1, policy_version 681149 (0.0009) [2023-12-27 10:20:42,307][20381] Updated weights for policy 1, policy_version 681159 (0.0007) [2023-12-27 10:20:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 348921856. Throughput: 0: 10123.8, 1: 10589.7. Samples: 348919660. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:42,363][19170] Avg episode reward: [(0, '451.240'), (1, '980.730')] [2023-12-27 10:20:42,374][20381] Updated weights for policy 1, policy_version 681169 (0.0007) [2023-12-27 10:20:42,423][20347] Updated weights for policy 0, policy_version 681654 (0.0008) [2023-12-27 10:20:42,481][20347] Updated weights for policy 0, policy_version 681664 (0.0007) [2023-12-27 10:20:42,539][20347] Updated weights for policy 0, policy_version 681674 (0.0008) [2023-12-27 10:20:43,044][20381] Updated weights for policy 1, policy_version 681179 (0.0009) [2023-12-27 10:20:43,101][20381] Updated weights for policy 1, policy_version 681189 (0.0005) [2023-12-27 10:20:43,159][20381] Updated weights for policy 1, policy_version 681199 (0.0006) [2023-12-27 10:20:43,191][20347] Updated weights for policy 0, policy_version 681684 (0.0006) [2023-12-27 10:20:43,250][20347] Updated weights for policy 0, policy_version 681694 (0.0006) [2023-12-27 10:20:43,305][20347] Updated weights for policy 0, policy_version 681704 (0.0005) [2023-12-27 10:20:43,724][20381] Updated weights for policy 1, policy_version 681209 (0.0007) [2023-12-27 10:20:43,777][20381] Updated weights for policy 1, policy_version 681219 (0.0010) [2023-12-27 10:20:43,834][20381] Updated weights for policy 1, policy_version 681229 (0.0010) [2023-12-27 10:20:43,895][20381] Updated weights for policy 1, policy_version 681239 (0.0010) [2023-12-27 10:20:43,921][20347] Updated weights for policy 0, policy_version 681714 (0.0006) [2023-12-27 10:20:43,979][20347] Updated weights for policy 0, policy_version 681724 (0.0007) [2023-12-27 10:20:44,039][20347] Updated weights for policy 0, policy_version 681734 (0.0007) [2023-12-27 10:20:44,089][20347] Updated weights for policy 0, policy_version 681744 (0.0007) [2023-12-27 10:20:44,546][20381] Updated weights for policy 1, policy_version 681249 (0.0010) [2023-12-27 10:20:44,597][20381] Updated weights for policy 1, policy_version 681259 (0.0010) [2023-12-27 10:20:44,644][20381] Updated weights for policy 1, policy_version 681269 (0.0010) [2023-12-27 10:20:44,725][20347] Updated weights for policy 0, policy_version 681754 (0.0007) [2023-12-27 10:20:44,784][20347] Updated weights for policy 0, policy_version 681764 (0.0007) [2023-12-27 10:20:44,843][20347] Updated weights for policy 0, policy_version 681774 (0.0008) [2023-12-27 10:20:45,359][20381] Updated weights for policy 1, policy_version 681279 (0.0007) [2023-12-27 10:20:45,421][20381] Updated weights for policy 1, policy_version 681289 (0.0006) [2023-12-27 10:20:45,478][20381] Updated weights for policy 1, policy_version 681299 (0.0007) [2023-12-27 10:20:45,509][20347] Updated weights for policy 0, policy_version 681784 (0.0009) [2023-12-27 10:20:45,570][20347] Updated weights for policy 0, policy_version 681794 (0.0010) [2023-12-27 10:20:45,626][20347] Updated weights for policy 0, policy_version 681804 (0.0010) [2023-12-27 10:20:46,069][20381] Updated weights for policy 1, policy_version 681309 (0.0005) [2023-12-27 10:20:46,128][20381] Updated weights for policy 1, policy_version 681319 (0.0006) [2023-12-27 10:20:46,184][20381] Updated weights for policy 1, policy_version 681329 (0.0005) [2023-12-27 10:20:46,297][20347] Updated weights for policy 0, policy_version 681814 (0.0010) [2023-12-27 10:20:46,357][20347] Updated weights for policy 0, policy_version 681824 (0.0010) [2023-12-27 10:20:46,414][20347] Updated weights for policy 0, policy_version 681834 (0.0010) [2023-12-27 10:20:46,741][20381] Updated weights for policy 1, policy_version 681339 (0.0006) [2023-12-27 10:20:46,792][20381] Updated weights for policy 1, policy_version 681349 (0.0007) [2023-12-27 10:20:46,843][20381] Updated weights for policy 1, policy_version 681359 (0.0007) [2023-12-27 10:20:47,085][20347] Updated weights for policy 0, policy_version 681844 (0.0010) [2023-12-27 10:20:47,146][20347] Updated weights for policy 0, policy_version 681854 (0.0010) [2023-12-27 10:20:47,204][20347] Updated weights for policy 0, policy_version 681864 (0.0010) [2023-12-27 10:20:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 349036544. Throughput: 0: 10188.7, 1: 10626.5. Samples: 349045352. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:47,363][19170] Avg episode reward: [(0, '614.740'), (1, '981.570')] [2023-12-27 10:20:47,550][20381] Updated weights for policy 1, policy_version 681369 (0.0008) [2023-12-27 10:20:47,608][20381] Updated weights for policy 1, policy_version 681379 (0.0007) [2023-12-27 10:20:47,662][20381] Updated weights for policy 1, policy_version 681389 (0.0007) [2023-12-27 10:20:47,716][20381] Updated weights for policy 1, policy_version 681399 (0.0007) [2023-12-27 10:20:47,871][20347] Updated weights for policy 0, policy_version 681874 (0.0011) [2023-12-27 10:20:47,908][20291] KL-divergence is very high: 119.9715 [2023-12-27 10:20:47,926][20347] Updated weights for policy 0, policy_version 681884 (0.0011) [2023-12-27 10:20:47,978][20347] Updated weights for policy 0, policy_version 681894 (0.0010) [2023-12-27 10:20:48,029][20347] Updated weights for policy 0, policy_version 681904 (0.0009) [2023-12-27 10:20:48,373][20381] Updated weights for policy 1, policy_version 681409 (0.0006) [2023-12-27 10:20:48,429][20381] Updated weights for policy 1, policy_version 681419 (0.0007) [2023-12-27 10:20:48,477][20381] Updated weights for policy 1, policy_version 681429 (0.0007) [2023-12-27 10:20:48,719][20347] Updated weights for policy 0, policy_version 681914 (0.0006) [2023-12-27 10:20:48,789][20347] Updated weights for policy 0, policy_version 681924 (0.0007) [2023-12-27 10:20:48,850][20347] Updated weights for policy 0, policy_version 681934 (0.0007) [2023-12-27 10:20:49,085][20381] Updated weights for policy 1, policy_version 681439 (0.0006) [2023-12-27 10:20:49,145][20381] Updated weights for policy 1, policy_version 681449 (0.0010) [2023-12-27 10:20:49,207][20381] Updated weights for policy 1, policy_version 681459 (0.0011) [2023-12-27 10:20:49,473][20347] Updated weights for policy 0, policy_version 681944 (0.0009) [2023-12-27 10:20:49,528][20347] Updated weights for policy 0, policy_version 681954 (0.0010) [2023-12-27 10:20:49,588][20347] Updated weights for policy 0, policy_version 681964 (0.0006) [2023-12-27 10:20:49,837][20381] Updated weights for policy 1, policy_version 681469 (0.0009) [2023-12-27 10:20:49,901][20381] Updated weights for policy 1, policy_version 681479 (0.0011) [2023-12-27 10:20:49,964][20381] Updated weights for policy 1, policy_version 681489 (0.0012) [2023-12-27 10:20:50,306][20347] Updated weights for policy 0, policy_version 681974 (0.0007) [2023-12-27 10:20:50,356][20347] Updated weights for policy 0, policy_version 681984 (0.0007) [2023-12-27 10:20:50,407][20347] Updated weights for policy 0, policy_version 681994 (0.0008) [2023-12-27 10:20:50,670][20381] Updated weights for policy 1, policy_version 681499 (0.0011) [2023-12-27 10:20:50,729][20381] Updated weights for policy 1, policy_version 681509 (0.0011) [2023-12-27 10:20:50,778][20381] Updated weights for policy 1, policy_version 681519 (0.0011) [2023-12-27 10:20:51,129][20347] Updated weights for policy 0, policy_version 682004 (0.0009) [2023-12-27 10:20:51,182][20347] Updated weights for policy 0, policy_version 682014 (0.0010) [2023-12-27 10:20:51,242][20347] Updated weights for policy 0, policy_version 682024 (0.0010) [2023-12-27 10:20:51,512][20381] Updated weights for policy 1, policy_version 681529 (0.0011) [2023-12-27 10:20:51,560][20381] Updated weights for policy 1, policy_version 681539 (0.0010) [2023-12-27 10:20:51,607][20381] Updated weights for policy 1, policy_version 681549 (0.0009) [2023-12-27 10:20:51,665][20381] Updated weights for policy 1, policy_version 681559 (0.0009) [2023-12-27 10:20:51,947][20347] Updated weights for policy 0, policy_version 682034 (0.0011) [2023-12-27 10:20:51,999][20347] Updated weights for policy 0, policy_version 682044 (0.0010) [2023-12-27 10:20:52,059][20347] Updated weights for policy 0, policy_version 682054 (0.0010) [2023-12-27 10:20:52,118][20347] Updated weights for policy 0, policy_version 682064 (0.0011) [2023-12-27 10:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 349134848. Throughput: 0: 10214.0, 1: 10623.6. Samples: 349107792. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:52,363][19170] Avg episode reward: [(0, '483.010'), (1, '972.580')] [2023-12-27 10:20:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000682064_174628864.pth... [2023-12-27 10:20:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000680848_174317568.pth [2023-12-27 10:20:52,386][20381] Updated weights for policy 1, policy_version 681569 (0.0009) [2023-12-27 10:20:52,441][20381] Updated weights for policy 1, policy_version 681579 (0.0008) [2023-12-27 10:20:52,494][20381] Updated weights for policy 1, policy_version 681589 (0.0009) [2023-12-27 10:20:52,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000681592_174514176.pth... [2023-12-27 10:20:52,513][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000680344_174194688.pth [2023-12-27 10:20:52,802][20347] Updated weights for policy 0, policy_version 682074 (0.0005) [2023-12-27 10:20:52,850][20347] Updated weights for policy 0, policy_version 682084 (0.0005) [2023-12-27 10:20:52,906][20347] Updated weights for policy 0, policy_version 682094 (0.0005) [2023-12-27 10:20:53,207][20381] Updated weights for policy 1, policy_version 681599 (0.0008) [2023-12-27 10:20:53,260][20381] Updated weights for policy 1, policy_version 681609 (0.0008) [2023-12-27 10:20:53,316][20381] Updated weights for policy 1, policy_version 681619 (0.0008) [2023-12-27 10:20:53,477][20347] Updated weights for policy 0, policy_version 682104 (0.0007) [2023-12-27 10:20:53,527][20347] Updated weights for policy 0, policy_version 682114 (0.0007) [2023-12-27 10:20:53,578][20347] Updated weights for policy 0, policy_version 682124 (0.0007) [2023-12-27 10:20:54,015][20381] Updated weights for policy 1, policy_version 681629 (0.0009) [2023-12-27 10:20:54,067][20381] Updated weights for policy 1, policy_version 681639 (0.0008) [2023-12-27 10:20:54,118][20381] Updated weights for policy 1, policy_version 681649 (0.0006) [2023-12-27 10:20:54,278][20347] Updated weights for policy 0, policy_version 682134 (0.0007) [2023-12-27 10:20:54,336][20347] Updated weights for policy 0, policy_version 682144 (0.0007) [2023-12-27 10:20:54,396][20347] Updated weights for policy 0, policy_version 682154 (0.0008) [2023-12-27 10:20:54,793][20381] Updated weights for policy 1, policy_version 681659 (0.0006) [2023-12-27 10:20:54,846][20381] Updated weights for policy 1, policy_version 681669 (0.0005) [2023-12-27 10:20:54,899][20381] Updated weights for policy 1, policy_version 681679 (0.0005) [2023-12-27 10:20:54,926][20292] KL-divergence is very high: 120.6840 [2023-12-27 10:20:55,103][20347] Updated weights for policy 0, policy_version 682164 (0.0007) [2023-12-27 10:20:55,163][20347] Updated weights for policy 0, policy_version 682174 (0.0007) [2023-12-27 10:20:55,224][20347] Updated weights for policy 0, policy_version 682184 (0.0009) [2023-12-27 10:20:55,549][20292] KL-divergence is very high: 112.2376 [2023-12-27 10:20:55,555][20381] Updated weights for policy 1, policy_version 681689 (0.0006) [2023-12-27 10:20:55,618][20381] Updated weights for policy 1, policy_version 681699 (0.0008) [2023-12-27 10:20:55,651][20292] KL-divergence is very high: 122.8106 [2023-12-27 10:20:55,684][20381] Updated weights for policy 1, policy_version 681709 (0.0008) [2023-12-27 10:20:55,749][20381] Updated weights for policy 1, policy_version 681719 (0.0008) [2023-12-27 10:20:55,908][20347] Updated weights for policy 0, policy_version 682194 (0.0007) [2023-12-27 10:20:55,957][20347] Updated weights for policy 0, policy_version 682204 (0.0006) [2023-12-27 10:20:56,001][20347] Updated weights for policy 0, policy_version 682214 (0.0010) [2023-12-27 10:20:56,061][20347] Updated weights for policy 0, policy_version 682224 (0.0010) [2023-12-27 10:20:56,385][20381] Updated weights for policy 1, policy_version 681729 (0.0007) [2023-12-27 10:20:56,435][20381] Updated weights for policy 1, policy_version 681739 (0.0008) [2023-12-27 10:20:56,492][20381] Updated weights for policy 1, policy_version 681749 (0.0008) [2023-12-27 10:20:56,740][20347] Updated weights for policy 0, policy_version 682234 (0.0005) [2023-12-27 10:20:56,799][20347] Updated weights for policy 0, policy_version 682244 (0.0005) [2023-12-27 10:20:56,844][20347] Updated weights for policy 0, policy_version 682254 (0.0005) [2023-12-27 10:20:57,204][20381] Updated weights for policy 1, policy_version 681759 (0.0007) [2023-12-27 10:20:57,258][20381] Updated weights for policy 1, policy_version 681769 (0.0007) [2023-12-27 10:20:57,306][20381] Updated weights for policy 1, policy_version 681779 (0.0008) [2023-12-27 10:20:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 349241344. Throughput: 0: 10283.3, 1: 10519.1. Samples: 349229300. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:20:57,363][19170] Avg episode reward: [(0, '332.460'), (1, '962.690')] [2023-12-27 10:20:57,364][20347] Updated weights for policy 0, policy_version 682264 (0.0009) [2023-12-27 10:20:57,418][20347] Updated weights for policy 0, policy_version 682274 (0.0010) [2023-12-27 10:20:57,478][20347] Updated weights for policy 0, policy_version 682284 (0.0010) [2023-12-27 10:20:57,990][20381] Updated weights for policy 1, policy_version 681789 (0.0007) [2023-12-27 10:20:58,045][20381] Updated weights for policy 1, policy_version 681799 (0.0005) [2023-12-27 10:20:58,098][20381] Updated weights for policy 1, policy_version 681809 (0.0005) [2023-12-27 10:20:58,101][20347] Updated weights for policy 0, policy_version 682294 (0.0010) [2023-12-27 10:20:58,150][20347] Updated weights for policy 0, policy_version 682304 (0.0010) [2023-12-27 10:20:58,206][20347] Updated weights for policy 0, policy_version 682314 (0.0010) [2023-12-27 10:20:58,788][20381] Updated weights for policy 1, policy_version 681819 (0.0008) [2023-12-27 10:20:58,852][20381] Updated weights for policy 1, policy_version 681829 (0.0007) [2023-12-27 10:20:58,916][20381] Updated weights for policy 1, policy_version 681839 (0.0008) [2023-12-27 10:20:58,945][20347] Updated weights for policy 0, policy_version 682324 (0.0010) [2023-12-27 10:20:59,000][20347] Updated weights for policy 0, policy_version 682334 (0.0006) [2023-12-27 10:20:59,059][20347] Updated weights for policy 0, policy_version 682344 (0.0007) [2023-12-27 10:20:59,632][20381] Updated weights for policy 1, policy_version 681849 (0.0007) [2023-12-27 10:20:59,689][20381] Updated weights for policy 1, policy_version 681859 (0.0005) [2023-12-27 10:20:59,744][20381] Updated weights for policy 1, policy_version 681869 (0.0006) [2023-12-27 10:20:59,762][20347] Updated weights for policy 0, policy_version 682354 (0.0006) [2023-12-27 10:20:59,800][20381] Updated weights for policy 1, policy_version 681879 (0.0006) [2023-12-27 10:20:59,813][20347] Updated weights for policy 0, policy_version 682364 (0.0006) [2023-12-27 10:20:59,868][20347] Updated weights for policy 0, policy_version 682374 (0.0008) [2023-12-27 10:20:59,928][20347] Updated weights for policy 0, policy_version 682384 (0.0008) [2023-12-27 10:21:00,481][20381] Updated weights for policy 1, policy_version 681889 (0.0005) [2023-12-27 10:21:00,530][20381] Updated weights for policy 1, policy_version 681899 (0.0005) [2023-12-27 10:21:00,571][20347] Updated weights for policy 0, policy_version 682394 (0.0005) [2023-12-27 10:21:00,577][20381] Updated weights for policy 1, policy_version 681909 (0.0007) [2023-12-27 10:21:00,622][20347] Updated weights for policy 0, policy_version 682404 (0.0006) [2023-12-27 10:21:00,680][20347] Updated weights for policy 0, policy_version 682414 (0.0007) [2023-12-27 10:21:01,251][20381] Updated weights for policy 1, policy_version 681919 (0.0010) [2023-12-27 10:21:01,307][20381] Updated weights for policy 1, policy_version 681929 (0.0011) [2023-12-27 10:21:01,331][20347] Updated weights for policy 0, policy_version 682424 (0.0006) [2023-12-27 10:21:01,371][20381] Updated weights for policy 1, policy_version 681939 (0.0012) [2023-12-27 10:21:01,390][20347] Updated weights for policy 0, policy_version 682434 (0.0008) [2023-12-27 10:21:01,439][20347] Updated weights for policy 0, policy_version 682444 (0.0008) [2023-12-27 10:21:01,985][20381] Updated weights for policy 1, policy_version 681949 (0.0009) [2023-12-27 10:21:02,048][20381] Updated weights for policy 1, policy_version 681959 (0.0010) [2023-12-27 10:21:02,073][20347] Updated weights for policy 0, policy_version 682454 (0.0007) [2023-12-27 10:21:02,094][20381] Updated weights for policy 1, policy_version 681969 (0.0007) [2023-12-27 10:21:02,130][20347] Updated weights for policy 0, policy_version 682464 (0.0008) [2023-12-27 10:21:02,185][20347] Updated weights for policy 0, policy_version 682474 (0.0007) [2023-12-27 10:21:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 349347840. Throughput: 0: 10364.4, 1: 10470.2. Samples: 349354064. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:02,363][19170] Avg episode reward: [(0, '272.100'), (1, '952.360')] [2023-12-27 10:21:02,709][20381] Updated weights for policy 1, policy_version 681979 (0.0010) [2023-12-27 10:21:02,765][20381] Updated weights for policy 1, policy_version 681989 (0.0011) [2023-12-27 10:21:02,818][20381] Updated weights for policy 1, policy_version 681999 (0.0010) [2023-12-27 10:21:02,821][20347] Updated weights for policy 0, policy_version 682484 (0.0006) [2023-12-27 10:21:02,869][20347] Updated weights for policy 0, policy_version 682494 (0.0005) [2023-12-27 10:21:02,944][20347] Updated weights for policy 0, policy_version 682504 (0.0005) [2023-12-27 10:21:03,486][20381] Updated weights for policy 1, policy_version 682009 (0.0009) [2023-12-27 10:21:03,550][20381] Updated weights for policy 1, policy_version 682019 (0.0005) [2023-12-27 10:21:03,580][20347] Updated weights for policy 0, policy_version 682514 (0.0007) [2023-12-27 10:21:03,607][20381] Updated weights for policy 1, policy_version 682029 (0.0006) [2023-12-27 10:21:03,632][20347] Updated weights for policy 0, policy_version 682524 (0.0007) [2023-12-27 10:21:03,657][20381] Updated weights for policy 1, policy_version 682039 (0.0005) [2023-12-27 10:21:03,689][20347] Updated weights for policy 0, policy_version 682534 (0.0008) [2023-12-27 10:21:03,740][20347] Updated weights for policy 0, policy_version 682544 (0.0008) [2023-12-27 10:21:04,269][20381] Updated weights for policy 1, policy_version 682049 (0.0009) [2023-12-27 10:21:04,331][20381] Updated weights for policy 1, policy_version 682059 (0.0008) [2023-12-27 10:21:04,398][20381] Updated weights for policy 1, policy_version 682069 (0.0009) [2023-12-27 10:21:04,460][20347] Updated weights for policy 0, policy_version 682554 (0.0011) [2023-12-27 10:21:04,519][20347] Updated weights for policy 0, policy_version 682564 (0.0010) [2023-12-27 10:21:04,579][20347] Updated weights for policy 0, policy_version 682574 (0.0010) [2023-12-27 10:21:05,066][20381] Updated weights for policy 1, policy_version 682079 (0.0009) [2023-12-27 10:21:05,124][20381] Updated weights for policy 1, policy_version 682089 (0.0010) [2023-12-27 10:21:05,185][20381] Updated weights for policy 1, policy_version 682099 (0.0011) [2023-12-27 10:21:05,253][20347] Updated weights for policy 0, policy_version 682584 (0.0007) [2023-12-27 10:21:05,314][20347] Updated weights for policy 0, policy_version 682594 (0.0007) [2023-12-27 10:21:05,380][20347] Updated weights for policy 0, policy_version 682604 (0.0008) [2023-12-27 10:21:05,795][20381] Updated weights for policy 1, policy_version 682109 (0.0008) [2023-12-27 10:21:05,851][20381] Updated weights for policy 1, policy_version 682119 (0.0005) [2023-12-27 10:21:05,913][20381] Updated weights for policy 1, policy_version 682129 (0.0007) [2023-12-27 10:21:06,046][20347] Updated weights for policy 0, policy_version 682614 (0.0007) [2023-12-27 10:21:06,099][20347] Updated weights for policy 0, policy_version 682624 (0.0006) [2023-12-27 10:21:06,155][20347] Updated weights for policy 0, policy_version 682634 (0.0007) [2023-12-27 10:21:06,582][20381] Updated weights for policy 1, policy_version 682139 (0.0011) [2023-12-27 10:21:06,639][20381] Updated weights for policy 1, policy_version 682149 (0.0010) [2023-12-27 10:21:06,702][20381] Updated weights for policy 1, policy_version 682159 (0.0011) [2023-12-27 10:21:06,838][20347] Updated weights for policy 0, policy_version 682644 (0.0007) [2023-12-27 10:21:06,892][20347] Updated weights for policy 0, policy_version 682654 (0.0007) [2023-12-27 10:21:06,952][20347] Updated weights for policy 0, policy_version 682664 (0.0007) [2023-12-27 10:21:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 349446144. Throughput: 0: 10410.9, 1: 10473.6. Samples: 349416872. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:07,363][19170] Avg episode reward: [(0, '363.850'), (1, '922.500')] [2023-12-27 10:21:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000682168_174661632.pth... [2023-12-27 10:21:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000682672_174784512.pth... [2023-12-27 10:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000680952_174350336.pth [2023-12-27 10:21:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000681488_174481408.pth [2023-12-27 10:21:07,390][20381] Updated weights for policy 1, policy_version 682169 (0.0011) [2023-12-27 10:21:07,436][20381] Updated weights for policy 1, policy_version 682179 (0.0010) [2023-12-27 10:21:07,489][20381] Updated weights for policy 1, policy_version 682189 (0.0007) [2023-12-27 10:21:07,552][20381] Updated weights for policy 1, policy_version 682199 (0.0007) [2023-12-27 10:21:07,653][20347] Updated weights for policy 0, policy_version 682674 (0.0007) [2023-12-27 10:21:07,702][20347] Updated weights for policy 0, policy_version 682684 (0.0008) [2023-12-27 10:21:07,761][20347] Updated weights for policy 0, policy_version 682694 (0.0008) [2023-12-27 10:21:07,821][20347] Updated weights for policy 0, policy_version 682704 (0.0007) [2023-12-27 10:21:08,172][20381] Updated weights for policy 1, policy_version 682209 (0.0007) [2023-12-27 10:21:08,231][20381] Updated weights for policy 1, policy_version 682219 (0.0010) [2023-12-27 10:21:08,291][20381] Updated weights for policy 1, policy_version 682229 (0.0008) [2023-12-27 10:21:08,541][20347] Updated weights for policy 0, policy_version 682714 (0.0007) [2023-12-27 10:21:08,592][20347] Updated weights for policy 0, policy_version 682724 (0.0007) [2023-12-27 10:21:08,647][20347] Updated weights for policy 0, policy_version 682734 (0.0007) [2023-12-27 10:21:08,917][20381] Updated weights for policy 1, policy_version 682239 (0.0007) [2023-12-27 10:21:08,967][20381] Updated weights for policy 1, policy_version 682249 (0.0007) [2023-12-27 10:21:09,014][20381] Updated weights for policy 1, policy_version 682259 (0.0010) [2023-12-27 10:21:09,409][20347] Updated weights for policy 0, policy_version 682744 (0.0009) [2023-12-27 10:21:09,468][20347] Updated weights for policy 0, policy_version 682754 (0.0008) [2023-12-27 10:21:09,522][20347] Updated weights for policy 0, policy_version 682764 (0.0009) [2023-12-27 10:21:09,648][20381] Updated weights for policy 1, policy_version 682269 (0.0009) [2023-12-27 10:21:09,707][20381] Updated weights for policy 1, policy_version 682279 (0.0007) [2023-12-27 10:21:09,760][20381] Updated weights for policy 1, policy_version 682289 (0.0007) [2023-12-27 10:21:10,254][20347] Updated weights for policy 0, policy_version 682774 (0.0008) [2023-12-27 10:21:10,323][20347] Updated weights for policy 0, policy_version 682784 (0.0009) [2023-12-27 10:21:10,390][20347] Updated weights for policy 0, policy_version 682794 (0.0008) [2023-12-27 10:21:10,425][20381] Updated weights for policy 1, policy_version 682299 (0.0008) [2023-12-27 10:21:10,487][20381] Updated weights for policy 1, policy_version 682309 (0.0005) [2023-12-27 10:21:10,549][20381] Updated weights for policy 1, policy_version 682319 (0.0005) [2023-12-27 10:21:11,107][20347] Updated weights for policy 0, policy_version 682804 (0.0008) [2023-12-27 10:21:11,119][20381] Updated weights for policy 1, policy_version 682329 (0.0007) [2023-12-27 10:21:11,175][20347] Updated weights for policy 0, policy_version 682814 (0.0007) [2023-12-27 10:21:11,186][20381] Updated weights for policy 1, policy_version 682339 (0.0008) [2023-12-27 10:21:11,240][20347] Updated weights for policy 0, policy_version 682824 (0.0007) [2023-12-27 10:21:11,247][20381] Updated weights for policy 1, policy_version 682349 (0.0007) [2023-12-27 10:21:11,311][20381] Updated weights for policy 1, policy_version 682359 (0.0007) [2023-12-27 10:21:11,981][20381] Updated weights for policy 1, policy_version 682369 (0.0007) [2023-12-27 10:21:12,000][20347] Updated weights for policy 0, policy_version 682834 (0.0008) [2023-12-27 10:21:12,039][20381] Updated weights for policy 1, policy_version 682379 (0.0007) [2023-12-27 10:21:12,062][20347] Updated weights for policy 0, policy_version 682844 (0.0007) [2023-12-27 10:21:12,102][20381] Updated weights for policy 1, policy_version 682389 (0.0007) [2023-12-27 10:21:12,129][20347] Updated weights for policy 0, policy_version 682854 (0.0007) [2023-12-27 10:21:12,190][20347] Updated weights for policy 0, policy_version 682864 (0.0008) [2023-12-27 10:21:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 349552640. Throughput: 0: 10347.3, 1: 10480.4. Samples: 349540272. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:12,363][19170] Avg episode reward: [(0, '400.230'), (1, '902.810')] [2023-12-27 10:21:12,807][20381] Updated weights for policy 1, policy_version 682399 (0.0006) [2023-12-27 10:21:12,832][20347] Updated weights for policy 0, policy_version 682874 (0.0010) [2023-12-27 10:21:12,860][20381] Updated weights for policy 1, policy_version 682409 (0.0005) [2023-12-27 10:21:12,888][20347] Updated weights for policy 0, policy_version 682884 (0.0010) [2023-12-27 10:21:12,912][20381] Updated weights for policy 1, policy_version 682419 (0.0006) [2023-12-27 10:21:12,941][20347] Updated weights for policy 0, policy_version 682894 (0.0010) [2023-12-27 10:21:13,576][20381] Updated weights for policy 1, policy_version 682429 (0.0007) [2023-12-27 10:21:13,625][20381] Updated weights for policy 1, policy_version 682439 (0.0006) [2023-12-27 10:21:13,626][20347] Updated weights for policy 0, policy_version 682904 (0.0010) [2023-12-27 10:21:13,672][20347] Updated weights for policy 0, policy_version 682914 (0.0010) [2023-12-27 10:21:13,677][20381] Updated weights for policy 1, policy_version 682449 (0.0005) [2023-12-27 10:21:13,729][20347] Updated weights for policy 0, policy_version 682924 (0.0010) [2023-12-27 10:21:14,312][20381] Updated weights for policy 1, policy_version 682459 (0.0006) [2023-12-27 10:21:14,369][20381] Updated weights for policy 1, policy_version 682469 (0.0006) [2023-12-27 10:21:14,370][20347] Updated weights for policy 0, policy_version 682934 (0.0011) [2023-12-27 10:21:14,422][20347] Updated weights for policy 0, policy_version 682944 (0.0011) [2023-12-27 10:21:14,426][20381] Updated weights for policy 1, policy_version 682479 (0.0006) [2023-12-27 10:21:14,474][20347] Updated weights for policy 0, policy_version 682954 (0.0010) [2023-12-27 10:21:15,069][20381] Updated weights for policy 1, policy_version 682489 (0.0008) [2023-12-27 10:21:15,118][20292] KL-divergence is very high: 118.9189 [2023-12-27 10:21:15,125][20292] KL-divergence is very high: 216.4180 [2023-12-27 10:21:15,136][20381] Updated weights for policy 1, policy_version 682499 (0.0008) [2023-12-27 10:21:15,153][20347] Updated weights for policy 0, policy_version 682964 (0.0010) [2023-12-27 10:21:15,197][20381] Updated weights for policy 1, policy_version 682509 (0.0005) [2023-12-27 10:21:15,205][20347] Updated weights for policy 0, policy_version 682974 (0.0010) [2023-12-27 10:21:15,254][20347] Updated weights for policy 0, policy_version 682984 (0.0010) [2023-12-27 10:21:15,256][20381] Updated weights for policy 1, policy_version 682519 (0.0006) [2023-12-27 10:21:15,899][20381] Updated weights for policy 1, policy_version 682529 (0.0007) [2023-12-27 10:21:15,942][20347] Updated weights for policy 0, policy_version 682994 (0.0010) [2023-12-27 10:21:15,945][20381] Updated weights for policy 1, policy_version 682539 (0.0005) [2023-12-27 10:21:15,998][20381] Updated weights for policy 1, policy_version 682549 (0.0006) [2023-12-27 10:21:16,003][20347] Updated weights for policy 0, policy_version 683004 (0.0006) [2023-12-27 10:21:16,058][20347] Updated weights for policy 0, policy_version 683014 (0.0005) [2023-12-27 10:21:16,107][20347] Updated weights for policy 0, policy_version 683024 (0.0005) [2023-12-27 10:21:16,606][20381] Updated weights for policy 1, policy_version 682559 (0.0007) [2023-12-27 10:21:16,663][20381] Updated weights for policy 1, policy_version 682569 (0.0008) [2023-12-27 10:21:16,685][20347] Updated weights for policy 0, policy_version 683034 (0.0005) [2023-12-27 10:21:16,720][20381] Updated weights for policy 1, policy_version 682579 (0.0007) [2023-12-27 10:21:16,737][20347] Updated weights for policy 0, policy_version 683044 (0.0010) [2023-12-27 10:21:16,784][20347] Updated weights for policy 0, policy_version 683054 (0.0010) [2023-12-27 10:21:17,320][20381] Updated weights for policy 1, policy_version 682589 (0.0007) [2023-12-27 10:21:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 349650944. Throughput: 0: 10253.8, 1: 10495.6. Samples: 349666288. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:17,363][19170] Avg episode reward: [(0, '351.370'), (1, '809.120')] [2023-12-27 10:21:17,374][20381] Updated weights for policy 1, policy_version 682599 (0.0008) [2023-12-27 10:21:17,423][20381] Updated weights for policy 1, policy_version 682609 (0.0007) [2023-12-27 10:21:17,460][20347] Updated weights for policy 0, policy_version 683064 (0.0010) [2023-12-27 10:21:17,512][20347] Updated weights for policy 0, policy_version 683074 (0.0010) [2023-12-27 10:21:17,567][20347] Updated weights for policy 0, policy_version 683084 (0.0010) [2023-12-27 10:21:18,112][20381] Updated weights for policy 1, policy_version 682619 (0.0008) [2023-12-27 10:21:18,173][20381] Updated weights for policy 1, policy_version 682629 (0.0008) [2023-12-27 10:21:18,211][20347] Updated weights for policy 0, policy_version 683094 (0.0010) [2023-12-27 10:21:18,228][20381] Updated weights for policy 1, policy_version 682639 (0.0005) [2023-12-27 10:21:18,266][20347] Updated weights for policy 0, policy_version 683104 (0.0010) [2023-12-27 10:21:18,325][20347] Updated weights for policy 0, policy_version 683114 (0.0010) [2023-12-27 10:21:18,819][20381] Updated weights for policy 1, policy_version 682649 (0.0005) [2023-12-27 10:21:18,881][20381] Updated weights for policy 1, policy_version 682659 (0.0007) [2023-12-27 10:21:18,937][20381] Updated weights for policy 1, policy_version 682669 (0.0008) [2023-12-27 10:21:18,994][20381] Updated weights for policy 1, policy_version 682679 (0.0006) [2023-12-27 10:21:19,019][20347] Updated weights for policy 0, policy_version 683124 (0.0008) [2023-12-27 10:21:19,080][20347] Updated weights for policy 0, policy_version 683134 (0.0007) [2023-12-27 10:21:19,142][20347] Updated weights for policy 0, policy_version 683144 (0.0008) [2023-12-27 10:21:19,615][20381] Updated weights for policy 1, policy_version 682689 (0.0008) [2023-12-27 10:21:19,676][20381] Updated weights for policy 1, policy_version 682699 (0.0009) [2023-12-27 10:21:19,743][20381] Updated weights for policy 1, policy_version 682709 (0.0008) [2023-12-27 10:21:19,824][20347] Updated weights for policy 0, policy_version 683154 (0.0008) [2023-12-27 10:21:19,885][20347] Updated weights for policy 0, policy_version 683164 (0.0007) [2023-12-27 10:21:19,948][20347] Updated weights for policy 0, policy_version 683174 (0.0006) [2023-12-27 10:21:20,015][20347] Updated weights for policy 0, policy_version 683184 (0.0006) [2023-12-27 10:21:20,487][20381] Updated weights for policy 1, policy_version 682719 (0.0007) [2023-12-27 10:21:20,539][20381] Updated weights for policy 1, policy_version 682729 (0.0008) [2023-12-27 10:21:20,602][20381] Updated weights for policy 1, policy_version 682739 (0.0008) [2023-12-27 10:21:20,636][20347] Updated weights for policy 0, policy_version 683194 (0.0005) [2023-12-27 10:21:20,690][20347] Updated weights for policy 0, policy_version 683204 (0.0005) [2023-12-27 10:21:20,752][20347] Updated weights for policy 0, policy_version 683214 (0.0006) [2023-12-27 10:21:21,344][20381] Updated weights for policy 1, policy_version 682749 (0.0007) [2023-12-27 10:21:21,388][20347] Updated weights for policy 0, policy_version 683224 (0.0007) [2023-12-27 10:21:21,415][20381] Updated weights for policy 1, policy_version 682759 (0.0007) [2023-12-27 10:21:21,450][20347] Updated weights for policy 0, policy_version 683234 (0.0008) [2023-12-27 10:21:21,474][20381] Updated weights for policy 1, policy_version 682769 (0.0006) [2023-12-27 10:21:21,500][20347] Updated weights for policy 0, policy_version 683244 (0.0008) [2023-12-27 10:21:22,118][20381] Updated weights for policy 1, policy_version 682779 (0.0007) [2023-12-27 10:21:22,175][20381] Updated weights for policy 1, policy_version 682789 (0.0008) [2023-12-27 10:21:22,203][20347] Updated weights for policy 0, policy_version 683254 (0.0008) [2023-12-27 10:21:22,229][20381] Updated weights for policy 1, policy_version 682799 (0.0006) [2023-12-27 10:21:22,254][20347] Updated weights for policy 0, policy_version 683264 (0.0007) [2023-12-27 10:21:22,314][20347] Updated weights for policy 0, policy_version 683274 (0.0006) [2023-12-27 10:21:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 349765632. Throughput: 0: 10218.2, 1: 10470.7. Samples: 349728244. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:22,363][19170] Avg episode reward: [(0, '425.320'), (1, '770.690')] [2023-12-27 10:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000683280_174940160.pth... [2023-12-27 10:21:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000682808_174825472.pth... [2023-12-27 10:21:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000682064_174628864.pth [2023-12-27 10:21:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000681592_174514176.pth [2023-12-27 10:21:22,942][20381] Updated weights for policy 1, policy_version 682809 (0.0010) [2023-12-27 10:21:23,002][20381] Updated weights for policy 1, policy_version 682819 (0.0010) [2023-12-27 10:21:23,037][20347] Updated weights for policy 0, policy_version 683284 (0.0007) [2023-12-27 10:21:23,058][20381] Updated weights for policy 1, policy_version 682829 (0.0010) [2023-12-27 10:21:23,099][20347] Updated weights for policy 0, policy_version 683294 (0.0008) [2023-12-27 10:21:23,107][20381] Updated weights for policy 1, policy_version 682839 (0.0010) [2023-12-27 10:21:23,155][20347] Updated weights for policy 0, policy_version 683304 (0.0007) [2023-12-27 10:21:23,749][20381] Updated weights for policy 1, policy_version 682849 (0.0010) [2023-12-27 10:21:23,806][20381] Updated weights for policy 1, policy_version 682859 (0.0010) [2023-12-27 10:21:23,810][20347] Updated weights for policy 0, policy_version 683314 (0.0008) [2023-12-27 10:21:23,860][20347] Updated weights for policy 0, policy_version 683324 (0.0005) [2023-12-27 10:21:23,861][20381] Updated weights for policy 1, policy_version 682869 (0.0010) [2023-12-27 10:21:23,917][20347] Updated weights for policy 0, policy_version 683334 (0.0007) [2023-12-27 10:21:23,972][20347] Updated weights for policy 0, policy_version 683344 (0.0007) [2023-12-27 10:21:24,504][20381] Updated weights for policy 1, policy_version 682879 (0.0010) [2023-12-27 10:21:24,559][20381] Updated weights for policy 1, policy_version 682889 (0.0010) [2023-12-27 10:21:24,613][20381] Updated weights for policy 1, policy_version 682899 (0.0010) [2023-12-27 10:21:24,683][20347] Updated weights for policy 0, policy_version 683354 (0.0007) [2023-12-27 10:21:24,729][20347] Updated weights for policy 0, policy_version 683364 (0.0007) [2023-12-27 10:21:24,771][20347] Updated weights for policy 0, policy_version 683374 (0.0006) [2023-12-27 10:21:25,296][20381] Updated weights for policy 1, policy_version 682909 (0.0011) [2023-12-27 10:21:25,349][20381] Updated weights for policy 1, policy_version 682919 (0.0011) [2023-12-27 10:21:25,407][20381] Updated weights for policy 1, policy_version 682929 (0.0010) [2023-12-27 10:21:25,446][20347] Updated weights for policy 0, policy_version 683384 (0.0005) [2023-12-27 10:21:25,497][20347] Updated weights for policy 0, policy_version 683394 (0.0007) [2023-12-27 10:21:25,552][20347] Updated weights for policy 0, policy_version 683404 (0.0007) [2023-12-27 10:21:26,114][20381] Updated weights for policy 1, policy_version 682939 (0.0010) [2023-12-27 10:21:26,172][20381] Updated weights for policy 1, policy_version 682949 (0.0008) [2023-12-27 10:21:26,218][20381] Updated weights for policy 1, policy_version 682959 (0.0008) [2023-12-27 10:21:26,270][20347] Updated weights for policy 0, policy_version 683414 (0.0006) [2023-12-27 10:21:26,316][20347] Updated weights for policy 0, policy_version 683424 (0.0007) [2023-12-27 10:21:26,372][20347] Updated weights for policy 0, policy_version 683434 (0.0008) [2023-12-27 10:21:26,894][20381] Updated weights for policy 1, policy_version 682969 (0.0007) [2023-12-27 10:21:26,944][20381] Updated weights for policy 1, policy_version 682979 (0.0008) [2023-12-27 10:21:26,997][20381] Updated weights for policy 1, policy_version 682989 (0.0008) [2023-12-27 10:21:27,044][20381] Updated weights for policy 1, policy_version 682999 (0.0008) [2023-12-27 10:21:27,046][20347] Updated weights for policy 0, policy_version 683444 (0.0008) [2023-12-27 10:21:27,096][20347] Updated weights for policy 0, policy_version 683454 (0.0006) [2023-12-27 10:21:27,140][20347] Updated weights for policy 0, policy_version 683464 (0.0007) [2023-12-27 10:21:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 349863936. Throughput: 0: 10243.8, 1: 10438.1. Samples: 349850348. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:27,363][19170] Avg episode reward: [(0, '423.880'), (1, '835.170')] [2023-12-27 10:21:27,739][20347] Updated weights for policy 0, policy_version 683474 (0.0007) [2023-12-27 10:21:27,752][20381] Updated weights for policy 1, policy_version 683009 (0.0006) [2023-12-27 10:21:27,789][20347] Updated weights for policy 0, policy_version 683484 (0.0007) [2023-12-27 10:21:27,809][20381] Updated weights for policy 1, policy_version 683019 (0.0006) [2023-12-27 10:21:27,842][20347] Updated weights for policy 0, policy_version 683494 (0.0008) [2023-12-27 10:21:27,863][20381] Updated weights for policy 1, policy_version 683029 (0.0008) [2023-12-27 10:21:27,897][20347] Updated weights for policy 0, policy_version 683504 (0.0007) [2023-12-27 10:21:28,563][20347] Updated weights for policy 0, policy_version 683514 (0.0006) [2023-12-27 10:21:28,569][20381] Updated weights for policy 1, policy_version 683039 (0.0006) [2023-12-27 10:21:28,614][20347] Updated weights for policy 0, policy_version 683524 (0.0006) [2023-12-27 10:21:28,617][20381] Updated weights for policy 1, policy_version 683049 (0.0007) [2023-12-27 10:21:28,663][20381] Updated weights for policy 1, policy_version 683059 (0.0008) [2023-12-27 10:21:28,666][20347] Updated weights for policy 0, policy_version 683534 (0.0005) [2023-12-27 10:21:29,283][20347] Updated weights for policy 0, policy_version 683544 (0.0008) [2023-12-27 10:21:29,341][20347] Updated weights for policy 0, policy_version 683554 (0.0006) [2023-12-27 10:21:29,368][20381] Updated weights for policy 1, policy_version 683069 (0.0008) [2023-12-27 10:21:29,404][20347] Updated weights for policy 0, policy_version 683564 (0.0008) [2023-12-27 10:21:29,422][20381] Updated weights for policy 1, policy_version 683079 (0.0005) [2023-12-27 10:21:29,470][20381] Updated weights for policy 1, policy_version 683089 (0.0007) [2023-12-27 10:21:30,090][20381] Updated weights for policy 1, policy_version 683099 (0.0008) [2023-12-27 10:21:30,099][20347] Updated weights for policy 0, policy_version 683574 (0.0007) [2023-12-27 10:21:30,137][20381] Updated weights for policy 1, policy_version 683109 (0.0006) [2023-12-27 10:21:30,150][20347] Updated weights for policy 0, policy_version 683584 (0.0008) [2023-12-27 10:21:30,198][20381] Updated weights for policy 1, policy_version 683119 (0.0006) [2023-12-27 10:21:30,208][20347] Updated weights for policy 0, policy_version 683594 (0.0008) [2023-12-27 10:21:30,827][20347] Updated weights for policy 0, policy_version 683604 (0.0008) [2023-12-27 10:21:30,835][20381] Updated weights for policy 1, policy_version 683129 (0.0006) [2023-12-27 10:21:30,885][20347] Updated weights for policy 0, policy_version 683614 (0.0006) [2023-12-27 10:21:30,887][20381] Updated weights for policy 1, policy_version 683139 (0.0009) [2023-12-27 10:21:30,943][20347] Updated weights for policy 0, policy_version 683624 (0.0008) [2023-12-27 10:21:30,951][20381] Updated weights for policy 1, policy_version 683149 (0.0006) [2023-12-27 10:21:31,011][20381] Updated weights for policy 1, policy_version 683159 (0.0009) [2023-12-27 10:21:31,628][20347] Updated weights for policy 0, policy_version 683634 (0.0010) [2023-12-27 10:21:31,685][20347] Updated weights for policy 0, policy_version 683644 (0.0008) [2023-12-27 10:21:31,692][20381] Updated weights for policy 1, policy_version 683169 (0.0008) [2023-12-27 10:21:31,747][20347] Updated weights for policy 0, policy_version 683654 (0.0007) [2023-12-27 10:21:31,751][20381] Updated weights for policy 1, policy_version 683179 (0.0009) [2023-12-27 10:21:31,796][20347] Updated weights for policy 0, policy_version 683664 (0.0005) [2023-12-27 10:21:31,804][20381] Updated weights for policy 1, policy_version 683189 (0.0008) [2023-12-27 10:21:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 349962240. Throughput: 0: 10303.4, 1: 10400.6. Samples: 349977036. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:32,363][19170] Avg episode reward: [(0, '515.320'), (1, '844.580')] [2023-12-27 10:21:32,373][20347] Updated weights for policy 0, policy_version 683674 (0.0007) [2023-12-27 10:21:32,439][20347] Updated weights for policy 0, policy_version 683684 (0.0006) [2023-12-27 10:21:32,474][20381] Updated weights for policy 1, policy_version 683199 (0.0007) [2023-12-27 10:21:32,496][20347] Updated weights for policy 0, policy_version 683694 (0.0008) [2023-12-27 10:21:32,523][20381] Updated weights for policy 1, policy_version 683209 (0.0005) [2023-12-27 10:21:32,578][20381] Updated weights for policy 1, policy_version 683219 (0.0006) [2023-12-27 10:21:33,103][20381] Updated weights for policy 1, policy_version 683229 (0.0005) [2023-12-27 10:21:33,155][20381] Updated weights for policy 1, policy_version 683239 (0.0005) [2023-12-27 10:21:33,193][20347] Updated weights for policy 0, policy_version 683704 (0.0008) [2023-12-27 10:21:33,213][20381] Updated weights for policy 1, policy_version 683249 (0.0005) [2023-12-27 10:21:33,250][20347] Updated weights for policy 0, policy_version 683714 (0.0008) [2023-12-27 10:21:33,301][20347] Updated weights for policy 0, policy_version 683724 (0.0009) [2023-12-27 10:21:33,725][20381] Updated weights for policy 1, policy_version 683259 (0.0006) [2023-12-27 10:21:33,796][20381] Updated weights for policy 1, policy_version 683269 (0.0007) [2023-12-27 10:21:33,844][20381] Updated weights for policy 1, policy_version 683279 (0.0005) [2023-12-27 10:21:34,006][20347] Updated weights for policy 0, policy_version 683734 (0.0008) [2023-12-27 10:21:34,062][20347] Updated weights for policy 0, policy_version 683744 (0.0008) [2023-12-27 10:21:34,121][20347] Updated weights for policy 0, policy_version 683754 (0.0007) [2023-12-27 10:21:34,421][20381] Updated weights for policy 1, policy_version 683289 (0.0005) [2023-12-27 10:21:34,481][20381] Updated weights for policy 1, policy_version 683299 (0.0008) [2023-12-27 10:21:34,526][20381] Updated weights for policy 1, policy_version 683309 (0.0008) [2023-12-27 10:21:34,576][20381] Updated weights for policy 1, policy_version 683319 (0.0008) [2023-12-27 10:21:34,817][20347] Updated weights for policy 0, policy_version 683764 (0.0007) [2023-12-27 10:21:34,879][20347] Updated weights for policy 0, policy_version 683774 (0.0007) [2023-12-27 10:21:34,932][20347] Updated weights for policy 0, policy_version 683784 (0.0008) [2023-12-27 10:21:35,297][20381] Updated weights for policy 1, policy_version 683329 (0.0007) [2023-12-27 10:21:35,355][20381] Updated weights for policy 1, policy_version 683339 (0.0006) [2023-12-27 10:21:35,407][20381] Updated weights for policy 1, policy_version 683349 (0.0007) [2023-12-27 10:21:35,638][20347] Updated weights for policy 0, policy_version 683794 (0.0008) [2023-12-27 10:21:35,694][20347] Updated weights for policy 0, policy_version 683804 (0.0008) [2023-12-27 10:21:35,748][20347] Updated weights for policy 0, policy_version 683814 (0.0006) [2023-12-27 10:21:35,802][20347] Updated weights for policy 0, policy_version 683824 (0.0006) [2023-12-27 10:21:36,069][20381] Updated weights for policy 1, policy_version 683359 (0.0007) [2023-12-27 10:21:36,130][20381] Updated weights for policy 1, policy_version 683369 (0.0008) [2023-12-27 10:21:36,198][20381] Updated weights for policy 1, policy_version 683379 (0.0007) [2023-12-27 10:21:36,526][20347] Updated weights for policy 0, policy_version 683834 (0.0008) [2023-12-27 10:21:36,582][20347] Updated weights for policy 0, policy_version 683844 (0.0008) [2023-12-27 10:21:36,645][20347] Updated weights for policy 0, policy_version 683854 (0.0008) [2023-12-27 10:21:36,904][20381] Updated weights for policy 1, policy_version 683389 (0.0007) [2023-12-27 10:21:36,952][20381] Updated weights for policy 1, policy_version 683399 (0.0007) [2023-12-27 10:21:37,003][20381] Updated weights for policy 1, policy_version 683409 (0.0005) [2023-12-27 10:21:37,360][20347] Updated weights for policy 0, policy_version 683864 (0.0008) [2023-12-27 10:21:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 350068736. Throughput: 0: 10278.4, 1: 10470.4. Samples: 350041488. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 10:21:37,363][19170] Avg episode reward: [(0, '593.420'), (1, '883.390')] [2023-12-27 10:21:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000683416_174981120.pth... [2023-12-27 10:21:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000682168_174661632.pth [2023-12-27 10:21:37,426][20347] Updated weights for policy 0, policy_version 683874 (0.0008) [2023-12-27 10:21:37,490][20347] Updated weights for policy 0, policy_version 683884 (0.0007) [2023-12-27 10:21:37,515][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000683888_175095808.pth... [2023-12-27 10:21:37,519][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000682672_174784512.pth [2023-12-27 10:21:37,695][20381] Updated weights for policy 1, policy_version 683419 (0.0006) [2023-12-27 10:21:37,752][20381] Updated weights for policy 1, policy_version 683429 (0.0008) [2023-12-27 10:21:37,807][20381] Updated weights for policy 1, policy_version 683439 (0.0005) [2023-12-27 10:21:38,098][20347] Updated weights for policy 0, policy_version 683894 (0.0005) [2023-12-27 10:21:38,149][20347] Updated weights for policy 0, policy_version 683904 (0.0007) [2023-12-27 10:21:38,205][20347] Updated weights for policy 0, policy_version 683914 (0.0005) [2023-12-27 10:21:38,486][20381] Updated weights for policy 1, policy_version 683449 (0.0006) [2023-12-27 10:21:38,545][20381] Updated weights for policy 1, policy_version 683459 (0.0007) [2023-12-27 10:21:38,601][20381] Updated weights for policy 1, policy_version 683469 (0.0007) [2023-12-27 10:21:38,657][20381] Updated weights for policy 1, policy_version 683479 (0.0008) [2023-12-27 10:21:38,817][20347] Updated weights for policy 0, policy_version 683924 (0.0006) [2023-12-27 10:21:38,877][20347] Updated weights for policy 0, policy_version 683934 (0.0006) [2023-12-27 10:21:38,935][20347] Updated weights for policy 0, policy_version 683944 (0.0005) [2023-12-27 10:21:39,423][20381] Updated weights for policy 1, policy_version 683489 (0.0008) [2023-12-27 10:21:39,483][20381] Updated weights for policy 1, policy_version 683499 (0.0008) [2023-12-27 10:21:39,546][20381] Updated weights for policy 1, policy_version 683509 (0.0008) [2023-12-27 10:21:39,580][20347] Updated weights for policy 0, policy_version 683954 (0.0006) [2023-12-27 10:21:39,636][20347] Updated weights for policy 0, policy_version 683964 (0.0008) [2023-12-27 10:21:39,696][20347] Updated weights for policy 0, policy_version 683974 (0.0008) [2023-12-27 10:21:39,750][20347] Updated weights for policy 0, policy_version 683984 (0.0008) [2023-12-27 10:21:40,270][20381] Updated weights for policy 1, policy_version 683519 (0.0008) [2023-12-27 10:21:40,320][20381] Updated weights for policy 1, policy_version 683529 (0.0008) [2023-12-27 10:21:40,384][20381] Updated weights for policy 1, policy_version 683539 (0.0008) [2023-12-27 10:21:40,429][20347] Updated weights for policy 0, policy_version 683994 (0.0007) [2023-12-27 10:21:40,488][20347] Updated weights for policy 0, policy_version 684004 (0.0008) [2023-12-27 10:21:40,510][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000010 [2023-12-27 10:21:41,064][20381] Updated weights for policy 1, policy_version 683549 (0.0007) [2023-12-27 10:21:41,120][20381] Updated weights for policy 1, policy_version 683559 (0.0007) [2023-12-27 10:21:41,180][20381] Updated weights for policy 1, policy_version 683569 (0.0007) [2023-12-27 10:21:41,254][20347] Updated weights for policy 0, policy_version 684014 (0.0008) [2023-12-27 10:21:41,312][20347] Updated weights for policy 0, policy_version 684024 (0.0007) [2023-12-27 10:21:41,373][20347] Updated weights for policy 0, policy_version 684034 (0.0008) [2023-12-27 10:21:41,868][20381] Updated weights for policy 1, policy_version 683579 (0.0007) [2023-12-27 10:21:41,920][20381] Updated weights for policy 1, policy_version 683589 (0.0006) [2023-12-27 10:21:41,969][20381] Updated weights for policy 1, policy_version 683599 (0.0006) [2023-12-27 10:21:42,099][20347] Updated weights for policy 0, policy_version 684044 (0.0008) [2023-12-27 10:21:42,158][20347] Updated weights for policy 0, policy_version 684054 (0.0006) [2023-12-27 10:21:42,222][20347] Updated weights for policy 0, policy_version 684064 (0.0006) [2023-12-27 10:21:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 350175232. Throughput: 0: 10282.7, 1: 10449.3. Samples: 350162244. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:21:42,363][19170] Avg episode reward: [(0, '718.610'), (1, '825.970')] [2023-12-27 10:21:42,648][20381] Updated weights for policy 1, policy_version 683609 (0.0007) [2023-12-27 10:21:42,707][20381] Updated weights for policy 1, policy_version 683619 (0.0008) [2023-12-27 10:21:42,765][20381] Updated weights for policy 1, policy_version 683629 (0.0008) [2023-12-27 10:21:42,822][20381] Updated weights for policy 1, policy_version 683639 (0.0009) [2023-12-27 10:21:42,905][20347] Updated weights for policy 0, policy_version 684074 (0.0007) [2023-12-27 10:21:42,965][20347] Updated weights for policy 0, policy_version 684084 (0.0007) [2023-12-27 10:21:43,024][20347] Updated weights for policy 0, policy_version 684094 (0.0008) [2023-12-27 10:21:43,079][20347] Updated weights for policy 0, policy_version 684104 (0.0008) [2023-12-27 10:21:43,467][20381] Updated weights for policy 1, policy_version 683649 (0.0008) [2023-12-27 10:21:43,518][20381] Updated weights for policy 1, policy_version 683659 (0.0005) [2023-12-27 10:21:43,564][20381] Updated weights for policy 1, policy_version 683669 (0.0005) [2023-12-27 10:21:43,799][20347] Updated weights for policy 0, policy_version 684114 (0.0005) [2023-12-27 10:21:43,857][20347] Updated weights for policy 0, policy_version 684124 (0.0005) [2023-12-27 10:21:43,909][20347] Updated weights for policy 0, policy_version 684134 (0.0007) [2023-12-27 10:21:44,157][20381] Updated weights for policy 1, policy_version 683679 (0.0005) [2023-12-27 10:21:44,220][20381] Updated weights for policy 1, policy_version 683689 (0.0006) [2023-12-27 10:21:44,277][20381] Updated weights for policy 1, policy_version 683699 (0.0005) [2023-12-27 10:21:44,522][20347] Updated weights for policy 0, policy_version 684144 (0.0009) [2023-12-27 10:21:44,574][20347] Updated weights for policy 0, policy_version 684154 (0.0007) [2023-12-27 10:21:44,622][20347] Updated weights for policy 0, policy_version 684164 (0.0008) [2023-12-27 10:21:44,802][20381] Updated weights for policy 1, policy_version 683709 (0.0006) [2023-12-27 10:21:44,857][20381] Updated weights for policy 1, policy_version 683719 (0.0009) [2023-12-27 10:21:44,922][20381] Updated weights for policy 1, policy_version 683729 (0.0007) [2023-12-27 10:21:45,379][20347] Updated weights for policy 0, policy_version 684174 (0.0008) [2023-12-27 10:21:45,439][20347] Updated weights for policy 0, policy_version 684184 (0.0007) [2023-12-27 10:21:45,500][20347] Updated weights for policy 0, policy_version 684194 (0.0007) [2023-12-27 10:21:45,528][20381] Updated weights for policy 1, policy_version 683739 (0.0008) [2023-12-27 10:21:45,580][20381] Updated weights for policy 1, policy_version 683749 (0.0008) [2023-12-27 10:21:45,642][20381] Updated weights for policy 1, policy_version 683759 (0.0007) [2023-12-27 10:21:46,181][20347] Updated weights for policy 0, policy_version 684204 (0.0005) [2023-12-27 10:21:46,241][20347] Updated weights for policy 0, policy_version 684214 (0.0005) [2023-12-27 10:21:46,294][20347] Updated weights for policy 0, policy_version 684224 (0.0007) [2023-12-27 10:21:46,307][20381] Updated weights for policy 1, policy_version 683769 (0.0007) [2023-12-27 10:21:46,355][20381] Updated weights for policy 1, policy_version 683779 (0.0006) [2023-12-27 10:21:46,405][20381] Updated weights for policy 1, policy_version 683789 (0.0008) [2023-12-27 10:21:46,453][20381] Updated weights for policy 1, policy_version 683799 (0.0007) [2023-12-27 10:21:46,958][20347] Updated weights for policy 0, policy_version 684234 (0.0006) [2023-12-27 10:21:47,005][20347] Updated weights for policy 0, policy_version 684244 (0.0007) [2023-12-27 10:21:47,050][20347] Updated weights for policy 0, policy_version 684254 (0.0007) [2023-12-27 10:21:47,105][20347] Updated weights for policy 0, policy_version 684264 (0.0007) [2023-12-27 10:21:47,168][20381] Updated weights for policy 1, policy_version 683809 (0.0010) [2023-12-27 10:21:47,217][20381] Updated weights for policy 1, policy_version 683819 (0.0010) [2023-12-27 10:21:47,272][20381] Updated weights for policy 1, policy_version 683829 (0.0010) [2023-12-27 10:21:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 350281728. Throughput: 0: 10210.1, 1: 10526.8. Samples: 350287224. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:21:47,363][19170] Avg episode reward: [(0, '639.350'), (1, '674.440')] [2023-12-27 10:21:47,755][20347] Updated weights for policy 0, policy_version 684274 (0.0007) [2023-12-27 10:21:47,812][20347] Updated weights for policy 0, policy_version 684284 (0.0006) [2023-12-27 10:21:47,863][20347] Updated weights for policy 0, policy_version 684294 (0.0006) [2023-12-27 10:21:47,941][20381] Updated weights for policy 1, policy_version 683839 (0.0009) [2023-12-27 10:21:48,004][20381] Updated weights for policy 1, policy_version 683849 (0.0007) [2023-12-27 10:21:48,064][20381] Updated weights for policy 1, policy_version 683859 (0.0008) [2023-12-27 10:21:48,429][20347] Updated weights for policy 0, policy_version 684304 (0.0008) [2023-12-27 10:21:48,484][20347] Updated weights for policy 0, policy_version 684314 (0.0008) [2023-12-27 10:21:48,543][20347] Updated weights for policy 0, policy_version 684324 (0.0011) [2023-12-27 10:21:48,730][20381] Updated weights for policy 1, policy_version 683869 (0.0008) [2023-12-27 10:21:48,779][20381] Updated weights for policy 1, policy_version 683879 (0.0006) [2023-12-27 10:21:48,842][20381] Updated weights for policy 1, policy_version 683889 (0.0006) [2023-12-27 10:21:49,196][20347] Updated weights for policy 0, policy_version 684334 (0.0010) [2023-12-27 10:21:49,261][20347] Updated weights for policy 0, policy_version 684344 (0.0009) [2023-12-27 10:21:49,326][20347] Updated weights for policy 0, policy_version 684354 (0.0008) [2023-12-27 10:21:49,511][20381] Updated weights for policy 1, policy_version 683899 (0.0006) [2023-12-27 10:21:49,573][20381] Updated weights for policy 1, policy_version 683909 (0.0008) [2023-12-27 10:21:49,625][20381] Updated weights for policy 1, policy_version 683919 (0.0009) [2023-12-27 10:21:49,966][20347] Updated weights for policy 0, policy_version 684364 (0.0008) [2023-12-27 10:21:50,018][20347] Updated weights for policy 0, policy_version 684374 (0.0008) [2023-12-27 10:21:50,077][20347] Updated weights for policy 0, policy_version 684384 (0.0008) [2023-12-27 10:21:50,222][20381] Updated weights for policy 1, policy_version 683929 (0.0006) [2023-12-27 10:21:50,270][20381] Updated weights for policy 1, policy_version 683939 (0.0006) [2023-12-27 10:21:50,329][20381] Updated weights for policy 1, policy_version 683949 (0.0007) [2023-12-27 10:21:50,383][20381] Updated weights for policy 1, policy_version 683959 (0.0006) [2023-12-27 10:21:50,788][20347] Updated weights for policy 0, policy_version 684394 (0.0008) [2023-12-27 10:21:50,848][20347] Updated weights for policy 0, policy_version 684404 (0.0010) [2023-12-27 10:21:50,906][20347] Updated weights for policy 0, policy_version 684414 (0.0010) [2023-12-27 10:21:50,962][20347] Updated weights for policy 0, policy_version 684424 (0.0010) [2023-12-27 10:21:50,989][20381] Updated weights for policy 1, policy_version 683969 (0.0006) [2023-12-27 10:21:51,050][20381] Updated weights for policy 1, policy_version 683979 (0.0009) [2023-12-27 10:21:51,117][20381] Updated weights for policy 1, policy_version 683989 (0.0008) [2023-12-27 10:21:51,708][20347] Updated weights for policy 0, policy_version 684434 (0.0010) [2023-12-27 10:21:51,773][20347] Updated weights for policy 0, policy_version 684444 (0.0010) [2023-12-27 10:21:51,774][20381] Updated weights for policy 1, policy_version 683999 (0.0007) [2023-12-27 10:21:51,826][20347] Updated weights for policy 0, policy_version 684454 (0.0010) [2023-12-27 10:21:51,831][20381] Updated weights for policy 1, policy_version 684009 (0.0006) [2023-12-27 10:21:51,889][20381] Updated weights for policy 1, policy_version 684019 (0.0007) [2023-12-27 10:21:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20771.6). Total num frames: 350380032. Throughput: 0: 10249.4, 1: 10525.8. Samples: 350351760. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:21:52,363][19170] Avg episode reward: [(0, '573.830'), (1, '680.880')] [2023-12-27 10:21:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000684024_175136768.pth... [2023-12-27 10:21:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000684456_175243264.pth... [2023-12-27 10:21:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000683280_174940160.pth [2023-12-27 10:21:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000682808_174825472.pth [2023-12-27 10:21:52,523][20347] Updated weights for policy 0, policy_version 684464 (0.0011) [2023-12-27 10:21:52,588][20347] Updated weights for policy 0, policy_version 684474 (0.0010) [2023-12-27 10:21:52,623][20381] Updated weights for policy 1, policy_version 684029 (0.0007) [2023-12-27 10:21:52,640][20347] Updated weights for policy 0, policy_version 684484 (0.0010) [2023-12-27 10:21:52,676][20381] Updated weights for policy 1, policy_version 684039 (0.0006) [2023-12-27 10:21:52,722][20292] KL-divergence is very high: 100.1224 [2023-12-27 10:21:52,728][20292] KL-divergence is very high: 303.6969 [2023-12-27 10:21:52,728][20381] Updated weights for policy 1, policy_version 684049 (0.0007) [2023-12-27 10:21:52,744][20292] KL-divergence is very high: 258.9073 [2023-12-27 10:21:53,251][20347] Updated weights for policy 0, policy_version 684494 (0.0007) [2023-12-27 10:21:53,305][20347] Updated weights for policy 0, policy_version 684504 (0.0006) [2023-12-27 10:21:53,362][20347] Updated weights for policy 0, policy_version 684514 (0.0007) [2023-12-27 10:21:53,458][20381] Updated weights for policy 1, policy_version 684059 (0.0008) [2023-12-27 10:21:53,487][20292] KL-divergence is very high: 139.9987 [2023-12-27 10:21:53,521][20381] Updated weights for policy 1, policy_version 684069 (0.0007) [2023-12-27 10:21:53,538][20292] KL-divergence is very high: 114.1641 [2023-12-27 10:21:53,579][20381] Updated weights for policy 1, policy_version 684079 (0.0008) [2023-12-27 10:21:53,953][20347] Updated weights for policy 0, policy_version 684524 (0.0009) [2023-12-27 10:21:54,004][20347] Updated weights for policy 0, policy_version 684534 (0.0010) [2023-12-27 10:21:54,058][20347] Updated weights for policy 0, policy_version 684544 (0.0010) [2023-12-27 10:21:54,297][20381] Updated weights for policy 1, policy_version 684089 (0.0007) [2023-12-27 10:21:54,356][20381] Updated weights for policy 1, policy_version 684099 (0.0007) [2023-12-27 10:21:54,419][20381] Updated weights for policy 1, policy_version 684109 (0.0007) [2023-12-27 10:21:54,480][20381] Updated weights for policy 1, policy_version 684119 (0.0007) [2023-12-27 10:21:54,677][20347] Updated weights for policy 0, policy_version 684554 (0.0009) [2023-12-27 10:21:54,735][20347] Updated weights for policy 0, policy_version 684564 (0.0010) [2023-12-27 10:21:54,786][20347] Updated weights for policy 0, policy_version 684574 (0.0010) [2023-12-27 10:21:54,838][20347] Updated weights for policy 0, policy_version 684584 (0.0010) [2023-12-27 10:21:55,113][20381] Updated weights for policy 1, policy_version 684129 (0.0007) [2023-12-27 10:21:55,176][20381] Updated weights for policy 1, policy_version 684139 (0.0007) [2023-12-27 10:21:55,227][20381] Updated weights for policy 1, policy_version 684149 (0.0007) [2023-12-27 10:21:55,497][20347] Updated weights for policy 0, policy_version 684594 (0.0010) [2023-12-27 10:21:55,556][20347] Updated weights for policy 0, policy_version 684604 (0.0010) [2023-12-27 10:21:55,612][20347] Updated weights for policy 0, policy_version 684614 (0.0011) [2023-12-27 10:21:55,852][20381] Updated weights for policy 1, policy_version 684159 (0.0008) [2023-12-27 10:21:55,902][20381] Updated weights for policy 1, policy_version 684169 (0.0006) [2023-12-27 10:21:55,953][20381] Updated weights for policy 1, policy_version 684179 (0.0008) [2023-12-27 10:21:56,198][20347] Updated weights for policy 0, policy_version 684624 (0.0006) [2023-12-27 10:21:56,252][20347] Updated weights for policy 0, policy_version 684634 (0.0007) [2023-12-27 10:21:56,299][20347] Updated weights for policy 0, policy_version 684644 (0.0007) [2023-12-27 10:21:56,655][20381] Updated weights for policy 1, policy_version 684189 (0.0007) [2023-12-27 10:21:56,700][20381] Updated weights for policy 1, policy_version 684199 (0.0006) [2023-12-27 10:21:56,747][20381] Updated weights for policy 1, policy_version 684209 (0.0007) [2023-12-27 10:21:56,940][20347] Updated weights for policy 0, policy_version 684654 (0.0008) [2023-12-27 10:21:56,994][20347] Updated weights for policy 0, policy_version 684664 (0.0008) [2023-12-27 10:21:57,054][20347] Updated weights for policy 0, policy_version 684674 (0.0007) [2023-12-27 10:21:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 350486528. Throughput: 0: 10348.9, 1: 10454.3. Samples: 350476420. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:21:57,364][19170] Avg episode reward: [(0, '651.450'), (1, '633.050')] [2023-12-27 10:21:57,441][20381] Updated weights for policy 1, policy_version 684219 (0.0008) [2023-12-27 10:21:57,493][20381] Updated weights for policy 1, policy_version 684229 (0.0009) [2023-12-27 10:21:57,505][20292] KL-divergence is very high: 225.0972 [2023-12-27 10:21:57,521][20292] KL-divergence is very high: 141.3208 [2023-12-27 10:21:57,543][20381] Updated weights for policy 1, policy_version 684239 (0.0007) [2023-12-27 10:21:57,650][20347] Updated weights for policy 0, policy_version 684684 (0.0006) [2023-12-27 10:21:57,701][20347] Updated weights for policy 0, policy_version 684694 (0.0006) [2023-12-27 10:21:57,760][20347] Updated weights for policy 0, policy_version 684704 (0.0007) [2023-12-27 10:21:58,234][20381] Updated weights for policy 1, policy_version 684249 (0.0008) [2023-12-27 10:21:58,288][20381] Updated weights for policy 1, policy_version 684259 (0.0011) [2023-12-27 10:21:58,345][20292] KL-divergence is very high: 175.8825 [2023-12-27 10:21:58,346][20381] Updated weights for policy 1, policy_version 684269 (0.0010) [2023-12-27 10:21:58,352][20292] KL-divergence is very high: 183.6212 [2023-12-27 10:21:58,385][20347] Updated weights for policy 0, policy_version 684714 (0.0008) [2023-12-27 10:21:58,404][20381] Updated weights for policy 1, policy_version 684279 (0.0011) [2023-12-27 10:21:58,446][20347] Updated weights for policy 0, policy_version 684724 (0.0011) [2023-12-27 10:21:58,509][20347] Updated weights for policy 0, policy_version 684734 (0.0011) [2023-12-27 10:21:58,572][20347] Updated weights for policy 0, policy_version 684744 (0.0009) [2023-12-27 10:21:59,128][20381] Updated weights for policy 1, policy_version 684289 (0.0010) [2023-12-27 10:21:59,182][20381] Updated weights for policy 1, policy_version 684299 (0.0010) [2023-12-27 10:21:59,243][20381] Updated weights for policy 1, policy_version 684309 (0.0011) [2023-12-27 10:21:59,271][20347] Updated weights for policy 0, policy_version 684754 (0.0011) [2023-12-27 10:21:59,334][20347] Updated weights for policy 0, policy_version 684764 (0.0008) [2023-12-27 10:21:59,396][20347] Updated weights for policy 0, policy_version 684774 (0.0007) [2023-12-27 10:21:59,854][20381] Updated weights for policy 1, policy_version 684319 (0.0008) [2023-12-27 10:21:59,909][20381] Updated weights for policy 1, policy_version 684329 (0.0008) [2023-12-27 10:21:59,970][20381] Updated weights for policy 1, policy_version 684339 (0.0007) [2023-12-27 10:22:00,116][20347] Updated weights for policy 0, policy_version 684784 (0.0009) [2023-12-27 10:22:00,171][20347] Updated weights for policy 0, policy_version 684794 (0.0010) [2023-12-27 10:22:00,224][20347] Updated weights for policy 0, policy_version 684804 (0.0010) [2023-12-27 10:22:00,635][20381] Updated weights for policy 1, policy_version 684349 (0.0006) [2023-12-27 10:22:00,696][20381] Updated weights for policy 1, policy_version 684359 (0.0005) [2023-12-27 10:22:00,758][20381] Updated weights for policy 1, policy_version 684369 (0.0007) [2023-12-27 10:22:00,872][20347] Updated weights for policy 0, policy_version 684814 (0.0007) [2023-12-27 10:22:00,933][20347] Updated weights for policy 0, policy_version 684824 (0.0008) [2023-12-27 10:22:00,988][20347] Updated weights for policy 0, policy_version 684834 (0.0008) [2023-12-27 10:22:01,402][20381] Updated weights for policy 1, policy_version 684379 (0.0006) [2023-12-27 10:22:01,461][20381] Updated weights for policy 1, policy_version 684389 (0.0008) [2023-12-27 10:22:01,522][20381] Updated weights for policy 1, policy_version 684399 (0.0008) [2023-12-27 10:22:01,684][20347] Updated weights for policy 0, policy_version 684844 (0.0008) [2023-12-27 10:22:01,749][20347] Updated weights for policy 0, policy_version 684854 (0.0008) [2023-12-27 10:22:01,801][20347] Updated weights for policy 0, policy_version 684864 (0.0008) [2023-12-27 10:22:02,172][20381] Updated weights for policy 1, policy_version 684409 (0.0010) [2023-12-27 10:22:02,226][20381] Updated weights for policy 1, policy_version 684419 (0.0006) [2023-12-27 10:22:02,285][20381] Updated weights for policy 1, policy_version 684429 (0.0007) [2023-12-27 10:22:02,346][20381] Updated weights for policy 1, policy_version 684439 (0.0007) [2023-12-27 10:22:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 350593024. Throughput: 0: 10349.6, 1: 10420.1. Samples: 350600924. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:02,363][19170] Avg episode reward: [(0, '810.760'), (1, '776.720')] [2023-12-27 10:22:02,543][20347] Updated weights for policy 0, policy_version 684874 (0.0007) [2023-12-27 10:22:02,594][20347] Updated weights for policy 0, policy_version 684884 (0.0007) [2023-12-27 10:22:02,656][20347] Updated weights for policy 0, policy_version 684894 (0.0007) [2023-12-27 10:22:02,678][20291] KL-divergence is very high: 292.4028 [2023-12-27 10:22:02,711][20347] Updated weights for policy 0, policy_version 684904 (0.0008) [2023-12-27 10:22:02,974][20381] Updated weights for policy 1, policy_version 684449 (0.0007) [2023-12-27 10:22:03,025][20381] Updated weights for policy 1, policy_version 684459 (0.0009) [2023-12-27 10:22:03,071][20381] Updated weights for policy 1, policy_version 684469 (0.0006) [2023-12-27 10:22:03,331][20291] KL-divergence is very high: 194.4085 [2023-12-27 10:22:03,347][20291] KL-divergence is very high: 168.2073 [2023-12-27 10:22:03,375][20347] Updated weights for policy 0, policy_version 684914 (0.0007) [2023-12-27 10:22:03,375][20291] KL-divergence is very high: 116.2527 [2023-12-27 10:22:03,430][20347] Updated weights for policy 0, policy_version 684924 (0.0007) [2023-12-27 10:22:03,489][20347] Updated weights for policy 0, policy_version 684934 (0.0008) [2023-12-27 10:22:03,718][20381] Updated weights for policy 1, policy_version 684479 (0.0008) [2023-12-27 10:22:03,772][20381] Updated weights for policy 1, policy_version 684489 (0.0008) [2023-12-27 10:22:03,823][20381] Updated weights for policy 1, policy_version 684499 (0.0007) [2023-12-27 10:22:04,207][20347] Updated weights for policy 0, policy_version 684944 (0.0007) [2023-12-27 10:22:04,271][20347] Updated weights for policy 0, policy_version 684954 (0.0008) [2023-12-27 10:22:04,327][20347] Updated weights for policy 0, policy_version 684964 (0.0007) [2023-12-27 10:22:04,534][20381] Updated weights for policy 1, policy_version 684509 (0.0008) [2023-12-27 10:22:04,595][20381] Updated weights for policy 1, policy_version 684519 (0.0007) [2023-12-27 10:22:04,657][20381] Updated weights for policy 1, policy_version 684529 (0.0007) [2023-12-27 10:22:05,062][20347] Updated weights for policy 0, policy_version 684974 (0.0007) [2023-12-27 10:22:05,122][20347] Updated weights for policy 0, policy_version 684984 (0.0008) [2023-12-27 10:22:05,181][20347] Updated weights for policy 0, policy_version 684994 (0.0008) [2023-12-27 10:22:05,285][20381] Updated weights for policy 1, policy_version 684539 (0.0006) [2023-12-27 10:22:05,333][20381] Updated weights for policy 1, policy_version 684549 (0.0006) [2023-12-27 10:22:05,395][20381] Updated weights for policy 1, policy_version 684559 (0.0005) [2023-12-27 10:22:05,916][20347] Updated weights for policy 0, policy_version 685004 (0.0008) [2023-12-27 10:22:05,964][20347] Updated weights for policy 0, policy_version 685014 (0.0007) [2023-12-27 10:22:06,010][20381] Updated weights for policy 1, policy_version 684569 (0.0006) [2023-12-27 10:22:06,015][20347] Updated weights for policy 0, policy_version 685024 (0.0008) [2023-12-27 10:22:06,059][20381] Updated weights for policy 1, policy_version 684579 (0.0006) [2023-12-27 10:22:06,117][20381] Updated weights for policy 1, policy_version 684589 (0.0008) [2023-12-27 10:22:06,166][20381] Updated weights for policy 1, policy_version 684599 (0.0009) [2023-12-27 10:22:06,774][20347] Updated weights for policy 0, policy_version 685034 (0.0006) [2023-12-27 10:22:06,831][20347] Updated weights for policy 0, policy_version 685044 (0.0008) [2023-12-27 10:22:06,891][20381] Updated weights for policy 1, policy_version 684609 (0.0007) [2023-12-27 10:22:06,893][20347] Updated weights for policy 0, policy_version 685054 (0.0006) [2023-12-27 10:22:06,941][20381] Updated weights for policy 1, policy_version 684619 (0.0005) [2023-12-27 10:22:06,946][20347] Updated weights for policy 0, policy_version 685064 (0.0007) [2023-12-27 10:22:06,991][20381] Updated weights for policy 1, policy_version 684629 (0.0008) [2023-12-27 10:22:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 350691328. Throughput: 0: 10331.3, 1: 10429.1. Samples: 350662460. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:07,363][19170] Avg episode reward: [(0, '835.000'), (1, '864.710')] [2023-12-27 10:22:07,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000685064_175398912.pth... [2023-12-27 10:22:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000684632_175292416.pth... [2023-12-27 10:22:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000683888_175095808.pth [2023-12-27 10:22:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000683416_174981120.pth [2023-12-27 10:22:07,621][20347] Updated weights for policy 0, policy_version 685074 (0.0010) [2023-12-27 10:22:07,673][20347] Updated weights for policy 0, policy_version 685084 (0.0010) [2023-12-27 10:22:07,720][20381] Updated weights for policy 1, policy_version 684639 (0.0007) [2023-12-27 10:22:07,732][20347] Updated weights for policy 0, policy_version 685094 (0.0010) [2023-12-27 10:22:07,780][20381] Updated weights for policy 1, policy_version 684649 (0.0007) [2023-12-27 10:22:07,835][20381] Updated weights for policy 1, policy_version 684659 (0.0007) [2023-12-27 10:22:08,391][20347] Updated weights for policy 0, policy_version 685104 (0.0010) [2023-12-27 10:22:08,433][20381] Updated weights for policy 1, policy_version 684669 (0.0008) [2023-12-27 10:22:08,455][20347] Updated weights for policy 0, policy_version 685114 (0.0011) [2023-12-27 10:22:08,482][20381] Updated weights for policy 1, policy_version 684679 (0.0010) [2023-12-27 10:22:08,517][20347] Updated weights for policy 0, policy_version 685124 (0.0010) [2023-12-27 10:22:08,532][20381] Updated weights for policy 1, policy_version 684689 (0.0010) [2023-12-27 10:22:09,188][20347] Updated weights for policy 0, policy_version 685134 (0.0008) [2023-12-27 10:22:09,217][20381] Updated weights for policy 1, policy_version 684699 (0.0010) [2023-12-27 10:22:09,256][20347] Updated weights for policy 0, policy_version 685144 (0.0009) [2023-12-27 10:22:09,283][20381] Updated weights for policy 1, policy_version 684709 (0.0008) [2023-12-27 10:22:09,323][20347] Updated weights for policy 0, policy_version 685154 (0.0009) [2023-12-27 10:22:09,346][20381] Updated weights for policy 1, policy_version 684719 (0.0010) [2023-12-27 10:22:09,994][20347] Updated weights for policy 0, policy_version 685164 (0.0009) [2023-12-27 10:22:10,014][20381] Updated weights for policy 1, policy_version 684729 (0.0008) [2023-12-27 10:22:10,050][20347] Updated weights for policy 0, policy_version 685174 (0.0008) [2023-12-27 10:22:10,070][20381] Updated weights for policy 1, policy_version 684739 (0.0011) [2023-12-27 10:22:10,107][20347] Updated weights for policy 0, policy_version 685184 (0.0008) [2023-12-27 10:22:10,127][20381] Updated weights for policy 1, policy_version 684749 (0.0011) [2023-12-27 10:22:10,183][20381] Updated weights for policy 1, policy_version 684759 (0.0011) [2023-12-27 10:22:10,820][20347] Updated weights for policy 0, policy_version 685194 (0.0008) [2023-12-27 10:22:10,872][20381] Updated weights for policy 1, policy_version 684769 (0.0010) [2023-12-27 10:22:10,884][20347] Updated weights for policy 0, policy_version 685204 (0.0006) [2023-12-27 10:22:10,926][20381] Updated weights for policy 1, policy_version 684779 (0.0011) [2023-12-27 10:22:10,944][20347] Updated weights for policy 0, policy_version 685214 (0.0006) [2023-12-27 10:22:10,986][20381] Updated weights for policy 1, policy_version 684789 (0.0011) [2023-12-27 10:22:11,004][20347] Updated weights for policy 0, policy_version 685224 (0.0009) [2023-12-27 10:22:11,663][20347] Updated weights for policy 0, policy_version 685234 (0.0011) [2023-12-27 10:22:11,718][20381] Updated weights for policy 1, policy_version 684799 (0.0011) [2023-12-27 10:22:11,726][20347] Updated weights for policy 0, policy_version 685244 (0.0011) [2023-12-27 10:22:11,779][20381] Updated weights for policy 1, policy_version 684809 (0.0007) [2023-12-27 10:22:11,786][20347] Updated weights for policy 0, policy_version 685254 (0.0010) [2023-12-27 10:22:11,836][20381] Updated weights for policy 1, policy_version 684819 (0.0006) [2023-12-27 10:22:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20716.0). Total num frames: 350789632. Throughput: 0: 10294.1, 1: 10442.2. Samples: 350783480. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:12,363][19170] Avg episode reward: [(0, '902.320'), (1, '806.590')] [2023-12-27 10:22:12,369][20347] Updated weights for policy 0, policy_version 685264 (0.0011) [2023-12-27 10:22:12,427][20347] Updated weights for policy 0, policy_version 685274 (0.0010) [2023-12-27 10:22:12,480][20347] Updated weights for policy 0, policy_version 685284 (0.0006) [2023-12-27 10:22:12,494][20381] Updated weights for policy 1, policy_version 684829 (0.0007) [2023-12-27 10:22:12,549][20381] Updated weights for policy 1, policy_version 684839 (0.0007) [2023-12-27 10:22:12,595][20381] Updated weights for policy 1, policy_version 684849 (0.0008) [2023-12-27 10:22:13,143][20347] Updated weights for policy 0, policy_version 685294 (0.0007) [2023-12-27 10:22:13,198][20347] Updated weights for policy 0, policy_version 685304 (0.0005) [2023-12-27 10:22:13,254][20347] Updated weights for policy 0, policy_version 685314 (0.0005) [2023-12-27 10:22:13,338][20381] Updated weights for policy 1, policy_version 684859 (0.0007) [2023-12-27 10:22:13,390][20381] Updated weights for policy 1, policy_version 684869 (0.0007) [2023-12-27 10:22:13,441][20381] Updated weights for policy 1, policy_version 684879 (0.0007) [2023-12-27 10:22:13,845][20347] Updated weights for policy 0, policy_version 685324 (0.0007) [2023-12-27 10:22:13,906][20347] Updated weights for policy 0, policy_version 685334 (0.0010) [2023-12-27 10:22:13,963][20347] Updated weights for policy 0, policy_version 685344 (0.0010) [2023-12-27 10:22:14,074][20381] Updated weights for policy 1, policy_version 684889 (0.0007) [2023-12-27 10:22:14,134][20381] Updated weights for policy 1, policy_version 684899 (0.0008) [2023-12-27 10:22:14,193][20381] Updated weights for policy 1, policy_version 684909 (0.0010) [2023-12-27 10:22:14,250][20381] Updated weights for policy 1, policy_version 684919 (0.0010) [2023-12-27 10:22:14,630][20347] Updated weights for policy 0, policy_version 685354 (0.0010) [2023-12-27 10:22:14,688][20347] Updated weights for policy 0, policy_version 685364 (0.0010) [2023-12-27 10:22:14,740][20347] Updated weights for policy 0, policy_version 685374 (0.0010) [2023-12-27 10:22:14,805][20347] Updated weights for policy 0, policy_version 685384 (0.0010) [2023-12-27 10:22:14,957][20381] Updated weights for policy 1, policy_version 684929 (0.0011) [2023-12-27 10:22:15,011][20381] Updated weights for policy 1, policy_version 684939 (0.0011) [2023-12-27 10:22:15,071][20381] Updated weights for policy 1, policy_version 684949 (0.0011) [2023-12-27 10:22:15,509][20347] Updated weights for policy 0, policy_version 685394 (0.0010) [2023-12-27 10:22:15,564][20347] Updated weights for policy 0, policy_version 685404 (0.0010) [2023-12-27 10:22:15,614][20347] Updated weights for policy 0, policy_version 685414 (0.0005) [2023-12-27 10:22:15,753][20381] Updated weights for policy 1, policy_version 684959 (0.0010) [2023-12-27 10:22:15,809][20381] Updated weights for policy 1, policy_version 684969 (0.0011) [2023-12-27 10:22:15,858][20381] Updated weights for policy 1, policy_version 684979 (0.0010) [2023-12-27 10:22:16,259][20347] Updated weights for policy 0, policy_version 685424 (0.0007) [2023-12-27 10:22:16,307][20347] Updated weights for policy 0, policy_version 685434 (0.0006) [2023-12-27 10:22:16,357][20347] Updated weights for policy 0, policy_version 685444 (0.0007) [2023-12-27 10:22:16,552][20381] Updated weights for policy 1, policy_version 684989 (0.0011) [2023-12-27 10:22:16,597][20381] Updated weights for policy 1, policy_version 684999 (0.0010) [2023-12-27 10:22:16,658][20381] Updated weights for policy 1, policy_version 685009 (0.0010) [2023-12-27 10:22:17,053][20347] Updated weights for policy 0, policy_version 685454 (0.0007) [2023-12-27 10:22:17,097][20347] Updated weights for policy 0, policy_version 685464 (0.0008) [2023-12-27 10:22:17,161][20347] Updated weights for policy 0, policy_version 685474 (0.0006) [2023-12-27 10:22:17,346][20381] Updated weights for policy 1, policy_version 685019 (0.0009) [2023-12-27 10:22:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 350896128. Throughput: 0: 10275.3, 1: 10408.9. Samples: 350907824. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:17,363][19170] Avg episode reward: [(0, '941.600'), (1, '679.870')] [2023-12-27 10:22:17,397][20381] Updated weights for policy 1, policy_version 685029 (0.0008) [2023-12-27 10:22:17,442][20381] Updated weights for policy 1, policy_version 685039 (0.0008) [2023-12-27 10:22:17,825][20347] Updated weights for policy 0, policy_version 685484 (0.0007) [2023-12-27 10:22:17,878][20347] Updated weights for policy 0, policy_version 685494 (0.0010) [2023-12-27 10:22:17,930][20347] Updated weights for policy 0, policy_version 685504 (0.0010) [2023-12-27 10:22:18,087][20381] Updated weights for policy 1, policy_version 685049 (0.0008) [2023-12-27 10:22:18,142][20381] Updated weights for policy 1, policy_version 685059 (0.0007) [2023-12-27 10:22:18,206][20381] Updated weights for policy 1, policy_version 685069 (0.0008) [2023-12-27 10:22:18,260][20381] Updated weights for policy 1, policy_version 685079 (0.0008) [2023-12-27 10:22:18,630][20347] Updated weights for policy 0, policy_version 685514 (0.0010) [2023-12-27 10:22:18,679][20347] Updated weights for policy 0, policy_version 685524 (0.0010) [2023-12-27 10:22:18,725][20347] Updated weights for policy 0, policy_version 685534 (0.0010) [2023-12-27 10:22:18,783][20347] Updated weights for policy 0, policy_version 685544 (0.0010) [2023-12-27 10:22:18,892][20381] Updated weights for policy 1, policy_version 685089 (0.0007) [2023-12-27 10:22:18,937][20381] Updated weights for policy 1, policy_version 685099 (0.0008) [2023-12-27 10:22:18,984][20381] Updated weights for policy 1, policy_version 685109 (0.0008) [2023-12-27 10:22:19,508][20347] Updated weights for policy 0, policy_version 685554 (0.0010) [2023-12-27 10:22:19,566][20347] Updated weights for policy 0, policy_version 685564 (0.0010) [2023-12-27 10:22:19,623][20347] Updated weights for policy 0, policy_version 685574 (0.0010) [2023-12-27 10:22:19,685][20381] Updated weights for policy 1, policy_version 685119 (0.0007) [2023-12-27 10:22:19,744][20381] Updated weights for policy 1, policy_version 685129 (0.0007) [2023-12-27 10:22:19,806][20381] Updated weights for policy 1, policy_version 685139 (0.0008) [2023-12-27 10:22:20,340][20347] Updated weights for policy 0, policy_version 685584 (0.0010) [2023-12-27 10:22:20,407][20347] Updated weights for policy 0, policy_version 685594 (0.0010) [2023-12-27 10:22:20,472][20381] Updated weights for policy 1, policy_version 685149 (0.0007) [2023-12-27 10:22:20,474][20347] Updated weights for policy 0, policy_version 685604 (0.0007) [2023-12-27 10:22:20,531][20381] Updated weights for policy 1, policy_version 685159 (0.0006) [2023-12-27 10:22:20,591][20381] Updated weights for policy 1, policy_version 685169 (0.0007) [2023-12-27 10:22:21,144][20347] Updated weights for policy 0, policy_version 685614 (0.0009) [2023-12-27 10:22:21,207][20347] Updated weights for policy 0, policy_version 685624 (0.0011) [2023-12-27 10:22:21,263][20381] Updated weights for policy 1, policy_version 685179 (0.0007) [2023-12-27 10:22:21,273][20347] Updated weights for policy 0, policy_version 685634 (0.0011) [2023-12-27 10:22:21,317][20381] Updated weights for policy 1, policy_version 685189 (0.0006) [2023-12-27 10:22:21,382][20381] Updated weights for policy 1, policy_version 685199 (0.0007) [2023-12-27 10:22:21,980][20381] Updated weights for policy 1, policy_version 685209 (0.0006) [2023-12-27 10:22:22,009][20347] Updated weights for policy 0, policy_version 685644 (0.0009) [2023-12-27 10:22:22,039][20381] Updated weights for policy 1, policy_version 685219 (0.0008) [2023-12-27 10:22:22,059][20347] Updated weights for policy 0, policy_version 685654 (0.0010) [2023-12-27 10:22:22,099][20381] Updated weights for policy 1, policy_version 685229 (0.0008) [2023-12-27 10:22:22,115][20347] Updated weights for policy 0, policy_version 685664 (0.0010) [2023-12-27 10:22:22,154][20381] Updated weights for policy 1, policy_version 685239 (0.0006) [2023-12-27 10:22:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 351002624. Throughput: 0: 10281.8, 1: 10332.4. Samples: 350969128. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:22,363][19170] Avg episode reward: [(0, '932.580'), (1, '720.280')] [2023-12-27 10:22:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000685240_175448064.pth... [2023-12-27 10:22:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000685672_175554560.pth... [2023-12-27 10:22:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000684024_175136768.pth [2023-12-27 10:22:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000684456_175243264.pth [2023-12-27 10:22:22,827][20347] Updated weights for policy 0, policy_version 685674 (0.0009) [2023-12-27 10:22:22,834][20381] Updated weights for policy 1, policy_version 685249 (0.0008) [2023-12-27 10:22:22,889][20381] Updated weights for policy 1, policy_version 685259 (0.0009) [2023-12-27 10:22:22,891][20347] Updated weights for policy 0, policy_version 685684 (0.0005) [2023-12-27 10:22:22,942][20381] Updated weights for policy 1, policy_version 685269 (0.0006) [2023-12-27 10:22:22,959][20347] Updated weights for policy 0, policy_version 685694 (0.0008) [2023-12-27 10:22:23,023][20347] Updated weights for policy 0, policy_version 685704 (0.0008) [2023-12-27 10:22:23,544][20347] Updated weights for policy 0, policy_version 685714 (0.0008) [2023-12-27 10:22:23,609][20347] Updated weights for policy 0, policy_version 685724 (0.0005) [2023-12-27 10:22:23,676][20347] Updated weights for policy 0, policy_version 685734 (0.0009) [2023-12-27 10:22:23,686][20381] Updated weights for policy 1, policy_version 685279 (0.0006) [2023-12-27 10:22:23,745][20381] Updated weights for policy 1, policy_version 685289 (0.0006) [2023-12-27 10:22:23,800][20381] Updated weights for policy 1, policy_version 685299 (0.0005) [2023-12-27 10:22:24,298][20347] Updated weights for policy 0, policy_version 685744 (0.0008) [2023-12-27 10:22:24,320][20381] Updated weights for policy 1, policy_version 685309 (0.0006) [2023-12-27 10:22:24,363][20347] Updated weights for policy 0, policy_version 685754 (0.0007) [2023-12-27 10:22:24,365][20381] Updated weights for policy 1, policy_version 685319 (0.0005) [2023-12-27 10:22:24,417][20347] Updated weights for policy 0, policy_version 685764 (0.0007) [2023-12-27 10:22:24,421][20381] Updated weights for policy 1, policy_version 685329 (0.0005) [2023-12-27 10:22:25,040][20347] Updated weights for policy 0, policy_version 685774 (0.0007) [2023-12-27 10:22:25,046][20381] Updated weights for policy 1, policy_version 685339 (0.0006) [2023-12-27 10:22:25,060][20291] KL-divergence is very high: 172.6434 [2023-12-27 10:22:25,098][20347] Updated weights for policy 0, policy_version 685784 (0.0007) [2023-12-27 10:22:25,101][20381] Updated weights for policy 1, policy_version 685349 (0.0006) [2023-12-27 10:22:25,104][20291] KL-divergence is very high: 158.5723 [2023-12-27 10:22:25,147][20291] KL-divergence is very high: 108.7185 [2023-12-27 10:22:25,152][20347] Updated weights for policy 0, policy_version 685794 (0.0006) [2023-12-27 10:22:25,154][20381] Updated weights for policy 1, policy_version 685359 (0.0007) [2023-12-27 10:22:25,751][20381] Updated weights for policy 1, policy_version 685369 (0.0006) [2023-12-27 10:22:25,757][20347] Updated weights for policy 0, policy_version 685804 (0.0008) [2023-12-27 10:22:25,810][20381] Updated weights for policy 1, policy_version 685379 (0.0007) [2023-12-27 10:22:25,812][20347] Updated weights for policy 0, policy_version 685814 (0.0009) [2023-12-27 10:22:25,870][20381] Updated weights for policy 1, policy_version 685389 (0.0008) [2023-12-27 10:22:25,873][20347] Updated weights for policy 0, policy_version 685824 (0.0005) [2023-12-27 10:22:25,929][20381] Updated weights for policy 1, policy_version 685399 (0.0008) [2023-12-27 10:22:26,529][20347] Updated weights for policy 0, policy_version 685834 (0.0008) [2023-12-27 10:22:26,547][20381] Updated weights for policy 1, policy_version 685409 (0.0006) [2023-12-27 10:22:26,580][20347] Updated weights for policy 0, policy_version 685844 (0.0010) [2023-12-27 10:22:26,603][20381] Updated weights for policy 1, policy_version 685419 (0.0006) [2023-12-27 10:22:26,633][20347] Updated weights for policy 0, policy_version 685854 (0.0010) [2023-12-27 10:22:26,653][20381] Updated weights for policy 1, policy_version 685429 (0.0005) [2023-12-27 10:22:26,687][20347] Updated weights for policy 0, policy_version 685864 (0.0010) [2023-12-27 10:22:27,314][20381] Updated weights for policy 1, policy_version 685439 (0.0006) [2023-12-27 10:22:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20716.0). Total num frames: 351100928. Throughput: 0: 10321.8, 1: 10435.1. Samples: 351096304. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:27,363][19170] Avg episode reward: [(0, '893.160'), (1, '766.940')] [2023-12-27 10:22:27,364][20347] Updated weights for policy 0, policy_version 685874 (0.0009) [2023-12-27 10:22:27,373][20381] Updated weights for policy 1, policy_version 685449 (0.0006) [2023-12-27 10:22:27,417][20347] Updated weights for policy 0, policy_version 685884 (0.0007) [2023-12-27 10:22:27,426][20381] Updated weights for policy 1, policy_version 685459 (0.0006) [2023-12-27 10:22:27,465][20347] Updated weights for policy 0, policy_version 685894 (0.0010) [2023-12-27 10:22:28,036][20381] Updated weights for policy 1, policy_version 685469 (0.0005) [2023-12-27 10:22:28,092][20381] Updated weights for policy 1, policy_version 685479 (0.0007) [2023-12-27 10:22:28,107][20347] Updated weights for policy 0, policy_version 685904 (0.0010) [2023-12-27 10:22:28,144][20381] Updated weights for policy 1, policy_version 685489 (0.0010) [2023-12-27 10:22:28,162][20347] Updated weights for policy 0, policy_version 685914 (0.0010) [2023-12-27 10:22:28,217][20347] Updated weights for policy 0, policy_version 685924 (0.0009) [2023-12-27 10:22:28,745][20381] Updated weights for policy 1, policy_version 685499 (0.0010) [2023-12-27 10:22:28,802][20381] Updated weights for policy 1, policy_version 685509 (0.0008) [2023-12-27 10:22:28,804][20347] Updated weights for policy 0, policy_version 685934 (0.0005) [2023-12-27 10:22:28,851][20381] Updated weights for policy 1, policy_version 685519 (0.0008) [2023-12-27 10:22:28,853][20347] Updated weights for policy 0, policy_version 685944 (0.0006) [2023-12-27 10:22:28,913][20347] Updated weights for policy 0, policy_version 685954 (0.0005) [2023-12-27 10:22:29,467][20347] Updated weights for policy 0, policy_version 685964 (0.0007) [2023-12-27 10:22:29,517][20347] Updated weights for policy 0, policy_version 685974 (0.0009) [2023-12-27 10:22:29,554][20381] Updated weights for policy 1, policy_version 685529 (0.0007) [2023-12-27 10:22:29,579][20347] Updated weights for policy 0, policy_version 685984 (0.0009) [2023-12-27 10:22:29,614][20381] Updated weights for policy 1, policy_version 685539 (0.0006) [2023-12-27 10:22:29,663][20381] Updated weights for policy 1, policy_version 685549 (0.0007) [2023-12-27 10:22:29,712][20381] Updated weights for policy 1, policy_version 685559 (0.0007) [2023-12-27 10:22:30,280][20347] Updated weights for policy 0, policy_version 685994 (0.0007) [2023-12-27 10:22:30,321][20381] Updated weights for policy 1, policy_version 685569 (0.0006) [2023-12-27 10:22:30,340][20347] Updated weights for policy 0, policy_version 686004 (0.0008) [2023-12-27 10:22:30,381][20381] Updated weights for policy 1, policy_version 685579 (0.0007) [2023-12-27 10:22:30,396][20347] Updated weights for policy 0, policy_version 686014 (0.0007) [2023-12-27 10:22:30,434][20381] Updated weights for policy 1, policy_version 685589 (0.0006) [2023-12-27 10:22:30,450][20347] Updated weights for policy 0, policy_version 686024 (0.0008) [2023-12-27 10:22:31,014][20381] Updated weights for policy 1, policy_version 685599 (0.0008) [2023-12-27 10:22:31,072][20381] Updated weights for policy 1, policy_version 685609 (0.0007) [2023-12-27 10:22:31,124][20381] Updated weights for policy 1, policy_version 685619 (0.0008) [2023-12-27 10:22:31,184][20347] Updated weights for policy 0, policy_version 686034 (0.0006) [2023-12-27 10:22:31,243][20347] Updated weights for policy 0, policy_version 686044 (0.0008) [2023-12-27 10:22:31,306][20347] Updated weights for policy 0, policy_version 686054 (0.0008) [2023-12-27 10:22:31,829][20381] Updated weights for policy 1, policy_version 685629 (0.0008) [2023-12-27 10:22:31,887][20381] Updated weights for policy 1, policy_version 685639 (0.0008) [2023-12-27 10:22:31,950][20381] Updated weights for policy 1, policy_version 685649 (0.0008) [2023-12-27 10:22:32,035][20347] Updated weights for policy 0, policy_version 686064 (0.0006) [2023-12-27 10:22:32,090][20347] Updated weights for policy 0, policy_version 686074 (0.0007) [2023-12-27 10:22:32,144][20347] Updated weights for policy 0, policy_version 686084 (0.0007) [2023-12-27 10:22:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 351215616. Throughput: 0: 10390.9, 1: 10435.8. Samples: 351224424. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:32,363][19170] Avg episode reward: [(0, '806.360'), (1, '748.230')] [2023-12-27 10:22:32,682][20381] Updated weights for policy 1, policy_version 685659 (0.0008) [2023-12-27 10:22:32,738][20381] Updated weights for policy 1, policy_version 685669 (0.0008) [2023-12-27 10:22:32,787][20347] Updated weights for policy 0, policy_version 686094 (0.0007) [2023-12-27 10:22:32,797][20381] Updated weights for policy 1, policy_version 685679 (0.0006) [2023-12-27 10:22:32,845][20347] Updated weights for policy 0, policy_version 686104 (0.0008) [2023-12-27 10:22:32,903][20347] Updated weights for policy 0, policy_version 686114 (0.0008) [2023-12-27 10:22:33,382][20381] Updated weights for policy 1, policy_version 685689 (0.0005) [2023-12-27 10:22:33,436][20381] Updated weights for policy 1, policy_version 685699 (0.0005) [2023-12-27 10:22:33,490][20381] Updated weights for policy 1, policy_version 685709 (0.0005) [2023-12-27 10:22:33,545][20381] Updated weights for policy 1, policy_version 685719 (0.0005) [2023-12-27 10:22:33,593][20347] Updated weights for policy 0, policy_version 686124 (0.0009) [2023-12-27 10:22:33,647][20347] Updated weights for policy 0, policy_version 686134 (0.0010) [2023-12-27 10:22:33,689][20347] Updated weights for policy 0, policy_version 686144 (0.0005) [2023-12-27 10:22:34,050][20381] Updated weights for policy 1, policy_version 685729 (0.0005) [2023-12-27 10:22:34,095][20381] Updated weights for policy 1, policy_version 685739 (0.0005) [2023-12-27 10:22:34,154][20381] Updated weights for policy 1, policy_version 685749 (0.0008) [2023-12-27 10:22:34,263][20347] Updated weights for policy 0, policy_version 686154 (0.0006) [2023-12-27 10:22:34,321][20347] Updated weights for policy 0, policy_version 686164 (0.0007) [2023-12-27 10:22:34,390][20347] Updated weights for policy 0, policy_version 686174 (0.0008) [2023-12-27 10:22:34,453][20347] Updated weights for policy 0, policy_version 686184 (0.0007) [2023-12-27 10:22:34,861][20381] Updated weights for policy 1, policy_version 685759 (0.0007) [2023-12-27 10:22:34,907][20381] Updated weights for policy 1, policy_version 685769 (0.0008) [2023-12-27 10:22:34,952][20381] Updated weights for policy 1, policy_version 685779 (0.0007) [2023-12-27 10:22:35,120][20347] Updated weights for policy 0, policy_version 686194 (0.0005) [2023-12-27 10:22:35,166][20347] Updated weights for policy 0, policy_version 686204 (0.0006) [2023-12-27 10:22:35,222][20347] Updated weights for policy 0, policy_version 686214 (0.0007) [2023-12-27 10:22:35,659][20381] Updated weights for policy 1, policy_version 685789 (0.0009) [2023-12-27 10:22:35,709][20381] Updated weights for policy 1, policy_version 685799 (0.0009) [2023-12-27 10:22:35,766][20381] Updated weights for policy 1, policy_version 685809 (0.0005) [2023-12-27 10:22:35,886][20347] Updated weights for policy 0, policy_version 686224 (0.0006) [2023-12-27 10:22:35,945][20347] Updated weights for policy 0, policy_version 686234 (0.0005) [2023-12-27 10:22:35,995][20347] Updated weights for policy 0, policy_version 686244 (0.0009) [2023-12-27 10:22:36,366][20381] Updated weights for policy 1, policy_version 685819 (0.0006) [2023-12-27 10:22:36,426][20381] Updated weights for policy 1, policy_version 685829 (0.0007) [2023-12-27 10:22:36,488][20381] Updated weights for policy 1, policy_version 685839 (0.0006) [2023-12-27 10:22:36,659][20347] Updated weights for policy 0, policy_version 686254 (0.0009) [2023-12-27 10:22:36,727][20347] Updated weights for policy 0, policy_version 686264 (0.0007) [2023-12-27 10:22:36,784][20347] Updated weights for policy 0, policy_version 686274 (0.0008) [2023-12-27 10:22:37,175][20381] Updated weights for policy 1, policy_version 685849 (0.0008) [2023-12-27 10:22:37,236][20381] Updated weights for policy 1, policy_version 685859 (0.0008) [2023-12-27 10:22:37,298][20381] Updated weights for policy 1, policy_version 685869 (0.0008) [2023-12-27 10:22:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 351313920. Throughput: 0: 10368.0, 1: 10458.2. Samples: 351288940. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:37,363][19170] Avg episode reward: [(0, '835.210'), (1, '737.580')] [2023-12-27 10:22:37,364][20381] Updated weights for policy 1, policy_version 685879 (0.0008) [2023-12-27 10:22:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000685880_175611904.pth... [2023-12-27 10:22:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000684632_175292416.pth [2023-12-27 10:22:37,383][20347] Updated weights for policy 0, policy_version 686284 (0.0006) [2023-12-27 10:22:37,441][20347] Updated weights for policy 0, policy_version 686294 (0.0009) [2023-12-27 10:22:37,502][20347] Updated weights for policy 0, policy_version 686304 (0.0008) [2023-12-27 10:22:37,543][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000686312_175718400.pth... [2023-12-27 10:22:37,546][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000685064_175398912.pth [2023-12-27 10:22:38,015][20381] Updated weights for policy 1, policy_version 685889 (0.0006) [2023-12-27 10:22:38,075][20381] Updated weights for policy 1, policy_version 685899 (0.0006) [2023-12-27 10:22:38,135][20347] Updated weights for policy 0, policy_version 686314 (0.0007) [2023-12-27 10:22:38,137][20381] Updated weights for policy 1, policy_version 685909 (0.0008) [2023-12-27 10:22:38,190][20347] Updated weights for policy 0, policy_version 686324 (0.0006) [2023-12-27 10:22:38,244][20347] Updated weights for policy 0, policy_version 686334 (0.0008) [2023-12-27 10:22:38,289][20347] Updated weights for policy 0, policy_version 686344 (0.0008) [2023-12-27 10:22:38,785][20381] Updated weights for policy 1, policy_version 685919 (0.0009) [2023-12-27 10:22:38,847][20381] Updated weights for policy 1, policy_version 685929 (0.0011) [2023-12-27 10:22:38,891][20347] Updated weights for policy 0, policy_version 686354 (0.0007) [2023-12-27 10:22:38,900][20381] Updated weights for policy 1, policy_version 685939 (0.0008) [2023-12-27 10:22:38,939][20347] Updated weights for policy 0, policy_version 686364 (0.0006) [2023-12-27 10:22:38,987][20347] Updated weights for policy 0, policy_version 686374 (0.0007) [2023-12-27 10:22:39,570][20292] KL-divergence is very high: 136.5152 [2023-12-27 10:22:39,593][20381] Updated weights for policy 1, policy_version 685949 (0.0009) [2023-12-27 10:22:39,618][20292] KL-divergence is very high: 136.0009 [2023-12-27 10:22:39,652][20381] Updated weights for policy 1, policy_version 685959 (0.0007) [2023-12-27 10:22:39,685][20347] Updated weights for policy 0, policy_version 686384 (0.0006) [2023-12-27 10:22:39,708][20292] KL-divergence is very high: 124.9149 [2023-12-27 10:22:39,708][20381] Updated weights for policy 1, policy_version 685969 (0.0006) [2023-12-27 10:22:39,745][20347] Updated weights for policy 0, policy_version 686394 (0.0006) [2023-12-27 10:22:39,807][20347] Updated weights for policy 0, policy_version 686404 (0.0008) [2023-12-27 10:22:40,437][20381] Updated weights for policy 1, policy_version 685979 (0.0007) [2023-12-27 10:22:40,491][20381] Updated weights for policy 1, policy_version 685989 (0.0007) [2023-12-27 10:22:40,525][20347] Updated weights for policy 0, policy_version 686414 (0.0009) [2023-12-27 10:22:40,550][20381] Updated weights for policy 1, policy_version 685999 (0.0008) [2023-12-27 10:22:40,586][20347] Updated weights for policy 0, policy_version 686424 (0.0007) [2023-12-27 10:22:40,644][20347] Updated weights for policy 0, policy_version 686434 (0.0007) [2023-12-27 10:22:41,284][20381] Updated weights for policy 1, policy_version 686009 (0.0008) [2023-12-27 10:22:41,336][20347] Updated weights for policy 0, policy_version 686444 (0.0007) [2023-12-27 10:22:41,342][20381] Updated weights for policy 1, policy_version 686019 (0.0008) [2023-12-27 10:22:41,404][20347] Updated weights for policy 0, policy_version 686454 (0.0008) [2023-12-27 10:22:41,406][20381] Updated weights for policy 1, policy_version 686029 (0.0007) [2023-12-27 10:22:41,468][20381] Updated weights for policy 1, policy_version 686039 (0.0007) [2023-12-27 10:22:41,470][20347] Updated weights for policy 0, policy_version 686464 (0.0006) [2023-12-27 10:22:42,120][20347] Updated weights for policy 0, policy_version 686474 (0.0007) [2023-12-27 10:22:42,178][20347] Updated weights for policy 0, policy_version 686484 (0.0008) [2023-12-27 10:22:42,223][20381] Updated weights for policy 1, policy_version 686049 (0.0006) [2023-12-27 10:22:42,232][20347] Updated weights for policy 0, policy_version 686494 (0.0006) [2023-12-27 10:22:42,283][20381] Updated weights for policy 1, policy_version 686059 (0.0006) [2023-12-27 10:22:42,290][20347] Updated weights for policy 0, policy_version 686504 (0.0007) [2023-12-27 10:22:42,345][20381] Updated weights for policy 1, policy_version 686069 (0.0008) [2023-12-27 10:22:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 351420416. Throughput: 0: 10366.3, 1: 10443.3. Samples: 351412848. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:42,363][19170] Avg episode reward: [(0, '883.250'), (1, '724.040')] [2023-12-27 10:22:42,996][20347] Updated weights for policy 0, policy_version 686514 (0.0006) [2023-12-27 10:22:42,997][20381] Updated weights for policy 1, policy_version 686079 (0.0006) [2023-12-27 10:22:43,051][20347] Updated weights for policy 0, policy_version 686524 (0.0006) [2023-12-27 10:22:43,053][20381] Updated weights for policy 1, policy_version 686089 (0.0010) [2023-12-27 10:22:43,105][20381] Updated weights for policy 1, policy_version 686099 (0.0010) [2023-12-27 10:22:43,109][20347] Updated weights for policy 0, policy_version 686534 (0.0006) [2023-12-27 10:22:43,745][20381] Updated weights for policy 1, policy_version 686109 (0.0011) [2023-12-27 10:22:43,777][20347] Updated weights for policy 0, policy_version 686544 (0.0006) [2023-12-27 10:22:43,798][20381] Updated weights for policy 1, policy_version 686119 (0.0011) [2023-12-27 10:22:43,830][20347] Updated weights for policy 0, policy_version 686554 (0.0006) [2023-12-27 10:22:43,844][20381] Updated weights for policy 1, policy_version 686129 (0.0011) [2023-12-27 10:22:43,880][20347] Updated weights for policy 0, policy_version 686564 (0.0006) [2023-12-27 10:22:44,531][20381] Updated weights for policy 1, policy_version 686139 (0.0010) [2023-12-27 10:22:44,561][20347] Updated weights for policy 0, policy_version 686574 (0.0008) [2023-12-27 10:22:44,587][20381] Updated weights for policy 1, policy_version 686149 (0.0011) [2023-12-27 10:22:44,614][20347] Updated weights for policy 0, policy_version 686584 (0.0006) [2023-12-27 10:22:44,639][20381] Updated weights for policy 1, policy_version 686159 (0.0010) [2023-12-27 10:22:44,669][20347] Updated weights for policy 0, policy_version 686594 (0.0005) [2023-12-27 10:22:45,329][20381] Updated weights for policy 1, policy_version 686169 (0.0011) [2023-12-27 10:22:45,354][20347] Updated weights for policy 0, policy_version 686604 (0.0006) [2023-12-27 10:22:45,385][20381] Updated weights for policy 1, policy_version 686179 (0.0008) [2023-12-27 10:22:45,412][20347] Updated weights for policy 0, policy_version 686614 (0.0006) [2023-12-27 10:22:45,434][20381] Updated weights for policy 1, policy_version 686189 (0.0008) [2023-12-27 10:22:45,471][20347] Updated weights for policy 0, policy_version 686624 (0.0008) [2023-12-27 10:22:45,486][20381] Updated weights for policy 1, policy_version 686199 (0.0007) [2023-12-27 10:22:46,151][20347] Updated weights for policy 0, policy_version 686634 (0.0007) [2023-12-27 10:22:46,162][20381] Updated weights for policy 1, policy_version 686209 (0.0010) [2023-12-27 10:22:46,199][20347] Updated weights for policy 0, policy_version 686644 (0.0005) [2023-12-27 10:22:46,214][20381] Updated weights for policy 1, policy_version 686219 (0.0010) [2023-12-27 10:22:46,252][20347] Updated weights for policy 0, policy_version 686654 (0.0006) [2023-12-27 10:22:46,265][20381] Updated weights for policy 1, policy_version 686229 (0.0010) [2023-12-27 10:22:46,311][20347] Updated weights for policy 0, policy_version 686664 (0.0006) [2023-12-27 10:22:46,921][20381] Updated weights for policy 1, policy_version 686239 (0.0011) [2023-12-27 10:22:46,970][20381] Updated weights for policy 1, policy_version 686249 (0.0010) [2023-12-27 10:22:47,019][20347] Updated weights for policy 0, policy_version 686674 (0.0006) [2023-12-27 10:22:47,026][20381] Updated weights for policy 1, policy_version 686259 (0.0011) [2023-12-27 10:22:47,077][20347] Updated weights for policy 0, policy_version 686684 (0.0006) [2023-12-27 10:22:47,144][20347] Updated weights for policy 0, policy_version 686694 (0.0008) [2023-12-27 10:22:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20743.8). Total num frames: 351526912. Throughput: 0: 10338.4, 1: 10417.2. Samples: 351534932. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:47,364][19170] Avg episode reward: [(0, '854.320'), (1, '683.360')] [2023-12-27 10:22:47,726][20381] Updated weights for policy 1, policy_version 686269 (0.0011) [2023-12-27 10:22:47,747][20347] Updated weights for policy 0, policy_version 686704 (0.0008) [2023-12-27 10:22:47,778][20381] Updated weights for policy 1, policy_version 686279 (0.0010) [2023-12-27 10:22:47,805][20347] Updated weights for policy 0, policy_version 686714 (0.0005) [2023-12-27 10:22:47,832][20381] Updated weights for policy 1, policy_version 686289 (0.0009) [2023-12-27 10:22:47,859][20347] Updated weights for policy 0, policy_version 686724 (0.0007) [2023-12-27 10:22:48,474][20381] Updated weights for policy 1, policy_version 686299 (0.0006) [2023-12-27 10:22:48,521][20347] Updated weights for policy 0, policy_version 686734 (0.0007) [2023-12-27 10:22:48,530][20381] Updated weights for policy 1, policy_version 686309 (0.0008) [2023-12-27 10:22:48,573][20347] Updated weights for policy 0, policy_version 686744 (0.0007) [2023-12-27 10:22:48,577][20381] Updated weights for policy 1, policy_version 686319 (0.0009) [2023-12-27 10:22:48,625][20347] Updated weights for policy 0, policy_version 686754 (0.0006) [2023-12-27 10:22:49,193][20347] Updated weights for policy 0, policy_version 686764 (0.0006) [2023-12-27 10:22:49,255][20347] Updated weights for policy 0, policy_version 686774 (0.0007) [2023-12-27 10:22:49,266][20381] Updated weights for policy 1, policy_version 686329 (0.0008) [2023-12-27 10:22:49,318][20347] Updated weights for policy 0, policy_version 686784 (0.0008) [2023-12-27 10:22:49,327][20381] Updated weights for policy 1, policy_version 686339 (0.0006) [2023-12-27 10:22:49,384][20381] Updated weights for policy 1, policy_version 686349 (0.0008) [2023-12-27 10:22:49,442][20381] Updated weights for policy 1, policy_version 686359 (0.0009) [2023-12-27 10:22:49,875][20347] Updated weights for policy 0, policy_version 686794 (0.0007) [2023-12-27 10:22:49,945][20347] Updated weights for policy 0, policy_version 686804 (0.0007) [2023-12-27 10:22:50,004][20347] Updated weights for policy 0, policy_version 686814 (0.0005) [2023-12-27 10:22:50,065][20347] Updated weights for policy 0, policy_version 686824 (0.0007) [2023-12-27 10:22:50,135][20381] Updated weights for policy 1, policy_version 686369 (0.0010) [2023-12-27 10:22:50,188][20381] Updated weights for policy 1, policy_version 686379 (0.0011) [2023-12-27 10:22:50,246][20381] Updated weights for policy 1, policy_version 686389 (0.0010) [2023-12-27 10:22:50,689][20347] Updated weights for policy 0, policy_version 686834 (0.0009) [2023-12-27 10:22:50,736][20347] Updated weights for policy 0, policy_version 686844 (0.0010) [2023-12-27 10:22:50,788][20347] Updated weights for policy 0, policy_version 686854 (0.0010) [2023-12-27 10:22:50,943][20381] Updated weights for policy 1, policy_version 686399 (0.0010) [2023-12-27 10:22:50,999][20381] Updated weights for policy 1, policy_version 686409 (0.0011) [2023-12-27 10:22:51,062][20381] Updated weights for policy 1, policy_version 686419 (0.0011) [2023-12-27 10:22:51,512][20347] Updated weights for policy 0, policy_version 686864 (0.0010) [2023-12-27 10:22:51,566][20347] Updated weights for policy 0, policy_version 686874 (0.0010) [2023-12-27 10:22:51,634][20347] Updated weights for policy 0, policy_version 686884 (0.0008) [2023-12-27 10:22:51,738][20381] Updated weights for policy 1, policy_version 686429 (0.0007) [2023-12-27 10:22:51,791][20381] Updated weights for policy 1, policy_version 686439 (0.0009) [2023-12-27 10:22:51,849][20381] Updated weights for policy 1, policy_version 686449 (0.0008) [2023-12-27 10:22:52,307][20347] Updated weights for policy 0, policy_version 686894 (0.0007) [2023-12-27 10:22:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20716.0). Total num frames: 351625216. Throughput: 0: 10423.6, 1: 10400.7. Samples: 351599556. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:52,363][19170] Avg episode reward: [(0, '902.750'), (1, '805.120')] [2023-12-27 10:22:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000686456_175759360.pth... [2023-12-27 10:22:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000685240_175448064.pth [2023-12-27 10:22:52,372][20347] Updated weights for policy 0, policy_version 686904 (0.0008) [2023-12-27 10:22:52,428][20347] Updated weights for policy 0, policy_version 686914 (0.0007) [2023-12-27 10:22:52,465][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000686920_175874048.pth... [2023-12-27 10:22:52,469][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000685672_175554560.pth [2023-12-27 10:22:52,502][20381] Updated weights for policy 1, policy_version 686459 (0.0008) [2023-12-27 10:22:52,556][20381] Updated weights for policy 1, policy_version 686469 (0.0008) [2023-12-27 10:22:52,608][20381] Updated weights for policy 1, policy_version 686479 (0.0007) [2023-12-27 10:22:53,120][20347] Updated weights for policy 0, policy_version 686924 (0.0008) [2023-12-27 10:22:53,174][20347] Updated weights for policy 0, policy_version 686934 (0.0007) [2023-12-27 10:22:53,228][20347] Updated weights for policy 0, policy_version 686944 (0.0005) [2023-12-27 10:22:53,286][20381] Updated weights for policy 1, policy_version 686489 (0.0007) [2023-12-27 10:22:53,335][20381] Updated weights for policy 1, policy_version 686499 (0.0005) [2023-12-27 10:22:53,391][20381] Updated weights for policy 1, policy_version 686509 (0.0005) [2023-12-27 10:22:53,439][20381] Updated weights for policy 1, policy_version 686519 (0.0005) [2023-12-27 10:22:53,763][20347] Updated weights for policy 0, policy_version 686954 (0.0005) [2023-12-27 10:22:53,809][20347] Updated weights for policy 0, policy_version 686964 (0.0006) [2023-12-27 10:22:53,866][20347] Updated weights for policy 0, policy_version 686974 (0.0007) [2023-12-27 10:22:53,917][20347] Updated weights for policy 0, policy_version 686984 (0.0007) [2023-12-27 10:22:54,116][20381] Updated weights for policy 1, policy_version 686529 (0.0010) [2023-12-27 10:22:54,165][20381] Updated weights for policy 1, policy_version 686539 (0.0010) [2023-12-27 10:22:54,216][20381] Updated weights for policy 1, policy_version 686549 (0.0010) [2023-12-27 10:22:54,603][20347] Updated weights for policy 0, policy_version 686994 (0.0005) [2023-12-27 10:22:54,655][20347] Updated weights for policy 0, policy_version 687004 (0.0007) [2023-12-27 10:22:54,703][20347] Updated weights for policy 0, policy_version 687014 (0.0009) [2023-12-27 10:22:54,869][20381] Updated weights for policy 1, policy_version 686559 (0.0007) [2023-12-27 10:22:54,932][20381] Updated weights for policy 1, policy_version 686569 (0.0011) [2023-12-27 10:22:54,995][20381] Updated weights for policy 1, policy_version 686579 (0.0010) [2023-12-27 10:22:55,313][20347] Updated weights for policy 0, policy_version 687024 (0.0010) [2023-12-27 10:22:55,374][20347] Updated weights for policy 0, policy_version 687034 (0.0010) [2023-12-27 10:22:55,430][20347] Updated weights for policy 0, policy_version 687044 (0.0009) [2023-12-27 10:22:55,634][20381] Updated weights for policy 1, policy_version 686589 (0.0011) [2023-12-27 10:22:55,682][20381] Updated weights for policy 1, policy_version 686599 (0.0010) [2023-12-27 10:22:55,749][20381] Updated weights for policy 1, policy_version 686609 (0.0009) [2023-12-27 10:22:56,064][20347] Updated weights for policy 0, policy_version 687054 (0.0007) [2023-12-27 10:22:56,124][20347] Updated weights for policy 0, policy_version 687064 (0.0008) [2023-12-27 10:22:56,187][20347] Updated weights for policy 0, policy_version 687074 (0.0010) [2023-12-27 10:22:56,373][20381] Updated weights for policy 1, policy_version 686619 (0.0007) [2023-12-27 10:22:56,427][20381] Updated weights for policy 1, policy_version 686629 (0.0010) [2023-12-27 10:22:56,485][20381] Updated weights for policy 1, policy_version 686639 (0.0010) [2023-12-27 10:22:56,858][20347] Updated weights for policy 0, policy_version 687084 (0.0011) [2023-12-27 10:22:56,906][20347] Updated weights for policy 0, policy_version 687094 (0.0010) [2023-12-27 10:22:56,961][20347] Updated weights for policy 0, policy_version 687104 (0.0010) [2023-12-27 10:22:57,048][20381] Updated weights for policy 1, policy_version 686649 (0.0010) [2023-12-27 10:22:57,107][20381] Updated weights for policy 1, policy_version 686659 (0.0009) [2023-12-27 10:22:57,163][20381] Updated weights for policy 1, policy_version 686669 (0.0007) [2023-12-27 10:22:57,226][20381] Updated weights for policy 1, policy_version 686679 (0.0006) [2023-12-27 10:22:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 351739904. Throughput: 0: 10512.5, 1: 10430.2. Samples: 351725904. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:22:57,363][19170] Avg episode reward: [(0, '931.700'), (1, '873.880')] [2023-12-27 10:22:57,630][20347] Updated weights for policy 0, policy_version 687114 (0.0010) [2023-12-27 10:22:57,675][20347] Updated weights for policy 0, policy_version 687124 (0.0010) [2023-12-27 10:22:57,717][20347] Updated weights for policy 0, policy_version 687134 (0.0009) [2023-12-27 10:22:57,767][20347] Updated weights for policy 0, policy_version 687144 (0.0006) [2023-12-27 10:22:57,821][20381] Updated weights for policy 1, policy_version 686689 (0.0010) [2023-12-27 10:22:57,869][20381] Updated weights for policy 1, policy_version 686699 (0.0010) [2023-12-27 10:22:57,917][20381] Updated weights for policy 1, policy_version 686709 (0.0010) [2023-12-27 10:22:58,387][20347] Updated weights for policy 0, policy_version 687154 (0.0008) [2023-12-27 10:22:58,450][20347] Updated weights for policy 0, policy_version 687164 (0.0009) [2023-12-27 10:22:58,517][20347] Updated weights for policy 0, policy_version 687174 (0.0007) [2023-12-27 10:22:58,640][20381] Updated weights for policy 1, policy_version 686719 (0.0010) [2023-12-27 10:22:58,709][20381] Updated weights for policy 1, policy_version 686729 (0.0008) [2023-12-27 10:22:58,774][20381] Updated weights for policy 1, policy_version 686739 (0.0009) [2023-12-27 10:22:59,258][20347] Updated weights for policy 0, policy_version 687184 (0.0008) [2023-12-27 10:22:59,322][20347] Updated weights for policy 0, policy_version 687194 (0.0007) [2023-12-27 10:22:59,384][20347] Updated weights for policy 0, policy_version 687204 (0.0009) [2023-12-27 10:22:59,533][20381] Updated weights for policy 1, policy_version 686749 (0.0012) [2023-12-27 10:22:59,588][20381] Updated weights for policy 1, policy_version 686759 (0.0010) [2023-12-27 10:22:59,651][20381] Updated weights for policy 1, policy_version 686769 (0.0010) [2023-12-27 10:23:00,034][20347] Updated weights for policy 0, policy_version 687214 (0.0010) [2023-12-27 10:23:00,089][20347] Updated weights for policy 0, policy_version 687224 (0.0010) [2023-12-27 10:23:00,150][20347] Updated weights for policy 0, policy_version 687234 (0.0010) [2023-12-27 10:23:00,269][20381] Updated weights for policy 1, policy_version 686779 (0.0011) [2023-12-27 10:23:00,326][20381] Updated weights for policy 1, policy_version 686789 (0.0008) [2023-12-27 10:23:00,382][20381] Updated weights for policy 1, policy_version 686799 (0.0009) [2023-12-27 10:23:00,817][20347] Updated weights for policy 0, policy_version 687244 (0.0010) [2023-12-27 10:23:00,872][20347] Updated weights for policy 0, policy_version 687254 (0.0010) [2023-12-27 10:23:00,917][20347] Updated weights for policy 0, policy_version 687264 (0.0010) [2023-12-27 10:23:00,944][20381] Updated weights for policy 1, policy_version 686809 (0.0009) [2023-12-27 10:23:01,012][20381] Updated weights for policy 1, policy_version 686819 (0.0008) [2023-12-27 10:23:01,075][20381] Updated weights for policy 1, policy_version 686829 (0.0008) [2023-12-27 10:23:01,131][20381] Updated weights for policy 1, policy_version 686839 (0.0006) [2023-12-27 10:23:01,606][20347] Updated weights for policy 0, policy_version 687274 (0.0010) [2023-12-27 10:23:01,667][20347] Updated weights for policy 0, policy_version 687284 (0.0009) [2023-12-27 10:23:01,723][20347] Updated weights for policy 0, policy_version 687294 (0.0009) [2023-12-27 10:23:01,752][20381] Updated weights for policy 1, policy_version 686849 (0.0008) [2023-12-27 10:23:01,781][20347] Updated weights for policy 0, policy_version 687304 (0.0007) [2023-12-27 10:23:01,810][20381] Updated weights for policy 1, policy_version 686859 (0.0007) [2023-12-27 10:23:01,865][20381] Updated weights for policy 1, policy_version 686869 (0.0009) [2023-12-27 10:23:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 351838208. Throughput: 0: 10486.4, 1: 10486.3. Samples: 351851596. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:23:02,363][19170] Avg episode reward: [(0, '970.900'), (1, '893.410')] [2023-12-27 10:23:02,418][20347] Updated weights for policy 0, policy_version 687314 (0.0008) [2023-12-27 10:23:02,477][20347] Updated weights for policy 0, policy_version 687324 (0.0008) [2023-12-27 10:23:02,527][20347] Updated weights for policy 0, policy_version 687334 (0.0007) [2023-12-27 10:23:02,597][20381] Updated weights for policy 1, policy_version 686879 (0.0009) [2023-12-27 10:23:02,660][20381] Updated weights for policy 1, policy_version 686889 (0.0007) [2023-12-27 10:23:02,727][20381] Updated weights for policy 1, policy_version 686899 (0.0006) [2023-12-27 10:23:03,219][20347] Updated weights for policy 0, policy_version 687344 (0.0006) [2023-12-27 10:23:03,279][20347] Updated weights for policy 0, policy_version 687354 (0.0005) [2023-12-27 10:23:03,327][20347] Updated weights for policy 0, policy_version 687364 (0.0005) [2023-12-27 10:23:03,357][20381] Updated weights for policy 1, policy_version 686909 (0.0008) [2023-12-27 10:23:03,402][20381] Updated weights for policy 1, policy_version 686919 (0.0008) [2023-12-27 10:23:03,447][20381] Updated weights for policy 1, policy_version 686929 (0.0009) [2023-12-27 10:23:03,884][20347] Updated weights for policy 0, policy_version 687374 (0.0006) [2023-12-27 10:23:03,943][20347] Updated weights for policy 0, policy_version 687384 (0.0006) [2023-12-27 10:23:04,005][20347] Updated weights for policy 0, policy_version 687394 (0.0010) [2023-12-27 10:23:04,104][20381] Updated weights for policy 1, policy_version 686939 (0.0007) [2023-12-27 10:23:04,169][20381] Updated weights for policy 1, policy_version 686949 (0.0006) [2023-12-27 10:23:04,238][20381] Updated weights for policy 1, policy_version 686959 (0.0006) [2023-12-27 10:23:04,581][20347] Updated weights for policy 0, policy_version 687404 (0.0009) [2023-12-27 10:23:04,644][20347] Updated weights for policy 0, policy_version 687414 (0.0006) [2023-12-27 10:23:04,696][20347] Updated weights for policy 0, policy_version 687424 (0.0008) [2023-12-27 10:23:04,805][20381] Updated weights for policy 1, policy_version 686969 (0.0007) [2023-12-27 10:23:04,868][20381] Updated weights for policy 1, policy_version 686979 (0.0005) [2023-12-27 10:23:04,922][20381] Updated weights for policy 1, policy_version 686989 (0.0005) [2023-12-27 10:23:04,971][20381] Updated weights for policy 1, policy_version 686999 (0.0005) [2023-12-27 10:23:05,428][20347] Updated weights for policy 0, policy_version 687434 (0.0008) [2023-12-27 10:23:05,481][20381] Updated weights for policy 1, policy_version 687009 (0.0005) [2023-12-27 10:23:05,488][20347] Updated weights for policy 0, policy_version 687444 (0.0007) [2023-12-27 10:23:05,539][20347] Updated weights for policy 0, policy_version 687454 (0.0007) [2023-12-27 10:23:05,544][20381] Updated weights for policy 1, policy_version 687019 (0.0005) [2023-12-27 10:23:05,594][20347] Updated weights for policy 0, policy_version 687464 (0.0007) [2023-12-27 10:23:05,609][20381] Updated weights for policy 1, policy_version 687029 (0.0006) [2023-12-27 10:23:06,145][20381] Updated weights for policy 1, policy_version 687039 (0.0008) [2023-12-27 10:23:06,199][20381] Updated weights for policy 1, policy_version 687049 (0.0008) [2023-12-27 10:23:06,258][20381] Updated weights for policy 1, policy_version 687059 (0.0008) [2023-12-27 10:23:06,300][20347] Updated weights for policy 0, policy_version 687474 (0.0007) [2023-12-27 10:23:06,356][20347] Updated weights for policy 0, policy_version 687484 (0.0007) [2023-12-27 10:23:06,413][20347] Updated weights for policy 0, policy_version 687494 (0.0007) [2023-12-27 10:23:06,961][20381] Updated weights for policy 1, policy_version 687069 (0.0007) [2023-12-27 10:23:07,012][20381] Updated weights for policy 1, policy_version 687079 (0.0007) [2023-12-27 10:23:07,058][20381] Updated weights for policy 1, policy_version 687089 (0.0007) [2023-12-27 10:23:07,062][20347] Updated weights for policy 0, policy_version 687504 (0.0010) [2023-12-27 10:23:07,112][20347] Updated weights for policy 0, policy_version 687514 (0.0006) [2023-12-27 10:23:07,173][20347] Updated weights for policy 0, policy_version 687524 (0.0006) [2023-12-27 10:23:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 351952896. Throughput: 0: 10540.9, 1: 10531.5. Samples: 351917384. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:23:07,363][19170] Avg episode reward: [(0, '951.500'), (1, '931.780')] [2023-12-27 10:23:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000687096_175923200.pth... [2023-12-27 10:23:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000687528_176029696.pth... [2023-12-27 10:23:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000685880_175611904.pth [2023-12-27 10:23:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000686312_175718400.pth [2023-12-27 10:23:07,698][20381] Updated weights for policy 1, policy_version 687099 (0.0008) [2023-12-27 10:23:07,753][20381] Updated weights for policy 1, policy_version 687109 (0.0011) [2023-12-27 10:23:07,814][20381] Updated weights for policy 1, policy_version 687119 (0.0011) [2023-12-27 10:23:07,845][20347] Updated weights for policy 0, policy_version 687534 (0.0005) [2023-12-27 10:23:07,899][20347] Updated weights for policy 0, policy_version 687544 (0.0006) [2023-12-27 10:23:07,953][20347] Updated weights for policy 0, policy_version 687554 (0.0008) [2023-12-27 10:23:08,487][20381] Updated weights for policy 1, policy_version 687129 (0.0011) [2023-12-27 10:23:08,547][20381] Updated weights for policy 1, policy_version 687139 (0.0011) [2023-12-27 10:23:08,607][20381] Updated weights for policy 1, policy_version 687149 (0.0011) [2023-12-27 10:23:08,663][20381] Updated weights for policy 1, policy_version 687159 (0.0011) [2023-12-27 10:23:08,670][20347] Updated weights for policy 0, policy_version 687564 (0.0008) [2023-12-27 10:23:08,728][20347] Updated weights for policy 0, policy_version 687574 (0.0007) [2023-12-27 10:23:08,787][20347] Updated weights for policy 0, policy_version 687584 (0.0006) [2023-12-27 10:23:09,366][20381] Updated weights for policy 1, policy_version 687169 (0.0008) [2023-12-27 10:23:09,431][20381] Updated weights for policy 1, policy_version 687179 (0.0009) [2023-12-27 10:23:09,444][20347] Updated weights for policy 0, policy_version 687594 (0.0008) [2023-12-27 10:23:09,489][20381] Updated weights for policy 1, policy_version 687189 (0.0007) [2023-12-27 10:23:09,498][20347] Updated weights for policy 0, policy_version 687604 (0.0010) [2023-12-27 10:23:09,550][20347] Updated weights for policy 0, policy_version 687614 (0.0009) [2023-12-27 10:23:09,608][20347] Updated weights for policy 0, policy_version 687624 (0.0007) [2023-12-27 10:23:10,214][20381] Updated weights for policy 1, policy_version 687199 (0.0009) [2023-12-27 10:23:10,269][20347] Updated weights for policy 0, policy_version 687634 (0.0011) [2023-12-27 10:23:10,271][20381] Updated weights for policy 1, policy_version 687209 (0.0011) [2023-12-27 10:23:10,329][20347] Updated weights for policy 0, policy_version 687644 (0.0010) [2023-12-27 10:23:10,329][20381] Updated weights for policy 1, policy_version 687219 (0.0011) [2023-12-27 10:23:10,387][20347] Updated weights for policy 0, policy_version 687654 (0.0010) [2023-12-27 10:23:10,954][20381] Updated weights for policy 1, policy_version 687229 (0.0008) [2023-12-27 10:23:11,014][20381] Updated weights for policy 1, policy_version 687239 (0.0007) [2023-12-27 10:23:11,074][20347] Updated weights for policy 0, policy_version 687664 (0.0010) [2023-12-27 10:23:11,077][20381] Updated weights for policy 1, policy_version 687249 (0.0008) [2023-12-27 10:23:11,132][20347] Updated weights for policy 0, policy_version 687674 (0.0009) [2023-12-27 10:23:11,197][20347] Updated weights for policy 0, policy_version 687684 (0.0011) [2023-12-27 10:23:11,871][20381] Updated weights for policy 1, policy_version 687259 (0.0006) [2023-12-27 10:23:11,889][20347] Updated weights for policy 0, policy_version 687694 (0.0010) [2023-12-27 10:23:11,930][20381] Updated weights for policy 1, policy_version 687269 (0.0006) [2023-12-27 10:23:11,950][20347] Updated weights for policy 0, policy_version 687704 (0.0010) [2023-12-27 10:23:11,987][20381] Updated weights for policy 1, policy_version 687279 (0.0008) [2023-12-27 10:23:12,008][20347] Updated weights for policy 0, policy_version 687714 (0.0008) [2023-12-27 10:23:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.2, 300 sec: 20743.8). Total num frames: 352051200. Throughput: 0: 10485.2, 1: 10493.4. Samples: 352040340. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:23:12,363][19170] Avg episode reward: [(0, '941.900'), (1, '893.580')] [2023-12-27 10:23:12,693][20381] Updated weights for policy 1, policy_version 687289 (0.0007) [2023-12-27 10:23:12,718][20347] Updated weights for policy 0, policy_version 687724 (0.0008) [2023-12-27 10:23:12,746][20381] Updated weights for policy 1, policy_version 687299 (0.0008) [2023-12-27 10:23:12,778][20347] Updated weights for policy 0, policy_version 687734 (0.0008) [2023-12-27 10:23:12,803][20381] Updated weights for policy 1, policy_version 687309 (0.0009) [2023-12-27 10:23:12,835][20347] Updated weights for policy 0, policy_version 687744 (0.0006) [2023-12-27 10:23:12,865][20381] Updated weights for policy 1, policy_version 687319 (0.0011) [2023-12-27 10:23:13,460][20381] Updated weights for policy 1, policy_version 687329 (0.0006) [2023-12-27 10:23:13,504][20347] Updated weights for policy 0, policy_version 687754 (0.0008) [2023-12-27 10:23:13,521][20381] Updated weights for policy 1, policy_version 687339 (0.0005) [2023-12-27 10:23:13,559][20347] Updated weights for policy 0, policy_version 687764 (0.0007) [2023-12-27 10:23:13,579][20381] Updated weights for policy 1, policy_version 687349 (0.0011) [2023-12-27 10:23:13,614][20347] Updated weights for policy 0, policy_version 687774 (0.0006) [2023-12-27 10:23:13,670][20347] Updated weights for policy 0, policy_version 687784 (0.0007) [2023-12-27 10:23:14,234][20381] Updated weights for policy 1, policy_version 687359 (0.0008) [2023-12-27 10:23:14,268][20347] Updated weights for policy 0, policy_version 687794 (0.0007) [2023-12-27 10:23:14,300][20381] Updated weights for policy 1, policy_version 687369 (0.0006) [2023-12-27 10:23:14,328][20347] Updated weights for policy 0, policy_version 687804 (0.0007) [2023-12-27 10:23:14,361][20381] Updated weights for policy 1, policy_version 687379 (0.0006) [2023-12-27 10:23:14,386][20347] Updated weights for policy 0, policy_version 687814 (0.0007) [2023-12-27 10:23:14,977][20381] Updated weights for policy 1, policy_version 687389 (0.0007) [2023-12-27 10:23:15,034][20381] Updated weights for policy 1, policy_version 687399 (0.0006) [2023-12-27 10:23:15,076][20347] Updated weights for policy 0, policy_version 687824 (0.0008) [2023-12-27 10:23:15,086][20381] Updated weights for policy 1, policy_version 687409 (0.0006) [2023-12-27 10:23:15,141][20347] Updated weights for policy 0, policy_version 687834 (0.0009) [2023-12-27 10:23:15,210][20347] Updated weights for policy 0, policy_version 687844 (0.0008) [2023-12-27 10:23:15,770][20347] Updated weights for policy 0, policy_version 687854 (0.0006) [2023-12-27 10:23:15,805][20381] Updated weights for policy 1, policy_version 687419 (0.0006) [2023-12-27 10:23:15,826][20347] Updated weights for policy 0, policy_version 687864 (0.0005) [2023-12-27 10:23:15,856][20381] Updated weights for policy 1, policy_version 687429 (0.0007) [2023-12-27 10:23:15,884][20347] Updated weights for policy 0, policy_version 687874 (0.0005) [2023-12-27 10:23:15,912][20381] Updated weights for policy 1, policy_version 687439 (0.0008) [2023-12-27 10:23:16,418][20347] Updated weights for policy 0, policy_version 687884 (0.0005) [2023-12-27 10:23:16,475][20347] Updated weights for policy 0, policy_version 687894 (0.0006) [2023-12-27 10:23:16,533][20347] Updated weights for policy 0, policy_version 687904 (0.0010) [2023-12-27 10:23:16,672][20381] Updated weights for policy 1, policy_version 687449 (0.0010) [2023-12-27 10:23:16,727][20381] Updated weights for policy 1, policy_version 687459 (0.0008) [2023-12-27 10:23:16,776][20381] Updated weights for policy 1, policy_version 687469 (0.0008) [2023-12-27 10:23:16,832][20381] Updated weights for policy 1, policy_version 687479 (0.0007) [2023-12-27 10:23:17,080][20347] Updated weights for policy 0, policy_version 687914 (0.0010) [2023-12-27 10:23:17,132][20347] Updated weights for policy 0, policy_version 687924 (0.0010) [2023-12-27 10:23:17,186][20347] Updated weights for policy 0, policy_version 687934 (0.0010) [2023-12-27 10:23:17,240][20347] Updated weights for policy 0, policy_version 687944 (0.0010) [2023-12-27 10:23:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 21026.2, 300 sec: 20771.6). Total num frames: 352157696. Throughput: 0: 10518.5, 1: 10412.3. Samples: 352166308. Policy #0 lag: (min: 7.0, avg: 9.3, max: 39.0) [2023-12-27 10:23:17,363][19170] Avg episode reward: [(0, '971.000'), (1, '786.320')] [2023-12-27 10:23:17,486][20381] Updated weights for policy 1, policy_version 687489 (0.0007) [2023-12-27 10:23:17,537][20381] Updated weights for policy 1, policy_version 687499 (0.0009) [2023-12-27 10:23:17,594][20381] Updated weights for policy 1, policy_version 687509 (0.0008) [2023-12-27 10:23:17,860][20347] Updated weights for policy 0, policy_version 687954 (0.0005) [2023-12-27 10:23:17,916][20347] Updated weights for policy 0, policy_version 687964 (0.0005) [2023-12-27 10:23:17,965][20347] Updated weights for policy 0, policy_version 687974 (0.0008) [2023-12-27 10:23:18,353][20381] Updated weights for policy 1, policy_version 687519 (0.0007) [2023-12-27 10:23:18,412][20381] Updated weights for policy 1, policy_version 687529 (0.0007) [2023-12-27 10:23:18,469][20381] Updated weights for policy 1, policy_version 687539 (0.0008) [2023-12-27 10:23:18,556][20347] Updated weights for policy 0, policy_version 687984 (0.0006) [2023-12-27 10:23:18,606][20347] Updated weights for policy 0, policy_version 687994 (0.0006) [2023-12-27 10:23:18,666][20347] Updated weights for policy 0, policy_version 688004 (0.0008) [2023-12-27 10:23:19,168][20381] Updated weights for policy 1, policy_version 687549 (0.0007) [2023-12-27 10:23:19,219][20381] Updated weights for policy 1, policy_version 687559 (0.0009) [2023-12-27 10:23:19,282][20381] Updated weights for policy 1, policy_version 687569 (0.0008) [2023-12-27 10:23:19,372][20347] Updated weights for policy 0, policy_version 688014 (0.0008) [2023-12-27 10:23:19,435][20347] Updated weights for policy 0, policy_version 688024 (0.0005) [2023-12-27 10:23:19,492][20347] Updated weights for policy 0, policy_version 688034 (0.0007) [2023-12-27 10:23:20,013][20381] Updated weights for policy 1, policy_version 687579 (0.0008) [2023-12-27 10:23:20,072][20381] Updated weights for policy 1, policy_version 687589 (0.0008) [2023-12-27 10:23:20,101][20347] Updated weights for policy 0, policy_version 688044 (0.0007) [2023-12-27 10:23:20,127][20381] Updated weights for policy 1, policy_version 687599 (0.0006) [2023-12-27 10:23:20,158][20347] Updated weights for policy 0, policy_version 688054 (0.0008) [2023-12-27 10:23:20,219][20347] Updated weights for policy 0, policy_version 688064 (0.0005) [2023-12-27 10:23:20,782][20381] Updated weights for policy 1, policy_version 687609 (0.0007) [2023-12-27 10:23:20,837][20381] Updated weights for policy 1, policy_version 687619 (0.0008) [2023-12-27 10:23:20,867][20347] Updated weights for policy 0, policy_version 688074 (0.0007) [2023-12-27 10:23:20,899][20381] Updated weights for policy 1, policy_version 687629 (0.0007) [2023-12-27 10:23:20,920][20347] Updated weights for policy 0, policy_version 688084 (0.0006) [2023-12-27 10:23:20,925][20291] KL-divergence is very high: 172.5953 [2023-12-27 10:23:20,960][20381] Updated weights for policy 1, policy_version 687639 (0.0005) [2023-12-27 10:23:20,977][20347] Updated weights for policy 0, policy_version 688094 (0.0007) [2023-12-27 10:23:21,030][20347] Updated weights for policy 0, policy_version 688104 (0.0008) [2023-12-27 10:23:21,643][20381] Updated weights for policy 1, policy_version 687649 (0.0008) [2023-12-27 10:23:21,708][20381] Updated weights for policy 1, policy_version 687659 (0.0008) [2023-12-27 10:23:21,765][20381] Updated weights for policy 1, policy_version 687669 (0.0008) [2023-12-27 10:23:21,767][20347] Updated weights for policy 0, policy_version 688114 (0.0007) [2023-12-27 10:23:21,828][20347] Updated weights for policy 0, policy_version 688124 (0.0007) [2023-12-27 10:23:21,887][20347] Updated weights for policy 0, policy_version 688134 (0.0007) [2023-12-27 10:23:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20716.0). Total num frames: 352256000. Throughput: 0: 10538.4, 1: 10326.8. Samples: 352227876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:22,363][19170] Avg episode reward: [(0, '971.450'), (1, '844.080')] [2023-12-27 10:23:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000687672_176070656.pth... [2023-12-27 10:23:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000688136_176185344.pth... [2023-12-27 10:23:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000686456_175759360.pth [2023-12-27 10:23:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000686920_175874048.pth [2023-12-27 10:23:22,453][20381] Updated weights for policy 1, policy_version 687679 (0.0007) [2023-12-27 10:23:22,516][20381] Updated weights for policy 1, policy_version 687689 (0.0008) [2023-12-27 10:23:22,572][20381] Updated weights for policy 1, policy_version 687699 (0.0007) [2023-12-27 10:23:22,576][20347] Updated weights for policy 0, policy_version 688144 (0.0006) [2023-12-27 10:23:22,630][20347] Updated weights for policy 0, policy_version 688154 (0.0007) [2023-12-27 10:23:22,692][20347] Updated weights for policy 0, policy_version 688164 (0.0007) [2023-12-27 10:23:23,279][20381] Updated weights for policy 1, policy_version 687709 (0.0008) [2023-12-27 10:23:23,348][20381] Updated weights for policy 1, policy_version 687719 (0.0007) [2023-12-27 10:23:23,358][20347] Updated weights for policy 0, policy_version 688174 (0.0009) [2023-12-27 10:23:23,410][20381] Updated weights for policy 1, policy_version 687729 (0.0006) [2023-12-27 10:23:23,419][20347] Updated weights for policy 0, policy_version 688184 (0.0009) [2023-12-27 10:23:23,481][20347] Updated weights for policy 0, policy_version 688194 (0.0010) [2023-12-27 10:23:24,072][20381] Updated weights for policy 1, policy_version 687739 (0.0007) [2023-12-27 10:23:24,085][20347] Updated weights for policy 0, policy_version 688204 (0.0008) [2023-12-27 10:23:24,121][20381] Updated weights for policy 1, policy_version 687749 (0.0010) [2023-12-27 10:23:24,144][20347] Updated weights for policy 0, policy_version 688214 (0.0008) [2023-12-27 10:23:24,173][20381] Updated weights for policy 1, policy_version 687759 (0.0010) [2023-12-27 10:23:24,203][20347] Updated weights for policy 0, policy_version 688224 (0.0007) [2023-12-27 10:23:24,816][20381] Updated weights for policy 1, policy_version 687769 (0.0006) [2023-12-27 10:23:24,817][20347] Updated weights for policy 0, policy_version 688234 (0.0005) [2023-12-27 10:23:24,868][20381] Updated weights for policy 1, policy_version 687779 (0.0011) [2023-12-27 10:23:24,868][20347] Updated weights for policy 0, policy_version 688244 (0.0009) [2023-12-27 10:23:24,911][20381] Updated weights for policy 1, policy_version 687789 (0.0006) [2023-12-27 10:23:24,926][20347] Updated weights for policy 0, policy_version 688254 (0.0010) [2023-12-27 10:23:24,957][20381] Updated weights for policy 1, policy_version 687799 (0.0005) [2023-12-27 10:23:24,977][20347] Updated weights for policy 0, policy_version 688264 (0.0010) [2023-12-27 10:23:25,523][20381] Updated weights for policy 1, policy_version 687809 (0.0005) [2023-12-27 10:23:25,571][20381] Updated weights for policy 1, policy_version 687819 (0.0005) [2023-12-27 10:23:25,626][20381] Updated weights for policy 1, policy_version 687829 (0.0008) [2023-12-27 10:23:25,644][20347] Updated weights for policy 0, policy_version 688274 (0.0010) [2023-12-27 10:23:25,702][20347] Updated weights for policy 0, policy_version 688284 (0.0008) [2023-12-27 10:23:25,762][20347] Updated weights for policy 0, policy_version 688294 (0.0010) [2023-12-27 10:23:26,261][20381] Updated weights for policy 1, policy_version 687839 (0.0005) [2023-12-27 10:23:26,329][20381] Updated weights for policy 1, policy_version 687849 (0.0007) [2023-12-27 10:23:26,380][20381] Updated weights for policy 1, policy_version 687859 (0.0005) [2023-12-27 10:23:26,427][20347] Updated weights for policy 0, policy_version 688304 (0.0010) [2023-12-27 10:23:26,475][20347] Updated weights for policy 0, policy_version 688314 (0.0010) [2023-12-27 10:23:26,535][20347] Updated weights for policy 0, policy_version 688324 (0.0009) [2023-12-27 10:23:26,973][20381] Updated weights for policy 1, policy_version 687869 (0.0007) [2023-12-27 10:23:27,017][20381] Updated weights for policy 1, policy_version 687879 (0.0008) [2023-12-27 10:23:27,068][20381] Updated weights for policy 1, policy_version 687889 (0.0008) [2023-12-27 10:23:27,132][20347] Updated weights for policy 0, policy_version 688334 (0.0007) [2023-12-27 10:23:27,194][20347] Updated weights for policy 0, policy_version 688344 (0.0010) [2023-12-27 10:23:27,248][20347] Updated weights for policy 0, policy_version 688354 (0.0010) [2023-12-27 10:23:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 21162.6, 300 sec: 20771.6). Total num frames: 352370688. Throughput: 0: 10519.0, 1: 10392.7. Samples: 352353876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:27,363][19170] Avg episode reward: [(0, '952.040'), (1, '893.620')] [2023-12-27 10:23:27,715][20381] Updated weights for policy 1, policy_version 687899 (0.0010) [2023-12-27 10:23:27,775][20381] Updated weights for policy 1, policy_version 687909 (0.0010) [2023-12-27 10:23:27,829][20381] Updated weights for policy 1, policy_version 687919 (0.0010) [2023-12-27 10:23:27,862][20347] Updated weights for policy 0, policy_version 688364 (0.0010) [2023-12-27 10:23:27,907][20347] Updated weights for policy 0, policy_version 688374 (0.0010) [2023-12-27 10:23:27,954][20347] Updated weights for policy 0, policy_version 688384 (0.0010) [2023-12-27 10:23:28,398][20381] Updated weights for policy 1, policy_version 687929 (0.0010) [2023-12-27 10:23:28,455][20381] Updated weights for policy 1, policy_version 687939 (0.0010) [2023-12-27 10:23:28,517][20381] Updated weights for policy 1, policy_version 687949 (0.0009) [2023-12-27 10:23:28,573][20381] Updated weights for policy 1, policy_version 687959 (0.0005) [2023-12-27 10:23:28,623][20347] Updated weights for policy 0, policy_version 688394 (0.0010) [2023-12-27 10:23:28,685][20347] Updated weights for policy 0, policy_version 688404 (0.0011) [2023-12-27 10:23:28,744][20347] Updated weights for policy 0, policy_version 688414 (0.0010) [2023-12-27 10:23:28,802][20347] Updated weights for policy 0, policy_version 688424 (0.0010) [2023-12-27 10:23:29,205][20381] Updated weights for policy 1, policy_version 687969 (0.0010) [2023-12-27 10:23:29,267][20381] Updated weights for policy 1, policy_version 687979 (0.0011) [2023-12-27 10:23:29,320][20381] Updated weights for policy 1, policy_version 687989 (0.0008) [2023-12-27 10:23:29,465][20347] Updated weights for policy 0, policy_version 688434 (0.0010) [2023-12-27 10:23:29,513][20347] Updated weights for policy 0, policy_version 688444 (0.0010) [2023-12-27 10:23:29,558][20347] Updated weights for policy 0, policy_version 688454 (0.0010) [2023-12-27 10:23:30,036][20381] Updated weights for policy 1, policy_version 687999 (0.0006) [2023-12-27 10:23:30,086][20381] Updated weights for policy 1, policy_version 688009 (0.0007) [2023-12-27 10:23:30,137][20381] Updated weights for policy 1, policy_version 688019 (0.0007) [2023-12-27 10:23:30,286][20347] Updated weights for policy 0, policy_version 688464 (0.0010) [2023-12-27 10:23:30,346][20347] Updated weights for policy 0, policy_version 688474 (0.0010) [2023-12-27 10:23:30,409][20347] Updated weights for policy 0, policy_version 688484 (0.0010) [2023-12-27 10:23:30,772][20381] Updated weights for policy 1, policy_version 688029 (0.0007) [2023-12-27 10:23:30,828][20381] Updated weights for policy 1, policy_version 688039 (0.0007) [2023-12-27 10:23:30,877][20381] Updated weights for policy 1, policy_version 688049 (0.0006) [2023-12-27 10:23:31,082][20347] Updated weights for policy 0, policy_version 688494 (0.0010) [2023-12-27 10:23:31,145][20347] Updated weights for policy 0, policy_version 688504 (0.0011) [2023-12-27 10:23:31,203][20347] Updated weights for policy 0, policy_version 688514 (0.0010) [2023-12-27 10:23:31,589][20381] Updated weights for policy 1, policy_version 688059 (0.0007) [2023-12-27 10:23:31,646][20381] Updated weights for policy 1, policy_version 688069 (0.0008) [2023-12-27 10:23:31,708][20381] Updated weights for policy 1, policy_version 688079 (0.0008) [2023-12-27 10:23:31,875][20347] Updated weights for policy 0, policy_version 688524 (0.0009) [2023-12-27 10:23:31,924][20347] Updated weights for policy 0, policy_version 688534 (0.0007) [2023-12-27 10:23:31,976][20347] Updated weights for policy 0, policy_version 688544 (0.0008) [2023-12-27 10:23:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 352468992. Throughput: 0: 10549.6, 1: 10457.2. Samples: 352480232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:32,363][19170] Avg episode reward: [(0, '980.430'), (1, '922.950')] [2023-12-27 10:23:32,372][20381] Updated weights for policy 1, policy_version 688089 (0.0006) [2023-12-27 10:23:32,426][20381] Updated weights for policy 1, policy_version 688099 (0.0008) [2023-12-27 10:23:32,476][20381] Updated weights for policy 1, policy_version 688109 (0.0007) [2023-12-27 10:23:32,531][20381] Updated weights for policy 1, policy_version 688119 (0.0005) [2023-12-27 10:23:32,653][20347] Updated weights for policy 0, policy_version 688554 (0.0007) [2023-12-27 10:23:32,713][20347] Updated weights for policy 0, policy_version 688564 (0.0005) [2023-12-27 10:23:32,777][20347] Updated weights for policy 0, policy_version 688574 (0.0006) [2023-12-27 10:23:32,834][20347] Updated weights for policy 0, policy_version 688584 (0.0008) [2023-12-27 10:23:33,170][20381] Updated weights for policy 1, policy_version 688129 (0.0007) [2023-12-27 10:23:33,222][20381] Updated weights for policy 1, policy_version 688139 (0.0007) [2023-12-27 10:23:33,271][20381] Updated weights for policy 1, policy_version 688149 (0.0007) [2023-12-27 10:23:33,454][20347] Updated weights for policy 0, policy_version 688594 (0.0007) [2023-12-27 10:23:33,513][20347] Updated weights for policy 0, policy_version 688604 (0.0007) [2023-12-27 10:23:33,561][20347] Updated weights for policy 0, policy_version 688614 (0.0009) [2023-12-27 10:23:33,878][20381] Updated weights for policy 1, policy_version 688159 (0.0006) [2023-12-27 10:23:33,935][20381] Updated weights for policy 1, policy_version 688169 (0.0005) [2023-12-27 10:23:33,985][20381] Updated weights for policy 1, policy_version 688179 (0.0005) [2023-12-27 10:23:34,242][20347] Updated weights for policy 0, policy_version 688624 (0.0006) [2023-12-27 10:23:34,304][20347] Updated weights for policy 0, policy_version 688634 (0.0008) [2023-12-27 10:23:34,363][20347] Updated weights for policy 0, policy_version 688644 (0.0008) [2023-12-27 10:23:34,554][20381] Updated weights for policy 1, policy_version 688189 (0.0006) [2023-12-27 10:23:34,615][20381] Updated weights for policy 1, policy_version 688199 (0.0006) [2023-12-27 10:23:34,673][20381] Updated weights for policy 1, policy_version 688209 (0.0008) [2023-12-27 10:23:35,085][20347] Updated weights for policy 0, policy_version 688654 (0.0009) [2023-12-27 10:23:35,139][20347] Updated weights for policy 0, policy_version 688664 (0.0006) [2023-12-27 10:23:35,196][20347] Updated weights for policy 0, policy_version 688674 (0.0006) [2023-12-27 10:23:35,305][20381] Updated weights for policy 1, policy_version 688219 (0.0007) [2023-12-27 10:23:35,358][20381] Updated weights for policy 1, policy_version 688229 (0.0005) [2023-12-27 10:23:35,416][20381] Updated weights for policy 1, policy_version 688239 (0.0005) [2023-12-27 10:23:35,733][20347] Updated weights for policy 0, policy_version 688684 (0.0006) [2023-12-27 10:23:35,791][20347] Updated weights for policy 0, policy_version 688694 (0.0005) [2023-12-27 10:23:35,851][20347] Updated weights for policy 0, policy_version 688704 (0.0005) [2023-12-27 10:23:36,086][20381] Updated weights for policy 1, policy_version 688249 (0.0006) [2023-12-27 10:23:36,143][20381] Updated weights for policy 1, policy_version 688259 (0.0007) [2023-12-27 10:23:36,196][20381] Updated weights for policy 1, policy_version 688269 (0.0007) [2023-12-27 10:23:36,257][20381] Updated weights for policy 1, policy_version 688279 (0.0007) [2023-12-27 10:23:36,440][20347] Updated weights for policy 0, policy_version 688714 (0.0006) [2023-12-27 10:23:36,506][20347] Updated weights for policy 0, policy_version 688724 (0.0008) [2023-12-27 10:23:36,561][20347] Updated weights for policy 0, policy_version 688734 (0.0008) [2023-12-27 10:23:36,614][20347] Updated weights for policy 0, policy_version 688744 (0.0007) [2023-12-27 10:23:36,942][20381] Updated weights for policy 1, policy_version 688289 (0.0007) [2023-12-27 10:23:36,997][20381] Updated weights for policy 1, policy_version 688299 (0.0007) [2023-12-27 10:23:37,049][20381] Updated weights for policy 1, policy_version 688309 (0.0008) [2023-12-27 10:23:37,304][20347] Updated weights for policy 0, policy_version 688754 (0.0008) [2023-12-27 10:23:37,361][20347] Updated weights for policy 0, policy_version 688764 (0.0008) [2023-12-27 10:23:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 352575488. Throughput: 0: 10498.4, 1: 10512.4. Samples: 352545040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:37,363][19170] Avg episode reward: [(0, '990.210'), (1, '941.810')] [2023-12-27 10:23:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000688312_176234496.pth... [2023-12-27 10:23:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000687096_175923200.pth [2023-12-27 10:23:37,417][20347] Updated weights for policy 0, policy_version 688774 (0.0007) [2023-12-27 10:23:37,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000688776_176349184.pth... [2023-12-27 10:23:37,429][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000687528_176029696.pth [2023-12-27 10:23:37,721][20381] Updated weights for policy 1, policy_version 688319 (0.0007) [2023-12-27 10:23:37,778][20381] Updated weights for policy 1, policy_version 688329 (0.0007) [2023-12-27 10:23:37,833][20381] Updated weights for policy 1, policy_version 688339 (0.0007) [2023-12-27 10:23:38,127][20347] Updated weights for policy 0, policy_version 688784 (0.0008) [2023-12-27 10:23:38,180][20347] Updated weights for policy 0, policy_version 688794 (0.0008) [2023-12-27 10:23:38,237][20347] Updated weights for policy 0, policy_version 688804 (0.0007) [2023-12-27 10:23:38,503][20381] Updated weights for policy 1, policy_version 688349 (0.0006) [2023-12-27 10:23:38,568][20381] Updated weights for policy 1, policy_version 688359 (0.0006) [2023-12-27 10:23:38,624][20381] Updated weights for policy 1, policy_version 688369 (0.0007) [2023-12-27 10:23:38,949][20347] Updated weights for policy 0, policy_version 688814 (0.0007) [2023-12-27 10:23:39,008][20347] Updated weights for policy 0, policy_version 688824 (0.0005) [2023-12-27 10:23:39,061][20347] Updated weights for policy 0, policy_version 688834 (0.0006) [2023-12-27 10:23:39,243][20381] Updated weights for policy 1, policy_version 688379 (0.0008) [2023-12-27 10:23:39,301][20381] Updated weights for policy 1, policy_version 688389 (0.0006) [2023-12-27 10:23:39,364][20381] Updated weights for policy 1, policy_version 688399 (0.0009) [2023-12-27 10:23:39,741][20347] Updated weights for policy 0, policy_version 688844 (0.0008) [2023-12-27 10:23:39,788][20347] Updated weights for policy 0, policy_version 688854 (0.0008) [2023-12-27 10:23:39,845][20347] Updated weights for policy 0, policy_version 688864 (0.0007) [2023-12-27 10:23:40,136][20381] Updated weights for policy 1, policy_version 688409 (0.0007) [2023-12-27 10:23:40,205][20381] Updated weights for policy 1, policy_version 688419 (0.0009) [2023-12-27 10:23:40,268][20381] Updated weights for policy 1, policy_version 688429 (0.0008) [2023-12-27 10:23:40,335][20381] Updated weights for policy 1, policy_version 688439 (0.0008) [2023-12-27 10:23:40,506][20347] Updated weights for policy 0, policy_version 688874 (0.0006) [2023-12-27 10:23:40,568][20347] Updated weights for policy 0, policy_version 688884 (0.0007) [2023-12-27 10:23:40,630][20347] Updated weights for policy 0, policy_version 688894 (0.0008) [2023-12-27 10:23:40,689][20347] Updated weights for policy 0, policy_version 688904 (0.0007) [2023-12-27 10:23:40,945][20381] Updated weights for policy 1, policy_version 688449 (0.0006) [2023-12-27 10:23:40,994][20381] Updated weights for policy 1, policy_version 688459 (0.0005) [2023-12-27 10:23:41,055][20381] Updated weights for policy 1, policy_version 688469 (0.0006) [2023-12-27 10:23:41,410][20347] Updated weights for policy 0, policy_version 688914 (0.0009) [2023-12-27 10:23:41,469][20347] Updated weights for policy 0, policy_version 688924 (0.0008) [2023-12-27 10:23:41,529][20347] Updated weights for policy 0, policy_version 688934 (0.0008) [2023-12-27 10:23:41,752][20381] Updated weights for policy 1, policy_version 688479 (0.0008) [2023-12-27 10:23:41,807][20381] Updated weights for policy 1, policy_version 688489 (0.0007) [2023-12-27 10:23:41,865][20381] Updated weights for policy 1, policy_version 688499 (0.0007) [2023-12-27 10:23:42,287][20347] Updated weights for policy 0, policy_version 688944 (0.0008) [2023-12-27 10:23:42,344][20347] Updated weights for policy 0, policy_version 688954 (0.0009) [2023-12-27 10:23:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20743.8). Total num frames: 352673792. Throughput: 0: 10470.9, 1: 10485.7. Samples: 352668952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:42,363][19170] Avg episode reward: [(0, '980.490'), (1, '922.280')] [2023-12-27 10:23:42,405][20347] Updated weights for policy 0, policy_version 688964 (0.0007) [2023-12-27 10:23:42,552][20381] Updated weights for policy 1, policy_version 688509 (0.0006) [2023-12-27 10:23:42,607][20381] Updated weights for policy 1, policy_version 688519 (0.0008) [2023-12-27 10:23:42,666][20381] Updated weights for policy 1, policy_version 688529 (0.0008) [2023-12-27 10:23:43,142][20347] Updated weights for policy 0, policy_version 688974 (0.0007) [2023-12-27 10:23:43,202][20347] Updated weights for policy 0, policy_version 688984 (0.0008) [2023-12-27 10:23:43,260][20347] Updated weights for policy 0, policy_version 688994 (0.0007) [2023-12-27 10:23:43,328][20381] Updated weights for policy 1, policy_version 688539 (0.0007) [2023-12-27 10:23:43,388][20381] Updated weights for policy 1, policy_version 688549 (0.0008) [2023-12-27 10:23:43,444][20381] Updated weights for policy 1, policy_version 688559 (0.0008) [2023-12-27 10:23:43,951][20347] Updated weights for policy 0, policy_version 689004 (0.0007) [2023-12-27 10:23:44,010][20347] Updated weights for policy 0, policy_version 689014 (0.0007) [2023-12-27 10:23:44,068][20347] Updated weights for policy 0, policy_version 689024 (0.0008) [2023-12-27 10:23:44,128][20381] Updated weights for policy 1, policy_version 688569 (0.0007) [2023-12-27 10:23:44,176][20381] Updated weights for policy 1, policy_version 688579 (0.0008) [2023-12-27 10:23:44,225][20381] Updated weights for policy 1, policy_version 688589 (0.0007) [2023-12-27 10:23:44,269][20381] Updated weights for policy 1, policy_version 688599 (0.0008) [2023-12-27 10:23:44,677][20347] Updated weights for policy 0, policy_version 689034 (0.0008) [2023-12-27 10:23:44,726][20347] Updated weights for policy 0, policy_version 689044 (0.0006) [2023-12-27 10:23:44,792][20347] Updated weights for policy 0, policy_version 689054 (0.0007) [2023-12-27 10:23:44,843][20347] Updated weights for policy 0, policy_version 689064 (0.0008) [2023-12-27 10:23:45,056][20381] Updated weights for policy 1, policy_version 688609 (0.0008) [2023-12-27 10:23:45,105][20381] Updated weights for policy 1, policy_version 688619 (0.0007) [2023-12-27 10:23:45,155][20381] Updated weights for policy 1, policy_version 688629 (0.0008) [2023-12-27 10:23:45,549][20347] Updated weights for policy 0, policy_version 689074 (0.0010) [2023-12-27 10:23:45,607][20347] Updated weights for policy 0, policy_version 689084 (0.0010) [2023-12-27 10:23:45,655][20347] Updated weights for policy 0, policy_version 689094 (0.0010) [2023-12-27 10:23:45,848][20381] Updated weights for policy 1, policy_version 688639 (0.0009) [2023-12-27 10:23:45,899][20381] Updated weights for policy 1, policy_version 688649 (0.0010) [2023-12-27 10:23:45,947][20381] Updated weights for policy 1, policy_version 688659 (0.0010) [2023-12-27 10:23:46,306][20347] Updated weights for policy 0, policy_version 689104 (0.0006) [2023-12-27 10:23:46,368][20347] Updated weights for policy 0, policy_version 689114 (0.0005) [2023-12-27 10:23:46,421][20347] Updated weights for policy 0, policy_version 689124 (0.0005) [2023-12-27 10:23:46,616][20381] Updated weights for policy 1, policy_version 688669 (0.0009) [2023-12-27 10:23:46,673][20381] Updated weights for policy 1, policy_version 688679 (0.0008) [2023-12-27 10:23:46,725][20381] Updated weights for policy 1, policy_version 688689 (0.0007) [2023-12-27 10:23:46,975][20347] Updated weights for policy 0, policy_version 689134 (0.0005) [2023-12-27 10:23:47,024][20347] Updated weights for policy 0, policy_version 689144 (0.0005) [2023-12-27 10:23:47,073][20347] Updated weights for policy 0, policy_version 689154 (0.0005) [2023-12-27 10:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.7, 300 sec: 20743.8). Total num frames: 352780288. Throughput: 0: 10453.1, 1: 10422.3. Samples: 352790988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:47,363][19170] Avg episode reward: [(0, '980.490'), (1, '922.450')] [2023-12-27 10:23:47,401][20381] Updated weights for policy 1, policy_version 688699 (0.0006) [2023-12-27 10:23:47,462][20381] Updated weights for policy 1, policy_version 688709 (0.0006) [2023-12-27 10:23:47,521][20381] Updated weights for policy 1, policy_version 688719 (0.0005) [2023-12-27 10:23:47,642][20347] Updated weights for policy 0, policy_version 689164 (0.0005) [2023-12-27 10:23:47,689][20347] Updated weights for policy 0, policy_version 689174 (0.0005) [2023-12-27 10:23:47,741][20347] Updated weights for policy 0, policy_version 689184 (0.0005) [2023-12-27 10:23:48,100][20381] Updated weights for policy 1, policy_version 688729 (0.0005) [2023-12-27 10:23:48,152][20381] Updated weights for policy 1, policy_version 688739 (0.0008) [2023-12-27 10:23:48,202][20381] Updated weights for policy 1, policy_version 688749 (0.0011) [2023-12-27 10:23:48,248][20381] Updated weights for policy 1, policy_version 688759 (0.0011) [2023-12-27 10:23:48,295][20347] Updated weights for policy 0, policy_version 689194 (0.0006) [2023-12-27 10:23:48,357][20347] Updated weights for policy 0, policy_version 689204 (0.0008) [2023-12-27 10:23:48,416][20347] Updated weights for policy 0, policy_version 689214 (0.0007) [2023-12-27 10:23:48,476][20347] Updated weights for policy 0, policy_version 689224 (0.0009) [2023-12-27 10:23:48,924][20381] Updated weights for policy 1, policy_version 688769 (0.0011) [2023-12-27 10:23:48,984][20381] Updated weights for policy 1, policy_version 688779 (0.0011) [2023-12-27 10:23:49,041][20381] Updated weights for policy 1, policy_version 688789 (0.0010) [2023-12-27 10:23:49,140][20347] Updated weights for policy 0, policy_version 689234 (0.0010) [2023-12-27 10:23:49,198][20347] Updated weights for policy 0, policy_version 689244 (0.0010) [2023-12-27 10:23:49,258][20347] Updated weights for policy 0, policy_version 689254 (0.0011) [2023-12-27 10:23:49,698][20381] Updated weights for policy 1, policy_version 688799 (0.0011) [2023-12-27 10:23:49,769][20381] Updated weights for policy 1, policy_version 688809 (0.0009) [2023-12-27 10:23:49,830][20381] Updated weights for policy 1, policy_version 688819 (0.0011) [2023-12-27 10:23:49,886][20347] Updated weights for policy 0, policy_version 689264 (0.0010) [2023-12-27 10:23:49,951][20347] Updated weights for policy 0, policy_version 689274 (0.0011) [2023-12-27 10:23:50,009][20347] Updated weights for policy 0, policy_version 689284 (0.0010) [2023-12-27 10:23:50,541][20381] Updated weights for policy 1, policy_version 688829 (0.0009) [2023-12-27 10:23:50,598][20381] Updated weights for policy 1, policy_version 688839 (0.0008) [2023-12-27 10:23:50,651][20381] Updated weights for policy 1, policy_version 688849 (0.0007) [2023-12-27 10:23:50,657][20347] Updated weights for policy 0, policy_version 689294 (0.0008) [2023-12-27 10:23:50,703][20347] Updated weights for policy 0, policy_version 689304 (0.0010) [2023-12-27 10:23:50,750][20347] Updated weights for policy 0, policy_version 689314 (0.0010) [2023-12-27 10:23:51,390][20347] Updated weights for policy 0, policy_version 689324 (0.0010) [2023-12-27 10:23:51,399][20381] Updated weights for policy 1, policy_version 688859 (0.0006) [2023-12-27 10:23:51,443][20347] Updated weights for policy 0, policy_version 689334 (0.0010) [2023-12-27 10:23:51,451][20381] Updated weights for policy 1, policy_version 688869 (0.0008) [2023-12-27 10:23:51,501][20347] Updated weights for policy 0, policy_version 689344 (0.0010) [2023-12-27 10:23:51,509][20381] Updated weights for policy 1, policy_version 688879 (0.0008) [2023-12-27 10:23:52,135][20347] Updated weights for policy 0, policy_version 689354 (0.0007) [2023-12-27 10:23:52,198][20347] Updated weights for policy 0, policy_version 689364 (0.0005) [2023-12-27 10:23:52,230][20381] Updated weights for policy 1, policy_version 688889 (0.0008) [2023-12-27 10:23:52,254][20347] Updated weights for policy 0, policy_version 689374 (0.0006) [2023-12-27 10:23:52,288][20381] Updated weights for policy 1, policy_version 688899 (0.0006) [2023-12-27 10:23:52,313][20347] Updated weights for policy 0, policy_version 689384 (0.0006) [2023-12-27 10:23:52,347][20381] Updated weights for policy 1, policy_version 688909 (0.0007) [2023-12-27 10:23:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 352886784. Throughput: 0: 10464.5, 1: 10388.3. Samples: 352855764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:52,363][19170] Avg episode reward: [(0, '990.310'), (1, '893.250')] [2023-12-27 10:23:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000689384_176504832.pth... [2023-12-27 10:23:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000688136_176185344.pth [2023-12-27 10:23:52,409][20381] Updated weights for policy 1, policy_version 688919 (0.0008) [2023-12-27 10:23:52,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000688920_176390144.pth... [2023-12-27 10:23:52,417][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000687672_176070656.pth [2023-12-27 10:23:52,983][20347] Updated weights for policy 0, policy_version 689394 (0.0007) [2023-12-27 10:23:53,032][20347] Updated weights for policy 0, policy_version 689404 (0.0006) [2023-12-27 10:23:53,082][20347] Updated weights for policy 0, policy_version 689414 (0.0005) [2023-12-27 10:23:53,105][20381] Updated weights for policy 1, policy_version 688929 (0.0007) [2023-12-27 10:23:53,160][20381] Updated weights for policy 1, policy_version 688939 (0.0008) [2023-12-27 10:23:53,221][20381] Updated weights for policy 1, policy_version 688949 (0.0008) [2023-12-27 10:23:53,756][20347] Updated weights for policy 0, policy_version 689424 (0.0007) [2023-12-27 10:23:53,816][20347] Updated weights for policy 0, policy_version 689434 (0.0006) [2023-12-27 10:23:53,867][20347] Updated weights for policy 0, policy_version 689444 (0.0005) [2023-12-27 10:23:53,907][20381] Updated weights for policy 1, policy_version 688959 (0.0008) [2023-12-27 10:23:53,956][20381] Updated weights for policy 1, policy_version 688969 (0.0008) [2023-12-27 10:23:54,013][20381] Updated weights for policy 1, policy_version 688979 (0.0005) [2023-12-27 10:23:54,518][20347] Updated weights for policy 0, policy_version 689454 (0.0005) [2023-12-27 10:23:54,565][20381] Updated weights for policy 1, policy_version 688989 (0.0008) [2023-12-27 10:23:54,575][20347] Updated weights for policy 0, policy_version 689464 (0.0009) [2023-12-27 10:23:54,621][20381] Updated weights for policy 1, policy_version 688999 (0.0010) [2023-12-27 10:23:54,630][20347] Updated weights for policy 0, policy_version 689474 (0.0010) [2023-12-27 10:23:54,675][20381] Updated weights for policy 1, policy_version 689009 (0.0010) [2023-12-27 10:23:55,212][20347] Updated weights for policy 0, policy_version 689484 (0.0008) [2023-12-27 10:23:55,258][20347] Updated weights for policy 0, policy_version 689494 (0.0005) [2023-12-27 10:23:55,308][20347] Updated weights for policy 0, policy_version 689504 (0.0007) [2023-12-27 10:23:55,369][20381] Updated weights for policy 1, policy_version 689019 (0.0010) [2023-12-27 10:23:55,425][20381] Updated weights for policy 1, policy_version 689029 (0.0010) [2023-12-27 10:23:55,482][20381] Updated weights for policy 1, policy_version 689039 (0.0010) [2023-12-27 10:23:55,904][20347] Updated weights for policy 0, policy_version 689514 (0.0007) [2023-12-27 10:23:55,959][20347] Updated weights for policy 0, policy_version 689524 (0.0006) [2023-12-27 10:23:56,013][20347] Updated weights for policy 0, policy_version 689534 (0.0005) [2023-12-27 10:23:56,070][20347] Updated weights for policy 0, policy_version 689544 (0.0005) [2023-12-27 10:23:56,152][20381] Updated weights for policy 1, policy_version 689049 (0.0010) [2023-12-27 10:23:56,212][20381] Updated weights for policy 1, policy_version 689059 (0.0007) [2023-12-27 10:23:56,266][20381] Updated weights for policy 1, policy_version 689069 (0.0010) [2023-12-27 10:23:56,317][20381] Updated weights for policy 1, policy_version 689079 (0.0010) [2023-12-27 10:23:56,602][20347] Updated weights for policy 0, policy_version 689554 (0.0006) [2023-12-27 10:23:56,654][20347] Updated weights for policy 0, policy_version 689564 (0.0005) [2023-12-27 10:23:56,708][20347] Updated weights for policy 0, policy_version 689574 (0.0005) [2023-12-27 10:23:57,009][20381] Updated weights for policy 1, policy_version 689089 (0.0009) [2023-12-27 10:23:57,063][20381] Updated weights for policy 1, policy_version 689099 (0.0007) [2023-12-27 10:23:57,120][20381] Updated weights for policy 1, policy_version 689109 (0.0008) [2023-12-27 10:23:57,253][20347] Updated weights for policy 0, policy_version 689584 (0.0006) [2023-12-27 10:23:57,300][20347] Updated weights for policy 0, policy_version 689594 (0.0007) [2023-12-27 10:23:57,349][20347] Updated weights for policy 0, policy_version 689604 (0.0008) [2023-12-27 10:23:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 352993280. Throughput: 0: 10572.6, 1: 10350.0. Samples: 352981856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:23:57,363][19170] Avg episode reward: [(0, '980.580'), (1, '922.340')] [2023-12-27 10:23:57,828][20381] Updated weights for policy 1, policy_version 689119 (0.0007) [2023-12-27 10:23:57,879][20381] Updated weights for policy 1, policy_version 689129 (0.0008) [2023-12-27 10:23:57,929][20381] Updated weights for policy 1, policy_version 689139 (0.0007) [2023-12-27 10:23:57,949][20347] Updated weights for policy 0, policy_version 689614 (0.0009) [2023-12-27 10:23:58,009][20347] Updated weights for policy 0, policy_version 689624 (0.0010) [2023-12-27 10:23:58,056][20347] Updated weights for policy 0, policy_version 689634 (0.0007) [2023-12-27 10:23:58,547][20381] Updated weights for policy 1, policy_version 689149 (0.0007) [2023-12-27 10:23:58,612][20381] Updated weights for policy 1, policy_version 689159 (0.0007) [2023-12-27 10:23:58,673][20381] Updated weights for policy 1, policy_version 689169 (0.0008) [2023-12-27 10:23:58,721][20347] Updated weights for policy 0, policy_version 689644 (0.0006) [2023-12-27 10:23:58,782][20347] Updated weights for policy 0, policy_version 689654 (0.0007) [2023-12-27 10:23:58,848][20347] Updated weights for policy 0, policy_version 689664 (0.0009) [2023-12-27 10:23:59,447][20381] Updated weights for policy 1, policy_version 689179 (0.0007) [2023-12-27 10:23:59,503][20381] Updated weights for policy 1, policy_version 689189 (0.0007) [2023-12-27 10:23:59,552][20381] Updated weights for policy 1, policy_version 689199 (0.0007) [2023-12-27 10:23:59,619][20347] Updated weights for policy 0, policy_version 689674 (0.0009) [2023-12-27 10:23:59,678][20347] Updated weights for policy 0, policy_version 689684 (0.0005) [2023-12-27 10:23:59,733][20347] Updated weights for policy 0, policy_version 689694 (0.0005) [2023-12-27 10:23:59,781][20347] Updated weights for policy 0, policy_version 689704 (0.0005) [2023-12-27 10:24:00,215][20381] Updated weights for policy 1, policy_version 689209 (0.0010) [2023-12-27 10:24:00,277][20381] Updated weights for policy 1, policy_version 689219 (0.0009) [2023-12-27 10:24:00,342][20381] Updated weights for policy 1, policy_version 689229 (0.0006) [2023-12-27 10:24:00,404][20381] Updated weights for policy 1, policy_version 689239 (0.0006) [2023-12-27 10:24:00,409][20347] Updated weights for policy 0, policy_version 689714 (0.0006) [2023-12-27 10:24:00,458][20347] Updated weights for policy 0, policy_version 689724 (0.0005) [2023-12-27 10:24:00,507][20347] Updated weights for policy 0, policy_version 689734 (0.0008) [2023-12-27 10:24:00,943][20381] Updated weights for policy 1, policy_version 689249 (0.0008) [2023-12-27 10:24:00,989][20381] Updated weights for policy 1, policy_version 689259 (0.0008) [2023-12-27 10:24:01,044][20381] Updated weights for policy 1, policy_version 689269 (0.0008) [2023-12-27 10:24:01,126][20347] Updated weights for policy 0, policy_version 689744 (0.0008) [2023-12-27 10:24:01,173][20347] Updated weights for policy 0, policy_version 689754 (0.0008) [2023-12-27 10:24:01,216][20347] Updated weights for policy 0, policy_version 689764 (0.0008) [2023-12-27 10:24:01,746][20381] Updated weights for policy 1, policy_version 689279 (0.0006) [2023-12-27 10:24:01,805][20381] Updated weights for policy 1, policy_version 689289 (0.0006) [2023-12-27 10:24:01,861][20381] Updated weights for policy 1, policy_version 689299 (0.0006) [2023-12-27 10:24:01,967][20291] KL-divergence is very high: 213.0306 [2023-12-27 10:24:01,974][20347] Updated weights for policy 0, policy_version 689774 (0.0008) [2023-12-27 10:24:02,035][20347] Updated weights for policy 0, policy_version 689784 (0.0008) [2023-12-27 10:24:02,102][20347] Updated weights for policy 0, policy_version 689794 (0.0008) [2023-12-27 10:24:02,122][20291] KL-divergence is very high: 571.4761 [2023-12-27 10:24:02,128][20291] KL-divergence is very high: 439.6683 [2023-12-27 10:24:02,134][20291] KL-divergence is very high: 371.1937 [2023-12-27 10:24:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20771.6). Total num frames: 353099776. Throughput: 0: 10562.9, 1: 10374.5. Samples: 353108492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:02,363][19170] Avg episode reward: [(0, '964.660'), (1, '941.860')] [2023-12-27 10:24:02,512][20381] Updated weights for policy 1, policy_version 689309 (0.0008) [2023-12-27 10:24:02,567][20381] Updated weights for policy 1, policy_version 689319 (0.0010) [2023-12-27 10:24:02,619][20381] Updated weights for policy 1, policy_version 689329 (0.0010) [2023-12-27 10:24:02,728][20347] Updated weights for policy 0, policy_version 689804 (0.0006) [2023-12-27 10:24:02,785][20347] Updated weights for policy 0, policy_version 689814 (0.0008) [2023-12-27 10:24:02,841][20347] Updated weights for policy 0, policy_version 689824 (0.0009) [2023-12-27 10:24:03,268][20381] Updated weights for policy 1, policy_version 689339 (0.0010) [2023-12-27 10:24:03,317][20381] Updated weights for policy 1, policy_version 689349 (0.0010) [2023-12-27 10:24:03,366][20381] Updated weights for policy 1, policy_version 689359 (0.0010) [2023-12-27 10:24:03,477][20347] Updated weights for policy 0, policy_version 689834 (0.0007) [2023-12-27 10:24:03,525][20347] Updated weights for policy 0, policy_version 689844 (0.0005) [2023-12-27 10:24:03,581][20347] Updated weights for policy 0, policy_version 689854 (0.0005) [2023-12-27 10:24:03,637][20347] Updated weights for policy 0, policy_version 689864 (0.0005) [2023-12-27 10:24:04,050][20381] Updated weights for policy 1, policy_version 689369 (0.0010) [2023-12-27 10:24:04,113][20381] Updated weights for policy 1, policy_version 689379 (0.0006) [2023-12-27 10:24:04,174][20381] Updated weights for policy 1, policy_version 689389 (0.0009) [2023-12-27 10:24:04,225][20381] Updated weights for policy 1, policy_version 689399 (0.0008) [2023-12-27 10:24:04,237][20347] Updated weights for policy 0, policy_version 689874 (0.0007) [2023-12-27 10:24:04,287][20347] Updated weights for policy 0, policy_version 689884 (0.0007) [2023-12-27 10:24:04,350][20347] Updated weights for policy 0, policy_version 689894 (0.0009) [2023-12-27 10:24:04,841][20381] Updated weights for policy 1, policy_version 689409 (0.0010) [2023-12-27 10:24:04,894][20381] Updated weights for policy 1, policy_version 689419 (0.0011) [2023-12-27 10:24:04,939][20381] Updated weights for policy 1, policy_version 689429 (0.0010) [2023-12-27 10:24:04,994][20347] Updated weights for policy 0, policy_version 689904 (0.0007) [2023-12-27 10:24:05,052][20347] Updated weights for policy 0, policy_version 689914 (0.0008) [2023-12-27 10:24:05,099][20347] Updated weights for policy 0, policy_version 689924 (0.0007) [2023-12-27 10:24:05,588][20381] Updated weights for policy 1, policy_version 689439 (0.0009) [2023-12-27 10:24:05,637][20381] Updated weights for policy 1, policy_version 689449 (0.0011) [2023-12-27 10:24:05,692][20381] Updated weights for policy 1, policy_version 689459 (0.0010) [2023-12-27 10:24:05,750][20347] Updated weights for policy 0, policy_version 689934 (0.0007) [2023-12-27 10:24:05,807][20347] Updated weights for policy 0, policy_version 689944 (0.0007) [2023-12-27 10:24:05,858][20347] Updated weights for policy 0, policy_version 689954 (0.0007) [2023-12-27 10:24:06,408][20381] Updated weights for policy 1, policy_version 689469 (0.0010) [2023-12-27 10:24:06,464][20381] Updated weights for policy 1, policy_version 689479 (0.0011) [2023-12-27 10:24:06,520][20381] Updated weights for policy 1, policy_version 689489 (0.0011) [2023-12-27 10:24:06,526][20347] Updated weights for policy 0, policy_version 689964 (0.0006) [2023-12-27 10:24:06,575][20347] Updated weights for policy 0, policy_version 689974 (0.0005) [2023-12-27 10:24:06,643][20347] Updated weights for policy 0, policy_version 689984 (0.0006) [2023-12-27 10:24:07,192][20381] Updated weights for policy 1, policy_version 689499 (0.0010) [2023-12-27 10:24:07,245][20381] Updated weights for policy 1, policy_version 689509 (0.0008) [2023-12-27 10:24:07,261][20347] Updated weights for policy 0, policy_version 689994 (0.0006) [2023-12-27 10:24:07,308][20381] Updated weights for policy 1, policy_version 689519 (0.0006) [2023-12-27 10:24:07,313][20347] Updated weights for policy 0, policy_version 690004 (0.0005) [2023-12-27 10:24:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 353206272. Throughput: 0: 10564.1, 1: 10422.3. Samples: 353172260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:07,363][19170] Avg episode reward: [(0, '948.310'), (1, '922.520')] [2023-12-27 10:24:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000689528_176545792.pth... [2023-12-27 10:24:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000688312_176234496.pth [2023-12-27 10:24:07,382][20347] Updated weights for policy 0, policy_version 690014 (0.0008) [2023-12-27 10:24:07,438][20347] Updated weights for policy 0, policy_version 690024 (0.0008) [2023-12-27 10:24:07,438][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000690024_176668672.pth... [2023-12-27 10:24:07,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000688776_176349184.pth [2023-12-27 10:24:07,941][20381] Updated weights for policy 1, policy_version 689529 (0.0006) [2023-12-27 10:24:07,991][20381] Updated weights for policy 1, policy_version 689539 (0.0008) [2023-12-27 10:24:08,039][20381] Updated weights for policy 1, policy_version 689549 (0.0005) [2023-12-27 10:24:08,093][20381] Updated weights for policy 1, policy_version 689559 (0.0008) [2023-12-27 10:24:08,123][20347] Updated weights for policy 0, policy_version 690034 (0.0006) [2023-12-27 10:24:08,177][20347] Updated weights for policy 0, policy_version 690044 (0.0007) [2023-12-27 10:24:08,221][20347] Updated weights for policy 0, policy_version 690054 (0.0007) [2023-12-27 10:24:08,796][20381] Updated weights for policy 1, policy_version 689569 (0.0010) [2023-12-27 10:24:08,857][20381] Updated weights for policy 1, policy_version 689579 (0.0007) [2023-12-27 10:24:08,909][20381] Updated weights for policy 1, policy_version 689589 (0.0010) [2023-12-27 10:24:08,945][20347] Updated weights for policy 0, policy_version 690064 (0.0007) [2023-12-27 10:24:08,996][20347] Updated weights for policy 0, policy_version 690074 (0.0008) [2023-12-27 10:24:09,048][20347] Updated weights for policy 0, policy_version 690084 (0.0008) [2023-12-27 10:24:09,611][20381] Updated weights for policy 1, policy_version 689599 (0.0010) [2023-12-27 10:24:09,672][20381] Updated weights for policy 1, policy_version 689609 (0.0011) [2023-12-27 10:24:09,736][20381] Updated weights for policy 1, policy_version 689619 (0.0008) [2023-12-27 10:24:09,833][20347] Updated weights for policy 0, policy_version 690094 (0.0008) [2023-12-27 10:24:09,899][20347] Updated weights for policy 0, policy_version 690104 (0.0008) [2023-12-27 10:24:09,967][20347] Updated weights for policy 0, policy_version 690114 (0.0008) [2023-12-27 10:24:10,470][20381] Updated weights for policy 1, policy_version 689629 (0.0007) [2023-12-27 10:24:10,532][20381] Updated weights for policy 1, policy_version 689639 (0.0008) [2023-12-27 10:24:10,589][20381] Updated weights for policy 1, policy_version 689649 (0.0009) [2023-12-27 10:24:10,675][20347] Updated weights for policy 0, policy_version 690124 (0.0007) [2023-12-27 10:24:10,727][20347] Updated weights for policy 0, policy_version 690134 (0.0008) [2023-12-27 10:24:10,788][20347] Updated weights for policy 0, policy_version 690144 (0.0008) [2023-12-27 10:24:11,342][20381] Updated weights for policy 1, policy_version 689659 (0.0008) [2023-12-27 10:24:11,406][20381] Updated weights for policy 1, policy_version 689669 (0.0008) [2023-12-27 10:24:11,477][20381] Updated weights for policy 1, policy_version 689679 (0.0006) [2023-12-27 10:24:11,549][20347] Updated weights for policy 0, policy_version 690154 (0.0008) [2023-12-27 10:24:11,617][20347] Updated weights for policy 0, policy_version 690164 (0.0009) [2023-12-27 10:24:11,689][20347] Updated weights for policy 0, policy_version 690174 (0.0008) [2023-12-27 10:24:11,761][20347] Updated weights for policy 0, policy_version 690184 (0.0011) [2023-12-27 10:24:12,247][20381] Updated weights for policy 1, policy_version 689689 (0.0007) [2023-12-27 10:24:12,323][20381] Updated weights for policy 1, policy_version 689699 (0.0008) [2023-12-27 10:24:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20716.0). Total num frames: 353296384. Throughput: 0: 10511.4, 1: 10351.5. Samples: 353292704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:12,363][19170] Avg episode reward: [(0, '963.330'), (1, '941.660')] [2023-12-27 10:24:12,396][20381] Updated weights for policy 1, policy_version 689709 (0.0009) [2023-12-27 10:24:12,461][20381] Updated weights for policy 1, policy_version 689719 (0.0009) [2023-12-27 10:24:12,664][20347] Updated weights for policy 0, policy_version 690194 (0.0009) [2023-12-27 10:24:12,721][20347] Updated weights for policy 0, policy_version 690204 (0.0009) [2023-12-27 10:24:12,782][20347] Updated weights for policy 0, policy_version 690214 (0.0010) [2023-12-27 10:24:13,200][20381] Updated weights for policy 1, policy_version 689729 (0.0007) [2023-12-27 10:24:13,265][20381] Updated weights for policy 1, policy_version 689739 (0.0006) [2023-12-27 10:24:13,327][20381] Updated weights for policy 1, policy_version 689749 (0.0007) [2023-12-27 10:24:13,681][20347] Updated weights for policy 0, policy_version 690224 (0.0008) [2023-12-27 10:24:13,749][20347] Updated weights for policy 0, policy_version 690234 (0.0011) [2023-12-27 10:24:13,812][20347] Updated weights for policy 0, policy_version 690244 (0.0010) [2023-12-27 10:24:14,102][20381] Updated weights for policy 1, policy_version 689759 (0.0008) [2023-12-27 10:24:14,173][20381] Updated weights for policy 1, policy_version 689769 (0.0008) [2023-12-27 10:24:14,240][20381] Updated weights for policy 1, policy_version 689779 (0.0009) [2023-12-27 10:24:14,627][20347] Updated weights for policy 0, policy_version 690254 (0.0010) [2023-12-27 10:24:14,691][20347] Updated weights for policy 0, policy_version 690264 (0.0011) [2023-12-27 10:24:14,758][20347] Updated weights for policy 0, policy_version 690274 (0.0007) [2023-12-27 10:24:14,977][20381] Updated weights for policy 1, policy_version 689789 (0.0009) [2023-12-27 10:24:15,044][20381] Updated weights for policy 1, policy_version 689799 (0.0011) [2023-12-27 10:24:15,106][20381] Updated weights for policy 1, policy_version 689809 (0.0011) [2023-12-27 10:24:15,467][20347] Updated weights for policy 0, policy_version 690284 (0.0009) [2023-12-27 10:24:15,526][20347] Updated weights for policy 0, policy_version 690294 (0.0011) [2023-12-27 10:24:15,590][20347] Updated weights for policy 0, policy_version 690304 (0.0008) [2023-12-27 10:24:15,832][20381] Updated weights for policy 1, policy_version 689819 (0.0011) [2023-12-27 10:24:15,888][20381] Updated weights for policy 1, policy_version 689829 (0.0008) [2023-12-27 10:24:15,936][20381] Updated weights for policy 1, policy_version 689839 (0.0008) [2023-12-27 10:24:16,298][20347] Updated weights for policy 0, policy_version 690314 (0.0010) [2023-12-27 10:24:16,356][20347] Updated weights for policy 0, policy_version 690324 (0.0008) [2023-12-27 10:24:16,409][20347] Updated weights for policy 0, policy_version 690334 (0.0008) [2023-12-27 10:24:16,475][20347] Updated weights for policy 0, policy_version 690344 (0.0006) [2023-12-27 10:24:16,617][20381] Updated weights for policy 1, policy_version 689849 (0.0010) [2023-12-27 10:24:16,681][20381] Updated weights for policy 1, policy_version 689859 (0.0011) [2023-12-27 10:24:16,743][20381] Updated weights for policy 1, policy_version 689869 (0.0010) [2023-12-27 10:24:16,801][20381] Updated weights for policy 1, policy_version 689879 (0.0010) [2023-12-27 10:24:17,146][20347] Updated weights for policy 0, policy_version 690354 (0.0008) [2023-12-27 10:24:17,198][20347] Updated weights for policy 0, policy_version 690364 (0.0007) [2023-12-27 10:24:17,256][20347] Updated weights for policy 0, policy_version 690374 (0.0008) [2023-12-27 10:24:17,362][19170] Fps is (10 sec: 18841.7, 60 sec: 20616.5, 300 sec: 20743.8). Total num frames: 353394688. Throughput: 0: 10321.2, 1: 10182.1. Samples: 353402876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:17,363][19170] Avg episode reward: [(0, '953.660'), (1, '961.070')] [2023-12-27 10:24:17,457][20381] Updated weights for policy 1, policy_version 689889 (0.0007) [2023-12-27 10:24:17,518][20381] Updated weights for policy 1, policy_version 689899 (0.0007) [2023-12-27 10:24:17,577][20381] Updated weights for policy 1, policy_version 689909 (0.0011) [2023-12-27 10:24:17,959][20347] Updated weights for policy 0, policy_version 690384 (0.0007) [2023-12-27 10:24:18,013][20347] Updated weights for policy 0, policy_version 690394 (0.0008) [2023-12-27 10:24:18,077][20347] Updated weights for policy 0, policy_version 690404 (0.0007) [2023-12-27 10:24:18,247][20381] Updated weights for policy 1, policy_version 689919 (0.0011) [2023-12-27 10:24:18,303][20381] Updated weights for policy 1, policy_version 689929 (0.0011) [2023-12-27 10:24:18,360][20381] Updated weights for policy 1, policy_version 689939 (0.0011) [2023-12-27 10:24:18,766][20347] Updated weights for policy 0, policy_version 690414 (0.0007) [2023-12-27 10:24:18,829][20347] Updated weights for policy 0, policy_version 690424 (0.0007) [2023-12-27 10:24:18,889][20347] Updated weights for policy 0, policy_version 690434 (0.0007) [2023-12-27 10:24:19,082][20381] Updated weights for policy 1, policy_version 689949 (0.0009) [2023-12-27 10:24:19,142][20381] Updated weights for policy 1, policy_version 689959 (0.0011) [2023-12-27 10:24:19,203][20381] Updated weights for policy 1, policy_version 689969 (0.0010) [2023-12-27 10:24:19,654][20347] Updated weights for policy 0, policy_version 690444 (0.0009) [2023-12-27 10:24:19,721][20347] Updated weights for policy 0, policy_version 690454 (0.0007) [2023-12-27 10:24:19,787][20347] Updated weights for policy 0, policy_version 690464 (0.0008) [2023-12-27 10:24:20,038][20381] Updated weights for policy 1, policy_version 689979 (0.0009) [2023-12-27 10:24:20,102][20381] Updated weights for policy 1, policy_version 689989 (0.0007) [2023-12-27 10:24:20,167][20381] Updated weights for policy 1, policy_version 689999 (0.0007) [2023-12-27 10:24:20,673][20347] Updated weights for policy 0, policy_version 690474 (0.0009) [2023-12-27 10:24:20,739][20347] Updated weights for policy 0, policy_version 690484 (0.0009) [2023-12-27 10:24:20,804][20347] Updated weights for policy 0, policy_version 690494 (0.0009) [2023-12-27 10:24:20,863][20347] Updated weights for policy 0, policy_version 690504 (0.0009) [2023-12-27 10:24:20,895][20381] Updated weights for policy 1, policy_version 690009 (0.0007) [2023-12-27 10:24:20,959][20381] Updated weights for policy 1, policy_version 690019 (0.0006) [2023-12-27 10:24:21,021][20381] Updated weights for policy 1, policy_version 690029 (0.0008) [2023-12-27 10:24:21,089][20381] Updated weights for policy 1, policy_version 690039 (0.0008) [2023-12-27 10:24:21,811][20347] Updated weights for policy 0, policy_version 690514 (0.0009) [2023-12-27 10:24:21,818][20381] Updated weights for policy 1, policy_version 690049 (0.0007) [2023-12-27 10:24:21,862][20347] Updated weights for policy 0, policy_version 690524 (0.0009) [2023-12-27 10:24:21,885][20381] Updated weights for policy 1, policy_version 690059 (0.0006) [2023-12-27 10:24:21,914][20347] Updated weights for policy 0, policy_version 690534 (0.0009) [2023-12-27 10:24:21,945][20381] Updated weights for policy 1, policy_version 690069 (0.0005) [2023-12-27 10:24:22,363][19170] Fps is (10 sec: 18841.4, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 353484800. Throughput: 0: 10257.9, 1: 10077.5. Samples: 353460136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:22,363][19170] Avg episode reward: [(0, '963.380'), (1, '990.280')] [2023-12-27 10:24:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000690072_176685056.pth... [2023-12-27 10:24:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000690536_176799744.pth... [2023-12-27 10:24:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000689384_176504832.pth [2023-12-27 10:24:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000688920_176390144.pth [2023-12-27 10:24:22,636][20381] Updated weights for policy 1, policy_version 690079 (0.0007) [2023-12-27 10:24:22,657][20347] Updated weights for policy 0, policy_version 690544 (0.0007) [2023-12-27 10:24:22,694][20381] Updated weights for policy 1, policy_version 690089 (0.0006) [2023-12-27 10:24:22,720][20347] Updated weights for policy 0, policy_version 690554 (0.0008) [2023-12-27 10:24:22,750][20381] Updated weights for policy 1, policy_version 690099 (0.0007) [2023-12-27 10:24:22,783][20347] Updated weights for policy 0, policy_version 690564 (0.0006) [2023-12-27 10:24:23,496][20381] Updated weights for policy 1, policy_version 690109 (0.0008) [2023-12-27 10:24:23,569][20347] Updated weights for policy 0, policy_version 690574 (0.0008) [2023-12-27 10:24:23,570][20381] Updated weights for policy 1, policy_version 690119 (0.0009) [2023-12-27 10:24:23,640][20381] Updated weights for policy 1, policy_version 690129 (0.0007) [2023-12-27 10:24:23,642][20347] Updated weights for policy 0, policy_version 690584 (0.0008) [2023-12-27 10:24:23,711][20347] Updated weights for policy 0, policy_version 690594 (0.0007) [2023-12-27 10:24:24,367][20381] Updated weights for policy 1, policy_version 690139 (0.0008) [2023-12-27 10:24:24,431][20381] Updated weights for policy 1, policy_version 690149 (0.0007) [2023-12-27 10:24:24,454][20347] Updated weights for policy 0, policy_version 690604 (0.0008) [2023-12-27 10:24:24,488][20381] Updated weights for policy 1, policy_version 690159 (0.0007) [2023-12-27 10:24:24,516][20347] Updated weights for policy 0, policy_version 690614 (0.0009) [2023-12-27 10:24:24,576][20347] Updated weights for policy 0, policy_version 690624 (0.0008) [2023-12-27 10:24:25,212][20381] Updated weights for policy 1, policy_version 690169 (0.0009) [2023-12-27 10:24:25,284][20381] Updated weights for policy 1, policy_version 690179 (0.0008) [2023-12-27 10:24:25,330][20347] Updated weights for policy 0, policy_version 690634 (0.0008) [2023-12-27 10:24:25,349][20381] Updated weights for policy 1, policy_version 690189 (0.0007) [2023-12-27 10:24:25,396][20347] Updated weights for policy 0, policy_version 690644 (0.0007) [2023-12-27 10:24:25,416][20381] Updated weights for policy 1, policy_version 690199 (0.0007) [2023-12-27 10:24:25,466][20347] Updated weights for policy 0, policy_version 690654 (0.0009) [2023-12-27 10:24:25,536][20347] Updated weights for policy 0, policy_version 690664 (0.0008) [2023-12-27 10:24:26,223][20381] Updated weights for policy 1, policy_version 690209 (0.0009) [2023-12-27 10:24:26,225][20347] Updated weights for policy 0, policy_version 690674 (0.0007) [2023-12-27 10:24:26,276][20347] Updated weights for policy 0, policy_version 690684 (0.0006) [2023-12-27 10:24:26,281][20381] Updated weights for policy 1, policy_version 690219 (0.0008) [2023-12-27 10:24:26,336][20347] Updated weights for policy 0, policy_version 690694 (0.0007) [2023-12-27 10:24:26,339][20381] Updated weights for policy 1, policy_version 690229 (0.0008) [2023-12-27 10:24:27,048][20347] Updated weights for policy 0, policy_version 690704 (0.0007) [2023-12-27 10:24:27,054][20381] Updated weights for policy 1, policy_version 690239 (0.0008) [2023-12-27 10:24:27,108][20347] Updated weights for policy 0, policy_version 690714 (0.0008) [2023-12-27 10:24:27,110][20381] Updated weights for policy 1, policy_version 690249 (0.0008) [2023-12-27 10:24:27,166][20347] Updated weights for policy 0, policy_version 690724 (0.0007) [2023-12-27 10:24:27,173][20381] Updated weights for policy 1, policy_version 690259 (0.0007) [2023-12-27 10:24:27,362][19170] Fps is (10 sec: 18841.6, 60 sec: 20207.0, 300 sec: 20660.5). Total num frames: 353583104. Throughput: 0: 10037.6, 1: 9942.3. Samples: 353568048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:27,363][19170] Avg episode reward: [(0, '941.840'), (1, '980.620')] [2023-12-27 10:24:27,891][20347] Updated weights for policy 0, policy_version 690734 (0.0008) [2023-12-27 10:24:27,934][20381] Updated weights for policy 1, policy_version 690269 (0.0006) [2023-12-27 10:24:27,951][20347] Updated weights for policy 0, policy_version 690744 (0.0007) [2023-12-27 10:24:28,000][20381] Updated weights for policy 1, policy_version 690279 (0.0008) [2023-12-27 10:24:28,012][20347] Updated weights for policy 0, policy_version 690754 (0.0006) [2023-12-27 10:24:28,062][20381] Updated weights for policy 1, policy_version 690289 (0.0007) [2023-12-27 10:24:28,766][20347] Updated weights for policy 0, policy_version 690764 (0.0006) [2023-12-27 10:24:28,831][20347] Updated weights for policy 0, policy_version 690774 (0.0006) [2023-12-27 10:24:28,892][20381] Updated weights for policy 1, policy_version 690299 (0.0008) [2023-12-27 10:24:28,898][20347] Updated weights for policy 0, policy_version 690784 (0.0010) [2023-12-27 10:24:28,953][20381] Updated weights for policy 1, policy_version 690309 (0.0009) [2023-12-27 10:24:29,022][20381] Updated weights for policy 1, policy_version 690319 (0.0007) [2023-12-27 10:24:29,582][20347] Updated weights for policy 0, policy_version 690794 (0.0008) [2023-12-27 10:24:29,635][20347] Updated weights for policy 0, policy_version 690804 (0.0007) [2023-12-27 10:24:29,700][20347] Updated weights for policy 0, policy_version 690814 (0.0008) [2023-12-27 10:24:29,744][20381] Updated weights for policy 1, policy_version 690329 (0.0007) [2023-12-27 10:24:29,759][20347] Updated weights for policy 0, policy_version 690824 (0.0008) [2023-12-27 10:24:29,811][20381] Updated weights for policy 1, policy_version 690339 (0.0009) [2023-12-27 10:24:29,878][20381] Updated weights for policy 1, policy_version 690349 (0.0008) [2023-12-27 10:24:29,940][20381] Updated weights for policy 1, policy_version 690359 (0.0008) [2023-12-27 10:24:30,605][20347] Updated weights for policy 0, policy_version 690834 (0.0008) [2023-12-27 10:24:30,666][20347] Updated weights for policy 0, policy_version 690844 (0.0008) [2023-12-27 10:24:30,733][20347] Updated weights for policy 0, policy_version 690854 (0.0008) [2023-12-27 10:24:30,737][20381] Updated weights for policy 1, policy_version 690369 (0.0008) [2023-12-27 10:24:30,797][20381] Updated weights for policy 1, policy_version 690379 (0.0008) [2023-12-27 10:24:30,864][20381] Updated weights for policy 1, policy_version 690389 (0.0008) [2023-12-27 10:24:31,396][20347] Updated weights for policy 0, policy_version 690864 (0.0008) [2023-12-27 10:24:31,447][20347] Updated weights for policy 0, policy_version 690874 (0.0008) [2023-12-27 10:24:31,498][20347] Updated weights for policy 0, policy_version 690884 (0.0008) [2023-12-27 10:24:31,603][20381] Updated weights for policy 1, policy_version 690399 (0.0009) [2023-12-27 10:24:31,671][20381] Updated weights for policy 1, policy_version 690409 (0.0009) [2023-12-27 10:24:31,737][20381] Updated weights for policy 1, policy_version 690419 (0.0009) [2023-12-27 10:24:32,191][20347] Updated weights for policy 0, policy_version 690894 (0.0006) [2023-12-27 10:24:32,246][20347] Updated weights for policy 0, policy_version 690904 (0.0007) [2023-12-27 10:24:32,309][20347] Updated weights for policy 0, policy_version 690914 (0.0009) [2023-12-27 10:24:32,362][19170] Fps is (10 sec: 18841.9, 60 sec: 20070.4, 300 sec: 20605.0). Total num frames: 353673216. Throughput: 0: 9954.7, 1: 9817.5. Samples: 353680740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:32,363][19170] Avg episode reward: [(0, '922.420'), (1, '970.900')] [2023-12-27 10:24:32,483][20381] Updated weights for policy 1, policy_version 690429 (0.0008) [2023-12-27 10:24:32,545][20381] Updated weights for policy 1, policy_version 690439 (0.0008) [2023-12-27 10:24:32,608][20381] Updated weights for policy 1, policy_version 690449 (0.0007) [2023-12-27 10:24:32,936][20347] Updated weights for policy 0, policy_version 690924 (0.0007) [2023-12-27 10:24:32,995][20347] Updated weights for policy 0, policy_version 690934 (0.0006) [2023-12-27 10:24:33,050][20347] Updated weights for policy 0, policy_version 690944 (0.0006) [2023-12-27 10:24:33,404][20381] Updated weights for policy 1, policy_version 690459 (0.0008) [2023-12-27 10:24:33,465][20381] Updated weights for policy 1, policy_version 690469 (0.0008) [2023-12-27 10:24:33,521][20381] Updated weights for policy 1, policy_version 690479 (0.0010) [2023-12-27 10:24:33,694][20347] Updated weights for policy 0, policy_version 690954 (0.0006) [2023-12-27 10:24:33,761][20347] Updated weights for policy 0, policy_version 690964 (0.0006) [2023-12-27 10:24:33,817][20347] Updated weights for policy 0, policy_version 690974 (0.0007) [2023-12-27 10:24:33,863][20347] Updated weights for policy 0, policy_version 690984 (0.0007) [2023-12-27 10:24:34,302][20381] Updated weights for policy 1, policy_version 690489 (0.0010) [2023-12-27 10:24:34,370][20381] Updated weights for policy 1, policy_version 690499 (0.0007) [2023-12-27 10:24:34,434][20381] Updated weights for policy 1, policy_version 690509 (0.0008) [2023-12-27 10:24:34,500][20381] Updated weights for policy 1, policy_version 690519 (0.0008) [2023-12-27 10:24:34,608][20347] Updated weights for policy 0, policy_version 690994 (0.0008) [2023-12-27 10:24:34,675][20347] Updated weights for policy 0, policy_version 691004 (0.0008) [2023-12-27 10:24:34,735][20347] Updated weights for policy 0, policy_version 691014 (0.0008) [2023-12-27 10:24:35,181][20381] Updated weights for policy 1, policy_version 690529 (0.0007) [2023-12-27 10:24:35,242][20381] Updated weights for policy 1, policy_version 690539 (0.0006) [2023-12-27 10:24:35,302][20381] Updated weights for policy 1, policy_version 690549 (0.0006) [2023-12-27 10:24:35,475][20347] Updated weights for policy 0, policy_version 691024 (0.0008) [2023-12-27 10:24:35,530][20347] Updated weights for policy 0, policy_version 691034 (0.0009) [2023-12-27 10:24:35,579][20291] KL-divergence is very high: 122.2595 [2023-12-27 10:24:35,591][20347] Updated weights for policy 0, policy_version 691044 (0.0011) [2023-12-27 10:24:35,946][20381] Updated weights for policy 1, policy_version 690559 (0.0008) [2023-12-27 10:24:36,007][20381] Updated weights for policy 1, policy_version 690569 (0.0006) [2023-12-27 10:24:36,068][20381] Updated weights for policy 1, policy_version 690579 (0.0008) [2023-12-27 10:24:36,262][20347] Updated weights for policy 0, policy_version 691054 (0.0011) [2023-12-27 10:24:36,323][20347] Updated weights for policy 0, policy_version 691064 (0.0011) [2023-12-27 10:24:36,388][20347] Updated weights for policy 0, policy_version 691074 (0.0010) [2023-12-27 10:24:36,836][20381] Updated weights for policy 1, policy_version 690589 (0.0009) [2023-12-27 10:24:36,906][20381] Updated weights for policy 1, policy_version 690599 (0.0008) [2023-12-27 10:24:36,962][20381] Updated weights for policy 1, policy_version 690609 (0.0008) [2023-12-27 10:24:37,109][20347] Updated weights for policy 0, policy_version 691084 (0.0006) [2023-12-27 10:24:37,174][20347] Updated weights for policy 0, policy_version 691094 (0.0007) [2023-12-27 10:24:37,241][20347] Updated weights for policy 0, policy_version 691104 (0.0006) [2023-12-27 10:24:37,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19933.9, 300 sec: 20632.7). Total num frames: 353771520. Throughput: 0: 9895.1, 1: 9733.8. Samples: 353739060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:37,363][19170] Avg episode reward: [(0, '884.010'), (1, '980.550')] [2023-12-27 10:24:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000690616_176824320.pth... [2023-12-27 10:24:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000691112_176947200.pth... [2023-12-27 10:24:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000690024_176668672.pth [2023-12-27 10:24:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000689528_176545792.pth [2023-12-27 10:24:37,698][20381] Updated weights for policy 1, policy_version 690619 (0.0009) [2023-12-27 10:24:37,769][20381] Updated weights for policy 1, policy_version 690629 (0.0009) [2023-12-27 10:24:37,829][20292] KL-divergence is very high: 171.8716 [2023-12-27 10:24:37,835][20381] Updated weights for policy 1, policy_version 690639 (0.0009) [2023-12-27 10:24:37,883][20347] Updated weights for policy 0, policy_version 691114 (0.0006) [2023-12-27 10:24:37,950][20347] Updated weights for policy 0, policy_version 691124 (0.0006) [2023-12-27 10:24:38,017][20347] Updated weights for policy 0, policy_version 691134 (0.0006) [2023-12-27 10:24:38,077][20347] Updated weights for policy 0, policy_version 691144 (0.0006) [2023-12-27 10:24:38,578][20381] Updated weights for policy 1, policy_version 690649 (0.0008) [2023-12-27 10:24:38,642][20381] Updated weights for policy 1, policy_version 690659 (0.0008) [2023-12-27 10:24:38,699][20381] Updated weights for policy 1, policy_version 690669 (0.0007) [2023-12-27 10:24:38,748][20347] Updated weights for policy 0, policy_version 691154 (0.0007) [2023-12-27 10:24:38,756][20381] Updated weights for policy 1, policy_version 690679 (0.0006) [2023-12-27 10:24:38,813][20347] Updated weights for policy 0, policy_version 691164 (0.0008) [2023-12-27 10:24:38,881][20347] Updated weights for policy 0, policy_version 691174 (0.0008) [2023-12-27 10:24:39,488][20381] Updated weights for policy 1, policy_version 690689 (0.0008) [2023-12-27 10:24:39,553][20381] Updated weights for policy 1, policy_version 690699 (0.0008) [2023-12-27 10:24:39,591][20347] Updated weights for policy 0, policy_version 691184 (0.0008) [2023-12-27 10:24:39,613][20381] Updated weights for policy 1, policy_version 690709 (0.0007) [2023-12-27 10:24:39,651][20347] Updated weights for policy 0, policy_version 691194 (0.0006) [2023-12-27 10:24:39,708][20347] Updated weights for policy 0, policy_version 691204 (0.0008) [2023-12-27 10:24:40,374][20381] Updated weights for policy 1, policy_version 690719 (0.0007) [2023-12-27 10:24:40,441][20381] Updated weights for policy 1, policy_version 690729 (0.0009) [2023-12-27 10:24:40,489][20381] Updated weights for policy 1, policy_version 690739 (0.0008) [2023-12-27 10:24:40,501][20347] Updated weights for policy 0, policy_version 691214 (0.0008) [2023-12-27 10:24:40,562][20347] Updated weights for policy 0, policy_version 691224 (0.0006) [2023-12-27 10:24:40,624][20347] Updated weights for policy 0, policy_version 691234 (0.0006) [2023-12-27 10:24:41,352][20381] Updated weights for policy 1, policy_version 690749 (0.0009) [2023-12-27 10:24:41,378][20347] Updated weights for policy 0, policy_version 691244 (0.0008) [2023-12-27 10:24:41,419][20381] Updated weights for policy 1, policy_version 690759 (0.0007) [2023-12-27 10:24:41,445][20347] Updated weights for policy 0, policy_version 691254 (0.0011) [2023-12-27 10:24:41,486][20381] Updated weights for policy 1, policy_version 690769 (0.0006) [2023-12-27 10:24:41,504][20347] Updated weights for policy 0, policy_version 691264 (0.0010) [2023-12-27 10:24:42,208][20381] Updated weights for policy 1, policy_version 690779 (0.0007) [2023-12-27 10:24:42,264][20347] Updated weights for policy 0, policy_version 691274 (0.0009) [2023-12-27 10:24:42,288][20381] Updated weights for policy 1, policy_version 690789 (0.0007) [2023-12-27 10:24:42,292][20291] KL-divergence is very high: 145.9057 [2023-12-27 10:24:42,329][20347] Updated weights for policy 0, policy_version 691284 (0.0008) [2023-12-27 10:24:42,345][20291] KL-divergence is very high: 164.0039 [2023-12-27 10:24:42,355][20381] Updated weights for policy 1, policy_version 690799 (0.0007) [2023-12-27 10:24:42,362][19170] Fps is (10 sec: 18022.3, 60 sec: 19660.8, 300 sec: 20549.4). Total num frames: 353853440. Throughput: 0: 9740.2, 1: 9624.8. Samples: 353853280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:42,363][19170] Avg episode reward: [(0, '875.290'), (1, '990.270')] [2023-12-27 10:24:42,396][20291] KL-divergence is very high: 134.1516 [2023-12-27 10:24:42,397][20347] Updated weights for policy 0, policy_version 691294 (0.0008) [2023-12-27 10:24:42,455][20347] Updated weights for policy 0, policy_version 691304 (0.0011) [2023-12-27 10:24:43,022][20381] Updated weights for policy 1, policy_version 690809 (0.0007) [2023-12-27 10:24:43,080][20381] Updated weights for policy 1, policy_version 690819 (0.0008) [2023-12-27 10:24:43,133][20381] Updated weights for policy 1, policy_version 690829 (0.0008) [2023-12-27 10:24:43,196][20381] Updated weights for policy 1, policy_version 690839 (0.0007) [2023-12-27 10:24:43,211][20347] Updated weights for policy 0, policy_version 691314 (0.0007) [2023-12-27 10:24:43,257][20347] Updated weights for policy 0, policy_version 691324 (0.0008) [2023-12-27 10:24:43,313][20347] Updated weights for policy 0, policy_version 691334 (0.0006) [2023-12-27 10:24:43,953][20381] Updated weights for policy 1, policy_version 690849 (0.0007) [2023-12-27 10:24:43,978][20347] Updated weights for policy 0, policy_version 691344 (0.0005) [2023-12-27 10:24:44,006][20381] Updated weights for policy 1, policy_version 690859 (0.0007) [2023-12-27 10:24:44,038][20347] Updated weights for policy 0, policy_version 691354 (0.0006) [2023-12-27 10:24:44,069][20381] Updated weights for policy 1, policy_version 690869 (0.0007) [2023-12-27 10:24:44,097][20347] Updated weights for policy 0, policy_version 691364 (0.0007) [2023-12-27 10:24:44,676][20347] Updated weights for policy 0, policy_version 691374 (0.0007) [2023-12-27 10:24:44,695][20291] KL-divergence is very high: 471.0325 [2023-12-27 10:24:44,702][20291] KL-divergence is very high: 145.4816 [2023-12-27 10:24:44,708][20291] KL-divergence is very high: 443.1709 [2023-12-27 10:24:44,721][20291] KL-divergence is very high: 440.0557 [2023-12-27 10:24:44,739][20347] Updated weights for policy 0, policy_version 691384 (0.0006) [2023-12-27 10:24:44,749][20291] KL-divergence is very high: 908.0632 [2023-12-27 10:24:44,756][20291] KL-divergence is very high: 135.0369 [2023-12-27 10:24:44,763][20291] KL-divergence is very high: 620.2741 [2023-12-27 10:24:44,777][20291] KL-divergence is very high: 314.4441 [2023-12-27 10:24:44,785][20381] Updated weights for policy 1, policy_version 690879 (0.0007) [2023-12-27 10:24:44,804][20291] KL-divergence is very high: 446.3918 [2023-12-27 10:24:44,812][20347] Updated weights for policy 0, policy_version 691394 (0.0006) [2023-12-27 10:24:44,818][20291] KL-divergence is very high: 226.6640 [2023-12-27 10:24:44,832][20291] KL-divergence is very high: 127.8672 [2023-12-27 10:24:44,840][20381] Updated weights for policy 1, policy_version 690889 (0.0007) [2023-12-27 10:24:44,901][20381] Updated weights for policy 1, policy_version 690899 (0.0006) [2023-12-27 10:24:45,503][20347] Updated weights for policy 0, policy_version 691404 (0.0009) [2023-12-27 10:24:45,527][20381] Updated weights for policy 1, policy_version 690909 (0.0010) [2023-12-27 10:24:45,555][20347] Updated weights for policy 0, policy_version 691414 (0.0009) [2023-12-27 10:24:45,587][20381] Updated weights for policy 1, policy_version 690919 (0.0011) [2023-12-27 10:24:45,617][20347] Updated weights for policy 0, policy_version 691424 (0.0007) [2023-12-27 10:24:45,651][20381] Updated weights for policy 1, policy_version 690929 (0.0007) [2023-12-27 10:24:46,340][20381] Updated weights for policy 1, policy_version 690939 (0.0007) [2023-12-27 10:24:46,355][20347] Updated weights for policy 0, policy_version 691434 (0.0009) [2023-12-27 10:24:46,403][20381] Updated weights for policy 1, policy_version 690949 (0.0006) [2023-12-27 10:24:46,420][20347] Updated weights for policy 0, policy_version 691444 (0.0008) [2023-12-27 10:24:46,465][20381] Updated weights for policy 1, policy_version 690959 (0.0007) [2023-12-27 10:24:46,484][20347] Updated weights for policy 0, policy_version 691454 (0.0007) [2023-12-27 10:24:46,552][20347] Updated weights for policy 0, policy_version 691464 (0.0008) [2023-12-27 10:24:47,182][20381] Updated weights for policy 1, policy_version 690969 (0.0008) [2023-12-27 10:24:47,241][20381] Updated weights for policy 1, policy_version 690979 (0.0010) [2023-12-27 10:24:47,304][20381] Updated weights for policy 1, policy_version 690989 (0.0010) [2023-12-27 10:24:47,316][20347] Updated weights for policy 0, policy_version 691474 (0.0006) [2023-12-27 10:24:47,362][19170] Fps is (10 sec: 18022.3, 60 sec: 19524.2, 300 sec: 20521.7). Total num frames: 353951744. Throughput: 0: 9595.2, 1: 9545.6. Samples: 353969828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:47,363][20381] Updated weights for policy 1, policy_version 690999 (0.0008) [2023-12-27 10:24:47,364][19170] Avg episode reward: [(0, '847.470'), (1, '1000.000')] [2023-12-27 10:24:47,377][20347] Updated weights for policy 0, policy_version 691484 (0.0006) [2023-12-27 10:24:47,433][20347] Updated weights for policy 0, policy_version 691494 (0.0007) [2023-12-27 10:24:48,093][20381] Updated weights for policy 1, policy_version 691009 (0.0009) [2023-12-27 10:24:48,155][20381] Updated weights for policy 1, policy_version 691019 (0.0011) [2023-12-27 10:24:48,214][20381] Updated weights for policy 1, policy_version 691029 (0.0011) [2023-12-27 10:24:48,228][20347] Updated weights for policy 0, policy_version 691504 (0.0008) [2023-12-27 10:24:48,289][20347] Updated weights for policy 0, policy_version 691514 (0.0007) [2023-12-27 10:24:48,356][20347] Updated weights for policy 0, policy_version 691524 (0.0008) [2023-12-27 10:24:48,940][20381] Updated weights for policy 1, policy_version 691039 (0.0009) [2023-12-27 10:24:48,994][20381] Updated weights for policy 1, policy_version 691049 (0.0008) [2023-12-27 10:24:49,052][20381] Updated weights for policy 1, policy_version 691059 (0.0008) [2023-12-27 10:24:49,054][20347] Updated weights for policy 0, policy_version 691534 (0.0008) [2023-12-27 10:24:49,116][20347] Updated weights for policy 0, policy_version 691544 (0.0007) [2023-12-27 10:24:49,179][20347] Updated weights for policy 0, policy_version 691554 (0.0007) [2023-12-27 10:24:49,693][20381] Updated weights for policy 1, policy_version 691069 (0.0006) [2023-12-27 10:24:49,749][20381] Updated weights for policy 1, policy_version 691079 (0.0005) [2023-12-27 10:24:49,797][20381] Updated weights for policy 1, policy_version 691089 (0.0005) [2023-12-27 10:24:49,887][20347] Updated weights for policy 0, policy_version 691564 (0.0007) [2023-12-27 10:24:49,949][20347] Updated weights for policy 0, policy_version 691574 (0.0008) [2023-12-27 10:24:50,005][20347] Updated weights for policy 0, policy_version 691584 (0.0008) [2023-12-27 10:24:50,451][20381] Updated weights for policy 1, policy_version 691099 (0.0006) [2023-12-27 10:24:50,513][20381] Updated weights for policy 1, policy_version 691109 (0.0006) [2023-12-27 10:24:50,569][20381] Updated weights for policy 1, policy_version 691119 (0.0008) [2023-12-27 10:24:50,715][20347] Updated weights for policy 0, policy_version 691594 (0.0007) [2023-12-27 10:24:50,776][20347] Updated weights for policy 0, policy_version 691604 (0.0007) [2023-12-27 10:24:50,836][20347] Updated weights for policy 0, policy_version 691614 (0.0008) [2023-12-27 10:24:50,896][20347] Updated weights for policy 0, policy_version 691624 (0.0008) [2023-12-27 10:24:51,258][20381] Updated weights for policy 1, policy_version 691129 (0.0007) [2023-12-27 10:24:51,317][20381] Updated weights for policy 1, policy_version 691139 (0.0008) [2023-12-27 10:24:51,386][20381] Updated weights for policy 1, policy_version 691149 (0.0008) [2023-12-27 10:24:51,448][20381] Updated weights for policy 1, policy_version 691159 (0.0006) [2023-12-27 10:24:51,618][20347] Updated weights for policy 0, policy_version 691634 (0.0008) [2023-12-27 10:24:51,679][20347] Updated weights for policy 0, policy_version 691644 (0.0009) [2023-12-27 10:24:51,740][20347] Updated weights for policy 0, policy_version 691654 (0.0007) [2023-12-27 10:24:52,129][20381] Updated weights for policy 1, policy_version 691169 (0.0007) [2023-12-27 10:24:52,181][20381] Updated weights for policy 1, policy_version 691179 (0.0008) [2023-12-27 10:24:52,241][20381] Updated weights for policy 1, policy_version 691189 (0.0008) [2023-12-27 10:24:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 20549.4). Total num frames: 354058240. Throughput: 0: 9492.8, 1: 9516.9. Samples: 354027696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:52,363][19170] Avg episode reward: [(0, '793.320'), (1, '1000.000')] [2023-12-27 10:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000691192_176971776.pth... [2023-12-27 10:24:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000691656_177086464.pth... [2023-12-27 10:24:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000690072_176685056.pth [2023-12-27 10:24:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000690536_176799744.pth [2023-12-27 10:24:52,478][20347] Updated weights for policy 0, policy_version 691664 (0.0008) [2023-12-27 10:24:52,538][20347] Updated weights for policy 0, policy_version 691674 (0.0008) [2023-12-27 10:24:52,598][20347] Updated weights for policy 0, policy_version 691684 (0.0008) [2023-12-27 10:24:52,930][20381] Updated weights for policy 1, policy_version 691199 (0.0006) [2023-12-27 10:24:52,985][20381] Updated weights for policy 1, policy_version 691209 (0.0005) [2023-12-27 10:24:53,049][20381] Updated weights for policy 1, policy_version 691219 (0.0008) [2023-12-27 10:24:53,267][20347] Updated weights for policy 0, policy_version 691694 (0.0006) [2023-12-27 10:24:53,323][20347] Updated weights for policy 0, policy_version 691704 (0.0005) [2023-12-27 10:24:53,368][20347] Updated weights for policy 0, policy_version 691714 (0.0006) [2023-12-27 10:24:53,687][20381] Updated weights for policy 1, policy_version 691229 (0.0008) [2023-12-27 10:24:53,750][20381] Updated weights for policy 1, policy_version 691239 (0.0011) [2023-12-27 10:24:53,805][20381] Updated weights for policy 1, policy_version 691249 (0.0011) [2023-12-27 10:24:54,044][20347] Updated weights for policy 0, policy_version 691724 (0.0008) [2023-12-27 10:24:54,093][20347] Updated weights for policy 0, policy_version 691734 (0.0007) [2023-12-27 10:24:54,145][20347] Updated weights for policy 0, policy_version 691744 (0.0008) [2023-12-27 10:24:54,390][20381] Updated weights for policy 1, policy_version 691259 (0.0009) [2023-12-27 10:24:54,445][20381] Updated weights for policy 1, policy_version 691269 (0.0008) [2023-12-27 10:24:54,498][20381] Updated weights for policy 1, policy_version 691279 (0.0005) [2023-12-27 10:24:54,808][20347] Updated weights for policy 0, policy_version 691754 (0.0006) [2023-12-27 10:24:54,867][20347] Updated weights for policy 0, policy_version 691764 (0.0005) [2023-12-27 10:24:54,928][20347] Updated weights for policy 0, policy_version 691774 (0.0006) [2023-12-27 10:24:54,985][20347] Updated weights for policy 0, policy_version 691784 (0.0007) [2023-12-27 10:24:55,126][20381] Updated weights for policy 1, policy_version 691289 (0.0009) [2023-12-27 10:24:55,177][20381] Updated weights for policy 1, policy_version 691299 (0.0005) [2023-12-27 10:24:55,222][20381] Updated weights for policy 1, policy_version 691309 (0.0008) [2023-12-27 10:24:55,272][20381] Updated weights for policy 1, policy_version 691319 (0.0010) [2023-12-27 10:24:55,591][20347] Updated weights for policy 0, policy_version 691794 (0.0007) [2023-12-27 10:24:55,641][20347] Updated weights for policy 0, policy_version 691804 (0.0007) [2023-12-27 10:24:55,699][20347] Updated weights for policy 0, policy_version 691814 (0.0007) [2023-12-27 10:24:55,925][20381] Updated weights for policy 1, policy_version 691329 (0.0007) [2023-12-27 10:24:55,974][20381] Updated weights for policy 1, policy_version 691339 (0.0008) [2023-12-27 10:24:56,030][20381] Updated weights for policy 1, policy_version 691349 (0.0008) [2023-12-27 10:24:56,421][20347] Updated weights for policy 0, policy_version 691824 (0.0008) [2023-12-27 10:24:56,477][20347] Updated weights for policy 0, policy_version 691834 (0.0008) [2023-12-27 10:24:56,529][20347] Updated weights for policy 0, policy_version 691844 (0.0008) [2023-12-27 10:24:56,642][20381] Updated weights for policy 1, policy_version 691359 (0.0006) [2023-12-27 10:24:56,687][20381] Updated weights for policy 1, policy_version 691369 (0.0005) [2023-12-27 10:24:56,727][20381] Updated weights for policy 1, policy_version 691379 (0.0005) [2023-12-27 10:24:57,241][20347] Updated weights for policy 0, policy_version 691854 (0.0007) [2023-12-27 10:24:57,295][20347] Updated weights for policy 0, policy_version 691864 (0.0007) [2023-12-27 10:24:57,353][20347] Updated weights for policy 0, policy_version 691874 (0.0007) [2023-12-27 10:24:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19387.7, 300 sec: 20521.7). Total num frames: 354156544. Throughput: 0: 9514.1, 1: 9564.7. Samples: 354151252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:24:57,363][19170] Avg episode reward: [(0, '726.580'), (1, '980.620')] [2023-12-27 10:24:57,373][20381] Updated weights for policy 1, policy_version 691389 (0.0007) [2023-12-27 10:24:57,424][20381] Updated weights for policy 1, policy_version 691399 (0.0008) [2023-12-27 10:24:57,482][20381] Updated weights for policy 1, policy_version 691409 (0.0006) [2023-12-27 10:24:58,067][20347] Updated weights for policy 0, policy_version 691884 (0.0007) [2023-12-27 10:24:58,109][20381] Updated weights for policy 1, policy_version 691419 (0.0005) [2023-12-27 10:24:58,115][20347] Updated weights for policy 0, policy_version 691894 (0.0007) [2023-12-27 10:24:58,154][20381] Updated weights for policy 1, policy_version 691429 (0.0006) [2023-12-27 10:24:58,160][20347] Updated weights for policy 0, policy_version 691904 (0.0005) [2023-12-27 10:24:58,217][20381] Updated weights for policy 1, policy_version 691439 (0.0007) [2023-12-27 10:24:59,011][20347] Updated weights for policy 0, policy_version 691914 (0.0008) [2023-12-27 10:24:59,048][20381] Updated weights for policy 1, policy_version 691449 (0.0007) [2023-12-27 10:24:59,069][20347] Updated weights for policy 0, policy_version 691924 (0.0007) [2023-12-27 10:24:59,104][20381] Updated weights for policy 1, policy_version 691459 (0.0007) [2023-12-27 10:24:59,123][20347] Updated weights for policy 0, policy_version 691934 (0.0006) [2023-12-27 10:24:59,163][20381] Updated weights for policy 1, policy_version 691469 (0.0007) [2023-12-27 10:24:59,181][20347] Updated weights for policy 0, policy_version 691944 (0.0007) [2023-12-27 10:24:59,228][20381] Updated weights for policy 1, policy_version 691479 (0.0007) [2023-12-27 10:24:59,886][20381] Updated weights for policy 1, policy_version 691489 (0.0006) [2023-12-27 10:24:59,915][20347] Updated weights for policy 0, policy_version 691954 (0.0010) [2023-12-27 10:24:59,941][20381] Updated weights for policy 1, policy_version 691499 (0.0006) [2023-12-27 10:24:59,979][20347] Updated weights for policy 0, policy_version 691964 (0.0007) [2023-12-27 10:24:59,998][20381] Updated weights for policy 1, policy_version 691509 (0.0006) [2023-12-27 10:25:00,045][20347] Updated weights for policy 0, policy_version 691974 (0.0007) [2023-12-27 10:25:00,628][20347] Updated weights for policy 0, policy_version 691984 (0.0006) [2023-12-27 10:25:00,688][20347] Updated weights for policy 0, policy_version 691994 (0.0005) [2023-12-27 10:25:00,706][20381] Updated weights for policy 1, policy_version 691519 (0.0009) [2023-12-27 10:25:00,742][20347] Updated weights for policy 0, policy_version 692004 (0.0008) [2023-12-27 10:25:00,757][20381] Updated weights for policy 1, policy_version 691529 (0.0010) [2023-12-27 10:25:00,809][20381] Updated weights for policy 1, policy_version 691539 (0.0007) [2023-12-27 10:25:01,349][20347] Updated weights for policy 0, policy_version 692014 (0.0009) [2023-12-27 10:25:01,412][20347] Updated weights for policy 0, policy_version 692024 (0.0010) [2023-12-27 10:25:01,474][20347] Updated weights for policy 0, policy_version 692034 (0.0006) [2023-12-27 10:25:01,540][20381] Updated weights for policy 1, policy_version 691549 (0.0009) [2023-12-27 10:25:01,593][20381] Updated weights for policy 1, policy_version 691559 (0.0007) [2023-12-27 10:25:01,655][20381] Updated weights for policy 1, policy_version 691569 (0.0007) [2023-12-27 10:25:02,147][20347] Updated weights for policy 0, policy_version 692044 (0.0009) [2023-12-27 10:25:02,195][20347] Updated weights for policy 0, policy_version 692054 (0.0010) [2023-12-27 10:25:02,244][20347] Updated weights for policy 0, policy_version 692064 (0.0009) [2023-12-27 10:25:02,320][20381] Updated weights for policy 1, policy_version 691579 (0.0007) [2023-12-27 10:25:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 20549.4). Total num frames: 354263040. Throughput: 0: 9661.8, 1: 9700.1. Samples: 354274160. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:02,363][19170] Avg episode reward: [(0, '799.540'), (1, '980.620')] [2023-12-27 10:25:02,387][20381] Updated weights for policy 1, policy_version 691589 (0.0007) [2023-12-27 10:25:02,438][20381] Updated weights for policy 1, policy_version 691599 (0.0006) [2023-12-27 10:25:02,856][20347] Updated weights for policy 0, policy_version 692074 (0.0008) [2023-12-27 10:25:02,921][20347] Updated weights for policy 0, policy_version 692084 (0.0005) [2023-12-27 10:25:02,984][20347] Updated weights for policy 0, policy_version 692094 (0.0007) [2023-12-27 10:25:03,036][20347] Updated weights for policy 0, policy_version 692104 (0.0008) [2023-12-27 10:25:03,125][20381] Updated weights for policy 1, policy_version 691609 (0.0008) [2023-12-27 10:25:03,170][20381] Updated weights for policy 1, policy_version 691619 (0.0010) [2023-12-27 10:25:03,224][20381] Updated weights for policy 1, policy_version 691629 (0.0010) [2023-12-27 10:25:03,272][20381] Updated weights for policy 1, policy_version 691639 (0.0010) [2023-12-27 10:25:03,636][20347] Updated weights for policy 0, policy_version 692114 (0.0006) [2023-12-27 10:25:03,690][20347] Updated weights for policy 0, policy_version 692124 (0.0006) [2023-12-27 10:25:03,747][20347] Updated weights for policy 0, policy_version 692134 (0.0010) [2023-12-27 10:25:03,929][20381] Updated weights for policy 1, policy_version 691649 (0.0011) [2023-12-27 10:25:03,980][20381] Updated weights for policy 1, policy_version 691659 (0.0010) [2023-12-27 10:25:04,042][20381] Updated weights for policy 1, policy_version 691669 (0.0011) [2023-12-27 10:25:04,365][20347] Updated weights for policy 0, policy_version 692144 (0.0006) [2023-12-27 10:25:04,427][20347] Updated weights for policy 0, policy_version 692154 (0.0006) [2023-12-27 10:25:04,487][20347] Updated weights for policy 0, policy_version 692164 (0.0005) [2023-12-27 10:25:04,748][20381] Updated weights for policy 1, policy_version 691679 (0.0011) [2023-12-27 10:25:04,804][20381] Updated weights for policy 1, policy_version 691689 (0.0007) [2023-12-27 10:25:04,866][20381] Updated weights for policy 1, policy_version 691699 (0.0010) [2023-12-27 10:25:05,085][20347] Updated weights for policy 0, policy_version 692174 (0.0005) [2023-12-27 10:25:05,155][20347] Updated weights for policy 0, policy_version 692184 (0.0005) [2023-12-27 10:25:05,210][20347] Updated weights for policy 0, policy_version 692194 (0.0010) [2023-12-27 10:25:05,535][20381] Updated weights for policy 1, policy_version 691709 (0.0009) [2023-12-27 10:25:05,585][20381] Updated weights for policy 1, policy_version 691719 (0.0009) [2023-12-27 10:25:05,636][20381] Updated weights for policy 1, policy_version 691729 (0.0010) [2023-12-27 10:25:05,844][20347] Updated weights for policy 0, policy_version 692204 (0.0010) [2023-12-27 10:25:05,898][20347] Updated weights for policy 0, policy_version 692214 (0.0010) [2023-12-27 10:25:05,950][20347] Updated weights for policy 0, policy_version 692224 (0.0005) [2023-12-27 10:25:06,324][20381] Updated weights for policy 1, policy_version 691739 (0.0010) [2023-12-27 10:25:06,379][20381] Updated weights for policy 1, policy_version 691749 (0.0009) [2023-12-27 10:25:06,445][20381] Updated weights for policy 1, policy_version 691759 (0.0009) [2023-12-27 10:25:06,597][20347] Updated weights for policy 0, policy_version 692234 (0.0006) [2023-12-27 10:25:06,660][20347] Updated weights for policy 0, policy_version 692244 (0.0008) [2023-12-27 10:25:06,724][20347] Updated weights for policy 0, policy_version 692254 (0.0008) [2023-12-27 10:25:06,784][20347] Updated weights for policy 0, policy_version 692264 (0.0008) [2023-12-27 10:25:07,185][20381] Updated weights for policy 1, policy_version 691769 (0.0008) [2023-12-27 10:25:07,243][20381] Updated weights for policy 1, policy_version 691779 (0.0007) [2023-12-27 10:25:07,292][20381] Updated weights for policy 1, policy_version 691789 (0.0008) [2023-12-27 10:25:07,344][20381] Updated weights for policy 1, policy_version 691799 (0.0008) [2023-12-27 10:25:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 19387.7, 300 sec: 20549.4). Total num frames: 354369536. Throughput: 0: 9776.3, 1: 9743.1. Samples: 354338508. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:07,363][19170] Avg episode reward: [(0, '854.460'), (1, '990.310')] [2023-12-27 10:25:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000691800_177127424.pth... [2023-12-27 10:25:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000690616_176824320.pth [2023-12-27 10:25:07,426][20347] Updated weights for policy 0, policy_version 692274 (0.0008) [2023-12-27 10:25:07,491][20347] Updated weights for policy 0, policy_version 692284 (0.0008) [2023-12-27 10:25:07,546][20347] Updated weights for policy 0, policy_version 692294 (0.0007) [2023-12-27 10:25:07,557][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000692296_177250304.pth... [2023-12-27 10:25:07,560][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000691112_176947200.pth [2023-12-27 10:25:07,967][20381] Updated weights for policy 1, policy_version 691809 (0.0005) [2023-12-27 10:25:08,017][20381] Updated weights for policy 1, policy_version 691819 (0.0007) [2023-12-27 10:25:08,071][20381] Updated weights for policy 1, policy_version 691829 (0.0005) [2023-12-27 10:25:08,203][20347] Updated weights for policy 0, policy_version 692304 (0.0005) [2023-12-27 10:25:08,249][20347] Updated weights for policy 0, policy_version 692314 (0.0005) [2023-12-27 10:25:08,306][20347] Updated weights for policy 0, policy_version 692324 (0.0006) [2023-12-27 10:25:08,686][20381] Updated weights for policy 1, policy_version 691839 (0.0005) [2023-12-27 10:25:08,750][20381] Updated weights for policy 1, policy_version 691849 (0.0007) [2023-12-27 10:25:08,812][20381] Updated weights for policy 1, policy_version 691859 (0.0008) [2023-12-27 10:25:08,986][20347] Updated weights for policy 0, policy_version 692334 (0.0006) [2023-12-27 10:25:09,039][20347] Updated weights for policy 0, policy_version 692344 (0.0006) [2023-12-27 10:25:09,086][20347] Updated weights for policy 0, policy_version 692354 (0.0005) [2023-12-27 10:25:09,503][20381] Updated weights for policy 1, policy_version 691869 (0.0008) [2023-12-27 10:25:09,568][20381] Updated weights for policy 1, policy_version 691879 (0.0007) [2023-12-27 10:25:09,639][20381] Updated weights for policy 1, policy_version 691889 (0.0009) [2023-12-27 10:25:09,775][20347] Updated weights for policy 0, policy_version 692364 (0.0007) [2023-12-27 10:25:09,858][20347] Updated weights for policy 0, policy_version 692374 (0.0008) [2023-12-27 10:25:09,918][20347] Updated weights for policy 0, policy_version 692384 (0.0008) [2023-12-27 10:25:10,276][20381] Updated weights for policy 1, policy_version 691899 (0.0008) [2023-12-27 10:25:10,342][20381] Updated weights for policy 1, policy_version 691909 (0.0006) [2023-12-27 10:25:10,398][20381] Updated weights for policy 1, policy_version 691919 (0.0006) [2023-12-27 10:25:10,626][20347] Updated weights for policy 0, policy_version 692394 (0.0009) [2023-12-27 10:25:10,683][20347] Updated weights for policy 0, policy_version 692404 (0.0010) [2023-12-27 10:25:10,739][20347] Updated weights for policy 0, policy_version 692414 (0.0010) [2023-12-27 10:25:10,803][20347] Updated weights for policy 0, policy_version 692424 (0.0010) [2023-12-27 10:25:10,975][20381] Updated weights for policy 1, policy_version 691929 (0.0006) [2023-12-27 10:25:11,038][20381] Updated weights for policy 1, policy_version 691939 (0.0009) [2023-12-27 10:25:11,101][20381] Updated weights for policy 1, policy_version 691949 (0.0011) [2023-12-27 10:25:11,167][20381] Updated weights for policy 1, policy_version 691959 (0.0008) [2023-12-27 10:25:11,511][20347] Updated weights for policy 0, policy_version 692434 (0.0008) [2023-12-27 10:25:11,566][20347] Updated weights for policy 0, policy_version 692444 (0.0008) [2023-12-27 10:25:11,634][20347] Updated weights for policy 0, policy_version 692454 (0.0012) [2023-12-27 10:25:11,887][20381] Updated weights for policy 1, policy_version 691969 (0.0007) [2023-12-27 10:25:11,948][20381] Updated weights for policy 1, policy_version 691979 (0.0007) [2023-12-27 10:25:12,004][20381] Updated weights for policy 1, policy_version 691989 (0.0008) [2023-12-27 10:25:12,349][20347] Updated weights for policy 0, policy_version 692464 (0.0009) [2023-12-27 10:25:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 20549.4). Total num frames: 354467840. Throughput: 0: 9987.0, 1: 9895.2. Samples: 354462748. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:12,363][19170] Avg episode reward: [(0, '854.400'), (1, '970.890')] [2023-12-27 10:25:12,402][20347] Updated weights for policy 0, policy_version 692474 (0.0010) [2023-12-27 10:25:12,455][20347] Updated weights for policy 0, policy_version 692484 (0.0010) [2023-12-27 10:25:12,744][20381] Updated weights for policy 1, policy_version 691999 (0.0007) [2023-12-27 10:25:12,802][20381] Updated weights for policy 1, policy_version 692009 (0.0010) [2023-12-27 10:25:12,868][20381] Updated weights for policy 1, policy_version 692019 (0.0010) [2023-12-27 10:25:13,059][20347] Updated weights for policy 0, policy_version 692494 (0.0008) [2023-12-27 10:25:13,125][20347] Updated weights for policy 0, policy_version 692504 (0.0005) [2023-12-27 10:25:13,184][20347] Updated weights for policy 0, policy_version 692514 (0.0005) [2023-12-27 10:25:13,542][20381] Updated weights for policy 1, policy_version 692029 (0.0010) [2023-12-27 10:25:13,588][20381] Updated weights for policy 1, policy_version 692039 (0.0011) [2023-12-27 10:25:13,650][20381] Updated weights for policy 1, policy_version 692049 (0.0011) [2023-12-27 10:25:13,701][20347] Updated weights for policy 0, policy_version 692524 (0.0007) [2023-12-27 10:25:13,767][20347] Updated weights for policy 0, policy_version 692534 (0.0010) [2023-12-27 10:25:13,828][20347] Updated weights for policy 0, policy_version 692544 (0.0005) [2023-12-27 10:25:14,342][20381] Updated weights for policy 1, policy_version 692059 (0.0009) [2023-12-27 10:25:14,396][20381] Updated weights for policy 1, policy_version 692069 (0.0005) [2023-12-27 10:25:14,411][20347] Updated weights for policy 0, policy_version 692554 (0.0006) [2023-12-27 10:25:14,445][20381] Updated weights for policy 1, policy_version 692079 (0.0010) [2023-12-27 10:25:14,469][20347] Updated weights for policy 0, policy_version 692564 (0.0008) [2023-12-27 10:25:14,525][20347] Updated weights for policy 0, policy_version 692574 (0.0005) [2023-12-27 10:25:14,590][20347] Updated weights for policy 0, policy_version 692584 (0.0005) [2023-12-27 10:25:15,053][20381] Updated weights for policy 1, policy_version 692089 (0.0010) [2023-12-27 10:25:15,117][20381] Updated weights for policy 1, policy_version 692099 (0.0006) [2023-12-27 10:25:15,180][20381] Updated weights for policy 1, policy_version 692109 (0.0008) [2023-12-27 10:25:15,191][20347] Updated weights for policy 0, policy_version 692594 (0.0010) [2023-12-27 10:25:15,235][20381] Updated weights for policy 1, policy_version 692119 (0.0010) [2023-12-27 10:25:15,246][20347] Updated weights for policy 0, policy_version 692604 (0.0010) [2023-12-27 10:25:15,304][20347] Updated weights for policy 0, policy_version 692614 (0.0010) [2023-12-27 10:25:15,806][20381] Updated weights for policy 1, policy_version 692129 (0.0008) [2023-12-27 10:25:15,860][20381] Updated weights for policy 1, policy_version 692139 (0.0008) [2023-12-27 10:25:15,914][20381] Updated weights for policy 1, policy_version 692149 (0.0008) [2023-12-27 10:25:15,973][20347] Updated weights for policy 0, policy_version 692624 (0.0007) [2023-12-27 10:25:16,037][20347] Updated weights for policy 0, policy_version 692634 (0.0007) [2023-12-27 10:25:16,101][20347] Updated weights for policy 0, policy_version 692644 (0.0005) [2023-12-27 10:25:16,565][20381] Updated weights for policy 1, policy_version 692159 (0.0007) [2023-12-27 10:25:16,615][20381] Updated weights for policy 1, policy_version 692169 (0.0008) [2023-12-27 10:25:16,665][20381] Updated weights for policy 1, policy_version 692179 (0.0008) [2023-12-27 10:25:16,737][20347] Updated weights for policy 0, policy_version 692654 (0.0007) [2023-12-27 10:25:16,789][20347] Updated weights for policy 0, policy_version 692664 (0.0010) [2023-12-27 10:25:16,838][20347] Updated weights for policy 0, policy_version 692674 (0.0010) [2023-12-27 10:25:17,326][20381] Updated weights for policy 1, policy_version 692189 (0.0007) [2023-12-27 10:25:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 19660.8, 300 sec: 20549.4). Total num frames: 354574336. Throughput: 0: 10141.3, 1: 10061.6. Samples: 354589868. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:17,363][19170] Avg episode reward: [(0, '912.380'), (1, '961.160')] [2023-12-27 10:25:17,383][20381] Updated weights for policy 1, policy_version 692199 (0.0006) [2023-12-27 10:25:17,436][20381] Updated weights for policy 1, policy_version 692209 (0.0007) [2023-12-27 10:25:17,481][20347] Updated weights for policy 0, policy_version 692684 (0.0009) [2023-12-27 10:25:17,526][20347] Updated weights for policy 0, policy_version 692694 (0.0010) [2023-12-27 10:25:17,578][20347] Updated weights for policy 0, policy_version 692704 (0.0009) [2023-12-27 10:25:18,099][20381] Updated weights for policy 1, policy_version 692219 (0.0007) [2023-12-27 10:25:18,159][20381] Updated weights for policy 1, policy_version 692229 (0.0008) [2023-12-27 10:25:18,214][20381] Updated weights for policy 1, policy_version 692239 (0.0008) [2023-12-27 10:25:18,230][20347] Updated weights for policy 0, policy_version 692714 (0.0006) [2023-12-27 10:25:18,288][20347] Updated weights for policy 0, policy_version 692724 (0.0007) [2023-12-27 10:25:18,348][20347] Updated weights for policy 0, policy_version 692734 (0.0008) [2023-12-27 10:25:18,406][20347] Updated weights for policy 0, policy_version 692744 (0.0008) [2023-12-27 10:25:18,906][20381] Updated weights for policy 1, policy_version 692249 (0.0006) [2023-12-27 10:25:18,968][20381] Updated weights for policy 1, policy_version 692259 (0.0007) [2023-12-27 10:25:19,027][20381] Updated weights for policy 1, policy_version 692269 (0.0008) [2023-12-27 10:25:19,088][20381] Updated weights for policy 1, policy_version 692279 (0.0006) [2023-12-27 10:25:19,101][20347] Updated weights for policy 0, policy_version 692754 (0.0006) [2023-12-27 10:25:19,152][20347] Updated weights for policy 0, policy_version 692764 (0.0008) [2023-12-27 10:25:19,207][20347] Updated weights for policy 0, policy_version 692774 (0.0007) [2023-12-27 10:25:19,809][20381] Updated weights for policy 1, policy_version 692289 (0.0008) [2023-12-27 10:25:19,870][20381] Updated weights for policy 1, policy_version 692299 (0.0008) [2023-12-27 10:25:19,926][20347] Updated weights for policy 0, policy_version 692784 (0.0006) [2023-12-27 10:25:19,938][20381] Updated weights for policy 1, policy_version 692309 (0.0008) [2023-12-27 10:25:19,987][20347] Updated weights for policy 0, policy_version 692794 (0.0007) [2023-12-27 10:25:20,048][20347] Updated weights for policy 0, policy_version 692804 (0.0008) [2023-12-27 10:25:20,636][20381] Updated weights for policy 1, policy_version 692319 (0.0006) [2023-12-27 10:25:20,688][20381] Updated weights for policy 1, policy_version 692329 (0.0006) [2023-12-27 10:25:20,743][20381] Updated weights for policy 1, policy_version 692339 (0.0007) [2023-12-27 10:25:20,759][20347] Updated weights for policy 0, policy_version 692814 (0.0008) [2023-12-27 10:25:20,818][20347] Updated weights for policy 0, policy_version 692824 (0.0009) [2023-12-27 10:25:20,880][20347] Updated weights for policy 0, policy_version 692834 (0.0010) [2023-12-27 10:25:21,502][20381] Updated weights for policy 1, policy_version 692349 (0.0007) [2023-12-27 10:25:21,566][20381] Updated weights for policy 1, policy_version 692359 (0.0007) [2023-12-27 10:25:21,578][20347] Updated weights for policy 0, policy_version 692844 (0.0009) [2023-12-27 10:25:21,622][20381] Updated weights for policy 1, policy_version 692369 (0.0007) [2023-12-27 10:25:21,643][20347] Updated weights for policy 0, policy_version 692854 (0.0008) [2023-12-27 10:25:21,705][20347] Updated weights for policy 0, policy_version 692864 (0.0007) [2023-12-27 10:25:22,293][20381] Updated weights for policy 1, policy_version 692379 (0.0007) [2023-12-27 10:25:22,359][20381] Updated weights for policy 1, policy_version 692389 (0.0008) [2023-12-27 10:25:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 19797.3, 300 sec: 20549.4). Total num frames: 354672640. Throughput: 0: 10149.6, 1: 10102.3. Samples: 354650396. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:22,363][19170] Avg episode reward: [(0, '922.890'), (1, '961.160')] [2023-12-27 10:25:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000692872_177397760.pth... [2023-12-27 10:25:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000691656_177086464.pth [2023-12-27 10:25:22,400][20347] Updated weights for policy 0, policy_version 692874 (0.0007) [2023-12-27 10:25:22,420][20381] Updated weights for policy 1, policy_version 692399 (0.0007) [2023-12-27 10:25:22,456][20347] Updated weights for policy 0, policy_version 692884 (0.0006) [2023-12-27 10:25:22,470][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000692408_177283072.pth... [2023-12-27 10:25:22,475][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000691192_176971776.pth [2023-12-27 10:25:22,513][20347] Updated weights for policy 0, policy_version 692894 (0.0007) [2023-12-27 10:25:22,572][20347] Updated weights for policy 0, policy_version 692904 (0.0007) [2023-12-27 10:25:23,030][20381] Updated weights for policy 1, policy_version 692409 (0.0006) [2023-12-27 10:25:23,079][20381] Updated weights for policy 1, policy_version 692419 (0.0010) [2023-12-27 10:25:23,135][20381] Updated weights for policy 1, policy_version 692429 (0.0010) [2023-12-27 10:25:23,186][20381] Updated weights for policy 1, policy_version 692439 (0.0010) [2023-12-27 10:25:23,229][20347] Updated weights for policy 0, policy_version 692914 (0.0010) [2023-12-27 10:25:23,278][20347] Updated weights for policy 0, policy_version 692924 (0.0010) [2023-12-27 10:25:23,323][20347] Updated weights for policy 0, policy_version 692934 (0.0010) [2023-12-27 10:25:23,847][20381] Updated weights for policy 1, policy_version 692449 (0.0007) [2023-12-27 10:25:23,907][20381] Updated weights for policy 1, policy_version 692459 (0.0008) [2023-12-27 10:25:23,972][20381] Updated weights for policy 1, policy_version 692469 (0.0007) [2023-12-27 10:25:24,006][20347] Updated weights for policy 0, policy_version 692944 (0.0006) [2023-12-27 10:25:24,064][20347] Updated weights for policy 0, policy_version 692954 (0.0009) [2023-12-27 10:25:24,109][20347] Updated weights for policy 0, policy_version 692964 (0.0010) [2023-12-27 10:25:24,607][20381] Updated weights for policy 1, policy_version 692479 (0.0006) [2023-12-27 10:25:24,660][20381] Updated weights for policy 1, policy_version 692489 (0.0008) [2023-12-27 10:25:24,714][20381] Updated weights for policy 1, policy_version 692499 (0.0008) [2023-12-27 10:25:24,743][20347] Updated weights for policy 0, policy_version 692974 (0.0007) [2023-12-27 10:25:24,804][20347] Updated weights for policy 0, policy_version 692984 (0.0006) [2023-12-27 10:25:24,863][20347] Updated weights for policy 0, policy_version 692994 (0.0005) [2023-12-27 10:25:25,412][20381] Updated weights for policy 1, policy_version 692509 (0.0009) [2023-12-27 10:25:25,460][20381] Updated weights for policy 1, policy_version 692519 (0.0010) [2023-12-27 10:25:25,469][20347] Updated weights for policy 0, policy_version 693004 (0.0005) [2023-12-27 10:25:25,510][20381] Updated weights for policy 1, policy_version 692529 (0.0010) [2023-12-27 10:25:25,532][20347] Updated weights for policy 0, policy_version 693014 (0.0005) [2023-12-27 10:25:25,591][20347] Updated weights for policy 0, policy_version 693024 (0.0005) [2023-12-27 10:25:26,074][20381] Updated weights for policy 1, policy_version 692539 (0.0009) [2023-12-27 10:25:26,127][20381] Updated weights for policy 1, policy_version 692549 (0.0006) [2023-12-27 10:25:26,129][20347] Updated weights for policy 0, policy_version 693034 (0.0005) [2023-12-27 10:25:26,180][20381] Updated weights for policy 1, policy_version 692559 (0.0010) [2023-12-27 10:25:26,181][20347] Updated weights for policy 0, policy_version 693044 (0.0005) [2023-12-27 10:25:26,237][20347] Updated weights for policy 0, policy_version 693054 (0.0006) [2023-12-27 10:25:26,292][20347] Updated weights for policy 0, policy_version 693064 (0.0007) [2023-12-27 10:25:26,760][20381] Updated weights for policy 1, policy_version 692569 (0.0009) [2023-12-27 10:25:26,809][20381] Updated weights for policy 1, policy_version 692579 (0.0007) [2023-12-27 10:25:26,872][20381] Updated weights for policy 1, policy_version 692589 (0.0007) [2023-12-27 10:25:26,935][20381] Updated weights for policy 1, policy_version 692599 (0.0005) [2023-12-27 10:25:27,036][20347] Updated weights for policy 0, policy_version 693074 (0.0008) [2023-12-27 10:25:27,084][20347] Updated weights for policy 0, policy_version 693084 (0.0009) [2023-12-27 10:25:27,132][20347] Updated weights for policy 0, policy_version 693094 (0.0008) [2023-12-27 10:25:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20070.4, 300 sec: 20549.4). Total num frames: 354787328. Throughput: 0: 10253.1, 1: 10263.7. Samples: 354776540. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:27,363][19170] Avg episode reward: [(0, '914.570'), (1, '980.530')] [2023-12-27 10:25:27,488][20381] Updated weights for policy 1, policy_version 692609 (0.0008) [2023-12-27 10:25:27,543][20381] Updated weights for policy 1, policy_version 692619 (0.0007) [2023-12-27 10:25:27,606][20381] Updated weights for policy 1, policy_version 692629 (0.0007) [2023-12-27 10:25:27,881][20347] Updated weights for policy 0, policy_version 693104 (0.0007) [2023-12-27 10:25:27,931][20347] Updated weights for policy 0, policy_version 693114 (0.0008) [2023-12-27 10:25:27,975][20347] Updated weights for policy 0, policy_version 693124 (0.0010) [2023-12-27 10:25:28,235][20381] Updated weights for policy 1, policy_version 692639 (0.0007) [2023-12-27 10:25:28,293][20381] Updated weights for policy 1, policy_version 692649 (0.0005) [2023-12-27 10:25:28,353][20381] Updated weights for policy 1, policy_version 692659 (0.0007) [2023-12-27 10:25:28,677][20347] Updated weights for policy 0, policy_version 693134 (0.0007) [2023-12-27 10:25:28,736][20347] Updated weights for policy 0, policy_version 693144 (0.0009) [2023-12-27 10:25:28,799][20347] Updated weights for policy 0, policy_version 693154 (0.0010) [2023-12-27 10:25:28,956][20381] Updated weights for policy 1, policy_version 692669 (0.0006) [2023-12-27 10:25:29,004][20381] Updated weights for policy 1, policy_version 692679 (0.0007) [2023-12-27 10:25:29,051][20381] Updated weights for policy 1, policy_version 692689 (0.0007) [2023-12-27 10:25:29,436][20347] Updated weights for policy 0, policy_version 693164 (0.0008) [2023-12-27 10:25:29,493][20347] Updated weights for policy 0, policy_version 693174 (0.0008) [2023-12-27 10:25:29,554][20347] Updated weights for policy 0, policy_version 693184 (0.0010) [2023-12-27 10:25:29,700][20381] Updated weights for policy 1, policy_version 692699 (0.0008) [2023-12-27 10:25:29,758][20381] Updated weights for policy 1, policy_version 692709 (0.0009) [2023-12-27 10:25:29,809][20381] Updated weights for policy 1, policy_version 692719 (0.0008) [2023-12-27 10:25:30,208][20347] Updated weights for policy 0, policy_version 693194 (0.0009) [2023-12-27 10:25:30,261][20347] Updated weights for policy 0, policy_version 693204 (0.0008) [2023-12-27 10:25:30,314][20347] Updated weights for policy 0, policy_version 693214 (0.0007) [2023-12-27 10:25:30,369][20347] Updated weights for policy 0, policy_version 693224 (0.0008) [2023-12-27 10:25:30,520][20381] Updated weights for policy 1, policy_version 692729 (0.0009) [2023-12-27 10:25:30,575][20381] Updated weights for policy 1, policy_version 692739 (0.0011) [2023-12-27 10:25:30,624][20381] Updated weights for policy 1, policy_version 692749 (0.0011) [2023-12-27 10:25:30,679][20381] Updated weights for policy 1, policy_version 692759 (0.0006) [2023-12-27 10:25:30,978][20347] Updated weights for policy 0, policy_version 693234 (0.0010) [2023-12-27 10:25:31,036][20347] Updated weights for policy 0, policy_version 693244 (0.0010) [2023-12-27 10:25:31,095][20347] Updated weights for policy 0, policy_version 693254 (0.0007) [2023-12-27 10:25:31,301][20381] Updated weights for policy 1, policy_version 692769 (0.0005) [2023-12-27 10:25:31,358][20381] Updated weights for policy 1, policy_version 692779 (0.0006) [2023-12-27 10:25:31,414][20381] Updated weights for policy 1, policy_version 692789 (0.0008) [2023-12-27 10:25:31,741][20347] Updated weights for policy 0, policy_version 693264 (0.0007) [2023-12-27 10:25:31,804][20347] Updated weights for policy 0, policy_version 693274 (0.0008) [2023-12-27 10:25:31,865][20347] Updated weights for policy 0, policy_version 693284 (0.0010) [2023-12-27 10:25:32,096][20381] Updated weights for policy 1, policy_version 692799 (0.0007) [2023-12-27 10:25:32,142][20381] Updated weights for policy 1, policy_version 692809 (0.0009) [2023-12-27 10:25:32,187][20381] Updated weights for policy 1, policy_version 692819 (0.0008) [2023-12-27 10:25:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 354893824. Throughput: 0: 10339.8, 1: 10401.1. Samples: 354903172. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:32,363][19170] Avg episode reward: [(0, '980.880'), (1, '980.560')] [2023-12-27 10:25:32,450][20347] Updated weights for policy 0, policy_version 693294 (0.0008) [2023-12-27 10:25:32,509][20347] Updated weights for policy 0, policy_version 693304 (0.0010) [2023-12-27 10:25:32,571][20347] Updated weights for policy 0, policy_version 693314 (0.0010) [2023-12-27 10:25:32,916][20381] Updated weights for policy 1, policy_version 692829 (0.0009) [2023-12-27 10:25:32,964][20381] Updated weights for policy 1, policy_version 692839 (0.0010) [2023-12-27 10:25:33,014][20381] Updated weights for policy 1, policy_version 692849 (0.0010) [2023-12-27 10:25:33,145][20347] Updated weights for policy 0, policy_version 693324 (0.0007) [2023-12-27 10:25:33,210][20347] Updated weights for policy 0, policy_version 693334 (0.0005) [2023-12-27 10:25:33,272][20347] Updated weights for policy 0, policy_version 693344 (0.0005) [2023-12-27 10:25:33,598][20381] Updated weights for policy 1, policy_version 692859 (0.0009) [2023-12-27 10:25:33,651][20381] Updated weights for policy 1, policy_version 692869 (0.0010) [2023-12-27 10:25:33,698][20381] Updated weights for policy 1, policy_version 692879 (0.0010) [2023-12-27 10:25:33,796][20347] Updated weights for policy 0, policy_version 693354 (0.0006) [2023-12-27 10:25:33,851][20347] Updated weights for policy 0, policy_version 693364 (0.0006) [2023-12-27 10:25:33,901][20347] Updated weights for policy 0, policy_version 693374 (0.0007) [2023-12-27 10:25:33,956][20347] Updated weights for policy 0, policy_version 693384 (0.0006) [2023-12-27 10:25:34,393][20381] Updated weights for policy 1, policy_version 692889 (0.0010) [2023-12-27 10:25:34,445][20381] Updated weights for policy 1, policy_version 692899 (0.0008) [2023-12-27 10:25:34,501][20381] Updated weights for policy 1, policy_version 692909 (0.0008) [2023-12-27 10:25:34,559][20381] Updated weights for policy 1, policy_version 692919 (0.0008) [2023-12-27 10:25:34,612][20347] Updated weights for policy 0, policy_version 693394 (0.0006) [2023-12-27 10:25:34,663][20347] Updated weights for policy 0, policy_version 693404 (0.0007) [2023-12-27 10:25:34,714][20347] Updated weights for policy 0, policy_version 693414 (0.0007) [2023-12-27 10:25:35,220][20381] Updated weights for policy 1, policy_version 692929 (0.0007) [2023-12-27 10:25:35,272][20381] Updated weights for policy 1, policy_version 692939 (0.0008) [2023-12-27 10:25:35,339][20381] Updated weights for policy 1, policy_version 692949 (0.0006) [2023-12-27 10:25:35,450][20347] Updated weights for policy 0, policy_version 693424 (0.0007) [2023-12-27 10:25:35,508][20347] Updated weights for policy 0, policy_version 693434 (0.0007) [2023-12-27 10:25:35,566][20347] Updated weights for policy 0, policy_version 693444 (0.0007) [2023-12-27 10:25:36,003][20381] Updated weights for policy 1, policy_version 692959 (0.0006) [2023-12-27 10:25:36,067][20381] Updated weights for policy 1, policy_version 692969 (0.0006) [2023-12-27 10:25:36,127][20381] Updated weights for policy 1, policy_version 692979 (0.0007) [2023-12-27 10:25:36,275][20347] Updated weights for policy 0, policy_version 693454 (0.0008) [2023-12-27 10:25:36,335][20347] Updated weights for policy 0, policy_version 693464 (0.0007) [2023-12-27 10:25:36,389][20347] Updated weights for policy 0, policy_version 693474 (0.0007) [2023-12-27 10:25:36,761][20381] Updated weights for policy 1, policy_version 692989 (0.0007) [2023-12-27 10:25:36,821][20381] Updated weights for policy 1, policy_version 692999 (0.0006) [2023-12-27 10:25:36,877][20381] Updated weights for policy 1, policy_version 693009 (0.0008) [2023-12-27 10:25:37,121][20347] Updated weights for policy 0, policy_version 693484 (0.0008) [2023-12-27 10:25:37,182][20347] Updated weights for policy 0, policy_version 693494 (0.0007) [2023-12-27 10:25:37,241][20347] Updated weights for policy 0, policy_version 693504 (0.0008) [2023-12-27 10:25:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 355000320. Throughput: 0: 10460.3, 1: 10447.5. Samples: 354968548. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:37,363][19170] Avg episode reward: [(0, '932.630'), (1, '980.620')] [2023-12-27 10:25:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000693512_177561600.pth... [2023-12-27 10:25:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000693016_177438720.pth... [2023-12-27 10:25:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000691800_177127424.pth [2023-12-27 10:25:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000692296_177250304.pth [2023-12-27 10:25:37,454][20381] Updated weights for policy 1, policy_version 693019 (0.0006) [2023-12-27 10:25:37,512][20381] Updated weights for policy 1, policy_version 693029 (0.0008) [2023-12-27 10:25:37,568][20381] Updated weights for policy 1, policy_version 693039 (0.0007) [2023-12-27 10:25:37,898][20347] Updated weights for policy 0, policy_version 693514 (0.0007) [2023-12-27 10:25:37,963][20347] Updated weights for policy 0, policy_version 693524 (0.0007) [2023-12-27 10:25:38,032][20347] Updated weights for policy 0, policy_version 693534 (0.0008) [2023-12-27 10:25:38,091][20347] Updated weights for policy 0, policy_version 693544 (0.0008) [2023-12-27 10:25:38,209][20381] Updated weights for policy 1, policy_version 693049 (0.0007) [2023-12-27 10:25:38,256][20381] Updated weights for policy 1, policy_version 693059 (0.0006) [2023-12-27 10:25:38,304][20381] Updated weights for policy 1, policy_version 693069 (0.0008) [2023-12-27 10:25:38,364][20381] Updated weights for policy 1, policy_version 693079 (0.0008) [2023-12-27 10:25:38,714][20347] Updated weights for policy 0, policy_version 693554 (0.0005) [2023-12-27 10:25:38,770][20347] Updated weights for policy 0, policy_version 693564 (0.0005) [2023-12-27 10:25:38,828][20347] Updated weights for policy 0, policy_version 693574 (0.0006) [2023-12-27 10:25:39,082][20381] Updated weights for policy 1, policy_version 693089 (0.0006) [2023-12-27 10:25:39,131][20381] Updated weights for policy 1, policy_version 693099 (0.0007) [2023-12-27 10:25:39,184][20381] Updated weights for policy 1, policy_version 693109 (0.0006) [2023-12-27 10:25:39,369][20347] Updated weights for policy 0, policy_version 693584 (0.0008) [2023-12-27 10:25:39,432][20347] Updated weights for policy 0, policy_version 693594 (0.0008) [2023-12-27 10:25:39,493][20347] Updated weights for policy 0, policy_version 693604 (0.0005) [2023-12-27 10:25:39,859][20381] Updated weights for policy 1, policy_version 693119 (0.0007) [2023-12-27 10:25:39,912][20381] Updated weights for policy 1, policy_version 693129 (0.0008) [2023-12-27 10:25:39,979][20381] Updated weights for policy 1, policy_version 693139 (0.0008) [2023-12-27 10:25:39,990][20292] KL-divergence is very high: 148.2148 [2023-12-27 10:25:40,167][20347] Updated weights for policy 0, policy_version 693614 (0.0007) [2023-12-27 10:25:40,227][20347] Updated weights for policy 0, policy_version 693624 (0.0007) [2023-12-27 10:25:40,280][20347] Updated weights for policy 0, policy_version 693634 (0.0007) [2023-12-27 10:25:40,688][20292] KL-divergence is very high: 143.0900 [2023-12-27 10:25:40,718][20381] Updated weights for policy 1, policy_version 693149 (0.0007) [2023-12-27 10:25:40,777][20381] Updated weights for policy 1, policy_version 693159 (0.0005) [2023-12-27 10:25:40,785][20292] KL-divergence is very high: 1445.2219 [2023-12-27 10:25:40,790][20292] KL-divergence is very high: 922.2692 [2023-12-27 10:25:40,828][20292] KL-divergence is very high: 185.9442 [2023-12-27 10:25:40,833][20381] Updated weights for policy 1, policy_version 693169 (0.0007) [2023-12-27 10:25:41,016][20347] Updated weights for policy 0, policy_version 693644 (0.0007) [2023-12-27 10:25:41,077][20347] Updated weights for policy 0, policy_version 693654 (0.0008) [2023-12-27 10:25:41,143][20347] Updated weights for policy 0, policy_version 693664 (0.0009) [2023-12-27 10:25:41,476][20381] Updated weights for policy 1, policy_version 693179 (0.0006) [2023-12-27 10:25:41,546][20381] Updated weights for policy 1, policy_version 693189 (0.0008) [2023-12-27 10:25:41,614][20381] Updated weights for policy 1, policy_version 693199 (0.0009) [2023-12-27 10:25:41,790][20347] Updated weights for policy 0, policy_version 693674 (0.0007) [2023-12-27 10:25:41,853][20347] Updated weights for policy 0, policy_version 693684 (0.0008) [2023-12-27 10:25:41,914][20347] Updated weights for policy 0, policy_version 693694 (0.0008) [2023-12-27 10:25:41,969][20347] Updated weights for policy 0, policy_version 693704 (0.0007) [2023-12-27 10:25:42,294][20381] Updated weights for policy 1, policy_version 693209 (0.0008) [2023-12-27 10:25:42,360][20381] Updated weights for policy 1, policy_version 693219 (0.0007) [2023-12-27 10:25:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 355098624. Throughput: 0: 10472.6, 1: 10459.8. Samples: 355093212. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:42,363][19170] Avg episode reward: [(0, '913.130'), (1, '961.120')] [2023-12-27 10:25:42,406][20381] Updated weights for policy 1, policy_version 693229 (0.0006) [2023-12-27 10:25:42,461][20381] Updated weights for policy 1, policy_version 693239 (0.0006) [2023-12-27 10:25:42,593][20347] Updated weights for policy 0, policy_version 693714 (0.0007) [2023-12-27 10:25:42,650][20347] Updated weights for policy 0, policy_version 693724 (0.0005) [2023-12-27 10:25:42,713][20347] Updated weights for policy 0, policy_version 693734 (0.0006) [2023-12-27 10:25:43,052][20381] Updated weights for policy 1, policy_version 693249 (0.0007) [2023-12-27 10:25:43,103][20381] Updated weights for policy 1, policy_version 693259 (0.0007) [2023-12-27 10:25:43,158][20381] Updated weights for policy 1, policy_version 693269 (0.0007) [2023-12-27 10:25:43,320][20347] Updated weights for policy 0, policy_version 693744 (0.0008) [2023-12-27 10:25:43,366][20347] Updated weights for policy 0, policy_version 693754 (0.0006) [2023-12-27 10:25:43,417][20347] Updated weights for policy 0, policy_version 693764 (0.0005) [2023-12-27 10:25:43,921][20381] Updated weights for policy 1, policy_version 693279 (0.0008) [2023-12-27 10:25:43,954][20347] Updated weights for policy 0, policy_version 693774 (0.0008) [2023-12-27 10:25:43,970][20381] Updated weights for policy 1, policy_version 693289 (0.0007) [2023-12-27 10:25:44,006][20347] Updated weights for policy 0, policy_version 693784 (0.0010) [2023-12-27 10:25:44,026][20381] Updated weights for policy 1, policy_version 693299 (0.0005) [2023-12-27 10:25:44,068][20347] Updated weights for policy 0, policy_version 693794 (0.0010) [2023-12-27 10:25:44,696][20347] Updated weights for policy 0, policy_version 693804 (0.0009) [2023-12-27 10:25:44,736][20381] Updated weights for policy 1, policy_version 693309 (0.0006) [2023-12-27 10:25:44,746][20347] Updated weights for policy 0, policy_version 693814 (0.0008) [2023-12-27 10:25:44,804][20381] Updated weights for policy 1, policy_version 693319 (0.0008) [2023-12-27 10:25:44,808][20347] Updated weights for policy 0, policy_version 693824 (0.0008) [2023-12-27 10:25:44,867][20381] Updated weights for policy 1, policy_version 693329 (0.0010) [2023-12-27 10:25:45,398][20347] Updated weights for policy 0, policy_version 693834 (0.0006) [2023-12-27 10:25:45,451][20347] Updated weights for policy 0, policy_version 693844 (0.0005) [2023-12-27 10:25:45,514][20347] Updated weights for policy 0, policy_version 693854 (0.0005) [2023-12-27 10:25:45,545][20381] Updated weights for policy 1, policy_version 693339 (0.0010) [2023-12-27 10:25:45,565][20347] Updated weights for policy 0, policy_version 693864 (0.0005) [2023-12-27 10:25:45,606][20381] Updated weights for policy 1, policy_version 693349 (0.0010) [2023-12-27 10:25:45,667][20381] Updated weights for policy 1, policy_version 693359 (0.0011) [2023-12-27 10:25:46,115][20347] Updated weights for policy 0, policy_version 693874 (0.0006) [2023-12-27 10:25:46,165][20347] Updated weights for policy 0, policy_version 693884 (0.0005) [2023-12-27 10:25:46,212][20347] Updated weights for policy 0, policy_version 693894 (0.0006) [2023-12-27 10:25:46,381][20381] Updated weights for policy 1, policy_version 693369 (0.0011) [2023-12-27 10:25:46,428][20381] Updated weights for policy 1, policy_version 693379 (0.0010) [2023-12-27 10:25:46,483][20381] Updated weights for policy 1, policy_version 693389 (0.0010) [2023-12-27 10:25:46,539][20381] Updated weights for policy 1, policy_version 693399 (0.0010) [2023-12-27 10:25:46,812][20347] Updated weights for policy 0, policy_version 693904 (0.0005) [2023-12-27 10:25:46,860][20347] Updated weights for policy 0, policy_version 693914 (0.0005) [2023-12-27 10:25:46,913][20347] Updated weights for policy 0, policy_version 693924 (0.0005) [2023-12-27 10:25:47,117][20381] Updated weights for policy 1, policy_version 693409 (0.0006) [2023-12-27 10:25:47,172][20381] Updated weights for policy 1, policy_version 693419 (0.0007) [2023-12-27 10:25:47,226][20381] Updated weights for policy 1, policy_version 693429 (0.0008) [2023-12-27 10:25:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 21026.1, 300 sec: 20605.0). Total num frames: 355213312. Throughput: 0: 10651.3, 1: 10429.7. Samples: 355222812. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:47,363][19170] Avg episode reward: [(0, '941.560'), (1, '961.120')] [2023-12-27 10:25:47,487][20347] Updated weights for policy 0, policy_version 693934 (0.0006) [2023-12-27 10:25:47,532][20347] Updated weights for policy 0, policy_version 693944 (0.0007) [2023-12-27 10:25:47,584][20347] Updated weights for policy 0, policy_version 693954 (0.0008) [2023-12-27 10:25:47,930][20381] Updated weights for policy 1, policy_version 693439 (0.0007) [2023-12-27 10:25:47,989][20381] Updated weights for policy 1, policy_version 693449 (0.0010) [2023-12-27 10:25:48,045][20381] Updated weights for policy 1, policy_version 693459 (0.0011) [2023-12-27 10:25:48,238][20347] Updated weights for policy 0, policy_version 693964 (0.0008) [2023-12-27 10:25:48,292][20347] Updated weights for policy 0, policy_version 693974 (0.0007) [2023-12-27 10:25:48,356][20347] Updated weights for policy 0, policy_version 693984 (0.0007) [2023-12-27 10:25:48,657][20381] Updated weights for policy 1, policy_version 693469 (0.0008) [2023-12-27 10:25:48,709][20381] Updated weights for policy 1, policy_version 693479 (0.0005) [2023-12-27 10:25:48,768][20381] Updated weights for policy 1, policy_version 693489 (0.0005) [2023-12-27 10:25:49,028][20347] Updated weights for policy 0, policy_version 693994 (0.0007) [2023-12-27 10:25:49,092][20347] Updated weights for policy 0, policy_version 694004 (0.0008) [2023-12-27 10:25:49,152][20347] Updated weights for policy 0, policy_version 694014 (0.0008) [2023-12-27 10:25:49,211][20347] Updated weights for policy 0, policy_version 694024 (0.0008) [2023-12-27 10:25:49,462][20381] Updated weights for policy 1, policy_version 693499 (0.0008) [2023-12-27 10:25:49,517][20381] Updated weights for policy 1, policy_version 693509 (0.0008) [2023-12-27 10:25:49,569][20381] Updated weights for policy 1, policy_version 693519 (0.0008) [2023-12-27 10:25:49,892][20347] Updated weights for policy 0, policy_version 694034 (0.0007) [2023-12-27 10:25:49,964][20347] Updated weights for policy 0, policy_version 694044 (0.0007) [2023-12-27 10:25:50,031][20347] Updated weights for policy 0, policy_version 694054 (0.0005) [2023-12-27 10:25:50,256][20381] Updated weights for policy 1, policy_version 693529 (0.0008) [2023-12-27 10:25:50,303][20381] Updated weights for policy 1, policy_version 693539 (0.0008) [2023-12-27 10:25:50,353][20381] Updated weights for policy 1, policy_version 693549 (0.0007) [2023-12-27 10:25:50,407][20381] Updated weights for policy 1, policy_version 693559 (0.0008) [2023-12-27 10:25:50,655][20347] Updated weights for policy 0, policy_version 694064 (0.0007) [2023-12-27 10:25:50,707][20347] Updated weights for policy 0, policy_version 694074 (0.0008) [2023-12-27 10:25:50,765][20347] Updated weights for policy 0, policy_version 694084 (0.0007) [2023-12-27 10:25:51,100][20381] Updated weights for policy 1, policy_version 693569 (0.0007) [2023-12-27 10:25:51,154][20381] Updated weights for policy 1, policy_version 693579 (0.0007) [2023-12-27 10:25:51,208][20381] Updated weights for policy 1, policy_version 693589 (0.0007) [2023-12-27 10:25:51,515][20347] Updated weights for policy 0, policy_version 694094 (0.0006) [2023-12-27 10:25:51,579][20347] Updated weights for policy 0, policy_version 694104 (0.0006) [2023-12-27 10:25:51,643][20347] Updated weights for policy 0, policy_version 694114 (0.0006) [2023-12-27 10:25:51,875][20381] Updated weights for policy 1, policy_version 693599 (0.0007) [2023-12-27 10:25:51,935][20381] Updated weights for policy 1, policy_version 693609 (0.0006) [2023-12-27 10:25:51,996][20381] Updated weights for policy 1, policy_version 693619 (0.0007) [2023-12-27 10:25:52,288][20347] Updated weights for policy 0, policy_version 694124 (0.0007) [2023-12-27 10:25:52,343][20347] Updated weights for policy 0, policy_version 694134 (0.0008) [2023-12-27 10:25:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.5, 300 sec: 20577.2). Total num frames: 355311616. Throughput: 0: 10599.5, 1: 10433.1. Samples: 355284972. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:52,363][19170] Avg episode reward: [(0, '922.600'), (1, '951.470')] [2023-12-27 10:25:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000693624_177594368.pth... [2023-12-27 10:25:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000692408_177283072.pth [2023-12-27 10:25:52,410][20347] Updated weights for policy 0, policy_version 694144 (0.0007) [2023-12-27 10:25:52,455][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000694152_177725440.pth... [2023-12-27 10:25:52,459][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000692872_177397760.pth [2023-12-27 10:25:52,674][20381] Updated weights for policy 1, policy_version 693629 (0.0008) [2023-12-27 10:25:52,737][20381] Updated weights for policy 1, policy_version 693639 (0.0005) [2023-12-27 10:25:52,798][20381] Updated weights for policy 1, policy_version 693649 (0.0008) [2023-12-27 10:25:53,102][20347] Updated weights for policy 0, policy_version 694154 (0.0008) [2023-12-27 10:25:53,153][20347] Updated weights for policy 0, policy_version 694164 (0.0008) [2023-12-27 10:25:53,210][20347] Updated weights for policy 0, policy_version 694174 (0.0007) [2023-12-27 10:25:53,265][20347] Updated weights for policy 0, policy_version 694184 (0.0008) [2023-12-27 10:25:53,410][20381] Updated weights for policy 1, policy_version 693659 (0.0007) [2023-12-27 10:25:53,461][20381] Updated weights for policy 1, policy_version 693669 (0.0008) [2023-12-27 10:25:53,519][20381] Updated weights for policy 1, policy_version 693679 (0.0007) [2023-12-27 10:25:53,878][20347] Updated weights for policy 0, policy_version 694194 (0.0005) [2023-12-27 10:25:53,923][20347] Updated weights for policy 0, policy_version 694204 (0.0006) [2023-12-27 10:25:53,967][20347] Updated weights for policy 0, policy_version 694214 (0.0005) [2023-12-27 10:25:54,232][20381] Updated weights for policy 1, policy_version 693689 (0.0007) [2023-12-27 10:25:54,286][20381] Updated weights for policy 1, policy_version 693699 (0.0007) [2023-12-27 10:25:54,340][20381] Updated weights for policy 1, policy_version 693709 (0.0007) [2023-12-27 10:25:54,390][20381] Updated weights for policy 1, policy_version 693719 (0.0005) [2023-12-27 10:25:54,612][20347] Updated weights for policy 0, policy_version 694224 (0.0010) [2023-12-27 10:25:54,658][20347] Updated weights for policy 0, policy_version 694234 (0.0010) [2023-12-27 10:25:54,703][20347] Updated weights for policy 0, policy_version 694244 (0.0010) [2023-12-27 10:25:55,037][20381] Updated weights for policy 1, policy_version 693729 (0.0007) [2023-12-27 10:25:55,081][20381] Updated weights for policy 1, policy_version 693739 (0.0008) [2023-12-27 10:25:55,144][20381] Updated weights for policy 1, policy_version 693749 (0.0007) [2023-12-27 10:25:55,417][20347] Updated weights for policy 0, policy_version 694254 (0.0010) [2023-12-27 10:25:55,471][20347] Updated weights for policy 0, policy_version 694264 (0.0010) [2023-12-27 10:25:55,525][20347] Updated weights for policy 0, policy_version 694274 (0.0010) [2023-12-27 10:25:55,817][20381] Updated weights for policy 1, policy_version 693759 (0.0006) [2023-12-27 10:25:55,869][20381] Updated weights for policy 1, policy_version 693769 (0.0005) [2023-12-27 10:25:55,920][20381] Updated weights for policy 1, policy_version 693779 (0.0009) [2023-12-27 10:25:56,196][20347] Updated weights for policy 0, policy_version 694284 (0.0010) [2023-12-27 10:25:56,247][20347] Updated weights for policy 0, policy_version 694294 (0.0010) [2023-12-27 10:25:56,299][20347] Updated weights for policy 0, policy_version 694304 (0.0010) [2023-12-27 10:25:56,542][20381] Updated weights for policy 1, policy_version 693789 (0.0009) [2023-12-27 10:25:56,588][20381] Updated weights for policy 1, policy_version 693799 (0.0007) [2023-12-27 10:25:56,639][20381] Updated weights for policy 1, policy_version 693809 (0.0007) [2023-12-27 10:25:56,981][20347] Updated weights for policy 0, policy_version 694314 (0.0010) [2023-12-27 10:25:57,025][20347] Updated weights for policy 0, policy_version 694324 (0.0010) [2023-12-27 10:25:57,082][20347] Updated weights for policy 0, policy_version 694334 (0.0010) [2023-12-27 10:25:57,136][20347] Updated weights for policy 0, policy_version 694344 (0.0010) [2023-12-27 10:25:57,324][20381] Updated weights for policy 1, policy_version 693819 (0.0007) [2023-12-27 10:25:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 21026.2, 300 sec: 20577.2). Total num frames: 355418112. Throughput: 0: 10593.1, 1: 10428.7. Samples: 355408728. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:25:57,363][19170] Avg episode reward: [(0, '894.910'), (1, '941.750')] [2023-12-27 10:25:57,375][20381] Updated weights for policy 1, policy_version 693829 (0.0005) [2023-12-27 10:25:57,430][20381] Updated weights for policy 1, policy_version 693839 (0.0005) [2023-12-27 10:25:57,765][20347] Updated weights for policy 0, policy_version 694354 (0.0010) [2023-12-27 10:25:57,820][20347] Updated weights for policy 0, policy_version 694364 (0.0010) [2023-12-27 10:25:57,878][20347] Updated weights for policy 0, policy_version 694374 (0.0010) [2023-12-27 10:25:58,000][20381] Updated weights for policy 1, policy_version 693849 (0.0005) [2023-12-27 10:25:58,058][20381] Updated weights for policy 1, policy_version 693859 (0.0007) [2023-12-27 10:25:58,117][20381] Updated weights for policy 1, policy_version 693869 (0.0007) [2023-12-27 10:25:58,177][20381] Updated weights for policy 1, policy_version 693879 (0.0007) [2023-12-27 10:25:58,545][20347] Updated weights for policy 0, policy_version 694384 (0.0008) [2023-12-27 10:25:58,604][20347] Updated weights for policy 0, policy_version 694394 (0.0008) [2023-12-27 10:25:58,666][20347] Updated weights for policy 0, policy_version 694404 (0.0007) [2023-12-27 10:25:58,991][20381] Updated weights for policy 1, policy_version 693889 (0.0009) [2023-12-27 10:25:59,050][20381] Updated weights for policy 1, policy_version 693899 (0.0011) [2023-12-27 10:25:59,105][20381] Updated weights for policy 1, policy_version 693909 (0.0011) [2023-12-27 10:25:59,513][20347] Updated weights for policy 0, policy_version 694414 (0.0007) [2023-12-27 10:25:59,557][20347] Updated weights for policy 0, policy_version 694424 (0.0007) [2023-12-27 10:25:59,613][20347] Updated weights for policy 0, policy_version 694434 (0.0007) [2023-12-27 10:25:59,826][20381] Updated weights for policy 1, policy_version 693919 (0.0010) [2023-12-27 10:25:59,877][20381] Updated weights for policy 1, policy_version 693929 (0.0011) [2023-12-27 10:25:59,935][20381] Updated weights for policy 1, policy_version 693939 (0.0011) [2023-12-27 10:26:00,335][20347] Updated weights for policy 0, policy_version 694444 (0.0007) [2023-12-27 10:26:00,392][20347] Updated weights for policy 0, policy_version 694454 (0.0008) [2023-12-27 10:26:00,448][20347] Updated weights for policy 0, policy_version 694464 (0.0008) [2023-12-27 10:26:00,604][20381] Updated weights for policy 1, policy_version 693949 (0.0009) [2023-12-27 10:26:00,657][20381] Updated weights for policy 1, policy_version 693959 (0.0008) [2023-12-27 10:26:00,715][20381] Updated weights for policy 1, policy_version 693969 (0.0006) [2023-12-27 10:26:01,147][20347] Updated weights for policy 0, policy_version 694474 (0.0008) [2023-12-27 10:26:01,196][20347] Updated weights for policy 0, policy_version 694484 (0.0008) [2023-12-27 10:26:01,249][20347] Updated weights for policy 0, policy_version 694494 (0.0007) [2023-12-27 10:26:01,311][20347] Updated weights for policy 0, policy_version 694504 (0.0008) [2023-12-27 10:26:01,403][20381] Updated weights for policy 1, policy_version 693979 (0.0006) [2023-12-27 10:26:01,452][20381] Updated weights for policy 1, policy_version 693989 (0.0005) [2023-12-27 10:26:01,511][20381] Updated weights for policy 1, policy_version 693999 (0.0006) [2023-12-27 10:26:02,014][20347] Updated weights for policy 0, policy_version 694514 (0.0007) [2023-12-27 10:26:02,062][20347] Updated weights for policy 0, policy_version 694524 (0.0007) [2023-12-27 10:26:02,116][20347] Updated weights for policy 0, policy_version 694534 (0.0007) [2023-12-27 10:26:02,203][20381] Updated weights for policy 1, policy_version 694009 (0.0008) [2023-12-27 10:26:02,263][20381] Updated weights for policy 1, policy_version 694019 (0.0010) [2023-12-27 10:26:02,318][20381] Updated weights for policy 1, policy_version 694029 (0.0010) [2023-12-27 10:26:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 355516416. Throughput: 0: 10490.2, 1: 10407.4. Samples: 355530264. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:02,363][19170] Avg episode reward: [(0, '864.900'), (1, '970.790')] [2023-12-27 10:26:02,368][20381] Updated weights for policy 1, policy_version 694039 (0.0010) [2023-12-27 10:26:02,846][20347] Updated weights for policy 0, policy_version 694544 (0.0007) [2023-12-27 10:26:02,893][20347] Updated weights for policy 0, policy_version 694554 (0.0006) [2023-12-27 10:26:02,941][20347] Updated weights for policy 0, policy_version 694564 (0.0007) [2023-12-27 10:26:03,043][20381] Updated weights for policy 1, policy_version 694049 (0.0010) [2023-12-27 10:26:03,091][20381] Updated weights for policy 1, policy_version 694059 (0.0010) [2023-12-27 10:26:03,141][20381] Updated weights for policy 1, policy_version 694069 (0.0005) [2023-12-27 10:26:03,615][20347] Updated weights for policy 0, policy_version 694574 (0.0008) [2023-12-27 10:26:03,677][20347] Updated weights for policy 0, policy_version 694584 (0.0007) [2023-12-27 10:26:03,736][20381] Updated weights for policy 1, policy_version 694079 (0.0006) [2023-12-27 10:26:03,741][20347] Updated weights for policy 0, policy_version 694594 (0.0008) [2023-12-27 10:26:03,791][20381] Updated weights for policy 1, policy_version 694089 (0.0006) [2023-12-27 10:26:03,849][20381] Updated weights for policy 1, policy_version 694099 (0.0006) [2023-12-27 10:26:04,453][20347] Updated weights for policy 0, policy_version 694604 (0.0007) [2023-12-27 10:26:04,504][20347] Updated weights for policy 0, policy_version 694614 (0.0008) [2023-12-27 10:26:04,526][20381] Updated weights for policy 1, policy_version 694109 (0.0007) [2023-12-27 10:26:04,552][20347] Updated weights for policy 0, policy_version 694624 (0.0006) [2023-12-27 10:26:04,585][20381] Updated weights for policy 1, policy_version 694119 (0.0007) [2023-12-27 10:26:04,644][20381] Updated weights for policy 1, policy_version 694129 (0.0007) [2023-12-27 10:26:05,259][20381] Updated weights for policy 1, policy_version 694139 (0.0007) [2023-12-27 10:26:05,307][20347] Updated weights for policy 0, policy_version 694634 (0.0006) [2023-12-27 10:26:05,319][20381] Updated weights for policy 1, policy_version 694149 (0.0007) [2023-12-27 10:26:05,355][20347] Updated weights for policy 0, policy_version 694644 (0.0008) [2023-12-27 10:26:05,368][20381] Updated weights for policy 1, policy_version 694159 (0.0005) [2023-12-27 10:26:05,401][20347] Updated weights for policy 0, policy_version 694654 (0.0007) [2023-12-27 10:26:05,448][20347] Updated weights for policy 0, policy_version 694664 (0.0008) [2023-12-27 10:26:05,899][20381] Updated weights for policy 1, policy_version 694169 (0.0005) [2023-12-27 10:26:05,947][20381] Updated weights for policy 1, policy_version 694179 (0.0006) [2023-12-27 10:26:05,997][20381] Updated weights for policy 1, policy_version 694189 (0.0005) [2023-12-27 10:26:06,045][20381] Updated weights for policy 1, policy_version 694199 (0.0005) [2023-12-27 10:26:06,214][20347] Updated weights for policy 0, policy_version 694674 (0.0005) [2023-12-27 10:26:06,282][20347] Updated weights for policy 0, policy_version 694684 (0.0007) [2023-12-27 10:26:06,345][20347] Updated weights for policy 0, policy_version 694694 (0.0007) [2023-12-27 10:26:06,737][20381] Updated weights for policy 1, policy_version 694209 (0.0010) [2023-12-27 10:26:06,799][20381] Updated weights for policy 1, policy_version 694219 (0.0010) [2023-12-27 10:26:06,858][20381] Updated weights for policy 1, policy_version 694229 (0.0010) [2023-12-27 10:26:07,026][20347] Updated weights for policy 0, policy_version 694704 (0.0007) [2023-12-27 10:26:07,038][20291] KL-divergence is very high: 402.5170 [2023-12-27 10:26:07,076][20347] Updated weights for policy 0, policy_version 694714 (0.0007) [2023-12-27 10:26:07,077][20291] KL-divergence is very high: 199.2235 [2023-12-27 10:26:07,127][20347] Updated weights for policy 0, policy_version 694724 (0.0009) [2023-12-27 10:26:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 355622912. Throughput: 0: 10463.7, 1: 10460.9. Samples: 355592004. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:07,364][19170] Avg episode reward: [(0, '855.540'), (1, '970.840')] [2023-12-27 10:26:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000694232_177750016.pth... [2023-12-27 10:26:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000694728_177872896.pth... [2023-12-27 10:26:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000693016_177438720.pth [2023-12-27 10:26:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000693512_177561600.pth [2023-12-27 10:26:07,483][20381] Updated weights for policy 1, policy_version 694239 (0.0011) [2023-12-27 10:26:07,535][20381] Updated weights for policy 1, policy_version 694249 (0.0010) [2023-12-27 10:26:07,583][20381] Updated weights for policy 1, policy_version 694259 (0.0010) [2023-12-27 10:26:07,826][20291] KL-divergence is very high: 118.4883 [2023-12-27 10:26:07,856][20347] Updated weights for policy 0, policy_version 694734 (0.0007) [2023-12-27 10:26:07,918][20347] Updated weights for policy 0, policy_version 694744 (0.0008) [2023-12-27 10:26:07,979][20347] Updated weights for policy 0, policy_version 694754 (0.0006) [2023-12-27 10:26:08,276][20381] Updated weights for policy 1, policy_version 694269 (0.0010) [2023-12-27 10:26:08,327][20381] Updated weights for policy 1, policy_version 694279 (0.0010) [2023-12-27 10:26:08,384][20381] Updated weights for policy 1, policy_version 694289 (0.0011) [2023-12-27 10:26:08,598][20347] Updated weights for policy 0, policy_version 694764 (0.0006) [2023-12-27 10:26:08,656][20347] Updated weights for policy 0, policy_version 694774 (0.0007) [2023-12-27 10:26:08,712][20347] Updated weights for policy 0, policy_version 694784 (0.0007) [2023-12-27 10:26:09,054][20381] Updated weights for policy 1, policy_version 694299 (0.0010) [2023-12-27 10:26:09,091][20292] KL-divergence is very high: 315.0563 [2023-12-27 10:26:09,100][20381] Updated weights for policy 1, policy_version 694309 (0.0008) [2023-12-27 10:26:09,130][20292] KL-divergence is very high: 351.7908 [2023-12-27 10:26:09,154][20381] Updated weights for policy 1, policy_version 694319 (0.0006) [2023-12-27 10:26:09,422][20347] Updated weights for policy 0, policy_version 694794 (0.0007) [2023-12-27 10:26:09,475][20347] Updated weights for policy 0, policy_version 694804 (0.0008) [2023-12-27 10:26:09,530][20347] Updated weights for policy 0, policy_version 694814 (0.0007) [2023-12-27 10:26:09,585][20347] Updated weights for policy 0, policy_version 694824 (0.0007) [2023-12-27 10:26:09,866][20381] Updated weights for policy 1, policy_version 694329 (0.0007) [2023-12-27 10:26:09,929][20381] Updated weights for policy 1, policy_version 694339 (0.0007) [2023-12-27 10:26:09,997][20381] Updated weights for policy 1, policy_version 694349 (0.0008) [2023-12-27 10:26:10,005][20292] KL-divergence is very high: 145.0073 [2023-12-27 10:26:10,039][20292] KL-divergence is very high: 337.9504 [2023-12-27 10:26:10,057][20292] KL-divergence is very high: 248.7185 [2023-12-27 10:26:10,064][20381] Updated weights for policy 1, policy_version 694359 (0.0008) [2023-12-27 10:26:10,327][20347] Updated weights for policy 0, policy_version 694834 (0.0007) [2023-12-27 10:26:10,387][20347] Updated weights for policy 0, policy_version 694844 (0.0006) [2023-12-27 10:26:10,448][20347] Updated weights for policy 0, policy_version 694854 (0.0006) [2023-12-27 10:26:10,651][20292] KL-divergence is very high: 246.9430 [2023-12-27 10:26:10,695][20381] Updated weights for policy 1, policy_version 694369 (0.0008) [2023-12-27 10:26:10,751][20381] Updated weights for policy 1, policy_version 694379 (0.0008) [2023-12-27 10:26:10,811][20381] Updated weights for policy 1, policy_version 694389 (0.0008) [2023-12-27 10:26:11,131][20347] Updated weights for policy 0, policy_version 694864 (0.0009) [2023-12-27 10:26:11,198][20347] Updated weights for policy 0, policy_version 694874 (0.0006) [2023-12-27 10:26:11,256][20347] Updated weights for policy 0, policy_version 694884 (0.0007) [2023-12-27 10:26:11,492][20381] Updated weights for policy 1, policy_version 694399 (0.0007) [2023-12-27 10:26:11,555][20381] Updated weights for policy 1, policy_version 694409 (0.0008) [2023-12-27 10:26:11,621][20381] Updated weights for policy 1, policy_version 694419 (0.0008) [2023-12-27 10:26:11,894][20347] Updated weights for policy 0, policy_version 694894 (0.0007) [2023-12-27 10:26:11,940][20347] Updated weights for policy 0, policy_version 694904 (0.0007) [2023-12-27 10:26:11,986][20347] Updated weights for policy 0, policy_version 694914 (0.0008) [2023-12-27 10:26:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 355721216. Throughput: 0: 10391.9, 1: 10455.0. Samples: 355714648. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:12,363][19170] Avg episode reward: [(0, '846.430'), (1, '951.360')] [2023-12-27 10:26:12,379][20381] Updated weights for policy 1, policy_version 694429 (0.0008) [2023-12-27 10:26:12,437][20381] Updated weights for policy 1, policy_version 694439 (0.0008) [2023-12-27 10:26:12,500][20381] Updated weights for policy 1, policy_version 694449 (0.0007) [2023-12-27 10:26:12,741][20347] Updated weights for policy 0, policy_version 694924 (0.0007) [2023-12-27 10:26:12,803][20347] Updated weights for policy 0, policy_version 694934 (0.0008) [2023-12-27 10:26:12,859][20347] Updated weights for policy 0, policy_version 694944 (0.0007) [2023-12-27 10:26:13,097][20381] Updated weights for policy 1, policy_version 694459 (0.0006) [2023-12-27 10:26:13,181][20381] Updated weights for policy 1, policy_version 694469 (0.0007) [2023-12-27 10:26:13,243][20381] Updated weights for policy 1, policy_version 694479 (0.0008) [2023-12-27 10:26:13,490][20347] Updated weights for policy 0, policy_version 694954 (0.0007) [2023-12-27 10:26:13,546][20347] Updated weights for policy 0, policy_version 694964 (0.0007) [2023-12-27 10:26:13,595][20347] Updated weights for policy 0, policy_version 694974 (0.0010) [2023-12-27 10:26:13,649][20347] Updated weights for policy 0, policy_version 694984 (0.0009) [2023-12-27 10:26:13,865][20381] Updated weights for policy 1, policy_version 694489 (0.0009) [2023-12-27 10:26:13,924][20381] Updated weights for policy 1, policy_version 694499 (0.0008) [2023-12-27 10:26:13,976][20381] Updated weights for policy 1, policy_version 694509 (0.0007) [2023-12-27 10:26:14,022][20381] Updated weights for policy 1, policy_version 694519 (0.0008) [2023-12-27 10:26:14,347][20347] Updated weights for policy 0, policy_version 694994 (0.0007) [2023-12-27 10:26:14,395][20347] Updated weights for policy 0, policy_version 695004 (0.0008) [2023-12-27 10:26:14,458][20347] Updated weights for policy 0, policy_version 695014 (0.0008) [2023-12-27 10:26:14,728][20381] Updated weights for policy 1, policy_version 694529 (0.0008) [2023-12-27 10:26:14,787][20381] Updated weights for policy 1, policy_version 694539 (0.0006) [2023-12-27 10:26:14,848][20381] Updated weights for policy 1, policy_version 694549 (0.0008) [2023-12-27 10:26:15,183][20347] Updated weights for policy 0, policy_version 695024 (0.0008) [2023-12-27 10:26:15,233][20347] Updated weights for policy 0, policy_version 695034 (0.0007) [2023-12-27 10:26:15,286][20347] Updated weights for policy 0, policy_version 695044 (0.0008) [2023-12-27 10:26:15,549][20381] Updated weights for policy 1, policy_version 694559 (0.0007) [2023-12-27 10:26:15,608][20381] Updated weights for policy 1, policy_version 694569 (0.0008) [2023-12-27 10:26:15,664][20381] Updated weights for policy 1, policy_version 694579 (0.0007) [2023-12-27 10:26:15,946][20347] Updated weights for policy 0, policy_version 695054 (0.0007) [2023-12-27 10:26:15,951][20291] KL-divergence is very high: 160.6362 [2023-12-27 10:26:15,994][20291] KL-divergence is very high: 103.4618 [2023-12-27 10:26:16,000][20347] Updated weights for policy 0, policy_version 695064 (0.0008) [2023-12-27 10:26:16,037][20291] KL-divergence is very high: 149.3898 [2023-12-27 10:26:16,055][20347] Updated weights for policy 0, policy_version 695074 (0.0008) [2023-12-27 10:26:16,079][20291] KL-divergence is very high: 104.6987 [2023-12-27 10:26:16,361][20381] Updated weights for policy 1, policy_version 694589 (0.0007) [2023-12-27 10:26:16,406][20381] Updated weights for policy 1, policy_version 694599 (0.0008) [2023-12-27 10:26:16,459][20381] Updated weights for policy 1, policy_version 694609 (0.0008) [2023-12-27 10:26:16,725][20347] Updated weights for policy 0, policy_version 695084 (0.0008) [2023-12-27 10:26:16,782][20347] Updated weights for policy 0, policy_version 695094 (0.0007) [2023-12-27 10:26:16,830][20347] Updated weights for policy 0, policy_version 695104 (0.0007) [2023-12-27 10:26:17,178][20381] Updated weights for policy 1, policy_version 694619 (0.0007) [2023-12-27 10:26:17,228][20381] Updated weights for policy 1, policy_version 694629 (0.0007) [2023-12-27 10:26:17,273][20381] Updated weights for policy 1, policy_version 694639 (0.0007) [2023-12-27 10:26:17,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.5, 300 sec: 20549.4). Total num frames: 355827712. Throughput: 0: 10357.9, 1: 10358.7. Samples: 355835424. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:17,364][19170] Avg episode reward: [(0, '817.200'), (1, '961.900')] [2023-12-27 10:26:17,521][20347] Updated weights for policy 0, policy_version 695114 (0.0008) [2023-12-27 10:26:17,570][20347] Updated weights for policy 0, policy_version 695124 (0.0008) [2023-12-27 10:26:17,628][20347] Updated weights for policy 0, policy_version 695134 (0.0007) [2023-12-27 10:26:17,681][20347] Updated weights for policy 0, policy_version 695144 (0.0008) [2023-12-27 10:26:17,976][20381] Updated weights for policy 1, policy_version 694649 (0.0007) [2023-12-27 10:26:18,030][20381] Updated weights for policy 1, policy_version 694659 (0.0007) [2023-12-27 10:26:18,077][20381] Updated weights for policy 1, policy_version 694669 (0.0005) [2023-12-27 10:26:18,130][20381] Updated weights for policy 1, policy_version 694679 (0.0006) [2023-12-27 10:26:18,405][20347] Updated weights for policy 0, policy_version 695154 (0.0006) [2023-12-27 10:26:18,463][20347] Updated weights for policy 0, policy_version 695164 (0.0005) [2023-12-27 10:26:18,523][20347] Updated weights for policy 0, policy_version 695174 (0.0007) [2023-12-27 10:26:18,802][20381] Updated weights for policy 1, policy_version 694689 (0.0010) [2023-12-27 10:26:18,858][20381] Updated weights for policy 1, policy_version 694699 (0.0010) [2023-12-27 10:26:18,917][20381] Updated weights for policy 1, policy_version 694709 (0.0011) [2023-12-27 10:26:19,179][20347] Updated weights for policy 0, policy_version 695184 (0.0010) [2023-12-27 10:26:19,237][20347] Updated weights for policy 0, policy_version 695194 (0.0010) [2023-12-27 10:26:19,300][20347] Updated weights for policy 0, policy_version 695204 (0.0010) [2023-12-27 10:26:19,588][20381] Updated weights for policy 1, policy_version 694719 (0.0006) [2023-12-27 10:26:19,649][20381] Updated weights for policy 1, policy_version 694729 (0.0007) [2023-12-27 10:26:19,702][20381] Updated weights for policy 1, policy_version 694739 (0.0008) [2023-12-27 10:26:20,065][20347] Updated weights for policy 0, policy_version 695214 (0.0009) [2023-12-27 10:26:20,117][20347] Updated weights for policy 0, policy_version 695224 (0.0006) [2023-12-27 10:26:20,176][20347] Updated weights for policy 0, policy_version 695234 (0.0008) [2023-12-27 10:26:20,352][20381] Updated weights for policy 1, policy_version 694749 (0.0008) [2023-12-27 10:26:20,409][20381] Updated weights for policy 1, policy_version 694759 (0.0007) [2023-12-27 10:26:20,466][20381] Updated weights for policy 1, policy_version 694769 (0.0008) [2023-12-27 10:26:20,841][20347] Updated weights for policy 0, policy_version 695244 (0.0007) [2023-12-27 10:26:20,907][20347] Updated weights for policy 0, policy_version 695254 (0.0007) [2023-12-27 10:26:20,976][20347] Updated weights for policy 0, policy_version 695264 (0.0008) [2023-12-27 10:26:21,184][20381] Updated weights for policy 1, policy_version 694779 (0.0008) [2023-12-27 10:26:21,238][20381] Updated weights for policy 1, policy_version 694789 (0.0008) [2023-12-27 10:26:21,312][20381] Updated weights for policy 1, policy_version 694799 (0.0009) [2023-12-27 10:26:21,658][20347] Updated weights for policy 0, policy_version 695274 (0.0008) [2023-12-27 10:26:21,726][20347] Updated weights for policy 0, policy_version 695284 (0.0007) [2023-12-27 10:26:21,783][20347] Updated weights for policy 0, policy_version 695294 (0.0008) [2023-12-27 10:26:21,838][20347] Updated weights for policy 0, policy_version 695304 (0.0007) [2023-12-27 10:26:22,046][20381] Updated weights for policy 1, policy_version 694809 (0.0009) [2023-12-27 10:26:22,097][20381] Updated weights for policy 1, policy_version 694819 (0.0011) [2023-12-27 10:26:22,147][20381] Updated weights for policy 1, policy_version 694829 (0.0010) [2023-12-27 10:26:22,197][20381] Updated weights for policy 1, policy_version 694839 (0.0011) [2023-12-27 10:26:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 355926016. Throughput: 0: 10278.7, 1: 10345.4. Samples: 355896632. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:22,363][19170] Avg episode reward: [(0, '788.650'), (1, '991.110')] [2023-12-27 10:26:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000694840_177905664.pth... [2023-12-27 10:26:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000695304_178020352.pth... [2023-12-27 10:26:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000694152_177725440.pth [2023-12-27 10:26:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000693624_177594368.pth [2023-12-27 10:26:22,499][20347] Updated weights for policy 0, policy_version 695314 (0.0006) [2023-12-27 10:26:22,554][20347] Updated weights for policy 0, policy_version 695324 (0.0005) [2023-12-27 10:26:22,602][20347] Updated weights for policy 0, policy_version 695334 (0.0006) [2023-12-27 10:26:22,942][20381] Updated weights for policy 1, policy_version 694849 (0.0011) [2023-12-27 10:26:23,003][20381] Updated weights for policy 1, policy_version 694859 (0.0011) [2023-12-27 10:26:23,060][20381] Updated weights for policy 1, policy_version 694869 (0.0011) [2023-12-27 10:26:23,251][20347] Updated weights for policy 0, policy_version 695344 (0.0008) [2023-12-27 10:26:23,307][20347] Updated weights for policy 0, policy_version 695354 (0.0006) [2023-12-27 10:26:23,363][20347] Updated weights for policy 0, policy_version 695364 (0.0007) [2023-12-27 10:26:23,741][20381] Updated weights for policy 1, policy_version 694879 (0.0010) [2023-12-27 10:26:23,801][20381] Updated weights for policy 1, policy_version 694889 (0.0008) [2023-12-27 10:26:23,852][20381] Updated weights for policy 1, policy_version 694899 (0.0010) [2023-12-27 10:26:23,984][20347] Updated weights for policy 0, policy_version 695374 (0.0007) [2023-12-27 10:26:24,038][20347] Updated weights for policy 0, policy_version 695384 (0.0007) [2023-12-27 10:26:24,099][20347] Updated weights for policy 0, policy_version 695394 (0.0007) [2023-12-27 10:26:24,537][20381] Updated weights for policy 1, policy_version 694909 (0.0010) [2023-12-27 10:26:24,589][20381] Updated weights for policy 1, policy_version 694919 (0.0010) [2023-12-27 10:26:24,654][20381] Updated weights for policy 1, policy_version 694929 (0.0011) [2023-12-27 10:26:24,767][20347] Updated weights for policy 0, policy_version 695404 (0.0006) [2023-12-27 10:26:24,812][20347] Updated weights for policy 0, policy_version 695414 (0.0005) [2023-12-27 10:26:24,862][20347] Updated weights for policy 0, policy_version 695424 (0.0005) [2023-12-27 10:26:25,344][20381] Updated weights for policy 1, policy_version 694939 (0.0010) [2023-12-27 10:26:25,399][20381] Updated weights for policy 1, policy_version 694949 (0.0010) [2023-12-27 10:26:25,448][20347] Updated weights for policy 0, policy_version 695434 (0.0005) [2023-12-27 10:26:25,450][20381] Updated weights for policy 1, policy_version 694959 (0.0010) [2023-12-27 10:26:25,503][20347] Updated weights for policy 0, policy_version 695444 (0.0006) [2023-12-27 10:26:25,552][20347] Updated weights for policy 0, policy_version 695454 (0.0007) [2023-12-27 10:26:25,603][20347] Updated weights for policy 0, policy_version 695464 (0.0007) [2023-12-27 10:26:26,064][20381] Updated weights for policy 1, policy_version 694969 (0.0010) [2023-12-27 10:26:26,113][20381] Updated weights for policy 1, policy_version 694979 (0.0008) [2023-12-27 10:26:26,164][20381] Updated weights for policy 1, policy_version 694989 (0.0008) [2023-12-27 10:26:26,208][20381] Updated weights for policy 1, policy_version 694999 (0.0007) [2023-12-27 10:26:26,304][20347] Updated weights for policy 0, policy_version 695474 (0.0007) [2023-12-27 10:26:26,350][20347] Updated weights for policy 0, policy_version 695484 (0.0007) [2023-12-27 10:26:26,412][20347] Updated weights for policy 0, policy_version 695494 (0.0005) [2023-12-27 10:26:26,833][20381] Updated weights for policy 1, policy_version 695009 (0.0006) [2023-12-27 10:26:26,885][20381] Updated weights for policy 1, policy_version 695019 (0.0005) [2023-12-27 10:26:26,932][20381] Updated weights for policy 1, policy_version 695029 (0.0005) [2023-12-27 10:26:27,083][20347] Updated weights for policy 0, policy_version 695504 (0.0007) [2023-12-27 10:26:27,127][20347] Updated weights for policy 0, policy_version 695514 (0.0008) [2023-12-27 10:26:27,172][20347] Updated weights for policy 0, policy_version 695524 (0.0007) [2023-12-27 10:26:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 356032512. Throughput: 0: 10314.4, 1: 10261.7. Samples: 356019136. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:27,363][19170] Avg episode reward: [(0, '827.130'), (1, '961.320')] [2023-12-27 10:26:27,504][20381] Updated weights for policy 1, policy_version 695039 (0.0005) [2023-12-27 10:26:27,561][20381] Updated weights for policy 1, policy_version 695049 (0.0005) [2023-12-27 10:26:27,609][20381] Updated weights for policy 1, policy_version 695059 (0.0005) [2023-12-27 10:26:27,853][20347] Updated weights for policy 0, policy_version 695534 (0.0007) [2023-12-27 10:26:27,915][20347] Updated weights for policy 0, policy_version 695544 (0.0006) [2023-12-27 10:26:27,977][20347] Updated weights for policy 0, policy_version 695554 (0.0006) [2023-12-27 10:26:28,256][20381] Updated weights for policy 1, policy_version 695069 (0.0006) [2023-12-27 10:26:28,311][20381] Updated weights for policy 1, policy_version 695079 (0.0007) [2023-12-27 10:26:28,374][20381] Updated weights for policy 1, policy_version 695089 (0.0007) [2023-12-27 10:26:28,588][20347] Updated weights for policy 0, policy_version 695564 (0.0009) [2023-12-27 10:26:28,644][20347] Updated weights for policy 0, policy_version 695574 (0.0011) [2023-12-27 10:26:28,699][20347] Updated weights for policy 0, policy_version 695584 (0.0010) [2023-12-27 10:26:29,067][20381] Updated weights for policy 1, policy_version 695099 (0.0007) [2023-12-27 10:26:29,120][20381] Updated weights for policy 1, policy_version 695109 (0.0005) [2023-12-27 10:26:29,176][20381] Updated weights for policy 1, policy_version 695119 (0.0008) [2023-12-27 10:26:29,204][20292] KL-divergence is very high: 205.7057 [2023-12-27 10:26:29,217][20292] KL-divergence is very high: 137.8350 [2023-12-27 10:26:29,393][20347] Updated weights for policy 0, policy_version 695594 (0.0010) [2023-12-27 10:26:29,442][20347] Updated weights for policy 0, policy_version 695604 (0.0009) [2023-12-27 10:26:29,494][20347] Updated weights for policy 0, policy_version 695614 (0.0007) [2023-12-27 10:26:29,550][20347] Updated weights for policy 0, policy_version 695624 (0.0007) [2023-12-27 10:26:29,741][20381] Updated weights for policy 1, policy_version 695129 (0.0008) [2023-12-27 10:26:29,799][20381] Updated weights for policy 1, policy_version 695139 (0.0007) [2023-12-27 10:26:29,852][20381] Updated weights for policy 1, policy_version 695149 (0.0008) [2023-12-27 10:26:29,906][20381] Updated weights for policy 1, policy_version 695159 (0.0005) [2023-12-27 10:26:30,280][20347] Updated weights for policy 0, policy_version 695634 (0.0008) [2023-12-27 10:26:30,340][20347] Updated weights for policy 0, policy_version 695644 (0.0008) [2023-12-27 10:26:30,400][20347] Updated weights for policy 0, policy_version 695654 (0.0008) [2023-12-27 10:26:30,521][20381] Updated weights for policy 1, policy_version 695169 (0.0007) [2023-12-27 10:26:30,576][20381] Updated weights for policy 1, policy_version 695179 (0.0008) [2023-12-27 10:26:30,632][20381] Updated weights for policy 1, policy_version 695189 (0.0008) [2023-12-27 10:26:31,052][20347] Updated weights for policy 0, policy_version 695664 (0.0008) [2023-12-27 10:26:31,105][20347] Updated weights for policy 0, policy_version 695674 (0.0008) [2023-12-27 10:26:31,166][20347] Updated weights for policy 0, policy_version 695684 (0.0007) [2023-12-27 10:26:31,258][20381] Updated weights for policy 1, policy_version 695199 (0.0008) [2023-12-27 10:26:31,320][20381] Updated weights for policy 1, policy_version 695209 (0.0008) [2023-12-27 10:26:31,383][20381] Updated weights for policy 1, policy_version 695219 (0.0008) [2023-12-27 10:26:31,875][20347] Updated weights for policy 0, policy_version 695694 (0.0008) [2023-12-27 10:26:31,941][20347] Updated weights for policy 0, policy_version 695704 (0.0007) [2023-12-27 10:26:32,000][20347] Updated weights for policy 0, policy_version 695714 (0.0008) [2023-12-27 10:26:32,145][20381] Updated weights for policy 1, policy_version 695229 (0.0009) [2023-12-27 10:26:32,201][20381] Updated weights for policy 1, policy_version 695239 (0.0008) [2023-12-27 10:26:32,261][20381] Updated weights for policy 1, policy_version 695249 (0.0008) [2023-12-27 10:26:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 356139008. Throughput: 0: 10154.2, 1: 10366.3. Samples: 356146228. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:32,363][19170] Avg episode reward: [(0, '912.720'), (1, '933.430')] [2023-12-27 10:26:32,706][20347] Updated weights for policy 0, policy_version 695724 (0.0008) [2023-12-27 10:26:32,758][20347] Updated weights for policy 0, policy_version 695734 (0.0008) [2023-12-27 10:26:32,806][20347] Updated weights for policy 0, policy_version 695744 (0.0008) [2023-12-27 10:26:32,952][20381] Updated weights for policy 1, policy_version 695259 (0.0007) [2023-12-27 10:26:33,005][20381] Updated weights for policy 1, policy_version 695269 (0.0010) [2023-12-27 10:26:33,054][20381] Updated weights for policy 1, policy_version 695279 (0.0009) [2023-12-27 10:26:33,419][20347] Updated weights for policy 0, policy_version 695754 (0.0007) [2023-12-27 10:26:33,474][20347] Updated weights for policy 0, policy_version 695764 (0.0005) [2023-12-27 10:26:33,530][20347] Updated weights for policy 0, policy_version 695774 (0.0007) [2023-12-27 10:26:33,559][20291] KL-divergence is very high: 232.6483 [2023-12-27 10:26:33,564][20291] KL-divergence is very high: 230.7881 [2023-12-27 10:26:33,578][20347] Updated weights for policy 0, policy_version 695784 (0.0007) [2023-12-27 10:26:33,793][20381] Updated weights for policy 1, policy_version 695289 (0.0008) [2023-12-27 10:26:33,848][20381] Updated weights for policy 1, policy_version 695299 (0.0009) [2023-12-27 10:26:33,894][20381] Updated weights for policy 1, policy_version 695309 (0.0010) [2023-12-27 10:26:33,939][20381] Updated weights for policy 1, policy_version 695319 (0.0008) [2023-12-27 10:26:34,149][20347] Updated weights for policy 0, policy_version 695794 (0.0006) [2023-12-27 10:26:34,204][20347] Updated weights for policy 0, policy_version 695804 (0.0007) [2023-12-27 10:26:34,261][20347] Updated weights for policy 0, policy_version 695814 (0.0005) [2023-12-27 10:26:34,663][20381] Updated weights for policy 1, policy_version 695329 (0.0008) [2023-12-27 10:26:34,724][20381] Updated weights for policy 1, policy_version 695339 (0.0007) [2023-12-27 10:26:34,779][20381] Updated weights for policy 1, policy_version 695349 (0.0006) [2023-12-27 10:26:34,838][20347] Updated weights for policy 0, policy_version 695824 (0.0007) [2023-12-27 10:26:34,890][20347] Updated weights for policy 0, policy_version 695834 (0.0008) [2023-12-27 10:26:34,948][20347] Updated weights for policy 0, policy_version 695844 (0.0008) [2023-12-27 10:26:35,408][20381] Updated weights for policy 1, policy_version 695359 (0.0008) [2023-12-27 10:26:35,453][20381] Updated weights for policy 1, policy_version 695369 (0.0008) [2023-12-27 10:26:35,499][20381] Updated weights for policy 1, policy_version 695379 (0.0008) [2023-12-27 10:26:35,656][20347] Updated weights for policy 0, policy_version 695854 (0.0007) [2023-12-27 10:26:35,710][20347] Updated weights for policy 0, policy_version 695864 (0.0007) [2023-12-27 10:26:35,766][20347] Updated weights for policy 0, policy_version 695874 (0.0007) [2023-12-27 10:26:36,255][20381] Updated weights for policy 1, policy_version 695389 (0.0008) [2023-12-27 10:26:36,314][20381] Updated weights for policy 1, policy_version 695399 (0.0008) [2023-12-27 10:26:36,352][20292] KL-divergence is very high: 451.1615 [2023-12-27 10:26:36,371][20381] Updated weights for policy 1, policy_version 695409 (0.0008) [2023-12-27 10:26:36,409][20347] Updated weights for policy 0, policy_version 695884 (0.0008) [2023-12-27 10:26:36,471][20347] Updated weights for policy 0, policy_version 695894 (0.0008) [2023-12-27 10:26:36,529][20347] Updated weights for policy 0, policy_version 695904 (0.0006) [2023-12-27 10:26:37,066][20381] Updated weights for policy 1, policy_version 695419 (0.0008) [2023-12-27 10:26:37,086][20292] KL-divergence is very high: 446.3497 [2023-12-27 10:26:37,122][20381] Updated weights for policy 1, policy_version 695429 (0.0008) [2023-12-27 10:26:37,131][20292] KL-divergence is very high: 211.6141 [2023-12-27 10:26:37,174][20381] Updated weights for policy 1, policy_version 695439 (0.0007) [2023-12-27 10:26:37,174][20292] KL-divergence is very high: 179.9027 [2023-12-27 10:26:37,176][20347] Updated weights for policy 0, policy_version 695914 (0.0007) [2023-12-27 10:26:37,216][20292] KL-divergence is very high: 232.6219 [2023-12-27 10:26:37,229][20347] Updated weights for policy 0, policy_version 695924 (0.0007) [2023-12-27 10:26:37,294][20347] Updated weights for policy 0, policy_version 695934 (0.0007) [2023-12-27 10:26:37,348][20347] Updated weights for policy 0, policy_version 695944 (0.0008) [2023-12-27 10:26:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 356245504. Throughput: 0: 10194.5, 1: 10348.4. Samples: 356209400. Policy #0 lag: (min: 24.0, avg: 48.1, max: 56.0) [2023-12-27 10:26:37,363][19170] Avg episode reward: [(0, '902.940'), (1, '935.260')] [2023-12-27 10:26:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000695448_178061312.pth... [2023-12-27 10:26:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000695944_178184192.pth... [2023-12-27 10:26:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000694728_177872896.pth [2023-12-27 10:26:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000694232_177750016.pth [2023-12-27 10:26:37,853][20381] Updated weights for policy 1, policy_version 695449 (0.0006) [2023-12-27 10:26:37,912][20381] Updated weights for policy 1, policy_version 695459 (0.0008) [2023-12-27 10:26:37,964][20381] Updated weights for policy 1, policy_version 695469 (0.0008) [2023-12-27 10:26:38,019][20381] Updated weights for policy 1, policy_version 695479 (0.0007) [2023-12-27 10:26:38,048][20347] Updated weights for policy 0, policy_version 695954 (0.0006) [2023-12-27 10:26:38,099][20347] Updated weights for policy 0, policy_version 695964 (0.0005) [2023-12-27 10:26:38,152][20347] Updated weights for policy 0, policy_version 695974 (0.0005) [2023-12-27 10:26:38,739][20381] Updated weights for policy 1, policy_version 695489 (0.0005) [2023-12-27 10:26:38,799][20381] Updated weights for policy 1, policy_version 695499 (0.0006) [2023-12-27 10:26:38,813][20347] Updated weights for policy 0, policy_version 695984 (0.0007) [2023-12-27 10:26:38,862][20381] Updated weights for policy 1, policy_version 695509 (0.0006) [2023-12-27 10:26:38,871][20347] Updated weights for policy 0, policy_version 695994 (0.0007) [2023-12-27 10:26:38,929][20347] Updated weights for policy 0, policy_version 696004 (0.0007) [2023-12-27 10:26:39,457][20381] Updated weights for policy 1, policy_version 695519 (0.0007) [2023-12-27 10:26:39,520][20381] Updated weights for policy 1, policy_version 695529 (0.0005) [2023-12-27 10:26:39,578][20381] Updated weights for policy 1, policy_version 695539 (0.0008) [2023-12-27 10:26:39,636][20347] Updated weights for policy 0, policy_version 696014 (0.0009) [2023-12-27 10:26:39,698][20347] Updated weights for policy 0, policy_version 696024 (0.0008) [2023-12-27 10:26:39,761][20347] Updated weights for policy 0, policy_version 696034 (0.0008) [2023-12-27 10:26:40,180][20381] Updated weights for policy 1, policy_version 695549 (0.0008) [2023-12-27 10:26:40,249][20381] Updated weights for policy 1, policy_version 695559 (0.0006) [2023-12-27 10:26:40,307][20381] Updated weights for policy 1, policy_version 695569 (0.0008) [2023-12-27 10:26:40,519][20347] Updated weights for policy 0, policy_version 696044 (0.0007) [2023-12-27 10:26:40,582][20347] Updated weights for policy 0, policy_version 696054 (0.0008) [2023-12-27 10:26:40,647][20347] Updated weights for policy 0, policy_version 696064 (0.0008) [2023-12-27 10:26:40,916][20381] Updated weights for policy 1, policy_version 695579 (0.0006) [2023-12-27 10:26:40,971][20381] Updated weights for policy 1, policy_version 695589 (0.0005) [2023-12-27 10:26:41,024][20381] Updated weights for policy 1, policy_version 695599 (0.0007) [2023-12-27 10:26:41,381][20347] Updated weights for policy 0, policy_version 696074 (0.0008) [2023-12-27 10:26:41,438][20347] Updated weights for policy 0, policy_version 696084 (0.0007) [2023-12-27 10:26:41,489][20347] Updated weights for policy 0, policy_version 696094 (0.0008) [2023-12-27 10:26:41,543][20347] Updated weights for policy 0, policy_version 696104 (0.0008) [2023-12-27 10:26:41,679][20381] Updated weights for policy 1, policy_version 695609 (0.0006) [2023-12-27 10:26:41,743][20381] Updated weights for policy 1, policy_version 695619 (0.0007) [2023-12-27 10:26:41,810][20381] Updated weights for policy 1, policy_version 695629 (0.0008) [2023-12-27 10:26:41,864][20381] Updated weights for policy 1, policy_version 695639 (0.0008) [2023-12-27 10:26:42,247][20347] Updated weights for policy 0, policy_version 696114 (0.0007) [2023-12-27 10:26:42,312][20347] Updated weights for policy 0, policy_version 696124 (0.0008) [2023-12-27 10:26:42,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 356335616. Throughput: 0: 10170.3, 1: 10352.8. Samples: 356332268. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:26:42,363][19170] Avg episode reward: [(0, '893.070'), (1, '916.240')] [2023-12-27 10:26:42,373][20347] Updated weights for policy 0, policy_version 696134 (0.0011) [2023-12-27 10:26:42,568][20381] Updated weights for policy 1, policy_version 695649 (0.0010) [2023-12-27 10:26:42,617][20381] Updated weights for policy 1, policy_version 695659 (0.0011) [2023-12-27 10:26:42,680][20381] Updated weights for policy 1, policy_version 695669 (0.0011) [2023-12-27 10:26:43,051][20347] Updated weights for policy 0, policy_version 696144 (0.0011) [2023-12-27 10:26:43,112][20347] Updated weights for policy 0, policy_version 696154 (0.0011) [2023-12-27 10:26:43,173][20347] Updated weights for policy 0, policy_version 696164 (0.0010) [2023-12-27 10:26:43,380][20381] Updated weights for policy 1, policy_version 695679 (0.0011) [2023-12-27 10:26:43,436][20381] Updated weights for policy 1, policy_version 695689 (0.0011) [2023-12-27 10:26:43,483][20381] Updated weights for policy 1, policy_version 695699 (0.0011) [2023-12-27 10:26:43,838][20347] Updated weights for policy 0, policy_version 696174 (0.0010) [2023-12-27 10:26:43,896][20347] Updated weights for policy 0, policy_version 696184 (0.0010) [2023-12-27 10:26:43,947][20347] Updated weights for policy 0, policy_version 696194 (0.0010) [2023-12-27 10:26:44,153][20381] Updated weights for policy 1, policy_version 695709 (0.0010) [2023-12-27 10:26:44,201][20381] Updated weights for policy 1, policy_version 695719 (0.0010) [2023-12-27 10:26:44,254][20381] Updated weights for policy 1, policy_version 695729 (0.0010) [2023-12-27 10:26:44,637][20347] Updated weights for policy 0, policy_version 696204 (0.0006) [2023-12-27 10:26:44,694][20347] Updated weights for policy 0, policy_version 696214 (0.0006) [2023-12-27 10:26:44,764][20347] Updated weights for policy 0, policy_version 696224 (0.0007) [2023-12-27 10:26:44,868][20381] Updated weights for policy 1, policy_version 695739 (0.0009) [2023-12-27 10:26:44,921][20381] Updated weights for policy 1, policy_version 695749 (0.0006) [2023-12-27 10:26:44,978][20381] Updated weights for policy 1, policy_version 695759 (0.0008) [2023-12-27 10:26:45,408][20347] Updated weights for policy 0, policy_version 696234 (0.0010) [2023-12-27 10:26:45,460][20347] Updated weights for policy 0, policy_version 696244 (0.0010) [2023-12-27 10:26:45,512][20347] Updated weights for policy 0, policy_version 696254 (0.0010) [2023-12-27 10:26:45,569][20347] Updated weights for policy 0, policy_version 696264 (0.0006) [2023-12-27 10:26:45,654][20381] Updated weights for policy 1, policy_version 695769 (0.0007) [2023-12-27 10:26:45,708][20381] Updated weights for policy 1, policy_version 695779 (0.0008) [2023-12-27 10:26:45,764][20381] Updated weights for policy 1, policy_version 695789 (0.0008) [2023-12-27 10:26:45,818][20381] Updated weights for policy 1, policy_version 695799 (0.0009) [2023-12-27 10:26:46,247][20347] Updated weights for policy 0, policy_version 696274 (0.0007) [2023-12-27 10:26:46,291][20347] Updated weights for policy 0, policy_version 696284 (0.0007) [2023-12-27 10:26:46,337][20347] Updated weights for policy 0, policy_version 696294 (0.0007) [2023-12-27 10:26:46,445][20381] Updated weights for policy 1, policy_version 695809 (0.0005) [2023-12-27 10:26:46,493][20381] Updated weights for policy 1, policy_version 695819 (0.0006) [2023-12-27 10:26:46,549][20381] Updated weights for policy 1, policy_version 695829 (0.0008) [2023-12-27 10:26:47,004][20347] Updated weights for policy 0, policy_version 696304 (0.0010) [2023-12-27 10:26:47,066][20347] Updated weights for policy 0, policy_version 696314 (0.0010) [2023-12-27 10:26:47,121][20347] Updated weights for policy 0, policy_version 696324 (0.0010) [2023-12-27 10:26:47,158][20381] Updated weights for policy 1, policy_version 695839 (0.0006) [2023-12-27 10:26:47,213][20381] Updated weights for policy 1, policy_version 695849 (0.0009) [2023-12-27 10:26:47,258][20381] Updated weights for policy 1, policy_version 695859 (0.0009) [2023-12-27 10:26:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 356450304. Throughput: 0: 10195.6, 1: 10395.8. Samples: 356456876. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:26:47,363][19170] Avg episode reward: [(0, '961.040'), (1, '962.990')] [2023-12-27 10:26:47,726][20347] Updated weights for policy 0, policy_version 696335 (0.0010) [2023-12-27 10:26:47,774][20347] Updated weights for policy 0, policy_version 696345 (0.0010) [2023-12-27 10:26:47,818][20347] Updated weights for policy 0, policy_version 696355 (0.0010) [2023-12-27 10:26:47,931][20381] Updated weights for policy 1, policy_version 695869 (0.0008) [2023-12-27 10:26:47,986][20381] Updated weights for policy 1, policy_version 695879 (0.0008) [2023-12-27 10:26:48,042][20381] Updated weights for policy 1, policy_version 695889 (0.0008) [2023-12-27 10:26:48,407][20347] Updated weights for policy 0, policy_version 696365 (0.0010) [2023-12-27 10:26:48,467][20347] Updated weights for policy 0, policy_version 696375 (0.0007) [2023-12-27 10:26:48,523][20347] Updated weights for policy 0, policy_version 696385 (0.0008) [2023-12-27 10:26:48,674][20381] Updated weights for policy 1, policy_version 695899 (0.0007) [2023-12-27 10:26:48,735][20381] Updated weights for policy 1, policy_version 695909 (0.0010) [2023-12-27 10:26:48,796][20381] Updated weights for policy 1, policy_version 695919 (0.0010) [2023-12-27 10:26:49,213][20347] Updated weights for policy 0, policy_version 696395 (0.0007) [2023-12-27 10:26:49,276][20347] Updated weights for policy 0, policy_version 696405 (0.0006) [2023-12-27 10:26:49,344][20347] Updated weights for policy 0, policy_version 696415 (0.0007) [2023-12-27 10:26:49,501][20381] Updated weights for policy 1, policy_version 695929 (0.0010) [2023-12-27 10:26:49,560][20381] Updated weights for policy 1, policy_version 695939 (0.0008) [2023-12-27 10:26:49,616][20381] Updated weights for policy 1, policy_version 695949 (0.0006) [2023-12-27 10:26:49,676][20381] Updated weights for policy 1, policy_version 695959 (0.0006) [2023-12-27 10:26:50,016][20347] Updated weights for policy 0, policy_version 696425 (0.0008) [2023-12-27 10:26:50,079][20347] Updated weights for policy 0, policy_version 696435 (0.0010) [2023-12-27 10:26:50,140][20347] Updated weights for policy 0, policy_version 696445 (0.0011) [2023-12-27 10:26:50,193][20347] Updated weights for policy 0, policy_version 696455 (0.0010) [2023-12-27 10:26:50,402][20381] Updated weights for policy 1, policy_version 695969 (0.0008) [2023-12-27 10:26:50,454][20381] Updated weights for policy 1, policy_version 695979 (0.0007) [2023-12-27 10:26:50,500][20381] Updated weights for policy 1, policy_version 695989 (0.0007) [2023-12-27 10:26:50,880][20347] Updated weights for policy 0, policy_version 696465 (0.0010) [2023-12-27 10:26:50,932][20347] Updated weights for policy 0, policy_version 696475 (0.0010) [2023-12-27 10:26:50,990][20347] Updated weights for policy 0, policy_version 696485 (0.0010) [2023-12-27 10:26:51,232][20381] Updated weights for policy 1, policy_version 695999 (0.0008) [2023-12-27 10:26:51,289][20381] Updated weights for policy 1, policy_version 696009 (0.0007) [2023-12-27 10:26:51,359][20381] Updated weights for policy 1, policy_version 696019 (0.0008) [2023-12-27 10:26:51,666][20347] Updated weights for policy 0, policy_version 696495 (0.0009) [2023-12-27 10:26:51,724][20347] Updated weights for policy 0, policy_version 696505 (0.0007) [2023-12-27 10:26:51,779][20347] Updated weights for policy 0, policy_version 696515 (0.0006) [2023-12-27 10:26:52,108][20381] Updated weights for policy 1, policy_version 696029 (0.0008) [2023-12-27 10:26:52,161][20381] Updated weights for policy 1, policy_version 696039 (0.0007) [2023-12-27 10:26:52,219][20381] Updated weights for policy 1, policy_version 696049 (0.0008) [2023-12-27 10:26:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 356548608. Throughput: 0: 10256.4, 1: 10353.9. Samples: 356519464. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:26:52,363][19170] Avg episode reward: [(0, '932.260'), (1, '981.440')] [2023-12-27 10:26:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000696056_178216960.pth... [2023-12-27 10:26:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000696520_178331648.pth... [2023-12-27 10:26:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000695304_178020352.pth [2023-12-27 10:26:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000694840_177905664.pth [2023-12-27 10:26:52,475][20347] Updated weights for policy 0, policy_version 696525 (0.0005) [2023-12-27 10:26:52,537][20347] Updated weights for policy 0, policy_version 696535 (0.0007) [2023-12-27 10:26:52,598][20347] Updated weights for policy 0, policy_version 696545 (0.0008) [2023-12-27 10:26:52,906][20381] Updated weights for policy 1, policy_version 696059 (0.0008) [2023-12-27 10:26:52,965][20381] Updated weights for policy 1, policy_version 696069 (0.0006) [2023-12-27 10:26:53,024][20381] Updated weights for policy 1, policy_version 696079 (0.0006) [2023-12-27 10:26:53,193][20347] Updated weights for policy 0, policy_version 696555 (0.0007) [2023-12-27 10:26:53,242][20347] Updated weights for policy 0, policy_version 696565 (0.0005) [2023-12-27 10:26:53,295][20347] Updated weights for policy 0, policy_version 696575 (0.0007) [2023-12-27 10:26:53,636][20381] Updated weights for policy 1, policy_version 696089 (0.0009) [2023-12-27 10:26:53,685][20381] Updated weights for policy 1, policy_version 696099 (0.0008) [2023-12-27 10:26:53,734][20381] Updated weights for policy 1, policy_version 696109 (0.0008) [2023-12-27 10:26:53,780][20381] Updated weights for policy 1, policy_version 696119 (0.0007) [2023-12-27 10:26:53,940][20347] Updated weights for policy 0, policy_version 696585 (0.0006) [2023-12-27 10:26:54,005][20347] Updated weights for policy 0, policy_version 696595 (0.0008) [2023-12-27 10:26:54,059][20347] Updated weights for policy 0, policy_version 696605 (0.0007) [2023-12-27 10:26:54,116][20347] Updated weights for policy 0, policy_version 696615 (0.0007) [2023-12-27 10:26:54,510][20381] Updated weights for policy 1, policy_version 696129 (0.0010) [2023-12-27 10:26:54,559][20381] Updated weights for policy 1, policy_version 696139 (0.0010) [2023-12-27 10:26:54,606][20381] Updated weights for policy 1, policy_version 696149 (0.0010) [2023-12-27 10:26:54,776][20347] Updated weights for policy 0, policy_version 696625 (0.0006) [2023-12-27 10:26:54,827][20347] Updated weights for policy 0, policy_version 696635 (0.0010) [2023-12-27 10:26:54,881][20347] Updated weights for policy 0, policy_version 696645 (0.0010) [2023-12-27 10:26:55,281][20381] Updated weights for policy 1, policy_version 696159 (0.0008) [2023-12-27 10:26:55,329][20381] Updated weights for policy 1, policy_version 696169 (0.0008) [2023-12-27 10:26:55,379][20381] Updated weights for policy 1, policy_version 696179 (0.0006) [2023-12-27 10:26:55,518][20347] Updated weights for policy 0, policy_version 696655 (0.0007) [2023-12-27 10:26:55,572][20347] Updated weights for policy 0, policy_version 696665 (0.0005) [2023-12-27 10:26:55,632][20347] Updated weights for policy 0, policy_version 696675 (0.0009) [2023-12-27 10:26:55,961][20381] Updated weights for policy 1, policy_version 696189 (0.0006) [2023-12-27 10:26:56,009][20381] Updated weights for policy 1, policy_version 696199 (0.0005) [2023-12-27 10:26:56,061][20381] Updated weights for policy 1, policy_version 696209 (0.0011) [2023-12-27 10:26:56,278][20347] Updated weights for policy 0, policy_version 696685 (0.0009) [2023-12-27 10:26:56,344][20347] Updated weights for policy 0, policy_version 696695 (0.0010) [2023-12-27 10:26:56,398][20347] Updated weights for policy 0, policy_version 696705 (0.0010) [2023-12-27 10:26:56,743][20381] Updated weights for policy 1, policy_version 696219 (0.0011) [2023-12-27 10:26:56,795][20381] Updated weights for policy 1, policy_version 696229 (0.0010) [2023-12-27 10:26:56,842][20381] Updated weights for policy 1, policy_version 696239 (0.0010) [2023-12-27 10:26:57,042][20347] Updated weights for policy 0, policy_version 696715 (0.0010) [2023-12-27 10:26:57,092][20347] Updated weights for policy 0, policy_version 696725 (0.0010) [2023-12-27 10:26:57,145][20347] Updated weights for policy 0, policy_version 696735 (0.0010) [2023-12-27 10:26:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 356655104. Throughput: 0: 10333.2, 1: 10318.5. Samples: 356643972. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:26:57,363][19170] Avg episode reward: [(0, '941.970'), (1, '980.570')] [2023-12-27 10:26:57,531][20381] Updated weights for policy 1, policy_version 696249 (0.0010) [2023-12-27 10:26:57,581][20381] Updated weights for policy 1, policy_version 696259 (0.0008) [2023-12-27 10:26:57,630][20381] Updated weights for policy 1, policy_version 696269 (0.0008) [2023-12-27 10:26:57,675][20381] Updated weights for policy 1, policy_version 696279 (0.0005) [2023-12-27 10:26:57,812][20347] Updated weights for policy 0, policy_version 696745 (0.0010) [2023-12-27 10:26:57,873][20347] Updated weights for policy 0, policy_version 696755 (0.0005) [2023-12-27 10:26:57,923][20347] Updated weights for policy 0, policy_version 696765 (0.0005) [2023-12-27 10:26:57,975][20347] Updated weights for policy 0, policy_version 696775 (0.0005) [2023-12-27 10:26:58,341][20381] Updated weights for policy 1, policy_version 696289 (0.0009) [2023-12-27 10:26:58,400][20381] Updated weights for policy 1, policy_version 696299 (0.0008) [2023-12-27 10:26:58,469][20381] Updated weights for policy 1, policy_version 696309 (0.0008) [2023-12-27 10:26:58,607][20347] Updated weights for policy 0, policy_version 696785 (0.0008) [2023-12-27 10:26:58,669][20347] Updated weights for policy 0, policy_version 696795 (0.0007) [2023-12-27 10:26:58,729][20347] Updated weights for policy 0, policy_version 696805 (0.0008) [2023-12-27 10:26:59,162][20381] Updated weights for policy 1, policy_version 696319 (0.0008) [2023-12-27 10:26:59,214][20381] Updated weights for policy 1, policy_version 696329 (0.0009) [2023-12-27 10:26:59,278][20381] Updated weights for policy 1, policy_version 696339 (0.0008) [2023-12-27 10:26:59,446][20347] Updated weights for policy 0, policy_version 696815 (0.0007) [2023-12-27 10:26:59,501][20347] Updated weights for policy 0, policy_version 696825 (0.0006) [2023-12-27 10:26:59,560][20347] Updated weights for policy 0, policy_version 696835 (0.0008) [2023-12-27 10:27:00,020][20381] Updated weights for policy 1, policy_version 696349 (0.0009) [2023-12-27 10:27:00,075][20381] Updated weights for policy 1, policy_version 696359 (0.0010) [2023-12-27 10:27:00,124][20381] Updated weights for policy 1, policy_version 696369 (0.0010) [2023-12-27 10:27:00,213][20347] Updated weights for policy 0, policy_version 696845 (0.0006) [2023-12-27 10:27:00,275][20347] Updated weights for policy 0, policy_version 696855 (0.0006) [2023-12-27 10:27:00,337][20347] Updated weights for policy 0, policy_version 696865 (0.0005) [2023-12-27 10:27:00,824][20381] Updated weights for policy 1, policy_version 696379 (0.0009) [2023-12-27 10:27:00,882][20381] Updated weights for policy 1, policy_version 696389 (0.0005) [2023-12-27 10:27:00,942][20381] Updated weights for policy 1, policy_version 696399 (0.0005) [2023-12-27 10:27:00,973][20347] Updated weights for policy 0, policy_version 696875 (0.0008) [2023-12-27 10:27:01,028][20347] Updated weights for policy 0, policy_version 696885 (0.0007) [2023-12-27 10:27:01,082][20347] Updated weights for policy 0, policy_version 696895 (0.0010) [2023-12-27 10:27:01,563][20381] Updated weights for policy 1, policy_version 696409 (0.0005) [2023-12-27 10:27:01,623][20381] Updated weights for policy 1, policy_version 696419 (0.0007) [2023-12-27 10:27:01,680][20381] Updated weights for policy 1, policy_version 696429 (0.0007) [2023-12-27 10:27:01,742][20381] Updated weights for policy 1, policy_version 696439 (0.0008) [2023-12-27 10:27:01,747][20347] Updated weights for policy 0, policy_version 696905 (0.0010) [2023-12-27 10:27:01,801][20347] Updated weights for policy 0, policy_version 696915 (0.0007) [2023-12-27 10:27:01,871][20347] Updated weights for policy 0, policy_version 696925 (0.0006) [2023-12-27 10:27:01,932][20347] Updated weights for policy 0, policy_version 696935 (0.0005) [2023-12-27 10:27:02,347][20381] Updated weights for policy 1, policy_version 696449 (0.0007) [2023-12-27 10:27:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 356753408. Throughput: 0: 10377.8, 1: 10338.7. Samples: 356767664. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:02,363][19170] Avg episode reward: [(0, '970.950'), (1, '990.280')] [2023-12-27 10:27:02,412][20381] Updated weights for policy 1, policy_version 696459 (0.0008) [2023-12-27 10:27:02,465][20381] Updated weights for policy 1, policy_version 696469 (0.0007) [2023-12-27 10:27:02,490][20347] Updated weights for policy 0, policy_version 696945 (0.0006) [2023-12-27 10:27:02,546][20347] Updated weights for policy 0, policy_version 696955 (0.0008) [2023-12-27 10:27:02,596][20347] Updated weights for policy 0, policy_version 696965 (0.0009) [2023-12-27 10:27:03,090][20381] Updated weights for policy 1, policy_version 696479 (0.0006) [2023-12-27 10:27:03,134][20381] Updated weights for policy 1, policy_version 696489 (0.0005) [2023-12-27 10:27:03,189][20381] Updated weights for policy 1, policy_version 696499 (0.0005) [2023-12-27 10:27:03,224][20347] Updated weights for policy 0, policy_version 696975 (0.0008) [2023-12-27 10:27:03,275][20347] Updated weights for policy 0, policy_version 696985 (0.0005) [2023-12-27 10:27:03,326][20347] Updated weights for policy 0, policy_version 696995 (0.0005) [2023-12-27 10:27:03,859][20381] Updated weights for policy 1, policy_version 696509 (0.0006) [2023-12-27 10:27:03,878][20347] Updated weights for policy 0, policy_version 697005 (0.0007) [2023-12-27 10:27:03,917][20381] Updated weights for policy 1, policy_version 696519 (0.0006) [2023-12-27 10:27:03,936][20347] Updated weights for policy 0, policy_version 697015 (0.0006) [2023-12-27 10:27:03,973][20381] Updated weights for policy 1, policy_version 696529 (0.0006) [2023-12-27 10:27:03,993][20347] Updated weights for policy 0, policy_version 697025 (0.0007) [2023-12-27 10:27:04,619][20347] Updated weights for policy 0, policy_version 697035 (0.0007) [2023-12-27 10:27:04,635][20381] Updated weights for policy 1, policy_version 696539 (0.0008) [2023-12-27 10:27:04,676][20347] Updated weights for policy 0, policy_version 697045 (0.0006) [2023-12-27 10:27:04,682][20381] Updated weights for policy 1, policy_version 696549 (0.0005) [2023-12-27 10:27:04,727][20381] Updated weights for policy 1, policy_version 696559 (0.0005) [2023-12-27 10:27:04,729][20347] Updated weights for policy 0, policy_version 697055 (0.0007) [2023-12-27 10:27:05,331][20347] Updated weights for policy 0, policy_version 697065 (0.0008) [2023-12-27 10:27:05,390][20347] Updated weights for policy 0, policy_version 697075 (0.0008) [2023-12-27 10:27:05,439][20347] Updated weights for policy 0, policy_version 697085 (0.0007) [2023-12-27 10:27:05,473][20381] Updated weights for policy 1, policy_version 696569 (0.0006) [2023-12-27 10:27:05,490][20347] Updated weights for policy 0, policy_version 697095 (0.0006) [2023-12-27 10:27:05,526][20381] Updated weights for policy 1, policy_version 696579 (0.0007) [2023-12-27 10:27:05,585][20381] Updated weights for policy 1, policy_version 696589 (0.0008) [2023-12-27 10:27:05,642][20381] Updated weights for policy 1, policy_version 696599 (0.0008) [2023-12-27 10:27:06,175][20347] Updated weights for policy 0, policy_version 697105 (0.0007) [2023-12-27 10:27:06,238][20347] Updated weights for policy 0, policy_version 697115 (0.0008) [2023-12-27 10:27:06,297][20347] Updated weights for policy 0, policy_version 697125 (0.0008) [2023-12-27 10:27:06,333][20381] Updated weights for policy 1, policy_version 696609 (0.0006) [2023-12-27 10:27:06,384][20381] Updated weights for policy 1, policy_version 696619 (0.0007) [2023-12-27 10:27:06,435][20381] Updated weights for policy 1, policy_version 696629 (0.0007) [2023-12-27 10:27:07,024][20381] Updated weights for policy 1, policy_version 696639 (0.0006) [2023-12-27 10:27:07,042][20347] Updated weights for policy 0, policy_version 697135 (0.0007) [2023-12-27 10:27:07,091][20381] Updated weights for policy 1, policy_version 696649 (0.0006) [2023-12-27 10:27:07,096][20347] Updated weights for policy 0, policy_version 697145 (0.0008) [2023-12-27 10:27:07,149][20347] Updated weights for policy 0, policy_version 697155 (0.0006) [2023-12-27 10:27:07,159][20381] Updated weights for policy 1, policy_version 696659 (0.0006) [2023-12-27 10:27:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 356868096. Throughput: 0: 10479.9, 1: 10330.3. Samples: 356833092. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:07,363][19170] Avg episode reward: [(0, '980.630'), (1, '1000.000')] [2023-12-27 10:27:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000696664_178372608.pth... [2023-12-27 10:27:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000697160_178495488.pth... [2023-12-27 10:27:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000695944_178184192.pth [2023-12-27 10:27:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000695448_178061312.pth [2023-12-27 10:27:07,803][20381] Updated weights for policy 1, policy_version 696669 (0.0007) [2023-12-27 10:27:07,825][20347] Updated weights for policy 0, policy_version 697165 (0.0007) [2023-12-27 10:27:07,853][20381] Updated weights for policy 1, policy_version 696679 (0.0006) [2023-12-27 10:27:07,880][20347] Updated weights for policy 0, policy_version 697175 (0.0007) [2023-12-27 10:27:07,911][20381] Updated weights for policy 1, policy_version 696689 (0.0006) [2023-12-27 10:27:07,938][20347] Updated weights for policy 0, policy_version 697185 (0.0006) [2023-12-27 10:27:08,569][20381] Updated weights for policy 1, policy_version 696699 (0.0007) [2023-12-27 10:27:08,599][20347] Updated weights for policy 0, policy_version 697195 (0.0007) [2023-12-27 10:27:08,625][20381] Updated weights for policy 1, policy_version 696709 (0.0010) [2023-12-27 10:27:08,650][20347] Updated weights for policy 0, policy_version 697205 (0.0006) [2023-12-27 10:27:08,678][20381] Updated weights for policy 1, policy_version 696719 (0.0006) [2023-12-27 10:27:08,709][20347] Updated weights for policy 0, policy_version 697215 (0.0008) [2023-12-27 10:27:09,247][20381] Updated weights for policy 1, policy_version 696729 (0.0008) [2023-12-27 10:27:09,298][20381] Updated weights for policy 1, policy_version 696739 (0.0008) [2023-12-27 10:27:09,373][20381] Updated weights for policy 1, policy_version 696749 (0.0009) [2023-12-27 10:27:09,424][20347] Updated weights for policy 0, policy_version 697225 (0.0009) [2023-12-27 10:27:09,436][20381] Updated weights for policy 1, policy_version 696759 (0.0009) [2023-12-27 10:27:09,483][20347] Updated weights for policy 0, policy_version 697235 (0.0006) [2023-12-27 10:27:09,546][20347] Updated weights for policy 0, policy_version 697245 (0.0007) [2023-12-27 10:27:09,612][20347] Updated weights for policy 0, policy_version 697255 (0.0007) [2023-12-27 10:27:10,090][20381] Updated weights for policy 1, policy_version 696769 (0.0008) [2023-12-27 10:27:10,144][20381] Updated weights for policy 1, policy_version 696779 (0.0008) [2023-12-27 10:27:10,194][20381] Updated weights for policy 1, policy_version 696789 (0.0008) [2023-12-27 10:27:10,312][20347] Updated weights for policy 0, policy_version 697265 (0.0008) [2023-12-27 10:27:10,376][20347] Updated weights for policy 0, policy_version 697275 (0.0007) [2023-12-27 10:27:10,438][20347] Updated weights for policy 0, policy_version 697285 (0.0008) [2023-12-27 10:27:10,848][20381] Updated weights for policy 1, policy_version 696799 (0.0007) [2023-12-27 10:27:10,910][20381] Updated weights for policy 1, policy_version 696809 (0.0007) [2023-12-27 10:27:10,968][20381] Updated weights for policy 1, policy_version 696819 (0.0005) [2023-12-27 10:27:11,164][20347] Updated weights for policy 0, policy_version 697295 (0.0009) [2023-12-27 10:27:11,227][20347] Updated weights for policy 0, policy_version 697305 (0.0007) [2023-12-27 10:27:11,296][20347] Updated weights for policy 0, policy_version 697315 (0.0008) [2023-12-27 10:27:11,720][20381] Updated weights for policy 1, policy_version 696829 (0.0007) [2023-12-27 10:27:11,786][20381] Updated weights for policy 1, policy_version 696839 (0.0007) [2023-12-27 10:27:11,847][20381] Updated weights for policy 1, policy_version 696849 (0.0009) [2023-12-27 10:27:11,976][20347] Updated weights for policy 0, policy_version 697325 (0.0006) [2023-12-27 10:27:12,026][20347] Updated weights for policy 0, policy_version 697335 (0.0005) [2023-12-27 10:27:12,078][20347] Updated weights for policy 0, policy_version 697345 (0.0005) [2023-12-27 10:27:12,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 356966400. Throughput: 0: 10409.0, 1: 10440.2. Samples: 356957352. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:12,363][19170] Avg episode reward: [(0, '990.310'), (1, '1000.000')] [2023-12-27 10:27:12,495][20381] Updated weights for policy 1, policy_version 696859 (0.0007) [2023-12-27 10:27:12,554][20381] Updated weights for policy 1, policy_version 696869 (0.0005) [2023-12-27 10:27:12,620][20381] Updated weights for policy 1, policy_version 696879 (0.0006) [2023-12-27 10:27:12,753][20347] Updated weights for policy 0, policy_version 697355 (0.0006) [2023-12-27 10:27:12,812][20347] Updated weights for policy 0, policy_version 697365 (0.0006) [2023-12-27 10:27:12,872][20347] Updated weights for policy 0, policy_version 697375 (0.0006) [2023-12-27 10:27:13,273][20381] Updated weights for policy 1, policy_version 696889 (0.0006) [2023-12-27 10:27:13,333][20381] Updated weights for policy 1, policy_version 696899 (0.0008) [2023-12-27 10:27:13,382][20381] Updated weights for policy 1, policy_version 696909 (0.0007) [2023-12-27 10:27:13,440][20381] Updated weights for policy 1, policy_version 696919 (0.0008) [2023-12-27 10:27:13,445][20347] Updated weights for policy 0, policy_version 697385 (0.0006) [2023-12-27 10:27:13,507][20347] Updated weights for policy 0, policy_version 697395 (0.0007) [2023-12-27 10:27:13,571][20347] Updated weights for policy 0, policy_version 697405 (0.0008) [2023-12-27 10:27:13,630][20347] Updated weights for policy 0, policy_version 697415 (0.0007) [2023-12-27 10:27:14,170][20381] Updated weights for policy 1, policy_version 696929 (0.0008) [2023-12-27 10:27:14,214][20381] Updated weights for policy 1, policy_version 696939 (0.0007) [2023-12-27 10:27:14,268][20381] Updated weights for policy 1, policy_version 696949 (0.0007) [2023-12-27 10:27:14,280][20347] Updated weights for policy 0, policy_version 697425 (0.0005) [2023-12-27 10:27:14,329][20347] Updated weights for policy 0, policy_version 697435 (0.0005) [2023-12-27 10:27:14,381][20347] Updated weights for policy 0, policy_version 697445 (0.0005) [2023-12-27 10:27:15,021][20381] Updated weights for policy 1, policy_version 696959 (0.0007) [2023-12-27 10:27:15,044][20347] Updated weights for policy 0, policy_version 697455 (0.0006) [2023-12-27 10:27:15,082][20381] Updated weights for policy 1, policy_version 696969 (0.0008) [2023-12-27 10:27:15,096][20347] Updated weights for policy 0, policy_version 697465 (0.0006) [2023-12-27 10:27:15,138][20381] Updated weights for policy 1, policy_version 696979 (0.0007) [2023-12-27 10:27:15,154][20347] Updated weights for policy 0, policy_version 697475 (0.0006) [2023-12-27 10:27:15,807][20347] Updated weights for policy 0, policy_version 697485 (0.0007) [2023-12-27 10:27:15,858][20347] Updated weights for policy 0, policy_version 697495 (0.0007) [2023-12-27 10:27:15,865][20381] Updated weights for policy 1, policy_version 696989 (0.0009) [2023-12-27 10:27:15,907][20347] Updated weights for policy 0, policy_version 697505 (0.0007) [2023-12-27 10:27:15,921][20381] Updated weights for policy 1, policy_version 696999 (0.0007) [2023-12-27 10:27:15,979][20381] Updated weights for policy 1, policy_version 697009 (0.0007) [2023-12-27 10:27:16,583][20381] Updated weights for policy 1, policy_version 697019 (0.0007) [2023-12-27 10:27:16,635][20381] Updated weights for policy 1, policy_version 697029 (0.0010) [2023-12-27 10:27:16,641][20347] Updated weights for policy 0, policy_version 697515 (0.0006) [2023-12-27 10:27:16,690][20381] Updated weights for policy 1, policy_version 697039 (0.0010) [2023-12-27 10:27:16,700][20347] Updated weights for policy 0, policy_version 697525 (0.0005) [2023-12-27 10:27:16,766][20347] Updated weights for policy 0, policy_version 697535 (0.0005) [2023-12-27 10:27:17,250][20381] Updated weights for policy 1, policy_version 697049 (0.0008) [2023-12-27 10:27:17,304][20381] Updated weights for policy 1, policy_version 697059 (0.0010) [2023-12-27 10:27:17,362][20381] Updated weights for policy 1, policy_version 697069 (0.0010) [2023-12-27 10:27:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 357064704. Throughput: 0: 10433.6, 1: 10349.1. Samples: 357081452. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 10:27:17,419][20381] Updated weights for policy 1, policy_version 697079 (0.0010) [2023-12-27 10:27:17,438][20347] Updated weights for policy 0, policy_version 697545 (0.0007) [2023-12-27 10:27:17,494][20347] Updated weights for policy 0, policy_version 697555 (0.0008) [2023-12-27 10:27:17,548][20347] Updated weights for policy 0, policy_version 697565 (0.0007) [2023-12-27 10:27:17,605][20347] Updated weights for policy 0, policy_version 697575 (0.0007) [2023-12-27 10:27:18,093][20381] Updated weights for policy 1, policy_version 697089 (0.0007) [2023-12-27 10:27:18,154][20381] Updated weights for policy 1, policy_version 697099 (0.0011) [2023-12-27 10:27:18,216][20381] Updated weights for policy 1, policy_version 697109 (0.0010) [2023-12-27 10:27:18,320][20347] Updated weights for policy 0, policy_version 697585 (0.0008) [2023-12-27 10:27:18,392][20347] Updated weights for policy 0, policy_version 697595 (0.0008) [2023-12-27 10:27:18,450][20347] Updated weights for policy 0, policy_version 697605 (0.0007) [2023-12-27 10:27:18,833][20381] Updated weights for policy 1, policy_version 697119 (0.0010) [2023-12-27 10:27:18,885][20381] Updated weights for policy 1, policy_version 697129 (0.0005) [2023-12-27 10:27:18,935][20381] Updated weights for policy 1, policy_version 697139 (0.0005) [2023-12-27 10:27:19,130][20347] Updated weights for policy 0, policy_version 697615 (0.0008) [2023-12-27 10:27:19,195][20347] Updated weights for policy 0, policy_version 697625 (0.0008) [2023-12-27 10:27:19,262][20347] Updated weights for policy 0, policy_version 697635 (0.0008) [2023-12-27 10:27:19,643][20381] Updated weights for policy 1, policy_version 697149 (0.0008) [2023-12-27 10:27:19,696][20381] Updated weights for policy 1, policy_version 697159 (0.0007) [2023-12-27 10:27:19,753][20381] Updated weights for policy 1, policy_version 697169 (0.0008) [2023-12-27 10:27:19,947][20347] Updated weights for policy 0, policy_version 697645 (0.0007) [2023-12-27 10:27:20,001][20347] Updated weights for policy 0, policy_version 697655 (0.0007) [2023-12-27 10:27:20,061][20347] Updated weights for policy 0, policy_version 697665 (0.0007) [2023-12-27 10:27:20,426][20381] Updated weights for policy 1, policy_version 697179 (0.0007) [2023-12-27 10:27:20,485][20381] Updated weights for policy 1, policy_version 697189 (0.0006) [2023-12-27 10:27:20,537][20381] Updated weights for policy 1, policy_version 697199 (0.0006) [2023-12-27 10:27:20,775][20347] Updated weights for policy 0, policy_version 697675 (0.0008) [2023-12-27 10:27:20,836][20347] Updated weights for policy 0, policy_version 697685 (0.0007) [2023-12-27 10:27:20,893][20347] Updated weights for policy 0, policy_version 697695 (0.0007) [2023-12-27 10:27:21,241][20381] Updated weights for policy 1, policy_version 697209 (0.0007) [2023-12-27 10:27:21,304][20381] Updated weights for policy 1, policy_version 697219 (0.0008) [2023-12-27 10:27:21,373][20381] Updated weights for policy 1, policy_version 697229 (0.0008) [2023-12-27 10:27:21,436][20381] Updated weights for policy 1, policy_version 697239 (0.0008) [2023-12-27 10:27:21,567][20347] Updated weights for policy 0, policy_version 697705 (0.0006) [2023-12-27 10:27:21,620][20347] Updated weights for policy 0, policy_version 697715 (0.0007) [2023-12-27 10:27:21,681][20347] Updated weights for policy 0, policy_version 697725 (0.0008) [2023-12-27 10:27:21,742][20347] Updated weights for policy 0, policy_version 697735 (0.0007) [2023-12-27 10:27:22,155][20381] Updated weights for policy 1, policy_version 697249 (0.0009) [2023-12-27 10:27:22,204][20381] Updated weights for policy 1, policy_version 697259 (0.0010) [2023-12-27 10:27:22,263][20381] Updated weights for policy 1, policy_version 697269 (0.0011) [2023-12-27 10:27:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 357171200. Throughput: 0: 10350.5, 1: 10361.5. Samples: 357141440. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:22,364][19170] Avg episode reward: [(0, '990.310'), (1, '1000.000')] [2023-12-27 10:27:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000697272_178528256.pth... [2023-12-27 10:27:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000697736_178642944.pth... [2023-12-27 10:27:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000696056_178216960.pth [2023-12-27 10:27:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000696520_178331648.pth [2023-12-27 10:27:22,452][20347] Updated weights for policy 0, policy_version 697745 (0.0006) [2023-12-27 10:27:22,501][20347] Updated weights for policy 0, policy_version 697755 (0.0006) [2023-12-27 10:27:22,565][20347] Updated weights for policy 0, policy_version 697765 (0.0007) [2023-12-27 10:27:22,952][20381] Updated weights for policy 1, policy_version 697279 (0.0010) [2023-12-27 10:27:23,017][20381] Updated weights for policy 1, policy_version 697289 (0.0010) [2023-12-27 10:27:23,069][20381] Updated weights for policy 1, policy_version 697299 (0.0008) [2023-12-27 10:27:23,216][20347] Updated weights for policy 0, policy_version 697775 (0.0007) [2023-12-27 10:27:23,277][20347] Updated weights for policy 0, policy_version 697785 (0.0006) [2023-12-27 10:27:23,341][20347] Updated weights for policy 0, policy_version 697795 (0.0006) [2023-12-27 10:27:23,719][20381] Updated weights for policy 1, policy_version 697309 (0.0008) [2023-12-27 10:27:23,780][20381] Updated weights for policy 1, policy_version 697319 (0.0010) [2023-12-27 10:27:23,837][20381] Updated weights for policy 1, policy_version 697329 (0.0010) [2023-12-27 10:27:23,884][20347] Updated weights for policy 0, policy_version 697805 (0.0008) [2023-12-27 10:27:23,943][20347] Updated weights for policy 0, policy_version 697815 (0.0010) [2023-12-27 10:27:23,994][20347] Updated weights for policy 0, policy_version 697825 (0.0010) [2023-12-27 10:27:24,515][20381] Updated weights for policy 1, policy_version 697339 (0.0009) [2023-12-27 10:27:24,587][20381] Updated weights for policy 1, policy_version 697349 (0.0006) [2023-12-27 10:27:24,648][20381] Updated weights for policy 1, policy_version 697359 (0.0009) [2023-12-27 10:27:24,649][20347] Updated weights for policy 0, policy_version 697835 (0.0010) [2023-12-27 10:27:24,708][20347] Updated weights for policy 0, policy_version 697845 (0.0010) [2023-12-27 10:27:24,769][20347] Updated weights for policy 0, policy_version 697855 (0.0010) [2023-12-27 10:27:25,181][20381] Updated weights for policy 1, policy_version 697369 (0.0010) [2023-12-27 10:27:25,231][20381] Updated weights for policy 1, policy_version 697379 (0.0006) [2023-12-27 10:27:25,283][20381] Updated weights for policy 1, policy_version 697389 (0.0010) [2023-12-27 10:27:25,341][20381] Updated weights for policy 1, policy_version 697399 (0.0011) [2023-12-27 10:27:25,365][20347] Updated weights for policy 0, policy_version 697865 (0.0010) [2023-12-27 10:27:25,422][20347] Updated weights for policy 0, policy_version 697875 (0.0010) [2023-12-27 10:27:25,477][20347] Updated weights for policy 0, policy_version 697885 (0.0010) [2023-12-27 10:27:25,536][20347] Updated weights for policy 0, policy_version 697895 (0.0009) [2023-12-27 10:27:26,020][20381] Updated weights for policy 1, policy_version 697409 (0.0011) [2023-12-27 10:27:26,076][20381] Updated weights for policy 1, policy_version 697419 (0.0011) [2023-12-27 10:27:26,126][20381] Updated weights for policy 1, policy_version 697429 (0.0011) [2023-12-27 10:27:26,201][20347] Updated weights for policy 0, policy_version 697905 (0.0009) [2023-12-27 10:27:26,266][20347] Updated weights for policy 0, policy_version 697915 (0.0005) [2023-12-27 10:27:26,323][20347] Updated weights for policy 0, policy_version 697925 (0.0005) [2023-12-27 10:27:26,775][20381] Updated weights for policy 1, policy_version 697439 (0.0007) [2023-12-27 10:27:26,838][20381] Updated weights for policy 1, policy_version 697449 (0.0005) [2023-12-27 10:27:26,892][20381] Updated weights for policy 1, policy_version 697459 (0.0005) [2023-12-27 10:27:26,910][20347] Updated weights for policy 0, policy_version 697935 (0.0010) [2023-12-27 10:27:26,964][20347] Updated weights for policy 0, policy_version 697945 (0.0010) [2023-12-27 10:27:27,015][20347] Updated weights for policy 0, policy_version 697955 (0.0010) [2023-12-27 10:27:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 357277696. Throughput: 0: 10410.8, 1: 10354.1. Samples: 357266684. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:27,363][19170] Avg episode reward: [(0, '990.310'), (1, '1000.000')] [2023-12-27 10:27:27,430][20381] Updated weights for policy 1, policy_version 697469 (0.0006) [2023-12-27 10:27:27,497][20381] Updated weights for policy 1, policy_version 697479 (0.0006) [2023-12-27 10:27:27,544][20381] Updated weights for policy 1, policy_version 697489 (0.0005) [2023-12-27 10:27:27,633][20347] Updated weights for policy 0, policy_version 697965 (0.0009) [2023-12-27 10:27:27,681][20347] Updated weights for policy 0, policy_version 697975 (0.0008) [2023-12-27 10:27:27,743][20347] Updated weights for policy 0, policy_version 697985 (0.0008) [2023-12-27 10:27:28,091][20381] Updated weights for policy 1, policy_version 697499 (0.0006) [2023-12-27 10:27:28,150][20381] Updated weights for policy 1, policy_version 697509 (0.0010) [2023-12-27 10:27:28,214][20381] Updated weights for policy 1, policy_version 697519 (0.0010) [2023-12-27 10:27:28,439][20347] Updated weights for policy 0, policy_version 697995 (0.0008) [2023-12-27 10:27:28,495][20347] Updated weights for policy 0, policy_version 698005 (0.0007) [2023-12-27 10:27:28,552][20347] Updated weights for policy 0, policy_version 698015 (0.0007) [2023-12-27 10:27:28,887][20381] Updated weights for policy 1, policy_version 697529 (0.0010) [2023-12-27 10:27:28,945][20381] Updated weights for policy 1, policy_version 697539 (0.0011) [2023-12-27 10:27:29,000][20381] Updated weights for policy 1, policy_version 697549 (0.0011) [2023-12-27 10:27:29,052][20381] Updated weights for policy 1, policy_version 697559 (0.0007) [2023-12-27 10:27:29,215][20347] Updated weights for policy 0, policy_version 698025 (0.0007) [2023-12-27 10:27:29,275][20347] Updated weights for policy 0, policy_version 698035 (0.0008) [2023-12-27 10:27:29,327][20347] Updated weights for policy 0, policy_version 698045 (0.0006) [2023-12-27 10:27:29,397][20347] Updated weights for policy 0, policy_version 698055 (0.0008) [2023-12-27 10:27:29,745][20381] Updated weights for policy 1, policy_version 697569 (0.0007) [2023-12-27 10:27:29,807][20381] Updated weights for policy 1, policy_version 697579 (0.0007) [2023-12-27 10:27:29,876][20381] Updated weights for policy 1, policy_version 697589 (0.0006) [2023-12-27 10:27:30,083][20347] Updated weights for policy 0, policy_version 698065 (0.0008) [2023-12-27 10:27:30,146][20347] Updated weights for policy 0, policy_version 698075 (0.0007) [2023-12-27 10:27:30,204][20347] Updated weights for policy 0, policy_version 698085 (0.0007) [2023-12-27 10:27:30,555][20381] Updated weights for policy 1, policy_version 697599 (0.0008) [2023-12-27 10:27:30,610][20381] Updated weights for policy 1, policy_version 697609 (0.0007) [2023-12-27 10:27:30,672][20381] Updated weights for policy 1, policy_version 697619 (0.0007) [2023-12-27 10:27:30,867][20347] Updated weights for policy 0, policy_version 698095 (0.0007) [2023-12-27 10:27:30,912][20347] Updated weights for policy 0, policy_version 698105 (0.0008) [2023-12-27 10:27:30,963][20347] Updated weights for policy 0, policy_version 698115 (0.0008) [2023-12-27 10:27:31,380][20381] Updated weights for policy 1, policy_version 697629 (0.0009) [2023-12-27 10:27:31,445][20381] Updated weights for policy 1, policy_version 697639 (0.0008) [2023-12-27 10:27:31,508][20381] Updated weights for policy 1, policy_version 697649 (0.0007) [2023-12-27 10:27:31,709][20347] Updated weights for policy 0, policy_version 698125 (0.0007) [2023-12-27 10:27:31,774][20347] Updated weights for policy 0, policy_version 698135 (0.0008) [2023-12-27 10:27:31,831][20347] Updated weights for policy 0, policy_version 698145 (0.0008) [2023-12-27 10:27:32,161][20381] Updated weights for policy 1, policy_version 697659 (0.0007) [2023-12-27 10:27:32,217][20381] Updated weights for policy 1, policy_version 697669 (0.0005) [2023-12-27 10:27:32,273][20381] Updated weights for policy 1, policy_version 697679 (0.0006) [2023-12-27 10:27:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 357384192. Throughput: 0: 10419.6, 1: 10366.8. Samples: 357392264. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 10:27:32,569][20347] Updated weights for policy 0, policy_version 698155 (0.0008) [2023-12-27 10:27:32,629][20347] Updated weights for policy 0, policy_version 698165 (0.0008) [2023-12-27 10:27:32,680][20347] Updated weights for policy 0, policy_version 698175 (0.0008) [2023-12-27 10:27:32,854][20381] Updated weights for policy 1, policy_version 697689 (0.0010) [2023-12-27 10:27:32,903][20381] Updated weights for policy 1, policy_version 697699 (0.0005) [2023-12-27 10:27:32,947][20381] Updated weights for policy 1, policy_version 697709 (0.0005) [2023-12-27 10:27:33,005][20381] Updated weights for policy 1, policy_version 697719 (0.0005) [2023-12-27 10:27:33,396][20347] Updated weights for policy 0, policy_version 698185 (0.0007) [2023-12-27 10:27:33,447][20347] Updated weights for policy 0, policy_version 698195 (0.0009) [2023-12-27 10:27:33,502][20347] Updated weights for policy 0, policy_version 698205 (0.0008) [2023-12-27 10:27:33,555][20347] Updated weights for policy 0, policy_version 698215 (0.0007) [2023-12-27 10:27:33,579][20381] Updated weights for policy 1, policy_version 697729 (0.0006) [2023-12-27 10:27:33,640][20381] Updated weights for policy 1, policy_version 697739 (0.0008) [2023-12-27 10:27:33,703][20381] Updated weights for policy 1, policy_version 697749 (0.0007) [2023-12-27 10:27:34,191][20347] Updated weights for policy 0, policy_version 698225 (0.0007) [2023-12-27 10:27:34,249][20347] Updated weights for policy 0, policy_version 698235 (0.0007) [2023-12-27 10:27:34,310][20347] Updated weights for policy 0, policy_version 698245 (0.0008) [2023-12-27 10:27:34,402][20381] Updated weights for policy 1, policy_version 697759 (0.0007) [2023-12-27 10:27:34,454][20381] Updated weights for policy 1, policy_version 697769 (0.0008) [2023-12-27 10:27:34,505][20381] Updated weights for policy 1, policy_version 697779 (0.0007) [2023-12-27 10:27:34,985][20347] Updated weights for policy 0, policy_version 698255 (0.0008) [2023-12-27 10:27:35,031][20347] Updated weights for policy 0, policy_version 698265 (0.0008) [2023-12-27 10:27:35,079][20347] Updated weights for policy 0, policy_version 698275 (0.0007) [2023-12-27 10:27:35,203][20381] Updated weights for policy 1, policy_version 697789 (0.0008) [2023-12-27 10:27:35,255][20381] Updated weights for policy 1, policy_version 697799 (0.0007) [2023-12-27 10:27:35,317][20381] Updated weights for policy 1, policy_version 697809 (0.0008) [2023-12-27 10:27:35,750][20347] Updated weights for policy 0, policy_version 698285 (0.0008) [2023-12-27 10:27:35,812][20347] Updated weights for policy 0, policy_version 698295 (0.0008) [2023-12-27 10:27:35,864][20347] Updated weights for policy 0, policy_version 698305 (0.0008) [2023-12-27 10:27:36,009][20381] Updated weights for policy 1, policy_version 697819 (0.0008) [2023-12-27 10:27:36,065][20381] Updated weights for policy 1, policy_version 697829 (0.0007) [2023-12-27 10:27:36,126][20381] Updated weights for policy 1, policy_version 697839 (0.0007) [2023-12-27 10:27:36,590][20347] Updated weights for policy 0, policy_version 698315 (0.0008) [2023-12-27 10:27:36,657][20347] Updated weights for policy 0, policy_version 698325 (0.0008) [2023-12-27 10:27:36,720][20347] Updated weights for policy 0, policy_version 698335 (0.0008) [2023-12-27 10:27:36,837][20381] Updated weights for policy 1, policy_version 697849 (0.0009) [2023-12-27 10:27:36,893][20381] Updated weights for policy 1, policy_version 697859 (0.0008) [2023-12-27 10:27:36,948][20381] Updated weights for policy 1, policy_version 697869 (0.0007) [2023-12-27 10:27:37,004][20381] Updated weights for policy 1, policy_version 697879 (0.0007) [2023-12-27 10:27:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 357482496. Throughput: 0: 10379.7, 1: 10412.5. Samples: 357455108. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 10:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000697880_178683904.pth... [2023-12-27 10:27:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000698344_178798592.pth... [2023-12-27 10:27:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000697160_178495488.pth [2023-12-27 10:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000696664_178372608.pth [2023-12-27 10:27:37,418][20347] Updated weights for policy 0, policy_version 698345 (0.0008) [2023-12-27 10:27:37,470][20347] Updated weights for policy 0, policy_version 698355 (0.0006) [2023-12-27 10:27:37,525][20347] Updated weights for policy 0, policy_version 698365 (0.0006) [2023-12-27 10:27:37,576][20347] Updated weights for policy 0, policy_version 698375 (0.0006) [2023-12-27 10:27:37,724][20381] Updated weights for policy 1, policy_version 697889 (0.0008) [2023-12-27 10:27:37,784][20381] Updated weights for policy 1, policy_version 697899 (0.0007) [2023-12-27 10:27:37,839][20381] Updated weights for policy 1, policy_version 697909 (0.0007) [2023-12-27 10:27:38,213][20347] Updated weights for policy 0, policy_version 698385 (0.0007) [2023-12-27 10:27:38,278][20347] Updated weights for policy 0, policy_version 698395 (0.0008) [2023-12-27 10:27:38,336][20347] Updated weights for policy 0, policy_version 698405 (0.0009) [2023-12-27 10:27:38,543][20381] Updated weights for policy 1, policy_version 697919 (0.0008) [2023-12-27 10:27:38,598][20381] Updated weights for policy 1, policy_version 697929 (0.0007) [2023-12-27 10:27:38,650][20381] Updated weights for policy 1, policy_version 697939 (0.0007) [2023-12-27 10:27:39,038][20347] Updated weights for policy 0, policy_version 698415 (0.0010) [2023-12-27 10:27:39,089][20347] Updated weights for policy 0, policy_version 698425 (0.0010) [2023-12-27 10:27:39,147][20347] Updated weights for policy 0, policy_version 698435 (0.0010) [2023-12-27 10:27:39,364][20381] Updated weights for policy 1, policy_version 697949 (0.0008) [2023-12-27 10:27:39,430][20381] Updated weights for policy 1, policy_version 697959 (0.0007) [2023-12-27 10:27:39,488][20381] Updated weights for policy 1, policy_version 697969 (0.0005) [2023-12-27 10:27:39,883][20347] Updated weights for policy 0, policy_version 698445 (0.0008) [2023-12-27 10:27:39,943][20347] Updated weights for policy 0, policy_version 698455 (0.0009) [2023-12-27 10:27:40,008][20347] Updated weights for policy 0, policy_version 698465 (0.0009) [2023-12-27 10:27:40,169][20381] Updated weights for policy 1, policy_version 697979 (0.0007) [2023-12-27 10:27:40,225][20381] Updated weights for policy 1, policy_version 697989 (0.0009) [2023-12-27 10:27:40,290][20381] Updated weights for policy 1, policy_version 697999 (0.0008) [2023-12-27 10:27:40,649][20347] Updated weights for policy 0, policy_version 698475 (0.0008) [2023-12-27 10:27:40,720][20347] Updated weights for policy 0, policy_version 698485 (0.0007) [2023-12-27 10:27:40,774][20347] Updated weights for policy 0, policy_version 698495 (0.0007) [2023-12-27 10:27:40,984][20381] Updated weights for policy 1, policy_version 698009 (0.0008) [2023-12-27 10:27:41,030][20381] Updated weights for policy 1, policy_version 698019 (0.0008) [2023-12-27 10:27:41,089][20381] Updated weights for policy 1, policy_version 698029 (0.0008) [2023-12-27 10:27:41,155][20381] Updated weights for policy 1, policy_version 698039 (0.0008) [2023-12-27 10:27:41,468][20347] Updated weights for policy 0, policy_version 698505 (0.0008) [2023-12-27 10:27:41,528][20347] Updated weights for policy 0, policy_version 698515 (0.0007) [2023-12-27 10:27:41,584][20347] Updated weights for policy 0, policy_version 698525 (0.0008) [2023-12-27 10:27:41,654][20347] Updated weights for policy 0, policy_version 698535 (0.0009) [2023-12-27 10:27:41,898][20381] Updated weights for policy 1, policy_version 698049 (0.0008) [2023-12-27 10:27:41,958][20381] Updated weights for policy 1, policy_version 698059 (0.0008) [2023-12-27 10:27:42,018][20381] Updated weights for policy 1, policy_version 698069 (0.0008) [2023-12-27 10:27:42,359][20347] Updated weights for policy 0, policy_version 698545 (0.0007) [2023-12-27 10:27:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 357580800. Throughput: 0: 10328.4, 1: 10356.1. Samples: 357574776. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:42,363][19170] Avg episode reward: [(0, '977.830'), (1, '1000.000')] [2023-12-27 10:27:42,418][20347] Updated weights for policy 0, policy_version 698555 (0.0007) [2023-12-27 10:27:42,476][20347] Updated weights for policy 0, policy_version 698565 (0.0007) [2023-12-27 10:27:42,790][20381] Updated weights for policy 1, policy_version 698079 (0.0007) [2023-12-27 10:27:42,851][20381] Updated weights for policy 1, policy_version 698089 (0.0007) [2023-12-27 10:27:42,911][20381] Updated weights for policy 1, policy_version 698099 (0.0006) [2023-12-27 10:27:43,173][20347] Updated weights for policy 0, policy_version 698575 (0.0008) [2023-12-27 10:27:43,231][20347] Updated weights for policy 0, policy_version 698585 (0.0007) [2023-12-27 10:27:43,282][20347] Updated weights for policy 0, policy_version 698595 (0.0005) [2023-12-27 10:27:43,507][20381] Updated weights for policy 1, policy_version 698109 (0.0007) [2023-12-27 10:27:43,572][20381] Updated weights for policy 1, policy_version 698119 (0.0008) [2023-12-27 10:27:43,635][20381] Updated weights for policy 1, policy_version 698129 (0.0008) [2023-12-27 10:27:43,926][20347] Updated weights for policy 0, policy_version 698605 (0.0007) [2023-12-27 10:27:43,979][20347] Updated weights for policy 0, policy_version 698615 (0.0007) [2023-12-27 10:27:44,038][20347] Updated weights for policy 0, policy_version 698625 (0.0007) [2023-12-27 10:27:44,336][20381] Updated weights for policy 1, policy_version 698139 (0.0006) [2023-12-27 10:27:44,383][20381] Updated weights for policy 1, policy_version 698149 (0.0007) [2023-12-27 10:27:44,434][20381] Updated weights for policy 1, policy_version 698159 (0.0007) [2023-12-27 10:27:44,640][20347] Updated weights for policy 0, policy_version 698635 (0.0007) [2023-12-27 10:27:44,690][20347] Updated weights for policy 0, policy_version 698645 (0.0005) [2023-12-27 10:27:44,741][20347] Updated weights for policy 0, policy_version 698655 (0.0005) [2023-12-27 10:27:45,179][20381] Updated weights for policy 1, policy_version 698169 (0.0009) [2023-12-27 10:27:45,240][20381] Updated weights for policy 1, policy_version 698179 (0.0006) [2023-12-27 10:27:45,301][20381] Updated weights for policy 1, policy_version 698189 (0.0007) [2023-12-27 10:27:45,358][20381] Updated weights for policy 1, policy_version 698199 (0.0005) [2023-12-27 10:27:45,447][20347] Updated weights for policy 0, policy_version 698665 (0.0008) [2023-12-27 10:27:45,505][20347] Updated weights for policy 0, policy_version 698675 (0.0006) [2023-12-27 10:27:45,563][20347] Updated weights for policy 0, policy_version 698685 (0.0006) [2023-12-27 10:27:45,626][20347] Updated weights for policy 0, policy_version 698695 (0.0006) [2023-12-27 10:27:45,933][20381] Updated weights for policy 1, policy_version 698209 (0.0008) [2023-12-27 10:27:45,982][20381] Updated weights for policy 1, policy_version 698219 (0.0008) [2023-12-27 10:27:46,040][20381] Updated weights for policy 1, policy_version 698229 (0.0008) [2023-12-27 10:27:46,281][20347] Updated weights for policy 0, policy_version 698705 (0.0008) [2023-12-27 10:27:46,336][20347] Updated weights for policy 0, policy_version 698715 (0.0008) [2023-12-27 10:27:46,395][20347] Updated weights for policy 0, policy_version 698725 (0.0007) [2023-12-27 10:27:46,668][20381] Updated weights for policy 1, policy_version 698239 (0.0008) [2023-12-27 10:27:46,734][20381] Updated weights for policy 1, policy_version 698249 (0.0006) [2023-12-27 10:27:46,798][20381] Updated weights for policy 1, policy_version 698259 (0.0006) [2023-12-27 10:27:47,024][20347] Updated weights for policy 0, policy_version 698735 (0.0008) [2023-12-27 10:27:47,074][20347] Updated weights for policy 0, policy_version 698745 (0.0008) [2023-12-27 10:27:47,122][20347] Updated weights for policy 0, policy_version 698755 (0.0008) [2023-12-27 10:27:47,355][20381] Updated weights for policy 1, policy_version 698269 (0.0007) [2023-12-27 10:27:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 357687296. Throughput: 0: 10310.7, 1: 10369.7. Samples: 357698284. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:47,363][19170] Avg episode reward: [(0, '948.240'), (1, '1000.000')] [2023-12-27 10:27:47,410][20381] Updated weights for policy 1, policy_version 698279 (0.0006) [2023-12-27 10:27:47,459][20381] Updated weights for policy 1, policy_version 698289 (0.0007) [2023-12-27 10:27:47,852][20347] Updated weights for policy 0, policy_version 698765 (0.0008) [2023-12-27 10:27:47,918][20347] Updated weights for policy 0, policy_version 698775 (0.0008) [2023-12-27 10:27:47,970][20347] Updated weights for policy 0, policy_version 698785 (0.0008) [2023-12-27 10:27:47,993][20381] Updated weights for policy 1, policy_version 698299 (0.0005) [2023-12-27 10:27:48,056][20381] Updated weights for policy 1, policy_version 698309 (0.0007) [2023-12-27 10:27:48,112][20381] Updated weights for policy 1, policy_version 698319 (0.0005) [2023-12-27 10:27:48,674][20381] Updated weights for policy 1, policy_version 698329 (0.0005) [2023-12-27 10:27:48,704][20347] Updated weights for policy 0, policy_version 698795 (0.0008) [2023-12-27 10:27:48,739][20381] Updated weights for policy 1, policy_version 698339 (0.0010) [2023-12-27 10:27:48,760][20347] Updated weights for policy 0, policy_version 698805 (0.0005) [2023-12-27 10:27:48,805][20381] Updated weights for policy 1, policy_version 698349 (0.0011) [2023-12-27 10:27:48,808][20347] Updated weights for policy 0, policy_version 698815 (0.0006) [2023-12-27 10:27:48,858][20381] Updated weights for policy 1, policy_version 698359 (0.0011) [2023-12-27 10:27:49,487][20381] Updated weights for policy 1, policy_version 698369 (0.0008) [2023-12-27 10:27:49,518][20347] Updated weights for policy 0, policy_version 698825 (0.0006) [2023-12-27 10:27:49,531][20381] Updated weights for policy 1, policy_version 698379 (0.0007) [2023-12-27 10:27:49,570][20347] Updated weights for policy 0, policy_version 698835 (0.0007) [2023-12-27 10:27:49,584][20381] Updated weights for policy 1, policy_version 698389 (0.0005) [2023-12-27 10:27:49,626][20347] Updated weights for policy 0, policy_version 698845 (0.0007) [2023-12-27 10:27:49,681][20347] Updated weights for policy 0, policy_version 698855 (0.0007) [2023-12-27 10:27:50,276][20381] Updated weights for policy 1, policy_version 698399 (0.0007) [2023-12-27 10:27:50,334][20381] Updated weights for policy 1, policy_version 698409 (0.0007) [2023-12-27 10:27:50,390][20381] Updated weights for policy 1, policy_version 698419 (0.0008) [2023-12-27 10:27:50,441][20347] Updated weights for policy 0, policy_version 698865 (0.0008) [2023-12-27 10:27:50,487][20347] Updated weights for policy 0, policy_version 698875 (0.0008) [2023-12-27 10:27:50,543][20347] Updated weights for policy 0, policy_version 698885 (0.0006) [2023-12-27 10:27:51,078][20381] Updated weights for policy 1, policy_version 698429 (0.0007) [2023-12-27 10:27:51,137][20381] Updated weights for policy 1, policy_version 698439 (0.0008) [2023-12-27 10:27:51,195][20381] Updated weights for policy 1, policy_version 698449 (0.0007) [2023-12-27 10:27:51,270][20347] Updated weights for policy 0, policy_version 698895 (0.0006) [2023-12-27 10:27:51,343][20347] Updated weights for policy 0, policy_version 698906 (0.0008) [2023-12-27 10:27:51,409][20347] Updated weights for policy 0, policy_version 698916 (0.0009) [2023-12-27 10:27:51,929][20381] Updated weights for policy 1, policy_version 698459 (0.0009) [2023-12-27 10:27:51,985][20381] Updated weights for policy 1, policy_version 698469 (0.0008) [2023-12-27 10:27:52,046][20381] Updated weights for policy 1, policy_version 698479 (0.0008) [2023-12-27 10:27:52,116][20347] Updated weights for policy 0, policy_version 698926 (0.0009) [2023-12-27 10:27:52,181][20347] Updated weights for policy 0, policy_version 698936 (0.0008) [2023-12-27 10:27:52,243][20347] Updated weights for policy 0, policy_version 698946 (0.0009) [2023-12-27 10:27:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 357793792. Throughput: 0: 10190.9, 1: 10437.6. Samples: 357761380. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:52,363][19170] Avg episode reward: [(0, '941.900'), (1, '1000.000')] [2023-12-27 10:27:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000698488_178839552.pth... [2023-12-27 10:27:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000698952_178954240.pth... [2023-12-27 10:27:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000697272_178528256.pth [2023-12-27 10:27:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000697736_178642944.pth [2023-12-27 10:27:52,765][20381] Updated weights for policy 1, policy_version 698489 (0.0008) [2023-12-27 10:27:52,820][20381] Updated weights for policy 1, policy_version 698499 (0.0007) [2023-12-27 10:27:52,882][20381] Updated weights for policy 1, policy_version 698509 (0.0008) [2023-12-27 10:27:52,919][20347] Updated weights for policy 0, policy_version 698956 (0.0009) [2023-12-27 10:27:52,942][20381] Updated weights for policy 1, policy_version 698519 (0.0006) [2023-12-27 10:27:52,983][20347] Updated weights for policy 0, policy_version 698966 (0.0010) [2023-12-27 10:27:53,048][20347] Updated weights for policy 0, policy_version 698976 (0.0010) [2023-12-27 10:27:53,555][20381] Updated weights for policy 1, policy_version 698529 (0.0009) [2023-12-27 10:27:53,605][20381] Updated weights for policy 1, policy_version 698539 (0.0007) [2023-12-27 10:27:53,636][20347] Updated weights for policy 0, policy_version 698986 (0.0009) [2023-12-27 10:27:53,658][20381] Updated weights for policy 1, policy_version 698549 (0.0006) [2023-12-27 10:27:53,690][20347] Updated weights for policy 0, policy_version 698996 (0.0005) [2023-12-27 10:27:53,744][20347] Updated weights for policy 0, policy_version 699006 (0.0005) [2023-12-27 10:27:53,793][20347] Updated weights for policy 0, policy_version 699016 (0.0005) [2023-12-27 10:27:54,256][20381] Updated weights for policy 1, policy_version 698559 (0.0006) [2023-12-27 10:27:54,317][20381] Updated weights for policy 1, policy_version 698569 (0.0007) [2023-12-27 10:27:54,359][20347] Updated weights for policy 0, policy_version 699026 (0.0010) [2023-12-27 10:27:54,379][20381] Updated weights for policy 1, policy_version 698579 (0.0005) [2023-12-27 10:27:54,414][20347] Updated weights for policy 0, policy_version 699036 (0.0010) [2023-12-27 10:27:54,468][20347] Updated weights for policy 0, policy_version 699046 (0.0010) [2023-12-27 10:27:54,956][20381] Updated weights for policy 1, policy_version 698589 (0.0008) [2023-12-27 10:27:55,014][20381] Updated weights for policy 1, policy_version 698599 (0.0011) [2023-12-27 10:27:55,064][20347] Updated weights for policy 0, policy_version 699056 (0.0010) [2023-12-27 10:27:55,064][20381] Updated weights for policy 1, policy_version 698609 (0.0011) [2023-12-27 10:27:55,121][20347] Updated weights for policy 0, policy_version 699066 (0.0010) [2023-12-27 10:27:55,176][20347] Updated weights for policy 0, policy_version 699076 (0.0010) [2023-12-27 10:27:55,769][20381] Updated weights for policy 1, policy_version 698619 (0.0011) [2023-12-27 10:27:55,824][20381] Updated weights for policy 1, policy_version 698629 (0.0011) [2023-12-27 10:27:55,873][20381] Updated weights for policy 1, policy_version 698639 (0.0011) [2023-12-27 10:27:55,881][20347] Updated weights for policy 0, policy_version 699086 (0.0011) [2023-12-27 10:27:55,935][20347] Updated weights for policy 0, policy_version 699096 (0.0005) [2023-12-27 10:27:55,993][20347] Updated weights for policy 0, policy_version 699106 (0.0007) [2023-12-27 10:27:56,545][20381] Updated weights for policy 1, policy_version 698649 (0.0011) [2023-12-27 10:27:56,605][20381] Updated weights for policy 1, policy_version 698659 (0.0010) [2023-12-27 10:27:56,644][20347] Updated weights for policy 0, policy_version 699116 (0.0006) [2023-12-27 10:27:56,669][20381] Updated weights for policy 1, policy_version 698669 (0.0006) [2023-12-27 10:27:56,697][20347] Updated weights for policy 0, policy_version 699126 (0.0005) [2023-12-27 10:27:56,719][20381] Updated weights for policy 1, policy_version 698679 (0.0005) [2023-12-27 10:27:56,746][20347] Updated weights for policy 0, policy_version 699136 (0.0005) [2023-12-27 10:27:57,298][20381] Updated weights for policy 1, policy_version 698689 (0.0007) [2023-12-27 10:27:57,353][20381] Updated weights for policy 1, policy_version 698699 (0.0006) [2023-12-27 10:27:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.4, 300 sec: 20521.6). Total num frames: 357892096. Throughput: 0: 10261.5, 1: 10384.8. Samples: 357886436. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:27:57,364][19170] Avg episode reward: [(0, '889.650'), (1, '1000.000')] [2023-12-27 10:27:57,389][20347] Updated weights for policy 0, policy_version 699146 (0.0009) [2023-12-27 10:27:57,412][20381] Updated weights for policy 1, policy_version 698709 (0.0006) [2023-12-27 10:27:57,435][20347] Updated weights for policy 0, policy_version 699156 (0.0009) [2023-12-27 10:27:57,435][20291] KL-divergence is very high: 199.2101 [2023-12-27 10:27:57,441][20291] KL-divergence is very high: 178.6447 [2023-12-27 10:27:57,495][20347] Updated weights for policy 0, policy_version 699166 (0.0010) [2023-12-27 10:27:57,549][20347] Updated weights for policy 0, policy_version 699176 (0.0010) [2023-12-27 10:27:58,025][20381] Updated weights for policy 1, policy_version 698719 (0.0007) [2023-12-27 10:27:58,088][20381] Updated weights for policy 1, policy_version 698729 (0.0008) [2023-12-27 10:27:58,148][20381] Updated weights for policy 1, policy_version 698739 (0.0007) [2023-12-27 10:27:58,208][20347] Updated weights for policy 0, policy_version 699186 (0.0008) [2023-12-27 10:27:58,272][20347] Updated weights for policy 0, policy_version 699196 (0.0007) [2023-12-27 10:27:58,334][20347] Updated weights for policy 0, policy_version 699206 (0.0008) [2023-12-27 10:27:58,954][20381] Updated weights for policy 1, policy_version 698749 (0.0008) [2023-12-27 10:27:59,012][20381] Updated weights for policy 1, policy_version 698759 (0.0008) [2023-12-27 10:27:59,049][20347] Updated weights for policy 0, policy_version 699216 (0.0006) [2023-12-27 10:27:59,069][20381] Updated weights for policy 1, policy_version 698769 (0.0007) [2023-12-27 10:27:59,105][20347] Updated weights for policy 0, policy_version 699226 (0.0008) [2023-12-27 10:27:59,166][20347] Updated weights for policy 0, policy_version 699236 (0.0008) [2023-12-27 10:27:59,831][20381] Updated weights for policy 1, policy_version 698779 (0.0006) [2023-12-27 10:27:59,834][20347] Updated weights for policy 0, policy_version 699246 (0.0008) [2023-12-27 10:27:59,883][20381] Updated weights for policy 1, policy_version 698789 (0.0007) [2023-12-27 10:27:59,891][20347] Updated weights for policy 0, policy_version 699256 (0.0007) [2023-12-27 10:27:59,948][20381] Updated weights for policy 1, policy_version 698799 (0.0007) [2023-12-27 10:27:59,953][20347] Updated weights for policy 0, policy_version 699266 (0.0008) [2023-12-27 10:28:00,536][20381] Updated weights for policy 1, policy_version 698809 (0.0006) [2023-12-27 10:28:00,584][20381] Updated weights for policy 1, policy_version 698819 (0.0007) [2023-12-27 10:28:00,629][20347] Updated weights for policy 0, policy_version 699276 (0.0006) [2023-12-27 10:28:00,639][20381] Updated weights for policy 1, policy_version 698829 (0.0006) [2023-12-27 10:28:00,681][20347] Updated weights for policy 0, policy_version 699286 (0.0006) [2023-12-27 10:28:00,687][20381] Updated weights for policy 1, policy_version 698839 (0.0005) [2023-12-27 10:28:00,739][20347] Updated weights for policy 0, policy_version 699296 (0.0006) [2023-12-27 10:28:01,345][20347] Updated weights for policy 0, policy_version 699306 (0.0005) [2023-12-27 10:28:01,410][20347] Updated weights for policy 0, policy_version 699316 (0.0007) [2023-12-27 10:28:01,464][20347] Updated weights for policy 0, policy_version 699326 (0.0007) [2023-12-27 10:28:01,470][20381] Updated weights for policy 1, policy_version 698849 (0.0006) [2023-12-27 10:28:01,513][20347] Updated weights for policy 0, policy_version 699336 (0.0007) [2023-12-27 10:28:01,522][20381] Updated weights for policy 1, policy_version 698859 (0.0006) [2023-12-27 10:28:01,577][20381] Updated weights for policy 1, policy_version 698869 (0.0008) [2023-12-27 10:28:02,220][20347] Updated weights for policy 0, policy_version 699346 (0.0010) [2023-12-27 10:28:02,268][20381] Updated weights for policy 1, policy_version 698879 (0.0007) [2023-12-27 10:28:02,276][20347] Updated weights for policy 0, policy_version 699356 (0.0010) [2023-12-27 10:28:02,322][20381] Updated weights for policy 1, policy_version 698889 (0.0006) [2023-12-27 10:28:02,331][20347] Updated weights for policy 0, policy_version 699366 (0.0010) [2023-12-27 10:28:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 357998592. Throughput: 0: 10256.3, 1: 10370.4. Samples: 358009652. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:28:02,363][19170] Avg episode reward: [(0, '831.990'), (1, '1000.000')] [2023-12-27 10:28:02,384][20381] Updated weights for policy 1, policy_version 698899 (0.0007) [2023-12-27 10:28:02,984][20347] Updated weights for policy 0, policy_version 699376 (0.0007) [2023-12-27 10:28:03,044][20347] Updated weights for policy 0, policy_version 699386 (0.0005) [2023-12-27 10:28:03,073][20381] Updated weights for policy 1, policy_version 698909 (0.0008) [2023-12-27 10:28:03,101][20347] Updated weights for policy 0, policy_version 699396 (0.0005) [2023-12-27 10:28:03,122][20381] Updated weights for policy 1, policy_version 698919 (0.0008) [2023-12-27 10:28:03,172][20381] Updated weights for policy 1, policy_version 698929 (0.0010) [2023-12-27 10:28:03,677][20347] Updated weights for policy 0, policy_version 699406 (0.0005) [2023-12-27 10:28:03,727][20347] Updated weights for policy 0, policy_version 699416 (0.0005) [2023-12-27 10:28:03,787][20347] Updated weights for policy 0, policy_version 699426 (0.0006) [2023-12-27 10:28:03,866][20381] Updated weights for policy 1, policy_version 698939 (0.0008) [2023-12-27 10:28:03,917][20381] Updated weights for policy 1, policy_version 698949 (0.0008) [2023-12-27 10:28:03,968][20381] Updated weights for policy 1, policy_version 698959 (0.0007) [2023-12-27 10:28:04,433][20347] Updated weights for policy 0, policy_version 699436 (0.0010) [2023-12-27 10:28:04,489][20347] Updated weights for policy 0, policy_version 699446 (0.0010) [2023-12-27 10:28:04,552][20347] Updated weights for policy 0, policy_version 699456 (0.0006) [2023-12-27 10:28:04,673][20381] Updated weights for policy 1, policy_version 698969 (0.0007) [2023-12-27 10:28:04,727][20381] Updated weights for policy 1, policy_version 698979 (0.0005) [2023-12-27 10:28:04,790][20381] Updated weights for policy 1, policy_version 698989 (0.0005) [2023-12-27 10:28:04,855][20381] Updated weights for policy 1, policy_version 698999 (0.0005) [2023-12-27 10:28:05,188][20347] Updated weights for policy 0, policy_version 699466 (0.0007) [2023-12-27 10:28:05,273][20347] Updated weights for policy 0, policy_version 699476 (0.0005) [2023-12-27 10:28:05,336][20347] Updated weights for policy 0, policy_version 699486 (0.0005) [2023-12-27 10:28:05,350][20381] Updated weights for policy 1, policy_version 699009 (0.0006) [2023-12-27 10:28:05,384][20347] Updated weights for policy 0, policy_version 699496 (0.0005) [2023-12-27 10:28:05,399][20381] Updated weights for policy 1, policy_version 699019 (0.0005) [2023-12-27 10:28:05,444][20381] Updated weights for policy 1, policy_version 699029 (0.0005) [2023-12-27 10:28:05,870][20347] Updated weights for policy 0, policy_version 699506 (0.0005) [2023-12-27 10:28:05,928][20347] Updated weights for policy 0, policy_version 699516 (0.0010) [2023-12-27 10:28:05,979][20347] Updated weights for policy 0, policy_version 699526 (0.0010) [2023-12-27 10:28:06,082][20381] Updated weights for policy 1, policy_version 699039 (0.0007) [2023-12-27 10:28:06,144][20381] Updated weights for policy 1, policy_version 699049 (0.0008) [2023-12-27 10:28:06,201][20381] Updated weights for policy 1, policy_version 699059 (0.0009) [2023-12-27 10:28:06,663][20347] Updated weights for policy 0, policy_version 699536 (0.0009) [2023-12-27 10:28:06,716][20347] Updated weights for policy 0, policy_version 699546 (0.0008) [2023-12-27 10:28:06,761][20347] Updated weights for policy 0, policy_version 699556 (0.0007) [2023-12-27 10:28:06,889][20381] Updated weights for policy 1, policy_version 699069 (0.0008) [2023-12-27 10:28:06,952][20381] Updated weights for policy 1, policy_version 699079 (0.0008) [2023-12-27 10:28:07,009][20381] Updated weights for policy 1, policy_version 699089 (0.0008) [2023-12-27 10:28:07,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 358105088. Throughput: 0: 10328.1, 1: 10394.9. Samples: 358073972. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:28:07,363][19170] Avg episode reward: [(0, '844.490'), (1, '1000.000')] [2023-12-27 10:28:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000699096_178995200.pth... [2023-12-27 10:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000699560_179109888.pth... [2023-12-27 10:28:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000698344_178798592.pth [2023-12-27 10:28:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000697880_178683904.pth [2023-12-27 10:28:07,415][20347] Updated weights for policy 0, policy_version 699566 (0.0009) [2023-12-27 10:28:07,471][20347] Updated weights for policy 0, policy_version 699576 (0.0010) [2023-12-27 10:28:07,530][20347] Updated weights for policy 0, policy_version 699586 (0.0011) [2023-12-27 10:28:07,759][20381] Updated weights for policy 1, policy_version 699099 (0.0008) [2023-12-27 10:28:07,813][20381] Updated weights for policy 1, policy_version 699109 (0.0007) [2023-12-27 10:28:07,857][20381] Updated weights for policy 1, policy_version 699119 (0.0007) [2023-12-27 10:28:08,186][20347] Updated weights for policy 0, policy_version 699596 (0.0009) [2023-12-27 10:28:08,242][20347] Updated weights for policy 0, policy_version 699606 (0.0008) [2023-12-27 10:28:08,293][20347] Updated weights for policy 0, policy_version 699616 (0.0010) [2023-12-27 10:28:08,558][20381] Updated weights for policy 1, policy_version 699129 (0.0008) [2023-12-27 10:28:08,616][20381] Updated weights for policy 1, policy_version 699139 (0.0006) [2023-12-27 10:28:08,680][20381] Updated weights for policy 1, policy_version 699149 (0.0010) [2023-12-27 10:28:08,738][20381] Updated weights for policy 1, policy_version 699159 (0.0007) [2023-12-27 10:28:08,952][20347] Updated weights for policy 0, policy_version 699626 (0.0008) [2023-12-27 10:28:09,002][20347] Updated weights for policy 0, policy_version 699636 (0.0005) [2023-12-27 10:28:09,061][20347] Updated weights for policy 0, policy_version 699646 (0.0005) [2023-12-27 10:28:09,121][20347] Updated weights for policy 0, policy_version 699656 (0.0005) [2023-12-27 10:28:09,357][20381] Updated weights for policy 1, policy_version 699169 (0.0011) [2023-12-27 10:28:09,419][20381] Updated weights for policy 1, policy_version 699179 (0.0011) [2023-12-27 10:28:09,473][20381] Updated weights for policy 1, policy_version 699189 (0.0010) [2023-12-27 10:28:09,735][20347] Updated weights for policy 0, policy_version 699666 (0.0010) [2023-12-27 10:28:09,798][20347] Updated weights for policy 0, policy_version 699676 (0.0010) [2023-12-27 10:28:09,862][20347] Updated weights for policy 0, policy_version 699686 (0.0008) [2023-12-27 10:28:10,198][20381] Updated weights for policy 1, policy_version 699199 (0.0011) [2023-12-27 10:28:10,263][20381] Updated weights for policy 1, policy_version 699209 (0.0008) [2023-12-27 10:28:10,324][20381] Updated weights for policy 1, policy_version 699219 (0.0008) [2023-12-27 10:28:10,584][20347] Updated weights for policy 0, policy_version 699696 (0.0010) [2023-12-27 10:28:10,636][20347] Updated weights for policy 0, policy_version 699706 (0.0010) [2023-12-27 10:28:10,687][20347] Updated weights for policy 0, policy_version 699716 (0.0010) [2023-12-27 10:28:10,872][20381] Updated weights for policy 1, policy_version 699229 (0.0009) [2023-12-27 10:28:10,921][20381] Updated weights for policy 1, policy_version 699239 (0.0007) [2023-12-27 10:28:10,973][20381] Updated weights for policy 1, policy_version 699249 (0.0008) [2023-12-27 10:28:11,411][20347] Updated weights for policy 0, policy_version 699726 (0.0009) [2023-12-27 10:28:11,471][20347] Updated weights for policy 0, policy_version 699736 (0.0008) [2023-12-27 10:28:11,537][20347] Updated weights for policy 0, policy_version 699746 (0.0009) [2023-12-27 10:28:11,710][20381] Updated weights for policy 1, policy_version 699259 (0.0007) [2023-12-27 10:28:11,772][20381] Updated weights for policy 1, policy_version 699269 (0.0006) [2023-12-27 10:28:11,824][20381] Updated weights for policy 1, policy_version 699279 (0.0005) [2023-12-27 10:28:12,198][20347] Updated weights for policy 0, policy_version 699756 (0.0008) [2023-12-27 10:28:12,250][20347] Updated weights for policy 0, policy_version 699766 (0.0007) [2023-12-27 10:28:12,308][20347] Updated weights for policy 0, policy_version 699776 (0.0006) [2023-12-27 10:28:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 358203392. Throughput: 0: 10341.6, 1: 10380.3. Samples: 358199168. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:28:12,363][19170] Avg episode reward: [(0, '893.000'), (1, '990.280')] [2023-12-27 10:28:12,538][20381] Updated weights for policy 1, policy_version 699289 (0.0007) [2023-12-27 10:28:12,601][20381] Updated weights for policy 1, policy_version 699299 (0.0010) [2023-12-27 10:28:12,661][20381] Updated weights for policy 1, policy_version 699309 (0.0011) [2023-12-27 10:28:12,712][20381] Updated weights for policy 1, policy_version 699319 (0.0011) [2023-12-27 10:28:12,903][20347] Updated weights for policy 0, policy_version 699786 (0.0007) [2023-12-27 10:28:12,948][20347] Updated weights for policy 0, policy_version 699796 (0.0007) [2023-12-27 10:28:12,997][20347] Updated weights for policy 0, policy_version 699806 (0.0006) [2023-12-27 10:28:13,057][20347] Updated weights for policy 0, policy_version 699816 (0.0007) [2023-12-27 10:28:13,350][20381] Updated weights for policy 1, policy_version 699329 (0.0011) [2023-12-27 10:28:13,398][20381] Updated weights for policy 1, policy_version 699339 (0.0010) [2023-12-27 10:28:13,444][20381] Updated weights for policy 1, policy_version 699349 (0.0010) [2023-12-27 10:28:13,698][20347] Updated weights for policy 0, policy_version 699826 (0.0008) [2023-12-27 10:28:13,755][20347] Updated weights for policy 0, policy_version 699836 (0.0007) [2023-12-27 10:28:13,822][20347] Updated weights for policy 0, policy_version 699846 (0.0006) [2023-12-27 10:28:14,166][20381] Updated weights for policy 1, policy_version 699359 (0.0009) [2023-12-27 10:28:14,226][20381] Updated weights for policy 1, policy_version 699369 (0.0009) [2023-12-27 10:28:14,292][20381] Updated weights for policy 1, policy_version 699379 (0.0007) [2023-12-27 10:28:14,452][20347] Updated weights for policy 0, policy_version 699856 (0.0008) [2023-12-27 10:28:14,513][20347] Updated weights for policy 0, policy_version 699866 (0.0006) [2023-12-27 10:28:14,562][20347] Updated weights for policy 0, policy_version 699876 (0.0007) [2023-12-27 10:28:14,976][20381] Updated weights for policy 1, policy_version 699389 (0.0008) [2023-12-27 10:28:15,047][20381] Updated weights for policy 1, policy_version 699399 (0.0011) [2023-12-27 10:28:15,111][20381] Updated weights for policy 1, policy_version 699409 (0.0011) [2023-12-27 10:28:15,273][20347] Updated weights for policy 0, policy_version 699886 (0.0007) [2023-12-27 10:28:15,332][20347] Updated weights for policy 0, policy_version 699896 (0.0007) [2023-12-27 10:28:15,386][20347] Updated weights for policy 0, policy_version 699906 (0.0007) [2023-12-27 10:28:15,789][20381] Updated weights for policy 1, policy_version 699419 (0.0009) [2023-12-27 10:28:15,849][20381] Updated weights for policy 1, policy_version 699429 (0.0005) [2023-12-27 10:28:15,917][20381] Updated weights for policy 1, policy_version 699439 (0.0006) [2023-12-27 10:28:15,994][20347] Updated weights for policy 0, policy_version 699916 (0.0006) [2023-12-27 10:28:16,036][20347] Updated weights for policy 0, policy_version 699926 (0.0005) [2023-12-27 10:28:16,085][20347] Updated weights for policy 0, policy_version 699936 (0.0005) [2023-12-27 10:28:16,452][20381] Updated weights for policy 1, policy_version 699449 (0.0010) [2023-12-27 10:28:16,509][20381] Updated weights for policy 1, policy_version 699459 (0.0005) [2023-12-27 10:28:16,566][20381] Updated weights for policy 1, policy_version 699469 (0.0005) [2023-12-27 10:28:16,631][20381] Updated weights for policy 1, policy_version 699479 (0.0005) [2023-12-27 10:28:16,679][20347] Updated weights for policy 0, policy_version 699946 (0.0006) [2023-12-27 10:28:16,730][20347] Updated weights for policy 0, policy_version 699956 (0.0008) [2023-12-27 10:28:16,785][20347] Updated weights for policy 0, policy_version 699966 (0.0009) [2023-12-27 10:28:16,838][20347] Updated weights for policy 0, policy_version 699976 (0.0008) [2023-12-27 10:28:17,145][20381] Updated weights for policy 1, policy_version 699489 (0.0010) [2023-12-27 10:28:17,194][20381] Updated weights for policy 1, policy_version 699499 (0.0010) [2023-12-27 10:28:17,239][20381] Updated weights for policy 1, policy_version 699509 (0.0010) [2023-12-27 10:28:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 358318080. Throughput: 0: 10384.7, 1: 10372.1. Samples: 358326320. Policy #0 lag: (min: 27.0, avg: 34.0, max: 59.0) [2023-12-27 10:28:17,363][19170] Avg episode reward: [(0, '853.790'), (1, '951.600')] [2023-12-27 10:28:17,572][20347] Updated weights for policy 0, policy_version 699986 (0.0008) [2023-12-27 10:28:17,630][20347] Updated weights for policy 0, policy_version 699996 (0.0006) [2023-12-27 10:28:17,690][20347] Updated weights for policy 0, policy_version 700006 (0.0007) [2023-12-27 10:28:17,874][20381] Updated weights for policy 1, policy_version 699519 (0.0011) [2023-12-27 10:28:17,929][20381] Updated weights for policy 1, policy_version 699529 (0.0011) [2023-12-27 10:28:17,990][20381] Updated weights for policy 1, policy_version 699539 (0.0011) [2023-12-27 10:28:18,319][20347] Updated weights for policy 0, policy_version 700016 (0.0006) [2023-12-27 10:28:18,383][20347] Updated weights for policy 0, policy_version 700026 (0.0008) [2023-12-27 10:28:18,443][20347] Updated weights for policy 0, policy_version 700036 (0.0007) [2023-12-27 10:28:18,680][20381] Updated weights for policy 1, policy_version 699549 (0.0008) [2023-12-27 10:28:18,737][20381] Updated weights for policy 1, policy_version 699559 (0.0008) [2023-12-27 10:28:18,789][20381] Updated weights for policy 1, policy_version 699569 (0.0008) [2023-12-27 10:28:19,123][20347] Updated weights for policy 0, policy_version 700046 (0.0008) [2023-12-27 10:28:19,179][20347] Updated weights for policy 0, policy_version 700056 (0.0009) [2023-12-27 10:28:19,237][20347] Updated weights for policy 0, policy_version 700066 (0.0008) [2023-12-27 10:28:19,455][20381] Updated weights for policy 1, policy_version 699579 (0.0008) [2023-12-27 10:28:19,518][20381] Updated weights for policy 1, policy_version 699589 (0.0007) [2023-12-27 10:28:19,576][20381] Updated weights for policy 1, policy_version 699599 (0.0007) [2023-12-27 10:28:19,944][20347] Updated weights for policy 0, policy_version 700076 (0.0008) [2023-12-27 10:28:20,000][20347] Updated weights for policy 0, policy_version 700086 (0.0007) [2023-12-27 10:28:20,059][20347] Updated weights for policy 0, policy_version 700096 (0.0008) [2023-12-27 10:28:20,171][20381] Updated weights for policy 1, policy_version 699609 (0.0006) [2023-12-27 10:28:20,236][20381] Updated weights for policy 1, policy_version 699619 (0.0006) [2023-12-27 10:28:20,297][20381] Updated weights for policy 1, policy_version 699629 (0.0006) [2023-12-27 10:28:20,355][20381] Updated weights for policy 1, policy_version 699639 (0.0007) [2023-12-27 10:28:20,777][20347] Updated weights for policy 0, policy_version 700106 (0.0008) [2023-12-27 10:28:20,840][20347] Updated weights for policy 0, policy_version 700116 (0.0008) [2023-12-27 10:28:20,910][20347] Updated weights for policy 0, policy_version 700126 (0.0006) [2023-12-27 10:28:20,963][20381] Updated weights for policy 1, policy_version 699649 (0.0007) [2023-12-27 10:28:20,969][20347] Updated weights for policy 0, policy_version 700136 (0.0006) [2023-12-27 10:28:21,030][20381] Updated weights for policy 1, policy_version 699659 (0.0008) [2023-12-27 10:28:21,093][20381] Updated weights for policy 1, policy_version 699669 (0.0008) [2023-12-27 10:28:21,650][20347] Updated weights for policy 0, policy_version 700146 (0.0008) [2023-12-27 10:28:21,717][20347] Updated weights for policy 0, policy_version 700156 (0.0007) [2023-12-27 10:28:21,782][20347] Updated weights for policy 0, policy_version 700166 (0.0007) [2023-12-27 10:28:21,789][20381] Updated weights for policy 1, policy_version 699679 (0.0008) [2023-12-27 10:28:21,842][20381] Updated weights for policy 1, policy_version 699689 (0.0008) [2023-12-27 10:28:21,903][20381] Updated weights for policy 1, policy_version 699699 (0.0008) [2023-12-27 10:28:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 358416384. Throughput: 0: 10392.1, 1: 10368.8. Samples: 358389348. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:22,363][19170] Avg episode reward: [(0, '844.010'), (1, '971.060')] [2023-12-27 10:28:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000699704_179150848.pth... [2023-12-27 10:28:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000700168_179265536.pth... [2023-12-27 10:28:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000698488_178839552.pth [2023-12-27 10:28:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000698952_178954240.pth [2023-12-27 10:28:22,439][20347] Updated weights for policy 0, policy_version 700176 (0.0007) [2023-12-27 10:28:22,503][20347] Updated weights for policy 0, policy_version 700186 (0.0008) [2023-12-27 10:28:22,561][20347] Updated weights for policy 0, policy_version 700196 (0.0007) [2023-12-27 10:28:22,578][20381] Updated weights for policy 1, policy_version 699709 (0.0008) [2023-12-27 10:28:22,638][20381] Updated weights for policy 1, policy_version 699719 (0.0008) [2023-12-27 10:28:22,694][20381] Updated weights for policy 1, policy_version 699729 (0.0007) [2023-12-27 10:28:23,182][20347] Updated weights for policy 0, policy_version 700206 (0.0008) [2023-12-27 10:28:23,242][20347] Updated weights for policy 0, policy_version 700216 (0.0010) [2023-12-27 10:28:23,296][20347] Updated weights for policy 0, policy_version 700226 (0.0010) [2023-12-27 10:28:23,454][20381] Updated weights for policy 1, policy_version 699739 (0.0008) [2023-12-27 10:28:23,510][20381] Updated weights for policy 1, policy_version 699749 (0.0008) [2023-12-27 10:28:23,570][20381] Updated weights for policy 1, policy_version 699759 (0.0007) [2023-12-27 10:28:23,852][20347] Updated weights for policy 0, policy_version 700236 (0.0010) [2023-12-27 10:28:23,904][20347] Updated weights for policy 0, policy_version 700246 (0.0008) [2023-12-27 10:28:23,965][20347] Updated weights for policy 0, policy_version 700256 (0.0009) [2023-12-27 10:28:24,308][20381] Updated weights for policy 1, policy_version 699769 (0.0008) [2023-12-27 10:28:24,366][20381] Updated weights for policy 1, policy_version 699779 (0.0007) [2023-12-27 10:28:24,419][20381] Updated weights for policy 1, policy_version 699789 (0.0007) [2023-12-27 10:28:24,467][20381] Updated weights for policy 1, policy_version 699799 (0.0008) [2023-12-27 10:28:24,681][20347] Updated weights for policy 0, policy_version 700266 (0.0010) [2023-12-27 10:28:24,726][20347] Updated weights for policy 0, policy_version 700276 (0.0010) [2023-12-27 10:28:24,771][20347] Updated weights for policy 0, policy_version 700286 (0.0010) [2023-12-27 10:28:24,826][20347] Updated weights for policy 0, policy_version 700296 (0.0010) [2023-12-27 10:28:25,136][20381] Updated weights for policy 1, policy_version 699809 (0.0005) [2023-12-27 10:28:25,185][20381] Updated weights for policy 1, policy_version 699819 (0.0008) [2023-12-27 10:28:25,234][20381] Updated weights for policy 1, policy_version 699829 (0.0010) [2023-12-27 10:28:25,530][20347] Updated weights for policy 0, policy_version 700306 (0.0005) [2023-12-27 10:28:25,564][20291] KL-divergence is very high: 317.2861 [2023-12-27 10:28:25,587][20347] Updated weights for policy 0, policy_version 700316 (0.0005) [2023-12-27 10:28:25,611][20291] KL-divergence is very high: 311.3563 [2023-12-27 10:28:25,642][20347] Updated weights for policy 0, policy_version 700326 (0.0005) [2023-12-27 10:28:25,948][20381] Updated weights for policy 1, policy_version 699839 (0.0007) [2023-12-27 10:28:26,002][20381] Updated weights for policy 1, policy_version 699849 (0.0005) [2023-12-27 10:28:26,057][20381] Updated weights for policy 1, policy_version 699859 (0.0007) [2023-12-27 10:28:26,156][20347] Updated weights for policy 0, policy_version 700336 (0.0006) [2023-12-27 10:28:26,204][20347] Updated weights for policy 0, policy_version 700346 (0.0007) [2023-12-27 10:28:26,257][20347] Updated weights for policy 0, policy_version 700356 (0.0005) [2023-12-27 10:28:26,690][20381] Updated weights for policy 1, policy_version 699869 (0.0008) [2023-12-27 10:28:26,747][20381] Updated weights for policy 1, policy_version 699879 (0.0010) [2023-12-27 10:28:26,806][20381] Updated weights for policy 1, policy_version 699889 (0.0008) [2023-12-27 10:28:26,868][20347] Updated weights for policy 0, policy_version 700366 (0.0006) [2023-12-27 10:28:26,928][20347] Updated weights for policy 0, policy_version 700376 (0.0005) [2023-12-27 10:28:26,982][20347] Updated weights for policy 0, policy_version 700386 (0.0005) [2023-12-27 10:28:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 358522880. Throughput: 0: 10442.0, 1: 10411.9. Samples: 358513204. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:27,363][19170] Avg episode reward: [(0, '951.090'), (1, '1000.000')] [2023-12-27 10:28:27,480][20381] Updated weights for policy 1, policy_version 699899 (0.0007) [2023-12-27 10:28:27,530][20381] Updated weights for policy 1, policy_version 699909 (0.0005) [2023-12-27 10:28:27,560][20347] Updated weights for policy 0, policy_version 700396 (0.0006) [2023-12-27 10:28:27,587][20381] Updated weights for policy 1, policy_version 699919 (0.0006) [2023-12-27 10:28:27,608][20347] Updated weights for policy 0, policy_version 700406 (0.0006) [2023-12-27 10:28:27,661][20347] Updated weights for policy 0, policy_version 700416 (0.0006) [2023-12-27 10:28:28,206][20381] Updated weights for policy 1, policy_version 699929 (0.0006) [2023-12-27 10:28:28,267][20381] Updated weights for policy 1, policy_version 699939 (0.0007) [2023-12-27 10:28:28,325][20381] Updated weights for policy 1, policy_version 699949 (0.0008) [2023-12-27 10:28:28,367][20347] Updated weights for policy 0, policy_version 700426 (0.0008) [2023-12-27 10:28:28,388][20381] Updated weights for policy 1, policy_version 699959 (0.0006) [2023-12-27 10:28:28,420][20347] Updated weights for policy 0, policy_version 700436 (0.0011) [2023-12-27 10:28:28,448][20291] KL-divergence is very high: 124.4090 [2023-12-27 10:28:28,477][20347] Updated weights for policy 0, policy_version 700446 (0.0010) [2023-12-27 10:28:28,541][20347] Updated weights for policy 0, policy_version 700456 (0.0010) [2023-12-27 10:28:29,129][20381] Updated weights for policy 1, policy_version 699969 (0.0008) [2023-12-27 10:28:29,148][20347] Updated weights for policy 0, policy_version 700466 (0.0008) [2023-12-27 10:28:29,184][20381] Updated weights for policy 1, policy_version 699979 (0.0008) [2023-12-27 10:28:29,214][20347] Updated weights for policy 0, policy_version 700476 (0.0008) [2023-12-27 10:28:29,243][20381] Updated weights for policy 1, policy_version 699989 (0.0008) [2023-12-27 10:28:29,281][20347] Updated weights for policy 0, policy_version 700486 (0.0009) [2023-12-27 10:28:29,861][20347] Updated weights for policy 0, policy_version 700496 (0.0007) [2023-12-27 10:28:29,917][20347] Updated weights for policy 0, policy_version 700506 (0.0007) [2023-12-27 10:28:29,972][20347] Updated weights for policy 0, policy_version 700516 (0.0007) [2023-12-27 10:28:29,981][20381] Updated weights for policy 1, policy_version 699999 (0.0007) [2023-12-27 10:28:30,037][20381] Updated weights for policy 1, policy_version 700009 (0.0007) [2023-12-27 10:28:30,095][20381] Updated weights for policy 1, policy_version 700019 (0.0008) [2023-12-27 10:28:30,680][20347] Updated weights for policy 0, policy_version 700526 (0.0006) [2023-12-27 10:28:30,702][20381] Updated weights for policy 1, policy_version 700029 (0.0007) [2023-12-27 10:28:30,736][20347] Updated weights for policy 0, policy_version 700536 (0.0005) [2023-12-27 10:28:30,757][20381] Updated weights for policy 1, policy_version 700039 (0.0007) [2023-12-27 10:28:30,794][20347] Updated weights for policy 0, policy_version 700546 (0.0005) [2023-12-27 10:28:30,813][20381] Updated weights for policy 1, policy_version 700049 (0.0007) [2023-12-27 10:28:31,348][20347] Updated weights for policy 0, policy_version 700556 (0.0007) [2023-12-27 10:28:31,406][20347] Updated weights for policy 0, policy_version 700566 (0.0008) [2023-12-27 10:28:31,452][20347] Updated weights for policy 0, policy_version 700576 (0.0007) [2023-12-27 10:28:31,512][20381] Updated weights for policy 1, policy_version 700059 (0.0008) [2023-12-27 10:28:31,569][20381] Updated weights for policy 1, policy_version 700069 (0.0006) [2023-12-27 10:28:31,631][20381] Updated weights for policy 1, policy_version 700079 (0.0007) [2023-12-27 10:28:32,179][20347] Updated weights for policy 0, policy_version 700586 (0.0008) [2023-12-27 10:28:32,239][20347] Updated weights for policy 0, policy_version 700596 (0.0007) [2023-12-27 10:28:32,300][20381] Updated weights for policy 1, policy_version 700089 (0.0009) [2023-12-27 10:28:32,301][20347] Updated weights for policy 0, policy_version 700606 (0.0007) [2023-12-27 10:28:32,358][20381] Updated weights for policy 1, policy_version 700099 (0.0007) [2023-12-27 10:28:32,362][20347] Updated weights for policy 0, policy_version 700616 (0.0007) [2023-12-27 10:28:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 358629376. Throughput: 0: 10511.1, 1: 10403.7. Samples: 358639452. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:32,363][19170] Avg episode reward: [(0, '800.710'), (1, '990.310')] [2023-12-27 10:28:32,417][20381] Updated weights for policy 1, policy_version 700109 (0.0008) [2023-12-27 10:28:32,478][20381] Updated weights for policy 1, policy_version 700119 (0.0008) [2023-12-27 10:28:32,961][20347] Updated weights for policy 0, policy_version 700626 (0.0007) [2023-12-27 10:28:33,012][20347] Updated weights for policy 0, policy_version 700636 (0.0007) [2023-12-27 10:28:33,070][20347] Updated weights for policy 0, policy_version 700646 (0.0007) [2023-12-27 10:28:33,143][20381] Updated weights for policy 1, policy_version 700129 (0.0008) [2023-12-27 10:28:33,202][20381] Updated weights for policy 1, policy_version 700139 (0.0008) [2023-12-27 10:28:33,260][20381] Updated weights for policy 1, policy_version 700149 (0.0007) [2023-12-27 10:28:33,693][20347] Updated weights for policy 0, policy_version 700657 (0.0006) [2023-12-27 10:28:33,739][20347] Updated weights for policy 0, policy_version 700667 (0.0005) [2023-12-27 10:28:33,790][20347] Updated weights for policy 0, policy_version 700677 (0.0005) [2023-12-27 10:28:33,967][20381] Updated weights for policy 1, policy_version 700159 (0.0009) [2023-12-27 10:28:34,017][20381] Updated weights for policy 1, policy_version 700169 (0.0010) [2023-12-27 10:28:34,086][20381] Updated weights for policy 1, policy_version 700179 (0.0008) [2023-12-27 10:28:34,381][20347] Updated weights for policy 0, policy_version 700687 (0.0006) [2023-12-27 10:28:34,442][20347] Updated weights for policy 0, policy_version 700697 (0.0005) [2023-12-27 10:28:34,493][20347] Updated weights for policy 0, policy_version 700707 (0.0006) [2023-12-27 10:28:34,791][20381] Updated weights for policy 1, policy_version 700189 (0.0007) [2023-12-27 10:28:34,822][20292] KL-divergence is very high: 130.3554 [2023-12-27 10:28:34,843][20381] Updated weights for policy 1, policy_version 700199 (0.0005) [2023-12-27 10:28:34,849][20292] KL-divergence is very high: 139.8154 [2023-12-27 10:28:34,904][20381] Updated weights for policy 1, policy_version 700209 (0.0005) [2023-12-27 10:28:35,127][20347] Updated weights for policy 0, policy_version 700717 (0.0006) [2023-12-27 10:28:35,180][20347] Updated weights for policy 0, policy_version 700727 (0.0008) [2023-12-27 10:28:35,239][20347] Updated weights for policy 0, policy_version 700737 (0.0008) [2023-12-27 10:28:35,530][20381] Updated weights for policy 1, policy_version 700219 (0.0007) [2023-12-27 10:28:35,586][20381] Updated weights for policy 1, policy_version 700229 (0.0008) [2023-12-27 10:28:35,648][20381] Updated weights for policy 1, policy_version 700239 (0.0009) [2023-12-27 10:28:35,868][20347] Updated weights for policy 0, policy_version 700747 (0.0007) [2023-12-27 10:28:35,921][20347] Updated weights for policy 0, policy_version 700757 (0.0005) [2023-12-27 10:28:35,976][20347] Updated weights for policy 0, policy_version 700767 (0.0005) [2023-12-27 10:28:36,301][20381] Updated weights for policy 1, policy_version 700249 (0.0010) [2023-12-27 10:28:36,357][20381] Updated weights for policy 1, policy_version 700259 (0.0008) [2023-12-27 10:28:36,415][20381] Updated weights for policy 1, policy_version 700269 (0.0007) [2023-12-27 10:28:36,476][20381] Updated weights for policy 1, policy_version 700279 (0.0006) [2023-12-27 10:28:36,684][20347] Updated weights for policy 0, policy_version 700777 (0.0008) [2023-12-27 10:28:36,744][20347] Updated weights for policy 0, policy_version 700787 (0.0009) [2023-12-27 10:28:36,800][20347] Updated weights for policy 0, policy_version 700797 (0.0008) [2023-12-27 10:28:36,860][20347] Updated weights for policy 0, policy_version 700807 (0.0006) [2023-12-27 10:28:37,038][20381] Updated weights for policy 1, policy_version 700289 (0.0008) [2023-12-27 10:28:37,095][20381] Updated weights for policy 1, policy_version 700299 (0.0008) [2023-12-27 10:28:37,147][20381] Updated weights for policy 1, policy_version 700309 (0.0007) [2023-12-27 10:28:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 358735872. Throughput: 0: 10603.2, 1: 10320.7. Samples: 358702948. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:37,363][19170] Avg episode reward: [(0, '404.530'), (1, '941.900')] [2023-12-27 10:28:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000700312_179306496.pth... [2023-12-27 10:28:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000700808_179429376.pth... [2023-12-27 10:28:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000699096_178995200.pth [2023-12-27 10:28:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000699560_179109888.pth [2023-12-27 10:28:37,556][20347] Updated weights for policy 0, policy_version 700817 (0.0008) [2023-12-27 10:28:37,616][20347] Updated weights for policy 0, policy_version 700827 (0.0008) [2023-12-27 10:28:37,671][20347] Updated weights for policy 0, policy_version 700837 (0.0007) [2023-12-27 10:28:37,850][20381] Updated weights for policy 1, policy_version 700319 (0.0008) [2023-12-27 10:28:37,902][20381] Updated weights for policy 1, policy_version 700329 (0.0007) [2023-12-27 10:28:37,958][20381] Updated weights for policy 1, policy_version 700339 (0.0008) [2023-12-27 10:28:38,297][20347] Updated weights for policy 0, policy_version 700847 (0.0009) [2023-12-27 10:28:38,363][20347] Updated weights for policy 0, policy_version 700857 (0.0010) [2023-12-27 10:28:38,416][20347] Updated weights for policy 0, policy_version 700867 (0.0010) [2023-12-27 10:28:38,673][20381] Updated weights for policy 1, policy_version 700349 (0.0010) [2023-12-27 10:28:38,726][20381] Updated weights for policy 1, policy_version 700359 (0.0011) [2023-12-27 10:28:38,783][20381] Updated weights for policy 1, policy_version 700369 (0.0011) [2023-12-27 10:28:39,052][20347] Updated weights for policy 0, policy_version 700877 (0.0010) [2023-12-27 10:28:39,110][20347] Updated weights for policy 0, policy_version 700887 (0.0010) [2023-12-27 10:28:39,167][20347] Updated weights for policy 0, policy_version 700897 (0.0010) [2023-12-27 10:28:39,506][20381] Updated weights for policy 1, policy_version 700379 (0.0010) [2023-12-27 10:28:39,558][20381] Updated weights for policy 1, policy_version 700389 (0.0008) [2023-12-27 10:28:39,613][20381] Updated weights for policy 1, policy_version 700399 (0.0009) [2023-12-27 10:28:39,876][20347] Updated weights for policy 0, policy_version 700907 (0.0009) [2023-12-27 10:28:39,929][20347] Updated weights for policy 0, policy_version 700917 (0.0010) [2023-12-27 10:28:39,995][20347] Updated weights for policy 0, policy_version 700927 (0.0011) [2023-12-27 10:28:40,336][20381] Updated weights for policy 1, policy_version 700409 (0.0010) [2023-12-27 10:28:40,406][20381] Updated weights for policy 1, policy_version 700419 (0.0006) [2023-12-27 10:28:40,475][20381] Updated weights for policy 1, policy_version 700429 (0.0005) [2023-12-27 10:28:40,536][20381] Updated weights for policy 1, policy_version 700439 (0.0006) [2023-12-27 10:28:40,682][20347] Updated weights for policy 0, policy_version 700937 (0.0011) [2023-12-27 10:28:40,744][20347] Updated weights for policy 0, policy_version 700947 (0.0011) [2023-12-27 10:28:40,810][20347] Updated weights for policy 0, policy_version 700957 (0.0011) [2023-12-27 10:28:40,878][20347] Updated weights for policy 0, policy_version 700967 (0.0008) [2023-12-27 10:28:41,077][20381] Updated weights for policy 1, policy_version 700449 (0.0010) [2023-12-27 10:28:41,144][20381] Updated weights for policy 1, policy_version 700459 (0.0011) [2023-12-27 10:28:41,204][20381] Updated weights for policy 1, policy_version 700469 (0.0008) [2023-12-27 10:28:41,566][20347] Updated weights for policy 0, policy_version 700977 (0.0008) [2023-12-27 10:28:41,627][20347] Updated weights for policy 0, policy_version 700987 (0.0009) [2023-12-27 10:28:41,694][20347] Updated weights for policy 0, policy_version 700997 (0.0011) [2023-12-27 10:28:41,926][20381] Updated weights for policy 1, policy_version 700479 (0.0008) [2023-12-27 10:28:41,968][20292] KL-divergence is very high: 129.8296 [2023-12-27 10:28:41,973][20381] Updated weights for policy 1, policy_version 700489 (0.0007) [2023-12-27 10:28:42,005][20292] KL-divergence is very high: 264.0408 [2023-12-27 10:28:42,022][20381] Updated weights for policy 1, policy_version 700499 (0.0006) [2023-12-27 10:28:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20521.6). Total num frames: 358834176. Throughput: 0: 10583.3, 1: 10320.6. Samples: 358827104. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:42,363][19170] Avg episode reward: [(0, '369.720'), (1, '942.770')] [2023-12-27 10:28:42,434][20347] Updated weights for policy 0, policy_version 701007 (0.0010) [2023-12-27 10:28:42,486][20347] Updated weights for policy 0, policy_version 701017 (0.0009) [2023-12-27 10:28:42,544][20347] Updated weights for policy 0, policy_version 701027 (0.0007) [2023-12-27 10:28:42,664][20381] Updated weights for policy 1, policy_version 700509 (0.0009) [2023-12-27 10:28:42,725][20381] Updated weights for policy 1, policy_version 700519 (0.0008) [2023-12-27 10:28:42,790][20381] Updated weights for policy 1, policy_version 700529 (0.0009) [2023-12-27 10:28:43,183][20347] Updated weights for policy 0, policy_version 701037 (0.0006) [2023-12-27 10:28:43,229][20347] Updated weights for policy 0, policy_version 701047 (0.0006) [2023-12-27 10:28:43,280][20347] Updated weights for policy 0, policy_version 701057 (0.0005) [2023-12-27 10:28:43,456][20381] Updated weights for policy 1, policy_version 700539 (0.0009) [2023-12-27 10:28:43,517][20381] Updated weights for policy 1, policy_version 700549 (0.0005) [2023-12-27 10:28:43,578][20381] Updated weights for policy 1, policy_version 700559 (0.0006) [2023-12-27 10:28:43,932][20347] Updated weights for policy 0, policy_version 701067 (0.0009) [2023-12-27 10:28:43,995][20347] Updated weights for policy 0, policy_version 701077 (0.0006) [2023-12-27 10:28:44,055][20347] Updated weights for policy 0, policy_version 701087 (0.0006) [2023-12-27 10:28:44,228][20381] Updated weights for policy 1, policy_version 700569 (0.0011) [2023-12-27 10:28:44,291][20381] Updated weights for policy 1, policy_version 700579 (0.0011) [2023-12-27 10:28:44,357][20381] Updated weights for policy 1, policy_version 700589 (0.0011) [2023-12-27 10:28:44,417][20381] Updated weights for policy 1, policy_version 700599 (0.0007) [2023-12-27 10:28:44,744][20347] Updated weights for policy 0, policy_version 701097 (0.0007) [2023-12-27 10:28:44,808][20347] Updated weights for policy 0, policy_version 701107 (0.0007) [2023-12-27 10:28:44,870][20347] Updated weights for policy 0, policy_version 701117 (0.0007) [2023-12-27 10:28:44,929][20347] Updated weights for policy 0, policy_version 701127 (0.0007) [2023-12-27 10:28:45,086][20381] Updated weights for policy 1, policy_version 700609 (0.0009) [2023-12-27 10:28:45,141][20381] Updated weights for policy 1, policy_version 700619 (0.0009) [2023-12-27 10:28:45,205][20381] Updated weights for policy 1, policy_version 700629 (0.0008) [2023-12-27 10:28:45,632][20347] Updated weights for policy 0, policy_version 701137 (0.0007) [2023-12-27 10:28:45,680][20347] Updated weights for policy 0, policy_version 701147 (0.0007) [2023-12-27 10:28:45,732][20347] Updated weights for policy 0, policy_version 701157 (0.0007) [2023-12-27 10:28:45,933][20381] Updated weights for policy 1, policy_version 700639 (0.0006) [2023-12-27 10:28:45,988][20381] Updated weights for policy 1, policy_version 700649 (0.0005) [2023-12-27 10:28:46,040][20381] Updated weights for policy 1, policy_version 700659 (0.0007) [2023-12-27 10:28:46,487][20347] Updated weights for policy 0, policy_version 701167 (0.0007) [2023-12-27 10:28:46,541][20347] Updated weights for policy 0, policy_version 701177 (0.0007) [2023-12-27 10:28:46,586][20381] Updated weights for policy 1, policy_version 700669 (0.0006) [2023-12-27 10:28:46,591][20347] Updated weights for policy 0, policy_version 701187 (0.0008) [2023-12-27 10:28:46,623][20292] KL-divergence is very high: 268.5560 [2023-12-27 10:28:46,643][20381] Updated weights for policy 1, policy_version 700679 (0.0006) [2023-12-27 10:28:46,692][20381] Updated weights for policy 1, policy_version 700689 (0.0007) [2023-12-27 10:28:47,177][20347] Updated weights for policy 0, policy_version 701197 (0.0007) [2023-12-27 10:28:47,223][20347] Updated weights for policy 0, policy_version 701207 (0.0007) [2023-12-27 10:28:47,274][20347] Updated weights for policy 0, policy_version 701217 (0.0008) [2023-12-27 10:28:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20521.7). Total num frames: 358940672. Throughput: 0: 10547.0, 1: 10342.4. Samples: 358949672. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:47,363][19170] Avg episode reward: [(0, '121.710'), (1, '963.230')] [2023-12-27 10:28:47,440][20381] Updated weights for policy 1, policy_version 700699 (0.0007) [2023-12-27 10:28:47,503][20381] Updated weights for policy 1, policy_version 700709 (0.0008) [2023-12-27 10:28:47,559][20381] Updated weights for policy 1, policy_version 700719 (0.0008) [2023-12-27 10:28:47,974][20347] Updated weights for policy 0, policy_version 701227 (0.0008) [2023-12-27 10:28:48,030][20347] Updated weights for policy 0, policy_version 701237 (0.0007) [2023-12-27 10:28:48,088][20347] Updated weights for policy 0, policy_version 701247 (0.0007) [2023-12-27 10:28:48,254][20381] Updated weights for policy 1, policy_version 700729 (0.0007) [2023-12-27 10:28:48,310][20381] Updated weights for policy 1, policy_version 700739 (0.0008) [2023-12-27 10:28:48,373][20381] Updated weights for policy 1, policy_version 700749 (0.0007) [2023-12-27 10:28:48,430][20381] Updated weights for policy 1, policy_version 700759 (0.0007) [2023-12-27 10:28:48,778][20347] Updated weights for policy 0, policy_version 701257 (0.0008) [2023-12-27 10:28:48,832][20347] Updated weights for policy 0, policy_version 701267 (0.0008) [2023-12-27 10:28:48,890][20347] Updated weights for policy 0, policy_version 701277 (0.0007) [2023-12-27 10:28:48,945][20347] Updated weights for policy 0, policy_version 701287 (0.0008) [2023-12-27 10:28:49,106][20381] Updated weights for policy 1, policy_version 700769 (0.0007) [2023-12-27 10:28:49,152][20381] Updated weights for policy 1, policy_version 700779 (0.0008) [2023-12-27 10:28:49,201][20381] Updated weights for policy 1, policy_version 700789 (0.0007) [2023-12-27 10:28:49,631][20347] Updated weights for policy 0, policy_version 701297 (0.0010) [2023-12-27 10:28:49,677][20347] Updated weights for policy 0, policy_version 701307 (0.0010) [2023-12-27 10:28:49,724][20347] Updated weights for policy 0, policy_version 701317 (0.0010) [2023-12-27 10:28:49,967][20381] Updated weights for policy 1, policy_version 700799 (0.0007) [2023-12-27 10:28:50,024][20381] Updated weights for policy 1, policy_version 700809 (0.0005) [2023-12-27 10:28:50,087][20381] Updated weights for policy 1, policy_version 700819 (0.0007) [2023-12-27 10:28:50,410][20347] Updated weights for policy 0, policy_version 701327 (0.0011) [2023-12-27 10:28:50,466][20347] Updated weights for policy 0, policy_version 701337 (0.0009) [2023-12-27 10:28:50,526][20347] Updated weights for policy 0, policy_version 701347 (0.0011) [2023-12-27 10:28:50,766][20381] Updated weights for policy 1, policy_version 700829 (0.0008) [2023-12-27 10:28:50,825][20381] Updated weights for policy 1, policy_version 700839 (0.0007) [2023-12-27 10:28:50,887][20381] Updated weights for policy 1, policy_version 700849 (0.0006) [2023-12-27 10:28:51,210][20347] Updated weights for policy 0, policy_version 701357 (0.0011) [2023-12-27 10:28:51,273][20347] Updated weights for policy 0, policy_version 701367 (0.0010) [2023-12-27 10:28:51,332][20347] Updated weights for policy 0, policy_version 701377 (0.0010) [2023-12-27 10:28:51,580][20381] Updated weights for policy 1, policy_version 700859 (0.0007) [2023-12-27 10:28:51,647][20381] Updated weights for policy 1, policy_version 700869 (0.0011) [2023-12-27 10:28:51,706][20381] Updated weights for policy 1, policy_version 700879 (0.0011) [2023-12-27 10:28:52,030][20347] Updated weights for policy 0, policy_version 701387 (0.0008) [2023-12-27 10:28:52,081][20347] Updated weights for policy 0, policy_version 701397 (0.0006) [2023-12-27 10:28:52,135][20347] Updated weights for policy 0, policy_version 701407 (0.0005) [2023-12-27 10:28:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 359038976. Throughput: 0: 10498.6, 1: 10289.5. Samples: 359009440. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:52,363][19170] Avg episode reward: [(0, '150.540'), (1, '972.050')] [2023-12-27 10:28:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000701416_179585024.pth... [2023-12-27 10:28:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000700168_179265536.pth [2023-12-27 10:28:52,386][20381] Updated weights for policy 1, policy_version 700889 (0.0008) [2023-12-27 10:28:52,452][20381] Updated weights for policy 1, policy_version 700899 (0.0007) [2023-12-27 10:28:52,516][20381] Updated weights for policy 1, policy_version 700909 (0.0008) [2023-12-27 10:28:52,587][20381] Updated weights for policy 1, policy_version 700919 (0.0008) [2023-12-27 10:28:52,591][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000700920_179462144.pth... [2023-12-27 10:28:52,596][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000699704_179150848.pth [2023-12-27 10:28:52,762][20347] Updated weights for policy 0, policy_version 701417 (0.0007) [2023-12-27 10:28:52,819][20347] Updated weights for policy 0, policy_version 701427 (0.0007) [2023-12-27 10:28:52,871][20347] Updated weights for policy 0, policy_version 701437 (0.0008) [2023-12-27 10:28:52,920][20347] Updated weights for policy 0, policy_version 701447 (0.0009) [2023-12-27 10:28:53,198][20381] Updated weights for policy 1, policy_version 700929 (0.0010) [2023-12-27 10:28:53,253][20381] Updated weights for policy 1, policy_version 700939 (0.0010) [2023-12-27 10:28:53,302][20381] Updated weights for policy 1, policy_version 700949 (0.0010) [2023-12-27 10:28:53,583][20347] Updated weights for policy 0, policy_version 701457 (0.0010) [2023-12-27 10:28:53,628][20347] Updated weights for policy 0, policy_version 701467 (0.0009) [2023-12-27 10:28:53,680][20347] Updated weights for policy 0, policy_version 701477 (0.0006) [2023-12-27 10:28:53,959][20381] Updated weights for policy 1, policy_version 700959 (0.0011) [2023-12-27 10:28:54,014][20381] Updated weights for policy 1, policy_version 700969 (0.0011) [2023-12-27 10:28:54,067][20381] Updated weights for policy 1, policy_version 700979 (0.0010) [2023-12-27 10:28:54,343][20347] Updated weights for policy 0, policy_version 701487 (0.0008) [2023-12-27 10:28:54,398][20347] Updated weights for policy 0, policy_version 701497 (0.0010) [2023-12-27 10:28:54,460][20347] Updated weights for policy 0, policy_version 701507 (0.0010) [2023-12-27 10:28:54,714][20381] Updated weights for policy 1, policy_version 700989 (0.0006) [2023-12-27 10:28:54,773][20381] Updated weights for policy 1, policy_version 700999 (0.0006) [2023-12-27 10:28:54,832][20381] Updated weights for policy 1, policy_version 701009 (0.0005) [2023-12-27 10:28:55,059][20347] Updated weights for policy 0, policy_version 701517 (0.0010) [2023-12-27 10:28:55,110][20347] Updated weights for policy 0, policy_version 701527 (0.0010) [2023-12-27 10:28:55,169][20347] Updated weights for policy 0, policy_version 701537 (0.0007) [2023-12-27 10:28:55,401][20381] Updated weights for policy 1, policy_version 701019 (0.0007) [2023-12-27 10:28:55,450][20381] Updated weights for policy 1, policy_version 701029 (0.0010) [2023-12-27 10:28:55,501][20381] Updated weights for policy 1, policy_version 701039 (0.0010) [2023-12-27 10:28:55,724][20347] Updated weights for policy 0, policy_version 701547 (0.0009) [2023-12-27 10:28:55,784][20347] Updated weights for policy 0, policy_version 701557 (0.0008) [2023-12-27 10:28:55,836][20347] Updated weights for policy 0, policy_version 701567 (0.0010) [2023-12-27 10:28:56,188][20381] Updated weights for policy 1, policy_version 701049 (0.0011) [2023-12-27 10:28:56,237][20381] Updated weights for policy 1, policy_version 701059 (0.0010) [2023-12-27 10:28:56,292][20381] Updated weights for policy 1, policy_version 701069 (0.0010) [2023-12-27 10:28:56,346][20381] Updated weights for policy 1, policy_version 701079 (0.0010) [2023-12-27 10:28:56,464][20347] Updated weights for policy 0, policy_version 701577 (0.0010) [2023-12-27 10:28:56,524][20347] Updated weights for policy 0, policy_version 701587 (0.0006) [2023-12-27 10:28:56,580][20347] Updated weights for policy 0, policy_version 701597 (0.0005) [2023-12-27 10:28:56,634][20347] Updated weights for policy 0, policy_version 701607 (0.0006) [2023-12-27 10:28:56,978][20381] Updated weights for policy 1, policy_version 701089 (0.0006) [2023-12-27 10:28:57,028][20381] Updated weights for policy 1, policy_version 701099 (0.0005) [2023-12-27 10:28:57,081][20381] Updated weights for policy 1, policy_version 701109 (0.0010) [2023-12-27 10:28:57,220][20347] Updated weights for policy 0, policy_version 701617 (0.0006) [2023-12-27 10:28:57,279][20347] Updated weights for policy 0, policy_version 701627 (0.0005) [2023-12-27 10:28:57,338][20347] Updated weights for policy 0, policy_version 701637 (0.0008) [2023-12-27 10:28:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.2, 300 sec: 20521.7). Total num frames: 359153664. Throughput: 0: 10519.7, 1: 10332.3. Samples: 359137504. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:28:57,363][19170] Avg episode reward: [(0, '209.090'), (1, '913.210')] [2023-12-27 10:28:57,693][20381] Updated weights for policy 1, policy_version 701119 (0.0009) [2023-12-27 10:28:57,752][20381] Updated weights for policy 1, policy_version 701129 (0.0005) [2023-12-27 10:28:57,805][20381] Updated weights for policy 1, policy_version 701139 (0.0005) [2023-12-27 10:28:57,918][20347] Updated weights for policy 0, policy_version 701647 (0.0007) [2023-12-27 10:28:57,965][20347] Updated weights for policy 0, policy_version 701657 (0.0005) [2023-12-27 10:28:58,013][20347] Updated weights for policy 0, policy_version 701667 (0.0005) [2023-12-27 10:28:58,463][20381] Updated weights for policy 1, policy_version 701149 (0.0006) [2023-12-27 10:28:58,520][20381] Updated weights for policy 1, policy_version 701159 (0.0007) [2023-12-27 10:28:58,578][20381] Updated weights for policy 1, policy_version 701169 (0.0008) [2023-12-27 10:28:58,692][20347] Updated weights for policy 0, policy_version 701677 (0.0007) [2023-12-27 10:28:58,756][20347] Updated weights for policy 0, policy_version 701687 (0.0007) [2023-12-27 10:28:58,820][20347] Updated weights for policy 0, policy_version 701697 (0.0008) [2023-12-27 10:28:59,277][20381] Updated weights for policy 1, policy_version 701179 (0.0008) [2023-12-27 10:28:59,336][20381] Updated weights for policy 1, policy_version 701189 (0.0007) [2023-12-27 10:28:59,386][20381] Updated weights for policy 1, policy_version 701199 (0.0008) [2023-12-27 10:28:59,475][20347] Updated weights for policy 0, policy_version 701707 (0.0008) [2023-12-27 10:28:59,530][20347] Updated weights for policy 0, policy_version 701717 (0.0007) [2023-12-27 10:28:59,588][20347] Updated weights for policy 0, policy_version 701727 (0.0007) [2023-12-27 10:29:00,087][20381] Updated weights for policy 1, policy_version 701209 (0.0008) [2023-12-27 10:29:00,137][20381] Updated weights for policy 1, policy_version 701219 (0.0008) [2023-12-27 10:29:00,183][20381] Updated weights for policy 1, policy_version 701229 (0.0008) [2023-12-27 10:29:00,242][20381] Updated weights for policy 1, policy_version 701239 (0.0008) [2023-12-27 10:29:00,276][20347] Updated weights for policy 0, policy_version 701737 (0.0007) [2023-12-27 10:29:00,334][20347] Updated weights for policy 0, policy_version 701747 (0.0008) [2023-12-27 10:29:00,385][20347] Updated weights for policy 0, policy_version 701757 (0.0007) [2023-12-27 10:29:00,434][20347] Updated weights for policy 0, policy_version 701767 (0.0008) [2023-12-27 10:29:00,900][20381] Updated weights for policy 1, policy_version 701249 (0.0005) [2023-12-27 10:29:00,959][20381] Updated weights for policy 1, policy_version 701259 (0.0005) [2023-12-27 10:29:01,010][20381] Updated weights for policy 1, policy_version 701269 (0.0005) [2023-12-27 10:29:01,161][20347] Updated weights for policy 0, policy_version 701777 (0.0007) [2023-12-27 10:29:01,225][20347] Updated weights for policy 0, policy_version 701787 (0.0007) [2023-12-27 10:29:01,286][20347] Updated weights for policy 0, policy_version 701797 (0.0008) [2023-12-27 10:29:01,624][20381] Updated weights for policy 1, policy_version 701279 (0.0008) [2023-12-27 10:29:01,682][20381] Updated weights for policy 1, policy_version 701289 (0.0008) [2023-12-27 10:29:01,744][20381] Updated weights for policy 1, policy_version 701299 (0.0009) [2023-12-27 10:29:01,958][20347] Updated weights for policy 0, policy_version 701807 (0.0008) [2023-12-27 10:29:02,013][20347] Updated weights for policy 0, policy_version 701817 (0.0009) [2023-12-27 10:29:02,065][20347] Updated weights for policy 0, policy_version 701827 (0.0007) [2023-12-27 10:29:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20493.9). Total num frames: 359251968. Throughput: 0: 10506.4, 1: 10327.7. Samples: 359263856. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:02,363][19170] Avg episode reward: [(0, '283.000'), (1, '883.790')] [2023-12-27 10:29:02,384][20381] Updated weights for policy 1, policy_version 701309 (0.0008) [2023-12-27 10:29:02,439][20381] Updated weights for policy 1, policy_version 701319 (0.0009) [2023-12-27 10:29:02,498][20381] Updated weights for policy 1, policy_version 701329 (0.0010) [2023-12-27 10:29:02,778][20347] Updated weights for policy 0, policy_version 701837 (0.0007) [2023-12-27 10:29:02,832][20347] Updated weights for policy 0, policy_version 701847 (0.0007) [2023-12-27 10:29:02,875][20347] Updated weights for policy 0, policy_version 701857 (0.0007) [2023-12-27 10:29:03,157][20381] Updated weights for policy 1, policy_version 701339 (0.0009) [2023-12-27 10:29:03,208][20381] Updated weights for policy 1, policy_version 701349 (0.0005) [2023-12-27 10:29:03,253][20381] Updated weights for policy 1, policy_version 701359 (0.0005) [2023-12-27 10:29:03,564][20347] Updated weights for policy 0, policy_version 701867 (0.0007) [2023-12-27 10:29:03,620][20347] Updated weights for policy 0, policy_version 701877 (0.0007) [2023-12-27 10:29:03,676][20347] Updated weights for policy 0, policy_version 701887 (0.0008) [2023-12-27 10:29:03,859][20381] Updated weights for policy 1, policy_version 701369 (0.0006) [2023-12-27 10:29:03,911][20381] Updated weights for policy 1, policy_version 701379 (0.0006) [2023-12-27 10:29:03,971][20381] Updated weights for policy 1, policy_version 701389 (0.0006) [2023-12-27 10:29:04,031][20381] Updated weights for policy 1, policy_version 701399 (0.0006) [2023-12-27 10:29:04,382][20347] Updated weights for policy 0, policy_version 701897 (0.0008) [2023-12-27 10:29:04,438][20347] Updated weights for policy 0, policy_version 701907 (0.0008) [2023-12-27 10:29:04,492][20347] Updated weights for policy 0, policy_version 701917 (0.0006) [2023-12-27 10:29:04,544][20347] Updated weights for policy 0, policy_version 701927 (0.0006) [2023-12-27 10:29:04,662][20381] Updated weights for policy 1, policy_version 701409 (0.0008) [2023-12-27 10:29:04,713][20381] Updated weights for policy 1, policy_version 701419 (0.0008) [2023-12-27 10:29:04,765][20381] Updated weights for policy 1, policy_version 701429 (0.0008) [2023-12-27 10:29:05,169][20347] Updated weights for policy 0, policy_version 701937 (0.0007) [2023-12-27 10:29:05,226][20347] Updated weights for policy 0, policy_version 701947 (0.0006) [2023-12-27 10:29:05,285][20347] Updated weights for policy 0, policy_version 701957 (0.0006) [2023-12-27 10:29:05,409][20381] Updated weights for policy 1, policy_version 701439 (0.0007) [2023-12-27 10:29:05,464][20381] Updated weights for policy 1, policy_version 701449 (0.0007) [2023-12-27 10:29:05,519][20381] Updated weights for policy 1, policy_version 701459 (0.0007) [2023-12-27 10:29:05,862][20347] Updated weights for policy 0, policy_version 701967 (0.0007) [2023-12-27 10:29:05,922][20347] Updated weights for policy 0, policy_version 701977 (0.0008) [2023-12-27 10:29:05,990][20347] Updated weights for policy 0, policy_version 701987 (0.0008) [2023-12-27 10:29:06,074][20381] Updated weights for policy 1, policy_version 701469 (0.0006) [2023-12-27 10:29:06,136][20381] Updated weights for policy 1, policy_version 701479 (0.0007) [2023-12-27 10:29:06,201][20381] Updated weights for policy 1, policy_version 701489 (0.0005) [2023-12-27 10:29:06,637][20347] Updated weights for policy 0, policy_version 701997 (0.0009) [2023-12-27 10:29:06,698][20347] Updated weights for policy 0, policy_version 702007 (0.0010) [2023-12-27 10:29:06,760][20347] Updated weights for policy 0, policy_version 702017 (0.0009) [2023-12-27 10:29:06,827][20381] Updated weights for policy 1, policy_version 701499 (0.0006) [2023-12-27 10:29:06,885][20381] Updated weights for policy 1, policy_version 701509 (0.0009) [2023-12-27 10:29:06,941][20381] Updated weights for policy 1, policy_version 701519 (0.0011) [2023-12-27 10:29:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 359358464. Throughput: 0: 10512.0, 1: 10339.0. Samples: 359327640. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:07,363][19170] Avg episode reward: [(0, '226.870'), (1, '912.610')] [2023-12-27 10:29:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000702024_179740672.pth... [2023-12-27 10:29:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000701528_179617792.pth... [2023-12-27 10:29:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000700808_179429376.pth [2023-12-27 10:29:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000700312_179306496.pth [2023-12-27 10:29:07,398][20347] Updated weights for policy 0, policy_version 702027 (0.0008) [2023-12-27 10:29:07,456][20347] Updated weights for policy 0, policy_version 702037 (0.0010) [2023-12-27 10:29:07,508][20347] Updated weights for policy 0, policy_version 702047 (0.0010) [2023-12-27 10:29:07,598][20381] Updated weights for policy 1, policy_version 701529 (0.0011) [2023-12-27 10:29:07,645][20381] Updated weights for policy 1, policy_version 701539 (0.0007) [2023-12-27 10:29:07,700][20381] Updated weights for policy 1, policy_version 701549 (0.0007) [2023-12-27 10:29:07,745][20381] Updated weights for policy 1, policy_version 701559 (0.0007) [2023-12-27 10:29:08,175][20347] Updated weights for policy 0, policy_version 702057 (0.0010) [2023-12-27 10:29:08,243][20347] Updated weights for policy 0, policy_version 702067 (0.0010) [2023-12-27 10:29:08,307][20347] Updated weights for policy 0, policy_version 702077 (0.0010) [2023-12-27 10:29:08,365][20347] Updated weights for policy 0, policy_version 702087 (0.0010) [2023-12-27 10:29:08,390][20381] Updated weights for policy 1, policy_version 701569 (0.0007) [2023-12-27 10:29:08,448][20381] Updated weights for policy 1, policy_version 701579 (0.0007) [2023-12-27 10:29:08,508][20381] Updated weights for policy 1, policy_version 701589 (0.0007) [2023-12-27 10:29:09,064][20347] Updated weights for policy 0, policy_version 702097 (0.0010) [2023-12-27 10:29:09,119][20347] Updated weights for policy 0, policy_version 702107 (0.0010) [2023-12-27 10:29:09,172][20347] Updated weights for policy 0, policy_version 702117 (0.0010) [2023-12-27 10:29:09,208][20381] Updated weights for policy 1, policy_version 701599 (0.0006) [2023-12-27 10:29:09,272][20381] Updated weights for policy 1, policy_version 701609 (0.0008) [2023-12-27 10:29:09,340][20381] Updated weights for policy 1, policy_version 701619 (0.0008) [2023-12-27 10:29:09,970][20347] Updated weights for policy 0, policy_version 702127 (0.0009) [2023-12-27 10:29:10,031][20347] Updated weights for policy 0, policy_version 702137 (0.0006) [2023-12-27 10:29:10,033][20381] Updated weights for policy 1, policy_version 701629 (0.0008) [2023-12-27 10:29:10,085][20347] Updated weights for policy 0, policy_version 702147 (0.0008) [2023-12-27 10:29:10,090][20381] Updated weights for policy 1, policy_version 701639 (0.0007) [2023-12-27 10:29:10,158][20381] Updated weights for policy 1, policy_version 701649 (0.0008) [2023-12-27 10:29:10,808][20381] Updated weights for policy 1, policy_version 701659 (0.0007) [2023-12-27 10:29:10,817][20347] Updated weights for policy 0, policy_version 702157 (0.0008) [2023-12-27 10:29:10,859][20381] Updated weights for policy 1, policy_version 701669 (0.0010) [2023-12-27 10:29:10,876][20347] Updated weights for policy 0, policy_version 702167 (0.0008) [2023-12-27 10:29:10,921][20381] Updated weights for policy 1, policy_version 701679 (0.0010) [2023-12-27 10:29:10,928][20347] Updated weights for policy 0, policy_version 702177 (0.0008) [2023-12-27 10:29:11,616][20381] Updated weights for policy 1, policy_version 701689 (0.0010) [2023-12-27 10:29:11,648][20347] Updated weights for policy 0, policy_version 702187 (0.0007) [2023-12-27 10:29:11,679][20381] Updated weights for policy 1, policy_version 701699 (0.0009) [2023-12-27 10:29:11,699][20292] KL-divergence is very high: 514.5803 [2023-12-27 10:29:11,706][20292] KL-divergence is very high: 352.9235 [2023-12-27 10:29:11,709][20347] Updated weights for policy 0, policy_version 702197 (0.0007) [2023-12-27 10:29:11,750][20381] Updated weights for policy 1, policy_version 701709 (0.0007) [2023-12-27 10:29:11,778][20347] Updated weights for policy 0, policy_version 702207 (0.0008) [2023-12-27 10:29:11,806][20381] Updated weights for policy 1, policy_version 701719 (0.0006) [2023-12-27 10:29:11,806][20292] KL-divergence is very high: 185.5369 [2023-12-27 10:29:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 359456768. Throughput: 0: 10455.1, 1: 10385.2. Samples: 359451016. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:12,363][19170] Avg episode reward: [(0, '389.000'), (1, '971.630')] [2023-12-27 10:29:12,471][20381] Updated weights for policy 1, policy_version 701729 (0.0008) [2023-12-27 10:29:12,502][20347] Updated weights for policy 0, policy_version 702217 (0.0008) [2023-12-27 10:29:12,523][20381] Updated weights for policy 1, policy_version 701739 (0.0007) [2023-12-27 10:29:12,559][20347] Updated weights for policy 0, policy_version 702227 (0.0005) [2023-12-27 10:29:12,575][20381] Updated weights for policy 1, policy_version 701749 (0.0007) [2023-12-27 10:29:12,613][20347] Updated weights for policy 0, policy_version 702237 (0.0006) [2023-12-27 10:29:12,673][20347] Updated weights for policy 0, policy_version 702247 (0.0006) [2023-12-27 10:29:13,185][20381] Updated weights for policy 1, policy_version 701759 (0.0007) [2023-12-27 10:29:13,240][20381] Updated weights for policy 1, policy_version 701769 (0.0008) [2023-12-27 10:29:13,289][20381] Updated weights for policy 1, policy_version 701779 (0.0008) [2023-12-27 10:29:13,371][20347] Updated weights for policy 0, policy_version 702257 (0.0007) [2023-12-27 10:29:13,430][20347] Updated weights for policy 0, policy_version 702267 (0.0007) [2023-12-27 10:29:13,488][20347] Updated weights for policy 0, policy_version 702277 (0.0006) [2023-12-27 10:29:13,923][20381] Updated weights for policy 1, policy_version 701789 (0.0007) [2023-12-27 10:29:13,941][20292] KL-divergence is very high: 100.0443 [2023-12-27 10:29:13,978][20381] Updated weights for policy 1, policy_version 701799 (0.0007) [2023-12-27 10:29:14,023][20292] KL-divergence is very high: 145.4606 [2023-12-27 10:29:14,034][20381] Updated weights for policy 1, policy_version 701809 (0.0008) [2023-12-27 10:29:14,034][20292] KL-divergence is very high: 232.7442 [2023-12-27 10:29:14,152][20347] Updated weights for policy 0, policy_version 702287 (0.0006) [2023-12-27 10:29:14,214][20347] Updated weights for policy 0, policy_version 702297 (0.0005) [2023-12-27 10:29:14,267][20347] Updated weights for policy 0, policy_version 702307 (0.0007) [2023-12-27 10:29:14,739][20381] Updated weights for policy 1, policy_version 701819 (0.0007) [2023-12-27 10:29:14,758][20292] KL-divergence is very high: 187.3340 [2023-12-27 10:29:14,798][20381] Updated weights for policy 1, policy_version 701829 (0.0008) [2023-12-27 10:29:14,859][20381] Updated weights for policy 1, policy_version 701839 (0.0008) [2023-12-27 10:29:14,931][20347] Updated weights for policy 0, policy_version 702317 (0.0006) [2023-12-27 10:29:14,994][20347] Updated weights for policy 0, policy_version 702327 (0.0006) [2023-12-27 10:29:15,059][20347] Updated weights for policy 0, policy_version 702337 (0.0006) [2023-12-27 10:29:15,456][20381] Updated weights for policy 1, policy_version 701849 (0.0007) [2023-12-27 10:29:15,504][20381] Updated weights for policy 1, policy_version 701859 (0.0005) [2023-12-27 10:29:15,557][20381] Updated weights for policy 1, policy_version 701869 (0.0007) [2023-12-27 10:29:15,612][20381] Updated weights for policy 1, policy_version 701879 (0.0007) [2023-12-27 10:29:15,747][20347] Updated weights for policy 0, policy_version 702347 (0.0007) [2023-12-27 10:29:15,791][20347] Updated weights for policy 0, policy_version 702357 (0.0008) [2023-12-27 10:29:15,840][20347] Updated weights for policy 0, policy_version 702367 (0.0008) [2023-12-27 10:29:16,198][20381] Updated weights for policy 1, policy_version 701889 (0.0010) [2023-12-27 10:29:16,257][20381] Updated weights for policy 1, policy_version 701899 (0.0010) [2023-12-27 10:29:16,315][20381] Updated weights for policy 1, policy_version 701909 (0.0007) [2023-12-27 10:29:16,473][20347] Updated weights for policy 0, policy_version 702377 (0.0009) [2023-12-27 10:29:16,523][20347] Updated weights for policy 0, policy_version 702387 (0.0006) [2023-12-27 10:29:16,571][20347] Updated weights for policy 0, policy_version 702397 (0.0008) [2023-12-27 10:29:16,624][20347] Updated weights for policy 0, policy_version 702407 (0.0006) [2023-12-27 10:29:16,997][20381] Updated weights for policy 1, policy_version 701919 (0.0009) [2023-12-27 10:29:17,054][20381] Updated weights for policy 1, policy_version 701929 (0.0008) [2023-12-27 10:29:17,104][20381] Updated weights for policy 1, policy_version 701939 (0.0005) [2023-12-27 10:29:17,195][20347] Updated weights for policy 0, policy_version 702417 (0.0006) [2023-12-27 10:29:17,245][20347] Updated weights for policy 0, policy_version 702427 (0.0008) [2023-12-27 10:29:17,303][20347] Updated weights for policy 0, policy_version 702437 (0.0008) [2023-12-27 10:29:17,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 359571456. Throughput: 0: 10384.7, 1: 10443.0. Samples: 359576700. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:17,363][19170] Avg episode reward: [(0, '545.050'), (1, '951.730')] [2023-12-27 10:29:17,648][20381] Updated weights for policy 1, policy_version 701949 (0.0007) [2023-12-27 10:29:17,668][20292] KL-divergence is very high: 221.0433 [2023-12-27 10:29:17,697][20381] Updated weights for policy 1, policy_version 701959 (0.0007) [2023-12-27 10:29:17,743][20381] Updated weights for policy 1, policy_version 701969 (0.0010) [2023-12-27 10:29:17,882][20347] Updated weights for policy 0, policy_version 702447 (0.0006) [2023-12-27 10:29:17,938][20347] Updated weights for policy 0, policy_version 702457 (0.0007) [2023-12-27 10:29:17,994][20347] Updated weights for policy 0, policy_version 702467 (0.0008) [2023-12-27 10:29:18,502][20381] Updated weights for policy 1, policy_version 701979 (0.0008) [2023-12-27 10:29:18,561][20381] Updated weights for policy 1, policy_version 701989 (0.0008) [2023-12-27 10:29:18,619][20381] Updated weights for policy 1, policy_version 701999 (0.0009) [2023-12-27 10:29:18,679][20347] Updated weights for policy 0, policy_version 702477 (0.0008) [2023-12-27 10:29:18,733][20347] Updated weights for policy 0, policy_version 702487 (0.0006) [2023-12-27 10:29:18,789][20347] Updated weights for policy 0, policy_version 702497 (0.0006) [2023-12-27 10:29:19,298][20381] Updated weights for policy 1, policy_version 702009 (0.0010) [2023-12-27 10:29:19,358][20381] Updated weights for policy 1, policy_version 702019 (0.0007) [2023-12-27 10:29:19,418][20381] Updated weights for policy 1, policy_version 702029 (0.0010) [2023-12-27 10:29:19,480][20381] Updated weights for policy 1, policy_version 702039 (0.0009) [2023-12-27 10:29:19,488][20347] Updated weights for policy 0, policy_version 702507 (0.0007) [2023-12-27 10:29:19,546][20347] Updated weights for policy 0, policy_version 702517 (0.0007) [2023-12-27 10:29:19,609][20347] Updated weights for policy 0, policy_version 702527 (0.0008) [2023-12-27 10:29:20,151][20381] Updated weights for policy 1, policy_version 702049 (0.0010) [2023-12-27 10:29:20,205][20381] Updated weights for policy 1, policy_version 702059 (0.0007) [2023-12-27 10:29:20,268][20381] Updated weights for policy 1, policy_version 702069 (0.0010) [2023-12-27 10:29:20,351][20347] Updated weights for policy 0, policy_version 702537 (0.0008) [2023-12-27 10:29:20,403][20347] Updated weights for policy 0, policy_version 702547 (0.0007) [2023-12-27 10:29:20,458][20347] Updated weights for policy 0, policy_version 702557 (0.0007) [2023-12-27 10:29:20,517][20347] Updated weights for policy 0, policy_version 702567 (0.0006) [2023-12-27 10:29:20,931][20381] Updated weights for policy 1, policy_version 702079 (0.0007) [2023-12-27 10:29:20,990][20381] Updated weights for policy 1, policy_version 702089 (0.0006) [2023-12-27 10:29:21,058][20381] Updated weights for policy 1, policy_version 702099 (0.0008) [2023-12-27 10:29:21,229][20347] Updated weights for policy 0, policy_version 702577 (0.0008) [2023-12-27 10:29:21,296][20347] Updated weights for policy 0, policy_version 702587 (0.0008) [2023-12-27 10:29:21,362][20347] Updated weights for policy 0, policy_version 702597 (0.0008) [2023-12-27 10:29:21,653][20381] Updated weights for policy 1, policy_version 702109 (0.0008) [2023-12-27 10:29:21,720][20381] Updated weights for policy 1, policy_version 702119 (0.0007) [2023-12-27 10:29:21,778][20381] Updated weights for policy 1, policy_version 702129 (0.0008) [2023-12-27 10:29:22,007][20347] Updated weights for policy 0, policy_version 702607 (0.0007) [2023-12-27 10:29:22,060][20347] Updated weights for policy 0, policy_version 702617 (0.0008) [2023-12-27 10:29:22,109][20347] Updated weights for policy 0, policy_version 702627 (0.0007) [2023-12-27 10:29:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 359669760. Throughput: 0: 10338.6, 1: 10460.9. Samples: 359638924. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:22,363][19170] Avg episode reward: [(0, '715.370'), (1, '902.310')] [2023-12-27 10:29:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000702632_179896320.pth... [2023-12-27 10:29:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000702136_179773440.pth... [2023-12-27 10:29:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000700920_179462144.pth [2023-12-27 10:29:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000701416_179585024.pth [2023-12-27 10:29:22,529][20381] Updated weights for policy 1, policy_version 702139 (0.0007) [2023-12-27 10:29:22,595][20381] Updated weights for policy 1, policy_version 702149 (0.0006) [2023-12-27 10:29:22,649][20381] Updated weights for policy 1, policy_version 702159 (0.0007) [2023-12-27 10:29:22,854][20347] Updated weights for policy 0, policy_version 702637 (0.0006) [2023-12-27 10:29:22,907][20347] Updated weights for policy 0, policy_version 702647 (0.0005) [2023-12-27 10:29:22,968][20347] Updated weights for policy 0, policy_version 702657 (0.0010) [2023-12-27 10:29:23,285][20381] Updated weights for policy 1, policy_version 702169 (0.0007) [2023-12-27 10:29:23,346][20381] Updated weights for policy 1, policy_version 702179 (0.0008) [2023-12-27 10:29:23,403][20381] Updated weights for policy 1, policy_version 702189 (0.0007) [2023-12-27 10:29:23,453][20381] Updated weights for policy 1, policy_version 702199 (0.0008) [2023-12-27 10:29:23,610][20347] Updated weights for policy 0, policy_version 702667 (0.0010) [2023-12-27 10:29:23,667][20347] Updated weights for policy 0, policy_version 702677 (0.0008) [2023-12-27 10:29:23,727][20347] Updated weights for policy 0, policy_version 702687 (0.0008) [2023-12-27 10:29:24,105][20381] Updated weights for policy 1, policy_version 702209 (0.0005) [2023-12-27 10:29:24,168][20381] Updated weights for policy 1, policy_version 702219 (0.0007) [2023-12-27 10:29:24,223][20381] Updated weights for policy 1, policy_version 702229 (0.0010) [2023-12-27 10:29:24,375][20347] Updated weights for policy 0, policy_version 702697 (0.0010) [2023-12-27 10:29:24,426][20347] Updated weights for policy 0, policy_version 702707 (0.0005) [2023-12-27 10:29:24,487][20347] Updated weights for policy 0, policy_version 702717 (0.0006) [2023-12-27 10:29:24,548][20347] Updated weights for policy 0, policy_version 702727 (0.0010) [2023-12-27 10:29:24,784][20381] Updated weights for policy 1, policy_version 702239 (0.0009) [2023-12-27 10:29:24,832][20381] Updated weights for policy 1, policy_version 702249 (0.0010) [2023-12-27 10:29:24,880][20381] Updated weights for policy 1, policy_version 702259 (0.0010) [2023-12-27 10:29:25,146][20347] Updated weights for policy 0, policy_version 702737 (0.0010) [2023-12-27 10:29:25,207][20347] Updated weights for policy 0, policy_version 702747 (0.0010) [2023-12-27 10:29:25,271][20347] Updated weights for policy 0, policy_version 702757 (0.0010) [2023-12-27 10:29:25,538][20381] Updated weights for policy 1, policy_version 702269 (0.0011) [2023-12-27 10:29:25,586][20381] Updated weights for policy 1, policy_version 702279 (0.0010) [2023-12-27 10:29:25,634][20381] Updated weights for policy 1, policy_version 702289 (0.0010) [2023-12-27 10:29:25,839][20347] Updated weights for policy 0, policy_version 702767 (0.0007) [2023-12-27 10:29:25,900][20347] Updated weights for policy 0, policy_version 702777 (0.0005) [2023-12-27 10:29:25,958][20347] Updated weights for policy 0, policy_version 702787 (0.0005) [2023-12-27 10:29:26,271][20381] Updated weights for policy 1, policy_version 702299 (0.0011) [2023-12-27 10:29:26,332][20381] Updated weights for policy 1, policy_version 702309 (0.0009) [2023-12-27 10:29:26,391][20381] Updated weights for policy 1, policy_version 702319 (0.0005) [2023-12-27 10:29:26,619][20347] Updated weights for policy 0, policy_version 702797 (0.0006) [2023-12-27 10:29:26,678][20347] Updated weights for policy 0, policy_version 702807 (0.0007) [2023-12-27 10:29:26,737][20347] Updated weights for policy 0, policy_version 702817 (0.0006) [2023-12-27 10:29:26,913][20381] Updated weights for policy 1, policy_version 702329 (0.0005) [2023-12-27 10:29:26,970][20381] Updated weights for policy 1, policy_version 702339 (0.0005) [2023-12-27 10:29:27,017][20381] Updated weights for policy 1, policy_version 702349 (0.0005) [2023-12-27 10:29:27,067][20381] Updated weights for policy 1, policy_version 702359 (0.0005) [2023-12-27 10:29:27,326][20347] Updated weights for policy 0, policy_version 702827 (0.0005) [2023-12-27 10:29:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20688.3). Total num frames: 359776256. Throughput: 0: 10361.6, 1: 10500.9. Samples: 359765916. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:27,363][19170] Avg episode reward: [(0, '771.390'), (1, '825.480')] [2023-12-27 10:29:27,385][20347] Updated weights for policy 0, policy_version 702837 (0.0005) [2023-12-27 10:29:27,439][20347] Updated weights for policy 0, policy_version 702847 (0.0005) [2023-12-27 10:29:27,654][20381] Updated weights for policy 1, policy_version 702369 (0.0010) [2023-12-27 10:29:27,705][20381] Updated weights for policy 1, policy_version 702379 (0.0010) [2023-12-27 10:29:27,760][20381] Updated weights for policy 1, policy_version 702389 (0.0005) [2023-12-27 10:29:27,978][20347] Updated weights for policy 0, policy_version 702857 (0.0005) [2023-12-27 10:29:28,030][20347] Updated weights for policy 0, policy_version 702867 (0.0007) [2023-12-27 10:29:28,074][20347] Updated weights for policy 0, policy_version 702877 (0.0008) [2023-12-27 10:29:28,122][20347] Updated weights for policy 0, policy_version 702887 (0.0008) [2023-12-27 10:29:28,315][20381] Updated weights for policy 1, policy_version 702399 (0.0005) [2023-12-27 10:29:28,374][20381] Updated weights for policy 1, policy_version 702409 (0.0008) [2023-12-27 10:29:28,421][20381] Updated weights for policy 1, policy_version 702419 (0.0008) [2023-12-27 10:29:28,828][20347] Updated weights for policy 0, policy_version 702897 (0.0007) [2023-12-27 10:29:28,882][20347] Updated weights for policy 0, policy_version 702907 (0.0010) [2023-12-27 10:29:28,937][20347] Updated weights for policy 0, policy_version 702917 (0.0010) [2023-12-27 10:29:28,987][20381] Updated weights for policy 1, policy_version 702429 (0.0006) [2023-12-27 10:29:29,035][20381] Updated weights for policy 1, policy_version 702439 (0.0005) [2023-12-27 10:29:29,086][20381] Updated weights for policy 1, policy_version 702449 (0.0005) [2023-12-27 10:29:29,550][20347] Updated weights for policy 0, policy_version 702927 (0.0007) [2023-12-27 10:29:29,610][20347] Updated weights for policy 0, policy_version 702937 (0.0007) [2023-12-27 10:29:29,660][20347] Updated weights for policy 0, policy_version 702947 (0.0009) [2023-12-27 10:29:29,715][20381] Updated weights for policy 1, policy_version 702459 (0.0005) [2023-12-27 10:29:29,775][20381] Updated weights for policy 1, policy_version 702469 (0.0006) [2023-12-27 10:29:29,839][20381] Updated weights for policy 1, policy_version 702479 (0.0007) [2023-12-27 10:29:30,373][20347] Updated weights for policy 0, policy_version 702957 (0.0008) [2023-12-27 10:29:30,427][20347] Updated weights for policy 0, policy_version 702967 (0.0008) [2023-12-27 10:29:30,486][20347] Updated weights for policy 0, policy_version 702977 (0.0008) [2023-12-27 10:29:30,504][20381] Updated weights for policy 1, policy_version 702489 (0.0008) [2023-12-27 10:29:30,558][20381] Updated weights for policy 1, policy_version 702499 (0.0007) [2023-12-27 10:29:30,617][20381] Updated weights for policy 1, policy_version 702509 (0.0008) [2023-12-27 10:29:30,671][20381] Updated weights for policy 1, policy_version 702519 (0.0008) [2023-12-27 10:29:31,182][20347] Updated weights for policy 0, policy_version 702987 (0.0007) [2023-12-27 10:29:31,230][20347] Updated weights for policy 0, policy_version 702997 (0.0008) [2023-12-27 10:29:31,288][20347] Updated weights for policy 0, policy_version 703007 (0.0007) [2023-12-27 10:29:31,365][20381] Updated weights for policy 1, policy_version 702529 (0.0009) [2023-12-27 10:29:31,424][20381] Updated weights for policy 1, policy_version 702539 (0.0008) [2023-12-27 10:29:31,474][20381] Updated weights for policy 1, policy_version 702549 (0.0008) [2023-12-27 10:29:32,023][20347] Updated weights for policy 0, policy_version 703017 (0.0007) [2023-12-27 10:29:32,077][20347] Updated weights for policy 0, policy_version 703027 (0.0007) [2023-12-27 10:29:32,128][20347] Updated weights for policy 0, policy_version 703037 (0.0008) [2023-12-27 10:29:32,177][20347] Updated weights for policy 0, policy_version 703047 (0.0006) [2023-12-27 10:29:32,187][20381] Updated weights for policy 1, policy_version 702559 (0.0006) [2023-12-27 10:29:32,242][20381] Updated weights for policy 1, policy_version 702569 (0.0008) [2023-12-27 10:29:32,303][20381] Updated weights for policy 1, policy_version 702579 (0.0009) [2023-12-27 10:29:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 21026.1, 300 sec: 20743.8). Total num frames: 359890944. Throughput: 0: 10417.4, 1: 10600.3. Samples: 359895472. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:32,363][19170] Avg episode reward: [(0, '780.680'), (1, '883.600')] [2023-12-27 10:29:32,806][20347] Updated weights for policy 0, policy_version 703057 (0.0007) [2023-12-27 10:29:32,863][20347] Updated weights for policy 0, policy_version 703067 (0.0008) [2023-12-27 10:29:32,921][20347] Updated weights for policy 0, policy_version 703077 (0.0007) [2023-12-27 10:29:32,969][20381] Updated weights for policy 1, policy_version 702589 (0.0007) [2023-12-27 10:29:33,025][20381] Updated weights for policy 1, policy_version 702599 (0.0008) [2023-12-27 10:29:33,083][20381] Updated weights for policy 1, policy_version 702609 (0.0007) [2023-12-27 10:29:33,595][20347] Updated weights for policy 0, policy_version 703087 (0.0007) [2023-12-27 10:29:33,640][20347] Updated weights for policy 0, policy_version 703097 (0.0007) [2023-12-27 10:29:33,685][20347] Updated weights for policy 0, policy_version 703107 (0.0007) [2023-12-27 10:29:33,765][20381] Updated weights for policy 1, policy_version 702619 (0.0008) [2023-12-27 10:29:33,822][20381] Updated weights for policy 1, policy_version 702629 (0.0010) [2023-12-27 10:29:33,880][20381] Updated weights for policy 1, policy_version 702639 (0.0010) [2023-12-27 10:29:34,442][20347] Updated weights for policy 0, policy_version 703117 (0.0008) [2023-12-27 10:29:34,503][20347] Updated weights for policy 0, policy_version 703127 (0.0007) [2023-12-27 10:29:34,560][20347] Updated weights for policy 0, policy_version 703137 (0.0007) [2023-12-27 10:29:34,617][20381] Updated weights for policy 1, policy_version 702649 (0.0010) [2023-12-27 10:29:34,677][20381] Updated weights for policy 1, policy_version 702659 (0.0007) [2023-12-27 10:29:34,736][20381] Updated weights for policy 1, policy_version 702669 (0.0007) [2023-12-27 10:29:34,786][20381] Updated weights for policy 1, policy_version 702679 (0.0006) [2023-12-27 10:29:35,175][20347] Updated weights for policy 0, policy_version 703147 (0.0006) [2023-12-27 10:29:35,230][20347] Updated weights for policy 0, policy_version 703157 (0.0005) [2023-12-27 10:29:35,288][20347] Updated weights for policy 0, policy_version 703167 (0.0005) [2023-12-27 10:29:35,499][20381] Updated weights for policy 1, policy_version 702689 (0.0007) [2023-12-27 10:29:35,553][20381] Updated weights for policy 1, policy_version 702699 (0.0007) [2023-12-27 10:29:35,599][20381] Updated weights for policy 1, policy_version 702709 (0.0007) [2023-12-27 10:29:35,919][20347] Updated weights for policy 0, policy_version 703177 (0.0008) [2023-12-27 10:29:35,984][20347] Updated weights for policy 0, policy_version 703187 (0.0006) [2023-12-27 10:29:36,040][20347] Updated weights for policy 0, policy_version 703197 (0.0005) [2023-12-27 10:29:36,106][20347] Updated weights for policy 0, policy_version 703207 (0.0007) [2023-12-27 10:29:36,347][20381] Updated weights for policy 1, policy_version 702719 (0.0007) [2023-12-27 10:29:36,412][20381] Updated weights for policy 1, policy_version 702729 (0.0008) [2023-12-27 10:29:36,475][20381] Updated weights for policy 1, policy_version 702739 (0.0008) [2023-12-27 10:29:36,757][20347] Updated weights for policy 0, policy_version 703217 (0.0010) [2023-12-27 10:29:36,816][20347] Updated weights for policy 0, policy_version 703227 (0.0010) [2023-12-27 10:29:36,866][20347] Updated weights for policy 0, policy_version 703237 (0.0010) [2023-12-27 10:29:37,156][20381] Updated weights for policy 1, policy_version 702749 (0.0008) [2023-12-27 10:29:37,205][20381] Updated weights for policy 1, policy_version 702759 (0.0010) [2023-12-27 10:29:37,251][20381] Updated weights for policy 1, policy_version 702769 (0.0008) [2023-12-27 10:29:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 359989248. Throughput: 0: 10435.3, 1: 10616.3. Samples: 359956760. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:37,363][19170] Avg episode reward: [(0, '377.150'), (1, '894.060')] [2023-12-27 10:29:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000702776_179937280.pth... [2023-12-27 10:29:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000703240_180051968.pth... [2023-12-27 10:29:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000701528_179617792.pth [2023-12-27 10:29:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000702024_179740672.pth [2023-12-27 10:29:37,465][20347] Updated weights for policy 0, policy_version 703247 (0.0007) [2023-12-27 10:29:37,523][20347] Updated weights for policy 0, policy_version 703257 (0.0007) [2023-12-27 10:29:37,582][20347] Updated weights for policy 0, policy_version 703267 (0.0010) [2023-12-27 10:29:37,920][20381] Updated weights for policy 1, policy_version 702779 (0.0006) [2023-12-27 10:29:37,974][20381] Updated weights for policy 1, policy_version 702789 (0.0006) [2023-12-27 10:29:38,032][20381] Updated weights for policy 1, policy_version 702799 (0.0009) [2023-12-27 10:29:38,175][20347] Updated weights for policy 0, policy_version 703277 (0.0009) [2023-12-27 10:29:38,230][20347] Updated weights for policy 0, policy_version 703287 (0.0006) [2023-12-27 10:29:38,286][20347] Updated weights for policy 0, policy_version 703297 (0.0005) [2023-12-27 10:29:38,720][20381] Updated weights for policy 1, policy_version 702809 (0.0006) [2023-12-27 10:29:38,789][20381] Updated weights for policy 1, policy_version 702819 (0.0007) [2023-12-27 10:29:38,851][20381] Updated weights for policy 1, policy_version 702829 (0.0009) [2023-12-27 10:29:38,876][20347] Updated weights for policy 0, policy_version 703307 (0.0006) [2023-12-27 10:29:38,909][20381] Updated weights for policy 1, policy_version 702839 (0.0007) [2023-12-27 10:29:38,938][20347] Updated weights for policy 0, policy_version 703317 (0.0007) [2023-12-27 10:29:38,999][20347] Updated weights for policy 0, policy_version 703327 (0.0010) [2023-12-27 10:29:39,532][20381] Updated weights for policy 1, policy_version 702849 (0.0006) [2023-12-27 10:29:39,590][20381] Updated weights for policy 1, policy_version 702859 (0.0007) [2023-12-27 10:29:39,646][20381] Updated weights for policy 1, policy_version 702869 (0.0008) [2023-12-27 10:29:39,704][20347] Updated weights for policy 0, policy_version 703337 (0.0010) [2023-12-27 10:29:39,768][20347] Updated weights for policy 0, policy_version 703347 (0.0008) [2023-12-27 10:29:39,826][20347] Updated weights for policy 0, policy_version 703357 (0.0008) [2023-12-27 10:29:39,893][20347] Updated weights for policy 0, policy_version 703367 (0.0009) [2023-12-27 10:29:40,392][20381] Updated weights for policy 1, policy_version 702879 (0.0007) [2023-12-27 10:29:40,443][20381] Updated weights for policy 1, policy_version 702889 (0.0008) [2023-12-27 10:29:40,496][20381] Updated weights for policy 1, policy_version 702899 (0.0006) [2023-12-27 10:29:40,520][20347] Updated weights for policy 0, policy_version 703377 (0.0011) [2023-12-27 10:29:40,569][20347] Updated weights for policy 0, policy_version 703387 (0.0010) [2023-12-27 10:29:40,631][20347] Updated weights for policy 0, policy_version 703397 (0.0010) [2023-12-27 10:29:41,233][20381] Updated weights for policy 1, policy_version 702909 (0.0007) [2023-12-27 10:29:41,298][20381] Updated weights for policy 1, policy_version 702919 (0.0009) [2023-12-27 10:29:41,374][20381] Updated weights for policy 1, policy_version 702929 (0.0008) [2023-12-27 10:29:41,430][20347] Updated weights for policy 0, policy_version 703407 (0.0009) [2023-12-27 10:29:41,483][20347] Updated weights for policy 0, policy_version 703417 (0.0008) [2023-12-27 10:29:41,536][20347] Updated weights for policy 0, policy_version 703427 (0.0008) [2023-12-27 10:29:42,086][20381] Updated weights for policy 1, policy_version 702939 (0.0007) [2023-12-27 10:29:42,151][20381] Updated weights for policy 1, policy_version 702949 (0.0007) [2023-12-27 10:29:42,212][20381] Updated weights for policy 1, policy_version 702959 (0.0007) [2023-12-27 10:29:42,361][20347] Updated weights for policy 0, policy_version 703437 (0.0008) [2023-12-27 10:29:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 360087552. Throughput: 0: 10421.7, 1: 10537.9. Samples: 360080688. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:42,363][19170] Avg episode reward: [(0, '570.010'), (1, '912.690')] [2023-12-27 10:29:42,427][20347] Updated weights for policy 0, policy_version 703447 (0.0008) [2023-12-27 10:29:42,488][20347] Updated weights for policy 0, policy_version 703457 (0.0008) [2023-12-27 10:29:42,856][20381] Updated weights for policy 1, policy_version 702969 (0.0008) [2023-12-27 10:29:42,921][20381] Updated weights for policy 1, policy_version 702979 (0.0010) [2023-12-27 10:29:42,981][20381] Updated weights for policy 1, policy_version 702989 (0.0010) [2023-12-27 10:29:43,013][20292] KL-divergence is very high: 147.8634 [2023-12-27 10:29:43,035][20381] Updated weights for policy 1, policy_version 702999 (0.0010) [2023-12-27 10:29:43,149][20347] Updated weights for policy 0, policy_version 703467 (0.0007) [2023-12-27 10:29:43,214][20347] Updated weights for policy 0, policy_version 703477 (0.0006) [2023-12-27 10:29:43,274][20347] Updated weights for policy 0, policy_version 703487 (0.0006) [2023-12-27 10:29:43,647][20381] Updated weights for policy 1, policy_version 703009 (0.0007) [2023-12-27 10:29:43,698][20381] Updated weights for policy 1, policy_version 703019 (0.0008) [2023-12-27 10:29:43,745][20381] Updated weights for policy 1, policy_version 703029 (0.0007) [2023-12-27 10:29:43,907][20347] Updated weights for policy 0, policy_version 703497 (0.0007) [2023-12-27 10:29:43,960][20347] Updated weights for policy 0, policy_version 703507 (0.0008) [2023-12-27 10:29:44,015][20347] Updated weights for policy 0, policy_version 703517 (0.0007) [2023-12-27 10:29:44,070][20347] Updated weights for policy 0, policy_version 703527 (0.0005) [2023-12-27 10:29:44,478][20381] Updated weights for policy 1, policy_version 703039 (0.0008) [2023-12-27 10:29:44,538][20381] Updated weights for policy 1, policy_version 703049 (0.0008) [2023-12-27 10:29:44,598][20381] Updated weights for policy 1, policy_version 703059 (0.0008) [2023-12-27 10:29:44,669][20347] Updated weights for policy 0, policy_version 703537 (0.0007) [2023-12-27 10:29:44,724][20347] Updated weights for policy 0, policy_version 703547 (0.0007) [2023-12-27 10:29:44,786][20347] Updated weights for policy 0, policy_version 703557 (0.0008) [2023-12-27 10:29:45,257][20381] Updated weights for policy 1, policy_version 703069 (0.0007) [2023-12-27 10:29:45,307][20381] Updated weights for policy 1, policy_version 703079 (0.0005) [2023-12-27 10:29:45,367][20381] Updated weights for policy 1, policy_version 703089 (0.0007) [2023-12-27 10:29:45,430][20347] Updated weights for policy 0, policy_version 703567 (0.0006) [2023-12-27 10:29:45,479][20347] Updated weights for policy 0, policy_version 703577 (0.0007) [2023-12-27 10:29:45,524][20347] Updated weights for policy 0, policy_version 703587 (0.0008) [2023-12-27 10:29:45,994][20381] Updated weights for policy 1, policy_version 703099 (0.0009) [2023-12-27 10:29:46,039][20381] Updated weights for policy 1, policy_version 703109 (0.0010) [2023-12-27 10:29:46,098][20381] Updated weights for policy 1, policy_version 703119 (0.0010) [2023-12-27 10:29:46,148][20347] Updated weights for policy 0, policy_version 703597 (0.0006) [2023-12-27 10:29:46,192][20347] Updated weights for policy 0, policy_version 703607 (0.0005) [2023-12-27 10:29:46,239][20347] Updated weights for policy 0, policy_version 703617 (0.0005) [2023-12-27 10:29:46,802][20381] Updated weights for policy 1, policy_version 703129 (0.0011) [2023-12-27 10:29:46,856][20381] Updated weights for policy 1, policy_version 703139 (0.0007) [2023-12-27 10:29:46,881][20347] Updated weights for policy 0, policy_version 703627 (0.0005) [2023-12-27 10:29:46,905][20381] Updated weights for policy 1, policy_version 703149 (0.0009) [2023-12-27 10:29:46,941][20347] Updated weights for policy 0, policy_version 703637 (0.0005) [2023-12-27 10:29:46,954][20381] Updated weights for policy 1, policy_version 703159 (0.0010) [2023-12-27 10:29:46,996][20347] Updated weights for policy 0, policy_version 703647 (0.0006) [2023-12-27 10:29:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 360194048. Throughput: 0: 10402.1, 1: 10487.1. Samples: 360203872. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:47,363][19170] Avg episode reward: [(0, '342.940'), (1, '951.460')] [2023-12-27 10:29:47,555][20381] Updated weights for policy 1, policy_version 703169 (0.0006) [2023-12-27 10:29:47,610][20381] Updated weights for policy 1, policy_version 703179 (0.0005) [2023-12-27 10:29:47,667][20347] Updated weights for policy 0, policy_version 703657 (0.0007) [2023-12-27 10:29:47,669][20381] Updated weights for policy 1, policy_version 703189 (0.0005) [2023-12-27 10:29:47,719][20347] Updated weights for policy 0, policy_version 703667 (0.0007) [2023-12-27 10:29:47,779][20347] Updated weights for policy 0, policy_version 703677 (0.0008) [2023-12-27 10:29:47,828][20347] Updated weights for policy 0, policy_version 703687 (0.0008) [2023-12-27 10:29:48,249][20381] Updated weights for policy 1, policy_version 703199 (0.0005) [2023-12-27 10:29:48,299][20381] Updated weights for policy 1, policy_version 703209 (0.0005) [2023-12-27 10:29:48,360][20381] Updated weights for policy 1, policy_version 703219 (0.0008) [2023-12-27 10:29:48,571][20347] Updated weights for policy 0, policy_version 703697 (0.0008) [2023-12-27 10:29:48,626][20347] Updated weights for policy 0, policy_version 703707 (0.0007) [2023-12-27 10:29:48,682][20347] Updated weights for policy 0, policy_version 703717 (0.0008) [2023-12-27 10:29:48,936][20381] Updated weights for policy 1, policy_version 703229 (0.0007) [2023-12-27 10:29:48,999][20381] Updated weights for policy 1, policy_version 703239 (0.0008) [2023-12-27 10:29:49,057][20381] Updated weights for policy 1, policy_version 703249 (0.0009) [2023-12-27 10:29:49,395][20347] Updated weights for policy 0, policy_version 703727 (0.0008) [2023-12-27 10:29:49,454][20347] Updated weights for policy 0, policy_version 703737 (0.0005) [2023-12-27 10:29:49,509][20347] Updated weights for policy 0, policy_version 703747 (0.0005) [2023-12-27 10:29:49,784][20381] Updated weights for policy 1, policy_version 703259 (0.0008) [2023-12-27 10:29:49,839][20381] Updated weights for policy 1, policy_version 703269 (0.0009) [2023-12-27 10:29:49,893][20381] Updated weights for policy 1, policy_version 703279 (0.0007) [2023-12-27 10:29:50,088][20347] Updated weights for policy 0, policy_version 703757 (0.0007) [2023-12-27 10:29:50,147][20347] Updated weights for policy 0, policy_version 703767 (0.0010) [2023-12-27 10:29:50,205][20347] Updated weights for policy 0, policy_version 703777 (0.0010) [2023-12-27 10:29:50,609][20381] Updated weights for policy 1, policy_version 703289 (0.0008) [2023-12-27 10:29:50,667][20381] Updated weights for policy 1, policy_version 703299 (0.0011) [2023-12-27 10:29:50,723][20381] Updated weights for policy 1, policy_version 703309 (0.0011) [2023-12-27 10:29:50,782][20381] Updated weights for policy 1, policy_version 703319 (0.0011) [2023-12-27 10:29:50,884][20347] Updated weights for policy 0, policy_version 703787 (0.0008) [2023-12-27 10:29:50,939][20347] Updated weights for policy 0, policy_version 703797 (0.0009) [2023-12-27 10:29:50,993][20347] Updated weights for policy 0, policy_version 703807 (0.0007) [2023-12-27 10:29:51,523][20381] Updated weights for policy 1, policy_version 703329 (0.0009) [2023-12-27 10:29:51,573][20381] Updated weights for policy 1, policy_version 703339 (0.0008) [2023-12-27 10:29:51,636][20381] Updated weights for policy 1, policy_version 703349 (0.0008) [2023-12-27 10:29:51,679][20347] Updated weights for policy 0, policy_version 703817 (0.0008) [2023-12-27 10:29:51,744][20347] Updated weights for policy 0, policy_version 703827 (0.0007) [2023-12-27 10:29:51,813][20347] Updated weights for policy 0, policy_version 703837 (0.0005) [2023-12-27 10:29:51,877][20347] Updated weights for policy 0, policy_version 703847 (0.0007) [2023-12-27 10:29:52,257][20381] Updated weights for policy 1, policy_version 703359 (0.0006) [2023-12-27 10:29:52,313][20381] Updated weights for policy 1, policy_version 703369 (0.0006) [2023-12-27 10:29:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 360292352. Throughput: 0: 10389.8, 1: 10486.1. Samples: 360267060. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:52,363][19170] Avg episode reward: [(0, '430.640'), (1, '990.280')] [2023-12-27 10:29:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000703848_180207616.pth... [2023-12-27 10:29:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000702632_179896320.pth [2023-12-27 10:29:52,373][20381] Updated weights for policy 1, policy_version 703379 (0.0008) [2023-12-27 10:29:52,402][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000703384_180092928.pth... [2023-12-27 10:29:52,405][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000702136_179773440.pth [2023-12-27 10:29:52,496][20347] Updated weights for policy 0, policy_version 703857 (0.0008) [2023-12-27 10:29:52,542][20347] Updated weights for policy 0, policy_version 703867 (0.0007) [2023-12-27 10:29:52,602][20347] Updated weights for policy 0, policy_version 703877 (0.0007) [2023-12-27 10:29:53,010][20381] Updated weights for policy 1, policy_version 703389 (0.0010) [2023-12-27 10:29:53,066][20381] Updated weights for policy 1, policy_version 703399 (0.0010) [2023-12-27 10:29:53,123][20381] Updated weights for policy 1, policy_version 703409 (0.0008) [2023-12-27 10:29:53,346][20347] Updated weights for policy 0, policy_version 703887 (0.0006) [2023-12-27 10:29:53,406][20347] Updated weights for policy 0, policy_version 703897 (0.0010) [2023-12-27 10:29:53,467][20347] Updated weights for policy 0, policy_version 703907 (0.0010) [2023-12-27 10:29:53,753][20381] Updated weights for policy 1, policy_version 703419 (0.0005) [2023-12-27 10:29:53,818][20381] Updated weights for policy 1, policy_version 703429 (0.0005) [2023-12-27 10:29:53,879][20381] Updated weights for policy 1, policy_version 703439 (0.0007) [2023-12-27 10:29:54,159][20347] Updated weights for policy 0, policy_version 703917 (0.0008) [2023-12-27 10:29:54,207][20347] Updated weights for policy 0, policy_version 703927 (0.0008) [2023-12-27 10:29:54,260][20347] Updated weights for policy 0, policy_version 703937 (0.0008) [2023-12-27 10:29:54,508][20381] Updated weights for policy 1, policy_version 703449 (0.0009) [2023-12-27 10:29:54,558][20381] Updated weights for policy 1, policy_version 703459 (0.0009) [2023-12-27 10:29:54,613][20381] Updated weights for policy 1, policy_version 703469 (0.0006) [2023-12-27 10:29:54,674][20381] Updated weights for policy 1, policy_version 703479 (0.0005) [2023-12-27 10:29:54,910][20347] Updated weights for policy 0, policy_version 703947 (0.0008) [2023-12-27 10:29:54,969][20347] Updated weights for policy 0, policy_version 703957 (0.0008) [2023-12-27 10:29:55,027][20347] Updated weights for policy 0, policy_version 703967 (0.0008) [2023-12-27 10:29:55,243][20381] Updated weights for policy 1, policy_version 703489 (0.0005) [2023-12-27 10:29:55,298][20381] Updated weights for policy 1, policy_version 703499 (0.0005) [2023-12-27 10:29:55,353][20381] Updated weights for policy 1, policy_version 703509 (0.0005) [2023-12-27 10:29:55,700][20347] Updated weights for policy 0, policy_version 703977 (0.0007) [2023-12-27 10:29:55,751][20347] Updated weights for policy 0, policy_version 703987 (0.0007) [2023-12-27 10:29:55,802][20347] Updated weights for policy 0, policy_version 703997 (0.0010) [2023-12-27 10:29:55,855][20347] Updated weights for policy 0, policy_version 704007 (0.0010) [2023-12-27 10:29:55,931][20381] Updated weights for policy 1, policy_version 703519 (0.0007) [2023-12-27 10:29:55,993][20381] Updated weights for policy 1, policy_version 703529 (0.0007) [2023-12-27 10:29:56,048][20381] Updated weights for policy 1, policy_version 703539 (0.0007) [2023-12-27 10:29:56,465][20347] Updated weights for policy 0, policy_version 704017 (0.0006) [2023-12-27 10:29:56,514][20347] Updated weights for policy 0, policy_version 704027 (0.0010) [2023-12-27 10:29:56,568][20347] Updated weights for policy 0, policy_version 704037 (0.0010) [2023-12-27 10:29:56,735][20381] Updated weights for policy 1, policy_version 703549 (0.0007) [2023-12-27 10:29:56,784][20381] Updated weights for policy 1, policy_version 703559 (0.0006) [2023-12-27 10:29:56,842][20381] Updated weights for policy 1, policy_version 703569 (0.0006) [2023-12-27 10:29:57,178][20347] Updated weights for policy 0, policy_version 704047 (0.0006) [2023-12-27 10:29:57,234][20347] Updated weights for policy 0, policy_version 704057 (0.0005) [2023-12-27 10:29:57,285][20347] Updated weights for policy 0, policy_version 704067 (0.0006) [2023-12-27 10:29:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 360407040. Throughput: 0: 10400.8, 1: 10519.6. Samples: 360392436. Policy #0 lag: (min: 3.0, avg: 13.4, max: 35.0) [2023-12-27 10:29:57,363][19170] Avg episode reward: [(0, '767.040'), (1, '1000.000')] [2023-12-27 10:29:57,394][20381] Updated weights for policy 1, policy_version 703579 (0.0009) [2023-12-27 10:29:57,448][20381] Updated weights for policy 1, policy_version 703589 (0.0005) [2023-12-27 10:29:57,498][20381] Updated weights for policy 1, policy_version 703599 (0.0005) [2023-12-27 10:29:57,896][20347] Updated weights for policy 0, policy_version 704077 (0.0007) [2023-12-27 10:29:57,940][20347] Updated weights for policy 0, policy_version 704087 (0.0008) [2023-12-27 10:29:57,993][20347] Updated weights for policy 0, policy_version 704097 (0.0009) [2023-12-27 10:29:58,136][20381] Updated weights for policy 1, policy_version 703609 (0.0008) [2023-12-27 10:29:58,197][20381] Updated weights for policy 1, policy_version 703619 (0.0008) [2023-12-27 10:29:58,258][20381] Updated weights for policy 1, policy_version 703629 (0.0009) [2023-12-27 10:29:58,311][20381] Updated weights for policy 1, policy_version 703639 (0.0011) [2023-12-27 10:29:58,705][20347] Updated weights for policy 0, policy_version 704107 (0.0009) [2023-12-27 10:29:58,752][20347] Updated weights for policy 0, policy_version 704117 (0.0010) [2023-12-27 10:29:58,806][20347] Updated weights for policy 0, policy_version 704127 (0.0010) [2023-12-27 10:29:59,045][20381] Updated weights for policy 1, policy_version 703649 (0.0007) [2023-12-27 10:29:59,104][20381] Updated weights for policy 1, policy_version 703659 (0.0009) [2023-12-27 10:29:59,158][20381] Updated weights for policy 1, policy_version 703669 (0.0010) [2023-12-27 10:29:59,527][20347] Updated weights for policy 0, policy_version 704137 (0.0008) [2023-12-27 10:29:59,584][20347] Updated weights for policy 0, policy_version 704147 (0.0010) [2023-12-27 10:29:59,632][20347] Updated weights for policy 0, policy_version 704157 (0.0008) [2023-12-27 10:29:59,691][20347] Updated weights for policy 0, policy_version 704167 (0.0009) [2023-12-27 10:29:59,889][20381] Updated weights for policy 1, policy_version 703679 (0.0007) [2023-12-27 10:29:59,957][20381] Updated weights for policy 1, policy_version 703689 (0.0008) [2023-12-27 10:30:00,006][20381] Updated weights for policy 1, policy_version 703699 (0.0008) [2023-12-27 10:30:00,380][20347] Updated weights for policy 0, policy_version 704177 (0.0006) [2023-12-27 10:30:00,443][20347] Updated weights for policy 0, policy_version 704187 (0.0005) [2023-12-27 10:30:00,504][20347] Updated weights for policy 0, policy_version 704197 (0.0008) [2023-12-27 10:30:00,691][20381] Updated weights for policy 1, policy_version 703709 (0.0006) [2023-12-27 10:30:00,740][20381] Updated weights for policy 1, policy_version 703719 (0.0006) [2023-12-27 10:30:00,788][20381] Updated weights for policy 1, policy_version 703729 (0.0007) [2023-12-27 10:30:01,123][20347] Updated weights for policy 0, policy_version 704207 (0.0009) [2023-12-27 10:30:01,178][20347] Updated weights for policy 0, policy_version 704217 (0.0008) [2023-12-27 10:30:01,244][20347] Updated weights for policy 0, policy_version 704227 (0.0008) [2023-12-27 10:30:01,491][20381] Updated weights for policy 1, policy_version 703739 (0.0007) [2023-12-27 10:30:01,545][20381] Updated weights for policy 1, policy_version 703749 (0.0006) [2023-12-27 10:30:01,597][20381] Updated weights for policy 1, policy_version 703759 (0.0005) [2023-12-27 10:30:01,976][20347] Updated weights for policy 0, policy_version 704237 (0.0007) [2023-12-27 10:30:02,039][20347] Updated weights for policy 0, policy_version 704247 (0.0008) [2023-12-27 10:30:02,103][20347] Updated weights for policy 0, policy_version 704257 (0.0007) [2023-12-27 10:30:02,187][20381] Updated weights for policy 1, policy_version 703769 (0.0008) [2023-12-27 10:30:02,246][20381] Updated weights for policy 1, policy_version 703779 (0.0006) [2023-12-27 10:30:02,305][20381] Updated weights for policy 1, policy_version 703789 (0.0006) [2023-12-27 10:30:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 360505344. Throughput: 0: 10434.3, 1: 10501.7. Samples: 360518820. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:02,363][19170] Avg episode reward: [(0, '893.090'), (1, '970.870')] [2023-12-27 10:30:02,370][20381] Updated weights for policy 1, policy_version 703799 (0.0008) [2023-12-27 10:30:02,845][20347] Updated weights for policy 0, policy_version 704267 (0.0006) [2023-12-27 10:30:02,899][20347] Updated weights for policy 0, policy_version 704277 (0.0005) [2023-12-27 10:30:02,948][20347] Updated weights for policy 0, policy_version 704287 (0.0005) [2023-12-27 10:30:02,956][20381] Updated weights for policy 1, policy_version 703809 (0.0006) [2023-12-27 10:30:03,013][20381] Updated weights for policy 1, policy_version 703819 (0.0006) [2023-12-27 10:30:03,066][20381] Updated weights for policy 1, policy_version 703829 (0.0005) [2023-12-27 10:30:03,525][20347] Updated weights for policy 0, policy_version 704297 (0.0005) [2023-12-27 10:30:03,576][20347] Updated weights for policy 0, policy_version 704307 (0.0005) [2023-12-27 10:30:03,630][20347] Updated weights for policy 0, policy_version 704317 (0.0005) [2023-12-27 10:30:03,683][20347] Updated weights for policy 0, policy_version 704327 (0.0005) [2023-12-27 10:30:03,694][20381] Updated weights for policy 1, policy_version 703839 (0.0005) [2023-12-27 10:30:03,748][20381] Updated weights for policy 1, policy_version 703849 (0.0007) [2023-12-27 10:30:03,796][20381] Updated weights for policy 1, policy_version 703859 (0.0010) [2023-12-27 10:30:04,266][20347] Updated weights for policy 0, policy_version 704337 (0.0010) [2023-12-27 10:30:04,330][20347] Updated weights for policy 0, policy_version 704347 (0.0010) [2023-12-27 10:30:04,382][20347] Updated weights for policy 0, policy_version 704357 (0.0010) [2023-12-27 10:30:04,505][20381] Updated weights for policy 1, policy_version 703869 (0.0008) [2023-12-27 10:30:04,562][20381] Updated weights for policy 1, policy_version 703879 (0.0005) [2023-12-27 10:30:04,619][20381] Updated weights for policy 1, policy_version 703889 (0.0009) [2023-12-27 10:30:04,979][20347] Updated weights for policy 0, policy_version 704367 (0.0009) [2023-12-27 10:30:05,047][20347] Updated weights for policy 0, policy_version 704377 (0.0007) [2023-12-27 10:30:05,111][20347] Updated weights for policy 0, policy_version 704387 (0.0008) [2023-12-27 10:30:05,280][20381] Updated weights for policy 1, policy_version 703899 (0.0009) [2023-12-27 10:30:05,326][20381] Updated weights for policy 1, policy_version 703909 (0.0005) [2023-12-27 10:30:05,376][20381] Updated weights for policy 1, policy_version 703919 (0.0006) [2023-12-27 10:30:05,710][20347] Updated weights for policy 0, policy_version 704397 (0.0007) [2023-12-27 10:30:05,769][20347] Updated weights for policy 0, policy_version 704407 (0.0005) [2023-12-27 10:30:05,827][20347] Updated weights for policy 0, policy_version 704417 (0.0005) [2023-12-27 10:30:06,014][20381] Updated weights for policy 1, policy_version 703929 (0.0008) [2023-12-27 10:30:06,067][20381] Updated weights for policy 1, policy_version 703939 (0.0007) [2023-12-27 10:30:06,121][20381] Updated weights for policy 1, policy_version 703949 (0.0007) [2023-12-27 10:30:06,179][20381] Updated weights for policy 1, policy_version 703959 (0.0006) [2023-12-27 10:30:06,479][20347] Updated weights for policy 0, policy_version 704427 (0.0007) [2023-12-27 10:30:06,531][20347] Updated weights for policy 0, policy_version 704437 (0.0007) [2023-12-27 10:30:06,593][20347] Updated weights for policy 0, policy_version 704447 (0.0008) [2023-12-27 10:30:06,848][20381] Updated weights for policy 1, policy_version 703969 (0.0008) [2023-12-27 10:30:06,903][20381] Updated weights for policy 1, policy_version 703979 (0.0008) [2023-12-27 10:30:06,951][20381] Updated weights for policy 1, policy_version 703989 (0.0007) [2023-12-27 10:30:07,307][20347] Updated weights for policy 0, policy_version 704457 (0.0008) [2023-12-27 10:30:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20889.5, 300 sec: 20827.1). Total num frames: 360611840. Throughput: 0: 10476.3, 1: 10518.2. Samples: 360583680. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:07,363][19170] Avg episode reward: [(0, '903.550'), (1, '961.070')] [2023-12-27 10:30:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000703992_180248576.pth... [2023-12-27 10:30:07,373][20347] Updated weights for policy 0, policy_version 704467 (0.0009) [2023-12-27 10:30:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000702776_179937280.pth [2023-12-27 10:30:07,436][20347] Updated weights for policy 0, policy_version 704477 (0.0007) [2023-12-27 10:30:07,495][20347] Updated weights for policy 0, policy_version 704487 (0.0008) [2023-12-27 10:30:07,500][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000704488_180371456.pth... [2023-12-27 10:30:07,503][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000703240_180051968.pth [2023-12-27 10:30:07,639][20381] Updated weights for policy 1, policy_version 703999 (0.0007) [2023-12-27 10:30:07,692][20381] Updated weights for policy 1, policy_version 704009 (0.0007) [2023-12-27 10:30:07,755][20381] Updated weights for policy 1, policy_version 704019 (0.0007) [2023-12-27 10:30:08,106][20347] Updated weights for policy 0, policy_version 704497 (0.0007) [2023-12-27 10:30:08,164][20347] Updated weights for policy 0, policy_version 704507 (0.0005) [2023-12-27 10:30:08,221][20347] Updated weights for policy 0, policy_version 704517 (0.0010) [2023-12-27 10:30:08,461][20381] Updated weights for policy 1, policy_version 704029 (0.0008) [2023-12-27 10:30:08,513][20381] Updated weights for policy 1, policy_version 704039 (0.0008) [2023-12-27 10:30:08,567][20381] Updated weights for policy 1, policy_version 704049 (0.0007) [2023-12-27 10:30:08,848][20347] Updated weights for policy 0, policy_version 704527 (0.0007) [2023-12-27 10:30:08,909][20347] Updated weights for policy 0, policy_version 704537 (0.0006) [2023-12-27 10:30:08,967][20347] Updated weights for policy 0, policy_version 704547 (0.0010) [2023-12-27 10:30:09,229][20381] Updated weights for policy 1, policy_version 704059 (0.0007) [2023-12-27 10:30:09,294][20381] Updated weights for policy 1, policy_version 704069 (0.0008) [2023-12-27 10:30:09,358][20381] Updated weights for policy 1, policy_version 704079 (0.0007) [2023-12-27 10:30:09,615][20347] Updated weights for policy 0, policy_version 704557 (0.0011) [2023-12-27 10:30:09,672][20347] Updated weights for policy 0, policy_version 704567 (0.0011) [2023-12-27 10:30:09,737][20347] Updated weights for policy 0, policy_version 704577 (0.0009) [2023-12-27 10:30:10,136][20381] Updated weights for policy 1, policy_version 704089 (0.0007) [2023-12-27 10:30:10,203][20381] Updated weights for policy 1, policy_version 704099 (0.0008) [2023-12-27 10:30:10,266][20381] Updated weights for policy 1, policy_version 704109 (0.0007) [2023-12-27 10:30:10,331][20381] Updated weights for policy 1, policy_version 704119 (0.0006) [2023-12-27 10:30:10,380][20347] Updated weights for policy 0, policy_version 704587 (0.0007) [2023-12-27 10:30:10,433][20347] Updated weights for policy 0, policy_version 704597 (0.0007) [2023-12-27 10:30:10,489][20347] Updated weights for policy 0, policy_version 704607 (0.0008) [2023-12-27 10:30:11,050][20381] Updated weights for policy 1, policy_version 704129 (0.0006) [2023-12-27 10:30:11,105][20347] Updated weights for policy 0, policy_version 704617 (0.0008) [2023-12-27 10:30:11,110][20381] Updated weights for policy 1, policy_version 704139 (0.0008) [2023-12-27 10:30:11,162][20347] Updated weights for policy 0, policy_version 704627 (0.0008) [2023-12-27 10:30:11,170][20381] Updated weights for policy 1, policy_version 704149 (0.0008) [2023-12-27 10:30:11,213][20347] Updated weights for policy 0, policy_version 704637 (0.0006) [2023-12-27 10:30:11,275][20347] Updated weights for policy 0, policy_version 704647 (0.0008) [2023-12-27 10:30:11,895][20381] Updated weights for policy 1, policy_version 704159 (0.0006) [2023-12-27 10:30:11,955][20381] Updated weights for policy 1, policy_version 704169 (0.0006) [2023-12-27 10:30:12,001][20347] Updated weights for policy 0, policy_version 704657 (0.0009) [2023-12-27 10:30:12,012][20381] Updated weights for policy 1, policy_version 704179 (0.0006) [2023-12-27 10:30:12,054][20347] Updated weights for policy 0, policy_version 704667 (0.0009) [2023-12-27 10:30:12,119][20347] Updated weights for policy 0, policy_version 704677 (0.0009) [2023-12-27 10:30:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 360718336. Throughput: 0: 10490.9, 1: 10435.8. Samples: 360707620. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:12,363][19170] Avg episode reward: [(0, '913.340'), (1, '951.500')] [2023-12-27 10:30:12,698][20381] Updated weights for policy 1, policy_version 704189 (0.0007) [2023-12-27 10:30:12,745][20381] Updated weights for policy 1, policy_version 704199 (0.0008) [2023-12-27 10:30:12,793][20292] KL-divergence is very high: 356.2330 [2023-12-27 10:30:12,797][20381] Updated weights for policy 1, policy_version 704209 (0.0006) [2023-12-27 10:30:12,878][20347] Updated weights for policy 0, policy_version 704687 (0.0010) [2023-12-27 10:30:12,934][20347] Updated weights for policy 0, policy_version 704697 (0.0010) [2023-12-27 10:30:12,984][20347] Updated weights for policy 0, policy_version 704707 (0.0010) [2023-12-27 10:30:13,498][20381] Updated weights for policy 1, policy_version 704219 (0.0008) [2023-12-27 10:30:13,547][20381] Updated weights for policy 1, policy_version 704229 (0.0005) [2023-12-27 10:30:13,599][20381] Updated weights for policy 1, policy_version 704239 (0.0006) [2023-12-27 10:30:13,631][20347] Updated weights for policy 0, policy_version 704717 (0.0010) [2023-12-27 10:30:13,676][20347] Updated weights for policy 0, policy_version 704727 (0.0010) [2023-12-27 10:30:13,724][20347] Updated weights for policy 0, policy_version 704737 (0.0010) [2023-12-27 10:30:14,161][20381] Updated weights for policy 1, policy_version 704249 (0.0006) [2023-12-27 10:30:14,221][20381] Updated weights for policy 1, policy_version 704259 (0.0006) [2023-12-27 10:30:14,285][20381] Updated weights for policy 1, policy_version 704269 (0.0006) [2023-12-27 10:30:14,344][20381] Updated weights for policy 1, policy_version 704279 (0.0006) [2023-12-27 10:30:14,390][20347] Updated weights for policy 0, policy_version 704747 (0.0009) [2023-12-27 10:30:14,452][20347] Updated weights for policy 0, policy_version 704757 (0.0005) [2023-12-27 10:30:14,500][20347] Updated weights for policy 0, policy_version 704767 (0.0005) [2023-12-27 10:30:15,041][20381] Updated weights for policy 1, policy_version 704289 (0.0008) [2023-12-27 10:30:15,111][20381] Updated weights for policy 1, policy_version 704299 (0.0009) [2023-12-27 10:30:15,116][20347] Updated weights for policy 0, policy_version 704777 (0.0009) [2023-12-27 10:30:15,172][20381] Updated weights for policy 1, policy_version 704309 (0.0008) [2023-12-27 10:30:15,182][20347] Updated weights for policy 0, policy_version 704787 (0.0006) [2023-12-27 10:30:15,242][20347] Updated weights for policy 0, policy_version 704797 (0.0006) [2023-12-27 10:30:15,303][20347] Updated weights for policy 0, policy_version 704807 (0.0005) [2023-12-27 10:30:15,844][20381] Updated weights for policy 1, policy_version 704319 (0.0007) [2023-12-27 10:30:15,897][20381] Updated weights for policy 1, policy_version 704329 (0.0005) [2023-12-27 10:30:15,945][20347] Updated weights for policy 0, policy_version 704817 (0.0006) [2023-12-27 10:30:15,960][20381] Updated weights for policy 1, policy_version 704339 (0.0005) [2023-12-27 10:30:15,998][20347] Updated weights for policy 0, policy_version 704827 (0.0008) [2023-12-27 10:30:16,045][20347] Updated weights for policy 0, policy_version 704837 (0.0008) [2023-12-27 10:30:16,567][20381] Updated weights for policy 1, policy_version 704349 (0.0010) [2023-12-27 10:30:16,613][20381] Updated weights for policy 1, policy_version 704359 (0.0010) [2023-12-27 10:30:16,658][20381] Updated weights for policy 1, policy_version 704369 (0.0008) [2023-12-27 10:30:16,719][20347] Updated weights for policy 0, policy_version 704847 (0.0007) [2023-12-27 10:30:16,767][20347] Updated weights for policy 0, policy_version 704857 (0.0007) [2023-12-27 10:30:16,808][20347] Updated weights for policy 0, policy_version 704867 (0.0007) [2023-12-27 10:30:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20827.1). Total num frames: 360816640. Throughput: 0: 10466.4, 1: 10322.3. Samples: 360830964. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:17,364][19170] Avg episode reward: [(0, '961.490'), (1, '923.010')] [2023-12-27 10:30:17,407][20347] Updated weights for policy 0, policy_version 704877 (0.0008) [2023-12-27 10:30:17,453][20381] Updated weights for policy 1, policy_version 704379 (0.0008) [2023-12-27 10:30:17,465][20347] Updated weights for policy 0, policy_version 704887 (0.0008) [2023-12-27 10:30:17,510][20381] Updated weights for policy 1, policy_version 704389 (0.0005) [2023-12-27 10:30:17,511][20347] Updated weights for policy 0, policy_version 704897 (0.0008) [2023-12-27 10:30:17,566][20381] Updated weights for policy 1, policy_version 704399 (0.0005) [2023-12-27 10:30:18,154][20381] Updated weights for policy 1, policy_version 704409 (0.0006) [2023-12-27 10:30:18,156][20347] Updated weights for policy 0, policy_version 704907 (0.0006) [2023-12-27 10:30:18,212][20347] Updated weights for policy 0, policy_version 704917 (0.0006) [2023-12-27 10:30:18,218][20381] Updated weights for policy 1, policy_version 704419 (0.0008) [2023-12-27 10:30:18,273][20347] Updated weights for policy 0, policy_version 704927 (0.0006) [2023-12-27 10:30:18,274][20381] Updated weights for policy 1, policy_version 704429 (0.0007) [2023-12-27 10:30:18,285][20291] KL-divergence is very high: 200.1928 [2023-12-27 10:30:18,341][20381] Updated weights for policy 1, policy_version 704439 (0.0007) [2023-12-27 10:30:19,006][20347] Updated weights for policy 0, policy_version 704937 (0.0008) [2023-12-27 10:30:19,017][20381] Updated weights for policy 1, policy_version 704449 (0.0007) [2023-12-27 10:30:19,065][20347] Updated weights for policy 0, policy_version 704947 (0.0010) [2023-12-27 10:30:19,067][20381] Updated weights for policy 1, policy_version 704459 (0.0006) [2023-12-27 10:30:19,118][20381] Updated weights for policy 1, policy_version 704469 (0.0011) [2023-12-27 10:30:19,121][20347] Updated weights for policy 0, policy_version 704957 (0.0010) [2023-12-27 10:30:19,180][20347] Updated weights for policy 0, policy_version 704967 (0.0010) [2023-12-27 10:30:19,783][20381] Updated weights for policy 1, policy_version 704479 (0.0010) [2023-12-27 10:30:19,845][20381] Updated weights for policy 1, policy_version 704489 (0.0011) [2023-12-27 10:30:19,862][20347] Updated weights for policy 0, policy_version 704977 (0.0007) [2023-12-27 10:30:19,904][20381] Updated weights for policy 1, policy_version 704499 (0.0008) [2023-12-27 10:30:19,927][20347] Updated weights for policy 0, policy_version 704987 (0.0006) [2023-12-27 10:30:19,988][20347] Updated weights for policy 0, policy_version 704997 (0.0006) [2023-12-27 10:30:20,533][20381] Updated weights for policy 1, policy_version 704509 (0.0010) [2023-12-27 10:30:20,591][20381] Updated weights for policy 1, policy_version 704519 (0.0010) [2023-12-27 10:30:20,660][20381] Updated weights for policy 1, policy_version 704529 (0.0006) [2023-12-27 10:30:20,716][20347] Updated weights for policy 0, policy_version 705007 (0.0007) [2023-12-27 10:30:20,766][20347] Updated weights for policy 0, policy_version 705017 (0.0006) [2023-12-27 10:30:20,825][20347] Updated weights for policy 0, policy_version 705027 (0.0005) [2023-12-27 10:30:21,336][20381] Updated weights for policy 1, policy_version 704539 (0.0009) [2023-12-27 10:30:21,374][20292] KL-divergence is very high: 222.5963 [2023-12-27 10:30:21,406][20381] Updated weights for policy 1, policy_version 704549 (0.0009) [2023-12-27 10:30:21,468][20381] Updated weights for policy 1, policy_version 704559 (0.0007) [2023-12-27 10:30:21,509][20347] Updated weights for policy 0, policy_version 705037 (0.0006) [2023-12-27 10:30:21,568][20347] Updated weights for policy 0, policy_version 705047 (0.0009) [2023-12-27 10:30:21,631][20347] Updated weights for policy 0, policy_version 705057 (0.0010) [2023-12-27 10:30:22,240][20381] Updated weights for policy 1, policy_version 704569 (0.0010) [2023-12-27 10:30:22,300][20381] Updated weights for policy 1, policy_version 704579 (0.0009) [2023-12-27 10:30:22,361][20381] Updated weights for policy 1, policy_version 704589 (0.0010) [2023-12-27 10:30:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 360914944. Throughput: 0: 10459.1, 1: 10352.7. Samples: 360893292. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:22,363][19170] Avg episode reward: [(0, '980.470'), (1, '942.270')] [2023-12-27 10:30:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000705064_180518912.pth... [2023-12-27 10:30:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000703848_180207616.pth [2023-12-27 10:30:22,414][20347] Updated weights for policy 0, policy_version 705067 (0.0008) [2023-12-27 10:30:22,425][20381] Updated weights for policy 1, policy_version 704599 (0.0011) [2023-12-27 10:30:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000704600_180404224.pth... [2023-12-27 10:30:22,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000703384_180092928.pth [2023-12-27 10:30:22,471][20347] Updated weights for policy 0, policy_version 705077 (0.0008) [2023-12-27 10:30:22,533][20347] Updated weights for policy 0, policy_version 705087 (0.0009) [2023-12-27 10:30:23,151][20347] Updated weights for policy 0, policy_version 705097 (0.0007) [2023-12-27 10:30:23,152][20381] Updated weights for policy 1, policy_version 704609 (0.0008) [2023-12-27 10:30:23,185][20292] KL-divergence is very high: 2363.7412 [2023-12-27 10:30:23,206][20381] Updated weights for policy 1, policy_version 704619 (0.0007) [2023-12-27 10:30:23,217][20347] Updated weights for policy 0, policy_version 705107 (0.0010) [2023-12-27 10:30:23,257][20381] Updated weights for policy 1, policy_version 704629 (0.0010) [2023-12-27 10:30:23,278][20347] Updated weights for policy 0, policy_version 705117 (0.0010) [2023-12-27 10:30:23,334][20347] Updated weights for policy 0, policy_version 705127 (0.0010) [2023-12-27 10:30:23,913][20381] Updated weights for policy 1, policy_version 704639 (0.0008) [2023-12-27 10:30:23,974][20381] Updated weights for policy 1, policy_version 704649 (0.0007) [2023-12-27 10:30:24,005][20347] Updated weights for policy 0, policy_version 705137 (0.0011) [2023-12-27 10:30:24,039][20381] Updated weights for policy 1, policy_version 704659 (0.0007) [2023-12-27 10:30:24,060][20347] Updated weights for policy 0, policy_version 705147 (0.0010) [2023-12-27 10:30:24,119][20347] Updated weights for policy 0, policy_version 705157 (0.0010) [2023-12-27 10:30:24,693][20381] Updated weights for policy 1, policy_version 704669 (0.0007) [2023-12-27 10:30:24,754][20381] Updated weights for policy 1, policy_version 704679 (0.0007) [2023-12-27 10:30:24,809][20381] Updated weights for policy 1, policy_version 704689 (0.0006) [2023-12-27 10:30:24,810][20347] Updated weights for policy 0, policy_version 705167 (0.0011) [2023-12-27 10:30:24,868][20347] Updated weights for policy 0, policy_version 705177 (0.0010) [2023-12-27 10:30:24,927][20347] Updated weights for policy 0, policy_version 705187 (0.0005) [2023-12-27 10:30:25,494][20347] Updated weights for policy 0, policy_version 705197 (0.0008) [2023-12-27 10:30:25,549][20381] Updated weights for policy 1, policy_version 704699 (0.0006) [2023-12-27 10:30:25,559][20347] Updated weights for policy 0, policy_version 705207 (0.0011) [2023-12-27 10:30:25,610][20381] Updated weights for policy 1, policy_version 704709 (0.0006) [2023-12-27 10:30:25,621][20347] Updated weights for policy 0, policy_version 705217 (0.0009) [2023-12-27 10:30:25,668][20381] Updated weights for policy 1, policy_version 704719 (0.0006) [2023-12-27 10:30:26,286][20347] Updated weights for policy 0, policy_version 705227 (0.0010) [2023-12-27 10:30:26,341][20347] Updated weights for policy 0, policy_version 705237 (0.0010) [2023-12-27 10:30:26,360][20381] Updated weights for policy 1, policy_version 704729 (0.0008) [2023-12-27 10:30:26,394][20347] Updated weights for policy 0, policy_version 705247 (0.0010) [2023-12-27 10:30:26,414][20381] Updated weights for policy 1, policy_version 704739 (0.0006) [2023-12-27 10:30:26,466][20381] Updated weights for policy 1, policy_version 704749 (0.0006) [2023-12-27 10:30:26,513][20381] Updated weights for policy 1, policy_version 704759 (0.0007) [2023-12-27 10:30:27,035][20347] Updated weights for policy 0, policy_version 705257 (0.0010) [2023-12-27 10:30:27,098][20347] Updated weights for policy 0, policy_version 705267 (0.0005) [2023-12-27 10:30:27,156][20381] Updated weights for policy 1, policy_version 704769 (0.0007) [2023-12-27 10:30:27,160][20347] Updated weights for policy 0, policy_version 705277 (0.0005) [2023-12-27 10:30:27,192][20292] KL-divergence is very high: 200.2351 [2023-12-27 10:30:27,213][20381] Updated weights for policy 1, policy_version 704779 (0.0007) [2023-12-27 10:30:27,226][20347] Updated weights for policy 0, policy_version 705287 (0.0010) [2023-12-27 10:30:27,243][20292] KL-divergence is very high: 191.7743 [2023-12-27 10:30:27,270][20381] Updated weights for policy 1, policy_version 704789 (0.0005) [2023-12-27 10:30:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.5, 300 sec: 20799.3). Total num frames: 361029632. Throughput: 0: 10383.9, 1: 10340.7. Samples: 361013296. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:27,363][19170] Avg episode reward: [(0, '951.170'), (1, '912.590')] [2023-12-27 10:30:27,832][20381] Updated weights for policy 1, policy_version 704799 (0.0007) [2023-12-27 10:30:27,867][20292] KL-divergence is very high: 116.3964 [2023-12-27 10:30:27,872][20347] Updated weights for policy 0, policy_version 705297 (0.0006) [2023-12-27 10:30:27,889][20381] Updated weights for policy 1, policy_version 704809 (0.0006) [2023-12-27 10:30:27,928][20347] Updated weights for policy 0, policy_version 705307 (0.0006) [2023-12-27 10:30:27,948][20381] Updated weights for policy 1, policy_version 704819 (0.0006) [2023-12-27 10:30:27,987][20347] Updated weights for policy 0, policy_version 705317 (0.0007) [2023-12-27 10:30:28,632][20381] Updated weights for policy 1, policy_version 704829 (0.0007) [2023-12-27 10:30:28,675][20347] Updated weights for policy 0, policy_version 705327 (0.0007) [2023-12-27 10:30:28,684][20381] Updated weights for policy 1, policy_version 704839 (0.0006) [2023-12-27 10:30:28,725][20347] Updated weights for policy 0, policy_version 705337 (0.0006) [2023-12-27 10:30:28,742][20381] Updated weights for policy 1, policy_version 704849 (0.0006) [2023-12-27 10:30:28,782][20347] Updated weights for policy 0, policy_version 705347 (0.0006) [2023-12-27 10:30:29,426][20381] Updated weights for policy 1, policy_version 704859 (0.0007) [2023-12-27 10:30:29,439][20347] Updated weights for policy 0, policy_version 705357 (0.0007) [2023-12-27 10:30:29,488][20381] Updated weights for policy 1, policy_version 704869 (0.0009) [2023-12-27 10:30:29,502][20347] Updated weights for policy 0, policy_version 705367 (0.0005) [2023-12-27 10:30:29,550][20381] Updated weights for policy 1, policy_version 704879 (0.0006) [2023-12-27 10:30:29,561][20347] Updated weights for policy 0, policy_version 705377 (0.0006) [2023-12-27 10:30:30,165][20347] Updated weights for policy 0, policy_version 705387 (0.0008) [2023-12-27 10:30:30,221][20381] Updated weights for policy 1, policy_version 704889 (0.0006) [2023-12-27 10:30:30,227][20347] Updated weights for policy 0, policy_version 705397 (0.0007) [2023-12-27 10:30:30,276][20381] Updated weights for policy 1, policy_version 704899 (0.0006) [2023-12-27 10:30:30,278][20347] Updated weights for policy 0, policy_version 705407 (0.0007) [2023-12-27 10:30:30,338][20381] Updated weights for policy 1, policy_version 704909 (0.0006) [2023-12-27 10:30:30,402][20381] Updated weights for policy 1, policy_version 704919 (0.0006) [2023-12-27 10:30:30,924][20347] Updated weights for policy 0, policy_version 705417 (0.0007) [2023-12-27 10:30:30,977][20347] Updated weights for policy 0, policy_version 705427 (0.0005) [2023-12-27 10:30:31,037][20347] Updated weights for policy 0, policy_version 705437 (0.0006) [2023-12-27 10:30:31,066][20381] Updated weights for policy 1, policy_version 704929 (0.0009) [2023-12-27 10:30:31,118][20347] Updated weights for policy 0, policy_version 705447 (0.0008) [2023-12-27 10:30:31,124][20381] Updated weights for policy 1, policy_version 704939 (0.0008) [2023-12-27 10:30:31,184][20381] Updated weights for policy 1, policy_version 704949 (0.0008) [2023-12-27 10:30:31,777][20347] Updated weights for policy 0, policy_version 705457 (0.0008) [2023-12-27 10:30:31,836][20347] Updated weights for policy 0, policy_version 705467 (0.0007) [2023-12-27 10:30:31,862][20381] Updated weights for policy 1, policy_version 704959 (0.0009) [2023-12-27 10:30:31,888][20347] Updated weights for policy 0, policy_version 705477 (0.0009) [2023-12-27 10:30:31,923][20381] Updated weights for policy 1, policy_version 704969 (0.0007) [2023-12-27 10:30:31,984][20381] Updated weights for policy 1, policy_version 704979 (0.0008) [2023-12-27 10:30:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20771.6). Total num frames: 361127936. Throughput: 0: 10395.2, 1: 10373.9. Samples: 361138480. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:32,363][19170] Avg episode reward: [(0, '951.170'), (1, '902.860')] [2023-12-27 10:30:32,601][20347] Updated weights for policy 0, policy_version 705487 (0.0007) [2023-12-27 10:30:32,634][20381] Updated weights for policy 1, policy_version 704989 (0.0006) [2023-12-27 10:30:32,655][20347] Updated weights for policy 0, policy_version 705497 (0.0007) [2023-12-27 10:30:32,695][20381] Updated weights for policy 1, policy_version 704999 (0.0006) [2023-12-27 10:30:32,709][20347] Updated weights for policy 0, policy_version 705507 (0.0005) [2023-12-27 10:30:32,753][20381] Updated weights for policy 1, policy_version 705009 (0.0007) [2023-12-27 10:30:33,379][20347] Updated weights for policy 0, policy_version 705517 (0.0006) [2023-12-27 10:30:33,429][20347] Updated weights for policy 0, policy_version 705527 (0.0005) [2023-12-27 10:30:33,483][20381] Updated weights for policy 1, policy_version 705019 (0.0008) [2023-12-27 10:30:33,487][20347] Updated weights for policy 0, policy_version 705537 (0.0005) [2023-12-27 10:30:33,530][20381] Updated weights for policy 1, policy_version 705029 (0.0006) [2023-12-27 10:30:33,575][20381] Updated weights for policy 1, policy_version 705039 (0.0007) [2023-12-27 10:30:34,058][20347] Updated weights for policy 0, policy_version 705547 (0.0006) [2023-12-27 10:30:34,120][20347] Updated weights for policy 0, policy_version 705557 (0.0008) [2023-12-27 10:30:34,185][20347] Updated weights for policy 0, policy_version 705567 (0.0008) [2023-12-27 10:30:34,282][20381] Updated weights for policy 1, policy_version 705049 (0.0006) [2023-12-27 10:30:34,341][20381] Updated weights for policy 1, policy_version 705059 (0.0007) [2023-12-27 10:30:34,399][20381] Updated weights for policy 1, policy_version 705069 (0.0007) [2023-12-27 10:30:34,456][20381] Updated weights for policy 1, policy_version 705079 (0.0007) [2023-12-27 10:30:34,815][20347] Updated weights for policy 0, policy_version 705577 (0.0008) [2023-12-27 10:30:34,867][20347] Updated weights for policy 0, policy_version 705587 (0.0005) [2023-12-27 10:30:34,927][20347] Updated weights for policy 0, policy_version 705597 (0.0006) [2023-12-27 10:30:34,983][20347] Updated weights for policy 0, policy_version 705607 (0.0008) [2023-12-27 10:30:35,197][20381] Updated weights for policy 1, policy_version 705089 (0.0007) [2023-12-27 10:30:35,246][20381] Updated weights for policy 1, policy_version 705099 (0.0008) [2023-12-27 10:30:35,294][20381] Updated weights for policy 1, policy_version 705109 (0.0007) [2023-12-27 10:30:35,536][20347] Updated weights for policy 0, policy_version 705617 (0.0005) [2023-12-27 10:30:35,594][20347] Updated weights for policy 0, policy_version 705627 (0.0005) [2023-12-27 10:30:35,651][20347] Updated weights for policy 0, policy_version 705637 (0.0005) [2023-12-27 10:30:35,943][20381] Updated weights for policy 1, policy_version 705119 (0.0006) [2023-12-27 10:30:35,991][20381] Updated weights for policy 1, policy_version 705129 (0.0005) [2023-12-27 10:30:36,043][20381] Updated weights for policy 1, policy_version 705139 (0.0006) [2023-12-27 10:30:36,282][20347] Updated weights for policy 0, policy_version 705647 (0.0007) [2023-12-27 10:30:36,344][20347] Updated weights for policy 0, policy_version 705657 (0.0009) [2023-12-27 10:30:36,413][20347] Updated weights for policy 0, policy_version 705667 (0.0008) [2023-12-27 10:30:36,657][20381] Updated weights for policy 1, policy_version 705149 (0.0007) [2023-12-27 10:30:36,717][20381] Updated weights for policy 1, policy_version 705159 (0.0007) [2023-12-27 10:30:36,773][20381] Updated weights for policy 1, policy_version 705169 (0.0007) [2023-12-27 10:30:37,069][20347] Updated weights for policy 0, policy_version 705677 (0.0006) [2023-12-27 10:30:37,113][20347] Updated weights for policy 0, policy_version 705687 (0.0005) [2023-12-27 10:30:37,164][20347] Updated weights for policy 0, policy_version 705697 (0.0006) [2023-12-27 10:30:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20799.3). Total num frames: 361234432. Throughput: 0: 10436.7, 1: 10302.3. Samples: 361200312. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:37,363][19170] Avg episode reward: [(0, '941.610'), (1, '941.840')] [2023-12-27 10:30:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000705176_180551680.pth... [2023-12-27 10:30:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000705704_180682752.pth... [2023-12-27 10:30:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000703992_180248576.pth [2023-12-27 10:30:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000704488_180371456.pth [2023-12-27 10:30:37,409][20381] Updated weights for policy 1, policy_version 705179 (0.0008) [2023-12-27 10:30:37,466][20381] Updated weights for policy 1, policy_version 705189 (0.0011) [2023-12-27 10:30:37,520][20381] Updated weights for policy 1, policy_version 705199 (0.0011) [2023-12-27 10:30:37,788][20347] Updated weights for policy 0, policy_version 705707 (0.0005) [2023-12-27 10:30:37,847][20347] Updated weights for policy 0, policy_version 705717 (0.0005) [2023-12-27 10:30:37,895][20347] Updated weights for policy 0, policy_version 705727 (0.0008) [2023-12-27 10:30:38,168][20381] Updated weights for policy 1, policy_version 705209 (0.0011) [2023-12-27 10:30:38,232][20381] Updated weights for policy 1, policy_version 705219 (0.0010) [2023-12-27 10:30:38,289][20381] Updated weights for policy 1, policy_version 705229 (0.0010) [2023-12-27 10:30:38,350][20381] Updated weights for policy 1, policy_version 705239 (0.0011) [2023-12-27 10:30:38,574][20347] Updated weights for policy 0, policy_version 705737 (0.0007) [2023-12-27 10:30:38,628][20347] Updated weights for policy 0, policy_version 705747 (0.0010) [2023-12-27 10:30:38,674][20347] Updated weights for policy 0, policy_version 705757 (0.0010) [2023-12-27 10:30:38,720][20347] Updated weights for policy 0, policy_version 705767 (0.0010) [2023-12-27 10:30:38,976][20381] Updated weights for policy 1, policy_version 705249 (0.0010) [2023-12-27 10:30:39,029][20381] Updated weights for policy 1, policy_version 705259 (0.0010) [2023-12-27 10:30:39,087][20381] Updated weights for policy 1, policy_version 705269 (0.0005) [2023-12-27 10:30:39,404][20347] Updated weights for policy 0, policy_version 705777 (0.0011) [2023-12-27 10:30:39,463][20347] Updated weights for policy 0, policy_version 705787 (0.0010) [2023-12-27 10:30:39,531][20347] Updated weights for policy 0, policy_version 705797 (0.0010) [2023-12-27 10:30:39,756][20381] Updated weights for policy 1, policy_version 705279 (0.0008) [2023-12-27 10:30:39,812][20381] Updated weights for policy 1, policy_version 705289 (0.0006) [2023-12-27 10:30:39,871][20381] Updated weights for policy 1, policy_version 705299 (0.0011) [2023-12-27 10:30:40,233][20347] Updated weights for policy 0, policy_version 705807 (0.0009) [2023-12-27 10:30:40,290][20347] Updated weights for policy 0, policy_version 705817 (0.0008) [2023-12-27 10:30:40,345][20347] Updated weights for policy 0, policy_version 705827 (0.0007) [2023-12-27 10:30:40,563][20381] Updated weights for policy 1, policy_version 705309 (0.0008) [2023-12-27 10:30:40,613][20381] Updated weights for policy 1, policy_version 705319 (0.0008) [2023-12-27 10:30:40,663][20381] Updated weights for policy 1, policy_version 705329 (0.0008) [2023-12-27 10:30:41,071][20347] Updated weights for policy 0, policy_version 705837 (0.0009) [2023-12-27 10:30:41,140][20347] Updated weights for policy 0, policy_version 705847 (0.0007) [2023-12-27 10:30:41,197][20347] Updated weights for policy 0, policy_version 705857 (0.0007) [2023-12-27 10:30:41,313][20381] Updated weights for policy 1, policy_version 705339 (0.0007) [2023-12-27 10:30:41,382][20381] Updated weights for policy 1, policy_version 705349 (0.0008) [2023-12-27 10:30:41,441][20381] Updated weights for policy 1, policy_version 705359 (0.0007) [2023-12-27 10:30:41,946][20347] Updated weights for policy 0, policy_version 705867 (0.0007) [2023-12-27 10:30:42,010][20347] Updated weights for policy 0, policy_version 705877 (0.0007) [2023-12-27 10:30:42,070][20347] Updated weights for policy 0, policy_version 705887 (0.0008) [2023-12-27 10:30:42,171][20381] Updated weights for policy 1, policy_version 705369 (0.0007) [2023-12-27 10:30:42,225][20381] Updated weights for policy 1, policy_version 705379 (0.0008) [2023-12-27 10:30:42,278][20381] Updated weights for policy 1, policy_version 705389 (0.0007) [2023-12-27 10:30:42,339][20381] Updated weights for policy 1, policy_version 705399 (0.0007) [2023-12-27 10:30:42,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20771.6). Total num frames: 361340928. Throughput: 0: 10471.1, 1: 10304.6. Samples: 361327344. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:42,363][19170] Avg episode reward: [(0, '941.660'), (1, '941.790')] [2023-12-27 10:30:42,821][20347] Updated weights for policy 0, policy_version 705897 (0.0008) [2023-12-27 10:30:42,878][20347] Updated weights for policy 0, policy_version 705907 (0.0007) [2023-12-27 10:30:42,923][20347] Updated weights for policy 0, policy_version 705917 (0.0007) [2023-12-27 10:30:42,972][20347] Updated weights for policy 0, policy_version 705927 (0.0007) [2023-12-27 10:30:43,042][20381] Updated weights for policy 1, policy_version 705409 (0.0007) [2023-12-27 10:30:43,096][20381] Updated weights for policy 1, policy_version 705419 (0.0008) [2023-12-27 10:30:43,146][20381] Updated weights for policy 1, policy_version 705429 (0.0007) [2023-12-27 10:30:43,686][20347] Updated weights for policy 0, policy_version 705937 (0.0008) [2023-12-27 10:30:43,731][20347] Updated weights for policy 0, policy_version 705947 (0.0009) [2023-12-27 10:30:43,786][20381] Updated weights for policy 1, policy_version 705439 (0.0006) [2023-12-27 10:30:43,787][20347] Updated weights for policy 0, policy_version 705957 (0.0007) [2023-12-27 10:30:43,840][20381] Updated weights for policy 1, policy_version 705449 (0.0010) [2023-12-27 10:30:43,889][20381] Updated weights for policy 1, policy_version 705459 (0.0010) [2023-12-27 10:30:44,475][20381] Updated weights for policy 1, policy_version 705469 (0.0010) [2023-12-27 10:30:44,531][20381] Updated weights for policy 1, policy_version 705479 (0.0011) [2023-12-27 10:30:44,533][20347] Updated weights for policy 0, policy_version 705967 (0.0006) [2023-12-27 10:30:44,592][20381] Updated weights for policy 1, policy_version 705489 (0.0009) [2023-12-27 10:30:44,597][20347] Updated weights for policy 0, policy_version 705977 (0.0007) [2023-12-27 10:30:44,657][20347] Updated weights for policy 0, policy_version 705987 (0.0007) [2023-12-27 10:30:45,300][20381] Updated weights for policy 1, policy_version 705499 (0.0009) [2023-12-27 10:30:45,338][20347] Updated weights for policy 0, policy_version 705997 (0.0008) [2023-12-27 10:30:45,362][20381] Updated weights for policy 1, policy_version 705509 (0.0007) [2023-12-27 10:30:45,391][20347] Updated weights for policy 0, policy_version 706007 (0.0006) [2023-12-27 10:30:45,427][20381] Updated weights for policy 1, policy_version 705519 (0.0008) [2023-12-27 10:30:45,436][20347] Updated weights for policy 0, policy_version 706017 (0.0007) [2023-12-27 10:30:46,094][20381] Updated weights for policy 1, policy_version 705529 (0.0011) [2023-12-27 10:30:46,116][20347] Updated weights for policy 0, policy_version 706027 (0.0006) [2023-12-27 10:30:46,146][20381] Updated weights for policy 1, policy_version 705539 (0.0011) [2023-12-27 10:30:46,165][20347] Updated weights for policy 0, policy_version 706037 (0.0005) [2023-12-27 10:30:46,198][20381] Updated weights for policy 1, policy_version 705549 (0.0010) [2023-12-27 10:30:46,211][20347] Updated weights for policy 0, policy_version 706047 (0.0005) [2023-12-27 10:30:46,252][20381] Updated weights for policy 1, policy_version 705559 (0.0010) [2023-12-27 10:30:46,794][20347] Updated weights for policy 0, policy_version 706057 (0.0006) [2023-12-27 10:30:46,850][20347] Updated weights for policy 0, policy_version 706067 (0.0011) [2023-12-27 10:30:46,873][20381] Updated weights for policy 1, policy_version 705569 (0.0006) [2023-12-27 10:30:46,905][20347] Updated weights for policy 0, policy_version 706077 (0.0009) [2023-12-27 10:30:46,936][20381] Updated weights for policy 1, policy_version 705579 (0.0010) [2023-12-27 10:30:46,964][20347] Updated weights for policy 0, policy_version 706087 (0.0008) [2023-12-27 10:30:46,988][20381] Updated weights for policy 1, policy_version 705589 (0.0009) [2023-12-27 10:30:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20771.6). Total num frames: 361439232. Throughput: 0: 10411.0, 1: 10299.6. Samples: 361450800. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:47,363][19170] Avg episode reward: [(0, '922.160'), (1, '980.540')] [2023-12-27 10:30:47,548][20381] Updated weights for policy 1, policy_version 705599 (0.0010) [2023-12-27 10:30:47,613][20381] Updated weights for policy 1, policy_version 705609 (0.0011) [2023-12-27 10:30:47,620][20347] Updated weights for policy 0, policy_version 706097 (0.0011) [2023-12-27 10:30:47,676][20381] Updated weights for policy 1, policy_version 705619 (0.0011) [2023-12-27 10:30:47,682][20347] Updated weights for policy 0, policy_version 706107 (0.0010) [2023-12-27 10:30:47,745][20347] Updated weights for policy 0, policy_version 706117 (0.0010) [2023-12-27 10:30:48,327][20381] Updated weights for policy 1, policy_version 705629 (0.0011) [2023-12-27 10:30:48,399][20381] Updated weights for policy 1, policy_version 705639 (0.0010) [2023-12-27 10:30:48,403][20347] Updated weights for policy 0, policy_version 706127 (0.0007) [2023-12-27 10:30:48,456][20347] Updated weights for policy 0, policy_version 706137 (0.0007) [2023-12-27 10:30:48,466][20381] Updated weights for policy 1, policy_version 705649 (0.0007) [2023-12-27 10:30:48,507][20347] Updated weights for policy 0, policy_version 706147 (0.0005) [2023-12-27 10:30:49,087][20347] Updated weights for policy 0, policy_version 706157 (0.0005) [2023-12-27 10:30:49,132][20347] Updated weights for policy 0, policy_version 706167 (0.0005) [2023-12-27 10:30:49,184][20381] Updated weights for policy 1, policy_version 705659 (0.0008) [2023-12-27 10:30:49,190][20347] Updated weights for policy 0, policy_version 706177 (0.0005) [2023-12-27 10:30:49,247][20381] Updated weights for policy 1, policy_version 705669 (0.0008) [2023-12-27 10:30:49,300][20381] Updated weights for policy 1, policy_version 705679 (0.0006) [2023-12-27 10:30:49,881][20381] Updated weights for policy 1, policy_version 705689 (0.0007) [2023-12-27 10:30:49,882][20347] Updated weights for policy 0, policy_version 706187 (0.0006) [2023-12-27 10:30:49,945][20381] Updated weights for policy 1, policy_version 705699 (0.0008) [2023-12-27 10:30:49,947][20347] Updated weights for policy 0, policy_version 706197 (0.0007) [2023-12-27 10:30:50,002][20381] Updated weights for policy 1, policy_version 705709 (0.0006) [2023-12-27 10:30:50,007][20347] Updated weights for policy 0, policy_version 706207 (0.0007) [2023-12-27 10:30:50,057][20381] Updated weights for policy 1, policy_version 705719 (0.0006) [2023-12-27 10:30:50,704][20347] Updated weights for policy 0, policy_version 706217 (0.0007) [2023-12-27 10:30:50,735][20381] Updated weights for policy 1, policy_version 705729 (0.0008) [2023-12-27 10:30:50,760][20347] Updated weights for policy 0, policy_version 706227 (0.0006) [2023-12-27 10:30:50,790][20381] Updated weights for policy 1, policy_version 705739 (0.0007) [2023-12-27 10:30:50,822][20347] Updated weights for policy 0, policy_version 706237 (0.0006) [2023-12-27 10:30:50,852][20381] Updated weights for policy 1, policy_version 705749 (0.0006) [2023-12-27 10:30:50,886][20347] Updated weights for policy 0, policy_version 706247 (0.0007) [2023-12-27 10:30:51,495][20381] Updated weights for policy 1, policy_version 705759 (0.0005) [2023-12-27 10:30:51,546][20347] Updated weights for policy 0, policy_version 706257 (0.0008) [2023-12-27 10:30:51,556][20381] Updated weights for policy 1, policy_version 705769 (0.0007) [2023-12-27 10:30:51,609][20347] Updated weights for policy 0, policy_version 706267 (0.0008) [2023-12-27 10:30:51,614][20381] Updated weights for policy 1, policy_version 705779 (0.0007) [2023-12-27 10:30:51,671][20347] Updated weights for policy 0, policy_version 706277 (0.0008) [2023-12-27 10:30:52,259][20381] Updated weights for policy 1, policy_version 705789 (0.0008) [2023-12-27 10:30:52,328][20381] Updated weights for policy 1, policy_version 705799 (0.0008) [2023-12-27 10:30:52,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20743.8). Total num frames: 361537536. Throughput: 0: 10378.0, 1: 10284.6. Samples: 361513492. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:52,363][19170] Avg episode reward: [(0, '883.500'), (1, '941.710')] [2023-12-27 10:30:52,374][20347] Updated weights for policy 0, policy_version 706287 (0.0008) [2023-12-27 10:30:52,392][20381] Updated weights for policy 1, policy_version 705809 (0.0009) [2023-12-27 10:30:52,435][20347] Updated weights for policy 0, policy_version 706297 (0.0006) [2023-12-27 10:30:52,435][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000705816_180715520.pth... [2023-12-27 10:30:52,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000704600_180404224.pth [2023-12-27 10:30:52,483][20347] Updated weights for policy 0, policy_version 706307 (0.0007) [2023-12-27 10:30:52,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000706312_180838400.pth... [2023-12-27 10:30:52,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000705064_180518912.pth [2023-12-27 10:30:53,059][20381] Updated weights for policy 1, policy_version 705819 (0.0011) [2023-12-27 10:30:53,103][20381] Updated weights for policy 1, policy_version 705829 (0.0010) [2023-12-27 10:30:53,152][20381] Updated weights for policy 1, policy_version 705839 (0.0010) [2023-12-27 10:30:53,159][20347] Updated weights for policy 0, policy_version 706317 (0.0006) [2023-12-27 10:30:53,215][20347] Updated weights for policy 0, policy_version 706327 (0.0005) [2023-12-27 10:30:53,276][20347] Updated weights for policy 0, policy_version 706337 (0.0007) [2023-12-27 10:30:53,793][20381] Updated weights for policy 1, policy_version 705849 (0.0009) [2023-12-27 10:30:53,847][20381] Updated weights for policy 1, policy_version 705859 (0.0006) [2023-12-27 10:30:53,871][20347] Updated weights for policy 0, policy_version 706347 (0.0006) [2023-12-27 10:30:53,903][20381] Updated weights for policy 1, policy_version 705869 (0.0005) [2023-12-27 10:30:53,927][20347] Updated weights for policy 0, policy_version 706357 (0.0007) [2023-12-27 10:30:53,955][20381] Updated weights for policy 1, policy_version 705879 (0.0005) [2023-12-27 10:30:53,983][20347] Updated weights for policy 0, policy_version 706367 (0.0009) [2023-12-27 10:30:54,500][20381] Updated weights for policy 1, policy_version 705889 (0.0005) [2023-12-27 10:30:54,554][20381] Updated weights for policy 1, policy_version 705899 (0.0005) [2023-12-27 10:30:54,608][20381] Updated weights for policy 1, policy_version 705909 (0.0005) [2023-12-27 10:30:54,638][20347] Updated weights for policy 0, policy_version 706377 (0.0010) [2023-12-27 10:30:54,694][20347] Updated weights for policy 0, policy_version 706387 (0.0005) [2023-12-27 10:30:54,756][20347] Updated weights for policy 0, policy_version 706397 (0.0005) [2023-12-27 10:30:54,825][20347] Updated weights for policy 0, policy_version 706407 (0.0006) [2023-12-27 10:30:55,135][20381] Updated weights for policy 1, policy_version 705919 (0.0005) [2023-12-27 10:30:55,198][20381] Updated weights for policy 1, policy_version 705929 (0.0005) [2023-12-27 10:30:55,254][20381] Updated weights for policy 1, policy_version 705939 (0.0006) [2023-12-27 10:30:55,383][20347] Updated weights for policy 0, policy_version 706417 (0.0009) [2023-12-27 10:30:55,440][20347] Updated weights for policy 0, policy_version 706427 (0.0010) [2023-12-27 10:30:55,501][20347] Updated weights for policy 0, policy_version 706437 (0.0010) [2023-12-27 10:30:55,887][20381] Updated weights for policy 1, policy_version 705949 (0.0006) [2023-12-27 10:30:55,942][20381] Updated weights for policy 1, policy_version 705959 (0.0007) [2023-12-27 10:30:55,993][20381] Updated weights for policy 1, policy_version 705969 (0.0007) [2023-12-27 10:30:56,132][20347] Updated weights for policy 0, policy_version 706447 (0.0010) [2023-12-27 10:30:56,188][20347] Updated weights for policy 0, policy_version 706457 (0.0010) [2023-12-27 10:30:56,246][20347] Updated weights for policy 0, policy_version 706467 (0.0009) [2023-12-27 10:30:56,692][20381] Updated weights for policy 1, policy_version 705979 (0.0007) [2023-12-27 10:30:56,736][20381] Updated weights for policy 1, policy_version 705989 (0.0008) [2023-12-27 10:30:56,780][20381] Updated weights for policy 1, policy_version 705999 (0.0008) [2023-12-27 10:30:56,853][20347] Updated weights for policy 0, policy_version 706477 (0.0005) [2023-12-27 10:30:56,901][20347] Updated weights for policy 0, policy_version 706487 (0.0005) [2023-12-27 10:30:56,951][20347] Updated weights for policy 0, policy_version 706497 (0.0005) [2023-12-27 10:30:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20799.4). Total num frames: 361652224. Throughput: 0: 10370.1, 1: 10413.6. Samples: 361642888. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:30:57,363][19170] Avg episode reward: [(0, '902.510'), (1, '922.300')] [2023-12-27 10:30:57,494][20381] Updated weights for policy 1, policy_version 706009 (0.0008) [2023-12-27 10:30:57,529][20347] Updated weights for policy 0, policy_version 706507 (0.0006) [2023-12-27 10:30:57,566][20381] Updated weights for policy 1, policy_version 706019 (0.0005) [2023-12-27 10:30:57,587][20347] Updated weights for policy 0, policy_version 706517 (0.0005) [2023-12-27 10:30:57,615][20381] Updated weights for policy 1, policy_version 706029 (0.0007) [2023-12-27 10:30:57,642][20347] Updated weights for policy 0, policy_version 706527 (0.0005) [2023-12-27 10:30:57,666][20381] Updated weights for policy 1, policy_version 706039 (0.0007) [2023-12-27 10:30:58,294][20347] Updated weights for policy 0, policy_version 706537 (0.0006) [2023-12-27 10:30:58,318][20381] Updated weights for policy 1, policy_version 706049 (0.0010) [2023-12-27 10:30:58,358][20347] Updated weights for policy 0, policy_version 706547 (0.0008) [2023-12-27 10:30:58,386][20381] Updated weights for policy 1, policy_version 706059 (0.0008) [2023-12-27 10:30:58,429][20347] Updated weights for policy 0, policy_version 706557 (0.0007) [2023-12-27 10:30:58,451][20381] Updated weights for policy 1, policy_version 706069 (0.0008) [2023-12-27 10:30:58,489][20347] Updated weights for policy 0, policy_version 706567 (0.0006) [2023-12-27 10:30:59,192][20381] Updated weights for policy 1, policy_version 706079 (0.0008) [2023-12-27 10:30:59,201][20347] Updated weights for policy 0, policy_version 706577 (0.0006) [2023-12-27 10:30:59,260][20381] Updated weights for policy 1, policy_version 706089 (0.0008) [2023-12-27 10:30:59,265][20347] Updated weights for policy 0, policy_version 706587 (0.0008) [2023-12-27 10:30:59,317][20381] Updated weights for policy 1, policy_version 706099 (0.0007) [2023-12-27 10:30:59,324][20347] Updated weights for policy 0, policy_version 706597 (0.0008) [2023-12-27 10:30:59,993][20381] Updated weights for policy 1, policy_version 706109 (0.0008) [2023-12-27 10:30:59,998][20347] Updated weights for policy 0, policy_version 706607 (0.0006) [2023-12-27 10:31:00,050][20381] Updated weights for policy 1, policy_version 706119 (0.0008) [2023-12-27 10:31:00,054][20347] Updated weights for policy 0, policy_version 706617 (0.0006) [2023-12-27 10:31:00,111][20347] Updated weights for policy 0, policy_version 706627 (0.0005) [2023-12-27 10:31:00,115][20381] Updated weights for policy 1, policy_version 706129 (0.0008) [2023-12-27 10:31:00,666][20347] Updated weights for policy 0, policy_version 706637 (0.0008) [2023-12-27 10:31:00,712][20347] Updated weights for policy 0, policy_version 706647 (0.0010) [2023-12-27 10:31:00,723][20381] Updated weights for policy 1, policy_version 706139 (0.0010) [2023-12-27 10:31:00,763][20347] Updated weights for policy 0, policy_version 706657 (0.0010) [2023-12-27 10:31:00,781][20381] Updated weights for policy 1, policy_version 706149 (0.0011) [2023-12-27 10:31:00,830][20381] Updated weights for policy 1, policy_version 706159 (0.0010) [2023-12-27 10:31:01,397][20347] Updated weights for policy 0, policy_version 706667 (0.0009) [2023-12-27 10:31:01,455][20347] Updated weights for policy 0, policy_version 706677 (0.0010) [2023-12-27 10:31:01,511][20347] Updated weights for policy 0, policy_version 706687 (0.0010) [2023-12-27 10:31:01,532][20381] Updated weights for policy 1, policy_version 706169 (0.0010) [2023-12-27 10:31:01,588][20381] Updated weights for policy 1, policy_version 706179 (0.0006) [2023-12-27 10:31:01,645][20381] Updated weights for policy 1, policy_version 706189 (0.0008) [2023-12-27 10:31:01,702][20381] Updated weights for policy 1, policy_version 706199 (0.0008) [2023-12-27 10:31:02,161][20347] Updated weights for policy 0, policy_version 706697 (0.0010) [2023-12-27 10:31:02,211][20347] Updated weights for policy 0, policy_version 706707 (0.0007) [2023-12-27 10:31:02,281][20347] Updated weights for policy 0, policy_version 706717 (0.0008) [2023-12-27 10:31:02,337][20347] Updated weights for policy 0, policy_version 706727 (0.0009) [2023-12-27 10:31:02,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20799.4). Total num frames: 361758720. Throughput: 0: 10445.5, 1: 10392.5. Samples: 361768676. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:02,363][19170] Avg episode reward: [(0, '960.910'), (1, '961.120')] [2023-12-27 10:31:02,394][20381] Updated weights for policy 1, policy_version 706209 (0.0007) [2023-12-27 10:31:02,452][20381] Updated weights for policy 1, policy_version 706219 (0.0007) [2023-12-27 10:31:02,503][20381] Updated weights for policy 1, policy_version 706229 (0.0007) [2023-12-27 10:31:03,031][20347] Updated weights for policy 0, policy_version 706737 (0.0007) [2023-12-27 10:31:03,075][20347] Updated weights for policy 0, policy_version 706747 (0.0007) [2023-12-27 10:31:03,127][20347] Updated weights for policy 0, policy_version 706757 (0.0007) [2023-12-27 10:31:03,163][20381] Updated weights for policy 1, policy_version 706239 (0.0007) [2023-12-27 10:31:03,219][20381] Updated weights for policy 1, policy_version 706249 (0.0009) [2023-12-27 10:31:03,270][20381] Updated weights for policy 1, policy_version 706259 (0.0007) [2023-12-27 10:31:03,715][20347] Updated weights for policy 0, policy_version 706767 (0.0005) [2023-12-27 10:31:03,774][20347] Updated weights for policy 0, policy_version 706777 (0.0006) [2023-12-27 10:31:03,850][20347] Updated weights for policy 0, policy_version 706787 (0.0006) [2023-12-27 10:31:04,009][20381] Updated weights for policy 1, policy_version 706269 (0.0008) [2023-12-27 10:31:04,061][20381] Updated weights for policy 1, policy_version 706279 (0.0007) [2023-12-27 10:31:04,121][20381] Updated weights for policy 1, policy_version 706289 (0.0008) [2023-12-27 10:31:04,499][20347] Updated weights for policy 0, policy_version 706797 (0.0008) [2023-12-27 10:31:04,560][20347] Updated weights for policy 0, policy_version 706807 (0.0005) [2023-12-27 10:31:04,608][20347] Updated weights for policy 0, policy_version 706817 (0.0005) [2023-12-27 10:31:04,848][20381] Updated weights for policy 1, policy_version 706299 (0.0008) [2023-12-27 10:31:04,906][20381] Updated weights for policy 1, policy_version 706309 (0.0011) [2023-12-27 10:31:04,955][20381] Updated weights for policy 1, policy_version 706319 (0.0011) [2023-12-27 10:31:05,181][20347] Updated weights for policy 0, policy_version 706827 (0.0007) [2023-12-27 10:31:05,240][20347] Updated weights for policy 0, policy_version 706837 (0.0010) [2023-12-27 10:31:05,292][20347] Updated weights for policy 0, policy_version 706847 (0.0010) [2023-12-27 10:31:05,575][20381] Updated weights for policy 1, policy_version 706329 (0.0011) [2023-12-27 10:31:05,627][20381] Updated weights for policy 1, policy_version 706339 (0.0009) [2023-12-27 10:31:05,680][20381] Updated weights for policy 1, policy_version 706349 (0.0005) [2023-12-27 10:31:05,730][20381] Updated weights for policy 1, policy_version 706359 (0.0005) [2023-12-27 10:31:05,964][20347] Updated weights for policy 0, policy_version 706857 (0.0010) [2023-12-27 10:31:06,015][20347] Updated weights for policy 0, policy_version 706867 (0.0010) [2023-12-27 10:31:06,067][20347] Updated weights for policy 0, policy_version 706877 (0.0010) [2023-12-27 10:31:06,126][20347] Updated weights for policy 0, policy_version 706887 (0.0010) [2023-12-27 10:31:06,276][20381] Updated weights for policy 1, policy_version 706369 (0.0010) [2023-12-27 10:31:06,331][20381] Updated weights for policy 1, policy_version 706379 (0.0010) [2023-12-27 10:31:06,386][20381] Updated weights for policy 1, policy_version 706389 (0.0010) [2023-12-27 10:31:06,834][20347] Updated weights for policy 0, policy_version 706897 (0.0008) [2023-12-27 10:31:06,886][20347] Updated weights for policy 0, policy_version 706907 (0.0008) [2023-12-27 10:31:06,933][20347] Updated weights for policy 0, policy_version 706917 (0.0008) [2023-12-27 10:31:07,086][20381] Updated weights for policy 1, policy_version 706399 (0.0011) [2023-12-27 10:31:07,144][20381] Updated weights for policy 1, policy_version 706409 (0.0010) [2023-12-27 10:31:07,192][20381] Updated weights for policy 1, policy_version 706419 (0.0010) [2023-12-27 10:31:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 361865216. Throughput: 0: 10478.7, 1: 10358.7. Samples: 361830972. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:07,363][19170] Avg episode reward: [(0, '970.720'), (1, '951.420')] [2023-12-27 10:31:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000706424_180871168.pth... [2023-12-27 10:31:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000706920_180994048.pth... [2023-12-27 10:31:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000705176_180551680.pth [2023-12-27 10:31:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000705704_180682752.pth [2023-12-27 10:31:07,606][20347] Updated weights for policy 0, policy_version 706927 (0.0006) [2023-12-27 10:31:07,652][20347] Updated weights for policy 0, policy_version 706937 (0.0006) [2023-12-27 10:31:07,698][20347] Updated weights for policy 0, policy_version 706947 (0.0010) [2023-12-27 10:31:07,860][20381] Updated weights for policy 1, policy_version 706429 (0.0008) [2023-12-27 10:31:07,911][20381] Updated weights for policy 1, policy_version 706439 (0.0007) [2023-12-27 10:31:07,966][20381] Updated weights for policy 1, policy_version 706449 (0.0007) [2023-12-27 10:31:08,340][20347] Updated weights for policy 0, policy_version 706957 (0.0010) [2023-12-27 10:31:08,401][20347] Updated weights for policy 0, policy_version 706967 (0.0010) [2023-12-27 10:31:08,463][20347] Updated weights for policy 0, policy_version 706977 (0.0010) [2023-12-27 10:31:08,577][20381] Updated weights for policy 1, policy_version 706459 (0.0009) [2023-12-27 10:31:08,626][20381] Updated weights for policy 1, policy_version 706469 (0.0010) [2023-12-27 10:31:08,684][20381] Updated weights for policy 1, policy_version 706479 (0.0010) [2023-12-27 10:31:09,152][20347] Updated weights for policy 0, policy_version 706987 (0.0009) [2023-12-27 10:31:09,209][20347] Updated weights for policy 0, policy_version 706997 (0.0007) [2023-12-27 10:31:09,272][20347] Updated weights for policy 0, policy_version 707007 (0.0008) [2023-12-27 10:31:09,409][20381] Updated weights for policy 1, policy_version 706489 (0.0010) [2023-12-27 10:31:09,464][20381] Updated weights for policy 1, policy_version 706499 (0.0006) [2023-12-27 10:31:09,528][20381] Updated weights for policy 1, policy_version 706509 (0.0008) [2023-12-27 10:31:09,579][20381] Updated weights for policy 1, policy_version 706519 (0.0007) [2023-12-27 10:31:09,994][20347] Updated weights for policy 0, policy_version 707017 (0.0007) [2023-12-27 10:31:10,058][20347] Updated weights for policy 0, policy_version 707027 (0.0010) [2023-12-27 10:31:10,119][20347] Updated weights for policy 0, policy_version 707037 (0.0010) [2023-12-27 10:31:10,181][20347] Updated weights for policy 0, policy_version 707047 (0.0010) [2023-12-27 10:31:10,239][20381] Updated weights for policy 1, policy_version 706529 (0.0010) [2023-12-27 10:31:10,303][20381] Updated weights for policy 1, policy_version 706539 (0.0009) [2023-12-27 10:31:10,363][20381] Updated weights for policy 1, policy_version 706549 (0.0009) [2023-12-27 10:31:10,767][20347] Updated weights for policy 0, policy_version 707057 (0.0006) [2023-12-27 10:31:10,817][20347] Updated weights for policy 0, policy_version 707067 (0.0008) [2023-12-27 10:31:10,879][20347] Updated weights for policy 0, policy_version 707077 (0.0008) [2023-12-27 10:31:11,004][20381] Updated weights for policy 1, policy_version 706559 (0.0007) [2023-12-27 10:31:11,063][20381] Updated weights for policy 1, policy_version 706569 (0.0008) [2023-12-27 10:31:11,123][20381] Updated weights for policy 1, policy_version 706579 (0.0007) [2023-12-27 10:31:11,564][20347] Updated weights for policy 0, policy_version 707087 (0.0008) [2023-12-27 10:31:11,625][20347] Updated weights for policy 0, policy_version 707097 (0.0007) [2023-12-27 10:31:11,690][20347] Updated weights for policy 0, policy_version 707107 (0.0009) [2023-12-27 10:31:11,804][20381] Updated weights for policy 1, policy_version 706589 (0.0007) [2023-12-27 10:31:11,863][20381] Updated weights for policy 1, policy_version 706599 (0.0005) [2023-12-27 10:31:11,920][20381] Updated weights for policy 1, policy_version 706609 (0.0005) [2023-12-27 10:31:12,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20799.3). Total num frames: 361963520. Throughput: 0: 10515.4, 1: 10467.0. Samples: 361957504. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:12,363][19170] Avg episode reward: [(0, '961.310'), (1, '961.340')] [2023-12-27 10:31:12,424][20347] Updated weights for policy 0, policy_version 707117 (0.0007) [2023-12-27 10:31:12,489][20347] Updated weights for policy 0, policy_version 707127 (0.0007) [2023-12-27 10:31:12,515][20381] Updated weights for policy 1, policy_version 706619 (0.0007) [2023-12-27 10:31:12,546][20347] Updated weights for policy 0, policy_version 707137 (0.0008) [2023-12-27 10:31:12,574][20381] Updated weights for policy 1, policy_version 706629 (0.0011) [2023-12-27 10:31:12,630][20381] Updated weights for policy 1, policy_version 706639 (0.0011) [2023-12-27 10:31:13,227][20347] Updated weights for policy 0, policy_version 707147 (0.0006) [2023-12-27 10:31:13,282][20347] Updated weights for policy 0, policy_version 707157 (0.0008) [2023-12-27 10:31:13,297][20381] Updated weights for policy 1, policy_version 706649 (0.0010) [2023-12-27 10:31:13,332][20347] Updated weights for policy 0, policy_version 707167 (0.0007) [2023-12-27 10:31:13,348][20381] Updated weights for policy 1, policy_version 706659 (0.0005) [2023-12-27 10:31:13,402][20381] Updated weights for policy 1, policy_version 706669 (0.0005) [2023-12-27 10:31:13,459][20381] Updated weights for policy 1, policy_version 706679 (0.0005) [2023-12-27 10:31:13,937][20347] Updated weights for policy 0, policy_version 707177 (0.0006) [2023-12-27 10:31:13,990][20347] Updated weights for policy 0, policy_version 707187 (0.0006) [2023-12-27 10:31:14,042][20347] Updated weights for policy 0, policy_version 707197 (0.0007) [2023-12-27 10:31:14,092][20347] Updated weights for policy 0, policy_version 707207 (0.0007) [2023-12-27 10:31:14,123][20381] Updated weights for policy 1, policy_version 706689 (0.0005) [2023-12-27 10:31:14,172][20381] Updated weights for policy 1, policy_version 706699 (0.0005) [2023-12-27 10:31:14,227][20381] Updated weights for policy 1, policy_version 706709 (0.0005) [2023-12-27 10:31:14,753][20347] Updated weights for policy 0, policy_version 707217 (0.0007) [2023-12-27 10:31:14,823][20347] Updated weights for policy 0, policy_version 707227 (0.0008) [2023-12-27 10:31:14,842][20381] Updated weights for policy 1, policy_version 706719 (0.0006) [2023-12-27 10:31:14,879][20347] Updated weights for policy 0, policy_version 707237 (0.0007) [2023-12-27 10:31:14,898][20381] Updated weights for policy 1, policy_version 706729 (0.0006) [2023-12-27 10:31:14,955][20381] Updated weights for policy 1, policy_version 706739 (0.0005) [2023-12-27 10:31:15,582][20347] Updated weights for policy 0, policy_version 707247 (0.0008) [2023-12-27 10:31:15,595][20381] Updated weights for policy 1, policy_version 706749 (0.0007) [2023-12-27 10:31:15,639][20347] Updated weights for policy 0, policy_version 707257 (0.0006) [2023-12-27 10:31:15,655][20381] Updated weights for policy 1, policy_version 706759 (0.0006) [2023-12-27 10:31:15,698][20347] Updated weights for policy 0, policy_version 707267 (0.0006) [2023-12-27 10:31:15,718][20381] Updated weights for policy 1, policy_version 706769 (0.0007) [2023-12-27 10:31:16,326][20381] Updated weights for policy 1, policy_version 706779 (0.0007) [2023-12-27 10:31:16,378][20381] Updated weights for policy 1, policy_version 706789 (0.0007) [2023-12-27 10:31:16,388][20347] Updated weights for policy 0, policy_version 707277 (0.0006) [2023-12-27 10:31:16,439][20381] Updated weights for policy 1, policy_version 706799 (0.0005) [2023-12-27 10:31:16,444][20347] Updated weights for policy 0, policy_version 707287 (0.0010) [2023-12-27 10:31:16,506][20347] Updated weights for policy 0, policy_version 707297 (0.0010) [2023-12-27 10:31:16,987][20381] Updated weights for policy 1, policy_version 706809 (0.0006) [2023-12-27 10:31:17,034][20381] Updated weights for policy 1, policy_version 706819 (0.0005) [2023-12-27 10:31:17,087][20381] Updated weights for policy 1, policy_version 706829 (0.0006) [2023-12-27 10:31:17,140][20381] Updated weights for policy 1, policy_version 706839 (0.0006) [2023-12-27 10:31:17,164][20347] Updated weights for policy 0, policy_version 707307 (0.0010) [2023-12-27 10:31:17,222][20347] Updated weights for policy 0, policy_version 707317 (0.0010) [2023-12-27 10:31:17,276][20347] Updated weights for policy 0, policy_version 707327 (0.0010) [2023-12-27 10:31:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 362078208. Throughput: 0: 10489.5, 1: 10533.0. Samples: 362084496. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:17,364][19170] Avg episode reward: [(0, '922.670'), (1, '952.130')] [2023-12-27 10:31:17,754][20381] Updated weights for policy 1, policy_version 706849 (0.0006) [2023-12-27 10:31:17,812][20381] Updated weights for policy 1, policy_version 706859 (0.0009) [2023-12-27 10:31:17,877][20381] Updated weights for policy 1, policy_version 706869 (0.0010) [2023-12-27 10:31:17,927][20347] Updated weights for policy 0, policy_version 707337 (0.0009) [2023-12-27 10:31:17,988][20347] Updated weights for policy 0, policy_version 707347 (0.0010) [2023-12-27 10:31:18,043][20347] Updated weights for policy 0, policy_version 707357 (0.0010) [2023-12-27 10:31:18,092][20347] Updated weights for policy 0, policy_version 707367 (0.0010) [2023-12-27 10:31:18,501][20381] Updated weights for policy 1, policy_version 706879 (0.0008) [2023-12-27 10:31:18,565][20381] Updated weights for policy 1, policy_version 706889 (0.0011) [2023-12-27 10:31:18,627][20381] Updated weights for policy 1, policy_version 706899 (0.0011) [2023-12-27 10:31:18,755][20347] Updated weights for policy 0, policy_version 707377 (0.0010) [2023-12-27 10:31:18,814][20347] Updated weights for policy 0, policy_version 707387 (0.0009) [2023-12-27 10:31:18,868][20347] Updated weights for policy 0, policy_version 707397 (0.0007) [2023-12-27 10:31:19,231][20381] Updated weights for policy 1, policy_version 706909 (0.0010) [2023-12-27 10:31:19,295][20381] Updated weights for policy 1, policy_version 706919 (0.0005) [2023-12-27 10:31:19,360][20381] Updated weights for policy 1, policy_version 706929 (0.0010) [2023-12-27 10:31:19,537][20347] Updated weights for policy 0, policy_version 707407 (0.0010) [2023-12-27 10:31:19,594][20347] Updated weights for policy 0, policy_version 707417 (0.0010) [2023-12-27 10:31:19,648][20347] Updated weights for policy 0, policy_version 707427 (0.0005) [2023-12-27 10:31:20,031][20381] Updated weights for policy 1, policy_version 706939 (0.0009) [2023-12-27 10:31:20,090][20381] Updated weights for policy 1, policy_version 706949 (0.0008) [2023-12-27 10:31:20,150][20381] Updated weights for policy 1, policy_version 706959 (0.0010) [2023-12-27 10:31:20,279][20347] Updated weights for policy 0, policy_version 707437 (0.0007) [2023-12-27 10:31:20,349][20347] Updated weights for policy 0, policy_version 707447 (0.0009) [2023-12-27 10:31:20,412][20347] Updated weights for policy 0, policy_version 707457 (0.0009) [2023-12-27 10:31:20,838][20381] Updated weights for policy 1, policy_version 706969 (0.0011) [2023-12-27 10:31:20,897][20381] Updated weights for policy 1, policy_version 706979 (0.0011) [2023-12-27 10:31:20,957][20381] Updated weights for policy 1, policy_version 706989 (0.0011) [2023-12-27 10:31:20,998][20347] Updated weights for policy 0, policy_version 707467 (0.0007) [2023-12-27 10:31:21,011][20381] Updated weights for policy 1, policy_version 706999 (0.0011) [2023-12-27 10:31:21,063][20347] Updated weights for policy 0, policy_version 707477 (0.0009) [2023-12-27 10:31:21,122][20347] Updated weights for policy 0, policy_version 707487 (0.0010) [2023-12-27 10:31:21,799][20381] Updated weights for policy 1, policy_version 707009 (0.0009) [2023-12-27 10:31:21,835][20347] Updated weights for policy 0, policy_version 707497 (0.0010) [2023-12-27 10:31:21,855][20381] Updated weights for policy 1, policy_version 707019 (0.0007) [2023-12-27 10:31:21,897][20347] Updated weights for policy 0, policy_version 707507 (0.0007) [2023-12-27 10:31:21,914][20381] Updated weights for policy 1, policy_version 707029 (0.0008) [2023-12-27 10:31:21,950][20347] Updated weights for policy 0, policy_version 707517 (0.0008) [2023-12-27 10:31:22,009][20347] Updated weights for policy 0, policy_version 707527 (0.0007) [2023-12-27 10:31:22,363][19170] Fps is (10 sec: 21299.3, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 362176512. Throughput: 0: 10479.2, 1: 10580.8. Samples: 362148016. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:22,363][19170] Avg episode reward: [(0, '980.580'), (1, '942.400')] [2023-12-27 10:31:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000707032_181026816.pth... [2023-12-27 10:31:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000707528_181149696.pth... [2023-12-27 10:31:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000705816_180715520.pth [2023-12-27 10:31:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000706312_180838400.pth [2023-12-27 10:31:22,633][20381] Updated weights for policy 1, policy_version 707039 (0.0010) [2023-12-27 10:31:22,646][20347] Updated weights for policy 0, policy_version 707537 (0.0006) [2023-12-27 10:31:22,689][20381] Updated weights for policy 1, policy_version 707049 (0.0010) [2023-12-27 10:31:22,693][20347] Updated weights for policy 0, policy_version 707547 (0.0008) [2023-12-27 10:31:22,740][20347] Updated weights for policy 0, policy_version 707557 (0.0008) [2023-12-27 10:31:22,743][20381] Updated weights for policy 1, policy_version 707059 (0.0011) [2023-12-27 10:31:23,421][20347] Updated weights for policy 0, policy_version 707567 (0.0007) [2023-12-27 10:31:23,444][20381] Updated weights for policy 1, policy_version 707069 (0.0010) [2023-12-27 10:31:23,479][20347] Updated weights for policy 0, policy_version 707577 (0.0009) [2023-12-27 10:31:23,497][20381] Updated weights for policy 1, policy_version 707079 (0.0007) [2023-12-27 10:31:23,534][20347] Updated weights for policy 0, policy_version 707587 (0.0010) [2023-12-27 10:31:23,551][20381] Updated weights for policy 1, policy_version 707089 (0.0010) [2023-12-27 10:31:24,128][20347] Updated weights for policy 0, policy_version 707597 (0.0009) [2023-12-27 10:31:24,184][20347] Updated weights for policy 0, policy_version 707607 (0.0010) [2023-12-27 10:31:24,198][20381] Updated weights for policy 1, policy_version 707099 (0.0009) [2023-12-27 10:31:24,231][20347] Updated weights for policy 0, policy_version 707617 (0.0010) [2023-12-27 10:31:24,258][20381] Updated weights for policy 1, policy_version 707109 (0.0007) [2023-12-27 10:31:24,312][20381] Updated weights for policy 1, policy_version 707119 (0.0010) [2023-12-27 10:31:24,875][20347] Updated weights for policy 0, policy_version 707627 (0.0009) [2023-12-27 10:31:24,922][20381] Updated weights for policy 1, policy_version 707129 (0.0010) [2023-12-27 10:31:24,939][20347] Updated weights for policy 0, policy_version 707637 (0.0006) [2023-12-27 10:31:24,973][20381] Updated weights for policy 1, policy_version 707139 (0.0005) [2023-12-27 10:31:24,997][20347] Updated weights for policy 0, policy_version 707647 (0.0005) [2023-12-27 10:31:25,035][20381] Updated weights for policy 1, policy_version 707149 (0.0005) [2023-12-27 10:31:25,091][20381] Updated weights for policy 1, policy_version 707159 (0.0005) [2023-12-27 10:31:25,605][20347] Updated weights for policy 0, policy_version 707657 (0.0006) [2023-12-27 10:31:25,641][20381] Updated weights for policy 1, policy_version 707169 (0.0007) [2023-12-27 10:31:25,668][20347] Updated weights for policy 0, policy_version 707667 (0.0006) [2023-12-27 10:31:25,692][20381] Updated weights for policy 1, policy_version 707179 (0.0009) [2023-12-27 10:31:25,729][20347] Updated weights for policy 0, policy_version 707677 (0.0007) [2023-12-27 10:31:25,742][20381] Updated weights for policy 1, policy_version 707189 (0.0007) [2023-12-27 10:31:25,787][20347] Updated weights for policy 0, policy_version 707687 (0.0008) [2023-12-27 10:31:26,340][20381] Updated weights for policy 1, policy_version 707199 (0.0009) [2023-12-27 10:31:26,376][20347] Updated weights for policy 0, policy_version 707697 (0.0010) [2023-12-27 10:31:26,397][20381] Updated weights for policy 1, policy_version 707209 (0.0007) [2023-12-27 10:31:26,436][20347] Updated weights for policy 0, policy_version 707707 (0.0010) [2023-12-27 10:31:26,451][20381] Updated weights for policy 1, policy_version 707219 (0.0011) [2023-12-27 10:31:26,484][20347] Updated weights for policy 0, policy_version 707717 (0.0008) [2023-12-27 10:31:27,088][20381] Updated weights for policy 1, policy_version 707229 (0.0010) [2023-12-27 10:31:27,102][20347] Updated weights for policy 0, policy_version 707727 (0.0005) [2023-12-27 10:31:27,142][20381] Updated weights for policy 1, policy_version 707239 (0.0010) [2023-12-27 10:31:27,155][20347] Updated weights for policy 0, policy_version 707737 (0.0005) [2023-12-27 10:31:27,187][20381] Updated weights for policy 1, policy_version 707249 (0.0010) [2023-12-27 10:31:27,210][20347] Updated weights for policy 0, policy_version 707747 (0.0007) [2023-12-27 10:31:27,363][19170] Fps is (10 sec: 21299.2, 60 sec: 21026.1, 300 sec: 20854.9). Total num frames: 362291200. Throughput: 0: 10512.0, 1: 10547.8. Samples: 362275036. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '980.590')] [2023-12-27 10:31:27,838][20347] Updated weights for policy 0, policy_version 707757 (0.0007) [2023-12-27 10:31:27,848][20381] Updated weights for policy 1, policy_version 707259 (0.0010) [2023-12-27 10:31:27,887][20347] Updated weights for policy 0, policy_version 707767 (0.0005) [2023-12-27 10:31:27,903][20381] Updated weights for policy 1, policy_version 707269 (0.0010) [2023-12-27 10:31:27,937][20347] Updated weights for policy 0, policy_version 707777 (0.0005) [2023-12-27 10:31:27,960][20381] Updated weights for policy 1, policy_version 707279 (0.0010) [2023-12-27 10:31:28,576][20347] Updated weights for policy 0, policy_version 707787 (0.0006) [2023-12-27 10:31:28,633][20347] Updated weights for policy 0, policy_version 707797 (0.0007) [2023-12-27 10:31:28,653][20381] Updated weights for policy 1, policy_version 707289 (0.0010) [2023-12-27 10:31:28,679][20347] Updated weights for policy 0, policy_version 707807 (0.0006) [2023-12-27 10:31:28,705][20381] Updated weights for policy 1, policy_version 707299 (0.0010) [2023-12-27 10:31:28,757][20381] Updated weights for policy 1, policy_version 707309 (0.0010) [2023-12-27 10:31:28,810][20381] Updated weights for policy 1, policy_version 707319 (0.0010) [2023-12-27 10:31:29,338][20347] Updated weights for policy 0, policy_version 707817 (0.0006) [2023-12-27 10:31:29,402][20347] Updated weights for policy 0, policy_version 707827 (0.0007) [2023-12-27 10:31:29,465][20347] Updated weights for policy 0, policy_version 707837 (0.0008) [2023-12-27 10:31:29,508][20381] Updated weights for policy 1, policy_version 707329 (0.0006) [2023-12-27 10:31:29,524][20347] Updated weights for policy 0, policy_version 707847 (0.0006) [2023-12-27 10:31:29,563][20381] Updated weights for policy 1, policy_version 707339 (0.0005) [2023-12-27 10:31:29,620][20381] Updated weights for policy 1, policy_version 707349 (0.0005) [2023-12-27 10:31:30,173][20347] Updated weights for policy 0, policy_version 707857 (0.0007) [2023-12-27 10:31:30,237][20347] Updated weights for policy 0, policy_version 707867 (0.0007) [2023-12-27 10:31:30,244][20381] Updated weights for policy 1, policy_version 707359 (0.0007) [2023-12-27 10:31:30,253][20291] KL-divergence is very high: 249.9844 [2023-12-27 10:31:30,291][20347] Updated weights for policy 0, policy_version 707877 (0.0005) [2023-12-27 10:31:30,298][20291] KL-divergence is very high: 181.6339 [2023-12-27 10:31:30,299][20381] Updated weights for policy 1, policy_version 707369 (0.0010) [2023-12-27 10:31:30,359][20381] Updated weights for policy 1, policy_version 707379 (0.0010) [2023-12-27 10:31:30,904][20381] Updated weights for policy 1, policy_version 707389 (0.0009) [2023-12-27 10:31:30,942][20292] KL-divergence is very high: 127.1499 [2023-12-27 10:31:30,943][20347] Updated weights for policy 0, policy_version 707887 (0.0005) [2023-12-27 10:31:30,959][20381] Updated weights for policy 1, policy_version 707399 (0.0010) [2023-12-27 10:31:30,994][20347] Updated weights for policy 0, policy_version 707897 (0.0009) [2023-12-27 10:31:31,011][20381] Updated weights for policy 1, policy_version 707409 (0.0010) [2023-12-27 10:31:31,054][20347] Updated weights for policy 0, policy_version 707907 (0.0010) [2023-12-27 10:31:31,659][20381] Updated weights for policy 1, policy_version 707419 (0.0009) [2023-12-27 10:31:31,711][20381] Updated weights for policy 1, policy_version 707429 (0.0008) [2023-12-27 10:31:31,727][20347] Updated weights for policy 0, policy_version 707917 (0.0008) [2023-12-27 10:31:31,744][20292] KL-divergence is very high: 319.7510 [2023-12-27 10:31:31,768][20292] KL-divergence is very high: 693.0721 [2023-12-27 10:31:31,774][20381] Updated weights for policy 1, policy_version 707439 (0.0006) [2023-12-27 10:31:31,786][20347] Updated weights for policy 0, policy_version 707927 (0.0008) [2023-12-27 10:31:31,790][20292] KL-divergence is very high: 206.6066 [2023-12-27 10:31:31,810][20292] KL-divergence is very high: 120.5126 [2023-12-27 10:31:31,814][20291] KL-divergence is very high: 158.4161 [2023-12-27 10:31:31,845][20347] Updated weights for policy 0, policy_version 707937 (0.0008) [2023-12-27 10:31:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 21026.1, 300 sec: 20827.1). Total num frames: 362389504. Throughput: 0: 10584.1, 1: 10573.3. Samples: 362402884. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:32,363][19170] Avg episode reward: [(0, '806.010'), (1, '970.800')] [2023-12-27 10:31:32,420][20381] Updated weights for policy 1, policy_version 707449 (0.0005) [2023-12-27 10:31:32,466][20381] Updated weights for policy 1, policy_version 707459 (0.0007) [2023-12-27 10:31:32,507][20347] Updated weights for policy 0, policy_version 707947 (0.0008) [2023-12-27 10:31:32,514][20381] Updated weights for policy 1, policy_version 707469 (0.0007) [2023-12-27 10:31:32,542][20292] KL-divergence is very high: 104.1336 [2023-12-27 10:31:32,570][20381] Updated weights for policy 1, policy_version 707479 (0.0006) [2023-12-27 10:31:32,570][20347] Updated weights for policy 0, policy_version 707957 (0.0006) [2023-12-27 10:31:32,630][20347] Updated weights for policy 0, policy_version 707967 (0.0006) [2023-12-27 10:31:33,082][20292] KL-divergence is very high: 111.8288 [2023-12-27 10:31:33,130][20381] Updated weights for policy 1, policy_version 707489 (0.0005) [2023-12-27 10:31:33,176][20292] KL-divergence is very high: 173.1964 [2023-12-27 10:31:33,187][20381] Updated weights for policy 1, policy_version 707499 (0.0006) [2023-12-27 10:31:33,194][20292] KL-divergence is very high: 123.7987 [2023-12-27 10:31:33,238][20381] Updated weights for policy 1, policy_version 707509 (0.0005) [2023-12-27 10:31:33,244][20347] Updated weights for policy 0, policy_version 707977 (0.0005) [2023-12-27 10:31:33,308][20347] Updated weights for policy 0, policy_version 707987 (0.0008) [2023-12-27 10:31:33,361][20347] Updated weights for policy 0, policy_version 707997 (0.0010) [2023-12-27 10:31:33,421][20347] Updated weights for policy 0, policy_version 708007 (0.0010) [2023-12-27 10:31:33,827][20381] Updated weights for policy 1, policy_version 707519 (0.0008) [2023-12-27 10:31:33,881][20292] KL-divergence is very high: 144.4066 [2023-12-27 10:31:33,881][20381] Updated weights for policy 1, policy_version 707529 (0.0008) [2023-12-27 10:31:33,929][20381] Updated weights for policy 1, policy_version 707539 (0.0008) [2023-12-27 10:31:33,999][20347] Updated weights for policy 0, policy_version 708017 (0.0005) [2023-12-27 10:31:34,068][20347] Updated weights for policy 0, policy_version 708027 (0.0005) [2023-12-27 10:31:34,120][20347] Updated weights for policy 0, policy_version 708037 (0.0007) [2023-12-27 10:31:34,629][20381] Updated weights for policy 1, policy_version 707549 (0.0009) [2023-12-27 10:31:34,652][20292] KL-divergence is very high: 127.1106 [2023-12-27 10:31:34,688][20381] Updated weights for policy 1, policy_version 707559 (0.0008) [2023-12-27 10:31:34,700][20292] KL-divergence is very high: 248.6349 [2023-12-27 10:31:34,722][20347] Updated weights for policy 0, policy_version 708047 (0.0009) [2023-12-27 10:31:34,724][20292] KL-divergence is very high: 153.7924 [2023-12-27 10:31:34,743][20292] KL-divergence is very high: 242.5658 [2023-12-27 10:31:34,748][20381] Updated weights for policy 1, policy_version 707569 (0.0008) [2023-12-27 10:31:34,749][20292] KL-divergence is very high: 361.6348 [2023-12-27 10:31:34,771][20292] KL-divergence is very high: 106.0801 [2023-12-27 10:31:34,783][20347] Updated weights for policy 0, policy_version 708057 (0.0010) [2023-12-27 10:31:34,834][20347] Updated weights for policy 0, policy_version 708067 (0.0010) [2023-12-27 10:31:35,342][20292] KL-divergence is very high: 658.8409 [2023-12-27 10:31:35,343][20381] Updated weights for policy 1, policy_version 707579 (0.0008) [2023-12-27 10:31:35,364][20292] KL-divergence is very high: 589.4039 [2023-12-27 10:31:35,385][20292] KL-divergence is very high: 270.7280 [2023-12-27 10:31:35,390][20292] KL-divergence is very high: 948.9973 [2023-12-27 10:31:35,396][20381] Updated weights for policy 1, policy_version 707589 (0.0010) [2023-12-27 10:31:35,407][20292] KL-divergence is very high: 303.2281 [2023-12-27 10:31:35,430][20292] KL-divergence is very high: 742.3228 [2023-12-27 10:31:35,435][20347] Updated weights for policy 0, policy_version 708077 (0.0008) [2023-12-27 10:31:35,455][20381] Updated weights for policy 1, policy_version 707599 (0.0011) [2023-12-27 10:31:35,477][20292] KL-divergence is very high: 306.9852 [2023-12-27 10:31:35,487][20347] Updated weights for policy 0, policy_version 708087 (0.0006) [2023-12-27 10:31:35,500][20292] KL-divergence is very high: 116.3330 [2023-12-27 10:31:35,545][20347] Updated weights for policy 0, policy_version 708097 (0.0005) [2023-12-27 10:31:36,061][20347] Updated weights for policy 0, policy_version 708107 (0.0007) [2023-12-27 10:31:36,116][20381] Updated weights for policy 1, policy_version 707609 (0.0010) [2023-12-27 10:31:36,119][20347] Updated weights for policy 0, policy_version 708117 (0.0009) [2023-12-27 10:31:36,177][20381] Updated weights for policy 1, policy_version 707619 (0.0006) [2023-12-27 10:31:36,177][20347] Updated weights for policy 0, policy_version 708127 (0.0007) [2023-12-27 10:31:36,243][20381] Updated weights for policy 1, policy_version 707629 (0.0006) [2023-12-27 10:31:36,303][20381] Updated weights for policy 1, policy_version 707639 (0.0006) [2023-12-27 10:31:36,891][20347] Updated weights for policy 0, policy_version 708137 (0.0010) [2023-12-27 10:31:36,958][20347] Updated weights for policy 0, policy_version 708147 (0.0008) [2023-12-27 10:31:36,990][20381] Updated weights for policy 1, policy_version 707649 (0.0006) [2023-12-27 10:31:37,023][20347] Updated weights for policy 0, policy_version 708157 (0.0010) [2023-12-27 10:31:37,049][20381] Updated weights for policy 1, policy_version 707659 (0.0006) [2023-12-27 10:31:37,087][20347] Updated weights for policy 0, policy_version 708167 (0.0010) [2023-12-27 10:31:37,108][20381] Updated weights for policy 1, policy_version 707669 (0.0007) [2023-12-27 10:31:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 21162.6, 300 sec: 20910.4). Total num frames: 362504192. Throughput: 0: 10617.9, 1: 10629.0. Samples: 362469604. Policy #0 lag: (min: 12.0, avg: 13.4, max: 28.0) [2023-12-27 10:31:37,363][19170] Avg episode reward: [(0, '695.640'), (1, '786.420')] [2023-12-27 10:31:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000707672_181190656.pth... [2023-12-27 10:31:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000708168_181313536.pth... [2023-12-27 10:31:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000706424_180871168.pth [2023-12-27 10:31:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000706920_180994048.pth [2023-12-27 10:31:37,772][20347] Updated weights for policy 0, policy_version 708177 (0.0007) [2023-12-27 10:31:37,830][20347] Updated weights for policy 0, policy_version 708187 (0.0006) [2023-12-27 10:31:37,863][20381] Updated weights for policy 1, policy_version 707679 (0.0009) [2023-12-27 10:31:37,891][20347] Updated weights for policy 0, policy_version 708197 (0.0006) [2023-12-27 10:31:37,918][20292] KL-divergence is very high: 164.8603 [2023-12-27 10:31:37,924][20381] Updated weights for policy 1, policy_version 707689 (0.0008) [2023-12-27 10:31:37,936][20292] KL-divergence is very high: 236.5668 [2023-12-27 10:31:37,943][20292] KL-divergence is very high: 143.7231 [2023-12-27 10:31:37,985][20381] Updated weights for policy 1, policy_version 707699 (0.0007) [2023-12-27 10:31:37,985][20292] KL-divergence is very high: 222.4904 [2023-12-27 10:31:37,991][20292] KL-divergence is very high: 232.6050 [2023-12-27 10:31:38,499][20347] Updated weights for policy 0, policy_version 708207 (0.0007) [2023-12-27 10:31:38,560][20347] Updated weights for policy 0, policy_version 708217 (0.0008) [2023-12-27 10:31:38,622][20347] Updated weights for policy 0, policy_version 708227 (0.0008) [2023-12-27 10:31:38,742][20292] KL-divergence is very high: 315.8810 [2023-12-27 10:31:38,750][20381] Updated weights for policy 1, policy_version 707709 (0.0010) [2023-12-27 10:31:38,792][20292] KL-divergence is very high: 167.1411 [2023-12-27 10:31:38,811][20381] Updated weights for policy 1, policy_version 707719 (0.0011) [2023-12-27 10:31:38,844][20292] KL-divergence is very high: 252.1642 [2023-12-27 10:31:38,878][20381] Updated weights for policy 1, policy_version 707729 (0.0011) [2023-12-27 10:31:38,898][20292] KL-divergence is very high: 280.0759 [2023-12-27 10:31:39,336][20347] Updated weights for policy 0, policy_version 708237 (0.0010) [2023-12-27 10:31:39,417][20347] Updated weights for policy 0, policy_version 708247 (0.0008) [2023-12-27 10:31:39,485][20347] Updated weights for policy 0, policy_version 708257 (0.0008) [2023-12-27 10:31:39,486][20291] KL-divergence is very high: 105.7563 [2023-12-27 10:31:39,635][20381] Updated weights for policy 1, policy_version 707739 (0.0010) [2023-12-27 10:31:39,698][20381] Updated weights for policy 1, policy_version 707749 (0.0009) [2023-12-27 10:31:39,760][20381] Updated weights for policy 1, policy_version 707759 (0.0009) [2023-12-27 10:31:40,185][20347] Updated weights for policy 0, policy_version 708267 (0.0007) [2023-12-27 10:31:40,246][20347] Updated weights for policy 0, policy_version 708277 (0.0007) [2023-12-27 10:31:40,311][20347] Updated weights for policy 0, policy_version 708287 (0.0007) [2023-12-27 10:31:40,425][20381] Updated weights for policy 1, policy_version 707769 (0.0008) [2023-12-27 10:31:40,485][20381] Updated weights for policy 1, policy_version 707779 (0.0008) [2023-12-27 10:31:40,531][20381] Updated weights for policy 1, policy_version 707789 (0.0006) [2023-12-27 10:31:40,582][20381] Updated weights for policy 1, policy_version 707799 (0.0007) [2023-12-27 10:31:41,008][20347] Updated weights for policy 0, policy_version 708297 (0.0007) [2023-12-27 10:31:41,075][20347] Updated weights for policy 0, policy_version 708307 (0.0008) [2023-12-27 10:31:41,145][20347] Updated weights for policy 0, policy_version 708317 (0.0009) [2023-12-27 10:31:41,210][20347] Updated weights for policy 0, policy_version 708327 (0.0008) [2023-12-27 10:31:41,309][20292] KL-divergence is very high: 111.0072 [2023-12-27 10:31:41,358][20381] Updated weights for policy 1, policy_version 707809 (0.0008) [2023-12-27 10:31:41,424][20381] Updated weights for policy 1, policy_version 707819 (0.0007) [2023-12-27 10:31:41,429][20292] KL-divergence is very high: 127.5149 [2023-12-27 10:31:41,475][20292] KL-divergence is very high: 113.0131 [2023-12-27 10:31:41,482][20381] Updated weights for policy 1, policy_version 707829 (0.0008) [2023-12-27 10:31:41,897][20347] Updated weights for policy 0, policy_version 708337 (0.0007) [2023-12-27 10:31:41,959][20347] Updated weights for policy 0, policy_version 708347 (0.0006) [2023-12-27 10:31:42,014][20347] Updated weights for policy 0, policy_version 708357 (0.0008) [2023-12-27 10:31:42,257][20381] Updated weights for policy 1, policy_version 707839 (0.0009) [2023-12-27 10:31:42,318][20381] Updated weights for policy 1, policy_version 707849 (0.0007) [2023-12-27 10:31:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 362594304. Throughput: 0: 10598.1, 1: 10458.0. Samples: 362590416. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:31:42,363][19170] Avg episode reward: [(0, '554.040'), (1, '718.520')] [2023-12-27 10:31:42,390][20381] Updated weights for policy 1, policy_version 707859 (0.0010) [2023-12-27 10:31:42,718][20347] Updated weights for policy 0, policy_version 708367 (0.0008) [2023-12-27 10:31:42,778][20347] Updated weights for policy 0, policy_version 708377 (0.0008) [2023-12-27 10:31:42,842][20347] Updated weights for policy 0, policy_version 708387 (0.0008) [2023-12-27 10:31:43,058][20292] KL-divergence is very high: 123.1399 [2023-12-27 10:31:43,065][20381] Updated weights for policy 1, policy_version 707869 (0.0007) [2023-12-27 10:31:43,067][20292] KL-divergence is very high: 118.2178 [2023-12-27 10:31:43,097][20292] KL-divergence is very high: 100.1373 [2023-12-27 10:31:43,103][20292] KL-divergence is very high: 136.3129 [2023-12-27 10:31:43,113][20381] Updated weights for policy 1, policy_version 707879 (0.0006) [2023-12-27 10:31:43,173][20381] Updated weights for policy 1, policy_version 707889 (0.0006) [2023-12-27 10:31:43,514][20347] Updated weights for policy 0, policy_version 708397 (0.0007) [2023-12-27 10:31:43,573][20347] Updated weights for policy 0, policy_version 708407 (0.0008) [2023-12-27 10:31:43,632][20347] Updated weights for policy 0, policy_version 708417 (0.0008) [2023-12-27 10:31:43,865][20381] Updated weights for policy 1, policy_version 707899 (0.0008) [2023-12-27 10:31:43,902][20292] KL-divergence is very high: 143.0992 [2023-12-27 10:31:43,908][20292] KL-divergence is very high: 153.0088 [2023-12-27 10:31:43,914][20292] KL-divergence is very high: 119.3743 [2023-12-27 10:31:43,924][20381] Updated weights for policy 1, policy_version 707909 (0.0008) [2023-12-27 10:31:43,931][20292] KL-divergence is very high: 105.9149 [2023-12-27 10:31:43,980][20381] Updated weights for policy 1, policy_version 707919 (0.0006) [2023-12-27 10:31:44,237][20347] Updated weights for policy 0, policy_version 708427 (0.0008) [2023-12-27 10:31:44,293][20347] Updated weights for policy 0, policy_version 708437 (0.0008) [2023-12-27 10:31:44,352][20347] Updated weights for policy 0, policy_version 708447 (0.0008) [2023-12-27 10:31:44,693][20381] Updated weights for policy 1, policy_version 707929 (0.0009) [2023-12-27 10:31:44,760][20381] Updated weights for policy 1, policy_version 707939 (0.0007) [2023-12-27 10:31:44,831][20381] Updated weights for policy 1, policy_version 707949 (0.0008) [2023-12-27 10:31:44,896][20381] Updated weights for policy 1, policy_version 707959 (0.0008) [2023-12-27 10:31:45,070][20347] Updated weights for policy 0, policy_version 708457 (0.0008) [2023-12-27 10:31:45,135][20347] Updated weights for policy 0, policy_version 708467 (0.0007) [2023-12-27 10:31:45,201][20347] Updated weights for policy 0, policy_version 708477 (0.0008) [2023-12-27 10:31:45,265][20347] Updated weights for policy 0, policy_version 708487 (0.0007) [2023-12-27 10:31:45,647][20381] Updated weights for policy 1, policy_version 707969 (0.0007) [2023-12-27 10:31:45,712][20381] Updated weights for policy 1, policy_version 707979 (0.0008) [2023-12-27 10:31:45,769][20381] Updated weights for policy 1, policy_version 707989 (0.0008) [2023-12-27 10:31:45,990][20347] Updated weights for policy 0, policy_version 708497 (0.0007) [2023-12-27 10:31:46,053][20347] Updated weights for policy 0, policy_version 708507 (0.0008) [2023-12-27 10:31:46,117][20347] Updated weights for policy 0, policy_version 708517 (0.0008) [2023-12-27 10:31:46,484][20381] Updated weights for policy 1, policy_version 707999 (0.0007) [2023-12-27 10:31:46,547][20381] Updated weights for policy 1, policy_version 708009 (0.0007) [2023-12-27 10:31:46,604][20381] Updated weights for policy 1, policy_version 708019 (0.0006) [2023-12-27 10:31:46,811][20347] Updated weights for policy 0, policy_version 708527 (0.0009) [2023-12-27 10:31:46,866][20347] Updated weights for policy 0, policy_version 708537 (0.0007) [2023-12-27 10:31:46,919][20347] Updated weights for policy 0, policy_version 708547 (0.0007) [2023-12-27 10:31:47,297][20381] Updated weights for policy 1, policy_version 708029 (0.0008) [2023-12-27 10:31:47,357][20381] Updated weights for policy 1, policy_version 708039 (0.0008) [2023-12-27 10:31:47,362][19170] Fps is (10 sec: 18841.8, 60 sec: 20889.6, 300 sec: 20827.1). Total num frames: 362692608. Throughput: 0: 10443.2, 1: 10396.0. Samples: 362706440. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:31:47,363][19170] Avg episode reward: [(0, '578.030'), (1, '178.950')] [2023-12-27 10:31:47,411][20381] Updated weights for policy 1, policy_version 708049 (0.0007) [2023-12-27 10:31:47,623][20347] Updated weights for policy 0, policy_version 708557 (0.0008) [2023-12-27 10:31:47,677][20347] Updated weights for policy 0, policy_version 708567 (0.0007) [2023-12-27 10:31:47,730][20347] Updated weights for policy 0, policy_version 708577 (0.0007) [2023-12-27 10:31:48,105][20381] Updated weights for policy 1, policy_version 708059 (0.0009) [2023-12-27 10:31:48,162][20381] Updated weights for policy 1, policy_version 708069 (0.0007) [2023-12-27 10:31:48,221][20381] Updated weights for policy 1, policy_version 708079 (0.0010) [2023-12-27 10:31:48,432][20347] Updated weights for policy 0, policy_version 708587 (0.0007) [2023-12-27 10:31:48,490][20347] Updated weights for policy 0, policy_version 708597 (0.0008) [2023-12-27 10:31:48,555][20347] Updated weights for policy 0, policy_version 708607 (0.0006) [2023-12-27 10:31:48,999][20381] Updated weights for policy 1, policy_version 708089 (0.0009) [2023-12-27 10:31:49,060][20381] Updated weights for policy 1, policy_version 708099 (0.0006) [2023-12-27 10:31:49,122][20381] Updated weights for policy 1, policy_version 708109 (0.0005) [2023-12-27 10:31:49,185][20381] Updated weights for policy 1, policy_version 708119 (0.0006) [2023-12-27 10:31:49,255][20347] Updated weights for policy 0, policy_version 708617 (0.0008) [2023-12-27 10:31:49,318][20347] Updated weights for policy 0, policy_version 708627 (0.0007) [2023-12-27 10:31:49,389][20347] Updated weights for policy 0, policy_version 708637 (0.0009) [2023-12-27 10:31:49,454][20347] Updated weights for policy 0, policy_version 708647 (0.0008) [2023-12-27 10:31:49,873][20381] Updated weights for policy 1, policy_version 708129 (0.0008) [2023-12-27 10:31:49,933][20381] Updated weights for policy 1, policy_version 708139 (0.0007) [2023-12-27 10:31:49,998][20381] Updated weights for policy 1, policy_version 708149 (0.0006) [2023-12-27 10:31:50,192][20347] Updated weights for policy 0, policy_version 708657 (0.0008) [2023-12-27 10:31:50,254][20347] Updated weights for policy 0, policy_version 708667 (0.0007) [2023-12-27 10:31:50,315][20347] Updated weights for policy 0, policy_version 708677 (0.0008) [2023-12-27 10:31:50,716][20381] Updated weights for policy 1, policy_version 708159 (0.0007) [2023-12-27 10:31:50,783][20381] Updated weights for policy 1, policy_version 708169 (0.0007) [2023-12-27 10:31:50,787][20292] KL-divergence is very high: 102.6402 [2023-12-27 10:31:50,839][20381] Updated weights for policy 1, policy_version 708179 (0.0008) [2023-12-27 10:31:51,072][20347] Updated weights for policy 0, policy_version 708687 (0.0008) [2023-12-27 10:31:51,137][20347] Updated weights for policy 0, policy_version 708697 (0.0008) [2023-12-27 10:31:51,203][20347] Updated weights for policy 0, policy_version 708707 (0.0008) [2023-12-27 10:31:51,620][20381] Updated weights for policy 1, policy_version 708189 (0.0008) [2023-12-27 10:31:51,688][20381] Updated weights for policy 1, policy_version 708199 (0.0009) [2023-12-27 10:31:51,753][20381] Updated weights for policy 1, policy_version 708209 (0.0009) [2023-12-27 10:31:52,020][20347] Updated weights for policy 0, policy_version 708717 (0.0008) [2023-12-27 10:31:52,080][20347] Updated weights for policy 0, policy_version 708727 (0.0008) [2023-12-27 10:31:52,147][20347] Updated weights for policy 0, policy_version 708737 (0.0007) [2023-12-27 10:31:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20889.6, 300 sec: 20799.3). Total num frames: 362790912. Throughput: 0: 10371.8, 1: 10367.8. Samples: 362764256. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:31:52,363][19170] Avg episode reward: [(0, '627.110'), (1, '112.990')] [2023-12-27 10:31:52,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000708216_181329920.pth... [2023-12-27 10:31:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000708744_181460992.pth... [2023-12-27 10:31:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000707032_181026816.pth [2023-12-27 10:31:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000707528_181149696.pth [2023-12-27 10:31:52,525][20381] Updated weights for policy 1, policy_version 708219 (0.0008) [2023-12-27 10:31:52,589][20381] Updated weights for policy 1, policy_version 708229 (0.0008) [2023-12-27 10:31:52,648][20381] Updated weights for policy 1, policy_version 708239 (0.0008) [2023-12-27 10:31:52,901][20347] Updated weights for policy 0, policy_version 708747 (0.0008) [2023-12-27 10:31:52,959][20347] Updated weights for policy 0, policy_version 708757 (0.0007) [2023-12-27 10:31:53,024][20347] Updated weights for policy 0, policy_version 708767 (0.0008) [2023-12-27 10:31:53,400][20381] Updated weights for policy 1, policy_version 708249 (0.0008) [2023-12-27 10:31:53,462][20381] Updated weights for policy 1, policy_version 708259 (0.0008) [2023-12-27 10:31:53,524][20381] Updated weights for policy 1, policy_version 708269 (0.0008) [2023-12-27 10:31:53,578][20381] Updated weights for policy 1, policy_version 708279 (0.0007) [2023-12-27 10:31:53,731][20347] Updated weights for policy 0, policy_version 708777 (0.0008) [2023-12-27 10:31:53,784][20347] Updated weights for policy 0, policy_version 708787 (0.0007) [2023-12-27 10:31:53,841][20347] Updated weights for policy 0, policy_version 708797 (0.0007) [2023-12-27 10:31:53,904][20347] Updated weights for policy 0, policy_version 708807 (0.0007) [2023-12-27 10:31:54,311][20381] Updated weights for policy 1, policy_version 708289 (0.0006) [2023-12-27 10:31:54,372][20381] Updated weights for policy 1, policy_version 708299 (0.0006) [2023-12-27 10:31:54,445][20381] Updated weights for policy 1, policy_version 708309 (0.0007) [2023-12-27 10:31:54,630][20347] Updated weights for policy 0, policy_version 708817 (0.0007) [2023-12-27 10:31:54,686][20347] Updated weights for policy 0, policy_version 708827 (0.0006) [2023-12-27 10:31:54,746][20347] Updated weights for policy 0, policy_version 708837 (0.0008) [2023-12-27 10:31:55,182][20381] Updated weights for policy 1, policy_version 708319 (0.0009) [2023-12-27 10:31:55,240][20381] Updated weights for policy 1, policy_version 708329 (0.0008) [2023-12-27 10:31:55,305][20381] Updated weights for policy 1, policy_version 708339 (0.0009) [2023-12-27 10:31:55,438][20347] Updated weights for policy 0, policy_version 708847 (0.0007) [2023-12-27 10:31:55,493][20347] Updated weights for policy 0, policy_version 708857 (0.0006) [2023-12-27 10:31:55,540][20347] Updated weights for policy 0, policy_version 708867 (0.0007) [2023-12-27 10:31:55,969][20381] Updated weights for policy 1, policy_version 708349 (0.0008) [2023-12-27 10:31:56,026][20381] Updated weights for policy 1, policy_version 708359 (0.0010) [2023-12-27 10:31:56,080][20381] Updated weights for policy 1, policy_version 708369 (0.0010) [2023-12-27 10:31:56,155][20347] Updated weights for policy 0, policy_version 708877 (0.0005) [2023-12-27 10:31:56,203][20347] Updated weights for policy 0, policy_version 708887 (0.0005) [2023-12-27 10:31:56,254][20347] Updated weights for policy 0, policy_version 708897 (0.0005) [2023-12-27 10:31:56,743][20381] Updated weights for policy 1, policy_version 708379 (0.0010) [2023-12-27 10:31:56,800][20381] Updated weights for policy 1, policy_version 708389 (0.0008) [2023-12-27 10:31:56,855][20381] Updated weights for policy 1, policy_version 708400 (0.0010) [2023-12-27 10:31:56,878][20347] Updated weights for policy 0, policy_version 708907 (0.0005) [2023-12-27 10:31:56,925][20347] Updated weights for policy 0, policy_version 708917 (0.0007) [2023-12-27 10:31:56,973][20347] Updated weights for policy 0, policy_version 708927 (0.0009) [2023-12-27 10:31:56,992][20291] KL-divergence is very high: 127.5736 [2023-12-27 10:31:57,001][20291] KL-divergence is very high: 163.5266 [2023-12-27 10:31:57,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20799.3). Total num frames: 362889216. Throughput: 0: 10247.9, 1: 10191.7. Samples: 362877284. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:31:57,363][19170] Avg episode reward: [(0, '708.330'), (1, '286.000')] [2023-12-27 10:31:57,543][20381] Updated weights for policy 1, policy_version 708410 (0.0009) [2023-12-27 10:31:57,591][20381] Updated weights for policy 1, policy_version 708420 (0.0008) [2023-12-27 10:31:57,618][20347] Updated weights for policy 0, policy_version 708937 (0.0010) [2023-12-27 10:31:57,643][20291] KL-divergence is very high: 168.3466 [2023-12-27 10:31:57,649][20381] Updated weights for policy 1, policy_version 708430 (0.0006) [2023-12-27 10:31:57,666][20347] Updated weights for policy 0, policy_version 708947 (0.0010) [2023-12-27 10:31:57,681][20291] KL-divergence is very high: 202.8272 [2023-12-27 10:31:57,704][20381] Updated weights for policy 1, policy_version 708440 (0.0006) [2023-12-27 10:31:57,715][20347] Updated weights for policy 0, policy_version 708957 (0.0010) [2023-12-27 10:31:57,719][20291] KL-divergence is very high: 156.5643 [2023-12-27 10:31:57,761][20291] KL-divergence is very high: 234.9566 [2023-12-27 10:31:57,767][20347] Updated weights for policy 0, policy_version 708967 (0.0009) [2023-12-27 10:31:58,351][20381] Updated weights for policy 1, policy_version 708451 (0.0008) [2023-12-27 10:31:58,409][20381] Updated weights for policy 1, policy_version 708461 (0.0010) [2023-12-27 10:31:58,472][20381] Updated weights for policy 1, policy_version 708471 (0.0010) [2023-12-27 10:31:58,487][20347] Updated weights for policy 0, policy_version 708977 (0.0010) [2023-12-27 10:31:58,547][20347] Updated weights for policy 0, policy_version 708987 (0.0008) [2023-12-27 10:31:58,615][20347] Updated weights for policy 0, policy_version 708997 (0.0008) [2023-12-27 10:31:59,197][20381] Updated weights for policy 1, policy_version 708481 (0.0010) [2023-12-27 10:31:59,260][20381] Updated weights for policy 1, policy_version 708491 (0.0008) [2023-12-27 10:31:59,307][20381] Updated weights for policy 1, policy_version 708501 (0.0009) [2023-12-27 10:31:59,347][20347] Updated weights for policy 0, policy_version 709007 (0.0008) [2023-12-27 10:31:59,417][20347] Updated weights for policy 0, policy_version 709017 (0.0006) [2023-12-27 10:31:59,471][20347] Updated weights for policy 0, policy_version 709027 (0.0008) [2023-12-27 10:32:00,004][20381] Updated weights for policy 1, policy_version 708511 (0.0006) [2023-12-27 10:32:00,059][20381] Updated weights for policy 1, policy_version 708521 (0.0008) [2023-12-27 10:32:00,112][20381] Updated weights for policy 1, policy_version 708531 (0.0008) [2023-12-27 10:32:00,207][20347] Updated weights for policy 0, policy_version 709037 (0.0008) [2023-12-27 10:32:00,264][20347] Updated weights for policy 0, policy_version 709047 (0.0007) [2023-12-27 10:32:00,321][20347] Updated weights for policy 0, policy_version 709057 (0.0007) [2023-12-27 10:32:00,795][20381] Updated weights for policy 1, policy_version 708541 (0.0008) [2023-12-27 10:32:00,850][20381] Updated weights for policy 1, policy_version 708551 (0.0008) [2023-12-27 10:32:00,899][20381] Updated weights for policy 1, policy_version 708561 (0.0007) [2023-12-27 10:32:01,042][20347] Updated weights for policy 0, policy_version 709067 (0.0008) [2023-12-27 10:32:01,096][20347] Updated weights for policy 0, policy_version 709077 (0.0005) [2023-12-27 10:32:01,156][20347] Updated weights for policy 0, policy_version 709087 (0.0008) [2023-12-27 10:32:01,592][20381] Updated weights for policy 1, policy_version 708571 (0.0008) [2023-12-27 10:32:01,654][20381] Updated weights for policy 1, policy_version 708581 (0.0007) [2023-12-27 10:32:01,703][20381] Updated weights for policy 1, policy_version 708591 (0.0008) [2023-12-27 10:32:01,826][20347] Updated weights for policy 0, policy_version 709097 (0.0008) [2023-12-27 10:32:01,883][20347] Updated weights for policy 0, policy_version 709107 (0.0005) [2023-12-27 10:32:01,942][20347] Updated weights for policy 0, policy_version 709117 (0.0005) [2023-12-27 10:32:02,004][20347] Updated weights for policy 0, policy_version 709127 (0.0007) [2023-12-27 10:32:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20743.8). Total num frames: 362987520. Throughput: 0: 10235.3, 1: 10093.7. Samples: 362999300. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:02,363][19170] Avg episode reward: [(0, '726.750'), (1, '406.250')] [2023-12-27 10:32:02,406][20381] Updated weights for policy 1, policy_version 708601 (0.0009) [2023-12-27 10:32:02,466][20381] Updated weights for policy 1, policy_version 708611 (0.0008) [2023-12-27 10:32:02,524][20381] Updated weights for policy 1, policy_version 708621 (0.0007) [2023-12-27 10:32:02,578][20381] Updated weights for policy 1, policy_version 708631 (0.0008) [2023-12-27 10:32:02,631][20347] Updated weights for policy 0, policy_version 709137 (0.0008) [2023-12-27 10:32:02,690][20347] Updated weights for policy 0, policy_version 709147 (0.0007) [2023-12-27 10:32:02,741][20347] Updated weights for policy 0, policy_version 709157 (0.0008) [2023-12-27 10:32:03,294][20381] Updated weights for policy 1, policy_version 708641 (0.0008) [2023-12-27 10:32:03,317][20291] KL-divergence is very high: 111.7717 [2023-12-27 10:32:03,350][20381] Updated weights for policy 1, policy_version 708651 (0.0007) [2023-12-27 10:32:03,352][20347] Updated weights for policy 0, policy_version 709167 (0.0009) [2023-12-27 10:32:03,404][20381] Updated weights for policy 1, policy_version 708661 (0.0005) [2023-12-27 10:32:03,406][20347] Updated weights for policy 0, policy_version 709177 (0.0010) [2023-12-27 10:32:03,457][20347] Updated weights for policy 0, policy_version 709187 (0.0010) [2023-12-27 10:32:04,099][20381] Updated weights for policy 1, policy_version 708671 (0.0006) [2023-12-27 10:32:04,146][20347] Updated weights for policy 0, policy_version 709197 (0.0009) [2023-12-27 10:32:04,159][20381] Updated weights for policy 1, policy_version 708681 (0.0006) [2023-12-27 10:32:04,206][20347] Updated weights for policy 0, policy_version 709207 (0.0008) [2023-12-27 10:32:04,216][20381] Updated weights for policy 1, policy_version 708691 (0.0006) [2023-12-27 10:32:04,265][20347] Updated weights for policy 0, policy_version 709217 (0.0007) [2023-12-27 10:32:04,903][20381] Updated weights for policy 1, policy_version 708701 (0.0006) [2023-12-27 10:32:04,948][20381] Updated weights for policy 1, policy_version 708711 (0.0007) [2023-12-27 10:32:04,956][20347] Updated weights for policy 0, policy_version 709227 (0.0007) [2023-12-27 10:32:04,997][20381] Updated weights for policy 1, policy_version 708721 (0.0006) [2023-12-27 10:32:05,007][20347] Updated weights for policy 0, policy_version 709237 (0.0006) [2023-12-27 10:32:05,062][20347] Updated weights for policy 0, policy_version 709247 (0.0007) [2023-12-27 10:32:05,595][20381] Updated weights for policy 1, policy_version 708731 (0.0006) [2023-12-27 10:32:05,640][20381] Updated weights for policy 1, policy_version 708741 (0.0007) [2023-12-27 10:32:05,692][20381] Updated weights for policy 1, policy_version 708751 (0.0008) [2023-12-27 10:32:05,730][20347] Updated weights for policy 0, policy_version 709257 (0.0008) [2023-12-27 10:32:05,790][20347] Updated weights for policy 0, policy_version 709267 (0.0006) [2023-12-27 10:32:05,845][20347] Updated weights for policy 0, policy_version 709277 (0.0006) [2023-12-27 10:32:05,904][20347] Updated weights for policy 0, policy_version 709287 (0.0007) [2023-12-27 10:32:06,405][20381] Updated weights for policy 1, policy_version 708761 (0.0006) [2023-12-27 10:32:06,464][20381] Updated weights for policy 1, policy_version 708771 (0.0008) [2023-12-27 10:32:06,527][20381] Updated weights for policy 1, policy_version 708781 (0.0008) [2023-12-27 10:32:06,583][20347] Updated weights for policy 0, policy_version 709297 (0.0006) [2023-12-27 10:32:06,585][20381] Updated weights for policy 1, policy_version 708791 (0.0006) [2023-12-27 10:32:06,642][20347] Updated weights for policy 0, policy_version 709307 (0.0008) [2023-12-27 10:32:06,703][20347] Updated weights for policy 0, policy_version 709317 (0.0008) [2023-12-27 10:32:07,250][20381] Updated weights for policy 1, policy_version 708801 (0.0008) [2023-12-27 10:32:07,299][20381] Updated weights for policy 1, policy_version 708811 (0.0007) [2023-12-27 10:32:07,350][20347] Updated weights for policy 0, policy_version 709327 (0.0010) [2023-12-27 10:32:07,351][20381] Updated weights for policy 1, policy_version 708821 (0.0006) [2023-12-27 10:32:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20743.8). Total num frames: 363085824. Throughput: 0: 10224.8, 1: 10051.2. Samples: 363060436. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:07,363][19170] Avg episode reward: [(0, '827.770'), (1, '545.180')] [2023-12-27 10:32:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000708824_181485568.pth... [2023-12-27 10:32:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000707672_181190656.pth [2023-12-27 10:32:07,409][20347] Updated weights for policy 0, policy_version 709337 (0.0010) [2023-12-27 10:32:07,467][20347] Updated weights for policy 0, policy_version 709347 (0.0010) [2023-12-27 10:32:07,499][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000709352_181616640.pth... [2023-12-27 10:32:07,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000708168_181313536.pth [2023-12-27 10:32:08,059][20381] Updated weights for policy 1, policy_version 708831 (0.0007) [2023-12-27 10:32:08,102][20347] Updated weights for policy 0, policy_version 709357 (0.0008) [2023-12-27 10:32:08,125][20381] Updated weights for policy 1, policy_version 708841 (0.0008) [2023-12-27 10:32:08,131][20292] KL-divergence is very high: 100.9252 [2023-12-27 10:32:08,138][20292] KL-divergence is very high: 112.6721 [2023-12-27 10:32:08,152][20347] Updated weights for policy 0, policy_version 709367 (0.0006) [2023-12-27 10:32:08,184][20381] Updated weights for policy 1, policy_version 708851 (0.0007) [2023-12-27 10:32:08,208][20347] Updated weights for policy 0, policy_version 709377 (0.0008) [2023-12-27 10:32:08,861][20347] Updated weights for policy 0, policy_version 709387 (0.0010) [2023-12-27 10:32:08,900][20381] Updated weights for policy 1, policy_version 708861 (0.0007) [2023-12-27 10:32:08,919][20347] Updated weights for policy 0, policy_version 709397 (0.0007) [2023-12-27 10:32:08,959][20381] Updated weights for policy 1, policy_version 708871 (0.0007) [2023-12-27 10:32:08,970][20347] Updated weights for policy 0, policy_version 709407 (0.0005) [2023-12-27 10:32:09,006][20381] Updated weights for policy 1, policy_version 708881 (0.0006) [2023-12-27 10:32:09,662][20347] Updated weights for policy 0, policy_version 709417 (0.0007) [2023-12-27 10:32:09,726][20347] Updated weights for policy 0, policy_version 709427 (0.0008) [2023-12-27 10:32:09,765][20381] Updated weights for policy 1, policy_version 708891 (0.0007) [2023-12-27 10:32:09,780][20347] Updated weights for policy 0, policy_version 709437 (0.0008) [2023-12-27 10:32:09,830][20381] Updated weights for policy 1, policy_version 708901 (0.0007) [2023-12-27 10:32:09,831][20347] Updated weights for policy 0, policy_version 709447 (0.0008) [2023-12-27 10:32:09,891][20381] Updated weights for policy 1, policy_version 708911 (0.0007) [2023-12-27 10:32:10,564][20347] Updated weights for policy 0, policy_version 709457 (0.0007) [2023-12-27 10:32:10,568][20381] Updated weights for policy 1, policy_version 708921 (0.0007) [2023-12-27 10:32:10,620][20347] Updated weights for policy 0, policy_version 709467 (0.0006) [2023-12-27 10:32:10,626][20381] Updated weights for policy 1, policy_version 708931 (0.0006) [2023-12-27 10:32:10,678][20347] Updated weights for policy 0, policy_version 709477 (0.0009) [2023-12-27 10:32:10,678][20381] Updated weights for policy 1, policy_version 708941 (0.0006) [2023-12-27 10:32:10,737][20381] Updated weights for policy 1, policy_version 708951 (0.0008) [2023-12-27 10:32:11,390][20347] Updated weights for policy 0, policy_version 709487 (0.0008) [2023-12-27 10:32:11,451][20347] Updated weights for policy 0, policy_version 709497 (0.0008) [2023-12-27 10:32:11,463][20381] Updated weights for policy 1, policy_version 708961 (0.0006) [2023-12-27 10:32:11,514][20347] Updated weights for policy 0, policy_version 709507 (0.0010) [2023-12-27 10:32:11,520][20381] Updated weights for policy 1, policy_version 708971 (0.0006) [2023-12-27 10:32:11,581][20381] Updated weights for policy 1, policy_version 708981 (0.0007) [2023-12-27 10:32:12,237][20347] Updated weights for policy 0, policy_version 709517 (0.0009) [2023-12-27 10:32:12,300][20347] Updated weights for policy 0, policy_version 709527 (0.0007) [2023-12-27 10:32:12,325][20381] Updated weights for policy 1, policy_version 708991 (0.0009) [2023-12-27 10:32:12,363][20347] Updated weights for policy 0, policy_version 709537 (0.0008) [2023-12-27 10:32:12,363][19170] Fps is (10 sec: 19659.9, 60 sec: 20343.4, 300 sec: 20743.8). Total num frames: 363184128. Throughput: 0: 10179.7, 1: 9994.3. Samples: 363182872. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:12,364][19170] Avg episode reward: [(0, '970.860'), (1, '698.020')] [2023-12-27 10:32:12,391][20381] Updated weights for policy 1, policy_version 709001 (0.0007) [2023-12-27 10:32:12,450][20381] Updated weights for policy 1, policy_version 709011 (0.0006) [2023-12-27 10:32:12,985][20347] Updated weights for policy 0, policy_version 709547 (0.0008) [2023-12-27 10:32:13,044][20347] Updated weights for policy 0, policy_version 709557 (0.0006) [2023-12-27 10:32:13,096][20347] Updated weights for policy 0, policy_version 709567 (0.0005) [2023-12-27 10:32:13,099][20381] Updated weights for policy 1, policy_version 709021 (0.0006) [2023-12-27 10:32:13,152][20381] Updated weights for policy 1, policy_version 709031 (0.0005) [2023-12-27 10:32:13,204][20381] Updated weights for policy 1, policy_version 709041 (0.0007) [2023-12-27 10:32:13,661][20347] Updated weights for policy 0, policy_version 709577 (0.0006) [2023-12-27 10:32:13,713][20347] Updated weights for policy 0, policy_version 709587 (0.0010) [2023-12-27 10:32:13,765][20347] Updated weights for policy 0, policy_version 709597 (0.0010) [2023-12-27 10:32:13,808][20381] Updated weights for policy 1, policy_version 709051 (0.0007) [2023-12-27 10:32:13,814][20347] Updated weights for policy 0, policy_version 709607 (0.0010) [2023-12-27 10:32:13,862][20381] Updated weights for policy 1, policy_version 709061 (0.0007) [2023-12-27 10:32:13,916][20381] Updated weights for policy 1, policy_version 709071 (0.0007) [2023-12-27 10:32:14,492][20347] Updated weights for policy 0, policy_version 709617 (0.0009) [2023-12-27 10:32:14,540][20347] Updated weights for policy 0, policy_version 709627 (0.0010) [2023-12-27 10:32:14,554][20381] Updated weights for policy 1, policy_version 709081 (0.0005) [2023-12-27 10:32:14,593][20347] Updated weights for policy 0, policy_version 709637 (0.0005) [2023-12-27 10:32:14,608][20381] Updated weights for policy 1, policy_version 709091 (0.0005) [2023-12-27 10:32:14,652][20381] Updated weights for policy 1, policy_version 709101 (0.0005) [2023-12-27 10:32:14,704][20381] Updated weights for policy 1, policy_version 709111 (0.0006) [2023-12-27 10:32:15,266][20347] Updated weights for policy 0, policy_version 709647 (0.0007) [2023-12-27 10:32:15,305][20381] Updated weights for policy 1, policy_version 709121 (0.0008) [2023-12-27 10:32:15,338][20347] Updated weights for policy 0, policy_version 709657 (0.0007) [2023-12-27 10:32:15,370][20381] Updated weights for policy 1, policy_version 709131 (0.0007) [2023-12-27 10:32:15,403][20347] Updated weights for policy 0, policy_version 709667 (0.0008) [2023-12-27 10:32:15,433][20381] Updated weights for policy 1, policy_version 709141 (0.0007) [2023-12-27 10:32:15,993][20347] Updated weights for policy 0, policy_version 709677 (0.0006) [2023-12-27 10:32:16,056][20347] Updated weights for policy 0, policy_version 709687 (0.0005) [2023-12-27 10:32:16,057][20381] Updated weights for policy 1, policy_version 709151 (0.0006) [2023-12-27 10:32:16,114][20381] Updated weights for policy 1, policy_version 709161 (0.0006) [2023-12-27 10:32:16,119][20347] Updated weights for policy 0, policy_version 709697 (0.0010) [2023-12-27 10:32:16,175][20381] Updated weights for policy 1, policy_version 709171 (0.0006) [2023-12-27 10:32:16,693][20347] Updated weights for policy 0, policy_version 709707 (0.0009) [2023-12-27 10:32:16,748][20347] Updated weights for policy 0, policy_version 709717 (0.0005) [2023-12-27 10:32:16,806][20347] Updated weights for policy 0, policy_version 709727 (0.0007) [2023-12-27 10:32:16,852][20381] Updated weights for policy 1, policy_version 709181 (0.0007) [2023-12-27 10:32:16,905][20381] Updated weights for policy 1, policy_version 709191 (0.0008) [2023-12-27 10:32:16,962][20381] Updated weights for policy 1, policy_version 709201 (0.0008) [2023-12-27 10:32:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20343.5, 300 sec: 20771.6). Total num frames: 363298816. Throughput: 0: 10179.3, 1: 9964.9. Samples: 363309372. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:17,363][19170] Avg episode reward: [(0, '971.030'), (1, '775.940')] [2023-12-27 10:32:17,441][20347] Updated weights for policy 0, policy_version 709737 (0.0010) [2023-12-27 10:32:17,496][20347] Updated weights for policy 0, policy_version 709747 (0.0010) [2023-12-27 10:32:17,547][20347] Updated weights for policy 0, policy_version 709757 (0.0010) [2023-12-27 10:32:17,581][20381] Updated weights for policy 1, policy_version 709211 (0.0007) [2023-12-27 10:32:17,604][20347] Updated weights for policy 0, policy_version 709767 (0.0010) [2023-12-27 10:32:17,636][20381] Updated weights for policy 1, policy_version 709221 (0.0006) [2023-12-27 10:32:17,683][20381] Updated weights for policy 1, policy_version 709231 (0.0007) [2023-12-27 10:32:18,279][20347] Updated weights for policy 0, policy_version 709777 (0.0010) [2023-12-27 10:32:18,336][20347] Updated weights for policy 0, policy_version 709787 (0.0009) [2023-12-27 10:32:18,381][20381] Updated weights for policy 1, policy_version 709241 (0.0007) [2023-12-27 10:32:18,399][20347] Updated weights for policy 0, policy_version 709797 (0.0007) [2023-12-27 10:32:18,445][20381] Updated weights for policy 1, policy_version 709251 (0.0006) [2023-12-27 10:32:18,503][20381] Updated weights for policy 1, policy_version 709261 (0.0005) [2023-12-27 10:32:18,564][20381] Updated weights for policy 1, policy_version 709271 (0.0007) [2023-12-27 10:32:19,060][20347] Updated weights for policy 0, policy_version 709807 (0.0010) [2023-12-27 10:32:19,105][20347] Updated weights for policy 0, policy_version 709817 (0.0010) [2023-12-27 10:32:19,151][20347] Updated weights for policy 0, policy_version 709827 (0.0010) [2023-12-27 10:32:19,187][20381] Updated weights for policy 1, policy_version 709281 (0.0009) [2023-12-27 10:32:19,251][20381] Updated weights for policy 1, policy_version 709291 (0.0011) [2023-12-27 10:32:19,308][20381] Updated weights for policy 1, policy_version 709301 (0.0008) [2023-12-27 10:32:19,904][20347] Updated weights for policy 0, policy_version 709837 (0.0008) [2023-12-27 10:32:19,968][20347] Updated weights for policy 0, policy_version 709847 (0.0008) [2023-12-27 10:32:20,025][20347] Updated weights for policy 0, policy_version 709857 (0.0006) [2023-12-27 10:32:20,031][20381] Updated weights for policy 1, policy_version 709311 (0.0008) [2023-12-27 10:32:20,086][20381] Updated weights for policy 1, policy_version 709321 (0.0008) [2023-12-27 10:32:20,150][20381] Updated weights for policy 1, policy_version 709331 (0.0008) [2023-12-27 10:32:20,744][20347] Updated weights for policy 0, policy_version 709867 (0.0007) [2023-12-27 10:32:20,794][20347] Updated weights for policy 0, policy_version 709877 (0.0007) [2023-12-27 10:32:20,854][20347] Updated weights for policy 0, policy_version 709887 (0.0007) [2023-12-27 10:32:20,864][20381] Updated weights for policy 1, policy_version 709341 (0.0007) [2023-12-27 10:32:20,926][20381] Updated weights for policy 1, policy_version 709351 (0.0006) [2023-12-27 10:32:20,984][20381] Updated weights for policy 1, policy_version 709361 (0.0008) [2023-12-27 10:32:21,616][20347] Updated weights for policy 0, policy_version 709897 (0.0006) [2023-12-27 10:32:21,681][20347] Updated weights for policy 0, policy_version 709907 (0.0007) [2023-12-27 10:32:21,741][20381] Updated weights for policy 1, policy_version 709371 (0.0007) [2023-12-27 10:32:21,753][20347] Updated weights for policy 0, policy_version 709917 (0.0008) [2023-12-27 10:32:21,806][20381] Updated weights for policy 1, policy_version 709381 (0.0008) [2023-12-27 10:32:21,817][20347] Updated weights for policy 0, policy_version 709927 (0.0007) [2023-12-27 10:32:21,865][20381] Updated weights for policy 1, policy_version 709391 (0.0008) [2023-12-27 10:32:22,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20343.5, 300 sec: 20743.8). Total num frames: 363397120. Throughput: 0: 10117.4, 1: 9905.0. Samples: 363370612. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:22,363][19170] Avg episode reward: [(0, '970.980'), (1, '736.800')] [2023-12-27 10:32:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000709400_181633024.pth... [2023-12-27 10:32:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000709928_181764096.pth... [2023-12-27 10:32:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000708216_181329920.pth [2023-12-27 10:32:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000708744_181460992.pth [2023-12-27 10:32:22,487][20347] Updated weights for policy 0, policy_version 709937 (0.0006) [2023-12-27 10:32:22,548][20347] Updated weights for policy 0, policy_version 709947 (0.0007) [2023-12-27 10:32:22,607][20347] Updated weights for policy 0, policy_version 709957 (0.0008) [2023-12-27 10:32:22,618][20381] Updated weights for policy 1, policy_version 709401 (0.0008) [2023-12-27 10:32:22,675][20381] Updated weights for policy 1, policy_version 709411 (0.0011) [2023-12-27 10:32:22,731][20381] Updated weights for policy 1, policy_version 709421 (0.0008) [2023-12-27 10:32:22,793][20381] Updated weights for policy 1, policy_version 709431 (0.0006) [2023-12-27 10:32:23,208][20347] Updated weights for policy 0, policy_version 709967 (0.0009) [2023-12-27 10:32:23,268][20347] Updated weights for policy 0, policy_version 709977 (0.0010) [2023-12-27 10:32:23,325][20347] Updated weights for policy 0, policy_version 709987 (0.0010) [2023-12-27 10:32:23,413][20381] Updated weights for policy 1, policy_version 709441 (0.0010) [2023-12-27 10:32:23,477][20381] Updated weights for policy 1, policy_version 709451 (0.0010) [2023-12-27 10:32:23,535][20381] Updated weights for policy 1, policy_version 709461 (0.0010) [2023-12-27 10:32:23,954][20347] Updated weights for policy 0, policy_version 709997 (0.0008) [2023-12-27 10:32:24,017][20347] Updated weights for policy 0, policy_version 710007 (0.0005) [2023-12-27 10:32:24,072][20347] Updated weights for policy 0, policy_version 710017 (0.0007) [2023-12-27 10:32:24,217][20381] Updated weights for policy 1, policy_version 709471 (0.0007) [2023-12-27 10:32:24,267][20381] Updated weights for policy 1, policy_version 709481 (0.0006) [2023-12-27 10:32:24,325][20381] Updated weights for policy 1, policy_version 709491 (0.0006) [2023-12-27 10:32:24,713][20347] Updated weights for policy 0, policy_version 710027 (0.0009) [2023-12-27 10:32:24,778][20347] Updated weights for policy 0, policy_version 710037 (0.0011) [2023-12-27 10:32:24,823][20347] Updated weights for policy 0, policy_version 710047 (0.0010) [2023-12-27 10:32:24,958][20381] Updated weights for policy 1, policy_version 709501 (0.0006) [2023-12-27 10:32:25,015][20381] Updated weights for policy 1, policy_version 709511 (0.0007) [2023-12-27 10:32:25,072][20381] Updated weights for policy 1, policy_version 709521 (0.0007) [2023-12-27 10:32:25,504][20347] Updated weights for policy 0, policy_version 710057 (0.0010) [2023-12-27 10:32:25,561][20347] Updated weights for policy 0, policy_version 710067 (0.0010) [2023-12-27 10:32:25,616][20347] Updated weights for policy 0, policy_version 710077 (0.0010) [2023-12-27 10:32:25,676][20347] Updated weights for policy 0, policy_version 710087 (0.0010) [2023-12-27 10:32:25,755][20381] Updated weights for policy 1, policy_version 709531 (0.0007) [2023-12-27 10:32:25,802][20381] Updated weights for policy 1, policy_version 709541 (0.0007) [2023-12-27 10:32:25,855][20381] Updated weights for policy 1, policy_version 709551 (0.0007) [2023-12-27 10:32:26,360][20347] Updated weights for policy 0, policy_version 710097 (0.0011) [2023-12-27 10:32:26,406][20347] Updated weights for policy 0, policy_version 710107 (0.0010) [2023-12-27 10:32:26,458][20347] Updated weights for policy 0, policy_version 710117 (0.0010) [2023-12-27 10:32:26,521][20381] Updated weights for policy 1, policy_version 709561 (0.0007) [2023-12-27 10:32:26,578][20381] Updated weights for policy 1, policy_version 709571 (0.0010) [2023-12-27 10:32:26,626][20381] Updated weights for policy 1, policy_version 709581 (0.0010) [2023-12-27 10:32:26,676][20381] Updated weights for policy 1, policy_version 709591 (0.0010) [2023-12-27 10:32:27,123][20347] Updated weights for policy 0, policy_version 710127 (0.0008) [2023-12-27 10:32:27,171][20347] Updated weights for policy 0, policy_version 710137 (0.0007) [2023-12-27 10:32:27,225][20347] Updated weights for policy 0, policy_version 710147 (0.0007) [2023-12-27 10:32:27,342][20381] Updated weights for policy 1, policy_version 709601 (0.0009) [2023-12-27 10:32:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20207.0, 300 sec: 20743.8). Total num frames: 363503616. Throughput: 0: 10097.2, 1: 9949.8. Samples: 363492528. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:27,363][19170] Avg episode reward: [(0, '941.570'), (1, '696.980')] [2023-12-27 10:32:27,396][20381] Updated weights for policy 1, policy_version 709611 (0.0010) [2023-12-27 10:32:27,451][20381] Updated weights for policy 1, policy_version 709621 (0.0006) [2023-12-27 10:32:27,924][20347] Updated weights for policy 0, policy_version 710157 (0.0007) [2023-12-27 10:32:27,973][20347] Updated weights for policy 0, policy_version 710167 (0.0005) [2023-12-27 10:32:28,025][20347] Updated weights for policy 0, policy_version 710177 (0.0005) [2023-12-27 10:32:28,095][20381] Updated weights for policy 1, policy_version 709631 (0.0009) [2023-12-27 10:32:28,155][20381] Updated weights for policy 1, policy_version 709641 (0.0010) [2023-12-27 10:32:28,209][20381] Updated weights for policy 1, policy_version 709651 (0.0010) [2023-12-27 10:32:28,642][20347] Updated weights for policy 0, policy_version 710187 (0.0006) [2023-12-27 10:32:28,690][20347] Updated weights for policy 0, policy_version 710197 (0.0007) [2023-12-27 10:32:28,747][20347] Updated weights for policy 0, policy_version 710207 (0.0007) [2023-12-27 10:32:28,889][20381] Updated weights for policy 1, policy_version 709661 (0.0010) [2023-12-27 10:32:28,950][20381] Updated weights for policy 1, policy_version 709671 (0.0010) [2023-12-27 10:32:29,008][20381] Updated weights for policy 1, policy_version 709681 (0.0010) [2023-12-27 10:32:29,428][20347] Updated weights for policy 0, policy_version 710217 (0.0009) [2023-12-27 10:32:29,482][20347] Updated weights for policy 0, policy_version 710227 (0.0007) [2023-12-27 10:32:29,540][20347] Updated weights for policy 0, policy_version 710237 (0.0007) [2023-12-27 10:32:29,588][20347] Updated weights for policy 0, policy_version 710247 (0.0007) [2023-12-27 10:32:29,728][20381] Updated weights for policy 1, policy_version 709691 (0.0010) [2023-12-27 10:32:29,777][20381] Updated weights for policy 1, policy_version 709701 (0.0010) [2023-12-27 10:32:29,836][20381] Updated weights for policy 1, policy_version 709711 (0.0010) [2023-12-27 10:32:30,347][20347] Updated weights for policy 0, policy_version 710257 (0.0005) [2023-12-27 10:32:30,410][20347] Updated weights for policy 0, policy_version 710267 (0.0007) [2023-12-27 10:32:30,454][20347] Updated weights for policy 0, policy_version 710277 (0.0010) [2023-12-27 10:32:30,553][20381] Updated weights for policy 1, policy_version 709721 (0.0008) [2023-12-27 10:32:30,609][20381] Updated weights for policy 1, policy_version 709731 (0.0009) [2023-12-27 10:32:30,667][20381] Updated weights for policy 1, policy_version 709741 (0.0010) [2023-12-27 10:32:30,726][20381] Updated weights for policy 1, policy_version 709752 (0.0009) [2023-12-27 10:32:31,123][20347] Updated weights for policy 0, policy_version 710287 (0.0010) [2023-12-27 10:32:31,184][20347] Updated weights for policy 0, policy_version 710297 (0.0010) [2023-12-27 10:32:31,246][20347] Updated weights for policy 0, policy_version 710307 (0.0006) [2023-12-27 10:32:31,403][20381] Updated weights for policy 1, policy_version 709762 (0.0007) [2023-12-27 10:32:31,459][20381] Updated weights for policy 1, policy_version 709772 (0.0006) [2023-12-27 10:32:31,512][20381] Updated weights for policy 1, policy_version 709782 (0.0006) [2023-12-27 10:32:31,978][20347] Updated weights for policy 0, policy_version 710317 (0.0007) [2023-12-27 10:32:32,036][20347] Updated weights for policy 0, policy_version 710327 (0.0008) [2023-12-27 10:32:32,095][20347] Updated weights for policy 0, policy_version 710337 (0.0008) [2023-12-27 10:32:32,173][20381] Updated weights for policy 1, policy_version 709792 (0.0005) [2023-12-27 10:32:32,220][20381] Updated weights for policy 1, policy_version 709802 (0.0005) [2023-12-27 10:32:32,278][20381] Updated weights for policy 1, policy_version 709812 (0.0008) [2023-12-27 10:32:32,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.4, 300 sec: 20771.6). Total num frames: 363610112. Throughput: 0: 10148.3, 1: 10043.0. Samples: 363615052. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:32,363][19170] Avg episode reward: [(0, '961.140'), (1, '718.920')] [2023-12-27 10:32:32,823][20347] Updated weights for policy 0, policy_version 710347 (0.0007) [2023-12-27 10:32:32,875][20347] Updated weights for policy 0, policy_version 710357 (0.0006) [2023-12-27 10:32:32,928][20347] Updated weights for policy 0, policy_version 710367 (0.0006) [2023-12-27 10:32:32,937][20381] Updated weights for policy 1, policy_version 709822 (0.0009) [2023-12-27 10:32:32,991][20381] Updated weights for policy 1, policy_version 709832 (0.0010) [2023-12-27 10:32:33,039][20381] Updated weights for policy 1, policy_version 709842 (0.0010) [2023-12-27 10:32:33,611][20347] Updated weights for policy 0, policy_version 710377 (0.0005) [2023-12-27 10:32:33,673][20347] Updated weights for policy 0, policy_version 710387 (0.0005) [2023-12-27 10:32:33,678][20381] Updated weights for policy 1, policy_version 709852 (0.0010) [2023-12-27 10:32:33,725][20381] Updated weights for policy 1, policy_version 709862 (0.0010) [2023-12-27 10:32:33,731][20347] Updated weights for policy 0, policy_version 710397 (0.0006) [2023-12-27 10:32:33,773][20381] Updated weights for policy 1, policy_version 709872 (0.0010) [2023-12-27 10:32:33,782][20347] Updated weights for policy 0, policy_version 710407 (0.0005) [2023-12-27 10:32:34,367][20347] Updated weights for policy 0, policy_version 710417 (0.0007) [2023-12-27 10:32:34,426][20347] Updated weights for policy 0, policy_version 710427 (0.0007) [2023-12-27 10:32:34,428][20381] Updated weights for policy 1, policy_version 709882 (0.0008) [2023-12-27 10:32:34,487][20347] Updated weights for policy 0, policy_version 710437 (0.0010) [2023-12-27 10:32:34,487][20381] Updated weights for policy 1, policy_version 709892 (0.0009) [2023-12-27 10:32:34,536][20381] Updated weights for policy 1, policy_version 709902 (0.0007) [2023-12-27 10:32:34,589][20381] Updated weights for policy 1, policy_version 709912 (0.0008) [2023-12-27 10:32:35,160][20347] Updated weights for policy 0, policy_version 710447 (0.0007) [2023-12-27 10:32:35,202][20347] Updated weights for policy 0, policy_version 710457 (0.0007) [2023-12-27 10:32:35,250][20347] Updated weights for policy 0, policy_version 710467 (0.0007) [2023-12-27 10:32:35,296][20381] Updated weights for policy 1, policy_version 709922 (0.0006) [2023-12-27 10:32:35,342][20381] Updated weights for policy 1, policy_version 709932 (0.0008) [2023-12-27 10:32:35,391][20381] Updated weights for policy 1, policy_version 709942 (0.0005) [2023-12-27 10:32:35,950][20347] Updated weights for policy 0, policy_version 710477 (0.0007) [2023-12-27 10:32:36,004][20347] Updated weights for policy 0, policy_version 710487 (0.0007) [2023-12-27 10:32:36,061][20347] Updated weights for policy 0, policy_version 710497 (0.0006) [2023-12-27 10:32:36,071][20381] Updated weights for policy 1, policy_version 709952 (0.0007) [2023-12-27 10:32:36,130][20381] Updated weights for policy 1, policy_version 709962 (0.0011) [2023-12-27 10:32:36,178][20381] Updated weights for policy 1, policy_version 709972 (0.0010) [2023-12-27 10:32:36,750][20347] Updated weights for policy 0, policy_version 710507 (0.0006) [2023-12-27 10:32:36,818][20347] Updated weights for policy 0, policy_version 710517 (0.0007) [2023-12-27 10:32:36,885][20347] Updated weights for policy 0, policy_version 710527 (0.0005) [2023-12-27 10:32:36,937][20381] Updated weights for policy 1, policy_version 709982 (0.0011) [2023-12-27 10:32:36,992][20381] Updated weights for policy 1, policy_version 709992 (0.0009) [2023-12-27 10:32:37,042][20381] Updated weights for policy 1, policy_version 710002 (0.0011) [2023-12-27 10:32:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 20771.6). Total num frames: 363708416. Throughput: 0: 10194.4, 1: 10102.1. Samples: 363677600. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:37,363][19170] Avg episode reward: [(0, '990.230'), (1, '727.170')] [2023-12-27 10:32:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000710008_181788672.pth... [2023-12-27 10:32:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000710536_181919744.pth... [2023-12-27 10:32:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000708824_181485568.pth [2023-12-27 10:32:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000709352_181616640.pth [2023-12-27 10:32:37,520][20347] Updated weights for policy 0, policy_version 710537 (0.0006) [2023-12-27 10:32:37,575][20347] Updated weights for policy 0, policy_version 710547 (0.0007) [2023-12-27 10:32:37,631][20347] Updated weights for policy 0, policy_version 710557 (0.0007) [2023-12-27 10:32:37,687][20347] Updated weights for policy 0, policy_version 710567 (0.0007) [2023-12-27 10:32:37,754][20381] Updated weights for policy 1, policy_version 710012 (0.0011) [2023-12-27 10:32:37,804][20381] Updated weights for policy 1, policy_version 710022 (0.0010) [2023-12-27 10:32:37,860][20381] Updated weights for policy 1, policy_version 710032 (0.0011) [2023-12-27 10:32:38,374][20347] Updated weights for policy 0, policy_version 710577 (0.0008) [2023-12-27 10:32:38,430][20347] Updated weights for policy 0, policy_version 710587 (0.0007) [2023-12-27 10:32:38,486][20347] Updated weights for policy 0, policy_version 710597 (0.0008) [2023-12-27 10:32:38,582][20381] Updated weights for policy 1, policy_version 710042 (0.0011) [2023-12-27 10:32:38,634][20381] Updated weights for policy 1, policy_version 710052 (0.0011) [2023-12-27 10:32:38,688][20381] Updated weights for policy 1, policy_version 710062 (0.0011) [2023-12-27 10:32:38,748][20381] Updated weights for policy 1, policy_version 710072 (0.0011) [2023-12-27 10:32:39,206][20347] Updated weights for policy 0, policy_version 710607 (0.0007) [2023-12-27 10:32:39,272][20347] Updated weights for policy 0, policy_version 710617 (0.0007) [2023-12-27 10:32:39,337][20347] Updated weights for policy 0, policy_version 710627 (0.0008) [2023-12-27 10:32:39,484][20381] Updated weights for policy 1, policy_version 710082 (0.0010) [2023-12-27 10:32:39,553][20381] Updated weights for policy 1, policy_version 710092 (0.0009) [2023-12-27 10:32:39,625][20381] Updated weights for policy 1, policy_version 710102 (0.0011) [2023-12-27 10:32:40,080][20347] Updated weights for policy 0, policy_version 710637 (0.0007) [2023-12-27 10:32:40,138][20347] Updated weights for policy 0, policy_version 710647 (0.0010) [2023-12-27 10:32:40,191][20347] Updated weights for policy 0, policy_version 710657 (0.0010) [2023-12-27 10:32:40,313][20381] Updated weights for policy 1, policy_version 710112 (0.0008) [2023-12-27 10:32:40,367][20381] Updated weights for policy 1, policy_version 710122 (0.0006) [2023-12-27 10:32:40,422][20381] Updated weights for policy 1, policy_version 710132 (0.0007) [2023-12-27 10:32:40,882][20347] Updated weights for policy 0, policy_version 710667 (0.0010) [2023-12-27 10:32:40,949][20347] Updated weights for policy 0, policy_version 710677 (0.0010) [2023-12-27 10:32:41,013][20347] Updated weights for policy 0, policy_version 710687 (0.0010) [2023-12-27 10:32:41,140][20381] Updated weights for policy 1, policy_version 710142 (0.0008) [2023-12-27 10:32:41,203][20381] Updated weights for policy 1, policy_version 710152 (0.0008) [2023-12-27 10:32:41,268][20381] Updated weights for policy 1, policy_version 710162 (0.0007) [2023-12-27 10:32:41,738][20347] Updated weights for policy 0, policy_version 710697 (0.0010) [2023-12-27 10:32:41,807][20347] Updated weights for policy 0, policy_version 710707 (0.0009) [2023-12-27 10:32:41,866][20347] Updated weights for policy 0, policy_version 710717 (0.0008) [2023-12-27 10:32:41,928][20347] Updated weights for policy 0, policy_version 710727 (0.0007) [2023-12-27 10:32:42,008][20381] Updated weights for policy 1, policy_version 710172 (0.0008) [2023-12-27 10:32:42,068][20381] Updated weights for policy 1, policy_version 710182 (0.0009) [2023-12-27 10:32:42,123][20381] Updated weights for policy 1, policy_version 710192 (0.0008) [2023-12-27 10:32:42,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20206.9, 300 sec: 20743.8). Total num frames: 363806720. Throughput: 0: 10261.5, 1: 10163.1. Samples: 363796392. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:42,363][19170] Avg episode reward: [(0, '990.230'), (1, '727.410')] [2023-12-27 10:32:42,622][20347] Updated weights for policy 0, policy_version 710737 (0.0006) [2023-12-27 10:32:42,681][20347] Updated weights for policy 0, policy_version 710747 (0.0007) [2023-12-27 10:32:42,741][20347] Updated weights for policy 0, policy_version 710757 (0.0007) [2023-12-27 10:32:42,845][20381] Updated weights for policy 1, policy_version 710202 (0.0007) [2023-12-27 10:32:42,908][20381] Updated weights for policy 1, policy_version 710212 (0.0007) [2023-12-27 10:32:42,963][20381] Updated weights for policy 1, policy_version 710222 (0.0008) [2023-12-27 10:32:43,023][20381] Updated weights for policy 1, policy_version 710232 (0.0008) [2023-12-27 10:32:43,403][20347] Updated weights for policy 0, policy_version 710767 (0.0007) [2023-12-27 10:32:43,455][20347] Updated weights for policy 0, policy_version 710777 (0.0007) [2023-12-27 10:32:43,509][20347] Updated weights for policy 0, policy_version 710787 (0.0008) [2023-12-27 10:32:43,702][20381] Updated weights for policy 1, policy_version 710242 (0.0008) [2023-12-27 10:32:43,751][20381] Updated weights for policy 1, policy_version 710252 (0.0008) [2023-12-27 10:32:43,808][20381] Updated weights for policy 1, policy_version 710263 (0.0010) [2023-12-27 10:32:44,166][20347] Updated weights for policy 0, policy_version 710797 (0.0006) [2023-12-27 10:32:44,218][20347] Updated weights for policy 0, policy_version 710807 (0.0005) [2023-12-27 10:32:44,282][20347] Updated weights for policy 0, policy_version 710817 (0.0008) [2023-12-27 10:32:44,413][20381] Updated weights for policy 1, policy_version 710273 (0.0009) [2023-12-27 10:32:44,465][20381] Updated weights for policy 1, policy_version 710283 (0.0007) [2023-12-27 10:32:44,526][20381] Updated weights for policy 1, policy_version 710293 (0.0009) [2023-12-27 10:32:44,910][20347] Updated weights for policy 0, policy_version 710827 (0.0008) [2023-12-27 10:32:44,975][20347] Updated weights for policy 0, policy_version 710837 (0.0008) [2023-12-27 10:32:45,036][20347] Updated weights for policy 0, policy_version 710847 (0.0007) [2023-12-27 10:32:45,167][20381] Updated weights for policy 1, policy_version 710303 (0.0010) [2023-12-27 10:32:45,226][20381] Updated weights for policy 1, policy_version 710313 (0.0007) [2023-12-27 10:32:45,293][20381] Updated weights for policy 1, policy_version 710323 (0.0011) [2023-12-27 10:32:45,682][20347] Updated weights for policy 0, policy_version 710857 (0.0008) [2023-12-27 10:32:45,727][20347] Updated weights for policy 0, policy_version 710867 (0.0010) [2023-12-27 10:32:45,774][20347] Updated weights for policy 0, policy_version 710877 (0.0008) [2023-12-27 10:32:45,834][20347] Updated weights for policy 0, policy_version 710887 (0.0005) [2023-12-27 10:32:45,908][20381] Updated weights for policy 1, policy_version 710333 (0.0008) [2023-12-27 10:32:45,966][20381] Updated weights for policy 1, policy_version 710343 (0.0005) [2023-12-27 10:32:46,024][20381] Updated weights for policy 1, policy_version 710353 (0.0005) [2023-12-27 10:32:46,499][20347] Updated weights for policy 0, policy_version 710897 (0.0010) [2023-12-27 10:32:46,555][20347] Updated weights for policy 0, policy_version 710907 (0.0010) [2023-12-27 10:32:46,607][20347] Updated weights for policy 0, policy_version 710917 (0.0010) [2023-12-27 10:32:46,661][20381] Updated weights for policy 1, policy_version 710363 (0.0007) [2023-12-27 10:32:46,718][20381] Updated weights for policy 1, policy_version 710373 (0.0011) [2023-12-27 10:32:46,774][20381] Updated weights for policy 1, policy_version 710383 (0.0011) [2023-12-27 10:32:47,299][20347] Updated weights for policy 0, policy_version 710927 (0.0007) [2023-12-27 10:32:47,349][20347] Updated weights for policy 0, policy_version 710937 (0.0005) [2023-12-27 10:32:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20716.1). Total num frames: 363905024. Throughput: 0: 10276.7, 1: 10191.4. Samples: 363920364. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.890')] [2023-12-27 10:32:47,398][20381] Updated weights for policy 1, policy_version 710393 (0.0008) [2023-12-27 10:32:47,408][20347] Updated weights for policy 0, policy_version 710947 (0.0005) [2023-12-27 10:32:47,447][20381] Updated weights for policy 1, policy_version 710403 (0.0010) [2023-12-27 10:32:47,506][20381] Updated weights for policy 1, policy_version 710413 (0.0011) [2023-12-27 10:32:47,567][20381] Updated weights for policy 1, policy_version 710423 (0.0011) [2023-12-27 10:32:48,016][20347] Updated weights for policy 0, policy_version 710957 (0.0008) [2023-12-27 10:32:48,068][20347] Updated weights for policy 0, policy_version 710967 (0.0010) [2023-12-27 10:32:48,124][20347] Updated weights for policy 0, policy_version 710977 (0.0010) [2023-12-27 10:32:48,266][20381] Updated weights for policy 1, policy_version 710433 (0.0008) [2023-12-27 10:32:48,331][20381] Updated weights for policy 1, policy_version 710443 (0.0008) [2023-12-27 10:32:48,395][20381] Updated weights for policy 1, policy_version 710453 (0.0010) [2023-12-27 10:32:48,776][20347] Updated weights for policy 0, policy_version 710987 (0.0009) [2023-12-27 10:32:48,835][20347] Updated weights for policy 0, policy_version 710997 (0.0007) [2023-12-27 10:32:48,893][20347] Updated weights for policy 0, policy_version 711007 (0.0007) [2023-12-27 10:32:49,057][20381] Updated weights for policy 1, policy_version 710463 (0.0007) [2023-12-27 10:32:49,106][20381] Updated weights for policy 1, policy_version 710473 (0.0010) [2023-12-27 10:32:49,154][20381] Updated weights for policy 1, policy_version 710483 (0.0009) [2023-12-27 10:32:49,584][20347] Updated weights for policy 0, policy_version 711017 (0.0007) [2023-12-27 10:32:49,638][20347] Updated weights for policy 0, policy_version 711027 (0.0009) [2023-12-27 10:32:49,701][20347] Updated weights for policy 0, policy_version 711037 (0.0010) [2023-12-27 10:32:49,761][20347] Updated weights for policy 0, policy_version 711047 (0.0007) [2023-12-27 10:32:49,792][20381] Updated weights for policy 1, policy_version 710493 (0.0008) [2023-12-27 10:32:49,857][20381] Updated weights for policy 1, policy_version 710503 (0.0011) [2023-12-27 10:32:49,919][20381] Updated weights for policy 1, policy_version 710513 (0.0008) [2023-12-27 10:32:50,514][20347] Updated weights for policy 0, policy_version 711057 (0.0008) [2023-12-27 10:32:50,566][20347] Updated weights for policy 0, policy_version 711067 (0.0007) [2023-12-27 10:32:50,625][20381] Updated weights for policy 1, policy_version 710523 (0.0011) [2023-12-27 10:32:50,631][20347] Updated weights for policy 0, policy_version 711077 (0.0006) [2023-12-27 10:32:50,677][20381] Updated weights for policy 1, policy_version 710533 (0.0011) [2023-12-27 10:32:50,739][20381] Updated weights for policy 1, policy_version 710543 (0.0011) [2023-12-27 10:32:51,328][20347] Updated weights for policy 0, policy_version 711087 (0.0006) [2023-12-27 10:32:51,392][20347] Updated weights for policy 0, policy_version 711097 (0.0008) [2023-12-27 10:32:51,449][20347] Updated weights for policy 0, policy_version 711107 (0.0007) [2023-12-27 10:32:51,492][20381] Updated weights for policy 1, policy_version 710553 (0.0010) [2023-12-27 10:32:51,551][20381] Updated weights for policy 1, policy_version 710563 (0.0008) [2023-12-27 10:32:51,605][20381] Updated weights for policy 1, policy_version 710573 (0.0008) [2023-12-27 10:32:51,667][20381] Updated weights for policy 1, policy_version 710583 (0.0007) [2023-12-27 10:32:52,176][20347] Updated weights for policy 0, policy_version 711117 (0.0007) [2023-12-27 10:32:52,224][20347] Updated weights for policy 0, policy_version 711127 (0.0007) [2023-12-27 10:32:52,279][20347] Updated weights for policy 0, policy_version 711137 (0.0007) [2023-12-27 10:32:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20743.8). Total num frames: 364011520. Throughput: 0: 10262.6, 1: 10213.2. Samples: 363981844. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.410')] [2023-12-27 10:32:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000711144_182075392.pth... [2023-12-27 10:32:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000709928_181764096.pth [2023-12-27 10:32:52,404][20381] Updated weights for policy 1, policy_version 710593 (0.0010) [2023-12-27 10:32:52,467][20381] Updated weights for policy 1, policy_version 710603 (0.0011) [2023-12-27 10:32:52,526][20381] Updated weights for policy 1, policy_version 710613 (0.0011) [2023-12-27 10:32:52,543][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000710616_181944320.pth... [2023-12-27 10:32:52,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000709400_181633024.pth [2023-12-27 10:32:53,027][20347] Updated weights for policy 0, policy_version 711147 (0.0008) [2023-12-27 10:32:53,076][20347] Updated weights for policy 0, policy_version 711157 (0.0007) [2023-12-27 10:32:53,120][20347] Updated weights for policy 0, policy_version 711167 (0.0007) [2023-12-27 10:32:53,198][20381] Updated weights for policy 1, policy_version 710623 (0.0010) [2023-12-27 10:32:53,259][20381] Updated weights for policy 1, policy_version 710633 (0.0010) [2023-12-27 10:32:53,314][20381] Updated weights for policy 1, policy_version 710643 (0.0011) [2023-12-27 10:32:53,822][20347] Updated weights for policy 0, policy_version 711177 (0.0007) [2023-12-27 10:32:53,887][20347] Updated weights for policy 0, policy_version 711187 (0.0008) [2023-12-27 10:32:53,941][20347] Updated weights for policy 0, policy_version 711197 (0.0010) [2023-12-27 10:32:53,974][20381] Updated weights for policy 1, policy_version 710653 (0.0009) [2023-12-27 10:32:53,999][20347] Updated weights for policy 0, policy_version 711207 (0.0010) [2023-12-27 10:32:54,041][20381] Updated weights for policy 1, policy_version 710663 (0.0007) [2023-12-27 10:32:54,095][20381] Updated weights for policy 1, policy_version 710673 (0.0005) [2023-12-27 10:32:54,620][20347] Updated weights for policy 0, policy_version 711217 (0.0006) [2023-12-27 10:32:54,674][20347] Updated weights for policy 0, policy_version 711227 (0.0007) [2023-12-27 10:32:54,678][20381] Updated weights for policy 1, policy_version 710683 (0.0007) [2023-12-27 10:32:54,727][20347] Updated weights for policy 0, policy_version 711237 (0.0008) [2023-12-27 10:32:54,731][20381] Updated weights for policy 1, policy_version 710693 (0.0011) [2023-12-27 10:32:54,780][20381] Updated weights for policy 1, policy_version 710703 (0.0011) [2023-12-27 10:32:55,341][20347] Updated weights for policy 0, policy_version 711247 (0.0010) [2023-12-27 10:32:55,402][20347] Updated weights for policy 0, policy_version 711257 (0.0010) [2023-12-27 10:32:55,406][20381] Updated weights for policy 1, policy_version 710713 (0.0010) [2023-12-27 10:32:55,460][20347] Updated weights for policy 0, policy_version 711267 (0.0010) [2023-12-27 10:32:55,464][20381] Updated weights for policy 1, policy_version 710723 (0.0011) [2023-12-27 10:32:55,523][20381] Updated weights for policy 1, policy_version 710733 (0.0011) [2023-12-27 10:32:55,578][20381] Updated weights for policy 1, policy_version 710743 (0.0010) [2023-12-27 10:32:56,021][20347] Updated weights for policy 0, policy_version 711277 (0.0010) [2023-12-27 10:32:56,071][20347] Updated weights for policy 0, policy_version 711287 (0.0010) [2023-12-27 10:32:56,118][20347] Updated weights for policy 0, policy_version 711297 (0.0010) [2023-12-27 10:32:56,275][20381] Updated weights for policy 1, policy_version 710753 (0.0006) [2023-12-27 10:32:56,333][20381] Updated weights for policy 1, policy_version 710763 (0.0005) [2023-12-27 10:32:56,391][20381] Updated weights for policy 1, policy_version 710773 (0.0005) [2023-12-27 10:32:56,707][20347] Updated weights for policy 0, policy_version 711307 (0.0008) [2023-12-27 10:32:56,751][20347] Updated weights for policy 0, policy_version 711317 (0.0005) [2023-12-27 10:32:56,811][20347] Updated weights for policy 0, policy_version 711327 (0.0007) [2023-12-27 10:32:56,956][20381] Updated weights for policy 1, policy_version 710783 (0.0005) [2023-12-27 10:32:57,011][20381] Updated weights for policy 1, policy_version 710793 (0.0010) [2023-12-27 10:32:57,066][20381] Updated weights for policy 1, policy_version 710803 (0.0011) [2023-12-27 10:32:57,327][20347] Updated weights for policy 0, policy_version 711337 (0.0006) [2023-12-27 10:32:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20743.8). Total num frames: 364118016. Throughput: 0: 10272.4, 1: 10237.7. Samples: 364105816. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:32:57,363][19170] Avg episode reward: [(0, '980.670'), (1, '776.090')] [2023-12-27 10:32:57,381][20347] Updated weights for policy 0, policy_version 711347 (0.0010) [2023-12-27 10:32:57,429][20347] Updated weights for policy 0, policy_version 711357 (0.0010) [2023-12-27 10:32:57,474][20347] Updated weights for policy 0, policy_version 711367 (0.0009) [2023-12-27 10:32:57,750][20381] Updated weights for policy 1, policy_version 710813 (0.0010) [2023-12-27 10:32:57,807][20381] Updated weights for policy 1, policy_version 710823 (0.0010) [2023-12-27 10:32:57,864][20381] Updated weights for policy 1, policy_version 710833 (0.0008) [2023-12-27 10:32:58,117][20347] Updated weights for policy 0, policy_version 711377 (0.0006) [2023-12-27 10:32:58,166][20347] Updated weights for policy 0, policy_version 711387 (0.0005) [2023-12-27 10:32:58,228][20347] Updated weights for policy 0, policy_version 711397 (0.0007) [2023-12-27 10:32:58,506][20381] Updated weights for policy 1, policy_version 710843 (0.0005) [2023-12-27 10:32:58,573][20381] Updated weights for policy 1, policy_version 710853 (0.0006) [2023-12-27 10:32:58,635][20381] Updated weights for policy 1, policy_version 710863 (0.0008) [2023-12-27 10:32:59,005][20347] Updated weights for policy 0, policy_version 711407 (0.0008) [2023-12-27 10:32:59,048][20347] Updated weights for policy 0, policy_version 711417 (0.0008) [2023-12-27 10:32:59,092][20347] Updated weights for policy 0, policy_version 711427 (0.0007) [2023-12-27 10:32:59,284][20381] Updated weights for policy 1, policy_version 710873 (0.0008) [2023-12-27 10:32:59,344][20381] Updated weights for policy 1, policy_version 710883 (0.0008) [2023-12-27 10:32:59,405][20381] Updated weights for policy 1, policy_version 710893 (0.0007) [2023-12-27 10:32:59,464][20381] Updated weights for policy 1, policy_version 710903 (0.0008) [2023-12-27 10:32:59,855][20347] Updated weights for policy 0, policy_version 711437 (0.0007) [2023-12-27 10:32:59,911][20347] Updated weights for policy 0, policy_version 711447 (0.0007) [2023-12-27 10:32:59,973][20347] Updated weights for policy 0, policy_version 711457 (0.0007) [2023-12-27 10:33:00,181][20381] Updated weights for policy 1, policy_version 710913 (0.0008) [2023-12-27 10:33:00,238][20381] Updated weights for policy 1, policy_version 710923 (0.0009) [2023-12-27 10:33:00,290][20381] Updated weights for policy 1, policy_version 710933 (0.0008) [2023-12-27 10:33:00,557][20347] Updated weights for policy 0, policy_version 711467 (0.0007) [2023-12-27 10:33:00,621][20347] Updated weights for policy 0, policy_version 711477 (0.0006) [2023-12-27 10:33:00,669][20347] Updated weights for policy 0, policy_version 711487 (0.0006) [2023-12-27 10:33:00,927][20381] Updated weights for policy 1, policy_version 710943 (0.0007) [2023-12-27 10:33:00,974][20381] Updated weights for policy 1, policy_version 710953 (0.0008) [2023-12-27 10:33:01,018][20381] Updated weights for policy 1, policy_version 710963 (0.0007) [2023-12-27 10:33:01,281][20347] Updated weights for policy 0, policy_version 711497 (0.0006) [2023-12-27 10:33:01,331][20347] Updated weights for policy 0, policy_version 711507 (0.0008) [2023-12-27 10:33:01,399][20347] Updated weights for policy 0, policy_version 711517 (0.0008) [2023-12-27 10:33:01,455][20347] Updated weights for policy 0, policy_version 711527 (0.0007) [2023-12-27 10:33:01,747][20381] Updated weights for policy 1, policy_version 710973 (0.0010) [2023-12-27 10:33:01,809][20381] Updated weights for policy 1, policy_version 710983 (0.0011) [2023-12-27 10:33:01,867][20381] Updated weights for policy 1, policy_version 710993 (0.0009) [2023-12-27 10:33:02,228][20347] Updated weights for policy 0, policy_version 711537 (0.0008) [2023-12-27 10:33:02,287][20347] Updated weights for policy 0, policy_version 711547 (0.0007) [2023-12-27 10:33:02,350][20347] Updated weights for policy 0, policy_version 711557 (0.0007) [2023-12-27 10:33:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 364216320. Throughput: 0: 10286.0, 1: 10234.6. Samples: 364232800. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:33:02,363][19170] Avg episode reward: [(0, '980.670'), (1, '718.380')] [2023-12-27 10:33:02,554][20381] Updated weights for policy 1, policy_version 711003 (0.0008) [2023-12-27 10:33:02,614][20381] Updated weights for policy 1, policy_version 711013 (0.0011) [2023-12-27 10:33:02,676][20381] Updated weights for policy 1, policy_version 711023 (0.0011) [2023-12-27 10:33:03,019][20347] Updated weights for policy 0, policy_version 711567 (0.0008) [2023-12-27 10:33:03,072][20347] Updated weights for policy 0, policy_version 711577 (0.0007) [2023-12-27 10:33:03,129][20347] Updated weights for policy 0, policy_version 711587 (0.0007) [2023-12-27 10:33:03,262][20381] Updated weights for policy 1, policy_version 711033 (0.0010) [2023-12-27 10:33:03,323][20381] Updated weights for policy 1, policy_version 711043 (0.0005) [2023-12-27 10:33:03,377][20381] Updated weights for policy 1, policy_version 711053 (0.0006) [2023-12-27 10:33:03,434][20381] Updated weights for policy 1, policy_version 711063 (0.0010) [2023-12-27 10:33:03,751][20347] Updated weights for policy 0, policy_version 711597 (0.0007) [2023-12-27 10:33:03,806][20347] Updated weights for policy 0, policy_version 711607 (0.0005) [2023-12-27 10:33:03,877][20347] Updated weights for policy 0, policy_version 711617 (0.0007) [2023-12-27 10:33:04,030][20381] Updated weights for policy 1, policy_version 711073 (0.0008) [2023-12-27 10:33:04,078][20381] Updated weights for policy 1, policy_version 711083 (0.0007) [2023-12-27 10:33:04,136][20381] Updated weights for policy 1, policy_version 711093 (0.0008) [2023-12-27 10:33:04,458][20347] Updated weights for policy 0, policy_version 711627 (0.0007) [2023-12-27 10:33:04,512][20347] Updated weights for policy 0, policy_version 711637 (0.0007) [2023-12-27 10:33:04,559][20347] Updated weights for policy 0, policy_version 711647 (0.0007) [2023-12-27 10:33:04,880][20381] Updated weights for policy 1, policy_version 711103 (0.0006) [2023-12-27 10:33:04,933][20381] Updated weights for policy 1, policy_version 711113 (0.0005) [2023-12-27 10:33:04,987][20381] Updated weights for policy 1, policy_version 711123 (0.0006) [2023-12-27 10:33:05,259][20347] Updated weights for policy 0, policy_version 711657 (0.0005) [2023-12-27 10:33:05,316][20347] Updated weights for policy 0, policy_version 711667 (0.0008) [2023-12-27 10:33:05,368][20347] Updated weights for policy 0, policy_version 711677 (0.0007) [2023-12-27 10:33:05,421][20347] Updated weights for policy 0, policy_version 711687 (0.0007) [2023-12-27 10:33:05,591][20381] Updated weights for policy 1, policy_version 711133 (0.0007) [2023-12-27 10:33:05,649][20381] Updated weights for policy 1, policy_version 711143 (0.0008) [2023-12-27 10:33:05,701][20381] Updated weights for policy 1, policy_version 711153 (0.0007) [2023-12-27 10:33:06,103][20347] Updated weights for policy 0, policy_version 711697 (0.0007) [2023-12-27 10:33:06,168][20347] Updated weights for policy 0, policy_version 711707 (0.0008) [2023-12-27 10:33:06,228][20347] Updated weights for policy 0, policy_version 711717 (0.0008) [2023-12-27 10:33:06,375][20381] Updated weights for policy 1, policy_version 711163 (0.0008) [2023-12-27 10:33:06,436][20381] Updated weights for policy 1, policy_version 711173 (0.0007) [2023-12-27 10:33:06,496][20381] Updated weights for policy 1, policy_version 711183 (0.0008) [2023-12-27 10:33:06,866][20347] Updated weights for policy 0, policy_version 711727 (0.0008) [2023-12-27 10:33:06,922][20347] Updated weights for policy 0, policy_version 711737 (0.0008) [2023-12-27 10:33:06,979][20347] Updated weights for policy 0, policy_version 711747 (0.0008) [2023-12-27 10:33:07,191][20381] Updated weights for policy 1, policy_version 711193 (0.0008) [2023-12-27 10:33:07,255][20381] Updated weights for policy 1, policy_version 711203 (0.0008) [2023-12-27 10:33:07,320][20381] Updated weights for policy 1, policy_version 711213 (0.0008) [2023-12-27 10:33:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20743.8). Total num frames: 364322816. Throughput: 0: 10325.6, 1: 10258.7. Samples: 364296904. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:33:07,363][19170] Avg episode reward: [(0, '951.700'), (1, '795.550')] [2023-12-27 10:33:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000711752_182231040.pth... [2023-12-27 10:33:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000710536_181919744.pth [2023-12-27 10:33:07,380][20381] Updated weights for policy 1, policy_version 711223 (0.0007) [2023-12-27 10:33:07,386][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000711224_182099968.pth... [2023-12-27 10:33:07,390][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000710008_181788672.pth [2023-12-27 10:33:07,645][20347] Updated weights for policy 0, policy_version 711757 (0.0007) [2023-12-27 10:33:07,693][20347] Updated weights for policy 0, policy_version 711767 (0.0007) [2023-12-27 10:33:07,734][20291] KL-divergence is very high: 125.2971 [2023-12-27 10:33:07,741][20291] KL-divergence is very high: 114.1085 [2023-12-27 10:33:07,747][20347] Updated weights for policy 0, policy_version 711777 (0.0007) [2023-12-27 10:33:08,044][20381] Updated weights for policy 1, policy_version 711233 (0.0008) [2023-12-27 10:33:08,098][20381] Updated weights for policy 1, policy_version 711243 (0.0006) [2023-12-27 10:33:08,154][20381] Updated weights for policy 1, policy_version 711253 (0.0008) [2023-12-27 10:33:08,455][20347] Updated weights for policy 0, policy_version 711787 (0.0007) [2023-12-27 10:33:08,511][20347] Updated weights for policy 0, policy_version 711797 (0.0005) [2023-12-27 10:33:08,570][20347] Updated weights for policy 0, policy_version 711807 (0.0006) [2023-12-27 10:33:08,829][20381] Updated weights for policy 1, policy_version 711263 (0.0006) [2023-12-27 10:33:08,888][20381] Updated weights for policy 1, policy_version 711273 (0.0006) [2023-12-27 10:33:08,948][20381] Updated weights for policy 1, policy_version 711283 (0.0006) [2023-12-27 10:33:09,225][20347] Updated weights for policy 0, policy_version 711817 (0.0007) [2023-12-27 10:33:09,289][20347] Updated weights for policy 0, policy_version 711827 (0.0007) [2023-12-27 10:33:09,354][20347] Updated weights for policy 0, policy_version 711837 (0.0008) [2023-12-27 10:33:09,428][20347] Updated weights for policy 0, policy_version 711847 (0.0008) [2023-12-27 10:33:09,600][20381] Updated weights for policy 1, policy_version 711293 (0.0009) [2023-12-27 10:33:09,658][20381] Updated weights for policy 1, policy_version 711303 (0.0008) [2023-12-27 10:33:09,719][20381] Updated weights for policy 1, policy_version 711313 (0.0007) [2023-12-27 10:33:10,087][20347] Updated weights for policy 0, policy_version 711857 (0.0008) [2023-12-27 10:33:10,147][20347] Updated weights for policy 0, policy_version 711867 (0.0008) [2023-12-27 10:33:10,208][20347] Updated weights for policy 0, policy_version 711877 (0.0008) [2023-12-27 10:33:10,478][20381] Updated weights for policy 1, policy_version 711323 (0.0008) [2023-12-27 10:33:10,538][20381] Updated weights for policy 1, policy_version 711333 (0.0008) [2023-12-27 10:33:10,593][20381] Updated weights for policy 1, policy_version 711343 (0.0008) [2023-12-27 10:33:10,927][20347] Updated weights for policy 0, policy_version 711887 (0.0007) [2023-12-27 10:33:10,986][20347] Updated weights for policy 0, policy_version 711897 (0.0008) [2023-12-27 10:33:11,045][20347] Updated weights for policy 0, policy_version 711907 (0.0008) [2023-12-27 10:33:11,292][20381] Updated weights for policy 1, policy_version 711353 (0.0008) [2023-12-27 10:33:11,355][20381] Updated weights for policy 1, policy_version 711363 (0.0008) [2023-12-27 10:33:11,416][20381] Updated weights for policy 1, policy_version 711373 (0.0008) [2023-12-27 10:33:11,468][20381] Updated weights for policy 1, policy_version 711383 (0.0008) [2023-12-27 10:33:11,824][20347] Updated weights for policy 0, policy_version 711917 (0.0007) [2023-12-27 10:33:11,887][20347] Updated weights for policy 0, policy_version 711927 (0.0008) [2023-12-27 10:33:11,942][20347] Updated weights for policy 0, policy_version 711937 (0.0008) [2023-12-27 10:33:12,193][20381] Updated weights for policy 1, policy_version 711393 (0.0008) [2023-12-27 10:33:12,250][20381] Updated weights for policy 1, policy_version 711403 (0.0007) [2023-12-27 10:33:12,311][20381] Updated weights for policy 1, policy_version 711413 (0.0008) [2023-12-27 10:33:12,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.2, 300 sec: 20716.0). Total num frames: 364429312. Throughput: 0: 10296.2, 1: 10265.3. Samples: 364417800. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:33:12,363][19170] Avg episode reward: [(0, '903.150'), (1, '756.260')] [2023-12-27 10:33:12,654][20347] Updated weights for policy 0, policy_version 711947 (0.0008) [2023-12-27 10:33:12,714][20347] Updated weights for policy 0, policy_version 711957 (0.0008) [2023-12-27 10:33:12,775][20347] Updated weights for policy 0, policy_version 711967 (0.0007) [2023-12-27 10:33:13,007][20381] Updated weights for policy 1, policy_version 711423 (0.0007) [2023-12-27 10:33:13,069][20381] Updated weights for policy 1, policy_version 711433 (0.0008) [2023-12-27 10:33:13,125][20381] Updated weights for policy 1, policy_version 711443 (0.0006) [2023-12-27 10:33:13,478][20347] Updated weights for policy 0, policy_version 711977 (0.0009) [2023-12-27 10:33:13,546][20347] Updated weights for policy 0, policy_version 711987 (0.0008) [2023-12-27 10:33:13,599][20347] Updated weights for policy 0, policy_version 711997 (0.0007) [2023-12-27 10:33:13,657][20347] Updated weights for policy 0, policy_version 712007 (0.0007) [2023-12-27 10:33:13,775][20381] Updated weights for policy 1, policy_version 711453 (0.0007) [2023-12-27 10:33:13,831][20381] Updated weights for policy 1, policy_version 711463 (0.0007) [2023-12-27 10:33:13,889][20381] Updated weights for policy 1, policy_version 711473 (0.0007) [2023-12-27 10:33:14,344][20347] Updated weights for policy 0, policy_version 712017 (0.0006) [2023-12-27 10:33:14,394][20347] Updated weights for policy 0, policy_version 712027 (0.0007) [2023-12-27 10:33:14,449][20347] Updated weights for policy 0, policy_version 712037 (0.0007) [2023-12-27 10:33:14,593][20381] Updated weights for policy 1, policy_version 711483 (0.0008) [2023-12-27 10:33:14,648][20381] Updated weights for policy 1, policy_version 711493 (0.0007) [2023-12-27 10:33:14,707][20381] Updated weights for policy 1, policy_version 711503 (0.0008) [2023-12-27 10:33:15,116][20347] Updated weights for policy 0, policy_version 712047 (0.0007) [2023-12-27 10:33:15,181][20347] Updated weights for policy 0, policy_version 712057 (0.0006) [2023-12-27 10:33:15,241][20347] Updated weights for policy 0, policy_version 712067 (0.0007) [2023-12-27 10:33:15,432][20381] Updated weights for policy 1, policy_version 711513 (0.0008) [2023-12-27 10:33:15,490][20381] Updated weights for policy 1, policy_version 711523 (0.0008) [2023-12-27 10:33:15,554][20381] Updated weights for policy 1, policy_version 711533 (0.0008) [2023-12-27 10:33:15,606][20381] Updated weights for policy 1, policy_version 711543 (0.0008) [2023-12-27 10:33:15,813][20347] Updated weights for policy 0, policy_version 712077 (0.0007) [2023-12-27 10:33:15,876][20347] Updated weights for policy 0, policy_version 712087 (0.0005) [2023-12-27 10:33:15,935][20347] Updated weights for policy 0, policy_version 712097 (0.0006) [2023-12-27 10:33:16,362][20381] Updated weights for policy 1, policy_version 711553 (0.0008) [2023-12-27 10:33:16,417][20381] Updated weights for policy 1, policy_version 711563 (0.0008) [2023-12-27 10:33:16,474][20381] Updated weights for policy 1, policy_version 711573 (0.0008) [2023-12-27 10:33:16,565][20347] Updated weights for policy 0, policy_version 712107 (0.0008) [2023-12-27 10:33:16,624][20347] Updated weights for policy 0, policy_version 712117 (0.0006) [2023-12-27 10:33:16,681][20347] Updated weights for policy 0, policy_version 712127 (0.0006) [2023-12-27 10:33:17,211][20381] Updated weights for policy 1, policy_version 711583 (0.0008) [2023-12-27 10:33:17,269][20381] Updated weights for policy 1, policy_version 711593 (0.0007) [2023-12-27 10:33:17,279][20347] Updated weights for policy 0, policy_version 712137 (0.0008) [2023-12-27 10:33:17,325][20381] Updated weights for policy 1, policy_version 711603 (0.0007) [2023-12-27 10:33:17,334][20347] Updated weights for policy 0, policy_version 712147 (0.0006) [2023-12-27 10:33:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20716.0). Total num frames: 364527616. Throughput: 0: 10314.1, 1: 10190.9. Samples: 364537772. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:33:17,363][19170] Avg episode reward: [(0, '893.300'), (1, '717.600')] [2023-12-27 10:33:17,387][20347] Updated weights for policy 0, policy_version 712157 (0.0007) [2023-12-27 10:33:17,449][20347] Updated weights for policy 0, policy_version 712167 (0.0005) [2023-12-27 10:33:18,031][20381] Updated weights for policy 1, policy_version 711613 (0.0006) [2023-12-27 10:33:18,090][20381] Updated weights for policy 1, policy_version 711623 (0.0006) [2023-12-27 10:33:18,099][20347] Updated weights for policy 0, policy_version 712177 (0.0006) [2023-12-27 10:33:18,143][20381] Updated weights for policy 1, policy_version 711633 (0.0007) [2023-12-27 10:33:18,159][20347] Updated weights for policy 0, policy_version 712187 (0.0007) [2023-12-27 10:33:18,217][20347] Updated weights for policy 0, policy_version 712197 (0.0007) [2023-12-27 10:33:18,793][20381] Updated weights for policy 1, policy_version 711643 (0.0007) [2023-12-27 10:33:18,853][20381] Updated weights for policy 1, policy_version 711653 (0.0008) [2023-12-27 10:33:18,915][20381] Updated weights for policy 1, policy_version 711663 (0.0007) [2023-12-27 10:33:18,935][20347] Updated weights for policy 0, policy_version 712207 (0.0008) [2023-12-27 10:33:18,985][20347] Updated weights for policy 0, policy_version 712217 (0.0006) [2023-12-27 10:33:19,041][20347] Updated weights for policy 0, policy_version 712227 (0.0007) [2023-12-27 10:33:19,578][20381] Updated weights for policy 1, policy_version 711673 (0.0006) [2023-12-27 10:33:19,641][20381] Updated weights for policy 1, policy_version 711683 (0.0006) [2023-12-27 10:33:19,701][20381] Updated weights for policy 1, policy_version 711693 (0.0008) [2023-12-27 10:33:19,763][20381] Updated weights for policy 1, policy_version 711703 (0.0007) [2023-12-27 10:33:19,777][20347] Updated weights for policy 0, policy_version 712237 (0.0009) [2023-12-27 10:33:19,842][20347] Updated weights for policy 0, policy_version 712247 (0.0010) [2023-12-27 10:33:19,909][20347] Updated weights for policy 0, policy_version 712257 (0.0009) [2023-12-27 10:33:20,412][20381] Updated weights for policy 1, policy_version 711713 (0.0010) [2023-12-27 10:33:20,473][20381] Updated weights for policy 1, policy_version 711723 (0.0011) [2023-12-27 10:33:20,530][20381] Updated weights for policy 1, policy_version 711733 (0.0011) [2023-12-27 10:33:20,627][20347] Updated weights for policy 0, policy_version 712267 (0.0008) [2023-12-27 10:33:20,688][20347] Updated weights for policy 0, policy_version 712277 (0.0006) [2023-12-27 10:33:20,755][20347] Updated weights for policy 0, policy_version 712287 (0.0007) [2023-12-27 10:33:21,277][20381] Updated weights for policy 1, policy_version 711743 (0.0011) [2023-12-27 10:33:21,334][20381] Updated weights for policy 1, policy_version 711753 (0.0011) [2023-12-27 10:33:21,398][20381] Updated weights for policy 1, policy_version 711763 (0.0010) [2023-12-27 10:33:21,529][20347] Updated weights for policy 0, policy_version 712297 (0.0008) [2023-12-27 10:33:21,585][20347] Updated weights for policy 0, policy_version 712307 (0.0007) [2023-12-27 10:33:21,645][20347] Updated weights for policy 0, policy_version 712317 (0.0009) [2023-12-27 10:33:21,701][20347] Updated weights for policy 0, policy_version 712327 (0.0011) [2023-12-27 10:33:22,131][20381] Updated weights for policy 1, policy_version 711773 (0.0011) [2023-12-27 10:33:22,192][20381] Updated weights for policy 1, policy_version 711783 (0.0010) [2023-12-27 10:33:22,253][20381] Updated weights for policy 1, policy_version 711793 (0.0011) [2023-12-27 10:33:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 364625920. Throughput: 0: 10272.6, 1: 10166.4. Samples: 364597356. Policy #0 lag: (min: 31.0, avg: 45.1, max: 63.0) [2023-12-27 10:33:22,363][19170] Avg episode reward: [(0, '941.650'), (1, '786.200')] [2023-12-27 10:33:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000711800_182247424.pth... [2023-12-27 10:33:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000712328_182378496.pth... [2023-12-27 10:33:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000710616_181944320.pth [2023-12-27 10:33:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000711144_182075392.pth [2023-12-27 10:33:22,454][20347] Updated weights for policy 0, policy_version 712337 (0.0011) [2023-12-27 10:33:22,518][20347] Updated weights for policy 0, policy_version 712347 (0.0009) [2023-12-27 10:33:22,583][20347] Updated weights for policy 0, policy_version 712357 (0.0010) [2023-12-27 10:33:22,939][20381] Updated weights for policy 1, policy_version 711803 (0.0010) [2023-12-27 10:33:23,008][20381] Updated weights for policy 1, policy_version 711813 (0.0008) [2023-12-27 10:33:23,071][20381] Updated weights for policy 1, policy_version 711823 (0.0008) [2023-12-27 10:33:23,212][20347] Updated weights for policy 0, policy_version 712367 (0.0011) [2023-12-27 10:33:23,273][20347] Updated weights for policy 0, policy_version 712377 (0.0010) [2023-12-27 10:33:23,330][20347] Updated weights for policy 0, policy_version 712387 (0.0010) [2023-12-27 10:33:23,746][20381] Updated weights for policy 1, policy_version 711833 (0.0008) [2023-12-27 10:33:23,808][20381] Updated weights for policy 1, policy_version 711843 (0.0008) [2023-12-27 10:33:23,865][20381] Updated weights for policy 1, policy_version 711853 (0.0010) [2023-12-27 10:33:23,921][20381] Updated weights for policy 1, policy_version 711863 (0.0009) [2023-12-27 10:33:23,991][20347] Updated weights for policy 0, policy_version 712397 (0.0010) [2023-12-27 10:33:24,048][20347] Updated weights for policy 0, policy_version 712407 (0.0010) [2023-12-27 10:33:24,109][20347] Updated weights for policy 0, policy_version 712417 (0.0010) [2023-12-27 10:33:24,515][20381] Updated weights for policy 1, policy_version 711873 (0.0010) [2023-12-27 10:33:24,574][20381] Updated weights for policy 1, policy_version 711883 (0.0010) [2023-12-27 10:33:24,626][20381] Updated weights for policy 1, policy_version 711893 (0.0011) [2023-12-27 10:33:24,703][20347] Updated weights for policy 0, policy_version 712427 (0.0010) [2023-12-27 10:33:24,760][20347] Updated weights for policy 0, policy_version 712437 (0.0005) [2023-12-27 10:33:24,814][20347] Updated weights for policy 0, policy_version 712447 (0.0005) [2023-12-27 10:33:25,298][20381] Updated weights for policy 1, policy_version 711903 (0.0009) [2023-12-27 10:33:25,349][20381] Updated weights for policy 1, policy_version 711913 (0.0007) [2023-12-27 10:33:25,406][20347] Updated weights for policy 0, policy_version 712457 (0.0006) [2023-12-27 10:33:25,406][20381] Updated weights for policy 1, policy_version 711923 (0.0008) [2023-12-27 10:33:25,459][20347] Updated weights for policy 0, policy_version 712467 (0.0010) [2023-12-27 10:33:25,514][20347] Updated weights for policy 0, policy_version 712477 (0.0010) [2023-12-27 10:33:25,564][20347] Updated weights for policy 0, policy_version 712487 (0.0010) [2023-12-27 10:33:26,050][20381] Updated weights for policy 1, policy_version 711933 (0.0008) [2023-12-27 10:33:26,111][20381] Updated weights for policy 1, policy_version 711943 (0.0008) [2023-12-27 10:33:26,176][20381] Updated weights for policy 1, policy_version 711953 (0.0008) [2023-12-27 10:33:26,219][20347] Updated weights for policy 0, policy_version 712497 (0.0010) [2023-12-27 10:33:26,264][20347] Updated weights for policy 0, policy_version 712507 (0.0010) [2023-12-27 10:33:26,312][20347] Updated weights for policy 0, policy_version 712517 (0.0010) [2023-12-27 10:33:26,818][20381] Updated weights for policy 1, policy_version 711963 (0.0008) [2023-12-27 10:33:26,881][20381] Updated weights for policy 1, policy_version 711974 (0.0008) [2023-12-27 10:33:26,963][20381] Updated weights for policy 1, policy_version 711984 (0.0007) [2023-12-27 10:33:27,004][20347] Updated weights for policy 0, policy_version 712527 (0.0010) [2023-12-27 10:33:27,064][20347] Updated weights for policy 0, policy_version 712537 (0.0010) [2023-12-27 10:33:27,118][20347] Updated weights for policy 0, policy_version 712547 (0.0008) [2023-12-27 10:33:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 364732416. Throughput: 0: 10309.5, 1: 10213.3. Samples: 364719916. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:27,363][19170] Avg episode reward: [(0, '980.430'), (1, '912.310')] [2023-12-27 10:33:27,603][20381] Updated weights for policy 1, policy_version 711994 (0.0007) [2023-12-27 10:33:27,661][20381] Updated weights for policy 1, policy_version 712004 (0.0005) [2023-12-27 10:33:27,725][20347] Updated weights for policy 0, policy_version 712557 (0.0005) [2023-12-27 10:33:27,725][20381] Updated weights for policy 1, policy_version 712014 (0.0008) [2023-12-27 10:33:27,777][20381] Updated weights for policy 1, policy_version 712024 (0.0008) [2023-12-27 10:33:27,784][20347] Updated weights for policy 0, policy_version 712567 (0.0005) [2023-12-27 10:33:27,840][20347] Updated weights for policy 0, policy_version 712577 (0.0005) [2023-12-27 10:33:28,425][20347] Updated weights for policy 0, policy_version 712587 (0.0005) [2023-12-27 10:33:28,459][20381] Updated weights for policy 1, policy_version 712034 (0.0008) [2023-12-27 10:33:28,480][20347] Updated weights for policy 0, policy_version 712597 (0.0010) [2023-12-27 10:33:28,513][20381] Updated weights for policy 1, policy_version 712044 (0.0007) [2023-12-27 10:33:28,536][20347] Updated weights for policy 0, policy_version 712607 (0.0010) [2023-12-27 10:33:28,574][20381] Updated weights for policy 1, policy_version 712054 (0.0010) [2023-12-27 10:33:29,205][20381] Updated weights for policy 1, policy_version 712064 (0.0007) [2023-12-27 10:33:29,216][20347] Updated weights for policy 0, policy_version 712617 (0.0010) [2023-12-27 10:33:29,265][20381] Updated weights for policy 1, policy_version 712074 (0.0008) [2023-12-27 10:33:29,280][20347] Updated weights for policy 0, policy_version 712627 (0.0010) [2023-12-27 10:33:29,321][20381] Updated weights for policy 1, policy_version 712084 (0.0011) [2023-12-27 10:33:29,344][20347] Updated weights for policy 0, policy_version 712637 (0.0009) [2023-12-27 10:33:29,407][20347] Updated weights for policy 0, policy_version 712647 (0.0008) [2023-12-27 10:33:29,988][20381] Updated weights for policy 1, policy_version 712094 (0.0008) [2023-12-27 10:33:30,044][20381] Updated weights for policy 1, policy_version 712104 (0.0007) [2023-12-27 10:33:30,081][20347] Updated weights for policy 0, policy_version 712657 (0.0006) [2023-12-27 10:33:30,101][20381] Updated weights for policy 1, policy_version 712114 (0.0007) [2023-12-27 10:33:30,137][20347] Updated weights for policy 0, policy_version 712667 (0.0006) [2023-12-27 10:33:30,198][20347] Updated weights for policy 0, policy_version 712677 (0.0006) [2023-12-27 10:33:30,676][20381] Updated weights for policy 1, policy_version 712124 (0.0007) [2023-12-27 10:33:30,729][20381] Updated weights for policy 1, policy_version 712134 (0.0007) [2023-12-27 10:33:30,776][20381] Updated weights for policy 1, policy_version 712144 (0.0007) [2023-12-27 10:33:30,873][20347] Updated weights for policy 0, policy_version 712687 (0.0007) [2023-12-27 10:33:30,920][20347] Updated weights for policy 0, policy_version 712697 (0.0007) [2023-12-27 10:33:30,967][20347] Updated weights for policy 0, policy_version 712707 (0.0007) [2023-12-27 10:33:31,501][20381] Updated weights for policy 1, policy_version 712154 (0.0007) [2023-12-27 10:33:31,566][20381] Updated weights for policy 1, policy_version 712164 (0.0008) [2023-12-27 10:33:31,629][20381] Updated weights for policy 1, policy_version 712174 (0.0008) [2023-12-27 10:33:31,695][20347] Updated weights for policy 0, policy_version 712717 (0.0009) [2023-12-27 10:33:31,697][20381] Updated weights for policy 1, policy_version 712184 (0.0007) [2023-12-27 10:33:31,755][20347] Updated weights for policy 0, policy_version 712727 (0.0010) [2023-12-27 10:33:31,816][20347] Updated weights for policy 0, policy_version 712737 (0.0010) [2023-12-27 10:33:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 364830720. Throughput: 0: 10341.0, 1: 10219.0. Samples: 364845568. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:32,363][19170] Avg episode reward: [(0, '980.750'), (1, '883.040')] [2023-12-27 10:33:32,379][20381] Updated weights for policy 1, policy_version 712194 (0.0008) [2023-12-27 10:33:32,436][20381] Updated weights for policy 1, policy_version 712204 (0.0006) [2023-12-27 10:33:32,490][20381] Updated weights for policy 1, policy_version 712214 (0.0006) [2023-12-27 10:33:32,492][20347] Updated weights for policy 0, policy_version 712747 (0.0011) [2023-12-27 10:33:32,540][20347] Updated weights for policy 0, policy_version 712757 (0.0010) [2023-12-27 10:33:32,598][20347] Updated weights for policy 0, policy_version 712767 (0.0010) [2023-12-27 10:33:33,179][20381] Updated weights for policy 1, policy_version 712224 (0.0007) [2023-12-27 10:33:33,224][20381] Updated weights for policy 1, policy_version 712234 (0.0007) [2023-12-27 10:33:33,276][20381] Updated weights for policy 1, policy_version 712244 (0.0008) [2023-12-27 10:33:33,281][20347] Updated weights for policy 0, policy_version 712777 (0.0010) [2023-12-27 10:33:33,342][20347] Updated weights for policy 0, policy_version 712787 (0.0010) [2023-12-27 10:33:33,396][20347] Updated weights for policy 0, policy_version 712797 (0.0010) [2023-12-27 10:33:33,450][20347] Updated weights for policy 0, policy_version 712807 (0.0010) [2023-12-27 10:33:33,981][20381] Updated weights for policy 1, policy_version 712254 (0.0007) [2023-12-27 10:33:34,025][20381] Updated weights for policy 1, policy_version 712264 (0.0007) [2023-12-27 10:33:34,082][20381] Updated weights for policy 1, policy_version 712274 (0.0007) [2023-12-27 10:33:34,128][20347] Updated weights for policy 0, policy_version 712817 (0.0010) [2023-12-27 10:33:34,186][20347] Updated weights for policy 0, policy_version 712827 (0.0008) [2023-12-27 10:33:34,237][20347] Updated weights for policy 0, policy_version 712837 (0.0007) [2023-12-27 10:33:34,785][20381] Updated weights for policy 1, policy_version 712284 (0.0006) [2023-12-27 10:33:34,842][20381] Updated weights for policy 1, policy_version 712294 (0.0005) [2023-12-27 10:33:34,888][20381] Updated weights for policy 1, policy_version 712304 (0.0005) [2023-12-27 10:33:34,929][20347] Updated weights for policy 0, policy_version 712847 (0.0009) [2023-12-27 10:33:34,985][20347] Updated weights for policy 0, policy_version 712857 (0.0010) [2023-12-27 10:33:35,034][20347] Updated weights for policy 0, policy_version 712867 (0.0010) [2023-12-27 10:33:35,551][20381] Updated weights for policy 1, policy_version 712314 (0.0006) [2023-12-27 10:33:35,609][20381] Updated weights for policy 1, policy_version 712324 (0.0007) [2023-12-27 10:33:35,666][20381] Updated weights for policy 1, policy_version 712334 (0.0007) [2023-12-27 10:33:35,667][20347] Updated weights for policy 0, policy_version 712877 (0.0010) [2023-12-27 10:33:35,716][20381] Updated weights for policy 1, policy_version 712344 (0.0007) [2023-12-27 10:33:35,726][20347] Updated weights for policy 0, policy_version 712887 (0.0010) [2023-12-27 10:33:35,780][20347] Updated weights for policy 0, policy_version 712897 (0.0010) [2023-12-27 10:33:36,418][20381] Updated weights for policy 1, policy_version 712354 (0.0007) [2023-12-27 10:33:36,478][20381] Updated weights for policy 1, policy_version 712364 (0.0007) [2023-12-27 10:33:36,490][20347] Updated weights for policy 0, policy_version 712907 (0.0010) [2023-12-27 10:33:36,537][20381] Updated weights for policy 1, policy_version 712374 (0.0006) [2023-12-27 10:33:36,550][20347] Updated weights for policy 0, policy_version 712917 (0.0010) [2023-12-27 10:33:36,613][20347] Updated weights for policy 0, policy_version 712927 (0.0011) [2023-12-27 10:33:37,249][20381] Updated weights for policy 1, policy_version 712384 (0.0007) [2023-12-27 10:33:37,295][20347] Updated weights for policy 0, policy_version 712937 (0.0011) [2023-12-27 10:33:37,301][20381] Updated weights for policy 1, policy_version 712394 (0.0008) [2023-12-27 10:33:37,352][20347] Updated weights for policy 0, policy_version 712947 (0.0011) [2023-12-27 10:33:37,354][20381] Updated weights for policy 1, policy_version 712404 (0.0005) [2023-12-27 10:33:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20660.5). Total num frames: 364929024. Throughput: 0: 10331.5, 1: 10221.2. Samples: 364906716. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:37,363][19170] Avg episode reward: [(0, '980.750'), (1, '853.730')] [2023-12-27 10:33:37,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000712408_182403072.pth... [2023-12-27 10:33:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000711224_182099968.pth [2023-12-27 10:33:37,399][20347] Updated weights for policy 0, policy_version 712957 (0.0007) [2023-12-27 10:33:37,454][20347] Updated weights for policy 0, policy_version 712967 (0.0009) [2023-12-27 10:33:37,456][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000712968_182542336.pth... [2023-12-27 10:33:37,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000711752_182231040.pth [2023-12-27 10:33:37,999][20381] Updated weights for policy 1, policy_version 712414 (0.0006) [2023-12-27 10:33:38,062][20381] Updated weights for policy 1, policy_version 712424 (0.0005) [2023-12-27 10:33:38,119][20381] Updated weights for policy 1, policy_version 712434 (0.0005) [2023-12-27 10:33:38,151][20347] Updated weights for policy 0, policy_version 712977 (0.0008) [2023-12-27 10:33:38,208][20347] Updated weights for policy 0, policy_version 712987 (0.0008) [2023-12-27 10:33:38,262][20347] Updated weights for policy 0, policy_version 712997 (0.0008) [2023-12-27 10:33:38,746][20381] Updated weights for policy 1, policy_version 712444 (0.0006) [2023-12-27 10:33:38,803][20381] Updated weights for policy 1, policy_version 712454 (0.0008) [2023-12-27 10:33:38,853][20381] Updated weights for policy 1, policy_version 712464 (0.0008) [2023-12-27 10:33:38,984][20347] Updated weights for policy 0, policy_version 713007 (0.0008) [2023-12-27 10:33:39,041][20347] Updated weights for policy 0, policy_version 713017 (0.0006) [2023-12-27 10:33:39,100][20347] Updated weights for policy 0, policy_version 713027 (0.0005) [2023-12-27 10:33:39,626][20381] Updated weights for policy 1, policy_version 712474 (0.0007) [2023-12-27 10:33:39,685][20381] Updated weights for policy 1, policy_version 712484 (0.0008) [2023-12-27 10:33:39,739][20347] Updated weights for policy 0, policy_version 713037 (0.0006) [2023-12-27 10:33:39,741][20381] Updated weights for policy 1, policy_version 712494 (0.0007) [2023-12-27 10:33:39,791][20347] Updated weights for policy 0, policy_version 713047 (0.0006) [2023-12-27 10:33:39,799][20381] Updated weights for policy 1, policy_version 712504 (0.0007) [2023-12-27 10:33:39,865][20347] Updated weights for policy 0, policy_version 713057 (0.0007) [2023-12-27 10:33:40,538][20347] Updated weights for policy 0, policy_version 713067 (0.0008) [2023-12-27 10:33:40,540][20381] Updated weights for policy 1, policy_version 712514 (0.0007) [2023-12-27 10:33:40,595][20381] Updated weights for policy 1, policy_version 712524 (0.0007) [2023-12-27 10:33:40,601][20347] Updated weights for policy 0, policy_version 713077 (0.0007) [2023-12-27 10:33:40,650][20347] Updated weights for policy 0, policy_version 713087 (0.0006) [2023-12-27 10:33:40,656][20381] Updated weights for policy 1, policy_version 712534 (0.0007) [2023-12-27 10:33:41,363][20347] Updated weights for policy 0, policy_version 713097 (0.0007) [2023-12-27 10:33:41,432][20347] Updated weights for policy 0, policy_version 713107 (0.0008) [2023-12-27 10:33:41,449][20381] Updated weights for policy 1, policy_version 712544 (0.0007) [2023-12-27 10:33:41,496][20347] Updated weights for policy 0, policy_version 713117 (0.0008) [2023-12-27 10:33:41,509][20381] Updated weights for policy 1, policy_version 712554 (0.0007) [2023-12-27 10:33:41,562][20347] Updated weights for policy 0, policy_version 713127 (0.0008) [2023-12-27 10:33:41,571][20381] Updated weights for policy 1, policy_version 712564 (0.0008) [2023-12-27 10:33:42,221][20381] Updated weights for policy 1, policy_version 712574 (0.0007) [2023-12-27 10:33:42,288][20347] Updated weights for policy 0, policy_version 713137 (0.0009) [2023-12-27 10:33:42,293][20381] Updated weights for policy 1, policy_version 712584 (0.0009) [2023-12-27 10:33:42,359][20347] Updated weights for policy 0, policy_version 713147 (0.0011) [2023-12-27 10:33:42,361][20381] Updated weights for policy 1, policy_version 712594 (0.0007) [2023-12-27 10:33:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 365027328. Throughput: 0: 10298.8, 1: 10166.4. Samples: 365026748. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:42,366][19170] Avg episode reward: [(0, '990.510'), (1, '844.730')] [2023-12-27 10:33:42,426][20347] Updated weights for policy 0, policy_version 713157 (0.0007) [2023-12-27 10:33:42,907][20381] Updated weights for policy 1, policy_version 712604 (0.0008) [2023-12-27 10:33:42,967][20381] Updated weights for policy 1, policy_version 712614 (0.0008) [2023-12-27 10:33:43,023][20381] Updated weights for policy 1, policy_version 712624 (0.0008) [2023-12-27 10:33:43,147][20347] Updated weights for policy 0, policy_version 713167 (0.0007) [2023-12-27 10:33:43,198][20347] Updated weights for policy 0, policy_version 713177 (0.0006) [2023-12-27 10:33:43,257][20347] Updated weights for policy 0, policy_version 713187 (0.0005) [2023-12-27 10:33:43,687][20381] Updated weights for policy 1, policy_version 712634 (0.0006) [2023-12-27 10:33:43,715][20292] KL-divergence is very high: 153.9696 [2023-12-27 10:33:43,737][20381] Updated weights for policy 1, policy_version 712644 (0.0007) [2023-12-27 10:33:43,792][20381] Updated weights for policy 1, policy_version 712654 (0.0005) [2023-12-27 10:33:43,841][20381] Updated weights for policy 1, policy_version 712664 (0.0005) [2023-12-27 10:33:43,868][20347] Updated weights for policy 0, policy_version 713197 (0.0008) [2023-12-27 10:33:43,915][20347] Updated weights for policy 0, policy_version 713207 (0.0010) [2023-12-27 10:33:43,967][20347] Updated weights for policy 0, policy_version 713217 (0.0009) [2023-12-27 10:33:44,408][20381] Updated weights for policy 1, policy_version 712674 (0.0008) [2023-12-27 10:33:44,460][20381] Updated weights for policy 1, policy_version 712684 (0.0008) [2023-12-27 10:33:44,521][20381] Updated weights for policy 1, policy_version 712694 (0.0007) [2023-12-27 10:33:44,645][20347] Updated weights for policy 0, policy_version 713227 (0.0010) [2023-12-27 10:33:44,710][20347] Updated weights for policy 0, policy_version 713237 (0.0009) [2023-12-27 10:33:44,771][20347] Updated weights for policy 0, policy_version 713247 (0.0010) [2023-12-27 10:33:45,242][20381] Updated weights for policy 1, policy_version 712704 (0.0008) [2023-12-27 10:33:45,297][20381] Updated weights for policy 1, policy_version 712714 (0.0007) [2023-12-27 10:33:45,347][20381] Updated weights for policy 1, policy_version 712724 (0.0007) [2023-12-27 10:33:45,470][20347] Updated weights for policy 0, policy_version 713257 (0.0009) [2023-12-27 10:33:45,534][20347] Updated weights for policy 0, policy_version 713267 (0.0005) [2023-12-27 10:33:45,591][20347] Updated weights for policy 0, policy_version 713277 (0.0005) [2023-12-27 10:33:45,635][20347] Updated weights for policy 0, policy_version 713287 (0.0005) [2023-12-27 10:33:46,046][20381] Updated weights for policy 1, policy_version 712734 (0.0006) [2023-12-27 10:33:46,098][20381] Updated weights for policy 1, policy_version 712744 (0.0005) [2023-12-27 10:33:46,155][20381] Updated weights for policy 1, policy_version 712754 (0.0007) [2023-12-27 10:33:46,160][20347] Updated weights for policy 0, policy_version 713297 (0.0007) [2023-12-27 10:33:46,219][20347] Updated weights for policy 0, policy_version 713307 (0.0007) [2023-12-27 10:33:46,278][20347] Updated weights for policy 0, policy_version 713317 (0.0008) [2023-12-27 10:33:46,736][20381] Updated weights for policy 1, policy_version 712764 (0.0008) [2023-12-27 10:33:46,784][20381] Updated weights for policy 1, policy_version 712774 (0.0008) [2023-12-27 10:33:46,837][20381] Updated weights for policy 1, policy_version 712784 (0.0008) [2023-12-27 10:33:46,959][20347] Updated weights for policy 0, policy_version 713327 (0.0009) [2023-12-27 10:33:47,022][20347] Updated weights for policy 0, policy_version 713337 (0.0011) [2023-12-27 10:33:47,079][20347] Updated weights for policy 0, policy_version 713347 (0.0010) [2023-12-27 10:33:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 365142016. Throughput: 0: 10248.3, 1: 10181.7. Samples: 365152148. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:47,363][19170] Avg episode reward: [(0, '990.510'), (1, '758.170')] [2023-12-27 10:33:47,528][20381] Updated weights for policy 1, policy_version 712794 (0.0009) [2023-12-27 10:33:47,580][20381] Updated weights for policy 1, policy_version 712804 (0.0010) [2023-12-27 10:33:47,627][20381] Updated weights for policy 1, policy_version 712814 (0.0010) [2023-12-27 10:33:47,682][20381] Updated weights for policy 1, policy_version 712824 (0.0010) [2023-12-27 10:33:47,708][20347] Updated weights for policy 0, policy_version 713357 (0.0007) [2023-12-27 10:33:47,763][20347] Updated weights for policy 0, policy_version 713367 (0.0007) [2023-12-27 10:33:47,820][20347] Updated weights for policy 0, policy_version 713377 (0.0005) [2023-12-27 10:33:48,346][20381] Updated weights for policy 1, policy_version 712834 (0.0011) [2023-12-27 10:33:48,360][20347] Updated weights for policy 0, policy_version 713387 (0.0007) [2023-12-27 10:33:48,408][20381] Updated weights for policy 1, policy_version 712844 (0.0011) [2023-12-27 10:33:48,421][20347] Updated weights for policy 0, policy_version 713397 (0.0006) [2023-12-27 10:33:48,465][20381] Updated weights for policy 1, policy_version 712854 (0.0011) [2023-12-27 10:33:48,474][20347] Updated weights for policy 0, policy_version 713407 (0.0006) [2023-12-27 10:33:49,162][20381] Updated weights for policy 1, policy_version 712864 (0.0010) [2023-12-27 10:33:49,167][20347] Updated weights for policy 0, policy_version 713417 (0.0007) [2023-12-27 10:33:49,217][20292] KL-divergence is very high: 169.0003 [2023-12-27 10:33:49,218][20381] Updated weights for policy 1, policy_version 712874 (0.0011) [2023-12-27 10:33:49,226][20347] Updated weights for policy 0, policy_version 713427 (0.0006) [2023-12-27 10:33:49,279][20381] Updated weights for policy 1, policy_version 712884 (0.0011) [2023-12-27 10:33:49,288][20347] Updated weights for policy 0, policy_version 713437 (0.0006) [2023-12-27 10:33:49,347][20347] Updated weights for policy 0, policy_version 713447 (0.0007) [2023-12-27 10:33:49,951][20347] Updated weights for policy 0, policy_version 713457 (0.0008) [2023-12-27 10:33:49,980][20381] Updated weights for policy 1, policy_version 712894 (0.0009) [2023-12-27 10:33:50,005][20347] Updated weights for policy 0, policy_version 713467 (0.0006) [2023-12-27 10:33:50,034][20381] Updated weights for policy 1, policy_version 712904 (0.0006) [2023-12-27 10:33:50,064][20347] Updated weights for policy 0, policy_version 713477 (0.0006) [2023-12-27 10:33:50,091][20381] Updated weights for policy 1, policy_version 712914 (0.0007) [2023-12-27 10:33:50,765][20347] Updated weights for policy 0, policy_version 713487 (0.0006) [2023-12-27 10:33:50,814][20381] Updated weights for policy 1, policy_version 712924 (0.0006) [2023-12-27 10:33:50,827][20347] Updated weights for policy 0, policy_version 713497 (0.0010) [2023-12-27 10:33:50,870][20381] Updated weights for policy 1, policy_version 712934 (0.0006) [2023-12-27 10:33:50,883][20347] Updated weights for policy 0, policy_version 713507 (0.0010) [2023-12-27 10:33:50,922][20381] Updated weights for policy 1, policy_version 712944 (0.0008) [2023-12-27 10:33:51,576][20347] Updated weights for policy 0, policy_version 713517 (0.0007) [2023-12-27 10:33:51,641][20381] Updated weights for policy 1, policy_version 712954 (0.0008) [2023-12-27 10:33:51,645][20347] Updated weights for policy 0, policy_version 713527 (0.0008) [2023-12-27 10:33:51,703][20381] Updated weights for policy 1, policy_version 712964 (0.0006) [2023-12-27 10:33:51,704][20347] Updated weights for policy 0, policy_version 713537 (0.0006) [2023-12-27 10:33:51,774][20381] Updated weights for policy 1, policy_version 712974 (0.0009) [2023-12-27 10:33:51,842][20381] Updated weights for policy 1, policy_version 712984 (0.0008) [2023-12-27 10:33:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 365240320. Throughput: 0: 10260.0, 1: 10145.3. Samples: 365215144. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '749.460')] [2023-12-27 10:33:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000712984_182550528.pth... [2023-12-27 10:33:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000711800_182247424.pth [2023-12-27 10:33:52,388][20347] Updated weights for policy 0, policy_version 713547 (0.0007) [2023-12-27 10:33:52,453][20347] Updated weights for policy 0, policy_version 713557 (0.0007) [2023-12-27 10:33:52,492][20381] Updated weights for policy 1, policy_version 712994 (0.0011) [2023-12-27 10:33:52,508][20347] Updated weights for policy 0, policy_version 713567 (0.0006) [2023-12-27 10:33:52,551][20381] Updated weights for policy 1, policy_version 713004 (0.0011) [2023-12-27 10:33:52,564][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000713576_182697984.pth... [2023-12-27 10:33:52,568][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000712328_182378496.pth [2023-12-27 10:33:52,605][20381] Updated weights for policy 1, policy_version 713014 (0.0011) [2023-12-27 10:33:53,175][20347] Updated weights for policy 0, policy_version 713577 (0.0005) [2023-12-27 10:33:53,216][20381] Updated weights for policy 1, policy_version 713024 (0.0008) [2023-12-27 10:33:53,242][20347] Updated weights for policy 0, policy_version 713587 (0.0006) [2023-12-27 10:33:53,271][20381] Updated weights for policy 1, policy_version 713034 (0.0007) [2023-12-27 10:33:53,308][20347] Updated weights for policy 0, policy_version 713597 (0.0006) [2023-12-27 10:33:53,328][20381] Updated weights for policy 1, policy_version 713044 (0.0006) [2023-12-27 10:33:53,367][20347] Updated weights for policy 0, policy_version 713607 (0.0005) [2023-12-27 10:33:53,950][20347] Updated weights for policy 0, policy_version 713617 (0.0009) [2023-12-27 10:33:53,972][20381] Updated weights for policy 1, policy_version 713054 (0.0005) [2023-12-27 10:33:54,007][20347] Updated weights for policy 0, policy_version 713627 (0.0008) [2023-12-27 10:33:54,018][20381] Updated weights for policy 1, policy_version 713064 (0.0005) [2023-12-27 10:33:54,057][20347] Updated weights for policy 0, policy_version 713637 (0.0008) [2023-12-27 10:33:54,075][20381] Updated weights for policy 1, policy_version 713074 (0.0006) [2023-12-27 10:33:54,760][20347] Updated weights for policy 0, policy_version 713647 (0.0008) [2023-12-27 10:33:54,771][20381] Updated weights for policy 1, policy_version 713084 (0.0008) [2023-12-27 10:33:54,814][20347] Updated weights for policy 0, policy_version 713657 (0.0009) [2023-12-27 10:33:54,830][20381] Updated weights for policy 1, policy_version 713094 (0.0005) [2023-12-27 10:33:54,872][20347] Updated weights for policy 0, policy_version 713667 (0.0008) [2023-12-27 10:33:54,891][20381] Updated weights for policy 1, policy_version 713104 (0.0006) [2023-12-27 10:33:55,514][20381] Updated weights for policy 1, policy_version 713114 (0.0007) [2023-12-27 10:33:55,549][20347] Updated weights for policy 0, policy_version 713677 (0.0009) [2023-12-27 10:33:55,570][20381] Updated weights for policy 1, policy_version 713124 (0.0011) [2023-12-27 10:33:55,612][20347] Updated weights for policy 0, policy_version 713687 (0.0009) [2023-12-27 10:33:55,623][20381] Updated weights for policy 1, policy_version 713134 (0.0010) [2023-12-27 10:33:55,663][20347] Updated weights for policy 0, policy_version 713697 (0.0010) [2023-12-27 10:33:55,675][20381] Updated weights for policy 1, policy_version 713144 (0.0011) [2023-12-27 10:33:56,398][20381] Updated weights for policy 1, policy_version 713154 (0.0011) [2023-12-27 10:33:56,429][20347] Updated weights for policy 0, policy_version 713707 (0.0009) [2023-12-27 10:33:56,457][20381] Updated weights for policy 1, policy_version 713164 (0.0011) [2023-12-27 10:33:56,475][20347] Updated weights for policy 0, policy_version 713717 (0.0006) [2023-12-27 10:33:56,513][20381] Updated weights for policy 1, policy_version 713174 (0.0010) [2023-12-27 10:33:56,531][20347] Updated weights for policy 0, policy_version 713727 (0.0010) [2023-12-27 10:33:57,180][20381] Updated weights for policy 1, policy_version 713184 (0.0010) [2023-12-27 10:33:57,225][20347] Updated weights for policy 0, policy_version 713737 (0.0007) [2023-12-27 10:33:57,232][20381] Updated weights for policy 1, policy_version 713194 (0.0011) [2023-12-27 10:33:57,275][20347] Updated weights for policy 0, policy_version 713747 (0.0005) [2023-12-27 10:33:57,285][20381] Updated weights for policy 1, policy_version 713204 (0.0011) [2023-12-27 10:33:57,329][20347] Updated weights for policy 0, policy_version 713757 (0.0006) [2023-12-27 10:33:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 365346816. Throughput: 0: 10268.1, 1: 10169.2. Samples: 365337476. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:33:57,363][19170] Avg episode reward: [(0, '951.480'), (1, '787.720')] [2023-12-27 10:33:57,385][20347] Updated weights for policy 0, policy_version 713767 (0.0007) [2023-12-27 10:33:58,001][20381] Updated weights for policy 1, policy_version 713214 (0.0008) [2023-12-27 10:33:58,059][20381] Updated weights for policy 1, policy_version 713224 (0.0005) [2023-12-27 10:33:58,115][20381] Updated weights for policy 1, policy_version 713234 (0.0005) [2023-12-27 10:33:58,128][20347] Updated weights for policy 0, policy_version 713777 (0.0008) [2023-12-27 10:33:58,198][20347] Updated weights for policy 0, policy_version 713787 (0.0008) [2023-12-27 10:33:58,259][20347] Updated weights for policy 0, policy_version 713797 (0.0007) [2023-12-27 10:33:58,823][20381] Updated weights for policy 1, policy_version 713244 (0.0007) [2023-12-27 10:33:58,890][20381] Updated weights for policy 1, policy_version 713254 (0.0010) [2023-12-27 10:33:58,955][20381] Updated weights for policy 1, policy_version 713264 (0.0008) [2023-12-27 10:33:59,099][20347] Updated weights for policy 0, policy_version 713807 (0.0007) [2023-12-27 10:33:59,148][20347] Updated weights for policy 0, policy_version 713817 (0.0005) [2023-12-27 10:33:59,214][20347] Updated weights for policy 0, policy_version 713827 (0.0007) [2023-12-27 10:33:59,698][20381] Updated weights for policy 1, policy_version 713274 (0.0008) [2023-12-27 10:33:59,752][20381] Updated weights for policy 1, policy_version 713284 (0.0007) [2023-12-27 10:33:59,798][20381] Updated weights for policy 1, policy_version 713294 (0.0008) [2023-12-27 10:33:59,856][20381] Updated weights for policy 1, policy_version 713304 (0.0007) [2023-12-27 10:33:59,898][20347] Updated weights for policy 0, policy_version 713837 (0.0008) [2023-12-27 10:33:59,953][20347] Updated weights for policy 0, policy_version 713847 (0.0008) [2023-12-27 10:34:00,002][20347] Updated weights for policy 0, policy_version 713857 (0.0009) [2023-12-27 10:34:00,542][20381] Updated weights for policy 1, policy_version 713314 (0.0011) [2023-12-27 10:34:00,598][20381] Updated weights for policy 1, policy_version 713324 (0.0011) [2023-12-27 10:34:00,660][20381] Updated weights for policy 1, policy_version 713334 (0.0009) [2023-12-27 10:34:00,745][20347] Updated weights for policy 0, policy_version 713867 (0.0008) [2023-12-27 10:34:00,792][20291] KL-divergence is very high: 307.0165 [2023-12-27 10:34:00,796][20347] Updated weights for policy 0, policy_version 713877 (0.0008) [2023-12-27 10:34:00,853][20347] Updated weights for policy 0, policy_version 713887 (0.0007) [2023-12-27 10:34:00,882][20291] KL-divergence is very high: 194.0709 [2023-12-27 10:34:01,362][20381] Updated weights for policy 1, policy_version 713344 (0.0009) [2023-12-27 10:34:01,423][20381] Updated weights for policy 1, policy_version 713354 (0.0008) [2023-12-27 10:34:01,483][20381] Updated weights for policy 1, policy_version 713364 (0.0007) [2023-12-27 10:34:01,673][20347] Updated weights for policy 0, policy_version 713897 (0.0008) [2023-12-27 10:34:01,737][20347] Updated weights for policy 0, policy_version 713907 (0.0008) [2023-12-27 10:34:01,801][20291] KL-divergence is very high: 152.9343 [2023-12-27 10:34:01,802][20347] Updated weights for policy 0, policy_version 713917 (0.0008) [2023-12-27 10:34:01,854][20347] Updated weights for policy 0, policy_version 713927 (0.0007) [2023-12-27 10:34:02,102][20381] Updated weights for policy 1, policy_version 713374 (0.0007) [2023-12-27 10:34:02,163][20381] Updated weights for policy 1, policy_version 713384 (0.0008) [2023-12-27 10:34:02,224][20381] Updated weights for policy 1, policy_version 713394 (0.0006) [2023-12-27 10:34:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 365445120. Throughput: 0: 10153.2, 1: 10197.9. Samples: 365453568. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:02,363][19170] Avg episode reward: [(0, '893.790'), (1, '758.100')] [2023-12-27 10:34:02,587][20347] Updated weights for policy 0, policy_version 713937 (0.0007) [2023-12-27 10:34:02,635][20347] Updated weights for policy 0, policy_version 713947 (0.0007) [2023-12-27 10:34:02,681][20347] Updated weights for policy 0, policy_version 713957 (0.0009) [2023-12-27 10:34:02,812][20381] Updated weights for policy 1, policy_version 713404 (0.0007) [2023-12-27 10:34:02,858][20381] Updated weights for policy 1, policy_version 713414 (0.0007) [2023-12-27 10:34:02,903][20381] Updated weights for policy 1, policy_version 713424 (0.0007) [2023-12-27 10:34:03,379][20347] Updated weights for policy 0, policy_version 713967 (0.0008) [2023-12-27 10:34:03,408][20291] KL-divergence is very high: 103.8159 [2023-12-27 10:34:03,425][20291] KL-divergence is very high: 140.0433 [2023-12-27 10:34:03,434][20347] Updated weights for policy 0, policy_version 713977 (0.0007) [2023-12-27 10:34:03,483][20347] Updated weights for policy 0, policy_version 713987 (0.0007) [2023-12-27 10:34:03,539][20381] Updated weights for policy 1, policy_version 713434 (0.0007) [2023-12-27 10:34:03,600][20381] Updated weights for policy 1, policy_version 713444 (0.0005) [2023-12-27 10:34:03,652][20381] Updated weights for policy 1, policy_version 713454 (0.0005) [2023-12-27 10:34:03,700][20381] Updated weights for policy 1, policy_version 713464 (0.0007) [2023-12-27 10:34:04,187][20347] Updated weights for policy 0, policy_version 713997 (0.0008) [2023-12-27 10:34:04,245][20347] Updated weights for policy 0, policy_version 714007 (0.0009) [2023-12-27 10:34:04,309][20347] Updated weights for policy 0, policy_version 714017 (0.0007) [2023-12-27 10:34:04,369][20381] Updated weights for policy 1, policy_version 713474 (0.0007) [2023-12-27 10:34:04,391][20292] KL-divergence is very high: 101.0055 [2023-12-27 10:34:04,434][20381] Updated weights for policy 1, policy_version 713484 (0.0007) [2023-12-27 10:34:04,493][20381] Updated weights for policy 1, policy_version 713494 (0.0007) [2023-12-27 10:34:04,943][20347] Updated weights for policy 0, policy_version 714027 (0.0006) [2023-12-27 10:34:04,998][20347] Updated weights for policy 0, policy_version 714037 (0.0006) [2023-12-27 10:34:05,058][20347] Updated weights for policy 0, policy_version 714047 (0.0007) [2023-12-27 10:34:05,088][20292] KL-divergence is very high: 192.2300 [2023-12-27 10:34:05,119][20381] Updated weights for policy 1, policy_version 713504 (0.0006) [2023-12-27 10:34:05,128][20292] KL-divergence is very high: 301.7170 [2023-12-27 10:34:05,138][20292] KL-divergence is very high: 614.4670 [2023-12-27 10:34:05,167][20381] Updated weights for policy 1, policy_version 713514 (0.0007) [2023-12-27 10:34:05,219][20381] Updated weights for policy 1, policy_version 713524 (0.0007) [2023-12-27 10:34:05,729][20347] Updated weights for policy 0, policy_version 714057 (0.0010) [2023-12-27 10:34:05,782][20347] Updated weights for policy 0, policy_version 714067 (0.0008) [2023-12-27 10:34:05,832][20347] Updated weights for policy 0, policy_version 714077 (0.0010) [2023-12-27 10:34:05,837][20381] Updated weights for policy 1, policy_version 713534 (0.0006) [2023-12-27 10:34:05,838][20292] KL-divergence is very high: 111.6485 [2023-12-27 10:34:05,883][20292] KL-divergence is very high: 113.2229 [2023-12-27 10:34:05,888][20347] Updated weights for policy 0, policy_version 714087 (0.0010) [2023-12-27 10:34:05,896][20381] Updated weights for policy 1, policy_version 713544 (0.0005) [2023-12-27 10:34:05,950][20381] Updated weights for policy 1, policy_version 713554 (0.0007) [2023-12-27 10:34:06,566][20347] Updated weights for policy 0, policy_version 714097 (0.0011) [2023-12-27 10:34:06,619][20347] Updated weights for policy 0, policy_version 714107 (0.0010) [2023-12-27 10:34:06,620][20381] Updated weights for policy 1, policy_version 713564 (0.0007) [2023-12-27 10:34:06,672][20347] Updated weights for policy 0, policy_version 714117 (0.0011) [2023-12-27 10:34:06,682][20381] Updated weights for policy 1, policy_version 713574 (0.0006) [2023-12-27 10:34:06,744][20381] Updated weights for policy 1, policy_version 713584 (0.0008) [2023-12-27 10:34:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20632.7). Total num frames: 365543424. Throughput: 0: 10179.1, 1: 10250.3. Samples: 365516672. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:07,363][19170] Avg episode reward: [(0, '866.160'), (1, '776.800')] [2023-12-27 10:34:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000713592_182706176.pth... [2023-12-27 10:34:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000712408_182403072.pth [2023-12-27 10:34:07,382][20347] Updated weights for policy 0, policy_version 714127 (0.0008) [2023-12-27 10:34:07,410][20381] Updated weights for policy 1, policy_version 713594 (0.0007) [2023-12-27 10:34:07,435][20347] Updated weights for policy 0, policy_version 714137 (0.0006) [2023-12-27 10:34:07,471][20381] Updated weights for policy 1, policy_version 713604 (0.0006) [2023-12-27 10:34:07,487][20347] Updated weights for policy 0, policy_version 714147 (0.0008) [2023-12-27 10:34:07,519][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000714152_182845440.pth... [2023-12-27 10:34:07,523][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000712968_182542336.pth [2023-12-27 10:34:07,527][20381] Updated weights for policy 1, policy_version 713614 (0.0005) [2023-12-27 10:34:07,583][20381] Updated weights for policy 1, policy_version 713624 (0.0005) [2023-12-27 10:34:08,153][20381] Updated weights for policy 1, policy_version 713634 (0.0007) [2023-12-27 10:34:08,191][20347] Updated weights for policy 0, policy_version 714157 (0.0007) [2023-12-27 10:34:08,205][20381] Updated weights for policy 1, policy_version 713644 (0.0008) [2023-12-27 10:34:08,210][20291] KL-divergence is very high: 159.9525 [2023-12-27 10:34:08,248][20347] Updated weights for policy 0, policy_version 714167 (0.0005) [2023-12-27 10:34:08,259][20381] Updated weights for policy 1, policy_version 713654 (0.0007) [2023-12-27 10:34:08,302][20347] Updated weights for policy 0, policy_version 714177 (0.0005) [2023-12-27 10:34:08,872][20347] Updated weights for policy 0, policy_version 714187 (0.0007) [2023-12-27 10:34:08,926][20347] Updated weights for policy 0, policy_version 714197 (0.0007) [2023-12-27 10:34:08,972][20347] Updated weights for policy 0, policy_version 714207 (0.0008) [2023-12-27 10:34:09,036][20381] Updated weights for policy 1, policy_version 713664 (0.0007) [2023-12-27 10:34:09,082][20381] Updated weights for policy 1, policy_version 713674 (0.0008) [2023-12-27 10:34:09,139][20381] Updated weights for policy 1, policy_version 713684 (0.0008) [2023-12-27 10:34:09,687][20347] Updated weights for policy 0, policy_version 714217 (0.0007) [2023-12-27 10:34:09,751][20347] Updated weights for policy 0, policy_version 714227 (0.0007) [2023-12-27 10:34:09,811][20347] Updated weights for policy 0, policy_version 714237 (0.0008) [2023-12-27 10:34:09,835][20381] Updated weights for policy 1, policy_version 713694 (0.0008) [2023-12-27 10:34:09,882][20347] Updated weights for policy 0, policy_version 714247 (0.0008) [2023-12-27 10:34:09,893][20381] Updated weights for policy 1, policy_version 713704 (0.0007) [2023-12-27 10:34:09,960][20381] Updated weights for policy 1, policy_version 713714 (0.0007) [2023-12-27 10:34:10,541][20347] Updated weights for policy 0, policy_version 714257 (0.0007) [2023-12-27 10:34:10,598][20347] Updated weights for policy 0, policy_version 714267 (0.0007) [2023-12-27 10:34:10,645][20347] Updated weights for policy 0, policy_version 714277 (0.0007) [2023-12-27 10:34:10,663][20381] Updated weights for policy 1, policy_version 713724 (0.0010) [2023-12-27 10:34:10,728][20381] Updated weights for policy 1, policy_version 713734 (0.0011) [2023-12-27 10:34:10,783][20381] Updated weights for policy 1, policy_version 713744 (0.0010) [2023-12-27 10:34:11,341][20347] Updated weights for policy 0, policy_version 714287 (0.0007) [2023-12-27 10:34:11,415][20347] Updated weights for policy 0, policy_version 714297 (0.0009) [2023-12-27 10:34:11,462][20347] Updated weights for policy 0, policy_version 714307 (0.0008) [2023-12-27 10:34:11,484][20381] Updated weights for policy 1, policy_version 713754 (0.0010) [2023-12-27 10:34:11,547][20381] Updated weights for policy 1, policy_version 713764 (0.0008) [2023-12-27 10:34:11,604][20381] Updated weights for policy 1, policy_version 713774 (0.0008) [2023-12-27 10:34:11,668][20381] Updated weights for policy 1, policy_version 713784 (0.0007) [2023-12-27 10:34:12,132][20347] Updated weights for policy 0, policy_version 714317 (0.0007) [2023-12-27 10:34:12,198][20347] Updated weights for policy 0, policy_version 714327 (0.0008) [2023-12-27 10:34:12,258][20347] Updated weights for policy 0, policy_version 714337 (0.0008) [2023-12-27 10:34:12,347][20381] Updated weights for policy 1, policy_version 713794 (0.0008) [2023-12-27 10:34:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 365649920. Throughput: 0: 10191.9, 1: 10270.6. Samples: 365640728. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:12,363][19170] Avg episode reward: [(0, '829.460'), (1, '795.540')] [2023-12-27 10:34:12,412][20381] Updated weights for policy 1, policy_version 713804 (0.0007) [2023-12-27 10:34:12,476][20381] Updated weights for policy 1, policy_version 713814 (0.0008) [2023-12-27 10:34:12,947][20347] Updated weights for policy 0, policy_version 714347 (0.0008) [2023-12-27 10:34:13,001][20347] Updated weights for policy 0, policy_version 714357 (0.0007) [2023-12-27 10:34:13,064][20347] Updated weights for policy 0, policy_version 714367 (0.0007) [2023-12-27 10:34:13,178][20381] Updated weights for policy 1, policy_version 713824 (0.0007) [2023-12-27 10:34:13,230][20381] Updated weights for policy 1, policy_version 713834 (0.0006) [2023-12-27 10:34:13,282][20381] Updated weights for policy 1, policy_version 713844 (0.0006) [2023-12-27 10:34:13,717][20347] Updated weights for policy 0, policy_version 714377 (0.0007) [2023-12-27 10:34:13,768][20347] Updated weights for policy 0, policy_version 714387 (0.0006) [2023-12-27 10:34:13,821][20347] Updated weights for policy 0, policy_version 714397 (0.0006) [2023-12-27 10:34:13,879][20347] Updated weights for policy 0, policy_version 714407 (0.0006) [2023-12-27 10:34:13,939][20381] Updated weights for policy 1, policy_version 713854 (0.0009) [2023-12-27 10:34:13,990][20381] Updated weights for policy 1, policy_version 713864 (0.0010) [2023-12-27 10:34:14,047][20381] Updated weights for policy 1, policy_version 713874 (0.0010) [2023-12-27 10:34:14,492][20347] Updated weights for policy 0, policy_version 714417 (0.0007) [2023-12-27 10:34:14,555][20347] Updated weights for policy 0, policy_version 714427 (0.0005) [2023-12-27 10:34:14,617][20347] Updated weights for policy 0, policy_version 714437 (0.0005) [2023-12-27 10:34:14,737][20381] Updated weights for policy 1, policy_version 713884 (0.0010) [2023-12-27 10:34:14,798][20381] Updated weights for policy 1, policy_version 713894 (0.0011) [2023-12-27 10:34:14,863][20381] Updated weights for policy 1, policy_version 713904 (0.0011) [2023-12-27 10:34:15,299][20347] Updated weights for policy 0, policy_version 714447 (0.0007) [2023-12-27 10:34:15,355][20347] Updated weights for policy 0, policy_version 714457 (0.0008) [2023-12-27 10:34:15,416][20347] Updated weights for policy 0, policy_version 714467 (0.0008) [2023-12-27 10:34:15,535][20381] Updated weights for policy 1, policy_version 713914 (0.0011) [2023-12-27 10:34:15,591][20381] Updated weights for policy 1, policy_version 713924 (0.0011) [2023-12-27 10:34:15,651][20381] Updated weights for policy 1, policy_version 713934 (0.0008) [2023-12-27 10:34:15,704][20381] Updated weights for policy 1, policy_version 713944 (0.0007) [2023-12-27 10:34:16,041][20347] Updated weights for policy 0, policy_version 714477 (0.0007) [2023-12-27 10:34:16,092][20347] Updated weights for policy 0, policy_version 714487 (0.0005) [2023-12-27 10:34:16,129][20291] KL-divergence is very high: 109.8144 [2023-12-27 10:34:16,139][20347] Updated weights for policy 0, policy_version 714497 (0.0005) [2023-12-27 10:34:16,310][20381] Updated weights for policy 1, policy_version 713954 (0.0010) [2023-12-27 10:34:16,376][20381] Updated weights for policy 1, policy_version 713964 (0.0008) [2023-12-27 10:34:16,426][20381] Updated weights for policy 1, policy_version 713974 (0.0008) [2023-12-27 10:34:16,766][20347] Updated weights for policy 0, policy_version 714507 (0.0006) [2023-12-27 10:34:16,821][20347] Updated weights for policy 0, policy_version 714517 (0.0006) [2023-12-27 10:34:16,885][20347] Updated weights for policy 0, policy_version 714527 (0.0008) [2023-12-27 10:34:17,053][20381] Updated weights for policy 1, policy_version 713984 (0.0007) [2023-12-27 10:34:17,116][20381] Updated weights for policy 1, policy_version 713994 (0.0008) [2023-12-27 10:34:17,172][20381] Updated weights for policy 1, policy_version 714004 (0.0007) [2023-12-27 10:34:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 365756416. Throughput: 0: 10175.9, 1: 10239.0. Samples: 365764240. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:17,363][19170] Avg episode reward: [(0, '818.970'), (1, '757.620')] [2023-12-27 10:34:17,563][20347] Updated weights for policy 0, policy_version 714537 (0.0008) [2023-12-27 10:34:17,618][20347] Updated weights for policy 0, policy_version 714547 (0.0009) [2023-12-27 10:34:17,678][20347] Updated weights for policy 0, policy_version 714557 (0.0007) [2023-12-27 10:34:17,732][20347] Updated weights for policy 0, policy_version 714567 (0.0007) [2023-12-27 10:34:17,752][20381] Updated weights for policy 1, policy_version 714014 (0.0008) [2023-12-27 10:34:17,806][20381] Updated weights for policy 1, policy_version 714024 (0.0008) [2023-12-27 10:34:17,854][20381] Updated weights for policy 1, policy_version 714034 (0.0007) [2023-12-27 10:34:18,408][20347] Updated weights for policy 0, policy_version 714577 (0.0006) [2023-12-27 10:34:18,463][20347] Updated weights for policy 0, policy_version 714587 (0.0005) [2023-12-27 10:34:18,515][20347] Updated weights for policy 0, policy_version 714597 (0.0005) [2023-12-27 10:34:18,596][20381] Updated weights for policy 1, policy_version 714044 (0.0010) [2023-12-27 10:34:18,652][20381] Updated weights for policy 1, policy_version 714054 (0.0011) [2023-12-27 10:34:18,711][20381] Updated weights for policy 1, policy_version 714064 (0.0010) [2023-12-27 10:34:19,079][20347] Updated weights for policy 0, policy_version 714607 (0.0006) [2023-12-27 10:34:19,137][20347] Updated weights for policy 0, policy_version 714617 (0.0008) [2023-12-27 10:34:19,186][20347] Updated weights for policy 0, policy_version 714627 (0.0007) [2023-12-27 10:34:19,371][20381] Updated weights for policy 1, policy_version 714074 (0.0011) [2023-12-27 10:34:19,432][20381] Updated weights for policy 1, policy_version 714084 (0.0009) [2023-12-27 10:34:19,487][20381] Updated weights for policy 1, policy_version 714094 (0.0009) [2023-12-27 10:34:19,543][20381] Updated weights for policy 1, policy_version 714104 (0.0009) [2023-12-27 10:34:19,976][20347] Updated weights for policy 0, policy_version 714637 (0.0008) [2023-12-27 10:34:20,035][20347] Updated weights for policy 0, policy_version 714647 (0.0007) [2023-12-27 10:34:20,097][20347] Updated weights for policy 0, policy_version 714657 (0.0008) [2023-12-27 10:34:20,361][20381] Updated weights for policy 1, policy_version 714114 (0.0007) [2023-12-27 10:34:20,424][20381] Updated weights for policy 1, policy_version 714124 (0.0006) [2023-12-27 10:34:20,483][20381] Updated weights for policy 1, policy_version 714134 (0.0008) [2023-12-27 10:34:20,805][20347] Updated weights for policy 0, policy_version 714667 (0.0008) [2023-12-27 10:34:20,872][20347] Updated weights for policy 0, policy_version 714677 (0.0008) [2023-12-27 10:34:20,936][20347] Updated weights for policy 0, policy_version 714687 (0.0008) [2023-12-27 10:34:21,213][20381] Updated weights for policy 1, policy_version 714144 (0.0008) [2023-12-27 10:34:21,280][20381] Updated weights for policy 1, policy_version 714154 (0.0008) [2023-12-27 10:34:21,349][20381] Updated weights for policy 1, policy_version 714164 (0.0008) [2023-12-27 10:34:21,658][20347] Updated weights for policy 0, policy_version 714697 (0.0008) [2023-12-27 10:34:21,728][20347] Updated weights for policy 0, policy_version 714707 (0.0007) [2023-12-27 10:34:21,791][20347] Updated weights for policy 0, policy_version 714717 (0.0008) [2023-12-27 10:34:21,854][20347] Updated weights for policy 0, policy_version 714727 (0.0008) [2023-12-27 10:34:22,135][20381] Updated weights for policy 1, policy_version 714174 (0.0008) [2023-12-27 10:34:22,194][20381] Updated weights for policy 1, policy_version 714184 (0.0009) [2023-12-27 10:34:22,254][20381] Updated weights for policy 1, policy_version 714194 (0.0009) [2023-12-27 10:34:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 365854720. Throughput: 0: 10188.4, 1: 10225.2. Samples: 365825328. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:22,363][19170] Avg episode reward: [(0, '835.930'), (1, '640.750')] [2023-12-27 10:34:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000714200_182861824.pth... [2023-12-27 10:34:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000714728_182992896.pth... [2023-12-27 10:34:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000712984_182550528.pth [2023-12-27 10:34:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000713576_182697984.pth [2023-12-27 10:34:22,555][20291] KL-divergence is very high: 194.8772 [2023-12-27 10:34:22,588][20347] Updated weights for policy 0, policy_version 714737 (0.0007) [2023-12-27 10:34:22,646][20347] Updated weights for policy 0, policy_version 714747 (0.0007) [2023-12-27 10:34:22,646][20291] KL-divergence is very high: 117.8472 [2023-12-27 10:34:22,691][20291] KL-divergence is very high: 116.3831 [2023-12-27 10:34:22,705][20347] Updated weights for policy 0, policy_version 714757 (0.0008) [2023-12-27 10:34:22,951][20381] Updated weights for policy 1, policy_version 714204 (0.0008) [2023-12-27 10:34:23,004][20381] Updated weights for policy 1, policy_version 714214 (0.0009) [2023-12-27 10:34:23,058][20381] Updated weights for policy 1, policy_version 714224 (0.0011) [2023-12-27 10:34:23,423][20347] Updated weights for policy 0, policy_version 714767 (0.0006) [2023-12-27 10:34:23,474][20347] Updated weights for policy 0, policy_version 714777 (0.0005) [2023-12-27 10:34:23,521][20347] Updated weights for policy 0, policy_version 714787 (0.0005) [2023-12-27 10:34:23,675][20381] Updated weights for policy 1, policy_version 714234 (0.0010) [2023-12-27 10:34:23,729][20381] Updated weights for policy 1, policy_version 714244 (0.0010) [2023-12-27 10:34:23,792][20381] Updated weights for policy 1, policy_version 714254 (0.0011) [2023-12-27 10:34:23,853][20381] Updated weights for policy 1, policy_version 714264 (0.0005) [2023-12-27 10:34:24,113][20347] Updated weights for policy 0, policy_version 714797 (0.0005) [2023-12-27 10:34:24,173][20347] Updated weights for policy 0, policy_version 714807 (0.0005) [2023-12-27 10:34:24,227][20347] Updated weights for policy 0, policy_version 714817 (0.0008) [2023-12-27 10:34:24,418][20381] Updated weights for policy 1, policy_version 714274 (0.0007) [2023-12-27 10:34:24,448][20292] KL-divergence is very high: 128.8073 [2023-12-27 10:34:24,480][20381] Updated weights for policy 1, policy_version 714284 (0.0005) [2023-12-27 10:34:24,529][20381] Updated weights for policy 1, policy_version 714294 (0.0006) [2023-12-27 10:34:24,819][20347] Updated weights for policy 0, policy_version 714827 (0.0009) [2023-12-27 10:34:24,870][20347] Updated weights for policy 0, policy_version 714837 (0.0010) [2023-12-27 10:34:24,918][20347] Updated weights for policy 0, policy_version 714847 (0.0010) [2023-12-27 10:34:25,053][20381] Updated weights for policy 1, policy_version 714304 (0.0007) [2023-12-27 10:34:25,103][20381] Updated weights for policy 1, policy_version 714314 (0.0006) [2023-12-27 10:34:25,155][20381] Updated weights for policy 1, policy_version 714324 (0.0006) [2023-12-27 10:34:25,596][20347] Updated weights for policy 0, policy_version 714857 (0.0010) [2023-12-27 10:34:25,658][20347] Updated weights for policy 0, policy_version 714867 (0.0006) [2023-12-27 10:34:25,713][20381] Updated weights for policy 1, policy_version 714334 (0.0007) [2023-12-27 10:34:25,718][20347] Updated weights for policy 0, policy_version 714877 (0.0008) [2023-12-27 10:34:25,773][20347] Updated weights for policy 0, policy_version 714887 (0.0006) [2023-12-27 10:34:25,778][20381] Updated weights for policy 1, policy_version 714344 (0.0006) [2023-12-27 10:34:25,834][20381] Updated weights for policy 1, policy_version 714354 (0.0005) [2023-12-27 10:34:26,410][20347] Updated weights for policy 0, policy_version 714897 (0.0010) [2023-12-27 10:34:26,435][20381] Updated weights for policy 1, policy_version 714364 (0.0005) [2023-12-27 10:34:26,466][20347] Updated weights for policy 0, policy_version 714907 (0.0010) [2023-12-27 10:34:26,486][20381] Updated weights for policy 1, policy_version 714374 (0.0005) [2023-12-27 10:34:26,523][20347] Updated weights for policy 0, policy_version 714917 (0.0010) [2023-12-27 10:34:26,540][20381] Updated weights for policy 1, policy_version 714384 (0.0006) [2023-12-27 10:34:27,107][20347] Updated weights for policy 0, policy_version 714927 (0.0010) [2023-12-27 10:34:27,145][20381] Updated weights for policy 1, policy_version 714394 (0.0010) [2023-12-27 10:34:27,170][20347] Updated weights for policy 0, policy_version 714937 (0.0010) [2023-12-27 10:34:27,199][20381] Updated weights for policy 1, policy_version 714404 (0.0010) [2023-12-27 10:34:27,221][20347] Updated weights for policy 0, policy_version 714947 (0.0010) [2023-12-27 10:34:27,247][20381] Updated weights for policy 1, policy_version 714414 (0.0008) [2023-12-27 10:34:27,300][20381] Updated weights for policy 1, policy_version 714424 (0.0009) [2023-12-27 10:34:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 365969408. Throughput: 0: 10191.6, 1: 10347.7. Samples: 365951016. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:27,363][19170] Avg episode reward: [(0, '855.130'), (1, '596.450')] [2023-12-27 10:34:27,878][20347] Updated weights for policy 0, policy_version 714957 (0.0010) [2023-12-27 10:34:27,922][20347] Updated weights for policy 0, policy_version 714967 (0.0010) [2023-12-27 10:34:27,964][20347] Updated weights for policy 0, policy_version 714977 (0.0010) [2023-12-27 10:34:27,969][20381] Updated weights for policy 1, policy_version 714434 (0.0010) [2023-12-27 10:34:28,024][20381] Updated weights for policy 1, policy_version 714444 (0.0010) [2023-12-27 10:34:28,073][20381] Updated weights for policy 1, policy_version 714454 (0.0008) [2023-12-27 10:34:28,669][20347] Updated weights for policy 0, policy_version 714987 (0.0010) [2023-12-27 10:34:28,703][20381] Updated weights for policy 1, policy_version 714464 (0.0010) [2023-12-27 10:34:28,727][20347] Updated weights for policy 0, policy_version 714997 (0.0010) [2023-12-27 10:34:28,761][20381] Updated weights for policy 1, policy_version 714474 (0.0011) [2023-12-27 10:34:28,784][20347] Updated weights for policy 0, policy_version 715007 (0.0010) [2023-12-27 10:34:28,816][20381] Updated weights for policy 1, policy_version 714484 (0.0010) [2023-12-27 10:34:29,432][20347] Updated weights for policy 0, policy_version 715017 (0.0010) [2023-12-27 10:34:29,455][20381] Updated weights for policy 1, policy_version 714494 (0.0007) [2023-12-27 10:34:29,492][20347] Updated weights for policy 0, policy_version 715027 (0.0007) [2023-12-27 10:34:29,509][20381] Updated weights for policy 1, policy_version 714504 (0.0007) [2023-12-27 10:34:29,551][20347] Updated weights for policy 0, policy_version 715037 (0.0005) [2023-12-27 10:34:29,557][20381] Updated weights for policy 1, policy_version 714514 (0.0010) [2023-12-27 10:34:29,610][20347] Updated weights for policy 0, policy_version 715047 (0.0007) [2023-12-27 10:34:30,244][20381] Updated weights for policy 1, policy_version 714524 (0.0010) [2023-12-27 10:34:30,253][20347] Updated weights for policy 0, policy_version 715057 (0.0007) [2023-12-27 10:34:30,299][20381] Updated weights for policy 1, policy_version 714534 (0.0010) [2023-12-27 10:34:30,301][20347] Updated weights for policy 0, policy_version 715067 (0.0005) [2023-12-27 10:34:30,353][20381] Updated weights for policy 1, policy_version 714544 (0.0010) [2023-12-27 10:34:30,356][20347] Updated weights for policy 0, policy_version 715077 (0.0007) [2023-12-27 10:34:30,980][20347] Updated weights for policy 0, policy_version 715087 (0.0005) [2023-12-27 10:34:31,043][20347] Updated weights for policy 0, policy_version 715097 (0.0007) [2023-12-27 10:34:31,047][20381] Updated weights for policy 1, policy_version 714554 (0.0010) [2023-12-27 10:34:31,097][20347] Updated weights for policy 0, policy_version 715107 (0.0007) [2023-12-27 10:34:31,109][20381] Updated weights for policy 1, policy_version 714564 (0.0011) [2023-12-27 10:34:31,170][20381] Updated weights for policy 1, policy_version 714574 (0.0010) [2023-12-27 10:34:31,230][20381] Updated weights for policy 1, policy_version 714584 (0.0007) [2023-12-27 10:34:31,728][20347] Updated weights for policy 0, policy_version 715117 (0.0007) [2023-12-27 10:34:31,792][20347] Updated weights for policy 0, policy_version 715127 (0.0008) [2023-12-27 10:34:31,854][20347] Updated weights for policy 0, policy_version 715137 (0.0007) [2023-12-27 10:34:31,903][20381] Updated weights for policy 1, policy_version 714594 (0.0007) [2023-12-27 10:34:31,963][20381] Updated weights for policy 1, policy_version 714604 (0.0007) [2023-12-27 10:34:32,030][20381] Updated weights for policy 1, policy_version 714614 (0.0007) [2023-12-27 10:34:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 366067712. Throughput: 0: 10238.3, 1: 10335.8. Samples: 366077984. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:32,363][19170] Avg episode reward: [(0, '970.800'), (1, '475.590')] [2023-12-27 10:34:32,443][20347] Updated weights for policy 0, policy_version 715147 (0.0006) [2023-12-27 10:34:32,490][20347] Updated weights for policy 0, policy_version 715157 (0.0008) [2023-12-27 10:34:32,546][20347] Updated weights for policy 0, policy_version 715167 (0.0005) [2023-12-27 10:34:32,721][20381] Updated weights for policy 1, policy_version 714624 (0.0008) [2023-12-27 10:34:32,775][20381] Updated weights for policy 1, policy_version 714634 (0.0008) [2023-12-27 10:34:32,828][20381] Updated weights for policy 1, policy_version 714644 (0.0009) [2023-12-27 10:34:33,187][20347] Updated weights for policy 0, policy_version 715177 (0.0007) [2023-12-27 10:34:33,244][20347] Updated weights for policy 0, policy_version 715187 (0.0005) [2023-12-27 10:34:33,299][20347] Updated weights for policy 0, policy_version 715197 (0.0005) [2023-12-27 10:34:33,347][20347] Updated weights for policy 0, policy_version 715207 (0.0005) [2023-12-27 10:34:33,523][20381] Updated weights for policy 1, policy_version 714654 (0.0009) [2023-12-27 10:34:33,575][20381] Updated weights for policy 1, policy_version 714664 (0.0007) [2023-12-27 10:34:33,625][20381] Updated weights for policy 1, policy_version 714674 (0.0008) [2023-12-27 10:34:33,989][20347] Updated weights for policy 0, policy_version 715217 (0.0008) [2023-12-27 10:34:34,046][20347] Updated weights for policy 0, policy_version 715227 (0.0007) [2023-12-27 10:34:34,093][20347] Updated weights for policy 0, policy_version 715237 (0.0007) [2023-12-27 10:34:34,278][20381] Updated weights for policy 1, policy_version 714684 (0.0007) [2023-12-27 10:34:34,336][20381] Updated weights for policy 1, policy_version 714694 (0.0006) [2023-12-27 10:34:34,397][20381] Updated weights for policy 1, policy_version 714704 (0.0009) [2023-12-27 10:34:34,828][20347] Updated weights for policy 0, policy_version 715247 (0.0008) [2023-12-27 10:34:34,886][20347] Updated weights for policy 0, policy_version 715257 (0.0006) [2023-12-27 10:34:34,940][20347] Updated weights for policy 0, policy_version 715267 (0.0006) [2023-12-27 10:34:35,078][20381] Updated weights for policy 1, policy_version 714714 (0.0008) [2023-12-27 10:34:35,140][20381] Updated weights for policy 1, policy_version 714724 (0.0008) [2023-12-27 10:34:35,193][20381] Updated weights for policy 1, policy_version 714734 (0.0008) [2023-12-27 10:34:35,252][20381] Updated weights for policy 1, policy_version 714744 (0.0008) [2023-12-27 10:34:35,585][20347] Updated weights for policy 0, policy_version 715277 (0.0008) [2023-12-27 10:34:35,643][20347] Updated weights for policy 0, policy_version 715287 (0.0007) [2023-12-27 10:34:35,699][20347] Updated weights for policy 0, policy_version 715297 (0.0007) [2023-12-27 10:34:35,936][20381] Updated weights for policy 1, policy_version 714754 (0.0005) [2023-12-27 10:34:35,988][20381] Updated weights for policy 1, policy_version 714764 (0.0006) [2023-12-27 10:34:36,046][20381] Updated weights for policy 1, policy_version 714774 (0.0007) [2023-12-27 10:34:36,406][20347] Updated weights for policy 0, policy_version 715307 (0.0006) [2023-12-27 10:34:36,470][20347] Updated weights for policy 0, policy_version 715317 (0.0008) [2023-12-27 10:34:36,530][20347] Updated weights for policy 0, policy_version 715327 (0.0008) [2023-12-27 10:34:36,718][20381] Updated weights for policy 1, policy_version 714784 (0.0010) [2023-12-27 10:34:36,775][20381] Updated weights for policy 1, policy_version 714794 (0.0011) [2023-12-27 10:34:36,835][20381] Updated weights for policy 1, policy_version 714804 (0.0011) [2023-12-27 10:34:37,153][20347] Updated weights for policy 0, policy_version 715337 (0.0007) [2023-12-27 10:34:37,205][20347] Updated weights for policy 0, policy_version 715347 (0.0006) [2023-12-27 10:34:37,254][20347] Updated weights for policy 0, policy_version 715357 (0.0005) [2023-12-27 10:34:37,302][20347] Updated weights for policy 0, policy_version 715367 (0.0005) [2023-12-27 10:34:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 366174208. Throughput: 0: 10215.8, 1: 10341.6. Samples: 366140228. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:37,363][19170] Avg episode reward: [(0, '980.510'), (1, '598.910')] [2023-12-27 10:34:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000714808_183017472.pth... [2023-12-27 10:34:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000715368_183156736.pth... [2023-12-27 10:34:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000714152_182845440.pth [2023-12-27 10:34:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000713592_182706176.pth [2023-12-27 10:34:37,503][20381] Updated weights for policy 1, policy_version 714814 (0.0008) [2023-12-27 10:34:37,572][20381] Updated weights for policy 1, policy_version 714824 (0.0006) [2023-12-27 10:34:37,623][20381] Updated weights for policy 1, policy_version 714834 (0.0005) [2023-12-27 10:34:37,948][20347] Updated weights for policy 0, policy_version 715377 (0.0007) [2023-12-27 10:34:37,998][20347] Updated weights for policy 0, policy_version 715387 (0.0007) [2023-12-27 10:34:38,057][20347] Updated weights for policy 0, policy_version 715397 (0.0007) [2023-12-27 10:34:38,177][20381] Updated weights for policy 1, policy_version 714844 (0.0007) [2023-12-27 10:34:38,235][20381] Updated weights for policy 1, policy_version 714854 (0.0010) [2023-12-27 10:34:38,284][20381] Updated weights for policy 1, policy_version 714864 (0.0008) [2023-12-27 10:34:38,706][20347] Updated weights for policy 0, policy_version 715407 (0.0007) [2023-12-27 10:34:38,761][20347] Updated weights for policy 0, policy_version 715417 (0.0008) [2023-12-27 10:34:38,821][20347] Updated weights for policy 0, policy_version 715427 (0.0008) [2023-12-27 10:34:38,979][20381] Updated weights for policy 1, policy_version 714874 (0.0009) [2023-12-27 10:34:39,039][20381] Updated weights for policy 1, policy_version 714884 (0.0006) [2023-12-27 10:34:39,095][20381] Updated weights for policy 1, policy_version 714894 (0.0007) [2023-12-27 10:34:39,145][20381] Updated weights for policy 1, policy_version 714904 (0.0008) [2023-12-27 10:34:39,592][20347] Updated weights for policy 0, policy_version 715437 (0.0008) [2023-12-27 10:34:39,654][20347] Updated weights for policy 0, policy_version 715447 (0.0007) [2023-12-27 10:34:39,713][20347] Updated weights for policy 0, policy_version 715457 (0.0008) [2023-12-27 10:34:39,933][20381] Updated weights for policy 1, policy_version 714914 (0.0011) [2023-12-27 10:34:39,997][20381] Updated weights for policy 1, policy_version 714924 (0.0009) [2023-12-27 10:34:40,060][20381] Updated weights for policy 1, policy_version 714934 (0.0008) [2023-12-27 10:34:40,440][20347] Updated weights for policy 0, policy_version 715467 (0.0008) [2023-12-27 10:34:40,493][20347] Updated weights for policy 0, policy_version 715477 (0.0007) [2023-12-27 10:34:40,546][20347] Updated weights for policy 0, policy_version 715487 (0.0005) [2023-12-27 10:34:40,737][20381] Updated weights for policy 1, policy_version 714944 (0.0010) [2023-12-27 10:34:40,793][20381] Updated weights for policy 1, policy_version 714954 (0.0011) [2023-12-27 10:34:40,852][20381] Updated weights for policy 1, policy_version 714964 (0.0011) [2023-12-27 10:34:41,226][20347] Updated weights for policy 0, policy_version 715497 (0.0006) [2023-12-27 10:34:41,287][20347] Updated weights for policy 0, policy_version 715507 (0.0008) [2023-12-27 10:34:41,347][20347] Updated weights for policy 0, policy_version 715517 (0.0007) [2023-12-27 10:34:41,422][20347] Updated weights for policy 0, policy_version 715527 (0.0009) [2023-12-27 10:34:41,662][20381] Updated weights for policy 1, policy_version 714974 (0.0009) [2023-12-27 10:34:41,724][20381] Updated weights for policy 1, policy_version 714984 (0.0007) [2023-12-27 10:34:41,794][20381] Updated weights for policy 1, policy_version 714994 (0.0008) [2023-12-27 10:34:42,098][20347] Updated weights for policy 0, policy_version 715537 (0.0005) [2023-12-27 10:34:42,156][20347] Updated weights for policy 0, policy_version 715547 (0.0006) [2023-12-27 10:34:42,214][20347] Updated weights for policy 0, policy_version 715557 (0.0005) [2023-12-27 10:34:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 366272512. Throughput: 0: 10227.9, 1: 10329.8. Samples: 366262568. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:42,363][19170] Avg episode reward: [(0, '941.600'), (1, '571.840')] [2023-12-27 10:34:42,489][20381] Updated weights for policy 1, policy_version 715004 (0.0008) [2023-12-27 10:34:42,556][20381] Updated weights for policy 1, policy_version 715014 (0.0008) [2023-12-27 10:34:42,624][20381] Updated weights for policy 1, policy_version 715024 (0.0007) [2023-12-27 10:34:42,818][20347] Updated weights for policy 0, policy_version 715567 (0.0006) [2023-12-27 10:34:42,883][20347] Updated weights for policy 0, policy_version 715577 (0.0005) [2023-12-27 10:34:42,943][20347] Updated weights for policy 0, policy_version 715587 (0.0005) [2023-12-27 10:34:43,227][20381] Updated weights for policy 1, policy_version 715034 (0.0008) [2023-12-27 10:34:43,297][20381] Updated weights for policy 1, policy_version 715044 (0.0007) [2023-12-27 10:34:43,362][20381] Updated weights for policy 1, policy_version 715054 (0.0008) [2023-12-27 10:34:43,414][20381] Updated weights for policy 1, policy_version 715064 (0.0008) [2023-12-27 10:34:43,510][20347] Updated weights for policy 0, policy_version 715597 (0.0006) [2023-12-27 10:34:43,564][20347] Updated weights for policy 0, policy_version 715607 (0.0006) [2023-12-27 10:34:43,626][20347] Updated weights for policy 0, policy_version 715617 (0.0006) [2023-12-27 10:34:43,983][20381] Updated weights for policy 1, policy_version 715074 (0.0005) [2023-12-27 10:34:44,045][20381] Updated weights for policy 1, policy_version 715084 (0.0005) [2023-12-27 10:34:44,098][20381] Updated weights for policy 1, policy_version 715094 (0.0008) [2023-12-27 10:34:44,293][20347] Updated weights for policy 0, policy_version 715627 (0.0007) [2023-12-27 10:34:44,345][20347] Updated weights for policy 0, policy_version 715637 (0.0006) [2023-12-27 10:34:44,399][20347] Updated weights for policy 0, policy_version 715647 (0.0007) [2023-12-27 10:34:44,760][20381] Updated weights for policy 1, policy_version 715104 (0.0010) [2023-12-27 10:34:44,823][20381] Updated weights for policy 1, policy_version 715114 (0.0009) [2023-12-27 10:34:44,886][20381] Updated weights for policy 1, policy_version 715124 (0.0011) [2023-12-27 10:34:45,070][20347] Updated weights for policy 0, policy_version 715657 (0.0007) [2023-12-27 10:34:45,131][20347] Updated weights for policy 0, policy_version 715667 (0.0006) [2023-12-27 10:34:45,196][20347] Updated weights for policy 0, policy_version 715677 (0.0007) [2023-12-27 10:34:45,256][20347] Updated weights for policy 0, policy_version 715687 (0.0007) [2023-12-27 10:34:45,564][20381] Updated weights for policy 1, policy_version 715134 (0.0009) [2023-12-27 10:34:45,625][20381] Updated weights for policy 1, policy_version 715144 (0.0010) [2023-12-27 10:34:45,678][20381] Updated weights for policy 1, policy_version 715154 (0.0011) [2023-12-27 10:34:45,894][20347] Updated weights for policy 0, policy_version 715697 (0.0007) [2023-12-27 10:34:45,940][20347] Updated weights for policy 0, policy_version 715707 (0.0009) [2023-12-27 10:34:45,999][20347] Updated weights for policy 0, policy_version 715717 (0.0007) [2023-12-27 10:34:46,310][20381] Updated weights for policy 1, policy_version 715164 (0.0011) [2023-12-27 10:34:46,368][20381] Updated weights for policy 1, policy_version 715174 (0.0010) [2023-12-27 10:34:46,420][20381] Updated weights for policy 1, policy_version 715184 (0.0007) [2023-12-27 10:34:46,734][20347] Updated weights for policy 0, policy_version 715727 (0.0008) [2023-12-27 10:34:46,780][20347] Updated weights for policy 0, policy_version 715737 (0.0008) [2023-12-27 10:34:46,825][20347] Updated weights for policy 0, policy_version 715747 (0.0007) [2023-12-27 10:34:47,018][20381] Updated weights for policy 1, policy_version 715194 (0.0007) [2023-12-27 10:34:47,084][20381] Updated weights for policy 1, policy_version 715204 (0.0007) [2023-12-27 10:34:47,145][20381] Updated weights for policy 1, policy_version 715214 (0.0007) [2023-12-27 10:34:47,206][20381] Updated weights for policy 1, policy_version 715224 (0.0008) [2023-12-27 10:34:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 366379008. Throughput: 0: 10354.0, 1: 10392.9. Samples: 366387176. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:47,363][19170] Avg episode reward: [(0, '932.080'), (1, '550.710')] [2023-12-27 10:34:47,552][20347] Updated weights for policy 0, policy_version 715757 (0.0007) [2023-12-27 10:34:47,608][20347] Updated weights for policy 0, policy_version 715767 (0.0007) [2023-12-27 10:34:47,663][20347] Updated weights for policy 0, policy_version 715777 (0.0007) [2023-12-27 10:34:47,681][20291] KL-divergence is very high: 550.4162 [2023-12-27 10:34:47,855][20381] Updated weights for policy 1, policy_version 715234 (0.0007) [2023-12-27 10:34:47,900][20381] Updated weights for policy 1, policy_version 715244 (0.0008) [2023-12-27 10:34:47,949][20381] Updated weights for policy 1, policy_version 715254 (0.0008) [2023-12-27 10:34:48,337][20291] KL-divergence is very high: 186.8696 [2023-12-27 10:34:48,360][20347] Updated weights for policy 0, policy_version 715787 (0.0008) [2023-12-27 10:34:48,390][20291] KL-divergence is very high: 229.4255 [2023-12-27 10:34:48,419][20347] Updated weights for policy 0, policy_version 715797 (0.0008) [2023-12-27 10:34:48,437][20291] KL-divergence is very high: 136.7599 [2023-12-27 10:34:48,475][20347] Updated weights for policy 0, policy_version 715807 (0.0007) [2023-12-27 10:34:48,481][20291] KL-divergence is very high: 334.9132 [2023-12-27 10:34:48,699][20381] Updated weights for policy 1, policy_version 715264 (0.0007) [2023-12-27 10:34:48,761][20381] Updated weights for policy 1, policy_version 715274 (0.0008) [2023-12-27 10:34:48,821][20381] Updated weights for policy 1, policy_version 715284 (0.0008) [2023-12-27 10:34:49,142][20347] Updated weights for policy 0, policy_version 715817 (0.0009) [2023-12-27 10:34:49,200][20347] Updated weights for policy 0, policy_version 715827 (0.0010) [2023-12-27 10:34:49,264][20347] Updated weights for policy 0, policy_version 715837 (0.0010) [2023-12-27 10:34:49,325][20347] Updated weights for policy 0, policy_version 715847 (0.0008) [2023-12-27 10:34:49,564][20381] Updated weights for policy 1, policy_version 715294 (0.0008) [2023-12-27 10:34:49,618][20381] Updated weights for policy 1, policy_version 715304 (0.0006) [2023-12-27 10:34:49,665][20381] Updated weights for policy 1, policy_version 715314 (0.0005) [2023-12-27 10:34:50,091][20347] Updated weights for policy 0, policy_version 715857 (0.0006) [2023-12-27 10:34:50,132][20291] KL-divergence is very high: 160.0940 [2023-12-27 10:34:50,151][20347] Updated weights for policy 0, policy_version 715867 (0.0008) [2023-12-27 10:34:50,212][20347] Updated weights for policy 0, policy_version 715877 (0.0008) [2023-12-27 10:34:50,288][20381] Updated weights for policy 1, policy_version 715324 (0.0006) [2023-12-27 10:34:50,348][20381] Updated weights for policy 1, policy_version 715334 (0.0008) [2023-12-27 10:34:50,409][20381] Updated weights for policy 1, policy_version 715344 (0.0006) [2023-12-27 10:34:50,903][20347] Updated weights for policy 0, policy_version 715887 (0.0007) [2023-12-27 10:34:50,966][20347] Updated weights for policy 0, policy_version 715897 (0.0006) [2023-12-27 10:34:50,996][20291] KL-divergence is very high: 106.4239 [2023-12-27 10:34:51,037][20347] Updated weights for policy 0, policy_version 715907 (0.0006) [2023-12-27 10:34:51,100][20381] Updated weights for policy 1, policy_version 715354 (0.0007) [2023-12-27 10:34:51,172][20381] Updated weights for policy 1, policy_version 715364 (0.0009) [2023-12-27 10:34:51,239][20381] Updated weights for policy 1, policy_version 715374 (0.0011) [2023-12-27 10:34:51,305][20381] Updated weights for policy 1, policy_version 715384 (0.0010) [2023-12-27 10:34:51,748][20347] Updated weights for policy 0, policy_version 715917 (0.0007) [2023-12-27 10:34:51,805][20347] Updated weights for policy 0, policy_version 715927 (0.0009) [2023-12-27 10:34:51,868][20347] Updated weights for policy 0, policy_version 715937 (0.0009) [2023-12-27 10:34:52,065][20381] Updated weights for policy 1, policy_version 715394 (0.0009) [2023-12-27 10:34:52,121][20381] Updated weights for policy 1, policy_version 715404 (0.0009) [2023-12-27 10:34:52,174][20381] Updated weights for policy 1, policy_version 715414 (0.0010) [2023-12-27 10:34:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 366477312. Throughput: 0: 10335.5, 1: 10347.0. Samples: 366447384. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:52,363][19170] Avg episode reward: [(0, '894.430'), (1, '542.040')] [2023-12-27 10:34:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000715416_183173120.pth... [2023-12-27 10:34:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000715944_183304192.pth... [2023-12-27 10:34:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000714200_182861824.pth [2023-12-27 10:34:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000714728_182992896.pth [2023-12-27 10:34:52,633][20347] Updated weights for policy 0, policy_version 715947 (0.0009) [2023-12-27 10:34:52,698][20347] Updated weights for policy 0, policy_version 715957 (0.0009) [2023-12-27 10:34:52,758][20347] Updated weights for policy 0, policy_version 715967 (0.0010) [2023-12-27 10:34:52,873][20381] Updated weights for policy 1, policy_version 715424 (0.0008) [2023-12-27 10:34:52,932][20381] Updated weights for policy 1, policy_version 715434 (0.0007) [2023-12-27 10:34:52,990][20381] Updated weights for policy 1, policy_version 715444 (0.0008) [2023-12-27 10:34:53,412][20347] Updated weights for policy 0, policy_version 715977 (0.0010) [2023-12-27 10:34:53,459][20347] Updated weights for policy 0, policy_version 715987 (0.0008) [2023-12-27 10:34:53,505][20347] Updated weights for policy 0, policy_version 715997 (0.0005) [2023-12-27 10:34:53,559][20347] Updated weights for policy 0, policy_version 716007 (0.0007) [2023-12-27 10:34:53,595][20381] Updated weights for policy 1, policy_version 715454 (0.0010) [2023-12-27 10:34:53,649][20381] Updated weights for policy 1, policy_version 715464 (0.0010) [2023-12-27 10:34:53,714][20381] Updated weights for policy 1, policy_version 715474 (0.0008) [2023-12-27 10:34:54,221][20347] Updated weights for policy 0, policy_version 716017 (0.0007) [2023-12-27 10:34:54,279][20347] Updated weights for policy 0, policy_version 716027 (0.0010) [2023-12-27 10:34:54,339][20347] Updated weights for policy 0, policy_version 716037 (0.0010) [2023-12-27 10:34:54,342][20381] Updated weights for policy 1, policy_version 715484 (0.0009) [2023-12-27 10:34:54,396][20381] Updated weights for policy 1, policy_version 715494 (0.0010) [2023-12-27 10:34:54,450][20381] Updated weights for policy 1, policy_version 715504 (0.0011) [2023-12-27 10:34:54,968][20347] Updated weights for policy 0, policy_version 716047 (0.0010) [2023-12-27 10:34:55,018][20347] Updated weights for policy 0, policy_version 716057 (0.0007) [2023-12-27 10:34:55,067][20347] Updated weights for policy 0, policy_version 716067 (0.0009) [2023-12-27 10:34:55,089][20381] Updated weights for policy 1, policy_version 715514 (0.0011) [2023-12-27 10:34:55,146][20381] Updated weights for policy 1, policy_version 715524 (0.0010) [2023-12-27 10:34:55,200][20381] Updated weights for policy 1, policy_version 715534 (0.0010) [2023-12-27 10:34:55,260][20381] Updated weights for policy 1, policy_version 715544 (0.0010) [2023-12-27 10:34:55,704][20347] Updated weights for policy 0, policy_version 716077 (0.0010) [2023-12-27 10:34:55,768][20347] Updated weights for policy 0, policy_version 716087 (0.0010) [2023-12-27 10:34:55,815][20347] Updated weights for policy 0, policy_version 716097 (0.0007) [2023-12-27 10:34:55,874][20381] Updated weights for policy 1, policy_version 715554 (0.0006) [2023-12-27 10:34:55,938][20381] Updated weights for policy 1, policy_version 715564 (0.0007) [2023-12-27 10:34:55,996][20381] Updated weights for policy 1, policy_version 715574 (0.0009) [2023-12-27 10:34:56,448][20347] Updated weights for policy 0, policy_version 716107 (0.0005) [2023-12-27 10:34:56,499][20347] Updated weights for policy 0, policy_version 716117 (0.0006) [2023-12-27 10:34:56,554][20347] Updated weights for policy 0, policy_version 716127 (0.0010) [2023-12-27 10:34:56,626][20381] Updated weights for policy 1, policy_version 715584 (0.0010) [2023-12-27 10:34:56,679][20381] Updated weights for policy 1, policy_version 715594 (0.0008) [2023-12-27 10:34:56,737][20381] Updated weights for policy 1, policy_version 715604 (0.0010) [2023-12-27 10:34:57,138][20347] Updated weights for policy 0, policy_version 716137 (0.0008) [2023-12-27 10:34:57,195][20347] Updated weights for policy 0, policy_version 716147 (0.0010) [2023-12-27 10:34:57,252][20347] Updated weights for policy 0, policy_version 716157 (0.0010) [2023-12-27 10:34:57,307][20347] Updated weights for policy 0, policy_version 716167 (0.0010) [2023-12-27 10:34:57,332][20381] Updated weights for policy 1, policy_version 715614 (0.0007) [2023-12-27 10:34:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 366583808. Throughput: 0: 10314.4, 1: 10347.3. Samples: 366570504. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:34:57,364][19170] Avg episode reward: [(0, '847.120'), (1, '436.330')] [2023-12-27 10:34:57,386][20381] Updated weights for policy 1, policy_version 715624 (0.0007) [2023-12-27 10:34:57,442][20381] Updated weights for policy 1, policy_version 715634 (0.0005) [2023-12-27 10:34:57,889][20347] Updated weights for policy 0, policy_version 716177 (0.0010) [2023-12-27 10:34:57,947][20347] Updated weights for policy 0, policy_version 716187 (0.0010) [2023-12-27 10:34:58,010][20347] Updated weights for policy 0, policy_version 716197 (0.0010) [2023-12-27 10:34:58,029][20381] Updated weights for policy 1, policy_version 715644 (0.0008) [2023-12-27 10:34:58,074][20381] Updated weights for policy 1, policy_version 715654 (0.0006) [2023-12-27 10:34:58,120][20381] Updated weights for policy 1, policy_version 715664 (0.0005) [2023-12-27 10:34:58,722][20347] Updated weights for policy 0, policy_version 716207 (0.0010) [2023-12-27 10:34:58,723][20291] KL-divergence is very high: 103.1638 [2023-12-27 10:34:58,785][20347] Updated weights for policy 0, policy_version 716217 (0.0009) [2023-12-27 10:34:58,851][20347] Updated weights for policy 0, policy_version 716227 (0.0010) [2023-12-27 10:34:58,885][20381] Updated weights for policy 1, policy_version 715674 (0.0006) [2023-12-27 10:34:58,947][20381] Updated weights for policy 1, policy_version 715684 (0.0007) [2023-12-27 10:34:59,000][20381] Updated weights for policy 1, policy_version 715694 (0.0007) [2023-12-27 10:34:59,064][20381] Updated weights for policy 1, policy_version 715704 (0.0007) [2023-12-27 10:34:59,490][20347] Updated weights for policy 0, policy_version 716237 (0.0008) [2023-12-27 10:34:59,543][20347] Updated weights for policy 0, policy_version 716247 (0.0007) [2023-12-27 10:34:59,600][20347] Updated weights for policy 0, policy_version 716257 (0.0008) [2023-12-27 10:34:59,737][20381] Updated weights for policy 1, policy_version 715714 (0.0008) [2023-12-27 10:34:59,796][20381] Updated weights for policy 1, policy_version 715724 (0.0007) [2023-12-27 10:34:59,862][20381] Updated weights for policy 1, policy_version 715734 (0.0008) [2023-12-27 10:35:00,299][20347] Updated weights for policy 0, policy_version 716267 (0.0006) [2023-12-27 10:35:00,360][20347] Updated weights for policy 0, policy_version 716277 (0.0006) [2023-12-27 10:35:00,422][20347] Updated weights for policy 0, policy_version 716287 (0.0006) [2023-12-27 10:35:00,525][20381] Updated weights for policy 1, policy_version 715744 (0.0008) [2023-12-27 10:35:00,586][20381] Updated weights for policy 1, policy_version 715754 (0.0009) [2023-12-27 10:35:00,633][20381] Updated weights for policy 1, policy_version 715764 (0.0007) [2023-12-27 10:35:00,976][20347] Updated weights for policy 0, policy_version 716297 (0.0008) [2023-12-27 10:35:01,034][20347] Updated weights for policy 0, policy_version 716307 (0.0007) [2023-12-27 10:35:01,101][20347] Updated weights for policy 0, policy_version 716317 (0.0008) [2023-12-27 10:35:01,178][20347] Updated weights for policy 0, policy_version 716327 (0.0008) [2023-12-27 10:35:01,210][20381] Updated weights for policy 1, policy_version 715774 (0.0006) [2023-12-27 10:35:01,273][20381] Updated weights for policy 1, policy_version 715784 (0.0008) [2023-12-27 10:35:01,333][20381] Updated weights for policy 1, policy_version 715794 (0.0009) [2023-12-27 10:35:01,854][20347] Updated weights for policy 0, policy_version 716337 (0.0007) [2023-12-27 10:35:01,909][20347] Updated weights for policy 0, policy_version 716348 (0.0008) [2023-12-27 10:35:01,958][20347] Updated weights for policy 0, policy_version 716358 (0.0010) [2023-12-27 10:35:02,050][20381] Updated weights for policy 1, policy_version 715804 (0.0008) [2023-12-27 10:35:02,110][20381] Updated weights for policy 1, policy_version 715814 (0.0008) [2023-12-27 10:35:02,173][20381] Updated weights for policy 1, policy_version 715824 (0.0008) [2023-12-27 10:35:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 366690304. Throughput: 0: 10352.6, 1: 10390.2. Samples: 366697660. Policy #0 lag: (min: 27.0, avg: 32.6, max: 59.0) [2023-12-27 10:35:02,363][19170] Avg episode reward: [(0, '864.990'), (1, '229.080')] [2023-12-27 10:35:02,615][20347] Updated weights for policy 0, policy_version 716368 (0.0007) [2023-12-27 10:35:02,667][20347] Updated weights for policy 0, policy_version 716378 (0.0005) [2023-12-27 10:35:02,720][20347] Updated weights for policy 0, policy_version 716388 (0.0005) [2023-12-27 10:35:02,881][20381] Updated weights for policy 1, policy_version 715834 (0.0008) [2023-12-27 10:35:02,930][20381] Updated weights for policy 1, policy_version 715844 (0.0008) [2023-12-27 10:35:02,988][20381] Updated weights for policy 1, policy_version 715854 (0.0008) [2023-12-27 10:35:03,039][20381] Updated weights for policy 1, policy_version 715864 (0.0007) [2023-12-27 10:35:03,377][20347] Updated weights for policy 0, policy_version 716398 (0.0007) [2023-12-27 10:35:03,421][20347] Updated weights for policy 0, policy_version 716408 (0.0008) [2023-12-27 10:35:03,475][20347] Updated weights for policy 0, policy_version 716418 (0.0007) [2023-12-27 10:35:03,716][20381] Updated weights for policy 1, policy_version 715874 (0.0005) [2023-12-27 10:35:03,774][20381] Updated weights for policy 1, policy_version 715884 (0.0009) [2023-12-27 10:35:03,822][20381] Updated weights for policy 1, policy_version 715894 (0.0010) [2023-12-27 10:35:04,162][20347] Updated weights for policy 0, policy_version 716428 (0.0006) [2023-12-27 10:35:04,225][20347] Updated weights for policy 0, policy_version 716438 (0.0009) [2023-12-27 10:35:04,281][20347] Updated weights for policy 0, policy_version 716448 (0.0007) [2023-12-27 10:35:04,476][20381] Updated weights for policy 1, policy_version 715904 (0.0010) [2023-12-27 10:35:04,528][20381] Updated weights for policy 1, policy_version 715914 (0.0010) [2023-12-27 10:35:04,580][20381] Updated weights for policy 1, policy_version 715924 (0.0010) [2023-12-27 10:35:05,000][20347] Updated weights for policy 0, policy_version 716458 (0.0008) [2023-12-27 10:35:05,059][20347] Updated weights for policy 0, policy_version 716468 (0.0007) [2023-12-27 10:35:05,116][20347] Updated weights for policy 0, policy_version 716478 (0.0008) [2023-12-27 10:35:05,166][20347] Updated weights for policy 0, policy_version 716488 (0.0007) [2023-12-27 10:35:05,274][20381] Updated weights for policy 1, policy_version 715934 (0.0008) [2023-12-27 10:35:05,338][20381] Updated weights for policy 1, policy_version 715944 (0.0006) [2023-12-27 10:35:05,399][20381] Updated weights for policy 1, policy_version 715954 (0.0005) [2023-12-27 10:35:05,898][20381] Updated weights for policy 1, policy_version 715964 (0.0005) [2023-12-27 10:35:05,921][20347] Updated weights for policy 0, policy_version 716498 (0.0008) [2023-12-27 10:35:05,954][20381] Updated weights for policy 1, policy_version 715974 (0.0005) [2023-12-27 10:35:05,977][20347] Updated weights for policy 0, policy_version 716508 (0.0007) [2023-12-27 10:35:06,006][20381] Updated weights for policy 1, policy_version 715984 (0.0006) [2023-12-27 10:35:06,026][20347] Updated weights for policy 0, policy_version 716518 (0.0006) [2023-12-27 10:35:06,664][20381] Updated weights for policy 1, policy_version 715994 (0.0006) [2023-12-27 10:35:06,665][20292] KL-divergence is very high: 101.7954 [2023-12-27 10:35:06,725][20381] Updated weights for policy 1, policy_version 716004 (0.0008) [2023-12-27 10:35:06,769][20347] Updated weights for policy 0, policy_version 716528 (0.0006) [2023-12-27 10:35:06,786][20381] Updated weights for policy 1, policy_version 716014 (0.0006) [2023-12-27 10:35:06,826][20347] Updated weights for policy 0, policy_version 716538 (0.0006) [2023-12-27 10:35:06,839][20381] Updated weights for policy 1, policy_version 716024 (0.0006) [2023-12-27 10:35:06,874][20347] Updated weights for policy 0, policy_version 716548 (0.0007) [2023-12-27 10:35:07,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 366788608. Throughput: 0: 10353.9, 1: 10398.5. Samples: 366759188. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:07,363][19170] Avg episode reward: [(0, '961.100'), (1, '440.590')] [2023-12-27 10:35:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000716024_183328768.pth... [2023-12-27 10:35:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000716552_183459840.pth... [2023-12-27 10:35:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000714808_183017472.pth [2023-12-27 10:35:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000715368_183156736.pth [2023-12-27 10:35:07,493][20381] Updated weights for policy 1, policy_version 716034 (0.0005) [2023-12-27 10:35:07,546][20381] Updated weights for policy 1, policy_version 716044 (0.0006) [2023-12-27 10:35:07,603][20347] Updated weights for policy 0, policy_version 716558 (0.0007) [2023-12-27 10:35:07,605][20381] Updated weights for policy 1, policy_version 716054 (0.0006) [2023-12-27 10:35:07,659][20347] Updated weights for policy 0, policy_version 716568 (0.0005) [2023-12-27 10:35:07,717][20347] Updated weights for policy 0, policy_version 716578 (0.0005) [2023-12-27 10:35:08,218][20381] Updated weights for policy 1, policy_version 716064 (0.0006) [2023-12-27 10:35:08,282][20381] Updated weights for policy 1, policy_version 716074 (0.0006) [2023-12-27 10:35:08,344][20381] Updated weights for policy 1, policy_version 716084 (0.0006) [2023-12-27 10:35:08,382][20347] Updated weights for policy 0, policy_version 716588 (0.0006) [2023-12-27 10:35:08,444][20347] Updated weights for policy 0, policy_version 716598 (0.0008) [2023-12-27 10:35:08,494][20347] Updated weights for policy 0, policy_version 716608 (0.0008) [2023-12-27 10:35:09,004][20381] Updated weights for policy 1, policy_version 716094 (0.0008) [2023-12-27 10:35:09,049][20381] Updated weights for policy 1, policy_version 716104 (0.0007) [2023-12-27 10:35:09,101][20381] Updated weights for policy 1, policy_version 716114 (0.0007) [2023-12-27 10:35:09,230][20347] Updated weights for policy 0, policy_version 716618 (0.0008) [2023-12-27 10:35:09,289][20347] Updated weights for policy 0, policy_version 716628 (0.0007) [2023-12-27 10:35:09,351][20347] Updated weights for policy 0, policy_version 716638 (0.0008) [2023-12-27 10:35:09,415][20347] Updated weights for policy 0, policy_version 716648 (0.0008) [2023-12-27 10:35:09,854][20381] Updated weights for policy 1, policy_version 716124 (0.0009) [2023-12-27 10:35:09,910][20381] Updated weights for policy 1, policy_version 716134 (0.0008) [2023-12-27 10:35:09,968][20381] Updated weights for policy 1, policy_version 716144 (0.0008) [2023-12-27 10:35:10,148][20347] Updated weights for policy 0, policy_version 716658 (0.0009) [2023-12-27 10:35:10,202][20347] Updated weights for policy 0, policy_version 716668 (0.0010) [2023-12-27 10:35:10,252][20347] Updated weights for policy 0, policy_version 716678 (0.0011) [2023-12-27 10:35:10,729][20381] Updated weights for policy 1, policy_version 716154 (0.0007) [2023-12-27 10:35:10,791][20381] Updated weights for policy 1, policy_version 716164 (0.0007) [2023-12-27 10:35:10,849][20381] Updated weights for policy 1, policy_version 716174 (0.0009) [2023-12-27 10:35:10,876][20347] Updated weights for policy 0, policy_version 716688 (0.0008) [2023-12-27 10:35:10,908][20381] Updated weights for policy 1, policy_version 716184 (0.0008) [2023-12-27 10:35:10,934][20347] Updated weights for policy 0, policy_version 716698 (0.0005) [2023-12-27 10:35:10,997][20347] Updated weights for policy 0, policy_version 716708 (0.0009) [2023-12-27 10:35:11,688][20381] Updated weights for policy 1, policy_version 716194 (0.0008) [2023-12-27 10:35:11,695][20347] Updated weights for policy 0, policy_version 716718 (0.0007) [2023-12-27 10:35:11,755][20381] Updated weights for policy 1, policy_version 716204 (0.0008) [2023-12-27 10:35:11,764][20347] Updated weights for policy 0, policy_version 716728 (0.0007) [2023-12-27 10:35:11,814][20381] Updated weights for policy 1, policy_version 716214 (0.0008) [2023-12-27 10:35:11,826][20347] Updated weights for policy 0, policy_version 716738 (0.0008) [2023-12-27 10:35:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 366886912. Throughput: 0: 10301.5, 1: 10347.5. Samples: 366880220. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:12,363][19170] Avg episode reward: [(0, '951.470'), (1, '374.890')] [2023-12-27 10:35:12,471][20347] Updated weights for policy 0, policy_version 716748 (0.0006) [2023-12-27 10:35:12,529][20347] Updated weights for policy 0, policy_version 716758 (0.0006) [2023-12-27 10:35:12,531][20381] Updated weights for policy 1, policy_version 716224 (0.0006) [2023-12-27 10:35:12,582][20347] Updated weights for policy 0, policy_version 716768 (0.0005) [2023-12-27 10:35:12,588][20381] Updated weights for policy 1, policy_version 716234 (0.0006) [2023-12-27 10:35:12,645][20381] Updated weights for policy 1, policy_version 716244 (0.0006) [2023-12-27 10:35:13,251][20347] Updated weights for policy 0, policy_version 716778 (0.0008) [2023-12-27 10:35:13,299][20347] Updated weights for policy 0, policy_version 716788 (0.0007) [2023-12-27 10:35:13,343][20381] Updated weights for policy 1, policy_version 716254 (0.0006) [2023-12-27 10:35:13,360][20347] Updated weights for policy 0, policy_version 716798 (0.0007) [2023-12-27 10:35:13,395][20381] Updated weights for policy 1, policy_version 716264 (0.0005) [2023-12-27 10:35:13,419][20347] Updated weights for policy 0, policy_version 716808 (0.0006) [2023-12-27 10:35:13,453][20381] Updated weights for policy 1, policy_version 716274 (0.0006) [2023-12-27 10:35:14,069][20347] Updated weights for policy 0, policy_version 716818 (0.0005) [2023-12-27 10:35:14,134][20347] Updated weights for policy 0, policy_version 716828 (0.0007) [2023-12-27 10:35:14,152][20381] Updated weights for policy 1, policy_version 716284 (0.0005) [2023-12-27 10:35:14,185][20347] Updated weights for policy 0, policy_version 716838 (0.0006) [2023-12-27 10:35:14,204][20381] Updated weights for policy 1, policy_version 716294 (0.0007) [2023-12-27 10:35:14,255][20381] Updated weights for policy 1, policy_version 716304 (0.0008) [2023-12-27 10:35:14,761][20347] Updated weights for policy 0, policy_version 716848 (0.0007) [2023-12-27 10:35:14,821][20347] Updated weights for policy 0, policy_version 716858 (0.0008) [2023-12-27 10:35:14,878][20347] Updated weights for policy 0, policy_version 716868 (0.0008) [2023-12-27 10:35:14,998][20381] Updated weights for policy 1, policy_version 716314 (0.0008) [2023-12-27 10:35:15,055][20381] Updated weights for policy 1, policy_version 716324 (0.0007) [2023-12-27 10:35:15,112][20381] Updated weights for policy 1, policy_version 716334 (0.0008) [2023-12-27 10:35:15,169][20381] Updated weights for policy 1, policy_version 716344 (0.0008) [2023-12-27 10:35:15,512][20347] Updated weights for policy 0, policy_version 716878 (0.0008) [2023-12-27 10:35:15,574][20347] Updated weights for policy 0, policy_version 716888 (0.0008) [2023-12-27 10:35:15,629][20347] Updated weights for policy 0, policy_version 716898 (0.0008) [2023-12-27 10:35:15,790][20381] Updated weights for policy 1, policy_version 716354 (0.0008) [2023-12-27 10:35:15,845][20381] Updated weights for policy 1, policy_version 716364 (0.0007) [2023-12-27 10:35:15,906][20381] Updated weights for policy 1, policy_version 716374 (0.0006) [2023-12-27 10:35:16,372][20347] Updated weights for policy 0, policy_version 716908 (0.0009) [2023-12-27 10:35:16,426][20347] Updated weights for policy 0, policy_version 716918 (0.0007) [2023-12-27 10:35:16,474][20347] Updated weights for policy 0, policy_version 716928 (0.0006) [2023-12-27 10:35:16,475][20381] Updated weights for policy 1, policy_version 716384 (0.0006) [2023-12-27 10:35:16,529][20381] Updated weights for policy 1, policy_version 716394 (0.0008) [2023-12-27 10:35:16,585][20381] Updated weights for policy 1, policy_version 716404 (0.0006) [2023-12-27 10:35:17,137][20347] Updated weights for policy 0, policy_version 716938 (0.0006) [2023-12-27 10:35:17,200][20347] Updated weights for policy 0, policy_version 716948 (0.0005) [2023-12-27 10:35:17,262][20347] Updated weights for policy 0, policy_version 716958 (0.0005) [2023-12-27 10:35:17,293][20381] Updated weights for policy 1, policy_version 716414 (0.0007) [2023-12-27 10:35:17,322][20347] Updated weights for policy 0, policy_version 716968 (0.0006) [2023-12-27 10:35:17,349][20381] Updated weights for policy 1, policy_version 716424 (0.0006) [2023-12-27 10:35:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 366993408. Throughput: 0: 10267.5, 1: 10303.8. Samples: 367003692. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:17,363][19170] Avg episode reward: [(0, '941.710'), (1, '464.530')] [2023-12-27 10:35:17,409][20381] Updated weights for policy 1, policy_version 716434 (0.0008) [2023-12-27 10:35:17,965][20347] Updated weights for policy 0, policy_version 716978 (0.0008) [2023-12-27 10:35:18,027][20347] Updated weights for policy 0, policy_version 716988 (0.0008) [2023-12-27 10:35:18,075][20347] Updated weights for policy 0, policy_version 716998 (0.0008) [2023-12-27 10:35:18,089][20381] Updated weights for policy 1, policy_version 716444 (0.0007) [2023-12-27 10:35:18,146][20381] Updated weights for policy 1, policy_version 716454 (0.0006) [2023-12-27 10:35:18,195][20381] Updated weights for policy 1, policy_version 716464 (0.0005) [2023-12-27 10:35:18,818][20347] Updated weights for policy 0, policy_version 717008 (0.0008) [2023-12-27 10:35:18,870][20347] Updated weights for policy 0, policy_version 717018 (0.0008) [2023-12-27 10:35:18,872][20381] Updated weights for policy 1, policy_version 716474 (0.0008) [2023-12-27 10:35:18,926][20381] Updated weights for policy 1, policy_version 716484 (0.0005) [2023-12-27 10:35:18,927][20347] Updated weights for policy 0, policy_version 717028 (0.0005) [2023-12-27 10:35:18,982][20381] Updated weights for policy 1, policy_version 716494 (0.0005) [2023-12-27 10:35:19,036][20381] Updated weights for policy 1, policy_version 716504 (0.0006) [2023-12-27 10:35:19,655][20381] Updated weights for policy 1, policy_version 716514 (0.0008) [2023-12-27 10:35:19,684][20347] Updated weights for policy 0, policy_version 717038 (0.0007) [2023-12-27 10:35:19,705][20381] Updated weights for policy 1, policy_version 716524 (0.0006) [2023-12-27 10:35:19,740][20347] Updated weights for policy 0, policy_version 717048 (0.0008) [2023-12-27 10:35:19,762][20381] Updated weights for policy 1, policy_version 716534 (0.0007) [2023-12-27 10:35:19,801][20347] Updated weights for policy 0, policy_version 717058 (0.0008) [2023-12-27 10:35:20,444][20381] Updated weights for policy 1, policy_version 716544 (0.0008) [2023-12-27 10:35:20,499][20381] Updated weights for policy 1, policy_version 716554 (0.0008) [2023-12-27 10:35:20,547][20347] Updated weights for policy 0, policy_version 717068 (0.0007) [2023-12-27 10:35:20,557][20381] Updated weights for policy 1, policy_version 716564 (0.0008) [2023-12-27 10:35:20,610][20347] Updated weights for policy 0, policy_version 717078 (0.0008) [2023-12-27 10:35:20,661][20347] Updated weights for policy 0, policy_version 717088 (0.0008) [2023-12-27 10:35:21,259][20381] Updated weights for policy 1, policy_version 716574 (0.0008) [2023-12-27 10:35:21,323][20381] Updated weights for policy 1, policy_version 716584 (0.0007) [2023-12-27 10:35:21,394][20381] Updated weights for policy 1, policy_version 716594 (0.0009) [2023-12-27 10:35:21,427][20347] Updated weights for policy 0, policy_version 717098 (0.0007) [2023-12-27 10:35:21,491][20347] Updated weights for policy 0, policy_version 717108 (0.0007) [2023-12-27 10:35:21,547][20347] Updated weights for policy 0, policy_version 717118 (0.0007) [2023-12-27 10:35:21,607][20347] Updated weights for policy 0, policy_version 717128 (0.0008) [2023-12-27 10:35:22,113][20381] Updated weights for policy 1, policy_version 716604 (0.0007) [2023-12-27 10:35:22,172][20381] Updated weights for policy 1, policy_version 716614 (0.0008) [2023-12-27 10:35:22,229][20381] Updated weights for policy 1, policy_version 716624 (0.0008) [2023-12-27 10:35:22,321][20347] Updated weights for policy 0, policy_version 717138 (0.0008) [2023-12-27 10:35:22,382][19170] Fps is (10 sec: 20440.1, 60 sec: 20609.8, 300 sec: 20548.1). Total num frames: 367091712. Throughput: 0: 10203.5, 1: 10322.8. Samples: 367064312. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:22,383][19170] Avg episode reward: [(0, '970.780'), (1, '560.750')] [2023-12-27 10:35:22,383][20347] Updated weights for policy 0, policy_version 717148 (0.0008) [2023-12-27 10:35:22,388][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000716632_183484416.pth... [2023-12-27 10:35:22,393][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000715416_183173120.pth [2023-12-27 10:35:22,445][20347] Updated weights for policy 0, policy_version 717158 (0.0007) [2023-12-27 10:35:22,457][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000717160_183615488.pth... [2023-12-27 10:35:22,461][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000715944_183304192.pth [2023-12-27 10:35:22,885][20381] Updated weights for policy 1, policy_version 716634 (0.0007) [2023-12-27 10:35:22,953][20381] Updated weights for policy 1, policy_version 716644 (0.0008) [2023-12-27 10:35:23,009][20381] Updated weights for policy 1, policy_version 716654 (0.0008) [2023-12-27 10:35:23,070][20381] Updated weights for policy 1, policy_version 716664 (0.0009) [2023-12-27 10:35:23,194][20347] Updated weights for policy 0, policy_version 717168 (0.0006) [2023-12-27 10:35:23,251][20347] Updated weights for policy 0, policy_version 717178 (0.0006) [2023-12-27 10:35:23,307][20347] Updated weights for policy 0, policy_version 717188 (0.0007) [2023-12-27 10:35:23,684][20381] Updated weights for policy 1, policy_version 716674 (0.0008) [2023-12-27 10:35:23,739][20381] Updated weights for policy 1, policy_version 716684 (0.0007) [2023-12-27 10:35:23,798][20381] Updated weights for policy 1, policy_version 716694 (0.0008) [2023-12-27 10:35:23,967][20347] Updated weights for policy 0, policy_version 717198 (0.0007) [2023-12-27 10:35:24,026][20347] Updated weights for policy 0, policy_version 717208 (0.0007) [2023-12-27 10:35:24,075][20347] Updated weights for policy 0, policy_version 717218 (0.0005) [2023-12-27 10:35:24,404][20381] Updated weights for policy 1, policy_version 716704 (0.0006) [2023-12-27 10:35:24,455][20381] Updated weights for policy 1, policy_version 716714 (0.0005) [2023-12-27 10:35:24,511][20381] Updated weights for policy 1, policy_version 716724 (0.0006) [2023-12-27 10:35:24,629][20347] Updated weights for policy 0, policy_version 717228 (0.0005) [2023-12-27 10:35:24,680][20347] Updated weights for policy 0, policy_version 717238 (0.0005) [2023-12-27 10:35:24,741][20347] Updated weights for policy 0, policy_version 717248 (0.0005) [2023-12-27 10:35:25,129][20381] Updated weights for policy 1, policy_version 716734 (0.0009) [2023-12-27 10:35:25,185][20381] Updated weights for policy 1, policy_version 716744 (0.0008) [2023-12-27 10:35:25,242][20381] Updated weights for policy 1, policy_version 716754 (0.0007) [2023-12-27 10:35:25,352][20347] Updated weights for policy 0, policy_version 717258 (0.0006) [2023-12-27 10:35:25,401][20347] Updated weights for policy 0, policy_version 717268 (0.0006) [2023-12-27 10:35:25,448][20347] Updated weights for policy 0, policy_version 717278 (0.0005) [2023-12-27 10:35:25,502][20347] Updated weights for policy 0, policy_version 717288 (0.0005) [2023-12-27 10:35:25,933][20381] Updated weights for policy 1, policy_version 716764 (0.0007) [2023-12-27 10:35:25,990][20381] Updated weights for policy 1, policy_version 716774 (0.0006) [2023-12-27 10:35:26,043][20381] Updated weights for policy 1, policy_version 716784 (0.0007) [2023-12-27 10:35:26,082][20347] Updated weights for policy 0, policy_version 717298 (0.0006) [2023-12-27 10:35:26,133][20347] Updated weights for policy 0, policy_version 717308 (0.0007) [2023-12-27 10:35:26,194][20347] Updated weights for policy 0, policy_version 717318 (0.0008) [2023-12-27 10:35:26,641][20381] Updated weights for policy 1, policy_version 716794 (0.0006) [2023-12-27 10:35:26,700][20381] Updated weights for policy 1, policy_version 716804 (0.0010) [2023-12-27 10:35:26,764][20381] Updated weights for policy 1, policy_version 716814 (0.0005) [2023-12-27 10:35:26,817][20381] Updated weights for policy 1, policy_version 716824 (0.0007) [2023-12-27 10:35:26,884][20347] Updated weights for policy 0, policy_version 717328 (0.0009) [2023-12-27 10:35:26,950][20347] Updated weights for policy 0, policy_version 717338 (0.0010) [2023-12-27 10:35:27,009][20347] Updated weights for policy 0, policy_version 717348 (0.0006) [2023-12-27 10:35:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 367198208. Throughput: 0: 10230.8, 1: 10364.2. Samples: 367189348. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:27,364][19170] Avg episode reward: [(0, '980.470'), (1, '602.920')] [2023-12-27 10:35:27,474][20381] Updated weights for policy 1, policy_version 716834 (0.0007) [2023-12-27 10:35:27,528][20381] Updated weights for policy 1, policy_version 716844 (0.0007) [2023-12-27 10:35:27,576][20381] Updated weights for policy 1, policy_version 716854 (0.0008) [2023-12-27 10:35:27,609][20347] Updated weights for policy 0, policy_version 717358 (0.0008) [2023-12-27 10:35:27,664][20347] Updated weights for policy 0, policy_version 717368 (0.0010) [2023-12-27 10:35:27,737][20347] Updated weights for policy 0, policy_version 717378 (0.0008) [2023-12-27 10:35:28,300][20381] Updated weights for policy 1, policy_version 716864 (0.0010) [2023-12-27 10:35:28,363][20381] Updated weights for policy 1, policy_version 716874 (0.0009) [2023-12-27 10:35:28,369][20347] Updated weights for policy 0, policy_version 717388 (0.0006) [2023-12-27 10:35:28,421][20381] Updated weights for policy 1, policy_version 716884 (0.0010) [2023-12-27 10:35:28,427][20347] Updated weights for policy 0, policy_version 717398 (0.0006) [2023-12-27 10:35:28,481][20347] Updated weights for policy 0, policy_version 717408 (0.0007) [2023-12-27 10:35:29,033][20381] Updated weights for policy 1, policy_version 716894 (0.0010) [2023-12-27 10:35:29,087][20381] Updated weights for policy 1, policy_version 716904 (0.0010) [2023-12-27 10:35:29,140][20381] Updated weights for policy 1, policy_version 716914 (0.0009) [2023-12-27 10:35:29,170][20347] Updated weights for policy 0, policy_version 717418 (0.0007) [2023-12-27 10:35:29,229][20347] Updated weights for policy 0, policy_version 717428 (0.0010) [2023-12-27 10:35:29,293][20347] Updated weights for policy 0, policy_version 717438 (0.0010) [2023-12-27 10:35:29,354][20347] Updated weights for policy 0, policy_version 717448 (0.0010) [2023-12-27 10:35:29,841][20381] Updated weights for policy 1, policy_version 716924 (0.0009) [2023-12-27 10:35:29,902][20381] Updated weights for policy 1, policy_version 716934 (0.0011) [2023-12-27 10:35:29,961][20381] Updated weights for policy 1, policy_version 716944 (0.0010) [2023-12-27 10:35:30,062][20347] Updated weights for policy 0, policy_version 717458 (0.0008) [2023-12-27 10:35:30,121][20347] Updated weights for policy 0, policy_version 717468 (0.0007) [2023-12-27 10:35:30,176][20347] Updated weights for policy 0, policy_version 717478 (0.0008) [2023-12-27 10:35:30,592][20381] Updated weights for policy 1, policy_version 716954 (0.0009) [2023-12-27 10:35:30,649][20381] Updated weights for policy 1, policy_version 716964 (0.0007) [2023-12-27 10:35:30,699][20381] Updated weights for policy 1, policy_version 716974 (0.0007) [2023-12-27 10:35:30,763][20381] Updated weights for policy 1, policy_version 716984 (0.0005) [2023-12-27 10:35:30,879][20347] Updated weights for policy 0, policy_version 717488 (0.0006) [2023-12-27 10:35:30,930][20347] Updated weights for policy 0, policy_version 717498 (0.0005) [2023-12-27 10:35:30,980][20347] Updated weights for policy 0, policy_version 717508 (0.0005) [2023-12-27 10:35:31,423][20381] Updated weights for policy 1, policy_version 716994 (0.0009) [2023-12-27 10:35:31,474][20381] Updated weights for policy 1, policy_version 717004 (0.0007) [2023-12-27 10:35:31,533][20381] Updated weights for policy 1, policy_version 717014 (0.0007) [2023-12-27 10:35:31,565][20347] Updated weights for policy 0, policy_version 717518 (0.0008) [2023-12-27 10:35:31,628][20347] Updated weights for policy 0, policy_version 717528 (0.0008) [2023-12-27 10:35:31,689][20347] Updated weights for policy 0, policy_version 717538 (0.0008) [2023-12-27 10:35:32,205][20381] Updated weights for policy 1, policy_version 717024 (0.0006) [2023-12-27 10:35:32,260][20381] Updated weights for policy 1, policy_version 717034 (0.0007) [2023-12-27 10:35:32,319][20381] Updated weights for policy 1, policy_version 717044 (0.0008) [2023-12-27 10:35:32,362][19170] Fps is (10 sec: 21340.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 367304704. Throughput: 0: 10221.7, 1: 10364.3. Samples: 367313548. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:32,363][19170] Avg episode reward: [(0, '971.130'), (1, '368.480')] [2023-12-27 10:35:32,417][20347] Updated weights for policy 0, policy_version 717548 (0.0008) [2023-12-27 10:35:32,479][20347] Updated weights for policy 0, policy_version 717558 (0.0008) [2023-12-27 10:35:32,534][20347] Updated weights for policy 0, policy_version 717568 (0.0008) [2023-12-27 10:35:32,924][20381] Updated weights for policy 1, policy_version 717054 (0.0007) [2023-12-27 10:35:32,977][20381] Updated weights for policy 1, policy_version 717064 (0.0005) [2023-12-27 10:35:33,026][20381] Updated weights for policy 1, policy_version 717074 (0.0005) [2023-12-27 10:35:33,235][20347] Updated weights for policy 0, policy_version 717578 (0.0007) [2023-12-27 10:35:33,292][20347] Updated weights for policy 0, policy_version 717588 (0.0005) [2023-12-27 10:35:33,348][20347] Updated weights for policy 0, policy_version 717598 (0.0005) [2023-12-27 10:35:33,409][20347] Updated weights for policy 0, policy_version 717608 (0.0006) [2023-12-27 10:35:33,570][20381] Updated weights for policy 1, policy_version 717084 (0.0005) [2023-12-27 10:35:33,629][20381] Updated weights for policy 1, policy_version 717094 (0.0006) [2023-12-27 10:35:33,674][20381] Updated weights for policy 1, policy_version 717104 (0.0008) [2023-12-27 10:35:34,033][20347] Updated weights for policy 0, policy_version 717618 (0.0008) [2023-12-27 10:35:34,096][20347] Updated weights for policy 0, policy_version 717628 (0.0005) [2023-12-27 10:35:34,160][20347] Updated weights for policy 0, policy_version 717638 (0.0007) [2023-12-27 10:35:34,218][20381] Updated weights for policy 1, policy_version 717114 (0.0006) [2023-12-27 10:35:34,284][20381] Updated weights for policy 1, policy_version 717124 (0.0011) [2023-12-27 10:35:34,350][20381] Updated weights for policy 1, policy_version 717134 (0.0011) [2023-12-27 10:35:34,410][20381] Updated weights for policy 1, policy_version 717144 (0.0011) [2023-12-27 10:35:34,774][20347] Updated weights for policy 0, policy_version 717648 (0.0007) [2023-12-27 10:35:34,832][20347] Updated weights for policy 0, policy_version 717658 (0.0007) [2023-12-27 10:35:34,888][20347] Updated weights for policy 0, policy_version 717668 (0.0008) [2023-12-27 10:35:35,023][20381] Updated weights for policy 1, policy_version 717154 (0.0006) [2023-12-27 10:35:35,073][20381] Updated weights for policy 1, policy_version 717164 (0.0005) [2023-12-27 10:35:35,119][20381] Updated weights for policy 1, policy_version 717174 (0.0009) [2023-12-27 10:35:35,640][20347] Updated weights for policy 0, policy_version 717678 (0.0007) [2023-12-27 10:35:35,672][20381] Updated weights for policy 1, policy_version 717184 (0.0006) [2023-12-27 10:35:35,698][20347] Updated weights for policy 0, policy_version 717688 (0.0007) [2023-12-27 10:35:35,730][20381] Updated weights for policy 1, policy_version 717194 (0.0005) [2023-12-27 10:35:35,744][20347] Updated weights for policy 0, policy_version 717698 (0.0008) [2023-12-27 10:35:35,785][20381] Updated weights for policy 1, policy_version 717204 (0.0005) [2023-12-27 10:35:36,339][20381] Updated weights for policy 1, policy_version 717214 (0.0008) [2023-12-27 10:35:36,391][20381] Updated weights for policy 1, policy_version 717224 (0.0011) [2023-12-27 10:35:36,447][20381] Updated weights for policy 1, policy_version 717234 (0.0011) [2023-12-27 10:35:36,507][20347] Updated weights for policy 0, policy_version 717708 (0.0008) [2023-12-27 10:35:36,555][20347] Updated weights for policy 0, policy_version 717718 (0.0007) [2023-12-27 10:35:36,608][20347] Updated weights for policy 0, policy_version 717728 (0.0007) [2023-12-27 10:35:37,098][20381] Updated weights for policy 1, policy_version 717244 (0.0011) [2023-12-27 10:35:37,152][20381] Updated weights for policy 1, policy_version 717254 (0.0010) [2023-12-27 10:35:37,209][20381] Updated weights for policy 1, policy_version 717264 (0.0007) [2023-12-27 10:35:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 367411200. Throughput: 0: 10249.8, 1: 10457.3. Samples: 367379204. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:37,363][19170] Avg episode reward: [(0, '971.140'), (1, '467.690')] [2023-12-27 10:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000717272_183648256.pth... [2023-12-27 10:35:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000716024_183328768.pth [2023-12-27 10:35:37,379][20347] Updated weights for policy 0, policy_version 717738 (0.0008) [2023-12-27 10:35:37,433][20347] Updated weights for policy 0, policy_version 717748 (0.0006) [2023-12-27 10:35:37,491][20347] Updated weights for policy 0, policy_version 717758 (0.0007) [2023-12-27 10:35:37,545][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000717768_183771136.pth... [2023-12-27 10:35:37,545][20347] Updated weights for policy 0, policy_version 717768 (0.0006) [2023-12-27 10:35:37,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000716552_183459840.pth [2023-12-27 10:35:37,867][20381] Updated weights for policy 1, policy_version 717274 (0.0006) [2023-12-27 10:35:37,926][20381] Updated weights for policy 1, policy_version 717284 (0.0008) [2023-12-27 10:35:37,990][20381] Updated weights for policy 1, policy_version 717294 (0.0011) [2023-12-27 10:35:38,053][20381] Updated weights for policy 1, policy_version 717304 (0.0011) [2023-12-27 10:35:38,213][20347] Updated weights for policy 0, policy_version 717778 (0.0007) [2023-12-27 10:35:38,259][20347] Updated weights for policy 0, policy_version 717788 (0.0007) [2023-12-27 10:35:38,312][20347] Updated weights for policy 0, policy_version 717798 (0.0007) [2023-12-27 10:35:38,682][20381] Updated weights for policy 1, policy_version 717314 (0.0011) [2023-12-27 10:35:38,745][20381] Updated weights for policy 1, policy_version 717324 (0.0011) [2023-12-27 10:35:38,804][20381] Updated weights for policy 1, policy_version 717334 (0.0011) [2023-12-27 10:35:39,038][20347] Updated weights for policy 0, policy_version 717808 (0.0007) [2023-12-27 10:35:39,090][20347] Updated weights for policy 0, policy_version 717818 (0.0005) [2023-12-27 10:35:39,147][20347] Updated weights for policy 0, policy_version 717828 (0.0007) [2023-12-27 10:35:39,512][20381] Updated weights for policy 1, policy_version 717344 (0.0011) [2023-12-27 10:35:39,572][20381] Updated weights for policy 1, policy_version 717354 (0.0011) [2023-12-27 10:35:39,636][20381] Updated weights for policy 1, policy_version 717364 (0.0011) [2023-12-27 10:35:39,788][20347] Updated weights for policy 0, policy_version 717838 (0.0008) [2023-12-27 10:35:39,859][20347] Updated weights for policy 0, policy_version 717848 (0.0008) [2023-12-27 10:35:39,921][20347] Updated weights for policy 0, policy_version 717858 (0.0007) [2023-12-27 10:35:40,390][20381] Updated weights for policy 1, policy_version 717374 (0.0009) [2023-12-27 10:35:40,445][20381] Updated weights for policy 1, policy_version 717384 (0.0008) [2023-12-27 10:35:40,506][20381] Updated weights for policy 1, policy_version 717394 (0.0008) [2023-12-27 10:35:40,609][20347] Updated weights for policy 0, policy_version 717868 (0.0008) [2023-12-27 10:35:40,669][20347] Updated weights for policy 0, policy_version 717878 (0.0008) [2023-12-27 10:35:40,719][20347] Updated weights for policy 0, policy_version 717888 (0.0007) [2023-12-27 10:35:41,212][20381] Updated weights for policy 1, policy_version 717404 (0.0007) [2023-12-27 10:35:41,275][20381] Updated weights for policy 1, policy_version 717414 (0.0007) [2023-12-27 10:35:41,338][20381] Updated weights for policy 1, policy_version 717424 (0.0008) [2023-12-27 10:35:41,499][20347] Updated weights for policy 0, policy_version 717898 (0.0007) [2023-12-27 10:35:41,559][20347] Updated weights for policy 0, policy_version 717908 (0.0007) [2023-12-27 10:35:41,623][20347] Updated weights for policy 0, policy_version 717918 (0.0008) [2023-12-27 10:35:41,682][20347] Updated weights for policy 0, policy_version 717928 (0.0008) [2023-12-27 10:35:42,062][20381] Updated weights for policy 1, policy_version 717434 (0.0009) [2023-12-27 10:35:42,121][20381] Updated weights for policy 1, policy_version 717444 (0.0011) [2023-12-27 10:35:42,179][20381] Updated weights for policy 1, policy_version 717454 (0.0011) [2023-12-27 10:35:42,243][20381] Updated weights for policy 1, policy_version 717464 (0.0011) [2023-12-27 10:35:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 367509504. Throughput: 0: 10200.1, 1: 10492.8. Samples: 367501684. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:42,363][19170] Avg episode reward: [(0, '942.170'), (1, '363.250')] [2023-12-27 10:35:42,392][20347] Updated weights for policy 0, policy_version 717938 (0.0008) [2023-12-27 10:35:42,452][20347] Updated weights for policy 0, policy_version 717948 (0.0007) [2023-12-27 10:35:42,516][20347] Updated weights for policy 0, policy_version 717958 (0.0008) [2023-12-27 10:35:42,870][20381] Updated weights for policy 1, policy_version 717474 (0.0008) [2023-12-27 10:35:42,930][20381] Updated weights for policy 1, policy_version 717484 (0.0010) [2023-12-27 10:35:42,987][20381] Updated weights for policy 1, policy_version 717494 (0.0011) [2023-12-27 10:35:43,252][20347] Updated weights for policy 0, policy_version 717968 (0.0008) [2023-12-27 10:35:43,303][20347] Updated weights for policy 0, policy_version 717978 (0.0007) [2023-12-27 10:35:43,349][20347] Updated weights for policy 0, policy_version 717988 (0.0010) [2023-12-27 10:35:43,604][20381] Updated weights for policy 1, policy_version 717504 (0.0009) [2023-12-27 10:35:43,655][20381] Updated weights for policy 1, policy_version 717514 (0.0008) [2023-12-27 10:35:43,701][20381] Updated weights for policy 1, policy_version 717524 (0.0008) [2023-12-27 10:35:44,081][20347] Updated weights for policy 0, policy_version 717998 (0.0008) [2023-12-27 10:35:44,144][20347] Updated weights for policy 0, policy_version 718008 (0.0008) [2023-12-27 10:35:44,201][20347] Updated weights for policy 0, policy_version 718018 (0.0007) [2023-12-27 10:35:44,337][20381] Updated weights for policy 1, policy_version 717534 (0.0007) [2023-12-27 10:35:44,385][20381] Updated weights for policy 1, policy_version 717544 (0.0005) [2023-12-27 10:35:44,436][20381] Updated weights for policy 1, policy_version 717554 (0.0007) [2023-12-27 10:35:44,892][20347] Updated weights for policy 0, policy_version 718028 (0.0008) [2023-12-27 10:35:44,948][20347] Updated weights for policy 0, policy_version 718038 (0.0007) [2023-12-27 10:35:45,005][20347] Updated weights for policy 0, policy_version 718048 (0.0007) [2023-12-27 10:35:45,147][20381] Updated weights for policy 1, policy_version 717564 (0.0007) [2023-12-27 10:35:45,206][20381] Updated weights for policy 1, policy_version 717574 (0.0008) [2023-12-27 10:35:45,267][20381] Updated weights for policy 1, policy_version 717584 (0.0007) [2023-12-27 10:35:45,671][20347] Updated weights for policy 0, policy_version 718058 (0.0007) [2023-12-27 10:35:45,736][20347] Updated weights for policy 0, policy_version 718068 (0.0005) [2023-12-27 10:35:45,798][20347] Updated weights for policy 0, policy_version 718078 (0.0005) [2023-12-27 10:35:45,860][20347] Updated weights for policy 0, policy_version 718088 (0.0006) [2023-12-27 10:35:45,898][20381] Updated weights for policy 1, policy_version 717594 (0.0007) [2023-12-27 10:35:45,953][20381] Updated weights for policy 1, policy_version 717604 (0.0007) [2023-12-27 10:35:45,999][20381] Updated weights for policy 1, policy_version 717614 (0.0008) [2023-12-27 10:35:46,044][20381] Updated weights for policy 1, policy_version 717624 (0.0008) [2023-12-27 10:35:46,471][20347] Updated weights for policy 0, policy_version 718098 (0.0008) [2023-12-27 10:35:46,523][20347] Updated weights for policy 0, policy_version 718108 (0.0008) [2023-12-27 10:35:46,569][20347] Updated weights for policy 0, policy_version 718118 (0.0008) [2023-12-27 10:35:46,677][20381] Updated weights for policy 1, policy_version 717634 (0.0010) [2023-12-27 10:35:46,728][20381] Updated weights for policy 1, policy_version 717644 (0.0008) [2023-12-27 10:35:46,774][20381] Updated weights for policy 1, policy_version 717654 (0.0005) [2023-12-27 10:35:47,299][20347] Updated weights for policy 0, policy_version 718128 (0.0009) [2023-12-27 10:35:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 367607808. Throughput: 0: 10104.4, 1: 10508.6. Samples: 367625244. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:47,363][20347] Updated weights for policy 0, policy_version 718138 (0.0007) [2023-12-27 10:35:47,363][19170] Avg episode reward: [(0, '942.200'), (1, '358.580')] [2023-12-27 10:35:47,382][20381] Updated weights for policy 1, policy_version 717664 (0.0010) [2023-12-27 10:35:47,411][20347] Updated weights for policy 0, policy_version 718148 (0.0006) [2023-12-27 10:35:47,440][20381] Updated weights for policy 1, policy_version 717674 (0.0006) [2023-12-27 10:35:47,501][20381] Updated weights for policy 1, policy_version 717684 (0.0006) [2023-12-27 10:35:48,056][20347] Updated weights for policy 0, policy_version 718158 (0.0009) [2023-12-27 10:35:48,119][20347] Updated weights for policy 0, policy_version 718168 (0.0010) [2023-12-27 10:35:48,166][20381] Updated weights for policy 1, policy_version 717694 (0.0009) [2023-12-27 10:35:48,175][20347] Updated weights for policy 0, policy_version 718178 (0.0010) [2023-12-27 10:35:48,216][20381] Updated weights for policy 1, policy_version 717704 (0.0011) [2023-12-27 10:35:48,270][20381] Updated weights for policy 1, policy_version 717714 (0.0011) [2023-12-27 10:35:48,830][20347] Updated weights for policy 0, policy_version 718188 (0.0009) [2023-12-27 10:35:48,879][20347] Updated weights for policy 0, policy_version 718198 (0.0010) [2023-12-27 10:35:48,933][20381] Updated weights for policy 1, policy_version 717724 (0.0009) [2023-12-27 10:35:48,940][20347] Updated weights for policy 0, policy_version 718208 (0.0010) [2023-12-27 10:35:48,998][20381] Updated weights for policy 1, policy_version 717734 (0.0007) [2023-12-27 10:35:49,048][20381] Updated weights for policy 1, policy_version 717744 (0.0005) [2023-12-27 10:35:49,656][20347] Updated weights for policy 0, policy_version 718218 (0.0008) [2023-12-27 10:35:49,688][20381] Updated weights for policy 1, policy_version 717754 (0.0005) [2023-12-27 10:35:49,722][20347] Updated weights for policy 0, policy_version 718228 (0.0007) [2023-12-27 10:35:49,746][20381] Updated weights for policy 1, policy_version 717764 (0.0006) [2023-12-27 10:35:49,782][20347] Updated weights for policy 0, policy_version 718238 (0.0009) [2023-12-27 10:35:49,815][20381] Updated weights for policy 1, policy_version 717774 (0.0007) [2023-12-27 10:35:49,841][20347] Updated weights for policy 0, policy_version 718248 (0.0006) [2023-12-27 10:35:49,882][20381] Updated weights for policy 1, policy_version 717784 (0.0006) [2023-12-27 10:35:50,467][20381] Updated weights for policy 1, policy_version 717794 (0.0007) [2023-12-27 10:35:50,515][20381] Updated weights for policy 1, policy_version 717804 (0.0007) [2023-12-27 10:35:50,517][20347] Updated weights for policy 0, policy_version 718258 (0.0006) [2023-12-27 10:35:50,548][20291] KL-divergence is very high: 117.2080 [2023-12-27 10:35:50,568][20347] Updated weights for policy 0, policy_version 718268 (0.0007) [2023-12-27 10:35:50,569][20381] Updated weights for policy 1, policy_version 717814 (0.0006) [2023-12-27 10:35:50,635][20347] Updated weights for policy 0, policy_version 718278 (0.0008) [2023-12-27 10:35:51,291][20291] KL-divergence is very high: 104.2971 [2023-12-27 10:35:51,291][20381] Updated weights for policy 1, policy_version 717824 (0.0008) [2023-12-27 10:35:51,342][20347] Updated weights for policy 0, policy_version 718288 (0.0008) [2023-12-27 10:35:51,342][20291] KL-divergence is very high: 194.9385 [2023-12-27 10:35:51,351][20381] Updated weights for policy 1, policy_version 717834 (0.0007) [2023-12-27 10:35:51,398][20291] KL-divergence is very high: 139.8230 [2023-12-27 10:35:51,408][20347] Updated weights for policy 0, policy_version 718298 (0.0008) [2023-12-27 10:35:51,425][20381] Updated weights for policy 1, policy_version 717844 (0.0008) [2023-12-27 10:35:51,460][20347] Updated weights for policy 0, policy_version 718308 (0.0008) [2023-12-27 10:35:52,098][20381] Updated weights for policy 1, policy_version 717854 (0.0007) [2023-12-27 10:35:52,161][20381] Updated weights for policy 1, policy_version 717864 (0.0007) [2023-12-27 10:35:52,206][20347] Updated weights for policy 0, policy_version 718318 (0.0006) [2023-12-27 10:35:52,219][20381] Updated weights for policy 1, policy_version 717874 (0.0007) [2023-12-27 10:35:52,268][20347] Updated weights for policy 0, policy_version 718328 (0.0007) [2023-12-27 10:35:52,325][20347] Updated weights for policy 0, policy_version 718338 (0.0007) [2023-12-27 10:35:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 367714304. Throughput: 0: 10092.8, 1: 10535.4. Samples: 367687456. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:52,363][19170] Avg episode reward: [(0, '883.950'), (1, '446.420')] [2023-12-27 10:35:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000717880_183803904.pth... [2023-12-27 10:35:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000718344_183918592.pth... [2023-12-27 10:35:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000716632_183484416.pth [2023-12-27 10:35:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000717160_183615488.pth [2023-12-27 10:35:52,890][20381] Updated weights for policy 1, policy_version 717884 (0.0008) [2023-12-27 10:35:52,942][20381] Updated weights for policy 1, policy_version 717894 (0.0010) [2023-12-27 10:35:52,995][20381] Updated weights for policy 1, policy_version 717904 (0.0010) [2023-12-27 10:35:52,995][20347] Updated weights for policy 0, policy_version 718348 (0.0008) [2023-12-27 10:35:53,059][20347] Updated weights for policy 0, policy_version 718358 (0.0007) [2023-12-27 10:35:53,117][20347] Updated weights for policy 0, policy_version 718368 (0.0007) [2023-12-27 10:35:53,697][20347] Updated weights for policy 0, policy_version 718378 (0.0007) [2023-12-27 10:35:53,719][20381] Updated weights for policy 1, policy_version 717914 (0.0011) [2023-12-27 10:35:53,755][20347] Updated weights for policy 0, policy_version 718388 (0.0005) [2023-12-27 10:35:53,775][20381] Updated weights for policy 1, policy_version 717924 (0.0011) [2023-12-27 10:35:53,811][20347] Updated weights for policy 0, policy_version 718398 (0.0005) [2023-12-27 10:35:53,821][20381] Updated weights for policy 1, policy_version 717934 (0.0010) [2023-12-27 10:35:53,868][20347] Updated weights for policy 0, policy_version 718408 (0.0007) [2023-12-27 10:35:53,877][20381] Updated weights for policy 1, policy_version 717944 (0.0010) [2023-12-27 10:35:54,499][20347] Updated weights for policy 0, policy_version 718418 (0.0008) [2023-12-27 10:35:54,548][20347] Updated weights for policy 0, policy_version 718428 (0.0007) [2023-12-27 10:35:54,571][20381] Updated weights for policy 1, policy_version 717954 (0.0005) [2023-12-27 10:35:54,594][20347] Updated weights for policy 0, policy_version 718438 (0.0008) [2023-12-27 10:35:54,627][20381] Updated weights for policy 1, policy_version 717964 (0.0007) [2023-12-27 10:35:54,680][20381] Updated weights for policy 1, policy_version 717974 (0.0008) [2023-12-27 10:35:55,267][20381] Updated weights for policy 1, policy_version 717984 (0.0006) [2023-12-27 10:35:55,317][20381] Updated weights for policy 1, policy_version 717994 (0.0005) [2023-12-27 10:35:55,346][20347] Updated weights for policy 0, policy_version 718448 (0.0007) [2023-12-27 10:35:55,367][20381] Updated weights for policy 1, policy_version 718004 (0.0006) [2023-12-27 10:35:55,404][20347] Updated weights for policy 0, policy_version 718458 (0.0007) [2023-12-27 10:35:55,463][20347] Updated weights for policy 0, policy_version 718468 (0.0008) [2023-12-27 10:35:55,932][20381] Updated weights for policy 1, policy_version 718014 (0.0006) [2023-12-27 10:35:55,991][20381] Updated weights for policy 1, policy_version 718024 (0.0005) [2023-12-27 10:35:56,044][20381] Updated weights for policy 1, policy_version 718034 (0.0006) [2023-12-27 10:35:56,153][20347] Updated weights for policy 0, policy_version 718478 (0.0009) [2023-12-27 10:35:56,215][20347] Updated weights for policy 0, policy_version 718488 (0.0007) [2023-12-27 10:35:56,277][20347] Updated weights for policy 0, policy_version 718498 (0.0010) [2023-12-27 10:35:56,650][20381] Updated weights for policy 1, policy_version 718044 (0.0005) [2023-12-27 10:35:56,710][20381] Updated weights for policy 1, policy_version 718054 (0.0006) [2023-12-27 10:35:56,773][20381] Updated weights for policy 1, policy_version 718064 (0.0005) [2023-12-27 10:35:56,896][20347] Updated weights for policy 0, policy_version 718508 (0.0009) [2023-12-27 10:35:56,944][20347] Updated weights for policy 0, policy_version 718518 (0.0008) [2023-12-27 10:35:56,993][20347] Updated weights for policy 0, policy_version 718528 (0.0009) [2023-12-27 10:35:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 367820800. Throughput: 0: 10133.5, 1: 10560.9. Samples: 367811468. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:35:57,363][19170] Avg episode reward: [(0, '835.320'), (1, '392.360')] [2023-12-27 10:35:57,397][20381] Updated weights for policy 1, policy_version 718074 (0.0006) [2023-12-27 10:35:57,453][20381] Updated weights for policy 1, policy_version 718084 (0.0007) [2023-12-27 10:35:57,506][20381] Updated weights for policy 1, policy_version 718094 (0.0007) [2023-12-27 10:35:57,555][20381] Updated weights for policy 1, policy_version 718104 (0.0006) [2023-12-27 10:35:57,606][20347] Updated weights for policy 0, policy_version 718538 (0.0007) [2023-12-27 10:35:57,663][20347] Updated weights for policy 0, policy_version 718548 (0.0010) [2023-12-27 10:35:57,718][20347] Updated weights for policy 0, policy_version 718558 (0.0010) [2023-12-27 10:35:57,777][20347] Updated weights for policy 0, policy_version 718568 (0.0009) [2023-12-27 10:35:58,194][20381] Updated weights for policy 1, policy_version 718114 (0.0010) [2023-12-27 10:35:58,252][20381] Updated weights for policy 1, policy_version 718124 (0.0007) [2023-12-27 10:35:58,317][20381] Updated weights for policy 1, policy_version 718134 (0.0008) [2023-12-27 10:35:58,496][20347] Updated weights for policy 0, policy_version 718578 (0.0011) [2023-12-27 10:35:58,561][20347] Updated weights for policy 0, policy_version 718588 (0.0008) [2023-12-27 10:35:58,628][20347] Updated weights for policy 0, policy_version 718598 (0.0007) [2023-12-27 10:35:59,090][20381] Updated weights for policy 1, policy_version 718144 (0.0010) [2023-12-27 10:35:59,147][20381] Updated weights for policy 1, policy_version 718154 (0.0010) [2023-12-27 10:35:59,202][20381] Updated weights for policy 1, policy_version 718164 (0.0011) [2023-12-27 10:35:59,320][20347] Updated weights for policy 0, policy_version 718608 (0.0009) [2023-12-27 10:35:59,385][20347] Updated weights for policy 0, policy_version 718618 (0.0009) [2023-12-27 10:35:59,435][20347] Updated weights for policy 0, policy_version 718628 (0.0010) [2023-12-27 10:35:59,927][20381] Updated weights for policy 1, policy_version 718174 (0.0009) [2023-12-27 10:35:59,989][20381] Updated weights for policy 1, policy_version 718184 (0.0010) [2023-12-27 10:36:00,045][20381] Updated weights for policy 1, policy_version 718194 (0.0010) [2023-12-27 10:36:00,084][20347] Updated weights for policy 0, policy_version 718638 (0.0007) [2023-12-27 10:36:00,138][20347] Updated weights for policy 0, policy_version 718648 (0.0006) [2023-12-27 10:36:00,187][20347] Updated weights for policy 0, policy_version 718658 (0.0010) [2023-12-27 10:36:00,737][20381] Updated weights for policy 1, policy_version 718204 (0.0010) [2023-12-27 10:36:00,796][20381] Updated weights for policy 1, policy_version 718214 (0.0008) [2023-12-27 10:36:00,805][20347] Updated weights for policy 0, policy_version 718668 (0.0009) [2023-12-27 10:36:00,846][20381] Updated weights for policy 1, policy_version 718224 (0.0008) [2023-12-27 10:36:00,863][20347] Updated weights for policy 0, policy_version 718678 (0.0006) [2023-12-27 10:36:00,912][20347] Updated weights for policy 0, policy_version 718688 (0.0007) [2023-12-27 10:36:01,504][20381] Updated weights for policy 1, policy_version 718234 (0.0010) [2023-12-27 10:36:01,554][20381] Updated weights for policy 1, policy_version 718244 (0.0008) [2023-12-27 10:36:01,600][20381] Updated weights for policy 1, policy_version 718254 (0.0007) [2023-12-27 10:36:01,653][20347] Updated weights for policy 0, policy_version 718698 (0.0007) [2023-12-27 10:36:01,659][20381] Updated weights for policy 1, policy_version 718264 (0.0009) [2023-12-27 10:36:01,708][20347] Updated weights for policy 0, policy_version 718708 (0.0007) [2023-12-27 10:36:01,769][20347] Updated weights for policy 0, policy_version 718718 (0.0007) [2023-12-27 10:36:01,819][20347] Updated weights for policy 0, policy_version 718728 (0.0006) [2023-12-27 10:36:02,306][20381] Updated weights for policy 1, policy_version 718274 (0.0008) [2023-12-27 10:36:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 367919104. Throughput: 0: 10108.4, 1: 10574.1. Samples: 367934404. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:02,363][19170] Avg episode reward: [(0, '922.360'), (1, '479.770')] [2023-12-27 10:36:02,372][20381] Updated weights for policy 1, policy_version 718284 (0.0008) [2023-12-27 10:36:02,428][20381] Updated weights for policy 1, policy_version 718294 (0.0008) [2023-12-27 10:36:02,573][20347] Updated weights for policy 0, policy_version 718738 (0.0007) [2023-12-27 10:36:02,634][20347] Updated weights for policy 0, policy_version 718748 (0.0007) [2023-12-27 10:36:02,694][20347] Updated weights for policy 0, policy_version 718758 (0.0010) [2023-12-27 10:36:03,081][20381] Updated weights for policy 1, policy_version 718304 (0.0006) [2023-12-27 10:36:03,147][20381] Updated weights for policy 1, policy_version 718314 (0.0005) [2023-12-27 10:36:03,199][20381] Updated weights for policy 1, policy_version 718324 (0.0009) [2023-12-27 10:36:03,359][20347] Updated weights for policy 0, policy_version 718768 (0.0007) [2023-12-27 10:36:03,416][20347] Updated weights for policy 0, policy_version 718778 (0.0007) [2023-12-27 10:36:03,473][20347] Updated weights for policy 0, policy_version 718788 (0.0006) [2023-12-27 10:36:03,899][20381] Updated weights for policy 1, policy_version 718334 (0.0008) [2023-12-27 10:36:03,953][20381] Updated weights for policy 1, policy_version 718344 (0.0008) [2023-12-27 10:36:04,016][20381] Updated weights for policy 1, policy_version 718354 (0.0009) [2023-12-27 10:36:04,113][20347] Updated weights for policy 0, policy_version 718798 (0.0007) [2023-12-27 10:36:04,171][20347] Updated weights for policy 0, policy_version 718808 (0.0008) [2023-12-27 10:36:04,233][20347] Updated weights for policy 0, policy_version 718818 (0.0007) [2023-12-27 10:36:04,704][20381] Updated weights for policy 1, policy_version 718364 (0.0008) [2023-12-27 10:36:04,756][20381] Updated weights for policy 1, policy_version 718374 (0.0009) [2023-12-27 10:36:04,810][20381] Updated weights for policy 1, policy_version 718384 (0.0007) [2023-12-27 10:36:04,900][20347] Updated weights for policy 0, policy_version 718828 (0.0006) [2023-12-27 10:36:04,963][20347] Updated weights for policy 0, policy_version 718838 (0.0006) [2023-12-27 10:36:05,013][20347] Updated weights for policy 0, policy_version 718848 (0.0007) [2023-12-27 10:36:05,456][20381] Updated weights for policy 1, policy_version 718394 (0.0005) [2023-12-27 10:36:05,510][20381] Updated weights for policy 1, policy_version 718404 (0.0005) [2023-12-27 10:36:05,568][20381] Updated weights for policy 1, policy_version 718414 (0.0005) [2023-12-27 10:36:05,607][20347] Updated weights for policy 0, policy_version 718858 (0.0006) [2023-12-27 10:36:05,622][20381] Updated weights for policy 1, policy_version 718424 (0.0007) [2023-12-27 10:36:05,664][20347] Updated weights for policy 0, policy_version 718868 (0.0006) [2023-12-27 10:36:05,712][20347] Updated weights for policy 0, policy_version 718878 (0.0006) [2023-12-27 10:36:05,766][20347] Updated weights for policy 0, policy_version 718888 (0.0005) [2023-12-27 10:36:06,239][20381] Updated weights for policy 1, policy_version 718434 (0.0008) [2023-12-27 10:36:06,302][20381] Updated weights for policy 1, policy_version 718444 (0.0007) [2023-12-27 10:36:06,336][20347] Updated weights for policy 0, policy_version 718898 (0.0007) [2023-12-27 10:36:06,365][20381] Updated weights for policy 1, policy_version 718454 (0.0008) [2023-12-27 10:36:06,399][20347] Updated weights for policy 0, policy_version 718908 (0.0006) [2023-12-27 10:36:06,457][20347] Updated weights for policy 0, policy_version 718918 (0.0008) [2023-12-27 10:36:07,000][20381] Updated weights for policy 1, policy_version 718464 (0.0006) [2023-12-27 10:36:07,056][20381] Updated weights for policy 1, policy_version 718474 (0.0006) [2023-12-27 10:36:07,107][20381] Updated weights for policy 1, policy_version 718484 (0.0007) [2023-12-27 10:36:07,166][20347] Updated weights for policy 0, policy_version 718928 (0.0007) [2023-12-27 10:36:07,221][20347] Updated weights for policy 0, policy_version 718938 (0.0007) [2023-12-27 10:36:07,275][20347] Updated weights for policy 0, policy_version 718948 (0.0007) [2023-12-27 10:36:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 368033792. Throughput: 0: 10161.5, 1: 10551.3. Samples: 367995984. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:07,363][19170] Avg episode reward: [(0, '932.430'), (1, '537.290')] [2023-12-27 10:36:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000718952_184074240.pth... [2023-12-27 10:36:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000718488_183959552.pth... [2023-12-27 10:36:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000717768_183771136.pth [2023-12-27 10:36:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000717272_183648256.pth [2023-12-27 10:36:07,713][20381] Updated weights for policy 1, policy_version 718494 (0.0008) [2023-12-27 10:36:07,762][20381] Updated weights for policy 1, policy_version 718504 (0.0007) [2023-12-27 10:36:07,808][20381] Updated weights for policy 1, policy_version 718514 (0.0007) [2023-12-27 10:36:07,983][20347] Updated weights for policy 0, policy_version 718958 (0.0010) [2023-12-27 10:36:08,034][20347] Updated weights for policy 0, policy_version 718968 (0.0010) [2023-12-27 10:36:08,082][20347] Updated weights for policy 0, policy_version 718978 (0.0010) [2023-12-27 10:36:08,428][20381] Updated weights for policy 1, policy_version 718524 (0.0007) [2023-12-27 10:36:08,489][20381] Updated weights for policy 1, policy_version 718534 (0.0008) [2023-12-27 10:36:08,548][20381] Updated weights for policy 1, policy_version 718544 (0.0007) [2023-12-27 10:36:08,798][20347] Updated weights for policy 0, policy_version 718988 (0.0008) [2023-12-27 10:36:08,863][20347] Updated weights for policy 0, policy_version 718998 (0.0006) [2023-12-27 10:36:08,923][20347] Updated weights for policy 0, policy_version 719008 (0.0007) [2023-12-27 10:36:09,280][20381] Updated weights for policy 1, policy_version 718554 (0.0008) [2023-12-27 10:36:09,344][20381] Updated weights for policy 1, policy_version 718564 (0.0008) [2023-12-27 10:36:09,405][20381] Updated weights for policy 1, policy_version 718574 (0.0008) [2023-12-27 10:36:09,465][20381] Updated weights for policy 1, policy_version 718584 (0.0008) [2023-12-27 10:36:09,521][20347] Updated weights for policy 0, policy_version 719018 (0.0006) [2023-12-27 10:36:09,582][20347] Updated weights for policy 0, policy_version 719028 (0.0006) [2023-12-27 10:36:09,647][20347] Updated weights for policy 0, policy_version 719038 (0.0007) [2023-12-27 10:36:09,710][20347] Updated weights for policy 0, policy_version 719048 (0.0005) [2023-12-27 10:36:10,199][20381] Updated weights for policy 1, policy_version 718594 (0.0008) [2023-12-27 10:36:10,259][20381] Updated weights for policy 1, policy_version 718604 (0.0008) [2023-12-27 10:36:10,321][20381] Updated weights for policy 1, policy_version 718614 (0.0007) [2023-12-27 10:36:10,371][20347] Updated weights for policy 0, policy_version 719058 (0.0010) [2023-12-27 10:36:10,426][20347] Updated weights for policy 0, policy_version 719068 (0.0008) [2023-12-27 10:36:10,479][20347] Updated weights for policy 0, policy_version 719078 (0.0005) [2023-12-27 10:36:11,000][20381] Updated weights for policy 1, policy_version 718624 (0.0008) [2023-12-27 10:36:11,067][20381] Updated weights for policy 1, policy_version 718634 (0.0007) [2023-12-27 10:36:11,109][20347] Updated weights for policy 0, policy_version 719088 (0.0007) [2023-12-27 10:36:11,132][20381] Updated weights for policy 1, policy_version 718644 (0.0006) [2023-12-27 10:36:11,174][20347] Updated weights for policy 0, policy_version 719098 (0.0009) [2023-12-27 10:36:11,237][20347] Updated weights for policy 0, policy_version 719108 (0.0007) [2023-12-27 10:36:11,899][20347] Updated weights for policy 0, policy_version 719118 (0.0006) [2023-12-27 10:36:11,959][20347] Updated weights for policy 0, policy_version 719128 (0.0005) [2023-12-27 10:36:11,964][20381] Updated weights for policy 1, policy_version 718654 (0.0009) [2023-12-27 10:36:12,021][20347] Updated weights for policy 0, policy_version 719138 (0.0006) [2023-12-27 10:36:12,023][20381] Updated weights for policy 1, policy_version 718664 (0.0008) [2023-12-27 10:36:12,084][20381] Updated weights for policy 1, policy_version 718674 (0.0008) [2023-12-27 10:36:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 368132096. Throughput: 0: 10190.9, 1: 10516.9. Samples: 368121196. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:12,363][19170] Avg episode reward: [(0, '971.240'), (1, '616.730')] [2023-12-27 10:36:12,636][20347] Updated weights for policy 0, policy_version 719148 (0.0006) [2023-12-27 10:36:12,693][20347] Updated weights for policy 0, policy_version 719158 (0.0006) [2023-12-27 10:36:12,758][20347] Updated weights for policy 0, policy_version 719168 (0.0006) [2023-12-27 10:36:12,806][20381] Updated weights for policy 1, policy_version 718684 (0.0007) [2023-12-27 10:36:12,863][20381] Updated weights for policy 1, policy_version 718694 (0.0007) [2023-12-27 10:36:12,913][20381] Updated weights for policy 1, policy_version 718704 (0.0008) [2023-12-27 10:36:13,404][20347] Updated weights for policy 0, policy_version 719178 (0.0006) [2023-12-27 10:36:13,457][20347] Updated weights for policy 0, policy_version 719188 (0.0007) [2023-12-27 10:36:13,514][20347] Updated weights for policy 0, policy_version 719198 (0.0008) [2023-12-27 10:36:13,564][20347] Updated weights for policy 0, policy_version 719208 (0.0008) [2023-12-27 10:36:13,616][20381] Updated weights for policy 1, policy_version 718714 (0.0007) [2023-12-27 10:36:13,665][20381] Updated weights for policy 1, policy_version 718724 (0.0008) [2023-12-27 10:36:13,719][20381] Updated weights for policy 1, policy_version 718734 (0.0008) [2023-12-27 10:36:13,775][20381] Updated weights for policy 1, policy_version 718744 (0.0008) [2023-12-27 10:36:14,209][20347] Updated weights for policy 0, policy_version 719218 (0.0008) [2023-12-27 10:36:14,271][20347] Updated weights for policy 0, policy_version 719228 (0.0008) [2023-12-27 10:36:14,331][20347] Updated weights for policy 0, policy_version 719238 (0.0007) [2023-12-27 10:36:14,502][20381] Updated weights for policy 1, policy_version 718754 (0.0007) [2023-12-27 10:36:14,550][20381] Updated weights for policy 1, policy_version 718764 (0.0007) [2023-12-27 10:36:14,601][20381] Updated weights for policy 1, policy_version 718774 (0.0007) [2023-12-27 10:36:15,068][20347] Updated weights for policy 0, policy_version 719248 (0.0008) [2023-12-27 10:36:15,129][20347] Updated weights for policy 0, policy_version 719258 (0.0008) [2023-12-27 10:36:15,200][20347] Updated weights for policy 0, policy_version 719268 (0.0007) [2023-12-27 10:36:15,243][20381] Updated weights for policy 1, policy_version 718784 (0.0007) [2023-12-27 10:36:15,302][20381] Updated weights for policy 1, policy_version 718794 (0.0008) [2023-12-27 10:36:15,356][20381] Updated weights for policy 1, policy_version 718804 (0.0008) [2023-12-27 10:36:15,923][20347] Updated weights for policy 0, policy_version 719278 (0.0006) [2023-12-27 10:36:15,932][20381] Updated weights for policy 1, policy_version 718814 (0.0007) [2023-12-27 10:36:15,982][20347] Updated weights for policy 0, policy_version 719288 (0.0008) [2023-12-27 10:36:15,988][20381] Updated weights for policy 1, policy_version 718824 (0.0005) [2023-12-27 10:36:16,042][20347] Updated weights for policy 0, policy_version 719298 (0.0007) [2023-12-27 10:36:16,045][20381] Updated weights for policy 1, policy_version 718834 (0.0005) [2023-12-27 10:36:16,708][20381] Updated weights for policy 1, policy_version 718844 (0.0006) [2023-12-27 10:36:16,731][20347] Updated weights for policy 0, policy_version 719308 (0.0007) [2023-12-27 10:36:16,767][20381] Updated weights for policy 1, policy_version 718854 (0.0006) [2023-12-27 10:36:16,783][20347] Updated weights for policy 0, policy_version 719318 (0.0005) [2023-12-27 10:36:16,826][20381] Updated weights for policy 1, policy_version 718864 (0.0006) [2023-12-27 10:36:16,838][20347] Updated weights for policy 0, policy_version 719328 (0.0008) [2023-12-27 10:36:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 368230400. Throughput: 0: 10161.7, 1: 10482.5. Samples: 368242536. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:17,364][19170] Avg episode reward: [(0, '970.830'), (1, '665.830')] [2023-12-27 10:36:17,491][20381] Updated weights for policy 1, policy_version 718874 (0.0008) [2023-12-27 10:36:17,544][20347] Updated weights for policy 0, policy_version 719338 (0.0007) [2023-12-27 10:36:17,552][20381] Updated weights for policy 1, policy_version 718884 (0.0008) [2023-12-27 10:36:17,608][20347] Updated weights for policy 0, policy_version 719348 (0.0007) [2023-12-27 10:36:17,612][20381] Updated weights for policy 1, policy_version 718894 (0.0005) [2023-12-27 10:36:17,658][20347] Updated weights for policy 0, policy_version 719358 (0.0008) [2023-12-27 10:36:17,668][20381] Updated weights for policy 1, policy_version 718904 (0.0006) [2023-12-27 10:36:17,709][20347] Updated weights for policy 0, policy_version 719368 (0.0008) [2023-12-27 10:36:18,266][20381] Updated weights for policy 1, policy_version 718914 (0.0008) [2023-12-27 10:36:18,328][20381] Updated weights for policy 1, policy_version 718924 (0.0008) [2023-12-27 10:36:18,389][20381] Updated weights for policy 1, policy_version 718934 (0.0007) [2023-12-27 10:36:18,441][20347] Updated weights for policy 0, policy_version 719378 (0.0008) [2023-12-27 10:36:18,507][20347] Updated weights for policy 0, policy_version 719388 (0.0007) [2023-12-27 10:36:18,566][20347] Updated weights for policy 0, policy_version 719398 (0.0007) [2023-12-27 10:36:19,112][20381] Updated weights for policy 1, policy_version 718944 (0.0007) [2023-12-27 10:36:19,167][20381] Updated weights for policy 1, policy_version 718954 (0.0007) [2023-12-27 10:36:19,231][20381] Updated weights for policy 1, policy_version 718964 (0.0008) [2023-12-27 10:36:19,275][20347] Updated weights for policy 0, policy_version 719408 (0.0009) [2023-12-27 10:36:19,335][20347] Updated weights for policy 0, policy_version 719418 (0.0007) [2023-12-27 10:36:19,405][20347] Updated weights for policy 0, policy_version 719428 (0.0008) [2023-12-27 10:36:19,991][20381] Updated weights for policy 1, policy_version 718974 (0.0008) [2023-12-27 10:36:20,056][20381] Updated weights for policy 1, policy_version 718984 (0.0007) [2023-12-27 10:36:20,117][20381] Updated weights for policy 1, policy_version 718994 (0.0006) [2023-12-27 10:36:20,192][20347] Updated weights for policy 0, policy_version 719438 (0.0008) [2023-12-27 10:36:20,255][20347] Updated weights for policy 0, policy_version 719448 (0.0008) [2023-12-27 10:36:20,315][20347] Updated weights for policy 0, policy_version 719458 (0.0008) [2023-12-27 10:36:20,835][20381] Updated weights for policy 1, policy_version 719004 (0.0007) [2023-12-27 10:36:20,889][20381] Updated weights for policy 1, policy_version 719014 (0.0008) [2023-12-27 10:36:20,940][20381] Updated weights for policy 1, policy_version 719024 (0.0008) [2023-12-27 10:36:21,092][20347] Updated weights for policy 0, policy_version 719468 (0.0008) [2023-12-27 10:36:21,149][20347] Updated weights for policy 0, policy_version 719478 (0.0008) [2023-12-27 10:36:21,210][20347] Updated weights for policy 0, policy_version 719488 (0.0009) [2023-12-27 10:36:21,678][20381] Updated weights for policy 1, policy_version 719034 (0.0007) [2023-12-27 10:36:21,740][20381] Updated weights for policy 1, policy_version 719044 (0.0008) [2023-12-27 10:36:21,801][20381] Updated weights for policy 1, policy_version 719054 (0.0008) [2023-12-27 10:36:21,861][20381] Updated weights for policy 1, policy_version 719064 (0.0008) [2023-12-27 10:36:22,001][20347] Updated weights for policy 0, policy_version 719498 (0.0009) [2023-12-27 10:36:22,057][20347] Updated weights for policy 0, policy_version 719508 (0.0007) [2023-12-27 10:36:22,115][20347] Updated weights for policy 0, policy_version 719518 (0.0005) [2023-12-27 10:36:22,161][20291] KL-divergence is very high: 207.5488 [2023-12-27 10:36:22,173][20291] KL-divergence is very high: 532.9731 [2023-12-27 10:36:22,179][20347] Updated weights for policy 0, policy_version 719528 (0.0007) [2023-12-27 10:36:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20623.2, 300 sec: 20466.1). Total num frames: 368328704. Throughput: 0: 10116.1, 1: 10363.4. Samples: 368300784. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:22,363][19170] Avg episode reward: [(0, '942.100'), (1, '762.370')] [2023-12-27 10:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000719064_184107008.pth... [2023-12-27 10:36:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000719528_184221696.pth... [2023-12-27 10:36:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000718344_183918592.pth [2023-12-27 10:36:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000717880_183803904.pth [2023-12-27 10:36:22,589][20381] Updated weights for policy 1, policy_version 719074 (0.0008) [2023-12-27 10:36:22,645][20381] Updated weights for policy 1, policy_version 719084 (0.0008) [2023-12-27 10:36:22,701][20381] Updated weights for policy 1, policy_version 719094 (0.0008) [2023-12-27 10:36:22,876][20291] KL-divergence is very high: 178.9365 [2023-12-27 10:36:22,905][20347] Updated weights for policy 0, policy_version 719538 (0.0007) [2023-12-27 10:36:22,911][20291] KL-divergence is very high: 109.8620 [2023-12-27 10:36:22,957][20347] Updated weights for policy 0, policy_version 719548 (0.0008) [2023-12-27 10:36:23,009][20291] KL-divergence is very high: 149.4354 [2023-12-27 10:36:23,021][20347] Updated weights for policy 0, policy_version 719558 (0.0006) [2023-12-27 10:36:23,408][20381] Updated weights for policy 1, policy_version 719104 (0.0008) [2023-12-27 10:36:23,469][20381] Updated weights for policy 1, policy_version 719114 (0.0008) [2023-12-27 10:36:23,528][20381] Updated weights for policy 1, policy_version 719124 (0.0009) [2023-12-27 10:36:23,603][20291] KL-divergence is very high: 122.7358 [2023-12-27 10:36:23,608][20347] Updated weights for policy 0, policy_version 719568 (0.0009) [2023-12-27 10:36:23,649][20291] KL-divergence is very high: 129.1169 [2023-12-27 10:36:23,666][20347] Updated weights for policy 0, policy_version 719578 (0.0010) [2023-12-27 10:36:23,695][20291] KL-divergence is very high: 138.4946 [2023-12-27 10:36:23,723][20347] Updated weights for policy 0, policy_version 719588 (0.0010) [2023-12-27 10:36:23,742][20291] KL-divergence is very high: 147.1280 [2023-12-27 10:36:24,149][20381] Updated weights for policy 1, policy_version 719134 (0.0006) [2023-12-27 10:36:24,210][20381] Updated weights for policy 1, policy_version 719144 (0.0008) [2023-12-27 10:36:24,268][20381] Updated weights for policy 1, policy_version 719154 (0.0008) [2023-12-27 10:36:24,275][20292] KL-divergence is very high: 105.1087 [2023-12-27 10:36:24,450][20347] Updated weights for policy 0, policy_version 719598 (0.0009) [2023-12-27 10:36:24,512][20347] Updated weights for policy 0, policy_version 719608 (0.0008) [2023-12-27 10:36:24,567][20347] Updated weights for policy 0, policy_version 719618 (0.0007) [2023-12-27 10:36:24,833][20381] Updated weights for policy 1, policy_version 719164 (0.0005) [2023-12-27 10:36:24,882][20381] Updated weights for policy 1, policy_version 719174 (0.0007) [2023-12-27 10:36:24,932][20381] Updated weights for policy 1, policy_version 719184 (0.0006) [2023-12-27 10:36:25,273][20347] Updated weights for policy 0, policy_version 719628 (0.0010) [2023-12-27 10:36:25,328][20347] Updated weights for policy 0, policy_version 719638 (0.0010) [2023-12-27 10:36:25,390][20347] Updated weights for policy 0, policy_version 719648 (0.0009) [2023-12-27 10:36:25,550][20381] Updated weights for policy 1, policy_version 719194 (0.0005) [2023-12-27 10:36:25,607][20381] Updated weights for policy 1, policy_version 719204 (0.0005) [2023-12-27 10:36:25,661][20381] Updated weights for policy 1, policy_version 719214 (0.0010) [2023-12-27 10:36:25,712][20381] Updated weights for policy 1, policy_version 719224 (0.0010) [2023-12-27 10:36:26,048][20347] Updated weights for policy 0, policy_version 719658 (0.0008) [2023-12-27 10:36:26,096][20347] Updated weights for policy 0, policy_version 719668 (0.0005) [2023-12-27 10:36:26,142][20347] Updated weights for policy 0, policy_version 719678 (0.0005) [2023-12-27 10:36:26,188][20347] Updated weights for policy 0, policy_version 719688 (0.0005) [2023-12-27 10:36:26,376][20381] Updated weights for policy 1, policy_version 719234 (0.0010) [2023-12-27 10:36:26,429][20381] Updated weights for policy 1, policy_version 719244 (0.0010) [2023-12-27 10:36:26,483][20381] Updated weights for policy 1, policy_version 719254 (0.0010) [2023-12-27 10:36:26,781][20347] Updated weights for policy 0, policy_version 719698 (0.0005) [2023-12-27 10:36:26,829][20347] Updated weights for policy 0, policy_version 719708 (0.0006) [2023-12-27 10:36:26,879][20347] Updated weights for policy 0, policy_version 719718 (0.0005) [2023-12-27 10:36:27,170][20381] Updated weights for policy 1, policy_version 719264 (0.0010) [2023-12-27 10:36:27,225][20381] Updated weights for policy 1, policy_version 719274 (0.0005) [2023-12-27 10:36:27,276][20381] Updated weights for policy 1, policy_version 719284 (0.0008) [2023-12-27 10:36:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 368435200. Throughput: 0: 10108.5, 1: 10329.1. Samples: 368421376. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:27,363][19170] Avg episode reward: [(0, '722.270'), (1, '805.090')] [2023-12-27 10:36:27,403][20347] Updated weights for policy 0, policy_version 719728 (0.0009) [2023-12-27 10:36:27,469][20347] Updated weights for policy 0, policy_version 719738 (0.0010) [2023-12-27 10:36:27,527][20347] Updated weights for policy 0, policy_version 719748 (0.0010) [2023-12-27 10:36:27,942][20381] Updated weights for policy 1, policy_version 719294 (0.0009) [2023-12-27 10:36:27,995][20381] Updated weights for policy 1, policy_version 719304 (0.0009) [2023-12-27 10:36:28,040][20381] Updated weights for policy 1, policy_version 719314 (0.0009) [2023-12-27 10:36:28,076][20347] Updated weights for policy 0, policy_version 719758 (0.0009) [2023-12-27 10:36:28,122][20347] Updated weights for policy 0, policy_version 719768 (0.0009) [2023-12-27 10:36:28,167][20347] Updated weights for policy 0, policy_version 719778 (0.0005) [2023-12-27 10:36:28,795][20381] Updated weights for policy 1, policy_version 719324 (0.0007) [2023-12-27 10:36:28,796][20347] Updated weights for policy 0, policy_version 719788 (0.0005) [2023-12-27 10:36:28,846][20347] Updated weights for policy 0, policy_version 719798 (0.0006) [2023-12-27 10:36:28,856][20381] Updated weights for policy 1, policy_version 719334 (0.0007) [2023-12-27 10:36:28,906][20347] Updated weights for policy 0, policy_version 719808 (0.0007) [2023-12-27 10:36:28,917][20381] Updated weights for policy 1, policy_version 719344 (0.0007) [2023-12-27 10:36:29,573][20291] KL-divergence is very high: 106.0448 [2023-12-27 10:36:29,580][20347] Updated weights for policy 0, policy_version 719818 (0.0007) [2023-12-27 10:36:29,634][20347] Updated weights for policy 0, policy_version 719828 (0.0009) [2023-12-27 10:36:29,635][20381] Updated weights for policy 1, policy_version 719354 (0.0007) [2023-12-27 10:36:29,697][20347] Updated weights for policy 0, policy_version 719838 (0.0011) [2023-12-27 10:36:29,699][20381] Updated weights for policy 1, policy_version 719364 (0.0007) [2023-12-27 10:36:29,749][20381] Updated weights for policy 1, policy_version 719374 (0.0006) [2023-12-27 10:36:29,754][20347] Updated weights for policy 0, policy_version 719848 (0.0011) [2023-12-27 10:36:29,802][20381] Updated weights for policy 1, policy_version 719384 (0.0008) [2023-12-27 10:36:30,473][20347] Updated weights for policy 0, policy_version 719858 (0.0010) [2023-12-27 10:36:30,514][20347] Updated weights for policy 0, policy_version 719868 (0.0010) [2023-12-27 10:36:30,519][20381] Updated weights for policy 1, policy_version 719394 (0.0006) [2023-12-27 10:36:30,569][20347] Updated weights for policy 0, policy_version 719878 (0.0010) [2023-12-27 10:36:30,577][20381] Updated weights for policy 1, policy_version 719404 (0.0005) [2023-12-27 10:36:30,634][20381] Updated weights for policy 1, policy_version 719414 (0.0007) [2023-12-27 10:36:31,186][20347] Updated weights for policy 0, policy_version 719888 (0.0008) [2023-12-27 10:36:31,240][20347] Updated weights for policy 0, policy_version 719898 (0.0006) [2023-12-27 10:36:31,280][20381] Updated weights for policy 1, policy_version 719424 (0.0007) [2023-12-27 10:36:31,295][20347] Updated weights for policy 0, policy_version 719908 (0.0008) [2023-12-27 10:36:31,341][20381] Updated weights for policy 1, policy_version 719434 (0.0009) [2023-12-27 10:36:31,410][20381] Updated weights for policy 1, policy_version 719444 (0.0010) [2023-12-27 10:36:31,990][20347] Updated weights for policy 0, policy_version 719918 (0.0006) [2023-12-27 10:36:32,043][20347] Updated weights for policy 0, policy_version 719928 (0.0006) [2023-12-27 10:36:32,102][20347] Updated weights for policy 0, policy_version 719938 (0.0008) [2023-12-27 10:36:32,152][20381] Updated weights for policy 1, policy_version 719454 (0.0008) [2023-12-27 10:36:32,199][20381] Updated weights for policy 1, policy_version 719464 (0.0007) [2023-12-27 10:36:32,244][20381] Updated weights for policy 1, policy_version 719474 (0.0009) [2023-12-27 10:36:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 368541696. Throughput: 0: 10264.3, 1: 10229.2. Samples: 368547452. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:32,363][19170] Avg episode reward: [(0, '774.450'), (1, '747.290')] [2023-12-27 10:36:32,757][20347] Updated weights for policy 0, policy_version 719948 (0.0006) [2023-12-27 10:36:32,815][20347] Updated weights for policy 0, policy_version 719958 (0.0008) [2023-12-27 10:36:32,864][20347] Updated weights for policy 0, policy_version 719968 (0.0009) [2023-12-27 10:36:32,964][20381] Updated weights for policy 1, policy_version 719484 (0.0007) [2023-12-27 10:36:33,030][20381] Updated weights for policy 1, policy_version 719494 (0.0005) [2023-12-27 10:36:33,092][20381] Updated weights for policy 1, policy_version 719504 (0.0006) [2023-12-27 10:36:33,591][20347] Updated weights for policy 0, policy_version 719978 (0.0008) [2023-12-27 10:36:33,647][20347] Updated weights for policy 0, policy_version 719988 (0.0010) [2023-12-27 10:36:33,699][20347] Updated weights for policy 0, policy_version 719998 (0.0010) [2023-12-27 10:36:33,711][20381] Updated weights for policy 1, policy_version 719514 (0.0006) [2023-12-27 10:36:33,754][20347] Updated weights for policy 0, policy_version 720008 (0.0010) [2023-12-27 10:36:33,760][20381] Updated weights for policy 1, policy_version 719524 (0.0005) [2023-12-27 10:36:33,809][20381] Updated weights for policy 1, policy_version 719534 (0.0007) [2023-12-27 10:36:33,859][20381] Updated weights for policy 1, policy_version 719544 (0.0007) [2023-12-27 10:36:34,417][20347] Updated weights for policy 0, policy_version 720018 (0.0006) [2023-12-27 10:36:34,477][20347] Updated weights for policy 0, policy_version 720028 (0.0007) [2023-12-27 10:36:34,536][20347] Updated weights for policy 0, policy_version 720038 (0.0007) [2023-12-27 10:36:34,567][20381] Updated weights for policy 1, policy_version 719554 (0.0008) [2023-12-27 10:36:34,616][20381] Updated weights for policy 1, policy_version 719564 (0.0007) [2023-12-27 10:36:34,666][20381] Updated weights for policy 1, policy_version 719574 (0.0007) [2023-12-27 10:36:35,214][20347] Updated weights for policy 0, policy_version 720048 (0.0007) [2023-12-27 10:36:35,269][20347] Updated weights for policy 0, policy_version 720058 (0.0007) [2023-12-27 10:36:35,324][20347] Updated weights for policy 0, policy_version 720068 (0.0007) [2023-12-27 10:36:35,351][20381] Updated weights for policy 1, policy_version 719584 (0.0006) [2023-12-27 10:36:35,407][20381] Updated weights for policy 1, policy_version 719594 (0.0006) [2023-12-27 10:36:35,463][20381] Updated weights for policy 1, policy_version 719604 (0.0005) [2023-12-27 10:36:35,887][20347] Updated weights for policy 0, policy_version 720078 (0.0007) [2023-12-27 10:36:35,947][20347] Updated weights for policy 0, policy_version 720088 (0.0008) [2023-12-27 10:36:36,006][20347] Updated weights for policy 0, policy_version 720098 (0.0006) [2023-12-27 10:36:36,065][20381] Updated weights for policy 1, policy_version 719614 (0.0007) [2023-12-27 10:36:36,133][20381] Updated weights for policy 1, policy_version 719624 (0.0008) [2023-12-27 10:36:36,187][20381] Updated weights for policy 1, policy_version 719634 (0.0008) [2023-12-27 10:36:36,677][20347] Updated weights for policy 0, policy_version 720108 (0.0008) [2023-12-27 10:36:36,730][20347] Updated weights for policy 0, policy_version 720118 (0.0011) [2023-12-27 10:36:36,786][20347] Updated weights for policy 0, policy_version 720128 (0.0010) [2023-12-27 10:36:36,892][20381] Updated weights for policy 1, policy_version 719644 (0.0008) [2023-12-27 10:36:36,945][20381] Updated weights for policy 1, policy_version 719654 (0.0008) [2023-12-27 10:36:37,005][20381] Updated weights for policy 1, policy_version 719664 (0.0008) [2023-12-27 10:36:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 368640000. Throughput: 0: 10259.5, 1: 10205.9. Samples: 368608396. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:37,363][19170] Avg episode reward: [(0, '295.930'), (1, '717.870')] [2023-12-27 10:36:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000719672_184262656.pth... [2023-12-27 10:36:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000720136_184377344.pth... [2023-12-27 10:36:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000718488_183959552.pth [2023-12-27 10:36:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000718952_184074240.pth [2023-12-27 10:36:37,459][20347] Updated weights for policy 0, policy_version 720138 (0.0010) [2023-12-27 10:36:37,508][20347] Updated weights for policy 0, policy_version 720148 (0.0010) [2023-12-27 10:36:37,513][20291] KL-divergence is very high: 137.9947 [2023-12-27 10:36:37,560][20347] Updated weights for policy 0, policy_version 720158 (0.0010) [2023-12-27 10:36:37,581][20291] KL-divergence is very high: 107.0345 [2023-12-27 10:36:37,588][20291] KL-divergence is very high: 171.6050 [2023-12-27 10:36:37,601][20291] KL-divergence is very high: 364.1041 [2023-12-27 10:36:37,619][20347] Updated weights for policy 0, policy_version 720168 (0.0010) [2023-12-27 10:36:37,725][20381] Updated weights for policy 1, policy_version 719674 (0.0008) [2023-12-27 10:36:37,784][20381] Updated weights for policy 1, policy_version 719684 (0.0007) [2023-12-27 10:36:37,839][20381] Updated weights for policy 1, policy_version 719694 (0.0007) [2023-12-27 10:36:37,887][20381] Updated weights for policy 1, policy_version 719704 (0.0007) [2023-12-27 10:36:38,252][20291] KL-divergence is very high: 150.4270 [2023-12-27 10:36:38,258][20291] KL-divergence is very high: 122.4423 [2023-12-27 10:36:38,293][20347] Updated weights for policy 0, policy_version 720178 (0.0010) [2023-12-27 10:36:38,293][20291] KL-divergence is very high: 115.0968 [2023-12-27 10:36:38,353][20347] Updated weights for policy 0, policy_version 720188 (0.0010) [2023-12-27 10:36:38,415][20347] Updated weights for policy 0, policy_version 720198 (0.0011) [2023-12-27 10:36:38,625][20381] Updated weights for policy 1, policy_version 719714 (0.0008) [2023-12-27 10:36:38,682][20381] Updated weights for policy 1, policy_version 719724 (0.0007) [2023-12-27 10:36:38,737][20381] Updated weights for policy 1, policy_version 719734 (0.0008) [2023-12-27 10:36:39,126][20347] Updated weights for policy 0, policy_version 720208 (0.0008) [2023-12-27 10:36:39,188][20347] Updated weights for policy 0, policy_version 720218 (0.0005) [2023-12-27 10:36:39,255][20347] Updated weights for policy 0, policy_version 720228 (0.0007) [2023-12-27 10:36:39,470][20381] Updated weights for policy 1, policy_version 719744 (0.0010) [2023-12-27 10:36:39,532][20381] Updated weights for policy 1, policy_version 719754 (0.0007) [2023-12-27 10:36:39,590][20381] Updated weights for policy 1, policy_version 719764 (0.0008) [2023-12-27 10:36:39,873][20347] Updated weights for policy 0, policy_version 720238 (0.0006) [2023-12-27 10:36:39,937][20347] Updated weights for policy 0, policy_version 720248 (0.0006) [2023-12-27 10:36:39,999][20347] Updated weights for policy 0, policy_version 720258 (0.0006) [2023-12-27 10:36:40,332][20381] Updated weights for policy 1, policy_version 719774 (0.0008) [2023-12-27 10:36:40,396][20381] Updated weights for policy 1, policy_version 719784 (0.0008) [2023-12-27 10:36:40,455][20381] Updated weights for policy 1, policy_version 719794 (0.0008) [2023-12-27 10:36:40,670][20347] Updated weights for policy 0, policy_version 720268 (0.0008) [2023-12-27 10:36:40,727][20347] Updated weights for policy 0, policy_version 720278 (0.0007) [2023-12-27 10:36:40,782][20347] Updated weights for policy 0, policy_version 720288 (0.0008) [2023-12-27 10:36:41,202][20381] Updated weights for policy 1, policy_version 719804 (0.0007) [2023-12-27 10:36:41,256][20381] Updated weights for policy 1, policy_version 719814 (0.0008) [2023-12-27 10:36:41,316][20381] Updated weights for policy 1, policy_version 719824 (0.0008) [2023-12-27 10:36:41,486][20347] Updated weights for policy 0, policy_version 720298 (0.0008) [2023-12-27 10:36:41,546][20347] Updated weights for policy 0, policy_version 720308 (0.0011) [2023-12-27 10:36:41,599][20347] Updated weights for policy 0, policy_version 720318 (0.0010) [2023-12-27 10:36:41,669][20347] Updated weights for policy 0, policy_version 720328 (0.0009) [2023-12-27 10:36:42,013][20381] Updated weights for policy 1, policy_version 719834 (0.0008) [2023-12-27 10:36:42,072][20381] Updated weights for policy 1, policy_version 719844 (0.0007) [2023-12-27 10:36:42,130][20381] Updated weights for policy 1, policy_version 719854 (0.0007) [2023-12-27 10:36:42,185][20381] Updated weights for policy 1, policy_version 719864 (0.0007) [2023-12-27 10:36:42,362][20347] Updated weights for policy 0, policy_version 720338 (0.0007) [2023-12-27 10:36:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 368738304. Throughput: 0: 10296.3, 1: 10108.8. Samples: 368729696. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 10:36:42,363][19170] Avg episode reward: [(0, '288.540'), (1, '737.100')] [2023-12-27 10:36:42,426][20347] Updated weights for policy 0, policy_version 720348 (0.0007) [2023-12-27 10:36:42,491][20347] Updated weights for policy 0, policy_version 720358 (0.0007) [2023-12-27 10:36:42,870][20381] Updated weights for policy 1, policy_version 719874 (0.0007) [2023-12-27 10:36:42,916][20381] Updated weights for policy 1, policy_version 719884 (0.0008) [2023-12-27 10:36:42,960][20381] Updated weights for policy 1, policy_version 719894 (0.0007) [2023-12-27 10:36:43,157][20347] Updated weights for policy 0, policy_version 720368 (0.0008) [2023-12-27 10:36:43,164][20291] KL-divergence is very high: 137.9965 [2023-12-27 10:36:43,213][20347] Updated weights for policy 0, policy_version 720378 (0.0009) [2023-12-27 10:36:43,270][20347] Updated weights for policy 0, policy_version 720388 (0.0009) [2023-12-27 10:36:43,614][20381] Updated weights for policy 1, policy_version 719904 (0.0007) [2023-12-27 10:36:43,665][20381] Updated weights for policy 1, policy_version 719914 (0.0007) [2023-12-27 10:36:43,725][20381] Updated weights for policy 1, policy_version 719924 (0.0008) [2023-12-27 10:36:43,890][20347] Updated weights for policy 0, policy_version 720398 (0.0007) [2023-12-27 10:36:43,943][20347] Updated weights for policy 0, policy_version 720408 (0.0006) [2023-12-27 10:36:43,996][20347] Updated weights for policy 0, policy_version 720418 (0.0006) [2023-12-27 10:36:44,381][20381] Updated weights for policy 1, policy_version 719934 (0.0008) [2023-12-27 10:36:44,439][20381] Updated weights for policy 1, policy_version 719944 (0.0007) [2023-12-27 10:36:44,501][20381] Updated weights for policy 1, policy_version 719954 (0.0008) [2023-12-27 10:36:44,659][20347] Updated weights for policy 0, policy_version 720428 (0.0007) [2023-12-27 10:36:44,725][20347] Updated weights for policy 0, policy_version 720438 (0.0005) [2023-12-27 10:36:44,791][20347] Updated weights for policy 0, policy_version 720448 (0.0008) [2023-12-27 10:36:45,149][20381] Updated weights for policy 1, policy_version 719964 (0.0007) [2023-12-27 10:36:45,212][20381] Updated weights for policy 1, policy_version 719974 (0.0008) [2023-12-27 10:36:45,278][20381] Updated weights for policy 1, policy_version 719984 (0.0008) [2023-12-27 10:36:45,468][20347] Updated weights for policy 0, policy_version 720458 (0.0008) [2023-12-27 10:36:45,522][20347] Updated weights for policy 0, policy_version 720468 (0.0008) [2023-12-27 10:36:45,583][20347] Updated weights for policy 0, policy_version 720478 (0.0007) [2023-12-27 10:36:45,632][20347] Updated weights for policy 0, policy_version 720488 (0.0007) [2023-12-27 10:36:45,954][20381] Updated weights for policy 1, policy_version 719994 (0.0008) [2023-12-27 10:36:46,000][20381] Updated weights for policy 1, policy_version 720004 (0.0009) [2023-12-27 10:36:46,051][20381] Updated weights for policy 1, policy_version 720014 (0.0005) [2023-12-27 10:36:46,105][20381] Updated weights for policy 1, policy_version 720024 (0.0007) [2023-12-27 10:36:46,258][20347] Updated weights for policy 0, policy_version 720498 (0.0005) [2023-12-27 10:36:46,315][20347] Updated weights for policy 0, policy_version 720508 (0.0005) [2023-12-27 10:36:46,364][20347] Updated weights for policy 0, policy_version 720518 (0.0008) [2023-12-27 10:36:46,682][20381] Updated weights for policy 1, policy_version 720034 (0.0007) [2023-12-27 10:36:46,733][20381] Updated weights for policy 1, policy_version 720044 (0.0006) [2023-12-27 10:36:46,781][20381] Updated weights for policy 1, policy_version 720054 (0.0005) [2023-12-27 10:36:47,074][20347] Updated weights for policy 0, policy_version 720528 (0.0008) [2023-12-27 10:36:47,122][20347] Updated weights for policy 0, policy_version 720538 (0.0007) [2023-12-27 10:36:47,174][20347] Updated weights for policy 0, policy_version 720548 (0.0007) [2023-12-27 10:36:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 368844800. Throughput: 0: 10300.1, 1: 10147.6. Samples: 368854552. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:36:47,363][19170] Avg episode reward: [(0, '208.470'), (1, '766.020')] [2023-12-27 10:36:47,444][20381] Updated weights for policy 1, policy_version 720064 (0.0006) [2023-12-27 10:36:47,507][20381] Updated weights for policy 1, policy_version 720074 (0.0005) [2023-12-27 10:36:47,560][20381] Updated weights for policy 1, policy_version 720084 (0.0005) [2023-12-27 10:36:47,906][20347] Updated weights for policy 0, policy_version 720558 (0.0007) [2023-12-27 10:36:47,959][20347] Updated weights for policy 0, policy_version 720568 (0.0008) [2023-12-27 10:36:48,016][20347] Updated weights for policy 0, policy_version 720578 (0.0008) [2023-12-27 10:36:48,122][20381] Updated weights for policy 1, policy_version 720094 (0.0005) [2023-12-27 10:36:48,174][20381] Updated weights for policy 1, policy_version 720104 (0.0010) [2023-12-27 10:36:48,233][20381] Updated weights for policy 1, policy_version 720114 (0.0011) [2023-12-27 10:36:48,733][20347] Updated weights for policy 0, policy_version 720588 (0.0008) [2023-12-27 10:36:48,789][20347] Updated weights for policy 0, policy_version 720598 (0.0008) [2023-12-27 10:36:48,847][20347] Updated weights for policy 0, policy_version 720608 (0.0008) [2023-12-27 10:36:48,896][20381] Updated weights for policy 1, policy_version 720124 (0.0009) [2023-12-27 10:36:48,948][20381] Updated weights for policy 1, policy_version 720134 (0.0005) [2023-12-27 10:36:48,996][20381] Updated weights for policy 1, policy_version 720144 (0.0008) [2023-12-27 10:36:49,570][20347] Updated weights for policy 0, policy_version 720618 (0.0007) [2023-12-27 10:36:49,621][20347] Updated weights for policy 0, policy_version 720628 (0.0007) [2023-12-27 10:36:49,665][20381] Updated weights for policy 1, policy_version 720154 (0.0009) [2023-12-27 10:36:49,680][20347] Updated weights for policy 0, policy_version 720638 (0.0007) [2023-12-27 10:36:49,725][20381] Updated weights for policy 1, policy_version 720164 (0.0009) [2023-12-27 10:36:49,738][20347] Updated weights for policy 0, policy_version 720648 (0.0007) [2023-12-27 10:36:49,789][20381] Updated weights for policy 1, policy_version 720174 (0.0011) [2023-12-27 10:36:49,852][20381] Updated weights for policy 1, policy_version 720184 (0.0011) [2023-12-27 10:36:50,466][20347] Updated weights for policy 0, policy_version 720658 (0.0006) [2023-12-27 10:36:50,520][20347] Updated weights for policy 0, policy_version 720668 (0.0006) [2023-12-27 10:36:50,537][20381] Updated weights for policy 1, policy_version 720194 (0.0007) [2023-12-27 10:36:50,584][20347] Updated weights for policy 0, policy_version 720678 (0.0007) [2023-12-27 10:36:50,602][20381] Updated weights for policy 1, policy_version 720204 (0.0008) [2023-12-27 10:36:50,662][20381] Updated weights for policy 1, policy_version 720214 (0.0007) [2023-12-27 10:36:51,279][20347] Updated weights for policy 0, policy_version 720688 (0.0008) [2023-12-27 10:36:51,339][20347] Updated weights for policy 0, policy_version 720698 (0.0008) [2023-12-27 10:36:51,415][20347] Updated weights for policy 0, policy_version 720708 (0.0008) [2023-12-27 10:36:51,436][20381] Updated weights for policy 1, policy_version 720224 (0.0007) [2023-12-27 10:36:51,483][20381] Updated weights for policy 1, policy_version 720234 (0.0009) [2023-12-27 10:36:51,544][20381] Updated weights for policy 1, policy_version 720244 (0.0009) [2023-12-27 10:36:52,125][20347] Updated weights for policy 0, policy_version 720718 (0.0006) [2023-12-27 10:36:52,184][20347] Updated weights for policy 0, policy_version 720728 (0.0006) [2023-12-27 10:36:52,238][20381] Updated weights for policy 1, policy_version 720254 (0.0008) [2023-12-27 10:36:52,243][20347] Updated weights for policy 0, policy_version 720738 (0.0006) [2023-12-27 10:36:52,305][20381] Updated weights for policy 1, policy_version 720264 (0.0008) [2023-12-27 10:36:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 368943104. Throughput: 0: 10254.6, 1: 10191.9. Samples: 368916076. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:36:52,363][19170] Avg episode reward: [(0, '274.370'), (1, '717.830')] [2023-12-27 10:36:52,364][20381] Updated weights for policy 1, policy_version 720274 (0.0008) [2023-12-27 10:36:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000720744_184532992.pth... [2023-12-27 10:36:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000719528_184221696.pth [2023-12-27 10:36:52,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000720280_184418304.pth... [2023-12-27 10:36:52,416][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000719064_184107008.pth [2023-12-27 10:36:52,972][20347] Updated weights for policy 0, policy_version 720748 (0.0007) [2023-12-27 10:36:53,027][20347] Updated weights for policy 0, policy_version 720758 (0.0008) [2023-12-27 10:36:53,084][20347] Updated weights for policy 0, policy_version 720768 (0.0007) [2023-12-27 10:36:53,102][20381] Updated weights for policy 1, policy_version 720284 (0.0008) [2023-12-27 10:36:53,153][20381] Updated weights for policy 1, policy_version 720294 (0.0008) [2023-12-27 10:36:53,203][20381] Updated weights for policy 1, policy_version 720304 (0.0008) [2023-12-27 10:36:53,763][20347] Updated weights for policy 0, policy_version 720778 (0.0008) [2023-12-27 10:36:53,814][20347] Updated weights for policy 0, policy_version 720788 (0.0006) [2023-12-27 10:36:53,863][20347] Updated weights for policy 0, policy_version 720798 (0.0005) [2023-12-27 10:36:53,870][20381] Updated weights for policy 1, policy_version 720314 (0.0007) [2023-12-27 10:36:53,917][20347] Updated weights for policy 0, policy_version 720808 (0.0007) [2023-12-27 10:36:53,932][20381] Updated weights for policy 1, policy_version 720324 (0.0005) [2023-12-27 10:36:53,988][20381] Updated weights for policy 1, policy_version 720334 (0.0005) [2023-12-27 10:36:54,038][20381] Updated weights for policy 1, policy_version 720344 (0.0005) [2023-12-27 10:36:54,562][20381] Updated weights for policy 1, policy_version 720354 (0.0007) [2023-12-27 10:36:54,603][20347] Updated weights for policy 0, policy_version 720818 (0.0009) [2023-12-27 10:36:54,622][20381] Updated weights for policy 1, policy_version 720364 (0.0010) [2023-12-27 10:36:54,646][20347] Updated weights for policy 0, policy_version 720828 (0.0005) [2023-12-27 10:36:54,667][20381] Updated weights for policy 1, policy_version 720374 (0.0010) [2023-12-27 10:36:54,696][20347] Updated weights for policy 0, policy_version 720838 (0.0006) [2023-12-27 10:36:55,352][20381] Updated weights for policy 1, policy_version 720384 (0.0007) [2023-12-27 10:36:55,403][20381] Updated weights for policy 1, policy_version 720394 (0.0007) [2023-12-27 10:36:55,449][20347] Updated weights for policy 0, policy_version 720848 (0.0008) [2023-12-27 10:36:55,460][20381] Updated weights for policy 1, policy_version 720404 (0.0005) [2023-12-27 10:36:55,495][20347] Updated weights for policy 0, policy_version 720858 (0.0007) [2023-12-27 10:36:55,545][20347] Updated weights for policy 0, policy_version 720868 (0.0009) [2023-12-27 10:36:56,034][20381] Updated weights for policy 1, policy_version 720414 (0.0006) [2023-12-27 10:36:56,088][20381] Updated weights for policy 1, policy_version 720424 (0.0008) [2023-12-27 10:36:56,135][20381] Updated weights for policy 1, policy_version 720434 (0.0008) [2023-12-27 10:36:56,288][20347] Updated weights for policy 0, policy_version 720878 (0.0008) [2023-12-27 10:36:56,339][20347] Updated weights for policy 0, policy_version 720888 (0.0008) [2023-12-27 10:36:56,382][20347] Updated weights for policy 0, policy_version 720898 (0.0006) [2023-12-27 10:36:56,791][20381] Updated weights for policy 1, policy_version 720444 (0.0007) [2023-12-27 10:36:56,843][20381] Updated weights for policy 1, policy_version 720454 (0.0005) [2023-12-27 10:36:56,895][20381] Updated weights for policy 1, policy_version 720464 (0.0009) [2023-12-27 10:36:57,121][20347] Updated weights for policy 0, policy_version 720908 (0.0008) [2023-12-27 10:36:57,171][20347] Updated weights for policy 0, policy_version 720918 (0.0007) [2023-12-27 10:36:57,234][20347] Updated weights for policy 0, policy_version 720928 (0.0007) [2023-12-27 10:36:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 369049600. Throughput: 0: 10152.2, 1: 10208.5. Samples: 369037428. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:36:57,363][19170] Avg episode reward: [(0, '224.640'), (1, '790.680')] [2023-12-27 10:36:57,515][20381] Updated weights for policy 1, policy_version 720474 (0.0010) [2023-12-27 10:36:57,563][20381] Updated weights for policy 1, policy_version 720484 (0.0005) [2023-12-27 10:36:57,622][20381] Updated weights for policy 1, policy_version 720494 (0.0005) [2023-12-27 10:36:57,678][20381] Updated weights for policy 1, policy_version 720504 (0.0005) [2023-12-27 10:36:57,913][20347] Updated weights for policy 0, policy_version 720938 (0.0008) [2023-12-27 10:36:57,967][20347] Updated weights for policy 0, policy_version 720948 (0.0010) [2023-12-27 10:36:58,016][20347] Updated weights for policy 0, policy_version 720958 (0.0010) [2023-12-27 10:36:58,074][20347] Updated weights for policy 0, policy_version 720968 (0.0010) [2023-12-27 10:36:58,199][20381] Updated weights for policy 1, policy_version 720514 (0.0006) [2023-12-27 10:36:58,256][20381] Updated weights for policy 1, policy_version 720524 (0.0008) [2023-12-27 10:36:58,316][20381] Updated weights for policy 1, policy_version 720534 (0.0007) [2023-12-27 10:36:58,795][20347] Updated weights for policy 0, policy_version 720978 (0.0008) [2023-12-27 10:36:58,860][20347] Updated weights for policy 0, policy_version 720988 (0.0008) [2023-12-27 10:36:58,924][20347] Updated weights for policy 0, policy_version 720998 (0.0007) [2023-12-27 10:36:59,091][20381] Updated weights for policy 1, policy_version 720544 (0.0010) [2023-12-27 10:36:59,147][20381] Updated weights for policy 1, policy_version 720554 (0.0011) [2023-12-27 10:36:59,211][20381] Updated weights for policy 1, policy_version 720564 (0.0010) [2023-12-27 10:36:59,672][20347] Updated weights for policy 0, policy_version 721008 (0.0007) [2023-12-27 10:36:59,729][20347] Updated weights for policy 0, policy_version 721018 (0.0006) [2023-12-27 10:36:59,779][20347] Updated weights for policy 0, policy_version 721028 (0.0006) [2023-12-27 10:36:59,957][20381] Updated weights for policy 1, policy_version 720574 (0.0010) [2023-12-27 10:37:00,010][20381] Updated weights for policy 1, policy_version 720584 (0.0011) [2023-12-27 10:37:00,065][20381] Updated weights for policy 1, policy_version 720594 (0.0008) [2023-12-27 10:37:00,453][20347] Updated weights for policy 0, policy_version 721038 (0.0007) [2023-12-27 10:37:00,504][20347] Updated weights for policy 0, policy_version 721048 (0.0007) [2023-12-27 10:37:00,555][20347] Updated weights for policy 0, policy_version 721058 (0.0006) [2023-12-27 10:37:00,757][20381] Updated weights for policy 1, policy_version 720604 (0.0007) [2023-12-27 10:37:00,814][20381] Updated weights for policy 1, policy_version 720614 (0.0006) [2023-12-27 10:37:00,870][20381] Updated weights for policy 1, policy_version 720624 (0.0006) [2023-12-27 10:37:01,179][20347] Updated weights for policy 0, policy_version 721068 (0.0006) [2023-12-27 10:37:01,241][20347] Updated weights for policy 0, policy_version 721078 (0.0008) [2023-12-27 10:37:01,304][20347] Updated weights for policy 0, policy_version 721088 (0.0007) [2023-12-27 10:37:01,573][20381] Updated weights for policy 1, policy_version 720634 (0.0008) [2023-12-27 10:37:01,628][20381] Updated weights for policy 1, policy_version 720644 (0.0009) [2023-12-27 10:37:01,681][20381] Updated weights for policy 1, policy_version 720654 (0.0008) [2023-12-27 10:37:01,744][20381] Updated weights for policy 1, policy_version 720664 (0.0008) [2023-12-27 10:37:01,968][20347] Updated weights for policy 0, policy_version 721098 (0.0007) [2023-12-27 10:37:02,022][20347] Updated weights for policy 0, policy_version 721108 (0.0007) [2023-12-27 10:37:02,073][20347] Updated weights for policy 0, policy_version 721118 (0.0008) [2023-12-27 10:37:02,139][20347] Updated weights for policy 0, policy_version 721128 (0.0008) [2023-12-27 10:37:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 369147904. Throughput: 0: 10154.2, 1: 10215.4. Samples: 369159164. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:02,363][19170] Avg episode reward: [(0, '191.230'), (1, '863.700')] [2023-12-27 10:37:02,417][20381] Updated weights for policy 1, policy_version 720674 (0.0007) [2023-12-27 10:37:02,463][20381] Updated weights for policy 1, policy_version 720684 (0.0007) [2023-12-27 10:37:02,513][20381] Updated weights for policy 1, policy_version 720694 (0.0007) [2023-12-27 10:37:02,900][20347] Updated weights for policy 0, policy_version 721138 (0.0008) [2023-12-27 10:37:02,957][20347] Updated weights for policy 0, policy_version 721148 (0.0008) [2023-12-27 10:37:03,011][20347] Updated weights for policy 0, policy_version 721158 (0.0008) [2023-12-27 10:37:03,167][20381] Updated weights for policy 1, policy_version 720704 (0.0006) [2023-12-27 10:37:03,226][20381] Updated weights for policy 1, policy_version 720714 (0.0005) [2023-12-27 10:37:03,275][20381] Updated weights for policy 1, policy_version 720724 (0.0006) [2023-12-27 10:37:03,616][20347] Updated weights for policy 0, policy_version 721168 (0.0008) [2023-12-27 10:37:03,674][20347] Updated weights for policy 0, policy_version 721178 (0.0007) [2023-12-27 10:37:03,733][20347] Updated weights for policy 0, policy_version 721188 (0.0007) [2023-12-27 10:37:03,969][20381] Updated weights for policy 1, policy_version 720734 (0.0008) [2023-12-27 10:37:04,030][20381] Updated weights for policy 1, policy_version 720744 (0.0008) [2023-12-27 10:37:04,079][20381] Updated weights for policy 1, policy_version 720754 (0.0008) [2023-12-27 10:37:04,422][20347] Updated weights for policy 0, policy_version 721198 (0.0007) [2023-12-27 10:37:04,475][20347] Updated weights for policy 0, policy_version 721208 (0.0008) [2023-12-27 10:37:04,527][20347] Updated weights for policy 0, policy_version 721218 (0.0008) [2023-12-27 10:37:04,738][20381] Updated weights for policy 1, policy_version 720764 (0.0008) [2023-12-27 10:37:04,799][20381] Updated weights for policy 1, policy_version 720774 (0.0007) [2023-12-27 10:37:04,855][20381] Updated weights for policy 1, policy_version 720784 (0.0008) [2023-12-27 10:37:05,191][20347] Updated weights for policy 0, policy_version 721228 (0.0008) [2023-12-27 10:37:05,243][20347] Updated weights for policy 0, policy_version 721238 (0.0007) [2023-12-27 10:37:05,285][20347] Updated weights for policy 0, policy_version 721248 (0.0008) [2023-12-27 10:37:05,552][20381] Updated weights for policy 1, policy_version 720794 (0.0008) [2023-12-27 10:37:05,609][20381] Updated weights for policy 1, policy_version 720804 (0.0008) [2023-12-27 10:37:05,665][20381] Updated weights for policy 1, policy_version 720814 (0.0007) [2023-12-27 10:37:05,719][20381] Updated weights for policy 1, policy_version 720824 (0.0007) [2023-12-27 10:37:05,948][20347] Updated weights for policy 0, policy_version 721258 (0.0008) [2023-12-27 10:37:06,008][20347] Updated weights for policy 0, policy_version 721268 (0.0006) [2023-12-27 10:37:06,071][20347] Updated weights for policy 0, policy_version 721278 (0.0008) [2023-12-27 10:37:06,134][20347] Updated weights for policy 0, policy_version 721288 (0.0009) [2023-12-27 10:37:06,369][20381] Updated weights for policy 1, policy_version 720834 (0.0006) [2023-12-27 10:37:06,435][20381] Updated weights for policy 1, policy_version 720844 (0.0006) [2023-12-27 10:37:06,495][20381] Updated weights for policy 1, policy_version 720854 (0.0007) [2023-12-27 10:37:06,824][20347] Updated weights for policy 0, policy_version 721298 (0.0008) [2023-12-27 10:37:06,881][20347] Updated weights for policy 0, policy_version 721308 (0.0008) [2023-12-27 10:37:06,939][20347] Updated weights for policy 0, policy_version 721318 (0.0008) [2023-12-27 10:37:07,153][20381] Updated weights for policy 1, policy_version 720864 (0.0008) [2023-12-27 10:37:07,209][20381] Updated weights for policy 1, policy_version 720874 (0.0008) [2023-12-27 10:37:07,263][20381] Updated weights for policy 1, policy_version 720884 (0.0007) [2023-12-27 10:37:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 369254400. Throughput: 0: 10196.1, 1: 10249.9. Samples: 369220852. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:07,363][19170] Avg episode reward: [(0, '236.940'), (1, '883.160')] [2023-12-27 10:37:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000720888_184573952.pth... [2023-12-27 10:37:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000721320_184680448.pth... [2023-12-27 10:37:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000719672_184262656.pth [2023-12-27 10:37:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000720136_184377344.pth [2023-12-27 10:37:07,625][20347] Updated weights for policy 0, policy_version 721328 (0.0008) [2023-12-27 10:37:07,686][20347] Updated weights for policy 0, policy_version 721338 (0.0008) [2023-12-27 10:37:07,746][20347] Updated weights for policy 0, policy_version 721348 (0.0008) [2023-12-27 10:37:07,931][20381] Updated weights for policy 1, policy_version 720894 (0.0007) [2023-12-27 10:37:07,980][20381] Updated weights for policy 1, policy_version 720904 (0.0010) [2023-12-27 10:37:08,042][20381] Updated weights for policy 1, policy_version 720914 (0.0011) [2023-12-27 10:37:08,467][20347] Updated weights for policy 0, policy_version 721358 (0.0008) [2023-12-27 10:37:08,526][20347] Updated weights for policy 0, policy_version 721368 (0.0005) [2023-12-27 10:37:08,587][20347] Updated weights for policy 0, policy_version 721378 (0.0011) [2023-12-27 10:37:08,655][20381] Updated weights for policy 1, policy_version 720924 (0.0010) [2023-12-27 10:37:08,708][20381] Updated weights for policy 1, policy_version 720934 (0.0007) [2023-12-27 10:37:08,756][20381] Updated weights for policy 1, policy_version 720944 (0.0007) [2023-12-27 10:37:09,173][20347] Updated weights for policy 0, policy_version 721388 (0.0009) [2023-12-27 10:37:09,225][20347] Updated weights for policy 0, policy_version 721398 (0.0008) [2023-12-27 10:37:09,286][20347] Updated weights for policy 0, policy_version 721408 (0.0006) [2023-12-27 10:37:09,522][20381] Updated weights for policy 1, policy_version 720954 (0.0007) [2023-12-27 10:37:09,586][20381] Updated weights for policy 1, policy_version 720964 (0.0007) [2023-12-27 10:37:09,642][20381] Updated weights for policy 1, policy_version 720974 (0.0011) [2023-12-27 10:37:09,705][20381] Updated weights for policy 1, policy_version 720984 (0.0009) [2023-12-27 10:37:10,129][20347] Updated weights for policy 0, policy_version 721418 (0.0007) [2023-12-27 10:37:10,183][20347] Updated weights for policy 0, policy_version 721428 (0.0009) [2023-12-27 10:37:10,239][20347] Updated weights for policy 0, policy_version 721438 (0.0008) [2023-12-27 10:37:10,305][20347] Updated weights for policy 0, policy_version 721448 (0.0006) [2023-12-27 10:37:10,442][20381] Updated weights for policy 1, policy_version 720994 (0.0011) [2023-12-27 10:37:10,502][20381] Updated weights for policy 1, policy_version 721004 (0.0011) [2023-12-27 10:37:10,559][20381] Updated weights for policy 1, policy_version 721014 (0.0011) [2023-12-27 10:37:10,935][20347] Updated weights for policy 0, policy_version 721458 (0.0008) [2023-12-27 10:37:10,988][20347] Updated weights for policy 0, policy_version 721468 (0.0008) [2023-12-27 10:37:11,043][20347] Updated weights for policy 0, policy_version 721478 (0.0008) [2023-12-27 10:37:11,299][20381] Updated weights for policy 1, policy_version 721024 (0.0009) [2023-12-27 10:37:11,374][20381] Updated weights for policy 1, policy_version 721034 (0.0008) [2023-12-27 10:37:11,463][20381] Updated weights for policy 1, policy_version 721044 (0.0006) [2023-12-27 10:37:11,809][20347] Updated weights for policy 0, policy_version 721488 (0.0008) [2023-12-27 10:37:11,865][20347] Updated weights for policy 0, policy_version 721498 (0.0008) [2023-12-27 10:37:11,925][20347] Updated weights for policy 0, policy_version 721508 (0.0008) [2023-12-27 10:37:12,192][20381] Updated weights for policy 1, policy_version 721054 (0.0005) [2023-12-27 10:37:12,251][20381] Updated weights for policy 1, policy_version 721064 (0.0005) [2023-12-27 10:37:12,313][20381] Updated weights for policy 1, policy_version 721074 (0.0007) [2023-12-27 10:37:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 369352704. Throughput: 0: 10230.6, 1: 10226.7. Samples: 369341960. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:12,363][19170] Avg episode reward: [(0, '532.940'), (1, '892.910')] [2023-12-27 10:37:12,683][20347] Updated weights for policy 0, policy_version 721518 (0.0007) [2023-12-27 10:37:12,745][20347] Updated weights for policy 0, policy_version 721528 (0.0006) [2023-12-27 10:37:12,797][20347] Updated weights for policy 0, policy_version 721538 (0.0007) [2023-12-27 10:37:12,871][20381] Updated weights for policy 1, policy_version 721084 (0.0007) [2023-12-27 10:37:12,931][20381] Updated weights for policy 1, policy_version 721094 (0.0005) [2023-12-27 10:37:12,986][20381] Updated weights for policy 1, policy_version 721104 (0.0007) [2023-12-27 10:37:13,481][20347] Updated weights for policy 0, policy_version 721548 (0.0008) [2023-12-27 10:37:13,533][20347] Updated weights for policy 0, policy_version 721558 (0.0008) [2023-12-27 10:37:13,594][20347] Updated weights for policy 0, policy_version 721568 (0.0008) [2023-12-27 10:37:13,614][20381] Updated weights for policy 1, policy_version 721114 (0.0008) [2023-12-27 10:37:13,664][20381] Updated weights for policy 1, policy_version 721124 (0.0010) [2023-12-27 10:37:13,721][20381] Updated weights for policy 1, policy_version 721134 (0.0009) [2023-12-27 10:37:13,779][20381] Updated weights for policy 1, policy_version 721144 (0.0008) [2023-12-27 10:37:14,268][20347] Updated weights for policy 0, policy_version 721578 (0.0008) [2023-12-27 10:37:14,316][20347] Updated weights for policy 0, policy_version 721588 (0.0010) [2023-12-27 10:37:14,364][20347] Updated weights for policy 0, policy_version 721598 (0.0010) [2023-12-27 10:37:14,419][20347] Updated weights for policy 0, policy_version 721608 (0.0008) [2023-12-27 10:37:14,447][20381] Updated weights for policy 1, policy_version 721154 (0.0011) [2023-12-27 10:37:14,503][20381] Updated weights for policy 1, policy_version 721164 (0.0010) [2023-12-27 10:37:14,558][20381] Updated weights for policy 1, policy_version 721174 (0.0010) [2023-12-27 10:37:15,143][20347] Updated weights for policy 0, policy_version 721618 (0.0008) [2023-12-27 10:37:15,183][20381] Updated weights for policy 1, policy_version 721184 (0.0011) [2023-12-27 10:37:15,192][20347] Updated weights for policy 0, policy_version 721628 (0.0005) [2023-12-27 10:37:15,245][20347] Updated weights for policy 0, policy_version 721638 (0.0006) [2023-12-27 10:37:15,246][20381] Updated weights for policy 1, policy_version 721194 (0.0011) [2023-12-27 10:37:15,308][20381] Updated weights for policy 1, policy_version 721204 (0.0011) [2023-12-27 10:37:15,869][20347] Updated weights for policy 0, policy_version 721648 (0.0006) [2023-12-27 10:37:15,922][20347] Updated weights for policy 0, policy_version 721658 (0.0006) [2023-12-27 10:37:15,969][20381] Updated weights for policy 1, policy_version 721214 (0.0011) [2023-12-27 10:37:15,974][20347] Updated weights for policy 0, policy_version 721668 (0.0008) [2023-12-27 10:37:16,037][20381] Updated weights for policy 1, policy_version 721224 (0.0011) [2023-12-27 10:37:16,111][20381] Updated weights for policy 1, policy_version 721234 (0.0007) [2023-12-27 10:37:16,597][20347] Updated weights for policy 0, policy_version 721678 (0.0007) [2023-12-27 10:37:16,645][20347] Updated weights for policy 0, policy_version 721688 (0.0007) [2023-12-27 10:37:16,701][20347] Updated weights for policy 0, policy_version 721698 (0.0005) [2023-12-27 10:37:16,743][20381] Updated weights for policy 1, policy_version 721244 (0.0009) [2023-12-27 10:37:16,802][20381] Updated weights for policy 1, policy_version 721254 (0.0005) [2023-12-27 10:37:16,855][20381] Updated weights for policy 1, policy_version 721264 (0.0005) [2023-12-27 10:37:17,340][20347] Updated weights for policy 0, policy_version 721708 (0.0007) [2023-12-27 10:37:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 369451008. Throughput: 0: 10104.0, 1: 10304.9. Samples: 369465852. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:17,363][19170] Avg episode reward: [(0, '350.810'), (1, '892.780')] [2023-12-27 10:37:17,406][20347] Updated weights for policy 0, policy_version 721718 (0.0005) [2023-12-27 10:37:17,461][20381] Updated weights for policy 1, policy_version 721274 (0.0005) [2023-12-27 10:37:17,463][20347] Updated weights for policy 0, policy_version 721728 (0.0005) [2023-12-27 10:37:17,517][20381] Updated weights for policy 1, policy_version 721284 (0.0005) [2023-12-27 10:37:17,576][20381] Updated weights for policy 1, policy_version 721294 (0.0010) [2023-12-27 10:37:17,636][20381] Updated weights for policy 1, policy_version 721304 (0.0010) [2023-12-27 10:37:18,040][20347] Updated weights for policy 0, policy_version 721738 (0.0007) [2023-12-27 10:37:18,089][20347] Updated weights for policy 0, policy_version 721748 (0.0010) [2023-12-27 10:37:18,140][20347] Updated weights for policy 0, policy_version 721758 (0.0005) [2023-12-27 10:37:18,193][20347] Updated weights for policy 0, policy_version 721768 (0.0005) [2023-12-27 10:37:18,249][20381] Updated weights for policy 1, policy_version 721314 (0.0006) [2023-12-27 10:37:18,295][20381] Updated weights for policy 1, policy_version 721324 (0.0008) [2023-12-27 10:37:18,358][20381] Updated weights for policy 1, policy_version 721334 (0.0009) [2023-12-27 10:37:18,875][20347] Updated weights for policy 0, policy_version 721778 (0.0008) [2023-12-27 10:37:18,938][20347] Updated weights for policy 0, policy_version 721788 (0.0008) [2023-12-27 10:37:19,000][20347] Updated weights for policy 0, policy_version 721798 (0.0008) [2023-12-27 10:37:19,132][20381] Updated weights for policy 1, policy_version 721344 (0.0007) [2023-12-27 10:37:19,187][20381] Updated weights for policy 1, policy_version 721354 (0.0008) [2023-12-27 10:37:19,252][20381] Updated weights for policy 1, policy_version 721364 (0.0007) [2023-12-27 10:37:19,723][20347] Updated weights for policy 0, policy_version 721808 (0.0008) [2023-12-27 10:37:19,777][20347] Updated weights for policy 0, policy_version 721818 (0.0008) [2023-12-27 10:37:19,836][20347] Updated weights for policy 0, policy_version 721828 (0.0008) [2023-12-27 10:37:19,941][20381] Updated weights for policy 1, policy_version 721374 (0.0009) [2023-12-27 10:37:20,005][20381] Updated weights for policy 1, policy_version 721384 (0.0010) [2023-12-27 10:37:20,069][20381] Updated weights for policy 1, policy_version 721394 (0.0010) [2023-12-27 10:37:20,499][20347] Updated weights for policy 0, policy_version 721838 (0.0008) [2023-12-27 10:37:20,551][20347] Updated weights for policy 0, policy_version 721848 (0.0008) [2023-12-27 10:37:20,612][20347] Updated weights for policy 0, policy_version 721858 (0.0008) [2023-12-27 10:37:20,786][20381] Updated weights for policy 1, policy_version 721404 (0.0009) [2023-12-27 10:37:20,835][20381] Updated weights for policy 1, policy_version 721414 (0.0007) [2023-12-27 10:37:20,891][20381] Updated weights for policy 1, policy_version 721424 (0.0009) [2023-12-27 10:37:21,410][20347] Updated weights for policy 0, policy_version 721868 (0.0007) [2023-12-27 10:37:21,475][20347] Updated weights for policy 0, policy_version 721878 (0.0007) [2023-12-27 10:37:21,538][20347] Updated weights for policy 0, policy_version 721888 (0.0009) [2023-12-27 10:37:21,669][20381] Updated weights for policy 1, policy_version 721434 (0.0008) [2023-12-27 10:37:21,727][20381] Updated weights for policy 1, policy_version 721444 (0.0009) [2023-12-27 10:37:21,789][20381] Updated weights for policy 1, policy_version 721454 (0.0008) [2023-12-27 10:37:21,847][20381] Updated weights for policy 1, policy_version 721464 (0.0008) [2023-12-27 10:37:22,278][20347] Updated weights for policy 0, policy_version 721898 (0.0008) [2023-12-27 10:37:22,341][20347] Updated weights for policy 0, policy_version 721908 (0.0008) [2023-12-27 10:37:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 369549312. Throughput: 0: 10120.4, 1: 10289.3. Samples: 369526836. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:22,363][19170] Avg episode reward: [(0, '534.040'), (1, '922.230')] [2023-12-27 10:37:22,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000721464_184721408.pth... [2023-12-27 10:37:22,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000720280_184418304.pth [2023-12-27 10:37:22,412][20347] Updated weights for policy 0, policy_version 721918 (0.0008) [2023-12-27 10:37:22,472][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000721928_184836096.pth... [2023-12-27 10:37:22,473][20347] Updated weights for policy 0, policy_version 721928 (0.0008) [2023-12-27 10:37:22,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000720744_184532992.pth [2023-12-27 10:37:22,569][20381] Updated weights for policy 1, policy_version 721474 (0.0007) [2023-12-27 10:37:22,627][20381] Updated weights for policy 1, policy_version 721484 (0.0007) [2023-12-27 10:37:22,686][20381] Updated weights for policy 1, policy_version 721494 (0.0008) [2023-12-27 10:37:23,212][20347] Updated weights for policy 0, policy_version 721938 (0.0008) [2023-12-27 10:37:23,268][20347] Updated weights for policy 0, policy_version 721948 (0.0008) [2023-12-27 10:37:23,327][20347] Updated weights for policy 0, policy_version 721958 (0.0007) [2023-12-27 10:37:23,391][20381] Updated weights for policy 1, policy_version 721504 (0.0008) [2023-12-27 10:37:23,436][20381] Updated weights for policy 1, policy_version 721514 (0.0008) [2023-12-27 10:37:23,489][20381] Updated weights for policy 1, policy_version 721524 (0.0008) [2023-12-27 10:37:24,039][20347] Updated weights for policy 0, policy_version 721968 (0.0008) [2023-12-27 10:37:24,095][20347] Updated weights for policy 0, policy_version 721978 (0.0007) [2023-12-27 10:37:24,112][20381] Updated weights for policy 1, policy_version 721534 (0.0006) [2023-12-27 10:37:24,150][20347] Updated weights for policy 0, policy_version 721988 (0.0007) [2023-12-27 10:37:24,158][20381] Updated weights for policy 1, policy_version 721544 (0.0008) [2023-12-27 10:37:24,214][20381] Updated weights for policy 1, policy_version 721554 (0.0007) [2023-12-27 10:37:24,802][20347] Updated weights for policy 0, policy_version 721998 (0.0005) [2023-12-27 10:37:24,852][20347] Updated weights for policy 0, policy_version 722008 (0.0005) [2023-12-27 10:37:24,905][20347] Updated weights for policy 0, policy_version 722018 (0.0006) [2023-12-27 10:37:24,940][20381] Updated weights for policy 1, policy_version 721564 (0.0009) [2023-12-27 10:37:24,999][20381] Updated weights for policy 1, policy_version 721574 (0.0010) [2023-12-27 10:37:25,048][20381] Updated weights for policy 1, policy_version 721584 (0.0010) [2023-12-27 10:37:25,573][20347] Updated weights for policy 0, policy_version 722028 (0.0007) [2023-12-27 10:37:25,626][20347] Updated weights for policy 0, policy_version 722038 (0.0006) [2023-12-27 10:37:25,672][20347] Updated weights for policy 0, policy_version 722048 (0.0006) [2023-12-27 10:37:25,706][20381] Updated weights for policy 1, policy_version 721594 (0.0010) [2023-12-27 10:37:25,758][20381] Updated weights for policy 1, policy_version 721604 (0.0010) [2023-12-27 10:37:25,802][20381] Updated weights for policy 1, policy_version 721614 (0.0010) [2023-12-27 10:37:25,859][20381] Updated weights for policy 1, policy_version 721624 (0.0010) [2023-12-27 10:37:26,373][20347] Updated weights for policy 0, policy_version 722058 (0.0008) [2023-12-27 10:37:26,431][20347] Updated weights for policy 0, policy_version 722068 (0.0007) [2023-12-27 10:37:26,486][20347] Updated weights for policy 0, policy_version 722078 (0.0008) [2023-12-27 10:37:26,536][20347] Updated weights for policy 0, policy_version 722088 (0.0007) [2023-12-27 10:37:26,543][20381] Updated weights for policy 1, policy_version 721634 (0.0009) [2023-12-27 10:37:26,590][20381] Updated weights for policy 1, policy_version 721644 (0.0010) [2023-12-27 10:37:26,638][20381] Updated weights for policy 1, policy_version 721654 (0.0010) [2023-12-27 10:37:27,183][20347] Updated weights for policy 0, policy_version 722098 (0.0007) [2023-12-27 10:37:27,229][20347] Updated weights for policy 0, policy_version 722108 (0.0007) [2023-12-27 10:37:27,276][20347] Updated weights for policy 0, policy_version 722118 (0.0007) [2023-12-27 10:37:27,340][20381] Updated weights for policy 1, policy_version 721664 (0.0011) [2023-12-27 10:37:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 369655808. Throughput: 0: 10062.1, 1: 10311.0. Samples: 369646484. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:27,363][19170] Avg episode reward: [(0, '572.480'), (1, '912.750')] [2023-12-27 10:37:27,390][20381] Updated weights for policy 1, policy_version 721674 (0.0010) [2023-12-27 10:37:27,451][20381] Updated weights for policy 1, policy_version 721684 (0.0010) [2023-12-27 10:37:27,938][20347] Updated weights for policy 0, policy_version 722128 (0.0008) [2023-12-27 10:37:27,985][20347] Updated weights for policy 0, policy_version 722138 (0.0009) [2023-12-27 10:37:28,046][20347] Updated weights for policy 0, policy_version 722148 (0.0009) [2023-12-27 10:37:28,074][20381] Updated weights for policy 1, policy_version 721694 (0.0008) [2023-12-27 10:37:28,120][20381] Updated weights for policy 1, policy_version 721704 (0.0007) [2023-12-27 10:37:28,168][20381] Updated weights for policy 1, policy_version 721714 (0.0010) [2023-12-27 10:37:28,741][20347] Updated weights for policy 0, policy_version 722158 (0.0007) [2023-12-27 10:37:28,801][20347] Updated weights for policy 0, policy_version 722168 (0.0007) [2023-12-27 10:37:28,806][20381] Updated weights for policy 1, policy_version 721724 (0.0008) [2023-12-27 10:37:28,860][20347] Updated weights for policy 0, policy_version 722178 (0.0006) [2023-12-27 10:37:28,867][20381] Updated weights for policy 1, policy_version 721734 (0.0010) [2023-12-27 10:37:28,924][20381] Updated weights for policy 1, policy_version 721744 (0.0011) [2023-12-27 10:37:29,600][20381] Updated weights for policy 1, policy_version 721754 (0.0011) [2023-12-27 10:37:29,617][20347] Updated weights for policy 0, policy_version 722188 (0.0009) [2023-12-27 10:37:29,655][20381] Updated weights for policy 1, policy_version 721764 (0.0011) [2023-12-27 10:37:29,663][20347] Updated weights for policy 0, policy_version 722198 (0.0008) [2023-12-27 10:37:29,708][20381] Updated weights for policy 1, policy_version 721774 (0.0011) [2023-12-27 10:37:29,719][20347] Updated weights for policy 0, policy_version 722208 (0.0007) [2023-12-27 10:37:29,767][20381] Updated weights for policy 1, policy_version 721784 (0.0006) [2023-12-27 10:37:30,377][20347] Updated weights for policy 0, policy_version 722218 (0.0009) [2023-12-27 10:37:30,411][20381] Updated weights for policy 1, policy_version 721794 (0.0011) [2023-12-27 10:37:30,433][20347] Updated weights for policy 0, policy_version 722228 (0.0008) [2023-12-27 10:37:30,469][20381] Updated weights for policy 1, policy_version 721804 (0.0010) [2023-12-27 10:37:30,485][20347] Updated weights for policy 0, policy_version 722238 (0.0010) [2023-12-27 10:37:30,526][20381] Updated weights for policy 1, policy_version 721814 (0.0006) [2023-12-27 10:37:30,543][20347] Updated weights for policy 0, policy_version 722248 (0.0007) [2023-12-27 10:37:31,150][20347] Updated weights for policy 0, policy_version 722258 (0.0008) [2023-12-27 10:37:31,173][20381] Updated weights for policy 1, policy_version 721824 (0.0007) [2023-12-27 10:37:31,212][20347] Updated weights for policy 0, policy_version 722268 (0.0010) [2023-12-27 10:37:31,227][20381] Updated weights for policy 1, policy_version 721834 (0.0006) [2023-12-27 10:37:31,270][20347] Updated weights for policy 0, policy_version 722278 (0.0010) [2023-12-27 10:37:31,295][20381] Updated weights for policy 1, policy_version 721844 (0.0007) [2023-12-27 10:37:31,947][20381] Updated weights for policy 1, policy_version 721854 (0.0006) [2023-12-27 10:37:31,971][20347] Updated weights for policy 0, policy_version 722288 (0.0006) [2023-12-27 10:37:32,004][20381] Updated weights for policy 1, policy_version 721864 (0.0006) [2023-12-27 10:37:32,023][20347] Updated weights for policy 0, policy_version 722298 (0.0006) [2023-12-27 10:37:32,067][20381] Updated weights for policy 1, policy_version 721874 (0.0007) [2023-12-27 10:37:32,078][20347] Updated weights for policy 0, policy_version 722308 (0.0006) [2023-12-27 10:37:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 369762304. Throughput: 0: 10064.5, 1: 10305.1. Samples: 369771184. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:32,363][19170] Avg episode reward: [(0, '645.010'), (1, '873.570')] [2023-12-27 10:37:32,742][20381] Updated weights for policy 1, policy_version 721884 (0.0008) [2023-12-27 10:37:32,747][20347] Updated weights for policy 0, policy_version 722318 (0.0005) [2023-12-27 10:37:32,798][20381] Updated weights for policy 1, policy_version 721894 (0.0011) [2023-12-27 10:37:32,803][20347] Updated weights for policy 0, policy_version 722328 (0.0006) [2023-12-27 10:37:32,850][20381] Updated weights for policy 1, policy_version 721904 (0.0010) [2023-12-27 10:37:32,852][20347] Updated weights for policy 0, policy_version 722338 (0.0005) [2023-12-27 10:37:33,459][20381] Updated weights for policy 1, policy_version 721914 (0.0010) [2023-12-27 10:37:33,475][20347] Updated weights for policy 0, policy_version 722348 (0.0006) [2023-12-27 10:37:33,513][20381] Updated weights for policy 1, policy_version 721924 (0.0006) [2023-12-27 10:37:33,527][20347] Updated weights for policy 0, policy_version 722358 (0.0006) [2023-12-27 10:37:33,568][20381] Updated weights for policy 1, policy_version 721934 (0.0007) [2023-12-27 10:37:33,584][20347] Updated weights for policy 0, policy_version 722368 (0.0007) [2023-12-27 10:37:33,624][20381] Updated weights for policy 1, policy_version 721944 (0.0005) [2023-12-27 10:37:34,277][20381] Updated weights for policy 1, policy_version 721954 (0.0008) [2023-12-27 10:37:34,298][20347] Updated weights for policy 0, policy_version 722378 (0.0007) [2023-12-27 10:37:34,341][20381] Updated weights for policy 1, policy_version 721964 (0.0011) [2023-12-27 10:37:34,354][20347] Updated weights for policy 0, policy_version 722388 (0.0007) [2023-12-27 10:37:34,401][20381] Updated weights for policy 1, policy_version 721974 (0.0011) [2023-12-27 10:37:34,415][20347] Updated weights for policy 0, policy_version 722398 (0.0008) [2023-12-27 10:37:34,477][20347] Updated weights for policy 0, policy_version 722408 (0.0008) [2023-12-27 10:37:35,086][20381] Updated weights for policy 1, policy_version 721984 (0.0008) [2023-12-27 10:37:35,130][20347] Updated weights for policy 0, policy_version 722418 (0.0010) [2023-12-27 10:37:35,140][20381] Updated weights for policy 1, policy_version 721994 (0.0006) [2023-12-27 10:37:35,183][20347] Updated weights for policy 0, policy_version 722428 (0.0010) [2023-12-27 10:37:35,197][20381] Updated weights for policy 1, policy_version 722004 (0.0006) [2023-12-27 10:37:35,235][20347] Updated weights for policy 0, policy_version 722438 (0.0010) [2023-12-27 10:37:35,889][20381] Updated weights for policy 1, policy_version 722014 (0.0007) [2023-12-27 10:37:35,928][20347] Updated weights for policy 0, policy_version 722448 (0.0010) [2023-12-27 10:37:35,948][20381] Updated weights for policy 1, policy_version 722024 (0.0005) [2023-12-27 10:37:35,979][20347] Updated weights for policy 0, policy_version 722458 (0.0010) [2023-12-27 10:37:36,006][20381] Updated weights for policy 1, policy_version 722034 (0.0005) [2023-12-27 10:37:36,033][20347] Updated weights for policy 0, policy_version 722468 (0.0010) [2023-12-27 10:37:36,702][20381] Updated weights for policy 1, policy_version 722044 (0.0006) [2023-12-27 10:37:36,711][20347] Updated weights for policy 0, policy_version 722478 (0.0008) [2023-12-27 10:37:36,758][20381] Updated weights for policy 1, policy_version 722054 (0.0005) [2023-12-27 10:37:36,767][20347] Updated weights for policy 0, policy_version 722488 (0.0006) [2023-12-27 10:37:36,818][20381] Updated weights for policy 1, policy_version 722064 (0.0006) [2023-12-27 10:37:36,827][20347] Updated weights for policy 0, policy_version 722498 (0.0006) [2023-12-27 10:37:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 369860608. Throughput: 0: 10116.6, 1: 10286.6. Samples: 369834220. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:37,363][19170] Avg episode reward: [(0, '635.860'), (1, '931.990')] [2023-12-27 10:37:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000722072_184877056.pth... [2023-12-27 10:37:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000722504_184983552.pth... [2023-12-27 10:37:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000720888_184573952.pth [2023-12-27 10:37:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000721320_184680448.pth [2023-12-27 10:37:37,450][20381] Updated weights for policy 1, policy_version 722074 (0.0006) [2023-12-27 10:37:37,509][20381] Updated weights for policy 1, policy_version 722084 (0.0006) [2023-12-27 10:37:37,515][20347] Updated weights for policy 0, policy_version 722508 (0.0006) [2023-12-27 10:37:37,571][20381] Updated weights for policy 1, policy_version 722094 (0.0006) [2023-12-27 10:37:37,574][20347] Updated weights for policy 0, policy_version 722518 (0.0005) [2023-12-27 10:37:37,629][20381] Updated weights for policy 1, policy_version 722104 (0.0007) [2023-12-27 10:37:37,640][20347] Updated weights for policy 0, policy_version 722528 (0.0005) [2023-12-27 10:37:38,170][20381] Updated weights for policy 1, policy_version 722114 (0.0006) [2023-12-27 10:37:38,206][20347] Updated weights for policy 0, policy_version 722538 (0.0006) [2023-12-27 10:37:38,224][20381] Updated weights for policy 1, policy_version 722124 (0.0005) [2023-12-27 10:37:38,259][20347] Updated weights for policy 0, policy_version 722548 (0.0007) [2023-12-27 10:37:38,269][20381] Updated weights for policy 1, policy_version 722134 (0.0005) [2023-12-27 10:37:38,317][20347] Updated weights for policy 0, policy_version 722558 (0.0006) [2023-12-27 10:37:38,379][20347] Updated weights for policy 0, policy_version 722568 (0.0007) [2023-12-27 10:37:38,908][20381] Updated weights for policy 1, policy_version 722144 (0.0005) [2023-12-27 10:37:38,958][20381] Updated weights for policy 1, policy_version 722154 (0.0006) [2023-12-27 10:37:39,003][20381] Updated weights for policy 1, policy_version 722164 (0.0007) [2023-12-27 10:37:39,084][20347] Updated weights for policy 0, policy_version 722578 (0.0010) [2023-12-27 10:37:39,130][20347] Updated weights for policy 0, policy_version 722588 (0.0010) [2023-12-27 10:37:39,182][20347] Updated weights for policy 0, policy_version 722598 (0.0010) [2023-12-27 10:37:39,732][20381] Updated weights for policy 1, policy_version 722174 (0.0010) [2023-12-27 10:37:39,791][20381] Updated weights for policy 1, policy_version 722184 (0.0011) [2023-12-27 10:37:39,854][20381] Updated weights for policy 1, policy_version 722194 (0.0009) [2023-12-27 10:37:39,860][20347] Updated weights for policy 0, policy_version 722608 (0.0009) [2023-12-27 10:37:39,923][20347] Updated weights for policy 0, policy_version 722618 (0.0009) [2023-12-27 10:37:39,983][20347] Updated weights for policy 0, policy_version 722628 (0.0008) [2023-12-27 10:37:40,597][20381] Updated weights for policy 1, policy_version 722204 (0.0010) [2023-12-27 10:37:40,659][20381] Updated weights for policy 1, policy_version 722214 (0.0010) [2023-12-27 10:37:40,713][20347] Updated weights for policy 0, policy_version 722638 (0.0006) [2023-12-27 10:37:40,716][20381] Updated weights for policy 1, policy_version 722224 (0.0011) [2023-12-27 10:37:40,772][20347] Updated weights for policy 0, policy_version 722648 (0.0006) [2023-12-27 10:37:40,828][20347] Updated weights for policy 0, policy_version 722658 (0.0005) [2023-12-27 10:37:41,413][20381] Updated weights for policy 1, policy_version 722234 (0.0009) [2023-12-27 10:37:41,473][20381] Updated weights for policy 1, policy_version 722244 (0.0010) [2023-12-27 10:37:41,533][20381] Updated weights for policy 1, policy_version 722254 (0.0011) [2023-12-27 10:37:41,543][20347] Updated weights for policy 0, policy_version 722668 (0.0008) [2023-12-27 10:37:41,597][20381] Updated weights for policy 1, policy_version 722264 (0.0011) [2023-12-27 10:37:41,609][20347] Updated weights for policy 0, policy_version 722678 (0.0007) [2023-12-27 10:37:41,672][20347] Updated weights for policy 0, policy_version 722688 (0.0008) [2023-12-27 10:37:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 369958912. Throughput: 0: 10155.4, 1: 10274.0. Samples: 369956752. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:42,363][19170] Avg episode reward: [(0, '647.530'), (1, '893.130')] [2023-12-27 10:37:42,363][20381] Updated weights for policy 1, policy_version 722274 (0.0011) [2023-12-27 10:37:42,422][20381] Updated weights for policy 1, policy_version 722284 (0.0008) [2023-12-27 10:37:42,446][20347] Updated weights for policy 0, policy_version 722698 (0.0008) [2023-12-27 10:37:42,485][20381] Updated weights for policy 1, policy_version 722294 (0.0010) [2023-12-27 10:37:42,505][20347] Updated weights for policy 0, policy_version 722708 (0.0008) [2023-12-27 10:37:42,558][20347] Updated weights for policy 0, policy_version 722718 (0.0009) [2023-12-27 10:37:42,611][20347] Updated weights for policy 0, policy_version 722728 (0.0009) [2023-12-27 10:37:43,124][20381] Updated weights for policy 1, policy_version 722304 (0.0010) [2023-12-27 10:37:43,189][20381] Updated weights for policy 1, policy_version 722314 (0.0010) [2023-12-27 10:37:43,255][20381] Updated weights for policy 1, policy_version 722324 (0.0006) [2023-12-27 10:37:43,284][20347] Updated weights for policy 0, policy_version 722738 (0.0005) [2023-12-27 10:37:43,343][20347] Updated weights for policy 0, policy_version 722748 (0.0005) [2023-12-27 10:37:43,406][20347] Updated weights for policy 0, policy_version 722758 (0.0005) [2023-12-27 10:37:43,905][20381] Updated weights for policy 1, policy_version 722334 (0.0009) [2023-12-27 10:37:43,954][20381] Updated weights for policy 1, policy_version 722344 (0.0011) [2023-12-27 10:37:43,971][20347] Updated weights for policy 0, policy_version 722768 (0.0008) [2023-12-27 10:37:44,003][20381] Updated weights for policy 1, policy_version 722354 (0.0011) [2023-12-27 10:37:44,023][20347] Updated weights for policy 0, policy_version 722778 (0.0005) [2023-12-27 10:37:44,085][20347] Updated weights for policy 0, policy_version 722788 (0.0006) [2023-12-27 10:37:44,643][20381] Updated weights for policy 1, policy_version 722364 (0.0010) [2023-12-27 10:37:44,708][20381] Updated weights for policy 1, policy_version 722374 (0.0011) [2023-12-27 10:37:44,725][20347] Updated weights for policy 0, policy_version 722798 (0.0006) [2023-12-27 10:37:44,770][20381] Updated weights for policy 1, policy_version 722384 (0.0011) [2023-12-27 10:37:44,782][20347] Updated weights for policy 0, policy_version 722808 (0.0006) [2023-12-27 10:37:44,836][20347] Updated weights for policy 0, policy_version 722818 (0.0008) [2023-12-27 10:37:45,393][20381] Updated weights for policy 1, policy_version 722394 (0.0010) [2023-12-27 10:37:45,461][20381] Updated weights for policy 1, policy_version 722404 (0.0009) [2023-12-27 10:37:45,516][20381] Updated weights for policy 1, policy_version 722414 (0.0008) [2023-12-27 10:37:45,536][20347] Updated weights for policy 0, policy_version 722828 (0.0007) [2023-12-27 10:37:45,568][20381] Updated weights for policy 1, policy_version 722424 (0.0008) [2023-12-27 10:37:45,596][20347] Updated weights for policy 0, policy_version 722838 (0.0011) [2023-12-27 10:37:45,657][20347] Updated weights for policy 0, policy_version 722848 (0.0011) [2023-12-27 10:37:46,175][20381] Updated weights for policy 1, policy_version 722434 (0.0008) [2023-12-27 10:37:46,235][20381] Updated weights for policy 1, policy_version 722444 (0.0007) [2023-12-27 10:37:46,291][20381] Updated weights for policy 1, policy_version 722454 (0.0007) [2023-12-27 10:37:46,331][20347] Updated weights for policy 0, policy_version 722858 (0.0010) [2023-12-27 10:37:46,386][20347] Updated weights for policy 0, policy_version 722868 (0.0010) [2023-12-27 10:37:46,434][20347] Updated weights for policy 0, policy_version 722878 (0.0010) [2023-12-27 10:37:46,491][20347] Updated weights for policy 0, policy_version 722888 (0.0008) [2023-12-27 10:37:46,996][20381] Updated weights for policy 1, policy_version 722464 (0.0008) [2023-12-27 10:37:47,055][20381] Updated weights for policy 1, policy_version 722474 (0.0008) [2023-12-27 10:37:47,105][20381] Updated weights for policy 1, policy_version 722484 (0.0007) [2023-12-27 10:37:47,115][20347] Updated weights for policy 0, policy_version 722898 (0.0005) [2023-12-27 10:37:47,169][20347] Updated weights for policy 0, policy_version 722908 (0.0005) [2023-12-27 10:37:47,227][20347] Updated weights for policy 0, policy_version 722918 (0.0005) [2023-12-27 10:37:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 370073600. Throughput: 0: 10188.4, 1: 10277.6. Samples: 370080132. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:47,363][19170] Avg episode reward: [(0, '589.680'), (1, '912.480')] [2023-12-27 10:37:47,766][20381] Updated weights for policy 1, policy_version 722494 (0.0006) [2023-12-27 10:37:47,803][20347] Updated weights for policy 0, policy_version 722928 (0.0005) [2023-12-27 10:37:47,815][20381] Updated weights for policy 1, policy_version 722504 (0.0006) [2023-12-27 10:37:47,860][20347] Updated weights for policy 0, policy_version 722938 (0.0005) [2023-12-27 10:37:47,862][20381] Updated weights for policy 1, policy_version 722514 (0.0006) [2023-12-27 10:37:47,922][20347] Updated weights for policy 0, policy_version 722948 (0.0005) [2023-12-27 10:37:48,512][20381] Updated weights for policy 1, policy_version 722524 (0.0006) [2023-12-27 10:37:48,570][20381] Updated weights for policy 1, policy_version 722534 (0.0008) [2023-12-27 10:37:48,571][20347] Updated weights for policy 0, policy_version 722958 (0.0007) [2023-12-27 10:37:48,625][20347] Updated weights for policy 0, policy_version 722968 (0.0006) [2023-12-27 10:37:48,626][20381] Updated weights for policy 1, policy_version 722544 (0.0005) [2023-12-27 10:37:48,681][20347] Updated weights for policy 0, policy_version 722978 (0.0005) [2023-12-27 10:37:49,323][20381] Updated weights for policy 1, policy_version 722554 (0.0007) [2023-12-27 10:37:49,394][20381] Updated weights for policy 1, policy_version 722564 (0.0008) [2023-12-27 10:37:49,420][20347] Updated weights for policy 0, policy_version 722988 (0.0007) [2023-12-27 10:37:49,458][20381] Updated weights for policy 1, policy_version 722574 (0.0006) [2023-12-27 10:37:49,483][20347] Updated weights for policy 0, policy_version 722998 (0.0010) [2023-12-27 10:37:49,520][20381] Updated weights for policy 1, policy_version 722584 (0.0006) [2023-12-27 10:37:49,544][20347] Updated weights for policy 0, policy_version 723008 (0.0006) [2023-12-27 10:37:50,143][20381] Updated weights for policy 1, policy_version 722595 (0.0008) [2023-12-27 10:37:50,200][20381] Updated weights for policy 1, policy_version 722605 (0.0006) [2023-12-27 10:37:50,230][20347] Updated weights for policy 0, policy_version 723018 (0.0010) [2023-12-27 10:37:50,261][20381] Updated weights for policy 1, policy_version 722615 (0.0007) [2023-12-27 10:37:50,292][20347] Updated weights for policy 0, policy_version 723028 (0.0010) [2023-12-27 10:37:50,359][20347] Updated weights for policy 0, policy_version 723038 (0.0010) [2023-12-27 10:37:50,421][20347] Updated weights for policy 0, policy_version 723048 (0.0010) [2023-12-27 10:37:50,959][20381] Updated weights for policy 1, policy_version 722625 (0.0008) [2023-12-27 10:37:51,017][20381] Updated weights for policy 1, policy_version 722635 (0.0009) [2023-12-27 10:37:51,079][20381] Updated weights for policy 1, policy_version 722645 (0.0007) [2023-12-27 10:37:51,125][20347] Updated weights for policy 0, policy_version 723058 (0.0008) [2023-12-27 10:37:51,188][20347] Updated weights for policy 0, policy_version 723068 (0.0008) [2023-12-27 10:37:51,243][20347] Updated weights for policy 0, policy_version 723078 (0.0008) [2023-12-27 10:37:51,758][20381] Updated weights for policy 1, policy_version 722655 (0.0007) [2023-12-27 10:37:51,814][20381] Updated weights for policy 1, policy_version 722665 (0.0007) [2023-12-27 10:37:51,870][20381] Updated weights for policy 1, policy_version 722675 (0.0008) [2023-12-27 10:37:51,998][20347] Updated weights for policy 0, policy_version 723088 (0.0010) [2023-12-27 10:37:52,065][20347] Updated weights for policy 0, policy_version 723098 (0.0010) [2023-12-27 10:37:52,120][20347] Updated weights for policy 0, policy_version 723108 (0.0010) [2023-12-27 10:37:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 370171904. Throughput: 0: 10194.8, 1: 10298.5. Samples: 370143052. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:52,363][19170] Avg episode reward: [(0, '657.640'), (1, '951.520')] [2023-12-27 10:37:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000722680_185032704.pth... [2023-12-27 10:37:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000723112_185139200.pth... [2023-12-27 10:37:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000721464_184721408.pth [2023-12-27 10:37:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000721928_184836096.pth [2023-12-27 10:37:52,574][20381] Updated weights for policy 1, policy_version 722685 (0.0006) [2023-12-27 10:37:52,624][20381] Updated weights for policy 1, policy_version 722695 (0.0007) [2023-12-27 10:37:52,683][20381] Updated weights for policy 1, policy_version 722705 (0.0007) [2023-12-27 10:37:52,766][20347] Updated weights for policy 0, policy_version 723118 (0.0009) [2023-12-27 10:37:52,814][20347] Updated weights for policy 0, policy_version 723128 (0.0008) [2023-12-27 10:37:52,862][20347] Updated weights for policy 0, policy_version 723138 (0.0007) [2023-12-27 10:37:53,370][20381] Updated weights for policy 1, policy_version 722715 (0.0008) [2023-12-27 10:37:53,419][20381] Updated weights for policy 1, policy_version 722725 (0.0010) [2023-12-27 10:37:53,472][20381] Updated weights for policy 1, policy_version 722735 (0.0011) [2023-12-27 10:37:53,472][20347] Updated weights for policy 0, policy_version 723148 (0.0008) [2023-12-27 10:37:53,532][20347] Updated weights for policy 0, policy_version 723158 (0.0005) [2023-12-27 10:37:53,592][20347] Updated weights for policy 0, policy_version 723168 (0.0005) [2023-12-27 10:37:54,156][20347] Updated weights for policy 0, policy_version 723178 (0.0006) [2023-12-27 10:37:54,161][20381] Updated weights for policy 1, policy_version 722745 (0.0011) [2023-12-27 10:37:54,215][20381] Updated weights for policy 1, policy_version 722755 (0.0010) [2023-12-27 10:37:54,216][20347] Updated weights for policy 0, policy_version 723188 (0.0010) [2023-12-27 10:37:54,268][20347] Updated weights for policy 0, policy_version 723198 (0.0009) [2023-12-27 10:37:54,271][20381] Updated weights for policy 1, policy_version 722765 (0.0011) [2023-12-27 10:37:54,329][20347] Updated weights for policy 0, policy_version 723208 (0.0011) [2023-12-27 10:37:54,331][20381] Updated weights for policy 1, policy_version 722775 (0.0010) [2023-12-27 10:37:54,982][20347] Updated weights for policy 0, policy_version 723218 (0.0010) [2023-12-27 10:37:55,035][20381] Updated weights for policy 1, policy_version 722785 (0.0009) [2023-12-27 10:37:55,047][20347] Updated weights for policy 0, policy_version 723228 (0.0011) [2023-12-27 10:37:55,099][20381] Updated weights for policy 1, policy_version 722795 (0.0009) [2023-12-27 10:37:55,103][20347] Updated weights for policy 0, policy_version 723238 (0.0009) [2023-12-27 10:37:55,159][20381] Updated weights for policy 1, policy_version 722805 (0.0007) [2023-12-27 10:37:55,783][20347] Updated weights for policy 0, policy_version 723248 (0.0010) [2023-12-27 10:37:55,794][20381] Updated weights for policy 1, policy_version 722815 (0.0007) [2023-12-27 10:37:55,839][20347] Updated weights for policy 0, policy_version 723258 (0.0011) [2023-12-27 10:37:55,849][20381] Updated weights for policy 1, policy_version 722825 (0.0007) [2023-12-27 10:37:55,884][20347] Updated weights for policy 0, policy_version 723268 (0.0007) [2023-12-27 10:37:55,904][20381] Updated weights for policy 1, policy_version 722835 (0.0008) [2023-12-27 10:37:56,528][20347] Updated weights for policy 0, policy_version 723278 (0.0007) [2023-12-27 10:37:56,540][20381] Updated weights for policy 1, policy_version 722845 (0.0008) [2023-12-27 10:37:56,575][20347] Updated weights for policy 0, policy_version 723288 (0.0007) [2023-12-27 10:37:56,601][20381] Updated weights for policy 1, policy_version 722855 (0.0006) [2023-12-27 10:37:56,621][20347] Updated weights for policy 0, policy_version 723298 (0.0006) [2023-12-27 10:37:56,660][20381] Updated weights for policy 1, policy_version 722865 (0.0008) [2023-12-27 10:37:57,263][20347] Updated weights for policy 0, policy_version 723308 (0.0007) [2023-12-27 10:37:57,319][20347] Updated weights for policy 0, policy_version 723318 (0.0007) [2023-12-27 10:37:57,345][20381] Updated weights for policy 1, policy_version 722875 (0.0008) [2023-12-27 10:37:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 370270208. Throughput: 0: 10229.9, 1: 10293.5. Samples: 370265512. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:37:57,363][19170] Avg episode reward: [(0, '693.840'), (1, '912.500')] [2023-12-27 10:37:57,375][20347] Updated weights for policy 0, policy_version 723328 (0.0006) [2023-12-27 10:37:57,408][20381] Updated weights for policy 1, policy_version 722885 (0.0008) [2023-12-27 10:37:57,465][20381] Updated weights for policy 1, policy_version 722895 (0.0007) [2023-12-27 10:37:57,951][20347] Updated weights for policy 0, policy_version 723338 (0.0006) [2023-12-27 10:37:58,010][20347] Updated weights for policy 0, policy_version 723348 (0.0008) [2023-12-27 10:37:58,062][20347] Updated weights for policy 0, policy_version 723358 (0.0008) [2023-12-27 10:37:58,114][20347] Updated weights for policy 0, policy_version 723368 (0.0007) [2023-12-27 10:37:58,172][20381] Updated weights for policy 1, policy_version 722905 (0.0007) [2023-12-27 10:37:58,241][20381] Updated weights for policy 1, policy_version 722915 (0.0008) [2023-12-27 10:37:58,309][20381] Updated weights for policy 1, policy_version 722925 (0.0008) [2023-12-27 10:37:58,380][20381] Updated weights for policy 1, policy_version 722935 (0.0008) [2023-12-27 10:37:58,911][20347] Updated weights for policy 0, policy_version 723378 (0.0008) [2023-12-27 10:37:58,962][20347] Updated weights for policy 0, policy_version 723388 (0.0008) [2023-12-27 10:37:59,012][20347] Updated weights for policy 0, policy_version 723398 (0.0008) [2023-12-27 10:37:59,134][20381] Updated weights for policy 1, policy_version 722945 (0.0008) [2023-12-27 10:37:59,180][20381] Updated weights for policy 1, policy_version 722955 (0.0009) [2023-12-27 10:37:59,237][20381] Updated weights for policy 1, policy_version 722965 (0.0007) [2023-12-27 10:37:59,683][20347] Updated weights for policy 0, policy_version 723408 (0.0006) [2023-12-27 10:37:59,735][20347] Updated weights for policy 0, policy_version 723418 (0.0005) [2023-12-27 10:37:59,792][20347] Updated weights for policy 0, policy_version 723428 (0.0005) [2023-12-27 10:37:59,985][20381] Updated weights for policy 1, policy_version 722975 (0.0007) [2023-12-27 10:38:00,049][20381] Updated weights for policy 1, policy_version 722985 (0.0006) [2023-12-27 10:38:00,109][20381] Updated weights for policy 1, policy_version 722995 (0.0006) [2023-12-27 10:38:00,490][20347] Updated weights for policy 0, policy_version 723438 (0.0009) [2023-12-27 10:38:00,544][20347] Updated weights for policy 0, policy_version 723448 (0.0008) [2023-12-27 10:38:00,595][20347] Updated weights for policy 0, policy_version 723458 (0.0007) [2023-12-27 10:38:00,712][20381] Updated weights for policy 1, policy_version 723005 (0.0006) [2023-12-27 10:38:00,770][20381] Updated weights for policy 1, policy_version 723015 (0.0005) [2023-12-27 10:38:00,830][20381] Updated weights for policy 1, policy_version 723025 (0.0005) [2023-12-27 10:38:01,252][20347] Updated weights for policy 0, policy_version 723468 (0.0006) [2023-12-27 10:38:01,312][20347] Updated weights for policy 0, policy_version 723478 (0.0006) [2023-12-27 10:38:01,383][20347] Updated weights for policy 0, policy_version 723488 (0.0008) [2023-12-27 10:38:01,465][20381] Updated weights for policy 1, policy_version 723035 (0.0006) [2023-12-27 10:38:01,522][20381] Updated weights for policy 1, policy_version 723045 (0.0008) [2023-12-27 10:38:01,575][20381] Updated weights for policy 1, policy_version 723055 (0.0008) [2023-12-27 10:38:02,031][20347] Updated weights for policy 0, policy_version 723498 (0.0005) [2023-12-27 10:38:02,098][20347] Updated weights for policy 0, policy_version 723508 (0.0007) [2023-12-27 10:38:02,159][20347] Updated weights for policy 0, policy_version 723518 (0.0008) [2023-12-27 10:38:02,217][20347] Updated weights for policy 0, policy_version 723528 (0.0007) [2023-12-27 10:38:02,312][20381] Updated weights for policy 1, policy_version 723065 (0.0008) [2023-12-27 10:38:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 370376704. Throughput: 0: 10274.4, 1: 10224.2. Samples: 370388288. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:02,363][19170] Avg episode reward: [(0, '779.670'), (1, '922.340')] [2023-12-27 10:38:02,371][20381] Updated weights for policy 1, policy_version 723075 (0.0007) [2023-12-27 10:38:02,433][20381] Updated weights for policy 1, policy_version 723085 (0.0007) [2023-12-27 10:38:02,487][20381] Updated weights for policy 1, policy_version 723095 (0.0007) [2023-12-27 10:38:02,856][20347] Updated weights for policy 0, policy_version 723538 (0.0005) [2023-12-27 10:38:02,914][20347] Updated weights for policy 0, policy_version 723548 (0.0008) [2023-12-27 10:38:02,971][20347] Updated weights for policy 0, policy_version 723558 (0.0008) [2023-12-27 10:38:03,122][20381] Updated weights for policy 1, policy_version 723105 (0.0005) [2023-12-27 10:38:03,173][20381] Updated weights for policy 1, policy_version 723115 (0.0005) [2023-12-27 10:38:03,228][20381] Updated weights for policy 1, policy_version 723125 (0.0005) [2023-12-27 10:38:03,672][20347] Updated weights for policy 0, policy_version 723568 (0.0010) [2023-12-27 10:38:03,726][20347] Updated weights for policy 0, policy_version 723578 (0.0010) [2023-12-27 10:38:03,778][20347] Updated weights for policy 0, policy_version 723588 (0.0010) [2023-12-27 10:38:03,828][20381] Updated weights for policy 1, policy_version 723135 (0.0007) [2023-12-27 10:38:03,892][20381] Updated weights for policy 1, policy_version 723145 (0.0007) [2023-12-27 10:38:03,953][20381] Updated weights for policy 1, policy_version 723155 (0.0008) [2023-12-27 10:38:04,488][20347] Updated weights for policy 0, policy_version 723598 (0.0010) [2023-12-27 10:38:04,540][20347] Updated weights for policy 0, policy_version 723608 (0.0010) [2023-12-27 10:38:04,597][20347] Updated weights for policy 0, policy_version 723618 (0.0009) [2023-12-27 10:38:04,655][20381] Updated weights for policy 1, policy_version 723165 (0.0007) [2023-12-27 10:38:04,713][20381] Updated weights for policy 1, policy_version 723175 (0.0007) [2023-12-27 10:38:04,763][20381] Updated weights for policy 1, policy_version 723185 (0.0008) [2023-12-27 10:38:05,226][20347] Updated weights for policy 0, policy_version 723628 (0.0010) [2023-12-27 10:38:05,277][20347] Updated weights for policy 0, policy_version 723638 (0.0010) [2023-12-27 10:38:05,332][20347] Updated weights for policy 0, policy_version 723648 (0.0010) [2023-12-27 10:38:05,443][20381] Updated weights for policy 1, policy_version 723195 (0.0008) [2023-12-27 10:38:05,501][20381] Updated weights for policy 1, policy_version 723205 (0.0010) [2023-12-27 10:38:05,558][20381] Updated weights for policy 1, policy_version 723215 (0.0009) [2023-12-27 10:38:05,977][20347] Updated weights for policy 0, policy_version 723658 (0.0010) [2023-12-27 10:38:06,041][20347] Updated weights for policy 0, policy_version 723668 (0.0010) [2023-12-27 10:38:06,105][20347] Updated weights for policy 0, policy_version 723678 (0.0008) [2023-12-27 10:38:06,166][20347] Updated weights for policy 0, policy_version 723688 (0.0010) [2023-12-27 10:38:06,178][20381] Updated weights for policy 1, policy_version 723225 (0.0006) [2023-12-27 10:38:06,242][20381] Updated weights for policy 1, policy_version 723235 (0.0007) [2023-12-27 10:38:06,305][20381] Updated weights for policy 1, policy_version 723245 (0.0008) [2023-12-27 10:38:06,372][20381] Updated weights for policy 1, policy_version 723255 (0.0008) [2023-12-27 10:38:06,741][20347] Updated weights for policy 0, policy_version 723698 (0.0007) [2023-12-27 10:38:06,808][20347] Updated weights for policy 0, policy_version 723708 (0.0011) [2023-12-27 10:38:06,868][20347] Updated weights for policy 0, policy_version 723718 (0.0010) [2023-12-27 10:38:07,039][20381] Updated weights for policy 1, policy_version 723265 (0.0008) [2023-12-27 10:38:07,099][20381] Updated weights for policy 1, policy_version 723275 (0.0007) [2023-12-27 10:38:07,165][20381] Updated weights for policy 1, policy_version 723285 (0.0006) [2023-12-27 10:38:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 370483200. Throughput: 0: 10257.1, 1: 10255.5. Samples: 370449904. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:07,363][19170] Avg episode reward: [(0, '749.850'), (1, '883.450')] [2023-12-27 10:38:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000723288_185188352.pth... [2023-12-27 10:38:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000723720_185294848.pth... [2023-12-27 10:38:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000722072_184877056.pth [2023-12-27 10:38:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000722504_184983552.pth [2023-12-27 10:38:07,473][20347] Updated weights for policy 0, policy_version 723728 (0.0006) [2023-12-27 10:38:07,539][20347] Updated weights for policy 0, policy_version 723738 (0.0007) [2023-12-27 10:38:07,603][20347] Updated weights for policy 0, policy_version 723748 (0.0006) [2023-12-27 10:38:07,779][20381] Updated weights for policy 1, policy_version 723295 (0.0007) [2023-12-27 10:38:07,830][20381] Updated weights for policy 1, policy_version 723305 (0.0007) [2023-12-27 10:38:07,883][20381] Updated weights for policy 1, policy_version 723315 (0.0006) [2023-12-27 10:38:08,219][20347] Updated weights for policy 0, policy_version 723758 (0.0007) [2023-12-27 10:38:08,262][20347] Updated weights for policy 0, policy_version 723768 (0.0009) [2023-12-27 10:38:08,317][20347] Updated weights for policy 0, policy_version 723778 (0.0009) [2023-12-27 10:38:08,502][20381] Updated weights for policy 1, policy_version 723325 (0.0006) [2023-12-27 10:38:08,563][20381] Updated weights for policy 1, policy_version 723335 (0.0006) [2023-12-27 10:38:08,628][20381] Updated weights for policy 1, policy_version 723345 (0.0007) [2023-12-27 10:38:09,086][20347] Updated weights for policy 0, policy_version 723788 (0.0007) [2023-12-27 10:38:09,140][20347] Updated weights for policy 0, policy_version 723798 (0.0005) [2023-12-27 10:38:09,164][20381] Updated weights for policy 1, policy_version 723355 (0.0006) [2023-12-27 10:38:09,201][20347] Updated weights for policy 0, policy_version 723808 (0.0006) [2023-12-27 10:38:09,226][20381] Updated weights for policy 1, policy_version 723365 (0.0007) [2023-12-27 10:38:09,284][20381] Updated weights for policy 1, policy_version 723375 (0.0007) [2023-12-27 10:38:09,956][20347] Updated weights for policy 0, policy_version 723818 (0.0008) [2023-12-27 10:38:09,976][20381] Updated weights for policy 1, policy_version 723385 (0.0008) [2023-12-27 10:38:10,013][20347] Updated weights for policy 0, policy_version 723828 (0.0008) [2023-12-27 10:38:10,037][20381] Updated weights for policy 1, policy_version 723395 (0.0011) [2023-12-27 10:38:10,078][20347] Updated weights for policy 0, policy_version 723838 (0.0007) [2023-12-27 10:38:10,099][20381] Updated weights for policy 1, policy_version 723405 (0.0011) [2023-12-27 10:38:10,143][20347] Updated weights for policy 0, policy_version 723848 (0.0006) [2023-12-27 10:38:10,149][20381] Updated weights for policy 1, policy_version 723415 (0.0011) [2023-12-27 10:38:10,812][20347] Updated weights for policy 0, policy_version 723858 (0.0007) [2023-12-27 10:38:10,860][20381] Updated weights for policy 1, policy_version 723425 (0.0007) [2023-12-27 10:38:10,864][20347] Updated weights for policy 0, policy_version 723868 (0.0008) [2023-12-27 10:38:10,920][20347] Updated weights for policy 0, policy_version 723878 (0.0007) [2023-12-27 10:38:10,923][20381] Updated weights for policy 1, policy_version 723435 (0.0007) [2023-12-27 10:38:10,981][20381] Updated weights for policy 1, policy_version 723445 (0.0007) [2023-12-27 10:38:11,659][20347] Updated weights for policy 0, policy_version 723888 (0.0008) [2023-12-27 10:38:11,725][20347] Updated weights for policy 0, policy_version 723898 (0.0008) [2023-12-27 10:38:11,764][20381] Updated weights for policy 1, policy_version 723455 (0.0009) [2023-12-27 10:38:11,789][20347] Updated weights for policy 0, policy_version 723908 (0.0007) [2023-12-27 10:38:11,823][20381] Updated weights for policy 1, policy_version 723465 (0.0007) [2023-12-27 10:38:11,883][20381] Updated weights for policy 1, policy_version 723475 (0.0009) [2023-12-27 10:38:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 370581504. Throughput: 0: 10317.0, 1: 10327.3. Samples: 370575480. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:12,363][19170] Avg episode reward: [(0, '770.750'), (1, '853.920')] [2023-12-27 10:38:12,531][20347] Updated weights for policy 0, policy_version 723918 (0.0006) [2023-12-27 10:38:12,592][20347] Updated weights for policy 0, policy_version 723928 (0.0006) [2023-12-27 10:38:12,606][20381] Updated weights for policy 1, policy_version 723485 (0.0008) [2023-12-27 10:38:12,656][20347] Updated weights for policy 0, policy_version 723938 (0.0007) [2023-12-27 10:38:12,665][20381] Updated weights for policy 1, policy_version 723495 (0.0008) [2023-12-27 10:38:12,722][20381] Updated weights for policy 1, policy_version 723505 (0.0007) [2023-12-27 10:38:13,227][20347] Updated weights for policy 0, policy_version 723948 (0.0007) [2023-12-27 10:38:13,288][20347] Updated weights for policy 0, policy_version 723958 (0.0008) [2023-12-27 10:38:13,336][20347] Updated weights for policy 0, policy_version 723968 (0.0007) [2023-12-27 10:38:13,436][20381] Updated weights for policy 1, policy_version 723515 (0.0007) [2023-12-27 10:38:13,485][20381] Updated weights for policy 1, policy_version 723525 (0.0008) [2023-12-27 10:38:13,538][20381] Updated weights for policy 1, policy_version 723535 (0.0008) [2023-12-27 10:38:14,051][20347] Updated weights for policy 0, policy_version 723978 (0.0007) [2023-12-27 10:38:14,099][20347] Updated weights for policy 0, policy_version 723988 (0.0007) [2023-12-27 10:38:14,150][20347] Updated weights for policy 0, policy_version 723998 (0.0008) [2023-12-27 10:38:14,195][20381] Updated weights for policy 1, policy_version 723545 (0.0008) [2023-12-27 10:38:14,205][20347] Updated weights for policy 0, policy_version 724008 (0.0007) [2023-12-27 10:38:14,243][20381] Updated weights for policy 1, policy_version 723555 (0.0007) [2023-12-27 10:38:14,288][20381] Updated weights for policy 1, policy_version 723565 (0.0007) [2023-12-27 10:38:14,334][20381] Updated weights for policy 1, policy_version 723575 (0.0009) [2023-12-27 10:38:14,898][20347] Updated weights for policy 0, policy_version 724018 (0.0008) [2023-12-27 10:38:14,957][20347] Updated weights for policy 0, policy_version 724028 (0.0007) [2023-12-27 10:38:15,009][20381] Updated weights for policy 1, policy_version 723585 (0.0010) [2023-12-27 10:38:15,017][20347] Updated weights for policy 0, policy_version 724038 (0.0007) [2023-12-27 10:38:15,065][20381] Updated weights for policy 1, policy_version 723595 (0.0005) [2023-12-27 10:38:15,119][20381] Updated weights for policy 1, policy_version 723605 (0.0006) [2023-12-27 10:38:15,666][20347] Updated weights for policy 0, policy_version 724048 (0.0006) [2023-12-27 10:38:15,715][20347] Updated weights for policy 0, policy_version 724058 (0.0005) [2023-12-27 10:38:15,778][20347] Updated weights for policy 0, policy_version 724068 (0.0008) [2023-12-27 10:38:15,800][20381] Updated weights for policy 1, policy_version 723615 (0.0010) [2023-12-27 10:38:15,854][20381] Updated weights for policy 1, policy_version 723625 (0.0010) [2023-12-27 10:38:15,913][20381] Updated weights for policy 1, policy_version 723635 (0.0010) [2023-12-27 10:38:16,437][20347] Updated weights for policy 0, policy_version 724078 (0.0007) [2023-12-27 10:38:16,494][20347] Updated weights for policy 0, policy_version 724088 (0.0009) [2023-12-27 10:38:16,559][20347] Updated weights for policy 0, policy_version 724098 (0.0010) [2023-12-27 10:38:16,578][20381] Updated weights for policy 1, policy_version 723645 (0.0010) [2023-12-27 10:38:16,644][20381] Updated weights for policy 1, policy_version 723655 (0.0011) [2023-12-27 10:38:16,703][20381] Updated weights for policy 1, policy_version 723665 (0.0011) [2023-12-27 10:38:17,219][20347] Updated weights for policy 0, policy_version 724108 (0.0011) [2023-12-27 10:38:17,278][20381] Updated weights for policy 1, policy_version 723675 (0.0010) [2023-12-27 10:38:17,280][20347] Updated weights for policy 0, policy_version 724118 (0.0010) [2023-12-27 10:38:17,334][20381] Updated weights for policy 1, policy_version 723685 (0.0008) [2023-12-27 10:38:17,342][20347] Updated weights for policy 0, policy_version 724128 (0.0011) [2023-12-27 10:38:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 370679808. Throughput: 0: 10308.4, 1: 10308.4. Samples: 370698944. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:17,363][19170] Avg episode reward: [(0, '797.240'), (1, '834.520')] [2023-12-27 10:38:17,390][20381] Updated weights for policy 1, policy_version 723695 (0.0006) [2023-12-27 10:38:17,983][20347] Updated weights for policy 0, policy_version 724138 (0.0011) [2023-12-27 10:38:18,028][20381] Updated weights for policy 1, policy_version 723705 (0.0006) [2023-12-27 10:38:18,041][20347] Updated weights for policy 0, policy_version 724148 (0.0009) [2023-12-27 10:38:18,093][20381] Updated weights for policy 1, policy_version 723715 (0.0008) [2023-12-27 10:38:18,104][20347] Updated weights for policy 0, policy_version 724158 (0.0007) [2023-12-27 10:38:18,155][20381] Updated weights for policy 1, policy_version 723725 (0.0009) [2023-12-27 10:38:18,163][20347] Updated weights for policy 0, policy_version 724168 (0.0011) [2023-12-27 10:38:18,211][20381] Updated weights for policy 1, policy_version 723735 (0.0008) [2023-12-27 10:38:18,865][20347] Updated weights for policy 0, policy_version 724178 (0.0010) [2023-12-27 10:38:18,896][20381] Updated weights for policy 1, policy_version 723745 (0.0009) [2023-12-27 10:38:18,927][20347] Updated weights for policy 0, policy_version 724188 (0.0010) [2023-12-27 10:38:18,948][20381] Updated weights for policy 1, policy_version 723755 (0.0007) [2023-12-27 10:38:18,994][20347] Updated weights for policy 0, policy_version 724199 (0.0010) [2023-12-27 10:38:19,003][20381] Updated weights for policy 1, policy_version 723765 (0.0007) [2023-12-27 10:38:19,767][20347] Updated weights for policy 0, policy_version 724209 (0.0010) [2023-12-27 10:38:19,771][20381] Updated weights for policy 1, policy_version 723775 (0.0009) [2023-12-27 10:38:19,830][20347] Updated weights for policy 0, policy_version 724219 (0.0009) [2023-12-27 10:38:19,845][20381] Updated weights for policy 1, policy_version 723785 (0.0008) [2023-12-27 10:38:19,893][20347] Updated weights for policy 0, policy_version 724229 (0.0008) [2023-12-27 10:38:19,909][20381] Updated weights for policy 1, policy_version 723795 (0.0007) [2023-12-27 10:38:20,590][20347] Updated weights for policy 0, policy_version 724239 (0.0010) [2023-12-27 10:38:20,631][20381] Updated weights for policy 1, policy_version 723805 (0.0008) [2023-12-27 10:38:20,649][20347] Updated weights for policy 0, policy_version 724249 (0.0008) [2023-12-27 10:38:20,692][20381] Updated weights for policy 1, policy_version 723815 (0.0006) [2023-12-27 10:38:20,699][20347] Updated weights for policy 0, policy_version 724259 (0.0010) [2023-12-27 10:38:20,752][20381] Updated weights for policy 1, policy_version 723825 (0.0007) [2023-12-27 10:38:21,493][20347] Updated weights for policy 0, policy_version 724269 (0.0010) [2023-12-27 10:38:21,520][20381] Updated weights for policy 1, policy_version 723835 (0.0008) [2023-12-27 10:38:21,558][20347] Updated weights for policy 0, policy_version 724279 (0.0011) [2023-12-27 10:38:21,576][20381] Updated weights for policy 1, policy_version 723845 (0.0007) [2023-12-27 10:38:21,613][20347] Updated weights for policy 0, policy_version 724289 (0.0008) [2023-12-27 10:38:21,651][20381] Updated weights for policy 1, policy_version 723855 (0.0007) [2023-12-27 10:38:22,319][20347] Updated weights for policy 0, policy_version 724299 (0.0009) [2023-12-27 10:38:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 370778112. Throughput: 0: 10260.3, 1: 10250.1. Samples: 370757188. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:22,363][19170] Avg episode reward: [(0, '826.370'), (1, '786.030')] [2023-12-27 10:38:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000723864_185335808.pth... [2023-12-27 10:38:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000722680_185032704.pth [2023-12-27 10:38:22,383][20347] Updated weights for policy 0, policy_version 724309 (0.0008) [2023-12-27 10:38:22,415][20381] Updated weights for policy 1, policy_version 723865 (0.0008) [2023-12-27 10:38:22,451][20347] Updated weights for policy 0, policy_version 724319 (0.0007) [2023-12-27 10:38:22,481][20381] Updated weights for policy 1, policy_version 723875 (0.0006) [2023-12-27 10:38:22,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000724328_185450496.pth... [2023-12-27 10:38:22,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000723112_185139200.pth [2023-12-27 10:38:22,538][20381] Updated weights for policy 1, policy_version 723885 (0.0007) [2023-12-27 10:38:22,599][20381] Updated weights for policy 1, policy_version 723895 (0.0008) [2023-12-27 10:38:23,140][20347] Updated weights for policy 0, policy_version 724329 (0.0006) [2023-12-27 10:38:23,192][20347] Updated weights for policy 0, policy_version 724339 (0.0005) [2023-12-27 10:38:23,247][20347] Updated weights for policy 0, policy_version 724349 (0.0006) [2023-12-27 10:38:23,271][20381] Updated weights for policy 1, policy_version 723905 (0.0006) [2023-12-27 10:38:23,299][20347] Updated weights for policy 0, policy_version 724359 (0.0006) [2023-12-27 10:38:23,324][20381] Updated weights for policy 1, policy_version 723915 (0.0006) [2023-12-27 10:38:23,380][20381] Updated weights for policy 1, policy_version 723925 (0.0007) [2023-12-27 10:38:23,979][20347] Updated weights for policy 0, policy_version 724369 (0.0007) [2023-12-27 10:38:24,039][20347] Updated weights for policy 0, policy_version 724379 (0.0007) [2023-12-27 10:38:24,089][20381] Updated weights for policy 1, policy_version 723935 (0.0008) [2023-12-27 10:38:24,104][20347] Updated weights for policy 0, policy_version 724389 (0.0005) [2023-12-27 10:38:24,153][20381] Updated weights for policy 1, policy_version 723945 (0.0008) [2023-12-27 10:38:24,199][20381] Updated weights for policy 1, policy_version 723955 (0.0009) [2023-12-27 10:38:24,683][20347] Updated weights for policy 0, policy_version 724399 (0.0007) [2023-12-27 10:38:24,735][20347] Updated weights for policy 0, policy_version 724409 (0.0007) [2023-12-27 10:38:24,793][20347] Updated weights for policy 0, policy_version 724419 (0.0008) [2023-12-27 10:38:24,927][20381] Updated weights for policy 1, policy_version 723965 (0.0008) [2023-12-27 10:38:24,984][20381] Updated weights for policy 1, policy_version 723975 (0.0008) [2023-12-27 10:38:25,041][20381] Updated weights for policy 1, policy_version 723985 (0.0008) [2023-12-27 10:38:25,493][20347] Updated weights for policy 0, policy_version 724429 (0.0007) [2023-12-27 10:38:25,550][20347] Updated weights for policy 0, policy_version 724439 (0.0008) [2023-12-27 10:38:25,601][20347] Updated weights for policy 0, policy_version 724449 (0.0008) [2023-12-27 10:38:25,737][20381] Updated weights for policy 1, policy_version 723995 (0.0008) [2023-12-27 10:38:25,799][20381] Updated weights for policy 1, policy_version 724005 (0.0009) [2023-12-27 10:38:25,858][20381] Updated weights for policy 1, policy_version 724015 (0.0008) [2023-12-27 10:38:26,323][20347] Updated weights for policy 0, policy_version 724459 (0.0008) [2023-12-27 10:38:26,384][20347] Updated weights for policy 0, policy_version 724469 (0.0008) [2023-12-27 10:38:26,447][20347] Updated weights for policy 0, policy_version 724479 (0.0007) [2023-12-27 10:38:26,557][20381] Updated weights for policy 1, policy_version 724025 (0.0008) [2023-12-27 10:38:26,627][20381] Updated weights for policy 1, policy_version 724035 (0.0008) [2023-12-27 10:38:26,682][20381] Updated weights for policy 1, policy_version 724045 (0.0008) [2023-12-27 10:38:26,738][20381] Updated weights for policy 1, policy_version 724055 (0.0008) [2023-12-27 10:38:27,138][20347] Updated weights for policy 0, policy_version 724489 (0.0008) [2023-12-27 10:38:27,197][20347] Updated weights for policy 0, policy_version 724499 (0.0007) [2023-12-27 10:38:27,247][20347] Updated weights for policy 0, policy_version 724509 (0.0007) [2023-12-27 10:38:27,302][20347] Updated weights for policy 0, policy_version 724519 (0.0007) [2023-12-27 10:38:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 370884608. Throughput: 0: 10244.1, 1: 10161.1. Samples: 370874984. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 10:38:27,363][19170] Avg episode reward: [(0, '874.870'), (1, '863.790')] [2023-12-27 10:38:27,401][20381] Updated weights for policy 1, policy_version 724065 (0.0006) [2023-12-27 10:38:27,467][20381] Updated weights for policy 1, policy_version 724075 (0.0006) [2023-12-27 10:38:27,523][20381] Updated weights for policy 1, policy_version 724085 (0.0005) [2023-12-27 10:38:27,956][20347] Updated weights for policy 0, policy_version 724529 (0.0008) [2023-12-27 10:38:28,008][20347] Updated weights for policy 0, policy_version 724539 (0.0007) [2023-12-27 10:38:28,062][20347] Updated weights for policy 0, policy_version 724549 (0.0007) [2023-12-27 10:38:28,151][20381] Updated weights for policy 1, policy_version 724095 (0.0007) [2023-12-27 10:38:28,201][20381] Updated weights for policy 1, policy_version 724105 (0.0007) [2023-12-27 10:38:28,255][20381] Updated weights for policy 1, policy_version 724115 (0.0008) [2023-12-27 10:38:28,710][20347] Updated weights for policy 0, policy_version 724559 (0.0008) [2023-12-27 10:38:28,771][20347] Updated weights for policy 0, policy_version 724569 (0.0008) [2023-12-27 10:38:28,830][20347] Updated weights for policy 0, policy_version 724579 (0.0008) [2023-12-27 10:38:28,979][20381] Updated weights for policy 1, policy_version 724125 (0.0007) [2023-12-27 10:38:29,034][20381] Updated weights for policy 1, policy_version 724135 (0.0008) [2023-12-27 10:38:29,094][20381] Updated weights for policy 1, policy_version 724145 (0.0008) [2023-12-27 10:38:29,466][20347] Updated weights for policy 0, policy_version 724589 (0.0007) [2023-12-27 10:38:29,518][20347] Updated weights for policy 0, policy_version 724599 (0.0007) [2023-12-27 10:38:29,587][20347] Updated weights for policy 0, policy_version 724609 (0.0007) [2023-12-27 10:38:29,794][20381] Updated weights for policy 1, policy_version 724155 (0.0008) [2023-12-27 10:38:29,854][20381] Updated weights for policy 1, policy_version 724165 (0.0006) [2023-12-27 10:38:29,917][20381] Updated weights for policy 1, policy_version 724175 (0.0007) [2023-12-27 10:38:30,291][20347] Updated weights for policy 0, policy_version 724619 (0.0007) [2023-12-27 10:38:30,347][20347] Updated weights for policy 0, policy_version 724629 (0.0008) [2023-12-27 10:38:30,400][20347] Updated weights for policy 0, policy_version 724639 (0.0007) [2023-12-27 10:38:30,620][20381] Updated weights for policy 1, policy_version 724185 (0.0007) [2023-12-27 10:38:30,675][20381] Updated weights for policy 1, policy_version 724195 (0.0005) [2023-12-27 10:38:30,727][20381] Updated weights for policy 1, policy_version 724205 (0.0008) [2023-12-27 10:38:30,783][20381] Updated weights for policy 1, policy_version 724215 (0.0008) [2023-12-27 10:38:31,106][20347] Updated weights for policy 0, policy_version 724649 (0.0008) [2023-12-27 10:38:31,167][20347] Updated weights for policy 0, policy_version 724659 (0.0007) [2023-12-27 10:38:31,224][20347] Updated weights for policy 0, policy_version 724669 (0.0008) [2023-12-27 10:38:31,283][20347] Updated weights for policy 0, policy_version 724679 (0.0008) [2023-12-27 10:38:31,510][20381] Updated weights for policy 1, policy_version 724225 (0.0010) [2023-12-27 10:38:31,569][20381] Updated weights for policy 1, policy_version 724235 (0.0011) [2023-12-27 10:38:31,625][20381] Updated weights for policy 1, policy_version 724245 (0.0010) [2023-12-27 10:38:32,058][20347] Updated weights for policy 0, policy_version 724689 (0.0008) [2023-12-27 10:38:32,108][20347] Updated weights for policy 0, policy_version 724699 (0.0007) [2023-12-27 10:38:32,162][20347] Updated weights for policy 0, policy_version 724709 (0.0007) [2023-12-27 10:38:32,261][20381] Updated weights for policy 1, policy_version 724255 (0.0009) [2023-12-27 10:38:32,317][20381] Updated weights for policy 1, policy_version 724265 (0.0010) [2023-12-27 10:38:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 370982912. Throughput: 0: 10212.4, 1: 10142.1. Samples: 370996088. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:32,363][19170] Avg episode reward: [(0, '893.890'), (1, '922.090')] [2023-12-27 10:38:32,381][20381] Updated weights for policy 1, policy_version 724275 (0.0009) [2023-12-27 10:38:32,841][20347] Updated weights for policy 0, policy_version 724719 (0.0009) [2023-12-27 10:38:32,896][20347] Updated weights for policy 0, policy_version 724729 (0.0010) [2023-12-27 10:38:32,951][20347] Updated weights for policy 0, policy_version 724739 (0.0010) [2023-12-27 10:38:33,024][20381] Updated weights for policy 1, policy_version 724285 (0.0007) [2023-12-27 10:38:33,071][20381] Updated weights for policy 1, policy_version 724295 (0.0005) [2023-12-27 10:38:33,128][20381] Updated weights for policy 1, policy_version 724305 (0.0008) [2023-12-27 10:38:33,602][20347] Updated weights for policy 0, policy_version 724749 (0.0008) [2023-12-27 10:38:33,670][20347] Updated weights for policy 0, policy_version 724759 (0.0006) [2023-12-27 10:38:33,725][20381] Updated weights for policy 1, policy_version 724315 (0.0009) [2023-12-27 10:38:33,726][20347] Updated weights for policy 0, policy_version 724769 (0.0006) [2023-12-27 10:38:33,774][20381] Updated weights for policy 1, policy_version 724325 (0.0006) [2023-12-27 10:38:33,835][20381] Updated weights for policy 1, policy_version 724335 (0.0008) [2023-12-27 10:38:34,347][20347] Updated weights for policy 0, policy_version 724779 (0.0007) [2023-12-27 10:38:34,402][20347] Updated weights for policy 0, policy_version 724789 (0.0007) [2023-12-27 10:38:34,460][20347] Updated weights for policy 0, policy_version 724799 (0.0007) [2023-12-27 10:38:34,541][20381] Updated weights for policy 1, policy_version 724345 (0.0007) [2023-12-27 10:38:34,598][20381] Updated weights for policy 1, policy_version 724355 (0.0007) [2023-12-27 10:38:34,647][20381] Updated weights for policy 1, policy_version 724365 (0.0009) [2023-12-27 10:38:34,700][20381] Updated weights for policy 1, policy_version 724375 (0.0008) [2023-12-27 10:38:35,122][20347] Updated weights for policy 0, policy_version 724809 (0.0007) [2023-12-27 10:38:35,185][20347] Updated weights for policy 0, policy_version 724819 (0.0010) [2023-12-27 10:38:35,243][20347] Updated weights for policy 0, policy_version 724829 (0.0010) [2023-12-27 10:38:35,304][20347] Updated weights for policy 0, policy_version 724839 (0.0010) [2023-12-27 10:38:35,358][20381] Updated weights for policy 1, policy_version 724385 (0.0005) [2023-12-27 10:38:35,404][20381] Updated weights for policy 1, policy_version 724395 (0.0005) [2023-12-27 10:38:35,461][20381] Updated weights for policy 1, policy_version 724405 (0.0006) [2023-12-27 10:38:35,934][20347] Updated weights for policy 0, policy_version 724849 (0.0010) [2023-12-27 10:38:35,991][20347] Updated weights for policy 0, policy_version 724859 (0.0010) [2023-12-27 10:38:36,046][20347] Updated weights for policy 0, policy_version 724869 (0.0010) [2023-12-27 10:38:36,082][20381] Updated weights for policy 1, policy_version 724415 (0.0008) [2023-12-27 10:38:36,146][20381] Updated weights for policy 1, policy_version 724425 (0.0007) [2023-12-27 10:38:36,207][20381] Updated weights for policy 1, policy_version 724435 (0.0007) [2023-12-27 10:38:36,739][20347] Updated weights for policy 0, policy_version 724879 (0.0010) [2023-12-27 10:38:36,787][20347] Updated weights for policy 0, policy_version 724889 (0.0010) [2023-12-27 10:38:36,832][20347] Updated weights for policy 0, policy_version 724899 (0.0010) [2023-12-27 10:38:36,909][20381] Updated weights for policy 1, policy_version 724445 (0.0007) [2023-12-27 10:38:36,970][20381] Updated weights for policy 1, policy_version 724455 (0.0007) [2023-12-27 10:38:37,021][20381] Updated weights for policy 1, policy_version 724465 (0.0007) [2023-12-27 10:38:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 371089408. Throughput: 0: 10223.7, 1: 10128.5. Samples: 371058896. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:37,363][19170] Avg episode reward: [(0, '923.380'), (1, '892.840')] [2023-12-27 10:38:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000724472_185491456.pth... [2023-12-27 10:38:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000724904_185597952.pth... [2023-12-27 10:38:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000723288_185188352.pth [2023-12-27 10:38:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000723720_185294848.pth [2023-12-27 10:38:37,502][20347] Updated weights for policy 0, policy_version 724909 (0.0009) [2023-12-27 10:38:37,556][20347] Updated weights for policy 0, policy_version 724919 (0.0008) [2023-12-27 10:38:37,615][20347] Updated weights for policy 0, policy_version 724929 (0.0008) [2023-12-27 10:38:37,735][20381] Updated weights for policy 1, policy_version 724475 (0.0008) [2023-12-27 10:38:37,785][20381] Updated weights for policy 1, policy_version 724485 (0.0007) [2023-12-27 10:38:37,818][20292] KL-divergence is very high: 111.6494 [2023-12-27 10:38:37,832][20381] Updated weights for policy 1, policy_version 724495 (0.0007) [2023-12-27 10:38:38,261][20347] Updated weights for policy 0, policy_version 724939 (0.0008) [2023-12-27 10:38:38,311][20347] Updated weights for policy 0, policy_version 724949 (0.0006) [2023-12-27 10:38:38,373][20347] Updated weights for policy 0, policy_version 724959 (0.0007) [2023-12-27 10:38:38,515][20381] Updated weights for policy 1, policy_version 724505 (0.0007) [2023-12-27 10:38:38,575][20381] Updated weights for policy 1, policy_version 724515 (0.0007) [2023-12-27 10:38:38,631][20381] Updated weights for policy 1, policy_version 724525 (0.0008) [2023-12-27 10:38:38,684][20381] Updated weights for policy 1, policy_version 724535 (0.0007) [2023-12-27 10:38:39,033][20347] Updated weights for policy 0, policy_version 724969 (0.0006) [2023-12-27 10:38:39,084][20347] Updated weights for policy 0, policy_version 724979 (0.0006) [2023-12-27 10:38:39,147][20347] Updated weights for policy 0, policy_version 724989 (0.0005) [2023-12-27 10:38:39,206][20347] Updated weights for policy 0, policy_version 724999 (0.0006) [2023-12-27 10:38:39,387][20381] Updated weights for policy 1, policy_version 724545 (0.0009) [2023-12-27 10:38:39,452][20381] Updated weights for policy 1, policy_version 724555 (0.0008) [2023-12-27 10:38:39,515][20381] Updated weights for policy 1, policy_version 724565 (0.0007) [2023-12-27 10:38:39,940][20347] Updated weights for policy 0, policy_version 725009 (0.0008) [2023-12-27 10:38:39,993][20347] Updated weights for policy 0, policy_version 725019 (0.0006) [2023-12-27 10:38:40,043][20347] Updated weights for policy 0, policy_version 725029 (0.0006) [2023-12-27 10:38:40,229][20381] Updated weights for policy 1, policy_version 724575 (0.0008) [2023-12-27 10:38:40,282][20381] Updated weights for policy 1, policy_version 724585 (0.0009) [2023-12-27 10:38:40,335][20381] Updated weights for policy 1, policy_version 724595 (0.0009) [2023-12-27 10:38:40,743][20347] Updated weights for policy 0, policy_version 725039 (0.0007) [2023-12-27 10:38:40,799][20347] Updated weights for policy 0, policy_version 725049 (0.0007) [2023-12-27 10:38:40,848][20347] Updated weights for policy 0, policy_version 725059 (0.0007) [2023-12-27 10:38:41,006][20381] Updated weights for policy 1, policy_version 724605 (0.0008) [2023-12-27 10:38:41,092][20381] Updated weights for policy 1, policy_version 724615 (0.0008) [2023-12-27 10:38:41,162][20381] Updated weights for policy 1, policy_version 724625 (0.0008) [2023-12-27 10:38:41,571][20347] Updated weights for policy 0, policy_version 725069 (0.0008) [2023-12-27 10:38:41,637][20347] Updated weights for policy 0, policy_version 725079 (0.0008) [2023-12-27 10:38:41,701][20347] Updated weights for policy 0, policy_version 725089 (0.0008) [2023-12-27 10:38:41,927][20381] Updated weights for policy 1, policy_version 724635 (0.0008) [2023-12-27 10:38:41,984][20381] Updated weights for policy 1, policy_version 724645 (0.0008) [2023-12-27 10:38:42,039][20381] Updated weights for policy 1, policy_version 724655 (0.0010) [2023-12-27 10:38:42,272][20347] Updated weights for policy 0, policy_version 725099 (0.0008) [2023-12-27 10:38:42,330][20347] Updated weights for policy 0, policy_version 725109 (0.0008) [2023-12-27 10:38:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 371187712. Throughput: 0: 10209.9, 1: 10122.4. Samples: 371180468. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:42,363][19170] Avg episode reward: [(0, '941.800'), (1, '824.670')] [2023-12-27 10:38:42,393][20347] Updated weights for policy 0, policy_version 725119 (0.0007) [2023-12-27 10:38:42,775][20381] Updated weights for policy 1, policy_version 724665 (0.0008) [2023-12-27 10:38:42,829][20381] Updated weights for policy 1, policy_version 724675 (0.0006) [2023-12-27 10:38:42,883][20381] Updated weights for policy 1, policy_version 724685 (0.0005) [2023-12-27 10:38:42,933][20381] Updated weights for policy 1, policy_version 724695 (0.0005) [2023-12-27 10:38:43,123][20347] Updated weights for policy 0, policy_version 725129 (0.0007) [2023-12-27 10:38:43,177][20347] Updated weights for policy 0, policy_version 725139 (0.0007) [2023-12-27 10:38:43,229][20347] Updated weights for policy 0, policy_version 725149 (0.0007) [2023-12-27 10:38:43,278][20347] Updated weights for policy 0, policy_version 725159 (0.0007) [2023-12-27 10:38:43,481][20381] Updated weights for policy 1, policy_version 724705 (0.0010) [2023-12-27 10:38:43,530][20381] Updated weights for policy 1, policy_version 724715 (0.0010) [2023-12-27 10:38:43,579][20381] Updated weights for policy 1, policy_version 724725 (0.0010) [2023-12-27 10:38:43,982][20347] Updated weights for policy 0, policy_version 725169 (0.0005) [2023-12-27 10:38:44,037][20347] Updated weights for policy 0, policy_version 725179 (0.0006) [2023-12-27 10:38:44,089][20347] Updated weights for policy 0, policy_version 725189 (0.0007) [2023-12-27 10:38:44,294][20381] Updated weights for policy 1, policy_version 724735 (0.0010) [2023-12-27 10:38:44,349][20381] Updated weights for policy 1, policy_version 724745 (0.0010) [2023-12-27 10:38:44,406][20381] Updated weights for policy 1, policy_version 724755 (0.0010) [2023-12-27 10:38:44,706][20347] Updated weights for policy 0, policy_version 725199 (0.0008) [2023-12-27 10:38:44,764][20347] Updated weights for policy 0, policy_version 725209 (0.0007) [2023-12-27 10:38:44,823][20347] Updated weights for policy 0, policy_version 725219 (0.0009) [2023-12-27 10:38:45,081][20381] Updated weights for policy 1, policy_version 724765 (0.0010) [2023-12-27 10:38:45,136][20381] Updated weights for policy 1, policy_version 724775 (0.0007) [2023-12-27 10:38:45,195][20381] Updated weights for policy 1, policy_version 724785 (0.0008) [2023-12-27 10:38:45,549][20347] Updated weights for policy 0, policy_version 725229 (0.0007) [2023-12-27 10:38:45,612][20347] Updated weights for policy 0, policy_version 725239 (0.0006) [2023-12-27 10:38:45,675][20347] Updated weights for policy 0, policy_version 725249 (0.0007) [2023-12-27 10:38:45,856][20381] Updated weights for policy 1, policy_version 724795 (0.0011) [2023-12-27 10:38:45,905][20381] Updated weights for policy 1, policy_version 724805 (0.0010) [2023-12-27 10:38:45,962][20381] Updated weights for policy 1, policy_version 724815 (0.0011) [2023-12-27 10:38:46,278][20347] Updated weights for policy 0, policy_version 725259 (0.0006) [2023-12-27 10:38:46,333][20347] Updated weights for policy 0, policy_version 725269 (0.0010) [2023-12-27 10:38:46,381][20347] Updated weights for policy 0, policy_version 725279 (0.0010) [2023-12-27 10:38:46,645][20381] Updated weights for policy 1, policy_version 724825 (0.0011) [2023-12-27 10:38:46,705][20381] Updated weights for policy 1, policy_version 724835 (0.0010) [2023-12-27 10:38:46,756][20381] Updated weights for policy 1, policy_version 724845 (0.0010) [2023-12-27 10:38:46,809][20381] Updated weights for policy 1, policy_version 724855 (0.0011) [2023-12-27 10:38:47,020][20347] Updated weights for policy 0, policy_version 725289 (0.0010) [2023-12-27 10:38:47,085][20347] Updated weights for policy 0, policy_version 725299 (0.0008) [2023-12-27 10:38:47,147][20347] Updated weights for policy 0, policy_version 725309 (0.0007) [2023-12-27 10:38:47,200][20347] Updated weights for policy 0, policy_version 725319 (0.0010) [2023-12-27 10:38:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 371294208. Throughput: 0: 10200.1, 1: 10157.4. Samples: 371304376. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:47,363][19170] Avg episode reward: [(0, '922.190'), (1, '785.680')] [2023-12-27 10:38:47,477][20381] Updated weights for policy 1, policy_version 724865 (0.0010) [2023-12-27 10:38:47,528][20381] Updated weights for policy 1, policy_version 724875 (0.0010) [2023-12-27 10:38:47,582][20381] Updated weights for policy 1, policy_version 724885 (0.0010) [2023-12-27 10:38:47,853][20347] Updated weights for policy 0, policy_version 725329 (0.0010) [2023-12-27 10:38:47,911][20347] Updated weights for policy 0, policy_version 725339 (0.0010) [2023-12-27 10:38:47,968][20347] Updated weights for policy 0, policy_version 725349 (0.0008) [2023-12-27 10:38:48,253][20381] Updated weights for policy 1, policy_version 724895 (0.0010) [2023-12-27 10:38:48,312][20381] Updated weights for policy 1, policy_version 724905 (0.0010) [2023-12-27 10:38:48,372][20381] Updated weights for policy 1, policy_version 724915 (0.0008) [2023-12-27 10:38:48,638][20347] Updated weights for policy 0, policy_version 725359 (0.0007) [2023-12-27 10:38:48,686][20347] Updated weights for policy 0, policy_version 725369 (0.0007) [2023-12-27 10:38:48,734][20347] Updated weights for policy 0, policy_version 725379 (0.0007) [2023-12-27 10:38:49,081][20381] Updated weights for policy 1, policy_version 724925 (0.0008) [2023-12-27 10:38:49,135][20381] Updated weights for policy 1, policy_version 724935 (0.0011) [2023-12-27 10:38:49,190][20381] Updated weights for policy 1, policy_version 724945 (0.0010) [2023-12-27 10:38:49,589][20347] Updated weights for policy 0, policy_version 725389 (0.0009) [2023-12-27 10:38:49,645][20347] Updated weights for policy 0, policy_version 725399 (0.0010) [2023-12-27 10:38:49,700][20347] Updated weights for policy 0, policy_version 725409 (0.0010) [2023-12-27 10:38:49,853][20381] Updated weights for policy 1, policy_version 724955 (0.0011) [2023-12-27 10:38:49,909][20381] Updated weights for policy 1, policy_version 724965 (0.0011) [2023-12-27 10:38:49,966][20381] Updated weights for policy 1, policy_version 724975 (0.0012) [2023-12-27 10:38:50,422][20347] Updated weights for policy 0, policy_version 725419 (0.0010) [2023-12-27 10:38:50,481][20347] Updated weights for policy 0, policy_version 725429 (0.0010) [2023-12-27 10:38:50,540][20347] Updated weights for policy 0, policy_version 725439 (0.0010) [2023-12-27 10:38:50,665][20381] Updated weights for policy 1, policy_version 724985 (0.0010) [2023-12-27 10:38:50,726][20381] Updated weights for policy 1, policy_version 724995 (0.0008) [2023-12-27 10:38:50,789][20381] Updated weights for policy 1, policy_version 725005 (0.0011) [2023-12-27 10:38:50,844][20381] Updated weights for policy 1, policy_version 725015 (0.0010) [2023-12-27 10:38:51,223][20347] Updated weights for policy 0, policy_version 725449 (0.0008) [2023-12-27 10:38:51,284][20347] Updated weights for policy 0, policy_version 725459 (0.0009) [2023-12-27 10:38:51,345][20347] Updated weights for policy 0, policy_version 725469 (0.0007) [2023-12-27 10:38:51,414][20347] Updated weights for policy 0, policy_version 725479 (0.0007) [2023-12-27 10:38:51,540][20381] Updated weights for policy 1, policy_version 725025 (0.0008) [2023-12-27 10:38:51,602][20381] Updated weights for policy 1, policy_version 725035 (0.0008) [2023-12-27 10:38:51,666][20381] Updated weights for policy 1, policy_version 725045 (0.0008) [2023-12-27 10:38:52,055][20347] Updated weights for policy 0, policy_version 725489 (0.0008) [2023-12-27 10:38:52,113][20347] Updated weights for policy 0, policy_version 725499 (0.0008) [2023-12-27 10:38:52,172][20347] Updated weights for policy 0, policy_version 725509 (0.0006) [2023-12-27 10:38:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 371392512. Throughput: 0: 10163.6, 1: 10140.5. Samples: 371363592. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:52,363][19170] Avg episode reward: [(0, '941.560'), (1, '824.950')] [2023-12-27 10:38:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000725512_185753600.pth... [2023-12-27 10:38:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000724328_185450496.pth [2023-12-27 10:38:52,404][20381] Updated weights for policy 1, policy_version 725055 (0.0008) [2023-12-27 10:38:52,465][20381] Updated weights for policy 1, policy_version 725065 (0.0007) [2023-12-27 10:38:52,523][20381] Updated weights for policy 1, policy_version 725075 (0.0008) [2023-12-27 10:38:52,555][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000725080_185647104.pth... [2023-12-27 10:38:52,560][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000723864_185335808.pth [2023-12-27 10:38:52,975][20347] Updated weights for policy 0, policy_version 725519 (0.0010) [2023-12-27 10:38:53,029][20347] Updated weights for policy 0, policy_version 725529 (0.0010) [2023-12-27 10:38:53,081][20347] Updated weights for policy 0, policy_version 725539 (0.0010) [2023-12-27 10:38:53,224][20381] Updated weights for policy 1, policy_version 725085 (0.0008) [2023-12-27 10:38:53,275][20381] Updated weights for policy 1, policy_version 725095 (0.0010) [2023-12-27 10:38:53,333][20381] Updated weights for policy 1, policy_version 725105 (0.0010) [2023-12-27 10:38:53,778][20347] Updated weights for policy 0, policy_version 725549 (0.0009) [2023-12-27 10:38:53,833][20347] Updated weights for policy 0, policy_version 725559 (0.0007) [2023-12-27 10:38:53,883][20347] Updated weights for policy 0, policy_version 725569 (0.0007) [2023-12-27 10:38:53,997][20381] Updated weights for policy 1, policy_version 725115 (0.0010) [2023-12-27 10:38:54,058][20381] Updated weights for policy 1, policy_version 725125 (0.0010) [2023-12-27 10:38:54,112][20381] Updated weights for policy 1, policy_version 725135 (0.0010) [2023-12-27 10:38:54,561][20347] Updated weights for policy 0, policy_version 725579 (0.0007) [2023-12-27 10:38:54,614][20347] Updated weights for policy 0, policy_version 725589 (0.0007) [2023-12-27 10:38:54,667][20347] Updated weights for policy 0, policy_version 725599 (0.0006) [2023-12-27 10:38:54,750][20381] Updated weights for policy 1, policy_version 725145 (0.0010) [2023-12-27 10:38:54,810][20381] Updated weights for policy 1, policy_version 725155 (0.0005) [2023-12-27 10:38:54,854][20381] Updated weights for policy 1, policy_version 725165 (0.0005) [2023-12-27 10:38:54,908][20381] Updated weights for policy 1, policy_version 725175 (0.0005) [2023-12-27 10:38:55,390][20347] Updated weights for policy 0, policy_version 725609 (0.0007) [2023-12-27 10:38:55,445][20347] Updated weights for policy 0, policy_version 725619 (0.0006) [2023-12-27 10:38:55,489][20347] Updated weights for policy 0, policy_version 725629 (0.0007) [2023-12-27 10:38:55,520][20381] Updated weights for policy 1, policy_version 725185 (0.0006) [2023-12-27 10:38:55,537][20347] Updated weights for policy 0, policy_version 725639 (0.0005) [2023-12-27 10:38:55,570][20381] Updated weights for policy 1, policy_version 725195 (0.0006) [2023-12-27 10:38:55,628][20381] Updated weights for policy 1, policy_version 725205 (0.0008) [2023-12-27 10:38:56,152][20347] Updated weights for policy 0, policy_version 725649 (0.0005) [2023-12-27 10:38:56,206][20347] Updated weights for policy 0, policy_version 725659 (0.0006) [2023-12-27 10:38:56,261][20347] Updated weights for policy 0, policy_version 725669 (0.0007) [2023-12-27 10:38:56,289][20381] Updated weights for policy 1, policy_version 725215 (0.0009) [2023-12-27 10:38:56,334][20381] Updated weights for policy 1, policy_version 725225 (0.0009) [2023-12-27 10:38:56,382][20381] Updated weights for policy 1, policy_version 725235 (0.0005) [2023-12-27 10:38:56,802][20347] Updated weights for policy 0, policy_version 725679 (0.0006) [2023-12-27 10:38:56,861][20347] Updated weights for policy 0, policy_version 725689 (0.0005) [2023-12-27 10:38:56,927][20347] Updated weights for policy 0, policy_version 725699 (0.0005) [2023-12-27 10:38:57,122][20381] Updated weights for policy 1, policy_version 725245 (0.0006) [2023-12-27 10:38:57,168][20381] Updated weights for policy 1, policy_version 725255 (0.0008) [2023-12-27 10:38:57,217][20381] Updated weights for policy 1, policy_version 725265 (0.0008) [2023-12-27 10:38:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 371499008. Throughput: 0: 10134.5, 1: 10104.9. Samples: 371486252. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:38:57,363][19170] Avg episode reward: [(0, '951.410'), (1, '786.530')] [2023-12-27 10:38:57,528][20347] Updated weights for policy 0, policy_version 725709 (0.0005) [2023-12-27 10:38:57,580][20347] Updated weights for policy 0, policy_version 725719 (0.0007) [2023-12-27 10:38:57,624][20347] Updated weights for policy 0, policy_version 725729 (0.0008) [2023-12-27 10:38:57,895][20381] Updated weights for policy 1, policy_version 725275 (0.0007) [2023-12-27 10:38:57,958][20381] Updated weights for policy 1, policy_version 725285 (0.0005) [2023-12-27 10:38:58,017][20381] Updated weights for policy 1, policy_version 725295 (0.0008) [2023-12-27 10:38:58,246][20347] Updated weights for policy 0, policy_version 725739 (0.0007) [2023-12-27 10:38:58,292][20347] Updated weights for policy 0, policy_version 725749 (0.0008) [2023-12-27 10:38:58,365][20347] Updated weights for policy 0, policy_version 725759 (0.0008) [2023-12-27 10:38:58,708][20381] Updated weights for policy 1, policy_version 725305 (0.0010) [2023-12-27 10:38:58,766][20381] Updated weights for policy 1, policy_version 725315 (0.0007) [2023-12-27 10:38:58,831][20381] Updated weights for policy 1, policy_version 725325 (0.0008) [2023-12-27 10:38:58,901][20381] Updated weights for policy 1, policy_version 725335 (0.0009) [2023-12-27 10:38:59,096][20347] Updated weights for policy 0, policy_version 725769 (0.0008) [2023-12-27 10:38:59,147][20347] Updated weights for policy 0, policy_version 725779 (0.0007) [2023-12-27 10:38:59,202][20347] Updated weights for policy 0, policy_version 725789 (0.0008) [2023-12-27 10:38:59,273][20347] Updated weights for policy 0, policy_version 725799 (0.0007) [2023-12-27 10:38:59,688][20381] Updated weights for policy 1, policy_version 725345 (0.0006) [2023-12-27 10:38:59,742][20381] Updated weights for policy 1, policy_version 725355 (0.0006) [2023-12-27 10:38:59,803][20381] Updated weights for policy 1, policy_version 725365 (0.0007) [2023-12-27 10:38:59,994][20347] Updated weights for policy 0, policy_version 725809 (0.0009) [2023-12-27 10:39:00,046][20347] Updated weights for policy 0, policy_version 725819 (0.0008) [2023-12-27 10:39:00,095][20347] Updated weights for policy 0, policy_version 725829 (0.0007) [2023-12-27 10:39:00,432][20381] Updated weights for policy 1, policy_version 725375 (0.0006) [2023-12-27 10:39:00,495][20381] Updated weights for policy 1, policy_version 725385 (0.0005) [2023-12-27 10:39:00,549][20381] Updated weights for policy 1, policy_version 725395 (0.0008) [2023-12-27 10:39:00,830][20347] Updated weights for policy 0, policy_version 725839 (0.0007) [2023-12-27 10:39:00,883][20347] Updated weights for policy 0, policy_version 725849 (0.0007) [2023-12-27 10:39:00,934][20347] Updated weights for policy 0, policy_version 725859 (0.0008) [2023-12-27 10:39:01,157][20381] Updated weights for policy 1, policy_version 725405 (0.0008) [2023-12-27 10:39:01,217][20381] Updated weights for policy 1, policy_version 725415 (0.0007) [2023-12-27 10:39:01,278][20381] Updated weights for policy 1, policy_version 725425 (0.0007) [2023-12-27 10:39:01,698][20347] Updated weights for policy 0, policy_version 725869 (0.0007) [2023-12-27 10:39:01,758][20347] Updated weights for policy 0, policy_version 725879 (0.0008) [2023-12-27 10:39:01,808][20347] Updated weights for policy 0, policy_version 725889 (0.0008) [2023-12-27 10:39:01,937][20381] Updated weights for policy 1, policy_version 725435 (0.0007) [2023-12-27 10:39:01,992][20381] Updated weights for policy 1, policy_version 725445 (0.0008) [2023-12-27 10:39:02,050][20381] Updated weights for policy 1, policy_version 725455 (0.0007) [2023-12-27 10:39:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 371597312. Throughput: 0: 10123.8, 1: 10081.3. Samples: 371608176. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:02,363][19170] Avg episode reward: [(0, '980.560'), (1, '718.260')] [2023-12-27 10:39:02,534][20347] Updated weights for policy 0, policy_version 725899 (0.0007) [2023-12-27 10:39:02,590][20347] Updated weights for policy 0, policy_version 725909 (0.0008) [2023-12-27 10:39:02,646][20347] Updated weights for policy 0, policy_version 725919 (0.0008) [2023-12-27 10:39:02,739][20381] Updated weights for policy 1, policy_version 725465 (0.0008) [2023-12-27 10:39:02,795][20381] Updated weights for policy 1, policy_version 725475 (0.0008) [2023-12-27 10:39:02,842][20381] Updated weights for policy 1, policy_version 725485 (0.0006) [2023-12-27 10:39:02,897][20381] Updated weights for policy 1, policy_version 725495 (0.0008) [2023-12-27 10:39:03,305][20347] Updated weights for policy 0, policy_version 725929 (0.0007) [2023-12-27 10:39:03,353][20347] Updated weights for policy 0, policy_version 725939 (0.0007) [2023-12-27 10:39:03,401][20347] Updated weights for policy 0, policy_version 725949 (0.0008) [2023-12-27 10:39:03,453][20347] Updated weights for policy 0, policy_version 725959 (0.0009) [2023-12-27 10:39:03,570][20381] Updated weights for policy 1, policy_version 725505 (0.0005) [2023-12-27 10:39:03,625][20381] Updated weights for policy 1, policy_version 725515 (0.0005) [2023-12-27 10:39:03,680][20381] Updated weights for policy 1, policy_version 725525 (0.0007) [2023-12-27 10:39:04,121][20347] Updated weights for policy 0, policy_version 725969 (0.0008) [2023-12-27 10:39:04,178][20347] Updated weights for policy 0, policy_version 725979 (0.0008) [2023-12-27 10:39:04,237][20347] Updated weights for policy 0, policy_version 725989 (0.0008) [2023-12-27 10:39:04,349][20381] Updated weights for policy 1, policy_version 725535 (0.0008) [2023-12-27 10:39:04,413][20381] Updated weights for policy 1, policy_version 725545 (0.0007) [2023-12-27 10:39:04,474][20381] Updated weights for policy 1, policy_version 725555 (0.0008) [2023-12-27 10:39:04,970][20347] Updated weights for policy 0, policy_version 725999 (0.0007) [2023-12-27 10:39:05,020][20347] Updated weights for policy 0, policy_version 726009 (0.0007) [2023-12-27 10:39:05,075][20347] Updated weights for policy 0, policy_version 726019 (0.0007) [2023-12-27 10:39:05,201][20381] Updated weights for policy 1, policy_version 725565 (0.0007) [2023-12-27 10:39:05,252][20381] Updated weights for policy 1, policy_version 725575 (0.0008) [2023-12-27 10:39:05,309][20381] Updated weights for policy 1, policy_version 725585 (0.0008) [2023-12-27 10:39:05,649][20347] Updated weights for policy 0, policy_version 726029 (0.0005) [2023-12-27 10:39:05,704][20347] Updated weights for policy 0, policy_version 726039 (0.0007) [2023-12-27 10:39:05,762][20347] Updated weights for policy 0, policy_version 726049 (0.0007) [2023-12-27 10:39:06,041][20381] Updated weights for policy 1, policy_version 725595 (0.0007) [2023-12-27 10:39:06,090][20381] Updated weights for policy 1, policy_version 725605 (0.0010) [2023-12-27 10:39:06,157][20381] Updated weights for policy 1, policy_version 725615 (0.0007) [2023-12-27 10:39:06,406][20347] Updated weights for policy 0, policy_version 726059 (0.0007) [2023-12-27 10:39:06,471][20347] Updated weights for policy 0, policy_version 726069 (0.0006) [2023-12-27 10:39:06,529][20347] Updated weights for policy 0, policy_version 726079 (0.0005) [2023-12-27 10:39:06,915][20381] Updated weights for policy 1, policy_version 725625 (0.0007) [2023-12-27 10:39:06,964][20381] Updated weights for policy 1, policy_version 725635 (0.0009) [2023-12-27 10:39:07,012][20381] Updated weights for policy 1, policy_version 725645 (0.0010) [2023-12-27 10:39:07,072][20381] Updated weights for policy 1, policy_version 725655 (0.0010) [2023-12-27 10:39:07,157][20347] Updated weights for policy 0, policy_version 726089 (0.0007) [2023-12-27 10:39:07,218][20347] Updated weights for policy 0, policy_version 726099 (0.0007) [2023-12-27 10:39:07,267][20347] Updated weights for policy 0, policy_version 726109 (0.0008) [2023-12-27 10:39:07,323][20347] Updated weights for policy 0, policy_version 726119 (0.0007) [2023-12-27 10:39:07,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 371703808. Throughput: 0: 10148.5, 1: 10107.6. Samples: 371668716. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:07,364][19170] Avg episode reward: [(0, '990.250'), (1, '728.120')] [2023-12-27 10:39:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000725656_185794560.pth... [2023-12-27 10:39:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000726120_185909248.pth... [2023-12-27 10:39:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000724904_185597952.pth [2023-12-27 10:39:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000724472_185491456.pth [2023-12-27 10:39:07,701][20381] Updated weights for policy 1, policy_version 725665 (0.0008) [2023-12-27 10:39:07,752][20381] Updated weights for policy 1, policy_version 725675 (0.0007) [2023-12-27 10:39:07,807][20381] Updated weights for policy 1, policy_version 725685 (0.0008) [2023-12-27 10:39:08,017][20347] Updated weights for policy 0, policy_version 726129 (0.0011) [2023-12-27 10:39:08,072][20347] Updated weights for policy 0, policy_version 726139 (0.0010) [2023-12-27 10:39:08,127][20347] Updated weights for policy 0, policy_version 726149 (0.0010) [2023-12-27 10:39:08,475][20381] Updated weights for policy 1, policy_version 725695 (0.0007) [2023-12-27 10:39:08,532][20381] Updated weights for policy 1, policy_version 725705 (0.0007) [2023-12-27 10:39:08,597][20381] Updated weights for policy 1, policy_version 725715 (0.0007) [2023-12-27 10:39:08,815][20347] Updated weights for policy 0, policy_version 726159 (0.0009) [2023-12-27 10:39:08,876][20347] Updated weights for policy 0, policy_version 726169 (0.0008) [2023-12-27 10:39:08,921][20347] Updated weights for policy 0, policy_version 726179 (0.0007) [2023-12-27 10:39:09,224][20381] Updated weights for policy 1, policy_version 725725 (0.0010) [2023-12-27 10:39:09,284][20381] Updated weights for policy 1, policy_version 725735 (0.0008) [2023-12-27 10:39:09,347][20381] Updated weights for policy 1, policy_version 725745 (0.0007) [2023-12-27 10:39:09,556][20347] Updated weights for policy 0, policy_version 726189 (0.0006) [2023-12-27 10:39:09,621][20347] Updated weights for policy 0, policy_version 726199 (0.0006) [2023-12-27 10:39:09,675][20347] Updated weights for policy 0, policy_version 726209 (0.0006) [2023-12-27 10:39:10,020][20381] Updated weights for policy 1, policy_version 725755 (0.0009) [2023-12-27 10:39:10,055][20292] KL-divergence is very high: 119.7305 [2023-12-27 10:39:10,080][20381] Updated weights for policy 1, policy_version 725765 (0.0008) [2023-12-27 10:39:10,135][20381] Updated weights for policy 1, policy_version 725775 (0.0008) [2023-12-27 10:39:10,357][20347] Updated weights for policy 0, policy_version 726219 (0.0007) [2023-12-27 10:39:10,415][20347] Updated weights for policy 0, policy_version 726229 (0.0007) [2023-12-27 10:39:10,465][20347] Updated weights for policy 0, policy_version 726239 (0.0006) [2023-12-27 10:39:10,788][20381] Updated weights for policy 1, policy_version 725785 (0.0008) [2023-12-27 10:39:10,856][20381] Updated weights for policy 1, policy_version 725795 (0.0008) [2023-12-27 10:39:10,924][20381] Updated weights for policy 1, policy_version 725805 (0.0007) [2023-12-27 10:39:10,991][20381] Updated weights for policy 1, policy_version 725815 (0.0007) [2023-12-27 10:39:11,187][20347] Updated weights for policy 0, policy_version 726249 (0.0007) [2023-12-27 10:39:11,255][20347] Updated weights for policy 0, policy_version 726259 (0.0009) [2023-12-27 10:39:11,313][20347] Updated weights for policy 0, policy_version 726269 (0.0008) [2023-12-27 10:39:11,381][20347] Updated weights for policy 0, policy_version 726279 (0.0010) [2023-12-27 10:39:11,680][20381] Updated weights for policy 1, policy_version 725825 (0.0008) [2023-12-27 10:39:11,742][20381] Updated weights for policy 1, policy_version 725835 (0.0007) [2023-12-27 10:39:11,805][20381] Updated weights for policy 1, policy_version 725845 (0.0006) [2023-12-27 10:39:12,118][20347] Updated weights for policy 0, policy_version 726289 (0.0008) [2023-12-27 10:39:12,176][20347] Updated weights for policy 0, policy_version 726299 (0.0006) [2023-12-27 10:39:12,242][20347] Updated weights for policy 0, policy_version 726309 (0.0006) [2023-12-27 10:39:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 371802112. Throughput: 0: 10224.0, 1: 10172.8. Samples: 371792840. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:12,363][19170] Avg episode reward: [(0, '970.870'), (1, '835.210')] [2023-12-27 10:39:12,496][20381] Updated weights for policy 1, policy_version 725855 (0.0007) [2023-12-27 10:39:12,548][20381] Updated weights for policy 1, policy_version 725865 (0.0007) [2023-12-27 10:39:12,594][20381] Updated weights for policy 1, policy_version 725875 (0.0009) [2023-12-27 10:39:12,840][20347] Updated weights for policy 0, policy_version 726319 (0.0006) [2023-12-27 10:39:12,891][20347] Updated weights for policy 0, policy_version 726329 (0.0007) [2023-12-27 10:39:12,944][20347] Updated weights for policy 0, policy_version 726339 (0.0005) [2023-12-27 10:39:13,348][20381] Updated weights for policy 1, policy_version 725885 (0.0009) [2023-12-27 10:39:13,395][20381] Updated weights for policy 1, policy_version 725895 (0.0007) [2023-12-27 10:39:13,443][20381] Updated weights for policy 1, policy_version 725905 (0.0007) [2023-12-27 10:39:13,548][20347] Updated weights for policy 0, policy_version 726349 (0.0008) [2023-12-27 10:39:13,593][20347] Updated weights for policy 0, policy_version 726359 (0.0010) [2023-12-27 10:39:13,646][20347] Updated weights for policy 0, policy_version 726369 (0.0010) [2023-12-27 10:39:14,150][20381] Updated weights for policy 1, policy_version 725915 (0.0008) [2023-12-27 10:39:14,207][20381] Updated weights for policy 1, policy_version 725925 (0.0007) [2023-12-27 10:39:14,262][20381] Updated weights for policy 1, policy_version 725935 (0.0007) [2023-12-27 10:39:14,342][20347] Updated weights for policy 0, policy_version 726379 (0.0010) [2023-12-27 10:39:14,406][20347] Updated weights for policy 0, policy_version 726389 (0.0010) [2023-12-27 10:39:14,461][20347] Updated weights for policy 0, policy_version 726399 (0.0010) [2023-12-27 10:39:14,944][20381] Updated weights for policy 1, policy_version 725945 (0.0007) [2023-12-27 10:39:15,004][20381] Updated weights for policy 1, policy_version 725955 (0.0008) [2023-12-27 10:39:15,064][20381] Updated weights for policy 1, policy_version 725965 (0.0008) [2023-12-27 10:39:15,128][20381] Updated weights for policy 1, policy_version 725975 (0.0007) [2023-12-27 10:39:15,149][20347] Updated weights for policy 0, policy_version 726409 (0.0010) [2023-12-27 10:39:15,215][20347] Updated weights for policy 0, policy_version 726419 (0.0009) [2023-12-27 10:39:15,278][20347] Updated weights for policy 0, policy_version 726429 (0.0008) [2023-12-27 10:39:15,337][20347] Updated weights for policy 0, policy_version 726439 (0.0007) [2023-12-27 10:39:15,802][20381] Updated weights for policy 1, policy_version 725985 (0.0006) [2023-12-27 10:39:15,853][20381] Updated weights for policy 1, policy_version 725995 (0.0007) [2023-12-27 10:39:15,898][20381] Updated weights for policy 1, policy_version 726005 (0.0008) [2023-12-27 10:39:16,005][20347] Updated weights for policy 0, policy_version 726449 (0.0006) [2023-12-27 10:39:16,064][20347] Updated weights for policy 0, policy_version 726459 (0.0005) [2023-12-27 10:39:16,123][20347] Updated weights for policy 0, policy_version 726469 (0.0005) [2023-12-27 10:39:16,634][20381] Updated weights for policy 1, policy_version 726015 (0.0007) [2023-12-27 10:39:16,643][20347] Updated weights for policy 0, policy_version 726479 (0.0005) [2023-12-27 10:39:16,680][20381] Updated weights for policy 1, policy_version 726025 (0.0007) [2023-12-27 10:39:16,701][20347] Updated weights for policy 0, policy_version 726489 (0.0005) [2023-12-27 10:39:16,725][20381] Updated weights for policy 1, policy_version 726035 (0.0008) [2023-12-27 10:39:16,763][20347] Updated weights for policy 0, policy_version 726499 (0.0005) [2023-12-27 10:39:17,352][20347] Updated weights for policy 0, policy_version 726509 (0.0006) [2023-12-27 10:39:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 371900416. Throughput: 0: 10295.4, 1: 10136.3. Samples: 371915512. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:17,363][19170] Avg episode reward: [(0, '970.800'), (1, '835.340')] [2023-12-27 10:39:17,401][20347] Updated weights for policy 0, policy_version 726519 (0.0008) [2023-12-27 10:39:17,461][20381] Updated weights for policy 1, policy_version 726045 (0.0008) [2023-12-27 10:39:17,462][20347] Updated weights for policy 0, policy_version 726529 (0.0006) [2023-12-27 10:39:17,507][20381] Updated weights for policy 1, policy_version 726055 (0.0006) [2023-12-27 10:39:17,558][20381] Updated weights for policy 1, policy_version 726065 (0.0007) [2023-12-27 10:39:18,142][20347] Updated weights for policy 0, policy_version 726539 (0.0007) [2023-12-27 10:39:18,204][20347] Updated weights for policy 0, policy_version 726549 (0.0006) [2023-12-27 10:39:18,255][20381] Updated weights for policy 1, policy_version 726075 (0.0007) [2023-12-27 10:39:18,259][20347] Updated weights for policy 0, policy_version 726559 (0.0005) [2023-12-27 10:39:18,317][20381] Updated weights for policy 1, policy_version 726085 (0.0007) [2023-12-27 10:39:18,382][20381] Updated weights for policy 1, policy_version 726095 (0.0006) [2023-12-27 10:39:18,979][20347] Updated weights for policy 0, policy_version 726569 (0.0006) [2023-12-27 10:39:19,024][20347] Updated weights for policy 0, policy_version 726579 (0.0007) [2023-12-27 10:39:19,037][20381] Updated weights for policy 1, policy_version 726105 (0.0006) [2023-12-27 10:39:19,084][20347] Updated weights for policy 0, policy_version 726589 (0.0007) [2023-12-27 10:39:19,091][20381] Updated weights for policy 1, policy_version 726115 (0.0005) [2023-12-27 10:39:19,140][20347] Updated weights for policy 0, policy_version 726599 (0.0007) [2023-12-27 10:39:19,148][20381] Updated weights for policy 1, policy_version 726125 (0.0005) [2023-12-27 10:39:19,204][20381] Updated weights for policy 1, policy_version 726135 (0.0005) [2023-12-27 10:39:19,885][20347] Updated weights for policy 0, policy_version 726609 (0.0006) [2023-12-27 10:39:19,922][20381] Updated weights for policy 1, policy_version 726145 (0.0008) [2023-12-27 10:39:19,938][20292] KL-divergence is very high: 134.4989 [2023-12-27 10:39:19,949][20347] Updated weights for policy 0, policy_version 726619 (0.0006) [2023-12-27 10:39:19,987][20292] KL-divergence is very high: 111.6929 [2023-12-27 10:39:19,988][20381] Updated weights for policy 1, policy_version 726155 (0.0008) [2023-12-27 10:39:20,015][20347] Updated weights for policy 0, policy_version 726629 (0.0008) [2023-12-27 10:39:20,044][20381] Updated weights for policy 1, policy_version 726165 (0.0007) [2023-12-27 10:39:20,641][20347] Updated weights for policy 0, policy_version 726639 (0.0007) [2023-12-27 10:39:20,700][20347] Updated weights for policy 0, policy_version 726649 (0.0007) [2023-12-27 10:39:20,762][20347] Updated weights for policy 0, policy_version 726659 (0.0008) [2023-12-27 10:39:20,816][20381] Updated weights for policy 1, policy_version 726175 (0.0006) [2023-12-27 10:39:20,879][20381] Updated weights for policy 1, policy_version 726185 (0.0008) [2023-12-27 10:39:20,936][20381] Updated weights for policy 1, policy_version 726195 (0.0010) [2023-12-27 10:39:21,497][20347] Updated weights for policy 0, policy_version 726669 (0.0009) [2023-12-27 10:39:21,556][20347] Updated weights for policy 0, policy_version 726679 (0.0010) [2023-12-27 10:39:21,631][20381] Updated weights for policy 1, policy_version 726205 (0.0010) [2023-12-27 10:39:21,635][20347] Updated weights for policy 0, policy_version 726689 (0.0010) [2023-12-27 10:39:21,693][20381] Updated weights for policy 1, policy_version 726215 (0.0007) [2023-12-27 10:39:21,754][20381] Updated weights for policy 1, policy_version 726225 (0.0008) [2023-12-27 10:39:22,307][20347] Updated weights for policy 0, policy_version 726699 (0.0009) [2023-12-27 10:39:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 371998720. Throughput: 0: 10252.2, 1: 10108.2. Samples: 371975112. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:22,363][19170] Avg episode reward: [(0, '942.280'), (1, '728.020')] [2023-12-27 10:39:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000726232_185942016.pth... [2023-12-27 10:39:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000725080_185647104.pth [2023-12-27 10:39:22,374][20347] Updated weights for policy 0, policy_version 726709 (0.0010) [2023-12-27 10:39:22,406][20381] Updated weights for policy 1, policy_version 726235 (0.0007) [2023-12-27 10:39:22,431][20347] Updated weights for policy 0, policy_version 726719 (0.0008) [2023-12-27 10:39:22,462][20381] Updated weights for policy 1, policy_version 726245 (0.0008) [2023-12-27 10:39:22,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000726728_186064896.pth... [2023-12-27 10:39:22,487][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000725512_185753600.pth [2023-12-27 10:39:22,513][20381] Updated weights for policy 1, policy_version 726255 (0.0008) [2023-12-27 10:39:23,055][20347] Updated weights for policy 0, policy_version 726729 (0.0007) [2023-12-27 10:39:23,117][20347] Updated weights for policy 0, policy_version 726739 (0.0010) [2023-12-27 10:39:23,179][20347] Updated weights for policy 0, policy_version 726749 (0.0011) [2023-12-27 10:39:23,237][20347] Updated weights for policy 0, policy_version 726759 (0.0010) [2023-12-27 10:39:23,242][20381] Updated weights for policy 1, policy_version 726265 (0.0008) [2023-12-27 10:39:23,300][20381] Updated weights for policy 1, policy_version 726275 (0.0007) [2023-12-27 10:39:23,364][20381] Updated weights for policy 1, policy_version 726285 (0.0007) [2023-12-27 10:39:23,427][20381] Updated weights for policy 1, policy_version 726295 (0.0007) [2023-12-27 10:39:23,897][20347] Updated weights for policy 0, policy_version 726769 (0.0009) [2023-12-27 10:39:23,945][20347] Updated weights for policy 0, policy_version 726779 (0.0010) [2023-12-27 10:39:23,997][20347] Updated weights for policy 0, policy_version 726789 (0.0008) [2023-12-27 10:39:24,041][20381] Updated weights for policy 1, policy_version 726305 (0.0006) [2023-12-27 10:39:24,100][20381] Updated weights for policy 1, policy_version 726315 (0.0010) [2023-12-27 10:39:24,162][20381] Updated weights for policy 1, policy_version 726325 (0.0011) [2023-12-27 10:39:24,626][20347] Updated weights for policy 0, policy_version 726799 (0.0009) [2023-12-27 10:39:24,685][20347] Updated weights for policy 0, policy_version 726809 (0.0010) [2023-12-27 10:39:24,743][20347] Updated weights for policy 0, policy_version 726819 (0.0010) [2023-12-27 10:39:24,774][20381] Updated weights for policy 1, policy_version 726335 (0.0007) [2023-12-27 10:39:24,832][20381] Updated weights for policy 1, policy_version 726345 (0.0005) [2023-12-27 10:39:24,880][20381] Updated weights for policy 1, policy_version 726355 (0.0005) [2023-12-27 10:39:25,394][20347] Updated weights for policy 0, policy_version 726829 (0.0010) [2023-12-27 10:39:25,449][20347] Updated weights for policy 0, policy_version 726839 (0.0010) [2023-12-27 10:39:25,483][20381] Updated weights for policy 1, policy_version 726365 (0.0006) [2023-12-27 10:39:25,501][20347] Updated weights for policy 0, policy_version 726849 (0.0010) [2023-12-27 10:39:25,540][20381] Updated weights for policy 1, policy_version 726375 (0.0005) [2023-12-27 10:39:25,597][20381] Updated weights for policy 1, policy_version 726385 (0.0007) [2023-12-27 10:39:26,160][20347] Updated weights for policy 0, policy_version 726859 (0.0009) [2023-12-27 10:39:26,210][20347] Updated weights for policy 0, policy_version 726869 (0.0007) [2023-12-27 10:39:26,262][20347] Updated weights for policy 0, policy_version 726879 (0.0010) [2023-12-27 10:39:26,265][20381] Updated weights for policy 1, policy_version 726395 (0.0009) [2023-12-27 10:39:26,313][20381] Updated weights for policy 1, policy_version 726405 (0.0007) [2023-12-27 10:39:26,371][20381] Updated weights for policy 1, policy_version 726415 (0.0007) [2023-12-27 10:39:26,896][20347] Updated weights for policy 0, policy_version 726889 (0.0010) [2023-12-27 10:39:26,899][20381] Updated weights for policy 1, policy_version 726425 (0.0005) [2023-12-27 10:39:26,950][20347] Updated weights for policy 0, policy_version 726899 (0.0010) [2023-12-27 10:39:26,954][20381] Updated weights for policy 1, policy_version 726435 (0.0005) [2023-12-27 10:39:27,010][20347] Updated weights for policy 0, policy_version 726909 (0.0010) [2023-12-27 10:39:27,012][20381] Updated weights for policy 1, policy_version 726445 (0.0005) [2023-12-27 10:39:27,065][20381] Updated weights for policy 1, policy_version 726455 (0.0005) [2023-12-27 10:39:27,067][20347] Updated weights for policy 0, policy_version 726919 (0.0010) [2023-12-27 10:39:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 372113408. Throughput: 0: 10264.5, 1: 10144.0. Samples: 372098852. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:27,363][19170] Avg episode reward: [(0, '913.190'), (1, '756.970')] [2023-12-27 10:39:27,667][20381] Updated weights for policy 1, policy_version 726465 (0.0010) [2023-12-27 10:39:27,710][20347] Updated weights for policy 0, policy_version 726929 (0.0010) [2023-12-27 10:39:27,715][20381] Updated weights for policy 1, policy_version 726475 (0.0010) [2023-12-27 10:39:27,756][20347] Updated weights for policy 0, policy_version 726939 (0.0009) [2023-12-27 10:39:27,774][20381] Updated weights for policy 1, policy_version 726485 (0.0009) [2023-12-27 10:39:27,804][20347] Updated weights for policy 0, policy_version 726949 (0.0010) [2023-12-27 10:39:28,444][20381] Updated weights for policy 1, policy_version 726495 (0.0009) [2023-12-27 10:39:28,494][20381] Updated weights for policy 1, policy_version 726505 (0.0011) [2023-12-27 10:39:28,503][20347] Updated weights for policy 0, policy_version 726959 (0.0010) [2023-12-27 10:39:28,544][20381] Updated weights for policy 1, policy_version 726515 (0.0011) [2023-12-27 10:39:28,559][20347] Updated weights for policy 0, policy_version 726969 (0.0010) [2023-12-27 10:39:28,614][20347] Updated weights for policy 0, policy_version 726979 (0.0010) [2023-12-27 10:39:29,270][20381] Updated weights for policy 1, policy_version 726525 (0.0010) [2023-12-27 10:39:29,291][20347] Updated weights for policy 0, policy_version 726989 (0.0011) [2023-12-27 10:39:29,328][20381] Updated weights for policy 1, policy_version 726535 (0.0008) [2023-12-27 10:39:29,352][20347] Updated weights for policy 0, policy_version 726999 (0.0011) [2023-12-27 10:39:29,388][20381] Updated weights for policy 1, policy_version 726545 (0.0009) [2023-12-27 10:39:29,400][20291] KL-divergence is very high: 597.2031 [2023-12-27 10:39:29,405][20347] Updated weights for policy 0, policy_version 727009 (0.0008) [2023-12-27 10:39:30,022][20291] KL-divergence is very high: 219.2300 [2023-12-27 10:39:30,037][20347] Updated weights for policy 0, policy_version 727019 (0.0008) [2023-12-27 10:39:30,039][20291] KL-divergence is very high: 303.3250 [2023-12-27 10:39:30,071][20291] KL-divergence is very high: 111.2020 [2023-12-27 10:39:30,089][20291] KL-divergence is very high: 233.4917 [2023-12-27 10:39:30,093][20381] Updated weights for policy 1, policy_version 726555 (0.0009) [2023-12-27 10:39:30,100][20347] Updated weights for policy 0, policy_version 727029 (0.0007) [2023-12-27 10:39:30,117][20291] KL-divergence is very high: 188.8351 [2023-12-27 10:39:30,134][20291] KL-divergence is very high: 228.9865 [2023-12-27 10:39:30,148][20381] Updated weights for policy 1, policy_version 726565 (0.0005) [2023-12-27 10:39:30,157][20347] Updated weights for policy 0, policy_version 727039 (0.0006) [2023-12-27 10:39:30,169][20291] KL-divergence is very high: 115.6820 [2023-12-27 10:39:30,181][20291] KL-divergence is very high: 138.8667 [2023-12-27 10:39:30,200][20381] Updated weights for policy 1, policy_version 726575 (0.0006) [2023-12-27 10:39:30,841][20381] Updated weights for policy 1, policy_version 726585 (0.0008) [2023-12-27 10:39:30,863][20347] Updated weights for policy 0, policy_version 727049 (0.0007) [2023-12-27 10:39:30,898][20381] Updated weights for policy 1, policy_version 726595 (0.0005) [2023-12-27 10:39:30,911][20347] Updated weights for policy 0, policy_version 727059 (0.0007) [2023-12-27 10:39:30,955][20381] Updated weights for policy 1, policy_version 726605 (0.0005) [2023-12-27 10:39:30,962][20347] Updated weights for policy 0, policy_version 727069 (0.0008) [2023-12-27 10:39:31,007][20347] Updated weights for policy 0, policy_version 727079 (0.0007) [2023-12-27 10:39:31,012][20381] Updated weights for policy 1, policy_version 726615 (0.0005) [2023-12-27 10:39:31,710][20347] Updated weights for policy 0, policy_version 727089 (0.0008) [2023-12-27 10:39:31,749][20381] Updated weights for policy 1, policy_version 726625 (0.0007) [2023-12-27 10:39:31,773][20347] Updated weights for policy 0, policy_version 727099 (0.0008) [2023-12-27 10:39:31,805][20381] Updated weights for policy 1, policy_version 726635 (0.0007) [2023-12-27 10:39:31,821][20347] Updated weights for policy 0, policy_version 727109 (0.0007) [2023-12-27 10:39:31,862][20381] Updated weights for policy 1, policy_version 726645 (0.0008) [2023-12-27 10:39:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 372211712. Throughput: 0: 10259.0, 1: 10175.8. Samples: 372223944. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:32,363][19170] Avg episode reward: [(0, '902.980'), (1, '816.020')] [2023-12-27 10:39:32,465][20347] Updated weights for policy 0, policy_version 727119 (0.0007) [2023-12-27 10:39:32,517][20347] Updated weights for policy 0, policy_version 727129 (0.0008) [2023-12-27 10:39:32,576][20347] Updated weights for policy 0, policy_version 727139 (0.0007) [2023-12-27 10:39:32,590][20381] Updated weights for policy 1, policy_version 726655 (0.0008) [2023-12-27 10:39:32,637][20381] Updated weights for policy 1, policy_version 726665 (0.0008) [2023-12-27 10:39:32,690][20381] Updated weights for policy 1, policy_version 726675 (0.0008) [2023-12-27 10:39:33,252][20347] Updated weights for policy 0, policy_version 727149 (0.0009) [2023-12-27 10:39:33,306][20347] Updated weights for policy 0, policy_version 727159 (0.0010) [2023-12-27 10:39:33,365][20347] Updated weights for policy 0, policy_version 727169 (0.0007) [2023-12-27 10:39:33,376][20381] Updated weights for policy 1, policy_version 726685 (0.0006) [2023-12-27 10:39:33,419][20381] Updated weights for policy 1, policy_version 726695 (0.0005) [2023-12-27 10:39:33,469][20381] Updated weights for policy 1, policy_version 726706 (0.0007) [2023-12-27 10:39:33,934][20347] Updated weights for policy 0, policy_version 727179 (0.0007) [2023-12-27 10:39:33,987][20347] Updated weights for policy 0, policy_version 727189 (0.0005) [2023-12-27 10:39:34,046][20347] Updated weights for policy 0, policy_version 727199 (0.0006) [2023-12-27 10:39:34,055][20381] Updated weights for policy 1, policy_version 726716 (0.0008) [2023-12-27 10:39:34,107][20381] Updated weights for policy 1, policy_version 726726 (0.0007) [2023-12-27 10:39:34,158][20381] Updated weights for policy 1, policy_version 726736 (0.0008) [2023-12-27 10:39:34,664][20347] Updated weights for policy 0, policy_version 727209 (0.0006) [2023-12-27 10:39:34,724][20347] Updated weights for policy 0, policy_version 727219 (0.0006) [2023-12-27 10:39:34,782][20347] Updated weights for policy 0, policy_version 727229 (0.0008) [2023-12-27 10:39:34,844][20347] Updated weights for policy 0, policy_version 727239 (0.0008) [2023-12-27 10:39:34,880][20381] Updated weights for policy 1, policy_version 726746 (0.0007) [2023-12-27 10:39:34,933][20381] Updated weights for policy 1, policy_version 726756 (0.0007) [2023-12-27 10:39:34,990][20381] Updated weights for policy 1, policy_version 726766 (0.0008) [2023-12-27 10:39:35,045][20381] Updated weights for policy 1, policy_version 726776 (0.0008) [2023-12-27 10:39:35,521][20347] Updated weights for policy 0, policy_version 727249 (0.0010) [2023-12-27 10:39:35,570][20347] Updated weights for policy 0, policy_version 727259 (0.0010) [2023-12-27 10:39:35,625][20347] Updated weights for policy 0, policy_version 727269 (0.0010) [2023-12-27 10:39:35,709][20381] Updated weights for policy 1, policy_version 726786 (0.0005) [2023-12-27 10:39:35,764][20381] Updated weights for policy 1, policy_version 726796 (0.0007) [2023-12-27 10:39:35,821][20381] Updated weights for policy 1, policy_version 726806 (0.0008) [2023-12-27 10:39:36,324][20347] Updated weights for policy 0, policy_version 727279 (0.0009) [2023-12-27 10:39:36,392][20347] Updated weights for policy 0, policy_version 727289 (0.0009) [2023-12-27 10:39:36,456][20347] Updated weights for policy 0, policy_version 727299 (0.0007) [2023-12-27 10:39:36,560][20381] Updated weights for policy 1, policy_version 726816 (0.0007) [2023-12-27 10:39:36,617][20381] Updated weights for policy 1, policy_version 726826 (0.0008) [2023-12-27 10:39:36,662][20381] Updated weights for policy 1, policy_version 726836 (0.0007) [2023-12-27 10:39:37,122][20347] Updated weights for policy 0, policy_version 727309 (0.0006) [2023-12-27 10:39:37,186][20347] Updated weights for policy 0, policy_version 727319 (0.0010) [2023-12-27 10:39:37,245][20347] Updated weights for policy 0, policy_version 727329 (0.0010) [2023-12-27 10:39:37,270][20381] Updated weights for policy 1, policy_version 726846 (0.0008) [2023-12-27 10:39:37,319][20381] Updated weights for policy 1, policy_version 726856 (0.0008) [2023-12-27 10:39:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 372318208. Throughput: 0: 10344.8, 1: 10183.7. Samples: 372287372. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:37,363][19170] Avg episode reward: [(0, '932.220'), (1, '708.480')] [2023-12-27 10:39:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000727336_186220544.pth... [2023-12-27 10:39:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000726120_185909248.pth [2023-12-27 10:39:37,379][20381] Updated weights for policy 1, policy_version 726866 (0.0008) [2023-12-27 10:39:37,411][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000726872_186105856.pth... [2023-12-27 10:39:37,415][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000725656_185794560.pth [2023-12-27 10:39:37,913][20347] Updated weights for policy 0, policy_version 727339 (0.0010) [2023-12-27 10:39:37,968][20347] Updated weights for policy 0, policy_version 727349 (0.0010) [2023-12-27 10:39:38,005][20381] Updated weights for policy 1, policy_version 726876 (0.0007) [2023-12-27 10:39:38,031][20347] Updated weights for policy 0, policy_version 727359 (0.0010) [2023-12-27 10:39:38,059][20381] Updated weights for policy 1, policy_version 726886 (0.0006) [2023-12-27 10:39:38,115][20381] Updated weights for policy 1, policy_version 726896 (0.0006) [2023-12-27 10:39:38,700][20347] Updated weights for policy 0, policy_version 727369 (0.0010) [2023-12-27 10:39:38,759][20347] Updated weights for policy 0, policy_version 727379 (0.0007) [2023-12-27 10:39:38,802][20381] Updated weights for policy 1, policy_version 726906 (0.0008) [2023-12-27 10:39:38,816][20347] Updated weights for policy 0, policy_version 727389 (0.0007) [2023-12-27 10:39:38,855][20381] Updated weights for policy 1, policy_version 726916 (0.0006) [2023-12-27 10:39:38,875][20347] Updated weights for policy 0, policy_version 727399 (0.0007) [2023-12-27 10:39:38,916][20381] Updated weights for policy 1, policy_version 726926 (0.0008) [2023-12-27 10:39:38,975][20381] Updated weights for policy 1, policy_version 726936 (0.0008) [2023-12-27 10:39:39,549][20347] Updated weights for policy 0, policy_version 727409 (0.0008) [2023-12-27 10:39:39,613][20347] Updated weights for policy 0, policy_version 727419 (0.0007) [2023-12-27 10:39:39,672][20347] Updated weights for policy 0, policy_version 727429 (0.0007) [2023-12-27 10:39:39,701][20381] Updated weights for policy 1, policy_version 726946 (0.0006) [2023-12-27 10:39:39,762][20381] Updated weights for policy 1, policy_version 726956 (0.0008) [2023-12-27 10:39:39,827][20381] Updated weights for policy 1, policy_version 726966 (0.0008) [2023-12-27 10:39:40,431][20347] Updated weights for policy 0, policy_version 727439 (0.0007) [2023-12-27 10:39:40,486][20347] Updated weights for policy 0, policy_version 727449 (0.0008) [2023-12-27 10:39:40,504][20381] Updated weights for policy 1, policy_version 726976 (0.0006) [2023-12-27 10:39:40,543][20347] Updated weights for policy 0, policy_version 727459 (0.0008) [2023-12-27 10:39:40,566][20381] Updated weights for policy 1, policy_version 726986 (0.0006) [2023-12-27 10:39:40,628][20381] Updated weights for policy 1, policy_version 726996 (0.0007) [2023-12-27 10:39:41,253][20347] Updated weights for policy 0, policy_version 727469 (0.0007) [2023-12-27 10:39:41,317][20347] Updated weights for policy 0, policy_version 727479 (0.0008) [2023-12-27 10:39:41,377][20381] Updated weights for policy 1, policy_version 727006 (0.0007) [2023-12-27 10:39:41,387][20347] Updated weights for policy 0, policy_version 727489 (0.0008) [2023-12-27 10:39:41,437][20381] Updated weights for policy 1, policy_version 727016 (0.0006) [2023-12-27 10:39:41,498][20381] Updated weights for policy 1, policy_version 727026 (0.0007) [2023-12-27 10:39:42,114][20381] Updated weights for policy 1, policy_version 727036 (0.0008) [2023-12-27 10:39:42,142][20347] Updated weights for policy 0, policy_version 727499 (0.0008) [2023-12-27 10:39:42,165][20381] Updated weights for policy 1, policy_version 727046 (0.0006) [2023-12-27 10:39:42,206][20347] Updated weights for policy 0, policy_version 727509 (0.0008) [2023-12-27 10:39:42,220][20381] Updated weights for policy 1, policy_version 727056 (0.0005) [2023-12-27 10:39:42,271][20347] Updated weights for policy 0, policy_version 727519 (0.0007) [2023-12-27 10:39:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 372424704. Throughput: 0: 10317.0, 1: 10183.2. Samples: 372408764. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:42,363][19170] Avg episode reward: [(0, '961.000'), (1, '738.420')] [2023-12-27 10:39:42,862][20381] Updated weights for policy 1, policy_version 727066 (0.0006) [2023-12-27 10:39:42,927][20381] Updated weights for policy 1, policy_version 727076 (0.0006) [2023-12-27 10:39:42,932][20347] Updated weights for policy 0, policy_version 727529 (0.0008) [2023-12-27 10:39:42,978][20381] Updated weights for policy 1, policy_version 727086 (0.0007) [2023-12-27 10:39:42,988][20347] Updated weights for policy 0, policy_version 727539 (0.0005) [2023-12-27 10:39:43,039][20381] Updated weights for policy 1, policy_version 727096 (0.0006) [2023-12-27 10:39:43,045][20347] Updated weights for policy 0, policy_version 727549 (0.0005) [2023-12-27 10:39:43,095][20347] Updated weights for policy 0, policy_version 727559 (0.0007) [2023-12-27 10:39:43,690][20381] Updated weights for policy 1, policy_version 727106 (0.0008) [2023-12-27 10:39:43,758][20381] Updated weights for policy 1, policy_version 727116 (0.0008) [2023-12-27 10:39:43,810][20347] Updated weights for policy 0, policy_version 727569 (0.0007) [2023-12-27 10:39:43,825][20381] Updated weights for policy 1, policy_version 727126 (0.0007) [2023-12-27 10:39:43,864][20347] Updated weights for policy 0, policy_version 727579 (0.0010) [2023-12-27 10:39:43,919][20347] Updated weights for policy 0, policy_version 727589 (0.0010) [2023-12-27 10:39:44,490][20381] Updated weights for policy 1, policy_version 727136 (0.0009) [2023-12-27 10:39:44,542][20381] Updated weights for policy 1, policy_version 727146 (0.0005) [2023-12-27 10:39:44,567][20347] Updated weights for policy 0, policy_version 727599 (0.0010) [2023-12-27 10:39:44,604][20381] Updated weights for policy 1, policy_version 727156 (0.0007) [2023-12-27 10:39:44,626][20347] Updated weights for policy 0, policy_version 727609 (0.0009) [2023-12-27 10:39:44,687][20347] Updated weights for policy 0, policy_version 727619 (0.0011) [2023-12-27 10:39:45,426][20381] Updated weights for policy 1, policy_version 727166 (0.0009) [2023-12-27 10:39:45,455][20347] Updated weights for policy 0, policy_version 727629 (0.0011) [2023-12-27 10:39:45,483][20381] Updated weights for policy 1, policy_version 727176 (0.0010) [2023-12-27 10:39:45,508][20347] Updated weights for policy 0, policy_version 727639 (0.0010) [2023-12-27 10:39:45,542][20381] Updated weights for policy 1, policy_version 727187 (0.0009) [2023-12-27 10:39:45,573][20347] Updated weights for policy 0, policy_version 727649 (0.0010) [2023-12-27 10:39:46,229][20347] Updated weights for policy 0, policy_version 727659 (0.0011) [2023-12-27 10:39:46,251][20381] Updated weights for policy 1, policy_version 727197 (0.0010) [2023-12-27 10:39:46,284][20347] Updated weights for policy 0, policy_version 727669 (0.0008) [2023-12-27 10:39:46,306][20381] Updated weights for policy 1, policy_version 727207 (0.0009) [2023-12-27 10:39:46,345][20347] Updated weights for policy 0, policy_version 727679 (0.0005) [2023-12-27 10:39:46,360][20381] Updated weights for policy 1, policy_version 727217 (0.0011) [2023-12-27 10:39:47,009][20347] Updated weights for policy 0, policy_version 727689 (0.0006) [2023-12-27 10:39:47,037][20381] Updated weights for policy 1, policy_version 727227 (0.0009) [2023-12-27 10:39:47,068][20347] Updated weights for policy 0, policy_version 727699 (0.0005) [2023-12-27 10:39:47,083][20381] Updated weights for policy 1, policy_version 727237 (0.0005) [2023-12-27 10:39:47,123][20347] Updated weights for policy 0, policy_version 727709 (0.0006) [2023-12-27 10:39:47,131][20381] Updated weights for policy 1, policy_version 727247 (0.0006) [2023-12-27 10:39:47,179][20347] Updated weights for policy 0, policy_version 727719 (0.0005) [2023-12-27 10:39:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 372523008. Throughput: 0: 10285.9, 1: 10164.1. Samples: 372528424. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:47,363][19170] Avg episode reward: [(0, '873.570'), (1, '786.260')] [2023-12-27 10:39:47,802][20381] Updated weights for policy 1, policy_version 727257 (0.0010) [2023-12-27 10:39:47,845][20347] Updated weights for policy 0, policy_version 727729 (0.0005) [2023-12-27 10:39:47,846][20381] Updated weights for policy 1, policy_version 727267 (0.0006) [2023-12-27 10:39:47,903][20381] Updated weights for policy 1, policy_version 727277 (0.0005) [2023-12-27 10:39:47,907][20347] Updated weights for policy 0, policy_version 727739 (0.0005) [2023-12-27 10:39:47,960][20381] Updated weights for policy 1, policy_version 727287 (0.0005) [2023-12-27 10:39:47,961][20347] Updated weights for policy 0, policy_version 727749 (0.0006) [2023-12-27 10:39:48,540][20347] Updated weights for policy 0, policy_version 727759 (0.0007) [2023-12-27 10:39:48,602][20347] Updated weights for policy 0, policy_version 727769 (0.0008) [2023-12-27 10:39:48,662][20347] Updated weights for policy 0, policy_version 727779 (0.0008) [2023-12-27 10:39:48,680][20381] Updated weights for policy 1, policy_version 727297 (0.0010) [2023-12-27 10:39:48,741][20381] Updated weights for policy 1, policy_version 727307 (0.0011) [2023-12-27 10:39:48,801][20381] Updated weights for policy 1, policy_version 727317 (0.0011) [2023-12-27 10:39:49,365][20347] Updated weights for policy 0, policy_version 727789 (0.0007) [2023-12-27 10:39:49,431][20347] Updated weights for policy 0, policy_version 727799 (0.0008) [2023-12-27 10:39:49,492][20347] Updated weights for policy 0, policy_version 727809 (0.0008) [2023-12-27 10:39:49,507][20381] Updated weights for policy 1, policy_version 727327 (0.0011) [2023-12-27 10:39:49,570][20381] Updated weights for policy 1, policy_version 727337 (0.0011) [2023-12-27 10:39:49,637][20381] Updated weights for policy 1, policy_version 727347 (0.0011) [2023-12-27 10:39:50,226][20347] Updated weights for policy 0, policy_version 727819 (0.0008) [2023-12-27 10:39:50,289][20347] Updated weights for policy 0, policy_version 727829 (0.0007) [2023-12-27 10:39:50,330][20381] Updated weights for policy 1, policy_version 727357 (0.0008) [2023-12-27 10:39:50,349][20347] Updated weights for policy 0, policy_version 727839 (0.0008) [2023-12-27 10:39:50,397][20381] Updated weights for policy 1, policy_version 727367 (0.0007) [2023-12-27 10:39:50,458][20381] Updated weights for policy 1, policy_version 727377 (0.0010) [2023-12-27 10:39:51,063][20347] Updated weights for policy 0, policy_version 727849 (0.0008) [2023-12-27 10:39:51,120][20347] Updated weights for policy 0, policy_version 727859 (0.0010) [2023-12-27 10:39:51,180][20347] Updated weights for policy 0, policy_version 727869 (0.0010) [2023-12-27 10:39:51,180][20381] Updated weights for policy 1, policy_version 727387 (0.0009) [2023-12-27 10:39:51,233][20347] Updated weights for policy 0, policy_version 727879 (0.0010) [2023-12-27 10:39:51,237][20381] Updated weights for policy 1, policy_version 727397 (0.0006) [2023-12-27 10:39:51,301][20381] Updated weights for policy 1, policy_version 727407 (0.0007) [2023-12-27 10:39:51,960][20381] Updated weights for policy 1, policy_version 727417 (0.0007) [2023-12-27 10:39:52,020][20381] Updated weights for policy 1, policy_version 727427 (0.0005) [2023-12-27 10:39:52,049][20347] Updated weights for policy 0, policy_version 727889 (0.0007) [2023-12-27 10:39:52,078][20381] Updated weights for policy 1, policy_version 727437 (0.0006) [2023-12-27 10:39:52,106][20347] Updated weights for policy 0, policy_version 727899 (0.0008) [2023-12-27 10:39:52,138][20381] Updated weights for policy 1, policy_version 727447 (0.0005) [2023-12-27 10:39:52,160][20347] Updated weights for policy 0, policy_version 727909 (0.0007) [2023-12-27 10:39:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 372621312. Throughput: 0: 10301.8, 1: 10157.0. Samples: 372589360. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:52,363][19170] Avg episode reward: [(0, '844.820'), (1, '805.350')] [2023-12-27 10:39:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000727912_186368000.pth... [2023-12-27 10:39:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000727448_186253312.pth... [2023-12-27 10:39:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000726728_186064896.pth [2023-12-27 10:39:52,372][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000727912_186368000.pth [2023-12-27 10:39:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000726232_185942016.pth [2023-12-27 10:39:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000727448_186253312.pth [2023-12-27 10:39:52,741][20381] Updated weights for policy 1, policy_version 727457 (0.0008) [2023-12-27 10:39:52,805][20381] Updated weights for policy 1, policy_version 727467 (0.0009) [2023-12-27 10:39:52,837][20347] Updated weights for policy 0, policy_version 727919 (0.0007) [2023-12-27 10:39:52,862][20381] Updated weights for policy 1, policy_version 727477 (0.0006) [2023-12-27 10:39:52,894][20347] Updated weights for policy 0, policy_version 727929 (0.0006) [2023-12-27 10:39:52,953][20347] Updated weights for policy 0, policy_version 727939 (0.0005) [2023-12-27 10:39:53,512][20347] Updated weights for policy 0, policy_version 727949 (0.0005) [2023-12-27 10:39:53,558][20347] Updated weights for policy 0, policy_version 727959 (0.0005) [2023-12-27 10:39:53,581][20381] Updated weights for policy 1, policy_version 727487 (0.0007) [2023-12-27 10:39:53,607][20347] Updated weights for policy 0, policy_version 727969 (0.0006) [2023-12-27 10:39:53,638][20381] Updated weights for policy 1, policy_version 727497 (0.0007) [2023-12-27 10:39:53,694][20381] Updated weights for policy 1, policy_version 727507 (0.0008) [2023-12-27 10:39:54,194][20347] Updated weights for policy 0, policy_version 727979 (0.0005) [2023-12-27 10:39:54,256][20347] Updated weights for policy 0, policy_version 727989 (0.0006) [2023-12-27 10:39:54,320][20347] Updated weights for policy 0, policy_version 727999 (0.0006) [2023-12-27 10:39:54,467][20381] Updated weights for policy 1, policy_version 727517 (0.0008) [2023-12-27 10:39:54,518][20381] Updated weights for policy 1, policy_version 727527 (0.0009) [2023-12-27 10:39:54,570][20381] Updated weights for policy 1, policy_version 727537 (0.0010) [2023-12-27 10:39:54,905][20347] Updated weights for policy 0, policy_version 728009 (0.0006) [2023-12-27 10:39:54,963][20347] Updated weights for policy 0, policy_version 728019 (0.0008) [2023-12-27 10:39:55,021][20347] Updated weights for policy 0, policy_version 728029 (0.0007) [2023-12-27 10:39:55,082][20347] Updated weights for policy 0, policy_version 728039 (0.0008) [2023-12-27 10:39:55,287][20381] Updated weights for policy 1, policy_version 727547 (0.0010) [2023-12-27 10:39:55,344][20381] Updated weights for policy 1, policy_version 727557 (0.0010) [2023-12-27 10:39:55,393][20381] Updated weights for policy 1, policy_version 727567 (0.0010) [2023-12-27 10:39:55,768][20347] Updated weights for policy 0, policy_version 728049 (0.0008) [2023-12-27 10:39:55,818][20347] Updated weights for policy 0, policy_version 728059 (0.0007) [2023-12-27 10:39:55,871][20347] Updated weights for policy 0, policy_version 728069 (0.0007) [2023-12-27 10:39:56,091][20381] Updated weights for policy 1, policy_version 727577 (0.0011) [2023-12-27 10:39:56,147][20381] Updated weights for policy 1, policy_version 727587 (0.0011) [2023-12-27 10:39:56,205][20381] Updated weights for policy 1, policy_version 727597 (0.0011) [2023-12-27 10:39:56,258][20381] Updated weights for policy 1, policy_version 727607 (0.0011) [2023-12-27 10:39:56,555][20347] Updated weights for policy 0, policy_version 728079 (0.0006) [2023-12-27 10:39:56,603][20347] Updated weights for policy 0, policy_version 728089 (0.0005) [2023-12-27 10:39:56,654][20347] Updated weights for policy 0, policy_version 728099 (0.0005) [2023-12-27 10:39:56,815][20381] Updated weights for policy 1, policy_version 727617 (0.0006) [2023-12-27 10:39:56,865][20381] Updated weights for policy 1, policy_version 727627 (0.0010) [2023-12-27 10:39:56,913][20381] Updated weights for policy 1, policy_version 727637 (0.0010) [2023-12-27 10:39:57,230][20347] Updated weights for policy 0, policy_version 728109 (0.0006) [2023-12-27 10:39:57,247][20291] KL-divergence is very high: 368.5839 [2023-12-27 10:39:57,286][20347] Updated weights for policy 0, policy_version 728119 (0.0007) [2023-12-27 10:39:57,344][20347] Updated weights for policy 0, policy_version 728129 (0.0007) [2023-12-27 10:39:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 372719616. Throughput: 0: 10272.2, 1: 10133.2. Samples: 372711080. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:39:57,363][19170] Avg episode reward: [(0, '750.600'), (1, '786.220')] [2023-12-27 10:39:57,586][20381] Updated weights for policy 1, policy_version 727647 (0.0010) [2023-12-27 10:39:57,632][20381] Updated weights for policy 1, policy_version 727657 (0.0008) [2023-12-27 10:39:57,682][20381] Updated weights for policy 1, policy_version 727667 (0.0005) [2023-12-27 10:39:58,019][20347] Updated weights for policy 0, policy_version 728139 (0.0006) [2023-12-27 10:39:58,075][20347] Updated weights for policy 0, policy_version 728149 (0.0009) [2023-12-27 10:39:58,132][20347] Updated weights for policy 0, policy_version 728159 (0.0008) [2023-12-27 10:39:58,253][20381] Updated weights for policy 1, policy_version 727677 (0.0008) [2023-12-27 10:39:58,314][20381] Updated weights for policy 1, policy_version 727687 (0.0010) [2023-12-27 10:39:58,382][20381] Updated weights for policy 1, policy_version 727697 (0.0008) [2023-12-27 10:39:58,908][20347] Updated weights for policy 0, policy_version 728169 (0.0008) [2023-12-27 10:39:58,957][20347] Updated weights for policy 0, policy_version 728179 (0.0008) [2023-12-27 10:39:59,017][20347] Updated weights for policy 0, policy_version 728189 (0.0008) [2023-12-27 10:39:59,054][20381] Updated weights for policy 1, policy_version 727707 (0.0008) [2023-12-27 10:39:59,068][20347] Updated weights for policy 0, policy_version 728199 (0.0007) [2023-12-27 10:39:59,114][20381] Updated weights for policy 1, policy_version 727717 (0.0007) [2023-12-27 10:39:59,172][20381] Updated weights for policy 1, policy_version 727727 (0.0007) [2023-12-27 10:39:59,812][20347] Updated weights for policy 0, policy_version 728209 (0.0007) [2023-12-27 10:39:59,875][20381] Updated weights for policy 1, policy_version 727737 (0.0006) [2023-12-27 10:39:59,877][20347] Updated weights for policy 0, policy_version 728219 (0.0008) [2023-12-27 10:39:59,940][20347] Updated weights for policy 0, policy_version 728229 (0.0007) [2023-12-27 10:39:59,943][20381] Updated weights for policy 1, policy_version 727747 (0.0007) [2023-12-27 10:40:00,000][20381] Updated weights for policy 1, policy_version 727757 (0.0008) [2023-12-27 10:40:00,054][20381] Updated weights for policy 1, policy_version 727767 (0.0008) [2023-12-27 10:40:00,536][20347] Updated weights for policy 0, policy_version 728239 (0.0008) [2023-12-27 10:40:00,576][20291] KL-divergence is very high: 107.9217 [2023-12-27 10:40:00,581][20347] Updated weights for policy 0, policy_version 728249 (0.0007) [2023-12-27 10:40:00,581][20291] KL-divergence is very high: 135.6071 [2023-12-27 10:40:00,586][20291] KL-divergence is very high: 162.0288 [2023-12-27 10:40:00,592][20291] KL-divergence is very high: 174.8873 [2023-12-27 10:40:00,635][20347] Updated weights for policy 0, policy_version 728259 (0.0007) [2023-12-27 10:40:00,662][20291] KL-divergence is very high: 133.2866 [2023-12-27 10:40:00,787][20381] Updated weights for policy 1, policy_version 727777 (0.0008) [2023-12-27 10:40:00,841][20381] Updated weights for policy 1, policy_version 727787 (0.0008) [2023-12-27 10:40:00,888][20381] Updated weights for policy 1, policy_version 727797 (0.0007) [2023-12-27 10:40:01,311][20291] KL-divergence is very high: 105.8910 [2023-12-27 10:40:01,337][20347] Updated weights for policy 0, policy_version 728269 (0.0006) [2023-12-27 10:40:01,399][20347] Updated weights for policy 0, policy_version 728279 (0.0007) [2023-12-27 10:40:01,455][20347] Updated weights for policy 0, policy_version 728289 (0.0007) [2023-12-27 10:40:01,634][20381] Updated weights for policy 1, policy_version 727807 (0.0008) [2023-12-27 10:40:01,692][20381] Updated weights for policy 1, policy_version 727817 (0.0008) [2023-12-27 10:40:01,758][20381] Updated weights for policy 1, policy_version 727827 (0.0008) [2023-12-27 10:40:02,203][20347] Updated weights for policy 0, policy_version 728299 (0.0006) [2023-12-27 10:40:02,231][20291] KL-divergence is very high: 101.9355 [2023-12-27 10:40:02,254][20291] KL-divergence is very high: 123.1289 [2023-12-27 10:40:02,259][20347] Updated weights for policy 0, policy_version 728309 (0.0006) [2023-12-27 10:40:02,277][20291] KL-divergence is very high: 142.8176 [2023-12-27 10:40:02,288][20291] KL-divergence is very high: 100.2613 [2023-12-27 10:40:02,316][20347] Updated weights for policy 0, policy_version 728319 (0.0006) [2023-12-27 10:40:02,323][20291] KL-divergence is very high: 141.7951 [2023-12-27 10:40:02,348][20291] KL-divergence is very high: 121.4054 [2023-12-27 10:40:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 372817920. Throughput: 0: 10210.0, 1: 10187.3. Samples: 372833392. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:40:02,363][19170] Avg episode reward: [(0, '697.550'), (1, '834.520')] [2023-12-27 10:40:02,504][20381] Updated weights for policy 1, policy_version 727837 (0.0008) [2023-12-27 10:40:02,559][20381] Updated weights for policy 1, policy_version 727847 (0.0007) [2023-12-27 10:40:02,621][20381] Updated weights for policy 1, policy_version 727857 (0.0008) [2023-12-27 10:40:02,932][20347] Updated weights for policy 0, policy_version 728329 (0.0007) [2023-12-27 10:40:02,986][20347] Updated weights for policy 0, policy_version 728339 (0.0005) [2023-12-27 10:40:03,047][20347] Updated weights for policy 0, policy_version 728349 (0.0005) [2023-12-27 10:40:03,111][20347] Updated weights for policy 0, policy_version 728359 (0.0005) [2023-12-27 10:40:03,254][20381] Updated weights for policy 1, policy_version 727867 (0.0008) [2023-12-27 10:40:03,298][20381] Updated weights for policy 1, policy_version 727877 (0.0008) [2023-12-27 10:40:03,343][20381] Updated weights for policy 1, policy_version 727887 (0.0008) [2023-12-27 10:40:03,623][20347] Updated weights for policy 0, policy_version 728369 (0.0005) [2023-12-27 10:40:03,671][20347] Updated weights for policy 0, policy_version 728379 (0.0005) [2023-12-27 10:40:03,716][20347] Updated weights for policy 0, policy_version 728389 (0.0005) [2023-12-27 10:40:03,969][20381] Updated weights for policy 1, policy_version 727897 (0.0008) [2023-12-27 10:40:04,032][20381] Updated weights for policy 1, policy_version 727907 (0.0008) [2023-12-27 10:40:04,088][20381] Updated weights for policy 1, policy_version 727917 (0.0008) [2023-12-27 10:40:04,153][20381] Updated weights for policy 1, policy_version 727927 (0.0008) [2023-12-27 10:40:04,372][20347] Updated weights for policy 0, policy_version 728399 (0.0007) [2023-12-27 10:40:04,434][20291] KL-divergence is very high: 137.6760 [2023-12-27 10:40:04,441][20347] Updated weights for policy 0, policy_version 728409 (0.0007) [2023-12-27 10:40:04,500][20347] Updated weights for policy 0, policy_version 728419 (0.0005) [2023-12-27 10:40:04,806][20381] Updated weights for policy 1, policy_version 727937 (0.0006) [2023-12-27 10:40:04,859][20381] Updated weights for policy 1, policy_version 727947 (0.0006) [2023-12-27 10:40:04,910][20381] Updated weights for policy 1, policy_version 727957 (0.0008) [2023-12-27 10:40:05,127][20291] KL-divergence is very high: 174.2135 [2023-12-27 10:40:05,144][20347] Updated weights for policy 0, policy_version 728429 (0.0006) [2023-12-27 10:40:05,174][20291] KL-divergence is very high: 136.4519 [2023-12-27 10:40:05,202][20347] Updated weights for policy 0, policy_version 728439 (0.0008) [2023-12-27 10:40:05,221][20291] KL-divergence is very high: 165.1847 [2023-12-27 10:40:05,265][20347] Updated weights for policy 0, policy_version 728449 (0.0007) [2023-12-27 10:40:05,628][20381] Updated weights for policy 1, policy_version 727967 (0.0008) [2023-12-27 10:40:05,684][20381] Updated weights for policy 1, policy_version 727977 (0.0008) [2023-12-27 10:40:05,739][20381] Updated weights for policy 1, policy_version 727987 (0.0008) [2023-12-27 10:40:05,950][20347] Updated weights for policy 0, policy_version 728459 (0.0008) [2023-12-27 10:40:06,009][20347] Updated weights for policy 0, policy_version 728469 (0.0007) [2023-12-27 10:40:06,067][20347] Updated weights for policy 0, policy_version 728479 (0.0007) [2023-12-27 10:40:06,458][20381] Updated weights for policy 1, policy_version 727997 (0.0006) [2023-12-27 10:40:06,518][20381] Updated weights for policy 1, policy_version 728007 (0.0007) [2023-12-27 10:40:06,574][20381] Updated weights for policy 1, policy_version 728017 (0.0007) [2023-12-27 10:40:06,803][20347] Updated weights for policy 0, policy_version 728489 (0.0007) [2023-12-27 10:40:06,860][20347] Updated weights for policy 0, policy_version 728499 (0.0008) [2023-12-27 10:40:06,924][20347] Updated weights for policy 0, policy_version 728509 (0.0008) [2023-12-27 10:40:06,986][20347] Updated weights for policy 0, policy_version 728519 (0.0007) [2023-12-27 10:40:07,238][20381] Updated weights for policy 1, policy_version 728027 (0.0006) [2023-12-27 10:40:07,288][20381] Updated weights for policy 1, policy_version 728037 (0.0008) [2023-12-27 10:40:07,343][20381] Updated weights for policy 1, policy_version 728047 (0.0008) [2023-12-27 10:40:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.6, 300 sec: 20466.1). Total num frames: 372924416. Throughput: 0: 10302.1, 1: 10226.3. Samples: 372898888. Policy #0 lag: (min: 25.0, avg: 51.3, max: 57.0) [2023-12-27 10:40:07,363][19170] Avg episode reward: [(0, '378.450'), (1, '883.180')] [2023-12-27 10:40:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000728520_186523648.pth... [2023-12-27 10:40:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000727336_186220544.pth [2023-12-27 10:40:07,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000728056_186408960.pth... [2023-12-27 10:40:07,387][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000726872_186105856.pth [2023-12-27 10:40:07,637][20347] Updated weights for policy 0, policy_version 728529 (0.0007) [2023-12-27 10:40:07,682][20347] Updated weights for policy 0, policy_version 728539 (0.0007) [2023-12-27 10:40:07,733][20347] Updated weights for policy 0, policy_version 728549 (0.0007) [2023-12-27 10:40:08,061][20381] Updated weights for policy 1, policy_version 728057 (0.0008) [2023-12-27 10:40:08,109][20381] Updated weights for policy 1, policy_version 728067 (0.0009) [2023-12-27 10:40:08,164][20381] Updated weights for policy 1, policy_version 728077 (0.0010) [2023-12-27 10:40:08,214][20381] Updated weights for policy 1, policy_version 728087 (0.0010) [2023-12-27 10:40:08,437][20347] Updated weights for policy 0, policy_version 728559 (0.0007) [2023-12-27 10:40:08,496][20347] Updated weights for policy 0, policy_version 728569 (0.0005) [2023-12-27 10:40:08,560][20347] Updated weights for policy 0, policy_version 728579 (0.0006) [2023-12-27 10:40:08,913][20381] Updated weights for policy 1, policy_version 728097 (0.0011) [2023-12-27 10:40:08,969][20381] Updated weights for policy 1, policy_version 728107 (0.0010) [2023-12-27 10:40:09,027][20381] Updated weights for policy 1, policy_version 728117 (0.0011) [2023-12-27 10:40:09,229][20347] Updated weights for policy 0, policy_version 728589 (0.0008) [2023-12-27 10:40:09,282][20347] Updated weights for policy 0, policy_version 728599 (0.0008) [2023-12-27 10:40:09,344][20347] Updated weights for policy 0, policy_version 728609 (0.0009) [2023-12-27 10:40:09,746][20381] Updated weights for policy 1, policy_version 728127 (0.0007) [2023-12-27 10:40:09,803][20381] Updated weights for policy 1, policy_version 728137 (0.0006) [2023-12-27 10:40:09,867][20381] Updated weights for policy 1, policy_version 728147 (0.0007) [2023-12-27 10:40:10,099][20347] Updated weights for policy 0, policy_version 728619 (0.0007) [2023-12-27 10:40:10,150][20347] Updated weights for policy 0, policy_version 728629 (0.0007) [2023-12-27 10:40:10,214][20347] Updated weights for policy 0, policy_version 728639 (0.0007) [2023-12-27 10:40:10,507][20381] Updated weights for policy 1, policy_version 728157 (0.0008) [2023-12-27 10:40:10,558][20381] Updated weights for policy 1, policy_version 728167 (0.0007) [2023-12-27 10:40:10,605][20381] Updated weights for policy 1, policy_version 728177 (0.0007) [2023-12-27 10:40:10,942][20347] Updated weights for policy 0, policy_version 728649 (0.0008) [2023-12-27 10:40:10,995][20347] Updated weights for policy 0, policy_version 728659 (0.0008) [2023-12-27 10:40:11,065][20347] Updated weights for policy 0, policy_version 728669 (0.0008) [2023-12-27 10:40:11,127][20347] Updated weights for policy 0, policy_version 728679 (0.0008) [2023-12-27 10:40:11,317][20381] Updated weights for policy 1, policy_version 728187 (0.0010) [2023-12-27 10:40:11,379][20381] Updated weights for policy 1, policy_version 728197 (0.0010) [2023-12-27 10:40:11,450][20381] Updated weights for policy 1, policy_version 728207 (0.0009) [2023-12-27 10:40:11,913][20347] Updated weights for policy 0, policy_version 728689 (0.0007) [2023-12-27 10:40:11,974][20347] Updated weights for policy 0, policy_version 728699 (0.0008) [2023-12-27 10:40:12,042][20347] Updated weights for policy 0, policy_version 728709 (0.0007) [2023-12-27 10:40:12,215][20381] Updated weights for policy 1, policy_version 728217 (0.0009) [2023-12-27 10:40:12,275][20381] Updated weights for policy 1, policy_version 728227 (0.0011) [2023-12-27 10:40:12,332][20381] Updated weights for policy 1, policy_version 728237 (0.0011) [2023-12-27 10:40:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 373022720. Throughput: 0: 10241.7, 1: 10191.4. Samples: 373018340. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:12,363][19170] Avg episode reward: [(0, '570.360'), (1, '922.050')] [2023-12-27 10:40:12,397][20381] Updated weights for policy 1, policy_version 728247 (0.0011) [2023-12-27 10:40:12,722][20347] Updated weights for policy 0, policy_version 728719 (0.0008) [2023-12-27 10:40:12,779][20347] Updated weights for policy 0, policy_version 728729 (0.0008) [2023-12-27 10:40:12,834][20347] Updated weights for policy 0, policy_version 728739 (0.0007) [2023-12-27 10:40:13,121][20381] Updated weights for policy 1, policy_version 728257 (0.0006) [2023-12-27 10:40:13,184][20381] Updated weights for policy 1, policy_version 728267 (0.0009) [2023-12-27 10:40:13,243][20381] Updated weights for policy 1, policy_version 728277 (0.0010) [2023-12-27 10:40:13,513][20347] Updated weights for policy 0, policy_version 728749 (0.0009) [2023-12-27 10:40:13,567][20347] Updated weights for policy 0, policy_version 728759 (0.0010) [2023-12-27 10:40:13,616][20347] Updated weights for policy 0, policy_version 728769 (0.0010) [2023-12-27 10:40:13,800][20381] Updated weights for policy 1, policy_version 728287 (0.0007) [2023-12-27 10:40:13,848][20381] Updated weights for policy 1, policy_version 728297 (0.0005) [2023-12-27 10:40:13,900][20381] Updated weights for policy 1, policy_version 728307 (0.0007) [2023-12-27 10:40:14,275][20347] Updated weights for policy 0, policy_version 728779 (0.0008) [2023-12-27 10:40:14,334][20347] Updated weights for policy 0, policy_version 728789 (0.0007) [2023-12-27 10:40:14,387][20347] Updated weights for policy 0, policy_version 728799 (0.0008) [2023-12-27 10:40:14,592][20381] Updated weights for policy 1, policy_version 728317 (0.0008) [2023-12-27 10:40:14,639][20381] Updated weights for policy 1, policy_version 728327 (0.0010) [2023-12-27 10:40:14,698][20381] Updated weights for policy 1, policy_version 728337 (0.0008) [2023-12-27 10:40:15,007][20347] Updated weights for policy 0, policy_version 728809 (0.0008) [2023-12-27 10:40:15,068][20347] Updated weights for policy 0, policy_version 728819 (0.0006) [2023-12-27 10:40:15,128][20347] Updated weights for policy 0, policy_version 728829 (0.0006) [2023-12-27 10:40:15,192][20347] Updated weights for policy 0, policy_version 728839 (0.0008) [2023-12-27 10:40:15,469][20381] Updated weights for policy 1, policy_version 728347 (0.0007) [2023-12-27 10:40:15,533][20381] Updated weights for policy 1, policy_version 728357 (0.0005) [2023-12-27 10:40:15,594][20381] Updated weights for policy 1, policy_version 728367 (0.0008) [2023-12-27 10:40:15,816][20347] Updated weights for policy 0, policy_version 728849 (0.0006) [2023-12-27 10:40:15,862][20347] Updated weights for policy 0, policy_version 728859 (0.0005) [2023-12-27 10:40:15,907][20347] Updated weights for policy 0, policy_version 728869 (0.0007) [2023-12-27 10:40:16,239][20381] Updated weights for policy 1, policy_version 728377 (0.0008) [2023-12-27 10:40:16,301][20381] Updated weights for policy 1, policy_version 728387 (0.0007) [2023-12-27 10:40:16,359][20381] Updated weights for policy 1, policy_version 728397 (0.0008) [2023-12-27 10:40:16,429][20381] Updated weights for policy 1, policy_version 728407 (0.0007) [2023-12-27 10:40:16,561][20347] Updated weights for policy 0, policy_version 728879 (0.0010) [2023-12-27 10:40:16,617][20347] Updated weights for policy 0, policy_version 728889 (0.0010) [2023-12-27 10:40:16,676][20347] Updated weights for policy 0, policy_version 728899 (0.0010) [2023-12-27 10:40:17,133][20381] Updated weights for policy 1, policy_version 728417 (0.0008) [2023-12-27 10:40:17,188][20381] Updated weights for policy 1, policy_version 728427 (0.0007) [2023-12-27 10:40:17,245][20381] Updated weights for policy 1, policy_version 728437 (0.0008) [2023-12-27 10:40:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20467.5). Total num frames: 373129216. Throughput: 0: 10227.7, 1: 10117.1. Samples: 373139456. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:17,363][19170] Avg episode reward: [(0, '808.370'), (1, '912.130')] [2023-12-27 10:40:17,365][20347] Updated weights for policy 0, policy_version 728909 (0.0010) [2023-12-27 10:40:17,421][20347] Updated weights for policy 0, policy_version 728919 (0.0010) [2023-12-27 10:40:17,481][20347] Updated weights for policy 0, policy_version 728929 (0.0010) [2023-12-27 10:40:17,886][20381] Updated weights for policy 1, policy_version 728447 (0.0006) [2023-12-27 10:40:17,934][20381] Updated weights for policy 1, policy_version 728457 (0.0006) [2023-12-27 10:40:17,981][20381] Updated weights for policy 1, policy_version 728467 (0.0008) [2023-12-27 10:40:18,104][20347] Updated weights for policy 0, policy_version 728939 (0.0009) [2023-12-27 10:40:18,163][20347] Updated weights for policy 0, policy_version 728949 (0.0007) [2023-12-27 10:40:18,217][20347] Updated weights for policy 0, policy_version 728959 (0.0010) [2023-12-27 10:40:18,680][20381] Updated weights for policy 1, policy_version 728477 (0.0007) [2023-12-27 10:40:18,738][20381] Updated weights for policy 1, policy_version 728487 (0.0008) [2023-12-27 10:40:18,796][20381] Updated weights for policy 1, policy_version 728497 (0.0007) [2023-12-27 10:40:18,916][20347] Updated weights for policy 0, policy_version 728969 (0.0010) [2023-12-27 10:40:18,970][20347] Updated weights for policy 0, policy_version 728979 (0.0009) [2023-12-27 10:40:19,016][20347] Updated weights for policy 0, policy_version 728989 (0.0010) [2023-12-27 10:40:19,065][20347] Updated weights for policy 0, policy_version 728999 (0.0010) [2023-12-27 10:40:19,446][20381] Updated weights for policy 1, policy_version 728507 (0.0008) [2023-12-27 10:40:19,508][20381] Updated weights for policy 1, policy_version 728517 (0.0008) [2023-12-27 10:40:19,568][20381] Updated weights for policy 1, policy_version 728527 (0.0008) [2023-12-27 10:40:19,845][20347] Updated weights for policy 0, policy_version 729009 (0.0008) [2023-12-27 10:40:19,913][20347] Updated weights for policy 0, policy_version 729019 (0.0008) [2023-12-27 10:40:19,981][20347] Updated weights for policy 0, policy_version 729029 (0.0007) [2023-12-27 10:40:20,285][20381] Updated weights for policy 1, policy_version 728537 (0.0007) [2023-12-27 10:40:20,347][20381] Updated weights for policy 1, policy_version 728547 (0.0008) [2023-12-27 10:40:20,410][20381] Updated weights for policy 1, policy_version 728557 (0.0006) [2023-12-27 10:40:20,476][20381] Updated weights for policy 1, policy_version 728567 (0.0006) [2023-12-27 10:40:20,671][20347] Updated weights for policy 0, policy_version 729039 (0.0007) [2023-12-27 10:40:20,735][20347] Updated weights for policy 0, policy_version 729049 (0.0007) [2023-12-27 10:40:20,792][20347] Updated weights for policy 0, policy_version 729059 (0.0007) [2023-12-27 10:40:21,122][20381] Updated weights for policy 1, policy_version 728577 (0.0007) [2023-12-27 10:40:21,193][20381] Updated weights for policy 1, policy_version 728587 (0.0008) [2023-12-27 10:40:21,263][20381] Updated weights for policy 1, policy_version 728597 (0.0008) [2023-12-27 10:40:21,552][20347] Updated weights for policy 0, policy_version 729069 (0.0007) [2023-12-27 10:40:21,613][20347] Updated weights for policy 0, policy_version 729079 (0.0006) [2023-12-27 10:40:21,679][20347] Updated weights for policy 0, policy_version 729089 (0.0008) [2023-12-27 10:40:22,039][20381] Updated weights for policy 1, policy_version 728607 (0.0008) [2023-12-27 10:40:22,100][20381] Updated weights for policy 1, policy_version 728617 (0.0008) [2023-12-27 10:40:22,165][20381] Updated weights for policy 1, policy_version 728627 (0.0008) [2023-12-27 10:40:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 373227520. Throughput: 0: 10167.4, 1: 10113.9. Samples: 373200036. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:22,363][19170] Avg episode reward: [(0, '700.980'), (1, '834.050')] [2023-12-27 10:40:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000728632_186556416.pth... [2023-12-27 10:40:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000729096_186671104.pth... [2023-12-27 10:40:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000727448_186253312.pth [2023-12-27 10:40:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000727912_186368000.pth [2023-12-27 10:40:22,437][20347] Updated weights for policy 0, policy_version 729099 (0.0007) [2023-12-27 10:40:22,508][20347] Updated weights for policy 0, policy_version 729109 (0.0009) [2023-12-27 10:40:22,580][20347] Updated weights for policy 0, policy_version 729119 (0.0007) [2023-12-27 10:40:22,896][20381] Updated weights for policy 1, policy_version 728637 (0.0007) [2023-12-27 10:40:22,948][20381] Updated weights for policy 1, policy_version 728647 (0.0008) [2023-12-27 10:40:23,005][20381] Updated weights for policy 1, policy_version 728657 (0.0008) [2023-12-27 10:40:23,305][20347] Updated weights for policy 0, policy_version 729129 (0.0008) [2023-12-27 10:40:23,357][20347] Updated weights for policy 0, policy_version 729139 (0.0008) [2023-12-27 10:40:23,413][20347] Updated weights for policy 0, policy_version 729149 (0.0007) [2023-12-27 10:40:23,474][20347] Updated weights for policy 0, policy_version 729159 (0.0005) [2023-12-27 10:40:23,751][20381] Updated weights for policy 1, policy_version 728667 (0.0008) [2023-12-27 10:40:23,806][20381] Updated weights for policy 1, policy_version 728677 (0.0007) [2023-12-27 10:40:23,854][20381] Updated weights for policy 1, policy_version 728687 (0.0010) [2023-12-27 10:40:24,090][20347] Updated weights for policy 0, policy_version 729169 (0.0006) [2023-12-27 10:40:24,153][20347] Updated weights for policy 0, policy_version 729179 (0.0005) [2023-12-27 10:40:24,211][20347] Updated weights for policy 0, policy_version 729189 (0.0006) [2023-12-27 10:40:24,530][20381] Updated weights for policy 1, policy_version 728697 (0.0009) [2023-12-27 10:40:24,588][20381] Updated weights for policy 1, policy_version 728707 (0.0005) [2023-12-27 10:40:24,648][20381] Updated weights for policy 1, policy_version 728717 (0.0008) [2023-12-27 10:40:24,714][20381] Updated weights for policy 1, policy_version 728727 (0.0006) [2023-12-27 10:40:24,840][20347] Updated weights for policy 0, policy_version 729199 (0.0010) [2023-12-27 10:40:24,896][20347] Updated weights for policy 0, policy_version 729209 (0.0010) [2023-12-27 10:40:24,953][20347] Updated weights for policy 0, policy_version 729219 (0.0008) [2023-12-27 10:40:25,238][20381] Updated weights for policy 1, policy_version 728737 (0.0007) [2023-12-27 10:40:25,291][20381] Updated weights for policy 1, policy_version 728747 (0.0009) [2023-12-27 10:40:25,350][20381] Updated weights for policy 1, policy_version 728757 (0.0008) [2023-12-27 10:40:25,490][20347] Updated weights for policy 0, policy_version 729229 (0.0005) [2023-12-27 10:40:25,550][20347] Updated weights for policy 0, policy_version 729239 (0.0005) [2023-12-27 10:40:25,610][20347] Updated weights for policy 0, policy_version 729249 (0.0006) [2023-12-27 10:40:26,056][20381] Updated weights for policy 1, policy_version 728767 (0.0007) [2023-12-27 10:40:26,102][20381] Updated weights for policy 1, policy_version 728777 (0.0008) [2023-12-27 10:40:26,156][20381] Updated weights for policy 1, policy_version 728787 (0.0008) [2023-12-27 10:40:26,228][20347] Updated weights for policy 0, policy_version 729259 (0.0008) [2023-12-27 10:40:26,274][20347] Updated weights for policy 0, policy_version 729269 (0.0005) [2023-12-27 10:40:26,321][20347] Updated weights for policy 0, policy_version 729279 (0.0005) [2023-12-27 10:40:26,899][20347] Updated weights for policy 0, policy_version 729289 (0.0005) [2023-12-27 10:40:26,903][20381] Updated weights for policy 1, policy_version 728797 (0.0006) [2023-12-27 10:40:26,954][20347] Updated weights for policy 0, policy_version 729299 (0.0005) [2023-12-27 10:40:26,958][20381] Updated weights for policy 1, policy_version 728807 (0.0005) [2023-12-27 10:40:27,008][20347] Updated weights for policy 0, policy_version 729309 (0.0006) [2023-12-27 10:40:27,015][20381] Updated weights for policy 1, policy_version 728817 (0.0007) [2023-12-27 10:40:27,063][20347] Updated weights for policy 0, policy_version 729319 (0.0007) [2023-12-27 10:40:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 373334016. Throughput: 0: 10212.7, 1: 10067.8. Samples: 373321388. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:27,363][19170] Avg episode reward: [(0, '700.830'), (1, '756.050')] [2023-12-27 10:40:27,656][20381] Updated weights for policy 1, policy_version 728827 (0.0006) [2023-12-27 10:40:27,672][20347] Updated weights for policy 0, policy_version 729329 (0.0010) [2023-12-27 10:40:27,710][20381] Updated weights for policy 1, policy_version 728837 (0.0005) [2023-12-27 10:40:27,720][20347] Updated weights for policy 0, policy_version 729339 (0.0010) [2023-12-27 10:40:27,760][20381] Updated weights for policy 1, policy_version 728847 (0.0005) [2023-12-27 10:40:27,768][20347] Updated weights for policy 0, policy_version 729349 (0.0009) [2023-12-27 10:40:28,401][20381] Updated weights for policy 1, policy_version 728857 (0.0006) [2023-12-27 10:40:28,416][20347] Updated weights for policy 0, policy_version 729359 (0.0008) [2023-12-27 10:40:28,460][20381] Updated weights for policy 1, policy_version 728867 (0.0008) [2023-12-27 10:40:28,473][20347] Updated weights for policy 0, policy_version 729369 (0.0007) [2023-12-27 10:40:28,516][20381] Updated weights for policy 1, policy_version 728877 (0.0006) [2023-12-27 10:40:28,526][20347] Updated weights for policy 0, policy_version 729379 (0.0005) [2023-12-27 10:40:28,567][20381] Updated weights for policy 1, policy_version 728887 (0.0006) [2023-12-27 10:40:29,119][20347] Updated weights for policy 0, policy_version 729389 (0.0006) [2023-12-27 10:40:29,182][20347] Updated weights for policy 0, policy_version 729399 (0.0005) [2023-12-27 10:40:29,247][20347] Updated weights for policy 0, policy_version 729409 (0.0008) [2023-12-27 10:40:29,318][20381] Updated weights for policy 1, policy_version 728897 (0.0009) [2023-12-27 10:40:29,379][20381] Updated weights for policy 1, policy_version 728907 (0.0008) [2023-12-27 10:40:29,435][20381] Updated weights for policy 1, policy_version 728917 (0.0007) [2023-12-27 10:40:29,909][20347] Updated weights for policy 0, policy_version 729419 (0.0007) [2023-12-27 10:40:29,962][20347] Updated weights for policy 0, policy_version 729429 (0.0007) [2023-12-27 10:40:30,016][20347] Updated weights for policy 0, policy_version 729439 (0.0008) [2023-12-27 10:40:30,093][20381] Updated weights for policy 1, policy_version 728927 (0.0007) [2023-12-27 10:40:30,152][20381] Updated weights for policy 1, policy_version 728937 (0.0007) [2023-12-27 10:40:30,205][20381] Updated weights for policy 1, policy_version 728947 (0.0007) [2023-12-27 10:40:30,653][20347] Updated weights for policy 0, policy_version 729449 (0.0006) [2023-12-27 10:40:30,700][20347] Updated weights for policy 0, policy_version 729459 (0.0005) [2023-12-27 10:40:30,746][20347] Updated weights for policy 0, policy_version 729469 (0.0005) [2023-12-27 10:40:30,807][20347] Updated weights for policy 0, policy_version 729479 (0.0005) [2023-12-27 10:40:30,857][20381] Updated weights for policy 1, policy_version 728957 (0.0008) [2023-12-27 10:40:30,906][20381] Updated weights for policy 1, policy_version 728967 (0.0007) [2023-12-27 10:40:30,958][20381] Updated weights for policy 1, policy_version 728977 (0.0008) [2023-12-27 10:40:31,454][20347] Updated weights for policy 0, policy_version 729489 (0.0009) [2023-12-27 10:40:31,514][20347] Updated weights for policy 0, policy_version 729499 (0.0008) [2023-12-27 10:40:31,573][20347] Updated weights for policy 0, policy_version 729509 (0.0008) [2023-12-27 10:40:31,711][20381] Updated weights for policy 1, policy_version 728987 (0.0009) [2023-12-27 10:40:31,772][20381] Updated weights for policy 1, policy_version 728997 (0.0006) [2023-12-27 10:40:31,832][20381] Updated weights for policy 1, policy_version 729007 (0.0006) [2023-12-27 10:40:32,264][20347] Updated weights for policy 0, policy_version 729519 (0.0009) [2023-12-27 10:40:32,326][20347] Updated weights for policy 0, policy_version 729529 (0.0006) [2023-12-27 10:40:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 373432320. Throughput: 0: 10351.1, 1: 10082.6. Samples: 373447944. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:32,363][19170] Avg episode reward: [(0, '760.560'), (1, '824.120')] [2023-12-27 10:40:32,397][20347] Updated weights for policy 0, policy_version 729539 (0.0007) [2023-12-27 10:40:32,498][20381] Updated weights for policy 1, policy_version 729017 (0.0007) [2023-12-27 10:40:32,568][20381] Updated weights for policy 1, policy_version 729027 (0.0006) [2023-12-27 10:40:32,632][20381] Updated weights for policy 1, policy_version 729037 (0.0009) [2023-12-27 10:40:32,686][20381] Updated weights for policy 1, policy_version 729047 (0.0007) [2023-12-27 10:40:32,953][20347] Updated weights for policy 0, policy_version 729549 (0.0006) [2023-12-27 10:40:33,013][20347] Updated weights for policy 0, policy_version 729559 (0.0005) [2023-12-27 10:40:33,072][20347] Updated weights for policy 0, policy_version 729569 (0.0005) [2023-12-27 10:40:33,356][20381] Updated weights for policy 1, policy_version 729057 (0.0006) [2023-12-27 10:40:33,402][20381] Updated weights for policy 1, policy_version 729067 (0.0005) [2023-12-27 10:40:33,460][20381] Updated weights for policy 1, policy_version 729077 (0.0006) [2023-12-27 10:40:33,725][20347] Updated weights for policy 0, policy_version 729579 (0.0008) [2023-12-27 10:40:33,777][20347] Updated weights for policy 0, policy_version 729589 (0.0007) [2023-12-27 10:40:33,828][20347] Updated weights for policy 0, policy_version 729599 (0.0007) [2023-12-27 10:40:34,059][20381] Updated weights for policy 1, policy_version 729087 (0.0007) [2023-12-27 10:40:34,110][20381] Updated weights for policy 1, policy_version 729097 (0.0006) [2023-12-27 10:40:34,170][20381] Updated weights for policy 1, policy_version 729107 (0.0008) [2023-12-27 10:40:34,570][20347] Updated weights for policy 0, policy_version 729609 (0.0007) [2023-12-27 10:40:34,629][20347] Updated weights for policy 0, policy_version 729619 (0.0008) [2023-12-27 10:40:34,687][20347] Updated weights for policy 0, policy_version 729629 (0.0008) [2023-12-27 10:40:34,740][20347] Updated weights for policy 0, policy_version 729639 (0.0007) [2023-12-27 10:40:34,817][20381] Updated weights for policy 1, policy_version 729117 (0.0007) [2023-12-27 10:40:34,869][20381] Updated weights for policy 1, policy_version 729127 (0.0005) [2023-12-27 10:40:34,913][20381] Updated weights for policy 1, policy_version 729137 (0.0008) [2023-12-27 10:40:35,466][20347] Updated weights for policy 0, policy_version 729649 (0.0007) [2023-12-27 10:40:35,533][20347] Updated weights for policy 0, policy_version 729659 (0.0008) [2023-12-27 10:40:35,549][20381] Updated weights for policy 1, policy_version 729147 (0.0010) [2023-12-27 10:40:35,587][20347] Updated weights for policy 0, policy_version 729669 (0.0007) [2023-12-27 10:40:35,609][20381] Updated weights for policy 1, policy_version 729157 (0.0007) [2023-12-27 10:40:35,672][20381] Updated weights for policy 1, policy_version 729167 (0.0009) [2023-12-27 10:40:36,312][20347] Updated weights for policy 0, policy_version 729679 (0.0009) [2023-12-27 10:40:36,351][20381] Updated weights for policy 1, policy_version 729177 (0.0007) [2023-12-27 10:40:36,375][20347] Updated weights for policy 0, policy_version 729689 (0.0008) [2023-12-27 10:40:36,412][20381] Updated weights for policy 1, policy_version 729187 (0.0007) [2023-12-27 10:40:36,439][20347] Updated weights for policy 0, policy_version 729699 (0.0006) [2023-12-27 10:40:36,472][20381] Updated weights for policy 1, policy_version 729197 (0.0011) [2023-12-27 10:40:36,533][20381] Updated weights for policy 1, policy_version 729207 (0.0010) [2023-12-27 10:40:37,137][20347] Updated weights for policy 0, policy_version 729709 (0.0007) [2023-12-27 10:40:37,148][20381] Updated weights for policy 1, policy_version 729217 (0.0008) [2023-12-27 10:40:37,194][20347] Updated weights for policy 0, policy_version 729719 (0.0005) [2023-12-27 10:40:37,200][20381] Updated weights for policy 1, policy_version 729227 (0.0006) [2023-12-27 10:40:37,251][20347] Updated weights for policy 0, policy_version 729729 (0.0005) [2023-12-27 10:40:37,253][20381] Updated weights for policy 1, policy_version 729237 (0.0007) [2023-12-27 10:40:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 373547008. Throughput: 0: 10343.3, 1: 10136.8. Samples: 373510964. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:37,363][19170] Avg episode reward: [(0, '856.300'), (1, '843.660')] [2023-12-27 10:40:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000729240_186712064.pth... [2023-12-27 10:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000729736_186834944.pth... [2023-12-27 10:40:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000728056_186408960.pth [2023-12-27 10:40:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000728520_186523648.pth [2023-12-27 10:40:37,864][20347] Updated weights for policy 0, policy_version 729739 (0.0006) [2023-12-27 10:40:37,915][20347] Updated weights for policy 0, policy_version 729749 (0.0008) [2023-12-27 10:40:37,951][20381] Updated weights for policy 1, policy_version 729247 (0.0007) [2023-12-27 10:40:37,971][20347] Updated weights for policy 0, policy_version 729759 (0.0007) [2023-12-27 10:40:38,016][20381] Updated weights for policy 1, policy_version 729257 (0.0006) [2023-12-27 10:40:38,089][20381] Updated weights for policy 1, policy_version 729267 (0.0007) [2023-12-27 10:40:38,645][20347] Updated weights for policy 0, policy_version 729769 (0.0008) [2023-12-27 10:40:38,699][20347] Updated weights for policy 0, policy_version 729779 (0.0007) [2023-12-27 10:40:38,712][20381] Updated weights for policy 1, policy_version 729277 (0.0007) [2023-12-27 10:40:38,756][20347] Updated weights for policy 0, policy_version 729789 (0.0006) [2023-12-27 10:40:38,769][20381] Updated weights for policy 1, policy_version 729287 (0.0006) [2023-12-27 10:40:38,817][20347] Updated weights for policy 0, policy_version 729799 (0.0006) [2023-12-27 10:40:38,827][20381] Updated weights for policy 1, policy_version 729297 (0.0007) [2023-12-27 10:40:39,522][20347] Updated weights for policy 0, policy_version 729809 (0.0007) [2023-12-27 10:40:39,555][20381] Updated weights for policy 1, policy_version 729307 (0.0007) [2023-12-27 10:40:39,580][20347] Updated weights for policy 0, policy_version 729819 (0.0006) [2023-12-27 10:40:39,612][20381] Updated weights for policy 1, policy_version 729317 (0.0007) [2023-12-27 10:40:39,640][20347] Updated weights for policy 0, policy_version 729829 (0.0007) [2023-12-27 10:40:39,670][20381] Updated weights for policy 1, policy_version 729327 (0.0006) [2023-12-27 10:40:40,304][20381] Updated weights for policy 1, policy_version 729337 (0.0006) [2023-12-27 10:40:40,317][20347] Updated weights for policy 0, policy_version 729839 (0.0007) [2023-12-27 10:40:40,360][20381] Updated weights for policy 1, policy_version 729347 (0.0008) [2023-12-27 10:40:40,378][20347] Updated weights for policy 0, policy_version 729849 (0.0006) [2023-12-27 10:40:40,414][20381] Updated weights for policy 1, policy_version 729357 (0.0006) [2023-12-27 10:40:40,432][20347] Updated weights for policy 0, policy_version 729859 (0.0007) [2023-12-27 10:40:40,438][20291] KL-divergence is very high: 104.7755 [2023-12-27 10:40:40,466][20381] Updated weights for policy 1, policy_version 729367 (0.0006) [2023-12-27 10:40:41,079][20381] Updated weights for policy 1, policy_version 729377 (0.0008) [2023-12-27 10:40:41,142][20381] Updated weights for policy 1, policy_version 729387 (0.0008) [2023-12-27 10:40:41,197][20381] Updated weights for policy 1, policy_version 729397 (0.0007) [2023-12-27 10:40:41,209][20347] Updated weights for policy 0, policy_version 729869 (0.0008) [2023-12-27 10:40:41,272][20347] Updated weights for policy 0, policy_version 729879 (0.0007) [2023-12-27 10:40:41,323][20347] Updated weights for policy 0, policy_version 729889 (0.0008) [2023-12-27 10:40:41,968][20381] Updated weights for policy 1, policy_version 729407 (0.0009) [2023-12-27 10:40:42,014][20381] Updated weights for policy 1, policy_version 729417 (0.0010) [2023-12-27 10:40:42,059][20381] Updated weights for policy 1, policy_version 729427 (0.0010) [2023-12-27 10:40:42,097][20347] Updated weights for policy 0, policy_version 729899 (0.0008) [2023-12-27 10:40:42,161][20347] Updated weights for policy 0, policy_version 729909 (0.0007) [2023-12-27 10:40:42,207][20347] Updated weights for policy 0, policy_version 729919 (0.0007) [2023-12-27 10:40:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 373645312. Throughput: 0: 10294.8, 1: 10192.1. Samples: 373632992. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:42,363][19170] Avg episode reward: [(0, '846.810'), (1, '626.380')] [2023-12-27 10:40:42,791][20381] Updated weights for policy 1, policy_version 729437 (0.0010) [2023-12-27 10:40:42,846][20381] Updated weights for policy 1, policy_version 729447 (0.0010) [2023-12-27 10:40:42,904][20381] Updated weights for policy 1, policy_version 729457 (0.0008) [2023-12-27 10:40:42,940][20347] Updated weights for policy 0, policy_version 729929 (0.0008) [2023-12-27 10:40:42,995][20347] Updated weights for policy 0, policy_version 729939 (0.0007) [2023-12-27 10:40:43,047][20347] Updated weights for policy 0, policy_version 729949 (0.0008) [2023-12-27 10:40:43,103][20347] Updated weights for policy 0, policy_version 729959 (0.0007) [2023-12-27 10:40:43,555][20381] Updated weights for policy 1, policy_version 729467 (0.0007) [2023-12-27 10:40:43,600][20381] Updated weights for policy 1, policy_version 729477 (0.0005) [2023-12-27 10:40:43,646][20381] Updated weights for policy 1, policy_version 729487 (0.0005) [2023-12-27 10:40:43,829][20347] Updated weights for policy 0, policy_version 729969 (0.0007) [2023-12-27 10:40:43,886][20347] Updated weights for policy 0, policy_version 729979 (0.0007) [2023-12-27 10:40:43,943][20347] Updated weights for policy 0, policy_version 729989 (0.0007) [2023-12-27 10:40:44,281][20381] Updated weights for policy 1, policy_version 729497 (0.0006) [2023-12-27 10:40:44,338][20381] Updated weights for policy 1, policy_version 729507 (0.0011) [2023-12-27 10:40:44,393][20381] Updated weights for policy 1, policy_version 729517 (0.0010) [2023-12-27 10:40:44,447][20381] Updated weights for policy 1, policy_version 729527 (0.0010) [2023-12-27 10:40:44,626][20347] Updated weights for policy 0, policy_version 729999 (0.0007) [2023-12-27 10:40:44,686][20347] Updated weights for policy 0, policy_version 730009 (0.0007) [2023-12-27 10:40:44,742][20347] Updated weights for policy 0, policy_version 730019 (0.0007) [2023-12-27 10:40:45,205][20381] Updated weights for policy 1, policy_version 729537 (0.0008) [2023-12-27 10:40:45,257][20381] Updated weights for policy 1, policy_version 729547 (0.0008) [2023-12-27 10:40:45,313][20381] Updated weights for policy 1, policy_version 729557 (0.0008) [2023-12-27 10:40:45,378][20347] Updated weights for policy 0, policy_version 730029 (0.0007) [2023-12-27 10:40:45,438][20347] Updated weights for policy 0, policy_version 730039 (0.0007) [2023-12-27 10:40:45,491][20347] Updated weights for policy 0, policy_version 730049 (0.0007) [2023-12-27 10:40:45,987][20381] Updated weights for policy 1, policy_version 729567 (0.0006) [2023-12-27 10:40:46,039][20381] Updated weights for policy 1, policy_version 729577 (0.0006) [2023-12-27 10:40:46,085][20381] Updated weights for policy 1, policy_version 729587 (0.0005) [2023-12-27 10:40:46,130][20347] Updated weights for policy 0, policy_version 730059 (0.0007) [2023-12-27 10:40:46,197][20347] Updated weights for policy 0, policy_version 730069 (0.0006) [2023-12-27 10:40:46,255][20347] Updated weights for policy 0, policy_version 730079 (0.0005) [2023-12-27 10:40:46,741][20381] Updated weights for policy 1, policy_version 729597 (0.0008) [2023-12-27 10:40:46,798][20381] Updated weights for policy 1, policy_version 729607 (0.0010) [2023-12-27 10:40:46,826][20347] Updated weights for policy 0, policy_version 730089 (0.0007) [2023-12-27 10:40:46,853][20381] Updated weights for policy 1, policy_version 729617 (0.0010) [2023-12-27 10:40:46,886][20347] Updated weights for policy 0, policy_version 730099 (0.0005) [2023-12-27 10:40:46,944][20347] Updated weights for policy 0, policy_version 730109 (0.0005) [2023-12-27 10:40:46,999][20347] Updated weights for policy 0, policy_version 730119 (0.0005) [2023-12-27 10:40:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 373743616. Throughput: 0: 10304.7, 1: 10181.9. Samples: 373755288. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:47,364][19170] Avg episode reward: [(0, '809.620'), (1, '508.300')] [2023-12-27 10:40:47,517][20381] Updated weights for policy 1, policy_version 729627 (0.0009) [2023-12-27 10:40:47,550][20347] Updated weights for policy 0, policy_version 730129 (0.0007) [2023-12-27 10:40:47,577][20381] Updated weights for policy 1, policy_version 729637 (0.0006) [2023-12-27 10:40:47,584][20291] KL-divergence is very high: 124.3113 [2023-12-27 10:40:47,595][20291] KL-divergence is very high: 214.9152 [2023-12-27 10:40:47,600][20291] KL-divergence is very high: 204.4277 [2023-12-27 10:40:47,606][20347] Updated weights for policy 0, policy_version 730139 (0.0005) [2023-12-27 10:40:47,606][20291] KL-divergence is very high: 155.1273 [2023-12-27 10:40:47,632][20381] Updated weights for policy 1, policy_version 729647 (0.0006) [2023-12-27 10:40:47,662][20347] Updated weights for policy 0, policy_version 730149 (0.0006) [2023-12-27 10:40:48,175][20291] KL-divergence is very high: 147.4026 [2023-12-27 10:40:48,187][20347] Updated weights for policy 0, policy_version 730159 (0.0006) [2023-12-27 10:40:48,222][20291] KL-divergence is very high: 106.2484 [2023-12-27 10:40:48,244][20347] Updated weights for policy 0, policy_version 730169 (0.0007) [2023-12-27 10:40:48,270][20381] Updated weights for policy 1, policy_version 729657 (0.0006) [2023-12-27 10:40:48,299][20347] Updated weights for policy 0, policy_version 730179 (0.0005) [2023-12-27 10:40:48,329][20381] Updated weights for policy 1, policy_version 729667 (0.0007) [2023-12-27 10:40:48,387][20381] Updated weights for policy 1, policy_version 729677 (0.0008) [2023-12-27 10:40:48,448][20381] Updated weights for policy 1, policy_version 729687 (0.0006) [2023-12-27 10:40:48,995][20347] Updated weights for policy 0, policy_version 730189 (0.0007) [2023-12-27 10:40:49,057][20381] Updated weights for policy 1, policy_version 729697 (0.0006) [2023-12-27 10:40:49,067][20347] Updated weights for policy 0, policy_version 730199 (0.0007) [2023-12-27 10:40:49,121][20381] Updated weights for policy 1, policy_version 729707 (0.0008) [2023-12-27 10:40:49,133][20347] Updated weights for policy 0, policy_version 730209 (0.0007) [2023-12-27 10:40:49,177][20381] Updated weights for policy 1, policy_version 729717 (0.0007) [2023-12-27 10:40:49,806][20347] Updated weights for policy 0, policy_version 730219 (0.0007) [2023-12-27 10:40:49,870][20347] Updated weights for policy 0, policy_version 730229 (0.0008) [2023-12-27 10:40:49,885][20381] Updated weights for policy 1, policy_version 729727 (0.0007) [2023-12-27 10:40:49,928][20347] Updated weights for policy 0, policy_version 730239 (0.0007) [2023-12-27 10:40:49,947][20381] Updated weights for policy 1, policy_version 729737 (0.0006) [2023-12-27 10:40:50,004][20381] Updated weights for policy 1, policy_version 729747 (0.0007) [2023-12-27 10:40:50,650][20347] Updated weights for policy 0, policy_version 730249 (0.0008) [2023-12-27 10:40:50,652][20381] Updated weights for policy 1, policy_version 729757 (0.0008) [2023-12-27 10:40:50,710][20347] Updated weights for policy 0, policy_version 730259 (0.0006) [2023-12-27 10:40:50,711][20381] Updated weights for policy 1, policy_version 729767 (0.0006) [2023-12-27 10:40:50,768][20381] Updated weights for policy 1, policy_version 729777 (0.0006) [2023-12-27 10:40:50,773][20347] Updated weights for policy 0, policy_version 730269 (0.0006) [2023-12-27 10:40:50,833][20347] Updated weights for policy 0, policy_version 730279 (0.0007) [2023-12-27 10:40:51,518][20347] Updated weights for policy 0, policy_version 730289 (0.0006) [2023-12-27 10:40:51,528][20381] Updated weights for policy 1, policy_version 729787 (0.0007) [2023-12-27 10:40:51,574][20347] Updated weights for policy 0, policy_version 730299 (0.0005) [2023-12-27 10:40:51,580][20381] Updated weights for policy 1, policy_version 729797 (0.0008) [2023-12-27 10:40:51,637][20347] Updated weights for policy 0, policy_version 730309 (0.0007) [2023-12-27 10:40:51,641][20381] Updated weights for policy 1, policy_version 729807 (0.0008) [2023-12-27 10:40:52,291][20347] Updated weights for policy 0, policy_version 730319 (0.0007) [2023-12-27 10:40:52,318][20381] Updated weights for policy 1, policy_version 729817 (0.0008) [2023-12-27 10:40:52,341][20347] Updated weights for policy 0, policy_version 730329 (0.0007) [2023-12-27 10:40:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 373841920. Throughput: 0: 10273.8, 1: 10171.4. Samples: 373818928. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:52,363][19170] Avg episode reward: [(0, '800.080'), (1, '400.220')] [2023-12-27 10:40:52,376][20381] Updated weights for policy 1, policy_version 729827 (0.0006) [2023-12-27 10:40:52,407][20347] Updated weights for policy 0, policy_version 730339 (0.0008) [2023-12-27 10:40:52,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000730344_186990592.pth... [2023-12-27 10:40:52,436][20381] Updated weights for policy 1, policy_version 729837 (0.0007) [2023-12-27 10:40:52,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000729096_186671104.pth [2023-12-27 10:40:52,491][20381] Updated weights for policy 1, policy_version 729847 (0.0008) [2023-12-27 10:40:52,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000729848_186867712.pth... [2023-12-27 10:40:52,497][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000728632_186556416.pth [2023-12-27 10:40:53,081][20347] Updated weights for policy 0, policy_version 730349 (0.0008) [2023-12-27 10:40:53,146][20347] Updated weights for policy 0, policy_version 730359 (0.0008) [2023-12-27 10:40:53,195][20381] Updated weights for policy 1, policy_version 729857 (0.0006) [2023-12-27 10:40:53,201][20347] Updated weights for policy 0, policy_version 730369 (0.0008) [2023-12-27 10:40:53,250][20381] Updated weights for policy 1, policy_version 729867 (0.0005) [2023-12-27 10:40:53,306][20381] Updated weights for policy 1, policy_version 729877 (0.0006) [2023-12-27 10:40:53,869][20381] Updated weights for policy 1, policy_version 729887 (0.0007) [2023-12-27 10:40:53,933][20381] Updated weights for policy 1, policy_version 729897 (0.0008) [2023-12-27 10:40:53,940][20347] Updated weights for policy 0, policy_version 730379 (0.0007) [2023-12-27 10:40:53,991][20347] Updated weights for policy 0, policy_version 730389 (0.0006) [2023-12-27 10:40:53,997][20381] Updated weights for policy 1, policy_version 729907 (0.0008) [2023-12-27 10:40:54,048][20347] Updated weights for policy 0, policy_version 730399 (0.0007) [2023-12-27 10:40:54,631][20381] Updated weights for policy 1, policy_version 729917 (0.0006) [2023-12-27 10:40:54,675][20381] Updated weights for policy 1, policy_version 729927 (0.0005) [2023-12-27 10:40:54,678][20347] Updated weights for policy 0, policy_version 730409 (0.0007) [2023-12-27 10:40:54,722][20381] Updated weights for policy 1, policy_version 729937 (0.0005) [2023-12-27 10:40:54,731][20347] Updated weights for policy 0, policy_version 730419 (0.0007) [2023-12-27 10:40:54,775][20347] Updated weights for policy 0, policy_version 730429 (0.0005) [2023-12-27 10:40:54,822][20347] Updated weights for policy 0, policy_version 730439 (0.0005) [2023-12-27 10:40:55,398][20347] Updated weights for policy 0, policy_version 730449 (0.0010) [2023-12-27 10:40:55,410][20381] Updated weights for policy 1, policy_version 729947 (0.0007) [2023-12-27 10:40:55,456][20347] Updated weights for policy 0, policy_version 730459 (0.0008) [2023-12-27 10:40:55,465][20381] Updated weights for policy 1, policy_version 729957 (0.0005) [2023-12-27 10:40:55,515][20347] Updated weights for policy 0, policy_version 730469 (0.0005) [2023-12-27 10:40:55,521][20381] Updated weights for policy 1, policy_version 729967 (0.0007) [2023-12-27 10:40:56,184][20381] Updated weights for policy 1, policy_version 729977 (0.0007) [2023-12-27 10:40:56,186][20347] Updated weights for policy 0, policy_version 730479 (0.0007) [2023-12-27 10:40:56,247][20347] Updated weights for policy 0, policy_version 730489 (0.0006) [2023-12-27 10:40:56,247][20381] Updated weights for policy 1, policy_version 729987 (0.0007) [2023-12-27 10:40:56,310][20347] Updated weights for policy 0, policy_version 730499 (0.0006) [2023-12-27 10:40:56,311][20381] Updated weights for policy 1, policy_version 729997 (0.0006) [2023-12-27 10:40:56,363][20381] Updated weights for policy 1, policy_version 730007 (0.0006) [2023-12-27 10:40:56,939][20381] Updated weights for policy 1, policy_version 730017 (0.0007) [2023-12-27 10:40:56,994][20381] Updated weights for policy 1, policy_version 730027 (0.0005) [2023-12-27 10:40:57,022][20347] Updated weights for policy 0, policy_version 730509 (0.0008) [2023-12-27 10:40:57,053][20381] Updated weights for policy 1, policy_version 730037 (0.0005) [2023-12-27 10:40:57,077][20347] Updated weights for policy 0, policy_version 730519 (0.0010) [2023-12-27 10:40:57,130][20347] Updated weights for policy 0, policy_version 730529 (0.0010) [2023-12-27 10:40:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 373956608. Throughput: 0: 10342.0, 1: 10214.4. Samples: 373943376. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:40:57,363][19170] Avg episode reward: [(0, '866.110'), (1, '323.080')] [2023-12-27 10:40:57,626][20381] Updated weights for policy 1, policy_version 730047 (0.0006) [2023-12-27 10:40:57,674][20381] Updated weights for policy 1, policy_version 730057 (0.0006) [2023-12-27 10:40:57,724][20292] KL-divergence is very high: 302.9776 [2023-12-27 10:40:57,729][20381] Updated weights for policy 1, policy_version 730067 (0.0007) [2023-12-27 10:40:57,768][20347] Updated weights for policy 0, policy_version 730539 (0.0010) [2023-12-27 10:40:57,827][20347] Updated weights for policy 0, policy_version 730549 (0.0010) [2023-12-27 10:40:57,885][20347] Updated weights for policy 0, policy_version 730559 (0.0010) [2023-12-27 10:40:58,461][20381] Updated weights for policy 1, policy_version 730077 (0.0007) [2023-12-27 10:40:58,524][20381] Updated weights for policy 1, policy_version 730087 (0.0007) [2023-12-27 10:40:58,589][20381] Updated weights for policy 1, policy_version 730097 (0.0008) [2023-12-27 10:40:58,599][20347] Updated weights for policy 0, policy_version 730569 (0.0010) [2023-12-27 10:40:58,659][20347] Updated weights for policy 0, policy_version 730579 (0.0008) [2023-12-27 10:40:58,725][20347] Updated weights for policy 0, policy_version 730589 (0.0009) [2023-12-27 10:40:58,793][20347] Updated weights for policy 0, policy_version 730599 (0.0010) [2023-12-27 10:40:59,361][20381] Updated weights for policy 1, policy_version 730107 (0.0007) [2023-12-27 10:40:59,415][20381] Updated weights for policy 1, policy_version 730117 (0.0008) [2023-12-27 10:40:59,471][20381] Updated weights for policy 1, policy_version 730127 (0.0007) [2023-12-27 10:40:59,558][20347] Updated weights for policy 0, policy_version 730609 (0.0006) [2023-12-27 10:40:59,614][20347] Updated weights for policy 0, policy_version 730619 (0.0005) [2023-12-27 10:40:59,627][20291] KL-divergence is very high: 123.8450 [2023-12-27 10:40:59,673][20347] Updated weights for policy 0, policy_version 730629 (0.0005) [2023-12-27 10:41:00,258][20381] Updated weights for policy 1, policy_version 730137 (0.0007) [2023-12-27 10:41:00,272][20347] Updated weights for policy 0, policy_version 730639 (0.0006) [2023-12-27 10:41:00,309][20381] Updated weights for policy 1, policy_version 730147 (0.0008) [2023-12-27 10:41:00,335][20347] Updated weights for policy 0, policy_version 730649 (0.0006) [2023-12-27 10:41:00,368][20381] Updated weights for policy 1, policy_version 730157 (0.0006) [2023-12-27 10:41:00,393][20347] Updated weights for policy 0, policy_version 730659 (0.0006) [2023-12-27 10:41:00,429][20381] Updated weights for policy 1, policy_version 730167 (0.0007) [2023-12-27 10:41:00,972][20347] Updated weights for policy 0, policy_version 730669 (0.0008) [2023-12-27 10:41:00,979][20381] Updated weights for policy 1, policy_version 730177 (0.0008) [2023-12-27 10:41:01,023][20347] Updated weights for policy 0, policy_version 730679 (0.0010) [2023-12-27 10:41:01,044][20381] Updated weights for policy 1, policy_version 730187 (0.0008) [2023-12-27 10:41:01,079][20347] Updated weights for policy 0, policy_version 730689 (0.0010) [2023-12-27 10:41:01,108][20381] Updated weights for policy 1, policy_version 730197 (0.0008) [2023-12-27 10:41:01,768][20381] Updated weights for policy 1, policy_version 730207 (0.0010) [2023-12-27 10:41:01,822][20381] Updated weights for policy 1, policy_version 730217 (0.0011) [2023-12-27 10:41:01,839][20347] Updated weights for policy 0, policy_version 730699 (0.0009) [2023-12-27 10:41:01,886][20381] Updated weights for policy 1, policy_version 730227 (0.0011) [2023-12-27 10:41:01,899][20347] Updated weights for policy 0, policy_version 730709 (0.0006) [2023-12-27 10:41:01,954][20347] Updated weights for policy 0, policy_version 730719 (0.0007) [2023-12-27 10:41:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20410.6). Total num frames: 374054912. Throughput: 0: 10310.2, 1: 10257.3. Samples: 374064996. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:02,363][19170] Avg episode reward: [(0, '834.820'), (1, '449.670')] [2023-12-27 10:41:02,627][20381] Updated weights for policy 1, policy_version 730237 (0.0009) [2023-12-27 10:41:02,648][20347] Updated weights for policy 0, policy_version 730729 (0.0007) [2023-12-27 10:41:02,685][20381] Updated weights for policy 1, policy_version 730247 (0.0006) [2023-12-27 10:41:02,709][20347] Updated weights for policy 0, policy_version 730739 (0.0006) [2023-12-27 10:41:02,742][20381] Updated weights for policy 1, policy_version 730257 (0.0005) [2023-12-27 10:41:02,767][20347] Updated weights for policy 0, policy_version 730749 (0.0006) [2023-12-27 10:41:02,829][20347] Updated weights for policy 0, policy_version 730759 (0.0006) [2023-12-27 10:41:03,429][20381] Updated weights for policy 1, policy_version 730267 (0.0006) [2023-12-27 10:41:03,485][20347] Updated weights for policy 0, policy_version 730769 (0.0006) [2023-12-27 10:41:03,486][20381] Updated weights for policy 1, policy_version 730277 (0.0006) [2023-12-27 10:41:03,541][20381] Updated weights for policy 1, policy_version 730287 (0.0008) [2023-12-27 10:41:03,544][20347] Updated weights for policy 0, policy_version 730779 (0.0006) [2023-12-27 10:41:03,603][20347] Updated weights for policy 0, policy_version 730789 (0.0007) [2023-12-27 10:41:04,243][20381] Updated weights for policy 1, policy_version 730297 (0.0007) [2023-12-27 10:41:04,278][20347] Updated weights for policy 0, policy_version 730799 (0.0009) [2023-12-27 10:41:04,306][20381] Updated weights for policy 1, policy_version 730307 (0.0008) [2023-12-27 10:41:04,341][20347] Updated weights for policy 0, policy_version 730809 (0.0011) [2023-12-27 10:41:04,369][20381] Updated weights for policy 1, policy_version 730317 (0.0006) [2023-12-27 10:41:04,401][20347] Updated weights for policy 0, policy_version 730819 (0.0008) [2023-12-27 10:41:04,432][20381] Updated weights for policy 1, policy_version 730327 (0.0006) [2023-12-27 10:41:05,012][20347] Updated weights for policy 0, policy_version 730829 (0.0008) [2023-12-27 10:41:05,031][20381] Updated weights for policy 1, policy_version 730337 (0.0005) [2023-12-27 10:41:05,068][20347] Updated weights for policy 0, policy_version 730839 (0.0007) [2023-12-27 10:41:05,082][20381] Updated weights for policy 1, policy_version 730347 (0.0005) [2023-12-27 10:41:05,127][20347] Updated weights for policy 0, policy_version 730849 (0.0007) [2023-12-27 10:41:05,129][20381] Updated weights for policy 1, policy_version 730357 (0.0005) [2023-12-27 10:41:05,742][20347] Updated weights for policy 0, policy_version 730859 (0.0007) [2023-12-27 10:41:05,794][20381] Updated weights for policy 1, policy_version 730367 (0.0008) [2023-12-27 10:41:05,795][20347] Updated weights for policy 0, policy_version 730869 (0.0007) [2023-12-27 10:41:05,845][20347] Updated weights for policy 0, policy_version 730879 (0.0008) [2023-12-27 10:41:05,847][20381] Updated weights for policy 1, policy_version 730377 (0.0007) [2023-12-27 10:41:05,904][20381] Updated weights for policy 1, policy_version 730387 (0.0007) [2023-12-27 10:41:06,593][20381] Updated weights for policy 1, policy_version 730397 (0.0007) [2023-12-27 10:41:06,609][20347] Updated weights for policy 0, policy_version 730889 (0.0006) [2023-12-27 10:41:06,657][20381] Updated weights for policy 1, policy_version 730407 (0.0007) [2023-12-27 10:41:06,663][20347] Updated weights for policy 0, policy_version 730899 (0.0007) [2023-12-27 10:41:06,722][20381] Updated weights for policy 1, policy_version 730417 (0.0006) [2023-12-27 10:41:06,726][20347] Updated weights for policy 0, policy_version 730909 (0.0006) [2023-12-27 10:41:06,776][20347] Updated weights for policy 0, policy_version 730919 (0.0006) [2023-12-27 10:41:07,262][20381] Updated weights for policy 1, policy_version 730427 (0.0005) [2023-12-27 10:41:07,312][20381] Updated weights for policy 1, policy_version 730437 (0.0006) [2023-12-27 10:41:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20410.6). Total num frames: 374153216. Throughput: 0: 10347.0, 1: 10255.8. Samples: 374127164. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:07,363][19170] Avg episode reward: [(0, '892.850'), (1, '387.280')] [2023-12-27 10:41:07,369][20381] Updated weights for policy 1, policy_version 730447 (0.0006) [2023-12-27 10:41:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000730920_187138048.pth... [2023-12-27 10:41:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000729736_186834944.pth [2023-12-27 10:41:07,410][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000730456_187023360.pth... [2023-12-27 10:41:07,413][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000729240_186712064.pth [2023-12-27 10:41:07,440][20347] Updated weights for policy 0, policy_version 730929 (0.0010) [2023-12-27 10:41:07,495][20347] Updated weights for policy 0, policy_version 730939 (0.0010) [2023-12-27 10:41:07,548][20347] Updated weights for policy 0, policy_version 730949 (0.0010) [2023-12-27 10:41:07,935][20381] Updated weights for policy 1, policy_version 730457 (0.0008) [2023-12-27 10:41:07,988][20381] Updated weights for policy 1, policy_version 730467 (0.0005) [2023-12-27 10:41:08,051][20381] Updated weights for policy 1, policy_version 730477 (0.0005) [2023-12-27 10:41:08,112][20381] Updated weights for policy 1, policy_version 730487 (0.0006) [2023-12-27 10:41:08,222][20347] Updated weights for policy 0, policy_version 730959 (0.0010) [2023-12-27 10:41:08,276][20347] Updated weights for policy 0, policy_version 730969 (0.0010) [2023-12-27 10:41:08,331][20347] Updated weights for policy 0, policy_version 730979 (0.0011) [2023-12-27 10:41:08,706][20381] Updated weights for policy 1, policy_version 730497 (0.0011) [2023-12-27 10:41:08,765][20381] Updated weights for policy 1, policy_version 730507 (0.0011) [2023-12-27 10:41:08,825][20381] Updated weights for policy 1, policy_version 730517 (0.0011) [2023-12-27 10:41:09,052][20347] Updated weights for policy 0, policy_version 730989 (0.0009) [2023-12-27 10:41:09,101][20347] Updated weights for policy 0, policy_version 730999 (0.0010) [2023-12-27 10:41:09,157][20347] Updated weights for policy 0, policy_version 731009 (0.0008) [2023-12-27 10:41:09,568][20381] Updated weights for policy 1, policy_version 730527 (0.0011) [2023-12-27 10:41:09,631][20381] Updated weights for policy 1, policy_version 730537 (0.0011) [2023-12-27 10:41:09,687][20381] Updated weights for policy 1, policy_version 730547 (0.0011) [2023-12-27 10:41:09,914][20347] Updated weights for policy 0, policy_version 731019 (0.0007) [2023-12-27 10:41:09,971][20347] Updated weights for policy 0, policy_version 731029 (0.0007) [2023-12-27 10:41:10,016][20347] Updated weights for policy 0, policy_version 731039 (0.0007) [2023-12-27 10:41:10,394][20381] Updated weights for policy 1, policy_version 730557 (0.0011) [2023-12-27 10:41:10,444][20381] Updated weights for policy 1, policy_version 730567 (0.0011) [2023-12-27 10:41:10,499][20381] Updated weights for policy 1, policy_version 730577 (0.0010) [2023-12-27 10:41:10,671][20347] Updated weights for policy 0, policy_version 731049 (0.0007) [2023-12-27 10:41:10,723][20347] Updated weights for policy 0, policy_version 731059 (0.0006) [2023-12-27 10:41:10,781][20347] Updated weights for policy 0, policy_version 731069 (0.0006) [2023-12-27 10:41:10,835][20347] Updated weights for policy 0, policy_version 731079 (0.0005) [2023-12-27 10:41:11,222][20381] Updated weights for policy 1, policy_version 730587 (0.0008) [2023-12-27 10:41:11,286][20381] Updated weights for policy 1, policy_version 730597 (0.0009) [2023-12-27 10:41:11,349][20381] Updated weights for policy 1, policy_version 730607 (0.0010) [2023-12-27 10:41:11,527][20347] Updated weights for policy 0, policy_version 731089 (0.0007) [2023-12-27 10:41:11,590][20347] Updated weights for policy 0, policy_version 731099 (0.0005) [2023-12-27 10:41:11,656][20347] Updated weights for policy 0, policy_version 731109 (0.0007) [2023-12-27 10:41:12,112][20381] Updated weights for policy 1, policy_version 730617 (0.0009) [2023-12-27 10:41:12,168][20381] Updated weights for policy 1, policy_version 730627 (0.0011) [2023-12-27 10:41:12,221][20381] Updated weights for policy 1, policy_version 730637 (0.0011) [2023-12-27 10:41:12,280][20381] Updated weights for policy 1, policy_version 730647 (0.0009) [2023-12-27 10:41:12,316][20347] Updated weights for policy 0, policy_version 731119 (0.0007) [2023-12-27 10:41:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 374259712. Throughput: 0: 10329.7, 1: 10334.7. Samples: 374251284. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:12,363][19170] Avg episode reward: [(0, '904.200'), (1, '267.490')] [2023-12-27 10:41:12,385][20347] Updated weights for policy 0, policy_version 731129 (0.0007) [2023-12-27 10:41:12,445][20347] Updated weights for policy 0, policy_version 731139 (0.0008) [2023-12-27 10:41:12,995][20381] Updated weights for policy 1, policy_version 730657 (0.0008) [2023-12-27 10:41:13,044][20381] Updated weights for policy 1, policy_version 730667 (0.0007) [2023-12-27 10:41:13,080][20347] Updated weights for policy 0, policy_version 731149 (0.0007) [2023-12-27 10:41:13,095][20381] Updated weights for policy 1, policy_version 730677 (0.0006) [2023-12-27 10:41:13,136][20347] Updated weights for policy 0, policy_version 731159 (0.0007) [2023-12-27 10:41:13,194][20347] Updated weights for policy 0, policy_version 731169 (0.0008) [2023-12-27 10:41:13,676][20381] Updated weights for policy 1, policy_version 730687 (0.0005) [2023-12-27 10:41:13,725][20381] Updated weights for policy 1, policy_version 730697 (0.0006) [2023-12-27 10:41:13,780][20381] Updated weights for policy 1, policy_version 730707 (0.0007) [2023-12-27 10:41:13,915][20347] Updated weights for policy 0, policy_version 731179 (0.0007) [2023-12-27 10:41:13,977][20347] Updated weights for policy 0, policy_version 731189 (0.0006) [2023-12-27 10:41:14,035][20347] Updated weights for policy 0, policy_version 731199 (0.0006) [2023-12-27 10:41:14,398][20381] Updated weights for policy 1, policy_version 730717 (0.0008) [2023-12-27 10:41:14,467][20381] Updated weights for policy 1, policy_version 730727 (0.0009) [2023-12-27 10:41:14,528][20381] Updated weights for policy 1, policy_version 730737 (0.0008) [2023-12-27 10:41:14,661][20347] Updated weights for policy 0, policy_version 731209 (0.0007) [2023-12-27 10:41:14,723][20347] Updated weights for policy 0, policy_version 731219 (0.0008) [2023-12-27 10:41:14,785][20347] Updated weights for policy 0, policy_version 731229 (0.0008) [2023-12-27 10:41:14,848][20347] Updated weights for policy 0, policy_version 731239 (0.0008) [2023-12-27 10:41:15,205][20381] Updated weights for policy 1, policy_version 730747 (0.0008) [2023-12-27 10:41:15,259][20381] Updated weights for policy 1, policy_version 730757 (0.0008) [2023-12-27 10:41:15,309][20381] Updated weights for policy 1, policy_version 730767 (0.0005) [2023-12-27 10:41:15,550][20347] Updated weights for policy 0, policy_version 731249 (0.0010) [2023-12-27 10:41:15,555][20291] KL-divergence is very high: 224.3189 [2023-12-27 10:41:15,614][20347] Updated weights for policy 0, policy_version 731259 (0.0010) [2023-12-27 10:41:15,675][20347] Updated weights for policy 0, policy_version 731269 (0.0009) [2023-12-27 10:41:15,974][20381] Updated weights for policy 1, policy_version 730777 (0.0008) [2023-12-27 10:41:16,022][20381] Updated weights for policy 1, policy_version 730787 (0.0007) [2023-12-27 10:41:16,079][20381] Updated weights for policy 1, policy_version 730797 (0.0008) [2023-12-27 10:41:16,133][20381] Updated weights for policy 1, policy_version 730807 (0.0007) [2023-12-27 10:41:16,313][20347] Updated weights for policy 0, policy_version 731279 (0.0010) [2023-12-27 10:41:16,361][20347] Updated weights for policy 0, policy_version 731289 (0.0010) [2023-12-27 10:41:16,412][20347] Updated weights for policy 0, policy_version 731299 (0.0010) [2023-12-27 10:41:16,813][20381] Updated weights for policy 1, policy_version 730817 (0.0007) [2023-12-27 10:41:16,867][20381] Updated weights for policy 1, policy_version 730827 (0.0008) [2023-12-27 10:41:16,921][20381] Updated weights for policy 1, policy_version 730837 (0.0007) [2023-12-27 10:41:17,107][20347] Updated weights for policy 0, policy_version 731309 (0.0010) [2023-12-27 10:41:17,161][20347] Updated weights for policy 0, policy_version 731319 (0.0010) [2023-12-27 10:41:17,218][20347] Updated weights for policy 0, policy_version 731329 (0.0010) [2023-12-27 10:41:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 374366208. Throughput: 0: 10230.4, 1: 10350.2. Samples: 374374072. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:17,363][19170] Avg episode reward: [(0, '914.210'), (1, '367.150')] [2023-12-27 10:41:17,646][20381] Updated weights for policy 1, policy_version 730847 (0.0008) [2023-12-27 10:41:17,697][20381] Updated weights for policy 1, policy_version 730857 (0.0008) [2023-12-27 10:41:17,753][20381] Updated weights for policy 1, policy_version 730867 (0.0006) [2023-12-27 10:41:17,809][20347] Updated weights for policy 0, policy_version 731339 (0.0007) [2023-12-27 10:41:17,857][20347] Updated weights for policy 0, policy_version 731349 (0.0007) [2023-12-27 10:41:17,905][20347] Updated weights for policy 0, policy_version 731359 (0.0007) [2023-12-27 10:41:18,446][20381] Updated weights for policy 1, policy_version 730877 (0.0007) [2023-12-27 10:41:18,502][20381] Updated weights for policy 1, policy_version 730887 (0.0008) [2023-12-27 10:41:18,555][20381] Updated weights for policy 1, policy_version 730897 (0.0008) [2023-12-27 10:41:18,607][20347] Updated weights for policy 0, policy_version 731369 (0.0007) [2023-12-27 10:41:18,660][20347] Updated weights for policy 0, policy_version 731379 (0.0007) [2023-12-27 10:41:18,706][20347] Updated weights for policy 0, policy_version 731389 (0.0007) [2023-12-27 10:41:18,765][20347] Updated weights for policy 0, policy_version 731399 (0.0008) [2023-12-27 10:41:19,265][20381] Updated weights for policy 1, policy_version 730907 (0.0009) [2023-12-27 10:41:19,319][20381] Updated weights for policy 1, policy_version 730917 (0.0007) [2023-12-27 10:41:19,386][20381] Updated weights for policy 1, policy_version 730927 (0.0007) [2023-12-27 10:41:19,499][20347] Updated weights for policy 0, policy_version 731409 (0.0007) [2023-12-27 10:41:19,563][20347] Updated weights for policy 0, policy_version 731419 (0.0008) [2023-12-27 10:41:19,626][20347] Updated weights for policy 0, policy_version 731429 (0.0006) [2023-12-27 10:41:20,122][20381] Updated weights for policy 1, policy_version 730937 (0.0008) [2023-12-27 10:41:20,182][20381] Updated weights for policy 1, policy_version 730947 (0.0011) [2023-12-27 10:41:20,247][20381] Updated weights for policy 1, policy_version 730957 (0.0010) [2023-12-27 10:41:20,255][20347] Updated weights for policy 0, policy_version 731439 (0.0007) [2023-12-27 10:41:20,314][20381] Updated weights for policy 1, policy_version 730967 (0.0011) [2023-12-27 10:41:20,317][20347] Updated weights for policy 0, policy_version 731449 (0.0007) [2023-12-27 10:41:20,369][20347] Updated weights for policy 0, policy_version 731459 (0.0006) [2023-12-27 10:41:21,013][20347] Updated weights for policy 0, policy_version 731469 (0.0007) [2023-12-27 10:41:21,046][20381] Updated weights for policy 1, policy_version 730977 (0.0010) [2023-12-27 10:41:21,076][20347] Updated weights for policy 0, policy_version 731479 (0.0008) [2023-12-27 10:41:21,109][20381] Updated weights for policy 1, policy_version 730987 (0.0008) [2023-12-27 10:41:21,138][20347] Updated weights for policy 0, policy_version 731489 (0.0008) [2023-12-27 10:41:21,176][20381] Updated weights for policy 1, policy_version 730997 (0.0010) [2023-12-27 10:41:21,786][20347] Updated weights for policy 0, policy_version 731499 (0.0008) [2023-12-27 10:41:21,844][20347] Updated weights for policy 0, policy_version 731509 (0.0007) [2023-12-27 10:41:21,901][20347] Updated weights for policy 0, policy_version 731519 (0.0006) [2023-12-27 10:41:21,943][20381] Updated weights for policy 1, policy_version 731007 (0.0009) [2023-12-27 10:41:21,999][20381] Updated weights for policy 1, policy_version 731017 (0.0008) [2023-12-27 10:41:22,056][20381] Updated weights for policy 1, policy_version 731027 (0.0008) [2023-12-27 10:41:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 374464512. Throughput: 0: 10243.4, 1: 10277.1. Samples: 374434392. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:22,364][19170] Avg episode reward: [(0, '865.030'), (1, '393.860')] [2023-12-27 10:41:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000731032_187170816.pth... [2023-12-27 10:41:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000729848_186867712.pth [2023-12-27 10:41:22,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000731528_187293696.pth... [2023-12-27 10:41:22,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000730344_186990592.pth [2023-12-27 10:41:22,609][20347] Updated weights for policy 0, policy_version 731529 (0.0006) [2023-12-27 10:41:22,675][20347] Updated weights for policy 0, policy_version 731539 (0.0007) [2023-12-27 10:41:22,737][20347] Updated weights for policy 0, policy_version 731549 (0.0007) [2023-12-27 10:41:22,789][20381] Updated weights for policy 1, policy_version 731037 (0.0009) [2023-12-27 10:41:22,794][20347] Updated weights for policy 0, policy_version 731559 (0.0006) [2023-12-27 10:41:22,848][20381] Updated weights for policy 1, policy_version 731047 (0.0011) [2023-12-27 10:41:22,908][20381] Updated weights for policy 1, policy_version 731057 (0.0009) [2023-12-27 10:41:23,474][20347] Updated weights for policy 0, policy_version 731569 (0.0009) [2023-12-27 10:41:23,532][20347] Updated weights for policy 0, policy_version 731579 (0.0010) [2023-12-27 10:41:23,564][20381] Updated weights for policy 1, policy_version 731067 (0.0006) [2023-12-27 10:41:23,583][20347] Updated weights for policy 0, policy_version 731589 (0.0010) [2023-12-27 10:41:23,624][20381] Updated weights for policy 1, policy_version 731077 (0.0005) [2023-12-27 10:41:23,680][20381] Updated weights for policy 1, policy_version 731087 (0.0008) [2023-12-27 10:41:24,221][20347] Updated weights for policy 0, policy_version 731599 (0.0007) [2023-12-27 10:41:24,278][20347] Updated weights for policy 0, policy_version 731609 (0.0005) [2023-12-27 10:41:24,330][20347] Updated weights for policy 0, policy_version 731619 (0.0005) [2023-12-27 10:41:24,342][20381] Updated weights for policy 1, policy_version 731097 (0.0008) [2023-12-27 10:41:24,401][20381] Updated weights for policy 1, policy_version 731107 (0.0006) [2023-12-27 10:41:24,448][20381] Updated weights for policy 1, policy_version 731117 (0.0006) [2023-12-27 10:41:24,497][20381] Updated weights for policy 1, policy_version 731127 (0.0005) [2023-12-27 10:41:25,020][20347] Updated weights for policy 0, policy_version 731629 (0.0007) [2023-12-27 10:41:25,067][20347] Updated weights for policy 0, policy_version 731639 (0.0007) [2023-12-27 10:41:25,081][20381] Updated weights for policy 1, policy_version 731137 (0.0005) [2023-12-27 10:41:25,126][20347] Updated weights for policy 0, policy_version 731649 (0.0006) [2023-12-27 10:41:25,133][20381] Updated weights for policy 1, policy_version 731147 (0.0005) [2023-12-27 10:41:25,187][20381] Updated weights for policy 1, policy_version 731157 (0.0006) [2023-12-27 10:41:25,820][20347] Updated weights for policy 0, policy_version 731659 (0.0008) [2023-12-27 10:41:25,836][20381] Updated weights for policy 1, policy_version 731167 (0.0007) [2023-12-27 10:41:25,876][20347] Updated weights for policy 0, policy_version 731669 (0.0008) [2023-12-27 10:41:25,892][20381] Updated weights for policy 1, policy_version 731177 (0.0005) [2023-12-27 10:41:25,920][20347] Updated weights for policy 0, policy_version 731679 (0.0007) [2023-12-27 10:41:25,947][20381] Updated weights for policy 1, policy_version 731187 (0.0005) [2023-12-27 10:41:26,465][20381] Updated weights for policy 1, policy_version 731197 (0.0005) [2023-12-27 10:41:26,519][20381] Updated weights for policy 1, policy_version 731207 (0.0005) [2023-12-27 10:41:26,568][20347] Updated weights for policy 0, policy_version 731689 (0.0007) [2023-12-27 10:41:26,574][20381] Updated weights for policy 1, policy_version 731217 (0.0006) [2023-12-27 10:41:26,618][20347] Updated weights for policy 0, policy_version 731699 (0.0005) [2023-12-27 10:41:26,666][20347] Updated weights for policy 0, policy_version 731709 (0.0005) [2023-12-27 10:41:26,715][20347] Updated weights for policy 0, policy_version 731719 (0.0005) [2023-12-27 10:41:27,171][20381] Updated weights for policy 1, policy_version 731227 (0.0008) [2023-12-27 10:41:27,224][20381] Updated weights for policy 1, policy_version 731237 (0.0008) [2023-12-27 10:41:27,286][20381] Updated weights for policy 1, policy_version 731247 (0.0009) [2023-12-27 10:41:27,326][20347] Updated weights for policy 0, policy_version 731729 (0.0006) [2023-12-27 10:41:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 374571008. Throughput: 0: 10264.4, 1: 10273.8. Samples: 374557212. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:27,363][19170] Avg episode reward: [(0, '883.380'), (1, '393.490')] [2023-12-27 10:41:27,379][20347] Updated weights for policy 0, policy_version 731739 (0.0006) [2023-12-27 10:41:27,426][20347] Updated weights for policy 0, policy_version 731749 (0.0007) [2023-12-27 10:41:27,992][20381] Updated weights for policy 1, policy_version 731257 (0.0008) [2023-12-27 10:41:28,020][20347] Updated weights for policy 0, policy_version 731759 (0.0006) [2023-12-27 10:41:28,048][20381] Updated weights for policy 1, policy_version 731267 (0.0006) [2023-12-27 10:41:28,068][20347] Updated weights for policy 0, policy_version 731769 (0.0005) [2023-12-27 10:41:28,099][20381] Updated weights for policy 1, policy_version 731277 (0.0008) [2023-12-27 10:41:28,121][20347] Updated weights for policy 0, policy_version 731779 (0.0006) [2023-12-27 10:41:28,157][20381] Updated weights for policy 1, policy_version 731287 (0.0007) [2023-12-27 10:41:28,731][20347] Updated weights for policy 0, policy_version 731789 (0.0006) [2023-12-27 10:41:28,787][20347] Updated weights for policy 0, policy_version 731799 (0.0006) [2023-12-27 10:41:28,845][20347] Updated weights for policy 0, policy_version 731809 (0.0008) [2023-12-27 10:41:28,900][20381] Updated weights for policy 1, policy_version 731297 (0.0007) [2023-12-27 10:41:28,963][20381] Updated weights for policy 1, policy_version 731307 (0.0007) [2023-12-27 10:41:29,016][20381] Updated weights for policy 1, policy_version 731317 (0.0005) [2023-12-27 10:41:29,439][20347] Updated weights for policy 0, policy_version 731819 (0.0007) [2023-12-27 10:41:29,497][20347] Updated weights for policy 0, policy_version 731829 (0.0005) [2023-12-27 10:41:29,557][20347] Updated weights for policy 0, policy_version 731839 (0.0006) [2023-12-27 10:41:29,721][20381] Updated weights for policy 1, policy_version 731327 (0.0007) [2023-12-27 10:41:29,780][20381] Updated weights for policy 1, policy_version 731337 (0.0009) [2023-12-27 10:41:29,832][20381] Updated weights for policy 1, policy_version 731347 (0.0007) [2023-12-27 10:41:30,219][20347] Updated weights for policy 0, policy_version 731849 (0.0008) [2023-12-27 10:41:30,272][20347] Updated weights for policy 0, policy_version 731859 (0.0007) [2023-12-27 10:41:30,333][20347] Updated weights for policy 0, policy_version 731869 (0.0008) [2023-12-27 10:41:30,393][20347] Updated weights for policy 0, policy_version 731879 (0.0007) [2023-12-27 10:41:30,528][20381] Updated weights for policy 1, policy_version 731357 (0.0007) [2023-12-27 10:41:30,583][20381] Updated weights for policy 1, policy_version 731367 (0.0008) [2023-12-27 10:41:30,638][20381] Updated weights for policy 1, policy_version 731377 (0.0008) [2023-12-27 10:41:31,084][20347] Updated weights for policy 0, policy_version 731889 (0.0008) [2023-12-27 10:41:31,142][20347] Updated weights for policy 0, policy_version 731899 (0.0006) [2023-12-27 10:41:31,201][20347] Updated weights for policy 0, policy_version 731909 (0.0008) [2023-12-27 10:41:31,349][20381] Updated weights for policy 1, policy_version 731387 (0.0006) [2023-12-27 10:41:31,409][20381] Updated weights for policy 1, policy_version 731397 (0.0007) [2023-12-27 10:41:31,466][20381] Updated weights for policy 1, policy_version 731407 (0.0007) [2023-12-27 10:41:31,896][20347] Updated weights for policy 0, policy_version 731919 (0.0006) [2023-12-27 10:41:31,955][20347] Updated weights for policy 0, policy_version 731929 (0.0006) [2023-12-27 10:41:32,017][20347] Updated weights for policy 0, policy_version 731939 (0.0006) [2023-12-27 10:41:32,215][20381] Updated weights for policy 1, policy_version 731417 (0.0008) [2023-12-27 10:41:32,281][20381] Updated weights for policy 1, policy_version 731427 (0.0008) [2023-12-27 10:41:32,349][20381] Updated weights for policy 1, policy_version 731437 (0.0008) [2023-12-27 10:41:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 374669312. Throughput: 0: 10354.2, 1: 10266.0. Samples: 374683196. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:32,363][19170] Avg episode reward: [(0, '902.860'), (1, '521.090')] [2023-12-27 10:41:32,408][20381] Updated weights for policy 1, policy_version 731447 (0.0009) [2023-12-27 10:41:32,697][20347] Updated weights for policy 0, policy_version 731949 (0.0008) [2023-12-27 10:41:32,752][20347] Updated weights for policy 0, policy_version 731959 (0.0008) [2023-12-27 10:41:32,796][20347] Updated weights for policy 0, policy_version 731969 (0.0007) [2023-12-27 10:41:33,072][20381] Updated weights for policy 1, policy_version 731457 (0.0007) [2023-12-27 10:41:33,126][20381] Updated weights for policy 1, policy_version 731467 (0.0010) [2023-12-27 10:41:33,184][20381] Updated weights for policy 1, policy_version 731477 (0.0010) [2023-12-27 10:41:33,517][20347] Updated weights for policy 0, policy_version 731979 (0.0008) [2023-12-27 10:41:33,565][20347] Updated weights for policy 0, policy_version 731989 (0.0008) [2023-12-27 10:41:33,617][20347] Updated weights for policy 0, policy_version 731999 (0.0009) [2023-12-27 10:41:33,742][20381] Updated weights for policy 1, policy_version 731487 (0.0006) [2023-12-27 10:41:33,797][20381] Updated weights for policy 1, policy_version 731497 (0.0005) [2023-12-27 10:41:33,848][20381] Updated weights for policy 1, policy_version 731507 (0.0005) [2023-12-27 10:41:34,347][20347] Updated weights for policy 0, policy_version 732009 (0.0008) [2023-12-27 10:41:34,415][20347] Updated weights for policy 0, policy_version 732019 (0.0006) [2023-12-27 10:41:34,480][20347] Updated weights for policy 0, policy_version 732029 (0.0006) [2023-12-27 10:41:34,503][20381] Updated weights for policy 1, policy_version 731517 (0.0005) [2023-12-27 10:41:34,549][20347] Updated weights for policy 0, policy_version 732039 (0.0007) [2023-12-27 10:41:34,565][20381] Updated weights for policy 1, policy_version 731527 (0.0006) [2023-12-27 10:41:34,632][20381] Updated weights for policy 1, policy_version 731537 (0.0007) [2023-12-27 10:41:35,181][20347] Updated weights for policy 0, policy_version 732049 (0.0009) [2023-12-27 10:41:35,234][20347] Updated weights for policy 0, policy_version 732059 (0.0009) [2023-12-27 10:41:35,245][20381] Updated weights for policy 1, policy_version 731547 (0.0007) [2023-12-27 10:41:35,285][20347] Updated weights for policy 0, policy_version 732069 (0.0008) [2023-12-27 10:41:35,291][20381] Updated weights for policy 1, policy_version 731557 (0.0005) [2023-12-27 10:41:35,340][20381] Updated weights for policy 1, policy_version 731567 (0.0005) [2023-12-27 10:41:35,998][20381] Updated weights for policy 1, policy_version 731577 (0.0006) [2023-12-27 10:41:36,053][20381] Updated weights for policy 1, policy_version 731587 (0.0006) [2023-12-27 10:41:36,055][20347] Updated weights for policy 0, policy_version 732079 (0.0006) [2023-12-27 10:41:36,112][20381] Updated weights for policy 1, policy_version 731597 (0.0006) [2023-12-27 10:41:36,122][20347] Updated weights for policy 0, policy_version 732089 (0.0008) [2023-12-27 10:41:36,175][20381] Updated weights for policy 1, policy_version 731607 (0.0008) [2023-12-27 10:41:36,177][20347] Updated weights for policy 0, policy_version 732099 (0.0007) [2023-12-27 10:41:36,827][20347] Updated weights for policy 0, policy_version 732109 (0.0006) [2023-12-27 10:41:36,884][20347] Updated weights for policy 0, policy_version 732119 (0.0007) [2023-12-27 10:41:36,905][20381] Updated weights for policy 1, policy_version 731617 (0.0007) [2023-12-27 10:41:36,941][20347] Updated weights for policy 0, policy_version 732129 (0.0006) [2023-12-27 10:41:36,964][20381] Updated weights for policy 1, policy_version 731627 (0.0008) [2023-12-27 10:41:37,023][20381] Updated weights for policy 1, policy_version 731637 (0.0008) [2023-12-27 10:41:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 374775808. Throughput: 0: 10304.5, 1: 10295.2. Samples: 374745916. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:37,363][19170] Avg episode reward: [(0, '931.860'), (1, '472.300')] [2023-12-27 10:41:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000731640_187326464.pth... [2023-12-27 10:41:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000732136_187449344.pth... [2023-12-27 10:41:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000730456_187023360.pth [2023-12-27 10:41:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000730920_187138048.pth [2023-12-27 10:41:37,617][20347] Updated weights for policy 0, policy_version 732139 (0.0006) [2023-12-27 10:41:37,655][20381] Updated weights for policy 1, policy_version 731647 (0.0007) [2023-12-27 10:41:37,671][20347] Updated weights for policy 0, policy_version 732149 (0.0005) [2023-12-27 10:41:37,712][20381] Updated weights for policy 1, policy_version 731657 (0.0007) [2023-12-27 10:41:37,725][20347] Updated weights for policy 0, policy_version 732159 (0.0006) [2023-12-27 10:41:37,768][20381] Updated weights for policy 1, policy_version 731667 (0.0007) [2023-12-27 10:41:38,367][20381] Updated weights for policy 1, policy_version 731677 (0.0007) [2023-12-27 10:41:38,414][20347] Updated weights for policy 0, policy_version 732169 (0.0006) [2023-12-27 10:41:38,422][20381] Updated weights for policy 1, policy_version 731687 (0.0008) [2023-12-27 10:41:38,471][20347] Updated weights for policy 0, policy_version 732179 (0.0007) [2023-12-27 10:41:38,481][20381] Updated weights for policy 1, policy_version 731697 (0.0008) [2023-12-27 10:41:38,529][20347] Updated weights for policy 0, policy_version 732189 (0.0006) [2023-12-27 10:41:38,587][20347] Updated weights for policy 0, policy_version 732199 (0.0006) [2023-12-27 10:41:39,215][20381] Updated weights for policy 1, policy_version 731707 (0.0008) [2023-12-27 10:41:39,217][20347] Updated weights for policy 0, policy_version 732209 (0.0007) [2023-12-27 10:41:39,277][20381] Updated weights for policy 1, policy_version 731717 (0.0007) [2023-12-27 10:41:39,287][20347] Updated weights for policy 0, policy_version 732219 (0.0008) [2023-12-27 10:41:39,333][20381] Updated weights for policy 1, policy_version 731727 (0.0008) [2023-12-27 10:41:39,351][20347] Updated weights for policy 0, policy_version 732229 (0.0007) [2023-12-27 10:41:40,038][20347] Updated weights for policy 0, policy_version 732239 (0.0010) [2023-12-27 10:41:40,089][20381] Updated weights for policy 1, policy_version 731737 (0.0007) [2023-12-27 10:41:40,099][20347] Updated weights for policy 0, policy_version 732249 (0.0010) [2023-12-27 10:41:40,149][20381] Updated weights for policy 1, policy_version 731747 (0.0006) [2023-12-27 10:41:40,164][20347] Updated weights for policy 0, policy_version 732259 (0.0011) [2023-12-27 10:41:40,212][20381] Updated weights for policy 1, policy_version 731757 (0.0008) [2023-12-27 10:41:40,267][20381] Updated weights for policy 1, policy_version 731767 (0.0007) [2023-12-27 10:41:40,853][20347] Updated weights for policy 0, policy_version 732269 (0.0011) [2023-12-27 10:41:40,907][20347] Updated weights for policy 0, policy_version 732279 (0.0007) [2023-12-27 10:41:40,955][20381] Updated weights for policy 1, policy_version 731777 (0.0008) [2023-12-27 10:41:40,961][20347] Updated weights for policy 0, policy_version 732289 (0.0005) [2023-12-27 10:41:41,013][20381] Updated weights for policy 1, policy_version 731787 (0.0008) [2023-12-27 10:41:41,082][20381] Updated weights for policy 1, policy_version 731797 (0.0009) [2023-12-27 10:41:41,686][20347] Updated weights for policy 0, policy_version 732299 (0.0006) [2023-12-27 10:41:41,749][20347] Updated weights for policy 0, policy_version 732309 (0.0008) [2023-12-27 10:41:41,817][20347] Updated weights for policy 0, policy_version 732319 (0.0010) [2023-12-27 10:41:41,823][20381] Updated weights for policy 1, policy_version 731807 (0.0007) [2023-12-27 10:41:41,887][20381] Updated weights for policy 1, policy_version 731817 (0.0006) [2023-12-27 10:41:41,943][20381] Updated weights for policy 1, policy_version 731827 (0.0008) [2023-12-27 10:41:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 374874112. Throughput: 0: 10262.9, 1: 10241.0. Samples: 374866052. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:42,363][19170] Avg episode reward: [(0, '951.300'), (1, '386.520')] [2023-12-27 10:41:42,467][20347] Updated weights for policy 0, policy_version 732329 (0.0010) [2023-12-27 10:41:42,523][20347] Updated weights for policy 0, policy_version 732339 (0.0007) [2023-12-27 10:41:42,576][20347] Updated weights for policy 0, policy_version 732349 (0.0008) [2023-12-27 10:41:42,639][20347] Updated weights for policy 0, policy_version 732359 (0.0008) [2023-12-27 10:41:42,666][20381] Updated weights for policy 1, policy_version 731837 (0.0008) [2023-12-27 10:41:42,722][20381] Updated weights for policy 1, policy_version 731847 (0.0008) [2023-12-27 10:41:42,780][20381] Updated weights for policy 1, policy_version 731857 (0.0008) [2023-12-27 10:41:43,280][20347] Updated weights for policy 0, policy_version 732369 (0.0007) [2023-12-27 10:41:43,323][20347] Updated weights for policy 0, policy_version 732379 (0.0005) [2023-12-27 10:41:43,370][20347] Updated weights for policy 0, policy_version 732389 (0.0005) [2023-12-27 10:41:43,483][20381] Updated weights for policy 1, policy_version 731867 (0.0007) [2023-12-27 10:41:43,538][20381] Updated weights for policy 1, policy_version 731877 (0.0005) [2023-12-27 10:41:43,587][20381] Updated weights for policy 1, policy_version 731887 (0.0006) [2023-12-27 10:41:44,037][20347] Updated weights for policy 0, policy_version 732399 (0.0006) [2023-12-27 10:41:44,091][20347] Updated weights for policy 0, policy_version 732409 (0.0008) [2023-12-27 10:41:44,145][20347] Updated weights for policy 0, policy_version 732419 (0.0007) [2023-12-27 10:41:44,229][20381] Updated weights for policy 1, policy_version 731897 (0.0008) [2023-12-27 10:41:44,274][20381] Updated weights for policy 1, policy_version 731907 (0.0007) [2023-12-27 10:41:44,320][20381] Updated weights for policy 1, policy_version 731917 (0.0008) [2023-12-27 10:41:44,383][20381] Updated weights for policy 1, policy_version 731927 (0.0007) [2023-12-27 10:41:44,767][20347] Updated weights for policy 0, policy_version 732429 (0.0008) [2023-12-27 10:41:44,832][20347] Updated weights for policy 0, policy_version 732439 (0.0008) [2023-12-27 10:41:44,894][20347] Updated weights for policy 0, policy_version 732449 (0.0008) [2023-12-27 10:41:45,060][20381] Updated weights for policy 1, policy_version 731937 (0.0008) [2023-12-27 10:41:45,124][20381] Updated weights for policy 1, policy_version 731947 (0.0008) [2023-12-27 10:41:45,187][20381] Updated weights for policy 1, policy_version 731957 (0.0008) [2023-12-27 10:41:45,595][20347] Updated weights for policy 0, policy_version 732459 (0.0008) [2023-12-27 10:41:45,653][20347] Updated weights for policy 0, policy_version 732469 (0.0008) [2023-12-27 10:41:45,705][20347] Updated weights for policy 0, policy_version 732479 (0.0008) [2023-12-27 10:41:45,829][20381] Updated weights for policy 1, policy_version 731967 (0.0008) [2023-12-27 10:41:45,885][20381] Updated weights for policy 1, policy_version 731977 (0.0007) [2023-12-27 10:41:45,943][20381] Updated weights for policy 1, policy_version 731987 (0.0008) [2023-12-27 10:41:46,372][20347] Updated weights for policy 0, policy_version 732489 (0.0008) [2023-12-27 10:41:46,428][20347] Updated weights for policy 0, policy_version 732499 (0.0006) [2023-12-27 10:41:46,481][20347] Updated weights for policy 0, policy_version 732509 (0.0007) [2023-12-27 10:41:46,530][20347] Updated weights for policy 0, policy_version 732519 (0.0008) [2023-12-27 10:41:46,656][20381] Updated weights for policy 1, policy_version 731997 (0.0007) [2023-12-27 10:41:46,711][20381] Updated weights for policy 1, policy_version 732007 (0.0008) [2023-12-27 10:41:46,768][20381] Updated weights for policy 1, policy_version 732017 (0.0007) [2023-12-27 10:41:47,177][20347] Updated weights for policy 0, policy_version 732529 (0.0007) [2023-12-27 10:41:47,231][20347] Updated weights for policy 0, policy_version 732539 (0.0007) [2023-12-27 10:41:47,288][20347] Updated weights for policy 0, policy_version 732549 (0.0008) [2023-12-27 10:41:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 374980608. Throughput: 0: 10305.1, 1: 10222.8. Samples: 374988756. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:47,364][19170] Avg episode reward: [(0, '970.780'), (1, '570.340')] [2023-12-27 10:41:47,429][20381] Updated weights for policy 1, policy_version 732027 (0.0007) [2023-12-27 10:41:47,489][20381] Updated weights for policy 1, policy_version 732037 (0.0008) [2023-12-27 10:41:47,544][20381] Updated weights for policy 1, policy_version 732047 (0.0009) [2023-12-27 10:41:47,905][20347] Updated weights for policy 0, policy_version 732559 (0.0007) [2023-12-27 10:41:47,964][20347] Updated weights for policy 0, policy_version 732569 (0.0007) [2023-12-27 10:41:48,014][20347] Updated weights for policy 0, policy_version 732579 (0.0007) [2023-12-27 10:41:48,204][20381] Updated weights for policy 1, policy_version 732057 (0.0006) [2023-12-27 10:41:48,253][20381] Updated weights for policy 1, policy_version 732067 (0.0005) [2023-12-27 10:41:48,309][20381] Updated weights for policy 1, policy_version 732077 (0.0009) [2023-12-27 10:41:48,373][20381] Updated weights for policy 1, policy_version 732087 (0.0011) [2023-12-27 10:41:48,719][20347] Updated weights for policy 0, policy_version 732589 (0.0007) [2023-12-27 10:41:48,782][20347] Updated weights for policy 0, policy_version 732599 (0.0008) [2023-12-27 10:41:48,850][20347] Updated weights for policy 0, policy_version 732609 (0.0008) [2023-12-27 10:41:49,031][20381] Updated weights for policy 1, policy_version 732097 (0.0007) [2023-12-27 10:41:49,076][20381] Updated weights for policy 1, policy_version 732107 (0.0005) [2023-12-27 10:41:49,124][20381] Updated weights for policy 1, policy_version 732117 (0.0005) [2023-12-27 10:41:49,565][20347] Updated weights for policy 0, policy_version 732619 (0.0007) [2023-12-27 10:41:49,617][20347] Updated weights for policy 0, policy_version 732629 (0.0008) [2023-12-27 10:41:49,672][20347] Updated weights for policy 0, policy_version 732639 (0.0008) [2023-12-27 10:41:49,751][20381] Updated weights for policy 1, policy_version 732127 (0.0007) [2023-12-27 10:41:49,807][20381] Updated weights for policy 1, policy_version 732137 (0.0008) [2023-12-27 10:41:49,873][20381] Updated weights for policy 1, policy_version 732147 (0.0007) [2023-12-27 10:41:50,413][20347] Updated weights for policy 0, policy_version 732649 (0.0007) [2023-12-27 10:41:50,472][20347] Updated weights for policy 0, policy_version 732659 (0.0008) [2023-12-27 10:41:50,529][20347] Updated weights for policy 0, policy_version 732669 (0.0007) [2023-12-27 10:41:50,580][20347] Updated weights for policy 0, policy_version 732679 (0.0008) [2023-12-27 10:41:50,607][20381] Updated weights for policy 1, policy_version 732157 (0.0007) [2023-12-27 10:41:50,656][20381] Updated weights for policy 1, policy_version 732167 (0.0008) [2023-12-27 10:41:50,713][20381] Updated weights for policy 1, policy_version 732177 (0.0007) [2023-12-27 10:41:51,310][20347] Updated weights for policy 0, policy_version 732689 (0.0008) [2023-12-27 10:41:51,369][20381] Updated weights for policy 1, policy_version 732187 (0.0008) [2023-12-27 10:41:51,388][20347] Updated weights for policy 0, policy_version 732699 (0.0008) [2023-12-27 10:41:51,435][20381] Updated weights for policy 1, policy_version 732197 (0.0008) [2023-12-27 10:41:51,444][20347] Updated weights for policy 0, policy_version 732709 (0.0007) [2023-12-27 10:41:51,488][20381] Updated weights for policy 1, policy_version 732207 (0.0007) [2023-12-27 10:41:52,147][20347] Updated weights for policy 0, policy_version 732719 (0.0006) [2023-12-27 10:41:52,193][20381] Updated weights for policy 1, policy_version 732217 (0.0007) [2023-12-27 10:41:52,208][20347] Updated weights for policy 0, policy_version 732729 (0.0006) [2023-12-27 10:41:52,242][20381] Updated weights for policy 1, policy_version 732227 (0.0007) [2023-12-27 10:41:52,268][20347] Updated weights for policy 0, policy_version 732739 (0.0006) [2023-12-27 10:41:52,303][20381] Updated weights for policy 1, policy_version 732237 (0.0008) [2023-12-27 10:41:52,353][20381] Updated weights for policy 1, policy_version 732247 (0.0007) [2023-12-27 10:41:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 375087104. Throughput: 0: 10275.3, 1: 10238.6. Samples: 375050288. Policy #0 lag: (min: 28.0, avg: 28.2, max: 36.0) [2023-12-27 10:41:52,363][19170] Avg episode reward: [(0, '990.220'), (1, '575.160')] [2023-12-27 10:41:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000732248_187482112.pth... [2023-12-27 10:41:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000732744_187604992.pth... [2023-12-27 10:41:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000731032_187170816.pth [2023-12-27 10:41:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000731528_187293696.pth [2023-12-27 10:41:52,920][20347] Updated weights for policy 0, policy_version 732749 (0.0009) [2023-12-27 10:41:52,971][20347] Updated weights for policy 0, policy_version 732759 (0.0007) [2023-12-27 10:41:52,992][20381] Updated weights for policy 1, policy_version 732257 (0.0005) [2023-12-27 10:41:53,031][20347] Updated weights for policy 0, policy_version 732769 (0.0007) [2023-12-27 10:41:53,046][20381] Updated weights for policy 1, policy_version 732267 (0.0005) [2023-12-27 10:41:53,105][20381] Updated weights for policy 1, policy_version 732277 (0.0006) [2023-12-27 10:41:53,680][20381] Updated weights for policy 1, policy_version 732287 (0.0006) [2023-12-27 10:41:53,725][20347] Updated weights for policy 0, policy_version 732779 (0.0008) [2023-12-27 10:41:53,736][20381] Updated weights for policy 1, policy_version 732297 (0.0007) [2023-12-27 10:41:53,782][20347] Updated weights for policy 0, policy_version 732789 (0.0009) [2023-12-27 10:41:53,790][20381] Updated weights for policy 1, policy_version 732307 (0.0007) [2023-12-27 10:41:53,841][20347] Updated weights for policy 0, policy_version 732799 (0.0008) [2023-12-27 10:41:54,400][20381] Updated weights for policy 1, policy_version 732317 (0.0007) [2023-12-27 10:41:54,407][20347] Updated weights for policy 0, policy_version 732809 (0.0010) [2023-12-27 10:41:54,460][20347] Updated weights for policy 0, policy_version 732819 (0.0005) [2023-12-27 10:41:54,462][20381] Updated weights for policy 1, policy_version 732327 (0.0007) [2023-12-27 10:41:54,505][20347] Updated weights for policy 0, policy_version 732829 (0.0005) [2023-12-27 10:41:54,522][20381] Updated weights for policy 1, policy_version 732337 (0.0008) [2023-12-27 10:41:54,558][20347] Updated weights for policy 0, policy_version 732839 (0.0005) [2023-12-27 10:41:55,084][20381] Updated weights for policy 1, policy_version 732347 (0.0007) [2023-12-27 10:41:55,138][20381] Updated weights for policy 1, policy_version 732357 (0.0006) [2023-12-27 10:41:55,192][20381] Updated weights for policy 1, policy_version 732367 (0.0006) [2023-12-27 10:41:55,207][20347] Updated weights for policy 0, policy_version 732849 (0.0007) [2023-12-27 10:41:55,258][20347] Updated weights for policy 0, policy_version 732859 (0.0007) [2023-12-27 10:41:55,318][20347] Updated weights for policy 0, policy_version 732869 (0.0008) [2023-12-27 10:41:55,772][20381] Updated weights for policy 1, policy_version 732377 (0.0005) [2023-12-27 10:41:55,823][20381] Updated weights for policy 1, policy_version 732387 (0.0005) [2023-12-27 10:41:55,872][20381] Updated weights for policy 1, policy_version 732397 (0.0005) [2023-12-27 10:41:55,921][20381] Updated weights for policy 1, policy_version 732407 (0.0005) [2023-12-27 10:41:55,979][20347] Updated weights for policy 0, policy_version 732879 (0.0006) [2023-12-27 10:41:56,034][20347] Updated weights for policy 0, policy_version 732889 (0.0005) [2023-12-27 10:41:56,097][20347] Updated weights for policy 0, policy_version 732899 (0.0005) [2023-12-27 10:41:56,570][20381] Updated weights for policy 1, policy_version 732417 (0.0008) [2023-12-27 10:41:56,625][20381] Updated weights for policy 1, policy_version 732427 (0.0008) [2023-12-27 10:41:56,673][20381] Updated weights for policy 1, policy_version 732437 (0.0008) [2023-12-27 10:41:56,721][20347] Updated weights for policy 0, policy_version 732909 (0.0008) [2023-12-27 10:41:56,772][20347] Updated weights for policy 0, policy_version 732919 (0.0009) [2023-12-27 10:41:56,826][20347] Updated weights for policy 0, policy_version 732929 (0.0010) [2023-12-27 10:41:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 375185408. Throughput: 0: 10297.2, 1: 10300.8. Samples: 375178196. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:41:57,363][19170] Avg episode reward: [(0, '980.650'), (1, '561.830')] [2023-12-27 10:41:57,404][20381] Updated weights for policy 1, policy_version 732447 (0.0009) [2023-12-27 10:41:57,448][20347] Updated weights for policy 0, policy_version 732939 (0.0010) [2023-12-27 10:41:57,455][20381] Updated weights for policy 1, policy_version 732457 (0.0010) [2023-12-27 10:41:57,501][20347] Updated weights for policy 0, policy_version 732949 (0.0009) [2023-12-27 10:41:57,506][20381] Updated weights for policy 1, policy_version 732467 (0.0010) [2023-12-27 10:41:57,552][20347] Updated weights for policy 0, policy_version 732959 (0.0005) [2023-12-27 10:41:58,106][20347] Updated weights for policy 0, policy_version 732969 (0.0006) [2023-12-27 10:41:58,108][20381] Updated weights for policy 1, policy_version 732477 (0.0009) [2023-12-27 10:41:58,158][20381] Updated weights for policy 1, policy_version 732487 (0.0005) [2023-12-27 10:41:58,164][20347] Updated weights for policy 0, policy_version 732979 (0.0010) [2023-12-27 10:41:58,220][20381] Updated weights for policy 1, policy_version 732497 (0.0007) [2023-12-27 10:41:58,229][20347] Updated weights for policy 0, policy_version 732989 (0.0009) [2023-12-27 10:41:58,281][20347] Updated weights for policy 0, policy_version 732999 (0.0007) [2023-12-27 10:41:59,006][20381] Updated weights for policy 1, policy_version 732507 (0.0007) [2023-12-27 10:41:59,026][20347] Updated weights for policy 0, policy_version 733009 (0.0007) [2023-12-27 10:41:59,066][20381] Updated weights for policy 1, policy_version 732517 (0.0008) [2023-12-27 10:41:59,079][20347] Updated weights for policy 0, policy_version 733019 (0.0008) [2023-12-27 10:41:59,130][20381] Updated weights for policy 1, policy_version 732527 (0.0008) [2023-12-27 10:41:59,133][20347] Updated weights for policy 0, policy_version 733029 (0.0007) [2023-12-27 10:41:59,795][20381] Updated weights for policy 1, policy_version 732537 (0.0008) [2023-12-27 10:41:59,859][20381] Updated weights for policy 1, policy_version 732547 (0.0010) [2023-12-27 10:41:59,887][20347] Updated weights for policy 0, policy_version 733039 (0.0006) [2023-12-27 10:41:59,922][20381] Updated weights for policy 1, policy_version 732557 (0.0011) [2023-12-27 10:41:59,946][20347] Updated weights for policy 0, policy_version 733049 (0.0007) [2023-12-27 10:41:59,985][20381] Updated weights for policy 1, policy_version 732567 (0.0010) [2023-12-27 10:42:00,008][20347] Updated weights for policy 0, policy_version 733059 (0.0006) [2023-12-27 10:42:00,589][20347] Updated weights for policy 0, policy_version 733069 (0.0006) [2023-12-27 10:42:00,650][20347] Updated weights for policy 0, policy_version 733079 (0.0006) [2023-12-27 10:42:00,663][20381] Updated weights for policy 1, policy_version 732577 (0.0010) [2023-12-27 10:42:00,704][20347] Updated weights for policy 0, policy_version 733089 (0.0006) [2023-12-27 10:42:00,717][20381] Updated weights for policy 1, policy_version 732587 (0.0010) [2023-12-27 10:42:00,772][20381] Updated weights for policy 1, policy_version 732597 (0.0010) [2023-12-27 10:42:01,378][20347] Updated weights for policy 0, policy_version 733099 (0.0007) [2023-12-27 10:42:01,436][20347] Updated weights for policy 0, policy_version 733109 (0.0007) [2023-12-27 10:42:01,438][20381] Updated weights for policy 1, policy_version 732607 (0.0007) [2023-12-27 10:42:01,490][20381] Updated weights for policy 1, policy_version 732617 (0.0005) [2023-12-27 10:42:01,491][20347] Updated weights for policy 0, policy_version 733119 (0.0006) [2023-12-27 10:42:01,547][20381] Updated weights for policy 1, policy_version 732627 (0.0007) [2023-12-27 10:42:02,204][20381] Updated weights for policy 1, policy_version 732637 (0.0008) [2023-12-27 10:42:02,208][20347] Updated weights for policy 0, policy_version 733129 (0.0007) [2023-12-27 10:42:02,263][20347] Updated weights for policy 0, policy_version 733139 (0.0010) [2023-12-27 10:42:02,264][20381] Updated weights for policy 1, policy_version 732647 (0.0006) [2023-12-27 10:42:02,319][20381] Updated weights for policy 1, policy_version 732657 (0.0005) [2023-12-27 10:42:02,324][20347] Updated weights for policy 0, policy_version 733149 (0.0010) [2023-12-27 10:42:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 375291904. Throughput: 0: 10323.5, 1: 10297.5. Samples: 375302020. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:02,363][19170] Avg episode reward: [(0, '941.700'), (1, '457.310')] [2023-12-27 10:42:02,385][20347] Updated weights for policy 0, policy_version 733159 (0.0008) [2023-12-27 10:42:02,979][20381] Updated weights for policy 1, policy_version 732667 (0.0008) [2023-12-27 10:42:03,039][20381] Updated weights for policy 1, policy_version 732677 (0.0007) [2023-12-27 10:42:03,097][20381] Updated weights for policy 1, policy_version 732687 (0.0005) [2023-12-27 10:42:03,104][20347] Updated weights for policy 0, policy_version 733169 (0.0007) [2023-12-27 10:42:03,155][20347] Updated weights for policy 0, policy_version 733179 (0.0008) [2023-12-27 10:42:03,213][20347] Updated weights for policy 0, policy_version 733189 (0.0008) [2023-12-27 10:42:03,626][20381] Updated weights for policy 1, policy_version 732697 (0.0005) [2023-12-27 10:42:03,676][20381] Updated weights for policy 1, policy_version 732707 (0.0005) [2023-12-27 10:42:03,732][20381] Updated weights for policy 1, policy_version 732717 (0.0006) [2023-12-27 10:42:03,789][20381] Updated weights for policy 1, policy_version 732727 (0.0008) [2023-12-27 10:42:03,804][20347] Updated weights for policy 0, policy_version 733199 (0.0007) [2023-12-27 10:42:03,855][20347] Updated weights for policy 0, policy_version 733209 (0.0006) [2023-12-27 10:42:03,906][20347] Updated weights for policy 0, policy_version 733219 (0.0006) [2023-12-27 10:42:04,363][20381] Updated weights for policy 1, policy_version 732737 (0.0010) [2023-12-27 10:42:04,423][20381] Updated weights for policy 1, policy_version 732747 (0.0007) [2023-12-27 10:42:04,483][20381] Updated weights for policy 1, policy_version 732757 (0.0005) [2023-12-27 10:42:04,597][20347] Updated weights for policy 0, policy_version 733229 (0.0005) [2023-12-27 10:42:04,661][20347] Updated weights for policy 0, policy_version 733239 (0.0007) [2023-12-27 10:42:04,728][20347] Updated weights for policy 0, policy_version 733249 (0.0006) [2023-12-27 10:42:05,059][20381] Updated weights for policy 1, policy_version 732767 (0.0006) [2023-12-27 10:42:05,118][20381] Updated weights for policy 1, policy_version 732777 (0.0006) [2023-12-27 10:42:05,171][20381] Updated weights for policy 1, policy_version 732787 (0.0005) [2023-12-27 10:42:05,418][20347] Updated weights for policy 0, policy_version 733259 (0.0007) [2023-12-27 10:42:05,482][20347] Updated weights for policy 0, policy_version 733269 (0.0008) [2023-12-27 10:42:05,548][20347] Updated weights for policy 0, policy_version 733279 (0.0008) [2023-12-27 10:42:05,731][20381] Updated weights for policy 1, policy_version 732797 (0.0006) [2023-12-27 10:42:05,798][20381] Updated weights for policy 1, policy_version 732807 (0.0009) [2023-12-27 10:42:05,857][20381] Updated weights for policy 1, policy_version 732817 (0.0007) [2023-12-27 10:42:06,285][20347] Updated weights for policy 0, policy_version 733289 (0.0007) [2023-12-27 10:42:06,351][20347] Updated weights for policy 0, policy_version 733299 (0.0007) [2023-12-27 10:42:06,414][20347] Updated weights for policy 0, policy_version 733309 (0.0007) [2023-12-27 10:42:06,475][20347] Updated weights for policy 0, policy_version 733319 (0.0007) [2023-12-27 10:42:06,545][20381] Updated weights for policy 1, policy_version 732827 (0.0011) [2023-12-27 10:42:06,608][20381] Updated weights for policy 1, policy_version 732837 (0.0011) [2023-12-27 10:42:06,667][20381] Updated weights for policy 1, policy_version 732847 (0.0010) [2023-12-27 10:42:07,164][20347] Updated weights for policy 0, policy_version 733329 (0.0007) [2023-12-27 10:42:07,227][20347] Updated weights for policy 0, policy_version 733339 (0.0007) [2023-12-27 10:42:07,286][20347] Updated weights for policy 0, policy_version 733349 (0.0007) [2023-12-27 10:42:07,351][20381] Updated weights for policy 1, policy_version 732857 (0.0011) [2023-12-27 10:42:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 375398400. Throughput: 0: 10322.6, 1: 10418.4. Samples: 375367736. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:07,363][19170] Avg episode reward: [(0, '941.700'), (1, '638.960')] [2023-12-27 10:42:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000733352_187760640.pth... [2023-12-27 10:42:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000732136_187449344.pth [2023-12-27 10:42:07,405][20381] Updated weights for policy 1, policy_version 732867 (0.0010) [2023-12-27 10:42:07,459][20381] Updated weights for policy 1, policy_version 732877 (0.0011) [2023-12-27 10:42:07,514][20381] Updated weights for policy 1, policy_version 732887 (0.0008) [2023-12-27 10:42:07,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000732888_187645952.pth... [2023-12-27 10:42:07,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000731640_187326464.pth [2023-12-27 10:42:07,986][20347] Updated weights for policy 0, policy_version 733359 (0.0007) [2023-12-27 10:42:08,040][20347] Updated weights for policy 0, policy_version 733369 (0.0007) [2023-12-27 10:42:08,096][20347] Updated weights for policy 0, policy_version 733379 (0.0007) [2023-12-27 10:42:08,149][20381] Updated weights for policy 1, policy_version 732897 (0.0007) [2023-12-27 10:42:08,201][20381] Updated weights for policy 1, policy_version 732907 (0.0007) [2023-12-27 10:42:08,257][20381] Updated weights for policy 1, policy_version 732917 (0.0008) [2023-12-27 10:42:08,849][20347] Updated weights for policy 0, policy_version 733389 (0.0006) [2023-12-27 10:42:08,879][20381] Updated weights for policy 1, policy_version 732927 (0.0007) [2023-12-27 10:42:08,899][20347] Updated weights for policy 0, policy_version 733399 (0.0006) [2023-12-27 10:42:08,939][20381] Updated weights for policy 1, policy_version 732937 (0.0007) [2023-12-27 10:42:08,961][20347] Updated weights for policy 0, policy_version 733409 (0.0005) [2023-12-27 10:42:09,005][20381] Updated weights for policy 1, policy_version 732947 (0.0007) [2023-12-27 10:42:09,679][20347] Updated weights for policy 0, policy_version 733419 (0.0007) [2023-12-27 10:42:09,710][20381] Updated weights for policy 1, policy_version 732957 (0.0009) [2023-12-27 10:42:09,741][20347] Updated weights for policy 0, policy_version 733429 (0.0008) [2023-12-27 10:42:09,764][20381] Updated weights for policy 1, policy_version 732967 (0.0011) [2023-12-27 10:42:09,804][20347] Updated weights for policy 0, policy_version 733439 (0.0007) [2023-12-27 10:42:09,825][20381] Updated weights for policy 1, policy_version 732977 (0.0011) [2023-12-27 10:42:10,447][20347] Updated weights for policy 0, policy_version 733449 (0.0007) [2023-12-27 10:42:10,498][20347] Updated weights for policy 0, policy_version 733459 (0.0005) [2023-12-27 10:42:10,551][20347] Updated weights for policy 0, policy_version 733469 (0.0005) [2023-12-27 10:42:10,570][20381] Updated weights for policy 1, policy_version 732987 (0.0010) [2023-12-27 10:42:10,599][20347] Updated weights for policy 0, policy_version 733479 (0.0005) [2023-12-27 10:42:10,622][20381] Updated weights for policy 1, policy_version 732997 (0.0006) [2023-12-27 10:42:10,672][20381] Updated weights for policy 1, policy_version 733007 (0.0005) [2023-12-27 10:42:11,294][20347] Updated weights for policy 0, policy_version 733489 (0.0007) [2023-12-27 10:42:11,351][20347] Updated weights for policy 0, policy_version 733499 (0.0008) [2023-12-27 10:42:11,388][20381] Updated weights for policy 1, policy_version 733017 (0.0007) [2023-12-27 10:42:11,418][20347] Updated weights for policy 0, policy_version 733509 (0.0009) [2023-12-27 10:42:11,452][20381] Updated weights for policy 1, policy_version 733027 (0.0006) [2023-12-27 10:42:11,515][20381] Updated weights for policy 1, policy_version 733037 (0.0008) [2023-12-27 10:42:11,583][20381] Updated weights for policy 1, policy_version 733047 (0.0007) [2023-12-27 10:42:12,180][20347] Updated weights for policy 0, policy_version 733519 (0.0008) [2023-12-27 10:42:12,214][20381] Updated weights for policy 1, policy_version 733057 (0.0007) [2023-12-27 10:42:12,243][20347] Updated weights for policy 0, policy_version 733529 (0.0006) [2023-12-27 10:42:12,265][20381] Updated weights for policy 1, policy_version 733067 (0.0006) [2023-12-27 10:42:12,308][20347] Updated weights for policy 0, policy_version 733539 (0.0006) [2023-12-27 10:42:12,328][20381] Updated weights for policy 1, policy_version 733077 (0.0006) [2023-12-27 10:42:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 375504896. Throughput: 0: 10272.5, 1: 10405.9. Samples: 375487740. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:12,363][19170] Avg episode reward: [(0, '980.460'), (1, '715.440')] [2023-12-27 10:42:12,926][20381] Updated weights for policy 1, policy_version 733087 (0.0007) [2023-12-27 10:42:12,976][20381] Updated weights for policy 1, policy_version 733097 (0.0006) [2023-12-27 10:42:13,028][20381] Updated weights for policy 1, policy_version 733107 (0.0005) [2023-12-27 10:42:13,065][20347] Updated weights for policy 0, policy_version 733549 (0.0007) [2023-12-27 10:42:13,120][20347] Updated weights for policy 0, policy_version 733559 (0.0008) [2023-12-27 10:42:13,170][20347] Updated weights for policy 0, policy_version 733569 (0.0007) [2023-12-27 10:42:13,701][20381] Updated weights for policy 1, policy_version 733117 (0.0005) [2023-12-27 10:42:13,762][20381] Updated weights for policy 1, policy_version 733127 (0.0005) [2023-12-27 10:42:13,810][20381] Updated weights for policy 1, policy_version 733137 (0.0005) [2023-12-27 10:42:13,887][20347] Updated weights for policy 0, policy_version 733579 (0.0008) [2023-12-27 10:42:13,936][20291] KL-divergence is very high: 211.1298 [2023-12-27 10:42:13,942][20347] Updated weights for policy 0, policy_version 733589 (0.0008) [2023-12-27 10:42:13,948][20291] KL-divergence is very high: 195.4503 [2023-12-27 10:42:13,999][20347] Updated weights for policy 0, policy_version 733599 (0.0006) [2023-12-27 10:42:14,436][20381] Updated weights for policy 1, policy_version 733147 (0.0006) [2023-12-27 10:42:14,482][20381] Updated weights for policy 1, policy_version 733157 (0.0008) [2023-12-27 10:42:14,526][20381] Updated weights for policy 1, policy_version 733167 (0.0007) [2023-12-27 10:42:14,671][20347] Updated weights for policy 0, policy_version 733609 (0.0008) [2023-12-27 10:42:14,735][20347] Updated weights for policy 0, policy_version 733619 (0.0008) [2023-12-27 10:42:14,798][20347] Updated weights for policy 0, policy_version 733629 (0.0007) [2023-12-27 10:42:14,858][20347] Updated weights for policy 0, policy_version 733639 (0.0008) [2023-12-27 10:42:15,228][20381] Updated weights for policy 1, policy_version 733177 (0.0007) [2023-12-27 10:42:15,282][20381] Updated weights for policy 1, policy_version 733187 (0.0008) [2023-12-27 10:42:15,345][20381] Updated weights for policy 1, policy_version 733197 (0.0008) [2023-12-27 10:42:15,396][20381] Updated weights for policy 1, policy_version 733207 (0.0007) [2023-12-27 10:42:15,522][20347] Updated weights for policy 0, policy_version 733649 (0.0007) [2023-12-27 10:42:15,573][20347] Updated weights for policy 0, policy_version 733659 (0.0007) [2023-12-27 10:42:15,627][20347] Updated weights for policy 0, policy_version 733669 (0.0007) [2023-12-27 10:42:16,100][20381] Updated weights for policy 1, policy_version 733217 (0.0007) [2023-12-27 10:42:16,145][20381] Updated weights for policy 1, policy_version 733227 (0.0008) [2023-12-27 10:42:16,205][20381] Updated weights for policy 1, policy_version 733237 (0.0008) [2023-12-27 10:42:16,275][20347] Updated weights for policy 0, policy_version 733679 (0.0006) [2023-12-27 10:42:16,338][20347] Updated weights for policy 0, policy_version 733689 (0.0006) [2023-12-27 10:42:16,393][20347] Updated weights for policy 0, policy_version 733699 (0.0007) [2023-12-27 10:42:16,971][20347] Updated weights for policy 0, policy_version 733709 (0.0006) [2023-12-27 10:42:16,983][20381] Updated weights for policy 1, policy_version 733247 (0.0007) [2023-12-27 10:42:17,026][20347] Updated weights for policy 0, policy_version 733719 (0.0005) [2023-12-27 10:42:17,040][20381] Updated weights for policy 1, policy_version 733257 (0.0007) [2023-12-27 10:42:17,079][20347] Updated weights for policy 0, policy_version 733729 (0.0006) [2023-12-27 10:42:17,095][20381] Updated weights for policy 1, policy_version 733267 (0.0006) [2023-12-27 10:42:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 375603200. Throughput: 0: 10164.5, 1: 10421.2. Samples: 375609552. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:17,363][19170] Avg episode reward: [(0, '951.330'), (1, '704.330')] [2023-12-27 10:42:17,647][20347] Updated weights for policy 0, policy_version 733739 (0.0006) [2023-12-27 10:42:17,703][20347] Updated weights for policy 0, policy_version 733749 (0.0005) [2023-12-27 10:42:17,764][20347] Updated weights for policy 0, policy_version 733759 (0.0005) [2023-12-27 10:42:17,828][20381] Updated weights for policy 1, policy_version 733277 (0.0007) [2023-12-27 10:42:17,879][20381] Updated weights for policy 1, policy_version 733287 (0.0008) [2023-12-27 10:42:17,935][20381] Updated weights for policy 1, policy_version 733297 (0.0008) [2023-12-27 10:42:18,322][20347] Updated weights for policy 0, policy_version 733769 (0.0006) [2023-12-27 10:42:18,387][20347] Updated weights for policy 0, policy_version 733779 (0.0007) [2023-12-27 10:42:18,442][20347] Updated weights for policy 0, policy_version 733789 (0.0007) [2023-12-27 10:42:18,499][20347] Updated weights for policy 0, policy_version 733799 (0.0005) [2023-12-27 10:42:18,695][20381] Updated weights for policy 1, policy_version 733307 (0.0008) [2023-12-27 10:42:18,753][20381] Updated weights for policy 1, policy_version 733317 (0.0008) [2023-12-27 10:42:18,809][20381] Updated weights for policy 1, policy_version 733327 (0.0008) [2023-12-27 10:42:19,147][20347] Updated weights for policy 0, policy_version 733809 (0.0007) [2023-12-27 10:42:19,203][20347] Updated weights for policy 0, policy_version 733819 (0.0007) [2023-12-27 10:42:19,272][20347] Updated weights for policy 0, policy_version 733829 (0.0008) [2023-12-27 10:42:19,563][20381] Updated weights for policy 1, policy_version 733337 (0.0008) [2023-12-27 10:42:19,617][20381] Updated weights for policy 1, policy_version 733347 (0.0007) [2023-12-27 10:42:19,672][20381] Updated weights for policy 1, policy_version 733357 (0.0007) [2023-12-27 10:42:19,723][20381] Updated weights for policy 1, policy_version 733367 (0.0007) [2023-12-27 10:42:20,058][20347] Updated weights for policy 0, policy_version 733839 (0.0008) [2023-12-27 10:42:20,119][20347] Updated weights for policy 0, policy_version 733849 (0.0008) [2023-12-27 10:42:20,181][20347] Updated weights for policy 0, policy_version 733859 (0.0008) [2023-12-27 10:42:20,416][20381] Updated weights for policy 1, policy_version 733377 (0.0008) [2023-12-27 10:42:20,470][20381] Updated weights for policy 1, policy_version 733387 (0.0008) [2023-12-27 10:42:20,515][20381] Updated weights for policy 1, policy_version 733397 (0.0008) [2023-12-27 10:42:20,905][20347] Updated weights for policy 0, policy_version 733869 (0.0008) [2023-12-27 10:42:20,962][20347] Updated weights for policy 0, policy_version 733879 (0.0008) [2023-12-27 10:42:21,026][20347] Updated weights for policy 0, policy_version 733889 (0.0007) [2023-12-27 10:42:21,291][20381] Updated weights for policy 1, policy_version 733407 (0.0007) [2023-12-27 10:42:21,359][20381] Updated weights for policy 1, policy_version 733417 (0.0008) [2023-12-27 10:42:21,430][20381] Updated weights for policy 1, policy_version 733427 (0.0007) [2023-12-27 10:42:21,750][20347] Updated weights for policy 0, policy_version 733899 (0.0007) [2023-12-27 10:42:21,820][20347] Updated weights for policy 0, policy_version 733909 (0.0008) [2023-12-27 10:42:21,888][20347] Updated weights for policy 0, policy_version 733919 (0.0008) [2023-12-27 10:42:22,082][20381] Updated weights for policy 1, policy_version 733437 (0.0007) [2023-12-27 10:42:22,141][20381] Updated weights for policy 1, policy_version 733447 (0.0006) [2023-12-27 10:42:22,207][20381] Updated weights for policy 1, policy_version 733457 (0.0006) [2023-12-27 10:42:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 375701504. Throughput: 0: 10205.7, 1: 10338.6. Samples: 375670408. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:22,363][19170] Avg episode reward: [(0, '970.770'), (1, '756.930')] [2023-12-27 10:42:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000733464_187793408.pth... [2023-12-27 10:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000733928_187908096.pth... [2023-12-27 10:42:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000732248_187482112.pth [2023-12-27 10:42:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000732744_187604992.pth [2023-12-27 10:42:22,583][20347] Updated weights for policy 0, policy_version 733929 (0.0008) [2023-12-27 10:42:22,643][20347] Updated weights for policy 0, policy_version 733939 (0.0008) [2023-12-27 10:42:22,697][20347] Updated weights for policy 0, policy_version 733949 (0.0008) [2023-12-27 10:42:22,759][20347] Updated weights for policy 0, policy_version 733959 (0.0007) [2023-12-27 10:42:22,831][20381] Updated weights for policy 1, policy_version 733467 (0.0008) [2023-12-27 10:42:22,897][20381] Updated weights for policy 1, policy_version 733477 (0.0009) [2023-12-27 10:42:22,955][20381] Updated weights for policy 1, policy_version 733487 (0.0007) [2023-12-27 10:42:23,346][20347] Updated weights for policy 0, policy_version 733969 (0.0006) [2023-12-27 10:42:23,395][20347] Updated weights for policy 0, policy_version 733979 (0.0007) [2023-12-27 10:42:23,443][20347] Updated weights for policy 0, policy_version 733989 (0.0007) [2023-12-27 10:42:23,584][20381] Updated weights for policy 1, policy_version 733497 (0.0006) [2023-12-27 10:42:23,633][20381] Updated weights for policy 1, policy_version 733507 (0.0008) [2023-12-27 10:42:23,685][20381] Updated weights for policy 1, policy_version 733517 (0.0006) [2023-12-27 10:42:23,728][20381] Updated weights for policy 1, policy_version 733527 (0.0008) [2023-12-27 10:42:24,106][20347] Updated weights for policy 0, policy_version 733999 (0.0008) [2023-12-27 10:42:24,159][20347] Updated weights for policy 0, policy_version 734009 (0.0007) [2023-12-27 10:42:24,208][20347] Updated weights for policy 0, policy_version 734019 (0.0007) [2023-12-27 10:42:24,424][20381] Updated weights for policy 1, policy_version 733537 (0.0006) [2023-12-27 10:42:24,479][20381] Updated weights for policy 1, policy_version 733547 (0.0006) [2023-12-27 10:42:24,531][20381] Updated weights for policy 1, policy_version 733557 (0.0005) [2023-12-27 10:42:24,869][20347] Updated weights for policy 0, policy_version 734029 (0.0008) [2023-12-27 10:42:24,928][20347] Updated weights for policy 0, policy_version 734039 (0.0006) [2023-12-27 10:42:24,986][20347] Updated weights for policy 0, policy_version 734049 (0.0006) [2023-12-27 10:42:25,092][20381] Updated weights for policy 1, policy_version 733567 (0.0005) [2023-12-27 10:42:25,150][20381] Updated weights for policy 1, policy_version 733577 (0.0005) [2023-12-27 10:42:25,206][20381] Updated weights for policy 1, policy_version 733587 (0.0005) [2023-12-27 10:42:25,698][20347] Updated weights for policy 0, policy_version 734059 (0.0007) [2023-12-27 10:42:25,753][20347] Updated weights for policy 0, policy_version 734069 (0.0007) [2023-12-27 10:42:25,755][20381] Updated weights for policy 1, policy_version 733597 (0.0006) [2023-12-27 10:42:25,804][20381] Updated weights for policy 1, policy_version 733607 (0.0005) [2023-12-27 10:42:25,809][20347] Updated weights for policy 0, policy_version 734079 (0.0006) [2023-12-27 10:42:25,859][20381] Updated weights for policy 1, policy_version 733617 (0.0007) [2023-12-27 10:42:26,432][20347] Updated weights for policy 0, policy_version 734089 (0.0008) [2023-12-27 10:42:26,488][20347] Updated weights for policy 0, policy_version 734099 (0.0005) [2023-12-27 10:42:26,515][20381] Updated weights for policy 1, policy_version 733627 (0.0007) [2023-12-27 10:42:26,551][20347] Updated weights for policy 0, policy_version 734109 (0.0006) [2023-12-27 10:42:26,565][20381] Updated weights for policy 1, policy_version 733637 (0.0008) [2023-12-27 10:42:26,608][20347] Updated weights for policy 0, policy_version 734119 (0.0006) [2023-12-27 10:42:26,622][20381] Updated weights for policy 1, policy_version 733647 (0.0008) [2023-12-27 10:42:27,220][20347] Updated weights for policy 0, policy_version 734129 (0.0007) [2023-12-27 10:42:27,273][20347] Updated weights for policy 0, policy_version 734139 (0.0007) [2023-12-27 10:42:27,330][20347] Updated weights for policy 0, policy_version 734149 (0.0009) [2023-12-27 10:42:27,339][20381] Updated weights for policy 1, policy_version 733657 (0.0009) [2023-12-27 10:42:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 375808000. Throughput: 0: 10214.3, 1: 10425.2. Samples: 375794832. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:27,363][19170] Avg episode reward: [(0, '980.690'), (1, '785.400')] [2023-12-27 10:42:27,386][20381] Updated weights for policy 1, policy_version 733667 (0.0007) [2023-12-27 10:42:27,433][20381] Updated weights for policy 1, policy_version 733677 (0.0007) [2023-12-27 10:42:27,479][20381] Updated weights for policy 1, policy_version 733687 (0.0007) [2023-12-27 10:42:27,931][20347] Updated weights for policy 0, policy_version 734159 (0.0007) [2023-12-27 10:42:27,986][20347] Updated weights for policy 0, policy_version 734169 (0.0006) [2023-12-27 10:42:28,041][20347] Updated weights for policy 0, policy_version 734179 (0.0006) [2023-12-27 10:42:28,262][20381] Updated weights for policy 1, policy_version 733697 (0.0007) [2023-12-27 10:42:28,323][20381] Updated weights for policy 1, policy_version 733707 (0.0007) [2023-12-27 10:42:28,386][20381] Updated weights for policy 1, policy_version 733717 (0.0011) [2023-12-27 10:42:28,627][20347] Updated weights for policy 0, policy_version 734189 (0.0006) [2023-12-27 10:42:28,682][20347] Updated weights for policy 0, policy_version 734199 (0.0007) [2023-12-27 10:42:28,743][20347] Updated weights for policy 0, policy_version 734209 (0.0008) [2023-12-27 10:42:29,059][20381] Updated weights for policy 1, policy_version 733727 (0.0008) [2023-12-27 10:42:29,115][20381] Updated weights for policy 1, policy_version 733737 (0.0008) [2023-12-27 10:42:29,174][20381] Updated weights for policy 1, policy_version 733747 (0.0007) [2023-12-27 10:42:29,389][20347] Updated weights for policy 0, policy_version 734219 (0.0008) [2023-12-27 10:42:29,438][20347] Updated weights for policy 0, policy_version 734229 (0.0008) [2023-12-27 10:42:29,485][20347] Updated weights for policy 0, policy_version 734239 (0.0007) [2023-12-27 10:42:29,845][20381] Updated weights for policy 1, policy_version 733757 (0.0008) [2023-12-27 10:42:29,911][20381] Updated weights for policy 1, policy_version 733767 (0.0007) [2023-12-27 10:42:29,962][20381] Updated weights for policy 1, policy_version 733777 (0.0009) [2023-12-27 10:42:30,234][20347] Updated weights for policy 0, policy_version 734249 (0.0009) [2023-12-27 10:42:30,292][20347] Updated weights for policy 0, policy_version 734259 (0.0010) [2023-12-27 10:42:30,347][20347] Updated weights for policy 0, policy_version 734269 (0.0010) [2023-12-27 10:42:30,399][20347] Updated weights for policy 0, policy_version 734279 (0.0010) [2023-12-27 10:42:30,645][20381] Updated weights for policy 1, policy_version 733787 (0.0007) [2023-12-27 10:42:30,712][20381] Updated weights for policy 1, policy_version 733797 (0.0007) [2023-12-27 10:42:30,769][20381] Updated weights for policy 1, policy_version 733807 (0.0008) [2023-12-27 10:42:31,091][20347] Updated weights for policy 0, policy_version 734289 (0.0008) [2023-12-27 10:42:31,151][20347] Updated weights for policy 0, policy_version 734299 (0.0006) [2023-12-27 10:42:31,211][20347] Updated weights for policy 0, policy_version 734309 (0.0007) [2023-12-27 10:42:31,466][20381] Updated weights for policy 1, policy_version 733817 (0.0007) [2023-12-27 10:42:31,524][20381] Updated weights for policy 1, policy_version 733827 (0.0008) [2023-12-27 10:42:31,573][20381] Updated weights for policy 1, policy_version 733837 (0.0007) [2023-12-27 10:42:31,632][20381] Updated weights for policy 1, policy_version 733847 (0.0007) [2023-12-27 10:42:31,912][20347] Updated weights for policy 0, policy_version 734319 (0.0008) [2023-12-27 10:42:31,964][20347] Updated weights for policy 0, policy_version 734329 (0.0009) [2023-12-27 10:42:32,024][20347] Updated weights for policy 0, policy_version 734339 (0.0008) [2023-12-27 10:42:32,297][20381] Updated weights for policy 1, policy_version 733857 (0.0008) [2023-12-27 10:42:32,355][20381] Updated weights for policy 1, policy_version 733867 (0.0007) [2023-12-27 10:42:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 375906304. Throughput: 0: 10246.5, 1: 10416.5. Samples: 375918588. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:32,363][19170] Avg episode reward: [(0, '883.560'), (1, '734.930')] [2023-12-27 10:42:32,414][20381] Updated weights for policy 1, policy_version 733877 (0.0009) [2023-12-27 10:42:32,677][20347] Updated weights for policy 0, policy_version 734349 (0.0007) [2023-12-27 10:42:32,745][20347] Updated weights for policy 0, policy_version 734359 (0.0008) [2023-12-27 10:42:32,813][20347] Updated weights for policy 0, policy_version 734369 (0.0011) [2023-12-27 10:42:33,019][20381] Updated weights for policy 1, policy_version 733887 (0.0009) [2023-12-27 10:42:33,068][20381] Updated weights for policy 1, policy_version 733897 (0.0009) [2023-12-27 10:42:33,115][20381] Updated weights for policy 1, policy_version 733907 (0.0007) [2023-12-27 10:42:33,398][20347] Updated weights for policy 0, policy_version 734379 (0.0006) [2023-12-27 10:42:33,453][20347] Updated weights for policy 0, policy_version 734389 (0.0006) [2023-12-27 10:42:33,509][20347] Updated weights for policy 0, policy_version 734399 (0.0010) [2023-12-27 10:42:33,815][20381] Updated weights for policy 1, policy_version 733917 (0.0007) [2023-12-27 10:42:33,859][20381] Updated weights for policy 1, policy_version 733927 (0.0007) [2023-12-27 10:42:33,911][20381] Updated weights for policy 1, policy_version 733937 (0.0007) [2023-12-27 10:42:34,130][20347] Updated weights for policy 0, policy_version 734409 (0.0010) [2023-12-27 10:42:34,192][20347] Updated weights for policy 0, policy_version 734419 (0.0011) [2023-12-27 10:42:34,254][20347] Updated weights for policy 0, policy_version 734429 (0.0010) [2023-12-27 10:42:34,316][20347] Updated weights for policy 0, policy_version 734439 (0.0010) [2023-12-27 10:42:34,573][20381] Updated weights for policy 1, policy_version 733947 (0.0006) [2023-12-27 10:42:34,636][20381] Updated weights for policy 1, policy_version 733957 (0.0006) [2023-12-27 10:42:34,696][20381] Updated weights for policy 1, policy_version 733967 (0.0007) [2023-12-27 10:42:34,963][20347] Updated weights for policy 0, policy_version 734449 (0.0010) [2023-12-27 10:42:35,012][20347] Updated weights for policy 0, policy_version 734459 (0.0010) [2023-12-27 10:42:35,068][20347] Updated weights for policy 0, policy_version 734469 (0.0007) [2023-12-27 10:42:35,336][20381] Updated weights for policy 1, policy_version 733977 (0.0007) [2023-12-27 10:42:35,395][20381] Updated weights for policy 1, policy_version 733987 (0.0008) [2023-12-27 10:42:35,449][20381] Updated weights for policy 1, policy_version 733997 (0.0009) [2023-12-27 10:42:35,502][20381] Updated weights for policy 1, policy_version 734007 (0.0008) [2023-12-27 10:42:35,654][20347] Updated weights for policy 0, policy_version 734479 (0.0005) [2023-12-27 10:42:35,714][20347] Updated weights for policy 0, policy_version 734489 (0.0005) [2023-12-27 10:42:35,770][20347] Updated weights for policy 0, policy_version 734499 (0.0005) [2023-12-27 10:42:36,210][20381] Updated weights for policy 1, policy_version 734017 (0.0010) [2023-12-27 10:42:36,269][20381] Updated weights for policy 1, policy_version 734027 (0.0011) [2023-12-27 10:42:36,335][20381] Updated weights for policy 1, policy_version 734037 (0.0011) [2023-12-27 10:42:36,361][20347] Updated weights for policy 0, policy_version 734509 (0.0008) [2023-12-27 10:42:36,413][20347] Updated weights for policy 0, policy_version 734519 (0.0010) [2023-12-27 10:42:36,466][20347] Updated weights for policy 0, policy_version 734529 (0.0010) [2023-12-27 10:42:37,011][20381] Updated weights for policy 1, policy_version 734047 (0.0007) [2023-12-27 10:42:37,066][20381] Updated weights for policy 1, policy_version 734057 (0.0005) [2023-12-27 10:42:37,130][20381] Updated weights for policy 1, policy_version 734067 (0.0007) [2023-12-27 10:42:37,162][20347] Updated weights for policy 0, policy_version 734539 (0.0011) [2023-12-27 10:42:37,218][20347] Updated weights for policy 0, policy_version 734549 (0.0010) [2023-12-27 10:42:37,274][20347] Updated weights for policy 0, policy_version 734559 (0.0011) [2023-12-27 10:42:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 376020992. Throughput: 0: 10303.0, 1: 10420.3. Samples: 375982836. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:37,363][19170] Avg episode reward: [(0, '854.160'), (1, '747.850')] [2023-12-27 10:42:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000734072_187949056.pth... [2023-12-27 10:42:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000734568_188071936.pth... [2023-12-27 10:42:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000733352_187760640.pth [2023-12-27 10:42:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000732888_187645952.pth [2023-12-27 10:42:37,815][20381] Updated weights for policy 1, policy_version 734077 (0.0008) [2023-12-27 10:42:37,871][20381] Updated weights for policy 1, policy_version 734087 (0.0008) [2023-12-27 10:42:37,933][20381] Updated weights for policy 1, policy_version 734097 (0.0007) [2023-12-27 10:42:37,976][20347] Updated weights for policy 0, policy_version 734569 (0.0011) [2023-12-27 10:42:38,035][20347] Updated weights for policy 0, policy_version 734579 (0.0010) [2023-12-27 10:42:38,096][20347] Updated weights for policy 0, policy_version 734589 (0.0010) [2023-12-27 10:42:38,151][20347] Updated weights for policy 0, policy_version 734599 (0.0010) [2023-12-27 10:42:38,627][20381] Updated weights for policy 1, policy_version 734107 (0.0009) [2023-12-27 10:42:38,691][20381] Updated weights for policy 1, policy_version 734117 (0.0011) [2023-12-27 10:42:38,746][20381] Updated weights for policy 1, policy_version 734127 (0.0011) [2023-12-27 10:42:38,837][20347] Updated weights for policy 0, policy_version 734609 (0.0006) [2023-12-27 10:42:38,899][20347] Updated weights for policy 0, policy_version 734619 (0.0007) [2023-12-27 10:42:38,961][20347] Updated weights for policy 0, policy_version 734629 (0.0005) [2023-12-27 10:42:39,397][20381] Updated weights for policy 1, policy_version 734137 (0.0011) [2023-12-27 10:42:39,462][20381] Updated weights for policy 1, policy_version 734147 (0.0006) [2023-12-27 10:42:39,523][20381] Updated weights for policy 1, policy_version 734157 (0.0006) [2023-12-27 10:42:39,581][20381] Updated weights for policy 1, policy_version 734167 (0.0007) [2023-12-27 10:42:39,645][20347] Updated weights for policy 0, policy_version 734639 (0.0007) [2023-12-27 10:42:39,695][20347] Updated weights for policy 0, policy_version 734649 (0.0008) [2023-12-27 10:42:39,747][20347] Updated weights for policy 0, policy_version 734659 (0.0008) [2023-12-27 10:42:40,315][20381] Updated weights for policy 1, policy_version 734177 (0.0007) [2023-12-27 10:42:40,375][20381] Updated weights for policy 1, policy_version 734187 (0.0007) [2023-12-27 10:42:40,429][20381] Updated weights for policy 1, policy_version 734197 (0.0008) [2023-12-27 10:42:40,468][20347] Updated weights for policy 0, policy_version 734669 (0.0007) [2023-12-27 10:42:40,530][20347] Updated weights for policy 0, policy_version 734679 (0.0008) [2023-12-27 10:42:40,578][20347] Updated weights for policy 0, policy_version 734689 (0.0008) [2023-12-27 10:42:40,994][20381] Updated weights for policy 1, policy_version 734207 (0.0006) [2023-12-27 10:42:41,059][20381] Updated weights for policy 1, policy_version 734217 (0.0007) [2023-12-27 10:42:41,116][20381] Updated weights for policy 1, policy_version 734227 (0.0006) [2023-12-27 10:42:41,360][20347] Updated weights for policy 0, policy_version 734699 (0.0008) [2023-12-27 10:42:41,426][20347] Updated weights for policy 0, policy_version 734709 (0.0008) [2023-12-27 10:42:41,479][20347] Updated weights for policy 0, policy_version 734719 (0.0008) [2023-12-27 10:42:41,855][20381] Updated weights for policy 1, policy_version 734237 (0.0007) [2023-12-27 10:42:41,907][20381] Updated weights for policy 1, policy_version 734247 (0.0008) [2023-12-27 10:42:41,957][20381] Updated weights for policy 1, policy_version 734257 (0.0007) [2023-12-27 10:42:42,223][20347] Updated weights for policy 0, policy_version 734729 (0.0007) [2023-12-27 10:42:42,277][20347] Updated weights for policy 0, policy_version 734739 (0.0008) [2023-12-27 10:42:42,337][20347] Updated weights for policy 0, policy_version 734749 (0.0006) [2023-12-27 10:42:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 376111104. Throughput: 0: 10285.5, 1: 10333.9. Samples: 376106072. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:42,363][19170] Avg episode reward: [(0, '921.970'), (1, '761.920')] [2023-12-27 10:42:42,403][20347] Updated weights for policy 0, policy_version 734759 (0.0008) [2023-12-27 10:42:42,658][20381] Updated weights for policy 1, policy_version 734267 (0.0008) [2023-12-27 10:42:42,716][20381] Updated weights for policy 1, policy_version 734277 (0.0007) [2023-12-27 10:42:42,780][20381] Updated weights for policy 1, policy_version 734287 (0.0008) [2023-12-27 10:42:43,011][20347] Updated weights for policy 0, policy_version 734769 (0.0005) [2023-12-27 10:42:43,072][20347] Updated weights for policy 0, policy_version 734779 (0.0007) [2023-12-27 10:42:43,128][20347] Updated weights for policy 0, policy_version 734789 (0.0008) [2023-12-27 10:42:43,421][20381] Updated weights for policy 1, policy_version 734297 (0.0007) [2023-12-27 10:42:43,483][20381] Updated weights for policy 1, policy_version 734307 (0.0008) [2023-12-27 10:42:43,528][20381] Updated weights for policy 1, policy_version 734317 (0.0007) [2023-12-27 10:42:43,583][20381] Updated weights for policy 1, policy_version 734327 (0.0007) [2023-12-27 10:42:43,774][20347] Updated weights for policy 0, policy_version 734799 (0.0006) [2023-12-27 10:42:43,828][20347] Updated weights for policy 0, policy_version 734809 (0.0007) [2023-12-27 10:42:43,893][20347] Updated weights for policy 0, policy_version 734819 (0.0008) [2023-12-27 10:42:44,271][20381] Updated weights for policy 1, policy_version 734337 (0.0008) [2023-12-27 10:42:44,327][20381] Updated weights for policy 1, policy_version 734347 (0.0008) [2023-12-27 10:42:44,382][20381] Updated weights for policy 1, policy_version 734357 (0.0006) [2023-12-27 10:42:44,578][20347] Updated weights for policy 0, policy_version 734829 (0.0007) [2023-12-27 10:42:44,637][20347] Updated weights for policy 0, policy_version 734839 (0.0007) [2023-12-27 10:42:44,684][20347] Updated weights for policy 0, policy_version 734849 (0.0007) [2023-12-27 10:42:45,057][20381] Updated weights for policy 1, policy_version 734367 (0.0007) [2023-12-27 10:42:45,111][20381] Updated weights for policy 1, policy_version 734377 (0.0007) [2023-12-27 10:42:45,172][20381] Updated weights for policy 1, policy_version 734387 (0.0008) [2023-12-27 10:42:45,371][20347] Updated weights for policy 0, policy_version 734859 (0.0009) [2023-12-27 10:42:45,422][20347] Updated weights for policy 0, policy_version 734869 (0.0007) [2023-12-27 10:42:45,473][20347] Updated weights for policy 0, policy_version 734879 (0.0007) [2023-12-27 10:42:45,866][20381] Updated weights for policy 1, policy_version 734397 (0.0007) [2023-12-27 10:42:45,924][20381] Updated weights for policy 1, policy_version 734407 (0.0008) [2023-12-27 10:42:45,983][20381] Updated weights for policy 1, policy_version 734417 (0.0008) [2023-12-27 10:42:46,154][20347] Updated weights for policy 0, policy_version 734889 (0.0008) [2023-12-27 10:42:46,206][20347] Updated weights for policy 0, policy_version 734899 (0.0008) [2023-12-27 10:42:46,255][20347] Updated weights for policy 0, policy_version 734909 (0.0007) [2023-12-27 10:42:46,301][20347] Updated weights for policy 0, policy_version 734919 (0.0007) [2023-12-27 10:42:46,642][20381] Updated weights for policy 1, policy_version 734427 (0.0008) [2023-12-27 10:42:46,690][20381] Updated weights for policy 1, policy_version 734437 (0.0010) [2023-12-27 10:42:46,735][20381] Updated weights for policy 1, policy_version 734447 (0.0010) [2023-12-27 10:42:46,957][20347] Updated weights for policy 0, policy_version 734929 (0.0006) [2023-12-27 10:42:47,006][20347] Updated weights for policy 0, policy_version 734939 (0.0009) [2023-12-27 10:42:47,059][20347] Updated weights for policy 0, policy_version 734949 (0.0009) [2023-12-27 10:42:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 376217600. Throughput: 0: 10267.7, 1: 10334.2. Samples: 376229108. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:47,363][19170] Avg episode reward: [(0, '970.800'), (1, '707.790')] [2023-12-27 10:42:47,371][20381] Updated weights for policy 1, policy_version 734457 (0.0011) [2023-12-27 10:42:47,433][20381] Updated weights for policy 1, policy_version 734467 (0.0008) [2023-12-27 10:42:47,447][20292] KL-divergence is very high: 277.6464 [2023-12-27 10:42:47,461][20292] KL-divergence is very high: 301.8236 [2023-12-27 10:42:47,498][20292] KL-divergence is very high: 210.0426 [2023-12-27 10:42:47,499][20381] Updated weights for policy 1, policy_version 734477 (0.0008) [2023-12-27 10:42:47,508][20292] KL-divergence is very high: 169.6954 [2023-12-27 10:42:47,549][20381] Updated weights for policy 1, policy_version 734487 (0.0010) [2023-12-27 10:42:47,650][20347] Updated weights for policy 0, policy_version 734959 (0.0008) [2023-12-27 10:42:47,696][20347] Updated weights for policy 0, policy_version 734969 (0.0007) [2023-12-27 10:42:47,748][20347] Updated weights for policy 0, policy_version 734979 (0.0007) [2023-12-27 10:42:48,187][20381] Updated weights for policy 1, policy_version 734497 (0.0011) [2023-12-27 10:42:48,232][20381] Updated weights for policy 1, policy_version 734507 (0.0010) [2023-12-27 10:42:48,278][20381] Updated weights for policy 1, policy_version 734517 (0.0010) [2023-12-27 10:42:48,379][20347] Updated weights for policy 0, policy_version 734989 (0.0006) [2023-12-27 10:42:48,437][20347] Updated weights for policy 0, policy_version 734999 (0.0007) [2023-12-27 10:42:48,490][20347] Updated weights for policy 0, policy_version 735009 (0.0007) [2023-12-27 10:42:48,977][20381] Updated weights for policy 1, policy_version 734527 (0.0010) [2023-12-27 10:42:49,044][20381] Updated weights for policy 1, policy_version 734537 (0.0010) [2023-12-27 10:42:49,111][20381] Updated weights for policy 1, policy_version 734547 (0.0011) [2023-12-27 10:42:49,173][20347] Updated weights for policy 0, policy_version 735019 (0.0007) [2023-12-27 10:42:49,230][20347] Updated weights for policy 0, policy_version 735029 (0.0007) [2023-12-27 10:42:49,285][20347] Updated weights for policy 0, policy_version 735039 (0.0008) [2023-12-27 10:42:49,808][20381] Updated weights for policy 1, policy_version 734557 (0.0011) [2023-12-27 10:42:49,870][20381] Updated weights for policy 1, policy_version 734567 (0.0011) [2023-12-27 10:42:49,928][20381] Updated weights for policy 1, policy_version 734577 (0.0010) [2023-12-27 10:42:50,056][20347] Updated weights for policy 0, policy_version 735049 (0.0008) [2023-12-27 10:42:50,103][20347] Updated weights for policy 0, policy_version 735059 (0.0010) [2023-12-27 10:42:50,154][20347] Updated weights for policy 0, policy_version 735069 (0.0010) [2023-12-27 10:42:50,200][20347] Updated weights for policy 0, policy_version 735079 (0.0010) [2023-12-27 10:42:50,619][20381] Updated weights for policy 1, policy_version 734587 (0.0007) [2023-12-27 10:42:50,673][20381] Updated weights for policy 1, policy_version 734597 (0.0006) [2023-12-27 10:42:50,731][20381] Updated weights for policy 1, policy_version 734607 (0.0007) [2023-12-27 10:42:50,935][20347] Updated weights for policy 0, policy_version 735089 (0.0006) [2023-12-27 10:42:50,985][20347] Updated weights for policy 0, policy_version 735099 (0.0006) [2023-12-27 10:42:51,041][20347] Updated weights for policy 0, policy_version 735109 (0.0008) [2023-12-27 10:42:51,422][20381] Updated weights for policy 1, policy_version 734617 (0.0006) [2023-12-27 10:42:51,480][20381] Updated weights for policy 1, policy_version 734627 (0.0006) [2023-12-27 10:42:51,537][20381] Updated weights for policy 1, policy_version 734637 (0.0007) [2023-12-27 10:42:51,596][20381] Updated weights for policy 1, policy_version 734647 (0.0007) [2023-12-27 10:42:51,750][20347] Updated weights for policy 0, policy_version 735119 (0.0008) [2023-12-27 10:42:51,808][20347] Updated weights for policy 0, policy_version 735129 (0.0007) [2023-12-27 10:42:51,810][20291] KL-divergence is very high: 109.7561 [2023-12-27 10:42:51,867][20347] Updated weights for policy 0, policy_version 735139 (0.0007) [2023-12-27 10:42:52,297][20381] Updated weights for policy 1, policy_version 734657 (0.0009) [2023-12-27 10:42:52,355][20381] Updated weights for policy 1, policy_version 734667 (0.0006) [2023-12-27 10:42:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 376315904. Throughput: 0: 10280.3, 1: 10237.0. Samples: 376291012. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:52,363][19170] Avg episode reward: [(0, '951.200'), (1, '737.410')] [2023-12-27 10:42:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000735144_188219392.pth... [2023-12-27 10:42:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000733928_187908096.pth [2023-12-27 10:42:52,422][20381] Updated weights for policy 1, policy_version 734677 (0.0010) [2023-12-27 10:42:52,441][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000734680_188104704.pth... [2023-12-27 10:42:52,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000733464_187793408.pth [2023-12-27 10:42:52,590][20347] Updated weights for policy 0, policy_version 735149 (0.0008) [2023-12-27 10:42:52,649][20347] Updated weights for policy 0, policy_version 735159 (0.0009) [2023-12-27 10:42:52,706][20347] Updated weights for policy 0, policy_version 735169 (0.0009) [2023-12-27 10:42:53,060][20381] Updated weights for policy 1, policy_version 734687 (0.0011) [2023-12-27 10:42:53,121][20381] Updated weights for policy 1, policy_version 734697 (0.0011) [2023-12-27 10:42:53,171][20381] Updated weights for policy 1, policy_version 734707 (0.0010) [2023-12-27 10:42:53,306][20347] Updated weights for policy 0, policy_version 735179 (0.0006) [2023-12-27 10:42:53,368][20347] Updated weights for policy 0, policy_version 735189 (0.0005) [2023-12-27 10:42:53,420][20347] Updated weights for policy 0, policy_version 735199 (0.0005) [2023-12-27 10:42:53,772][20381] Updated weights for policy 1, policy_version 734717 (0.0010) [2023-12-27 10:42:53,833][20381] Updated weights for policy 1, policy_version 734727 (0.0005) [2023-12-27 10:42:53,893][20381] Updated weights for policy 1, policy_version 734737 (0.0005) [2023-12-27 10:42:53,933][20347] Updated weights for policy 0, policy_version 735209 (0.0005) [2023-12-27 10:42:53,984][20347] Updated weights for policy 0, policy_version 735219 (0.0010) [2023-12-27 10:42:54,044][20347] Updated weights for policy 0, policy_version 735229 (0.0010) [2023-12-27 10:42:54,108][20347] Updated weights for policy 0, policy_version 735239 (0.0010) [2023-12-27 10:42:54,505][20381] Updated weights for policy 1, policy_version 734747 (0.0007) [2023-12-27 10:42:54,560][20381] Updated weights for policy 1, policy_version 734757 (0.0010) [2023-12-27 10:42:54,615][20381] Updated weights for policy 1, policy_version 734767 (0.0010) [2023-12-27 10:42:54,738][20347] Updated weights for policy 0, policy_version 735249 (0.0006) [2023-12-27 10:42:54,791][20347] Updated weights for policy 0, policy_version 735259 (0.0010) [2023-12-27 10:42:54,853][20347] Updated weights for policy 0, policy_version 735269 (0.0010) [2023-12-27 10:42:55,226][20381] Updated weights for policy 1, policy_version 734777 (0.0011) [2023-12-27 10:42:55,286][20381] Updated weights for policy 1, policy_version 734787 (0.0011) [2023-12-27 10:42:55,332][20381] Updated weights for policy 1, policy_version 734797 (0.0010) [2023-12-27 10:42:55,390][20381] Updated weights for policy 1, policy_version 734807 (0.0010) [2023-12-27 10:42:55,542][20347] Updated weights for policy 0, policy_version 735279 (0.0010) [2023-12-27 10:42:55,590][20347] Updated weights for policy 0, policy_version 735289 (0.0010) [2023-12-27 10:42:55,647][20347] Updated weights for policy 0, policy_version 735299 (0.0010) [2023-12-27 10:42:56,012][20381] Updated weights for policy 1, policy_version 734817 (0.0010) [2023-12-27 10:42:56,058][20381] Updated weights for policy 1, policy_version 734827 (0.0011) [2023-12-27 10:42:56,104][20381] Updated weights for policy 1, policy_version 734837 (0.0010) [2023-12-27 10:42:56,293][20347] Updated weights for policy 0, policy_version 735309 (0.0010) [2023-12-27 10:42:56,349][20347] Updated weights for policy 0, policy_version 735319 (0.0010) [2023-12-27 10:42:56,403][20347] Updated weights for policy 0, policy_version 735329 (0.0010) [2023-12-27 10:42:56,733][20381] Updated weights for policy 1, policy_version 734847 (0.0009) [2023-12-27 10:42:56,784][20381] Updated weights for policy 1, policy_version 734857 (0.0010) [2023-12-27 10:42:56,836][20381] Updated weights for policy 1, policy_version 734867 (0.0007) [2023-12-27 10:42:57,050][20347] Updated weights for policy 0, policy_version 735339 (0.0010) [2023-12-27 10:42:57,109][20347] Updated weights for policy 0, policy_version 735349 (0.0010) [2023-12-27 10:42:57,159][20347] Updated weights for policy 0, policy_version 735359 (0.0010) [2023-12-27 10:42:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 376430592. Throughput: 0: 10376.8, 1: 10303.2. Samples: 376418340. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:42:57,363][19170] Avg episode reward: [(0, '931.650'), (1, '739.140')] [2023-12-27 10:42:57,408][20381] Updated weights for policy 1, policy_version 734877 (0.0005) [2023-12-27 10:42:57,480][20381] Updated weights for policy 1, policy_version 734887 (0.0010) [2023-12-27 10:42:57,539][20381] Updated weights for policy 1, policy_version 734897 (0.0011) [2023-12-27 10:42:57,831][20347] Updated weights for policy 0, policy_version 735369 (0.0010) [2023-12-27 10:42:57,892][20347] Updated weights for policy 0, policy_version 735379 (0.0010) [2023-12-27 10:42:57,916][20291] KL-divergence is very high: 105.4240 [2023-12-27 10:42:57,945][20347] Updated weights for policy 0, policy_version 735389 (0.0010) [2023-12-27 10:42:57,995][20347] Updated weights for policy 0, policy_version 735399 (0.0010) [2023-12-27 10:42:58,124][20381] Updated weights for policy 1, policy_version 734907 (0.0009) [2023-12-27 10:42:58,189][20381] Updated weights for policy 1, policy_version 734917 (0.0006) [2023-12-27 10:42:58,252][20381] Updated weights for policy 1, policy_version 734927 (0.0006) [2023-12-27 10:42:58,701][20347] Updated weights for policy 0, policy_version 735409 (0.0011) [2023-12-27 10:42:58,764][20347] Updated weights for policy 0, policy_version 735419 (0.0010) [2023-12-27 10:42:58,821][20347] Updated weights for policy 0, policy_version 735429 (0.0010) [2023-12-27 10:42:58,966][20381] Updated weights for policy 1, policy_version 734937 (0.0006) [2023-12-27 10:42:59,025][20381] Updated weights for policy 1, policy_version 734947 (0.0011) [2023-12-27 10:42:59,080][20381] Updated weights for policy 1, policy_version 734957 (0.0010) [2023-12-27 10:42:59,137][20381] Updated weights for policy 1, policy_version 734967 (0.0010) [2023-12-27 10:42:59,515][20347] Updated weights for policy 0, policy_version 735439 (0.0007) [2023-12-27 10:42:59,577][20347] Updated weights for policy 0, policy_version 735449 (0.0007) [2023-12-27 10:42:59,636][20347] Updated weights for policy 0, policy_version 735459 (0.0008) [2023-12-27 10:42:59,809][20381] Updated weights for policy 1, policy_version 734977 (0.0010) [2023-12-27 10:42:59,866][20381] Updated weights for policy 1, policy_version 734987 (0.0008) [2023-12-27 10:42:59,929][20381] Updated weights for policy 1, policy_version 734997 (0.0009) [2023-12-27 10:43:00,219][20347] Updated weights for policy 0, policy_version 735469 (0.0006) [2023-12-27 10:43:00,273][20347] Updated weights for policy 0, policy_version 735479 (0.0007) [2023-12-27 10:43:00,324][20347] Updated weights for policy 0, policy_version 735489 (0.0007) [2023-12-27 10:43:00,603][20381] Updated weights for policy 1, policy_version 735007 (0.0009) [2023-12-27 10:43:00,655][20381] Updated weights for policy 1, policy_version 735017 (0.0010) [2023-12-27 10:43:00,702][20381] Updated weights for policy 1, policy_version 735027 (0.0007) [2023-12-27 10:43:01,018][20347] Updated weights for policy 0, policy_version 735499 (0.0007) [2023-12-27 10:43:01,079][20347] Updated weights for policy 0, policy_version 735509 (0.0010) [2023-12-27 10:43:01,131][20347] Updated weights for policy 0, policy_version 735519 (0.0010) [2023-12-27 10:43:01,344][20381] Updated weights for policy 1, policy_version 735037 (0.0007) [2023-12-27 10:43:01,409][20381] Updated weights for policy 1, policy_version 735047 (0.0008) [2023-12-27 10:43:01,467][20381] Updated weights for policy 1, policy_version 735057 (0.0008) [2023-12-27 10:43:01,896][20347] Updated weights for policy 0, policy_version 735529 (0.0009) [2023-12-27 10:43:01,948][20347] Updated weights for policy 0, policy_version 735539 (0.0010) [2023-12-27 10:43:02,003][20347] Updated weights for policy 0, policy_version 735549 (0.0010) [2023-12-27 10:43:02,059][20347] Updated weights for policy 0, policy_version 735559 (0.0010) [2023-12-27 10:43:02,151][20381] Updated weights for policy 1, policy_version 735067 (0.0008) [2023-12-27 10:43:02,204][20381] Updated weights for policy 1, policy_version 735077 (0.0007) [2023-12-27 10:43:02,256][20381] Updated weights for policy 1, policy_version 735087 (0.0007) [2023-12-27 10:43:02,362][19170] Fps is (10 sec: 22118.7, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 376537088. Throughput: 0: 10398.2, 1: 10353.7. Samples: 376543388. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:02,363][19170] Avg episode reward: [(0, '932.040'), (1, '735.340')] [2023-12-27 10:43:02,753][20347] Updated weights for policy 0, policy_version 735569 (0.0010) [2023-12-27 10:43:02,815][20347] Updated weights for policy 0, policy_version 735579 (0.0010) [2023-12-27 10:43:02,863][20347] Updated weights for policy 0, policy_version 735589 (0.0010) [2023-12-27 10:43:02,902][20381] Updated weights for policy 1, policy_version 735097 (0.0008) [2023-12-27 10:43:02,958][20381] Updated weights for policy 1, policy_version 735107 (0.0005) [2023-12-27 10:43:03,016][20381] Updated weights for policy 1, policy_version 735117 (0.0005) [2023-12-27 10:43:03,070][20381] Updated weights for policy 1, policy_version 735127 (0.0005) [2023-12-27 10:43:03,432][20347] Updated weights for policy 0, policy_version 735599 (0.0007) [2023-12-27 10:43:03,487][20347] Updated weights for policy 0, policy_version 735609 (0.0005) [2023-12-27 10:43:03,543][20347] Updated weights for policy 0, policy_version 735619 (0.0005) [2023-12-27 10:43:03,679][20381] Updated weights for policy 1, policy_version 735137 (0.0005) [2023-12-27 10:43:03,739][20381] Updated weights for policy 1, policy_version 735147 (0.0005) [2023-12-27 10:43:03,787][20381] Updated weights for policy 1, policy_version 735157 (0.0005) [2023-12-27 10:43:04,159][20347] Updated weights for policy 0, policy_version 735629 (0.0006) [2023-12-27 10:43:04,220][20347] Updated weights for policy 0, policy_version 735639 (0.0006) [2023-12-27 10:43:04,274][20347] Updated weights for policy 0, policy_version 735649 (0.0008) [2023-12-27 10:43:04,455][20381] Updated weights for policy 1, policy_version 735167 (0.0007) [2023-12-27 10:43:04,510][20381] Updated weights for policy 1, policy_version 735177 (0.0007) [2023-12-27 10:43:04,561][20381] Updated weights for policy 1, policy_version 735187 (0.0007) [2023-12-27 10:43:04,895][20347] Updated weights for policy 0, policy_version 735659 (0.0007) [2023-12-27 10:43:04,950][20347] Updated weights for policy 0, policy_version 735669 (0.0007) [2023-12-27 10:43:05,012][20347] Updated weights for policy 0, policy_version 735679 (0.0008) [2023-12-27 10:43:05,142][20381] Updated weights for policy 1, policy_version 735197 (0.0009) [2023-12-27 10:43:05,188][20381] Updated weights for policy 1, policy_version 735207 (0.0008) [2023-12-27 10:43:05,243][20381] Updated weights for policy 1, policy_version 735217 (0.0007) [2023-12-27 10:43:05,667][20347] Updated weights for policy 0, policy_version 735689 (0.0008) [2023-12-27 10:43:05,718][20347] Updated weights for policy 0, policy_version 735699 (0.0008) [2023-12-27 10:43:05,772][20347] Updated weights for policy 0, policy_version 735709 (0.0008) [2023-12-27 10:43:05,829][20347] Updated weights for policy 0, policy_version 735719 (0.0008) [2023-12-27 10:43:05,839][20381] Updated weights for policy 1, policy_version 735227 (0.0006) [2023-12-27 10:43:05,892][20381] Updated weights for policy 1, policy_version 735237 (0.0006) [2023-12-27 10:43:05,942][20381] Updated weights for policy 1, policy_version 735247 (0.0006) [2023-12-27 10:43:06,539][20347] Updated weights for policy 0, policy_version 735729 (0.0007) [2023-12-27 10:43:06,594][20347] Updated weights for policy 0, policy_version 735739 (0.0008) [2023-12-27 10:43:06,634][20381] Updated weights for policy 1, policy_version 735257 (0.0005) [2023-12-27 10:43:06,651][20347] Updated weights for policy 0, policy_version 735749 (0.0008) [2023-12-27 10:43:06,696][20381] Updated weights for policy 1, policy_version 735267 (0.0007) [2023-12-27 10:43:06,762][20381] Updated weights for policy 1, policy_version 735277 (0.0006) [2023-12-27 10:43:06,830][20381] Updated weights for policy 1, policy_version 735287 (0.0006) [2023-12-27 10:43:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 376635392. Throughput: 0: 10414.0, 1: 10443.6. Samples: 376608996. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:07,363][19170] Avg episode reward: [(0, '951.780'), (1, '437.930')] [2023-12-27 10:43:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000735288_188260352.pth... [2023-12-27 10:43:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000734072_187949056.pth [2023-12-27 10:43:07,388][20347] Updated weights for policy 0, policy_version 735759 (0.0007) [2023-12-27 10:43:07,437][20347] Updated weights for policy 0, policy_version 735769 (0.0007) [2023-12-27 10:43:07,446][20381] Updated weights for policy 1, policy_version 735297 (0.0008) [2023-12-27 10:43:07,492][20347] Updated weights for policy 0, policy_version 735779 (0.0008) [2023-12-27 10:43:07,495][20381] Updated weights for policy 1, policy_version 735307 (0.0007) [2023-12-27 10:43:07,519][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000735784_188383232.pth... [2023-12-27 10:43:07,522][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000734568_188071936.pth [2023-12-27 10:43:07,553][20381] Updated weights for policy 1, policy_version 735317 (0.0008) [2023-12-27 10:43:08,198][20347] Updated weights for policy 0, policy_version 735789 (0.0009) [2023-12-27 10:43:08,234][20381] Updated weights for policy 1, policy_version 735327 (0.0006) [2023-12-27 10:43:08,252][20347] Updated weights for policy 0, policy_version 735799 (0.0007) [2023-12-27 10:43:08,300][20381] Updated weights for policy 1, policy_version 735337 (0.0006) [2023-12-27 10:43:08,311][20347] Updated weights for policy 0, policy_version 735809 (0.0005) [2023-12-27 10:43:08,372][20381] Updated weights for policy 1, policy_version 735347 (0.0010) [2023-12-27 10:43:09,020][20347] Updated weights for policy 0, policy_version 735819 (0.0006) [2023-12-27 10:43:09,029][20381] Updated weights for policy 1, policy_version 735357 (0.0011) [2023-12-27 10:43:09,072][20347] Updated weights for policy 0, policy_version 735829 (0.0005) [2023-12-27 10:43:09,082][20381] Updated weights for policy 1, policy_version 735367 (0.0010) [2023-12-27 10:43:09,129][20347] Updated weights for policy 0, policy_version 735839 (0.0005) [2023-12-27 10:43:09,141][20381] Updated weights for policy 1, policy_version 735377 (0.0010) [2023-12-27 10:43:09,800][20347] Updated weights for policy 0, policy_version 735849 (0.0006) [2023-12-27 10:43:09,864][20347] Updated weights for policy 0, policy_version 735859 (0.0007) [2023-12-27 10:43:09,867][20381] Updated weights for policy 1, policy_version 735387 (0.0010) [2023-12-27 10:43:09,930][20381] Updated weights for policy 1, policy_version 735397 (0.0006) [2023-12-27 10:43:09,936][20347] Updated weights for policy 0, policy_version 735869 (0.0007) [2023-12-27 10:43:09,994][20381] Updated weights for policy 1, policy_version 735407 (0.0006) [2023-12-27 10:43:09,997][20347] Updated weights for policy 0, policy_version 735879 (0.0007) [2023-12-27 10:43:10,646][20381] Updated weights for policy 1, policy_version 735417 (0.0008) [2023-12-27 10:43:10,691][20381] Updated weights for policy 1, policy_version 735427 (0.0007) [2023-12-27 10:43:10,694][20347] Updated weights for policy 0, policy_version 735889 (0.0006) [2023-12-27 10:43:10,741][20291] KL-divergence is very high: 154.3508 [2023-12-27 10:43:10,747][20347] Updated weights for policy 0, policy_version 735899 (0.0005) [2023-12-27 10:43:10,749][20381] Updated weights for policy 1, policy_version 735437 (0.0006) [2023-12-27 10:43:10,798][20381] Updated weights for policy 1, policy_version 735447 (0.0006) [2023-12-27 10:43:10,800][20347] Updated weights for policy 0, policy_version 735909 (0.0007) [2023-12-27 10:43:11,513][20291] KL-divergence is very high: 211.6108 [2023-12-27 10:43:11,519][20347] Updated weights for policy 0, policy_version 735919 (0.0006) [2023-12-27 10:43:11,576][20347] Updated weights for policy 0, policy_version 735929 (0.0006) [2023-12-27 10:43:11,618][20381] Updated weights for policy 1, policy_version 735457 (0.0007) [2023-12-27 10:43:11,645][20347] Updated weights for policy 0, policy_version 735939 (0.0006) [2023-12-27 10:43:11,681][20381] Updated weights for policy 1, policy_version 735467 (0.0007) [2023-12-27 10:43:11,751][20381] Updated weights for policy 1, policy_version 735477 (0.0009) [2023-12-27 10:43:12,310][20347] Updated weights for policy 0, policy_version 735949 (0.0009) [2023-12-27 10:43:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 376733696. Throughput: 0: 10396.2, 1: 10395.7. Samples: 376730472. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:12,363][19170] Avg episode reward: [(0, '913.590'), (1, '477.210')] [2023-12-27 10:43:12,368][20347] Updated weights for policy 0, policy_version 735959 (0.0008) [2023-12-27 10:43:12,430][20347] Updated weights for policy 0, policy_version 735969 (0.0008) [2023-12-27 10:43:12,466][20381] Updated weights for policy 1, policy_version 735487 (0.0007) [2023-12-27 10:43:12,524][20381] Updated weights for policy 1, policy_version 735497 (0.0007) [2023-12-27 10:43:12,577][20381] Updated weights for policy 1, policy_version 735507 (0.0008) [2023-12-27 10:43:13,140][20347] Updated weights for policy 0, policy_version 735979 (0.0008) [2023-12-27 10:43:13,196][20347] Updated weights for policy 0, policy_version 735989 (0.0008) [2023-12-27 10:43:13,246][20347] Updated weights for policy 0, policy_version 735999 (0.0007) [2023-12-27 10:43:13,289][20381] Updated weights for policy 1, policy_version 735517 (0.0007) [2023-12-27 10:43:13,350][20381] Updated weights for policy 1, policy_version 735527 (0.0006) [2023-12-27 10:43:13,417][20381] Updated weights for policy 1, policy_version 735537 (0.0007) [2023-12-27 10:43:13,894][20347] Updated weights for policy 0, policy_version 736009 (0.0008) [2023-12-27 10:43:13,955][20347] Updated weights for policy 0, policy_version 736019 (0.0005) [2023-12-27 10:43:14,014][20347] Updated weights for policy 0, policy_version 736029 (0.0006) [2023-12-27 10:43:14,022][20381] Updated weights for policy 1, policy_version 735547 (0.0009) [2023-12-27 10:43:14,067][20347] Updated weights for policy 0, policy_version 736039 (0.0005) [2023-12-27 10:43:14,071][20381] Updated weights for policy 1, policy_version 735557 (0.0010) [2023-12-27 10:43:14,116][20381] Updated weights for policy 1, policy_version 735567 (0.0010) [2023-12-27 10:43:14,642][20347] Updated weights for policy 0, policy_version 736049 (0.0008) [2023-12-27 10:43:14,692][20347] Updated weights for policy 0, policy_version 736060 (0.0009) [2023-12-27 10:43:14,746][20347] Updated weights for policy 0, policy_version 736070 (0.0007) [2023-12-27 10:43:14,780][20381] Updated weights for policy 1, policy_version 735577 (0.0010) [2023-12-27 10:43:14,835][20381] Updated weights for policy 1, policy_version 735587 (0.0007) [2023-12-27 10:43:14,898][20381] Updated weights for policy 1, policy_version 735597 (0.0008) [2023-12-27 10:43:14,961][20381] Updated weights for policy 1, policy_version 735607 (0.0006) [2023-12-27 10:43:15,470][20347] Updated weights for policy 0, policy_version 736080 (0.0007) [2023-12-27 10:43:15,514][20347] Updated weights for policy 0, policy_version 736090 (0.0007) [2023-12-27 10:43:15,561][20347] Updated weights for policy 0, policy_version 736100 (0.0007) [2023-12-27 10:43:15,643][20381] Updated weights for policy 1, policy_version 735617 (0.0010) [2023-12-27 10:43:15,699][20381] Updated weights for policy 1, policy_version 735627 (0.0011) [2023-12-27 10:43:15,750][20381] Updated weights for policy 1, policy_version 735637 (0.0010) [2023-12-27 10:43:16,238][20347] Updated weights for policy 0, policy_version 736110 (0.0006) [2023-12-27 10:43:16,291][20347] Updated weights for policy 0, policy_version 736120 (0.0008) [2023-12-27 10:43:16,344][20347] Updated weights for policy 0, policy_version 736130 (0.0008) [2023-12-27 10:43:16,414][20381] Updated weights for policy 1, policy_version 735647 (0.0010) [2023-12-27 10:43:16,466][20381] Updated weights for policy 1, policy_version 735657 (0.0011) [2023-12-27 10:43:16,514][20381] Updated weights for policy 1, policy_version 735667 (0.0010) [2023-12-27 10:43:17,036][20347] Updated weights for policy 0, policy_version 736140 (0.0008) [2023-12-27 10:43:17,095][20347] Updated weights for policy 0, policy_version 736150 (0.0008) [2023-12-27 10:43:17,153][20347] Updated weights for policy 0, policy_version 736160 (0.0007) [2023-12-27 10:43:17,200][20381] Updated weights for policy 1, policy_version 735677 (0.0010) [2023-12-27 10:43:17,259][20381] Updated weights for policy 1, policy_version 735687 (0.0010) [2023-12-27 10:43:17,319][20381] Updated weights for policy 1, policy_version 735697 (0.0010) [2023-12-27 10:43:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 376848384. Throughput: 0: 10359.7, 1: 10413.2. Samples: 376853372. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:17,363][19170] Avg episode reward: [(0, '922.890'), (1, '490.320')] [2023-12-27 10:43:17,818][20347] Updated weights for policy 0, policy_version 736170 (0.0007) [2023-12-27 10:43:17,868][20347] Updated weights for policy 0, policy_version 736180 (0.0007) [2023-12-27 10:43:17,924][20347] Updated weights for policy 0, policy_version 736190 (0.0008) [2023-12-27 10:43:17,966][20381] Updated weights for policy 1, policy_version 735707 (0.0009) [2023-12-27 10:43:17,977][20347] Updated weights for policy 0, policy_version 736200 (0.0010) [2023-12-27 10:43:18,021][20381] Updated weights for policy 1, policy_version 735717 (0.0006) [2023-12-27 10:43:18,080][20381] Updated weights for policy 1, policy_version 735727 (0.0011) [2023-12-27 10:43:18,694][20347] Updated weights for policy 0, policy_version 736210 (0.0008) [2023-12-27 10:43:18,741][20381] Updated weights for policy 1, policy_version 735737 (0.0010) [2023-12-27 10:43:18,746][20347] Updated weights for policy 0, policy_version 736220 (0.0007) [2023-12-27 10:43:18,791][20381] Updated weights for policy 1, policy_version 735747 (0.0006) [2023-12-27 10:43:18,815][20347] Updated weights for policy 0, policy_version 736230 (0.0008) [2023-12-27 10:43:18,847][20381] Updated weights for policy 1, policy_version 735757 (0.0009) [2023-12-27 10:43:18,894][20381] Updated weights for policy 1, policy_version 735767 (0.0008) [2023-12-27 10:43:19,532][20347] Updated weights for policy 0, policy_version 736240 (0.0008) [2023-12-27 10:43:19,590][20347] Updated weights for policy 0, policy_version 736250 (0.0007) [2023-12-27 10:43:19,649][20381] Updated weights for policy 1, policy_version 735777 (0.0010) [2023-12-27 10:43:19,651][20347] Updated weights for policy 0, policy_version 736260 (0.0007) [2023-12-27 10:43:19,711][20381] Updated weights for policy 1, policy_version 735787 (0.0011) [2023-12-27 10:43:19,765][20381] Updated weights for policy 1, policy_version 735797 (0.0011) [2023-12-27 10:43:20,353][20347] Updated weights for policy 0, policy_version 736270 (0.0007) [2023-12-27 10:43:20,411][20347] Updated weights for policy 0, policy_version 736280 (0.0007) [2023-12-27 10:43:20,462][20381] Updated weights for policy 1, policy_version 735807 (0.0011) [2023-12-27 10:43:20,464][20347] Updated weights for policy 0, policy_version 736290 (0.0008) [2023-12-27 10:43:20,522][20381] Updated weights for policy 1, policy_version 735817 (0.0010) [2023-12-27 10:43:20,595][20381] Updated weights for policy 1, policy_version 735827 (0.0008) [2023-12-27 10:43:21,214][20347] Updated weights for policy 0, policy_version 736300 (0.0007) [2023-12-27 10:43:21,265][20381] Updated weights for policy 1, policy_version 735837 (0.0008) [2023-12-27 10:43:21,276][20347] Updated weights for policy 0, policy_version 736310 (0.0007) [2023-12-27 10:43:21,326][20381] Updated weights for policy 1, policy_version 735847 (0.0009) [2023-12-27 10:43:21,333][20347] Updated weights for policy 0, policy_version 736320 (0.0007) [2023-12-27 10:43:21,396][20381] Updated weights for policy 1, policy_version 735857 (0.0008) [2023-12-27 10:43:22,011][20347] Updated weights for policy 0, policy_version 736330 (0.0007) [2023-12-27 10:43:22,073][20347] Updated weights for policy 0, policy_version 736340 (0.0006) [2023-12-27 10:43:22,124][20347] Updated weights for policy 0, policy_version 736350 (0.0007) [2023-12-27 10:43:22,159][20381] Updated weights for policy 1, policy_version 735867 (0.0007) [2023-12-27 10:43:22,183][20347] Updated weights for policy 0, policy_version 736360 (0.0006) [2023-12-27 10:43:22,212][20381] Updated weights for policy 1, policy_version 735877 (0.0011) [2023-12-27 10:43:22,272][20381] Updated weights for policy 1, policy_version 735887 (0.0011) [2023-12-27 10:43:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 376946688. Throughput: 0: 10290.8, 1: 10378.6. Samples: 376912960. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:22,363][19170] Avg episode reward: [(0, '971.030'), (1, '493.810')] [2023-12-27 10:43:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000735896_188416000.pth... [2023-12-27 10:43:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000736360_188530688.pth... [2023-12-27 10:43:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000735144_188219392.pth [2023-12-27 10:43:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000734680_188104704.pth [2023-12-27 10:43:22,884][20347] Updated weights for policy 0, policy_version 736370 (0.0008) [2023-12-27 10:43:22,937][20381] Updated weights for policy 1, policy_version 735897 (0.0010) [2023-12-27 10:43:22,945][20347] Updated weights for policy 0, policy_version 736380 (0.0008) [2023-12-27 10:43:23,001][20347] Updated weights for policy 0, policy_version 736390 (0.0007) [2023-12-27 10:43:23,002][20381] Updated weights for policy 1, policy_version 735907 (0.0008) [2023-12-27 10:43:23,055][20381] Updated weights for policy 1, policy_version 735917 (0.0007) [2023-12-27 10:43:23,113][20381] Updated weights for policy 1, policy_version 735927 (0.0008) [2023-12-27 10:43:23,702][20347] Updated weights for policy 0, policy_version 736400 (0.0007) [2023-12-27 10:43:23,755][20347] Updated weights for policy 0, policy_version 736410 (0.0008) [2023-12-27 10:43:23,801][20381] Updated weights for policy 1, policy_version 735937 (0.0006) [2023-12-27 10:43:23,802][20347] Updated weights for policy 0, policy_version 736420 (0.0006) [2023-12-27 10:43:23,850][20381] Updated weights for policy 1, policy_version 735947 (0.0005) [2023-12-27 10:43:23,908][20381] Updated weights for policy 1, policy_version 735957 (0.0005) [2023-12-27 10:43:24,516][20381] Updated weights for policy 1, policy_version 735967 (0.0006) [2023-12-27 10:43:24,517][20347] Updated weights for policy 0, policy_version 736430 (0.0006) [2023-12-27 10:43:24,575][20381] Updated weights for policy 1, policy_version 735977 (0.0006) [2023-12-27 10:43:24,575][20347] Updated weights for policy 0, policy_version 736440 (0.0006) [2023-12-27 10:43:24,631][20381] Updated weights for policy 1, policy_version 735987 (0.0006) [2023-12-27 10:43:24,632][20347] Updated weights for policy 0, policy_version 736450 (0.0007) [2023-12-27 10:43:25,288][20381] Updated weights for policy 1, policy_version 735997 (0.0008) [2023-12-27 10:43:25,296][20347] Updated weights for policy 0, policy_version 736460 (0.0007) [2023-12-27 10:43:25,347][20381] Updated weights for policy 1, policy_version 736007 (0.0011) [2023-12-27 10:43:25,354][20347] Updated weights for policy 0, policy_version 736470 (0.0007) [2023-12-27 10:43:25,399][20381] Updated weights for policy 1, policy_version 736017 (0.0010) [2023-12-27 10:43:25,435][20347] Updated weights for policy 0, policy_version 736480 (0.0005) [2023-12-27 10:43:26,073][20381] Updated weights for policy 1, policy_version 736027 (0.0009) [2023-12-27 10:43:26,080][20347] Updated weights for policy 0, policy_version 736490 (0.0007) [2023-12-27 10:43:26,121][20381] Updated weights for policy 1, policy_version 736037 (0.0005) [2023-12-27 10:43:26,134][20347] Updated weights for policy 0, policy_version 736500 (0.0007) [2023-12-27 10:43:26,180][20381] Updated weights for policy 1, policy_version 736047 (0.0009) [2023-12-27 10:43:26,186][20347] Updated weights for policy 0, policy_version 736510 (0.0007) [2023-12-27 10:43:26,238][20347] Updated weights for policy 0, policy_version 736520 (0.0007) [2023-12-27 10:43:26,847][20381] Updated weights for policy 1, policy_version 736057 (0.0010) [2023-12-27 10:43:26,900][20381] Updated weights for policy 1, policy_version 736067 (0.0010) [2023-12-27 10:43:26,920][20347] Updated weights for policy 0, policy_version 736530 (0.0008) [2023-12-27 10:43:26,957][20381] Updated weights for policy 1, policy_version 736077 (0.0006) [2023-12-27 10:43:26,976][20347] Updated weights for policy 0, policy_version 736540 (0.0010) [2023-12-27 10:43:27,015][20381] Updated weights for policy 1, policy_version 736087 (0.0006) [2023-12-27 10:43:27,031][20347] Updated weights for policy 0, policy_version 736550 (0.0010) [2023-12-27 10:43:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 377044992. Throughput: 0: 10256.7, 1: 10364.6. Samples: 377034028. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:27,363][19170] Avg episode reward: [(0, '990.310'), (1, '573.130')] [2023-12-27 10:43:27,633][20347] Updated weights for policy 0, policy_version 736560 (0.0009) [2023-12-27 10:43:27,677][20347] Updated weights for policy 0, policy_version 736570 (0.0009) [2023-12-27 10:43:27,716][20381] Updated weights for policy 1, policy_version 736097 (0.0005) [2023-12-27 10:43:27,732][20347] Updated weights for policy 0, policy_version 736580 (0.0010) [2023-12-27 10:43:27,760][20381] Updated weights for policy 1, policy_version 736107 (0.0005) [2023-12-27 10:43:27,804][20381] Updated weights for policy 1, policy_version 736117 (0.0007) [2023-12-27 10:43:28,379][20347] Updated weights for policy 0, policy_version 736590 (0.0011) [2023-12-27 10:43:28,438][20347] Updated weights for policy 0, policy_version 736600 (0.0010) [2023-12-27 10:43:28,497][20347] Updated weights for policy 0, policy_version 736610 (0.0007) [2023-12-27 10:43:28,537][20381] Updated weights for policy 1, policy_version 736127 (0.0006) [2023-12-27 10:43:28,585][20381] Updated weights for policy 1, policy_version 736137 (0.0007) [2023-12-27 10:43:28,638][20381] Updated weights for policy 1, policy_version 736147 (0.0007) [2023-12-27 10:43:29,146][20347] Updated weights for policy 0, policy_version 736620 (0.0007) [2023-12-27 10:43:29,203][20347] Updated weights for policy 0, policy_version 736630 (0.0005) [2023-12-27 10:43:29,271][20347] Updated weights for policy 0, policy_version 736640 (0.0008) [2023-12-27 10:43:29,282][20381] Updated weights for policy 1, policy_version 736157 (0.0007) [2023-12-27 10:43:29,335][20381] Updated weights for policy 1, policy_version 736167 (0.0007) [2023-12-27 10:43:29,398][20381] Updated weights for policy 1, policy_version 736177 (0.0008) [2023-12-27 10:43:29,888][20347] Updated weights for policy 0, policy_version 736650 (0.0006) [2023-12-27 10:43:29,946][20347] Updated weights for policy 0, policy_version 736660 (0.0008) [2023-12-27 10:43:30,013][20347] Updated weights for policy 0, policy_version 736670 (0.0008) [2023-12-27 10:43:30,041][20381] Updated weights for policy 1, policy_version 736187 (0.0006) [2023-12-27 10:43:30,068][20347] Updated weights for policy 0, policy_version 736680 (0.0008) [2023-12-27 10:43:30,097][20381] Updated weights for policy 1, policy_version 736197 (0.0006) [2023-12-27 10:43:30,156][20381] Updated weights for policy 1, policy_version 736207 (0.0006) [2023-12-27 10:43:30,760][20347] Updated weights for policy 0, policy_version 736690 (0.0007) [2023-12-27 10:43:30,793][20381] Updated weights for policy 1, policy_version 736217 (0.0007) [2023-12-27 10:43:30,806][20347] Updated weights for policy 0, policy_version 736700 (0.0006) [2023-12-27 10:43:30,848][20381] Updated weights for policy 1, policy_version 736227 (0.0006) [2023-12-27 10:43:30,858][20347] Updated weights for policy 0, policy_version 736710 (0.0006) [2023-12-27 10:43:30,904][20381] Updated weights for policy 1, policy_version 736237 (0.0005) [2023-12-27 10:43:30,957][20381] Updated weights for policy 1, policy_version 736247 (0.0005) [2023-12-27 10:43:31,615][20381] Updated weights for policy 1, policy_version 736257 (0.0006) [2023-12-27 10:43:31,630][20347] Updated weights for policy 0, policy_version 736720 (0.0009) [2023-12-27 10:43:31,673][20381] Updated weights for policy 1, policy_version 736267 (0.0007) [2023-12-27 10:43:31,683][20347] Updated weights for policy 0, policy_version 736730 (0.0008) [2023-12-27 10:43:31,733][20381] Updated weights for policy 1, policy_version 736277 (0.0007) [2023-12-27 10:43:31,754][20347] Updated weights for policy 0, policy_version 736740 (0.0008) [2023-12-27 10:43:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 377143296. Throughput: 0: 10268.4, 1: 10379.5. Samples: 377158268. Policy #0 lag: (min: 31.0, avg: 41.2, max: 63.0) [2023-12-27 10:43:32,363][19170] Avg episode reward: [(0, '951.230'), (1, '473.620')] [2023-12-27 10:43:32,410][20381] Updated weights for policy 1, policy_version 736287 (0.0008) [2023-12-27 10:43:32,470][20347] Updated weights for policy 0, policy_version 736750 (0.0008) [2023-12-27 10:43:32,471][20381] Updated weights for policy 1, policy_version 736297 (0.0006) [2023-12-27 10:43:32,529][20381] Updated weights for policy 1, policy_version 736307 (0.0006) [2023-12-27 10:43:32,531][20347] Updated weights for policy 0, policy_version 736760 (0.0006) [2023-12-27 10:43:32,601][20347] Updated weights for policy 0, policy_version 736770 (0.0006) [2023-12-27 10:43:33,133][20381] Updated weights for policy 1, policy_version 736317 (0.0006) [2023-12-27 10:43:33,183][20381] Updated weights for policy 1, policy_version 736327 (0.0005) [2023-12-27 10:43:33,235][20381] Updated weights for policy 1, policy_version 736337 (0.0005) [2023-12-27 10:43:33,267][20347] Updated weights for policy 0, policy_version 736780 (0.0010) [2023-12-27 10:43:33,325][20347] Updated weights for policy 0, policy_version 736790 (0.0008) [2023-12-27 10:43:33,376][20347] Updated weights for policy 0, policy_version 736800 (0.0007) [2023-12-27 10:43:33,796][20381] Updated weights for policy 1, policy_version 736347 (0.0005) [2023-12-27 10:43:33,849][20381] Updated weights for policy 1, policy_version 736357 (0.0005) [2023-12-27 10:43:33,903][20381] Updated weights for policy 1, policy_version 736367 (0.0010) [2023-12-27 10:43:34,125][20347] Updated weights for policy 0, policy_version 736810 (0.0008) [2023-12-27 10:43:34,187][20347] Updated weights for policy 0, policy_version 736820 (0.0008) [2023-12-27 10:43:34,240][20347] Updated weights for policy 0, policy_version 736830 (0.0007) [2023-12-27 10:43:34,295][20347] Updated weights for policy 0, policy_version 736840 (0.0007) [2023-12-27 10:43:34,558][20381] Updated weights for policy 1, policy_version 736377 (0.0008) [2023-12-27 10:43:34,619][20381] Updated weights for policy 1, policy_version 736387 (0.0011) [2023-12-27 10:43:34,680][20381] Updated weights for policy 1, policy_version 736397 (0.0011) [2023-12-27 10:43:34,738][20381] Updated weights for policy 1, policy_version 736407 (0.0008) [2023-12-27 10:43:34,975][20347] Updated weights for policy 0, policy_version 736850 (0.0008) [2023-12-27 10:43:35,031][20347] Updated weights for policy 0, policy_version 736860 (0.0008) [2023-12-27 10:43:35,083][20347] Updated weights for policy 0, policy_version 736870 (0.0009) [2023-12-27 10:43:35,302][20381] Updated weights for policy 1, policy_version 736417 (0.0005) [2023-12-27 10:43:35,365][20381] Updated weights for policy 1, policy_version 736427 (0.0005) [2023-12-27 10:43:35,424][20381] Updated weights for policy 1, policy_version 736437 (0.0005) [2023-12-27 10:43:35,740][20347] Updated weights for policy 0, policy_version 736880 (0.0006) [2023-12-27 10:43:35,801][20347] Updated weights for policy 0, policy_version 736890 (0.0005) [2023-12-27 10:43:35,859][20347] Updated weights for policy 0, policy_version 736900 (0.0005) [2023-12-27 10:43:35,980][20381] Updated weights for policy 1, policy_version 736447 (0.0006) [2023-12-27 10:43:36,026][20381] Updated weights for policy 1, policy_version 736457 (0.0010) [2023-12-27 10:43:36,076][20381] Updated weights for policy 1, policy_version 736467 (0.0008) [2023-12-27 10:43:36,448][20347] Updated weights for policy 0, policy_version 736910 (0.0006) [2023-12-27 10:43:36,514][20347] Updated weights for policy 0, policy_version 736920 (0.0008) [2023-12-27 10:43:36,585][20347] Updated weights for policy 0, policy_version 736930 (0.0008) [2023-12-27 10:43:36,784][20381] Updated weights for policy 1, policy_version 736477 (0.0008) [2023-12-27 10:43:36,844][20381] Updated weights for policy 1, policy_version 736487 (0.0008) [2023-12-27 10:43:36,904][20381] Updated weights for policy 1, policy_version 736497 (0.0008) [2023-12-27 10:43:37,205][20347] Updated weights for policy 0, policy_version 736940 (0.0009) [2023-12-27 10:43:37,269][20347] Updated weights for policy 0, policy_version 736950 (0.0007) [2023-12-27 10:43:37,332][20347] Updated weights for policy 0, policy_version 736960 (0.0007) [2023-12-27 10:43:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 377249792. Throughput: 0: 10216.0, 1: 10456.6. Samples: 377221280. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:43:37,363][19170] Avg episode reward: [(0, '912.310'), (1, '536.810')] [2023-12-27 10:43:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000736504_188571648.pth... [2023-12-27 10:43:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000735288_188260352.pth [2023-12-27 10:43:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000736968_188686336.pth... [2023-12-27 10:43:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000735784_188383232.pth [2023-12-27 10:43:37,608][20381] Updated weights for policy 1, policy_version 736507 (0.0006) [2023-12-27 10:43:37,669][20381] Updated weights for policy 1, policy_version 736517 (0.0008) [2023-12-27 10:43:37,733][20381] Updated weights for policy 1, policy_version 736527 (0.0007) [2023-12-27 10:43:37,991][20347] Updated weights for policy 0, policy_version 736970 (0.0005) [2023-12-27 10:43:38,049][20347] Updated weights for policy 0, policy_version 736980 (0.0006) [2023-12-27 10:43:38,110][20347] Updated weights for policy 0, policy_version 736990 (0.0005) [2023-12-27 10:43:38,166][20347] Updated weights for policy 0, policy_version 737000 (0.0007) [2023-12-27 10:43:38,335][20381] Updated weights for policy 1, policy_version 736537 (0.0007) [2023-12-27 10:43:38,395][20381] Updated weights for policy 1, policy_version 736547 (0.0007) [2023-12-27 10:43:38,445][20381] Updated weights for policy 1, policy_version 736557 (0.0007) [2023-12-27 10:43:38,502][20381] Updated weights for policy 1, policy_version 736567 (0.0007) [2023-12-27 10:43:38,847][20347] Updated weights for policy 0, policy_version 737010 (0.0008) [2023-12-27 10:43:38,918][20347] Updated weights for policy 0, policy_version 737020 (0.0007) [2023-12-27 10:43:38,971][20347] Updated weights for policy 0, policy_version 737030 (0.0007) [2023-12-27 10:43:39,198][20381] Updated weights for policy 1, policy_version 736577 (0.0006) [2023-12-27 10:43:39,272][20381] Updated weights for policy 1, policy_version 736587 (0.0009) [2023-12-27 10:43:39,334][20381] Updated weights for policy 1, policy_version 736597 (0.0008) [2023-12-27 10:43:39,723][20347] Updated weights for policy 0, policy_version 737040 (0.0008) [2023-12-27 10:43:39,793][20347] Updated weights for policy 0, policy_version 737050 (0.0008) [2023-12-27 10:43:39,868][20347] Updated weights for policy 0, policy_version 737060 (0.0008) [2023-12-27 10:43:40,130][20381] Updated weights for policy 1, policy_version 736607 (0.0008) [2023-12-27 10:43:40,196][20381] Updated weights for policy 1, policy_version 736617 (0.0008) [2023-12-27 10:43:40,264][20381] Updated weights for policy 1, policy_version 736627 (0.0009) [2023-12-27 10:43:40,620][20347] Updated weights for policy 0, policy_version 737070 (0.0007) [2023-12-27 10:43:40,670][20347] Updated weights for policy 0, policy_version 737080 (0.0005) [2023-12-27 10:43:40,731][20347] Updated weights for policy 0, policy_version 737090 (0.0006) [2023-12-27 10:43:40,986][20381] Updated weights for policy 1, policy_version 736637 (0.0007) [2023-12-27 10:43:41,059][20381] Updated weights for policy 1, policy_version 736647 (0.0007) [2023-12-27 10:43:41,127][20381] Updated weights for policy 1, policy_version 736657 (0.0008) [2023-12-27 10:43:41,484][20347] Updated weights for policy 0, policy_version 737100 (0.0008) [2023-12-27 10:43:41,547][20347] Updated weights for policy 0, policy_version 737110 (0.0008) [2023-12-27 10:43:41,610][20347] Updated weights for policy 0, policy_version 737120 (0.0008) [2023-12-27 10:43:41,959][20381] Updated weights for policy 1, policy_version 736667 (0.0008) [2023-12-27 10:43:42,017][20381] Updated weights for policy 1, policy_version 736677 (0.0008) [2023-12-27 10:43:42,075][20381] Updated weights for policy 1, policy_version 736687 (0.0007) [2023-12-27 10:43:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 377348096. Throughput: 0: 10160.8, 1: 10351.6. Samples: 377341400. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:43:42,363][19170] Avg episode reward: [(0, '941.520'), (1, '363.610')] [2023-12-27 10:43:42,383][20347] Updated weights for policy 0, policy_version 737130 (0.0009) [2023-12-27 10:43:42,438][20347] Updated weights for policy 0, policy_version 737140 (0.0010) [2023-12-27 10:43:42,502][20347] Updated weights for policy 0, policy_version 737150 (0.0010) [2023-12-27 10:43:42,562][20347] Updated weights for policy 0, policy_version 737160 (0.0010) [2023-12-27 10:43:42,828][20381] Updated weights for policy 1, policy_version 736697 (0.0009) [2023-12-27 10:43:42,887][20381] Updated weights for policy 1, policy_version 736707 (0.0011) [2023-12-27 10:43:42,942][20381] Updated weights for policy 1, policy_version 736717 (0.0006) [2023-12-27 10:43:43,007][20381] Updated weights for policy 1, policy_version 736727 (0.0008) [2023-12-27 10:43:43,264][20347] Updated weights for policy 0, policy_version 737170 (0.0007) [2023-12-27 10:43:43,315][20347] Updated weights for policy 0, policy_version 737180 (0.0007) [2023-12-27 10:43:43,370][20347] Updated weights for policy 0, policy_version 737190 (0.0007) [2023-12-27 10:43:43,705][20381] Updated weights for policy 1, policy_version 736737 (0.0008) [2023-12-27 10:43:43,761][20381] Updated weights for policy 1, policy_version 736747 (0.0008) [2023-12-27 10:43:43,816][20381] Updated weights for policy 1, policy_version 736757 (0.0008) [2023-12-27 10:43:44,060][20347] Updated weights for policy 0, policy_version 737200 (0.0009) [2023-12-27 10:43:44,121][20347] Updated weights for policy 0, policy_version 737210 (0.0011) [2023-12-27 10:43:44,191][20347] Updated weights for policy 0, policy_version 737220 (0.0009) [2023-12-27 10:43:44,495][20381] Updated weights for policy 1, policy_version 736767 (0.0008) [2023-12-27 10:43:44,541][20381] Updated weights for policy 1, policy_version 736777 (0.0007) [2023-12-27 10:43:44,584][20381] Updated weights for policy 1, policy_version 736787 (0.0005) [2023-12-27 10:43:44,839][20347] Updated weights for policy 0, policy_version 737230 (0.0007) [2023-12-27 10:43:44,901][20347] Updated weights for policy 0, policy_version 737240 (0.0007) [2023-12-27 10:43:44,964][20347] Updated weights for policy 0, policy_version 737250 (0.0006) [2023-12-27 10:43:45,298][20381] Updated weights for policy 1, policy_version 736797 (0.0008) [2023-12-27 10:43:45,364][20381] Updated weights for policy 1, policy_version 736807 (0.0007) [2023-12-27 10:43:45,418][20381] Updated weights for policy 1, policy_version 736817 (0.0008) [2023-12-27 10:43:45,659][20347] Updated weights for policy 0, policy_version 737260 (0.0008) [2023-12-27 10:43:45,719][20347] Updated weights for policy 0, policy_version 737270 (0.0008) [2023-12-27 10:43:45,773][20347] Updated weights for policy 0, policy_version 737280 (0.0007) [2023-12-27 10:43:46,143][20381] Updated weights for policy 1, policy_version 736827 (0.0007) [2023-12-27 10:43:46,197][20381] Updated weights for policy 1, policy_version 736837 (0.0006) [2023-12-27 10:43:46,257][20381] Updated weights for policy 1, policy_version 736847 (0.0005) [2023-12-27 10:43:46,409][20347] Updated weights for policy 0, policy_version 737290 (0.0007) [2023-12-27 10:43:46,464][20347] Updated weights for policy 0, policy_version 737300 (0.0007) [2023-12-27 10:43:46,512][20347] Updated weights for policy 0, policy_version 737310 (0.0009) [2023-12-27 10:43:46,570][20347] Updated weights for policy 0, policy_version 737320 (0.0006) [2023-12-27 10:43:46,882][20381] Updated weights for policy 1, policy_version 736857 (0.0005) [2023-12-27 10:43:46,935][20381] Updated weights for policy 1, policy_version 736867 (0.0008) [2023-12-27 10:43:46,992][20381] Updated weights for policy 1, policy_version 736877 (0.0009) [2023-12-27 10:43:47,048][20381] Updated weights for policy 1, policy_version 736887 (0.0008) [2023-12-27 10:43:47,190][20347] Updated weights for policy 0, policy_version 737330 (0.0011) [2023-12-27 10:43:47,238][20347] Updated weights for policy 0, policy_version 737340 (0.0010) [2023-12-27 10:43:47,290][20347] Updated weights for policy 0, policy_version 737350 (0.0010) [2023-12-27 10:43:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 377454592. Throughput: 0: 10131.5, 1: 10237.5. Samples: 377459992. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:43:47,363][19170] Avg episode reward: [(0, '951.200'), (1, '459.020')] [2023-12-27 10:43:47,767][20381] Updated weights for policy 1, policy_version 736897 (0.0007) [2023-12-27 10:43:47,814][20381] Updated weights for policy 1, policy_version 736907 (0.0008) [2023-12-27 10:43:47,875][20381] Updated weights for policy 1, policy_version 736917 (0.0007) [2023-12-27 10:43:47,961][20347] Updated weights for policy 0, policy_version 737360 (0.0011) [2023-12-27 10:43:48,017][20347] Updated weights for policy 0, policy_version 737370 (0.0011) [2023-12-27 10:43:48,063][20347] Updated weights for policy 0, policy_version 737380 (0.0010) [2023-12-27 10:43:48,562][20381] Updated weights for policy 1, policy_version 736927 (0.0006) [2023-12-27 10:43:48,617][20381] Updated weights for policy 1, policy_version 736937 (0.0005) [2023-12-27 10:43:48,663][20381] Updated weights for policy 1, policy_version 736947 (0.0006) [2023-12-27 10:43:48,760][20347] Updated weights for policy 0, policy_version 737390 (0.0010) [2023-12-27 10:43:48,816][20347] Updated weights for policy 0, policy_version 737400 (0.0011) [2023-12-27 10:43:48,872][20347] Updated weights for policy 0, policy_version 737410 (0.0010) [2023-12-27 10:43:49,318][20381] Updated weights for policy 1, policy_version 736957 (0.0007) [2023-12-27 10:43:49,397][20381] Updated weights for policy 1, policy_version 736967 (0.0007) [2023-12-27 10:43:49,454][20381] Updated weights for policy 1, policy_version 736977 (0.0007) [2023-12-27 10:43:49,626][20347] Updated weights for policy 0, policy_version 737420 (0.0010) [2023-12-27 10:43:49,681][20347] Updated weights for policy 0, policy_version 737430 (0.0008) [2023-12-27 10:43:49,737][20347] Updated weights for policy 0, policy_version 737440 (0.0007) [2023-12-27 10:43:50,085][20381] Updated weights for policy 1, policy_version 736987 (0.0006) [2023-12-27 10:43:50,142][20381] Updated weights for policy 1, policy_version 736997 (0.0008) [2023-12-27 10:43:50,199][20381] Updated weights for policy 1, policy_version 737007 (0.0009) [2023-12-27 10:43:50,453][20347] Updated weights for policy 0, policy_version 737450 (0.0007) [2023-12-27 10:43:50,501][20347] Updated weights for policy 0, policy_version 737460 (0.0007) [2023-12-27 10:43:50,550][20347] Updated weights for policy 0, policy_version 737470 (0.0007) [2023-12-27 10:43:50,610][20347] Updated weights for policy 0, policy_version 737480 (0.0008) [2023-12-27 10:43:50,913][20381] Updated weights for policy 1, policy_version 737017 (0.0007) [2023-12-27 10:43:50,974][20381] Updated weights for policy 1, policy_version 737027 (0.0008) [2023-12-27 10:43:51,038][20381] Updated weights for policy 1, policy_version 737037 (0.0008) [2023-12-27 10:43:51,102][20381] Updated weights for policy 1, policy_version 737047 (0.0009) [2023-12-27 10:43:51,353][20347] Updated weights for policy 0, policy_version 737490 (0.0008) [2023-12-27 10:43:51,420][20347] Updated weights for policy 0, policy_version 737500 (0.0009) [2023-12-27 10:43:51,484][20347] Updated weights for policy 0, policy_version 737510 (0.0008) [2023-12-27 10:43:51,860][20381] Updated weights for policy 1, policy_version 737057 (0.0008) [2023-12-27 10:43:51,919][20381] Updated weights for policy 1, policy_version 737067 (0.0008) [2023-12-27 10:43:51,982][20381] Updated weights for policy 1, policy_version 737077 (0.0008) [2023-12-27 10:43:52,219][20347] Updated weights for policy 0, policy_version 737520 (0.0006) [2023-12-27 10:43:52,285][20347] Updated weights for policy 0, policy_version 737530 (0.0006) [2023-12-27 10:43:52,343][20347] Updated weights for policy 0, policy_version 737540 (0.0007) [2023-12-27 10:43:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 377544704. Throughput: 0: 10061.9, 1: 10193.1. Samples: 377520476. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:43:52,363][19170] Avg episode reward: [(0, '970.650'), (1, '417.180')] [2023-12-27 10:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000737544_188833792.pth... [2023-12-27 10:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000737080_188719104.pth... [2023-12-27 10:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000735896_188416000.pth [2023-12-27 10:43:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000736360_188530688.pth [2023-12-27 10:43:52,718][20381] Updated weights for policy 1, policy_version 737087 (0.0010) [2023-12-27 10:43:52,773][20381] Updated weights for policy 1, policy_version 737097 (0.0008) [2023-12-27 10:43:52,824][20381] Updated weights for policy 1, policy_version 737107 (0.0009) [2023-12-27 10:43:53,037][20347] Updated weights for policy 0, policy_version 737550 (0.0008) [2023-12-27 10:43:53,090][20347] Updated weights for policy 0, policy_version 737560 (0.0006) [2023-12-27 10:43:53,147][20347] Updated weights for policy 0, policy_version 737570 (0.0005) [2023-12-27 10:43:53,422][20381] Updated weights for policy 1, policy_version 737117 (0.0008) [2023-12-27 10:43:53,473][20381] Updated weights for policy 1, policy_version 737127 (0.0005) [2023-12-27 10:43:53,527][20381] Updated weights for policy 1, policy_version 737137 (0.0005) [2023-12-27 10:43:53,799][20347] Updated weights for policy 0, policy_version 737580 (0.0007) [2023-12-27 10:43:53,859][20347] Updated weights for policy 0, policy_version 737590 (0.0007) [2023-12-27 10:43:53,922][20347] Updated weights for policy 0, policy_version 737600 (0.0007) [2023-12-27 10:43:54,049][20381] Updated weights for policy 1, policy_version 737147 (0.0005) [2023-12-27 10:43:54,107][20381] Updated weights for policy 1, policy_version 737157 (0.0005) [2023-12-27 10:43:54,156][20381] Updated weights for policy 1, policy_version 737167 (0.0005) [2023-12-27 10:43:54,649][20347] Updated weights for policy 0, policy_version 737610 (0.0008) [2023-12-27 10:43:54,699][20347] Updated weights for policy 0, policy_version 737620 (0.0007) [2023-12-27 10:43:54,745][20347] Updated weights for policy 0, policy_version 737630 (0.0005) [2023-12-27 10:43:54,764][20381] Updated weights for policy 1, policy_version 737177 (0.0007) [2023-12-27 10:43:54,793][20347] Updated weights for policy 0, policy_version 737640 (0.0005) [2023-12-27 10:43:54,823][20381] Updated weights for policy 1, policy_version 737187 (0.0006) [2023-12-27 10:43:54,889][20381] Updated weights for policy 1, policy_version 737197 (0.0005) [2023-12-27 10:43:54,945][20381] Updated weights for policy 1, policy_version 737207 (0.0006) [2023-12-27 10:43:55,510][20347] Updated weights for policy 0, policy_version 737650 (0.0007) [2023-12-27 10:43:55,546][20381] Updated weights for policy 1, policy_version 737217 (0.0007) [2023-12-27 10:43:55,563][20347] Updated weights for policy 0, policy_version 737660 (0.0005) [2023-12-27 10:43:55,603][20381] Updated weights for policy 1, policy_version 737227 (0.0008) [2023-12-27 10:43:55,619][20347] Updated weights for policy 0, policy_version 737670 (0.0005) [2023-12-27 10:43:55,660][20381] Updated weights for policy 1, policy_version 737237 (0.0007) [2023-12-27 10:43:56,243][20347] Updated weights for policy 0, policy_version 737680 (0.0008) [2023-12-27 10:43:56,290][20347] Updated weights for policy 0, policy_version 737690 (0.0008) [2023-12-27 10:43:56,337][20347] Updated weights for policy 0, policy_version 737700 (0.0007) [2023-12-27 10:43:56,353][20381] Updated weights for policy 1, policy_version 737247 (0.0007) [2023-12-27 10:43:56,404][20381] Updated weights for policy 1, policy_version 737257 (0.0008) [2023-12-27 10:43:56,465][20381] Updated weights for policy 1, policy_version 737267 (0.0006) [2023-12-27 10:43:57,026][20347] Updated weights for policy 0, policy_version 737710 (0.0008) [2023-12-27 10:43:57,086][20347] Updated weights for policy 0, policy_version 737720 (0.0006) [2023-12-27 10:43:57,140][20347] Updated weights for policy 0, policy_version 737730 (0.0007) [2023-12-27 10:43:57,152][20381] Updated weights for policy 1, policy_version 737277 (0.0009) [2023-12-27 10:43:57,203][20381] Updated weights for policy 1, policy_version 737287 (0.0010) [2023-12-27 10:43:57,257][20381] Updated weights for policy 1, policy_version 737297 (0.0007) [2023-12-27 10:43:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 377659392. Throughput: 0: 10062.8, 1: 10232.6. Samples: 377643764. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:43:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '404.870')] [2023-12-27 10:43:57,758][20347] Updated weights for policy 0, policy_version 737740 (0.0006) [2023-12-27 10:43:57,811][20347] Updated weights for policy 0, policy_version 737750 (0.0005) [2023-12-27 10:43:57,823][20381] Updated weights for policy 1, policy_version 737307 (0.0006) [2023-12-27 10:43:57,871][20381] Updated weights for policy 1, policy_version 737317 (0.0005) [2023-12-27 10:43:57,875][20347] Updated weights for policy 0, policy_version 737760 (0.0008) [2023-12-27 10:43:57,918][20381] Updated weights for policy 1, policy_version 737327 (0.0005) [2023-12-27 10:43:58,550][20347] Updated weights for policy 0, policy_version 737770 (0.0010) [2023-12-27 10:43:58,584][20381] Updated weights for policy 1, policy_version 737337 (0.0006) [2023-12-27 10:43:58,619][20347] Updated weights for policy 0, policy_version 737780 (0.0010) [2023-12-27 10:43:58,644][20381] Updated weights for policy 1, policy_version 737347 (0.0010) [2023-12-27 10:43:58,674][20347] Updated weights for policy 0, policy_version 737790 (0.0010) [2023-12-27 10:43:58,708][20381] Updated weights for policy 1, policy_version 737357 (0.0011) [2023-12-27 10:43:58,740][20347] Updated weights for policy 0, policy_version 737800 (0.0010) [2023-12-27 10:43:58,769][20381] Updated weights for policy 1, policy_version 737367 (0.0009) [2023-12-27 10:43:59,582][20347] Updated weights for policy 0, policy_version 737810 (0.0011) [2023-12-27 10:43:59,621][20381] Updated weights for policy 1, policy_version 737377 (0.0007) [2023-12-27 10:43:59,639][20347] Updated weights for policy 0, policy_version 737820 (0.0010) [2023-12-27 10:43:59,678][20381] Updated weights for policy 1, policy_version 737387 (0.0007) [2023-12-27 10:43:59,702][20347] Updated weights for policy 0, policy_version 737830 (0.0011) [2023-12-27 10:43:59,741][20381] Updated weights for policy 1, policy_version 737397 (0.0008) [2023-12-27 10:44:00,400][20347] Updated weights for policy 0, policy_version 737840 (0.0009) [2023-12-27 10:44:00,418][20381] Updated weights for policy 1, policy_version 737407 (0.0010) [2023-12-27 10:44:00,455][20347] Updated weights for policy 0, policy_version 737850 (0.0008) [2023-12-27 10:44:00,473][20381] Updated weights for policy 1, policy_version 737417 (0.0010) [2023-12-27 10:44:00,500][20347] Updated weights for policy 0, policy_version 737860 (0.0006) [2023-12-27 10:44:00,521][20381] Updated weights for policy 1, policy_version 737427 (0.0010) [2023-12-27 10:44:01,093][20347] Updated weights for policy 0, policy_version 737870 (0.0006) [2023-12-27 10:44:01,158][20347] Updated weights for policy 0, policy_version 737880 (0.0008) [2023-12-27 10:44:01,202][20381] Updated weights for policy 1, policy_version 737437 (0.0008) [2023-12-27 10:44:01,218][20347] Updated weights for policy 0, policy_version 737890 (0.0006) [2023-12-27 10:44:01,261][20381] Updated weights for policy 1, policy_version 737447 (0.0009) [2023-12-27 10:44:01,322][20381] Updated weights for policy 1, policy_version 737457 (0.0006) [2023-12-27 10:44:01,897][20347] Updated weights for policy 0, policy_version 737900 (0.0008) [2023-12-27 10:44:01,957][20347] Updated weights for policy 0, policy_version 737910 (0.0008) [2023-12-27 10:44:02,012][20347] Updated weights for policy 0, policy_version 737920 (0.0010) [2023-12-27 10:44:02,029][20381] Updated weights for policy 1, policy_version 737467 (0.0007) [2023-12-27 10:44:02,085][20381] Updated weights for policy 1, policy_version 737477 (0.0008) [2023-12-27 10:44:02,132][20381] Updated weights for policy 1, policy_version 737487 (0.0009) [2023-12-27 10:44:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 377757696. Throughput: 0: 10032.6, 1: 10218.6. Samples: 377764676. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:02,363][19170] Avg episode reward: [(0, '990.270'), (1, '559.350')] [2023-12-27 10:44:02,674][20347] Updated weights for policy 0, policy_version 737930 (0.0007) [2023-12-27 10:44:02,727][20347] Updated weights for policy 0, policy_version 737940 (0.0007) [2023-12-27 10:44:02,781][20347] Updated weights for policy 0, policy_version 737950 (0.0005) [2023-12-27 10:44:02,836][20347] Updated weights for policy 0, policy_version 737960 (0.0005) [2023-12-27 10:44:02,868][20381] Updated weights for policy 1, policy_version 737497 (0.0007) [2023-12-27 10:44:02,921][20381] Updated weights for policy 1, policy_version 737507 (0.0008) [2023-12-27 10:44:02,969][20381] Updated weights for policy 1, policy_version 737517 (0.0007) [2023-12-27 10:44:03,017][20381] Updated weights for policy 1, policy_version 737527 (0.0008) [2023-12-27 10:44:03,470][20347] Updated weights for policy 0, policy_version 737970 (0.0010) [2023-12-27 10:44:03,517][20347] Updated weights for policy 0, policy_version 737980 (0.0010) [2023-12-27 10:44:03,561][20347] Updated weights for policy 0, policy_version 737990 (0.0010) [2023-12-27 10:44:03,664][20381] Updated weights for policy 1, policy_version 737537 (0.0007) [2023-12-27 10:44:03,728][20381] Updated weights for policy 1, policy_version 737547 (0.0008) [2023-12-27 10:44:03,780][20381] Updated weights for policy 1, policy_version 737557 (0.0008) [2023-12-27 10:44:04,337][20347] Updated weights for policy 0, policy_version 738000 (0.0011) [2023-12-27 10:44:04,402][20347] Updated weights for policy 0, policy_version 738010 (0.0011) [2023-12-27 10:44:04,466][20347] Updated weights for policy 0, policy_version 738020 (0.0011) [2023-12-27 10:44:04,512][20381] Updated weights for policy 1, policy_version 737567 (0.0007) [2023-12-27 10:44:04,566][20381] Updated weights for policy 1, policy_version 737577 (0.0007) [2023-12-27 10:44:04,624][20381] Updated weights for policy 1, policy_version 737587 (0.0007) [2023-12-27 10:44:05,111][20347] Updated weights for policy 0, policy_version 738030 (0.0009) [2023-12-27 10:44:05,159][20347] Updated weights for policy 0, policy_version 738040 (0.0007) [2023-12-27 10:44:05,218][20347] Updated weights for policy 0, policy_version 738050 (0.0007) [2023-12-27 10:44:05,284][20381] Updated weights for policy 1, policy_version 737597 (0.0007) [2023-12-27 10:44:05,338][20381] Updated weights for policy 1, policy_version 737607 (0.0008) [2023-12-27 10:44:05,394][20381] Updated weights for policy 1, policy_version 737617 (0.0011) [2023-12-27 10:44:05,939][20347] Updated weights for policy 0, policy_version 738060 (0.0008) [2023-12-27 10:44:05,996][20347] Updated weights for policy 0, policy_version 738070 (0.0007) [2023-12-27 10:44:06,046][20347] Updated weights for policy 0, policy_version 738080 (0.0007) [2023-12-27 10:44:06,076][20381] Updated weights for policy 1, policy_version 737627 (0.0011) [2023-12-27 10:44:06,134][20381] Updated weights for policy 1, policy_version 737637 (0.0009) [2023-12-27 10:44:06,200][20381] Updated weights for policy 1, policy_version 737647 (0.0006) [2023-12-27 10:44:06,750][20347] Updated weights for policy 0, policy_version 738090 (0.0007) [2023-12-27 10:44:06,796][20381] Updated weights for policy 1, policy_version 737657 (0.0006) [2023-12-27 10:44:06,801][20347] Updated weights for policy 0, policy_version 738100 (0.0007) [2023-12-27 10:44:06,844][20381] Updated weights for policy 1, policy_version 737667 (0.0010) [2023-12-27 10:44:06,853][20347] Updated weights for policy 0, policy_version 738110 (0.0005) [2023-12-27 10:44:06,898][20381] Updated weights for policy 1, policy_version 737677 (0.0008) [2023-12-27 10:44:06,913][20347] Updated weights for policy 0, policy_version 738120 (0.0009) [2023-12-27 10:44:06,950][20381] Updated weights for policy 1, policy_version 737687 (0.0011) [2023-12-27 10:44:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 377856000. Throughput: 0: 10071.1, 1: 10230.6. Samples: 377826536. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '636.510')] [2023-12-27 10:44:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000738120_188981248.pth... [2023-12-27 10:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000737688_188874752.pth... [2023-12-27 10:44:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000736968_188686336.pth [2023-12-27 10:44:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000736504_188571648.pth [2023-12-27 10:44:07,573][20347] Updated weights for policy 0, policy_version 738130 (0.0007) [2023-12-27 10:44:07,632][20347] Updated weights for policy 0, policy_version 738140 (0.0007) [2023-12-27 10:44:07,694][20347] Updated weights for policy 0, policy_version 738150 (0.0006) [2023-12-27 10:44:07,716][20381] Updated weights for policy 1, policy_version 737697 (0.0007) [2023-12-27 10:44:07,771][20381] Updated weights for policy 1, policy_version 737707 (0.0007) [2023-12-27 10:44:07,817][20381] Updated weights for policy 1, policy_version 737717 (0.0007) [2023-12-27 10:44:08,307][20347] Updated weights for policy 0, policy_version 738160 (0.0007) [2023-12-27 10:44:08,375][20347] Updated weights for policy 0, policy_version 738170 (0.0008) [2023-12-27 10:44:08,434][20347] Updated weights for policy 0, policy_version 738180 (0.0008) [2023-12-27 10:44:08,463][20381] Updated weights for policy 1, policy_version 737727 (0.0007) [2023-12-27 10:44:08,519][20381] Updated weights for policy 1, policy_version 737737 (0.0008) [2023-12-27 10:44:08,578][20381] Updated weights for policy 1, policy_version 737747 (0.0008) [2023-12-27 10:44:09,139][20347] Updated weights for policy 0, policy_version 738190 (0.0007) [2023-12-27 10:44:09,199][20347] Updated weights for policy 0, policy_version 738200 (0.0007) [2023-12-27 10:44:09,257][20347] Updated weights for policy 0, policy_version 738210 (0.0008) [2023-12-27 10:44:09,298][20381] Updated weights for policy 1, policy_version 737757 (0.0007) [2023-12-27 10:44:09,362][20381] Updated weights for policy 1, policy_version 737767 (0.0008) [2023-12-27 10:44:09,430][20381] Updated weights for policy 1, policy_version 737777 (0.0008) [2023-12-27 10:44:10,033][20347] Updated weights for policy 0, policy_version 738220 (0.0008) [2023-12-27 10:44:10,097][20347] Updated weights for policy 0, policy_version 738230 (0.0006) [2023-12-27 10:44:10,145][20381] Updated weights for policy 1, policy_version 737787 (0.0009) [2023-12-27 10:44:10,156][20347] Updated weights for policy 0, policy_version 738240 (0.0008) [2023-12-27 10:44:10,199][20381] Updated weights for policy 1, policy_version 737797 (0.0011) [2023-12-27 10:44:10,249][20381] Updated weights for policy 1, policy_version 737807 (0.0011) [2023-12-27 10:44:10,865][20347] Updated weights for policy 0, policy_version 738250 (0.0008) [2023-12-27 10:44:10,918][20347] Updated weights for policy 0, policy_version 738260 (0.0006) [2023-12-27 10:44:10,950][20381] Updated weights for policy 1, policy_version 737817 (0.0008) [2023-12-27 10:44:10,973][20347] Updated weights for policy 0, policy_version 738270 (0.0008) [2023-12-27 10:44:11,013][20381] Updated weights for policy 1, policy_version 737827 (0.0006) [2023-12-27 10:44:11,038][20347] Updated weights for policy 0, policy_version 738280 (0.0008) [2023-12-27 10:44:11,075][20381] Updated weights for policy 1, policy_version 737837 (0.0010) [2023-12-27 10:44:11,136][20381] Updated weights for policy 1, policy_version 737847 (0.0008) [2023-12-27 10:44:11,785][20347] Updated weights for policy 0, policy_version 738290 (0.0007) [2023-12-27 10:44:11,849][20347] Updated weights for policy 0, policy_version 738300 (0.0008) [2023-12-27 10:44:11,883][20381] Updated weights for policy 1, policy_version 737857 (0.0006) [2023-12-27 10:44:11,914][20347] Updated weights for policy 0, policy_version 738310 (0.0009) [2023-12-27 10:44:11,936][20381] Updated weights for policy 1, policy_version 737867 (0.0007) [2023-12-27 10:44:11,997][20381] Updated weights for policy 1, policy_version 737877 (0.0008) [2023-12-27 10:44:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 377954304. Throughput: 0: 10070.3, 1: 10209.5. Samples: 377946620. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '671.140')] [2023-12-27 10:44:12,642][20347] Updated weights for policy 0, policy_version 738320 (0.0008) [2023-12-27 10:44:12,703][20347] Updated weights for policy 0, policy_version 738330 (0.0008) [2023-12-27 10:44:12,740][20381] Updated weights for policy 1, policy_version 737887 (0.0007) [2023-12-27 10:44:12,764][20347] Updated weights for policy 0, policy_version 738340 (0.0006) [2023-12-27 10:44:12,802][20381] Updated weights for policy 1, policy_version 737897 (0.0006) [2023-12-27 10:44:12,864][20381] Updated weights for policy 1, policy_version 737907 (0.0007) [2023-12-27 10:44:13,468][20347] Updated weights for policy 0, policy_version 738350 (0.0006) [2023-12-27 10:44:13,507][20381] Updated weights for policy 1, policy_version 737917 (0.0008) [2023-12-27 10:44:13,518][20347] Updated weights for policy 0, policy_version 738360 (0.0006) [2023-12-27 10:44:13,567][20381] Updated weights for policy 1, policy_version 737927 (0.0007) [2023-12-27 10:44:13,573][20347] Updated weights for policy 0, policy_version 738370 (0.0005) [2023-12-27 10:44:13,620][20381] Updated weights for policy 1, policy_version 737937 (0.0006) [2023-12-27 10:44:14,179][20381] Updated weights for policy 1, policy_version 737947 (0.0006) [2023-12-27 10:44:14,239][20381] Updated weights for policy 1, policy_version 737957 (0.0006) [2023-12-27 10:44:14,298][20347] Updated weights for policy 0, policy_version 738380 (0.0007) [2023-12-27 10:44:14,302][20381] Updated weights for policy 1, policy_version 737967 (0.0005) [2023-12-27 10:44:14,351][20347] Updated weights for policy 0, policy_version 738390 (0.0007) [2023-12-27 10:44:14,403][20347] Updated weights for policy 0, policy_version 738400 (0.0007) [2023-12-27 10:44:14,930][20381] Updated weights for policy 1, policy_version 737977 (0.0007) [2023-12-27 10:44:14,988][20381] Updated weights for policy 1, policy_version 737987 (0.0008) [2023-12-27 10:44:15,046][20381] Updated weights for policy 1, policy_version 737997 (0.0006) [2023-12-27 10:44:15,108][20381] Updated weights for policy 1, policy_version 738007 (0.0006) [2023-12-27 10:44:15,152][20347] Updated weights for policy 0, policy_version 738410 (0.0008) [2023-12-27 10:44:15,215][20347] Updated weights for policy 0, policy_version 738420 (0.0008) [2023-12-27 10:44:15,269][20347] Updated weights for policy 0, policy_version 738430 (0.0008) [2023-12-27 10:44:15,327][20347] Updated weights for policy 0, policy_version 738440 (0.0008) [2023-12-27 10:44:15,804][20381] Updated weights for policy 1, policy_version 738017 (0.0008) [2023-12-27 10:44:15,853][20381] Updated weights for policy 1, policy_version 738027 (0.0007) [2023-12-27 10:44:15,905][20381] Updated weights for policy 1, policy_version 738037 (0.0007) [2023-12-27 10:44:16,050][20347] Updated weights for policy 0, policy_version 738450 (0.0007) [2023-12-27 10:44:16,094][20347] Updated weights for policy 0, policy_version 738460 (0.0007) [2023-12-27 10:44:16,151][20347] Updated weights for policy 0, policy_version 738470 (0.0007) [2023-12-27 10:44:16,610][20381] Updated weights for policy 1, policy_version 738047 (0.0008) [2023-12-27 10:44:16,659][20381] Updated weights for policy 1, policy_version 738057 (0.0008) [2023-12-27 10:44:16,720][20381] Updated weights for policy 1, policy_version 738067 (0.0008) [2023-12-27 10:44:16,853][20347] Updated weights for policy 0, policy_version 738480 (0.0008) [2023-12-27 10:44:16,911][20347] Updated weights for policy 0, policy_version 738490 (0.0008) [2023-12-27 10:44:16,966][20347] Updated weights for policy 0, policy_version 738500 (0.0008) [2023-12-27 10:44:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 20521.7). Total num frames: 378052608. Throughput: 0: 9964.7, 1: 10197.5. Samples: 378065568. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:17,363][19170] Avg episode reward: [(0, '990.230'), (1, '675.940')] [2023-12-27 10:44:17,379][20381] Updated weights for policy 1, policy_version 738077 (0.0007) [2023-12-27 10:44:17,434][20381] Updated weights for policy 1, policy_version 738087 (0.0008) [2023-12-27 10:44:17,482][20381] Updated weights for policy 1, policy_version 738097 (0.0007) [2023-12-27 10:44:17,668][20347] Updated weights for policy 0, policy_version 738510 (0.0008) [2023-12-27 10:44:17,723][20347] Updated weights for policy 0, policy_version 738520 (0.0007) [2023-12-27 10:44:17,778][20347] Updated weights for policy 0, policy_version 738530 (0.0007) [2023-12-27 10:44:18,187][20381] Updated weights for policy 1, policy_version 738107 (0.0008) [2023-12-27 10:44:18,237][20381] Updated weights for policy 1, policy_version 738117 (0.0008) [2023-12-27 10:44:18,280][20381] Updated weights for policy 1, policy_version 738127 (0.0008) [2023-12-27 10:44:18,461][20347] Updated weights for policy 0, policy_version 738540 (0.0008) [2023-12-27 10:44:18,510][20347] Updated weights for policy 0, policy_version 738550 (0.0008) [2023-12-27 10:44:18,558][20347] Updated weights for policy 0, policy_version 738560 (0.0007) [2023-12-27 10:44:18,995][20381] Updated weights for policy 1, policy_version 738137 (0.0008) [2023-12-27 10:44:19,054][20381] Updated weights for policy 1, policy_version 738147 (0.0011) [2023-12-27 10:44:19,112][20381] Updated weights for policy 1, policy_version 738157 (0.0008) [2023-12-27 10:44:19,168][20381] Updated weights for policy 1, policy_version 738167 (0.0010) [2023-12-27 10:44:19,288][20347] Updated weights for policy 0, policy_version 738570 (0.0007) [2023-12-27 10:44:19,347][20347] Updated weights for policy 0, policy_version 738580 (0.0007) [2023-12-27 10:44:19,408][20347] Updated weights for policy 0, policy_version 738590 (0.0008) [2023-12-27 10:44:19,462][20347] Updated weights for policy 0, policy_version 738600 (0.0007) [2023-12-27 10:44:19,878][20381] Updated weights for policy 1, policy_version 738177 (0.0011) [2023-12-27 10:44:19,930][20381] Updated weights for policy 1, policy_version 738187 (0.0011) [2023-12-27 10:44:19,981][20381] Updated weights for policy 1, policy_version 738197 (0.0011) [2023-12-27 10:44:20,241][20347] Updated weights for policy 0, policy_version 738610 (0.0007) [2023-12-27 10:44:20,289][20347] Updated weights for policy 0, policy_version 738620 (0.0006) [2023-12-27 10:44:20,346][20347] Updated weights for policy 0, policy_version 738630 (0.0008) [2023-12-27 10:44:20,740][20381] Updated weights for policy 1, policy_version 738207 (0.0009) [2023-12-27 10:44:20,804][20381] Updated weights for policy 1, policy_version 738217 (0.0007) [2023-12-27 10:44:20,864][20381] Updated weights for policy 1, policy_version 738227 (0.0008) [2023-12-27 10:44:21,064][20347] Updated weights for policy 0, policy_version 738640 (0.0007) [2023-12-27 10:44:21,110][20291] KL-divergence is very high: 120.2759 [2023-12-27 10:44:21,128][20347] Updated weights for policy 0, policy_version 738650 (0.0008) [2023-12-27 10:44:21,188][20347] Updated weights for policy 0, policy_version 738660 (0.0008) [2023-12-27 10:44:21,611][20381] Updated weights for policy 1, policy_version 738237 (0.0007) [2023-12-27 10:44:21,677][20381] Updated weights for policy 1, policy_version 738247 (0.0008) [2023-12-27 10:44:21,742][20381] Updated weights for policy 1, policy_version 738257 (0.0007) [2023-12-27 10:44:21,959][20347] Updated weights for policy 0, policy_version 738670 (0.0008) [2023-12-27 10:44:22,021][20347] Updated weights for policy 0, policy_version 738680 (0.0006) [2023-12-27 10:44:22,087][20347] Updated weights for policy 0, policy_version 738690 (0.0008) [2023-12-27 10:44:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 378150912. Throughput: 0: 9965.5, 1: 10105.1. Samples: 378124456. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:22,363][19170] Avg episode reward: [(0, '980.540'), (1, '565.060')] [2023-12-27 10:44:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000738264_189022208.pth... [2023-12-27 10:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000738696_189128704.pth... [2023-12-27 10:44:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000737544_188833792.pth [2023-12-27 10:44:22,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000737080_188719104.pth [2023-12-27 10:44:22,438][20381] Updated weights for policy 1, policy_version 738267 (0.0007) [2023-12-27 10:44:22,505][20381] Updated weights for policy 1, policy_version 738277 (0.0008) [2023-12-27 10:44:22,565][20381] Updated weights for policy 1, policy_version 738287 (0.0008) [2023-12-27 10:44:22,746][20347] Updated weights for policy 0, policy_version 738700 (0.0007) [2023-12-27 10:44:22,814][20347] Updated weights for policy 0, policy_version 738710 (0.0005) [2023-12-27 10:44:22,870][20347] Updated weights for policy 0, policy_version 738720 (0.0007) [2023-12-27 10:44:23,250][20381] Updated weights for policy 1, policy_version 738297 (0.0007) [2023-12-27 10:44:23,301][20381] Updated weights for policy 1, policy_version 738307 (0.0005) [2023-12-27 10:44:23,355][20381] Updated weights for policy 1, policy_version 738317 (0.0007) [2023-12-27 10:44:23,403][20381] Updated weights for policy 1, policy_version 738327 (0.0007) [2023-12-27 10:44:23,546][20347] Updated weights for policy 0, policy_version 738730 (0.0008) [2023-12-27 10:44:23,606][20347] Updated weights for policy 0, policy_version 738740 (0.0008) [2023-12-27 10:44:23,663][20347] Updated weights for policy 0, policy_version 738750 (0.0008) [2023-12-27 10:44:23,724][20347] Updated weights for policy 0, policy_version 738760 (0.0007) [2023-12-27 10:44:24,064][20381] Updated weights for policy 1, policy_version 738337 (0.0007) [2023-12-27 10:44:24,111][20381] Updated weights for policy 1, policy_version 738347 (0.0008) [2023-12-27 10:44:24,169][20381] Updated weights for policy 1, policy_version 738357 (0.0008) [2023-12-27 10:44:24,419][20347] Updated weights for policy 0, policy_version 738770 (0.0007) [2023-12-27 10:44:24,477][20347] Updated weights for policy 0, policy_version 738780 (0.0009) [2023-12-27 10:44:24,522][20347] Updated weights for policy 0, policy_version 738790 (0.0007) [2023-12-27 10:44:24,867][20381] Updated weights for policy 1, policy_version 738367 (0.0008) [2023-12-27 10:44:24,920][20381] Updated weights for policy 1, policy_version 738377 (0.0009) [2023-12-27 10:44:24,974][20381] Updated weights for policy 1, policy_version 738387 (0.0008) [2023-12-27 10:44:25,234][20347] Updated weights for policy 0, policy_version 738800 (0.0007) [2023-12-27 10:44:25,298][20347] Updated weights for policy 0, policy_version 738810 (0.0007) [2023-12-27 10:44:25,352][20347] Updated weights for policy 0, policy_version 738820 (0.0007) [2023-12-27 10:44:25,553][20381] Updated weights for policy 1, policy_version 738397 (0.0005) [2023-12-27 10:44:25,602][20381] Updated weights for policy 1, policy_version 738407 (0.0006) [2023-12-27 10:44:25,647][20381] Updated weights for policy 1, policy_version 738417 (0.0006) [2023-12-27 10:44:26,079][20347] Updated weights for policy 0, policy_version 738830 (0.0008) [2023-12-27 10:44:26,132][20347] Updated weights for policy 0, policy_version 738840 (0.0007) [2023-12-27 10:44:26,181][20291] KL-divergence is very high: 147.5446 [2023-12-27 10:44:26,183][20347] Updated weights for policy 0, policy_version 738850 (0.0009) [2023-12-27 10:44:26,194][20291] KL-divergence is very high: 103.2722 [2023-12-27 10:44:26,261][20381] Updated weights for policy 1, policy_version 738427 (0.0007) [2023-12-27 10:44:26,319][20381] Updated weights for policy 1, policy_version 738437 (0.0008) [2023-12-27 10:44:26,372][20381] Updated weights for policy 1, policy_version 738447 (0.0008) [2023-12-27 10:44:26,876][20347] Updated weights for policy 0, policy_version 738860 (0.0009) [2023-12-27 10:44:26,926][20347] Updated weights for policy 0, policy_version 738870 (0.0009) [2023-12-27 10:44:26,981][20347] Updated weights for policy 0, policy_version 738880 (0.0006) [2023-12-27 10:44:26,982][20381] Updated weights for policy 1, policy_version 738457 (0.0007) [2023-12-27 10:44:27,035][20381] Updated weights for policy 1, policy_version 738467 (0.0006) [2023-12-27 10:44:27,091][20381] Updated weights for policy 1, policy_version 738477 (0.0007) [2023-12-27 10:44:27,144][20381] Updated weights for policy 1, policy_version 738487 (0.0007) [2023-12-27 10:44:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 378257408. Throughput: 0: 9929.2, 1: 10150.9. Samples: 378245008. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:27,364][19170] Avg episode reward: [(0, '961.550'), (1, '520.240')] [2023-12-27 10:44:27,630][20347] Updated weights for policy 0, policy_version 738890 (0.0010) [2023-12-27 10:44:27,679][20347] Updated weights for policy 0, policy_version 738900 (0.0010) [2023-12-27 10:44:27,739][20347] Updated weights for policy 0, policy_version 738910 (0.0010) [2023-12-27 10:44:27,755][20381] Updated weights for policy 1, policy_version 738497 (0.0010) [2023-12-27 10:44:27,798][20347] Updated weights for policy 0, policy_version 738920 (0.0010) [2023-12-27 10:44:27,807][20381] Updated weights for policy 1, policy_version 738507 (0.0006) [2023-12-27 10:44:27,868][20381] Updated weights for policy 1, policy_version 738517 (0.0006) [2023-12-27 10:44:28,467][20381] Updated weights for policy 1, policy_version 738527 (0.0007) [2023-12-27 10:44:28,476][20347] Updated weights for policy 0, policy_version 738930 (0.0010) [2023-12-27 10:44:28,513][20381] Updated weights for policy 1, policy_version 738537 (0.0007) [2023-12-27 10:44:28,528][20347] Updated weights for policy 0, policy_version 738940 (0.0010) [2023-12-27 10:44:28,562][20381] Updated weights for policy 1, policy_version 738547 (0.0010) [2023-12-27 10:44:28,574][20347] Updated weights for policy 0, policy_version 738950 (0.0009) [2023-12-27 10:44:29,214][20381] Updated weights for policy 1, policy_version 738557 (0.0009) [2023-12-27 10:44:29,242][20347] Updated weights for policy 0, policy_version 738960 (0.0007) [2023-12-27 10:44:29,291][20381] Updated weights for policy 1, policy_version 738567 (0.0009) [2023-12-27 10:44:29,299][20347] Updated weights for policy 0, policy_version 738970 (0.0006) [2023-12-27 10:44:29,355][20381] Updated weights for policy 1, policy_version 738577 (0.0008) [2023-12-27 10:44:29,372][20347] Updated weights for policy 0, policy_version 738980 (0.0010) [2023-12-27 10:44:30,047][20347] Updated weights for policy 0, policy_version 738990 (0.0009) [2023-12-27 10:44:30,064][20381] Updated weights for policy 1, policy_version 738587 (0.0008) [2023-12-27 10:44:30,107][20347] Updated weights for policy 0, policy_version 739000 (0.0008) [2023-12-27 10:44:30,127][20381] Updated weights for policy 1, policy_version 738597 (0.0006) [2023-12-27 10:44:30,168][20347] Updated weights for policy 0, policy_version 739010 (0.0008) [2023-12-27 10:44:30,185][20381] Updated weights for policy 1, policy_version 738607 (0.0007) [2023-12-27 10:44:30,797][20347] Updated weights for policy 0, policy_version 739020 (0.0007) [2023-12-27 10:44:30,801][20381] Updated weights for policy 1, policy_version 738617 (0.0007) [2023-12-27 10:44:30,848][20347] Updated weights for policy 0, policy_version 739030 (0.0005) [2023-12-27 10:44:30,849][20381] Updated weights for policy 1, policy_version 738627 (0.0007) [2023-12-27 10:44:30,897][20347] Updated weights for policy 0, policy_version 739040 (0.0005) [2023-12-27 10:44:30,903][20381] Updated weights for policy 1, policy_version 738637 (0.0007) [2023-12-27 10:44:30,952][20381] Updated weights for policy 1, policy_version 738647 (0.0005) [2023-12-27 10:44:31,516][20347] Updated weights for policy 0, policy_version 739050 (0.0007) [2023-12-27 10:44:31,574][20347] Updated weights for policy 0, policy_version 739060 (0.0007) [2023-12-27 10:44:31,606][20381] Updated weights for policy 1, policy_version 738657 (0.0006) [2023-12-27 10:44:31,637][20347] Updated weights for policy 0, policy_version 739070 (0.0007) [2023-12-27 10:44:31,669][20381] Updated weights for policy 1, policy_version 738667 (0.0008) [2023-12-27 10:44:31,701][20347] Updated weights for policy 0, policy_version 739080 (0.0008) [2023-12-27 10:44:31,739][20381] Updated weights for policy 1, policy_version 738677 (0.0007) [2023-12-27 10:44:32,319][20381] Updated weights for policy 1, policy_version 738687 (0.0010) [2023-12-27 10:44:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 378355712. Throughput: 0: 9962.6, 1: 10290.1. Samples: 378371368. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:32,363][19170] Avg episode reward: [(0, '942.510'), (1, '591.850')] [2023-12-27 10:44:32,383][20381] Updated weights for policy 1, policy_version 738697 (0.0010) [2023-12-27 10:44:32,410][20347] Updated weights for policy 0, policy_version 739090 (0.0008) [2023-12-27 10:44:32,442][20381] Updated weights for policy 1, policy_version 738707 (0.0006) [2023-12-27 10:44:32,462][20347] Updated weights for policy 0, policy_version 739100 (0.0005) [2023-12-27 10:44:32,524][20347] Updated weights for policy 0, policy_version 739110 (0.0007) [2023-12-27 10:44:33,121][20381] Updated weights for policy 1, policy_version 738717 (0.0007) [2023-12-27 10:44:33,186][20381] Updated weights for policy 1, policy_version 738727 (0.0008) [2023-12-27 10:44:33,222][20347] Updated weights for policy 0, policy_version 739120 (0.0006) [2023-12-27 10:44:33,240][20381] Updated weights for policy 1, policy_version 738737 (0.0006) [2023-12-27 10:44:33,270][20347] Updated weights for policy 0, policy_version 739130 (0.0007) [2023-12-27 10:44:33,315][20347] Updated weights for policy 0, policy_version 739140 (0.0008) [2023-12-27 10:44:33,830][20381] Updated weights for policy 1, policy_version 738747 (0.0007) [2023-12-27 10:44:33,881][20381] Updated weights for policy 1, policy_version 738757 (0.0010) [2023-12-27 10:44:33,932][20381] Updated weights for policy 1, policy_version 738767 (0.0010) [2023-12-27 10:44:34,015][20347] Updated weights for policy 0, policy_version 739150 (0.0006) [2023-12-27 10:44:34,071][20347] Updated weights for policy 0, policy_version 739160 (0.0005) [2023-12-27 10:44:34,128][20347] Updated weights for policy 0, policy_version 739170 (0.0007) [2023-12-27 10:44:34,593][20381] Updated weights for policy 1, policy_version 738777 (0.0006) [2023-12-27 10:44:34,651][20381] Updated weights for policy 1, policy_version 738787 (0.0008) [2023-12-27 10:44:34,711][20381] Updated weights for policy 1, policy_version 738797 (0.0007) [2023-12-27 10:44:34,772][20381] Updated weights for policy 1, policy_version 738807 (0.0008) [2023-12-27 10:44:34,821][20347] Updated weights for policy 0, policy_version 739180 (0.0008) [2023-12-27 10:44:34,869][20347] Updated weights for policy 0, policy_version 739190 (0.0007) [2023-12-27 10:44:34,921][20347] Updated weights for policy 0, policy_version 739200 (0.0008) [2023-12-27 10:44:35,461][20381] Updated weights for policy 1, policy_version 738817 (0.0006) [2023-12-27 10:44:35,521][20381] Updated weights for policy 1, policy_version 738827 (0.0006) [2023-12-27 10:44:35,571][20381] Updated weights for policy 1, policy_version 738837 (0.0008) [2023-12-27 10:44:35,644][20347] Updated weights for policy 0, policy_version 739210 (0.0008) [2023-12-27 10:44:35,698][20347] Updated weights for policy 0, policy_version 739220 (0.0007) [2023-12-27 10:44:35,753][20347] Updated weights for policy 0, policy_version 739230 (0.0007) [2023-12-27 10:44:35,807][20347] Updated weights for policy 0, policy_version 739240 (0.0007) [2023-12-27 10:44:36,209][20381] Updated weights for policy 1, policy_version 738847 (0.0008) [2023-12-27 10:44:36,266][20381] Updated weights for policy 1, policy_version 738857 (0.0006) [2023-12-27 10:44:36,318][20381] Updated weights for policy 1, policy_version 738867 (0.0005) [2023-12-27 10:44:36,577][20347] Updated weights for policy 0, policy_version 739250 (0.0008) [2023-12-27 10:44:36,633][20347] Updated weights for policy 0, policy_version 739260 (0.0008) [2023-12-27 10:44:36,688][20347] Updated weights for policy 0, policy_version 739270 (0.0008) [2023-12-27 10:44:36,935][20381] Updated weights for policy 1, policy_version 738877 (0.0006) [2023-12-27 10:44:36,984][20381] Updated weights for policy 1, policy_version 738887 (0.0005) [2023-12-27 10:44:37,037][20381] Updated weights for policy 1, policy_version 738897 (0.0006) [2023-12-27 10:44:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 378462208. Throughput: 0: 9967.7, 1: 10299.7. Samples: 378432508. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:37,363][19170] Avg episode reward: [(0, '933.190'), (1, '287.120')] [2023-12-27 10:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000738904_189186048.pth... [2023-12-27 10:44:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000739272_189276160.pth... [2023-12-27 10:44:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000737688_188874752.pth [2023-12-27 10:44:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000738120_188981248.pth [2023-12-27 10:44:37,445][20347] Updated weights for policy 0, policy_version 739280 (0.0007) [2023-12-27 10:44:37,503][20347] Updated weights for policy 0, policy_version 739290 (0.0007) [2023-12-27 10:44:37,562][20347] Updated weights for policy 0, policy_version 739300 (0.0008) [2023-12-27 10:44:37,724][20381] Updated weights for policy 1, policy_version 738907 (0.0008) [2023-12-27 10:44:37,785][20381] Updated weights for policy 1, policy_version 738917 (0.0006) [2023-12-27 10:44:37,840][20381] Updated weights for policy 1, policy_version 738927 (0.0006) [2023-12-27 10:44:38,303][20347] Updated weights for policy 0, policy_version 739310 (0.0008) [2023-12-27 10:44:38,371][20347] Updated weights for policy 0, policy_version 739320 (0.0008) [2023-12-27 10:44:38,410][20381] Updated weights for policy 1, policy_version 738937 (0.0006) [2023-12-27 10:44:38,425][20347] Updated weights for policy 0, policy_version 739330 (0.0007) [2023-12-27 10:44:38,466][20381] Updated weights for policy 1, policy_version 738947 (0.0007) [2023-12-27 10:44:38,519][20381] Updated weights for policy 1, policy_version 738957 (0.0008) [2023-12-27 10:44:38,573][20381] Updated weights for policy 1, policy_version 738967 (0.0008) [2023-12-27 10:44:39,064][20347] Updated weights for policy 0, policy_version 739340 (0.0007) [2023-12-27 10:44:39,108][20347] Updated weights for policy 0, policy_version 739350 (0.0008) [2023-12-27 10:44:39,163][20347] Updated weights for policy 0, policy_version 739360 (0.0008) [2023-12-27 10:44:39,216][20381] Updated weights for policy 1, policy_version 738977 (0.0006) [2023-12-27 10:44:39,274][20381] Updated weights for policy 1, policy_version 738987 (0.0007) [2023-12-27 10:44:39,327][20381] Updated weights for policy 1, policy_version 738997 (0.0007) [2023-12-27 10:44:39,897][20347] Updated weights for policy 0, policy_version 739370 (0.0007) [2023-12-27 10:44:39,959][20347] Updated weights for policy 0, policy_version 739380 (0.0007) [2023-12-27 10:44:40,021][20347] Updated weights for policy 0, policy_version 739390 (0.0008) [2023-12-27 10:44:40,079][20381] Updated weights for policy 1, policy_version 739007 (0.0009) [2023-12-27 10:44:40,080][20347] Updated weights for policy 0, policy_version 739400 (0.0006) [2023-12-27 10:44:40,143][20381] Updated weights for policy 1, policy_version 739017 (0.0009) [2023-12-27 10:44:40,203][20381] Updated weights for policy 1, policy_version 739027 (0.0010) [2023-12-27 10:44:40,767][20347] Updated weights for policy 0, policy_version 739410 (0.0008) [2023-12-27 10:44:40,829][20347] Updated weights for policy 0, policy_version 739420 (0.0007) [2023-12-27 10:44:40,889][20347] Updated weights for policy 0, policy_version 739430 (0.0005) [2023-12-27 10:44:40,896][20381] Updated weights for policy 1, policy_version 739037 (0.0011) [2023-12-27 10:44:40,949][20381] Updated weights for policy 1, policy_version 739047 (0.0010) [2023-12-27 10:44:41,001][20381] Updated weights for policy 1, policy_version 739057 (0.0010) [2023-12-27 10:44:41,555][20347] Updated weights for policy 0, policy_version 739440 (0.0008) [2023-12-27 10:44:41,621][20347] Updated weights for policy 0, policy_version 739450 (0.0008) [2023-12-27 10:44:41,688][20347] Updated weights for policy 0, policy_version 739460 (0.0007) [2023-12-27 10:44:41,776][20381] Updated weights for policy 1, policy_version 739067 (0.0009) [2023-12-27 10:44:41,829][20381] Updated weights for policy 1, policy_version 739077 (0.0007) [2023-12-27 10:44:41,888][20381] Updated weights for policy 1, policy_version 739087 (0.0008) [2023-12-27 10:44:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 378560512. Throughput: 0: 9932.3, 1: 10280.9. Samples: 378553360. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:42,363][19170] Avg episode reward: [(0, '951.720'), (1, '159.540')] [2023-12-27 10:44:42,409][20347] Updated weights for policy 0, policy_version 739470 (0.0007) [2023-12-27 10:44:42,458][20347] Updated weights for policy 0, policy_version 739480 (0.0008) [2023-12-27 10:44:42,519][20347] Updated weights for policy 0, policy_version 739490 (0.0007) [2023-12-27 10:44:42,579][20381] Updated weights for policy 1, policy_version 739097 (0.0008) [2023-12-27 10:44:42,638][20381] Updated weights for policy 1, policy_version 739107 (0.0010) [2023-12-27 10:44:42,694][20381] Updated weights for policy 1, policy_version 739117 (0.0010) [2023-12-27 10:44:42,754][20381] Updated weights for policy 1, policy_version 739127 (0.0011) [2023-12-27 10:44:43,219][20347] Updated weights for policy 0, policy_version 739500 (0.0007) [2023-12-27 10:44:43,271][20347] Updated weights for policy 0, policy_version 739510 (0.0007) [2023-12-27 10:44:43,326][20347] Updated weights for policy 0, policy_version 739520 (0.0007) [2023-12-27 10:44:43,443][20381] Updated weights for policy 1, policy_version 739137 (0.0011) [2023-12-27 10:44:43,504][20381] Updated weights for policy 1, policy_version 739147 (0.0008) [2023-12-27 10:44:43,564][20381] Updated weights for policy 1, policy_version 739157 (0.0011) [2023-12-27 10:44:43,961][20347] Updated weights for policy 0, policy_version 739530 (0.0007) [2023-12-27 10:44:44,030][20347] Updated weights for policy 0, policy_version 739540 (0.0007) [2023-12-27 10:44:44,079][20347] Updated weights for policy 0, policy_version 739550 (0.0008) [2023-12-27 10:44:44,137][20347] Updated weights for policy 0, policy_version 739560 (0.0008) [2023-12-27 10:44:44,205][20381] Updated weights for policy 1, policy_version 739167 (0.0007) [2023-12-27 10:44:44,254][20381] Updated weights for policy 1, policy_version 739177 (0.0006) [2023-12-27 10:44:44,307][20381] Updated weights for policy 1, policy_version 739187 (0.0008) [2023-12-27 10:44:44,854][20347] Updated weights for policy 0, policy_version 739570 (0.0008) [2023-12-27 10:44:44,916][20347] Updated weights for policy 0, policy_version 739580 (0.0007) [2023-12-27 10:44:44,947][20381] Updated weights for policy 1, policy_version 739197 (0.0006) [2023-12-27 10:44:44,978][20347] Updated weights for policy 0, policy_version 739590 (0.0006) [2023-12-27 10:44:45,007][20381] Updated weights for policy 1, policy_version 739207 (0.0007) [2023-12-27 10:44:45,060][20381] Updated weights for policy 1, policy_version 739217 (0.0008) [2023-12-27 10:44:45,590][20347] Updated weights for policy 0, policy_version 739600 (0.0007) [2023-12-27 10:44:45,642][20347] Updated weights for policy 0, policy_version 739610 (0.0007) [2023-12-27 10:44:45,689][20347] Updated weights for policy 0, policy_version 739620 (0.0007) [2023-12-27 10:44:45,789][20381] Updated weights for policy 1, policy_version 739227 (0.0009) [2023-12-27 10:44:45,848][20381] Updated weights for policy 1, policy_version 739237 (0.0011) [2023-12-27 10:44:45,908][20381] Updated weights for policy 1, policy_version 739247 (0.0011) [2023-12-27 10:44:46,394][20347] Updated weights for policy 0, policy_version 739630 (0.0008) [2023-12-27 10:44:46,460][20347] Updated weights for policy 0, policy_version 739640 (0.0008) [2023-12-27 10:44:46,514][20347] Updated weights for policy 0, policy_version 739650 (0.0009) [2023-12-27 10:44:46,550][20381] Updated weights for policy 1, policy_version 739257 (0.0010) [2023-12-27 10:44:46,598][20381] Updated weights for policy 1, policy_version 739267 (0.0007) [2023-12-27 10:44:46,654][20381] Updated weights for policy 1, policy_version 739277 (0.0008) [2023-12-27 10:44:46,702][20381] Updated weights for policy 1, policy_version 739287 (0.0008) [2023-12-27 10:44:47,230][20347] Updated weights for policy 0, policy_version 739660 (0.0008) [2023-12-27 10:44:47,285][20347] Updated weights for policy 0, policy_version 739670 (0.0008) [2023-12-27 10:44:47,331][20381] Updated weights for policy 1, policy_version 739297 (0.0008) [2023-12-27 10:44:47,336][20347] Updated weights for policy 0, policy_version 739680 (0.0005) [2023-12-27 10:44:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 378658816. Throughput: 0: 9933.2, 1: 10305.9. Samples: 378675436. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:47,363][19170] Avg episode reward: [(0, '980.510'), (1, '91.140')] [2023-12-27 10:44:47,400][20381] Updated weights for policy 1, policy_version 739307 (0.0007) [2023-12-27 10:44:47,455][20381] Updated weights for policy 1, policy_version 739317 (0.0006) [2023-12-27 10:44:48,041][20347] Updated weights for policy 0, policy_version 739690 (0.0007) [2023-12-27 10:44:48,083][20381] Updated weights for policy 1, policy_version 739327 (0.0009) [2023-12-27 10:44:48,092][20347] Updated weights for policy 0, policy_version 739700 (0.0005) [2023-12-27 10:44:48,137][20381] Updated weights for policy 1, policy_version 739337 (0.0010) [2023-12-27 10:44:48,151][20347] Updated weights for policy 0, policy_version 739710 (0.0007) [2023-12-27 10:44:48,185][20381] Updated weights for policy 1, policy_version 739347 (0.0010) [2023-12-27 10:44:48,214][20347] Updated weights for policy 0, policy_version 739720 (0.0006) [2023-12-27 10:44:48,902][20347] Updated weights for policy 0, policy_version 739730 (0.0006) [2023-12-27 10:44:48,914][20381] Updated weights for policy 1, policy_version 739357 (0.0009) [2023-12-27 10:44:48,952][20347] Updated weights for policy 0, policy_version 739740 (0.0007) [2023-12-27 10:44:48,969][20381] Updated weights for policy 1, policy_version 739367 (0.0007) [2023-12-27 10:44:49,010][20347] Updated weights for policy 0, policy_version 739750 (0.0006) [2023-12-27 10:44:49,030][20381] Updated weights for policy 1, policy_version 739377 (0.0008) [2023-12-27 10:44:49,753][20381] Updated weights for policy 1, policy_version 739387 (0.0007) [2023-12-27 10:44:49,761][20347] Updated weights for policy 0, policy_version 739760 (0.0008) [2023-12-27 10:44:49,808][20381] Updated weights for policy 1, policy_version 739397 (0.0008) [2023-12-27 10:44:49,814][20347] Updated weights for policy 0, policy_version 739770 (0.0007) [2023-12-27 10:44:49,876][20381] Updated weights for policy 1, policy_version 739407 (0.0008) [2023-12-27 10:44:49,882][20347] Updated weights for policy 0, policy_version 739780 (0.0007) [2023-12-27 10:44:50,535][20347] Updated weights for policy 0, policy_version 739790 (0.0006) [2023-12-27 10:44:50,598][20347] Updated weights for policy 0, policy_version 739800 (0.0007) [2023-12-27 10:44:50,615][20381] Updated weights for policy 1, policy_version 739417 (0.0008) [2023-12-27 10:44:50,654][20347] Updated weights for policy 0, policy_version 739810 (0.0010) [2023-12-27 10:44:50,670][20381] Updated weights for policy 1, policy_version 739427 (0.0010) [2023-12-27 10:44:50,734][20381] Updated weights for policy 1, policy_version 739437 (0.0007) [2023-12-27 10:44:50,790][20381] Updated weights for policy 1, policy_version 739447 (0.0011) [2023-12-27 10:44:51,347][20347] Updated weights for policy 0, policy_version 739820 (0.0007) [2023-12-27 10:44:51,412][20347] Updated weights for policy 0, policy_version 739830 (0.0008) [2023-12-27 10:44:51,477][20347] Updated weights for policy 0, policy_version 739840 (0.0008) [2023-12-27 10:44:51,520][20381] Updated weights for policy 1, policy_version 739457 (0.0011) [2023-12-27 10:44:51,579][20381] Updated weights for policy 1, policy_version 739467 (0.0011) [2023-12-27 10:44:51,641][20381] Updated weights for policy 1, policy_version 739477 (0.0010) [2023-12-27 10:44:52,219][20347] Updated weights for policy 0, policy_version 739850 (0.0008) [2023-12-27 10:44:52,276][20347] Updated weights for policy 0, policy_version 739860 (0.0008) [2023-12-27 10:44:52,328][20381] Updated weights for policy 1, policy_version 739487 (0.0006) [2023-12-27 10:44:52,332][20347] Updated weights for policy 0, policy_version 739870 (0.0007) [2023-12-27 10:44:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 378757120. Throughput: 0: 9882.0, 1: 10280.7. Samples: 378733860. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:52,363][19170] Avg episode reward: [(0, '990.280'), (1, '130.390')] [2023-12-27 10:44:52,392][20381] Updated weights for policy 1, policy_version 739497 (0.0008) [2023-12-27 10:44:52,396][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000739880_189431808.pth... [2023-12-27 10:44:52,398][20347] Updated weights for policy 0, policy_version 739880 (0.0008) [2023-12-27 10:44:52,400][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000738696_189128704.pth [2023-12-27 10:44:52,457][20381] Updated weights for policy 1, policy_version 739507 (0.0008) [2023-12-27 10:44:52,486][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000739512_189341696.pth... [2023-12-27 10:44:52,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000738264_189022208.pth [2023-12-27 10:44:53,042][20347] Updated weights for policy 0, policy_version 739890 (0.0008) [2023-12-27 10:44:53,101][20347] Updated weights for policy 0, policy_version 739900 (0.0008) [2023-12-27 10:44:53,149][20381] Updated weights for policy 1, policy_version 739517 (0.0006) [2023-12-27 10:44:53,162][20347] Updated weights for policy 0, policy_version 739910 (0.0006) [2023-12-27 10:44:53,195][20381] Updated weights for policy 1, policy_version 739527 (0.0006) [2023-12-27 10:44:53,244][20381] Updated weights for policy 1, policy_version 739537 (0.0008) [2023-12-27 10:44:53,861][20347] Updated weights for policy 0, policy_version 739920 (0.0008) [2023-12-27 10:44:53,914][20347] Updated weights for policy 0, policy_version 739930 (0.0008) [2023-12-27 10:44:53,953][20381] Updated weights for policy 1, policy_version 739547 (0.0007) [2023-12-27 10:44:53,967][20347] Updated weights for policy 0, policy_version 739940 (0.0007) [2023-12-27 10:44:54,003][20381] Updated weights for policy 1, policy_version 739557 (0.0006) [2023-12-27 10:44:54,057][20381] Updated weights for policy 1, policy_version 739567 (0.0007) [2023-12-27 10:44:54,680][20347] Updated weights for policy 0, policy_version 739950 (0.0006) [2023-12-27 10:44:54,719][20291] KL-divergence is very high: 121.9478 [2023-12-27 10:44:54,728][20381] Updated weights for policy 1, policy_version 739577 (0.0007) [2023-12-27 10:44:54,742][20347] Updated weights for policy 0, policy_version 739960 (0.0007) [2023-12-27 10:44:54,789][20381] Updated weights for policy 1, policy_version 739587 (0.0008) [2023-12-27 10:44:54,798][20347] Updated weights for policy 0, policy_version 739970 (0.0007) [2023-12-27 10:44:54,809][20291] KL-divergence is very high: 140.8718 [2023-12-27 10:44:54,854][20381] Updated weights for policy 1, policy_version 739597 (0.0007) [2023-12-27 10:44:54,916][20381] Updated weights for policy 1, policy_version 739607 (0.0007) [2023-12-27 10:44:55,489][20347] Updated weights for policy 0, policy_version 739980 (0.0006) [2023-12-27 10:44:55,546][20347] Updated weights for policy 0, policy_version 739990 (0.0007) [2023-12-27 10:44:55,555][20381] Updated weights for policy 1, policy_version 739617 (0.0010) [2023-12-27 10:44:55,604][20347] Updated weights for policy 0, policy_version 740000 (0.0006) [2023-12-27 10:44:55,610][20381] Updated weights for policy 1, policy_version 739627 (0.0010) [2023-12-27 10:44:55,669][20381] Updated weights for policy 1, policy_version 739637 (0.0010) [2023-12-27 10:44:56,254][20347] Updated weights for policy 0, policy_version 740010 (0.0009) [2023-12-27 10:44:56,297][20381] Updated weights for policy 1, policy_version 739647 (0.0011) [2023-12-27 10:44:56,314][20347] Updated weights for policy 0, policy_version 740020 (0.0005) [2023-12-27 10:44:56,357][20381] Updated weights for policy 1, policy_version 739657 (0.0011) [2023-12-27 10:44:56,374][20347] Updated weights for policy 0, policy_version 740030 (0.0006) [2023-12-27 10:44:56,419][20381] Updated weights for policy 1, policy_version 739667 (0.0011) [2023-12-27 10:44:56,436][20347] Updated weights for policy 0, policy_version 740040 (0.0005) [2023-12-27 10:44:57,105][20381] Updated weights for policy 1, policy_version 739677 (0.0011) [2023-12-27 10:44:57,121][20347] Updated weights for policy 0, policy_version 740050 (0.0008) [2023-12-27 10:44:57,155][20381] Updated weights for policy 1, policy_version 739687 (0.0011) [2023-12-27 10:44:57,176][20347] Updated weights for policy 0, policy_version 740060 (0.0007) [2023-12-27 10:44:57,207][20381] Updated weights for policy 1, policy_version 739697 (0.0010) [2023-12-27 10:44:57,229][20347] Updated weights for policy 0, policy_version 740070 (0.0006) [2023-12-27 10:44:57,235][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-27 10:44:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 378871808. Throughput: 0: 9898.8, 1: 10302.4. Samples: 378855676. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:44:57,363][19170] Avg episode reward: [(0, '960.940'), (1, '144.570')] [2023-12-27 10:44:57,904][20381] Updated weights for policy 1, policy_version 739707 (0.0010) [2023-12-27 10:44:57,917][20347] Updated weights for policy 0, policy_version 740080 (0.0006) [2023-12-27 10:44:57,949][20381] Updated weights for policy 1, policy_version 739717 (0.0010) [2023-12-27 10:44:57,970][20347] Updated weights for policy 0, policy_version 740090 (0.0007) [2023-12-27 10:44:57,994][20381] Updated weights for policy 1, policy_version 739727 (0.0010) [2023-12-27 10:44:58,022][20347] Updated weights for policy 0, policy_version 740100 (0.0005) [2023-12-27 10:44:58,761][20347] Updated weights for policy 0, policy_version 740110 (0.0008) [2023-12-27 10:44:58,776][20381] Updated weights for policy 1, policy_version 739737 (0.0011) [2023-12-27 10:44:58,823][20347] Updated weights for policy 0, policy_version 740120 (0.0008) [2023-12-27 10:44:58,842][20381] Updated weights for policy 1, policy_version 739747 (0.0008) [2023-12-27 10:44:58,899][20347] Updated weights for policy 0, policy_version 740130 (0.0008) [2023-12-27 10:44:58,911][20381] Updated weights for policy 1, policy_version 739757 (0.0009) [2023-12-27 10:44:58,970][20381] Updated weights for policy 1, policy_version 739767 (0.0011) [2023-12-27 10:44:59,631][20381] Updated weights for policy 1, policy_version 739777 (0.0006) [2023-12-27 10:44:59,686][20381] Updated weights for policy 1, policy_version 739787 (0.0005) [2023-12-27 10:44:59,700][20347] Updated weights for policy 0, policy_version 740140 (0.0008) [2023-12-27 10:44:59,740][20381] Updated weights for policy 1, policy_version 739797 (0.0007) [2023-12-27 10:44:59,765][20347] Updated weights for policy 0, policy_version 740150 (0.0005) [2023-12-27 10:44:59,825][20347] Updated weights for policy 0, policy_version 740160 (0.0008) [2023-12-27 10:45:00,447][20381] Updated weights for policy 1, policy_version 739807 (0.0008) [2023-12-27 10:45:00,465][20347] Updated weights for policy 0, policy_version 740170 (0.0008) [2023-12-27 10:45:00,503][20381] Updated weights for policy 1, policy_version 739817 (0.0006) [2023-12-27 10:45:00,513][20347] Updated weights for policy 0, policy_version 740180 (0.0006) [2023-12-27 10:45:00,559][20381] Updated weights for policy 1, policy_version 739827 (0.0005) [2023-12-27 10:45:00,574][20347] Updated weights for policy 0, policy_version 740190 (0.0008) [2023-12-27 10:45:00,629][20347] Updated weights for policy 0, policy_version 740200 (0.0008) [2023-12-27 10:45:01,233][20347] Updated weights for policy 0, policy_version 740210 (0.0007) [2023-12-27 10:45:01,254][20381] Updated weights for policy 1, policy_version 739837 (0.0006) [2023-12-27 10:45:01,297][20347] Updated weights for policy 0, policy_version 740220 (0.0010) [2023-12-27 10:45:01,313][20381] Updated weights for policy 1, policy_version 739847 (0.0005) [2023-12-27 10:45:01,357][20347] Updated weights for policy 0, policy_version 740230 (0.0010) [2023-12-27 10:45:01,374][20381] Updated weights for policy 1, policy_version 739857 (0.0006) [2023-12-27 10:45:02,078][20347] Updated weights for policy 0, policy_version 740240 (0.0010) [2023-12-27 10:45:02,106][20381] Updated weights for policy 1, policy_version 739867 (0.0007) [2023-12-27 10:45:02,127][20347] Updated weights for policy 0, policy_version 740250 (0.0010) [2023-12-27 10:45:02,162][20381] Updated weights for policy 1, policy_version 739877 (0.0006) [2023-12-27 10:45:02,186][20347] Updated weights for policy 0, policy_version 740260 (0.0010) [2023-12-27 10:45:02,220][20381] Updated weights for policy 1, policy_version 739887 (0.0006) [2023-12-27 10:45:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 378970112. Throughput: 0: 9955.7, 1: 10231.2. Samples: 378973980. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:45:02,363][19170] Avg episode reward: [(0, '689.080'), (1, '47.740')] [2023-12-27 10:45:02,868][20347] Updated weights for policy 0, policy_version 740270 (0.0010) [2023-12-27 10:45:02,907][20381] Updated weights for policy 1, policy_version 739897 (0.0007) [2023-12-27 10:45:02,913][20347] Updated weights for policy 0, policy_version 740280 (0.0010) [2023-12-27 10:45:02,961][20347] Updated weights for policy 0, policy_version 740290 (0.0010) [2023-12-27 10:45:02,962][20381] Updated weights for policy 1, policy_version 739907 (0.0005) [2023-12-27 10:45:03,011][20381] Updated weights for policy 1, policy_version 739917 (0.0007) [2023-12-27 10:45:03,060][20381] Updated weights for policy 1, policy_version 739927 (0.0009) [2023-12-27 10:45:03,570][20347] Updated weights for policy 0, policy_version 740300 (0.0007) [2023-12-27 10:45:03,627][20347] Updated weights for policy 0, policy_version 740310 (0.0010) [2023-12-27 10:45:03,634][20381] Updated weights for policy 1, policy_version 739937 (0.0006) [2023-12-27 10:45:03,682][20381] Updated weights for policy 1, policy_version 739947 (0.0010) [2023-12-27 10:45:03,687][20347] Updated weights for policy 0, policy_version 740320 (0.0006) [2023-12-27 10:45:03,739][20381] Updated weights for policy 1, policy_version 739957 (0.0010) [2023-12-27 10:45:04,341][20347] Updated weights for policy 0, policy_version 740330 (0.0006) [2023-12-27 10:45:04,400][20381] Updated weights for policy 1, policy_version 739967 (0.0008) [2023-12-27 10:45:04,402][20347] Updated weights for policy 0, policy_version 740340 (0.0007) [2023-12-27 10:45:04,458][20381] Updated weights for policy 1, policy_version 739977 (0.0006) [2023-12-27 10:45:04,462][20347] Updated weights for policy 0, policy_version 740350 (0.0006) [2023-12-27 10:45:04,513][20381] Updated weights for policy 1, policy_version 739987 (0.0005) [2023-12-27 10:45:04,519][20347] Updated weights for policy 0, policy_version 740360 (0.0008) [2023-12-27 10:45:05,192][20347] Updated weights for policy 0, policy_version 740370 (0.0008) [2023-12-27 10:45:05,205][20381] Updated weights for policy 1, policy_version 739997 (0.0007) [2023-12-27 10:45:05,241][20347] Updated weights for policy 0, policy_version 740380 (0.0007) [2023-12-27 10:45:05,263][20381] Updated weights for policy 1, policy_version 740007 (0.0007) [2023-12-27 10:45:05,290][20347] Updated weights for policy 0, policy_version 740390 (0.0005) [2023-12-27 10:45:05,315][20381] Updated weights for policy 1, policy_version 740017 (0.0010) [2023-12-27 10:45:05,994][20381] Updated weights for policy 1, policy_version 740027 (0.0010) [2023-12-27 10:45:06,005][20347] Updated weights for policy 0, policy_version 740400 (0.0008) [2023-12-27 10:45:06,048][20381] Updated weights for policy 1, policy_version 740037 (0.0010) [2023-12-27 10:45:06,063][20347] Updated weights for policy 0, policy_version 740410 (0.0008) [2023-12-27 10:45:06,103][20381] Updated weights for policy 1, policy_version 740047 (0.0010) [2023-12-27 10:45:06,122][20347] Updated weights for policy 0, policy_version 740420 (0.0009) [2023-12-27 10:45:06,786][20381] Updated weights for policy 1, policy_version 740057 (0.0008) [2023-12-27 10:45:06,839][20381] Updated weights for policy 1, policy_version 740067 (0.0011) [2023-12-27 10:45:06,847][20347] Updated weights for policy 0, policy_version 740430 (0.0007) [2023-12-27 10:45:06,895][20381] Updated weights for policy 1, policy_version 740077 (0.0011) [2023-12-27 10:45:06,909][20347] Updated weights for policy 0, policy_version 740440 (0.0009) [2023-12-27 10:45:06,951][20381] Updated weights for policy 1, policy_version 740087 (0.0011) [2023-12-27 10:45:06,970][20347] Updated weights for policy 0, policy_version 740450 (0.0008) [2023-12-27 10:45:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 379068416. Throughput: 0: 10001.0, 1: 10294.7. Samples: 379037760. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:45:07,363][19170] Avg episode reward: [(0, '92.230'), (1, '51.750')] [2023-12-27 10:45:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000740088_189489152.pth... [2023-12-27 10:45:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000740456_189579264.pth... [2023-12-27 10:45:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000738904_189186048.pth [2023-12-27 10:45:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000739272_189276160.pth [2023-12-27 10:45:07,636][20347] Updated weights for policy 0, policy_version 740460 (0.0007) [2023-12-27 10:45:07,647][20381] Updated weights for policy 1, policy_version 740097 (0.0011) [2023-12-27 10:45:07,691][20347] Updated weights for policy 0, policy_version 740470 (0.0005) [2023-12-27 10:45:07,704][20381] Updated weights for policy 1, policy_version 740107 (0.0011) [2023-12-27 10:45:07,748][20347] Updated weights for policy 0, policy_version 740480 (0.0006) [2023-12-27 10:45:07,761][20381] Updated weights for policy 1, policy_version 740117 (0.0011) [2023-12-27 10:45:08,454][20381] Updated weights for policy 1, policy_version 740127 (0.0011) [2023-12-27 10:45:08,486][20347] Updated weights for policy 0, policy_version 740490 (0.0007) [2023-12-27 10:45:08,513][20381] Updated weights for policy 1, policy_version 740137 (0.0011) [2023-12-27 10:45:08,540][20347] Updated weights for policy 0, policy_version 740500 (0.0008) [2023-12-27 10:45:08,576][20381] Updated weights for policy 1, policy_version 740147 (0.0011) [2023-12-27 10:45:08,597][20347] Updated weights for policy 0, policy_version 740510 (0.0007) [2023-12-27 10:45:08,661][20347] Updated weights for policy 0, policy_version 740520 (0.0006) [2023-12-27 10:45:09,266][20381] Updated weights for policy 1, policy_version 740157 (0.0009) [2023-12-27 10:45:09,321][20381] Updated weights for policy 1, policy_version 740167 (0.0007) [2023-12-27 10:45:09,353][20347] Updated weights for policy 0, policy_version 740530 (0.0007) [2023-12-27 10:45:09,389][20381] Updated weights for policy 1, policy_version 740177 (0.0008) [2023-12-27 10:45:09,423][20347] Updated weights for policy 0, policy_version 740540 (0.0008) [2023-12-27 10:45:09,474][20347] Updated weights for policy 0, policy_version 740550 (0.0008) [2023-12-27 10:45:10,049][20381] Updated weights for policy 1, policy_version 740187 (0.0008) [2023-12-27 10:45:10,114][20381] Updated weights for policy 1, policy_version 740197 (0.0008) [2023-12-27 10:45:10,159][20347] Updated weights for policy 0, policy_version 740560 (0.0007) [2023-12-27 10:45:10,172][20381] Updated weights for policy 1, policy_version 740207 (0.0006) [2023-12-27 10:45:10,214][20347] Updated weights for policy 0, policy_version 740570 (0.0006) [2023-12-27 10:45:10,277][20347] Updated weights for policy 0, policy_version 740580 (0.0007) [2023-12-27 10:45:10,789][20381] Updated weights for policy 1, policy_version 740217 (0.0008) [2023-12-27 10:45:10,851][20381] Updated weights for policy 1, policy_version 740227 (0.0008) [2023-12-27 10:45:10,907][20381] Updated weights for policy 1, policy_version 740237 (0.0007) [2023-12-27 10:45:10,963][20381] Updated weights for policy 1, policy_version 740247 (0.0008) [2023-12-27 10:45:10,999][20347] Updated weights for policy 0, policy_version 740590 (0.0008) [2023-12-27 10:45:11,058][20347] Updated weights for policy 0, policy_version 740600 (0.0008) [2023-12-27 10:45:11,123][20347] Updated weights for policy 0, policy_version 740610 (0.0008) [2023-12-27 10:45:11,695][20381] Updated weights for policy 1, policy_version 740257 (0.0009) [2023-12-27 10:45:11,766][20381] Updated weights for policy 1, policy_version 740267 (0.0008) [2023-12-27 10:45:11,818][20381] Updated weights for policy 1, policy_version 740277 (0.0008) [2023-12-27 10:45:11,868][20347] Updated weights for policy 0, policy_version 740620 (0.0007) [2023-12-27 10:45:11,925][20347] Updated weights for policy 0, policy_version 740630 (0.0008) [2023-12-27 10:45:11,989][20347] Updated weights for policy 0, policy_version 740640 (0.0008) [2023-12-27 10:45:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 379166720. Throughput: 0: 10006.9, 1: 10290.2. Samples: 379158372. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:45:12,363][19170] Avg episode reward: [(0, '164.270'), (1, '222.020')] [2023-12-27 10:45:12,512][20381] Updated weights for policy 1, policy_version 740287 (0.0008) [2023-12-27 10:45:12,567][20381] Updated weights for policy 1, policy_version 740297 (0.0008) [2023-12-27 10:45:12,620][20381] Updated weights for policy 1, policy_version 740307 (0.0006) [2023-12-27 10:45:12,812][20347] Updated weights for policy 0, policy_version 740650 (0.0008) [2023-12-27 10:45:12,872][20347] Updated weights for policy 0, policy_version 740660 (0.0007) [2023-12-27 10:45:12,931][20347] Updated weights for policy 0, policy_version 740670 (0.0008) [2023-12-27 10:45:12,988][20347] Updated weights for policy 0, policy_version 740680 (0.0007) [2023-12-27 10:45:13,251][20381] Updated weights for policy 1, policy_version 740317 (0.0007) [2023-12-27 10:45:13,310][20381] Updated weights for policy 1, policy_version 740327 (0.0008) [2023-12-27 10:45:13,368][20381] Updated weights for policy 1, policy_version 740337 (0.0007) [2023-12-27 10:45:13,661][20347] Updated weights for policy 0, policy_version 740690 (0.0008) [2023-12-27 10:45:13,708][20347] Updated weights for policy 0, policy_version 740700 (0.0009) [2023-12-27 10:45:13,757][20347] Updated weights for policy 0, policy_version 740710 (0.0007) [2023-12-27 10:45:13,998][20381] Updated weights for policy 1, policy_version 740347 (0.0008) [2023-12-27 10:45:14,053][20381] Updated weights for policy 1, policy_version 740357 (0.0007) [2023-12-27 10:45:14,113][20381] Updated weights for policy 1, policy_version 740367 (0.0008) [2023-12-27 10:45:14,472][20347] Updated weights for policy 0, policy_version 740720 (0.0007) [2023-12-27 10:45:14,525][20347] Updated weights for policy 0, policy_version 740730 (0.0008) [2023-12-27 10:45:14,585][20347] Updated weights for policy 0, policy_version 740740 (0.0008) [2023-12-27 10:45:14,826][20381] Updated weights for policy 1, policy_version 740377 (0.0008) [2023-12-27 10:45:14,881][20381] Updated weights for policy 1, policy_version 740387 (0.0007) [2023-12-27 10:45:14,939][20381] Updated weights for policy 1, policy_version 740397 (0.0008) [2023-12-27 10:45:14,999][20381] Updated weights for policy 1, policy_version 740407 (0.0007) [2023-12-27 10:45:15,321][20347] Updated weights for policy 0, policy_version 740750 (0.0008) [2023-12-27 10:45:15,379][20347] Updated weights for policy 0, policy_version 740760 (0.0008) [2023-12-27 10:45:15,445][20347] Updated weights for policy 0, policy_version 740770 (0.0007) [2023-12-27 10:45:15,697][20381] Updated weights for policy 1, policy_version 740417 (0.0006) [2023-12-27 10:45:15,760][20381] Updated weights for policy 1, policy_version 740427 (0.0007) [2023-12-27 10:45:15,819][20381] Updated weights for policy 1, policy_version 740437 (0.0005) [2023-12-27 10:45:16,163][20347] Updated weights for policy 0, policy_version 740780 (0.0008) [2023-12-27 10:45:16,214][20347] Updated weights for policy 0, policy_version 740790 (0.0007) [2023-12-27 10:45:16,266][20347] Updated weights for policy 0, policy_version 740800 (0.0007) [2023-12-27 10:45:16,462][20381] Updated weights for policy 1, policy_version 740447 (0.0007) [2023-12-27 10:45:16,518][20381] Updated weights for policy 1, policy_version 740457 (0.0006) [2023-12-27 10:45:16,567][20381] Updated weights for policy 1, policy_version 740467 (0.0008) [2023-12-27 10:45:16,908][20347] Updated weights for policy 0, policy_version 740810 (0.0008) [2023-12-27 10:45:16,965][20347] Updated weights for policy 0, policy_version 740820 (0.0007) [2023-12-27 10:45:17,019][20347] Updated weights for policy 0, policy_version 740830 (0.0006) [2023-12-27 10:45:17,072][20347] Updated weights for policy 0, policy_version 740840 (0.0007) [2023-12-27 10:45:17,261][20381] Updated weights for policy 1, policy_version 740477 (0.0009) [2023-12-27 10:45:17,316][20381] Updated weights for policy 1, policy_version 740487 (0.0007) [2023-12-27 10:45:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 379265024. Throughput: 0: 9945.9, 1: 10209.3. Samples: 379278352. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 10:45:17,363][19170] Avg episode reward: [(0, '427.880'), (1, '332.300')] [2023-12-27 10:45:17,371][20381] Updated weights for policy 1, policy_version 740497 (0.0007) [2023-12-27 10:45:17,771][20347] Updated weights for policy 0, policy_version 740850 (0.0007) [2023-12-27 10:45:17,828][20347] Updated weights for policy 0, policy_version 740860 (0.0007) [2023-12-27 10:45:17,887][20347] Updated weights for policy 0, policy_version 740870 (0.0008) [2023-12-27 10:45:18,056][20381] Updated weights for policy 1, policy_version 740507 (0.0006) [2023-12-27 10:45:18,108][20381] Updated weights for policy 1, policy_version 740517 (0.0005) [2023-12-27 10:45:18,167][20381] Updated weights for policy 1, policy_version 740527 (0.0005) [2023-12-27 10:45:18,597][20347] Updated weights for policy 0, policy_version 740880 (0.0007) [2023-12-27 10:45:18,653][20347] Updated weights for policy 0, policy_version 740890 (0.0007) [2023-12-27 10:45:18,717][20347] Updated weights for policy 0, policy_version 740900 (0.0008) [2023-12-27 10:45:18,873][20381] Updated weights for policy 1, policy_version 740537 (0.0010) [2023-12-27 10:45:18,933][20381] Updated weights for policy 1, policy_version 740547 (0.0011) [2023-12-27 10:45:18,993][20381] Updated weights for policy 1, policy_version 740557 (0.0010) [2023-12-27 10:45:19,050][20381] Updated weights for policy 1, policy_version 740567 (0.0005) [2023-12-27 10:45:19,420][20347] Updated weights for policy 0, policy_version 740910 (0.0008) [2023-12-27 10:45:19,488][20347] Updated weights for policy 0, policy_version 740920 (0.0008) [2023-12-27 10:45:19,550][20347] Updated weights for policy 0, policy_version 740930 (0.0007) [2023-12-27 10:45:19,683][20381] Updated weights for policy 1, policy_version 740577 (0.0006) [2023-12-27 10:45:19,748][20381] Updated weights for policy 1, policy_version 740587 (0.0008) [2023-12-27 10:45:19,808][20381] Updated weights for policy 1, policy_version 740597 (0.0008) [2023-12-27 10:45:20,276][20347] Updated weights for policy 0, policy_version 740940 (0.0007) [2023-12-27 10:45:20,342][20347] Updated weights for policy 0, policy_version 740950 (0.0007) [2023-12-27 10:45:20,402][20347] Updated weights for policy 0, policy_version 740960 (0.0007) [2023-12-27 10:45:20,506][20381] Updated weights for policy 1, policy_version 740607 (0.0008) [2023-12-27 10:45:20,561][20381] Updated weights for policy 1, policy_version 740617 (0.0008) [2023-12-27 10:45:20,623][20381] Updated weights for policy 1, policy_version 740627 (0.0009) [2023-12-27 10:45:21,132][20347] Updated weights for policy 0, policy_version 740970 (0.0008) [2023-12-27 10:45:21,198][20347] Updated weights for policy 0, policy_version 740980 (0.0007) [2023-12-27 10:45:21,264][20347] Updated weights for policy 0, policy_version 740990 (0.0007) [2023-12-27 10:45:21,321][20347] Updated weights for policy 0, policy_version 741000 (0.0008) [2023-12-27 10:45:21,370][20381] Updated weights for policy 1, policy_version 740637 (0.0008) [2023-12-27 10:45:21,439][20381] Updated weights for policy 1, policy_version 740647 (0.0006) [2023-12-27 10:45:21,494][20381] Updated weights for policy 1, policy_version 740657 (0.0006) [2023-12-27 10:45:21,995][20347] Updated weights for policy 0, policy_version 741010 (0.0008) [2023-12-27 10:45:22,054][20347] Updated weights for policy 0, policy_version 741020 (0.0008) [2023-12-27 10:45:22,119][20347] Updated weights for policy 0, policy_version 741030 (0.0007) [2023-12-27 10:45:22,205][20381] Updated weights for policy 1, policy_version 740667 (0.0007) [2023-12-27 10:45:22,263][20381] Updated weights for policy 1, policy_version 740677 (0.0006) [2023-12-27 10:45:22,325][20381] Updated weights for policy 1, policy_version 740687 (0.0008) [2023-12-27 10:45:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20438.3). Total num frames: 379363328. Throughput: 0: 9927.7, 1: 10188.5. Samples: 379337740. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:22,363][19170] Avg episode reward: [(0, '450.050'), (1, '392.470')] [2023-12-27 10:45:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000741032_189726720.pth... [2023-12-27 10:45:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000739880_189431808.pth [2023-12-27 10:45:22,392][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000740696_189644800.pth... [2023-12-27 10:45:22,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000739512_189341696.pth [2023-12-27 10:45:22,855][20347] Updated weights for policy 0, policy_version 741040 (0.0008) [2023-12-27 10:45:22,922][20347] Updated weights for policy 0, policy_version 741050 (0.0009) [2023-12-27 10:45:22,986][20347] Updated weights for policy 0, policy_version 741060 (0.0008) [2023-12-27 10:45:23,030][20381] Updated weights for policy 1, policy_version 740697 (0.0008) [2023-12-27 10:45:23,097][20381] Updated weights for policy 1, policy_version 740707 (0.0009) [2023-12-27 10:45:23,154][20381] Updated weights for policy 1, policy_version 740717 (0.0006) [2023-12-27 10:45:23,209][20381] Updated weights for policy 1, policy_version 740727 (0.0007) [2023-12-27 10:45:23,633][20347] Updated weights for policy 0, policy_version 741070 (0.0007) [2023-12-27 10:45:23,694][20347] Updated weights for policy 0, policy_version 741080 (0.0009) [2023-12-27 10:45:23,756][20347] Updated weights for policy 0, policy_version 741090 (0.0011) [2023-12-27 10:45:23,846][20381] Updated weights for policy 1, policy_version 740737 (0.0007) [2023-12-27 10:45:23,898][20381] Updated weights for policy 1, policy_version 740747 (0.0008) [2023-12-27 10:45:23,954][20381] Updated weights for policy 1, policy_version 740757 (0.0007) [2023-12-27 10:45:24,438][20347] Updated weights for policy 0, policy_version 741100 (0.0009) [2023-12-27 10:45:24,505][20347] Updated weights for policy 0, policy_version 741110 (0.0007) [2023-12-27 10:45:24,571][20347] Updated weights for policy 0, policy_version 741120 (0.0006) [2023-12-27 10:45:24,608][20381] Updated weights for policy 1, policy_version 740767 (0.0007) [2023-12-27 10:45:24,661][20381] Updated weights for policy 1, policy_version 740777 (0.0007) [2023-12-27 10:45:24,718][20381] Updated weights for policy 1, policy_version 740787 (0.0011) [2023-12-27 10:45:25,253][20347] Updated weights for policy 0, policy_version 741130 (0.0007) [2023-12-27 10:45:25,301][20347] Updated weights for policy 0, policy_version 741140 (0.0010) [2023-12-27 10:45:25,356][20347] Updated weights for policy 0, policy_version 741150 (0.0009) [2023-12-27 10:45:25,380][20381] Updated weights for policy 1, policy_version 740797 (0.0009) [2023-12-27 10:45:25,416][20347] Updated weights for policy 0, policy_version 741160 (0.0006) [2023-12-27 10:45:25,439][20381] Updated weights for policy 1, policy_version 740807 (0.0008) [2023-12-27 10:45:25,497][20381] Updated weights for policy 1, policy_version 740817 (0.0007) [2023-12-27 10:45:26,063][20347] Updated weights for policy 0, policy_version 741170 (0.0010) [2023-12-27 10:45:26,111][20347] Updated weights for policy 0, policy_version 741180 (0.0008) [2023-12-27 10:45:26,163][20347] Updated weights for policy 0, policy_version 741190 (0.0006) [2023-12-27 10:45:26,172][20381] Updated weights for policy 1, policy_version 740827 (0.0007) [2023-12-27 10:45:26,225][20381] Updated weights for policy 1, policy_version 740837 (0.0008) [2023-12-27 10:45:26,282][20381] Updated weights for policy 1, policy_version 740847 (0.0007) [2023-12-27 10:45:26,834][20347] Updated weights for policy 0, policy_version 741200 (0.0007) [2023-12-27 10:45:26,884][20347] Updated weights for policy 0, policy_version 741210 (0.0009) [2023-12-27 10:45:26,907][20381] Updated weights for policy 1, policy_version 740857 (0.0007) [2023-12-27 10:45:26,937][20347] Updated weights for policy 0, policy_version 741220 (0.0009) [2023-12-27 10:45:26,966][20381] Updated weights for policy 1, policy_version 740867 (0.0008) [2023-12-27 10:45:27,022][20381] Updated weights for policy 1, policy_version 740877 (0.0008) [2023-12-27 10:45:27,078][20381] Updated weights for policy 1, policy_version 740887 (0.0007) [2023-12-27 10:45:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 379469824. Throughput: 0: 9960.7, 1: 10142.7. Samples: 379458012. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:27,363][19170] Avg episode reward: [(0, '630.450'), (1, '319.530')] [2023-12-27 10:45:27,655][20347] Updated weights for policy 0, policy_version 741230 (0.0008) [2023-12-27 10:45:27,705][20347] Updated weights for policy 0, policy_version 741240 (0.0008) [2023-12-27 10:45:27,757][20347] Updated weights for policy 0, policy_version 741250 (0.0008) [2023-12-27 10:45:27,777][20381] Updated weights for policy 1, policy_version 740897 (0.0007) [2023-12-27 10:45:27,828][20381] Updated weights for policy 1, policy_version 740907 (0.0006) [2023-12-27 10:45:27,888][20381] Updated weights for policy 1, policy_version 740917 (0.0007) [2023-12-27 10:45:28,459][20347] Updated weights for policy 0, policy_version 741260 (0.0007) [2023-12-27 10:45:28,516][20347] Updated weights for policy 0, policy_version 741270 (0.0008) [2023-12-27 10:45:28,582][20347] Updated weights for policy 0, policy_version 741280 (0.0008) [2023-12-27 10:45:28,617][20381] Updated weights for policy 1, policy_version 740927 (0.0007) [2023-12-27 10:45:28,673][20381] Updated weights for policy 1, policy_version 740937 (0.0007) [2023-12-27 10:45:28,727][20381] Updated weights for policy 1, policy_version 740947 (0.0008) [2023-12-27 10:45:29,318][20347] Updated weights for policy 0, policy_version 741290 (0.0008) [2023-12-27 10:45:29,378][20347] Updated weights for policy 0, policy_version 741300 (0.0009) [2023-12-27 10:45:29,433][20347] Updated weights for policy 0, policy_version 741311 (0.0010) [2023-12-27 10:45:29,443][20381] Updated weights for policy 1, policy_version 740957 (0.0009) [2023-12-27 10:45:29,496][20381] Updated weights for policy 1, policy_version 740967 (0.0009) [2023-12-27 10:45:29,550][20381] Updated weights for policy 1, policy_version 740977 (0.0009) [2023-12-27 10:45:30,198][20347] Updated weights for policy 0, policy_version 741321 (0.0008) [2023-12-27 10:45:30,262][20347] Updated weights for policy 0, policy_version 741331 (0.0010) [2023-12-27 10:45:30,277][20381] Updated weights for policy 1, policy_version 740987 (0.0009) [2023-12-27 10:45:30,325][20347] Updated weights for policy 0, policy_version 741341 (0.0008) [2023-12-27 10:45:30,334][20381] Updated weights for policy 1, policy_version 740997 (0.0011) [2023-12-27 10:45:30,362][20291] KL-divergence is very high: 141.5088 [2023-12-27 10:45:30,368][20291] KL-divergence is very high: 130.5165 [2023-12-27 10:45:30,385][20347] Updated weights for policy 0, policy_version 741351 (0.0008) [2023-12-27 10:45:30,392][20381] Updated weights for policy 1, policy_version 741007 (0.0011) [2023-12-27 10:45:30,983][20347] Updated weights for policy 0, policy_version 741361 (0.0005) [2023-12-27 10:45:31,031][20381] Updated weights for policy 1, policy_version 741017 (0.0011) [2023-12-27 10:45:31,046][20347] Updated weights for policy 0, policy_version 741371 (0.0011) [2023-12-27 10:45:31,090][20381] Updated weights for policy 1, policy_version 741027 (0.0010) [2023-12-27 10:45:31,106][20347] Updated weights for policy 0, policy_version 741381 (0.0010) [2023-12-27 10:45:31,147][20381] Updated weights for policy 1, policy_version 741037 (0.0011) [2023-12-27 10:45:31,209][20381] Updated weights for policy 1, policy_version 741047 (0.0010) [2023-12-27 10:45:31,796][20347] Updated weights for policy 0, policy_version 741391 (0.0008) [2023-12-27 10:45:31,849][20347] Updated weights for policy 0, policy_version 741401 (0.0008) [2023-12-27 10:45:31,899][20347] Updated weights for policy 0, policy_version 741411 (0.0006) [2023-12-27 10:45:31,955][20381] Updated weights for policy 1, policy_version 741057 (0.0007) [2023-12-27 10:45:32,015][20381] Updated weights for policy 1, policy_version 741067 (0.0009) [2023-12-27 10:45:32,080][20381] Updated weights for policy 1, policy_version 741077 (0.0007) [2023-12-27 10:45:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 379568128. Throughput: 0: 9943.7, 1: 10102.6. Samples: 379577516. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:32,363][19170] Avg episode reward: [(0, '247.930'), (1, '321.340')] [2023-12-27 10:45:32,561][20347] Updated weights for policy 0, policy_version 741421 (0.0006) [2023-12-27 10:45:32,623][20347] Updated weights for policy 0, policy_version 741431 (0.0006) [2023-12-27 10:45:32,687][20347] Updated weights for policy 0, policy_version 741441 (0.0006) [2023-12-27 10:45:32,716][20381] Updated weights for policy 1, policy_version 741087 (0.0007) [2023-12-27 10:45:32,766][20381] Updated weights for policy 1, policy_version 741097 (0.0007) [2023-12-27 10:45:32,819][20381] Updated weights for policy 1, policy_version 741107 (0.0007) [2023-12-27 10:45:33,335][20347] Updated weights for policy 0, policy_version 741451 (0.0006) [2023-12-27 10:45:33,383][20347] Updated weights for policy 0, policy_version 741461 (0.0005) [2023-12-27 10:45:33,406][20381] Updated weights for policy 1, policy_version 741117 (0.0007) [2023-12-27 10:45:33,441][20347] Updated weights for policy 0, policy_version 741471 (0.0007) [2023-12-27 10:45:33,464][20381] Updated weights for policy 1, policy_version 741127 (0.0006) [2023-12-27 10:45:33,522][20381] Updated weights for policy 1, policy_version 741137 (0.0006) [2023-12-27 10:45:34,073][20347] Updated weights for policy 0, policy_version 741481 (0.0006) [2023-12-27 10:45:34,133][20347] Updated weights for policy 0, policy_version 741491 (0.0006) [2023-12-27 10:45:34,196][20347] Updated weights for policy 0, policy_version 741501 (0.0008) [2023-12-27 10:45:34,205][20381] Updated weights for policy 1, policy_version 741147 (0.0008) [2023-12-27 10:45:34,256][20347] Updated weights for policy 0, policy_version 741511 (0.0009) [2023-12-27 10:45:34,271][20381] Updated weights for policy 1, policy_version 741157 (0.0007) [2023-12-27 10:45:34,321][20381] Updated weights for policy 1, policy_version 741167 (0.0008) [2023-12-27 10:45:34,951][20347] Updated weights for policy 0, policy_version 741521 (0.0008) [2023-12-27 10:45:34,962][20381] Updated weights for policy 1, policy_version 741177 (0.0008) [2023-12-27 10:45:35,006][20347] Updated weights for policy 0, policy_version 741531 (0.0009) [2023-12-27 10:45:35,025][20381] Updated weights for policy 1, policy_version 741187 (0.0006) [2023-12-27 10:45:35,061][20347] Updated weights for policy 0, policy_version 741541 (0.0007) [2023-12-27 10:45:35,078][20381] Updated weights for policy 1, policy_version 741197 (0.0006) [2023-12-27 10:45:35,140][20381] Updated weights for policy 1, policy_version 741207 (0.0008) [2023-12-27 10:45:35,772][20347] Updated weights for policy 0, policy_version 741551 (0.0006) [2023-12-27 10:45:35,785][20381] Updated weights for policy 1, policy_version 741217 (0.0006) [2023-12-27 10:45:35,831][20347] Updated weights for policy 0, policy_version 741561 (0.0006) [2023-12-27 10:45:35,845][20381] Updated weights for policy 1, policy_version 741227 (0.0006) [2023-12-27 10:45:35,888][20347] Updated weights for policy 0, policy_version 741571 (0.0007) [2023-12-27 10:45:35,898][20381] Updated weights for policy 1, policy_version 741237 (0.0006) [2023-12-27 10:45:36,585][20347] Updated weights for policy 0, policy_version 741581 (0.0006) [2023-12-27 10:45:36,633][20381] Updated weights for policy 1, policy_version 741247 (0.0007) [2023-12-27 10:45:36,646][20347] Updated weights for policy 0, policy_version 741591 (0.0005) [2023-12-27 10:45:36,695][20381] Updated weights for policy 1, policy_version 741257 (0.0008) [2023-12-27 10:45:36,705][20347] Updated weights for policy 0, policy_version 741601 (0.0006) [2023-12-27 10:45:36,755][20381] Updated weights for policy 1, policy_version 741267 (0.0008) [2023-12-27 10:45:37,334][20347] Updated weights for policy 0, policy_version 741611 (0.0006) [2023-12-27 10:45:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 20410.6). Total num frames: 379666432. Throughput: 0: 9994.6, 1: 10166.3. Samples: 379641096. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:37,363][19170] Avg episode reward: [(0, '297.950'), (1, '536.340')] [2023-12-27 10:45:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000741272_189792256.pth... [2023-12-27 10:45:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000740088_189489152.pth [2023-12-27 10:45:37,396][20347] Updated weights for policy 0, policy_version 741621 (0.0006) [2023-12-27 10:45:37,453][20347] Updated weights for policy 0, policy_version 741631 (0.0007) [2023-12-27 10:45:37,465][20381] Updated weights for policy 1, policy_version 741277 (0.0009) [2023-12-27 10:45:37,502][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000741640_189882368.pth... [2023-12-27 10:45:37,505][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000740456_189579264.pth [2023-12-27 10:45:37,523][20381] Updated weights for policy 1, policy_version 741287 (0.0011) [2023-12-27 10:45:37,586][20381] Updated weights for policy 1, policy_version 741297 (0.0010) [2023-12-27 10:45:38,144][20347] Updated weights for policy 0, policy_version 741641 (0.0006) [2023-12-27 10:45:38,207][20347] Updated weights for policy 0, policy_version 741651 (0.0008) [2023-12-27 10:45:38,268][20347] Updated weights for policy 0, policy_version 741661 (0.0008) [2023-12-27 10:45:38,318][20381] Updated weights for policy 1, policy_version 741307 (0.0007) [2023-12-27 10:45:38,336][20347] Updated weights for policy 0, policy_version 741671 (0.0009) [2023-12-27 10:45:38,378][20381] Updated weights for policy 1, policy_version 741317 (0.0008) [2023-12-27 10:45:38,443][20381] Updated weights for policy 1, policy_version 741327 (0.0007) [2023-12-27 10:45:39,061][20347] Updated weights for policy 0, policy_version 741681 (0.0006) [2023-12-27 10:45:39,108][20347] Updated weights for policy 0, policy_version 741691 (0.0005) [2023-12-27 10:45:39,159][20347] Updated weights for policy 0, policy_version 741701 (0.0005) [2023-12-27 10:45:39,186][20381] Updated weights for policy 1, policy_version 741337 (0.0008) [2023-12-27 10:45:39,245][20381] Updated weights for policy 1, policy_version 741347 (0.0007) [2023-12-27 10:45:39,303][20381] Updated weights for policy 1, policy_version 741357 (0.0006) [2023-12-27 10:45:39,372][20381] Updated weights for policy 1, policy_version 741367 (0.0008) [2023-12-27 10:45:39,864][20347] Updated weights for policy 0, policy_version 741711 (0.0008) [2023-12-27 10:45:39,928][20347] Updated weights for policy 0, policy_version 741721 (0.0008) [2023-12-27 10:45:39,984][20381] Updated weights for policy 1, policy_version 741377 (0.0008) [2023-12-27 10:45:39,994][20347] Updated weights for policy 0, policy_version 741731 (0.0007) [2023-12-27 10:45:40,043][20381] Updated weights for policy 1, policy_version 741387 (0.0006) [2023-12-27 10:45:40,101][20381] Updated weights for policy 1, policy_version 741397 (0.0006) [2023-12-27 10:45:40,723][20381] Updated weights for policy 1, policy_version 741407 (0.0007) [2023-12-27 10:45:40,740][20347] Updated weights for policy 0, policy_version 741741 (0.0007) [2023-12-27 10:45:40,781][20381] Updated weights for policy 1, policy_version 741417 (0.0006) [2023-12-27 10:45:40,799][20347] Updated weights for policy 0, policy_version 741751 (0.0006) [2023-12-27 10:45:40,835][20381] Updated weights for policy 1, policy_version 741427 (0.0008) [2023-12-27 10:45:40,857][20347] Updated weights for policy 0, policy_version 741761 (0.0008) [2023-12-27 10:45:41,570][20381] Updated weights for policy 1, policy_version 741437 (0.0009) [2023-12-27 10:45:41,615][20347] Updated weights for policy 0, policy_version 741771 (0.0008) [2023-12-27 10:45:41,632][20381] Updated weights for policy 1, policy_version 741447 (0.0008) [2023-12-27 10:45:41,683][20347] Updated weights for policy 0, policy_version 741781 (0.0008) [2023-12-27 10:45:41,692][20381] Updated weights for policy 1, policy_version 741457 (0.0007) [2023-12-27 10:45:41,749][20347] Updated weights for policy 0, policy_version 741791 (0.0007) [2023-12-27 10:45:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20410.6). Total num frames: 379764736. Throughput: 0: 9950.8, 1: 10135.2. Samples: 379759548. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:42,363][19170] Avg episode reward: [(0, '219.400'), (1, '397.740')] [2023-12-27 10:45:42,411][20381] Updated weights for policy 1, policy_version 741467 (0.0008) [2023-12-27 10:45:42,450][20347] Updated weights for policy 0, policy_version 741801 (0.0006) [2023-12-27 10:45:42,469][20381] Updated weights for policy 1, policy_version 741477 (0.0007) [2023-12-27 10:45:42,510][20347] Updated weights for policy 0, policy_version 741811 (0.0006) [2023-12-27 10:45:42,527][20381] Updated weights for policy 1, policy_version 741487 (0.0008) [2023-12-27 10:45:42,571][20347] Updated weights for policy 0, policy_version 741821 (0.0006) [2023-12-27 10:45:42,635][20347] Updated weights for policy 0, policy_version 741831 (0.0007) [2023-12-27 10:45:43,261][20381] Updated weights for policy 1, policy_version 741497 (0.0006) [2023-12-27 10:45:43,290][20347] Updated weights for policy 0, policy_version 741841 (0.0006) [2023-12-27 10:45:43,318][20381] Updated weights for policy 1, policy_version 741507 (0.0006) [2023-12-27 10:45:43,345][20347] Updated weights for policy 0, policy_version 741851 (0.0006) [2023-12-27 10:45:43,376][20381] Updated weights for policy 1, policy_version 741517 (0.0006) [2023-12-27 10:45:43,401][20347] Updated weights for policy 0, policy_version 741861 (0.0005) [2023-12-27 10:45:43,428][20381] Updated weights for policy 1, policy_version 741527 (0.0006) [2023-12-27 10:45:43,972][20347] Updated weights for policy 0, policy_version 741871 (0.0007) [2023-12-27 10:45:44,030][20347] Updated weights for policy 0, policy_version 741881 (0.0009) [2023-12-27 10:45:44,088][20347] Updated weights for policy 0, policy_version 741891 (0.0009) [2023-12-27 10:45:44,128][20381] Updated weights for policy 1, policy_version 741537 (0.0008) [2023-12-27 10:45:44,187][20381] Updated weights for policy 1, policy_version 741547 (0.0006) [2023-12-27 10:45:44,249][20381] Updated weights for policy 1, policy_version 741557 (0.0006) [2023-12-27 10:45:44,736][20347] Updated weights for policy 0, policy_version 741901 (0.0007) [2023-12-27 10:45:44,802][20347] Updated weights for policy 0, policy_version 741911 (0.0008) [2023-12-27 10:45:44,858][20347] Updated weights for policy 0, policy_version 741921 (0.0010) [2023-12-27 10:45:44,952][20381] Updated weights for policy 1, policy_version 741567 (0.0008) [2023-12-27 10:45:45,012][20381] Updated weights for policy 1, policy_version 741577 (0.0007) [2023-12-27 10:45:45,077][20381] Updated weights for policy 1, policy_version 741587 (0.0008) [2023-12-27 10:45:45,446][20347] Updated weights for policy 0, policy_version 741931 (0.0006) [2023-12-27 10:45:45,494][20347] Updated weights for policy 0, policy_version 741941 (0.0010) [2023-12-27 10:45:45,542][20347] Updated weights for policy 0, policy_version 741951 (0.0006) [2023-12-27 10:45:45,838][20381] Updated weights for policy 1, policy_version 741597 (0.0008) [2023-12-27 10:45:45,903][20381] Updated weights for policy 1, policy_version 741607 (0.0006) [2023-12-27 10:45:45,959][20381] Updated weights for policy 1, policy_version 741617 (0.0006) [2023-12-27 10:45:46,136][20347] Updated weights for policy 0, policy_version 741961 (0.0007) [2023-12-27 10:45:46,201][20347] Updated weights for policy 0, policy_version 741971 (0.0005) [2023-12-27 10:45:46,265][20347] Updated weights for policy 0, policy_version 741981 (0.0005) [2023-12-27 10:45:46,317][20347] Updated weights for policy 0, policy_version 741991 (0.0010) [2023-12-27 10:45:46,584][20381] Updated weights for policy 1, policy_version 741627 (0.0008) [2023-12-27 10:45:46,632][20381] Updated weights for policy 1, policy_version 741637 (0.0007) [2023-12-27 10:45:46,693][20381] Updated weights for policy 1, policy_version 741647 (0.0009) [2023-12-27 10:45:46,902][20347] Updated weights for policy 0, policy_version 742001 (0.0007) [2023-12-27 10:45:46,967][20347] Updated weights for policy 0, policy_version 742011 (0.0005) [2023-12-27 10:45:47,025][20347] Updated weights for policy 0, policy_version 742021 (0.0009) [2023-12-27 10:45:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20438.4). Total num frames: 379871232. Throughput: 0: 10059.0, 1: 10142.0. Samples: 379883024. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:47,363][19170] Avg episode reward: [(0, '361.830'), (1, '602.260')] [2023-12-27 10:45:47,381][20381] Updated weights for policy 1, policy_version 741657 (0.0008) [2023-12-27 10:45:47,441][20381] Updated weights for policy 1, policy_version 741667 (0.0005) [2023-12-27 10:45:47,501][20381] Updated weights for policy 1, policy_version 741677 (0.0006) [2023-12-27 10:45:47,557][20381] Updated weights for policy 1, policy_version 741687 (0.0007) [2023-12-27 10:45:47,660][20347] Updated weights for policy 0, policy_version 742031 (0.0007) [2023-12-27 10:45:47,715][20347] Updated weights for policy 0, policy_version 742041 (0.0006) [2023-12-27 10:45:47,771][20347] Updated weights for policy 0, policy_version 742051 (0.0007) [2023-12-27 10:45:48,212][20381] Updated weights for policy 1, policy_version 741697 (0.0007) [2023-12-27 10:45:48,271][20381] Updated weights for policy 1, policy_version 741707 (0.0005) [2023-12-27 10:45:48,338][20381] Updated weights for policy 1, policy_version 741717 (0.0008) [2023-12-27 10:45:48,470][20347] Updated weights for policy 0, policy_version 742061 (0.0007) [2023-12-27 10:45:48,523][20347] Updated weights for policy 0, policy_version 742071 (0.0007) [2023-12-27 10:45:48,584][20347] Updated weights for policy 0, policy_version 742081 (0.0008) [2023-12-27 10:45:48,999][20381] Updated weights for policy 1, policy_version 741727 (0.0007) [2023-12-27 10:45:49,060][20381] Updated weights for policy 1, policy_version 741737 (0.0008) [2023-12-27 10:45:49,120][20381] Updated weights for policy 1, policy_version 741747 (0.0008) [2023-12-27 10:45:49,305][20347] Updated weights for policy 0, policy_version 742091 (0.0007) [2023-12-27 10:45:49,371][20347] Updated weights for policy 0, policy_version 742101 (0.0008) [2023-12-27 10:45:49,428][20347] Updated weights for policy 0, policy_version 742111 (0.0008) [2023-12-27 10:45:49,919][20381] Updated weights for policy 1, policy_version 741757 (0.0008) [2023-12-27 10:45:49,987][20381] Updated weights for policy 1, policy_version 741767 (0.0007) [2023-12-27 10:45:50,044][20381] Updated weights for policy 1, policy_version 741777 (0.0007) [2023-12-27 10:45:50,161][20347] Updated weights for policy 0, policy_version 742121 (0.0007) [2023-12-27 10:45:50,212][20347] Updated weights for policy 0, policy_version 742131 (0.0008) [2023-12-27 10:45:50,265][20347] Updated weights for policy 0, policy_version 742141 (0.0007) [2023-12-27 10:45:50,322][20347] Updated weights for policy 0, policy_version 742151 (0.0007) [2023-12-27 10:45:50,676][20381] Updated weights for policy 1, policy_version 741787 (0.0007) [2023-12-27 10:45:50,740][20381] Updated weights for policy 1, policy_version 741797 (0.0006) [2023-12-27 10:45:50,796][20381] Updated weights for policy 1, policy_version 741807 (0.0006) [2023-12-27 10:45:51,116][20347] Updated weights for policy 0, policy_version 742161 (0.0008) [2023-12-27 10:45:51,179][20347] Updated weights for policy 0, policy_version 742171 (0.0008) [2023-12-27 10:45:51,250][20347] Updated weights for policy 0, policy_version 742181 (0.0009) [2023-12-27 10:45:51,466][20381] Updated weights for policy 1, policy_version 741817 (0.0008) [2023-12-27 10:45:51,530][20381] Updated weights for policy 1, policy_version 741827 (0.0008) [2023-12-27 10:45:51,594][20381] Updated weights for policy 1, policy_version 741837 (0.0008) [2023-12-27 10:45:51,672][20381] Updated weights for policy 1, policy_version 741847 (0.0007) [2023-12-27 10:45:51,984][20347] Updated weights for policy 0, policy_version 742191 (0.0008) [2023-12-27 10:45:52,043][20347] Updated weights for policy 0, policy_version 742201 (0.0007) [2023-12-27 10:45:52,102][20347] Updated weights for policy 0, policy_version 742211 (0.0007) [2023-12-27 10:45:52,346][20381] Updated weights for policy 1, policy_version 741857 (0.0008) [2023-12-27 10:45:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20382.8). Total num frames: 379969536. Throughput: 0: 10018.2, 1: 10093.4. Samples: 379942784. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:52,363][19170] Avg episode reward: [(0, '249.270'), (1, '463.120')] [2023-12-27 10:45:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000742216_190029824.pth... [2023-12-27 10:45:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000741032_189726720.pth [2023-12-27 10:45:52,400][20381] Updated weights for policy 1, policy_version 741867 (0.0007) [2023-12-27 10:45:52,458][20381] Updated weights for policy 1, policy_version 741877 (0.0007) [2023-12-27 10:45:52,474][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000741880_189947904.pth... [2023-12-27 10:45:52,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000740696_189644800.pth [2023-12-27 10:45:52,808][20347] Updated weights for policy 0, policy_version 742221 (0.0008) [2023-12-27 10:45:52,863][20347] Updated weights for policy 0, policy_version 742231 (0.0006) [2023-12-27 10:45:52,922][20347] Updated weights for policy 0, policy_version 742241 (0.0008) [2023-12-27 10:45:53,163][20381] Updated weights for policy 1, policy_version 741887 (0.0008) [2023-12-27 10:45:53,222][20381] Updated weights for policy 1, policy_version 741897 (0.0007) [2023-12-27 10:45:53,283][20381] Updated weights for policy 1, policy_version 741907 (0.0007) [2023-12-27 10:45:53,576][20347] Updated weights for policy 0, policy_version 742251 (0.0008) [2023-12-27 10:45:53,639][20347] Updated weights for policy 0, policy_version 742261 (0.0005) [2023-12-27 10:45:53,700][20347] Updated weights for policy 0, policy_version 742271 (0.0008) [2023-12-27 10:45:54,027][20381] Updated weights for policy 1, policy_version 741917 (0.0007) [2023-12-27 10:45:54,081][20381] Updated weights for policy 1, policy_version 741927 (0.0007) [2023-12-27 10:45:54,136][20381] Updated weights for policy 1, policy_version 741937 (0.0007) [2023-12-27 10:45:54,301][20347] Updated weights for policy 0, policy_version 742281 (0.0008) [2023-12-27 10:45:54,358][20347] Updated weights for policy 0, policy_version 742291 (0.0010) [2023-12-27 10:45:54,412][20347] Updated weights for policy 0, policy_version 742301 (0.0010) [2023-12-27 10:45:54,463][20347] Updated weights for policy 0, policy_version 742311 (0.0010) [2023-12-27 10:45:54,866][20381] Updated weights for policy 1, policy_version 741947 (0.0008) [2023-12-27 10:45:54,924][20381] Updated weights for policy 1, policy_version 741957 (0.0007) [2023-12-27 10:45:54,987][20381] Updated weights for policy 1, policy_version 741967 (0.0008) [2023-12-27 10:45:55,133][20347] Updated weights for policy 0, policy_version 742321 (0.0009) [2023-12-27 10:45:55,185][20347] Updated weights for policy 0, policy_version 742331 (0.0007) [2023-12-27 10:45:55,244][20347] Updated weights for policy 0, policy_version 742341 (0.0007) [2023-12-27 10:45:55,671][20381] Updated weights for policy 1, policy_version 741977 (0.0009) [2023-12-27 10:45:55,725][20381] Updated weights for policy 1, policy_version 741987 (0.0011) [2023-12-27 10:45:55,787][20381] Updated weights for policy 1, policy_version 741997 (0.0011) [2023-12-27 10:45:55,843][20381] Updated weights for policy 1, policy_version 742007 (0.0011) [2023-12-27 10:45:55,855][20347] Updated weights for policy 0, policy_version 742351 (0.0005) [2023-12-27 10:45:55,908][20347] Updated weights for policy 0, policy_version 742361 (0.0008) [2023-12-27 10:45:55,964][20347] Updated weights for policy 0, policy_version 742371 (0.0007) [2023-12-27 10:45:56,512][20381] Updated weights for policy 1, policy_version 742017 (0.0011) [2023-12-27 10:45:56,559][20381] Updated weights for policy 1, policy_version 742027 (0.0010) [2023-12-27 10:45:56,610][20381] Updated weights for policy 1, policy_version 742037 (0.0010) [2023-12-27 10:45:56,641][20347] Updated weights for policy 0, policy_version 742381 (0.0007) [2023-12-27 10:45:56,687][20347] Updated weights for policy 0, policy_version 742391 (0.0007) [2023-12-27 10:45:56,734][20347] Updated weights for policy 0, policy_version 742401 (0.0007) [2023-12-27 10:45:57,285][20381] Updated weights for policy 1, policy_version 742047 (0.0010) [2023-12-27 10:45:57,346][20381] Updated weights for policy 1, policy_version 742057 (0.0010) [2023-12-27 10:45:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19933.9, 300 sec: 20382.8). Total num frames: 380067840. Throughput: 0: 10051.8, 1: 10062.6. Samples: 380063516. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:45:57,363][19170] Avg episode reward: [(0, '240.670'), (1, '511.980')] [2023-12-27 10:45:57,404][20381] Updated weights for policy 1, policy_version 742067 (0.0010) [2023-12-27 10:45:57,426][20347] Updated weights for policy 0, policy_version 742411 (0.0007) [2023-12-27 10:45:57,488][20347] Updated weights for policy 0, policy_version 742421 (0.0007) [2023-12-27 10:45:57,546][20347] Updated weights for policy 0, policy_version 742431 (0.0007) [2023-12-27 10:45:58,033][20381] Updated weights for policy 1, policy_version 742077 (0.0008) [2023-12-27 10:45:58,093][20381] Updated weights for policy 1, policy_version 742087 (0.0007) [2023-12-27 10:45:58,147][20381] Updated weights for policy 1, policy_version 742097 (0.0008) [2023-12-27 10:45:58,220][20347] Updated weights for policy 0, policy_version 742441 (0.0010) [2023-12-27 10:45:58,280][20347] Updated weights for policy 0, policy_version 742451 (0.0007) [2023-12-27 10:45:58,348][20347] Updated weights for policy 0, policy_version 742461 (0.0007) [2023-12-27 10:45:58,411][20347] Updated weights for policy 0, policy_version 742471 (0.0008) [2023-12-27 10:45:58,934][20381] Updated weights for policy 1, policy_version 742107 (0.0008) [2023-12-27 10:45:58,992][20381] Updated weights for policy 1, policy_version 742117 (0.0011) [2023-12-27 10:45:59,053][20381] Updated weights for policy 1, policy_version 742127 (0.0011) [2023-12-27 10:45:59,220][20347] Updated weights for policy 0, policy_version 742481 (0.0009) [2023-12-27 10:45:59,287][20347] Updated weights for policy 0, policy_version 742491 (0.0009) [2023-12-27 10:45:59,353][20347] Updated weights for policy 0, policy_version 742501 (0.0010) [2023-12-27 10:45:59,708][20381] Updated weights for policy 1, policy_version 742137 (0.0009) [2023-12-27 10:45:59,770][20381] Updated weights for policy 1, policy_version 742147 (0.0008) [2023-12-27 10:45:59,838][20381] Updated weights for policy 1, policy_version 742157 (0.0007) [2023-12-27 10:45:59,901][20381] Updated weights for policy 1, policy_version 742167 (0.0007) [2023-12-27 10:46:00,080][20347] Updated weights for policy 0, policy_version 742511 (0.0009) [2023-12-27 10:46:00,135][20347] Updated weights for policy 0, policy_version 742521 (0.0008) [2023-12-27 10:46:00,187][20347] Updated weights for policy 0, policy_version 742531 (0.0010) [2023-12-27 10:46:00,590][20381] Updated weights for policy 1, policy_version 742177 (0.0007) [2023-12-27 10:46:00,640][20381] Updated weights for policy 1, policy_version 742187 (0.0007) [2023-12-27 10:46:00,686][20381] Updated weights for policy 1, policy_version 742197 (0.0009) [2023-12-27 10:46:00,808][20347] Updated weights for policy 0, policy_version 742541 (0.0008) [2023-12-27 10:46:00,864][20347] Updated weights for policy 0, policy_version 742551 (0.0008) [2023-12-27 10:46:00,912][20347] Updated weights for policy 0, policy_version 742561 (0.0008) [2023-12-27 10:46:01,340][20381] Updated weights for policy 1, policy_version 742207 (0.0007) [2023-12-27 10:46:01,402][20381] Updated weights for policy 1, policy_version 742217 (0.0007) [2023-12-27 10:46:01,455][20381] Updated weights for policy 1, policy_version 742227 (0.0008) [2023-12-27 10:46:01,641][20347] Updated weights for policy 0, policy_version 742571 (0.0009) [2023-12-27 10:46:01,702][20347] Updated weights for policy 0, policy_version 742581 (0.0007) [2023-12-27 10:46:01,766][20347] Updated weights for policy 0, policy_version 742591 (0.0008) [2023-12-27 10:46:02,150][20381] Updated weights for policy 1, policy_version 742237 (0.0007) [2023-12-27 10:46:02,211][20381] Updated weights for policy 1, policy_version 742247 (0.0008) [2023-12-27 10:46:02,273][20381] Updated weights for policy 1, policy_version 742257 (0.0008) [2023-12-27 10:46:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 20410.6). Total num frames: 380174336. Throughput: 0: 10064.5, 1: 10042.4. Samples: 380183164. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:02,363][19170] Avg episode reward: [(0, '176.650'), (1, '455.980')] [2023-12-27 10:46:02,443][20347] Updated weights for policy 0, policy_version 742601 (0.0007) [2023-12-27 10:46:02,507][20347] Updated weights for policy 0, policy_version 742611 (0.0008) [2023-12-27 10:46:02,569][20347] Updated weights for policy 0, policy_version 742621 (0.0006) [2023-12-27 10:46:02,623][20347] Updated weights for policy 0, policy_version 742631 (0.0007) [2023-12-27 10:46:03,005][20381] Updated weights for policy 1, policy_version 742267 (0.0008) [2023-12-27 10:46:03,064][20381] Updated weights for policy 1, policy_version 742277 (0.0008) [2023-12-27 10:46:03,121][20381] Updated weights for policy 1, policy_version 742287 (0.0006) [2023-12-27 10:46:03,258][20347] Updated weights for policy 0, policy_version 742641 (0.0010) [2023-12-27 10:46:03,307][20347] Updated weights for policy 0, policy_version 742651 (0.0010) [2023-12-27 10:46:03,362][20347] Updated weights for policy 0, policy_version 742661 (0.0009) [2023-12-27 10:46:03,728][20381] Updated weights for policy 1, policy_version 742297 (0.0006) [2023-12-27 10:46:03,778][20381] Updated weights for policy 1, policy_version 742307 (0.0008) [2023-12-27 10:46:03,831][20381] Updated weights for policy 1, policy_version 742317 (0.0008) [2023-12-27 10:46:03,897][20381] Updated weights for policy 1, policy_version 742327 (0.0007) [2023-12-27 10:46:04,048][20347] Updated weights for policy 0, policy_version 742671 (0.0007) [2023-12-27 10:46:04,106][20347] Updated weights for policy 0, policy_version 742681 (0.0008) [2023-12-27 10:46:04,166][20347] Updated weights for policy 0, policy_version 742691 (0.0008) [2023-12-27 10:46:04,582][20381] Updated weights for policy 1, policy_version 742337 (0.0008) [2023-12-27 10:46:04,644][20381] Updated weights for policy 1, policy_version 742347 (0.0008) [2023-12-27 10:46:04,694][20381] Updated weights for policy 1, policy_version 742357 (0.0008) [2023-12-27 10:46:04,840][20347] Updated weights for policy 0, policy_version 742701 (0.0007) [2023-12-27 10:46:04,896][20347] Updated weights for policy 0, policy_version 742711 (0.0008) [2023-12-27 10:46:04,948][20347] Updated weights for policy 0, policy_version 742721 (0.0008) [2023-12-27 10:46:05,390][20381] Updated weights for policy 1, policy_version 742367 (0.0007) [2023-12-27 10:46:05,443][20381] Updated weights for policy 1, policy_version 742377 (0.0006) [2023-12-27 10:46:05,495][20381] Updated weights for policy 1, policy_version 742387 (0.0007) [2023-12-27 10:46:05,658][20347] Updated weights for policy 0, policy_version 742731 (0.0008) [2023-12-27 10:46:05,722][20347] Updated weights for policy 0, policy_version 742741 (0.0010) [2023-12-27 10:46:05,786][20347] Updated weights for policy 0, policy_version 742751 (0.0010) [2023-12-27 10:46:06,140][20381] Updated weights for policy 1, policy_version 742397 (0.0008) [2023-12-27 10:46:06,201][20381] Updated weights for policy 1, policy_version 742407 (0.0006) [2023-12-27 10:46:06,260][20381] Updated weights for policy 1, policy_version 742417 (0.0006) [2023-12-27 10:46:06,448][20347] Updated weights for policy 0, policy_version 742761 (0.0010) [2023-12-27 10:46:06,507][20347] Updated weights for policy 0, policy_version 742771 (0.0008) [2023-12-27 10:46:06,567][20347] Updated weights for policy 0, policy_version 742781 (0.0008) [2023-12-27 10:46:06,631][20347] Updated weights for policy 0, policy_version 742791 (0.0008) [2023-12-27 10:46:06,913][20381] Updated weights for policy 1, policy_version 742427 (0.0008) [2023-12-27 10:46:06,966][20381] Updated weights for policy 1, policy_version 742437 (0.0011) [2023-12-27 10:46:07,019][20381] Updated weights for policy 1, policy_version 742447 (0.0011) [2023-12-27 10:46:07,303][20347] Updated weights for policy 0, policy_version 742801 (0.0008) [2023-12-27 10:46:07,354][20347] Updated weights for policy 0, policy_version 742811 (0.0007) [2023-12-27 10:46:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 20382.8). Total num frames: 380272640. Throughput: 0: 10095.7, 1: 10043.5. Samples: 380244004. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:07,363][19170] Avg episode reward: [(0, '242.140'), (1, '655.510')] [2023-12-27 10:46:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000742456_190095360.pth... [2023-12-27 10:46:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000741272_189792256.pth [2023-12-27 10:46:07,409][20347] Updated weights for policy 0, policy_version 742821 (0.0005) [2023-12-27 10:46:07,427][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000742824_190185472.pth... [2023-12-27 10:46:07,434][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000741640_189882368.pth [2023-12-27 10:46:07,641][20381] Updated weights for policy 1, policy_version 742457 (0.0009) [2023-12-27 10:46:07,703][20381] Updated weights for policy 1, policy_version 742467 (0.0009) [2023-12-27 10:46:07,758][20381] Updated weights for policy 1, policy_version 742477 (0.0008) [2023-12-27 10:46:07,824][20381] Updated weights for policy 1, policy_version 742487 (0.0007) [2023-12-27 10:46:08,047][20347] Updated weights for policy 0, policy_version 742831 (0.0007) [2023-12-27 10:46:08,114][20347] Updated weights for policy 0, policy_version 742841 (0.0009) [2023-12-27 10:46:08,170][20347] Updated weights for policy 0, policy_version 742851 (0.0008) [2023-12-27 10:46:08,600][20381] Updated weights for policy 1, policy_version 742497 (0.0007) [2023-12-27 10:46:08,663][20381] Updated weights for policy 1, policy_version 742507 (0.0008) [2023-12-27 10:46:08,724][20381] Updated weights for policy 1, policy_version 742517 (0.0008) [2023-12-27 10:46:08,839][20347] Updated weights for policy 0, policy_version 742861 (0.0007) [2023-12-27 10:46:08,897][20347] Updated weights for policy 0, policy_version 742871 (0.0008) [2023-12-27 10:46:08,952][20347] Updated weights for policy 0, policy_version 742881 (0.0010) [2023-12-27 10:46:09,385][20381] Updated weights for policy 1, policy_version 742527 (0.0010) [2023-12-27 10:46:09,446][20381] Updated weights for policy 1, policy_version 742537 (0.0007) [2023-12-27 10:46:09,505][20381] Updated weights for policy 1, policy_version 742547 (0.0007) [2023-12-27 10:46:09,642][20347] Updated weights for policy 0, policy_version 742891 (0.0010) [2023-12-27 10:46:09,698][20347] Updated weights for policy 0, policy_version 742901 (0.0010) [2023-12-27 10:46:09,760][20347] Updated weights for policy 0, policy_version 742911 (0.0007) [2023-12-27 10:46:10,233][20381] Updated weights for policy 1, policy_version 742557 (0.0007) [2023-12-27 10:46:10,283][20381] Updated weights for policy 1, policy_version 742567 (0.0011) [2023-12-27 10:46:10,333][20381] Updated weights for policy 1, policy_version 742577 (0.0011) [2023-12-27 10:46:10,512][20347] Updated weights for policy 0, policy_version 742921 (0.0006) [2023-12-27 10:46:10,573][20347] Updated weights for policy 0, policy_version 742931 (0.0010) [2023-12-27 10:46:10,632][20347] Updated weights for policy 0, policy_version 742941 (0.0010) [2023-12-27 10:46:10,680][20347] Updated weights for policy 0, policy_version 742951 (0.0010) [2023-12-27 10:46:11,025][20381] Updated weights for policy 1, policy_version 742587 (0.0010) [2023-12-27 10:46:11,089][20381] Updated weights for policy 1, policy_version 742597 (0.0011) [2023-12-27 10:46:11,154][20381] Updated weights for policy 1, policy_version 742607 (0.0011) [2023-12-27 10:46:11,414][20347] Updated weights for policy 0, policy_version 742961 (0.0008) [2023-12-27 10:46:11,480][20347] Updated weights for policy 0, policy_version 742971 (0.0007) [2023-12-27 10:46:11,539][20347] Updated weights for policy 0, policy_version 742981 (0.0008) [2023-12-27 10:46:11,920][20381] Updated weights for policy 1, policy_version 742617 (0.0011) [2023-12-27 10:46:11,978][20381] Updated weights for policy 1, policy_version 742627 (0.0006) [2023-12-27 10:46:12,039][20381] Updated weights for policy 1, policy_version 742637 (0.0007) [2023-12-27 10:46:12,099][20381] Updated weights for policy 1, policy_version 742647 (0.0008) [2023-12-27 10:46:12,205][20347] Updated weights for policy 0, policy_version 742991 (0.0008) [2023-12-27 10:46:12,266][20347] Updated weights for policy 0, policy_version 743001 (0.0009) [2023-12-27 10:46:12,330][20347] Updated weights for policy 0, policy_version 743011 (0.0009) [2023-12-27 10:46:12,374][19170] Fps is (10 sec: 20456.1, 60 sec: 20203.0, 300 sec: 20382.0). Total num frames: 380379136. Throughput: 0: 10111.3, 1: 10041.4. Samples: 380365120. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:12,375][19170] Avg episode reward: [(0, '296.740'), (1, '595.960')] [2023-12-27 10:46:12,801][20381] Updated weights for policy 1, policy_version 742657 (0.0008) [2023-12-27 10:46:12,861][20381] Updated weights for policy 1, policy_version 742667 (0.0006) [2023-12-27 10:46:12,916][20381] Updated weights for policy 1, policy_version 742677 (0.0005) [2023-12-27 10:46:12,997][20347] Updated weights for policy 0, policy_version 743021 (0.0007) [2023-12-27 10:46:13,054][20347] Updated weights for policy 0, policy_version 743031 (0.0006) [2023-12-27 10:46:13,107][20347] Updated weights for policy 0, policy_version 743041 (0.0006) [2023-12-27 10:46:13,495][20381] Updated weights for policy 1, policy_version 742687 (0.0006) [2023-12-27 10:46:13,545][20381] Updated weights for policy 1, policy_version 742697 (0.0005) [2023-12-27 10:46:13,601][20381] Updated weights for policy 1, policy_version 742707 (0.0005) [2023-12-27 10:46:13,808][20347] Updated weights for policy 0, policy_version 743051 (0.0007) [2023-12-27 10:46:13,863][20347] Updated weights for policy 0, policy_version 743061 (0.0007) [2023-12-27 10:46:13,912][20347] Updated weights for policy 0, policy_version 743071 (0.0007) [2023-12-27 10:46:14,184][20381] Updated weights for policy 1, policy_version 742717 (0.0008) [2023-12-27 10:46:14,236][20381] Updated weights for policy 1, policy_version 742727 (0.0010) [2023-12-27 10:46:14,288][20381] Updated weights for policy 1, policy_version 742737 (0.0010) [2023-12-27 10:46:14,550][20347] Updated weights for policy 0, policy_version 743081 (0.0006) [2023-12-27 10:46:14,616][20347] Updated weights for policy 0, policy_version 743091 (0.0007) [2023-12-27 10:46:14,675][20347] Updated weights for policy 0, policy_version 743101 (0.0007) [2023-12-27 10:46:14,736][20347] Updated weights for policy 0, policy_version 743111 (0.0008) [2023-12-27 10:46:14,929][20381] Updated weights for policy 1, policy_version 742747 (0.0010) [2023-12-27 10:46:14,986][20381] Updated weights for policy 1, policy_version 742757 (0.0008) [2023-12-27 10:46:15,041][20381] Updated weights for policy 1, policy_version 742767 (0.0008) [2023-12-27 10:46:15,426][20347] Updated weights for policy 0, policy_version 743121 (0.0008) [2023-12-27 10:46:15,486][20347] Updated weights for policy 0, policy_version 743131 (0.0007) [2023-12-27 10:46:15,541][20347] Updated weights for policy 0, policy_version 743141 (0.0008) [2023-12-27 10:46:15,761][20381] Updated weights for policy 1, policy_version 742777 (0.0009) [2023-12-27 10:46:15,811][20381] Updated weights for policy 1, policy_version 742787 (0.0008) [2023-12-27 10:46:15,867][20381] Updated weights for policy 1, policy_version 742797 (0.0006) [2023-12-27 10:46:15,921][20381] Updated weights for policy 1, policy_version 742807 (0.0006) [2023-12-27 10:46:16,250][20347] Updated weights for policy 0, policy_version 743151 (0.0007) [2023-12-27 10:46:16,309][20347] Updated weights for policy 0, policy_version 743161 (0.0007) [2023-12-27 10:46:16,364][20347] Updated weights for policy 0, policy_version 743171 (0.0008) [2023-12-27 10:46:16,574][20381] Updated weights for policy 1, policy_version 742817 (0.0010) [2023-12-27 10:46:16,627][20381] Updated weights for policy 1, policy_version 742827 (0.0006) [2023-12-27 10:46:16,674][20381] Updated weights for policy 1, policy_version 742837 (0.0005) [2023-12-27 10:46:17,041][20347] Updated weights for policy 0, policy_version 743181 (0.0006) [2023-12-27 10:46:17,094][20347] Updated weights for policy 0, policy_version 743191 (0.0007) [2023-12-27 10:46:17,148][20347] Updated weights for policy 0, policy_version 743201 (0.0008) [2023-12-27 10:46:17,250][20381] Updated weights for policy 1, policy_version 742847 (0.0007) [2023-12-27 10:46:17,295][20381] Updated weights for policy 1, policy_version 742857 (0.0010) [2023-12-27 10:46:17,344][20381] Updated weights for policy 1, policy_version 742867 (0.0008) [2023-12-27 10:46:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20382.8). Total num frames: 380477440. Throughput: 0: 10129.6, 1: 10139.4. Samples: 380489620. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:17,363][19170] Avg episode reward: [(0, '383.080'), (1, '559.960')] [2023-12-27 10:46:17,789][20347] Updated weights for policy 0, policy_version 743211 (0.0008) [2023-12-27 10:46:17,840][20347] Updated weights for policy 0, policy_version 743221 (0.0008) [2023-12-27 10:46:17,888][20347] Updated weights for policy 0, policy_version 743231 (0.0008) [2023-12-27 10:46:18,020][20381] Updated weights for policy 1, policy_version 742877 (0.0007) [2023-12-27 10:46:18,074][20381] Updated weights for policy 1, policy_version 742887 (0.0008) [2023-12-27 10:46:18,125][20381] Updated weights for policy 1, policy_version 742897 (0.0007) [2023-12-27 10:46:18,608][20347] Updated weights for policy 0, policy_version 743241 (0.0007) [2023-12-27 10:46:18,667][20347] Updated weights for policy 0, policy_version 743251 (0.0008) [2023-12-27 10:46:18,725][20347] Updated weights for policy 0, policy_version 743261 (0.0006) [2023-12-27 10:46:18,780][20347] Updated weights for policy 0, policy_version 743271 (0.0006) [2023-12-27 10:46:18,781][20381] Updated weights for policy 1, policy_version 742907 (0.0006) [2023-12-27 10:46:18,837][20381] Updated weights for policy 1, policy_version 742917 (0.0008) [2023-12-27 10:46:18,893][20381] Updated weights for policy 1, policy_version 742927 (0.0008) [2023-12-27 10:46:19,397][20347] Updated weights for policy 0, policy_version 743281 (0.0008) [2023-12-27 10:46:19,459][20347] Updated weights for policy 0, policy_version 743291 (0.0008) [2023-12-27 10:46:19,525][20347] Updated weights for policy 0, policy_version 743301 (0.0008) [2023-12-27 10:46:19,654][20381] Updated weights for policy 1, policy_version 742937 (0.0009) [2023-12-27 10:46:19,715][20381] Updated weights for policy 1, policy_version 742947 (0.0008) [2023-12-27 10:46:19,777][20381] Updated weights for policy 1, policy_version 742957 (0.0009) [2023-12-27 10:46:19,838][20381] Updated weights for policy 1, policy_version 742967 (0.0007) [2023-12-27 10:46:20,247][20347] Updated weights for policy 0, policy_version 743311 (0.0008) [2023-12-27 10:46:20,305][20347] Updated weights for policy 0, policy_version 743321 (0.0008) [2023-12-27 10:46:20,366][20347] Updated weights for policy 0, policy_version 743331 (0.0008) [2023-12-27 10:46:20,451][20381] Updated weights for policy 1, policy_version 742977 (0.0006) [2023-12-27 10:46:20,517][20381] Updated weights for policy 1, policy_version 742987 (0.0006) [2023-12-27 10:46:20,578][20381] Updated weights for policy 1, policy_version 742997 (0.0006) [2023-12-27 10:46:21,115][20347] Updated weights for policy 0, policy_version 743341 (0.0008) [2023-12-27 10:46:21,174][20347] Updated weights for policy 0, policy_version 743351 (0.0007) [2023-12-27 10:46:21,236][20347] Updated weights for policy 0, policy_version 743361 (0.0008) [2023-12-27 10:46:21,360][20381] Updated weights for policy 1, policy_version 743007 (0.0008) [2023-12-27 10:46:21,427][20381] Updated weights for policy 1, policy_version 743017 (0.0008) [2023-12-27 10:46:21,483][20381] Updated weights for policy 1, policy_version 743027 (0.0008) [2023-12-27 10:46:21,914][20347] Updated weights for policy 0, policy_version 743371 (0.0008) [2023-12-27 10:46:21,972][20347] Updated weights for policy 0, policy_version 743381 (0.0005) [2023-12-27 10:46:22,034][20347] Updated weights for policy 0, policy_version 743391 (0.0006) [2023-12-27 10:46:22,280][20381] Updated weights for policy 1, policy_version 743037 (0.0009) [2023-12-27 10:46:22,343][20381] Updated weights for policy 1, policy_version 743047 (0.0008) [2023-12-27 10:46:22,362][19170] Fps is (10 sec: 19683.7, 60 sec: 20206.9, 300 sec: 20355.0). Total num frames: 380575744. Throughput: 0: 10130.5, 1: 10098.9. Samples: 380551420. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:22,363][19170] Avg episode reward: [(0, '402.290'), (1, '642.890')] [2023-12-27 10:46:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000743400_190332928.pth... [2023-12-27 10:46:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000742216_190029824.pth [2023-12-27 10:46:22,411][20381] Updated weights for policy 1, policy_version 743057 (0.0008) [2023-12-27 10:46:22,452][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000743064_190251008.pth... [2023-12-27 10:46:22,455][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000741880_189947904.pth [2023-12-27 10:46:22,719][20347] Updated weights for policy 0, policy_version 743401 (0.0008) [2023-12-27 10:46:22,786][20347] Updated weights for policy 0, policy_version 743411 (0.0008) [2023-12-27 10:46:22,854][20347] Updated weights for policy 0, policy_version 743421 (0.0005) [2023-12-27 10:46:22,917][20347] Updated weights for policy 0, policy_version 743431 (0.0006) [2023-12-27 10:46:23,088][20381] Updated weights for policy 1, policy_version 743067 (0.0008) [2023-12-27 10:46:23,149][20381] Updated weights for policy 1, policy_version 743077 (0.0008) [2023-12-27 10:46:23,211][20381] Updated weights for policy 1, policy_version 743087 (0.0008) [2023-12-27 10:46:23,542][20347] Updated weights for policy 0, policy_version 743441 (0.0008) [2023-12-27 10:46:23,597][20347] Updated weights for policy 0, policy_version 743451 (0.0008) [2023-12-27 10:46:23,660][20347] Updated weights for policy 0, policy_version 743461 (0.0008) [2023-12-27 10:46:23,836][20381] Updated weights for policy 1, policy_version 743097 (0.0008) [2023-12-27 10:46:23,883][20381] Updated weights for policy 1, policy_version 743107 (0.0007) [2023-12-27 10:46:23,935][20381] Updated weights for policy 1, policy_version 743117 (0.0006) [2023-12-27 10:46:23,997][20381] Updated weights for policy 1, policy_version 743127 (0.0007) [2023-12-27 10:46:24,259][20347] Updated weights for policy 0, policy_version 743471 (0.0009) [2023-12-27 10:46:24,307][20347] Updated weights for policy 0, policy_version 743481 (0.0010) [2023-12-27 10:46:24,356][20347] Updated weights for policy 0, policy_version 743491 (0.0010) [2023-12-27 10:46:24,703][20381] Updated weights for policy 1, policy_version 743137 (0.0008) [2023-12-27 10:46:24,765][20381] Updated weights for policy 1, policy_version 743147 (0.0008) [2023-12-27 10:46:24,822][20381] Updated weights for policy 1, policy_version 743157 (0.0007) [2023-12-27 10:46:24,982][20347] Updated weights for policy 0, policy_version 743501 (0.0008) [2023-12-27 10:46:25,040][20347] Updated weights for policy 0, policy_version 743511 (0.0005) [2023-12-27 10:46:25,101][20347] Updated weights for policy 0, policy_version 743521 (0.0005) [2023-12-27 10:46:25,564][20381] Updated weights for policy 1, policy_version 743167 (0.0007) [2023-12-27 10:46:25,610][20347] Updated weights for policy 0, policy_version 743531 (0.0005) [2023-12-27 10:46:25,628][20381] Updated weights for policy 1, policy_version 743177 (0.0007) [2023-12-27 10:46:25,662][20347] Updated weights for policy 0, policy_version 743541 (0.0005) [2023-12-27 10:46:25,677][20381] Updated weights for policy 1, policy_version 743187 (0.0007) [2023-12-27 10:46:25,718][20347] Updated weights for policy 0, policy_version 743551 (0.0005) [2023-12-27 10:46:26,337][20381] Updated weights for policy 1, policy_version 743197 (0.0007) [2023-12-27 10:46:26,338][20347] Updated weights for policy 0, policy_version 743561 (0.0007) [2023-12-27 10:46:26,391][20347] Updated weights for policy 0, policy_version 743571 (0.0010) [2023-12-27 10:46:26,393][20381] Updated weights for policy 1, policy_version 743207 (0.0006) [2023-12-27 10:46:26,446][20381] Updated weights for policy 1, policy_version 743217 (0.0006) [2023-12-27 10:46:26,450][20347] Updated weights for policy 0, policy_version 743581 (0.0010) [2023-12-27 10:46:26,508][20347] Updated weights for policy 0, policy_version 743591 (0.0008) [2023-12-27 10:46:27,126][20381] Updated weights for policy 1, policy_version 743227 (0.0009) [2023-12-27 10:46:27,161][20347] Updated weights for policy 0, policy_version 743601 (0.0005) [2023-12-27 10:46:27,185][20381] Updated weights for policy 1, policy_version 743237 (0.0008) [2023-12-27 10:46:27,209][20347] Updated weights for policy 0, policy_version 743611 (0.0005) [2023-12-27 10:46:27,233][20381] Updated weights for policy 1, policy_version 743247 (0.0007) [2023-12-27 10:46:27,257][20347] Updated weights for policy 0, policy_version 743621 (0.0005) [2023-12-27 10:46:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 380690432. Throughput: 0: 10245.2, 1: 10064.7. Samples: 380673496. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:27,363][19170] Avg episode reward: [(0, '302.560'), (1, '721.690')] [2023-12-27 10:46:27,843][20347] Updated weights for policy 0, policy_version 743631 (0.0008) [2023-12-27 10:46:27,900][20347] Updated weights for policy 0, policy_version 743641 (0.0010) [2023-12-27 10:46:27,926][20381] Updated weights for policy 1, policy_version 743257 (0.0008) [2023-12-27 10:46:27,949][20347] Updated weights for policy 0, policy_version 743651 (0.0007) [2023-12-27 10:46:27,970][20381] Updated weights for policy 1, policy_version 743267 (0.0008) [2023-12-27 10:46:28,028][20381] Updated weights for policy 1, policy_version 743277 (0.0007) [2023-12-27 10:46:28,083][20381] Updated weights for policy 1, policy_version 743287 (0.0006) [2023-12-27 10:46:28,561][20347] Updated weights for policy 0, policy_version 743661 (0.0008) [2023-12-27 10:46:28,606][20347] Updated weights for policy 0, policy_version 743671 (0.0010) [2023-12-27 10:46:28,651][20347] Updated weights for policy 0, policy_version 743681 (0.0006) [2023-12-27 10:46:28,772][20381] Updated weights for policy 1, policy_version 743297 (0.0008) [2023-12-27 10:46:28,833][20381] Updated weights for policy 1, policy_version 743307 (0.0008) [2023-12-27 10:46:28,885][20381] Updated weights for policy 1, policy_version 743317 (0.0007) [2023-12-27 10:46:29,302][20347] Updated weights for policy 0, policy_version 743691 (0.0006) [2023-12-27 10:46:29,363][20347] Updated weights for policy 0, policy_version 743701 (0.0008) [2023-12-27 10:46:29,423][20347] Updated weights for policy 0, policy_version 743711 (0.0007) [2023-12-27 10:46:29,601][20381] Updated weights for policy 1, policy_version 743327 (0.0008) [2023-12-27 10:46:29,657][20381] Updated weights for policy 1, policy_version 743337 (0.0007) [2023-12-27 10:46:29,713][20381] Updated weights for policy 1, policy_version 743347 (0.0008) [2023-12-27 10:46:30,089][20347] Updated weights for policy 0, policy_version 743721 (0.0006) [2023-12-27 10:46:30,147][20347] Updated weights for policy 0, policy_version 743731 (0.0008) [2023-12-27 10:46:30,203][20347] Updated weights for policy 0, policy_version 743741 (0.0007) [2023-12-27 10:46:30,258][20347] Updated weights for policy 0, policy_version 743751 (0.0008) [2023-12-27 10:46:30,379][20381] Updated weights for policy 1, policy_version 743357 (0.0007) [2023-12-27 10:46:30,437][20381] Updated weights for policy 1, policy_version 743367 (0.0008) [2023-12-27 10:46:30,497][20381] Updated weights for policy 1, policy_version 743377 (0.0008) [2023-12-27 10:46:30,953][20347] Updated weights for policy 0, policy_version 743761 (0.0008) [2023-12-27 10:46:31,000][20347] Updated weights for policy 0, policy_version 743771 (0.0006) [2023-12-27 10:46:31,057][20347] Updated weights for policy 0, policy_version 743781 (0.0008) [2023-12-27 10:46:31,202][20381] Updated weights for policy 1, policy_version 743387 (0.0007) [2023-12-27 10:46:31,259][20381] Updated weights for policy 1, policy_version 743397 (0.0007) [2023-12-27 10:46:31,311][20381] Updated weights for policy 1, policy_version 743407 (0.0007) [2023-12-27 10:46:31,728][20347] Updated weights for policy 0, policy_version 743791 (0.0008) [2023-12-27 10:46:31,792][20347] Updated weights for policy 0, policy_version 743801 (0.0005) [2023-12-27 10:46:31,855][20347] Updated weights for policy 0, policy_version 743811 (0.0008) [2023-12-27 10:46:32,043][20381] Updated weights for policy 1, policy_version 743417 (0.0008) [2023-12-27 10:46:32,107][20381] Updated weights for policy 1, policy_version 743427 (0.0007) [2023-12-27 10:46:32,166][20381] Updated weights for policy 1, policy_version 743437 (0.0009) [2023-12-27 10:46:32,227][20381] Updated weights for policy 1, policy_version 743447 (0.0008) [2023-12-27 10:46:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20382.8). Total num frames: 380788736. Throughput: 0: 10233.4, 1: 10082.2. Samples: 380797228. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:32,363][19170] Avg episode reward: [(0, '476.460'), (1, '706.550')] [2023-12-27 10:46:32,500][20347] Updated weights for policy 0, policy_version 743821 (0.0007) [2023-12-27 10:46:32,559][20347] Updated weights for policy 0, policy_version 743831 (0.0008) [2023-12-27 10:46:32,618][20347] Updated weights for policy 0, policy_version 743841 (0.0005) [2023-12-27 10:46:32,892][20381] Updated weights for policy 1, policy_version 743457 (0.0008) [2023-12-27 10:46:32,948][20381] Updated weights for policy 1, policy_version 743467 (0.0008) [2023-12-27 10:46:32,995][20381] Updated weights for policy 1, policy_version 743477 (0.0007) [2023-12-27 10:46:33,242][20347] Updated weights for policy 0, policy_version 743851 (0.0007) [2023-12-27 10:46:33,296][20347] Updated weights for policy 0, policy_version 743861 (0.0006) [2023-12-27 10:46:33,340][20347] Updated weights for policy 0, policy_version 743871 (0.0009) [2023-12-27 10:46:33,560][20381] Updated weights for policy 1, policy_version 743487 (0.0007) [2023-12-27 10:46:33,617][20381] Updated weights for policy 1, policy_version 743497 (0.0007) [2023-12-27 10:46:33,671][20381] Updated weights for policy 1, policy_version 743508 (0.0010) [2023-12-27 10:46:33,929][20347] Updated weights for policy 0, policy_version 743881 (0.0007) [2023-12-27 10:46:33,982][20347] Updated weights for policy 0, policy_version 743891 (0.0005) [2023-12-27 10:46:34,034][20347] Updated weights for policy 0, policy_version 743901 (0.0005) [2023-12-27 10:46:34,077][20347] Updated weights for policy 0, policy_version 743911 (0.0005) [2023-12-27 10:46:34,383][20381] Updated weights for policy 1, policy_version 743518 (0.0010) [2023-12-27 10:46:34,440][20381] Updated weights for policy 1, policy_version 743528 (0.0011) [2023-12-27 10:46:34,499][20381] Updated weights for policy 1, policy_version 743538 (0.0011) [2023-12-27 10:46:34,737][20347] Updated weights for policy 0, policy_version 743921 (0.0010) [2023-12-27 10:46:34,790][20347] Updated weights for policy 0, policy_version 743931 (0.0010) [2023-12-27 10:46:34,850][20347] Updated weights for policy 0, policy_version 743941 (0.0008) [2023-12-27 10:46:35,166][20381] Updated weights for policy 1, policy_version 743548 (0.0011) [2023-12-27 10:46:35,221][20381] Updated weights for policy 1, policy_version 743558 (0.0009) [2023-12-27 10:46:35,269][20381] Updated weights for policy 1, policy_version 743568 (0.0005) [2023-12-27 10:46:35,443][20347] Updated weights for policy 0, policy_version 743951 (0.0006) [2023-12-27 10:46:35,504][20347] Updated weights for policy 0, policy_version 743962 (0.0008) [2023-12-27 10:46:35,556][20347] Updated weights for policy 0, policy_version 743972 (0.0007) [2023-12-27 10:46:35,884][20381] Updated weights for policy 1, policy_version 743578 (0.0006) [2023-12-27 10:46:35,943][20381] Updated weights for policy 1, policy_version 743588 (0.0007) [2023-12-27 10:46:36,005][20381] Updated weights for policy 1, policy_version 743598 (0.0007) [2023-12-27 10:46:36,064][20381] Updated weights for policy 1, policy_version 743608 (0.0007) [2023-12-27 10:46:36,282][20347] Updated weights for policy 0, policy_version 743982 (0.0010) [2023-12-27 10:46:36,346][20347] Updated weights for policy 0, policy_version 743992 (0.0011) [2023-12-27 10:46:36,404][20347] Updated weights for policy 0, policy_version 744002 (0.0010) [2023-12-27 10:46:36,770][20381] Updated weights for policy 1, policy_version 743618 (0.0005) [2023-12-27 10:46:36,836][20381] Updated weights for policy 1, policy_version 743628 (0.0007) [2023-12-27 10:46:36,891][20381] Updated weights for policy 1, policy_version 743638 (0.0007) [2023-12-27 10:46:37,052][20347] Updated weights for policy 0, policy_version 744012 (0.0008) [2023-12-27 10:46:37,101][20347] Updated weights for policy 0, policy_version 744022 (0.0005) [2023-12-27 10:46:37,156][20347] Updated weights for policy 0, policy_version 744032 (0.0006) [2023-12-27 10:46:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 380895232. Throughput: 0: 10304.6, 1: 10118.6. Samples: 380861824. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:37,363][19170] Avg episode reward: [(0, '362.220'), (1, '774.750')] [2023-12-27 10:46:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000743640_190398464.pth... [2023-12-27 10:46:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000744040_190496768.pth... [2023-12-27 10:46:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000742456_190095360.pth [2023-12-27 10:46:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000742824_190185472.pth [2023-12-27 10:46:37,577][20381] Updated weights for policy 1, policy_version 743648 (0.0011) [2023-12-27 10:46:37,633][20381] Updated weights for policy 1, policy_version 743658 (0.0011) [2023-12-27 10:46:37,696][20381] Updated weights for policy 1, policy_version 743668 (0.0011) [2023-12-27 10:46:37,836][20347] Updated weights for policy 0, policy_version 744042 (0.0008) [2023-12-27 10:46:37,894][20347] Updated weights for policy 0, policy_version 744052 (0.0010) [2023-12-27 10:46:37,954][20347] Updated weights for policy 0, policy_version 744062 (0.0010) [2023-12-27 10:46:38,009][20347] Updated weights for policy 0, policy_version 744072 (0.0010) [2023-12-27 10:46:38,414][20381] Updated weights for policy 1, policy_version 743678 (0.0009) [2023-12-27 10:46:38,473][20381] Updated weights for policy 1, policy_version 743688 (0.0007) [2023-12-27 10:46:38,532][20381] Updated weights for policy 1, policy_version 743698 (0.0008) [2023-12-27 10:46:38,728][20347] Updated weights for policy 0, policy_version 744082 (0.0007) [2023-12-27 10:46:38,790][20347] Updated weights for policy 0, policy_version 744092 (0.0007) [2023-12-27 10:46:38,844][20347] Updated weights for policy 0, policy_version 744102 (0.0006) [2023-12-27 10:46:39,222][20381] Updated weights for policy 1, policy_version 743708 (0.0007) [2023-12-27 10:46:39,290][20381] Updated weights for policy 1, policy_version 743718 (0.0008) [2023-12-27 10:46:39,359][20381] Updated weights for policy 1, policy_version 743728 (0.0008) [2023-12-27 10:46:39,468][20347] Updated weights for policy 0, policy_version 744112 (0.0008) [2023-12-27 10:46:39,519][20347] Updated weights for policy 0, policy_version 744122 (0.0008) [2023-12-27 10:46:39,571][20347] Updated weights for policy 0, policy_version 744132 (0.0008) [2023-12-27 10:46:40,107][20381] Updated weights for policy 1, policy_version 743738 (0.0008) [2023-12-27 10:46:40,168][20381] Updated weights for policy 1, policy_version 743748 (0.0008) [2023-12-27 10:46:40,232][20381] Updated weights for policy 1, policy_version 743758 (0.0007) [2023-12-27 10:46:40,287][20381] Updated weights for policy 1, policy_version 743768 (0.0005) [2023-12-27 10:46:40,330][20347] Updated weights for policy 0, policy_version 744142 (0.0008) [2023-12-27 10:46:40,385][20347] Updated weights for policy 0, policy_version 744152 (0.0007) [2023-12-27 10:46:40,442][20347] Updated weights for policy 0, policy_version 744162 (0.0007) [2023-12-27 10:46:40,971][20381] Updated weights for policy 1, policy_version 743778 (0.0005) [2023-12-27 10:46:41,031][20381] Updated weights for policy 1, policy_version 743788 (0.0007) [2023-12-27 10:46:41,093][20381] Updated weights for policy 1, policy_version 743798 (0.0009) [2023-12-27 10:46:41,172][20347] Updated weights for policy 0, policy_version 744172 (0.0007) [2023-12-27 10:46:41,222][20347] Updated weights for policy 0, policy_version 744182 (0.0008) [2023-12-27 10:46:41,287][20347] Updated weights for policy 0, policy_version 744192 (0.0009) [2023-12-27 10:46:41,764][20381] Updated weights for policy 1, policy_version 743808 (0.0007) [2023-12-27 10:46:41,826][20381] Updated weights for policy 1, policy_version 743818 (0.0006) [2023-12-27 10:46:41,896][20381] Updated weights for policy 1, policy_version 743828 (0.0006) [2023-12-27 10:46:42,085][20347] Updated weights for policy 0, policy_version 744202 (0.0008) [2023-12-27 10:46:42,144][20347] Updated weights for policy 0, policy_version 744212 (0.0009) [2023-12-27 10:46:42,206][20347] Updated weights for policy 0, policy_version 744222 (0.0009) [2023-12-27 10:46:42,271][20347] Updated weights for policy 0, policy_version 744232 (0.0009) [2023-12-27 10:46:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 380993536. Throughput: 0: 10312.3, 1: 10126.2. Samples: 380983248. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:42,363][19170] Avg episode reward: [(0, '324.160'), (1, '804.110')] [2023-12-27 10:46:42,578][20381] Updated weights for policy 1, policy_version 743838 (0.0007) [2023-12-27 10:46:42,641][20381] Updated weights for policy 1, policy_version 743848 (0.0008) [2023-12-27 10:46:42,705][20381] Updated weights for policy 1, policy_version 743858 (0.0007) [2023-12-27 10:46:43,048][20347] Updated weights for policy 0, policy_version 744242 (0.0007) [2023-12-27 10:46:43,100][20347] Updated weights for policy 0, policy_version 744252 (0.0008) [2023-12-27 10:46:43,165][20347] Updated weights for policy 0, policy_version 744262 (0.0008) [2023-12-27 10:46:43,369][20381] Updated weights for policy 1, policy_version 743868 (0.0007) [2023-12-27 10:46:43,427][20381] Updated weights for policy 1, policy_version 743878 (0.0007) [2023-12-27 10:46:43,490][20381] Updated weights for policy 1, policy_version 743888 (0.0005) [2023-12-27 10:46:43,886][20347] Updated weights for policy 0, policy_version 744272 (0.0007) [2023-12-27 10:46:43,944][20347] Updated weights for policy 0, policy_version 744282 (0.0007) [2023-12-27 10:46:43,995][20347] Updated weights for policy 0, policy_version 744292 (0.0008) [2023-12-27 10:46:44,059][20381] Updated weights for policy 1, policy_version 743898 (0.0006) [2023-12-27 10:46:44,107][20381] Updated weights for policy 1, policy_version 743908 (0.0007) [2023-12-27 10:46:44,153][20381] Updated weights for policy 1, policy_version 743918 (0.0005) [2023-12-27 10:46:44,200][20381] Updated weights for policy 1, policy_version 743928 (0.0005) [2023-12-27 10:46:44,706][20347] Updated weights for policy 0, policy_version 744302 (0.0008) [2023-12-27 10:46:44,769][20347] Updated weights for policy 0, policy_version 744312 (0.0008) [2023-12-27 10:46:44,824][20347] Updated weights for policy 0, policy_version 744322 (0.0008) [2023-12-27 10:46:44,928][20381] Updated weights for policy 1, policy_version 743938 (0.0008) [2023-12-27 10:46:44,987][20381] Updated weights for policy 1, policy_version 743948 (0.0008) [2023-12-27 10:46:45,044][20381] Updated weights for policy 1, policy_version 743958 (0.0009) [2023-12-27 10:46:45,451][20347] Updated weights for policy 0, policy_version 744332 (0.0008) [2023-12-27 10:46:45,509][20347] Updated weights for policy 0, policy_version 744342 (0.0008) [2023-12-27 10:46:45,563][20347] Updated weights for policy 0, policy_version 744352 (0.0006) [2023-12-27 10:46:45,713][20381] Updated weights for policy 1, policy_version 743968 (0.0006) [2023-12-27 10:46:45,766][20381] Updated weights for policy 1, policy_version 743978 (0.0006) [2023-12-27 10:46:45,814][20381] Updated weights for policy 1, policy_version 743988 (0.0006) [2023-12-27 10:46:46,212][20347] Updated weights for policy 0, policy_version 744362 (0.0005) [2023-12-27 10:46:46,265][20347] Updated weights for policy 0, policy_version 744372 (0.0005) [2023-12-27 10:46:46,316][20347] Updated weights for policy 0, policy_version 744382 (0.0005) [2023-12-27 10:46:46,371][20347] Updated weights for policy 0, policy_version 744392 (0.0005) [2023-12-27 10:46:46,385][20381] Updated weights for policy 1, policy_version 743998 (0.0009) [2023-12-27 10:46:46,440][20381] Updated weights for policy 1, policy_version 744008 (0.0010) [2023-12-27 10:46:46,493][20381] Updated weights for policy 1, policy_version 744018 (0.0010) [2023-12-27 10:46:47,009][20347] Updated weights for policy 0, policy_version 744402 (0.0011) [2023-12-27 10:46:47,074][20347] Updated weights for policy 0, policy_version 744412 (0.0010) [2023-12-27 10:46:47,138][20347] Updated weights for policy 0, policy_version 744422 (0.0010) [2023-12-27 10:46:47,152][20381] Updated weights for policy 1, policy_version 744028 (0.0009) [2023-12-27 10:46:47,208][20381] Updated weights for policy 1, policy_version 744038 (0.0007) [2023-12-27 10:46:47,263][20381] Updated weights for policy 1, policy_version 744048 (0.0005) [2023-12-27 10:46:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 381100032. Throughput: 0: 10300.0, 1: 10198.3. Samples: 381105588. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:47,363][19170] Avg episode reward: [(0, '422.610'), (1, '712.440')] [2023-12-27 10:46:47,788][20381] Updated weights for policy 1, policy_version 744058 (0.0006) [2023-12-27 10:46:47,839][20381] Updated weights for policy 1, policy_version 744068 (0.0010) [2023-12-27 10:46:47,856][20347] Updated weights for policy 0, policy_version 744432 (0.0007) [2023-12-27 10:46:47,889][20381] Updated weights for policy 1, policy_version 744078 (0.0009) [2023-12-27 10:46:47,912][20347] Updated weights for policy 0, policy_version 744442 (0.0006) [2023-12-27 10:46:47,951][20381] Updated weights for policy 1, policy_version 744088 (0.0007) [2023-12-27 10:46:47,970][20347] Updated weights for policy 0, policy_version 744452 (0.0005) [2023-12-27 10:46:48,554][20347] Updated weights for policy 0, policy_version 744462 (0.0007) [2023-12-27 10:46:48,611][20347] Updated weights for policy 0, policy_version 744472 (0.0006) [2023-12-27 10:46:48,624][20381] Updated weights for policy 1, policy_version 744098 (0.0008) [2023-12-27 10:46:48,661][20347] Updated weights for policy 0, policy_version 744482 (0.0006) [2023-12-27 10:46:48,692][20381] Updated weights for policy 1, policy_version 744108 (0.0011) [2023-12-27 10:46:48,745][20381] Updated weights for policy 1, policy_version 744118 (0.0011) [2023-12-27 10:46:49,398][20347] Updated weights for policy 0, policy_version 744492 (0.0007) [2023-12-27 10:46:49,464][20381] Updated weights for policy 1, policy_version 744128 (0.0011) [2023-12-27 10:46:49,467][20347] Updated weights for policy 0, policy_version 744502 (0.0006) [2023-12-27 10:46:49,528][20381] Updated weights for policy 1, policy_version 744138 (0.0008) [2023-12-27 10:46:49,536][20347] Updated weights for policy 0, policy_version 744512 (0.0005) [2023-12-27 10:46:49,585][20381] Updated weights for policy 1, policy_version 744148 (0.0011) [2023-12-27 10:46:50,196][20347] Updated weights for policy 0, policy_version 744522 (0.0006) [2023-12-27 10:46:50,258][20347] Updated weights for policy 0, policy_version 744532 (0.0007) [2023-12-27 10:46:50,311][20381] Updated weights for policy 1, policy_version 744158 (0.0011) [2023-12-27 10:46:50,317][20347] Updated weights for policy 0, policy_version 744542 (0.0007) [2023-12-27 10:46:50,369][20381] Updated weights for policy 1, policy_version 744168 (0.0011) [2023-12-27 10:46:50,374][20347] Updated weights for policy 0, policy_version 744552 (0.0006) [2023-12-27 10:46:50,422][20381] Updated weights for policy 1, policy_version 744178 (0.0011) [2023-12-27 10:46:51,085][20381] Updated weights for policy 1, policy_version 744188 (0.0011) [2023-12-27 10:46:51,104][20347] Updated weights for policy 0, policy_version 744562 (0.0010) [2023-12-27 10:46:51,146][20381] Updated weights for policy 1, policy_version 744198 (0.0010) [2023-12-27 10:46:51,163][20347] Updated weights for policy 0, policy_version 744572 (0.0007) [2023-12-27 10:46:51,207][20381] Updated weights for policy 1, policy_version 744208 (0.0007) [2023-12-27 10:46:51,216][20347] Updated weights for policy 0, policy_version 744582 (0.0008) [2023-12-27 10:46:51,891][20381] Updated weights for policy 1, policy_version 744218 (0.0007) [2023-12-27 10:46:51,952][20347] Updated weights for policy 0, policy_version 744592 (0.0006) [2023-12-27 10:46:51,975][20381] Updated weights for policy 1, policy_version 744228 (0.0007) [2023-12-27 10:46:52,013][20347] Updated weights for policy 0, policy_version 744602 (0.0007) [2023-12-27 10:46:52,035][20381] Updated weights for policy 1, policy_version 744238 (0.0006) [2023-12-27 10:46:52,077][20347] Updated weights for policy 0, policy_version 744612 (0.0007) [2023-12-27 10:46:52,095][20381] Updated weights for policy 1, policy_version 744248 (0.0007) [2023-12-27 10:46:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 381198336. Throughput: 0: 10293.1, 1: 10216.8. Samples: 381166948. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:52,363][19170] Avg episode reward: [(0, '377.010'), (1, '605.520')] [2023-12-27 10:46:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000744248_190554112.pth... [2023-12-27 10:46:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000744616_190644224.pth... [2023-12-27 10:46:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000743400_190332928.pth [2023-12-27 10:46:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000743064_190251008.pth [2023-12-27 10:46:52,749][20347] Updated weights for policy 0, policy_version 744622 (0.0006) [2023-12-27 10:46:52,755][20381] Updated weights for policy 1, policy_version 744258 (0.0006) [2023-12-27 10:46:52,812][20381] Updated weights for policy 1, policy_version 744268 (0.0006) [2023-12-27 10:46:52,813][20347] Updated weights for policy 0, policy_version 744632 (0.0007) [2023-12-27 10:46:52,866][20381] Updated weights for policy 1, policy_version 744278 (0.0007) [2023-12-27 10:46:52,874][20347] Updated weights for policy 0, policy_version 744642 (0.0006) [2023-12-27 10:46:53,539][20347] Updated weights for policy 0, policy_version 744652 (0.0008) [2023-12-27 10:46:53,560][20381] Updated weights for policy 1, policy_version 744288 (0.0006) [2023-12-27 10:46:53,600][20347] Updated weights for policy 0, policy_version 744662 (0.0007) [2023-12-27 10:46:53,617][20381] Updated weights for policy 1, policy_version 744298 (0.0006) [2023-12-27 10:46:53,661][20347] Updated weights for policy 0, policy_version 744672 (0.0007) [2023-12-27 10:46:53,673][20381] Updated weights for policy 1, policy_version 744308 (0.0006) [2023-12-27 10:46:54,317][20347] Updated weights for policy 0, policy_version 744682 (0.0009) [2023-12-27 10:46:54,375][20347] Updated weights for policy 0, policy_version 744692 (0.0010) [2023-12-27 10:46:54,385][20381] Updated weights for policy 1, policy_version 744318 (0.0006) [2023-12-27 10:46:54,432][20347] Updated weights for policy 0, policy_version 744702 (0.0009) [2023-12-27 10:46:54,444][20381] Updated weights for policy 1, policy_version 744328 (0.0006) [2023-12-27 10:46:54,483][20347] Updated weights for policy 0, policy_version 744712 (0.0005) [2023-12-27 10:46:54,500][20381] Updated weights for policy 1, policy_version 744338 (0.0008) [2023-12-27 10:46:55,141][20347] Updated weights for policy 0, policy_version 744722 (0.0010) [2023-12-27 10:46:55,166][20381] Updated weights for policy 1, policy_version 744348 (0.0007) [2023-12-27 10:46:55,196][20347] Updated weights for policy 0, policy_version 744732 (0.0010) [2023-12-27 10:46:55,216][20381] Updated weights for policy 1, policy_version 744358 (0.0006) [2023-12-27 10:46:55,257][20347] Updated weights for policy 0, policy_version 744742 (0.0010) [2023-12-27 10:46:55,268][20381] Updated weights for policy 1, policy_version 744368 (0.0007) [2023-12-27 10:46:55,939][20347] Updated weights for policy 0, policy_version 744752 (0.0010) [2023-12-27 10:46:55,954][20381] Updated weights for policy 1, policy_version 744378 (0.0008) [2023-12-27 10:46:55,994][20347] Updated weights for policy 0, policy_version 744762 (0.0010) [2023-12-27 10:46:56,003][20381] Updated weights for policy 1, policy_version 744388 (0.0006) [2023-12-27 10:46:56,044][20347] Updated weights for policy 0, policy_version 744772 (0.0010) [2023-12-27 10:46:56,056][20381] Updated weights for policy 1, policy_version 744398 (0.0005) [2023-12-27 10:46:56,109][20381] Updated weights for policy 1, policy_version 744408 (0.0007) [2023-12-27 10:46:56,696][20347] Updated weights for policy 0, policy_version 744782 (0.0007) [2023-12-27 10:46:56,728][20381] Updated weights for policy 1, policy_version 744418 (0.0007) [2023-12-27 10:46:56,746][20347] Updated weights for policy 0, policy_version 744792 (0.0005) [2023-12-27 10:46:56,780][20381] Updated weights for policy 1, policy_version 744428 (0.0007) [2023-12-27 10:46:56,801][20347] Updated weights for policy 0, policy_version 744802 (0.0007) [2023-12-27 10:46:56,825][20381] Updated weights for policy 1, policy_version 744438 (0.0006) [2023-12-27 10:46:57,348][20347] Updated weights for policy 0, policy_version 744812 (0.0008) [2023-12-27 10:46:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 381296640. Throughput: 0: 10298.0, 1: 10213.7. Samples: 381287904. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:46:57,363][19170] Avg episode reward: [(0, '547.230'), (1, '713.820')] [2023-12-27 10:46:57,395][20347] Updated weights for policy 0, policy_version 744822 (0.0005) [2023-12-27 10:46:57,440][20347] Updated weights for policy 0, policy_version 744832 (0.0005) [2023-12-27 10:46:57,573][20381] Updated weights for policy 1, policy_version 744448 (0.0008) [2023-12-27 10:46:57,627][20381] Updated weights for policy 1, policy_version 744458 (0.0008) [2023-12-27 10:46:57,677][20381] Updated weights for policy 1, policy_version 744468 (0.0007) [2023-12-27 10:46:58,069][20347] Updated weights for policy 0, policy_version 744842 (0.0007) [2023-12-27 10:46:58,128][20347] Updated weights for policy 0, policy_version 744852 (0.0007) [2023-12-27 10:46:58,193][20347] Updated weights for policy 0, policy_version 744862 (0.0006) [2023-12-27 10:46:58,253][20347] Updated weights for policy 0, policy_version 744872 (0.0008) [2023-12-27 10:46:58,384][20381] Updated weights for policy 1, policy_version 744478 (0.0008) [2023-12-27 10:46:58,447][20381] Updated weights for policy 1, policy_version 744488 (0.0008) [2023-12-27 10:46:58,507][20381] Updated weights for policy 1, policy_version 744498 (0.0007) [2023-12-27 10:46:59,011][20347] Updated weights for policy 0, policy_version 744882 (0.0008) [2023-12-27 10:46:59,070][20347] Updated weights for policy 0, policy_version 744892 (0.0008) [2023-12-27 10:46:59,126][20347] Updated weights for policy 0, policy_version 744902 (0.0008) [2023-12-27 10:46:59,225][20381] Updated weights for policy 1, policy_version 744508 (0.0008) [2023-12-27 10:46:59,288][20381] Updated weights for policy 1, policy_version 744518 (0.0008) [2023-12-27 10:46:59,358][20381] Updated weights for policy 1, policy_version 744528 (0.0009) [2023-12-27 10:46:59,851][20347] Updated weights for policy 0, policy_version 744912 (0.0008) [2023-12-27 10:46:59,905][20347] Updated weights for policy 0, policy_version 744922 (0.0006) [2023-12-27 10:46:59,962][20347] Updated weights for policy 0, policy_version 744932 (0.0008) [2023-12-27 10:47:00,103][20381] Updated weights for policy 1, policy_version 744538 (0.0007) [2023-12-27 10:47:00,155][20381] Updated weights for policy 1, policy_version 744548 (0.0007) [2023-12-27 10:47:00,207][20381] Updated weights for policy 1, policy_version 744558 (0.0009) [2023-12-27 10:47:00,269][20381] Updated weights for policy 1, policy_version 744568 (0.0008) [2023-12-27 10:47:00,618][20347] Updated weights for policy 0, policy_version 744942 (0.0009) [2023-12-27 10:47:00,675][20347] Updated weights for policy 0, policy_version 744952 (0.0009) [2023-12-27 10:47:00,726][20347] Updated weights for policy 0, policy_version 744962 (0.0010) [2023-12-27 10:47:00,978][20381] Updated weights for policy 1, policy_version 744578 (0.0008) [2023-12-27 10:47:01,034][20381] Updated weights for policy 1, policy_version 744588 (0.0007) [2023-12-27 10:47:01,095][20381] Updated weights for policy 1, policy_version 744598 (0.0006) [2023-12-27 10:47:01,408][20347] Updated weights for policy 0, policy_version 744972 (0.0008) [2023-12-27 10:47:01,454][20347] Updated weights for policy 0, policy_version 744982 (0.0005) [2023-12-27 10:47:01,503][20347] Updated weights for policy 0, policy_version 744992 (0.0006) [2023-12-27 10:47:01,852][20381] Updated weights for policy 1, policy_version 744608 (0.0008) [2023-12-27 10:47:01,899][20381] Updated weights for policy 1, policy_version 744618 (0.0009) [2023-12-27 10:47:01,953][20381] Updated weights for policy 1, policy_version 744628 (0.0009) [2023-12-27 10:47:02,139][20347] Updated weights for policy 0, policy_version 745002 (0.0006) [2023-12-27 10:47:02,201][20347] Updated weights for policy 0, policy_version 745012 (0.0007) [2023-12-27 10:47:02,267][20347] Updated weights for policy 0, policy_version 745022 (0.0007) [2023-12-27 10:47:02,329][20347] Updated weights for policy 0, policy_version 745032 (0.0007) [2023-12-27 10:47:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 381403136. Throughput: 0: 10360.8, 1: 10079.7. Samples: 381409444. Policy #0 lag: (min: 31.0, avg: 32.4, max: 61.0) [2023-12-27 10:47:02,363][19170] Avg episode reward: [(0, '606.200'), (1, '783.560')] [2023-12-27 10:47:02,730][20381] Updated weights for policy 1, policy_version 744638 (0.0010) [2023-12-27 10:47:02,791][20381] Updated weights for policy 1, policy_version 744648 (0.0011) [2023-12-27 10:47:02,850][20381] Updated weights for policy 1, policy_version 744658 (0.0010) [2023-12-27 10:47:03,014][20347] Updated weights for policy 0, policy_version 745042 (0.0007) [2023-12-27 10:47:03,075][20347] Updated weights for policy 0, policy_version 745052 (0.0008) [2023-12-27 10:47:03,136][20347] Updated weights for policy 0, policy_version 745062 (0.0010) [2023-12-27 10:47:03,525][20381] Updated weights for policy 1, policy_version 744668 (0.0008) [2023-12-27 10:47:03,571][20381] Updated weights for policy 1, policy_version 744678 (0.0007) [2023-12-27 10:47:03,623][20381] Updated weights for policy 1, policy_version 744688 (0.0011) [2023-12-27 10:47:03,821][20347] Updated weights for policy 0, policy_version 745072 (0.0007) [2023-12-27 10:47:03,882][20347] Updated weights for policy 0, policy_version 745082 (0.0010) [2023-12-27 10:47:03,942][20347] Updated weights for policy 0, policy_version 745092 (0.0010) [2023-12-27 10:47:04,302][20381] Updated weights for policy 1, policy_version 744699 (0.0012) [2023-12-27 10:47:04,355][20381] Updated weights for policy 1, policy_version 744709 (0.0010) [2023-12-27 10:47:04,405][20381] Updated weights for policy 1, policy_version 744719 (0.0011) [2023-12-27 10:47:04,595][20347] Updated weights for policy 0, policy_version 745102 (0.0010) [2023-12-27 10:47:04,643][20347] Updated weights for policy 0, policy_version 745112 (0.0010) [2023-12-27 10:47:04,689][20347] Updated weights for policy 0, policy_version 745122 (0.0010) [2023-12-27 10:47:05,035][20381] Updated weights for policy 1, policy_version 744729 (0.0010) [2023-12-27 10:47:05,090][20381] Updated weights for policy 1, policy_version 744739 (0.0011) [2023-12-27 10:47:05,142][20381] Updated weights for policy 1, policy_version 744749 (0.0010) [2023-12-27 10:47:05,200][20381] Updated weights for policy 1, policy_version 744759 (0.0011) [2023-12-27 10:47:05,364][20347] Updated weights for policy 0, policy_version 745132 (0.0009) [2023-12-27 10:47:05,414][20347] Updated weights for policy 0, policy_version 745142 (0.0007) [2023-12-27 10:47:05,466][20347] Updated weights for policy 0, policy_version 745152 (0.0007) [2023-12-27 10:47:05,879][20381] Updated weights for policy 1, policy_version 744769 (0.0011) [2023-12-27 10:47:05,934][20381] Updated weights for policy 1, policy_version 744779 (0.0011) [2023-12-27 10:47:05,986][20381] Updated weights for policy 1, policy_version 744789 (0.0010) [2023-12-27 10:47:06,138][20347] Updated weights for policy 0, policy_version 745162 (0.0007) [2023-12-27 10:47:06,195][20347] Updated weights for policy 0, policy_version 745172 (0.0008) [2023-12-27 10:47:06,249][20347] Updated weights for policy 0, policy_version 745182 (0.0008) [2023-12-27 10:47:06,313][20347] Updated weights for policy 0, policy_version 745192 (0.0008) [2023-12-27 10:47:06,645][20381] Updated weights for policy 1, policy_version 744799 (0.0007) [2023-12-27 10:47:06,711][20381] Updated weights for policy 1, policy_version 744809 (0.0007) [2023-12-27 10:47:06,764][20381] Updated weights for policy 1, policy_version 744819 (0.0008) [2023-12-27 10:47:07,019][20347] Updated weights for policy 0, policy_version 745202 (0.0005) [2023-12-27 10:47:07,075][20347] Updated weights for policy 0, policy_version 745212 (0.0005) [2023-12-27 10:47:07,134][20347] Updated weights for policy 0, policy_version 745222 (0.0006) [2023-12-27 10:47:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20327.3). Total num frames: 381501440. Throughput: 0: 10340.7, 1: 10082.1. Samples: 381470448. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:07,363][19170] Avg episode reward: [(0, '682.710'), (1, '814.200')] [2023-12-27 10:47:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000745224_190799872.pth... [2023-12-27 10:47:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000744040_190496768.pth [2023-12-27 10:47:07,388][20381] Updated weights for policy 1, policy_version 744829 (0.0007) [2023-12-27 10:47:07,438][20381] Updated weights for policy 1, policy_version 744839 (0.0007) [2023-12-27 10:47:07,485][20381] Updated weights for policy 1, policy_version 744849 (0.0007) [2023-12-27 10:47:07,517][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000744856_190709760.pth... [2023-12-27 10:47:07,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000743640_190398464.pth [2023-12-27 10:47:07,763][20347] Updated weights for policy 0, policy_version 745232 (0.0009) [2023-12-27 10:47:07,811][20347] Updated weights for policy 0, policy_version 745242 (0.0010) [2023-12-27 10:47:07,862][20347] Updated weights for policy 0, policy_version 745252 (0.0010) [2023-12-27 10:47:08,152][20381] Updated weights for policy 1, policy_version 744859 (0.0007) [2023-12-27 10:47:08,206][20381] Updated weights for policy 1, policy_version 744869 (0.0007) [2023-12-27 10:47:08,258][20381] Updated weights for policy 1, policy_version 744879 (0.0007) [2023-12-27 10:47:08,564][20347] Updated weights for policy 0, policy_version 745262 (0.0009) [2023-12-27 10:47:08,624][20347] Updated weights for policy 0, policy_version 745272 (0.0008) [2023-12-27 10:47:08,681][20347] Updated weights for policy 0, policy_version 745282 (0.0008) [2023-12-27 10:47:08,990][20381] Updated weights for policy 1, policy_version 744889 (0.0006) [2023-12-27 10:47:09,047][20381] Updated weights for policy 1, policy_version 744899 (0.0008) [2023-12-27 10:47:09,104][20381] Updated weights for policy 1, policy_version 744909 (0.0008) [2023-12-27 10:47:09,153][20381] Updated weights for policy 1, policy_version 744919 (0.0008) [2023-12-27 10:47:09,389][20347] Updated weights for policy 0, policy_version 745292 (0.0008) [2023-12-27 10:47:09,457][20347] Updated weights for policy 0, policy_version 745302 (0.0008) [2023-12-27 10:47:09,526][20347] Updated weights for policy 0, policy_version 745312 (0.0008) [2023-12-27 10:47:09,808][20381] Updated weights for policy 1, policy_version 744929 (0.0007) [2023-12-27 10:47:09,870][20381] Updated weights for policy 1, policy_version 744939 (0.0008) [2023-12-27 10:47:09,937][20381] Updated weights for policy 1, policy_version 744949 (0.0007) [2023-12-27 10:47:10,242][20347] Updated weights for policy 0, policy_version 745322 (0.0009) [2023-12-27 10:47:10,307][20347] Updated weights for policy 0, policy_version 745332 (0.0008) [2023-12-27 10:47:10,367][20347] Updated weights for policy 0, policy_version 745342 (0.0008) [2023-12-27 10:47:10,427][20347] Updated weights for policy 0, policy_version 745352 (0.0008) [2023-12-27 10:47:10,570][20381] Updated weights for policy 1, policy_version 744959 (0.0007) [2023-12-27 10:47:10,624][20381] Updated weights for policy 1, policy_version 744969 (0.0008) [2023-12-27 10:47:10,671][20381] Updated weights for policy 1, policy_version 744979 (0.0008) [2023-12-27 10:47:11,094][20347] Updated weights for policy 0, policy_version 745362 (0.0007) [2023-12-27 10:47:11,158][20347] Updated weights for policy 0, policy_version 745372 (0.0008) [2023-12-27 10:47:11,222][20347] Updated weights for policy 0, policy_version 745382 (0.0009) [2023-12-27 10:47:11,480][20381] Updated weights for policy 1, policy_version 744989 (0.0007) [2023-12-27 10:47:11,540][20381] Updated weights for policy 1, policy_version 744999 (0.0008) [2023-12-27 10:47:11,595][20381] Updated weights for policy 1, policy_version 745009 (0.0007) [2023-12-27 10:47:11,895][20347] Updated weights for policy 0, policy_version 745392 (0.0006) [2023-12-27 10:47:11,948][20347] Updated weights for policy 0, policy_version 745402 (0.0008) [2023-12-27 10:47:12,001][20347] Updated weights for policy 0, policy_version 745412 (0.0007) [2023-12-27 10:47:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20347.4, 300 sec: 20327.3). Total num frames: 381599744. Throughput: 0: 10267.0, 1: 10161.7. Samples: 381592788. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:12,363][19170] Avg episode reward: [(0, '693.910'), (1, '804.300')] [2023-12-27 10:47:12,368][20381] Updated weights for policy 1, policy_version 745019 (0.0008) [2023-12-27 10:47:12,424][20381] Updated weights for policy 1, policy_version 745029 (0.0011) [2023-12-27 10:47:12,476][20381] Updated weights for policy 1, policy_version 745039 (0.0010) [2023-12-27 10:47:12,677][20347] Updated weights for policy 0, policy_version 745422 (0.0008) [2023-12-27 10:47:12,740][20347] Updated weights for policy 0, policy_version 745432 (0.0007) [2023-12-27 10:47:12,810][20347] Updated weights for policy 0, policy_version 745442 (0.0007) [2023-12-27 10:47:13,094][20381] Updated weights for policy 1, policy_version 745049 (0.0010) [2023-12-27 10:47:13,142][20381] Updated weights for policy 1, policy_version 745059 (0.0005) [2023-12-27 10:47:13,194][20381] Updated weights for policy 1, policy_version 745069 (0.0007) [2023-12-27 10:47:13,241][20381] Updated weights for policy 1, policy_version 745079 (0.0008) [2023-12-27 10:47:13,468][20347] Updated weights for policy 0, policy_version 745452 (0.0010) [2023-12-27 10:47:13,514][20347] Updated weights for policy 0, policy_version 745462 (0.0008) [2023-12-27 10:47:13,563][20347] Updated weights for policy 0, policy_version 745472 (0.0008) [2023-12-27 10:47:13,932][20381] Updated weights for policy 1, policy_version 745089 (0.0008) [2023-12-27 10:47:13,979][20381] Updated weights for policy 1, policy_version 745099 (0.0006) [2023-12-27 10:47:14,026][20381] Updated weights for policy 1, policy_version 745109 (0.0008) [2023-12-27 10:47:14,264][20347] Updated weights for policy 0, policy_version 745482 (0.0009) [2023-12-27 10:47:14,314][20347] Updated weights for policy 0, policy_version 745492 (0.0008) [2023-12-27 10:47:14,359][20347] Updated weights for policy 0, policy_version 745502 (0.0007) [2023-12-27 10:47:14,402][20347] Updated weights for policy 0, policy_version 745512 (0.0007) [2023-12-27 10:47:14,724][20381] Updated weights for policy 1, policy_version 745119 (0.0007) [2023-12-27 10:47:14,779][20381] Updated weights for policy 1, policy_version 745129 (0.0007) [2023-12-27 10:47:14,838][20381] Updated weights for policy 1, policy_version 745139 (0.0008) [2023-12-27 10:47:15,156][20347] Updated weights for policy 0, policy_version 745522 (0.0007) [2023-12-27 10:47:15,214][20347] Updated weights for policy 0, policy_version 745532 (0.0006) [2023-12-27 10:47:15,273][20347] Updated weights for policy 0, policy_version 745542 (0.0007) [2023-12-27 10:47:15,606][20381] Updated weights for policy 1, policy_version 745149 (0.0008) [2023-12-27 10:47:15,664][20381] Updated weights for policy 1, policy_version 745159 (0.0007) [2023-12-27 10:47:15,721][20381] Updated weights for policy 1, policy_version 745169 (0.0006) [2023-12-27 10:47:15,956][20347] Updated weights for policy 0, policy_version 745552 (0.0007) [2023-12-27 10:47:16,011][20347] Updated weights for policy 0, policy_version 745562 (0.0007) [2023-12-27 10:47:16,069][20347] Updated weights for policy 0, policy_version 745572 (0.0007) [2023-12-27 10:47:16,396][20381] Updated weights for policy 1, policy_version 745179 (0.0008) [2023-12-27 10:47:16,445][20381] Updated weights for policy 1, policy_version 745189 (0.0008) [2023-12-27 10:47:16,497][20381] Updated weights for policy 1, policy_version 745199 (0.0008) [2023-12-27 10:47:16,753][20347] Updated weights for policy 0, policy_version 745582 (0.0008) [2023-12-27 10:47:16,802][20347] Updated weights for policy 0, policy_version 745592 (0.0007) [2023-12-27 10:47:16,860][20347] Updated weights for policy 0, policy_version 745602 (0.0007) [2023-12-27 10:47:17,124][20381] Updated weights for policy 1, policy_version 745209 (0.0007) [2023-12-27 10:47:17,171][20381] Updated weights for policy 1, policy_version 745219 (0.0008) [2023-12-27 10:47:17,225][20381] Updated weights for policy 1, policy_version 745229 (0.0007) [2023-12-27 10:47:17,276][20381] Updated weights for policy 1, policy_version 745239 (0.0008) [2023-12-27 10:47:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 381706240. Throughput: 0: 10174.4, 1: 10188.8. Samples: 381713572. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:17,363][19170] Avg episode reward: [(0, '717.620'), (1, '774.690')] [2023-12-27 10:47:17,593][20347] Updated weights for policy 0, policy_version 745612 (0.0008) [2023-12-27 10:47:17,657][20347] Updated weights for policy 0, policy_version 745622 (0.0008) [2023-12-27 10:47:17,714][20347] Updated weights for policy 0, policy_version 745632 (0.0007) [2023-12-27 10:47:17,896][20381] Updated weights for policy 1, policy_version 745249 (0.0007) [2023-12-27 10:47:17,950][20381] Updated weights for policy 1, policy_version 745259 (0.0006) [2023-12-27 10:47:18,006][20381] Updated weights for policy 1, policy_version 745269 (0.0006) [2023-12-27 10:47:18,428][20347] Updated weights for policy 0, policy_version 745642 (0.0008) [2023-12-27 10:47:18,487][20347] Updated weights for policy 0, policy_version 745652 (0.0007) [2023-12-27 10:47:18,540][20347] Updated weights for policy 0, policy_version 745662 (0.0008) [2023-12-27 10:47:18,548][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 10:47:18,620][20381] Updated weights for policy 1, policy_version 745279 (0.0007) [2023-12-27 10:47:18,676][20381] Updated weights for policy 1, policy_version 745289 (0.0008) [2023-12-27 10:47:18,735][20381] Updated weights for policy 1, policy_version 745299 (0.0006) [2023-12-27 10:47:19,277][20347] Updated weights for policy 0, policy_version 745672 (0.0009) [2023-12-27 10:47:19,348][20347] Updated weights for policy 0, policy_version 745682 (0.0009) [2023-12-27 10:47:19,358][20381] Updated weights for policy 1, policy_version 745309 (0.0008) [2023-12-27 10:47:19,412][20347] Updated weights for policy 0, policy_version 745692 (0.0008) [2023-12-27 10:47:19,422][20381] Updated weights for policy 1, policy_version 745319 (0.0006) [2023-12-27 10:47:19,472][20381] Updated weights for policy 1, policy_version 745329 (0.0006) [2023-12-27 10:47:20,108][20381] Updated weights for policy 1, policy_version 745339 (0.0008) [2023-12-27 10:47:20,131][20347] Updated weights for policy 0, policy_version 745702 (0.0008) [2023-12-27 10:47:20,168][20381] Updated weights for policy 1, policy_version 745349 (0.0008) [2023-12-27 10:47:20,189][20347] Updated weights for policy 0, policy_version 745712 (0.0007) [2023-12-27 10:47:20,235][20381] Updated weights for policy 1, policy_version 745359 (0.0006) [2023-12-27 10:47:20,251][20347] Updated weights for policy 0, policy_version 745722 (0.0007) [2023-12-27 10:47:20,868][20347] Updated weights for policy 0, policy_version 745732 (0.0009) [2023-12-27 10:47:20,926][20347] Updated weights for policy 0, policy_version 745742 (0.0008) [2023-12-27 10:47:20,972][20381] Updated weights for policy 1, policy_version 745369 (0.0009) [2023-12-27 10:47:20,978][20347] Updated weights for policy 0, policy_version 745752 (0.0007) [2023-12-27 10:47:21,035][20381] Updated weights for policy 1, policy_version 745379 (0.0007) [2023-12-27 10:47:21,101][20381] Updated weights for policy 1, policy_version 745389 (0.0008) [2023-12-27 10:47:21,165][20381] Updated weights for policy 1, policy_version 745399 (0.0008) [2023-12-27 10:47:21,768][20347] Updated weights for policy 0, policy_version 745762 (0.0007) [2023-12-27 10:47:21,825][20347] Updated weights for policy 0, policy_version 745772 (0.0008) [2023-12-27 10:47:21,884][20347] Updated weights for policy 0, policy_version 745782 (0.0007) [2023-12-27 10:47:21,935][20381] Updated weights for policy 1, policy_version 745409 (0.0008) [2023-12-27 10:47:21,937][20347] Updated weights for policy 0, policy_version 745792 (0.0008) [2023-12-27 10:47:21,997][20381] Updated weights for policy 1, policy_version 745419 (0.0007) [2023-12-27 10:47:22,058][20381] Updated weights for policy 1, policy_version 745429 (0.0009) [2023-12-27 10:47:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20327.3). Total num frames: 381804544. Throughput: 0: 10093.6, 1: 10214.5. Samples: 381775688. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:22,363][19170] Avg episode reward: [(0, '685.520'), (1, '765.360')] [2023-12-27 10:47:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000745432_190857216.pth... [2023-12-27 10:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000745792_190947328.pth... [2023-12-27 10:47:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000744248_190554112.pth [2023-12-27 10:47:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000744616_190644224.pth [2023-12-27 10:47:22,603][20347] Updated weights for policy 0, policy_version 745802 (0.0005) [2023-12-27 10:47:22,662][20347] Updated weights for policy 0, policy_version 745812 (0.0010) [2023-12-27 10:47:22,722][20347] Updated weights for policy 0, policy_version 745822 (0.0010) [2023-12-27 10:47:22,789][20381] Updated weights for policy 1, policy_version 745439 (0.0008) [2023-12-27 10:47:22,840][20381] Updated weights for policy 1, policy_version 745449 (0.0009) [2023-12-27 10:47:22,893][20381] Updated weights for policy 1, policy_version 745459 (0.0011) [2023-12-27 10:47:23,383][20347] Updated weights for policy 0, policy_version 745832 (0.0007) [2023-12-27 10:47:23,432][20347] Updated weights for policy 0, policy_version 745842 (0.0007) [2023-12-27 10:47:23,480][20347] Updated weights for policy 0, policy_version 745852 (0.0007) [2023-12-27 10:47:23,596][20381] Updated weights for policy 1, policy_version 745469 (0.0009) [2023-12-27 10:47:23,645][20381] Updated weights for policy 1, policy_version 745479 (0.0008) [2023-12-27 10:47:23,697][20381] Updated weights for policy 1, policy_version 745489 (0.0007) [2023-12-27 10:47:24,245][20347] Updated weights for policy 0, policy_version 745862 (0.0007) [2023-12-27 10:47:24,298][20347] Updated weights for policy 0, policy_version 745872 (0.0006) [2023-12-27 10:47:24,350][20381] Updated weights for policy 1, policy_version 745499 (0.0007) [2023-12-27 10:47:24,363][20347] Updated weights for policy 0, policy_version 745882 (0.0007) [2023-12-27 10:47:24,417][20381] Updated weights for policy 1, policy_version 745509 (0.0006) [2023-12-27 10:47:24,476][20381] Updated weights for policy 1, policy_version 745519 (0.0007) [2023-12-27 10:47:25,041][20347] Updated weights for policy 0, policy_version 745892 (0.0007) [2023-12-27 10:47:25,089][20347] Updated weights for policy 0, policy_version 745902 (0.0007) [2023-12-27 10:47:25,145][20347] Updated weights for policy 0, policy_version 745912 (0.0006) [2023-12-27 10:47:25,148][20381] Updated weights for policy 1, policy_version 745529 (0.0007) [2023-12-27 10:47:25,207][20381] Updated weights for policy 1, policy_version 745539 (0.0007) [2023-12-27 10:47:25,270][20381] Updated weights for policy 1, policy_version 745549 (0.0007) [2023-12-27 10:47:25,336][20381] Updated weights for policy 1, policy_version 745559 (0.0006) [2023-12-27 10:47:25,797][20347] Updated weights for policy 0, policy_version 745922 (0.0007) [2023-12-27 10:47:25,858][20347] Updated weights for policy 0, policy_version 745932 (0.0008) [2023-12-27 10:47:25,908][20347] Updated weights for policy 0, policy_version 745942 (0.0006) [2023-12-27 10:47:25,922][20381] Updated weights for policy 1, policy_version 745569 (0.0010) [2023-12-27 10:47:25,964][20347] Updated weights for policy 0, policy_version 745952 (0.0005) [2023-12-27 10:47:25,980][20381] Updated weights for policy 1, policy_version 745579 (0.0010) [2023-12-27 10:47:26,041][20381] Updated weights for policy 1, policy_version 745589 (0.0011) [2023-12-27 10:47:26,607][20347] Updated weights for policy 0, policy_version 745962 (0.0005) [2023-12-27 10:47:26,627][20381] Updated weights for policy 1, policy_version 745599 (0.0007) [2023-12-27 10:47:26,661][20347] Updated weights for policy 0, policy_version 745972 (0.0005) [2023-12-27 10:47:26,684][20381] Updated weights for policy 1, policy_version 745609 (0.0005) [2023-12-27 10:47:26,712][20347] Updated weights for policy 0, policy_version 745982 (0.0005) [2023-12-27 10:47:26,742][20381] Updated weights for policy 1, policy_version 745619 (0.0006) [2023-12-27 10:47:27,237][20347] Updated weights for policy 0, policy_version 745992 (0.0007) [2023-12-27 10:47:27,281][20347] Updated weights for policy 0, policy_version 746002 (0.0007) [2023-12-27 10:47:27,334][20347] Updated weights for policy 0, policy_version 746012 (0.0008) [2023-12-27 10:47:27,359][20381] Updated weights for policy 1, policy_version 745629 (0.0006) [2023-12-27 10:47:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20355.0). Total num frames: 381911040. Throughput: 0: 10077.3, 1: 10198.1. Samples: 381895644. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:27,363][19170] Avg episode reward: [(0, '744.010'), (1, '804.460')] [2023-12-27 10:47:27,418][20381] Updated weights for policy 1, policy_version 745639 (0.0010) [2023-12-27 10:47:27,468][20381] Updated weights for policy 1, policy_version 745649 (0.0010) [2023-12-27 10:47:28,084][20347] Updated weights for policy 0, policy_version 746022 (0.0008) [2023-12-27 10:47:28,097][20381] Updated weights for policy 1, policy_version 745659 (0.0009) [2023-12-27 10:47:28,140][20347] Updated weights for policy 0, policy_version 746032 (0.0006) [2023-12-27 10:47:28,157][20381] Updated weights for policy 1, policy_version 745669 (0.0006) [2023-12-27 10:47:28,196][20347] Updated weights for policy 0, policy_version 746042 (0.0006) [2023-12-27 10:47:28,203][20381] Updated weights for policy 1, policy_version 745679 (0.0005) [2023-12-27 10:47:28,887][20381] Updated weights for policy 1, policy_version 745689 (0.0007) [2023-12-27 10:47:28,934][20347] Updated weights for policy 0, policy_version 746052 (0.0006) [2023-12-27 10:47:28,945][20381] Updated weights for policy 1, policy_version 745699 (0.0006) [2023-12-27 10:47:28,988][20347] Updated weights for policy 0, policy_version 746062 (0.0007) [2023-12-27 10:47:29,001][20381] Updated weights for policy 1, policy_version 745709 (0.0005) [2023-12-27 10:47:29,044][20347] Updated weights for policy 0, policy_version 746073 (0.0008) [2023-12-27 10:47:29,054][20381] Updated weights for policy 1, policy_version 745719 (0.0007) [2023-12-27 10:47:29,681][20347] Updated weights for policy 0, policy_version 746083 (0.0005) [2023-12-27 10:47:29,739][20347] Updated weights for policy 0, policy_version 746093 (0.0006) [2023-12-27 10:47:29,784][20381] Updated weights for policy 1, policy_version 745729 (0.0008) [2023-12-27 10:47:29,798][20347] Updated weights for policy 0, policy_version 746103 (0.0007) [2023-12-27 10:47:29,848][20381] Updated weights for policy 1, policy_version 745739 (0.0007) [2023-12-27 10:47:29,916][20381] Updated weights for policy 1, policy_version 745749 (0.0008) [2023-12-27 10:47:30,420][20347] Updated weights for policy 0, policy_version 746113 (0.0006) [2023-12-27 10:47:30,475][20347] Updated weights for policy 0, policy_version 746123 (0.0007) [2023-12-27 10:47:30,532][20347] Updated weights for policy 0, policy_version 746133 (0.0007) [2023-12-27 10:47:30,592][20347] Updated weights for policy 0, policy_version 746143 (0.0006) [2023-12-27 10:47:30,647][20381] Updated weights for policy 1, policy_version 745759 (0.0008) [2023-12-27 10:47:30,705][20381] Updated weights for policy 1, policy_version 745769 (0.0008) [2023-12-27 10:47:30,757][20381] Updated weights for policy 1, policy_version 745779 (0.0008) [2023-12-27 10:47:31,222][20347] Updated weights for policy 0, policy_version 746153 (0.0008) [2023-12-27 10:47:31,287][20347] Updated weights for policy 0, policy_version 746163 (0.0009) [2023-12-27 10:47:31,351][20347] Updated weights for policy 0, policy_version 746173 (0.0008) [2023-12-27 10:47:31,421][20381] Updated weights for policy 1, policy_version 745789 (0.0008) [2023-12-27 10:47:31,481][20381] Updated weights for policy 1, policy_version 745799 (0.0007) [2023-12-27 10:47:31,540][20381] Updated weights for policy 1, policy_version 745809 (0.0007) [2023-12-27 10:47:32,006][20347] Updated weights for policy 0, policy_version 746183 (0.0008) [2023-12-27 10:47:32,055][20347] Updated weights for policy 0, policy_version 746193 (0.0007) [2023-12-27 10:47:32,107][20347] Updated weights for policy 0, policy_version 746203 (0.0008) [2023-12-27 10:47:32,264][20381] Updated weights for policy 1, policy_version 745819 (0.0009) [2023-12-27 10:47:32,318][20381] Updated weights for policy 1, policy_version 745829 (0.0011) [2023-12-27 10:47:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20299.5). Total num frames: 382009344. Throughput: 0: 10190.6, 1: 10164.7. Samples: 382021576. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:32,363][19170] Avg episode reward: [(0, '740.680'), (1, '813.860')] [2023-12-27 10:47:32,386][20381] Updated weights for policy 1, policy_version 745839 (0.0011) [2023-12-27 10:47:32,844][20347] Updated weights for policy 0, policy_version 746213 (0.0006) [2023-12-27 10:47:32,892][20347] Updated weights for policy 0, policy_version 746223 (0.0006) [2023-12-27 10:47:32,944][20347] Updated weights for policy 0, policy_version 746233 (0.0007) [2023-12-27 10:47:33,085][20381] Updated weights for policy 1, policy_version 745849 (0.0011) [2023-12-27 10:47:33,139][20381] Updated weights for policy 1, policy_version 745859 (0.0010) [2023-12-27 10:47:33,190][20381] Updated weights for policy 1, policy_version 745869 (0.0010) [2023-12-27 10:47:33,241][20381] Updated weights for policy 1, policy_version 745879 (0.0007) [2023-12-27 10:47:33,585][20347] Updated weights for policy 0, policy_version 746243 (0.0009) [2023-12-27 10:47:33,647][20347] Updated weights for policy 0, policy_version 746253 (0.0006) [2023-12-27 10:47:33,706][20347] Updated weights for policy 0, policy_version 746263 (0.0005) [2023-12-27 10:47:33,803][20381] Updated weights for policy 1, policy_version 745889 (0.0005) [2023-12-27 10:47:33,860][20381] Updated weights for policy 1, policy_version 745899 (0.0005) [2023-12-27 10:47:33,907][20381] Updated weights for policy 1, policy_version 745909 (0.0005) [2023-12-27 10:47:34,313][20347] Updated weights for policy 0, policy_version 746273 (0.0009) [2023-12-27 10:47:34,368][20347] Updated weights for policy 0, policy_version 746283 (0.0010) [2023-12-27 10:47:34,430][20347] Updated weights for policy 0, policy_version 746293 (0.0009) [2023-12-27 10:47:34,489][20347] Updated weights for policy 0, policy_version 746303 (0.0010) [2023-12-27 10:47:34,546][20381] Updated weights for policy 1, policy_version 745919 (0.0009) [2023-12-27 10:47:34,608][20381] Updated weights for policy 1, policy_version 745929 (0.0011) [2023-12-27 10:47:34,670][20381] Updated weights for policy 1, policy_version 745939 (0.0011) [2023-12-27 10:47:35,190][20347] Updated weights for policy 0, policy_version 746313 (0.0010) [2023-12-27 10:47:35,250][20347] Updated weights for policy 0, policy_version 746323 (0.0011) [2023-12-27 10:47:35,317][20347] Updated weights for policy 0, policy_version 746333 (0.0011) [2023-12-27 10:47:35,355][20381] Updated weights for policy 1, policy_version 745949 (0.0009) [2023-12-27 10:47:35,409][20381] Updated weights for policy 1, policy_version 745959 (0.0007) [2023-12-27 10:47:35,468][20381] Updated weights for policy 1, policy_version 745969 (0.0007) [2023-12-27 10:47:35,988][20347] Updated weights for policy 0, policy_version 746343 (0.0010) [2023-12-27 10:47:36,047][20347] Updated weights for policy 0, policy_version 746353 (0.0010) [2023-12-27 10:47:36,103][20347] Updated weights for policy 0, policy_version 746363 (0.0010) [2023-12-27 10:47:36,189][20381] Updated weights for policy 1, policy_version 745979 (0.0006) [2023-12-27 10:47:36,252][20381] Updated weights for policy 1, policy_version 745989 (0.0007) [2023-12-27 10:47:36,316][20381] Updated weights for policy 1, policy_version 745999 (0.0008) [2023-12-27 10:47:36,788][20347] Updated weights for policy 0, policy_version 746373 (0.0007) [2023-12-27 10:47:36,851][20347] Updated weights for policy 0, policy_version 746383 (0.0006) [2023-12-27 10:47:36,917][20347] Updated weights for policy 0, policy_version 746393 (0.0006) [2023-12-27 10:47:37,040][20381] Updated weights for policy 1, policy_version 746009 (0.0007) [2023-12-27 10:47:37,103][20381] Updated weights for policy 1, policy_version 746019 (0.0009) [2023-12-27 10:47:37,164][20381] Updated weights for policy 1, policy_version 746029 (0.0010) [2023-12-27 10:47:37,215][20381] Updated weights for policy 1, policy_version 746039 (0.0011) [2023-12-27 10:47:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20343.4, 300 sec: 20355.0). Total num frames: 382115840. Throughput: 0: 10216.2, 1: 10179.1. Samples: 382084744. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:37,364][19170] Avg episode reward: [(0, '733.090'), (1, '726.240')] [2023-12-27 10:47:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000746040_191012864.pth... [2023-12-27 10:47:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000746400_191102976.pth... [2023-12-27 10:47:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000745224_190799872.pth [2023-12-27 10:47:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000744856_190709760.pth [2023-12-27 10:47:37,541][20347] Updated weights for policy 0, policy_version 746403 (0.0007) [2023-12-27 10:47:37,604][20347] Updated weights for policy 0, policy_version 746413 (0.0009) [2023-12-27 10:47:37,658][20347] Updated weights for policy 0, policy_version 746423 (0.0007) [2023-12-27 10:47:37,911][20381] Updated weights for policy 1, policy_version 746049 (0.0010) [2023-12-27 10:47:37,972][20381] Updated weights for policy 1, policy_version 746059 (0.0010) [2023-12-27 10:47:38,031][20381] Updated weights for policy 1, policy_version 746069 (0.0010) [2023-12-27 10:47:38,280][20347] Updated weights for policy 0, policy_version 746433 (0.0008) [2023-12-27 10:47:38,329][20347] Updated weights for policy 0, policy_version 746443 (0.0010) [2023-12-27 10:47:38,384][20347] Updated weights for policy 0, policy_version 746453 (0.0007) [2023-12-27 10:47:38,441][20347] Updated weights for policy 0, policy_version 746463 (0.0009) [2023-12-27 10:47:38,739][20381] Updated weights for policy 1, policy_version 746079 (0.0008) [2023-12-27 10:47:38,795][20381] Updated weights for policy 1, policy_version 746089 (0.0006) [2023-12-27 10:47:38,864][20381] Updated weights for policy 1, policy_version 746099 (0.0011) [2023-12-27 10:47:39,218][20347] Updated weights for policy 0, policy_version 746473 (0.0010) [2023-12-27 10:47:39,283][20347] Updated weights for policy 0, policy_version 746483 (0.0009) [2023-12-27 10:47:39,349][20347] Updated weights for policy 0, policy_version 746493 (0.0008) [2023-12-27 10:47:39,580][20381] Updated weights for policy 1, policy_version 746109 (0.0009) [2023-12-27 10:47:39,637][20381] Updated weights for policy 1, policy_version 746119 (0.0009) [2023-12-27 10:47:39,701][20381] Updated weights for policy 1, policy_version 746129 (0.0008) [2023-12-27 10:47:40,005][20347] Updated weights for policy 0, policy_version 746503 (0.0008) [2023-12-27 10:47:40,065][20347] Updated weights for policy 0, policy_version 746513 (0.0008) [2023-12-27 10:47:40,134][20347] Updated weights for policy 0, policy_version 746523 (0.0008) [2023-12-27 10:47:40,485][20381] Updated weights for policy 1, policy_version 746139 (0.0007) [2023-12-27 10:47:40,543][20381] Updated weights for policy 1, policy_version 746149 (0.0007) [2023-12-27 10:47:40,600][20381] Updated weights for policy 1, policy_version 746159 (0.0006) [2023-12-27 10:47:40,844][20347] Updated weights for policy 0, policy_version 746533 (0.0009) [2023-12-27 10:47:40,893][20347] Updated weights for policy 0, policy_version 746543 (0.0010) [2023-12-27 10:47:40,951][20347] Updated weights for policy 0, policy_version 746553 (0.0010) [2023-12-27 10:47:41,241][20381] Updated weights for policy 1, policy_version 746169 (0.0007) [2023-12-27 10:47:41,300][20381] Updated weights for policy 1, policy_version 746179 (0.0006) [2023-12-27 10:47:41,362][20381] Updated weights for policy 1, policy_version 746189 (0.0009) [2023-12-27 10:47:41,431][20381] Updated weights for policy 1, policy_version 746199 (0.0008) [2023-12-27 10:47:41,656][20347] Updated weights for policy 0, policy_version 746563 (0.0010) [2023-12-27 10:47:41,723][20347] Updated weights for policy 0, policy_version 746573 (0.0008) [2023-12-27 10:47:41,779][20347] Updated weights for policy 0, policy_version 746583 (0.0006) [2023-12-27 10:47:42,119][20381] Updated weights for policy 1, policy_version 746209 (0.0008) [2023-12-27 10:47:42,165][20381] Updated weights for policy 1, policy_version 746219 (0.0007) [2023-12-27 10:47:42,225][20381] Updated weights for policy 1, policy_version 746229 (0.0008) [2023-12-27 10:47:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20327.3). Total num frames: 382214144. Throughput: 0: 10213.2, 1: 10142.1. Samples: 382203896. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:42,363][19170] Avg episode reward: [(0, '694.510'), (1, '784.810')] [2023-12-27 10:47:42,502][20347] Updated weights for policy 0, policy_version 746593 (0.0008) [2023-12-27 10:47:42,563][20347] Updated weights for policy 0, policy_version 746603 (0.0009) [2023-12-27 10:47:42,626][20347] Updated weights for policy 0, policy_version 746613 (0.0008) [2023-12-27 10:47:42,683][20347] Updated weights for policy 0, policy_version 746623 (0.0010) [2023-12-27 10:47:42,908][20381] Updated weights for policy 1, policy_version 746239 (0.0007) [2023-12-27 10:47:42,953][20381] Updated weights for policy 1, policy_version 746249 (0.0008) [2023-12-27 10:47:43,014][20381] Updated weights for policy 1, policy_version 746259 (0.0008) [2023-12-27 10:47:43,315][20347] Updated weights for policy 0, policy_version 746633 (0.0006) [2023-12-27 10:47:43,365][20347] Updated weights for policy 0, policy_version 746643 (0.0005) [2023-12-27 10:47:43,423][20347] Updated weights for policy 0, policy_version 746653 (0.0005) [2023-12-27 10:47:43,588][20381] Updated weights for policy 1, policy_version 746269 (0.0008) [2023-12-27 10:47:43,634][20381] Updated weights for policy 1, policy_version 746279 (0.0007) [2023-12-27 10:47:43,692][20381] Updated weights for policy 1, policy_version 746289 (0.0006) [2023-12-27 10:47:44,037][20347] Updated weights for policy 0, policy_version 746663 (0.0009) [2023-12-27 10:47:44,079][20347] Updated weights for policy 0, policy_version 746673 (0.0010) [2023-12-27 10:47:44,130][20347] Updated weights for policy 0, policy_version 746683 (0.0009) [2023-12-27 10:47:44,295][20381] Updated weights for policy 1, policy_version 746299 (0.0005) [2023-12-27 10:47:44,349][20381] Updated weights for policy 1, policy_version 746309 (0.0006) [2023-12-27 10:47:44,413][20381] Updated weights for policy 1, policy_version 746319 (0.0007) [2023-12-27 10:47:44,802][20347] Updated weights for policy 0, policy_version 746693 (0.0009) [2023-12-27 10:47:44,862][20347] Updated weights for policy 0, policy_version 746703 (0.0009) [2023-12-27 10:47:44,921][20347] Updated weights for policy 0, policy_version 746713 (0.0010) [2023-12-27 10:47:45,099][20381] Updated weights for policy 1, policy_version 746329 (0.0007) [2023-12-27 10:47:45,156][20381] Updated weights for policy 1, policy_version 746339 (0.0007) [2023-12-27 10:47:45,212][20381] Updated weights for policy 1, policy_version 746349 (0.0008) [2023-12-27 10:47:45,268][20381] Updated weights for policy 1, policy_version 746359 (0.0007) [2023-12-27 10:47:45,540][20347] Updated weights for policy 0, policy_version 746723 (0.0010) [2023-12-27 10:47:45,604][20347] Updated weights for policy 0, policy_version 746733 (0.0008) [2023-12-27 10:47:45,666][20347] Updated weights for policy 0, policy_version 746743 (0.0009) [2023-12-27 10:47:45,958][20381] Updated weights for policy 1, policy_version 746369 (0.0006) [2023-12-27 10:47:46,021][20381] Updated weights for policy 1, policy_version 746379 (0.0005) [2023-12-27 10:47:46,073][20381] Updated weights for policy 1, policy_version 746389 (0.0005) [2023-12-27 10:47:46,330][20347] Updated weights for policy 0, policy_version 746753 (0.0010) [2023-12-27 10:47:46,382][20347] Updated weights for policy 0, policy_version 746763 (0.0010) [2023-12-27 10:47:46,437][20347] Updated weights for policy 0, policy_version 746773 (0.0010) [2023-12-27 10:47:46,492][20347] Updated weights for policy 0, policy_version 746783 (0.0010) [2023-12-27 10:47:46,650][20381] Updated weights for policy 1, policy_version 746399 (0.0007) [2023-12-27 10:47:46,715][20381] Updated weights for policy 1, policy_version 746409 (0.0008) [2023-12-27 10:47:46,773][20381] Updated weights for policy 1, policy_version 746419 (0.0008) [2023-12-27 10:47:47,191][20347] Updated weights for policy 0, policy_version 746793 (0.0010) [2023-12-27 10:47:47,255][20347] Updated weights for policy 0, policy_version 746803 (0.0010) [2023-12-27 10:47:47,313][20347] Updated weights for policy 0, policy_version 746813 (0.0010) [2023-12-27 10:47:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20343.5, 300 sec: 20355.0). Total num frames: 382320640. Throughput: 0: 10191.8, 1: 10269.0. Samples: 382330180. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:47,363][19170] Avg episode reward: [(0, '800.150'), (1, '765.420')] [2023-12-27 10:47:47,379][20381] Updated weights for policy 1, policy_version 746429 (0.0007) [2023-12-27 10:47:47,430][20381] Updated weights for policy 1, policy_version 746439 (0.0005) [2023-12-27 10:47:47,489][20381] Updated weights for policy 1, policy_version 746449 (0.0006) [2023-12-27 10:47:47,489][20292] KL-divergence is very high: 100.9452 [2023-12-27 10:47:47,511][20292] KL-divergence is very high: 125.4471 [2023-12-27 10:47:47,972][20347] Updated weights for policy 0, policy_version 746823 (0.0010) [2023-12-27 10:47:48,029][20381] Updated weights for policy 1, policy_version 746459 (0.0007) [2023-12-27 10:47:48,036][20347] Updated weights for policy 0, policy_version 746833 (0.0010) [2023-12-27 10:47:48,083][20381] Updated weights for policy 1, policy_version 746469 (0.0010) [2023-12-27 10:47:48,094][20347] Updated weights for policy 0, policy_version 746843 (0.0010) [2023-12-27 10:47:48,135][20381] Updated weights for policy 1, policy_version 746479 (0.0010) [2023-12-27 10:47:48,730][20381] Updated weights for policy 1, policy_version 746489 (0.0010) [2023-12-27 10:47:48,780][20347] Updated weights for policy 0, policy_version 746853 (0.0010) [2023-12-27 10:47:48,785][20381] Updated weights for policy 1, policy_version 746499 (0.0009) [2023-12-27 10:47:48,839][20347] Updated weights for policy 0, policy_version 746863 (0.0010) [2023-12-27 10:47:48,841][20381] Updated weights for policy 1, policy_version 746509 (0.0010) [2023-12-27 10:47:48,895][20347] Updated weights for policy 0, policy_version 746873 (0.0010) [2023-12-27 10:47:48,897][20381] Updated weights for policy 1, policy_version 746519 (0.0010) [2023-12-27 10:47:49,575][20381] Updated weights for policy 1, policy_version 746529 (0.0007) [2023-12-27 10:47:49,583][20347] Updated weights for policy 0, policy_version 746883 (0.0009) [2023-12-27 10:47:49,628][20381] Updated weights for policy 1, policy_version 746539 (0.0007) [2023-12-27 10:47:49,643][20347] Updated weights for policy 0, policy_version 746893 (0.0007) [2023-12-27 10:47:49,682][20381] Updated weights for policy 1, policy_version 746549 (0.0006) [2023-12-27 10:47:49,698][20347] Updated weights for policy 0, policy_version 746903 (0.0007) [2023-12-27 10:47:50,382][20347] Updated weights for policy 0, policy_version 746913 (0.0008) [2023-12-27 10:47:50,414][20381] Updated weights for policy 1, policy_version 746559 (0.0007) [2023-12-27 10:47:50,435][20347] Updated weights for policy 0, policy_version 746923 (0.0006) [2023-12-27 10:47:50,466][20381] Updated weights for policy 1, policy_version 746569 (0.0006) [2023-12-27 10:47:50,482][20347] Updated weights for policy 0, policy_version 746933 (0.0007) [2023-12-27 10:47:50,519][20381] Updated weights for policy 1, policy_version 746579 (0.0006) [2023-12-27 10:47:50,529][20347] Updated weights for policy 0, policy_version 746943 (0.0006) [2023-12-27 10:47:51,177][20381] Updated weights for policy 1, policy_version 746589 (0.0006) [2023-12-27 10:47:51,237][20381] Updated weights for policy 1, policy_version 746599 (0.0007) [2023-12-27 10:47:51,241][20347] Updated weights for policy 0, policy_version 746953 (0.0006) [2023-12-27 10:47:51,299][20381] Updated weights for policy 1, policy_version 746609 (0.0007) [2023-12-27 10:47:51,304][20347] Updated weights for policy 0, policy_version 746963 (0.0006) [2023-12-27 10:47:51,369][20347] Updated weights for policy 0, policy_version 746973 (0.0008) [2023-12-27 10:47:51,959][20381] Updated weights for policy 1, policy_version 746619 (0.0008) [2023-12-27 10:47:52,022][20381] Updated weights for policy 1, policy_version 746629 (0.0008) [2023-12-27 10:47:52,042][20347] Updated weights for policy 0, policy_version 746983 (0.0007) [2023-12-27 10:47:52,078][20381] Updated weights for policy 1, policy_version 746639 (0.0006) [2023-12-27 10:47:52,105][20347] Updated weights for policy 0, policy_version 746993 (0.0007) [2023-12-27 10:47:52,159][20347] Updated weights for policy 0, policy_version 747003 (0.0006) [2023-12-27 10:47:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20327.3). Total num frames: 382427136. Throughput: 0: 10181.1, 1: 10318.2. Samples: 382392920. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:52,363][19170] Avg episode reward: [(0, '752.670'), (1, '726.380')] [2023-12-27 10:47:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000746648_191168512.pth... [2023-12-27 10:47:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000747008_191258624.pth... [2023-12-27 10:47:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000745792_190947328.pth [2023-12-27 10:47:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000745432_190857216.pth [2023-12-27 10:47:52,821][20381] Updated weights for policy 1, policy_version 746649 (0.0008) [2023-12-27 10:47:52,824][20347] Updated weights for policy 0, policy_version 747013 (0.0006) [2023-12-27 10:47:52,880][20381] Updated weights for policy 1, policy_version 746659 (0.0007) [2023-12-27 10:47:52,885][20347] Updated weights for policy 0, policy_version 747023 (0.0006) [2023-12-27 10:47:52,934][20381] Updated weights for policy 1, policy_version 746669 (0.0009) [2023-12-27 10:47:52,943][20347] Updated weights for policy 0, policy_version 747033 (0.0005) [2023-12-27 10:47:52,984][20381] Updated weights for policy 1, policy_version 746679 (0.0007) [2023-12-27 10:47:53,563][20347] Updated weights for policy 0, policy_version 747043 (0.0006) [2023-12-27 10:47:53,615][20347] Updated weights for policy 0, policy_version 747053 (0.0006) [2023-12-27 10:47:53,643][20381] Updated weights for policy 1, policy_version 746689 (0.0008) [2023-12-27 10:47:53,666][20347] Updated weights for policy 0, policy_version 747063 (0.0005) [2023-12-27 10:47:53,696][20381] Updated weights for policy 1, policy_version 746699 (0.0010) [2023-12-27 10:47:53,751][20381] Updated weights for policy 1, policy_version 746709 (0.0010) [2023-12-27 10:47:54,355][20347] Updated weights for policy 0, policy_version 747073 (0.0006) [2023-12-27 10:47:54,419][20347] Updated weights for policy 0, policy_version 747083 (0.0006) [2023-12-27 10:47:54,428][20381] Updated weights for policy 1, policy_version 746719 (0.0008) [2023-12-27 10:47:54,476][20347] Updated weights for policy 0, policy_version 747093 (0.0006) [2023-12-27 10:47:54,485][20381] Updated weights for policy 1, policy_version 746729 (0.0006) [2023-12-27 10:47:54,533][20347] Updated weights for policy 0, policy_version 747103 (0.0006) [2023-12-27 10:47:54,542][20381] Updated weights for policy 1, policy_version 746739 (0.0005) [2023-12-27 10:47:55,117][20347] Updated weights for policy 0, policy_version 747113 (0.0005) [2023-12-27 10:47:55,174][20347] Updated weights for policy 0, policy_version 747123 (0.0007) [2023-12-27 10:47:55,176][20381] Updated weights for policy 1, policy_version 746749 (0.0009) [2023-12-27 10:47:55,228][20381] Updated weights for policy 1, policy_version 746759 (0.0010) [2023-12-27 10:47:55,230][20347] Updated weights for policy 0, policy_version 747133 (0.0007) [2023-12-27 10:47:55,276][20381] Updated weights for policy 1, policy_version 746769 (0.0010) [2023-12-27 10:47:55,825][20347] Updated weights for policy 0, policy_version 747143 (0.0009) [2023-12-27 10:47:55,884][20347] Updated weights for policy 0, policy_version 747153 (0.0010) [2023-12-27 10:47:55,929][20381] Updated weights for policy 1, policy_version 746779 (0.0010) [2023-12-27 10:47:55,937][20347] Updated weights for policy 0, policy_version 747163 (0.0010) [2023-12-27 10:47:55,985][20381] Updated weights for policy 1, policy_version 746789 (0.0011) [2023-12-27 10:47:56,037][20381] Updated weights for policy 1, policy_version 746799 (0.0010) [2023-12-27 10:47:56,591][20347] Updated weights for policy 0, policy_version 747173 (0.0010) [2023-12-27 10:47:56,638][20347] Updated weights for policy 0, policy_version 747183 (0.0010) [2023-12-27 10:47:56,688][20347] Updated weights for policy 0, policy_version 747193 (0.0010) [2023-12-27 10:47:56,716][20381] Updated weights for policy 1, policy_version 746809 (0.0010) [2023-12-27 10:47:56,779][20381] Updated weights for policy 1, policy_version 746819 (0.0010) [2023-12-27 10:47:56,830][20381] Updated weights for policy 1, policy_version 746829 (0.0010) [2023-12-27 10:47:56,890][20381] Updated weights for policy 1, policy_version 746839 (0.0006) [2023-12-27 10:47:57,347][20347] Updated weights for policy 0, policy_version 747203 (0.0010) [2023-12-27 10:47:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 382525440. Throughput: 0: 10245.5, 1: 10315.4. Samples: 382518028. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:47:57,363][19170] Avg episode reward: [(0, '600.190'), (1, '700.190')] [2023-12-27 10:47:57,395][20347] Updated weights for policy 0, policy_version 747213 (0.0010) [2023-12-27 10:47:57,446][20347] Updated weights for policy 0, policy_version 747223 (0.0010) [2023-12-27 10:47:57,459][20381] Updated weights for policy 1, policy_version 746849 (0.0005) [2023-12-27 10:47:57,504][20381] Updated weights for policy 1, policy_version 746859 (0.0005) [2023-12-27 10:47:57,550][20381] Updated weights for policy 1, policy_version 746869 (0.0005) [2023-12-27 10:47:58,086][20347] Updated weights for policy 0, policy_version 747233 (0.0010) [2023-12-27 10:47:58,133][20347] Updated weights for policy 0, policy_version 747243 (0.0010) [2023-12-27 10:47:58,184][20347] Updated weights for policy 0, policy_version 747253 (0.0010) [2023-12-27 10:47:58,188][20381] Updated weights for policy 1, policy_version 746879 (0.0007) [2023-12-27 10:47:58,244][20347] Updated weights for policy 0, policy_version 747263 (0.0010) [2023-12-27 10:47:58,251][20381] Updated weights for policy 1, policy_version 746889 (0.0009) [2023-12-27 10:47:58,311][20381] Updated weights for policy 1, policy_version 746899 (0.0011) [2023-12-27 10:47:59,020][20347] Updated weights for policy 0, policy_version 747273 (0.0008) [2023-12-27 10:47:59,059][20381] Updated weights for policy 1, policy_version 746909 (0.0009) [2023-12-27 10:47:59,074][20347] Updated weights for policy 0, policy_version 747283 (0.0007) [2023-12-27 10:47:59,117][20381] Updated weights for policy 1, policy_version 746919 (0.0008) [2023-12-27 10:47:59,137][20347] Updated weights for policy 0, policy_version 747293 (0.0007) [2023-12-27 10:47:59,181][20381] Updated weights for policy 1, policy_version 746929 (0.0008) [2023-12-27 10:47:59,778][20347] Updated weights for policy 0, policy_version 747303 (0.0006) [2023-12-27 10:47:59,839][20347] Updated weights for policy 0, policy_version 747313 (0.0008) [2023-12-27 10:47:59,901][20347] Updated weights for policy 0, policy_version 747323 (0.0008) [2023-12-27 10:47:59,928][20381] Updated weights for policy 1, policy_version 746939 (0.0008) [2023-12-27 10:47:59,993][20381] Updated weights for policy 1, policy_version 746949 (0.0010) [2023-12-27 10:48:00,052][20381] Updated weights for policy 1, policy_version 746959 (0.0010) [2023-12-27 10:48:00,647][20347] Updated weights for policy 0, policy_version 747333 (0.0007) [2023-12-27 10:48:00,654][20381] Updated weights for policy 1, policy_version 746969 (0.0010) [2023-12-27 10:48:00,702][20347] Updated weights for policy 0, policy_version 747343 (0.0005) [2023-12-27 10:48:00,712][20381] Updated weights for policy 1, policy_version 746979 (0.0010) [2023-12-27 10:48:00,757][20347] Updated weights for policy 0, policy_version 747353 (0.0007) [2023-12-27 10:48:00,764][20381] Updated weights for policy 1, policy_version 746989 (0.0006) [2023-12-27 10:48:00,819][20381] Updated weights for policy 1, policy_version 746999 (0.0005) [2023-12-27 10:48:01,432][20381] Updated weights for policy 1, policy_version 747009 (0.0008) [2023-12-27 10:48:01,487][20381] Updated weights for policy 1, policy_version 747019 (0.0008) [2023-12-27 10:48:01,490][20347] Updated weights for policy 0, policy_version 747363 (0.0007) [2023-12-27 10:48:01,534][20381] Updated weights for policy 1, policy_version 747029 (0.0008) [2023-12-27 10:48:01,541][20347] Updated weights for policy 0, policy_version 747373 (0.0007) [2023-12-27 10:48:01,599][20347] Updated weights for policy 0, policy_version 747383 (0.0007) [2023-12-27 10:48:02,241][20381] Updated weights for policy 1, policy_version 747039 (0.0008) [2023-12-27 10:48:02,298][20381] Updated weights for policy 1, policy_version 747049 (0.0010) [2023-12-27 10:48:02,311][20347] Updated weights for policy 0, policy_version 747393 (0.0008) [2023-12-27 10:48:02,357][20381] Updated weights for policy 1, policy_version 747059 (0.0010) [2023-12-27 10:48:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20299.5). Total num frames: 382623744. Throughput: 0: 10251.6, 1: 10355.0. Samples: 382640872. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:02,363][19170] Avg episode reward: [(0, '759.100'), (1, '649.960')] [2023-12-27 10:48:02,374][20347] Updated weights for policy 0, policy_version 747403 (0.0008) [2023-12-27 10:48:02,430][20347] Updated weights for policy 0, policy_version 747413 (0.0007) [2023-12-27 10:48:02,486][20347] Updated weights for policy 0, policy_version 747423 (0.0007) [2023-12-27 10:48:03,036][20381] Updated weights for policy 1, policy_version 747069 (0.0009) [2023-12-27 10:48:03,096][20381] Updated weights for policy 1, policy_version 747079 (0.0011) [2023-12-27 10:48:03,153][20381] Updated weights for policy 1, policy_version 747089 (0.0008) [2023-12-27 10:48:03,199][20347] Updated weights for policy 0, policy_version 747433 (0.0006) [2023-12-27 10:48:03,244][20347] Updated weights for policy 0, policy_version 747443 (0.0006) [2023-12-27 10:48:03,286][20347] Updated weights for policy 0, policy_version 747453 (0.0005) [2023-12-27 10:48:03,695][20381] Updated weights for policy 1, policy_version 747099 (0.0009) [2023-12-27 10:48:03,751][20381] Updated weights for policy 1, policy_version 747109 (0.0007) [2023-12-27 10:48:03,805][20381] Updated weights for policy 1, policy_version 747119 (0.0010) [2023-12-27 10:48:03,857][20347] Updated weights for policy 0, policy_version 747463 (0.0007) [2023-12-27 10:48:03,909][20347] Updated weights for policy 0, policy_version 747473 (0.0007) [2023-12-27 10:48:03,965][20347] Updated weights for policy 0, policy_version 747483 (0.0007) [2023-12-27 10:48:04,498][20381] Updated weights for policy 1, policy_version 747129 (0.0008) [2023-12-27 10:48:04,550][20381] Updated weights for policy 1, policy_version 747139 (0.0007) [2023-12-27 10:48:04,604][20381] Updated weights for policy 1, policy_version 747149 (0.0008) [2023-12-27 10:48:04,624][20347] Updated weights for policy 0, policy_version 747493 (0.0006) [2023-12-27 10:48:04,660][20381] Updated weights for policy 1, policy_version 747159 (0.0006) [2023-12-27 10:48:04,672][20347] Updated weights for policy 0, policy_version 747503 (0.0005) [2023-12-27 10:48:04,729][20347] Updated weights for policy 0, policy_version 747513 (0.0005) [2023-12-27 10:48:05,273][20347] Updated weights for policy 0, policy_version 747523 (0.0005) [2023-12-27 10:48:05,329][20347] Updated weights for policy 0, policy_version 747533 (0.0005) [2023-12-27 10:48:05,391][20347] Updated weights for policy 0, policy_version 747543 (0.0006) [2023-12-27 10:48:05,404][20381] Updated weights for policy 1, policy_version 747169 (0.0005) [2023-12-27 10:48:05,452][20381] Updated weights for policy 1, policy_version 747179 (0.0005) [2023-12-27 10:48:05,500][20381] Updated weights for policy 1, policy_version 747189 (0.0005) [2023-12-27 10:48:05,991][20347] Updated weights for policy 0, policy_version 747553 (0.0006) [2023-12-27 10:48:06,051][20347] Updated weights for policy 0, policy_version 747563 (0.0010) [2023-12-27 10:48:06,117][20347] Updated weights for policy 0, policy_version 747573 (0.0010) [2023-12-27 10:48:06,164][20381] Updated weights for policy 1, policy_version 747199 (0.0006) [2023-12-27 10:48:06,174][20347] Updated weights for policy 0, policy_version 747583 (0.0010) [2023-12-27 10:48:06,224][20381] Updated weights for policy 1, policy_version 747209 (0.0007) [2023-12-27 10:48:06,279][20381] Updated weights for policy 1, policy_version 747219 (0.0008) [2023-12-27 10:48:06,839][20347] Updated weights for policy 0, policy_version 747593 (0.0010) [2023-12-27 10:48:06,894][20347] Updated weights for policy 0, policy_version 747603 (0.0005) [2023-12-27 10:48:06,898][20381] Updated weights for policy 1, policy_version 747229 (0.0006) [2023-12-27 10:48:06,948][20347] Updated weights for policy 0, policy_version 747613 (0.0006) [2023-12-27 10:48:06,957][20381] Updated weights for policy 1, policy_version 747239 (0.0006) [2023-12-27 10:48:07,014][20381] Updated weights for policy 1, policy_version 747249 (0.0005) [2023-12-27 10:48:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20355.0). Total num frames: 382738432. Throughput: 0: 10330.7, 1: 10319.0. Samples: 382704920. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:07,363][19170] Avg episode reward: [(0, '884.510'), (1, '716.290')] [2023-12-27 10:48:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000747256_191324160.pth... [2023-12-27 10:48:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000747616_191414272.pth... [2023-12-27 10:48:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000746040_191012864.pth [2023-12-27 10:48:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000746400_191102976.pth [2023-12-27 10:48:07,514][20347] Updated weights for policy 0, policy_version 747623 (0.0009) [2023-12-27 10:48:07,570][20381] Updated weights for policy 1, policy_version 747259 (0.0006) [2023-12-27 10:48:07,570][20347] Updated weights for policy 0, policy_version 747633 (0.0010) [2023-12-27 10:48:07,623][20347] Updated weights for policy 0, policy_version 747643 (0.0010) [2023-12-27 10:48:07,625][20381] Updated weights for policy 1, policy_version 747269 (0.0006) [2023-12-27 10:48:07,681][20381] Updated weights for policy 1, policy_version 747279 (0.0006) [2023-12-27 10:48:08,263][20347] Updated weights for policy 0, policy_version 747653 (0.0008) [2023-12-27 10:48:08,319][20347] Updated weights for policy 0, policy_version 747663 (0.0006) [2023-12-27 10:48:08,340][20381] Updated weights for policy 1, policy_version 747289 (0.0007) [2023-12-27 10:48:08,383][20347] Updated weights for policy 0, policy_version 747673 (0.0011) [2023-12-27 10:48:08,406][20381] Updated weights for policy 1, policy_version 747299 (0.0011) [2023-12-27 10:48:08,455][20381] Updated weights for policy 1, policy_version 747309 (0.0010) [2023-12-27 10:48:08,507][20381] Updated weights for policy 1, policy_version 747319 (0.0010) [2023-12-27 10:48:09,054][20347] Updated weights for policy 0, policy_version 747683 (0.0010) [2023-12-27 10:48:09,101][20347] Updated weights for policy 0, policy_version 747693 (0.0007) [2023-12-27 10:48:09,149][20347] Updated weights for policy 0, policy_version 747703 (0.0007) [2023-12-27 10:48:09,202][20381] Updated weights for policy 1, policy_version 747329 (0.0010) [2023-12-27 10:48:09,266][20381] Updated weights for policy 1, policy_version 747339 (0.0011) [2023-12-27 10:48:09,324][20381] Updated weights for policy 1, policy_version 747349 (0.0011) [2023-12-27 10:48:09,919][20347] Updated weights for policy 0, policy_version 747713 (0.0008) [2023-12-27 10:48:09,984][20347] Updated weights for policy 0, policy_version 747723 (0.0008) [2023-12-27 10:48:10,050][20347] Updated weights for policy 0, policy_version 747733 (0.0008) [2023-12-27 10:48:10,074][20381] Updated weights for policy 1, policy_version 747359 (0.0009) [2023-12-27 10:48:10,103][20347] Updated weights for policy 0, policy_version 747743 (0.0007) [2023-12-27 10:48:10,126][20381] Updated weights for policy 1, policy_version 747369 (0.0008) [2023-12-27 10:48:10,187][20381] Updated weights for policy 1, policy_version 747379 (0.0007) [2023-12-27 10:48:10,822][20347] Updated weights for policy 0, policy_version 747753 (0.0009) [2023-12-27 10:48:10,843][20381] Updated weights for policy 1, policy_version 747389 (0.0008) [2023-12-27 10:48:10,874][20347] Updated weights for policy 0, policy_version 747763 (0.0007) [2023-12-27 10:48:10,896][20381] Updated weights for policy 1, policy_version 747399 (0.0010) [2023-12-27 10:48:10,930][20347] Updated weights for policy 0, policy_version 747773 (0.0010) [2023-12-27 10:48:10,956][20381] Updated weights for policy 1, policy_version 747409 (0.0006) [2023-12-27 10:48:11,659][20347] Updated weights for policy 0, policy_version 747783 (0.0008) [2023-12-27 10:48:11,662][20381] Updated weights for policy 1, policy_version 747419 (0.0007) [2023-12-27 10:48:11,722][20347] Updated weights for policy 0, policy_version 747793 (0.0009) [2023-12-27 10:48:11,729][20381] Updated weights for policy 1, policy_version 747429 (0.0009) [2023-12-27 10:48:11,788][20347] Updated weights for policy 0, policy_version 747803 (0.0007) [2023-12-27 10:48:11,794][20381] Updated weights for policy 1, policy_version 747439 (0.0008) [2023-12-27 10:48:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20299.5). Total num frames: 382836736. Throughput: 0: 10379.8, 1: 10387.0. Samples: 382830152. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:12,363][19170] Avg episode reward: [(0, '903.790'), (1, '804.410')] [2023-12-27 10:48:12,466][20347] Updated weights for policy 0, policy_version 747813 (0.0006) [2023-12-27 10:48:12,516][20347] Updated weights for policy 0, policy_version 747823 (0.0007) [2023-12-27 10:48:12,552][20381] Updated weights for policy 1, policy_version 747449 (0.0008) [2023-12-27 10:48:12,573][20347] Updated weights for policy 0, policy_version 747833 (0.0007) [2023-12-27 10:48:12,608][20381] Updated weights for policy 1, policy_version 747459 (0.0011) [2023-12-27 10:48:12,660][20381] Updated weights for policy 1, policy_version 747469 (0.0008) [2023-12-27 10:48:12,718][20381] Updated weights for policy 1, policy_version 747479 (0.0006) [2023-12-27 10:48:13,267][20347] Updated weights for policy 0, policy_version 747843 (0.0007) [2023-12-27 10:48:13,276][20381] Updated weights for policy 1, policy_version 747489 (0.0006) [2023-12-27 10:48:13,311][20347] Updated weights for policy 0, policy_version 747853 (0.0005) [2023-12-27 10:48:13,327][20381] Updated weights for policy 1, policy_version 747499 (0.0008) [2023-12-27 10:48:13,367][20347] Updated weights for policy 0, policy_version 747863 (0.0008) [2023-12-27 10:48:13,380][20381] Updated weights for policy 1, policy_version 747509 (0.0007) [2023-12-27 10:48:13,946][20381] Updated weights for policy 1, policy_version 747519 (0.0007) [2023-12-27 10:48:14,002][20381] Updated weights for policy 1, policy_version 747529 (0.0005) [2023-12-27 10:48:14,034][20347] Updated weights for policy 0, policy_version 747873 (0.0006) [2023-12-27 10:48:14,056][20381] Updated weights for policy 1, policy_version 747539 (0.0010) [2023-12-27 10:48:14,083][20347] Updated weights for policy 0, policy_version 747883 (0.0008) [2023-12-27 10:48:14,143][20347] Updated weights for policy 0, policy_version 747893 (0.0005) [2023-12-27 10:48:14,190][20347] Updated weights for policy 0, policy_version 747903 (0.0005) [2023-12-27 10:48:14,602][20381] Updated weights for policy 1, policy_version 747549 (0.0008) [2023-12-27 10:48:14,660][20381] Updated weights for policy 1, policy_version 747559 (0.0005) [2023-12-27 10:48:14,712][20381] Updated weights for policy 1, policy_version 747569 (0.0005) [2023-12-27 10:48:14,946][20347] Updated weights for policy 0, policy_version 747913 (0.0007) [2023-12-27 10:48:15,008][20347] Updated weights for policy 0, policy_version 747923 (0.0007) [2023-12-27 10:48:15,070][20347] Updated weights for policy 0, policy_version 747933 (0.0008) [2023-12-27 10:48:15,337][20381] Updated weights for policy 1, policy_version 747579 (0.0007) [2023-12-27 10:48:15,387][20381] Updated weights for policy 1, policy_version 747589 (0.0010) [2023-12-27 10:48:15,435][20381] Updated weights for policy 1, policy_version 747599 (0.0010) [2023-12-27 10:48:15,684][20347] Updated weights for policy 0, policy_version 747943 (0.0007) [2023-12-27 10:48:15,735][20347] Updated weights for policy 0, policy_version 747953 (0.0007) [2023-12-27 10:48:15,793][20347] Updated weights for policy 0, policy_version 747963 (0.0007) [2023-12-27 10:48:16,115][20381] Updated weights for policy 1, policy_version 747609 (0.0010) [2023-12-27 10:48:16,174][20381] Updated weights for policy 1, policy_version 747619 (0.0011) [2023-12-27 10:48:16,227][20381] Updated weights for policy 1, policy_version 747629 (0.0011) [2023-12-27 10:48:16,288][20381] Updated weights for policy 1, policy_version 747639 (0.0010) [2023-12-27 10:48:16,504][20347] Updated weights for policy 0, policy_version 747973 (0.0006) [2023-12-27 10:48:16,557][20347] Updated weights for policy 0, policy_version 747983 (0.0005) [2023-12-27 10:48:16,606][20347] Updated weights for policy 0, policy_version 747993 (0.0005) [2023-12-27 10:48:16,868][20381] Updated weights for policy 1, policy_version 747649 (0.0006) [2023-12-27 10:48:16,933][20381] Updated weights for policy 1, policy_version 747659 (0.0006) [2023-12-27 10:48:16,995][20381] Updated weights for policy 1, policy_version 747669 (0.0007) [2023-12-27 10:48:17,229][20347] Updated weights for policy 0, policy_version 748003 (0.0007) [2023-12-27 10:48:17,280][20347] Updated weights for policy 0, policy_version 748013 (0.0010) [2023-12-27 10:48:17,332][20347] Updated weights for policy 0, policy_version 748023 (0.0010) [2023-12-27 10:48:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20327.3). Total num frames: 382943232. Throughput: 0: 10325.6, 1: 10482.1. Samples: 382957920. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:17,363][19170] Avg episode reward: [(0, '951.240'), (1, '706.550')] [2023-12-27 10:48:17,597][20381] Updated weights for policy 1, policy_version 747679 (0.0008) [2023-12-27 10:48:17,644][20381] Updated weights for policy 1, policy_version 747689 (0.0007) [2023-12-27 10:48:17,699][20381] Updated weights for policy 1, policy_version 747699 (0.0010) [2023-12-27 10:48:17,983][20347] Updated weights for policy 0, policy_version 748033 (0.0010) [2023-12-27 10:48:18,037][20347] Updated weights for policy 0, policy_version 748043 (0.0007) [2023-12-27 10:48:18,098][20347] Updated weights for policy 0, policy_version 748053 (0.0007) [2023-12-27 10:48:18,158][20347] Updated weights for policy 0, policy_version 748063 (0.0008) [2023-12-27 10:48:18,343][20381] Updated weights for policy 1, policy_version 747709 (0.0009) [2023-12-27 10:48:18,403][20381] Updated weights for policy 1, policy_version 747719 (0.0010) [2023-12-27 10:48:18,462][20381] Updated weights for policy 1, policy_version 747729 (0.0011) [2023-12-27 10:48:18,892][20347] Updated weights for policy 0, policy_version 748073 (0.0007) [2023-12-27 10:48:18,948][20347] Updated weights for policy 0, policy_version 748083 (0.0008) [2023-12-27 10:48:19,003][20347] Updated weights for policy 0, policy_version 748093 (0.0007) [2023-12-27 10:48:19,096][20381] Updated weights for policy 1, policy_version 747739 (0.0009) [2023-12-27 10:48:19,143][20381] Updated weights for policy 1, policy_version 747749 (0.0005) [2023-12-27 10:48:19,193][20381] Updated weights for policy 1, policy_version 747759 (0.0005) [2023-12-27 10:48:19,752][20347] Updated weights for policy 0, policy_version 748103 (0.0008) [2023-12-27 10:48:19,808][20347] Updated weights for policy 0, policy_version 748113 (0.0007) [2023-12-27 10:48:19,878][20347] Updated weights for policy 0, policy_version 748123 (0.0007) [2023-12-27 10:48:19,879][20381] Updated weights for policy 1, policy_version 747769 (0.0010) [2023-12-27 10:48:19,947][20381] Updated weights for policy 1, policy_version 747779 (0.0008) [2023-12-27 10:48:20,000][20381] Updated weights for policy 1, policy_version 747789 (0.0008) [2023-12-27 10:48:20,050][20381] Updated weights for policy 1, policy_version 747799 (0.0008) [2023-12-27 10:48:20,565][20347] Updated weights for policy 0, policy_version 748133 (0.0006) [2023-12-27 10:48:20,632][20347] Updated weights for policy 0, policy_version 748143 (0.0008) [2023-12-27 10:48:20,696][20347] Updated weights for policy 0, policy_version 748153 (0.0008) [2023-12-27 10:48:20,830][20381] Updated weights for policy 1, policy_version 747809 (0.0008) [2023-12-27 10:48:20,890][20381] Updated weights for policy 1, policy_version 747819 (0.0008) [2023-12-27 10:48:20,956][20381] Updated weights for policy 1, policy_version 747829 (0.0008) [2023-12-27 10:48:21,400][20347] Updated weights for policy 0, policy_version 748163 (0.0008) [2023-12-27 10:48:21,459][20347] Updated weights for policy 0, policy_version 748173 (0.0008) [2023-12-27 10:48:21,513][20347] Updated weights for policy 0, policy_version 748183 (0.0008) [2023-12-27 10:48:21,601][20381] Updated weights for policy 1, policy_version 747839 (0.0007) [2023-12-27 10:48:21,663][20381] Updated weights for policy 1, policy_version 747849 (0.0008) [2023-12-27 10:48:21,717][20381] Updated weights for policy 1, policy_version 747859 (0.0008) [2023-12-27 10:48:22,271][20347] Updated weights for policy 0, policy_version 748193 (0.0008) [2023-12-27 10:48:22,332][20347] Updated weights for policy 0, policy_version 748203 (0.0008) [2023-12-27 10:48:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20327.3). Total num frames: 383041536. Throughput: 0: 10279.3, 1: 10465.8. Samples: 383018264. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:22,363][19170] Avg episode reward: [(0, '961.230'), (1, '736.030')] [2023-12-27 10:48:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000747864_191479808.pth... [2023-12-27 10:48:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000746648_191168512.pth [2023-12-27 10:48:22,398][20347] Updated weights for policy 0, policy_version 748213 (0.0008) [2023-12-27 10:48:22,441][20381] Updated weights for policy 1, policy_version 747869 (0.0007) [2023-12-27 10:48:22,461][20347] Updated weights for policy 0, policy_version 748223 (0.0008) [2023-12-27 10:48:22,463][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000748224_191569920.pth... [2023-12-27 10:48:22,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000747008_191258624.pth [2023-12-27 10:48:22,504][20381] Updated weights for policy 1, policy_version 747879 (0.0006) [2023-12-27 10:48:22,563][20381] Updated weights for policy 1, policy_version 747889 (0.0006) [2023-12-27 10:48:23,202][20347] Updated weights for policy 0, policy_version 748233 (0.0007) [2023-12-27 10:48:23,216][20381] Updated weights for policy 1, policy_version 747899 (0.0008) [2023-12-27 10:48:23,251][20347] Updated weights for policy 0, policy_version 748243 (0.0005) [2023-12-27 10:48:23,268][20381] Updated weights for policy 1, policy_version 747909 (0.0010) [2023-12-27 10:48:23,311][20347] Updated weights for policy 0, policy_version 748253 (0.0006) [2023-12-27 10:48:23,324][20381] Updated weights for policy 1, policy_version 747919 (0.0010) [2023-12-27 10:48:24,013][20381] Updated weights for policy 1, policy_version 747929 (0.0010) [2023-12-27 10:48:24,032][20347] Updated weights for policy 0, policy_version 748263 (0.0006) [2023-12-27 10:48:24,066][20381] Updated weights for policy 1, policy_version 747939 (0.0008) [2023-12-27 10:48:24,088][20347] Updated weights for policy 0, policy_version 748273 (0.0005) [2023-12-27 10:48:24,118][20381] Updated weights for policy 1, policy_version 747949 (0.0010) [2023-12-27 10:48:24,147][20347] Updated weights for policy 0, policy_version 748283 (0.0007) [2023-12-27 10:48:24,171][20381] Updated weights for policy 1, policy_version 747959 (0.0011) [2023-12-27 10:48:24,787][20347] Updated weights for policy 0, policy_version 748293 (0.0007) [2023-12-27 10:48:24,839][20347] Updated weights for policy 0, policy_version 748303 (0.0008) [2023-12-27 10:48:24,880][20381] Updated weights for policy 1, policy_version 747969 (0.0007) [2023-12-27 10:48:24,898][20347] Updated weights for policy 0, policy_version 748313 (0.0006) [2023-12-27 10:48:24,940][20381] Updated weights for policy 1, policy_version 747979 (0.0007) [2023-12-27 10:48:24,999][20381] Updated weights for policy 1, policy_version 747989 (0.0008) [2023-12-27 10:48:25,585][20347] Updated weights for policy 0, policy_version 748323 (0.0006) [2023-12-27 10:48:25,631][20347] Updated weights for policy 0, policy_version 748333 (0.0006) [2023-12-27 10:48:25,632][20381] Updated weights for policy 1, policy_version 747999 (0.0007) [2023-12-27 10:48:25,689][20381] Updated weights for policy 1, policy_version 748009 (0.0006) [2023-12-27 10:48:25,690][20347] Updated weights for policy 0, policy_version 748343 (0.0006) [2023-12-27 10:48:25,743][20381] Updated weights for policy 1, policy_version 748019 (0.0006) [2023-12-27 10:48:26,332][20381] Updated weights for policy 1, policy_version 748029 (0.0007) [2023-12-27 10:48:26,378][20381] Updated weights for policy 1, policy_version 748039 (0.0008) [2023-12-27 10:48:26,429][20381] Updated weights for policy 1, policy_version 748049 (0.0008) [2023-12-27 10:48:26,430][20347] Updated weights for policy 0, policy_version 748353 (0.0008) [2023-12-27 10:48:26,477][20347] Updated weights for policy 0, policy_version 748363 (0.0007) [2023-12-27 10:48:26,525][20347] Updated weights for policy 0, policy_version 748373 (0.0008) [2023-12-27 10:48:26,576][20347] Updated weights for policy 0, policy_version 748383 (0.0008) [2023-12-27 10:48:27,123][20381] Updated weights for policy 1, policy_version 748059 (0.0008) [2023-12-27 10:48:27,176][20381] Updated weights for policy 1, policy_version 748069 (0.0007) [2023-12-27 10:48:27,227][20381] Updated weights for policy 1, policy_version 748079 (0.0008) [2023-12-27 10:48:27,266][20347] Updated weights for policy 0, policy_version 748393 (0.0006) [2023-12-27 10:48:27,325][20291] KL-divergence is very high: 110.4535 [2023-12-27 10:48:27,341][20347] Updated weights for policy 0, policy_version 748403 (0.0007) [2023-12-27 10:48:27,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20355.0). Total num frames: 383148032. Throughput: 0: 10241.2, 1: 10527.5. Samples: 383138492. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:27,363][19170] Avg episode reward: [(0, '846.450'), (1, '833.900')] [2023-12-27 10:48:27,389][20347] Updated weights for policy 0, policy_version 748413 (0.0008) [2023-12-27 10:48:27,880][20381] Updated weights for policy 1, policy_version 748089 (0.0006) [2023-12-27 10:48:27,938][20381] Updated weights for policy 1, policy_version 748099 (0.0005) [2023-12-27 10:48:27,980][20381] Updated weights for policy 1, policy_version 748109 (0.0005) [2023-12-27 10:48:28,027][20381] Updated weights for policy 1, policy_version 748119 (0.0005) [2023-12-27 10:48:28,089][20347] Updated weights for policy 0, policy_version 748423 (0.0009) [2023-12-27 10:48:28,143][20347] Updated weights for policy 0, policy_version 748433 (0.0008) [2023-12-27 10:48:28,204][20347] Updated weights for policy 0, policy_version 748443 (0.0005) [2023-12-27 10:48:28,659][20381] Updated weights for policy 1, policy_version 748129 (0.0007) [2023-12-27 10:48:28,716][20381] Updated weights for policy 1, policy_version 748139 (0.0007) [2023-12-27 10:48:28,778][20381] Updated weights for policy 1, policy_version 748149 (0.0008) [2023-12-27 10:48:28,852][20347] Updated weights for policy 0, policy_version 748453 (0.0006) [2023-12-27 10:48:28,906][20347] Updated weights for policy 0, policy_version 748463 (0.0008) [2023-12-27 10:48:28,962][20347] Updated weights for policy 0, policy_version 748473 (0.0007) [2023-12-27 10:48:29,514][20381] Updated weights for policy 1, policy_version 748159 (0.0008) [2023-12-27 10:48:29,570][20381] Updated weights for policy 1, policy_version 748169 (0.0005) [2023-12-27 10:48:29,625][20347] Updated weights for policy 0, policy_version 748483 (0.0008) [2023-12-27 10:48:29,632][20381] Updated weights for policy 1, policy_version 748179 (0.0008) [2023-12-27 10:48:29,684][20347] Updated weights for policy 0, policy_version 748493 (0.0009) [2023-12-27 10:48:29,740][20347] Updated weights for policy 0, policy_version 748503 (0.0008) [2023-12-27 10:48:30,324][20381] Updated weights for policy 1, policy_version 748189 (0.0009) [2023-12-27 10:48:30,377][20381] Updated weights for policy 1, policy_version 748199 (0.0008) [2023-12-27 10:48:30,429][20381] Updated weights for policy 1, policy_version 748209 (0.0007) [2023-12-27 10:48:30,435][20347] Updated weights for policy 0, policy_version 748513 (0.0007) [2023-12-27 10:48:30,486][20347] Updated weights for policy 0, policy_version 748523 (0.0005) [2023-12-27 10:48:30,537][20347] Updated weights for policy 0, policy_version 748533 (0.0007) [2023-12-27 10:48:30,584][20347] Updated weights for policy 0, policy_version 748543 (0.0007) [2023-12-27 10:48:31,167][20347] Updated weights for policy 0, policy_version 748553 (0.0007) [2023-12-27 10:48:31,214][20381] Updated weights for policy 1, policy_version 748219 (0.0007) [2023-12-27 10:48:31,219][20347] Updated weights for policy 0, policy_version 748563 (0.0006) [2023-12-27 10:48:31,279][20347] Updated weights for policy 0, policy_version 748573 (0.0007) [2023-12-27 10:48:31,280][20381] Updated weights for policy 1, policy_version 748229 (0.0007) [2023-12-27 10:48:31,339][20381] Updated weights for policy 1, policy_version 748239 (0.0008) [2023-12-27 10:48:31,931][20347] Updated weights for policy 0, policy_version 748583 (0.0007) [2023-12-27 10:48:31,986][20347] Updated weights for policy 0, policy_version 748593 (0.0008) [2023-12-27 10:48:32,003][20381] Updated weights for policy 1, policy_version 748249 (0.0008) [2023-12-27 10:48:32,042][20347] Updated weights for policy 0, policy_version 748603 (0.0006) [2023-12-27 10:48:32,057][20381] Updated weights for policy 1, policy_version 748259 (0.0005) [2023-12-27 10:48:32,105][20381] Updated weights for policy 1, policy_version 748269 (0.0005) [2023-12-27 10:48:32,159][20381] Updated weights for policy 1, policy_version 748279 (0.0005) [2023-12-27 10:48:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20355.0). Total num frames: 383254528. Throughput: 0: 10249.9, 1: 10468.9. Samples: 383262528. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:32,363][19170] Avg episode reward: [(0, '894.650'), (1, '872.960')] [2023-12-27 10:48:32,739][20347] Updated weights for policy 0, policy_version 748613 (0.0007) [2023-12-27 10:48:32,789][20347] Updated weights for policy 0, policy_version 748623 (0.0007) [2023-12-27 10:48:32,824][20381] Updated weights for policy 1, policy_version 748289 (0.0006) [2023-12-27 10:48:32,848][20347] Updated weights for policy 0, policy_version 748633 (0.0007) [2023-12-27 10:48:32,877][20381] Updated weights for policy 1, policy_version 748299 (0.0006) [2023-12-27 10:48:32,936][20381] Updated weights for policy 1, policy_version 748309 (0.0007) [2023-12-27 10:48:33,496][20347] Updated weights for policy 0, policy_version 748643 (0.0008) [2023-12-27 10:48:33,551][20347] Updated weights for policy 0, policy_version 748653 (0.0006) [2023-12-27 10:48:33,602][20381] Updated weights for policy 1, policy_version 748319 (0.0006) [2023-12-27 10:48:33,608][20347] Updated weights for policy 0, policy_version 748663 (0.0006) [2023-12-27 10:48:33,649][20381] Updated weights for policy 1, policy_version 748329 (0.0005) [2023-12-27 10:48:33,697][20381] Updated weights for policy 1, policy_version 748339 (0.0005) [2023-12-27 10:48:34,172][20347] Updated weights for policy 0, policy_version 748673 (0.0006) [2023-12-27 10:48:34,231][20347] Updated weights for policy 0, policy_version 748683 (0.0009) [2023-12-27 10:48:34,269][20381] Updated weights for policy 1, policy_version 748349 (0.0005) [2023-12-27 10:48:34,293][20347] Updated weights for policy 0, policy_version 748693 (0.0010) [2023-12-27 10:48:34,325][20381] Updated weights for policy 1, policy_version 748359 (0.0008) [2023-12-27 10:48:34,358][20347] Updated weights for policy 0, policy_version 748703 (0.0007) [2023-12-27 10:48:34,383][20381] Updated weights for policy 1, policy_version 748369 (0.0007) [2023-12-27 10:48:35,038][20381] Updated weights for policy 1, policy_version 748379 (0.0007) [2023-12-27 10:48:35,051][20347] Updated weights for policy 0, policy_version 748713 (0.0006) [2023-12-27 10:48:35,091][20381] Updated weights for policy 1, policy_version 748389 (0.0005) [2023-12-27 10:48:35,104][20347] Updated weights for policy 0, policy_version 748723 (0.0006) [2023-12-27 10:48:35,132][20291] KL-divergence is very high: 102.0642 [2023-12-27 10:48:35,142][20381] Updated weights for policy 1, policy_version 748399 (0.0005) [2023-12-27 10:48:35,160][20347] Updated weights for policy 0, policy_version 748733 (0.0008) [2023-12-27 10:48:35,771][20381] Updated weights for policy 1, policy_version 748409 (0.0005) [2023-12-27 10:48:35,813][20347] Updated weights for policy 0, policy_version 748743 (0.0009) [2023-12-27 10:48:35,821][20381] Updated weights for policy 1, policy_version 748419 (0.0007) [2023-12-27 10:48:35,866][20347] Updated weights for policy 0, policy_version 748753 (0.0007) [2023-12-27 10:48:35,873][20381] Updated weights for policy 1, policy_version 748429 (0.0007) [2023-12-27 10:48:35,921][20347] Updated weights for policy 0, policy_version 748763 (0.0005) [2023-12-27 10:48:35,932][20381] Updated weights for policy 1, policy_version 748439 (0.0008) [2023-12-27 10:48:36,547][20347] Updated weights for policy 0, policy_version 748773 (0.0010) [2023-12-27 10:48:36,603][20347] Updated weights for policy 0, policy_version 748783 (0.0010) [2023-12-27 10:48:36,662][20347] Updated weights for policy 0, policy_version 748793 (0.0006) [2023-12-27 10:48:36,710][20381] Updated weights for policy 1, policy_version 748449 (0.0007) [2023-12-27 10:48:36,768][20381] Updated weights for policy 1, policy_version 748459 (0.0008) [2023-12-27 10:48:36,831][20381] Updated weights for policy 1, policy_version 748469 (0.0008) [2023-12-27 10:48:37,310][20347] Updated weights for policy 0, policy_version 748803 (0.0008) [2023-12-27 10:48:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20355.0). Total num frames: 383352832. Throughput: 0: 10288.4, 1: 10451.5. Samples: 383326216. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:37,363][19170] Avg episode reward: [(0, '941.840'), (1, '833.840')] [2023-12-27 10:48:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000748472_191635456.pth... [2023-12-27 10:48:37,373][20347] Updated weights for policy 0, policy_version 748813 (0.0008) [2023-12-27 10:48:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000747256_191324160.pth [2023-12-27 10:48:37,436][20347] Updated weights for policy 0, policy_version 748823 (0.0007) [2023-12-27 10:48:37,495][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000748832_191725568.pth... [2023-12-27 10:48:37,499][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000747616_191414272.pth [2023-12-27 10:48:37,535][20381] Updated weights for policy 1, policy_version 748479 (0.0010) [2023-12-27 10:48:37,587][20381] Updated weights for policy 1, policy_version 748489 (0.0006) [2023-12-27 10:48:37,639][20381] Updated weights for policy 1, policy_version 748499 (0.0005) [2023-12-27 10:48:38,025][20347] Updated weights for policy 0, policy_version 748833 (0.0008) [2023-12-27 10:48:38,084][20347] Updated weights for policy 0, policy_version 748843 (0.0005) [2023-12-27 10:48:38,136][20347] Updated weights for policy 0, policy_version 748853 (0.0008) [2023-12-27 10:48:38,195][20347] Updated weights for policy 0, policy_version 748863 (0.0010) [2023-12-27 10:48:38,230][20381] Updated weights for policy 1, policy_version 748509 (0.0006) [2023-12-27 10:48:38,292][20381] Updated weights for policy 1, policy_version 748519 (0.0011) [2023-12-27 10:48:38,358][20381] Updated weights for policy 1, policy_version 748529 (0.0011) [2023-12-27 10:48:38,886][20347] Updated weights for policy 0, policy_version 748873 (0.0011) [2023-12-27 10:48:38,950][20347] Updated weights for policy 0, policy_version 748883 (0.0011) [2023-12-27 10:48:39,011][20347] Updated weights for policy 0, policy_version 748893 (0.0010) [2023-12-27 10:48:39,062][20381] Updated weights for policy 1, policy_version 748539 (0.0009) [2023-12-27 10:48:39,122][20381] Updated weights for policy 1, policy_version 748549 (0.0005) [2023-12-27 10:48:39,180][20381] Updated weights for policy 1, policy_version 748559 (0.0007) [2023-12-27 10:48:39,746][20347] Updated weights for policy 0, policy_version 748903 (0.0010) [2023-12-27 10:48:39,796][20347] Updated weights for policy 0, policy_version 748913 (0.0010) [2023-12-27 10:48:39,857][20347] Updated weights for policy 0, policy_version 748923 (0.0011) [2023-12-27 10:48:39,884][20381] Updated weights for policy 1, policy_version 748569 (0.0007) [2023-12-27 10:48:39,948][20381] Updated weights for policy 1, policy_version 748579 (0.0008) [2023-12-27 10:48:40,004][20381] Updated weights for policy 1, policy_version 748589 (0.0007) [2023-12-27 10:48:40,053][20381] Updated weights for policy 1, policy_version 748599 (0.0007) [2023-12-27 10:48:40,542][20347] Updated weights for policy 0, policy_version 748933 (0.0010) [2023-12-27 10:48:40,591][20291] KL-divergence is very high: 560.3023 [2023-12-27 10:48:40,591][20347] Updated weights for policy 0, policy_version 748943 (0.0010) [2023-12-27 10:48:40,629][20291] KL-divergence is very high: 345.4013 [2023-12-27 10:48:40,640][20347] Updated weights for policy 0, policy_version 748953 (0.0010) [2023-12-27 10:48:40,670][20291] KL-divergence is very high: 152.3230 [2023-12-27 10:48:40,798][20381] Updated weights for policy 1, policy_version 748609 (0.0006) [2023-12-27 10:48:40,854][20381] Updated weights for policy 1, policy_version 748619 (0.0006) [2023-12-27 10:48:40,912][20381] Updated weights for policy 1, policy_version 748629 (0.0007) [2023-12-27 10:48:41,392][20347] Updated weights for policy 0, policy_version 748963 (0.0009) [2023-12-27 10:48:41,458][20347] Updated weights for policy 0, policy_version 748973 (0.0008) [2023-12-27 10:48:41,521][20347] Updated weights for policy 0, policy_version 748983 (0.0008) [2023-12-27 10:48:41,665][20381] Updated weights for policy 1, policy_version 748639 (0.0009) [2023-12-27 10:48:41,729][20381] Updated weights for policy 1, policy_version 748649 (0.0009) [2023-12-27 10:48:41,797][20381] Updated weights for policy 1, policy_version 748659 (0.0006) [2023-12-27 10:48:42,247][20347] Updated weights for policy 0, policy_version 748993 (0.0007) [2023-12-27 10:48:42,313][20347] Updated weights for policy 0, policy_version 749003 (0.0008) [2023-12-27 10:48:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20327.3). Total num frames: 383451136. Throughput: 0: 10263.2, 1: 10411.1. Samples: 383448372. Policy #0 lag: (min: 31.0, avg: 40.4, max: 63.0) [2023-12-27 10:48:42,363][19170] Avg episode reward: [(0, '912.570'), (1, '795.050')] [2023-12-27 10:48:42,375][20347] Updated weights for policy 0, policy_version 749013 (0.0008) [2023-12-27 10:48:42,429][20347] Updated weights for policy 0, policy_version 749023 (0.0006) [2023-12-27 10:48:42,489][20381] Updated weights for policy 1, policy_version 748669 (0.0007) [2023-12-27 10:48:42,515][20292] KL-divergence is very high: 199.0732 [2023-12-27 10:48:42,553][20381] Updated weights for policy 1, policy_version 748679 (0.0008) [2023-12-27 10:48:42,614][20381] Updated weights for policy 1, policy_version 748689 (0.0008) [2023-12-27 10:48:43,120][20347] Updated weights for policy 0, policy_version 749033 (0.0007) [2023-12-27 10:48:43,171][20347] Updated weights for policy 0, policy_version 749043 (0.0005) [2023-12-27 10:48:43,217][20347] Updated weights for policy 0, policy_version 749053 (0.0005) [2023-12-27 10:48:43,305][20381] Updated weights for policy 1, policy_version 748699 (0.0008) [2023-12-27 10:48:43,360][20381] Updated weights for policy 1, policy_version 748709 (0.0005) [2023-12-27 10:48:43,419][20381] Updated weights for policy 1, policy_version 748719 (0.0005) [2023-12-27 10:48:43,946][20347] Updated weights for policy 0, policy_version 749063 (0.0010) [2023-12-27 10:48:43,963][20381] Updated weights for policy 1, policy_version 748729 (0.0006) [2023-12-27 10:48:43,997][20347] Updated weights for policy 0, policy_version 749073 (0.0010) [2023-12-27 10:48:44,026][20381] Updated weights for policy 1, policy_version 748739 (0.0009) [2023-12-27 10:48:44,045][20347] Updated weights for policy 0, policy_version 749083 (0.0007) [2023-12-27 10:48:44,079][20381] Updated weights for policy 1, policy_version 748749 (0.0011) [2023-12-27 10:48:44,125][20292] KL-divergence is very high: 483.2726 [2023-12-27 10:48:44,126][20381] Updated weights for policy 1, policy_version 748759 (0.0007) [2023-12-27 10:48:44,679][20347] Updated weights for policy 0, policy_version 749093 (0.0006) [2023-12-27 10:48:44,735][20347] Updated weights for policy 0, policy_version 749103 (0.0007) [2023-12-27 10:48:44,748][20381] Updated weights for policy 1, policy_version 748769 (0.0010) [2023-12-27 10:48:44,794][20347] Updated weights for policy 0, policy_version 749113 (0.0008) [2023-12-27 10:48:44,812][20381] Updated weights for policy 1, policy_version 748779 (0.0010) [2023-12-27 10:48:44,860][20292] KL-divergence is very high: 115.4039 [2023-12-27 10:48:44,866][20292] KL-divergence is very high: 156.4701 [2023-12-27 10:48:44,873][20381] Updated weights for policy 1, policy_version 748789 (0.0011) [2023-12-27 10:48:44,886][20292] KL-divergence is very high: 167.0679 [2023-12-27 10:48:45,425][20347] Updated weights for policy 0, policy_version 749123 (0.0006) [2023-12-27 10:48:45,484][20347] Updated weights for policy 0, policy_version 749133 (0.0006) [2023-12-27 10:48:45,529][20347] Updated weights for policy 0, policy_version 749143 (0.0005) [2023-12-27 10:48:45,584][20381] Updated weights for policy 1, policy_version 748799 (0.0011) [2023-12-27 10:48:45,639][20381] Updated weights for policy 1, policy_version 748809 (0.0010) [2023-12-27 10:48:45,695][20381] Updated weights for policy 1, policy_version 748819 (0.0011) [2023-12-27 10:48:46,080][20347] Updated weights for policy 0, policy_version 749153 (0.0008) [2023-12-27 10:48:46,128][20347] Updated weights for policy 0, policy_version 749163 (0.0005) [2023-12-27 10:48:46,176][20347] Updated weights for policy 0, policy_version 749173 (0.0006) [2023-12-27 10:48:46,231][20347] Updated weights for policy 0, policy_version 749183 (0.0010) [2023-12-27 10:48:46,397][20381] Updated weights for policy 1, policy_version 748829 (0.0011) [2023-12-27 10:48:46,451][20381] Updated weights for policy 1, policy_version 748839 (0.0010) [2023-12-27 10:48:46,506][20381] Updated weights for policy 1, policy_version 748849 (0.0006) [2023-12-27 10:48:46,891][20347] Updated weights for policy 0, policy_version 749193 (0.0006) [2023-12-27 10:48:46,947][20347] Updated weights for policy 0, policy_version 749203 (0.0007) [2023-12-27 10:48:47,001][20347] Updated weights for policy 0, policy_version 749213 (0.0009) [2023-12-27 10:48:47,128][20381] Updated weights for policy 1, policy_version 748859 (0.0007) [2023-12-27 10:48:47,190][20381] Updated weights for policy 1, policy_version 748869 (0.0008) [2023-12-27 10:48:47,251][20381] Updated weights for policy 1, policy_version 748879 (0.0007) [2023-12-27 10:48:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20410.6). Total num frames: 383565824. Throughput: 0: 10304.9, 1: 10401.2. Samples: 383572644. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:48:47,363][19170] Avg episode reward: [(0, '922.160'), (1, '701.750')] [2023-12-27 10:48:47,664][20347] Updated weights for policy 0, policy_version 749223 (0.0007) [2023-12-27 10:48:47,707][20347] Updated weights for policy 0, policy_version 749233 (0.0007) [2023-12-27 10:48:47,765][20347] Updated weights for policy 0, policy_version 749243 (0.0007) [2023-12-27 10:48:47,824][20381] Updated weights for policy 1, policy_version 748889 (0.0006) [2023-12-27 10:48:47,871][20381] Updated weights for policy 1, policy_version 748899 (0.0009) [2023-12-27 10:48:47,924][20381] Updated weights for policy 1, policy_version 748909 (0.0009) [2023-12-27 10:48:47,970][20381] Updated weights for policy 1, policy_version 748919 (0.0009) [2023-12-27 10:48:48,457][20347] Updated weights for policy 0, policy_version 749253 (0.0007) [2023-12-27 10:48:48,506][20347] Updated weights for policy 0, policy_version 749263 (0.0007) [2023-12-27 10:48:48,560][20347] Updated weights for policy 0, policy_version 749273 (0.0007) [2023-12-27 10:48:48,637][20381] Updated weights for policy 1, policy_version 748929 (0.0011) [2023-12-27 10:48:48,694][20381] Updated weights for policy 1, policy_version 748939 (0.0011) [2023-12-27 10:48:48,751][20381] Updated weights for policy 1, policy_version 748949 (0.0010) [2023-12-27 10:48:49,304][20347] Updated weights for policy 0, policy_version 749283 (0.0007) [2023-12-27 10:48:49,361][20347] Updated weights for policy 0, policy_version 749293 (0.0008) [2023-12-27 10:48:49,419][20347] Updated weights for policy 0, policy_version 749303 (0.0007) [2023-12-27 10:48:49,436][20381] Updated weights for policy 1, policy_version 748959 (0.0006) [2023-12-27 10:48:49,495][20381] Updated weights for policy 1, policy_version 748969 (0.0008) [2023-12-27 10:48:49,551][20381] Updated weights for policy 1, policy_version 748979 (0.0008) [2023-12-27 10:48:50,105][20347] Updated weights for policy 0, policy_version 749313 (0.0008) [2023-12-27 10:48:50,168][20347] Updated weights for policy 0, policy_version 749323 (0.0008) [2023-12-27 10:48:50,230][20347] Updated weights for policy 0, policy_version 749333 (0.0008) [2023-12-27 10:48:50,292][20347] Updated weights for policy 0, policy_version 749343 (0.0007) [2023-12-27 10:48:50,305][20381] Updated weights for policy 1, policy_version 748989 (0.0007) [2023-12-27 10:48:50,361][20381] Updated weights for policy 1, policy_version 748999 (0.0008) [2023-12-27 10:48:50,419][20381] Updated weights for policy 1, policy_version 749009 (0.0009) [2023-12-27 10:48:50,910][20347] Updated weights for policy 0, policy_version 749353 (0.0006) [2023-12-27 10:48:50,973][20347] Updated weights for policy 0, policy_version 749363 (0.0006) [2023-12-27 10:48:51,036][20347] Updated weights for policy 0, policy_version 749373 (0.0006) [2023-12-27 10:48:51,155][20381] Updated weights for policy 1, policy_version 749019 (0.0007) [2023-12-27 10:48:51,209][20381] Updated weights for policy 1, policy_version 749029 (0.0006) [2023-12-27 10:48:51,273][20381] Updated weights for policy 1, policy_version 749039 (0.0006) [2023-12-27 10:48:51,690][20347] Updated weights for policy 0, policy_version 749383 (0.0009) [2023-12-27 10:48:51,752][20347] Updated weights for policy 0, policy_version 749393 (0.0008) [2023-12-27 10:48:51,814][20347] Updated weights for policy 0, policy_version 749403 (0.0005) [2023-12-27 10:48:51,951][20381] Updated weights for policy 1, policy_version 749049 (0.0007) [2023-12-27 10:48:52,011][20381] Updated weights for policy 1, policy_version 749059 (0.0007) [2023-12-27 10:48:52,076][20381] Updated weights for policy 1, policy_version 749069 (0.0007) [2023-12-27 10:48:52,128][20381] Updated weights for policy 1, policy_version 749079 (0.0007) [2023-12-27 10:48:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20355.0). Total num frames: 383664128. Throughput: 0: 10250.0, 1: 10399.8. Samples: 383634164. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:48:52,363][19170] Avg episode reward: [(0, '931.720'), (1, '728.820')] [2023-12-27 10:48:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000749080_191791104.pth... [2023-12-27 10:48:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000749408_191873024.pth... [2023-12-27 10:48:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000747864_191479808.pth [2023-12-27 10:48:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000748224_191569920.pth [2023-12-27 10:48:52,499][20347] Updated weights for policy 0, policy_version 749413 (0.0007) [2023-12-27 10:48:52,558][20347] Updated weights for policy 0, policy_version 749423 (0.0008) [2023-12-27 10:48:52,618][20347] Updated weights for policy 0, policy_version 749433 (0.0008) [2023-12-27 10:48:52,788][20381] Updated weights for policy 1, policy_version 749089 (0.0007) [2023-12-27 10:48:52,849][20381] Updated weights for policy 1, policy_version 749099 (0.0008) [2023-12-27 10:48:52,908][20381] Updated weights for policy 1, policy_version 749109 (0.0008) [2023-12-27 10:48:53,310][20347] Updated weights for policy 0, policy_version 749443 (0.0007) [2023-12-27 10:48:53,364][20347] Updated weights for policy 0, policy_version 749453 (0.0007) [2023-12-27 10:48:53,420][20347] Updated weights for policy 0, policy_version 749463 (0.0007) [2023-12-27 10:48:53,630][20381] Updated weights for policy 1, policy_version 749119 (0.0007) [2023-12-27 10:48:53,691][20381] Updated weights for policy 1, policy_version 749129 (0.0008) [2023-12-27 10:48:53,742][20381] Updated weights for policy 1, policy_version 749139 (0.0007) [2023-12-27 10:48:54,068][20347] Updated weights for policy 0, policy_version 749473 (0.0006) [2023-12-27 10:48:54,128][20347] Updated weights for policy 0, policy_version 749483 (0.0010) [2023-12-27 10:48:54,184][20347] Updated weights for policy 0, policy_version 749493 (0.0011) [2023-12-27 10:48:54,243][20347] Updated weights for policy 0, policy_version 749503 (0.0010) [2023-12-27 10:48:54,396][20381] Updated weights for policy 1, policy_version 749149 (0.0007) [2023-12-27 10:48:54,449][20381] Updated weights for policy 1, policy_version 749159 (0.0005) [2023-12-27 10:48:54,511][20381] Updated weights for policy 1, policy_version 749169 (0.0008) [2023-12-27 10:48:54,942][20347] Updated weights for policy 0, policy_version 749513 (0.0007) [2023-12-27 10:48:55,012][20347] Updated weights for policy 0, policy_version 749523 (0.0010) [2023-12-27 10:48:55,077][20347] Updated weights for policy 0, policy_version 749533 (0.0007) [2023-12-27 10:48:55,174][20381] Updated weights for policy 1, policy_version 749179 (0.0008) [2023-12-27 10:48:55,227][20381] Updated weights for policy 1, policy_version 749189 (0.0008) [2023-12-27 10:48:55,278][20381] Updated weights for policy 1, policy_version 749199 (0.0007) [2023-12-27 10:48:55,737][20347] Updated weights for policy 0, policy_version 749543 (0.0008) [2023-12-27 10:48:55,789][20347] Updated weights for policy 0, policy_version 749553 (0.0007) [2023-12-27 10:48:55,840][20347] Updated weights for policy 0, policy_version 749563 (0.0005) [2023-12-27 10:48:55,983][20381] Updated weights for policy 1, policy_version 749209 (0.0008) [2023-12-27 10:48:56,036][20381] Updated weights for policy 1, policy_version 749219 (0.0007) [2023-12-27 10:48:56,083][20381] Updated weights for policy 1, policy_version 749229 (0.0008) [2023-12-27 10:48:56,133][20381] Updated weights for policy 1, policy_version 749239 (0.0008) [2023-12-27 10:48:56,437][20347] Updated weights for policy 0, policy_version 749573 (0.0008) [2023-12-27 10:48:56,484][20347] Updated weights for policy 0, policy_version 749583 (0.0010) [2023-12-27 10:48:56,548][20347] Updated weights for policy 0, policy_version 749593 (0.0010) [2023-12-27 10:48:56,755][20381] Updated weights for policy 1, policy_version 749249 (0.0006) [2023-12-27 10:48:56,807][20381] Updated weights for policy 1, policy_version 749259 (0.0005) [2023-12-27 10:48:56,857][20381] Updated weights for policy 1, policy_version 749269 (0.0005) [2023-12-27 10:48:57,189][20347] Updated weights for policy 0, policy_version 749603 (0.0009) [2023-12-27 10:48:57,242][20347] Updated weights for policy 0, policy_version 749613 (0.0005) [2023-12-27 10:48:57,299][20347] Updated weights for policy 0, policy_version 749623 (0.0005) [2023-12-27 10:48:57,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20382.8). Total num frames: 383770624. Throughput: 0: 10223.3, 1: 10331.1. Samples: 383755100. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:48:57,364][19170] Avg episode reward: [(0, '912.300'), (1, '774.960')] [2023-12-27 10:48:57,381][20381] Updated weights for policy 1, policy_version 749279 (0.0008) [2023-12-27 10:48:57,429][20381] Updated weights for policy 1, policy_version 749289 (0.0010) [2023-12-27 10:48:57,483][20381] Updated weights for policy 1, policy_version 749299 (0.0010) [2023-12-27 10:48:57,946][20347] Updated weights for policy 0, policy_version 749633 (0.0006) [2023-12-27 10:48:57,989][20347] Updated weights for policy 0, policy_version 749643 (0.0006) [2023-12-27 10:48:58,050][20347] Updated weights for policy 0, policy_version 749653 (0.0006) [2023-12-27 10:48:58,112][20347] Updated weights for policy 0, policy_version 749663 (0.0006) [2023-12-27 10:48:58,122][20381] Updated weights for policy 1, policy_version 749309 (0.0008) [2023-12-27 10:48:58,170][20381] Updated weights for policy 1, policy_version 749319 (0.0010) [2023-12-27 10:48:58,235][20381] Updated weights for policy 1, policy_version 749329 (0.0010) [2023-12-27 10:48:58,785][20347] Updated weights for policy 0, policy_version 749673 (0.0008) [2023-12-27 10:48:58,849][20347] Updated weights for policy 0, policy_version 749683 (0.0008) [2023-12-27 10:48:58,912][20347] Updated weights for policy 0, policy_version 749693 (0.0007) [2023-12-27 10:48:58,966][20381] Updated weights for policy 1, policy_version 749339 (0.0009) [2023-12-27 10:48:59,024][20381] Updated weights for policy 1, policy_version 749349 (0.0005) [2023-12-27 10:48:59,076][20381] Updated weights for policy 1, policy_version 749359 (0.0007) [2023-12-27 10:48:59,655][20347] Updated weights for policy 0, policy_version 749703 (0.0007) [2023-12-27 10:48:59,709][20347] Updated weights for policy 0, policy_version 749713 (0.0008) [2023-12-27 10:48:59,727][20381] Updated weights for policy 1, policy_version 749369 (0.0007) [2023-12-27 10:48:59,768][20347] Updated weights for policy 0, policy_version 749723 (0.0007) [2023-12-27 10:48:59,783][20381] Updated weights for policy 1, policy_version 749379 (0.0006) [2023-12-27 10:48:59,843][20381] Updated weights for policy 1, policy_version 749389 (0.0007) [2023-12-27 10:48:59,902][20381] Updated weights for policy 1, policy_version 749399 (0.0008) [2023-12-27 10:49:00,480][20347] Updated weights for policy 0, policy_version 749733 (0.0008) [2023-12-27 10:49:00,530][20347] Updated weights for policy 0, policy_version 749743 (0.0008) [2023-12-27 10:49:00,542][20381] Updated weights for policy 1, policy_version 749409 (0.0005) [2023-12-27 10:49:00,584][20347] Updated weights for policy 0, policy_version 749753 (0.0007) [2023-12-27 10:49:00,600][20381] Updated weights for policy 1, policy_version 749419 (0.0009) [2023-12-27 10:49:00,654][20381] Updated weights for policy 1, policy_version 749429 (0.0010) [2023-12-27 10:49:01,241][20347] Updated weights for policy 0, policy_version 749763 (0.0006) [2023-12-27 10:49:01,299][20347] Updated weights for policy 0, policy_version 749773 (0.0007) [2023-12-27 10:49:01,305][20381] Updated weights for policy 1, policy_version 749439 (0.0007) [2023-12-27 10:49:01,358][20347] Updated weights for policy 0, policy_version 749783 (0.0008) [2023-12-27 10:49:01,368][20381] Updated weights for policy 1, policy_version 749449 (0.0008) [2023-12-27 10:49:01,427][20381] Updated weights for policy 1, policy_version 749459 (0.0006) [2023-12-27 10:49:02,021][20381] Updated weights for policy 1, policy_version 749469 (0.0008) [2023-12-27 10:49:02,080][20381] Updated weights for policy 1, policy_version 749479 (0.0010) [2023-12-27 10:49:02,085][20347] Updated weights for policy 0, policy_version 749793 (0.0008) [2023-12-27 10:49:02,135][20291] KL-divergence is very high: 128.5047 [2023-12-27 10:49:02,140][20347] Updated weights for policy 0, policy_version 749803 (0.0010) [2023-12-27 10:49:02,141][20381] Updated weights for policy 1, policy_version 749489 (0.0010) [2023-12-27 10:49:02,203][20347] Updated weights for policy 0, policy_version 749813 (0.0010) [2023-12-27 10:49:02,264][20347] Updated weights for policy 0, policy_version 749823 (0.0010) [2023-12-27 10:49:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20410.6). Total num frames: 383877120. Throughput: 0: 10210.0, 1: 10308.1. Samples: 383881236. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:02,363][19170] Avg episode reward: [(0, '931.740'), (1, '706.180')] [2023-12-27 10:49:02,824][20381] Updated weights for policy 1, policy_version 749499 (0.0010) [2023-12-27 10:49:02,878][20381] Updated weights for policy 1, policy_version 749509 (0.0010) [2023-12-27 10:49:02,897][20347] Updated weights for policy 0, policy_version 749833 (0.0010) [2023-12-27 10:49:02,923][20381] Updated weights for policy 1, policy_version 749519 (0.0010) [2023-12-27 10:49:02,954][20347] Updated weights for policy 0, policy_version 749843 (0.0010) [2023-12-27 10:49:03,005][20347] Updated weights for policy 0, policy_version 749853 (0.0007) [2023-12-27 10:49:03,574][20347] Updated weights for policy 0, policy_version 749863 (0.0007) [2023-12-27 10:49:03,604][20381] Updated weights for policy 1, policy_version 749529 (0.0010) [2023-12-27 10:49:03,624][20347] Updated weights for policy 0, policy_version 749873 (0.0007) [2023-12-27 10:49:03,653][20381] Updated weights for policy 1, policy_version 749539 (0.0010) [2023-12-27 10:49:03,672][20347] Updated weights for policy 0, policy_version 749883 (0.0006) [2023-12-27 10:49:03,704][20381] Updated weights for policy 1, policy_version 749549 (0.0010) [2023-12-27 10:49:03,755][20381] Updated weights for policy 1, policy_version 749559 (0.0010) [2023-12-27 10:49:04,364][20347] Updated weights for policy 0, policy_version 749893 (0.0006) [2023-12-27 10:49:04,421][20347] Updated weights for policy 0, policy_version 749903 (0.0008) [2023-12-27 10:49:04,469][20381] Updated weights for policy 1, policy_version 749569 (0.0008) [2023-12-27 10:49:04,470][20347] Updated weights for policy 0, policy_version 749913 (0.0008) [2023-12-27 10:49:04,531][20381] Updated weights for policy 1, policy_version 749579 (0.0007) [2023-12-27 10:49:04,597][20381] Updated weights for policy 1, policy_version 749589 (0.0007) [2023-12-27 10:49:05,175][20347] Updated weights for policy 0, policy_version 749923 (0.0008) [2023-12-27 10:49:05,232][20347] Updated weights for policy 0, policy_version 749933 (0.0008) [2023-12-27 10:49:05,243][20381] Updated weights for policy 1, policy_version 749599 (0.0006) [2023-12-27 10:49:05,284][20347] Updated weights for policy 0, policy_version 749943 (0.0010) [2023-12-27 10:49:05,298][20381] Updated weights for policy 1, policy_version 749609 (0.0006) [2023-12-27 10:49:05,348][20381] Updated weights for policy 1, policy_version 749619 (0.0010) [2023-12-27 10:49:05,974][20347] Updated weights for policy 0, policy_version 749953 (0.0010) [2023-12-27 10:49:06,024][20381] Updated weights for policy 1, policy_version 749629 (0.0010) [2023-12-27 10:49:06,028][20347] Updated weights for policy 0, policy_version 749963 (0.0010) [2023-12-27 10:49:06,074][20381] Updated weights for policy 1, policy_version 749639 (0.0010) [2023-12-27 10:49:06,084][20347] Updated weights for policy 0, policy_version 749973 (0.0010) [2023-12-27 10:49:06,131][20381] Updated weights for policy 1, policy_version 749649 (0.0010) [2023-12-27 10:49:06,142][20347] Updated weights for policy 0, policy_version 749983 (0.0008) [2023-12-27 10:49:06,812][20347] Updated weights for policy 0, policy_version 749993 (0.0010) [2023-12-27 10:49:06,856][20381] Updated weights for policy 1, policy_version 749659 (0.0010) [2023-12-27 10:49:06,864][20347] Updated weights for policy 0, policy_version 750003 (0.0010) [2023-12-27 10:49:06,909][20381] Updated weights for policy 1, policy_version 749669 (0.0011) [2023-12-27 10:49:06,913][20347] Updated weights for policy 0, policy_version 750013 (0.0010) [2023-12-27 10:49:06,968][20381] Updated weights for policy 1, policy_version 749679 (0.0011) [2023-12-27 10:49:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 383975424. Throughput: 0: 10268.1, 1: 10301.1. Samples: 383943884. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:07,363][19170] Avg episode reward: [(0, '961.000'), (1, '637.810')] [2023-12-27 10:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000749688_191946752.pth... [2023-12-27 10:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000750016_192028672.pth... [2023-12-27 10:49:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000748832_191725568.pth [2023-12-27 10:49:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000748472_191635456.pth [2023-12-27 10:49:07,612][20347] Updated weights for policy 0, policy_version 750023 (0.0010) [2023-12-27 10:49:07,646][20381] Updated weights for policy 1, policy_version 749689 (0.0010) [2023-12-27 10:49:07,667][20347] Updated weights for policy 0, policy_version 750033 (0.0010) [2023-12-27 10:49:07,701][20381] Updated weights for policy 1, policy_version 749699 (0.0006) [2023-12-27 10:49:07,722][20347] Updated weights for policy 0, policy_version 750043 (0.0010) [2023-12-27 10:49:07,756][20381] Updated weights for policy 1, policy_version 749709 (0.0007) [2023-12-27 10:49:07,812][20381] Updated weights for policy 1, policy_version 749719 (0.0008) [2023-12-27 10:49:08,415][20347] Updated weights for policy 0, policy_version 750053 (0.0008) [2023-12-27 10:49:08,472][20347] Updated weights for policy 0, policy_version 750063 (0.0009) [2023-12-27 10:49:08,524][20381] Updated weights for policy 1, policy_version 749729 (0.0010) [2023-12-27 10:49:08,536][20347] Updated weights for policy 0, policy_version 750073 (0.0007) [2023-12-27 10:49:08,583][20381] Updated weights for policy 1, policy_version 749739 (0.0010) [2023-12-27 10:49:08,639][20381] Updated weights for policy 1, policy_version 749749 (0.0011) [2023-12-27 10:49:09,232][20347] Updated weights for policy 0, policy_version 750083 (0.0006) [2023-12-27 10:49:09,289][20347] Updated weights for policy 0, policy_version 750093 (0.0007) [2023-12-27 10:49:09,352][20347] Updated weights for policy 0, policy_version 750103 (0.0008) [2023-12-27 10:49:09,372][20381] Updated weights for policy 1, policy_version 749759 (0.0010) [2023-12-27 10:49:09,435][20381] Updated weights for policy 1, policy_version 749769 (0.0008) [2023-12-27 10:49:09,487][20381] Updated weights for policy 1, policy_version 749779 (0.0008) [2023-12-27 10:49:10,122][20347] Updated weights for policy 0, policy_version 750113 (0.0007) [2023-12-27 10:49:10,185][20347] Updated weights for policy 0, policy_version 750123 (0.0007) [2023-12-27 10:49:10,233][20381] Updated weights for policy 1, policy_version 749789 (0.0007) [2023-12-27 10:49:10,251][20347] Updated weights for policy 0, policy_version 750133 (0.0007) [2023-12-27 10:49:10,292][20381] Updated weights for policy 1, policy_version 749799 (0.0007) [2023-12-27 10:49:10,317][20347] Updated weights for policy 0, policy_version 750143 (0.0007) [2023-12-27 10:49:10,351][20381] Updated weights for policy 1, policy_version 749809 (0.0007) [2023-12-27 10:49:10,964][20381] Updated weights for policy 1, policy_version 749819 (0.0007) [2023-12-27 10:49:10,969][20347] Updated weights for policy 0, policy_version 750153 (0.0006) [2023-12-27 10:49:11,027][20381] Updated weights for policy 1, policy_version 749829 (0.0007) [2023-12-27 10:49:11,033][20347] Updated weights for policy 0, policy_version 750163 (0.0006) [2023-12-27 10:49:11,089][20381] Updated weights for policy 1, policy_version 749839 (0.0007) [2023-12-27 10:49:11,091][20347] Updated weights for policy 0, policy_version 750173 (0.0006) [2023-12-27 10:49:11,798][20381] Updated weights for policy 1, policy_version 749849 (0.0008) [2023-12-27 10:49:11,841][20347] Updated weights for policy 0, policy_version 750183 (0.0007) [2023-12-27 10:49:11,857][20381] Updated weights for policy 1, policy_version 749859 (0.0007) [2023-12-27 10:49:11,898][20347] Updated weights for policy 0, policy_version 750193 (0.0006) [2023-12-27 10:49:11,914][20381] Updated weights for policy 1, policy_version 749869 (0.0007) [2023-12-27 10:49:11,954][20347] Updated weights for policy 0, policy_version 750203 (0.0006) [2023-12-27 10:49:11,968][20381] Updated weights for policy 1, policy_version 749879 (0.0007) [2023-12-27 10:49:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 384073728. Throughput: 0: 10297.3, 1: 10260.0. Samples: 384063572. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:12,363][19170] Avg episode reward: [(0, '980.560'), (1, '618.530')] [2023-12-27 10:49:12,681][20347] Updated weights for policy 0, policy_version 750213 (0.0007) [2023-12-27 10:49:12,712][20381] Updated weights for policy 1, policy_version 749889 (0.0007) [2023-12-27 10:49:12,742][20347] Updated weights for policy 0, policy_version 750223 (0.0007) [2023-12-27 10:49:12,765][20381] Updated weights for policy 1, policy_version 749899 (0.0006) [2023-12-27 10:49:12,805][20347] Updated weights for policy 0, policy_version 750233 (0.0006) [2023-12-27 10:49:12,821][20381] Updated weights for policy 1, policy_version 749909 (0.0007) [2023-12-27 10:49:13,479][20381] Updated weights for policy 1, policy_version 749919 (0.0007) [2023-12-27 10:49:13,485][20347] Updated weights for policy 0, policy_version 750243 (0.0007) [2023-12-27 10:49:13,527][20381] Updated weights for policy 1, policy_version 749929 (0.0010) [2023-12-27 10:49:13,543][20347] Updated weights for policy 0, policy_version 750253 (0.0005) [2023-12-27 10:49:13,591][20381] Updated weights for policy 1, policy_version 749939 (0.0010) [2023-12-27 10:49:13,600][20347] Updated weights for policy 0, policy_version 750263 (0.0005) [2023-12-27 10:49:14,201][20347] Updated weights for policy 0, policy_version 750273 (0.0006) [2023-12-27 10:49:14,254][20347] Updated weights for policy 0, policy_version 750283 (0.0005) [2023-12-27 10:49:14,278][20381] Updated weights for policy 1, policy_version 749949 (0.0010) [2023-12-27 10:49:14,305][20347] Updated weights for policy 0, policy_version 750293 (0.0005) [2023-12-27 10:49:14,330][20381] Updated weights for policy 1, policy_version 749959 (0.0011) [2023-12-27 10:49:14,353][20347] Updated weights for policy 0, policy_version 750303 (0.0005) [2023-12-27 10:49:14,388][20381] Updated weights for policy 1, policy_version 749969 (0.0011) [2023-12-27 10:49:14,939][20347] Updated weights for policy 0, policy_version 750313 (0.0007) [2023-12-27 10:49:15,004][20347] Updated weights for policy 0, policy_version 750323 (0.0007) [2023-12-27 10:49:15,062][20347] Updated weights for policy 0, policy_version 750333 (0.0007) [2023-12-27 10:49:15,084][20381] Updated weights for policy 1, policy_version 749979 (0.0011) [2023-12-27 10:49:15,149][20381] Updated weights for policy 1, policy_version 749989 (0.0010) [2023-12-27 10:49:15,211][20381] Updated weights for policy 1, policy_version 749999 (0.0008) [2023-12-27 10:49:15,671][20347] Updated weights for policy 0, policy_version 750343 (0.0007) [2023-12-27 10:49:15,719][20347] Updated weights for policy 0, policy_version 750353 (0.0007) [2023-12-27 10:49:15,771][20347] Updated weights for policy 0, policy_version 750363 (0.0008) [2023-12-27 10:49:15,880][20381] Updated weights for policy 1, policy_version 750009 (0.0008) [2023-12-27 10:49:15,934][20381] Updated weights for policy 1, policy_version 750019 (0.0008) [2023-12-27 10:49:15,980][20381] Updated weights for policy 1, policy_version 750029 (0.0008) [2023-12-27 10:49:16,035][20381] Updated weights for policy 1, policy_version 750039 (0.0005) [2023-12-27 10:49:16,487][20347] Updated weights for policy 0, policy_version 750373 (0.0008) [2023-12-27 10:49:16,548][20347] Updated weights for policy 0, policy_version 750383 (0.0007) [2023-12-27 10:49:16,598][20347] Updated weights for policy 0, policy_version 750393 (0.0007) [2023-12-27 10:49:16,640][20381] Updated weights for policy 1, policy_version 750049 (0.0008) [2023-12-27 10:49:16,688][20381] Updated weights for policy 1, policy_version 750059 (0.0008) [2023-12-27 10:49:16,734][20381] Updated weights for policy 1, policy_version 750069 (0.0007) [2023-12-27 10:49:17,224][20347] Updated weights for policy 0, policy_version 750403 (0.0006) [2023-12-27 10:49:17,282][20347] Updated weights for policy 0, policy_version 750413 (0.0008) [2023-12-27 10:49:17,339][20347] Updated weights for policy 0, policy_version 750423 (0.0007) [2023-12-27 10:49:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20410.6). Total num frames: 384172032. Throughput: 0: 10299.1, 1: 10265.7. Samples: 384187948. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:17,364][19170] Avg episode reward: [(0, '892.710'), (1, '638.800')] [2023-12-27 10:49:17,435][20381] Updated weights for policy 1, policy_version 750079 (0.0009) [2023-12-27 10:49:17,495][20381] Updated weights for policy 1, policy_version 750089 (0.0011) [2023-12-27 10:49:17,554][20381] Updated weights for policy 1, policy_version 750099 (0.0011) [2023-12-27 10:49:18,031][20347] Updated weights for policy 0, policy_version 750433 (0.0007) [2023-12-27 10:49:18,079][20347] Updated weights for policy 0, policy_version 750443 (0.0007) [2023-12-27 10:49:18,142][20347] Updated weights for policy 0, policy_version 750453 (0.0007) [2023-12-27 10:49:18,198][20347] Updated weights for policy 0, policy_version 750463 (0.0007) [2023-12-27 10:49:18,231][20381] Updated weights for policy 1, policy_version 750109 (0.0010) [2023-12-27 10:49:18,292][20381] Updated weights for policy 1, policy_version 750119 (0.0010) [2023-12-27 10:49:18,352][20381] Updated weights for policy 1, policy_version 750129 (0.0010) [2023-12-27 10:49:18,912][20347] Updated weights for policy 0, policy_version 750473 (0.0008) [2023-12-27 10:49:18,968][20347] Updated weights for policy 0, policy_version 750483 (0.0007) [2023-12-27 10:49:19,027][20347] Updated weights for policy 0, policy_version 750493 (0.0007) [2023-12-27 10:49:19,060][20381] Updated weights for policy 1, policy_version 750139 (0.0010) [2023-12-27 10:49:19,106][20381] Updated weights for policy 1, policy_version 750149 (0.0009) [2023-12-27 10:49:19,161][20381] Updated weights for policy 1, policy_version 750159 (0.0011) [2023-12-27 10:49:19,723][20347] Updated weights for policy 0, policy_version 750503 (0.0006) [2023-12-27 10:49:19,780][20347] Updated weights for policy 0, policy_version 750513 (0.0005) [2023-12-27 10:49:19,846][20347] Updated weights for policy 0, policy_version 750523 (0.0007) [2023-12-27 10:49:19,880][20381] Updated weights for policy 1, policy_version 750169 (0.0009) [2023-12-27 10:49:19,938][20381] Updated weights for policy 1, policy_version 750179 (0.0011) [2023-12-27 10:49:20,028][20381] Updated weights for policy 1, policy_version 750189 (0.0008) [2023-12-27 10:49:20,085][20381] Updated weights for policy 1, policy_version 750199 (0.0008) [2023-12-27 10:49:20,521][20347] Updated weights for policy 0, policy_version 750533 (0.0007) [2023-12-27 10:49:20,576][20347] Updated weights for policy 0, policy_version 750543 (0.0007) [2023-12-27 10:49:20,642][20347] Updated weights for policy 0, policy_version 750553 (0.0006) [2023-12-27 10:49:20,846][20381] Updated weights for policy 1, policy_version 750209 (0.0008) [2023-12-27 10:49:20,901][20381] Updated weights for policy 1, policy_version 750219 (0.0006) [2023-12-27 10:49:20,950][20381] Updated weights for policy 1, policy_version 750229 (0.0005) [2023-12-27 10:49:21,319][20347] Updated weights for policy 0, policy_version 750563 (0.0009) [2023-12-27 10:49:21,383][20347] Updated weights for policy 0, policy_version 750573 (0.0009) [2023-12-27 10:49:21,452][20347] Updated weights for policy 0, policy_version 750583 (0.0008) [2023-12-27 10:49:21,699][20381] Updated weights for policy 1, policy_version 750239 (0.0006) [2023-12-27 10:49:21,762][20381] Updated weights for policy 1, policy_version 750249 (0.0007) [2023-12-27 10:49:21,818][20381] Updated weights for policy 1, policy_version 750259 (0.0007) [2023-12-27 10:49:22,190][20347] Updated weights for policy 0, policy_version 750593 (0.0011) [2023-12-27 10:49:22,253][20347] Updated weights for policy 0, policy_version 750603 (0.0007) [2023-12-27 10:49:22,314][20347] Updated weights for policy 0, policy_version 750613 (0.0008) [2023-12-27 10:49:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 384270336. Throughput: 0: 10260.1, 1: 10202.6. Samples: 384247036. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:22,363][19170] Avg episode reward: [(0, '902.460'), (1, '611.570')] [2023-12-27 10:49:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000750264_192094208.pth... [2023-12-27 10:49:22,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000749080_191791104.pth [2023-12-27 10:49:22,388][20347] Updated weights for policy 0, policy_version 750623 (0.0007) [2023-12-27 10:49:22,392][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000750624_192184320.pth... [2023-12-27 10:49:22,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000749408_191873024.pth [2023-12-27 10:49:22,509][20381] Updated weights for policy 1, policy_version 750269 (0.0008) [2023-12-27 10:49:22,569][20381] Updated weights for policy 1, policy_version 750279 (0.0008) [2023-12-27 10:49:22,639][20381] Updated weights for policy 1, policy_version 750289 (0.0006) [2023-12-27 10:49:23,093][20347] Updated weights for policy 0, policy_version 750633 (0.0008) [2023-12-27 10:49:23,148][20347] Updated weights for policy 0, policy_version 750643 (0.0007) [2023-12-27 10:49:23,198][20347] Updated weights for policy 0, policy_version 750653 (0.0007) [2023-12-27 10:49:23,274][20381] Updated weights for policy 1, policy_version 750299 (0.0006) [2023-12-27 10:49:23,332][20381] Updated weights for policy 1, policy_version 750309 (0.0007) [2023-12-27 10:49:23,388][20381] Updated weights for policy 1, policy_version 750319 (0.0008) [2023-12-27 10:49:23,895][20347] Updated weights for policy 0, policy_version 750663 (0.0005) [2023-12-27 10:49:23,958][20347] Updated weights for policy 0, policy_version 750673 (0.0005) [2023-12-27 10:49:24,009][20381] Updated weights for policy 1, policy_version 750329 (0.0008) [2023-12-27 10:49:24,011][20347] Updated weights for policy 0, policy_version 750683 (0.0005) [2023-12-27 10:49:24,054][20381] Updated weights for policy 1, policy_version 750339 (0.0010) [2023-12-27 10:49:24,109][20381] Updated weights for policy 1, policy_version 750349 (0.0010) [2023-12-27 10:49:24,158][20381] Updated weights for policy 1, policy_version 750359 (0.0010) [2023-12-27 10:49:24,620][20347] Updated weights for policy 0, policy_version 750693 (0.0008) [2023-12-27 10:49:24,678][20347] Updated weights for policy 0, policy_version 750703 (0.0010) [2023-12-27 10:49:24,735][20347] Updated weights for policy 0, policy_version 750713 (0.0010) [2023-12-27 10:49:24,761][20381] Updated weights for policy 1, policy_version 750369 (0.0010) [2023-12-27 10:49:24,815][20381] Updated weights for policy 1, policy_version 750379 (0.0010) [2023-12-27 10:49:24,866][20381] Updated weights for policy 1, policy_version 750389 (0.0010) [2023-12-27 10:49:25,382][20347] Updated weights for policy 0, policy_version 750723 (0.0010) [2023-12-27 10:49:25,441][20347] Updated weights for policy 0, policy_version 750733 (0.0011) [2023-12-27 10:49:25,499][20347] Updated weights for policy 0, policy_version 750743 (0.0010) [2023-12-27 10:49:25,617][20381] Updated weights for policy 1, policy_version 750399 (0.0010) [2023-12-27 10:49:25,672][20381] Updated weights for policy 1, policy_version 750409 (0.0010) [2023-12-27 10:49:25,730][20381] Updated weights for policy 1, policy_version 750419 (0.0011) [2023-12-27 10:49:26,162][20347] Updated weights for policy 0, policy_version 750753 (0.0010) [2023-12-27 10:49:26,236][20347] Updated weights for policy 0, policy_version 750763 (0.0006) [2023-12-27 10:49:26,290][20347] Updated weights for policy 0, policy_version 750773 (0.0008) [2023-12-27 10:49:26,338][20347] Updated weights for policy 0, policy_version 750783 (0.0010) [2023-12-27 10:49:26,348][20381] Updated weights for policy 1, policy_version 750429 (0.0008) [2023-12-27 10:49:26,410][20381] Updated weights for policy 1, policy_version 750439 (0.0005) [2023-12-27 10:49:26,481][20381] Updated weights for policy 1, policy_version 750449 (0.0006) [2023-12-27 10:49:26,889][20347] Updated weights for policy 0, policy_version 750793 (0.0010) [2023-12-27 10:49:26,905][20291] KL-divergence is very high: 164.8203 [2023-12-27 10:49:26,933][20347] Updated weights for policy 0, policy_version 750803 (0.0010) [2023-12-27 10:49:26,996][20347] Updated weights for policy 0, policy_version 750813 (0.0010) [2023-12-27 10:49:27,005][20381] Updated weights for policy 1, policy_version 750459 (0.0005) [2023-12-27 10:49:27,067][20381] Updated weights for policy 1, policy_version 750469 (0.0007) [2023-12-27 10:49:27,125][20381] Updated weights for policy 1, policy_version 750479 (0.0005) [2023-12-27 10:49:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 384385024. Throughput: 0: 10226.7, 1: 10259.1. Samples: 384370232. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:27,363][19170] Avg episode reward: [(0, '970.800'), (1, '670.680')] [2023-12-27 10:49:27,585][20347] Updated weights for policy 0, policy_version 750823 (0.0007) [2023-12-27 10:49:27,628][20347] Updated weights for policy 0, policy_version 750833 (0.0005) [2023-12-27 10:49:27,681][20347] Updated weights for policy 0, policy_version 750843 (0.0010) [2023-12-27 10:49:27,722][20381] Updated weights for policy 1, policy_version 750489 (0.0009) [2023-12-27 10:49:27,768][20381] Updated weights for policy 1, policy_version 750499 (0.0005) [2023-12-27 10:49:27,811][20381] Updated weights for policy 1, policy_version 750509 (0.0005) [2023-12-27 10:49:27,858][20381] Updated weights for policy 1, policy_version 750519 (0.0005) [2023-12-27 10:49:28,330][20347] Updated weights for policy 0, policy_version 750853 (0.0011) [2023-12-27 10:49:28,389][20347] Updated weights for policy 0, policy_version 750863 (0.0009) [2023-12-27 10:49:28,410][20381] Updated weights for policy 1, policy_version 750529 (0.0010) [2023-12-27 10:49:28,441][20347] Updated weights for policy 0, policy_version 750873 (0.0010) [2023-12-27 10:49:28,468][20381] Updated weights for policy 1, policy_version 750539 (0.0010) [2023-12-27 10:49:28,529][20381] Updated weights for policy 1, policy_version 750549 (0.0010) [2023-12-27 10:49:29,075][20381] Updated weights for policy 1, policy_version 750559 (0.0007) [2023-12-27 10:49:29,123][20381] Updated weights for policy 1, policy_version 750569 (0.0005) [2023-12-27 10:49:29,127][20347] Updated weights for policy 0, policy_version 750883 (0.0010) [2023-12-27 10:49:29,174][20381] Updated weights for policy 1, policy_version 750579 (0.0005) [2023-12-27 10:49:29,180][20347] Updated weights for policy 0, policy_version 750893 (0.0009) [2023-12-27 10:49:29,241][20347] Updated weights for policy 0, policy_version 750903 (0.0009) [2023-12-27 10:49:29,796][20381] Updated weights for policy 1, policy_version 750589 (0.0007) [2023-12-27 10:49:29,860][20381] Updated weights for policy 1, policy_version 750599 (0.0009) [2023-12-27 10:49:29,908][20347] Updated weights for policy 0, policy_version 750913 (0.0008) [2023-12-27 10:49:29,920][20381] Updated weights for policy 1, policy_version 750609 (0.0009) [2023-12-27 10:49:29,969][20347] Updated weights for policy 0, policy_version 750923 (0.0006) [2023-12-27 10:49:30,028][20347] Updated weights for policy 0, policy_version 750933 (0.0006) [2023-12-27 10:49:30,084][20347] Updated weights for policy 0, policy_version 750943 (0.0007) [2023-12-27 10:49:30,604][20381] Updated weights for policy 1, policy_version 750619 (0.0007) [2023-12-27 10:49:30,660][20381] Updated weights for policy 1, policy_version 750629 (0.0008) [2023-12-27 10:49:30,716][20347] Updated weights for policy 0, policy_version 750953 (0.0006) [2023-12-27 10:49:30,717][20381] Updated weights for policy 1, policy_version 750639 (0.0008) [2023-12-27 10:49:30,773][20347] Updated weights for policy 0, policy_version 750963 (0.0007) [2023-12-27 10:49:30,827][20347] Updated weights for policy 0, policy_version 750973 (0.0007) [2023-12-27 10:49:31,398][20381] Updated weights for policy 1, policy_version 750649 (0.0006) [2023-12-27 10:49:31,459][20381] Updated weights for policy 1, policy_version 750659 (0.0007) [2023-12-27 10:49:31,499][20347] Updated weights for policy 0, policy_version 750983 (0.0008) [2023-12-27 10:49:31,515][20381] Updated weights for policy 1, policy_version 750669 (0.0010) [2023-12-27 10:49:31,555][20347] Updated weights for policy 0, policy_version 750993 (0.0007) [2023-12-27 10:49:31,568][20381] Updated weights for policy 1, policy_version 750679 (0.0006) [2023-12-27 10:49:31,612][20347] Updated weights for policy 0, policy_version 751003 (0.0007) [2023-12-27 10:49:32,197][20381] Updated weights for policy 1, policy_version 750689 (0.0008) [2023-12-27 10:49:32,257][20381] Updated weights for policy 1, policy_version 750699 (0.0008) [2023-12-27 10:49:32,313][20347] Updated weights for policy 0, policy_version 751013 (0.0006) [2023-12-27 10:49:32,323][20381] Updated weights for policy 1, policy_version 750709 (0.0007) [2023-12-27 10:49:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 384491520. Throughput: 0: 10254.2, 1: 10373.1. Samples: 384500876. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:32,363][19170] Avg episode reward: [(0, '961.060'), (1, '648.270')] [2023-12-27 10:49:32,377][20347] Updated weights for policy 0, policy_version 751023 (0.0008) [2023-12-27 10:49:32,439][20347] Updated weights for policy 0, policy_version 751033 (0.0008) [2023-12-27 10:49:32,888][20381] Updated weights for policy 1, policy_version 750719 (0.0005) [2023-12-27 10:49:32,944][20381] Updated weights for policy 1, policy_version 750729 (0.0007) [2023-12-27 10:49:32,994][20381] Updated weights for policy 1, policy_version 750739 (0.0006) [2023-12-27 10:49:33,169][20347] Updated weights for policy 0, policy_version 751043 (0.0008) [2023-12-27 10:49:33,223][20347] Updated weights for policy 0, policy_version 751053 (0.0007) [2023-12-27 10:49:33,278][20347] Updated weights for policy 0, policy_version 751063 (0.0006) [2023-12-27 10:49:33,585][20381] Updated weights for policy 1, policy_version 750749 (0.0005) [2023-12-27 10:49:33,631][20381] Updated weights for policy 1, policy_version 750759 (0.0005) [2023-12-27 10:49:33,682][20381] Updated weights for policy 1, policy_version 750769 (0.0008) [2023-12-27 10:49:33,937][20347] Updated weights for policy 0, policy_version 751073 (0.0007) [2023-12-27 10:49:33,993][20347] Updated weights for policy 0, policy_version 751083 (0.0007) [2023-12-27 10:49:34,048][20347] Updated weights for policy 0, policy_version 751093 (0.0006) [2023-12-27 10:49:34,101][20347] Updated weights for policy 0, policy_version 751103 (0.0005) [2023-12-27 10:49:34,299][20381] Updated weights for policy 1, policy_version 750779 (0.0010) [2023-12-27 10:49:34,354][20381] Updated weights for policy 1, policy_version 750789 (0.0011) [2023-12-27 10:49:34,414][20381] Updated weights for policy 1, policy_version 750799 (0.0010) [2023-12-27 10:49:34,787][20347] Updated weights for policy 0, policy_version 751113 (0.0007) [2023-12-27 10:49:34,848][20347] Updated weights for policy 0, policy_version 751123 (0.0007) [2023-12-27 10:49:34,903][20347] Updated weights for policy 0, policy_version 751133 (0.0007) [2023-12-27 10:49:35,062][20381] Updated weights for policy 1, policy_version 750809 (0.0010) [2023-12-27 10:49:35,120][20381] Updated weights for policy 1, policy_version 750819 (0.0011) [2023-12-27 10:49:35,178][20381] Updated weights for policy 1, policy_version 750829 (0.0010) [2023-12-27 10:49:35,228][20381] Updated weights for policy 1, policy_version 750839 (0.0007) [2023-12-27 10:49:35,587][20347] Updated weights for policy 0, policy_version 751143 (0.0008) [2023-12-27 10:49:35,639][20347] Updated weights for policy 0, policy_version 751153 (0.0008) [2023-12-27 10:49:35,692][20347] Updated weights for policy 0, policy_version 751163 (0.0007) [2023-12-27 10:49:35,864][20381] Updated weights for policy 1, policy_version 750849 (0.0008) [2023-12-27 10:49:35,916][20381] Updated weights for policy 1, policy_version 750859 (0.0008) [2023-12-27 10:49:35,968][20381] Updated weights for policy 1, policy_version 750869 (0.0007) [2023-12-27 10:49:36,440][20347] Updated weights for policy 0, policy_version 751173 (0.0007) [2023-12-27 10:49:36,501][20347] Updated weights for policy 0, policy_version 751183 (0.0008) [2023-12-27 10:49:36,567][20347] Updated weights for policy 0, policy_version 751193 (0.0007) [2023-12-27 10:49:36,642][20381] Updated weights for policy 1, policy_version 750879 (0.0007) [2023-12-27 10:49:36,701][20381] Updated weights for policy 1, policy_version 750889 (0.0008) [2023-12-27 10:49:36,758][20381] Updated weights for policy 1, policy_version 750899 (0.0006) [2023-12-27 10:49:37,270][20347] Updated weights for policy 0, policy_version 751203 (0.0007) [2023-12-27 10:49:37,325][20347] Updated weights for policy 0, policy_version 751213 (0.0007) [2023-12-27 10:49:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 384589824. Throughput: 0: 10255.5, 1: 10428.7. Samples: 384564952. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:37,363][19170] Avg episode reward: [(0, '951.400'), (1, '628.790')] [2023-12-27 10:49:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000750904_192258048.pth... [2023-12-27 10:49:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000749688_191946752.pth [2023-12-27 10:49:37,383][20347] Updated weights for policy 0, policy_version 751223 (0.0009) [2023-12-27 10:49:37,434][20381] Updated weights for policy 1, policy_version 750909 (0.0007) [2023-12-27 10:49:37,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000751232_192339968.pth... [2023-12-27 10:49:37,439][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000750016_192028672.pth [2023-12-27 10:49:37,490][20381] Updated weights for policy 1, policy_version 750919 (0.0007) [2023-12-27 10:49:37,549][20381] Updated weights for policy 1, policy_version 750929 (0.0008) [2023-12-27 10:49:37,996][20347] Updated weights for policy 0, policy_version 751233 (0.0007) [2023-12-27 10:49:38,060][20347] Updated weights for policy 0, policy_version 751243 (0.0005) [2023-12-27 10:49:38,115][20347] Updated weights for policy 0, policy_version 751253 (0.0007) [2023-12-27 10:49:38,160][20347] Updated weights for policy 0, policy_version 751263 (0.0005) [2023-12-27 10:49:38,289][20381] Updated weights for policy 1, policy_version 750939 (0.0008) [2023-12-27 10:49:38,354][20381] Updated weights for policy 1, policy_version 750949 (0.0008) [2023-12-27 10:49:38,406][20381] Updated weights for policy 1, policy_version 750959 (0.0006) [2023-12-27 10:49:38,855][20347] Updated weights for policy 0, policy_version 751273 (0.0010) [2023-12-27 10:49:38,908][20347] Updated weights for policy 0, policy_version 751283 (0.0008) [2023-12-27 10:49:38,956][20347] Updated weights for policy 0, policy_version 751293 (0.0007) [2023-12-27 10:49:39,083][20381] Updated weights for policy 1, policy_version 750969 (0.0006) [2023-12-27 10:49:39,140][20381] Updated weights for policy 1, policy_version 750979 (0.0008) [2023-12-27 10:49:39,202][20381] Updated weights for policy 1, policy_version 750989 (0.0006) [2023-12-27 10:49:39,267][20381] Updated weights for policy 1, policy_version 750999 (0.0008) [2023-12-27 10:49:39,730][20347] Updated weights for policy 0, policy_version 751303 (0.0007) [2023-12-27 10:49:39,784][20347] Updated weights for policy 0, policy_version 751313 (0.0007) [2023-12-27 10:49:39,846][20347] Updated weights for policy 0, policy_version 751323 (0.0008) [2023-12-27 10:49:40,042][20381] Updated weights for policy 1, policy_version 751009 (0.0010) [2023-12-27 10:49:40,103][20381] Updated weights for policy 1, policy_version 751019 (0.0011) [2023-12-27 10:49:40,166][20381] Updated weights for policy 1, policy_version 751029 (0.0009) [2023-12-27 10:49:40,548][20347] Updated weights for policy 0, policy_version 751333 (0.0008) [2023-12-27 10:49:40,599][20347] Updated weights for policy 0, policy_version 751343 (0.0007) [2023-12-27 10:49:40,655][20347] Updated weights for policy 0, policy_version 751353 (0.0007) [2023-12-27 10:49:40,863][20381] Updated weights for policy 1, policy_version 751039 (0.0011) [2023-12-27 10:49:40,916][20381] Updated weights for policy 1, policy_version 751049 (0.0011) [2023-12-27 10:49:40,969][20381] Updated weights for policy 1, policy_version 751059 (0.0011) [2023-12-27 10:49:41,387][20347] Updated weights for policy 0, policy_version 751363 (0.0008) [2023-12-27 10:49:41,452][20347] Updated weights for policy 0, policy_version 751373 (0.0007) [2023-12-27 10:49:41,509][20347] Updated weights for policy 0, policy_version 751383 (0.0008) [2023-12-27 10:49:41,766][20381] Updated weights for policy 1, policy_version 751069 (0.0009) [2023-12-27 10:49:41,832][20381] Updated weights for policy 1, policy_version 751079 (0.0006) [2023-12-27 10:49:41,896][20381] Updated weights for policy 1, policy_version 751089 (0.0008) [2023-12-27 10:49:42,164][20347] Updated weights for policy 0, policy_version 751393 (0.0007) [2023-12-27 10:49:42,230][20347] Updated weights for policy 0, policy_version 751403 (0.0007) [2023-12-27 10:49:42,295][20347] Updated weights for policy 0, policy_version 751413 (0.0007) [2023-12-27 10:49:42,359][20347] Updated weights for policy 0, policy_version 751423 (0.0007) [2023-12-27 10:49:42,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 384696320. Throughput: 0: 10230.0, 1: 10424.7. Samples: 384684556. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:42,363][19170] Avg episode reward: [(0, '970.900'), (1, '706.890')] [2023-12-27 10:49:42,610][20381] Updated weights for policy 1, policy_version 751099 (0.0008) [2023-12-27 10:49:42,674][20381] Updated weights for policy 1, policy_version 751109 (0.0009) [2023-12-27 10:49:42,739][20381] Updated weights for policy 1, policy_version 751119 (0.0008) [2023-12-27 10:49:43,040][20347] Updated weights for policy 0, policy_version 751433 (0.0008) [2023-12-27 10:49:43,094][20347] Updated weights for policy 0, policy_version 751443 (0.0010) [2023-12-27 10:49:43,149][20347] Updated weights for policy 0, policy_version 751453 (0.0010) [2023-12-27 10:49:43,448][20381] Updated weights for policy 1, policy_version 751129 (0.0008) [2023-12-27 10:49:43,497][20381] Updated weights for policy 1, policy_version 751139 (0.0007) [2023-12-27 10:49:43,546][20381] Updated weights for policy 1, policy_version 751149 (0.0008) [2023-12-27 10:49:43,592][20381] Updated weights for policy 1, policy_version 751159 (0.0008) [2023-12-27 10:49:43,726][20347] Updated weights for policy 0, policy_version 751463 (0.0007) [2023-12-27 10:49:43,785][20347] Updated weights for policy 0, policy_version 751473 (0.0005) [2023-12-27 10:49:43,837][20347] Updated weights for policy 0, policy_version 751483 (0.0005) [2023-12-27 10:49:44,334][20381] Updated weights for policy 1, policy_version 751169 (0.0006) [2023-12-27 10:49:44,386][20381] Updated weights for policy 1, policy_version 751179 (0.0008) [2023-12-27 10:49:44,431][20347] Updated weights for policy 0, policy_version 751493 (0.0007) [2023-12-27 10:49:44,435][20381] Updated weights for policy 1, policy_version 751189 (0.0008) [2023-12-27 10:49:44,486][20347] Updated weights for policy 0, policy_version 751503 (0.0006) [2023-12-27 10:49:44,542][20347] Updated weights for policy 0, policy_version 751513 (0.0010) [2023-12-27 10:49:45,080][20381] Updated weights for policy 1, policy_version 751199 (0.0007) [2023-12-27 10:49:45,140][20381] Updated weights for policy 1, policy_version 751209 (0.0006) [2023-12-27 10:49:45,200][20381] Updated weights for policy 1, policy_version 751219 (0.0007) [2023-12-27 10:49:45,258][20347] Updated weights for policy 0, policy_version 751523 (0.0010) [2023-12-27 10:49:45,316][20347] Updated weights for policy 0, policy_version 751533 (0.0009) [2023-12-27 10:49:45,377][20347] Updated weights for policy 0, policy_version 751543 (0.0010) [2023-12-27 10:49:45,863][20381] Updated weights for policy 1, policy_version 751229 (0.0007) [2023-12-27 10:49:45,912][20381] Updated weights for policy 1, policy_version 751239 (0.0006) [2023-12-27 10:49:45,964][20381] Updated weights for policy 1, policy_version 751249 (0.0006) [2023-12-27 10:49:46,003][20347] Updated weights for policy 0, policy_version 751553 (0.0010) [2023-12-27 10:49:46,050][20347] Updated weights for policy 0, policy_version 751563 (0.0005) [2023-12-27 10:49:46,094][20347] Updated weights for policy 0, policy_version 751573 (0.0005) [2023-12-27 10:49:46,140][20347] Updated weights for policy 0, policy_version 751583 (0.0005) [2023-12-27 10:49:46,584][20381] Updated weights for policy 1, policy_version 751259 (0.0006) [2023-12-27 10:49:46,636][20381] Updated weights for policy 1, policy_version 751269 (0.0006) [2023-12-27 10:49:46,699][20381] Updated weights for policy 1, policy_version 751279 (0.0005) [2023-12-27 10:49:46,784][20347] Updated weights for policy 0, policy_version 751593 (0.0009) [2023-12-27 10:49:46,835][20347] Updated weights for policy 0, policy_version 751603 (0.0008) [2023-12-27 10:49:46,888][20347] Updated weights for policy 0, policy_version 751613 (0.0006) [2023-12-27 10:49:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 384794624. Throughput: 0: 10288.7, 1: 10317.8. Samples: 384808528. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:47,363][19170] Avg episode reward: [(0, '970.820'), (1, '706.990')] [2023-12-27 10:49:47,364][20381] Updated weights for policy 1, policy_version 751289 (0.0008) [2023-12-27 10:49:47,415][20381] Updated weights for policy 1, policy_version 751299 (0.0007) [2023-12-27 10:49:47,462][20381] Updated weights for policy 1, policy_version 751309 (0.0007) [2023-12-27 10:49:47,510][20347] Updated weights for policy 0, policy_version 751623 (0.0005) [2023-12-27 10:49:47,518][20381] Updated weights for policy 1, policy_version 751319 (0.0007) [2023-12-27 10:49:47,561][20347] Updated weights for policy 0, policy_version 751633 (0.0007) [2023-12-27 10:49:47,610][20347] Updated weights for policy 0, policy_version 751643 (0.0007) [2023-12-27 10:49:48,143][20381] Updated weights for policy 1, policy_version 751329 (0.0008) [2023-12-27 10:49:48,202][20381] Updated weights for policy 1, policy_version 751339 (0.0008) [2023-12-27 10:49:48,254][20347] Updated weights for policy 0, policy_version 751653 (0.0008) [2023-12-27 10:49:48,256][20381] Updated weights for policy 1, policy_version 751349 (0.0006) [2023-12-27 10:49:48,307][20347] Updated weights for policy 0, policy_version 751663 (0.0007) [2023-12-27 10:49:48,367][20347] Updated weights for policy 0, policy_version 751673 (0.0007) [2023-12-27 10:49:48,969][20381] Updated weights for policy 1, policy_version 751359 (0.0009) [2023-12-27 10:49:49,028][20381] Updated weights for policy 1, policy_version 751369 (0.0010) [2023-12-27 10:49:49,032][20347] Updated weights for policy 0, policy_version 751683 (0.0009) [2023-12-27 10:49:49,084][20381] Updated weights for policy 1, policy_version 751379 (0.0005) [2023-12-27 10:49:49,086][20347] Updated weights for policy 0, policy_version 751693 (0.0005) [2023-12-27 10:49:49,140][20347] Updated weights for policy 0, policy_version 751703 (0.0010) [2023-12-27 10:49:49,730][20381] Updated weights for policy 1, policy_version 751389 (0.0007) [2023-12-27 10:49:49,780][20347] Updated weights for policy 0, policy_version 751713 (0.0010) [2023-12-27 10:49:49,784][20381] Updated weights for policy 1, policy_version 751399 (0.0007) [2023-12-27 10:49:49,846][20347] Updated weights for policy 0, policy_version 751723 (0.0008) [2023-12-27 10:49:49,847][20381] Updated weights for policy 1, policy_version 751409 (0.0007) [2023-12-27 10:49:49,899][20347] Updated weights for policy 0, policy_version 751733 (0.0006) [2023-12-27 10:49:49,962][20347] Updated weights for policy 0, policy_version 751743 (0.0006) [2023-12-27 10:49:50,568][20381] Updated weights for policy 1, policy_version 751419 (0.0008) [2023-12-27 10:49:50,627][20381] Updated weights for policy 1, policy_version 751429 (0.0010) [2023-12-27 10:49:50,664][20347] Updated weights for policy 0, policy_version 751753 (0.0006) [2023-12-27 10:49:50,683][20381] Updated weights for policy 1, policy_version 751439 (0.0010) [2023-12-27 10:49:50,711][20347] Updated weights for policy 0, policy_version 751763 (0.0006) [2023-12-27 10:49:50,761][20347] Updated weights for policy 0, policy_version 751773 (0.0007) [2023-12-27 10:49:51,389][20381] Updated weights for policy 1, policy_version 751449 (0.0010) [2023-12-27 10:49:51,450][20381] Updated weights for policy 1, policy_version 751459 (0.0006) [2023-12-27 10:49:51,516][20381] Updated weights for policy 1, policy_version 751469 (0.0006) [2023-12-27 10:49:51,542][20347] Updated weights for policy 0, policy_version 751783 (0.0007) [2023-12-27 10:49:51,573][20381] Updated weights for policy 1, policy_version 751479 (0.0006) [2023-12-27 10:49:51,604][20347] Updated weights for policy 0, policy_version 751793 (0.0007) [2023-12-27 10:49:51,670][20347] Updated weights for policy 0, policy_version 751803 (0.0007) [2023-12-27 10:49:52,232][20381] Updated weights for policy 1, policy_version 751489 (0.0007) [2023-12-27 10:49:52,291][20381] Updated weights for policy 1, policy_version 751499 (0.0009) [2023-12-27 10:49:52,293][20347] Updated weights for policy 0, policy_version 751813 (0.0007) [2023-12-27 10:49:52,349][20381] Updated weights for policy 1, policy_version 751509 (0.0006) [2023-12-27 10:49:52,351][20347] Updated weights for policy 0, policy_version 751823 (0.0006) [2023-12-27 10:49:52,362][19170] Fps is (10 sec: 19660.2, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 384892928. Throughput: 0: 10289.3, 1: 10318.0. Samples: 384871216. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:52,363][19170] Avg episode reward: [(0, '970.810'), (1, '775.400')] [2023-12-27 10:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000751512_192413696.pth... [2023-12-27 10:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000750264_192094208.pth [2023-12-27 10:49:52,408][20347] Updated weights for policy 0, policy_version 751833 (0.0007) [2023-12-27 10:49:52,445][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000751840_192495616.pth... [2023-12-27 10:49:52,448][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000750624_192184320.pth [2023-12-27 10:49:53,046][20347] Updated weights for policy 0, policy_version 751843 (0.0007) [2023-12-27 10:49:53,079][20381] Updated weights for policy 1, policy_version 751519 (0.0007) [2023-12-27 10:49:53,101][20347] Updated weights for policy 0, policy_version 751853 (0.0006) [2023-12-27 10:49:53,107][20292] KL-divergence is very high: 130.6422 [2023-12-27 10:49:53,122][20292] KL-divergence is very high: 100.2103 [2023-12-27 10:49:53,125][20381] Updated weights for policy 1, policy_version 751529 (0.0007) [2023-12-27 10:49:53,156][20347] Updated weights for policy 0, policy_version 751863 (0.0007) [2023-12-27 10:49:53,171][20381] Updated weights for policy 1, policy_version 751539 (0.0007) [2023-12-27 10:49:53,837][20347] Updated weights for policy 0, policy_version 751873 (0.0006) [2023-12-27 10:49:53,867][20381] Updated weights for policy 1, policy_version 751549 (0.0006) [2023-12-27 10:49:53,885][20347] Updated weights for policy 0, policy_version 751883 (0.0006) [2023-12-27 10:49:53,912][20381] Updated weights for policy 1, policy_version 751559 (0.0007) [2023-12-27 10:49:53,933][20347] Updated weights for policy 0, policy_version 751893 (0.0006) [2023-12-27 10:49:53,957][20381] Updated weights for policy 1, policy_version 751569 (0.0006) [2023-12-27 10:49:53,977][20347] Updated weights for policy 0, policy_version 751903 (0.0005) [2023-12-27 10:49:54,578][20347] Updated weights for policy 0, policy_version 751913 (0.0007) [2023-12-27 10:49:54,602][20381] Updated weights for policy 1, policy_version 751579 (0.0005) [2023-12-27 10:49:54,634][20347] Updated weights for policy 0, policy_version 751923 (0.0007) [2023-12-27 10:49:54,652][20381] Updated weights for policy 1, policy_version 751589 (0.0005) [2023-12-27 10:49:54,683][20347] Updated weights for policy 0, policy_version 751933 (0.0008) [2023-12-27 10:49:54,708][20381] Updated weights for policy 1, policy_version 751599 (0.0007) [2023-12-27 10:49:55,369][20381] Updated weights for policy 1, policy_version 751609 (0.0005) [2023-12-27 10:49:55,415][20347] Updated weights for policy 0, policy_version 751943 (0.0007) [2023-12-27 10:49:55,429][20381] Updated weights for policy 1, policy_version 751619 (0.0006) [2023-12-27 10:49:55,476][20347] Updated weights for policy 0, policy_version 751953 (0.0006) [2023-12-27 10:49:55,486][20381] Updated weights for policy 1, policy_version 751629 (0.0008) [2023-12-27 10:49:55,534][20347] Updated weights for policy 0, policy_version 751963 (0.0006) [2023-12-27 10:49:55,540][20381] Updated weights for policy 1, policy_version 751639 (0.0011) [2023-12-27 10:49:56,153][20347] Updated weights for policy 0, policy_version 751973 (0.0007) [2023-12-27 10:49:56,188][20381] Updated weights for policy 1, policy_version 751649 (0.0006) [2023-12-27 10:49:56,201][20347] Updated weights for policy 0, policy_version 751983 (0.0007) [2023-12-27 10:49:56,247][20381] Updated weights for policy 1, policy_version 751659 (0.0009) [2023-12-27 10:49:56,251][20347] Updated weights for policy 0, policy_version 751993 (0.0009) [2023-12-27 10:49:56,298][20381] Updated weights for policy 1, policy_version 751669 (0.0008) [2023-12-27 10:49:56,953][20381] Updated weights for policy 1, policy_version 751679 (0.0006) [2023-12-27 10:49:56,964][20347] Updated weights for policy 0, policy_version 752003 (0.0006) [2023-12-27 10:49:57,001][20381] Updated weights for policy 1, policy_version 751689 (0.0005) [2023-12-27 10:49:57,014][20347] Updated weights for policy 0, policy_version 752013 (0.0007) [2023-12-27 10:49:57,051][20381] Updated weights for policy 1, policy_version 751699 (0.0005) [2023-12-27 10:49:57,067][20347] Updated weights for policy 0, policy_version 752023 (0.0008) [2023-12-27 10:49:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 385007616. Throughput: 0: 10343.1, 1: 10367.1. Samples: 384995532. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:49:57,363][19170] Avg episode reward: [(0, '970.940'), (1, '804.610')] [2023-12-27 10:49:57,692][20381] Updated weights for policy 1, policy_version 751709 (0.0008) [2023-12-27 10:49:57,736][20347] Updated weights for policy 0, policy_version 752033 (0.0006) [2023-12-27 10:49:57,749][20381] Updated weights for policy 1, policy_version 751719 (0.0008) [2023-12-27 10:49:57,789][20347] Updated weights for policy 0, policy_version 752043 (0.0005) [2023-12-27 10:49:57,801][20381] Updated weights for policy 1, policy_version 751729 (0.0009) [2023-12-27 10:49:57,838][20347] Updated weights for policy 0, policy_version 752053 (0.0008) [2023-12-27 10:49:57,884][20347] Updated weights for policy 0, policy_version 752063 (0.0007) [2023-12-27 10:49:58,511][20347] Updated weights for policy 0, policy_version 752073 (0.0008) [2023-12-27 10:49:58,532][20381] Updated weights for policy 1, policy_version 751739 (0.0009) [2023-12-27 10:49:58,582][20347] Updated weights for policy 0, policy_version 752083 (0.0008) [2023-12-27 10:49:58,598][20381] Updated weights for policy 1, policy_version 751749 (0.0007) [2023-12-27 10:49:58,650][20347] Updated weights for policy 0, policy_version 752093 (0.0008) [2023-12-27 10:49:58,667][20381] Updated weights for policy 1, policy_version 751759 (0.0007) [2023-12-27 10:49:59,370][20347] Updated weights for policy 0, policy_version 752103 (0.0007) [2023-12-27 10:49:59,422][20347] Updated weights for policy 0, policy_version 752113 (0.0008) [2023-12-27 10:49:59,473][20347] Updated weights for policy 0, policy_version 752123 (0.0008) [2023-12-27 10:49:59,480][20381] Updated weights for policy 1, policy_version 751769 (0.0007) [2023-12-27 10:49:59,541][20381] Updated weights for policy 1, policy_version 751779 (0.0005) [2023-12-27 10:49:59,596][20381] Updated weights for policy 1, policy_version 751789 (0.0005) [2023-12-27 10:49:59,653][20381] Updated weights for policy 1, policy_version 751799 (0.0005) [2023-12-27 10:50:00,215][20347] Updated weights for policy 0, policy_version 752133 (0.0009) [2023-12-27 10:50:00,276][20347] Updated weights for policy 0, policy_version 752143 (0.0010) [2023-12-27 10:50:00,324][20381] Updated weights for policy 1, policy_version 751809 (0.0006) [2023-12-27 10:50:00,330][20347] Updated weights for policy 0, policy_version 752153 (0.0010) [2023-12-27 10:50:00,381][20381] Updated weights for policy 1, policy_version 751819 (0.0006) [2023-12-27 10:50:00,441][20381] Updated weights for policy 1, policy_version 751829 (0.0008) [2023-12-27 10:50:00,931][20347] Updated weights for policy 0, policy_version 752163 (0.0010) [2023-12-27 10:50:00,976][20347] Updated weights for policy 0, policy_version 752173 (0.0010) [2023-12-27 10:50:01,025][20347] Updated weights for policy 0, policy_version 752183 (0.0010) [2023-12-27 10:50:01,150][20381] Updated weights for policy 1, policy_version 751839 (0.0008) [2023-12-27 10:50:01,200][20381] Updated weights for policy 1, policy_version 751849 (0.0006) [2023-12-27 10:50:01,260][20381] Updated weights for policy 1, policy_version 751859 (0.0008) [2023-12-27 10:50:01,796][20347] Updated weights for policy 0, policy_version 752193 (0.0009) [2023-12-27 10:50:01,850][20347] Updated weights for policy 0, policy_version 752203 (0.0008) [2023-12-27 10:50:01,903][20347] Updated weights for policy 0, policy_version 752213 (0.0007) [2023-12-27 10:50:01,930][20381] Updated weights for policy 1, policy_version 751869 (0.0009) [2023-12-27 10:50:01,962][20347] Updated weights for policy 0, policy_version 752223 (0.0006) [2023-12-27 10:50:01,980][20381] Updated weights for policy 1, policy_version 751879 (0.0011) [2023-12-27 10:50:02,026][20381] Updated weights for policy 1, policy_version 751889 (0.0010) [2023-12-27 10:50:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 385105920. Throughput: 0: 10293.7, 1: 10321.7. Samples: 385115632. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:02,363][19170] Avg episode reward: [(0, '922.430'), (1, '833.870')] [2023-12-27 10:50:02,691][20347] Updated weights for policy 0, policy_version 752233 (0.0008) [2023-12-27 10:50:02,692][20381] Updated weights for policy 1, policy_version 751899 (0.0010) [2023-12-27 10:50:02,743][20381] Updated weights for policy 1, policy_version 751909 (0.0011) [2023-12-27 10:50:02,744][20347] Updated weights for policy 0, policy_version 752243 (0.0005) [2023-12-27 10:50:02,800][20381] Updated weights for policy 1, policy_version 751919 (0.0011) [2023-12-27 10:50:02,805][20347] Updated weights for policy 0, policy_version 752253 (0.0006) [2023-12-27 10:50:03,430][20347] Updated weights for policy 0, policy_version 752263 (0.0005) [2023-12-27 10:50:03,484][20347] Updated weights for policy 0, policy_version 752273 (0.0005) [2023-12-27 10:50:03,497][20381] Updated weights for policy 1, policy_version 751929 (0.0011) [2023-12-27 10:50:03,539][20347] Updated weights for policy 0, policy_version 752283 (0.0005) [2023-12-27 10:50:03,541][20381] Updated weights for policy 1, policy_version 751939 (0.0008) [2023-12-27 10:50:03,591][20381] Updated weights for policy 1, policy_version 751949 (0.0005) [2023-12-27 10:50:03,643][20381] Updated weights for policy 1, policy_version 751959 (0.0007) [2023-12-27 10:50:04,129][20347] Updated weights for policy 0, policy_version 752293 (0.0008) [2023-12-27 10:50:04,184][20291] KL-divergence is very high: 122.3406 [2023-12-27 10:50:04,191][20347] Updated weights for policy 0, policy_version 752303 (0.0010) [2023-12-27 10:50:04,256][20347] Updated weights for policy 0, policy_version 752313 (0.0010) [2023-12-27 10:50:04,313][20381] Updated weights for policy 1, policy_version 751969 (0.0011) [2023-12-27 10:50:04,376][20381] Updated weights for policy 1, policy_version 751979 (0.0011) [2023-12-27 10:50:04,439][20381] Updated weights for policy 1, policy_version 751989 (0.0010) [2023-12-27 10:50:04,879][20291] KL-divergence is very high: 170.6533 [2023-12-27 10:50:04,889][20347] Updated weights for policy 0, policy_version 752323 (0.0008) [2023-12-27 10:50:04,949][20347] Updated weights for policy 0, policy_version 752333 (0.0006) [2023-12-27 10:50:05,006][20347] Updated weights for policy 0, policy_version 752343 (0.0007) [2023-12-27 10:50:05,156][20381] Updated weights for policy 1, policy_version 751999 (0.0010) [2023-12-27 10:50:05,211][20381] Updated weights for policy 1, policy_version 752009 (0.0009) [2023-12-27 10:50:05,256][20381] Updated weights for policy 1, policy_version 752019 (0.0010) [2023-12-27 10:50:05,571][20347] Updated weights for policy 0, policy_version 752353 (0.0007) [2023-12-27 10:50:05,623][20347] Updated weights for policy 0, policy_version 752363 (0.0005) [2023-12-27 10:50:05,681][20347] Updated weights for policy 0, policy_version 752373 (0.0005) [2023-12-27 10:50:05,730][20347] Updated weights for policy 0, policy_version 752383 (0.0006) [2023-12-27 10:50:05,913][20381] Updated weights for policy 1, policy_version 752029 (0.0009) [2023-12-27 10:50:05,960][20381] Updated weights for policy 1, policy_version 752039 (0.0010) [2023-12-27 10:50:06,005][20381] Updated weights for policy 1, policy_version 752049 (0.0010) [2023-12-27 10:50:06,312][20347] Updated weights for policy 0, policy_version 752393 (0.0010) [2023-12-27 10:50:06,375][20347] Updated weights for policy 0, policy_version 752403 (0.0010) [2023-12-27 10:50:06,427][20347] Updated weights for policy 0, policy_version 752413 (0.0010) [2023-12-27 10:50:06,725][20381] Updated weights for policy 1, policy_version 752059 (0.0008) [2023-12-27 10:50:06,791][20381] Updated weights for policy 1, policy_version 752069 (0.0009) [2023-12-27 10:50:06,850][20381] Updated weights for policy 1, policy_version 752079 (0.0007) [2023-12-27 10:50:07,091][20347] Updated weights for policy 0, policy_version 752423 (0.0009) [2023-12-27 10:50:07,150][20347] Updated weights for policy 0, policy_version 752433 (0.0010) [2023-12-27 10:50:07,196][20347] Updated weights for policy 0, policy_version 752443 (0.0010) [2023-12-27 10:50:07,363][19170] Fps is (10 sec: 20479.1, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 385212416. Throughput: 0: 10350.0, 1: 10357.6. Samples: 385178888. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:07,364][19170] Avg episode reward: [(0, '941.860'), (1, '824.140')] [2023-12-27 10:50:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000752088_192561152.pth... [2023-12-27 10:50:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000752448_192651264.pth... [2023-12-27 10:50:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000751232_192339968.pth [2023-12-27 10:50:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000750904_192258048.pth [2023-12-27 10:50:07,435][20381] Updated weights for policy 1, policy_version 752089 (0.0007) [2023-12-27 10:50:07,491][20381] Updated weights for policy 1, policy_version 752099 (0.0007) [2023-12-27 10:50:07,535][20381] Updated weights for policy 1, policy_version 752109 (0.0007) [2023-12-27 10:50:07,587][20381] Updated weights for policy 1, policy_version 752119 (0.0006) [2023-12-27 10:50:07,839][20347] Updated weights for policy 0, policy_version 752453 (0.0008) [2023-12-27 10:50:07,899][20347] Updated weights for policy 0, policy_version 752463 (0.0005) [2023-12-27 10:50:07,958][20347] Updated weights for policy 0, policy_version 752473 (0.0008) [2023-12-27 10:50:08,272][20381] Updated weights for policy 1, policy_version 752129 (0.0009) [2023-12-27 10:50:08,332][20381] Updated weights for policy 1, policy_version 752139 (0.0009) [2023-12-27 10:50:08,393][20381] Updated weights for policy 1, policy_version 752149 (0.0007) [2023-12-27 10:50:08,537][20347] Updated weights for policy 0, policy_version 752483 (0.0010) [2023-12-27 10:50:08,595][20347] Updated weights for policy 0, policy_version 752493 (0.0010) [2023-12-27 10:50:08,659][20347] Updated weights for policy 0, policy_version 752503 (0.0010) [2023-12-27 10:50:09,093][20381] Updated weights for policy 1, policy_version 752159 (0.0006) [2023-12-27 10:50:09,145][20381] Updated weights for policy 1, policy_version 752169 (0.0005) [2023-12-27 10:50:09,192][20381] Updated weights for policy 1, policy_version 752179 (0.0005) [2023-12-27 10:50:09,263][20347] Updated weights for policy 0, policy_version 752513 (0.0010) [2023-12-27 10:50:09,325][20347] Updated weights for policy 0, policy_version 752523 (0.0005) [2023-12-27 10:50:09,392][20347] Updated weights for policy 0, policy_version 752533 (0.0008) [2023-12-27 10:50:09,455][20347] Updated weights for policy 0, policy_version 752543 (0.0008) [2023-12-27 10:50:09,853][20381] Updated weights for policy 1, policy_version 752189 (0.0008) [2023-12-27 10:50:09,909][20381] Updated weights for policy 1, policy_version 752199 (0.0006) [2023-12-27 10:50:09,968][20381] Updated weights for policy 1, policy_version 752209 (0.0008) [2023-12-27 10:50:10,154][20291] KL-divergence is very high: 133.5037 [2023-12-27 10:50:10,178][20347] Updated weights for policy 0, policy_version 752553 (0.0008) [2023-12-27 10:50:10,240][20347] Updated weights for policy 0, policy_version 752563 (0.0008) [2023-12-27 10:50:10,251][20291] KL-divergence is very high: 273.2339 [2023-12-27 10:50:10,291][20347] Updated weights for policy 0, policy_version 752573 (0.0007) [2023-12-27 10:50:10,667][20381] Updated weights for policy 1, policy_version 752219 (0.0007) [2023-12-27 10:50:10,729][20381] Updated weights for policy 1, policy_version 752229 (0.0008) [2023-12-27 10:50:10,784][20381] Updated weights for policy 1, policy_version 752239 (0.0008) [2023-12-27 10:50:10,966][20291] KL-divergence is very high: 121.6578 [2023-12-27 10:50:10,993][20347] Updated weights for policy 0, policy_version 752583 (0.0008) [2023-12-27 10:50:11,049][20347] Updated weights for policy 0, policy_version 752593 (0.0009) [2023-12-27 10:50:11,106][20347] Updated weights for policy 0, policy_version 752603 (0.0008) [2023-12-27 10:50:11,552][20381] Updated weights for policy 1, policy_version 752249 (0.0008) [2023-12-27 10:50:11,609][20381] Updated weights for policy 1, policy_version 752259 (0.0008) [2023-12-27 10:50:11,675][20381] Updated weights for policy 1, policy_version 752269 (0.0008) [2023-12-27 10:50:11,731][20381] Updated weights for policy 1, policy_version 752279 (0.0008) [2023-12-27 10:50:11,918][20347] Updated weights for policy 0, policy_version 752613 (0.0009) [2023-12-27 10:50:11,980][20347] Updated weights for policy 0, policy_version 752623 (0.0008) [2023-12-27 10:50:12,038][20347] Updated weights for policy 0, policy_version 752633 (0.0008) [2023-12-27 10:50:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 385310720. Throughput: 0: 10421.3, 1: 10329.8. Samples: 385304036. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:12,363][19170] Avg episode reward: [(0, '893.030'), (1, '756.210')] [2023-12-27 10:50:12,496][20381] Updated weights for policy 1, policy_version 752289 (0.0007) [2023-12-27 10:50:12,557][20381] Updated weights for policy 1, policy_version 752299 (0.0009) [2023-12-27 10:50:12,619][20381] Updated weights for policy 1, policy_version 752309 (0.0008) [2023-12-27 10:50:12,703][20347] Updated weights for policy 0, policy_version 752643 (0.0007) [2023-12-27 10:50:12,763][20347] Updated weights for policy 0, policy_version 752653 (0.0005) [2023-12-27 10:50:12,830][20347] Updated weights for policy 0, policy_version 752663 (0.0006) [2023-12-27 10:50:13,277][20381] Updated weights for policy 1, policy_version 752319 (0.0010) [2023-12-27 10:50:13,334][20381] Updated weights for policy 1, policy_version 752329 (0.0011) [2023-12-27 10:50:13,390][20381] Updated weights for policy 1, policy_version 752339 (0.0011) [2023-12-27 10:50:13,449][20347] Updated weights for policy 0, policy_version 752673 (0.0008) [2023-12-27 10:50:13,499][20347] Updated weights for policy 0, policy_version 752683 (0.0007) [2023-12-27 10:50:13,544][20347] Updated weights for policy 0, policy_version 752693 (0.0007) [2023-12-27 10:50:13,587][20347] Updated weights for policy 0, policy_version 752703 (0.0007) [2023-12-27 10:50:14,085][20381] Updated weights for policy 1, policy_version 752349 (0.0010) [2023-12-27 10:50:14,134][20381] Updated weights for policy 1, policy_version 752359 (0.0008) [2023-12-27 10:50:14,179][20381] Updated weights for policy 1, policy_version 752369 (0.0008) [2023-12-27 10:50:14,285][20347] Updated weights for policy 0, policy_version 752713 (0.0007) [2023-12-27 10:50:14,334][20347] Updated weights for policy 0, policy_version 752723 (0.0007) [2023-12-27 10:50:14,382][20347] Updated weights for policy 0, policy_version 752733 (0.0006) [2023-12-27 10:50:14,885][20381] Updated weights for policy 1, policy_version 752379 (0.0008) [2023-12-27 10:50:14,945][20381] Updated weights for policy 1, policy_version 752389 (0.0011) [2023-12-27 10:50:15,005][20381] Updated weights for policy 1, policy_version 752399 (0.0011) [2023-12-27 10:50:15,040][20347] Updated weights for policy 0, policy_version 752743 (0.0006) [2023-12-27 10:50:15,095][20347] Updated weights for policy 0, policy_version 752753 (0.0006) [2023-12-27 10:50:15,148][20347] Updated weights for policy 0, policy_version 752763 (0.0008) [2023-12-27 10:50:15,676][20381] Updated weights for policy 1, policy_version 752409 (0.0010) [2023-12-27 10:50:15,726][20381] Updated weights for policy 1, policy_version 752419 (0.0005) [2023-12-27 10:50:15,773][20381] Updated weights for policy 1, policy_version 752429 (0.0005) [2023-12-27 10:50:15,793][20347] Updated weights for policy 0, policy_version 752773 (0.0008) [2023-12-27 10:50:15,817][20381] Updated weights for policy 1, policy_version 752439 (0.0008) [2023-12-27 10:50:15,845][20347] Updated weights for policy 0, policy_version 752783 (0.0006) [2023-12-27 10:50:15,903][20347] Updated weights for policy 0, policy_version 752793 (0.0008) [2023-12-27 10:50:16,496][20381] Updated weights for policy 1, policy_version 752449 (0.0010) [2023-12-27 10:50:16,540][20381] Updated weights for policy 1, policy_version 752459 (0.0009) [2023-12-27 10:50:16,585][20381] Updated weights for policy 1, policy_version 752469 (0.0007) [2023-12-27 10:50:16,596][20347] Updated weights for policy 0, policy_version 752803 (0.0006) [2023-12-27 10:50:16,640][20347] Updated weights for policy 0, policy_version 752813 (0.0006) [2023-12-27 10:50:16,689][20347] Updated weights for policy 0, policy_version 752823 (0.0005) [2023-12-27 10:50:17,159][20381] Updated weights for policy 1, policy_version 752479 (0.0005) [2023-12-27 10:50:17,212][20381] Updated weights for policy 1, policy_version 752489 (0.0005) [2023-12-27 10:50:17,246][20347] Updated weights for policy 0, policy_version 752833 (0.0005) [2023-12-27 10:50:17,265][20381] Updated weights for policy 1, policy_version 752499 (0.0006) [2023-12-27 10:50:17,299][20347] Updated weights for policy 0, policy_version 752843 (0.0007) [2023-12-27 10:50:17,354][20347] Updated weights for policy 0, policy_version 752853 (0.0008) [2023-12-27 10:50:17,362][19170] Fps is (10 sec: 20480.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 385417216. Throughput: 0: 10394.2, 1: 10201.9. Samples: 385427700. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:17,363][19170] Avg episode reward: [(0, '863.830'), (1, '759.130')] [2023-12-27 10:50:17,413][20347] Updated weights for policy 0, policy_version 752863 (0.0008) [2023-12-27 10:50:17,836][20381] Updated weights for policy 1, policy_version 752509 (0.0006) [2023-12-27 10:50:17,888][20381] Updated weights for policy 1, policy_version 752519 (0.0007) [2023-12-27 10:50:17,939][20381] Updated weights for policy 1, policy_version 752529 (0.0008) [2023-12-27 10:50:18,034][20347] Updated weights for policy 0, policy_version 752873 (0.0008) [2023-12-27 10:50:18,094][20347] Updated weights for policy 0, policy_version 752883 (0.0007) [2023-12-27 10:50:18,151][20347] Updated weights for policy 0, policy_version 752893 (0.0008) [2023-12-27 10:50:18,588][20381] Updated weights for policy 1, policy_version 752539 (0.0008) [2023-12-27 10:50:18,651][20381] Updated weights for policy 1, policy_version 752549 (0.0008) [2023-12-27 10:50:18,713][20381] Updated weights for policy 1, policy_version 752559 (0.0009) [2023-12-27 10:50:18,891][20347] Updated weights for policy 0, policy_version 752903 (0.0007) [2023-12-27 10:50:18,946][20347] Updated weights for policy 0, policy_version 752913 (0.0010) [2023-12-27 10:50:18,998][20347] Updated weights for policy 0, policy_version 752923 (0.0006) [2023-12-27 10:50:19,398][20381] Updated weights for policy 1, policy_version 752569 (0.0009) [2023-12-27 10:50:19,449][20381] Updated weights for policy 1, policy_version 752579 (0.0011) [2023-12-27 10:50:19,509][20381] Updated weights for policy 1, policy_version 752589 (0.0010) [2023-12-27 10:50:19,566][20381] Updated weights for policy 1, policy_version 752599 (0.0008) [2023-12-27 10:50:19,644][20347] Updated weights for policy 0, policy_version 752933 (0.0005) [2023-12-27 10:50:19,702][20347] Updated weights for policy 0, policy_version 752943 (0.0006) [2023-12-27 10:50:19,765][20347] Updated weights for policy 0, policy_version 752953 (0.0006) [2023-12-27 10:50:20,227][20381] Updated weights for policy 1, policy_version 752609 (0.0010) [2023-12-27 10:50:20,284][20381] Updated weights for policy 1, policy_version 752619 (0.0011) [2023-12-27 10:50:20,343][20381] Updated weights for policy 1, policy_version 752629 (0.0010) [2023-12-27 10:50:20,463][20347] Updated weights for policy 0, policy_version 752963 (0.0007) [2023-12-27 10:50:20,525][20347] Updated weights for policy 0, policy_version 752973 (0.0007) [2023-12-27 10:50:20,593][20347] Updated weights for policy 0, policy_version 752983 (0.0008) [2023-12-27 10:50:21,000][20381] Updated weights for policy 1, policy_version 752639 (0.0010) [2023-12-27 10:50:21,069][20381] Updated weights for policy 1, policy_version 752649 (0.0011) [2023-12-27 10:50:21,130][20381] Updated weights for policy 1, policy_version 752659 (0.0010) [2023-12-27 10:50:21,275][20347] Updated weights for policy 0, policy_version 752993 (0.0007) [2023-12-27 10:50:21,329][20347] Updated weights for policy 0, policy_version 753003 (0.0007) [2023-12-27 10:50:21,400][20347] Updated weights for policy 0, policy_version 753013 (0.0009) [2023-12-27 10:50:21,458][20347] Updated weights for policy 0, policy_version 753023 (0.0008) [2023-12-27 10:50:21,841][20381] Updated weights for policy 1, policy_version 752669 (0.0007) [2023-12-27 10:50:21,901][20381] Updated weights for policy 1, policy_version 752679 (0.0007) [2023-12-27 10:50:21,972][20381] Updated weights for policy 1, policy_version 752689 (0.0006) [2023-12-27 10:50:22,226][20347] Updated weights for policy 0, policy_version 753033 (0.0010) [2023-12-27 10:50:22,283][20347] Updated weights for policy 0, policy_version 753043 (0.0008) [2023-12-27 10:50:22,346][20347] Updated weights for policy 0, policy_version 753053 (0.0007) [2023-12-27 10:50:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 385515520. Throughput: 0: 10424.8, 1: 10163.2. Samples: 385491412. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:22,363][19170] Avg episode reward: [(0, '942.210'), (1, '760.020')] [2023-12-27 10:50:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000752696_192716800.pth... [2023-12-27 10:50:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000753056_192806912.pth... [2023-12-27 10:50:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000751512_192413696.pth [2023-12-27 10:50:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000751840_192495616.pth [2023-12-27 10:50:22,679][20381] Updated weights for policy 1, policy_version 752699 (0.0006) [2023-12-27 10:50:22,743][20381] Updated weights for policy 1, policy_version 752709 (0.0008) [2023-12-27 10:50:22,805][20381] Updated weights for policy 1, policy_version 752719 (0.0008) [2023-12-27 10:50:23,044][20347] Updated weights for policy 0, policy_version 753063 (0.0009) [2023-12-27 10:50:23,107][20347] Updated weights for policy 0, policy_version 753073 (0.0011) [2023-12-27 10:50:23,163][20347] Updated weights for policy 0, policy_version 753083 (0.0008) [2023-12-27 10:50:23,459][20381] Updated weights for policy 1, policy_version 752729 (0.0006) [2023-12-27 10:50:23,516][20381] Updated weights for policy 1, policy_version 752739 (0.0005) [2023-12-27 10:50:23,579][20381] Updated weights for policy 1, policy_version 752749 (0.0007) [2023-12-27 10:50:23,627][20381] Updated weights for policy 1, policy_version 752759 (0.0007) [2023-12-27 10:50:23,785][20347] Updated weights for policy 0, policy_version 753093 (0.0007) [2023-12-27 10:50:23,845][20347] Updated weights for policy 0, policy_version 753103 (0.0008) [2023-12-27 10:50:23,905][20347] Updated weights for policy 0, policy_version 753113 (0.0007) [2023-12-27 10:50:24,317][20381] Updated weights for policy 1, policy_version 752769 (0.0008) [2023-12-27 10:50:24,374][20381] Updated weights for policy 1, policy_version 752779 (0.0008) [2023-12-27 10:50:24,427][20381] Updated weights for policy 1, policy_version 752789 (0.0007) [2023-12-27 10:50:24,475][20347] Updated weights for policy 0, policy_version 753123 (0.0007) [2023-12-27 10:50:24,521][20347] Updated weights for policy 0, policy_version 753133 (0.0007) [2023-12-27 10:50:24,577][20347] Updated weights for policy 0, policy_version 753143 (0.0006) [2023-12-27 10:50:25,142][20381] Updated weights for policy 1, policy_version 752799 (0.0007) [2023-12-27 10:50:25,190][20381] Updated weights for policy 1, policy_version 752809 (0.0007) [2023-12-27 10:50:25,217][20347] Updated weights for policy 0, policy_version 753153 (0.0005) [2023-12-27 10:50:25,244][20381] Updated weights for policy 1, policy_version 752819 (0.0008) [2023-12-27 10:50:25,276][20347] Updated weights for policy 0, policy_version 753163 (0.0005) [2023-12-27 10:50:25,332][20347] Updated weights for policy 0, policy_version 753173 (0.0007) [2023-12-27 10:50:25,385][20347] Updated weights for policy 0, policy_version 753183 (0.0009) [2023-12-27 10:50:25,892][20381] Updated weights for policy 1, policy_version 752829 (0.0006) [2023-12-27 10:50:25,940][20381] Updated weights for policy 1, policy_version 752839 (0.0006) [2023-12-27 10:50:25,994][20381] Updated weights for policy 1, policy_version 752849 (0.0009) [2023-12-27 10:50:26,075][20347] Updated weights for policy 0, policy_version 753193 (0.0008) [2023-12-27 10:50:26,134][20347] Updated weights for policy 0, policy_version 753203 (0.0008) [2023-12-27 10:50:26,196][20347] Updated weights for policy 0, policy_version 753213 (0.0005) [2023-12-27 10:50:26,616][20381] Updated weights for policy 1, policy_version 752859 (0.0007) [2023-12-27 10:50:26,663][20381] Updated weights for policy 1, policy_version 752869 (0.0005) [2023-12-27 10:50:26,717][20381] Updated weights for policy 1, policy_version 752879 (0.0005) [2023-12-27 10:50:26,819][20347] Updated weights for policy 0, policy_version 753223 (0.0006) [2023-12-27 10:50:26,870][20347] Updated weights for policy 0, policy_version 753233 (0.0005) [2023-12-27 10:50:26,918][20347] Updated weights for policy 0, policy_version 753243 (0.0005) [2023-12-27 10:50:27,296][20381] Updated weights for policy 1, policy_version 752889 (0.0005) [2023-12-27 10:50:27,362][20381] Updated weights for policy 1, policy_version 752899 (0.0006) [2023-12-27 10:50:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 385622016. Throughput: 0: 10446.8, 1: 10208.9. Samples: 385614068. Policy #0 lag: (min: 31.0, avg: 31.7, max: 51.0) [2023-12-27 10:50:27,364][19170] Avg episode reward: [(0, '942.590'), (1, '816.160')] [2023-12-27 10:50:27,420][20381] Updated weights for policy 1, policy_version 752909 (0.0006) [2023-12-27 10:50:27,475][20381] Updated weights for policy 1, policy_version 752919 (0.0006) [2023-12-27 10:50:27,533][20347] Updated weights for policy 0, policy_version 753253 (0.0006) [2023-12-27 10:50:27,597][20347] Updated weights for policy 0, policy_version 753263 (0.0005) [2023-12-27 10:50:27,650][20347] Updated weights for policy 0, policy_version 753273 (0.0005) [2023-12-27 10:50:28,077][20381] Updated weights for policy 1, policy_version 752929 (0.0005) [2023-12-27 10:50:28,135][20381] Updated weights for policy 1, policy_version 752939 (0.0005) [2023-12-27 10:50:28,192][20381] Updated weights for policy 1, policy_version 752949 (0.0005) [2023-12-27 10:50:28,334][20347] Updated weights for policy 0, policy_version 753283 (0.0007) [2023-12-27 10:50:28,381][20347] Updated weights for policy 0, policy_version 753293 (0.0006) [2023-12-27 10:50:28,446][20347] Updated weights for policy 0, policy_version 753303 (0.0006) [2023-12-27 10:50:28,810][20381] Updated weights for policy 1, policy_version 752959 (0.0006) [2023-12-27 10:50:28,868][20381] Updated weights for policy 1, policy_version 752969 (0.0005) [2023-12-27 10:50:28,914][20381] Updated weights for policy 1, policy_version 752979 (0.0005) [2023-12-27 10:50:29,158][20347] Updated weights for policy 0, policy_version 753313 (0.0007) [2023-12-27 10:50:29,206][20347] Updated weights for policy 0, policy_version 753323 (0.0008) [2023-12-27 10:50:29,264][20347] Updated weights for policy 0, policy_version 753333 (0.0007) [2023-12-27 10:50:29,327][20347] Updated weights for policy 0, policy_version 753343 (0.0007) [2023-12-27 10:50:29,515][20381] Updated weights for policy 1, policy_version 752989 (0.0008) [2023-12-27 10:50:29,580][20381] Updated weights for policy 1, policy_version 752999 (0.0011) [2023-12-27 10:50:29,639][20381] Updated weights for policy 1, policy_version 753009 (0.0010) [2023-12-27 10:50:30,035][20347] Updated weights for policy 0, policy_version 753353 (0.0007) [2023-12-27 10:50:30,092][20347] Updated weights for policy 0, policy_version 753363 (0.0007) [2023-12-27 10:50:30,142][20347] Updated weights for policy 0, policy_version 753373 (0.0007) [2023-12-27 10:50:30,336][20381] Updated weights for policy 1, policy_version 753019 (0.0011) [2023-12-27 10:50:30,397][20381] Updated weights for policy 1, policy_version 753029 (0.0010) [2023-12-27 10:50:30,454][20381] Updated weights for policy 1, policy_version 753039 (0.0010) [2023-12-27 10:50:30,841][20347] Updated weights for policy 0, policy_version 753383 (0.0007) [2023-12-27 10:50:30,894][20347] Updated weights for policy 0, policy_version 753393 (0.0007) [2023-12-27 10:50:30,944][20347] Updated weights for policy 0, policy_version 753403 (0.0007) [2023-12-27 10:50:31,141][20381] Updated weights for policy 1, policy_version 753049 (0.0010) [2023-12-27 10:50:31,204][20381] Updated weights for policy 1, policy_version 753059 (0.0007) [2023-12-27 10:50:31,255][20381] Updated weights for policy 1, policy_version 753069 (0.0008) [2023-12-27 10:50:31,314][20381] Updated weights for policy 1, policy_version 753079 (0.0008) [2023-12-27 10:50:31,694][20347] Updated weights for policy 0, policy_version 753413 (0.0007) [2023-12-27 10:50:31,761][20347] Updated weights for policy 0, policy_version 753423 (0.0008) [2023-12-27 10:50:31,818][20347] Updated weights for policy 0, policy_version 753433 (0.0007) [2023-12-27 10:50:31,959][20381] Updated weights for policy 1, policy_version 753089 (0.0005) [2023-12-27 10:50:32,018][20381] Updated weights for policy 1, policy_version 753099 (0.0006) [2023-12-27 10:50:32,077][20381] Updated weights for policy 1, policy_version 753109 (0.0007) [2023-12-27 10:50:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 385728512. Throughput: 0: 10369.4, 1: 10313.7. Samples: 385739268. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:32,363][19170] Avg episode reward: [(0, '961.510'), (1, '824.170')] [2023-12-27 10:50:32,523][20347] Updated weights for policy 0, policy_version 753443 (0.0008) [2023-12-27 10:50:32,574][20347] Updated weights for policy 0, policy_version 753453 (0.0007) [2023-12-27 10:50:32,626][20347] Updated weights for policy 0, policy_version 753463 (0.0006) [2023-12-27 10:50:32,652][20381] Updated weights for policy 1, policy_version 753119 (0.0010) [2023-12-27 10:50:32,700][20381] Updated weights for policy 1, policy_version 753129 (0.0010) [2023-12-27 10:50:32,755][20381] Updated weights for policy 1, policy_version 753139 (0.0011) [2023-12-27 10:50:33,245][20347] Updated weights for policy 0, policy_version 753473 (0.0010) [2023-12-27 10:50:33,295][20347] Updated weights for policy 0, policy_version 753483 (0.0005) [2023-12-27 10:50:33,342][20347] Updated weights for policy 0, policy_version 753493 (0.0007) [2023-12-27 10:50:33,380][20381] Updated weights for policy 1, policy_version 753149 (0.0008) [2023-12-27 10:50:33,390][20347] Updated weights for policy 0, policy_version 753503 (0.0010) [2023-12-27 10:50:33,425][20381] Updated weights for policy 1, policy_version 753159 (0.0005) [2023-12-27 10:50:33,476][20381] Updated weights for policy 1, policy_version 753169 (0.0007) [2023-12-27 10:50:34,045][20347] Updated weights for policy 0, policy_version 753513 (0.0010) [2023-12-27 10:50:34,100][20347] Updated weights for policy 0, policy_version 753523 (0.0010) [2023-12-27 10:50:34,118][20381] Updated weights for policy 1, policy_version 753179 (0.0010) [2023-12-27 10:50:34,157][20347] Updated weights for policy 0, policy_version 753533 (0.0009) [2023-12-27 10:50:34,178][20381] Updated weights for policy 1, policy_version 753189 (0.0010) [2023-12-27 10:50:34,240][20381] Updated weights for policy 1, policy_version 753199 (0.0009) [2023-12-27 10:50:34,924][20347] Updated weights for policy 0, policy_version 753543 (0.0006) [2023-12-27 10:50:34,937][20291] KL-divergence is very high: 309.3005 [2023-12-27 10:50:34,980][20347] Updated weights for policy 0, policy_version 753553 (0.0005) [2023-12-27 10:50:34,999][20381] Updated weights for policy 1, policy_version 753209 (0.0009) [2023-12-27 10:50:35,029][20291] KL-divergence is very high: 106.6441 [2023-12-27 10:50:35,039][20347] Updated weights for policy 0, policy_version 753563 (0.0008) [2023-12-27 10:50:35,054][20381] Updated weights for policy 1, policy_version 753219 (0.0009) [2023-12-27 10:50:35,103][20381] Updated weights for policy 1, policy_version 753229 (0.0010) [2023-12-27 10:50:35,160][20381] Updated weights for policy 1, policy_version 753239 (0.0009) [2023-12-27 10:50:35,697][20347] Updated weights for policy 0, policy_version 753573 (0.0008) [2023-12-27 10:50:35,760][20347] Updated weights for policy 0, policy_version 753583 (0.0008) [2023-12-27 10:50:35,808][20347] Updated weights for policy 0, policy_version 753593 (0.0010) [2023-12-27 10:50:35,851][20381] Updated weights for policy 1, policy_version 753249 (0.0007) [2023-12-27 10:50:35,902][20381] Updated weights for policy 1, policy_version 753259 (0.0009) [2023-12-27 10:50:35,958][20381] Updated weights for policy 1, policy_version 753269 (0.0010) [2023-12-27 10:50:36,518][20347] Updated weights for policy 0, policy_version 753603 (0.0009) [2023-12-27 10:50:36,567][20291] KL-divergence is very high: 468.0360 [2023-12-27 10:50:36,572][20347] Updated weights for policy 0, policy_version 753613 (0.0007) [2023-12-27 10:50:36,617][20291] KL-divergence is very high: 381.6583 [2023-12-27 10:50:36,637][20381] Updated weights for policy 1, policy_version 753279 (0.0009) [2023-12-27 10:50:36,638][20347] Updated weights for policy 0, policy_version 753623 (0.0006) [2023-12-27 10:50:36,692][20381] Updated weights for policy 1, policy_version 753289 (0.0008) [2023-12-27 10:50:36,748][20381] Updated weights for policy 1, policy_version 753299 (0.0007) [2023-12-27 10:50:37,277][20347] Updated weights for policy 0, policy_version 753633 (0.0010) [2023-12-27 10:50:37,301][20291] KL-divergence is very high: 162.8220 [2023-12-27 10:50:37,337][20347] Updated weights for policy 0, policy_version 753643 (0.0007) [2023-12-27 10:50:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 385826816. Throughput: 0: 10353.6, 1: 10320.3. Samples: 385801540. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:37,363][19170] Avg episode reward: [(0, '931.770'), (1, '765.580')] [2023-12-27 10:50:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000753304_192872448.pth... [2023-12-27 10:50:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000752088_192561152.pth [2023-12-27 10:50:37,398][20347] Updated weights for policy 0, policy_version 753653 (0.0006) [2023-12-27 10:50:37,426][20381] Updated weights for policy 1, policy_version 753309 (0.0009) [2023-12-27 10:50:37,453][20347] Updated weights for policy 0, policy_version 753663 (0.0010) [2023-12-27 10:50:37,457][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000753664_192962560.pth... [2023-12-27 10:50:37,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000752448_192651264.pth [2023-12-27 10:50:37,484][20381] Updated weights for policy 1, policy_version 753319 (0.0011) [2023-12-27 10:50:37,541][20381] Updated weights for policy 1, policy_version 753329 (0.0011) [2023-12-27 10:50:38,147][20347] Updated weights for policy 0, policy_version 753673 (0.0008) [2023-12-27 10:50:38,200][20347] Updated weights for policy 0, policy_version 753683 (0.0009) [2023-12-27 10:50:38,245][20381] Updated weights for policy 1, policy_version 753339 (0.0011) [2023-12-27 10:50:38,250][20347] Updated weights for policy 0, policy_version 753693 (0.0006) [2023-12-27 10:50:38,294][20381] Updated weights for policy 1, policy_version 753349 (0.0011) [2023-12-27 10:50:38,357][20381] Updated weights for policy 1, policy_version 753359 (0.0010) [2023-12-27 10:50:38,948][20347] Updated weights for policy 0, policy_version 753703 (0.0007) [2023-12-27 10:50:38,998][20347] Updated weights for policy 0, policy_version 753713 (0.0007) [2023-12-27 10:50:39,048][20347] Updated weights for policy 0, policy_version 753723 (0.0007) [2023-12-27 10:50:39,076][20381] Updated weights for policy 1, policy_version 753369 (0.0011) [2023-12-27 10:50:39,132][20381] Updated weights for policy 1, policy_version 753379 (0.0010) [2023-12-27 10:50:39,187][20381] Updated weights for policy 1, policy_version 753389 (0.0010) [2023-12-27 10:50:39,253][20381] Updated weights for policy 1, policy_version 753399 (0.0011) [2023-12-27 10:50:39,808][20347] Updated weights for policy 0, policy_version 753733 (0.0006) [2023-12-27 10:50:39,873][20347] Updated weights for policy 0, policy_version 753743 (0.0010) [2023-12-27 10:50:39,933][20347] Updated weights for policy 0, policy_version 753753 (0.0009) [2023-12-27 10:50:40,054][20381] Updated weights for policy 1, policy_version 753409 (0.0011) [2023-12-27 10:50:40,113][20381] Updated weights for policy 1, policy_version 753419 (0.0010) [2023-12-27 10:50:40,170][20381] Updated weights for policy 1, policy_version 753429 (0.0009) [2023-12-27 10:50:40,599][20347] Updated weights for policy 0, policy_version 753763 (0.0009) [2023-12-27 10:50:40,664][20347] Updated weights for policy 0, policy_version 753773 (0.0007) [2023-12-27 10:50:40,726][20347] Updated weights for policy 0, policy_version 753783 (0.0006) [2023-12-27 10:50:40,876][20381] Updated weights for policy 1, policy_version 753439 (0.0009) [2023-12-27 10:50:40,942][20381] Updated weights for policy 1, policy_version 753449 (0.0008) [2023-12-27 10:50:41,009][20381] Updated weights for policy 1, policy_version 753459 (0.0008) [2023-12-27 10:50:41,405][20347] Updated weights for policy 0, policy_version 753793 (0.0007) [2023-12-27 10:50:41,467][20347] Updated weights for policy 0, policy_version 753803 (0.0007) [2023-12-27 10:50:41,526][20347] Updated weights for policy 0, policy_version 753813 (0.0008) [2023-12-27 10:50:41,586][20347] Updated weights for policy 0, policy_version 753823 (0.0008) [2023-12-27 10:50:41,757][20381] Updated weights for policy 1, policy_version 753469 (0.0008) [2023-12-27 10:50:41,815][20381] Updated weights for policy 1, policy_version 753479 (0.0008) [2023-12-27 10:50:41,882][20381] Updated weights for policy 1, policy_version 753489 (0.0007) [2023-12-27 10:50:42,350][20347] Updated weights for policy 0, policy_version 753833 (0.0007) [2023-12-27 10:50:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20479.9, 300 sec: 20521.7). Total num frames: 385925120. Throughput: 0: 10309.4, 1: 10256.2. Samples: 385920984. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:42,363][19170] Avg episode reward: [(0, '931.740'), (1, '765.460')] [2023-12-27 10:50:42,408][20347] Updated weights for policy 0, policy_version 753843 (0.0008) [2023-12-27 10:50:42,461][20347] Updated weights for policy 0, policy_version 753853 (0.0008) [2023-12-27 10:50:42,527][20381] Updated weights for policy 1, policy_version 753499 (0.0008) [2023-12-27 10:50:42,582][20381] Updated weights for policy 1, policy_version 753509 (0.0007) [2023-12-27 10:50:42,632][20381] Updated weights for policy 1, policy_version 753519 (0.0006) [2023-12-27 10:50:43,197][20347] Updated weights for policy 0, policy_version 753863 (0.0006) [2023-12-27 10:50:43,233][20381] Updated weights for policy 1, policy_version 753529 (0.0008) [2023-12-27 10:50:43,256][20347] Updated weights for policy 0, policy_version 753873 (0.0005) [2023-12-27 10:50:43,289][20381] Updated weights for policy 1, policy_version 753539 (0.0005) [2023-12-27 10:50:43,306][20347] Updated weights for policy 0, policy_version 753883 (0.0007) [2023-12-27 10:50:43,339][20381] Updated weights for policy 1, policy_version 753549 (0.0005) [2023-12-27 10:50:43,391][20381] Updated weights for policy 1, policy_version 753559 (0.0007) [2023-12-27 10:50:43,933][20347] Updated weights for policy 0, policy_version 753893 (0.0006) [2023-12-27 10:50:44,002][20347] Updated weights for policy 0, policy_version 753903 (0.0007) [2023-12-27 10:50:44,039][20381] Updated weights for policy 1, policy_version 753569 (0.0006) [2023-12-27 10:50:44,062][20347] Updated weights for policy 0, policy_version 753913 (0.0005) [2023-12-27 10:50:44,090][20381] Updated weights for policy 1, policy_version 753579 (0.0007) [2023-12-27 10:50:44,143][20381] Updated weights for policy 1, policy_version 753589 (0.0006) [2023-12-27 10:50:44,715][20381] Updated weights for policy 1, policy_version 753599 (0.0005) [2023-12-27 10:50:44,735][20347] Updated weights for policy 0, policy_version 753923 (0.0006) [2023-12-27 10:50:44,765][20381] Updated weights for policy 1, policy_version 753609 (0.0007) [2023-12-27 10:50:44,799][20347] Updated weights for policy 0, policy_version 753933 (0.0006) [2023-12-27 10:50:44,828][20381] Updated weights for policy 1, policy_version 753619 (0.0007) [2023-12-27 10:50:44,859][20347] Updated weights for policy 0, policy_version 753943 (0.0007) [2023-12-27 10:50:45,518][20381] Updated weights for policy 1, policy_version 753629 (0.0006) [2023-12-27 10:50:45,544][20347] Updated weights for policy 0, policy_version 753953 (0.0007) [2023-12-27 10:50:45,565][20381] Updated weights for policy 1, policy_version 753639 (0.0006) [2023-12-27 10:50:45,600][20347] Updated weights for policy 0, policy_version 753963 (0.0007) [2023-12-27 10:50:45,610][20381] Updated weights for policy 1, policy_version 753649 (0.0005) [2023-12-27 10:50:45,653][20347] Updated weights for policy 0, policy_version 753973 (0.0005) [2023-12-27 10:50:45,710][20347] Updated weights for policy 0, policy_version 753983 (0.0005) [2023-12-27 10:50:46,302][20347] Updated weights for policy 0, policy_version 753993 (0.0006) [2023-12-27 10:50:46,358][20381] Updated weights for policy 1, policy_version 753659 (0.0006) [2023-12-27 10:50:46,359][20347] Updated weights for policy 0, policy_version 754003 (0.0005) [2023-12-27 10:50:46,410][20381] Updated weights for policy 1, policy_version 753669 (0.0005) [2023-12-27 10:50:46,414][20347] Updated weights for policy 0, policy_version 754013 (0.0006) [2023-12-27 10:50:46,474][20381] Updated weights for policy 1, policy_version 753679 (0.0005) [2023-12-27 10:50:47,025][20347] Updated weights for policy 0, policy_version 754023 (0.0007) [2023-12-27 10:50:47,071][20347] Updated weights for policy 0, policy_version 754033 (0.0008) [2023-12-27 10:50:47,111][20381] Updated weights for policy 1, policy_version 753689 (0.0006) [2023-12-27 10:50:47,124][20347] Updated weights for policy 0, policy_version 754043 (0.0007) [2023-12-27 10:50:47,160][20381] Updated weights for policy 1, policy_version 753699 (0.0006) [2023-12-27 10:50:47,217][20381] Updated weights for policy 1, policy_version 753709 (0.0008) [2023-12-27 10:50:47,270][20381] Updated weights for policy 1, policy_version 753719 (0.0008) [2023-12-27 10:50:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 386039808. Throughput: 0: 10321.2, 1: 10338.1. Samples: 386045300. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:47,363][19170] Avg episode reward: [(0, '970.960'), (1, '843.590')] [2023-12-27 10:50:47,744][20347] Updated weights for policy 0, policy_version 754053 (0.0005) [2023-12-27 10:50:47,796][20347] Updated weights for policy 0, policy_version 754063 (0.0005) [2023-12-27 10:50:47,842][20347] Updated weights for policy 0, policy_version 754073 (0.0005) [2023-12-27 10:50:48,035][20381] Updated weights for policy 1, policy_version 753729 (0.0006) [2023-12-27 10:50:48,089][20381] Updated weights for policy 1, policy_version 753739 (0.0005) [2023-12-27 10:50:48,142][20381] Updated weights for policy 1, policy_version 753749 (0.0005) [2023-12-27 10:50:48,414][20347] Updated weights for policy 0, policy_version 754083 (0.0006) [2023-12-27 10:50:48,473][20347] Updated weights for policy 0, policy_version 754093 (0.0006) [2023-12-27 10:50:48,524][20347] Updated weights for policy 0, policy_version 754103 (0.0008) [2023-12-27 10:50:48,876][20381] Updated weights for policy 1, policy_version 753759 (0.0005) [2023-12-27 10:50:48,926][20381] Updated weights for policy 1, policy_version 753769 (0.0005) [2023-12-27 10:50:48,977][20381] Updated weights for policy 1, policy_version 753779 (0.0005) [2023-12-27 10:50:49,094][20347] Updated weights for policy 0, policy_version 754113 (0.0007) [2023-12-27 10:50:49,161][20347] Updated weights for policy 0, policy_version 754123 (0.0005) [2023-12-27 10:50:49,233][20347] Updated weights for policy 0, policy_version 754133 (0.0007) [2023-12-27 10:50:49,297][20347] Updated weights for policy 0, policy_version 754143 (0.0008) [2023-12-27 10:50:49,619][20381] Updated weights for policy 1, policy_version 753789 (0.0006) [2023-12-27 10:50:49,669][20381] Updated weights for policy 1, policy_version 753799 (0.0008) [2023-12-27 10:50:49,720][20381] Updated weights for policy 1, policy_version 753809 (0.0008) [2023-12-27 10:50:49,976][20347] Updated weights for policy 0, policy_version 754153 (0.0007) [2023-12-27 10:50:50,035][20347] Updated weights for policy 0, policy_version 754163 (0.0008) [2023-12-27 10:50:50,095][20347] Updated weights for policy 0, policy_version 754173 (0.0007) [2023-12-27 10:50:50,419][20381] Updated weights for policy 1, policy_version 753819 (0.0008) [2023-12-27 10:50:50,472][20381] Updated weights for policy 1, policy_version 753829 (0.0007) [2023-12-27 10:50:50,530][20381] Updated weights for policy 1, policy_version 753839 (0.0005) [2023-12-27 10:50:50,825][20347] Updated weights for policy 0, policy_version 754183 (0.0006) [2023-12-27 10:50:50,888][20347] Updated weights for policy 0, policy_version 754193 (0.0007) [2023-12-27 10:50:50,946][20347] Updated weights for policy 0, policy_version 754203 (0.0008) [2023-12-27 10:50:51,231][20381] Updated weights for policy 1, policy_version 753849 (0.0007) [2023-12-27 10:50:51,292][20381] Updated weights for policy 1, policy_version 753859 (0.0006) [2023-12-27 10:50:51,355][20381] Updated weights for policy 1, policy_version 753869 (0.0008) [2023-12-27 10:50:51,415][20381] Updated weights for policy 1, policy_version 753879 (0.0009) [2023-12-27 10:50:51,611][20347] Updated weights for policy 0, policy_version 754213 (0.0009) [2023-12-27 10:50:51,676][20347] Updated weights for policy 0, policy_version 754223 (0.0007) [2023-12-27 10:50:51,743][20347] Updated weights for policy 0, policy_version 754233 (0.0008) [2023-12-27 10:50:52,070][20381] Updated weights for policy 1, policy_version 753889 (0.0006) [2023-12-27 10:50:52,120][20381] Updated weights for policy 1, policy_version 753899 (0.0009) [2023-12-27 10:50:52,176][20381] Updated weights for policy 1, policy_version 753909 (0.0008) [2023-12-27 10:50:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 386138112. Throughput: 0: 10342.2, 1: 10320.1. Samples: 386108684. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:52,363][19170] Avg episode reward: [(0, '980.540'), (1, '794.780')] [2023-12-27 10:50:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000753912_193028096.pth... [2023-12-27 10:50:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000754240_193110016.pth... [2023-12-27 10:50:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000753056_192806912.pth [2023-12-27 10:50:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000752696_192716800.pth [2023-12-27 10:50:52,411][20347] Updated weights for policy 0, policy_version 754243 (0.0008) [2023-12-27 10:50:52,481][20347] Updated weights for policy 0, policy_version 754253 (0.0008) [2023-12-27 10:50:52,547][20347] Updated weights for policy 0, policy_version 754263 (0.0008) [2023-12-27 10:50:52,868][20381] Updated weights for policy 1, policy_version 753919 (0.0008) [2023-12-27 10:50:52,924][20381] Updated weights for policy 1, policy_version 753929 (0.0008) [2023-12-27 10:50:52,981][20381] Updated weights for policy 1, policy_version 753939 (0.0008) [2023-12-27 10:50:53,196][20347] Updated weights for policy 0, policy_version 754273 (0.0008) [2023-12-27 10:50:53,254][20347] Updated weights for policy 0, policy_version 754283 (0.0006) [2023-12-27 10:50:53,310][20347] Updated weights for policy 0, policy_version 754293 (0.0008) [2023-12-27 10:50:53,365][20347] Updated weights for policy 0, policy_version 754303 (0.0008) [2023-12-27 10:50:53,704][20381] Updated weights for policy 1, policy_version 753949 (0.0009) [2023-12-27 10:50:53,763][20381] Updated weights for policy 1, policy_version 753959 (0.0009) [2023-12-27 10:50:53,818][20381] Updated weights for policy 1, policy_version 753969 (0.0008) [2023-12-27 10:50:53,955][20347] Updated weights for policy 0, policy_version 754313 (0.0005) [2023-12-27 10:50:54,006][20347] Updated weights for policy 0, policy_version 754323 (0.0007) [2023-12-27 10:50:54,054][20347] Updated weights for policy 0, policy_version 754333 (0.0007) [2023-12-27 10:50:54,559][20381] Updated weights for policy 1, policy_version 753979 (0.0008) [2023-12-27 10:50:54,613][20381] Updated weights for policy 1, policy_version 753989 (0.0010) [2023-12-27 10:50:54,662][20347] Updated weights for policy 0, policy_version 754343 (0.0007) [2023-12-27 10:50:54,664][20381] Updated weights for policy 1, policy_version 753999 (0.0010) [2023-12-27 10:50:54,706][20347] Updated weights for policy 0, policy_version 754353 (0.0005) [2023-12-27 10:50:54,758][20347] Updated weights for policy 0, policy_version 754363 (0.0007) [2023-12-27 10:50:55,353][20381] Updated weights for policy 1, policy_version 754009 (0.0010) [2023-12-27 10:50:55,408][20381] Updated weights for policy 1, policy_version 754019 (0.0011) [2023-12-27 10:50:55,464][20381] Updated weights for policy 1, policy_version 754029 (0.0011) [2023-12-27 10:50:55,466][20347] Updated weights for policy 0, policy_version 754373 (0.0006) [2023-12-27 10:50:55,514][20381] Updated weights for policy 1, policy_version 754039 (0.0011) [2023-12-27 10:50:55,515][20347] Updated weights for policy 0, policy_version 754383 (0.0006) [2023-12-27 10:50:55,560][20347] Updated weights for policy 0, policy_version 754393 (0.0007) [2023-12-27 10:50:56,165][20381] Updated weights for policy 1, policy_version 754049 (0.0006) [2023-12-27 10:50:56,223][20381] Updated weights for policy 1, policy_version 754059 (0.0005) [2023-12-27 10:50:56,278][20381] Updated weights for policy 1, policy_version 754069 (0.0006) [2023-12-27 10:50:56,318][20347] Updated weights for policy 0, policy_version 754403 (0.0008) [2023-12-27 10:50:56,376][20347] Updated weights for policy 0, policy_version 754413 (0.0007) [2023-12-27 10:50:56,433][20347] Updated weights for policy 0, policy_version 754423 (0.0009) [2023-12-27 10:50:56,800][20381] Updated weights for policy 1, policy_version 754079 (0.0007) [2023-12-27 10:50:56,850][20381] Updated weights for policy 1, policy_version 754089 (0.0005) [2023-12-27 10:50:56,893][20381] Updated weights for policy 1, policy_version 754099 (0.0005) [2023-12-27 10:50:57,075][20347] Updated weights for policy 0, policy_version 754433 (0.0009) [2023-12-27 10:50:57,138][20347] Updated weights for policy 0, policy_version 754443 (0.0005) [2023-12-27 10:50:57,204][20347] Updated weights for policy 0, policy_version 754453 (0.0005) [2023-12-27 10:50:57,267][20347] Updated weights for policy 0, policy_version 754463 (0.0005) [2023-12-27 10:50:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 386244608. Throughput: 0: 10303.9, 1: 10297.5. Samples: 386231100. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:50:57,363][19170] Avg episode reward: [(0, '951.340'), (1, '804.600')] [2023-12-27 10:50:57,433][20381] Updated weights for policy 1, policy_version 754109 (0.0008) [2023-12-27 10:50:57,484][20381] Updated weights for policy 1, policy_version 754119 (0.0010) [2023-12-27 10:50:57,541][20381] Updated weights for policy 1, policy_version 754129 (0.0010) [2023-12-27 10:50:57,875][20347] Updated weights for policy 0, policy_version 754473 (0.0007) [2023-12-27 10:50:57,926][20347] Updated weights for policy 0, policy_version 754483 (0.0007) [2023-12-27 10:50:57,974][20347] Updated weights for policy 0, policy_version 754493 (0.0007) [2023-12-27 10:50:58,197][20381] Updated weights for policy 1, policy_version 754139 (0.0007) [2023-12-27 10:50:58,260][20381] Updated weights for policy 1, policy_version 754149 (0.0008) [2023-12-27 10:50:58,309][20381] Updated weights for policy 1, policy_version 754159 (0.0008) [2023-12-27 10:50:58,831][20347] Updated weights for policy 0, policy_version 754503 (0.0008) [2023-12-27 10:50:58,905][20347] Updated weights for policy 0, policy_version 754513 (0.0008) [2023-12-27 10:50:58,960][20347] Updated weights for policy 0, policy_version 754523 (0.0007) [2023-12-27 10:50:59,102][20381] Updated weights for policy 1, policy_version 754169 (0.0007) [2023-12-27 10:50:59,163][20381] Updated weights for policy 1, policy_version 754179 (0.0007) [2023-12-27 10:50:59,227][20381] Updated weights for policy 1, policy_version 754189 (0.0006) [2023-12-27 10:50:59,293][20381] Updated weights for policy 1, policy_version 754199 (0.0006) [2023-12-27 10:50:59,705][20347] Updated weights for policy 0, policy_version 754533 (0.0007) [2023-12-27 10:50:59,761][20347] Updated weights for policy 0, policy_version 754543 (0.0007) [2023-12-27 10:50:59,818][20347] Updated weights for policy 0, policy_version 754553 (0.0008) [2023-12-27 10:50:59,910][20381] Updated weights for policy 1, policy_version 754209 (0.0008) [2023-12-27 10:50:59,968][20381] Updated weights for policy 1, policy_version 754219 (0.0007) [2023-12-27 10:51:00,024][20381] Updated weights for policy 1, policy_version 754229 (0.0008) [2023-12-27 10:51:00,509][20347] Updated weights for policy 0, policy_version 754563 (0.0007) [2023-12-27 10:51:00,565][20347] Updated weights for policy 0, policy_version 754573 (0.0007) [2023-12-27 10:51:00,625][20347] Updated weights for policy 0, policy_version 754583 (0.0008) [2023-12-27 10:51:00,722][20381] Updated weights for policy 1, policy_version 754239 (0.0010) [2023-12-27 10:51:00,768][20381] Updated weights for policy 1, policy_version 754249 (0.0010) [2023-12-27 10:51:00,813][20381] Updated weights for policy 1, policy_version 754259 (0.0010) [2023-12-27 10:51:01,336][20347] Updated weights for policy 0, policy_version 754593 (0.0008) [2023-12-27 10:51:01,401][20347] Updated weights for policy 0, policy_version 754603 (0.0008) [2023-12-27 10:51:01,459][20347] Updated weights for policy 0, policy_version 754613 (0.0007) [2023-12-27 10:51:01,494][20381] Updated weights for policy 1, policy_version 754269 (0.0010) [2023-12-27 10:51:01,507][20347] Updated weights for policy 0, policy_version 754623 (0.0005) [2023-12-27 10:51:01,553][20381] Updated weights for policy 1, policy_version 754279 (0.0011) [2023-12-27 10:51:01,614][20381] Updated weights for policy 1, policy_version 754289 (0.0011) [2023-12-27 10:51:02,257][20347] Updated weights for policy 0, policy_version 754633 (0.0007) [2023-12-27 10:51:02,269][20381] Updated weights for policy 1, policy_version 754299 (0.0010) [2023-12-27 10:51:02,311][20347] Updated weights for policy 0, policy_version 754643 (0.0007) [2023-12-27 10:51:02,326][20381] Updated weights for policy 1, policy_version 754309 (0.0007) [2023-12-27 10:51:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 386334720. Throughput: 0: 10215.5, 1: 10350.6. Samples: 386353172. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:02,363][19170] Avg episode reward: [(0, '941.560'), (1, '804.550')] [2023-12-27 10:51:02,368][20347] Updated weights for policy 0, policy_version 754653 (0.0006) [2023-12-27 10:51:02,389][20381] Updated weights for policy 1, policy_version 754319 (0.0007) [2023-12-27 10:51:03,036][20381] Updated weights for policy 1, policy_version 754329 (0.0008) [2023-12-27 10:51:03,077][20347] Updated weights for policy 0, policy_version 754663 (0.0006) [2023-12-27 10:51:03,086][20381] Updated weights for policy 1, policy_version 754339 (0.0006) [2023-12-27 10:51:03,130][20347] Updated weights for policy 0, policy_version 754673 (0.0005) [2023-12-27 10:51:03,140][20381] Updated weights for policy 1, policy_version 754349 (0.0006) [2023-12-27 10:51:03,180][20347] Updated weights for policy 0, policy_version 754683 (0.0005) [2023-12-27 10:51:03,196][20381] Updated weights for policy 1, policy_version 754359 (0.0007) [2023-12-27 10:51:03,784][20347] Updated weights for policy 0, policy_version 754693 (0.0007) [2023-12-27 10:51:03,843][20347] Updated weights for policy 0, policy_version 754703 (0.0006) [2023-12-27 10:51:03,905][20347] Updated weights for policy 0, policy_version 754713 (0.0006) [2023-12-27 10:51:03,951][20381] Updated weights for policy 1, policy_version 754369 (0.0006) [2023-12-27 10:51:04,008][20381] Updated weights for policy 1, policy_version 754379 (0.0007) [2023-12-27 10:51:04,060][20381] Updated weights for policy 1, policy_version 754389 (0.0007) [2023-12-27 10:51:04,600][20347] Updated weights for policy 0, policy_version 754723 (0.0007) [2023-12-27 10:51:04,657][20347] Updated weights for policy 0, policy_version 754733 (0.0008) [2023-12-27 10:51:04,712][20347] Updated weights for policy 0, policy_version 754743 (0.0007) [2023-12-27 10:51:04,787][20381] Updated weights for policy 1, policy_version 754399 (0.0007) [2023-12-27 10:51:04,845][20381] Updated weights for policy 1, policy_version 754409 (0.0008) [2023-12-27 10:51:04,895][20381] Updated weights for policy 1, policy_version 754419 (0.0007) [2023-12-27 10:51:05,425][20347] Updated weights for policy 0, policy_version 754753 (0.0007) [2023-12-27 10:51:05,486][20347] Updated weights for policy 0, policy_version 754763 (0.0008) [2023-12-27 10:51:05,508][20381] Updated weights for policy 1, policy_version 754429 (0.0006) [2023-12-27 10:51:05,533][20347] Updated weights for policy 0, policy_version 754773 (0.0009) [2023-12-27 10:51:05,558][20381] Updated weights for policy 1, policy_version 754439 (0.0006) [2023-12-27 10:51:05,584][20347] Updated weights for policy 0, policy_version 754783 (0.0010) [2023-12-27 10:51:05,614][20381] Updated weights for policy 1, policy_version 754449 (0.0010) [2023-12-27 10:51:06,177][20381] Updated weights for policy 1, policy_version 754459 (0.0010) [2023-12-27 10:51:06,238][20381] Updated weights for policy 1, policy_version 754469 (0.0011) [2023-12-27 10:51:06,300][20381] Updated weights for policy 1, policy_version 754479 (0.0011) [2023-12-27 10:51:06,327][20347] Updated weights for policy 0, policy_version 754793 (0.0010) [2023-12-27 10:51:06,386][20347] Updated weights for policy 0, policy_version 754803 (0.0010) [2023-12-27 10:51:06,445][20347] Updated weights for policy 0, policy_version 754813 (0.0010) [2023-12-27 10:51:07,006][20381] Updated weights for policy 1, policy_version 754489 (0.0011) [2023-12-27 10:51:07,036][20347] Updated weights for policy 0, policy_version 754823 (0.0005) [2023-12-27 10:51:07,062][20381] Updated weights for policy 1, policy_version 754499 (0.0011) [2023-12-27 10:51:07,095][20347] Updated weights for policy 0, policy_version 754833 (0.0005) [2023-12-27 10:51:07,117][20381] Updated weights for policy 1, policy_version 754509 (0.0011) [2023-12-27 10:51:07,155][20347] Updated weights for policy 0, policy_version 754843 (0.0005) [2023-12-27 10:51:07,170][20381] Updated weights for policy 1, policy_version 754519 (0.0010) [2023-12-27 10:51:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.6, 300 sec: 20578.0). Total num frames: 386449408. Throughput: 0: 10192.7, 1: 10292.9. Samples: 386413268. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:07,364][19170] Avg episode reward: [(0, '970.670'), (1, '843.580')] [2023-12-27 10:51:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000754848_193265664.pth... [2023-12-27 10:51:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000754520_193183744.pth... [2023-12-27 10:51:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000753664_192962560.pth [2023-12-27 10:51:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000753304_192872448.pth [2023-12-27 10:51:07,732][20347] Updated weights for policy 0, policy_version 754853 (0.0006) [2023-12-27 10:51:07,792][20347] Updated weights for policy 0, policy_version 754863 (0.0005) [2023-12-27 10:51:07,811][20381] Updated weights for policy 1, policy_version 754529 (0.0010) [2023-12-27 10:51:07,849][20347] Updated weights for policy 0, policy_version 754873 (0.0005) [2023-12-27 10:51:07,856][20381] Updated weights for policy 1, policy_version 754539 (0.0010) [2023-12-27 10:51:07,903][20381] Updated weights for policy 1, policy_version 754549 (0.0005) [2023-12-27 10:51:08,442][20347] Updated weights for policy 0, policy_version 754883 (0.0006) [2023-12-27 10:51:08,498][20347] Updated weights for policy 0, policy_version 754893 (0.0008) [2023-12-27 10:51:08,553][20347] Updated weights for policy 0, policy_version 754903 (0.0008) [2023-12-27 10:51:08,625][20381] Updated weights for policy 1, policy_version 754559 (0.0006) [2023-12-27 10:51:08,691][20381] Updated weights for policy 1, policy_version 754569 (0.0006) [2023-12-27 10:51:08,753][20381] Updated weights for policy 1, policy_version 754579 (0.0007) [2023-12-27 10:51:09,171][20347] Updated weights for policy 0, policy_version 754913 (0.0008) [2023-12-27 10:51:09,226][20347] Updated weights for policy 0, policy_version 754923 (0.0007) [2023-12-27 10:51:09,292][20347] Updated weights for policy 0, policy_version 754933 (0.0008) [2023-12-27 10:51:09,353][20347] Updated weights for policy 0, policy_version 754943 (0.0008) [2023-12-27 10:51:09,363][20381] Updated weights for policy 1, policy_version 754589 (0.0008) [2023-12-27 10:51:09,431][20381] Updated weights for policy 1, policy_version 754599 (0.0008) [2023-12-27 10:51:09,495][20381] Updated weights for policy 1, policy_version 754609 (0.0007) [2023-12-27 10:51:10,113][20381] Updated weights for policy 1, policy_version 754619 (0.0007) [2023-12-27 10:51:10,140][20347] Updated weights for policy 0, policy_version 754953 (0.0009) [2023-12-27 10:51:10,170][20381] Updated weights for policy 1, policy_version 754629 (0.0009) [2023-12-27 10:51:10,189][20347] Updated weights for policy 0, policy_version 754963 (0.0007) [2023-12-27 10:51:10,226][20381] Updated weights for policy 1, policy_version 754639 (0.0009) [2023-12-27 10:51:10,245][20347] Updated weights for policy 0, policy_version 754973 (0.0006) [2023-12-27 10:51:10,920][20347] Updated weights for policy 0, policy_version 754983 (0.0008) [2023-12-27 10:51:10,978][20347] Updated weights for policy 0, policy_version 754993 (0.0008) [2023-12-27 10:51:10,980][20381] Updated weights for policy 1, policy_version 754649 (0.0008) [2023-12-27 10:51:11,044][20381] Updated weights for policy 1, policy_version 754659 (0.0008) [2023-12-27 10:51:11,045][20347] Updated weights for policy 0, policy_version 755003 (0.0007) [2023-12-27 10:51:11,114][20381] Updated weights for policy 1, policy_version 754669 (0.0007) [2023-12-27 10:51:11,176][20381] Updated weights for policy 1, policy_version 754679 (0.0009) [2023-12-27 10:51:11,776][20347] Updated weights for policy 0, policy_version 755013 (0.0008) [2023-12-27 10:51:11,838][20347] Updated weights for policy 0, policy_version 755023 (0.0008) [2023-12-27 10:51:11,894][20347] Updated weights for policy 0, policy_version 755033 (0.0008) [2023-12-27 10:51:11,915][20381] Updated weights for policy 1, policy_version 754689 (0.0007) [2023-12-27 10:51:11,981][20381] Updated weights for policy 1, policy_version 754699 (0.0008) [2023-12-27 10:51:12,042][20381] Updated weights for policy 1, policy_version 754709 (0.0008) [2023-12-27 10:51:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 386547712. Throughput: 0: 10226.8, 1: 10319.1. Samples: 386538628. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:12,363][19170] Avg episode reward: [(0, '980.540'), (1, '911.990')] [2023-12-27 10:51:12,568][20347] Updated weights for policy 0, policy_version 755043 (0.0006) [2023-12-27 10:51:12,626][20347] Updated weights for policy 0, policy_version 755053 (0.0007) [2023-12-27 10:51:12,693][20347] Updated weights for policy 0, policy_version 755063 (0.0006) [2023-12-27 10:51:12,748][20381] Updated weights for policy 1, policy_version 754719 (0.0006) [2023-12-27 10:51:12,814][20381] Updated weights for policy 1, policy_version 754729 (0.0007) [2023-12-27 10:51:12,877][20381] Updated weights for policy 1, policy_version 754739 (0.0006) [2023-12-27 10:51:13,369][20347] Updated weights for policy 0, policy_version 755073 (0.0006) [2023-12-27 10:51:13,432][20347] Updated weights for policy 0, policy_version 755083 (0.0008) [2023-12-27 10:51:13,488][20347] Updated weights for policy 0, policy_version 755093 (0.0006) [2023-12-27 10:51:13,511][20381] Updated weights for policy 1, policy_version 754749 (0.0007) [2023-12-27 10:51:13,537][20347] Updated weights for policy 0, policy_version 755103 (0.0008) [2023-12-27 10:51:13,563][20381] Updated weights for policy 1, policy_version 754759 (0.0007) [2023-12-27 10:51:13,619][20381] Updated weights for policy 1, policy_version 754769 (0.0007) [2023-12-27 10:51:14,201][20381] Updated weights for policy 1, policy_version 754779 (0.0007) [2023-12-27 10:51:14,237][20347] Updated weights for policy 0, policy_version 755113 (0.0007) [2023-12-27 10:51:14,253][20381] Updated weights for policy 1, policy_version 754789 (0.0005) [2023-12-27 10:51:14,295][20347] Updated weights for policy 0, policy_version 755123 (0.0008) [2023-12-27 10:51:14,309][20381] Updated weights for policy 1, policy_version 754799 (0.0007) [2023-12-27 10:51:14,362][20347] Updated weights for policy 0, policy_version 755133 (0.0008) [2023-12-27 10:51:14,998][20347] Updated weights for policy 0, policy_version 755143 (0.0007) [2023-12-27 10:51:15,032][20381] Updated weights for policy 1, policy_version 754809 (0.0008) [2023-12-27 10:51:15,065][20347] Updated weights for policy 0, policy_version 755153 (0.0008) [2023-12-27 10:51:15,078][20381] Updated weights for policy 1, policy_version 754819 (0.0007) [2023-12-27 10:51:15,125][20347] Updated weights for policy 0, policy_version 755163 (0.0007) [2023-12-27 10:51:15,134][20381] Updated weights for policy 1, policy_version 754829 (0.0005) [2023-12-27 10:51:15,193][20381] Updated weights for policy 1, policy_version 754839 (0.0007) [2023-12-27 10:51:15,803][20347] Updated weights for policy 0, policy_version 755173 (0.0006) [2023-12-27 10:51:15,833][20381] Updated weights for policy 1, policy_version 754849 (0.0007) [2023-12-27 10:51:15,857][20347] Updated weights for policy 0, policy_version 755183 (0.0006) [2023-12-27 10:51:15,880][20381] Updated weights for policy 1, policy_version 754859 (0.0006) [2023-12-27 10:51:15,912][20347] Updated weights for policy 0, policy_version 755193 (0.0005) [2023-12-27 10:51:15,928][20381] Updated weights for policy 1, policy_version 754869 (0.0006) [2023-12-27 10:51:16,499][20381] Updated weights for policy 1, policy_version 754879 (0.0006) [2023-12-27 10:51:16,558][20381] Updated weights for policy 1, policy_version 754889 (0.0008) [2023-12-27 10:51:16,618][20381] Updated weights for policy 1, policy_version 754899 (0.0007) [2023-12-27 10:51:16,619][20347] Updated weights for policy 0, policy_version 755203 (0.0008) [2023-12-27 10:51:16,671][20291] KL-divergence is very high: 121.9658 [2023-12-27 10:51:16,678][20347] Updated weights for policy 0, policy_version 755213 (0.0008) [2023-12-27 10:51:16,715][20291] KL-divergence is very high: 123.1639 [2023-12-27 10:51:16,731][20347] Updated weights for policy 0, policy_version 755223 (0.0008) [2023-12-27 10:51:17,277][20381] Updated weights for policy 1, policy_version 754909 (0.0005) [2023-12-27 10:51:17,323][20381] Updated weights for policy 1, policy_version 754919 (0.0005) [2023-12-27 10:51:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 386646016. Throughput: 0: 10222.5, 1: 10284.4. Samples: 386662076. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:17,363][19170] Avg episode reward: [(0, '951.420'), (1, '892.640')] [2023-12-27 10:51:17,368][20381] Updated weights for policy 1, policy_version 754929 (0.0005) [2023-12-27 10:51:17,431][20347] Updated weights for policy 0, policy_version 755233 (0.0008) [2023-12-27 10:51:17,486][20347] Updated weights for policy 0, policy_version 755243 (0.0007) [2023-12-27 10:51:17,534][20347] Updated weights for policy 0, policy_version 755253 (0.0007) [2023-12-27 10:51:17,593][20347] Updated weights for policy 0, policy_version 755263 (0.0007) [2023-12-27 10:51:18,037][20381] Updated weights for policy 1, policy_version 754939 (0.0006) [2023-12-27 10:51:18,086][20381] Updated weights for policy 1, policy_version 754949 (0.0010) [2023-12-27 10:51:18,132][20381] Updated weights for policy 1, policy_version 754959 (0.0010) [2023-12-27 10:51:18,204][20347] Updated weights for policy 0, policy_version 755273 (0.0009) [2023-12-27 10:51:18,250][20347] Updated weights for policy 0, policy_version 755283 (0.0008) [2023-12-27 10:51:18,300][20347] Updated weights for policy 0, policy_version 755293 (0.0008) [2023-12-27 10:51:18,824][20381] Updated weights for policy 1, policy_version 754969 (0.0010) [2023-12-27 10:51:18,885][20381] Updated weights for policy 1, policy_version 754979 (0.0008) [2023-12-27 10:51:18,937][20381] Updated weights for policy 1, policy_version 754989 (0.0008) [2023-12-27 10:51:18,992][20381] Updated weights for policy 1, policy_version 754999 (0.0008) [2023-12-27 10:51:19,039][20347] Updated weights for policy 0, policy_version 755303 (0.0010) [2023-12-27 10:51:19,094][20347] Updated weights for policy 0, policy_version 755313 (0.0010) [2023-12-27 10:51:19,154][20347] Updated weights for policy 0, policy_version 755323 (0.0007) [2023-12-27 10:51:19,708][20381] Updated weights for policy 1, policy_version 755009 (0.0007) [2023-12-27 10:51:19,769][20381] Updated weights for policy 1, policy_version 755019 (0.0008) [2023-12-27 10:51:19,834][20381] Updated weights for policy 1, policy_version 755029 (0.0008) [2023-12-27 10:51:19,863][20347] Updated weights for policy 0, policy_version 755333 (0.0007) [2023-12-27 10:51:19,926][20347] Updated weights for policy 0, policy_version 755343 (0.0008) [2023-12-27 10:51:19,990][20347] Updated weights for policy 0, policy_version 755353 (0.0008) [2023-12-27 10:51:20,470][20381] Updated weights for policy 1, policy_version 755039 (0.0006) [2023-12-27 10:51:20,526][20381] Updated weights for policy 1, policy_version 755049 (0.0007) [2023-12-27 10:51:20,586][20381] Updated weights for policy 1, policy_version 755059 (0.0010) [2023-12-27 10:51:20,755][20347] Updated weights for policy 0, policy_version 755363 (0.0008) [2023-12-27 10:51:20,818][20347] Updated weights for policy 0, policy_version 755373 (0.0007) [2023-12-27 10:51:20,881][20347] Updated weights for policy 0, policy_version 755383 (0.0008) [2023-12-27 10:51:21,268][20381] Updated weights for policy 1, policy_version 755069 (0.0010) [2023-12-27 10:51:21,319][20381] Updated weights for policy 1, policy_version 755079 (0.0007) [2023-12-27 10:51:21,384][20381] Updated weights for policy 1, policy_version 755089 (0.0008) [2023-12-27 10:51:21,664][20347] Updated weights for policy 0, policy_version 755393 (0.0008) [2023-12-27 10:51:21,732][20347] Updated weights for policy 0, policy_version 755403 (0.0007) [2023-12-27 10:51:21,795][20347] Updated weights for policy 0, policy_version 755413 (0.0008) [2023-12-27 10:51:21,850][20347] Updated weights for policy 0, policy_version 755423 (0.0008) [2023-12-27 10:51:22,147][20381] Updated weights for policy 1, policy_version 755099 (0.0007) [2023-12-27 10:51:22,194][20381] Updated weights for policy 1, policy_version 755109 (0.0007) [2023-12-27 10:51:22,246][20381] Updated weights for policy 1, policy_version 755119 (0.0006) [2023-12-27 10:51:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 386752512. Throughput: 0: 10212.0, 1: 10273.5. Samples: 386723388. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:22,363][19170] Avg episode reward: [(0, '873.740'), (1, '844.300')] [2023-12-27 10:51:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000755128_193339392.pth... [2023-12-27 10:51:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000755424_193413120.pth... [2023-12-27 10:51:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000754240_193110016.pth [2023-12-27 10:51:22,391][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000753912_193028096.pth [2023-12-27 10:51:22,542][20347] Updated weights for policy 0, policy_version 755433 (0.0007) [2023-12-27 10:51:22,602][20347] Updated weights for policy 0, policy_version 755443 (0.0008) [2023-12-27 10:51:22,666][20347] Updated weights for policy 0, policy_version 755453 (0.0009) [2023-12-27 10:51:22,901][20381] Updated weights for policy 1, policy_version 755129 (0.0007) [2023-12-27 10:51:22,963][20381] Updated weights for policy 1, policy_version 755139 (0.0008) [2023-12-27 10:51:23,030][20381] Updated weights for policy 1, policy_version 755149 (0.0008) [2023-12-27 10:51:23,097][20381] Updated weights for policy 1, policy_version 755159 (0.0008) [2023-12-27 10:51:23,322][20347] Updated weights for policy 0, policy_version 755463 (0.0008) [2023-12-27 10:51:23,378][20347] Updated weights for policy 0, policy_version 755473 (0.0009) [2023-12-27 10:51:23,426][20347] Updated weights for policy 0, policy_version 755483 (0.0008) [2023-12-27 10:51:23,701][20381] Updated weights for policy 1, policy_version 755169 (0.0007) [2023-12-27 10:51:23,758][20381] Updated weights for policy 1, policy_version 755179 (0.0005) [2023-12-27 10:51:23,810][20381] Updated weights for policy 1, policy_version 755189 (0.0005) [2023-12-27 10:51:24,150][20347] Updated weights for policy 0, policy_version 755493 (0.0007) [2023-12-27 10:51:24,212][20347] Updated weights for policy 0, policy_version 755503 (0.0006) [2023-12-27 10:51:24,265][20347] Updated weights for policy 0, policy_version 755513 (0.0007) [2023-12-27 10:51:24,378][20381] Updated weights for policy 1, policy_version 755199 (0.0007) [2023-12-27 10:51:24,434][20381] Updated weights for policy 1, policy_version 755209 (0.0008) [2023-12-27 10:51:24,490][20381] Updated weights for policy 1, policy_version 755219 (0.0008) [2023-12-27 10:51:24,950][20347] Updated weights for policy 0, policy_version 755523 (0.0008) [2023-12-27 10:51:25,016][20347] Updated weights for policy 0, policy_version 755533 (0.0008) [2023-12-27 10:51:25,062][20381] Updated weights for policy 1, policy_version 755229 (0.0007) [2023-12-27 10:51:25,079][20347] Updated weights for policy 0, policy_version 755543 (0.0007) [2023-12-27 10:51:25,112][20381] Updated weights for policy 1, policy_version 755239 (0.0006) [2023-12-27 10:51:25,170][20381] Updated weights for policy 1, policy_version 755249 (0.0008) [2023-12-27 10:51:25,624][20347] Updated weights for policy 0, policy_version 755553 (0.0006) [2023-12-27 10:51:25,687][20347] Updated weights for policy 0, policy_version 755563 (0.0005) [2023-12-27 10:51:25,749][20347] Updated weights for policy 0, policy_version 755573 (0.0006) [2023-12-27 10:51:25,795][20381] Updated weights for policy 1, policy_version 755259 (0.0008) [2023-12-27 10:51:25,798][20347] Updated weights for policy 0, policy_version 755583 (0.0008) [2023-12-27 10:51:25,860][20381] Updated weights for policy 1, policy_version 755269 (0.0008) [2023-12-27 10:51:25,917][20381] Updated weights for policy 1, policy_version 755279 (0.0006) [2023-12-27 10:51:26,382][20347] Updated weights for policy 0, policy_version 755593 (0.0008) [2023-12-27 10:51:26,438][20347] Updated weights for policy 0, policy_version 755603 (0.0008) [2023-12-27 10:51:26,496][20347] Updated weights for policy 0, policy_version 755613 (0.0008) [2023-12-27 10:51:26,537][20381] Updated weights for policy 1, policy_version 755289 (0.0006) [2023-12-27 10:51:26,597][20381] Updated weights for policy 1, policy_version 755299 (0.0008) [2023-12-27 10:51:26,655][20381] Updated weights for policy 1, policy_version 755309 (0.0007) [2023-12-27 10:51:26,711][20381] Updated weights for policy 1, policy_version 755319 (0.0007) [2023-12-27 10:51:27,097][20347] Updated weights for policy 0, policy_version 755623 (0.0009) [2023-12-27 10:51:27,152][20347] Updated weights for policy 0, policy_version 755633 (0.0010) [2023-12-27 10:51:27,207][20347] Updated weights for policy 0, policy_version 755643 (0.0010) [2023-12-27 10:51:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 386859008. Throughput: 0: 10226.8, 1: 10405.2. Samples: 386849420. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:27,363][19170] Avg episode reward: [(0, '873.550'), (1, '844.910')] [2023-12-27 10:51:27,423][20381] Updated weights for policy 1, policy_version 755329 (0.0006) [2023-12-27 10:51:27,473][20381] Updated weights for policy 1, policy_version 755339 (0.0005) [2023-12-27 10:51:27,520][20381] Updated weights for policy 1, policy_version 755349 (0.0005) [2023-12-27 10:51:27,870][20347] Updated weights for policy 0, policy_version 755653 (0.0009) [2023-12-27 10:51:27,921][20347] Updated weights for policy 0, policy_version 755663 (0.0007) [2023-12-27 10:51:27,970][20347] Updated weights for policy 0, policy_version 755673 (0.0007) [2023-12-27 10:51:28,140][20381] Updated weights for policy 1, policy_version 755359 (0.0007) [2023-12-27 10:51:28,191][20381] Updated weights for policy 1, policy_version 755369 (0.0007) [2023-12-27 10:51:28,236][20381] Updated weights for policy 1, policy_version 755379 (0.0007) [2023-12-27 10:51:28,664][20347] Updated weights for policy 0, policy_version 755683 (0.0007) [2023-12-27 10:51:28,726][20347] Updated weights for policy 0, policy_version 755693 (0.0008) [2023-12-27 10:51:28,777][20347] Updated weights for policy 0, policy_version 755703 (0.0008) [2023-12-27 10:51:28,923][20381] Updated weights for policy 1, policy_version 755389 (0.0008) [2023-12-27 10:51:28,976][20381] Updated weights for policy 1, policy_version 755399 (0.0008) [2023-12-27 10:51:29,033][20381] Updated weights for policy 1, policy_version 755409 (0.0007) [2023-12-27 10:51:29,468][20347] Updated weights for policy 0, policy_version 755713 (0.0007) [2023-12-27 10:51:29,520][20347] Updated weights for policy 0, policy_version 755723 (0.0007) [2023-12-27 10:51:29,578][20347] Updated weights for policy 0, policy_version 755733 (0.0008) [2023-12-27 10:51:29,627][20347] Updated weights for policy 0, policy_version 755743 (0.0007) [2023-12-27 10:51:29,755][20381] Updated weights for policy 1, policy_version 755419 (0.0008) [2023-12-27 10:51:29,814][20381] Updated weights for policy 1, policy_version 755429 (0.0007) [2023-12-27 10:51:29,874][20381] Updated weights for policy 1, policy_version 755439 (0.0007) [2023-12-27 10:51:30,343][20347] Updated weights for policy 0, policy_version 755753 (0.0008) [2023-12-27 10:51:30,394][20347] Updated weights for policy 0, policy_version 755763 (0.0007) [2023-12-27 10:51:30,446][20347] Updated weights for policy 0, policy_version 755773 (0.0008) [2023-12-27 10:51:30,550][20381] Updated weights for policy 1, policy_version 755449 (0.0007) [2023-12-27 10:51:30,597][20381] Updated weights for policy 1, policy_version 755459 (0.0005) [2023-12-27 10:51:30,645][20381] Updated weights for policy 1, policy_version 755469 (0.0005) [2023-12-27 10:51:30,699][20381] Updated weights for policy 1, policy_version 755479 (0.0005) [2023-12-27 10:51:31,103][20347] Updated weights for policy 0, policy_version 755783 (0.0008) [2023-12-27 10:51:31,166][20347] Updated weights for policy 0, policy_version 755793 (0.0007) [2023-12-27 10:51:31,224][20347] Updated weights for policy 0, policy_version 755803 (0.0008) [2023-12-27 10:51:31,375][20292] KL-divergence is very high: 176.6596 [2023-12-27 10:51:31,375][20381] Updated weights for policy 1, policy_version 755489 (0.0007) [2023-12-27 10:51:31,434][20381] Updated weights for policy 1, policy_version 755499 (0.0007) [2023-12-27 10:51:31,493][20381] Updated weights for policy 1, policy_version 755509 (0.0007) [2023-12-27 10:51:31,962][20347] Updated weights for policy 0, policy_version 755813 (0.0008) [2023-12-27 10:51:31,989][20291] KL-divergence is very high: 1952.2697 [2023-12-27 10:51:32,011][20347] Updated weights for policy 0, policy_version 755823 (0.0007) [2023-12-27 10:51:32,066][20347] Updated weights for policy 0, policy_version 755833 (0.0007) [2023-12-27 10:51:32,175][20381] Updated weights for policy 1, policy_version 755519 (0.0007) [2023-12-27 10:51:32,228][20381] Updated weights for policy 1, policy_version 755529 (0.0008) [2023-12-27 10:51:32,286][20381] Updated weights for policy 1, policy_version 755539 (0.0007) [2023-12-27 10:51:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 386965504. Throughput: 0: 10234.7, 1: 10362.8. Samples: 386972192. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:32,363][19170] Avg episode reward: [(0, '892.790'), (1, '843.790')] [2023-12-27 10:51:32,761][20347] Updated weights for policy 0, policy_version 755843 (0.0008) [2023-12-27 10:51:32,767][20291] KL-divergence is very high: 242.4085 [2023-12-27 10:51:32,808][20291] KL-divergence is very high: 148.9335 [2023-12-27 10:51:32,813][20347] Updated weights for policy 0, policy_version 755853 (0.0010) [2023-12-27 10:51:32,866][20347] Updated weights for policy 0, policy_version 755863 (0.0010) [2023-12-27 10:51:33,010][20381] Updated weights for policy 1, policy_version 755549 (0.0007) [2023-12-27 10:51:33,067][20381] Updated weights for policy 1, policy_version 755559 (0.0007) [2023-12-27 10:51:33,091][20292] KL-divergence is very high: 129.0985 [2023-12-27 10:51:33,125][20381] Updated weights for policy 1, policy_version 755569 (0.0010) [2023-12-27 10:51:33,533][20291] KL-divergence is very high: 286.7714 [2023-12-27 10:51:33,533][20347] Updated weights for policy 0, policy_version 755873 (0.0010) [2023-12-27 10:51:33,553][20291] KL-divergence is very high: 141.7467 [2023-12-27 10:51:33,580][20291] KL-divergence is very high: 222.0428 [2023-12-27 10:51:33,592][20347] Updated weights for policy 0, policy_version 755883 (0.0010) [2023-12-27 10:51:33,596][20291] KL-divergence is very high: 127.7191 [2023-12-27 10:51:33,628][20291] KL-divergence is very high: 240.4061 [2023-12-27 10:51:33,642][20291] KL-divergence is very high: 124.9821 [2023-12-27 10:51:33,649][20347] Updated weights for policy 0, policy_version 755893 (0.0010) [2023-12-27 10:51:33,664][20291] KL-divergence is very high: 310.1459 [2023-12-27 10:51:33,679][20291] KL-divergence is very high: 133.8946 [2023-12-27 10:51:33,693][20347] Updated weights for policy 0, policy_version 755903 (0.0010) [2023-12-27 10:51:33,747][20381] Updated weights for policy 1, policy_version 755579 (0.0009) [2023-12-27 10:51:33,795][20381] Updated weights for policy 1, policy_version 755589 (0.0005) [2023-12-27 10:51:33,848][20381] Updated weights for policy 1, policy_version 755599 (0.0005) [2023-12-27 10:51:34,366][20347] Updated weights for policy 0, policy_version 755913 (0.0007) [2023-12-27 10:51:34,424][20347] Updated weights for policy 0, policy_version 755923 (0.0007) [2023-12-27 10:51:34,464][20381] Updated weights for policy 1, policy_version 755609 (0.0006) [2023-12-27 10:51:34,484][20347] Updated weights for policy 0, policy_version 755933 (0.0006) [2023-12-27 10:51:34,519][20381] Updated weights for policy 1, policy_version 755619 (0.0008) [2023-12-27 10:51:34,581][20381] Updated weights for policy 1, policy_version 755629 (0.0008) [2023-12-27 10:51:34,641][20381] Updated weights for policy 1, policy_version 755639 (0.0007) [2023-12-27 10:51:35,099][20347] Updated weights for policy 0, policy_version 755943 (0.0005) [2023-12-27 10:51:35,153][20347] Updated weights for policy 0, policy_version 755953 (0.0006) [2023-12-27 10:51:35,205][20347] Updated weights for policy 0, policy_version 755963 (0.0007) [2023-12-27 10:51:35,256][20381] Updated weights for policy 1, policy_version 755649 (0.0007) [2023-12-27 10:51:35,306][20381] Updated weights for policy 1, policy_version 755659 (0.0007) [2023-12-27 10:51:35,358][20381] Updated weights for policy 1, policy_version 755669 (0.0008) [2023-12-27 10:51:35,812][20347] Updated weights for policy 0, policy_version 755973 (0.0007) [2023-12-27 10:51:35,861][20347] Updated weights for policy 0, policy_version 755983 (0.0008) [2023-12-27 10:51:35,903][20291] KL-divergence is very high: 106.1497 [2023-12-27 10:51:35,908][20347] Updated weights for policy 0, policy_version 755993 (0.0007) [2023-12-27 10:51:35,926][20291] KL-divergence is very high: 151.3852 [2023-12-27 10:51:36,032][20381] Updated weights for policy 1, policy_version 755679 (0.0006) [2023-12-27 10:51:36,094][20381] Updated weights for policy 1, policy_version 755689 (0.0005) [2023-12-27 10:51:36,158][20381] Updated weights for policy 1, policy_version 755699 (0.0008) [2023-12-27 10:51:36,520][20347] Updated weights for policy 0, policy_version 756003 (0.0007) [2023-12-27 10:51:36,570][20347] Updated weights for policy 0, policy_version 756013 (0.0008) [2023-12-27 10:51:36,629][20347] Updated weights for policy 0, policy_version 756023 (0.0007) [2023-12-27 10:51:36,845][20381] Updated weights for policy 1, policy_version 755709 (0.0008) [2023-12-27 10:51:36,903][20381] Updated weights for policy 1, policy_version 755719 (0.0006) [2023-12-27 10:51:36,962][20381] Updated weights for policy 1, policy_version 755729 (0.0006) [2023-12-27 10:51:37,344][20347] Updated weights for policy 0, policy_version 756033 (0.0008) [2023-12-27 10:51:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 387063808. Throughput: 0: 10193.1, 1: 10415.0. Samples: 387036048. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:37,363][19170] Avg episode reward: [(0, '535.260'), (1, '873.000')] [2023-12-27 10:51:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000755736_193495040.pth... [2023-12-27 10:51:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000754520_193183744.pth [2023-12-27 10:51:37,399][20347] Updated weights for policy 0, policy_version 756043 (0.0006) [2023-12-27 10:51:37,448][20347] Updated weights for policy 0, policy_version 756053 (0.0010) [2023-12-27 10:51:37,501][20347] Updated weights for policy 0, policy_version 756063 (0.0010) [2023-12-27 10:51:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000756064_193576960.pth... [2023-12-27 10:51:37,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000754848_193265664.pth [2023-12-27 10:51:37,539][20381] Updated weights for policy 1, policy_version 755739 (0.0006) [2023-12-27 10:51:37,595][20381] Updated weights for policy 1, policy_version 755749 (0.0007) [2023-12-27 10:51:37,654][20381] Updated weights for policy 1, policy_version 755759 (0.0008) [2023-12-27 10:51:38,165][20347] Updated weights for policy 0, policy_version 756073 (0.0010) [2023-12-27 10:51:38,218][20347] Updated weights for policy 0, policy_version 756083 (0.0011) [2023-12-27 10:51:38,273][20347] Updated weights for policy 0, policy_version 756093 (0.0010) [2023-12-27 10:51:38,292][20381] Updated weights for policy 1, policy_version 755769 (0.0008) [2023-12-27 10:51:38,349][20381] Updated weights for policy 1, policy_version 755779 (0.0008) [2023-12-27 10:51:38,405][20381] Updated weights for policy 1, policy_version 755789 (0.0006) [2023-12-27 10:51:38,463][20381] Updated weights for policy 1, policy_version 755799 (0.0006) [2023-12-27 10:51:38,947][20347] Updated weights for policy 0, policy_version 756103 (0.0007) [2023-12-27 10:51:39,003][20347] Updated weights for policy 0, policy_version 756113 (0.0005) [2023-12-27 10:51:39,054][20347] Updated weights for policy 0, policy_version 756123 (0.0005) [2023-12-27 10:51:39,163][20381] Updated weights for policy 1, policy_version 755809 (0.0008) [2023-12-27 10:51:39,229][20381] Updated weights for policy 1, policy_version 755819 (0.0008) [2023-12-27 10:51:39,293][20381] Updated weights for policy 1, policy_version 755829 (0.0008) [2023-12-27 10:51:39,708][20347] Updated weights for policy 0, policy_version 756133 (0.0006) [2023-12-27 10:51:39,770][20347] Updated weights for policy 0, policy_version 756143 (0.0006) [2023-12-27 10:51:39,830][20347] Updated weights for policy 0, policy_version 756153 (0.0009) [2023-12-27 10:51:40,067][20381] Updated weights for policy 1, policy_version 755839 (0.0007) [2023-12-27 10:51:40,081][20292] KL-divergence is very high: 171.1151 [2023-12-27 10:51:40,088][20292] KL-divergence is very high: 138.2249 [2023-12-27 10:51:40,129][20381] Updated weights for policy 1, policy_version 755849 (0.0007) [2023-12-27 10:51:40,185][20381] Updated weights for policy 1, policy_version 755859 (0.0008) [2023-12-27 10:51:40,521][20347] Updated weights for policy 0, policy_version 756163 (0.0012) [2023-12-27 10:51:40,579][20347] Updated weights for policy 0, policy_version 756173 (0.0010) [2023-12-27 10:51:40,638][20347] Updated weights for policy 0, policy_version 756183 (0.0010) [2023-12-27 10:51:40,642][20291] KL-divergence is very high: 113.2728 [2023-12-27 10:51:40,663][20291] KL-divergence is very high: 135.4258 [2023-12-27 10:51:40,858][20381] Updated weights for policy 1, policy_version 755869 (0.0007) [2023-12-27 10:51:40,917][20381] Updated weights for policy 1, policy_version 755879 (0.0006) [2023-12-27 10:51:40,968][20381] Updated weights for policy 1, policy_version 755889 (0.0007) [2023-12-27 10:51:41,271][20347] Updated weights for policy 0, policy_version 756193 (0.0010) [2023-12-27 10:51:41,277][20291] KL-divergence is very high: 146.7167 [2023-12-27 10:51:41,329][20347] Updated weights for policy 0, policy_version 756203 (0.0009) [2023-12-27 10:51:41,337][20291] KL-divergence is very high: 100.5108 [2023-12-27 10:51:41,380][20291] KL-divergence is very high: 163.1019 [2023-12-27 10:51:41,402][20347] Updated weights for policy 0, policy_version 756213 (0.0009) [2023-12-27 10:51:41,434][20291] KL-divergence is very high: 201.6087 [2023-12-27 10:51:41,464][20347] Updated weights for policy 0, policy_version 756223 (0.0006) [2023-12-27 10:51:41,723][20381] Updated weights for policy 1, policy_version 755899 (0.0008) [2023-12-27 10:51:41,781][20381] Updated weights for policy 1, policy_version 755909 (0.0008) [2023-12-27 10:51:41,837][20381] Updated weights for policy 1, policy_version 755919 (0.0007) [2023-12-27 10:51:42,117][20347] Updated weights for policy 0, policy_version 756233 (0.0007) [2023-12-27 10:51:42,177][20347] Updated weights for policy 0, policy_version 756243 (0.0007) [2023-12-27 10:51:42,241][20347] Updated weights for policy 0, policy_version 756253 (0.0006) [2023-12-27 10:51:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 387170304. Throughput: 0: 10219.0, 1: 10432.6. Samples: 387160424. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:42,363][19170] Avg episode reward: [(0, '50.680'), (1, '892.840')] [2023-12-27 10:51:42,582][20381] Updated weights for policy 1, policy_version 755929 (0.0009) [2023-12-27 10:51:42,635][20381] Updated weights for policy 1, policy_version 755939 (0.0005) [2023-12-27 10:51:42,685][20381] Updated weights for policy 1, policy_version 755949 (0.0005) [2023-12-27 10:51:42,742][20381] Updated weights for policy 1, policy_version 755959 (0.0007) [2023-12-27 10:51:42,874][20347] Updated weights for policy 0, policy_version 756263 (0.0008) [2023-12-27 10:51:42,939][20347] Updated weights for policy 0, policy_version 756273 (0.0007) [2023-12-27 10:51:43,006][20347] Updated weights for policy 0, policy_version 756283 (0.0006) [2023-12-27 10:51:43,401][20381] Updated weights for policy 1, policy_version 755969 (0.0005) [2023-12-27 10:51:43,458][20381] Updated weights for policy 1, policy_version 755979 (0.0006) [2023-12-27 10:51:43,506][20381] Updated weights for policy 1, policy_version 755989 (0.0005) [2023-12-27 10:51:43,719][20347] Updated weights for policy 0, policy_version 756293 (0.0005) [2023-12-27 10:51:43,772][20347] Updated weights for policy 0, policy_version 756303 (0.0005) [2023-12-27 10:51:43,828][20347] Updated weights for policy 0, policy_version 756313 (0.0005) [2023-12-27 10:51:44,158][20381] Updated weights for policy 1, policy_version 755999 (0.0008) [2023-12-27 10:51:44,217][20381] Updated weights for policy 1, policy_version 756009 (0.0007) [2023-12-27 10:51:44,274][20381] Updated weights for policy 1, policy_version 756019 (0.0006) [2023-12-27 10:51:44,403][20347] Updated weights for policy 0, policy_version 756323 (0.0007) [2023-12-27 10:51:44,449][20347] Updated weights for policy 0, policy_version 756333 (0.0009) [2023-12-27 10:51:44,497][20347] Updated weights for policy 0, policy_version 756343 (0.0008) [2023-12-27 10:51:44,921][20381] Updated weights for policy 1, policy_version 756029 (0.0007) [2023-12-27 10:51:44,978][20381] Updated weights for policy 1, policy_version 756039 (0.0009) [2023-12-27 10:51:45,046][20381] Updated weights for policy 1, policy_version 756049 (0.0007) [2023-12-27 10:51:45,233][20347] Updated weights for policy 0, policy_version 756353 (0.0008) [2023-12-27 10:51:45,300][20347] Updated weights for policy 0, policy_version 756363 (0.0010) [2023-12-27 10:51:45,363][20347] Updated weights for policy 0, policy_version 756373 (0.0008) [2023-12-27 10:51:45,412][20347] Updated weights for policy 0, policy_version 756383 (0.0008) [2023-12-27 10:51:45,772][20381] Updated weights for policy 1, policy_version 756059 (0.0008) [2023-12-27 10:51:45,825][20381] Updated weights for policy 1, policy_version 756069 (0.0005) [2023-12-27 10:51:45,878][20381] Updated weights for policy 1, policy_version 756079 (0.0010) [2023-12-27 10:51:46,124][20347] Updated weights for policy 0, policy_version 756393 (0.0010) [2023-12-27 10:51:46,173][20347] Updated weights for policy 0, policy_version 756403 (0.0006) [2023-12-27 10:51:46,222][20347] Updated weights for policy 0, policy_version 756413 (0.0005) [2023-12-27 10:51:46,498][20381] Updated weights for policy 1, policy_version 756089 (0.0007) [2023-12-27 10:51:46,557][20381] Updated weights for policy 1, policy_version 756099 (0.0008) [2023-12-27 10:51:46,615][20381] Updated weights for policy 1, policy_version 756109 (0.0005) [2023-12-27 10:51:46,663][20381] Updated weights for policy 1, policy_version 756119 (0.0005) [2023-12-27 10:51:46,845][20347] Updated weights for policy 0, policy_version 756423 (0.0005) [2023-12-27 10:51:46,894][20347] Updated weights for policy 0, policy_version 756433 (0.0005) [2023-12-27 10:51:46,945][20347] Updated weights for policy 0, policy_version 756443 (0.0005) [2023-12-27 10:51:47,312][20381] Updated weights for policy 1, policy_version 756129 (0.0010) [2023-12-27 10:51:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 387268608. Throughput: 0: 10308.4, 1: 10381.5. Samples: 387284220. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:47,364][19170] Avg episode reward: [(0, '47.200'), (1, '814.990')] [2023-12-27 10:51:47,372][20381] Updated weights for policy 1, policy_version 756139 (0.0007) [2023-12-27 10:51:47,423][20381] Updated weights for policy 1, policy_version 756149 (0.0006) [2023-12-27 10:51:47,511][20347] Updated weights for policy 0, policy_version 756453 (0.0006) [2023-12-27 10:51:47,572][20347] Updated weights for policy 0, policy_version 756463 (0.0007) [2023-12-27 10:51:47,637][20347] Updated weights for policy 0, policy_version 756473 (0.0006) [2023-12-27 10:51:48,092][20381] Updated weights for policy 1, policy_version 756159 (0.0011) [2023-12-27 10:51:48,151][20381] Updated weights for policy 1, policy_version 756169 (0.0006) [2023-12-27 10:51:48,219][20381] Updated weights for policy 1, policy_version 756179 (0.0008) [2023-12-27 10:51:48,258][20347] Updated weights for policy 0, policy_version 756483 (0.0006) [2023-12-27 10:51:48,305][20347] Updated weights for policy 0, policy_version 756493 (0.0007) [2023-12-27 10:51:48,368][20347] Updated weights for policy 0, policy_version 756503 (0.0008) [2023-12-27 10:51:48,847][20381] Updated weights for policy 1, policy_version 756189 (0.0007) [2023-12-27 10:51:48,898][20381] Updated weights for policy 1, policy_version 756199 (0.0006) [2023-12-27 10:51:48,950][20381] Updated weights for policy 1, policy_version 756209 (0.0008) [2023-12-27 10:51:49,094][20347] Updated weights for policy 0, policy_version 756513 (0.0007) [2023-12-27 10:51:49,150][20347] Updated weights for policy 0, policy_version 756523 (0.0005) [2023-12-27 10:51:49,196][20347] Updated weights for policy 0, policy_version 756533 (0.0005) [2023-12-27 10:51:49,262][20347] Updated weights for policy 0, policy_version 756543 (0.0009) [2023-12-27 10:51:49,641][20381] Updated weights for policy 1, policy_version 756219 (0.0008) [2023-12-27 10:51:49,700][20381] Updated weights for policy 1, policy_version 756229 (0.0010) [2023-12-27 10:51:49,755][20381] Updated weights for policy 1, policy_version 756239 (0.0011) [2023-12-27 10:51:49,970][20347] Updated weights for policy 0, policy_version 756553 (0.0010) [2023-12-27 10:51:50,030][20347] Updated weights for policy 0, policy_version 756563 (0.0010) [2023-12-27 10:51:50,089][20347] Updated weights for policy 0, policy_version 756573 (0.0011) [2023-12-27 10:51:50,483][20381] Updated weights for policy 1, policy_version 756249 (0.0010) [2023-12-27 10:51:50,544][20381] Updated weights for policy 1, policy_version 756259 (0.0011) [2023-12-27 10:51:50,606][20381] Updated weights for policy 1, policy_version 756269 (0.0010) [2023-12-27 10:51:50,668][20381] Updated weights for policy 1, policy_version 756279 (0.0011) [2023-12-27 10:51:50,791][20347] Updated weights for policy 0, policy_version 756583 (0.0010) [2023-12-27 10:51:50,843][20347] Updated weights for policy 0, policy_version 756593 (0.0010) [2023-12-27 10:51:50,899][20347] Updated weights for policy 0, policy_version 756603 (0.0010) [2023-12-27 10:51:51,360][20381] Updated weights for policy 1, policy_version 756289 (0.0011) [2023-12-27 10:51:51,425][20381] Updated weights for policy 1, policy_version 756299 (0.0006) [2023-12-27 10:51:51,482][20381] Updated weights for policy 1, policy_version 756309 (0.0008) [2023-12-27 10:51:51,498][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000003 [2023-12-27 10:51:51,611][20347] Updated weights for policy 0, policy_version 756613 (0.0011) [2023-12-27 10:51:51,673][20347] Updated weights for policy 0, policy_version 756623 (0.0010) [2023-12-27 10:51:51,742][20347] Updated weights for policy 0, policy_version 756633 (0.0011) [2023-12-27 10:51:52,246][20381] Updated weights for policy 1, policy_version 756319 (0.0006) [2023-12-27 10:51:52,310][20381] Updated weights for policy 1, policy_version 756329 (0.0007) [2023-12-27 10:51:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 387366912. Throughput: 0: 10318.6, 1: 10404.6. Samples: 387345808. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:52,363][19170] Avg episode reward: [(0, '54.000'), (1, '892.720')] [2023-12-27 10:51:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000756640_193724416.pth... [2023-12-27 10:51:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000755424_193413120.pth [2023-12-27 10:51:52,372][20381] Updated weights for policy 1, policy_version 756339 (0.0008) [2023-12-27 10:51:52,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000756344_193650688.pth... [2023-12-27 10:51:52,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000755128_193339392.pth [2023-12-27 10:51:52,435][20347] Updated weights for policy 0, policy_version 756643 (0.0008) [2023-12-27 10:51:52,499][20347] Updated weights for policy 0, policy_version 756653 (0.0009) [2023-12-27 10:51:52,567][20347] Updated weights for policy 0, policy_version 756663 (0.0008) [2023-12-27 10:51:53,064][20381] Updated weights for policy 1, policy_version 756349 (0.0007) [2023-12-27 10:51:53,111][20381] Updated weights for policy 1, policy_version 756359 (0.0007) [2023-12-27 10:51:53,160][20381] Updated weights for policy 1, policy_version 756369 (0.0007) [2023-12-27 10:51:53,174][20347] Updated weights for policy 0, policy_version 756673 (0.0007) [2023-12-27 10:51:53,228][20347] Updated weights for policy 0, policy_version 756683 (0.0009) [2023-12-27 10:51:53,284][20347] Updated weights for policy 0, policy_version 756693 (0.0008) [2023-12-27 10:51:53,339][20347] Updated weights for policy 0, policy_version 756703 (0.0007) [2023-12-27 10:51:53,733][20381] Updated weights for policy 1, policy_version 756379 (0.0007) [2023-12-27 10:51:53,781][20381] Updated weights for policy 1, policy_version 756389 (0.0007) [2023-12-27 10:51:53,835][20381] Updated weights for policy 1, policy_version 756399 (0.0006) [2023-12-27 10:51:54,072][20347] Updated weights for policy 0, policy_version 756713 (0.0008) [2023-12-27 10:51:54,122][20347] Updated weights for policy 0, policy_version 756723 (0.0008) [2023-12-27 10:51:54,178][20347] Updated weights for policy 0, policy_version 756733 (0.0007) [2023-12-27 10:51:54,426][20381] Updated weights for policy 1, policy_version 756409 (0.0007) [2023-12-27 10:51:54,475][20381] Updated weights for policy 1, policy_version 756419 (0.0007) [2023-12-27 10:51:54,533][20381] Updated weights for policy 1, policy_version 756429 (0.0005) [2023-12-27 10:51:54,593][20381] Updated weights for policy 1, policy_version 756439 (0.0007) [2023-12-27 10:51:54,916][20347] Updated weights for policy 0, policy_version 756743 (0.0008) [2023-12-27 10:51:54,977][20347] Updated weights for policy 0, policy_version 756753 (0.0007) [2023-12-27 10:51:55,036][20347] Updated weights for policy 0, policy_version 756763 (0.0006) [2023-12-27 10:51:55,247][20381] Updated weights for policy 1, policy_version 756449 (0.0007) [2023-12-27 10:51:55,304][20381] Updated weights for policy 1, policy_version 756459 (0.0008) [2023-12-27 10:51:55,352][20381] Updated weights for policy 1, policy_version 756469 (0.0005) [2023-12-27 10:51:55,763][20347] Updated weights for policy 0, policy_version 756773 (0.0008) [2023-12-27 10:51:55,814][20347] Updated weights for policy 0, policy_version 756783 (0.0006) [2023-12-27 10:51:55,863][20347] Updated weights for policy 0, policy_version 756793 (0.0008) [2023-12-27 10:51:55,906][20381] Updated weights for policy 1, policy_version 756479 (0.0005) [2023-12-27 10:51:55,972][20381] Updated weights for policy 1, policy_version 756489 (0.0007) [2023-12-27 10:51:56,039][20381] Updated weights for policy 1, policy_version 756499 (0.0008) [2023-12-27 10:51:56,585][20381] Updated weights for policy 1, policy_version 756509 (0.0007) [2023-12-27 10:51:56,613][20347] Updated weights for policy 0, policy_version 756803 (0.0008) [2023-12-27 10:51:56,635][20381] Updated weights for policy 1, policy_version 756519 (0.0006) [2023-12-27 10:51:56,665][20347] Updated weights for policy 0, policy_version 756813 (0.0007) [2023-12-27 10:51:56,692][20381] Updated weights for policy 1, policy_version 756529 (0.0005) [2023-12-27 10:51:56,725][20347] Updated weights for policy 0, policy_version 756823 (0.0007) [2023-12-27 10:51:57,229][20381] Updated weights for policy 1, policy_version 756539 (0.0006) [2023-12-27 10:51:57,280][20381] Updated weights for policy 1, policy_version 756549 (0.0006) [2023-12-27 10:51:57,335][20381] Updated weights for policy 1, policy_version 756559 (0.0007) [2023-12-27 10:51:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 387473408. Throughput: 0: 10236.8, 1: 10436.4. Samples: 387468920. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:51:57,363][19170] Avg episode reward: [(0, '45.620'), (1, '951.140')] [2023-12-27 10:51:57,470][20347] Updated weights for policy 0, policy_version 756833 (0.0009) [2023-12-27 10:51:57,529][20347] Updated weights for policy 0, policy_version 756843 (0.0008) [2023-12-27 10:51:57,582][20347] Updated weights for policy 0, policy_version 756853 (0.0006) [2023-12-27 10:51:57,634][20347] Updated weights for policy 0, policy_version 756863 (0.0006) [2023-12-27 10:51:57,991][20381] Updated weights for policy 1, policy_version 756569 (0.0006) [2023-12-27 10:51:58,038][20381] Updated weights for policy 1, policy_version 756579 (0.0008) [2023-12-27 10:51:58,087][20381] Updated weights for policy 1, policy_version 756589 (0.0007) [2023-12-27 10:51:58,132][20381] Updated weights for policy 1, policy_version 756599 (0.0008) [2023-12-27 10:51:58,279][20347] Updated weights for policy 0, policy_version 756873 (0.0008) [2023-12-27 10:51:58,346][20347] Updated weights for policy 0, policy_version 756883 (0.0008) [2023-12-27 10:51:58,411][20347] Updated weights for policy 0, policy_version 756893 (0.0012) [2023-12-27 10:51:58,925][20381] Updated weights for policy 1, policy_version 756609 (0.0008) [2023-12-27 10:51:58,991][20381] Updated weights for policy 1, policy_version 756619 (0.0007) [2023-12-27 10:51:59,048][20381] Updated weights for policy 1, policy_version 756629 (0.0006) [2023-12-27 10:51:59,205][20347] Updated weights for policy 0, policy_version 756903 (0.0009) [2023-12-27 10:51:59,275][20347] Updated weights for policy 0, policy_version 756913 (0.0008) [2023-12-27 10:51:59,340][20347] Updated weights for policy 0, policy_version 756923 (0.0008) [2023-12-27 10:51:59,747][20381] Updated weights for policy 1, policy_version 756639 (0.0007) [2023-12-27 10:51:59,751][20292] KL-divergence is very high: 146.4789 [2023-12-27 10:51:59,807][20381] Updated weights for policy 1, policy_version 756649 (0.0008) [2023-12-27 10:51:59,861][20381] Updated weights for policy 1, policy_version 756659 (0.0007) [2023-12-27 10:52:00,038][20347] Updated weights for policy 0, policy_version 756933 (0.0008) [2023-12-27 10:52:00,086][20347] Updated weights for policy 0, policy_version 756943 (0.0007) [2023-12-27 10:52:00,148][20347] Updated weights for policy 0, policy_version 756953 (0.0007) [2023-12-27 10:52:00,574][20381] Updated weights for policy 1, policy_version 756669 (0.0009) [2023-12-27 10:52:00,620][20381] Updated weights for policy 1, policy_version 756679 (0.0007) [2023-12-27 10:52:00,671][20381] Updated weights for policy 1, policy_version 756689 (0.0007) [2023-12-27 10:52:00,825][20347] Updated weights for policy 0, policy_version 756963 (0.0006) [2023-12-27 10:52:00,873][20347] Updated weights for policy 0, policy_version 756973 (0.0007) [2023-12-27 10:52:00,920][20347] Updated weights for policy 0, policy_version 756983 (0.0007) [2023-12-27 10:52:01,387][20381] Updated weights for policy 1, policy_version 756699 (0.0006) [2023-12-27 10:52:01,447][20381] Updated weights for policy 1, policy_version 756709 (0.0008) [2023-12-27 10:52:01,503][20381] Updated weights for policy 1, policy_version 756719 (0.0008) [2023-12-27 10:52:01,597][20347] Updated weights for policy 0, policy_version 756993 (0.0007) [2023-12-27 10:52:01,660][20347] Updated weights for policy 0, policy_version 757003 (0.0008) [2023-12-27 10:52:01,723][20347] Updated weights for policy 0, policy_version 757013 (0.0008) [2023-12-27 10:52:01,782][20347] Updated weights for policy 0, policy_version 757023 (0.0008) [2023-12-27 10:52:02,221][20381] Updated weights for policy 1, policy_version 756729 (0.0007) [2023-12-27 10:52:02,281][20381] Updated weights for policy 1, policy_version 756739 (0.0006) [2023-12-27 10:52:02,336][20381] Updated weights for policy 1, policy_version 756749 (0.0006) [2023-12-27 10:52:02,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 387571712. Throughput: 0: 10201.3, 1: 10386.2. Samples: 387588516. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:52:02,363][19170] Avg episode reward: [(0, '35.120'), (1, '912.140')] [2023-12-27 10:52:02,401][20381] Updated weights for policy 1, policy_version 756759 (0.0007) [2023-12-27 10:52:02,450][20347] Updated weights for policy 0, policy_version 757033 (0.0007) [2023-12-27 10:52:02,502][20347] Updated weights for policy 0, policy_version 757043 (0.0007) [2023-12-27 10:52:02,557][20347] Updated weights for policy 0, policy_version 757053 (0.0005) [2023-12-27 10:52:03,045][20381] Updated weights for policy 1, policy_version 756769 (0.0005) [2023-12-27 10:52:03,098][20381] Updated weights for policy 1, policy_version 756779 (0.0010) [2023-12-27 10:52:03,146][20381] Updated weights for policy 1, policy_version 756789 (0.0010) [2023-12-27 10:52:03,166][20347] Updated weights for policy 0, policy_version 757063 (0.0006) [2023-12-27 10:52:03,226][20347] Updated weights for policy 0, policy_version 757073 (0.0007) [2023-12-27 10:52:03,278][20347] Updated weights for policy 0, policy_version 757083 (0.0007) [2023-12-27 10:52:03,762][20381] Updated weights for policy 1, policy_version 756799 (0.0010) [2023-12-27 10:52:03,812][20381] Updated weights for policy 1, policy_version 756809 (0.0010) [2023-12-27 10:52:03,876][20381] Updated weights for policy 1, policy_version 756819 (0.0009) [2023-12-27 10:52:03,967][20347] Updated weights for policy 0, policy_version 757093 (0.0007) [2023-12-27 10:52:04,020][20347] Updated weights for policy 0, policy_version 757103 (0.0008) [2023-12-27 10:52:04,074][20347] Updated weights for policy 0, policy_version 757113 (0.0008) [2023-12-27 10:52:04,538][20381] Updated weights for policy 1, policy_version 756829 (0.0006) [2023-12-27 10:52:04,601][20381] Updated weights for policy 1, policy_version 756839 (0.0008) [2023-12-27 10:52:04,656][20381] Updated weights for policy 1, policy_version 756849 (0.0008) [2023-12-27 10:52:04,833][20347] Updated weights for policy 0, policy_version 757123 (0.0008) [2023-12-27 10:52:04,880][20347] Updated weights for policy 0, policy_version 757133 (0.0009) [2023-12-27 10:52:04,928][20347] Updated weights for policy 0, policy_version 757143 (0.0005) [2023-12-27 10:52:05,274][20381] Updated weights for policy 1, policy_version 756859 (0.0007) [2023-12-27 10:52:05,319][20381] Updated weights for policy 1, policy_version 756869 (0.0011) [2023-12-27 10:52:05,374][20381] Updated weights for policy 1, policy_version 756879 (0.0010) [2023-12-27 10:52:05,534][20347] Updated weights for policy 0, policy_version 757153 (0.0005) [2023-12-27 10:52:05,582][20347] Updated weights for policy 0, policy_version 757163 (0.0005) [2023-12-27 10:52:05,631][20347] Updated weights for policy 0, policy_version 757173 (0.0005) [2023-12-27 10:52:05,682][20347] Updated weights for policy 0, policy_version 757183 (0.0005) [2023-12-27 10:52:06,021][20381] Updated weights for policy 1, policy_version 756889 (0.0006) [2023-12-27 10:52:06,073][20381] Updated weights for policy 1, policy_version 756899 (0.0005) [2023-12-27 10:52:06,136][20381] Updated weights for policy 1, policy_version 756909 (0.0007) [2023-12-27 10:52:06,197][20381] Updated weights for policy 1, policy_version 756919 (0.0006) [2023-12-27 10:52:06,318][20347] Updated weights for policy 0, policy_version 757193 (0.0010) [2023-12-27 10:52:06,372][20347] Updated weights for policy 0, policy_version 757203 (0.0009) [2023-12-27 10:52:06,431][20347] Updated weights for policy 0, policy_version 757213 (0.0008) [2023-12-27 10:52:06,840][20381] Updated weights for policy 1, policy_version 756929 (0.0005) [2023-12-27 10:52:06,888][20381] Updated weights for policy 1, policy_version 756939 (0.0005) [2023-12-27 10:52:06,936][20381] Updated weights for policy 1, policy_version 756949 (0.0005) [2023-12-27 10:52:07,080][20347] Updated weights for policy 0, policy_version 757223 (0.0008) [2023-12-27 10:52:07,136][20347] Updated weights for policy 0, policy_version 757233 (0.0010) [2023-12-27 10:52:07,194][20347] Updated weights for policy 0, policy_version 757243 (0.0010) [2023-12-27 10:52:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 387686400. Throughput: 0: 10212.9, 1: 10419.7. Samples: 387651852. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:52:07,363][19170] Avg episode reward: [(0, '34.250'), (1, '931.820')] [2023-12-27 10:52:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000757248_193880064.pth... [2023-12-27 10:52:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000756952_193806336.pth... [2023-12-27 10:52:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000756064_193576960.pth [2023-12-27 10:52:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000755736_193495040.pth [2023-12-27 10:52:07,569][20381] Updated weights for policy 1, policy_version 756959 (0.0008) [2023-12-27 10:52:07,619][20381] Updated weights for policy 1, policy_version 756969 (0.0007) [2023-12-27 10:52:07,669][20381] Updated weights for policy 1, policy_version 756979 (0.0005) [2023-12-27 10:52:07,847][20347] Updated weights for policy 0, policy_version 757253 (0.0010) [2023-12-27 10:52:07,896][20347] Updated weights for policy 0, policy_version 757263 (0.0010) [2023-12-27 10:52:07,948][20347] Updated weights for policy 0, policy_version 757273 (0.0010) [2023-12-27 10:52:08,228][20381] Updated weights for policy 1, policy_version 756989 (0.0005) [2023-12-27 10:52:08,274][20381] Updated weights for policy 1, policy_version 756999 (0.0005) [2023-12-27 10:52:08,325][20381] Updated weights for policy 1, policy_version 757009 (0.0006) [2023-12-27 10:52:08,636][20347] Updated weights for policy 0, policy_version 757283 (0.0009) [2023-12-27 10:52:08,694][20347] Updated weights for policy 0, policy_version 757293 (0.0010) [2023-12-27 10:52:08,749][20347] Updated weights for policy 0, policy_version 757303 (0.0010) [2023-12-27 10:52:08,997][20381] Updated weights for policy 1, policy_version 757019 (0.0011) [2023-12-27 10:52:09,045][20381] Updated weights for policy 1, policy_version 757029 (0.0010) [2023-12-27 10:52:09,090][20381] Updated weights for policy 1, policy_version 757039 (0.0010) [2023-12-27 10:52:09,471][20347] Updated weights for policy 0, policy_version 757313 (0.0010) [2023-12-27 10:52:09,534][20347] Updated weights for policy 0, policy_version 757323 (0.0010) [2023-12-27 10:52:09,594][20347] Updated weights for policy 0, policy_version 757333 (0.0011) [2023-12-27 10:52:09,654][20347] Updated weights for policy 0, policy_version 757343 (0.0010) [2023-12-27 10:52:09,847][20381] Updated weights for policy 1, policy_version 757049 (0.0010) [2023-12-27 10:52:09,907][20381] Updated weights for policy 1, policy_version 757059 (0.0011) [2023-12-27 10:52:09,972][20381] Updated weights for policy 1, policy_version 757069 (0.0008) [2023-12-27 10:52:10,033][20381] Updated weights for policy 1, policy_version 757079 (0.0008) [2023-12-27 10:52:10,365][20347] Updated weights for policy 0, policy_version 757353 (0.0007) [2023-12-27 10:52:10,426][20347] Updated weights for policy 0, policy_version 757363 (0.0008) [2023-12-27 10:52:10,486][20347] Updated weights for policy 0, policy_version 757373 (0.0008) [2023-12-27 10:52:10,733][20381] Updated weights for policy 1, policy_version 757089 (0.0006) [2023-12-27 10:52:10,790][20381] Updated weights for policy 1, policy_version 757099 (0.0007) [2023-12-27 10:52:10,842][20381] Updated weights for policy 1, policy_version 757109 (0.0008) [2023-12-27 10:52:11,204][20347] Updated weights for policy 0, policy_version 757383 (0.0008) [2023-12-27 10:52:11,266][20347] Updated weights for policy 0, policy_version 757393 (0.0007) [2023-12-27 10:52:11,323][20347] Updated weights for policy 0, policy_version 757403 (0.0008) [2023-12-27 10:52:11,574][20381] Updated weights for policy 1, policy_version 757119 (0.0008) [2023-12-27 10:52:11,630][20381] Updated weights for policy 1, policy_version 757129 (0.0009) [2023-12-27 10:52:11,704][20381] Updated weights for policy 1, policy_version 757139 (0.0008) [2023-12-27 10:52:12,042][20347] Updated weights for policy 0, policy_version 757413 (0.0009) [2023-12-27 10:52:12,091][20347] Updated weights for policy 0, policy_version 757423 (0.0008) [2023-12-27 10:52:12,147][20347] Updated weights for policy 0, policy_version 757433 (0.0009) [2023-12-27 10:52:12,332][20381] Updated weights for policy 1, policy_version 757149 (0.0007) [2023-12-27 10:52:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 387784704. Throughput: 0: 10230.4, 1: 10373.8. Samples: 387776608. Policy #0 lag: (min: 31.0, avg: 35.8, max: 63.0) [2023-12-27 10:52:12,363][19170] Avg episode reward: [(0, '51.790'), (1, '844.140')] [2023-12-27 10:52:12,400][20381] Updated weights for policy 1, policy_version 757159 (0.0008) [2023-12-27 10:52:12,458][20381] Updated weights for policy 1, policy_version 757169 (0.0007) [2023-12-27 10:52:12,864][20347] Updated weights for policy 0, policy_version 757443 (0.0007) [2023-12-27 10:52:12,925][20347] Updated weights for policy 0, policy_version 757453 (0.0008) [2023-12-27 10:52:12,985][20347] Updated weights for policy 0, policy_version 757463 (0.0009) [2023-12-27 10:52:13,162][20381] Updated weights for policy 1, policy_version 757179 (0.0008) [2023-12-27 10:52:13,210][20381] Updated weights for policy 1, policy_version 757189 (0.0007) [2023-12-27 10:52:13,254][20381] Updated weights for policy 1, policy_version 757199 (0.0007) [2023-12-27 10:52:13,614][20347] Updated weights for policy 0, policy_version 757473 (0.0010) [2023-12-27 10:52:13,668][20347] Updated weights for policy 0, policy_version 757483 (0.0005) [2023-12-27 10:52:13,724][20347] Updated weights for policy 0, policy_version 757493 (0.0006) [2023-12-27 10:52:13,781][20347] Updated weights for policy 0, policy_version 757503 (0.0010) [2023-12-27 10:52:13,869][20381] Updated weights for policy 1, policy_version 757209 (0.0007) [2023-12-27 10:52:13,925][20381] Updated weights for policy 1, policy_version 757219 (0.0006) [2023-12-27 10:52:13,989][20381] Updated weights for policy 1, policy_version 757229 (0.0008) [2023-12-27 10:52:14,038][20381] Updated weights for policy 1, policy_version 757239 (0.0008) [2023-12-27 10:52:14,428][20347] Updated weights for policy 0, policy_version 757513 (0.0010) [2023-12-27 10:52:14,486][20347] Updated weights for policy 0, policy_version 757523 (0.0010) [2023-12-27 10:52:14,550][20347] Updated weights for policy 0, policy_version 757533 (0.0010) [2023-12-27 10:52:14,713][20381] Updated weights for policy 1, policy_version 757249 (0.0010) [2023-12-27 10:52:14,775][20381] Updated weights for policy 1, policy_version 757259 (0.0009) [2023-12-27 10:52:14,838][20381] Updated weights for policy 1, policy_version 757269 (0.0006) [2023-12-27 10:52:15,246][20347] Updated weights for policy 0, policy_version 757543 (0.0008) [2023-12-27 10:52:15,310][20347] Updated weights for policy 0, policy_version 757553 (0.0009) [2023-12-27 10:52:15,362][20347] Updated weights for policy 0, policy_version 757563 (0.0007) [2023-12-27 10:52:15,419][20381] Updated weights for policy 1, policy_version 757279 (0.0007) [2023-12-27 10:52:15,483][20381] Updated weights for policy 1, policy_version 757289 (0.0008) [2023-12-27 10:52:15,538][20381] Updated weights for policy 1, policy_version 757299 (0.0008) [2023-12-27 10:52:15,913][20347] Updated weights for policy 0, policy_version 757573 (0.0006) [2023-12-27 10:52:15,967][20347] Updated weights for policy 0, policy_version 757583 (0.0005) [2023-12-27 10:52:16,034][20347] Updated weights for policy 0, policy_version 757593 (0.0008) [2023-12-27 10:52:16,201][20381] Updated weights for policy 1, policy_version 757309 (0.0006) [2023-12-27 10:52:16,259][20381] Updated weights for policy 1, policy_version 757319 (0.0007) [2023-12-27 10:52:16,318][20381] Updated weights for policy 1, policy_version 757329 (0.0008) [2023-12-27 10:52:16,641][20347] Updated weights for policy 0, policy_version 757603 (0.0006) [2023-12-27 10:52:16,687][20347] Updated weights for policy 0, policy_version 757613 (0.0005) [2023-12-27 10:52:16,747][20347] Updated weights for policy 0, policy_version 757623 (0.0007) [2023-12-27 10:52:16,992][20381] Updated weights for policy 1, policy_version 757339 (0.0008) [2023-12-27 10:52:17,047][20381] Updated weights for policy 1, policy_version 757349 (0.0007) [2023-12-27 10:52:17,104][20381] Updated weights for policy 1, policy_version 757359 (0.0010) [2023-12-27 10:52:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 387891200. Throughput: 0: 10243.9, 1: 10405.4. Samples: 387901412. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:17,363][19170] Avg episode reward: [(0, '142.010'), (1, '863.510')] [2023-12-27 10:52:17,394][20347] Updated weights for policy 0, policy_version 757633 (0.0010) [2023-12-27 10:52:17,456][20347] Updated weights for policy 0, policy_version 757643 (0.0008) [2023-12-27 10:52:17,509][20347] Updated weights for policy 0, policy_version 757653 (0.0005) [2023-12-27 10:52:17,569][20347] Updated weights for policy 0, policy_version 757663 (0.0008) [2023-12-27 10:52:17,649][20381] Updated weights for policy 1, policy_version 757369 (0.0009) [2023-12-27 10:52:17,708][20381] Updated weights for policy 1, policy_version 757379 (0.0008) [2023-12-27 10:52:17,765][20381] Updated weights for policy 1, policy_version 757389 (0.0010) [2023-12-27 10:52:17,817][20381] Updated weights for policy 1, policy_version 757399 (0.0010) [2023-12-27 10:52:18,207][20347] Updated weights for policy 0, policy_version 757673 (0.0010) [2023-12-27 10:52:18,267][20347] Updated weights for policy 0, policy_version 757683 (0.0010) [2023-12-27 10:52:18,325][20347] Updated weights for policy 0, policy_version 757693 (0.0010) [2023-12-27 10:52:18,377][20381] Updated weights for policy 1, policy_version 757409 (0.0009) [2023-12-27 10:52:18,430][20381] Updated weights for policy 1, policy_version 757419 (0.0011) [2023-12-27 10:52:18,491][20381] Updated weights for policy 1, policy_version 757429 (0.0011) [2023-12-27 10:52:19,012][20347] Updated weights for policy 0, policy_version 757703 (0.0007) [2023-12-27 10:52:19,068][20347] Updated weights for policy 0, policy_version 757713 (0.0007) [2023-12-27 10:52:19,116][20347] Updated weights for policy 0, policy_version 757723 (0.0008) [2023-12-27 10:52:19,159][20381] Updated weights for policy 1, policy_version 757439 (0.0010) [2023-12-27 10:52:19,208][20381] Updated weights for policy 1, policy_version 757449 (0.0010) [2023-12-27 10:52:19,269][20381] Updated weights for policy 1, policy_version 757459 (0.0009) [2023-12-27 10:52:19,814][20347] Updated weights for policy 0, policy_version 757733 (0.0008) [2023-12-27 10:52:19,892][20347] Updated weights for policy 0, policy_version 757743 (0.0008) [2023-12-27 10:52:19,964][20347] Updated weights for policy 0, policy_version 757753 (0.0008) [2023-12-27 10:52:20,010][20381] Updated weights for policy 1, policy_version 757469 (0.0007) [2023-12-27 10:52:20,071][20381] Updated weights for policy 1, policy_version 757479 (0.0008) [2023-12-27 10:52:20,138][20381] Updated weights for policy 1, policy_version 757489 (0.0008) [2023-12-27 10:52:20,607][20347] Updated weights for policy 0, policy_version 757763 (0.0008) [2023-12-27 10:52:20,667][20347] Updated weights for policy 0, policy_version 757773 (0.0007) [2023-12-27 10:52:20,724][20347] Updated weights for policy 0, policy_version 757783 (0.0007) [2023-12-27 10:52:20,823][20381] Updated weights for policy 1, policy_version 757499 (0.0007) [2023-12-27 10:52:20,886][20381] Updated weights for policy 1, policy_version 757509 (0.0005) [2023-12-27 10:52:20,953][20381] Updated weights for policy 1, policy_version 757519 (0.0007) [2023-12-27 10:52:21,445][20347] Updated weights for policy 0, policy_version 757793 (0.0007) [2023-12-27 10:52:21,502][20347] Updated weights for policy 0, policy_version 757803 (0.0007) [2023-12-27 10:52:21,561][20347] Updated weights for policy 0, policy_version 757813 (0.0007) [2023-12-27 10:52:21,632][20347] Updated weights for policy 0, policy_version 757823 (0.0008) [2023-12-27 10:52:21,669][20381] Updated weights for policy 1, policy_version 757529 (0.0008) [2023-12-27 10:52:21,715][20381] Updated weights for policy 1, policy_version 757539 (0.0006) [2023-12-27 10:52:21,782][20381] Updated weights for policy 1, policy_version 757549 (0.0009) [2023-12-27 10:52:21,842][20381] Updated weights for policy 1, policy_version 757559 (0.0008) [2023-12-27 10:52:22,361][20347] Updated weights for policy 0, policy_version 757833 (0.0009) [2023-12-27 10:52:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 387989504. Throughput: 0: 10226.9, 1: 10404.4. Samples: 387964460. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:22,363][19170] Avg episode reward: [(0, '293.260'), (1, '941.460')] [2023-12-27 10:52:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000757560_193961984.pth... [2023-12-27 10:52:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000756344_193650688.pth [2023-12-27 10:52:22,421][20347] Updated weights for policy 0, policy_version 757843 (0.0008) [2023-12-27 10:52:22,485][20347] Updated weights for policy 0, policy_version 757853 (0.0008) [2023-12-27 10:52:22,501][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000757856_194035712.pth... [2023-12-27 10:52:22,503][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000756640_193724416.pth [2023-12-27 10:52:22,540][20381] Updated weights for policy 1, policy_version 757569 (0.0008) [2023-12-27 10:52:22,587][20381] Updated weights for policy 1, policy_version 757579 (0.0007) [2023-12-27 10:52:22,645][20381] Updated weights for policy 1, policy_version 757589 (0.0008) [2023-12-27 10:52:23,144][20347] Updated weights for policy 0, policy_version 757863 (0.0008) [2023-12-27 10:52:23,197][20347] Updated weights for policy 0, policy_version 757873 (0.0008) [2023-12-27 10:52:23,241][20347] Updated weights for policy 0, policy_version 757883 (0.0007) [2023-12-27 10:52:23,291][20381] Updated weights for policy 1, policy_version 757599 (0.0007) [2023-12-27 10:52:23,349][20381] Updated weights for policy 1, policy_version 757609 (0.0008) [2023-12-27 10:52:23,405][20381] Updated weights for policy 1, policy_version 757619 (0.0008) [2023-12-27 10:52:23,870][20347] Updated weights for policy 0, policy_version 757893 (0.0007) [2023-12-27 10:52:23,925][20347] Updated weights for policy 0, policy_version 757903 (0.0006) [2023-12-27 10:52:23,976][20347] Updated weights for policy 0, policy_version 757913 (0.0005) [2023-12-27 10:52:24,143][20381] Updated weights for policy 1, policy_version 757629 (0.0008) [2023-12-27 10:52:24,191][20381] Updated weights for policy 1, policy_version 757639 (0.0007) [2023-12-27 10:52:24,243][20381] Updated weights for policy 1, policy_version 757649 (0.0007) [2023-12-27 10:52:24,557][20347] Updated weights for policy 0, policy_version 757923 (0.0005) [2023-12-27 10:52:24,608][20347] Updated weights for policy 0, policy_version 757933 (0.0005) [2023-12-27 10:52:24,658][20347] Updated weights for policy 0, policy_version 757943 (0.0005) [2023-12-27 10:52:24,902][20381] Updated weights for policy 1, policy_version 757659 (0.0005) [2023-12-27 10:52:24,949][20381] Updated weights for policy 1, policy_version 757669 (0.0006) [2023-12-27 10:52:24,973][20292] KL-divergence is very high: 534.9357 [2023-12-27 10:52:24,996][20381] Updated weights for policy 1, policy_version 757679 (0.0005) [2023-12-27 10:52:25,300][20347] Updated weights for policy 0, policy_version 757953 (0.0006) [2023-12-27 10:52:25,356][20347] Updated weights for policy 0, policy_version 757963 (0.0008) [2023-12-27 10:52:25,413][20347] Updated weights for policy 0, policy_version 757973 (0.0008) [2023-12-27 10:52:25,467][20347] Updated weights for policy 0, policy_version 757983 (0.0007) [2023-12-27 10:52:25,625][20381] Updated weights for policy 1, policy_version 757689 (0.0005) [2023-12-27 10:52:25,685][20381] Updated weights for policy 1, policy_version 757699 (0.0007) [2023-12-27 10:52:25,738][20381] Updated weights for policy 1, policy_version 757709 (0.0010) [2023-12-27 10:52:25,788][20381] Updated weights for policy 1, policy_version 757719 (0.0005) [2023-12-27 10:52:26,068][20347] Updated weights for policy 0, policy_version 757993 (0.0006) [2023-12-27 10:52:26,129][20347] Updated weights for policy 0, policy_version 758003 (0.0009) [2023-12-27 10:52:26,192][20347] Updated weights for policy 0, policy_version 758013 (0.0010) [2023-12-27 10:52:26,435][20381] Updated weights for policy 1, policy_version 757729 (0.0005) [2023-12-27 10:52:26,486][20381] Updated weights for policy 1, policy_version 757739 (0.0005) [2023-12-27 10:52:26,514][20292] KL-divergence is very high: 109.8371 [2023-12-27 10:52:26,540][20381] Updated weights for policy 1, policy_version 757749 (0.0005) [2023-12-27 10:52:26,803][20347] Updated weights for policy 0, policy_version 758023 (0.0008) [2023-12-27 10:52:26,864][20347] Updated weights for policy 0, policy_version 758033 (0.0008) [2023-12-27 10:52:26,919][20347] Updated weights for policy 0, policy_version 758043 (0.0007) [2023-12-27 10:52:27,182][20381] Updated weights for policy 1, policy_version 757759 (0.0007) [2023-12-27 10:52:27,241][20381] Updated weights for policy 1, policy_version 757769 (0.0010) [2023-12-27 10:52:27,302][20381] Updated weights for policy 1, policy_version 757779 (0.0010) [2023-12-27 10:52:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 388104192. Throughput: 0: 10234.2, 1: 10417.5. Samples: 388089752. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:27,363][19170] Avg episode reward: [(0, '553.740'), (1, '931.710')] [2023-12-27 10:52:27,555][20347] Updated weights for policy 0, policy_version 758053 (0.0006) [2023-12-27 10:52:27,615][20347] Updated weights for policy 0, policy_version 758063 (0.0005) [2023-12-27 10:52:27,676][20347] Updated weights for policy 0, policy_version 758073 (0.0006) [2023-12-27 10:52:27,951][20381] Updated weights for policy 1, policy_version 757789 (0.0009) [2023-12-27 10:52:28,000][20381] Updated weights for policy 1, policy_version 757799 (0.0007) [2023-12-27 10:52:28,049][20381] Updated weights for policy 1, policy_version 757809 (0.0007) [2023-12-27 10:52:28,288][20347] Updated weights for policy 0, policy_version 758083 (0.0006) [2023-12-27 10:52:28,350][20347] Updated weights for policy 0, policy_version 758093 (0.0008) [2023-12-27 10:52:28,410][20347] Updated weights for policy 0, policy_version 758103 (0.0007) [2023-12-27 10:52:28,818][20381] Updated weights for policy 1, policy_version 757819 (0.0008) [2023-12-27 10:52:28,872][20381] Updated weights for policy 1, policy_version 757829 (0.0007) [2023-12-27 10:52:28,931][20381] Updated weights for policy 1, policy_version 757839 (0.0007) [2023-12-27 10:52:29,041][20347] Updated weights for policy 0, policy_version 758113 (0.0006) [2023-12-27 10:52:29,104][20347] Updated weights for policy 0, policy_version 758123 (0.0010) [2023-12-27 10:52:29,171][20347] Updated weights for policy 0, policy_version 758133 (0.0010) [2023-12-27 10:52:29,231][20347] Updated weights for policy 0, policy_version 758143 (0.0009) [2023-12-27 10:52:29,659][20381] Updated weights for policy 1, policy_version 757849 (0.0007) [2023-12-27 10:52:29,711][20381] Updated weights for policy 1, policy_version 757859 (0.0008) [2023-12-27 10:52:29,770][20381] Updated weights for policy 1, policy_version 757869 (0.0008) [2023-12-27 10:52:29,822][20381] Updated weights for policy 1, policy_version 757879 (0.0007) [2023-12-27 10:52:29,924][20347] Updated weights for policy 0, policy_version 758153 (0.0008) [2023-12-27 10:52:29,983][20347] Updated weights for policy 0, policy_version 758163 (0.0008) [2023-12-27 10:52:30,040][20347] Updated weights for policy 0, policy_version 758173 (0.0008) [2023-12-27 10:52:30,534][20381] Updated weights for policy 1, policy_version 757889 (0.0007) [2023-12-27 10:52:30,590][20381] Updated weights for policy 1, policy_version 757899 (0.0008) [2023-12-27 10:52:30,650][20381] Updated weights for policy 1, policy_version 757909 (0.0008) [2023-12-27 10:52:30,727][20347] Updated weights for policy 0, policy_version 758183 (0.0007) [2023-12-27 10:52:30,779][20347] Updated weights for policy 0, policy_version 758193 (0.0005) [2023-12-27 10:52:30,824][20347] Updated weights for policy 0, policy_version 758203 (0.0005) [2023-12-27 10:52:31,300][20381] Updated weights for policy 1, policy_version 757919 (0.0007) [2023-12-27 10:52:31,355][20381] Updated weights for policy 1, policy_version 757929 (0.0007) [2023-12-27 10:52:31,417][20347] Updated weights for policy 0, policy_version 758213 (0.0007) [2023-12-27 10:52:31,421][20381] Updated weights for policy 1, policy_version 757939 (0.0008) [2023-12-27 10:52:31,474][20347] Updated weights for policy 0, policy_version 758223 (0.0009) [2023-12-27 10:52:31,537][20347] Updated weights for policy 0, policy_version 758233 (0.0010) [2023-12-27 10:52:32,083][20381] Updated weights for policy 1, policy_version 757949 (0.0006) [2023-12-27 10:52:32,145][20381] Updated weights for policy 1, policy_version 757959 (0.0007) [2023-12-27 10:52:32,210][20381] Updated weights for policy 1, policy_version 757969 (0.0007) [2023-12-27 10:52:32,238][20347] Updated weights for policy 0, policy_version 758243 (0.0010) [2023-12-27 10:52:32,297][20347] Updated weights for policy 0, policy_version 758253 (0.0010) [2023-12-27 10:52:32,360][20347] Updated weights for policy 0, policy_version 758263 (0.0010) [2023-12-27 10:52:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20632.8). Total num frames: 388202496. Throughput: 0: 10245.3, 1: 10401.4. Samples: 388213316. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:32,363][19170] Avg episode reward: [(0, '100.070'), (1, '922.000')] [2023-12-27 10:52:32,817][20381] Updated weights for policy 1, policy_version 757979 (0.0006) [2023-12-27 10:52:32,865][20381] Updated weights for policy 1, policy_version 757989 (0.0007) [2023-12-27 10:52:32,919][20381] Updated weights for policy 1, policy_version 757999 (0.0007) [2023-12-27 10:52:32,983][20347] Updated weights for policy 0, policy_version 758273 (0.0008) [2023-12-27 10:52:33,031][20347] Updated weights for policy 0, policy_version 758283 (0.0009) [2023-12-27 10:52:33,077][20347] Updated weights for policy 0, policy_version 758293 (0.0006) [2023-12-27 10:52:33,126][20347] Updated weights for policy 0, policy_version 758303 (0.0005) [2023-12-27 10:52:33,496][20381] Updated weights for policy 1, policy_version 758009 (0.0007) [2023-12-27 10:52:33,562][20381] Updated weights for policy 1, policy_version 758019 (0.0007) [2023-12-27 10:52:33,622][20381] Updated weights for policy 1, policy_version 758029 (0.0006) [2023-12-27 10:52:33,673][20381] Updated weights for policy 1, policy_version 758039 (0.0005) [2023-12-27 10:52:33,705][20347] Updated weights for policy 0, policy_version 758313 (0.0005) [2023-12-27 10:52:33,768][20347] Updated weights for policy 0, policy_version 758323 (0.0005) [2023-12-27 10:52:33,823][20347] Updated weights for policy 0, policy_version 758333 (0.0010) [2023-12-27 10:52:34,262][20381] Updated weights for policy 1, policy_version 758049 (0.0005) [2023-12-27 10:52:34,323][20381] Updated weights for policy 1, policy_version 758059 (0.0007) [2023-12-27 10:52:34,383][20381] Updated weights for policy 1, policy_version 758069 (0.0008) [2023-12-27 10:52:34,450][20347] Updated weights for policy 0, policy_version 758343 (0.0010) [2023-12-27 10:52:34,504][20347] Updated weights for policy 0, policy_version 758353 (0.0010) [2023-12-27 10:52:34,564][20347] Updated weights for policy 0, policy_version 758363 (0.0010) [2023-12-27 10:52:35,020][20381] Updated weights for policy 1, policy_version 758079 (0.0007) [2023-12-27 10:52:35,093][20381] Updated weights for policy 1, policy_version 758089 (0.0006) [2023-12-27 10:52:35,150][20381] Updated weights for policy 1, policy_version 758099 (0.0007) [2023-12-27 10:52:35,198][20347] Updated weights for policy 0, policy_version 758373 (0.0008) [2023-12-27 10:52:35,243][20347] Updated weights for policy 0, policy_version 758383 (0.0009) [2023-12-27 10:52:35,288][20347] Updated weights for policy 0, policy_version 758393 (0.0010) [2023-12-27 10:52:35,772][20381] Updated weights for policy 1, policy_version 758109 (0.0009) [2023-12-27 10:52:35,826][20381] Updated weights for policy 1, policy_version 758119 (0.0011) [2023-12-27 10:52:35,883][20381] Updated weights for policy 1, policy_version 758129 (0.0011) [2023-12-27 10:52:35,990][20347] Updated weights for policy 0, policy_version 758403 (0.0010) [2023-12-27 10:52:36,053][20347] Updated weights for policy 0, policy_version 758413 (0.0007) [2023-12-27 10:52:36,109][20347] Updated weights for policy 0, policy_version 758423 (0.0007) [2023-12-27 10:52:36,585][20381] Updated weights for policy 1, policy_version 758139 (0.0011) [2023-12-27 10:52:36,639][20381] Updated weights for policy 1, policy_version 758149 (0.0011) [2023-12-27 10:52:36,692][20381] Updated weights for policy 1, policy_version 758159 (0.0011) [2023-12-27 10:52:36,725][20347] Updated weights for policy 0, policy_version 758433 (0.0007) [2023-12-27 10:52:36,784][20347] Updated weights for policy 0, policy_version 758443 (0.0007) [2023-12-27 10:52:36,838][20347] Updated weights for policy 0, policy_version 758453 (0.0007) [2023-12-27 10:52:36,885][20347] Updated weights for policy 0, policy_version 758463 (0.0007) [2023-12-27 10:52:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 388308992. Throughput: 0: 10289.0, 1: 10472.4. Samples: 388280072. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:37,363][19170] Avg episode reward: [(0, '55.700'), (1, '902.440')] [2023-12-27 10:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000758168_194117632.pth... [2023-12-27 10:52:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000758464_194191360.pth... [2023-12-27 10:52:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000756952_193806336.pth [2023-12-27 10:52:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000757248_193880064.pth [2023-12-27 10:52:37,398][20381] Updated weights for policy 1, policy_version 758169 (0.0011) [2023-12-27 10:52:37,464][20381] Updated weights for policy 1, policy_version 758179 (0.0011) [2023-12-27 10:52:37,517][20347] Updated weights for policy 0, policy_version 758473 (0.0009) [2023-12-27 10:52:37,521][20381] Updated weights for policy 1, policy_version 758189 (0.0011) [2023-12-27 10:52:37,572][20347] Updated weights for policy 0, policy_version 758483 (0.0010) [2023-12-27 10:52:37,576][20381] Updated weights for policy 1, policy_version 758199 (0.0011) [2023-12-27 10:52:37,621][20347] Updated weights for policy 0, policy_version 758493 (0.0009) [2023-12-27 10:52:38,248][20347] Updated weights for policy 0, policy_version 758503 (0.0009) [2023-12-27 10:52:38,274][20381] Updated weights for policy 1, policy_version 758209 (0.0008) [2023-12-27 10:52:38,310][20347] Updated weights for policy 0, policy_version 758513 (0.0010) [2023-12-27 10:52:38,336][20381] Updated weights for policy 1, policy_version 758219 (0.0007) [2023-12-27 10:52:38,375][20347] Updated weights for policy 0, policy_version 758523 (0.0010) [2023-12-27 10:52:38,400][20381] Updated weights for policy 1, policy_version 758229 (0.0008) [2023-12-27 10:52:39,034][20347] Updated weights for policy 0, policy_version 758533 (0.0009) [2023-12-27 10:52:39,061][20381] Updated weights for policy 1, policy_version 758239 (0.0010) [2023-12-27 10:52:39,090][20347] Updated weights for policy 0, policy_version 758543 (0.0010) [2023-12-27 10:52:39,117][20381] Updated weights for policy 1, policy_version 758249 (0.0011) [2023-12-27 10:52:39,144][20347] Updated weights for policy 0, policy_version 758553 (0.0010) [2023-12-27 10:52:39,173][20381] Updated weights for policy 1, policy_version 758259 (0.0011) [2023-12-27 10:52:39,888][20347] Updated weights for policy 0, policy_version 758563 (0.0009) [2023-12-27 10:52:39,913][20381] Updated weights for policy 1, policy_version 758269 (0.0011) [2023-12-27 10:52:39,952][20347] Updated weights for policy 0, policy_version 758573 (0.0007) [2023-12-27 10:52:39,980][20381] Updated weights for policy 1, policy_version 758279 (0.0007) [2023-12-27 10:52:40,010][20347] Updated weights for policy 0, policy_version 758583 (0.0007) [2023-12-27 10:52:40,045][20381] Updated weights for policy 1, policy_version 758289 (0.0010) [2023-12-27 10:52:40,644][20347] Updated weights for policy 0, policy_version 758593 (0.0006) [2023-12-27 10:52:40,707][20347] Updated weights for policy 0, policy_version 758603 (0.0005) [2023-12-27 10:52:40,729][20381] Updated weights for policy 1, policy_version 758299 (0.0010) [2023-12-27 10:52:40,765][20347] Updated weights for policy 0, policy_version 758613 (0.0007) [2023-12-27 10:52:40,787][20381] Updated weights for policy 1, policy_version 758309 (0.0006) [2023-12-27 10:52:40,824][20347] Updated weights for policy 0, policy_version 758623 (0.0008) [2023-12-27 10:52:40,856][20381] Updated weights for policy 1, policy_version 758319 (0.0006) [2023-12-27 10:52:41,516][20347] Updated weights for policy 0, policy_version 758633 (0.0010) [2023-12-27 10:52:41,560][20381] Updated weights for policy 1, policy_version 758329 (0.0006) [2023-12-27 10:52:41,569][20347] Updated weights for policy 0, policy_version 758643 (0.0010) [2023-12-27 10:52:41,623][20381] Updated weights for policy 1, policy_version 758339 (0.0009) [2023-12-27 10:52:41,640][20347] Updated weights for policy 0, policy_version 758653 (0.0010) [2023-12-27 10:52:41,690][20381] Updated weights for policy 1, policy_version 758349 (0.0011) [2023-12-27 10:52:41,758][20381] Updated weights for policy 1, policy_version 758359 (0.0010) [2023-12-27 10:52:42,349][20347] Updated weights for policy 0, policy_version 758663 (0.0008) [2023-12-27 10:52:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 388407296. Throughput: 0: 10382.4, 1: 10388.9. Samples: 388403628. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:42,363][19170] Avg episode reward: [(0, '314.720'), (1, '892.540')] [2023-12-27 10:52:42,413][20347] Updated weights for policy 0, policy_version 758673 (0.0008) [2023-12-27 10:52:42,419][20381] Updated weights for policy 1, policy_version 758369 (0.0006) [2023-12-27 10:52:42,475][20347] Updated weights for policy 0, policy_version 758683 (0.0008) [2023-12-27 10:52:42,481][20381] Updated weights for policy 1, policy_version 758379 (0.0006) [2023-12-27 10:52:42,543][20381] Updated weights for policy 1, policy_version 758389 (0.0007) [2023-12-27 10:52:43,112][20381] Updated weights for policy 1, policy_version 758399 (0.0007) [2023-12-27 10:52:43,176][20381] Updated weights for policy 1, policy_version 758409 (0.0008) [2023-12-27 10:52:43,210][20347] Updated weights for policy 0, policy_version 758693 (0.0007) [2023-12-27 10:52:43,233][20381] Updated weights for policy 1, policy_version 758419 (0.0008) [2023-12-27 10:52:43,265][20347] Updated weights for policy 0, policy_version 758703 (0.0005) [2023-12-27 10:52:43,310][20347] Updated weights for policy 0, policy_version 758713 (0.0006) [2023-12-27 10:52:43,853][20381] Updated weights for policy 1, policy_version 758429 (0.0007) [2023-12-27 10:52:43,902][20381] Updated weights for policy 1, policy_version 758439 (0.0005) [2023-12-27 10:52:43,954][20381] Updated weights for policy 1, policy_version 758449 (0.0005) [2023-12-27 10:52:43,961][20347] Updated weights for policy 0, policy_version 758723 (0.0009) [2023-12-27 10:52:44,028][20347] Updated weights for policy 0, policy_version 758733 (0.0010) [2023-12-27 10:52:44,096][20347] Updated weights for policy 0, policy_version 758743 (0.0010) [2023-12-27 10:52:44,541][20381] Updated weights for policy 1, policy_version 758459 (0.0007) [2023-12-27 10:52:44,596][20381] Updated weights for policy 1, policy_version 758469 (0.0010) [2023-12-27 10:52:44,647][20381] Updated weights for policy 1, policy_version 758479 (0.0010) [2023-12-27 10:52:44,673][20347] Updated weights for policy 0, policy_version 758753 (0.0007) [2023-12-27 10:52:44,729][20347] Updated weights for policy 0, policy_version 758763 (0.0005) [2023-12-27 10:52:44,791][20347] Updated weights for policy 0, policy_version 758773 (0.0005) [2023-12-27 10:52:44,853][20347] Updated weights for policy 0, policy_version 758783 (0.0010) [2023-12-27 10:52:45,367][20381] Updated weights for policy 1, policy_version 758489 (0.0010) [2023-12-27 10:52:45,428][20381] Updated weights for policy 1, policy_version 758499 (0.0010) [2023-12-27 10:52:45,469][20347] Updated weights for policy 0, policy_version 758793 (0.0010) [2023-12-27 10:52:45,486][20381] Updated weights for policy 1, policy_version 758509 (0.0010) [2023-12-27 10:52:45,515][20347] Updated weights for policy 0, policy_version 758803 (0.0010) [2023-12-27 10:52:45,537][20381] Updated weights for policy 1, policy_version 758519 (0.0011) [2023-12-27 10:52:45,561][20347] Updated weights for policy 0, policy_version 758813 (0.0010) [2023-12-27 10:52:46,224][20381] Updated weights for policy 1, policy_version 758529 (0.0010) [2023-12-27 10:52:46,267][20347] Updated weights for policy 0, policy_version 758823 (0.0007) [2023-12-27 10:52:46,283][20381] Updated weights for policy 1, policy_version 758539 (0.0010) [2023-12-27 10:52:46,311][20347] Updated weights for policy 0, policy_version 758833 (0.0008) [2023-12-27 10:52:46,339][20381] Updated weights for policy 1, policy_version 758549 (0.0011) [2023-12-27 10:52:46,357][20347] Updated weights for policy 0, policy_version 758843 (0.0005) [2023-12-27 10:52:47,017][20381] Updated weights for policy 1, policy_version 758559 (0.0008) [2023-12-27 10:52:47,086][20347] Updated weights for policy 0, policy_version 758853 (0.0007) [2023-12-27 10:52:47,088][20381] Updated weights for policy 1, policy_version 758569 (0.0007) [2023-12-27 10:52:47,146][20381] Updated weights for policy 1, policy_version 758579 (0.0006) [2023-12-27 10:52:47,148][20347] Updated weights for policy 0, policy_version 758863 (0.0005) [2023-12-27 10:52:47,206][20347] Updated weights for policy 0, policy_version 758873 (0.0007) [2023-12-27 10:52:47,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 388521984. Throughput: 0: 10461.7, 1: 10418.4. Samples: 388528124. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:47,364][19170] Avg episode reward: [(0, '450.520'), (1, '892.640')] [2023-12-27 10:52:47,759][20381] Updated weights for policy 1, policy_version 758589 (0.0005) [2023-12-27 10:52:47,803][20381] Updated weights for policy 1, policy_version 758599 (0.0005) [2023-12-27 10:52:47,838][20347] Updated weights for policy 0, policy_version 758883 (0.0006) [2023-12-27 10:52:47,851][20381] Updated weights for policy 1, policy_version 758609 (0.0010) [2023-12-27 10:52:47,892][20347] Updated weights for policy 0, policy_version 758893 (0.0007) [2023-12-27 10:52:47,952][20347] Updated weights for policy 0, policy_version 758903 (0.0007) [2023-12-27 10:52:48,475][20381] Updated weights for policy 1, policy_version 758619 (0.0010) [2023-12-27 10:52:48,522][20381] Updated weights for policy 1, policy_version 758629 (0.0010) [2023-12-27 10:52:48,564][20347] Updated weights for policy 0, policy_version 758913 (0.0008) [2023-12-27 10:52:48,569][20381] Updated weights for policy 1, policy_version 758639 (0.0011) [2023-12-27 10:52:48,623][20347] Updated weights for policy 0, policy_version 758923 (0.0010) [2023-12-27 10:52:48,680][20347] Updated weights for policy 0, policy_version 758933 (0.0010) [2023-12-27 10:52:48,742][20347] Updated weights for policy 0, policy_version 758943 (0.0011) [2023-12-27 10:52:49,301][20381] Updated weights for policy 1, policy_version 758649 (0.0010) [2023-12-27 10:52:49,361][20381] Updated weights for policy 1, policy_version 758659 (0.0010) [2023-12-27 10:52:49,422][20381] Updated weights for policy 1, policy_version 758669 (0.0007) [2023-12-27 10:52:49,464][20347] Updated weights for policy 0, policy_version 758953 (0.0011) [2023-12-27 10:52:49,477][20381] Updated weights for policy 1, policy_version 758679 (0.0005) [2023-12-27 10:52:49,522][20347] Updated weights for policy 0, policy_version 758963 (0.0010) [2023-12-27 10:52:49,585][20347] Updated weights for policy 0, policy_version 758973 (0.0010) [2023-12-27 10:52:50,168][20381] Updated weights for policy 1, policy_version 758689 (0.0010) [2023-12-27 10:52:50,227][20381] Updated weights for policy 1, policy_version 758699 (0.0010) [2023-12-27 10:52:50,290][20381] Updated weights for policy 1, policy_version 758709 (0.0011) [2023-12-27 10:52:50,293][20347] Updated weights for policy 0, policy_version 758983 (0.0008) [2023-12-27 10:52:50,345][20347] Updated weights for policy 0, policy_version 758993 (0.0008) [2023-12-27 10:52:50,403][20347] Updated weights for policy 0, policy_version 759003 (0.0007) [2023-12-27 10:52:50,974][20381] Updated weights for policy 1, policy_version 758719 (0.0007) [2023-12-27 10:52:51,035][20381] Updated weights for policy 1, policy_version 758729 (0.0006) [2023-12-27 10:52:51,098][20347] Updated weights for policy 0, policy_version 759013 (0.0010) [2023-12-27 10:52:51,098][20381] Updated weights for policy 1, policy_version 758739 (0.0010) [2023-12-27 10:52:51,162][20347] Updated weights for policy 0, policy_version 759023 (0.0009) [2023-12-27 10:52:51,227][20347] Updated weights for policy 0, policy_version 759033 (0.0006) [2023-12-27 10:52:51,764][20381] Updated weights for policy 1, policy_version 758749 (0.0010) [2023-12-27 10:52:51,822][20381] Updated weights for policy 1, policy_version 758759 (0.0009) [2023-12-27 10:52:51,874][20347] Updated weights for policy 0, policy_version 759043 (0.0010) [2023-12-27 10:52:51,881][20381] Updated weights for policy 1, policy_version 758769 (0.0010) [2023-12-27 10:52:51,927][20347] Updated weights for policy 0, policy_version 759053 (0.0009) [2023-12-27 10:52:51,986][20347] Updated weights for policy 0, policy_version 759063 (0.0006) [2023-12-27 10:52:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20889.5, 300 sec: 20660.5). Total num frames: 388620288. Throughput: 0: 10455.1, 1: 10403.7. Samples: 388590500. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:52,363][19170] Avg episode reward: [(0, '296.250'), (1, '873.330')] [2023-12-27 10:52:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000758776_194273280.pth... [2023-12-27 10:52:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000759072_194347008.pth... [2023-12-27 10:52:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000757560_193961984.pth [2023-12-27 10:52:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000757856_194035712.pth [2023-12-27 10:52:52,592][20381] Updated weights for policy 1, policy_version 758779 (0.0010) [2023-12-27 10:52:52,653][20381] Updated weights for policy 1, policy_version 758789 (0.0007) [2023-12-27 10:52:52,670][20347] Updated weights for policy 0, policy_version 759073 (0.0005) [2023-12-27 10:52:52,717][20381] Updated weights for policy 1, policy_version 758799 (0.0006) [2023-12-27 10:52:52,725][20347] Updated weights for policy 0, policy_version 759083 (0.0006) [2023-12-27 10:52:52,778][20347] Updated weights for policy 0, policy_version 759093 (0.0006) [2023-12-27 10:52:52,840][20347] Updated weights for policy 0, policy_version 759103 (0.0007) [2023-12-27 10:52:53,388][20381] Updated weights for policy 1, policy_version 758809 (0.0007) [2023-12-27 10:52:53,441][20381] Updated weights for policy 1, policy_version 758819 (0.0007) [2023-12-27 10:52:53,453][20347] Updated weights for policy 0, policy_version 759113 (0.0007) [2023-12-27 10:52:53,492][20381] Updated weights for policy 1, policy_version 758829 (0.0006) [2023-12-27 10:52:53,500][20347] Updated weights for policy 0, policy_version 759123 (0.0006) [2023-12-27 10:52:53,550][20381] Updated weights for policy 1, policy_version 758839 (0.0007) [2023-12-27 10:52:53,556][20347] Updated weights for policy 0, policy_version 759133 (0.0005) [2023-12-27 10:52:54,130][20381] Updated weights for policy 1, policy_version 758849 (0.0008) [2023-12-27 10:52:54,136][20347] Updated weights for policy 0, policy_version 759143 (0.0005) [2023-12-27 10:52:54,179][20381] Updated weights for policy 1, policy_version 758859 (0.0008) [2023-12-27 10:52:54,192][20347] Updated weights for policy 0, policy_version 759153 (0.0006) [2023-12-27 10:52:54,231][20381] Updated weights for policy 1, policy_version 758869 (0.0005) [2023-12-27 10:52:54,238][20347] Updated weights for policy 0, policy_version 759163 (0.0007) [2023-12-27 10:52:54,824][20381] Updated weights for policy 1, policy_version 758879 (0.0005) [2023-12-27 10:52:54,885][20381] Updated weights for policy 1, policy_version 758889 (0.0005) [2023-12-27 10:52:54,886][20347] Updated weights for policy 0, policy_version 759173 (0.0006) [2023-12-27 10:52:54,946][20381] Updated weights for policy 1, policy_version 758899 (0.0007) [2023-12-27 10:52:54,947][20347] Updated weights for policy 0, policy_version 759183 (0.0005) [2023-12-27 10:52:55,011][20347] Updated weights for policy 0, policy_version 759193 (0.0006) [2023-12-27 10:52:55,473][20381] Updated weights for policy 1, policy_version 758909 (0.0010) [2023-12-27 10:52:55,531][20381] Updated weights for policy 1, policy_version 758919 (0.0010) [2023-12-27 10:52:55,588][20381] Updated weights for policy 1, policy_version 758929 (0.0010) [2023-12-27 10:52:55,589][20347] Updated weights for policy 0, policy_version 759203 (0.0009) [2023-12-27 10:52:55,641][20347] Updated weights for policy 0, policy_version 759213 (0.0009) [2023-12-27 10:52:55,704][20347] Updated weights for policy 0, policy_version 759223 (0.0007) [2023-12-27 10:52:56,257][20381] Updated weights for policy 1, policy_version 758939 (0.0009) [2023-12-27 10:52:56,304][20381] Updated weights for policy 1, policy_version 758949 (0.0005) [2023-12-27 10:52:56,334][20347] Updated weights for policy 0, policy_version 759233 (0.0010) [2023-12-27 10:52:56,352][20381] Updated weights for policy 1, policy_version 758959 (0.0005) [2023-12-27 10:52:56,394][20347] Updated weights for policy 0, policy_version 759243 (0.0010) [2023-12-27 10:52:56,448][20347] Updated weights for policy 0, policy_version 759253 (0.0010) [2023-12-27 10:52:56,505][20347] Updated weights for policy 0, policy_version 759263 (0.0010) [2023-12-27 10:52:56,888][20381] Updated weights for policy 1, policy_version 758969 (0.0005) [2023-12-27 10:52:56,946][20381] Updated weights for policy 1, policy_version 758979 (0.0005) [2023-12-27 10:52:57,011][20381] Updated weights for policy 1, policy_version 758989 (0.0009) [2023-12-27 10:52:57,069][20381] Updated weights for policy 1, policy_version 758999 (0.0011) [2023-12-27 10:52:57,188][20347] Updated weights for policy 0, policy_version 759273 (0.0010) [2023-12-27 10:52:57,241][20347] Updated weights for policy 0, policy_version 759283 (0.0010) [2023-12-27 10:52:57,303][20347] Updated weights for policy 0, policy_version 759293 (0.0011) [2023-12-27 10:52:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 21026.1, 300 sec: 20716.0). Total num frames: 388734976. Throughput: 0: 10504.1, 1: 10437.2. Samples: 388718964. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:52:57,363][19170] Avg episode reward: [(0, '637.470'), (1, '921.920')] [2023-12-27 10:52:57,789][20381] Updated weights for policy 1, policy_version 759009 (0.0010) [2023-12-27 10:52:57,837][20381] Updated weights for policy 1, policy_version 759019 (0.0010) [2023-12-27 10:52:57,888][20381] Updated weights for policy 1, policy_version 759029 (0.0010) [2023-12-27 10:52:57,920][20347] Updated weights for policy 0, policy_version 759303 (0.0007) [2023-12-27 10:52:57,976][20347] Updated weights for policy 0, policy_version 759313 (0.0006) [2023-12-27 10:52:58,036][20347] Updated weights for policy 0, policy_version 759323 (0.0010) [2023-12-27 10:52:58,604][20381] Updated weights for policy 1, policy_version 759039 (0.0008) [2023-12-27 10:52:58,668][20381] Updated weights for policy 1, policy_version 759049 (0.0011) [2023-12-27 10:52:58,716][20347] Updated weights for policy 0, policy_version 759333 (0.0010) [2023-12-27 10:52:58,734][20381] Updated weights for policy 1, policy_version 759059 (0.0010) [2023-12-27 10:52:58,780][20347] Updated weights for policy 0, policy_version 759343 (0.0009) [2023-12-27 10:52:58,838][20347] Updated weights for policy 0, policy_version 759353 (0.0008) [2023-12-27 10:52:59,483][20381] Updated weights for policy 1, policy_version 759069 (0.0009) [2023-12-27 10:52:59,532][20381] Updated weights for policy 1, policy_version 759079 (0.0008) [2023-12-27 10:52:59,557][20347] Updated weights for policy 0, policy_version 759363 (0.0007) [2023-12-27 10:52:59,587][20381] Updated weights for policy 1, policy_version 759089 (0.0007) [2023-12-27 10:52:59,613][20347] Updated weights for policy 0, policy_version 759373 (0.0007) [2023-12-27 10:52:59,668][20347] Updated weights for policy 0, policy_version 759383 (0.0006) [2023-12-27 10:53:00,258][20381] Updated weights for policy 1, policy_version 759099 (0.0006) [2023-12-27 10:53:00,314][20381] Updated weights for policy 1, policy_version 759109 (0.0005) [2023-12-27 10:53:00,364][20347] Updated weights for policy 0, policy_version 759393 (0.0008) [2023-12-27 10:53:00,372][20381] Updated weights for policy 1, policy_version 759119 (0.0007) [2023-12-27 10:53:00,423][20347] Updated weights for policy 0, policy_version 759403 (0.0008) [2023-12-27 10:53:00,477][20347] Updated weights for policy 0, policy_version 759413 (0.0006) [2023-12-27 10:53:00,531][20347] Updated weights for policy 0, policy_version 759423 (0.0007) [2023-12-27 10:53:01,030][20381] Updated weights for policy 1, policy_version 759129 (0.0006) [2023-12-27 10:53:01,088][20381] Updated weights for policy 1, policy_version 759139 (0.0009) [2023-12-27 10:53:01,151][20381] Updated weights for policy 1, policy_version 759149 (0.0007) [2023-12-27 10:53:01,172][20347] Updated weights for policy 0, policy_version 759433 (0.0008) [2023-12-27 10:53:01,203][20381] Updated weights for policy 1, policy_version 759159 (0.0008) [2023-12-27 10:53:01,227][20347] Updated weights for policy 0, policy_version 759443 (0.0008) [2023-12-27 10:53:01,294][20347] Updated weights for policy 0, policy_version 759453 (0.0009) [2023-12-27 10:53:01,919][20381] Updated weights for policy 1, policy_version 759169 (0.0007) [2023-12-27 10:53:01,978][20381] Updated weights for policy 1, policy_version 759179 (0.0008) [2023-12-27 10:53:01,989][20347] Updated weights for policy 0, policy_version 759463 (0.0009) [2023-12-27 10:53:02,027][20381] Updated weights for policy 1, policy_version 759189 (0.0006) [2023-12-27 10:53:02,037][20347] Updated weights for policy 0, policy_version 759473 (0.0010) [2023-12-27 10:53:02,082][20347] Updated weights for policy 0, policy_version 759483 (0.0010) [2023-12-27 10:53:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 21026.2, 300 sec: 20660.5). Total num frames: 388833280. Throughput: 0: 10487.1, 1: 10406.5. Samples: 388841624. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:02,363][19170] Avg episode reward: [(0, '363.890'), (1, '912.350')] [2023-12-27 10:53:02,685][20381] Updated weights for policy 1, policy_version 759199 (0.0007) [2023-12-27 10:53:02,755][20381] Updated weights for policy 1, policy_version 759209 (0.0008) [2023-12-27 10:53:02,762][20347] Updated weights for policy 0, policy_version 759493 (0.0008) [2023-12-27 10:53:02,810][20381] Updated weights for policy 1, policy_version 759219 (0.0007) [2023-12-27 10:53:02,826][20347] Updated weights for policy 0, policy_version 759503 (0.0006) [2023-12-27 10:53:02,877][20347] Updated weights for policy 0, policy_version 759513 (0.0005) [2023-12-27 10:53:03,391][20347] Updated weights for policy 0, policy_version 759523 (0.0006) [2023-12-27 10:53:03,451][20347] Updated weights for policy 0, policy_version 759533 (0.0010) [2023-12-27 10:53:03,502][20347] Updated weights for policy 0, policy_version 759543 (0.0006) [2023-12-27 10:53:03,541][20381] Updated weights for policy 1, policy_version 759229 (0.0008) [2023-12-27 10:53:03,593][20381] Updated weights for policy 1, policy_version 759239 (0.0007) [2023-12-27 10:53:03,637][20381] Updated weights for policy 1, policy_version 759249 (0.0008) [2023-12-27 10:53:04,118][20347] Updated weights for policy 0, policy_version 759553 (0.0006) [2023-12-27 10:53:04,185][20347] Updated weights for policy 0, policy_version 759563 (0.0010) [2023-12-27 10:53:04,244][20347] Updated weights for policy 0, policy_version 759573 (0.0010) [2023-12-27 10:53:04,300][20347] Updated weights for policy 0, policy_version 759583 (0.0011) [2023-12-27 10:53:04,371][20381] Updated weights for policy 1, policy_version 759259 (0.0007) [2023-12-27 10:53:04,429][20381] Updated weights for policy 1, policy_version 759269 (0.0008) [2023-12-27 10:53:04,483][20381] Updated weights for policy 1, policy_version 759279 (0.0008) [2023-12-27 10:53:04,984][20347] Updated weights for policy 0, policy_version 759593 (0.0010) [2023-12-27 10:53:05,040][20347] Updated weights for policy 0, policy_version 759603 (0.0010) [2023-12-27 10:53:05,100][20347] Updated weights for policy 0, policy_version 759613 (0.0010) [2023-12-27 10:53:05,177][20381] Updated weights for policy 1, policy_version 759289 (0.0008) [2023-12-27 10:53:05,228][20381] Updated weights for policy 1, policy_version 759299 (0.0009) [2023-12-27 10:53:05,287][20381] Updated weights for policy 1, policy_version 759309 (0.0007) [2023-12-27 10:53:05,333][20381] Updated weights for policy 1, policy_version 759319 (0.0008) [2023-12-27 10:53:05,652][20347] Updated weights for policy 0, policy_version 759623 (0.0007) [2023-12-27 10:53:05,703][20347] Updated weights for policy 0, policy_version 759633 (0.0005) [2023-12-27 10:53:05,752][20347] Updated weights for policy 0, policy_version 759643 (0.0007) [2023-12-27 10:53:06,090][20381] Updated weights for policy 1, policy_version 759329 (0.0008) [2023-12-27 10:53:06,151][20381] Updated weights for policy 1, policy_version 759339 (0.0008) [2023-12-27 10:53:06,209][20381] Updated weights for policy 1, policy_version 759349 (0.0007) [2023-12-27 10:53:06,423][20347] Updated weights for policy 0, policy_version 759653 (0.0007) [2023-12-27 10:53:06,483][20347] Updated weights for policy 0, policy_version 759663 (0.0008) [2023-12-27 10:53:06,547][20347] Updated weights for policy 0, policy_version 759673 (0.0007) [2023-12-27 10:53:06,929][20381] Updated weights for policy 1, policy_version 759359 (0.0007) [2023-12-27 10:53:06,994][20381] Updated weights for policy 1, policy_version 759369 (0.0008) [2023-12-27 10:53:07,061][20381] Updated weights for policy 1, policy_version 759379 (0.0008) [2023-12-27 10:53:07,241][20347] Updated weights for policy 0, policy_version 759683 (0.0008) [2023-12-27 10:53:07,293][20347] Updated weights for policy 0, policy_version 759693 (0.0007) [2023-12-27 10:53:07,347][20347] Updated weights for policy 0, policy_version 759703 (0.0008) [2023-12-27 10:53:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 388931584. Throughput: 0: 10533.4, 1: 10350.6. Samples: 388904236. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:07,363][19170] Avg episode reward: [(0, '589.000'), (1, '902.690')] [2023-12-27 10:53:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000759384_194428928.pth... [2023-12-27 10:53:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000758168_194117632.pth [2023-12-27 10:53:07,396][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000759712_194510848.pth... [2023-12-27 10:53:07,400][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000758464_194191360.pth [2023-12-27 10:53:07,715][20381] Updated weights for policy 1, policy_version 759389 (0.0010) [2023-12-27 10:53:07,772][20381] Updated weights for policy 1, policy_version 759399 (0.0011) [2023-12-27 10:53:07,832][20381] Updated weights for policy 1, policy_version 759409 (0.0010) [2023-12-27 10:53:08,015][20347] Updated weights for policy 0, policy_version 759713 (0.0007) [2023-12-27 10:53:08,073][20347] Updated weights for policy 0, policy_version 759723 (0.0005) [2023-12-27 10:53:08,129][20347] Updated weights for policy 0, policy_version 759733 (0.0007) [2023-12-27 10:53:08,179][20347] Updated weights for policy 0, policy_version 759743 (0.0007) [2023-12-27 10:53:08,488][20381] Updated weights for policy 1, policy_version 759419 (0.0011) [2023-12-27 10:53:08,541][20381] Updated weights for policy 1, policy_version 759429 (0.0011) [2023-12-27 10:53:08,594][20381] Updated weights for policy 1, policy_version 759439 (0.0010) [2023-12-27 10:53:08,878][20347] Updated weights for policy 0, policy_version 759753 (0.0006) [2023-12-27 10:53:08,933][20347] Updated weights for policy 0, policy_version 759763 (0.0007) [2023-12-27 10:53:08,984][20347] Updated weights for policy 0, policy_version 759773 (0.0007) [2023-12-27 10:53:09,298][20381] Updated weights for policy 1, policy_version 759449 (0.0011) [2023-12-27 10:53:09,361][20381] Updated weights for policy 1, policy_version 759459 (0.0010) [2023-12-27 10:53:09,428][20381] Updated weights for policy 1, policy_version 759469 (0.0013) [2023-12-27 10:53:09,489][20381] Updated weights for policy 1, policy_version 759479 (0.0006) [2023-12-27 10:53:09,690][20347] Updated weights for policy 0, policy_version 759783 (0.0008) [2023-12-27 10:53:09,758][20347] Updated weights for policy 0, policy_version 759793 (0.0008) [2023-12-27 10:53:09,825][20347] Updated weights for policy 0, policy_version 759803 (0.0007) [2023-12-27 10:53:10,175][20381] Updated weights for policy 1, policy_version 759489 (0.0009) [2023-12-27 10:53:10,234][20381] Updated weights for policy 1, policy_version 759499 (0.0009) [2023-12-27 10:53:10,291][20381] Updated weights for policy 1, policy_version 759509 (0.0008) [2023-12-27 10:53:10,521][20347] Updated weights for policy 0, policy_version 759813 (0.0007) [2023-12-27 10:53:10,577][20347] Updated weights for policy 0, policy_version 759823 (0.0007) [2023-12-27 10:53:10,632][20347] Updated weights for policy 0, policy_version 759833 (0.0006) [2023-12-27 10:53:10,949][20381] Updated weights for policy 1, policy_version 759519 (0.0006) [2023-12-27 10:53:10,998][20381] Updated weights for policy 1, policy_version 759529 (0.0007) [2023-12-27 10:53:11,055][20381] Updated weights for policy 1, policy_version 759539 (0.0009) [2023-12-27 10:53:11,366][20347] Updated weights for policy 0, policy_version 759843 (0.0008) [2023-12-27 10:53:11,431][20347] Updated weights for policy 0, policy_version 759853 (0.0008) [2023-12-27 10:53:11,492][20347] Updated weights for policy 0, policy_version 759863 (0.0008) [2023-12-27 10:53:11,792][20381] Updated weights for policy 1, policy_version 759549 (0.0008) [2023-12-27 10:53:11,849][20381] Updated weights for policy 1, policy_version 759559 (0.0006) [2023-12-27 10:53:11,911][20381] Updated weights for policy 1, policy_version 759569 (0.0007) [2023-12-27 10:53:12,227][20347] Updated weights for policy 0, policy_version 759873 (0.0009) [2023-12-27 10:53:12,292][20347] Updated weights for policy 0, policy_version 759883 (0.0007) [2023-12-27 10:53:12,355][20347] Updated weights for policy 0, policy_version 759893 (0.0007) [2023-12-27 10:53:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 389029888. Throughput: 0: 10485.4, 1: 10303.0. Samples: 389025232. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:12,363][19170] Avg episode reward: [(0, '659.110'), (1, '931.730')] [2023-12-27 10:53:12,417][20347] Updated weights for policy 0, policy_version 759903 (0.0007) [2023-12-27 10:53:12,633][20381] Updated weights for policy 1, policy_version 759579 (0.0008) [2023-12-27 10:53:12,684][20381] Updated weights for policy 1, policy_version 759589 (0.0007) [2023-12-27 10:53:12,738][20381] Updated weights for policy 1, policy_version 759599 (0.0008) [2023-12-27 10:53:13,142][20347] Updated weights for policy 0, policy_version 759913 (0.0008) [2023-12-27 10:53:13,194][20347] Updated weights for policy 0, policy_version 759923 (0.0009) [2023-12-27 10:53:13,259][20347] Updated weights for policy 0, policy_version 759933 (0.0008) [2023-12-27 10:53:13,433][20381] Updated weights for policy 1, policy_version 759609 (0.0008) [2023-12-27 10:53:13,486][20381] Updated weights for policy 1, policy_version 759619 (0.0008) [2023-12-27 10:53:13,533][20381] Updated weights for policy 1, policy_version 759629 (0.0008) [2023-12-27 10:53:13,586][20381] Updated weights for policy 1, policy_version 759639 (0.0008) [2023-12-27 10:53:13,854][20347] Updated weights for policy 0, policy_version 759943 (0.0008) [2023-12-27 10:53:13,905][20347] Updated weights for policy 0, policy_version 759953 (0.0007) [2023-12-27 10:53:13,950][20347] Updated weights for policy 0, policy_version 759963 (0.0007) [2023-12-27 10:53:14,290][20381] Updated weights for policy 1, policy_version 759649 (0.0010) [2023-12-27 10:53:14,341][20381] Updated weights for policy 1, policy_version 759659 (0.0011) [2023-12-27 10:53:14,399][20381] Updated weights for policy 1, policy_version 759669 (0.0010) [2023-12-27 10:53:14,671][20347] Updated weights for policy 0, policy_version 759973 (0.0007) [2023-12-27 10:53:14,728][20347] Updated weights for policy 0, policy_version 759983 (0.0008) [2023-12-27 10:53:14,795][20347] Updated weights for policy 0, policy_version 759993 (0.0007) [2023-12-27 10:53:15,085][20381] Updated weights for policy 1, policy_version 759679 (0.0010) [2023-12-27 10:53:15,149][20381] Updated weights for policy 1, policy_version 759689 (0.0010) [2023-12-27 10:53:15,209][20381] Updated weights for policy 1, policy_version 759699 (0.0011) [2023-12-27 10:53:15,506][20347] Updated weights for policy 0, policy_version 760003 (0.0007) [2023-12-27 10:53:15,565][20347] Updated weights for policy 0, policy_version 760013 (0.0007) [2023-12-27 10:53:15,616][20347] Updated weights for policy 0, policy_version 760023 (0.0007) [2023-12-27 10:53:15,842][20381] Updated weights for policy 1, policy_version 759709 (0.0009) [2023-12-27 10:53:15,895][20381] Updated weights for policy 1, policy_version 759719 (0.0009) [2023-12-27 10:53:15,941][20381] Updated weights for policy 1, policy_version 759729 (0.0006) [2023-12-27 10:53:16,225][20347] Updated weights for policy 0, policy_version 760033 (0.0008) [2023-12-27 10:53:16,280][20347] Updated weights for policy 0, policy_version 760043 (0.0005) [2023-12-27 10:53:16,333][20347] Updated weights for policy 0, policy_version 760053 (0.0005) [2023-12-27 10:53:16,389][20347] Updated weights for policy 0, policy_version 760063 (0.0005) [2023-12-27 10:53:16,616][20381] Updated weights for policy 1, policy_version 759739 (0.0006) [2023-12-27 10:53:16,675][20381] Updated weights for policy 1, policy_version 759749 (0.0007) [2023-12-27 10:53:16,735][20381] Updated weights for policy 1, policy_version 759759 (0.0009) [2023-12-27 10:53:16,950][20347] Updated weights for policy 0, policy_version 760073 (0.0007) [2023-12-27 10:53:17,004][20347] Updated weights for policy 0, policy_version 760083 (0.0006) [2023-12-27 10:53:17,061][20347] Updated weights for policy 0, policy_version 760093 (0.0007) [2023-12-27 10:53:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 389136384. Throughput: 0: 10456.5, 1: 10309.0. Samples: 389147764. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:17,363][19170] Avg episode reward: [(0, '512.080'), (1, '882.700')] [2023-12-27 10:53:17,369][20381] Updated weights for policy 1, policy_version 759769 (0.0006) [2023-12-27 10:53:17,413][20381] Updated weights for policy 1, policy_version 759779 (0.0008) [2023-12-27 10:53:17,461][20381] Updated weights for policy 1, policy_version 759789 (0.0007) [2023-12-27 10:53:17,520][20381] Updated weights for policy 1, policy_version 759799 (0.0008) [2023-12-27 10:53:17,733][20347] Updated weights for policy 0, policy_version 760103 (0.0006) [2023-12-27 10:53:17,780][20347] Updated weights for policy 0, policy_version 760113 (0.0007) [2023-12-27 10:53:17,834][20347] Updated weights for policy 0, policy_version 760123 (0.0007) [2023-12-27 10:53:18,242][20381] Updated weights for policy 1, policy_version 759809 (0.0007) [2023-12-27 10:53:18,304][20381] Updated weights for policy 1, policy_version 759819 (0.0007) [2023-12-27 10:53:18,367][20381] Updated weights for policy 1, policy_version 759829 (0.0008) [2023-12-27 10:53:18,578][20347] Updated weights for policy 0, policy_version 760133 (0.0008) [2023-12-27 10:53:18,638][20347] Updated weights for policy 0, policy_version 760143 (0.0008) [2023-12-27 10:53:18,690][20347] Updated weights for policy 0, policy_version 760153 (0.0006) [2023-12-27 10:53:18,968][20381] Updated weights for policy 1, policy_version 759839 (0.0006) [2023-12-27 10:53:19,031][20381] Updated weights for policy 1, policy_version 759849 (0.0005) [2023-12-27 10:53:19,093][20381] Updated weights for policy 1, policy_version 759859 (0.0006) [2023-12-27 10:53:19,362][20347] Updated weights for policy 0, policy_version 760163 (0.0006) [2023-12-27 10:53:19,424][20347] Updated weights for policy 0, policy_version 760173 (0.0006) [2023-12-27 10:53:19,481][20347] Updated weights for policy 0, policy_version 760183 (0.0006) [2023-12-27 10:53:19,806][20381] Updated weights for policy 1, policy_version 759869 (0.0007) [2023-12-27 10:53:19,872][20381] Updated weights for policy 1, policy_version 759879 (0.0008) [2023-12-27 10:53:19,938][20381] Updated weights for policy 1, policy_version 759889 (0.0008) [2023-12-27 10:53:20,117][20347] Updated weights for policy 0, policy_version 760193 (0.0007) [2023-12-27 10:53:20,180][20347] Updated weights for policy 0, policy_version 760203 (0.0008) [2023-12-27 10:53:20,237][20347] Updated weights for policy 0, policy_version 760213 (0.0007) [2023-12-27 10:53:20,287][20347] Updated weights for policy 0, policy_version 760223 (0.0008) [2023-12-27 10:53:20,665][20381] Updated weights for policy 1, policy_version 759899 (0.0008) [2023-12-27 10:53:20,725][20381] Updated weights for policy 1, policy_version 759909 (0.0008) [2023-12-27 10:53:20,774][20381] Updated weights for policy 1, policy_version 759919 (0.0008) [2023-12-27 10:53:21,011][20347] Updated weights for policy 0, policy_version 760233 (0.0010) [2023-12-27 10:53:21,072][20347] Updated weights for policy 0, policy_version 760243 (0.0009) [2023-12-27 10:53:21,139][20347] Updated weights for policy 0, policy_version 760253 (0.0009) [2023-12-27 10:53:21,527][20381] Updated weights for policy 1, policy_version 759929 (0.0008) [2023-12-27 10:53:21,580][20381] Updated weights for policy 1, policy_version 759939 (0.0010) [2023-12-27 10:53:21,642][20381] Updated weights for policy 1, policy_version 759949 (0.0010) [2023-12-27 10:53:21,711][20381] Updated weights for policy 1, policy_version 759959 (0.0009) [2023-12-27 10:53:21,896][20347] Updated weights for policy 0, policy_version 760263 (0.0007) [2023-12-27 10:53:21,960][20347] Updated weights for policy 0, policy_version 760273 (0.0008) [2023-12-27 10:53:22,024][20347] Updated weights for policy 0, policy_version 760283 (0.0008) [2023-12-27 10:53:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 389234688. Throughput: 0: 10402.5, 1: 10227.1. Samples: 389208404. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:22,363][19170] Avg episode reward: [(0, '560.860'), (1, '873.130')] [2023-12-27 10:53:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000759960_194576384.pth... [2023-12-27 10:53:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000760288_194658304.pth... [2023-12-27 10:53:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000758776_194273280.pth [2023-12-27 10:53:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000759072_194347008.pth [2023-12-27 10:53:22,523][20381] Updated weights for policy 1, policy_version 759969 (0.0008) [2023-12-27 10:53:22,589][20381] Updated weights for policy 1, policy_version 759979 (0.0008) [2023-12-27 10:53:22,653][20381] Updated weights for policy 1, policy_version 759989 (0.0008) [2023-12-27 10:53:22,677][20347] Updated weights for policy 0, policy_version 760293 (0.0006) [2023-12-27 10:53:22,740][20347] Updated weights for policy 0, policy_version 760303 (0.0006) [2023-12-27 10:53:22,801][20347] Updated weights for policy 0, policy_version 760313 (0.0006) [2023-12-27 10:53:23,358][20381] Updated weights for policy 1, policy_version 759999 (0.0009) [2023-12-27 10:53:23,414][20381] Updated weights for policy 1, policy_version 760009 (0.0011) [2023-12-27 10:53:23,438][20347] Updated weights for policy 0, policy_version 760323 (0.0006) [2023-12-27 10:53:23,459][20381] Updated weights for policy 1, policy_version 760019 (0.0010) [2023-12-27 10:53:23,522][20347] Updated weights for policy 0, policy_version 760333 (0.0006) [2023-12-27 10:53:23,580][20347] Updated weights for policy 0, policy_version 760343 (0.0007) [2023-12-27 10:53:24,201][20381] Updated weights for policy 1, policy_version 760029 (0.0008) [2023-12-27 10:53:24,208][20347] Updated weights for policy 0, policy_version 760353 (0.0008) [2023-12-27 10:53:24,251][20381] Updated weights for policy 1, policy_version 760039 (0.0006) [2023-12-27 10:53:24,260][20347] Updated weights for policy 0, policy_version 760363 (0.0009) [2023-12-27 10:53:24,297][20381] Updated weights for policy 1, policy_version 760049 (0.0006) [2023-12-27 10:53:24,309][20347] Updated weights for policy 0, policy_version 760373 (0.0006) [2023-12-27 10:53:24,366][20347] Updated weights for policy 0, policy_version 760383 (0.0005) [2023-12-27 10:53:24,955][20347] Updated weights for policy 0, policy_version 760393 (0.0006) [2023-12-27 10:53:24,959][20381] Updated weights for policy 1, policy_version 760059 (0.0007) [2023-12-27 10:53:25,015][20347] Updated weights for policy 0, policy_version 760403 (0.0008) [2023-12-27 10:53:25,015][20381] Updated weights for policy 1, policy_version 760069 (0.0005) [2023-12-27 10:53:25,063][20381] Updated weights for policy 1, policy_version 760079 (0.0005) [2023-12-27 10:53:25,071][20347] Updated weights for policy 0, policy_version 760413 (0.0006) [2023-12-27 10:53:25,623][20381] Updated weights for policy 1, policy_version 760089 (0.0006) [2023-12-27 10:53:25,656][20347] Updated weights for policy 0, policy_version 760423 (0.0007) [2023-12-27 10:53:25,687][20381] Updated weights for policy 1, policy_version 760099 (0.0007) [2023-12-27 10:53:25,707][20347] Updated weights for policy 0, policy_version 760433 (0.0006) [2023-12-27 10:53:25,745][20381] Updated weights for policy 1, policy_version 760109 (0.0008) [2023-12-27 10:53:25,761][20347] Updated weights for policy 0, policy_version 760443 (0.0006) [2023-12-27 10:53:25,798][20381] Updated weights for policy 1, policy_version 760119 (0.0007) [2023-12-27 10:53:26,328][20347] Updated weights for policy 0, policy_version 760453 (0.0006) [2023-12-27 10:53:26,386][20347] Updated weights for policy 0, policy_version 760463 (0.0005) [2023-12-27 10:53:26,443][20347] Updated weights for policy 0, policy_version 760473 (0.0005) [2023-12-27 10:53:26,542][20381] Updated weights for policy 1, policy_version 760129 (0.0008) [2023-12-27 10:53:26,600][20381] Updated weights for policy 1, policy_version 760139 (0.0007) [2023-12-27 10:53:26,656][20381] Updated weights for policy 1, policy_version 760149 (0.0007) [2023-12-27 10:53:27,076][20347] Updated weights for policy 0, policy_version 760483 (0.0008) [2023-12-27 10:53:27,120][20347] Updated weights for policy 0, policy_version 760493 (0.0007) [2023-12-27 10:53:27,175][20347] Updated weights for policy 0, policy_version 760503 (0.0007) [2023-12-27 10:53:27,317][20381] Updated weights for policy 1, policy_version 760159 (0.0008) [2023-12-27 10:53:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 389341184. Throughput: 0: 10397.0, 1: 10210.5. Samples: 389330964. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:27,363][19170] Avg episode reward: [(0, '736.290'), (1, '902.340')] [2023-12-27 10:53:27,375][20381] Updated weights for policy 1, policy_version 760169 (0.0006) [2023-12-27 10:53:27,436][20381] Updated weights for policy 1, policy_version 760179 (0.0005) [2023-12-27 10:53:27,864][20347] Updated weights for policy 0, policy_version 760513 (0.0007) [2023-12-27 10:53:27,912][20347] Updated weights for policy 0, policy_version 760523 (0.0007) [2023-12-27 10:53:27,969][20347] Updated weights for policy 0, policy_version 760533 (0.0006) [2023-12-27 10:53:28,013][20347] Updated weights for policy 0, policy_version 760543 (0.0005) [2023-12-27 10:53:28,063][20381] Updated weights for policy 1, policy_version 760189 (0.0006) [2023-12-27 10:53:28,111][20381] Updated weights for policy 1, policy_version 760199 (0.0008) [2023-12-27 10:53:28,159][20381] Updated weights for policy 1, policy_version 760209 (0.0008) [2023-12-27 10:53:28,671][20347] Updated weights for policy 0, policy_version 760553 (0.0007) [2023-12-27 10:53:28,722][20347] Updated weights for policy 0, policy_version 760563 (0.0008) [2023-12-27 10:53:28,776][20347] Updated weights for policy 0, policy_version 760573 (0.0006) [2023-12-27 10:53:28,902][20381] Updated weights for policy 1, policy_version 760219 (0.0007) [2023-12-27 10:53:28,959][20381] Updated weights for policy 1, policy_version 760229 (0.0007) [2023-12-27 10:53:29,009][20381] Updated weights for policy 1, policy_version 760239 (0.0007) [2023-12-27 10:53:29,491][20347] Updated weights for policy 0, policy_version 760583 (0.0008) [2023-12-27 10:53:29,543][20347] Updated weights for policy 0, policy_version 760593 (0.0008) [2023-12-27 10:53:29,598][20347] Updated weights for policy 0, policy_version 760603 (0.0007) [2023-12-27 10:53:29,708][20381] Updated weights for policy 1, policy_version 760249 (0.0006) [2023-12-27 10:53:29,756][20381] Updated weights for policy 1, policy_version 760259 (0.0007) [2023-12-27 10:53:29,806][20381] Updated weights for policy 1, policy_version 760269 (0.0007) [2023-12-27 10:53:29,869][20381] Updated weights for policy 1, policy_version 760279 (0.0008) [2023-12-27 10:53:30,314][20347] Updated weights for policy 0, policy_version 760613 (0.0007) [2023-12-27 10:53:30,360][20347] Updated weights for policy 0, policy_version 760623 (0.0007) [2023-12-27 10:53:30,409][20347] Updated weights for policy 0, policy_version 760633 (0.0006) [2023-12-27 10:53:30,628][20381] Updated weights for policy 1, policy_version 760289 (0.0008) [2023-12-27 10:53:30,690][20381] Updated weights for policy 1, policy_version 760299 (0.0007) [2023-12-27 10:53:30,743][20381] Updated weights for policy 1, policy_version 760309 (0.0007) [2023-12-27 10:53:31,035][20347] Updated weights for policy 0, policy_version 760643 (0.0007) [2023-12-27 10:53:31,091][20347] Updated weights for policy 0, policy_version 760653 (0.0008) [2023-12-27 10:53:31,154][20347] Updated weights for policy 0, policy_version 760663 (0.0007) [2023-12-27 10:53:31,361][20381] Updated weights for policy 1, policy_version 760319 (0.0007) [2023-12-27 10:53:31,422][20381] Updated weights for policy 1, policy_version 760329 (0.0008) [2023-12-27 10:53:31,470][20381] Updated weights for policy 1, policy_version 760339 (0.0008) [2023-12-27 10:53:31,874][20347] Updated weights for policy 0, policy_version 760673 (0.0007) [2023-12-27 10:53:31,932][20347] Updated weights for policy 0, policy_version 760683 (0.0007) [2023-12-27 10:53:31,991][20347] Updated weights for policy 0, policy_version 760693 (0.0007) [2023-12-27 10:53:32,047][20347] Updated weights for policy 0, policy_version 760703 (0.0006) [2023-12-27 10:53:32,199][20381] Updated weights for policy 1, policy_version 760349 (0.0007) [2023-12-27 10:53:32,248][20381] Updated weights for policy 1, policy_version 760359 (0.0006) [2023-12-27 10:53:32,309][20381] Updated weights for policy 1, policy_version 760369 (0.0008) [2023-12-27 10:53:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 389447680. Throughput: 0: 10407.3, 1: 10160.4. Samples: 389453664. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:32,363][19170] Avg episode reward: [(0, '687.260'), (1, '882.730')] [2023-12-27 10:53:32,772][20347] Updated weights for policy 0, policy_version 760713 (0.0008) [2023-12-27 10:53:32,830][20347] Updated weights for policy 0, policy_version 760723 (0.0007) [2023-12-27 10:53:32,872][20347] Updated weights for policy 0, policy_version 760733 (0.0007) [2023-12-27 10:53:32,950][20381] Updated weights for policy 1, policy_version 760379 (0.0008) [2023-12-27 10:53:33,001][20381] Updated weights for policy 1, policy_version 760389 (0.0008) [2023-12-27 10:53:33,056][20381] Updated weights for policy 1, policy_version 760399 (0.0008) [2023-12-27 10:53:33,567][20347] Updated weights for policy 0, policy_version 760743 (0.0008) [2023-12-27 10:53:33,625][20347] Updated weights for policy 0, policy_version 760753 (0.0008) [2023-12-27 10:53:33,671][20347] Updated weights for policy 0, policy_version 760763 (0.0007) [2023-12-27 10:53:33,731][20381] Updated weights for policy 1, policy_version 760409 (0.0007) [2023-12-27 10:53:33,775][20381] Updated weights for policy 1, policy_version 760419 (0.0008) [2023-12-27 10:53:33,835][20381] Updated weights for policy 1, policy_version 760429 (0.0008) [2023-12-27 10:53:33,885][20381] Updated weights for policy 1, policy_version 760439 (0.0010) [2023-12-27 10:53:34,265][20347] Updated weights for policy 0, policy_version 760773 (0.0008) [2023-12-27 10:53:34,315][20347] Updated weights for policy 0, policy_version 760783 (0.0010) [2023-12-27 10:53:34,361][20347] Updated weights for policy 0, policy_version 760793 (0.0008) [2023-12-27 10:53:34,625][20381] Updated weights for policy 1, policy_version 760449 (0.0009) [2023-12-27 10:53:34,673][20381] Updated weights for policy 1, policy_version 760459 (0.0007) [2023-12-27 10:53:34,723][20381] Updated weights for policy 1, policy_version 760469 (0.0007) [2023-12-27 10:53:35,032][20347] Updated weights for policy 0, policy_version 760803 (0.0008) [2023-12-27 10:53:35,090][20347] Updated weights for policy 0, policy_version 760813 (0.0009) [2023-12-27 10:53:35,138][20347] Updated weights for policy 0, policy_version 760823 (0.0010) [2023-12-27 10:53:35,460][20381] Updated weights for policy 1, policy_version 760479 (0.0008) [2023-12-27 10:53:35,513][20381] Updated weights for policy 1, policy_version 760489 (0.0009) [2023-12-27 10:53:35,562][20381] Updated weights for policy 1, policy_version 760499 (0.0009) [2023-12-27 10:53:35,729][20347] Updated weights for policy 0, policy_version 760833 (0.0010) [2023-12-27 10:53:35,786][20347] Updated weights for policy 0, policy_version 760843 (0.0007) [2023-12-27 10:53:35,844][20347] Updated weights for policy 0, policy_version 760853 (0.0010) [2023-12-27 10:53:35,900][20347] Updated weights for policy 0, policy_version 760863 (0.0009) [2023-12-27 10:53:36,240][20381] Updated weights for policy 1, policy_version 760509 (0.0009) [2023-12-27 10:53:36,302][20381] Updated weights for policy 1, policy_version 760519 (0.0011) [2023-12-27 10:53:36,362][20381] Updated weights for policy 1, policy_version 760529 (0.0011) [2023-12-27 10:53:36,525][20347] Updated weights for policy 0, policy_version 760873 (0.0008) [2023-12-27 10:53:36,587][20347] Updated weights for policy 0, policy_version 760883 (0.0008) [2023-12-27 10:53:36,643][20347] Updated weights for policy 0, policy_version 760893 (0.0007) [2023-12-27 10:53:37,062][20381] Updated weights for policy 1, policy_version 760539 (0.0009) [2023-12-27 10:53:37,122][20381] Updated weights for policy 1, policy_version 760549 (0.0006) [2023-12-27 10:53:37,181][20381] Updated weights for policy 1, policy_version 760559 (0.0005) [2023-12-27 10:53:37,341][20347] Updated weights for policy 0, policy_version 760903 (0.0009) [2023-12-27 10:53:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 389545984. Throughput: 0: 10421.2, 1: 10122.8. Samples: 389514976. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:37,364][19170] Avg episode reward: [(0, '726.960'), (1, '921.860')] [2023-12-27 10:53:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000760568_194732032.pth... [2023-12-27 10:53:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000759384_194428928.pth [2023-12-27 10:53:37,391][20347] Updated weights for policy 0, policy_version 760913 (0.0011) [2023-12-27 10:53:37,440][20347] Updated weights for policy 0, policy_version 760923 (0.0010) [2023-12-27 10:53:37,465][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000760928_194822144.pth... [2023-12-27 10:53:37,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000759712_194510848.pth [2023-12-27 10:53:37,753][20381] Updated weights for policy 1, policy_version 760569 (0.0006) [2023-12-27 10:53:37,808][20381] Updated weights for policy 1, policy_version 760579 (0.0007) [2023-12-27 10:53:37,864][20381] Updated weights for policy 1, policy_version 760589 (0.0007) [2023-12-27 10:53:37,919][20381] Updated weights for policy 1, policy_version 760599 (0.0005) [2023-12-27 10:53:38,062][20347] Updated weights for policy 0, policy_version 760933 (0.0008) [2023-12-27 10:53:38,126][20347] Updated weights for policy 0, policy_version 760943 (0.0005) [2023-12-27 10:53:38,182][20347] Updated weights for policy 0, policy_version 760953 (0.0010) [2023-12-27 10:53:38,557][20381] Updated weights for policy 1, policy_version 760609 (0.0010) [2023-12-27 10:53:38,621][20381] Updated weights for policy 1, policy_version 760619 (0.0011) [2023-12-27 10:53:38,682][20381] Updated weights for policy 1, policy_version 760629 (0.0010) [2023-12-27 10:53:38,827][20347] Updated weights for policy 0, policy_version 760963 (0.0007) [2023-12-27 10:53:38,886][20347] Updated weights for policy 0, policy_version 760973 (0.0008) [2023-12-27 10:53:38,935][20347] Updated weights for policy 0, policy_version 760983 (0.0007) [2023-12-27 10:53:39,373][20381] Updated weights for policy 1, policy_version 760639 (0.0009) [2023-12-27 10:53:39,442][20381] Updated weights for policy 1, policy_version 760649 (0.0009) [2023-12-27 10:53:39,503][20381] Updated weights for policy 1, policy_version 760659 (0.0008) [2023-12-27 10:53:39,639][20347] Updated weights for policy 0, policy_version 760993 (0.0007) [2023-12-27 10:53:39,701][20347] Updated weights for policy 0, policy_version 761003 (0.0008) [2023-12-27 10:53:39,756][20347] Updated weights for policy 0, policy_version 761013 (0.0009) [2023-12-27 10:53:39,810][20347] Updated weights for policy 0, policy_version 761023 (0.0009) [2023-12-27 10:53:40,232][20381] Updated weights for policy 1, policy_version 760669 (0.0008) [2023-12-27 10:53:40,288][20381] Updated weights for policy 1, policy_version 760679 (0.0008) [2023-12-27 10:53:40,342][20381] Updated weights for policy 1, policy_version 760689 (0.0008) [2023-12-27 10:53:40,537][20347] Updated weights for policy 0, policy_version 761033 (0.0008) [2023-12-27 10:53:40,595][20347] Updated weights for policy 0, policy_version 761043 (0.0008) [2023-12-27 10:53:40,652][20347] Updated weights for policy 0, policy_version 761053 (0.0008) [2023-12-27 10:53:40,998][20381] Updated weights for policy 1, policy_version 760699 (0.0008) [2023-12-27 10:53:41,075][20381] Updated weights for policy 1, policy_version 760709 (0.0008) [2023-12-27 10:53:41,135][20381] Updated weights for policy 1, policy_version 760719 (0.0008) [2023-12-27 10:53:41,401][20347] Updated weights for policy 0, policy_version 761063 (0.0010) [2023-12-27 10:53:41,465][20347] Updated weights for policy 0, policy_version 761073 (0.0011) [2023-12-27 10:53:41,523][20347] Updated weights for policy 0, policy_version 761083 (0.0010) [2023-12-27 10:53:41,930][20381] Updated weights for policy 1, policy_version 760729 (0.0007) [2023-12-27 10:53:41,992][20381] Updated weights for policy 1, policy_version 760739 (0.0008) [2023-12-27 10:53:42,049][20381] Updated weights for policy 1, policy_version 760749 (0.0007) [2023-12-27 10:53:42,101][20381] Updated weights for policy 1, policy_version 760759 (0.0008) [2023-12-27 10:53:42,181][20347] Updated weights for policy 0, policy_version 761093 (0.0010) [2023-12-27 10:53:42,237][20347] Updated weights for policy 0, policy_version 761103 (0.0010) [2023-12-27 10:53:42,302][20347] Updated weights for policy 0, policy_version 761113 (0.0010) [2023-12-27 10:53:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 389652480. Throughput: 0: 10384.0, 1: 10052.5. Samples: 389638604. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:42,363][19170] Avg episode reward: [(0, '736.470'), (1, '892.570')] [2023-12-27 10:53:42,838][20381] Updated weights for policy 1, policy_version 760769 (0.0008) [2023-12-27 10:53:42,890][20381] Updated weights for policy 1, policy_version 760779 (0.0008) [2023-12-27 10:53:42,951][20381] Updated weights for policy 1, policy_version 760789 (0.0008) [2023-12-27 10:53:42,996][20347] Updated weights for policy 0, policy_version 761123 (0.0008) [2023-12-27 10:53:43,045][20347] Updated weights for policy 0, policy_version 761133 (0.0005) [2023-12-27 10:53:43,091][20347] Updated weights for policy 0, policy_version 761143 (0.0005) [2023-12-27 10:53:43,656][20347] Updated weights for policy 0, policy_version 761153 (0.0005) [2023-12-27 10:53:43,698][20381] Updated weights for policy 1, policy_version 760799 (0.0006) [2023-12-27 10:53:43,713][20347] Updated weights for policy 0, policy_version 761163 (0.0007) [2023-12-27 10:53:43,764][20381] Updated weights for policy 1, policy_version 760809 (0.0006) [2023-12-27 10:53:43,770][20347] Updated weights for policy 0, policy_version 761173 (0.0008) [2023-12-27 10:53:43,819][20381] Updated weights for policy 1, policy_version 760819 (0.0006) [2023-12-27 10:53:43,825][20347] Updated weights for policy 0, policy_version 761183 (0.0005) [2023-12-27 10:53:44,438][20347] Updated weights for policy 0, policy_version 761193 (0.0007) [2023-12-27 10:53:44,492][20347] Updated weights for policy 0, policy_version 761203 (0.0007) [2023-12-27 10:53:44,509][20381] Updated weights for policy 1, policy_version 760829 (0.0006) [2023-12-27 10:53:44,541][20347] Updated weights for policy 0, policy_version 761213 (0.0005) [2023-12-27 10:53:44,562][20381] Updated weights for policy 1, policy_version 760839 (0.0007) [2023-12-27 10:53:44,618][20381] Updated weights for policy 1, policy_version 760849 (0.0008) [2023-12-27 10:53:45,268][20347] Updated weights for policy 0, policy_version 761223 (0.0007) [2023-12-27 10:53:45,334][20381] Updated weights for policy 1, policy_version 760859 (0.0007) [2023-12-27 10:53:45,334][20347] Updated weights for policy 0, policy_version 761233 (0.0008) [2023-12-27 10:53:45,385][20381] Updated weights for policy 1, policy_version 760869 (0.0006) [2023-12-27 10:53:45,397][20347] Updated weights for policy 0, policy_version 761243 (0.0008) [2023-12-27 10:53:45,441][20381] Updated weights for policy 1, policy_version 760879 (0.0009) [2023-12-27 10:53:46,077][20347] Updated weights for policy 0, policy_version 761253 (0.0010) [2023-12-27 10:53:46,129][20347] Updated weights for policy 0, policy_version 761263 (0.0010) [2023-12-27 10:53:46,138][20381] Updated weights for policy 1, policy_version 760889 (0.0007) [2023-12-27 10:53:46,184][20347] Updated weights for policy 0, policy_version 761273 (0.0010) [2023-12-27 10:53:46,193][20381] Updated weights for policy 1, policy_version 760899 (0.0005) [2023-12-27 10:53:46,244][20381] Updated weights for policy 1, policy_version 760909 (0.0007) [2023-12-27 10:53:46,295][20381] Updated weights for policy 1, policy_version 760919 (0.0007) [2023-12-27 10:53:46,832][20347] Updated weights for policy 0, policy_version 761283 (0.0010) [2023-12-27 10:53:46,900][20347] Updated weights for policy 0, policy_version 761293 (0.0010) [2023-12-27 10:53:46,968][20347] Updated weights for policy 0, policy_version 761303 (0.0007) [2023-12-27 10:53:46,986][20381] Updated weights for policy 1, policy_version 760929 (0.0006) [2023-12-27 10:53:47,039][20381] Updated weights for policy 1, policy_version 760939 (0.0006) [2023-12-27 10:53:47,088][20381] Updated weights for policy 1, policy_version 760949 (0.0009) [2023-12-27 10:53:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 389750784. Throughput: 0: 10404.4, 1: 9997.0. Samples: 389759688. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:47,363][19170] Avg episode reward: [(0, '804.320'), (1, '912.120')] [2023-12-27 10:53:47,485][20347] Updated weights for policy 0, policy_version 761313 (0.0006) [2023-12-27 10:53:47,540][20347] Updated weights for policy 0, policy_version 761323 (0.0010) [2023-12-27 10:53:47,602][20347] Updated weights for policy 0, policy_version 761333 (0.0010) [2023-12-27 10:53:47,664][20347] Updated weights for policy 0, policy_version 761343 (0.0010) [2023-12-27 10:53:47,716][20381] Updated weights for policy 1, policy_version 760959 (0.0007) [2023-12-27 10:53:47,772][20381] Updated weights for policy 1, policy_version 760969 (0.0007) [2023-12-27 10:53:47,826][20381] Updated weights for policy 1, policy_version 760979 (0.0007) [2023-12-27 10:53:48,305][20347] Updated weights for policy 0, policy_version 761353 (0.0010) [2023-12-27 10:53:48,370][20347] Updated weights for policy 0, policy_version 761363 (0.0010) [2023-12-27 10:53:48,425][20347] Updated weights for policy 0, policy_version 761373 (0.0008) [2023-12-27 10:53:48,544][20381] Updated weights for policy 1, policy_version 760989 (0.0008) [2023-12-27 10:53:48,603][20381] Updated weights for policy 1, policy_version 760999 (0.0008) [2023-12-27 10:53:48,657][20381] Updated weights for policy 1, policy_version 761009 (0.0007) [2023-12-27 10:53:49,115][20347] Updated weights for policy 0, policy_version 761383 (0.0010) [2023-12-27 10:53:49,178][20347] Updated weights for policy 0, policy_version 761393 (0.0010) [2023-12-27 10:53:49,242][20347] Updated weights for policy 0, policy_version 761403 (0.0011) [2023-12-27 10:53:49,295][20381] Updated weights for policy 1, policy_version 761019 (0.0007) [2023-12-27 10:53:49,351][20381] Updated weights for policy 1, policy_version 761029 (0.0008) [2023-12-27 10:53:49,412][20381] Updated weights for policy 1, policy_version 761039 (0.0008) [2023-12-27 10:53:49,984][20347] Updated weights for policy 0, policy_version 761413 (0.0008) [2023-12-27 10:53:50,038][20347] Updated weights for policy 0, policy_version 761423 (0.0007) [2023-12-27 10:53:50,103][20347] Updated weights for policy 0, policy_version 761433 (0.0007) [2023-12-27 10:53:50,151][20381] Updated weights for policy 1, policy_version 761049 (0.0008) [2023-12-27 10:53:50,216][20381] Updated weights for policy 1, policy_version 761059 (0.0007) [2023-12-27 10:53:50,279][20381] Updated weights for policy 1, policy_version 761069 (0.0007) [2023-12-27 10:53:50,343][20381] Updated weights for policy 1, policy_version 761079 (0.0007) [2023-12-27 10:53:50,834][20347] Updated weights for policy 0, policy_version 761443 (0.0007) [2023-12-27 10:53:50,884][20347] Updated weights for policy 0, policy_version 761453 (0.0006) [2023-12-27 10:53:50,945][20347] Updated weights for policy 0, policy_version 761463 (0.0008) [2023-12-27 10:53:51,045][20381] Updated weights for policy 1, policy_version 761089 (0.0007) [2023-12-27 10:53:51,128][20381] Updated weights for policy 1, policy_version 761099 (0.0008) [2023-12-27 10:53:51,186][20381] Updated weights for policy 1, policy_version 761109 (0.0008) [2023-12-27 10:53:51,633][20347] Updated weights for policy 0, policy_version 761473 (0.0008) [2023-12-27 10:53:51,691][20347] Updated weights for policy 0, policy_version 761483 (0.0008) [2023-12-27 10:53:51,754][20347] Updated weights for policy 0, policy_version 761493 (0.0008) [2023-12-27 10:53:51,812][20347] Updated weights for policy 0, policy_version 761503 (0.0008) [2023-12-27 10:53:51,850][20381] Updated weights for policy 1, policy_version 761119 (0.0008) [2023-12-27 10:53:51,909][20381] Updated weights for policy 1, policy_version 761129 (0.0007) [2023-12-27 10:53:51,961][20381] Updated weights for policy 1, policy_version 761139 (0.0005) [2023-12-27 10:53:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 389849088. Throughput: 0: 10367.5, 1: 10013.7. Samples: 389821396. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:52,363][19170] Avg episode reward: [(0, '765.260'), (1, '951.170')] [2023-12-27 10:53:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000761144_194879488.pth... [2023-12-27 10:53:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000761504_194969600.pth... [2023-12-27 10:53:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000759960_194576384.pth [2023-12-27 10:53:52,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000760288_194658304.pth [2023-12-27 10:53:52,583][20347] Updated weights for policy 0, policy_version 761513 (0.0008) [2023-12-27 10:53:52,614][20381] Updated weights for policy 1, policy_version 761149 (0.0009) [2023-12-27 10:53:52,639][20347] Updated weights for policy 0, policy_version 761523 (0.0007) [2023-12-27 10:53:52,675][20381] Updated weights for policy 1, policy_version 761159 (0.0009) [2023-12-27 10:53:52,695][20347] Updated weights for policy 0, policy_version 761533 (0.0007) [2023-12-27 10:53:52,731][20381] Updated weights for policy 1, policy_version 761169 (0.0010) [2023-12-27 10:53:53,290][20347] Updated weights for policy 0, policy_version 761543 (0.0005) [2023-12-27 10:53:53,316][20381] Updated weights for policy 1, policy_version 761179 (0.0008) [2023-12-27 10:53:53,344][20347] Updated weights for policy 0, policy_version 761553 (0.0005) [2023-12-27 10:53:53,366][20381] Updated weights for policy 1, policy_version 761189 (0.0006) [2023-12-27 10:53:53,398][20347] Updated weights for policy 0, policy_version 761563 (0.0006) [2023-12-27 10:53:53,424][20381] Updated weights for policy 1, policy_version 761199 (0.0006) [2023-12-27 10:53:53,955][20347] Updated weights for policy 0, policy_version 761573 (0.0006) [2023-12-27 10:53:54,008][20381] Updated weights for policy 1, policy_version 761209 (0.0006) [2023-12-27 10:53:54,016][20347] Updated weights for policy 0, policy_version 761583 (0.0008) [2023-12-27 10:53:54,071][20347] Updated weights for policy 0, policy_version 761593 (0.0007) [2023-12-27 10:53:54,076][20381] Updated weights for policy 1, policy_version 761219 (0.0005) [2023-12-27 10:53:54,134][20381] Updated weights for policy 1, policy_version 761229 (0.0006) [2023-12-27 10:53:54,191][20381] Updated weights for policy 1, policy_version 761239 (0.0008) [2023-12-27 10:53:54,763][20347] Updated weights for policy 0, policy_version 761603 (0.0007) [2023-12-27 10:53:54,821][20347] Updated weights for policy 0, policy_version 761613 (0.0006) [2023-12-27 10:53:54,857][20381] Updated weights for policy 1, policy_version 761249 (0.0008) [2023-12-27 10:53:54,878][20347] Updated weights for policy 0, policy_version 761623 (0.0005) [2023-12-27 10:53:54,905][20381] Updated weights for policy 1, policy_version 761259 (0.0007) [2023-12-27 10:53:54,955][20381] Updated weights for policy 1, policy_version 761269 (0.0007) [2023-12-27 10:53:55,451][20347] Updated weights for policy 0, policy_version 761633 (0.0006) [2023-12-27 10:53:55,504][20347] Updated weights for policy 0, policy_version 761643 (0.0005) [2023-12-27 10:53:55,553][20347] Updated weights for policy 0, policy_version 761653 (0.0007) [2023-12-27 10:53:55,604][20347] Updated weights for policy 0, policy_version 761663 (0.0007) [2023-12-27 10:53:55,720][20381] Updated weights for policy 1, policy_version 761279 (0.0008) [2023-12-27 10:53:55,777][20381] Updated weights for policy 1, policy_version 761289 (0.0008) [2023-12-27 10:53:55,834][20381] Updated weights for policy 1, policy_version 761299 (0.0008) [2023-12-27 10:53:56,192][20347] Updated weights for policy 0, policy_version 761673 (0.0006) [2023-12-27 10:53:56,245][20291] KL-divergence is very high: 128.6091 [2023-12-27 10:53:56,250][20347] Updated weights for policy 0, policy_version 761683 (0.0006) [2023-12-27 10:53:56,286][20291] KL-divergence is very high: 129.7582 [2023-12-27 10:53:56,299][20347] Updated weights for policy 0, policy_version 761693 (0.0005) [2023-12-27 10:53:56,526][20381] Updated weights for policy 1, policy_version 761309 (0.0007) [2023-12-27 10:53:56,583][20381] Updated weights for policy 1, policy_version 761319 (0.0005) [2023-12-27 10:53:56,638][20381] Updated weights for policy 1, policy_version 761329 (0.0005) [2023-12-27 10:53:56,916][20347] Updated weights for policy 0, policy_version 761703 (0.0009) [2023-12-27 10:53:56,972][20347] Updated weights for policy 0, policy_version 761713 (0.0009) [2023-12-27 10:53:57,030][20347] Updated weights for policy 0, policy_version 761723 (0.0006) [2023-12-27 10:53:57,240][20381] Updated weights for policy 1, policy_version 761339 (0.0005) [2023-12-27 10:53:57,292][20381] Updated weights for policy 1, policy_version 761349 (0.0005) [2023-12-27 10:53:57,344][20381] Updated weights for policy 1, policy_version 761359 (0.0006) [2023-12-27 10:53:57,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 389955584. Throughput: 0: 10415.4, 1: 10054.8. Samples: 389946396. Policy #0 lag: (min: 21.0, avg: 29.9, max: 53.0) [2023-12-27 10:53:57,363][19170] Avg episode reward: [(0, '814.150'), (1, '922.020')] [2023-12-27 10:53:57,643][20347] Updated weights for policy 0, policy_version 761733 (0.0006) [2023-12-27 10:53:57,698][20347] Updated weights for policy 0, policy_version 761743 (0.0006) [2023-12-27 10:53:57,762][20347] Updated weights for policy 0, policy_version 761753 (0.0010) [2023-12-27 10:53:57,934][20381] Updated weights for policy 1, policy_version 761369 (0.0005) [2023-12-27 10:53:57,991][20381] Updated weights for policy 1, policy_version 761379 (0.0010) [2023-12-27 10:53:58,054][20381] Updated weights for policy 1, policy_version 761389 (0.0005) [2023-12-27 10:53:58,108][20381] Updated weights for policy 1, policy_version 761399 (0.0005) [2023-12-27 10:53:58,473][20347] Updated weights for policy 0, policy_version 761763 (0.0010) [2023-12-27 10:53:58,532][20347] Updated weights for policy 0, policy_version 761773 (0.0011) [2023-12-27 10:53:58,596][20347] Updated weights for policy 0, policy_version 761783 (0.0010) [2023-12-27 10:53:58,858][20381] Updated weights for policy 1, policy_version 761409 (0.0008) [2023-12-27 10:53:58,925][20381] Updated weights for policy 1, policy_version 761419 (0.0008) [2023-12-27 10:53:58,985][20381] Updated weights for policy 1, policy_version 761429 (0.0007) [2023-12-27 10:53:59,415][20347] Updated weights for policy 0, policy_version 761793 (0.0008) [2023-12-27 10:53:59,479][20347] Updated weights for policy 0, policy_version 761803 (0.0010) [2023-12-27 10:53:59,546][20347] Updated weights for policy 0, policy_version 761813 (0.0010) [2023-12-27 10:53:59,611][20347] Updated weights for policy 0, policy_version 761823 (0.0010) [2023-12-27 10:53:59,706][20381] Updated weights for policy 1, policy_version 761439 (0.0010) [2023-12-27 10:53:59,767][20381] Updated weights for policy 1, policy_version 761449 (0.0008) [2023-12-27 10:53:59,827][20381] Updated weights for policy 1, policy_version 761459 (0.0010) [2023-12-27 10:54:00,284][20347] Updated weights for policy 0, policy_version 761833 (0.0010) [2023-12-27 10:54:00,339][20347] Updated weights for policy 0, policy_version 761843 (0.0008) [2023-12-27 10:54:00,397][20347] Updated weights for policy 0, policy_version 761853 (0.0008) [2023-12-27 10:54:00,505][20381] Updated weights for policy 1, policy_version 761469 (0.0011) [2023-12-27 10:54:00,554][20381] Updated weights for policy 1, policy_version 761479 (0.0009) [2023-12-27 10:54:00,612][20381] Updated weights for policy 1, policy_version 761489 (0.0005) [2023-12-27 10:54:01,127][20347] Updated weights for policy 0, policy_version 761863 (0.0008) [2023-12-27 10:54:01,190][20347] Updated weights for policy 0, policy_version 761873 (0.0008) [2023-12-27 10:54:01,194][20381] Updated weights for policy 1, policy_version 761499 (0.0006) [2023-12-27 10:54:01,245][20347] Updated weights for policy 0, policy_version 761883 (0.0008) [2023-12-27 10:54:01,253][20381] Updated weights for policy 1, policy_version 761509 (0.0008) [2023-12-27 10:54:01,319][20381] Updated weights for policy 1, policy_version 761519 (0.0011) [2023-12-27 10:54:01,909][20347] Updated weights for policy 0, policy_version 761893 (0.0008) [2023-12-27 10:54:01,966][20347] Updated weights for policy 0, policy_version 761903 (0.0007) [2023-12-27 10:54:02,017][20347] Updated weights for policy 0, policy_version 761913 (0.0006) [2023-12-27 10:54:02,026][20381] Updated weights for policy 1, policy_version 761529 (0.0010) [2023-12-27 10:54:02,081][20381] Updated weights for policy 1, policy_version 761539 (0.0007) [2023-12-27 10:54:02,129][20381] Updated weights for policy 1, policy_version 761549 (0.0010) [2023-12-27 10:54:02,188][20381] Updated weights for policy 1, policy_version 761559 (0.0007) [2023-12-27 10:54:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 390062080. Throughput: 0: 10396.5, 1: 10093.0. Samples: 390069796. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:02,363][19170] Avg episode reward: [(0, '912.030'), (1, '931.770')] [2023-12-27 10:54:02,701][20347] Updated weights for policy 0, policy_version 761923 (0.0005) [2023-12-27 10:54:02,760][20347] Updated weights for policy 0, policy_version 761933 (0.0006) [2023-12-27 10:54:02,817][20347] Updated weights for policy 0, policy_version 761943 (0.0006) [2023-12-27 10:54:02,881][20381] Updated weights for policy 1, policy_version 761569 (0.0008) [2023-12-27 10:54:02,933][20381] Updated weights for policy 1, policy_version 761579 (0.0011) [2023-12-27 10:54:02,979][20381] Updated weights for policy 1, policy_version 761589 (0.0010) [2023-12-27 10:54:03,445][20347] Updated weights for policy 0, policy_version 761953 (0.0005) [2023-12-27 10:54:03,493][20347] Updated weights for policy 0, policy_version 761963 (0.0008) [2023-12-27 10:54:03,544][20347] Updated weights for policy 0, policy_version 761973 (0.0007) [2023-12-27 10:54:03,591][20347] Updated weights for policy 0, policy_version 761983 (0.0008) [2023-12-27 10:54:03,649][20381] Updated weights for policy 1, policy_version 761599 (0.0008) [2023-12-27 10:54:03,697][20381] Updated weights for policy 1, policy_version 761609 (0.0005) [2023-12-27 10:54:03,742][20381] Updated weights for policy 1, policy_version 761619 (0.0009) [2023-12-27 10:54:04,310][20347] Updated weights for policy 0, policy_version 761993 (0.0006) [2023-12-27 10:54:04,377][20347] Updated weights for policy 0, policy_version 762003 (0.0007) [2023-12-27 10:54:04,432][20381] Updated weights for policy 1, policy_version 761629 (0.0011) [2023-12-27 10:54:04,437][20347] Updated weights for policy 0, policy_version 762013 (0.0006) [2023-12-27 10:54:04,481][20381] Updated weights for policy 1, policy_version 761639 (0.0010) [2023-12-27 10:54:04,538][20381] Updated weights for policy 1, policy_version 761649 (0.0005) [2023-12-27 10:54:05,085][20347] Updated weights for policy 0, policy_version 762023 (0.0005) [2023-12-27 10:54:05,131][20347] Updated weights for policy 0, policy_version 762033 (0.0005) [2023-12-27 10:54:05,143][20381] Updated weights for policy 1, policy_version 761659 (0.0007) [2023-12-27 10:54:05,190][20347] Updated weights for policy 0, policy_version 762043 (0.0005) [2023-12-27 10:54:05,199][20381] Updated weights for policy 1, policy_version 761669 (0.0010) [2023-12-27 10:54:05,255][20381] Updated weights for policy 1, policy_version 761679 (0.0010) [2023-12-27 10:54:05,724][20347] Updated weights for policy 0, policy_version 762053 (0.0007) [2023-12-27 10:54:05,777][20347] Updated weights for policy 0, policy_version 762063 (0.0009) [2023-12-27 10:54:05,825][20347] Updated weights for policy 0, policy_version 762073 (0.0008) [2023-12-27 10:54:05,844][20381] Updated weights for policy 1, policy_version 761689 (0.0010) [2023-12-27 10:54:05,896][20381] Updated weights for policy 1, policy_version 761699 (0.0007) [2023-12-27 10:54:05,948][20381] Updated weights for policy 1, policy_version 761709 (0.0009) [2023-12-27 10:54:06,001][20381] Updated weights for policy 1, policy_version 761719 (0.0007) [2023-12-27 10:54:06,547][20347] Updated weights for policy 0, policy_version 762083 (0.0007) [2023-12-27 10:54:06,601][20347] Updated weights for policy 0, policy_version 762093 (0.0007) [2023-12-27 10:54:06,660][20347] Updated weights for policy 0, policy_version 762103 (0.0007) [2023-12-27 10:54:06,716][20381] Updated weights for policy 1, policy_version 761729 (0.0007) [2023-12-27 10:54:06,765][20381] Updated weights for policy 1, policy_version 761739 (0.0008) [2023-12-27 10:54:06,809][20381] Updated weights for policy 1, policy_version 761749 (0.0005) [2023-12-27 10:54:07,259][20347] Updated weights for policy 0, policy_version 762113 (0.0009) [2023-12-27 10:54:07,313][20347] Updated weights for policy 0, policy_version 762123 (0.0009) [2023-12-27 10:54:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 390160384. Throughput: 0: 10419.0, 1: 10126.9. Samples: 390132972. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:07,363][19170] Avg episode reward: [(0, '941.310'), (1, '990.310')] [2023-12-27 10:54:07,370][20347] Updated weights for policy 0, policy_version 762133 (0.0008) [2023-12-27 10:54:07,408][20381] Updated weights for policy 1, policy_version 761759 (0.0005) [2023-12-27 10:54:07,429][20347] Updated weights for policy 0, policy_version 762143 (0.0008) [2023-12-27 10:54:07,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000762144_195133440.pth... [2023-12-27 10:54:07,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000760928_194822144.pth [2023-12-27 10:54:07,462][20381] Updated weights for policy 1, policy_version 761769 (0.0005) [2023-12-27 10:54:07,512][20381] Updated weights for policy 1, policy_version 761779 (0.0007) [2023-12-27 10:54:07,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000761784_195043328.pth... [2023-12-27 10:54:07,536][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000760568_194732032.pth [2023-12-27 10:54:08,019][20347] Updated weights for policy 0, policy_version 762153 (0.0007) [2023-12-27 10:54:08,068][20347] Updated weights for policy 0, policy_version 762164 (0.0008) [2023-12-27 10:54:08,116][20347] Updated weights for policy 0, policy_version 762174 (0.0007) [2023-12-27 10:54:08,125][20381] Updated weights for policy 1, policy_version 761789 (0.0008) [2023-12-27 10:54:08,173][20381] Updated weights for policy 1, policy_version 761799 (0.0007) [2023-12-27 10:54:08,224][20381] Updated weights for policy 1, policy_version 761809 (0.0006) [2023-12-27 10:54:08,840][20347] Updated weights for policy 0, policy_version 762184 (0.0007) [2023-12-27 10:54:08,899][20347] Updated weights for policy 0, policy_version 762194 (0.0008) [2023-12-27 10:54:08,917][20381] Updated weights for policy 1, policy_version 761819 (0.0005) [2023-12-27 10:54:08,952][20347] Updated weights for policy 0, policy_version 762204 (0.0006) [2023-12-27 10:54:08,975][20381] Updated weights for policy 1, policy_version 761829 (0.0006) [2023-12-27 10:54:09,033][20381] Updated weights for policy 1, policy_version 761839 (0.0008) [2023-12-27 10:54:09,663][20347] Updated weights for policy 0, policy_version 762214 (0.0008) [2023-12-27 10:54:09,700][20381] Updated weights for policy 1, policy_version 761849 (0.0006) [2023-12-27 10:54:09,718][20347] Updated weights for policy 0, policy_version 762224 (0.0008) [2023-12-27 10:54:09,765][20381] Updated weights for policy 1, policy_version 761859 (0.0006) [2023-12-27 10:54:09,778][20347] Updated weights for policy 0, policy_version 762234 (0.0007) [2023-12-27 10:54:09,832][20381] Updated weights for policy 1, policy_version 761869 (0.0007) [2023-12-27 10:54:09,893][20381] Updated weights for policy 1, policy_version 761879 (0.0007) [2023-12-27 10:54:10,520][20347] Updated weights for policy 0, policy_version 762244 (0.0007) [2023-12-27 10:54:10,567][20381] Updated weights for policy 1, policy_version 761889 (0.0007) [2023-12-27 10:54:10,578][20347] Updated weights for policy 0, policy_version 762254 (0.0006) [2023-12-27 10:54:10,629][20381] Updated weights for policy 1, policy_version 761899 (0.0006) [2023-12-27 10:54:10,636][20347] Updated weights for policy 0, policy_version 762264 (0.0005) [2023-12-27 10:54:10,687][20381] Updated weights for policy 1, policy_version 761909 (0.0008) [2023-12-27 10:54:11,321][20347] Updated weights for policy 0, policy_version 762274 (0.0006) [2023-12-27 10:54:11,379][20347] Updated weights for policy 0, policy_version 762284 (0.0010) [2023-12-27 10:54:11,412][20381] Updated weights for policy 1, policy_version 761919 (0.0007) [2023-12-27 10:54:11,450][20347] Updated weights for policy 0, policy_version 762294 (0.0009) [2023-12-27 10:54:11,470][20381] Updated weights for policy 1, policy_version 761929 (0.0007) [2023-12-27 10:54:11,513][20347] Updated weights for policy 0, policy_version 762304 (0.0008) [2023-12-27 10:54:11,535][20381] Updated weights for policy 1, policy_version 761939 (0.0007) [2023-12-27 10:54:12,199][20381] Updated weights for policy 1, policy_version 761949 (0.0006) [2023-12-27 10:54:12,256][20381] Updated weights for policy 1, policy_version 761959 (0.0006) [2023-12-27 10:54:12,293][20347] Updated weights for policy 0, policy_version 762314 (0.0008) [2023-12-27 10:54:12,315][20381] Updated weights for policy 1, policy_version 761969 (0.0007) [2023-12-27 10:54:12,353][20347] Updated weights for policy 0, policy_version 762324 (0.0007) [2023-12-27 10:54:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 390266880. Throughput: 0: 10411.1, 1: 10209.3. Samples: 390258884. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:12,363][19170] Avg episode reward: [(0, '941.320'), (1, '960.920')] [2023-12-27 10:54:12,413][20347] Updated weights for policy 0, policy_version 762334 (0.0007) [2023-12-27 10:54:13,049][20381] Updated weights for policy 1, policy_version 761979 (0.0007) [2023-12-27 10:54:13,077][20347] Updated weights for policy 0, policy_version 762344 (0.0006) [2023-12-27 10:54:13,095][20381] Updated weights for policy 1, policy_version 761989 (0.0007) [2023-12-27 10:54:13,136][20347] Updated weights for policy 0, policy_version 762354 (0.0005) [2023-12-27 10:54:13,142][20381] Updated weights for policy 1, policy_version 761999 (0.0008) [2023-12-27 10:54:13,194][20347] Updated weights for policy 0, policy_version 762364 (0.0006) [2023-12-27 10:54:13,706][20347] Updated weights for policy 0, policy_version 762374 (0.0008) [2023-12-27 10:54:13,753][20347] Updated weights for policy 0, policy_version 762384 (0.0007) [2023-12-27 10:54:13,798][20347] Updated weights for policy 0, policy_version 762394 (0.0005) [2023-12-27 10:54:13,909][20381] Updated weights for policy 1, policy_version 762009 (0.0008) [2023-12-27 10:54:13,971][20381] Updated weights for policy 1, policy_version 762019 (0.0007) [2023-12-27 10:54:14,016][20381] Updated weights for policy 1, policy_version 762029 (0.0010) [2023-12-27 10:54:14,063][20381] Updated weights for policy 1, policy_version 762039 (0.0008) [2023-12-27 10:54:14,450][20347] Updated weights for policy 0, policy_version 762404 (0.0006) [2023-12-27 10:54:14,510][20347] Updated weights for policy 0, policy_version 762414 (0.0006) [2023-12-27 10:54:14,572][20347] Updated weights for policy 0, policy_version 762424 (0.0006) [2023-12-27 10:54:14,789][20381] Updated weights for policy 1, policy_version 762049 (0.0008) [2023-12-27 10:54:14,850][20381] Updated weights for policy 1, policy_version 762059 (0.0008) [2023-12-27 10:54:14,911][20381] Updated weights for policy 1, policy_version 762069 (0.0009) [2023-12-27 10:54:15,185][20347] Updated weights for policy 0, policy_version 762434 (0.0008) [2023-12-27 10:54:15,246][20347] Updated weights for policy 0, policy_version 762444 (0.0006) [2023-12-27 10:54:15,306][20347] Updated weights for policy 0, policy_version 762454 (0.0007) [2023-12-27 10:54:15,367][20347] Updated weights for policy 0, policy_version 762464 (0.0007) [2023-12-27 10:54:15,643][20381] Updated weights for policy 1, policy_version 762079 (0.0007) [2023-12-27 10:54:15,695][20381] Updated weights for policy 1, policy_version 762089 (0.0008) [2023-12-27 10:54:15,743][20381] Updated weights for policy 1, policy_version 762099 (0.0007) [2023-12-27 10:54:16,034][20347] Updated weights for policy 0, policy_version 762474 (0.0008) [2023-12-27 10:54:16,087][20347] Updated weights for policy 0, policy_version 762484 (0.0007) [2023-12-27 10:54:16,144][20347] Updated weights for policy 0, policy_version 762494 (0.0005) [2023-12-27 10:54:16,445][20381] Updated weights for policy 1, policy_version 762109 (0.0008) [2023-12-27 10:54:16,505][20381] Updated weights for policy 1, policy_version 762119 (0.0008) [2023-12-27 10:54:16,557][20381] Updated weights for policy 1, policy_version 762129 (0.0008) [2023-12-27 10:54:16,809][20347] Updated weights for policy 0, policy_version 762504 (0.0009) [2023-12-27 10:54:16,867][20347] Updated weights for policy 0, policy_version 762514 (0.0007) [2023-12-27 10:54:16,913][20347] Updated weights for policy 0, policy_version 762524 (0.0010) [2023-12-27 10:54:17,273][20381] Updated weights for policy 1, policy_version 762139 (0.0008) [2023-12-27 10:54:17,325][20381] Updated weights for policy 1, policy_version 762149 (0.0008) [2023-12-27 10:54:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 390365184. Throughput: 0: 10422.6, 1: 10168.4. Samples: 390380256. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:17,363][19170] Avg episode reward: [(0, '921.740'), (1, '941.430')] [2023-12-27 10:54:17,369][20381] Updated weights for policy 1, policy_version 762159 (0.0008) [2023-12-27 10:54:17,549][20347] Updated weights for policy 0, policy_version 762534 (0.0007) [2023-12-27 10:54:17,614][20347] Updated weights for policy 0, policy_version 762544 (0.0005) [2023-12-27 10:54:17,673][20347] Updated weights for policy 0, policy_version 762554 (0.0006) [2023-12-27 10:54:18,104][20381] Updated weights for policy 1, policy_version 762169 (0.0008) [2023-12-27 10:54:18,150][20381] Updated weights for policy 1, policy_version 762179 (0.0008) [2023-12-27 10:54:18,195][20381] Updated weights for policy 1, policy_version 762189 (0.0007) [2023-12-27 10:54:18,240][20381] Updated weights for policy 1, policy_version 762199 (0.0008) [2023-12-27 10:54:18,268][20347] Updated weights for policy 0, policy_version 762564 (0.0006) [2023-12-27 10:54:18,324][20347] Updated weights for policy 0, policy_version 762574 (0.0008) [2023-12-27 10:54:18,389][20347] Updated weights for policy 0, policy_version 762584 (0.0009) [2023-12-27 10:54:18,980][20381] Updated weights for policy 1, policy_version 762209 (0.0010) [2023-12-27 10:54:19,037][20381] Updated weights for policy 1, policy_version 762219 (0.0010) [2023-12-27 10:54:19,095][20347] Updated weights for policy 0, policy_version 762594 (0.0007) [2023-12-27 10:54:19,103][20381] Updated weights for policy 1, policy_version 762229 (0.0009) [2023-12-27 10:54:19,146][20347] Updated weights for policy 0, policy_version 762604 (0.0007) [2023-12-27 10:54:19,195][20347] Updated weights for policy 0, policy_version 762614 (0.0007) [2023-12-27 10:54:19,255][20347] Updated weights for policy 0, policy_version 762624 (0.0007) [2023-12-27 10:54:19,774][20381] Updated weights for policy 1, policy_version 762239 (0.0008) [2023-12-27 10:54:19,836][20381] Updated weights for policy 1, policy_version 762249 (0.0008) [2023-12-27 10:54:19,888][20381] Updated weights for policy 1, policy_version 762259 (0.0008) [2023-12-27 10:54:20,004][20347] Updated weights for policy 0, policy_version 762634 (0.0008) [2023-12-27 10:54:20,062][20347] Updated weights for policy 0, policy_version 762644 (0.0008) [2023-12-27 10:54:20,119][20347] Updated weights for policy 0, policy_version 762654 (0.0008) [2023-12-27 10:54:20,621][20381] Updated weights for policy 1, policy_version 762269 (0.0008) [2023-12-27 10:54:20,671][20381] Updated weights for policy 1, policy_version 762279 (0.0008) [2023-12-27 10:54:20,721][20381] Updated weights for policy 1, policy_version 762289 (0.0008) [2023-12-27 10:54:20,822][20347] Updated weights for policy 0, policy_version 762664 (0.0007) [2023-12-27 10:54:20,878][20347] Updated weights for policy 0, policy_version 762674 (0.0008) [2023-12-27 10:54:20,936][20347] Updated weights for policy 0, policy_version 762684 (0.0008) [2023-12-27 10:54:21,487][20381] Updated weights for policy 1, policy_version 762299 (0.0008) [2023-12-27 10:54:21,546][20381] Updated weights for policy 1, policy_version 762309 (0.0008) [2023-12-27 10:54:21,606][20381] Updated weights for policy 1, policy_version 762319 (0.0007) [2023-12-27 10:54:21,709][20347] Updated weights for policy 0, policy_version 762694 (0.0010) [2023-12-27 10:54:21,780][20347] Updated weights for policy 0, policy_version 762704 (0.0008) [2023-12-27 10:54:21,842][20347] Updated weights for policy 0, policy_version 762714 (0.0008) [2023-12-27 10:54:22,341][20381] Updated weights for policy 1, policy_version 762329 (0.0008) [2023-12-27 10:54:22,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 390463488. Throughput: 0: 10409.5, 1: 10155.8. Samples: 390440416. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:22,363][19170] Avg episode reward: [(0, '921.800'), (1, '941.570')] [2023-12-27 10:54:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000762720_195280896.pth... [2023-12-27 10:54:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000761504_194969600.pth [2023-12-27 10:54:22,410][20381] Updated weights for policy 1, policy_version 762339 (0.0008) [2023-12-27 10:54:22,463][20381] Updated weights for policy 1, policy_version 762349 (0.0008) [2023-12-27 10:54:22,516][20347] Updated weights for policy 0, policy_version 762724 (0.0007) [2023-12-27 10:54:22,518][20381] Updated weights for policy 1, policy_version 762359 (0.0007) [2023-12-27 10:54:22,520][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000762360_195190784.pth... [2023-12-27 10:54:22,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000761144_194879488.pth [2023-12-27 10:54:22,572][20347] Updated weights for policy 0, policy_version 762734 (0.0007) [2023-12-27 10:54:22,629][20347] Updated weights for policy 0, policy_version 762744 (0.0008) [2023-12-27 10:54:23,238][20381] Updated weights for policy 1, policy_version 762369 (0.0008) [2023-12-27 10:54:23,302][20347] Updated weights for policy 0, policy_version 762754 (0.0008) [2023-12-27 10:54:23,307][20381] Updated weights for policy 1, policy_version 762379 (0.0007) [2023-12-27 10:54:23,350][20347] Updated weights for policy 0, policy_version 762764 (0.0007) [2023-12-27 10:54:23,367][20381] Updated weights for policy 1, policy_version 762389 (0.0006) [2023-12-27 10:54:23,397][20347] Updated weights for policy 0, policy_version 762774 (0.0006) [2023-12-27 10:54:23,445][20347] Updated weights for policy 0, policy_version 762784 (0.0006) [2023-12-27 10:54:24,039][20381] Updated weights for policy 1, policy_version 762399 (0.0008) [2023-12-27 10:54:24,102][20381] Updated weights for policy 1, policy_version 762409 (0.0008) [2023-12-27 10:54:24,126][20347] Updated weights for policy 0, policy_version 762794 (0.0008) [2023-12-27 10:54:24,161][20381] Updated weights for policy 1, policy_version 762419 (0.0005) [2023-12-27 10:54:24,177][20347] Updated weights for policy 0, policy_version 762804 (0.0008) [2023-12-27 10:54:24,228][20347] Updated weights for policy 0, policy_version 762814 (0.0008) [2023-12-27 10:54:24,752][20381] Updated weights for policy 1, policy_version 762429 (0.0006) [2023-12-27 10:54:24,800][20381] Updated weights for policy 1, policy_version 762439 (0.0007) [2023-12-27 10:54:24,843][20381] Updated weights for policy 1, policy_version 762449 (0.0007) [2023-12-27 10:54:24,912][20347] Updated weights for policy 0, policy_version 762824 (0.0010) [2023-12-27 10:54:24,957][20347] Updated weights for policy 0, policy_version 762834 (0.0010) [2023-12-27 10:54:25,011][20347] Updated weights for policy 0, policy_version 762844 (0.0010) [2023-12-27 10:54:25,576][20381] Updated weights for policy 1, policy_version 762459 (0.0007) [2023-12-27 10:54:25,634][20347] Updated weights for policy 0, policy_version 762854 (0.0009) [2023-12-27 10:54:25,641][20381] Updated weights for policy 1, policy_version 762469 (0.0008) [2023-12-27 10:54:25,692][20292] KL-divergence is very high: 120.4443 [2023-12-27 10:54:25,694][20347] Updated weights for policy 0, policy_version 762864 (0.0011) [2023-12-27 10:54:25,696][20292] KL-divergence is very high: 106.7177 [2023-12-27 10:54:25,697][20381] Updated weights for policy 1, policy_version 762479 (0.0006) [2023-12-27 10:54:25,743][20347] Updated weights for policy 0, policy_version 762874 (0.0010) [2023-12-27 10:54:26,330][20381] Updated weights for policy 1, policy_version 762489 (0.0006) [2023-12-27 10:54:26,344][20292] KL-divergence is very high: 166.3745 [2023-12-27 10:54:26,346][20347] Updated weights for policy 0, policy_version 762884 (0.0008) [2023-12-27 10:54:26,382][20381] Updated weights for policy 1, policy_version 762499 (0.0008) [2023-12-27 10:54:26,388][20292] KL-divergence is very high: 104.8683 [2023-12-27 10:54:26,405][20347] Updated weights for policy 0, policy_version 762894 (0.0005) [2023-12-27 10:54:26,438][20381] Updated weights for policy 1, policy_version 762509 (0.0008) [2023-12-27 10:54:26,452][20347] Updated weights for policy 0, policy_version 762904 (0.0006) [2023-12-27 10:54:26,494][20381] Updated weights for policy 1, policy_version 762519 (0.0006) [2023-12-27 10:54:27,004][20347] Updated weights for policy 0, policy_version 762914 (0.0005) [2023-12-27 10:54:27,056][20347] Updated weights for policy 0, policy_version 762924 (0.0005) [2023-12-27 10:54:27,110][20347] Updated weights for policy 0, policy_version 762934 (0.0006) [2023-12-27 10:54:27,159][20347] Updated weights for policy 0, policy_version 762944 (0.0010) [2023-12-27 10:54:27,218][20381] Updated weights for policy 1, policy_version 762529 (0.0008) [2023-12-27 10:54:27,273][20381] Updated weights for policy 1, policy_version 762539 (0.0008) [2023-12-27 10:54:27,326][20381] Updated weights for policy 1, policy_version 762549 (0.0008) [2023-12-27 10:54:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 390578176. Throughput: 0: 10371.7, 1: 10134.3. Samples: 390561376. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:27,363][19170] Avg episode reward: [(0, '921.990'), (1, '931.970')] [2023-12-27 10:54:27,763][20347] Updated weights for policy 0, policy_version 762954 (0.0008) [2023-12-27 10:54:27,817][20347] Updated weights for policy 0, policy_version 762964 (0.0008) [2023-12-27 10:54:27,872][20347] Updated weights for policy 0, policy_version 762974 (0.0008) [2023-12-27 10:54:27,970][20381] Updated weights for policy 1, policy_version 762559 (0.0006) [2023-12-27 10:54:28,026][20381] Updated weights for policy 1, policy_version 762569 (0.0007) [2023-12-27 10:54:28,083][20381] Updated weights for policy 1, policy_version 762579 (0.0008) [2023-12-27 10:54:28,562][20347] Updated weights for policy 0, policy_version 762984 (0.0010) [2023-12-27 10:54:28,623][20347] Updated weights for policy 0, policy_version 762994 (0.0010) [2023-12-27 10:54:28,682][20347] Updated weights for policy 0, policy_version 763004 (0.0008) [2023-12-27 10:54:28,736][20381] Updated weights for policy 1, policy_version 762589 (0.0008) [2023-12-27 10:54:28,794][20381] Updated weights for policy 1, policy_version 762599 (0.0007) [2023-12-27 10:54:28,845][20381] Updated weights for policy 1, policy_version 762609 (0.0007) [2023-12-27 10:54:29,368][20347] Updated weights for policy 0, policy_version 763014 (0.0007) [2023-12-27 10:54:29,417][20347] Updated weights for policy 0, policy_version 763024 (0.0008) [2023-12-27 10:54:29,468][20347] Updated weights for policy 0, policy_version 763034 (0.0007) [2023-12-27 10:54:29,528][20381] Updated weights for policy 1, policy_version 762619 (0.0008) [2023-12-27 10:54:29,587][20381] Updated weights for policy 1, policy_version 762629 (0.0005) [2023-12-27 10:54:29,655][20381] Updated weights for policy 1, policy_version 762639 (0.0006) [2023-12-27 10:54:30,169][20347] Updated weights for policy 0, policy_version 763044 (0.0006) [2023-12-27 10:54:30,225][20347] Updated weights for policy 0, policy_version 763054 (0.0008) [2023-12-27 10:54:30,275][20347] Updated weights for policy 0, policy_version 763064 (0.0007) [2023-12-27 10:54:30,302][20381] Updated weights for policy 1, policy_version 762649 (0.0011) [2023-12-27 10:54:30,361][20381] Updated weights for policy 1, policy_version 762659 (0.0011) [2023-12-27 10:54:30,423][20381] Updated weights for policy 1, policy_version 762669 (0.0011) [2023-12-27 10:54:30,481][20381] Updated weights for policy 1, policy_version 762679 (0.0011) [2023-12-27 10:54:30,957][20347] Updated weights for policy 0, policy_version 763074 (0.0006) [2023-12-27 10:54:31,009][20347] Updated weights for policy 0, policy_version 763084 (0.0005) [2023-12-27 10:54:31,070][20347] Updated weights for policy 0, policy_version 763094 (0.0007) [2023-12-27 10:54:31,128][20347] Updated weights for policy 0, policy_version 763104 (0.0008) [2023-12-27 10:54:31,143][20381] Updated weights for policy 1, policy_version 762689 (0.0008) [2023-12-27 10:54:31,201][20381] Updated weights for policy 1, policy_version 762699 (0.0009) [2023-12-27 10:54:31,259][20381] Updated weights for policy 1, policy_version 762709 (0.0007) [2023-12-27 10:54:31,827][20347] Updated weights for policy 0, policy_version 763114 (0.0006) [2023-12-27 10:54:31,883][20347] Updated weights for policy 0, policy_version 763124 (0.0007) [2023-12-27 10:54:31,933][20347] Updated weights for policy 0, policy_version 763134 (0.0005) [2023-12-27 10:54:31,934][20381] Updated weights for policy 1, policy_version 762719 (0.0007) [2023-12-27 10:54:31,987][20381] Updated weights for policy 1, policy_version 762729 (0.0008) [2023-12-27 10:54:32,036][20381] Updated weights for policy 1, policy_version 762739 (0.0008) [2023-12-27 10:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 390676480. Throughput: 0: 10411.9, 1: 10198.3. Samples: 390687148. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:32,363][19170] Avg episode reward: [(0, '912.230'), (1, '932.030')] [2023-12-27 10:54:32,585][20347] Updated weights for policy 0, policy_version 763144 (0.0007) [2023-12-27 10:54:32,648][20347] Updated weights for policy 0, policy_version 763154 (0.0008) [2023-12-27 10:54:32,706][20347] Updated weights for policy 0, policy_version 763164 (0.0010) [2023-12-27 10:54:32,726][20381] Updated weights for policy 1, policy_version 762749 (0.0007) [2023-12-27 10:54:32,784][20381] Updated weights for policy 1, policy_version 762759 (0.0007) [2023-12-27 10:54:32,839][20381] Updated weights for policy 1, policy_version 762769 (0.0007) [2023-12-27 10:54:33,324][20347] Updated weights for policy 0, policy_version 763174 (0.0007) [2023-12-27 10:54:33,369][20347] Updated weights for policy 0, policy_version 763184 (0.0005) [2023-12-27 10:54:33,415][20347] Updated weights for policy 0, policy_version 763194 (0.0005) [2023-12-27 10:54:33,418][20381] Updated weights for policy 1, policy_version 762779 (0.0007) [2023-12-27 10:54:33,482][20381] Updated weights for policy 1, policy_version 762789 (0.0008) [2023-12-27 10:54:33,543][20381] Updated weights for policy 1, policy_version 762799 (0.0007) [2023-12-27 10:54:33,942][20347] Updated weights for policy 0, policy_version 763204 (0.0007) [2023-12-27 10:54:33,993][20347] Updated weights for policy 0, policy_version 763214 (0.0010) [2023-12-27 10:54:34,050][20347] Updated weights for policy 0, policy_version 763224 (0.0010) [2023-12-27 10:54:34,163][20381] Updated weights for policy 1, policy_version 762809 (0.0005) [2023-12-27 10:54:34,228][20381] Updated weights for policy 1, policy_version 762819 (0.0007) [2023-12-27 10:54:34,288][20381] Updated weights for policy 1, policy_version 762829 (0.0007) [2023-12-27 10:54:34,347][20381] Updated weights for policy 1, policy_version 762839 (0.0008) [2023-12-27 10:54:34,731][20347] Updated weights for policy 0, policy_version 763234 (0.0010) [2023-12-27 10:54:34,789][20347] Updated weights for policy 0, policy_version 763244 (0.0010) [2023-12-27 10:54:34,843][20347] Updated weights for policy 0, policy_version 763254 (0.0010) [2023-12-27 10:54:34,894][20347] Updated weights for policy 0, policy_version 763264 (0.0010) [2023-12-27 10:54:34,970][20381] Updated weights for policy 1, policy_version 762849 (0.0005) [2023-12-27 10:54:35,035][20381] Updated weights for policy 1, policy_version 762859 (0.0006) [2023-12-27 10:54:35,091][20381] Updated weights for policy 1, policy_version 762869 (0.0007) [2023-12-27 10:54:35,534][20347] Updated weights for policy 0, policy_version 763274 (0.0010) [2023-12-27 10:54:35,589][20347] Updated weights for policy 0, policy_version 763284 (0.0009) [2023-12-27 10:54:35,653][20347] Updated weights for policy 0, policy_version 763294 (0.0008) [2023-12-27 10:54:35,699][20381] Updated weights for policy 1, policy_version 762879 (0.0005) [2023-12-27 10:54:35,757][20381] Updated weights for policy 1, policy_version 762889 (0.0005) [2023-12-27 10:54:35,808][20381] Updated weights for policy 1, policy_version 762899 (0.0007) [2023-12-27 10:54:36,325][20347] Updated weights for policy 0, policy_version 763304 (0.0010) [2023-12-27 10:54:36,389][20347] Updated weights for policy 0, policy_version 763314 (0.0007) [2023-12-27 10:54:36,425][20381] Updated weights for policy 1, policy_version 762909 (0.0007) [2023-12-27 10:54:36,451][20347] Updated weights for policy 0, policy_version 763324 (0.0010) [2023-12-27 10:54:36,482][20381] Updated weights for policy 1, policy_version 762919 (0.0006) [2023-12-27 10:54:36,540][20381] Updated weights for policy 1, policy_version 762929 (0.0007) [2023-12-27 10:54:37,061][20347] Updated weights for policy 0, policy_version 763334 (0.0010) [2023-12-27 10:54:37,117][20347] Updated weights for policy 0, policy_version 763344 (0.0010) [2023-12-27 10:54:37,179][20347] Updated weights for policy 0, policy_version 763354 (0.0008) [2023-12-27 10:54:37,195][20381] Updated weights for policy 1, policy_version 762939 (0.0007) [2023-12-27 10:54:37,224][20292] KL-divergence is very high: 120.7064 [2023-12-27 10:54:37,253][20381] Updated weights for policy 1, policy_version 762949 (0.0011) [2023-12-27 10:54:37,312][20381] Updated weights for policy 1, policy_version 762959 (0.0010) [2023-12-27 10:54:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 390791168. Throughput: 0: 10456.0, 1: 10253.3. Samples: 390753312. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:37,363][19170] Avg episode reward: [(0, '931.550'), (1, '951.300')] [2023-12-27 10:54:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000762968_195346432.pth... [2023-12-27 10:54:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000763360_195444736.pth... [2023-12-27 10:54:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000761784_195043328.pth [2023-12-27 10:54:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000762144_195133440.pth [2023-12-27 10:54:37,767][20347] Updated weights for policy 0, policy_version 763364 (0.0007) [2023-12-27 10:54:37,835][20347] Updated weights for policy 0, policy_version 763374 (0.0010) [2023-12-27 10:54:37,884][20347] Updated weights for policy 0, policy_version 763384 (0.0007) [2023-12-27 10:54:37,956][20381] Updated weights for policy 1, policy_version 762969 (0.0009) [2023-12-27 10:54:38,011][20381] Updated weights for policy 1, policy_version 762979 (0.0010) [2023-12-27 10:54:38,067][20381] Updated weights for policy 1, policy_version 762989 (0.0010) [2023-12-27 10:54:38,126][20381] Updated weights for policy 1, policy_version 762999 (0.0010) [2023-12-27 10:54:38,607][20347] Updated weights for policy 0, policy_version 763394 (0.0008) [2023-12-27 10:54:38,658][20347] Updated weights for policy 0, policy_version 763404 (0.0008) [2023-12-27 10:54:38,710][20347] Updated weights for policy 0, policy_version 763414 (0.0007) [2023-12-27 10:54:38,728][20381] Updated weights for policy 1, policy_version 763009 (0.0008) [2023-12-27 10:54:38,758][20347] Updated weights for policy 0, policy_version 763424 (0.0006) [2023-12-27 10:54:38,790][20381] Updated weights for policy 1, policy_version 763019 (0.0008) [2023-12-27 10:54:38,848][20381] Updated weights for policy 1, policy_version 763029 (0.0008) [2023-12-27 10:54:39,452][20347] Updated weights for policy 0, policy_version 763434 (0.0008) [2023-12-27 10:54:39,512][20347] Updated weights for policy 0, policy_version 763444 (0.0008) [2023-12-27 10:54:39,574][20347] Updated weights for policy 0, policy_version 763454 (0.0007) [2023-12-27 10:54:39,637][20381] Updated weights for policy 1, policy_version 763039 (0.0008) [2023-12-27 10:54:39,705][20381] Updated weights for policy 1, policy_version 763049 (0.0008) [2023-12-27 10:54:39,765][20381] Updated weights for policy 1, policy_version 763059 (0.0009) [2023-12-27 10:54:40,256][20347] Updated weights for policy 0, policy_version 763464 (0.0007) [2023-12-27 10:54:40,315][20347] Updated weights for policy 0, policy_version 763474 (0.0008) [2023-12-27 10:54:40,378][20347] Updated weights for policy 0, policy_version 763484 (0.0007) [2023-12-27 10:54:40,415][20381] Updated weights for policy 1, policy_version 763069 (0.0008) [2023-12-27 10:54:40,464][20381] Updated weights for policy 1, policy_version 763079 (0.0007) [2023-12-27 10:54:40,517][20381] Updated weights for policy 1, policy_version 763089 (0.0008) [2023-12-27 10:54:41,049][20347] Updated weights for policy 0, policy_version 763494 (0.0008) [2023-12-27 10:54:41,114][20347] Updated weights for policy 0, policy_version 763504 (0.0009) [2023-12-27 10:54:41,186][20347] Updated weights for policy 0, policy_version 763514 (0.0008) [2023-12-27 10:54:41,247][20381] Updated weights for policy 1, policy_version 763099 (0.0006) [2023-12-27 10:54:41,308][20381] Updated weights for policy 1, policy_version 763109 (0.0008) [2023-12-27 10:54:41,383][20381] Updated weights for policy 1, policy_version 763119 (0.0007) [2023-12-27 10:54:41,911][20347] Updated weights for policy 0, policy_version 763524 (0.0008) [2023-12-27 10:54:41,972][20347] Updated weights for policy 0, policy_version 763534 (0.0006) [2023-12-27 10:54:42,042][20347] Updated weights for policy 0, policy_version 763544 (0.0007) [2023-12-27 10:54:42,091][20381] Updated weights for policy 1, policy_version 763129 (0.0006) [2023-12-27 10:54:42,157][20381] Updated weights for policy 1, policy_version 763139 (0.0008) [2023-12-27 10:54:42,217][20381] Updated weights for policy 1, policy_version 763149 (0.0007) [2023-12-27 10:54:42,278][20381] Updated weights for policy 1, policy_version 763159 (0.0009) [2023-12-27 10:54:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 390889472. Throughput: 0: 10431.5, 1: 10274.3. Samples: 390878152. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:42,363][19170] Avg episode reward: [(0, '912.070'), (1, '941.500')] [2023-12-27 10:54:42,711][20347] Updated weights for policy 0, policy_version 763554 (0.0008) [2023-12-27 10:54:42,771][20347] Updated weights for policy 0, policy_version 763564 (0.0008) [2023-12-27 10:54:42,831][20347] Updated weights for policy 0, policy_version 763574 (0.0008) [2023-12-27 10:54:42,896][20347] Updated weights for policy 0, policy_version 763584 (0.0008) [2023-12-27 10:54:43,024][20381] Updated weights for policy 1, policy_version 763169 (0.0007) [2023-12-27 10:54:43,086][20381] Updated weights for policy 1, policy_version 763179 (0.0008) [2023-12-27 10:54:43,142][20381] Updated weights for policy 1, policy_version 763189 (0.0008) [2023-12-27 10:54:43,535][20347] Updated weights for policy 0, policy_version 763594 (0.0008) [2023-12-27 10:54:43,588][20347] Updated weights for policy 0, policy_version 763604 (0.0007) [2023-12-27 10:54:43,653][20347] Updated weights for policy 0, policy_version 763614 (0.0008) [2023-12-27 10:54:43,810][20381] Updated weights for policy 1, policy_version 763199 (0.0008) [2023-12-27 10:54:43,862][20381] Updated weights for policy 1, policy_version 763209 (0.0008) [2023-12-27 10:54:43,912][20381] Updated weights for policy 1, policy_version 763219 (0.0007) [2023-12-27 10:54:44,354][20347] Updated weights for policy 0, policy_version 763624 (0.0007) [2023-12-27 10:54:44,375][20291] KL-divergence is very high: 200.3630 [2023-12-27 10:54:44,381][20291] KL-divergence is very high: 195.3886 [2023-12-27 10:54:44,388][20291] KL-divergence is very high: 134.3485 [2023-12-27 10:54:44,409][20347] Updated weights for policy 0, policy_version 763634 (0.0007) [2023-12-27 10:54:44,467][20347] Updated weights for policy 0, policy_version 763644 (0.0008) [2023-12-27 10:54:44,599][20381] Updated weights for policy 1, policy_version 763229 (0.0007) [2023-12-27 10:54:44,655][20381] Updated weights for policy 1, policy_version 763239 (0.0005) [2023-12-27 10:54:44,712][20381] Updated weights for policy 1, policy_version 763249 (0.0005) [2023-12-27 10:54:45,197][20291] KL-divergence is very high: 195.7948 [2023-12-27 10:54:45,214][20347] Updated weights for policy 0, policy_version 763654 (0.0008) [2023-12-27 10:54:45,237][20291] KL-divergence is very high: 434.8042 [2023-12-27 10:54:45,260][20347] Updated weights for policy 0, policy_version 763664 (0.0007) [2023-12-27 10:54:45,320][20347] Updated weights for policy 0, policy_version 763674 (0.0007) [2023-12-27 10:54:45,345][20291] KL-divergence is very high: 128.1481 [2023-12-27 10:54:45,394][20381] Updated weights for policy 1, policy_version 763259 (0.0006) [2023-12-27 10:54:45,444][20381] Updated weights for policy 1, policy_version 763269 (0.0008) [2023-12-27 10:54:45,493][20381] Updated weights for policy 1, policy_version 763279 (0.0008) [2023-12-27 10:54:45,912][20347] Updated weights for policy 0, policy_version 763684 (0.0006) [2023-12-27 10:54:45,965][20347] Updated weights for policy 0, policy_version 763694 (0.0005) [2023-12-27 10:54:46,024][20347] Updated weights for policy 0, policy_version 763704 (0.0007) [2023-12-27 10:54:46,221][20381] Updated weights for policy 1, policy_version 763289 (0.0008) [2023-12-27 10:54:46,273][20381] Updated weights for policy 1, policy_version 763299 (0.0011) [2023-12-27 10:54:46,322][20381] Updated weights for policy 1, policy_version 763309 (0.0011) [2023-12-27 10:54:46,377][20381] Updated weights for policy 1, policy_version 763319 (0.0010) [2023-12-27 10:54:46,617][20347] Updated weights for policy 0, policy_version 763714 (0.0007) [2023-12-27 10:54:46,681][20347] Updated weights for policy 0, policy_version 763724 (0.0008) [2023-12-27 10:54:46,747][20347] Updated weights for policy 0, policy_version 763734 (0.0007) [2023-12-27 10:54:46,800][20347] Updated weights for policy 0, policy_version 763744 (0.0006) [2023-12-27 10:54:47,070][20381] Updated weights for policy 1, policy_version 763329 (0.0011) [2023-12-27 10:54:47,121][20381] Updated weights for policy 1, policy_version 763339 (0.0010) [2023-12-27 10:54:47,169][20381] Updated weights for policy 1, policy_version 763349 (0.0010) [2023-12-27 10:54:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 390987776. Throughput: 0: 10439.6, 1: 10197.4. Samples: 390998460. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:47,363][19170] Avg episode reward: [(0, '863.240'), (1, '931.750')] [2023-12-27 10:54:47,466][20347] Updated weights for policy 0, policy_version 763754 (0.0007) [2023-12-27 10:54:47,523][20347] Updated weights for policy 0, policy_version 763764 (0.0007) [2023-12-27 10:54:47,574][20347] Updated weights for policy 0, policy_version 763774 (0.0007) [2023-12-27 10:54:47,852][20381] Updated weights for policy 1, policy_version 763359 (0.0008) [2023-12-27 10:54:47,904][20381] Updated weights for policy 1, policy_version 763369 (0.0008) [2023-12-27 10:54:47,957][20381] Updated weights for policy 1, policy_version 763379 (0.0008) [2023-12-27 10:54:48,261][20347] Updated weights for policy 0, policy_version 763784 (0.0008) [2023-12-27 10:54:48,325][20347] Updated weights for policy 0, policy_version 763794 (0.0008) [2023-12-27 10:54:48,392][20347] Updated weights for policy 0, policy_version 763804 (0.0007) [2023-12-27 10:54:48,643][20381] Updated weights for policy 1, policy_version 763389 (0.0007) [2023-12-27 10:54:48,698][20381] Updated weights for policy 1, policy_version 763399 (0.0007) [2023-12-27 10:54:48,756][20381] Updated weights for policy 1, policy_version 763409 (0.0008) [2023-12-27 10:54:49,104][20347] Updated weights for policy 0, policy_version 763814 (0.0007) [2023-12-27 10:54:49,159][20347] Updated weights for policy 0, policy_version 763824 (0.0007) [2023-12-27 10:54:49,220][20347] Updated weights for policy 0, policy_version 763834 (0.0006) [2023-12-27 10:54:49,474][20381] Updated weights for policy 1, policy_version 763419 (0.0008) [2023-12-27 10:54:49,531][20381] Updated weights for policy 1, policy_version 763429 (0.0005) [2023-12-27 10:54:49,580][20381] Updated weights for policy 1, policy_version 763439 (0.0005) [2023-12-27 10:54:49,907][20347] Updated weights for policy 0, policy_version 763844 (0.0007) [2023-12-27 10:54:49,969][20347] Updated weights for policy 0, policy_version 763854 (0.0009) [2023-12-27 10:54:50,031][20347] Updated weights for policy 0, policy_version 763864 (0.0008) [2023-12-27 10:54:50,198][20381] Updated weights for policy 1, policy_version 763449 (0.0005) [2023-12-27 10:54:50,248][20381] Updated weights for policy 1, policy_version 763459 (0.0007) [2023-12-27 10:54:50,303][20381] Updated weights for policy 1, policy_version 763469 (0.0008) [2023-12-27 10:54:50,363][20381] Updated weights for policy 1, policy_version 763479 (0.0008) [2023-12-27 10:54:50,753][20347] Updated weights for policy 0, policy_version 763874 (0.0008) [2023-12-27 10:54:50,813][20347] Updated weights for policy 0, policy_version 763884 (0.0007) [2023-12-27 10:54:50,868][20347] Updated weights for policy 0, policy_version 763894 (0.0008) [2023-12-27 10:54:50,912][20347] Updated weights for policy 0, policy_version 763904 (0.0007) [2023-12-27 10:54:51,059][20381] Updated weights for policy 1, policy_version 763489 (0.0010) [2023-12-27 10:54:51,115][20381] Updated weights for policy 1, policy_version 763499 (0.0011) [2023-12-27 10:54:51,176][20381] Updated weights for policy 1, policy_version 763509 (0.0011) [2023-12-27 10:54:51,646][20347] Updated weights for policy 0, policy_version 763914 (0.0008) [2023-12-27 10:54:51,706][20347] Updated weights for policy 0, policy_version 763924 (0.0007) [2023-12-27 10:54:51,764][20347] Updated weights for policy 0, policy_version 763934 (0.0009) [2023-12-27 10:54:51,885][20381] Updated weights for policy 1, policy_version 763519 (0.0010) [2023-12-27 10:54:51,943][20381] Updated weights for policy 1, policy_version 763529 (0.0010) [2023-12-27 10:54:52,006][20381] Updated weights for policy 1, policy_version 763539 (0.0009) [2023-12-27 10:54:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 391086080. Throughput: 0: 10392.3, 1: 10191.4. Samples: 391059236. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:52,363][19170] Avg episode reward: [(0, '911.950'), (1, '951.220')] [2023-12-27 10:54:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000763544_195493888.pth... [2023-12-27 10:54:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000763936_195592192.pth... [2023-12-27 10:54:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000762360_195190784.pth [2023-12-27 10:54:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000762720_195280896.pth [2023-12-27 10:54:52,480][20347] Updated weights for policy 0, policy_version 763944 (0.0007) [2023-12-27 10:54:52,529][20347] Updated weights for policy 0, policy_version 763954 (0.0008) [2023-12-27 10:54:52,583][20347] Updated weights for policy 0, policy_version 763964 (0.0008) [2023-12-27 10:54:52,678][20381] Updated weights for policy 1, policy_version 763549 (0.0010) [2023-12-27 10:54:52,738][20381] Updated weights for policy 1, policy_version 763559 (0.0011) [2023-12-27 10:54:52,794][20381] Updated weights for policy 1, policy_version 763569 (0.0011) [2023-12-27 10:54:53,327][20347] Updated weights for policy 0, policy_version 763974 (0.0007) [2023-12-27 10:54:53,375][20347] Updated weights for policy 0, policy_version 763984 (0.0007) [2023-12-27 10:54:53,420][20347] Updated weights for policy 0, policy_version 763994 (0.0007) [2023-12-27 10:54:53,453][20381] Updated weights for policy 1, policy_version 763579 (0.0011) [2023-12-27 10:54:53,501][20381] Updated weights for policy 1, policy_version 763589 (0.0010) [2023-12-27 10:54:53,563][20381] Updated weights for policy 1, policy_version 763599 (0.0011) [2023-12-27 10:54:54,105][20347] Updated weights for policy 0, policy_version 764004 (0.0007) [2023-12-27 10:54:54,156][20347] Updated weights for policy 0, policy_version 764014 (0.0008) [2023-12-27 10:54:54,202][20347] Updated weights for policy 0, policy_version 764024 (0.0006) [2023-12-27 10:54:54,250][20381] Updated weights for policy 1, policy_version 763609 (0.0011) [2023-12-27 10:54:54,301][20381] Updated weights for policy 1, policy_version 763619 (0.0010) [2023-12-27 10:54:54,354][20381] Updated weights for policy 1, policy_version 763629 (0.0011) [2023-12-27 10:54:54,409][20381] Updated weights for policy 1, policy_version 763639 (0.0011) [2023-12-27 10:54:54,898][20347] Updated weights for policy 0, policy_version 764034 (0.0006) [2023-12-27 10:54:54,959][20347] Updated weights for policy 0, policy_version 764044 (0.0006) [2023-12-27 10:54:55,013][20347] Updated weights for policy 0, policy_version 764054 (0.0007) [2023-12-27 10:54:55,061][20347] Updated weights for policy 0, policy_version 764064 (0.0007) [2023-12-27 10:54:55,110][20381] Updated weights for policy 1, policy_version 763649 (0.0011) [2023-12-27 10:54:55,173][20381] Updated weights for policy 1, policy_version 763659 (0.0011) [2023-12-27 10:54:55,222][20381] Updated weights for policy 1, policy_version 763669 (0.0010) [2023-12-27 10:54:55,748][20347] Updated weights for policy 0, policy_version 764074 (0.0008) [2023-12-27 10:54:55,809][20347] Updated weights for policy 0, policy_version 764084 (0.0009) [2023-12-27 10:54:55,867][20347] Updated weights for policy 0, policy_version 764094 (0.0007) [2023-12-27 10:54:55,869][20381] Updated weights for policy 1, policy_version 763679 (0.0007) [2023-12-27 10:54:55,921][20381] Updated weights for policy 1, policy_version 763689 (0.0005) [2023-12-27 10:54:55,974][20381] Updated weights for policy 1, policy_version 763699 (0.0005) [2023-12-27 10:54:56,500][20381] Updated weights for policy 1, policy_version 763709 (0.0005) [2023-12-27 10:54:56,557][20381] Updated weights for policy 1, policy_version 763719 (0.0006) [2023-12-27 10:54:56,612][20381] Updated weights for policy 1, policy_version 763729 (0.0005) [2023-12-27 10:54:56,626][20347] Updated weights for policy 0, policy_version 764104 (0.0007) [2023-12-27 10:54:56,685][20347] Updated weights for policy 0, policy_version 764114 (0.0008) [2023-12-27 10:54:56,743][20347] Updated weights for policy 0, policy_version 764124 (0.0007) [2023-12-27 10:54:57,146][20381] Updated weights for policy 1, policy_version 763739 (0.0006) [2023-12-27 10:54:57,197][20381] Updated weights for policy 1, policy_version 763749 (0.0008) [2023-12-27 10:54:57,252][20381] Updated weights for policy 1, policy_version 763759 (0.0008) [2023-12-27 10:54:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 391192576. Throughput: 0: 10311.3, 1: 10149.4. Samples: 391179616. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:54:57,363][19170] Avg episode reward: [(0, '951.110'), (1, '980.470')] [2023-12-27 10:54:57,466][20347] Updated weights for policy 0, policy_version 764134 (0.0008) [2023-12-27 10:54:57,518][20347] Updated weights for policy 0, policy_version 764144 (0.0009) [2023-12-27 10:54:57,565][20347] Updated weights for policy 0, policy_version 764154 (0.0007) [2023-12-27 10:54:57,815][20381] Updated weights for policy 1, policy_version 763769 (0.0007) [2023-12-27 10:54:57,867][20381] Updated weights for policy 1, policy_version 763779 (0.0005) [2023-12-27 10:54:57,929][20381] Updated weights for policy 1, policy_version 763789 (0.0005) [2023-12-27 10:54:57,994][20381] Updated weights for policy 1, policy_version 763799 (0.0006) [2023-12-27 10:54:58,307][20347] Updated weights for policy 0, policy_version 764164 (0.0007) [2023-12-27 10:54:58,373][20347] Updated weights for policy 0, policy_version 764174 (0.0009) [2023-12-27 10:54:58,431][20347] Updated weights for policy 0, policy_version 764184 (0.0007) [2023-12-27 10:54:58,690][20381] Updated weights for policy 1, policy_version 763809 (0.0009) [2023-12-27 10:54:58,753][20381] Updated weights for policy 1, policy_version 763819 (0.0009) [2023-12-27 10:54:58,823][20381] Updated weights for policy 1, policy_version 763829 (0.0007) [2023-12-27 10:54:59,210][20347] Updated weights for policy 0, policy_version 764194 (0.0008) [2023-12-27 10:54:59,272][20347] Updated weights for policy 0, policy_version 764204 (0.0007) [2023-12-27 10:54:59,335][20347] Updated weights for policy 0, policy_version 764214 (0.0007) [2023-12-27 10:54:59,396][20347] Updated weights for policy 0, policy_version 764224 (0.0007) [2023-12-27 10:54:59,548][20381] Updated weights for policy 1, policy_version 763839 (0.0006) [2023-12-27 10:54:59,601][20381] Updated weights for policy 1, policy_version 763849 (0.0005) [2023-12-27 10:54:59,657][20381] Updated weights for policy 1, policy_version 763859 (0.0005) [2023-12-27 10:55:00,121][20347] Updated weights for policy 0, policy_version 764234 (0.0007) [2023-12-27 10:55:00,177][20347] Updated weights for policy 0, policy_version 764244 (0.0008) [2023-12-27 10:55:00,232][20381] Updated weights for policy 1, policy_version 763869 (0.0006) [2023-12-27 10:55:00,237][20347] Updated weights for policy 0, policy_version 764254 (0.0006) [2023-12-27 10:55:00,291][20381] Updated weights for policy 1, policy_version 763879 (0.0007) [2023-12-27 10:55:00,349][20381] Updated weights for policy 1, policy_version 763889 (0.0008) [2023-12-27 10:55:00,898][20347] Updated weights for policy 0, policy_version 764264 (0.0007) [2023-12-27 10:55:00,947][20347] Updated weights for policy 0, policy_version 764274 (0.0007) [2023-12-27 10:55:00,992][20347] Updated weights for policy 0, policy_version 764284 (0.0007) [2023-12-27 10:55:01,012][20381] Updated weights for policy 1, policy_version 763899 (0.0007) [2023-12-27 10:55:01,073][20381] Updated weights for policy 1, policy_version 763909 (0.0008) [2023-12-27 10:55:01,135][20381] Updated weights for policy 1, policy_version 763919 (0.0008) [2023-12-27 10:55:01,789][20347] Updated weights for policy 0, policy_version 764294 (0.0009) [2023-12-27 10:55:01,847][20347] Updated weights for policy 0, policy_version 764304 (0.0010) [2023-12-27 10:55:01,872][20381] Updated weights for policy 1, policy_version 763929 (0.0006) [2023-12-27 10:55:01,904][20347] Updated weights for policy 0, policy_version 764314 (0.0011) [2023-12-27 10:55:01,921][20381] Updated weights for policy 1, policy_version 763939 (0.0005) [2023-12-27 10:55:01,973][20381] Updated weights for policy 1, policy_version 763949 (0.0007) [2023-12-27 10:55:02,020][20381] Updated weights for policy 1, policy_version 763959 (0.0007) [2023-12-27 10:55:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 391290880. Throughput: 0: 10169.1, 1: 10268.2. Samples: 391299932. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:02,363][19170] Avg episode reward: [(0, '931.810'), (1, '980.490')] [2023-12-27 10:55:02,620][20347] Updated weights for policy 0, policy_version 764324 (0.0010) [2023-12-27 10:55:02,670][20347] Updated weights for policy 0, policy_version 764334 (0.0010) [2023-12-27 10:55:02,722][20347] Updated weights for policy 0, policy_version 764344 (0.0010) [2023-12-27 10:55:02,761][20381] Updated weights for policy 1, policy_version 763969 (0.0008) [2023-12-27 10:55:02,818][20381] Updated weights for policy 1, policy_version 763979 (0.0007) [2023-12-27 10:55:02,875][20381] Updated weights for policy 1, policy_version 763989 (0.0007) [2023-12-27 10:55:03,308][20347] Updated weights for policy 0, policy_version 764354 (0.0010) [2023-12-27 10:55:03,358][20347] Updated weights for policy 0, policy_version 764364 (0.0008) [2023-12-27 10:55:03,406][20347] Updated weights for policy 0, policy_version 764374 (0.0010) [2023-12-27 10:55:03,459][20347] Updated weights for policy 0, policy_version 764384 (0.0007) [2023-12-27 10:55:03,553][20381] Updated weights for policy 1, policy_version 763999 (0.0007) [2023-12-27 10:55:03,606][20381] Updated weights for policy 1, policy_version 764009 (0.0005) [2023-12-27 10:55:03,653][20381] Updated weights for policy 1, policy_version 764019 (0.0008) [2023-12-27 10:55:04,033][20347] Updated weights for policy 0, policy_version 764394 (0.0007) [2023-12-27 10:55:04,094][20347] Updated weights for policy 0, policy_version 764404 (0.0010) [2023-12-27 10:55:04,156][20347] Updated weights for policy 0, policy_version 764414 (0.0010) [2023-12-27 10:55:04,301][20381] Updated weights for policy 1, policy_version 764029 (0.0011) [2023-12-27 10:55:04,355][20381] Updated weights for policy 1, policy_version 764039 (0.0011) [2023-12-27 10:55:04,405][20381] Updated weights for policy 1, policy_version 764049 (0.0010) [2023-12-27 10:55:04,815][20347] Updated weights for policy 0, policy_version 764424 (0.0010) [2023-12-27 10:55:04,866][20347] Updated weights for policy 0, policy_version 764434 (0.0009) [2023-12-27 10:55:04,915][20347] Updated weights for policy 0, policy_version 764444 (0.0010) [2023-12-27 10:55:05,028][20381] Updated weights for policy 1, policy_version 764059 (0.0009) [2023-12-27 10:55:05,080][20381] Updated weights for policy 1, policy_version 764069 (0.0005) [2023-12-27 10:55:05,124][20381] Updated weights for policy 1, policy_version 764079 (0.0006) [2023-12-27 10:55:05,662][20347] Updated weights for policy 0, policy_version 764454 (0.0008) [2023-12-27 10:55:05,711][20347] Updated weights for policy 0, policy_version 764464 (0.0010) [2023-12-27 10:55:05,764][20347] Updated weights for policy 0, policy_version 764474 (0.0010) [2023-12-27 10:55:05,832][20381] Updated weights for policy 1, policy_version 764089 (0.0006) [2023-12-27 10:55:05,881][20381] Updated weights for policy 1, policy_version 764099 (0.0008) [2023-12-27 10:55:05,931][20381] Updated weights for policy 1, policy_version 764109 (0.0008) [2023-12-27 10:55:05,978][20381] Updated weights for policy 1, policy_version 764119 (0.0007) [2023-12-27 10:55:06,436][20347] Updated weights for policy 0, policy_version 764484 (0.0008) [2023-12-27 10:55:06,496][20347] Updated weights for policy 0, policy_version 764494 (0.0007) [2023-12-27 10:55:06,554][20347] Updated weights for policy 0, policy_version 764504 (0.0007) [2023-12-27 10:55:06,764][20381] Updated weights for policy 1, policy_version 764129 (0.0007) [2023-12-27 10:55:06,829][20381] Updated weights for policy 1, policy_version 764139 (0.0007) [2023-12-27 10:55:06,897][20381] Updated weights for policy 1, policy_version 764149 (0.0007) [2023-12-27 10:55:07,176][20347] Updated weights for policy 0, policy_version 764514 (0.0006) [2023-12-27 10:55:07,243][20347] Updated weights for policy 0, policy_version 764524 (0.0008) [2023-12-27 10:55:07,303][20347] Updated weights for policy 0, policy_version 764534 (0.0008) [2023-12-27 10:55:07,352][20347] Updated weights for policy 0, policy_version 764544 (0.0007) [2023-12-27 10:55:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 391397376. Throughput: 0: 10213.2, 1: 10334.8. Samples: 391365076. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:07,363][19170] Avg episode reward: [(0, '834.630'), (1, '980.490')] [2023-12-27 10:55:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000764152_195649536.pth... [2023-12-27 10:55:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000764544_195747840.pth... [2023-12-27 10:55:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000763360_195444736.pth [2023-12-27 10:55:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000762968_195346432.pth [2023-12-27 10:55:07,624][20381] Updated weights for policy 1, policy_version 764159 (0.0009) [2023-12-27 10:55:07,679][20381] Updated weights for policy 1, policy_version 764169 (0.0010) [2023-12-27 10:55:07,734][20381] Updated weights for policy 1, policy_version 764179 (0.0010) [2023-12-27 10:55:08,006][20347] Updated weights for policy 0, policy_version 764554 (0.0005) [2023-12-27 10:55:08,062][20347] Updated weights for policy 0, policy_version 764564 (0.0005) [2023-12-27 10:55:08,086][20291] KL-divergence is very high: 130.2247 [2023-12-27 10:55:08,102][20291] KL-divergence is very high: 183.9762 [2023-12-27 10:55:08,108][20291] KL-divergence is very high: 164.8399 [2023-12-27 10:55:08,119][20347] Updated weights for policy 0, policy_version 764574 (0.0006) [2023-12-27 10:55:08,440][20381] Updated weights for policy 1, policy_version 764189 (0.0007) [2023-12-27 10:55:08,497][20381] Updated weights for policy 1, policy_version 764199 (0.0008) [2023-12-27 10:55:08,555][20381] Updated weights for policy 1, policy_version 764209 (0.0007) [2023-12-27 10:55:08,780][20347] Updated weights for policy 0, policy_version 764584 (0.0010) [2023-12-27 10:55:08,836][20347] Updated weights for policy 0, policy_version 764594 (0.0010) [2023-12-27 10:55:08,889][20347] Updated weights for policy 0, policy_version 764604 (0.0011) [2023-12-27 10:55:09,270][20381] Updated weights for policy 1, policy_version 764219 (0.0007) [2023-12-27 10:55:09,327][20381] Updated weights for policy 1, policy_version 764229 (0.0007) [2023-12-27 10:55:09,402][20381] Updated weights for policy 1, policy_version 764239 (0.0009) [2023-12-27 10:55:09,630][20347] Updated weights for policy 0, policy_version 764614 (0.0009) [2023-12-27 10:55:09,686][20347] Updated weights for policy 0, policy_version 764624 (0.0007) [2023-12-27 10:55:09,744][20347] Updated weights for policy 0, policy_version 764634 (0.0008) [2023-12-27 10:55:10,148][20381] Updated weights for policy 1, policy_version 764249 (0.0008) [2023-12-27 10:55:10,209][20381] Updated weights for policy 1, policy_version 764259 (0.0011) [2023-12-27 10:55:10,264][20381] Updated weights for policy 1, policy_version 764269 (0.0011) [2023-12-27 10:55:10,331][20381] Updated weights for policy 1, policy_version 764279 (0.0011) [2023-12-27 10:55:10,447][20347] Updated weights for policy 0, policy_version 764644 (0.0008) [2023-12-27 10:55:10,502][20347] Updated weights for policy 0, policy_version 764654 (0.0008) [2023-12-27 10:55:10,557][20347] Updated weights for policy 0, policy_version 764664 (0.0007) [2023-12-27 10:55:10,937][20381] Updated weights for policy 1, policy_version 764289 (0.0006) [2023-12-27 10:55:10,993][20381] Updated weights for policy 1, policy_version 764299 (0.0006) [2023-12-27 10:55:11,058][20381] Updated weights for policy 1, policy_version 764309 (0.0008) [2023-12-27 10:55:11,294][20347] Updated weights for policy 0, policy_version 764674 (0.0008) [2023-12-27 10:55:11,361][20347] Updated weights for policy 0, policy_version 764684 (0.0007) [2023-12-27 10:55:11,427][20347] Updated weights for policy 0, policy_version 764694 (0.0008) [2023-12-27 10:55:11,491][20347] Updated weights for policy 0, policy_version 764704 (0.0007) [2023-12-27 10:55:11,828][20381] Updated weights for policy 1, policy_version 764319 (0.0008) [2023-12-27 10:55:11,890][20381] Updated weights for policy 1, policy_version 764329 (0.0009) [2023-12-27 10:55:11,939][20381] Updated weights for policy 1, policy_version 764339 (0.0007) [2023-12-27 10:55:12,211][20347] Updated weights for policy 0, policy_version 764714 (0.0008) [2023-12-27 10:55:12,268][20347] Updated weights for policy 0, policy_version 764724 (0.0008) [2023-12-27 10:55:12,329][20347] Updated weights for policy 0, policy_version 764734 (0.0007) [2023-12-27 10:55:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 391495680. Throughput: 0: 10206.0, 1: 10294.2. Samples: 391483884. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:12,363][19170] Avg episode reward: [(0, '740.980'), (1, '1000.000')] [2023-12-27 10:55:12,645][20381] Updated weights for policy 1, policy_version 764349 (0.0006) [2023-12-27 10:55:12,708][20381] Updated weights for policy 1, policy_version 764359 (0.0007) [2023-12-27 10:55:12,773][20381] Updated weights for policy 1, policy_version 764369 (0.0007) [2023-12-27 10:55:13,042][20347] Updated weights for policy 0, policy_version 764744 (0.0010) [2023-12-27 10:55:13,102][20347] Updated weights for policy 0, policy_version 764754 (0.0011) [2023-12-27 10:55:13,166][20347] Updated weights for policy 0, policy_version 764764 (0.0009) [2023-12-27 10:55:13,398][20381] Updated weights for policy 1, policy_version 764379 (0.0007) [2023-12-27 10:55:13,455][20381] Updated weights for policy 1, policy_version 764389 (0.0006) [2023-12-27 10:55:13,515][20381] Updated weights for policy 1, policy_version 764399 (0.0007) [2023-12-27 10:55:13,922][20347] Updated weights for policy 0, policy_version 764774 (0.0010) [2023-12-27 10:55:13,985][20347] Updated weights for policy 0, policy_version 764784 (0.0007) [2023-12-27 10:55:14,050][20347] Updated weights for policy 0, policy_version 764794 (0.0006) [2023-12-27 10:55:14,206][20381] Updated weights for policy 1, policy_version 764409 (0.0007) [2023-12-27 10:55:14,268][20381] Updated weights for policy 1, policy_version 764419 (0.0008) [2023-12-27 10:55:14,337][20381] Updated weights for policy 1, policy_version 764429 (0.0008) [2023-12-27 10:55:14,400][20381] Updated weights for policy 1, policy_version 764439 (0.0007) [2023-12-27 10:55:14,703][20347] Updated weights for policy 0, policy_version 764804 (0.0007) [2023-12-27 10:55:14,758][20347] Updated weights for policy 0, policy_version 764814 (0.0008) [2023-12-27 10:55:14,819][20347] Updated weights for policy 0, policy_version 764824 (0.0009) [2023-12-27 10:55:15,010][20381] Updated weights for policy 1, policy_version 764449 (0.0008) [2023-12-27 10:55:15,072][20381] Updated weights for policy 1, policy_version 764459 (0.0008) [2023-12-27 10:55:15,135][20381] Updated weights for policy 1, policy_version 764469 (0.0008) [2023-12-27 10:55:15,498][20347] Updated weights for policy 0, policy_version 764834 (0.0010) [2023-12-27 10:55:15,562][20347] Updated weights for policy 0, policy_version 764844 (0.0011) [2023-12-27 10:55:15,619][20347] Updated weights for policy 0, policy_version 764854 (0.0010) [2023-12-27 10:55:15,677][20347] Updated weights for policy 0, policy_version 764864 (0.0010) [2023-12-27 10:55:15,757][20381] Updated weights for policy 1, policy_version 764479 (0.0008) [2023-12-27 10:55:15,810][20381] Updated weights for policy 1, policy_version 764489 (0.0008) [2023-12-27 10:55:15,858][20381] Updated weights for policy 1, policy_version 764499 (0.0008) [2023-12-27 10:55:16,314][20347] Updated weights for policy 0, policy_version 764874 (0.0007) [2023-12-27 10:55:16,372][20347] Updated weights for policy 0, policy_version 764884 (0.0008) [2023-12-27 10:55:16,427][20347] Updated weights for policy 0, policy_version 764894 (0.0007) [2023-12-27 10:55:16,579][20381] Updated weights for policy 1, policy_version 764509 (0.0009) [2023-12-27 10:55:16,637][20381] Updated weights for policy 1, policy_version 764519 (0.0008) [2023-12-27 10:55:16,688][20381] Updated weights for policy 1, policy_version 764529 (0.0008) [2023-12-27 10:55:17,096][20347] Updated weights for policy 0, policy_version 764904 (0.0008) [2023-12-27 10:55:17,158][20347] Updated weights for policy 0, policy_version 764914 (0.0008) [2023-12-27 10:55:17,216][20347] Updated weights for policy 0, policy_version 764924 (0.0007) [2023-12-27 10:55:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 391593984. Throughput: 0: 10098.2, 1: 10282.8. Samples: 391604296. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:17,363][19170] Avg episode reward: [(0, '552.100'), (1, '1000.000')] [2023-12-27 10:55:17,369][20381] Updated weights for policy 1, policy_version 764539 (0.0007) [2023-12-27 10:55:17,415][20381] Updated weights for policy 1, policy_version 764549 (0.0008) [2023-12-27 10:55:17,475][20381] Updated weights for policy 1, policy_version 764559 (0.0007) [2023-12-27 10:55:17,863][20347] Updated weights for policy 0, policy_version 764934 (0.0008) [2023-12-27 10:55:17,915][20347] Updated weights for policy 0, policy_version 764944 (0.0007) [2023-12-27 10:55:17,963][20347] Updated weights for policy 0, policy_version 764954 (0.0010) [2023-12-27 10:55:18,179][20381] Updated weights for policy 1, policy_version 764569 (0.0008) [2023-12-27 10:55:18,227][20381] Updated weights for policy 1, policy_version 764579 (0.0006) [2023-12-27 10:55:18,273][20381] Updated weights for policy 1, policy_version 764589 (0.0010) [2023-12-27 10:55:18,329][20381] Updated weights for policy 1, policy_version 764599 (0.0011) [2023-12-27 10:55:18,674][20347] Updated weights for policy 0, policy_version 764965 (0.0010) [2023-12-27 10:55:18,730][20347] Updated weights for policy 0, policy_version 764975 (0.0010) [2023-12-27 10:55:18,781][20347] Updated weights for policy 0, policy_version 764985 (0.0007) [2023-12-27 10:55:19,052][20381] Updated weights for policy 1, policy_version 764609 (0.0007) [2023-12-27 10:55:19,105][20381] Updated weights for policy 1, policy_version 764619 (0.0008) [2023-12-27 10:55:19,157][20381] Updated weights for policy 1, policy_version 764629 (0.0008) [2023-12-27 10:55:19,479][20347] Updated weights for policy 0, policy_version 764995 (0.0007) [2023-12-27 10:55:19,544][20347] Updated weights for policy 0, policy_version 765005 (0.0009) [2023-12-27 10:55:19,608][20347] Updated weights for policy 0, policy_version 765015 (0.0008) [2023-12-27 10:55:19,884][20381] Updated weights for policy 1, policy_version 764639 (0.0008) [2023-12-27 10:55:19,953][20381] Updated weights for policy 1, policy_version 764649 (0.0008) [2023-12-27 10:55:20,018][20381] Updated weights for policy 1, policy_version 764659 (0.0008) [2023-12-27 10:55:20,379][20347] Updated weights for policy 0, policy_version 765025 (0.0008) [2023-12-27 10:55:20,433][20347] Updated weights for policy 0, policy_version 765035 (0.0008) [2023-12-27 10:55:20,486][20347] Updated weights for policy 0, policy_version 765045 (0.0007) [2023-12-27 10:55:20,548][20347] Updated weights for policy 0, policy_version 765055 (0.0008) [2023-12-27 10:55:20,741][20381] Updated weights for policy 1, policy_version 764669 (0.0007) [2023-12-27 10:55:20,801][20381] Updated weights for policy 1, policy_version 764679 (0.0005) [2023-12-27 10:55:20,858][20381] Updated weights for policy 1, policy_version 764689 (0.0005) [2023-12-27 10:55:21,302][20347] Updated weights for policy 0, policy_version 765065 (0.0008) [2023-12-27 10:55:21,369][20347] Updated weights for policy 0, policy_version 765075 (0.0008) [2023-12-27 10:55:21,437][20347] Updated weights for policy 0, policy_version 765085 (0.0007) [2023-12-27 10:55:21,543][20381] Updated weights for policy 1, policy_version 764699 (0.0006) [2023-12-27 10:55:21,600][20381] Updated weights for policy 1, policy_version 764709 (0.0008) [2023-12-27 10:55:21,661][20381] Updated weights for policy 1, policy_version 764719 (0.0008) [2023-12-27 10:55:22,098][20347] Updated weights for policy 0, policy_version 765095 (0.0007) [2023-12-27 10:55:22,163][20347] Updated weights for policy 0, policy_version 765105 (0.0007) [2023-12-27 10:55:22,223][20347] Updated weights for policy 0, policy_version 765115 (0.0007) [2023-12-27 10:55:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 391692288. Throughput: 0: 10031.3, 1: 10192.1. Samples: 391663364. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:22,363][19170] Avg episode reward: [(0, '487.380'), (1, '980.490')] [2023-12-27 10:55:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000764728_195796992.pth... [2023-12-27 10:55:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000765120_195895296.pth... [2023-12-27 10:55:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000763544_195493888.pth [2023-12-27 10:55:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000763936_195592192.pth [2023-12-27 10:55:22,402][20381] Updated weights for policy 1, policy_version 764729 (0.0007) [2023-12-27 10:55:22,465][20381] Updated weights for policy 1, policy_version 764739 (0.0006) [2023-12-27 10:55:22,527][20381] Updated weights for policy 1, policy_version 764749 (0.0006) [2023-12-27 10:55:22,588][20381] Updated weights for policy 1, policy_version 764759 (0.0008) [2023-12-27 10:55:22,927][20347] Updated weights for policy 0, policy_version 765125 (0.0008) [2023-12-27 10:55:22,977][20347] Updated weights for policy 0, policy_version 765135 (0.0007) [2023-12-27 10:55:23,026][20347] Updated weights for policy 0, policy_version 765145 (0.0008) [2023-12-27 10:55:23,188][20292] KL-divergence is very high: 141.1058 [2023-12-27 10:55:23,229][20381] Updated weights for policy 1, policy_version 764769 (0.0005) [2023-12-27 10:55:23,286][20381] Updated weights for policy 1, policy_version 764779 (0.0005) [2023-12-27 10:55:23,341][20381] Updated weights for policy 1, policy_version 764789 (0.0005) [2023-12-27 10:55:23,763][20347] Updated weights for policy 0, policy_version 765155 (0.0007) [2023-12-27 10:55:23,824][20347] Updated weights for policy 0, policy_version 765165 (0.0005) [2023-12-27 10:55:23,880][20347] Updated weights for policy 0, policy_version 765175 (0.0006) [2023-12-27 10:55:23,955][20381] Updated weights for policy 1, policy_version 764799 (0.0007) [2023-12-27 10:55:24,004][20381] Updated weights for policy 1, policy_version 764809 (0.0008) [2023-12-27 10:55:24,059][20381] Updated weights for policy 1, policy_version 764819 (0.0008) [2023-12-27 10:55:24,515][20347] Updated weights for policy 0, policy_version 765185 (0.0008) [2023-12-27 10:55:24,576][20347] Updated weights for policy 0, policy_version 765195 (0.0010) [2023-12-27 10:55:24,630][20347] Updated weights for policy 0, policy_version 765205 (0.0010) [2023-12-27 10:55:24,686][20347] Updated weights for policy 0, policy_version 765215 (0.0010) [2023-12-27 10:55:24,718][20381] Updated weights for policy 1, policy_version 764829 (0.0009) [2023-12-27 10:55:24,773][20381] Updated weights for policy 1, policy_version 764839 (0.0010) [2023-12-27 10:55:24,819][20381] Updated weights for policy 1, policy_version 764849 (0.0010) [2023-12-27 10:55:25,367][20347] Updated weights for policy 0, policy_version 765225 (0.0010) [2023-12-27 10:55:25,421][20347] Updated weights for policy 0, policy_version 765235 (0.0010) [2023-12-27 10:55:25,472][20347] Updated weights for policy 0, policy_version 765245 (0.0010) [2023-12-27 10:55:25,520][20381] Updated weights for policy 1, policy_version 764859 (0.0010) [2023-12-27 10:55:25,582][20381] Updated weights for policy 1, policy_version 764869 (0.0011) [2023-12-27 10:55:25,639][20381] Updated weights for policy 1, policy_version 764879 (0.0010) [2023-12-27 10:55:26,148][20347] Updated weights for policy 0, policy_version 765255 (0.0010) [2023-12-27 10:55:26,197][20347] Updated weights for policy 0, policy_version 765265 (0.0010) [2023-12-27 10:55:26,248][20347] Updated weights for policy 0, policy_version 765275 (0.0010) [2023-12-27 10:55:26,319][20381] Updated weights for policy 1, policy_version 764889 (0.0010) [2023-12-27 10:55:26,371][20381] Updated weights for policy 1, policy_version 764899 (0.0010) [2023-12-27 10:55:26,425][20381] Updated weights for policy 1, policy_version 764909 (0.0010) [2023-12-27 10:55:26,484][20381] Updated weights for policy 1, policy_version 764919 (0.0011) [2023-12-27 10:55:26,936][20347] Updated weights for policy 0, policy_version 765285 (0.0009) [2023-12-27 10:55:27,001][20347] Updated weights for policy 0, policy_version 765295 (0.0006) [2023-12-27 10:55:27,054][20347] Updated weights for policy 0, policy_version 765305 (0.0008) [2023-12-27 10:55:27,177][20381] Updated weights for policy 1, policy_version 764929 (0.0007) [2023-12-27 10:55:27,229][20381] Updated weights for policy 1, policy_version 764939 (0.0006) [2023-12-27 10:55:27,281][20381] Updated weights for policy 1, policy_version 764949 (0.0007) [2023-12-27 10:55:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 391798784. Throughput: 0: 9965.6, 1: 10168.5. Samples: 391784188. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:27,363][19170] Avg episode reward: [(0, '666.720'), (1, '922.060')] [2023-12-27 10:55:27,684][20347] Updated weights for policy 0, policy_version 765315 (0.0007) [2023-12-27 10:55:27,734][20347] Updated weights for policy 0, policy_version 765325 (0.0007) [2023-12-27 10:55:27,789][20347] Updated weights for policy 0, policy_version 765335 (0.0007) [2023-12-27 10:55:27,867][20381] Updated weights for policy 1, policy_version 764959 (0.0007) [2023-12-27 10:55:27,927][20381] Updated weights for policy 1, policy_version 764969 (0.0005) [2023-12-27 10:55:27,987][20381] Updated weights for policy 1, policy_version 764979 (0.0005) [2023-12-27 10:55:28,358][20347] Updated weights for policy 0, policy_version 765345 (0.0007) [2023-12-27 10:55:28,418][20347] Updated weights for policy 0, policy_version 765355 (0.0005) [2023-12-27 10:55:28,472][20347] Updated weights for policy 0, policy_version 765365 (0.0005) [2023-12-27 10:55:28,526][20347] Updated weights for policy 0, policy_version 765375 (0.0005) [2023-12-27 10:55:28,535][20381] Updated weights for policy 1, policy_version 764989 (0.0008) [2023-12-27 10:55:28,599][20381] Updated weights for policy 1, policy_version 764999 (0.0010) [2023-12-27 10:55:28,657][20381] Updated weights for policy 1, policy_version 765009 (0.0011) [2023-12-27 10:55:29,112][20347] Updated weights for policy 0, policy_version 765385 (0.0007) [2023-12-27 10:55:29,160][20347] Updated weights for policy 0, policy_version 765395 (0.0007) [2023-12-27 10:55:29,210][20347] Updated weights for policy 0, policy_version 765405 (0.0006) [2023-12-27 10:55:29,332][20381] Updated weights for policy 1, policy_version 765019 (0.0010) [2023-12-27 10:55:29,394][20381] Updated weights for policy 1, policy_version 765029 (0.0007) [2023-12-27 10:55:29,449][20381] Updated weights for policy 1, policy_version 765039 (0.0007) [2023-12-27 10:55:29,899][20347] Updated weights for policy 0, policy_version 765415 (0.0010) [2023-12-27 10:55:29,960][20347] Updated weights for policy 0, policy_version 765425 (0.0009) [2023-12-27 10:55:30,021][20347] Updated weights for policy 0, policy_version 765435 (0.0007) [2023-12-27 10:55:30,143][20381] Updated weights for policy 1, policy_version 765049 (0.0008) [2023-12-27 10:55:30,203][20381] Updated weights for policy 1, policy_version 765059 (0.0008) [2023-12-27 10:55:30,259][20381] Updated weights for policy 1, policy_version 765069 (0.0008) [2023-12-27 10:55:30,308][20381] Updated weights for policy 1, policy_version 765079 (0.0007) [2023-12-27 10:55:30,615][20347] Updated weights for policy 0, policy_version 765445 (0.0009) [2023-12-27 10:55:30,664][20347] Updated weights for policy 0, policy_version 765455 (0.0005) [2023-12-27 10:55:30,709][20347] Updated weights for policy 0, policy_version 765465 (0.0007) [2023-12-27 10:55:31,026][20381] Updated weights for policy 1, policy_version 765089 (0.0006) [2023-12-27 10:55:31,088][20381] Updated weights for policy 1, policy_version 765099 (0.0007) [2023-12-27 10:55:31,154][20381] Updated weights for policy 1, policy_version 765109 (0.0007) [2023-12-27 10:55:31,418][20347] Updated weights for policy 0, policy_version 765475 (0.0009) [2023-12-27 10:55:31,467][20347] Updated weights for policy 0, policy_version 765485 (0.0007) [2023-12-27 10:55:31,514][20347] Updated weights for policy 0, policy_version 765495 (0.0007) [2023-12-27 10:55:31,845][20381] Updated weights for policy 1, policy_version 765119 (0.0007) [2023-12-27 10:55:31,901][20381] Updated weights for policy 1, policy_version 765129 (0.0008) [2023-12-27 10:55:31,960][20381] Updated weights for policy 1, policy_version 765139 (0.0008) [2023-12-27 10:55:32,187][20347] Updated weights for policy 0, policy_version 765505 (0.0007) [2023-12-27 10:55:32,257][20347] Updated weights for policy 0, policy_version 765515 (0.0006) [2023-12-27 10:55:32,318][20347] Updated weights for policy 0, policy_version 765525 (0.0006) [2023-12-27 10:55:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 391897088. Throughput: 0: 10046.6, 1: 10230.4. Samples: 391910924. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:32,363][19170] Avg episode reward: [(0, '796.080'), (1, '921.090')] [2023-12-27 10:55:32,386][20347] Updated weights for policy 0, policy_version 765535 (0.0007) [2023-12-27 10:55:32,657][20381] Updated weights for policy 1, policy_version 765149 (0.0007) [2023-12-27 10:55:32,713][20381] Updated weights for policy 1, policy_version 765159 (0.0006) [2023-12-27 10:55:32,762][20381] Updated weights for policy 1, policy_version 765169 (0.0008) [2023-12-27 10:55:33,041][20347] Updated weights for policy 0, policy_version 765545 (0.0007) [2023-12-27 10:55:33,100][20347] Updated weights for policy 0, policy_version 765555 (0.0005) [2023-12-27 10:55:33,168][20347] Updated weights for policy 0, policy_version 765565 (0.0006) [2023-12-27 10:55:33,302][20381] Updated weights for policy 1, policy_version 765179 (0.0006) [2023-12-27 10:55:33,350][20381] Updated weights for policy 1, policy_version 765189 (0.0005) [2023-12-27 10:55:33,404][20381] Updated weights for policy 1, policy_version 765199 (0.0009) [2023-12-27 10:55:33,764][20347] Updated weights for policy 0, policy_version 765575 (0.0009) [2023-12-27 10:55:33,812][20347] Updated weights for policy 0, policy_version 765585 (0.0010) [2023-12-27 10:55:33,857][20347] Updated weights for policy 0, policy_version 765595 (0.0010) [2023-12-27 10:55:34,000][20381] Updated weights for policy 1, policy_version 765209 (0.0005) [2023-12-27 10:55:34,051][20381] Updated weights for policy 1, policy_version 765219 (0.0008) [2023-12-27 10:55:34,099][20381] Updated weights for policy 1, policy_version 765229 (0.0008) [2023-12-27 10:55:34,159][20381] Updated weights for policy 1, policy_version 765239 (0.0009) [2023-12-27 10:55:34,499][20347] Updated weights for policy 0, policy_version 765605 (0.0008) [2023-12-27 10:55:34,554][20347] Updated weights for policy 0, policy_version 765615 (0.0007) [2023-12-27 10:55:34,610][20347] Updated weights for policy 0, policy_version 765625 (0.0008) [2023-12-27 10:55:34,882][20381] Updated weights for policy 1, policy_version 765249 (0.0007) [2023-12-27 10:55:34,933][20381] Updated weights for policy 1, policy_version 765259 (0.0010) [2023-12-27 10:55:34,981][20381] Updated weights for policy 1, policy_version 765269 (0.0010) [2023-12-27 10:55:35,255][20347] Updated weights for policy 0, policy_version 765635 (0.0007) [2023-12-27 10:55:35,303][20347] Updated weights for policy 0, policy_version 765645 (0.0005) [2023-12-27 10:55:35,359][20347] Updated weights for policy 0, policy_version 765655 (0.0005) [2023-12-27 10:55:35,635][20381] Updated weights for policy 1, policy_version 765279 (0.0006) [2023-12-27 10:55:35,683][20381] Updated weights for policy 1, policy_version 765289 (0.0005) [2023-12-27 10:55:35,736][20381] Updated weights for policy 1, policy_version 765299 (0.0007) [2023-12-27 10:55:35,995][20347] Updated weights for policy 0, policy_version 765665 (0.0006) [2023-12-27 10:55:36,034][20291] KL-divergence is very high: 159.2106 [2023-12-27 10:55:36,050][20347] Updated weights for policy 0, policy_version 765675 (0.0009) [2023-12-27 10:55:36,052][20291] KL-divergence is very high: 324.4159 [2023-12-27 10:55:36,057][20291] KL-divergence is very high: 218.0925 [2023-12-27 10:55:36,069][20291] KL-divergence is very high: 143.0502 [2023-12-27 10:55:36,081][20291] KL-divergence is very high: 126.6025 [2023-12-27 10:55:36,100][20291] KL-divergence is very high: 121.4689 [2023-12-27 10:55:36,111][20347] Updated weights for policy 0, policy_version 765685 (0.0007) [2023-12-27 10:55:36,175][20347] Updated weights for policy 0, policy_version 765695 (0.0008) [2023-12-27 10:55:36,365][20381] Updated weights for policy 1, policy_version 765309 (0.0008) [2023-12-27 10:55:36,432][20381] Updated weights for policy 1, policy_version 765319 (0.0009) [2023-12-27 10:55:36,491][20381] Updated weights for policy 1, policy_version 765329 (0.0006) [2023-12-27 10:55:36,889][20347] Updated weights for policy 0, policy_version 765705 (0.0007) [2023-12-27 10:55:36,940][20347] Updated weights for policy 0, policy_version 765715 (0.0007) [2023-12-27 10:55:36,998][20347] Updated weights for policy 0, policy_version 765725 (0.0007) [2023-12-27 10:55:37,137][20381] Updated weights for policy 1, policy_version 765339 (0.0007) [2023-12-27 10:55:37,188][20381] Updated weights for policy 1, policy_version 765349 (0.0010) [2023-12-27 10:55:37,239][20381] Updated weights for policy 1, policy_version 765359 (0.0007) [2023-12-27 10:55:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20343.4, 300 sec: 20632.7). Total num frames: 392011776. Throughput: 0: 10090.8, 1: 10272.0. Samples: 391975564. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:37,363][19170] Avg episode reward: [(0, '835.650'), (1, '958.340')] [2023-12-27 10:55:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000765728_196050944.pth... [2023-12-27 10:55:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000765368_195960832.pth... [2023-12-27 10:55:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000764544_195747840.pth [2023-12-27 10:55:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000764152_195649536.pth [2023-12-27 10:55:37,681][20347] Updated weights for policy 0, policy_version 765735 (0.0007) [2023-12-27 10:55:37,743][20347] Updated weights for policy 0, policy_version 765745 (0.0007) [2023-12-27 10:55:37,804][20347] Updated weights for policy 0, policy_version 765755 (0.0008) [2023-12-27 10:55:37,892][20381] Updated weights for policy 1, policy_version 765369 (0.0010) [2023-12-27 10:55:37,941][20381] Updated weights for policy 1, policy_version 765379 (0.0005) [2023-12-27 10:55:37,998][20381] Updated weights for policy 1, policy_version 765389 (0.0007) [2023-12-27 10:55:38,047][20381] Updated weights for policy 1, policy_version 765399 (0.0008) [2023-12-27 10:55:38,454][20347] Updated weights for policy 0, policy_version 765765 (0.0009) [2023-12-27 10:55:38,508][20347] Updated weights for policy 0, policy_version 765775 (0.0009) [2023-12-27 10:55:38,568][20347] Updated weights for policy 0, policy_version 765785 (0.0009) [2023-12-27 10:55:38,652][20381] Updated weights for policy 1, policy_version 765409 (0.0007) [2023-12-27 10:55:38,709][20381] Updated weights for policy 1, policy_version 765419 (0.0007) [2023-12-27 10:55:38,768][20381] Updated weights for policy 1, policy_version 765429 (0.0007) [2023-12-27 10:55:39,259][20347] Updated weights for policy 0, policy_version 765795 (0.0010) [2023-12-27 10:55:39,318][20347] Updated weights for policy 0, policy_version 765805 (0.0008) [2023-12-27 10:55:39,382][20347] Updated weights for policy 0, policy_version 765815 (0.0008) [2023-12-27 10:55:39,462][20381] Updated weights for policy 1, policy_version 765439 (0.0009) [2023-12-27 10:55:39,518][20381] Updated weights for policy 1, policy_version 765449 (0.0010) [2023-12-27 10:55:39,577][20381] Updated weights for policy 1, policy_version 765459 (0.0011) [2023-12-27 10:55:40,135][20347] Updated weights for policy 0, policy_version 765825 (0.0008) [2023-12-27 10:55:40,197][20347] Updated weights for policy 0, policy_version 765835 (0.0008) [2023-12-27 10:55:40,260][20347] Updated weights for policy 0, policy_version 765845 (0.0008) [2023-12-27 10:55:40,289][20381] Updated weights for policy 1, policy_version 765469 (0.0009) [2023-12-27 10:55:40,318][20347] Updated weights for policy 0, policy_version 765855 (0.0005) [2023-12-27 10:55:40,348][20381] Updated weights for policy 1, policy_version 765479 (0.0006) [2023-12-27 10:55:40,403][20381] Updated weights for policy 1, policy_version 765489 (0.0007) [2023-12-27 10:55:40,988][20347] Updated weights for policy 0, policy_version 765865 (0.0007) [2023-12-27 10:55:41,054][20347] Updated weights for policy 0, policy_version 765875 (0.0008) [2023-12-27 10:55:41,091][20381] Updated weights for policy 1, policy_version 765499 (0.0005) [2023-12-27 10:55:41,118][20347] Updated weights for policy 0, policy_version 765885 (0.0006) [2023-12-27 10:55:41,159][20381] Updated weights for policy 1, policy_version 765509 (0.0007) [2023-12-27 10:55:41,229][20381] Updated weights for policy 1, policy_version 765519 (0.0008) [2023-12-27 10:55:41,882][20347] Updated weights for policy 0, policy_version 765895 (0.0007) [2023-12-27 10:55:41,888][20291] KL-divergence is very high: 407.4822 [2023-12-27 10:55:41,940][20291] KL-divergence is very high: 411.6541 [2023-12-27 10:55:41,945][20347] Updated weights for policy 0, policy_version 765905 (0.0007) [2023-12-27 10:55:41,991][20381] Updated weights for policy 1, policy_version 765529 (0.0008) [2023-12-27 10:55:41,991][20291] KL-divergence is very high: 129.7229 [2023-12-27 10:55:42,010][20347] Updated weights for policy 0, policy_version 765915 (0.0007) [2023-12-27 10:55:42,046][20381] Updated weights for policy 1, policy_version 765539 (0.0005) [2023-12-27 10:55:42,079][20292] KL-divergence is very high: 113.2497 [2023-12-27 10:55:42,109][20381] Updated weights for policy 1, policy_version 765549 (0.0008) [2023-12-27 10:55:42,170][20381] Updated weights for policy 1, policy_version 765559 (0.0008) [2023-12-27 10:55:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 392110080. Throughput: 0: 10138.4, 1: 10299.7. Samples: 392099332. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-12-27 10:55:42,363][19170] Avg episode reward: [(0, '815.720'), (1, '980.520')] [2023-12-27 10:55:42,642][20291] KL-divergence is very high: 144.4063 [2023-12-27 10:55:42,657][20347] Updated weights for policy 0, policy_version 765925 (0.0006) [2023-12-27 10:55:42,707][20347] Updated weights for policy 0, policy_version 765935 (0.0006) [2023-12-27 10:55:42,763][20347] Updated weights for policy 0, policy_version 765945 (0.0008) [2023-12-27 10:55:42,885][20381] Updated weights for policy 1, policy_version 765569 (0.0006) [2023-12-27 10:55:42,944][20381] Updated weights for policy 1, policy_version 765579 (0.0006) [2023-12-27 10:55:42,990][20381] Updated weights for policy 1, policy_version 765589 (0.0008) [2023-12-27 10:55:43,409][20347] Updated weights for policy 0, policy_version 765955 (0.0008) [2023-12-27 10:55:43,471][20347] Updated weights for policy 0, policy_version 765965 (0.0008) [2023-12-27 10:55:43,535][20347] Updated weights for policy 0, policy_version 765975 (0.0008) [2023-12-27 10:55:43,666][20381] Updated weights for policy 1, policy_version 765599 (0.0008) [2023-12-27 10:55:43,724][20381] Updated weights for policy 1, policy_version 765609 (0.0007) [2023-12-27 10:55:43,781][20381] Updated weights for policy 1, policy_version 765619 (0.0007) [2023-12-27 10:55:44,224][20347] Updated weights for policy 0, policy_version 765985 (0.0008) [2023-12-27 10:55:44,292][20347] Updated weights for policy 0, policy_version 765995 (0.0010) [2023-12-27 10:55:44,350][20347] Updated weights for policy 0, policy_version 766005 (0.0010) [2023-12-27 10:55:44,405][20347] Updated weights for policy 0, policy_version 766015 (0.0010) [2023-12-27 10:55:44,428][20381] Updated weights for policy 1, policy_version 765629 (0.0006) [2023-12-27 10:55:44,483][20381] Updated weights for policy 1, policy_version 765639 (0.0006) [2023-12-27 10:55:44,545][20381] Updated weights for policy 1, policy_version 765649 (0.0007) [2023-12-27 10:55:45,105][20347] Updated weights for policy 0, policy_version 766025 (0.0011) [2023-12-27 10:55:45,169][20347] Updated weights for policy 0, policy_version 766035 (0.0010) [2023-12-27 10:55:45,209][20381] Updated weights for policy 1, policy_version 765659 (0.0007) [2023-12-27 10:55:45,226][20347] Updated weights for policy 0, policy_version 766045 (0.0006) [2023-12-27 10:55:45,264][20381] Updated weights for policy 1, policy_version 765669 (0.0007) [2023-12-27 10:55:45,325][20381] Updated weights for policy 1, policy_version 765679 (0.0008) [2023-12-27 10:55:45,905][20347] Updated weights for policy 0, policy_version 766055 (0.0007) [2023-12-27 10:55:45,963][20347] Updated weights for policy 0, policy_version 766065 (0.0009) [2023-12-27 10:55:45,972][20381] Updated weights for policy 1, policy_version 765689 (0.0008) [2023-12-27 10:55:46,015][20347] Updated weights for policy 0, policy_version 766075 (0.0007) [2023-12-27 10:55:46,024][20381] Updated weights for policy 1, policy_version 765699 (0.0008) [2023-12-27 10:55:46,075][20381] Updated weights for policy 1, policy_version 765709 (0.0009) [2023-12-27 10:55:46,127][20381] Updated weights for policy 1, policy_version 765719 (0.0008) [2023-12-27 10:55:46,646][20291] KL-divergence is very high: 115.5932 [2023-12-27 10:55:46,668][20347] Updated weights for policy 0, policy_version 766085 (0.0006) [2023-12-27 10:55:46,717][20347] Updated weights for policy 0, policy_version 766095 (0.0007) [2023-12-27 10:55:46,727][20381] Updated weights for policy 1, policy_version 765729 (0.0006) [2023-12-27 10:55:46,766][20347] Updated weights for policy 0, policy_version 766105 (0.0005) [2023-12-27 10:55:46,782][20381] Updated weights for policy 1, policy_version 765739 (0.0006) [2023-12-27 10:55:46,840][20381] Updated weights for policy 1, policy_version 765749 (0.0007) [2023-12-27 10:55:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 392208384. Throughput: 0: 10227.5, 1: 10246.2. Samples: 392221248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:55:47,363][19170] Avg episode reward: [(0, '776.900'), (1, '951.540')] [2023-12-27 10:55:47,479][20347] Updated weights for policy 0, policy_version 766115 (0.0005) [2023-12-27 10:55:47,503][20381] Updated weights for policy 1, policy_version 765759 (0.0007) [2023-12-27 10:55:47,504][20291] KL-divergence is very high: 523.5836 [2023-12-27 10:55:47,510][20291] KL-divergence is very high: 632.8586 [2023-12-27 10:55:47,516][20291] KL-divergence is very high: 109.9437 [2023-12-27 10:55:47,541][20347] Updated weights for policy 0, policy_version 766125 (0.0007) [2023-12-27 10:55:47,553][20291] KL-divergence is very high: 115.6971 [2023-12-27 10:55:47,559][20291] KL-divergence is very high: 143.4491 [2023-12-27 10:55:47,561][20381] Updated weights for policy 1, policy_version 765769 (0.0005) [2023-12-27 10:55:47,576][20291] KL-divergence is very high: 157.0138 [2023-12-27 10:55:47,599][20347] Updated weights for policy 0, policy_version 766135 (0.0007) [2023-12-27 10:55:47,609][20291] KL-divergence is very high: 126.7198 [2023-12-27 10:55:47,615][20381] Updated weights for policy 1, policy_version 765779 (0.0006) [2023-12-27 10:55:48,181][20347] Updated weights for policy 0, policy_version 766145 (0.0006) [2023-12-27 10:55:48,190][20291] KL-divergence is very high: 330.3022 [2023-12-27 10:55:48,220][20291] KL-divergence is very high: 109.8153 [2023-12-27 10:55:48,230][20291] KL-divergence is very high: 125.5483 [2023-12-27 10:55:48,234][20347] Updated weights for policy 0, policy_version 766155 (0.0006) [2023-12-27 10:55:48,257][20291] KL-divergence is very high: 201.7145 [2023-12-27 10:55:48,262][20291] KL-divergence is very high: 126.4839 [2023-12-27 10:55:48,267][20291] KL-divergence is very high: 123.3118 [2023-12-27 10:55:48,282][20347] Updated weights for policy 0, policy_version 766165 (0.0005) [2023-12-27 10:55:48,308][20291] KL-divergence is very high: 338.6862 [2023-12-27 10:55:48,338][20347] Updated weights for policy 0, policy_version 766175 (0.0006) [2023-12-27 10:55:48,353][20381] Updated weights for policy 1, policy_version 765789 (0.0006) [2023-12-27 10:55:48,409][20381] Updated weights for policy 1, policy_version 765799 (0.0007) [2023-12-27 10:55:48,472][20381] Updated weights for policy 1, policy_version 765809 (0.0007) [2023-12-27 10:55:48,966][20347] Updated weights for policy 0, policy_version 766185 (0.0008) [2023-12-27 10:55:49,021][20347] Updated weights for policy 0, policy_version 766195 (0.0007) [2023-12-27 10:55:49,078][20347] Updated weights for policy 0, policy_version 766205 (0.0008) [2023-12-27 10:55:49,167][20381] Updated weights for policy 1, policy_version 765819 (0.0006) [2023-12-27 10:55:49,223][20381] Updated weights for policy 1, policy_version 765829 (0.0008) [2023-12-27 10:55:49,288][20381] Updated weights for policy 1, policy_version 765839 (0.0008) [2023-12-27 10:55:49,722][20347] Updated weights for policy 0, policy_version 766215 (0.0008) [2023-12-27 10:55:49,781][20347] Updated weights for policy 0, policy_version 766225 (0.0008) [2023-12-27 10:55:49,847][20347] Updated weights for policy 0, policy_version 766235 (0.0008) [2023-12-27 10:55:50,000][20381] Updated weights for policy 1, policy_version 765849 (0.0008) [2023-12-27 10:55:50,070][20381] Updated weights for policy 1, policy_version 765859 (0.0010) [2023-12-27 10:55:50,134][20381] Updated weights for policy 1, policy_version 765869 (0.0008) [2023-12-27 10:55:50,202][20381] Updated weights for policy 1, policy_version 765879 (0.0011) [2023-12-27 10:55:50,535][20347] Updated weights for policy 0, policy_version 766245 (0.0008) [2023-12-27 10:55:50,597][20347] Updated weights for policy 0, policy_version 766255 (0.0009) [2023-12-27 10:55:50,650][20347] Updated weights for policy 0, policy_version 766265 (0.0007) [2023-12-27 10:55:50,878][20381] Updated weights for policy 1, policy_version 765889 (0.0008) [2023-12-27 10:55:50,941][20381] Updated weights for policy 1, policy_version 765899 (0.0010) [2023-12-27 10:55:50,997][20381] Updated weights for policy 1, policy_version 765909 (0.0006) [2023-12-27 10:55:51,350][20347] Updated weights for policy 0, policy_version 766275 (0.0008) [2023-12-27 10:55:51,416][20347] Updated weights for policy 0, policy_version 766285 (0.0007) [2023-12-27 10:55:51,499][20347] Updated weights for policy 0, policy_version 766295 (0.0007) [2023-12-27 10:55:51,772][20381] Updated weights for policy 1, policy_version 765919 (0.0008) [2023-12-27 10:55:51,830][20381] Updated weights for policy 1, policy_version 765929 (0.0011) [2023-12-27 10:55:51,888][20381] Updated weights for policy 1, policy_version 765939 (0.0011) [2023-12-27 10:55:52,179][20347] Updated weights for policy 0, policy_version 766305 (0.0006) [2023-12-27 10:55:52,235][20347] Updated weights for policy 0, policy_version 766315 (0.0007) [2023-12-27 10:55:52,299][20347] Updated weights for policy 0, policy_version 766325 (0.0007) [2023-12-27 10:55:52,358][20347] Updated weights for policy 0, policy_version 766335 (0.0007) [2023-12-27 10:55:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 392306688. Throughput: 0: 10206.3, 1: 10173.8. Samples: 392282180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:55:52,364][19170] Avg episode reward: [(0, '766.200'), (1, '922.290')] [2023-12-27 10:55:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000766336_196206592.pth... [2023-12-27 10:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000765944_196108288.pth... [2023-12-27 10:55:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000764728_195796992.pth [2023-12-27 10:55:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000765120_195895296.pth [2023-12-27 10:55:52,594][20381] Updated weights for policy 1, policy_version 765949 (0.0010) [2023-12-27 10:55:52,619][20292] KL-divergence is very high: 102.5747 [2023-12-27 10:55:52,655][20381] Updated weights for policy 1, policy_version 765959 (0.0007) [2023-12-27 10:55:52,718][20381] Updated weights for policy 1, policy_version 765969 (0.0008) [2023-12-27 10:55:53,020][20291] KL-divergence is very high: 150.8430 [2023-12-27 10:55:53,026][20347] Updated weights for policy 0, policy_version 766345 (0.0009) [2023-12-27 10:55:53,032][20291] KL-divergence is very high: 191.2764 [2023-12-27 10:55:53,037][20291] KL-divergence is very high: 160.8141 [2023-12-27 10:55:53,047][20291] KL-divergence is very high: 112.2729 [2023-12-27 10:55:53,076][20347] Updated weights for policy 0, policy_version 766355 (0.0009) [2023-12-27 10:55:53,122][20347] Updated weights for policy 0, policy_version 766365 (0.0009) [2023-12-27 10:55:53,349][20381] Updated weights for policy 1, policy_version 765979 (0.0010) [2023-12-27 10:55:53,394][20381] Updated weights for policy 1, policy_version 765989 (0.0010) [2023-12-27 10:55:53,444][20381] Updated weights for policy 1, policy_version 765999 (0.0010) [2023-12-27 10:55:53,812][20347] Updated weights for policy 0, policy_version 766375 (0.0010) [2023-12-27 10:55:53,863][20347] Updated weights for policy 0, policy_version 766385 (0.0010) [2023-12-27 10:55:53,912][20347] Updated weights for policy 0, policy_version 766395 (0.0005) [2023-12-27 10:55:54,134][20381] Updated weights for policy 1, policy_version 766009 (0.0010) [2023-12-27 10:55:54,183][20381] Updated weights for policy 1, policy_version 766019 (0.0005) [2023-12-27 10:55:54,247][20381] Updated weights for policy 1, policy_version 766029 (0.0006) [2023-12-27 10:55:54,303][20381] Updated weights for policy 1, policy_version 766039 (0.0008) [2023-12-27 10:55:54,595][20347] Updated weights for policy 0, policy_version 766405 (0.0007) [2023-12-27 10:55:54,640][20347] Updated weights for policy 0, policy_version 766415 (0.0007) [2023-12-27 10:55:54,691][20347] Updated weights for policy 0, policy_version 766425 (0.0007) [2023-12-27 10:55:54,894][20381] Updated weights for policy 1, policy_version 766049 (0.0010) [2023-12-27 10:55:54,952][20381] Updated weights for policy 1, policy_version 766059 (0.0010) [2023-12-27 10:55:55,013][20381] Updated weights for policy 1, policy_version 766069 (0.0011) [2023-12-27 10:55:55,361][20347] Updated weights for policy 0, policy_version 766435 (0.0010) [2023-12-27 10:55:55,410][20347] Updated weights for policy 0, policy_version 766445 (0.0010) [2023-12-27 10:55:55,460][20347] Updated weights for policy 0, policy_version 766455 (0.0010) [2023-12-27 10:55:55,656][20381] Updated weights for policy 1, policy_version 766079 (0.0007) [2023-12-27 10:55:55,711][20381] Updated weights for policy 1, policy_version 766089 (0.0005) [2023-12-27 10:55:55,774][20381] Updated weights for policy 1, policy_version 766099 (0.0006) [2023-12-27 10:55:56,141][20347] Updated weights for policy 0, policy_version 766465 (0.0011) [2023-12-27 10:55:56,193][20347] Updated weights for policy 0, policy_version 766475 (0.0010) [2023-12-27 10:55:56,241][20347] Updated weights for policy 0, policy_version 766485 (0.0010) [2023-12-27 10:55:56,293][20347] Updated weights for policy 0, policy_version 766495 (0.0010) [2023-12-27 10:55:56,411][20381] Updated weights for policy 1, policy_version 766109 (0.0008) [2023-12-27 10:55:56,461][20381] Updated weights for policy 1, policy_version 766119 (0.0008) [2023-12-27 10:55:56,508][20381] Updated weights for policy 1, policy_version 766129 (0.0008) [2023-12-27 10:55:56,945][20347] Updated weights for policy 0, policy_version 766505 (0.0006) [2023-12-27 10:55:56,993][20347] Updated weights for policy 0, policy_version 766515 (0.0005) [2023-12-27 10:55:57,048][20347] Updated weights for policy 0, policy_version 766525 (0.0010) [2023-12-27 10:55:57,199][20381] Updated weights for policy 1, policy_version 766139 (0.0008) [2023-12-27 10:55:57,248][20381] Updated weights for policy 1, policy_version 766149 (0.0010) [2023-12-27 10:55:57,296][20381] Updated weights for policy 1, policy_version 766159 (0.0009) [2023-12-27 10:55:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 392421376. Throughput: 0: 10207.3, 1: 10258.0. Samples: 392404820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:55:57,363][19170] Avg episode reward: [(0, '775.880'), (1, '922.110')] [2023-12-27 10:55:57,679][20347] Updated weights for policy 0, policy_version 766535 (0.0010) [2023-12-27 10:55:57,736][20347] Updated weights for policy 0, policy_version 766545 (0.0010) [2023-12-27 10:55:57,790][20347] Updated weights for policy 0, policy_version 766555 (0.0009) [2023-12-27 10:55:57,988][20381] Updated weights for policy 1, policy_version 766169 (0.0008) [2023-12-27 10:55:58,058][20381] Updated weights for policy 1, policy_version 766179 (0.0010) [2023-12-27 10:55:58,116][20381] Updated weights for policy 1, policy_version 766189 (0.0010) [2023-12-27 10:55:58,167][20381] Updated weights for policy 1, policy_version 766199 (0.0010) [2023-12-27 10:55:58,487][20347] Updated weights for policy 0, policy_version 766565 (0.0010) [2023-12-27 10:55:58,546][20347] Updated weights for policy 0, policy_version 766575 (0.0010) [2023-12-27 10:55:58,620][20347] Updated weights for policy 0, policy_version 766585 (0.0009) [2023-12-27 10:55:58,863][20381] Updated weights for policy 1, policy_version 766209 (0.0011) [2023-12-27 10:55:58,934][20381] Updated weights for policy 1, policy_version 766219 (0.0010) [2023-12-27 10:55:58,992][20381] Updated weights for policy 1, policy_version 766229 (0.0011) [2023-12-27 10:55:59,298][20347] Updated weights for policy 0, policy_version 766595 (0.0008) [2023-12-27 10:55:59,364][20347] Updated weights for policy 0, policy_version 766605 (0.0007) [2023-12-27 10:55:59,415][20347] Updated weights for policy 0, policy_version 766615 (0.0007) [2023-12-27 10:55:59,661][20381] Updated weights for policy 1, policy_version 766239 (0.0010) [2023-12-27 10:55:59,716][20381] Updated weights for policy 1, policy_version 766249 (0.0010) [2023-12-27 10:55:59,774][20381] Updated weights for policy 1, policy_version 766259 (0.0010) [2023-12-27 10:56:00,073][20347] Updated weights for policy 0, policy_version 766625 (0.0007) [2023-12-27 10:56:00,133][20347] Updated weights for policy 0, policy_version 766635 (0.0007) [2023-12-27 10:56:00,186][20347] Updated weights for policy 0, policy_version 766645 (0.0007) [2023-12-27 10:56:00,241][20347] Updated weights for policy 0, policy_version 766655 (0.0007) [2023-12-27 10:56:00,410][20381] Updated weights for policy 1, policy_version 766269 (0.0008) [2023-12-27 10:56:00,460][20381] Updated weights for policy 1, policy_version 766279 (0.0005) [2023-12-27 10:56:00,505][20381] Updated weights for policy 1, policy_version 766289 (0.0005) [2023-12-27 10:56:00,944][20347] Updated weights for policy 0, policy_version 766665 (0.0006) [2023-12-27 10:56:00,992][20347] Updated weights for policy 0, policy_version 766675 (0.0009) [2023-12-27 10:56:01,031][20381] Updated weights for policy 1, policy_version 766299 (0.0007) [2023-12-27 10:56:01,059][20347] Updated weights for policy 0, policy_version 766685 (0.0007) [2023-12-27 10:56:01,096][20381] Updated weights for policy 1, policy_version 766309 (0.0009) [2023-12-27 10:56:01,166][20381] Updated weights for policy 1, policy_version 766319 (0.0009) [2023-12-27 10:56:01,777][20381] Updated weights for policy 1, policy_version 766329 (0.0006) [2023-12-27 10:56:01,830][20381] Updated weights for policy 1, policy_version 766339 (0.0006) [2023-12-27 10:56:01,849][20347] Updated weights for policy 0, policy_version 766695 (0.0007) [2023-12-27 10:56:01,880][20381] Updated weights for policy 1, policy_version 766349 (0.0007) [2023-12-27 10:56:01,915][20347] Updated weights for policy 0, policy_version 766705 (0.0007) [2023-12-27 10:56:01,935][20381] Updated weights for policy 1, policy_version 766359 (0.0008) [2023-12-27 10:56:01,974][20347] Updated weights for policy 0, policy_version 766715 (0.0007) [2023-12-27 10:56:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 392519680. Throughput: 0: 10247.3, 1: 10308.5. Samples: 392529304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:02,363][19170] Avg episode reward: [(0, '824.300'), (1, '951.290')] [2023-12-27 10:56:02,622][20381] Updated weights for policy 1, policy_version 766369 (0.0007) [2023-12-27 10:56:02,649][20347] Updated weights for policy 0, policy_version 766725 (0.0008) [2023-12-27 10:56:02,669][20381] Updated weights for policy 1, policy_version 766379 (0.0005) [2023-12-27 10:56:02,706][20347] Updated weights for policy 0, policy_version 766735 (0.0006) [2023-12-27 10:56:02,716][20381] Updated weights for policy 1, policy_version 766389 (0.0007) [2023-12-27 10:56:02,763][20347] Updated weights for policy 0, policy_version 766745 (0.0007) [2023-12-27 10:56:03,413][20381] Updated weights for policy 1, policy_version 766399 (0.0006) [2023-12-27 10:56:03,417][20347] Updated weights for policy 0, policy_version 766755 (0.0008) [2023-12-27 10:56:03,464][20381] Updated weights for policy 1, policy_version 766409 (0.0005) [2023-12-27 10:56:03,474][20347] Updated weights for policy 0, policy_version 766765 (0.0008) [2023-12-27 10:56:03,515][20381] Updated weights for policy 1, policy_version 766419 (0.0006) [2023-12-27 10:56:03,530][20347] Updated weights for policy 0, policy_version 766775 (0.0006) [2023-12-27 10:56:04,167][20381] Updated weights for policy 1, policy_version 766429 (0.0007) [2023-12-27 10:56:04,221][20347] Updated weights for policy 0, policy_version 766785 (0.0009) [2023-12-27 10:56:04,230][20381] Updated weights for policy 1, policy_version 766439 (0.0008) [2023-12-27 10:56:04,279][20347] Updated weights for policy 0, policy_version 766795 (0.0010) [2023-12-27 10:56:04,288][20381] Updated weights for policy 1, policy_version 766449 (0.0006) [2023-12-27 10:56:04,338][20347] Updated weights for policy 0, policy_version 766805 (0.0010) [2023-12-27 10:56:04,400][20347] Updated weights for policy 0, policy_version 766815 (0.0011) [2023-12-27 10:56:04,993][20381] Updated weights for policy 1, policy_version 766459 (0.0007) [2023-12-27 10:56:05,052][20381] Updated weights for policy 1, policy_version 766469 (0.0006) [2023-12-27 10:56:05,063][20347] Updated weights for policy 0, policy_version 766825 (0.0008) [2023-12-27 10:56:05,099][20381] Updated weights for policy 1, policy_version 766479 (0.0007) [2023-12-27 10:56:05,109][20347] Updated weights for policy 0, policy_version 766835 (0.0006) [2023-12-27 10:56:05,159][20347] Updated weights for policy 0, policy_version 766845 (0.0007) [2023-12-27 10:56:05,762][20381] Updated weights for policy 1, policy_version 766489 (0.0007) [2023-12-27 10:56:05,778][20347] Updated weights for policy 0, policy_version 766855 (0.0006) [2023-12-27 10:56:05,817][20381] Updated weights for policy 1, policy_version 766499 (0.0005) [2023-12-27 10:56:05,834][20347] Updated weights for policy 0, policy_version 766865 (0.0006) [2023-12-27 10:56:05,866][20381] Updated weights for policy 1, policy_version 766509 (0.0005) [2023-12-27 10:56:05,883][20347] Updated weights for policy 0, policy_version 766875 (0.0006) [2023-12-27 10:56:05,915][20381] Updated weights for policy 1, policy_version 766519 (0.0005) [2023-12-27 10:56:06,575][20381] Updated weights for policy 1, policy_version 766529 (0.0007) [2023-12-27 10:56:06,623][20347] Updated weights for policy 0, policy_version 766885 (0.0006) [2023-12-27 10:56:06,629][20381] Updated weights for policy 1, policy_version 766539 (0.0010) [2023-12-27 10:56:06,630][20292] KL-divergence is very high: 795.6708 [2023-12-27 10:56:06,635][20292] KL-divergence is very high: 726.3255 [2023-12-27 10:56:06,673][20292] KL-divergence is very high: 101.8199 [2023-12-27 10:56:06,680][20347] Updated weights for policy 0, policy_version 766895 (0.0005) [2023-12-27 10:56:06,686][20381] Updated weights for policy 1, policy_version 766549 (0.0011) [2023-12-27 10:56:06,741][20347] Updated weights for policy 0, policy_version 766905 (0.0007) [2023-12-27 10:56:07,352][20292] KL-divergence is very high: 610.9084 [2023-12-27 10:56:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 392617984. Throughput: 0: 10259.8, 1: 10348.3. Samples: 392590724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:07,363][19170] Avg episode reward: [(0, '815.200'), (1, '873.350')] [2023-12-27 10:56:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000766912_196354048.pth... [2023-12-27 10:56:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000765728_196050944.pth [2023-12-27 10:56:07,388][20381] Updated weights for policy 1, policy_version 766559 (0.0010) [2023-12-27 10:56:07,399][20292] KL-divergence is very high: 397.7640 [2023-12-27 10:56:07,439][20347] Updated weights for policy 0, policy_version 766915 (0.0007) [2023-12-27 10:56:07,445][20381] Updated weights for policy 1, policy_version 766569 (0.0009) [2023-12-27 10:56:07,447][20292] KL-divergence is very high: 680.8421 [2023-12-27 10:56:07,493][20292] KL-divergence is very high: 641.3344 [2023-12-27 10:56:07,497][20347] Updated weights for policy 0, policy_version 766925 (0.0010) [2023-12-27 10:56:07,503][20381] Updated weights for policy 1, policy_version 766579 (0.0010) [2023-12-27 10:56:07,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000766584_196272128.pth... [2023-12-27 10:56:07,533][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000765368_195960832.pth [2023-12-27 10:56:07,554][20347] Updated weights for policy 0, policy_version 766935 (0.0007) [2023-12-27 10:56:08,150][20347] Updated weights for policy 0, policy_version 766945 (0.0008) [2023-12-27 10:56:08,168][20381] Updated weights for policy 1, policy_version 766589 (0.0011) [2023-12-27 10:56:08,202][20347] Updated weights for policy 0, policy_version 766955 (0.0010) [2023-12-27 10:56:08,222][20381] Updated weights for policy 1, policy_version 766599 (0.0010) [2023-12-27 10:56:08,256][20347] Updated weights for policy 0, policy_version 766965 (0.0010) [2023-12-27 10:56:08,274][20381] Updated weights for policy 1, policy_version 766609 (0.0010) [2023-12-27 10:56:08,308][20347] Updated weights for policy 0, policy_version 766975 (0.0010) [2023-12-27 10:56:08,933][20381] Updated weights for policy 1, policy_version 766619 (0.0011) [2023-12-27 10:56:08,980][20381] Updated weights for policy 1, policy_version 766629 (0.0010) [2023-12-27 10:56:08,991][20347] Updated weights for policy 0, policy_version 766985 (0.0010) [2023-12-27 10:56:09,033][20381] Updated weights for policy 1, policy_version 766639 (0.0011) [2023-12-27 10:56:09,040][20347] Updated weights for policy 0, policy_version 766995 (0.0008) [2023-12-27 10:56:09,095][20347] Updated weights for policy 0, policy_version 767005 (0.0007) [2023-12-27 10:56:09,756][20381] Updated weights for policy 1, policy_version 766649 (0.0011) [2023-12-27 10:56:09,817][20381] Updated weights for policy 1, policy_version 766659 (0.0011) [2023-12-27 10:56:09,823][20347] Updated weights for policy 0, policy_version 767015 (0.0009) [2023-12-27 10:56:09,881][20381] Updated weights for policy 1, policy_version 766669 (0.0011) [2023-12-27 10:56:09,890][20347] Updated weights for policy 0, policy_version 767025 (0.0007) [2023-12-27 10:56:09,942][20381] Updated weights for policy 1, policy_version 766679 (0.0011) [2023-12-27 10:56:09,949][20347] Updated weights for policy 0, policy_version 767035 (0.0006) [2023-12-27 10:56:10,613][20381] Updated weights for policy 1, policy_version 766689 (0.0007) [2023-12-27 10:56:10,619][20347] Updated weights for policy 0, policy_version 767045 (0.0006) [2023-12-27 10:56:10,663][20347] Updated weights for policy 0, policy_version 767055 (0.0005) [2023-12-27 10:56:10,669][20381] Updated weights for policy 1, policy_version 766699 (0.0006) [2023-12-27 10:56:10,713][20347] Updated weights for policy 0, policy_version 767065 (0.0006) [2023-12-27 10:56:10,716][20381] Updated weights for policy 1, policy_version 766709 (0.0007) [2023-12-27 10:56:11,406][20381] Updated weights for policy 1, policy_version 766719 (0.0008) [2023-12-27 10:56:11,412][20347] Updated weights for policy 0, policy_version 767075 (0.0008) [2023-12-27 10:56:11,467][20347] Updated weights for policy 0, policy_version 767085 (0.0008) [2023-12-27 10:56:11,470][20381] Updated weights for policy 1, policy_version 766729 (0.0008) [2023-12-27 10:56:11,526][20347] Updated weights for policy 0, policy_version 767095 (0.0009) [2023-12-27 10:56:11,531][20381] Updated weights for policy 1, policy_version 766739 (0.0009) [2023-12-27 10:56:12,239][20381] Updated weights for policy 1, policy_version 766749 (0.0007) [2023-12-27 10:56:12,265][20347] Updated weights for policy 0, policy_version 767105 (0.0008) [2023-12-27 10:56:12,304][20381] Updated weights for policy 1, policy_version 766759 (0.0007) [2023-12-27 10:56:12,329][20347] Updated weights for policy 0, policy_version 767115 (0.0010) [2023-12-27 10:56:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 392716288. Throughput: 0: 10325.9, 1: 10363.1. Samples: 392715192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:12,363][19170] Avg episode reward: [(0, '873.470'), (1, '824.490')] [2023-12-27 10:56:12,366][20381] Updated weights for policy 1, policy_version 766769 (0.0008) [2023-12-27 10:56:12,393][20347] Updated weights for policy 0, policy_version 767125 (0.0009) [2023-12-27 10:56:12,452][20347] Updated weights for policy 0, policy_version 767135 (0.0008) [2023-12-27 10:56:13,082][20381] Updated weights for policy 1, policy_version 766779 (0.0006) [2023-12-27 10:56:13,138][20381] Updated weights for policy 1, policy_version 766789 (0.0006) [2023-12-27 10:56:13,139][20347] Updated weights for policy 0, policy_version 767145 (0.0010) [2023-12-27 10:56:13,189][20381] Updated weights for policy 1, policy_version 766799 (0.0007) [2023-12-27 10:56:13,197][20347] Updated weights for policy 0, policy_version 767155 (0.0010) [2023-12-27 10:56:13,249][20347] Updated weights for policy 0, policy_version 767165 (0.0010) [2023-12-27 10:56:13,909][20381] Updated weights for policy 1, policy_version 766809 (0.0007) [2023-12-27 10:56:13,930][20347] Updated weights for policy 0, policy_version 767175 (0.0009) [2023-12-27 10:56:13,970][20381] Updated weights for policy 1, policy_version 766819 (0.0007) [2023-12-27 10:56:13,984][20347] Updated weights for policy 0, policy_version 767185 (0.0006) [2023-12-27 10:56:14,035][20381] Updated weights for policy 1, policy_version 766829 (0.0009) [2023-12-27 10:56:14,045][20347] Updated weights for policy 0, policy_version 767195 (0.0007) [2023-12-27 10:56:14,089][20381] Updated weights for policy 1, policy_version 766839 (0.0008) [2023-12-27 10:56:14,719][20347] Updated weights for policy 0, policy_version 767205 (0.0007) [2023-12-27 10:56:14,734][20381] Updated weights for policy 1, policy_version 766849 (0.0007) [2023-12-27 10:56:14,776][20347] Updated weights for policy 0, policy_version 767215 (0.0006) [2023-12-27 10:56:14,801][20381] Updated weights for policy 1, policy_version 766859 (0.0008) [2023-12-27 10:56:14,846][20347] Updated weights for policy 0, policy_version 767225 (0.0008) [2023-12-27 10:56:14,864][20381] Updated weights for policy 1, policy_version 766869 (0.0006) [2023-12-27 10:56:15,418][20347] Updated weights for policy 0, policy_version 767235 (0.0007) [2023-12-27 10:56:15,465][20347] Updated weights for policy 0, policy_version 767245 (0.0007) [2023-12-27 10:56:15,511][20347] Updated weights for policy 0, policy_version 767255 (0.0006) [2023-12-27 10:56:15,611][20381] Updated weights for policy 1, policy_version 766879 (0.0006) [2023-12-27 10:56:15,674][20381] Updated weights for policy 1, policy_version 766889 (0.0005) [2023-12-27 10:56:15,739][20381] Updated weights for policy 1, policy_version 766899 (0.0007) [2023-12-27 10:56:16,185][20347] Updated weights for policy 0, policy_version 767265 (0.0007) [2023-12-27 10:56:16,238][20347] Updated weights for policy 0, policy_version 767275 (0.0007) [2023-12-27 10:56:16,292][20347] Updated weights for policy 0, policy_version 767285 (0.0006) [2023-12-27 10:56:16,329][20381] Updated weights for policy 1, policy_version 766909 (0.0007) [2023-12-27 10:56:16,348][20347] Updated weights for policy 0, policy_version 767295 (0.0007) [2023-12-27 10:56:16,390][20381] Updated weights for policy 1, policy_version 766919 (0.0006) [2023-12-27 10:56:16,438][20381] Updated weights for policy 1, policy_version 766929 (0.0008) [2023-12-27 10:56:16,974][20347] Updated weights for policy 0, policy_version 767305 (0.0007) [2023-12-27 10:56:17,031][20347] Updated weights for policy 0, policy_version 767315 (0.0006) [2023-12-27 10:56:17,080][20347] Updated weights for policy 0, policy_version 767325 (0.0007) [2023-12-27 10:56:17,144][20381] Updated weights for policy 1, policy_version 766939 (0.0010) [2023-12-27 10:56:17,197][20381] Updated weights for policy 1, policy_version 766949 (0.0007) [2023-12-27 10:56:17,243][20381] Updated weights for policy 1, policy_version 766959 (0.0008) [2023-12-27 10:56:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 392830976. Throughput: 0: 10257.1, 1: 10310.4. Samples: 392836464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:17,363][19170] Avg episode reward: [(0, '921.900'), (1, '824.480')] [2023-12-27 10:56:17,649][20347] Updated weights for policy 0, policy_version 767335 (0.0006) [2023-12-27 10:56:17,699][20347] Updated weights for policy 0, policy_version 767345 (0.0005) [2023-12-27 10:56:17,753][20347] Updated weights for policy 0, policy_version 767355 (0.0005) [2023-12-27 10:56:18,024][20381] Updated weights for policy 1, policy_version 766969 (0.0008) [2023-12-27 10:56:18,078][20381] Updated weights for policy 1, policy_version 766979 (0.0008) [2023-12-27 10:56:18,143][20381] Updated weights for policy 1, policy_version 766989 (0.0007) [2023-12-27 10:56:18,207][20381] Updated weights for policy 1, policy_version 766999 (0.0007) [2023-12-27 10:56:18,343][20347] Updated weights for policy 0, policy_version 767365 (0.0007) [2023-12-27 10:56:18,405][20347] Updated weights for policy 0, policy_version 767375 (0.0007) [2023-12-27 10:56:18,465][20347] Updated weights for policy 0, policy_version 767385 (0.0008) [2023-12-27 10:56:18,876][20381] Updated weights for policy 1, policy_version 767009 (0.0008) [2023-12-27 10:56:18,943][20381] Updated weights for policy 1, policy_version 767019 (0.0006) [2023-12-27 10:56:19,012][20381] Updated weights for policy 1, policy_version 767029 (0.0007) [2023-12-27 10:56:19,157][20347] Updated weights for policy 0, policy_version 767395 (0.0007) [2023-12-27 10:56:19,210][20347] Updated weights for policy 0, policy_version 767405 (0.0007) [2023-12-27 10:56:19,272][20347] Updated weights for policy 0, policy_version 767415 (0.0008) [2023-12-27 10:56:19,703][20381] Updated weights for policy 1, policy_version 767039 (0.0008) [2023-12-27 10:56:19,767][20381] Updated weights for policy 1, policy_version 767049 (0.0007) [2023-12-27 10:56:19,859][20381] Updated weights for policy 1, policy_version 767059 (0.0008) [2023-12-27 10:56:20,031][20347] Updated weights for policy 0, policy_version 767425 (0.0008) [2023-12-27 10:56:20,095][20347] Updated weights for policy 0, policy_version 767435 (0.0009) [2023-12-27 10:56:20,155][20347] Updated weights for policy 0, policy_version 767445 (0.0011) [2023-12-27 10:56:20,215][20347] Updated weights for policy 0, policy_version 767455 (0.0011) [2023-12-27 10:56:20,546][20381] Updated weights for policy 1, policy_version 767069 (0.0007) [2023-12-27 10:56:20,609][20381] Updated weights for policy 1, policy_version 767079 (0.0008) [2023-12-27 10:56:20,671][20381] Updated weights for policy 1, policy_version 767089 (0.0007) [2023-12-27 10:56:20,898][20347] Updated weights for policy 0, policy_version 767465 (0.0010) [2023-12-27 10:56:20,955][20347] Updated weights for policy 0, policy_version 767475 (0.0008) [2023-12-27 10:56:21,021][20347] Updated weights for policy 0, policy_version 767485 (0.0010) [2023-12-27 10:56:21,383][20381] Updated weights for policy 1, policy_version 767099 (0.0008) [2023-12-27 10:56:21,444][20381] Updated weights for policy 1, policy_version 767109 (0.0008) [2023-12-27 10:56:21,497][20381] Updated weights for policy 1, policy_version 767119 (0.0006) [2023-12-27 10:56:21,738][20347] Updated weights for policy 0, policy_version 767495 (0.0009) [2023-12-27 10:56:21,805][20347] Updated weights for policy 0, policy_version 767505 (0.0008) [2023-12-27 10:56:21,865][20347] Updated weights for policy 0, policy_version 767515 (0.0007) [2023-12-27 10:56:22,163][20381] Updated weights for policy 1, policy_version 767129 (0.0008) [2023-12-27 10:56:22,216][20381] Updated weights for policy 1, policy_version 767139 (0.0006) [2023-12-27 10:56:22,274][20381] Updated weights for policy 1, policy_version 767149 (0.0008) [2023-12-27 10:56:22,332][20381] Updated weights for policy 1, policy_version 767159 (0.0008) [2023-12-27 10:56:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 392929280. Throughput: 0: 10258.3, 1: 10223.0. Samples: 392897220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:22,363][19170] Avg episode reward: [(0, '931.730'), (1, '833.580')] [2023-12-27 10:56:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000767160_196419584.pth... [2023-12-27 10:56:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000767520_196509696.pth... [2023-12-27 10:56:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000766336_196206592.pth [2023-12-27 10:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000765944_196108288.pth [2023-12-27 10:56:22,558][20347] Updated weights for policy 0, policy_version 767525 (0.0006) [2023-12-27 10:56:22,617][20347] Updated weights for policy 0, policy_version 767535 (0.0007) [2023-12-27 10:56:22,674][20347] Updated weights for policy 0, policy_version 767545 (0.0008) [2023-12-27 10:56:23,079][20381] Updated weights for policy 1, policy_version 767169 (0.0008) [2023-12-27 10:56:23,136][20381] Updated weights for policy 1, policy_version 767179 (0.0007) [2023-12-27 10:56:23,189][20381] Updated weights for policy 1, policy_version 767189 (0.0008) [2023-12-27 10:56:23,343][20347] Updated weights for policy 0, policy_version 767555 (0.0008) [2023-12-27 10:56:23,397][20347] Updated weights for policy 0, policy_version 767565 (0.0008) [2023-12-27 10:56:23,452][20347] Updated weights for policy 0, policy_version 767575 (0.0007) [2023-12-27 10:56:23,823][20381] Updated weights for policy 1, policy_version 767199 (0.0008) [2023-12-27 10:56:23,880][20381] Updated weights for policy 1, policy_version 767209 (0.0008) [2023-12-27 10:56:23,940][20381] Updated weights for policy 1, policy_version 767219 (0.0009) [2023-12-27 10:56:24,065][20347] Updated weights for policy 0, policy_version 767585 (0.0007) [2023-12-27 10:56:24,124][20347] Updated weights for policy 0, policy_version 767595 (0.0006) [2023-12-27 10:56:24,180][20347] Updated weights for policy 0, policy_version 767605 (0.0007) [2023-12-27 10:56:24,236][20347] Updated weights for policy 0, policy_version 767615 (0.0005) [2023-12-27 10:56:24,634][20381] Updated weights for policy 1, policy_version 767229 (0.0007) [2023-12-27 10:56:24,676][20292] KL-divergence is very high: 170.5419 [2023-12-27 10:56:24,682][20292] KL-divergence is very high: 123.0123 [2023-12-27 10:56:24,694][20381] Updated weights for policy 1, policy_version 767239 (0.0006) [2023-12-27 10:56:24,755][20381] Updated weights for policy 1, policy_version 767249 (0.0008) [2023-12-27 10:56:24,857][20347] Updated weights for policy 0, policy_version 767625 (0.0009) [2023-12-27 10:56:24,902][20347] Updated weights for policy 0, policy_version 767635 (0.0010) [2023-12-27 10:56:24,953][20347] Updated weights for policy 0, policy_version 767645 (0.0010) [2023-12-27 10:56:25,448][20381] Updated weights for policy 1, policy_version 767259 (0.0008) [2023-12-27 10:56:25,504][20381] Updated weights for policy 1, policy_version 767269 (0.0008) [2023-12-27 10:56:25,555][20381] Updated weights for policy 1, policy_version 767279 (0.0008) [2023-12-27 10:56:25,570][20347] Updated weights for policy 0, policy_version 767655 (0.0006) [2023-12-27 10:56:25,621][20347] Updated weights for policy 0, policy_version 767665 (0.0005) [2023-12-27 10:56:25,671][20347] Updated weights for policy 0, policy_version 767675 (0.0005) [2023-12-27 10:56:26,204][20347] Updated weights for policy 0, policy_version 767685 (0.0008) [2023-12-27 10:56:26,224][20381] Updated weights for policy 1, policy_version 767289 (0.0008) [2023-12-27 10:56:26,261][20347] Updated weights for policy 0, policy_version 767695 (0.0011) [2023-12-27 10:56:26,281][20381] Updated weights for policy 1, policy_version 767299 (0.0006) [2023-12-27 10:56:26,313][20347] Updated weights for policy 0, policy_version 767705 (0.0010) [2023-12-27 10:56:26,337][20381] Updated weights for policy 1, policy_version 767309 (0.0005) [2023-12-27 10:56:26,385][20381] Updated weights for policy 1, policy_version 767319 (0.0010) [2023-12-27 10:56:26,985][20347] Updated weights for policy 0, policy_version 767715 (0.0010) [2023-12-27 10:56:27,040][20381] Updated weights for policy 1, policy_version 767329 (0.0011) [2023-12-27 10:56:27,042][20347] Updated weights for policy 0, policy_version 767725 (0.0010) [2023-12-27 10:56:27,095][20381] Updated weights for policy 1, policy_version 767339 (0.0006) [2023-12-27 10:56:27,096][20347] Updated weights for policy 0, policy_version 767735 (0.0010) [2023-12-27 10:56:27,152][20381] Updated weights for policy 1, policy_version 767349 (0.0005) [2023-12-27 10:56:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 393035776. Throughput: 0: 10309.8, 1: 10166.1. Samples: 393020752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:27,364][19170] Avg episode reward: [(0, '941.390'), (1, '881.460')] [2023-12-27 10:56:27,688][20381] Updated weights for policy 1, policy_version 767359 (0.0009) [2023-12-27 10:56:27,741][20381] Updated weights for policy 1, policy_version 767369 (0.0005) [2023-12-27 10:56:27,763][20347] Updated weights for policy 0, policy_version 767745 (0.0010) [2023-12-27 10:56:27,791][20381] Updated weights for policy 1, policy_version 767379 (0.0005) [2023-12-27 10:56:27,814][20347] Updated weights for policy 0, policy_version 767755 (0.0010) [2023-12-27 10:56:27,868][20347] Updated weights for policy 0, policy_version 767765 (0.0010) [2023-12-27 10:56:27,919][20347] Updated weights for policy 0, policy_version 767775 (0.0010) [2023-12-27 10:56:28,465][20381] Updated weights for policy 1, policy_version 767389 (0.0008) [2023-12-27 10:56:28,517][20381] Updated weights for policy 1, policy_version 767399 (0.0010) [2023-12-27 10:56:28,549][20347] Updated weights for policy 0, policy_version 767785 (0.0006) [2023-12-27 10:56:28,569][20381] Updated weights for policy 1, policy_version 767409 (0.0010) [2023-12-27 10:56:28,600][20347] Updated weights for policy 0, policy_version 767795 (0.0006) [2023-12-27 10:56:28,661][20347] Updated weights for policy 0, policy_version 767805 (0.0007) [2023-12-27 10:56:29,299][20381] Updated weights for policy 1, policy_version 767419 (0.0009) [2023-12-27 10:56:29,359][20381] Updated weights for policy 1, policy_version 767429 (0.0008) [2023-12-27 10:56:29,362][20347] Updated weights for policy 0, policy_version 767815 (0.0007) [2023-12-27 10:56:29,413][20292] KL-divergence is very high: 228.2797 [2023-12-27 10:56:29,418][20347] Updated weights for policy 0, policy_version 767825 (0.0006) [2023-12-27 10:56:29,418][20381] Updated weights for policy 1, policy_version 767439 (0.0006) [2023-12-27 10:56:29,431][20292] KL-divergence is very high: 252.7850 [2023-12-27 10:56:29,442][20292] KL-divergence is very high: 380.8920 [2023-12-27 10:56:29,460][20292] KL-divergence is very high: 190.5599 [2023-12-27 10:56:29,474][20347] Updated weights for policy 0, policy_version 767835 (0.0006) [2023-12-27 10:56:30,041][20381] Updated weights for policy 1, policy_version 767449 (0.0006) [2023-12-27 10:56:30,098][20381] Updated weights for policy 1, policy_version 767459 (0.0008) [2023-12-27 10:56:30,124][20347] Updated weights for policy 0, policy_version 767845 (0.0006) [2023-12-27 10:56:30,153][20381] Updated weights for policy 1, policy_version 767469 (0.0008) [2023-12-27 10:56:30,179][20347] Updated weights for policy 0, policy_version 767855 (0.0006) [2023-12-27 10:56:30,211][20381] Updated weights for policy 1, policy_version 767479 (0.0008) [2023-12-27 10:56:30,236][20347] Updated weights for policy 0, policy_version 767865 (0.0005) [2023-12-27 10:56:30,880][20381] Updated weights for policy 1, policy_version 767489 (0.0006) [2023-12-27 10:56:30,900][20347] Updated weights for policy 0, policy_version 767875 (0.0005) [2023-12-27 10:56:30,934][20381] Updated weights for policy 1, policy_version 767499 (0.0005) [2023-12-27 10:56:30,959][20347] Updated weights for policy 0, policy_version 767885 (0.0005) [2023-12-27 10:56:30,984][20381] Updated weights for policy 1, policy_version 767509 (0.0005) [2023-12-27 10:56:31,018][20347] Updated weights for policy 0, policy_version 767895 (0.0006) [2023-12-27 10:56:31,617][20381] Updated weights for policy 1, policy_version 767519 (0.0007) [2023-12-27 10:56:31,636][20347] Updated weights for policy 0, policy_version 767905 (0.0007) [2023-12-27 10:56:31,675][20381] Updated weights for policy 1, policy_version 767529 (0.0007) [2023-12-27 10:56:31,696][20347] Updated weights for policy 0, policy_version 767915 (0.0005) [2023-12-27 10:56:31,732][20381] Updated weights for policy 1, policy_version 767539 (0.0008) [2023-12-27 10:56:31,761][20347] Updated weights for policy 0, policy_version 767925 (0.0007) [2023-12-27 10:56:31,813][20347] Updated weights for policy 0, policy_version 767935 (0.0009) [2023-12-27 10:56:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 393134080. Throughput: 0: 10368.5, 1: 10223.8. Samples: 393147900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:32,363][19170] Avg episode reward: [(0, '960.950'), (1, '776.150')] [2023-12-27 10:56:32,400][20381] Updated weights for policy 1, policy_version 767549 (0.0006) [2023-12-27 10:56:32,414][20292] KL-divergence is very high: 193.4450 [2023-12-27 10:56:32,451][20381] Updated weights for policy 1, policy_version 767559 (0.0007) [2023-12-27 10:56:32,455][20347] Updated weights for policy 0, policy_version 767945 (0.0006) [2023-12-27 10:56:32,502][20381] Updated weights for policy 1, policy_version 767569 (0.0007) [2023-12-27 10:56:32,517][20347] Updated weights for policy 0, policy_version 767955 (0.0005) [2023-12-27 10:56:32,580][20347] Updated weights for policy 0, policy_version 767965 (0.0006) [2023-12-27 10:56:33,145][20347] Updated weights for policy 0, policy_version 767975 (0.0005) [2023-12-27 10:56:33,195][20347] Updated weights for policy 0, policy_version 767985 (0.0005) [2023-12-27 10:56:33,211][20381] Updated weights for policy 1, policy_version 767579 (0.0007) [2023-12-27 10:56:33,247][20347] Updated weights for policy 0, policy_version 767995 (0.0005) [2023-12-27 10:56:33,259][20381] Updated weights for policy 1, policy_version 767589 (0.0008) [2023-12-27 10:56:33,315][20381] Updated weights for policy 1, policy_version 767599 (0.0007) [2023-12-27 10:56:33,832][20347] Updated weights for policy 0, policy_version 768005 (0.0008) [2023-12-27 10:56:33,890][20347] Updated weights for policy 0, policy_version 768015 (0.0009) [2023-12-27 10:56:33,947][20347] Updated weights for policy 0, policy_version 768025 (0.0008) [2023-12-27 10:56:33,969][20381] Updated weights for policy 1, policy_version 767609 (0.0008) [2023-12-27 10:56:34,027][20381] Updated weights for policy 1, policy_version 767619 (0.0007) [2023-12-27 10:56:34,080][20381] Updated weights for policy 1, policy_version 767629 (0.0011) [2023-12-27 10:56:34,141][20381] Updated weights for policy 1, policy_version 767639 (0.0009) [2023-12-27 10:56:34,664][20347] Updated weights for policy 0, policy_version 768035 (0.0006) [2023-12-27 10:56:34,728][20347] Updated weights for policy 0, policy_version 768045 (0.0005) [2023-12-27 10:56:34,790][20347] Updated weights for policy 0, policy_version 768055 (0.0007) [2023-12-27 10:56:34,805][20381] Updated weights for policy 1, policy_version 767649 (0.0010) [2023-12-27 10:56:34,857][20381] Updated weights for policy 1, policy_version 767659 (0.0010) [2023-12-27 10:56:34,902][20381] Updated weights for policy 1, policy_version 767669 (0.0009) [2023-12-27 10:56:35,419][20347] Updated weights for policy 0, policy_version 768065 (0.0006) [2023-12-27 10:56:35,477][20347] Updated weights for policy 0, policy_version 768075 (0.0008) [2023-12-27 10:56:35,542][20347] Updated weights for policy 0, policy_version 768085 (0.0007) [2023-12-27 10:56:35,547][20381] Updated weights for policy 1, policy_version 767679 (0.0006) [2023-12-27 10:56:35,597][20347] Updated weights for policy 0, policy_version 768095 (0.0005) [2023-12-27 10:56:35,598][20381] Updated weights for policy 1, policy_version 767689 (0.0009) [2023-12-27 10:56:35,658][20381] Updated weights for policy 1, policy_version 767699 (0.0011) [2023-12-27 10:56:36,206][20381] Updated weights for policy 1, policy_version 767709 (0.0008) [2023-12-27 10:56:36,272][20381] Updated weights for policy 1, policy_version 767719 (0.0006) [2023-12-27 10:56:36,300][20347] Updated weights for policy 0, policy_version 768105 (0.0010) [2023-12-27 10:56:36,333][20381] Updated weights for policy 1, policy_version 767729 (0.0011) [2023-12-27 10:56:36,352][20347] Updated weights for policy 0, policy_version 768115 (0.0007) [2023-12-27 10:56:36,404][20347] Updated weights for policy 0, policy_version 768125 (0.0009) [2023-12-27 10:56:36,971][20381] Updated weights for policy 1, policy_version 767739 (0.0011) [2023-12-27 10:56:36,991][20292] KL-divergence is very high: 157.2893 [2023-12-27 10:56:37,020][20381] Updated weights for policy 1, policy_version 767749 (0.0011) [2023-12-27 10:56:37,076][20381] Updated weights for policy 1, policy_version 767759 (0.0011) [2023-12-27 10:56:37,086][20347] Updated weights for policy 0, policy_version 768135 (0.0010) [2023-12-27 10:56:37,108][20291] KL-divergence is very high: 163.7952 [2023-12-27 10:56:37,142][20347] Updated weights for policy 0, policy_version 768145 (0.0007) [2023-12-27 10:56:37,151][20291] KL-divergence is very high: 119.3079 [2023-12-27 10:56:37,200][20347] Updated weights for policy 0, policy_version 768155 (0.0008) [2023-12-27 10:56:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 393248768. Throughput: 0: 10390.9, 1: 10275.3. Samples: 393212160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:37,363][19170] Avg episode reward: [(0, '931.700'), (1, '689.610')] [2023-12-27 10:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000767768_196575232.pth... [2023-12-27 10:56:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000768160_196673536.pth... [2023-12-27 10:56:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000766584_196272128.pth [2023-12-27 10:56:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000766912_196354048.pth [2023-12-27 10:56:37,698][20381] Updated weights for policy 1, policy_version 767769 (0.0010) [2023-12-27 10:56:37,757][20381] Updated weights for policy 1, policy_version 767779 (0.0007) [2023-12-27 10:56:37,817][20381] Updated weights for policy 1, policy_version 767789 (0.0010) [2023-12-27 10:56:37,872][20381] Updated weights for policy 1, policy_version 767799 (0.0010) [2023-12-27 10:56:37,883][20347] Updated weights for policy 0, policy_version 768165 (0.0010) [2023-12-27 10:56:37,945][20347] Updated weights for policy 0, policy_version 768175 (0.0005) [2023-12-27 10:56:38,004][20347] Updated weights for policy 0, policy_version 768185 (0.0006) [2023-12-27 10:56:38,493][20381] Updated weights for policy 1, policy_version 767809 (0.0006) [2023-12-27 10:56:38,542][20381] Updated weights for policy 1, policy_version 767819 (0.0006) [2023-12-27 10:56:38,568][20347] Updated weights for policy 0, policy_version 768195 (0.0007) [2023-12-27 10:56:38,592][20381] Updated weights for policy 1, policy_version 767829 (0.0010) [2023-12-27 10:56:38,627][20347] Updated weights for policy 0, policy_version 768205 (0.0006) [2023-12-27 10:56:38,681][20347] Updated weights for policy 0, policy_version 768215 (0.0006) [2023-12-27 10:56:39,208][20381] Updated weights for policy 1, policy_version 767839 (0.0010) [2023-12-27 10:56:39,276][20381] Updated weights for policy 1, policy_version 767849 (0.0008) [2023-12-27 10:56:39,314][20347] Updated weights for policy 0, policy_version 768225 (0.0006) [2023-12-27 10:56:39,345][20381] Updated weights for policy 1, policy_version 767859 (0.0008) [2023-12-27 10:56:39,383][20347] Updated weights for policy 0, policy_version 768235 (0.0009) [2023-12-27 10:56:39,446][20347] Updated weights for policy 0, policy_version 768245 (0.0008) [2023-12-27 10:56:39,513][20347] Updated weights for policy 0, policy_version 768255 (0.0009) [2023-12-27 10:56:40,038][20381] Updated weights for policy 1, policy_version 767869 (0.0008) [2023-12-27 10:56:40,095][20381] Updated weights for policy 1, policy_version 767879 (0.0008) [2023-12-27 10:56:40,155][20381] Updated weights for policy 1, policy_version 767889 (0.0007) [2023-12-27 10:56:40,240][20347] Updated weights for policy 0, policy_version 768265 (0.0010) [2023-12-27 10:56:40,299][20347] Updated weights for policy 0, policy_version 768275 (0.0010) [2023-12-27 10:56:40,360][20347] Updated weights for policy 0, policy_version 768285 (0.0009) [2023-12-27 10:56:40,883][20381] Updated weights for policy 1, policy_version 767899 (0.0008) [2023-12-27 10:56:40,934][20381] Updated weights for policy 1, policy_version 767909 (0.0009) [2023-12-27 10:56:40,967][20347] Updated weights for policy 0, policy_version 768295 (0.0007) [2023-12-27 10:56:40,987][20381] Updated weights for policy 1, policy_version 767919 (0.0010) [2023-12-27 10:56:41,025][20347] Updated weights for policy 0, policy_version 768305 (0.0006) [2023-12-27 10:56:41,090][20347] Updated weights for policy 0, policy_version 768315 (0.0007) [2023-12-27 10:56:41,683][20381] Updated weights for policy 1, policy_version 767929 (0.0010) [2023-12-27 10:56:41,750][20381] Updated weights for policy 1, policy_version 767939 (0.0008) [2023-12-27 10:56:41,809][20381] Updated weights for policy 1, policy_version 767949 (0.0008) [2023-12-27 10:56:41,836][20347] Updated weights for policy 0, policy_version 768325 (0.0007) [2023-12-27 10:56:41,868][20381] Updated weights for policy 1, policy_version 767959 (0.0008) [2023-12-27 10:56:41,892][20347] Updated weights for policy 0, policy_version 768335 (0.0007) [2023-12-27 10:56:41,945][20347] Updated weights for policy 0, policy_version 768345 (0.0006) [2023-12-27 10:56:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 393347072. Throughput: 0: 10433.7, 1: 10312.3. Samples: 393338392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:42,363][19170] Avg episode reward: [(0, '941.520'), (1, '737.440')] [2023-12-27 10:56:42,571][20347] Updated weights for policy 0, policy_version 768355 (0.0005) [2023-12-27 10:56:42,602][20381] Updated weights for policy 1, policy_version 767969 (0.0007) [2023-12-27 10:56:42,628][20347] Updated weights for policy 0, policy_version 768365 (0.0006) [2023-12-27 10:56:42,662][20381] Updated weights for policy 1, policy_version 767979 (0.0006) [2023-12-27 10:56:42,686][20347] Updated weights for policy 0, policy_version 768375 (0.0006) [2023-12-27 10:56:42,721][20381] Updated weights for policy 1, policy_version 767989 (0.0007) [2023-12-27 10:56:43,251][20347] Updated weights for policy 0, policy_version 768385 (0.0006) [2023-12-27 10:56:43,281][20381] Updated weights for policy 1, policy_version 767999 (0.0007) [2023-12-27 10:56:43,300][20347] Updated weights for policy 0, policy_version 768395 (0.0005) [2023-12-27 10:56:43,345][20381] Updated weights for policy 1, policy_version 768009 (0.0010) [2023-12-27 10:56:43,348][20347] Updated weights for policy 0, policy_version 768405 (0.0005) [2023-12-27 10:56:43,399][20381] Updated weights for policy 1, policy_version 768019 (0.0011) [2023-12-27 10:56:43,400][20347] Updated weights for policy 0, policy_version 768415 (0.0006) [2023-12-27 10:56:44,011][20347] Updated weights for policy 0, policy_version 768425 (0.0006) [2023-12-27 10:56:44,071][20347] Updated weights for policy 0, policy_version 768435 (0.0006) [2023-12-27 10:56:44,076][20381] Updated weights for policy 1, policy_version 768029 (0.0009) [2023-12-27 10:56:44,130][20347] Updated weights for policy 0, policy_version 768445 (0.0007) [2023-12-27 10:56:44,133][20381] Updated weights for policy 1, policy_version 768039 (0.0008) [2023-12-27 10:56:44,189][20381] Updated weights for policy 1, policy_version 768049 (0.0008) [2023-12-27 10:56:44,803][20347] Updated weights for policy 0, policy_version 768455 (0.0009) [2023-12-27 10:56:44,862][20347] Updated weights for policy 0, policy_version 768465 (0.0009) [2023-12-27 10:56:44,916][20381] Updated weights for policy 1, policy_version 768059 (0.0009) [2023-12-27 10:56:44,923][20347] Updated weights for policy 0, policy_version 768475 (0.0005) [2023-12-27 10:56:44,976][20381] Updated weights for policy 1, policy_version 768069 (0.0010) [2023-12-27 10:56:45,039][20381] Updated weights for policy 1, policy_version 768079 (0.0011) [2023-12-27 10:56:45,545][20347] Updated weights for policy 0, policy_version 768485 (0.0007) [2023-12-27 10:56:45,606][20347] Updated weights for policy 0, policy_version 768495 (0.0006) [2023-12-27 10:56:45,659][20347] Updated weights for policy 0, policy_version 768505 (0.0010) [2023-12-27 10:56:45,726][20381] Updated weights for policy 1, policy_version 768089 (0.0011) [2023-12-27 10:56:45,778][20381] Updated weights for policy 1, policy_version 768099 (0.0010) [2023-12-27 10:56:45,828][20381] Updated weights for policy 1, policy_version 768109 (0.0010) [2023-12-27 10:56:45,876][20381] Updated weights for policy 1, policy_version 768119 (0.0010) [2023-12-27 10:56:46,312][20347] Updated weights for policy 0, policy_version 768515 (0.0010) [2023-12-27 10:56:46,366][20347] Updated weights for policy 0, policy_version 768525 (0.0010) [2023-12-27 10:56:46,414][20347] Updated weights for policy 0, policy_version 768535 (0.0010) [2023-12-27 10:56:46,526][20381] Updated weights for policy 1, policy_version 768129 (0.0008) [2023-12-27 10:56:46,573][20381] Updated weights for policy 1, policy_version 768139 (0.0009) [2023-12-27 10:56:46,621][20381] Updated weights for policy 1, policy_version 768149 (0.0007) [2023-12-27 10:56:47,102][20347] Updated weights for policy 0, policy_version 768545 (0.0010) [2023-12-27 10:56:47,162][20347] Updated weights for policy 0, policy_version 768555 (0.0008) [2023-12-27 10:56:47,218][20347] Updated weights for policy 0, policy_version 768565 (0.0007) [2023-12-27 10:56:47,273][20381] Updated weights for policy 1, policy_version 768159 (0.0011) [2023-12-27 10:56:47,280][20347] Updated weights for policy 0, policy_version 768575 (0.0008) [2023-12-27 10:56:47,322][20381] Updated weights for policy 1, policy_version 768169 (0.0010) [2023-12-27 10:56:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 393453568. Throughput: 0: 10496.6, 1: 10279.0. Samples: 393464208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:47,363][19170] Avg episode reward: [(0, '951.390'), (1, '739.940')] [2023-12-27 10:56:47,374][20381] Updated weights for policy 1, policy_version 768179 (0.0010) [2023-12-27 10:56:47,951][20347] Updated weights for policy 0, policy_version 768585 (0.0010) [2023-12-27 10:56:48,008][20347] Updated weights for policy 0, policy_version 768595 (0.0010) [2023-12-27 10:56:48,062][20347] Updated weights for policy 0, policy_version 768605 (0.0010) [2023-12-27 10:56:48,077][20381] Updated weights for policy 1, policy_version 768189 (0.0008) [2023-12-27 10:56:48,143][20381] Updated weights for policy 1, policy_version 768199 (0.0005) [2023-12-27 10:56:48,210][20381] Updated weights for policy 1, policy_version 768209 (0.0006) [2023-12-27 10:56:48,713][20347] Updated weights for policy 0, policy_version 768615 (0.0010) [2023-12-27 10:56:48,764][20347] Updated weights for policy 0, policy_version 768625 (0.0010) [2023-12-27 10:56:48,815][20381] Updated weights for policy 1, policy_version 768219 (0.0009) [2023-12-27 10:56:48,823][20347] Updated weights for policy 0, policy_version 768635 (0.0011) [2023-12-27 10:56:48,872][20381] Updated weights for policy 1, policy_version 768229 (0.0010) [2023-12-27 10:56:48,927][20381] Updated weights for policy 1, policy_version 768239 (0.0010) [2023-12-27 10:56:49,599][20347] Updated weights for policy 0, policy_version 768645 (0.0008) [2023-12-27 10:56:49,631][20381] Updated weights for policy 1, policy_version 768249 (0.0010) [2023-12-27 10:56:49,666][20347] Updated weights for policy 0, policy_version 768655 (0.0008) [2023-12-27 10:56:49,695][20381] Updated weights for policy 1, policy_version 768259 (0.0006) [2023-12-27 10:56:49,730][20347] Updated weights for policy 0, policy_version 768665 (0.0006) [2023-12-27 10:56:49,760][20381] Updated weights for policy 1, policy_version 768269 (0.0007) [2023-12-27 10:56:49,825][20381] Updated weights for policy 1, policy_version 768279 (0.0007) [2023-12-27 10:56:50,407][20347] Updated weights for policy 0, policy_version 768675 (0.0008) [2023-12-27 10:56:50,456][20347] Updated weights for policy 0, policy_version 768685 (0.0007) [2023-12-27 10:56:50,503][20347] Updated weights for policy 0, policy_version 768695 (0.0007) [2023-12-27 10:56:50,514][20381] Updated weights for policy 1, policy_version 768289 (0.0011) [2023-12-27 10:56:50,572][20381] Updated weights for policy 1, policy_version 768299 (0.0010) [2023-12-27 10:56:50,634][20381] Updated weights for policy 1, policy_version 768309 (0.0010) [2023-12-27 10:56:51,160][20347] Updated weights for policy 0, policy_version 768705 (0.0007) [2023-12-27 10:56:51,222][20347] Updated weights for policy 0, policy_version 768715 (0.0008) [2023-12-27 10:56:51,282][20347] Updated weights for policy 0, policy_version 768725 (0.0008) [2023-12-27 10:56:51,346][20347] Updated weights for policy 0, policy_version 768735 (0.0008) [2023-12-27 10:56:51,352][20381] Updated weights for policy 1, policy_version 768319 (0.0008) [2023-12-27 10:56:51,413][20381] Updated weights for policy 1, policy_version 768329 (0.0008) [2023-12-27 10:56:51,470][20381] Updated weights for policy 1, policy_version 768339 (0.0008) [2023-12-27 10:56:52,112][20347] Updated weights for policy 0, policy_version 768745 (0.0008) [2023-12-27 10:56:52,169][20347] Updated weights for policy 0, policy_version 768755 (0.0007) [2023-12-27 10:56:52,184][20381] Updated weights for policy 1, policy_version 768349 (0.0007) [2023-12-27 10:56:52,220][20347] Updated weights for policy 0, policy_version 768765 (0.0006) [2023-12-27 10:56:52,239][20381] Updated weights for policy 1, policy_version 768359 (0.0006) [2023-12-27 10:56:52,305][20381] Updated weights for policy 1, policy_version 768369 (0.0006) [2023-12-27 10:56:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 393560064. Throughput: 0: 10478.0, 1: 10275.9. Samples: 393524652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:52,363][19170] Avg episode reward: [(0, '931.860'), (1, '748.400')] [2023-12-27 10:56:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000768376_196730880.pth... [2023-12-27 10:56:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000768768_196829184.pth... [2023-12-27 10:56:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000767160_196419584.pth [2023-12-27 10:56:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000767520_196509696.pth [2023-12-27 10:56:52,913][20381] Updated weights for policy 1, policy_version 768379 (0.0007) [2023-12-27 10:56:52,926][20347] Updated weights for policy 0, policy_version 768775 (0.0006) [2023-12-27 10:56:52,964][20381] Updated weights for policy 1, policy_version 768389 (0.0007) [2023-12-27 10:56:52,977][20347] Updated weights for policy 0, policy_version 768785 (0.0006) [2023-12-27 10:56:53,020][20381] Updated weights for policy 1, policy_version 768399 (0.0006) [2023-12-27 10:56:53,029][20347] Updated weights for policy 0, policy_version 768795 (0.0006) [2023-12-27 10:56:53,683][20347] Updated weights for policy 0, policy_version 768805 (0.0007) [2023-12-27 10:56:53,690][20381] Updated weights for policy 1, policy_version 768409 (0.0007) [2023-12-27 10:56:53,736][20347] Updated weights for policy 0, policy_version 768815 (0.0007) [2023-12-27 10:56:53,738][20381] Updated weights for policy 1, policy_version 768419 (0.0005) [2023-12-27 10:56:53,790][20381] Updated weights for policy 1, policy_version 768429 (0.0005) [2023-12-27 10:56:53,794][20347] Updated weights for policy 0, policy_version 768825 (0.0006) [2023-12-27 10:56:53,836][20381] Updated weights for policy 1, policy_version 768439 (0.0009) [2023-12-27 10:56:54,358][20347] Updated weights for policy 0, policy_version 768835 (0.0010) [2023-12-27 10:56:54,419][20347] Updated weights for policy 0, policy_version 768845 (0.0007) [2023-12-27 10:56:54,480][20347] Updated weights for policy 0, policy_version 768855 (0.0005) [2023-12-27 10:56:54,583][20381] Updated weights for policy 1, policy_version 768449 (0.0010) [2023-12-27 10:56:54,632][20381] Updated weights for policy 1, policy_version 768459 (0.0010) [2023-12-27 10:56:54,688][20381] Updated weights for policy 1, policy_version 768469 (0.0008) [2023-12-27 10:56:55,074][20347] Updated weights for policy 0, policy_version 768865 (0.0009) [2023-12-27 10:56:55,137][20347] Updated weights for policy 0, policy_version 768875 (0.0007) [2023-12-27 10:56:55,189][20347] Updated weights for policy 0, policy_version 768885 (0.0010) [2023-12-27 10:56:55,241][20347] Updated weights for policy 0, policy_version 768895 (0.0010) [2023-12-27 10:56:55,348][20381] Updated weights for policy 1, policy_version 768479 (0.0006) [2023-12-27 10:56:55,403][20381] Updated weights for policy 1, policy_version 768489 (0.0006) [2023-12-27 10:56:55,459][20381] Updated weights for policy 1, policy_version 768499 (0.0007) [2023-12-27 10:56:55,911][20347] Updated weights for policy 0, policy_version 768905 (0.0007) [2023-12-27 10:56:55,969][20347] Updated weights for policy 0, policy_version 768915 (0.0006) [2023-12-27 10:56:56,017][20347] Updated weights for policy 0, policy_version 768925 (0.0007) [2023-12-27 10:56:56,081][20381] Updated weights for policy 1, policy_version 768509 (0.0007) [2023-12-27 10:56:56,140][20381] Updated weights for policy 1, policy_version 768519 (0.0008) [2023-12-27 10:56:56,193][20381] Updated weights for policy 1, policy_version 768529 (0.0008) [2023-12-27 10:56:56,645][20347] Updated weights for policy 0, policy_version 768935 (0.0006) [2023-12-27 10:56:56,696][20347] Updated weights for policy 0, policy_version 768945 (0.0005) [2023-12-27 10:56:56,740][20347] Updated weights for policy 0, policy_version 768955 (0.0005) [2023-12-27 10:56:56,850][20381] Updated weights for policy 1, policy_version 768539 (0.0008) [2023-12-27 10:56:56,905][20381] Updated weights for policy 1, policy_version 768549 (0.0007) [2023-12-27 10:56:56,971][20381] Updated weights for policy 1, policy_version 768559 (0.0006) [2023-12-27 10:56:57,314][20347] Updated weights for policy 0, policy_version 768965 (0.0007) [2023-12-27 10:56:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 393658368. Throughput: 0: 10479.5, 1: 10263.4. Samples: 393648620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:56:57,363][19170] Avg episode reward: [(0, '912.120'), (1, '756.640')] [2023-12-27 10:56:57,366][20347] Updated weights for policy 0, policy_version 768975 (0.0008) [2023-12-27 10:56:57,416][20347] Updated weights for policy 0, policy_version 768985 (0.0007) [2023-12-27 10:56:57,583][20381] Updated weights for policy 1, policy_version 768569 (0.0006) [2023-12-27 10:56:57,635][20381] Updated weights for policy 1, policy_version 768579 (0.0008) [2023-12-27 10:56:57,687][20381] Updated weights for policy 1, policy_version 768589 (0.0007) [2023-12-27 10:56:57,741][20381] Updated weights for policy 1, policy_version 768599 (0.0008) [2023-12-27 10:56:58,000][20347] Updated weights for policy 0, policy_version 768995 (0.0007) [2023-12-27 10:56:58,045][20347] Updated weights for policy 0, policy_version 769005 (0.0008) [2023-12-27 10:56:58,093][20347] Updated weights for policy 0, policy_version 769015 (0.0007) [2023-12-27 10:56:58,480][20381] Updated weights for policy 1, policy_version 768609 (0.0008) [2023-12-27 10:56:58,543][20381] Updated weights for policy 1, policy_version 768619 (0.0008) [2023-12-27 10:56:58,609][20381] Updated weights for policy 1, policy_version 768630 (0.0007) [2023-12-27 10:56:58,791][20347] Updated weights for policy 0, policy_version 769025 (0.0007) [2023-12-27 10:56:58,854][20347] Updated weights for policy 0, policy_version 769035 (0.0008) [2023-12-27 10:56:58,920][20347] Updated weights for policy 0, policy_version 769045 (0.0009) [2023-12-27 10:56:58,984][20347] Updated weights for policy 0, policy_version 769055 (0.0009) [2023-12-27 10:56:59,395][20381] Updated weights for policy 1, policy_version 768640 (0.0007) [2023-12-27 10:56:59,453][20381] Updated weights for policy 1, policy_version 768650 (0.0007) [2023-12-27 10:56:59,502][20381] Updated weights for policy 1, policy_version 768660 (0.0008) [2023-12-27 10:56:59,716][20347] Updated weights for policy 0, policy_version 769065 (0.0008) [2023-12-27 10:56:59,771][20347] Updated weights for policy 0, policy_version 769075 (0.0008) [2023-12-27 10:56:59,839][20347] Updated weights for policy 0, policy_version 769085 (0.0008) [2023-12-27 10:57:00,243][20381] Updated weights for policy 1, policy_version 768670 (0.0008) [2023-12-27 10:57:00,299][20381] Updated weights for policy 1, policy_version 768680 (0.0006) [2023-12-27 10:57:00,361][20381] Updated weights for policy 1, policy_version 768690 (0.0005) [2023-12-27 10:57:00,517][20347] Updated weights for policy 0, policy_version 769095 (0.0007) [2023-12-27 10:57:00,572][20347] Updated weights for policy 0, policy_version 769105 (0.0008) [2023-12-27 10:57:00,631][20347] Updated weights for policy 0, policy_version 769115 (0.0007) [2023-12-27 10:57:00,945][20381] Updated weights for policy 1, policy_version 768700 (0.0006) [2023-12-27 10:57:01,008][20381] Updated weights for policy 1, policy_version 768710 (0.0006) [2023-12-27 10:57:01,072][20381] Updated weights for policy 1, policy_version 768720 (0.0008) [2023-12-27 10:57:01,379][20347] Updated weights for policy 0, policy_version 769125 (0.0007) [2023-12-27 10:57:01,429][20347] Updated weights for policy 0, policy_version 769135 (0.0005) [2023-12-27 10:57:01,484][20347] Updated weights for policy 0, policy_version 769145 (0.0007) [2023-12-27 10:57:01,783][20381] Updated weights for policy 1, policy_version 768730 (0.0008) [2023-12-27 10:57:01,837][20381] Updated weights for policy 1, policy_version 768740 (0.0008) [2023-12-27 10:57:01,888][20381] Updated weights for policy 1, policy_version 768750 (0.0008) [2023-12-27 10:57:01,942][20381] Updated weights for policy 1, policy_version 768760 (0.0008) [2023-12-27 10:57:02,117][20347] Updated weights for policy 0, policy_version 769155 (0.0007) [2023-12-27 10:57:02,177][20347] Updated weights for policy 0, policy_version 769165 (0.0005) [2023-12-27 10:57:02,233][20347] Updated weights for policy 0, policy_version 769175 (0.0005) [2023-12-27 10:57:02,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 393764864. Throughput: 0: 10513.6, 1: 10271.0. Samples: 393771772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:02,363][19170] Avg episode reward: [(0, '921.930'), (1, '795.790')] [2023-12-27 10:57:02,694][20381] Updated weights for policy 1, policy_version 768770 (0.0008) [2023-12-27 10:57:02,740][20381] Updated weights for policy 1, policy_version 768780 (0.0008) [2023-12-27 10:57:02,795][20381] Updated weights for policy 1, policy_version 768790 (0.0008) [2023-12-27 10:57:02,837][20347] Updated weights for policy 0, policy_version 769185 (0.0007) [2023-12-27 10:57:02,889][20347] Updated weights for policy 0, policy_version 769195 (0.0008) [2023-12-27 10:57:02,944][20347] Updated weights for policy 0, policy_version 769205 (0.0008) [2023-12-27 10:57:02,994][20347] Updated weights for policy 0, policy_version 769215 (0.0007) [2023-12-27 10:57:03,522][20381] Updated weights for policy 1, policy_version 768800 (0.0007) [2023-12-27 10:57:03,583][20381] Updated weights for policy 1, policy_version 768810 (0.0008) [2023-12-27 10:57:03,643][20381] Updated weights for policy 1, policy_version 768820 (0.0008) [2023-12-27 10:57:03,664][20347] Updated weights for policy 0, policy_version 769225 (0.0006) [2023-12-27 10:57:03,711][20347] Updated weights for policy 0, policy_version 769235 (0.0007) [2023-12-27 10:57:03,761][20347] Updated weights for policy 0, policy_version 769245 (0.0008) [2023-12-27 10:57:04,288][20381] Updated weights for policy 1, policy_version 768830 (0.0005) [2023-12-27 10:57:04,342][20381] Updated weights for policy 1, policy_version 768840 (0.0006) [2023-12-27 10:57:04,413][20381] Updated weights for policy 1, policy_version 768850 (0.0008) [2023-12-27 10:57:04,478][20347] Updated weights for policy 0, policy_version 769255 (0.0007) [2023-12-27 10:57:04,536][20347] Updated weights for policy 0, policy_version 769265 (0.0008) [2023-12-27 10:57:04,603][20347] Updated weights for policy 0, policy_version 769275 (0.0008) [2023-12-27 10:57:04,993][20381] Updated weights for policy 1, policy_version 768860 (0.0007) [2023-12-27 10:57:05,048][20381] Updated weights for policy 1, policy_version 768870 (0.0008) [2023-12-27 10:57:05,109][20381] Updated weights for policy 1, policy_version 768880 (0.0006) [2023-12-27 10:57:05,288][20347] Updated weights for policy 0, policy_version 769285 (0.0007) [2023-12-27 10:57:05,340][20347] Updated weights for policy 0, policy_version 769295 (0.0007) [2023-12-27 10:57:05,389][20347] Updated weights for policy 0, policy_version 769305 (0.0008) [2023-12-27 10:57:05,742][20381] Updated weights for policy 1, policy_version 768890 (0.0006) [2023-12-27 10:57:05,799][20381] Updated weights for policy 1, policy_version 768900 (0.0007) [2023-12-27 10:57:05,852][20381] Updated weights for policy 1, policy_version 768910 (0.0006) [2023-12-27 10:57:05,901][20381] Updated weights for policy 1, policy_version 768920 (0.0005) [2023-12-27 10:57:06,107][20347] Updated weights for policy 0, policy_version 769315 (0.0008) [2023-12-27 10:57:06,166][20347] Updated weights for policy 0, policy_version 769325 (0.0010) [2023-12-27 10:57:06,222][20347] Updated weights for policy 0, policy_version 769335 (0.0009) [2023-12-27 10:57:06,560][20381] Updated weights for policy 1, policy_version 768930 (0.0008) [2023-12-27 10:57:06,619][20381] Updated weights for policy 1, policy_version 768940 (0.0008) [2023-12-27 10:57:06,676][20381] Updated weights for policy 1, policy_version 768950 (0.0008) [2023-12-27 10:57:06,882][20347] Updated weights for policy 0, policy_version 769345 (0.0006) [2023-12-27 10:57:06,938][20347] Updated weights for policy 0, policy_version 769355 (0.0008) [2023-12-27 10:57:06,997][20347] Updated weights for policy 0, policy_version 769365 (0.0008) [2023-12-27 10:57:07,041][20291] KL-divergence is very high: 107.8210 [2023-12-27 10:57:07,052][20291] KL-divergence is very high: 107.7872 [2023-12-27 10:57:07,059][20347] Updated weights for policy 0, policy_version 769375 (0.0008) [2023-12-27 10:57:07,345][20381] Updated weights for policy 1, policy_version 768960 (0.0007) [2023-12-27 10:57:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 393863168. Throughput: 0: 10496.9, 1: 10318.9. Samples: 393833932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:07,363][19170] Avg episode reward: [(0, '922.180'), (1, '864.820')] [2023-12-27 10:57:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000769376_196984832.pth... [2023-12-27 10:57:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000768160_196673536.pth [2023-12-27 10:57:07,395][20381] Updated weights for policy 1, policy_version 768970 (0.0007) [2023-12-27 10:57:07,453][20381] Updated weights for policy 1, policy_version 768980 (0.0006) [2023-12-27 10:57:07,473][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000768984_196886528.pth... [2023-12-27 10:57:07,476][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000767768_196575232.pth [2023-12-27 10:57:07,735][20347] Updated weights for policy 0, policy_version 769385 (0.0005) [2023-12-27 10:57:07,783][20347] Updated weights for policy 0, policy_version 769395 (0.0005) [2023-12-27 10:57:07,829][20347] Updated weights for policy 0, policy_version 769405 (0.0005) [2023-12-27 10:57:08,108][20381] Updated weights for policy 1, policy_version 768990 (0.0008) [2023-12-27 10:57:08,168][20381] Updated weights for policy 1, policy_version 769000 (0.0007) [2023-12-27 10:57:08,189][20292] KL-divergence is very high: 123.2121 [2023-12-27 10:57:08,222][20381] Updated weights for policy 1, policy_version 769011 (0.0008) [2023-12-27 10:57:08,377][20347] Updated weights for policy 0, policy_version 769415 (0.0006) [2023-12-27 10:57:08,434][20347] Updated weights for policy 0, policy_version 769425 (0.0006) [2023-12-27 10:57:08,493][20347] Updated weights for policy 0, policy_version 769435 (0.0007) [2023-12-27 10:57:08,921][20381] Updated weights for policy 1, policy_version 769021 (0.0009) [2023-12-27 10:57:08,973][20381] Updated weights for policy 1, policy_version 769031 (0.0010) [2023-12-27 10:57:09,021][20381] Updated weights for policy 1, policy_version 769041 (0.0009) [2023-12-27 10:57:09,153][20347] Updated weights for policy 0, policy_version 769445 (0.0011) [2023-12-27 10:57:09,209][20347] Updated weights for policy 0, policy_version 769455 (0.0009) [2023-12-27 10:57:09,266][20347] Updated weights for policy 0, policy_version 769465 (0.0007) [2023-12-27 10:57:09,733][20381] Updated weights for policy 1, policy_version 769051 (0.0009) [2023-12-27 10:57:09,795][20381] Updated weights for policy 1, policy_version 769061 (0.0008) [2023-12-27 10:57:09,863][20381] Updated weights for policy 1, policy_version 769072 (0.0009) [2023-12-27 10:57:09,979][20347] Updated weights for policy 0, policy_version 769475 (0.0007) [2023-12-27 10:57:10,033][20347] Updated weights for policy 0, policy_version 769485 (0.0009) [2023-12-27 10:57:10,082][20347] Updated weights for policy 0, policy_version 769495 (0.0009) [2023-12-27 10:57:10,616][20381] Updated weights for policy 1, policy_version 769082 (0.0011) [2023-12-27 10:57:10,666][20381] Updated weights for policy 1, policy_version 769092 (0.0010) [2023-12-27 10:57:10,713][20381] Updated weights for policy 1, policy_version 769102 (0.0010) [2023-12-27 10:57:10,770][20381] Updated weights for policy 1, policy_version 769112 (0.0005) [2023-12-27 10:57:10,810][20347] Updated weights for policy 0, policy_version 769505 (0.0007) [2023-12-27 10:57:10,858][20347] Updated weights for policy 0, policy_version 769515 (0.0006) [2023-12-27 10:57:10,912][20347] Updated weights for policy 0, policy_version 769525 (0.0008) [2023-12-27 10:57:10,967][20347] Updated weights for policy 0, policy_version 769535 (0.0006) [2023-12-27 10:57:11,469][20381] Updated weights for policy 1, policy_version 769122 (0.0008) [2023-12-27 10:57:11,528][20381] Updated weights for policy 1, policy_version 769132 (0.0008) [2023-12-27 10:57:11,589][20381] Updated weights for policy 1, policy_version 769142 (0.0007) [2023-12-27 10:57:11,710][20347] Updated weights for policy 0, policy_version 769545 (0.0006) [2023-12-27 10:57:11,755][20291] KL-divergence is very high: 119.1543 [2023-12-27 10:57:11,774][20347] Updated weights for policy 0, policy_version 769555 (0.0008) [2023-12-27 10:57:11,831][20347] Updated weights for policy 0, policy_version 769565 (0.0007) [2023-12-27 10:57:12,342][20381] Updated weights for policy 1, policy_version 769152 (0.0009) [2023-12-27 10:57:12,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 393961472. Throughput: 0: 10463.9, 1: 10342.0. Samples: 393957016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:12,363][19170] Avg episode reward: [(0, '931.950'), (1, '815.910')] [2023-12-27 10:57:12,401][20381] Updated weights for policy 1, policy_version 769162 (0.0008) [2023-12-27 10:57:12,452][20381] Updated weights for policy 1, policy_version 769172 (0.0007) [2023-12-27 10:57:12,496][20347] Updated weights for policy 0, policy_version 769575 (0.0007) [2023-12-27 10:57:12,546][20347] Updated weights for policy 0, policy_version 769585 (0.0006) [2023-12-27 10:57:12,595][20347] Updated weights for policy 0, policy_version 769595 (0.0008) [2023-12-27 10:57:13,191][20347] Updated weights for policy 0, policy_version 769605 (0.0006) [2023-12-27 10:57:13,194][20381] Updated weights for policy 1, policy_version 769182 (0.0009) [2023-12-27 10:57:13,244][20381] Updated weights for policy 1, policy_version 769192 (0.0011) [2023-12-27 10:57:13,249][20347] Updated weights for policy 0, policy_version 769615 (0.0007) [2023-12-27 10:57:13,290][20381] Updated weights for policy 1, policy_version 769202 (0.0010) [2023-12-27 10:57:13,299][20347] Updated weights for policy 0, policy_version 769625 (0.0006) [2023-12-27 10:57:13,942][20381] Updated weights for policy 1, policy_version 769212 (0.0009) [2023-12-27 10:57:13,986][20347] Updated weights for policy 0, policy_version 769635 (0.0007) [2023-12-27 10:57:13,992][20381] Updated weights for policy 1, policy_version 769222 (0.0007) [2023-12-27 10:57:14,038][20347] Updated weights for policy 0, policy_version 769645 (0.0006) [2023-12-27 10:57:14,042][20381] Updated weights for policy 1, policy_version 769232 (0.0005) [2023-12-27 10:57:14,098][20347] Updated weights for policy 0, policy_version 769655 (0.0008) [2023-12-27 10:57:14,697][20381] Updated weights for policy 1, policy_version 769242 (0.0005) [2023-12-27 10:57:14,748][20381] Updated weights for policy 1, policy_version 769252 (0.0008) [2023-12-27 10:57:14,795][20347] Updated weights for policy 0, policy_version 769665 (0.0009) [2023-12-27 10:57:14,810][20381] Updated weights for policy 1, policy_version 769262 (0.0008) [2023-12-27 10:57:14,832][20291] KL-divergence is very high: 334.0708 [2023-12-27 10:57:14,853][20291] KL-divergence is very high: 492.0175 [2023-12-27 10:57:14,854][20347] Updated weights for policy 0, policy_version 769675 (0.0005) [2023-12-27 10:57:14,866][20291] KL-divergence is very high: 103.5983 [2023-12-27 10:57:14,867][20381] Updated weights for policy 1, policy_version 769272 (0.0006) [2023-12-27 10:57:14,877][20291] KL-divergence is very high: 187.1755 [2023-12-27 10:57:14,904][20291] KL-divergence is very high: 136.1874 [2023-12-27 10:57:14,910][20347] Updated weights for policy 0, policy_version 769685 (0.0007) [2023-12-27 10:57:14,970][20347] Updated weights for policy 0, policy_version 769695 (0.0007) [2023-12-27 10:57:15,502][20381] Updated weights for policy 1, policy_version 769282 (0.0007) [2023-12-27 10:57:15,554][20381] Updated weights for policy 1, policy_version 769292 (0.0006) [2023-12-27 10:57:15,605][20381] Updated weights for policy 1, policy_version 769302 (0.0005) [2023-12-27 10:57:15,708][20347] Updated weights for policy 0, policy_version 769705 (0.0008) [2023-12-27 10:57:15,761][20347] Updated weights for policy 0, policy_version 769715 (0.0006) [2023-12-27 10:57:15,808][20347] Updated weights for policy 0, policy_version 769725 (0.0008) [2023-12-27 10:57:16,228][20381] Updated weights for policy 1, policy_version 769312 (0.0007) [2023-12-27 10:57:16,275][20381] Updated weights for policy 1, policy_version 769322 (0.0005) [2023-12-27 10:57:16,320][20381] Updated weights for policy 1, policy_version 769332 (0.0005) [2023-12-27 10:57:16,502][20347] Updated weights for policy 0, policy_version 769735 (0.0006) [2023-12-27 10:57:16,554][20347] Updated weights for policy 0, policy_version 769745 (0.0005) [2023-12-27 10:57:16,608][20347] Updated weights for policy 0, policy_version 769755 (0.0005) [2023-12-27 10:57:16,993][20381] Updated weights for policy 1, policy_version 769342 (0.0005) [2023-12-27 10:57:17,048][20381] Updated weights for policy 1, policy_version 769352 (0.0010) [2023-12-27 10:57:17,106][20381] Updated weights for policy 1, policy_version 769362 (0.0010) [2023-12-27 10:57:17,233][20347] Updated weights for policy 0, policy_version 769765 (0.0006) [2023-12-27 10:57:17,284][20347] Updated weights for policy 0, policy_version 769775 (0.0007) [2023-12-27 10:57:17,333][20347] Updated weights for policy 0, policy_version 769785 (0.0007) [2023-12-27 10:57:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 394067968. Throughput: 0: 10420.3, 1: 10286.7. Samples: 394079712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:17,363][19170] Avg episode reward: [(0, '921.850'), (1, '854.490')] [2023-12-27 10:57:17,739][20381] Updated weights for policy 1, policy_version 769372 (0.0010) [2023-12-27 10:57:17,797][20381] Updated weights for policy 1, policy_version 769382 (0.0010) [2023-12-27 10:57:17,855][20381] Updated weights for policy 1, policy_version 769392 (0.0010) [2023-12-27 10:57:18,043][20347] Updated weights for policy 0, policy_version 769795 (0.0007) [2023-12-27 10:57:18,094][20347] Updated weights for policy 0, policy_version 769805 (0.0007) [2023-12-27 10:57:18,142][20347] Updated weights for policy 0, policy_version 769815 (0.0009) [2023-12-27 10:57:18,410][20381] Updated weights for policy 1, policy_version 769402 (0.0009) [2023-12-27 10:57:18,478][20381] Updated weights for policy 1, policy_version 769412 (0.0010) [2023-12-27 10:57:18,547][20381] Updated weights for policy 1, policy_version 769422 (0.0011) [2023-12-27 10:57:18,616][20381] Updated weights for policy 1, policy_version 769432 (0.0011) [2023-12-27 10:57:18,836][20347] Updated weights for policy 0, policy_version 769825 (0.0009) [2023-12-27 10:57:18,900][20347] Updated weights for policy 0, policy_version 769835 (0.0008) [2023-12-27 10:57:18,953][20347] Updated weights for policy 0, policy_version 769845 (0.0007) [2023-12-27 10:57:19,002][20347] Updated weights for policy 0, policy_version 769855 (0.0007) [2023-12-27 10:57:19,280][20381] Updated weights for policy 1, policy_version 769442 (0.0009) [2023-12-27 10:57:19,339][20381] Updated weights for policy 1, policy_version 769452 (0.0011) [2023-12-27 10:57:19,404][20381] Updated weights for policy 1, policy_version 769462 (0.0010) [2023-12-27 10:57:19,796][20347] Updated weights for policy 0, policy_version 769865 (0.0008) [2023-12-27 10:57:19,858][20347] Updated weights for policy 0, policy_version 769875 (0.0008) [2023-12-27 10:57:19,921][20347] Updated weights for policy 0, policy_version 769885 (0.0007) [2023-12-27 10:57:20,117][20381] Updated weights for policy 1, policy_version 769472 (0.0007) [2023-12-27 10:57:20,176][20381] Updated weights for policy 1, policy_version 769482 (0.0008) [2023-12-27 10:57:20,239][20381] Updated weights for policy 1, policy_version 769492 (0.0008) [2023-12-27 10:57:20,600][20347] Updated weights for policy 0, policy_version 769895 (0.0008) [2023-12-27 10:57:20,664][20347] Updated weights for policy 0, policy_version 769905 (0.0007) [2023-12-27 10:57:20,726][20347] Updated weights for policy 0, policy_version 769915 (0.0007) [2023-12-27 10:57:20,942][20381] Updated weights for policy 1, policy_version 769502 (0.0008) [2023-12-27 10:57:21,000][20381] Updated weights for policy 1, policy_version 769512 (0.0008) [2023-12-27 10:57:21,066][20381] Updated weights for policy 1, policy_version 769522 (0.0008) [2023-12-27 10:57:21,495][20347] Updated weights for policy 0, policy_version 769925 (0.0008) [2023-12-27 10:57:21,545][20347] Updated weights for policy 0, policy_version 769935 (0.0007) [2023-12-27 10:57:21,599][20347] Updated weights for policy 0, policy_version 769945 (0.0008) [2023-12-27 10:57:21,856][20381] Updated weights for policy 1, policy_version 769532 (0.0007) [2023-12-27 10:57:21,914][20381] Updated weights for policy 1, policy_version 769542 (0.0008) [2023-12-27 10:57:21,970][20381] Updated weights for policy 1, policy_version 769552 (0.0008) [2023-12-27 10:57:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 394166272. Throughput: 0: 10339.1, 1: 10281.7. Samples: 394140096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:22,363][19170] Avg episode reward: [(0, '931.780'), (1, '874.250')] [2023-12-27 10:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000769560_197033984.pth... [2023-12-27 10:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000769952_197132288.pth... [2023-12-27 10:57:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000768768_196829184.pth [2023-12-27 10:57:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000768376_196730880.pth [2023-12-27 10:57:22,413][20347] Updated weights for policy 0, policy_version 769955 (0.0008) [2023-12-27 10:57:22,473][20347] Updated weights for policy 0, policy_version 769965 (0.0008) [2023-12-27 10:57:22,530][20347] Updated weights for policy 0, policy_version 769975 (0.0008) [2023-12-27 10:57:22,720][20381] Updated weights for policy 1, policy_version 769562 (0.0008) [2023-12-27 10:57:22,776][20381] Updated weights for policy 1, policy_version 769572 (0.0007) [2023-12-27 10:57:22,833][20381] Updated weights for policy 1, policy_version 769582 (0.0008) [2023-12-27 10:57:22,890][20381] Updated weights for policy 1, policy_version 769592 (0.0008) [2023-12-27 10:57:23,253][20347] Updated weights for policy 0, policy_version 769985 (0.0008) [2023-12-27 10:57:23,306][20347] Updated weights for policy 0, policy_version 769995 (0.0008) [2023-12-27 10:57:23,356][20347] Updated weights for policy 0, policy_version 770005 (0.0007) [2023-12-27 10:57:23,402][20347] Updated weights for policy 0, policy_version 770015 (0.0007) [2023-12-27 10:57:23,570][20381] Updated weights for policy 1, policy_version 769602 (0.0005) [2023-12-27 10:57:23,621][20381] Updated weights for policy 1, policy_version 769612 (0.0005) [2023-12-27 10:57:23,659][20292] KL-divergence is very high: 176.7010 [2023-12-27 10:57:23,682][20381] Updated weights for policy 1, policy_version 769622 (0.0005) [2023-12-27 10:57:24,086][20347] Updated weights for policy 0, policy_version 770025 (0.0006) [2023-12-27 10:57:24,144][20347] Updated weights for policy 0, policy_version 770035 (0.0005) [2023-12-27 10:57:24,200][20347] Updated weights for policy 0, policy_version 770045 (0.0008) [2023-12-27 10:57:24,215][20381] Updated weights for policy 1, policy_version 769632 (0.0010) [2023-12-27 10:57:24,267][20381] Updated weights for policy 1, policy_version 769642 (0.0010) [2023-12-27 10:57:24,316][20381] Updated weights for policy 1, policy_version 769652 (0.0010) [2023-12-27 10:57:24,744][20347] Updated weights for policy 0, policy_version 770055 (0.0007) [2023-12-27 10:57:24,795][20347] Updated weights for policy 0, policy_version 770065 (0.0005) [2023-12-27 10:57:24,850][20347] Updated weights for policy 0, policy_version 770075 (0.0007) [2023-12-27 10:57:24,944][20381] Updated weights for policy 1, policy_version 769662 (0.0007) [2023-12-27 10:57:25,007][20381] Updated weights for policy 1, policy_version 769672 (0.0005) [2023-12-27 10:57:25,065][20381] Updated weights for policy 1, policy_version 769682 (0.0009) [2023-12-27 10:57:25,404][20347] Updated weights for policy 0, policy_version 770085 (0.0008) [2023-12-27 10:57:25,470][20347] Updated weights for policy 0, policy_version 770095 (0.0009) [2023-12-27 10:57:25,533][20347] Updated weights for policy 0, policy_version 770105 (0.0005) [2023-12-27 10:57:25,694][20381] Updated weights for policy 1, policy_version 769692 (0.0010) [2023-12-27 10:57:25,745][20381] Updated weights for policy 1, policy_version 769702 (0.0010) [2023-12-27 10:57:25,800][20381] Updated weights for policy 1, policy_version 769712 (0.0009) [2023-12-27 10:57:25,812][20292] KL-divergence is very high: 239.1202 [2023-12-27 10:57:26,090][20347] Updated weights for policy 0, policy_version 770115 (0.0005) [2023-12-27 10:57:26,150][20347] Updated weights for policy 0, policy_version 770125 (0.0009) [2023-12-27 10:57:26,201][20347] Updated weights for policy 0, policy_version 770135 (0.0010) [2023-12-27 10:57:26,478][20381] Updated weights for policy 1, policy_version 769722 (0.0010) [2023-12-27 10:57:26,529][20381] Updated weights for policy 1, policy_version 769732 (0.0010) [2023-12-27 10:57:26,575][20381] Updated weights for policy 1, policy_version 769742 (0.0010) [2023-12-27 10:57:26,636][20381] Updated weights for policy 1, policy_version 769752 (0.0010) [2023-12-27 10:57:26,833][20347] Updated weights for policy 0, policy_version 770145 (0.0010) [2023-12-27 10:57:26,888][20347] Updated weights for policy 0, policy_version 770155 (0.0010) [2023-12-27 10:57:26,943][20347] Updated weights for policy 0, policy_version 770165 (0.0007) [2023-12-27 10:57:27,005][20347] Updated weights for policy 0, policy_version 770175 (0.0005) [2023-12-27 10:57:27,315][20381] Updated weights for policy 1, policy_version 769762 (0.0010) [2023-12-27 10:57:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 394272768. Throughput: 0: 10357.2, 1: 10249.9. Samples: 394265708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 10:57:27,363][19170] Avg episode reward: [(0, '941.490'), (1, '824.720')] [2023-12-27 10:57:27,371][20381] Updated weights for policy 1, policy_version 769772 (0.0010) [2023-12-27 10:57:27,422][20381] Updated weights for policy 1, policy_version 769782 (0.0010) [2023-12-27 10:57:27,597][20347] Updated weights for policy 0, policy_version 770185 (0.0010) [2023-12-27 10:57:27,658][20347] Updated weights for policy 0, policy_version 770195 (0.0010) [2023-12-27 10:57:27,718][20347] Updated weights for policy 0, policy_version 770205 (0.0010) [2023-12-27 10:57:28,126][20381] Updated weights for policy 1, policy_version 769792 (0.0010) [2023-12-27 10:57:28,180][20381] Updated weights for policy 1, policy_version 769802 (0.0010) [2023-12-27 10:57:28,237][20381] Updated weights for policy 1, policy_version 769812 (0.0010) [2023-12-27 10:57:28,371][20347] Updated weights for policy 0, policy_version 770215 (0.0008) [2023-12-27 10:57:28,430][20347] Updated weights for policy 0, policy_version 770225 (0.0007) [2023-12-27 10:57:28,483][20347] Updated weights for policy 0, policy_version 770235 (0.0007) [2023-12-27 10:57:28,912][20381] Updated weights for policy 1, policy_version 769822 (0.0009) [2023-12-27 10:57:28,965][20381] Updated weights for policy 1, policy_version 769832 (0.0010) [2023-12-27 10:57:29,020][20381] Updated weights for policy 1, policy_version 769842 (0.0006) [2023-12-27 10:57:29,144][20347] Updated weights for policy 0, policy_version 770245 (0.0008) [2023-12-27 10:57:29,193][20347] Updated weights for policy 0, policy_version 770255 (0.0009) [2023-12-27 10:57:29,247][20347] Updated weights for policy 0, policy_version 770265 (0.0007) [2023-12-27 10:57:29,612][20381] Updated weights for policy 1, policy_version 769852 (0.0007) [2023-12-27 10:57:29,664][20381] Updated weights for policy 1, policy_version 769862 (0.0010) [2023-12-27 10:57:29,718][20381] Updated weights for policy 1, policy_version 769872 (0.0011) [2023-12-27 10:57:29,912][20347] Updated weights for policy 0, policy_version 770275 (0.0007) [2023-12-27 10:57:29,968][20347] Updated weights for policy 0, policy_version 770285 (0.0007) [2023-12-27 10:57:30,027][20347] Updated weights for policy 0, policy_version 770295 (0.0010) [2023-12-27 10:57:30,379][20381] Updated weights for policy 1, policy_version 769882 (0.0010) [2023-12-27 10:57:30,438][20381] Updated weights for policy 1, policy_version 769892 (0.0011) [2023-12-27 10:57:30,492][20381] Updated weights for policy 1, policy_version 769902 (0.0010) [2023-12-27 10:57:30,550][20381] Updated weights for policy 1, policy_version 769912 (0.0007) [2023-12-27 10:57:30,721][20347] Updated weights for policy 0, policy_version 770305 (0.0007) [2023-12-27 10:57:30,769][20347] Updated weights for policy 0, policy_version 770315 (0.0005) [2023-12-27 10:57:30,824][20347] Updated weights for policy 0, policy_version 770325 (0.0005) [2023-12-27 10:57:30,882][20347] Updated weights for policy 0, policy_version 770335 (0.0005) [2023-12-27 10:57:31,178][20381] Updated weights for policy 1, policy_version 769922 (0.0011) [2023-12-27 10:57:31,230][20381] Updated weights for policy 1, policy_version 769932 (0.0010) [2023-12-27 10:57:31,295][20381] Updated weights for policy 1, policy_version 769942 (0.0009) [2023-12-27 10:57:31,484][20347] Updated weights for policy 0, policy_version 770345 (0.0007) [2023-12-27 10:57:31,540][20347] Updated weights for policy 0, policy_version 770355 (0.0008) [2023-12-27 10:57:31,596][20347] Updated weights for policy 0, policy_version 770365 (0.0008) [2023-12-27 10:57:32,014][20381] Updated weights for policy 1, policy_version 769952 (0.0006) [2023-12-27 10:57:32,065][20381] Updated weights for policy 1, policy_version 769962 (0.0007) [2023-12-27 10:57:32,118][20381] Updated weights for policy 1, policy_version 769972 (0.0010) [2023-12-27 10:57:32,344][20347] Updated weights for policy 0, policy_version 770375 (0.0008) [2023-12-27 10:57:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 394379264. Throughput: 0: 10339.6, 1: 10270.0. Samples: 394391636. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:32,363][19170] Avg episode reward: [(0, '892.700'), (1, '805.120')] [2023-12-27 10:57:32,404][20347] Updated weights for policy 0, policy_version 770385 (0.0006) [2023-12-27 10:57:32,461][20347] Updated weights for policy 0, policy_version 770395 (0.0008) [2023-12-27 10:57:32,738][20381] Updated weights for policy 1, policy_version 769982 (0.0007) [2023-12-27 10:57:32,746][20292] KL-divergence is very high: 377.5164 [2023-12-27 10:57:32,797][20292] KL-divergence is very high: 140.1481 [2023-12-27 10:57:32,802][20381] Updated weights for policy 1, policy_version 769992 (0.0005) [2023-12-27 10:57:32,859][20381] Updated weights for policy 1, policy_version 770002 (0.0006) [2023-12-27 10:57:33,135][20347] Updated weights for policy 0, policy_version 770405 (0.0005) [2023-12-27 10:57:33,188][20347] Updated weights for policy 0, policy_version 770415 (0.0005) [2023-12-27 10:57:33,247][20347] Updated weights for policy 0, policy_version 770425 (0.0007) [2023-12-27 10:57:33,460][20381] Updated weights for policy 1, policy_version 770012 (0.0007) [2023-12-27 10:57:33,518][20381] Updated weights for policy 1, policy_version 770022 (0.0010) [2023-12-27 10:57:33,565][20381] Updated weights for policy 1, policy_version 770032 (0.0010) [2023-12-27 10:57:33,864][20347] Updated weights for policy 0, policy_version 770435 (0.0010) [2023-12-27 10:57:33,918][20347] Updated weights for policy 0, policy_version 770445 (0.0010) [2023-12-27 10:57:33,972][20347] Updated weights for policy 0, policy_version 770455 (0.0009) [2023-12-27 10:57:34,238][20381] Updated weights for policy 1, policy_version 770042 (0.0009) [2023-12-27 10:57:34,297][20381] Updated weights for policy 1, policy_version 770052 (0.0006) [2023-12-27 10:57:34,354][20381] Updated weights for policy 1, policy_version 770062 (0.0006) [2023-12-27 10:57:34,417][20381] Updated weights for policy 1, policy_version 770072 (0.0006) [2023-12-27 10:57:34,626][20347] Updated weights for policy 0, policy_version 770465 (0.0006) [2023-12-27 10:57:34,671][20347] Updated weights for policy 0, policy_version 770475 (0.0010) [2023-12-27 10:57:34,723][20347] Updated weights for policy 0, policy_version 770485 (0.0010) [2023-12-27 10:57:34,778][20347] Updated weights for policy 0, policy_version 770495 (0.0010) [2023-12-27 10:57:35,041][20381] Updated weights for policy 1, policy_version 770082 (0.0007) [2023-12-27 10:57:35,099][20381] Updated weights for policy 1, policy_version 770092 (0.0008) [2023-12-27 10:57:35,159][20381] Updated weights for policy 1, policy_version 770102 (0.0008) [2023-12-27 10:57:35,469][20347] Updated weights for policy 0, policy_version 770505 (0.0010) [2023-12-27 10:57:35,523][20347] Updated weights for policy 0, policy_version 770515 (0.0010) [2023-12-27 10:57:35,571][20347] Updated weights for policy 0, policy_version 770525 (0.0006) [2023-12-27 10:57:35,877][20381] Updated weights for policy 1, policy_version 770112 (0.0008) [2023-12-27 10:57:35,933][20381] Updated weights for policy 1, policy_version 770122 (0.0009) [2023-12-27 10:57:35,985][20381] Updated weights for policy 1, policy_version 770132 (0.0008) [2023-12-27 10:57:36,128][20347] Updated weights for policy 0, policy_version 770535 (0.0007) [2023-12-27 10:57:36,188][20347] Updated weights for policy 0, policy_version 770545 (0.0008) [2023-12-27 10:57:36,250][20347] Updated weights for policy 0, policy_version 770555 (0.0008) [2023-12-27 10:57:36,718][20381] Updated weights for policy 1, policy_version 770142 (0.0010) [2023-12-27 10:57:36,785][20381] Updated weights for policy 1, policy_version 770152 (0.0011) [2023-12-27 10:57:36,848][20381] Updated weights for policy 1, policy_version 770162 (0.0011) [2023-12-27 10:57:36,874][20347] Updated weights for policy 0, policy_version 770565 (0.0008) [2023-12-27 10:57:36,930][20347] Updated weights for policy 0, policy_version 770575 (0.0007) [2023-12-27 10:57:36,986][20347] Updated weights for policy 0, policy_version 770585 (0.0008) [2023-12-27 10:57:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 394485760. Throughput: 0: 10371.9, 1: 10300.0. Samples: 394454888. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:37,363][19170] Avg episode reward: [(0, '921.920'), (1, '892.680')] [2023-12-27 10:57:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000770168_197189632.pth... [2023-12-27 10:57:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000770592_197296128.pth... [2023-12-27 10:57:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000769376_196984832.pth [2023-12-27 10:57:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000768984_196886528.pth [2023-12-27 10:57:37,530][20381] Updated weights for policy 1, policy_version 770172 (0.0010) [2023-12-27 10:57:37,583][20381] Updated weights for policy 1, policy_version 770182 (0.0008) [2023-12-27 10:57:37,639][20381] Updated weights for policy 1, policy_version 770192 (0.0008) [2023-12-27 10:57:37,695][20347] Updated weights for policy 0, policy_version 770595 (0.0008) [2023-12-27 10:57:37,750][20347] Updated weights for policy 0, policy_version 770605 (0.0008) [2023-12-27 10:57:37,806][20347] Updated weights for policy 0, policy_version 770615 (0.0007) [2023-12-27 10:57:38,265][20381] Updated weights for policy 1, policy_version 770202 (0.0008) [2023-12-27 10:57:38,317][20381] Updated weights for policy 1, policy_version 770212 (0.0009) [2023-12-27 10:57:38,376][20381] Updated weights for policy 1, policy_version 770222 (0.0008) [2023-12-27 10:57:38,432][20381] Updated weights for policy 1, policy_version 770232 (0.0008) [2023-12-27 10:57:38,522][20347] Updated weights for policy 0, policy_version 770625 (0.0008) [2023-12-27 10:57:38,582][20347] Updated weights for policy 0, policy_version 770635 (0.0007) [2023-12-27 10:57:38,637][20347] Updated weights for policy 0, policy_version 770645 (0.0007) [2023-12-27 10:57:38,685][20347] Updated weights for policy 0, policy_version 770655 (0.0007) [2023-12-27 10:57:39,129][20381] Updated weights for policy 1, policy_version 770242 (0.0007) [2023-12-27 10:57:39,186][20381] Updated weights for policy 1, policy_version 770252 (0.0008) [2023-12-27 10:57:39,248][20381] Updated weights for policy 1, policy_version 770262 (0.0009) [2023-12-27 10:57:39,411][20347] Updated weights for policy 0, policy_version 770665 (0.0007) [2023-12-27 10:57:39,476][20347] Updated weights for policy 0, policy_version 770675 (0.0007) [2023-12-27 10:57:39,543][20347] Updated weights for policy 0, policy_version 770685 (0.0006) [2023-12-27 10:57:39,989][20381] Updated weights for policy 1, policy_version 770272 (0.0007) [2023-12-27 10:57:40,053][20381] Updated weights for policy 1, policy_version 770282 (0.0006) [2023-12-27 10:57:40,116][20381] Updated weights for policy 1, policy_version 770292 (0.0007) [2023-12-27 10:57:40,175][20347] Updated weights for policy 0, policy_version 770695 (0.0008) [2023-12-27 10:57:40,230][20347] Updated weights for policy 0, policy_version 770705 (0.0007) [2023-12-27 10:57:40,292][20347] Updated weights for policy 0, policy_version 770715 (0.0008) [2023-12-27 10:57:40,782][20381] Updated weights for policy 1, policy_version 770302 (0.0007) [2023-12-27 10:57:40,833][20381] Updated weights for policy 1, policy_version 770312 (0.0006) [2023-12-27 10:57:40,896][20381] Updated weights for policy 1, policy_version 770322 (0.0006) [2023-12-27 10:57:40,903][20347] Updated weights for policy 0, policy_version 770725 (0.0006) [2023-12-27 10:57:40,957][20347] Updated weights for policy 0, policy_version 770735 (0.0009) [2023-12-27 10:57:41,004][20347] Updated weights for policy 0, policy_version 770745 (0.0010) [2023-12-27 10:57:41,569][20381] Updated weights for policy 1, policy_version 770332 (0.0006) [2023-12-27 10:57:41,630][20381] Updated weights for policy 1, policy_version 770342 (0.0007) [2023-12-27 10:57:41,687][20381] Updated weights for policy 1, policy_version 770352 (0.0007) [2023-12-27 10:57:41,792][20347] Updated weights for policy 0, policy_version 770755 (0.0008) [2023-12-27 10:57:41,848][20347] Updated weights for policy 0, policy_version 770765 (0.0008) [2023-12-27 10:57:41,897][20347] Updated weights for policy 0, policy_version 770775 (0.0008) [2023-12-27 10:57:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 394584064. Throughput: 0: 10366.4, 1: 10269.5. Samples: 394577236. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:42,363][19170] Avg episode reward: [(0, '902.440'), (1, '814.430')] [2023-12-27 10:57:42,465][20381] Updated weights for policy 1, policy_version 770362 (0.0008) [2023-12-27 10:57:42,524][20381] Updated weights for policy 1, policy_version 770372 (0.0011) [2023-12-27 10:57:42,589][20381] Updated weights for policy 1, policy_version 770382 (0.0011) [2023-12-27 10:57:42,623][20347] Updated weights for policy 0, policy_version 770785 (0.0008) [2023-12-27 10:57:42,652][20381] Updated weights for policy 1, policy_version 770392 (0.0007) [2023-12-27 10:57:42,678][20291] KL-divergence is very high: 178.3021 [2023-12-27 10:57:42,682][20347] Updated weights for policy 0, policy_version 770795 (0.0007) [2023-12-27 10:57:42,703][20291] KL-divergence is very high: 372.3679 [2023-12-27 10:57:42,742][20347] Updated weights for policy 0, policy_version 770805 (0.0007) [2023-12-27 10:57:42,799][20347] Updated weights for policy 0, policy_version 770815 (0.0008) [2023-12-27 10:57:43,299][20381] Updated weights for policy 1, policy_version 770402 (0.0010) [2023-12-27 10:57:43,353][20381] Updated weights for policy 1, policy_version 770412 (0.0010) [2023-12-27 10:57:43,394][20291] KL-divergence is very high: 109.4218 [2023-12-27 10:57:43,408][20381] Updated weights for policy 1, policy_version 770422 (0.0010) [2023-12-27 10:57:43,429][20347] Updated weights for policy 0, policy_version 770825 (0.0005) [2023-12-27 10:57:43,488][20291] KL-divergence is very high: 107.6064 [2023-12-27 10:57:43,490][20347] Updated weights for policy 0, policy_version 770835 (0.0007) [2023-12-27 10:57:43,527][20291] KL-divergence is very high: 126.4171 [2023-12-27 10:57:43,537][20347] Updated weights for policy 0, policy_version 770845 (0.0007) [2023-12-27 10:57:44,047][20381] Updated weights for policy 1, policy_version 770432 (0.0006) [2023-12-27 10:57:44,103][20381] Updated weights for policy 1, policy_version 770442 (0.0005) [2023-12-27 10:57:44,168][20381] Updated weights for policy 1, policy_version 770452 (0.0006) [2023-12-27 10:57:44,186][20347] Updated weights for policy 0, policy_version 770855 (0.0006) [2023-12-27 10:57:44,237][20347] Updated weights for policy 0, policy_version 770865 (0.0007) [2023-12-27 10:57:44,285][20347] Updated weights for policy 0, policy_version 770875 (0.0007) [2023-12-27 10:57:44,742][20381] Updated weights for policy 1, policy_version 770462 (0.0008) [2023-12-27 10:57:44,754][20292] KL-divergence is very high: 103.6398 [2023-12-27 10:57:44,781][20292] KL-divergence is very high: 134.2858 [2023-12-27 10:57:44,804][20381] Updated weights for policy 1, policy_version 770472 (0.0009) [2023-12-27 10:57:44,866][20381] Updated weights for policy 1, policy_version 770482 (0.0009) [2023-12-27 10:57:45,058][20347] Updated weights for policy 0, policy_version 770885 (0.0008) [2023-12-27 10:57:45,116][20347] Updated weights for policy 0, policy_version 770895 (0.0006) [2023-12-27 10:57:45,173][20347] Updated weights for policy 0, policy_version 770905 (0.0006) [2023-12-27 10:57:45,596][20381] Updated weights for policy 1, policy_version 770492 (0.0009) [2023-12-27 10:57:45,645][20381] Updated weights for policy 1, policy_version 770502 (0.0008) [2023-12-27 10:57:45,690][20381] Updated weights for policy 1, policy_version 770512 (0.0009) [2023-12-27 10:57:45,769][20347] Updated weights for policy 0, policy_version 770915 (0.0006) [2023-12-27 10:57:45,824][20347] Updated weights for policy 0, policy_version 770925 (0.0005) [2023-12-27 10:57:45,891][20347] Updated weights for policy 0, policy_version 770935 (0.0007) [2023-12-27 10:57:46,429][20381] Updated weights for policy 1, policy_version 770522 (0.0008) [2023-12-27 10:57:46,468][20347] Updated weights for policy 0, policy_version 770945 (0.0010) [2023-12-27 10:57:46,482][20381] Updated weights for policy 1, policy_version 770532 (0.0008) [2023-12-27 10:57:46,523][20347] Updated weights for policy 0, policy_version 770955 (0.0006) [2023-12-27 10:57:46,536][20381] Updated weights for policy 1, policy_version 770542 (0.0008) [2023-12-27 10:57:46,578][20347] Updated weights for policy 0, policy_version 770965 (0.0006) [2023-12-27 10:57:46,594][20381] Updated weights for policy 1, policy_version 770552 (0.0009) [2023-12-27 10:57:46,627][20347] Updated weights for policy 0, policy_version 770975 (0.0009) [2023-12-27 10:57:47,258][20381] Updated weights for policy 1, policy_version 770562 (0.0006) [2023-12-27 10:57:47,285][20347] Updated weights for policy 0, policy_version 770985 (0.0007) [2023-12-27 10:57:47,315][20381] Updated weights for policy 1, policy_version 770572 (0.0005) [2023-12-27 10:57:47,334][20347] Updated weights for policy 0, policy_version 770995 (0.0007) [2023-12-27 10:57:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 394682368. Throughput: 0: 10346.5, 1: 10288.3. Samples: 394700332. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:47,363][19170] Avg episode reward: [(0, '883.100'), (1, '795.170')] [2023-12-27 10:57:47,365][20381] Updated weights for policy 1, policy_version 770582 (0.0005) [2023-12-27 10:57:47,391][20347] Updated weights for policy 0, policy_version 771005 (0.0007) [2023-12-27 10:57:47,891][20381] Updated weights for policy 1, policy_version 770592 (0.0006) [2023-12-27 10:57:47,944][20381] Updated weights for policy 1, policy_version 770602 (0.0005) [2023-12-27 10:57:47,986][20292] KL-divergence is very high: 121.7660 [2023-12-27 10:57:48,002][20381] Updated weights for policy 1, policy_version 770612 (0.0005) [2023-12-27 10:57:48,092][20347] Updated weights for policy 0, policy_version 771015 (0.0008) [2023-12-27 10:57:48,151][20347] Updated weights for policy 0, policy_version 771025 (0.0007) [2023-12-27 10:57:48,207][20347] Updated weights for policy 0, policy_version 771035 (0.0007) [2023-12-27 10:57:48,662][20381] Updated weights for policy 1, policy_version 770622 (0.0008) [2023-12-27 10:57:48,712][20381] Updated weights for policy 1, policy_version 770632 (0.0008) [2023-12-27 10:57:48,763][20381] Updated weights for policy 1, policy_version 770642 (0.0008) [2023-12-27 10:57:48,852][20347] Updated weights for policy 0, policy_version 771045 (0.0006) [2023-12-27 10:57:48,906][20347] Updated weights for policy 0, policy_version 771055 (0.0007) [2023-12-27 10:57:48,954][20347] Updated weights for policy 0, policy_version 771065 (0.0007) [2023-12-27 10:57:49,483][20381] Updated weights for policy 1, policy_version 770652 (0.0007) [2023-12-27 10:57:49,540][20381] Updated weights for policy 1, policy_version 770662 (0.0008) [2023-12-27 10:57:49,571][20347] Updated weights for policy 0, policy_version 771075 (0.0007) [2023-12-27 10:57:49,593][20381] Updated weights for policy 1, policy_version 770672 (0.0008) [2023-12-27 10:57:49,635][20347] Updated weights for policy 0, policy_version 771085 (0.0007) [2023-12-27 10:57:49,696][20347] Updated weights for policy 0, policy_version 771095 (0.0008) [2023-12-27 10:57:50,294][20381] Updated weights for policy 1, policy_version 770682 (0.0007) [2023-12-27 10:57:50,355][20381] Updated weights for policy 1, policy_version 770692 (0.0007) [2023-12-27 10:57:50,395][20347] Updated weights for policy 0, policy_version 771105 (0.0007) [2023-12-27 10:57:50,413][20381] Updated weights for policy 1, policy_version 770702 (0.0007) [2023-12-27 10:57:50,453][20347] Updated weights for policy 0, policy_version 771115 (0.0006) [2023-12-27 10:57:50,470][20381] Updated weights for policy 1, policy_version 770712 (0.0006) [2023-12-27 10:57:50,508][20347] Updated weights for policy 0, policy_version 771125 (0.0007) [2023-12-27 10:57:50,571][20347] Updated weights for policy 0, policy_version 771135 (0.0007) [2023-12-27 10:57:51,265][20381] Updated weights for policy 1, policy_version 770722 (0.0007) [2023-12-27 10:57:51,295][20347] Updated weights for policy 0, policy_version 771145 (0.0008) [2023-12-27 10:57:51,327][20381] Updated weights for policy 1, policy_version 770732 (0.0006) [2023-12-27 10:57:51,356][20347] Updated weights for policy 0, policy_version 771155 (0.0008) [2023-12-27 10:57:51,390][20381] Updated weights for policy 1, policy_version 770742 (0.0008) [2023-12-27 10:57:51,424][20347] Updated weights for policy 0, policy_version 771165 (0.0008) [2023-12-27 10:57:52,052][20381] Updated weights for policy 1, policy_version 770752 (0.0007) [2023-12-27 10:57:52,112][20381] Updated weights for policy 1, policy_version 770762 (0.0008) [2023-12-27 10:57:52,167][20381] Updated weights for policy 1, policy_version 770772 (0.0008) [2023-12-27 10:57:52,170][20347] Updated weights for policy 0, policy_version 771175 (0.0007) [2023-12-27 10:57:52,221][20347] Updated weights for policy 0, policy_version 771185 (0.0007) [2023-12-27 10:57:52,282][20347] Updated weights for policy 0, policy_version 771195 (0.0008) [2023-12-27 10:57:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 394797056. Throughput: 0: 10357.2, 1: 10299.7. Samples: 394763496. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:52,363][19170] Avg episode reward: [(0, '873.340'), (1, '863.760')] [2023-12-27 10:57:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000770776_197345280.pth... [2023-12-27 10:57:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000771200_197451776.pth... [2023-12-27 10:57:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000769560_197033984.pth [2023-12-27 10:57:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000769952_197132288.pth [2023-12-27 10:57:52,813][20381] Updated weights for policy 1, policy_version 770782 (0.0007) [2023-12-27 10:57:52,866][20381] Updated weights for policy 1, policy_version 770792 (0.0008) [2023-12-27 10:57:52,922][20381] Updated weights for policy 1, policy_version 770802 (0.0009) [2023-12-27 10:57:52,969][20347] Updated weights for policy 0, policy_version 771205 (0.0006) [2023-12-27 10:57:53,024][20347] Updated weights for policy 0, policy_version 771215 (0.0007) [2023-12-27 10:57:53,080][20347] Updated weights for policy 0, policy_version 771225 (0.0006) [2023-12-27 10:57:53,638][20381] Updated weights for policy 1, policy_version 770812 (0.0007) [2023-12-27 10:57:53,692][20381] Updated weights for policy 1, policy_version 770822 (0.0006) [2023-12-27 10:57:53,707][20347] Updated weights for policy 0, policy_version 771235 (0.0007) [2023-12-27 10:57:53,737][20381] Updated weights for policy 1, policy_version 770832 (0.0005) [2023-12-27 10:57:53,758][20347] Updated weights for policy 0, policy_version 771245 (0.0007) [2023-12-27 10:57:53,809][20347] Updated weights for policy 0, policy_version 771255 (0.0007) [2023-12-27 10:57:54,365][20381] Updated weights for policy 1, policy_version 770842 (0.0006) [2023-12-27 10:57:54,423][20381] Updated weights for policy 1, policy_version 770852 (0.0008) [2023-12-27 10:57:54,448][20347] Updated weights for policy 0, policy_version 771265 (0.0008) [2023-12-27 10:57:54,477][20381] Updated weights for policy 1, policy_version 770862 (0.0008) [2023-12-27 10:57:54,496][20347] Updated weights for policy 0, policy_version 771275 (0.0005) [2023-12-27 10:57:54,535][20381] Updated weights for policy 1, policy_version 770872 (0.0008) [2023-12-27 10:57:54,545][20347] Updated weights for policy 0, policy_version 771285 (0.0005) [2023-12-27 10:57:54,597][20347] Updated weights for policy 0, policy_version 771295 (0.0005) [2023-12-27 10:57:55,149][20347] Updated weights for policy 0, policy_version 771305 (0.0005) [2023-12-27 10:57:55,205][20347] Updated weights for policy 0, policy_version 771315 (0.0005) [2023-12-27 10:57:55,262][20347] Updated weights for policy 0, policy_version 771325 (0.0006) [2023-12-27 10:57:55,269][20381] Updated weights for policy 1, policy_version 770882 (0.0010) [2023-12-27 10:57:55,323][20381] Updated weights for policy 1, policy_version 770892 (0.0010) [2023-12-27 10:57:55,377][20381] Updated weights for policy 1, policy_version 770902 (0.0006) [2023-12-27 10:57:55,804][20347] Updated weights for policy 0, policy_version 771335 (0.0007) [2023-12-27 10:57:55,854][20347] Updated weights for policy 0, policy_version 771345 (0.0007) [2023-12-27 10:57:55,907][20347] Updated weights for policy 0, policy_version 771355 (0.0007) [2023-12-27 10:57:55,989][20381] Updated weights for policy 1, policy_version 770912 (0.0006) [2023-12-27 10:57:56,040][20381] Updated weights for policy 1, policy_version 770922 (0.0005) [2023-12-27 10:57:56,096][20381] Updated weights for policy 1, policy_version 770932 (0.0005) [2023-12-27 10:57:56,632][20381] Updated weights for policy 1, policy_version 770942 (0.0006) [2023-12-27 10:57:56,658][20347] Updated weights for policy 0, policy_version 771365 (0.0007) [2023-12-27 10:57:56,680][20381] Updated weights for policy 1, policy_version 770952 (0.0006) [2023-12-27 10:57:56,706][20347] Updated weights for policy 0, policy_version 771375 (0.0005) [2023-12-27 10:57:56,734][20381] Updated weights for policy 1, policy_version 770962 (0.0007) [2023-12-27 10:57:56,751][20347] Updated weights for policy 0, policy_version 771385 (0.0005) [2023-12-27 10:57:57,362][20347] Updated weights for policy 0, policy_version 771395 (0.0007) [2023-12-27 10:57:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 394895360. Throughput: 0: 10371.9, 1: 10315.2. Samples: 394887936. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:57:57,363][19170] Avg episode reward: [(0, '882.940'), (1, '941.530')] [2023-12-27 10:57:57,370][20381] Updated weights for policy 1, policy_version 770972 (0.0007) [2023-12-27 10:57:57,419][20347] Updated weights for policy 0, policy_version 771405 (0.0005) [2023-12-27 10:57:57,422][20381] Updated weights for policy 1, policy_version 770982 (0.0010) [2023-12-27 10:57:57,471][20381] Updated weights for policy 1, policy_version 770992 (0.0010) [2023-12-27 10:57:57,476][20347] Updated weights for policy 0, policy_version 771415 (0.0006) [2023-12-27 10:57:58,124][20347] Updated weights for policy 0, policy_version 771425 (0.0007) [2023-12-27 10:57:58,150][20381] Updated weights for policy 1, policy_version 771002 (0.0010) [2023-12-27 10:57:58,176][20347] Updated weights for policy 0, policy_version 771435 (0.0008) [2023-12-27 10:57:58,211][20381] Updated weights for policy 1, policy_version 771012 (0.0010) [2023-12-27 10:57:58,237][20347] Updated weights for policy 0, policy_version 771445 (0.0007) [2023-12-27 10:57:58,269][20381] Updated weights for policy 1, policy_version 771022 (0.0011) [2023-12-27 10:57:58,300][20347] Updated weights for policy 0, policy_version 771455 (0.0006) [2023-12-27 10:57:58,330][20381] Updated weights for policy 1, policy_version 771032 (0.0010) [2023-12-27 10:57:59,069][20381] Updated weights for policy 1, policy_version 771042 (0.0008) [2023-12-27 10:57:59,124][20381] Updated weights for policy 1, policy_version 771052 (0.0008) [2023-12-27 10:57:59,161][20347] Updated weights for policy 0, policy_version 771465 (0.0008) [2023-12-27 10:57:59,184][20381] Updated weights for policy 1, policy_version 771062 (0.0006) [2023-12-27 10:57:59,233][20347] Updated weights for policy 0, policy_version 771475 (0.0009) [2023-12-27 10:57:59,296][20347] Updated weights for policy 0, policy_version 771485 (0.0008) [2023-12-27 10:57:59,929][20381] Updated weights for policy 1, policy_version 771072 (0.0007) [2023-12-27 10:57:59,984][20381] Updated weights for policy 1, policy_version 771082 (0.0008) [2023-12-27 10:58:00,036][20381] Updated weights for policy 1, policy_version 771092 (0.0008) [2023-12-27 10:58:00,055][20347] Updated weights for policy 0, policy_version 771495 (0.0006) [2023-12-27 10:58:00,114][20347] Updated weights for policy 0, policy_version 771505 (0.0005) [2023-12-27 10:58:00,168][20347] Updated weights for policy 0, policy_version 771515 (0.0005) [2023-12-27 10:58:00,720][20347] Updated weights for policy 0, policy_version 771525 (0.0007) [2023-12-27 10:58:00,763][20381] Updated weights for policy 1, policy_version 771102 (0.0008) [2023-12-27 10:58:00,779][20347] Updated weights for policy 0, policy_version 771535 (0.0008) [2023-12-27 10:58:00,818][20381] Updated weights for policy 1, policy_version 771112 (0.0006) [2023-12-27 10:58:00,838][20347] Updated weights for policy 0, policy_version 771545 (0.0006) [2023-12-27 10:58:00,870][20381] Updated weights for policy 1, policy_version 771122 (0.0006) [2023-12-27 10:58:01,494][20347] Updated weights for policy 0, policy_version 771555 (0.0007) [2023-12-27 10:58:01,539][20347] Updated weights for policy 0, policy_version 771565 (0.0007) [2023-12-27 10:58:01,592][20347] Updated weights for policy 0, policy_version 771575 (0.0006) [2023-12-27 10:58:01,593][20381] Updated weights for policy 1, policy_version 771132 (0.0007) [2023-12-27 10:58:01,655][20381] Updated weights for policy 1, policy_version 771142 (0.0008) [2023-12-27 10:58:01,715][20381] Updated weights for policy 1, policy_version 771152 (0.0008) [2023-12-27 10:58:02,313][20347] Updated weights for policy 0, policy_version 771585 (0.0006) [2023-12-27 10:58:02,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 394993664. Throughput: 0: 10371.2, 1: 10273.1. Samples: 395008712. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:02,363][19170] Avg episode reward: [(0, '941.420'), (1, '971.930')] [2023-12-27 10:58:02,375][20347] Updated weights for policy 0, policy_version 771595 (0.0008) [2023-12-27 10:58:02,430][20347] Updated weights for policy 0, policy_version 771605 (0.0006) [2023-12-27 10:58:02,431][20381] Updated weights for policy 1, policy_version 771162 (0.0008) [2023-12-27 10:58:02,483][20381] Updated weights for policy 1, policy_version 771172 (0.0005) [2023-12-27 10:58:02,489][20347] Updated weights for policy 0, policy_version 771615 (0.0007) [2023-12-27 10:58:02,536][20381] Updated weights for policy 1, policy_version 771182 (0.0007) [2023-12-27 10:58:02,586][20381] Updated weights for policy 1, policy_version 771192 (0.0007) [2023-12-27 10:58:03,130][20347] Updated weights for policy 0, policy_version 771625 (0.0007) [2023-12-27 10:58:03,179][20347] Updated weights for policy 0, policy_version 771635 (0.0008) [2023-12-27 10:58:03,229][20347] Updated weights for policy 0, policy_version 771645 (0.0009) [2023-12-27 10:58:03,268][20381] Updated weights for policy 1, policy_version 771202 (0.0005) [2023-12-27 10:58:03,314][20381] Updated weights for policy 1, policy_version 771212 (0.0006) [2023-12-27 10:58:03,368][20381] Updated weights for policy 1, policy_version 771222 (0.0007) [2023-12-27 10:58:03,927][20347] Updated weights for policy 0, policy_version 771655 (0.0007) [2023-12-27 10:58:03,987][20347] Updated weights for policy 0, policy_version 771665 (0.0008) [2023-12-27 10:58:04,030][20381] Updated weights for policy 1, policy_version 771232 (0.0008) [2023-12-27 10:58:04,047][20347] Updated weights for policy 0, policy_version 771675 (0.0006) [2023-12-27 10:58:04,088][20381] Updated weights for policy 1, policy_version 771242 (0.0007) [2023-12-27 10:58:04,146][20381] Updated weights for policy 1, policy_version 771252 (0.0008) [2023-12-27 10:58:04,649][20347] Updated weights for policy 0, policy_version 771685 (0.0007) [2023-12-27 10:58:04,698][20347] Updated weights for policy 0, policy_version 771695 (0.0005) [2023-12-27 10:58:04,744][20347] Updated weights for policy 0, policy_version 771705 (0.0005) [2023-12-27 10:58:04,878][20381] Updated weights for policy 1, policy_version 771262 (0.0008) [2023-12-27 10:58:04,945][20381] Updated weights for policy 1, policy_version 771272 (0.0009) [2023-12-27 10:58:05,011][20381] Updated weights for policy 1, policy_version 771282 (0.0008) [2023-12-27 10:58:05,297][20347] Updated weights for policy 0, policy_version 771715 (0.0005) [2023-12-27 10:58:05,347][20347] Updated weights for policy 0, policy_version 771725 (0.0005) [2023-12-27 10:58:05,397][20347] Updated weights for policy 0, policy_version 771735 (0.0005) [2023-12-27 10:58:05,696][20381] Updated weights for policy 1, policy_version 771292 (0.0007) [2023-12-27 10:58:05,766][20381] Updated weights for policy 1, policy_version 771302 (0.0008) [2023-12-27 10:58:05,811][20381] Updated weights for policy 1, policy_version 771312 (0.0007) [2023-12-27 10:58:05,926][20347] Updated weights for policy 0, policy_version 771745 (0.0005) [2023-12-27 10:58:05,976][20347] Updated weights for policy 0, policy_version 771755 (0.0005) [2023-12-27 10:58:06,022][20347] Updated weights for policy 0, policy_version 771765 (0.0006) [2023-12-27 10:58:06,079][20347] Updated weights for policy 0, policy_version 771775 (0.0007) [2023-12-27 10:58:06,425][20381] Updated weights for policy 1, policy_version 771322 (0.0009) [2023-12-27 10:58:06,478][20381] Updated weights for policy 1, policy_version 771332 (0.0008) [2023-12-27 10:58:06,535][20381] Updated weights for policy 1, policy_version 771342 (0.0008) [2023-12-27 10:58:06,596][20381] Updated weights for policy 1, policy_version 771352 (0.0007) [2023-12-27 10:58:06,747][20347] Updated weights for policy 0, policy_version 771785 (0.0007) [2023-12-27 10:58:06,793][20347] Updated weights for policy 0, policy_version 771795 (0.0008) [2023-12-27 10:58:06,843][20347] Updated weights for policy 0, policy_version 771805 (0.0008) [2023-12-27 10:58:07,302][20381] Updated weights for policy 1, policy_version 771362 (0.0008) [2023-12-27 10:58:07,362][20381] Updated weights for policy 1, policy_version 771372 (0.0008) [2023-12-27 10:58:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 395100160. Throughput: 0: 10439.6, 1: 10244.5. Samples: 395070884. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:07,363][19170] Avg episode reward: [(0, '902.330'), (1, '939.380')] [2023-12-27 10:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000771808_197607424.pth... [2023-12-27 10:58:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000770592_197296128.pth [2023-12-27 10:58:07,418][20381] Updated weights for policy 1, policy_version 771382 (0.0008) [2023-12-27 10:58:07,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000771384_197500928.pth... [2023-12-27 10:58:07,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000770168_197189632.pth [2023-12-27 10:58:07,551][20347] Updated weights for policy 0, policy_version 771815 (0.0008) [2023-12-27 10:58:07,608][20347] Updated weights for policy 0, policy_version 771825 (0.0007) [2023-12-27 10:58:07,663][20347] Updated weights for policy 0, policy_version 771835 (0.0005) [2023-12-27 10:58:08,116][20381] Updated weights for policy 1, policy_version 771392 (0.0007) [2023-12-27 10:58:08,172][20381] Updated weights for policy 1, policy_version 771402 (0.0008) [2023-12-27 10:58:08,227][20381] Updated weights for policy 1, policy_version 771412 (0.0008) [2023-12-27 10:58:08,305][20347] Updated weights for policy 0, policy_version 771845 (0.0005) [2023-12-27 10:58:08,344][20291] KL-divergence is very high: 109.6117 [2023-12-27 10:58:08,374][20347] Updated weights for policy 0, policy_version 771855 (0.0008) [2023-12-27 10:58:08,425][20347] Updated weights for policy 0, policy_version 771865 (0.0007) [2023-12-27 10:58:08,913][20381] Updated weights for policy 1, policy_version 771422 (0.0009) [2023-12-27 10:58:08,964][20381] Updated weights for policy 1, policy_version 771432 (0.0007) [2023-12-27 10:58:09,023][20381] Updated weights for policy 1, policy_version 771442 (0.0008) [2023-12-27 10:58:09,123][20347] Updated weights for policy 0, policy_version 771875 (0.0008) [2023-12-27 10:58:09,169][20291] KL-divergence is very high: 146.7548 [2023-12-27 10:58:09,179][20347] Updated weights for policy 0, policy_version 771885 (0.0007) [2023-12-27 10:58:09,257][20347] Updated weights for policy 0, policy_version 771895 (0.0007) [2023-12-27 10:58:09,728][20381] Updated weights for policy 1, policy_version 771452 (0.0008) [2023-12-27 10:58:09,789][20381] Updated weights for policy 1, policy_version 771462 (0.0008) [2023-12-27 10:58:09,812][20292] KL-divergence is very high: 121.3326 [2023-12-27 10:58:09,852][20381] Updated weights for policy 1, policy_version 771472 (0.0008) [2023-12-27 10:58:09,936][20347] Updated weights for policy 0, policy_version 771905 (0.0008) [2023-12-27 10:58:10,000][20347] Updated weights for policy 0, policy_version 771915 (0.0006) [2023-12-27 10:58:10,059][20347] Updated weights for policy 0, policy_version 771925 (0.0007) [2023-12-27 10:58:10,121][20347] Updated weights for policy 0, policy_version 771935 (0.0008) [2023-12-27 10:58:10,591][20381] Updated weights for policy 1, policy_version 771482 (0.0008) [2023-12-27 10:58:10,642][20381] Updated weights for policy 1, policy_version 771492 (0.0009) [2023-12-27 10:58:10,691][20381] Updated weights for policy 1, policy_version 771502 (0.0008) [2023-12-27 10:58:10,741][20347] Updated weights for policy 0, policy_version 771945 (0.0006) [2023-12-27 10:58:10,745][20381] Updated weights for policy 1, policy_version 771512 (0.0008) [2023-12-27 10:58:10,811][20347] Updated weights for policy 0, policy_version 771955 (0.0007) [2023-12-27 10:58:10,875][20347] Updated weights for policy 0, policy_version 771965 (0.0006) [2023-12-27 10:58:11,557][20381] Updated weights for policy 1, policy_version 771522 (0.0009) [2023-12-27 10:58:11,620][20381] Updated weights for policy 1, policy_version 771532 (0.0009) [2023-12-27 10:58:11,680][20381] Updated weights for policy 1, policy_version 771542 (0.0009) [2023-12-27 10:58:11,681][20347] Updated weights for policy 0, policy_version 771975 (0.0008) [2023-12-27 10:58:11,742][20347] Updated weights for policy 0, policy_version 771985 (0.0009) [2023-12-27 10:58:11,796][20347] Updated weights for policy 0, policy_version 771995 (0.0009) [2023-12-27 10:58:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 395198464. Throughput: 0: 10447.6, 1: 10185.6. Samples: 395194204. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:12,363][19170] Avg episode reward: [(0, '872.910'), (1, '881.140')] [2023-12-27 10:58:12,503][20381] Updated weights for policy 1, policy_version 771552 (0.0007) [2023-12-27 10:58:12,551][20381] Updated weights for policy 1, policy_version 771562 (0.0007) [2023-12-27 10:58:12,601][20347] Updated weights for policy 0, policy_version 772005 (0.0010) [2023-12-27 10:58:12,609][20381] Updated weights for policy 1, policy_version 771572 (0.0008) [2023-12-27 10:58:12,662][20347] Updated weights for policy 0, policy_version 772015 (0.0007) [2023-12-27 10:58:12,721][20347] Updated weights for policy 0, policy_version 772025 (0.0008) [2023-12-27 10:58:13,287][20381] Updated weights for policy 1, policy_version 771582 (0.0006) [2023-12-27 10:58:13,337][20381] Updated weights for policy 1, policy_version 771592 (0.0005) [2023-12-27 10:58:13,342][20347] Updated weights for policy 0, policy_version 772035 (0.0007) [2023-12-27 10:58:13,385][20381] Updated weights for policy 1, policy_version 771602 (0.0005) [2023-12-27 10:58:13,392][20347] Updated weights for policy 0, policy_version 772045 (0.0008) [2023-12-27 10:58:13,438][20347] Updated weights for policy 0, policy_version 772055 (0.0008) [2023-12-27 10:58:13,928][20381] Updated weights for policy 1, policy_version 771612 (0.0005) [2023-12-27 10:58:13,982][20381] Updated weights for policy 1, policy_version 771622 (0.0010) [2023-12-27 10:58:14,034][20381] Updated weights for policy 1, policy_version 771632 (0.0010) [2023-12-27 10:58:14,158][20347] Updated weights for policy 0, policy_version 772065 (0.0007) [2023-12-27 10:58:14,218][20347] Updated weights for policy 0, policy_version 772075 (0.0006) [2023-12-27 10:58:14,264][20347] Updated weights for policy 0, policy_version 772085 (0.0007) [2023-12-27 10:58:14,321][20347] Updated weights for policy 0, policy_version 772095 (0.0007) [2023-12-27 10:58:14,650][20381] Updated weights for policy 1, policy_version 771642 (0.0010) [2023-12-27 10:58:14,705][20381] Updated weights for policy 1, policy_version 771652 (0.0008) [2023-12-27 10:58:14,761][20381] Updated weights for policy 1, policy_version 771662 (0.0008) [2023-12-27 10:58:14,823][20381] Updated weights for policy 1, policy_version 771672 (0.0007) [2023-12-27 10:58:15,040][20347] Updated weights for policy 0, policy_version 772105 (0.0010) [2023-12-27 10:58:15,101][20347] Updated weights for policy 0, policy_version 772115 (0.0011) [2023-12-27 10:58:15,159][20347] Updated weights for policy 0, policy_version 772125 (0.0009) [2023-12-27 10:58:15,547][20381] Updated weights for policy 1, policy_version 771682 (0.0007) [2023-12-27 10:58:15,604][20381] Updated weights for policy 1, policy_version 771692 (0.0005) [2023-12-27 10:58:15,665][20381] Updated weights for policy 1, policy_version 771702 (0.0005) [2023-12-27 10:58:15,844][20347] Updated weights for policy 0, policy_version 772135 (0.0009) [2023-12-27 10:58:15,901][20347] Updated weights for policy 0, policy_version 772145 (0.0008) [2023-12-27 10:58:15,946][20347] Updated weights for policy 0, policy_version 772155 (0.0007) [2023-12-27 10:58:16,202][20381] Updated weights for policy 1, policy_version 771712 (0.0005) [2023-12-27 10:58:16,267][20381] Updated weights for policy 1, policy_version 771722 (0.0005) [2023-12-27 10:58:16,327][20381] Updated weights for policy 1, policy_version 771732 (0.0005) [2023-12-27 10:58:16,692][20347] Updated weights for policy 0, policy_version 772165 (0.0009) [2023-12-27 10:58:16,753][20347] Updated weights for policy 0, policy_version 772175 (0.0010) [2023-12-27 10:58:16,801][20347] Updated weights for policy 0, policy_version 772185 (0.0010) [2023-12-27 10:58:16,881][20381] Updated weights for policy 1, policy_version 771742 (0.0007) [2023-12-27 10:58:16,927][20381] Updated weights for policy 1, policy_version 771752 (0.0007) [2023-12-27 10:58:16,984][20381] Updated weights for policy 1, policy_version 771762 (0.0007) [2023-12-27 10:58:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 395304960. Throughput: 0: 10336.4, 1: 10198.0. Samples: 395315688. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:17,363][19170] Avg episode reward: [(0, '911.940'), (1, '872.950')] [2023-12-27 10:58:17,458][20347] Updated weights for policy 0, policy_version 772195 (0.0009) [2023-12-27 10:58:17,508][20347] Updated weights for policy 0, policy_version 772205 (0.0005) [2023-12-27 10:58:17,561][20347] Updated weights for policy 0, policy_version 772215 (0.0005) [2023-12-27 10:58:17,667][20381] Updated weights for policy 1, policy_version 771772 (0.0007) [2023-12-27 10:58:17,718][20381] Updated weights for policy 1, policy_version 771782 (0.0010) [2023-12-27 10:58:17,769][20381] Updated weights for policy 1, policy_version 771792 (0.0010) [2023-12-27 10:58:18,175][20347] Updated weights for policy 0, policy_version 772225 (0.0006) [2023-12-27 10:58:18,228][20347] Updated weights for policy 0, policy_version 772235 (0.0010) [2023-12-27 10:58:18,279][20347] Updated weights for policy 0, policy_version 772245 (0.0010) [2023-12-27 10:58:18,350][20347] Updated weights for policy 0, policy_version 772255 (0.0010) [2023-12-27 10:58:18,475][20381] Updated weights for policy 1, policy_version 771802 (0.0010) [2023-12-27 10:58:18,533][20381] Updated weights for policy 1, policy_version 771812 (0.0007) [2023-12-27 10:58:18,588][20381] Updated weights for policy 1, policy_version 771822 (0.0007) [2023-12-27 10:58:18,641][20381] Updated weights for policy 1, policy_version 771832 (0.0006) [2023-12-27 10:58:19,012][20347] Updated weights for policy 0, policy_version 772265 (0.0006) [2023-12-27 10:58:19,072][20347] Updated weights for policy 0, policy_version 772275 (0.0005) [2023-12-27 10:58:19,135][20347] Updated weights for policy 0, policy_version 772285 (0.0007) [2023-12-27 10:58:19,346][20381] Updated weights for policy 1, policy_version 771842 (0.0008) [2023-12-27 10:58:19,404][20381] Updated weights for policy 1, policy_version 771852 (0.0007) [2023-12-27 10:58:19,466][20381] Updated weights for policy 1, policy_version 771862 (0.0007) [2023-12-27 10:58:19,959][20347] Updated weights for policy 0, policy_version 772295 (0.0008) [2023-12-27 10:58:20,019][20347] Updated weights for policy 0, policy_version 772305 (0.0008) [2023-12-27 10:58:20,080][20347] Updated weights for policy 0, policy_version 772315 (0.0008) [2023-12-27 10:58:20,144][20381] Updated weights for policy 1, policy_version 771872 (0.0008) [2023-12-27 10:58:20,200][20381] Updated weights for policy 1, policy_version 771882 (0.0008) [2023-12-27 10:58:20,259][20381] Updated weights for policy 1, policy_version 771892 (0.0006) [2023-12-27 10:58:20,765][20347] Updated weights for policy 0, policy_version 772325 (0.0007) [2023-12-27 10:58:20,822][20347] Updated weights for policy 0, policy_version 772335 (0.0006) [2023-12-27 10:58:20,875][20347] Updated weights for policy 0, policy_version 772345 (0.0006) [2023-12-27 10:58:20,995][20381] Updated weights for policy 1, policy_version 771902 (0.0008) [2023-12-27 10:58:21,051][20381] Updated weights for policy 1, policy_version 771912 (0.0011) [2023-12-27 10:58:21,112][20381] Updated weights for policy 1, policy_version 771922 (0.0011) [2023-12-27 10:58:21,556][20347] Updated weights for policy 0, policy_version 772355 (0.0006) [2023-12-27 10:58:21,619][20347] Updated weights for policy 0, policy_version 772365 (0.0008) [2023-12-27 10:58:21,685][20347] Updated weights for policy 0, policy_version 772375 (0.0008) [2023-12-27 10:58:21,899][20381] Updated weights for policy 1, policy_version 771932 (0.0010) [2023-12-27 10:58:21,952][20381] Updated weights for policy 1, policy_version 771942 (0.0010) [2023-12-27 10:58:22,001][20381] Updated weights for policy 1, policy_version 771952 (0.0010) [2023-12-27 10:58:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 395403264. Throughput: 0: 10310.1, 1: 10156.5. Samples: 395375888. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:22,363][19170] Avg episode reward: [(0, '941.420'), (1, '902.520')] [2023-12-27 10:58:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000771960_197648384.pth... [2023-12-27 10:58:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000772384_197754880.pth... [2023-12-27 10:58:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000770776_197345280.pth [2023-12-27 10:58:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000771200_197451776.pth [2023-12-27 10:58:22,405][20347] Updated weights for policy 0, policy_version 772385 (0.0010) [2023-12-27 10:58:22,469][20347] Updated weights for policy 0, policy_version 772395 (0.0011) [2023-12-27 10:58:22,532][20347] Updated weights for policy 0, policy_version 772405 (0.0010) [2023-12-27 10:58:22,589][20347] Updated weights for policy 0, policy_version 772415 (0.0010) [2023-12-27 10:58:22,727][20381] Updated weights for policy 1, policy_version 771962 (0.0010) [2023-12-27 10:58:22,791][20381] Updated weights for policy 1, policy_version 771972 (0.0011) [2023-12-27 10:58:22,854][20381] Updated weights for policy 1, policy_version 771982 (0.0010) [2023-12-27 10:58:22,916][20381] Updated weights for policy 1, policy_version 771992 (0.0011) [2023-12-27 10:58:23,294][20347] Updated weights for policy 0, policy_version 772425 (0.0006) [2023-12-27 10:58:23,349][20347] Updated weights for policy 0, policy_version 772435 (0.0005) [2023-12-27 10:58:23,409][20347] Updated weights for policy 0, policy_version 772445 (0.0005) [2023-12-27 10:58:23,545][20381] Updated weights for policy 1, policy_version 772002 (0.0007) [2023-12-27 10:58:23,596][20381] Updated weights for policy 1, policy_version 772012 (0.0010) [2023-12-27 10:58:23,662][20381] Updated weights for policy 1, policy_version 772022 (0.0005) [2023-12-27 10:58:23,927][20347] Updated weights for policy 0, policy_version 772455 (0.0009) [2023-12-27 10:58:23,990][20347] Updated weights for policy 0, policy_version 772465 (0.0011) [2023-12-27 10:58:24,049][20347] Updated weights for policy 0, policy_version 772475 (0.0011) [2023-12-27 10:58:24,344][20381] Updated weights for policy 1, policy_version 772032 (0.0010) [2023-12-27 10:58:24,407][20381] Updated weights for policy 1, policy_version 772042 (0.0011) [2023-12-27 10:58:24,464][20381] Updated weights for policy 1, policy_version 772052 (0.0008) [2023-12-27 10:58:24,775][20347] Updated weights for policy 0, policy_version 772485 (0.0009) [2023-12-27 10:58:24,823][20347] Updated weights for policy 0, policy_version 772495 (0.0008) [2023-12-27 10:58:24,868][20347] Updated weights for policy 0, policy_version 772505 (0.0008) [2023-12-27 10:58:25,033][20381] Updated weights for policy 1, policy_version 772062 (0.0007) [2023-12-27 10:58:25,094][20381] Updated weights for policy 1, policy_version 772072 (0.0010) [2023-12-27 10:58:25,147][20381] Updated weights for policy 1, policy_version 772082 (0.0005) [2023-12-27 10:58:25,522][20347] Updated weights for policy 0, policy_version 772515 (0.0007) [2023-12-27 10:58:25,583][20347] Updated weights for policy 0, policy_version 772525 (0.0007) [2023-12-27 10:58:25,635][20347] Updated weights for policy 0, policy_version 772535 (0.0008) [2023-12-27 10:58:25,786][20381] Updated weights for policy 1, policy_version 772092 (0.0006) [2023-12-27 10:58:25,849][20381] Updated weights for policy 1, policy_version 772102 (0.0005) [2023-12-27 10:58:25,912][20381] Updated weights for policy 1, policy_version 772112 (0.0006) [2023-12-27 10:58:26,263][20347] Updated weights for policy 0, policy_version 772545 (0.0007) [2023-12-27 10:58:26,316][20347] Updated weights for policy 0, policy_version 772555 (0.0008) [2023-12-27 10:58:26,364][20347] Updated weights for policy 0, policy_version 772565 (0.0010) [2023-12-27 10:58:26,422][20347] Updated weights for policy 0, policy_version 772575 (0.0010) [2023-12-27 10:58:26,463][20381] Updated weights for policy 1, policy_version 772122 (0.0006) [2023-12-27 10:58:26,517][20381] Updated weights for policy 1, policy_version 772132 (0.0005) [2023-12-27 10:58:26,574][20381] Updated weights for policy 1, policy_version 772142 (0.0005) [2023-12-27 10:58:26,637][20381] Updated weights for policy 1, policy_version 772152 (0.0005) [2023-12-27 10:58:27,055][20347] Updated weights for policy 0, policy_version 772585 (0.0010) [2023-12-27 10:58:27,106][20347] Updated weights for policy 0, policy_version 772595 (0.0010) [2023-12-27 10:58:27,146][20291] KL-divergence is very high: 1532.1355 [2023-12-27 10:58:27,157][20347] Updated weights for policy 0, policy_version 772605 (0.0010) [2023-12-27 10:58:27,157][20291] KL-divergence is very high: 824.9543 [2023-12-27 10:58:27,197][20381] Updated weights for policy 1, policy_version 772162 (0.0006) [2023-12-27 10:58:27,245][20381] Updated weights for policy 1, policy_version 772172 (0.0006) [2023-12-27 10:58:27,295][20381] Updated weights for policy 1, policy_version 772182 (0.0005) [2023-12-27 10:58:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 395517952. Throughput: 0: 10299.9, 1: 10207.0. Samples: 395500048. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:27,363][19170] Avg episode reward: [(0, '921.940'), (1, '913.210')] [2023-12-27 10:58:27,804][20291] KL-divergence is very high: 1146.3865 [2023-12-27 10:58:27,817][20347] Updated weights for policy 0, policy_version 772615 (0.0010) [2023-12-27 10:58:27,844][20291] KL-divergence is very high: 486.1389 [2023-12-27 10:58:27,868][20347] Updated weights for policy 0, policy_version 772625 (0.0010) [2023-12-27 10:58:27,877][20381] Updated weights for policy 1, policy_version 772192 (0.0006) [2023-12-27 10:58:27,885][20291] KL-divergence is very high: 401.1418 [2023-12-27 10:58:27,924][20347] Updated weights for policy 0, policy_version 772635 (0.0010) [2023-12-27 10:58:27,926][20381] Updated weights for policy 1, policy_version 772202 (0.0005) [2023-12-27 10:58:27,928][20291] KL-divergence is very high: 545.8249 [2023-12-27 10:58:27,974][20381] Updated weights for policy 1, policy_version 772212 (0.0008) [2023-12-27 10:58:28,509][20347] Updated weights for policy 0, policy_version 772645 (0.0006) [2023-12-27 10:58:28,568][20347] Updated weights for policy 0, policy_version 772655 (0.0007) [2023-12-27 10:58:28,618][20347] Updated weights for policy 0, policy_version 772665 (0.0008) [2023-12-27 10:58:28,677][20381] Updated weights for policy 1, policy_version 772222 (0.0008) [2023-12-27 10:58:28,729][20381] Updated weights for policy 1, policy_version 772232 (0.0008) [2023-12-27 10:58:28,784][20381] Updated weights for policy 1, policy_version 772242 (0.0007) [2023-12-27 10:58:29,246][20347] Updated weights for policy 0, policy_version 772675 (0.0009) [2023-12-27 10:58:29,307][20347] Updated weights for policy 0, policy_version 772685 (0.0007) [2023-12-27 10:58:29,337][20291] KL-divergence is very high: 104.0929 [2023-12-27 10:58:29,368][20347] Updated weights for policy 0, policy_version 772695 (0.0007) [2023-12-27 10:58:29,523][20381] Updated weights for policy 1, policy_version 772252 (0.0008) [2023-12-27 10:58:29,586][20381] Updated weights for policy 1, policy_version 772262 (0.0007) [2023-12-27 10:58:29,647][20381] Updated weights for policy 1, policy_version 772272 (0.0008) [2023-12-27 10:58:30,006][20347] Updated weights for policy 0, policy_version 772705 (0.0007) [2023-12-27 10:58:30,068][20347] Updated weights for policy 0, policy_version 772715 (0.0007) [2023-12-27 10:58:30,126][20347] Updated weights for policy 0, policy_version 772725 (0.0008) [2023-12-27 10:58:30,187][20347] Updated weights for policy 0, policy_version 772735 (0.0007) [2023-12-27 10:58:30,325][20381] Updated weights for policy 1, policy_version 772282 (0.0009) [2023-12-27 10:58:30,388][20381] Updated weights for policy 1, policy_version 772292 (0.0006) [2023-12-27 10:58:30,437][20381] Updated weights for policy 1, policy_version 772302 (0.0008) [2023-12-27 10:58:30,495][20381] Updated weights for policy 1, policy_version 772312 (0.0006) [2023-12-27 10:58:30,772][20347] Updated weights for policy 0, policy_version 772745 (0.0006) [2023-12-27 10:58:30,818][20347] Updated weights for policy 0, policy_version 772755 (0.0005) [2023-12-27 10:58:30,878][20347] Updated weights for policy 0, policy_version 772765 (0.0005) [2023-12-27 10:58:31,248][20381] Updated weights for policy 1, policy_version 772322 (0.0008) [2023-12-27 10:58:31,308][20381] Updated weights for policy 1, policy_version 772332 (0.0005) [2023-12-27 10:58:31,366][20381] Updated weights for policy 1, policy_version 772342 (0.0007) [2023-12-27 10:58:31,520][20347] Updated weights for policy 0, policy_version 772775 (0.0007) [2023-12-27 10:58:31,577][20347] Updated weights for policy 0, policy_version 772785 (0.0009) [2023-12-27 10:58:31,644][20347] Updated weights for policy 0, policy_version 772795 (0.0009) [2023-12-27 10:58:32,045][20381] Updated weights for policy 1, policy_version 772352 (0.0007) [2023-12-27 10:58:32,107][20381] Updated weights for policy 1, policy_version 772362 (0.0009) [2023-12-27 10:58:32,169][20381] Updated weights for policy 1, policy_version 772372 (0.0008) [2023-12-27 10:58:32,255][20347] Updated weights for policy 0, policy_version 772805 (0.0007) [2023-12-27 10:58:32,310][20347] Updated weights for policy 0, policy_version 772815 (0.0008) [2023-12-27 10:58:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 395616256. Throughput: 0: 10378.0, 1: 10244.0. Samples: 395628324. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:32,363][19170] Avg episode reward: [(0, '853.710'), (1, '900.090')] [2023-12-27 10:58:32,366][20347] Updated weights for policy 0, policy_version 772825 (0.0008) [2023-12-27 10:58:32,871][20381] Updated weights for policy 1, policy_version 772382 (0.0006) [2023-12-27 10:58:32,915][20381] Updated weights for policy 1, policy_version 772392 (0.0007) [2023-12-27 10:58:32,965][20381] Updated weights for policy 1, policy_version 772402 (0.0005) [2023-12-27 10:58:33,060][20347] Updated weights for policy 0, policy_version 772835 (0.0008) [2023-12-27 10:58:33,117][20347] Updated weights for policy 0, policy_version 772845 (0.0009) [2023-12-27 10:58:33,176][20347] Updated weights for policy 0, policy_version 772855 (0.0009) [2023-12-27 10:58:33,554][20381] Updated weights for policy 1, policy_version 772412 (0.0006) [2023-12-27 10:58:33,598][20381] Updated weights for policy 1, policy_version 772422 (0.0007) [2023-12-27 10:58:33,645][20381] Updated weights for policy 1, policy_version 772432 (0.0007) [2023-12-27 10:58:33,787][20347] Updated weights for policy 0, policy_version 772865 (0.0010) [2023-12-27 10:58:33,842][20347] Updated weights for policy 0, policy_version 772875 (0.0008) [2023-12-27 10:58:33,895][20347] Updated weights for policy 0, policy_version 772885 (0.0007) [2023-12-27 10:58:33,946][20347] Updated weights for policy 0, policy_version 772895 (0.0007) [2023-12-27 10:58:34,373][20381] Updated weights for policy 1, policy_version 772442 (0.0007) [2023-12-27 10:58:34,429][20381] Updated weights for policy 1, policy_version 772452 (0.0008) [2023-12-27 10:58:34,488][20381] Updated weights for policy 1, policy_version 772462 (0.0007) [2023-12-27 10:58:34,549][20381] Updated weights for policy 1, policy_version 772472 (0.0008) [2023-12-27 10:58:34,644][20347] Updated weights for policy 0, policy_version 772905 (0.0007) [2023-12-27 10:58:34,707][20347] Updated weights for policy 0, policy_version 772915 (0.0007) [2023-12-27 10:58:34,766][20347] Updated weights for policy 0, policy_version 772925 (0.0007) [2023-12-27 10:58:35,220][20381] Updated weights for policy 1, policy_version 772482 (0.0008) [2023-12-27 10:58:35,269][20381] Updated weights for policy 1, policy_version 772492 (0.0008) [2023-12-27 10:58:35,315][20381] Updated weights for policy 1, policy_version 772502 (0.0008) [2023-12-27 10:58:35,469][20347] Updated weights for policy 0, policy_version 772935 (0.0007) [2023-12-27 10:58:35,525][20347] Updated weights for policy 0, policy_version 772945 (0.0008) [2023-12-27 10:58:35,586][20347] Updated weights for policy 0, policy_version 772955 (0.0008) [2023-12-27 10:58:36,051][20381] Updated weights for policy 1, policy_version 772512 (0.0009) [2023-12-27 10:58:36,100][20381] Updated weights for policy 1, policy_version 772522 (0.0008) [2023-12-27 10:58:36,151][20347] Updated weights for policy 0, policy_version 772965 (0.0009) [2023-12-27 10:58:36,160][20381] Updated weights for policy 1, policy_version 772532 (0.0006) [2023-12-27 10:58:36,208][20347] Updated weights for policy 0, policy_version 772975 (0.0010) [2023-12-27 10:58:36,255][20347] Updated weights for policy 0, policy_version 772985 (0.0009) [2023-12-27 10:58:36,838][20381] Updated weights for policy 1, policy_version 772542 (0.0005) [2023-12-27 10:58:36,884][20381] Updated weights for policy 1, policy_version 772552 (0.0005) [2023-12-27 10:58:36,940][20381] Updated weights for policy 1, policy_version 772562 (0.0005) [2023-12-27 10:58:36,967][20347] Updated weights for policy 0, policy_version 772995 (0.0008) [2023-12-27 10:58:37,016][20347] Updated weights for policy 0, policy_version 773005 (0.0010) [2023-12-27 10:58:37,064][20347] Updated weights for policy 0, policy_version 773015 (0.0010) [2023-12-27 10:58:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 395722752. Throughput: 0: 10366.2, 1: 10236.5. Samples: 395690620. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:37,364][19170] Avg episode reward: [(0, '902.510'), (1, '921.410')] [2023-12-27 10:58:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000772568_197804032.pth... [2023-12-27 10:58:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000773024_197918720.pth... [2023-12-27 10:58:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000771384_197500928.pth [2023-12-27 10:58:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000771808_197607424.pth [2023-12-27 10:58:37,568][20381] Updated weights for policy 1, policy_version 772572 (0.0006) [2023-12-27 10:58:37,627][20381] Updated weights for policy 1, policy_version 772582 (0.0007) [2023-12-27 10:58:37,673][20381] Updated weights for policy 1, policy_version 772592 (0.0005) [2023-12-27 10:58:37,762][20347] Updated weights for policy 0, policy_version 773025 (0.0010) [2023-12-27 10:58:37,813][20347] Updated weights for policy 0, policy_version 773035 (0.0010) [2023-12-27 10:58:37,867][20347] Updated weights for policy 0, policy_version 773045 (0.0010) [2023-12-27 10:58:37,916][20347] Updated weights for policy 0, policy_version 773055 (0.0005) [2023-12-27 10:58:38,324][20381] Updated weights for policy 1, policy_version 772602 (0.0007) [2023-12-27 10:58:38,390][20381] Updated weights for policy 1, policy_version 772612 (0.0008) [2023-12-27 10:58:38,451][20381] Updated weights for policy 1, policy_version 772622 (0.0008) [2023-12-27 10:58:38,509][20381] Updated weights for policy 1, policy_version 772632 (0.0006) [2023-12-27 10:58:38,591][20347] Updated weights for policy 0, policy_version 773065 (0.0010) [2023-12-27 10:58:38,640][20347] Updated weights for policy 0, policy_version 773075 (0.0010) [2023-12-27 10:58:38,696][20347] Updated weights for policy 0, policy_version 773085 (0.0010) [2023-12-27 10:58:39,191][20381] Updated weights for policy 1, policy_version 772642 (0.0006) [2023-12-27 10:58:39,252][20381] Updated weights for policy 1, policy_version 772652 (0.0007) [2023-12-27 10:58:39,314][20381] Updated weights for policy 1, policy_version 772662 (0.0008) [2023-12-27 10:58:39,426][20347] Updated weights for policy 0, policy_version 773095 (0.0011) [2023-12-27 10:58:39,488][20347] Updated weights for policy 0, policy_version 773105 (0.0010) [2023-12-27 10:58:39,547][20347] Updated weights for policy 0, policy_version 773115 (0.0010) [2023-12-27 10:58:39,985][20381] Updated weights for policy 1, policy_version 772672 (0.0010) [2023-12-27 10:58:40,050][20381] Updated weights for policy 1, policy_version 772682 (0.0010) [2023-12-27 10:58:40,115][20381] Updated weights for policy 1, policy_version 772692 (0.0009) [2023-12-27 10:58:40,215][20347] Updated weights for policy 0, policy_version 773125 (0.0010) [2023-12-27 10:58:40,281][20347] Updated weights for policy 0, policy_version 773135 (0.0010) [2023-12-27 10:58:40,349][20347] Updated weights for policy 0, policy_version 773145 (0.0010) [2023-12-27 10:58:40,802][20381] Updated weights for policy 1, policy_version 772702 (0.0010) [2023-12-27 10:58:40,852][20381] Updated weights for policy 1, policy_version 772712 (0.0011) [2023-12-27 10:58:40,901][20381] Updated weights for policy 1, policy_version 772722 (0.0010) [2023-12-27 10:58:40,943][20347] Updated weights for policy 0, policy_version 773155 (0.0010) [2023-12-27 10:58:40,999][20347] Updated weights for policy 0, policy_version 773165 (0.0010) [2023-12-27 10:58:41,067][20347] Updated weights for policy 0, policy_version 773175 (0.0009) [2023-12-27 10:58:41,666][20381] Updated weights for policy 1, policy_version 772732 (0.0010) [2023-12-27 10:58:41,734][20381] Updated weights for policy 1, policy_version 772742 (0.0008) [2023-12-27 10:58:41,793][20381] Updated weights for policy 1, policy_version 772752 (0.0008) [2023-12-27 10:58:41,816][20347] Updated weights for policy 0, policy_version 773185 (0.0010) [2023-12-27 10:58:41,870][20347] Updated weights for policy 0, policy_version 773195 (0.0007) [2023-12-27 10:58:41,921][20347] Updated weights for policy 0, policy_version 773205 (0.0005) [2023-12-27 10:58:41,977][20347] Updated weights for policy 0, policy_version 773215 (0.0005) [2023-12-27 10:58:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 395821056. Throughput: 0: 10361.8, 1: 10213.3. Samples: 395813812. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:42,363][19170] Avg episode reward: [(0, '941.240'), (1, '930.870')] [2023-12-27 10:58:42,519][20381] Updated weights for policy 1, policy_version 772762 (0.0008) [2023-12-27 10:58:42,577][20381] Updated weights for policy 1, policy_version 772772 (0.0006) [2023-12-27 10:58:42,634][20381] Updated weights for policy 1, policy_version 772782 (0.0008) [2023-12-27 10:58:42,684][20347] Updated weights for policy 0, policy_version 773225 (0.0008) [2023-12-27 10:58:42,687][20381] Updated weights for policy 1, policy_version 772792 (0.0006) [2023-12-27 10:58:42,738][20347] Updated weights for policy 0, policy_version 773235 (0.0007) [2023-12-27 10:58:42,797][20347] Updated weights for policy 0, policy_version 773245 (0.0006) [2023-12-27 10:58:43,381][20381] Updated weights for policy 1, policy_version 772802 (0.0011) [2023-12-27 10:58:43,426][20381] Updated weights for policy 1, policy_version 772812 (0.0010) [2023-12-27 10:58:43,475][20381] Updated weights for policy 1, policy_version 772822 (0.0010) [2023-12-27 10:58:43,506][20347] Updated weights for policy 0, policy_version 773255 (0.0010) [2023-12-27 10:58:43,563][20347] Updated weights for policy 0, policy_version 773265 (0.0010) [2023-12-27 10:58:43,615][20347] Updated weights for policy 0, policy_version 773275 (0.0011) [2023-12-27 10:58:44,188][20381] Updated weights for policy 1, policy_version 772832 (0.0010) [2023-12-27 10:58:44,248][20381] Updated weights for policy 1, policy_version 772842 (0.0010) [2023-12-27 10:58:44,301][20381] Updated weights for policy 1, policy_version 772852 (0.0011) [2023-12-27 10:58:44,301][20347] Updated weights for policy 0, policy_version 773285 (0.0010) [2023-12-27 10:58:44,355][20347] Updated weights for policy 0, policy_version 773295 (0.0007) [2023-12-27 10:58:44,418][20347] Updated weights for policy 0, policy_version 773305 (0.0005) [2023-12-27 10:58:45,016][20381] Updated weights for policy 1, policy_version 772862 (0.0010) [2023-12-27 10:58:45,060][20347] Updated weights for policy 0, policy_version 773315 (0.0007) [2023-12-27 10:58:45,085][20381] Updated weights for policy 1, policy_version 772872 (0.0011) [2023-12-27 10:58:45,126][20347] Updated weights for policy 0, policy_version 773325 (0.0011) [2023-12-27 10:58:45,139][20381] Updated weights for policy 1, policy_version 772882 (0.0009) [2023-12-27 10:58:45,186][20347] Updated weights for policy 0, policy_version 773335 (0.0011) [2023-12-27 10:58:45,822][20347] Updated weights for policy 0, policy_version 773345 (0.0010) [2023-12-27 10:58:45,838][20381] Updated weights for policy 1, policy_version 772892 (0.0008) [2023-12-27 10:58:45,872][20347] Updated weights for policy 0, policy_version 773355 (0.0005) [2023-12-27 10:58:45,886][20381] Updated weights for policy 1, policy_version 772902 (0.0005) [2023-12-27 10:58:45,926][20347] Updated weights for policy 0, policy_version 773365 (0.0005) [2023-12-27 10:58:45,934][20381] Updated weights for policy 1, policy_version 772912 (0.0005) [2023-12-27 10:58:45,987][20347] Updated weights for policy 0, policy_version 773375 (0.0007) [2023-12-27 10:58:46,515][20381] Updated weights for policy 1, policy_version 772922 (0.0006) [2023-12-27 10:58:46,575][20381] Updated weights for policy 1, policy_version 772932 (0.0005) [2023-12-27 10:58:46,634][20381] Updated weights for policy 1, policy_version 772942 (0.0006) [2023-12-27 10:58:46,636][20347] Updated weights for policy 0, policy_version 773385 (0.0010) [2023-12-27 10:58:46,680][20381] Updated weights for policy 1, policy_version 772952 (0.0007) [2023-12-27 10:58:46,690][20347] Updated weights for policy 0, policy_version 773395 (0.0010) [2023-12-27 10:58:46,747][20347] Updated weights for policy 0, policy_version 773405 (0.0010) [2023-12-27 10:58:47,332][20381] Updated weights for policy 1, policy_version 772962 (0.0008) [2023-12-27 10:58:47,350][20347] Updated weights for policy 0, policy_version 773415 (0.0007) [2023-12-27 10:58:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 395919360. Throughput: 0: 10364.1, 1: 10248.9. Samples: 395936296. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:47,363][19170] Avg episode reward: [(0, '941.290'), (1, '932.000')] [2023-12-27 10:58:47,385][20381] Updated weights for policy 1, policy_version 772972 (0.0007) [2023-12-27 10:58:47,395][20347] Updated weights for policy 0, policy_version 773425 (0.0005) [2023-12-27 10:58:47,439][20381] Updated weights for policy 1, policy_version 772982 (0.0007) [2023-12-27 10:58:47,442][20347] Updated weights for policy 0, policy_version 773435 (0.0006) [2023-12-27 10:58:48,069][20347] Updated weights for policy 0, policy_version 773445 (0.0005) [2023-12-27 10:58:48,125][20347] Updated weights for policy 0, policy_version 773455 (0.0005) [2023-12-27 10:58:48,157][20381] Updated weights for policy 1, policy_version 772992 (0.0008) [2023-12-27 10:58:48,173][20347] Updated weights for policy 0, policy_version 773465 (0.0005) [2023-12-27 10:58:48,222][20381] Updated weights for policy 1, policy_version 773002 (0.0007) [2023-12-27 10:58:48,273][20381] Updated weights for policy 1, policy_version 773012 (0.0005) [2023-12-27 10:58:48,851][20347] Updated weights for policy 0, policy_version 773475 (0.0006) [2023-12-27 10:58:48,908][20347] Updated weights for policy 0, policy_version 773485 (0.0007) [2023-12-27 10:58:48,956][20381] Updated weights for policy 1, policy_version 773022 (0.0005) [2023-12-27 10:58:48,961][20347] Updated weights for policy 0, policy_version 773495 (0.0006) [2023-12-27 10:58:49,025][20381] Updated weights for policy 1, policy_version 773032 (0.0007) [2023-12-27 10:58:49,071][20381] Updated weights for policy 1, policy_version 773042 (0.0008) [2023-12-27 10:58:49,677][20347] Updated weights for policy 0, policy_version 773505 (0.0006) [2023-12-27 10:58:49,740][20347] Updated weights for policy 0, policy_version 773515 (0.0010) [2023-12-27 10:58:49,764][20381] Updated weights for policy 1, policy_version 773052 (0.0007) [2023-12-27 10:58:49,796][20347] Updated weights for policy 0, policy_version 773525 (0.0010) [2023-12-27 10:58:49,826][20381] Updated weights for policy 1, policy_version 773062 (0.0006) [2023-12-27 10:58:49,866][20347] Updated weights for policy 0, policy_version 773535 (0.0010) [2023-12-27 10:58:49,887][20381] Updated weights for policy 1, policy_version 773072 (0.0007) [2023-12-27 10:58:50,508][20381] Updated weights for policy 1, policy_version 773082 (0.0007) [2023-12-27 10:58:50,538][20347] Updated weights for policy 0, policy_version 773545 (0.0006) [2023-12-27 10:58:50,560][20381] Updated weights for policy 1, policy_version 773092 (0.0008) [2023-12-27 10:58:50,601][20347] Updated weights for policy 0, policy_version 773555 (0.0007) [2023-12-27 10:58:50,627][20381] Updated weights for policy 1, policy_version 773102 (0.0007) [2023-12-27 10:58:50,655][20347] Updated weights for policy 0, policy_version 773565 (0.0007) [2023-12-27 10:58:50,690][20381] Updated weights for policy 1, policy_version 773112 (0.0007) [2023-12-27 10:58:51,379][20347] Updated weights for policy 0, policy_version 773575 (0.0007) [2023-12-27 10:58:51,412][20291] KL-divergence is very high: 110.3702 [2023-12-27 10:58:51,416][20381] Updated weights for policy 1, policy_version 773122 (0.0008) [2023-12-27 10:58:51,418][20291] KL-divergence is very high: 118.7040 [2023-12-27 10:58:51,441][20347] Updated weights for policy 0, policy_version 773585 (0.0007) [2023-12-27 10:58:51,459][20291] KL-divergence is very high: 151.5912 [2023-12-27 10:58:51,466][20291] KL-divergence is very high: 129.3162 [2023-12-27 10:58:51,475][20381] Updated weights for policy 1, policy_version 773132 (0.0008) [2023-12-27 10:58:51,499][20347] Updated weights for policy 0, policy_version 773595 (0.0007) [2023-12-27 10:58:51,535][20381] Updated weights for policy 1, policy_version 773142 (0.0008) [2023-12-27 10:58:52,127][20347] Updated weights for policy 0, policy_version 773605 (0.0006) [2023-12-27 10:58:52,185][20347] Updated weights for policy 0, policy_version 773615 (0.0008) [2023-12-27 10:58:52,246][20347] Updated weights for policy 0, policy_version 773625 (0.0006) [2023-12-27 10:58:52,315][20381] Updated weights for policy 1, policy_version 773152 (0.0007) [2023-12-27 10:58:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 396025856. Throughput: 0: 10348.1, 1: 10256.2. Samples: 395998076. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:52,363][19170] Avg episode reward: [(0, '931.700'), (1, '922.280')] [2023-12-27 10:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000773632_198074368.pth... [2023-12-27 10:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000772384_197754880.pth [2023-12-27 10:58:52,375][20381] Updated weights for policy 1, policy_version 773162 (0.0008) [2023-12-27 10:58:52,444][20381] Updated weights for policy 1, policy_version 773172 (0.0008) [2023-12-27 10:58:52,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000773176_197959680.pth... [2023-12-27 10:58:52,472][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000771960_197648384.pth [2023-12-27 10:58:52,868][20347] Updated weights for policy 0, policy_version 773635 (0.0007) [2023-12-27 10:58:52,919][20347] Updated weights for policy 0, policy_version 773645 (0.0006) [2023-12-27 10:58:52,970][20347] Updated weights for policy 0, policy_version 773655 (0.0009) [2023-12-27 10:58:53,098][20381] Updated weights for policy 1, policy_version 773182 (0.0008) [2023-12-27 10:58:53,162][20381] Updated weights for policy 1, policy_version 773192 (0.0007) [2023-12-27 10:58:53,225][20381] Updated weights for policy 1, policy_version 773202 (0.0008) [2023-12-27 10:58:53,608][20347] Updated weights for policy 0, policy_version 773665 (0.0010) [2023-12-27 10:58:53,664][20347] Updated weights for policy 0, policy_version 773675 (0.0006) [2023-12-27 10:58:53,727][20347] Updated weights for policy 0, policy_version 773685 (0.0005) [2023-12-27 10:58:53,793][20347] Updated weights for policy 0, policy_version 773695 (0.0006) [2023-12-27 10:58:53,957][20381] Updated weights for policy 1, policy_version 773212 (0.0007) [2023-12-27 10:58:54,011][20381] Updated weights for policy 1, policy_version 773222 (0.0008) [2023-12-27 10:58:54,068][20381] Updated weights for policy 1, policy_version 773232 (0.0009) [2023-12-27 10:58:54,325][20347] Updated weights for policy 0, policy_version 773705 (0.0008) [2023-12-27 10:58:54,390][20347] Updated weights for policy 0, policy_version 773715 (0.0008) [2023-12-27 10:58:54,444][20347] Updated weights for policy 0, policy_version 773725 (0.0008) [2023-12-27 10:58:54,822][20381] Updated weights for policy 1, policy_version 773242 (0.0008) [2023-12-27 10:58:54,890][20381] Updated weights for policy 1, policy_version 773252 (0.0006) [2023-12-27 10:58:54,947][20381] Updated weights for policy 1, policy_version 773262 (0.0007) [2023-12-27 10:58:55,007][20381] Updated weights for policy 1, policy_version 773272 (0.0008) [2023-12-27 10:58:55,058][20347] Updated weights for policy 0, policy_version 773735 (0.0009) [2023-12-27 10:58:55,109][20347] Updated weights for policy 0, policy_version 773745 (0.0010) [2023-12-27 10:58:55,158][20347] Updated weights for policy 0, policy_version 773755 (0.0010) [2023-12-27 10:58:55,630][20381] Updated weights for policy 1, policy_version 773282 (0.0008) [2023-12-27 10:58:55,683][20381] Updated weights for policy 1, policy_version 773292 (0.0008) [2023-12-27 10:58:55,737][20381] Updated weights for policy 1, policy_version 773302 (0.0009) [2023-12-27 10:58:55,825][20347] Updated weights for policy 0, policy_version 773765 (0.0008) [2023-12-27 10:58:55,870][20347] Updated weights for policy 0, policy_version 773775 (0.0007) [2023-12-27 10:58:55,870][20291] KL-divergence is very high: 149.5427 [2023-12-27 10:58:55,908][20291] KL-divergence is very high: 206.5257 [2023-12-27 10:58:55,917][20347] Updated weights for policy 0, policy_version 773785 (0.0008) [2023-12-27 10:58:55,948][20291] KL-divergence is very high: 133.6876 [2023-12-27 10:58:56,284][20381] Updated weights for policy 1, policy_version 773312 (0.0006) [2023-12-27 10:58:56,339][20381] Updated weights for policy 1, policy_version 773322 (0.0010) [2023-12-27 10:58:56,397][20381] Updated weights for policy 1, policy_version 773332 (0.0006) [2023-12-27 10:58:56,608][20347] Updated weights for policy 0, policy_version 773795 (0.0008) [2023-12-27 10:58:56,666][20291] KL-divergence is very high: 117.2981 [2023-12-27 10:58:56,667][20347] Updated weights for policy 0, policy_version 773805 (0.0010) [2023-12-27 10:58:56,720][20347] Updated weights for policy 0, policy_version 773815 (0.0010) [2023-12-27 10:58:56,937][20381] Updated weights for policy 1, policy_version 773342 (0.0005) [2023-12-27 10:58:56,989][20381] Updated weights for policy 1, policy_version 773352 (0.0005) [2023-12-27 10:58:57,037][20381] Updated weights for policy 1, policy_version 773362 (0.0005) [2023-12-27 10:58:57,356][20347] Updated weights for policy 0, policy_version 773825 (0.0010) [2023-12-27 10:58:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 396132352. Throughput: 0: 10372.5, 1: 10269.0. Samples: 396123068. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:58:57,363][19170] Avg episode reward: [(0, '922.070'), (1, '922.120')] [2023-12-27 10:58:57,407][20347] Updated weights for policy 0, policy_version 773835 (0.0005) [2023-12-27 10:58:57,457][20347] Updated weights for policy 0, policy_version 773845 (0.0005) [2023-12-27 10:58:57,503][20347] Updated weights for policy 0, policy_version 773855 (0.0005) [2023-12-27 10:58:57,620][20381] Updated weights for policy 1, policy_version 773372 (0.0005) [2023-12-27 10:58:57,669][20381] Updated weights for policy 1, policy_version 773382 (0.0005) [2023-12-27 10:58:57,720][20381] Updated weights for policy 1, policy_version 773392 (0.0005) [2023-12-27 10:58:58,115][20347] Updated weights for policy 0, policy_version 773865 (0.0005) [2023-12-27 10:58:58,169][20347] Updated weights for policy 0, policy_version 773875 (0.0009) [2023-12-27 10:58:58,229][20347] Updated weights for policy 0, policy_version 773885 (0.0008) [2023-12-27 10:58:58,452][20381] Updated weights for policy 1, policy_version 773402 (0.0006) [2023-12-27 10:58:58,520][20381] Updated weights for policy 1, policy_version 773412 (0.0007) [2023-12-27 10:58:58,582][20381] Updated weights for policy 1, policy_version 773422 (0.0008) [2023-12-27 10:58:58,651][20381] Updated weights for policy 1, policy_version 773432 (0.0008) [2023-12-27 10:58:58,980][20347] Updated weights for policy 0, policy_version 773895 (0.0009) [2023-12-27 10:58:59,029][20347] Updated weights for policy 0, policy_version 773905 (0.0009) [2023-12-27 10:58:59,083][20347] Updated weights for policy 0, policy_version 773915 (0.0010) [2023-12-27 10:58:59,401][20381] Updated weights for policy 1, policy_version 773442 (0.0010) [2023-12-27 10:58:59,454][20381] Updated weights for policy 1, policy_version 773452 (0.0006) [2023-12-27 10:58:59,506][20381] Updated weights for policy 1, policy_version 773462 (0.0005) [2023-12-27 10:58:59,822][20347] Updated weights for policy 0, policy_version 773925 (0.0010) [2023-12-27 10:58:59,881][20347] Updated weights for policy 0, policy_version 773935 (0.0008) [2023-12-27 10:58:59,943][20347] Updated weights for policy 0, policy_version 773945 (0.0008) [2023-12-27 10:59:00,142][20381] Updated weights for policy 1, policy_version 773472 (0.0007) [2023-12-27 10:59:00,198][20381] Updated weights for policy 1, policy_version 773482 (0.0007) [2023-12-27 10:59:00,256][20381] Updated weights for policy 1, policy_version 773492 (0.0005) [2023-12-27 10:59:00,631][20347] Updated weights for policy 0, policy_version 773955 (0.0007) [2023-12-27 10:59:00,693][20347] Updated weights for policy 0, policy_version 773965 (0.0008) [2023-12-27 10:59:00,747][20347] Updated weights for policy 0, policy_version 773975 (0.0007) [2023-12-27 10:59:00,877][20381] Updated weights for policy 1, policy_version 773502 (0.0007) [2023-12-27 10:59:00,923][20381] Updated weights for policy 1, policy_version 773512 (0.0010) [2023-12-27 10:59:00,980][20381] Updated weights for policy 1, policy_version 773522 (0.0007) [2023-12-27 10:59:01,377][20347] Updated weights for policy 0, policy_version 773985 (0.0005) [2023-12-27 10:59:01,437][20347] Updated weights for policy 0, policy_version 773995 (0.0008) [2023-12-27 10:59:01,502][20347] Updated weights for policy 0, policy_version 774005 (0.0007) [2023-12-27 10:59:01,559][20347] Updated weights for policy 0, policy_version 774015 (0.0007) [2023-12-27 10:59:01,671][20381] Updated weights for policy 1, policy_version 773532 (0.0008) [2023-12-27 10:59:01,737][20381] Updated weights for policy 1, policy_version 773542 (0.0008) [2023-12-27 10:59:01,797][20381] Updated weights for policy 1, policy_version 773552 (0.0007) [2023-12-27 10:59:02,316][20347] Updated weights for policy 0, policy_version 774025 (0.0007) [2023-12-27 10:59:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 396230656. Throughput: 0: 10433.8, 1: 10257.1. Samples: 396246776. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:59:02,363][19170] Avg episode reward: [(0, '921.920'), (1, '951.380')] [2023-12-27 10:59:02,373][20347] Updated weights for policy 0, policy_version 774035 (0.0008) [2023-12-27 10:59:02,435][20347] Updated weights for policy 0, policy_version 774045 (0.0008) [2023-12-27 10:59:02,478][20381] Updated weights for policy 1, policy_version 773562 (0.0006) [2023-12-27 10:59:02,535][20381] Updated weights for policy 1, policy_version 773572 (0.0008) [2023-12-27 10:59:02,592][20381] Updated weights for policy 1, policy_version 773582 (0.0006) [2023-12-27 10:59:02,656][20381] Updated weights for policy 1, policy_version 773592 (0.0007) [2023-12-27 10:59:03,089][20347] Updated weights for policy 0, policy_version 774055 (0.0007) [2023-12-27 10:59:03,137][20347] Updated weights for policy 0, policy_version 774065 (0.0007) [2023-12-27 10:59:03,196][20347] Updated weights for policy 0, policy_version 774075 (0.0005) [2023-12-27 10:59:03,346][20381] Updated weights for policy 1, policy_version 773602 (0.0008) [2023-12-27 10:59:03,396][20381] Updated weights for policy 1, policy_version 773612 (0.0010) [2023-12-27 10:59:03,456][20381] Updated weights for policy 1, policy_version 773622 (0.0006) [2023-12-27 10:59:03,809][20347] Updated weights for policy 0, policy_version 774085 (0.0007) [2023-12-27 10:59:03,869][20347] Updated weights for policy 0, policy_version 774095 (0.0008) [2023-12-27 10:59:03,931][20347] Updated weights for policy 0, policy_version 774105 (0.0007) [2023-12-27 10:59:04,100][20381] Updated weights for policy 1, policy_version 773632 (0.0008) [2023-12-27 10:59:04,164][20381] Updated weights for policy 1, policy_version 773642 (0.0008) [2023-12-27 10:59:04,220][20381] Updated weights for policy 1, policy_version 773652 (0.0007) [2023-12-27 10:59:04,599][20347] Updated weights for policy 0, policy_version 774115 (0.0008) [2023-12-27 10:59:04,661][20347] Updated weights for policy 0, policy_version 774125 (0.0006) [2023-12-27 10:59:04,730][20347] Updated weights for policy 0, policy_version 774135 (0.0006) [2023-12-27 10:59:04,918][20381] Updated weights for policy 1, policy_version 773662 (0.0006) [2023-12-27 10:59:04,983][20381] Updated weights for policy 1, policy_version 773672 (0.0005) [2023-12-27 10:59:05,044][20381] Updated weights for policy 1, policy_version 773682 (0.0009) [2023-12-27 10:59:05,374][20347] Updated weights for policy 0, policy_version 774145 (0.0006) [2023-12-27 10:59:05,432][20347] Updated weights for policy 0, policy_version 774155 (0.0008) [2023-12-27 10:59:05,486][20347] Updated weights for policy 0, policy_version 774165 (0.0008) [2023-12-27 10:59:05,537][20347] Updated weights for policy 0, policy_version 774175 (0.0008) [2023-12-27 10:59:05,589][20381] Updated weights for policy 1, policy_version 773692 (0.0009) [2023-12-27 10:59:05,639][20381] Updated weights for policy 1, policy_version 773702 (0.0006) [2023-12-27 10:59:05,689][20381] Updated weights for policy 1, policy_version 773712 (0.0007) [2023-12-27 10:59:06,240][20347] Updated weights for policy 0, policy_version 774185 (0.0010) [2023-12-27 10:59:06,302][20347] Updated weights for policy 0, policy_version 774195 (0.0011) [2023-12-27 10:59:06,369][20381] Updated weights for policy 1, policy_version 773722 (0.0011) [2023-12-27 10:59:06,371][20347] Updated weights for policy 0, policy_version 774205 (0.0009) [2023-12-27 10:59:06,439][20381] Updated weights for policy 1, policy_version 773732 (0.0011) [2023-12-27 10:59:06,505][20381] Updated weights for policy 1, policy_version 773742 (0.0011) [2023-12-27 10:59:06,571][20381] Updated weights for policy 1, policy_version 773752 (0.0011) [2023-12-27 10:59:07,023][20347] Updated weights for policy 0, policy_version 774215 (0.0009) [2023-12-27 10:59:07,078][20347] Updated weights for policy 0, policy_version 774225 (0.0010) [2023-12-27 10:59:07,140][20347] Updated weights for policy 0, policy_version 774235 (0.0010) [2023-12-27 10:59:07,238][20381] Updated weights for policy 1, policy_version 773762 (0.0011) [2023-12-27 10:59:07,300][20381] Updated weights for policy 1, policy_version 773772 (0.0011) [2023-12-27 10:59:07,357][20381] Updated weights for policy 1, policy_version 773782 (0.0010) [2023-12-27 10:59:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 396337152. Throughput: 0: 10472.6, 1: 10274.5. Samples: 396309508. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:59:07,363][19170] Avg episode reward: [(0, '941.410'), (1, '990.280')] [2023-12-27 10:59:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000774240_198230016.pth... [2023-12-27 10:59:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000773784_198115328.pth... [2023-12-27 10:59:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000773024_197918720.pth [2023-12-27 10:59:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000772568_197804032.pth [2023-12-27 10:59:07,753][20347] Updated weights for policy 0, policy_version 774245 (0.0010) [2023-12-27 10:59:07,798][20347] Updated weights for policy 0, policy_version 774255 (0.0010) [2023-12-27 10:59:07,845][20347] Updated weights for policy 0, policy_version 774265 (0.0007) [2023-12-27 10:59:08,053][20381] Updated weights for policy 1, policy_version 773792 (0.0008) [2023-12-27 10:59:08,099][20381] Updated weights for policy 1, policy_version 773802 (0.0010) [2023-12-27 10:59:08,145][20381] Updated weights for policy 1, policy_version 773812 (0.0010) [2023-12-27 10:59:08,489][20347] Updated weights for policy 0, policy_version 774275 (0.0006) [2023-12-27 10:59:08,548][20347] Updated weights for policy 0, policy_version 774285 (0.0008) [2023-12-27 10:59:08,604][20347] Updated weights for policy 0, policy_version 774295 (0.0008) [2023-12-27 10:59:08,856][20381] Updated weights for policy 1, policy_version 773822 (0.0011) [2023-12-27 10:59:08,917][20381] Updated weights for policy 1, policy_version 773832 (0.0006) [2023-12-27 10:59:08,981][20381] Updated weights for policy 1, policy_version 773842 (0.0009) [2023-12-27 10:59:09,295][20347] Updated weights for policy 0, policy_version 774305 (0.0008) [2023-12-27 10:59:09,362][20347] Updated weights for policy 0, policy_version 774315 (0.0009) [2023-12-27 10:59:09,433][20347] Updated weights for policy 0, policy_version 774325 (0.0008) [2023-12-27 10:59:09,493][20347] Updated weights for policy 0, policy_version 774335 (0.0008) [2023-12-27 10:59:09,652][20381] Updated weights for policy 1, policy_version 773852 (0.0009) [2023-12-27 10:59:09,712][20381] Updated weights for policy 1, policy_version 773862 (0.0008) [2023-12-27 10:59:09,773][20381] Updated weights for policy 1, policy_version 773872 (0.0008) [2023-12-27 10:59:10,203][20347] Updated weights for policy 0, policy_version 774345 (0.0010) [2023-12-27 10:59:10,264][20347] Updated weights for policy 0, policy_version 774355 (0.0007) [2023-12-27 10:59:10,322][20347] Updated weights for policy 0, policy_version 774365 (0.0005) [2023-12-27 10:59:10,529][20381] Updated weights for policy 1, policy_version 773882 (0.0007) [2023-12-27 10:59:10,595][20381] Updated weights for policy 1, policy_version 773892 (0.0008) [2023-12-27 10:59:10,650][20381] Updated weights for policy 1, policy_version 773902 (0.0008) [2023-12-27 10:59:10,711][20381] Updated weights for policy 1, policy_version 773912 (0.0008) [2023-12-27 10:59:10,939][20347] Updated weights for policy 0, policy_version 774375 (0.0008) [2023-12-27 10:59:11,009][20347] Updated weights for policy 0, policy_version 774385 (0.0005) [2023-12-27 10:59:11,073][20347] Updated weights for policy 0, policy_version 774395 (0.0009) [2023-12-27 10:59:11,379][20381] Updated weights for policy 1, policy_version 773922 (0.0008) [2023-12-27 10:59:11,448][20381] Updated weights for policy 1, policy_version 773932 (0.0008) [2023-12-27 10:59:11,509][20381] Updated weights for policy 1, policy_version 773942 (0.0006) [2023-12-27 10:59:11,793][20347] Updated weights for policy 0, policy_version 774405 (0.0009) [2023-12-27 10:59:11,853][20347] Updated weights for policy 0, policy_version 774415 (0.0007) [2023-12-27 10:59:11,915][20347] Updated weights for policy 0, policy_version 774425 (0.0008) [2023-12-27 10:59:12,211][20381] Updated weights for policy 1, policy_version 773952 (0.0006) [2023-12-27 10:59:12,278][20381] Updated weights for policy 1, policy_version 773962 (0.0006) [2023-12-27 10:59:12,338][20381] Updated weights for policy 1, policy_version 773972 (0.0007) [2023-12-27 10:59:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 396443648. Throughput: 0: 10471.4, 1: 10242.1. Samples: 396432152. Policy #0 lag: (min: 23.0, avg: 30.3, max: 55.0) [2023-12-27 10:59:12,363][19170] Avg episode reward: [(0, '970.760'), (1, '951.250')] [2023-12-27 10:59:12,666][20347] Updated weights for policy 0, policy_version 774435 (0.0008) [2023-12-27 10:59:12,726][20347] Updated weights for policy 0, policy_version 774445 (0.0007) [2023-12-27 10:59:12,786][20347] Updated weights for policy 0, policy_version 774455 (0.0008) [2023-12-27 10:59:12,993][20381] Updated weights for policy 1, policy_version 773982 (0.0007) [2023-12-27 10:59:13,041][20381] Updated weights for policy 1, policy_version 773992 (0.0007) [2023-12-27 10:59:13,093][20381] Updated weights for policy 1, policy_version 774002 (0.0008) [2023-12-27 10:59:13,482][20347] Updated weights for policy 0, policy_version 774465 (0.0007) [2023-12-27 10:59:13,537][20347] Updated weights for policy 0, policy_version 774475 (0.0005) [2023-12-27 10:59:13,595][20347] Updated weights for policy 0, policy_version 774485 (0.0005) [2023-12-27 10:59:13,648][20347] Updated weights for policy 0, policy_version 774495 (0.0005) [2023-12-27 10:59:13,814][20381] Updated weights for policy 1, policy_version 774012 (0.0006) [2023-12-27 10:59:13,872][20381] Updated weights for policy 1, policy_version 774022 (0.0007) [2023-12-27 10:59:13,925][20381] Updated weights for policy 1, policy_version 774032 (0.0005) [2023-12-27 10:59:14,238][20347] Updated weights for policy 0, policy_version 774505 (0.0008) [2023-12-27 10:59:14,292][20347] Updated weights for policy 0, policy_version 774515 (0.0007) [2023-12-27 10:59:14,347][20347] Updated weights for policy 0, policy_version 774525 (0.0007) [2023-12-27 10:59:14,597][20381] Updated weights for policy 1, policy_version 774042 (0.0007) [2023-12-27 10:59:14,652][20381] Updated weights for policy 1, policy_version 774052 (0.0005) [2023-12-27 10:59:14,709][20381] Updated weights for policy 1, policy_version 774062 (0.0005) [2023-12-27 10:59:14,768][20381] Updated weights for policy 1, policy_version 774072 (0.0006) [2023-12-27 10:59:15,049][20347] Updated weights for policy 0, policy_version 774535 (0.0007) [2023-12-27 10:59:15,108][20347] Updated weights for policy 0, policy_version 774545 (0.0006) [2023-12-27 10:59:15,168][20347] Updated weights for policy 0, policy_version 774555 (0.0011) [2023-12-27 10:59:15,429][20381] Updated weights for policy 1, policy_version 774082 (0.0009) [2023-12-27 10:59:15,494][20381] Updated weights for policy 1, policy_version 774092 (0.0007) [2023-12-27 10:59:15,543][20381] Updated weights for policy 1, policy_version 774102 (0.0007) [2023-12-27 10:59:15,771][20347] Updated weights for policy 0, policy_version 774565 (0.0008) [2023-12-27 10:59:15,827][20347] Updated weights for policy 0, policy_version 774575 (0.0007) [2023-12-27 10:59:15,886][20347] Updated weights for policy 0, policy_version 774585 (0.0008) [2023-12-27 10:59:16,263][20381] Updated weights for policy 1, policy_version 774112 (0.0008) [2023-12-27 10:59:16,328][20381] Updated weights for policy 1, policy_version 774122 (0.0008) [2023-12-27 10:59:16,384][20381] Updated weights for policy 1, policy_version 774132 (0.0007) [2023-12-27 10:59:16,493][20347] Updated weights for policy 0, policy_version 774595 (0.0009) [2023-12-27 10:59:16,554][20347] Updated weights for policy 0, policy_version 774605 (0.0005) [2023-12-27 10:59:16,600][20347] Updated weights for policy 0, policy_version 774615 (0.0005) [2023-12-27 10:59:17,104][20381] Updated weights for policy 1, policy_version 774142 (0.0008) [2023-12-27 10:59:17,154][20381] Updated weights for policy 1, policy_version 774152 (0.0008) [2023-12-27 10:59:17,188][20347] Updated weights for policy 0, policy_version 774625 (0.0005) [2023-12-27 10:59:17,202][20381] Updated weights for policy 1, policy_version 774162 (0.0007) [2023-12-27 10:59:17,244][20347] Updated weights for policy 0, policy_version 774635 (0.0006) [2023-12-27 10:59:17,299][20347] Updated weights for policy 0, policy_version 774645 (0.0008) [2023-12-27 10:59:17,359][20347] Updated weights for policy 0, policy_version 774655 (0.0008) [2023-12-27 10:59:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 396550144. Throughput: 0: 10396.5, 1: 10181.8. Samples: 396554348. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:17,363][19170] Avg episode reward: [(0, '961.030'), (1, '922.250')] [2023-12-27 10:59:17,840][20381] Updated weights for policy 1, policy_version 774172 (0.0007) [2023-12-27 10:59:17,895][20381] Updated weights for policy 1, policy_version 774182 (0.0008) [2023-12-27 10:59:17,955][20381] Updated weights for policy 1, policy_version 774192 (0.0009) [2023-12-27 10:59:18,008][20347] Updated weights for policy 0, policy_version 774665 (0.0005) [2023-12-27 10:59:18,069][20347] Updated weights for policy 0, policy_version 774675 (0.0008) [2023-12-27 10:59:18,129][20347] Updated weights for policy 0, policy_version 774685 (0.0005) [2023-12-27 10:59:18,677][20381] Updated weights for policy 1, policy_version 774202 (0.0008) [2023-12-27 10:59:18,701][20347] Updated weights for policy 0, policy_version 774695 (0.0009) [2023-12-27 10:59:18,733][20381] Updated weights for policy 1, policy_version 774212 (0.0005) [2023-12-27 10:59:18,760][20347] Updated weights for policy 0, policy_version 774705 (0.0010) [2023-12-27 10:59:18,788][20381] Updated weights for policy 1, policy_version 774222 (0.0005) [2023-12-27 10:59:18,819][20347] Updated weights for policy 0, policy_version 774715 (0.0010) [2023-12-27 10:59:18,845][20381] Updated weights for policy 1, policy_version 774232 (0.0005) [2023-12-27 10:59:19,437][20347] Updated weights for policy 0, policy_version 774725 (0.0009) [2023-12-27 10:59:19,499][20347] Updated weights for policy 0, policy_version 774735 (0.0008) [2023-12-27 10:59:19,517][20381] Updated weights for policy 1, policy_version 774242 (0.0007) [2023-12-27 10:59:19,550][20347] Updated weights for policy 0, policy_version 774745 (0.0011) [2023-12-27 10:59:19,577][20381] Updated weights for policy 1, policy_version 774252 (0.0008) [2023-12-27 10:59:19,639][20381] Updated weights for policy 1, policy_version 774262 (0.0008) [2023-12-27 10:59:20,269][20347] Updated weights for policy 0, policy_version 774755 (0.0010) [2023-12-27 10:59:20,310][20381] Updated weights for policy 1, policy_version 774272 (0.0009) [2023-12-27 10:59:20,333][20347] Updated weights for policy 0, policy_version 774765 (0.0010) [2023-12-27 10:59:20,372][20381] Updated weights for policy 1, policy_version 774282 (0.0007) [2023-12-27 10:59:20,394][20347] Updated weights for policy 0, policy_version 774775 (0.0010) [2023-12-27 10:59:20,435][20381] Updated weights for policy 1, policy_version 774292 (0.0006) [2023-12-27 10:59:21,078][20347] Updated weights for policy 0, policy_version 774785 (0.0010) [2023-12-27 10:59:21,145][20347] Updated weights for policy 0, policy_version 774795 (0.0007) [2023-12-27 10:59:21,171][20381] Updated weights for policy 1, policy_version 774302 (0.0007) [2023-12-27 10:59:21,205][20347] Updated weights for policy 0, policy_version 774805 (0.0008) [2023-12-27 10:59:21,231][20381] Updated weights for policy 1, policy_version 774312 (0.0007) [2023-12-27 10:59:21,271][20347] Updated weights for policy 0, policy_version 774815 (0.0008) [2023-12-27 10:59:21,297][20381] Updated weights for policy 1, policy_version 774322 (0.0007) [2023-12-27 10:59:22,001][20381] Updated weights for policy 1, policy_version 774332 (0.0007) [2023-12-27 10:59:22,014][20347] Updated weights for policy 0, policy_version 774825 (0.0006) [2023-12-27 10:59:22,069][20381] Updated weights for policy 1, policy_version 774342 (0.0006) [2023-12-27 10:59:22,083][20347] Updated weights for policy 0, policy_version 774835 (0.0006) [2023-12-27 10:59:22,127][20381] Updated weights for policy 1, policy_version 774352 (0.0006) [2023-12-27 10:59:22,137][20347] Updated weights for policy 0, policy_version 774845 (0.0006) [2023-12-27 10:59:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 396648448. Throughput: 0: 10425.0, 1: 10167.1. Samples: 396617256. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:22,363][19170] Avg episode reward: [(0, '921.860'), (1, '941.830')] [2023-12-27 10:59:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000774848_198385664.pth... [2023-12-27 10:59:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000774360_198262784.pth... [2023-12-27 10:59:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000773176_197959680.pth [2023-12-27 10:59:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000773632_198074368.pth [2023-12-27 10:59:22,846][20381] Updated weights for policy 1, policy_version 774362 (0.0006) [2023-12-27 10:59:22,860][20347] Updated weights for policy 0, policy_version 774855 (0.0006) [2023-12-27 10:59:22,868][20291] KL-divergence is very high: 105.4197 [2023-12-27 10:59:22,875][20291] KL-divergence is very high: 114.7945 [2023-12-27 10:59:22,893][20291] KL-divergence is very high: 131.5866 [2023-12-27 10:59:22,909][20381] Updated weights for policy 1, policy_version 774372 (0.0007) [2023-12-27 10:59:22,923][20347] Updated weights for policy 0, policy_version 774865 (0.0006) [2023-12-27 10:59:22,964][20381] Updated weights for policy 1, policy_version 774382 (0.0006) [2023-12-27 10:59:22,982][20347] Updated weights for policy 0, policy_version 774875 (0.0006) [2023-12-27 10:59:23,022][20381] Updated weights for policy 1, policy_version 774392 (0.0007) [2023-12-27 10:59:23,678][20381] Updated weights for policy 1, policy_version 774402 (0.0007) [2023-12-27 10:59:23,699][20347] Updated weights for policy 0, policy_version 774885 (0.0008) [2023-12-27 10:59:23,726][20381] Updated weights for policy 1, policy_version 774412 (0.0005) [2023-12-27 10:59:23,757][20347] Updated weights for policy 0, policy_version 774895 (0.0010) [2023-12-27 10:59:23,777][20381] Updated weights for policy 1, policy_version 774422 (0.0005) [2023-12-27 10:59:23,818][20347] Updated weights for policy 0, policy_version 774905 (0.0010) [2023-12-27 10:59:24,367][20347] Updated weights for policy 0, policy_version 774915 (0.0010) [2023-12-27 10:59:24,422][20347] Updated weights for policy 0, policy_version 774925 (0.0010) [2023-12-27 10:59:24,480][20347] Updated weights for policy 0, policy_version 774935 (0.0010) [2023-12-27 10:59:24,530][20381] Updated weights for policy 1, policy_version 774432 (0.0007) [2023-12-27 10:59:24,581][20381] Updated weights for policy 1, policy_version 774442 (0.0007) [2023-12-27 10:59:24,601][20292] KL-divergence is very high: 115.2648 [2023-12-27 10:59:24,633][20381] Updated weights for policy 1, policy_version 774452 (0.0007) [2023-12-27 10:59:25,163][20347] Updated weights for policy 0, policy_version 774945 (0.0011) [2023-12-27 10:59:25,216][20347] Updated weights for policy 0, policy_version 774955 (0.0011) [2023-12-27 10:59:25,271][20347] Updated weights for policy 0, policy_version 774965 (0.0010) [2023-12-27 10:59:25,280][20381] Updated weights for policy 1, policy_version 774462 (0.0006) [2023-12-27 10:59:25,328][20347] Updated weights for policy 0, policy_version 774975 (0.0007) [2023-12-27 10:59:25,333][20381] Updated weights for policy 1, policy_version 774472 (0.0007) [2023-12-27 10:59:25,390][20381] Updated weights for policy 1, policy_version 774482 (0.0008) [2023-12-27 10:59:25,948][20347] Updated weights for policy 0, policy_version 774985 (0.0010) [2023-12-27 10:59:25,993][20347] Updated weights for policy 0, policy_version 774995 (0.0010) [2023-12-27 10:59:26,024][20381] Updated weights for policy 1, policy_version 774492 (0.0007) [2023-12-27 10:59:26,051][20347] Updated weights for policy 0, policy_version 775005 (0.0009) [2023-12-27 10:59:26,076][20381] Updated weights for policy 1, policy_version 774502 (0.0005) [2023-12-27 10:59:26,131][20381] Updated weights for policy 1, policy_version 774512 (0.0005) [2023-12-27 10:59:26,726][20347] Updated weights for policy 0, policy_version 775015 (0.0008) [2023-12-27 10:59:26,742][20381] Updated weights for policy 1, policy_version 774522 (0.0006) [2023-12-27 10:59:26,771][20347] Updated weights for policy 0, policy_version 775025 (0.0010) [2023-12-27 10:59:26,790][20381] Updated weights for policy 1, policy_version 774532 (0.0009) [2023-12-27 10:59:26,827][20347] Updated weights for policy 0, policy_version 775035 (0.0005) [2023-12-27 10:59:26,841][20381] Updated weights for policy 1, policy_version 774542 (0.0007) [2023-12-27 10:59:26,889][20381] Updated weights for policy 1, policy_version 774552 (0.0007) [2023-12-27 10:59:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 396746752. Throughput: 0: 10386.3, 1: 10168.7. Samples: 396738788. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:27,363][19170] Avg episode reward: [(0, '912.090'), (1, '941.670')] [2023-12-27 10:59:27,423][20347] Updated weights for policy 0, policy_version 775045 (0.0005) [2023-12-27 10:59:27,474][20347] Updated weights for policy 0, policy_version 775055 (0.0005) [2023-12-27 10:59:27,520][20347] Updated weights for policy 0, policy_version 775065 (0.0005) [2023-12-27 10:59:27,537][20381] Updated weights for policy 1, policy_version 774562 (0.0011) [2023-12-27 10:59:27,581][20381] Updated weights for policy 1, policy_version 774572 (0.0010) [2023-12-27 10:59:27,626][20381] Updated weights for policy 1, policy_version 774582 (0.0010) [2023-12-27 10:59:28,132][20347] Updated weights for policy 0, policy_version 775075 (0.0008) [2023-12-27 10:59:28,189][20347] Updated weights for policy 0, policy_version 775085 (0.0007) [2023-12-27 10:59:28,249][20347] Updated weights for policy 0, policy_version 775095 (0.0010) [2023-12-27 10:59:28,324][20381] Updated weights for policy 1, policy_version 774592 (0.0007) [2023-12-27 10:59:28,385][20381] Updated weights for policy 1, policy_version 774602 (0.0008) [2023-12-27 10:59:28,432][20381] Updated weights for policy 1, policy_version 774612 (0.0008) [2023-12-27 10:59:28,897][20347] Updated weights for policy 0, policy_version 775105 (0.0005) [2023-12-27 10:59:28,960][20291] KL-divergence is very high: 137.7819 [2023-12-27 10:59:28,965][20347] Updated weights for policy 0, policy_version 775115 (0.0005) [2023-12-27 10:59:28,981][20291] KL-divergence is very high: 159.1272 [2023-12-27 10:59:29,026][20347] Updated weights for policy 0, policy_version 775125 (0.0005) [2023-12-27 10:59:29,052][20291] KL-divergence is very high: 123.9835 [2023-12-27 10:59:29,066][20381] Updated weights for policy 1, policy_version 774622 (0.0010) [2023-12-27 10:59:29,068][20291] KL-divergence is very high: 120.2467 [2023-12-27 10:59:29,077][20347] Updated weights for policy 0, policy_version 775135 (0.0005) [2023-12-27 10:59:29,120][20381] Updated weights for policy 1, policy_version 774632 (0.0010) [2023-12-27 10:59:29,168][20381] Updated weights for policy 1, policy_version 774642 (0.0010) [2023-12-27 10:59:29,707][20347] Updated weights for policy 0, policy_version 775145 (0.0006) [2023-12-27 10:59:29,738][20291] KL-divergence is very high: 320.8217 [2023-12-27 10:59:29,759][20347] Updated weights for policy 0, policy_version 775155 (0.0006) [2023-12-27 10:59:29,782][20291] KL-divergence is very high: 211.5731 [2023-12-27 10:59:29,816][20347] Updated weights for policy 0, policy_version 775165 (0.0008) [2023-12-27 10:59:29,871][20381] Updated weights for policy 1, policy_version 774652 (0.0010) [2023-12-27 10:59:29,921][20381] Updated weights for policy 1, policy_version 774662 (0.0008) [2023-12-27 10:59:29,982][20381] Updated weights for policy 1, policy_version 774672 (0.0007) [2023-12-27 10:59:30,550][20347] Updated weights for policy 0, policy_version 775175 (0.0008) [2023-12-27 10:59:30,606][20347] Updated weights for policy 0, policy_version 775185 (0.0008) [2023-12-27 10:59:30,651][20381] Updated weights for policy 1, policy_version 774682 (0.0005) [2023-12-27 10:59:30,668][20347] Updated weights for policy 0, policy_version 775195 (0.0008) [2023-12-27 10:59:30,704][20381] Updated weights for policy 1, policy_version 774692 (0.0007) [2023-12-27 10:59:30,770][20381] Updated weights for policy 1, policy_version 774702 (0.0008) [2023-12-27 10:59:30,823][20381] Updated weights for policy 1, policy_version 774712 (0.0009) [2023-12-27 10:59:31,351][20347] Updated weights for policy 0, policy_version 775205 (0.0007) [2023-12-27 10:59:31,371][20291] KL-divergence is very high: 213.2223 [2023-12-27 10:59:31,391][20291] KL-divergence is very high: 142.2873 [2023-12-27 10:59:31,412][20347] Updated weights for policy 0, policy_version 775215 (0.0008) [2023-12-27 10:59:31,461][20291] KL-divergence is very high: 118.2092 [2023-12-27 10:59:31,467][20347] Updated weights for policy 0, policy_version 775225 (0.0007) [2023-12-27 10:59:31,476][20291] KL-divergence is very high: 248.4052 [2023-12-27 10:59:31,499][20381] Updated weights for policy 1, policy_version 774722 (0.0007) [2023-12-27 10:59:31,550][20381] Updated weights for policy 1, policy_version 774732 (0.0007) [2023-12-27 10:59:31,595][20381] Updated weights for policy 1, policy_version 774742 (0.0008) [2023-12-27 10:59:32,200][20347] Updated weights for policy 0, policy_version 775235 (0.0008) [2023-12-27 10:59:32,253][20347] Updated weights for policy 0, policy_version 775245 (0.0007) [2023-12-27 10:59:32,307][20347] Updated weights for policy 0, policy_version 775255 (0.0006) [2023-12-27 10:59:32,323][20381] Updated weights for policy 1, policy_version 774752 (0.0007) [2023-12-27 10:59:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 396853248. Throughput: 0: 10430.3, 1: 10192.3. Samples: 396864312. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:32,363][19170] Avg episode reward: [(0, '882.870'), (1, '902.770')] [2023-12-27 10:59:32,384][20381] Updated weights for policy 1, policy_version 774762 (0.0009) [2023-12-27 10:59:32,433][20381] Updated weights for policy 1, policy_version 774772 (0.0007) [2023-12-27 10:59:32,957][20347] Updated weights for policy 0, policy_version 775265 (0.0008) [2023-12-27 10:59:33,009][20347] Updated weights for policy 0, policy_version 775275 (0.0008) [2023-12-27 10:59:33,057][20347] Updated weights for policy 0, policy_version 775285 (0.0007) [2023-12-27 10:59:33,108][20347] Updated weights for policy 0, policy_version 775295 (0.0007) [2023-12-27 10:59:33,137][20381] Updated weights for policy 1, policy_version 774782 (0.0007) [2023-12-27 10:59:33,188][20381] Updated weights for policy 1, policy_version 774792 (0.0007) [2023-12-27 10:59:33,239][20381] Updated weights for policy 1, policy_version 774802 (0.0007) [2023-12-27 10:59:33,757][20347] Updated weights for policy 0, policy_version 775305 (0.0007) [2023-12-27 10:59:33,804][20347] Updated weights for policy 0, policy_version 775315 (0.0007) [2023-12-27 10:59:33,854][20381] Updated weights for policy 1, policy_version 774812 (0.0005) [2023-12-27 10:59:33,863][20347] Updated weights for policy 0, policy_version 775325 (0.0005) [2023-12-27 10:59:33,910][20381] Updated weights for policy 1, policy_version 774822 (0.0008) [2023-12-27 10:59:33,957][20381] Updated weights for policy 1, policy_version 774832 (0.0007) [2023-12-27 10:59:34,550][20347] Updated weights for policy 0, policy_version 775335 (0.0007) [2023-12-27 10:59:34,609][20347] Updated weights for policy 0, policy_version 775345 (0.0006) [2023-12-27 10:59:34,665][20381] Updated weights for policy 1, policy_version 774842 (0.0007) [2023-12-27 10:59:34,666][20347] Updated weights for policy 0, policy_version 775355 (0.0008) [2023-12-27 10:59:34,715][20381] Updated weights for policy 1, policy_version 774852 (0.0007) [2023-12-27 10:59:34,770][20381] Updated weights for policy 1, policy_version 774862 (0.0008) [2023-12-27 10:59:34,824][20381] Updated weights for policy 1, policy_version 774872 (0.0007) [2023-12-27 10:59:35,257][20347] Updated weights for policy 0, policy_version 775365 (0.0007) [2023-12-27 10:59:35,315][20347] Updated weights for policy 0, policy_version 775375 (0.0006) [2023-12-27 10:59:35,369][20347] Updated weights for policy 0, policy_version 775385 (0.0007) [2023-12-27 10:59:35,568][20381] Updated weights for policy 1, policy_version 774882 (0.0009) [2023-12-27 10:59:35,621][20381] Updated weights for policy 1, policy_version 774892 (0.0008) [2023-12-27 10:59:35,676][20381] Updated weights for policy 1, policy_version 774902 (0.0008) [2023-12-27 10:59:35,948][20347] Updated weights for policy 0, policy_version 775395 (0.0007) [2023-12-27 10:59:36,003][20347] Updated weights for policy 0, policy_version 775405 (0.0007) [2023-12-27 10:59:36,062][20347] Updated weights for policy 0, policy_version 775415 (0.0007) [2023-12-27 10:59:36,336][20381] Updated weights for policy 1, policy_version 774912 (0.0008) [2023-12-27 10:59:36,397][20381] Updated weights for policy 1, policy_version 774922 (0.0008) [2023-12-27 10:59:36,456][20381] Updated weights for policy 1, policy_version 774932 (0.0008) [2023-12-27 10:59:36,763][20347] Updated weights for policy 0, policy_version 775425 (0.0008) [2023-12-27 10:59:36,835][20347] Updated weights for policy 0, policy_version 775435 (0.0008) [2023-12-27 10:59:36,891][20347] Updated weights for policy 0, policy_version 775445 (0.0007) [2023-12-27 10:59:36,949][20347] Updated weights for policy 0, policy_version 775455 (0.0008) [2023-12-27 10:59:37,100][20381] Updated weights for policy 1, policy_version 774942 (0.0007) [2023-12-27 10:59:37,146][20381] Updated weights for policy 1, policy_version 774952 (0.0007) [2023-12-27 10:59:37,202][20381] Updated weights for policy 1, policy_version 774962 (0.0008) [2023-12-27 10:59:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 396959744. Throughput: 0: 10436.2, 1: 10196.0. Samples: 396926524. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:37,363][19170] Avg episode reward: [(0, '765.760'), (1, '912.330')] [2023-12-27 10:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000774968_198418432.pth... [2023-12-27 10:59:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000775456_198541312.pth... [2023-12-27 10:59:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000773784_198115328.pth [2023-12-27 10:59:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000774240_198230016.pth [2023-12-27 10:59:37,657][20347] Updated weights for policy 0, policy_version 775465 (0.0008) [2023-12-27 10:59:37,713][20347] Updated weights for policy 0, policy_version 775475 (0.0008) [2023-12-27 10:59:37,773][20347] Updated weights for policy 0, policy_version 775485 (0.0009) [2023-12-27 10:59:37,860][20381] Updated weights for policy 1, policy_version 774972 (0.0007) [2023-12-27 10:59:37,924][20381] Updated weights for policy 1, policy_version 774982 (0.0006) [2023-12-27 10:59:37,972][20381] Updated weights for policy 1, policy_version 774992 (0.0005) [2023-12-27 10:59:38,497][20347] Updated weights for policy 0, policy_version 775495 (0.0008) [2023-12-27 10:59:38,555][20347] Updated weights for policy 0, policy_version 775505 (0.0006) [2023-12-27 10:59:38,615][20347] Updated weights for policy 0, policy_version 775515 (0.0007) [2023-12-27 10:59:38,675][20381] Updated weights for policy 1, policy_version 775002 (0.0006) [2023-12-27 10:59:38,740][20381] Updated weights for policy 1, policy_version 775012 (0.0008) [2023-12-27 10:59:38,809][20381] Updated weights for policy 1, policy_version 775022 (0.0008) [2023-12-27 10:59:38,873][20381] Updated weights for policy 1, policy_version 775032 (0.0007) [2023-12-27 10:59:39,286][20347] Updated weights for policy 0, policy_version 775525 (0.0009) [2023-12-27 10:59:39,351][20347] Updated weights for policy 0, policy_version 775535 (0.0011) [2023-12-27 10:59:39,418][20347] Updated weights for policy 0, policy_version 775545 (0.0010) [2023-12-27 10:59:39,606][20381] Updated weights for policy 1, policy_version 775042 (0.0006) [2023-12-27 10:59:39,654][20381] Updated weights for policy 1, policy_version 775052 (0.0008) [2023-12-27 10:59:39,706][20381] Updated weights for policy 1, policy_version 775062 (0.0006) [2023-12-27 10:59:40,111][20347] Updated weights for policy 0, policy_version 775555 (0.0010) [2023-12-27 10:59:40,167][20347] Updated weights for policy 0, policy_version 775565 (0.0010) [2023-12-27 10:59:40,221][20347] Updated weights for policy 0, policy_version 775575 (0.0010) [2023-12-27 10:59:40,375][20381] Updated weights for policy 1, policy_version 775072 (0.0008) [2023-12-27 10:59:40,436][20381] Updated weights for policy 1, policy_version 775082 (0.0007) [2023-12-27 10:59:40,503][20381] Updated weights for policy 1, policy_version 775092 (0.0008) [2023-12-27 10:59:40,844][20347] Updated weights for policy 0, policy_version 775585 (0.0008) [2023-12-27 10:59:40,913][20347] Updated weights for policy 0, policy_version 775595 (0.0005) [2023-12-27 10:59:40,976][20347] Updated weights for policy 0, policy_version 775605 (0.0007) [2023-12-27 10:59:41,047][20347] Updated weights for policy 0, policy_version 775615 (0.0009) [2023-12-27 10:59:41,166][20381] Updated weights for policy 1, policy_version 775102 (0.0007) [2023-12-27 10:59:41,234][20381] Updated weights for policy 1, policy_version 775112 (0.0009) [2023-12-27 10:59:41,296][20381] Updated weights for policy 1, policy_version 775122 (0.0007) [2023-12-27 10:59:41,738][20347] Updated weights for policy 0, policy_version 775625 (0.0009) [2023-12-27 10:59:41,793][20347] Updated weights for policy 0, policy_version 775635 (0.0006) [2023-12-27 10:59:41,857][20347] Updated weights for policy 0, policy_version 775645 (0.0007) [2023-12-27 10:59:41,966][20381] Updated weights for policy 1, policy_version 775132 (0.0007) [2023-12-27 10:59:42,026][20381] Updated weights for policy 1, policy_version 775142 (0.0006) [2023-12-27 10:59:42,084][20381] Updated weights for policy 1, policy_version 775152 (0.0005) [2023-12-27 10:59:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 397058048. Throughput: 0: 10367.7, 1: 10221.5. Samples: 397049584. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:42,363][19170] Avg episode reward: [(0, '613.750'), (1, '941.590')] [2023-12-27 10:59:42,587][20347] Updated weights for policy 0, policy_version 775655 (0.0009) [2023-12-27 10:59:42,648][20347] Updated weights for policy 0, policy_version 775665 (0.0010) [2023-12-27 10:59:42,703][20347] Updated weights for policy 0, policy_version 775675 (0.0010) [2023-12-27 10:59:42,772][20381] Updated weights for policy 1, policy_version 775162 (0.0007) [2023-12-27 10:59:42,830][20381] Updated weights for policy 1, policy_version 775172 (0.0007) [2023-12-27 10:59:42,886][20381] Updated weights for policy 1, policy_version 775182 (0.0008) [2023-12-27 10:59:42,947][20381] Updated weights for policy 1, policy_version 775192 (0.0007) [2023-12-27 10:59:43,389][20347] Updated weights for policy 0, policy_version 775685 (0.0010) [2023-12-27 10:59:43,439][20347] Updated weights for policy 0, policy_version 775695 (0.0010) [2023-12-27 10:59:43,496][20347] Updated weights for policy 0, policy_version 775705 (0.0005) [2023-12-27 10:59:43,498][20381] Updated weights for policy 1, policy_version 775202 (0.0007) [2023-12-27 10:59:43,554][20381] Updated weights for policy 1, policy_version 775212 (0.0006) [2023-12-27 10:59:43,604][20381] Updated weights for policy 1, policy_version 775222 (0.0007) [2023-12-27 10:59:44,173][20347] Updated weights for policy 0, policy_version 775715 (0.0007) [2023-12-27 10:59:44,235][20347] Updated weights for policy 0, policy_version 775725 (0.0010) [2023-12-27 10:59:44,261][20381] Updated weights for policy 1, policy_version 775232 (0.0008) [2023-12-27 10:59:44,290][20347] Updated weights for policy 0, policy_version 775735 (0.0010) [2023-12-27 10:59:44,323][20381] Updated weights for policy 1, policy_version 775242 (0.0009) [2023-12-27 10:59:44,380][20381] Updated weights for policy 1, policy_version 775252 (0.0007) [2023-12-27 10:59:44,935][20347] Updated weights for policy 0, policy_version 775745 (0.0010) [2023-12-27 10:59:44,960][20381] Updated weights for policy 1, policy_version 775262 (0.0006) [2023-12-27 10:59:44,995][20347] Updated weights for policy 0, policy_version 775755 (0.0011) [2023-12-27 10:59:45,013][20381] Updated weights for policy 1, policy_version 775272 (0.0006) [2023-12-27 10:59:45,051][20347] Updated weights for policy 0, policy_version 775765 (0.0011) [2023-12-27 10:59:45,071][20381] Updated weights for policy 1, policy_version 775282 (0.0006) [2023-12-27 10:59:45,114][20347] Updated weights for policy 0, policy_version 775775 (0.0011) [2023-12-27 10:59:45,644][20381] Updated weights for policy 1, policy_version 775292 (0.0006) [2023-12-27 10:59:45,699][20381] Updated weights for policy 1, policy_version 775302 (0.0007) [2023-12-27 10:59:45,753][20381] Updated weights for policy 1, policy_version 775312 (0.0007) [2023-12-27 10:59:45,796][20347] Updated weights for policy 0, policy_version 775785 (0.0006) [2023-12-27 10:59:45,857][20347] Updated weights for policy 0, policy_version 775795 (0.0010) [2023-12-27 10:59:45,915][20347] Updated weights for policy 0, policy_version 775805 (0.0010) [2023-12-27 10:59:46,393][20381] Updated weights for policy 1, policy_version 775322 (0.0008) [2023-12-27 10:59:46,459][20381] Updated weights for policy 1, policy_version 775332 (0.0007) [2023-12-27 10:59:46,507][20381] Updated weights for policy 1, policy_version 775342 (0.0007) [2023-12-27 10:59:46,538][20347] Updated weights for policy 0, policy_version 775815 (0.0010) [2023-12-27 10:59:46,562][20381] Updated weights for policy 1, policy_version 775352 (0.0005) [2023-12-27 10:59:46,589][20347] Updated weights for policy 0, policy_version 775825 (0.0010) [2023-12-27 10:59:46,646][20347] Updated weights for policy 0, policy_version 775835 (0.0008) [2023-12-27 10:59:47,245][20347] Updated weights for policy 0, policy_version 775845 (0.0008) [2023-12-27 10:59:47,272][20381] Updated weights for policy 1, policy_version 775362 (0.0007) [2023-12-27 10:59:47,295][20347] Updated weights for policy 0, policy_version 775855 (0.0006) [2023-12-27 10:59:47,331][20381] Updated weights for policy 1, policy_version 775372 (0.0008) [2023-12-27 10:59:47,351][20347] Updated weights for policy 0, policy_version 775865 (0.0005) [2023-12-27 10:59:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 397156352. Throughput: 0: 10372.1, 1: 10259.2. Samples: 397175188. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:47,364][19170] Avg episode reward: [(0, '522.590'), (1, '922.360')] [2023-12-27 10:59:47,390][20381] Updated weights for policy 1, policy_version 775382 (0.0006) [2023-12-27 10:59:47,980][20347] Updated weights for policy 0, policy_version 775875 (0.0006) [2023-12-27 10:59:47,989][20381] Updated weights for policy 1, policy_version 775392 (0.0005) [2023-12-27 10:59:48,039][20347] Updated weights for policy 0, policy_version 775885 (0.0007) [2023-12-27 10:59:48,045][20381] Updated weights for policy 1, policy_version 775402 (0.0005) [2023-12-27 10:59:48,095][20347] Updated weights for policy 0, policy_version 775895 (0.0007) [2023-12-27 10:59:48,102][20381] Updated weights for policy 1, policy_version 775412 (0.0006) [2023-12-27 10:59:48,114][20291] KL-divergence is very high: 129.7823 [2023-12-27 10:59:48,713][20381] Updated weights for policy 1, policy_version 775422 (0.0006) [2023-12-27 10:59:48,772][20381] Updated weights for policy 1, policy_version 775432 (0.0006) [2023-12-27 10:59:48,786][20347] Updated weights for policy 0, policy_version 775905 (0.0007) [2023-12-27 10:59:48,827][20381] Updated weights for policy 1, policy_version 775442 (0.0005) [2023-12-27 10:59:48,841][20347] Updated weights for policy 0, policy_version 775915 (0.0006) [2023-12-27 10:59:48,893][20347] Updated weights for policy 0, policy_version 775925 (0.0007) [2023-12-27 10:59:48,952][20347] Updated weights for policy 0, policy_version 775935 (0.0008) [2023-12-27 10:59:49,494][20381] Updated weights for policy 1, policy_version 775452 (0.0007) [2023-12-27 10:59:49,551][20381] Updated weights for policy 1, policy_version 775462 (0.0007) [2023-12-27 10:59:49,579][20292] KL-divergence is very high: 105.0323 [2023-12-27 10:59:49,592][20292] KL-divergence is very high: 111.3210 [2023-12-27 10:59:49,610][20381] Updated weights for policy 1, policy_version 775472 (0.0007) [2023-12-27 10:59:49,641][20347] Updated weights for policy 0, policy_version 775945 (0.0007) [2023-12-27 10:59:49,653][20292] KL-divergence is very high: 103.8057 [2023-12-27 10:59:49,679][20291] KL-divergence is very high: 124.0923 [2023-12-27 10:59:49,697][20347] Updated weights for policy 0, policy_version 775955 (0.0008) [2023-12-27 10:59:49,757][20347] Updated weights for policy 0, policy_version 775965 (0.0008) [2023-12-27 10:59:50,262][20381] Updated weights for policy 1, policy_version 775482 (0.0006) [2023-12-27 10:59:50,322][20381] Updated weights for policy 1, policy_version 775492 (0.0008) [2023-12-27 10:59:50,386][20381] Updated weights for policy 1, policy_version 775502 (0.0008) [2023-12-27 10:59:50,438][20381] Updated weights for policy 1, policy_version 775512 (0.0008) [2023-12-27 10:59:50,461][20347] Updated weights for policy 0, policy_version 775975 (0.0006) [2023-12-27 10:59:50,515][20347] Updated weights for policy 0, policy_version 775985 (0.0005) [2023-12-27 10:59:50,567][20347] Updated weights for policy 0, policy_version 775995 (0.0007) [2023-12-27 10:59:51,158][20381] Updated weights for policy 1, policy_version 775522 (0.0008) [2023-12-27 10:59:51,208][20381] Updated weights for policy 1, policy_version 775532 (0.0009) [2023-12-27 10:59:51,253][20347] Updated weights for policy 0, policy_version 776005 (0.0008) [2023-12-27 10:59:51,268][20381] Updated weights for policy 1, policy_version 775542 (0.0009) [2023-12-27 10:59:51,312][20347] Updated weights for policy 0, policy_version 776015 (0.0006) [2023-12-27 10:59:51,366][20347] Updated weights for policy 0, policy_version 776025 (0.0006) [2023-12-27 10:59:52,031][20347] Updated weights for policy 0, policy_version 776035 (0.0008) [2023-12-27 10:59:52,033][20381] Updated weights for policy 1, policy_version 775552 (0.0007) [2023-12-27 10:59:52,087][20381] Updated weights for policy 1, policy_version 775562 (0.0007) [2023-12-27 10:59:52,089][20347] Updated weights for policy 0, policy_version 776045 (0.0006) [2023-12-27 10:59:52,140][20381] Updated weights for policy 1, policy_version 775572 (0.0006) [2023-12-27 10:59:52,149][20347] Updated weights for policy 0, policy_version 776055 (0.0006) [2023-12-27 10:59:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 397271040. Throughput: 0: 10348.7, 1: 10293.8. Samples: 397238424. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:52,363][19170] Avg episode reward: [(0, '496.360'), (1, '874.670')] [2023-12-27 10:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000775576_198574080.pth... [2023-12-27 10:59:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000776064_198696960.pth... [2023-12-27 10:59:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000774360_198262784.pth [2023-12-27 10:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000775576_198574080.pth [2023-12-27 10:59:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000774848_198385664.pth [2023-12-27 10:59:52,376][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000776064_198696960.pth [2023-12-27 10:59:52,808][20381] Updated weights for policy 1, policy_version 775582 (0.0010) [2023-12-27 10:59:52,863][20381] Updated weights for policy 1, policy_version 775592 (0.0010) [2023-12-27 10:59:52,880][20347] Updated weights for policy 0, policy_version 776065 (0.0008) [2023-12-27 10:59:52,916][20381] Updated weights for policy 1, policy_version 775602 (0.0009) [2023-12-27 10:59:52,934][20347] Updated weights for policy 0, policy_version 776075 (0.0005) [2023-12-27 10:59:52,985][20347] Updated weights for policy 0, policy_version 776085 (0.0005) [2023-12-27 10:59:53,020][20291] KL-divergence is very high: 193.0928 [2023-12-27 10:59:53,039][20347] Updated weights for policy 0, policy_version 776095 (0.0005) [2023-12-27 10:59:53,041][20291] KL-divergence is very high: 334.8122 [2023-12-27 10:59:53,591][20381] Updated weights for policy 1, policy_version 775612 (0.0007) [2023-12-27 10:59:53,619][20291] KL-divergence is very high: 253.2770 [2023-12-27 10:59:53,638][20381] Updated weights for policy 1, policy_version 775622 (0.0006) [2023-12-27 10:59:53,657][20347] Updated weights for policy 0, policy_version 776105 (0.0008) [2023-12-27 10:59:53,686][20381] Updated weights for policy 1, policy_version 775632 (0.0006) [2023-12-27 10:59:53,703][20347] Updated weights for policy 0, policy_version 776115 (0.0005) [2023-12-27 10:59:53,756][20347] Updated weights for policy 0, policy_version 776125 (0.0006) [2023-12-27 10:59:54,395][20381] Updated weights for policy 1, policy_version 775642 (0.0006) [2023-12-27 10:59:54,446][20347] Updated weights for policy 0, policy_version 776135 (0.0008) [2023-12-27 10:59:54,453][20381] Updated weights for policy 1, policy_version 775652 (0.0007) [2023-12-27 10:59:54,499][20347] Updated weights for policy 0, policy_version 776145 (0.0006) [2023-12-27 10:59:54,504][20381] Updated weights for policy 1, policy_version 775662 (0.0005) [2023-12-27 10:59:54,556][20347] Updated weights for policy 0, policy_version 776155 (0.0008) [2023-12-27 10:59:54,561][20381] Updated weights for policy 1, policy_version 775672 (0.0008) [2023-12-27 10:59:55,160][20381] Updated weights for policy 1, policy_version 775682 (0.0007) [2023-12-27 10:59:55,217][20381] Updated weights for policy 1, policy_version 775692 (0.0007) [2023-12-27 10:59:55,251][20347] Updated weights for policy 0, policy_version 776165 (0.0008) [2023-12-27 10:59:55,283][20381] Updated weights for policy 1, policy_version 775702 (0.0006) [2023-12-27 10:59:55,311][20347] Updated weights for policy 0, policy_version 776175 (0.0007) [2023-12-27 10:59:55,363][20347] Updated weights for policy 0, policy_version 776185 (0.0010) [2023-12-27 10:59:55,960][20381] Updated weights for policy 1, policy_version 775712 (0.0008) [2023-12-27 10:59:56,020][20381] Updated weights for policy 1, policy_version 775722 (0.0007) [2023-12-27 10:59:56,059][20347] Updated weights for policy 0, policy_version 776195 (0.0009) [2023-12-27 10:59:56,073][20381] Updated weights for policy 1, policy_version 775732 (0.0007) [2023-12-27 10:59:56,112][20347] Updated weights for policy 0, policy_version 776205 (0.0007) [2023-12-27 10:59:56,171][20347] Updated weights for policy 0, policy_version 776215 (0.0009) [2023-12-27 10:59:56,751][20381] Updated weights for policy 1, policy_version 775742 (0.0007) [2023-12-27 10:59:56,806][20381] Updated weights for policy 1, policy_version 775752 (0.0007) [2023-12-27 10:59:56,826][20347] Updated weights for policy 0, policy_version 776225 (0.0008) [2023-12-27 10:59:56,875][20381] Updated weights for policy 1, policy_version 775762 (0.0010) [2023-12-27 10:59:56,883][20347] Updated weights for policy 0, policy_version 776235 (0.0010) [2023-12-27 10:59:56,934][20347] Updated weights for policy 0, policy_version 776245 (0.0010) [2023-12-27 10:59:56,982][20347] Updated weights for policy 0, policy_version 776255 (0.0010) [2023-12-27 10:59:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 397369344. Throughput: 0: 10346.2, 1: 10288.7. Samples: 397360724. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 10:59:57,363][19170] Avg episode reward: [(0, '708.720'), (1, '873.460')] [2023-12-27 10:59:57,566][20381] Updated weights for policy 1, policy_version 775772 (0.0006) [2023-12-27 10:59:57,599][20347] Updated weights for policy 0, policy_version 776265 (0.0010) [2023-12-27 10:59:57,624][20381] Updated weights for policy 1, policy_version 775782 (0.0005) [2023-12-27 10:59:57,644][20347] Updated weights for policy 0, policy_version 776275 (0.0010) [2023-12-27 10:59:57,668][20291] KL-divergence is very high: 670.1682 [2023-12-27 10:59:57,672][20381] Updated weights for policy 1, policy_version 775792 (0.0005) [2023-12-27 10:59:57,695][20347] Updated weights for policy 0, policy_version 776285 (0.0010) [2023-12-27 10:59:58,249][20291] KL-divergence is very high: 100.9397 [2023-12-27 10:59:58,281][20347] Updated weights for policy 0, policy_version 776295 (0.0008) [2023-12-27 10:59:58,334][20381] Updated weights for policy 1, policy_version 775802 (0.0006) [2023-12-27 10:59:58,347][20291] KL-divergence is very high: 189.5912 [2023-12-27 10:59:58,348][20347] Updated weights for policy 0, policy_version 776305 (0.0008) [2023-12-27 10:59:58,399][20381] Updated weights for policy 1, policy_version 775812 (0.0010) [2023-12-27 10:59:58,405][20347] Updated weights for policy 0, policy_version 776315 (0.0006) [2023-12-27 10:59:58,457][20381] Updated weights for policy 1, policy_version 775822 (0.0009) [2023-12-27 10:59:58,516][20381] Updated weights for policy 1, policy_version 775832 (0.0008) [2023-12-27 10:59:59,138][20291] KL-divergence is very high: 168.8508 [2023-12-27 10:59:59,151][20347] Updated weights for policy 0, policy_version 776325 (0.0006) [2023-12-27 10:59:59,185][20291] KL-divergence is very high: 148.5361 [2023-12-27 10:59:59,211][20347] Updated weights for policy 0, policy_version 776335 (0.0006) [2023-12-27 10:59:59,247][20291] KL-divergence is very high: 137.7910 [2023-12-27 10:59:59,284][20347] Updated weights for policy 0, policy_version 776345 (0.0007) [2023-12-27 10:59:59,286][20381] Updated weights for policy 1, policy_version 775842 (0.0010) [2023-12-27 10:59:59,299][20291] KL-divergence is very high: 236.9095 [2023-12-27 10:59:59,349][20381] Updated weights for policy 1, policy_version 775852 (0.0010) [2023-12-27 10:59:59,419][20381] Updated weights for policy 1, policy_version 775862 (0.0009) [2023-12-27 10:59:59,978][20347] Updated weights for policy 0, policy_version 776355 (0.0007) [2023-12-27 11:00:00,040][20347] Updated weights for policy 0, policy_version 776365 (0.0007) [2023-12-27 11:00:00,078][20381] Updated weights for policy 1, policy_version 775872 (0.0008) [2023-12-27 11:00:00,106][20347] Updated weights for policy 0, policy_version 776375 (0.0007) [2023-12-27 11:00:00,139][20381] Updated weights for policy 1, policy_version 775882 (0.0007) [2023-12-27 11:00:00,207][20381] Updated weights for policy 1, policy_version 775892 (0.0008) [2023-12-27 11:00:00,674][20347] Updated weights for policy 0, policy_version 776385 (0.0006) [2023-12-27 11:00:00,737][20347] Updated weights for policy 0, policy_version 776395 (0.0007) [2023-12-27 11:00:00,793][20347] Updated weights for policy 0, policy_version 776405 (0.0009) [2023-12-27 11:00:00,826][20381] Updated weights for policy 1, policy_version 775902 (0.0010) [2023-12-27 11:00:00,848][20347] Updated weights for policy 0, policy_version 776415 (0.0007) [2023-12-27 11:00:00,885][20381] Updated weights for policy 1, policy_version 775912 (0.0011) [2023-12-27 11:00:00,944][20381] Updated weights for policy 1, policy_version 775922 (0.0011) [2023-12-27 11:00:01,544][20347] Updated weights for policy 0, policy_version 776425 (0.0009) [2023-12-27 11:00:01,597][20347] Updated weights for policy 0, policy_version 776435 (0.0010) [2023-12-27 11:00:01,659][20347] Updated weights for policy 0, policy_version 776445 (0.0008) [2023-12-27 11:00:01,690][20381] Updated weights for policy 1, policy_version 775932 (0.0011) [2023-12-27 11:00:01,758][20381] Updated weights for policy 1, policy_version 775942 (0.0010) [2023-12-27 11:00:01,812][20381] Updated weights for policy 1, policy_version 775952 (0.0010) [2023-12-27 11:00:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 397467648. Throughput: 0: 10343.4, 1: 10275.8. Samples: 397482212. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:02,363][19170] Avg episode reward: [(0, '70.350'), (1, '902.980')] [2023-12-27 11:00:02,378][20347] Updated weights for policy 0, policy_version 776455 (0.0007) [2023-12-27 11:00:02,430][20347] Updated weights for policy 0, policy_version 776465 (0.0007) [2023-12-27 11:00:02,485][20347] Updated weights for policy 0, policy_version 776475 (0.0007) [2023-12-27 11:00:02,512][20381] Updated weights for policy 1, policy_version 775962 (0.0010) [2023-12-27 11:00:02,560][20381] Updated weights for policy 1, policy_version 775972 (0.0010) [2023-12-27 11:00:02,612][20381] Updated weights for policy 1, policy_version 775982 (0.0007) [2023-12-27 11:00:02,664][20381] Updated weights for policy 1, policy_version 775992 (0.0008) [2023-12-27 11:00:03,121][20347] Updated weights for policy 0, policy_version 776485 (0.0006) [2023-12-27 11:00:03,176][20347] Updated weights for policy 0, policy_version 776495 (0.0005) [2023-12-27 11:00:03,234][20347] Updated weights for policy 0, policy_version 776505 (0.0005) [2023-12-27 11:00:03,417][20381] Updated weights for policy 1, policy_version 776002 (0.0007) [2023-12-27 11:00:03,473][20381] Updated weights for policy 1, policy_version 776012 (0.0008) [2023-12-27 11:00:03,528][20381] Updated weights for policy 1, policy_version 776022 (0.0009) [2023-12-27 11:00:03,763][20347] Updated weights for policy 0, policy_version 776515 (0.0005) [2023-12-27 11:00:03,813][20347] Updated weights for policy 0, policy_version 776525 (0.0005) [2023-12-27 11:00:03,875][20347] Updated weights for policy 0, policy_version 776535 (0.0007) [2023-12-27 11:00:04,261][20381] Updated weights for policy 1, policy_version 776032 (0.0008) [2023-12-27 11:00:04,317][20381] Updated weights for policy 1, policy_version 776042 (0.0008) [2023-12-27 11:00:04,375][20381] Updated weights for policy 1, policy_version 776052 (0.0008) [2023-12-27 11:00:04,488][20347] Updated weights for policy 0, policy_version 776545 (0.0007) [2023-12-27 11:00:04,543][20347] Updated weights for policy 0, policy_version 776555 (0.0008) [2023-12-27 11:00:04,603][20347] Updated weights for policy 0, policy_version 776565 (0.0007) [2023-12-27 11:00:04,662][20347] Updated weights for policy 0, policy_version 776575 (0.0008) [2023-12-27 11:00:04,986][20381] Updated weights for policy 1, policy_version 776062 (0.0006) [2023-12-27 11:00:05,039][20381] Updated weights for policy 1, policy_version 776072 (0.0005) [2023-12-27 11:00:05,091][20381] Updated weights for policy 1, policy_version 776082 (0.0005) [2023-12-27 11:00:05,382][20347] Updated weights for policy 0, policy_version 776585 (0.0011) [2023-12-27 11:00:05,435][20347] Updated weights for policy 0, policy_version 776595 (0.0006) [2023-12-27 11:00:05,496][20347] Updated weights for policy 0, policy_version 776605 (0.0007) [2023-12-27 11:00:05,646][20381] Updated weights for policy 1, policy_version 776092 (0.0005) [2023-12-27 11:00:05,703][20381] Updated weights for policy 1, policy_version 776102 (0.0005) [2023-12-27 11:00:05,764][20381] Updated weights for policy 1, policy_version 776112 (0.0005) [2023-12-27 11:00:06,131][20347] Updated weights for policy 0, policy_version 776615 (0.0009) [2023-12-27 11:00:06,195][20347] Updated weights for policy 0, policy_version 776625 (0.0007) [2023-12-27 11:00:06,260][20347] Updated weights for policy 0, policy_version 776635 (0.0010) [2023-12-27 11:00:06,365][20381] Updated weights for policy 1, policy_version 776122 (0.0006) [2023-12-27 11:00:06,426][20381] Updated weights for policy 1, policy_version 776132 (0.0007) [2023-12-27 11:00:06,481][20381] Updated weights for policy 1, policy_version 776142 (0.0008) [2023-12-27 11:00:06,537][20381] Updated weights for policy 1, policy_version 776152 (0.0007) [2023-12-27 11:00:06,943][20347] Updated weights for policy 0, policy_version 776645 (0.0011) [2023-12-27 11:00:06,992][20347] Updated weights for policy 0, policy_version 776655 (0.0010) [2023-12-27 11:00:07,044][20347] Updated weights for policy 0, policy_version 776665 (0.0010) [2023-12-27 11:00:07,176][20381] Updated weights for policy 1, policy_version 776162 (0.0006) [2023-12-27 11:00:07,230][20381] Updated weights for policy 1, policy_version 776172 (0.0006) [2023-12-27 11:00:07,293][20381] Updated weights for policy 1, policy_version 776182 (0.0007) [2023-12-27 11:00:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 397582336. Throughput: 0: 10350.5, 1: 10270.3. Samples: 397545192. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:07,363][19170] Avg episode reward: [(0, '76.820'), (1, '951.680')] [2023-12-27 11:00:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000776672_198852608.pth... [2023-12-27 11:00:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000776184_198729728.pth... [2023-12-27 11:00:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000775456_198541312.pth [2023-12-27 11:00:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000774968_198418432.pth [2023-12-27 11:00:07,736][20347] Updated weights for policy 0, policy_version 776675 (0.0010) [2023-12-27 11:00:07,788][20347] Updated weights for policy 0, policy_version 776685 (0.0010) [2023-12-27 11:00:07,843][20347] Updated weights for policy 0, policy_version 776695 (0.0010) [2023-12-27 11:00:07,955][20381] Updated weights for policy 1, policy_version 776192 (0.0007) [2023-12-27 11:00:08,015][20381] Updated weights for policy 1, policy_version 776202 (0.0006) [2023-12-27 11:00:08,072][20381] Updated weights for policy 1, policy_version 776212 (0.0005) [2023-12-27 11:00:08,503][20347] Updated weights for policy 0, policy_version 776705 (0.0009) [2023-12-27 11:00:08,553][20347] Updated weights for policy 0, policy_version 776715 (0.0007) [2023-12-27 11:00:08,605][20347] Updated weights for policy 0, policy_version 776725 (0.0006) [2023-12-27 11:00:08,669][20347] Updated weights for policy 0, policy_version 776735 (0.0007) [2023-12-27 11:00:08,783][20381] Updated weights for policy 1, policy_version 776222 (0.0007) [2023-12-27 11:00:08,844][20381] Updated weights for policy 1, policy_version 776232 (0.0007) [2023-12-27 11:00:08,900][20381] Updated weights for policy 1, policy_version 776242 (0.0007) [2023-12-27 11:00:09,319][20347] Updated weights for policy 0, policy_version 776745 (0.0006) [2023-12-27 11:00:09,389][20347] Updated weights for policy 0, policy_version 776755 (0.0008) [2023-12-27 11:00:09,450][20347] Updated weights for policy 0, policy_version 776765 (0.0006) [2023-12-27 11:00:09,633][20381] Updated weights for policy 1, policy_version 776252 (0.0007) [2023-12-27 11:00:09,695][20381] Updated weights for policy 1, policy_version 776262 (0.0006) [2023-12-27 11:00:09,751][20381] Updated weights for policy 1, policy_version 776272 (0.0005) [2023-12-27 11:00:10,141][20347] Updated weights for policy 0, policy_version 776775 (0.0007) [2023-12-27 11:00:10,200][20347] Updated weights for policy 0, policy_version 776785 (0.0007) [2023-12-27 11:00:10,257][20347] Updated weights for policy 0, policy_version 776795 (0.0008) [2023-12-27 11:00:10,438][20381] Updated weights for policy 1, policy_version 776282 (0.0007) [2023-12-27 11:00:10,492][20381] Updated weights for policy 1, policy_version 776292 (0.0007) [2023-12-27 11:00:10,550][20381] Updated weights for policy 1, policy_version 776302 (0.0008) [2023-12-27 11:00:10,598][20381] Updated weights for policy 1, policy_version 776312 (0.0008) [2023-12-27 11:00:10,871][20347] Updated weights for policy 0, policy_version 776805 (0.0007) [2023-12-27 11:00:10,923][20347] Updated weights for policy 0, policy_version 776815 (0.0006) [2023-12-27 11:00:10,968][20347] Updated weights for policy 0, policy_version 776825 (0.0008) [2023-12-27 11:00:11,361][20381] Updated weights for policy 1, policy_version 776322 (0.0008) [2023-12-27 11:00:11,432][20381] Updated weights for policy 1, policy_version 776332 (0.0008) [2023-12-27 11:00:11,484][20381] Updated weights for policy 1, policy_version 776342 (0.0008) [2023-12-27 11:00:11,681][20347] Updated weights for policy 0, policy_version 776835 (0.0008) [2023-12-27 11:00:11,741][20347] Updated weights for policy 0, policy_version 776845 (0.0007) [2023-12-27 11:00:11,804][20347] Updated weights for policy 0, policy_version 776855 (0.0007) [2023-12-27 11:00:12,242][20381] Updated weights for policy 1, policy_version 776352 (0.0008) [2023-12-27 11:00:12,301][20381] Updated weights for policy 1, policy_version 776362 (0.0009) [2023-12-27 11:00:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 397672448. Throughput: 0: 10385.0, 1: 10294.6. Samples: 397669372. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:12,363][19170] Avg episode reward: [(0, '104.510'), (1, '932.070')] [2023-12-27 11:00:12,363][20381] Updated weights for policy 1, policy_version 776372 (0.0008) [2023-12-27 11:00:12,453][20347] Updated weights for policy 0, policy_version 776865 (0.0008) [2023-12-27 11:00:12,504][20347] Updated weights for policy 0, policy_version 776875 (0.0005) [2023-12-27 11:00:12,557][20347] Updated weights for policy 0, policy_version 776885 (0.0008) [2023-12-27 11:00:12,603][20347] Updated weights for policy 0, policy_version 776895 (0.0007) [2023-12-27 11:00:13,103][20381] Updated weights for policy 1, policy_version 776382 (0.0007) [2023-12-27 11:00:13,159][20381] Updated weights for policy 1, policy_version 776392 (0.0008) [2023-12-27 11:00:13,214][20381] Updated weights for policy 1, policy_version 776402 (0.0007) [2023-12-27 11:00:13,260][20347] Updated weights for policy 0, policy_version 776905 (0.0007) [2023-12-27 11:00:13,305][20347] Updated weights for policy 0, policy_version 776915 (0.0006) [2023-12-27 11:00:13,356][20347] Updated weights for policy 0, policy_version 776925 (0.0008) [2023-12-27 11:00:13,896][20381] Updated weights for policy 1, policy_version 776412 (0.0006) [2023-12-27 11:00:13,948][20381] Updated weights for policy 1, policy_version 776422 (0.0007) [2023-12-27 11:00:14,007][20381] Updated weights for policy 1, policy_version 776432 (0.0005) [2023-12-27 11:00:14,055][20347] Updated weights for policy 0, policy_version 776935 (0.0007) [2023-12-27 11:00:14,110][20347] Updated weights for policy 0, policy_version 776945 (0.0008) [2023-12-27 11:00:14,164][20347] Updated weights for policy 0, policy_version 776955 (0.0007) [2023-12-27 11:00:14,629][20381] Updated weights for policy 1, policy_version 776442 (0.0006) [2023-12-27 11:00:14,675][20381] Updated weights for policy 1, policy_version 776452 (0.0010) [2023-12-27 11:00:14,731][20381] Updated weights for policy 1, policy_version 776462 (0.0011) [2023-12-27 11:00:14,785][20347] Updated weights for policy 0, policy_version 776965 (0.0008) [2023-12-27 11:00:14,794][20381] Updated weights for policy 1, policy_version 776472 (0.0009) [2023-12-27 11:00:14,846][20347] Updated weights for policy 0, policy_version 776975 (0.0007) [2023-12-27 11:00:14,907][20347] Updated weights for policy 0, policy_version 776985 (0.0007) [2023-12-27 11:00:15,534][20347] Updated weights for policy 0, policy_version 776995 (0.0006) [2023-12-27 11:00:15,555][20381] Updated weights for policy 1, policy_version 776482 (0.0011) [2023-12-27 11:00:15,583][20347] Updated weights for policy 0, policy_version 777005 (0.0005) [2023-12-27 11:00:15,611][20381] Updated weights for policy 1, policy_version 776492 (0.0011) [2023-12-27 11:00:15,639][20347] Updated weights for policy 0, policy_version 777015 (0.0005) [2023-12-27 11:00:15,671][20381] Updated weights for policy 1, policy_version 776502 (0.0011) [2023-12-27 11:00:16,360][20347] Updated weights for policy 0, policy_version 777025 (0.0006) [2023-12-27 11:00:16,396][20381] Updated weights for policy 1, policy_version 776512 (0.0011) [2023-12-27 11:00:16,409][20347] Updated weights for policy 0, policy_version 777035 (0.0005) [2023-12-27 11:00:16,452][20381] Updated weights for policy 1, policy_version 776522 (0.0011) [2023-12-27 11:00:16,461][20347] Updated weights for policy 0, policy_version 777045 (0.0005) [2023-12-27 11:00:16,508][20381] Updated weights for policy 1, policy_version 776532 (0.0011) [2023-12-27 11:00:16,517][20347] Updated weights for policy 0, policy_version 777055 (0.0006) [2023-12-27 11:00:17,130][20347] Updated weights for policy 0, policy_version 777065 (0.0008) [2023-12-27 11:00:17,184][20347] Updated weights for policy 0, policy_version 777075 (0.0008) [2023-12-27 11:00:17,204][20381] Updated weights for policy 1, policy_version 776542 (0.0008) [2023-12-27 11:00:17,233][20347] Updated weights for policy 0, policy_version 777085 (0.0007) [2023-12-27 11:00:17,267][20381] Updated weights for policy 1, policy_version 776552 (0.0005) [2023-12-27 11:00:17,327][20381] Updated weights for policy 1, policy_version 776562 (0.0007) [2023-12-27 11:00:17,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 397787136. Throughput: 0: 10388.5, 1: 10216.6. Samples: 397791548. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:17,364][19170] Avg episode reward: [(0, '350.290'), (1, '922.170')] [2023-12-27 11:00:17,863][20347] Updated weights for policy 0, policy_version 777095 (0.0007) [2023-12-27 11:00:17,909][20347] Updated weights for policy 0, policy_version 777105 (0.0009) [2023-12-27 11:00:17,964][20347] Updated weights for policy 0, policy_version 777115 (0.0008) [2023-12-27 11:00:18,003][20381] Updated weights for policy 1, policy_version 776572 (0.0007) [2023-12-27 11:00:18,058][20381] Updated weights for policy 1, policy_version 776582 (0.0005) [2023-12-27 11:00:18,115][20381] Updated weights for policy 1, policy_version 776592 (0.0005) [2023-12-27 11:00:18,677][20347] Updated weights for policy 0, policy_version 777125 (0.0008) [2023-12-27 11:00:18,721][20381] Updated weights for policy 1, policy_version 776602 (0.0007) [2023-12-27 11:00:18,735][20347] Updated weights for policy 0, policy_version 777135 (0.0006) [2023-12-27 11:00:18,781][20381] Updated weights for policy 1, policy_version 776612 (0.0007) [2023-12-27 11:00:18,791][20347] Updated weights for policy 0, policy_version 777145 (0.0006) [2023-12-27 11:00:18,838][20381] Updated weights for policy 1, policy_version 776622 (0.0006) [2023-12-27 11:00:18,895][20381] Updated weights for policy 1, policy_version 776632 (0.0008) [2023-12-27 11:00:19,494][20347] Updated weights for policy 0, policy_version 777155 (0.0007) [2023-12-27 11:00:19,558][20347] Updated weights for policy 0, policy_version 777165 (0.0006) [2023-12-27 11:00:19,582][20381] Updated weights for policy 1, policy_version 776642 (0.0007) [2023-12-27 11:00:19,622][20347] Updated weights for policy 0, policy_version 777175 (0.0006) [2023-12-27 11:00:19,643][20381] Updated weights for policy 1, policy_version 776652 (0.0007) [2023-12-27 11:00:19,697][20381] Updated weights for policy 1, policy_version 776662 (0.0007) [2023-12-27 11:00:20,338][20347] Updated weights for policy 0, policy_version 777185 (0.0007) [2023-12-27 11:00:20,403][20347] Updated weights for policy 0, policy_version 777195 (0.0007) [2023-12-27 11:00:20,404][20381] Updated weights for policy 1, policy_version 776672 (0.0006) [2023-12-27 11:00:20,463][20347] Updated weights for policy 0, policy_version 777205 (0.0007) [2023-12-27 11:00:20,464][20381] Updated weights for policy 1, policy_version 776682 (0.0009) [2023-12-27 11:00:20,517][20347] Updated weights for policy 0, policy_version 777215 (0.0005) [2023-12-27 11:00:20,525][20381] Updated weights for policy 1, policy_version 776692 (0.0007) [2023-12-27 11:00:21,212][20347] Updated weights for policy 0, policy_version 777225 (0.0008) [2023-12-27 11:00:21,274][20381] Updated weights for policy 1, policy_version 776702 (0.0008) [2023-12-27 11:00:21,276][20347] Updated weights for policy 0, policy_version 777235 (0.0008) [2023-12-27 11:00:21,338][20381] Updated weights for policy 1, policy_version 776712 (0.0007) [2023-12-27 11:00:21,341][20347] Updated weights for policy 0, policy_version 777245 (0.0008) [2023-12-27 11:00:21,411][20381] Updated weights for policy 1, policy_version 776722 (0.0008) [2023-12-27 11:00:22,096][20347] Updated weights for policy 0, policy_version 777255 (0.0008) [2023-12-27 11:00:22,150][20381] Updated weights for policy 1, policy_version 776732 (0.0007) [2023-12-27 11:00:22,161][20347] Updated weights for policy 0, policy_version 777265 (0.0008) [2023-12-27 11:00:22,210][20381] Updated weights for policy 1, policy_version 776742 (0.0007) [2023-12-27 11:00:22,223][20347] Updated weights for policy 0, policy_version 777275 (0.0008) [2023-12-27 11:00:22,275][20381] Updated weights for policy 1, policy_version 776752 (0.0007) [2023-12-27 11:00:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 397885440. Throughput: 0: 10350.5, 1: 10224.6. Samples: 397852404. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:22,363][19170] Avg episode reward: [(0, '438.950'), (1, '931.930')] [2023-12-27 11:00:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000777280_199008256.pth... [2023-12-27 11:00:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000776760_198877184.pth... [2023-12-27 11:00:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000775576_198574080.pth [2023-12-27 11:00:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000776064_198696960.pth [2023-12-27 11:00:22,958][20347] Updated weights for policy 0, policy_version 777285 (0.0007) [2023-12-27 11:00:22,963][20381] Updated weights for policy 1, policy_version 776762 (0.0006) [2023-12-27 11:00:23,019][20347] Updated weights for policy 0, policy_version 777295 (0.0008) [2023-12-27 11:00:23,022][20381] Updated weights for policy 1, policy_version 776772 (0.0011) [2023-12-27 11:00:23,073][20347] Updated weights for policy 0, policy_version 777305 (0.0006) [2023-12-27 11:00:23,076][20381] Updated weights for policy 1, policy_version 776782 (0.0010) [2023-12-27 11:00:23,131][20381] Updated weights for policy 1, policy_version 776792 (0.0011) [2023-12-27 11:00:23,758][20347] Updated weights for policy 0, policy_version 777315 (0.0007) [2023-12-27 11:00:23,819][20381] Updated weights for policy 1, policy_version 776802 (0.0005) [2023-12-27 11:00:23,821][20347] Updated weights for policy 0, policy_version 777325 (0.0010) [2023-12-27 11:00:23,876][20381] Updated weights for policy 1, policy_version 776812 (0.0005) [2023-12-27 11:00:23,877][20347] Updated weights for policy 0, policy_version 777335 (0.0011) [2023-12-27 11:00:23,928][20381] Updated weights for policy 1, policy_version 776822 (0.0007) [2023-12-27 11:00:24,521][20347] Updated weights for policy 0, policy_version 777345 (0.0007) [2023-12-27 11:00:24,582][20347] Updated weights for policy 0, policy_version 777355 (0.0008) [2023-12-27 11:00:24,592][20381] Updated weights for policy 1, policy_version 776832 (0.0006) [2023-12-27 11:00:24,634][20347] Updated weights for policy 0, policy_version 777365 (0.0010) [2023-12-27 11:00:24,650][20381] Updated weights for policy 1, policy_version 776842 (0.0005) [2023-12-27 11:00:24,689][20347] Updated weights for policy 0, policy_version 777375 (0.0010) [2023-12-27 11:00:24,698][20381] Updated weights for policy 1, policy_version 776852 (0.0005) [2023-12-27 11:00:25,334][20381] Updated weights for policy 1, policy_version 776862 (0.0007) [2023-12-27 11:00:25,357][20347] Updated weights for policy 0, policy_version 777385 (0.0006) [2023-12-27 11:00:25,391][20381] Updated weights for policy 1, policy_version 776872 (0.0006) [2023-12-27 11:00:25,414][20347] Updated weights for policy 0, policy_version 777395 (0.0008) [2023-12-27 11:00:25,451][20381] Updated weights for policy 1, policy_version 776882 (0.0007) [2023-12-27 11:00:25,471][20347] Updated weights for policy 0, policy_version 777405 (0.0005) [2023-12-27 11:00:26,043][20347] Updated weights for policy 0, policy_version 777415 (0.0009) [2023-12-27 11:00:26,110][20347] Updated weights for policy 0, policy_version 777425 (0.0008) [2023-12-27 11:00:26,142][20381] Updated weights for policy 1, policy_version 776892 (0.0007) [2023-12-27 11:00:26,178][20347] Updated weights for policy 0, policy_version 777435 (0.0010) [2023-12-27 11:00:26,197][20381] Updated weights for policy 1, policy_version 776902 (0.0006) [2023-12-27 11:00:26,256][20381] Updated weights for policy 1, policy_version 776912 (0.0007) [2023-12-27 11:00:26,835][20347] Updated weights for policy 0, policy_version 777445 (0.0010) [2023-12-27 11:00:26,888][20347] Updated weights for policy 0, policy_version 777455 (0.0010) [2023-12-27 11:00:26,924][20381] Updated weights for policy 1, policy_version 776922 (0.0007) [2023-12-27 11:00:26,941][20347] Updated weights for policy 0, policy_version 777465 (0.0008) [2023-12-27 11:00:26,975][20381] Updated weights for policy 1, policy_version 776932 (0.0006) [2023-12-27 11:00:27,023][20381] Updated weights for policy 1, policy_version 776942 (0.0007) [2023-12-27 11:00:27,076][20381] Updated weights for policy 1, policy_version 776952 (0.0008) [2023-12-27 11:00:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 397983744. Throughput: 0: 10325.5, 1: 10178.0. Samples: 397972244. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:27,363][19170] Avg episode reward: [(0, '466.580'), (1, '961.070')] [2023-12-27 11:00:27,635][20347] Updated weights for policy 0, policy_version 777475 (0.0007) [2023-12-27 11:00:27,700][20347] Updated weights for policy 0, policy_version 777485 (0.0006) [2023-12-27 11:00:27,764][20347] Updated weights for policy 0, policy_version 777495 (0.0008) [2023-12-27 11:00:27,770][20381] Updated weights for policy 1, policy_version 776962 (0.0005) [2023-12-27 11:00:27,820][20381] Updated weights for policy 1, policy_version 776972 (0.0005) [2023-12-27 11:00:27,871][20381] Updated weights for policy 1, policy_version 776982 (0.0005) [2023-12-27 11:00:28,450][20347] Updated weights for policy 0, policy_version 777505 (0.0008) [2023-12-27 11:00:28,506][20347] Updated weights for policy 0, policy_version 777515 (0.0008) [2023-12-27 11:00:28,561][20381] Updated weights for policy 1, policy_version 776992 (0.0009) [2023-12-27 11:00:28,563][20347] Updated weights for policy 0, policy_version 777525 (0.0008) [2023-12-27 11:00:28,619][20381] Updated weights for policy 1, policy_version 777002 (0.0007) [2023-12-27 11:00:28,622][20347] Updated weights for policy 0, policy_version 777535 (0.0006) [2023-12-27 11:00:28,673][20381] Updated weights for policy 1, policy_version 777012 (0.0008) [2023-12-27 11:00:29,247][20347] Updated weights for policy 0, policy_version 777545 (0.0008) [2023-12-27 11:00:29,305][20347] Updated weights for policy 0, policy_version 777555 (0.0007) [2023-12-27 11:00:29,368][20347] Updated weights for policy 0, policy_version 777565 (0.0009) [2023-12-27 11:00:29,448][20381] Updated weights for policy 1, policy_version 777022 (0.0010) [2023-12-27 11:00:29,494][20381] Updated weights for policy 1, policy_version 777032 (0.0007) [2023-12-27 11:00:29,555][20381] Updated weights for policy 1, policy_version 777042 (0.0005) [2023-12-27 11:00:30,073][20347] Updated weights for policy 0, policy_version 777575 (0.0010) [2023-12-27 11:00:30,130][20347] Updated weights for policy 0, policy_version 777585 (0.0010) [2023-12-27 11:00:30,189][20347] Updated weights for policy 0, policy_version 777595 (0.0011) [2023-12-27 11:00:30,195][20381] Updated weights for policy 1, policy_version 777052 (0.0011) [2023-12-27 11:00:30,244][20381] Updated weights for policy 1, policy_version 777062 (0.0010) [2023-12-27 11:00:30,293][20381] Updated weights for policy 1, policy_version 777072 (0.0010) [2023-12-27 11:00:30,844][20347] Updated weights for policy 0, policy_version 777605 (0.0010) [2023-12-27 11:00:30,892][20347] Updated weights for policy 0, policy_version 777615 (0.0010) [2023-12-27 11:00:30,943][20347] Updated weights for policy 0, policy_version 777625 (0.0010) [2023-12-27 11:00:30,989][20381] Updated weights for policy 1, policy_version 777082 (0.0010) [2023-12-27 11:00:31,051][20381] Updated weights for policy 1, policy_version 777092 (0.0007) [2023-12-27 11:00:31,118][20381] Updated weights for policy 1, policy_version 777102 (0.0008) [2023-12-27 11:00:31,176][20381] Updated weights for policy 1, policy_version 777112 (0.0008) [2023-12-27 11:00:31,702][20347] Updated weights for policy 0, policy_version 777635 (0.0009) [2023-12-27 11:00:31,775][20347] Updated weights for policy 0, policy_version 777645 (0.0007) [2023-12-27 11:00:31,835][20347] Updated weights for policy 0, policy_version 777655 (0.0006) [2023-12-27 11:00:31,915][20381] Updated weights for policy 1, policy_version 777122 (0.0006) [2023-12-27 11:00:31,976][20381] Updated weights for policy 1, policy_version 777132 (0.0007) [2023-12-27 11:00:32,029][20381] Updated weights for policy 1, policy_version 777142 (0.0008) [2023-12-27 11:00:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 398082048. Throughput: 0: 10308.6, 1: 10067.2. Samples: 398092100. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:32,363][19170] Avg episode reward: [(0, '640.270'), (1, '970.780')] [2023-12-27 11:00:32,504][20347] Updated weights for policy 0, policy_version 777665 (0.0006) [2023-12-27 11:00:32,561][20347] Updated weights for policy 0, policy_version 777675 (0.0007) [2023-12-27 11:00:32,618][20347] Updated weights for policy 0, policy_version 777685 (0.0007) [2023-12-27 11:00:32,680][20347] Updated weights for policy 0, policy_version 777695 (0.0007) [2023-12-27 11:00:32,728][20381] Updated weights for policy 1, policy_version 777152 (0.0008) [2023-12-27 11:00:32,783][20381] Updated weights for policy 1, policy_version 777162 (0.0008) [2023-12-27 11:00:32,845][20381] Updated weights for policy 1, policy_version 777172 (0.0008) [2023-12-27 11:00:33,337][20347] Updated weights for policy 0, policy_version 777705 (0.0008) [2023-12-27 11:00:33,384][20347] Updated weights for policy 0, policy_version 777715 (0.0008) [2023-12-27 11:00:33,436][20347] Updated weights for policy 0, policy_version 777725 (0.0008) [2023-12-27 11:00:33,508][20381] Updated weights for policy 1, policy_version 777182 (0.0008) [2023-12-27 11:00:33,563][20381] Updated weights for policy 1, policy_version 777192 (0.0007) [2023-12-27 11:00:33,609][20381] Updated weights for policy 1, policy_version 777202 (0.0011) [2023-12-27 11:00:34,181][20347] Updated weights for policy 0, policy_version 777735 (0.0008) [2023-12-27 11:00:34,233][20347] Updated weights for policy 0, policy_version 777745 (0.0006) [2023-12-27 11:00:34,251][20381] Updated weights for policy 1, policy_version 777212 (0.0009) [2023-12-27 11:00:34,282][20347] Updated weights for policy 0, policy_version 777755 (0.0006) [2023-12-27 11:00:34,310][20381] Updated weights for policy 1, policy_version 777222 (0.0008) [2023-12-27 11:00:34,372][20381] Updated weights for policy 1, policy_version 777232 (0.0008) [2023-12-27 11:00:35,003][20381] Updated weights for policy 1, policy_version 777242 (0.0008) [2023-12-27 11:00:35,037][20347] Updated weights for policy 0, policy_version 777765 (0.0006) [2023-12-27 11:00:35,059][20381] Updated weights for policy 1, policy_version 777252 (0.0008) [2023-12-27 11:00:35,103][20347] Updated weights for policy 0, policy_version 777775 (0.0006) [2023-12-27 11:00:35,125][20381] Updated weights for policy 1, policy_version 777262 (0.0007) [2023-12-27 11:00:35,169][20347] Updated weights for policy 0, policy_version 777785 (0.0006) [2023-12-27 11:00:35,184][20381] Updated weights for policy 1, policy_version 777272 (0.0006) [2023-12-27 11:00:35,751][20381] Updated weights for policy 1, policy_version 777282 (0.0008) [2023-12-27 11:00:35,757][20347] Updated weights for policy 0, policy_version 777795 (0.0008) [2023-12-27 11:00:35,801][20381] Updated weights for policy 1, policy_version 777292 (0.0008) [2023-12-27 11:00:35,814][20347] Updated weights for policy 0, policy_version 777805 (0.0005) [2023-12-27 11:00:35,853][20381] Updated weights for policy 1, policy_version 777302 (0.0011) [2023-12-27 11:00:35,866][20347] Updated weights for policy 0, policy_version 777815 (0.0005) [2023-12-27 11:00:36,434][20381] Updated weights for policy 1, policy_version 777312 (0.0011) [2023-12-27 11:00:36,495][20381] Updated weights for policy 1, policy_version 777322 (0.0011) [2023-12-27 11:00:36,552][20381] Updated weights for policy 1, policy_version 777332 (0.0011) [2023-12-27 11:00:36,562][20347] Updated weights for policy 0, policy_version 777825 (0.0007) [2023-12-27 11:00:36,623][20347] Updated weights for policy 0, policy_version 777835 (0.0007) [2023-12-27 11:00:36,679][20347] Updated weights for policy 0, policy_version 777845 (0.0008) [2023-12-27 11:00:36,738][20347] Updated weights for policy 0, policy_version 777855 (0.0005) [2023-12-27 11:00:37,241][20381] Updated weights for policy 1, policy_version 777342 (0.0007) [2023-12-27 11:00:37,288][20381] Updated weights for policy 1, policy_version 777352 (0.0005) [2023-12-27 11:00:37,339][20381] Updated weights for policy 1, policy_version 777362 (0.0006) [2023-12-27 11:00:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 398180352. Throughput: 0: 10284.3, 1: 10047.5. Samples: 398153352. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:37,363][19170] Avg episode reward: [(0, '746.380'), (1, '932.020')] [2023-12-27 11:00:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000777368_199032832.pth... [2023-12-27 11:00:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000776184_198729728.pth [2023-12-27 11:00:37,413][20347] Updated weights for policy 0, policy_version 777865 (0.0008) [2023-12-27 11:00:37,470][20347] Updated weights for policy 0, policy_version 777875 (0.0007) [2023-12-27 11:00:37,518][20347] Updated weights for policy 0, policy_version 777885 (0.0006) [2023-12-27 11:00:37,533][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000777888_199163904.pth... [2023-12-27 11:00:37,537][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000776672_198852608.pth [2023-12-27 11:00:38,031][20381] Updated weights for policy 1, policy_version 777372 (0.0008) [2023-12-27 11:00:38,086][20381] Updated weights for policy 1, policy_version 777382 (0.0011) [2023-12-27 11:00:38,138][20381] Updated weights for policy 1, policy_version 777392 (0.0011) [2023-12-27 11:00:38,194][20347] Updated weights for policy 0, policy_version 777895 (0.0007) [2023-12-27 11:00:38,242][20347] Updated weights for policy 0, policy_version 777905 (0.0007) [2023-12-27 11:00:38,291][20347] Updated weights for policy 0, policy_version 777915 (0.0008) [2023-12-27 11:00:38,845][20381] Updated weights for policy 1, policy_version 777402 (0.0011) [2023-12-27 11:00:38,899][20381] Updated weights for policy 1, policy_version 777412 (0.0011) [2023-12-27 11:00:38,958][20381] Updated weights for policy 1, policy_version 777422 (0.0011) [2023-12-27 11:00:39,007][20381] Updated weights for policy 1, policy_version 777432 (0.0010) [2023-12-27 11:00:39,034][20347] Updated weights for policy 0, policy_version 777925 (0.0007) [2023-12-27 11:00:39,085][20347] Updated weights for policy 0, policy_version 777935 (0.0007) [2023-12-27 11:00:39,142][20347] Updated weights for policy 0, policy_version 777945 (0.0007) [2023-12-27 11:00:39,696][20381] Updated weights for policy 1, policy_version 777442 (0.0010) [2023-12-27 11:00:39,754][20381] Updated weights for policy 1, policy_version 777452 (0.0010) [2023-12-27 11:00:39,814][20381] Updated weights for policy 1, policy_version 777462 (0.0011) [2023-12-27 11:00:39,935][20347] Updated weights for policy 0, policy_version 777955 (0.0007) [2023-12-27 11:00:39,993][20347] Updated weights for policy 0, policy_version 777965 (0.0007) [2023-12-27 11:00:40,050][20347] Updated weights for policy 0, policy_version 777975 (0.0008) [2023-12-27 11:00:40,578][20381] Updated weights for policy 1, policy_version 777472 (0.0009) [2023-12-27 11:00:40,634][20381] Updated weights for policy 1, policy_version 777482 (0.0008) [2023-12-27 11:00:40,689][20381] Updated weights for policy 1, policy_version 777492 (0.0008) [2023-12-27 11:00:40,778][20347] Updated weights for policy 0, policy_version 777985 (0.0007) [2023-12-27 11:00:40,836][20347] Updated weights for policy 0, policy_version 777995 (0.0008) [2023-12-27 11:00:40,896][20347] Updated weights for policy 0, policy_version 778005 (0.0007) [2023-12-27 11:00:40,956][20347] Updated weights for policy 0, policy_version 778015 (0.0007) [2023-12-27 11:00:41,463][20381] Updated weights for policy 1, policy_version 777502 (0.0010) [2023-12-27 11:00:41,525][20381] Updated weights for policy 1, policy_version 777512 (0.0011) [2023-12-27 11:00:41,578][20381] Updated weights for policy 1, policy_version 777522 (0.0011) [2023-12-27 11:00:41,692][20347] Updated weights for policy 0, policy_version 778025 (0.0006) [2023-12-27 11:00:41,763][20347] Updated weights for policy 0, policy_version 778035 (0.0008) [2023-12-27 11:00:41,825][20347] Updated weights for policy 0, policy_version 778045 (0.0007) [2023-12-27 11:00:42,286][20381] Updated weights for policy 1, policy_version 777532 (0.0009) [2023-12-27 11:00:42,353][20381] Updated weights for policy 1, policy_version 777542 (0.0007) [2023-12-27 11:00:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 398278656. Throughput: 0: 10228.4, 1: 10086.5. Samples: 398274892. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:42,365][19170] Avg episode reward: [(0, '707.290'), (1, '941.810')] [2023-12-27 11:00:42,414][20381] Updated weights for policy 1, policy_version 777552 (0.0009) [2023-12-27 11:00:42,496][20347] Updated weights for policy 0, policy_version 778055 (0.0006) [2023-12-27 11:00:42,551][20347] Updated weights for policy 0, policy_version 778065 (0.0005) [2023-12-27 11:00:42,610][20347] Updated weights for policy 0, policy_version 778075 (0.0006) [2023-12-27 11:00:43,135][20381] Updated weights for policy 1, policy_version 777562 (0.0007) [2023-12-27 11:00:43,185][20381] Updated weights for policy 1, policy_version 777572 (0.0005) [2023-12-27 11:00:43,242][20381] Updated weights for policy 1, policy_version 777582 (0.0005) [2023-12-27 11:00:43,255][20347] Updated weights for policy 0, policy_version 778085 (0.0008) [2023-12-27 11:00:43,295][20381] Updated weights for policy 1, policy_version 777592 (0.0005) [2023-12-27 11:00:43,312][20347] Updated weights for policy 0, policy_version 778095 (0.0008) [2023-12-27 11:00:43,364][20347] Updated weights for policy 0, policy_version 778105 (0.0007) [2023-12-27 11:00:43,849][20381] Updated weights for policy 1, policy_version 777602 (0.0007) [2023-12-27 11:00:43,901][20381] Updated weights for policy 1, policy_version 777612 (0.0009) [2023-12-27 11:00:43,954][20381] Updated weights for policy 1, policy_version 777622 (0.0007) [2023-12-27 11:00:44,113][20347] Updated weights for policy 0, policy_version 778115 (0.0007) [2023-12-27 11:00:44,160][20347] Updated weights for policy 0, policy_version 778125 (0.0007) [2023-12-27 11:00:44,209][20347] Updated weights for policy 0, policy_version 778135 (0.0009) [2023-12-27 11:00:44,528][20381] Updated weights for policy 1, policy_version 777632 (0.0006) [2023-12-27 11:00:44,578][20381] Updated weights for policy 1, policy_version 777642 (0.0010) [2023-12-27 11:00:44,633][20381] Updated weights for policy 1, policy_version 777652 (0.0011) [2023-12-27 11:00:44,937][20347] Updated weights for policy 0, policy_version 778145 (0.0008) [2023-12-27 11:00:44,998][20347] Updated weights for policy 0, policy_version 778155 (0.0007) [2023-12-27 11:00:45,055][20347] Updated weights for policy 0, policy_version 778165 (0.0007) [2023-12-27 11:00:45,104][20347] Updated weights for policy 0, policy_version 778175 (0.0007) [2023-12-27 11:00:45,331][20381] Updated weights for policy 1, policy_version 777662 (0.0010) [2023-12-27 11:00:45,384][20381] Updated weights for policy 1, policy_version 777672 (0.0010) [2023-12-27 11:00:45,440][20381] Updated weights for policy 1, policy_version 777682 (0.0011) [2023-12-27 11:00:45,793][20347] Updated weights for policy 0, policy_version 778185 (0.0007) [2023-12-27 11:00:45,847][20347] Updated weights for policy 0, policy_version 778195 (0.0007) [2023-12-27 11:00:45,894][20347] Updated weights for policy 0, policy_version 778205 (0.0008) [2023-12-27 11:00:46,086][20381] Updated weights for policy 1, policy_version 777692 (0.0008) [2023-12-27 11:00:46,142][20381] Updated weights for policy 1, policy_version 777702 (0.0010) [2023-12-27 11:00:46,198][20381] Updated weights for policy 1, policy_version 777712 (0.0010) [2023-12-27 11:00:46,683][20347] Updated weights for policy 0, policy_version 778215 (0.0007) [2023-12-27 11:00:46,765][20347] Updated weights for policy 0, policy_version 778225 (0.0008) [2023-12-27 11:00:46,812][20347] Updated weights for policy 0, policy_version 778235 (0.0007) [2023-12-27 11:00:46,865][20381] Updated weights for policy 1, policy_version 777722 (0.0010) [2023-12-27 11:00:46,930][20381] Updated weights for policy 1, policy_version 777732 (0.0007) [2023-12-27 11:00:46,975][20381] Updated weights for policy 1, policy_version 777742 (0.0007) [2023-12-27 11:00:47,026][20381] Updated weights for policy 1, policy_version 777752 (0.0007) [2023-12-27 11:00:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 398385152. Throughput: 0: 10163.3, 1: 10164.3. Samples: 398396952. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:47,363][19170] Avg episode reward: [(0, '697.670'), (1, '961.230')] [2023-12-27 11:00:47,502][20347] Updated weights for policy 0, policy_version 778245 (0.0007) [2023-12-27 11:00:47,565][20347] Updated weights for policy 0, policy_version 778255 (0.0009) [2023-12-27 11:00:47,627][20347] Updated weights for policy 0, policy_version 778265 (0.0006) [2023-12-27 11:00:47,629][20381] Updated weights for policy 1, policy_version 777762 (0.0007) [2023-12-27 11:00:47,676][20381] Updated weights for policy 1, policy_version 777772 (0.0006) [2023-12-27 11:00:47,724][20381] Updated weights for policy 1, policy_version 777782 (0.0006) [2023-12-27 11:00:48,326][20347] Updated weights for policy 0, policy_version 778275 (0.0007) [2023-12-27 11:00:48,359][20381] Updated weights for policy 1, policy_version 777792 (0.0007) [2023-12-27 11:00:48,394][20347] Updated weights for policy 0, policy_version 778285 (0.0007) [2023-12-27 11:00:48,426][20381] Updated weights for policy 1, policy_version 777802 (0.0006) [2023-12-27 11:00:48,460][20347] Updated weights for policy 0, policy_version 778295 (0.0006) [2023-12-27 11:00:48,490][20381] Updated weights for policy 1, policy_version 777812 (0.0006) [2023-12-27 11:00:49,087][20381] Updated weights for policy 1, policy_version 777822 (0.0009) [2023-12-27 11:00:49,148][20347] Updated weights for policy 0, policy_version 778305 (0.0006) [2023-12-27 11:00:49,149][20381] Updated weights for policy 1, policy_version 777832 (0.0010) [2023-12-27 11:00:49,204][20381] Updated weights for policy 1, policy_version 777842 (0.0008) [2023-12-27 11:00:49,217][20347] Updated weights for policy 0, policy_version 778315 (0.0007) [2023-12-27 11:00:49,286][20347] Updated weights for policy 0, policy_version 778325 (0.0006) [2023-12-27 11:00:49,353][20347] Updated weights for policy 0, policy_version 778335 (0.0006) [2023-12-27 11:00:49,934][20381] Updated weights for policy 1, policy_version 777852 (0.0011) [2023-12-27 11:00:49,986][20381] Updated weights for policy 1, policy_version 777862 (0.0010) [2023-12-27 11:00:50,011][20347] Updated weights for policy 0, policy_version 778345 (0.0006) [2023-12-27 11:00:50,039][20381] Updated weights for policy 1, policy_version 777872 (0.0010) [2023-12-27 11:00:50,065][20347] Updated weights for policy 0, policy_version 778355 (0.0007) [2023-12-27 11:00:50,122][20347] Updated weights for policy 0, policy_version 778365 (0.0007) [2023-12-27 11:00:50,735][20381] Updated weights for policy 1, policy_version 777882 (0.0011) [2023-12-27 11:00:50,788][20381] Updated weights for policy 1, policy_version 777892 (0.0011) [2023-12-27 11:00:50,840][20347] Updated weights for policy 0, policy_version 778375 (0.0008) [2023-12-27 11:00:50,846][20381] Updated weights for policy 1, policy_version 777902 (0.0008) [2023-12-27 11:00:50,896][20347] Updated weights for policy 0, policy_version 778385 (0.0005) [2023-12-27 11:00:50,908][20381] Updated weights for policy 1, policy_version 777912 (0.0007) [2023-12-27 11:00:50,964][20347] Updated weights for policy 0, policy_version 778395 (0.0008) [2023-12-27 11:00:51,623][20381] Updated weights for policy 1, policy_version 777922 (0.0011) [2023-12-27 11:00:51,691][20381] Updated weights for policy 1, policy_version 777932 (0.0010) [2023-12-27 11:00:51,746][20347] Updated weights for policy 0, policy_version 778405 (0.0008) [2023-12-27 11:00:51,750][20381] Updated weights for policy 1, policy_version 777942 (0.0009) [2023-12-27 11:00:51,805][20347] Updated weights for policy 0, policy_version 778415 (0.0008) [2023-12-27 11:00:51,870][20347] Updated weights for policy 0, policy_version 778425 (0.0008) [2023-12-27 11:00:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 398483456. Throughput: 0: 10097.3, 1: 10194.1. Samples: 398458308. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:52,363][19170] Avg episode reward: [(0, '717.580'), (1, '941.050')] [2023-12-27 11:00:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000778432_199303168.pth... [2023-12-27 11:00:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000777280_199008256.pth [2023-12-27 11:00:52,391][20381] Updated weights for policy 1, policy_version 777952 (0.0007) [2023-12-27 11:00:52,444][20381] Updated weights for policy 1, policy_version 777962 (0.0007) [2023-12-27 11:00:52,503][20381] Updated weights for policy 1, policy_version 777972 (0.0008) [2023-12-27 11:00:52,507][20347] Updated weights for policy 0, policy_version 778435 (0.0007) [2023-12-27 11:00:52,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000777976_199188480.pth... [2023-12-27 11:00:52,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000776760_198877184.pth [2023-12-27 11:00:52,571][20347] Updated weights for policy 0, policy_version 778445 (0.0008) [2023-12-27 11:00:52,635][20347] Updated weights for policy 0, policy_version 778455 (0.0007) [2023-12-27 11:00:53,108][20381] Updated weights for policy 1, policy_version 777982 (0.0009) [2023-12-27 11:00:53,174][20381] Updated weights for policy 1, policy_version 777992 (0.0008) [2023-12-27 11:00:53,239][20381] Updated weights for policy 1, policy_version 778002 (0.0008) [2023-12-27 11:00:53,311][20347] Updated weights for policy 0, policy_version 778465 (0.0008) [2023-12-27 11:00:53,362][20347] Updated weights for policy 0, policy_version 778475 (0.0005) [2023-12-27 11:00:53,408][20347] Updated weights for policy 0, policy_version 778485 (0.0005) [2023-12-27 11:00:53,461][20347] Updated weights for policy 0, policy_version 778495 (0.0005) [2023-12-27 11:00:53,791][20381] Updated weights for policy 1, policy_version 778012 (0.0005) [2023-12-27 11:00:53,841][20381] Updated weights for policy 1, policy_version 778022 (0.0007) [2023-12-27 11:00:53,883][20381] Updated weights for policy 1, policy_version 778032 (0.0007) [2023-12-27 11:00:54,050][20347] Updated weights for policy 0, policy_version 778505 (0.0006) [2023-12-27 11:00:54,109][20347] Updated weights for policy 0, policy_version 778515 (0.0005) [2023-12-27 11:00:54,165][20347] Updated weights for policy 0, policy_version 778525 (0.0005) [2023-12-27 11:00:54,542][20381] Updated weights for policy 1, policy_version 778042 (0.0006) [2023-12-27 11:00:54,588][20381] Updated weights for policy 1, policy_version 778052 (0.0005) [2023-12-27 11:00:54,646][20381] Updated weights for policy 1, policy_version 778062 (0.0008) [2023-12-27 11:00:54,698][20381] Updated weights for policy 1, policy_version 778072 (0.0008) [2023-12-27 11:00:54,788][20347] Updated weights for policy 0, policy_version 778535 (0.0007) [2023-12-27 11:00:54,836][20347] Updated weights for policy 0, policy_version 778545 (0.0007) [2023-12-27 11:00:54,880][20347] Updated weights for policy 0, policy_version 778555 (0.0007) [2023-12-27 11:00:55,369][20381] Updated weights for policy 1, policy_version 778082 (0.0005) [2023-12-27 11:00:55,425][20381] Updated weights for policy 1, policy_version 778092 (0.0005) [2023-12-27 11:00:55,471][20381] Updated weights for policy 1, policy_version 778102 (0.0007) [2023-12-27 11:00:55,497][20347] Updated weights for policy 0, policy_version 778565 (0.0008) [2023-12-27 11:00:55,550][20347] Updated weights for policy 0, policy_version 778575 (0.0006) [2023-12-27 11:00:55,605][20347] Updated weights for policy 0, policy_version 778585 (0.0005) [2023-12-27 11:00:56,068][20381] Updated weights for policy 1, policy_version 778112 (0.0010) [2023-12-27 11:00:56,123][20381] Updated weights for policy 1, policy_version 778122 (0.0010) [2023-12-27 11:00:56,172][20347] Updated weights for policy 0, policy_version 778595 (0.0005) [2023-12-27 11:00:56,178][20381] Updated weights for policy 1, policy_version 778132 (0.0009) [2023-12-27 11:00:56,238][20347] Updated weights for policy 0, policy_version 778605 (0.0009) [2023-12-27 11:00:56,304][20347] Updated weights for policy 0, policy_version 778615 (0.0010) [2023-12-27 11:00:56,864][20381] Updated weights for policy 1, policy_version 778142 (0.0007) [2023-12-27 11:00:56,924][20381] Updated weights for policy 1, policy_version 778152 (0.0007) [2023-12-27 11:00:56,934][20347] Updated weights for policy 0, policy_version 778625 (0.0010) [2023-12-27 11:00:56,978][20381] Updated weights for policy 1, policy_version 778162 (0.0005) [2023-12-27 11:00:56,989][20347] Updated weights for policy 0, policy_version 778635 (0.0007) [2023-12-27 11:00:57,044][20347] Updated weights for policy 0, policy_version 778645 (0.0006) [2023-12-27 11:00:57,097][20347] Updated weights for policy 0, policy_version 778655 (0.0007) [2023-12-27 11:00:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 398598144. Throughput: 0: 10128.1, 1: 10238.8. Samples: 398585880. Policy #0 lag: (min: 12.0, avg: 14.1, max: 36.0) [2023-12-27 11:00:57,363][19170] Avg episode reward: [(0, '805.150'), (1, '960.450')] [2023-12-27 11:00:57,611][20381] Updated weights for policy 1, policy_version 778172 (0.0007) [2023-12-27 11:00:57,667][20381] Updated weights for policy 1, policy_version 778182 (0.0007) [2023-12-27 11:00:57,727][20381] Updated weights for policy 1, policy_version 778192 (0.0007) [2023-12-27 11:00:57,743][20347] Updated weights for policy 0, policy_version 778665 (0.0005) [2023-12-27 11:00:57,791][20347] Updated weights for policy 0, policy_version 778675 (0.0005) [2023-12-27 11:00:57,843][20347] Updated weights for policy 0, policy_version 778685 (0.0005) [2023-12-27 11:00:58,459][20381] Updated weights for policy 1, policy_version 778202 (0.0007) [2023-12-27 11:00:58,519][20381] Updated weights for policy 1, policy_version 778212 (0.0008) [2023-12-27 11:00:58,525][20347] Updated weights for policy 0, policy_version 778695 (0.0009) [2023-12-27 11:00:58,579][20381] Updated weights for policy 1, policy_version 778222 (0.0008) [2023-12-27 11:00:58,586][20347] Updated weights for policy 0, policy_version 778705 (0.0010) [2023-12-27 11:00:58,638][20381] Updated weights for policy 1, policy_version 778232 (0.0008) [2023-12-27 11:00:58,646][20347] Updated weights for policy 0, policy_version 778715 (0.0009) [2023-12-27 11:00:59,471][20381] Updated weights for policy 1, policy_version 778242 (0.0006) [2023-12-27 11:00:59,474][20347] Updated weights for policy 0, policy_version 778725 (0.0009) [2023-12-27 11:00:59,523][20381] Updated weights for policy 1, policy_version 778252 (0.0006) [2023-12-27 11:00:59,533][20347] Updated weights for policy 0, policy_version 778735 (0.0010) [2023-12-27 11:00:59,579][20381] Updated weights for policy 1, policy_version 778262 (0.0007) [2023-12-27 11:00:59,593][20347] Updated weights for policy 0, policy_version 778745 (0.0011) [2023-12-27 11:01:00,246][20381] Updated weights for policy 1, policy_version 778272 (0.0007) [2023-12-27 11:01:00,258][20347] Updated weights for policy 0, policy_version 778755 (0.0010) [2023-12-27 11:01:00,298][20381] Updated weights for policy 1, policy_version 778282 (0.0006) [2023-12-27 11:01:00,313][20347] Updated weights for policy 0, policy_version 778765 (0.0010) [2023-12-27 11:01:00,350][20381] Updated weights for policy 1, policy_version 778292 (0.0006) [2023-12-27 11:01:00,370][20347] Updated weights for policy 0, policy_version 778775 (0.0011) [2023-12-27 11:01:00,983][20347] Updated weights for policy 0, policy_version 778785 (0.0010) [2023-12-27 11:01:01,025][20381] Updated weights for policy 1, policy_version 778302 (0.0006) [2023-12-27 11:01:01,045][20347] Updated weights for policy 0, policy_version 778795 (0.0007) [2023-12-27 11:01:01,087][20381] Updated weights for policy 1, policy_version 778312 (0.0006) [2023-12-27 11:01:01,100][20347] Updated weights for policy 0, policy_version 778805 (0.0007) [2023-12-27 11:01:01,150][20381] Updated weights for policy 1, policy_version 778322 (0.0007) [2023-12-27 11:01:01,162][20347] Updated weights for policy 0, policy_version 778815 (0.0008) [2023-12-27 11:01:01,785][20381] Updated weights for policy 1, policy_version 778332 (0.0008) [2023-12-27 11:01:01,836][20381] Updated weights for policy 1, policy_version 778342 (0.0007) [2023-12-27 11:01:01,891][20381] Updated weights for policy 1, policy_version 778352 (0.0008) [2023-12-27 11:01:01,934][20347] Updated weights for policy 0, policy_version 778825 (0.0008) [2023-12-27 11:01:01,993][20347] Updated weights for policy 0, policy_version 778835 (0.0008) [2023-12-27 11:01:02,054][20347] Updated weights for policy 0, policy_version 778845 (0.0008) [2023-12-27 11:01:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 398696448. Throughput: 0: 10056.6, 1: 10258.8. Samples: 398705736. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:02,363][19170] Avg episode reward: [(0, '746.040'), (1, '951.540')] [2023-12-27 11:01:02,604][20381] Updated weights for policy 1, policy_version 778362 (0.0007) [2023-12-27 11:01:02,660][20381] Updated weights for policy 1, policy_version 778372 (0.0005) [2023-12-27 11:01:02,720][20381] Updated weights for policy 1, policy_version 778382 (0.0007) [2023-12-27 11:01:02,738][20347] Updated weights for policy 0, policy_version 778855 (0.0009) [2023-12-27 11:01:02,776][20381] Updated weights for policy 1, policy_version 778392 (0.0006) [2023-12-27 11:01:02,795][20347] Updated weights for policy 0, policy_version 778865 (0.0011) [2023-12-27 11:01:02,856][20347] Updated weights for policy 0, policy_version 778875 (0.0010) [2023-12-27 11:01:03,397][20381] Updated weights for policy 1, policy_version 778402 (0.0006) [2023-12-27 11:01:03,445][20381] Updated weights for policy 1, policy_version 778412 (0.0007) [2023-12-27 11:01:03,448][20347] Updated weights for policy 0, policy_version 778885 (0.0008) [2023-12-27 11:01:03,500][20381] Updated weights for policy 1, policy_version 778422 (0.0008) [2023-12-27 11:01:03,500][20347] Updated weights for policy 0, policy_version 778895 (0.0006) [2023-12-27 11:01:03,548][20347] Updated weights for policy 0, policy_version 778905 (0.0009) [2023-12-27 11:01:04,133][20347] Updated weights for policy 0, policy_version 778915 (0.0007) [2023-12-27 11:01:04,187][20381] Updated weights for policy 1, policy_version 778432 (0.0008) [2023-12-27 11:01:04,192][20347] Updated weights for policy 0, policy_version 778925 (0.0006) [2023-12-27 11:01:04,246][20381] Updated weights for policy 1, policy_version 778442 (0.0008) [2023-12-27 11:01:04,257][20347] Updated weights for policy 0, policy_version 778935 (0.0009) [2023-12-27 11:01:04,304][20381] Updated weights for policy 1, policy_version 778452 (0.0005) [2023-12-27 11:01:04,921][20347] Updated weights for policy 0, policy_version 778945 (0.0010) [2023-12-27 11:01:04,972][20347] Updated weights for policy 0, policy_version 778955 (0.0010) [2023-12-27 11:01:05,007][20381] Updated weights for policy 1, policy_version 778462 (0.0008) [2023-12-27 11:01:05,029][20347] Updated weights for policy 0, policy_version 778965 (0.0010) [2023-12-27 11:01:05,063][20381] Updated weights for policy 1, policy_version 778472 (0.0007) [2023-12-27 11:01:05,080][20347] Updated weights for policy 0, policy_version 778975 (0.0010) [2023-12-27 11:01:05,109][20381] Updated weights for policy 1, policy_version 778482 (0.0006) [2023-12-27 11:01:05,662][20347] Updated weights for policy 0, policy_version 778985 (0.0010) [2023-12-27 11:01:05,692][20381] Updated weights for policy 1, policy_version 778492 (0.0007) [2023-12-27 11:01:05,714][20347] Updated weights for policy 0, policy_version 778995 (0.0010) [2023-12-27 11:01:05,743][20381] Updated weights for policy 1, policy_version 778502 (0.0005) [2023-12-27 11:01:05,768][20347] Updated weights for policy 0, policy_version 779005 (0.0010) [2023-12-27 11:01:05,796][20381] Updated weights for policy 1, policy_version 778512 (0.0005) [2023-12-27 11:01:06,412][20347] Updated weights for policy 0, policy_version 779015 (0.0008) [2023-12-27 11:01:06,417][20381] Updated weights for policy 1, policy_version 778522 (0.0006) [2023-12-27 11:01:06,475][20381] Updated weights for policy 1, policy_version 778532 (0.0009) [2023-12-27 11:01:06,478][20347] Updated weights for policy 0, policy_version 779025 (0.0006) [2023-12-27 11:01:06,535][20381] Updated weights for policy 1, policy_version 778542 (0.0007) [2023-12-27 11:01:06,546][20347] Updated weights for policy 0, policy_version 779035 (0.0006) [2023-12-27 11:01:06,595][20381] Updated weights for policy 1, policy_version 778552 (0.0007) [2023-12-27 11:01:07,154][20347] Updated weights for policy 0, policy_version 779045 (0.0006) [2023-12-27 11:01:07,203][20347] Updated weights for policy 0, policy_version 779055 (0.0007) [2023-12-27 11:01:07,250][20347] Updated weights for policy 0, policy_version 779065 (0.0005) [2023-12-27 11:01:07,315][20381] Updated weights for policy 1, policy_version 778562 (0.0007) [2023-12-27 11:01:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20632.7). Total num frames: 398802944. Throughput: 0: 10114.2, 1: 10250.4. Samples: 398768812. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:07,363][19170] Avg episode reward: [(0, '736.890'), (1, '951.500')] [2023-12-27 11:01:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000779072_199467008.pth... [2023-12-27 11:01:07,368][20381] Updated weights for policy 1, policy_version 778572 (0.0007) [2023-12-27 11:01:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000777888_199163904.pth [2023-12-27 11:01:07,428][20381] Updated weights for policy 1, policy_version 778582 (0.0005) [2023-12-27 11:01:07,437][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000778584_199344128.pth... [2023-12-27 11:01:07,440][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000777368_199032832.pth [2023-12-27 11:01:07,848][20347] Updated weights for policy 0, policy_version 779075 (0.0006) [2023-12-27 11:01:07,893][20347] Updated weights for policy 0, policy_version 779085 (0.0007) [2023-12-27 11:01:07,938][20347] Updated weights for policy 0, policy_version 779095 (0.0007) [2023-12-27 11:01:08,107][20381] Updated weights for policy 1, policy_version 778592 (0.0007) [2023-12-27 11:01:08,165][20381] Updated weights for policy 1, policy_version 778602 (0.0008) [2023-12-27 11:01:08,222][20381] Updated weights for policy 1, policy_version 778612 (0.0008) [2023-12-27 11:01:08,678][20347] Updated weights for policy 0, policy_version 779105 (0.0007) [2023-12-27 11:01:08,734][20347] Updated weights for policy 0, policy_version 779115 (0.0007) [2023-12-27 11:01:08,784][20347] Updated weights for policy 0, policy_version 779125 (0.0008) [2023-12-27 11:01:08,847][20347] Updated weights for policy 0, policy_version 779135 (0.0007) [2023-12-27 11:01:08,920][20381] Updated weights for policy 1, policy_version 778622 (0.0008) [2023-12-27 11:01:08,979][20381] Updated weights for policy 1, policy_version 778632 (0.0008) [2023-12-27 11:01:09,036][20381] Updated weights for policy 1, policy_version 778642 (0.0007) [2023-12-27 11:01:09,546][20347] Updated weights for policy 0, policy_version 779145 (0.0007) [2023-12-27 11:01:09,597][20347] Updated weights for policy 0, policy_version 779155 (0.0008) [2023-12-27 11:01:09,657][20347] Updated weights for policy 0, policy_version 779165 (0.0009) [2023-12-27 11:01:09,732][20381] Updated weights for policy 1, policy_version 778652 (0.0007) [2023-12-27 11:01:09,792][20381] Updated weights for policy 1, policy_version 778662 (0.0008) [2023-12-27 11:01:09,857][20381] Updated weights for policy 1, policy_version 778672 (0.0007) [2023-12-27 11:01:10,407][20347] Updated weights for policy 0, policy_version 779175 (0.0008) [2023-12-27 11:01:10,460][20347] Updated weights for policy 0, policy_version 779185 (0.0008) [2023-12-27 11:01:10,514][20347] Updated weights for policy 0, policy_version 779195 (0.0007) [2023-12-27 11:01:10,583][20381] Updated weights for policy 1, policy_version 778682 (0.0008) [2023-12-27 11:01:10,632][20381] Updated weights for policy 1, policy_version 778692 (0.0008) [2023-12-27 11:01:10,677][20381] Updated weights for policy 1, policy_version 778702 (0.0009) [2023-12-27 11:01:10,731][20381] Updated weights for policy 1, policy_version 778712 (0.0008) [2023-12-27 11:01:11,124][20347] Updated weights for policy 0, policy_version 779205 (0.0007) [2023-12-27 11:01:11,193][20347] Updated weights for policy 0, policy_version 779215 (0.0007) [2023-12-27 11:01:11,255][20347] Updated weights for policy 0, policy_version 779225 (0.0008) [2023-12-27 11:01:11,537][20381] Updated weights for policy 1, policy_version 778722 (0.0007) [2023-12-27 11:01:11,601][20381] Updated weights for policy 1, policy_version 778732 (0.0007) [2023-12-27 11:01:11,664][20381] Updated weights for policy 1, policy_version 778742 (0.0010) [2023-12-27 11:01:11,967][20291] KL-divergence is very high: 127.9814 [2023-12-27 11:01:11,981][20347] Updated weights for policy 0, policy_version 779235 (0.0007) [2023-12-27 11:01:12,031][20347] Updated weights for policy 0, policy_version 779245 (0.0005) [2023-12-27 11:01:12,085][20347] Updated weights for policy 0, policy_version 779255 (0.0005) [2023-12-27 11:01:12,099][20291] KL-divergence is very high: 112.4550 [2023-12-27 11:01:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 398901248. Throughput: 0: 10180.1, 1: 10284.4. Samples: 398893144. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:12,363][19170] Avg episode reward: [(0, '765.600'), (1, '970.870')] [2023-12-27 11:01:12,435][20381] Updated weights for policy 1, policy_version 778752 (0.0008) [2023-12-27 11:01:12,497][20381] Updated weights for policy 1, policy_version 778762 (0.0007) [2023-12-27 11:01:12,554][20381] Updated weights for policy 1, policy_version 778772 (0.0007) [2023-12-27 11:01:12,711][20347] Updated weights for policy 0, policy_version 779265 (0.0006) [2023-12-27 11:01:12,772][20347] Updated weights for policy 0, policy_version 779275 (0.0006) [2023-12-27 11:01:12,837][20347] Updated weights for policy 0, policy_version 779285 (0.0008) [2023-12-27 11:01:12,889][20347] Updated weights for policy 0, policy_version 779295 (0.0007) [2023-12-27 11:01:13,201][20381] Updated weights for policy 1, policy_version 778782 (0.0008) [2023-12-27 11:01:13,257][20381] Updated weights for policy 1, policy_version 778792 (0.0008) [2023-12-27 11:01:13,319][20381] Updated weights for policy 1, policy_version 778802 (0.0008) [2023-12-27 11:01:13,562][20347] Updated weights for policy 0, policy_version 779305 (0.0006) [2023-12-27 11:01:13,616][20347] Updated weights for policy 0, policy_version 779315 (0.0007) [2023-12-27 11:01:13,664][20347] Updated weights for policy 0, policy_version 779325 (0.0007) [2023-12-27 11:01:13,998][20381] Updated weights for policy 1, policy_version 778812 (0.0008) [2023-12-27 11:01:14,053][20381] Updated weights for policy 1, policy_version 778822 (0.0007) [2023-12-27 11:01:14,108][20381] Updated weights for policy 1, policy_version 778832 (0.0008) [2023-12-27 11:01:14,373][20347] Updated weights for policy 0, policy_version 779335 (0.0006) [2023-12-27 11:01:14,424][20347] Updated weights for policy 0, policy_version 779345 (0.0005) [2023-12-27 11:01:14,481][20347] Updated weights for policy 0, policy_version 779355 (0.0005) [2023-12-27 11:01:14,749][20381] Updated weights for policy 1, policy_version 778842 (0.0007) [2023-12-27 11:01:14,814][20381] Updated weights for policy 1, policy_version 778852 (0.0008) [2023-12-27 11:01:14,878][20381] Updated weights for policy 1, policy_version 778862 (0.0007) [2023-12-27 11:01:14,942][20381] Updated weights for policy 1, policy_version 778872 (0.0009) [2023-12-27 11:01:15,095][20347] Updated weights for policy 0, policy_version 779365 (0.0006) [2023-12-27 11:01:15,161][20347] Updated weights for policy 0, policy_version 779375 (0.0007) [2023-12-27 11:01:15,225][20347] Updated weights for policy 0, policy_version 779385 (0.0008) [2023-12-27 11:01:15,649][20381] Updated weights for policy 1, policy_version 778882 (0.0008) [2023-12-27 11:01:15,698][20381] Updated weights for policy 1, policy_version 778892 (0.0008) [2023-12-27 11:01:15,751][20381] Updated weights for policy 1, policy_version 778902 (0.0007) [2023-12-27 11:01:15,853][20347] Updated weights for policy 0, policy_version 779395 (0.0007) [2023-12-27 11:01:15,911][20347] Updated weights for policy 0, policy_version 779405 (0.0008) [2023-12-27 11:01:15,960][20347] Updated weights for policy 0, policy_version 779415 (0.0008) [2023-12-27 11:01:16,437][20381] Updated weights for policy 1, policy_version 778912 (0.0007) [2023-12-27 11:01:16,500][20381] Updated weights for policy 1, policy_version 778922 (0.0008) [2023-12-27 11:01:16,559][20381] Updated weights for policy 1, policy_version 778932 (0.0007) [2023-12-27 11:01:16,683][20347] Updated weights for policy 0, policy_version 779425 (0.0008) [2023-12-27 11:01:16,745][20347] Updated weights for policy 0, policy_version 779435 (0.0007) [2023-12-27 11:01:16,805][20347] Updated weights for policy 0, policy_version 779445 (0.0007) [2023-12-27 11:01:16,854][20347] Updated weights for policy 0, policy_version 779455 (0.0007) [2023-12-27 11:01:17,263][20381] Updated weights for policy 1, policy_version 778942 (0.0008) [2023-12-27 11:01:17,309][20381] Updated weights for policy 1, policy_version 778952 (0.0008) [2023-12-27 11:01:17,355][20381] Updated weights for policy 1, policy_version 778962 (0.0008) [2023-12-27 11:01:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 20577.2). Total num frames: 398999552. Throughput: 0: 10195.7, 1: 10290.9. Samples: 399014000. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:17,363][19170] Avg episode reward: [(0, '804.470'), (1, '980.590')] [2023-12-27 11:01:17,538][20347] Updated weights for policy 0, policy_version 779465 (0.0008) [2023-12-27 11:01:17,594][20347] Updated weights for policy 0, policy_version 779475 (0.0008) [2023-12-27 11:01:17,655][20347] Updated weights for policy 0, policy_version 779485 (0.0008) [2023-12-27 11:01:18,065][20381] Updated weights for policy 1, policy_version 778972 (0.0007) [2023-12-27 11:01:18,114][20381] Updated weights for policy 1, policy_version 778982 (0.0008) [2023-12-27 11:01:18,186][20381] Updated weights for policy 1, policy_version 778992 (0.0008) [2023-12-27 11:01:18,335][20347] Updated weights for policy 0, policy_version 779495 (0.0008) [2023-12-27 11:01:18,392][20347] Updated weights for policy 0, policy_version 779505 (0.0008) [2023-12-27 11:01:18,445][20347] Updated weights for policy 0, policy_version 779515 (0.0008) [2023-12-27 11:01:18,913][20381] Updated weights for policy 1, policy_version 779002 (0.0007) [2023-12-27 11:01:18,972][20381] Updated weights for policy 1, policy_version 779012 (0.0007) [2023-12-27 11:01:19,025][20381] Updated weights for policy 1, policy_version 779022 (0.0009) [2023-12-27 11:01:19,090][20381] Updated weights for policy 1, policy_version 779032 (0.0008) [2023-12-27 11:01:19,169][20347] Updated weights for policy 0, policy_version 779525 (0.0008) [2023-12-27 11:01:19,227][20347] Updated weights for policy 0, policy_version 779535 (0.0007) [2023-12-27 11:01:19,288][20347] Updated weights for policy 0, policy_version 779545 (0.0008) [2023-12-27 11:01:19,836][20381] Updated weights for policy 1, policy_version 779042 (0.0007) [2023-12-27 11:01:19,899][20381] Updated weights for policy 1, policy_version 779052 (0.0006) [2023-12-27 11:01:19,962][20347] Updated weights for policy 0, policy_version 779555 (0.0007) [2023-12-27 11:01:19,964][20381] Updated weights for policy 1, policy_version 779062 (0.0006) [2023-12-27 11:01:20,021][20347] Updated weights for policy 0, policy_version 779565 (0.0008) [2023-12-27 11:01:20,086][20347] Updated weights for policy 0, policy_version 779575 (0.0008) [2023-12-27 11:01:20,622][20381] Updated weights for policy 1, policy_version 779072 (0.0007) [2023-12-27 11:01:20,674][20381] Updated weights for policy 1, policy_version 779082 (0.0007) [2023-12-27 11:01:20,736][20381] Updated weights for policy 1, policy_version 779092 (0.0006) [2023-12-27 11:01:20,797][20347] Updated weights for policy 0, policy_version 779585 (0.0008) [2023-12-27 11:01:20,860][20347] Updated weights for policy 0, policy_version 779595 (0.0006) [2023-12-27 11:01:20,928][20347] Updated weights for policy 0, policy_version 779605 (0.0006) [2023-12-27 11:01:20,994][20347] Updated weights for policy 0, policy_version 779615 (0.0006) [2023-12-27 11:01:21,456][20381] Updated weights for policy 1, policy_version 779102 (0.0008) [2023-12-27 11:01:21,518][20381] Updated weights for policy 1, policy_version 779112 (0.0008) [2023-12-27 11:01:21,576][20381] Updated weights for policy 1, policy_version 779122 (0.0008) [2023-12-27 11:01:21,672][20347] Updated weights for policy 0, policy_version 779625 (0.0007) [2023-12-27 11:01:21,736][20347] Updated weights for policy 0, policy_version 779635 (0.0008) [2023-12-27 11:01:21,797][20347] Updated weights for policy 0, policy_version 779645 (0.0006) [2023-12-27 11:01:22,233][20381] Updated weights for policy 1, policy_version 779132 (0.0007) [2023-12-27 11:01:22,301][20381] Updated weights for policy 1, policy_version 779142 (0.0006) [2023-12-27 11:01:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 399097856. Throughput: 0: 10203.1, 1: 10244.5. Samples: 399073496. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:22,363][19170] Avg episode reward: [(0, '795.070'), (1, '980.610')] [2023-12-27 11:01:22,364][20381] Updated weights for policy 1, policy_version 779152 (0.0007) [2023-12-27 11:01:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000779648_199614464.pth... [2023-12-27 11:01:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000778432_199303168.pth [2023-12-27 11:01:22,414][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000779160_199491584.pth... [2023-12-27 11:01:22,419][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000777976_199188480.pth [2023-12-27 11:01:22,543][20347] Updated weights for policy 0, policy_version 779655 (0.0007) [2023-12-27 11:01:22,602][20347] Updated weights for policy 0, policy_version 779665 (0.0008) [2023-12-27 11:01:22,663][20347] Updated weights for policy 0, policy_version 779675 (0.0007) [2023-12-27 11:01:23,045][20381] Updated weights for policy 1, policy_version 779162 (0.0007) [2023-12-27 11:01:23,098][20381] Updated weights for policy 1, policy_version 779172 (0.0008) [2023-12-27 11:01:23,157][20381] Updated weights for policy 1, policy_version 779182 (0.0008) [2023-12-27 11:01:23,211][20381] Updated weights for policy 1, policy_version 779192 (0.0008) [2023-12-27 11:01:23,379][20347] Updated weights for policy 0, policy_version 779685 (0.0008) [2023-12-27 11:01:23,431][20347] Updated weights for policy 0, policy_version 779695 (0.0005) [2023-12-27 11:01:23,482][20347] Updated weights for policy 0, policy_version 779705 (0.0008) [2023-12-27 11:01:23,950][20381] Updated weights for policy 1, policy_version 779202 (0.0010) [2023-12-27 11:01:23,998][20381] Updated weights for policy 1, policy_version 779212 (0.0010) [2023-12-27 11:01:24,043][20347] Updated weights for policy 0, policy_version 779715 (0.0009) [2023-12-27 11:01:24,043][20381] Updated weights for policy 1, policy_version 779222 (0.0010) [2023-12-27 11:01:24,093][20291] KL-divergence is very high: 284.0525 [2023-12-27 11:01:24,106][20347] Updated weights for policy 0, policy_version 779725 (0.0005) [2023-12-27 11:01:24,147][20291] KL-divergence is very high: 380.5521 [2023-12-27 11:01:24,169][20347] Updated weights for policy 0, policy_version 779735 (0.0006) [2023-12-27 11:01:24,196][20291] KL-divergence is very high: 106.7298 [2023-12-27 11:01:24,654][20381] Updated weights for policy 1, policy_version 779232 (0.0010) [2023-12-27 11:01:24,709][20381] Updated weights for policy 1, policy_version 779242 (0.0010) [2023-12-27 11:01:24,758][20381] Updated weights for policy 1, policy_version 779252 (0.0009) [2023-12-27 11:01:24,800][20347] Updated weights for policy 0, policy_version 779745 (0.0010) [2023-12-27 11:01:24,858][20347] Updated weights for policy 0, policy_version 779755 (0.0010) [2023-12-27 11:01:24,918][20347] Updated weights for policy 0, policy_version 779765 (0.0009) [2023-12-27 11:01:24,975][20347] Updated weights for policy 0, policy_version 779775 (0.0007) [2023-12-27 11:01:25,443][20381] Updated weights for policy 1, policy_version 779262 (0.0010) [2023-12-27 11:01:25,490][20381] Updated weights for policy 1, policy_version 779272 (0.0010) [2023-12-27 11:01:25,537][20381] Updated weights for policy 1, policy_version 779282 (0.0010) [2023-12-27 11:01:25,641][20347] Updated weights for policy 0, policy_version 779785 (0.0010) [2023-12-27 11:01:25,693][20347] Updated weights for policy 0, policy_version 779795 (0.0010) [2023-12-27 11:01:25,751][20347] Updated weights for policy 0, policy_version 779805 (0.0010) [2023-12-27 11:01:26,245][20381] Updated weights for policy 1, policy_version 779292 (0.0010) [2023-12-27 11:01:26,289][20381] Updated weights for policy 1, policy_version 779302 (0.0010) [2023-12-27 11:01:26,343][20381] Updated weights for policy 1, policy_version 779312 (0.0010) [2023-12-27 11:01:26,435][20347] Updated weights for policy 0, policy_version 779815 (0.0008) [2023-12-27 11:01:26,494][20347] Updated weights for policy 0, policy_version 779825 (0.0006) [2023-12-27 11:01:26,556][20347] Updated weights for policy 0, policy_version 779835 (0.0005) [2023-12-27 11:01:26,980][20381] Updated weights for policy 1, policy_version 779322 (0.0009) [2023-12-27 11:01:27,053][20381] Updated weights for policy 1, policy_version 779332 (0.0006) [2023-12-27 11:01:27,118][20381] Updated weights for policy 1, policy_version 779342 (0.0006) [2023-12-27 11:01:27,162][20347] Updated weights for policy 0, policy_version 779845 (0.0008) [2023-12-27 11:01:27,177][20381] Updated weights for policy 1, policy_version 779352 (0.0006) [2023-12-27 11:01:27,205][20347] Updated weights for policy 0, policy_version 779855 (0.0005) [2023-12-27 11:01:27,254][20347] Updated weights for policy 0, policy_version 779865 (0.0006) [2023-12-27 11:01:27,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 399212544. Throughput: 0: 10234.4, 1: 10218.6. Samples: 399195276. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:27,363][19170] Avg episode reward: [(0, '785.590'), (1, '989.030')] [2023-12-27 11:01:27,679][20381] Updated weights for policy 1, policy_version 779362 (0.0005) [2023-12-27 11:01:27,725][20381] Updated weights for policy 1, policy_version 779372 (0.0006) [2023-12-27 11:01:27,775][20381] Updated weights for policy 1, policy_version 779382 (0.0005) [2023-12-27 11:01:27,935][20347] Updated weights for policy 0, policy_version 779875 (0.0008) [2023-12-27 11:01:27,982][20347] Updated weights for policy 0, policy_version 779885 (0.0005) [2023-12-27 11:01:28,032][20347] Updated weights for policy 0, policy_version 779895 (0.0009) [2023-12-27 11:01:28,306][20381] Updated weights for policy 1, policy_version 779392 (0.0005) [2023-12-27 11:01:28,368][20381] Updated weights for policy 1, policy_version 779402 (0.0007) [2023-12-27 11:01:28,430][20381] Updated weights for policy 1, policy_version 779412 (0.0008) [2023-12-27 11:01:28,694][20347] Updated weights for policy 0, policy_version 779905 (0.0008) [2023-12-27 11:01:28,757][20347] Updated weights for policy 0, policy_version 779915 (0.0008) [2023-12-27 11:01:28,813][20347] Updated weights for policy 0, policy_version 779925 (0.0008) [2023-12-27 11:01:28,866][20347] Updated weights for policy 0, policy_version 779935 (0.0006) [2023-12-27 11:01:29,101][20381] Updated weights for policy 1, policy_version 779422 (0.0008) [2023-12-27 11:01:29,144][20381] Updated weights for policy 1, policy_version 779432 (0.0007) [2023-12-27 11:01:29,203][20381] Updated weights for policy 1, policy_version 779442 (0.0008) [2023-12-27 11:01:29,517][20347] Updated weights for policy 0, policy_version 779945 (0.0007) [2023-12-27 11:01:29,574][20347] Updated weights for policy 0, policy_version 779955 (0.0007) [2023-12-27 11:01:29,624][20347] Updated weights for policy 0, policy_version 779965 (0.0008) [2023-12-27 11:01:29,966][20381] Updated weights for policy 1, policy_version 779452 (0.0007) [2023-12-27 11:01:30,027][20381] Updated weights for policy 1, policy_version 779462 (0.0008) [2023-12-27 11:01:30,089][20381] Updated weights for policy 1, policy_version 779472 (0.0007) [2023-12-27 11:01:30,324][20347] Updated weights for policy 0, policy_version 779975 (0.0008) [2023-12-27 11:01:30,381][20347] Updated weights for policy 0, policy_version 779985 (0.0007) [2023-12-27 11:01:30,434][20347] Updated weights for policy 0, policy_version 779995 (0.0007) [2023-12-27 11:01:30,787][20381] Updated weights for policy 1, policy_version 779482 (0.0008) [2023-12-27 11:01:30,843][20381] Updated weights for policy 1, policy_version 779492 (0.0008) [2023-12-27 11:01:30,902][20381] Updated weights for policy 1, policy_version 779502 (0.0008) [2023-12-27 11:01:30,943][20381] Updated weights for policy 1, policy_version 779512 (0.0007) [2023-12-27 11:01:31,060][20347] Updated weights for policy 0, policy_version 780005 (0.0005) [2023-12-27 11:01:31,118][20347] Updated weights for policy 0, policy_version 780015 (0.0008) [2023-12-27 11:01:31,179][20347] Updated weights for policy 0, policy_version 780025 (0.0008) [2023-12-27 11:01:31,684][20381] Updated weights for policy 1, policy_version 779522 (0.0008) [2023-12-27 11:01:31,749][20381] Updated weights for policy 1, policy_version 779532 (0.0009) [2023-12-27 11:01:31,816][20381] Updated weights for policy 1, policy_version 779542 (0.0008) [2023-12-27 11:01:31,990][20347] Updated weights for policy 0, policy_version 780035 (0.0008) [2023-12-27 11:01:32,053][20347] Updated weights for policy 0, policy_version 780045 (0.0008) [2023-12-27 11:01:32,114][20347] Updated weights for policy 0, policy_version 780055 (0.0008) [2023-12-27 11:01:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 399310848. Throughput: 0: 10292.9, 1: 10238.3. Samples: 399320856. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:32,363][19170] Avg episode reward: [(0, '853.470'), (1, '988.920')] [2023-12-27 11:01:32,453][20381] Updated weights for policy 1, policy_version 779552 (0.0007) [2023-12-27 11:01:32,512][20381] Updated weights for policy 1, policy_version 779562 (0.0007) [2023-12-27 11:01:32,566][20381] Updated weights for policy 1, policy_version 779572 (0.0007) [2023-12-27 11:01:32,817][20347] Updated weights for policy 0, policy_version 780065 (0.0006) [2023-12-27 11:01:32,870][20347] Updated weights for policy 0, policy_version 780075 (0.0008) [2023-12-27 11:01:32,927][20347] Updated weights for policy 0, policy_version 780085 (0.0009) [2023-12-27 11:01:32,987][20347] Updated weights for policy 0, policy_version 780095 (0.0010) [2023-12-27 11:01:33,198][20381] Updated weights for policy 1, policy_version 779582 (0.0009) [2023-12-27 11:01:33,249][20381] Updated weights for policy 1, policy_version 779592 (0.0010) [2023-12-27 11:01:33,294][20381] Updated weights for policy 1, policy_version 779602 (0.0010) [2023-12-27 11:01:33,646][20347] Updated weights for policy 0, policy_version 780105 (0.0010) [2023-12-27 11:01:33,704][20347] Updated weights for policy 0, policy_version 780115 (0.0010) [2023-12-27 11:01:33,764][20347] Updated weights for policy 0, policy_version 780125 (0.0010) [2023-12-27 11:01:33,981][20381] Updated weights for policy 1, policy_version 779612 (0.0011) [2023-12-27 11:01:34,026][20381] Updated weights for policy 1, policy_version 779622 (0.0010) [2023-12-27 11:01:34,072][20381] Updated weights for policy 1, policy_version 779632 (0.0007) [2023-12-27 11:01:34,407][20347] Updated weights for policy 0, policy_version 780135 (0.0010) [2023-12-27 11:01:34,467][20347] Updated weights for policy 0, policy_version 780145 (0.0010) [2023-12-27 11:01:34,517][20347] Updated weights for policy 0, policy_version 780155 (0.0010) [2023-12-27 11:01:34,775][20381] Updated weights for policy 1, policy_version 779642 (0.0006) [2023-12-27 11:01:34,836][20381] Updated weights for policy 1, policy_version 779652 (0.0011) [2023-12-27 11:01:34,902][20381] Updated weights for policy 1, policy_version 779662 (0.0010) [2023-12-27 11:01:34,969][20381] Updated weights for policy 1, policy_version 779672 (0.0008) [2023-12-27 11:01:35,247][20347] Updated weights for policy 0, policy_version 780165 (0.0009) [2023-12-27 11:01:35,306][20347] Updated weights for policy 0, policy_version 780175 (0.0007) [2023-12-27 11:01:35,361][20347] Updated weights for policy 0, policy_version 780185 (0.0007) [2023-12-27 11:01:35,580][20381] Updated weights for policy 1, policy_version 779682 (0.0007) [2023-12-27 11:01:35,632][20381] Updated weights for policy 1, policy_version 779692 (0.0008) [2023-12-27 11:01:35,691][20381] Updated weights for policy 1, policy_version 779702 (0.0006) [2023-12-27 11:01:35,978][20347] Updated weights for policy 0, policy_version 780195 (0.0008) [2023-12-27 11:01:36,025][20347] Updated weights for policy 0, policy_version 780205 (0.0008) [2023-12-27 11:01:36,082][20347] Updated weights for policy 0, policy_version 780215 (0.0008) [2023-12-27 11:01:36,325][20381] Updated weights for policy 1, policy_version 779712 (0.0007) [2023-12-27 11:01:36,375][20381] Updated weights for policy 1, policy_version 779722 (0.0007) [2023-12-27 11:01:36,422][20381] Updated weights for policy 1, policy_version 779732 (0.0008) [2023-12-27 11:01:36,781][20347] Updated weights for policy 0, policy_version 780225 (0.0008) [2023-12-27 11:01:36,837][20347] Updated weights for policy 0, policy_version 780235 (0.0006) [2023-12-27 11:01:36,900][20347] Updated weights for policy 0, policy_version 780245 (0.0008) [2023-12-27 11:01:36,961][20347] Updated weights for policy 0, policy_version 780255 (0.0005) [2023-12-27 11:01:37,117][20381] Updated weights for policy 1, policy_version 779742 (0.0009) [2023-12-27 11:01:37,172][20381] Updated weights for policy 1, policy_version 779752 (0.0005) [2023-12-27 11:01:37,227][20381] Updated weights for policy 1, policy_version 779762 (0.0005) [2023-12-27 11:01:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 399417344. Throughput: 0: 10323.7, 1: 10223.1. Samples: 399382916. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:37,363][19170] Avg episode reward: [(0, '853.500'), (1, '980.530')] [2023-12-27 11:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000779768_199647232.pth... [2023-12-27 11:01:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000780256_199770112.pth... [2023-12-27 11:01:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000778584_199344128.pth [2023-12-27 11:01:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000779072_199467008.pth [2023-12-27 11:01:37,590][20347] Updated weights for policy 0, policy_version 780265 (0.0007) [2023-12-27 11:01:37,653][20347] Updated weights for policy 0, policy_version 780275 (0.0007) [2023-12-27 11:01:37,714][20347] Updated weights for policy 0, policy_version 780285 (0.0008) [2023-12-27 11:01:37,881][20381] Updated weights for policy 1, policy_version 779772 (0.0007) [2023-12-27 11:01:37,937][20381] Updated weights for policy 1, policy_version 779782 (0.0008) [2023-12-27 11:01:37,990][20381] Updated weights for policy 1, policy_version 779792 (0.0007) [2023-12-27 11:01:38,392][20347] Updated weights for policy 0, policy_version 780295 (0.0007) [2023-12-27 11:01:38,448][20347] Updated weights for policy 0, policy_version 780305 (0.0007) [2023-12-27 11:01:38,510][20347] Updated weights for policy 0, policy_version 780315 (0.0008) [2023-12-27 11:01:38,674][20381] Updated weights for policy 1, policy_version 779802 (0.0006) [2023-12-27 11:01:38,734][20381] Updated weights for policy 1, policy_version 779812 (0.0008) [2023-12-27 11:01:38,789][20381] Updated weights for policy 1, policy_version 779822 (0.0007) [2023-12-27 11:01:38,844][20381] Updated weights for policy 1, policy_version 779832 (0.0007) [2023-12-27 11:01:39,168][20347] Updated weights for policy 0, policy_version 780325 (0.0007) [2023-12-27 11:01:39,236][20347] Updated weights for policy 0, policy_version 780335 (0.0007) [2023-12-27 11:01:39,298][20347] Updated weights for policy 0, policy_version 780345 (0.0007) [2023-12-27 11:01:39,554][20381] Updated weights for policy 1, policy_version 779842 (0.0007) [2023-12-27 11:01:39,615][20381] Updated weights for policy 1, policy_version 779852 (0.0008) [2023-12-27 11:01:39,671][20381] Updated weights for policy 1, policy_version 779862 (0.0008) [2023-12-27 11:01:39,975][20347] Updated weights for policy 0, policy_version 780355 (0.0008) [2023-12-27 11:01:40,038][20347] Updated weights for policy 0, policy_version 780365 (0.0008) [2023-12-27 11:01:40,100][20347] Updated weights for policy 0, policy_version 780375 (0.0008) [2023-12-27 11:01:40,365][20381] Updated weights for policy 1, policy_version 779872 (0.0007) [2023-12-27 11:01:40,421][20381] Updated weights for policy 1, policy_version 779882 (0.0008) [2023-12-27 11:01:40,480][20381] Updated weights for policy 1, policy_version 779892 (0.0007) [2023-12-27 11:01:40,787][20347] Updated weights for policy 0, policy_version 780385 (0.0008) [2023-12-27 11:01:40,842][20347] Updated weights for policy 0, policy_version 780395 (0.0007) [2023-12-27 11:01:40,897][20347] Updated weights for policy 0, policy_version 780405 (0.0008) [2023-12-27 11:01:40,954][20347] Updated weights for policy 0, policy_version 780415 (0.0008) [2023-12-27 11:01:41,098][20381] Updated weights for policy 1, policy_version 779902 (0.0007) [2023-12-27 11:01:41,166][20381] Updated weights for policy 1, policy_version 779912 (0.0008) [2023-12-27 11:01:41,226][20381] Updated weights for policy 1, policy_version 779922 (0.0006) [2023-12-27 11:01:41,772][20347] Updated weights for policy 0, policy_version 780425 (0.0008) [2023-12-27 11:01:41,823][20347] Updated weights for policy 0, policy_version 780435 (0.0007) [2023-12-27 11:01:41,872][20347] Updated weights for policy 0, policy_version 780445 (0.0008) [2023-12-27 11:01:41,958][20381] Updated weights for policy 1, policy_version 779932 (0.0009) [2023-12-27 11:01:42,014][20381] Updated weights for policy 1, policy_version 779942 (0.0011) [2023-12-27 11:01:42,073][20381] Updated weights for policy 1, policy_version 779952 (0.0010) [2023-12-27 11:01:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 399515648. Throughput: 0: 10272.3, 1: 10194.2. Samples: 399506872. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:42,363][19170] Avg episode reward: [(0, '873.010'), (1, '990.300')] [2023-12-27 11:01:42,645][20347] Updated weights for policy 0, policy_version 780455 (0.0010) [2023-12-27 11:01:42,704][20347] Updated weights for policy 0, policy_version 780465 (0.0010) [2023-12-27 11:01:42,724][20381] Updated weights for policy 1, policy_version 779962 (0.0010) [2023-12-27 11:01:42,761][20347] Updated weights for policy 0, policy_version 780475 (0.0011) [2023-12-27 11:01:42,784][20381] Updated weights for policy 1, policy_version 779972 (0.0006) [2023-12-27 11:01:42,839][20381] Updated weights for policy 1, policy_version 779982 (0.0007) [2023-12-27 11:01:42,905][20381] Updated weights for policy 1, policy_version 779992 (0.0008) [2023-12-27 11:01:43,389][20347] Updated weights for policy 0, policy_version 780485 (0.0008) [2023-12-27 11:01:43,444][20347] Updated weights for policy 0, policy_version 780495 (0.0005) [2023-12-27 11:01:43,497][20347] Updated weights for policy 0, policy_version 780505 (0.0005) [2023-12-27 11:01:43,548][20381] Updated weights for policy 1, policy_version 780002 (0.0006) [2023-12-27 11:01:43,602][20381] Updated weights for policy 1, policy_version 780012 (0.0007) [2023-12-27 11:01:43,659][20381] Updated weights for policy 1, policy_version 780022 (0.0006) [2023-12-27 11:01:44,102][20347] Updated weights for policy 0, policy_version 780515 (0.0009) [2023-12-27 11:01:44,146][20347] Updated weights for policy 0, policy_version 780525 (0.0007) [2023-12-27 11:01:44,189][20347] Updated weights for policy 0, policy_version 780535 (0.0008) [2023-12-27 11:01:44,278][20381] Updated weights for policy 1, policy_version 780032 (0.0005) [2023-12-27 11:01:44,333][20381] Updated weights for policy 1, policy_version 780042 (0.0006) [2023-12-27 11:01:44,390][20381] Updated weights for policy 1, policy_version 780052 (0.0005) [2023-12-27 11:01:44,914][20347] Updated weights for policy 0, policy_version 780545 (0.0009) [2023-12-27 11:01:44,978][20347] Updated weights for policy 0, policy_version 780555 (0.0007) [2023-12-27 11:01:45,017][20381] Updated weights for policy 1, policy_version 780062 (0.0007) [2023-12-27 11:01:45,034][20347] Updated weights for policy 0, policy_version 780565 (0.0006) [2023-12-27 11:01:45,076][20381] Updated weights for policy 1, policy_version 780072 (0.0006) [2023-12-27 11:01:45,085][20347] Updated weights for policy 0, policy_version 780575 (0.0006) [2023-12-27 11:01:45,128][20381] Updated weights for policy 1, policy_version 780082 (0.0007) [2023-12-27 11:01:45,695][20347] Updated weights for policy 0, policy_version 780585 (0.0006) [2023-12-27 11:01:45,750][20347] Updated weights for policy 0, policy_version 780595 (0.0006) [2023-12-27 11:01:45,806][20347] Updated weights for policy 0, policy_version 780605 (0.0008) [2023-12-27 11:01:45,854][20381] Updated weights for policy 1, policy_version 780092 (0.0007) [2023-12-27 11:01:45,905][20381] Updated weights for policy 1, policy_version 780102 (0.0005) [2023-12-27 11:01:45,959][20381] Updated weights for policy 1, policy_version 780112 (0.0007) [2023-12-27 11:01:46,503][20347] Updated weights for policy 0, policy_version 780615 (0.0007) [2023-12-27 11:01:46,548][20381] Updated weights for policy 1, policy_version 780122 (0.0006) [2023-12-27 11:01:46,550][20347] Updated weights for policy 0, policy_version 780625 (0.0007) [2023-12-27 11:01:46,601][20347] Updated weights for policy 0, policy_version 780635 (0.0006) [2023-12-27 11:01:46,603][20381] Updated weights for policy 1, policy_version 780132 (0.0005) [2023-12-27 11:01:46,660][20381] Updated weights for policy 1, policy_version 780142 (0.0007) [2023-12-27 11:01:46,722][20381] Updated weights for policy 1, policy_version 780152 (0.0007) [2023-12-27 11:01:47,287][20347] Updated weights for policy 0, policy_version 780645 (0.0007) [2023-12-27 11:01:47,345][20347] Updated weights for policy 0, policy_version 780655 (0.0008) [2023-12-27 11:01:47,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 399613952. Throughput: 0: 10291.3, 1: 10258.1. Samples: 399630468. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:47,364][19170] Avg episode reward: [(0, '824.260'), (1, '990.300')] [2023-12-27 11:01:47,398][20347] Updated weights for policy 0, policy_version 780665 (0.0006) [2023-12-27 11:01:47,419][20381] Updated weights for policy 1, policy_version 780162 (0.0005) [2023-12-27 11:01:47,470][20381] Updated weights for policy 1, policy_version 780172 (0.0008) [2023-12-27 11:01:47,526][20381] Updated weights for policy 1, policy_version 780182 (0.0008) [2023-12-27 11:01:48,117][20347] Updated weights for policy 0, policy_version 780675 (0.0006) [2023-12-27 11:01:48,137][20381] Updated weights for policy 1, policy_version 780192 (0.0007) [2023-12-27 11:01:48,171][20347] Updated weights for policy 0, policy_version 780685 (0.0007) [2023-12-27 11:01:48,193][20381] Updated weights for policy 1, policy_version 780202 (0.0007) [2023-12-27 11:01:48,231][20347] Updated weights for policy 0, policy_version 780695 (0.0007) [2023-12-27 11:01:48,240][20381] Updated weights for policy 1, policy_version 780212 (0.0006) [2023-12-27 11:01:48,912][20381] Updated weights for policy 1, policy_version 780222 (0.0006) [2023-12-27 11:01:48,942][20347] Updated weights for policy 0, policy_version 780705 (0.0008) [2023-12-27 11:01:48,969][20381] Updated weights for policy 1, policy_version 780232 (0.0007) [2023-12-27 11:01:49,000][20347] Updated weights for policy 0, policy_version 780715 (0.0005) [2023-12-27 11:01:49,025][20381] Updated weights for policy 1, policy_version 780242 (0.0007) [2023-12-27 11:01:49,053][20347] Updated weights for policy 0, policy_version 780725 (0.0005) [2023-12-27 11:01:49,108][20347] Updated weights for policy 0, policy_version 780735 (0.0007) [2023-12-27 11:01:49,726][20381] Updated weights for policy 1, policy_version 780252 (0.0008) [2023-12-27 11:01:49,758][20347] Updated weights for policy 0, policy_version 780745 (0.0006) [2023-12-27 11:01:49,782][20381] Updated weights for policy 1, policy_version 780262 (0.0011) [2023-12-27 11:01:49,810][20347] Updated weights for policy 0, policy_version 780755 (0.0005) [2023-12-27 11:01:49,847][20381] Updated weights for policy 1, policy_version 780272 (0.0010) [2023-12-27 11:01:49,872][20347] Updated weights for policy 0, policy_version 780765 (0.0007) [2023-12-27 11:01:50,492][20347] Updated weights for policy 0, policy_version 780775 (0.0007) [2023-12-27 11:01:50,547][20347] Updated weights for policy 0, policy_version 780785 (0.0007) [2023-12-27 11:01:50,610][20347] Updated weights for policy 0, policy_version 780795 (0.0007) [2023-12-27 11:01:50,631][20381] Updated weights for policy 1, policy_version 780282 (0.0008) [2023-12-27 11:01:50,689][20381] Updated weights for policy 1, policy_version 780292 (0.0008) [2023-12-27 11:01:50,751][20381] Updated weights for policy 1, policy_version 780302 (0.0008) [2023-12-27 11:01:50,815][20381] Updated weights for policy 1, policy_version 780312 (0.0007) [2023-12-27 11:01:51,322][20347] Updated weights for policy 0, policy_version 780805 (0.0007) [2023-12-27 11:01:51,385][20347] Updated weights for policy 0, policy_version 780815 (0.0007) [2023-12-27 11:01:51,443][20347] Updated weights for policy 0, policy_version 780825 (0.0006) [2023-12-27 11:01:51,531][20381] Updated weights for policy 1, policy_version 780322 (0.0008) [2023-12-27 11:01:51,580][20381] Updated weights for policy 1, policy_version 780332 (0.0008) [2023-12-27 11:01:51,643][20381] Updated weights for policy 1, policy_version 780342 (0.0008) [2023-12-27 11:01:52,168][20347] Updated weights for policy 0, policy_version 780835 (0.0008) [2023-12-27 11:01:52,217][20347] Updated weights for policy 0, policy_version 780845 (0.0007) [2023-12-27 11:01:52,276][20347] Updated weights for policy 0, policy_version 780855 (0.0008) [2023-12-27 11:01:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 399720448. Throughput: 0: 10238.7, 1: 10265.1. Samples: 399691488. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:52,363][19170] Avg episode reward: [(0, '892.500'), (1, '990.250')] [2023-12-27 11:01:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000780864_199925760.pth... [2023-12-27 11:01:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000779648_199614464.pth [2023-12-27 11:01:52,385][20381] Updated weights for policy 1, policy_version 780352 (0.0008) [2023-12-27 11:01:52,441][20381] Updated weights for policy 1, policy_version 780362 (0.0008) [2023-12-27 11:01:52,497][20381] Updated weights for policy 1, policy_version 780372 (0.0008) [2023-12-27 11:01:52,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000780376_199802880.pth... [2023-12-27 11:01:52,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000779160_199491584.pth [2023-12-27 11:01:52,965][20347] Updated weights for policy 0, policy_version 780865 (0.0007) [2023-12-27 11:01:53,019][20347] Updated weights for policy 0, policy_version 780875 (0.0005) [2023-12-27 11:01:53,062][20347] Updated weights for policy 0, policy_version 780885 (0.0008) [2023-12-27 11:01:53,111][20347] Updated weights for policy 0, policy_version 780895 (0.0008) [2023-12-27 11:01:53,196][20381] Updated weights for policy 1, policy_version 780382 (0.0008) [2023-12-27 11:01:53,245][20381] Updated weights for policy 1, policy_version 780392 (0.0008) [2023-12-27 11:01:53,303][20381] Updated weights for policy 1, policy_version 780402 (0.0007) [2023-12-27 11:01:53,737][20347] Updated weights for policy 0, policy_version 780905 (0.0010) [2023-12-27 11:01:53,781][20347] Updated weights for policy 0, policy_version 780915 (0.0010) [2023-12-27 11:01:53,826][20347] Updated weights for policy 0, policy_version 780925 (0.0010) [2023-12-27 11:01:54,034][20381] Updated weights for policy 1, policy_version 780412 (0.0007) [2023-12-27 11:01:54,079][20381] Updated weights for policy 1, policy_version 780422 (0.0007) [2023-12-27 11:01:54,131][20381] Updated weights for policy 1, policy_version 780432 (0.0008) [2023-12-27 11:01:54,514][20347] Updated weights for policy 0, policy_version 780935 (0.0010) [2023-12-27 11:01:54,571][20347] Updated weights for policy 0, policy_version 780945 (0.0010) [2023-12-27 11:01:54,625][20347] Updated weights for policy 0, policy_version 780955 (0.0010) [2023-12-27 11:01:54,849][20381] Updated weights for policy 1, policy_version 780442 (0.0007) [2023-12-27 11:01:54,905][20381] Updated weights for policy 1, policy_version 780452 (0.0010) [2023-12-27 11:01:54,966][20381] Updated weights for policy 1, policy_version 780462 (0.0006) [2023-12-27 11:01:55,032][20381] Updated weights for policy 1, policy_version 780472 (0.0007) [2023-12-27 11:01:55,276][20347] Updated weights for policy 0, policy_version 780965 (0.0009) [2023-12-27 11:01:55,324][20347] Updated weights for policy 0, policy_version 780975 (0.0010) [2023-12-27 11:01:55,374][20347] Updated weights for policy 0, policy_version 780985 (0.0010) [2023-12-27 11:01:55,652][20381] Updated weights for policy 1, policy_version 780482 (0.0008) [2023-12-27 11:01:55,709][20381] Updated weights for policy 1, policy_version 780492 (0.0009) [2023-12-27 11:01:55,766][20381] Updated weights for policy 1, policy_version 780502 (0.0006) [2023-12-27 11:01:56,093][20347] Updated weights for policy 0, policy_version 780995 (0.0010) [2023-12-27 11:01:56,148][20347] Updated weights for policy 0, policy_version 781005 (0.0006) [2023-12-27 11:01:56,208][20347] Updated weights for policy 0, policy_version 781015 (0.0006) [2023-12-27 11:01:56,311][20381] Updated weights for policy 1, policy_version 780512 (0.0007) [2023-12-27 11:01:56,357][20381] Updated weights for policy 1, policy_version 780522 (0.0008) [2023-12-27 11:01:56,402][20381] Updated weights for policy 1, policy_version 780532 (0.0008) [2023-12-27 11:01:56,791][20347] Updated weights for policy 0, policy_version 781025 (0.0006) [2023-12-27 11:01:56,840][20347] Updated weights for policy 0, policy_version 781035 (0.0006) [2023-12-27 11:01:56,894][20347] Updated weights for policy 0, policy_version 781045 (0.0005) [2023-12-27 11:01:56,948][20347] Updated weights for policy 0, policy_version 781055 (0.0006) [2023-12-27 11:01:57,123][20381] Updated weights for policy 1, policy_version 780542 (0.0007) [2023-12-27 11:01:57,176][20381] Updated weights for policy 1, policy_version 780552 (0.0006) [2023-12-27 11:01:57,225][20381] Updated weights for policy 1, policy_version 780562 (0.0009) [2023-12-27 11:01:57,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 399826944. Throughput: 0: 10207.5, 1: 10262.3. Samples: 399814284. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:01:57,363][19170] Avg episode reward: [(0, '873.080'), (1, '980.550')] [2023-12-27 11:01:57,528][20347] Updated weights for policy 0, policy_version 781065 (0.0008) [2023-12-27 11:01:57,577][20347] Updated weights for policy 0, policy_version 781075 (0.0010) [2023-12-27 11:01:57,625][20347] Updated weights for policy 0, policy_version 781085 (0.0010) [2023-12-27 11:01:57,879][20381] Updated weights for policy 1, policy_version 780572 (0.0008) [2023-12-27 11:01:57,929][20381] Updated weights for policy 1, policy_version 780582 (0.0009) [2023-12-27 11:01:57,980][20381] Updated weights for policy 1, policy_version 780592 (0.0007) [2023-12-27 11:01:58,285][20347] Updated weights for policy 0, policy_version 781095 (0.0008) [2023-12-27 11:01:58,343][20347] Updated weights for policy 0, policy_version 781105 (0.0008) [2023-12-27 11:01:58,411][20347] Updated weights for policy 0, policy_version 781115 (0.0007) [2023-12-27 11:01:58,668][20381] Updated weights for policy 1, policy_version 780602 (0.0007) [2023-12-27 11:01:58,733][20381] Updated weights for policy 1, policy_version 780612 (0.0007) [2023-12-27 11:01:58,799][20381] Updated weights for policy 1, policy_version 780622 (0.0008) [2023-12-27 11:01:58,865][20381] Updated weights for policy 1, policy_version 780632 (0.0011) [2023-12-27 11:01:59,260][20347] Updated weights for policy 0, policy_version 781125 (0.0008) [2023-12-27 11:01:59,328][20347] Updated weights for policy 0, policy_version 781135 (0.0008) [2023-12-27 11:01:59,388][20347] Updated weights for policy 0, policy_version 781145 (0.0008) [2023-12-27 11:01:59,662][20381] Updated weights for policy 1, policy_version 780642 (0.0011) [2023-12-27 11:01:59,723][20381] Updated weights for policy 1, policy_version 780652 (0.0009) [2023-12-27 11:01:59,784][20381] Updated weights for policy 1, policy_version 780662 (0.0008) [2023-12-27 11:02:00,015][20347] Updated weights for policy 0, policy_version 781155 (0.0007) [2023-12-27 11:02:00,070][20347] Updated weights for policy 0, policy_version 781165 (0.0007) [2023-12-27 11:02:00,129][20347] Updated weights for policy 0, policy_version 781175 (0.0007) [2023-12-27 11:02:00,401][20381] Updated weights for policy 1, policy_version 780672 (0.0008) [2023-12-27 11:02:00,460][20381] Updated weights for policy 1, policy_version 780682 (0.0006) [2023-12-27 11:02:00,522][20381] Updated weights for policy 1, policy_version 780692 (0.0008) [2023-12-27 11:02:00,866][20347] Updated weights for policy 0, policy_version 781185 (0.0007) [2023-12-27 11:02:00,919][20347] Updated weights for policy 0, policy_version 781195 (0.0007) [2023-12-27 11:02:00,979][20347] Updated weights for policy 0, policy_version 781205 (0.0008) [2023-12-27 11:02:01,040][20347] Updated weights for policy 0, policy_version 781215 (0.0008) [2023-12-27 11:02:01,167][20381] Updated weights for policy 1, policy_version 780702 (0.0008) [2023-12-27 11:02:01,216][20381] Updated weights for policy 1, policy_version 780712 (0.0008) [2023-12-27 11:02:01,274][20381] Updated weights for policy 1, policy_version 780722 (0.0008) [2023-12-27 11:02:01,744][20347] Updated weights for policy 0, policy_version 781225 (0.0007) [2023-12-27 11:02:01,800][20347] Updated weights for policy 0, policy_version 781235 (0.0008) [2023-12-27 11:02:01,857][20347] Updated weights for policy 0, policy_version 781245 (0.0009) [2023-12-27 11:02:01,924][20381] Updated weights for policy 1, policy_version 780732 (0.0008) [2023-12-27 11:02:01,982][20381] Updated weights for policy 1, policy_version 780742 (0.0008) [2023-12-27 11:02:02,040][20381] Updated weights for policy 1, policy_version 780752 (0.0005) [2023-12-27 11:02:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 399925248. Throughput: 0: 10202.3, 1: 10292.2. Samples: 399936252. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:02,363][19170] Avg episode reward: [(0, '874.500'), (1, '980.570')] [2023-12-27 11:02:02,590][20347] Updated weights for policy 0, policy_version 781255 (0.0007) [2023-12-27 11:02:02,621][20381] Updated weights for policy 1, policy_version 780762 (0.0006) [2023-12-27 11:02:02,653][20347] Updated weights for policy 0, policy_version 781265 (0.0008) [2023-12-27 11:02:02,688][20381] Updated weights for policy 1, policy_version 780772 (0.0006) [2023-12-27 11:02:02,714][20347] Updated weights for policy 0, policy_version 781275 (0.0007) [2023-12-27 11:02:02,747][20381] Updated weights for policy 1, policy_version 780782 (0.0006) [2023-12-27 11:02:02,805][20381] Updated weights for policy 1, policy_version 780792 (0.0007) [2023-12-27 11:02:03,349][20347] Updated weights for policy 0, policy_version 781285 (0.0009) [2023-12-27 11:02:03,402][20347] Updated weights for policy 0, policy_version 781295 (0.0008) [2023-12-27 11:02:03,456][20347] Updated weights for policy 0, policy_version 781305 (0.0007) [2023-12-27 11:02:03,492][20381] Updated weights for policy 1, policy_version 780802 (0.0006) [2023-12-27 11:02:03,532][20381] Updated weights for policy 1, policy_version 780812 (0.0007) [2023-12-27 11:02:03,577][20381] Updated weights for policy 1, policy_version 780822 (0.0007) [2023-12-27 11:02:04,083][20347] Updated weights for policy 0, policy_version 781315 (0.0007) [2023-12-27 11:02:04,145][20347] Updated weights for policy 0, policy_version 781325 (0.0011) [2023-12-27 11:02:04,210][20347] Updated weights for policy 0, policy_version 781335 (0.0010) [2023-12-27 11:02:04,282][20381] Updated weights for policy 1, policy_version 780832 (0.0008) [2023-12-27 11:02:04,344][20381] Updated weights for policy 1, policy_version 780842 (0.0007) [2023-12-27 11:02:04,407][20381] Updated weights for policy 1, policy_version 780852 (0.0008) [2023-12-27 11:02:04,836][20347] Updated weights for policy 0, policy_version 781345 (0.0010) [2023-12-27 11:02:04,892][20347] Updated weights for policy 0, policy_version 781355 (0.0005) [2023-12-27 11:02:04,946][20347] Updated weights for policy 0, policy_version 781365 (0.0005) [2023-12-27 11:02:04,996][20347] Updated weights for policy 0, policy_version 781375 (0.0005) [2023-12-27 11:02:05,111][20381] Updated weights for policy 1, policy_version 780862 (0.0007) [2023-12-27 11:02:05,159][20381] Updated weights for policy 1, policy_version 780872 (0.0007) [2023-12-27 11:02:05,219][20381] Updated weights for policy 1, policy_version 780882 (0.0008) [2023-12-27 11:02:05,614][20347] Updated weights for policy 0, policy_version 781385 (0.0010) [2023-12-27 11:02:05,669][20347] Updated weights for policy 0, policy_version 781395 (0.0010) [2023-12-27 11:02:05,727][20347] Updated weights for policy 0, policy_version 781405 (0.0005) [2023-12-27 11:02:05,941][20381] Updated weights for policy 1, policy_version 780892 (0.0007) [2023-12-27 11:02:05,994][20381] Updated weights for policy 1, policy_version 780902 (0.0007) [2023-12-27 11:02:06,047][20381] Updated weights for policy 1, policy_version 780912 (0.0008) [2023-12-27 11:02:06,418][20347] Updated weights for policy 0, policy_version 781415 (0.0011) [2023-12-27 11:02:06,477][20347] Updated weights for policy 0, policy_version 781425 (0.0011) [2023-12-27 11:02:06,535][20347] Updated weights for policy 0, policy_version 781435 (0.0010) [2023-12-27 11:02:06,775][20381] Updated weights for policy 1, policy_version 780922 (0.0008) [2023-12-27 11:02:06,829][20381] Updated weights for policy 1, policy_version 780932 (0.0007) [2023-12-27 11:02:06,885][20381] Updated weights for policy 1, policy_version 780942 (0.0008) [2023-12-27 11:02:06,943][20381] Updated weights for policy 1, policy_version 780952 (0.0008) [2023-12-27 11:02:07,170][20347] Updated weights for policy 0, policy_version 781445 (0.0011) [2023-12-27 11:02:07,226][20347] Updated weights for policy 0, policy_version 781455 (0.0010) [2023-12-27 11:02:07,282][20347] Updated weights for policy 0, policy_version 781465 (0.0010) [2023-12-27 11:02:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 400031744. Throughput: 0: 10247.4, 1: 10308.7. Samples: 399998520. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:07,363][19170] Avg episode reward: [(0, '884.180'), (1, '990.270')] [2023-12-27 11:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000780952_199950336.pth... [2023-12-27 11:02:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000781472_200081408.pth... [2023-12-27 11:02:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000779768_199647232.pth [2023-12-27 11:02:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000780256_199770112.pth [2023-12-27 11:02:07,671][20381] Updated weights for policy 1, policy_version 780962 (0.0010) [2023-12-27 11:02:07,723][20381] Updated weights for policy 1, policy_version 780972 (0.0010) [2023-12-27 11:02:07,781][20381] Updated weights for policy 1, policy_version 780982 (0.0006) [2023-12-27 11:02:07,898][20347] Updated weights for policy 0, policy_version 781475 (0.0009) [2023-12-27 11:02:07,949][20347] Updated weights for policy 0, policy_version 781485 (0.0007) [2023-12-27 11:02:08,007][20347] Updated weights for policy 0, policy_version 781495 (0.0006) [2023-12-27 11:02:08,438][20381] Updated weights for policy 1, policy_version 780992 (0.0010) [2023-12-27 11:02:08,490][20381] Updated weights for policy 1, policy_version 781002 (0.0010) [2023-12-27 11:02:08,548][20381] Updated weights for policy 1, policy_version 781012 (0.0010) [2023-12-27 11:02:08,663][20347] Updated weights for policy 0, policy_version 781505 (0.0009) [2023-12-27 11:02:08,727][20347] Updated weights for policy 0, policy_version 781515 (0.0010) [2023-12-27 11:02:08,786][20347] Updated weights for policy 0, policy_version 781525 (0.0010) [2023-12-27 11:02:08,835][20347] Updated weights for policy 0, policy_version 781535 (0.0010) [2023-12-27 11:02:09,175][20381] Updated weights for policy 1, policy_version 781022 (0.0007) [2023-12-27 11:02:09,237][20381] Updated weights for policy 1, policy_version 781032 (0.0006) [2023-12-27 11:02:09,297][20381] Updated weights for policy 1, policy_version 781042 (0.0008) [2023-12-27 11:02:09,587][20347] Updated weights for policy 0, policy_version 781545 (0.0010) [2023-12-27 11:02:09,649][20347] Updated weights for policy 0, policy_version 781555 (0.0010) [2023-12-27 11:02:09,716][20347] Updated weights for policy 0, policy_version 781565 (0.0009) [2023-12-27 11:02:09,997][20381] Updated weights for policy 1, policy_version 781052 (0.0008) [2023-12-27 11:02:10,058][20381] Updated weights for policy 1, policy_version 781062 (0.0007) [2023-12-27 11:02:10,121][20381] Updated weights for policy 1, policy_version 781072 (0.0008) [2023-12-27 11:02:10,436][20347] Updated weights for policy 0, policy_version 781575 (0.0007) [2023-12-27 11:02:10,494][20347] Updated weights for policy 0, policy_version 781585 (0.0007) [2023-12-27 11:02:10,545][20347] Updated weights for policy 0, policy_version 781595 (0.0006) [2023-12-27 11:02:10,771][20381] Updated weights for policy 1, policy_version 781082 (0.0007) [2023-12-27 11:02:10,825][20381] Updated weights for policy 1, policy_version 781092 (0.0005) [2023-12-27 11:02:10,890][20381] Updated weights for policy 1, policy_version 781102 (0.0008) [2023-12-27 11:02:10,946][20381] Updated weights for policy 1, policy_version 781112 (0.0011) [2023-12-27 11:02:11,191][20347] Updated weights for policy 0, policy_version 781605 (0.0008) [2023-12-27 11:02:11,257][20347] Updated weights for policy 0, policy_version 781615 (0.0010) [2023-12-27 11:02:11,320][20347] Updated weights for policy 0, policy_version 781625 (0.0011) [2023-12-27 11:02:11,686][20381] Updated weights for policy 1, policy_version 781122 (0.0008) [2023-12-27 11:02:11,754][20381] Updated weights for policy 1, policy_version 781132 (0.0009) [2023-12-27 11:02:11,816][20381] Updated weights for policy 1, policy_version 781142 (0.0007) [2023-12-27 11:02:12,013][20347] Updated weights for policy 0, policy_version 781635 (0.0011) [2023-12-27 11:02:12,077][20347] Updated weights for policy 0, policy_version 781645 (0.0006) [2023-12-27 11:02:12,138][20347] Updated weights for policy 0, policy_version 781655 (0.0008) [2023-12-27 11:02:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 400130048. Throughput: 0: 10282.7, 1: 10298.0. Samples: 400121404. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:12,363][19170] Avg episode reward: [(0, '893.400'), (1, '990.250')] [2023-12-27 11:02:12,477][20381] Updated weights for policy 1, policy_version 781152 (0.0008) [2023-12-27 11:02:12,532][20381] Updated weights for policy 1, policy_version 781162 (0.0008) [2023-12-27 11:02:12,593][20381] Updated weights for policy 1, policy_version 781172 (0.0007) [2023-12-27 11:02:12,735][20347] Updated weights for policy 0, policy_version 781665 (0.0006) [2023-12-27 11:02:12,799][20347] Updated weights for policy 0, policy_version 781675 (0.0006) [2023-12-27 11:02:12,864][20347] Updated weights for policy 0, policy_version 781685 (0.0005) [2023-12-27 11:02:12,926][20347] Updated weights for policy 0, policy_version 781695 (0.0005) [2023-12-27 11:02:13,252][20381] Updated weights for policy 1, policy_version 781182 (0.0007) [2023-12-27 11:02:13,299][20381] Updated weights for policy 1, policy_version 781192 (0.0007) [2023-12-27 11:02:13,346][20381] Updated weights for policy 1, policy_version 781202 (0.0007) [2023-12-27 11:02:13,512][20347] Updated weights for policy 0, policy_version 781705 (0.0007) [2023-12-27 11:02:13,557][20347] Updated weights for policy 0, policy_version 781715 (0.0008) [2023-12-27 11:02:13,605][20347] Updated weights for policy 0, policy_version 781725 (0.0008) [2023-12-27 11:02:13,953][20381] Updated weights for policy 1, policy_version 781212 (0.0007) [2023-12-27 11:02:14,004][20381] Updated weights for policy 1, policy_version 781222 (0.0007) [2023-12-27 11:02:14,060][20381] Updated weights for policy 1, policy_version 781232 (0.0007) [2023-12-27 11:02:14,333][20347] Updated weights for policy 0, policy_version 781735 (0.0008) [2023-12-27 11:02:14,381][20347] Updated weights for policy 0, policy_version 781745 (0.0007) [2023-12-27 11:02:14,435][20347] Updated weights for policy 0, policy_version 781755 (0.0007) [2023-12-27 11:02:14,709][20381] Updated weights for policy 1, policy_version 781242 (0.0008) [2023-12-27 11:02:14,766][20381] Updated weights for policy 1, policy_version 781252 (0.0008) [2023-12-27 11:02:14,826][20381] Updated weights for policy 1, policy_version 781262 (0.0007) [2023-12-27 11:02:14,886][20381] Updated weights for policy 1, policy_version 781272 (0.0008) [2023-12-27 11:02:15,192][20347] Updated weights for policy 0, policy_version 781765 (0.0007) [2023-12-27 11:02:15,253][20347] Updated weights for policy 0, policy_version 781775 (0.0007) [2023-12-27 11:02:15,311][20347] Updated weights for policy 0, policy_version 781785 (0.0008) [2023-12-27 11:02:15,512][20381] Updated weights for policy 1, policy_version 781282 (0.0006) [2023-12-27 11:02:15,564][20381] Updated weights for policy 1, policy_version 781292 (0.0008) [2023-12-27 11:02:15,616][20381] Updated weights for policy 1, policy_version 781302 (0.0008) [2023-12-27 11:02:15,983][20347] Updated weights for policy 0, policy_version 781795 (0.0008) [2023-12-27 11:02:16,032][20347] Updated weights for policy 0, policy_version 781805 (0.0007) [2023-12-27 11:02:16,085][20347] Updated weights for policy 0, policy_version 781815 (0.0007) [2023-12-27 11:02:16,336][20381] Updated weights for policy 1, policy_version 781312 (0.0007) [2023-12-27 11:02:16,392][20381] Updated weights for policy 1, policy_version 781322 (0.0006) [2023-12-27 11:02:16,441][20381] Updated weights for policy 1, policy_version 781332 (0.0005) [2023-12-27 11:02:16,785][20347] Updated weights for policy 0, policy_version 781825 (0.0007) [2023-12-27 11:02:16,832][20347] Updated weights for policy 0, policy_version 781835 (0.0010) [2023-12-27 11:02:16,879][20347] Updated weights for policy 0, policy_version 781845 (0.0008) [2023-12-27 11:02:16,930][20347] Updated weights for policy 0, policy_version 781855 (0.0008) [2023-12-27 11:02:17,050][20381] Updated weights for policy 1, policy_version 781342 (0.0007) [2023-12-27 11:02:17,105][20381] Updated weights for policy 1, policy_version 781352 (0.0009) [2023-12-27 11:02:17,156][20381] Updated weights for policy 1, policy_version 781362 (0.0008) [2023-12-27 11:02:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 400236544. Throughput: 0: 10256.6, 1: 10273.4. Samples: 400244704. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:17,363][19170] Avg episode reward: [(0, '873.200'), (1, '961.050')] [2023-12-27 11:02:17,653][20347] Updated weights for policy 0, policy_version 781865 (0.0009) [2023-12-27 11:02:17,703][20347] Updated weights for policy 0, policy_version 781875 (0.0010) [2023-12-27 11:02:17,759][20347] Updated weights for policy 0, policy_version 781885 (0.0009) [2023-12-27 11:02:17,767][20381] Updated weights for policy 1, policy_version 781372 (0.0007) [2023-12-27 11:02:17,818][20381] Updated weights for policy 1, policy_version 781382 (0.0005) [2023-12-27 11:02:17,867][20381] Updated weights for policy 1, policy_version 781392 (0.0005) [2023-12-27 11:02:18,413][20347] Updated weights for policy 0, policy_version 781895 (0.0007) [2023-12-27 11:02:18,460][20381] Updated weights for policy 1, policy_version 781402 (0.0005) [2023-12-27 11:02:18,474][20347] Updated weights for policy 0, policy_version 781905 (0.0006) [2023-12-27 11:02:18,516][20381] Updated weights for policy 1, policy_version 781412 (0.0007) [2023-12-27 11:02:18,537][20347] Updated weights for policy 0, policy_version 781915 (0.0006) [2023-12-27 11:02:18,578][20381] Updated weights for policy 1, policy_version 781422 (0.0007) [2023-12-27 11:02:18,627][20381] Updated weights for policy 1, policy_version 781432 (0.0007) [2023-12-27 11:02:19,174][20347] Updated weights for policy 0, policy_version 781925 (0.0008) [2023-12-27 11:02:19,227][20347] Updated weights for policy 0, policy_version 781935 (0.0009) [2023-12-27 11:02:19,287][20347] Updated weights for policy 0, policy_version 781945 (0.0008) [2023-12-27 11:02:19,299][20381] Updated weights for policy 1, policy_version 781442 (0.0007) [2023-12-27 11:02:19,366][20381] Updated weights for policy 1, policy_version 781452 (0.0007) [2023-12-27 11:02:19,427][20381] Updated weights for policy 1, policy_version 781462 (0.0006) [2023-12-27 11:02:20,032][20347] Updated weights for policy 0, policy_version 781955 (0.0007) [2023-12-27 11:02:20,054][20381] Updated weights for policy 1, policy_version 781472 (0.0006) [2023-12-27 11:02:20,092][20347] Updated weights for policy 0, policy_version 781965 (0.0008) [2023-12-27 11:02:20,113][20381] Updated weights for policy 1, policy_version 781482 (0.0008) [2023-12-27 11:02:20,153][20347] Updated weights for policy 0, policy_version 781975 (0.0009) [2023-12-27 11:02:20,178][20381] Updated weights for policy 1, policy_version 781492 (0.0006) [2023-12-27 11:02:20,775][20381] Updated weights for policy 1, policy_version 781502 (0.0007) [2023-12-27 11:02:20,839][20381] Updated weights for policy 1, policy_version 781512 (0.0008) [2023-12-27 11:02:20,885][20347] Updated weights for policy 0, policy_version 781985 (0.0006) [2023-12-27 11:02:20,900][20381] Updated weights for policy 1, policy_version 781522 (0.0008) [2023-12-27 11:02:20,951][20347] Updated weights for policy 0, policy_version 781995 (0.0007) [2023-12-27 11:02:21,018][20347] Updated weights for policy 0, policy_version 782005 (0.0007) [2023-12-27 11:02:21,081][20347] Updated weights for policy 0, policy_version 782015 (0.0008) [2023-12-27 11:02:21,588][20381] Updated weights for policy 1, policy_version 781532 (0.0006) [2023-12-27 11:02:21,658][20381] Updated weights for policy 1, policy_version 781542 (0.0008) [2023-12-27 11:02:21,714][20381] Updated weights for policy 1, policy_version 781552 (0.0008) [2023-12-27 11:02:21,834][20347] Updated weights for policy 0, policy_version 782025 (0.0011) [2023-12-27 11:02:21,883][20347] Updated weights for policy 0, policy_version 782035 (0.0010) [2023-12-27 11:02:21,935][20347] Updated weights for policy 0, policy_version 782045 (0.0010) [2023-12-27 11:02:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 400334848. Throughput: 0: 10256.9, 1: 10307.3. Samples: 400308308. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:22,363][19170] Avg episode reward: [(0, '902.830'), (1, '951.300')] [2023-12-27 11:02:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000781560_200105984.pth... [2023-12-27 11:02:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000782048_200228864.pth... [2023-12-27 11:02:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000780376_199802880.pth [2023-12-27 11:02:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000780864_199925760.pth [2023-12-27 11:02:22,473][20381] Updated weights for policy 1, policy_version 781562 (0.0008) [2023-12-27 11:02:22,529][20381] Updated weights for policy 1, policy_version 781572 (0.0008) [2023-12-27 11:02:22,576][20381] Updated weights for policy 1, policy_version 781582 (0.0007) [2023-12-27 11:02:22,632][20381] Updated weights for policy 1, policy_version 781592 (0.0008) [2023-12-27 11:02:22,650][20347] Updated weights for policy 0, policy_version 782055 (0.0008) [2023-12-27 11:02:22,709][20347] Updated weights for policy 0, policy_version 782065 (0.0008) [2023-12-27 11:02:22,768][20347] Updated weights for policy 0, policy_version 782075 (0.0007) [2023-12-27 11:02:23,380][20347] Updated weights for policy 0, policy_version 782085 (0.0009) [2023-12-27 11:02:23,382][20381] Updated weights for policy 1, policy_version 781602 (0.0006) [2023-12-27 11:02:23,433][20381] Updated weights for policy 1, policy_version 781612 (0.0005) [2023-12-27 11:02:23,435][20347] Updated weights for policy 0, policy_version 782095 (0.0010) [2023-12-27 11:02:23,489][20381] Updated weights for policy 1, policy_version 781622 (0.0005) [2023-12-27 11:02:23,491][20347] Updated weights for policy 0, policy_version 782105 (0.0010) [2023-12-27 11:02:24,071][20347] Updated weights for policy 0, policy_version 782115 (0.0009) [2023-12-27 11:02:24,077][20381] Updated weights for policy 1, policy_version 781632 (0.0006) [2023-12-27 11:02:24,134][20347] Updated weights for policy 0, policy_version 782125 (0.0009) [2023-12-27 11:02:24,134][20381] Updated weights for policy 1, policy_version 781642 (0.0008) [2023-12-27 11:02:24,190][20381] Updated weights for policy 1, policy_version 781652 (0.0008) [2023-12-27 11:02:24,191][20347] Updated weights for policy 0, policy_version 782135 (0.0006) [2023-12-27 11:02:24,771][20347] Updated weights for policy 0, policy_version 782145 (0.0008) [2023-12-27 11:02:24,822][20347] Updated weights for policy 0, policy_version 782155 (0.0010) [2023-12-27 11:02:24,824][20381] Updated weights for policy 1, policy_version 781662 (0.0006) [2023-12-27 11:02:24,875][20381] Updated weights for policy 1, policy_version 781672 (0.0005) [2023-12-27 11:02:24,877][20347] Updated weights for policy 0, policy_version 782165 (0.0010) [2023-12-27 11:02:24,923][20381] Updated weights for policy 1, policy_version 781682 (0.0005) [2023-12-27 11:02:24,932][20347] Updated weights for policy 0, policy_version 782175 (0.0008) [2023-12-27 11:02:25,591][20381] Updated weights for policy 1, policy_version 781692 (0.0006) [2023-12-27 11:02:25,615][20347] Updated weights for policy 0, policy_version 782185 (0.0007) [2023-12-27 11:02:25,655][20381] Updated weights for policy 1, policy_version 781702 (0.0006) [2023-12-27 11:02:25,671][20347] Updated weights for policy 0, policy_version 782195 (0.0008) [2023-12-27 11:02:25,722][20381] Updated weights for policy 1, policy_version 781712 (0.0008) [2023-12-27 11:02:25,725][20347] Updated weights for policy 0, policy_version 782205 (0.0007) [2023-12-27 11:02:26,330][20381] Updated weights for policy 1, policy_version 781722 (0.0008) [2023-12-27 11:02:26,341][20347] Updated weights for policy 0, policy_version 782215 (0.0007) [2023-12-27 11:02:26,395][20381] Updated weights for policy 1, policy_version 781732 (0.0005) [2023-12-27 11:02:26,403][20347] Updated weights for policy 0, policy_version 782225 (0.0005) [2023-12-27 11:02:26,458][20381] Updated weights for policy 1, policy_version 781742 (0.0005) [2023-12-27 11:02:26,458][20347] Updated weights for policy 0, policy_version 782235 (0.0005) [2023-12-27 11:02:26,513][20381] Updated weights for policy 1, policy_version 781752 (0.0005) [2023-12-27 11:02:27,098][20347] Updated weights for policy 0, policy_version 782245 (0.0008) [2023-12-27 11:02:27,099][20381] Updated weights for policy 1, policy_version 781762 (0.0011) [2023-12-27 11:02:27,148][20381] Updated weights for policy 1, policy_version 781772 (0.0011) [2023-12-27 11:02:27,149][20347] Updated weights for policy 0, policy_version 782255 (0.0010) [2023-12-27 11:02:27,197][20381] Updated weights for policy 1, policy_version 781782 (0.0011) [2023-12-27 11:02:27,201][20347] Updated weights for policy 0, policy_version 782265 (0.0010) [2023-12-27 11:02:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 400449536. Throughput: 0: 10268.3, 1: 10302.6. Samples: 400432560. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:27,363][19170] Avg episode reward: [(0, '912.750'), (1, '951.280')] [2023-12-27 11:02:27,870][20381] Updated weights for policy 1, policy_version 781792 (0.0009) [2023-12-27 11:02:27,887][20347] Updated weights for policy 0, policy_version 782275 (0.0010) [2023-12-27 11:02:27,915][20381] Updated weights for policy 1, policy_version 781802 (0.0010) [2023-12-27 11:02:27,944][20347] Updated weights for policy 0, policy_version 782285 (0.0010) [2023-12-27 11:02:27,965][20381] Updated weights for policy 1, policy_version 781812 (0.0011) [2023-12-27 11:02:28,003][20347] Updated weights for policy 0, policy_version 782295 (0.0010) [2023-12-27 11:02:28,613][20347] Updated weights for policy 0, policy_version 782305 (0.0010) [2023-12-27 11:02:28,667][20347] Updated weights for policy 0, policy_version 782315 (0.0008) [2023-12-27 11:02:28,690][20381] Updated weights for policy 1, policy_version 781822 (0.0010) [2023-12-27 11:02:28,718][20347] Updated weights for policy 0, policy_version 782325 (0.0005) [2023-12-27 11:02:28,749][20381] Updated weights for policy 1, policy_version 781832 (0.0011) [2023-12-27 11:02:28,772][20347] Updated weights for policy 0, policy_version 782335 (0.0006) [2023-12-27 11:02:28,802][20381] Updated weights for policy 1, policy_version 781842 (0.0010) [2023-12-27 11:02:29,467][20381] Updated weights for policy 1, policy_version 781852 (0.0011) [2023-12-27 11:02:29,504][20347] Updated weights for policy 0, policy_version 782345 (0.0007) [2023-12-27 11:02:29,513][20381] Updated weights for policy 1, policy_version 781862 (0.0011) [2023-12-27 11:02:29,559][20381] Updated weights for policy 1, policy_version 781872 (0.0010) [2023-12-27 11:02:29,567][20347] Updated weights for policy 0, policy_version 782355 (0.0007) [2023-12-27 11:02:29,626][20347] Updated weights for policy 0, policy_version 782365 (0.0008) [2023-12-27 11:02:30,281][20347] Updated weights for policy 0, policy_version 782375 (0.0007) [2023-12-27 11:02:30,285][20381] Updated weights for policy 1, policy_version 781882 (0.0011) [2023-12-27 11:02:30,341][20381] Updated weights for policy 1, policy_version 781892 (0.0010) [2023-12-27 11:02:30,343][20347] Updated weights for policy 0, policy_version 782385 (0.0007) [2023-12-27 11:02:30,392][20347] Updated weights for policy 0, policy_version 782395 (0.0008) [2023-12-27 11:02:30,398][20381] Updated weights for policy 1, policy_version 781902 (0.0010) [2023-12-27 11:02:30,452][20381] Updated weights for policy 1, policy_version 781912 (0.0010) [2023-12-27 11:02:30,981][20347] Updated weights for policy 0, policy_version 782405 (0.0008) [2023-12-27 11:02:31,034][20347] Updated weights for policy 0, policy_version 782415 (0.0010) [2023-12-27 11:02:31,095][20347] Updated weights for policy 0, policy_version 782425 (0.0007) [2023-12-27 11:02:31,096][20381] Updated weights for policy 1, policy_version 781922 (0.0011) [2023-12-27 11:02:31,161][20381] Updated weights for policy 1, policy_version 781932 (0.0011) [2023-12-27 11:02:31,214][20381] Updated weights for policy 1, policy_version 781942 (0.0011) [2023-12-27 11:02:31,845][20347] Updated weights for policy 0, policy_version 782435 (0.0008) [2023-12-27 11:02:31,902][20347] Updated weights for policy 0, policy_version 782445 (0.0008) [2023-12-27 11:02:31,946][20381] Updated weights for policy 1, policy_version 781952 (0.0006) [2023-12-27 11:02:31,960][20347] Updated weights for policy 0, policy_version 782455 (0.0006) [2023-12-27 11:02:31,996][20381] Updated weights for policy 1, policy_version 781962 (0.0006) [2023-12-27 11:02:32,050][20381] Updated weights for policy 1, policy_version 781972 (0.0007) [2023-12-27 11:02:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 400547840. Throughput: 0: 10298.4, 1: 10286.5. Samples: 400556780. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:32,363][19170] Avg episode reward: [(0, '902.610'), (1, '912.700')] [2023-12-27 11:02:32,635][20347] Updated weights for policy 0, policy_version 782465 (0.0007) [2023-12-27 11:02:32,692][20347] Updated weights for policy 0, policy_version 782475 (0.0009) [2023-12-27 11:02:32,750][20381] Updated weights for policy 1, policy_version 781982 (0.0007) [2023-12-27 11:02:32,751][20347] Updated weights for policy 0, policy_version 782485 (0.0009) [2023-12-27 11:02:32,811][20381] Updated weights for policy 1, policy_version 781992 (0.0006) [2023-12-27 11:02:32,812][20347] Updated weights for policy 0, policy_version 782495 (0.0007) [2023-12-27 11:02:32,871][20381] Updated weights for policy 1, policy_version 782002 (0.0007) [2023-12-27 11:02:33,487][20381] Updated weights for policy 1, policy_version 782012 (0.0007) [2023-12-27 11:02:33,496][20347] Updated weights for policy 0, policy_version 782505 (0.0008) [2023-12-27 11:02:33,537][20381] Updated weights for policy 1, policy_version 782022 (0.0006) [2023-12-27 11:02:33,539][20347] Updated weights for policy 0, policy_version 782515 (0.0006) [2023-12-27 11:02:33,591][20381] Updated weights for policy 1, policy_version 782032 (0.0005) [2023-12-27 11:02:33,598][20347] Updated weights for policy 0, policy_version 782525 (0.0007) [2023-12-27 11:02:34,175][20381] Updated weights for policy 1, policy_version 782042 (0.0006) [2023-12-27 11:02:34,234][20381] Updated weights for policy 1, policy_version 782052 (0.0007) [2023-12-27 11:02:34,286][20381] Updated weights for policy 1, policy_version 782062 (0.0007) [2023-12-27 11:02:34,338][20381] Updated weights for policy 1, policy_version 782072 (0.0007) [2023-12-27 11:02:34,362][20347] Updated weights for policy 0, policy_version 782535 (0.0008) [2023-12-27 11:02:34,422][20347] Updated weights for policy 0, policy_version 782545 (0.0008) [2023-12-27 11:02:34,481][20347] Updated weights for policy 0, policy_version 782555 (0.0008) [2023-12-27 11:02:35,015][20381] Updated weights for policy 1, policy_version 782082 (0.0008) [2023-12-27 11:02:35,069][20381] Updated weights for policy 1, policy_version 782092 (0.0009) [2023-12-27 11:02:35,122][20381] Updated weights for policy 1, policy_version 782102 (0.0009) [2023-12-27 11:02:35,213][20347] Updated weights for policy 0, policy_version 782565 (0.0007) [2023-12-27 11:02:35,262][20347] Updated weights for policy 0, policy_version 782575 (0.0006) [2023-12-27 11:02:35,310][20347] Updated weights for policy 0, policy_version 782585 (0.0008) [2023-12-27 11:02:35,819][20381] Updated weights for policy 1, policy_version 782112 (0.0006) [2023-12-27 11:02:35,867][20381] Updated weights for policy 1, policy_version 782122 (0.0005) [2023-12-27 11:02:35,922][20381] Updated weights for policy 1, policy_version 782132 (0.0005) [2023-12-27 11:02:35,976][20347] Updated weights for policy 0, policy_version 782595 (0.0007) [2023-12-27 11:02:36,022][20347] Updated weights for policy 0, policy_version 782605 (0.0005) [2023-12-27 11:02:36,071][20347] Updated weights for policy 0, policy_version 782615 (0.0005) [2023-12-27 11:02:36,581][20381] Updated weights for policy 1, policy_version 782142 (0.0009) [2023-12-27 11:02:36,637][20381] Updated weights for policy 1, policy_version 782152 (0.0008) [2023-12-27 11:02:36,702][20381] Updated weights for policy 1, policy_version 782162 (0.0007) [2023-12-27 11:02:36,766][20347] Updated weights for policy 0, policy_version 782625 (0.0006) [2023-12-27 11:02:36,823][20347] Updated weights for policy 0, policy_version 782635 (0.0008) [2023-12-27 11:02:36,877][20347] Updated weights for policy 0, policy_version 782645 (0.0007) [2023-12-27 11:02:36,929][20347] Updated weights for policy 0, policy_version 782655 (0.0008) [2023-12-27 11:02:37,319][20381] Updated weights for policy 1, policy_version 782172 (0.0009) [2023-12-27 11:02:37,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 400646144. Throughput: 0: 10269.1, 1: 10323.2. Samples: 400618140. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:37,363][19170] Avg episode reward: [(0, '912.290'), (1, '922.330')] [2023-12-27 11:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000782656_200384512.pth... [2023-12-27 11:02:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000781472_200081408.pth [2023-12-27 11:02:37,375][20381] Updated weights for policy 1, policy_version 782182 (0.0011) [2023-12-27 11:02:37,427][20381] Updated weights for policy 1, policy_version 782192 (0.0010) [2023-12-27 11:02:37,469][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000782200_200269824.pth... [2023-12-27 11:02:37,473][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000780952_199950336.pth [2023-12-27 11:02:37,638][20347] Updated weights for policy 0, policy_version 782665 (0.0007) [2023-12-27 11:02:37,697][20347] Updated weights for policy 0, policy_version 782675 (0.0007) [2023-12-27 11:02:37,763][20347] Updated weights for policy 0, policy_version 782685 (0.0008) [2023-12-27 11:02:38,043][20381] Updated weights for policy 1, policy_version 782202 (0.0009) [2023-12-27 11:02:38,099][20381] Updated weights for policy 1, policy_version 782212 (0.0008) [2023-12-27 11:02:38,165][20381] Updated weights for policy 1, policy_version 782222 (0.0010) [2023-12-27 11:02:38,218][20381] Updated weights for policy 1, policy_version 782232 (0.0006) [2023-12-27 11:02:38,457][20347] Updated weights for policy 0, policy_version 782695 (0.0006) [2023-12-27 11:02:38,520][20347] Updated weights for policy 0, policy_version 782705 (0.0007) [2023-12-27 11:02:38,580][20347] Updated weights for policy 0, policy_version 782715 (0.0006) [2023-12-27 11:02:38,890][20381] Updated weights for policy 1, policy_version 782242 (0.0010) [2023-12-27 11:02:38,947][20381] Updated weights for policy 1, policy_version 782252 (0.0007) [2023-12-27 11:02:39,018][20381] Updated weights for policy 1, policy_version 782262 (0.0007) [2023-12-27 11:02:39,202][20347] Updated weights for policy 0, policy_version 782725 (0.0007) [2023-12-27 11:02:39,270][20347] Updated weights for policy 0, policy_version 782735 (0.0008) [2023-12-27 11:02:39,334][20347] Updated weights for policy 0, policy_version 782745 (0.0009) [2023-12-27 11:02:39,646][20381] Updated weights for policy 1, policy_version 782272 (0.0007) [2023-12-27 11:02:39,702][20381] Updated weights for policy 1, policy_version 782282 (0.0008) [2023-12-27 11:02:39,756][20381] Updated weights for policy 1, policy_version 782292 (0.0007) [2023-12-27 11:02:40,115][20347] Updated weights for policy 0, policy_version 782755 (0.0009) [2023-12-27 11:02:40,174][20347] Updated weights for policy 0, policy_version 782765 (0.0010) [2023-12-27 11:02:40,234][20347] Updated weights for policy 0, policy_version 782775 (0.0011) [2023-12-27 11:02:40,491][20381] Updated weights for policy 1, policy_version 782302 (0.0007) [2023-12-27 11:02:40,544][20381] Updated weights for policy 1, policy_version 782312 (0.0007) [2023-12-27 11:02:40,604][20381] Updated weights for policy 1, policy_version 782322 (0.0007) [2023-12-27 11:02:40,923][20347] Updated weights for policy 0, policy_version 782785 (0.0011) [2023-12-27 11:02:40,985][20347] Updated weights for policy 0, policy_version 782795 (0.0011) [2023-12-27 11:02:41,044][20347] Updated weights for policy 0, policy_version 782805 (0.0010) [2023-12-27 11:02:41,107][20347] Updated weights for policy 0, policy_version 782815 (0.0008) [2023-12-27 11:02:41,283][20381] Updated weights for policy 1, policy_version 782332 (0.0007) [2023-12-27 11:02:41,343][20381] Updated weights for policy 1, policy_version 782342 (0.0007) [2023-12-27 11:02:41,416][20381] Updated weights for policy 1, policy_version 782352 (0.0008) [2023-12-27 11:02:41,902][20347] Updated weights for policy 0, policy_version 782825 (0.0010) [2023-12-27 11:02:41,955][20347] Updated weights for policy 0, policy_version 782835 (0.0010) [2023-12-27 11:02:42,011][20347] Updated weights for policy 0, policy_version 782845 (0.0010) [2023-12-27 11:02:42,053][20381] Updated weights for policy 1, policy_version 782362 (0.0006) [2023-12-27 11:02:42,106][20381] Updated weights for policy 1, policy_version 782372 (0.0005) [2023-12-27 11:02:42,151][20381] Updated weights for policy 1, policy_version 782382 (0.0006) [2023-12-27 11:02:42,197][20381] Updated weights for policy 1, policy_version 782392 (0.0006) [2023-12-27 11:02:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 400752640. Throughput: 0: 10223.4, 1: 10361.5. Samples: 400740604. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:42,363][19170] Avg episode reward: [(0, '961.110'), (1, '941.680')] [2023-12-27 11:02:42,717][20347] Updated weights for policy 0, policy_version 782855 (0.0010) [2023-12-27 11:02:42,768][20347] Updated weights for policy 0, policy_version 782865 (0.0010) [2023-12-27 11:02:42,823][20347] Updated weights for policy 0, policy_version 782875 (0.0008) [2023-12-27 11:02:42,880][20381] Updated weights for policy 1, policy_version 782402 (0.0010) [2023-12-27 11:02:42,932][20381] Updated weights for policy 1, policy_version 782412 (0.0006) [2023-12-27 11:02:42,980][20381] Updated weights for policy 1, policy_version 782422 (0.0008) [2023-12-27 11:02:43,502][20347] Updated weights for policy 0, policy_version 782885 (0.0010) [2023-12-27 11:02:43,547][20347] Updated weights for policy 0, policy_version 782895 (0.0010) [2023-12-27 11:02:43,562][20291] KL-divergence is very high: 130.3178 [2023-12-27 11:02:43,602][20347] Updated weights for policy 0, policy_version 782905 (0.0010) [2023-12-27 11:02:43,608][20291] KL-divergence is very high: 133.8046 [2023-12-27 11:02:43,655][20381] Updated weights for policy 1, policy_version 782432 (0.0010) [2023-12-27 11:02:43,715][20381] Updated weights for policy 1, policy_version 782442 (0.0006) [2023-12-27 11:02:43,773][20381] Updated weights for policy 1, policy_version 782452 (0.0006) [2023-12-27 11:02:44,308][20347] Updated weights for policy 0, policy_version 782915 (0.0010) [2023-12-27 11:02:44,353][20381] Updated weights for policy 1, policy_version 782462 (0.0005) [2023-12-27 11:02:44,363][20347] Updated weights for policy 0, policy_version 782925 (0.0010) [2023-12-27 11:02:44,400][20381] Updated weights for policy 1, policy_version 782472 (0.0005) [2023-12-27 11:02:44,409][20347] Updated weights for policy 0, policy_version 782935 (0.0008) [2023-12-27 11:02:44,450][20381] Updated weights for policy 1, policy_version 782482 (0.0005) [2023-12-27 11:02:45,057][20381] Updated weights for policy 1, policy_version 782492 (0.0006) [2023-12-27 11:02:45,104][20347] Updated weights for policy 0, policy_version 782945 (0.0008) [2023-12-27 11:02:45,110][20381] Updated weights for policy 1, policy_version 782502 (0.0008) [2023-12-27 11:02:45,160][20347] Updated weights for policy 0, policy_version 782955 (0.0008) [2023-12-27 11:02:45,165][20381] Updated weights for policy 1, policy_version 782512 (0.0008) [2023-12-27 11:02:45,211][20347] Updated weights for policy 0, policy_version 782965 (0.0008) [2023-12-27 11:02:45,259][20347] Updated weights for policy 0, policy_version 782975 (0.0008) [2023-12-27 11:02:45,880][20347] Updated weights for policy 0, policy_version 782985 (0.0006) [2023-12-27 11:02:45,880][20381] Updated weights for policy 1, policy_version 782522 (0.0010) [2023-12-27 11:02:45,929][20381] Updated weights for policy 1, policy_version 782532 (0.0011) [2023-12-27 11:02:45,937][20347] Updated weights for policy 0, policy_version 782995 (0.0006) [2023-12-27 11:02:45,982][20381] Updated weights for policy 1, policy_version 782542 (0.0010) [2023-12-27 11:02:45,989][20347] Updated weights for policy 0, policy_version 783005 (0.0008) [2023-12-27 11:02:46,033][20381] Updated weights for policy 1, policy_version 782552 (0.0006) [2023-12-27 11:02:46,598][20347] Updated weights for policy 0, policy_version 783015 (0.0009) [2023-12-27 11:02:46,651][20347] Updated weights for policy 0, policy_version 783025 (0.0010) [2023-12-27 11:02:46,661][20381] Updated weights for policy 1, policy_version 782562 (0.0010) [2023-12-27 11:02:46,707][20347] Updated weights for policy 0, policy_version 783035 (0.0010) [2023-12-27 11:02:46,714][20381] Updated weights for policy 1, policy_version 782572 (0.0010) [2023-12-27 11:02:46,773][20381] Updated weights for policy 1, policy_version 782582 (0.0010) [2023-12-27 11:02:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.7, 300 sec: 20521.7). Total num frames: 400850944. Throughput: 0: 10223.4, 1: 10438.1. Samples: 400866020. Policy #0 lag: (min: 28.0, avg: 34.1, max: 60.0) [2023-12-27 11:02:47,363][19170] Avg episode reward: [(0, '951.260'), (1, '970.780')] [2023-12-27 11:02:47,378][20381] Updated weights for policy 1, policy_version 782592 (0.0007) [2023-12-27 11:02:47,392][20347] Updated weights for policy 0, policy_version 783045 (0.0008) [2023-12-27 11:02:47,435][20381] Updated weights for policy 1, policy_version 782602 (0.0006) [2023-12-27 11:02:47,448][20347] Updated weights for policy 0, policy_version 783055 (0.0006) [2023-12-27 11:02:47,492][20381] Updated weights for policy 1, policy_version 782612 (0.0005) [2023-12-27 11:02:47,505][20347] Updated weights for policy 0, policy_version 783065 (0.0007) [2023-12-27 11:02:48,117][20381] Updated weights for policy 1, policy_version 782622 (0.0007) [2023-12-27 11:02:48,166][20381] Updated weights for policy 1, policy_version 782632 (0.0008) [2023-12-27 11:02:48,208][20347] Updated weights for policy 0, policy_version 783075 (0.0007) [2023-12-27 11:02:48,221][20381] Updated weights for policy 1, policy_version 782642 (0.0007) [2023-12-27 11:02:48,271][20347] Updated weights for policy 0, policy_version 783085 (0.0005) [2023-12-27 11:02:48,338][20347] Updated weights for policy 0, policy_version 783095 (0.0006) [2023-12-27 11:02:48,948][20381] Updated weights for policy 1, policy_version 782652 (0.0008) [2023-12-27 11:02:48,984][20347] Updated weights for policy 0, policy_version 783105 (0.0008) [2023-12-27 11:02:49,002][20381] Updated weights for policy 1, policy_version 782662 (0.0008) [2023-12-27 11:02:49,044][20347] Updated weights for policy 0, policy_version 783115 (0.0006) [2023-12-27 11:02:49,054][20381] Updated weights for policy 1, policy_version 782672 (0.0006) [2023-12-27 11:02:49,099][20347] Updated weights for policy 0, policy_version 783125 (0.0006) [2023-12-27 11:02:49,151][20347] Updated weights for policy 0, policy_version 783135 (0.0006) [2023-12-27 11:02:49,819][20347] Updated weights for policy 0, policy_version 783145 (0.0008) [2023-12-27 11:02:49,870][20381] Updated weights for policy 1, policy_version 782682 (0.0006) [2023-12-27 11:02:49,887][20347] Updated weights for policy 0, policy_version 783155 (0.0007) [2023-12-27 11:02:49,923][20381] Updated weights for policy 1, policy_version 782692 (0.0007) [2023-12-27 11:02:49,950][20347] Updated weights for policy 0, policy_version 783165 (0.0007) [2023-12-27 11:02:49,985][20381] Updated weights for policy 1, policy_version 782702 (0.0008) [2023-12-27 11:02:50,041][20381] Updated weights for policy 1, policy_version 782712 (0.0008) [2023-12-27 11:02:50,656][20347] Updated weights for policy 0, policy_version 783175 (0.0006) [2023-12-27 11:02:50,715][20347] Updated weights for policy 0, policy_version 783185 (0.0005) [2023-12-27 11:02:50,728][20381] Updated weights for policy 1, policy_version 782722 (0.0008) [2023-12-27 11:02:50,773][20347] Updated weights for policy 0, policy_version 783195 (0.0006) [2023-12-27 11:02:50,791][20381] Updated weights for policy 1, policy_version 782732 (0.0007) [2023-12-27 11:02:50,845][20381] Updated weights for policy 1, policy_version 782742 (0.0007) [2023-12-27 11:02:51,419][20347] Updated weights for policy 0, policy_version 783205 (0.0009) [2023-12-27 11:02:51,478][20347] Updated weights for policy 0, policy_version 783215 (0.0010) [2023-12-27 11:02:51,540][20347] Updated weights for policy 0, policy_version 783225 (0.0009) [2023-12-27 11:02:51,557][20381] Updated weights for policy 1, policy_version 782752 (0.0006) [2023-12-27 11:02:51,612][20381] Updated weights for policy 1, policy_version 782762 (0.0007) [2023-12-27 11:02:51,672][20381] Updated weights for policy 1, policy_version 782772 (0.0007) [2023-12-27 11:02:52,228][20347] Updated weights for policy 0, policy_version 783235 (0.0009) [2023-12-27 11:02:52,279][20347] Updated weights for policy 0, policy_version 783245 (0.0008) [2023-12-27 11:02:52,342][20347] Updated weights for policy 0, policy_version 783255 (0.0008) [2023-12-27 11:02:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 400949248. Throughput: 0: 10185.6, 1: 10418.5. Samples: 400925704. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:02:52,363][19170] Avg episode reward: [(0, '970.660'), (1, '951.410')] [2023-12-27 11:02:52,371][20381] Updated weights for policy 1, policy_version 782782 (0.0008) [2023-12-27 11:02:52,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000783264_200540160.pth... [2023-12-27 11:02:52,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000782048_200228864.pth [2023-12-27 11:02:52,430][20381] Updated weights for policy 1, policy_version 782792 (0.0006) [2023-12-27 11:02:52,486][20381] Updated weights for policy 1, policy_version 782802 (0.0008) [2023-12-27 11:02:52,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000782808_200425472.pth... [2023-12-27 11:02:52,523][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000781560_200105984.pth [2023-12-27 11:02:53,012][20347] Updated weights for policy 0, policy_version 783265 (0.0008) [2023-12-27 11:02:53,071][20347] Updated weights for policy 0, policy_version 783275 (0.0007) [2023-12-27 11:02:53,130][20347] Updated weights for policy 0, policy_version 783285 (0.0007) [2023-12-27 11:02:53,157][20381] Updated weights for policy 1, policy_version 782812 (0.0007) [2023-12-27 11:02:53,194][20347] Updated weights for policy 0, policy_version 783295 (0.0008) [2023-12-27 11:02:53,219][20381] Updated weights for policy 1, policy_version 782822 (0.0006) [2023-12-27 11:02:53,271][20381] Updated weights for policy 1, policy_version 782832 (0.0008) [2023-12-27 11:02:53,847][20347] Updated weights for policy 0, policy_version 783305 (0.0009) [2023-12-27 11:02:53,853][20381] Updated weights for policy 1, policy_version 782842 (0.0008) [2023-12-27 11:02:53,906][20347] Updated weights for policy 0, policy_version 783315 (0.0010) [2023-12-27 11:02:53,910][20381] Updated weights for policy 1, policy_version 782852 (0.0008) [2023-12-27 11:02:53,956][20347] Updated weights for policy 0, policy_version 783325 (0.0010) [2023-12-27 11:02:53,965][20381] Updated weights for policy 1, policy_version 782862 (0.0009) [2023-12-27 11:02:54,021][20381] Updated weights for policy 1, policy_version 782872 (0.0010) [2023-12-27 11:02:54,647][20347] Updated weights for policy 0, policy_version 783335 (0.0010) [2023-12-27 11:02:54,690][20381] Updated weights for policy 1, policy_version 782882 (0.0011) [2023-12-27 11:02:54,701][20347] Updated weights for policy 0, policy_version 783345 (0.0010) [2023-12-27 11:02:54,742][20381] Updated weights for policy 1, policy_version 782892 (0.0011) [2023-12-27 11:02:54,758][20347] Updated weights for policy 0, policy_version 783355 (0.0010) [2023-12-27 11:02:54,803][20381] Updated weights for policy 1, policy_version 782902 (0.0010) [2023-12-27 11:02:55,411][20347] Updated weights for policy 0, policy_version 783365 (0.0010) [2023-12-27 11:02:55,427][20381] Updated weights for policy 1, policy_version 782912 (0.0006) [2023-12-27 11:02:55,456][20347] Updated weights for policy 0, policy_version 783375 (0.0010) [2023-12-27 11:02:55,473][20381] Updated weights for policy 1, policy_version 782922 (0.0007) [2023-12-27 11:02:55,507][20347] Updated weights for policy 0, policy_version 783385 (0.0010) [2023-12-27 11:02:55,525][20381] Updated weights for policy 1, policy_version 782932 (0.0010) [2023-12-27 11:02:56,129][20347] Updated weights for policy 0, policy_version 783395 (0.0010) [2023-12-27 11:02:56,144][20381] Updated weights for policy 1, policy_version 782942 (0.0011) [2023-12-27 11:02:56,175][20347] Updated weights for policy 0, policy_version 783405 (0.0011) [2023-12-27 11:02:56,194][20381] Updated weights for policy 1, policy_version 782952 (0.0011) [2023-12-27 11:02:56,224][20347] Updated weights for policy 0, policy_version 783415 (0.0010) [2023-12-27 11:02:56,251][20381] Updated weights for policy 1, policy_version 782962 (0.0011) [2023-12-27 11:02:56,873][20347] Updated weights for policy 0, policy_version 783425 (0.0011) [2023-12-27 11:02:56,885][20381] Updated weights for policy 1, policy_version 782972 (0.0005) [2023-12-27 11:02:56,927][20347] Updated weights for policy 0, policy_version 783435 (0.0010) [2023-12-27 11:02:56,932][20381] Updated weights for policy 1, policy_version 782982 (0.0006) [2023-12-27 11:02:56,974][20347] Updated weights for policy 0, policy_version 783445 (0.0010) [2023-12-27 11:02:56,980][20381] Updated weights for policy 1, policy_version 782992 (0.0005) [2023-12-27 11:02:57,022][20347] Updated weights for policy 0, policy_version 783455 (0.0010) [2023-12-27 11:02:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 401063936. Throughput: 0: 10189.8, 1: 10464.7. Samples: 401050856. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:02:57,363][19170] Avg episode reward: [(0, '941.540'), (1, '980.630')] [2023-12-27 11:02:57,641][20381] Updated weights for policy 1, policy_version 783002 (0.0006) [2023-12-27 11:02:57,693][20381] Updated weights for policy 1, policy_version 783012 (0.0007) [2023-12-27 11:02:57,695][20347] Updated weights for policy 0, policy_version 783465 (0.0006) [2023-12-27 11:02:57,742][20381] Updated weights for policy 1, policy_version 783022 (0.0006) [2023-12-27 11:02:57,753][20347] Updated weights for policy 0, policy_version 783475 (0.0006) [2023-12-27 11:02:57,789][20381] Updated weights for policy 1, policy_version 783032 (0.0007) [2023-12-27 11:02:57,812][20347] Updated weights for policy 0, policy_version 783485 (0.0006) [2023-12-27 11:02:58,477][20347] Updated weights for policy 0, policy_version 783495 (0.0008) [2023-12-27 11:02:58,515][20381] Updated weights for policy 1, policy_version 783042 (0.0008) [2023-12-27 11:02:58,536][20347] Updated weights for policy 0, policy_version 783505 (0.0008) [2023-12-27 11:02:58,577][20381] Updated weights for policy 1, policy_version 783052 (0.0007) [2023-12-27 11:02:58,602][20347] Updated weights for policy 0, policy_version 783515 (0.0008) [2023-12-27 11:02:58,644][20381] Updated weights for policy 1, policy_version 783062 (0.0008) [2023-12-27 11:02:59,433][20347] Updated weights for policy 0, policy_version 783525 (0.0008) [2023-12-27 11:02:59,489][20381] Updated weights for policy 1, policy_version 783072 (0.0007) [2023-12-27 11:02:59,499][20347] Updated weights for policy 0, policy_version 783535 (0.0008) [2023-12-27 11:02:59,542][20381] Updated weights for policy 1, policy_version 783082 (0.0007) [2023-12-27 11:02:59,559][20347] Updated weights for policy 0, policy_version 783545 (0.0006) [2023-12-27 11:02:59,565][20292] KL-divergence is very high: 106.7336 [2023-12-27 11:02:59,599][20381] Updated weights for policy 1, policy_version 783092 (0.0007) [2023-12-27 11:03:00,249][20347] Updated weights for policy 0, policy_version 783555 (0.0007) [2023-12-27 11:03:00,299][20381] Updated weights for policy 1, policy_version 783102 (0.0007) [2023-12-27 11:03:00,308][20347] Updated weights for policy 0, policy_version 783565 (0.0006) [2023-12-27 11:03:00,349][20381] Updated weights for policy 1, policy_version 783112 (0.0007) [2023-12-27 11:03:00,367][20347] Updated weights for policy 0, policy_version 783575 (0.0007) [2023-12-27 11:03:00,399][20381] Updated weights for policy 1, policy_version 783122 (0.0006) [2023-12-27 11:03:01,022][20381] Updated weights for policy 1, policy_version 783132 (0.0008) [2023-12-27 11:03:01,078][20347] Updated weights for policy 0, policy_version 783585 (0.0007) [2023-12-27 11:03:01,082][20381] Updated weights for policy 1, policy_version 783142 (0.0007) [2023-12-27 11:03:01,140][20347] Updated weights for policy 0, policy_version 783595 (0.0008) [2023-12-27 11:03:01,141][20381] Updated weights for policy 1, policy_version 783152 (0.0007) [2023-12-27 11:03:01,200][20347] Updated weights for policy 0, policy_version 783605 (0.0007) [2023-12-27 11:03:01,261][20347] Updated weights for policy 0, policy_version 783615 (0.0006) [2023-12-27 11:03:01,821][20381] Updated weights for policy 1, policy_version 783162 (0.0008) [2023-12-27 11:03:01,871][20381] Updated weights for policy 1, policy_version 783172 (0.0006) [2023-12-27 11:03:01,920][20381] Updated weights for policy 1, policy_version 783182 (0.0007) [2023-12-27 11:03:01,969][20381] Updated weights for policy 1, policy_version 783192 (0.0005) [2023-12-27 11:03:01,974][20347] Updated weights for policy 0, policy_version 783625 (0.0007) [2023-12-27 11:03:02,030][20347] Updated weights for policy 0, policy_version 783635 (0.0007) [2023-12-27 11:03:02,089][20347] Updated weights for policy 0, policy_version 783645 (0.0007) [2023-12-27 11:03:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 401162240. Throughput: 0: 10141.8, 1: 10428.2. Samples: 401170352. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:02,363][19170] Avg episode reward: [(0, '970.880'), (1, '930.490')] [2023-12-27 11:03:02,622][20381] Updated weights for policy 1, policy_version 783202 (0.0008) [2023-12-27 11:03:02,674][20381] Updated weights for policy 1, policy_version 783212 (0.0008) [2023-12-27 11:03:02,723][20381] Updated weights for policy 1, policy_version 783222 (0.0007) [2023-12-27 11:03:02,809][20347] Updated weights for policy 0, policy_version 783655 (0.0007) [2023-12-27 11:03:02,864][20347] Updated weights for policy 0, policy_version 783665 (0.0007) [2023-12-27 11:03:02,915][20347] Updated weights for policy 0, policy_version 783675 (0.0007) [2023-12-27 11:03:03,413][20381] Updated weights for policy 1, policy_version 783232 (0.0007) [2023-12-27 11:03:03,463][20381] Updated weights for policy 1, policy_version 783242 (0.0007) [2023-12-27 11:03:03,512][20381] Updated weights for policy 1, policy_version 783252 (0.0007) [2023-12-27 11:03:03,604][20347] Updated weights for policy 0, policy_version 783685 (0.0009) [2023-12-27 11:03:03,658][20347] Updated weights for policy 0, policy_version 783695 (0.0010) [2023-12-27 11:03:03,708][20347] Updated weights for policy 0, policy_version 783705 (0.0010) [2023-12-27 11:03:04,179][20381] Updated weights for policy 1, policy_version 783262 (0.0007) [2023-12-27 11:03:04,242][20381] Updated weights for policy 1, policy_version 783272 (0.0008) [2023-12-27 11:03:04,300][20381] Updated weights for policy 1, policy_version 783282 (0.0008) [2023-12-27 11:03:04,413][20347] Updated weights for policy 0, policy_version 783715 (0.0008) [2023-12-27 11:03:04,469][20347] Updated weights for policy 0, policy_version 783725 (0.0008) [2023-12-27 11:03:04,521][20347] Updated weights for policy 0, policy_version 783735 (0.0006) [2023-12-27 11:03:05,013][20381] Updated weights for policy 1, policy_version 783292 (0.0008) [2023-12-27 11:03:05,081][20347] Updated weights for policy 0, policy_version 783745 (0.0006) [2023-12-27 11:03:05,083][20381] Updated weights for policy 1, policy_version 783302 (0.0008) [2023-12-27 11:03:05,137][20347] Updated weights for policy 0, policy_version 783755 (0.0007) [2023-12-27 11:03:05,141][20381] Updated weights for policy 1, policy_version 783312 (0.0006) [2023-12-27 11:03:05,183][20347] Updated weights for policy 0, policy_version 783765 (0.0008) [2023-12-27 11:03:05,228][20347] Updated weights for policy 0, policy_version 783775 (0.0006) [2023-12-27 11:03:05,783][20381] Updated weights for policy 1, policy_version 783322 (0.0005) [2023-12-27 11:03:05,836][20381] Updated weights for policy 1, policy_version 783332 (0.0008) [2023-12-27 11:03:05,857][20347] Updated weights for policy 0, policy_version 783785 (0.0006) [2023-12-27 11:03:05,893][20381] Updated weights for policy 1, policy_version 783342 (0.0006) [2023-12-27 11:03:05,914][20347] Updated weights for policy 0, policy_version 783795 (0.0008) [2023-12-27 11:03:05,946][20381] Updated weights for policy 1, policy_version 783352 (0.0009) [2023-12-27 11:03:05,969][20347] Updated weights for policy 0, policy_version 783805 (0.0008) [2023-12-27 11:03:06,686][20347] Updated weights for policy 0, policy_version 783815 (0.0008) [2023-12-27 11:03:06,689][20381] Updated weights for policy 1, policy_version 783362 (0.0006) [2023-12-27 11:03:06,748][20381] Updated weights for policy 1, policy_version 783372 (0.0007) [2023-12-27 11:03:06,748][20347] Updated weights for policy 0, policy_version 783825 (0.0007) [2023-12-27 11:03:06,805][20347] Updated weights for policy 0, policy_version 783835 (0.0006) [2023-12-27 11:03:06,811][20381] Updated weights for policy 1, policy_version 783382 (0.0007) [2023-12-27 11:03:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 401260544. Throughput: 0: 10150.4, 1: 10388.2. Samples: 401232544. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:07,363][19170] Avg episode reward: [(0, '970.740'), (1, '833.310')] [2023-12-27 11:03:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000783840_200687616.pth... [2023-12-27 11:03:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000783384_200572928.pth... [2023-12-27 11:03:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000782656_200384512.pth [2023-12-27 11:03:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000782200_200269824.pth [2023-12-27 11:03:07,497][20347] Updated weights for policy 0, policy_version 783845 (0.0007) [2023-12-27 11:03:07,501][20381] Updated weights for policy 1, policy_version 783392 (0.0005) [2023-12-27 11:03:07,507][20292] KL-divergence is very high: 320.0860 [2023-12-27 11:03:07,552][20347] Updated weights for policy 0, policy_version 783855 (0.0007) [2023-12-27 11:03:07,552][20381] Updated weights for policy 1, policy_version 783402 (0.0005) [2023-12-27 11:03:07,599][20381] Updated weights for policy 1, policy_version 783412 (0.0005) [2023-12-27 11:03:07,606][20347] Updated weights for policy 0, policy_version 783865 (0.0007) [2023-12-27 11:03:08,193][20381] Updated weights for policy 1, policy_version 783422 (0.0006) [2023-12-27 11:03:08,249][20381] Updated weights for policy 1, policy_version 783432 (0.0007) [2023-12-27 11:03:08,295][20381] Updated weights for policy 1, policy_version 783442 (0.0009) [2023-12-27 11:03:08,330][20347] Updated weights for policy 0, policy_version 783875 (0.0007) [2023-12-27 11:03:08,395][20347] Updated weights for policy 0, policy_version 783885 (0.0006) [2023-12-27 11:03:08,460][20347] Updated weights for policy 0, policy_version 783895 (0.0006) [2023-12-27 11:03:08,968][20381] Updated weights for policy 1, policy_version 783452 (0.0007) [2023-12-27 11:03:09,033][20381] Updated weights for policy 1, policy_version 783462 (0.0007) [2023-12-27 11:03:09,096][20381] Updated weights for policy 1, policy_version 783472 (0.0007) [2023-12-27 11:03:09,159][20347] Updated weights for policy 0, policy_version 783905 (0.0007) [2023-12-27 11:03:09,218][20347] Updated weights for policy 0, policy_version 783915 (0.0006) [2023-12-27 11:03:09,274][20347] Updated weights for policy 0, policy_version 783925 (0.0007) [2023-12-27 11:03:09,333][20347] Updated weights for policy 0, policy_version 783935 (0.0007) [2023-12-27 11:03:09,822][20381] Updated weights for policy 1, policy_version 783482 (0.0008) [2023-12-27 11:03:09,884][20381] Updated weights for policy 1, policy_version 783492 (0.0008) [2023-12-27 11:03:09,941][20381] Updated weights for policy 1, policy_version 783502 (0.0006) [2023-12-27 11:03:09,990][20381] Updated weights for policy 1, policy_version 783512 (0.0007) [2023-12-27 11:03:10,041][20347] Updated weights for policy 0, policy_version 783945 (0.0007) [2023-12-27 11:03:10,105][20347] Updated weights for policy 0, policy_version 783955 (0.0007) [2023-12-27 11:03:10,162][20347] Updated weights for policy 0, policy_version 783965 (0.0007) [2023-12-27 11:03:10,726][20381] Updated weights for policy 1, policy_version 783522 (0.0008) [2023-12-27 11:03:10,773][20347] Updated weights for policy 0, policy_version 783975 (0.0005) [2023-12-27 11:03:10,786][20381] Updated weights for policy 1, policy_version 783532 (0.0006) [2023-12-27 11:03:10,832][20347] Updated weights for policy 0, policy_version 783985 (0.0006) [2023-12-27 11:03:10,845][20381] Updated weights for policy 1, policy_version 783542 (0.0006) [2023-12-27 11:03:10,887][20347] Updated weights for policy 0, policy_version 783995 (0.0007) [2023-12-27 11:03:11,541][20347] Updated weights for policy 0, policy_version 784005 (0.0008) [2023-12-27 11:03:11,599][20347] Updated weights for policy 0, policy_version 784015 (0.0007) [2023-12-27 11:03:11,601][20381] Updated weights for policy 1, policy_version 783552 (0.0006) [2023-12-27 11:03:11,664][20347] Updated weights for policy 0, policy_version 784025 (0.0006) [2023-12-27 11:03:11,672][20381] Updated weights for policy 1, policy_version 783562 (0.0008) [2023-12-27 11:03:11,739][20381] Updated weights for policy 1, policy_version 783572 (0.0008) [2023-12-27 11:03:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 401358848. Throughput: 0: 10128.5, 1: 10343.9. Samples: 401353820. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:12,363][19170] Avg episode reward: [(0, '932.910'), (1, '864.320')] [2023-12-27 11:03:12,366][20347] Updated weights for policy 0, policy_version 784035 (0.0008) [2023-12-27 11:03:12,423][20347] Updated weights for policy 0, policy_version 784045 (0.0008) [2023-12-27 11:03:12,437][20381] Updated weights for policy 1, policy_version 783582 (0.0007) [2023-12-27 11:03:12,477][20347] Updated weights for policy 0, policy_version 784055 (0.0007) [2023-12-27 11:03:12,495][20381] Updated weights for policy 1, policy_version 783592 (0.0006) [2023-12-27 11:03:12,546][20381] Updated weights for policy 1, policy_version 783602 (0.0007) [2023-12-27 11:03:13,177][20347] Updated weights for policy 0, policy_version 784065 (0.0006) [2023-12-27 11:03:13,230][20347] Updated weights for policy 0, policy_version 784075 (0.0007) [2023-12-27 11:03:13,258][20381] Updated weights for policy 1, policy_version 783612 (0.0008) [2023-12-27 11:03:13,283][20347] Updated weights for policy 0, policy_version 784085 (0.0006) [2023-12-27 11:03:13,307][20381] Updated weights for policy 1, policy_version 783622 (0.0006) [2023-12-27 11:03:13,332][20347] Updated weights for policy 0, policy_version 784095 (0.0006) [2023-12-27 11:03:13,359][20381] Updated weights for policy 1, policy_version 783632 (0.0006) [2023-12-27 11:03:13,998][20347] Updated weights for policy 0, policy_version 784105 (0.0005) [2023-12-27 11:03:14,041][20381] Updated weights for policy 1, policy_version 783642 (0.0008) [2023-12-27 11:03:14,064][20347] Updated weights for policy 0, policy_version 784115 (0.0007) [2023-12-27 11:03:14,105][20381] Updated weights for policy 1, policy_version 783652 (0.0006) [2023-12-27 11:03:14,114][20347] Updated weights for policy 0, policy_version 784125 (0.0006) [2023-12-27 11:03:14,164][20381] Updated weights for policy 1, policy_version 783662 (0.0007) [2023-12-27 11:03:14,215][20381] Updated weights for policy 1, policy_version 783672 (0.0008) [2023-12-27 11:03:14,680][20347] Updated weights for policy 0, policy_version 784135 (0.0006) [2023-12-27 11:03:14,743][20347] Updated weights for policy 0, policy_version 784145 (0.0008) [2023-12-27 11:03:14,801][20347] Updated weights for policy 0, policy_version 784155 (0.0007) [2023-12-27 11:03:14,920][20381] Updated weights for policy 1, policy_version 783682 (0.0007) [2023-12-27 11:03:14,984][20381] Updated weights for policy 1, policy_version 783692 (0.0010) [2023-12-27 11:03:15,045][20381] Updated weights for policy 1, policy_version 783702 (0.0011) [2023-12-27 11:03:15,505][20347] Updated weights for policy 0, policy_version 784165 (0.0006) [2023-12-27 11:03:15,557][20347] Updated weights for policy 0, policy_version 784175 (0.0005) [2023-12-27 11:03:15,644][20347] Updated weights for policy 0, policy_version 784185 (0.0005) [2023-12-27 11:03:15,724][20381] Updated weights for policy 1, policy_version 783712 (0.0008) [2023-12-27 11:03:15,778][20381] Updated weights for policy 1, policy_version 783722 (0.0005) [2023-12-27 11:03:15,834][20381] Updated weights for policy 1, policy_version 783732 (0.0005) [2023-12-27 11:03:16,259][20347] Updated weights for policy 0, policy_version 784195 (0.0006) [2023-12-27 11:03:16,314][20347] Updated weights for policy 0, policy_version 784205 (0.0008) [2023-12-27 11:03:16,369][20347] Updated weights for policy 0, policy_version 784215 (0.0007) [2023-12-27 11:03:16,413][20381] Updated weights for policy 1, policy_version 783742 (0.0005) [2023-12-27 11:03:16,479][20381] Updated weights for policy 1, policy_version 783752 (0.0005) [2023-12-27 11:03:16,544][20381] Updated weights for policy 1, policy_version 783762 (0.0005) [2023-12-27 11:03:16,983][20347] Updated weights for policy 0, policy_version 784225 (0.0006) [2023-12-27 11:03:17,037][20347] Updated weights for policy 0, policy_version 784235 (0.0005) [2023-12-27 11:03:17,045][20291] KL-divergence is very high: 104.0760 [2023-12-27 11:03:17,094][20347] Updated weights for policy 0, policy_version 784245 (0.0005) [2023-12-27 11:03:17,130][20381] Updated weights for policy 1, policy_version 783772 (0.0006) [2023-12-27 11:03:17,150][20347] Updated weights for policy 0, policy_version 784255 (0.0006) [2023-12-27 11:03:17,176][20381] Updated weights for policy 1, policy_version 783782 (0.0007) [2023-12-27 11:03:17,226][20381] Updated weights for policy 1, policy_version 783792 (0.0006) [2023-12-27 11:03:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 401473536. Throughput: 0: 10155.9, 1: 10332.3. Samples: 401478752. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:17,364][19170] Avg episode reward: [(0, '923.350'), (1, '844.060')] [2023-12-27 11:03:17,732][20291] KL-divergence is very high: 132.1905 [2023-12-27 11:03:17,787][20347] Updated weights for policy 0, policy_version 784265 (0.0007) [2023-12-27 11:03:17,832][20291] KL-divergence is very high: 100.2983 [2023-12-27 11:03:17,852][20347] Updated weights for policy 0, policy_version 784275 (0.0006) [2023-12-27 11:03:17,906][20347] Updated weights for policy 0, policy_version 784285 (0.0008) [2023-12-27 11:03:17,912][20381] Updated weights for policy 1, policy_version 783802 (0.0007) [2023-12-27 11:03:17,967][20381] Updated weights for policy 1, policy_version 783812 (0.0008) [2023-12-27 11:03:18,030][20381] Updated weights for policy 1, policy_version 783822 (0.0006) [2023-12-27 11:03:18,092][20381] Updated weights for policy 1, policy_version 783832 (0.0005) [2023-12-27 11:03:18,657][20347] Updated weights for policy 0, policy_version 784295 (0.0006) [2023-12-27 11:03:18,715][20347] Updated weights for policy 0, policy_version 784305 (0.0006) [2023-12-27 11:03:18,728][20381] Updated weights for policy 1, policy_version 783842 (0.0008) [2023-12-27 11:03:18,771][20347] Updated weights for policy 0, policy_version 784315 (0.0006) [2023-12-27 11:03:18,783][20381] Updated weights for policy 1, policy_version 783852 (0.0007) [2023-12-27 11:03:18,841][20381] Updated weights for policy 1, policy_version 783862 (0.0007) [2023-12-27 11:03:19,454][20347] Updated weights for policy 0, policy_version 784325 (0.0008) [2023-12-27 11:03:19,473][20381] Updated weights for policy 1, policy_version 783872 (0.0007) [2023-12-27 11:03:19,521][20347] Updated weights for policy 0, policy_version 784335 (0.0008) [2023-12-27 11:03:19,534][20381] Updated weights for policy 1, policy_version 783882 (0.0006) [2023-12-27 11:03:19,577][20347] Updated weights for policy 0, policy_version 784345 (0.0007) [2023-12-27 11:03:19,591][20381] Updated weights for policy 1, policy_version 783892 (0.0005) [2023-12-27 11:03:20,257][20381] Updated weights for policy 1, policy_version 783902 (0.0007) [2023-12-27 11:03:20,270][20347] Updated weights for policy 0, policy_version 784355 (0.0007) [2023-12-27 11:03:20,318][20381] Updated weights for policy 1, policy_version 783912 (0.0006) [2023-12-27 11:03:20,339][20347] Updated weights for policy 0, policy_version 784365 (0.0008) [2023-12-27 11:03:20,379][20381] Updated weights for policy 1, policy_version 783922 (0.0006) [2023-12-27 11:03:20,401][20347] Updated weights for policy 0, policy_version 784375 (0.0006) [2023-12-27 11:03:21,045][20347] Updated weights for policy 0, policy_version 784385 (0.0006) [2023-12-27 11:03:21,115][20347] Updated weights for policy 0, policy_version 784395 (0.0008) [2023-12-27 11:03:21,122][20381] Updated weights for policy 1, policy_version 783932 (0.0006) [2023-12-27 11:03:21,180][20347] Updated weights for policy 0, policy_version 784405 (0.0008) [2023-12-27 11:03:21,189][20381] Updated weights for policy 1, policy_version 783942 (0.0008) [2023-12-27 11:03:21,244][20347] Updated weights for policy 0, policy_version 784415 (0.0008) [2023-12-27 11:03:21,252][20381] Updated weights for policy 1, policy_version 783952 (0.0007) [2023-12-27 11:03:21,963][20347] Updated weights for policy 0, policy_version 784425 (0.0008) [2023-12-27 11:03:21,968][20381] Updated weights for policy 1, policy_version 783962 (0.0009) [2023-12-27 11:03:22,019][20381] Updated weights for policy 1, policy_version 783972 (0.0006) [2023-12-27 11:03:22,020][20347] Updated weights for policy 0, policy_version 784435 (0.0009) [2023-12-27 11:03:22,075][20347] Updated weights for policy 0, policy_version 784445 (0.0008) [2023-12-27 11:03:22,077][20381] Updated weights for policy 1, policy_version 783982 (0.0007) [2023-12-27 11:03:22,134][20381] Updated weights for policy 1, policy_version 783992 (0.0005) [2023-12-27 11:03:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 401571840. Throughput: 0: 10176.2, 1: 10321.7. Samples: 401540544. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:22,363][19170] Avg episode reward: [(0, '922.070'), (1, '843.330')] [2023-12-27 11:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000783992_200728576.pth... [2023-12-27 11:03:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000784448_200843264.pth... [2023-12-27 11:03:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000782808_200425472.pth [2023-12-27 11:03:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000783264_200540160.pth [2023-12-27 11:03:22,785][20347] Updated weights for policy 0, policy_version 784455 (0.0006) [2023-12-27 11:03:22,851][20347] Updated weights for policy 0, policy_version 784465 (0.0007) [2023-12-27 11:03:22,876][20381] Updated weights for policy 1, policy_version 784002 (0.0006) [2023-12-27 11:03:22,907][20347] Updated weights for policy 0, policy_version 784475 (0.0007) [2023-12-27 11:03:22,930][20381] Updated weights for policy 1, policy_version 784012 (0.0006) [2023-12-27 11:03:22,979][20381] Updated weights for policy 1, policy_version 784022 (0.0009) [2023-12-27 11:03:23,539][20347] Updated weights for policy 0, policy_version 784485 (0.0007) [2023-12-27 11:03:23,578][20291] KL-divergence is very high: 529.2112 [2023-12-27 11:03:23,602][20347] Updated weights for policy 0, policy_version 784495 (0.0005) [2023-12-27 11:03:23,641][20381] Updated weights for policy 1, policy_version 784032 (0.0007) [2023-12-27 11:03:23,658][20347] Updated weights for policy 0, policy_version 784505 (0.0006) [2023-12-27 11:03:23,705][20381] Updated weights for policy 1, policy_version 784042 (0.0007) [2023-12-27 11:03:23,765][20381] Updated weights for policy 1, policy_version 784052 (0.0008) [2023-12-27 11:03:24,160][20347] Updated weights for policy 0, policy_version 784515 (0.0005) [2023-12-27 11:03:24,210][20347] Updated weights for policy 0, policy_version 784525 (0.0006) [2023-12-27 11:03:24,260][20347] Updated weights for policy 0, policy_version 784535 (0.0007) [2023-12-27 11:03:24,399][20381] Updated weights for policy 1, policy_version 784062 (0.0006) [2023-12-27 11:03:24,456][20381] Updated weights for policy 1, policy_version 784072 (0.0006) [2023-12-27 11:03:24,511][20381] Updated weights for policy 1, policy_version 784082 (0.0008) [2023-12-27 11:03:24,967][20347] Updated weights for policy 0, policy_version 784545 (0.0008) [2023-12-27 11:03:25,024][20347] Updated weights for policy 0, policy_version 784555 (0.0007) [2023-12-27 11:03:25,077][20347] Updated weights for policy 0, policy_version 784565 (0.0007) [2023-12-27 11:03:25,085][20381] Updated weights for policy 1, policy_version 784092 (0.0009) [2023-12-27 11:03:25,125][20347] Updated weights for policy 0, policy_version 784575 (0.0006) [2023-12-27 11:03:25,133][20381] Updated weights for policy 1, policy_version 784102 (0.0008) [2023-12-27 11:03:25,183][20381] Updated weights for policy 1, policy_version 784112 (0.0007) [2023-12-27 11:03:25,833][20347] Updated weights for policy 0, policy_version 784585 (0.0007) [2023-12-27 11:03:25,859][20381] Updated weights for policy 1, policy_version 784122 (0.0007) [2023-12-27 11:03:25,876][20347] Updated weights for policy 0, policy_version 784595 (0.0007) [2023-12-27 11:03:25,911][20381] Updated weights for policy 1, policy_version 784132 (0.0005) [2023-12-27 11:03:25,925][20347] Updated weights for policy 0, policy_version 784605 (0.0006) [2023-12-27 11:03:25,970][20381] Updated weights for policy 1, policy_version 784142 (0.0009) [2023-12-27 11:03:26,031][20381] Updated weights for policy 1, policy_version 784152 (0.0005) [2023-12-27 11:03:26,542][20347] Updated weights for policy 0, policy_version 784615 (0.0005) [2023-12-27 11:03:26,581][20381] Updated weights for policy 1, policy_version 784162 (0.0007) [2023-12-27 11:03:26,588][20347] Updated weights for policy 0, policy_version 784625 (0.0005) [2023-12-27 11:03:26,630][20347] Updated weights for policy 0, policy_version 784635 (0.0009) [2023-12-27 11:03:26,633][20381] Updated weights for policy 1, policy_version 784172 (0.0008) [2023-12-27 11:03:26,686][20381] Updated weights for policy 1, policy_version 784182 (0.0008) [2023-12-27 11:03:27,244][20347] Updated weights for policy 0, policy_version 784645 (0.0008) [2023-12-27 11:03:27,305][20347] Updated weights for policy 0, policy_version 784655 (0.0008) [2023-12-27 11:03:27,312][20381] Updated weights for policy 1, policy_version 784192 (0.0009) [2023-12-27 11:03:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 401670144. Throughput: 0: 10222.5, 1: 10309.6. Samples: 401664548. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:27,363][20347] Updated weights for policy 0, policy_version 784665 (0.0010) [2023-12-27 11:03:27,363][19170] Avg episode reward: [(0, '883.540'), (1, '902.880')] [2023-12-27 11:03:27,366][20381] Updated weights for policy 1, policy_version 784202 (0.0008) [2023-12-27 11:03:27,422][20381] Updated weights for policy 1, policy_version 784212 (0.0010) [2023-12-27 11:03:27,961][20347] Updated weights for policy 0, policy_version 784675 (0.0009) [2023-12-27 11:03:28,018][20347] Updated weights for policy 0, policy_version 784685 (0.0005) [2023-12-27 11:03:28,067][20347] Updated weights for policy 0, policy_version 784695 (0.0006) [2023-12-27 11:03:28,109][20381] Updated weights for policy 1, policy_version 784222 (0.0007) [2023-12-27 11:03:28,163][20381] Updated weights for policy 1, policy_version 784232 (0.0008) [2023-12-27 11:03:28,222][20381] Updated weights for policy 1, policy_version 784242 (0.0006) [2023-12-27 11:03:28,661][20347] Updated weights for policy 0, policy_version 784705 (0.0010) [2023-12-27 11:03:28,722][20347] Updated weights for policy 0, policy_version 784715 (0.0007) [2023-12-27 11:03:28,777][20347] Updated weights for policy 0, policy_version 784725 (0.0005) [2023-12-27 11:03:28,845][20347] Updated weights for policy 0, policy_version 784735 (0.0007) [2023-12-27 11:03:28,845][20291] KL-divergence is very high: 180.4705 [2023-12-27 11:03:28,962][20381] Updated weights for policy 1, policy_version 784252 (0.0009) [2023-12-27 11:03:29,022][20381] Updated weights for policy 1, policy_version 784262 (0.0008) [2023-12-27 11:03:29,082][20381] Updated weights for policy 1, policy_version 784272 (0.0009) [2023-12-27 11:03:29,488][20347] Updated weights for policy 0, policy_version 784745 (0.0007) [2023-12-27 11:03:29,536][20347] Updated weights for policy 0, policy_version 784755 (0.0007) [2023-12-27 11:03:29,581][20347] Updated weights for policy 0, policy_version 784765 (0.0008) [2023-12-27 11:03:29,800][20381] Updated weights for policy 1, policy_version 784282 (0.0008) [2023-12-27 11:03:29,862][20381] Updated weights for policy 1, policy_version 784292 (0.0008) [2023-12-27 11:03:29,920][20381] Updated weights for policy 1, policy_version 784302 (0.0007) [2023-12-27 11:03:29,989][20381] Updated weights for policy 1, policy_version 784312 (0.0006) [2023-12-27 11:03:30,306][20347] Updated weights for policy 0, policy_version 784775 (0.0006) [2023-12-27 11:03:30,324][20291] KL-divergence is very high: 108.2680 [2023-12-27 11:03:30,364][20347] Updated weights for policy 0, policy_version 784785 (0.0008) [2023-12-27 11:03:30,410][20291] KL-divergence is very high: 104.1975 [2023-12-27 11:03:30,414][20347] Updated weights for policy 0, policy_version 784795 (0.0009) [2023-12-27 11:03:30,612][20381] Updated weights for policy 1, policy_version 784322 (0.0005) [2023-12-27 11:03:30,659][20381] Updated weights for policy 1, policy_version 784332 (0.0005) [2023-12-27 11:03:30,713][20381] Updated weights for policy 1, policy_version 784342 (0.0006) [2023-12-27 11:03:31,073][20347] Updated weights for policy 0, policy_version 784805 (0.0008) [2023-12-27 11:03:31,079][20291] KL-divergence is very high: 138.0725 [2023-12-27 11:03:31,121][20291] KL-divergence is very high: 110.5206 [2023-12-27 11:03:31,125][20347] Updated weights for policy 0, policy_version 784815 (0.0005) [2023-12-27 11:03:31,193][20347] Updated weights for policy 0, policy_version 784825 (0.0008) [2023-12-27 11:03:31,402][20381] Updated weights for policy 1, policy_version 784352 (0.0007) [2023-12-27 11:03:31,457][20381] Updated weights for policy 1, policy_version 784362 (0.0007) [2023-12-27 11:03:31,506][20381] Updated weights for policy 1, policy_version 784372 (0.0008) [2023-12-27 11:03:31,862][20291] KL-divergence is very high: 165.0490 [2023-12-27 11:03:31,869][20347] Updated weights for policy 0, policy_version 784835 (0.0007) [2023-12-27 11:03:31,920][20347] Updated weights for policy 0, policy_version 784845 (0.0008) [2023-12-27 11:03:31,967][20347] Updated weights for policy 0, policy_version 784855 (0.0008) [2023-12-27 11:03:32,201][20381] Updated weights for policy 1, policy_version 784382 (0.0008) [2023-12-27 11:03:32,255][20381] Updated weights for policy 1, policy_version 784392 (0.0008) [2023-12-27 11:03:32,306][20381] Updated weights for policy 1, policy_version 784402 (0.0007) [2023-12-27 11:03:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 401784832. Throughput: 0: 10299.3, 1: 10250.8. Samples: 401790772. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:32,363][19170] Avg episode reward: [(0, '912.360'), (1, '903.560')] [2023-12-27 11:03:32,654][20347] Updated weights for policy 0, policy_version 784865 (0.0008) [2023-12-27 11:03:32,716][20347] Updated weights for policy 0, policy_version 784875 (0.0008) [2023-12-27 11:03:32,772][20347] Updated weights for policy 0, policy_version 784885 (0.0009) [2023-12-27 11:03:32,822][20347] Updated weights for policy 0, policy_version 784895 (0.0007) [2023-12-27 11:03:32,948][20381] Updated weights for policy 1, policy_version 784412 (0.0007) [2023-12-27 11:03:33,004][20381] Updated weights for policy 1, policy_version 784422 (0.0005) [2023-12-27 11:03:33,065][20381] Updated weights for policy 1, policy_version 784432 (0.0006) [2023-12-27 11:03:33,415][20347] Updated weights for policy 0, policy_version 784905 (0.0007) [2023-12-27 11:03:33,473][20347] Updated weights for policy 0, policy_version 784915 (0.0008) [2023-12-27 11:03:33,535][20347] Updated weights for policy 0, policy_version 784925 (0.0008) [2023-12-27 11:03:33,729][20381] Updated weights for policy 1, policy_version 784442 (0.0008) [2023-12-27 11:03:33,783][20381] Updated weights for policy 1, policy_version 784452 (0.0007) [2023-12-27 11:03:33,835][20381] Updated weights for policy 1, policy_version 784462 (0.0007) [2023-12-27 11:03:33,891][20381] Updated weights for policy 1, policy_version 784472 (0.0007) [2023-12-27 11:03:34,166][20347] Updated weights for policy 0, policy_version 784935 (0.0010) [2023-12-27 11:03:34,222][20347] Updated weights for policy 0, policy_version 784945 (0.0010) [2023-12-27 11:03:34,276][20347] Updated weights for policy 0, policy_version 784955 (0.0010) [2023-12-27 11:03:34,615][20381] Updated weights for policy 1, policy_version 784482 (0.0008) [2023-12-27 11:03:34,678][20381] Updated weights for policy 1, policy_version 784492 (0.0008) [2023-12-27 11:03:34,735][20381] Updated weights for policy 1, policy_version 784502 (0.0007) [2023-12-27 11:03:34,963][20347] Updated weights for policy 0, policy_version 784965 (0.0010) [2023-12-27 11:03:35,015][20347] Updated weights for policy 0, policy_version 784975 (0.0010) [2023-12-27 11:03:35,057][20347] Updated weights for policy 0, policy_version 784985 (0.0010) [2023-12-27 11:03:35,409][20381] Updated weights for policy 1, policy_version 784512 (0.0007) [2023-12-27 11:03:35,470][20381] Updated weights for policy 1, policy_version 784522 (0.0007) [2023-12-27 11:03:35,530][20381] Updated weights for policy 1, policy_version 784532 (0.0007) [2023-12-27 11:03:35,751][20347] Updated weights for policy 0, policy_version 784995 (0.0010) [2023-12-27 11:03:35,804][20347] Updated weights for policy 0, policy_version 785005 (0.0009) [2023-12-27 11:03:35,857][20347] Updated weights for policy 0, policy_version 785015 (0.0010) [2023-12-27 11:03:36,229][20381] Updated weights for policy 1, policy_version 784542 (0.0007) [2023-12-27 11:03:36,291][20381] Updated weights for policy 1, policy_version 784552 (0.0009) [2023-12-27 11:03:36,354][20381] Updated weights for policy 1, policy_version 784562 (0.0008) [2023-12-27 11:03:36,583][20347] Updated weights for policy 0, policy_version 785025 (0.0010) [2023-12-27 11:03:36,644][20347] Updated weights for policy 0, policy_version 785035 (0.0007) [2023-12-27 11:03:36,693][20347] Updated weights for policy 0, policy_version 785045 (0.0007) [2023-12-27 11:03:36,750][20347] Updated weights for policy 0, policy_version 785055 (0.0006) [2023-12-27 11:03:37,086][20381] Updated weights for policy 1, policy_version 784572 (0.0009) [2023-12-27 11:03:37,138][20381] Updated weights for policy 1, policy_version 784582 (0.0008) [2023-12-27 11:03:37,192][20381] Updated weights for policy 1, policy_version 784592 (0.0008) [2023-12-27 11:03:37,333][20347] Updated weights for policy 0, policy_version 785065 (0.0008) [2023-12-27 11:03:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 401883136. Throughput: 0: 10335.3, 1: 10283.2. Samples: 401853540. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:37,363][19170] Avg episode reward: [(0, '892.320'), (1, '902.460')] [2023-12-27 11:03:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000784600_200884224.pth... [2023-12-27 11:03:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000783384_200572928.pth [2023-12-27 11:03:37,399][20347] Updated weights for policy 0, policy_version 785075 (0.0007) [2023-12-27 11:03:37,451][20347] Updated weights for policy 0, policy_version 785085 (0.0007) [2023-12-27 11:03:37,467][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000785088_201007104.pth... [2023-12-27 11:03:37,471][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000783840_200687616.pth [2023-12-27 11:03:37,928][20381] Updated weights for policy 1, policy_version 784602 (0.0008) [2023-12-27 11:03:37,977][20381] Updated weights for policy 1, policy_version 784612 (0.0009) [2023-12-27 11:03:38,022][20381] Updated weights for policy 1, policy_version 784622 (0.0009) [2023-12-27 11:03:38,041][20347] Updated weights for policy 0, policy_version 785095 (0.0005) [2023-12-27 11:03:38,073][20381] Updated weights for policy 1, policy_version 784632 (0.0006) [2023-12-27 11:03:38,089][20347] Updated weights for policy 0, policy_version 785105 (0.0005) [2023-12-27 11:03:38,136][20347] Updated weights for policy 0, policy_version 785115 (0.0005) [2023-12-27 11:03:38,744][20381] Updated weights for policy 1, policy_version 784642 (0.0008) [2023-12-27 11:03:38,795][20381] Updated weights for policy 1, policy_version 784652 (0.0008) [2023-12-27 11:03:38,797][20347] Updated weights for policy 0, policy_version 785125 (0.0006) [2023-12-27 11:03:38,851][20347] Updated weights for policy 0, policy_version 785135 (0.0005) [2023-12-27 11:03:38,853][20381] Updated weights for policy 1, policy_version 784662 (0.0007) [2023-12-27 11:03:38,911][20347] Updated weights for policy 0, policy_version 785145 (0.0007) [2023-12-27 11:03:39,562][20381] Updated weights for policy 1, policy_version 784672 (0.0007) [2023-12-27 11:03:39,625][20381] Updated weights for policy 1, policy_version 784682 (0.0008) [2023-12-27 11:03:39,653][20347] Updated weights for policy 0, policy_version 785155 (0.0007) [2023-12-27 11:03:39,684][20381] Updated weights for policy 1, policy_version 784692 (0.0008) [2023-12-27 11:03:39,707][20347] Updated weights for policy 0, policy_version 785165 (0.0006) [2023-12-27 11:03:39,766][20347] Updated weights for policy 0, policy_version 785175 (0.0007) [2023-12-27 11:03:40,429][20347] Updated weights for policy 0, policy_version 785185 (0.0007) [2023-12-27 11:03:40,457][20381] Updated weights for policy 1, policy_version 784702 (0.0007) [2023-12-27 11:03:40,489][20347] Updated weights for policy 0, policy_version 785195 (0.0005) [2023-12-27 11:03:40,517][20381] Updated weights for policy 1, policy_version 784712 (0.0006) [2023-12-27 11:03:40,545][20347] Updated weights for policy 0, policy_version 785205 (0.0006) [2023-12-27 11:03:40,570][20381] Updated weights for policy 1, policy_version 784722 (0.0006) [2023-12-27 11:03:40,602][20347] Updated weights for policy 0, policy_version 785215 (0.0006) [2023-12-27 11:03:41,215][20381] Updated weights for policy 1, policy_version 784732 (0.0006) [2023-12-27 11:03:41,281][20381] Updated weights for policy 1, policy_version 784742 (0.0008) [2023-12-27 11:03:41,300][20347] Updated weights for policy 0, policy_version 785225 (0.0008) [2023-12-27 11:03:41,340][20381] Updated weights for policy 1, policy_version 784752 (0.0008) [2023-12-27 11:03:41,363][20347] Updated weights for policy 0, policy_version 785235 (0.0008) [2023-12-27 11:03:41,427][20347] Updated weights for policy 0, policy_version 785245 (0.0006) [2023-12-27 11:03:42,030][20381] Updated weights for policy 1, policy_version 784762 (0.0009) [2023-12-27 11:03:42,081][20381] Updated weights for policy 1, policy_version 784772 (0.0008) [2023-12-27 11:03:42,130][20381] Updated weights for policy 1, policy_version 784782 (0.0007) [2023-12-27 11:03:42,181][20381] Updated weights for policy 1, policy_version 784792 (0.0007) [2023-12-27 11:03:42,187][20347] Updated weights for policy 0, policy_version 785255 (0.0007) [2023-12-27 11:03:42,239][20347] Updated weights for policy 0, policy_version 785265 (0.0008) [2023-12-27 11:03:42,291][20347] Updated weights for policy 0, policy_version 785275 (0.0007) [2023-12-27 11:03:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 401989632. Throughput: 0: 10349.9, 1: 10205.9. Samples: 401975872. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:42,363][19170] Avg episode reward: [(0, '873.010'), (1, '864.630')] [2023-12-27 11:03:42,946][20381] Updated weights for policy 1, policy_version 784802 (0.0006) [2023-12-27 11:03:42,997][20381] Updated weights for policy 1, policy_version 784812 (0.0005) [2023-12-27 11:03:43,040][20347] Updated weights for policy 0, policy_version 785285 (0.0006) [2023-12-27 11:03:43,053][20381] Updated weights for policy 1, policy_version 784822 (0.0007) [2023-12-27 11:03:43,101][20347] Updated weights for policy 0, policy_version 785295 (0.0006) [2023-12-27 11:03:43,165][20347] Updated weights for policy 0, policy_version 785305 (0.0007) [2023-12-27 11:03:43,610][20381] Updated weights for policy 1, policy_version 784832 (0.0006) [2023-12-27 11:03:43,664][20381] Updated weights for policy 1, policy_version 784842 (0.0005) [2023-12-27 11:03:43,717][20381] Updated weights for policy 1, policy_version 784852 (0.0005) [2023-12-27 11:03:43,898][20347] Updated weights for policy 0, policy_version 785315 (0.0008) [2023-12-27 11:03:43,957][20347] Updated weights for policy 0, policy_version 785325 (0.0008) [2023-12-27 11:03:44,012][20347] Updated weights for policy 0, policy_version 785335 (0.0007) [2023-12-27 11:03:44,300][20381] Updated weights for policy 1, policy_version 784862 (0.0006) [2023-12-27 11:03:44,360][20381] Updated weights for policy 1, policy_version 784872 (0.0006) [2023-12-27 11:03:44,422][20381] Updated weights for policy 1, policy_version 784882 (0.0008) [2023-12-27 11:03:44,721][20347] Updated weights for policy 0, policy_version 785345 (0.0008) [2023-12-27 11:03:44,783][20347] Updated weights for policy 0, policy_version 785355 (0.0008) [2023-12-27 11:03:44,796][20291] KL-divergence is very high: 327.5199 [2023-12-27 11:03:44,835][20347] Updated weights for policy 0, policy_version 785365 (0.0007) [2023-12-27 11:03:44,893][20347] Updated weights for policy 0, policy_version 785375 (0.0009) [2023-12-27 11:03:45,095][20381] Updated weights for policy 1, policy_version 784892 (0.0007) [2023-12-27 11:03:45,161][20381] Updated weights for policy 1, policy_version 784902 (0.0007) [2023-12-27 11:03:45,220][20381] Updated weights for policy 1, policy_version 784912 (0.0007) [2023-12-27 11:03:45,534][20347] Updated weights for policy 0, policy_version 785385 (0.0006) [2023-12-27 11:03:45,587][20347] Updated weights for policy 0, policy_version 785395 (0.0006) [2023-12-27 11:03:45,647][20347] Updated weights for policy 0, policy_version 785405 (0.0009) [2023-12-27 11:03:45,934][20381] Updated weights for policy 1, policy_version 784922 (0.0009) [2023-12-27 11:03:46,003][20381] Updated weights for policy 1, policy_version 784932 (0.0006) [2023-12-27 11:03:46,067][20381] Updated weights for policy 1, policy_version 784942 (0.0007) [2023-12-27 11:03:46,142][20381] Updated weights for policy 1, policy_version 784952 (0.0006) [2023-12-27 11:03:46,236][20347] Updated weights for policy 0, policy_version 785415 (0.0007) [2023-12-27 11:03:46,282][20347] Updated weights for policy 0, policy_version 785425 (0.0005) [2023-12-27 11:03:46,336][20347] Updated weights for policy 0, policy_version 785435 (0.0005) [2023-12-27 11:03:46,820][20381] Updated weights for policy 1, policy_version 784962 (0.0007) [2023-12-27 11:03:46,873][20381] Updated weights for policy 1, policy_version 784972 (0.0008) [2023-12-27 11:03:46,918][20347] Updated weights for policy 0, policy_version 785445 (0.0006) [2023-12-27 11:03:46,931][20381] Updated weights for policy 1, policy_version 784982 (0.0008) [2023-12-27 11:03:46,972][20347] Updated weights for policy 0, policy_version 785455 (0.0007) [2023-12-27 11:03:47,023][20347] Updated weights for policy 0, policy_version 785465 (0.0007) [2023-12-27 11:03:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 402087936. Throughput: 0: 10389.9, 1: 10213.1. Samples: 402097492. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:47,364][19170] Avg episode reward: [(0, '892.810'), (1, '904.500')] [2023-12-27 11:03:47,639][20381] Updated weights for policy 1, policy_version 784992 (0.0007) [2023-12-27 11:03:47,681][20347] Updated weights for policy 0, policy_version 785475 (0.0007) [2023-12-27 11:03:47,687][20381] Updated weights for policy 1, policy_version 785002 (0.0008) [2023-12-27 11:03:47,744][20347] Updated weights for policy 0, policy_version 785485 (0.0007) [2023-12-27 11:03:47,754][20381] Updated weights for policy 1, policy_version 785012 (0.0008) [2023-12-27 11:03:47,809][20347] Updated weights for policy 0, policy_version 785495 (0.0006) [2023-12-27 11:03:48,403][20347] Updated weights for policy 0, policy_version 785505 (0.0006) [2023-12-27 11:03:48,462][20347] Updated weights for policy 0, policy_version 785515 (0.0007) [2023-12-27 11:03:48,500][20381] Updated weights for policy 1, policy_version 785022 (0.0008) [2023-12-27 11:03:48,522][20347] Updated weights for policy 0, policy_version 785525 (0.0006) [2023-12-27 11:03:48,548][20381] Updated weights for policy 1, policy_version 785032 (0.0007) [2023-12-27 11:03:48,577][20347] Updated weights for policy 0, policy_version 785535 (0.0007) [2023-12-27 11:03:48,602][20381] Updated weights for policy 1, policy_version 785042 (0.0007) [2023-12-27 11:03:49,235][20347] Updated weights for policy 0, policy_version 785545 (0.0008) [2023-12-27 11:03:49,285][20381] Updated weights for policy 1, policy_version 785052 (0.0008) [2023-12-27 11:03:49,292][20347] Updated weights for policy 0, policy_version 785555 (0.0006) [2023-12-27 11:03:49,348][20381] Updated weights for policy 1, policy_version 785062 (0.0009) [2023-12-27 11:03:49,355][20347] Updated weights for policy 0, policy_version 785565 (0.0007) [2023-12-27 11:03:49,407][20381] Updated weights for policy 1, policy_version 785072 (0.0010) [2023-12-27 11:03:49,983][20347] Updated weights for policy 0, policy_version 785575 (0.0008) [2023-12-27 11:03:50,038][20347] Updated weights for policy 0, policy_version 785585 (0.0008) [2023-12-27 11:03:50,096][20347] Updated weights for policy 0, policy_version 785595 (0.0008) [2023-12-27 11:03:50,126][20381] Updated weights for policy 1, policy_version 785082 (0.0009) [2023-12-27 11:03:50,181][20381] Updated weights for policy 1, policy_version 785092 (0.0005) [2023-12-27 11:03:50,231][20381] Updated weights for policy 1, policy_version 785102 (0.0006) [2023-12-27 11:03:50,289][20381] Updated weights for policy 1, policy_version 785112 (0.0006) [2023-12-27 11:03:50,834][20381] Updated weights for policy 1, policy_version 785122 (0.0005) [2023-12-27 11:03:50,841][20347] Updated weights for policy 0, policy_version 785605 (0.0009) [2023-12-27 11:03:50,889][20381] Updated weights for policy 1, policy_version 785132 (0.0006) [2023-12-27 11:03:50,890][20347] Updated weights for policy 0, policy_version 785615 (0.0010) [2023-12-27 11:03:50,944][20347] Updated weights for policy 0, policy_version 785625 (0.0009) [2023-12-27 11:03:50,946][20381] Updated weights for policy 1, policy_version 785142 (0.0006) [2023-12-27 11:03:51,677][20381] Updated weights for policy 1, policy_version 785152 (0.0007) [2023-12-27 11:03:51,686][20347] Updated weights for policy 0, policy_version 785635 (0.0010) [2023-12-27 11:03:51,730][20381] Updated weights for policy 1, policy_version 785162 (0.0007) [2023-12-27 11:03:51,746][20347] Updated weights for policy 0, policy_version 785645 (0.0008) [2023-12-27 11:03:51,794][20381] Updated weights for policy 1, policy_version 785172 (0.0007) [2023-12-27 11:03:51,808][20347] Updated weights for policy 0, policy_version 785655 (0.0006) [2023-12-27 11:03:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 402186240. Throughput: 0: 10406.4, 1: 10173.6. Samples: 402158644. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:52,363][19170] Avg episode reward: [(0, '902.780'), (1, '923.100')] [2023-12-27 11:03:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000785176_201031680.pth... [2023-12-27 11:03:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000785664_201154560.pth... [2023-12-27 11:03:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000784448_200843264.pth [2023-12-27 11:03:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000783992_200728576.pth [2023-12-27 11:03:52,514][20347] Updated weights for policy 0, policy_version 785665 (0.0008) [2023-12-27 11:03:52,549][20381] Updated weights for policy 1, policy_version 785182 (0.0006) [2023-12-27 11:03:52,570][20347] Updated weights for policy 0, policy_version 785675 (0.0006) [2023-12-27 11:03:52,609][20381] Updated weights for policy 1, policy_version 785192 (0.0006) [2023-12-27 11:03:52,618][20347] Updated weights for policy 0, policy_version 785685 (0.0006) [2023-12-27 11:03:52,666][20381] Updated weights for policy 1, policy_version 785202 (0.0005) [2023-12-27 11:03:52,668][20347] Updated weights for policy 0, policy_version 785695 (0.0006) [2023-12-27 11:03:53,301][20381] Updated weights for policy 1, policy_version 785212 (0.0007) [2023-12-27 11:03:53,347][20381] Updated weights for policy 1, policy_version 785222 (0.0006) [2023-12-27 11:03:53,348][20347] Updated weights for policy 0, policy_version 785705 (0.0010) [2023-12-27 11:03:53,390][20381] Updated weights for policy 1, policy_version 785232 (0.0007) [2023-12-27 11:03:53,409][20347] Updated weights for policy 0, policy_version 785715 (0.0010) [2023-12-27 11:03:53,470][20347] Updated weights for policy 0, policy_version 785725 (0.0010) [2023-12-27 11:03:54,062][20381] Updated weights for policy 1, policy_version 785242 (0.0008) [2023-12-27 11:03:54,115][20347] Updated weights for policy 0, policy_version 785735 (0.0007) [2023-12-27 11:03:54,124][20381] Updated weights for policy 1, policy_version 785252 (0.0006) [2023-12-27 11:03:54,174][20347] Updated weights for policy 0, policy_version 785745 (0.0007) [2023-12-27 11:03:54,180][20381] Updated weights for policy 1, policy_version 785262 (0.0007) [2023-12-27 11:03:54,227][20347] Updated weights for policy 0, policy_version 785755 (0.0006) [2023-12-27 11:03:54,236][20381] Updated weights for policy 1, policy_version 785272 (0.0007) [2023-12-27 11:03:54,870][20347] Updated weights for policy 0, policy_version 785765 (0.0007) [2023-12-27 11:03:54,920][20347] Updated weights for policy 0, policy_version 785775 (0.0005) [2023-12-27 11:03:54,952][20381] Updated weights for policy 1, policy_version 785282 (0.0007) [2023-12-27 11:03:54,972][20347] Updated weights for policy 0, policy_version 785785 (0.0006) [2023-12-27 11:03:55,007][20381] Updated weights for policy 1, policy_version 785292 (0.0007) [2023-12-27 11:03:55,056][20381] Updated weights for policy 1, policy_version 785302 (0.0007) [2023-12-27 11:03:55,543][20347] Updated weights for policy 0, policy_version 785795 (0.0006) [2023-12-27 11:03:55,603][20347] Updated weights for policy 0, policy_version 785805 (0.0007) [2023-12-27 11:03:55,661][20347] Updated weights for policy 0, policy_version 785815 (0.0007) [2023-12-27 11:03:55,802][20381] Updated weights for policy 1, policy_version 785312 (0.0008) [2023-12-27 11:03:55,852][20381] Updated weights for policy 1, policy_version 785322 (0.0007) [2023-12-27 11:03:55,898][20381] Updated weights for policy 1, policy_version 785332 (0.0008) [2023-12-27 11:03:56,327][20347] Updated weights for policy 0, policy_version 785825 (0.0008) [2023-12-27 11:03:56,377][20347] Updated weights for policy 0, policy_version 785835 (0.0007) [2023-12-27 11:03:56,422][20347] Updated weights for policy 0, policy_version 785845 (0.0007) [2023-12-27 11:03:56,473][20347] Updated weights for policy 0, policy_version 785855 (0.0008) [2023-12-27 11:03:56,577][20381] Updated weights for policy 1, policy_version 785342 (0.0006) [2023-12-27 11:03:56,630][20381] Updated weights for policy 1, policy_version 785352 (0.0005) [2023-12-27 11:03:56,677][20381] Updated weights for policy 1, policy_version 785362 (0.0005) [2023-12-27 11:03:57,080][20347] Updated weights for policy 0, policy_version 785865 (0.0007) [2023-12-27 11:03:57,140][20347] Updated weights for policy 0, policy_version 785875 (0.0008) [2023-12-27 11:03:57,196][20347] Updated weights for policy 0, policy_version 785885 (0.0008) [2023-12-27 11:03:57,236][20381] Updated weights for policy 1, policy_version 785372 (0.0006) [2023-12-27 11:03:57,281][20381] Updated weights for policy 1, policy_version 785382 (0.0005) [2023-12-27 11:03:57,339][20381] Updated weights for policy 1, policy_version 785392 (0.0007) [2023-12-27 11:03:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 402292736. Throughput: 0: 10428.2, 1: 10186.9. Samples: 402281496. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:03:57,363][19170] Avg episode reward: [(0, '883.320'), (1, '931.870')] [2023-12-27 11:03:57,923][20381] Updated weights for policy 1, policy_version 785402 (0.0008) [2023-12-27 11:03:57,927][20347] Updated weights for policy 0, policy_version 785895 (0.0007) [2023-12-27 11:03:57,963][20292] KL-divergence is very high: 113.5457 [2023-12-27 11:03:57,967][20381] Updated weights for policy 1, policy_version 785412 (0.0006) [2023-12-27 11:03:57,981][20347] Updated weights for policy 0, policy_version 785905 (0.0007) [2023-12-27 11:03:57,999][20292] KL-divergence is very high: 106.0843 [2023-12-27 11:03:58,013][20381] Updated weights for policy 1, policy_version 785422 (0.0005) [2023-12-27 11:03:58,042][20347] Updated weights for policy 0, policy_version 785915 (0.0007) [2023-12-27 11:03:58,072][20381] Updated weights for policy 1, policy_version 785432 (0.0009) [2023-12-27 11:03:58,754][20381] Updated weights for policy 1, policy_version 785442 (0.0010) [2023-12-27 11:03:58,782][20347] Updated weights for policy 0, policy_version 785925 (0.0005) [2023-12-27 11:03:58,814][20381] Updated weights for policy 1, policy_version 785452 (0.0009) [2023-12-27 11:03:58,847][20347] Updated weights for policy 0, policy_version 785935 (0.0006) [2023-12-27 11:03:58,877][20381] Updated weights for policy 1, policy_version 785462 (0.0008) [2023-12-27 11:03:58,909][20347] Updated weights for policy 0, policy_version 785945 (0.0007) [2023-12-27 11:03:59,530][20381] Updated weights for policy 1, policy_version 785472 (0.0009) [2023-12-27 11:03:59,586][20347] Updated weights for policy 0, policy_version 785955 (0.0006) [2023-12-27 11:03:59,595][20381] Updated weights for policy 1, policy_version 785482 (0.0010) [2023-12-27 11:03:59,646][20347] Updated weights for policy 0, policy_version 785965 (0.0006) [2023-12-27 11:03:59,651][20381] Updated weights for policy 1, policy_version 785492 (0.0011) [2023-12-27 11:03:59,705][20347] Updated weights for policy 0, policy_version 785975 (0.0007) [2023-12-27 11:04:00,337][20381] Updated weights for policy 1, policy_version 785502 (0.0009) [2023-12-27 11:04:00,401][20347] Updated weights for policy 0, policy_version 785985 (0.0009) [2023-12-27 11:04:00,404][20381] Updated weights for policy 1, policy_version 785512 (0.0007) [2023-12-27 11:04:00,455][20347] Updated weights for policy 0, policy_version 785995 (0.0010) [2023-12-27 11:04:00,456][20381] Updated weights for policy 1, policy_version 785522 (0.0007) [2023-12-27 11:04:00,509][20347] Updated weights for policy 0, policy_version 786005 (0.0010) [2023-12-27 11:04:00,565][20347] Updated weights for policy 0, policy_version 786015 (0.0010) [2023-12-27 11:04:01,154][20381] Updated weights for policy 1, policy_version 785532 (0.0006) [2023-12-27 11:04:01,218][20381] Updated weights for policy 1, policy_version 785542 (0.0007) [2023-12-27 11:04:01,258][20347] Updated weights for policy 0, policy_version 786025 (0.0008) [2023-12-27 11:04:01,283][20381] Updated weights for policy 1, policy_version 785552 (0.0006) [2023-12-27 11:04:01,329][20347] Updated weights for policy 0, policy_version 786035 (0.0007) [2023-12-27 11:04:01,396][20347] Updated weights for policy 0, policy_version 786045 (0.0008) [2023-12-27 11:04:02,003][20381] Updated weights for policy 1, policy_version 785562 (0.0008) [2023-12-27 11:04:02,056][20347] Updated weights for policy 0, policy_version 786055 (0.0005) [2023-12-27 11:04:02,057][20381] Updated weights for policy 1, policy_version 785572 (0.0008) [2023-12-27 11:04:02,105][20347] Updated weights for policy 0, policy_version 786065 (0.0006) [2023-12-27 11:04:02,116][20381] Updated weights for policy 1, policy_version 785582 (0.0008) [2023-12-27 11:04:02,152][20347] Updated weights for policy 0, policy_version 786075 (0.0006) [2023-12-27 11:04:02,173][20381] Updated weights for policy 1, policy_version 785592 (0.0007) [2023-12-27 11:04:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 402399232. Throughput: 0: 10355.7, 1: 10222.9. Samples: 402404784. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:02,363][19170] Avg episode reward: [(0, '611.690'), (1, '970.690')] [2023-12-27 11:04:02,777][20381] Updated weights for policy 1, policy_version 785602 (0.0008) [2023-12-27 11:04:02,841][20381] Updated weights for policy 1, policy_version 785612 (0.0007) [2023-12-27 11:04:02,905][20381] Updated weights for policy 1, policy_version 785622 (0.0007) [2023-12-27 11:04:02,939][20347] Updated weights for policy 0, policy_version 786085 (0.0008) [2023-12-27 11:04:02,999][20347] Updated weights for policy 0, policy_version 786095 (0.0007) [2023-12-27 11:04:03,063][20347] Updated weights for policy 0, policy_version 786105 (0.0008) [2023-12-27 11:04:03,557][20381] Updated weights for policy 1, policy_version 785632 (0.0007) [2023-12-27 11:04:03,613][20381] Updated weights for policy 1, policy_version 785642 (0.0007) [2023-12-27 11:04:03,663][20381] Updated weights for policy 1, policy_version 785652 (0.0007) [2023-12-27 11:04:03,677][20347] Updated weights for policy 0, policy_version 786115 (0.0008) [2023-12-27 11:04:03,735][20347] Updated weights for policy 0, policy_version 786125 (0.0007) [2023-12-27 11:04:03,790][20347] Updated weights for policy 0, policy_version 786135 (0.0006) [2023-12-27 11:04:04,333][20381] Updated weights for policy 1, policy_version 785662 (0.0006) [2023-12-27 11:04:04,390][20381] Updated weights for policy 1, policy_version 785672 (0.0007) [2023-12-27 11:04:04,431][20347] Updated weights for policy 0, policy_version 786145 (0.0009) [2023-12-27 11:04:04,445][20381] Updated weights for policy 1, policy_version 785682 (0.0008) [2023-12-27 11:04:04,485][20347] Updated weights for policy 0, policy_version 786155 (0.0008) [2023-12-27 11:04:04,542][20347] Updated weights for policy 0, policy_version 786165 (0.0008) [2023-12-27 11:04:04,604][20347] Updated weights for policy 0, policy_version 786175 (0.0008) [2023-12-27 11:04:05,016][20381] Updated weights for policy 1, policy_version 785692 (0.0006) [2023-12-27 11:04:05,070][20381] Updated weights for policy 1, policy_version 785702 (0.0007) [2023-12-27 11:04:05,118][20381] Updated weights for policy 1, policy_version 785712 (0.0007) [2023-12-27 11:04:05,305][20347] Updated weights for policy 0, policy_version 786185 (0.0008) [2023-12-27 11:04:05,364][20347] Updated weights for policy 0, policy_version 786195 (0.0007) [2023-12-27 11:04:05,415][20347] Updated weights for policy 0, policy_version 786205 (0.0005) [2023-12-27 11:04:05,762][20381] Updated weights for policy 1, policy_version 785722 (0.0007) [2023-12-27 11:04:05,831][20381] Updated weights for policy 1, policy_version 785732 (0.0006) [2023-12-27 11:04:05,885][20381] Updated weights for policy 1, policy_version 785742 (0.0005) [2023-12-27 11:04:05,935][20381] Updated weights for policy 1, policy_version 785752 (0.0006) [2023-12-27 11:04:06,018][20347] Updated weights for policy 0, policy_version 786215 (0.0005) [2023-12-27 11:04:06,077][20347] Updated weights for policy 0, policy_version 786225 (0.0006) [2023-12-27 11:04:06,138][20347] Updated weights for policy 0, policy_version 786235 (0.0007) [2023-12-27 11:04:06,609][20381] Updated weights for policy 1, policy_version 785762 (0.0007) [2023-12-27 11:04:06,671][20381] Updated weights for policy 1, policy_version 785772 (0.0008) [2023-12-27 11:04:06,728][20381] Updated weights for policy 1, policy_version 785782 (0.0007) [2023-12-27 11:04:06,775][20347] Updated weights for policy 0, policy_version 786245 (0.0009) [2023-12-27 11:04:06,827][20347] Updated weights for policy 0, policy_version 786255 (0.0007) [2023-12-27 11:04:06,882][20347] Updated weights for policy 0, policy_version 786265 (0.0007) [2023-12-27 11:04:07,356][20381] Updated weights for policy 1, policy_version 785792 (0.0008) [2023-12-27 11:04:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 402497536. Throughput: 0: 10369.7, 1: 10241.3. Samples: 402468036. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:07,363][19170] Avg episode reward: [(0, '632.320'), (1, '990.320')] [2023-12-27 11:04:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000786272_201310208.pth... [2023-12-27 11:04:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000785088_201007104.pth [2023-12-27 11:04:07,408][20381] Updated weights for policy 1, policy_version 785802 (0.0008) [2023-12-27 11:04:07,463][20381] Updated weights for policy 1, policy_version 785812 (0.0007) [2023-12-27 11:04:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000785816_201195520.pth... [2023-12-27 11:04:07,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000784600_200884224.pth [2023-12-27 11:04:07,607][20347] Updated weights for policy 0, policy_version 786275 (0.0007) [2023-12-27 11:04:07,661][20347] Updated weights for policy 0, policy_version 786285 (0.0007) [2023-12-27 11:04:07,717][20347] Updated weights for policy 0, policy_version 786295 (0.0008) [2023-12-27 11:04:08,088][20381] Updated weights for policy 1, policy_version 785822 (0.0005) [2023-12-27 11:04:08,150][20381] Updated weights for policy 1, policy_version 785832 (0.0005) [2023-12-27 11:04:08,210][20381] Updated weights for policy 1, policy_version 785842 (0.0006) [2023-12-27 11:04:08,420][20347] Updated weights for policy 0, policy_version 786305 (0.0008) [2023-12-27 11:04:08,480][20347] Updated weights for policy 0, policy_version 786315 (0.0006) [2023-12-27 11:04:08,539][20347] Updated weights for policy 0, policy_version 786325 (0.0006) [2023-12-27 11:04:08,601][20347] Updated weights for policy 0, policy_version 786335 (0.0006) [2023-12-27 11:04:08,861][20381] Updated weights for policy 1, policy_version 785852 (0.0007) [2023-12-27 11:04:08,915][20381] Updated weights for policy 1, policy_version 785862 (0.0008) [2023-12-27 11:04:08,982][20381] Updated weights for policy 1, policy_version 785872 (0.0008) [2023-12-27 11:04:09,212][20347] Updated weights for policy 0, policy_version 786345 (0.0008) [2023-12-27 11:04:09,272][20347] Updated weights for policy 0, policy_version 786355 (0.0008) [2023-12-27 11:04:09,331][20347] Updated weights for policy 0, policy_version 786365 (0.0008) [2023-12-27 11:04:09,717][20381] Updated weights for policy 1, policy_version 785882 (0.0008) [2023-12-27 11:04:09,779][20381] Updated weights for policy 1, policy_version 785892 (0.0006) [2023-12-27 11:04:09,853][20381] Updated weights for policy 1, policy_version 785902 (0.0008) [2023-12-27 11:04:09,908][20381] Updated weights for policy 1, policy_version 785912 (0.0008) [2023-12-27 11:04:10,110][20347] Updated weights for policy 0, policy_version 786375 (0.0008) [2023-12-27 11:04:10,177][20347] Updated weights for policy 0, policy_version 786385 (0.0006) [2023-12-27 11:04:10,231][20347] Updated weights for policy 0, policy_version 786395 (0.0008) [2023-12-27 11:04:10,494][20381] Updated weights for policy 1, policy_version 785922 (0.0006) [2023-12-27 11:04:10,554][20381] Updated weights for policy 1, policy_version 785932 (0.0006) [2023-12-27 11:04:10,602][20381] Updated weights for policy 1, policy_version 785942 (0.0005) [2023-12-27 11:04:10,824][20347] Updated weights for policy 0, policy_version 786405 (0.0007) [2023-12-27 11:04:10,872][20347] Updated weights for policy 0, policy_version 786415 (0.0007) [2023-12-27 11:04:10,927][20347] Updated weights for policy 0, policy_version 786425 (0.0008) [2023-12-27 11:04:11,216][20381] Updated weights for policy 1, policy_version 785952 (0.0007) [2023-12-27 11:04:11,274][20381] Updated weights for policy 1, policy_version 785962 (0.0008) [2023-12-27 11:04:11,335][20381] Updated weights for policy 1, policy_version 785972 (0.0008) [2023-12-27 11:04:11,659][20347] Updated weights for policy 0, policy_version 786435 (0.0008) [2023-12-27 11:04:11,726][20347] Updated weights for policy 0, policy_version 786445 (0.0008) [2023-12-27 11:04:11,782][20347] Updated weights for policy 0, policy_version 786455 (0.0007) [2023-12-27 11:04:12,059][20381] Updated weights for policy 1, policy_version 785982 (0.0007) [2023-12-27 11:04:12,120][20381] Updated weights for policy 1, policy_version 785992 (0.0006) [2023-12-27 11:04:12,174][20381] Updated weights for policy 1, policy_version 786002 (0.0008) [2023-12-27 11:04:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 402604032. Throughput: 0: 10366.4, 1: 10275.6. Samples: 402593440. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:12,363][19170] Avg episode reward: [(0, '577.720'), (1, '990.260')] [2023-12-27 11:04:12,502][20347] Updated weights for policy 0, policy_version 786465 (0.0008) [2023-12-27 11:04:12,561][20347] Updated weights for policy 0, policy_version 786475 (0.0009) [2023-12-27 11:04:12,623][20347] Updated weights for policy 0, policy_version 786485 (0.0011) [2023-12-27 11:04:12,679][20347] Updated weights for policy 0, policy_version 786495 (0.0011) [2023-12-27 11:04:12,847][20381] Updated weights for policy 1, policy_version 786012 (0.0007) [2023-12-27 11:04:12,902][20381] Updated weights for policy 1, policy_version 786022 (0.0010) [2023-12-27 11:04:12,960][20381] Updated weights for policy 1, policy_version 786032 (0.0007) [2023-12-27 11:04:13,327][20347] Updated weights for policy 0, policy_version 786505 (0.0010) [2023-12-27 11:04:13,385][20347] Updated weights for policy 0, policy_version 786515 (0.0010) [2023-12-27 11:04:13,443][20347] Updated weights for policy 0, policy_version 786525 (0.0010) [2023-12-27 11:04:13,543][20381] Updated weights for policy 1, policy_version 786042 (0.0005) [2023-12-27 11:04:13,594][20381] Updated weights for policy 1, policy_version 786052 (0.0005) [2023-12-27 11:04:13,639][20381] Updated weights for policy 1, policy_version 786062 (0.0005) [2023-12-27 11:04:13,688][20381] Updated weights for policy 1, policy_version 786072 (0.0006) [2023-12-27 11:04:14,073][20347] Updated weights for policy 0, policy_version 786535 (0.0010) [2023-12-27 11:04:14,135][20347] Updated weights for policy 0, policy_version 786545 (0.0006) [2023-12-27 11:04:14,196][20347] Updated weights for policy 0, policy_version 786555 (0.0006) [2023-12-27 11:04:14,247][20381] Updated weights for policy 1, policy_version 786082 (0.0006) [2023-12-27 11:04:14,295][20381] Updated weights for policy 1, policy_version 786092 (0.0007) [2023-12-27 11:04:14,343][20381] Updated weights for policy 1, policy_version 786102 (0.0006) [2023-12-27 11:04:14,822][20347] Updated weights for policy 0, policy_version 786565 (0.0008) [2023-12-27 11:04:14,880][20347] Updated weights for policy 0, policy_version 786575 (0.0010) [2023-12-27 11:04:14,944][20347] Updated weights for policy 0, policy_version 786585 (0.0011) [2023-12-27 11:04:15,035][20381] Updated weights for policy 1, policy_version 786112 (0.0007) [2023-12-27 11:04:15,095][20381] Updated weights for policy 1, policy_version 786122 (0.0007) [2023-12-27 11:04:15,159][20381] Updated weights for policy 1, policy_version 786132 (0.0008) [2023-12-27 11:04:15,616][20347] Updated weights for policy 0, policy_version 786595 (0.0010) [2023-12-27 11:04:15,670][20347] Updated weights for policy 0, policy_version 786605 (0.0010) [2023-12-27 11:04:15,728][20347] Updated weights for policy 0, policy_version 786615 (0.0008) [2023-12-27 11:04:15,812][20381] Updated weights for policy 1, policy_version 786142 (0.0008) [2023-12-27 11:04:15,858][20381] Updated weights for policy 1, policy_version 786152 (0.0008) [2023-12-27 11:04:15,904][20381] Updated weights for policy 1, policy_version 786162 (0.0010) [2023-12-27 11:04:16,355][20347] Updated weights for policy 0, policy_version 786625 (0.0007) [2023-12-27 11:04:16,411][20347] Updated weights for policy 0, policy_version 786635 (0.0007) [2023-12-27 11:04:16,418][20291] KL-divergence is very high: 364.3165 [2023-12-27 11:04:16,462][20291] KL-divergence is very high: 163.6328 [2023-12-27 11:04:16,469][20347] Updated weights for policy 0, policy_version 786645 (0.0008) [2023-12-27 11:04:16,521][20347] Updated weights for policy 0, policy_version 786655 (0.0008) [2023-12-27 11:04:16,634][20381] Updated weights for policy 1, policy_version 786172 (0.0008) [2023-12-27 11:04:16,690][20381] Updated weights for policy 1, policy_version 786182 (0.0007) [2023-12-27 11:04:16,746][20381] Updated weights for policy 1, policy_version 786192 (0.0008) [2023-12-27 11:04:17,209][20347] Updated weights for policy 0, policy_version 786665 (0.0008) [2023-12-27 11:04:17,267][20347] Updated weights for policy 0, policy_version 786675 (0.0008) [2023-12-27 11:04:17,324][20347] Updated weights for policy 0, policy_version 786685 (0.0008) [2023-12-27 11:04:17,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 402710528. Throughput: 0: 10310.5, 1: 10298.0. Samples: 402718156. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:17,363][19170] Avg episode reward: [(0, '270.150'), (1, '980.590')] [2023-12-27 11:04:17,446][20381] Updated weights for policy 1, policy_version 786202 (0.0008) [2023-12-27 11:04:17,505][20381] Updated weights for policy 1, policy_version 786212 (0.0008) [2023-12-27 11:04:17,564][20381] Updated weights for policy 1, policy_version 786222 (0.0008) [2023-12-27 11:04:17,626][20381] Updated weights for policy 1, policy_version 786232 (0.0007) [2023-12-27 11:04:17,976][20347] Updated weights for policy 0, policy_version 786695 (0.0008) [2023-12-27 11:04:18,037][20347] Updated weights for policy 0, policy_version 786705 (0.0006) [2023-12-27 11:04:18,098][20347] Updated weights for policy 0, policy_version 786715 (0.0005) [2023-12-27 11:04:18,367][20381] Updated weights for policy 1, policy_version 786242 (0.0008) [2023-12-27 11:04:18,424][20381] Updated weights for policy 1, policy_version 786252 (0.0007) [2023-12-27 11:04:18,485][20381] Updated weights for policy 1, policy_version 786262 (0.0008) [2023-12-27 11:04:18,725][20347] Updated weights for policy 0, policy_version 786725 (0.0007) [2023-12-27 11:04:18,780][20347] Updated weights for policy 0, policy_version 786735 (0.0007) [2023-12-27 11:04:18,835][20347] Updated weights for policy 0, policy_version 786745 (0.0008) [2023-12-27 11:04:19,172][20381] Updated weights for policy 1, policy_version 786272 (0.0007) [2023-12-27 11:04:19,223][20381] Updated weights for policy 1, policy_version 786282 (0.0008) [2023-12-27 11:04:19,288][20381] Updated weights for policy 1, policy_version 786292 (0.0007) [2023-12-27 11:04:19,567][20347] Updated weights for policy 0, policy_version 786755 (0.0008) [2023-12-27 11:04:19,626][20347] Updated weights for policy 0, policy_version 786765 (0.0008) [2023-12-27 11:04:19,680][20347] Updated weights for policy 0, policy_version 786775 (0.0007) [2023-12-27 11:04:20,011][20381] Updated weights for policy 1, policy_version 786302 (0.0006) [2023-12-27 11:04:20,076][20381] Updated weights for policy 1, policy_version 786312 (0.0006) [2023-12-27 11:04:20,137][20381] Updated weights for policy 1, policy_version 786322 (0.0007) [2023-12-27 11:04:20,387][20347] Updated weights for policy 0, policy_version 786785 (0.0008) [2023-12-27 11:04:20,451][20347] Updated weights for policy 0, policy_version 786795 (0.0008) [2023-12-27 11:04:20,500][20347] Updated weights for policy 0, policy_version 786805 (0.0008) [2023-12-27 11:04:20,546][20347] Updated weights for policy 0, policy_version 786815 (0.0008) [2023-12-27 11:04:20,855][20381] Updated weights for policy 1, policy_version 786332 (0.0008) [2023-12-27 11:04:20,891][20292] KL-divergence is very high: 182.9904 [2023-12-27 11:04:20,915][20381] Updated weights for policy 1, policy_version 786342 (0.0008) [2023-12-27 11:04:20,969][20381] Updated weights for policy 1, policy_version 786352 (0.0008) [2023-12-27 11:04:21,290][20347] Updated weights for policy 0, policy_version 786825 (0.0008) [2023-12-27 11:04:21,354][20347] Updated weights for policy 0, policy_version 786835 (0.0008) [2023-12-27 11:04:21,427][20347] Updated weights for policy 0, policy_version 786845 (0.0008) [2023-12-27 11:04:21,695][20381] Updated weights for policy 1, policy_version 786362 (0.0007) [2023-12-27 11:04:21,759][20381] Updated weights for policy 1, policy_version 786372 (0.0009) [2023-12-27 11:04:21,819][20381] Updated weights for policy 1, policy_version 786382 (0.0008) [2023-12-27 11:04:21,872][20381] Updated weights for policy 1, policy_version 786392 (0.0008) [2023-12-27 11:04:22,126][20347] Updated weights for policy 0, policy_version 786855 (0.0007) [2023-12-27 11:04:22,193][20347] Updated weights for policy 0, policy_version 786865 (0.0009) [2023-12-27 11:04:22,253][20347] Updated weights for policy 0, policy_version 786875 (0.0007) [2023-12-27 11:04:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 402808832. Throughput: 0: 10269.6, 1: 10262.8. Samples: 402777496. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:22,363][19170] Avg episode reward: [(0, '318.270'), (1, '970.930')] [2023-12-27 11:04:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000786392_201342976.pth... [2023-12-27 11:04:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000786880_201465856.pth... [2023-12-27 11:04:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000785176_201031680.pth [2023-12-27 11:04:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000785664_201154560.pth [2023-12-27 11:04:22,544][20381] Updated weights for policy 1, policy_version 786402 (0.0006) [2023-12-27 11:04:22,605][20381] Updated weights for policy 1, policy_version 786412 (0.0006) [2023-12-27 11:04:22,669][20381] Updated weights for policy 1, policy_version 786422 (0.0008) [2023-12-27 11:04:22,988][20347] Updated weights for policy 0, policy_version 786885 (0.0008) [2023-12-27 11:04:23,050][20347] Updated weights for policy 0, policy_version 786895 (0.0007) [2023-12-27 11:04:23,110][20347] Updated weights for policy 0, policy_version 786905 (0.0008) [2023-12-27 11:04:23,299][20381] Updated weights for policy 1, policy_version 786432 (0.0006) [2023-12-27 11:04:23,358][20381] Updated weights for policy 1, policy_version 786442 (0.0006) [2023-12-27 11:04:23,429][20381] Updated weights for policy 1, policy_version 786452 (0.0011) [2023-12-27 11:04:23,739][20347] Updated weights for policy 0, policy_version 786915 (0.0008) [2023-12-27 11:04:23,787][20347] Updated weights for policy 0, policy_version 786925 (0.0008) [2023-12-27 11:04:23,833][20347] Updated weights for policy 0, policy_version 786935 (0.0008) [2023-12-27 11:04:24,011][20381] Updated weights for policy 1, policy_version 786462 (0.0007) [2023-12-27 11:04:24,068][20381] Updated weights for policy 1, policy_version 786472 (0.0006) [2023-12-27 11:04:24,132][20381] Updated weights for policy 1, policy_version 786482 (0.0006) [2023-12-27 11:04:24,539][20347] Updated weights for policy 0, policy_version 786945 (0.0009) [2023-12-27 11:04:24,591][20347] Updated weights for policy 0, policy_version 786955 (0.0007) [2023-12-27 11:04:24,642][20347] Updated weights for policy 0, policy_version 786965 (0.0007) [2023-12-27 11:04:24,693][20347] Updated weights for policy 0, policy_version 786975 (0.0007) [2023-12-27 11:04:24,721][20381] Updated weights for policy 1, policy_version 786492 (0.0008) [2023-12-27 11:04:24,779][20381] Updated weights for policy 1, policy_version 786502 (0.0011) [2023-12-27 11:04:24,837][20381] Updated weights for policy 1, policy_version 786512 (0.0010) [2023-12-27 11:04:25,334][20347] Updated weights for policy 0, policy_version 786985 (0.0008) [2023-12-27 11:04:25,392][20347] Updated weights for policy 0, policy_version 786995 (0.0008) [2023-12-27 11:04:25,417][20381] Updated weights for policy 1, policy_version 786522 (0.0011) [2023-12-27 11:04:25,448][20347] Updated weights for policy 0, policy_version 787005 (0.0006) [2023-12-27 11:04:25,470][20381] Updated weights for policy 1, policy_version 786532 (0.0011) [2023-12-27 11:04:25,518][20381] Updated weights for policy 1, policy_version 786542 (0.0010) [2023-12-27 11:04:25,567][20381] Updated weights for policy 1, policy_version 786552 (0.0010) [2023-12-27 11:04:26,101][20347] Updated weights for policy 0, policy_version 787015 (0.0010) [2023-12-27 11:04:26,157][20347] Updated weights for policy 0, policy_version 787025 (0.0010) [2023-12-27 11:04:26,217][20347] Updated weights for policy 0, policy_version 787035 (0.0010) [2023-12-27 11:04:26,274][20381] Updated weights for policy 1, policy_version 786562 (0.0007) [2023-12-27 11:04:26,295][20292] KL-divergence is very high: 151.2297 [2023-12-27 11:04:26,324][20381] Updated weights for policy 1, policy_version 786572 (0.0007) [2023-12-27 11:04:26,336][20292] KL-divergence is very high: 111.5027 [2023-12-27 11:04:26,374][20381] Updated weights for policy 1, policy_version 786582 (0.0007) [2023-12-27 11:04:26,835][20347] Updated weights for policy 0, policy_version 787045 (0.0008) [2023-12-27 11:04:26,891][20347] Updated weights for policy 0, policy_version 787055 (0.0005) [2023-12-27 11:04:26,946][20347] Updated weights for policy 0, policy_version 787065 (0.0010) [2023-12-27 11:04:27,040][20292] KL-divergence is very high: 392.6349 [2023-12-27 11:04:27,073][20381] Updated weights for policy 1, policy_version 786592 (0.0010) [2023-12-27 11:04:27,129][20381] Updated weights for policy 1, policy_version 786602 (0.0009) [2023-12-27 11:04:27,179][20381] Updated weights for policy 1, policy_version 786612 (0.0005) [2023-12-27 11:04:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 402915328. Throughput: 0: 10220.9, 1: 10349.5. Samples: 402901540. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:27,363][19170] Avg episode reward: [(0, '450.240'), (1, '942.010')] [2023-12-27 11:04:27,578][20347] Updated weights for policy 0, policy_version 787075 (0.0010) [2023-12-27 11:04:27,635][20347] Updated weights for policy 0, policy_version 787085 (0.0006) [2023-12-27 11:04:27,695][20347] Updated weights for policy 0, policy_version 787095 (0.0006) [2023-12-27 11:04:27,811][20381] Updated weights for policy 1, policy_version 786622 (0.0008) [2023-12-27 11:04:27,859][20381] Updated weights for policy 1, policy_version 786632 (0.0010) [2023-12-27 11:04:27,911][20381] Updated weights for policy 1, policy_version 786642 (0.0010) [2023-12-27 11:04:28,319][20347] Updated weights for policy 0, policy_version 787105 (0.0007) [2023-12-27 11:04:28,380][20347] Updated weights for policy 0, policy_version 787115 (0.0008) [2023-12-27 11:04:28,439][20347] Updated weights for policy 0, policy_version 787125 (0.0008) [2023-12-27 11:04:28,490][20347] Updated weights for policy 0, policy_version 787135 (0.0008) [2023-12-27 11:04:28,626][20381] Updated weights for policy 1, policy_version 786652 (0.0011) [2023-12-27 11:04:28,682][20381] Updated weights for policy 1, policy_version 786662 (0.0008) [2023-12-27 11:04:28,739][20381] Updated weights for policy 1, policy_version 786672 (0.0007) [2023-12-27 11:04:29,164][20347] Updated weights for policy 0, policy_version 787145 (0.0006) [2023-12-27 11:04:29,223][20347] Updated weights for policy 0, policy_version 787155 (0.0008) [2023-12-27 11:04:29,280][20347] Updated weights for policy 0, policy_version 787165 (0.0007) [2023-12-27 11:04:29,394][20381] Updated weights for policy 1, policy_version 786682 (0.0007) [2023-12-27 11:04:29,447][20381] Updated weights for policy 1, policy_version 786692 (0.0008) [2023-12-27 11:04:29,498][20381] Updated weights for policy 1, policy_version 786702 (0.0008) [2023-12-27 11:04:29,549][20381] Updated weights for policy 1, policy_version 786712 (0.0007) [2023-12-27 11:04:29,966][20347] Updated weights for policy 0, policy_version 787175 (0.0007) [2023-12-27 11:04:30,028][20347] Updated weights for policy 0, policy_version 787185 (0.0007) [2023-12-27 11:04:30,093][20347] Updated weights for policy 0, policy_version 787195 (0.0006) [2023-12-27 11:04:30,325][20381] Updated weights for policy 1, policy_version 786722 (0.0008) [2023-12-27 11:04:30,389][20381] Updated weights for policy 1, policy_version 786732 (0.0008) [2023-12-27 11:04:30,442][20381] Updated weights for policy 1, policy_version 786742 (0.0008) [2023-12-27 11:04:30,658][20347] Updated weights for policy 0, policy_version 787205 (0.0007) [2023-12-27 11:04:30,709][20347] Updated weights for policy 0, policy_version 787215 (0.0008) [2023-12-27 11:04:30,762][20347] Updated weights for policy 0, policy_version 787225 (0.0006) [2023-12-27 11:04:31,210][20381] Updated weights for policy 1, policy_version 786752 (0.0009) [2023-12-27 11:04:31,274][20381] Updated weights for policy 1, policy_version 786762 (0.0008) [2023-12-27 11:04:31,324][20381] Updated weights for policy 1, policy_version 786772 (0.0008) [2023-12-27 11:04:31,351][20347] Updated weights for policy 0, policy_version 787235 (0.0005) [2023-12-27 11:04:31,419][20347] Updated weights for policy 0, policy_version 787245 (0.0009) [2023-12-27 11:04:31,477][20347] Updated weights for policy 0, policy_version 787255 (0.0010) [2023-12-27 11:04:32,055][20381] Updated weights for policy 1, policy_version 786782 (0.0008) [2023-12-27 11:04:32,104][20381] Updated weights for policy 1, policy_version 786792 (0.0009) [2023-12-27 11:04:32,159][20347] Updated weights for policy 0, policy_version 787265 (0.0010) [2023-12-27 11:04:32,161][20381] Updated weights for policy 1, policy_version 786802 (0.0007) [2023-12-27 11:04:32,217][20347] Updated weights for policy 0, policy_version 787275 (0.0010) [2023-12-27 11:04:32,275][20347] Updated weights for policy 0, policy_version 787285 (0.0010) [2023-12-27 11:04:32,332][20347] Updated weights for policy 0, policy_version 787295 (0.0010) [2023-12-27 11:04:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 403021824. Throughput: 0: 10315.3, 1: 10307.3. Samples: 403025508. Policy #0 lag: (min: 2.0, avg: 10.5, max: 34.0) [2023-12-27 11:04:32,363][19170] Avg episode reward: [(0, '444.670'), (1, '932.250')] [2023-12-27 11:04:32,865][20381] Updated weights for policy 1, policy_version 786812 (0.0006) [2023-12-27 11:04:32,915][20381] Updated weights for policy 1, policy_version 786822 (0.0007) [2023-12-27 11:04:32,968][20381] Updated weights for policy 1, policy_version 786832 (0.0008) [2023-12-27 11:04:32,978][20347] Updated weights for policy 0, policy_version 787305 (0.0010) [2023-12-27 11:04:33,032][20347] Updated weights for policy 0, policy_version 787315 (0.0010) [2023-12-27 11:04:33,083][20347] Updated weights for policy 0, policy_version 787325 (0.0009) [2023-12-27 11:04:33,699][20381] Updated weights for policy 1, policy_version 786842 (0.0006) [2023-12-27 11:04:33,744][20347] Updated weights for policy 0, policy_version 787335 (0.0007) [2023-12-27 11:04:33,753][20381] Updated weights for policy 1, policy_version 786852 (0.0005) [2023-12-27 11:04:33,770][20291] KL-divergence is very high: 102.8838 [2023-12-27 11:04:33,805][20347] Updated weights for policy 0, policy_version 787345 (0.0006) [2023-12-27 11:04:33,808][20381] Updated weights for policy 1, policy_version 786862 (0.0005) [2023-12-27 11:04:33,859][20381] Updated weights for policy 1, policy_version 786872 (0.0006) [2023-12-27 11:04:33,864][20291] KL-divergence is very high: 104.5837 [2023-12-27 11:04:33,865][20347] Updated weights for policy 0, policy_version 787355 (0.0006) [2023-12-27 11:04:34,507][20347] Updated weights for policy 0, policy_version 787365 (0.0007) [2023-12-27 11:04:34,570][20347] Updated weights for policy 0, policy_version 787375 (0.0008) [2023-12-27 11:04:34,583][20381] Updated weights for policy 1, policy_version 786882 (0.0006) [2023-12-27 11:04:34,620][20347] Updated weights for policy 0, policy_version 787385 (0.0010) [2023-12-27 11:04:34,641][20381] Updated weights for policy 1, policy_version 786892 (0.0006) [2023-12-27 11:04:34,702][20381] Updated weights for policy 1, policy_version 786902 (0.0007) [2023-12-27 11:04:35,286][20347] Updated weights for policy 0, policy_version 787395 (0.0009) [2023-12-27 11:04:35,338][20347] Updated weights for policy 0, policy_version 787405 (0.0006) [2023-12-27 11:04:35,391][20347] Updated weights for policy 0, policy_version 787415 (0.0007) [2023-12-27 11:04:35,412][20381] Updated weights for policy 1, policy_version 786912 (0.0006) [2023-12-27 11:04:35,470][20381] Updated weights for policy 1, policy_version 786922 (0.0007) [2023-12-27 11:04:35,523][20381] Updated weights for policy 1, policy_version 786932 (0.0008) [2023-12-27 11:04:36,006][20347] Updated weights for policy 0, policy_version 787425 (0.0007) [2023-12-27 11:04:36,063][20347] Updated weights for policy 0, policy_version 787435 (0.0008) [2023-12-27 11:04:36,122][20347] Updated weights for policy 0, policy_version 787445 (0.0007) [2023-12-27 11:04:36,179][20347] Updated weights for policy 0, policy_version 787455 (0.0007) [2023-12-27 11:04:36,222][20381] Updated weights for policy 1, policy_version 786942 (0.0008) [2023-12-27 11:04:36,290][20381] Updated weights for policy 1, policy_version 786952 (0.0007) [2023-12-27 11:04:36,349][20381] Updated weights for policy 1, policy_version 786962 (0.0007) [2023-12-27 11:04:36,881][20347] Updated weights for policy 0, policy_version 787465 (0.0006) [2023-12-27 11:04:36,936][20347] Updated weights for policy 0, policy_version 787475 (0.0007) [2023-12-27 11:04:36,989][20347] Updated weights for policy 0, policy_version 787485 (0.0007) [2023-12-27 11:04:37,055][20381] Updated weights for policy 1, policy_version 786972 (0.0007) [2023-12-27 11:04:37,118][20381] Updated weights for policy 1, policy_version 786982 (0.0008) [2023-12-27 11:04:37,177][20381] Updated weights for policy 1, policy_version 786992 (0.0008) [2023-12-27 11:04:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 403120128. Throughput: 0: 10306.5, 1: 10318.8. Samples: 403086780. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:04:37,363][19170] Avg episode reward: [(0, '567.310'), (1, '951.460')] [2023-12-27 11:04:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000787000_201498624.pth... [2023-12-27 11:04:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000787488_201621504.pth... [2023-12-27 11:04:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000785816_201195520.pth [2023-12-27 11:04:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000786272_201310208.pth [2023-12-27 11:04:37,654][20347] Updated weights for policy 0, policy_version 787495 (0.0008) [2023-12-27 11:04:37,722][20347] Updated weights for policy 0, policy_version 787505 (0.0008) [2023-12-27 11:04:37,785][20347] Updated weights for policy 0, policy_version 787515 (0.0009) [2023-12-27 11:04:37,860][20381] Updated weights for policy 1, policy_version 787002 (0.0008) [2023-12-27 11:04:37,922][20381] Updated weights for policy 1, policy_version 787012 (0.0007) [2023-12-27 11:04:37,985][20381] Updated weights for policy 1, policy_version 787022 (0.0009) [2023-12-27 11:04:38,036][20381] Updated weights for policy 1, policy_version 787032 (0.0007) [2023-12-27 11:04:38,422][20347] Updated weights for policy 0, policy_version 787525 (0.0007) [2023-12-27 11:04:38,484][20347] Updated weights for policy 0, policy_version 787535 (0.0010) [2023-12-27 11:04:38,544][20347] Updated weights for policy 0, policy_version 787545 (0.0007) [2023-12-27 11:04:38,714][20381] Updated weights for policy 1, policy_version 787042 (0.0007) [2023-12-27 11:04:38,774][20381] Updated weights for policy 1, policy_version 787052 (0.0006) [2023-12-27 11:04:38,827][20381] Updated weights for policy 1, policy_version 787062 (0.0006) [2023-12-27 11:04:39,185][20347] Updated weights for policy 0, policy_version 787555 (0.0007) [2023-12-27 11:04:39,250][20347] Updated weights for policy 0, policy_version 787565 (0.0008) [2023-12-27 11:04:39,309][20347] Updated weights for policy 0, policy_version 787575 (0.0008) [2023-12-27 11:04:39,522][20381] Updated weights for policy 1, policy_version 787072 (0.0006) [2023-12-27 11:04:39,585][20381] Updated weights for policy 1, policy_version 787082 (0.0008) [2023-12-27 11:04:39,647][20381] Updated weights for policy 1, policy_version 787092 (0.0006) [2023-12-27 11:04:40,073][20347] Updated weights for policy 0, policy_version 787585 (0.0007) [2023-12-27 11:04:40,134][20347] Updated weights for policy 0, policy_version 787595 (0.0008) [2023-12-27 11:04:40,194][20347] Updated weights for policy 0, policy_version 787605 (0.0008) [2023-12-27 11:04:40,257][20347] Updated weights for policy 0, policy_version 787615 (0.0007) [2023-12-27 11:04:40,318][20381] Updated weights for policy 1, policy_version 787102 (0.0008) [2023-12-27 11:04:40,377][20381] Updated weights for policy 1, policy_version 787112 (0.0007) [2023-12-27 11:04:40,433][20381] Updated weights for policy 1, policy_version 787122 (0.0007) [2023-12-27 11:04:40,997][20347] Updated weights for policy 0, policy_version 787625 (0.0009) [2023-12-27 11:04:41,057][20347] Updated weights for policy 0, policy_version 787635 (0.0007) [2023-12-27 11:04:41,068][20381] Updated weights for policy 1, policy_version 787132 (0.0008) [2023-12-27 11:04:41,120][20381] Updated weights for policy 1, policy_version 787142 (0.0007) [2023-12-27 11:04:41,120][20347] Updated weights for policy 0, policy_version 787645 (0.0008) [2023-12-27 11:04:41,182][20381] Updated weights for policy 1, policy_version 787152 (0.0009) [2023-12-27 11:04:41,921][20347] Updated weights for policy 0, policy_version 787655 (0.0007) [2023-12-27 11:04:41,967][20381] Updated weights for policy 1, policy_version 787162 (0.0007) [2023-12-27 11:04:41,980][20347] Updated weights for policy 0, policy_version 787665 (0.0005) [2023-12-27 11:04:42,020][20292] KL-divergence is very high: 185.5737 [2023-12-27 11:04:42,026][20381] Updated weights for policy 1, policy_version 787172 (0.0008) [2023-12-27 11:04:42,040][20347] Updated weights for policy 0, policy_version 787675 (0.0005) [2023-12-27 11:04:42,085][20381] Updated weights for policy 1, policy_version 787182 (0.0007) [2023-12-27 11:04:42,143][20381] Updated weights for policy 1, policy_version 787192 (0.0008) [2023-12-27 11:04:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 403218432. Throughput: 0: 10285.7, 1: 10296.3. Samples: 403207684. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:04:42,363][19170] Avg episode reward: [(0, '556.450'), (1, '941.740')] [2023-12-27 11:04:42,725][20347] Updated weights for policy 0, policy_version 787685 (0.0009) [2023-12-27 11:04:42,788][20347] Updated weights for policy 0, policy_version 787695 (0.0008) [2023-12-27 11:04:42,848][20347] Updated weights for policy 0, policy_version 787705 (0.0006) [2023-12-27 11:04:42,865][20381] Updated weights for policy 1, policy_version 787202 (0.0008) [2023-12-27 11:04:42,923][20381] Updated weights for policy 1, policy_version 787212 (0.0007) [2023-12-27 11:04:42,986][20381] Updated weights for policy 1, policy_version 787222 (0.0008) [2023-12-27 11:04:43,464][20347] Updated weights for policy 0, policy_version 787715 (0.0005) [2023-12-27 11:04:43,521][20347] Updated weights for policy 0, policy_version 787725 (0.0006) [2023-12-27 11:04:43,569][20347] Updated weights for policy 0, policy_version 787735 (0.0007) [2023-12-27 11:04:43,712][20381] Updated weights for policy 1, policy_version 787232 (0.0008) [2023-12-27 11:04:43,770][20381] Updated weights for policy 1, policy_version 787242 (0.0009) [2023-12-27 11:04:43,836][20381] Updated weights for policy 1, policy_version 787252 (0.0008) [2023-12-27 11:04:44,123][20347] Updated weights for policy 0, policy_version 787745 (0.0007) [2023-12-27 11:04:44,168][20347] Updated weights for policy 0, policy_version 787755 (0.0005) [2023-12-27 11:04:44,203][20291] KL-divergence is very high: 181.3305 [2023-12-27 11:04:44,230][20347] Updated weights for policy 0, policy_version 787765 (0.0010) [2023-12-27 11:04:44,282][20347] Updated weights for policy 0, policy_version 787775 (0.0006) [2023-12-27 11:04:44,537][20381] Updated weights for policy 1, policy_version 787262 (0.0007) [2023-12-27 11:04:44,592][20381] Updated weights for policy 1, policy_version 787272 (0.0005) [2023-12-27 11:04:44,647][20381] Updated weights for policy 1, policy_version 787282 (0.0007) [2023-12-27 11:04:44,878][20347] Updated weights for policy 0, policy_version 787785 (0.0008) [2023-12-27 11:04:44,942][20347] Updated weights for policy 0, policy_version 787795 (0.0010) [2023-12-27 11:04:45,004][20347] Updated weights for policy 0, policy_version 787805 (0.0010) [2023-12-27 11:04:45,352][20381] Updated weights for policy 1, policy_version 787292 (0.0007) [2023-12-27 11:04:45,411][20381] Updated weights for policy 1, policy_version 787302 (0.0007) [2023-12-27 11:04:45,459][20381] Updated weights for policy 1, policy_version 787312 (0.0008) [2023-12-27 11:04:45,690][20347] Updated weights for policy 0, policy_version 787815 (0.0009) [2023-12-27 11:04:45,750][20347] Updated weights for policy 0, policy_version 787825 (0.0006) [2023-12-27 11:04:45,807][20347] Updated weights for policy 0, policy_version 787835 (0.0005) [2023-12-27 11:04:46,212][20381] Updated weights for policy 1, policy_version 787322 (0.0008) [2023-12-27 11:04:46,263][20381] Updated weights for policy 1, policy_version 787332 (0.0008) [2023-12-27 11:04:46,311][20381] Updated weights for policy 1, policy_version 787342 (0.0008) [2023-12-27 11:04:46,333][20347] Updated weights for policy 0, policy_version 787845 (0.0006) [2023-12-27 11:04:46,361][20381] Updated weights for policy 1, policy_version 787352 (0.0007) [2023-12-27 11:04:46,397][20347] Updated weights for policy 0, policy_version 787855 (0.0005) [2023-12-27 11:04:46,450][20347] Updated weights for policy 0, policy_version 787865 (0.0005) [2023-12-27 11:04:47,097][20381] Updated weights for policy 1, policy_version 787362 (0.0005) [2023-12-27 11:04:47,103][20347] Updated weights for policy 0, policy_version 787875 (0.0007) [2023-12-27 11:04:47,152][20381] Updated weights for policy 1, policy_version 787372 (0.0006) [2023-12-27 11:04:47,162][20347] Updated weights for policy 0, policy_version 787885 (0.0008) [2023-12-27 11:04:47,203][20381] Updated weights for policy 1, policy_version 787382 (0.0006) [2023-12-27 11:04:47,208][20347] Updated weights for policy 0, policy_version 787895 (0.0008) [2023-12-27 11:04:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 403324928. Throughput: 0: 10376.9, 1: 10201.6. Samples: 403330820. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:04:47,364][19170] Avg episode reward: [(0, '661.620'), (1, '865.340')] [2023-12-27 11:04:47,757][20381] Updated weights for policy 1, policy_version 787392 (0.0008) [2023-12-27 11:04:47,806][20381] Updated weights for policy 1, policy_version 787402 (0.0008) [2023-12-27 11:04:47,836][20347] Updated weights for policy 0, policy_version 787905 (0.0007) [2023-12-27 11:04:47,860][20381] Updated weights for policy 1, policy_version 787412 (0.0007) [2023-12-27 11:04:47,882][20347] Updated weights for policy 0, policy_version 787915 (0.0005) [2023-12-27 11:04:47,931][20347] Updated weights for policy 0, policy_version 787925 (0.0005) [2023-12-27 11:04:47,980][20347] Updated weights for policy 0, policy_version 787935 (0.0005) [2023-12-27 11:04:48,552][20381] Updated weights for policy 1, policy_version 787422 (0.0007) [2023-12-27 11:04:48,605][20381] Updated weights for policy 1, policy_version 787432 (0.0008) [2023-12-27 11:04:48,664][20381] Updated weights for policy 1, policy_version 787442 (0.0011) [2023-12-27 11:04:48,714][20347] Updated weights for policy 0, policy_version 787945 (0.0006) [2023-12-27 11:04:48,763][20347] Updated weights for policy 0, policy_version 787955 (0.0007) [2023-12-27 11:04:48,812][20347] Updated weights for policy 0, policy_version 787965 (0.0007) [2023-12-27 11:04:49,361][20381] Updated weights for policy 1, policy_version 787452 (0.0010) [2023-12-27 11:04:49,393][20347] Updated weights for policy 0, policy_version 787975 (0.0007) [2023-12-27 11:04:49,422][20381] Updated weights for policy 1, policy_version 787462 (0.0007) [2023-12-27 11:04:49,446][20347] Updated weights for policy 0, policy_version 787985 (0.0006) [2023-12-27 11:04:49,469][20381] Updated weights for policy 1, policy_version 787472 (0.0006) [2023-12-27 11:04:49,498][20347] Updated weights for policy 0, policy_version 787995 (0.0006) [2023-12-27 11:04:50,137][20381] Updated weights for policy 1, policy_version 787482 (0.0006) [2023-12-27 11:04:50,195][20381] Updated weights for policy 1, policy_version 787492 (0.0008) [2023-12-27 11:04:50,231][20347] Updated weights for policy 0, policy_version 788005 (0.0007) [2023-12-27 11:04:50,255][20381] Updated weights for policy 1, policy_version 787502 (0.0008) [2023-12-27 11:04:50,292][20347] Updated weights for policy 0, policy_version 788015 (0.0006) [2023-12-27 11:04:50,316][20381] Updated weights for policy 1, policy_version 787512 (0.0006) [2023-12-27 11:04:50,350][20347] Updated weights for policy 0, policy_version 788025 (0.0006) [2023-12-27 11:04:51,011][20347] Updated weights for policy 0, policy_version 788035 (0.0007) [2023-12-27 11:04:51,014][20381] Updated weights for policy 1, policy_version 787522 (0.0006) [2023-12-27 11:04:51,078][20381] Updated weights for policy 1, policy_version 787532 (0.0007) [2023-12-27 11:04:51,080][20347] Updated weights for policy 0, policy_version 788045 (0.0007) [2023-12-27 11:04:51,141][20381] Updated weights for policy 1, policy_version 787542 (0.0008) [2023-12-27 11:04:51,151][20347] Updated weights for policy 0, policy_version 788055 (0.0006) [2023-12-27 11:04:51,771][20381] Updated weights for policy 1, policy_version 787552 (0.0006) [2023-12-27 11:04:51,832][20381] Updated weights for policy 1, policy_version 787562 (0.0005) [2023-12-27 11:04:51,874][20347] Updated weights for policy 0, policy_version 788065 (0.0007) [2023-12-27 11:04:51,889][20381] Updated weights for policy 1, policy_version 787572 (0.0005) [2023-12-27 11:04:51,933][20347] Updated weights for policy 0, policy_version 788075 (0.0007) [2023-12-27 11:04:51,993][20347] Updated weights for policy 0, policy_version 788085 (0.0008) [2023-12-27 11:04:52,051][20347] Updated weights for policy 0, policy_version 788095 (0.0008) [2023-12-27 11:04:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 403423232. Throughput: 0: 10400.8, 1: 10173.9. Samples: 403393904. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:04:52,363][19170] Avg episode reward: [(0, '730.210'), (1, '884.210')] [2023-12-27 11:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000788096_201777152.pth... [2023-12-27 11:04:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000787576_201646080.pth... [2023-12-27 11:04:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000786392_201342976.pth [2023-12-27 11:04:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000786880_201465856.pth [2023-12-27 11:04:52,576][20381] Updated weights for policy 1, policy_version 787582 (0.0007) [2023-12-27 11:04:52,626][20381] Updated weights for policy 1, policy_version 787592 (0.0008) [2023-12-27 11:04:52,688][20381] Updated weights for policy 1, policy_version 787602 (0.0008) [2023-12-27 11:04:52,786][20347] Updated weights for policy 0, policy_version 788105 (0.0008) [2023-12-27 11:04:52,843][20347] Updated weights for policy 0, policy_version 788115 (0.0007) [2023-12-27 11:04:52,900][20347] Updated weights for policy 0, policy_version 788125 (0.0008) [2023-12-27 11:04:53,266][20381] Updated weights for policy 1, policy_version 787612 (0.0008) [2023-12-27 11:04:53,313][20381] Updated weights for policy 1, policy_version 787622 (0.0009) [2023-12-27 11:04:53,363][20381] Updated weights for policy 1, policy_version 787632 (0.0008) [2023-12-27 11:04:53,495][20347] Updated weights for policy 0, policy_version 788135 (0.0006) [2023-12-27 11:04:53,540][20347] Updated weights for policy 0, policy_version 788145 (0.0008) [2023-12-27 11:04:53,590][20347] Updated weights for policy 0, policy_version 788155 (0.0008) [2023-12-27 11:04:54,098][20381] Updated weights for policy 1, policy_version 787642 (0.0007) [2023-12-27 11:04:54,147][20381] Updated weights for policy 1, policy_version 787652 (0.0008) [2023-12-27 11:04:54,196][20381] Updated weights for policy 1, policy_version 787662 (0.0008) [2023-12-27 11:04:54,255][20381] Updated weights for policy 1, policy_version 787672 (0.0008) [2023-12-27 11:04:54,288][20347] Updated weights for policy 0, policy_version 788165 (0.0007) [2023-12-27 11:04:54,346][20347] Updated weights for policy 0, policy_version 788175 (0.0007) [2023-12-27 11:04:54,412][20347] Updated weights for policy 0, policy_version 788185 (0.0007) [2023-12-27 11:04:54,964][20381] Updated weights for policy 1, policy_version 787682 (0.0008) [2023-12-27 11:04:55,015][20381] Updated weights for policy 1, policy_version 787692 (0.0006) [2023-12-27 11:04:55,074][20381] Updated weights for policy 1, policy_version 787702 (0.0005) [2023-12-27 11:04:55,086][20347] Updated weights for policy 0, policy_version 788195 (0.0007) [2023-12-27 11:04:55,142][20347] Updated weights for policy 0, policy_version 788205 (0.0008) [2023-12-27 11:04:55,192][20347] Updated weights for policy 0, policy_version 788215 (0.0008) [2023-12-27 11:04:55,667][20381] Updated weights for policy 1, policy_version 787712 (0.0005) [2023-12-27 11:04:55,723][20381] Updated weights for policy 1, policy_version 787722 (0.0005) [2023-12-27 11:04:55,786][20381] Updated weights for policy 1, policy_version 787732 (0.0005) [2023-12-27 11:04:55,905][20347] Updated weights for policy 0, policy_version 788225 (0.0008) [2023-12-27 11:04:55,964][20347] Updated weights for policy 0, policy_version 788235 (0.0006) [2023-12-27 11:04:56,026][20347] Updated weights for policy 0, policy_version 788245 (0.0010) [2023-12-27 11:04:56,091][20347] Updated weights for policy 0, policy_version 788255 (0.0010) [2023-12-27 11:04:56,317][20381] Updated weights for policy 1, policy_version 787742 (0.0008) [2023-12-27 11:04:56,367][20381] Updated weights for policy 1, policy_version 787752 (0.0010) [2023-12-27 11:04:56,425][20381] Updated weights for policy 1, policy_version 787762 (0.0011) [2023-12-27 11:04:56,678][20347] Updated weights for policy 0, policy_version 788265 (0.0010) [2023-12-27 11:04:56,736][20347] Updated weights for policy 0, policy_version 788275 (0.0010) [2023-12-27 11:04:56,788][20347] Updated weights for policy 0, policy_version 788285 (0.0010) [2023-12-27 11:04:56,972][20381] Updated weights for policy 1, policy_version 787772 (0.0006) [2023-12-27 11:04:57,030][20381] Updated weights for policy 1, policy_version 787782 (0.0008) [2023-12-27 11:04:57,077][20381] Updated weights for policy 1, policy_version 787792 (0.0010) [2023-12-27 11:04:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 403529728. Throughput: 0: 10369.2, 1: 10185.7. Samples: 403518408. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:04:57,363][19170] Avg episode reward: [(0, '680.110'), (1, '941.720')] [2023-12-27 11:04:57,470][20347] Updated weights for policy 0, policy_version 788295 (0.0008) [2023-12-27 11:04:57,520][20347] Updated weights for policy 0, policy_version 788305 (0.0005) [2023-12-27 11:04:57,571][20347] Updated weights for policy 0, policy_version 788315 (0.0008) [2023-12-27 11:04:57,705][20381] Updated weights for policy 1, policy_version 787802 (0.0009) [2023-12-27 11:04:57,759][20381] Updated weights for policy 1, policy_version 787812 (0.0006) [2023-12-27 11:04:57,809][20381] Updated weights for policy 1, policy_version 787822 (0.0010) [2023-12-27 11:04:57,854][20381] Updated weights for policy 1, policy_version 787832 (0.0010) [2023-12-27 11:04:58,210][20347] Updated weights for policy 0, policy_version 788325 (0.0007) [2023-12-27 11:04:58,270][20347] Updated weights for policy 0, policy_version 788335 (0.0008) [2023-12-27 11:04:58,343][20347] Updated weights for policy 0, policy_version 788345 (0.0008) [2023-12-27 11:04:58,527][20381] Updated weights for policy 1, policy_version 787842 (0.0008) [2023-12-27 11:04:58,592][20381] Updated weights for policy 1, policy_version 787852 (0.0008) [2023-12-27 11:04:58,660][20381] Updated weights for policy 1, policy_version 787862 (0.0007) [2023-12-27 11:04:59,087][20347] Updated weights for policy 0, policy_version 788355 (0.0008) [2023-12-27 11:04:59,146][20347] Updated weights for policy 0, policy_version 788365 (0.0007) [2023-12-27 11:04:59,210][20347] Updated weights for policy 0, policy_version 788375 (0.0007) [2023-12-27 11:04:59,469][20381] Updated weights for policy 1, policy_version 787872 (0.0008) [2023-12-27 11:04:59,528][20381] Updated weights for policy 1, policy_version 787882 (0.0009) [2023-12-27 11:04:59,581][20381] Updated weights for policy 1, policy_version 787892 (0.0008) [2023-12-27 11:04:59,857][20347] Updated weights for policy 0, policy_version 788385 (0.0008) [2023-12-27 11:04:59,918][20347] Updated weights for policy 0, policy_version 788395 (0.0011) [2023-12-27 11:04:59,977][20347] Updated weights for policy 0, policy_version 788405 (0.0009) [2023-12-27 11:05:00,036][20347] Updated weights for policy 0, policy_version 788415 (0.0008) [2023-12-27 11:05:00,358][20381] Updated weights for policy 1, policy_version 787902 (0.0008) [2023-12-27 11:05:00,414][20381] Updated weights for policy 1, policy_version 787912 (0.0008) [2023-12-27 11:05:00,470][20381] Updated weights for policy 1, policy_version 787922 (0.0009) [2023-12-27 11:05:00,647][20347] Updated weights for policy 0, policy_version 788425 (0.0005) [2023-12-27 11:05:00,707][20347] Updated weights for policy 0, policy_version 788435 (0.0005) [2023-12-27 11:05:00,771][20347] Updated weights for policy 0, policy_version 788445 (0.0006) [2023-12-27 11:05:01,145][20381] Updated weights for policy 1, policy_version 787932 (0.0007) [2023-12-27 11:05:01,199][20381] Updated weights for policy 1, policy_version 787942 (0.0007) [2023-12-27 11:05:01,259][20381] Updated weights for policy 1, policy_version 787952 (0.0007) [2023-12-27 11:05:01,369][20347] Updated weights for policy 0, policy_version 788455 (0.0007) [2023-12-27 11:05:01,433][20347] Updated weights for policy 0, policy_version 788465 (0.0007) [2023-12-27 11:05:01,481][20347] Updated weights for policy 0, policy_version 788475 (0.0007) [2023-12-27 11:05:02,010][20381] Updated weights for policy 1, policy_version 787962 (0.0008) [2023-12-27 11:05:02,063][20381] Updated weights for policy 1, policy_version 787972 (0.0007) [2023-12-27 11:05:02,113][20381] Updated weights for policy 1, policy_version 787982 (0.0007) [2023-12-27 11:05:02,152][20347] Updated weights for policy 0, policy_version 788485 (0.0007) [2023-12-27 11:05:02,166][20381] Updated weights for policy 1, policy_version 787992 (0.0007) [2023-12-27 11:05:02,203][20347] Updated weights for policy 0, policy_version 788495 (0.0006) [2023-12-27 11:05:02,257][20347] Updated weights for policy 0, policy_version 788505 (0.0006) [2023-12-27 11:05:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 403636224. Throughput: 0: 10393.3, 1: 10128.6. Samples: 403641636. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:02,363][19170] Avg episode reward: [(0, '680.700'), (1, '961.020')] [2023-12-27 11:05:02,884][20347] Updated weights for policy 0, policy_version 788515 (0.0007) [2023-12-27 11:05:02,909][20381] Updated weights for policy 1, policy_version 788002 (0.0008) [2023-12-27 11:05:02,944][20347] Updated weights for policy 0, policy_version 788525 (0.0005) [2023-12-27 11:05:02,963][20381] Updated weights for policy 1, policy_version 788012 (0.0007) [2023-12-27 11:05:03,002][20347] Updated weights for policy 0, policy_version 788535 (0.0005) [2023-12-27 11:05:03,013][20381] Updated weights for policy 1, policy_version 788022 (0.0008) [2023-12-27 11:05:03,651][20347] Updated weights for policy 0, policy_version 788545 (0.0007) [2023-12-27 11:05:03,712][20347] Updated weights for policy 0, policy_version 788555 (0.0008) [2023-12-27 11:05:03,723][20381] Updated weights for policy 1, policy_version 788032 (0.0006) [2023-12-27 11:05:03,766][20347] Updated weights for policy 0, policy_version 788565 (0.0007) [2023-12-27 11:05:03,773][20381] Updated weights for policy 1, policy_version 788042 (0.0006) [2023-12-27 11:05:03,828][20381] Updated weights for policy 1, policy_version 788052 (0.0008) [2023-12-27 11:05:03,829][20347] Updated weights for policy 0, policy_version 788575 (0.0007) [2023-12-27 11:05:04,501][20347] Updated weights for policy 0, policy_version 788585 (0.0010) [2023-12-27 11:05:04,503][20381] Updated weights for policy 1, policy_version 788062 (0.0007) [2023-12-27 11:05:04,556][20381] Updated weights for policy 1, policy_version 788072 (0.0006) [2023-12-27 11:05:04,556][20347] Updated weights for policy 0, policy_version 788595 (0.0008) [2023-12-27 11:05:04,607][20347] Updated weights for policy 0, policy_version 788605 (0.0007) [2023-12-27 11:05:04,616][20381] Updated weights for policy 1, policy_version 788082 (0.0008) [2023-12-27 11:05:05,202][20347] Updated weights for policy 0, policy_version 788615 (0.0007) [2023-12-27 11:05:05,253][20347] Updated weights for policy 0, policy_version 788625 (0.0005) [2023-12-27 11:05:05,304][20347] Updated weights for policy 0, policy_version 788635 (0.0005) [2023-12-27 11:05:05,350][20381] Updated weights for policy 1, policy_version 788092 (0.0008) [2023-12-27 11:05:05,401][20381] Updated weights for policy 1, policy_version 788102 (0.0007) [2023-12-27 11:05:05,458][20381] Updated weights for policy 1, policy_version 788112 (0.0008) [2023-12-27 11:05:05,946][20347] Updated weights for policy 0, policy_version 788645 (0.0008) [2023-12-27 11:05:06,001][20347] Updated weights for policy 0, policy_version 788655 (0.0010) [2023-12-27 11:05:06,055][20347] Updated weights for policy 0, policy_version 788665 (0.0009) [2023-12-27 11:05:06,067][20381] Updated weights for policy 1, policy_version 788122 (0.0007) [2023-12-27 11:05:06,129][20381] Updated weights for policy 1, policy_version 788132 (0.0007) [2023-12-27 11:05:06,194][20381] Updated weights for policy 1, policy_version 788142 (0.0009) [2023-12-27 11:05:06,254][20381] Updated weights for policy 1, policy_version 788152 (0.0009) [2023-12-27 11:05:06,821][20347] Updated weights for policy 0, policy_version 788675 (0.0010) [2023-12-27 11:05:06,880][20347] Updated weights for policy 0, policy_version 788685 (0.0011) [2023-12-27 11:05:06,926][20381] Updated weights for policy 1, policy_version 788162 (0.0011) [2023-12-27 11:05:06,929][20347] Updated weights for policy 0, policy_version 788695 (0.0010) [2023-12-27 11:05:06,983][20381] Updated weights for policy 1, policy_version 788172 (0.0009) [2023-12-27 11:05:07,050][20381] Updated weights for policy 1, policy_version 788182 (0.0008) [2023-12-27 11:05:07,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 403734528. Throughput: 0: 10439.6, 1: 10133.7. Samples: 403703300. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:07,364][19170] Avg episode reward: [(0, '680.700'), (1, '913.180')] [2023-12-27 11:05:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000788184_201801728.pth... [2023-12-27 11:05:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000788704_201932800.pth... [2023-12-27 11:05:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000787488_201621504.pth [2023-12-27 11:05:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000787000_201498624.pth [2023-12-27 11:05:07,593][20347] Updated weights for policy 0, policy_version 788705 (0.0009) [2023-12-27 11:05:07,622][20381] Updated weights for policy 1, policy_version 788192 (0.0005) [2023-12-27 11:05:07,652][20347] Updated weights for policy 0, policy_version 788715 (0.0009) [2023-12-27 11:05:07,671][20381] Updated weights for policy 1, policy_version 788202 (0.0005) [2023-12-27 11:05:07,702][20347] Updated weights for policy 0, policy_version 788725 (0.0010) [2023-12-27 11:05:07,716][20381] Updated weights for policy 1, policy_version 788212 (0.0005) [2023-12-27 11:05:07,753][20347] Updated weights for policy 0, policy_version 788735 (0.0006) [2023-12-27 11:05:08,364][20381] Updated weights for policy 1, policy_version 788222 (0.0007) [2023-12-27 11:05:08,422][20381] Updated weights for policy 1, policy_version 788232 (0.0007) [2023-12-27 11:05:08,435][20347] Updated weights for policy 0, policy_version 788745 (0.0006) [2023-12-27 11:05:08,479][20381] Updated weights for policy 1, policy_version 788242 (0.0006) [2023-12-27 11:05:08,492][20347] Updated weights for policy 0, policy_version 788755 (0.0006) [2023-12-27 11:05:08,554][20347] Updated weights for policy 0, policy_version 788765 (0.0007) [2023-12-27 11:05:09,069][20381] Updated weights for policy 1, policy_version 788252 (0.0006) [2023-12-27 11:05:09,117][20381] Updated weights for policy 1, policy_version 788262 (0.0005) [2023-12-27 11:05:09,171][20381] Updated weights for policy 1, policy_version 788272 (0.0005) [2023-12-27 11:05:09,305][20347] Updated weights for policy 0, policy_version 788775 (0.0009) [2023-12-27 11:05:09,370][20347] Updated weights for policy 0, policy_version 788785 (0.0008) [2023-12-27 11:05:09,433][20347] Updated weights for policy 0, policy_version 788795 (0.0008) [2023-12-27 11:05:09,818][20381] Updated weights for policy 1, policy_version 788282 (0.0007) [2023-12-27 11:05:09,882][20381] Updated weights for policy 1, policy_version 788292 (0.0009) [2023-12-27 11:05:09,939][20381] Updated weights for policy 1, policy_version 788302 (0.0008) [2023-12-27 11:05:10,001][20381] Updated weights for policy 1, policy_version 788312 (0.0009) [2023-12-27 11:05:10,125][20347] Updated weights for policy 0, policy_version 788805 (0.0007) [2023-12-27 11:05:10,186][20347] Updated weights for policy 0, policy_version 788815 (0.0006) [2023-12-27 11:05:10,248][20347] Updated weights for policy 0, policy_version 788825 (0.0006) [2023-12-27 11:05:10,717][20381] Updated weights for policy 1, policy_version 788322 (0.0010) [2023-12-27 11:05:10,773][20381] Updated weights for policy 1, policy_version 788332 (0.0011) [2023-12-27 11:05:10,822][20347] Updated weights for policy 0, policy_version 788835 (0.0007) [2023-12-27 11:05:10,829][20381] Updated weights for policy 1, policy_version 788342 (0.0010) [2023-12-27 11:05:10,882][20347] Updated weights for policy 0, policy_version 788845 (0.0009) [2023-12-27 11:05:10,948][20347] Updated weights for policy 0, policy_version 788855 (0.0008) [2023-12-27 11:05:11,626][20381] Updated weights for policy 1, policy_version 788352 (0.0009) [2023-12-27 11:05:11,693][20381] Updated weights for policy 1, policy_version 788362 (0.0008) [2023-12-27 11:05:11,729][20347] Updated weights for policy 0, policy_version 788865 (0.0011) [2023-12-27 11:05:11,758][20381] Updated weights for policy 1, policy_version 788372 (0.0008) [2023-12-27 11:05:11,793][20347] Updated weights for policy 0, policy_version 788875 (0.0009) [2023-12-27 11:05:11,858][20347] Updated weights for policy 0, policy_version 788885 (0.0009) [2023-12-27 11:05:11,919][20347] Updated weights for policy 0, policy_version 788895 (0.0009) [2023-12-27 11:05:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 403832832. Throughput: 0: 10451.9, 1: 10136.2. Samples: 403828004. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:12,363][19170] Avg episode reward: [(0, '737.570'), (1, '913.290')] [2023-12-27 11:05:12,464][20381] Updated weights for policy 1, policy_version 788382 (0.0006) [2023-12-27 11:05:12,522][20381] Updated weights for policy 1, policy_version 788392 (0.0008) [2023-12-27 11:05:12,580][20381] Updated weights for policy 1, policy_version 788402 (0.0008) [2023-12-27 11:05:12,774][20347] Updated weights for policy 0, policy_version 788905 (0.0008) [2023-12-27 11:05:12,836][20347] Updated weights for policy 0, policy_version 788915 (0.0008) [2023-12-27 11:05:12,894][20347] Updated weights for policy 0, policy_version 788925 (0.0007) [2023-12-27 11:05:13,307][20381] Updated weights for policy 1, policy_version 788412 (0.0007) [2023-12-27 11:05:13,369][20381] Updated weights for policy 1, policy_version 788422 (0.0005) [2023-12-27 11:05:13,429][20381] Updated weights for policy 1, policy_version 788432 (0.0006) [2023-12-27 11:05:13,558][20347] Updated weights for policy 0, policy_version 788935 (0.0010) [2023-12-27 11:05:13,614][20347] Updated weights for policy 0, policy_version 788945 (0.0011) [2023-12-27 11:05:13,672][20347] Updated weights for policy 0, policy_version 788955 (0.0009) [2023-12-27 11:05:14,103][20381] Updated weights for policy 1, policy_version 788442 (0.0008) [2023-12-27 11:05:14,161][20381] Updated weights for policy 1, policy_version 788452 (0.0007) [2023-12-27 11:05:14,220][20381] Updated weights for policy 1, policy_version 788462 (0.0007) [2023-12-27 11:05:14,268][20381] Updated weights for policy 1, policy_version 788472 (0.0007) [2023-12-27 11:05:14,331][20347] Updated weights for policy 0, policy_version 788965 (0.0009) [2023-12-27 11:05:14,375][20347] Updated weights for policy 0, policy_version 788975 (0.0007) [2023-12-27 11:05:14,435][20347] Updated weights for policy 0, policy_version 788985 (0.0007) [2023-12-27 11:05:14,875][20381] Updated weights for policy 1, policy_version 788482 (0.0007) [2023-12-27 11:05:14,925][20381] Updated weights for policy 1, policy_version 788492 (0.0008) [2023-12-27 11:05:14,977][20381] Updated weights for policy 1, policy_version 788502 (0.0007) [2023-12-27 11:05:15,132][20347] Updated weights for policy 0, policy_version 788995 (0.0007) [2023-12-27 11:05:15,186][20347] Updated weights for policy 0, policy_version 789005 (0.0008) [2023-12-27 11:05:15,247][20347] Updated weights for policy 0, policy_version 789015 (0.0008) [2023-12-27 11:05:15,708][20381] Updated weights for policy 1, policy_version 788512 (0.0008) [2023-12-27 11:05:15,762][20381] Updated weights for policy 1, policy_version 788522 (0.0009) [2023-12-27 11:05:15,826][20381] Updated weights for policy 1, policy_version 788532 (0.0008) [2023-12-27 11:05:15,867][20347] Updated weights for policy 0, policy_version 789025 (0.0007) [2023-12-27 11:05:15,929][20347] Updated weights for policy 0, policy_version 789035 (0.0006) [2023-12-27 11:05:15,993][20347] Updated weights for policy 0, policy_version 789045 (0.0006) [2023-12-27 11:05:16,053][20347] Updated weights for policy 0, policy_version 789055 (0.0008) [2023-12-27 11:05:16,495][20381] Updated weights for policy 1, policy_version 788542 (0.0007) [2023-12-27 11:05:16,547][20381] Updated weights for policy 1, policy_version 788552 (0.0007) [2023-12-27 11:05:16,593][20381] Updated weights for policy 1, policy_version 788562 (0.0008) [2023-12-27 11:05:16,672][20347] Updated weights for policy 0, policy_version 789065 (0.0007) [2023-12-27 11:05:16,729][20347] Updated weights for policy 0, policy_version 789075 (0.0006) [2023-12-27 11:05:16,787][20347] Updated weights for policy 0, policy_version 789085 (0.0007) [2023-12-27 11:05:17,237][20381] Updated weights for policy 1, policy_version 788572 (0.0008) [2023-12-27 11:05:17,298][20381] Updated weights for policy 1, policy_version 788582 (0.0008) [2023-12-27 11:05:17,362][20381] Updated weights for policy 1, policy_version 788592 (0.0008) [2023-12-27 11:05:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 403931136. Throughput: 0: 10339.0, 1: 10158.6. Samples: 403947896. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:17,363][19170] Avg episode reward: [(0, '835.370'), (1, '961.080')] [2023-12-27 11:05:17,447][20347] Updated weights for policy 0, policy_version 789095 (0.0008) [2023-12-27 11:05:17,496][20347] Updated weights for policy 0, policy_version 789105 (0.0008) [2023-12-27 11:05:17,556][20347] Updated weights for policy 0, policy_version 789115 (0.0007) [2023-12-27 11:05:18,042][20381] Updated weights for policy 1, policy_version 788602 (0.0008) [2023-12-27 11:05:18,094][20381] Updated weights for policy 1, policy_version 788612 (0.0009) [2023-12-27 11:05:18,146][20381] Updated weights for policy 1, policy_version 788622 (0.0006) [2023-12-27 11:05:18,168][20347] Updated weights for policy 0, policy_version 789125 (0.0006) [2023-12-27 11:05:18,198][20381] Updated weights for policy 1, policy_version 788632 (0.0005) [2023-12-27 11:05:18,228][20347] Updated weights for policy 0, policy_version 789135 (0.0006) [2023-12-27 11:05:18,282][20347] Updated weights for policy 0, policy_version 789145 (0.0009) [2023-12-27 11:05:18,928][20381] Updated weights for policy 1, policy_version 788642 (0.0008) [2023-12-27 11:05:18,938][20347] Updated weights for policy 0, policy_version 789155 (0.0007) [2023-12-27 11:05:18,974][20381] Updated weights for policy 1, policy_version 788652 (0.0006) [2023-12-27 11:05:18,994][20347] Updated weights for policy 0, policy_version 789165 (0.0010) [2023-12-27 11:05:19,025][20381] Updated weights for policy 1, policy_version 788662 (0.0005) [2023-12-27 11:05:19,045][20347] Updated weights for policy 0, policy_version 789175 (0.0007) [2023-12-27 11:05:19,740][20347] Updated weights for policy 0, policy_version 789185 (0.0008) [2023-12-27 11:05:19,766][20381] Updated weights for policy 1, policy_version 788672 (0.0006) [2023-12-27 11:05:19,806][20347] Updated weights for policy 0, policy_version 789195 (0.0008) [2023-12-27 11:05:19,825][20381] Updated weights for policy 1, policy_version 788682 (0.0007) [2023-12-27 11:05:19,872][20347] Updated weights for policy 0, policy_version 789205 (0.0008) [2023-12-27 11:05:19,883][20381] Updated weights for policy 1, policy_version 788692 (0.0007) [2023-12-27 11:05:19,931][20347] Updated weights for policy 0, policy_version 789215 (0.0008) [2023-12-27 11:05:20,606][20381] Updated weights for policy 1, policy_version 788702 (0.0007) [2023-12-27 11:05:20,635][20347] Updated weights for policy 0, policy_version 789225 (0.0008) [2023-12-27 11:05:20,662][20381] Updated weights for policy 1, policy_version 788712 (0.0008) [2023-12-27 11:05:20,691][20347] Updated weights for policy 0, policy_version 789235 (0.0008) [2023-12-27 11:05:20,720][20381] Updated weights for policy 1, policy_version 788722 (0.0006) [2023-12-27 11:05:20,750][20347] Updated weights for policy 0, policy_version 789245 (0.0008) [2023-12-27 11:05:21,364][20381] Updated weights for policy 1, policy_version 788732 (0.0009) [2023-12-27 11:05:21,436][20381] Updated weights for policy 1, policy_version 788742 (0.0009) [2023-12-27 11:05:21,503][20381] Updated weights for policy 1, policy_version 788752 (0.0008) [2023-12-27 11:05:21,506][20347] Updated weights for policy 0, policy_version 789255 (0.0007) [2023-12-27 11:05:21,559][20347] Updated weights for policy 0, policy_version 789265 (0.0006) [2023-12-27 11:05:21,615][20347] Updated weights for policy 0, policy_version 789275 (0.0006) [2023-12-27 11:05:22,263][20381] Updated weights for policy 1, policy_version 788762 (0.0008) [2023-12-27 11:05:22,314][20347] Updated weights for policy 0, policy_version 789285 (0.0008) [2023-12-27 11:05:22,327][20381] Updated weights for policy 1, policy_version 788772 (0.0007) [2023-12-27 11:05:22,329][20292] KL-divergence is very high: 206.5446 [2023-12-27 11:05:22,335][20292] KL-divergence is very high: 212.2411 [2023-12-27 11:05:22,361][20292] KL-divergence is very high: 107.2714 [2023-12-27 11:05:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 404029440. Throughput: 0: 10327.4, 1: 10148.4. Samples: 404008192. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:22,363][19170] Avg episode reward: [(0, '864.200'), (1, '920.990')] [2023-12-27 11:05:22,384][20347] Updated weights for policy 0, policy_version 789295 (0.0008) [2023-12-27 11:05:22,395][20381] Updated weights for policy 1, policy_version 788782 (0.0009) [2023-12-27 11:05:22,442][20347] Updated weights for policy 0, policy_version 789305 (0.0006) [2023-12-27 11:05:22,451][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000788792_201957376.pth... [2023-12-27 11:05:22,451][20381] Updated weights for policy 1, policy_version 788792 (0.0006) [2023-12-27 11:05:22,455][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000787576_201646080.pth [2023-12-27 11:05:22,482][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000789312_202088448.pth... [2023-12-27 11:05:22,486][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000788096_201777152.pth [2023-12-27 11:05:23,113][20347] Updated weights for policy 0, policy_version 789315 (0.0007) [2023-12-27 11:05:23,167][20381] Updated weights for policy 1, policy_version 788802 (0.0006) [2023-12-27 11:05:23,170][20347] Updated weights for policy 0, policy_version 789325 (0.0006) [2023-12-27 11:05:23,218][20381] Updated weights for policy 1, policy_version 788812 (0.0007) [2023-12-27 11:05:23,219][20347] Updated weights for policy 0, policy_version 789335 (0.0006) [2023-12-27 11:05:23,270][20381] Updated weights for policy 1, policy_version 788822 (0.0005) [2023-12-27 11:05:23,919][20347] Updated weights for policy 0, policy_version 789345 (0.0006) [2023-12-27 11:05:23,951][20381] Updated weights for policy 1, policy_version 788832 (0.0008) [2023-12-27 11:05:23,979][20347] Updated weights for policy 0, policy_version 789355 (0.0010) [2023-12-27 11:05:24,010][20381] Updated weights for policy 1, policy_version 788842 (0.0006) [2023-12-27 11:05:24,043][20347] Updated weights for policy 0, policy_version 789365 (0.0010) [2023-12-27 11:05:24,057][20381] Updated weights for policy 1, policy_version 788852 (0.0008) [2023-12-27 11:05:24,088][20347] Updated weights for policy 0, policy_version 789375 (0.0010) [2023-12-27 11:05:24,721][20381] Updated weights for policy 1, policy_version 788862 (0.0008) [2023-12-27 11:05:24,757][20347] Updated weights for policy 0, policy_version 789385 (0.0007) [2023-12-27 11:05:24,771][20381] Updated weights for policy 1, policy_version 788872 (0.0005) [2023-12-27 11:05:24,807][20347] Updated weights for policy 0, policy_version 789395 (0.0007) [2023-12-27 11:05:24,824][20381] Updated weights for policy 1, policy_version 788882 (0.0006) [2023-12-27 11:05:24,859][20347] Updated weights for policy 0, policy_version 789405 (0.0007) [2023-12-27 11:05:25,489][20347] Updated weights for policy 0, policy_version 789415 (0.0005) [2023-12-27 11:05:25,504][20381] Updated weights for policy 1, policy_version 788892 (0.0007) [2023-12-27 11:05:25,543][20347] Updated weights for policy 0, policy_version 789425 (0.0005) [2023-12-27 11:05:25,560][20381] Updated weights for policy 1, policy_version 788902 (0.0010) [2023-12-27 11:05:25,600][20347] Updated weights for policy 0, policy_version 789435 (0.0005) [2023-12-27 11:05:25,615][20381] Updated weights for policy 1, policy_version 788912 (0.0010) [2023-12-27 11:05:26,113][20347] Updated weights for policy 0, policy_version 789445 (0.0006) [2023-12-27 11:05:26,161][20347] Updated weights for policy 0, policy_version 789455 (0.0007) [2023-12-27 11:05:26,215][20347] Updated weights for policy 0, policy_version 789465 (0.0007) [2023-12-27 11:05:26,261][20381] Updated weights for policy 1, policy_version 788922 (0.0010) [2023-12-27 11:05:26,312][20381] Updated weights for policy 1, policy_version 788932 (0.0005) [2023-12-27 11:05:26,357][20381] Updated weights for policy 1, policy_version 788942 (0.0010) [2023-12-27 11:05:26,402][20381] Updated weights for policy 1, policy_version 788952 (0.0010) [2023-12-27 11:05:26,897][20347] Updated weights for policy 0, policy_version 789475 (0.0007) [2023-12-27 11:05:26,950][20347] Updated weights for policy 0, policy_version 789485 (0.0007) [2023-12-27 11:05:27,009][20347] Updated weights for policy 0, policy_version 789495 (0.0007) [2023-12-27 11:05:27,041][20381] Updated weights for policy 1, policy_version 788962 (0.0010) [2023-12-27 11:05:27,095][20381] Updated weights for policy 1, policy_version 788972 (0.0010) [2023-12-27 11:05:27,154][20381] Updated weights for policy 1, policy_version 788982 (0.0011) [2023-12-27 11:05:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 404144128. Throughput: 0: 10348.7, 1: 10168.2. Samples: 404130944. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:27,363][19170] Avg episode reward: [(0, '951.380'), (1, '906.920')] [2023-12-27 11:05:27,708][20347] Updated weights for policy 0, policy_version 789505 (0.0005) [2023-12-27 11:05:27,765][20347] Updated weights for policy 0, policy_version 789515 (0.0007) [2023-12-27 11:05:27,818][20347] Updated weights for policy 0, policy_version 789525 (0.0006) [2023-12-27 11:05:27,840][20381] Updated weights for policy 1, policy_version 788992 (0.0010) [2023-12-27 11:05:27,876][20347] Updated weights for policy 0, policy_version 789535 (0.0005) [2023-12-27 11:05:27,894][20381] Updated weights for policy 1, policy_version 789002 (0.0010) [2023-12-27 11:05:27,942][20381] Updated weights for policy 1, policy_version 789012 (0.0010) [2023-12-27 11:05:28,461][20347] Updated weights for policy 0, policy_version 789545 (0.0007) [2023-12-27 11:05:28,520][20347] Updated weights for policy 0, policy_version 789555 (0.0007) [2023-12-27 11:05:28,583][20347] Updated weights for policy 0, policy_version 789565 (0.0007) [2023-12-27 11:05:28,615][20381] Updated weights for policy 1, policy_version 789022 (0.0010) [2023-12-27 11:05:28,670][20381] Updated weights for policy 1, policy_version 789032 (0.0010) [2023-12-27 11:05:28,730][20381] Updated weights for policy 1, policy_version 789042 (0.0010) [2023-12-27 11:05:29,265][20347] Updated weights for policy 0, policy_version 789575 (0.0009) [2023-12-27 11:05:29,324][20347] Updated weights for policy 0, policy_version 789585 (0.0011) [2023-12-27 11:05:29,391][20347] Updated weights for policy 0, policy_version 789595 (0.0008) [2023-12-27 11:05:29,408][20381] Updated weights for policy 1, policy_version 789052 (0.0009) [2023-12-27 11:05:29,467][20381] Updated weights for policy 1, policy_version 789062 (0.0008) [2023-12-27 11:05:29,530][20381] Updated weights for policy 1, policy_version 789072 (0.0008) [2023-12-27 11:05:29,983][20347] Updated weights for policy 0, policy_version 789605 (0.0006) [2023-12-27 11:05:30,049][20347] Updated weights for policy 0, policy_version 789615 (0.0006) [2023-12-27 11:05:30,104][20347] Updated weights for policy 0, policy_version 789625 (0.0007) [2023-12-27 11:05:30,220][20381] Updated weights for policy 1, policy_version 789082 (0.0007) [2023-12-27 11:05:30,281][20381] Updated weights for policy 1, policy_version 789092 (0.0008) [2023-12-27 11:05:30,342][20381] Updated weights for policy 1, policy_version 789102 (0.0009) [2023-12-27 11:05:30,406][20381] Updated weights for policy 1, policy_version 789112 (0.0007) [2023-12-27 11:05:30,728][20347] Updated weights for policy 0, policy_version 789635 (0.0007) [2023-12-27 11:05:30,778][20347] Updated weights for policy 0, policy_version 789645 (0.0005) [2023-12-27 11:05:30,824][20347] Updated weights for policy 0, policy_version 789655 (0.0005) [2023-12-27 11:05:31,050][20381] Updated weights for policy 1, policy_version 789122 (0.0007) [2023-12-27 11:05:31,109][20381] Updated weights for policy 1, policy_version 789132 (0.0007) [2023-12-27 11:05:31,171][20381] Updated weights for policy 1, policy_version 789142 (0.0009) [2023-12-27 11:05:31,433][20347] Updated weights for policy 0, policy_version 789665 (0.0005) [2023-12-27 11:05:31,496][20347] Updated weights for policy 0, policy_version 789675 (0.0005) [2023-12-27 11:05:31,557][20347] Updated weights for policy 0, policy_version 789685 (0.0006) [2023-12-27 11:05:31,616][20347] Updated weights for policy 0, policy_version 789695 (0.0007) [2023-12-27 11:05:31,935][20381] Updated weights for policy 1, policy_version 789152 (0.0008) [2023-12-27 11:05:32,002][20381] Updated weights for policy 1, policy_version 789162 (0.0006) [2023-12-27 11:05:32,051][20381] Updated weights for policy 1, policy_version 789172 (0.0005) [2023-12-27 11:05:32,231][20347] Updated weights for policy 0, policy_version 789705 (0.0008) [2023-12-27 11:05:32,297][20347] Updated weights for policy 0, policy_version 789715 (0.0008) [2023-12-27 11:05:32,357][20347] Updated weights for policy 0, policy_version 789725 (0.0007) [2023-12-27 11:05:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 404242432. Throughput: 0: 10351.2, 1: 10222.0. Samples: 404256612. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:32,363][19170] Avg episode reward: [(0, '921.910'), (1, '956.530')] [2023-12-27 11:05:32,688][20381] Updated weights for policy 1, policy_version 789182 (0.0007) [2023-12-27 11:05:32,741][20381] Updated weights for policy 1, policy_version 789192 (0.0008) [2023-12-27 11:05:32,797][20381] Updated weights for policy 1, policy_version 789202 (0.0008) [2023-12-27 11:05:33,037][20347] Updated weights for policy 0, policy_version 789735 (0.0008) [2023-12-27 11:05:33,085][20347] Updated weights for policy 0, policy_version 789745 (0.0008) [2023-12-27 11:05:33,142][20347] Updated weights for policy 0, policy_version 789755 (0.0008) [2023-12-27 11:05:33,386][20381] Updated weights for policy 1, policy_version 789212 (0.0008) [2023-12-27 11:05:33,432][20381] Updated weights for policy 1, policy_version 789222 (0.0005) [2023-12-27 11:05:33,476][20381] Updated weights for policy 1, policy_version 789232 (0.0006) [2023-12-27 11:05:33,878][20347] Updated weights for policy 0, policy_version 789765 (0.0007) [2023-12-27 11:05:33,931][20347] Updated weights for policy 0, policy_version 789775 (0.0007) [2023-12-27 11:05:33,982][20347] Updated weights for policy 0, policy_version 789785 (0.0009) [2023-12-27 11:05:34,180][20381] Updated weights for policy 1, policy_version 789242 (0.0008) [2023-12-27 11:05:34,239][20381] Updated weights for policy 1, policy_version 789252 (0.0011) [2023-12-27 11:05:34,300][20381] Updated weights for policy 1, policy_version 789262 (0.0010) [2023-12-27 11:05:34,358][20381] Updated weights for policy 1, policy_version 789272 (0.0011) [2023-12-27 11:05:34,682][20347] Updated weights for policy 0, policy_version 789795 (0.0010) [2023-12-27 11:05:34,739][20347] Updated weights for policy 0, policy_version 789805 (0.0011) [2023-12-27 11:05:34,792][20347] Updated weights for policy 0, policy_version 789815 (0.0008) [2023-12-27 11:05:35,053][20381] Updated weights for policy 1, policy_version 789282 (0.0005) [2023-12-27 11:05:35,101][20381] Updated weights for policy 1, policy_version 789292 (0.0005) [2023-12-27 11:05:35,150][20381] Updated weights for policy 1, policy_version 789302 (0.0005) [2023-12-27 11:05:35,512][20347] Updated weights for policy 0, policy_version 789825 (0.0007) [2023-12-27 11:05:35,565][20347] Updated weights for policy 0, policy_version 789835 (0.0007) [2023-12-27 11:05:35,626][20347] Updated weights for policy 0, policy_version 789845 (0.0007) [2023-12-27 11:05:35,680][20347] Updated weights for policy 0, policy_version 789855 (0.0007) [2023-12-27 11:05:35,786][20381] Updated weights for policy 1, policy_version 789312 (0.0009) [2023-12-27 11:05:35,842][20381] Updated weights for policy 1, policy_version 789322 (0.0010) [2023-12-27 11:05:35,901][20381] Updated weights for policy 1, policy_version 789332 (0.0010) [2023-12-27 11:05:36,351][20347] Updated weights for policy 0, policy_version 789865 (0.0008) [2023-12-27 11:05:36,406][20347] Updated weights for policy 0, policy_version 789875 (0.0007) [2023-12-27 11:05:36,462][20347] Updated weights for policy 0, policy_version 789885 (0.0008) [2023-12-27 11:05:36,615][20381] Updated weights for policy 1, policy_version 789342 (0.0008) [2023-12-27 11:05:36,675][20381] Updated weights for policy 1, policy_version 789352 (0.0007) [2023-12-27 11:05:36,733][20381] Updated weights for policy 1, policy_version 789362 (0.0008) [2023-12-27 11:05:37,221][20347] Updated weights for policy 0, policy_version 789895 (0.0007) [2023-12-27 11:05:37,276][20347] Updated weights for policy 0, policy_version 789905 (0.0006) [2023-12-27 11:05:37,282][20381] Updated weights for policy 1, policy_version 789372 (0.0009) [2023-12-27 11:05:37,337][20347] Updated weights for policy 0, policy_version 789915 (0.0005) [2023-12-27 11:05:37,338][20381] Updated weights for policy 1, policy_version 789382 (0.0011) [2023-12-27 11:05:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 404348928. Throughput: 0: 10307.1, 1: 10218.0. Samples: 404317528. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:37,363][19170] Avg episode reward: [(0, '902.500'), (1, '936.880')] [2023-12-27 11:05:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000789920_202244096.pth... [2023-12-27 11:05:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000788704_201932800.pth [2023-12-27 11:05:37,395][20381] Updated weights for policy 1, policy_version 789392 (0.0011) [2023-12-27 11:05:37,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000789400_202113024.pth... [2023-12-27 11:05:37,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000788184_201801728.pth [2023-12-27 11:05:38,059][20347] Updated weights for policy 0, policy_version 789925 (0.0006) [2023-12-27 11:05:38,064][20381] Updated weights for policy 1, policy_version 789402 (0.0011) [2023-12-27 11:05:38,114][20347] Updated weights for policy 0, policy_version 789935 (0.0006) [2023-12-27 11:05:38,122][20381] Updated weights for policy 1, policy_version 789412 (0.0010) [2023-12-27 11:05:38,160][20347] Updated weights for policy 0, policy_version 789945 (0.0007) [2023-12-27 11:05:38,173][20381] Updated weights for policy 1, policy_version 789422 (0.0010) [2023-12-27 11:05:38,212][20292] KL-divergence is very high: 112.1874 [2023-12-27 11:05:38,228][20381] Updated weights for policy 1, policy_version 789432 (0.0010) [2023-12-27 11:05:38,901][20347] Updated weights for policy 0, policy_version 789955 (0.0006) [2023-12-27 11:05:38,915][20381] Updated weights for policy 1, policy_version 789442 (0.0007) [2023-12-27 11:05:38,965][20347] Updated weights for policy 0, policy_version 789965 (0.0008) [2023-12-27 11:05:38,972][20381] Updated weights for policy 1, policy_version 789452 (0.0005) [2023-12-27 11:05:39,015][20347] Updated weights for policy 0, policy_version 789975 (0.0006) [2023-12-27 11:05:39,024][20381] Updated weights for policy 1, policy_version 789462 (0.0006) [2023-12-27 11:05:39,710][20381] Updated weights for policy 1, policy_version 789472 (0.0006) [2023-12-27 11:05:39,772][20381] Updated weights for policy 1, policy_version 789482 (0.0006) [2023-12-27 11:05:39,792][20347] Updated weights for policy 0, policy_version 789985 (0.0006) [2023-12-27 11:05:39,843][20381] Updated weights for policy 1, policy_version 789492 (0.0007) [2023-12-27 11:05:39,863][20347] Updated weights for policy 0, policy_version 789995 (0.0008) [2023-12-27 11:05:39,924][20347] Updated weights for policy 0, policy_version 790005 (0.0008) [2023-12-27 11:05:39,979][20347] Updated weights for policy 0, policy_version 790015 (0.0010) [2023-12-27 11:05:40,452][20381] Updated weights for policy 1, policy_version 789502 (0.0006) [2023-12-27 11:05:40,505][20381] Updated weights for policy 1, policy_version 789512 (0.0008) [2023-12-27 11:05:40,566][20381] Updated weights for policy 1, policy_version 789522 (0.0007) [2023-12-27 11:05:40,586][20347] Updated weights for policy 0, policy_version 790025 (0.0010) [2023-12-27 11:05:40,633][20347] Updated weights for policy 0, policy_version 790035 (0.0009) [2023-12-27 11:05:40,683][20347] Updated weights for policy 0, policy_version 790045 (0.0006) [2023-12-27 11:05:41,268][20381] Updated weights for policy 1, policy_version 789532 (0.0007) [2023-12-27 11:05:41,324][20381] Updated weights for policy 1, policy_version 789542 (0.0007) [2023-12-27 11:05:41,347][20347] Updated weights for policy 0, policy_version 790055 (0.0007) [2023-12-27 11:05:41,392][20381] Updated weights for policy 1, policy_version 789552 (0.0008) [2023-12-27 11:05:41,416][20347] Updated weights for policy 0, policy_version 790065 (0.0008) [2023-12-27 11:05:41,476][20347] Updated weights for policy 0, policy_version 790075 (0.0008) [2023-12-27 11:05:42,083][20381] Updated weights for policy 1, policy_version 789562 (0.0007) [2023-12-27 11:05:42,152][20381] Updated weights for policy 1, policy_version 789572 (0.0008) [2023-12-27 11:05:42,209][20347] Updated weights for policy 0, policy_version 790085 (0.0008) [2023-12-27 11:05:42,212][20381] Updated weights for policy 1, policy_version 789582 (0.0007) [2023-12-27 11:05:42,269][20381] Updated weights for policy 1, policy_version 789592 (0.0006) [2023-12-27 11:05:42,271][20347] Updated weights for policy 0, policy_version 790095 (0.0006) [2023-12-27 11:05:42,341][20347] Updated weights for policy 0, policy_version 790105 (0.0009) [2023-12-27 11:05:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 404447232. Throughput: 0: 10278.2, 1: 10204.3. Samples: 404440124. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:42,363][19170] Avg episode reward: [(0, '912.140'), (1, '932.150')] [2023-12-27 11:05:42,926][20381] Updated weights for policy 1, policy_version 789602 (0.0007) [2023-12-27 11:05:42,988][20381] Updated weights for policy 1, policy_version 789612 (0.0008) [2023-12-27 11:05:43,014][20347] Updated weights for policy 0, policy_version 790115 (0.0006) [2023-12-27 11:05:43,039][20381] Updated weights for policy 1, policy_version 789622 (0.0007) [2023-12-27 11:05:43,074][20347] Updated weights for policy 0, policy_version 790125 (0.0006) [2023-12-27 11:05:43,134][20347] Updated weights for policy 0, policy_version 790135 (0.0005) [2023-12-27 11:05:43,733][20381] Updated weights for policy 1, policy_version 789632 (0.0007) [2023-12-27 11:05:43,781][20347] Updated weights for policy 0, policy_version 790145 (0.0007) [2023-12-27 11:05:43,794][20381] Updated weights for policy 1, policy_version 789642 (0.0007) [2023-12-27 11:05:43,840][20347] Updated weights for policy 0, policy_version 790155 (0.0010) [2023-12-27 11:05:43,849][20381] Updated weights for policy 1, policy_version 789652 (0.0005) [2023-12-27 11:05:43,894][20347] Updated weights for policy 0, policy_version 790165 (0.0010) [2023-12-27 11:05:43,953][20347] Updated weights for policy 0, policy_version 790175 (0.0010) [2023-12-27 11:05:44,513][20381] Updated weights for policy 1, policy_version 789662 (0.0007) [2023-12-27 11:05:44,555][20292] KL-divergence is very high: 328.6802 [2023-12-27 11:05:44,569][20381] Updated weights for policy 1, policy_version 789672 (0.0006) [2023-12-27 11:05:44,574][20292] KL-divergence is very high: 146.4466 [2023-12-27 11:05:44,594][20292] KL-divergence is very high: 135.9082 [2023-12-27 11:05:44,596][20347] Updated weights for policy 0, policy_version 790185 (0.0010) [2023-12-27 11:05:44,620][20381] Updated weights for policy 1, policy_version 789682 (0.0007) [2023-12-27 11:05:44,650][20347] Updated weights for policy 0, policy_version 790195 (0.0010) [2023-12-27 11:05:44,698][20347] Updated weights for policy 0, policy_version 790205 (0.0010) [2023-12-27 11:05:45,317][20292] KL-divergence is very high: 156.2821 [2023-12-27 11:05:45,322][20381] Updated weights for policy 1, policy_version 789692 (0.0007) [2023-12-27 11:05:45,329][20292] KL-divergence is very high: 200.0682 [2023-12-27 11:05:45,365][20292] KL-divergence is very high: 272.3423 [2023-12-27 11:05:45,371][20347] Updated weights for policy 0, policy_version 790215 (0.0007) [2023-12-27 11:05:45,384][20381] Updated weights for policy 1, policy_version 789702 (0.0010) [2023-12-27 11:05:45,420][20292] KL-divergence is very high: 108.2487 [2023-12-27 11:05:45,423][20347] Updated weights for policy 0, policy_version 790225 (0.0007) [2023-12-27 11:05:45,434][20381] Updated weights for policy 1, policy_version 789712 (0.0010) [2023-12-27 11:05:45,473][20347] Updated weights for policy 0, policy_version 790235 (0.0006) [2023-12-27 11:05:46,101][20381] Updated weights for policy 1, policy_version 789722 (0.0010) [2023-12-27 11:05:46,139][20347] Updated weights for policy 0, policy_version 790245 (0.0007) [2023-12-27 11:05:46,152][20381] Updated weights for policy 1, policy_version 789732 (0.0010) [2023-12-27 11:05:46,188][20347] Updated weights for policy 0, policy_version 790255 (0.0005) [2023-12-27 11:05:46,201][20381] Updated weights for policy 1, policy_version 789742 (0.0010) [2023-12-27 11:05:46,244][20347] Updated weights for policy 0, policy_version 790265 (0.0006) [2023-12-27 11:05:46,263][20381] Updated weights for policy 1, policy_version 789752 (0.0010) [2023-12-27 11:05:46,870][20381] Updated weights for policy 1, policy_version 789762 (0.0006) [2023-12-27 11:05:46,876][20292] KL-divergence is very high: 571.6923 [2023-12-27 11:05:46,890][20347] Updated weights for policy 0, policy_version 790275 (0.0005) [2023-12-27 11:05:46,919][20292] KL-divergence is very high: 221.9716 [2023-12-27 11:05:46,924][20381] Updated weights for policy 1, policy_version 789772 (0.0007) [2023-12-27 11:05:46,952][20347] Updated weights for policy 0, policy_version 790285 (0.0007) [2023-12-27 11:05:46,981][20381] Updated weights for policy 1, policy_version 789782 (0.0005) [2023-12-27 11:05:47,008][20347] Updated weights for policy 0, policy_version 790295 (0.0007) [2023-12-27 11:05:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 404553728. Throughput: 0: 10241.2, 1: 10234.7. Samples: 404563048. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:47,363][19170] Avg episode reward: [(0, '883.030'), (1, '913.210')] [2023-12-27 11:05:47,625][20381] Updated weights for policy 1, policy_version 789792 (0.0005) [2023-12-27 11:05:47,642][20347] Updated weights for policy 0, policy_version 790305 (0.0007) [2023-12-27 11:05:47,688][20381] Updated weights for policy 1, policy_version 789802 (0.0005) [2023-12-27 11:05:47,697][20347] Updated weights for policy 0, policy_version 790315 (0.0007) [2023-12-27 11:05:47,748][20381] Updated weights for policy 1, policy_version 789812 (0.0008) [2023-12-27 11:05:47,753][20347] Updated weights for policy 0, policy_version 790325 (0.0007) [2023-12-27 11:05:47,816][20347] Updated weights for policy 0, policy_version 790335 (0.0007) [2023-12-27 11:05:48,384][20381] Updated weights for policy 1, policy_version 789822 (0.0011) [2023-12-27 11:05:48,441][20381] Updated weights for policy 1, policy_version 789832 (0.0006) [2023-12-27 11:05:48,448][20347] Updated weights for policy 0, policy_version 790345 (0.0008) [2023-12-27 11:05:48,496][20381] Updated weights for policy 1, policy_version 789842 (0.0005) [2023-12-27 11:05:48,506][20347] Updated weights for policy 0, policy_version 790355 (0.0008) [2023-12-27 11:05:48,564][20347] Updated weights for policy 0, policy_version 790365 (0.0008) [2023-12-27 11:05:49,117][20381] Updated weights for policy 1, policy_version 789852 (0.0006) [2023-12-27 11:05:49,168][20381] Updated weights for policy 1, policy_version 789862 (0.0006) [2023-12-27 11:05:49,232][20381] Updated weights for policy 1, policy_version 789872 (0.0007) [2023-12-27 11:05:49,351][20347] Updated weights for policy 0, policy_version 790375 (0.0008) [2023-12-27 11:05:49,416][20347] Updated weights for policy 0, policy_version 790385 (0.0008) [2023-12-27 11:05:49,472][20291] KL-divergence is very high: 233.3199 [2023-12-27 11:05:49,478][20347] Updated weights for policy 0, policy_version 790395 (0.0008) [2023-12-27 11:05:49,485][20291] KL-divergence is very high: 254.8822 [2023-12-27 11:05:49,843][20381] Updated weights for policy 1, policy_version 789882 (0.0006) [2023-12-27 11:05:49,903][20381] Updated weights for policy 1, policy_version 789892 (0.0008) [2023-12-27 11:05:49,962][20381] Updated weights for policy 1, policy_version 789902 (0.0009) [2023-12-27 11:05:50,020][20381] Updated weights for policy 1, policy_version 789912 (0.0007) [2023-12-27 11:05:50,194][20347] Updated weights for policy 0, policy_version 790405 (0.0008) [2023-12-27 11:05:50,244][20347] Updated weights for policy 0, policy_version 790415 (0.0008) [2023-12-27 11:05:50,298][20347] Updated weights for policy 0, policy_version 790425 (0.0008) [2023-12-27 11:05:50,766][20381] Updated weights for policy 1, policy_version 789922 (0.0010) [2023-12-27 11:05:50,816][20381] Updated weights for policy 1, policy_version 789932 (0.0011) [2023-12-27 11:05:50,866][20381] Updated weights for policy 1, policy_version 789942 (0.0011) [2023-12-27 11:05:50,962][20347] Updated weights for policy 0, policy_version 790435 (0.0007) [2023-12-27 11:05:51,016][20347] Updated weights for policy 0, policy_version 790445 (0.0006) [2023-12-27 11:05:51,076][20347] Updated weights for policy 0, policy_version 790455 (0.0008) [2023-12-27 11:05:51,564][20381] Updated weights for policy 1, policy_version 789952 (0.0008) [2023-12-27 11:05:51,624][20292] KL-divergence is very high: 303.5712 [2023-12-27 11:05:51,625][20381] Updated weights for policy 1, policy_version 789962 (0.0008) [2023-12-27 11:05:51,672][20292] KL-divergence is very high: 164.4673 [2023-12-27 11:05:51,685][20381] Updated weights for policy 1, policy_version 789972 (0.0007) [2023-12-27 11:05:51,799][20347] Updated weights for policy 0, policy_version 790465 (0.0009) [2023-12-27 11:05:51,863][20347] Updated weights for policy 0, policy_version 790475 (0.0010) [2023-12-27 11:05:51,921][20347] Updated weights for policy 0, policy_version 790485 (0.0010) [2023-12-27 11:05:51,984][20347] Updated weights for policy 0, policy_version 790495 (0.0011) [2023-12-27 11:05:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 404652032. Throughput: 0: 10192.2, 1: 10282.5. Samples: 404624660. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:52,363][19170] Avg episode reward: [(0, '892.670'), (1, '903.550')] [2023-12-27 11:05:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000790496_202391552.pth... [2023-12-27 11:05:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000789312_202088448.pth [2023-12-27 11:05:52,391][20381] Updated weights for policy 1, policy_version 789982 (0.0008) [2023-12-27 11:05:52,446][20381] Updated weights for policy 1, policy_version 789992 (0.0008) [2023-12-27 11:05:52,501][20381] Updated weights for policy 1, policy_version 790002 (0.0007) [2023-12-27 11:05:52,534][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000790008_202268672.pth... [2023-12-27 11:05:52,537][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000788792_201957376.pth [2023-12-27 11:05:52,674][20347] Updated weights for policy 0, policy_version 790505 (0.0007) [2023-12-27 11:05:52,735][20347] Updated weights for policy 0, policy_version 790515 (0.0009) [2023-12-27 11:05:52,791][20347] Updated weights for policy 0, policy_version 790525 (0.0008) [2023-12-27 11:05:53,195][20381] Updated weights for policy 1, policy_version 790012 (0.0008) [2023-12-27 11:05:53,247][20381] Updated weights for policy 1, policy_version 790022 (0.0007) [2023-12-27 11:05:53,293][20381] Updated weights for policy 1, policy_version 790032 (0.0005) [2023-12-27 11:05:53,456][20347] Updated weights for policy 0, policy_version 790535 (0.0008) [2023-12-27 11:05:53,508][20347] Updated weights for policy 0, policy_version 790545 (0.0007) [2023-12-27 11:05:53,556][20347] Updated weights for policy 0, policy_version 790555 (0.0007) [2023-12-27 11:05:53,940][20381] Updated weights for policy 1, policy_version 790042 (0.0005) [2023-12-27 11:05:53,987][20381] Updated weights for policy 1, policy_version 790052 (0.0006) [2023-12-27 11:05:54,035][20381] Updated weights for policy 1, policy_version 790062 (0.0008) [2023-12-27 11:05:54,084][20381] Updated weights for policy 1, policy_version 790072 (0.0008) [2023-12-27 11:05:54,202][20347] Updated weights for policy 0, policy_version 790565 (0.0007) [2023-12-27 11:05:54,257][20347] Updated weights for policy 0, policy_version 790575 (0.0007) [2023-12-27 11:05:54,310][20347] Updated weights for policy 0, policy_version 790585 (0.0007) [2023-12-27 11:05:54,777][20381] Updated weights for policy 1, policy_version 790082 (0.0007) [2023-12-27 11:05:54,827][20381] Updated weights for policy 1, policy_version 790092 (0.0007) [2023-12-27 11:05:54,874][20381] Updated weights for policy 1, policy_version 790102 (0.0008) [2023-12-27 11:05:55,039][20347] Updated weights for policy 0, policy_version 790595 (0.0007) [2023-12-27 11:05:55,096][20347] Updated weights for policy 0, policy_version 790605 (0.0007) [2023-12-27 11:05:55,150][20347] Updated weights for policy 0, policy_version 790615 (0.0007) [2023-12-27 11:05:55,563][20381] Updated weights for policy 1, policy_version 790112 (0.0008) [2023-12-27 11:05:55,621][20381] Updated weights for policy 1, policy_version 790122 (0.0010) [2023-12-27 11:05:55,679][20381] Updated weights for policy 1, policy_version 790132 (0.0007) [2023-12-27 11:05:55,804][20347] Updated weights for policy 0, policy_version 790625 (0.0007) [2023-12-27 11:05:55,860][20347] Updated weights for policy 0, policy_version 790635 (0.0007) [2023-12-27 11:05:55,925][20347] Updated weights for policy 0, policy_version 790645 (0.0008) [2023-12-27 11:05:55,981][20347] Updated weights for policy 0, policy_version 790655 (0.0010) [2023-12-27 11:05:56,313][20381] Updated weights for policy 1, policy_version 790142 (0.0007) [2023-12-27 11:05:56,378][20381] Updated weights for policy 1, policy_version 790152 (0.0005) [2023-12-27 11:05:56,436][20381] Updated weights for policy 1, policy_version 790162 (0.0007) [2023-12-27 11:05:56,620][20347] Updated weights for policy 0, policy_version 790665 (0.0006) [2023-12-27 11:05:56,673][20347] Updated weights for policy 0, policy_version 790675 (0.0006) [2023-12-27 11:05:56,723][20347] Updated weights for policy 0, policy_version 790685 (0.0007) [2023-12-27 11:05:57,115][20381] Updated weights for policy 1, policy_version 790172 (0.0008) [2023-12-27 11:05:57,174][20381] Updated weights for policy 1, policy_version 790182 (0.0008) [2023-12-27 11:05:57,233][20381] Updated weights for policy 1, policy_version 790192 (0.0008) [2023-12-27 11:05:57,331][20347] Updated weights for policy 0, policy_version 790695 (0.0008) [2023-12-27 11:05:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 404758528. Throughput: 0: 10182.4, 1: 10243.8. Samples: 404747184. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:05:57,363][19170] Avg episode reward: [(0, '931.700'), (1, '932.120')] [2023-12-27 11:05:57,383][20347] Updated weights for policy 0, policy_version 790705 (0.0008) [2023-12-27 11:05:57,441][20347] Updated weights for policy 0, policy_version 790715 (0.0010) [2023-12-27 11:05:57,864][20381] Updated weights for policy 1, policy_version 790202 (0.0007) [2023-12-27 11:05:57,923][20381] Updated weights for policy 1, policy_version 790212 (0.0006) [2023-12-27 11:05:57,974][20381] Updated weights for policy 1, policy_version 790222 (0.0007) [2023-12-27 11:05:58,030][20381] Updated weights for policy 1, policy_version 790232 (0.0008) [2023-12-27 11:05:58,073][20347] Updated weights for policy 0, policy_version 790725 (0.0008) [2023-12-27 11:05:58,125][20347] Updated weights for policy 0, policy_version 790735 (0.0005) [2023-12-27 11:05:58,174][20347] Updated weights for policy 0, policy_version 790745 (0.0010) [2023-12-27 11:05:58,739][20381] Updated weights for policy 1, policy_version 790242 (0.0008) [2023-12-27 11:05:58,805][20381] Updated weights for policy 1, policy_version 790252 (0.0008) [2023-12-27 11:05:58,877][20381] Updated weights for policy 1, policy_version 790262 (0.0008) [2023-12-27 11:05:58,932][20347] Updated weights for policy 0, policy_version 790755 (0.0008) [2023-12-27 11:05:59,009][20347] Updated weights for policy 0, policy_version 790765 (0.0009) [2023-12-27 11:05:59,072][20347] Updated weights for policy 0, policy_version 790775 (0.0007) [2023-12-27 11:05:59,660][20381] Updated weights for policy 1, policy_version 790272 (0.0008) [2023-12-27 11:05:59,724][20381] Updated weights for policy 1, policy_version 790282 (0.0008) [2023-12-27 11:05:59,769][20347] Updated weights for policy 0, policy_version 790785 (0.0007) [2023-12-27 11:05:59,782][20381] Updated weights for policy 1, policy_version 790292 (0.0008) [2023-12-27 11:05:59,836][20347] Updated weights for policy 0, policy_version 790795 (0.0007) [2023-12-27 11:05:59,894][20347] Updated weights for policy 0, policy_version 790805 (0.0008) [2023-12-27 11:05:59,957][20347] Updated weights for policy 0, policy_version 790815 (0.0007) [2023-12-27 11:06:00,545][20381] Updated weights for policy 1, policy_version 790302 (0.0009) [2023-12-27 11:06:00,599][20381] Updated weights for policy 1, policy_version 790312 (0.0010) [2023-12-27 11:06:00,625][20347] Updated weights for policy 0, policy_version 790825 (0.0010) [2023-12-27 11:06:00,647][20381] Updated weights for policy 1, policy_version 790322 (0.0010) [2023-12-27 11:06:00,682][20347] Updated weights for policy 0, policy_version 790835 (0.0010) [2023-12-27 11:06:00,737][20347] Updated weights for policy 0, policy_version 790845 (0.0010) [2023-12-27 11:06:01,325][20381] Updated weights for policy 1, policy_version 790332 (0.0009) [2023-12-27 11:06:01,390][20381] Updated weights for policy 1, policy_version 790342 (0.0007) [2023-12-27 11:06:01,444][20381] Updated weights for policy 1, policy_version 790352 (0.0006) [2023-12-27 11:06:01,457][20347] Updated weights for policy 0, policy_version 790855 (0.0007) [2023-12-27 11:06:01,514][20347] Updated weights for policy 0, policy_version 790865 (0.0006) [2023-12-27 11:06:01,573][20347] Updated weights for policy 0, policy_version 790875 (0.0005) [2023-12-27 11:06:02,195][20381] Updated weights for policy 1, policy_version 790362 (0.0006) [2023-12-27 11:06:02,249][20381] Updated weights for policy 1, policy_version 790372 (0.0008) [2023-12-27 11:06:02,309][20381] Updated weights for policy 1, policy_version 790382 (0.0008) [2023-12-27 11:06:02,309][20347] Updated weights for policy 0, policy_version 790885 (0.0008) [2023-12-27 11:06:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 404848640. Throughput: 0: 10207.5, 1: 10205.6. Samples: 404866488. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:06:02,363][19170] Avg episode reward: [(0, '970.720'), (1, '970.940')] [2023-12-27 11:06:02,375][20347] Updated weights for policy 0, policy_version 790895 (0.0008) [2023-12-27 11:06:02,376][20381] Updated weights for policy 1, policy_version 790392 (0.0008) [2023-12-27 11:06:02,438][20347] Updated weights for policy 0, policy_version 790905 (0.0008) [2023-12-27 11:06:03,020][20381] Updated weights for policy 1, policy_version 790402 (0.0008) [2023-12-27 11:06:03,024][20347] Updated weights for policy 0, policy_version 790915 (0.0006) [2023-12-27 11:06:03,067][20381] Updated weights for policy 1, policy_version 790412 (0.0007) [2023-12-27 11:06:03,072][20347] Updated weights for policy 0, policy_version 790925 (0.0006) [2023-12-27 11:06:03,112][20381] Updated weights for policy 1, policy_version 790422 (0.0005) [2023-12-27 11:06:03,125][20347] Updated weights for policy 0, policy_version 790935 (0.0006) [2023-12-27 11:06:03,791][20381] Updated weights for policy 1, policy_version 790432 (0.0005) [2023-12-27 11:06:03,813][20347] Updated weights for policy 0, policy_version 790945 (0.0007) [2023-12-27 11:06:03,841][20381] Updated weights for policy 1, policy_version 790442 (0.0007) [2023-12-27 11:06:03,873][20347] Updated weights for policy 0, policy_version 790955 (0.0007) [2023-12-27 11:06:03,902][20381] Updated weights for policy 1, policy_version 790452 (0.0006) [2023-12-27 11:06:03,930][20347] Updated weights for policy 0, policy_version 790965 (0.0007) [2023-12-27 11:06:03,996][20347] Updated weights for policy 0, policy_version 790975 (0.0007) [2023-12-27 11:06:04,656][20381] Updated weights for policy 1, policy_version 790462 (0.0007) [2023-12-27 11:06:04,668][20347] Updated weights for policy 0, policy_version 790985 (0.0006) [2023-12-27 11:06:04,710][20381] Updated weights for policy 1, policy_version 790472 (0.0007) [2023-12-27 11:06:04,721][20347] Updated weights for policy 0, policy_version 790995 (0.0006) [2023-12-27 11:06:04,772][20381] Updated weights for policy 1, policy_version 790482 (0.0008) [2023-12-27 11:06:04,778][20347] Updated weights for policy 0, policy_version 791005 (0.0006) [2023-12-27 11:06:05,442][20347] Updated weights for policy 0, policy_version 791015 (0.0007) [2023-12-27 11:06:05,488][20381] Updated weights for policy 1, policy_version 790492 (0.0007) [2023-12-27 11:06:05,498][20347] Updated weights for policy 0, policy_version 791025 (0.0006) [2023-12-27 11:06:05,543][20381] Updated weights for policy 1, policy_version 790502 (0.0008) [2023-12-27 11:06:05,559][20347] Updated weights for policy 0, policy_version 791035 (0.0008) [2023-12-27 11:06:05,588][20381] Updated weights for policy 1, policy_version 790512 (0.0008) [2023-12-27 11:06:06,189][20347] Updated weights for policy 0, policy_version 791045 (0.0008) [2023-12-27 11:06:06,253][20347] Updated weights for policy 0, policy_version 791055 (0.0008) [2023-12-27 11:06:06,304][20381] Updated weights for policy 1, policy_version 790522 (0.0008) [2023-12-27 11:06:06,318][20347] Updated weights for policy 0, policy_version 791065 (0.0008) [2023-12-27 11:06:06,363][20381] Updated weights for policy 1, policy_version 790532 (0.0006) [2023-12-27 11:06:06,424][20381] Updated weights for policy 1, policy_version 790542 (0.0008) [2023-12-27 11:06:06,492][20381] Updated weights for policy 1, policy_version 790552 (0.0006) [2023-12-27 11:06:06,966][20347] Updated weights for policy 0, policy_version 791075 (0.0006) [2023-12-27 11:06:07,023][20347] Updated weights for policy 0, policy_version 791085 (0.0007) [2023-12-27 11:06:07,080][20347] Updated weights for policy 0, policy_version 791095 (0.0007) [2023-12-27 11:06:07,170][20381] Updated weights for policy 1, policy_version 790562 (0.0007) [2023-12-27 11:06:07,222][20381] Updated weights for policy 1, policy_version 790572 (0.0008) [2023-12-27 11:06:07,275][20381] Updated weights for policy 1, policy_version 790582 (0.0008) [2023-12-27 11:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 404963328. Throughput: 0: 10204.9, 1: 10223.0. Samples: 404927448. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:06:07,363][19170] Avg episode reward: [(0, '970.690'), (1, '990.330')] [2023-12-27 11:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000790584_202416128.pth... [2023-12-27 11:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000791104_202547200.pth... [2023-12-27 11:06:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000789920_202244096.pth [2023-12-27 11:06:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000789400_202113024.pth [2023-12-27 11:06:07,748][20347] Updated weights for policy 0, policy_version 791105 (0.0006) [2023-12-27 11:06:07,803][20347] Updated weights for policy 0, policy_version 791115 (0.0005) [2023-12-27 11:06:07,858][20347] Updated weights for policy 0, policy_version 791125 (0.0006) [2023-12-27 11:06:07,902][20381] Updated weights for policy 1, policy_version 790592 (0.0006) [2023-12-27 11:06:07,904][20347] Updated weights for policy 0, policy_version 791135 (0.0006) [2023-12-27 11:06:07,946][20381] Updated weights for policy 1, policy_version 790602 (0.0006) [2023-12-27 11:06:07,992][20381] Updated weights for policy 1, policy_version 790612 (0.0007) [2023-12-27 11:06:08,554][20347] Updated weights for policy 0, policy_version 791145 (0.0008) [2023-12-27 11:06:08,623][20347] Updated weights for policy 0, policy_version 791155 (0.0008) [2023-12-27 11:06:08,675][20347] Updated weights for policy 0, policy_version 791165 (0.0007) [2023-12-27 11:06:08,713][20381] Updated weights for policy 1, policy_version 790622 (0.0008) [2023-12-27 11:06:08,769][20381] Updated weights for policy 1, policy_version 790632 (0.0010) [2023-12-27 11:06:08,824][20381] Updated weights for policy 1, policy_version 790642 (0.0011) [2023-12-27 11:06:09,309][20347] Updated weights for policy 0, policy_version 791175 (0.0007) [2023-12-27 11:06:09,376][20347] Updated weights for policy 0, policy_version 791185 (0.0009) [2023-12-27 11:06:09,442][20347] Updated weights for policy 0, policy_version 791195 (0.0009) [2023-12-27 11:06:09,509][20381] Updated weights for policy 1, policy_version 790652 (0.0011) [2023-12-27 11:06:09,562][20381] Updated weights for policy 1, policy_version 790662 (0.0011) [2023-12-27 11:06:09,623][20381] Updated weights for policy 1, policy_version 790672 (0.0011) [2023-12-27 11:06:10,198][20347] Updated weights for policy 0, policy_version 791205 (0.0008) [2023-12-27 11:06:10,246][20347] Updated weights for policy 0, policy_version 791215 (0.0007) [2023-12-27 11:06:10,276][20381] Updated weights for policy 1, policy_version 790682 (0.0011) [2023-12-27 11:06:10,302][20347] Updated weights for policy 0, policy_version 791225 (0.0007) [2023-12-27 11:06:10,337][20381] Updated weights for policy 1, policy_version 790692 (0.0009) [2023-12-27 11:06:10,398][20381] Updated weights for policy 1, policy_version 790702 (0.0009) [2023-12-27 11:06:10,454][20381] Updated weights for policy 1, policy_version 790712 (0.0009) [2023-12-27 11:06:11,015][20347] Updated weights for policy 0, policy_version 791235 (0.0006) [2023-12-27 11:06:11,081][20347] Updated weights for policy 0, policy_version 791245 (0.0007) [2023-12-27 11:06:11,111][20381] Updated weights for policy 1, policy_version 790722 (0.0011) [2023-12-27 11:06:11,149][20347] Updated weights for policy 0, policy_version 791255 (0.0009) [2023-12-27 11:06:11,182][20381] Updated weights for policy 1, policy_version 790732 (0.0008) [2023-12-27 11:06:11,245][20381] Updated weights for policy 1, policy_version 790742 (0.0008) [2023-12-27 11:06:11,906][20347] Updated weights for policy 0, policy_version 791265 (0.0007) [2023-12-27 11:06:11,928][20381] Updated weights for policy 1, policy_version 790752 (0.0006) [2023-12-27 11:06:11,955][20347] Updated weights for policy 0, policy_version 791275 (0.0008) [2023-12-27 11:06:11,984][20381] Updated weights for policy 1, policy_version 790762 (0.0006) [2023-12-27 11:06:12,008][20347] Updated weights for policy 0, policy_version 791285 (0.0008) [2023-12-27 11:06:12,039][20381] Updated weights for policy 1, policy_version 790772 (0.0005) [2023-12-27 11:06:12,062][20347] Updated weights for policy 0, policy_version 791295 (0.0008) [2023-12-27 11:06:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 405061632. Throughput: 0: 10208.2, 1: 10228.7. Samples: 405050608. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:06:12,363][19170] Avg episode reward: [(0, '970.690'), (1, '980.600')] [2023-12-27 11:06:12,704][20381] Updated weights for policy 1, policy_version 790782 (0.0007) [2023-12-27 11:06:12,753][20381] Updated weights for policy 1, policy_version 790792 (0.0008) [2023-12-27 11:06:12,809][20381] Updated weights for policy 1, policy_version 790802 (0.0007) [2023-12-27 11:06:12,810][20347] Updated weights for policy 0, policy_version 791305 (0.0006) [2023-12-27 11:06:12,870][20347] Updated weights for policy 0, policy_version 791315 (0.0006) [2023-12-27 11:06:12,921][20347] Updated weights for policy 0, policy_version 791325 (0.0007) [2023-12-27 11:06:13,442][20381] Updated weights for policy 1, policy_version 790812 (0.0006) [2023-12-27 11:06:13,502][20381] Updated weights for policy 1, policy_version 790822 (0.0007) [2023-12-27 11:06:13,564][20381] Updated weights for policy 1, policy_version 790832 (0.0008) [2023-12-27 11:06:13,604][20347] Updated weights for policy 0, policy_version 791335 (0.0006) [2023-12-27 11:06:13,665][20347] Updated weights for policy 0, policy_version 791345 (0.0008) [2023-12-27 11:06:13,712][20347] Updated weights for policy 0, policy_version 791355 (0.0007) [2023-12-27 11:06:14,256][20381] Updated weights for policy 1, policy_version 790842 (0.0008) [2023-12-27 11:06:14,312][20381] Updated weights for policy 1, policy_version 790852 (0.0008) [2023-12-27 11:06:14,372][20381] Updated weights for policy 1, policy_version 790862 (0.0007) [2023-12-27 11:06:14,393][20347] Updated weights for policy 0, policy_version 791365 (0.0007) [2023-12-27 11:06:14,429][20381] Updated weights for policy 1, policy_version 790872 (0.0007) [2023-12-27 11:06:14,446][20347] Updated weights for policy 0, policy_version 791375 (0.0006) [2023-12-27 11:06:14,493][20347] Updated weights for policy 0, policy_version 791385 (0.0005) [2023-12-27 11:06:15,152][20347] Updated weights for policy 0, policy_version 791395 (0.0007) [2023-12-27 11:06:15,209][20381] Updated weights for policy 1, policy_version 790882 (0.0006) [2023-12-27 11:06:15,214][20347] Updated weights for policy 0, policy_version 791405 (0.0007) [2023-12-27 11:06:15,271][20347] Updated weights for policy 0, policy_version 791415 (0.0007) [2023-12-27 11:06:15,274][20381] Updated weights for policy 1, policy_version 790892 (0.0006) [2023-12-27 11:06:15,334][20381] Updated weights for policy 1, policy_version 790902 (0.0008) [2023-12-27 11:06:15,982][20347] Updated weights for policy 0, policy_version 791425 (0.0007) [2023-12-27 11:06:15,984][20381] Updated weights for policy 1, policy_version 790912 (0.0008) [2023-12-27 11:06:16,037][20381] Updated weights for policy 1, policy_version 790922 (0.0006) [2023-12-27 11:06:16,040][20347] Updated weights for policy 0, policy_version 791435 (0.0006) [2023-12-27 11:06:16,086][20381] Updated weights for policy 1, policy_version 790932 (0.0008) [2023-12-27 11:06:16,098][20347] Updated weights for policy 0, policy_version 791445 (0.0007) [2023-12-27 11:06:16,159][20347] Updated weights for policy 0, policy_version 791455 (0.0008) [2023-12-27 11:06:16,685][20381] Updated weights for policy 1, policy_version 790942 (0.0007) [2023-12-27 11:06:16,740][20381] Updated weights for policy 1, policy_version 790952 (0.0006) [2023-12-27 11:06:16,768][20347] Updated weights for policy 0, policy_version 791465 (0.0007) [2023-12-27 11:06:16,793][20381] Updated weights for policy 1, policy_version 790962 (0.0006) [2023-12-27 11:06:16,825][20347] Updated weights for policy 0, policy_version 791475 (0.0008) [2023-12-27 11:06:16,880][20347] Updated weights for policy 0, policy_version 791485 (0.0006) [2023-12-27 11:06:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 405159936. Throughput: 0: 10114.0, 1: 10245.5. Samples: 405172792. Policy #0 lag: (min: 13.0, avg: 13.2, max: 24.0) [2023-12-27 11:06:17,363][19170] Avg episode reward: [(0, '931.580'), (1, '990.270')] [2023-12-27 11:06:17,510][20381] Updated weights for policy 1, policy_version 790972 (0.0007) [2023-12-27 11:06:17,513][20347] Updated weights for policy 0, policy_version 791495 (0.0008) [2023-12-27 11:06:17,558][20381] Updated weights for policy 1, policy_version 790982 (0.0007) [2023-12-27 11:06:17,571][20347] Updated weights for policy 0, policy_version 791505 (0.0009) [2023-12-27 11:06:17,603][20381] Updated weights for policy 1, policy_version 790992 (0.0006) [2023-12-27 11:06:17,623][20347] Updated weights for policy 0, policy_version 791515 (0.0005) [2023-12-27 11:06:18,276][20347] Updated weights for policy 0, policy_version 791525 (0.0007) [2023-12-27 11:06:18,322][20381] Updated weights for policy 1, policy_version 791002 (0.0008) [2023-12-27 11:06:18,335][20347] Updated weights for policy 0, policy_version 791535 (0.0007) [2023-12-27 11:06:18,390][20381] Updated weights for policy 1, policy_version 791012 (0.0008) [2023-12-27 11:06:18,392][20347] Updated weights for policy 0, policy_version 791545 (0.0007) [2023-12-27 11:06:18,453][20381] Updated weights for policy 1, policy_version 791022 (0.0008) [2023-12-27 11:06:18,517][20381] Updated weights for policy 1, policy_version 791032 (0.0008) [2023-12-27 11:06:19,055][20347] Updated weights for policy 0, policy_version 791555 (0.0006) [2023-12-27 11:06:19,121][20347] Updated weights for policy 0, policy_version 791565 (0.0005) [2023-12-27 11:06:19,181][20347] Updated weights for policy 0, policy_version 791575 (0.0008) [2023-12-27 11:06:19,204][20381] Updated weights for policy 1, policy_version 791042 (0.0005) [2023-12-27 11:06:19,275][20381] Updated weights for policy 1, policy_version 791052 (0.0008) [2023-12-27 11:06:19,337][20381] Updated weights for policy 1, policy_version 791062 (0.0008) [2023-12-27 11:06:19,838][20347] Updated weights for policy 0, policy_version 791585 (0.0008) [2023-12-27 11:06:19,903][20347] Updated weights for policy 0, policy_version 791595 (0.0010) [2023-12-27 11:06:19,971][20347] Updated weights for policy 0, policy_version 791605 (0.0008) [2023-12-27 11:06:20,024][20381] Updated weights for policy 1, policy_version 791072 (0.0006) [2023-12-27 11:06:20,035][20347] Updated weights for policy 0, policy_version 791615 (0.0009) [2023-12-27 11:06:20,077][20381] Updated weights for policy 1, policy_version 791082 (0.0007) [2023-12-27 11:06:20,129][20381] Updated weights for policy 1, policy_version 791092 (0.0008) [2023-12-27 11:06:20,750][20347] Updated weights for policy 0, policy_version 791625 (0.0008) [2023-12-27 11:06:20,763][20381] Updated weights for policy 1, policy_version 791102 (0.0007) [2023-12-27 11:06:20,800][20347] Updated weights for policy 0, policy_version 791635 (0.0006) [2023-12-27 11:06:20,818][20381] Updated weights for policy 1, policy_version 791112 (0.0006) [2023-12-27 11:06:20,860][20347] Updated weights for policy 0, policy_version 791645 (0.0007) [2023-12-27 11:06:20,874][20381] Updated weights for policy 1, policy_version 791122 (0.0006) [2023-12-27 11:06:21,597][20381] Updated weights for policy 1, policy_version 791132 (0.0008) [2023-12-27 11:06:21,606][20347] Updated weights for policy 0, policy_version 791655 (0.0009) [2023-12-27 11:06:21,612][20291] KL-divergence is very high: 174.0640 [2023-12-27 11:06:21,640][20291] KL-divergence is very high: 236.0981 [2023-12-27 11:06:21,666][20381] Updated weights for policy 1, policy_version 791142 (0.0008) [2023-12-27 11:06:21,668][20291] KL-divergence is very high: 187.0904 [2023-12-27 11:06:21,673][20347] Updated weights for policy 0, policy_version 791665 (0.0008) [2023-12-27 11:06:21,735][20381] Updated weights for policy 1, policy_version 791152 (0.0009) [2023-12-27 11:06:21,741][20347] Updated weights for policy 0, policy_version 791675 (0.0008) [2023-12-27 11:06:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 405258240. Throughput: 0: 10147.5, 1: 10199.8. Samples: 405233160. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:22,363][19170] Avg episode reward: [(0, '931.700'), (1, '990.310')] [2023-12-27 11:06:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000791160_202563584.pth... [2023-12-27 11:06:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000791680_202694656.pth... [2023-12-27 11:06:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000790008_202268672.pth [2023-12-27 11:06:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000790496_202391552.pth [2023-12-27 11:06:22,439][20381] Updated weights for policy 1, policy_version 791162 (0.0008) [2023-12-27 11:06:22,469][20347] Updated weights for policy 0, policy_version 791685 (0.0006) [2023-12-27 11:06:22,501][20381] Updated weights for policy 1, policy_version 791172 (0.0008) [2023-12-27 11:06:22,530][20347] Updated weights for policy 0, policy_version 791695 (0.0006) [2023-12-27 11:06:22,559][20381] Updated weights for policy 1, policy_version 791182 (0.0008) [2023-12-27 11:06:22,595][20347] Updated weights for policy 0, policy_version 791705 (0.0006) [2023-12-27 11:06:22,615][20381] Updated weights for policy 1, policy_version 791192 (0.0007) [2023-12-27 11:06:23,242][20381] Updated weights for policy 1, policy_version 791202 (0.0006) [2023-12-27 11:06:23,293][20381] Updated weights for policy 1, policy_version 791212 (0.0006) [2023-12-27 11:06:23,314][20347] Updated weights for policy 0, policy_version 791715 (0.0006) [2023-12-27 11:06:23,350][20381] Updated weights for policy 1, policy_version 791222 (0.0006) [2023-12-27 11:06:23,365][20347] Updated weights for policy 0, policy_version 791725 (0.0007) [2023-12-27 11:06:23,417][20347] Updated weights for policy 0, policy_version 791735 (0.0007) [2023-12-27 11:06:23,912][20381] Updated weights for policy 1, policy_version 791232 (0.0006) [2023-12-27 11:06:23,957][20381] Updated weights for policy 1, policy_version 791242 (0.0009) [2023-12-27 11:06:24,004][20347] Updated weights for policy 0, policy_version 791745 (0.0006) [2023-12-27 11:06:24,006][20381] Updated weights for policy 1, policy_version 791252 (0.0009) [2023-12-27 11:06:24,057][20347] Updated weights for policy 0, policy_version 791755 (0.0005) [2023-12-27 11:06:24,103][20347] Updated weights for policy 0, policy_version 791765 (0.0005) [2023-12-27 11:06:24,154][20347] Updated weights for policy 0, policy_version 791775 (0.0005) [2023-12-27 11:06:24,711][20347] Updated weights for policy 0, policy_version 791785 (0.0008) [2023-12-27 11:06:24,729][20381] Updated weights for policy 1, policy_version 791262 (0.0007) [2023-12-27 11:06:24,769][20347] Updated weights for policy 0, policy_version 791795 (0.0008) [2023-12-27 11:06:24,785][20381] Updated weights for policy 1, policy_version 791272 (0.0010) [2023-12-27 11:06:24,823][20347] Updated weights for policy 0, policy_version 791805 (0.0007) [2023-12-27 11:06:24,840][20381] Updated weights for policy 1, policy_version 791282 (0.0006) [2023-12-27 11:06:25,487][20381] Updated weights for policy 1, policy_version 791292 (0.0006) [2023-12-27 11:06:25,527][20347] Updated weights for policy 0, policy_version 791815 (0.0007) [2023-12-27 11:06:25,544][20381] Updated weights for policy 1, policy_version 791302 (0.0005) [2023-12-27 11:06:25,583][20347] Updated weights for policy 0, policy_version 791825 (0.0007) [2023-12-27 11:06:25,602][20381] Updated weights for policy 1, policy_version 791312 (0.0005) [2023-12-27 11:06:25,633][20347] Updated weights for policy 0, policy_version 791835 (0.0005) [2023-12-27 11:06:26,213][20381] Updated weights for policy 1, policy_version 791322 (0.0005) [2023-12-27 11:06:26,275][20381] Updated weights for policy 1, policy_version 791332 (0.0008) [2023-12-27 11:06:26,306][20347] Updated weights for policy 0, policy_version 791845 (0.0007) [2023-12-27 11:06:26,327][20381] Updated weights for policy 1, policy_version 791342 (0.0007) [2023-12-27 11:06:26,364][20347] Updated weights for policy 0, policy_version 791855 (0.0008) [2023-12-27 11:06:26,375][20381] Updated weights for policy 1, policy_version 791352 (0.0006) [2023-12-27 11:06:26,418][20347] Updated weights for policy 0, policy_version 791865 (0.0007) [2023-12-27 11:06:26,922][20381] Updated weights for policy 1, policy_version 791362 (0.0005) [2023-12-27 11:06:26,973][20381] Updated weights for policy 1, policy_version 791372 (0.0005) [2023-12-27 11:06:27,021][20381] Updated weights for policy 1, policy_version 791382 (0.0005) [2023-12-27 11:06:27,163][20347] Updated weights for policy 0, policy_version 791875 (0.0008) [2023-12-27 11:06:27,213][20347] Updated weights for policy 0, policy_version 791885 (0.0007) [2023-12-27 11:06:27,269][20347] Updated weights for policy 0, policy_version 791895 (0.0008) [2023-12-27 11:06:27,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 405372928. Throughput: 0: 10194.4, 1: 10189.0. Samples: 405357376. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:27,364][19170] Avg episode reward: [(0, '892.410'), (1, '990.310')] [2023-12-27 11:06:27,588][20381] Updated weights for policy 1, policy_version 791392 (0.0005) [2023-12-27 11:06:27,638][20381] Updated weights for policy 1, policy_version 791402 (0.0005) [2023-12-27 11:06:27,693][20381] Updated weights for policy 1, policy_version 791412 (0.0005) [2023-12-27 11:06:27,931][20347] Updated weights for policy 0, policy_version 791905 (0.0007) [2023-12-27 11:06:27,977][20347] Updated weights for policy 0, policy_version 791915 (0.0006) [2023-12-27 11:06:28,031][20347] Updated weights for policy 0, policy_version 791925 (0.0008) [2023-12-27 11:06:28,086][20347] Updated weights for policy 0, policy_version 791935 (0.0008) [2023-12-27 11:06:28,272][20381] Updated weights for policy 1, policy_version 791422 (0.0008) [2023-12-27 11:06:28,325][20381] Updated weights for policy 1, policy_version 791432 (0.0006) [2023-12-27 11:06:28,391][20381] Updated weights for policy 1, policy_version 791442 (0.0007) [2023-12-27 11:06:28,789][20347] Updated weights for policy 0, policy_version 791945 (0.0008) [2023-12-27 11:06:28,837][20347] Updated weights for policy 0, policy_version 791955 (0.0007) [2023-12-27 11:06:28,897][20347] Updated weights for policy 0, policy_version 791965 (0.0007) [2023-12-27 11:06:29,041][20381] Updated weights for policy 1, policy_version 791452 (0.0011) [2023-12-27 11:06:29,097][20381] Updated weights for policy 1, policy_version 791462 (0.0010) [2023-12-27 11:06:29,150][20381] Updated weights for policy 1, policy_version 791472 (0.0009) [2023-12-27 11:06:29,607][20347] Updated weights for policy 0, policy_version 791975 (0.0007) [2023-12-27 11:06:29,664][20347] Updated weights for policy 0, policy_version 791985 (0.0007) [2023-12-27 11:06:29,717][20347] Updated weights for policy 0, policy_version 791995 (0.0008) [2023-12-27 11:06:29,836][20381] Updated weights for policy 1, policy_version 791482 (0.0010) [2023-12-27 11:06:29,895][20381] Updated weights for policy 1, policy_version 791492 (0.0006) [2023-12-27 11:06:29,960][20381] Updated weights for policy 1, policy_version 791502 (0.0007) [2023-12-27 11:06:30,015][20381] Updated weights for policy 1, policy_version 791512 (0.0008) [2023-12-27 11:06:30,465][20347] Updated weights for policy 0, policy_version 792005 (0.0008) [2023-12-27 11:06:30,520][20347] Updated weights for policy 0, policy_version 792015 (0.0008) [2023-12-27 11:06:30,579][20347] Updated weights for policy 0, policy_version 792025 (0.0007) [2023-12-27 11:06:30,665][20381] Updated weights for policy 1, policy_version 791522 (0.0006) [2023-12-27 11:06:30,729][20381] Updated weights for policy 1, policy_version 791532 (0.0005) [2023-12-27 11:06:30,785][20381] Updated weights for policy 1, policy_version 791542 (0.0005) [2023-12-27 11:06:31,297][20347] Updated weights for policy 0, policy_version 792035 (0.0006) [2023-12-27 11:06:31,358][20347] Updated weights for policy 0, policy_version 792045 (0.0007) [2023-12-27 11:06:31,423][20347] Updated weights for policy 0, policy_version 792055 (0.0008) [2023-12-27 11:06:31,452][20381] Updated weights for policy 1, policy_version 791552 (0.0007) [2023-12-27 11:06:31,509][20381] Updated weights for policy 1, policy_version 791562 (0.0007) [2023-12-27 11:06:31,569][20381] Updated weights for policy 1, policy_version 791572 (0.0008) [2023-12-27 11:06:32,027][20347] Updated weights for policy 0, policy_version 792065 (0.0007) [2023-12-27 11:06:32,088][20347] Updated weights for policy 0, policy_version 792075 (0.0008) [2023-12-27 11:06:32,148][20347] Updated weights for policy 0, policy_version 792085 (0.0008) [2023-12-27 11:06:32,210][20347] Updated weights for policy 0, policy_version 792095 (0.0007) [2023-12-27 11:06:32,279][20381] Updated weights for policy 1, policy_version 791582 (0.0008) [2023-12-27 11:06:32,336][20381] Updated weights for policy 1, policy_version 791592 (0.0008) [2023-12-27 11:06:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 405471232. Throughput: 0: 10164.3, 1: 10268.2. Samples: 405482508. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:32,363][19170] Avg episode reward: [(0, '892.430'), (1, '990.360')] [2023-12-27 11:06:32,403][20381] Updated weights for policy 1, policy_version 791602 (0.0007) [2023-12-27 11:06:32,882][20347] Updated weights for policy 0, policy_version 792105 (0.0007) [2023-12-27 11:06:32,940][20347] Updated weights for policy 0, policy_version 792115 (0.0007) [2023-12-27 11:06:32,993][20291] KL-divergence is very high: 120.7323 [2023-12-27 11:06:33,004][20347] Updated weights for policy 0, policy_version 792125 (0.0008) [2023-12-27 11:06:33,118][20381] Updated weights for policy 1, policy_version 791612 (0.0007) [2023-12-27 11:06:33,178][20381] Updated weights for policy 1, policy_version 791622 (0.0008) [2023-12-27 11:06:33,230][20381] Updated weights for policy 1, policy_version 791632 (0.0008) [2023-12-27 11:06:33,702][20347] Updated weights for policy 0, policy_version 792135 (0.0008) [2023-12-27 11:06:33,749][20347] Updated weights for policy 0, policy_version 792145 (0.0007) [2023-12-27 11:06:33,800][20347] Updated weights for policy 0, policy_version 792155 (0.0008) [2023-12-27 11:06:33,833][20381] Updated weights for policy 1, policy_version 791642 (0.0008) [2023-12-27 11:06:33,878][20381] Updated weights for policy 1, policy_version 791652 (0.0006) [2023-12-27 11:06:33,923][20381] Updated weights for policy 1, policy_version 791662 (0.0005) [2023-12-27 11:06:33,976][20381] Updated weights for policy 1, policy_version 791672 (0.0005) [2023-12-27 11:06:34,513][20347] Updated weights for policy 0, policy_version 792165 (0.0009) [2023-12-27 11:06:34,578][20347] Updated weights for policy 0, policy_version 792175 (0.0011) [2023-12-27 11:06:34,636][20347] Updated weights for policy 0, policy_version 792185 (0.0010) [2023-12-27 11:06:34,661][20381] Updated weights for policy 1, policy_version 791682 (0.0007) [2023-12-27 11:06:34,715][20381] Updated weights for policy 1, policy_version 791692 (0.0008) [2023-12-27 11:06:34,763][20381] Updated weights for policy 1, policy_version 791702 (0.0007) [2023-12-27 11:06:35,305][20347] Updated weights for policy 0, policy_version 792195 (0.0010) [2023-12-27 11:06:35,365][20347] Updated weights for policy 0, policy_version 792205 (0.0010) [2023-12-27 11:06:35,419][20347] Updated weights for policy 0, policy_version 792215 (0.0010) [2023-12-27 11:06:35,471][20381] Updated weights for policy 1, policy_version 791712 (0.0005) [2023-12-27 11:06:35,522][20381] Updated weights for policy 1, policy_version 791722 (0.0007) [2023-12-27 11:06:35,573][20381] Updated weights for policy 1, policy_version 791732 (0.0007) [2023-12-27 11:06:36,076][20347] Updated weights for policy 0, policy_version 792225 (0.0010) [2023-12-27 11:06:36,143][20347] Updated weights for policy 0, policy_version 792235 (0.0009) [2023-12-27 11:06:36,196][20347] Updated weights for policy 0, policy_version 792245 (0.0008) [2023-12-27 11:06:36,245][20347] Updated weights for policy 0, policy_version 792255 (0.0007) [2023-12-27 11:06:36,277][20381] Updated weights for policy 1, policy_version 791742 (0.0007) [2023-12-27 11:06:36,337][20381] Updated weights for policy 1, policy_version 791752 (0.0007) [2023-12-27 11:06:36,402][20381] Updated weights for policy 1, policy_version 791762 (0.0008) [2023-12-27 11:06:36,889][20347] Updated weights for policy 0, policy_version 792265 (0.0005) [2023-12-27 11:06:36,938][20347] Updated weights for policy 0, policy_version 792275 (0.0005) [2023-12-27 11:06:36,989][20347] Updated weights for policy 0, policy_version 792285 (0.0005) [2023-12-27 11:06:37,109][20381] Updated weights for policy 1, policy_version 791772 (0.0008) [2023-12-27 11:06:37,167][20381] Updated weights for policy 1, policy_version 791782 (0.0007) [2023-12-27 11:06:37,221][20381] Updated weights for policy 1, policy_version 791792 (0.0007) [2023-12-27 11:06:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 405577728. Throughput: 0: 10158.3, 1: 10257.4. Samples: 405543368. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:37,363][19170] Avg episode reward: [(0, '882.590'), (1, '990.360')] [2023-12-27 11:06:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000792288_202850304.pth... [2023-12-27 11:06:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000791800_202727424.pth... [2023-12-27 11:06:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000791104_202547200.pth [2023-12-27 11:06:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000790584_202416128.pth [2023-12-27 11:06:37,596][20347] Updated weights for policy 0, policy_version 792295 (0.0007) [2023-12-27 11:06:37,644][20347] Updated weights for policy 0, policy_version 792305 (0.0010) [2023-12-27 11:06:37,701][20347] Updated weights for policy 0, policy_version 792315 (0.0010) [2023-12-27 11:06:37,947][20381] Updated weights for policy 1, policy_version 791802 (0.0007) [2023-12-27 11:06:38,003][20381] Updated weights for policy 1, policy_version 791812 (0.0005) [2023-12-27 11:06:38,063][20381] Updated weights for policy 1, policy_version 791822 (0.0005) [2023-12-27 11:06:38,125][20381] Updated weights for policy 1, policy_version 791832 (0.0005) [2023-12-27 11:06:38,404][20347] Updated weights for policy 0, policy_version 792325 (0.0010) [2023-12-27 11:06:38,459][20347] Updated weights for policy 0, policy_version 792335 (0.0010) [2023-12-27 11:06:38,514][20347] Updated weights for policy 0, policy_version 792345 (0.0008) [2023-12-27 11:06:38,689][20381] Updated weights for policy 1, policy_version 791842 (0.0007) [2023-12-27 11:06:38,741][20381] Updated weights for policy 1, policy_version 791852 (0.0007) [2023-12-27 11:06:38,796][20381] Updated weights for policy 1, policy_version 791862 (0.0008) [2023-12-27 11:06:39,206][20347] Updated weights for policy 0, policy_version 792355 (0.0010) [2023-12-27 11:06:39,271][20347] Updated weights for policy 0, policy_version 792365 (0.0009) [2023-12-27 11:06:39,333][20347] Updated weights for policy 0, policy_version 792375 (0.0007) [2023-12-27 11:06:39,517][20381] Updated weights for policy 1, policy_version 791872 (0.0009) [2023-12-27 11:06:39,579][20381] Updated weights for policy 1, policy_version 791882 (0.0008) [2023-12-27 11:06:39,637][20381] Updated weights for policy 1, policy_version 791892 (0.0008) [2023-12-27 11:06:40,038][20347] Updated weights for policy 0, policy_version 792385 (0.0009) [2023-12-27 11:06:40,100][20347] Updated weights for policy 0, policy_version 792395 (0.0007) [2023-12-27 11:06:40,154][20347] Updated weights for policy 0, policy_version 792405 (0.0008) [2023-12-27 11:06:40,214][20347] Updated weights for policy 0, policy_version 792415 (0.0008) [2023-12-27 11:06:40,350][20381] Updated weights for policy 1, policy_version 791902 (0.0009) [2023-12-27 11:06:40,411][20381] Updated weights for policy 1, policy_version 791912 (0.0007) [2023-12-27 11:06:40,470][20381] Updated weights for policy 1, policy_version 791922 (0.0008) [2023-12-27 11:06:40,915][20347] Updated weights for policy 0, policy_version 792425 (0.0007) [2023-12-27 11:06:40,972][20347] Updated weights for policy 0, policy_version 792435 (0.0007) [2023-12-27 11:06:41,024][20347] Updated weights for policy 0, policy_version 792445 (0.0008) [2023-12-27 11:06:41,171][20381] Updated weights for policy 1, policy_version 791932 (0.0008) [2023-12-27 11:06:41,236][20381] Updated weights for policy 1, policy_version 791942 (0.0008) [2023-12-27 11:06:41,297][20381] Updated weights for policy 1, policy_version 791952 (0.0007) [2023-12-27 11:06:41,792][20347] Updated weights for policy 0, policy_version 792455 (0.0007) [2023-12-27 11:06:41,848][20347] Updated weights for policy 0, policy_version 792465 (0.0008) [2023-12-27 11:06:41,911][20347] Updated weights for policy 0, policy_version 792475 (0.0008) [2023-12-27 11:06:42,017][20381] Updated weights for policy 1, policy_version 791962 (0.0008) [2023-12-27 11:06:42,073][20381] Updated weights for policy 1, policy_version 791972 (0.0007) [2023-12-27 11:06:42,131][20381] Updated weights for policy 1, policy_version 791982 (0.0006) [2023-12-27 11:06:42,194][20381] Updated weights for policy 1, policy_version 791992 (0.0005) [2023-12-27 11:06:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 405676032. Throughput: 0: 10171.2, 1: 10220.7. Samples: 405664820. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:42,363][19170] Avg episode reward: [(0, '872.760'), (1, '1000.000')] [2023-12-27 11:06:42,648][20347] Updated weights for policy 0, policy_version 792485 (0.0008) [2023-12-27 11:06:42,699][20347] Updated weights for policy 0, policy_version 792495 (0.0006) [2023-12-27 11:06:42,752][20347] Updated weights for policy 0, policy_version 792505 (0.0005) [2023-12-27 11:06:42,858][20381] Updated weights for policy 1, policy_version 792002 (0.0007) [2023-12-27 11:06:42,919][20381] Updated weights for policy 1, policy_version 792012 (0.0006) [2023-12-27 11:06:42,979][20381] Updated weights for policy 1, policy_version 792022 (0.0008) [2023-12-27 11:06:43,418][20347] Updated weights for policy 0, policy_version 792515 (0.0007) [2023-12-27 11:06:43,476][20347] Updated weights for policy 0, policy_version 792525 (0.0010) [2023-12-27 11:06:43,521][20347] Updated weights for policy 0, policy_version 792535 (0.0010) [2023-12-27 11:06:43,585][20381] Updated weights for policy 1, policy_version 792032 (0.0008) [2023-12-27 11:06:43,648][20381] Updated weights for policy 1, policy_version 792042 (0.0007) [2023-12-27 11:06:43,708][20381] Updated weights for policy 1, policy_version 792052 (0.0007) [2023-12-27 11:06:44,207][20347] Updated weights for policy 0, policy_version 792545 (0.0010) [2023-12-27 11:06:44,259][20347] Updated weights for policy 0, policy_version 792555 (0.0010) [2023-12-27 11:06:44,308][20347] Updated weights for policy 0, policy_version 792565 (0.0010) [2023-12-27 11:06:44,366][20347] Updated weights for policy 0, policy_version 792575 (0.0010) [2023-12-27 11:06:44,399][20381] Updated weights for policy 1, policy_version 792062 (0.0007) [2023-12-27 11:06:44,456][20381] Updated weights for policy 1, policy_version 792072 (0.0008) [2023-12-27 11:06:44,517][20381] Updated weights for policy 1, policy_version 792082 (0.0008) [2023-12-27 11:06:45,126][20347] Updated weights for policy 0, policy_version 792585 (0.0009) [2023-12-27 11:06:45,175][20381] Updated weights for policy 1, policy_version 792092 (0.0006) [2023-12-27 11:06:45,184][20347] Updated weights for policy 0, policy_version 792595 (0.0007) [2023-12-27 11:06:45,237][20347] Updated weights for policy 0, policy_version 792605 (0.0006) [2023-12-27 11:06:45,238][20381] Updated weights for policy 1, policy_version 792102 (0.0008) [2023-12-27 11:06:45,298][20381] Updated weights for policy 1, policy_version 792112 (0.0007) [2023-12-27 11:06:45,891][20347] Updated weights for policy 0, policy_version 792615 (0.0007) [2023-12-27 11:06:45,961][20347] Updated weights for policy 0, policy_version 792625 (0.0007) [2023-12-27 11:06:45,973][20381] Updated weights for policy 1, policy_version 792122 (0.0008) [2023-12-27 11:06:46,021][20347] Updated weights for policy 0, policy_version 792635 (0.0006) [2023-12-27 11:06:46,026][20381] Updated weights for policy 1, policy_version 792132 (0.0008) [2023-12-27 11:06:46,092][20381] Updated weights for policy 1, policy_version 792142 (0.0010) [2023-12-27 11:06:46,156][20381] Updated weights for policy 1, policy_version 792152 (0.0011) [2023-12-27 11:06:46,701][20347] Updated weights for policy 0, policy_version 792645 (0.0009) [2023-12-27 11:06:46,759][20347] Updated weights for policy 0, policy_version 792655 (0.0010) [2023-12-27 11:06:46,811][20347] Updated weights for policy 0, policy_version 792665 (0.0007) [2023-12-27 11:06:46,813][20381] Updated weights for policy 1, policy_version 792162 (0.0005) [2023-12-27 11:06:46,864][20381] Updated weights for policy 1, policy_version 792172 (0.0005) [2023-12-27 11:06:46,920][20381] Updated weights for policy 1, policy_version 792182 (0.0005) [2023-12-27 11:06:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 405774336. Throughput: 0: 10149.8, 1: 10287.3. Samples: 405786156. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:47,363][19170] Avg episode reward: [(0, '902.170'), (1, '990.270')] [2023-12-27 11:06:47,431][20347] Updated weights for policy 0, policy_version 792675 (0.0009) [2023-12-27 11:06:47,485][20347] Updated weights for policy 0, policy_version 792685 (0.0005) [2023-12-27 11:06:47,520][20381] Updated weights for policy 1, policy_version 792192 (0.0010) [2023-12-27 11:06:47,535][20347] Updated weights for policy 0, policy_version 792695 (0.0005) [2023-12-27 11:06:47,565][20381] Updated weights for policy 1, policy_version 792202 (0.0010) [2023-12-27 11:06:47,613][20381] Updated weights for policy 1, policy_version 792212 (0.0010) [2023-12-27 11:06:48,154][20347] Updated weights for policy 0, policy_version 792705 (0.0006) [2023-12-27 11:06:48,205][20347] Updated weights for policy 0, policy_version 792715 (0.0007) [2023-12-27 11:06:48,258][20347] Updated weights for policy 0, policy_version 792725 (0.0006) [2023-12-27 11:06:48,260][20381] Updated weights for policy 1, policy_version 792222 (0.0010) [2023-12-27 11:06:48,309][20381] Updated weights for policy 1, policy_version 792232 (0.0011) [2023-12-27 11:06:48,310][20347] Updated weights for policy 0, policy_version 792735 (0.0005) [2023-12-27 11:06:48,369][20381] Updated weights for policy 1, policy_version 792242 (0.0010) [2023-12-27 11:06:48,991][20347] Updated weights for policy 0, policy_version 792745 (0.0007) [2023-12-27 11:06:49,052][20347] Updated weights for policy 0, policy_version 792755 (0.0007) [2023-12-27 11:06:49,074][20381] Updated weights for policy 1, policy_version 792252 (0.0009) [2023-12-27 11:06:49,100][20347] Updated weights for policy 0, policy_version 792765 (0.0006) [2023-12-27 11:06:49,128][20381] Updated weights for policy 1, policy_version 792262 (0.0011) [2023-12-27 11:06:49,183][20381] Updated weights for policy 1, policy_version 792272 (0.0011) [2023-12-27 11:06:49,838][20347] Updated weights for policy 0, policy_version 792775 (0.0008) [2023-12-27 11:06:49,896][20347] Updated weights for policy 0, policy_version 792785 (0.0011) [2023-12-27 11:06:49,922][20381] Updated weights for policy 1, policy_version 792282 (0.0010) [2023-12-27 11:06:49,952][20347] Updated weights for policy 0, policy_version 792795 (0.0011) [2023-12-27 11:06:49,981][20381] Updated weights for policy 1, policy_version 792292 (0.0006) [2023-12-27 11:06:50,043][20381] Updated weights for policy 1, policy_version 792302 (0.0006) [2023-12-27 11:06:50,101][20381] Updated weights for policy 1, policy_version 792312 (0.0007) [2023-12-27 11:06:50,628][20347] Updated weights for policy 0, policy_version 792805 (0.0008) [2023-12-27 11:06:50,683][20347] Updated weights for policy 0, policy_version 792815 (0.0009) [2023-12-27 11:06:50,740][20347] Updated weights for policy 0, policy_version 792825 (0.0011) [2023-12-27 11:06:50,759][20381] Updated weights for policy 1, policy_version 792322 (0.0007) [2023-12-27 11:06:50,778][20292] KL-divergence is very high: 653.0248 [2023-12-27 11:06:50,812][20381] Updated weights for policy 1, policy_version 792332 (0.0006) [2023-12-27 11:06:50,863][20381] Updated weights for policy 1, policy_version 792342 (0.0005) [2023-12-27 11:06:51,431][20347] Updated weights for policy 0, policy_version 792835 (0.0008) [2023-12-27 11:06:51,481][20347] Updated weights for policy 0, policy_version 792845 (0.0009) [2023-12-27 11:06:51,537][20381] Updated weights for policy 1, policy_version 792352 (0.0008) [2023-12-27 11:06:51,540][20347] Updated weights for policy 0, policy_version 792855 (0.0010) [2023-12-27 11:06:51,598][20381] Updated weights for policy 1, policy_version 792362 (0.0006) [2023-12-27 11:06:51,663][20381] Updated weights for policy 1, policy_version 792372 (0.0008) [2023-12-27 11:06:52,236][20347] Updated weights for policy 0, policy_version 792865 (0.0010) [2023-12-27 11:06:52,297][20347] Updated weights for policy 0, policy_version 792875 (0.0007) [2023-12-27 11:06:52,352][20381] Updated weights for policy 1, policy_version 792382 (0.0008) [2023-12-27 11:06:52,360][20347] Updated weights for policy 0, policy_version 792885 (0.0007) [2023-12-27 11:06:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 405872640. Throughput: 0: 10163.0, 1: 10299.1. Samples: 405848244. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:52,363][19170] Avg episode reward: [(0, '921.720'), (1, '966.810')] [2023-12-27 11:06:52,416][20347] Updated weights for policy 0, policy_version 792895 (0.0008) [2023-12-27 11:06:52,416][20381] Updated weights for policy 1, policy_version 792392 (0.0009) [2023-12-27 11:06:52,419][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000792896_203005952.pth... [2023-12-27 11:06:52,422][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000791680_202694656.pth [2023-12-27 11:06:52,465][20381] Updated weights for policy 1, policy_version 792402 (0.0008) [2023-12-27 11:06:52,495][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000792408_202883072.pth... [2023-12-27 11:06:52,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000791160_202563584.pth [2023-12-27 11:06:53,107][20347] Updated weights for policy 0, policy_version 792905 (0.0006) [2023-12-27 11:06:53,169][20347] Updated weights for policy 0, policy_version 792915 (0.0005) [2023-12-27 11:06:53,173][20381] Updated weights for policy 1, policy_version 792412 (0.0008) [2023-12-27 11:06:53,225][20347] Updated weights for policy 0, policy_version 792925 (0.0005) [2023-12-27 11:06:53,229][20381] Updated weights for policy 1, policy_version 792422 (0.0006) [2023-12-27 11:06:53,284][20381] Updated weights for policy 1, policy_version 792432 (0.0008) [2023-12-27 11:06:53,763][20347] Updated weights for policy 0, policy_version 792935 (0.0005) [2023-12-27 11:06:53,829][20347] Updated weights for policy 0, policy_version 792945 (0.0005) [2023-12-27 11:06:53,894][20347] Updated weights for policy 0, policy_version 792955 (0.0005) [2023-12-27 11:06:53,961][20381] Updated weights for policy 1, policy_version 792442 (0.0007) [2023-12-27 11:06:54,013][20381] Updated weights for policy 1, policy_version 792452 (0.0006) [2023-12-27 11:06:54,058][20381] Updated weights for policy 1, policy_version 792462 (0.0007) [2023-12-27 11:06:54,113][20381] Updated weights for policy 1, policy_version 792472 (0.0007) [2023-12-27 11:06:54,533][20347] Updated weights for policy 0, policy_version 792965 (0.0005) [2023-12-27 11:06:54,595][20347] Updated weights for policy 0, policy_version 792975 (0.0005) [2023-12-27 11:06:54,657][20347] Updated weights for policy 0, policy_version 792985 (0.0005) [2023-12-27 11:06:54,717][20381] Updated weights for policy 1, policy_version 792482 (0.0005) [2023-12-27 11:06:54,773][20381] Updated weights for policy 1, policy_version 792492 (0.0005) [2023-12-27 11:06:54,836][20381] Updated weights for policy 1, policy_version 792502 (0.0005) [2023-12-27 11:06:55,260][20347] Updated weights for policy 0, policy_version 792995 (0.0006) [2023-12-27 11:06:55,320][20347] Updated weights for policy 0, policy_version 793005 (0.0008) [2023-12-27 11:06:55,369][20347] Updated weights for policy 0, policy_version 793015 (0.0008) [2023-12-27 11:06:55,410][20381] Updated weights for policy 1, policy_version 792512 (0.0007) [2023-12-27 11:06:55,454][20381] Updated weights for policy 1, policy_version 792522 (0.0007) [2023-12-27 11:06:55,507][20381] Updated weights for policy 1, policy_version 792532 (0.0008) [2023-12-27 11:06:56,016][20347] Updated weights for policy 0, policy_version 793025 (0.0007) [2023-12-27 11:06:56,084][20347] Updated weights for policy 0, policy_version 793035 (0.0009) [2023-12-27 11:06:56,150][20347] Updated weights for policy 0, policy_version 793045 (0.0006) [2023-12-27 11:06:56,168][20381] Updated weights for policy 1, policy_version 792542 (0.0008) [2023-12-27 11:06:56,204][20347] Updated weights for policy 0, policy_version 793055 (0.0006) [2023-12-27 11:06:56,222][20381] Updated weights for policy 1, policy_version 792552 (0.0009) [2023-12-27 11:06:56,277][20381] Updated weights for policy 1, policy_version 792562 (0.0009) [2023-12-27 11:06:56,835][20347] Updated weights for policy 0, policy_version 793065 (0.0007) [2023-12-27 11:06:56,853][20381] Updated weights for policy 1, policy_version 792572 (0.0007) [2023-12-27 11:06:56,891][20347] Updated weights for policy 0, policy_version 793075 (0.0007) [2023-12-27 11:06:56,906][20381] Updated weights for policy 1, policy_version 792582 (0.0005) [2023-12-27 11:06:56,950][20347] Updated weights for policy 0, policy_version 793085 (0.0007) [2023-12-27 11:06:56,955][20381] Updated weights for policy 1, policy_version 792592 (0.0006) [2023-12-27 11:06:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 405987328. Throughput: 0: 10203.9, 1: 10347.8. Samples: 405975436. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:06:57,364][19170] Avg episode reward: [(0, '970.590'), (1, '964.890')] [2023-12-27 11:06:57,604][20381] Updated weights for policy 1, policy_version 792602 (0.0006) [2023-12-27 11:06:57,650][20347] Updated weights for policy 0, policy_version 793095 (0.0007) [2023-12-27 11:06:57,664][20381] Updated weights for policy 1, policy_version 792612 (0.0007) [2023-12-27 11:06:57,712][20347] Updated weights for policy 0, policy_version 793105 (0.0006) [2023-12-27 11:06:57,717][20381] Updated weights for policy 1, policy_version 792622 (0.0007) [2023-12-27 11:06:57,769][20347] Updated weights for policy 0, policy_version 793115 (0.0007) [2023-12-27 11:06:57,770][20381] Updated weights for policy 1, policy_version 792632 (0.0006) [2023-12-27 11:06:58,295][20381] Updated weights for policy 1, policy_version 792642 (0.0009) [2023-12-27 11:06:58,358][20381] Updated weights for policy 1, policy_version 792652 (0.0009) [2023-12-27 11:06:58,425][20381] Updated weights for policy 1, policy_version 792662 (0.0009) [2023-12-27 11:06:58,466][20347] Updated weights for policy 0, policy_version 793125 (0.0008) [2023-12-27 11:06:58,527][20347] Updated weights for policy 0, policy_version 793135 (0.0008) [2023-12-27 11:06:58,591][20347] Updated weights for policy 0, policy_version 793145 (0.0008) [2023-12-27 11:06:59,170][20381] Updated weights for policy 1, policy_version 792672 (0.0010) [2023-12-27 11:06:59,230][20381] Updated weights for policy 1, policy_version 792682 (0.0010) [2023-12-27 11:06:59,289][20381] Updated weights for policy 1, policy_version 792692 (0.0011) [2023-12-27 11:06:59,316][20347] Updated weights for policy 0, policy_version 793155 (0.0008) [2023-12-27 11:06:59,383][20347] Updated weights for policy 0, policy_version 793165 (0.0007) [2023-12-27 11:06:59,443][20347] Updated weights for policy 0, policy_version 793175 (0.0008) [2023-12-27 11:06:59,987][20381] Updated weights for policy 1, policy_version 792702 (0.0008) [2023-12-27 11:07:00,043][20381] Updated weights for policy 1, policy_version 792712 (0.0008) [2023-12-27 11:07:00,101][20381] Updated weights for policy 1, policy_version 792722 (0.0008) [2023-12-27 11:07:00,144][20347] Updated weights for policy 0, policy_version 793185 (0.0007) [2023-12-27 11:07:00,196][20347] Updated weights for policy 0, policy_version 793195 (0.0010) [2023-12-27 11:07:00,242][20347] Updated weights for policy 0, policy_version 793205 (0.0010) [2023-12-27 11:07:00,298][20347] Updated weights for policy 0, policy_version 793215 (0.0011) [2023-12-27 11:07:00,733][20381] Updated weights for policy 1, policy_version 792732 (0.0008) [2023-12-27 11:07:00,784][20381] Updated weights for policy 1, policy_version 792742 (0.0006) [2023-12-27 11:07:00,836][20381] Updated weights for policy 1, policy_version 792752 (0.0005) [2023-12-27 11:07:01,006][20347] Updated weights for policy 0, policy_version 793225 (0.0008) [2023-12-27 11:07:01,073][20347] Updated weights for policy 0, policy_version 793235 (0.0008) [2023-12-27 11:07:01,138][20347] Updated weights for policy 0, policy_version 793245 (0.0007) [2023-12-27 11:07:01,543][20381] Updated weights for policy 1, policy_version 792762 (0.0010) [2023-12-27 11:07:01,603][20381] Updated weights for policy 1, policy_version 792772 (0.0006) [2023-12-27 11:07:01,669][20381] Updated weights for policy 1, policy_version 792782 (0.0008) [2023-12-27 11:07:01,726][20381] Updated weights for policy 1, policy_version 792792 (0.0006) [2023-12-27 11:07:01,820][20347] Updated weights for policy 0, policy_version 793255 (0.0006) [2023-12-27 11:07:01,878][20347] Updated weights for policy 0, policy_version 793265 (0.0007) [2023-12-27 11:07:01,935][20347] Updated weights for policy 0, policy_version 793275 (0.0008) [2023-12-27 11:07:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 406085632. Throughput: 0: 10195.8, 1: 10391.0. Samples: 406099192. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:02,363][19170] Avg episode reward: [(0, '970.700'), (1, '988.350')] [2023-12-27 11:07:02,387][20381] Updated weights for policy 1, policy_version 792802 (0.0008) [2023-12-27 11:07:02,446][20381] Updated weights for policy 1, policy_version 792812 (0.0006) [2023-12-27 11:07:02,500][20381] Updated weights for policy 1, policy_version 792822 (0.0005) [2023-12-27 11:07:02,601][20347] Updated weights for policy 0, policy_version 793285 (0.0008) [2023-12-27 11:07:02,653][20347] Updated weights for policy 0, policy_version 793295 (0.0008) [2023-12-27 11:07:02,705][20347] Updated weights for policy 0, policy_version 793305 (0.0007) [2023-12-27 11:07:03,051][20381] Updated weights for policy 1, policy_version 792832 (0.0005) [2023-12-27 11:07:03,100][20381] Updated weights for policy 1, policy_version 792842 (0.0006) [2023-12-27 11:07:03,156][20381] Updated weights for policy 1, policy_version 792852 (0.0007) [2023-12-27 11:07:03,392][20347] Updated weights for policy 0, policy_version 793315 (0.0008) [2023-12-27 11:07:03,459][20347] Updated weights for policy 0, policy_version 793325 (0.0005) [2023-12-27 11:07:03,519][20347] Updated weights for policy 0, policy_version 793335 (0.0005) [2023-12-27 11:07:03,816][20381] Updated weights for policy 1, policy_version 792862 (0.0009) [2023-12-27 11:07:03,866][20381] Updated weights for policy 1, policy_version 792872 (0.0009) [2023-12-27 11:07:03,930][20381] Updated weights for policy 1, policy_version 792882 (0.0007) [2023-12-27 11:07:04,147][20347] Updated weights for policy 0, policy_version 793345 (0.0006) [2023-12-27 11:07:04,201][20347] Updated weights for policy 0, policy_version 793355 (0.0008) [2023-12-27 11:07:04,260][20347] Updated weights for policy 0, policy_version 793365 (0.0008) [2023-12-27 11:07:04,318][20347] Updated weights for policy 0, policy_version 793375 (0.0008) [2023-12-27 11:07:04,690][20381] Updated weights for policy 1, policy_version 792892 (0.0007) [2023-12-27 11:07:04,740][20381] Updated weights for policy 1, policy_version 792902 (0.0008) [2023-12-27 11:07:04,789][20381] Updated weights for policy 1, policy_version 792912 (0.0008) [2023-12-27 11:07:04,989][20347] Updated weights for policy 0, policy_version 793385 (0.0007) [2023-12-27 11:07:05,040][20347] Updated weights for policy 0, policy_version 793395 (0.0007) [2023-12-27 11:07:05,091][20347] Updated weights for policy 0, policy_version 793405 (0.0007) [2023-12-27 11:07:05,438][20381] Updated weights for policy 1, policy_version 792922 (0.0007) [2023-12-27 11:07:05,497][20381] Updated weights for policy 1, policy_version 792932 (0.0006) [2023-12-27 11:07:05,553][20381] Updated weights for policy 1, policy_version 792942 (0.0008) [2023-12-27 11:07:05,609][20381] Updated weights for policy 1, policy_version 792952 (0.0008) [2023-12-27 11:07:05,814][20347] Updated weights for policy 0, policy_version 793415 (0.0006) [2023-12-27 11:07:05,864][20347] Updated weights for policy 0, policy_version 793425 (0.0005) [2023-12-27 11:07:05,915][20347] Updated weights for policy 0, policy_version 793435 (0.0005) [2023-12-27 11:07:06,294][20381] Updated weights for policy 1, policy_version 792962 (0.0006) [2023-12-27 11:07:06,354][20381] Updated weights for policy 1, policy_version 792972 (0.0010) [2023-12-27 11:07:06,419][20381] Updated weights for policy 1, policy_version 792982 (0.0005) [2023-12-27 11:07:06,552][20347] Updated weights for policy 0, policy_version 793445 (0.0007) [2023-12-27 11:07:06,603][20347] Updated weights for policy 0, policy_version 793455 (0.0008) [2023-12-27 11:07:06,668][20347] Updated weights for policy 0, policy_version 793465 (0.0006) [2023-12-27 11:07:07,016][20381] Updated weights for policy 1, policy_version 792992 (0.0007) [2023-12-27 11:07:07,070][20381] Updated weights for policy 1, policy_version 793002 (0.0008) [2023-12-27 11:07:07,129][20381] Updated weights for policy 1, policy_version 793012 (0.0007) [2023-12-27 11:07:07,277][20347] Updated weights for policy 0, policy_version 793475 (0.0006) [2023-12-27 11:07:07,332][20347] Updated weights for policy 0, policy_version 793485 (0.0005) [2023-12-27 11:07:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 406192128. Throughput: 0: 10198.8, 1: 10439.5. Samples: 406161880. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:07,363][19170] Avg episode reward: [(0, '951.280'), (1, '990.270')] [2023-12-27 11:07:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000793016_203038720.pth... [2023-12-27 11:07:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000791800_202727424.pth [2023-12-27 11:07:07,390][20347] Updated weights for policy 0, policy_version 793495 (0.0008) [2023-12-27 11:07:07,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000793504_203161600.pth... [2023-12-27 11:07:07,436][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000792288_202850304.pth [2023-12-27 11:07:07,801][20381] Updated weights for policy 1, policy_version 793022 (0.0008) [2023-12-27 11:07:07,855][20381] Updated weights for policy 1, policy_version 793032 (0.0006) [2023-12-27 11:07:07,910][20381] Updated weights for policy 1, policy_version 793042 (0.0005) [2023-12-27 11:07:07,942][20347] Updated weights for policy 0, policy_version 793505 (0.0010) [2023-12-27 11:07:08,001][20347] Updated weights for policy 0, policy_version 793515 (0.0005) [2023-12-27 11:07:08,051][20347] Updated weights for policy 0, policy_version 793525 (0.0005) [2023-12-27 11:07:08,110][20347] Updated weights for policy 0, policy_version 793535 (0.0006) [2023-12-27 11:07:08,547][20381] Updated weights for policy 1, policy_version 793052 (0.0006) [2023-12-27 11:07:08,604][20381] Updated weights for policy 1, policy_version 793062 (0.0007) [2023-12-27 11:07:08,662][20381] Updated weights for policy 1, policy_version 793072 (0.0007) [2023-12-27 11:07:08,694][20347] Updated weights for policy 0, policy_version 793545 (0.0011) [2023-12-27 11:07:08,752][20347] Updated weights for policy 0, policy_version 793555 (0.0010) [2023-12-27 11:07:08,808][20347] Updated weights for policy 0, policy_version 793565 (0.0010) [2023-12-27 11:07:09,327][20381] Updated weights for policy 1, policy_version 793082 (0.0009) [2023-12-27 11:07:09,391][20381] Updated weights for policy 1, policy_version 793092 (0.0009) [2023-12-27 11:07:09,461][20381] Updated weights for policy 1, policy_version 793102 (0.0006) [2023-12-27 11:07:09,485][20347] Updated weights for policy 0, policy_version 793575 (0.0010) [2023-12-27 11:07:09,520][20381] Updated weights for policy 1, policy_version 793112 (0.0006) [2023-12-27 11:07:09,541][20347] Updated weights for policy 0, policy_version 793585 (0.0010) [2023-12-27 11:07:09,605][20347] Updated weights for policy 0, policy_version 793595 (0.0008) [2023-12-27 11:07:10,183][20381] Updated weights for policy 1, policy_version 793122 (0.0007) [2023-12-27 11:07:10,244][20381] Updated weights for policy 1, policy_version 793132 (0.0007) [2023-12-27 11:07:10,304][20381] Updated weights for policy 1, policy_version 793142 (0.0007) [2023-12-27 11:07:10,317][20347] Updated weights for policy 0, policy_version 793605 (0.0009) [2023-12-27 11:07:10,377][20347] Updated weights for policy 0, policy_version 793615 (0.0008) [2023-12-27 11:07:10,434][20347] Updated weights for policy 0, policy_version 793625 (0.0007) [2023-12-27 11:07:11,007][20381] Updated weights for policy 1, policy_version 793152 (0.0010) [2023-12-27 11:07:11,055][20347] Updated weights for policy 0, policy_version 793635 (0.0008) [2023-12-27 11:07:11,071][20381] Updated weights for policy 1, policy_version 793162 (0.0009) [2023-12-27 11:07:11,114][20347] Updated weights for policy 0, policy_version 793645 (0.0008) [2023-12-27 11:07:11,126][20381] Updated weights for policy 1, policy_version 793172 (0.0010) [2023-12-27 11:07:11,178][20347] Updated weights for policy 0, policy_version 793655 (0.0007) [2023-12-27 11:07:11,855][20381] Updated weights for policy 1, policy_version 793182 (0.0007) [2023-12-27 11:07:11,912][20347] Updated weights for policy 0, policy_version 793665 (0.0006) [2023-12-27 11:07:11,915][20381] Updated weights for policy 1, policy_version 793192 (0.0007) [2023-12-27 11:07:11,964][20347] Updated weights for policy 0, policy_version 793675 (0.0010) [2023-12-27 11:07:11,976][20381] Updated weights for policy 1, policy_version 793202 (0.0006) [2023-12-27 11:07:12,018][20347] Updated weights for policy 0, policy_version 793685 (0.0010) [2023-12-27 11:07:12,073][20347] Updated weights for policy 0, policy_version 793695 (0.0010) [2023-12-27 11:07:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 406298624. Throughput: 0: 10269.1, 1: 10420.8. Samples: 406288420. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:12,363][19170] Avg episode reward: [(0, '961.050'), (1, '970.850')] [2023-12-27 11:07:12,634][20381] Updated weights for policy 1, policy_version 793212 (0.0007) [2023-12-27 11:07:12,690][20381] Updated weights for policy 1, policy_version 793222 (0.0007) [2023-12-27 11:07:12,747][20381] Updated weights for policy 1, policy_version 793232 (0.0007) [2023-12-27 11:07:12,784][20347] Updated weights for policy 0, policy_version 793705 (0.0007) [2023-12-27 11:07:12,840][20347] Updated weights for policy 0, policy_version 793715 (0.0007) [2023-12-27 11:07:12,898][20347] Updated weights for policy 0, policy_version 793725 (0.0006) [2023-12-27 11:07:13,389][20381] Updated weights for policy 1, policy_version 793242 (0.0007) [2023-12-27 11:07:13,447][20381] Updated weights for policy 1, policy_version 793252 (0.0006) [2023-12-27 11:07:13,507][20381] Updated weights for policy 1, policy_version 793262 (0.0006) [2023-12-27 11:07:13,533][20347] Updated weights for policy 0, policy_version 793735 (0.0009) [2023-12-27 11:07:13,561][20381] Updated weights for policy 1, policy_version 793272 (0.0006) [2023-12-27 11:07:13,584][20347] Updated weights for policy 0, policy_version 793745 (0.0010) [2023-12-27 11:07:13,642][20347] Updated weights for policy 0, policy_version 793755 (0.0010) [2023-12-27 11:07:14,222][20381] Updated weights for policy 1, policy_version 793282 (0.0008) [2023-12-27 11:07:14,276][20381] Updated weights for policy 1, policy_version 793292 (0.0007) [2023-12-27 11:07:14,298][20347] Updated weights for policy 0, policy_version 793765 (0.0010) [2023-12-27 11:07:14,330][20381] Updated weights for policy 1, policy_version 793302 (0.0007) [2023-12-27 11:07:14,353][20347] Updated weights for policy 0, policy_version 793775 (0.0010) [2023-12-27 11:07:14,411][20347] Updated weights for policy 0, policy_version 793785 (0.0010) [2023-12-27 11:07:15,022][20381] Updated weights for policy 1, policy_version 793312 (0.0008) [2023-12-27 11:07:15,073][20381] Updated weights for policy 1, policy_version 793322 (0.0007) [2023-12-27 11:07:15,124][20347] Updated weights for policy 0, policy_version 793795 (0.0010) [2023-12-27 11:07:15,127][20381] Updated weights for policy 1, policy_version 793332 (0.0008) [2023-12-27 11:07:15,179][20347] Updated weights for policy 0, policy_version 793805 (0.0010) [2023-12-27 11:07:15,231][20347] Updated weights for policy 0, policy_version 793815 (0.0010) [2023-12-27 11:07:15,857][20381] Updated weights for policy 1, policy_version 793342 (0.0008) [2023-12-27 11:07:15,897][20347] Updated weights for policy 0, policy_version 793825 (0.0010) [2023-12-27 11:07:15,913][20381] Updated weights for policy 1, policy_version 793352 (0.0007) [2023-12-27 11:07:15,945][20347] Updated weights for policy 0, policy_version 793835 (0.0010) [2023-12-27 11:07:15,968][20381] Updated weights for policy 1, policy_version 793362 (0.0005) [2023-12-27 11:07:15,993][20347] Updated weights for policy 0, policy_version 793845 (0.0010) [2023-12-27 11:07:16,041][20347] Updated weights for policy 0, policy_version 793855 (0.0010) [2023-12-27 11:07:16,654][20347] Updated weights for policy 0, policy_version 793865 (0.0010) [2023-12-27 11:07:16,660][20381] Updated weights for policy 1, policy_version 793372 (0.0005) [2023-12-27 11:07:16,712][20347] Updated weights for policy 0, policy_version 793875 (0.0007) [2023-12-27 11:07:16,712][20381] Updated weights for policy 1, policy_version 793382 (0.0009) [2023-12-27 11:07:16,772][20347] Updated weights for policy 0, policy_version 793885 (0.0007) [2023-12-27 11:07:16,779][20381] Updated weights for policy 1, policy_version 793392 (0.0010) [2023-12-27 11:07:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 406396928. Throughput: 0: 10308.2, 1: 10333.7. Samples: 406411392. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:17,363][19170] Avg episode reward: [(0, '980.770'), (1, '970.870')] [2023-12-27 11:07:17,389][20347] Updated weights for policy 0, policy_version 793895 (0.0006) [2023-12-27 11:07:17,431][20381] Updated weights for policy 1, policy_version 793402 (0.0011) [2023-12-27 11:07:17,444][20347] Updated weights for policy 0, policy_version 793905 (0.0007) [2023-12-27 11:07:17,493][20381] Updated weights for policy 1, policy_version 793412 (0.0011) [2023-12-27 11:07:17,501][20347] Updated weights for policy 0, policy_version 793915 (0.0005) [2023-12-27 11:07:17,549][20381] Updated weights for policy 1, policy_version 793422 (0.0010) [2023-12-27 11:07:17,603][20381] Updated weights for policy 1, policy_version 793432 (0.0010) [2023-12-27 11:07:18,064][20347] Updated weights for policy 0, policy_version 793925 (0.0006) [2023-12-27 11:07:18,120][20347] Updated weights for policy 0, policy_version 793935 (0.0008) [2023-12-27 11:07:18,172][20347] Updated weights for policy 0, policy_version 793945 (0.0007) [2023-12-27 11:07:18,264][20381] Updated weights for policy 1, policy_version 793442 (0.0006) [2023-12-27 11:07:18,325][20381] Updated weights for policy 1, policy_version 793452 (0.0008) [2023-12-27 11:07:18,385][20381] Updated weights for policy 1, policy_version 793462 (0.0008) [2023-12-27 11:07:18,904][20347] Updated weights for policy 0, policy_version 793955 (0.0007) [2023-12-27 11:07:18,954][20347] Updated weights for policy 0, policy_version 793965 (0.0007) [2023-12-27 11:07:19,002][20347] Updated weights for policy 0, policy_version 793975 (0.0008) [2023-12-27 11:07:19,051][20381] Updated weights for policy 1, policy_version 793472 (0.0008) [2023-12-27 11:07:19,097][20381] Updated weights for policy 1, policy_version 793482 (0.0007) [2023-12-27 11:07:19,144][20381] Updated weights for policy 1, policy_version 793492 (0.0007) [2023-12-27 11:07:19,724][20347] Updated weights for policy 0, policy_version 793985 (0.0009) [2023-12-27 11:07:19,784][20347] Updated weights for policy 0, policy_version 793995 (0.0007) [2023-12-27 11:07:19,846][20347] Updated weights for policy 0, policy_version 794005 (0.0008) [2023-12-27 11:07:19,859][20381] Updated weights for policy 1, policy_version 793502 (0.0009) [2023-12-27 11:07:19,905][20347] Updated weights for policy 0, policy_version 794015 (0.0006) [2023-12-27 11:07:19,919][20381] Updated weights for policy 1, policy_version 793512 (0.0007) [2023-12-27 11:07:19,980][20381] Updated weights for policy 1, policy_version 793522 (0.0007) [2023-12-27 11:07:20,614][20347] Updated weights for policy 0, policy_version 794025 (0.0007) [2023-12-27 11:07:20,677][20347] Updated weights for policy 0, policy_version 794035 (0.0007) [2023-12-27 11:07:20,692][20381] Updated weights for policy 1, policy_version 793532 (0.0009) [2023-12-27 11:07:20,737][20347] Updated weights for policy 0, policy_version 794045 (0.0006) [2023-12-27 11:07:20,747][20381] Updated weights for policy 1, policy_version 793542 (0.0011) [2023-12-27 11:07:20,804][20381] Updated weights for policy 1, policy_version 793552 (0.0011) [2023-12-27 11:07:21,475][20347] Updated weights for policy 0, policy_version 794055 (0.0007) [2023-12-27 11:07:21,487][20381] Updated weights for policy 1, policy_version 793562 (0.0009) [2023-12-27 11:07:21,535][20347] Updated weights for policy 0, policy_version 794065 (0.0007) [2023-12-27 11:07:21,546][20381] Updated weights for policy 1, policy_version 793572 (0.0008) [2023-12-27 11:07:21,593][20347] Updated weights for policy 0, policy_version 794075 (0.0007) [2023-12-27 11:07:21,606][20381] Updated weights for policy 1, policy_version 793582 (0.0007) [2023-12-27 11:07:21,674][20381] Updated weights for policy 1, policy_version 793592 (0.0009) [2023-12-27 11:07:22,320][20347] Updated weights for policy 0, policy_version 794085 (0.0007) [2023-12-27 11:07:22,351][20381] Updated weights for policy 1, policy_version 793602 (0.0008) [2023-12-27 11:07:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 406495232. Throughput: 0: 10339.8, 1: 10319.2. Samples: 406473024. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:22,363][19170] Avg episode reward: [(0, '970.970'), (1, '970.910')] [2023-12-27 11:07:22,387][20347] Updated weights for policy 0, policy_version 794095 (0.0006) [2023-12-27 11:07:22,412][20381] Updated weights for policy 1, policy_version 793612 (0.0007) [2023-12-27 11:07:22,448][20347] Updated weights for policy 0, policy_version 794105 (0.0006) [2023-12-27 11:07:22,465][20381] Updated weights for policy 1, policy_version 793622 (0.0006) [2023-12-27 11:07:22,476][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000793624_203194368.pth... [2023-12-27 11:07:22,480][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000792408_202883072.pth [2023-12-27 11:07:22,482][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000794112_203317248.pth... [2023-12-27 11:07:22,485][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000792896_203005952.pth [2023-12-27 11:07:23,142][20381] Updated weights for policy 1, policy_version 793632 (0.0007) [2023-12-27 11:07:23,150][20347] Updated weights for policy 0, policy_version 794115 (0.0006) [2023-12-27 11:07:23,200][20381] Updated weights for policy 1, policy_version 793642 (0.0008) [2023-12-27 11:07:23,209][20347] Updated weights for policy 0, policy_version 794125 (0.0007) [2023-12-27 11:07:23,255][20381] Updated weights for policy 1, policy_version 793652 (0.0009) [2023-12-27 11:07:23,259][20347] Updated weights for policy 0, policy_version 794135 (0.0010) [2023-12-27 11:07:23,885][20347] Updated weights for policy 0, policy_version 794145 (0.0007) [2023-12-27 11:07:23,944][20347] Updated weights for policy 0, policy_version 794155 (0.0010) [2023-12-27 11:07:23,959][20381] Updated weights for policy 1, policy_version 793662 (0.0006) [2023-12-27 11:07:24,002][20347] Updated weights for policy 0, policy_version 794165 (0.0010) [2023-12-27 11:07:24,015][20381] Updated weights for policy 1, policy_version 793672 (0.0005) [2023-12-27 11:07:24,060][20347] Updated weights for policy 0, policy_version 794175 (0.0010) [2023-12-27 11:07:24,070][20381] Updated weights for policy 1, policy_version 793682 (0.0005) [2023-12-27 11:07:24,660][20347] Updated weights for policy 0, policy_version 794185 (0.0008) [2023-12-27 11:07:24,709][20381] Updated weights for policy 1, policy_version 793692 (0.0006) [2023-12-27 11:07:24,711][20347] Updated weights for policy 0, policy_version 794195 (0.0008) [2023-12-27 11:07:24,760][20347] Updated weights for policy 0, policy_version 794205 (0.0006) [2023-12-27 11:07:24,765][20381] Updated weights for policy 1, policy_version 793702 (0.0006) [2023-12-27 11:07:24,813][20381] Updated weights for policy 1, policy_version 793712 (0.0007) [2023-12-27 11:07:25,428][20347] Updated weights for policy 0, policy_version 794215 (0.0005) [2023-12-27 11:07:25,481][20347] Updated weights for policy 0, policy_version 794225 (0.0005) [2023-12-27 11:07:25,504][20381] Updated weights for policy 1, policy_version 793722 (0.0007) [2023-12-27 11:07:25,535][20347] Updated weights for policy 0, policy_version 794235 (0.0006) [2023-12-27 11:07:25,555][20381] Updated weights for policy 1, policy_version 793732 (0.0005) [2023-12-27 11:07:25,614][20381] Updated weights for policy 1, policy_version 793742 (0.0008) [2023-12-27 11:07:25,659][20381] Updated weights for policy 1, policy_version 793752 (0.0006) [2023-12-27 11:07:26,161][20347] Updated weights for policy 0, policy_version 794245 (0.0007) [2023-12-27 11:07:26,206][20347] Updated weights for policy 0, policy_version 794255 (0.0005) [2023-12-27 11:07:26,277][20347] Updated weights for policy 0, policy_version 794265 (0.0006) [2023-12-27 11:07:26,367][20381] Updated weights for policy 1, policy_version 793762 (0.0007) [2023-12-27 11:07:26,418][20381] Updated weights for policy 1, policy_version 793772 (0.0007) [2023-12-27 11:07:26,464][20381] Updated weights for policy 1, policy_version 793782 (0.0008) [2023-12-27 11:07:26,896][20347] Updated weights for policy 0, policy_version 794275 (0.0007) [2023-12-27 11:07:26,947][20347] Updated weights for policy 0, policy_version 794285 (0.0006) [2023-12-27 11:07:26,992][20347] Updated weights for policy 0, policy_version 794295 (0.0009) [2023-12-27 11:07:27,137][20381] Updated weights for policy 1, policy_version 793792 (0.0009) [2023-12-27 11:07:27,191][20381] Updated weights for policy 1, policy_version 793802 (0.0006) [2023-12-27 11:07:27,249][20381] Updated weights for policy 1, policy_version 793812 (0.0008) [2023-12-27 11:07:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 406609920. Throughput: 0: 10328.2, 1: 10346.2. Samples: 406595168. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:27,363][19170] Avg episode reward: [(0, '960.900'), (1, '970.860')] [2023-12-27 11:07:27,591][20347] Updated weights for policy 0, policy_version 794305 (0.0009) [2023-12-27 11:07:27,635][20347] Updated weights for policy 0, policy_version 794315 (0.0010) [2023-12-27 11:07:27,695][20347] Updated weights for policy 0, policy_version 794325 (0.0010) [2023-12-27 11:07:27,749][20347] Updated weights for policy 0, policy_version 794335 (0.0010) [2023-12-27 11:07:27,917][20381] Updated weights for policy 1, policy_version 793822 (0.0008) [2023-12-27 11:07:27,962][20381] Updated weights for policy 1, policy_version 793832 (0.0007) [2023-12-27 11:07:28,017][20381] Updated weights for policy 1, policy_version 793842 (0.0007) [2023-12-27 11:07:28,415][20347] Updated weights for policy 0, policy_version 794345 (0.0009) [2023-12-27 11:07:28,474][20347] Updated weights for policy 0, policy_version 794355 (0.0007) [2023-12-27 11:07:28,529][20347] Updated weights for policy 0, policy_version 794365 (0.0005) [2023-12-27 11:07:28,762][20381] Updated weights for policy 1, policy_version 793852 (0.0008) [2023-12-27 11:07:28,818][20381] Updated weights for policy 1, policy_version 793862 (0.0006) [2023-12-27 11:07:28,878][20381] Updated weights for policy 1, policy_version 793872 (0.0006) [2023-12-27 11:07:29,121][20347] Updated weights for policy 0, policy_version 794375 (0.0007) [2023-12-27 11:07:29,171][20347] Updated weights for policy 0, policy_version 794385 (0.0008) [2023-12-27 11:07:29,224][20347] Updated weights for policy 0, policy_version 794395 (0.0008) [2023-12-27 11:07:29,553][20381] Updated weights for policy 1, policy_version 793882 (0.0005) [2023-12-27 11:07:29,613][20381] Updated weights for policy 1, policy_version 793892 (0.0006) [2023-12-27 11:07:29,667][20381] Updated weights for policy 1, policy_version 793902 (0.0005) [2023-12-27 11:07:29,722][20381] Updated weights for policy 1, policy_version 793912 (0.0007) [2023-12-27 11:07:29,989][20347] Updated weights for policy 0, policy_version 794405 (0.0009) [2023-12-27 11:07:30,051][20347] Updated weights for policy 0, policy_version 794415 (0.0008) [2023-12-27 11:07:30,101][20347] Updated weights for policy 0, policy_version 794425 (0.0008) [2023-12-27 11:07:30,377][20381] Updated weights for policy 1, policy_version 793922 (0.0006) [2023-12-27 11:07:30,427][20381] Updated weights for policy 1, policy_version 793932 (0.0005) [2023-12-27 11:07:30,483][20381] Updated weights for policy 1, policy_version 793942 (0.0006) [2023-12-27 11:07:30,756][20347] Updated weights for policy 0, policy_version 794435 (0.0007) [2023-12-27 11:07:30,802][20347] Updated weights for policy 0, policy_version 794445 (0.0006) [2023-12-27 11:07:30,847][20347] Updated weights for policy 0, policy_version 794455 (0.0005) [2023-12-27 11:07:31,122][20381] Updated weights for policy 1, policy_version 793952 (0.0008) [2023-12-27 11:07:31,188][20381] Updated weights for policy 1, policy_version 793962 (0.0010) [2023-12-27 11:07:31,248][20381] Updated weights for policy 1, policy_version 793972 (0.0007) [2023-12-27 11:07:31,537][20347] Updated weights for policy 0, policy_version 794465 (0.0008) [2023-12-27 11:07:31,602][20347] Updated weights for policy 0, policy_version 794475 (0.0007) [2023-12-27 11:07:31,666][20347] Updated weights for policy 0, policy_version 794485 (0.0007) [2023-12-27 11:07:31,732][20347] Updated weights for policy 0, policy_version 794495 (0.0008) [2023-12-27 11:07:31,913][20381] Updated weights for policy 1, policy_version 793982 (0.0006) [2023-12-27 11:07:31,961][20381] Updated weights for policy 1, policy_version 793992 (0.0010) [2023-12-27 11:07:32,014][20381] Updated weights for policy 1, policy_version 794002 (0.0009) [2023-12-27 11:07:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 406708224. Throughput: 0: 10409.8, 1: 10337.9. Samples: 406719804. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:32,363][19170] Avg episode reward: [(0, '951.170'), (1, '961.170')] [2023-12-27 11:07:32,408][20347] Updated weights for policy 0, policy_version 794505 (0.0008) [2023-12-27 11:07:32,463][20347] Updated weights for policy 0, policy_version 794515 (0.0008) [2023-12-27 11:07:32,528][20347] Updated weights for policy 0, policy_version 794525 (0.0008) [2023-12-27 11:07:32,727][20381] Updated weights for policy 1, policy_version 794012 (0.0007) [2023-12-27 11:07:32,785][20381] Updated weights for policy 1, policy_version 794022 (0.0007) [2023-12-27 11:07:32,843][20381] Updated weights for policy 1, policy_version 794032 (0.0007) [2023-12-27 11:07:33,231][20347] Updated weights for policy 0, policy_version 794535 (0.0008) [2023-12-27 11:07:33,262][20291] KL-divergence is very high: 131.5954 [2023-12-27 11:07:33,284][20347] Updated weights for policy 0, policy_version 794545 (0.0007) [2023-12-27 11:07:33,337][20347] Updated weights for policy 0, policy_version 794555 (0.0007) [2023-12-27 11:07:33,474][20381] Updated weights for policy 1, policy_version 794042 (0.0008) [2023-12-27 11:07:33,527][20381] Updated weights for policy 1, policy_version 794052 (0.0010) [2023-12-27 11:07:33,581][20381] Updated weights for policy 1, policy_version 794062 (0.0010) [2023-12-27 11:07:33,639][20381] Updated weights for policy 1, policy_version 794072 (0.0010) [2023-12-27 11:07:34,025][20347] Updated weights for policy 0, policy_version 794565 (0.0008) [2023-12-27 11:07:34,083][20347] Updated weights for policy 0, policy_version 794575 (0.0007) [2023-12-27 11:07:34,137][20347] Updated weights for policy 0, policy_version 794585 (0.0007) [2023-12-27 11:07:34,344][20381] Updated weights for policy 1, policy_version 794082 (0.0010) [2023-12-27 11:07:34,404][20381] Updated weights for policy 1, policy_version 794092 (0.0008) [2023-12-27 11:07:34,467][20381] Updated weights for policy 1, policy_version 794102 (0.0006) [2023-12-27 11:07:34,789][20347] Updated weights for policy 0, policy_version 794595 (0.0008) [2023-12-27 11:07:34,846][20347] Updated weights for policy 0, policy_version 794605 (0.0007) [2023-12-27 11:07:34,911][20347] Updated weights for policy 0, policy_version 794615 (0.0007) [2023-12-27 11:07:35,131][20381] Updated weights for policy 1, policy_version 794112 (0.0006) [2023-12-27 11:07:35,182][20381] Updated weights for policy 1, policy_version 794122 (0.0008) [2023-12-27 11:07:35,239][20381] Updated weights for policy 1, policy_version 794132 (0.0007) [2023-12-27 11:07:35,615][20347] Updated weights for policy 0, policy_version 794625 (0.0008) [2023-12-27 11:07:35,678][20347] Updated weights for policy 0, policy_version 794635 (0.0009) [2023-12-27 11:07:35,742][20347] Updated weights for policy 0, policy_version 794645 (0.0007) [2023-12-27 11:07:35,792][20347] Updated weights for policy 0, policy_version 794655 (0.0006) [2023-12-27 11:07:35,854][20381] Updated weights for policy 1, policy_version 794142 (0.0006) [2023-12-27 11:07:35,905][20381] Updated weights for policy 1, policy_version 794152 (0.0005) [2023-12-27 11:07:35,960][20381] Updated weights for policy 1, policy_version 794162 (0.0005) [2023-12-27 11:07:36,438][20347] Updated weights for policy 0, policy_version 794665 (0.0007) [2023-12-27 11:07:36,496][20347] Updated weights for policy 0, policy_version 794675 (0.0008) [2023-12-27 11:07:36,556][20347] Updated weights for policy 0, policy_version 794685 (0.0008) [2023-12-27 11:07:36,586][20381] Updated weights for policy 1, policy_version 794172 (0.0006) [2023-12-27 11:07:36,642][20381] Updated weights for policy 1, policy_version 794182 (0.0007) [2023-12-27 11:07:36,698][20381] Updated weights for policy 1, policy_version 794192 (0.0008) [2023-12-27 11:07:37,222][20347] Updated weights for policy 0, policy_version 794695 (0.0010) [2023-12-27 11:07:37,279][20347] Updated weights for policy 0, policy_version 794705 (0.0010) [2023-12-27 11:07:37,341][20347] Updated weights for policy 0, policy_version 794715 (0.0010) [2023-12-27 11:07:37,343][20381] Updated weights for policy 1, policy_version 794202 (0.0007) [2023-12-27 11:07:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 406806528. Throughput: 0: 10373.7, 1: 10351.4. Samples: 406780872. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:37,363][19170] Avg episode reward: [(0, '941.440'), (1, '961.430')] [2023-12-27 11:07:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000794720_203472896.pth... [2023-12-27 11:07:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000793504_203161600.pth [2023-12-27 11:07:37,400][20381] Updated weights for policy 1, policy_version 794212 (0.0006) [2023-12-27 11:07:37,459][20381] Updated weights for policy 1, policy_version 794222 (0.0008) [2023-12-27 11:07:37,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000794232_203350016.pth... [2023-12-27 11:07:37,521][20381] Updated weights for policy 1, policy_version 794232 (0.0007) [2023-12-27 11:07:37,522][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000793016_203038720.pth [2023-12-27 11:07:38,067][20347] Updated weights for policy 0, policy_version 794725 (0.0008) [2023-12-27 11:07:38,102][20381] Updated weights for policy 1, policy_version 794242 (0.0007) [2023-12-27 11:07:38,119][20347] Updated weights for policy 0, policy_version 794735 (0.0006) [2023-12-27 11:07:38,148][20381] Updated weights for policy 1, policy_version 794252 (0.0005) [2023-12-27 11:07:38,172][20347] Updated weights for policy 0, policy_version 794745 (0.0007) [2023-12-27 11:07:38,193][20381] Updated weights for policy 1, policy_version 794262 (0.0005) [2023-12-27 11:07:38,831][20347] Updated weights for policy 0, policy_version 794755 (0.0006) [2023-12-27 11:07:38,871][20381] Updated weights for policy 1, policy_version 794272 (0.0006) [2023-12-27 11:07:38,887][20347] Updated weights for policy 0, policy_version 794765 (0.0006) [2023-12-27 11:07:38,929][20381] Updated weights for policy 1, policy_version 794282 (0.0006) [2023-12-27 11:07:38,946][20347] Updated weights for policy 0, policy_version 794775 (0.0007) [2023-12-27 11:07:38,988][20381] Updated weights for policy 1, policy_version 794292 (0.0006) [2023-12-27 11:07:39,688][20347] Updated weights for policy 0, policy_version 794785 (0.0007) [2023-12-27 11:07:39,691][20381] Updated weights for policy 1, policy_version 794302 (0.0008) [2023-12-27 11:07:39,745][20347] Updated weights for policy 0, policy_version 794795 (0.0005) [2023-12-27 11:07:39,746][20381] Updated weights for policy 1, policy_version 794312 (0.0007) [2023-12-27 11:07:39,803][20381] Updated weights for policy 1, policy_version 794322 (0.0007) [2023-12-27 11:07:39,806][20347] Updated weights for policy 0, policy_version 794805 (0.0006) [2023-12-27 11:07:39,871][20347] Updated weights for policy 0, policy_version 794815 (0.0008) [2023-12-27 11:07:40,538][20381] Updated weights for policy 1, policy_version 794332 (0.0008) [2023-12-27 11:07:40,574][20347] Updated weights for policy 0, policy_version 794825 (0.0006) [2023-12-27 11:07:40,598][20381] Updated weights for policy 1, policy_version 794342 (0.0006) [2023-12-27 11:07:40,633][20347] Updated weights for policy 0, policy_version 794835 (0.0006) [2023-12-27 11:07:40,659][20381] Updated weights for policy 1, policy_version 794352 (0.0007) [2023-12-27 11:07:40,689][20347] Updated weights for policy 0, policy_version 794845 (0.0006) [2023-12-27 11:07:41,309][20347] Updated weights for policy 0, policy_version 794855 (0.0008) [2023-12-27 11:07:41,359][20381] Updated weights for policy 1, policy_version 794362 (0.0008) [2023-12-27 11:07:41,372][20347] Updated weights for policy 0, policy_version 794865 (0.0007) [2023-12-27 11:07:41,427][20381] Updated weights for policy 1, policy_version 794372 (0.0008) [2023-12-27 11:07:41,443][20347] Updated weights for policy 0, policy_version 794875 (0.0007) [2023-12-27 11:07:41,483][20381] Updated weights for policy 1, policy_version 794382 (0.0008) [2023-12-27 11:07:41,533][20381] Updated weights for policy 1, policy_version 794392 (0.0008) [2023-12-27 11:07:42,124][20347] Updated weights for policy 0, policy_version 794885 (0.0008) [2023-12-27 11:07:42,176][20347] Updated weights for policy 0, policy_version 794895 (0.0010) [2023-12-27 11:07:42,232][20347] Updated weights for policy 0, policy_version 794905 (0.0010) [2023-12-27 11:07:42,270][20381] Updated weights for policy 1, policy_version 794402 (0.0006) [2023-12-27 11:07:42,327][20381] Updated weights for policy 1, policy_version 794412 (0.0008) [2023-12-27 11:07:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 406913024. Throughput: 0: 10306.5, 1: 10344.5. Samples: 406904728. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:42,363][19170] Avg episode reward: [(0, '970.690'), (1, '980.760')] [2023-12-27 11:07:42,392][20381] Updated weights for policy 1, policy_version 794422 (0.0008) [2023-12-27 11:07:42,884][20347] Updated weights for policy 0, policy_version 794915 (0.0010) [2023-12-27 11:07:42,938][20347] Updated weights for policy 0, policy_version 794925 (0.0007) [2023-12-27 11:07:43,001][20347] Updated weights for policy 0, policy_version 794935 (0.0007) [2023-12-27 11:07:43,092][20381] Updated weights for policy 1, policy_version 794432 (0.0007) [2023-12-27 11:07:43,146][20381] Updated weights for policy 1, policy_version 794442 (0.0008) [2023-12-27 11:07:43,193][20381] Updated weights for policy 1, policy_version 794452 (0.0009) [2023-12-27 11:07:43,628][20347] Updated weights for policy 0, policy_version 794945 (0.0005) [2023-12-27 11:07:43,692][20347] Updated weights for policy 0, policy_version 794955 (0.0007) [2023-12-27 11:07:43,748][20347] Updated weights for policy 0, policy_version 794965 (0.0007) [2023-12-27 11:07:43,806][20347] Updated weights for policy 0, policy_version 794975 (0.0008) [2023-12-27 11:07:43,881][20381] Updated weights for policy 1, policy_version 794462 (0.0009) [2023-12-27 11:07:43,927][20381] Updated weights for policy 1, policy_version 794472 (0.0010) [2023-12-27 11:07:43,979][20381] Updated weights for policy 1, policy_version 794482 (0.0009) [2023-12-27 11:07:44,423][20347] Updated weights for policy 0, policy_version 794985 (0.0007) [2023-12-27 11:07:44,487][20347] Updated weights for policy 0, policy_version 794995 (0.0007) [2023-12-27 11:07:44,542][20347] Updated weights for policy 0, policy_version 795005 (0.0006) [2023-12-27 11:07:44,673][20381] Updated weights for policy 1, policy_version 794492 (0.0009) [2023-12-27 11:07:44,725][20381] Updated weights for policy 1, policy_version 794502 (0.0010) [2023-12-27 11:07:44,788][20381] Updated weights for policy 1, policy_version 794512 (0.0009) [2023-12-27 11:07:45,181][20347] Updated weights for policy 0, policy_version 795015 (0.0009) [2023-12-27 11:07:45,248][20347] Updated weights for policy 0, policy_version 795025 (0.0010) [2023-12-27 11:07:45,303][20347] Updated weights for policy 0, policy_version 795035 (0.0010) [2023-12-27 11:07:45,491][20381] Updated weights for policy 1, policy_version 794522 (0.0008) [2023-12-27 11:07:45,544][20381] Updated weights for policy 1, policy_version 794532 (0.0011) [2023-12-27 11:07:45,599][20381] Updated weights for policy 1, policy_version 794542 (0.0010) [2023-12-27 11:07:45,655][20381] Updated weights for policy 1, policy_version 794552 (0.0006) [2023-12-27 11:07:45,965][20347] Updated weights for policy 0, policy_version 795045 (0.0009) [2023-12-27 11:07:46,009][20347] Updated weights for policy 0, policy_version 795055 (0.0007) [2023-12-27 11:07:46,056][20347] Updated weights for policy 0, policy_version 795065 (0.0007) [2023-12-27 11:07:46,262][20381] Updated weights for policy 1, policy_version 794562 (0.0010) [2023-12-27 11:07:46,316][20381] Updated weights for policy 1, policy_version 794572 (0.0010) [2023-12-27 11:07:46,370][20381] Updated weights for policy 1, policy_version 794582 (0.0010) [2023-12-27 11:07:46,714][20347] Updated weights for policy 0, policy_version 795075 (0.0008) [2023-12-27 11:07:46,762][20347] Updated weights for policy 0, policy_version 795085 (0.0008) [2023-12-27 11:07:46,820][20347] Updated weights for policy 0, policy_version 795095 (0.0007) [2023-12-27 11:07:47,074][20381] Updated weights for policy 1, policy_version 794592 (0.0008) [2023-12-27 11:07:47,139][20381] Updated weights for policy 1, policy_version 794602 (0.0008) [2023-12-27 11:07:47,195][20381] Updated weights for policy 1, policy_version 794612 (0.0007) [2023-12-27 11:07:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 407019520. Throughput: 0: 10369.0, 1: 10255.5. Samples: 407027296. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:47,363][19170] Avg episode reward: [(0, '941.540'), (1, '980.540')] [2023-12-27 11:07:47,534][20347] Updated weights for policy 0, policy_version 795105 (0.0008) [2023-12-27 11:07:47,592][20347] Updated weights for policy 0, policy_version 795115 (0.0010) [2023-12-27 11:07:47,650][20347] Updated weights for policy 0, policy_version 795125 (0.0010) [2023-12-27 11:07:47,714][20347] Updated weights for policy 0, policy_version 795135 (0.0010) [2023-12-27 11:07:47,834][20381] Updated weights for policy 1, policy_version 794622 (0.0006) [2023-12-27 11:07:47,888][20381] Updated weights for policy 1, policy_version 794632 (0.0005) [2023-12-27 11:07:47,943][20381] Updated weights for policy 1, policy_version 794642 (0.0005) [2023-12-27 11:07:48,366][20347] Updated weights for policy 0, policy_version 795145 (0.0008) [2023-12-27 11:07:48,422][20347] Updated weights for policy 0, policy_version 795155 (0.0007) [2023-12-27 11:07:48,477][20347] Updated weights for policy 0, policy_version 795165 (0.0007) [2023-12-27 11:07:48,545][20381] Updated weights for policy 1, policy_version 794652 (0.0005) [2023-12-27 11:07:48,611][20381] Updated weights for policy 1, policy_version 794662 (0.0007) [2023-12-27 11:07:48,671][20381] Updated weights for policy 1, policy_version 794672 (0.0010) [2023-12-27 11:07:49,194][20347] Updated weights for policy 0, policy_version 795175 (0.0008) [2023-12-27 11:07:49,257][20347] Updated weights for policy 0, policy_version 795185 (0.0008) [2023-12-27 11:07:49,325][20347] Updated weights for policy 0, policy_version 795195 (0.0006) [2023-12-27 11:07:49,334][20381] Updated weights for policy 1, policy_version 794682 (0.0009) [2023-12-27 11:07:49,398][20381] Updated weights for policy 1, policy_version 794692 (0.0007) [2023-12-27 11:07:49,453][20381] Updated weights for policy 1, policy_version 794702 (0.0008) [2023-12-27 11:07:49,514][20381] Updated weights for policy 1, policy_version 794712 (0.0008) [2023-12-27 11:07:50,053][20347] Updated weights for policy 0, policy_version 795205 (0.0006) [2023-12-27 11:07:50,110][20347] Updated weights for policy 0, policy_version 795215 (0.0007) [2023-12-27 11:07:50,171][20347] Updated weights for policy 0, policy_version 795225 (0.0006) [2023-12-27 11:07:50,261][20381] Updated weights for policy 1, policy_version 794722 (0.0007) [2023-12-27 11:07:50,306][20381] Updated weights for policy 1, policy_version 794732 (0.0008) [2023-12-27 11:07:50,357][20381] Updated weights for policy 1, policy_version 794742 (0.0007) [2023-12-27 11:07:50,789][20347] Updated weights for policy 0, policy_version 795235 (0.0007) [2023-12-27 11:07:50,842][20347] Updated weights for policy 0, policy_version 795245 (0.0008) [2023-12-27 11:07:50,893][20347] Updated weights for policy 0, policy_version 795255 (0.0007) [2023-12-27 11:07:51,072][20381] Updated weights for policy 1, policy_version 794752 (0.0008) [2023-12-27 11:07:51,138][20381] Updated weights for policy 1, policy_version 794762 (0.0008) [2023-12-27 11:07:51,195][20381] Updated weights for policy 1, policy_version 794772 (0.0008) [2023-12-27 11:07:51,576][20347] Updated weights for policy 0, policy_version 795265 (0.0007) [2023-12-27 11:07:51,638][20347] Updated weights for policy 0, policy_version 795275 (0.0009) [2023-12-27 11:07:51,708][20347] Updated weights for policy 0, policy_version 795285 (0.0009) [2023-12-27 11:07:51,770][20347] Updated weights for policy 0, policy_version 795295 (0.0007) [2023-12-27 11:07:51,914][20381] Updated weights for policy 1, policy_version 794782 (0.0006) [2023-12-27 11:07:51,980][20381] Updated weights for policy 1, policy_version 794792 (0.0007) [2023-12-27 11:07:52,040][20381] Updated weights for policy 1, policy_version 794802 (0.0008) [2023-12-27 11:07:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 407117824. Throughput: 0: 10340.4, 1: 10251.6. Samples: 407088516. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:52,363][19170] Avg episode reward: [(0, '922.060'), (1, '990.280')] [2023-12-27 11:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000794808_203497472.pth... [2023-12-27 11:07:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000795296_203620352.pth... [2023-12-27 11:07:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000794112_203317248.pth [2023-12-27 11:07:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000793624_203194368.pth [2023-12-27 11:07:52,457][20347] Updated weights for policy 0, policy_version 795305 (0.0007) [2023-12-27 11:07:52,512][20347] Updated weights for policy 0, policy_version 795315 (0.0006) [2023-12-27 11:07:52,570][20347] Updated weights for policy 0, policy_version 795325 (0.0006) [2023-12-27 11:07:52,725][20381] Updated weights for policy 1, policy_version 794812 (0.0009) [2023-12-27 11:07:52,785][20381] Updated weights for policy 1, policy_version 794822 (0.0010) [2023-12-27 11:07:52,845][20381] Updated weights for policy 1, policy_version 794832 (0.0011) [2023-12-27 11:07:53,212][20347] Updated weights for policy 0, policy_version 795335 (0.0008) [2023-12-27 11:07:53,270][20347] Updated weights for policy 0, policy_version 795345 (0.0006) [2023-12-27 11:07:53,328][20347] Updated weights for policy 0, policy_version 795355 (0.0007) [2023-12-27 11:07:53,420][20381] Updated weights for policy 1, policy_version 794842 (0.0009) [2023-12-27 11:07:53,484][20381] Updated weights for policy 1, policy_version 794852 (0.0011) [2023-12-27 11:07:53,538][20381] Updated weights for policy 1, policy_version 794862 (0.0008) [2023-12-27 11:07:53,590][20381] Updated weights for policy 1, policy_version 794872 (0.0006) [2023-12-27 11:07:53,945][20347] Updated weights for policy 0, policy_version 795365 (0.0008) [2023-12-27 11:07:53,997][20347] Updated weights for policy 0, policy_version 795375 (0.0006) [2023-12-27 11:07:54,046][20347] Updated weights for policy 0, policy_version 795385 (0.0007) [2023-12-27 11:07:54,198][20381] Updated weights for policy 1, policy_version 794882 (0.0008) [2023-12-27 11:07:54,244][20381] Updated weights for policy 1, policy_version 794892 (0.0006) [2023-12-27 11:07:54,298][20381] Updated weights for policy 1, policy_version 794902 (0.0005) [2023-12-27 11:07:54,706][20347] Updated weights for policy 0, policy_version 795395 (0.0007) [2023-12-27 11:07:54,761][20347] Updated weights for policy 0, policy_version 795405 (0.0008) [2023-12-27 11:07:54,814][20347] Updated weights for policy 0, policy_version 795415 (0.0007) [2023-12-27 11:07:54,943][20381] Updated weights for policy 1, policy_version 794912 (0.0006) [2023-12-27 11:07:55,009][20381] Updated weights for policy 1, policy_version 794922 (0.0007) [2023-12-27 11:07:55,064][20381] Updated weights for policy 1, policy_version 794932 (0.0005) [2023-12-27 11:07:55,572][20347] Updated weights for policy 0, policy_version 795425 (0.0008) [2023-12-27 11:07:55,619][20347] Updated weights for policy 0, policy_version 795435 (0.0008) [2023-12-27 11:07:55,630][20381] Updated weights for policy 1, policy_version 794942 (0.0008) [2023-12-27 11:07:55,676][20347] Updated weights for policy 0, policy_version 795445 (0.0008) [2023-12-27 11:07:55,688][20381] Updated weights for policy 1, policy_version 794952 (0.0010) [2023-12-27 11:07:55,735][20347] Updated weights for policy 0, policy_version 795455 (0.0006) [2023-12-27 11:07:55,742][20381] Updated weights for policy 1, policy_version 794962 (0.0009) [2023-12-27 11:07:56,413][20347] Updated weights for policy 0, policy_version 795465 (0.0007) [2023-12-27 11:07:56,427][20381] Updated weights for policy 1, policy_version 794972 (0.0010) [2023-12-27 11:07:56,461][20347] Updated weights for policy 0, policy_version 795475 (0.0005) [2023-12-27 11:07:56,479][20381] Updated weights for policy 1, policy_version 794982 (0.0008) [2023-12-27 11:07:56,510][20347] Updated weights for policy 0, policy_version 795485 (0.0007) [2023-12-27 11:07:56,526][20381] Updated weights for policy 1, policy_version 794992 (0.0005) [2023-12-27 11:07:57,131][20381] Updated weights for policy 1, policy_version 795002 (0.0008) [2023-12-27 11:07:57,139][20347] Updated weights for policy 0, policy_version 795495 (0.0008) [2023-12-27 11:07:57,185][20381] Updated weights for policy 1, policy_version 795012 (0.0007) [2023-12-27 11:07:57,192][20347] Updated weights for policy 0, policy_version 795505 (0.0007) [2023-12-27 11:07:57,232][20381] Updated weights for policy 1, policy_version 795022 (0.0010) [2023-12-27 11:07:57,237][20347] Updated weights for policy 0, policy_version 795515 (0.0010) [2023-12-27 11:07:57,286][20381] Updated weights for policy 1, policy_version 795032 (0.0010) [2023-12-27 11:07:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 407232512. Throughput: 0: 10288.6, 1: 10282.3. Samples: 407214112. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:07:57,363][19170] Avg episode reward: [(0, '931.950'), (1, '990.270')] [2023-12-27 11:07:57,879][20347] Updated weights for policy 0, policy_version 795525 (0.0008) [2023-12-27 11:07:57,906][20381] Updated weights for policy 1, policy_version 795042 (0.0005) [2023-12-27 11:07:57,934][20347] Updated weights for policy 0, policy_version 795535 (0.0010) [2023-12-27 11:07:57,958][20381] Updated weights for policy 1, policy_version 795052 (0.0005) [2023-12-27 11:07:57,985][20347] Updated weights for policy 0, policy_version 795545 (0.0010) [2023-12-27 11:07:58,012][20381] Updated weights for policy 1, policy_version 795062 (0.0005) [2023-12-27 11:07:58,625][20347] Updated weights for policy 0, policy_version 795555 (0.0007) [2023-12-27 11:07:58,668][20381] Updated weights for policy 1, policy_version 795072 (0.0007) [2023-12-27 11:07:58,681][20347] Updated weights for policy 0, policy_version 795565 (0.0008) [2023-12-27 11:07:58,733][20381] Updated weights for policy 1, policy_version 795082 (0.0006) [2023-12-27 11:07:58,743][20347] Updated weights for policy 0, policy_version 795575 (0.0008) [2023-12-27 11:07:58,794][20381] Updated weights for policy 1, policy_version 795092 (0.0007) [2023-12-27 11:07:59,487][20347] Updated weights for policy 0, policy_version 795585 (0.0008) [2023-12-27 11:07:59,543][20347] Updated weights for policy 0, policy_version 795595 (0.0005) [2023-12-27 11:07:59,590][20347] Updated weights for policy 0, policy_version 795605 (0.0006) [2023-12-27 11:07:59,591][20381] Updated weights for policy 1, policy_version 795102 (0.0007) [2023-12-27 11:07:59,647][20347] Updated weights for policy 0, policy_version 795615 (0.0007) [2023-12-27 11:07:59,660][20381] Updated weights for policy 1, policy_version 795112 (0.0007) [2023-12-27 11:07:59,714][20381] Updated weights for policy 1, policy_version 795122 (0.0007) [2023-12-27 11:08:00,330][20347] Updated weights for policy 0, policy_version 795625 (0.0007) [2023-12-27 11:08:00,387][20347] Updated weights for policy 0, policy_version 795635 (0.0007) [2023-12-27 11:08:00,437][20347] Updated weights for policy 0, policy_version 795645 (0.0007) [2023-12-27 11:08:00,462][20381] Updated weights for policy 1, policy_version 795132 (0.0007) [2023-12-27 11:08:00,518][20381] Updated weights for policy 1, policy_version 795142 (0.0006) [2023-12-27 11:08:00,579][20381] Updated weights for policy 1, policy_version 795152 (0.0005) [2023-12-27 11:08:01,076][20347] Updated weights for policy 0, policy_version 795655 (0.0007) [2023-12-27 11:08:01,127][20347] Updated weights for policy 0, policy_version 795665 (0.0007) [2023-12-27 11:08:01,197][20347] Updated weights for policy 0, policy_version 795675 (0.0007) [2023-12-27 11:08:01,244][20381] Updated weights for policy 1, policy_version 795162 (0.0006) [2023-12-27 11:08:01,307][20381] Updated weights for policy 1, policy_version 795172 (0.0008) [2023-12-27 11:08:01,371][20381] Updated weights for policy 1, policy_version 795182 (0.0008) [2023-12-27 11:08:01,437][20381] Updated weights for policy 1, policy_version 795192 (0.0006) [2023-12-27 11:08:01,892][20347] Updated weights for policy 0, policy_version 795685 (0.0007) [2023-12-27 11:08:01,937][20347] Updated weights for policy 0, policy_version 795695 (0.0007) [2023-12-27 11:08:01,993][20347] Updated weights for policy 0, policy_version 795705 (0.0008) [2023-12-27 11:08:02,072][20381] Updated weights for policy 1, policy_version 795202 (0.0007) [2023-12-27 11:08:02,127][20381] Updated weights for policy 1, policy_version 795212 (0.0008) [2023-12-27 11:08:02,188][20381] Updated weights for policy 1, policy_version 795222 (0.0006) [2023-12-27 11:08:02,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 407330816. Throughput: 0: 10299.8, 1: 10287.4. Samples: 407337824. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:08:02,364][19170] Avg episode reward: [(0, '961.080'), (1, '990.270')] [2023-12-27 11:08:02,758][20347] Updated weights for policy 0, policy_version 795715 (0.0008) [2023-12-27 11:08:02,816][20347] Updated weights for policy 0, policy_version 795725 (0.0009) [2023-12-27 11:08:02,823][20381] Updated weights for policy 1, policy_version 795232 (0.0010) [2023-12-27 11:08:02,870][20347] Updated weights for policy 0, policy_version 795735 (0.0007) [2023-12-27 11:08:02,879][20381] Updated weights for policy 1, policy_version 795242 (0.0011) [2023-12-27 11:08:02,934][20381] Updated weights for policy 1, policy_version 795252 (0.0011) [2023-12-27 11:08:03,511][20347] Updated weights for policy 0, policy_version 795745 (0.0005) [2023-12-27 11:08:03,564][20347] Updated weights for policy 0, policy_version 795755 (0.0005) [2023-12-27 11:08:03,574][20381] Updated weights for policy 1, policy_version 795262 (0.0010) [2023-12-27 11:08:03,610][20347] Updated weights for policy 0, policy_version 795765 (0.0005) [2023-12-27 11:08:03,628][20381] Updated weights for policy 1, policy_version 795272 (0.0010) [2023-12-27 11:08:03,660][20347] Updated weights for policy 0, policy_version 795775 (0.0005) [2023-12-27 11:08:03,684][20381] Updated weights for policy 1, policy_version 795282 (0.0009) [2023-12-27 11:08:04,276][20347] Updated weights for policy 0, policy_version 795785 (0.0008) [2023-12-27 11:08:04,329][20381] Updated weights for policy 1, policy_version 795292 (0.0010) [2023-12-27 11:08:04,331][20347] Updated weights for policy 0, policy_version 795795 (0.0007) [2023-12-27 11:08:04,382][20381] Updated weights for policy 1, policy_version 795302 (0.0011) [2023-12-27 11:08:04,388][20347] Updated weights for policy 0, policy_version 795805 (0.0005) [2023-12-27 11:08:04,437][20381] Updated weights for policy 1, policy_version 795312 (0.0009) [2023-12-27 11:08:05,124][20347] Updated weights for policy 0, policy_version 795815 (0.0007) [2023-12-27 11:08:05,126][20381] Updated weights for policy 1, policy_version 795322 (0.0008) [2023-12-27 11:08:05,183][20347] Updated weights for policy 0, policy_version 795825 (0.0005) [2023-12-27 11:08:05,185][20381] Updated weights for policy 1, policy_version 795332 (0.0011) [2023-12-27 11:08:05,241][20347] Updated weights for policy 0, policy_version 795835 (0.0006) [2023-12-27 11:08:05,247][20381] Updated weights for policy 1, policy_version 795342 (0.0010) [2023-12-27 11:08:05,306][20381] Updated weights for policy 1, policy_version 795352 (0.0010) [2023-12-27 11:08:05,943][20347] Updated weights for policy 0, policy_version 795845 (0.0006) [2023-12-27 11:08:05,996][20381] Updated weights for policy 1, policy_version 795362 (0.0011) [2023-12-27 11:08:05,998][20347] Updated weights for policy 0, policy_version 795855 (0.0005) [2023-12-27 11:08:06,045][20381] Updated weights for policy 1, policy_version 795372 (0.0010) [2023-12-27 11:08:06,054][20347] Updated weights for policy 0, policy_version 795865 (0.0006) [2023-12-27 11:08:06,093][20381] Updated weights for policy 1, policy_version 795382 (0.0010) [2023-12-27 11:08:06,754][20381] Updated weights for policy 1, policy_version 795392 (0.0010) [2023-12-27 11:08:06,760][20347] Updated weights for policy 0, policy_version 795875 (0.0005) [2023-12-27 11:08:06,809][20381] Updated weights for policy 1, policy_version 795402 (0.0010) [2023-12-27 11:08:06,815][20347] Updated weights for policy 0, policy_version 795885 (0.0005) [2023-12-27 11:08:06,862][20381] Updated weights for policy 1, policy_version 795412 (0.0010) [2023-12-27 11:08:06,872][20347] Updated weights for policy 0, policy_version 795895 (0.0006) [2023-12-27 11:08:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 407429120. Throughput: 0: 10289.5, 1: 10314.7. Samples: 407400216. Policy #0 lag: (min: 12.0, avg: 19.0, max: 44.0) [2023-12-27 11:08:07,363][19170] Avg episode reward: [(0, '970.750'), (1, '970.870')] [2023-12-27 11:08:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000795416_203653120.pth... [2023-12-27 11:08:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000795904_203776000.pth... [2023-12-27 11:08:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000794232_203350016.pth [2023-12-27 11:08:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000794720_203472896.pth [2023-12-27 11:08:07,570][20381] Updated weights for policy 1, policy_version 795422 (0.0008) [2023-12-27 11:08:07,577][20347] Updated weights for policy 0, policy_version 795905 (0.0008) [2023-12-27 11:08:07,618][20381] Updated weights for policy 1, policy_version 795432 (0.0006) [2023-12-27 11:08:07,631][20347] Updated weights for policy 0, policy_version 795915 (0.0007) [2023-12-27 11:08:07,666][20381] Updated weights for policy 1, policy_version 795442 (0.0005) [2023-12-27 11:08:07,688][20347] Updated weights for policy 0, policy_version 795925 (0.0008) [2023-12-27 11:08:07,741][20347] Updated weights for policy 0, policy_version 795935 (0.0007) [2023-12-27 11:08:08,300][20381] Updated weights for policy 1, policy_version 795452 (0.0005) [2023-12-27 11:08:08,363][20381] Updated weights for policy 1, policy_version 795462 (0.0009) [2023-12-27 11:08:08,399][20291] KL-divergence is very high: 180.1011 [2023-12-27 11:08:08,405][20347] Updated weights for policy 0, policy_version 795945 (0.0009) [2023-12-27 11:08:08,422][20291] KL-divergence is very high: 157.5023 [2023-12-27 11:08:08,424][20381] Updated weights for policy 1, policy_version 795472 (0.0008) [2023-12-27 11:08:08,475][20347] Updated weights for policy 0, policy_version 795955 (0.0007) [2023-12-27 11:08:08,543][20347] Updated weights for policy 0, policy_version 795965 (0.0007) [2023-12-27 11:08:09,010][20381] Updated weights for policy 1, policy_version 795482 (0.0008) [2023-12-27 11:08:09,062][20381] Updated weights for policy 1, policy_version 795492 (0.0011) [2023-12-27 11:08:09,114][20381] Updated weights for policy 1, policy_version 795502 (0.0010) [2023-12-27 11:08:09,159][20381] Updated weights for policy 1, policy_version 795512 (0.0010) [2023-12-27 11:08:09,238][20347] Updated weights for policy 0, policy_version 795975 (0.0008) [2023-12-27 11:08:09,293][20347] Updated weights for policy 0, policy_version 795985 (0.0007) [2023-12-27 11:08:09,352][20347] Updated weights for policy 0, policy_version 795995 (0.0008) [2023-12-27 11:08:09,888][20381] Updated weights for policy 1, policy_version 795522 (0.0008) [2023-12-27 11:08:09,956][20381] Updated weights for policy 1, policy_version 795532 (0.0009) [2023-12-27 11:08:10,021][20381] Updated weights for policy 1, policy_version 795542 (0.0008) [2023-12-27 11:08:10,059][20347] Updated weights for policy 0, policy_version 796005 (0.0008) [2023-12-27 11:08:10,120][20347] Updated weights for policy 0, policy_version 796015 (0.0007) [2023-12-27 11:08:10,182][20347] Updated weights for policy 0, policy_version 796025 (0.0006) [2023-12-27 11:08:10,719][20381] Updated weights for policy 1, policy_version 795552 (0.0007) [2023-12-27 11:08:10,768][20381] Updated weights for policy 1, policy_version 795562 (0.0008) [2023-12-27 11:08:10,813][20347] Updated weights for policy 0, policy_version 796035 (0.0007) [2023-12-27 11:08:10,822][20381] Updated weights for policy 1, policy_version 795572 (0.0007) [2023-12-27 11:08:10,882][20347] Updated weights for policy 0, policy_version 796045 (0.0008) [2023-12-27 11:08:10,931][20347] Updated weights for policy 0, policy_version 796055 (0.0007) [2023-12-27 11:08:11,583][20381] Updated weights for policy 1, policy_version 795582 (0.0007) [2023-12-27 11:08:11,651][20381] Updated weights for policy 1, policy_version 795592 (0.0008) [2023-12-27 11:08:11,674][20347] Updated weights for policy 0, policy_version 796065 (0.0008) [2023-12-27 11:08:11,715][20381] Updated weights for policy 1, policy_version 795602 (0.0008) [2023-12-27 11:08:11,743][20347] Updated weights for policy 0, policy_version 796075 (0.0008) [2023-12-27 11:08:11,805][20347] Updated weights for policy 0, policy_version 796085 (0.0007) [2023-12-27 11:08:11,870][20347] Updated weights for policy 0, policy_version 796095 (0.0006) [2023-12-27 11:08:12,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 407527424. Throughput: 0: 10261.2, 1: 10323.2. Samples: 407521468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:12,363][19170] Avg episode reward: [(0, '960.990'), (1, '970.870')] [2023-12-27 11:08:12,422][20381] Updated weights for policy 1, policy_version 795612 (0.0007) [2023-12-27 11:08:12,487][20381] Updated weights for policy 1, policy_version 795622 (0.0007) [2023-12-27 11:08:12,545][20381] Updated weights for policy 1, policy_version 795632 (0.0007) [2023-12-27 11:08:12,576][20347] Updated weights for policy 0, policy_version 796105 (0.0008) [2023-12-27 11:08:12,637][20347] Updated weights for policy 0, policy_version 796115 (0.0008) [2023-12-27 11:08:12,698][20347] Updated weights for policy 0, policy_version 796125 (0.0008) [2023-12-27 11:08:13,179][20381] Updated weights for policy 1, policy_version 795642 (0.0007) [2023-12-27 11:08:13,238][20381] Updated weights for policy 1, policy_version 795652 (0.0009) [2023-12-27 11:08:13,297][20381] Updated weights for policy 1, policy_version 795662 (0.0011) [2023-12-27 11:08:13,313][20347] Updated weights for policy 0, policy_version 796135 (0.0006) [2023-12-27 11:08:13,352][20381] Updated weights for policy 1, policy_version 795672 (0.0011) [2023-12-27 11:08:13,361][20347] Updated weights for policy 0, policy_version 796145 (0.0005) [2023-12-27 11:08:13,413][20347] Updated weights for policy 0, policy_version 796155 (0.0005) [2023-12-27 11:08:13,996][20347] Updated weights for policy 0, policy_version 796165 (0.0005) [2023-12-27 11:08:14,014][20381] Updated weights for policy 1, policy_version 795682 (0.0010) [2023-12-27 11:08:14,043][20347] Updated weights for policy 0, policy_version 796175 (0.0007) [2023-12-27 11:08:14,069][20381] Updated weights for policy 1, policy_version 795692 (0.0010) [2023-12-27 11:08:14,096][20347] Updated weights for policy 0, policy_version 796185 (0.0005) [2023-12-27 11:08:14,121][20381] Updated weights for policy 1, policy_version 795702 (0.0011) [2023-12-27 11:08:14,728][20347] Updated weights for policy 0, policy_version 796195 (0.0005) [2023-12-27 11:08:14,795][20347] Updated weights for policy 0, policy_version 796205 (0.0007) [2023-12-27 11:08:14,821][20381] Updated weights for policy 1, policy_version 795712 (0.0011) [2023-12-27 11:08:14,847][20347] Updated weights for policy 0, policy_version 796215 (0.0008) [2023-12-27 11:08:14,871][20381] Updated weights for policy 1, policy_version 795722 (0.0011) [2023-12-27 11:08:14,918][20381] Updated weights for policy 1, policy_version 795732 (0.0011) [2023-12-27 11:08:15,554][20347] Updated weights for policy 0, policy_version 796225 (0.0008) [2023-12-27 11:08:15,602][20347] Updated weights for policy 0, policy_version 796235 (0.0007) [2023-12-27 11:08:15,647][20381] Updated weights for policy 1, policy_version 795742 (0.0010) [2023-12-27 11:08:15,650][20347] Updated weights for policy 0, policy_version 796245 (0.0008) [2023-12-27 11:08:15,695][20381] Updated weights for policy 1, policy_version 795752 (0.0010) [2023-12-27 11:08:15,697][20347] Updated weights for policy 0, policy_version 796255 (0.0005) [2023-12-27 11:08:15,751][20381] Updated weights for policy 1, policy_version 795762 (0.0010) [2023-12-27 11:08:16,357][20347] Updated weights for policy 0, policy_version 796265 (0.0010) [2023-12-27 11:08:16,409][20347] Updated weights for policy 0, policy_version 796275 (0.0010) [2023-12-27 11:08:16,450][20381] Updated weights for policy 1, policy_version 795772 (0.0010) [2023-12-27 11:08:16,464][20347] Updated weights for policy 0, policy_version 796285 (0.0007) [2023-12-27 11:08:16,506][20381] Updated weights for policy 1, policy_version 795782 (0.0008) [2023-12-27 11:08:16,567][20381] Updated weights for policy 1, policy_version 795792 (0.0010) [2023-12-27 11:08:17,142][20347] Updated weights for policy 0, policy_version 796295 (0.0007) [2023-12-27 11:08:17,183][20381] Updated weights for policy 1, policy_version 795802 (0.0009) [2023-12-27 11:08:17,199][20347] Updated weights for policy 0, policy_version 796305 (0.0007) [2023-12-27 11:08:17,233][20381] Updated weights for policy 1, policy_version 795812 (0.0008) [2023-12-27 11:08:17,250][20347] Updated weights for policy 0, policy_version 796315 (0.0008) [2023-12-27 11:08:17,278][20381] Updated weights for policy 1, policy_version 795822 (0.0006) [2023-12-27 11:08:17,326][20381] Updated weights for policy 1, policy_version 795832 (0.0008) [2023-12-27 11:08:17,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 407642112. Throughput: 0: 10250.4, 1: 10316.5. Samples: 407645316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:17,364][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 11:08:17,862][20347] Updated weights for policy 0, policy_version 796325 (0.0008) [2023-12-27 11:08:17,908][20347] Updated weights for policy 0, policy_version 796335 (0.0008) [2023-12-27 11:08:17,972][20347] Updated weights for policy 0, policy_version 796345 (0.0006) [2023-12-27 11:08:18,061][20381] Updated weights for policy 1, policy_version 795842 (0.0008) [2023-12-27 11:08:18,121][20381] Updated weights for policy 1, policy_version 795852 (0.0007) [2023-12-27 11:08:18,179][20381] Updated weights for policy 1, policy_version 795862 (0.0008) [2023-12-27 11:08:18,579][20347] Updated weights for policy 0, policy_version 796355 (0.0006) [2023-12-27 11:08:18,639][20347] Updated weights for policy 0, policy_version 796365 (0.0006) [2023-12-27 11:08:18,703][20347] Updated weights for policy 0, policy_version 796375 (0.0006) [2023-12-27 11:08:18,847][20381] Updated weights for policy 1, policy_version 795872 (0.0008) [2023-12-27 11:08:18,905][20381] Updated weights for policy 1, policy_version 795882 (0.0008) [2023-12-27 11:08:18,954][20381] Updated weights for policy 1, policy_version 795893 (0.0009) [2023-12-27 11:08:19,262][20347] Updated weights for policy 0, policy_version 796385 (0.0006) [2023-12-27 11:08:19,319][20347] Updated weights for policy 0, policy_version 796395 (0.0006) [2023-12-27 11:08:19,385][20347] Updated weights for policy 0, policy_version 796405 (0.0008) [2023-12-27 11:08:19,440][20347] Updated weights for policy 0, policy_version 796415 (0.0006) [2023-12-27 11:08:19,676][20381] Updated weights for policy 1, policy_version 795903 (0.0008) [2023-12-27 11:08:19,733][20381] Updated weights for policy 1, policy_version 795913 (0.0008) [2023-12-27 11:08:19,790][20381] Updated weights for policy 1, policy_version 795923 (0.0008) [2023-12-27 11:08:20,117][20347] Updated weights for policy 0, policy_version 796425 (0.0008) [2023-12-27 11:08:20,174][20347] Updated weights for policy 0, policy_version 796435 (0.0008) [2023-12-27 11:08:20,240][20347] Updated weights for policy 0, policy_version 796445 (0.0007) [2023-12-27 11:08:20,563][20381] Updated weights for policy 1, policy_version 795933 (0.0007) [2023-12-27 11:08:20,629][20381] Updated weights for policy 1, policy_version 795943 (0.0007) [2023-12-27 11:08:20,689][20381] Updated weights for policy 1, policy_version 795953 (0.0008) [2023-12-27 11:08:20,937][20347] Updated weights for policy 0, policy_version 796455 (0.0007) [2023-12-27 11:08:21,003][20347] Updated weights for policy 0, policy_version 796465 (0.0007) [2023-12-27 11:08:21,069][20347] Updated weights for policy 0, policy_version 796475 (0.0008) [2023-12-27 11:08:21,471][20381] Updated weights for policy 1, policy_version 795963 (0.0008) [2023-12-27 11:08:21,537][20381] Updated weights for policy 1, policy_version 795973 (0.0010) [2023-12-27 11:08:21,605][20381] Updated weights for policy 1, policy_version 795983 (0.0010) [2023-12-27 11:08:21,797][20347] Updated weights for policy 0, policy_version 796485 (0.0006) [2023-12-27 11:08:21,846][20347] Updated weights for policy 0, policy_version 796495 (0.0008) [2023-12-27 11:08:21,905][20347] Updated weights for policy 0, policy_version 796505 (0.0007) [2023-12-27 11:08:22,336][20381] Updated weights for policy 1, policy_version 795993 (0.0009) [2023-12-27 11:08:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 407732224. Throughput: 0: 10318.0, 1: 10275.8. Samples: 407707596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.600')] [2023-12-27 11:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000796512_203931648.pth... [2023-12-27 11:08:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000795296_203620352.pth [2023-12-27 11:08:22,404][20381] Updated weights for policy 1, policy_version 796003 (0.0009) [2023-12-27 11:08:22,431][20292] KL-divergence is very high: 1339.5033 [2023-12-27 11:08:22,457][20381] Updated weights for policy 1, policy_version 796013 (0.0007) [2023-12-27 11:08:22,476][20292] KL-divergence is very high: 917.9007 [2023-12-27 11:08:22,523][20381] Updated weights for policy 1, policy_version 796023 (0.0008) [2023-12-27 11:08:22,527][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000796024_203808768.pth... [2023-12-27 11:08:22,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000794808_203497472.pth [2023-12-27 11:08:22,613][20347] Updated weights for policy 0, policy_version 796515 (0.0008) [2023-12-27 11:08:22,671][20347] Updated weights for policy 0, policy_version 796525 (0.0007) [2023-12-27 11:08:22,727][20347] Updated weights for policy 0, policy_version 796535 (0.0008) [2023-12-27 11:08:23,212][20381] Updated weights for policy 1, policy_version 796033 (0.0010) [2023-12-27 11:08:23,271][20381] Updated weights for policy 1, policy_version 796043 (0.0010) [2023-12-27 11:08:23,333][20381] Updated weights for policy 1, policy_version 796053 (0.0011) [2023-12-27 11:08:23,364][20347] Updated weights for policy 0, policy_version 796545 (0.0008) [2023-12-27 11:08:23,410][20347] Updated weights for policy 0, policy_version 796555 (0.0007) [2023-12-27 11:08:23,464][20347] Updated weights for policy 0, policy_version 796565 (0.0007) [2023-12-27 11:08:23,518][20347] Updated weights for policy 0, policy_version 796575 (0.0007) [2023-12-27 11:08:23,988][20381] Updated weights for policy 1, policy_version 796063 (0.0010) [2023-12-27 11:08:24,049][20381] Updated weights for policy 1, policy_version 796073 (0.0010) [2023-12-27 11:08:24,094][20381] Updated weights for policy 1, policy_version 796083 (0.0010) [2023-12-27 11:08:24,192][20347] Updated weights for policy 0, policy_version 796585 (0.0006) [2023-12-27 11:08:24,253][20347] Updated weights for policy 0, policy_version 796595 (0.0010) [2023-12-27 11:08:24,315][20347] Updated weights for policy 0, policy_version 796605 (0.0010) [2023-12-27 11:08:24,650][20381] Updated weights for policy 1, policy_version 796093 (0.0008) [2023-12-27 11:08:24,703][20381] Updated weights for policy 1, policy_version 796103 (0.0005) [2023-12-27 11:08:24,758][20381] Updated weights for policy 1, policy_version 796113 (0.0005) [2023-12-27 11:08:24,918][20347] Updated weights for policy 0, policy_version 796615 (0.0007) [2023-12-27 11:08:24,973][20347] Updated weights for policy 0, policy_version 796625 (0.0007) [2023-12-27 11:08:25,025][20347] Updated weights for policy 0, policy_version 796635 (0.0005) [2023-12-27 11:08:25,342][20381] Updated weights for policy 1, policy_version 796123 (0.0007) [2023-12-27 11:08:25,396][20381] Updated weights for policy 1, policy_version 796133 (0.0010) [2023-12-27 11:08:25,460][20381] Updated weights for policy 1, policy_version 796143 (0.0009) [2023-12-27 11:08:25,634][20347] Updated weights for policy 0, policy_version 796645 (0.0008) [2023-12-27 11:08:25,691][20347] Updated weights for policy 0, policy_version 796655 (0.0009) [2023-12-27 11:08:25,740][20347] Updated weights for policy 0, policy_version 796665 (0.0005) [2023-12-27 11:08:26,131][20381] Updated weights for policy 1, policy_version 796153 (0.0009) [2023-12-27 11:08:26,177][20381] Updated weights for policy 1, policy_version 796163 (0.0010) [2023-12-27 11:08:26,223][20381] Updated weights for policy 1, policy_version 796173 (0.0009) [2023-12-27 11:08:26,273][20381] Updated weights for policy 1, policy_version 796183 (0.0006) [2023-12-27 11:08:26,325][20347] Updated weights for policy 0, policy_version 796675 (0.0007) [2023-12-27 11:08:26,378][20347] Updated weights for policy 0, policy_version 796685 (0.0010) [2023-12-27 11:08:26,428][20347] Updated weights for policy 0, policy_version 796695 (0.0008) [2023-12-27 11:08:26,834][20381] Updated weights for policy 1, policy_version 796193 (0.0007) [2023-12-27 11:08:26,878][20381] Updated weights for policy 1, policy_version 796203 (0.0010) [2023-12-27 11:08:26,922][20381] Updated weights for policy 1, policy_version 796213 (0.0010) [2023-12-27 11:08:27,052][20347] Updated weights for policy 0, policy_version 796705 (0.0005) [2023-12-27 11:08:27,106][20347] Updated weights for policy 0, policy_version 796715 (0.0005) [2023-12-27 11:08:27,168][20347] Updated weights for policy 0, policy_version 796725 (0.0005) [2023-12-27 11:08:27,225][20347] Updated weights for policy 0, policy_version 796735 (0.0010) [2023-12-27 11:08:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 407846912. Throughput: 0: 10358.3, 1: 10238.7. Samples: 407831588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:27,363][19170] Avg episode reward: [(0, '990.310'), (1, '970.940')] [2023-12-27 11:08:27,576][20381] Updated weights for policy 1, policy_version 796223 (0.0010) [2023-12-27 11:08:27,627][20381] Updated weights for policy 1, policy_version 796233 (0.0010) [2023-12-27 11:08:27,670][20381] Updated weights for policy 1, policy_version 796243 (0.0010) [2023-12-27 11:08:27,849][20347] Updated weights for policy 0, policy_version 796745 (0.0010) [2023-12-27 11:08:27,905][20347] Updated weights for policy 0, policy_version 796755 (0.0010) [2023-12-27 11:08:27,963][20347] Updated weights for policy 0, policy_version 796765 (0.0010) [2023-12-27 11:08:28,355][20381] Updated weights for policy 1, policy_version 796253 (0.0010) [2023-12-27 11:08:28,410][20381] Updated weights for policy 1, policy_version 796263 (0.0010) [2023-12-27 11:08:28,467][20381] Updated weights for policy 1, policy_version 796273 (0.0010) [2023-12-27 11:08:28,620][20347] Updated weights for policy 0, policy_version 796775 (0.0010) [2023-12-27 11:08:28,677][20347] Updated weights for policy 0, policy_version 796785 (0.0010) [2023-12-27 11:08:28,734][20347] Updated weights for policy 0, policy_version 796795 (0.0010) [2023-12-27 11:08:29,095][20381] Updated weights for policy 1, policy_version 796283 (0.0009) [2023-12-27 11:08:29,149][20381] Updated weights for policy 1, policy_version 796293 (0.0005) [2023-12-27 11:08:29,206][20381] Updated weights for policy 1, policy_version 796303 (0.0005) [2023-12-27 11:08:29,454][20347] Updated weights for policy 0, policy_version 796805 (0.0008) [2023-12-27 11:08:29,502][20347] Updated weights for policy 0, policy_version 796815 (0.0007) [2023-12-27 11:08:29,562][20347] Updated weights for policy 0, policy_version 796825 (0.0010) [2023-12-27 11:08:29,880][20381] Updated weights for policy 1, policy_version 796313 (0.0008) [2023-12-27 11:08:29,945][20381] Updated weights for policy 1, policy_version 796323 (0.0010) [2023-12-27 11:08:30,003][20381] Updated weights for policy 1, policy_version 796333 (0.0010) [2023-12-27 11:08:30,068][20381] Updated weights for policy 1, policy_version 796343 (0.0011) [2023-12-27 11:08:30,274][20347] Updated weights for policy 0, policy_version 796835 (0.0009) [2023-12-27 11:08:30,332][20347] Updated weights for policy 0, policy_version 796845 (0.0008) [2023-12-27 11:08:30,388][20347] Updated weights for policy 0, policy_version 796855 (0.0007) [2023-12-27 11:08:30,794][20381] Updated weights for policy 1, policy_version 796353 (0.0010) [2023-12-27 11:08:30,845][20381] Updated weights for policy 1, policy_version 796363 (0.0007) [2023-12-27 11:08:30,897][20381] Updated weights for policy 1, policy_version 796373 (0.0010) [2023-12-27 11:08:30,942][20347] Updated weights for policy 0, policy_version 796865 (0.0009) [2023-12-27 11:08:30,993][20347] Updated weights for policy 0, policy_version 796875 (0.0010) [2023-12-27 11:08:31,051][20347] Updated weights for policy 0, policy_version 796885 (0.0009) [2023-12-27 11:08:31,118][20347] Updated weights for policy 0, policy_version 796895 (0.0007) [2023-12-27 11:08:31,696][20381] Updated weights for policy 1, policy_version 796383 (0.0008) [2023-12-27 11:08:31,757][20381] Updated weights for policy 1, policy_version 796393 (0.0008) [2023-12-27 11:08:31,812][20381] Updated weights for policy 1, policy_version 796403 (0.0008) [2023-12-27 11:08:31,858][20347] Updated weights for policy 0, policy_version 796905 (0.0008) [2023-12-27 11:08:31,910][20347] Updated weights for policy 0, policy_version 796915 (0.0007) [2023-12-27 11:08:31,966][20347] Updated weights for policy 0, policy_version 796925 (0.0008) [2023-12-27 11:08:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 407945216. Throughput: 0: 10353.2, 1: 10287.3. Samples: 407956116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:32,363][19170] Avg episode reward: [(0, '980.650'), (1, '980.740')] [2023-12-27 11:08:32,512][20381] Updated weights for policy 1, policy_version 796413 (0.0007) [2023-12-27 11:08:32,570][20381] Updated weights for policy 1, policy_version 796423 (0.0007) [2023-12-27 11:08:32,622][20381] Updated weights for policy 1, policy_version 796433 (0.0007) [2023-12-27 11:08:32,640][20347] Updated weights for policy 0, policy_version 796935 (0.0006) [2023-12-27 11:08:32,700][20347] Updated weights for policy 0, policy_version 796945 (0.0005) [2023-12-27 11:08:32,760][20347] Updated weights for policy 0, policy_version 796955 (0.0005) [2023-12-27 11:08:33,303][20381] Updated weights for policy 1, policy_version 796443 (0.0006) [2023-12-27 11:08:33,330][20347] Updated weights for policy 0, policy_version 796965 (0.0008) [2023-12-27 11:08:33,352][20381] Updated weights for policy 1, policy_version 796453 (0.0006) [2023-12-27 11:08:33,378][20347] Updated weights for policy 0, policy_version 796975 (0.0010) [2023-12-27 11:08:33,408][20381] Updated weights for policy 1, policy_version 796463 (0.0005) [2023-12-27 11:08:33,438][20347] Updated weights for policy 0, policy_version 796985 (0.0009) [2023-12-27 11:08:33,949][20381] Updated weights for policy 1, policy_version 796473 (0.0005) [2023-12-27 11:08:34,000][20381] Updated weights for policy 1, policy_version 796483 (0.0005) [2023-12-27 11:08:34,056][20347] Updated weights for policy 0, policy_version 796995 (0.0009) [2023-12-27 11:08:34,060][20381] Updated weights for policy 1, policy_version 796493 (0.0005) [2023-12-27 11:08:34,110][20347] Updated weights for policy 0, policy_version 797005 (0.0005) [2023-12-27 11:08:34,112][20381] Updated weights for policy 1, policy_version 796503 (0.0005) [2023-12-27 11:08:34,167][20347] Updated weights for policy 0, policy_version 797015 (0.0008) [2023-12-27 11:08:34,775][20381] Updated weights for policy 1, policy_version 796513 (0.0006) [2023-12-27 11:08:34,822][20381] Updated weights for policy 1, policy_version 796523 (0.0009) [2023-12-27 11:08:34,825][20347] Updated weights for policy 0, policy_version 797025 (0.0007) [2023-12-27 11:08:34,871][20347] Updated weights for policy 0, policy_version 797035 (0.0005) [2023-12-27 11:08:34,872][20381] Updated weights for policy 1, policy_version 796533 (0.0005) [2023-12-27 11:08:34,893][20291] KL-divergence is very high: 105.1849 [2023-12-27 11:08:34,921][20347] Updated weights for policy 0, policy_version 797045 (0.0006) [2023-12-27 11:08:34,977][20347] Updated weights for policy 0, policy_version 797055 (0.0008) [2023-12-27 11:08:35,588][20381] Updated weights for policy 1, policy_version 796543 (0.0008) [2023-12-27 11:08:35,649][20381] Updated weights for policy 1, policy_version 796553 (0.0007) [2023-12-27 11:08:35,709][20381] Updated weights for policy 1, policy_version 796563 (0.0006) [2023-12-27 11:08:35,710][20347] Updated weights for policy 0, policy_version 797065 (0.0006) [2023-12-27 11:08:35,760][20347] Updated weights for policy 0, policy_version 797075 (0.0005) [2023-12-27 11:08:35,809][20347] Updated weights for policy 0, policy_version 797085 (0.0007) [2023-12-27 11:08:36,362][20381] Updated weights for policy 1, policy_version 796573 (0.0007) [2023-12-27 11:08:36,421][20381] Updated weights for policy 1, policy_version 796583 (0.0006) [2023-12-27 11:08:36,450][20347] Updated weights for policy 0, policy_version 797095 (0.0008) [2023-12-27 11:08:36,481][20381] Updated weights for policy 1, policy_version 796593 (0.0006) [2023-12-27 11:08:36,512][20347] Updated weights for policy 0, policy_version 797105 (0.0009) [2023-12-27 11:08:36,571][20347] Updated weights for policy 0, policy_version 797115 (0.0007) [2023-12-27 11:08:37,109][20381] Updated weights for policy 1, policy_version 796603 (0.0008) [2023-12-27 11:08:37,168][20381] Updated weights for policy 1, policy_version 796613 (0.0007) [2023-12-27 11:08:37,222][20381] Updated weights for policy 1, policy_version 796623 (0.0005) [2023-12-27 11:08:37,224][20347] Updated weights for policy 0, policy_version 797125 (0.0008) [2023-12-27 11:08:37,278][20347] Updated weights for policy 0, policy_version 797135 (0.0006) [2023-12-27 11:08:37,333][20347] Updated weights for policy 0, policy_version 797145 (0.0007) [2023-12-27 11:08:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 408051712. Throughput: 0: 10406.7, 1: 10315.3. Samples: 408021008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:37,363][19170] Avg episode reward: [(0, '951.540'), (1, '990.340')] [2023-12-27 11:08:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000796632_203964416.pth... [2023-12-27 11:08:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000797152_204095488.pth... [2023-12-27 11:08:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000795904_203776000.pth [2023-12-27 11:08:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000795416_203653120.pth [2023-12-27 11:08:37,917][20381] Updated weights for policy 1, policy_version 796633 (0.0006) [2023-12-27 11:08:37,974][20381] Updated weights for policy 1, policy_version 796643 (0.0006) [2023-12-27 11:08:37,985][20347] Updated weights for policy 0, policy_version 797155 (0.0008) [2023-12-27 11:08:38,033][20381] Updated weights for policy 1, policy_version 796653 (0.0006) [2023-12-27 11:08:38,039][20347] Updated weights for policy 0, policy_version 797165 (0.0010) [2023-12-27 11:08:38,092][20381] Updated weights for policy 1, policy_version 796663 (0.0006) [2023-12-27 11:08:38,095][20347] Updated weights for policy 0, policy_version 797175 (0.0010) [2023-12-27 11:08:38,773][20381] Updated weights for policy 1, policy_version 796673 (0.0007) [2023-12-27 11:08:38,829][20347] Updated weights for policy 0, policy_version 797185 (0.0010) [2023-12-27 11:08:38,833][20381] Updated weights for policy 1, policy_version 796683 (0.0008) [2023-12-27 11:08:38,890][20347] Updated weights for policy 0, policy_version 797195 (0.0008) [2023-12-27 11:08:38,895][20381] Updated weights for policy 1, policy_version 796693 (0.0008) [2023-12-27 11:08:38,948][20347] Updated weights for policy 0, policy_version 797205 (0.0009) [2023-12-27 11:08:39,009][20347] Updated weights for policy 0, policy_version 797215 (0.0010) [2023-12-27 11:08:39,670][20381] Updated weights for policy 1, policy_version 796703 (0.0007) [2023-12-27 11:08:39,733][20381] Updated weights for policy 1, policy_version 796713 (0.0008) [2023-12-27 11:08:39,791][20347] Updated weights for policy 0, policy_version 797225 (0.0011) [2023-12-27 11:08:39,801][20381] Updated weights for policy 1, policy_version 796723 (0.0007) [2023-12-27 11:08:39,860][20347] Updated weights for policy 0, policy_version 797235 (0.0009) [2023-12-27 11:08:39,928][20347] Updated weights for policy 0, policy_version 797246 (0.0009) [2023-12-27 11:08:40,549][20381] Updated weights for policy 1, policy_version 796733 (0.0007) [2023-12-27 11:08:40,614][20381] Updated weights for policy 1, policy_version 796743 (0.0008) [2023-12-27 11:08:40,680][20381] Updated weights for policy 1, policy_version 796753 (0.0007) [2023-12-27 11:08:40,729][20347] Updated weights for policy 0, policy_version 797256 (0.0010) [2023-12-27 11:08:40,797][20347] Updated weights for policy 0, policy_version 797266 (0.0011) [2023-12-27 11:08:40,862][20347] Updated weights for policy 0, policy_version 797276 (0.0011) [2023-12-27 11:08:41,478][20381] Updated weights for policy 1, policy_version 796763 (0.0007) [2023-12-27 11:08:41,545][20381] Updated weights for policy 1, policy_version 796773 (0.0008) [2023-12-27 11:08:41,606][20381] Updated weights for policy 1, policy_version 796783 (0.0008) [2023-12-27 11:08:41,611][20347] Updated weights for policy 0, policy_version 797286 (0.0009) [2023-12-27 11:08:41,655][20291] KL-divergence is very high: 113.0684 [2023-12-27 11:08:41,684][20347] Updated weights for policy 0, policy_version 797296 (0.0008) [2023-12-27 11:08:41,771][20347] Updated weights for policy 0, policy_version 797306 (0.0009) [2023-12-27 11:08:42,348][20381] Updated weights for policy 1, policy_version 796793 (0.0008) [2023-12-27 11:08:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 408141824. Throughput: 0: 10339.4, 1: 10209.1. Samples: 408138796. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:42,363][19170] Avg episode reward: [(0, '913.860'), (1, '991.220')] [2023-12-27 11:08:42,417][20381] Updated weights for policy 1, policy_version 796803 (0.0007) [2023-12-27 11:08:42,470][20381] Updated weights for policy 1, policy_version 796813 (0.0005) [2023-12-27 11:08:42,523][20381] Updated weights for policy 1, policy_version 796823 (0.0007) [2023-12-27 11:08:42,540][20347] Updated weights for policy 0, policy_version 797316 (0.0007) [2023-12-27 11:08:42,541][20291] KL-divergence is very high: 103.2422 [2023-12-27 11:08:42,587][20291] KL-divergence is very high: 113.3916 [2023-12-27 11:08:42,598][20347] Updated weights for policy 0, policy_version 797326 (0.0008) [2023-12-27 11:08:42,666][20347] Updated weights for policy 0, policy_version 797336 (0.0008) [2023-12-27 11:08:43,256][20381] Updated weights for policy 1, policy_version 796833 (0.0008) [2023-12-27 11:08:43,313][20381] Updated weights for policy 1, policy_version 796843 (0.0008) [2023-12-27 11:08:43,362][20381] Updated weights for policy 1, policy_version 796853 (0.0007) [2023-12-27 11:08:43,367][20347] Updated weights for policy 0, policy_version 797346 (0.0007) [2023-12-27 11:08:43,433][20347] Updated weights for policy 0, policy_version 797356 (0.0005) [2023-12-27 11:08:43,494][20347] Updated weights for policy 0, policy_version 797366 (0.0006) [2023-12-27 11:08:43,556][20347] Updated weights for policy 0, policy_version 797376 (0.0009) [2023-12-27 11:08:44,015][20381] Updated weights for policy 1, policy_version 796863 (0.0008) [2023-12-27 11:08:44,082][20381] Updated weights for policy 1, policy_version 796873 (0.0008) [2023-12-27 11:08:44,163][20381] Updated weights for policy 1, policy_version 796883 (0.0007) [2023-12-27 11:08:44,231][20347] Updated weights for policy 0, policy_version 797386 (0.0008) [2023-12-27 11:08:44,280][20347] Updated weights for policy 0, policy_version 797396 (0.0008) [2023-12-27 11:08:44,285][20291] KL-divergence is very high: 115.0378 [2023-12-27 11:08:44,329][20347] Updated weights for policy 0, policy_version 797406 (0.0008) [2023-12-27 11:08:44,784][20381] Updated weights for policy 1, policy_version 796893 (0.0008) [2023-12-27 11:08:44,837][20381] Updated weights for policy 1, policy_version 796903 (0.0006) [2023-12-27 11:08:44,890][20381] Updated weights for policy 1, policy_version 796913 (0.0005) [2023-12-27 11:08:45,054][20347] Updated weights for policy 0, policy_version 797416 (0.0010) [2023-12-27 11:08:45,116][20347] Updated weights for policy 0, policy_version 797426 (0.0010) [2023-12-27 11:08:45,176][20347] Updated weights for policy 0, policy_version 797436 (0.0008) [2023-12-27 11:08:45,566][20381] Updated weights for policy 1, policy_version 796923 (0.0008) [2023-12-27 11:08:45,615][20381] Updated weights for policy 1, policy_version 796933 (0.0007) [2023-12-27 11:08:45,662][20381] Updated weights for policy 1, policy_version 796943 (0.0008) [2023-12-27 11:08:45,798][20347] Updated weights for policy 0, policy_version 797446 (0.0009) [2023-12-27 11:08:45,855][20347] Updated weights for policy 0, policy_version 797456 (0.0011) [2023-12-27 11:08:45,909][20347] Updated weights for policy 0, policy_version 797466 (0.0011) [2023-12-27 11:08:46,395][20381] Updated weights for policy 1, policy_version 796953 (0.0008) [2023-12-27 11:08:46,453][20381] Updated weights for policy 1, policy_version 796963 (0.0008) [2023-12-27 11:08:46,509][20381] Updated weights for policy 1, policy_version 796973 (0.0007) [2023-12-27 11:08:46,560][20381] Updated weights for policy 1, policy_version 796983 (0.0007) [2023-12-27 11:08:46,636][20347] Updated weights for policy 0, policy_version 797476 (0.0009) [2023-12-27 11:08:46,694][20347] Updated weights for policy 0, policy_version 797486 (0.0006) [2023-12-27 11:08:46,750][20347] Updated weights for policy 0, policy_version 797496 (0.0009) [2023-12-27 11:08:47,294][20381] Updated weights for policy 1, policy_version 796993 (0.0008) [2023-12-27 11:08:47,350][20381] Updated weights for policy 1, policy_version 797003 (0.0008) [2023-12-27 11:08:47,362][19170] Fps is (10 sec: 18841.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 408240128. Throughput: 0: 10243.5, 1: 10145.9. Samples: 408255340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:47,363][19170] Avg episode reward: [(0, '913.560'), (1, '991.220')] [2023-12-27 11:08:47,413][20381] Updated weights for policy 1, policy_version 797013 (0.0009) [2023-12-27 11:08:47,432][20347] Updated weights for policy 0, policy_version 797506 (0.0010) [2023-12-27 11:08:47,496][20347] Updated weights for policy 0, policy_version 797516 (0.0011) [2023-12-27 11:08:47,550][20347] Updated weights for policy 0, policy_version 797526 (0.0008) [2023-12-27 11:08:47,611][20347] Updated weights for policy 0, policy_version 797536 (0.0011) [2023-12-27 11:08:48,145][20381] Updated weights for policy 1, policy_version 797023 (0.0008) [2023-12-27 11:08:48,202][20381] Updated weights for policy 1, policy_version 797033 (0.0008) [2023-12-27 11:08:48,263][20381] Updated weights for policy 1, policy_version 797043 (0.0008) [2023-12-27 11:08:48,272][20347] Updated weights for policy 0, policy_version 797546 (0.0007) [2023-12-27 11:08:48,327][20347] Updated weights for policy 0, policy_version 797556 (0.0009) [2023-12-27 11:08:48,394][20347] Updated weights for policy 0, policy_version 797566 (0.0010) [2023-12-27 11:08:49,031][20381] Updated weights for policy 1, policy_version 797053 (0.0006) [2023-12-27 11:08:49,062][20347] Updated weights for policy 0, policy_version 797576 (0.0010) [2023-12-27 11:08:49,087][20381] Updated weights for policy 1, policy_version 797063 (0.0006) [2023-12-27 11:08:49,130][20347] Updated weights for policy 0, policy_version 797586 (0.0010) [2023-12-27 11:08:49,156][20381] Updated weights for policy 1, policy_version 797073 (0.0007) [2023-12-27 11:08:49,201][20347] Updated weights for policy 0, policy_version 797596 (0.0008) [2023-12-27 11:08:49,893][20381] Updated weights for policy 1, policy_version 797083 (0.0007) [2023-12-27 11:08:49,968][20381] Updated weights for policy 1, policy_version 797093 (0.0009) [2023-12-27 11:08:50,022][20381] Updated weights for policy 1, policy_version 797103 (0.0007) [2023-12-27 11:08:50,032][20347] Updated weights for policy 0, policy_version 797606 (0.0008) [2023-12-27 11:08:50,081][20347] Updated weights for policy 0, policy_version 797616 (0.0008) [2023-12-27 11:08:50,130][20347] Updated weights for policy 0, policy_version 797626 (0.0008) [2023-12-27 11:08:50,713][20381] Updated weights for policy 1, policy_version 797113 (0.0006) [2023-12-27 11:08:50,775][20381] Updated weights for policy 1, policy_version 797123 (0.0010) [2023-12-27 11:08:50,837][20381] Updated weights for policy 1, policy_version 797133 (0.0010) [2023-12-27 11:08:50,897][20381] Updated weights for policy 1, policy_version 797143 (0.0011) [2023-12-27 11:08:50,941][20347] Updated weights for policy 0, policy_version 797636 (0.0008) [2023-12-27 11:08:51,006][20347] Updated weights for policy 0, policy_version 797646 (0.0008) [2023-12-27 11:08:51,068][20347] Updated weights for policy 0, policy_version 797656 (0.0008) [2023-12-27 11:08:51,685][20381] Updated weights for policy 1, policy_version 797153 (0.0011) [2023-12-27 11:08:51,752][20381] Updated weights for policy 1, policy_version 797163 (0.0009) [2023-12-27 11:08:51,812][20381] Updated weights for policy 1, policy_version 797173 (0.0008) [2023-12-27 11:08:51,848][20347] Updated weights for policy 0, policy_version 797666 (0.0007) [2023-12-27 11:08:51,910][20347] Updated weights for policy 0, policy_version 797676 (0.0008) [2023-12-27 11:08:51,968][20347] Updated weights for policy 0, policy_version 797686 (0.0007) [2023-12-27 11:08:52,030][20347] Updated weights for policy 0, policy_version 797696 (0.0008) [2023-12-27 11:08:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 408338432. Throughput: 0: 10209.9, 1: 10065.3. Samples: 408312604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:52,363][19170] Avg episode reward: [(0, '931.780'), (1, '1000.000')] [2023-12-27 11:08:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000797176_204103680.pth... [2023-12-27 11:08:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000797696_204234752.pth... [2023-12-27 11:08:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000796024_203808768.pth [2023-12-27 11:08:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000796512_203931648.pth [2023-12-27 11:08:52,585][20381] Updated weights for policy 1, policy_version 797183 (0.0008) [2023-12-27 11:08:52,653][20381] Updated weights for policy 1, policy_version 797193 (0.0007) [2023-12-27 11:08:52,720][20381] Updated weights for policy 1, policy_version 797203 (0.0006) [2023-12-27 11:08:52,804][20347] Updated weights for policy 0, policy_version 797706 (0.0009) [2023-12-27 11:08:52,873][20347] Updated weights for policy 0, policy_version 797716 (0.0008) [2023-12-27 11:08:52,939][20347] Updated weights for policy 0, policy_version 797726 (0.0008) [2023-12-27 11:08:53,285][20381] Updated weights for policy 1, policy_version 797213 (0.0007) [2023-12-27 11:08:53,334][20381] Updated weights for policy 1, policy_version 797223 (0.0008) [2023-12-27 11:08:53,387][20381] Updated weights for policy 1, policy_version 797233 (0.0008) [2023-12-27 11:08:53,633][20347] Updated weights for policy 0, policy_version 797736 (0.0008) [2023-12-27 11:08:53,689][20347] Updated weights for policy 0, policy_version 797746 (0.0008) [2023-12-27 11:08:53,744][20347] Updated weights for policy 0, policy_version 797756 (0.0008) [2023-12-27 11:08:54,095][20381] Updated weights for policy 1, policy_version 797243 (0.0006) [2023-12-27 11:08:54,151][20381] Updated weights for policy 1, policy_version 797253 (0.0007) [2023-12-27 11:08:54,213][20381] Updated weights for policy 1, policy_version 797263 (0.0006) [2023-12-27 11:08:54,321][20347] Updated weights for policy 0, policy_version 797766 (0.0008) [2023-12-27 11:08:54,367][20347] Updated weights for policy 0, policy_version 797776 (0.0008) [2023-12-27 11:08:54,423][20347] Updated weights for policy 0, policy_version 797786 (0.0008) [2023-12-27 11:08:54,839][20381] Updated weights for policy 1, policy_version 797273 (0.0005) [2023-12-27 11:08:54,908][20381] Updated weights for policy 1, policy_version 797283 (0.0006) [2023-12-27 11:08:54,971][20381] Updated weights for policy 1, policy_version 797293 (0.0006) [2023-12-27 11:08:55,031][20381] Updated weights for policy 1, policy_version 797303 (0.0006) [2023-12-27 11:08:55,148][20347] Updated weights for policy 0, policy_version 797796 (0.0008) [2023-12-27 11:08:55,203][20347] Updated weights for policy 0, policy_version 797806 (0.0008) [2023-12-27 11:08:55,256][20347] Updated weights for policy 0, policy_version 797816 (0.0008) [2023-12-27 11:08:55,636][20381] Updated weights for policy 1, policy_version 797313 (0.0006) [2023-12-27 11:08:55,688][20381] Updated weights for policy 1, policy_version 797323 (0.0007) [2023-12-27 11:08:55,745][20381] Updated weights for policy 1, policy_version 797333 (0.0007) [2023-12-27 11:08:56,000][20347] Updated weights for policy 0, policy_version 797826 (0.0007) [2023-12-27 11:08:56,056][20347] Updated weights for policy 0, policy_version 797836 (0.0007) [2023-12-27 11:08:56,125][20347] Updated weights for policy 0, policy_version 797846 (0.0007) [2023-12-27 11:08:56,187][20347] Updated weights for policy 0, policy_version 797856 (0.0007) [2023-12-27 11:08:56,370][20381] Updated weights for policy 1, policy_version 797343 (0.0008) [2023-12-27 11:08:56,424][20381] Updated weights for policy 1, policy_version 797353 (0.0011) [2023-12-27 11:08:56,481][20381] Updated weights for policy 1, policy_version 797363 (0.0011) [2023-12-27 11:08:56,884][20347] Updated weights for policy 0, policy_version 797866 (0.0005) [2023-12-27 11:08:56,935][20347] Updated weights for policy 0, policy_version 797876 (0.0005) [2023-12-27 11:08:56,994][20347] Updated weights for policy 0, policy_version 797886 (0.0007) [2023-12-27 11:08:57,137][20381] Updated weights for policy 1, policy_version 797373 (0.0010) [2023-12-27 11:08:57,192][20381] Updated weights for policy 1, policy_version 797383 (0.0010) [2023-12-27 11:08:57,247][20381] Updated weights for policy 1, policy_version 797393 (0.0010) [2023-12-27 11:08:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 408444928. Throughput: 0: 10150.2, 1: 10058.5. Samples: 408430860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:08:57,363][19170] Avg episode reward: [(0, '873.930'), (1, '1000.000')] [2023-12-27 11:08:57,558][20291] KL-divergence is very high: 135.8152 [2023-12-27 11:08:57,564][20291] KL-divergence is very high: 110.9452 [2023-12-27 11:08:57,613][20347] Updated weights for policy 0, policy_version 797896 (0.0006) [2023-12-27 11:08:57,614][20291] KL-divergence is very high: 112.4721 [2023-12-27 11:08:57,660][20291] KL-divergence is very high: 152.5318 [2023-12-27 11:08:57,665][20291] KL-divergence is very high: 173.7525 [2023-12-27 11:08:57,670][20347] Updated weights for policy 0, policy_version 797906 (0.0006) [2023-12-27 11:08:57,670][20291] KL-divergence is very high: 144.9084 [2023-12-27 11:08:57,675][20291] KL-divergence is very high: 117.5696 [2023-12-27 11:08:57,701][20291] KL-divergence is very high: 100.5489 [2023-12-27 11:08:57,712][20291] KL-divergence is very high: 105.8258 [2023-12-27 11:08:57,717][20291] KL-divergence is very high: 136.6981 [2023-12-27 11:08:57,723][20347] Updated weights for policy 0, policy_version 797916 (0.0005) [2023-12-27 11:08:57,916][20381] Updated weights for policy 1, policy_version 797403 (0.0010) [2023-12-27 11:08:57,968][20381] Updated weights for policy 1, policy_version 797413 (0.0010) [2023-12-27 11:08:58,015][20381] Updated weights for policy 1, policy_version 797423 (0.0008) [2023-12-27 11:08:58,302][20347] Updated weights for policy 0, policy_version 797926 (0.0009) [2023-12-27 11:08:58,367][20347] Updated weights for policy 0, policy_version 797936 (0.0009) [2023-12-27 11:08:58,431][20347] Updated weights for policy 0, policy_version 797946 (0.0008) [2023-12-27 11:08:58,688][20381] Updated weights for policy 1, policy_version 797433 (0.0005) [2023-12-27 11:08:58,747][20381] Updated weights for policy 1, policy_version 797443 (0.0008) [2023-12-27 11:08:58,814][20381] Updated weights for policy 1, policy_version 797453 (0.0009) [2023-12-27 11:08:58,874][20381] Updated weights for policy 1, policy_version 797463 (0.0008) [2023-12-27 11:08:59,241][20347] Updated weights for policy 0, policy_version 797956 (0.0008) [2023-12-27 11:08:59,308][20347] Updated weights for policy 0, policy_version 797966 (0.0008) [2023-12-27 11:08:59,378][20347] Updated weights for policy 0, policy_version 797976 (0.0008) [2023-12-27 11:08:59,655][20381] Updated weights for policy 1, policy_version 797473 (0.0010) [2023-12-27 11:08:59,717][20381] Updated weights for policy 1, policy_version 797483 (0.0011) [2023-12-27 11:08:59,768][20381] Updated weights for policy 1, policy_version 797493 (0.0009) [2023-12-27 11:09:00,123][20347] Updated weights for policy 0, policy_version 797986 (0.0008) [2023-12-27 11:09:00,185][20347] Updated weights for policy 0, policy_version 797996 (0.0009) [2023-12-27 11:09:00,228][20291] KL-divergence is very high: 146.0871 [2023-12-27 11:09:00,235][20291] KL-divergence is very high: 130.2945 [2023-12-27 11:09:00,247][20347] Updated weights for policy 0, policy_version 798006 (0.0010) [2023-12-27 11:09:00,307][20347] Updated weights for policy 0, policy_version 798016 (0.0011) [2023-12-27 11:09:00,475][20381] Updated weights for policy 1, policy_version 797503 (0.0009) [2023-12-27 11:09:00,523][20381] Updated weights for policy 1, policy_version 797513 (0.0010) [2023-12-27 11:09:00,571][20381] Updated weights for policy 1, policy_version 797523 (0.0010) [2023-12-27 11:09:00,991][20347] Updated weights for policy 0, policy_version 798026 (0.0007) [2023-12-27 11:09:00,993][20291] KL-divergence is very high: 145.1877 [2023-12-27 11:09:01,044][20291] KL-divergence is very high: 281.3470 [2023-12-27 11:09:01,056][20347] Updated weights for policy 0, policy_version 798036 (0.0010) [2023-12-27 11:09:01,086][20291] KL-divergence is very high: 251.0350 [2023-12-27 11:09:01,109][20347] Updated weights for policy 0, policy_version 798046 (0.0010) [2023-12-27 11:09:01,224][20381] Updated weights for policy 1, policy_version 797533 (0.0011) [2023-12-27 11:09:01,287][20381] Updated weights for policy 1, policy_version 797543 (0.0011) [2023-12-27 11:09:01,346][20381] Updated weights for policy 1, policy_version 797553 (0.0011) [2023-12-27 11:09:01,800][20347] Updated weights for policy 0, policy_version 798056 (0.0009) [2023-12-27 11:09:01,856][20347] Updated weights for policy 0, policy_version 798066 (0.0008) [2023-12-27 11:09:01,904][20347] Updated weights for policy 0, policy_version 798076 (0.0007) [2023-12-27 11:09:02,010][20381] Updated weights for policy 1, policy_version 797563 (0.0008) [2023-12-27 11:09:02,074][20381] Updated weights for policy 1, policy_version 797573 (0.0006) [2023-12-27 11:09:02,139][20381] Updated weights for policy 1, policy_version 797583 (0.0007) [2023-12-27 11:09:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 408543232. Throughput: 0: 10083.7, 1: 10055.9. Samples: 408551600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:02,363][19170] Avg episode reward: [(0, '616.970'), (1, '1000.000')] [2023-12-27 11:09:02,662][20347] Updated weights for policy 0, policy_version 798086 (0.0008) [2023-12-27 11:09:02,717][20347] Updated weights for policy 0, policy_version 798096 (0.0007) [2023-12-27 11:09:02,768][20347] Updated weights for policy 0, policy_version 798106 (0.0006) [2023-12-27 11:09:02,782][20381] Updated weights for policy 1, policy_version 797593 (0.0008) [2023-12-27 11:09:02,838][20381] Updated weights for policy 1, policy_version 797603 (0.0011) [2023-12-27 11:09:02,889][20381] Updated weights for policy 1, policy_version 797613 (0.0010) [2023-12-27 11:09:02,942][20381] Updated weights for policy 1, policy_version 797623 (0.0010) [2023-12-27 11:09:03,472][20347] Updated weights for policy 0, policy_version 798116 (0.0006) [2023-12-27 11:09:03,516][20347] Updated weights for policy 0, policy_version 798126 (0.0007) [2023-12-27 11:09:03,559][20347] Updated weights for policy 0, policy_version 798136 (0.0007) [2023-12-27 11:09:03,629][20381] Updated weights for policy 1, policy_version 797633 (0.0011) [2023-12-27 11:09:03,687][20381] Updated weights for policy 1, policy_version 797643 (0.0011) [2023-12-27 11:09:03,750][20381] Updated weights for policy 1, policy_version 797653 (0.0011) [2023-12-27 11:09:04,203][20347] Updated weights for policy 0, policy_version 798146 (0.0008) [2023-12-27 11:09:04,265][20347] Updated weights for policy 0, policy_version 798156 (0.0010) [2023-12-27 11:09:04,324][20347] Updated weights for policy 0, policy_version 798166 (0.0010) [2023-12-27 11:09:04,386][20381] Updated weights for policy 1, policy_version 797663 (0.0011) [2023-12-27 11:09:04,392][20347] Updated weights for policy 0, policy_version 798176 (0.0012) [2023-12-27 11:09:04,450][20381] Updated weights for policy 1, policy_version 797673 (0.0007) [2023-12-27 11:09:04,499][20381] Updated weights for policy 1, policy_version 797683 (0.0007) [2023-12-27 11:09:05,054][20347] Updated weights for policy 0, policy_version 798186 (0.0010) [2023-12-27 11:09:05,106][20347] Updated weights for policy 0, policy_version 798196 (0.0007) [2023-12-27 11:09:05,158][20381] Updated weights for policy 1, policy_version 797693 (0.0007) [2023-12-27 11:09:05,167][20347] Updated weights for policy 0, policy_version 798206 (0.0009) [2023-12-27 11:09:05,215][20381] Updated weights for policy 1, policy_version 797703 (0.0008) [2023-12-27 11:09:05,270][20381] Updated weights for policy 1, policy_version 797713 (0.0009) [2023-12-27 11:09:05,913][20347] Updated weights for policy 0, policy_version 798216 (0.0010) [2023-12-27 11:09:05,959][20381] Updated weights for policy 1, policy_version 797723 (0.0010) [2023-12-27 11:09:05,971][20347] Updated weights for policy 0, policy_version 798226 (0.0010) [2023-12-27 11:09:06,015][20381] Updated weights for policy 1, policy_version 797733 (0.0011) [2023-12-27 11:09:06,028][20347] Updated weights for policy 0, policy_version 798236 (0.0011) [2023-12-27 11:09:06,070][20381] Updated weights for policy 1, policy_version 797743 (0.0009) [2023-12-27 11:09:06,714][20347] Updated weights for policy 0, policy_version 798246 (0.0010) [2023-12-27 11:09:06,769][20347] Updated weights for policy 0, policy_version 798256 (0.0010) [2023-12-27 11:09:06,781][20381] Updated weights for policy 1, policy_version 797753 (0.0008) [2023-12-27 11:09:06,830][20347] Updated weights for policy 0, policy_version 798266 (0.0010) [2023-12-27 11:09:06,831][20381] Updated weights for policy 1, policy_version 797763 (0.0007) [2023-12-27 11:09:06,877][20381] Updated weights for policy 1, policy_version 797773 (0.0007) [2023-12-27 11:09:06,930][20381] Updated weights for policy 1, policy_version 797783 (0.0008) [2023-12-27 11:09:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 408641536. Throughput: 0: 10036.3, 1: 10104.1. Samples: 408613912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:07,363][19170] Avg episode reward: [(0, '651.810'), (1, '969.360')] [2023-12-27 11:09:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000797784_204259328.pth... [2023-12-27 11:09:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000798272_204382208.pth... [2023-12-27 11:09:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000796632_203964416.pth [2023-12-27 11:09:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000797152_204095488.pth [2023-12-27 11:09:07,515][20347] Updated weights for policy 0, policy_version 798276 (0.0011) [2023-12-27 11:09:07,564][20347] Updated weights for policy 0, policy_version 798286 (0.0010) [2023-12-27 11:09:07,618][20381] Updated weights for policy 1, policy_version 797793 (0.0007) [2023-12-27 11:09:07,621][20347] Updated weights for policy 0, policy_version 798296 (0.0010) [2023-12-27 11:09:07,668][20381] Updated weights for policy 1, policy_version 797803 (0.0006) [2023-12-27 11:09:07,711][20381] Updated weights for policy 1, policy_version 797813 (0.0006) [2023-12-27 11:09:08,290][20347] Updated weights for policy 0, policy_version 798306 (0.0010) [2023-12-27 11:09:08,340][20381] Updated weights for policy 1, policy_version 797823 (0.0006) [2023-12-27 11:09:08,350][20347] Updated weights for policy 0, policy_version 798316 (0.0008) [2023-12-27 11:09:08,399][20381] Updated weights for policy 1, policy_version 797833 (0.0007) [2023-12-27 11:09:08,415][20347] Updated weights for policy 0, policy_version 798326 (0.0007) [2023-12-27 11:09:08,457][20381] Updated weights for policy 1, policy_version 797843 (0.0006) [2023-12-27 11:09:08,476][20347] Updated weights for policy 0, policy_version 798336 (0.0006) [2023-12-27 11:09:09,080][20347] Updated weights for policy 0, policy_version 798346 (0.0006) [2023-12-27 11:09:09,141][20347] Updated weights for policy 0, policy_version 798356 (0.0008) [2023-12-27 11:09:09,158][20381] Updated weights for policy 1, policy_version 797853 (0.0006) [2023-12-27 11:09:09,197][20347] Updated weights for policy 0, policy_version 798366 (0.0006) [2023-12-27 11:09:09,207][20381] Updated weights for policy 1, policy_version 797863 (0.0006) [2023-12-27 11:09:09,267][20381] Updated weights for policy 1, policy_version 797873 (0.0008) [2023-12-27 11:09:09,912][20347] Updated weights for policy 0, policy_version 798376 (0.0007) [2023-12-27 11:09:09,979][20347] Updated weights for policy 0, policy_version 798386 (0.0008) [2023-12-27 11:09:10,038][20347] Updated weights for policy 0, policy_version 798396 (0.0008) [2023-12-27 11:09:10,040][20381] Updated weights for policy 1, policy_version 797883 (0.0007) [2023-12-27 11:09:10,096][20381] Updated weights for policy 1, policy_version 797893 (0.0007) [2023-12-27 11:09:10,153][20381] Updated weights for policy 1, policy_version 797903 (0.0008) [2023-12-27 11:09:10,744][20347] Updated weights for policy 0, policy_version 798406 (0.0007) [2023-12-27 11:09:10,798][20347] Updated weights for policy 0, policy_version 798416 (0.0006) [2023-12-27 11:09:10,842][20381] Updated weights for policy 1, policy_version 797913 (0.0009) [2023-12-27 11:09:10,856][20347] Updated weights for policy 0, policy_version 798426 (0.0005) [2023-12-27 11:09:10,906][20381] Updated weights for policy 1, policy_version 797923 (0.0007) [2023-12-27 11:09:10,963][20381] Updated weights for policy 1, policy_version 797933 (0.0005) [2023-12-27 11:09:11,017][20381] Updated weights for policy 1, policy_version 797943 (0.0006) [2023-12-27 11:09:11,508][20347] Updated weights for policy 0, policy_version 798436 (0.0007) [2023-12-27 11:09:11,557][20347] Updated weights for policy 0, policy_version 798446 (0.0009) [2023-12-27 11:09:11,611][20347] Updated weights for policy 0, policy_version 798456 (0.0007) [2023-12-27 11:09:11,709][20381] Updated weights for policy 1, policy_version 797953 (0.0006) [2023-12-27 11:09:11,776][20381] Updated weights for policy 1, policy_version 797963 (0.0007) [2023-12-27 11:09:11,840][20381] Updated weights for policy 1, policy_version 797973 (0.0008) [2023-12-27 11:09:12,314][20347] Updated weights for policy 0, policy_version 798466 (0.0008) [2023-12-27 11:09:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20438.4). Total num frames: 408739840. Throughput: 0: 9991.6, 1: 10086.7. Samples: 408735116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:12,363][19170] Avg episode reward: [(0, '784.200'), (1, '966.150')] [2023-12-27 11:09:12,379][20347] Updated weights for policy 0, policy_version 798476 (0.0008) [2023-12-27 11:09:12,436][20347] Updated weights for policy 0, policy_version 798486 (0.0008) [2023-12-27 11:09:12,487][20347] Updated weights for policy 0, policy_version 798496 (0.0007) [2023-12-27 11:09:12,551][20381] Updated weights for policy 1, policy_version 797983 (0.0007) [2023-12-27 11:09:12,618][20381] Updated weights for policy 1, policy_version 797993 (0.0007) [2023-12-27 11:09:12,667][20381] Updated weights for policy 1, policy_version 798003 (0.0010) [2023-12-27 11:09:13,117][20291] KL-divergence is very high: 116.2857 [2023-12-27 11:09:13,123][20291] KL-divergence is very high: 121.5439 [2023-12-27 11:09:13,161][20347] Updated weights for policy 0, policy_version 798506 (0.0006) [2023-12-27 11:09:13,217][20347] Updated weights for policy 0, policy_version 798516 (0.0006) [2023-12-27 11:09:13,270][20347] Updated weights for policy 0, policy_version 798526 (0.0005) [2023-12-27 11:09:13,333][20381] Updated weights for policy 1, policy_version 798013 (0.0008) [2023-12-27 11:09:13,381][20381] Updated weights for policy 1, policy_version 798023 (0.0007) [2023-12-27 11:09:13,431][20381] Updated weights for policy 1, policy_version 798033 (0.0006) [2023-12-27 11:09:13,918][20347] Updated weights for policy 0, policy_version 798536 (0.0007) [2023-12-27 11:09:13,922][20291] KL-divergence is very high: 106.9216 [2023-12-27 11:09:13,928][20291] KL-divergence is very high: 112.7342 [2023-12-27 11:09:13,971][20347] Updated weights for policy 0, policy_version 798546 (0.0008) [2023-12-27 11:09:14,020][20347] Updated weights for policy 0, policy_version 798556 (0.0007) [2023-12-27 11:09:14,102][20381] Updated weights for policy 1, policy_version 798043 (0.0006) [2023-12-27 11:09:14,145][20381] Updated weights for policy 1, policy_version 798053 (0.0007) [2023-12-27 11:09:14,200][20381] Updated weights for policy 1, policy_version 798063 (0.0008) [2023-12-27 11:09:14,732][20347] Updated weights for policy 0, policy_version 798566 (0.0007) [2023-12-27 11:09:14,798][20347] Updated weights for policy 0, policy_version 798576 (0.0007) [2023-12-27 11:09:14,800][20291] KL-divergence is very high: 157.2546 [2023-12-27 11:09:14,811][20291] KL-divergence is very high: 317.7271 [2023-12-27 11:09:14,816][20291] KL-divergence is very high: 451.1061 [2023-12-27 11:09:14,823][20291] KL-divergence is very high: 835.0377 [2023-12-27 11:09:14,839][20291] KL-divergence is very high: 730.0707 [2023-12-27 11:09:14,846][20291] KL-divergence is very high: 814.6868 [2023-12-27 11:09:14,856][20291] KL-divergence is very high: 153.2562 [2023-12-27 11:09:14,856][20347] Updated weights for policy 0, policy_version 798586 (0.0008) [2023-12-27 11:09:14,868][20291] KL-divergence is very high: 1005.1108 [2023-12-27 11:09:14,924][20381] Updated weights for policy 1, policy_version 798073 (0.0007) [2023-12-27 11:09:14,975][20381] Updated weights for policy 1, policy_version 798083 (0.0007) [2023-12-27 11:09:15,028][20381] Updated weights for policy 1, policy_version 798093 (0.0008) [2023-12-27 11:09:15,080][20381] Updated weights for policy 1, policy_version 798103 (0.0007) [2023-12-27 11:09:15,555][20291] KL-divergence is very high: 1096.6263 [2023-12-27 11:09:15,561][20291] KL-divergence is very high: 1098.4396 [2023-12-27 11:09:15,567][20291] KL-divergence is very high: 496.3020 [2023-12-27 11:09:15,568][20347] Updated weights for policy 0, policy_version 798596 (0.0007) [2023-12-27 11:09:15,578][20291] KL-divergence is very high: 106.3767 [2023-12-27 11:09:15,601][20291] KL-divergence is very high: 441.1469 [2023-12-27 11:09:15,607][20291] KL-divergence is very high: 510.3150 [2023-12-27 11:09:15,613][20291] KL-divergence is very high: 461.9492 [2023-12-27 11:09:15,623][20347] Updated weights for policy 0, policy_version 798606 (0.0007) [2023-12-27 11:09:15,628][20291] KL-divergence is very high: 321.5403 [2023-12-27 11:09:15,642][20291] KL-divergence is very high: 671.3417 [2023-12-27 11:09:15,647][20291] KL-divergence is very high: 1721.0886 [2023-12-27 11:09:15,653][20291] KL-divergence is very high: 805.2621 [2023-12-27 11:09:15,675][20347] Updated weights for policy 0, policy_version 798616 (0.0007) [2023-12-27 11:09:15,684][20291] KL-divergence is very high: 839.5934 [2023-12-27 11:09:15,690][20291] KL-divergence is very high: 2027.3635 [2023-12-27 11:09:15,696][20291] KL-divergence is very high: 1372.1011 [2023-12-27 11:09:15,712][20291] KL-divergence is very high: 195.3285 [2023-12-27 11:09:15,797][20381] Updated weights for policy 1, policy_version 798113 (0.0008) [2023-12-27 11:09:15,850][20381] Updated weights for policy 1, policy_version 798123 (0.0008) [2023-12-27 11:09:15,903][20381] Updated weights for policy 1, policy_version 798133 (0.0008) [2023-12-27 11:09:16,275][20291] KL-divergence is very high: 432.1574 [2023-12-27 11:09:16,281][20291] KL-divergence is very high: 344.3372 [2023-12-27 11:09:16,286][20347] Updated weights for policy 0, policy_version 798626 (0.0008) [2023-12-27 11:09:16,286][20291] KL-divergence is very high: 479.9261 [2023-12-27 11:09:16,291][20291] KL-divergence is very high: 626.3647 [2023-12-27 11:09:16,296][20291] KL-divergence is very high: 179.8226 [2023-12-27 11:09:16,300][20291] KL-divergence is very high: 530.4537 [2023-12-27 11:09:16,306][20291] KL-divergence is very high: 785.8524 [2023-12-27 11:09:16,315][20291] KL-divergence is very high: 853.1962 [2023-12-27 11:09:16,320][20291] KL-divergence is very high: 802.8082 [2023-12-27 11:09:16,326][20291] KL-divergence is very high: 581.7670 [2023-12-27 11:09:16,331][20291] KL-divergence is very high: 384.5350 [2023-12-27 11:09:16,336][20291] KL-divergence is very high: 203.0798 [2023-12-27 11:09:16,338][20347] Updated weights for policy 0, policy_version 798636 (0.0008) [2023-12-27 11:09:16,341][20291] KL-divergence is very high: 384.6302 [2023-12-27 11:09:16,347][20291] KL-divergence is very high: 278.0672 [2023-12-27 11:09:16,356][20291] KL-divergence is very high: 526.0518 [2023-12-27 11:09:16,363][20291] KL-divergence is very high: 799.2134 [2023-12-27 11:09:16,369][20291] KL-divergence is very high: 893.6877 [2023-12-27 11:09:16,375][20291] KL-divergence is very high: 485.7396 [2023-12-27 11:09:16,383][20291] KL-divergence is very high: 380.9424 [2023-12-27 11:09:16,391][20291] KL-divergence is very high: 477.9617 [2023-12-27 11:09:16,396][20347] Updated weights for policy 0, policy_version 798646 (0.0007) [2023-12-27 11:09:16,398][20291] KL-divergence is very high: 317.0996 [2023-12-27 11:09:16,411][20291] KL-divergence is very high: 246.6709 [2023-12-27 11:09:16,417][20291] KL-divergence is very high: 396.2048 [2023-12-27 11:09:16,424][20291] KL-divergence is very high: 475.8665 [2023-12-27 11:09:16,431][20291] KL-divergence is very high: 252.8723 [2023-12-27 11:09:16,437][20291] KL-divergence is very high: 285.3506 [2023-12-27 11:09:16,444][20291] KL-divergence is very high: 533.1207 [2023-12-27 11:09:16,450][20291] KL-divergence is very high: 507.6340 [2023-12-27 11:09:16,460][20347] Updated weights for policy 0, policy_version 798656 (0.0005) [2023-12-27 11:09:16,545][20381] Updated weights for policy 1, policy_version 798143 (0.0007) [2023-12-27 11:09:16,598][20381] Updated weights for policy 1, policy_version 798153 (0.0005) [2023-12-27 11:09:16,666][20381] Updated weights for policy 1, policy_version 798163 (0.0007) [2023-12-27 11:09:17,022][20291] KL-divergence is very high: 224.8568 [2023-12-27 11:09:17,028][20291] KL-divergence is very high: 231.4016 [2023-12-27 11:09:17,034][20291] KL-divergence is very high: 259.3748 [2023-12-27 11:09:17,040][20291] KL-divergence is very high: 160.6712 [2023-12-27 11:09:17,046][20291] KL-divergence is very high: 187.6191 [2023-12-27 11:09:17,052][20291] KL-divergence is very high: 159.2619 [2023-12-27 11:09:17,063][20291] KL-divergence is very high: 200.6222 [2023-12-27 11:09:17,069][20291] KL-divergence is very high: 260.4113 [2023-12-27 11:09:17,075][20291] KL-divergence is very high: 232.0554 [2023-12-27 11:09:17,080][20347] Updated weights for policy 0, policy_version 798666 (0.0006) [2023-12-27 11:09:17,081][20291] KL-divergence is very high: 217.2154 [2023-12-27 11:09:17,087][20291] KL-divergence is very high: 108.9940 [2023-12-27 11:09:17,092][20291] KL-divergence is very high: 130.4996 [2023-12-27 11:09:17,098][20291] KL-divergence is very high: 142.0137 [2023-12-27 11:09:17,112][20291] KL-divergence is very high: 109.0853 [2023-12-27 11:09:17,118][20291] KL-divergence is very high: 189.4865 [2023-12-27 11:09:17,124][20291] KL-divergence is very high: 217.8880 [2023-12-27 11:09:17,130][20291] KL-divergence is very high: 261.5522 [2023-12-27 11:09:17,136][20291] KL-divergence is very high: 169.6702 [2023-12-27 11:09:17,141][20347] Updated weights for policy 0, policy_version 798676 (0.0005) [2023-12-27 11:09:17,142][20291] KL-divergence is very high: 183.4597 [2023-12-27 11:09:17,148][20291] KL-divergence is very high: 140.6680 [2023-12-27 11:09:17,161][20291] KL-divergence is very high: 155.3242 [2023-12-27 11:09:17,168][20291] KL-divergence is very high: 216.4578 [2023-12-27 11:09:17,175][20291] KL-divergence is very high: 205.0085 [2023-12-27 11:09:17,181][20291] KL-divergence is very high: 218.7294 [2023-12-27 11:09:17,188][20291] KL-divergence is very high: 125.6505 [2023-12-27 11:09:17,194][20291] KL-divergence is very high: 138.5846 [2023-12-27 11:09:17,201][20291] KL-divergence is very high: 107.0205 [2023-12-27 11:09:17,205][20347] Updated weights for policy 0, policy_version 798686 (0.0005) [2023-12-27 11:09:17,212][20291] KL-divergence is very high: 137.1637 [2023-12-27 11:09:17,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 408846336. Throughput: 0: 9984.8, 1: 10060.7. Samples: 408858164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:17,364][19170] Avg episode reward: [(0, '53.590'), (1, '995.250')] [2023-12-27 11:09:17,377][20381] Updated weights for policy 1, policy_version 798173 (0.0008) [2023-12-27 11:09:17,430][20381] Updated weights for policy 1, policy_version 798183 (0.0009) [2023-12-27 11:09:17,477][20381] Updated weights for policy 1, policy_version 798193 (0.0008) [2023-12-27 11:09:17,785][20347] Updated weights for policy 0, policy_version 798696 (0.0007) [2023-12-27 11:09:17,847][20347] Updated weights for policy 0, policy_version 798706 (0.0006) [2023-12-27 11:09:17,898][20347] Updated weights for policy 0, policy_version 798716 (0.0005) [2023-12-27 11:09:18,206][20381] Updated weights for policy 1, policy_version 798203 (0.0008) [2023-12-27 11:09:18,255][20381] Updated weights for policy 1, policy_version 798213 (0.0008) [2023-12-27 11:09:18,312][20381] Updated weights for policy 1, policy_version 798223 (0.0009) [2023-12-27 11:09:18,470][20347] Updated weights for policy 0, policy_version 798726 (0.0007) [2023-12-27 11:09:18,526][20347] Updated weights for policy 0, policy_version 798736 (0.0008) [2023-12-27 11:09:18,592][20347] Updated weights for policy 0, policy_version 798746 (0.0005) [2023-12-27 11:09:19,067][20381] Updated weights for policy 1, policy_version 798233 (0.0008) [2023-12-27 11:09:19,129][20381] Updated weights for policy 1, policy_version 798243 (0.0008) [2023-12-27 11:09:19,151][20347] Updated weights for policy 0, policy_version 798756 (0.0006) [2023-12-27 11:09:19,182][20381] Updated weights for policy 1, policy_version 798253 (0.0007) [2023-12-27 11:09:19,200][20347] Updated weights for policy 0, policy_version 798766 (0.0005) [2023-12-27 11:09:19,240][20381] Updated weights for policy 1, policy_version 798263 (0.0007) [2023-12-27 11:09:19,265][20347] Updated weights for policy 0, policy_version 798776 (0.0008) [2023-12-27 11:09:19,914][20347] Updated weights for policy 0, policy_version 798786 (0.0007) [2023-12-27 11:09:19,979][20347] Updated weights for policy 0, policy_version 798796 (0.0006) [2023-12-27 11:09:20,023][20381] Updated weights for policy 1, policy_version 798273 (0.0007) [2023-12-27 11:09:20,041][20347] Updated weights for policy 0, policy_version 798806 (0.0007) [2023-12-27 11:09:20,081][20381] Updated weights for policy 1, policy_version 798283 (0.0006) [2023-12-27 11:09:20,098][20347] Updated weights for policy 0, policy_version 798816 (0.0008) [2023-12-27 11:09:20,141][20381] Updated weights for policy 1, policy_version 798293 (0.0007) [2023-12-27 11:09:20,794][20381] Updated weights for policy 1, policy_version 798303 (0.0008) [2023-12-27 11:09:20,809][20347] Updated weights for policy 0, policy_version 798826 (0.0007) [2023-12-27 11:09:20,853][20381] Updated weights for policy 1, policy_version 798313 (0.0008) [2023-12-27 11:09:20,869][20347] Updated weights for policy 0, policy_version 798836 (0.0006) [2023-12-27 11:09:20,909][20381] Updated weights for policy 1, policy_version 798323 (0.0008) [2023-12-27 11:09:20,927][20347] Updated weights for policy 0, policy_version 798846 (0.0007) [2023-12-27 11:09:21,665][20347] Updated weights for policy 0, policy_version 798856 (0.0008) [2023-12-27 11:09:21,705][20381] Updated weights for policy 1, policy_version 798333 (0.0008) [2023-12-27 11:09:21,731][20347] Updated weights for policy 0, policy_version 798866 (0.0008) [2023-12-27 11:09:21,771][20381] Updated weights for policy 1, policy_version 798343 (0.0007) [2023-12-27 11:09:21,796][20347] Updated weights for policy 0, policy_version 798876 (0.0008) [2023-12-27 11:09:21,832][20381] Updated weights for policy 1, policy_version 798353 (0.0007) [2023-12-27 11:09:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 408944640. Throughput: 0: 10019.9, 1: 9958.9. Samples: 408920056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:22,363][19170] Avg episode reward: [(0, '63.270'), (1, '980.900')] [2023-12-27 11:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000798360_204406784.pth... [2023-12-27 11:09:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000798880_204537856.pth... [2023-12-27 11:09:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000797696_204234752.pth [2023-12-27 11:09:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000797176_204103680.pth [2023-12-27 11:09:22,530][20381] Updated weights for policy 1, policy_version 798363 (0.0007) [2023-12-27 11:09:22,569][20347] Updated weights for policy 0, policy_version 798886 (0.0007) [2023-12-27 11:09:22,586][20381] Updated weights for policy 1, policy_version 798373 (0.0008) [2023-12-27 11:09:22,622][20347] Updated weights for policy 0, policy_version 798896 (0.0007) [2023-12-27 11:09:22,642][20381] Updated weights for policy 1, policy_version 798383 (0.0008) [2023-12-27 11:09:22,675][20347] Updated weights for policy 0, policy_version 798906 (0.0007) [2023-12-27 11:09:23,307][20347] Updated weights for policy 0, policy_version 798916 (0.0008) [2023-12-27 11:09:23,331][20381] Updated weights for policy 1, policy_version 798393 (0.0008) [2023-12-27 11:09:23,358][20347] Updated weights for policy 0, policy_version 798926 (0.0009) [2023-12-27 11:09:23,384][20381] Updated weights for policy 1, policy_version 798403 (0.0011) [2023-12-27 11:09:23,404][20347] Updated weights for policy 0, policy_version 798936 (0.0007) [2023-12-27 11:09:23,446][20381] Updated weights for policy 1, policy_version 798413 (0.0011) [2023-12-27 11:09:23,501][20381] Updated weights for policy 1, policy_version 798423 (0.0010) [2023-12-27 11:09:24,092][20381] Updated weights for policy 1, policy_version 798433 (0.0006) [2023-12-27 11:09:24,131][20347] Updated weights for policy 0, policy_version 798946 (0.0005) [2023-12-27 11:09:24,155][20381] Updated weights for policy 1, policy_version 798443 (0.0009) [2023-12-27 11:09:24,194][20347] Updated weights for policy 0, policy_version 798956 (0.0008) [2023-12-27 11:09:24,207][20381] Updated weights for policy 1, policy_version 798453 (0.0007) [2023-12-27 11:09:24,258][20347] Updated weights for policy 0, policy_version 798966 (0.0007) [2023-12-27 11:09:24,311][20347] Updated weights for policy 0, policy_version 798976 (0.0007) [2023-12-27 11:09:24,871][20381] Updated weights for policy 1, policy_version 798463 (0.0010) [2023-12-27 11:09:24,918][20381] Updated weights for policy 1, policy_version 798473 (0.0010) [2023-12-27 11:09:24,973][20381] Updated weights for policy 1, policy_version 798483 (0.0007) [2023-12-27 11:09:24,984][20347] Updated weights for policy 0, policy_version 798986 (0.0006) [2023-12-27 11:09:25,040][20347] Updated weights for policy 0, policy_version 798996 (0.0006) [2023-12-27 11:09:25,086][20347] Updated weights for policy 0, policy_version 799006 (0.0005) [2023-12-27 11:09:25,556][20381] Updated weights for policy 1, policy_version 798493 (0.0008) [2023-12-27 11:09:25,623][20381] Updated weights for policy 1, policy_version 798503 (0.0009) [2023-12-27 11:09:25,636][20347] Updated weights for policy 0, policy_version 799016 (0.0005) [2023-12-27 11:09:25,683][20381] Updated weights for policy 1, policy_version 798513 (0.0005) [2023-12-27 11:09:25,690][20347] Updated weights for policy 0, policy_version 799026 (0.0005) [2023-12-27 11:09:25,751][20347] Updated weights for policy 0, policy_version 799036 (0.0005) [2023-12-27 11:09:26,284][20381] Updated weights for policy 1, policy_version 798523 (0.0008) [2023-12-27 11:09:26,342][20381] Updated weights for policy 1, policy_version 798533 (0.0010) [2023-12-27 11:09:26,351][20347] Updated weights for policy 0, policy_version 799046 (0.0005) [2023-12-27 11:09:26,396][20381] Updated weights for policy 1, policy_version 798543 (0.0010) [2023-12-27 11:09:26,400][20347] Updated weights for policy 0, policy_version 799056 (0.0008) [2023-12-27 11:09:26,446][20347] Updated weights for policy 0, policy_version 799066 (0.0007) [2023-12-27 11:09:27,060][20381] Updated weights for policy 1, policy_version 798553 (0.0010) [2023-12-27 11:09:27,103][20347] Updated weights for policy 0, policy_version 799076 (0.0005) [2023-12-27 11:09:27,124][20381] Updated weights for policy 1, policy_version 798563 (0.0006) [2023-12-27 11:09:27,151][20347] Updated weights for policy 0, policy_version 799086 (0.0007) [2023-12-27 11:09:27,181][20381] Updated weights for policy 1, policy_version 798573 (0.0005) [2023-12-27 11:09:27,207][20347] Updated weights for policy 0, policy_version 799096 (0.0005) [2023-12-27 11:09:27,237][20381] Updated weights for policy 1, policy_version 798583 (0.0005) [2023-12-27 11:09:27,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 409059328. Throughput: 0: 10065.9, 1: 10052.8. Samples: 409044136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:27,363][19170] Avg episode reward: [(0, '67.270'), (1, '971.410')] [2023-12-27 11:09:27,852][20347] Updated weights for policy 0, policy_version 799106 (0.0007) [2023-12-27 11:09:27,907][20381] Updated weights for policy 1, policy_version 798593 (0.0009) [2023-12-27 11:09:27,907][20347] Updated weights for policy 0, policy_version 799116 (0.0009) [2023-12-27 11:09:27,955][20381] Updated weights for policy 1, policy_version 798603 (0.0010) [2023-12-27 11:09:27,957][20347] Updated weights for policy 0, policy_version 799126 (0.0008) [2023-12-27 11:09:28,011][20347] Updated weights for policy 0, policy_version 799136 (0.0008) [2023-12-27 11:09:28,012][20381] Updated weights for policy 1, policy_version 798613 (0.0007) [2023-12-27 11:09:28,673][20347] Updated weights for policy 0, policy_version 799146 (0.0008) [2023-12-27 11:09:28,717][20381] Updated weights for policy 1, policy_version 798623 (0.0006) [2023-12-27 11:09:28,734][20347] Updated weights for policy 0, policy_version 799156 (0.0006) [2023-12-27 11:09:28,785][20381] Updated weights for policy 1, policy_version 798633 (0.0007) [2023-12-27 11:09:28,793][20347] Updated weights for policy 0, policy_version 799166 (0.0006) [2023-12-27 11:09:28,846][20381] Updated weights for policy 1, policy_version 798643 (0.0005) [2023-12-27 11:09:29,448][20347] Updated weights for policy 0, policy_version 799176 (0.0007) [2023-12-27 11:09:29,512][20347] Updated weights for policy 0, policy_version 799186 (0.0008) [2023-12-27 11:09:29,515][20381] Updated weights for policy 1, policy_version 798653 (0.0008) [2023-12-27 11:09:29,571][20347] Updated weights for policy 0, policy_version 799196 (0.0006) [2023-12-27 11:09:29,576][20381] Updated weights for policy 1, policy_version 798663 (0.0008) [2023-12-27 11:09:29,606][20292] KL-divergence is very high: 331.4377 [2023-12-27 11:09:29,635][20381] Updated weights for policy 1, policy_version 798673 (0.0011) [2023-12-27 11:09:30,293][20381] Updated weights for policy 1, policy_version 798683 (0.0011) [2023-12-27 11:09:30,298][20347] Updated weights for policy 0, policy_version 799206 (0.0006) [2023-12-27 11:09:30,352][20381] Updated weights for policy 1, policy_version 798693 (0.0008) [2023-12-27 11:09:30,357][20347] Updated weights for policy 0, policy_version 799216 (0.0006) [2023-12-27 11:09:30,410][20381] Updated weights for policy 1, policy_version 798703 (0.0006) [2023-12-27 11:09:30,410][20347] Updated weights for policy 0, policy_version 799226 (0.0008) [2023-12-27 11:09:30,428][20292] KL-divergence is very high: 139.5432 [2023-12-27 11:09:31,003][20381] Updated weights for policy 1, policy_version 798713 (0.0011) [2023-12-27 11:09:31,056][20381] Updated weights for policy 1, policy_version 798723 (0.0011) [2023-12-27 11:09:31,090][20347] Updated weights for policy 0, policy_version 799236 (0.0006) [2023-12-27 11:09:31,113][20381] Updated weights for policy 1, policy_version 798733 (0.0011) [2023-12-27 11:09:31,141][20347] Updated weights for policy 0, policy_version 799246 (0.0007) [2023-12-27 11:09:31,176][20381] Updated weights for policy 1, policy_version 798743 (0.0011) [2023-12-27 11:09:31,204][20347] Updated weights for policy 0, policy_version 799256 (0.0008) [2023-12-27 11:09:31,876][20347] Updated weights for policy 0, policy_version 799266 (0.0006) [2023-12-27 11:09:31,900][20381] Updated weights for policy 1, policy_version 798753 (0.0011) [2023-12-27 11:09:31,930][20347] Updated weights for policy 0, policy_version 799276 (0.0005) [2023-12-27 11:09:31,952][20381] Updated weights for policy 1, policy_version 798763 (0.0011) [2023-12-27 11:09:31,989][20347] Updated weights for policy 0, policy_version 799286 (0.0005) [2023-12-27 11:09:32,008][20381] Updated weights for policy 1, policy_version 798773 (0.0010) [2023-12-27 11:09:32,043][20347] Updated weights for policy 0, policy_version 799296 (0.0005) [2023-12-27 11:09:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 409157632. Throughput: 0: 10166.1, 1: 10104.9. Samples: 409167536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:32,363][19170] Avg episode reward: [(0, '164.640'), (1, '942.370')] [2023-12-27 11:09:32,640][20347] Updated weights for policy 0, policy_version 799306 (0.0008) [2023-12-27 11:09:32,699][20347] Updated weights for policy 0, policy_version 799316 (0.0006) [2023-12-27 11:09:32,708][20381] Updated weights for policy 1, policy_version 798783 (0.0011) [2023-12-27 11:09:32,751][20347] Updated weights for policy 0, policy_version 799326 (0.0005) [2023-12-27 11:09:32,764][20381] Updated weights for policy 1, policy_version 798793 (0.0011) [2023-12-27 11:09:32,820][20381] Updated weights for policy 1, policy_version 798803 (0.0011) [2023-12-27 11:09:33,443][20347] Updated weights for policy 0, policy_version 799336 (0.0006) [2023-12-27 11:09:33,496][20347] Updated weights for policy 0, policy_version 799346 (0.0008) [2023-12-27 11:09:33,507][20381] Updated weights for policy 1, policy_version 798813 (0.0011) [2023-12-27 11:09:33,543][20347] Updated weights for policy 0, policy_version 799356 (0.0010) [2023-12-27 11:09:33,553][20381] Updated weights for policy 1, policy_version 798823 (0.0010) [2023-12-27 11:09:33,612][20381] Updated weights for policy 1, policy_version 798833 (0.0010) [2023-12-27 11:09:34,162][20347] Updated weights for policy 0, policy_version 799366 (0.0008) [2023-12-27 11:09:34,214][20347] Updated weights for policy 0, policy_version 799376 (0.0011) [2023-12-27 11:09:34,264][20347] Updated weights for policy 0, policy_version 799386 (0.0010) [2023-12-27 11:09:34,320][20381] Updated weights for policy 1, policy_version 798843 (0.0010) [2023-12-27 11:09:34,373][20381] Updated weights for policy 1, policy_version 798853 (0.0010) [2023-12-27 11:09:34,434][20381] Updated weights for policy 1, policy_version 798863 (0.0011) [2023-12-27 11:09:34,974][20347] Updated weights for policy 0, policy_version 799396 (0.0008) [2023-12-27 11:09:35,030][20347] Updated weights for policy 0, policy_version 799406 (0.0007) [2023-12-27 11:09:35,079][20347] Updated weights for policy 0, policy_version 799416 (0.0010) [2023-12-27 11:09:35,103][20381] Updated weights for policy 1, policy_version 798873 (0.0010) [2023-12-27 11:09:35,150][20381] Updated weights for policy 1, policy_version 798883 (0.0005) [2023-12-27 11:09:35,200][20381] Updated weights for policy 1, policy_version 798893 (0.0005) [2023-12-27 11:09:35,258][20381] Updated weights for policy 1, policy_version 798903 (0.0008) [2023-12-27 11:09:35,747][20347] Updated weights for policy 0, policy_version 799426 (0.0010) [2023-12-27 11:09:35,807][20347] Updated weights for policy 0, policy_version 799436 (0.0011) [2023-12-27 11:09:35,872][20347] Updated weights for policy 0, policy_version 799446 (0.0008) [2023-12-27 11:09:35,888][20381] Updated weights for policy 1, policy_version 798913 (0.0010) [2023-12-27 11:09:35,934][20347] Updated weights for policy 0, policy_version 799456 (0.0010) [2023-12-27 11:09:35,954][20381] Updated weights for policy 1, policy_version 798923 (0.0010) [2023-12-27 11:09:36,011][20381] Updated weights for policy 1, policy_version 798933 (0.0008) [2023-12-27 11:09:36,589][20347] Updated weights for policy 0, policy_version 799466 (0.0010) [2023-12-27 11:09:36,650][20381] Updated weights for policy 1, policy_version 798943 (0.0006) [2023-12-27 11:09:36,651][20347] Updated weights for policy 0, policy_version 799476 (0.0010) [2023-12-27 11:09:36,703][20381] Updated weights for policy 1, policy_version 798953 (0.0005) [2023-12-27 11:09:36,706][20347] Updated weights for policy 0, policy_version 799486 (0.0010) [2023-12-27 11:09:36,761][20381] Updated weights for policy 1, policy_version 798963 (0.0005) [2023-12-27 11:09:37,354][20347] Updated weights for policy 0, policy_version 799496 (0.0011) [2023-12-27 11:09:37,354][20381] Updated weights for policy 1, policy_version 798973 (0.0008) [2023-12-27 11:09:37,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 409255936. Throughput: 0: 10226.1, 1: 10159.3. Samples: 409229944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:37,363][19170] Avg episode reward: [(0, '279.690'), (1, '876.380')] [2023-12-27 11:09:37,404][20381] Updated weights for policy 1, policy_version 798983 (0.0010) [2023-12-27 11:09:37,413][20347] Updated weights for policy 0, policy_version 799506 (0.0010) [2023-12-27 11:09:37,452][20381] Updated weights for policy 1, policy_version 798993 (0.0009) [2023-12-27 11:09:37,469][20347] Updated weights for policy 0, policy_version 799516 (0.0008) [2023-12-27 11:09:37,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000799520_204701696.pth... [2023-12-27 11:09:37,492][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000799000_204570624.pth... [2023-12-27 11:09:37,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000798272_204382208.pth [2023-12-27 11:09:37,496][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000797784_204259328.pth [2023-12-27 11:09:38,071][20381] Updated weights for policy 1, policy_version 799003 (0.0007) [2023-12-27 11:09:38,083][20347] Updated weights for policy 0, policy_version 799526 (0.0005) [2023-12-27 11:09:38,130][20381] Updated weights for policy 1, policy_version 799013 (0.0010) [2023-12-27 11:09:38,134][20347] Updated weights for policy 0, policy_version 799536 (0.0006) [2023-12-27 11:09:38,186][20347] Updated weights for policy 0, policy_version 799546 (0.0006) [2023-12-27 11:09:38,191][20381] Updated weights for policy 1, policy_version 799023 (0.0011) [2023-12-27 11:09:38,846][20381] Updated weights for policy 1, policy_version 799033 (0.0009) [2023-12-27 11:09:38,887][20347] Updated weights for policy 0, policy_version 799556 (0.0006) [2023-12-27 11:09:38,912][20381] Updated weights for policy 1, policy_version 799043 (0.0007) [2023-12-27 11:09:38,947][20347] Updated weights for policy 0, policy_version 799566 (0.0007) [2023-12-27 11:09:38,968][20381] Updated weights for policy 1, policy_version 799053 (0.0006) [2023-12-27 11:09:39,007][20347] Updated weights for policy 0, policy_version 799576 (0.0006) [2023-12-27 11:09:39,026][20381] Updated weights for policy 1, policy_version 799063 (0.0007) [2023-12-27 11:09:39,660][20347] Updated weights for policy 0, policy_version 799586 (0.0006) [2023-12-27 11:09:39,726][20347] Updated weights for policy 0, policy_version 799596 (0.0007) [2023-12-27 11:09:39,736][20381] Updated weights for policy 1, policy_version 799073 (0.0007) [2023-12-27 11:09:39,784][20347] Updated weights for policy 0, policy_version 799606 (0.0007) [2023-12-27 11:09:39,786][20381] Updated weights for policy 1, policy_version 799083 (0.0005) [2023-12-27 11:09:39,846][20347] Updated weights for policy 0, policy_version 799616 (0.0008) [2023-12-27 11:09:39,846][20381] Updated weights for policy 1, policy_version 799093 (0.0007) [2023-12-27 11:09:40,521][20347] Updated weights for policy 0, policy_version 799626 (0.0008) [2023-12-27 11:09:40,578][20347] Updated weights for policy 0, policy_version 799636 (0.0007) [2023-12-27 11:09:40,584][20381] Updated weights for policy 1, policy_version 799103 (0.0006) [2023-12-27 11:09:40,634][20347] Updated weights for policy 0, policy_version 799646 (0.0008) [2023-12-27 11:09:40,640][20381] Updated weights for policy 1, policy_version 799113 (0.0006) [2023-12-27 11:09:40,693][20381] Updated weights for policy 1, policy_version 799123 (0.0008) [2023-12-27 11:09:41,352][20347] Updated weights for policy 0, policy_version 799656 (0.0007) [2023-12-27 11:09:41,407][20381] Updated weights for policy 1, policy_version 799133 (0.0008) [2023-12-27 11:09:41,422][20347] Updated weights for policy 0, policy_version 799666 (0.0009) [2023-12-27 11:09:41,467][20381] Updated weights for policy 1, policy_version 799143 (0.0010) [2023-12-27 11:09:41,485][20347] Updated weights for policy 0, policy_version 799676 (0.0009) [2023-12-27 11:09:41,527][20381] Updated weights for policy 1, policy_version 799153 (0.0010) [2023-12-27 11:09:42,238][20381] Updated weights for policy 1, policy_version 799163 (0.0009) [2023-12-27 11:09:42,247][20347] Updated weights for policy 0, policy_version 799686 (0.0010) [2023-12-27 11:09:42,303][20381] Updated weights for policy 1, policy_version 799173 (0.0007) [2023-12-27 11:09:42,309][20347] Updated weights for policy 0, policy_version 799696 (0.0011) [2023-12-27 11:09:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 20438.4). Total num frames: 409354240. Throughput: 0: 10324.9, 1: 10182.7. Samples: 409353700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:42,363][19170] Avg episode reward: [(0, '298.180'), (1, '816.720')] [2023-12-27 11:09:42,368][20381] Updated weights for policy 1, policy_version 799183 (0.0008) [2023-12-27 11:09:42,384][20347] Updated weights for policy 0, policy_version 799706 (0.0009) [2023-12-27 11:09:43,048][20381] Updated weights for policy 1, policy_version 799193 (0.0007) [2023-12-27 11:09:43,085][20347] Updated weights for policy 0, policy_version 799716 (0.0010) [2023-12-27 11:09:43,101][20381] Updated weights for policy 1, policy_version 799203 (0.0006) [2023-12-27 11:09:43,143][20347] Updated weights for policy 0, policy_version 799726 (0.0010) [2023-12-27 11:09:43,159][20381] Updated weights for policy 1, policy_version 799213 (0.0005) [2023-12-27 11:09:43,201][20347] Updated weights for policy 0, policy_version 799736 (0.0010) [2023-12-27 11:09:43,217][20381] Updated weights for policy 1, policy_version 799223 (0.0005) [2023-12-27 11:09:43,815][20347] Updated weights for policy 0, policy_version 799746 (0.0010) [2023-12-27 11:09:43,864][20347] Updated weights for policy 0, policy_version 799756 (0.0010) [2023-12-27 11:09:43,887][20381] Updated weights for policy 1, policy_version 799233 (0.0005) [2023-12-27 11:09:43,921][20347] Updated weights for policy 0, policy_version 799766 (0.0010) [2023-12-27 11:09:43,943][20381] Updated weights for policy 1, policy_version 799243 (0.0007) [2023-12-27 11:09:43,979][20347] Updated weights for policy 0, policy_version 799776 (0.0010) [2023-12-27 11:09:43,996][20381] Updated weights for policy 1, policy_version 799253 (0.0006) [2023-12-27 11:09:44,601][20347] Updated weights for policy 0, policy_version 799786 (0.0010) [2023-12-27 11:09:44,630][20381] Updated weights for policy 1, policy_version 799263 (0.0009) [2023-12-27 11:09:44,653][20347] Updated weights for policy 0, policy_version 799796 (0.0010) [2023-12-27 11:09:44,687][20381] Updated weights for policy 1, policy_version 799273 (0.0010) [2023-12-27 11:09:44,706][20347] Updated weights for policy 0, policy_version 799806 (0.0010) [2023-12-27 11:09:44,744][20381] Updated weights for policy 1, policy_version 799283 (0.0009) [2023-12-27 11:09:45,363][20381] Updated weights for policy 1, policy_version 799293 (0.0010) [2023-12-27 11:09:45,367][20347] Updated weights for policy 0, policy_version 799816 (0.0009) [2023-12-27 11:09:45,419][20381] Updated weights for policy 1, policy_version 799303 (0.0007) [2023-12-27 11:09:45,423][20347] Updated weights for policy 0, policy_version 799826 (0.0010) [2023-12-27 11:09:45,475][20381] Updated weights for policy 1, policy_version 799313 (0.0006) [2023-12-27 11:09:45,480][20347] Updated weights for policy 0, policy_version 799836 (0.0011) [2023-12-27 11:09:46,074][20381] Updated weights for policy 1, policy_version 799323 (0.0009) [2023-12-27 11:09:46,132][20381] Updated weights for policy 1, policy_version 799333 (0.0010) [2023-12-27 11:09:46,160][20347] Updated weights for policy 0, policy_version 799846 (0.0010) [2023-12-27 11:09:46,184][20381] Updated weights for policy 1, policy_version 799343 (0.0010) [2023-12-27 11:09:46,215][20347] Updated weights for policy 0, policy_version 799856 (0.0010) [2023-12-27 11:09:46,261][20347] Updated weights for policy 0, policy_version 799866 (0.0010) [2023-12-27 11:09:46,813][20381] Updated weights for policy 1, policy_version 799353 (0.0011) [2023-12-27 11:09:46,868][20292] KL-divergence is very high: 160.8614 [2023-12-27 11:09:46,874][20381] Updated weights for policy 1, policy_version 799363 (0.0010) [2023-12-27 11:09:46,874][20292] KL-divergence is very high: 124.0550 [2023-12-27 11:09:46,935][20381] Updated weights for policy 1, policy_version 799373 (0.0011) [2023-12-27 11:09:46,940][20347] Updated weights for policy 0, policy_version 799876 (0.0010) [2023-12-27 11:09:46,990][20381] Updated weights for policy 1, policy_version 799383 (0.0010) [2023-12-27 11:09:46,994][20347] Updated weights for policy 0, policy_version 799886 (0.0010) [2023-12-27 11:09:47,056][20347] Updated weights for policy 0, policy_version 799896 (0.0010) [2023-12-27 11:09:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 409468928. Throughput: 0: 10346.9, 1: 10227.1. Samples: 409477428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:47,363][19170] Avg episode reward: [(0, '361.980'), (1, '726.830')] [2023-12-27 11:09:47,662][20381] Updated weights for policy 1, policy_version 799393 (0.0010) [2023-12-27 11:09:47,720][20381] Updated weights for policy 1, policy_version 799403 (0.0010) [2023-12-27 11:09:47,727][20347] Updated weights for policy 0, policy_version 799906 (0.0010) [2023-12-27 11:09:47,779][20381] Updated weights for policy 1, policy_version 799413 (0.0010) [2023-12-27 11:09:47,789][20347] Updated weights for policy 0, policy_version 799916 (0.0010) [2023-12-27 11:09:47,844][20347] Updated weights for policy 0, policy_version 799926 (0.0010) [2023-12-27 11:09:47,901][20347] Updated weights for policy 0, policy_version 799936 (0.0010) [2023-12-27 11:09:48,380][20381] Updated weights for policy 1, policy_version 799423 (0.0009) [2023-12-27 11:09:48,387][20292] KL-divergence is very high: 147.3483 [2023-12-27 11:09:48,393][20292] KL-divergence is very high: 127.3669 [2023-12-27 11:09:48,439][20381] Updated weights for policy 1, policy_version 799433 (0.0008) [2023-12-27 11:09:48,484][20381] Updated weights for policy 1, policy_version 799443 (0.0008) [2023-12-27 11:09:48,576][20347] Updated weights for policy 0, policy_version 799946 (0.0005) [2023-12-27 11:09:48,638][20347] Updated weights for policy 0, policy_version 799956 (0.0005) [2023-12-27 11:09:48,697][20347] Updated weights for policy 0, policy_version 799966 (0.0005) [2023-12-27 11:09:49,208][20292] KL-divergence is very high: 107.4400 [2023-12-27 11:09:49,242][20381] Updated weights for policy 1, policy_version 799453 (0.0008) [2023-12-27 11:09:49,262][20292] KL-divergence is very high: 124.0971 [2023-12-27 11:09:49,296][20347] Updated weights for policy 0, policy_version 799976 (0.0009) [2023-12-27 11:09:49,298][20381] Updated weights for policy 1, policy_version 799463 (0.0006) [2023-12-27 11:09:49,352][20292] KL-divergence is very high: 138.5936 [2023-12-27 11:09:49,358][20381] Updated weights for policy 1, policy_version 799473 (0.0007) [2023-12-27 11:09:49,359][20347] Updated weights for policy 0, policy_version 799986 (0.0011) [2023-12-27 11:09:49,419][20347] Updated weights for policy 0, policy_version 799996 (0.0010) [2023-12-27 11:09:50,031][20381] Updated weights for policy 1, policy_version 799483 (0.0006) [2023-12-27 11:09:50,087][20381] Updated weights for policy 1, policy_version 799493 (0.0007) [2023-12-27 11:09:50,144][20381] Updated weights for policy 1, policy_version 799503 (0.0007) [2023-12-27 11:09:50,151][20347] Updated weights for policy 0, policy_version 800006 (0.0010) [2023-12-27 11:09:50,208][20347] Updated weights for policy 0, policy_version 800016 (0.0010) [2023-12-27 11:09:50,261][20347] Updated weights for policy 0, policy_version 800026 (0.0010) [2023-12-27 11:09:50,879][20381] Updated weights for policy 1, policy_version 799513 (0.0008) [2023-12-27 11:09:50,936][20347] Updated weights for policy 0, policy_version 800036 (0.0010) [2023-12-27 11:09:50,941][20381] Updated weights for policy 1, policy_version 799523 (0.0007) [2023-12-27 11:09:50,993][20347] Updated weights for policy 0, policy_version 800046 (0.0010) [2023-12-27 11:09:50,998][20381] Updated weights for policy 1, policy_version 799533 (0.0006) [2023-12-27 11:09:51,055][20347] Updated weights for policy 0, policy_version 800056 (0.0009) [2023-12-27 11:09:51,060][20381] Updated weights for policy 1, policy_version 799543 (0.0007) [2023-12-27 11:09:51,758][20292] KL-divergence is very high: 104.1465 [2023-12-27 11:09:51,781][20347] Updated weights for policy 0, policy_version 800066 (0.0009) [2023-12-27 11:09:51,801][20381] Updated weights for policy 1, policy_version 799553 (0.0006) [2023-12-27 11:09:51,843][20347] Updated weights for policy 0, policy_version 800076 (0.0011) [2023-12-27 11:09:51,853][20381] Updated weights for policy 1, policy_version 799563 (0.0008) [2023-12-27 11:09:51,902][20381] Updated weights for policy 1, policy_version 799573 (0.0007) [2023-12-27 11:09:51,908][20347] Updated weights for policy 0, policy_version 800086 (0.0010) [2023-12-27 11:09:51,966][20347] Updated weights for policy 0, policy_version 800096 (0.0009) [2023-12-27 11:09:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 409567232. Throughput: 0: 10361.7, 1: 10196.4. Samples: 409539024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:52,363][19170] Avg episode reward: [(0, '289.220'), (1, '275.290')] [2023-12-27 11:09:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000799576_204718080.pth... [2023-12-27 11:09:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000800096_204849152.pth... [2023-12-27 11:09:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000798360_204406784.pth [2023-12-27 11:09:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000798880_204537856.pth [2023-12-27 11:09:52,610][20381] Updated weights for policy 1, policy_version 799583 (0.0006) [2023-12-27 11:09:52,616][20347] Updated weights for policy 0, policy_version 800106 (0.0010) [2023-12-27 11:09:52,669][20381] Updated weights for policy 1, policy_version 799593 (0.0005) [2023-12-27 11:09:52,673][20347] Updated weights for policy 0, policy_version 800116 (0.0006) [2023-12-27 11:09:52,725][20381] Updated weights for policy 1, policy_version 799603 (0.0006) [2023-12-27 11:09:52,733][20347] Updated weights for policy 0, policy_version 800126 (0.0010) [2023-12-27 11:09:53,332][20381] Updated weights for policy 1, policy_version 799613 (0.0008) [2023-12-27 11:09:53,382][20347] Updated weights for policy 0, policy_version 800136 (0.0010) [2023-12-27 11:09:53,384][20381] Updated weights for policy 1, policy_version 799623 (0.0006) [2023-12-27 11:09:53,435][20347] Updated weights for policy 0, policy_version 800146 (0.0010) [2023-12-27 11:09:53,439][20381] Updated weights for policy 1, policy_version 799633 (0.0005) [2023-12-27 11:09:53,491][20347] Updated weights for policy 0, policy_version 800156 (0.0010) [2023-12-27 11:09:54,033][20381] Updated weights for policy 1, policy_version 799643 (0.0005) [2023-12-27 11:09:54,088][20381] Updated weights for policy 1, policy_version 799653 (0.0005) [2023-12-27 11:09:54,146][20381] Updated weights for policy 1, policy_version 799663 (0.0008) [2023-12-27 11:09:54,155][20347] Updated weights for policy 0, policy_version 800166 (0.0008) [2023-12-27 11:09:54,199][20347] Updated weights for policy 0, policy_version 800176 (0.0005) [2023-12-27 11:09:54,250][20347] Updated weights for policy 0, policy_version 800186 (0.0008) [2023-12-27 11:09:54,836][20381] Updated weights for policy 1, policy_version 799673 (0.0006) [2023-12-27 11:09:54,891][20381] Updated weights for policy 1, policy_version 799683 (0.0007) [2023-12-27 11:09:54,911][20347] Updated weights for policy 0, policy_version 800196 (0.0008) [2023-12-27 11:09:54,943][20381] Updated weights for policy 1, policy_version 799693 (0.0005) [2023-12-27 11:09:54,971][20347] Updated weights for policy 0, policy_version 800206 (0.0006) [2023-12-27 11:09:55,002][20381] Updated weights for policy 1, policy_version 799703 (0.0006) [2023-12-27 11:09:55,031][20347] Updated weights for policy 0, policy_version 800216 (0.0006) [2023-12-27 11:09:55,623][20347] Updated weights for policy 0, policy_version 800226 (0.0006) [2023-12-27 11:09:55,678][20347] Updated weights for policy 0, policy_version 800236 (0.0007) [2023-12-27 11:09:55,702][20381] Updated weights for policy 1, policy_version 799713 (0.0005) [2023-12-27 11:09:55,730][20347] Updated weights for policy 0, policy_version 800246 (0.0007) [2023-12-27 11:09:55,746][20381] Updated weights for policy 1, policy_version 799723 (0.0007) [2023-12-27 11:09:55,789][20347] Updated weights for policy 0, policy_version 800256 (0.0006) [2023-12-27 11:09:55,799][20381] Updated weights for policy 1, policy_version 799733 (0.0005) [2023-12-27 11:09:56,451][20347] Updated weights for policy 0, policy_version 800266 (0.0007) [2023-12-27 11:09:56,494][20347] Updated weights for policy 0, policy_version 800276 (0.0007) [2023-12-27 11:09:56,511][20381] Updated weights for policy 1, policy_version 799743 (0.0005) [2023-12-27 11:09:56,542][20347] Updated weights for policy 0, policy_version 800286 (0.0005) [2023-12-27 11:09:56,555][20381] Updated weights for policy 1, policy_version 799753 (0.0005) [2023-12-27 11:09:56,609][20381] Updated weights for policy 1, policy_version 799763 (0.0008) [2023-12-27 11:09:57,243][20347] Updated weights for policy 0, policy_version 800296 (0.0009) [2023-12-27 11:09:57,295][20347] Updated weights for policy 0, policy_version 800306 (0.0008) [2023-12-27 11:09:57,322][20381] Updated weights for policy 1, policy_version 799773 (0.0007) [2023-12-27 11:09:57,348][20347] Updated weights for policy 0, policy_version 800316 (0.0008) [2023-12-27 11:09:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 409665536. Throughput: 0: 10395.6, 1: 10211.0. Samples: 409662412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:09:57,363][19170] Avg episode reward: [(0, '340.820'), (1, '251.030')] [2023-12-27 11:09:57,368][20381] Updated weights for policy 1, policy_version 799783 (0.0005) [2023-12-27 11:09:57,412][20381] Updated weights for policy 1, policy_version 799793 (0.0007) [2023-12-27 11:09:57,934][20347] Updated weights for policy 0, policy_version 800326 (0.0010) [2023-12-27 11:09:57,988][20347] Updated weights for policy 0, policy_version 800336 (0.0010) [2023-12-27 11:09:58,045][20347] Updated weights for policy 0, policy_version 800346 (0.0010) [2023-12-27 11:09:58,141][20381] Updated weights for policy 1, policy_version 799803 (0.0009) [2023-12-27 11:09:58,196][20381] Updated weights for policy 1, policy_version 799813 (0.0010) [2023-12-27 11:09:58,258][20381] Updated weights for policy 1, policy_version 799823 (0.0011) [2023-12-27 11:09:58,810][20347] Updated weights for policy 0, policy_version 800356 (0.0009) [2023-12-27 11:09:58,888][20347] Updated weights for policy 0, policy_version 800366 (0.0008) [2023-12-27 11:09:58,963][20347] Updated weights for policy 0, policy_version 800376 (0.0008) [2023-12-27 11:09:59,077][20381] Updated weights for policy 1, policy_version 799833 (0.0010) [2023-12-27 11:09:59,137][20381] Updated weights for policy 1, policy_version 799843 (0.0008) [2023-12-27 11:09:59,195][20381] Updated weights for policy 1, policy_version 799853 (0.0008) [2023-12-27 11:09:59,254][20381] Updated weights for policy 1, policy_version 799863 (0.0008) [2023-12-27 11:09:59,639][20347] Updated weights for policy 0, policy_version 800386 (0.0007) [2023-12-27 11:09:59,691][20347] Updated weights for policy 0, policy_version 800396 (0.0005) [2023-12-27 11:09:59,742][20347] Updated weights for policy 0, policy_version 800406 (0.0006) [2023-12-27 11:09:59,789][20347] Updated weights for policy 0, policy_version 800416 (0.0007) [2023-12-27 11:09:59,949][20381] Updated weights for policy 1, policy_version 799873 (0.0008) [2023-12-27 11:09:59,994][20381] Updated weights for policy 1, policy_version 799883 (0.0007) [2023-12-27 11:10:00,051][20381] Updated weights for policy 1, policy_version 799893 (0.0011) [2023-12-27 11:10:00,484][20347] Updated weights for policy 0, policy_version 800426 (0.0010) [2023-12-27 11:10:00,532][20347] Updated weights for policy 0, policy_version 800436 (0.0010) [2023-12-27 11:10:00,586][20347] Updated weights for policy 0, policy_version 800446 (0.0007) [2023-12-27 11:10:00,658][20381] Updated weights for policy 1, policy_version 799903 (0.0007) [2023-12-27 11:10:00,704][20381] Updated weights for policy 1, policy_version 799913 (0.0005) [2023-12-27 11:10:00,758][20381] Updated weights for policy 1, policy_version 799923 (0.0005) [2023-12-27 11:10:01,217][20347] Updated weights for policy 0, policy_version 800456 (0.0008) [2023-12-27 11:10:01,287][20347] Updated weights for policy 0, policy_version 800466 (0.0008) [2023-12-27 11:10:01,338][20347] Updated weights for policy 0, policy_version 800476 (0.0008) [2023-12-27 11:10:01,435][20381] Updated weights for policy 1, policy_version 799933 (0.0006) [2023-12-27 11:10:01,487][20381] Updated weights for policy 1, policy_version 799943 (0.0007) [2023-12-27 11:10:01,546][20381] Updated weights for policy 1, policy_version 799953 (0.0008) [2023-12-27 11:10:02,019][20347] Updated weights for policy 0, policy_version 800486 (0.0007) [2023-12-27 11:10:02,078][20347] Updated weights for policy 0, policy_version 800496 (0.0006) [2023-12-27 11:10:02,135][20347] Updated weights for policy 0, policy_version 800506 (0.0005) [2023-12-27 11:10:02,323][20381] Updated weights for policy 1, policy_version 799963 (0.0007) [2023-12-27 11:10:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 409772032. Throughput: 0: 10379.8, 1: 10188.1. Samples: 409783712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:02,363][19170] Avg episode reward: [(0, '533.960'), (1, '355.200')] [2023-12-27 11:10:02,382][20381] Updated weights for policy 1, policy_version 799973 (0.0008) [2023-12-27 11:10:02,445][20381] Updated weights for policy 1, policy_version 799983 (0.0008) [2023-12-27 11:10:02,784][20347] Updated weights for policy 0, policy_version 800516 (0.0006) [2023-12-27 11:10:02,839][20347] Updated weights for policy 0, policy_version 800526 (0.0006) [2023-12-27 11:10:02,890][20347] Updated weights for policy 0, policy_version 800536 (0.0005) [2023-12-27 11:10:03,155][20381] Updated weights for policy 1, policy_version 799993 (0.0007) [2023-12-27 11:10:03,211][20381] Updated weights for policy 1, policy_version 800003 (0.0008) [2023-12-27 11:10:03,262][20381] Updated weights for policy 1, policy_version 800013 (0.0007) [2023-12-27 11:10:03,320][20381] Updated weights for policy 1, policy_version 800023 (0.0006) [2023-12-27 11:10:03,581][20347] Updated weights for policy 0, policy_version 800546 (0.0008) [2023-12-27 11:10:03,636][20347] Updated weights for policy 0, policy_version 800556 (0.0007) [2023-12-27 11:10:03,688][20347] Updated weights for policy 0, policy_version 800566 (0.0007) [2023-12-27 11:10:03,735][20347] Updated weights for policy 0, policy_version 800576 (0.0007) [2023-12-27 11:10:03,960][20381] Updated weights for policy 1, policy_version 800033 (0.0008) [2023-12-27 11:10:04,015][20381] Updated weights for policy 1, policy_version 800043 (0.0008) [2023-12-27 11:10:04,071][20381] Updated weights for policy 1, policy_version 800053 (0.0008) [2023-12-27 11:10:04,451][20347] Updated weights for policy 0, policy_version 800586 (0.0007) [2023-12-27 11:10:04,510][20347] Updated weights for policy 0, policy_version 800596 (0.0007) [2023-12-27 11:10:04,570][20347] Updated weights for policy 0, policy_version 800606 (0.0007) [2023-12-27 11:10:04,756][20381] Updated weights for policy 1, policy_version 800063 (0.0006) [2023-12-27 11:10:04,801][20381] Updated weights for policy 1, policy_version 800073 (0.0005) [2023-12-27 11:10:04,857][20381] Updated weights for policy 1, policy_version 800083 (0.0005) [2023-12-27 11:10:05,263][20347] Updated weights for policy 0, policy_version 800616 (0.0010) [2023-12-27 11:10:05,314][20347] Updated weights for policy 0, policy_version 800626 (0.0010) [2023-12-27 11:10:05,363][20347] Updated weights for policy 0, policy_version 800636 (0.0009) [2023-12-27 11:10:05,468][20381] Updated weights for policy 1, policy_version 800093 (0.0006) [2023-12-27 11:10:05,515][20381] Updated weights for policy 1, policy_version 800103 (0.0008) [2023-12-27 11:10:05,562][20381] Updated weights for policy 1, policy_version 800113 (0.0007) [2023-12-27 11:10:06,014][20347] Updated weights for policy 0, policy_version 800646 (0.0008) [2023-12-27 11:10:06,075][20347] Updated weights for policy 0, policy_version 800656 (0.0008) [2023-12-27 11:10:06,134][20347] Updated weights for policy 0, policy_version 800666 (0.0007) [2023-12-27 11:10:06,195][20381] Updated weights for policy 1, policy_version 800123 (0.0008) [2023-12-27 11:10:06,248][20381] Updated weights for policy 1, policy_version 800133 (0.0007) [2023-12-27 11:10:06,307][20381] Updated weights for policy 1, policy_version 800143 (0.0007) [2023-12-27 11:10:06,843][20347] Updated weights for policy 0, policy_version 800676 (0.0008) [2023-12-27 11:10:06,894][20347] Updated weights for policy 0, policy_version 800686 (0.0006) [2023-12-27 11:10:06,959][20347] Updated weights for policy 0, policy_version 800696 (0.0005) [2023-12-27 11:10:07,015][20381] Updated weights for policy 1, policy_version 800153 (0.0008) [2023-12-27 11:10:07,077][20381] Updated weights for policy 1, policy_version 800163 (0.0009) [2023-12-27 11:10:07,143][20381] Updated weights for policy 1, policy_version 800173 (0.0008) [2023-12-27 11:10:07,198][20381] Updated weights for policy 1, policy_version 800183 (0.0007) [2023-12-27 11:10:07,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 409878528. Throughput: 0: 10298.1, 1: 10260.3. Samples: 409845188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:07,364][19170] Avg episode reward: [(0, '525.490'), (1, '329.410')] [2023-12-27 11:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000800704_205004800.pth... [2023-12-27 11:10:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000800184_204873728.pth... [2023-12-27 11:10:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000799520_204701696.pth [2023-12-27 11:10:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000799000_204570624.pth [2023-12-27 11:10:07,547][20347] Updated weights for policy 0, policy_version 800706 (0.0006) [2023-12-27 11:10:07,592][20347] Updated weights for policy 0, policy_version 800716 (0.0006) [2023-12-27 11:10:07,643][20347] Updated weights for policy 0, policy_version 800726 (0.0005) [2023-12-27 11:10:07,689][20347] Updated weights for policy 0, policy_version 800736 (0.0006) [2023-12-27 11:10:07,924][20381] Updated weights for policy 1, policy_version 800193 (0.0007) [2023-12-27 11:10:07,977][20381] Updated weights for policy 1, policy_version 800203 (0.0007) [2023-12-27 11:10:08,032][20381] Updated weights for policy 1, policy_version 800213 (0.0007) [2023-12-27 11:10:08,351][20347] Updated weights for policy 0, policy_version 800746 (0.0011) [2023-12-27 11:10:08,411][20347] Updated weights for policy 0, policy_version 800756 (0.0010) [2023-12-27 11:10:08,472][20347] Updated weights for policy 0, policy_version 800766 (0.0006) [2023-12-27 11:10:08,629][20381] Updated weights for policy 1, policy_version 800223 (0.0007) [2023-12-27 11:10:08,686][20381] Updated weights for policy 1, policy_version 800233 (0.0007) [2023-12-27 11:10:08,742][20381] Updated weights for policy 1, policy_version 800243 (0.0008) [2023-12-27 11:10:09,076][20347] Updated weights for policy 0, policy_version 800776 (0.0005) [2023-12-27 11:10:09,125][20347] Updated weights for policy 0, policy_version 800786 (0.0005) [2023-12-27 11:10:09,171][20347] Updated weights for policy 0, policy_version 800796 (0.0005) [2023-12-27 11:10:09,536][20381] Updated weights for policy 1, policy_version 800253 (0.0008) [2023-12-27 11:10:09,600][20381] Updated weights for policy 1, policy_version 800263 (0.0008) [2023-12-27 11:10:09,663][20381] Updated weights for policy 1, policy_version 800273 (0.0008) [2023-12-27 11:10:09,822][20347] Updated weights for policy 0, policy_version 800806 (0.0006) [2023-12-27 11:10:09,894][20347] Updated weights for policy 0, policy_version 800816 (0.0008) [2023-12-27 11:10:09,960][20347] Updated weights for policy 0, policy_version 800826 (0.0008) [2023-12-27 11:10:10,348][20381] Updated weights for policy 1, policy_version 800283 (0.0008) [2023-12-27 11:10:10,403][20381] Updated weights for policy 1, policy_version 800293 (0.0007) [2023-12-27 11:10:10,463][20381] Updated weights for policy 1, policy_version 800303 (0.0008) [2023-12-27 11:10:10,652][20347] Updated weights for policy 0, policy_version 800836 (0.0009) [2023-12-27 11:10:10,701][20347] Updated weights for policy 0, policy_version 800846 (0.0010) [2023-12-27 11:10:10,762][20347] Updated weights for policy 0, policy_version 800856 (0.0010) [2023-12-27 11:10:11,141][20381] Updated weights for policy 1, policy_version 800313 (0.0007) [2023-12-27 11:10:11,218][20381] Updated weights for policy 1, policy_version 800323 (0.0008) [2023-12-27 11:10:11,277][20381] Updated weights for policy 1, policy_version 800333 (0.0006) [2023-12-27 11:10:11,341][20381] Updated weights for policy 1, policy_version 800343 (0.0008) [2023-12-27 11:10:11,466][20347] Updated weights for policy 0, policy_version 800866 (0.0010) [2023-12-27 11:10:11,528][20347] Updated weights for policy 0, policy_version 800876 (0.0007) [2023-12-27 11:10:11,594][20347] Updated weights for policy 0, policy_version 800886 (0.0007) [2023-12-27 11:10:11,663][20347] Updated weights for policy 0, policy_version 800896 (0.0009) [2023-12-27 11:10:12,105][20381] Updated weights for policy 1, policy_version 800353 (0.0008) [2023-12-27 11:10:12,164][20381] Updated weights for policy 1, policy_version 800363 (0.0008) [2023-12-27 11:10:12,225][20381] Updated weights for policy 1, policy_version 800373 (0.0008) [2023-12-27 11:10:12,359][20347] Updated weights for policy 0, policy_version 800906 (0.0008) [2023-12-27 11:10:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 409976832. Throughput: 0: 10331.8, 1: 10212.4. Samples: 409968628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:12,363][19170] Avg episode reward: [(0, '365.360'), (1, '517.430')] [2023-12-27 11:10:12,431][20347] Updated weights for policy 0, policy_version 800916 (0.0008) [2023-12-27 11:10:12,492][20347] Updated weights for policy 0, policy_version 800926 (0.0008) [2023-12-27 11:10:12,937][20381] Updated weights for policy 1, policy_version 800383 (0.0007) [2023-12-27 11:10:12,982][20381] Updated weights for policy 1, policy_version 800393 (0.0008) [2023-12-27 11:10:13,034][20381] Updated weights for policy 1, policy_version 800403 (0.0006) [2023-12-27 11:10:13,156][20347] Updated weights for policy 0, policy_version 800936 (0.0008) [2023-12-27 11:10:13,205][20347] Updated weights for policy 0, policy_version 800946 (0.0007) [2023-12-27 11:10:13,260][20347] Updated weights for policy 0, policy_version 800956 (0.0005) [2023-12-27 11:10:13,793][20347] Updated weights for policy 0, policy_version 800966 (0.0006) [2023-12-27 11:10:13,800][20381] Updated weights for policy 1, policy_version 800413 (0.0006) [2023-12-27 11:10:13,840][20347] Updated weights for policy 0, policy_version 800976 (0.0007) [2023-12-27 11:10:13,861][20381] Updated weights for policy 1, policy_version 800423 (0.0005) [2023-12-27 11:10:13,887][20347] Updated weights for policy 0, policy_version 800986 (0.0007) [2023-12-27 11:10:13,921][20381] Updated weights for policy 1, policy_version 800433 (0.0006) [2023-12-27 11:10:14,481][20347] Updated weights for policy 0, policy_version 800996 (0.0006) [2023-12-27 11:10:14,530][20347] Updated weights for policy 0, policy_version 801006 (0.0006) [2023-12-27 11:10:14,546][20381] Updated weights for policy 1, policy_version 800443 (0.0007) [2023-12-27 11:10:14,594][20347] Updated weights for policy 0, policy_version 801016 (0.0008) [2023-12-27 11:10:14,604][20381] Updated weights for policy 1, policy_version 800453 (0.0007) [2023-12-27 11:10:14,651][20381] Updated weights for policy 1, policy_version 800463 (0.0006) [2023-12-27 11:10:15,241][20347] Updated weights for policy 0, policy_version 801026 (0.0005) [2023-12-27 11:10:15,296][20381] Updated weights for policy 1, policy_version 800473 (0.0006) [2023-12-27 11:10:15,302][20347] Updated weights for policy 0, policy_version 801036 (0.0006) [2023-12-27 11:10:15,351][20381] Updated weights for policy 1, policy_version 800483 (0.0011) [2023-12-27 11:10:15,367][20347] Updated weights for policy 0, policy_version 801046 (0.0007) [2023-12-27 11:10:15,407][20381] Updated weights for policy 1, policy_version 800493 (0.0011) [2023-12-27 11:10:15,427][20347] Updated weights for policy 0, policy_version 801056 (0.0006) [2023-12-27 11:10:15,459][20381] Updated weights for policy 1, policy_version 800503 (0.0010) [2023-12-27 11:10:15,986][20347] Updated weights for policy 0, policy_version 801066 (0.0005) [2023-12-27 11:10:16,037][20347] Updated weights for policy 0, policy_version 801076 (0.0009) [2023-12-27 11:10:16,091][20347] Updated weights for policy 0, policy_version 801086 (0.0010) [2023-12-27 11:10:16,140][20381] Updated weights for policy 1, policy_version 800513 (0.0008) [2023-12-27 11:10:16,188][20381] Updated weights for policy 1, policy_version 800523 (0.0005) [2023-12-27 11:10:16,236][20381] Updated weights for policy 1, policy_version 800533 (0.0006) [2023-12-27 11:10:16,740][20347] Updated weights for policy 0, policy_version 801096 (0.0007) [2023-12-27 11:10:16,797][20347] Updated weights for policy 0, policy_version 801106 (0.0009) [2023-12-27 11:10:16,811][20381] Updated weights for policy 1, policy_version 800543 (0.0007) [2023-12-27 11:10:16,851][20347] Updated weights for policy 0, policy_version 801116 (0.0008) [2023-12-27 11:10:16,864][20381] Updated weights for policy 1, policy_version 800553 (0.0008) [2023-12-27 11:10:16,917][20381] Updated weights for policy 1, policy_version 800563 (0.0007) [2023-12-27 11:10:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 410083328. Throughput: 0: 10389.8, 1: 10210.9. Samples: 410094572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:17,364][19170] Avg episode reward: [(0, '517.870'), (1, '537.460')] [2023-12-27 11:10:17,535][20347] Updated weights for policy 0, policy_version 801126 (0.0006) [2023-12-27 11:10:17,591][20347] Updated weights for policy 0, policy_version 801136 (0.0005) [2023-12-27 11:10:17,644][20381] Updated weights for policy 1, policy_version 800573 (0.0007) [2023-12-27 11:10:17,647][20347] Updated weights for policy 0, policy_version 801146 (0.0005) [2023-12-27 11:10:17,694][20381] Updated weights for policy 1, policy_version 800583 (0.0007) [2023-12-27 11:10:17,740][20381] Updated weights for policy 1, policy_version 800593 (0.0009) [2023-12-27 11:10:18,219][20347] Updated weights for policy 0, policy_version 801156 (0.0005) [2023-12-27 11:10:18,269][20347] Updated weights for policy 0, policy_version 801166 (0.0005) [2023-12-27 11:10:18,328][20347] Updated weights for policy 0, policy_version 801176 (0.0007) [2023-12-27 11:10:18,496][20381] Updated weights for policy 1, policy_version 800603 (0.0009) [2023-12-27 11:10:18,556][20381] Updated weights for policy 1, policy_version 800613 (0.0008) [2023-12-27 11:10:18,606][20381] Updated weights for policy 1, policy_version 800623 (0.0007) [2023-12-27 11:10:18,936][20347] Updated weights for policy 0, policy_version 801186 (0.0008) [2023-12-27 11:10:18,981][20347] Updated weights for policy 0, policy_version 801196 (0.0007) [2023-12-27 11:10:19,032][20347] Updated weights for policy 0, policy_version 801206 (0.0006) [2023-12-27 11:10:19,079][20347] Updated weights for policy 0, policy_version 801216 (0.0009) [2023-12-27 11:10:19,243][20381] Updated weights for policy 1, policy_version 800633 (0.0008) [2023-12-27 11:10:19,300][20381] Updated weights for policy 1, policy_version 800643 (0.0008) [2023-12-27 11:10:19,372][20381] Updated weights for policy 1, policy_version 800653 (0.0008) [2023-12-27 11:10:19,434][20381] Updated weights for policy 1, policy_version 800663 (0.0008) [2023-12-27 11:10:19,810][20347] Updated weights for policy 0, policy_version 801226 (0.0007) [2023-12-27 11:10:19,883][20347] Updated weights for policy 0, policy_version 801236 (0.0007) [2023-12-27 11:10:19,946][20347] Updated weights for policy 0, policy_version 801246 (0.0008) [2023-12-27 11:10:20,197][20381] Updated weights for policy 1, policy_version 800673 (0.0009) [2023-12-27 11:10:20,260][20381] Updated weights for policy 1, policy_version 800683 (0.0009) [2023-12-27 11:10:20,320][20381] Updated weights for policy 1, policy_version 800693 (0.0008) [2023-12-27 11:10:20,655][20347] Updated weights for policy 0, policy_version 801256 (0.0009) [2023-12-27 11:10:20,712][20347] Updated weights for policy 0, policy_version 801266 (0.0010) [2023-12-27 11:10:20,762][20347] Updated weights for policy 0, policy_version 801276 (0.0010) [2023-12-27 11:10:21,014][20381] Updated weights for policy 1, policy_version 800703 (0.0008) [2023-12-27 11:10:21,081][20381] Updated weights for policy 1, policy_version 800713 (0.0007) [2023-12-27 11:10:21,152][20381] Updated weights for policy 1, policy_version 800723 (0.0007) [2023-12-27 11:10:21,522][20347] Updated weights for policy 0, policy_version 801286 (0.0009) [2023-12-27 11:10:21,567][20347] Updated weights for policy 0, policy_version 801296 (0.0008) [2023-12-27 11:10:21,624][20347] Updated weights for policy 0, policy_version 801306 (0.0008) [2023-12-27 11:10:21,864][20381] Updated weights for policy 1, policy_version 800733 (0.0008) [2023-12-27 11:10:21,924][20381] Updated weights for policy 1, policy_version 800743 (0.0011) [2023-12-27 11:10:21,984][20381] Updated weights for policy 1, policy_version 800753 (0.0010) [2023-12-27 11:10:22,350][20347] Updated weights for policy 0, policy_version 801316 (0.0007) [2023-12-27 11:10:22,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 410181632. Throughput: 0: 10396.1, 1: 10181.7. Samples: 410155948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:22,363][19170] Avg episode reward: [(0, '453.870'), (1, '688.140')] [2023-12-27 11:10:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000800760_205021184.pth... [2023-12-27 11:10:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000799576_204718080.pth [2023-12-27 11:10:22,419][20347] Updated weights for policy 0, policy_version 801326 (0.0007) [2023-12-27 11:10:22,480][20347] Updated weights for policy 0, policy_version 801336 (0.0008) [2023-12-27 11:10:22,531][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000801344_205168640.pth... [2023-12-27 11:10:22,535][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000800096_204849152.pth [2023-12-27 11:10:22,712][20381] Updated weights for policy 1, policy_version 800763 (0.0011) [2023-12-27 11:10:22,772][20381] Updated weights for policy 1, policy_version 800773 (0.0011) [2023-12-27 11:10:22,830][20381] Updated weights for policy 1, policy_version 800783 (0.0010) [2023-12-27 11:10:23,175][20347] Updated weights for policy 0, policy_version 801346 (0.0006) [2023-12-27 11:10:23,221][20347] Updated weights for policy 0, policy_version 801356 (0.0007) [2023-12-27 11:10:23,266][20347] Updated weights for policy 0, policy_version 801366 (0.0007) [2023-12-27 11:10:23,309][20347] Updated weights for policy 0, policy_version 801376 (0.0007) [2023-12-27 11:10:23,419][20381] Updated weights for policy 1, policy_version 800793 (0.0008) [2023-12-27 11:10:23,461][20381] Updated weights for policy 1, policy_version 800803 (0.0008) [2023-12-27 11:10:23,509][20381] Updated weights for policy 1, policy_version 800813 (0.0007) [2023-12-27 11:10:23,559][20381] Updated weights for policy 1, policy_version 800823 (0.0005) [2023-12-27 11:10:24,061][20347] Updated weights for policy 0, policy_version 801386 (0.0007) [2023-12-27 11:10:24,107][20347] Updated weights for policy 0, policy_version 801396 (0.0010) [2023-12-27 11:10:24,155][20381] Updated weights for policy 1, policy_version 800833 (0.0009) [2023-12-27 11:10:24,163][20347] Updated weights for policy 0, policy_version 801406 (0.0010) [2023-12-27 11:10:24,208][20381] Updated weights for policy 1, policy_version 800843 (0.0008) [2023-12-27 11:10:24,260][20381] Updated weights for policy 1, policy_version 800853 (0.0005) [2023-12-27 11:10:24,825][20347] Updated weights for policy 0, policy_version 801416 (0.0009) [2023-12-27 11:10:24,849][20381] Updated weights for policy 1, policy_version 800863 (0.0006) [2023-12-27 11:10:24,863][20292] KL-divergence is very high: 169.5875 [2023-12-27 11:10:24,875][20292] KL-divergence is very high: 188.0136 [2023-12-27 11:10:24,876][20347] Updated weights for policy 0, policy_version 801426 (0.0005) [2023-12-27 11:10:24,886][20292] KL-divergence is very high: 154.1674 [2023-12-27 11:10:24,906][20381] Updated weights for policy 1, policy_version 800873 (0.0008) [2023-12-27 11:10:24,934][20347] Updated weights for policy 0, policy_version 801436 (0.0006) [2023-12-27 11:10:24,963][20381] Updated weights for policy 1, policy_version 800883 (0.0007) [2023-12-27 11:10:25,492][20347] Updated weights for policy 0, policy_version 801446 (0.0008) [2023-12-27 11:10:25,538][20347] Updated weights for policy 0, policy_version 801456 (0.0010) [2023-12-27 11:10:25,591][20347] Updated weights for policy 0, policy_version 801466 (0.0007) [2023-12-27 11:10:25,646][20292] KL-divergence is very high: 143.5957 [2023-12-27 11:10:25,659][20381] Updated weights for policy 1, policy_version 800893 (0.0008) [2023-12-27 11:10:25,693][20292] KL-divergence is very high: 159.1579 [2023-12-27 11:10:25,716][20381] Updated weights for policy 1, policy_version 800903 (0.0007) [2023-12-27 11:10:25,737][20292] KL-divergence is very high: 112.2368 [2023-12-27 11:10:25,763][20381] Updated weights for policy 1, policy_version 800913 (0.0007) [2023-12-27 11:10:25,772][20292] KL-divergence is very high: 109.0641 [2023-12-27 11:10:26,227][20347] Updated weights for policy 0, policy_version 801476 (0.0008) [2023-12-27 11:10:26,286][20347] Updated weights for policy 0, policy_version 801486 (0.0011) [2023-12-27 11:10:26,338][20347] Updated weights for policy 0, policy_version 801496 (0.0010) [2023-12-27 11:10:26,384][20381] Updated weights for policy 1, policy_version 800923 (0.0007) [2023-12-27 11:10:26,442][20381] Updated weights for policy 1, policy_version 800933 (0.0007) [2023-12-27 11:10:26,501][20381] Updated weights for policy 1, policy_version 800943 (0.0007) [2023-12-27 11:10:26,942][20347] Updated weights for policy 0, policy_version 801506 (0.0010) [2023-12-27 11:10:27,002][20347] Updated weights for policy 0, policy_version 801516 (0.0010) [2023-12-27 11:10:27,054][20347] Updated weights for policy 0, policy_version 801526 (0.0010) [2023-12-27 11:10:27,103][20347] Updated weights for policy 0, policy_version 801536 (0.0010) [2023-12-27 11:10:27,190][20381] Updated weights for policy 1, policy_version 800953 (0.0007) [2023-12-27 11:10:27,242][20381] Updated weights for policy 1, policy_version 800963 (0.0005) [2023-12-27 11:10:27,289][20381] Updated weights for policy 1, policy_version 800973 (0.0005) [2023-12-27 11:10:27,350][20381] Updated weights for policy 1, policy_version 800983 (0.0007) [2023-12-27 11:10:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 410296320. Throughput: 0: 10390.2, 1: 10201.3. Samples: 410280320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:27,363][19170] Avg episode reward: [(0, '331.460'), (1, '650.700')] [2023-12-27 11:10:27,772][20347] Updated weights for policy 0, policy_version 801546 (0.0007) [2023-12-27 11:10:27,815][20347] Updated weights for policy 0, policy_version 801556 (0.0008) [2023-12-27 11:10:27,860][20347] Updated weights for policy 0, policy_version 801566 (0.0007) [2023-12-27 11:10:27,909][20381] Updated weights for policy 1, policy_version 800993 (0.0008) [2023-12-27 11:10:27,969][20381] Updated weights for policy 1, policy_version 801003 (0.0008) [2023-12-27 11:10:28,017][20381] Updated weights for policy 1, policy_version 801013 (0.0008) [2023-12-27 11:10:28,517][20347] Updated weights for policy 0, policy_version 801576 (0.0007) [2023-12-27 11:10:28,573][20347] Updated weights for policy 0, policy_version 801586 (0.0006) [2023-12-27 11:10:28,626][20347] Updated weights for policy 0, policy_version 801596 (0.0006) [2023-12-27 11:10:28,724][20381] Updated weights for policy 1, policy_version 801023 (0.0008) [2023-12-27 11:10:28,773][20381] Updated weights for policy 1, policy_version 801033 (0.0008) [2023-12-27 11:10:28,823][20381] Updated weights for policy 1, policy_version 801043 (0.0007) [2023-12-27 11:10:29,252][20347] Updated weights for policy 0, policy_version 801606 (0.0008) [2023-12-27 11:10:29,315][20347] Updated weights for policy 0, policy_version 801616 (0.0010) [2023-12-27 11:10:29,378][20347] Updated weights for policy 0, policy_version 801626 (0.0008) [2023-12-27 11:10:29,530][20381] Updated weights for policy 1, policy_version 801053 (0.0008) [2023-12-27 11:10:29,591][20381] Updated weights for policy 1, policy_version 801063 (0.0007) [2023-12-27 11:10:29,653][20381] Updated weights for policy 1, policy_version 801073 (0.0008) [2023-12-27 11:10:30,078][20347] Updated weights for policy 0, policy_version 801636 (0.0008) [2023-12-27 11:10:30,127][20347] Updated weights for policy 0, policy_version 801646 (0.0010) [2023-12-27 11:10:30,185][20347] Updated weights for policy 0, policy_version 801656 (0.0010) [2023-12-27 11:10:30,304][20381] Updated weights for policy 1, policy_version 801083 (0.0007) [2023-12-27 11:10:30,362][20381] Updated weights for policy 1, policy_version 801093 (0.0005) [2023-12-27 11:10:30,416][20381] Updated weights for policy 1, policy_version 801103 (0.0006) [2023-12-27 11:10:30,828][20347] Updated weights for policy 0, policy_version 801666 (0.0011) [2023-12-27 11:10:30,883][20347] Updated weights for policy 0, policy_version 801676 (0.0010) [2023-12-27 11:10:30,934][20347] Updated weights for policy 0, policy_version 801686 (0.0006) [2023-12-27 11:10:30,984][20347] Updated weights for policy 0, policy_version 801696 (0.0007) [2023-12-27 11:10:31,038][20381] Updated weights for policy 1, policy_version 801113 (0.0010) [2023-12-27 11:10:31,097][20381] Updated weights for policy 1, policy_version 801123 (0.0011) [2023-12-27 11:10:31,159][20381] Updated weights for policy 1, policy_version 801133 (0.0009) [2023-12-27 11:10:31,226][20381] Updated weights for policy 1, policy_version 801143 (0.0006) [2023-12-27 11:10:31,736][20347] Updated weights for policy 0, policy_version 801706 (0.0008) [2023-12-27 11:10:31,793][20347] Updated weights for policy 0, policy_version 801716 (0.0006) [2023-12-27 11:10:31,845][20347] Updated weights for policy 0, policy_version 801726 (0.0008) [2023-12-27 11:10:31,901][20381] Updated weights for policy 1, policy_version 801153 (0.0006) [2023-12-27 11:10:31,947][20292] KL-divergence is very high: 123.8815 [2023-12-27 11:10:31,961][20381] Updated weights for policy 1, policy_version 801163 (0.0006) [2023-12-27 11:10:31,997][20292] KL-divergence is very high: 209.8391 [2023-12-27 11:10:32,021][20381] Updated weights for policy 1, policy_version 801173 (0.0008) [2023-12-27 11:10:32,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 410394624. Throughput: 0: 10433.3, 1: 10196.6. Samples: 410405776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:32,363][19170] Avg episode reward: [(0, '364.580'), (1, '772.540')] [2023-12-27 11:10:32,543][20347] Updated weights for policy 0, policy_version 801736 (0.0008) [2023-12-27 11:10:32,589][20347] Updated weights for policy 0, policy_version 801746 (0.0008) [2023-12-27 11:10:32,634][20347] Updated weights for policy 0, policy_version 801756 (0.0007) [2023-12-27 11:10:32,640][20381] Updated weights for policy 1, policy_version 801183 (0.0007) [2023-12-27 11:10:32,686][20381] Updated weights for policy 1, policy_version 801193 (0.0006) [2023-12-27 11:10:32,739][20381] Updated weights for policy 1, policy_version 801203 (0.0005) [2023-12-27 11:10:33,293][20381] Updated weights for policy 1, policy_version 801213 (0.0006) [2023-12-27 11:10:33,340][20347] Updated weights for policy 0, policy_version 801766 (0.0007) [2023-12-27 11:10:33,349][20381] Updated weights for policy 1, policy_version 801223 (0.0007) [2023-12-27 11:10:33,389][20347] Updated weights for policy 0, policy_version 801776 (0.0005) [2023-12-27 11:10:33,405][20381] Updated weights for policy 1, policy_version 801233 (0.0007) [2023-12-27 11:10:33,437][20347] Updated weights for policy 0, policy_version 801786 (0.0006) [2023-12-27 11:10:34,064][20347] Updated weights for policy 0, policy_version 801796 (0.0007) [2023-12-27 11:10:34,110][20381] Updated weights for policy 1, policy_version 801243 (0.0006) [2023-12-27 11:10:34,116][20347] Updated weights for policy 0, policy_version 801806 (0.0010) [2023-12-27 11:10:34,176][20381] Updated weights for policy 1, policy_version 801253 (0.0008) [2023-12-27 11:10:34,178][20347] Updated weights for policy 0, policy_version 801816 (0.0009) [2023-12-27 11:10:34,232][20381] Updated weights for policy 1, policy_version 801263 (0.0006) [2023-12-27 11:10:34,862][20347] Updated weights for policy 0, policy_version 801826 (0.0007) [2023-12-27 11:10:34,924][20347] Updated weights for policy 0, policy_version 801836 (0.0007) [2023-12-27 11:10:34,926][20381] Updated weights for policy 1, policy_version 801273 (0.0007) [2023-12-27 11:10:34,979][20381] Updated weights for policy 1, policy_version 801283 (0.0006) [2023-12-27 11:10:34,981][20347] Updated weights for policy 0, policy_version 801846 (0.0007) [2023-12-27 11:10:35,034][20381] Updated weights for policy 1, policy_version 801293 (0.0007) [2023-12-27 11:10:35,036][20347] Updated weights for policy 0, policy_version 801856 (0.0005) [2023-12-27 11:10:35,086][20381] Updated weights for policy 1, policy_version 801303 (0.0008) [2023-12-27 11:10:35,656][20347] Updated weights for policy 0, policy_version 801866 (0.0006) [2023-12-27 11:10:35,708][20347] Updated weights for policy 0, policy_version 801876 (0.0006) [2023-12-27 11:10:35,763][20381] Updated weights for policy 1, policy_version 801313 (0.0007) [2023-12-27 11:10:35,766][20347] Updated weights for policy 0, policy_version 801886 (0.0006) [2023-12-27 11:10:35,819][20381] Updated weights for policy 1, policy_version 801323 (0.0007) [2023-12-27 11:10:35,871][20381] Updated weights for policy 1, policy_version 801333 (0.0007) [2023-12-27 11:10:36,442][20347] Updated weights for policy 0, policy_version 801896 (0.0009) [2023-12-27 11:10:36,500][20347] Updated weights for policy 0, policy_version 801906 (0.0010) [2023-12-27 11:10:36,558][20347] Updated weights for policy 0, policy_version 801916 (0.0008) [2023-12-27 11:10:36,573][20381] Updated weights for policy 1, policy_version 801343 (0.0007) [2023-12-27 11:10:36,637][20381] Updated weights for policy 1, policy_version 801353 (0.0008) [2023-12-27 11:10:36,689][20381] Updated weights for policy 1, policy_version 801363 (0.0007) [2023-12-27 11:10:37,244][20347] Updated weights for policy 0, policy_version 801926 (0.0010) [2023-12-27 11:10:37,291][20347] Updated weights for policy 0, policy_version 801936 (0.0010) [2023-12-27 11:10:37,344][20347] Updated weights for policy 0, policy_version 801946 (0.0010) [2023-12-27 11:10:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 410492928. Throughput: 0: 10419.9, 1: 10231.5. Samples: 410468332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:37,363][19170] Avg episode reward: [(0, '371.330'), (1, '674.250')] [2023-12-27 11:10:37,378][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000801952_205324288.pth... [2023-12-27 11:10:37,380][20381] Updated weights for policy 1, policy_version 801373 (0.0007) [2023-12-27 11:10:37,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000800704_205004800.pth [2023-12-27 11:10:37,439][20381] Updated weights for policy 1, policy_version 801383 (0.0006) [2023-12-27 11:10:37,497][20381] Updated weights for policy 1, policy_version 801393 (0.0006) [2023-12-27 11:10:37,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000801400_205185024.pth... [2023-12-27 11:10:37,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000800184_204873728.pth [2023-12-27 11:10:38,040][20347] Updated weights for policy 0, policy_version 801956 (0.0010) [2023-12-27 11:10:38,041][20381] Updated weights for policy 1, policy_version 801403 (0.0008) [2023-12-27 11:10:38,096][20347] Updated weights for policy 0, policy_version 801966 (0.0009) [2023-12-27 11:10:38,098][20381] Updated weights for policy 1, policy_version 801413 (0.0007) [2023-12-27 11:10:38,148][20347] Updated weights for policy 0, policy_version 801976 (0.0008) [2023-12-27 11:10:38,155][20381] Updated weights for policy 1, policy_version 801423 (0.0005) [2023-12-27 11:10:38,833][20381] Updated weights for policy 1, policy_version 801433 (0.0006) [2023-12-27 11:10:38,877][20347] Updated weights for policy 0, policy_version 801986 (0.0007) [2023-12-27 11:10:38,890][20381] Updated weights for policy 1, policy_version 801443 (0.0006) [2023-12-27 11:10:38,939][20347] Updated weights for policy 0, policy_version 801996 (0.0006) [2023-12-27 11:10:38,941][20381] Updated weights for policy 1, policy_version 801453 (0.0006) [2023-12-27 11:10:38,991][20381] Updated weights for policy 1, policy_version 801463 (0.0005) [2023-12-27 11:10:39,002][20347] Updated weights for policy 0, policy_version 802006 (0.0007) [2023-12-27 11:10:39,057][20347] Updated weights for policy 0, policy_version 802016 (0.0007) [2023-12-27 11:10:39,617][20381] Updated weights for policy 1, policy_version 801473 (0.0005) [2023-12-27 11:10:39,666][20381] Updated weights for policy 1, policy_version 801483 (0.0005) [2023-12-27 11:10:39,717][20381] Updated weights for policy 1, policy_version 801493 (0.0007) [2023-12-27 11:10:39,787][20347] Updated weights for policy 0, policy_version 802026 (0.0008) [2023-12-27 11:10:39,851][20347] Updated weights for policy 0, policy_version 802036 (0.0008) [2023-12-27 11:10:39,911][20347] Updated weights for policy 0, policy_version 802046 (0.0006) [2023-12-27 11:10:40,466][20381] Updated weights for policy 1, policy_version 801503 (0.0007) [2023-12-27 11:10:40,525][20381] Updated weights for policy 1, policy_version 801513 (0.0007) [2023-12-27 11:10:40,535][20347] Updated weights for policy 0, policy_version 802056 (0.0006) [2023-12-27 11:10:40,581][20381] Updated weights for policy 1, policy_version 801523 (0.0006) [2023-12-27 11:10:40,587][20347] Updated weights for policy 0, policy_version 802066 (0.0005) [2023-12-27 11:10:40,638][20347] Updated weights for policy 0, policy_version 802076 (0.0007) [2023-12-27 11:10:41,330][20381] Updated weights for policy 1, policy_version 801533 (0.0006) [2023-12-27 11:10:41,371][20347] Updated weights for policy 0, policy_version 802086 (0.0008) [2023-12-27 11:10:41,396][20381] Updated weights for policy 1, policy_version 801543 (0.0008) [2023-12-27 11:10:41,434][20347] Updated weights for policy 0, policy_version 802096 (0.0008) [2023-12-27 11:10:41,455][20381] Updated weights for policy 1, policy_version 801553 (0.0007) [2023-12-27 11:10:41,488][20347] Updated weights for policy 0, policy_version 802106 (0.0006) [2023-12-27 11:10:42,095][20381] Updated weights for policy 1, policy_version 801563 (0.0007) [2023-12-27 11:10:42,152][20381] Updated weights for policy 1, policy_version 801573 (0.0008) [2023-12-27 11:10:42,208][20381] Updated weights for policy 1, policy_version 801583 (0.0008) [2023-12-27 11:10:42,256][20347] Updated weights for policy 0, policy_version 802116 (0.0007) [2023-12-27 11:10:42,316][20347] Updated weights for policy 0, policy_version 802126 (0.0009) [2023-12-27 11:10:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 410599424. Throughput: 0: 10390.6, 1: 10244.6. Samples: 410590996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:42,363][19170] Avg episode reward: [(0, '320.520'), (1, '633.620')] [2023-12-27 11:10:42,379][20347] Updated weights for policy 0, policy_version 802136 (0.0008) [2023-12-27 11:10:42,933][20381] Updated weights for policy 1, policy_version 801593 (0.0007) [2023-12-27 11:10:42,996][20381] Updated weights for policy 1, policy_version 801603 (0.0009) [2023-12-27 11:10:43,058][20381] Updated weights for policy 1, policy_version 801613 (0.0007) [2023-12-27 11:10:43,085][20347] Updated weights for policy 0, policy_version 802146 (0.0007) [2023-12-27 11:10:43,117][20381] Updated weights for policy 1, policy_version 801623 (0.0006) [2023-12-27 11:10:43,139][20347] Updated weights for policy 0, policy_version 802156 (0.0007) [2023-12-27 11:10:43,184][20347] Updated weights for policy 0, policy_version 802166 (0.0008) [2023-12-27 11:10:43,230][20347] Updated weights for policy 0, policy_version 802176 (0.0008) [2023-12-27 11:10:43,776][20381] Updated weights for policy 1, policy_version 801633 (0.0007) [2023-12-27 11:10:43,835][20381] Updated weights for policy 1, policy_version 801643 (0.0007) [2023-12-27 11:10:43,896][20381] Updated weights for policy 1, policy_version 801653 (0.0008) [2023-12-27 11:10:43,930][20347] Updated weights for policy 0, policy_version 802186 (0.0005) [2023-12-27 11:10:43,981][20347] Updated weights for policy 0, policy_version 802196 (0.0006) [2023-12-27 11:10:44,026][20347] Updated weights for policy 0, policy_version 802206 (0.0007) [2023-12-27 11:10:44,496][20381] Updated weights for policy 1, policy_version 801663 (0.0006) [2023-12-27 11:10:44,552][20381] Updated weights for policy 1, policy_version 801673 (0.0010) [2023-12-27 11:10:44,600][20381] Updated weights for policy 1, policy_version 801683 (0.0010) [2023-12-27 11:10:44,773][20347] Updated weights for policy 0, policy_version 802216 (0.0007) [2023-12-27 11:10:44,836][20347] Updated weights for policy 0, policy_version 802226 (0.0007) [2023-12-27 11:10:44,889][20347] Updated weights for policy 0, policy_version 802236 (0.0008) [2023-12-27 11:10:45,260][20381] Updated weights for policy 1, policy_version 801693 (0.0009) [2023-12-27 11:10:45,317][20381] Updated weights for policy 1, policy_version 801703 (0.0007) [2023-12-27 11:10:45,376][20381] Updated weights for policy 1, policy_version 801713 (0.0008) [2023-12-27 11:10:45,571][20347] Updated weights for policy 0, policy_version 802246 (0.0006) [2023-12-27 11:10:45,620][20347] Updated weights for policy 0, policy_version 802256 (0.0005) [2023-12-27 11:10:45,683][20347] Updated weights for policy 0, policy_version 802266 (0.0006) [2023-12-27 11:10:46,111][20381] Updated weights for policy 1, policy_version 801723 (0.0008) [2023-12-27 11:10:46,173][20381] Updated weights for policy 1, policy_version 801733 (0.0008) [2023-12-27 11:10:46,232][20381] Updated weights for policy 1, policy_version 801743 (0.0008) [2023-12-27 11:10:46,278][20347] Updated weights for policy 0, policy_version 802276 (0.0007) [2023-12-27 11:10:46,330][20347] Updated weights for policy 0, policy_version 802286 (0.0005) [2023-12-27 11:10:46,394][20347] Updated weights for policy 0, policy_version 802296 (0.0005) [2023-12-27 11:10:46,912][20347] Updated weights for policy 0, policy_version 802306 (0.0005) [2023-12-27 11:10:46,952][20381] Updated weights for policy 1, policy_version 801753 (0.0007) [2023-12-27 11:10:46,970][20347] Updated weights for policy 0, policy_version 802316 (0.0009) [2023-12-27 11:10:47,010][20381] Updated weights for policy 1, policy_version 801763 (0.0005) [2023-12-27 11:10:47,029][20347] Updated weights for policy 0, policy_version 802326 (0.0008) [2023-12-27 11:10:47,067][20381] Updated weights for policy 1, policy_version 801773 (0.0005) [2023-12-27 11:10:47,088][20347] Updated weights for policy 0, policy_version 802336 (0.0005) [2023-12-27 11:10:47,125][20381] Updated weights for policy 1, policy_version 801783 (0.0005) [2023-12-27 11:10:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 410705920. Throughput: 0: 10396.6, 1: 10276.3. Samples: 410713992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:47,363][19170] Avg episode reward: [(0, '264.960'), (1, '577.970')] [2023-12-27 11:10:47,639][20381] Updated weights for policy 1, policy_version 801793 (0.0007) [2023-12-27 11:10:47,697][20347] Updated weights for policy 0, policy_version 802346 (0.0006) [2023-12-27 11:10:47,698][20381] Updated weights for policy 1, policy_version 801803 (0.0006) [2023-12-27 11:10:47,755][20381] Updated weights for policy 1, policy_version 801813 (0.0008) [2023-12-27 11:10:47,757][20347] Updated weights for policy 0, policy_version 802356 (0.0005) [2023-12-27 11:10:47,815][20347] Updated weights for policy 0, policy_version 802366 (0.0006) [2023-12-27 11:10:48,371][20381] Updated weights for policy 1, policy_version 801823 (0.0008) [2023-12-27 11:10:48,430][20381] Updated weights for policy 1, policy_version 801833 (0.0006) [2023-12-27 11:10:48,440][20347] Updated weights for policy 0, policy_version 802376 (0.0006) [2023-12-27 11:10:48,486][20381] Updated weights for policy 1, policy_version 801843 (0.0007) [2023-12-27 11:10:48,499][20347] Updated weights for policy 0, policy_version 802386 (0.0008) [2023-12-27 11:10:48,548][20347] Updated weights for policy 0, policy_version 802396 (0.0010) [2023-12-27 11:10:49,153][20381] Updated weights for policy 1, policy_version 801853 (0.0006) [2023-12-27 11:10:49,207][20381] Updated weights for policy 1, policy_version 801863 (0.0007) [2023-12-27 11:10:49,262][20347] Updated weights for policy 0, policy_version 802406 (0.0007) [2023-12-27 11:10:49,268][20381] Updated weights for policy 1, policy_version 801873 (0.0008) [2023-12-27 11:10:49,324][20347] Updated weights for policy 0, policy_version 802416 (0.0008) [2023-12-27 11:10:49,387][20347] Updated weights for policy 0, policy_version 802426 (0.0008) [2023-12-27 11:10:49,901][20381] Updated weights for policy 1, policy_version 801883 (0.0008) [2023-12-27 11:10:49,967][20381] Updated weights for policy 1, policy_version 801893 (0.0007) [2023-12-27 11:10:50,026][20381] Updated weights for policy 1, policy_version 801903 (0.0007) [2023-12-27 11:10:50,106][20347] Updated weights for policy 0, policy_version 802436 (0.0009) [2023-12-27 11:10:50,169][20347] Updated weights for policy 0, policy_version 802446 (0.0010) [2023-12-27 11:10:50,219][20347] Updated weights for policy 0, policy_version 802456 (0.0008) [2023-12-27 11:10:50,636][20381] Updated weights for policy 1, policy_version 801913 (0.0006) [2023-12-27 11:10:50,687][20381] Updated weights for policy 1, policy_version 801923 (0.0008) [2023-12-27 11:10:50,733][20381] Updated weights for policy 1, policy_version 801933 (0.0008) [2023-12-27 11:10:50,781][20381] Updated weights for policy 1, policy_version 801943 (0.0005) [2023-12-27 11:10:50,932][20347] Updated weights for policy 0, policy_version 802466 (0.0010) [2023-12-27 11:10:50,993][20347] Updated weights for policy 0, policy_version 802476 (0.0008) [2023-12-27 11:10:51,058][20347] Updated weights for policy 0, policy_version 802486 (0.0008) [2023-12-27 11:10:51,116][20347] Updated weights for policy 0, policy_version 802496 (0.0008) [2023-12-27 11:10:51,534][20381] Updated weights for policy 1, policy_version 801953 (0.0008) [2023-12-27 11:10:51,592][20381] Updated weights for policy 1, policy_version 801963 (0.0007) [2023-12-27 11:10:51,660][20381] Updated weights for policy 1, policy_version 801973 (0.0008) [2023-12-27 11:10:51,799][20347] Updated weights for policy 0, policy_version 802506 (0.0007) [2023-12-27 11:10:51,857][20347] Updated weights for policy 0, policy_version 802516 (0.0007) [2023-12-27 11:10:51,919][20347] Updated weights for policy 0, policy_version 802526 (0.0005) [2023-12-27 11:10:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 410804224. Throughput: 0: 10411.6, 1: 10327.3. Samples: 410778436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:52,363][19170] Avg episode reward: [(0, '369.710'), (1, '585.830')] [2023-12-27 11:10:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000802528_205471744.pth... [2023-12-27 11:10:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000801344_205168640.pth [2023-12-27 11:10:52,403][20381] Updated weights for policy 1, policy_version 801983 (0.0008) [2023-12-27 11:10:52,461][20381] Updated weights for policy 1, policy_version 801993 (0.0008) [2023-12-27 11:10:52,520][20381] Updated weights for policy 1, policy_version 802003 (0.0008) [2023-12-27 11:10:52,545][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000802008_205340672.pth... [2023-12-27 11:10:52,550][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000800760_205021184.pth [2023-12-27 11:10:52,574][20347] Updated weights for policy 0, policy_version 802536 (0.0007) [2023-12-27 11:10:52,627][20347] Updated weights for policy 0, policy_version 802546 (0.0007) [2023-12-27 11:10:52,683][20347] Updated weights for policy 0, policy_version 802556 (0.0008) [2023-12-27 11:10:53,145][20381] Updated weights for policy 1, policy_version 802013 (0.0007) [2023-12-27 11:10:53,188][20381] Updated weights for policy 1, policy_version 802023 (0.0007) [2023-12-27 11:10:53,239][20381] Updated weights for policy 1, policy_version 802033 (0.0005) [2023-12-27 11:10:53,405][20347] Updated weights for policy 0, policy_version 802566 (0.0007) [2023-12-27 11:10:53,454][20347] Updated weights for policy 0, policy_version 802576 (0.0007) [2023-12-27 11:10:53,505][20347] Updated weights for policy 0, policy_version 802586 (0.0007) [2023-12-27 11:10:53,876][20381] Updated weights for policy 1, policy_version 802043 (0.0005) [2023-12-27 11:10:53,935][20381] Updated weights for policy 1, policy_version 802053 (0.0005) [2023-12-27 11:10:53,988][20381] Updated weights for policy 1, policy_version 802063 (0.0005) [2023-12-27 11:10:54,188][20347] Updated weights for policy 0, policy_version 802596 (0.0007) [2023-12-27 11:10:54,248][20347] Updated weights for policy 0, policy_version 802606 (0.0007) [2023-12-27 11:10:54,300][20347] Updated weights for policy 0, policy_version 802616 (0.0007) [2023-12-27 11:10:54,524][20381] Updated weights for policy 1, policy_version 802073 (0.0006) [2023-12-27 11:10:54,582][20381] Updated weights for policy 1, policy_version 802083 (0.0008) [2023-12-27 11:10:54,643][20381] Updated weights for policy 1, policy_version 802093 (0.0007) [2023-12-27 11:10:54,699][20381] Updated weights for policy 1, policy_version 802103 (0.0009) [2023-12-27 11:10:54,906][20347] Updated weights for policy 0, policy_version 802626 (0.0007) [2023-12-27 11:10:54,972][20347] Updated weights for policy 0, policy_version 802636 (0.0007) [2023-12-27 11:10:55,028][20347] Updated weights for policy 0, policy_version 802646 (0.0008) [2023-12-27 11:10:55,079][20347] Updated weights for policy 0, policy_version 802656 (0.0010) [2023-12-27 11:10:55,331][20381] Updated weights for policy 1, policy_version 802113 (0.0006) [2023-12-27 11:10:55,383][20381] Updated weights for policy 1, policy_version 802123 (0.0008) [2023-12-27 11:10:55,437][20381] Updated weights for policy 1, policy_version 802133 (0.0007) [2023-12-27 11:10:55,726][20347] Updated weights for policy 0, policy_version 802666 (0.0010) [2023-12-27 11:10:55,770][20347] Updated weights for policy 0, policy_version 802676 (0.0005) [2023-12-27 11:10:55,826][20347] Updated weights for policy 0, policy_version 802686 (0.0007) [2023-12-27 11:10:56,050][20381] Updated weights for policy 1, policy_version 802143 (0.0006) [2023-12-27 11:10:56,109][20381] Updated weights for policy 1, policy_version 802153 (0.0005) [2023-12-27 11:10:56,176][20381] Updated weights for policy 1, policy_version 802163 (0.0005) [2023-12-27 11:10:56,472][20347] Updated weights for policy 0, policy_version 802696 (0.0010) [2023-12-27 11:10:56,519][20347] Updated weights for policy 0, policy_version 802706 (0.0010) [2023-12-27 11:10:56,573][20347] Updated weights for policy 0, policy_version 802716 (0.0010) [2023-12-27 11:10:56,666][20381] Updated weights for policy 1, policy_version 802173 (0.0005) [2023-12-27 11:10:56,725][20381] Updated weights for policy 1, policy_version 802183 (0.0007) [2023-12-27 11:10:56,779][20381] Updated weights for policy 1, policy_version 802193 (0.0007) [2023-12-27 11:10:57,252][20347] Updated weights for policy 0, policy_version 802726 (0.0007) [2023-12-27 11:10:57,317][20347] Updated weights for policy 0, policy_version 802736 (0.0005) [2023-12-27 11:10:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 410910720. Throughput: 0: 10380.9, 1: 10411.9. Samples: 410904304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:10:57,363][19170] Avg episode reward: [(0, '458.410'), (1, '618.570')] [2023-12-27 11:10:57,373][20347] Updated weights for policy 0, policy_version 802746 (0.0008) [2023-12-27 11:10:57,426][20381] Updated weights for policy 1, policy_version 802203 (0.0007) [2023-12-27 11:10:57,485][20381] Updated weights for policy 1, policy_version 802213 (0.0005) [2023-12-27 11:10:57,541][20381] Updated weights for policy 1, policy_version 802223 (0.0005) [2023-12-27 11:10:58,004][20347] Updated weights for policy 0, policy_version 802756 (0.0008) [2023-12-27 11:10:58,055][20347] Updated weights for policy 0, policy_version 802766 (0.0010) [2023-12-27 11:10:58,097][20381] Updated weights for policy 1, policy_version 802233 (0.0006) [2023-12-27 11:10:58,107][20347] Updated weights for policy 0, policy_version 802776 (0.0010) [2023-12-27 11:10:58,145][20381] Updated weights for policy 1, policy_version 802243 (0.0006) [2023-12-27 11:10:58,201][20381] Updated weights for policy 1, policy_version 802253 (0.0007) [2023-12-27 11:10:58,264][20381] Updated weights for policy 1, policy_version 802263 (0.0006) [2023-12-27 11:10:58,991][20347] Updated weights for policy 0, policy_version 802786 (0.0008) [2023-12-27 11:10:59,040][20381] Updated weights for policy 1, policy_version 802273 (0.0010) [2023-12-27 11:10:59,040][20347] Updated weights for policy 0, policy_version 802796 (0.0007) [2023-12-27 11:10:59,099][20347] Updated weights for policy 0, policy_version 802806 (0.0007) [2023-12-27 11:10:59,101][20381] Updated weights for policy 1, policy_version 802283 (0.0008) [2023-12-27 11:10:59,154][20381] Updated weights for policy 1, policy_version 802293 (0.0009) [2023-12-27 11:10:59,157][20347] Updated weights for policy 0, policy_version 802816 (0.0007) [2023-12-27 11:10:59,865][20347] Updated weights for policy 0, policy_version 802826 (0.0007) [2023-12-27 11:10:59,938][20347] Updated weights for policy 0, policy_version 802836 (0.0008) [2023-12-27 11:10:59,951][20381] Updated weights for policy 1, policy_version 802303 (0.0009) [2023-12-27 11:10:59,994][20347] Updated weights for policy 0, policy_version 802846 (0.0009) [2023-12-27 11:10:59,997][20381] Updated weights for policy 1, policy_version 802313 (0.0007) [2023-12-27 11:11:00,048][20381] Updated weights for policy 1, policy_version 802323 (0.0010) [2023-12-27 11:11:00,616][20347] Updated weights for policy 0, policy_version 802856 (0.0010) [2023-12-27 11:11:00,671][20347] Updated weights for policy 0, policy_version 802866 (0.0010) [2023-12-27 11:11:00,695][20381] Updated weights for policy 1, policy_version 802333 (0.0008) [2023-12-27 11:11:00,729][20347] Updated weights for policy 0, policy_version 802876 (0.0010) [2023-12-27 11:11:00,754][20381] Updated weights for policy 1, policy_version 802343 (0.0005) [2023-12-27 11:11:00,814][20381] Updated weights for policy 1, policy_version 802353 (0.0006) [2023-12-27 11:11:01,370][20347] Updated weights for policy 0, policy_version 802886 (0.0010) [2023-12-27 11:11:01,432][20347] Updated weights for policy 0, policy_version 802896 (0.0007) [2023-12-27 11:11:01,474][20381] Updated weights for policy 1, policy_version 802363 (0.0009) [2023-12-27 11:11:01,491][20347] Updated weights for policy 0, policy_version 802906 (0.0007) [2023-12-27 11:11:01,529][20381] Updated weights for policy 1, policy_version 802373 (0.0007) [2023-12-27 11:11:01,587][20381] Updated weights for policy 1, policy_version 802383 (0.0007) [2023-12-27 11:11:02,131][20347] Updated weights for policy 0, policy_version 802916 (0.0007) [2023-12-27 11:11:02,198][20347] Updated weights for policy 0, policy_version 802926 (0.0006) [2023-12-27 11:11:02,260][20347] Updated weights for policy 0, policy_version 802936 (0.0008) [2023-12-27 11:11:02,287][20381] Updated weights for policy 1, policy_version 802393 (0.0009) [2023-12-27 11:11:02,348][20381] Updated weights for policy 1, policy_version 802403 (0.0007) [2023-12-27 11:11:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 411017216. Throughput: 0: 10279.4, 1: 10448.0. Samples: 411027304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:02,363][19170] Avg episode reward: [(0, '389.480'), (1, '719.700')] [2023-12-27 11:11:02,414][20381] Updated weights for policy 1, policy_version 802413 (0.0009) [2023-12-27 11:11:02,480][20381] Updated weights for policy 1, policy_version 802423 (0.0011) [2023-12-27 11:11:02,881][20347] Updated weights for policy 0, policy_version 802946 (0.0008) [2023-12-27 11:11:02,932][20347] Updated weights for policy 0, policy_version 802956 (0.0005) [2023-12-27 11:11:02,979][20347] Updated weights for policy 0, policy_version 802966 (0.0008) [2023-12-27 11:11:03,033][20347] Updated weights for policy 0, policy_version 802976 (0.0005) [2023-12-27 11:11:03,125][20381] Updated weights for policy 1, policy_version 802433 (0.0010) [2023-12-27 11:11:03,181][20381] Updated weights for policy 1, policy_version 802443 (0.0005) [2023-12-27 11:11:03,239][20381] Updated weights for policy 1, policy_version 802453 (0.0010) [2023-12-27 11:11:03,692][20347] Updated weights for policy 0, policy_version 802986 (0.0005) [2023-12-27 11:11:03,748][20347] Updated weights for policy 0, policy_version 802996 (0.0005) [2023-12-27 11:11:03,801][20347] Updated weights for policy 0, policy_version 803006 (0.0005) [2023-12-27 11:11:03,889][20381] Updated weights for policy 1, policy_version 802463 (0.0011) [2023-12-27 11:11:03,948][20381] Updated weights for policy 1, policy_version 802473 (0.0011) [2023-12-27 11:11:04,010][20381] Updated weights for policy 1, policy_version 802483 (0.0011) [2023-12-27 11:11:04,453][20347] Updated weights for policy 0, policy_version 803016 (0.0007) [2023-12-27 11:11:04,508][20347] Updated weights for policy 0, policy_version 803026 (0.0005) [2023-12-27 11:11:04,567][20347] Updated weights for policy 0, policy_version 803036 (0.0010) [2023-12-27 11:11:04,715][20381] Updated weights for policy 1, policy_version 802493 (0.0011) [2023-12-27 11:11:04,785][20381] Updated weights for policy 1, policy_version 802503 (0.0010) [2023-12-27 11:11:04,851][20381] Updated weights for policy 1, policy_version 802513 (0.0009) [2023-12-27 11:11:05,160][20347] Updated weights for policy 0, policy_version 803046 (0.0008) [2023-12-27 11:11:05,214][20347] Updated weights for policy 0, policy_version 803056 (0.0005) [2023-12-27 11:11:05,263][20347] Updated weights for policy 0, policy_version 803066 (0.0005) [2023-12-27 11:11:05,502][20381] Updated weights for policy 1, policy_version 802523 (0.0007) [2023-12-27 11:11:05,565][20381] Updated weights for policy 1, policy_version 802533 (0.0006) [2023-12-27 11:11:05,633][20381] Updated weights for policy 1, policy_version 802543 (0.0009) [2023-12-27 11:11:05,781][20347] Updated weights for policy 0, policy_version 803076 (0.0005) [2023-12-27 11:11:05,838][20347] Updated weights for policy 0, policy_version 803086 (0.0006) [2023-12-27 11:11:05,893][20347] Updated weights for policy 0, policy_version 803096 (0.0006) [2023-12-27 11:11:06,285][20381] Updated weights for policy 1, policy_version 802553 (0.0011) [2023-12-27 11:11:06,349][20381] Updated weights for policy 1, policy_version 802563 (0.0011) [2023-12-27 11:11:06,415][20381] Updated weights for policy 1, policy_version 802573 (0.0008) [2023-12-27 11:11:06,475][20381] Updated weights for policy 1, policy_version 802583 (0.0006) [2023-12-27 11:11:06,547][20347] Updated weights for policy 0, policy_version 803106 (0.0007) [2023-12-27 11:11:06,606][20347] Updated weights for policy 0, policy_version 803116 (0.0007) [2023-12-27 11:11:06,671][20347] Updated weights for policy 0, policy_version 803126 (0.0007) [2023-12-27 11:11:06,731][20347] Updated weights for policy 0, policy_version 803136 (0.0006) [2023-12-27 11:11:07,105][20381] Updated weights for policy 1, policy_version 802593 (0.0005) [2023-12-27 11:11:07,160][20381] Updated weights for policy 1, policy_version 802603 (0.0005) [2023-12-27 11:11:07,213][20381] Updated weights for policy 1, policy_version 802613 (0.0006) [2023-12-27 11:11:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 411123712. Throughput: 0: 10283.9, 1: 10472.4. Samples: 411089980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:07,364][19170] Avg episode reward: [(0, '287.200'), (1, '613.910')] [2023-12-27 11:11:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000802616_205496320.pth... [2023-12-27 11:11:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000801400_205185024.pth [2023-12-27 11:11:07,403][20347] Updated weights for policy 0, policy_version 803146 (0.0007) [2023-12-27 11:11:07,449][20347] Updated weights for policy 0, policy_version 803156 (0.0006) [2023-12-27 11:11:07,502][20347] Updated weights for policy 0, policy_version 803166 (0.0005) [2023-12-27 11:11:07,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000803168_205635584.pth... [2023-12-27 11:11:07,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000801952_205324288.pth [2023-12-27 11:11:07,834][20381] Updated weights for policy 1, policy_version 802623 (0.0009) [2023-12-27 11:11:07,892][20381] Updated weights for policy 1, policy_version 802633 (0.0011) [2023-12-27 11:11:07,958][20381] Updated weights for policy 1, policy_version 802643 (0.0011) [2023-12-27 11:11:08,157][20347] Updated weights for policy 0, policy_version 803176 (0.0008) [2023-12-27 11:11:08,213][20347] Updated weights for policy 0, policy_version 803186 (0.0009) [2023-12-27 11:11:08,270][20347] Updated weights for policy 0, policy_version 803196 (0.0007) [2023-12-27 11:11:08,610][20381] Updated weights for policy 1, policy_version 802653 (0.0007) [2023-12-27 11:11:08,661][20381] Updated weights for policy 1, policy_version 802663 (0.0008) [2023-12-27 11:11:08,706][20381] Updated weights for policy 1, policy_version 802673 (0.0007) [2023-12-27 11:11:08,996][20347] Updated weights for policy 0, policy_version 803206 (0.0007) [2023-12-27 11:11:09,044][20347] Updated weights for policy 0, policy_version 803216 (0.0007) [2023-12-27 11:11:09,100][20347] Updated weights for policy 0, policy_version 803226 (0.0007) [2023-12-27 11:11:09,438][20381] Updated weights for policy 1, policy_version 802683 (0.0008) [2023-12-27 11:11:09,498][20381] Updated weights for policy 1, policy_version 802693 (0.0011) [2023-12-27 11:11:09,548][20381] Updated weights for policy 1, policy_version 802703 (0.0011) [2023-12-27 11:11:09,861][20347] Updated weights for policy 0, policy_version 803236 (0.0007) [2023-12-27 11:11:09,929][20347] Updated weights for policy 0, policy_version 803246 (0.0007) [2023-12-27 11:11:09,994][20347] Updated weights for policy 0, policy_version 803256 (0.0007) [2023-12-27 11:11:10,318][20381] Updated weights for policy 1, policy_version 802713 (0.0011) [2023-12-27 11:11:10,381][20381] Updated weights for policy 1, policy_version 802723 (0.0010) [2023-12-27 11:11:10,437][20381] Updated weights for policy 1, policy_version 802733 (0.0011) [2023-12-27 11:11:10,488][20381] Updated weights for policy 1, policy_version 802743 (0.0011) [2023-12-27 11:11:10,717][20347] Updated weights for policy 0, policy_version 803266 (0.0007) [2023-12-27 11:11:10,774][20347] Updated weights for policy 0, policy_version 803276 (0.0010) [2023-12-27 11:11:10,827][20347] Updated weights for policy 0, policy_version 803286 (0.0009) [2023-12-27 11:11:10,874][20347] Updated weights for policy 0, policy_version 803296 (0.0009) [2023-12-27 11:11:11,114][20381] Updated weights for policy 1, policy_version 802753 (0.0011) [2023-12-27 11:11:11,188][20381] Updated weights for policy 1, policy_version 802763 (0.0010) [2023-12-27 11:11:11,255][20381] Updated weights for policy 1, policy_version 802773 (0.0007) [2023-12-27 11:11:11,614][20347] Updated weights for policy 0, policy_version 803306 (0.0010) [2023-12-27 11:11:11,679][20347] Updated weights for policy 0, policy_version 803316 (0.0010) [2023-12-27 11:11:11,737][20347] Updated weights for policy 0, policy_version 803326 (0.0010) [2023-12-27 11:11:11,915][20381] Updated weights for policy 1, policy_version 802783 (0.0006) [2023-12-27 11:11:11,978][20381] Updated weights for policy 1, policy_version 802793 (0.0006) [2023-12-27 11:11:12,030][20381] Updated weights for policy 1, policy_version 802803 (0.0007) [2023-12-27 11:11:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 411222016. Throughput: 0: 10302.0, 1: 10435.9. Samples: 411213528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:12,363][19170] Avg episode reward: [(0, '311.360'), (1, '651.800')] [2023-12-27 11:11:12,465][20347] Updated weights for policy 0, policy_version 803336 (0.0006) [2023-12-27 11:11:12,521][20347] Updated weights for policy 0, policy_version 803346 (0.0005) [2023-12-27 11:11:12,579][20347] Updated weights for policy 0, policy_version 803356 (0.0005) [2023-12-27 11:11:12,637][20381] Updated weights for policy 1, policy_version 802813 (0.0007) [2023-12-27 11:11:12,696][20381] Updated weights for policy 1, policy_version 802823 (0.0008) [2023-12-27 11:11:12,750][20381] Updated weights for policy 1, policy_version 802833 (0.0007) [2023-12-27 11:11:13,173][20347] Updated weights for policy 0, policy_version 803366 (0.0006) [2023-12-27 11:11:13,231][20347] Updated weights for policy 0, policy_version 803376 (0.0005) [2023-12-27 11:11:13,290][20347] Updated weights for policy 0, policy_version 803386 (0.0008) [2023-12-27 11:11:13,435][20381] Updated weights for policy 1, policy_version 802843 (0.0007) [2023-12-27 11:11:13,479][20381] Updated weights for policy 1, policy_version 802853 (0.0007) [2023-12-27 11:11:13,523][20381] Updated weights for policy 1, policy_version 802863 (0.0007) [2023-12-27 11:11:13,947][20347] Updated weights for policy 0, policy_version 803396 (0.0008) [2023-12-27 11:11:14,005][20347] Updated weights for policy 0, policy_version 803406 (0.0005) [2023-12-27 11:11:14,061][20347] Updated weights for policy 0, policy_version 803416 (0.0006) [2023-12-27 11:11:14,165][20381] Updated weights for policy 1, policy_version 802873 (0.0006) [2023-12-27 11:11:14,229][20381] Updated weights for policy 1, policy_version 802883 (0.0009) [2023-12-27 11:11:14,288][20381] Updated weights for policy 1, policy_version 802893 (0.0008) [2023-12-27 11:11:14,343][20381] Updated weights for policy 1, policy_version 802903 (0.0008) [2023-12-27 11:11:14,612][20347] Updated weights for policy 0, policy_version 803426 (0.0005) [2023-12-27 11:11:14,672][20347] Updated weights for policy 0, policy_version 803436 (0.0007) [2023-12-27 11:11:14,727][20347] Updated weights for policy 0, policy_version 803446 (0.0007) [2023-12-27 11:11:14,790][20347] Updated weights for policy 0, policy_version 803456 (0.0008) [2023-12-27 11:11:15,052][20381] Updated weights for policy 1, policy_version 802913 (0.0008) [2023-12-27 11:11:15,113][20381] Updated weights for policy 1, policy_version 802923 (0.0007) [2023-12-27 11:11:15,174][20381] Updated weights for policy 1, policy_version 802933 (0.0008) [2023-12-27 11:11:15,493][20347] Updated weights for policy 0, policy_version 803466 (0.0007) [2023-12-27 11:11:15,554][20347] Updated weights for policy 0, policy_version 803476 (0.0007) [2023-12-27 11:11:15,607][20347] Updated weights for policy 0, policy_version 803486 (0.0008) [2023-12-27 11:11:15,852][20381] Updated weights for policy 1, policy_version 802943 (0.0008) [2023-12-27 11:11:15,901][20381] Updated weights for policy 1, policy_version 802953 (0.0009) [2023-12-27 11:11:15,949][20381] Updated weights for policy 1, policy_version 802963 (0.0010) [2023-12-27 11:11:16,238][20347] Updated weights for policy 0, policy_version 803496 (0.0007) [2023-12-27 11:11:16,295][20347] Updated weights for policy 0, policy_version 803506 (0.0007) [2023-12-27 11:11:16,354][20347] Updated weights for policy 0, policy_version 803516 (0.0007) [2023-12-27 11:11:16,561][20381] Updated weights for policy 1, policy_version 802973 (0.0008) [2023-12-27 11:11:16,606][20381] Updated weights for policy 1, policy_version 802983 (0.0008) [2023-12-27 11:11:16,652][20381] Updated weights for policy 1, policy_version 802993 (0.0005) [2023-12-27 11:11:17,026][20347] Updated weights for policy 0, policy_version 803526 (0.0007) [2023-12-27 11:11:17,084][20347] Updated weights for policy 0, policy_version 803536 (0.0010) [2023-12-27 11:11:17,136][20347] Updated weights for policy 0, policy_version 803546 (0.0008) [2023-12-27 11:11:17,333][20381] Updated weights for policy 1, policy_version 803003 (0.0010) [2023-12-27 11:11:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 411328512. Throughput: 0: 10318.9, 1: 10454.6. Samples: 411340584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:17,363][19170] Avg episode reward: [(0, '200.950'), (1, '425.670')] [2023-12-27 11:11:17,403][20381] Updated weights for policy 1, policy_version 803013 (0.0010) [2023-12-27 11:11:17,456][20381] Updated weights for policy 1, policy_version 803023 (0.0007) [2023-12-27 11:11:17,705][20347] Updated weights for policy 0, policy_version 803556 (0.0007) [2023-12-27 11:11:17,763][20347] Updated weights for policy 0, policy_version 803566 (0.0010) [2023-12-27 11:11:17,820][20347] Updated weights for policy 0, policy_version 803576 (0.0010) [2023-12-27 11:11:18,074][20381] Updated weights for policy 1, policy_version 803033 (0.0009) [2023-12-27 11:11:18,129][20381] Updated weights for policy 1, policy_version 803043 (0.0010) [2023-12-27 11:11:18,189][20381] Updated weights for policy 1, policy_version 803053 (0.0011) [2023-12-27 11:11:18,246][20381] Updated weights for policy 1, policy_version 803063 (0.0010) [2023-12-27 11:11:18,452][20347] Updated weights for policy 0, policy_version 803586 (0.0009) [2023-12-27 11:11:18,513][20347] Updated weights for policy 0, policy_version 803596 (0.0007) [2023-12-27 11:11:18,569][20347] Updated weights for policy 0, policy_version 803606 (0.0006) [2023-12-27 11:11:18,628][20347] Updated weights for policy 0, policy_version 803616 (0.0007) [2023-12-27 11:11:18,953][20381] Updated weights for policy 1, policy_version 803073 (0.0011) [2023-12-27 11:11:19,003][20381] Updated weights for policy 1, policy_version 803083 (0.0011) [2023-12-27 11:11:19,055][20381] Updated weights for policy 1, policy_version 803093 (0.0011) [2023-12-27 11:11:19,275][20347] Updated weights for policy 0, policy_version 803626 (0.0008) [2023-12-27 11:11:19,337][20347] Updated weights for policy 0, policy_version 803636 (0.0007) [2023-12-27 11:11:19,402][20347] Updated weights for policy 0, policy_version 803646 (0.0008) [2023-12-27 11:11:19,811][20381] Updated weights for policy 1, policy_version 803103 (0.0011) [2023-12-27 11:11:19,872][20381] Updated weights for policy 1, policy_version 803113 (0.0009) [2023-12-27 11:11:19,925][20381] Updated weights for policy 1, policy_version 803123 (0.0007) [2023-12-27 11:11:20,119][20347] Updated weights for policy 0, policy_version 803656 (0.0008) [2023-12-27 11:11:20,175][20347] Updated weights for policy 0, policy_version 803666 (0.0005) [2023-12-27 11:11:20,229][20347] Updated weights for policy 0, policy_version 803676 (0.0006) [2023-12-27 11:11:20,661][20381] Updated weights for policy 1, policy_version 803133 (0.0009) [2023-12-27 11:11:20,728][20381] Updated weights for policy 1, policy_version 803143 (0.0006) [2023-12-27 11:11:20,796][20381] Updated weights for policy 1, policy_version 803153 (0.0006) [2023-12-27 11:11:20,892][20347] Updated weights for policy 0, policy_version 803686 (0.0008) [2023-12-27 11:11:20,952][20347] Updated weights for policy 0, policy_version 803696 (0.0008) [2023-12-27 11:11:20,999][20347] Updated weights for policy 0, policy_version 803706 (0.0008) [2023-12-27 11:11:21,427][20381] Updated weights for policy 1, policy_version 803163 (0.0008) [2023-12-27 11:11:21,473][20381] Updated weights for policy 1, policy_version 803173 (0.0011) [2023-12-27 11:11:21,541][20381] Updated weights for policy 1, policy_version 803184 (0.0009) [2023-12-27 11:11:21,791][20347] Updated weights for policy 0, policy_version 803716 (0.0008) [2023-12-27 11:11:21,857][20347] Updated weights for policy 0, policy_version 803726 (0.0008) [2023-12-27 11:11:21,919][20347] Updated weights for policy 0, policy_version 803736 (0.0007) [2023-12-27 11:11:22,255][20381] Updated weights for policy 1, policy_version 803194 (0.0007) [2023-12-27 11:11:22,311][20381] Updated weights for policy 1, policy_version 803204 (0.0009) [2023-12-27 11:11:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 411426816. Throughput: 0: 10331.9, 1: 10411.5. Samples: 411401788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:22,363][19170] Avg episode reward: [(0, '254.110'), (1, '596.500')] [2023-12-27 11:11:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000803744_205783040.pth... [2023-12-27 11:11:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000802528_205471744.pth [2023-12-27 11:11:22,379][20381] Updated weights for policy 1, policy_version 803214 (0.0008) [2023-12-27 11:11:22,445][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000803224_205651968.pth... [2023-12-27 11:11:22,446][20381] Updated weights for policy 1, policy_version 803224 (0.0008) [2023-12-27 11:11:22,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000802008_205340672.pth [2023-12-27 11:11:22,615][20347] Updated weights for policy 0, policy_version 803746 (0.0007) [2023-12-27 11:11:22,673][20347] Updated weights for policy 0, policy_version 803756 (0.0007) [2023-12-27 11:11:22,735][20347] Updated weights for policy 0, policy_version 803766 (0.0007) [2023-12-27 11:11:22,795][20347] Updated weights for policy 0, policy_version 803776 (0.0007) [2023-12-27 11:11:23,172][20381] Updated weights for policy 1, policy_version 803234 (0.0009) [2023-12-27 11:11:23,233][20381] Updated weights for policy 1, policy_version 803244 (0.0008) [2023-12-27 11:11:23,278][20381] Updated weights for policy 1, policy_version 803254 (0.0007) [2023-12-27 11:11:23,398][20347] Updated weights for policy 0, policy_version 803786 (0.0005) [2023-12-27 11:11:23,461][20347] Updated weights for policy 0, policy_version 803796 (0.0005) [2023-12-27 11:11:23,510][20347] Updated weights for policy 0, policy_version 803806 (0.0006) [2023-12-27 11:11:23,991][20381] Updated weights for policy 1, policy_version 803264 (0.0010) [2023-12-27 11:11:24,041][20381] Updated weights for policy 1, policy_version 803274 (0.0011) [2023-12-27 11:11:24,091][20381] Updated weights for policy 1, policy_version 803284 (0.0010) [2023-12-27 11:11:24,115][20347] Updated weights for policy 0, policy_version 803816 (0.0006) [2023-12-27 11:11:24,161][20347] Updated weights for policy 0, policy_version 803826 (0.0005) [2023-12-27 11:11:24,212][20347] Updated weights for policy 0, policy_version 803836 (0.0006) [2023-12-27 11:11:24,788][20381] Updated weights for policy 1, policy_version 803294 (0.0010) [2023-12-27 11:11:24,835][20381] Updated weights for policy 1, policy_version 803304 (0.0007) [2023-12-27 11:11:24,867][20347] Updated weights for policy 0, policy_version 803846 (0.0007) [2023-12-27 11:11:24,888][20381] Updated weights for policy 1, policy_version 803314 (0.0005) [2023-12-27 11:11:24,922][20347] Updated weights for policy 0, policy_version 803856 (0.0008) [2023-12-27 11:11:24,981][20347] Updated weights for policy 0, policy_version 803866 (0.0006) [2023-12-27 11:11:25,526][20381] Updated weights for policy 1, policy_version 803324 (0.0007) [2023-12-27 11:11:25,574][20381] Updated weights for policy 1, policy_version 803334 (0.0010) [2023-12-27 11:11:25,612][20347] Updated weights for policy 0, policy_version 803876 (0.0007) [2023-12-27 11:11:25,626][20381] Updated weights for policy 1, policy_version 803344 (0.0010) [2023-12-27 11:11:25,666][20347] Updated weights for policy 0, policy_version 803886 (0.0007) [2023-12-27 11:11:25,725][20347] Updated weights for policy 0, policy_version 803896 (0.0007) [2023-12-27 11:11:26,264][20381] Updated weights for policy 1, policy_version 803354 (0.0009) [2023-12-27 11:11:26,311][20347] Updated weights for policy 0, policy_version 803906 (0.0006) [2023-12-27 11:11:26,322][20381] Updated weights for policy 1, policy_version 803364 (0.0005) [2023-12-27 11:11:26,366][20347] Updated weights for policy 0, policy_version 803916 (0.0006) [2023-12-27 11:11:26,377][20381] Updated weights for policy 1, policy_version 803374 (0.0005) [2023-12-27 11:11:26,424][20347] Updated weights for policy 0, policy_version 803926 (0.0006) [2023-12-27 11:11:26,435][20381] Updated weights for policy 1, policy_version 803384 (0.0006) [2023-12-27 11:11:26,474][20347] Updated weights for policy 0, policy_version 803936 (0.0005) [2023-12-27 11:11:26,948][20381] Updated weights for policy 1, policy_version 803394 (0.0005) [2023-12-27 11:11:27,009][20381] Updated weights for policy 1, policy_version 803404 (0.0005) [2023-12-27 11:11:27,009][20347] Updated weights for policy 0, policy_version 803946 (0.0005) [2023-12-27 11:11:27,064][20347] Updated weights for policy 0, policy_version 803956 (0.0005) [2023-12-27 11:11:27,068][20381] Updated weights for policy 1, policy_version 803414 (0.0005) [2023-12-27 11:11:27,116][20347] Updated weights for policy 0, policy_version 803966 (0.0007) [2023-12-27 11:11:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 411541504. Throughput: 0: 10363.8, 1: 10381.5. Samples: 411524536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:27,364][19170] Avg episode reward: [(0, '233.960'), (1, '823.420')] [2023-12-27 11:11:27,632][20347] Updated weights for policy 0, policy_version 803976 (0.0005) [2023-12-27 11:11:27,636][20381] Updated weights for policy 1, policy_version 803424 (0.0005) [2023-12-27 11:11:27,683][20347] Updated weights for policy 0, policy_version 803986 (0.0006) [2023-12-27 11:11:27,688][20381] Updated weights for policy 1, policy_version 803434 (0.0006) [2023-12-27 11:11:27,738][20347] Updated weights for policy 0, policy_version 803996 (0.0005) [2023-12-27 11:11:27,743][20381] Updated weights for policy 1, policy_version 803444 (0.0006) [2023-12-27 11:11:28,300][20381] Updated weights for policy 1, policy_version 803454 (0.0005) [2023-12-27 11:11:28,354][20381] Updated weights for policy 1, policy_version 803464 (0.0006) [2023-12-27 11:11:28,416][20381] Updated weights for policy 1, policy_version 803474 (0.0006) [2023-12-27 11:11:28,452][20347] Updated weights for policy 0, policy_version 804006 (0.0008) [2023-12-27 11:11:28,510][20347] Updated weights for policy 0, policy_version 804016 (0.0010) [2023-12-27 11:11:28,571][20347] Updated weights for policy 0, policy_version 804026 (0.0010) [2023-12-27 11:11:29,048][20381] Updated weights for policy 1, policy_version 803484 (0.0007) [2023-12-27 11:11:29,104][20381] Updated weights for policy 1, policy_version 803494 (0.0007) [2023-12-27 11:11:29,163][20381] Updated weights for policy 1, policy_version 803504 (0.0007) [2023-12-27 11:11:29,226][20347] Updated weights for policy 0, policy_version 804036 (0.0010) [2023-12-27 11:11:29,288][20347] Updated weights for policy 0, policy_version 804046 (0.0007) [2023-12-27 11:11:29,351][20347] Updated weights for policy 0, policy_version 804056 (0.0008) [2023-12-27 11:11:29,815][20381] Updated weights for policy 1, policy_version 803514 (0.0008) [2023-12-27 11:11:29,877][20381] Updated weights for policy 1, policy_version 803524 (0.0010) [2023-12-27 11:11:29,939][20381] Updated weights for policy 1, policy_version 803534 (0.0008) [2023-12-27 11:11:29,999][20381] Updated weights for policy 1, policy_version 803544 (0.0007) [2023-12-27 11:11:30,079][20347] Updated weights for policy 0, policy_version 804066 (0.0008) [2023-12-27 11:11:30,138][20347] Updated weights for policy 0, policy_version 804076 (0.0008) [2023-12-27 11:11:30,192][20347] Updated weights for policy 0, policy_version 804086 (0.0008) [2023-12-27 11:11:30,243][20347] Updated weights for policy 0, policy_version 804096 (0.0008) [2023-12-27 11:11:30,637][20381] Updated weights for policy 1, policy_version 803554 (0.0005) [2023-12-27 11:11:30,697][20381] Updated weights for policy 1, policy_version 803564 (0.0007) [2023-12-27 11:11:30,753][20381] Updated weights for policy 1, policy_version 803574 (0.0009) [2023-12-27 11:11:30,934][20347] Updated weights for policy 0, policy_version 804106 (0.0007) [2023-12-27 11:11:30,988][20347] Updated weights for policy 0, policy_version 804116 (0.0007) [2023-12-27 11:11:31,053][20347] Updated weights for policy 0, policy_version 804126 (0.0007) [2023-12-27 11:11:31,472][20381] Updated weights for policy 1, policy_version 803584 (0.0010) [2023-12-27 11:11:31,531][20381] Updated weights for policy 1, policy_version 803594 (0.0010) [2023-12-27 11:11:31,582][20381] Updated weights for policy 1, policy_version 803604 (0.0010) [2023-12-27 11:11:31,782][20347] Updated weights for policy 0, policy_version 804136 (0.0007) [2023-12-27 11:11:31,846][20347] Updated weights for policy 0, policy_version 804146 (0.0006) [2023-12-27 11:11:31,910][20347] Updated weights for policy 0, policy_version 804156 (0.0006) [2023-12-27 11:11:32,232][20381] Updated weights for policy 1, policy_version 803614 (0.0008) [2023-12-27 11:11:32,300][20381] Updated weights for policy 1, policy_version 803624 (0.0008) [2023-12-27 11:11:32,360][20381] Updated weights for policy 1, policy_version 803634 (0.0007) [2023-12-27 11:11:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 411639808. Throughput: 0: 10418.3, 1: 10495.0. Samples: 411655092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:32,363][19170] Avg episode reward: [(0, '317.760'), (1, '796.180')] [2023-12-27 11:11:32,504][20347] Updated weights for policy 0, policy_version 804166 (0.0008) [2023-12-27 11:11:32,566][20347] Updated weights for policy 0, policy_version 804176 (0.0008) [2023-12-27 11:11:32,627][20347] Updated weights for policy 0, policy_version 804186 (0.0007) [2023-12-27 11:11:32,992][20381] Updated weights for policy 1, policy_version 803644 (0.0008) [2023-12-27 11:11:33,043][20381] Updated weights for policy 1, policy_version 803654 (0.0007) [2023-12-27 11:11:33,101][20381] Updated weights for policy 1, policy_version 803664 (0.0005) [2023-12-27 11:11:33,283][20347] Updated weights for policy 0, policy_version 804196 (0.0007) [2023-12-27 11:11:33,336][20347] Updated weights for policy 0, policy_version 804206 (0.0007) [2023-12-27 11:11:33,393][20347] Updated weights for policy 0, policy_version 804216 (0.0007) [2023-12-27 11:11:33,751][20381] Updated weights for policy 1, policy_version 803674 (0.0006) [2023-12-27 11:11:33,806][20381] Updated weights for policy 1, policy_version 803684 (0.0009) [2023-12-27 11:11:33,863][20381] Updated weights for policy 1, policy_version 803694 (0.0009) [2023-12-27 11:11:33,909][20381] Updated weights for policy 1, policy_version 803704 (0.0007) [2023-12-27 11:11:34,003][20347] Updated weights for policy 0, policy_version 804226 (0.0007) [2023-12-27 11:11:34,055][20347] Updated weights for policy 0, policy_version 804236 (0.0007) [2023-12-27 11:11:34,114][20347] Updated weights for policy 0, policy_version 804246 (0.0007) [2023-12-27 11:11:34,175][20347] Updated weights for policy 0, policy_version 804256 (0.0007) [2023-12-27 11:11:34,642][20381] Updated weights for policy 1, policy_version 803714 (0.0011) [2023-12-27 11:11:34,700][20381] Updated weights for policy 1, policy_version 803724 (0.0010) [2023-12-27 11:11:34,759][20381] Updated weights for policy 1, policy_version 803734 (0.0010) [2023-12-27 11:11:34,907][20347] Updated weights for policy 0, policy_version 804266 (0.0007) [2023-12-27 11:11:34,958][20347] Updated weights for policy 0, policy_version 804276 (0.0007) [2023-12-27 11:11:35,004][20347] Updated weights for policy 0, policy_version 804286 (0.0010) [2023-12-27 11:11:35,436][20381] Updated weights for policy 1, policy_version 803744 (0.0011) [2023-12-27 11:11:35,491][20381] Updated weights for policy 1, policy_version 803754 (0.0010) [2023-12-27 11:11:35,543][20381] Updated weights for policy 1, policy_version 803764 (0.0010) [2023-12-27 11:11:35,604][20347] Updated weights for policy 0, policy_version 804296 (0.0010) [2023-12-27 11:11:35,665][20347] Updated weights for policy 0, policy_version 804307 (0.0010) [2023-12-27 11:11:35,717][20347] Updated weights for policy 0, policy_version 804317 (0.0005) [2023-12-27 11:11:36,191][20381] Updated weights for policy 1, policy_version 803774 (0.0011) [2023-12-27 11:11:36,255][20381] Updated weights for policy 1, policy_version 803784 (0.0011) [2023-12-27 11:11:36,317][20381] Updated weights for policy 1, policy_version 803794 (0.0011) [2023-12-27 11:11:36,370][20347] Updated weights for policy 0, policy_version 804327 (0.0009) [2023-12-27 11:11:36,427][20347] Updated weights for policy 0, policy_version 804337 (0.0010) [2023-12-27 11:11:36,478][20347] Updated weights for policy 0, policy_version 804347 (0.0010) [2023-12-27 11:11:37,003][20381] Updated weights for policy 1, policy_version 803804 (0.0009) [2023-12-27 11:11:37,060][20381] Updated weights for policy 1, policy_version 803814 (0.0010) [2023-12-27 11:11:37,107][20381] Updated weights for policy 1, policy_version 803824 (0.0009) [2023-12-27 11:11:37,198][20347] Updated weights for policy 0, policy_version 804357 (0.0009) [2023-12-27 11:11:37,252][20347] Updated weights for policy 0, policy_version 804367 (0.0008) [2023-12-27 11:11:37,310][20347] Updated weights for policy 0, policy_version 804377 (0.0008) [2023-12-27 11:11:37,362][19170] Fps is (10 sec: 21299.7, 60 sec: 21026.1, 300 sec: 20605.0). Total num frames: 411754496. Throughput: 0: 10429.0, 1: 10439.5. Samples: 411717516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:37,363][19170] Avg episode reward: [(0, '324.680'), (1, '699.180')] [2023-12-27 11:11:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000803832_205807616.pth... [2023-12-27 11:11:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000804384_205946880.pth... [2023-12-27 11:11:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000802616_205496320.pth [2023-12-27 11:11:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000803168_205635584.pth [2023-12-27 11:11:37,752][20381] Updated weights for policy 1, policy_version 803834 (0.0008) [2023-12-27 11:11:37,806][20381] Updated weights for policy 1, policy_version 803844 (0.0005) [2023-12-27 11:11:37,855][20381] Updated weights for policy 1, policy_version 803854 (0.0009) [2023-12-27 11:11:37,901][20381] Updated weights for policy 1, policy_version 803864 (0.0010) [2023-12-27 11:11:37,965][20347] Updated weights for policy 0, policy_version 804387 (0.0007) [2023-12-27 11:11:38,024][20347] Updated weights for policy 0, policy_version 804397 (0.0008) [2023-12-27 11:11:38,081][20347] Updated weights for policy 0, policy_version 804407 (0.0009) [2023-12-27 11:11:38,563][20381] Updated weights for policy 1, policy_version 803874 (0.0006) [2023-12-27 11:11:38,624][20381] Updated weights for policy 1, policy_version 803884 (0.0006) [2023-12-27 11:11:38,680][20381] Updated weights for policy 1, policy_version 803894 (0.0008) [2023-12-27 11:11:38,775][20347] Updated weights for policy 0, policy_version 804417 (0.0007) [2023-12-27 11:11:38,831][20347] Updated weights for policy 0, policy_version 804427 (0.0008) [2023-12-27 11:11:38,892][20347] Updated weights for policy 0, policy_version 804437 (0.0007) [2023-12-27 11:11:38,950][20347] Updated weights for policy 0, policy_version 804447 (0.0008) [2023-12-27 11:11:39,329][20381] Updated weights for policy 1, policy_version 803904 (0.0008) [2023-12-27 11:11:39,389][20381] Updated weights for policy 1, policy_version 803914 (0.0009) [2023-12-27 11:11:39,453][20381] Updated weights for policy 1, policy_version 803924 (0.0008) [2023-12-27 11:11:39,680][20347] Updated weights for policy 0, policy_version 804457 (0.0008) [2023-12-27 11:11:39,738][20347] Updated weights for policy 0, policy_version 804467 (0.0008) [2023-12-27 11:11:39,799][20347] Updated weights for policy 0, policy_version 804477 (0.0008) [2023-12-27 11:11:40,171][20381] Updated weights for policy 1, policy_version 803934 (0.0010) [2023-12-27 11:11:40,233][20381] Updated weights for policy 1, policy_version 803944 (0.0011) [2023-12-27 11:11:40,288][20381] Updated weights for policy 1, policy_version 803954 (0.0010) [2023-12-27 11:11:40,579][20347] Updated weights for policy 0, policy_version 804487 (0.0006) [2023-12-27 11:11:40,642][20347] Updated weights for policy 0, policy_version 804497 (0.0006) [2023-12-27 11:11:40,698][20347] Updated weights for policy 0, policy_version 804507 (0.0007) [2023-12-27 11:11:40,960][20381] Updated weights for policy 1, policy_version 803964 (0.0011) [2023-12-27 11:11:41,012][20381] Updated weights for policy 1, policy_version 803974 (0.0011) [2023-12-27 11:11:41,076][20381] Updated weights for policy 1, policy_version 803984 (0.0010) [2023-12-27 11:11:41,411][20347] Updated weights for policy 0, policy_version 804517 (0.0008) [2023-12-27 11:11:41,480][20347] Updated weights for policy 0, policy_version 804527 (0.0007) [2023-12-27 11:11:41,557][20347] Updated weights for policy 0, policy_version 804537 (0.0008) [2023-12-27 11:11:41,800][20381] Updated weights for policy 1, policy_version 803994 (0.0010) [2023-12-27 11:11:41,852][20381] Updated weights for policy 1, policy_version 804004 (0.0009) [2023-12-27 11:11:41,906][20381] Updated weights for policy 1, policy_version 804014 (0.0011) [2023-12-27 11:11:41,954][20381] Updated weights for policy 1, policy_version 804024 (0.0011) [2023-12-27 11:11:42,242][20347] Updated weights for policy 0, policy_version 804547 (0.0008) [2023-12-27 11:11:42,300][20347] Updated weights for policy 0, policy_version 804557 (0.0010) [2023-12-27 11:11:42,360][20347] Updated weights for policy 0, policy_version 804567 (0.0010) [2023-12-27 11:11:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 411844608. Throughput: 0: 10420.5, 1: 10367.4. Samples: 411839756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 11:11:42,363][19170] Avg episode reward: [(0, '214.720'), (1, '796.130')] [2023-12-27 11:11:42,706][20381] Updated weights for policy 1, policy_version 804034 (0.0010) [2023-12-27 11:11:42,764][20381] Updated weights for policy 1, policy_version 804044 (0.0008) [2023-12-27 11:11:42,824][20381] Updated weights for policy 1, policy_version 804054 (0.0007) [2023-12-27 11:11:42,949][20347] Updated weights for policy 0, policy_version 804577 (0.0007) [2023-12-27 11:11:43,001][20347] Updated weights for policy 0, policy_version 804587 (0.0008) [2023-12-27 11:11:43,057][20347] Updated weights for policy 0, policy_version 804597 (0.0010) [2023-12-27 11:11:43,116][20347] Updated weights for policy 0, policy_version 804607 (0.0010) [2023-12-27 11:11:43,511][20381] Updated weights for policy 1, policy_version 804064 (0.0007) [2023-12-27 11:11:43,568][20381] Updated weights for policy 1, policy_version 804074 (0.0007) [2023-12-27 11:11:43,630][20381] Updated weights for policy 1, policy_version 804084 (0.0007) [2023-12-27 11:11:43,774][20347] Updated weights for policy 0, policy_version 804617 (0.0006) [2023-12-27 11:11:43,819][20347] Updated weights for policy 0, policy_version 804627 (0.0010) [2023-12-27 11:11:43,873][20347] Updated weights for policy 0, policy_version 804637 (0.0005) [2023-12-27 11:11:44,323][20381] Updated weights for policy 1, policy_version 804094 (0.0008) [2023-12-27 11:11:44,380][20381] Updated weights for policy 1, policy_version 804104 (0.0006) [2023-12-27 11:11:44,443][20381] Updated weights for policy 1, policy_version 804114 (0.0007) [2023-12-27 11:11:44,493][20347] Updated weights for policy 0, policy_version 804647 (0.0006) [2023-12-27 11:11:44,551][20347] Updated weights for policy 0, policy_version 804657 (0.0007) [2023-12-27 11:11:44,605][20347] Updated weights for policy 0, policy_version 804667 (0.0007) [2023-12-27 11:11:45,074][20381] Updated weights for policy 1, policy_version 804124 (0.0009) [2023-12-27 11:11:45,135][20381] Updated weights for policy 1, policy_version 804134 (0.0011) [2023-12-27 11:11:45,194][20381] Updated weights for policy 1, policy_version 804144 (0.0011) [2023-12-27 11:11:45,312][20347] Updated weights for policy 0, policy_version 804677 (0.0007) [2023-12-27 11:11:45,375][20347] Updated weights for policy 0, policy_version 804687 (0.0007) [2023-12-27 11:11:45,436][20347] Updated weights for policy 0, policy_version 804697 (0.0009) [2023-12-27 11:11:45,821][20381] Updated weights for policy 1, policy_version 804154 (0.0011) [2023-12-27 11:11:45,876][20381] Updated weights for policy 1, policy_version 804164 (0.0010) [2023-12-27 11:11:45,929][20381] Updated weights for policy 1, policy_version 804174 (0.0010) [2023-12-27 11:11:45,991][20381] Updated weights for policy 1, policy_version 804184 (0.0005) [2023-12-27 11:11:46,151][20347] Updated weights for policy 0, policy_version 804707 (0.0007) [2023-12-27 11:11:46,212][20347] Updated weights for policy 0, policy_version 804717 (0.0009) [2023-12-27 11:11:46,266][20347] Updated weights for policy 0, policy_version 804727 (0.0007) [2023-12-27 11:11:46,569][20381] Updated weights for policy 1, policy_version 804194 (0.0008) [2023-12-27 11:11:46,633][20381] Updated weights for policy 1, policy_version 804204 (0.0008) [2023-12-27 11:11:46,695][20381] Updated weights for policy 1, policy_version 804214 (0.0010) [2023-12-27 11:11:46,874][20347] Updated weights for policy 0, policy_version 804737 (0.0007) [2023-12-27 11:11:46,929][20347] Updated weights for policy 0, policy_version 804747 (0.0010) [2023-12-27 11:11:46,983][20347] Updated weights for policy 0, policy_version 804757 (0.0009) [2023-12-27 11:11:47,038][20347] Updated weights for policy 0, policy_version 804767 (0.0008) [2023-12-27 11:11:47,320][20381] Updated weights for policy 1, policy_version 804224 (0.0008) [2023-12-27 11:11:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 411951104. Throughput: 0: 10442.0, 1: 10357.6. Samples: 411963284. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:11:47,363][19170] Avg episode reward: [(0, '245.360'), (1, '756.980')] [2023-12-27 11:11:47,376][20381] Updated weights for policy 1, policy_version 804234 (0.0008) [2023-12-27 11:11:47,428][20381] Updated weights for policy 1, policy_version 804244 (0.0008) [2023-12-27 11:11:47,735][20347] Updated weights for policy 0, policy_version 804777 (0.0007) [2023-12-27 11:11:47,799][20347] Updated weights for policy 0, policy_version 804787 (0.0005) [2023-12-27 11:11:47,851][20347] Updated weights for policy 0, policy_version 804797 (0.0005) [2023-12-27 11:11:48,102][20381] Updated weights for policy 1, policy_version 804254 (0.0007) [2023-12-27 11:11:48,156][20381] Updated weights for policy 1, policy_version 804264 (0.0009) [2023-12-27 11:11:48,209][20381] Updated weights for policy 1, policy_version 804274 (0.0009) [2023-12-27 11:11:48,417][20347] Updated weights for policy 0, policy_version 804807 (0.0006) [2023-12-27 11:11:48,475][20347] Updated weights for policy 0, policy_version 804817 (0.0005) [2023-12-27 11:11:48,538][20347] Updated weights for policy 0, policy_version 804827 (0.0007) [2023-12-27 11:11:48,891][20381] Updated weights for policy 1, policy_version 804284 (0.0008) [2023-12-27 11:11:48,954][20381] Updated weights for policy 1, policy_version 804294 (0.0006) [2023-12-27 11:11:49,015][20381] Updated weights for policy 1, policy_version 804304 (0.0007) [2023-12-27 11:11:49,143][20347] Updated weights for policy 0, policy_version 804837 (0.0008) [2023-12-27 11:11:49,201][20347] Updated weights for policy 0, policy_version 804847 (0.0008) [2023-12-27 11:11:49,262][20347] Updated weights for policy 0, policy_version 804857 (0.0008) [2023-12-27 11:11:49,717][20381] Updated weights for policy 1, policy_version 804314 (0.0007) [2023-12-27 11:11:49,770][20381] Updated weights for policy 1, policy_version 804324 (0.0009) [2023-12-27 11:11:49,834][20381] Updated weights for policy 1, policy_version 804334 (0.0006) [2023-12-27 11:11:49,902][20381] Updated weights for policy 1, policy_version 804344 (0.0006) [2023-12-27 11:11:49,968][20347] Updated weights for policy 0, policy_version 804867 (0.0008) [2023-12-27 11:11:50,030][20347] Updated weights for policy 0, policy_version 804877 (0.0007) [2023-12-27 11:11:50,090][20347] Updated weights for policy 0, policy_version 804887 (0.0007) [2023-12-27 11:11:50,534][20381] Updated weights for policy 1, policy_version 804354 (0.0007) [2023-12-27 11:11:50,595][20381] Updated weights for policy 1, policy_version 804364 (0.0008) [2023-12-27 11:11:50,646][20381] Updated weights for policy 1, policy_version 804374 (0.0008) [2023-12-27 11:11:50,805][20347] Updated weights for policy 0, policy_version 804897 (0.0008) [2023-12-27 11:11:50,857][20347] Updated weights for policy 0, policy_version 804907 (0.0008) [2023-12-27 11:11:50,911][20347] Updated weights for policy 0, policy_version 804917 (0.0007) [2023-12-27 11:11:50,968][20347] Updated weights for policy 0, policy_version 804927 (0.0007) [2023-12-27 11:11:51,342][20381] Updated weights for policy 1, policy_version 804384 (0.0010) [2023-12-27 11:11:51,409][20381] Updated weights for policy 1, policy_version 804394 (0.0008) [2023-12-27 11:11:51,460][20381] Updated weights for policy 1, policy_version 804404 (0.0008) [2023-12-27 11:11:51,684][20347] Updated weights for policy 0, policy_version 804937 (0.0008) [2023-12-27 11:11:51,752][20347] Updated weights for policy 0, policy_version 804947 (0.0008) [2023-12-27 11:11:51,816][20347] Updated weights for policy 0, policy_version 804957 (0.0008) [2023-12-27 11:11:52,179][20381] Updated weights for policy 1, policy_version 804414 (0.0010) [2023-12-27 11:11:52,229][20381] Updated weights for policy 1, policy_version 804424 (0.0011) [2023-12-27 11:11:52,290][20381] Updated weights for policy 1, policy_version 804434 (0.0011) [2023-12-27 11:11:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 412057600. Throughput: 0: 10434.8, 1: 10372.7. Samples: 412026316. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:11:52,363][19170] Avg episode reward: [(0, '377.790'), (1, '768.890')] [2023-12-27 11:11:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000804440_205963264.pth... [2023-12-27 11:11:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000803224_205651968.pth [2023-12-27 11:11:52,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000804960_206094336.pth... [2023-12-27 11:11:52,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000803744_205783040.pth [2023-12-27 11:11:52,595][20347] Updated weights for policy 0, policy_version 804967 (0.0007) [2023-12-27 11:11:52,654][20347] Updated weights for policy 0, policy_version 804977 (0.0008) [2023-12-27 11:11:52,719][20347] Updated weights for policy 0, policy_version 804987 (0.0008) [2023-12-27 11:11:52,993][20381] Updated weights for policy 1, policy_version 804444 (0.0009) [2023-12-27 11:11:53,053][20381] Updated weights for policy 1, policy_version 804454 (0.0010) [2023-12-27 11:11:53,115][20381] Updated weights for policy 1, policy_version 804464 (0.0011) [2023-12-27 11:11:53,397][20347] Updated weights for policy 0, policy_version 804997 (0.0007) [2023-12-27 11:11:53,450][20347] Updated weights for policy 0, policy_version 805007 (0.0007) [2023-12-27 11:11:53,505][20347] Updated weights for policy 0, policy_version 805017 (0.0007) [2023-12-27 11:11:53,786][20381] Updated weights for policy 1, policy_version 804474 (0.0010) [2023-12-27 11:11:53,841][20381] Updated weights for policy 1, policy_version 804484 (0.0011) [2023-12-27 11:11:53,903][20381] Updated weights for policy 1, policy_version 804494 (0.0011) [2023-12-27 11:11:53,967][20381] Updated weights for policy 1, policy_version 804504 (0.0009) [2023-12-27 11:11:54,217][20347] Updated weights for policy 0, policy_version 805027 (0.0007) [2023-12-27 11:11:54,271][20347] Updated weights for policy 0, policy_version 805037 (0.0007) [2023-12-27 11:11:54,336][20347] Updated weights for policy 0, policy_version 805047 (0.0006) [2023-12-27 11:11:54,572][20381] Updated weights for policy 1, policy_version 804514 (0.0011) [2023-12-27 11:11:54,617][20381] Updated weights for policy 1, policy_version 804524 (0.0010) [2023-12-27 11:11:54,677][20381] Updated weights for policy 1, policy_version 804534 (0.0010) [2023-12-27 11:11:55,028][20347] Updated weights for policy 0, policy_version 805057 (0.0005) [2023-12-27 11:11:55,077][20347] Updated weights for policy 0, policy_version 805067 (0.0007) [2023-12-27 11:11:55,134][20347] Updated weights for policy 0, policy_version 805077 (0.0007) [2023-12-27 11:11:55,187][20347] Updated weights for policy 0, policy_version 805087 (0.0007) [2023-12-27 11:11:55,370][20381] Updated weights for policy 1, policy_version 804544 (0.0011) [2023-12-27 11:11:55,431][20381] Updated weights for policy 1, policy_version 804554 (0.0010) [2023-12-27 11:11:55,489][20381] Updated weights for policy 1, policy_version 804564 (0.0011) [2023-12-27 11:11:55,899][20347] Updated weights for policy 0, policy_version 805097 (0.0007) [2023-12-27 11:11:55,952][20347] Updated weights for policy 0, policy_version 805107 (0.0007) [2023-12-27 11:11:55,999][20347] Updated weights for policy 0, policy_version 805117 (0.0007) [2023-12-27 11:11:56,139][20381] Updated weights for policy 1, policy_version 804574 (0.0007) [2023-12-27 11:11:56,200][20381] Updated weights for policy 1, policy_version 804584 (0.0005) [2023-12-27 11:11:56,263][20381] Updated weights for policy 1, policy_version 804594 (0.0010) [2023-12-27 11:11:56,740][20347] Updated weights for policy 0, policy_version 805127 (0.0008) [2023-12-27 11:11:56,797][20347] Updated weights for policy 0, policy_version 805137 (0.0008) [2023-12-27 11:11:56,811][20381] Updated weights for policy 1, policy_version 804604 (0.0008) [2023-12-27 11:11:56,852][20347] Updated weights for policy 0, policy_version 805147 (0.0006) [2023-12-27 11:11:56,868][20381] Updated weights for policy 1, policy_version 804614 (0.0005) [2023-12-27 11:11:56,914][20381] Updated weights for policy 1, policy_version 804624 (0.0005) [2023-12-27 11:11:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 412155904. Throughput: 0: 10351.7, 1: 10377.4. Samples: 412146336. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:11:57,363][19170] Avg episode reward: [(0, '433.700'), (1, '731.650')] [2023-12-27 11:11:57,508][20347] Updated weights for policy 0, policy_version 805157 (0.0007) [2023-12-27 11:11:57,522][20381] Updated weights for policy 1, policy_version 804634 (0.0005) [2023-12-27 11:11:57,555][20347] Updated weights for policy 0, policy_version 805167 (0.0007) [2023-12-27 11:11:57,580][20381] Updated weights for policy 1, policy_version 804644 (0.0005) [2023-12-27 11:11:57,602][20347] Updated weights for policy 0, policy_version 805177 (0.0006) [2023-12-27 11:11:57,642][20381] Updated weights for policy 1, policy_version 804654 (0.0005) [2023-12-27 11:11:57,706][20381] Updated weights for policy 1, policy_version 804664 (0.0005) [2023-12-27 11:11:58,331][20381] Updated weights for policy 1, policy_version 804674 (0.0008) [2023-12-27 11:11:58,348][20347] Updated weights for policy 0, policy_version 805187 (0.0007) [2023-12-27 11:11:58,399][20381] Updated weights for policy 1, policy_version 804684 (0.0008) [2023-12-27 11:11:58,411][20347] Updated weights for policy 0, policy_version 805197 (0.0008) [2023-12-27 11:11:58,463][20381] Updated weights for policy 1, policy_version 804694 (0.0008) [2023-12-27 11:11:58,464][20347] Updated weights for policy 0, policy_version 805207 (0.0008) [2023-12-27 11:11:59,236][20381] Updated weights for policy 1, policy_version 804704 (0.0007) [2023-12-27 11:11:59,255][20347] Updated weights for policy 0, policy_version 805217 (0.0008) [2023-12-27 11:11:59,301][20381] Updated weights for policy 1, policy_version 804714 (0.0007) [2023-12-27 11:11:59,314][20347] Updated weights for policy 0, policy_version 805227 (0.0008) [2023-12-27 11:11:59,376][20381] Updated weights for policy 1, policy_version 804724 (0.0008) [2023-12-27 11:11:59,379][20347] Updated weights for policy 0, policy_version 805237 (0.0008) [2023-12-27 11:11:59,433][20347] Updated weights for policy 0, policy_version 805247 (0.0007) [2023-12-27 11:12:00,046][20381] Updated weights for policy 1, policy_version 804734 (0.0007) [2023-12-27 11:12:00,075][20347] Updated weights for policy 0, policy_version 805257 (0.0007) [2023-12-27 11:12:00,096][20381] Updated weights for policy 1, policy_version 804744 (0.0007) [2023-12-27 11:12:00,131][20347] Updated weights for policy 0, policy_version 805267 (0.0006) [2023-12-27 11:12:00,149][20381] Updated weights for policy 1, policy_version 804754 (0.0005) [2023-12-27 11:12:00,181][20347] Updated weights for policy 0, policy_version 805277 (0.0006) [2023-12-27 11:12:00,799][20347] Updated weights for policy 0, policy_version 805287 (0.0006) [2023-12-27 11:12:00,807][20381] Updated weights for policy 1, policy_version 804764 (0.0006) [2023-12-27 11:12:00,858][20381] Updated weights for policy 1, policy_version 804774 (0.0007) [2023-12-27 11:12:00,859][20347] Updated weights for policy 0, policy_version 805297 (0.0006) [2023-12-27 11:12:00,914][20347] Updated weights for policy 0, policy_version 805307 (0.0007) [2023-12-27 11:12:00,915][20381] Updated weights for policy 1, policy_version 804784 (0.0006) [2023-12-27 11:12:01,543][20381] Updated weights for policy 1, policy_version 804794 (0.0006) [2023-12-27 11:12:01,608][20381] Updated weights for policy 1, policy_version 804804 (0.0007) [2023-12-27 11:12:01,631][20347] Updated weights for policy 0, policy_version 805317 (0.0008) [2023-12-27 11:12:01,667][20381] Updated weights for policy 1, policy_version 804814 (0.0007) [2023-12-27 11:12:01,691][20347] Updated weights for policy 0, policy_version 805327 (0.0008) [2023-12-27 11:12:01,729][20381] Updated weights for policy 1, policy_version 804824 (0.0008) [2023-12-27 11:12:01,759][20347] Updated weights for policy 0, policy_version 805337 (0.0008) [2023-12-27 11:12:02,354][20381] Updated weights for policy 1, policy_version 804834 (0.0008) [2023-12-27 11:12:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 412254208. Throughput: 0: 10248.0, 1: 10377.4. Samples: 412268728. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:02,363][19170] Avg episode reward: [(0, '682.500'), (1, '605.060')] [2023-12-27 11:12:02,413][20381] Updated weights for policy 1, policy_version 804844 (0.0008) [2023-12-27 11:12:02,471][20381] Updated weights for policy 1, policy_version 804854 (0.0007) [2023-12-27 11:12:02,523][20347] Updated weights for policy 0, policy_version 805347 (0.0008) [2023-12-27 11:12:02,571][20347] Updated weights for policy 0, policy_version 805357 (0.0007) [2023-12-27 11:12:02,620][20347] Updated weights for policy 0, policy_version 805367 (0.0007) [2023-12-27 11:12:03,096][20381] Updated weights for policy 1, policy_version 804864 (0.0006) [2023-12-27 11:12:03,148][20381] Updated weights for policy 1, policy_version 804874 (0.0009) [2023-12-27 11:12:03,199][20381] Updated weights for policy 1, policy_version 804884 (0.0010) [2023-12-27 11:12:03,371][20347] Updated weights for policy 0, policy_version 805377 (0.0008) [2023-12-27 11:12:03,428][20347] Updated weights for policy 0, policy_version 805387 (0.0007) [2023-12-27 11:12:03,478][20347] Updated weights for policy 0, policy_version 805397 (0.0007) [2023-12-27 11:12:03,534][20347] Updated weights for policy 0, policy_version 805407 (0.0007) [2023-12-27 11:12:03,873][20381] Updated weights for policy 1, policy_version 804894 (0.0011) [2023-12-27 11:12:03,933][20381] Updated weights for policy 1, policy_version 804904 (0.0011) [2023-12-27 11:12:03,996][20381] Updated weights for policy 1, policy_version 804914 (0.0011) [2023-12-27 11:12:04,233][20347] Updated weights for policy 0, policy_version 805417 (0.0008) [2023-12-27 11:12:04,289][20347] Updated weights for policy 0, policy_version 805427 (0.0008) [2023-12-27 11:12:04,339][20347] Updated weights for policy 0, policy_version 805437 (0.0009) [2023-12-27 11:12:04,649][20381] Updated weights for policy 1, policy_version 804924 (0.0008) [2023-12-27 11:12:04,700][20381] Updated weights for policy 1, policy_version 804934 (0.0005) [2023-12-27 11:12:04,755][20381] Updated weights for policy 1, policy_version 804944 (0.0006) [2023-12-27 11:12:05,106][20347] Updated weights for policy 0, policy_version 805447 (0.0007) [2023-12-27 11:12:05,158][20347] Updated weights for policy 0, policy_version 805457 (0.0008) [2023-12-27 11:12:05,208][20347] Updated weights for policy 0, policy_version 805467 (0.0008) [2023-12-27 11:12:05,316][20381] Updated weights for policy 1, policy_version 804954 (0.0006) [2023-12-27 11:12:05,374][20381] Updated weights for policy 1, policy_version 804964 (0.0007) [2023-12-27 11:12:05,435][20381] Updated weights for policy 1, policy_version 804974 (0.0006) [2023-12-27 11:12:05,497][20381] Updated weights for policy 1, policy_version 804984 (0.0006) [2023-12-27 11:12:05,873][20347] Updated weights for policy 0, policy_version 805477 (0.0009) [2023-12-27 11:12:05,938][20347] Updated weights for policy 0, policy_version 805487 (0.0010) [2023-12-27 11:12:05,993][20347] Updated weights for policy 0, policy_version 805497 (0.0010) [2023-12-27 11:12:06,120][20381] Updated weights for policy 1, policy_version 804994 (0.0007) [2023-12-27 11:12:06,178][20381] Updated weights for policy 1, policy_version 805004 (0.0006) [2023-12-27 11:12:06,248][20381] Updated weights for policy 1, policy_version 805014 (0.0008) [2023-12-27 11:12:06,702][20347] Updated weights for policy 0, policy_version 805507 (0.0009) [2023-12-27 11:12:06,764][20347] Updated weights for policy 0, policy_version 805517 (0.0005) [2023-12-27 11:12:06,830][20347] Updated weights for policy 0, policy_version 805527 (0.0010) [2023-12-27 11:12:06,866][20381] Updated weights for policy 1, policy_version 805024 (0.0008) [2023-12-27 11:12:06,932][20381] Updated weights for policy 1, policy_version 805034 (0.0009) [2023-12-27 11:12:06,992][20381] Updated weights for policy 1, policy_version 805044 (0.0008) [2023-12-27 11:12:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 412360704. Throughput: 0: 10192.2, 1: 10443.7. Samples: 412330408. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:07,363][19170] Avg episode reward: [(0, '619.300'), (1, '719.330')] [2023-12-27 11:12:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000805048_206118912.pth... [2023-12-27 11:12:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000805536_206241792.pth... [2023-12-27 11:12:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000804384_205946880.pth [2023-12-27 11:12:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000803832_205807616.pth [2023-12-27 11:12:07,493][20347] Updated weights for policy 0, policy_version 805537 (0.0010) [2023-12-27 11:12:07,552][20347] Updated weights for policy 0, policy_version 805547 (0.0009) [2023-12-27 11:12:07,610][20347] Updated weights for policy 0, policy_version 805557 (0.0006) [2023-12-27 11:12:07,624][20381] Updated weights for policy 1, policy_version 805054 (0.0007) [2023-12-27 11:12:07,656][20347] Updated weights for policy 0, policy_version 805567 (0.0007) [2023-12-27 11:12:07,684][20381] Updated weights for policy 1, policy_version 805064 (0.0005) [2023-12-27 11:12:07,740][20381] Updated weights for policy 1, policy_version 805074 (0.0008) [2023-12-27 11:12:08,301][20347] Updated weights for policy 0, policy_version 805577 (0.0008) [2023-12-27 11:12:08,365][20347] Updated weights for policy 0, policy_version 805587 (0.0007) [2023-12-27 11:12:08,375][20381] Updated weights for policy 1, policy_version 805084 (0.0008) [2023-12-27 11:12:08,415][20347] Updated weights for policy 0, policy_version 805597 (0.0007) [2023-12-27 11:12:08,434][20381] Updated weights for policy 1, policy_version 805094 (0.0008) [2023-12-27 11:12:08,493][20381] Updated weights for policy 1, policy_version 805104 (0.0008) [2023-12-27 11:12:09,075][20347] Updated weights for policy 0, policy_version 805607 (0.0007) [2023-12-27 11:12:09,122][20347] Updated weights for policy 0, policy_version 805617 (0.0008) [2023-12-27 11:12:09,177][20347] Updated weights for policy 0, policy_version 805627 (0.0008) [2023-12-27 11:12:09,198][20381] Updated weights for policy 1, policy_version 805114 (0.0008) [2023-12-27 11:12:09,260][20381] Updated weights for policy 1, policy_version 805124 (0.0009) [2023-12-27 11:12:09,317][20381] Updated weights for policy 1, policy_version 805134 (0.0006) [2023-12-27 11:12:09,394][20381] Updated weights for policy 1, policy_version 805144 (0.0008) [2023-12-27 11:12:09,932][20347] Updated weights for policy 0, policy_version 805637 (0.0007) [2023-12-27 11:12:09,995][20347] Updated weights for policy 0, policy_version 805647 (0.0009) [2023-12-27 11:12:10,048][20347] Updated weights for policy 0, policy_version 805657 (0.0009) [2023-12-27 11:12:10,114][20381] Updated weights for policy 1, policy_version 805154 (0.0007) [2023-12-27 11:12:10,172][20381] Updated weights for policy 1, policy_version 805164 (0.0008) [2023-12-27 11:12:10,233][20381] Updated weights for policy 1, policy_version 805174 (0.0008) [2023-12-27 11:12:10,771][20347] Updated weights for policy 0, policy_version 805667 (0.0007) [2023-12-27 11:12:10,820][20347] Updated weights for policy 0, policy_version 805677 (0.0007) [2023-12-27 11:12:10,876][20347] Updated weights for policy 0, policy_version 805687 (0.0008) [2023-12-27 11:12:10,913][20381] Updated weights for policy 1, policy_version 805184 (0.0006) [2023-12-27 11:12:10,963][20381] Updated weights for policy 1, policy_version 805194 (0.0008) [2023-12-27 11:12:11,011][20381] Updated weights for policy 1, policy_version 805204 (0.0008) [2023-12-27 11:12:11,639][20347] Updated weights for policy 0, policy_version 805697 (0.0007) [2023-12-27 11:12:11,698][20347] Updated weights for policy 0, policy_version 805707 (0.0007) [2023-12-27 11:12:11,765][20347] Updated weights for policy 0, policy_version 805717 (0.0008) [2023-12-27 11:12:11,818][20381] Updated weights for policy 1, policy_version 805214 (0.0008) [2023-12-27 11:12:11,828][20347] Updated weights for policy 0, policy_version 805727 (0.0007) [2023-12-27 11:12:11,874][20381] Updated weights for policy 1, policy_version 805224 (0.0008) [2023-12-27 11:12:11,930][20381] Updated weights for policy 1, policy_version 805234 (0.0008) [2023-12-27 11:12:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 412459008. Throughput: 0: 10149.3, 1: 10460.9. Samples: 412451992. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:12,363][19170] Avg episode reward: [(0, '612.040'), (1, '810.650')] [2023-12-27 11:12:12,497][20347] Updated weights for policy 0, policy_version 805737 (0.0007) [2023-12-27 11:12:12,560][20347] Updated weights for policy 0, policy_version 805747 (0.0005) [2023-12-27 11:12:12,624][20347] Updated weights for policy 0, policy_version 805757 (0.0006) [2023-12-27 11:12:12,686][20381] Updated weights for policy 1, policy_version 805244 (0.0009) [2023-12-27 11:12:12,751][20381] Updated weights for policy 1, policy_version 805254 (0.0010) [2023-12-27 11:12:12,809][20381] Updated weights for policy 1, policy_version 805264 (0.0008) [2023-12-27 11:12:13,239][20347] Updated weights for policy 0, policy_version 805767 (0.0007) [2023-12-27 11:12:13,293][20347] Updated weights for policy 0, policy_version 805777 (0.0007) [2023-12-27 11:12:13,345][20347] Updated weights for policy 0, policy_version 805787 (0.0007) [2023-12-27 11:12:13,446][20381] Updated weights for policy 1, policy_version 805274 (0.0008) [2023-12-27 11:12:13,510][20381] Updated weights for policy 1, policy_version 805284 (0.0010) [2023-12-27 11:12:13,561][20381] Updated weights for policy 1, policy_version 805294 (0.0010) [2023-12-27 11:12:13,621][20381] Updated weights for policy 1, policy_version 805304 (0.0010) [2023-12-27 11:12:14,013][20347] Updated weights for policy 0, policy_version 805797 (0.0009) [2023-12-27 11:12:14,071][20347] Updated weights for policy 0, policy_version 805807 (0.0007) [2023-12-27 11:12:14,122][20347] Updated weights for policy 0, policy_version 805817 (0.0005) [2023-12-27 11:12:14,284][20381] Updated weights for policy 1, policy_version 805314 (0.0006) [2023-12-27 11:12:14,346][20381] Updated weights for policy 1, policy_version 805324 (0.0009) [2023-12-27 11:12:14,408][20381] Updated weights for policy 1, policy_version 805334 (0.0010) [2023-12-27 11:12:14,775][20347] Updated weights for policy 0, policy_version 805827 (0.0006) [2023-12-27 11:12:14,832][20347] Updated weights for policy 0, policy_version 805837 (0.0006) [2023-12-27 11:12:14,888][20347] Updated weights for policy 0, policy_version 805847 (0.0006) [2023-12-27 11:12:15,064][20381] Updated weights for policy 1, policy_version 805344 (0.0006) [2023-12-27 11:12:15,119][20381] Updated weights for policy 1, policy_version 805354 (0.0007) [2023-12-27 11:12:15,173][20381] Updated weights for policy 1, policy_version 805364 (0.0005) [2023-12-27 11:12:15,555][20347] Updated weights for policy 0, policy_version 805857 (0.0008) [2023-12-27 11:12:15,611][20347] Updated weights for policy 0, policy_version 805867 (0.0010) [2023-12-27 11:12:15,665][20347] Updated weights for policy 0, policy_version 805877 (0.0010) [2023-12-27 11:12:15,730][20381] Updated weights for policy 1, policy_version 805374 (0.0005) [2023-12-27 11:12:15,733][20347] Updated weights for policy 0, policy_version 805887 (0.0010) [2023-12-27 11:12:15,781][20381] Updated weights for policy 1, policy_version 805384 (0.0005) [2023-12-27 11:12:15,834][20381] Updated weights for policy 1, policy_version 805394 (0.0005) [2023-12-27 11:12:16,392][20347] Updated weights for policy 0, policy_version 805897 (0.0010) [2023-12-27 11:12:16,452][20347] Updated weights for policy 0, policy_version 805907 (0.0010) [2023-12-27 11:12:16,482][20381] Updated weights for policy 1, policy_version 805404 (0.0007) [2023-12-27 11:12:16,509][20347] Updated weights for policy 0, policy_version 805917 (0.0010) [2023-12-27 11:12:16,536][20381] Updated weights for policy 1, policy_version 805414 (0.0006) [2023-12-27 11:12:16,590][20381] Updated weights for policy 1, policy_version 805424 (0.0007) [2023-12-27 11:12:17,164][20347] Updated weights for policy 0, policy_version 805927 (0.0010) [2023-12-27 11:12:17,218][20347] Updated weights for policy 0, policy_version 805937 (0.0010) [2023-12-27 11:12:17,255][20381] Updated weights for policy 1, policy_version 805434 (0.0007) [2023-12-27 11:12:17,276][20347] Updated weights for policy 0, policy_version 805947 (0.0010) [2023-12-27 11:12:17,313][20381] Updated weights for policy 1, policy_version 805444 (0.0005) [2023-12-27 11:12:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 412565504. Throughput: 0: 10105.8, 1: 10371.7. Samples: 412576576. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:17,363][19170] Avg episode reward: [(0, '589.420'), (1, '853.590')] [2023-12-27 11:12:17,372][20381] Updated weights for policy 1, policy_version 805454 (0.0005) [2023-12-27 11:12:17,431][20381] Updated weights for policy 1, policy_version 805464 (0.0005) [2023-12-27 11:12:17,957][20347] Updated weights for policy 0, policy_version 805957 (0.0010) [2023-12-27 11:12:17,978][20381] Updated weights for policy 1, policy_version 805474 (0.0005) [2023-12-27 11:12:18,011][20347] Updated weights for policy 0, policy_version 805967 (0.0010) [2023-12-27 11:12:18,025][20381] Updated weights for policy 1, policy_version 805484 (0.0005) [2023-12-27 11:12:18,069][20347] Updated weights for policy 0, policy_version 805977 (0.0010) [2023-12-27 11:12:18,079][20381] Updated weights for policy 1, policy_version 805494 (0.0005) [2023-12-27 11:12:18,766][20381] Updated weights for policy 1, policy_version 805504 (0.0009) [2023-12-27 11:12:18,775][20347] Updated weights for policy 0, policy_version 805987 (0.0010) [2023-12-27 11:12:18,825][20381] Updated weights for policy 1, policy_version 805514 (0.0010) [2023-12-27 11:12:18,834][20347] Updated weights for policy 0, policy_version 805997 (0.0011) [2023-12-27 11:12:18,887][20381] Updated weights for policy 1, policy_version 805524 (0.0011) [2023-12-27 11:12:18,893][20347] Updated weights for policy 0, policy_version 806007 (0.0010) [2023-12-27 11:12:19,547][20381] Updated weights for policy 1, policy_version 805534 (0.0009) [2023-12-27 11:12:19,605][20381] Updated weights for policy 1, policy_version 805544 (0.0008) [2023-12-27 11:12:19,616][20347] Updated weights for policy 0, policy_version 806017 (0.0010) [2023-12-27 11:12:19,661][20381] Updated weights for policy 1, policy_version 805554 (0.0010) [2023-12-27 11:12:19,678][20347] Updated weights for policy 0, policy_version 806027 (0.0010) [2023-12-27 11:12:19,746][20347] Updated weights for policy 0, policy_version 806037 (0.0010) [2023-12-27 11:12:19,808][20347] Updated weights for policy 0, policy_version 806047 (0.0010) [2023-12-27 11:12:20,367][20381] Updated weights for policy 1, policy_version 805564 (0.0010) [2023-12-27 11:12:20,424][20381] Updated weights for policy 1, policy_version 805574 (0.0009) [2023-12-27 11:12:20,479][20381] Updated weights for policy 1, policy_version 805584 (0.0005) [2023-12-27 11:12:20,493][20347] Updated weights for policy 0, policy_version 806057 (0.0007) [2023-12-27 11:12:20,545][20347] Updated weights for policy 0, policy_version 806067 (0.0008) [2023-12-27 11:12:20,601][20347] Updated weights for policy 0, policy_version 806077 (0.0008) [2023-12-27 11:12:21,167][20381] Updated weights for policy 1, policy_version 805594 (0.0006) [2023-12-27 11:12:21,234][20381] Updated weights for policy 1, policy_version 805604 (0.0011) [2023-12-27 11:12:21,298][20381] Updated weights for policy 1, policy_version 805614 (0.0012) [2023-12-27 11:12:21,309][20347] Updated weights for policy 0, policy_version 806087 (0.0008) [2023-12-27 11:12:21,363][20381] Updated weights for policy 1, policy_version 805624 (0.0011) [2023-12-27 11:12:21,373][20347] Updated weights for policy 0, policy_version 806097 (0.0009) [2023-12-27 11:12:21,433][20347] Updated weights for policy 0, policy_version 806107 (0.0008) [2023-12-27 11:12:22,072][20381] Updated weights for policy 1, policy_version 805634 (0.0009) [2023-12-27 11:12:22,131][20381] Updated weights for policy 1, policy_version 805644 (0.0007) [2023-12-27 11:12:22,195][20347] Updated weights for policy 0, policy_version 806117 (0.0009) [2023-12-27 11:12:22,196][20381] Updated weights for policy 1, policy_version 805654 (0.0006) [2023-12-27 11:12:22,257][20347] Updated weights for policy 0, policy_version 806127 (0.0011) [2023-12-27 11:12:22,320][20347] Updated weights for policy 0, policy_version 806137 (0.0011) [2023-12-27 11:12:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 412663808. Throughput: 0: 10064.7, 1: 10386.3. Samples: 412637812. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:22,363][19170] Avg episode reward: [(0, '598.050'), (1, '854.550')] [2023-12-27 11:12:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000805656_206274560.pth... [2023-12-27 11:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000806144_206397440.pth... [2023-12-27 11:12:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000804960_206094336.pth [2023-12-27 11:12:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000804440_205963264.pth [2023-12-27 11:12:22,918][20381] Updated weights for policy 1, policy_version 805664 (0.0005) [2023-12-27 11:12:22,977][20381] Updated weights for policy 1, policy_version 805674 (0.0006) [2023-12-27 11:12:23,015][20347] Updated weights for policy 0, policy_version 806147 (0.0010) [2023-12-27 11:12:23,024][20381] Updated weights for policy 1, policy_version 805684 (0.0007) [2023-12-27 11:12:23,072][20347] Updated weights for policy 0, policy_version 806157 (0.0005) [2023-12-27 11:12:23,121][20347] Updated weights for policy 0, policy_version 806167 (0.0007) [2023-12-27 11:12:23,568][20381] Updated weights for policy 1, policy_version 805694 (0.0006) [2023-12-27 11:12:23,626][20381] Updated weights for policy 1, policy_version 805704 (0.0005) [2023-12-27 11:12:23,684][20381] Updated weights for policy 1, policy_version 805714 (0.0006) [2023-12-27 11:12:23,766][20347] Updated weights for policy 0, policy_version 806177 (0.0007) [2023-12-27 11:12:23,854][20347] Updated weights for policy 0, policy_version 806187 (0.0006) [2023-12-27 11:12:23,904][20347] Updated weights for policy 0, policy_version 806197 (0.0007) [2023-12-27 11:12:23,957][20347] Updated weights for policy 0, policy_version 806207 (0.0007) [2023-12-27 11:12:24,244][20381] Updated weights for policy 1, policy_version 805724 (0.0006) [2023-12-27 11:12:24,294][20381] Updated weights for policy 1, policy_version 805734 (0.0005) [2023-12-27 11:12:24,354][20381] Updated weights for policy 1, policy_version 805744 (0.0005) [2023-12-27 11:12:24,605][20347] Updated weights for policy 0, policy_version 806217 (0.0010) [2023-12-27 11:12:24,659][20347] Updated weights for policy 0, policy_version 806227 (0.0010) [2023-12-27 11:12:24,722][20347] Updated weights for policy 0, policy_version 806237 (0.0010) [2023-12-27 11:12:24,991][20381] Updated weights for policy 1, policy_version 805754 (0.0006) [2023-12-27 11:12:25,050][20381] Updated weights for policy 1, policy_version 805764 (0.0007) [2023-12-27 11:12:25,107][20381] Updated weights for policy 1, policy_version 805774 (0.0010) [2023-12-27 11:12:25,161][20381] Updated weights for policy 1, policy_version 805784 (0.0010) [2023-12-27 11:12:25,349][20347] Updated weights for policy 0, policy_version 806247 (0.0007) [2023-12-27 11:12:25,397][20347] Updated weights for policy 0, policy_version 806257 (0.0006) [2023-12-27 11:12:25,455][20347] Updated weights for policy 0, policy_version 806267 (0.0007) [2023-12-27 11:12:25,782][20381] Updated weights for policy 1, policy_version 805794 (0.0007) [2023-12-27 11:12:25,837][20381] Updated weights for policy 1, policy_version 805804 (0.0010) [2023-12-27 11:12:25,891][20381] Updated weights for policy 1, policy_version 805814 (0.0009) [2023-12-27 11:12:26,042][20347] Updated weights for policy 0, policy_version 806277 (0.0007) [2023-12-27 11:12:26,092][20347] Updated weights for policy 0, policy_version 806287 (0.0007) [2023-12-27 11:12:26,149][20347] Updated weights for policy 0, policy_version 806297 (0.0007) [2023-12-27 11:12:26,552][20381] Updated weights for policy 1, policy_version 805824 (0.0009) [2023-12-27 11:12:26,597][20381] Updated weights for policy 1, policy_version 805834 (0.0010) [2023-12-27 11:12:26,642][20381] Updated weights for policy 1, policy_version 805844 (0.0010) [2023-12-27 11:12:26,832][20347] Updated weights for policy 0, policy_version 806307 (0.0007) [2023-12-27 11:12:26,879][20347] Updated weights for policy 0, policy_version 806317 (0.0007) [2023-12-27 11:12:26,929][20347] Updated weights for policy 0, policy_version 806327 (0.0007) [2023-12-27 11:12:27,346][20381] Updated weights for policy 1, policy_version 805854 (0.0007) [2023-12-27 11:12:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 412770304. Throughput: 0: 10078.6, 1: 10442.2. Samples: 412763196. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:27,363][19170] Avg episode reward: [(0, '633.500'), (1, '873.920')] [2023-12-27 11:12:27,409][20381] Updated weights for policy 1, policy_version 805864 (0.0010) [2023-12-27 11:12:27,457][20381] Updated weights for policy 1, policy_version 805874 (0.0010) [2023-12-27 11:12:27,625][20347] Updated weights for policy 0, policy_version 806337 (0.0007) [2023-12-27 11:12:27,680][20347] Updated weights for policy 0, policy_version 806347 (0.0007) [2023-12-27 11:12:27,737][20347] Updated weights for policy 0, policy_version 806357 (0.0006) [2023-12-27 11:12:27,799][20347] Updated weights for policy 0, policy_version 806367 (0.0007) [2023-12-27 11:12:28,090][20381] Updated weights for policy 1, policy_version 805884 (0.0009) [2023-12-27 11:12:28,137][20381] Updated weights for policy 1, policy_version 805894 (0.0010) [2023-12-27 11:12:28,185][20381] Updated weights for policy 1, policy_version 805904 (0.0006) [2023-12-27 11:12:28,491][20347] Updated weights for policy 0, policy_version 806377 (0.0008) [2023-12-27 11:12:28,550][20347] Updated weights for policy 0, policy_version 806387 (0.0007) [2023-12-27 11:12:28,597][20347] Updated weights for policy 0, policy_version 806397 (0.0007) [2023-12-27 11:12:28,832][20381] Updated weights for policy 1, policy_version 805914 (0.0009) [2023-12-27 11:12:28,888][20381] Updated weights for policy 1, policy_version 805924 (0.0009) [2023-12-27 11:12:28,946][20381] Updated weights for policy 1, policy_version 805934 (0.0008) [2023-12-27 11:12:29,004][20381] Updated weights for policy 1, policy_version 805944 (0.0010) [2023-12-27 11:12:29,302][20347] Updated weights for policy 0, policy_version 806407 (0.0008) [2023-12-27 11:12:29,368][20347] Updated weights for policy 0, policy_version 806417 (0.0007) [2023-12-27 11:12:29,422][20347] Updated weights for policy 0, policy_version 806427 (0.0007) [2023-12-27 11:12:29,724][20381] Updated weights for policy 1, policy_version 805954 (0.0008) [2023-12-27 11:12:29,776][20381] Updated weights for policy 1, policy_version 805964 (0.0008) [2023-12-27 11:12:29,834][20381] Updated weights for policy 1, policy_version 805974 (0.0007) [2023-12-27 11:12:30,059][20347] Updated weights for policy 0, policy_version 806437 (0.0008) [2023-12-27 11:12:30,121][20347] Updated weights for policy 0, policy_version 806447 (0.0008) [2023-12-27 11:12:30,183][20347] Updated weights for policy 0, policy_version 806457 (0.0008) [2023-12-27 11:12:30,508][20381] Updated weights for policy 1, policy_version 805984 (0.0008) [2023-12-27 11:12:30,566][20381] Updated weights for policy 1, policy_version 805994 (0.0007) [2023-12-27 11:12:30,622][20381] Updated weights for policy 1, policy_version 806004 (0.0008) [2023-12-27 11:12:30,821][20347] Updated weights for policy 0, policy_version 806467 (0.0008) [2023-12-27 11:12:30,875][20347] Updated weights for policy 0, policy_version 806477 (0.0005) [2023-12-27 11:12:30,923][20347] Updated weights for policy 0, policy_version 806487 (0.0006) [2023-12-27 11:12:31,348][20381] Updated weights for policy 1, policy_version 806014 (0.0008) [2023-12-27 11:12:31,412][20381] Updated weights for policy 1, policy_version 806024 (0.0008) [2023-12-27 11:12:31,467][20381] Updated weights for policy 1, policy_version 806034 (0.0007) [2023-12-27 11:12:31,578][20347] Updated weights for policy 0, policy_version 806497 (0.0005) [2023-12-27 11:12:31,636][20347] Updated weights for policy 0, policy_version 806507 (0.0008) [2023-12-27 11:12:31,690][20347] Updated weights for policy 0, policy_version 806517 (0.0008) [2023-12-27 11:12:31,753][20347] Updated weights for policy 0, policy_version 806527 (0.0008) [2023-12-27 11:12:32,179][20381] Updated weights for policy 1, policy_version 806044 (0.0008) [2023-12-27 11:12:32,228][20381] Updated weights for policy 1, policy_version 806054 (0.0008) [2023-12-27 11:12:32,288][20381] Updated weights for policy 1, policy_version 806064 (0.0008) [2023-12-27 11:12:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 412876800. Throughput: 0: 10076.4, 1: 10414.8. Samples: 412885392. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:32,363][19170] Avg episode reward: [(0, '607.240'), (1, '785.680')] [2023-12-27 11:12:32,463][20347] Updated weights for policy 0, policy_version 806537 (0.0007) [2023-12-27 11:12:32,516][20347] Updated weights for policy 0, policy_version 806547 (0.0007) [2023-12-27 11:12:32,567][20347] Updated weights for policy 0, policy_version 806557 (0.0007) [2023-12-27 11:12:33,010][20381] Updated weights for policy 1, policy_version 806074 (0.0007) [2023-12-27 11:12:33,071][20381] Updated weights for policy 1, policy_version 806084 (0.0006) [2023-12-27 11:12:33,124][20381] Updated weights for policy 1, policy_version 806094 (0.0006) [2023-12-27 11:12:33,177][20381] Updated weights for policy 1, policy_version 806104 (0.0005) [2023-12-27 11:12:33,197][20347] Updated weights for policy 0, policy_version 806567 (0.0006) [2023-12-27 11:12:33,254][20347] Updated weights for policy 0, policy_version 806577 (0.0007) [2023-12-27 11:12:33,312][20347] Updated weights for policy 0, policy_version 806587 (0.0007) [2023-12-27 11:12:33,714][20381] Updated weights for policy 1, policy_version 806114 (0.0005) [2023-12-27 11:12:33,778][20381] Updated weights for policy 1, policy_version 806124 (0.0006) [2023-12-27 11:12:33,828][20381] Updated weights for policy 1, policy_version 806134 (0.0005) [2023-12-27 11:12:33,887][20347] Updated weights for policy 0, policy_version 806597 (0.0008) [2023-12-27 11:12:33,941][20347] Updated weights for policy 0, policy_version 806607 (0.0007) [2023-12-27 11:12:33,987][20347] Updated weights for policy 0, policy_version 806617 (0.0009) [2023-12-27 11:12:34,450][20381] Updated weights for policy 1, policy_version 806144 (0.0009) [2023-12-27 11:12:34,504][20381] Updated weights for policy 1, policy_version 806154 (0.0009) [2023-12-27 11:12:34,563][20381] Updated weights for policy 1, policy_version 806164 (0.0007) [2023-12-27 11:12:34,748][20347] Updated weights for policy 0, policy_version 806627 (0.0007) [2023-12-27 11:12:34,809][20347] Updated weights for policy 0, policy_version 806637 (0.0008) [2023-12-27 11:12:34,866][20347] Updated weights for policy 0, policy_version 806647 (0.0008) [2023-12-27 11:12:35,219][20381] Updated weights for policy 1, policy_version 806174 (0.0008) [2023-12-27 11:12:35,271][20381] Updated weights for policy 1, policy_version 806184 (0.0010) [2023-12-27 11:12:35,325][20381] Updated weights for policy 1, policy_version 806194 (0.0007) [2023-12-27 11:12:35,612][20347] Updated weights for policy 0, policy_version 806657 (0.0008) [2023-12-27 11:12:35,663][20347] Updated weights for policy 0, policy_version 806667 (0.0009) [2023-12-27 11:12:35,706][20347] Updated weights for policy 0, policy_version 806677 (0.0009) [2023-12-27 11:12:35,753][20347] Updated weights for policy 0, policy_version 806687 (0.0007) [2023-12-27 11:12:35,892][20381] Updated weights for policy 1, policy_version 806204 (0.0007) [2023-12-27 11:12:35,947][20381] Updated weights for policy 1, policy_version 806214 (0.0010) [2023-12-27 11:12:35,998][20381] Updated weights for policy 1, policy_version 806224 (0.0010) [2023-12-27 11:12:36,503][20347] Updated weights for policy 0, policy_version 806697 (0.0006) [2023-12-27 11:12:36,568][20347] Updated weights for policy 0, policy_version 806707 (0.0005) [2023-12-27 11:12:36,634][20347] Updated weights for policy 0, policy_version 806717 (0.0006) [2023-12-27 11:12:36,677][20381] Updated weights for policy 1, policy_version 806234 (0.0011) [2023-12-27 11:12:36,742][20381] Updated weights for policy 1, policy_version 806244 (0.0010) [2023-12-27 11:12:36,812][20381] Updated weights for policy 1, policy_version 806254 (0.0008) [2023-12-27 11:12:36,874][20381] Updated weights for policy 1, policy_version 806264 (0.0005) [2023-12-27 11:12:37,239][20347] Updated weights for policy 0, policy_version 806727 (0.0007) [2023-12-27 11:12:37,291][20347] Updated weights for policy 0, policy_version 806737 (0.0006) [2023-12-27 11:12:37,347][20347] Updated weights for policy 0, policy_version 806747 (0.0006) [2023-12-27 11:12:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 412975104. Throughput: 0: 10048.4, 1: 10451.4. Samples: 412948804. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:37,363][19170] Avg episode reward: [(0, '671.730'), (1, '816.280')] [2023-12-27 11:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000806264_206430208.pth... [2023-12-27 11:12:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000805048_206118912.pth [2023-12-27 11:12:37,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000806752_206553088.pth... [2023-12-27 11:12:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000805536_206241792.pth [2023-12-27 11:12:37,513][20381] Updated weights for policy 1, policy_version 806274 (0.0007) [2023-12-27 11:12:37,600][20381] Updated weights for policy 1, policy_version 806284 (0.0006) [2023-12-27 11:12:37,666][20381] Updated weights for policy 1, policy_version 806294 (0.0006) [2023-12-27 11:12:38,035][20347] Updated weights for policy 0, policy_version 806757 (0.0006) [2023-12-27 11:12:38,092][20347] Updated weights for policy 0, policy_version 806767 (0.0007) [2023-12-27 11:12:38,152][20347] Updated weights for policy 0, policy_version 806777 (0.0007) [2023-12-27 11:12:38,244][20381] Updated weights for policy 1, policy_version 806304 (0.0006) [2023-12-27 11:12:38,294][20381] Updated weights for policy 1, policy_version 806314 (0.0007) [2023-12-27 11:12:38,353][20381] Updated weights for policy 1, policy_version 806324 (0.0008) [2023-12-27 11:12:38,868][20347] Updated weights for policy 0, policy_version 806787 (0.0007) [2023-12-27 11:12:38,924][20347] Updated weights for policy 0, policy_version 806797 (0.0007) [2023-12-27 11:12:38,982][20347] Updated weights for policy 0, policy_version 806807 (0.0008) [2023-12-27 11:12:39,041][20381] Updated weights for policy 1, policy_version 806334 (0.0007) [2023-12-27 11:12:39,089][20381] Updated weights for policy 1, policy_version 806344 (0.0007) [2023-12-27 11:12:39,140][20381] Updated weights for policy 1, policy_version 806354 (0.0008) [2023-12-27 11:12:39,711][20347] Updated weights for policy 0, policy_version 806817 (0.0007) [2023-12-27 11:12:39,772][20347] Updated weights for policy 0, policy_version 806827 (0.0008) [2023-12-27 11:12:39,824][20381] Updated weights for policy 1, policy_version 806364 (0.0007) [2023-12-27 11:12:39,839][20347] Updated weights for policy 0, policy_version 806837 (0.0008) [2023-12-27 11:12:39,883][20381] Updated weights for policy 1, policy_version 806374 (0.0006) [2023-12-27 11:12:39,901][20347] Updated weights for policy 0, policy_version 806847 (0.0006) [2023-12-27 11:12:39,944][20381] Updated weights for policy 1, policy_version 806384 (0.0007) [2023-12-27 11:12:40,591][20347] Updated weights for policy 0, policy_version 806857 (0.0009) [2023-12-27 11:12:40,599][20381] Updated weights for policy 1, policy_version 806394 (0.0007) [2023-12-27 11:12:40,640][20347] Updated weights for policy 0, policy_version 806867 (0.0008) [2023-12-27 11:12:40,658][20381] Updated weights for policy 1, policy_version 806404 (0.0005) [2023-12-27 11:12:40,696][20347] Updated weights for policy 0, policy_version 806877 (0.0008) [2023-12-27 11:12:40,719][20381] Updated weights for policy 1, policy_version 806414 (0.0005) [2023-12-27 11:12:40,779][20381] Updated weights for policy 1, policy_version 806424 (0.0005) [2023-12-27 11:12:41,415][20381] Updated weights for policy 1, policy_version 806434 (0.0007) [2023-12-27 11:12:41,475][20381] Updated weights for policy 1, policy_version 806444 (0.0008) [2023-12-27 11:12:41,477][20347] Updated weights for policy 0, policy_version 806887 (0.0007) [2023-12-27 11:12:41,536][20381] Updated weights for policy 1, policy_version 806454 (0.0009) [2023-12-27 11:12:41,537][20347] Updated weights for policy 0, policy_version 806897 (0.0008) [2023-12-27 11:12:41,594][20347] Updated weights for policy 0, policy_version 806907 (0.0008) [2023-12-27 11:12:42,276][20381] Updated weights for policy 1, policy_version 806464 (0.0008) [2023-12-27 11:12:42,286][20347] Updated weights for policy 0, policy_version 806917 (0.0007) [2023-12-27 11:12:42,337][20381] Updated weights for policy 1, policy_version 806474 (0.0007) [2023-12-27 11:12:42,351][20347] Updated weights for policy 0, policy_version 806927 (0.0007) [2023-12-27 11:12:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 413073408. Throughput: 0: 10060.0, 1: 10493.8. Samples: 413071256. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:42,363][19170] Avg episode reward: [(0, '692.750'), (1, '825.950')] [2023-12-27 11:12:42,399][20381] Updated weights for policy 1, policy_version 806484 (0.0008) [2023-12-27 11:12:42,413][20347] Updated weights for policy 0, policy_version 806937 (0.0007) [2023-12-27 11:12:43,042][20347] Updated weights for policy 0, policy_version 806947 (0.0008) [2023-12-27 11:12:43,093][20347] Updated weights for policy 0, policy_version 806957 (0.0009) [2023-12-27 11:12:43,103][20381] Updated weights for policy 1, policy_version 806494 (0.0006) [2023-12-27 11:12:43,147][20347] Updated weights for policy 0, policy_version 806967 (0.0009) [2023-12-27 11:12:43,162][20381] Updated weights for policy 1, policy_version 806504 (0.0005) [2023-12-27 11:12:43,221][20381] Updated weights for policy 1, policy_version 806514 (0.0005) [2023-12-27 11:12:43,794][20347] Updated weights for policy 0, policy_version 806977 (0.0007) [2023-12-27 11:12:43,842][20381] Updated weights for policy 1, policy_version 806524 (0.0006) [2023-12-27 11:12:43,845][20347] Updated weights for policy 0, policy_version 806987 (0.0010) [2023-12-27 11:12:43,887][20381] Updated weights for policy 1, policy_version 806534 (0.0005) [2023-12-27 11:12:43,890][20347] Updated weights for policy 0, policy_version 806997 (0.0010) [2023-12-27 11:12:43,936][20381] Updated weights for policy 1, policy_version 806544 (0.0005) [2023-12-27 11:12:43,938][20347] Updated weights for policy 0, policy_version 807007 (0.0008) [2023-12-27 11:12:44,513][20381] Updated weights for policy 1, policy_version 806554 (0.0007) [2023-12-27 11:12:44,569][20381] Updated weights for policy 1, policy_version 806564 (0.0006) [2023-12-27 11:12:44,610][20347] Updated weights for policy 0, policy_version 807017 (0.0007) [2023-12-27 11:12:44,629][20381] Updated weights for policy 1, policy_version 806574 (0.0005) [2023-12-27 11:12:44,674][20347] Updated weights for policy 0, policy_version 807027 (0.0008) [2023-12-27 11:12:44,689][20381] Updated weights for policy 1, policy_version 806584 (0.0005) [2023-12-27 11:12:44,733][20347] Updated weights for policy 0, policy_version 807037 (0.0007) [2023-12-27 11:12:44,739][20291] KL-divergence is very high: 122.0979 [2023-12-27 11:12:45,369][20381] Updated weights for policy 1, policy_version 806594 (0.0007) [2023-12-27 11:12:45,408][20291] KL-divergence is very high: 126.8634 [2023-12-27 11:12:45,427][20381] Updated weights for policy 1, policy_version 806604 (0.0007) [2023-12-27 11:12:45,444][20347] Updated weights for policy 0, policy_version 807047 (0.0008) [2023-12-27 11:12:45,455][20291] KL-divergence is very high: 123.8988 [2023-12-27 11:12:45,482][20381] Updated weights for policy 1, policy_version 806614 (0.0006) [2023-12-27 11:12:45,499][20347] Updated weights for policy 0, policy_version 807057 (0.0005) [2023-12-27 11:12:45,500][20291] KL-divergence is very high: 118.0091 [2023-12-27 11:12:45,547][20291] KL-divergence is very high: 122.5359 [2023-12-27 11:12:45,558][20347] Updated weights for policy 0, policy_version 807067 (0.0007) [2023-12-27 11:12:46,087][20381] Updated weights for policy 1, policy_version 806624 (0.0005) [2023-12-27 11:12:46,136][20381] Updated weights for policy 1, policy_version 806634 (0.0007) [2023-12-27 11:12:46,184][20381] Updated weights for policy 1, policy_version 806644 (0.0008) [2023-12-27 11:12:46,248][20347] Updated weights for policy 0, policy_version 807077 (0.0007) [2023-12-27 11:12:46,307][20347] Updated weights for policy 0, policy_version 807087 (0.0006) [2023-12-27 11:12:46,361][20347] Updated weights for policy 0, policy_version 807097 (0.0005) [2023-12-27 11:12:46,884][20381] Updated weights for policy 1, policy_version 806654 (0.0007) [2023-12-27 11:12:46,938][20381] Updated weights for policy 1, policy_version 806664 (0.0007) [2023-12-27 11:12:46,999][20381] Updated weights for policy 1, policy_version 806674 (0.0008) [2023-12-27 11:12:47,002][20347] Updated weights for policy 0, policy_version 807107 (0.0007) [2023-12-27 11:12:47,058][20347] Updated weights for policy 0, policy_version 807117 (0.0007) [2023-12-27 11:12:47,113][20347] Updated weights for policy 0, policy_version 807127 (0.0008) [2023-12-27 11:12:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 413188096. Throughput: 0: 10122.1, 1: 10471.9. Samples: 413195456. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:47,363][19170] Avg episode reward: [(0, '690.830'), (1, '824.550')] [2023-12-27 11:12:47,655][20381] Updated weights for policy 1, policy_version 806684 (0.0008) [2023-12-27 11:12:47,710][20381] Updated weights for policy 1, policy_version 806694 (0.0008) [2023-12-27 11:12:47,772][20381] Updated weights for policy 1, policy_version 806704 (0.0007) [2023-12-27 11:12:47,805][20347] Updated weights for policy 0, policy_version 807137 (0.0008) [2023-12-27 11:12:47,852][20347] Updated weights for policy 0, policy_version 807147 (0.0007) [2023-12-27 11:12:47,906][20347] Updated weights for policy 0, policy_version 807157 (0.0007) [2023-12-27 11:12:47,952][20347] Updated weights for policy 0, policy_version 807167 (0.0007) [2023-12-27 11:12:48,444][20381] Updated weights for policy 1, policy_version 806714 (0.0007) [2023-12-27 11:12:48,499][20381] Updated weights for policy 1, policy_version 806724 (0.0007) [2023-12-27 11:12:48,558][20381] Updated weights for policy 1, policy_version 806734 (0.0008) [2023-12-27 11:12:48,619][20381] Updated weights for policy 1, policy_version 806744 (0.0007) [2023-12-27 11:12:48,676][20347] Updated weights for policy 0, policy_version 807177 (0.0010) [2023-12-27 11:12:48,721][20347] Updated weights for policy 0, policy_version 807187 (0.0010) [2023-12-27 11:12:48,787][20347] Updated weights for policy 0, policy_version 807197 (0.0010) [2023-12-27 11:12:49,282][20381] Updated weights for policy 1, policy_version 806754 (0.0007) [2023-12-27 11:12:49,332][20381] Updated weights for policy 1, policy_version 806764 (0.0008) [2023-12-27 11:12:49,395][20381] Updated weights for policy 1, policy_version 806774 (0.0008) [2023-12-27 11:12:49,517][20347] Updated weights for policy 0, policy_version 807207 (0.0009) [2023-12-27 11:12:49,576][20347] Updated weights for policy 0, policy_version 807217 (0.0008) [2023-12-27 11:12:49,632][20347] Updated weights for policy 0, policy_version 807227 (0.0009) [2023-12-27 11:12:50,094][20381] Updated weights for policy 1, policy_version 806784 (0.0006) [2023-12-27 11:12:50,149][20381] Updated weights for policy 1, policy_version 806794 (0.0007) [2023-12-27 11:12:50,205][20381] Updated weights for policy 1, policy_version 806804 (0.0007) [2023-12-27 11:12:50,370][20347] Updated weights for policy 0, policy_version 807237 (0.0008) [2023-12-27 11:12:50,427][20347] Updated weights for policy 0, policy_version 807247 (0.0005) [2023-12-27 11:12:50,476][20347] Updated weights for policy 0, policy_version 807257 (0.0005) [2023-12-27 11:12:50,874][20381] Updated weights for policy 1, policy_version 806814 (0.0006) [2023-12-27 11:12:50,940][20381] Updated weights for policy 1, policy_version 806824 (0.0008) [2023-12-27 11:12:51,007][20381] Updated weights for policy 1, policy_version 806834 (0.0010) [2023-12-27 11:12:51,073][20347] Updated weights for policy 0, policy_version 807267 (0.0006) [2023-12-27 11:12:51,137][20347] Updated weights for policy 0, policy_version 807277 (0.0007) [2023-12-27 11:12:51,199][20347] Updated weights for policy 0, policy_version 807287 (0.0007) [2023-12-27 11:12:51,713][20381] Updated weights for policy 1, policy_version 806844 (0.0008) [2023-12-27 11:12:51,783][20381] Updated weights for policy 1, policy_version 806854 (0.0008) [2023-12-27 11:12:51,850][20381] Updated weights for policy 1, policy_version 806864 (0.0008) [2023-12-27 11:12:51,883][20347] Updated weights for policy 0, policy_version 807297 (0.0006) [2023-12-27 11:12:51,954][20347] Updated weights for policy 0, policy_version 807307 (0.0006) [2023-12-27 11:12:52,020][20347] Updated weights for policy 0, policy_version 807317 (0.0008) [2023-12-27 11:12:52,084][20347] Updated weights for policy 0, policy_version 807327 (0.0008) [2023-12-27 11:12:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 413286400. Throughput: 0: 10135.0, 1: 10423.3. Samples: 413255528. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:52,363][19170] Avg episode reward: [(0, '873.570'), (1, '813.890')] [2023-12-27 11:12:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000806872_206585856.pth... [2023-12-27 11:12:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000807328_206700544.pth... [2023-12-27 11:12:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000805656_206274560.pth [2023-12-27 11:12:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000806144_206397440.pth [2023-12-27 11:12:52,530][20381] Updated weights for policy 1, policy_version 806874 (0.0008) [2023-12-27 11:12:52,584][20381] Updated weights for policy 1, policy_version 806884 (0.0008) [2023-12-27 11:12:52,633][20381] Updated weights for policy 1, policy_version 806894 (0.0010) [2023-12-27 11:12:52,681][20381] Updated weights for policy 1, policy_version 806904 (0.0008) [2023-12-27 11:12:52,761][20347] Updated weights for policy 0, policy_version 807337 (0.0007) [2023-12-27 11:12:52,817][20347] Updated weights for policy 0, policy_version 807347 (0.0007) [2023-12-27 11:12:52,867][20347] Updated weights for policy 0, policy_version 807357 (0.0007) [2023-12-27 11:12:53,413][20381] Updated weights for policy 1, policy_version 806914 (0.0011) [2023-12-27 11:12:53,464][20381] Updated weights for policy 1, policy_version 806924 (0.0010) [2023-12-27 11:12:53,526][20381] Updated weights for policy 1, policy_version 806934 (0.0011) [2023-12-27 11:12:53,550][20347] Updated weights for policy 0, policy_version 807367 (0.0006) [2023-12-27 11:12:53,600][20347] Updated weights for policy 0, policy_version 807377 (0.0007) [2023-12-27 11:12:53,644][20347] Updated weights for policy 0, policy_version 807387 (0.0008) [2023-12-27 11:12:54,197][20381] Updated weights for policy 1, policy_version 806944 (0.0008) [2023-12-27 11:12:54,252][20381] Updated weights for policy 1, policy_version 806954 (0.0008) [2023-12-27 11:12:54,316][20381] Updated weights for policy 1, policy_version 806964 (0.0006) [2023-12-27 11:12:54,360][20347] Updated weights for policy 0, policy_version 807397 (0.0007) [2023-12-27 11:12:54,421][20347] Updated weights for policy 0, policy_version 807407 (0.0008) [2023-12-27 11:12:54,481][20347] Updated weights for policy 0, policy_version 807417 (0.0007) [2023-12-27 11:12:54,970][20381] Updated weights for policy 1, policy_version 806974 (0.0009) [2023-12-27 11:12:55,026][20381] Updated weights for policy 1, policy_version 806984 (0.0011) [2023-12-27 11:12:55,084][20381] Updated weights for policy 1, policy_version 806994 (0.0010) [2023-12-27 11:12:55,182][20347] Updated weights for policy 0, policy_version 807427 (0.0008) [2023-12-27 11:12:55,232][20347] Updated weights for policy 0, policy_version 807437 (0.0008) [2023-12-27 11:12:55,283][20347] Updated weights for policy 0, policy_version 807447 (0.0007) [2023-12-27 11:12:55,777][20381] Updated weights for policy 1, policy_version 807004 (0.0011) [2023-12-27 11:12:55,826][20381] Updated weights for policy 1, policy_version 807014 (0.0010) [2023-12-27 11:12:55,873][20381] Updated weights for policy 1, policy_version 807024 (0.0007) [2023-12-27 11:12:55,970][20347] Updated weights for policy 0, policy_version 807457 (0.0007) [2023-12-27 11:12:56,023][20347] Updated weights for policy 0, policy_version 807467 (0.0008) [2023-12-27 11:12:56,077][20347] Updated weights for policy 0, policy_version 807477 (0.0008) [2023-12-27 11:12:56,135][20347] Updated weights for policy 0, policy_version 807487 (0.0006) [2023-12-27 11:12:56,514][20381] Updated weights for policy 1, policy_version 807034 (0.0005) [2023-12-27 11:12:56,559][20381] Updated weights for policy 1, policy_version 807044 (0.0005) [2023-12-27 11:12:56,612][20381] Updated weights for policy 1, policy_version 807054 (0.0007) [2023-12-27 11:12:56,662][20381] Updated weights for policy 1, policy_version 807064 (0.0010) [2023-12-27 11:12:56,791][20347] Updated weights for policy 0, policy_version 807497 (0.0010) [2023-12-27 11:12:56,846][20347] Updated weights for policy 0, policy_version 807507 (0.0011) [2023-12-27 11:12:56,908][20347] Updated weights for policy 0, policy_version 807517 (0.0005) [2023-12-27 11:12:57,298][20381] Updated weights for policy 1, policy_version 807074 (0.0011) [2023-12-27 11:12:57,347][20381] Updated weights for policy 1, policy_version 807084 (0.0010) [2023-12-27 11:12:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 413384704. Throughput: 0: 10149.7, 1: 10420.3. Samples: 413377640. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:12:57,363][19170] Avg episode reward: [(0, '882.640'), (1, '888.580')] [2023-12-27 11:12:57,399][20381] Updated weights for policy 1, policy_version 807094 (0.0010) [2023-12-27 11:12:57,553][20347] Updated weights for policy 0, policy_version 807527 (0.0005) [2023-12-27 11:12:57,604][20347] Updated weights for policy 0, policy_version 807537 (0.0005) [2023-12-27 11:12:57,657][20347] Updated weights for policy 0, policy_version 807547 (0.0005) [2023-12-27 11:12:58,031][20381] Updated weights for policy 1, policy_version 807104 (0.0010) [2023-12-27 11:12:58,091][20381] Updated weights for policy 1, policy_version 807114 (0.0010) [2023-12-27 11:12:58,149][20381] Updated weights for policy 1, policy_version 807124 (0.0011) [2023-12-27 11:12:58,168][20347] Updated weights for policy 0, policy_version 807557 (0.0005) [2023-12-27 11:12:58,229][20347] Updated weights for policy 0, policy_version 807567 (0.0007) [2023-12-27 11:12:58,287][20347] Updated weights for policy 0, policy_version 807577 (0.0007) [2023-12-27 11:12:58,895][20381] Updated weights for policy 1, policy_version 807134 (0.0009) [2023-12-27 11:12:58,955][20381] Updated weights for policy 1, policy_version 807144 (0.0007) [2023-12-27 11:12:59,003][20381] Updated weights for policy 1, policy_version 807154 (0.0006) [2023-12-27 11:12:59,008][20347] Updated weights for policy 0, policy_version 807587 (0.0008) [2023-12-27 11:12:59,055][20347] Updated weights for policy 0, policy_version 807597 (0.0009) [2023-12-27 11:12:59,108][20347] Updated weights for policy 0, policy_version 807607 (0.0007) [2023-12-27 11:12:59,698][20381] Updated weights for policy 1, policy_version 807164 (0.0006) [2023-12-27 11:12:59,758][20381] Updated weights for policy 1, policy_version 807174 (0.0009) [2023-12-27 11:12:59,771][20292] KL-divergence is very high: 187.7117 [2023-12-27 11:12:59,802][20347] Updated weights for policy 0, policy_version 807617 (0.0006) [2023-12-27 11:12:59,811][20381] Updated weights for policy 1, policy_version 807184 (0.0010) [2023-12-27 11:12:59,865][20347] Updated weights for policy 0, policy_version 807627 (0.0010) [2023-12-27 11:12:59,934][20347] Updated weights for policy 0, policy_version 807637 (0.0011) [2023-12-27 11:12:59,995][20347] Updated weights for policy 0, policy_version 807647 (0.0011) [2023-12-27 11:13:00,486][20381] Updated weights for policy 1, policy_version 807194 (0.0007) [2023-12-27 11:13:00,545][20381] Updated weights for policy 1, policy_version 807204 (0.0006) [2023-12-27 11:13:00,594][20381] Updated weights for policy 1, policy_version 807214 (0.0006) [2023-12-27 11:13:00,653][20381] Updated weights for policy 1, policy_version 807224 (0.0009) [2023-12-27 11:13:00,675][20347] Updated weights for policy 0, policy_version 807657 (0.0010) [2023-12-27 11:13:00,733][20347] Updated weights for policy 0, policy_version 807667 (0.0010) [2023-12-27 11:13:00,791][20347] Updated weights for policy 0, policy_version 807677 (0.0010) [2023-12-27 11:13:01,277][20381] Updated weights for policy 1, policy_version 807234 (0.0010) [2023-12-27 11:13:01,336][20381] Updated weights for policy 1, policy_version 807244 (0.0011) [2023-12-27 11:13:01,405][20381] Updated weights for policy 1, policy_version 807254 (0.0009) [2023-12-27 11:13:01,485][20347] Updated weights for policy 0, policy_version 807687 (0.0007) [2023-12-27 11:13:01,543][20347] Updated weights for policy 0, policy_version 807697 (0.0008) [2023-12-27 11:13:01,594][20347] Updated weights for policy 0, policy_version 807707 (0.0008) [2023-12-27 11:13:02,026][20381] Updated weights for policy 1, policy_version 807264 (0.0007) [2023-12-27 11:13:02,084][20381] Updated weights for policy 1, policy_version 807274 (0.0007) [2023-12-27 11:13:02,142][20381] Updated weights for policy 1, policy_version 807284 (0.0007) [2023-12-27 11:13:02,190][20347] Updated weights for policy 0, policy_version 807717 (0.0007) [2023-12-27 11:13:02,259][20347] Updated weights for policy 0, policy_version 807727 (0.0006) [2023-12-27 11:13:02,320][20347] Updated weights for policy 0, policy_version 807737 (0.0007) [2023-12-27 11:13:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 413499392. Throughput: 0: 10169.0, 1: 10430.5. Samples: 413503560. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:02,363][19170] Avg episode reward: [(0, '892.870'), (1, '949.880')] [2023-12-27 11:13:02,831][20381] Updated weights for policy 1, policy_version 807294 (0.0008) [2023-12-27 11:13:02,879][20381] Updated weights for policy 1, policy_version 807304 (0.0007) [2023-12-27 11:13:02,926][20381] Updated weights for policy 1, policy_version 807314 (0.0007) [2023-12-27 11:13:02,932][20292] KL-divergence is very high: 228.4983 [2023-12-27 11:13:02,946][20292] KL-divergence is very high: 239.1291 [2023-12-27 11:13:02,971][20347] Updated weights for policy 0, policy_version 807747 (0.0007) [2023-12-27 11:13:03,030][20347] Updated weights for policy 0, policy_version 807757 (0.0006) [2023-12-27 11:13:03,097][20347] Updated weights for policy 0, policy_version 807767 (0.0005) [2023-12-27 11:13:03,540][20381] Updated weights for policy 1, policy_version 807324 (0.0008) [2023-12-27 11:13:03,595][20381] Updated weights for policy 1, policy_version 807334 (0.0010) [2023-12-27 11:13:03,652][20381] Updated weights for policy 1, policy_version 807344 (0.0010) [2023-12-27 11:13:03,746][20347] Updated weights for policy 0, policy_version 807777 (0.0006) [2023-12-27 11:13:03,806][20347] Updated weights for policy 0, policy_version 807787 (0.0007) [2023-12-27 11:13:03,866][20347] Updated weights for policy 0, policy_version 807797 (0.0008) [2023-12-27 11:13:03,925][20347] Updated weights for policy 0, policy_version 807807 (0.0006) [2023-12-27 11:13:04,304][20381] Updated weights for policy 1, policy_version 807354 (0.0009) [2023-12-27 11:13:04,356][20381] Updated weights for policy 1, policy_version 807364 (0.0010) [2023-12-27 11:13:04,414][20381] Updated weights for policy 1, policy_version 807374 (0.0010) [2023-12-27 11:13:04,474][20381] Updated weights for policy 1, policy_version 807384 (0.0011) [2023-12-27 11:13:04,589][20347] Updated weights for policy 0, policy_version 807817 (0.0005) [2023-12-27 11:13:04,642][20347] Updated weights for policy 0, policy_version 807827 (0.0008) [2023-12-27 11:13:04,695][20347] Updated weights for policy 0, policy_version 807837 (0.0007) [2023-12-27 11:13:05,101][20381] Updated weights for policy 1, policy_version 807394 (0.0007) [2023-12-27 11:13:05,155][20381] Updated weights for policy 1, policy_version 807404 (0.0010) [2023-12-27 11:13:05,204][20381] Updated weights for policy 1, policy_version 807414 (0.0010) [2023-12-27 11:13:05,329][20347] Updated weights for policy 0, policy_version 807847 (0.0008) [2023-12-27 11:13:05,379][20347] Updated weights for policy 0, policy_version 807857 (0.0007) [2023-12-27 11:13:05,436][20347] Updated weights for policy 0, policy_version 807867 (0.0006) [2023-12-27 11:13:05,876][20381] Updated weights for policy 1, policy_version 807424 (0.0009) [2023-12-27 11:13:05,938][20381] Updated weights for policy 1, policy_version 807434 (0.0006) [2023-12-27 11:13:05,988][20381] Updated weights for policy 1, policy_version 807444 (0.0010) [2023-12-27 11:13:06,020][20347] Updated weights for policy 0, policy_version 807877 (0.0006) [2023-12-27 11:13:06,081][20347] Updated weights for policy 0, policy_version 807887 (0.0006) [2023-12-27 11:13:06,151][20347] Updated weights for policy 0, policy_version 807897 (0.0008) [2023-12-27 11:13:06,704][20381] Updated weights for policy 1, policy_version 807454 (0.0007) [2023-12-27 11:13:06,760][20381] Updated weights for policy 1, policy_version 807464 (0.0007) [2023-12-27 11:13:06,799][20347] Updated weights for policy 0, policy_version 807907 (0.0006) [2023-12-27 11:13:06,817][20381] Updated weights for policy 1, policy_version 807474 (0.0006) [2023-12-27 11:13:06,853][20347] Updated weights for policy 0, policy_version 807917 (0.0006) [2023-12-27 11:13:06,912][20347] Updated weights for policy 0, policy_version 807927 (0.0009) [2023-12-27 11:13:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 413597696. Throughput: 0: 10210.0, 1: 10448.7. Samples: 413567448. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:07,363][19170] Avg episode reward: [(0, '931.740'), (1, '923.070')] [2023-12-27 11:13:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000807480_206741504.pth... [2023-12-27 11:13:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000807936_206856192.pth... [2023-12-27 11:13:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000806752_206553088.pth [2023-12-27 11:13:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000806264_206430208.pth [2023-12-27 11:13:07,413][20381] Updated weights for policy 1, policy_version 807484 (0.0007) [2023-12-27 11:13:07,479][20381] Updated weights for policy 1, policy_version 807494 (0.0008) [2023-12-27 11:13:07,541][20381] Updated weights for policy 1, policy_version 807504 (0.0008) [2023-12-27 11:13:07,563][20347] Updated weights for policy 0, policy_version 807937 (0.0009) [2023-12-27 11:13:07,626][20347] Updated weights for policy 0, policy_version 807947 (0.0007) [2023-12-27 11:13:07,685][20347] Updated weights for policy 0, policy_version 807957 (0.0007) [2023-12-27 11:13:07,747][20347] Updated weights for policy 0, policy_version 807967 (0.0007) [2023-12-27 11:13:08,249][20381] Updated weights for policy 1, policy_version 807514 (0.0007) [2023-12-27 11:13:08,311][20381] Updated weights for policy 1, policy_version 807524 (0.0009) [2023-12-27 11:13:08,380][20381] Updated weights for policy 1, policy_version 807534 (0.0007) [2023-12-27 11:13:08,440][20381] Updated weights for policy 1, policy_version 807544 (0.0008) [2023-12-27 11:13:08,503][20347] Updated weights for policy 0, policy_version 807977 (0.0008) [2023-12-27 11:13:08,567][20347] Updated weights for policy 0, policy_version 807987 (0.0008) [2023-12-27 11:13:08,632][20347] Updated weights for policy 0, policy_version 807997 (0.0007) [2023-12-27 11:13:09,193][20381] Updated weights for policy 1, policy_version 807554 (0.0006) [2023-12-27 11:13:09,274][20381] Updated weights for policy 1, policy_version 807564 (0.0008) [2023-12-27 11:13:09,342][20381] Updated weights for policy 1, policy_version 807574 (0.0008) [2023-12-27 11:13:09,360][20347] Updated weights for policy 0, policy_version 808007 (0.0008) [2023-12-27 11:13:09,430][20347] Updated weights for policy 0, policy_version 808017 (0.0008) [2023-12-27 11:13:09,483][20347] Updated weights for policy 0, policy_version 808027 (0.0006) [2023-12-27 11:13:10,061][20381] Updated weights for policy 1, policy_version 807584 (0.0008) [2023-12-27 11:13:10,124][20381] Updated weights for policy 1, policy_version 807594 (0.0008) [2023-12-27 11:13:10,154][20347] Updated weights for policy 0, policy_version 808037 (0.0007) [2023-12-27 11:13:10,186][20381] Updated weights for policy 1, policy_version 807604 (0.0008) [2023-12-27 11:13:10,224][20347] Updated weights for policy 0, policy_version 808047 (0.0006) [2023-12-27 11:13:10,293][20347] Updated weights for policy 0, policy_version 808057 (0.0007) [2023-12-27 11:13:10,912][20381] Updated weights for policy 1, policy_version 807614 (0.0009) [2023-12-27 11:13:10,974][20347] Updated weights for policy 0, policy_version 808067 (0.0006) [2023-12-27 11:13:10,983][20381] Updated weights for policy 1, policy_version 807624 (0.0008) [2023-12-27 11:13:11,031][20347] Updated weights for policy 0, policy_version 808077 (0.0006) [2023-12-27 11:13:11,050][20381] Updated weights for policy 1, policy_version 807634 (0.0007) [2023-12-27 11:13:11,088][20347] Updated weights for policy 0, policy_version 808087 (0.0007) [2023-12-27 11:13:11,823][20381] Updated weights for policy 1, policy_version 807644 (0.0007) [2023-12-27 11:13:11,886][20381] Updated weights for policy 1, policy_version 807654 (0.0008) [2023-12-27 11:13:11,894][20347] Updated weights for policy 0, policy_version 808097 (0.0007) [2023-12-27 11:13:11,952][20381] Updated weights for policy 1, policy_version 807664 (0.0007) [2023-12-27 11:13:11,960][20347] Updated weights for policy 0, policy_version 808107 (0.0008) [2023-12-27 11:13:12,026][20347] Updated weights for policy 0, policy_version 808117 (0.0008) [2023-12-27 11:13:12,096][20347] Updated weights for policy 0, policy_version 808127 (0.0007) [2023-12-27 11:13:12,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 413696000. Throughput: 0: 10184.3, 1: 10324.5. Samples: 413686088. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:12,363][19170] Avg episode reward: [(0, '912.130'), (1, '923.100')] [2023-12-27 11:13:12,654][20381] Updated weights for policy 1, policy_version 807674 (0.0008) [2023-12-27 11:13:12,716][20381] Updated weights for policy 1, policy_version 807684 (0.0007) [2023-12-27 11:13:12,776][20381] Updated weights for policy 1, policy_version 807694 (0.0007) [2023-12-27 11:13:12,842][20381] Updated weights for policy 1, policy_version 807704 (0.0009) [2023-12-27 11:13:12,860][20347] Updated weights for policy 0, policy_version 808137 (0.0010) [2023-12-27 11:13:12,911][20347] Updated weights for policy 0, policy_version 808147 (0.0007) [2023-12-27 11:13:12,971][20347] Updated weights for policy 0, policy_version 808157 (0.0006) [2023-12-27 11:13:13,601][20347] Updated weights for policy 0, policy_version 808167 (0.0006) [2023-12-27 11:13:13,603][20381] Updated weights for policy 1, policy_version 807714 (0.0008) [2023-12-27 11:13:13,662][20381] Updated weights for policy 1, policy_version 807724 (0.0006) [2023-12-27 11:13:13,663][20347] Updated weights for policy 0, policy_version 808177 (0.0008) [2023-12-27 11:13:13,723][20381] Updated weights for policy 1, policy_version 807734 (0.0006) [2023-12-27 11:13:13,724][20347] Updated weights for policy 0, policy_version 808187 (0.0009) [2023-12-27 11:13:14,414][20381] Updated weights for policy 1, policy_version 807744 (0.0008) [2023-12-27 11:13:14,431][20347] Updated weights for policy 0, policy_version 808197 (0.0008) [2023-12-27 11:13:14,474][20381] Updated weights for policy 1, policy_version 807754 (0.0007) [2023-12-27 11:13:14,489][20347] Updated weights for policy 0, policy_version 808207 (0.0008) [2023-12-27 11:13:14,534][20381] Updated weights for policy 1, policy_version 807764 (0.0007) [2023-12-27 11:13:14,550][20347] Updated weights for policy 0, policy_version 808217 (0.0008) [2023-12-27 11:13:15,221][20347] Updated weights for policy 0, policy_version 808227 (0.0008) [2023-12-27 11:13:15,274][20381] Updated weights for policy 1, policy_version 807774 (0.0007) [2023-12-27 11:13:15,285][20347] Updated weights for policy 0, policy_version 808237 (0.0007) [2023-12-27 11:13:15,334][20381] Updated weights for policy 1, policy_version 807784 (0.0008) [2023-12-27 11:13:15,352][20347] Updated weights for policy 0, policy_version 808247 (0.0011) [2023-12-27 11:13:15,367][20291] KL-divergence is very high: 167.7657 [2023-12-27 11:13:15,372][20291] KL-divergence is very high: 237.8345 [2023-12-27 11:13:15,377][20291] KL-divergence is very high: 181.8084 [2023-12-27 11:13:15,388][20291] KL-divergence is very high: 103.2476 [2023-12-27 11:13:15,392][20381] Updated weights for policy 1, policy_version 807794 (0.0007) [2023-12-27 11:13:15,973][20347] Updated weights for policy 0, policy_version 808257 (0.0008) [2023-12-27 11:13:16,000][20381] Updated weights for policy 1, policy_version 807804 (0.0006) [2023-12-27 11:13:16,030][20347] Updated weights for policy 0, policy_version 808267 (0.0010) [2023-12-27 11:13:16,054][20381] Updated weights for policy 1, policy_version 807814 (0.0006) [2023-12-27 11:13:16,096][20347] Updated weights for policy 0, policy_version 808277 (0.0009) [2023-12-27 11:13:16,116][20381] Updated weights for policy 1, policy_version 807824 (0.0008) [2023-12-27 11:13:16,149][20347] Updated weights for policy 0, policy_version 808287 (0.0008) [2023-12-27 11:13:16,763][20347] Updated weights for policy 0, policy_version 808297 (0.0008) [2023-12-27 11:13:16,803][20381] Updated weights for policy 1, policy_version 807834 (0.0007) [2023-12-27 11:13:16,817][20347] Updated weights for policy 0, policy_version 808307 (0.0010) [2023-12-27 11:13:16,855][20381] Updated weights for policy 1, policy_version 807844 (0.0006) [2023-12-27 11:13:16,872][20347] Updated weights for policy 0, policy_version 808317 (0.0009) [2023-12-27 11:13:16,911][20381] Updated weights for policy 1, policy_version 807854 (0.0005) [2023-12-27 11:13:16,978][20381] Updated weights for policy 1, policy_version 807864 (0.0006) [2023-12-27 11:13:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 413794304. Throughput: 0: 10155.1, 1: 10289.8. Samples: 413805412. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:17,363][19170] Avg episode reward: [(0, '892.700'), (1, '970.690')] [2023-12-27 11:13:17,522][20347] Updated weights for policy 0, policy_version 808327 (0.0009) [2023-12-27 11:13:17,575][20347] Updated weights for policy 0, policy_version 808337 (0.0010) [2023-12-27 11:13:17,630][20347] Updated weights for policy 0, policy_version 808347 (0.0008) [2023-12-27 11:13:17,634][20381] Updated weights for policy 1, policy_version 807874 (0.0010) [2023-12-27 11:13:17,690][20381] Updated weights for policy 1, policy_version 807884 (0.0007) [2023-12-27 11:13:17,739][20381] Updated weights for policy 1, policy_version 807894 (0.0007) [2023-12-27 11:13:18,350][20347] Updated weights for policy 0, policy_version 808357 (0.0011) [2023-12-27 11:13:18,409][20347] Updated weights for policy 0, policy_version 808367 (0.0011) [2023-12-27 11:13:18,473][20347] Updated weights for policy 0, policy_version 808377 (0.0011) [2023-12-27 11:13:18,473][20381] Updated weights for policy 1, policy_version 807904 (0.0007) [2023-12-27 11:13:18,540][20381] Updated weights for policy 1, policy_version 807914 (0.0007) [2023-12-27 11:13:18,593][20381] Updated weights for policy 1, policy_version 807924 (0.0007) [2023-12-27 11:13:19,171][20347] Updated weights for policy 0, policy_version 808387 (0.0010) [2023-12-27 11:13:19,236][20347] Updated weights for policy 0, policy_version 808397 (0.0011) [2023-12-27 11:13:19,292][20381] Updated weights for policy 1, policy_version 807934 (0.0006) [2023-12-27 11:13:19,301][20347] Updated weights for policy 0, policy_version 808407 (0.0010) [2023-12-27 11:13:19,362][20381] Updated weights for policy 1, policy_version 807944 (0.0008) [2023-12-27 11:13:19,428][20381] Updated weights for policy 1, policy_version 807954 (0.0007) [2023-12-27 11:13:20,088][20347] Updated weights for policy 0, policy_version 808417 (0.0011) [2023-12-27 11:13:20,151][20381] Updated weights for policy 1, policy_version 807964 (0.0007) [2023-12-27 11:13:20,153][20347] Updated weights for policy 0, policy_version 808427 (0.0008) [2023-12-27 11:13:20,207][20381] Updated weights for policy 1, policy_version 807974 (0.0007) [2023-12-27 11:13:20,214][20347] Updated weights for policy 0, policy_version 808437 (0.0008) [2023-12-27 11:13:20,270][20381] Updated weights for policy 1, policy_version 807984 (0.0007) [2023-12-27 11:13:20,274][20347] Updated weights for policy 0, policy_version 808447 (0.0008) [2023-12-27 11:13:20,982][20381] Updated weights for policy 1, policy_version 807994 (0.0009) [2023-12-27 11:13:21,046][20347] Updated weights for policy 0, policy_version 808457 (0.0008) [2023-12-27 11:13:21,046][20381] Updated weights for policy 1, policy_version 808004 (0.0009) [2023-12-27 11:13:21,108][20347] Updated weights for policy 0, policy_version 808467 (0.0007) [2023-12-27 11:13:21,111][20381] Updated weights for policy 1, policy_version 808014 (0.0007) [2023-12-27 11:13:21,173][20381] Updated weights for policy 1, policy_version 808024 (0.0009) [2023-12-27 11:13:21,174][20347] Updated weights for policy 0, policy_version 808477 (0.0008) [2023-12-27 11:13:21,920][20347] Updated weights for policy 0, policy_version 808487 (0.0008) [2023-12-27 11:13:21,963][20381] Updated weights for policy 1, policy_version 808034 (0.0008) [2023-12-27 11:13:21,980][20347] Updated weights for policy 0, policy_version 808497 (0.0008) [2023-12-27 11:13:22,031][20381] Updated weights for policy 1, policy_version 808044 (0.0008) [2023-12-27 11:13:22,042][20347] Updated weights for policy 0, policy_version 808507 (0.0007) [2023-12-27 11:13:22,091][20381] Updated weights for policy 1, policy_version 808054 (0.0008) [2023-12-27 11:13:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 413892608. Throughput: 0: 10108.0, 1: 10209.6. Samples: 413863096. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:22,363][19170] Avg episode reward: [(0, '941.490'), (1, '1000.000')] [2023-12-27 11:13:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000808056_206888960.pth... [2023-12-27 11:13:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000808512_207003648.pth... [2023-12-27 11:13:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000806872_206585856.pth [2023-12-27 11:13:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000807328_206700544.pth [2023-12-27 11:13:22,811][20347] Updated weights for policy 0, policy_version 808517 (0.0007) [2023-12-27 11:13:22,879][20347] Updated weights for policy 0, policy_version 808527 (0.0007) [2023-12-27 11:13:22,881][20381] Updated weights for policy 1, policy_version 808064 (0.0010) [2023-12-27 11:13:22,946][20381] Updated weights for policy 1, policy_version 808074 (0.0010) [2023-12-27 11:13:22,946][20347] Updated weights for policy 0, policy_version 808537 (0.0007) [2023-12-27 11:13:23,007][20381] Updated weights for policy 1, policy_version 808084 (0.0008) [2023-12-27 11:13:23,684][20381] Updated weights for policy 1, policy_version 808094 (0.0009) [2023-12-27 11:13:23,710][20347] Updated weights for policy 0, policy_version 808547 (0.0008) [2023-12-27 11:13:23,744][20381] Updated weights for policy 1, policy_version 808104 (0.0011) [2023-12-27 11:13:23,769][20347] Updated weights for policy 0, policy_version 808557 (0.0007) [2023-12-27 11:13:23,795][20381] Updated weights for policy 1, policy_version 808114 (0.0010) [2023-12-27 11:13:23,824][20347] Updated weights for policy 0, policy_version 808567 (0.0006) [2023-12-27 11:13:24,492][20347] Updated weights for policy 0, policy_version 808577 (0.0006) [2023-12-27 11:13:24,513][20381] Updated weights for policy 1, policy_version 808124 (0.0010) [2023-12-27 11:13:24,558][20347] Updated weights for policy 0, policy_version 808587 (0.0006) [2023-12-27 11:13:24,578][20381] Updated weights for policy 1, policy_version 808134 (0.0008) [2023-12-27 11:13:24,614][20347] Updated weights for policy 0, policy_version 808597 (0.0005) [2023-12-27 11:13:24,638][20381] Updated weights for policy 1, policy_version 808144 (0.0008) [2023-12-27 11:13:24,668][20347] Updated weights for policy 0, policy_version 808607 (0.0006) [2023-12-27 11:13:25,281][20381] Updated weights for policy 1, policy_version 808154 (0.0009) [2023-12-27 11:13:25,339][20381] Updated weights for policy 1, policy_version 808164 (0.0011) [2023-12-27 11:13:25,382][20347] Updated weights for policy 0, policy_version 808617 (0.0008) [2023-12-27 11:13:25,398][20381] Updated weights for policy 1, policy_version 808174 (0.0009) [2023-12-27 11:13:25,398][20292] KL-divergence is very high: 510.8534 [2023-12-27 11:13:25,444][20347] Updated weights for policy 0, policy_version 808627 (0.0008) [2023-12-27 11:13:25,448][20292] KL-divergence is very high: 193.0508 [2023-12-27 11:13:25,461][20381] Updated weights for policy 1, policy_version 808184 (0.0007) [2023-12-27 11:13:25,493][20347] Updated weights for policy 0, policy_version 808637 (0.0006) [2023-12-27 11:13:26,139][20381] Updated weights for policy 1, policy_version 808194 (0.0008) [2023-12-27 11:13:26,168][20347] Updated weights for policy 0, policy_version 808647 (0.0006) [2023-12-27 11:13:26,201][20381] Updated weights for policy 1, policy_version 808204 (0.0007) [2023-12-27 11:13:26,224][20347] Updated weights for policy 0, policy_version 808657 (0.0006) [2023-12-27 11:13:26,267][20381] Updated weights for policy 1, policy_version 808214 (0.0006) [2023-12-27 11:13:26,280][20347] Updated weights for policy 0, policy_version 808667 (0.0007) [2023-12-27 11:13:26,885][20381] Updated weights for policy 1, policy_version 808224 (0.0006) [2023-12-27 11:13:26,941][20381] Updated weights for policy 1, policy_version 808234 (0.0008) [2023-12-27 11:13:26,999][20381] Updated weights for policy 1, policy_version 808244 (0.0007) [2023-12-27 11:13:27,001][20347] Updated weights for policy 0, policy_version 808677 (0.0007) [2023-12-27 11:13:27,056][20347] Updated weights for policy 0, policy_version 808687 (0.0007) [2023-12-27 11:13:27,113][20347] Updated weights for policy 0, policy_version 808697 (0.0008) [2023-12-27 11:13:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 413990912. Throughput: 0: 10064.5, 1: 10071.8. Samples: 413977388. Policy #0 lag: (min: 22.0, avg: 31.7, max: 32.0) [2023-12-27 11:13:27,363][19170] Avg episode reward: [(0, '961.000'), (1, '990.230')] [2023-12-27 11:13:27,692][20381] Updated weights for policy 1, policy_version 808254 (0.0008) [2023-12-27 11:13:27,747][20381] Updated weights for policy 1, policy_version 808264 (0.0007) [2023-12-27 11:13:27,806][20381] Updated weights for policy 1, policy_version 808274 (0.0008) [2023-12-27 11:13:27,808][20347] Updated weights for policy 0, policy_version 808707 (0.0007) [2023-12-27 11:13:27,862][20347] Updated weights for policy 0, policy_version 808717 (0.0005) [2023-12-27 11:13:27,930][20347] Updated weights for policy 0, policy_version 808727 (0.0007) [2023-12-27 11:13:28,485][20381] Updated weights for policy 1, policy_version 808284 (0.0007) [2023-12-27 11:13:28,543][20381] Updated weights for policy 1, policy_version 808294 (0.0009) [2023-12-27 11:13:28,604][20381] Updated weights for policy 1, policy_version 808304 (0.0008) [2023-12-27 11:13:28,607][20347] Updated weights for policy 0, policy_version 808737 (0.0008) [2023-12-27 11:13:28,666][20347] Updated weights for policy 0, policy_version 808747 (0.0007) [2023-12-27 11:13:28,727][20347] Updated weights for policy 0, policy_version 808757 (0.0008) [2023-12-27 11:13:28,787][20347] Updated weights for policy 0, policy_version 808767 (0.0007) [2023-12-27 11:13:29,325][20381] Updated weights for policy 1, policy_version 808314 (0.0010) [2023-12-27 11:13:29,392][20381] Updated weights for policy 1, policy_version 808324 (0.0006) [2023-12-27 11:13:29,451][20381] Updated weights for policy 1, policy_version 808334 (0.0007) [2023-12-27 11:13:29,502][20381] Updated weights for policy 1, policy_version 808344 (0.0007) [2023-12-27 11:13:29,515][20347] Updated weights for policy 0, policy_version 808777 (0.0008) [2023-12-27 11:13:29,576][20347] Updated weights for policy 0, policy_version 808787 (0.0008) [2023-12-27 11:13:29,638][20347] Updated weights for policy 0, policy_version 808797 (0.0009) [2023-12-27 11:13:30,203][20381] Updated weights for policy 1, policy_version 808354 (0.0007) [2023-12-27 11:13:30,260][20381] Updated weights for policy 1, policy_version 808364 (0.0007) [2023-12-27 11:13:30,326][20381] Updated weights for policy 1, policy_version 808374 (0.0007) [2023-12-27 11:13:30,411][20347] Updated weights for policy 0, policy_version 808807 (0.0007) [2023-12-27 11:13:30,462][20347] Updated weights for policy 0, policy_version 808817 (0.0006) [2023-12-27 11:13:30,509][20347] Updated weights for policy 0, policy_version 808827 (0.0005) [2023-12-27 11:13:31,014][20381] Updated weights for policy 1, policy_version 808384 (0.0006) [2023-12-27 11:13:31,081][20381] Updated weights for policy 1, policy_version 808394 (0.0011) [2023-12-27 11:13:31,140][20347] Updated weights for policy 0, policy_version 808837 (0.0008) [2023-12-27 11:13:31,141][20381] Updated weights for policy 1, policy_version 808404 (0.0010) [2023-12-27 11:13:31,205][20347] Updated weights for policy 0, policy_version 808847 (0.0010) [2023-12-27 11:13:31,266][20347] Updated weights for policy 0, policy_version 808857 (0.0011) [2023-12-27 11:13:31,876][20381] Updated weights for policy 1, policy_version 808414 (0.0008) [2023-12-27 11:13:31,935][20381] Updated weights for policy 1, policy_version 808424 (0.0008) [2023-12-27 11:13:31,962][20347] Updated weights for policy 0, policy_version 808867 (0.0009) [2023-12-27 11:13:31,990][20381] Updated weights for policy 1, policy_version 808434 (0.0008) [2023-12-27 11:13:32,023][20347] Updated weights for policy 0, policy_version 808877 (0.0006) [2023-12-27 11:13:32,084][20347] Updated weights for policy 0, policy_version 808887 (0.0007) [2023-12-27 11:13:32,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20206.8, 300 sec: 20466.1). Total num frames: 414089216. Throughput: 0: 10011.0, 1: 10007.1. Samples: 414096280. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:32,363][19170] Avg episode reward: [(0, '922.300'), (1, '970.820')] [2023-12-27 11:13:32,660][20381] Updated weights for policy 1, policy_version 808444 (0.0007) [2023-12-27 11:13:32,719][20381] Updated weights for policy 1, policy_version 808454 (0.0007) [2023-12-27 11:13:32,778][20381] Updated weights for policy 1, policy_version 808464 (0.0009) [2023-12-27 11:13:32,797][20347] Updated weights for policy 0, policy_version 808897 (0.0010) [2023-12-27 11:13:32,860][20347] Updated weights for policy 0, policy_version 808907 (0.0006) [2023-12-27 11:13:32,922][20347] Updated weights for policy 0, policy_version 808917 (0.0006) [2023-12-27 11:13:32,988][20347] Updated weights for policy 0, policy_version 808927 (0.0006) [2023-12-27 11:13:33,517][20381] Updated weights for policy 1, policy_version 808474 (0.0008) [2023-12-27 11:13:33,570][20381] Updated weights for policy 1, policy_version 808484 (0.0007) [2023-12-27 11:13:33,607][20347] Updated weights for policy 0, policy_version 808937 (0.0007) [2023-12-27 11:13:33,617][20381] Updated weights for policy 1, policy_version 808494 (0.0007) [2023-12-27 11:13:33,665][20347] Updated weights for policy 0, policy_version 808947 (0.0007) [2023-12-27 11:13:33,678][20381] Updated weights for policy 1, policy_version 808504 (0.0008) [2023-12-27 11:13:33,722][20347] Updated weights for policy 0, policy_version 808957 (0.0007) [2023-12-27 11:13:34,434][20347] Updated weights for policy 0, policy_version 808967 (0.0008) [2023-12-27 11:13:34,465][20381] Updated weights for policy 1, policy_version 808514 (0.0006) [2023-12-27 11:13:34,498][20347] Updated weights for policy 0, policy_version 808977 (0.0008) [2023-12-27 11:13:34,531][20381] Updated weights for policy 1, policy_version 808524 (0.0006) [2023-12-27 11:13:34,561][20347] Updated weights for policy 0, policy_version 808987 (0.0008) [2023-12-27 11:13:34,599][20381] Updated weights for policy 1, policy_version 808534 (0.0007) [2023-12-27 11:13:35,240][20381] Updated weights for policy 1, policy_version 808544 (0.0006) [2023-12-27 11:13:35,274][20347] Updated weights for policy 0, policy_version 808997 (0.0007) [2023-12-27 11:13:35,298][20381] Updated weights for policy 1, policy_version 808554 (0.0009) [2023-12-27 11:13:35,336][20347] Updated weights for policy 0, policy_version 809007 (0.0009) [2023-12-27 11:13:35,357][20381] Updated weights for policy 1, policy_version 808564 (0.0010) [2023-12-27 11:13:35,391][20347] Updated weights for policy 0, policy_version 809017 (0.0007) [2023-12-27 11:13:36,066][20381] Updated weights for policy 1, policy_version 808574 (0.0006) [2023-12-27 11:13:36,068][20347] Updated weights for policy 0, policy_version 809027 (0.0008) [2023-12-27 11:13:36,130][20381] Updated weights for policy 1, policy_version 808584 (0.0008) [2023-12-27 11:13:36,141][20347] Updated weights for policy 0, policy_version 809037 (0.0007) [2023-12-27 11:13:36,196][20381] Updated weights for policy 1, policy_version 808594 (0.0009) [2023-12-27 11:13:36,199][20347] Updated weights for policy 0, policy_version 809047 (0.0007) [2023-12-27 11:13:36,879][20381] Updated weights for policy 1, policy_version 808604 (0.0006) [2023-12-27 11:13:36,948][20381] Updated weights for policy 1, policy_version 808614 (0.0006) [2023-12-27 11:13:36,952][20347] Updated weights for policy 0, policy_version 809057 (0.0007) [2023-12-27 11:13:37,001][20347] Updated weights for policy 0, policy_version 809067 (0.0007) [2023-12-27 11:13:37,015][20381] Updated weights for policy 1, policy_version 808624 (0.0009) [2023-12-27 11:13:37,056][20347] Updated weights for policy 0, policy_version 809077 (0.0006) [2023-12-27 11:13:37,112][20347] Updated weights for policy 0, policy_version 809087 (0.0007) [2023-12-27 11:13:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 414187520. Throughput: 0: 10024.0, 1: 9990.7. Samples: 414156192. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:37,363][19170] Avg episode reward: [(0, '941.640'), (1, '961.120')] [2023-12-27 11:13:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000808632_207036416.pth... [2023-12-27 11:13:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000809088_207151104.pth... [2023-12-27 11:13:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000807480_206741504.pth [2023-12-27 11:13:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000807936_206856192.pth [2023-12-27 11:13:37,705][20381] Updated weights for policy 1, policy_version 808634 (0.0007) [2023-12-27 11:13:37,774][20381] Updated weights for policy 1, policy_version 808644 (0.0009) [2023-12-27 11:13:37,840][20381] Updated weights for policy 1, policy_version 808654 (0.0007) [2023-12-27 11:13:37,862][20347] Updated weights for policy 0, policy_version 809097 (0.0007) [2023-12-27 11:13:37,906][20381] Updated weights for policy 1, policy_version 808664 (0.0007) [2023-12-27 11:13:37,931][20347] Updated weights for policy 0, policy_version 809107 (0.0008) [2023-12-27 11:13:37,993][20347] Updated weights for policy 0, policy_version 809117 (0.0008) [2023-12-27 11:13:38,587][20381] Updated weights for policy 1, policy_version 808674 (0.0008) [2023-12-27 11:13:38,647][20381] Updated weights for policy 1, policy_version 808684 (0.0007) [2023-12-27 11:13:38,685][20347] Updated weights for policy 0, policy_version 809127 (0.0008) [2023-12-27 11:13:38,711][20381] Updated weights for policy 1, policy_version 808694 (0.0007) [2023-12-27 11:13:38,739][20347] Updated weights for policy 0, policy_version 809137 (0.0006) [2023-12-27 11:13:38,804][20347] Updated weights for policy 0, policy_version 809147 (0.0006) [2023-12-27 11:13:39,459][20381] Updated weights for policy 1, policy_version 808704 (0.0008) [2023-12-27 11:13:39,511][20347] Updated weights for policy 0, policy_version 809157 (0.0009) [2023-12-27 11:13:39,521][20381] Updated weights for policy 1, policy_version 808714 (0.0007) [2023-12-27 11:13:39,575][20347] Updated weights for policy 0, policy_version 809167 (0.0007) [2023-12-27 11:13:39,592][20381] Updated weights for policy 1, policy_version 808724 (0.0007) [2023-12-27 11:13:39,641][20347] Updated weights for policy 0, policy_version 809177 (0.0008) [2023-12-27 11:13:40,374][20381] Updated weights for policy 1, policy_version 808734 (0.0009) [2023-12-27 11:13:40,434][20347] Updated weights for policy 0, policy_version 809187 (0.0008) [2023-12-27 11:13:40,435][20381] Updated weights for policy 1, policy_version 808744 (0.0010) [2023-12-27 11:13:40,494][20381] Updated weights for policy 1, policy_version 808754 (0.0009) [2023-12-27 11:13:40,497][20347] Updated weights for policy 0, policy_version 809197 (0.0010) [2023-12-27 11:13:40,564][20347] Updated weights for policy 0, policy_version 809207 (0.0010) [2023-12-27 11:13:41,217][20381] Updated weights for policy 1, policy_version 808764 (0.0008) [2023-12-27 11:13:41,266][20347] Updated weights for policy 0, policy_version 809217 (0.0010) [2023-12-27 11:13:41,289][20381] Updated weights for policy 1, policy_version 808774 (0.0008) [2023-12-27 11:13:41,330][20347] Updated weights for policy 0, policy_version 809227 (0.0009) [2023-12-27 11:13:41,351][20381] Updated weights for policy 1, policy_version 808784 (0.0009) [2023-12-27 11:13:41,398][20347] Updated weights for policy 0, policy_version 809237 (0.0009) [2023-12-27 11:13:41,465][20347] Updated weights for policy 0, policy_version 809247 (0.0011) [2023-12-27 11:13:42,173][20381] Updated weights for policy 1, policy_version 808794 (0.0009) [2023-12-27 11:13:42,224][20347] Updated weights for policy 0, policy_version 809257 (0.0006) [2023-12-27 11:13:42,233][20381] Updated weights for policy 1, policy_version 808804 (0.0010) [2023-12-27 11:13:42,285][20347] Updated weights for policy 0, policy_version 809267 (0.0006) [2023-12-27 11:13:42,297][20381] Updated weights for policy 1, policy_version 808814 (0.0009) [2023-12-27 11:13:42,348][20347] Updated weights for policy 0, policy_version 809277 (0.0007) [2023-12-27 11:13:42,362][20381] Updated weights for policy 1, policy_version 808824 (0.0009) [2023-12-27 11:13:42,362][19170] Fps is (10 sec: 18842.2, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 414277632. Throughput: 0: 9957.2, 1: 9899.2. Samples: 414271180. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:42,363][19170] Avg episode reward: [(0, '961.000'), (1, '961.080')] [2023-12-27 11:13:43,079][20347] Updated weights for policy 0, policy_version 809287 (0.0007) [2023-12-27 11:13:43,109][20381] Updated weights for policy 1, policy_version 808834 (0.0011) [2023-12-27 11:13:43,145][20347] Updated weights for policy 0, policy_version 809297 (0.0006) [2023-12-27 11:13:43,170][20381] Updated weights for policy 1, policy_version 808844 (0.0011) [2023-12-27 11:13:43,196][20347] Updated weights for policy 0, policy_version 809307 (0.0006) [2023-12-27 11:13:43,224][20381] Updated weights for policy 1, policy_version 808854 (0.0011) [2023-12-27 11:13:43,883][20381] Updated weights for policy 1, policy_version 808864 (0.0010) [2023-12-27 11:13:43,922][20347] Updated weights for policy 0, policy_version 809317 (0.0006) [2023-12-27 11:13:43,944][20381] Updated weights for policy 1, policy_version 808874 (0.0011) [2023-12-27 11:13:43,981][20347] Updated weights for policy 0, policy_version 809327 (0.0008) [2023-12-27 11:13:43,998][20381] Updated weights for policy 1, policy_version 808884 (0.0010) [2023-12-27 11:13:44,044][20347] Updated weights for policy 0, policy_version 809337 (0.0006) [2023-12-27 11:13:44,692][20381] Updated weights for policy 1, policy_version 808894 (0.0011) [2023-12-27 11:13:44,745][20381] Updated weights for policy 1, policy_version 808904 (0.0011) [2023-12-27 11:13:44,760][20347] Updated weights for policy 0, policy_version 809347 (0.0008) [2023-12-27 11:13:44,803][20381] Updated weights for policy 1, policy_version 808914 (0.0010) [2023-12-27 11:13:44,828][20347] Updated weights for policy 0, policy_version 809357 (0.0006) [2023-12-27 11:13:44,889][20347] Updated weights for policy 0, policy_version 809367 (0.0008) [2023-12-27 11:13:45,530][20381] Updated weights for policy 1, policy_version 808924 (0.0009) [2023-12-27 11:13:45,595][20381] Updated weights for policy 1, policy_version 808934 (0.0008) [2023-12-27 11:13:45,644][20347] Updated weights for policy 0, policy_version 809377 (0.0009) [2023-12-27 11:13:45,655][20381] Updated weights for policy 1, policy_version 808944 (0.0007) [2023-12-27 11:13:45,698][20347] Updated weights for policy 0, policy_version 809387 (0.0007) [2023-12-27 11:13:45,758][20347] Updated weights for policy 0, policy_version 809397 (0.0007) [2023-12-27 11:13:45,817][20347] Updated weights for policy 0, policy_version 809407 (0.0008) [2023-12-27 11:13:46,318][20381] Updated weights for policy 1, policy_version 808954 (0.0006) [2023-12-27 11:13:46,379][20381] Updated weights for policy 1, policy_version 808964 (0.0008) [2023-12-27 11:13:46,441][20381] Updated weights for policy 1, policy_version 808974 (0.0008) [2023-12-27 11:13:46,501][20381] Updated weights for policy 1, policy_version 808984 (0.0007) [2023-12-27 11:13:46,541][20347] Updated weights for policy 0, policy_version 809417 (0.0008) [2023-12-27 11:13:46,589][20347] Updated weights for policy 0, policy_version 809427 (0.0009) [2023-12-27 11:13:46,654][20347] Updated weights for policy 0, policy_version 809437 (0.0007) [2023-12-27 11:13:47,183][20381] Updated weights for policy 1, policy_version 808994 (0.0007) [2023-12-27 11:13:47,247][20381] Updated weights for policy 1, policy_version 809004 (0.0007) [2023-12-27 11:13:47,307][20381] Updated weights for policy 1, policy_version 809014 (0.0008) [2023-12-27 11:13:47,362][19170] Fps is (10 sec: 18841.9, 60 sec: 19797.3, 300 sec: 20466.1). Total num frames: 414375936. Throughput: 0: 9806.0, 1: 9799.1. Samples: 414385788. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:47,363][19170] Avg episode reward: [(0, '970.790'), (1, '970.950')] [2023-12-27 11:13:47,378][20347] Updated weights for policy 0, policy_version 809447 (0.0008) [2023-12-27 11:13:47,436][20347] Updated weights for policy 0, policy_version 809457 (0.0007) [2023-12-27 11:13:47,495][20347] Updated weights for policy 0, policy_version 809467 (0.0006) [2023-12-27 11:13:48,019][20381] Updated weights for policy 1, policy_version 809024 (0.0008) [2023-12-27 11:13:48,083][20381] Updated weights for policy 1, policy_version 809034 (0.0008) [2023-12-27 11:13:48,140][20381] Updated weights for policy 1, policy_version 809044 (0.0008) [2023-12-27 11:13:48,211][20347] Updated weights for policy 0, policy_version 809477 (0.0007) [2023-12-27 11:13:48,272][20347] Updated weights for policy 0, policy_version 809487 (0.0008) [2023-12-27 11:13:48,337][20347] Updated weights for policy 0, policy_version 809497 (0.0007) [2023-12-27 11:13:48,895][20381] Updated weights for policy 1, policy_version 809054 (0.0008) [2023-12-27 11:13:48,953][20381] Updated weights for policy 1, policy_version 809064 (0.0007) [2023-12-27 11:13:49,011][20381] Updated weights for policy 1, policy_version 809074 (0.0007) [2023-12-27 11:13:49,096][20347] Updated weights for policy 0, policy_version 809507 (0.0009) [2023-12-27 11:13:49,154][20347] Updated weights for policy 0, policy_version 809517 (0.0006) [2023-12-27 11:13:49,204][20347] Updated weights for policy 0, policy_version 809527 (0.0007) [2023-12-27 11:13:49,698][20381] Updated weights for policy 1, policy_version 809084 (0.0007) [2023-12-27 11:13:49,759][20381] Updated weights for policy 1, policy_version 809094 (0.0009) [2023-12-27 11:13:49,820][20381] Updated weights for policy 1, policy_version 809104 (0.0007) [2023-12-27 11:13:49,870][20347] Updated weights for policy 0, policy_version 809537 (0.0009) [2023-12-27 11:13:49,937][20347] Updated weights for policy 0, policy_version 809547 (0.0008) [2023-12-27 11:13:49,996][20347] Updated weights for policy 0, policy_version 809557 (0.0009) [2023-12-27 11:13:50,060][20347] Updated weights for policy 0, policy_version 809567 (0.0008) [2023-12-27 11:13:50,563][20381] Updated weights for policy 1, policy_version 809114 (0.0009) [2023-12-27 11:13:50,621][20381] Updated weights for policy 1, policy_version 809124 (0.0011) [2023-12-27 11:13:50,678][20381] Updated weights for policy 1, policy_version 809134 (0.0011) [2023-12-27 11:13:50,735][20381] Updated weights for policy 1, policy_version 809144 (0.0010) [2023-12-27 11:13:50,765][20347] Updated weights for policy 0, policy_version 809577 (0.0007) [2023-12-27 11:13:50,831][20347] Updated weights for policy 0, policy_version 809587 (0.0008) [2023-12-27 11:13:50,891][20347] Updated weights for policy 0, policy_version 809597 (0.0008) [2023-12-27 11:13:51,518][20381] Updated weights for policy 1, policy_version 809154 (0.0008) [2023-12-27 11:13:51,579][20381] Updated weights for policy 1, policy_version 809164 (0.0006) [2023-12-27 11:13:51,581][20347] Updated weights for policy 0, policy_version 809607 (0.0007) [2023-12-27 11:13:51,646][20347] Updated weights for policy 0, policy_version 809617 (0.0007) [2023-12-27 11:13:51,659][20381] Updated weights for policy 1, policy_version 809174 (0.0008) [2023-12-27 11:13:51,721][20347] Updated weights for policy 0, policy_version 809627 (0.0007) [2023-12-27 11:13:52,363][19170] Fps is (10 sec: 18841.3, 60 sec: 19660.8, 300 sec: 20410.6). Total num frames: 414466048. Throughput: 0: 9748.6, 1: 9717.9. Samples: 414443444. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:52,363][19170] Avg episode reward: [(0, '970.750'), (1, '980.670')] [2023-12-27 11:13:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000809176_207175680.pth... [2023-12-27 11:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000809632_207290368.pth... [2023-12-27 11:13:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000808056_206888960.pth [2023-12-27 11:13:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000808512_207003648.pth [2023-12-27 11:13:52,452][20347] Updated weights for policy 0, policy_version 809637 (0.0008) [2023-12-27 11:13:52,513][20381] Updated weights for policy 1, policy_version 809184 (0.0008) [2023-12-27 11:13:52,514][20347] Updated weights for policy 0, policy_version 809647 (0.0007) [2023-12-27 11:13:52,574][20347] Updated weights for policy 0, policy_version 809657 (0.0007) [2023-12-27 11:13:52,580][20381] Updated weights for policy 1, policy_version 809194 (0.0008) [2023-12-27 11:13:52,642][20381] Updated weights for policy 1, policy_version 809204 (0.0008) [2023-12-27 11:13:53,297][20347] Updated weights for policy 0, policy_version 809667 (0.0007) [2023-12-27 11:13:53,357][20347] Updated weights for policy 0, policy_version 809677 (0.0008) [2023-12-27 11:13:53,405][20381] Updated weights for policy 1, policy_version 809214 (0.0007) [2023-12-27 11:13:53,423][20347] Updated weights for policy 0, policy_version 809687 (0.0006) [2023-12-27 11:13:53,468][20381] Updated weights for policy 1, policy_version 809224 (0.0006) [2023-12-27 11:13:53,524][20381] Updated weights for policy 1, policy_version 809234 (0.0007) [2023-12-27 11:13:54,130][20347] Updated weights for policy 0, policy_version 809697 (0.0006) [2023-12-27 11:13:54,188][20347] Updated weights for policy 0, policy_version 809707 (0.0008) [2023-12-27 11:13:54,250][20347] Updated weights for policy 0, policy_version 809717 (0.0009) [2023-12-27 11:13:54,251][20381] Updated weights for policy 1, policy_version 809244 (0.0007) [2023-12-27 11:13:54,310][20381] Updated weights for policy 1, policy_version 809254 (0.0008) [2023-12-27 11:13:54,320][20347] Updated weights for policy 0, policy_version 809727 (0.0007) [2023-12-27 11:13:54,374][20381] Updated weights for policy 1, policy_version 809264 (0.0009) [2023-12-27 11:13:54,994][20347] Updated weights for policy 0, policy_version 809737 (0.0008) [2023-12-27 11:13:55,054][20347] Updated weights for policy 0, policy_version 809747 (0.0008) [2023-12-27 11:13:55,072][20381] Updated weights for policy 1, policy_version 809274 (0.0008) [2023-12-27 11:13:55,105][20347] Updated weights for policy 0, policy_version 809757 (0.0007) [2023-12-27 11:13:55,130][20381] Updated weights for policy 1, policy_version 809284 (0.0006) [2023-12-27 11:13:55,195][20381] Updated weights for policy 1, policy_version 809294 (0.0007) [2023-12-27 11:13:55,257][20381] Updated weights for policy 1, policy_version 809304 (0.0006) [2023-12-27 11:13:55,832][20347] Updated weights for policy 0, policy_version 809767 (0.0008) [2023-12-27 11:13:55,846][20381] Updated weights for policy 1, policy_version 809314 (0.0008) [2023-12-27 11:13:55,891][20347] Updated weights for policy 0, policy_version 809777 (0.0007) [2023-12-27 11:13:55,903][20381] Updated weights for policy 1, policy_version 809324 (0.0008) [2023-12-27 11:13:55,953][20347] Updated weights for policy 0, policy_version 809787 (0.0007) [2023-12-27 11:13:55,966][20381] Updated weights for policy 1, policy_version 809334 (0.0007) [2023-12-27 11:13:56,626][20381] Updated weights for policy 1, policy_version 809344 (0.0007) [2023-12-27 11:13:56,651][20347] Updated weights for policy 0, policy_version 809797 (0.0007) [2023-12-27 11:13:56,685][20381] Updated weights for policy 1, policy_version 809354 (0.0011) [2023-12-27 11:13:56,706][20347] Updated weights for policy 0, policy_version 809807 (0.0008) [2023-12-27 11:13:56,742][20381] Updated weights for policy 1, policy_version 809364 (0.0006) [2023-12-27 11:13:56,771][20347] Updated weights for policy 0, policy_version 809817 (0.0007) [2023-12-27 11:13:57,362][20381] Updated weights for policy 1, policy_version 809374 (0.0006) [2023-12-27 11:13:57,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19660.8, 300 sec: 20410.6). Total num frames: 414564352. Throughput: 0: 9689.9, 1: 9692.7. Samples: 414558308. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:13:57,363][19170] Avg episode reward: [(0, '961.160'), (1, '970.840')] [2023-12-27 11:13:57,409][20347] Updated weights for policy 0, policy_version 809827 (0.0007) [2023-12-27 11:13:57,418][20381] Updated weights for policy 1, policy_version 809384 (0.0008) [2023-12-27 11:13:57,468][20347] Updated weights for policy 0, policy_version 809837 (0.0006) [2023-12-27 11:13:57,470][20381] Updated weights for policy 1, policy_version 809394 (0.0006) [2023-12-27 11:13:57,519][20347] Updated weights for policy 0, policy_version 809847 (0.0006) [2023-12-27 11:13:58,128][20381] Updated weights for policy 1, policy_version 809404 (0.0007) [2023-12-27 11:13:58,182][20381] Updated weights for policy 1, policy_version 809414 (0.0007) [2023-12-27 11:13:58,244][20347] Updated weights for policy 0, policy_version 809857 (0.0007) [2023-12-27 11:13:58,251][20381] Updated weights for policy 1, policy_version 809424 (0.0008) [2023-12-27 11:13:58,305][20347] Updated weights for policy 0, policy_version 809867 (0.0008) [2023-12-27 11:13:58,371][20347] Updated weights for policy 0, policy_version 809877 (0.0008) [2023-12-27 11:13:58,434][20347] Updated weights for policy 0, policy_version 809887 (0.0007) [2023-12-27 11:13:59,063][20381] Updated weights for policy 1, policy_version 809434 (0.0008) [2023-12-27 11:13:59,130][20381] Updated weights for policy 1, policy_version 809444 (0.0011) [2023-12-27 11:13:59,185][20381] Updated weights for policy 1, policy_version 809454 (0.0009) [2023-12-27 11:13:59,250][20381] Updated weights for policy 1, policy_version 809464 (0.0009) [2023-12-27 11:13:59,297][20347] Updated weights for policy 0, policy_version 809897 (0.0008) [2023-12-27 11:13:59,382][20347] Updated weights for policy 0, policy_version 809907 (0.0008) [2023-12-27 11:13:59,454][20347] Updated weights for policy 0, policy_version 809917 (0.0007) [2023-12-27 11:14:00,007][20381] Updated weights for policy 1, policy_version 809474 (0.0006) [2023-12-27 11:14:00,064][20381] Updated weights for policy 1, policy_version 809484 (0.0006) [2023-12-27 11:14:00,125][20381] Updated weights for policy 1, policy_version 809494 (0.0006) [2023-12-27 11:14:00,139][20347] Updated weights for policy 0, policy_version 809927 (0.0007) [2023-12-27 11:14:00,198][20347] Updated weights for policy 0, policy_version 809937 (0.0006) [2023-12-27 11:14:00,263][20347] Updated weights for policy 0, policy_version 809947 (0.0009) [2023-12-27 11:14:00,779][20381] Updated weights for policy 1, policy_version 809504 (0.0006) [2023-12-27 11:14:00,836][20381] Updated weights for policy 1, policy_version 809514 (0.0006) [2023-12-27 11:14:00,879][20347] Updated weights for policy 0, policy_version 809957 (0.0006) [2023-12-27 11:14:00,891][20381] Updated weights for policy 1, policy_version 809524 (0.0007) [2023-12-27 11:14:00,942][20347] Updated weights for policy 0, policy_version 809967 (0.0007) [2023-12-27 11:14:00,994][20347] Updated weights for policy 0, policy_version 809977 (0.0007) [2023-12-27 11:14:01,568][20381] Updated weights for policy 1, policy_version 809534 (0.0007) [2023-12-27 11:14:01,631][20381] Updated weights for policy 1, policy_version 809544 (0.0008) [2023-12-27 11:14:01,687][20381] Updated weights for policy 1, policy_version 809554 (0.0008) [2023-12-27 11:14:01,752][20347] Updated weights for policy 0, policy_version 809987 (0.0007) [2023-12-27 11:14:01,804][20347] Updated weights for policy 0, policy_version 809997 (0.0007) [2023-12-27 11:14:01,866][20347] Updated weights for policy 0, policy_version 810007 (0.0007) [2023-12-27 11:14:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 20410.6). Total num frames: 414662656. Throughput: 0: 9640.3, 1: 9682.8. Samples: 414674956. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:02,363][19170] Avg episode reward: [(0, '970.920'), (1, '970.840')] [2023-12-27 11:14:02,379][20381] Updated weights for policy 1, policy_version 809564 (0.0008) [2023-12-27 11:14:02,444][20381] Updated weights for policy 1, policy_version 809574 (0.0008) [2023-12-27 11:14:02,508][20381] Updated weights for policy 1, policy_version 809584 (0.0008) [2023-12-27 11:14:02,615][20347] Updated weights for policy 0, policy_version 810017 (0.0008) [2023-12-27 11:14:02,672][20347] Updated weights for policy 0, policy_version 810027 (0.0007) [2023-12-27 11:14:02,729][20347] Updated weights for policy 0, policy_version 810037 (0.0007) [2023-12-27 11:14:02,787][20347] Updated weights for policy 0, policy_version 810047 (0.0008) [2023-12-27 11:14:03,129][20381] Updated weights for policy 1, policy_version 809594 (0.0008) [2023-12-27 11:14:03,186][20381] Updated weights for policy 1, policy_version 809604 (0.0009) [2023-12-27 11:14:03,240][20381] Updated weights for policy 1, policy_version 809614 (0.0007) [2023-12-27 11:14:03,292][20381] Updated weights for policy 1, policy_version 809624 (0.0006) [2023-12-27 11:14:03,523][20347] Updated weights for policy 0, policy_version 810057 (0.0006) [2023-12-27 11:14:03,582][20347] Updated weights for policy 0, policy_version 810067 (0.0005) [2023-12-27 11:14:03,633][20347] Updated weights for policy 0, policy_version 810077 (0.0005) [2023-12-27 11:14:04,019][20381] Updated weights for policy 1, policy_version 809634 (0.0008) [2023-12-27 11:14:04,081][20381] Updated weights for policy 1, policy_version 809644 (0.0008) [2023-12-27 11:14:04,147][20381] Updated weights for policy 1, policy_version 809654 (0.0007) [2023-12-27 11:14:04,283][20347] Updated weights for policy 0, policy_version 810087 (0.0007) [2023-12-27 11:14:04,350][20347] Updated weights for policy 0, policy_version 810097 (0.0007) [2023-12-27 11:14:04,413][20347] Updated weights for policy 0, policy_version 810107 (0.0008) [2023-12-27 11:14:04,849][20381] Updated weights for policy 1, policy_version 809664 (0.0008) [2023-12-27 11:14:04,905][20381] Updated weights for policy 1, policy_version 809674 (0.0008) [2023-12-27 11:14:04,957][20381] Updated weights for policy 1, policy_version 809684 (0.0007) [2023-12-27 11:14:05,133][20347] Updated weights for policy 0, policy_version 810117 (0.0007) [2023-12-27 11:14:05,189][20347] Updated weights for policy 0, policy_version 810127 (0.0008) [2023-12-27 11:14:05,246][20347] Updated weights for policy 0, policy_version 810137 (0.0006) [2023-12-27 11:14:05,661][20381] Updated weights for policy 1, policy_version 809694 (0.0007) [2023-12-27 11:14:05,720][20381] Updated weights for policy 1, policy_version 809704 (0.0008) [2023-12-27 11:14:05,771][20381] Updated weights for policy 1, policy_version 809714 (0.0008) [2023-12-27 11:14:05,883][20347] Updated weights for policy 0, policy_version 810147 (0.0006) [2023-12-27 11:14:05,937][20347] Updated weights for policy 0, policy_version 810157 (0.0005) [2023-12-27 11:14:06,002][20347] Updated weights for policy 0, policy_version 810167 (0.0008) [2023-12-27 11:14:06,470][20381] Updated weights for policy 1, policy_version 809724 (0.0009) [2023-12-27 11:14:06,535][20381] Updated weights for policy 1, policy_version 809734 (0.0007) [2023-12-27 11:14:06,600][20381] Updated weights for policy 1, policy_version 809744 (0.0007) [2023-12-27 11:14:06,773][20347] Updated weights for policy 0, policy_version 810177 (0.0007) [2023-12-27 11:14:06,832][20347] Updated weights for policy 0, policy_version 810187 (0.0008) [2023-12-27 11:14:06,892][20347] Updated weights for policy 0, policy_version 810197 (0.0006) [2023-12-27 11:14:06,954][20347] Updated weights for policy 0, policy_version 810207 (0.0006) [2023-12-27 11:14:07,319][20381] Updated weights for policy 1, policy_version 809754 (0.0008) [2023-12-27 11:14:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 20410.6). Total num frames: 414760960. Throughput: 0: 9658.0, 1: 9709.2. Samples: 414734616. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:07,363][19170] Avg episode reward: [(0, '970.870'), (1, '1000.000')] [2023-12-27 11:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000810208_207437824.pth... [2023-12-27 11:14:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000809088_207151104.pth [2023-12-27 11:14:07,379][20381] Updated weights for policy 1, policy_version 809764 (0.0007) [2023-12-27 11:14:07,433][20381] Updated weights for policy 1, policy_version 809774 (0.0007) [2023-12-27 11:14:07,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000809784_207331328.pth... [2023-12-27 11:14:07,494][20381] Updated weights for policy 1, policy_version 809784 (0.0008) [2023-12-27 11:14:07,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000808632_207036416.pth [2023-12-27 11:14:07,615][20347] Updated weights for policy 0, policy_version 810217 (0.0008) [2023-12-27 11:14:07,672][20347] Updated weights for policy 0, policy_version 810227 (0.0006) [2023-12-27 11:14:07,730][20347] Updated weights for policy 0, policy_version 810237 (0.0007) [2023-12-27 11:14:08,248][20381] Updated weights for policy 1, policy_version 809794 (0.0007) [2023-12-27 11:14:08,311][20381] Updated weights for policy 1, policy_version 809804 (0.0007) [2023-12-27 11:14:08,378][20381] Updated weights for policy 1, policy_version 809814 (0.0008) [2023-12-27 11:14:08,396][20347] Updated weights for policy 0, policy_version 810247 (0.0007) [2023-12-27 11:14:08,460][20347] Updated weights for policy 0, policy_version 810257 (0.0007) [2023-12-27 11:14:08,525][20347] Updated weights for policy 0, policy_version 810267 (0.0008) [2023-12-27 11:14:09,053][20381] Updated weights for policy 1, policy_version 809824 (0.0006) [2023-12-27 11:14:09,114][20381] Updated weights for policy 1, policy_version 809834 (0.0006) [2023-12-27 11:14:09,174][20381] Updated weights for policy 1, policy_version 809844 (0.0007) [2023-12-27 11:14:09,234][20347] Updated weights for policy 0, policy_version 810277 (0.0008) [2023-12-27 11:14:09,301][20347] Updated weights for policy 0, policy_version 810287 (0.0008) [2023-12-27 11:14:09,370][20347] Updated weights for policy 0, policy_version 810297 (0.0008) [2023-12-27 11:14:09,851][20381] Updated weights for policy 1, policy_version 809854 (0.0009) [2023-12-27 11:14:09,902][20381] Updated weights for policy 1, policy_version 809864 (0.0009) [2023-12-27 11:14:09,969][20381] Updated weights for policy 1, policy_version 809874 (0.0008) [2023-12-27 11:14:10,162][20347] Updated weights for policy 0, policy_version 810307 (0.0008) [2023-12-27 11:14:10,223][20347] Updated weights for policy 0, policy_version 810317 (0.0008) [2023-12-27 11:14:10,290][20347] Updated weights for policy 0, policy_version 810327 (0.0009) [2023-12-27 11:14:10,727][20381] Updated weights for policy 1, policy_version 809884 (0.0008) [2023-12-27 11:14:10,782][20381] Updated weights for policy 1, policy_version 809894 (0.0008) [2023-12-27 11:14:10,841][20381] Updated weights for policy 1, policy_version 809904 (0.0008) [2023-12-27 11:14:11,000][20347] Updated weights for policy 0, policy_version 810337 (0.0008) [2023-12-27 11:14:11,073][20347] Updated weights for policy 0, policy_version 810347 (0.0010) [2023-12-27 11:14:11,138][20347] Updated weights for policy 0, policy_version 810357 (0.0008) [2023-12-27 11:14:11,204][20347] Updated weights for policy 0, policy_version 810367 (0.0007) [2023-12-27 11:14:11,639][20381] Updated weights for policy 1, policy_version 809914 (0.0007) [2023-12-27 11:14:11,710][20381] Updated weights for policy 1, policy_version 809924 (0.0007) [2023-12-27 11:14:11,777][20381] Updated weights for policy 1, policy_version 809934 (0.0007) [2023-12-27 11:14:11,846][20381] Updated weights for policy 1, policy_version 809944 (0.0008) [2023-12-27 11:14:12,048][20347] Updated weights for policy 0, policy_version 810377 (0.0008) [2023-12-27 11:14:12,112][20347] Updated weights for policy 0, policy_version 810387 (0.0008) [2023-12-27 11:14:12,177][20347] Updated weights for policy 0, policy_version 810397 (0.0008) [2023-12-27 11:14:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 20382.8). Total num frames: 414859264. Throughput: 0: 9685.0, 1: 9710.0. Samples: 414850168. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:12,363][19170] Avg episode reward: [(0, '990.360'), (1, '1000.000')] [2023-12-27 11:14:12,591][20381] Updated weights for policy 1, policy_version 809954 (0.0007) [2023-12-27 11:14:12,653][20381] Updated weights for policy 1, policy_version 809964 (0.0008) [2023-12-27 11:14:12,710][20381] Updated weights for policy 1, policy_version 809974 (0.0009) [2023-12-27 11:14:12,899][20347] Updated weights for policy 0, policy_version 810407 (0.0008) [2023-12-27 11:14:12,963][20347] Updated weights for policy 0, policy_version 810417 (0.0007) [2023-12-27 11:14:13,027][20347] Updated weights for policy 0, policy_version 810427 (0.0007) [2023-12-27 11:14:13,419][20381] Updated weights for policy 1, policy_version 809984 (0.0009) [2023-12-27 11:14:13,472][20381] Updated weights for policy 1, policy_version 809994 (0.0009) [2023-12-27 11:14:13,530][20381] Updated weights for policy 1, policy_version 810004 (0.0008) [2023-12-27 11:14:13,694][20347] Updated weights for policy 0, policy_version 810437 (0.0009) [2023-12-27 11:14:13,748][20347] Updated weights for policy 0, policy_version 810447 (0.0009) [2023-12-27 11:14:13,804][20347] Updated weights for policy 0, policy_version 810457 (0.0006) [2023-12-27 11:14:14,278][20381] Updated weights for policy 1, policy_version 810014 (0.0009) [2023-12-27 11:14:14,340][20381] Updated weights for policy 1, policy_version 810024 (0.0008) [2023-12-27 11:14:14,405][20381] Updated weights for policy 1, policy_version 810034 (0.0008) [2023-12-27 11:14:14,515][20347] Updated weights for policy 0, policy_version 810467 (0.0007) [2023-12-27 11:14:14,582][20347] Updated weights for policy 0, policy_version 810477 (0.0008) [2023-12-27 11:14:14,649][20347] Updated weights for policy 0, policy_version 810487 (0.0008) [2023-12-27 11:14:15,158][20381] Updated weights for policy 1, policy_version 810044 (0.0007) [2023-12-27 11:14:15,229][20381] Updated weights for policy 1, policy_version 810054 (0.0008) [2023-12-27 11:14:15,292][20381] Updated weights for policy 1, policy_version 810064 (0.0009) [2023-12-27 11:14:15,426][20347] Updated weights for policy 0, policy_version 810497 (0.0008) [2023-12-27 11:14:15,480][20347] Updated weights for policy 0, policy_version 810507 (0.0008) [2023-12-27 11:14:15,539][20347] Updated weights for policy 0, policy_version 810517 (0.0008) [2023-12-27 11:14:15,599][20347] Updated weights for policy 0, policy_version 810527 (0.0008) [2023-12-27 11:14:15,983][20381] Updated weights for policy 1, policy_version 810074 (0.0008) [2023-12-27 11:14:16,037][20381] Updated weights for policy 1, policy_version 810084 (0.0006) [2023-12-27 11:14:16,098][20381] Updated weights for policy 1, policy_version 810094 (0.0006) [2023-12-27 11:14:16,165][20381] Updated weights for policy 1, policy_version 810104 (0.0007) [2023-12-27 11:14:16,343][20347] Updated weights for policy 0, policy_version 810537 (0.0007) [2023-12-27 11:14:16,410][20347] Updated weights for policy 0, policy_version 810547 (0.0007) [2023-12-27 11:14:16,464][20347] Updated weights for policy 0, policy_version 810557 (0.0007) [2023-12-27 11:14:16,835][20381] Updated weights for policy 1, policy_version 810114 (0.0007) [2023-12-27 11:14:16,902][20381] Updated weights for policy 1, policy_version 810124 (0.0008) [2023-12-27 11:14:16,958][20381] Updated weights for policy 1, policy_version 810134 (0.0008) [2023-12-27 11:14:17,128][20347] Updated weights for policy 0, policy_version 810567 (0.0007) [2023-12-27 11:14:17,177][20347] Updated weights for policy 0, policy_version 810577 (0.0007) [2023-12-27 11:14:17,235][20347] Updated weights for policy 0, policy_version 810587 (0.0006) [2023-12-27 11:14:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 20382.8). Total num frames: 414957568. Throughput: 0: 9632.7, 1: 9661.4. Samples: 414964508. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:14:17,651][20381] Updated weights for policy 1, policy_version 810144 (0.0007) [2023-12-27 11:14:17,710][20381] Updated weights for policy 1, policy_version 810154 (0.0005) [2023-12-27 11:14:17,770][20381] Updated weights for policy 1, policy_version 810164 (0.0006) [2023-12-27 11:14:17,943][20347] Updated weights for policy 0, policy_version 810597 (0.0007) [2023-12-27 11:14:18,008][20347] Updated weights for policy 0, policy_version 810607 (0.0006) [2023-12-27 11:14:18,071][20347] Updated weights for policy 0, policy_version 810617 (0.0007) [2023-12-27 11:14:18,462][20381] Updated weights for policy 1, policy_version 810174 (0.0007) [2023-12-27 11:14:18,520][20381] Updated weights for policy 1, policy_version 810184 (0.0007) [2023-12-27 11:14:18,580][20381] Updated weights for policy 1, policy_version 810194 (0.0005) [2023-12-27 11:14:18,783][20347] Updated weights for policy 0, policy_version 810627 (0.0007) [2023-12-27 11:14:18,843][20347] Updated weights for policy 0, policy_version 810637 (0.0008) [2023-12-27 11:14:18,907][20347] Updated weights for policy 0, policy_version 810647 (0.0007) [2023-12-27 11:14:19,134][20381] Updated weights for policy 1, policy_version 810204 (0.0006) [2023-12-27 11:14:19,186][20381] Updated weights for policy 1, policy_version 810214 (0.0008) [2023-12-27 11:14:19,245][20381] Updated weights for policy 1, policy_version 810224 (0.0008) [2023-12-27 11:14:19,607][20347] Updated weights for policy 0, policy_version 810657 (0.0007) [2023-12-27 11:14:19,665][20347] Updated weights for policy 0, policy_version 810667 (0.0007) [2023-12-27 11:14:19,726][20347] Updated weights for policy 0, policy_version 810677 (0.0007) [2023-12-27 11:14:19,786][20347] Updated weights for policy 0, policy_version 810687 (0.0007) [2023-12-27 11:14:20,027][20381] Updated weights for policy 1, policy_version 810234 (0.0009) [2023-12-27 11:14:20,088][20381] Updated weights for policy 1, policy_version 810244 (0.0009) [2023-12-27 11:14:20,156][20381] Updated weights for policy 1, policy_version 810254 (0.0008) [2023-12-27 11:14:20,218][20381] Updated weights for policy 1, policy_version 810264 (0.0008) [2023-12-27 11:14:20,519][20347] Updated weights for policy 0, policy_version 810697 (0.0008) [2023-12-27 11:14:20,585][20347] Updated weights for policy 0, policy_version 810707 (0.0008) [2023-12-27 11:14:20,651][20347] Updated weights for policy 0, policy_version 810717 (0.0007) [2023-12-27 11:14:20,949][20381] Updated weights for policy 1, policy_version 810274 (0.0007) [2023-12-27 11:14:21,016][20381] Updated weights for policy 1, policy_version 810284 (0.0008) [2023-12-27 11:14:21,085][20381] Updated weights for policy 1, policy_version 810295 (0.0008) [2023-12-27 11:14:21,395][20347] Updated weights for policy 0, policy_version 810727 (0.0008) [2023-12-27 11:14:21,464][20347] Updated weights for policy 0, policy_version 810737 (0.0008) [2023-12-27 11:14:21,525][20347] Updated weights for policy 0, policy_version 810747 (0.0007) [2023-12-27 11:14:21,895][20381] Updated weights for policy 1, policy_version 810305 (0.0007) [2023-12-27 11:14:21,953][20381] Updated weights for policy 1, policy_version 810315 (0.0010) [2023-12-27 11:14:22,018][20381] Updated weights for policy 1, policy_version 810325 (0.0011) [2023-12-27 11:14:22,243][20347] Updated weights for policy 0, policy_version 810757 (0.0007) [2023-12-27 11:14:22,299][20347] Updated weights for policy 0, policy_version 810767 (0.0007) [2023-12-27 11:14:22,352][20347] Updated weights for policy 0, policy_version 810777 (0.0007) [2023-12-27 11:14:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 20299.5). Total num frames: 415047680. Throughput: 0: 9613.1, 1: 9668.5. Samples: 415023864. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '966.780')] [2023-12-27 11:14:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000810328_207470592.pth... [2023-12-27 11:14:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000809176_207175680.pth [2023-12-27 11:14:22,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000810784_207585280.pth... [2023-12-27 11:14:22,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000809632_207290368.pth [2023-12-27 11:14:22,778][20381] Updated weights for policy 1, policy_version 810335 (0.0009) [2023-12-27 11:14:22,845][20381] Updated weights for policy 1, policy_version 810345 (0.0007) [2023-12-27 11:14:22,902][20381] Updated weights for policy 1, policy_version 810355 (0.0008) [2023-12-27 11:14:23,114][20347] Updated weights for policy 0, policy_version 810787 (0.0009) [2023-12-27 11:14:23,183][20347] Updated weights for policy 0, policy_version 810797 (0.0009) [2023-12-27 11:14:23,246][20347] Updated weights for policy 0, policy_version 810807 (0.0008) [2023-12-27 11:14:23,696][20381] Updated weights for policy 1, policy_version 810365 (0.0008) [2023-12-27 11:14:23,746][20381] Updated weights for policy 1, policy_version 810375 (0.0008) [2023-12-27 11:14:23,806][20381] Updated weights for policy 1, policy_version 810385 (0.0009) [2023-12-27 11:14:23,958][20347] Updated weights for policy 0, policy_version 810817 (0.0008) [2023-12-27 11:14:24,008][20347] Updated weights for policy 0, policy_version 810827 (0.0007) [2023-12-27 11:14:24,068][20347] Updated weights for policy 0, policy_version 810837 (0.0006) [2023-12-27 11:14:24,126][20347] Updated weights for policy 0, policy_version 810847 (0.0007) [2023-12-27 11:14:24,547][20381] Updated weights for policy 1, policy_version 810395 (0.0008) [2023-12-27 11:14:24,605][20381] Updated weights for policy 1, policy_version 810405 (0.0008) [2023-12-27 11:14:24,667][20381] Updated weights for policy 1, policy_version 810415 (0.0007) [2023-12-27 11:14:24,850][20347] Updated weights for policy 0, policy_version 810857 (0.0007) [2023-12-27 11:14:24,912][20347] Updated weights for policy 0, policy_version 810867 (0.0008) [2023-12-27 11:14:24,975][20347] Updated weights for policy 0, policy_version 810877 (0.0008) [2023-12-27 11:14:25,358][20381] Updated weights for policy 1, policy_version 810425 (0.0007) [2023-12-27 11:14:25,415][20381] Updated weights for policy 1, policy_version 810435 (0.0008) [2023-12-27 11:14:25,477][20381] Updated weights for policy 1, policy_version 810445 (0.0008) [2023-12-27 11:14:25,530][20381] Updated weights for policy 1, policy_version 810455 (0.0008) [2023-12-27 11:14:25,677][20347] Updated weights for policy 0, policy_version 810887 (0.0008) [2023-12-27 11:14:25,741][20347] Updated weights for policy 0, policy_version 810897 (0.0008) [2023-12-27 11:14:25,801][20347] Updated weights for policy 0, policy_version 810907 (0.0008) [2023-12-27 11:14:26,201][20381] Updated weights for policy 1, policy_version 810465 (0.0007) [2023-12-27 11:14:26,262][20381] Updated weights for policy 1, policy_version 810475 (0.0007) [2023-12-27 11:14:26,324][20381] Updated weights for policy 1, policy_version 810485 (0.0007) [2023-12-27 11:14:26,544][20347] Updated weights for policy 0, policy_version 810917 (0.0009) [2023-12-27 11:14:26,604][20347] Updated weights for policy 0, policy_version 810927 (0.0007) [2023-12-27 11:14:26,663][20347] Updated weights for policy 0, policy_version 810937 (0.0008) [2023-12-27 11:14:26,976][20381] Updated weights for policy 1, policy_version 810495 (0.0006) [2023-12-27 11:14:27,034][20381] Updated weights for policy 1, policy_version 810505 (0.0007) [2023-12-27 11:14:27,091][20381] Updated weights for policy 1, policy_version 810515 (0.0008) [2023-12-27 11:14:27,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 20299.5). Total num frames: 415145984. Throughput: 0: 9586.9, 1: 9660.0. Samples: 415137292. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '897.100')] [2023-12-27 11:14:27,364][20347] Updated weights for policy 0, policy_version 810947 (0.0008) [2023-12-27 11:14:27,416][20347] Updated weights for policy 0, policy_version 810957 (0.0007) [2023-12-27 11:14:27,478][20347] Updated weights for policy 0, policy_version 810967 (0.0006) [2023-12-27 11:14:27,791][20381] Updated weights for policy 1, policy_version 810525 (0.0007) [2023-12-27 11:14:27,855][20381] Updated weights for policy 1, policy_version 810535 (0.0006) [2023-12-27 11:14:27,910][20381] Updated weights for policy 1, policy_version 810545 (0.0005) [2023-12-27 11:14:28,113][20347] Updated weights for policy 0, policy_version 810977 (0.0007) [2023-12-27 11:14:28,168][20347] Updated weights for policy 0, policy_version 810987 (0.0008) [2023-12-27 11:14:28,224][20347] Updated weights for policy 0, policy_version 810997 (0.0007) [2023-12-27 11:14:28,280][20347] Updated weights for policy 0, policy_version 811007 (0.0008) [2023-12-27 11:14:28,581][20381] Updated weights for policy 1, policy_version 810555 (0.0008) [2023-12-27 11:14:28,643][20381] Updated weights for policy 1, policy_version 810565 (0.0007) [2023-12-27 11:14:28,705][20381] Updated weights for policy 1, policy_version 810575 (0.0010) [2023-12-27 11:14:29,075][20347] Updated weights for policy 0, policy_version 811017 (0.0008) [2023-12-27 11:14:29,134][20347] Updated weights for policy 0, policy_version 811027 (0.0007) [2023-12-27 11:14:29,195][20347] Updated weights for policy 0, policy_version 811037 (0.0006) [2023-12-27 11:14:29,433][20381] Updated weights for policy 1, policy_version 810585 (0.0010) [2023-12-27 11:14:29,498][20381] Updated weights for policy 1, policy_version 810595 (0.0008) [2023-12-27 11:14:29,554][20381] Updated weights for policy 1, policy_version 810605 (0.0008) [2023-12-27 11:14:29,613][20381] Updated weights for policy 1, policy_version 810615 (0.0008) [2023-12-27 11:14:30,012][20347] Updated weights for policy 0, policy_version 811047 (0.0007) [2023-12-27 11:14:30,072][20347] Updated weights for policy 0, policy_version 811057 (0.0008) [2023-12-27 11:14:30,129][20347] Updated weights for policy 0, policy_version 811067 (0.0008) [2023-12-27 11:14:30,373][20381] Updated weights for policy 1, policy_version 810625 (0.0008) [2023-12-27 11:14:30,431][20381] Updated weights for policy 1, policy_version 810635 (0.0008) [2023-12-27 11:14:30,490][20381] Updated weights for policy 1, policy_version 810645 (0.0008) [2023-12-27 11:14:30,776][20347] Updated weights for policy 0, policy_version 811077 (0.0007) [2023-12-27 11:14:30,831][20347] Updated weights for policy 0, policy_version 811087 (0.0007) [2023-12-27 11:14:30,883][20347] Updated weights for policy 0, policy_version 811097 (0.0007) [2023-12-27 11:14:31,161][20381] Updated weights for policy 1, policy_version 810655 (0.0009) [2023-12-27 11:14:31,222][20381] Updated weights for policy 1, policy_version 810665 (0.0006) [2023-12-27 11:14:31,283][20381] Updated weights for policy 1, policy_version 810675 (0.0008) [2023-12-27 11:14:31,656][20347] Updated weights for policy 0, policy_version 811107 (0.0007) [2023-12-27 11:14:31,714][20347] Updated weights for policy 0, policy_version 811117 (0.0008) [2023-12-27 11:14:31,789][20347] Updated weights for policy 0, policy_version 811127 (0.0008) [2023-12-27 11:14:32,039][20381] Updated weights for policy 1, policy_version 810685 (0.0006) [2023-12-27 11:14:32,100][20381] Updated weights for policy 1, policy_version 810695 (0.0008) [2023-12-27 11:14:32,163][20381] Updated weights for policy 1, policy_version 810705 (0.0007) [2023-12-27 11:14:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.3, 300 sec: 20299.5). Total num frames: 415244288. Throughput: 0: 9613.8, 1: 9669.0. Samples: 415253520. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.490')] [2023-12-27 11:14:32,585][20347] Updated weights for policy 0, policy_version 811137 (0.0008) [2023-12-27 11:14:32,641][20347] Updated weights for policy 0, policy_version 811147 (0.0008) [2023-12-27 11:14:32,701][20347] Updated weights for policy 0, policy_version 811157 (0.0008) [2023-12-27 11:14:32,756][20347] Updated weights for policy 0, policy_version 811167 (0.0008) [2023-12-27 11:14:32,885][20381] Updated weights for policy 1, policy_version 810715 (0.0007) [2023-12-27 11:14:32,947][20381] Updated weights for policy 1, policy_version 810725 (0.0007) [2023-12-27 11:14:33,002][20381] Updated weights for policy 1, policy_version 810735 (0.0008) [2023-12-27 11:14:33,487][20347] Updated weights for policy 0, policy_version 811177 (0.0006) [2023-12-27 11:14:33,552][20347] Updated weights for policy 0, policy_version 811187 (0.0006) [2023-12-27 11:14:33,609][20347] Updated weights for policy 0, policy_version 811197 (0.0006) [2023-12-27 11:14:33,710][20381] Updated weights for policy 1, policy_version 810745 (0.0008) [2023-12-27 11:14:33,767][20381] Updated weights for policy 1, policy_version 810755 (0.0009) [2023-12-27 11:14:33,811][20381] Updated weights for policy 1, policy_version 810765 (0.0008) [2023-12-27 11:14:33,858][20381] Updated weights for policy 1, policy_version 810775 (0.0008) [2023-12-27 11:14:34,229][20347] Updated weights for policy 0, policy_version 811207 (0.0008) [2023-12-27 11:14:34,289][20347] Updated weights for policy 0, policy_version 811217 (0.0008) [2023-12-27 11:14:34,357][20347] Updated weights for policy 0, policy_version 811227 (0.0008) [2023-12-27 11:14:34,637][20381] Updated weights for policy 1, policy_version 810785 (0.0007) [2023-12-27 11:14:34,705][20381] Updated weights for policy 1, policy_version 810795 (0.0008) [2023-12-27 11:14:34,764][20381] Updated weights for policy 1, policy_version 810805 (0.0008) [2023-12-27 11:14:35,068][20347] Updated weights for policy 0, policy_version 811237 (0.0008) [2023-12-27 11:14:35,134][20347] Updated weights for policy 0, policy_version 811247 (0.0008) [2023-12-27 11:14:35,193][20347] Updated weights for policy 0, policy_version 811257 (0.0007) [2023-12-27 11:14:35,498][20381] Updated weights for policy 1, policy_version 810815 (0.0007) [2023-12-27 11:14:35,558][20381] Updated weights for policy 1, policy_version 810825 (0.0008) [2023-12-27 11:14:35,616][20381] Updated weights for policy 1, policy_version 810835 (0.0007) [2023-12-27 11:14:35,888][20347] Updated weights for policy 0, policy_version 811267 (0.0007) [2023-12-27 11:14:35,949][20347] Updated weights for policy 0, policy_version 811277 (0.0006) [2023-12-27 11:14:36,009][20347] Updated weights for policy 0, policy_version 811287 (0.0006) [2023-12-27 11:14:36,338][20381] Updated weights for policy 1, policy_version 810845 (0.0008) [2023-12-27 11:14:36,401][20381] Updated weights for policy 1, policy_version 810855 (0.0009) [2023-12-27 11:14:36,467][20381] Updated weights for policy 1, policy_version 810865 (0.0007) [2023-12-27 11:14:36,720][20347] Updated weights for policy 0, policy_version 811297 (0.0007) [2023-12-27 11:14:36,781][20347] Updated weights for policy 0, policy_version 811307 (0.0007) [2023-12-27 11:14:36,839][20347] Updated weights for policy 0, policy_version 811317 (0.0007) [2023-12-27 11:14:36,902][20347] Updated weights for policy 0, policy_version 811327 (0.0009) [2023-12-27 11:14:37,171][20381] Updated weights for policy 1, policy_version 810875 (0.0010) [2023-12-27 11:14:37,235][20381] Updated weights for policy 1, policy_version 810885 (0.0006) [2023-12-27 11:14:37,293][20381] Updated weights for policy 1, policy_version 810895 (0.0008) [2023-12-27 11:14:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19251.2, 300 sec: 20299.5). Total num frames: 415342592. Throughput: 0: 9625.1, 1: 9665.0. Samples: 415311496. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '867.510')] [2023-12-27 11:14:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000810904_207618048.pth... [2023-12-27 11:14:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000811328_207724544.pth... [2023-12-27 11:14:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000809784_207331328.pth [2023-12-27 11:14:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000810208_207437824.pth [2023-12-27 11:14:37,610][20347] Updated weights for policy 0, policy_version 811337 (0.0008) [2023-12-27 11:14:37,667][20347] Updated weights for policy 0, policy_version 811347 (0.0008) [2023-12-27 11:14:37,724][20347] Updated weights for policy 0, policy_version 811357 (0.0007) [2023-12-27 11:14:37,993][20381] Updated weights for policy 1, policy_version 810905 (0.0008) [2023-12-27 11:14:38,054][20381] Updated weights for policy 1, policy_version 810915 (0.0009) [2023-12-27 11:14:38,108][20381] Updated weights for policy 1, policy_version 810925 (0.0009) [2023-12-27 11:14:38,167][20381] Updated weights for policy 1, policy_version 810935 (0.0007) [2023-12-27 11:14:38,415][20347] Updated weights for policy 0, policy_version 811367 (0.0007) [2023-12-27 11:14:38,482][20347] Updated weights for policy 0, policy_version 811377 (0.0008) [2023-12-27 11:14:38,540][20347] Updated weights for policy 0, policy_version 811387 (0.0007) [2023-12-27 11:14:38,958][20381] Updated weights for policy 1, policy_version 810945 (0.0008) [2023-12-27 11:14:39,022][20381] Updated weights for policy 1, policy_version 810955 (0.0008) [2023-12-27 11:14:39,081][20381] Updated weights for policy 1, policy_version 810965 (0.0008) [2023-12-27 11:14:39,189][20347] Updated weights for policy 0, policy_version 811397 (0.0008) [2023-12-27 11:14:39,254][20347] Updated weights for policy 0, policy_version 811407 (0.0007) [2023-12-27 11:14:39,316][20347] Updated weights for policy 0, policy_version 811417 (0.0009) [2023-12-27 11:14:39,862][20381] Updated weights for policy 1, policy_version 810975 (0.0008) [2023-12-27 11:14:39,926][20381] Updated weights for policy 1, policy_version 810985 (0.0007) [2023-12-27 11:14:39,992][20381] Updated weights for policy 1, policy_version 810995 (0.0009) [2023-12-27 11:14:40,051][20347] Updated weights for policy 0, policy_version 811427 (0.0009) [2023-12-27 11:14:40,104][20347] Updated weights for policy 0, policy_version 811437 (0.0008) [2023-12-27 11:14:40,167][20347] Updated weights for policy 0, policy_version 811447 (0.0008) [2023-12-27 11:14:40,731][20381] Updated weights for policy 1, policy_version 811005 (0.0006) [2023-12-27 11:14:40,782][20381] Updated weights for policy 1, policy_version 811015 (0.0008) [2023-12-27 11:14:40,842][20381] Updated weights for policy 1, policy_version 811025 (0.0009) [2023-12-27 11:14:40,925][20347] Updated weights for policy 0, policy_version 811457 (0.0008) [2023-12-27 11:14:40,992][20347] Updated weights for policy 0, policy_version 811467 (0.0008) [2023-12-27 11:14:41,061][20347] Updated weights for policy 0, policy_version 811477 (0.0008) [2023-12-27 11:14:41,132][20347] Updated weights for policy 0, policy_version 811487 (0.0007) [2023-12-27 11:14:41,595][20381] Updated weights for policy 1, policy_version 811035 (0.0008) [2023-12-27 11:14:41,671][20381] Updated weights for policy 1, policy_version 811045 (0.0008) [2023-12-27 11:14:41,745][20381] Updated weights for policy 1, policy_version 811055 (0.0008) [2023-12-27 11:14:41,947][20347] Updated weights for policy 0, policy_version 811497 (0.0009) [2023-12-27 11:14:42,016][20347] Updated weights for policy 0, policy_version 811507 (0.0008) [2023-12-27 11:14:42,087][20347] Updated weights for policy 0, policy_version 811517 (0.0008) [2023-12-27 11:14:42,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 20216.2). Total num frames: 415432704. Throughput: 0: 9631.7, 1: 9639.9. Samples: 415425532. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '915.320')] [2023-12-27 11:14:42,495][20381] Updated weights for policy 1, policy_version 811065 (0.0008) [2023-12-27 11:14:42,556][20381] Updated weights for policy 1, policy_version 811075 (0.0007) [2023-12-27 11:14:42,618][20381] Updated weights for policy 1, policy_version 811085 (0.0008) [2023-12-27 11:14:42,680][20381] Updated weights for policy 1, policy_version 811095 (0.0007) [2023-12-27 11:14:42,880][20347] Updated weights for policy 0, policy_version 811527 (0.0007) [2023-12-27 11:14:42,938][20347] Updated weights for policy 0, policy_version 811537 (0.0007) [2023-12-27 11:14:42,991][20347] Updated weights for policy 0, policy_version 811547 (0.0007) [2023-12-27 11:14:43,396][20381] Updated weights for policy 1, policy_version 811105 (0.0006) [2023-12-27 11:14:43,456][20381] Updated weights for policy 1, policy_version 811115 (0.0006) [2023-12-27 11:14:43,513][20381] Updated weights for policy 1, policy_version 811125 (0.0008) [2023-12-27 11:14:43,673][20347] Updated weights for policy 0, policy_version 811557 (0.0007) [2023-12-27 11:14:43,726][20347] Updated weights for policy 0, policy_version 811567 (0.0008) [2023-12-27 11:14:43,780][20347] Updated weights for policy 0, policy_version 811577 (0.0007) [2023-12-27 11:14:44,160][20381] Updated weights for policy 1, policy_version 811135 (0.0007) [2023-12-27 11:14:44,220][20381] Updated weights for policy 1, policy_version 811145 (0.0007) [2023-12-27 11:14:44,279][20381] Updated weights for policy 1, policy_version 811155 (0.0007) [2023-12-27 11:14:44,511][20347] Updated weights for policy 0, policy_version 811587 (0.0007) [2023-12-27 11:14:44,569][20347] Updated weights for policy 0, policy_version 811597 (0.0006) [2023-12-27 11:14:44,632][20347] Updated weights for policy 0, policy_version 811607 (0.0006) [2023-12-27 11:14:45,022][20381] Updated weights for policy 1, policy_version 811165 (0.0009) [2023-12-27 11:14:45,084][20381] Updated weights for policy 1, policy_version 811175 (0.0008) [2023-12-27 11:14:45,146][20381] Updated weights for policy 1, policy_version 811185 (0.0008) [2023-12-27 11:14:45,324][20347] Updated weights for policy 0, policy_version 811617 (0.0007) [2023-12-27 11:14:45,381][20347] Updated weights for policy 0, policy_version 811627 (0.0008) [2023-12-27 11:14:45,428][20347] Updated weights for policy 0, policy_version 811637 (0.0006) [2023-12-27 11:14:45,481][20347] Updated weights for policy 0, policy_version 811647 (0.0006) [2023-12-27 11:14:45,900][20381] Updated weights for policy 1, policy_version 811195 (0.0008) [2023-12-27 11:14:45,960][20381] Updated weights for policy 1, policy_version 811205 (0.0008) [2023-12-27 11:14:46,015][20381] Updated weights for policy 1, policy_version 811215 (0.0008) [2023-12-27 11:14:46,171][20347] Updated weights for policy 0, policy_version 811657 (0.0008) [2023-12-27 11:14:46,234][20347] Updated weights for policy 0, policy_version 811667 (0.0007) [2023-12-27 11:14:46,281][20347] Updated weights for policy 0, policy_version 811677 (0.0007) [2023-12-27 11:14:46,762][20381] Updated weights for policy 1, policy_version 811225 (0.0008) [2023-12-27 11:14:46,816][20381] Updated weights for policy 1, policy_version 811235 (0.0008) [2023-12-27 11:14:46,875][20381] Updated weights for policy 1, policy_version 811245 (0.0008) [2023-12-27 11:14:46,928][20381] Updated weights for policy 1, policy_version 811255 (0.0008) [2023-12-27 11:14:47,004][20347] Updated weights for policy 0, policy_version 811687 (0.0008) [2023-12-27 11:14:47,063][20347] Updated weights for policy 0, policy_version 811697 (0.0007) [2023-12-27 11:14:47,123][20347] Updated weights for policy 0, policy_version 811707 (0.0006) [2023-12-27 11:14:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 20216.2). Total num frames: 415531008. Throughput: 0: 9615.6, 1: 9596.4. Samples: 415539496. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '969.420')] [2023-12-27 11:14:47,671][20381] Updated weights for policy 1, policy_version 811265 (0.0008) [2023-12-27 11:14:47,737][20381] Updated weights for policy 1, policy_version 811275 (0.0006) [2023-12-27 11:14:47,795][20381] Updated weights for policy 1, policy_version 811285 (0.0007) [2023-12-27 11:14:47,817][20347] Updated weights for policy 0, policy_version 811717 (0.0006) [2023-12-27 11:14:47,872][20347] Updated weights for policy 0, policy_version 811727 (0.0006) [2023-12-27 11:14:47,931][20347] Updated weights for policy 0, policy_version 811737 (0.0006) [2023-12-27 11:14:48,510][20381] Updated weights for policy 1, policy_version 811295 (0.0007) [2023-12-27 11:14:48,564][20381] Updated weights for policy 1, policy_version 811305 (0.0008) [2023-12-27 11:14:48,613][20347] Updated weights for policy 0, policy_version 811747 (0.0007) [2023-12-27 11:14:48,623][20381] Updated weights for policy 1, policy_version 811315 (0.0009) [2023-12-27 11:14:48,672][20347] Updated weights for policy 0, policy_version 811757 (0.0007) [2023-12-27 11:14:48,734][20347] Updated weights for policy 0, policy_version 811767 (0.0008) [2023-12-27 11:14:49,386][20381] Updated weights for policy 1, policy_version 811325 (0.0007) [2023-12-27 11:14:49,430][20347] Updated weights for policy 0, policy_version 811777 (0.0008) [2023-12-27 11:14:49,446][20381] Updated weights for policy 1, policy_version 811335 (0.0009) [2023-12-27 11:14:49,492][20347] Updated weights for policy 0, policy_version 811787 (0.0006) [2023-12-27 11:14:49,506][20381] Updated weights for policy 1, policy_version 811345 (0.0007) [2023-12-27 11:14:49,559][20347] Updated weights for policy 0, policy_version 811797 (0.0006) [2023-12-27 11:14:49,622][20347] Updated weights for policy 0, policy_version 811807 (0.0008) [2023-12-27 11:14:50,230][20381] Updated weights for policy 1, policy_version 811355 (0.0007) [2023-12-27 11:14:50,280][20292] KL-divergence is very high: 314.1043 [2023-12-27 11:14:50,291][20381] Updated weights for policy 1, policy_version 811365 (0.0008) [2023-12-27 11:14:50,325][20292] KL-divergence is very high: 345.7114 [2023-12-27 11:14:50,335][20347] Updated weights for policy 0, policy_version 811817 (0.0008) [2023-12-27 11:14:50,351][20381] Updated weights for policy 1, policy_version 811375 (0.0007) [2023-12-27 11:14:50,376][20292] KL-divergence is very high: 151.8460 [2023-12-27 11:14:50,400][20347] Updated weights for policy 0, policy_version 811827 (0.0008) [2023-12-27 11:14:50,459][20347] Updated weights for policy 0, policy_version 811837 (0.0008) [2023-12-27 11:14:51,111][20381] Updated weights for policy 1, policy_version 811385 (0.0008) [2023-12-27 11:14:51,178][20381] Updated weights for policy 1, policy_version 811395 (0.0007) [2023-12-27 11:14:51,226][20347] Updated weights for policy 0, policy_version 811847 (0.0007) [2023-12-27 11:14:51,243][20381] Updated weights for policy 1, policy_version 811405 (0.0006) [2023-12-27 11:14:51,288][20347] Updated weights for policy 0, policy_version 811857 (0.0009) [2023-12-27 11:14:51,315][20381] Updated weights for policy 1, policy_version 811415 (0.0007) [2023-12-27 11:14:51,353][20347] Updated weights for policy 0, policy_version 811867 (0.0009) [2023-12-27 11:14:52,081][20381] Updated weights for policy 1, policy_version 811425 (0.0006) [2023-12-27 11:14:52,091][20347] Updated weights for policy 0, policy_version 811877 (0.0009) [2023-12-27 11:14:52,138][20381] Updated weights for policy 1, policy_version 811435 (0.0009) [2023-12-27 11:14:52,150][20347] Updated weights for policy 0, policy_version 811887 (0.0007) [2023-12-27 11:14:52,203][20381] Updated weights for policy 1, policy_version 811445 (0.0006) [2023-12-27 11:14:52,210][20347] Updated weights for policy 0, policy_version 811897 (0.0008) [2023-12-27 11:14:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19387.8, 300 sec: 20216.2). Total num frames: 415629312. Throughput: 0: 9620.5, 1: 9561.9. Samples: 415597828. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.100')] [2023-12-27 11:14:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000811448_207757312.pth... [2023-12-27 11:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000811904_207872000.pth... [2023-12-27 11:14:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000810328_207470592.pth [2023-12-27 11:14:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000810784_207585280.pth [2023-12-27 11:14:52,911][20347] Updated weights for policy 0, policy_version 811907 (0.0007) [2023-12-27 11:14:52,932][20381] Updated weights for policy 1, policy_version 811455 (0.0007) [2023-12-27 11:14:52,968][20347] Updated weights for policy 0, policy_version 811917 (0.0007) [2023-12-27 11:14:52,990][20381] Updated weights for policy 1, policy_version 811465 (0.0007) [2023-12-27 11:14:53,026][20347] Updated weights for policy 0, policy_version 811927 (0.0007) [2023-12-27 11:14:53,044][20381] Updated weights for policy 1, policy_version 811475 (0.0006) [2023-12-27 11:14:53,736][20381] Updated weights for policy 1, policy_version 811485 (0.0007) [2023-12-27 11:14:53,768][20347] Updated weights for policy 0, policy_version 811937 (0.0007) [2023-12-27 11:14:53,795][20381] Updated weights for policy 1, policy_version 811495 (0.0008) [2023-12-27 11:14:53,831][20347] Updated weights for policy 0, policy_version 811947 (0.0006) [2023-12-27 11:14:53,847][20381] Updated weights for policy 1, policy_version 811505 (0.0008) [2023-12-27 11:14:53,894][20347] Updated weights for policy 0, policy_version 811957 (0.0006) [2023-12-27 11:14:53,955][20347] Updated weights for policy 0, policy_version 811967 (0.0008) [2023-12-27 11:14:54,576][20347] Updated weights for policy 0, policy_version 811977 (0.0007) [2023-12-27 11:14:54,589][20381] Updated weights for policy 1, policy_version 811515 (0.0006) [2023-12-27 11:14:54,634][20347] Updated weights for policy 0, policy_version 811987 (0.0011) [2023-12-27 11:14:54,640][20381] Updated weights for policy 1, policy_version 811525 (0.0006) [2023-12-27 11:14:54,692][20347] Updated weights for policy 0, policy_version 811997 (0.0011) [2023-12-27 11:14:54,698][20381] Updated weights for policy 1, policy_version 811535 (0.0006) [2023-12-27 11:14:55,376][20347] Updated weights for policy 0, policy_version 812007 (0.0009) [2023-12-27 11:14:55,391][20291] KL-divergence is very high: 116.6662 [2023-12-27 11:14:55,432][20347] Updated weights for policy 0, policy_version 812017 (0.0010) [2023-12-27 11:14:55,435][20381] Updated weights for policy 1, policy_version 811545 (0.0007) [2023-12-27 11:14:55,491][20381] Updated weights for policy 1, policy_version 811555 (0.0008) [2023-12-27 11:14:55,494][20347] Updated weights for policy 0, policy_version 812027 (0.0010) [2023-12-27 11:14:55,546][20381] Updated weights for policy 1, policy_version 811565 (0.0007) [2023-12-27 11:14:55,608][20381] Updated weights for policy 1, policy_version 811575 (0.0009) [2023-12-27 11:14:56,160][20347] Updated weights for policy 0, policy_version 812037 (0.0009) [2023-12-27 11:14:56,215][20347] Updated weights for policy 0, policy_version 812047 (0.0010) [2023-12-27 11:14:56,272][20347] Updated weights for policy 0, policy_version 812057 (0.0011) [2023-12-27 11:14:56,336][20381] Updated weights for policy 1, policy_version 811585 (0.0007) [2023-12-27 11:14:56,390][20381] Updated weights for policy 1, policy_version 811595 (0.0006) [2023-12-27 11:14:56,459][20381] Updated weights for policy 1, policy_version 811605 (0.0007) [2023-12-27 11:14:56,968][20347] Updated weights for policy 0, policy_version 812067 (0.0011) [2023-12-27 11:14:57,020][20347] Updated weights for policy 0, policy_version 812077 (0.0008) [2023-12-27 11:14:57,073][20347] Updated weights for policy 0, policy_version 812087 (0.0007) [2023-12-27 11:14:57,168][20381] Updated weights for policy 1, policy_version 811615 (0.0009) [2023-12-27 11:14:57,229][20381] Updated weights for policy 1, policy_version 811625 (0.0010) [2023-12-27 11:14:57,291][20381] Updated weights for policy 1, policy_version 811635 (0.0010) [2023-12-27 11:14:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 20188.4). Total num frames: 415727616. Throughput: 0: 9634.1, 1: 9549.8. Samples: 415713444. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:14:57,363][19170] Avg episode reward: [(0, '971.010'), (1, '961.100')] [2023-12-27 11:14:57,771][20347] Updated weights for policy 0, policy_version 812097 (0.0010) [2023-12-27 11:14:57,827][20347] Updated weights for policy 0, policy_version 812107 (0.0010) [2023-12-27 11:14:57,880][20347] Updated weights for policy 0, policy_version 812117 (0.0009) [2023-12-27 11:14:57,943][20347] Updated weights for policy 0, policy_version 812127 (0.0010) [2023-12-27 11:14:57,975][20381] Updated weights for policy 1, policy_version 811645 (0.0008) [2023-12-27 11:14:58,030][20381] Updated weights for policy 1, policy_version 811655 (0.0010) [2023-12-27 11:14:58,088][20381] Updated weights for policy 1, policy_version 811665 (0.0010) [2023-12-27 11:14:58,730][20347] Updated weights for policy 0, policy_version 812137 (0.0008) [2023-12-27 11:14:58,792][20347] Updated weights for policy 0, policy_version 812147 (0.0008) [2023-12-27 11:14:58,859][20347] Updated weights for policy 0, policy_version 812157 (0.0008) [2023-12-27 11:14:58,901][20381] Updated weights for policy 1, policy_version 811675 (0.0010) [2023-12-27 11:14:58,979][20381] Updated weights for policy 1, policy_version 811685 (0.0008) [2023-12-27 11:14:59,042][20381] Updated weights for policy 1, policy_version 811695 (0.0008) [2023-12-27 11:14:59,603][20347] Updated weights for policy 0, policy_version 812167 (0.0007) [2023-12-27 11:14:59,667][20347] Updated weights for policy 0, policy_version 812177 (0.0006) [2023-12-27 11:14:59,734][20347] Updated weights for policy 0, policy_version 812187 (0.0008) [2023-12-27 11:14:59,824][20381] Updated weights for policy 1, policy_version 811705 (0.0007) [2023-12-27 11:14:59,884][20381] Updated weights for policy 1, policy_version 811715 (0.0007) [2023-12-27 11:14:59,946][20381] Updated weights for policy 1, policy_version 811725 (0.0008) [2023-12-27 11:15:00,003][20381] Updated weights for policy 1, policy_version 811735 (0.0007) [2023-12-27 11:15:00,441][20347] Updated weights for policy 0, policy_version 812197 (0.0008) [2023-12-27 11:15:00,496][20347] Updated weights for policy 0, policy_version 812207 (0.0008) [2023-12-27 11:15:00,551][20347] Updated weights for policy 0, policy_version 812217 (0.0007) [2023-12-27 11:15:00,671][20381] Updated weights for policy 1, policy_version 811745 (0.0007) [2023-12-27 11:15:00,722][20381] Updated weights for policy 1, policy_version 811755 (0.0007) [2023-12-27 11:15:00,771][20381] Updated weights for policy 1, policy_version 811765 (0.0007) [2023-12-27 11:15:01,239][20347] Updated weights for policy 0, policy_version 812227 (0.0007) [2023-12-27 11:15:01,302][20347] Updated weights for policy 0, policy_version 812237 (0.0006) [2023-12-27 11:15:01,359][20291] KL-divergence is very high: 1754.2085 [2023-12-27 11:15:01,367][20347] Updated weights for policy 0, policy_version 812247 (0.0008) [2023-12-27 11:15:01,409][20291] KL-divergence is very high: 122.5765 [2023-12-27 11:15:01,502][20381] Updated weights for policy 1, policy_version 811775 (0.0006) [2023-12-27 11:15:01,559][20381] Updated weights for policy 1, policy_version 811785 (0.0009) [2023-12-27 11:15:01,625][20381] Updated weights for policy 1, policy_version 811795 (0.0008) [2023-12-27 11:15:02,091][20347] Updated weights for policy 0, policy_version 812257 (0.0007) [2023-12-27 11:15:02,157][20347] Updated weights for policy 0, policy_version 812267 (0.0008) [2023-12-27 11:15:02,217][20347] Updated weights for policy 0, policy_version 812277 (0.0007) [2023-12-27 11:15:02,278][20347] Updated weights for policy 0, policy_version 812287 (0.0008) [2023-12-27 11:15:02,323][20381] Updated weights for policy 1, policy_version 811805 (0.0007) [2023-12-27 11:15:02,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 20132.9). Total num frames: 415817728. Throughput: 0: 9654.3, 1: 9541.4. Samples: 415828316. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:15:02,363][19170] Avg episode reward: [(0, '961.240'), (1, '980.570')] [2023-12-27 11:15:02,390][20381] Updated weights for policy 1, policy_version 811815 (0.0007) [2023-12-27 11:15:02,459][20381] Updated weights for policy 1, policy_version 811825 (0.0008) [2023-12-27 11:15:02,995][20347] Updated weights for policy 0, policy_version 812297 (0.0009) [2023-12-27 11:15:03,052][20347] Updated weights for policy 0, policy_version 812307 (0.0007) [2023-12-27 11:15:03,105][20381] Updated weights for policy 1, policy_version 811835 (0.0007) [2023-12-27 11:15:03,110][20347] Updated weights for policy 0, policy_version 812317 (0.0007) [2023-12-27 11:15:03,157][20381] Updated weights for policy 1, policy_version 811845 (0.0008) [2023-12-27 11:15:03,211][20381] Updated weights for policy 1, policy_version 811855 (0.0007) [2023-12-27 11:15:03,679][20347] Updated weights for policy 0, policy_version 812327 (0.0010) [2023-12-27 11:15:03,741][20347] Updated weights for policy 0, policy_version 812337 (0.0010) [2023-12-27 11:15:03,799][20347] Updated weights for policy 0, policy_version 812347 (0.0006) [2023-12-27 11:15:03,954][20381] Updated weights for policy 1, policy_version 811865 (0.0007) [2023-12-27 11:15:04,015][20381] Updated weights for policy 1, policy_version 811875 (0.0008) [2023-12-27 11:15:04,074][20381] Updated weights for policy 1, policy_version 811885 (0.0008) [2023-12-27 11:15:04,140][20381] Updated weights for policy 1, policy_version 811895 (0.0008) [2023-12-27 11:15:04,474][20347] Updated weights for policy 0, policy_version 812357 (0.0008) [2023-12-27 11:15:04,533][20347] Updated weights for policy 0, policy_version 812367 (0.0010) [2023-12-27 11:15:04,586][20347] Updated weights for policy 0, policy_version 812377 (0.0010) [2023-12-27 11:15:04,820][20381] Updated weights for policy 1, policy_version 811905 (0.0010) [2023-12-27 11:15:04,876][20381] Updated weights for policy 1, policy_version 811915 (0.0010) [2023-12-27 11:15:04,934][20381] Updated weights for policy 1, policy_version 811925 (0.0008) [2023-12-27 11:15:05,251][20347] Updated weights for policy 0, policy_version 812387 (0.0011) [2023-12-27 11:15:05,308][20347] Updated weights for policy 0, policy_version 812397 (0.0009) [2023-12-27 11:15:05,374][20347] Updated weights for policy 0, policy_version 812407 (0.0005) [2023-12-27 11:15:05,618][20381] Updated weights for policy 1, policy_version 811935 (0.0011) [2023-12-27 11:15:05,673][20381] Updated weights for policy 1, policy_version 811945 (0.0010) [2023-12-27 11:15:05,733][20381] Updated weights for policy 1, policy_version 811955 (0.0011) [2023-12-27 11:15:06,020][20347] Updated weights for policy 0, policy_version 812417 (0.0006) [2023-12-27 11:15:06,086][20347] Updated weights for policy 0, policy_version 812427 (0.0006) [2023-12-27 11:15:06,147][20347] Updated weights for policy 0, policy_version 812437 (0.0010) [2023-12-27 11:15:06,209][20347] Updated weights for policy 0, policy_version 812447 (0.0009) [2023-12-27 11:15:06,454][20381] Updated weights for policy 1, policy_version 811965 (0.0011) [2023-12-27 11:15:06,518][20381] Updated weights for policy 1, policy_version 811975 (0.0011) [2023-12-27 11:15:06,584][20381] Updated weights for policy 1, policy_version 811985 (0.0011) [2023-12-27 11:15:06,880][20347] Updated weights for policy 0, policy_version 812457 (0.0010) [2023-12-27 11:15:06,944][20347] Updated weights for policy 0, policy_version 812467 (0.0008) [2023-12-27 11:15:06,995][20347] Updated weights for policy 0, policy_version 812477 (0.0010) [2023-12-27 11:15:07,266][20381] Updated weights for policy 1, policy_version 811995 (0.0008) [2023-12-27 11:15:07,327][20381] Updated weights for policy 1, policy_version 812005 (0.0008) [2023-12-27 11:15:07,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 20132.9). Total num frames: 415916032. Throughput: 0: 9693.8, 1: 9534.5. Samples: 415889136. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:15:07,363][19170] Avg episode reward: [(0, '970.670'), (1, '990.290')] [2023-12-27 11:15:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000812480_208019456.pth... [2023-12-27 11:15:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000811328_207724544.pth [2023-12-27 11:15:07,390][20381] Updated weights for policy 1, policy_version 812015 (0.0006) [2023-12-27 11:15:07,445][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000812024_207904768.pth... [2023-12-27 11:15:07,448][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000810904_207618048.pth [2023-12-27 11:15:07,679][20347] Updated weights for policy 0, policy_version 812487 (0.0010) [2023-12-27 11:15:07,744][20347] Updated weights for policy 0, policy_version 812497 (0.0010) [2023-12-27 11:15:07,805][20347] Updated weights for policy 0, policy_version 812507 (0.0008) [2023-12-27 11:15:08,063][20381] Updated weights for policy 1, policy_version 812025 (0.0011) [2023-12-27 11:15:08,121][20381] Updated weights for policy 1, policy_version 812035 (0.0006) [2023-12-27 11:15:08,174][20381] Updated weights for policy 1, policy_version 812045 (0.0006) [2023-12-27 11:15:08,237][20381] Updated weights for policy 1, policy_version 812055 (0.0009) [2023-12-27 11:15:08,463][20347] Updated weights for policy 0, policy_version 812517 (0.0008) [2023-12-27 11:15:08,521][20347] Updated weights for policy 0, policy_version 812527 (0.0011) [2023-12-27 11:15:08,583][20347] Updated weights for policy 0, policy_version 812537 (0.0010) [2023-12-27 11:15:08,938][20381] Updated weights for policy 1, policy_version 812065 (0.0010) [2023-12-27 11:15:08,996][20381] Updated weights for policy 1, policy_version 812075 (0.0011) [2023-12-27 11:15:09,061][20381] Updated weights for policy 1, policy_version 812085 (0.0009) [2023-12-27 11:15:09,357][20347] Updated weights for policy 0, policy_version 812547 (0.0010) [2023-12-27 11:15:09,428][20347] Updated weights for policy 0, policy_version 812557 (0.0009) [2023-12-27 11:15:09,494][20347] Updated weights for policy 0, policy_version 812567 (0.0008) [2023-12-27 11:15:09,767][20381] Updated weights for policy 1, policy_version 812095 (0.0010) [2023-12-27 11:15:09,838][20381] Updated weights for policy 1, policy_version 812105 (0.0010) [2023-12-27 11:15:09,893][20381] Updated weights for policy 1, policy_version 812115 (0.0008) [2023-12-27 11:15:10,307][20347] Updated weights for policy 0, policy_version 812577 (0.0008) [2023-12-27 11:15:10,378][20347] Updated weights for policy 0, policy_version 812587 (0.0007) [2023-12-27 11:15:10,440][20347] Updated weights for policy 0, policy_version 812597 (0.0007) [2023-12-27 11:15:10,501][20347] Updated weights for policy 0, policy_version 812607 (0.0008) [2023-12-27 11:15:10,608][20381] Updated weights for policy 1, policy_version 812125 (0.0007) [2023-12-27 11:15:10,658][20381] Updated weights for policy 1, policy_version 812135 (0.0006) [2023-12-27 11:15:10,709][20381] Updated weights for policy 1, policy_version 812145 (0.0006) [2023-12-27 11:15:11,229][20347] Updated weights for policy 0, policy_version 812617 (0.0008) [2023-12-27 11:15:11,295][20347] Updated weights for policy 0, policy_version 812627 (0.0008) [2023-12-27 11:15:11,351][20347] Updated weights for policy 0, policy_version 812637 (0.0008) [2023-12-27 11:15:11,488][20381] Updated weights for policy 1, policy_version 812155 (0.0008) [2023-12-27 11:15:11,549][20381] Updated weights for policy 1, policy_version 812165 (0.0008) [2023-12-27 11:15:11,607][20381] Updated weights for policy 1, policy_version 812175 (0.0008) [2023-12-27 11:15:12,118][20347] Updated weights for policy 0, policy_version 812647 (0.0007) [2023-12-27 11:15:12,189][20347] Updated weights for policy 0, policy_version 812657 (0.0007) [2023-12-27 11:15:12,252][20347] Updated weights for policy 0, policy_version 812667 (0.0007) [2023-12-27 11:15:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 20105.1). Total num frames: 416014336. Throughput: 0: 9730.1, 1: 9590.4. Samples: 416006716. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:15:12,363][19170] Avg episode reward: [(0, '970.700'), (1, '980.580')] [2023-12-27 11:15:12,379][20381] Updated weights for policy 1, policy_version 812185 (0.0007) [2023-12-27 11:15:12,443][20381] Updated weights for policy 1, policy_version 812195 (0.0008) [2023-12-27 11:15:12,505][20381] Updated weights for policy 1, policy_version 812205 (0.0007) [2023-12-27 11:15:12,572][20381] Updated weights for policy 1, policy_version 812215 (0.0008) [2023-12-27 11:15:13,001][20347] Updated weights for policy 0, policy_version 812677 (0.0006) [2023-12-27 11:15:13,058][20347] Updated weights for policy 0, policy_version 812687 (0.0008) [2023-12-27 11:15:13,109][20347] Updated weights for policy 0, policy_version 812697 (0.0008) [2023-12-27 11:15:13,279][20381] Updated weights for policy 1, policy_version 812225 (0.0008) [2023-12-27 11:15:13,332][20381] Updated weights for policy 1, policy_version 812235 (0.0008) [2023-12-27 11:15:13,386][20381] Updated weights for policy 1, policy_version 812245 (0.0008) [2023-12-27 11:15:13,781][20347] Updated weights for policy 0, policy_version 812707 (0.0006) [2023-12-27 11:15:13,834][20347] Updated weights for policy 0, policy_version 812717 (0.0008) [2023-12-27 11:15:13,895][20347] Updated weights for policy 0, policy_version 812727 (0.0008) [2023-12-27 11:15:14,064][20381] Updated weights for policy 1, policy_version 812255 (0.0007) [2023-12-27 11:15:14,119][20381] Updated weights for policy 1, policy_version 812265 (0.0008) [2023-12-27 11:15:14,180][20381] Updated weights for policy 1, policy_version 812275 (0.0007) [2023-12-27 11:15:14,553][20347] Updated weights for policy 0, policy_version 812737 (0.0008) [2023-12-27 11:15:14,602][20347] Updated weights for policy 0, policy_version 812747 (0.0008) [2023-12-27 11:15:14,658][20347] Updated weights for policy 0, policy_version 812757 (0.0007) [2023-12-27 11:15:14,716][20347] Updated weights for policy 0, policy_version 812767 (0.0007) [2023-12-27 11:15:14,914][20381] Updated weights for policy 1, policy_version 812285 (0.0006) [2023-12-27 11:15:14,974][20381] Updated weights for policy 1, policy_version 812295 (0.0007) [2023-12-27 11:15:15,031][20381] Updated weights for policy 1, policy_version 812305 (0.0007) [2023-12-27 11:15:15,448][20347] Updated weights for policy 0, policy_version 812777 (0.0007) [2023-12-27 11:15:15,501][20347] Updated weights for policy 0, policy_version 812787 (0.0007) [2023-12-27 11:15:15,558][20347] Updated weights for policy 0, policy_version 812797 (0.0008) [2023-12-27 11:15:15,726][20381] Updated weights for policy 1, policy_version 812315 (0.0007) [2023-12-27 11:15:15,779][20381] Updated weights for policy 1, policy_version 812325 (0.0006) [2023-12-27 11:15:15,832][20381] Updated weights for policy 1, policy_version 812335 (0.0006) [2023-12-27 11:15:16,219][20347] Updated weights for policy 0, policy_version 812807 (0.0007) [2023-12-27 11:15:16,278][20347] Updated weights for policy 0, policy_version 812817 (0.0007) [2023-12-27 11:15:16,337][20347] Updated weights for policy 0, policy_version 812827 (0.0007) [2023-12-27 11:15:16,525][20381] Updated weights for policy 1, policy_version 812345 (0.0006) [2023-12-27 11:15:16,586][20381] Updated weights for policy 1, policy_version 812355 (0.0006) [2023-12-27 11:15:16,650][20381] Updated weights for policy 1, policy_version 812365 (0.0006) [2023-12-27 11:15:16,711][20381] Updated weights for policy 1, policy_version 812375 (0.0008) [2023-12-27 11:15:17,077][20347] Updated weights for policy 0, policy_version 812837 (0.0008) [2023-12-27 11:15:17,137][20347] Updated weights for policy 0, policy_version 812847 (0.0007) [2023-12-27 11:15:17,200][20347] Updated weights for policy 0, policy_version 812857 (0.0007) [2023-12-27 11:15:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 20105.1). Total num frames: 416112640. Throughput: 0: 9747.1, 1: 9584.5. Samples: 416123444. Policy #0 lag: (min: 3.0, avg: 11.2, max: 35.0) [2023-12-27 11:15:17,363][19170] Avg episode reward: [(0, '960.920'), (1, '970.890')] [2023-12-27 11:15:17,392][20381] Updated weights for policy 1, policy_version 812385 (0.0009) [2023-12-27 11:15:17,457][20381] Updated weights for policy 1, policy_version 812395 (0.0008) [2023-12-27 11:15:17,518][20381] Updated weights for policy 1, policy_version 812405 (0.0008) [2023-12-27 11:15:17,832][20347] Updated weights for policy 0, policy_version 812867 (0.0006) [2023-12-27 11:15:17,885][20347] Updated weights for policy 0, policy_version 812877 (0.0006) [2023-12-27 11:15:17,943][20347] Updated weights for policy 0, policy_version 812887 (0.0006) [2023-12-27 11:15:18,262][20381] Updated weights for policy 1, policy_version 812415 (0.0007) [2023-12-27 11:15:18,331][20381] Updated weights for policy 1, policy_version 812425 (0.0007) [2023-12-27 11:15:18,396][20381] Updated weights for policy 1, policy_version 812435 (0.0007) [2023-12-27 11:15:18,565][20347] Updated weights for policy 0, policy_version 812897 (0.0007) [2023-12-27 11:15:18,626][20347] Updated weights for policy 0, policy_version 812907 (0.0006) [2023-12-27 11:15:18,690][20347] Updated weights for policy 0, policy_version 812917 (0.0008) [2023-12-27 11:15:18,750][20347] Updated weights for policy 0, policy_version 812927 (0.0007) [2023-12-27 11:15:19,098][20381] Updated weights for policy 1, policy_version 812445 (0.0009) [2023-12-27 11:15:19,165][20381] Updated weights for policy 1, policy_version 812455 (0.0009) [2023-12-27 11:15:19,231][20381] Updated weights for policy 1, policy_version 812465 (0.0008) [2023-12-27 11:15:19,426][20347] Updated weights for policy 0, policy_version 812937 (0.0007) [2023-12-27 11:15:19,489][20347] Updated weights for policy 0, policy_version 812947 (0.0007) [2023-12-27 11:15:19,548][20347] Updated weights for policy 0, policy_version 812957 (0.0007) [2023-12-27 11:15:19,987][20381] Updated weights for policy 1, policy_version 812475 (0.0008) [2023-12-27 11:15:20,056][20381] Updated weights for policy 1, policy_version 812485 (0.0008) [2023-12-27 11:15:20,121][20381] Updated weights for policy 1, policy_version 812495 (0.0009) [2023-12-27 11:15:20,257][20347] Updated weights for policy 0, policy_version 812967 (0.0007) [2023-12-27 11:15:20,318][20347] Updated weights for policy 0, policy_version 812977 (0.0007) [2023-12-27 11:15:20,379][20347] Updated weights for policy 0, policy_version 812987 (0.0008) [2023-12-27 11:15:20,886][20381] Updated weights for policy 1, policy_version 812505 (0.0008) [2023-12-27 11:15:20,949][20381] Updated weights for policy 1, policy_version 812515 (0.0007) [2023-12-27 11:15:21,010][20381] Updated weights for policy 1, policy_version 812525 (0.0008) [2023-12-27 11:15:21,055][20347] Updated weights for policy 0, policy_version 812997 (0.0011) [2023-12-27 11:15:21,077][20381] Updated weights for policy 1, policy_version 812535 (0.0008) [2023-12-27 11:15:21,112][20347] Updated weights for policy 0, policy_version 813007 (0.0008) [2023-12-27 11:15:21,179][20347] Updated weights for policy 0, policy_version 813017 (0.0010) [2023-12-27 11:15:21,894][20381] Updated weights for policy 1, policy_version 812545 (0.0008) [2023-12-27 11:15:21,964][20347] Updated weights for policy 0, policy_version 813027 (0.0007) [2023-12-27 11:15:21,965][20381] Updated weights for policy 1, policy_version 812555 (0.0006) [2023-12-27 11:15:22,026][20381] Updated weights for policy 1, policy_version 812565 (0.0009) [2023-12-27 11:15:22,033][20347] Updated weights for policy 0, policy_version 813037 (0.0006) [2023-12-27 11:15:22,101][20347] Updated weights for policy 0, policy_version 813047 (0.0007) [2023-12-27 11:15:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 20049.6). Total num frames: 416210944. Throughput: 0: 9777.7, 1: 9561.6. Samples: 416181764. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:22,363][19170] Avg episode reward: [(0, '960.930'), (1, '990.310')] [2023-12-27 11:15:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000813056_208166912.pth... [2023-12-27 11:15:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000812568_208044032.pth... [2023-12-27 11:15:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000811904_207872000.pth [2023-12-27 11:15:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000811448_207757312.pth [2023-12-27 11:15:22,752][20381] Updated weights for policy 1, policy_version 812575 (0.0008) [2023-12-27 11:15:22,793][20347] Updated weights for policy 0, policy_version 813057 (0.0008) [2023-12-27 11:15:22,811][20381] Updated weights for policy 1, policy_version 812585 (0.0007) [2023-12-27 11:15:22,853][20347] Updated weights for policy 0, policy_version 813067 (0.0008) [2023-12-27 11:15:22,871][20381] Updated weights for policy 1, policy_version 812595 (0.0006) [2023-12-27 11:15:22,914][20347] Updated weights for policy 0, policy_version 813077 (0.0008) [2023-12-27 11:15:22,981][20347] Updated weights for policy 0, policy_version 813087 (0.0007) [2023-12-27 11:15:23,569][20381] Updated weights for policy 1, policy_version 812605 (0.0007) [2023-12-27 11:15:23,630][20381] Updated weights for policy 1, policy_version 812615 (0.0006) [2023-12-27 11:15:23,695][20381] Updated weights for policy 1, policy_version 812625 (0.0007) [2023-12-27 11:15:23,700][20347] Updated weights for policy 0, policy_version 813097 (0.0006) [2023-12-27 11:15:23,766][20347] Updated weights for policy 0, policy_version 813107 (0.0008) [2023-12-27 11:15:23,841][20347] Updated weights for policy 0, policy_version 813117 (0.0007) [2023-12-27 11:15:24,427][20381] Updated weights for policy 1, policy_version 812635 (0.0009) [2023-12-27 11:15:24,487][20381] Updated weights for policy 1, policy_version 812645 (0.0007) [2023-12-27 11:15:24,547][20381] Updated weights for policy 1, policy_version 812655 (0.0007) [2023-12-27 11:15:24,570][20347] Updated weights for policy 0, policy_version 813127 (0.0008) [2023-12-27 11:15:24,621][20347] Updated weights for policy 0, policy_version 813137 (0.0007) [2023-12-27 11:15:24,681][20347] Updated weights for policy 0, policy_version 813147 (0.0008) [2023-12-27 11:15:25,247][20381] Updated weights for policy 1, policy_version 812665 (0.0008) [2023-12-27 11:15:25,307][20381] Updated weights for policy 1, policy_version 812675 (0.0009) [2023-12-27 11:15:25,351][20347] Updated weights for policy 0, policy_version 813157 (0.0008) [2023-12-27 11:15:25,363][20381] Updated weights for policy 1, policy_version 812685 (0.0007) [2023-12-27 11:15:25,413][20347] Updated weights for policy 0, policy_version 813167 (0.0008) [2023-12-27 11:15:25,413][20381] Updated weights for policy 1, policy_version 812695 (0.0006) [2023-12-27 11:15:25,474][20347] Updated weights for policy 0, policy_version 813177 (0.0008) [2023-12-27 11:15:26,137][20381] Updated weights for policy 1, policy_version 812705 (0.0008) [2023-12-27 11:15:26,154][20347] Updated weights for policy 0, policy_version 813187 (0.0007) [2023-12-27 11:15:26,200][20381] Updated weights for policy 1, policy_version 812715 (0.0008) [2023-12-27 11:15:26,212][20347] Updated weights for policy 0, policy_version 813197 (0.0007) [2023-12-27 11:15:26,265][20381] Updated weights for policy 1, policy_version 812725 (0.0009) [2023-12-27 11:15:26,272][20347] Updated weights for policy 0, policy_version 813207 (0.0007) [2023-12-27 11:15:26,949][20381] Updated weights for policy 1, policy_version 812735 (0.0009) [2023-12-27 11:15:26,958][20347] Updated weights for policy 0, policy_version 813217 (0.0008) [2023-12-27 11:15:27,008][20381] Updated weights for policy 1, policy_version 812745 (0.0007) [2023-12-27 11:15:27,025][20347] Updated weights for policy 0, policy_version 813227 (0.0008) [2023-12-27 11:15:27,071][20381] Updated weights for policy 1, policy_version 812755 (0.0007) [2023-12-27 11:15:27,093][20347] Updated weights for policy 0, policy_version 813237 (0.0008) [2023-12-27 11:15:27,153][20347] Updated weights for policy 0, policy_version 813247 (0.0008) [2023-12-27 11:15:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 20049.6). Total num frames: 416309248. Throughput: 0: 9778.8, 1: 9570.0. Samples: 416296232. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:27,363][19170] Avg episode reward: [(0, '941.600'), (1, '1000.000')] [2023-12-27 11:15:27,746][20381] Updated weights for policy 1, policy_version 812765 (0.0007) [2023-12-27 11:15:27,799][20347] Updated weights for policy 0, policy_version 813257 (0.0009) [2023-12-27 11:15:27,804][20381] Updated weights for policy 1, policy_version 812775 (0.0009) [2023-12-27 11:15:27,857][20347] Updated weights for policy 0, policy_version 813267 (0.0008) [2023-12-27 11:15:27,865][20381] Updated weights for policy 1, policy_version 812785 (0.0010) [2023-12-27 11:15:27,916][20347] Updated weights for policy 0, policy_version 813277 (0.0006) [2023-12-27 11:15:28,546][20347] Updated weights for policy 0, policy_version 813287 (0.0007) [2023-12-27 11:15:28,616][20347] Updated weights for policy 0, policy_version 813297 (0.0007) [2023-12-27 11:15:28,636][20381] Updated weights for policy 1, policy_version 812795 (0.0010) [2023-12-27 11:15:28,681][20347] Updated weights for policy 0, policy_version 813307 (0.0008) [2023-12-27 11:15:28,700][20381] Updated weights for policy 1, policy_version 812805 (0.0007) [2023-12-27 11:15:28,757][20381] Updated weights for policy 1, policy_version 812815 (0.0007) [2023-12-27 11:15:29,428][20347] Updated weights for policy 0, policy_version 813317 (0.0007) [2023-12-27 11:15:29,484][20381] Updated weights for policy 1, policy_version 812825 (0.0007) [2023-12-27 11:15:29,495][20347] Updated weights for policy 0, policy_version 813327 (0.0007) [2023-12-27 11:15:29,540][20381] Updated weights for policy 1, policy_version 812835 (0.0008) [2023-12-27 11:15:29,557][20347] Updated weights for policy 0, policy_version 813337 (0.0007) [2023-12-27 11:15:29,601][20381] Updated weights for policy 1, policy_version 812845 (0.0006) [2023-12-27 11:15:29,667][20381] Updated weights for policy 1, policy_version 812855 (0.0007) [2023-12-27 11:15:30,296][20347] Updated weights for policy 0, policy_version 813347 (0.0007) [2023-12-27 11:15:30,358][20347] Updated weights for policy 0, policy_version 813357 (0.0007) [2023-12-27 11:15:30,394][20291] KL-divergence is very high: 202.2258 [2023-12-27 11:15:30,413][20291] KL-divergence is very high: 104.0996 [2023-12-27 11:15:30,418][20347] Updated weights for policy 0, policy_version 813367 (0.0006) [2023-12-27 11:15:30,421][20381] Updated weights for policy 1, policy_version 812865 (0.0008) [2023-12-27 11:15:30,441][20291] KL-divergence is very high: 159.2251 [2023-12-27 11:15:30,476][20381] Updated weights for policy 1, policy_version 812875 (0.0008) [2023-12-27 11:15:30,529][20381] Updated weights for policy 1, policy_version 812885 (0.0008) [2023-12-27 11:15:31,077][20347] Updated weights for policy 0, policy_version 813377 (0.0007) [2023-12-27 11:15:31,136][20347] Updated weights for policy 0, policy_version 813387 (0.0006) [2023-12-27 11:15:31,208][20347] Updated weights for policy 0, policy_version 813397 (0.0008) [2023-12-27 11:15:31,265][20291] KL-divergence is very high: 128.7788 [2023-12-27 11:15:31,271][20347] Updated weights for policy 0, policy_version 813407 (0.0008) [2023-12-27 11:15:31,306][20381] Updated weights for policy 1, policy_version 812895 (0.0008) [2023-12-27 11:15:31,367][20381] Updated weights for policy 1, policy_version 812905 (0.0008) [2023-12-27 11:15:31,441][20381] Updated weights for policy 1, policy_version 812915 (0.0008) [2023-12-27 11:15:32,112][20347] Updated weights for policy 0, policy_version 813417 (0.0008) [2023-12-27 11:15:32,143][20381] Updated weights for policy 1, policy_version 812925 (0.0007) [2023-12-27 11:15:32,173][20347] Updated weights for policy 0, policy_version 813427 (0.0008) [2023-12-27 11:15:32,203][20381] Updated weights for policy 1, policy_version 812935 (0.0007) [2023-12-27 11:15:32,236][20347] Updated weights for policy 0, policy_version 813437 (0.0008) [2023-12-27 11:15:32,273][20381] Updated weights for policy 1, policy_version 812945 (0.0007) [2023-12-27 11:15:32,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 20049.6). Total num frames: 416407552. Throughput: 0: 9816.3, 1: 9566.7. Samples: 416411728. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:32,364][19170] Avg episode reward: [(0, '863.880'), (1, '1000.000')] [2023-12-27 11:15:32,976][20381] Updated weights for policy 1, policy_version 812955 (0.0007) [2023-12-27 11:15:33,008][20347] Updated weights for policy 0, policy_version 813447 (0.0008) [2023-12-27 11:15:33,031][20381] Updated weights for policy 1, policy_version 812965 (0.0007) [2023-12-27 11:15:33,068][20347] Updated weights for policy 0, policy_version 813457 (0.0008) [2023-12-27 11:15:33,088][20381] Updated weights for policy 1, policy_version 812975 (0.0007) [2023-12-27 11:15:33,133][20347] Updated weights for policy 0, policy_version 813467 (0.0008) [2023-12-27 11:15:33,743][20381] Updated weights for policy 1, policy_version 812985 (0.0007) [2023-12-27 11:15:33,805][20381] Updated weights for policy 1, policy_version 812995 (0.0008) [2023-12-27 11:15:33,862][20381] Updated weights for policy 1, policy_version 813005 (0.0007) [2023-12-27 11:15:33,885][20347] Updated weights for policy 0, policy_version 813477 (0.0007) [2023-12-27 11:15:33,928][20381] Updated weights for policy 1, policy_version 813015 (0.0008) [2023-12-27 11:15:33,949][20347] Updated weights for policy 0, policy_version 813487 (0.0007) [2023-12-27 11:15:34,012][20347] Updated weights for policy 0, policy_version 813497 (0.0008) [2023-12-27 11:15:34,694][20381] Updated weights for policy 1, policy_version 813025 (0.0008) [2023-12-27 11:15:34,751][20347] Updated weights for policy 0, policy_version 813507 (0.0008) [2023-12-27 11:15:34,757][20381] Updated weights for policy 1, policy_version 813035 (0.0007) [2023-12-27 11:15:34,805][20347] Updated weights for policy 0, policy_version 813517 (0.0011) [2023-12-27 11:15:34,815][20381] Updated weights for policy 1, policy_version 813045 (0.0006) [2023-12-27 11:15:34,863][20347] Updated weights for policy 0, policy_version 813527 (0.0011) [2023-12-27 11:15:35,557][20381] Updated weights for policy 1, policy_version 813055 (0.0006) [2023-12-27 11:15:35,563][20347] Updated weights for policy 0, policy_version 813537 (0.0010) [2023-12-27 11:15:35,617][20381] Updated weights for policy 1, policy_version 813065 (0.0007) [2023-12-27 11:15:35,623][20347] Updated weights for policy 0, policy_version 813547 (0.0009) [2023-12-27 11:15:35,676][20381] Updated weights for policy 1, policy_version 813075 (0.0006) [2023-12-27 11:15:35,685][20347] Updated weights for policy 0, policy_version 813557 (0.0010) [2023-12-27 11:15:35,741][20347] Updated weights for policy 0, policy_version 813567 (0.0010) [2023-12-27 11:15:36,445][20347] Updated weights for policy 0, policy_version 813577 (0.0011) [2023-12-27 11:15:36,453][20381] Updated weights for policy 1, policy_version 813085 (0.0007) [2023-12-27 11:15:36,512][20347] Updated weights for policy 0, policy_version 813587 (0.0011) [2023-12-27 11:15:36,514][20381] Updated weights for policy 1, policy_version 813095 (0.0008) [2023-12-27 11:15:36,574][20347] Updated weights for policy 0, policy_version 813597 (0.0010) [2023-12-27 11:15:36,575][20381] Updated weights for policy 1, policy_version 813105 (0.0008) [2023-12-27 11:15:37,245][20347] Updated weights for policy 0, policy_version 813607 (0.0006) [2023-12-27 11:15:37,308][20381] Updated weights for policy 1, policy_version 813115 (0.0008) [2023-12-27 11:15:37,310][20347] Updated weights for policy 0, policy_version 813617 (0.0008) [2023-12-27 11:15:37,362][19170] Fps is (10 sec: 18022.4, 60 sec: 19114.7, 300 sec: 19966.3). Total num frames: 416489472. Throughput: 0: 9772.2, 1: 9574.5. Samples: 416468428. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:37,363][19170] Avg episode reward: [(0, '804.980'), (1, '1000.000')] [2023-12-27 11:15:37,372][20381] Updated weights for policy 1, policy_version 813125 (0.0006) [2023-12-27 11:15:37,374][20347] Updated weights for policy 0, policy_version 813627 (0.0006) [2023-12-27 11:15:37,404][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000813632_208314368.pth... [2023-12-27 11:15:37,409][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000812480_208019456.pth [2023-12-27 11:15:37,436][20381] Updated weights for policy 1, policy_version 813135 (0.0007) [2023-12-27 11:15:37,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000813144_208191488.pth... [2023-12-27 11:15:37,496][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000812024_207904768.pth [2023-12-27 11:15:38,160][20347] Updated weights for policy 0, policy_version 813637 (0.0008) [2023-12-27 11:15:38,215][20347] Updated weights for policy 0, policy_version 813647 (0.0009) [2023-12-27 11:15:38,235][20381] Updated weights for policy 1, policy_version 813145 (0.0007) [2023-12-27 11:15:38,268][20347] Updated weights for policy 0, policy_version 813657 (0.0009) [2023-12-27 11:15:38,288][20381] Updated weights for policy 1, policy_version 813155 (0.0009) [2023-12-27 11:15:38,349][20381] Updated weights for policy 1, policy_version 813165 (0.0009) [2023-12-27 11:15:38,408][20381] Updated weights for policy 1, policy_version 813175 (0.0008) [2023-12-27 11:15:39,105][20347] Updated weights for policy 0, policy_version 813667 (0.0009) [2023-12-27 11:15:39,161][20347] Updated weights for policy 0, policy_version 813677 (0.0008) [2023-12-27 11:15:39,219][20347] Updated weights for policy 0, policy_version 813687 (0.0008) [2023-12-27 11:15:39,230][20381] Updated weights for policy 1, policy_version 813185 (0.0008) [2023-12-27 11:15:39,290][20381] Updated weights for policy 1, policy_version 813195 (0.0008) [2023-12-27 11:15:39,352][20381] Updated weights for policy 1, policy_version 813205 (0.0008) [2023-12-27 11:15:40,009][20347] Updated weights for policy 0, policy_version 813697 (0.0008) [2023-12-27 11:15:40,064][20347] Updated weights for policy 0, policy_version 813707 (0.0008) [2023-12-27 11:15:40,125][20347] Updated weights for policy 0, policy_version 813717 (0.0007) [2023-12-27 11:15:40,168][20381] Updated weights for policy 1, policy_version 813215 (0.0008) [2023-12-27 11:15:40,177][20347] Updated weights for policy 0, policy_version 813727 (0.0007) [2023-12-27 11:15:40,219][20381] Updated weights for policy 1, policy_version 813225 (0.0008) [2023-12-27 11:15:40,279][20381] Updated weights for policy 1, policy_version 813235 (0.0008) [2023-12-27 11:15:40,856][20347] Updated weights for policy 0, policy_version 813737 (0.0007) [2023-12-27 11:15:40,909][20347] Updated weights for policy 0, policy_version 813747 (0.0008) [2023-12-27 11:15:40,960][20347] Updated weights for policy 0, policy_version 813757 (0.0007) [2023-12-27 11:15:40,973][20381] Updated weights for policy 1, policy_version 813245 (0.0008) [2023-12-27 11:15:41,031][20381] Updated weights for policy 1, policy_version 813255 (0.0007) [2023-12-27 11:15:41,090][20381] Updated weights for policy 1, policy_version 813265 (0.0007) [2023-12-27 11:15:41,746][20347] Updated weights for policy 0, policy_version 813767 (0.0007) [2023-12-27 11:15:41,810][20347] Updated weights for policy 0, policy_version 813777 (0.0008) [2023-12-27 11:15:41,816][20381] Updated weights for policy 1, policy_version 813275 (0.0007) [2023-12-27 11:15:41,869][20347] Updated weights for policy 0, policy_version 813787 (0.0008) [2023-12-27 11:15:41,880][20381] Updated weights for policy 1, policy_version 813285 (0.0007) [2023-12-27 11:15:41,939][20381] Updated weights for policy 1, policy_version 813295 (0.0007) [2023-12-27 11:15:42,362][19170] Fps is (10 sec: 18022.5, 60 sec: 19251.2, 300 sec: 19938.5). Total num frames: 416587776. Throughput: 0: 9720.7, 1: 9529.8. Samples: 416579716. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:42,363][19170] Avg episode reward: [(0, '815.040'), (1, '1000.000')] [2023-12-27 11:15:42,614][20347] Updated weights for policy 0, policy_version 813797 (0.0007) [2023-12-27 11:15:42,659][20381] Updated weights for policy 1, policy_version 813305 (0.0008) [2023-12-27 11:15:42,673][20347] Updated weights for policy 0, policy_version 813807 (0.0007) [2023-12-27 11:15:42,724][20381] Updated weights for policy 1, policy_version 813315 (0.0007) [2023-12-27 11:15:42,734][20347] Updated weights for policy 0, policy_version 813817 (0.0006) [2023-12-27 11:15:42,789][20381] Updated weights for policy 1, policy_version 813325 (0.0006) [2023-12-27 11:15:42,858][20381] Updated weights for policy 1, policy_version 813335 (0.0010) [2023-12-27 11:15:43,451][20347] Updated weights for policy 0, policy_version 813827 (0.0006) [2023-12-27 11:15:43,476][20381] Updated weights for policy 1, policy_version 813345 (0.0011) [2023-12-27 11:15:43,501][20292] KL-divergence is very high: 616.4577 [2023-12-27 11:15:43,501][20347] Updated weights for policy 0, policy_version 813837 (0.0006) [2023-12-27 11:15:43,536][20381] Updated weights for policy 1, policy_version 813355 (0.0009) [2023-12-27 11:15:43,555][20347] Updated weights for policy 0, policy_version 813847 (0.0007) [2023-12-27 11:15:43,566][20292] KL-divergence is very high: 104.5439 [2023-12-27 11:15:43,596][20381] Updated weights for policy 1, policy_version 813365 (0.0008) [2023-12-27 11:15:43,596][20292] KL-divergence is very high: 1491.8959 [2023-12-27 11:15:44,204][20381] Updated weights for policy 1, policy_version 813375 (0.0010) [2023-12-27 11:15:44,262][20381] Updated weights for policy 1, policy_version 813385 (0.0010) [2023-12-27 11:15:44,288][20347] Updated weights for policy 0, policy_version 813857 (0.0007) [2023-12-27 11:15:44,320][20381] Updated weights for policy 1, policy_version 813395 (0.0010) [2023-12-27 11:15:44,340][20347] Updated weights for policy 0, policy_version 813867 (0.0005) [2023-12-27 11:15:44,399][20347] Updated weights for policy 0, policy_version 813877 (0.0007) [2023-12-27 11:15:44,457][20347] Updated weights for policy 0, policy_version 813887 (0.0007) [2023-12-27 11:15:44,943][20381] Updated weights for policy 1, policy_version 813405 (0.0011) [2023-12-27 11:15:45,006][20381] Updated weights for policy 1, policy_version 813415 (0.0011) [2023-12-27 11:15:45,069][20381] Updated weights for policy 1, policy_version 813425 (0.0011) [2023-12-27 11:15:45,212][20347] Updated weights for policy 0, policy_version 813897 (0.0008) [2023-12-27 11:15:45,274][20347] Updated weights for policy 0, policy_version 813907 (0.0007) [2023-12-27 11:15:45,330][20347] Updated weights for policy 0, policy_version 813917 (0.0007) [2023-12-27 11:15:45,735][20381] Updated weights for policy 1, policy_version 813435 (0.0011) [2023-12-27 11:15:45,786][20381] Updated weights for policy 1, policy_version 813445 (0.0010) [2023-12-27 11:15:45,844][20381] Updated weights for policy 1, policy_version 813455 (0.0010) [2023-12-27 11:15:45,992][20347] Updated weights for policy 0, policy_version 813927 (0.0009) [2023-12-27 11:15:46,037][20347] Updated weights for policy 0, policy_version 813937 (0.0009) [2023-12-27 11:15:46,081][20347] Updated weights for policy 0, policy_version 813947 (0.0006) [2023-12-27 11:15:46,452][20381] Updated weights for policy 1, policy_version 813465 (0.0009) [2023-12-27 11:15:46,499][20381] Updated weights for policy 1, policy_version 813475 (0.0005) [2023-12-27 11:15:46,550][20381] Updated weights for policy 1, policy_version 813485 (0.0010) [2023-12-27 11:15:46,602][20381] Updated weights for policy 1, policy_version 813495 (0.0010) [2023-12-27 11:15:46,645][20347] Updated weights for policy 0, policy_version 813957 (0.0005) [2023-12-27 11:15:46,705][20347] Updated weights for policy 0, policy_version 813967 (0.0007) [2023-12-27 11:15:46,752][20347] Updated weights for policy 0, policy_version 813977 (0.0008) [2023-12-27 11:15:47,263][20381] Updated weights for policy 1, policy_version 813505 (0.0010) [2023-12-27 11:15:47,317][20381] Updated weights for policy 1, policy_version 813515 (0.0010) [2023-12-27 11:15:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19938.5). Total num frames: 416686080. Throughput: 0: 9754.1, 1: 9657.6. Samples: 416701840. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:47,363][19170] Avg episode reward: [(0, '834.940'), (1, '990.350')] [2023-12-27 11:15:47,369][20381] Updated weights for policy 1, policy_version 813525 (0.0010) [2023-12-27 11:15:47,387][20347] Updated weights for policy 0, policy_version 813987 (0.0007) [2023-12-27 11:15:47,445][20347] Updated weights for policy 0, policy_version 813997 (0.0007) [2023-12-27 11:15:47,506][20347] Updated weights for policy 0, policy_version 814007 (0.0007) [2023-12-27 11:15:48,068][20381] Updated weights for policy 1, policy_version 813535 (0.0008) [2023-12-27 11:15:48,122][20381] Updated weights for policy 1, policy_version 813545 (0.0005) [2023-12-27 11:15:48,152][20347] Updated weights for policy 0, policy_version 814017 (0.0007) [2023-12-27 11:15:48,176][20381] Updated weights for policy 1, policy_version 813555 (0.0005) [2023-12-27 11:15:48,213][20347] Updated weights for policy 0, policy_version 814027 (0.0006) [2023-12-27 11:15:48,278][20347] Updated weights for policy 0, policy_version 814037 (0.0008) [2023-12-27 11:15:48,351][20347] Updated weights for policy 0, policy_version 814047 (0.0008) [2023-12-27 11:15:48,848][20381] Updated weights for policy 1, policy_version 813565 (0.0007) [2023-12-27 11:15:48,911][20381] Updated weights for policy 1, policy_version 813575 (0.0007) [2023-12-27 11:15:48,965][20381] Updated weights for policy 1, policy_version 813585 (0.0008) [2023-12-27 11:15:49,024][20347] Updated weights for policy 0, policy_version 814057 (0.0008) [2023-12-27 11:15:49,074][20347] Updated weights for policy 0, policy_version 814067 (0.0008) [2023-12-27 11:15:49,120][20347] Updated weights for policy 0, policy_version 814077 (0.0007) [2023-12-27 11:15:49,668][20381] Updated weights for policy 1, policy_version 813595 (0.0007) [2023-12-27 11:15:49,728][20381] Updated weights for policy 1, policy_version 813605 (0.0011) [2023-12-27 11:15:49,793][20381] Updated weights for policy 1, policy_version 813615 (0.0008) [2023-12-27 11:15:49,867][20347] Updated weights for policy 0, policy_version 814087 (0.0007) [2023-12-27 11:15:49,933][20347] Updated weights for policy 0, policy_version 814097 (0.0008) [2023-12-27 11:15:49,993][20347] Updated weights for policy 0, policy_version 814107 (0.0008) [2023-12-27 11:15:50,502][20381] Updated weights for policy 1, policy_version 813625 (0.0010) [2023-12-27 11:15:50,554][20381] Updated weights for policy 1, policy_version 813635 (0.0009) [2023-12-27 11:15:50,614][20381] Updated weights for policy 1, policy_version 813645 (0.0007) [2023-12-27 11:15:50,674][20381] Updated weights for policy 1, policy_version 813655 (0.0006) [2023-12-27 11:15:50,698][20347] Updated weights for policy 0, policy_version 814117 (0.0008) [2023-12-27 11:15:50,762][20347] Updated weights for policy 0, policy_version 814127 (0.0007) [2023-12-27 11:15:50,823][20347] Updated weights for policy 0, policy_version 814137 (0.0007) [2023-12-27 11:15:51,344][20381] Updated weights for policy 1, policy_version 813665 (0.0010) [2023-12-27 11:15:51,412][20381] Updated weights for policy 1, policy_version 813675 (0.0009) [2023-12-27 11:15:51,465][20381] Updated weights for policy 1, policy_version 813685 (0.0010) [2023-12-27 11:15:51,532][20347] Updated weights for policy 0, policy_version 814147 (0.0007) [2023-12-27 11:15:51,593][20347] Updated weights for policy 0, policy_version 814158 (0.0008) [2023-12-27 11:15:51,659][20347] Updated weights for policy 0, policy_version 814168 (0.0006) [2023-12-27 11:15:52,188][20381] Updated weights for policy 1, policy_version 813695 (0.0008) [2023-12-27 11:15:52,248][20381] Updated weights for policy 1, policy_version 813705 (0.0008) [2023-12-27 11:15:52,301][20381] Updated weights for policy 1, policy_version 813715 (0.0008) [2023-12-27 11:15:52,360][20347] Updated weights for policy 0, policy_version 814178 (0.0010) [2023-12-27 11:15:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19387.7, 300 sec: 19938.5). Total num frames: 416792576. Throughput: 0: 9739.2, 1: 9663.4. Samples: 416762252. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:52,363][19170] Avg episode reward: [(0, '863.750'), (1, '990.350')] [2023-12-27 11:15:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000813720_208338944.pth... [2023-12-27 11:15:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000812568_208044032.pth [2023-12-27 11:15:52,417][20347] Updated weights for policy 0, policy_version 814188 (0.0011) [2023-12-27 11:15:52,466][20347] Updated weights for policy 0, policy_version 814198 (0.0010) [2023-12-27 11:15:52,516][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000814208_208461824.pth... [2023-12-27 11:15:52,518][20347] Updated weights for policy 0, policy_version 814208 (0.0010) [2023-12-27 11:15:52,520][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000813056_208166912.pth [2023-12-27 11:15:53,023][20381] Updated weights for policy 1, policy_version 813725 (0.0006) [2023-12-27 11:15:53,084][20381] Updated weights for policy 1, policy_version 813735 (0.0005) [2023-12-27 11:15:53,141][20381] Updated weights for policy 1, policy_version 813745 (0.0007) [2023-12-27 11:15:53,168][20347] Updated weights for policy 0, policy_version 814218 (0.0005) [2023-12-27 11:15:53,223][20347] Updated weights for policy 0, policy_version 814228 (0.0008) [2023-12-27 11:15:53,281][20347] Updated weights for policy 0, policy_version 814238 (0.0007) [2023-12-27 11:15:53,725][20381] Updated weights for policy 1, policy_version 813755 (0.0008) [2023-12-27 11:15:53,784][20381] Updated weights for policy 1, policy_version 813765 (0.0008) [2023-12-27 11:15:53,833][20381] Updated weights for policy 1, policy_version 813775 (0.0008) [2023-12-27 11:15:53,961][20347] Updated weights for policy 0, policy_version 814248 (0.0008) [2023-12-27 11:15:54,019][20347] Updated weights for policy 0, policy_version 814258 (0.0007) [2023-12-27 11:15:54,081][20347] Updated weights for policy 0, policy_version 814268 (0.0008) [2023-12-27 11:15:54,488][20381] Updated weights for policy 1, policy_version 813785 (0.0007) [2023-12-27 11:15:54,548][20381] Updated weights for policy 1, policy_version 813795 (0.0008) [2023-12-27 11:15:54,619][20381] Updated weights for policy 1, policy_version 813805 (0.0008) [2023-12-27 11:15:54,687][20381] Updated weights for policy 1, policy_version 813815 (0.0008) [2023-12-27 11:15:54,822][20347] Updated weights for policy 0, policy_version 814278 (0.0008) [2023-12-27 11:15:54,886][20347] Updated weights for policy 0, policy_version 814288 (0.0008) [2023-12-27 11:15:54,946][20347] Updated weights for policy 0, policy_version 814298 (0.0006) [2023-12-27 11:15:55,351][20381] Updated weights for policy 1, policy_version 813825 (0.0009) [2023-12-27 11:15:55,413][20381] Updated weights for policy 1, policy_version 813835 (0.0007) [2023-12-27 11:15:55,471][20381] Updated weights for policy 1, policy_version 813845 (0.0008) [2023-12-27 11:15:55,616][20347] Updated weights for policy 0, policy_version 814308 (0.0008) [2023-12-27 11:15:55,683][20347] Updated weights for policy 0, policy_version 814318 (0.0007) [2023-12-27 11:15:55,746][20347] Updated weights for policy 0, policy_version 814328 (0.0008) [2023-12-27 11:15:56,130][20381] Updated weights for policy 1, policy_version 813855 (0.0008) [2023-12-27 11:15:56,183][20381] Updated weights for policy 1, policy_version 813865 (0.0007) [2023-12-27 11:15:56,243][20381] Updated weights for policy 1, policy_version 813875 (0.0008) [2023-12-27 11:15:56,466][20347] Updated weights for policy 0, policy_version 814338 (0.0008) [2023-12-27 11:15:56,525][20347] Updated weights for policy 0, policy_version 814348 (0.0008) [2023-12-27 11:15:56,586][20347] Updated weights for policy 0, policy_version 814358 (0.0008) [2023-12-27 11:15:56,641][20347] Updated weights for policy 0, policy_version 814368 (0.0008) [2023-12-27 11:15:56,919][20381] Updated weights for policy 1, policy_version 813885 (0.0007) [2023-12-27 11:15:56,971][20381] Updated weights for policy 1, policy_version 813895 (0.0007) [2023-12-27 11:15:57,027][20381] Updated weights for policy 1, policy_version 813905 (0.0007) [2023-12-27 11:15:57,323][20347] Updated weights for policy 0, policy_version 814378 (0.0008) [2023-12-27 11:15:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19387.8, 300 sec: 19910.7). Total num frames: 416890880. Throughput: 0: 9759.5, 1: 9704.1. Samples: 416882576. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:15:57,363][19170] Avg episode reward: [(0, '843.800'), (1, '1000.000')] [2023-12-27 11:15:57,373][20347] Updated weights for policy 0, policy_version 814388 (0.0007) [2023-12-27 11:15:57,426][20347] Updated weights for policy 0, policy_version 814398 (0.0008) [2023-12-27 11:15:57,729][20381] Updated weights for policy 1, policy_version 813915 (0.0007) [2023-12-27 11:15:57,785][20381] Updated weights for policy 1, policy_version 813925 (0.0008) [2023-12-27 11:15:57,846][20381] Updated weights for policy 1, policy_version 813935 (0.0008) [2023-12-27 11:15:58,125][20347] Updated weights for policy 0, policy_version 814408 (0.0007) [2023-12-27 11:15:58,179][20347] Updated weights for policy 0, policy_version 814418 (0.0006) [2023-12-27 11:15:58,245][20347] Updated weights for policy 0, policy_version 814428 (0.0008) [2023-12-27 11:15:58,610][20381] Updated weights for policy 1, policy_version 813945 (0.0008) [2023-12-27 11:15:58,670][20381] Updated weights for policy 1, policy_version 813955 (0.0009) [2023-12-27 11:15:58,734][20381] Updated weights for policy 1, policy_version 813965 (0.0011) [2023-12-27 11:15:58,798][20381] Updated weights for policy 1, policy_version 813975 (0.0008) [2023-12-27 11:15:59,108][20347] Updated weights for policy 0, policy_version 814438 (0.0008) [2023-12-27 11:15:59,171][20347] Updated weights for policy 0, policy_version 814448 (0.0009) [2023-12-27 11:15:59,242][20347] Updated weights for policy 0, policy_version 814459 (0.0011) [2023-12-27 11:15:59,618][20381] Updated weights for policy 1, policy_version 813985 (0.0007) [2023-12-27 11:15:59,675][20381] Updated weights for policy 1, policy_version 813995 (0.0008) [2023-12-27 11:15:59,737][20381] Updated weights for policy 1, policy_version 814005 (0.0008) [2023-12-27 11:16:00,036][20347] Updated weights for policy 0, policy_version 814470 (0.0014) [2023-12-27 11:16:00,091][20347] Updated weights for policy 0, policy_version 814480 (0.0007) [2023-12-27 11:16:00,149][20347] Updated weights for policy 0, policy_version 814490 (0.0007) [2023-12-27 11:16:00,532][20381] Updated weights for policy 1, policy_version 814015 (0.0007) [2023-12-27 11:16:00,579][20381] Updated weights for policy 1, policy_version 814025 (0.0007) [2023-12-27 11:16:00,638][20381] Updated weights for policy 1, policy_version 814035 (0.0007) [2023-12-27 11:16:00,837][20347] Updated weights for policy 0, policy_version 814500 (0.0007) [2023-12-27 11:16:00,897][20347] Updated weights for policy 0, policy_version 814510 (0.0008) [2023-12-27 11:16:00,947][20347] Updated weights for policy 0, policy_version 814520 (0.0008) [2023-12-27 11:16:01,382][20381] Updated weights for policy 1, policy_version 814045 (0.0008) [2023-12-27 11:16:01,442][20381] Updated weights for policy 1, policy_version 814055 (0.0008) [2023-12-27 11:16:01,501][20381] Updated weights for policy 1, policy_version 814065 (0.0008) [2023-12-27 11:16:01,673][20347] Updated weights for policy 0, policy_version 814530 (0.0007) [2023-12-27 11:16:01,741][20347] Updated weights for policy 0, policy_version 814540 (0.0009) [2023-12-27 11:16:01,793][20347] Updated weights for policy 0, policy_version 814550 (0.0009) [2023-12-27 11:16:01,846][20347] Updated weights for policy 0, policy_version 814560 (0.0008) [2023-12-27 11:16:02,267][20381] Updated weights for policy 1, policy_version 814075 (0.0008) [2023-12-27 11:16:02,319][20381] Updated weights for policy 1, policy_version 814085 (0.0006) [2023-12-27 11:16:02,362][19170] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19855.2). Total num frames: 416980992. Throughput: 0: 9720.6, 1: 9666.2. Samples: 416995848. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:02,363][19170] Avg episode reward: [(0, '892.800'), (1, '1000.000')] [2023-12-27 11:16:02,381][20381] Updated weights for policy 1, policy_version 814095 (0.0007) [2023-12-27 11:16:02,540][20347] Updated weights for policy 0, policy_version 814570 (0.0007) [2023-12-27 11:16:02,602][20347] Updated weights for policy 0, policy_version 814580 (0.0006) [2023-12-27 11:16:02,655][20347] Updated weights for policy 0, policy_version 814590 (0.0006) [2023-12-27 11:16:03,035][20381] Updated weights for policy 1, policy_version 814105 (0.0009) [2023-12-27 11:16:03,096][20381] Updated weights for policy 1, policy_version 814115 (0.0007) [2023-12-27 11:16:03,151][20381] Updated weights for policy 1, policy_version 814125 (0.0007) [2023-12-27 11:16:03,204][20381] Updated weights for policy 1, policy_version 814135 (0.0006) [2023-12-27 11:16:03,337][20347] Updated weights for policy 0, policy_version 814600 (0.0006) [2023-12-27 11:16:03,396][20347] Updated weights for policy 0, policy_version 814610 (0.0006) [2023-12-27 11:16:03,454][20347] Updated weights for policy 0, policy_version 814620 (0.0008) [2023-12-27 11:16:03,838][20381] Updated weights for policy 1, policy_version 814145 (0.0006) [2023-12-27 11:16:03,905][20381] Updated weights for policy 1, policy_version 814155 (0.0008) [2023-12-27 11:16:03,965][20381] Updated weights for policy 1, policy_version 814165 (0.0008) [2023-12-27 11:16:04,114][20347] Updated weights for policy 0, policy_version 814630 (0.0008) [2023-12-27 11:16:04,171][20347] Updated weights for policy 0, policy_version 814640 (0.0007) [2023-12-27 11:16:04,232][20347] Updated weights for policy 0, policy_version 814650 (0.0007) [2023-12-27 11:16:04,700][20381] Updated weights for policy 1, policy_version 814175 (0.0007) [2023-12-27 11:16:04,750][20381] Updated weights for policy 1, policy_version 814185 (0.0007) [2023-12-27 11:16:04,808][20381] Updated weights for policy 1, policy_version 814195 (0.0008) [2023-12-27 11:16:04,968][20347] Updated weights for policy 0, policy_version 814660 (0.0008) [2023-12-27 11:16:05,026][20347] Updated weights for policy 0, policy_version 814670 (0.0007) [2023-12-27 11:16:05,091][20347] Updated weights for policy 0, policy_version 814680 (0.0008) [2023-12-27 11:16:05,528][20381] Updated weights for policy 1, policy_version 814205 (0.0008) [2023-12-27 11:16:05,594][20381] Updated weights for policy 1, policy_version 814215 (0.0008) [2023-12-27 11:16:05,651][20381] Updated weights for policy 1, policy_version 814225 (0.0008) [2023-12-27 11:16:05,791][20347] Updated weights for policy 0, policy_version 814690 (0.0007) [2023-12-27 11:16:05,854][20347] Updated weights for policy 0, policy_version 814700 (0.0008) [2023-12-27 11:16:05,921][20347] Updated weights for policy 0, policy_version 814710 (0.0007) [2023-12-27 11:16:05,981][20347] Updated weights for policy 0, policy_version 814720 (0.0007) [2023-12-27 11:16:06,403][20381] Updated weights for policy 1, policy_version 814235 (0.0008) [2023-12-27 11:16:06,463][20381] Updated weights for policy 1, policy_version 814245 (0.0008) [2023-12-27 11:16:06,526][20381] Updated weights for policy 1, policy_version 814255 (0.0008) [2023-12-27 11:16:06,670][20347] Updated weights for policy 0, policy_version 814730 (0.0008) [2023-12-27 11:16:06,730][20347] Updated weights for policy 0, policy_version 814740 (0.0008) [2023-12-27 11:16:06,792][20347] Updated weights for policy 0, policy_version 814750 (0.0007) [2023-12-27 11:16:07,326][20381] Updated weights for policy 1, policy_version 814265 (0.0008) [2023-12-27 11:16:07,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19855.2). Total num frames: 417079296. Throughput: 0: 9704.8, 1: 9717.2. Samples: 417055752. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:07,363][19170] Avg episode reward: [(0, '951.360'), (1, '1000.000')] [2023-12-27 11:16:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000814752_208601088.pth... [2023-12-27 11:16:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000813632_208314368.pth [2023-12-27 11:16:07,382][20381] Updated weights for policy 1, policy_version 814275 (0.0008) [2023-12-27 11:16:07,440][20381] Updated weights for policy 1, policy_version 814285 (0.0008) [2023-12-27 11:16:07,498][20381] Updated weights for policy 1, policy_version 814295 (0.0008) [2023-12-27 11:16:07,501][20347] Updated weights for policy 0, policy_version 814760 (0.0008) [2023-12-27 11:16:07,501][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000814296_208486400.pth... [2023-12-27 11:16:07,505][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000813144_208191488.pth [2023-12-27 11:16:07,590][20347] Updated weights for policy 0, policy_version 814770 (0.0007) [2023-12-27 11:16:07,656][20347] Updated weights for policy 0, policy_version 814780 (0.0008) [2023-12-27 11:16:08,193][20381] Updated weights for policy 1, policy_version 814305 (0.0006) [2023-12-27 11:16:08,260][20381] Updated weights for policy 1, policy_version 814315 (0.0010) [2023-12-27 11:16:08,321][20381] Updated weights for policy 1, policy_version 814325 (0.0008) [2023-12-27 11:16:08,322][20347] Updated weights for policy 0, policy_version 814790 (0.0006) [2023-12-27 11:16:08,382][20347] Updated weights for policy 0, policy_version 814800 (0.0006) [2023-12-27 11:16:08,438][20347] Updated weights for policy 0, policy_version 814810 (0.0007) [2023-12-27 11:16:09,070][20381] Updated weights for policy 1, policy_version 814335 (0.0008) [2023-12-27 11:16:09,125][20381] Updated weights for policy 1, policy_version 814345 (0.0007) [2023-12-27 11:16:09,150][20347] Updated weights for policy 0, policy_version 814820 (0.0007) [2023-12-27 11:16:09,187][20381] Updated weights for policy 1, policy_version 814355 (0.0008) [2023-12-27 11:16:09,217][20347] Updated weights for policy 0, policy_version 814830 (0.0007) [2023-12-27 11:16:09,281][20347] Updated weights for policy 0, policy_version 814840 (0.0008) [2023-12-27 11:16:09,945][20381] Updated weights for policy 1, policy_version 814365 (0.0008) [2023-12-27 11:16:10,008][20381] Updated weights for policy 1, policy_version 814375 (0.0007) [2023-12-27 11:16:10,071][20381] Updated weights for policy 1, policy_version 814385 (0.0008) [2023-12-27 11:16:10,090][20347] Updated weights for policy 0, policy_version 814850 (0.0009) [2023-12-27 11:16:10,154][20347] Updated weights for policy 0, policy_version 814860 (0.0008) [2023-12-27 11:16:10,219][20347] Updated weights for policy 0, policy_version 814870 (0.0008) [2023-12-27 11:16:10,280][20347] Updated weights for policy 0, policy_version 814880 (0.0008) [2023-12-27 11:16:10,792][20381] Updated weights for policy 1, policy_version 814395 (0.0007) [2023-12-27 11:16:10,854][20381] Updated weights for policy 1, policy_version 814405 (0.0009) [2023-12-27 11:16:10,907][20381] Updated weights for policy 1, policy_version 814415 (0.0007) [2023-12-27 11:16:11,042][20347] Updated weights for policy 0, policy_version 814890 (0.0009) [2023-12-27 11:16:11,101][20347] Updated weights for policy 0, policy_version 814900 (0.0010) [2023-12-27 11:16:11,171][20347] Updated weights for policy 0, policy_version 814910 (0.0009) [2023-12-27 11:16:11,660][20381] Updated weights for policy 1, policy_version 814425 (0.0006) [2023-12-27 11:16:11,726][20381] Updated weights for policy 1, policy_version 814435 (0.0008) [2023-12-27 11:16:11,788][20381] Updated weights for policy 1, policy_version 814445 (0.0007) [2023-12-27 11:16:11,852][20381] Updated weights for policy 1, policy_version 814455 (0.0009) [2023-12-27 11:16:11,981][20347] Updated weights for policy 0, policy_version 814920 (0.0008) [2023-12-27 11:16:12,047][20347] Updated weights for policy 0, policy_version 814930 (0.0008) [2023-12-27 11:16:12,102][20347] Updated weights for policy 0, policy_version 814940 (0.0008) [2023-12-27 11:16:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19827.4). Total num frames: 417177600. Throughput: 0: 9681.4, 1: 9706.7. Samples: 417168696. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:12,363][19170] Avg episode reward: [(0, '951.300'), (1, '1000.000')] [2023-12-27 11:16:12,592][20381] Updated weights for policy 1, policy_version 814465 (0.0006) [2023-12-27 11:16:12,661][20381] Updated weights for policy 1, policy_version 814475 (0.0007) [2023-12-27 11:16:12,725][20381] Updated weights for policy 1, policy_version 814485 (0.0007) [2023-12-27 11:16:12,800][20347] Updated weights for policy 0, policy_version 814950 (0.0007) [2023-12-27 11:16:12,858][20347] Updated weights for policy 0, policy_version 814960 (0.0007) [2023-12-27 11:16:12,921][20347] Updated weights for policy 0, policy_version 814970 (0.0007) [2023-12-27 11:16:13,441][20381] Updated weights for policy 1, policy_version 814495 (0.0006) [2023-12-27 11:16:13,506][20381] Updated weights for policy 1, policy_version 814505 (0.0009) [2023-12-27 11:16:13,555][20347] Updated weights for policy 0, policy_version 814980 (0.0006) [2023-12-27 11:16:13,565][20381] Updated weights for policy 1, policy_version 814515 (0.0008) [2023-12-27 11:16:13,612][20347] Updated weights for policy 0, policy_version 814990 (0.0005) [2023-12-27 11:16:13,669][20347] Updated weights for policy 0, policy_version 815000 (0.0008) [2023-12-27 11:16:14,222][20381] Updated weights for policy 1, policy_version 814525 (0.0008) [2023-12-27 11:16:14,278][20381] Updated weights for policy 1, policy_version 814535 (0.0008) [2023-12-27 11:16:14,340][20381] Updated weights for policy 1, policy_version 814545 (0.0007) [2023-12-27 11:16:14,408][20347] Updated weights for policy 0, policy_version 815010 (0.0007) [2023-12-27 11:16:14,466][20347] Updated weights for policy 0, policy_version 815020 (0.0008) [2023-12-27 11:16:14,521][20347] Updated weights for policy 0, policy_version 815030 (0.0008) [2023-12-27 11:16:14,579][20347] Updated weights for policy 0, policy_version 815040 (0.0007) [2023-12-27 11:16:15,061][20381] Updated weights for policy 1, policy_version 814555 (0.0007) [2023-12-27 11:16:15,119][20381] Updated weights for policy 1, policy_version 814565 (0.0008) [2023-12-27 11:16:15,185][20381] Updated weights for policy 1, policy_version 814575 (0.0008) [2023-12-27 11:16:15,310][20347] Updated weights for policy 0, policy_version 815050 (0.0007) [2023-12-27 11:16:15,373][20347] Updated weights for policy 0, policy_version 815060 (0.0007) [2023-12-27 11:16:15,439][20347] Updated weights for policy 0, policy_version 815070 (0.0006) [2023-12-27 11:16:15,910][20381] Updated weights for policy 1, policy_version 814585 (0.0009) [2023-12-27 11:16:15,964][20381] Updated weights for policy 1, policy_version 814595 (0.0008) [2023-12-27 11:16:16,018][20381] Updated weights for policy 1, policy_version 814605 (0.0008) [2023-12-27 11:16:16,075][20381] Updated weights for policy 1, policy_version 814615 (0.0007) [2023-12-27 11:16:16,091][20347] Updated weights for policy 0, policy_version 815080 (0.0008) [2023-12-27 11:16:16,152][20347] Updated weights for policy 0, policy_version 815090 (0.0008) [2023-12-27 11:16:16,221][20347] Updated weights for policy 0, policy_version 815100 (0.0007) [2023-12-27 11:16:16,804][20381] Updated weights for policy 1, policy_version 814625 (0.0008) [2023-12-27 11:16:16,859][20381] Updated weights for policy 1, policy_version 814635 (0.0007) [2023-12-27 11:16:16,899][20347] Updated weights for policy 0, policy_version 815110 (0.0008) [2023-12-27 11:16:16,917][20381] Updated weights for policy 1, policy_version 814645 (0.0006) [2023-12-27 11:16:16,952][20347] Updated weights for policy 0, policy_version 815120 (0.0007) [2023-12-27 11:16:17,006][20347] Updated weights for policy 0, policy_version 815130 (0.0008) [2023-12-27 11:16:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19827.4). Total num frames: 417275904. Throughput: 0: 9672.9, 1: 9735.9. Samples: 417285124. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:17,363][19170] Avg episode reward: [(0, '951.300'), (1, '1000.000')] [2023-12-27 11:16:17,638][20381] Updated weights for policy 1, policy_version 814655 (0.0006) [2023-12-27 11:16:17,707][20381] Updated weights for policy 1, policy_version 814665 (0.0009) [2023-12-27 11:16:17,709][20347] Updated weights for policy 0, policy_version 815140 (0.0007) [2023-12-27 11:16:17,766][20347] Updated weights for policy 0, policy_version 815150 (0.0006) [2023-12-27 11:16:17,767][20381] Updated weights for policy 1, policy_version 814675 (0.0006) [2023-12-27 11:16:17,821][20347] Updated weights for policy 0, policy_version 815160 (0.0006) [2023-12-27 11:16:18,480][20347] Updated weights for policy 0, policy_version 815170 (0.0007) [2023-12-27 11:16:18,509][20381] Updated weights for policy 1, policy_version 814685 (0.0006) [2023-12-27 11:16:18,538][20347] Updated weights for policy 0, policy_version 815180 (0.0011) [2023-12-27 11:16:18,569][20381] Updated weights for policy 1, policy_version 814695 (0.0007) [2023-12-27 11:16:18,599][20347] Updated weights for policy 0, policy_version 815190 (0.0008) [2023-12-27 11:16:18,632][20381] Updated weights for policy 1, policy_version 814705 (0.0007) [2023-12-27 11:16:18,661][20347] Updated weights for policy 0, policy_version 815200 (0.0011) [2023-12-27 11:16:19,312][20347] Updated weights for policy 0, policy_version 815210 (0.0008) [2023-12-27 11:16:19,387][20347] Updated weights for policy 0, policy_version 815220 (0.0008) [2023-12-27 11:16:19,398][20381] Updated weights for policy 1, policy_version 814715 (0.0008) [2023-12-27 11:16:19,446][20347] Updated weights for policy 0, policy_version 815230 (0.0008) [2023-12-27 11:16:19,455][20381] Updated weights for policy 1, policy_version 814725 (0.0007) [2023-12-27 11:16:19,522][20381] Updated weights for policy 1, policy_version 814735 (0.0008) [2023-12-27 11:16:20,201][20347] Updated weights for policy 0, policy_version 815240 (0.0007) [2023-12-27 11:16:20,260][20347] Updated weights for policy 0, policy_version 815250 (0.0008) [2023-12-27 11:16:20,311][20381] Updated weights for policy 1, policy_version 814745 (0.0008) [2023-12-27 11:16:20,318][20347] Updated weights for policy 0, policy_version 815260 (0.0007) [2023-12-27 11:16:20,377][20381] Updated weights for policy 1, policy_version 814755 (0.0008) [2023-12-27 11:16:20,437][20381] Updated weights for policy 1, policy_version 814765 (0.0008) [2023-12-27 11:16:20,503][20381] Updated weights for policy 1, policy_version 814775 (0.0008) [2023-12-27 11:16:21,080][20347] Updated weights for policy 0, policy_version 815270 (0.0007) [2023-12-27 11:16:21,151][20347] Updated weights for policy 0, policy_version 815280 (0.0008) [2023-12-27 11:16:21,190][20381] Updated weights for policy 1, policy_version 814785 (0.0007) [2023-12-27 11:16:21,212][20347] Updated weights for policy 0, policy_version 815290 (0.0008) [2023-12-27 11:16:21,253][20381] Updated weights for policy 1, policy_version 814795 (0.0007) [2023-12-27 11:16:21,316][20381] Updated weights for policy 1, policy_version 814805 (0.0007) [2023-12-27 11:16:21,971][20347] Updated weights for policy 0, policy_version 815300 (0.0007) [2023-12-27 11:16:22,031][20347] Updated weights for policy 0, policy_version 815310 (0.0006) [2023-12-27 11:16:22,037][20291] KL-divergence is very high: 402.8911 [2023-12-27 11:16:22,061][20381] Updated weights for policy 1, policy_version 814815 (0.0007) [2023-12-27 11:16:22,089][20291] KL-divergence is very high: 126.5612 [2023-12-27 11:16:22,094][20347] Updated weights for policy 0, policy_version 815320 (0.0006) [2023-12-27 11:16:22,120][20381] Updated weights for policy 1, policy_version 814825 (0.0008) [2023-12-27 11:16:22,185][20381] Updated weights for policy 1, policy_version 814835 (0.0008) [2023-12-27 11:16:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19771.9). Total num frames: 417374208. Throughput: 0: 9724.2, 1: 9704.5. Samples: 417342720. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:22,363][19170] Avg episode reward: [(0, '980.510'), (1, '1000.000')] [2023-12-27 11:16:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000814840_208625664.pth... [2023-12-27 11:16:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000815328_208748544.pth... [2023-12-27 11:16:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000814208_208461824.pth [2023-12-27 11:16:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000813720_208338944.pth [2023-12-27 11:16:22,839][20347] Updated weights for policy 0, policy_version 815330 (0.0007) [2023-12-27 11:16:22,896][20347] Updated weights for policy 0, policy_version 815340 (0.0008) [2023-12-27 11:16:22,931][20381] Updated weights for policy 1, policy_version 814845 (0.0006) [2023-12-27 11:16:22,957][20347] Updated weights for policy 0, policy_version 815350 (0.0008) [2023-12-27 11:16:22,985][20381] Updated weights for policy 1, policy_version 814855 (0.0007) [2023-12-27 11:16:23,017][20347] Updated weights for policy 0, policy_version 815360 (0.0008) [2023-12-27 11:16:23,047][20381] Updated weights for policy 1, policy_version 814865 (0.0007) [2023-12-27 11:16:23,733][20347] Updated weights for policy 0, policy_version 815370 (0.0008) [2023-12-27 11:16:23,761][20381] Updated weights for policy 1, policy_version 814875 (0.0008) [2023-12-27 11:16:23,790][20347] Updated weights for policy 0, policy_version 815380 (0.0006) [2023-12-27 11:16:23,815][20381] Updated weights for policy 1, policy_version 814885 (0.0008) [2023-12-27 11:16:23,844][20347] Updated weights for policy 0, policy_version 815390 (0.0006) [2023-12-27 11:16:23,870][20381] Updated weights for policy 1, policy_version 814895 (0.0006) [2023-12-27 11:16:24,601][20381] Updated weights for policy 1, policy_version 814905 (0.0008) [2023-12-27 11:16:24,624][20347] Updated weights for policy 0, policy_version 815400 (0.0006) [2023-12-27 11:16:24,657][20381] Updated weights for policy 1, policy_version 814915 (0.0008) [2023-12-27 11:16:24,682][20347] Updated weights for policy 0, policy_version 815410 (0.0006) [2023-12-27 11:16:24,716][20381] Updated weights for policy 1, policy_version 814925 (0.0008) [2023-12-27 11:16:24,744][20347] Updated weights for policy 0, policy_version 815420 (0.0007) [2023-12-27 11:16:24,779][20381] Updated weights for policy 1, policy_version 814935 (0.0008) [2023-12-27 11:16:25,380][20347] Updated weights for policy 0, policy_version 815430 (0.0007) [2023-12-27 11:16:25,436][20347] Updated weights for policy 0, policy_version 815440 (0.0008) [2023-12-27 11:16:25,466][20381] Updated weights for policy 1, policy_version 814945 (0.0010) [2023-12-27 11:16:25,498][20347] Updated weights for policy 0, policy_version 815450 (0.0006) [2023-12-27 11:16:25,519][20381] Updated weights for policy 1, policy_version 814955 (0.0011) [2023-12-27 11:16:25,573][20381] Updated weights for policy 1, policy_version 814965 (0.0010) [2023-12-27 11:16:26,201][20347] Updated weights for policy 0, policy_version 815460 (0.0007) [2023-12-27 11:16:26,257][20347] Updated weights for policy 0, policy_version 815470 (0.0007) [2023-12-27 11:16:26,280][20381] Updated weights for policy 1, policy_version 814975 (0.0011) [2023-12-27 11:16:26,318][20347] Updated weights for policy 0, policy_version 815480 (0.0006) [2023-12-27 11:16:26,340][20381] Updated weights for policy 1, policy_version 814985 (0.0009) [2023-12-27 11:16:26,400][20381] Updated weights for policy 1, policy_version 814995 (0.0011) [2023-12-27 11:16:27,025][20347] Updated weights for policy 0, policy_version 815490 (0.0006) [2023-12-27 11:16:27,077][20347] Updated weights for policy 0, policy_version 815500 (0.0007) [2023-12-27 11:16:27,091][20381] Updated weights for policy 1, policy_version 815005 (0.0011) [2023-12-27 11:16:27,132][20347] Updated weights for policy 0, policy_version 815510 (0.0007) [2023-12-27 11:16:27,149][20381] Updated weights for policy 1, policy_version 815015 (0.0009) [2023-12-27 11:16:27,189][20347] Updated weights for policy 0, policy_version 815520 (0.0006) [2023-12-27 11:16:27,198][20381] Updated weights for policy 1, policy_version 815025 (0.0009) [2023-12-27 11:16:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19771.9). Total num frames: 417472512. Throughput: 0: 9738.3, 1: 9774.6. Samples: 417457796. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:27,363][19170] Avg episode reward: [(0, '990.260'), (1, '1000.000')] [2023-12-27 11:16:27,880][20381] Updated weights for policy 1, policy_version 815035 (0.0006) [2023-12-27 11:16:27,883][20347] Updated weights for policy 0, policy_version 815530 (0.0007) [2023-12-27 11:16:27,931][20381] Updated weights for policy 1, policy_version 815045 (0.0005) [2023-12-27 11:16:27,940][20347] Updated weights for policy 0, policy_version 815540 (0.0005) [2023-12-27 11:16:27,993][20381] Updated weights for policy 1, policy_version 815055 (0.0007) [2023-12-27 11:16:28,003][20347] Updated weights for policy 0, policy_version 815550 (0.0006) [2023-12-27 11:16:28,675][20347] Updated weights for policy 0, policy_version 815560 (0.0007) [2023-12-27 11:16:28,696][20381] Updated weights for policy 1, policy_version 815065 (0.0007) [2023-12-27 11:16:28,733][20347] Updated weights for policy 0, policy_version 815570 (0.0006) [2023-12-27 11:16:28,757][20381] Updated weights for policy 1, policy_version 815075 (0.0010) [2023-12-27 11:16:28,790][20347] Updated weights for policy 0, policy_version 815580 (0.0006) [2023-12-27 11:16:28,814][20381] Updated weights for policy 1, policy_version 815085 (0.0008) [2023-12-27 11:16:28,877][20381] Updated weights for policy 1, policy_version 815095 (0.0007) [2023-12-27 11:16:29,528][20347] Updated weights for policy 0, policy_version 815590 (0.0008) [2023-12-27 11:16:29,585][20347] Updated weights for policy 0, policy_version 815600 (0.0007) [2023-12-27 11:16:29,613][20381] Updated weights for policy 1, policy_version 815105 (0.0007) [2023-12-27 11:16:29,651][20347] Updated weights for policy 0, policy_version 815610 (0.0007) [2023-12-27 11:16:29,675][20381] Updated weights for policy 1, policy_version 815115 (0.0007) [2023-12-27 11:16:29,737][20381] Updated weights for policy 1, policy_version 815125 (0.0007) [2023-12-27 11:16:30,351][20347] Updated weights for policy 0, policy_version 815620 (0.0006) [2023-12-27 11:16:30,409][20347] Updated weights for policy 0, policy_version 815630 (0.0008) [2023-12-27 11:16:30,461][20347] Updated weights for policy 0, policy_version 815640 (0.0007) [2023-12-27 11:16:30,476][20381] Updated weights for policy 1, policy_version 815135 (0.0007) [2023-12-27 11:16:30,529][20381] Updated weights for policy 1, policy_version 815145 (0.0007) [2023-12-27 11:16:30,583][20381] Updated weights for policy 1, policy_version 815155 (0.0008) [2023-12-27 11:16:31,128][20347] Updated weights for policy 0, policy_version 815650 (0.0007) [2023-12-27 11:16:31,198][20347] Updated weights for policy 0, policy_version 815660 (0.0008) [2023-12-27 11:16:31,257][20347] Updated weights for policy 0, policy_version 815670 (0.0008) [2023-12-27 11:16:31,310][20347] Updated weights for policy 0, policy_version 815680 (0.0007) [2023-12-27 11:16:31,310][20381] Updated weights for policy 1, policy_version 815165 (0.0007) [2023-12-27 11:16:31,368][20381] Updated weights for policy 1, policy_version 815175 (0.0007) [2023-12-27 11:16:31,442][20381] Updated weights for policy 1, policy_version 815185 (0.0008) [2023-12-27 11:16:32,037][20347] Updated weights for policy 0, policy_version 815690 (0.0008) [2023-12-27 11:16:32,099][20347] Updated weights for policy 0, policy_version 815700 (0.0008) [2023-12-27 11:16:32,157][20347] Updated weights for policy 0, policy_version 815710 (0.0007) [2023-12-27 11:16:32,166][20381] Updated weights for policy 1, policy_version 815195 (0.0008) [2023-12-27 11:16:32,227][20381] Updated weights for policy 1, policy_version 815205 (0.0008) [2023-12-27 11:16:32,289][20381] Updated weights for policy 1, policy_version 815215 (0.0008) [2023-12-27 11:16:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19716.3). Total num frames: 417570816. Throughput: 0: 9735.2, 1: 9680.7. Samples: 417575556. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:32,363][19170] Avg episode reward: [(0, '990.260'), (1, '1000.000')] [2023-12-27 11:16:32,901][20347] Updated weights for policy 0, policy_version 815720 (0.0007) [2023-12-27 11:16:32,959][20347] Updated weights for policy 0, policy_version 815730 (0.0008) [2023-12-27 11:16:33,009][20347] Updated weights for policy 0, policy_version 815740 (0.0006) [2023-12-27 11:16:33,014][20381] Updated weights for policy 1, policy_version 815225 (0.0008) [2023-12-27 11:16:33,070][20381] Updated weights for policy 1, policy_version 815235 (0.0009) [2023-12-27 11:16:33,127][20381] Updated weights for policy 1, policy_version 815245 (0.0008) [2023-12-27 11:16:33,177][20381] Updated weights for policy 1, policy_version 815255 (0.0008) [2023-12-27 11:16:33,675][20347] Updated weights for policy 0, policy_version 815750 (0.0008) [2023-12-27 11:16:33,724][20347] Updated weights for policy 0, policy_version 815760 (0.0007) [2023-12-27 11:16:33,783][20347] Updated weights for policy 0, policy_version 815770 (0.0007) [2023-12-27 11:16:33,883][20381] Updated weights for policy 1, policy_version 815265 (0.0007) [2023-12-27 11:16:33,935][20381] Updated weights for policy 1, policy_version 815275 (0.0008) [2023-12-27 11:16:33,993][20381] Updated weights for policy 1, policy_version 815285 (0.0006) [2023-12-27 11:16:34,483][20347] Updated weights for policy 0, policy_version 815780 (0.0008) [2023-12-27 11:16:34,538][20347] Updated weights for policy 0, policy_version 815790 (0.0008) [2023-12-27 11:16:34,596][20347] Updated weights for policy 0, policy_version 815800 (0.0008) [2023-12-27 11:16:34,678][20381] Updated weights for policy 1, policy_version 815295 (0.0007) [2023-12-27 11:16:34,734][20381] Updated weights for policy 1, policy_version 815305 (0.0007) [2023-12-27 11:16:34,788][20381] Updated weights for policy 1, policy_version 815315 (0.0008) [2023-12-27 11:16:35,312][20347] Updated weights for policy 0, policy_version 815810 (0.0007) [2023-12-27 11:16:35,374][20347] Updated weights for policy 0, policy_version 815820 (0.0008) [2023-12-27 11:16:35,426][20347] Updated weights for policy 0, policy_version 815830 (0.0008) [2023-12-27 11:16:35,448][20381] Updated weights for policy 1, policy_version 815325 (0.0007) [2023-12-27 11:16:35,479][20347] Updated weights for policy 0, policy_version 815840 (0.0008) [2023-12-27 11:16:35,505][20381] Updated weights for policy 1, policy_version 815335 (0.0006) [2023-12-27 11:16:35,570][20381] Updated weights for policy 1, policy_version 815345 (0.0006) [2023-12-27 11:16:36,210][20381] Updated weights for policy 1, policy_version 815355 (0.0006) [2023-12-27 11:16:36,223][20347] Updated weights for policy 0, policy_version 815850 (0.0008) [2023-12-27 11:16:36,269][20381] Updated weights for policy 1, policy_version 815365 (0.0006) [2023-12-27 11:16:36,287][20347] Updated weights for policy 0, policy_version 815860 (0.0006) [2023-12-27 11:16:36,328][20381] Updated weights for policy 1, policy_version 815375 (0.0007) [2023-12-27 11:16:36,349][20347] Updated weights for policy 0, policy_version 815870 (0.0007) [2023-12-27 11:16:37,010][20381] Updated weights for policy 1, policy_version 815385 (0.0008) [2023-12-27 11:16:37,071][20381] Updated weights for policy 1, policy_version 815395 (0.0006) [2023-12-27 11:16:37,083][20347] Updated weights for policy 0, policy_version 815880 (0.0006) [2023-12-27 11:16:37,125][20381] Updated weights for policy 1, policy_version 815405 (0.0007) [2023-12-27 11:16:37,148][20347] Updated weights for policy 0, policy_version 815890 (0.0007) [2023-12-27 11:16:37,193][20381] Updated weights for policy 1, policy_version 815415 (0.0007) [2023-12-27 11:16:37,210][20347] Updated weights for policy 0, policy_version 815900 (0.0006) [2023-12-27 11:16:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 417669120. Throughput: 0: 9720.6, 1: 9668.7. Samples: 417634768. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:16:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000815416_208773120.pth... [2023-12-27 11:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000815904_208896000.pth... [2023-12-27 11:16:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000814296_208486400.pth [2023-12-27 11:16:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000814752_208601088.pth [2023-12-27 11:16:37,880][20381] Updated weights for policy 1, policy_version 815425 (0.0007) [2023-12-27 11:16:37,915][20347] Updated weights for policy 0, policy_version 815910 (0.0008) [2023-12-27 11:16:37,942][20381] Updated weights for policy 1, policy_version 815435 (0.0007) [2023-12-27 11:16:37,977][20347] Updated weights for policy 0, policy_version 815920 (0.0005) [2023-12-27 11:16:37,994][20381] Updated weights for policy 1, policy_version 815445 (0.0006) [2023-12-27 11:16:38,045][20347] Updated weights for policy 0, policy_version 815930 (0.0007) [2023-12-27 11:16:38,737][20381] Updated weights for policy 1, policy_version 815455 (0.0009) [2023-12-27 11:16:38,740][20347] Updated weights for policy 0, policy_version 815940 (0.0008) [2023-12-27 11:16:38,797][20347] Updated weights for policy 0, policy_version 815950 (0.0007) [2023-12-27 11:16:38,798][20381] Updated weights for policy 1, policy_version 815465 (0.0007) [2023-12-27 11:16:38,857][20381] Updated weights for policy 1, policy_version 815475 (0.0007) [2023-12-27 11:16:38,860][20347] Updated weights for policy 0, policy_version 815960 (0.0007) [2023-12-27 11:16:39,592][20381] Updated weights for policy 1, policy_version 815485 (0.0007) [2023-12-27 11:16:39,628][20347] Updated weights for policy 0, policy_version 815970 (0.0007) [2023-12-27 11:16:39,655][20381] Updated weights for policy 1, policy_version 815495 (0.0008) [2023-12-27 11:16:39,693][20347] Updated weights for policy 0, policy_version 815980 (0.0008) [2023-12-27 11:16:39,723][20381] Updated weights for policy 1, policy_version 815505 (0.0009) [2023-12-27 11:16:39,756][20347] Updated weights for policy 0, policy_version 815990 (0.0007) [2023-12-27 11:16:39,825][20347] Updated weights for policy 0, policy_version 816000 (0.0006) [2023-12-27 11:16:40,427][20381] Updated weights for policy 1, policy_version 815515 (0.0007) [2023-12-27 11:16:40,492][20381] Updated weights for policy 1, policy_version 815525 (0.0007) [2023-12-27 11:16:40,549][20381] Updated weights for policy 1, policy_version 815535 (0.0008) [2023-12-27 11:16:40,551][20347] Updated weights for policy 0, policy_version 816010 (0.0005) [2023-12-27 11:16:40,600][20347] Updated weights for policy 0, policy_version 816020 (0.0005) [2023-12-27 11:16:40,668][20347] Updated weights for policy 0, policy_version 816030 (0.0008) [2023-12-27 11:16:41,324][20381] Updated weights for policy 1, policy_version 815545 (0.0007) [2023-12-27 11:16:41,348][20347] Updated weights for policy 0, policy_version 816040 (0.0007) [2023-12-27 11:16:41,402][20381] Updated weights for policy 1, policy_version 815555 (0.0008) [2023-12-27 11:16:41,420][20347] Updated weights for policy 0, policy_version 816050 (0.0008) [2023-12-27 11:16:41,467][20381] Updated weights for policy 1, policy_version 815565 (0.0008) [2023-12-27 11:16:41,486][20347] Updated weights for policy 0, policy_version 816060 (0.0006) [2023-12-27 11:16:41,532][20381] Updated weights for policy 1, policy_version 815575 (0.0006) [2023-12-27 11:16:42,222][20347] Updated weights for policy 0, policy_version 816070 (0.0006) [2023-12-27 11:16:42,233][20381] Updated weights for policy 1, policy_version 815585 (0.0007) [2023-12-27 11:16:42,285][20347] Updated weights for policy 0, policy_version 816080 (0.0008) [2023-12-27 11:16:42,291][20381] Updated weights for policy 1, policy_version 815595 (0.0008) [2023-12-27 11:16:42,350][20347] Updated weights for policy 0, policy_version 816090 (0.0006) [2023-12-27 11:16:42,356][20381] Updated weights for policy 1, policy_version 815605 (0.0006) [2023-12-27 11:16:42,362][19170] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 417751040. Throughput: 0: 9680.1, 1: 9623.8. Samples: 417751252. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:16:42,994][20347] Updated weights for policy 0, policy_version 816100 (0.0008) [2023-12-27 11:16:43,052][20347] Updated weights for policy 0, policy_version 816110 (0.0007) [2023-12-27 11:16:43,117][20347] Updated weights for policy 0, policy_version 816120 (0.0008) [2023-12-27 11:16:43,146][20381] Updated weights for policy 1, policy_version 815615 (0.0008) [2023-12-27 11:16:43,224][20381] Updated weights for policy 1, policy_version 815625 (0.0008) [2023-12-27 11:16:43,276][20381] Updated weights for policy 1, policy_version 815635 (0.0006) [2023-12-27 11:16:43,830][20347] Updated weights for policy 0, policy_version 816130 (0.0007) [2023-12-27 11:16:43,893][20347] Updated weights for policy 0, policy_version 816140 (0.0008) [2023-12-27 11:16:43,948][20381] Updated weights for policy 1, policy_version 815645 (0.0006) [2023-12-27 11:16:43,954][20347] Updated weights for policy 0, policy_version 816150 (0.0006) [2023-12-27 11:16:44,001][20381] Updated weights for policy 1, policy_version 815655 (0.0006) [2023-12-27 11:16:44,015][20347] Updated weights for policy 0, policy_version 816160 (0.0006) [2023-12-27 11:16:44,061][20381] Updated weights for policy 1, policy_version 815665 (0.0007) [2023-12-27 11:16:44,724][20381] Updated weights for policy 1, policy_version 815675 (0.0007) [2023-12-27 11:16:44,738][20347] Updated weights for policy 0, policy_version 816170 (0.0007) [2023-12-27 11:16:44,790][20381] Updated weights for policy 1, policy_version 815685 (0.0007) [2023-12-27 11:16:44,811][20347] Updated weights for policy 0, policy_version 816180 (0.0007) [2023-12-27 11:16:44,852][20381] Updated weights for policy 1, policy_version 815695 (0.0007) [2023-12-27 11:16:44,868][20347] Updated weights for policy 0, policy_version 816190 (0.0007) [2023-12-27 11:16:45,580][20347] Updated weights for policy 0, policy_version 816200 (0.0007) [2023-12-27 11:16:45,629][20381] Updated weights for policy 1, policy_version 815705 (0.0008) [2023-12-27 11:16:45,635][20347] Updated weights for policy 0, policy_version 816210 (0.0007) [2023-12-27 11:16:45,685][20381] Updated weights for policy 1, policy_version 815715 (0.0005) [2023-12-27 11:16:45,694][20347] Updated weights for policy 0, policy_version 816220 (0.0007) [2023-12-27 11:16:45,745][20381] Updated weights for policy 1, policy_version 815725 (0.0005) [2023-12-27 11:16:45,808][20381] Updated weights for policy 1, policy_version 815735 (0.0007) [2023-12-27 11:16:46,388][20347] Updated weights for policy 0, policy_version 816230 (0.0008) [2023-12-27 11:16:46,449][20347] Updated weights for policy 0, policy_version 816240 (0.0007) [2023-12-27 11:16:46,472][20381] Updated weights for policy 1, policy_version 815745 (0.0007) [2023-12-27 11:16:46,505][20347] Updated weights for policy 0, policy_version 816250 (0.0006) [2023-12-27 11:16:46,529][20381] Updated weights for policy 1, policy_version 815755 (0.0007) [2023-12-27 11:16:46,585][20381] Updated weights for policy 1, policy_version 815765 (0.0009) [2023-12-27 11:16:47,197][20347] Updated weights for policy 0, policy_version 816260 (0.0007) [2023-12-27 11:16:47,254][20347] Updated weights for policy 0, policy_version 816270 (0.0008) [2023-12-27 11:16:47,257][20381] Updated weights for policy 1, policy_version 815775 (0.0007) [2023-12-27 11:16:47,313][20347] Updated weights for policy 0, policy_version 816280 (0.0007) [2023-12-27 11:16:47,319][20381] Updated weights for policy 1, policy_version 815785 (0.0007) [2023-12-27 11:16:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19660.8). Total num frames: 417857536. Throughput: 0: 9719.2, 1: 9661.4. Samples: 417867976. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:47,363][19170] Avg episode reward: [(0, '970.780'), (1, '1000.000')] [2023-12-27 11:16:47,379][20381] Updated weights for policy 1, policy_version 815795 (0.0006) [2023-12-27 11:16:47,954][20381] Updated weights for policy 1, policy_version 815805 (0.0006) [2023-12-27 11:16:48,020][20381] Updated weights for policy 1, policy_version 815815 (0.0007) [2023-12-27 11:16:48,030][20347] Updated weights for policy 0, policy_version 816290 (0.0007) [2023-12-27 11:16:48,075][20381] Updated weights for policy 1, policy_version 815825 (0.0006) [2023-12-27 11:16:48,085][20347] Updated weights for policy 0, policy_version 816300 (0.0006) [2023-12-27 11:16:48,144][20347] Updated weights for policy 0, policy_version 816310 (0.0007) [2023-12-27 11:16:48,209][20347] Updated weights for policy 0, policy_version 816320 (0.0008) [2023-12-27 11:16:48,774][20381] Updated weights for policy 1, policy_version 815835 (0.0007) [2023-12-27 11:16:48,844][20381] Updated weights for policy 1, policy_version 815845 (0.0008) [2023-12-27 11:16:48,869][20347] Updated weights for policy 0, policy_version 816330 (0.0008) [2023-12-27 11:16:48,899][20381] Updated weights for policy 1, policy_version 815855 (0.0006) [2023-12-27 11:16:48,928][20347] Updated weights for policy 0, policy_version 816340 (0.0007) [2023-12-27 11:16:48,980][20347] Updated weights for policy 0, policy_version 816350 (0.0007) [2023-12-27 11:16:49,582][20381] Updated weights for policy 1, policy_version 815865 (0.0007) [2023-12-27 11:16:49,644][20381] Updated weights for policy 1, policy_version 815875 (0.0007) [2023-12-27 11:16:49,691][20347] Updated weights for policy 0, policy_version 816360 (0.0007) [2023-12-27 11:16:49,703][20381] Updated weights for policy 1, policy_version 815885 (0.0006) [2023-12-27 11:16:49,748][20347] Updated weights for policy 0, policy_version 816370 (0.0007) [2023-12-27 11:16:49,760][20381] Updated weights for policy 1, policy_version 815895 (0.0006) [2023-12-27 11:16:49,813][20347] Updated weights for policy 0, policy_version 816380 (0.0009) [2023-12-27 11:16:50,514][20381] Updated weights for policy 1, policy_version 815905 (0.0008) [2023-12-27 11:16:50,578][20381] Updated weights for policy 1, policy_version 815915 (0.0008) [2023-12-27 11:16:50,588][20347] Updated weights for policy 0, policy_version 816390 (0.0008) [2023-12-27 11:16:50,642][20381] Updated weights for policy 1, policy_version 815925 (0.0008) [2023-12-27 11:16:50,643][20347] Updated weights for policy 0, policy_version 816400 (0.0007) [2023-12-27 11:16:50,694][20347] Updated weights for policy 0, policy_version 816410 (0.0007) [2023-12-27 11:16:51,427][20347] Updated weights for policy 0, policy_version 816420 (0.0007) [2023-12-27 11:16:51,441][20381] Updated weights for policy 1, policy_version 815935 (0.0008) [2023-12-27 11:16:51,490][20347] Updated weights for policy 0, policy_version 816430 (0.0007) [2023-12-27 11:16:51,504][20381] Updated weights for policy 1, policy_version 815945 (0.0008) [2023-12-27 11:16:51,552][20347] Updated weights for policy 0, policy_version 816440 (0.0006) [2023-12-27 11:16:51,564][20381] Updated weights for policy 1, policy_version 815955 (0.0008) [2023-12-27 11:16:52,227][20347] Updated weights for policy 0, policy_version 816450 (0.0007) [2023-12-27 11:16:52,289][20347] Updated weights for policy 0, policy_version 816460 (0.0007) [2023-12-27 11:16:52,351][20381] Updated weights for policy 1, policy_version 815965 (0.0007) [2023-12-27 11:16:52,358][20347] Updated weights for policy 0, policy_version 816470 (0.0006) [2023-12-27 11:16:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19633.0). Total num frames: 417947648. Throughput: 0: 9697.8, 1: 9668.5. Samples: 417927236. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:52,363][19170] Avg episode reward: [(0, '951.290'), (1, '1000.000')] [2023-12-27 11:16:52,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000816480_209043456.pth... [2023-12-27 11:16:52,419][20347] Updated weights for policy 0, policy_version 816480 (0.0006) [2023-12-27 11:16:52,420][20381] Updated weights for policy 1, policy_version 815975 (0.0008) [2023-12-27 11:16:52,420][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000815328_208748544.pth [2023-12-27 11:16:52,484][20381] Updated weights for policy 1, policy_version 815985 (0.0008) [2023-12-27 11:16:52,526][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000815992_208920576.pth... [2023-12-27 11:16:52,531][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000814840_208625664.pth [2023-12-27 11:16:53,051][20347] Updated weights for policy 0, policy_version 816490 (0.0006) [2023-12-27 11:16:53,112][20347] Updated weights for policy 0, policy_version 816500 (0.0006) [2023-12-27 11:16:53,176][20347] Updated weights for policy 0, policy_version 816510 (0.0005) [2023-12-27 11:16:53,242][20381] Updated weights for policy 1, policy_version 815995 (0.0008) [2023-12-27 11:16:53,305][20381] Updated weights for policy 1, policy_version 816005 (0.0008) [2023-12-27 11:16:53,363][20381] Updated weights for policy 1, policy_version 816015 (0.0008) [2023-12-27 11:16:53,824][20347] Updated weights for policy 0, policy_version 816520 (0.0006) [2023-12-27 11:16:53,885][20347] Updated weights for policy 0, policy_version 816530 (0.0006) [2023-12-27 11:16:53,945][20347] Updated weights for policy 0, policy_version 816540 (0.0006) [2023-12-27 11:16:54,045][20381] Updated weights for policy 1, policy_version 816025 (0.0007) [2023-12-27 11:16:54,108][20381] Updated weights for policy 1, policy_version 816035 (0.0008) [2023-12-27 11:16:54,171][20381] Updated weights for policy 1, policy_version 816045 (0.0008) [2023-12-27 11:16:54,232][20381] Updated weights for policy 1, policy_version 816055 (0.0008) [2023-12-27 11:16:54,623][20347] Updated weights for policy 0, policy_version 816550 (0.0008) [2023-12-27 11:16:54,684][20347] Updated weights for policy 0, policy_version 816560 (0.0008) [2023-12-27 11:16:54,749][20347] Updated weights for policy 0, policy_version 816570 (0.0008) [2023-12-27 11:16:54,931][20381] Updated weights for policy 1, policy_version 816065 (0.0008) [2023-12-27 11:16:54,995][20381] Updated weights for policy 1, policy_version 816075 (0.0008) [2023-12-27 11:16:55,057][20381] Updated weights for policy 1, policy_version 816085 (0.0008) [2023-12-27 11:16:55,441][20347] Updated weights for policy 0, policy_version 816580 (0.0007) [2023-12-27 11:16:55,499][20347] Updated weights for policy 0, policy_version 816590 (0.0006) [2023-12-27 11:16:55,549][20347] Updated weights for policy 0, policy_version 816600 (0.0008) [2023-12-27 11:16:55,757][20381] Updated weights for policy 1, policy_version 816095 (0.0007) [2023-12-27 11:16:55,814][20381] Updated weights for policy 1, policy_version 816105 (0.0008) [2023-12-27 11:16:55,864][20381] Updated weights for policy 1, policy_version 816115 (0.0007) [2023-12-27 11:16:56,241][20347] Updated weights for policy 0, policy_version 816610 (0.0008) [2023-12-27 11:16:56,288][20347] Updated weights for policy 0, policy_version 816620 (0.0008) [2023-12-27 11:16:56,349][20347] Updated weights for policy 0, policy_version 816630 (0.0008) [2023-12-27 11:16:56,402][20347] Updated weights for policy 0, policy_version 816640 (0.0007) [2023-12-27 11:16:56,570][20381] Updated weights for policy 1, policy_version 816125 (0.0007) [2023-12-27 11:16:56,618][20381] Updated weights for policy 1, policy_version 816135 (0.0006) [2023-12-27 11:16:56,667][20381] Updated weights for policy 1, policy_version 816145 (0.0006) [2023-12-27 11:16:57,126][20347] Updated weights for policy 0, policy_version 816650 (0.0007) [2023-12-27 11:16:57,179][20347] Updated weights for policy 0, policy_version 816660 (0.0008) [2023-12-27 11:16:57,235][20347] Updated weights for policy 0, policy_version 816670 (0.0007) [2023-12-27 11:16:57,321][20381] Updated weights for policy 1, policy_version 816155 (0.0007) [2023-12-27 11:16:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19660.8). Total num frames: 418054144. Throughput: 0: 9766.9, 1: 9677.1. Samples: 418043680. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:16:57,363][19170] Avg episode reward: [(0, '951.320'), (1, '1000.000')] [2023-12-27 11:16:57,381][20381] Updated weights for policy 1, policy_version 816165 (0.0008) [2023-12-27 11:16:57,433][20381] Updated weights for policy 1, policy_version 816175 (0.0007) [2023-12-27 11:16:57,940][20347] Updated weights for policy 0, policy_version 816680 (0.0007) [2023-12-27 11:16:57,995][20347] Updated weights for policy 0, policy_version 816690 (0.0008) [2023-12-27 11:16:58,050][20347] Updated weights for policy 0, policy_version 816700 (0.0007) [2023-12-27 11:16:58,135][20381] Updated weights for policy 1, policy_version 816185 (0.0008) [2023-12-27 11:16:58,201][20381] Updated weights for policy 1, policy_version 816195 (0.0008) [2023-12-27 11:16:58,267][20381] Updated weights for policy 1, policy_version 816205 (0.0007) [2023-12-27 11:16:58,330][20381] Updated weights for policy 1, policy_version 816215 (0.0008) [2023-12-27 11:16:58,856][20347] Updated weights for policy 0, policy_version 816710 (0.0008) [2023-12-27 11:16:58,927][20347] Updated weights for policy 0, policy_version 816720 (0.0008) [2023-12-27 11:16:58,990][20347] Updated weights for policy 0, policy_version 816730 (0.0008) [2023-12-27 11:16:59,124][20381] Updated weights for policy 1, policy_version 816225 (0.0008) [2023-12-27 11:16:59,181][20381] Updated weights for policy 1, policy_version 816235 (0.0007) [2023-12-27 11:16:59,235][20381] Updated weights for policy 1, policy_version 816245 (0.0008) [2023-12-27 11:16:59,705][20347] Updated weights for policy 0, policy_version 816740 (0.0007) [2023-12-27 11:16:59,766][20347] Updated weights for policy 0, policy_version 816750 (0.0006) [2023-12-27 11:16:59,830][20347] Updated weights for policy 0, policy_version 816760 (0.0006) [2023-12-27 11:16:59,981][20381] Updated weights for policy 1, policy_version 816255 (0.0007) [2023-12-27 11:17:00,041][20381] Updated weights for policy 1, policy_version 816265 (0.0008) [2023-12-27 11:17:00,101][20381] Updated weights for policy 1, policy_version 816275 (0.0008) [2023-12-27 11:17:00,480][20347] Updated weights for policy 0, policy_version 816770 (0.0007) [2023-12-27 11:17:00,547][20347] Updated weights for policy 0, policy_version 816780 (0.0007) [2023-12-27 11:17:00,610][20347] Updated weights for policy 0, policy_version 816790 (0.0006) [2023-12-27 11:17:00,662][20291] KL-divergence is very high: 110.6225 [2023-12-27 11:17:00,675][20347] Updated weights for policy 0, policy_version 816800 (0.0007) [2023-12-27 11:17:00,794][20381] Updated weights for policy 1, policy_version 816285 (0.0008) [2023-12-27 11:17:00,854][20381] Updated weights for policy 1, policy_version 816295 (0.0006) [2023-12-27 11:17:00,906][20381] Updated weights for policy 1, policy_version 816305 (0.0006) [2023-12-27 11:17:01,384][20347] Updated weights for policy 0, policy_version 816810 (0.0008) [2023-12-27 11:17:01,443][20347] Updated weights for policy 0, policy_version 816820 (0.0008) [2023-12-27 11:17:01,501][20347] Updated weights for policy 0, policy_version 816830 (0.0008) [2023-12-27 11:17:01,568][20381] Updated weights for policy 1, policy_version 816315 (0.0006) [2023-12-27 11:17:01,635][20381] Updated weights for policy 1, policy_version 816325 (0.0007) [2023-12-27 11:17:01,697][20381] Updated weights for policy 1, policy_version 816335 (0.0005) [2023-12-27 11:17:02,201][20347] Updated weights for policy 0, policy_version 816840 (0.0007) [2023-12-27 11:17:02,265][20347] Updated weights for policy 0, policy_version 816850 (0.0007) [2023-12-27 11:17:02,320][20347] Updated weights for policy 0, policy_version 816860 (0.0005) [2023-12-27 11:17:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19524.2, 300 sec: 19633.0). Total num frames: 418152448. Throughput: 0: 9773.1, 1: 9700.6. Samples: 418161444. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 11:17:02,363][19170] Avg episode reward: [(0, '961.010'), (1, '1000.000')] [2023-12-27 11:17:02,388][20381] Updated weights for policy 1, policy_version 816345 (0.0008) [2023-12-27 11:17:02,454][20381] Updated weights for policy 1, policy_version 816355 (0.0008) [2023-12-27 11:17:02,518][20381] Updated weights for policy 1, policy_version 816365 (0.0010) [2023-12-27 11:17:02,580][20381] Updated weights for policy 1, policy_version 816375 (0.0008) [2023-12-27 11:17:02,988][20347] Updated weights for policy 0, policy_version 816870 (0.0007) [2023-12-27 11:17:03,045][20347] Updated weights for policy 0, policy_version 816880 (0.0008) [2023-12-27 11:17:03,101][20347] Updated weights for policy 0, policy_version 816890 (0.0008) [2023-12-27 11:17:03,261][20381] Updated weights for policy 1, policy_version 816385 (0.0007) [2023-12-27 11:17:03,320][20381] Updated weights for policy 1, policy_version 816395 (0.0008) [2023-12-27 11:17:03,374][20381] Updated weights for policy 1, policy_version 816405 (0.0007) [2023-12-27 11:17:03,745][20347] Updated weights for policy 0, policy_version 816900 (0.0006) [2023-12-27 11:17:03,795][20347] Updated weights for policy 0, policy_version 816910 (0.0005) [2023-12-27 11:17:03,855][20347] Updated weights for policy 0, policy_version 816920 (0.0006) [2023-12-27 11:17:04,119][20381] Updated weights for policy 1, policy_version 816415 (0.0007) [2023-12-27 11:17:04,182][20381] Updated weights for policy 1, policy_version 816425 (0.0007) [2023-12-27 11:17:04,244][20381] Updated weights for policy 1, policy_version 816435 (0.0008) [2023-12-27 11:17:04,533][20347] Updated weights for policy 0, policy_version 816930 (0.0007) [2023-12-27 11:17:04,601][20347] Updated weights for policy 0, policy_version 816940 (0.0008) [2023-12-27 11:17:04,663][20347] Updated weights for policy 0, policy_version 816950 (0.0008) [2023-12-27 11:17:04,726][20347] Updated weights for policy 0, policy_version 816960 (0.0006) [2023-12-27 11:17:04,949][20381] Updated weights for policy 1, policy_version 816445 (0.0007) [2023-12-27 11:17:05,010][20381] Updated weights for policy 1, policy_version 816455 (0.0006) [2023-12-27 11:17:05,072][20381] Updated weights for policy 1, policy_version 816465 (0.0008) [2023-12-27 11:17:05,344][20347] Updated weights for policy 0, policy_version 816970 (0.0007) [2023-12-27 11:17:05,401][20347] Updated weights for policy 0, policy_version 816980 (0.0008) [2023-12-27 11:17:05,455][20347] Updated weights for policy 0, policy_version 816990 (0.0008) [2023-12-27 11:17:05,769][20381] Updated weights for policy 1, policy_version 816475 (0.0008) [2023-12-27 11:17:05,831][20381] Updated weights for policy 1, policy_version 816485 (0.0008) [2023-12-27 11:17:05,892][20381] Updated weights for policy 1, policy_version 816495 (0.0007) [2023-12-27 11:17:06,147][20347] Updated weights for policy 0, policy_version 817000 (0.0009) [2023-12-27 11:17:06,207][20347] Updated weights for policy 0, policy_version 817010 (0.0008) [2023-12-27 11:17:06,272][20347] Updated weights for policy 0, policy_version 817020 (0.0008) [2023-12-27 11:17:06,626][20381] Updated weights for policy 1, policy_version 816505 (0.0008) [2023-12-27 11:17:06,688][20381] Updated weights for policy 1, policy_version 816515 (0.0008) [2023-12-27 11:17:06,748][20381] Updated weights for policy 1, policy_version 816525 (0.0008) [2023-12-27 11:17:06,804][20381] Updated weights for policy 1, policy_version 816535 (0.0008) [2023-12-27 11:17:06,992][20347] Updated weights for policy 0, policy_version 817030 (0.0007) [2023-12-27 11:17:07,054][20347] Updated weights for policy 0, policy_version 817040 (0.0006) [2023-12-27 11:17:07,106][20347] Updated weights for policy 0, policy_version 817050 (0.0005) [2023-12-27 11:17:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19633.0). Total num frames: 418250752. Throughput: 0: 9796.2, 1: 9729.8. Samples: 418221388. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:07,363][19170] Avg episode reward: [(0, '970.780'), (1, '998.870')] [2023-12-27 11:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000816536_209059840.pth... [2023-12-27 11:17:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000817056_209190912.pth... [2023-12-27 11:17:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000815416_208773120.pth [2023-12-27 11:17:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000815904_208896000.pth [2023-12-27 11:17:07,472][20381] Updated weights for policy 1, policy_version 816545 (0.0006) [2023-12-27 11:17:07,529][20381] Updated weights for policy 1, policy_version 816555 (0.0006) [2023-12-27 11:17:07,585][20381] Updated weights for policy 1, policy_version 816565 (0.0006) [2023-12-27 11:17:07,800][20347] Updated weights for policy 0, policy_version 817060 (0.0007) [2023-12-27 11:17:07,848][20347] Updated weights for policy 0, policy_version 817070 (0.0007) [2023-12-27 11:17:07,910][20347] Updated weights for policy 0, policy_version 817080 (0.0009) [2023-12-27 11:17:08,246][20381] Updated weights for policy 1, policy_version 816575 (0.0007) [2023-12-27 11:17:08,304][20381] Updated weights for policy 1, policy_version 816585 (0.0007) [2023-12-27 11:17:08,370][20381] Updated weights for policy 1, policy_version 816595 (0.0007) [2023-12-27 11:17:08,698][20347] Updated weights for policy 0, policy_version 817090 (0.0008) [2023-12-27 11:17:08,755][20347] Updated weights for policy 0, policy_version 817100 (0.0008) [2023-12-27 11:17:08,820][20347] Updated weights for policy 0, policy_version 817110 (0.0008) [2023-12-27 11:17:08,884][20347] Updated weights for policy 0, policy_version 817120 (0.0008) [2023-12-27 11:17:09,086][20381] Updated weights for policy 1, policy_version 816605 (0.0008) [2023-12-27 11:17:09,151][20381] Updated weights for policy 1, policy_version 816615 (0.0008) [2023-12-27 11:17:09,219][20381] Updated weights for policy 1, policy_version 816625 (0.0008) [2023-12-27 11:17:09,659][20347] Updated weights for policy 0, policy_version 817130 (0.0007) [2023-12-27 11:17:09,732][20347] Updated weights for policy 0, policy_version 817140 (0.0007) [2023-12-27 11:17:09,798][20347] Updated weights for policy 0, policy_version 817150 (0.0006) [2023-12-27 11:17:09,984][20381] Updated weights for policy 1, policy_version 816635 (0.0008) [2023-12-27 11:17:10,050][20381] Updated weights for policy 1, policy_version 816645 (0.0007) [2023-12-27 11:17:10,111][20381] Updated weights for policy 1, policy_version 816655 (0.0007) [2023-12-27 11:17:10,502][20347] Updated weights for policy 0, policy_version 817160 (0.0008) [2023-12-27 11:17:10,568][20347] Updated weights for policy 0, policy_version 817170 (0.0008) [2023-12-27 11:17:10,632][20347] Updated weights for policy 0, policy_version 817180 (0.0008) [2023-12-27 11:17:10,784][20381] Updated weights for policy 1, policy_version 816665 (0.0008) [2023-12-27 11:17:10,841][20381] Updated weights for policy 1, policy_version 816675 (0.0006) [2023-12-27 11:17:10,900][20381] Updated weights for policy 1, policy_version 816685 (0.0007) [2023-12-27 11:17:10,958][20381] Updated weights for policy 1, policy_version 816695 (0.0008) [2023-12-27 11:17:11,377][20347] Updated weights for policy 0, policy_version 817190 (0.0009) [2023-12-27 11:17:11,445][20347] Updated weights for policy 0, policy_version 817200 (0.0009) [2023-12-27 11:17:11,519][20347] Updated weights for policy 0, policy_version 817210 (0.0008) [2023-12-27 11:17:11,844][20381] Updated weights for policy 1, policy_version 816705 (0.0008) [2023-12-27 11:17:11,920][20381] Updated weights for policy 1, policy_version 816715 (0.0008) [2023-12-27 11:17:11,991][20381] Updated weights for policy 1, policy_version 816725 (0.0008) [2023-12-27 11:17:12,340][20347] Updated weights for policy 0, policy_version 817220 (0.0008) [2023-12-27 11:17:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 418340864. Throughput: 0: 9813.9, 1: 9740.1. Samples: 418337728. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:12,363][19170] Avg episode reward: [(0, '990.220'), (1, '998.870')] [2023-12-27 11:17:12,406][20347] Updated weights for policy 0, policy_version 817230 (0.0007) [2023-12-27 11:17:12,467][20347] Updated weights for policy 0, policy_version 817240 (0.0007) [2023-12-27 11:17:12,716][20381] Updated weights for policy 1, policy_version 816735 (0.0010) [2023-12-27 11:17:12,777][20381] Updated weights for policy 1, policy_version 816745 (0.0011) [2023-12-27 11:17:12,844][20381] Updated weights for policy 1, policy_version 816755 (0.0007) [2023-12-27 11:17:13,141][20347] Updated weights for policy 0, policy_version 817250 (0.0008) [2023-12-27 11:17:13,205][20347] Updated weights for policy 0, policy_version 817260 (0.0007) [2023-12-27 11:17:13,270][20347] Updated weights for policy 0, policy_version 817270 (0.0007) [2023-12-27 11:17:13,328][20347] Updated weights for policy 0, policy_version 817280 (0.0006) [2023-12-27 11:17:13,541][20381] Updated weights for policy 1, policy_version 816765 (0.0008) [2023-12-27 11:17:13,598][20381] Updated weights for policy 1, policy_version 816775 (0.0011) [2023-12-27 11:17:13,648][20381] Updated weights for policy 1, policy_version 816785 (0.0007) [2023-12-27 11:17:14,003][20347] Updated weights for policy 0, policy_version 817290 (0.0008) [2023-12-27 11:17:14,059][20347] Updated weights for policy 0, policy_version 817300 (0.0006) [2023-12-27 11:17:14,119][20347] Updated weights for policy 0, policy_version 817310 (0.0006) [2023-12-27 11:17:14,355][20381] Updated weights for policy 1, policy_version 816795 (0.0006) [2023-12-27 11:17:14,421][20381] Updated weights for policy 1, policy_version 816805 (0.0006) [2023-12-27 11:17:14,487][20381] Updated weights for policy 1, policy_version 816815 (0.0006) [2023-12-27 11:17:14,815][20347] Updated weights for policy 0, policy_version 817320 (0.0006) [2023-12-27 11:17:14,880][20347] Updated weights for policy 0, policy_version 817330 (0.0008) [2023-12-27 11:17:14,948][20347] Updated weights for policy 0, policy_version 817340 (0.0008) [2023-12-27 11:17:15,173][20381] Updated weights for policy 1, policy_version 816825 (0.0011) [2023-12-27 11:17:15,237][20381] Updated weights for policy 1, policy_version 816835 (0.0011) [2023-12-27 11:17:15,301][20381] Updated weights for policy 1, policy_version 816845 (0.0011) [2023-12-27 11:17:15,374][20381] Updated weights for policy 1, policy_version 816855 (0.0011) [2023-12-27 11:17:15,698][20347] Updated weights for policy 0, policy_version 817350 (0.0007) [2023-12-27 11:17:15,755][20347] Updated weights for policy 0, policy_version 817360 (0.0005) [2023-12-27 11:17:15,806][20347] Updated weights for policy 0, policy_version 817370 (0.0006) [2023-12-27 11:17:16,083][20381] Updated weights for policy 1, policy_version 816865 (0.0010) [2023-12-27 11:17:16,136][20381] Updated weights for policy 1, policy_version 816875 (0.0006) [2023-12-27 11:17:16,192][20381] Updated weights for policy 1, policy_version 816885 (0.0005) [2023-12-27 11:17:16,458][20347] Updated weights for policy 0, policy_version 817380 (0.0007) [2023-12-27 11:17:16,521][20347] Updated weights for policy 0, policy_version 817390 (0.0007) [2023-12-27 11:17:16,576][20347] Updated weights for policy 0, policy_version 817400 (0.0007) [2023-12-27 11:17:16,855][20381] Updated weights for policy 1, policy_version 816895 (0.0010) [2023-12-27 11:17:16,913][20381] Updated weights for policy 1, policy_version 816905 (0.0009) [2023-12-27 11:17:16,968][20381] Updated weights for policy 1, policy_version 816915 (0.0007) [2023-12-27 11:17:17,245][20347] Updated weights for policy 0, policy_version 817410 (0.0007) [2023-12-27 11:17:17,298][20347] Updated weights for policy 0, policy_version 817420 (0.0010) [2023-12-27 11:17:17,357][20347] Updated weights for policy 0, policy_version 817430 (0.0008) [2023-12-27 11:17:17,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19577.5). Total num frames: 418439168. Throughput: 0: 9788.9, 1: 9721.3. Samples: 418453512. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 11:17:17,414][20347] Updated weights for policy 0, policy_version 817440 (0.0006) [2023-12-27 11:17:17,565][20381] Updated weights for policy 1, policy_version 816925 (0.0006) [2023-12-27 11:17:17,620][20381] Updated weights for policy 1, policy_version 816935 (0.0008) [2023-12-27 11:17:17,675][20381] Updated weights for policy 1, policy_version 816945 (0.0007) [2023-12-27 11:17:18,145][20347] Updated weights for policy 0, policy_version 817450 (0.0008) [2023-12-27 11:17:18,203][20347] Updated weights for policy 0, policy_version 817460 (0.0009) [2023-12-27 11:17:18,251][20347] Updated weights for policy 0, policy_version 817470 (0.0007) [2023-12-27 11:17:18,319][20381] Updated weights for policy 1, policy_version 816955 (0.0008) [2023-12-27 11:17:18,381][20381] Updated weights for policy 1, policy_version 816965 (0.0008) [2023-12-27 11:17:18,439][20381] Updated weights for policy 1, policy_version 816975 (0.0008) [2023-12-27 11:17:18,990][20347] Updated weights for policy 0, policy_version 817480 (0.0008) [2023-12-27 11:17:19,043][20347] Updated weights for policy 0, policy_version 817490 (0.0007) [2023-12-27 11:17:19,104][20347] Updated weights for policy 0, policy_version 817500 (0.0008) [2023-12-27 11:17:19,125][20381] Updated weights for policy 1, policy_version 816985 (0.0008) [2023-12-27 11:17:19,182][20381] Updated weights for policy 1, policy_version 816995 (0.0009) [2023-12-27 11:17:19,244][20381] Updated weights for policy 1, policy_version 817005 (0.0008) [2023-12-27 11:17:19,303][20381] Updated weights for policy 1, policy_version 817015 (0.0009) [2023-12-27 11:17:19,812][20347] Updated weights for policy 0, policy_version 817510 (0.0007) [2023-12-27 11:17:19,874][20347] Updated weights for policy 0, policy_version 817520 (0.0008) [2023-12-27 11:17:19,939][20347] Updated weights for policy 0, policy_version 817530 (0.0006) [2023-12-27 11:17:20,115][20381] Updated weights for policy 1, policy_version 817025 (0.0007) [2023-12-27 11:17:20,180][20381] Updated weights for policy 1, policy_version 817035 (0.0008) [2023-12-27 11:17:20,253][20381] Updated weights for policy 1, policy_version 817045 (0.0008) [2023-12-27 11:17:20,669][20347] Updated weights for policy 0, policy_version 817540 (0.0007) [2023-12-27 11:17:20,735][20347] Updated weights for policy 0, policy_version 817550 (0.0006) [2023-12-27 11:17:20,799][20347] Updated weights for policy 0, policy_version 817560 (0.0008) [2023-12-27 11:17:21,011][20381] Updated weights for policy 1, policy_version 817055 (0.0007) [2023-12-27 11:17:21,079][20381] Updated weights for policy 1, policy_version 817065 (0.0008) [2023-12-27 11:17:21,152][20381] Updated weights for policy 1, policy_version 817075 (0.0007) [2023-12-27 11:17:21,545][20347] Updated weights for policy 0, policy_version 817570 (0.0008) [2023-12-27 11:17:21,609][20347] Updated weights for policy 0, policy_version 817580 (0.0008) [2023-12-27 11:17:21,688][20347] Updated weights for policy 0, policy_version 817590 (0.0008) [2023-12-27 11:17:21,767][20347] Updated weights for policy 0, policy_version 817600 (0.0009) [2023-12-27 11:17:21,928][20381] Updated weights for policy 1, policy_version 817085 (0.0008) [2023-12-27 11:17:22,006][20381] Updated weights for policy 1, policy_version 817095 (0.0010) [2023-12-27 11:17:22,077][20381] Updated weights for policy 1, policy_version 817105 (0.0010) [2023-12-27 11:17:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 418537472. Throughput: 0: 9762.7, 1: 9719.6. Samples: 418511476. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:22,369][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 11:17:22,376][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000817112_209207296.pth... [2023-12-27 11:17:22,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000817600_209330176.pth... [2023-12-27 11:17:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000815992_208920576.pth [2023-12-27 11:17:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000816480_209043456.pth [2023-12-27 11:17:22,682][20347] Updated weights for policy 0, policy_version 817610 (0.0008) [2023-12-27 11:17:22,748][20347] Updated weights for policy 0, policy_version 817620 (0.0007) [2023-12-27 11:17:22,813][20347] Updated weights for policy 0, policy_version 817630 (0.0009) [2023-12-27 11:17:23,007][20381] Updated weights for policy 1, policy_version 817115 (0.0009) [2023-12-27 11:17:23,070][20381] Updated weights for policy 1, policy_version 817125 (0.0007) [2023-12-27 11:17:23,132][20381] Updated weights for policy 1, policy_version 817135 (0.0008) [2023-12-27 11:17:23,523][20347] Updated weights for policy 0, policy_version 817640 (0.0006) [2023-12-27 11:17:23,587][20347] Updated weights for policy 0, policy_version 817650 (0.0006) [2023-12-27 11:17:23,647][20347] Updated weights for policy 0, policy_version 817660 (0.0006) [2023-12-27 11:17:23,816][20381] Updated weights for policy 1, policy_version 817145 (0.0007) [2023-12-27 11:17:23,866][20381] Updated weights for policy 1, policy_version 817155 (0.0008) [2023-12-27 11:17:23,926][20381] Updated weights for policy 1, policy_version 817165 (0.0006) [2023-12-27 11:17:23,983][20381] Updated weights for policy 1, policy_version 817175 (0.0006) [2023-12-27 11:17:24,260][20347] Updated weights for policy 0, policy_version 817670 (0.0007) [2023-12-27 11:17:24,321][20347] Updated weights for policy 0, policy_version 817680 (0.0010) [2023-12-27 11:17:24,378][20347] Updated weights for policy 0, policy_version 817690 (0.0010) [2023-12-27 11:17:24,626][20381] Updated weights for policy 1, policy_version 817185 (0.0008) [2023-12-27 11:17:24,682][20381] Updated weights for policy 1, policy_version 817195 (0.0008) [2023-12-27 11:17:24,743][20381] Updated weights for policy 1, policy_version 817205 (0.0006) [2023-12-27 11:17:25,068][20347] Updated weights for policy 0, policy_version 817700 (0.0011) [2023-12-27 11:17:25,129][20347] Updated weights for policy 0, policy_version 817710 (0.0010) [2023-12-27 11:17:25,190][20347] Updated weights for policy 0, policy_version 817720 (0.0010) [2023-12-27 11:17:25,466][20381] Updated weights for policy 1, policy_version 817215 (0.0007) [2023-12-27 11:17:25,530][20381] Updated weights for policy 1, policy_version 817225 (0.0007) [2023-12-27 11:17:25,577][20381] Updated weights for policy 1, policy_version 817235 (0.0008) [2023-12-27 11:17:25,871][20347] Updated weights for policy 0, policy_version 817730 (0.0009) [2023-12-27 11:17:25,936][20347] Updated weights for policy 0, policy_version 817740 (0.0006) [2023-12-27 11:17:25,993][20347] Updated weights for policy 0, policy_version 817750 (0.0009) [2023-12-27 11:17:26,052][20347] Updated weights for policy 0, policy_version 817760 (0.0006) [2023-12-27 11:17:26,238][20381] Updated weights for policy 1, policy_version 817245 (0.0007) [2023-12-27 11:17:26,291][20381] Updated weights for policy 1, policy_version 817255 (0.0006) [2023-12-27 11:17:26,350][20381] Updated weights for policy 1, policy_version 817265 (0.0006) [2023-12-27 11:17:26,703][20347] Updated weights for policy 0, policy_version 817770 (0.0010) [2023-12-27 11:17:26,761][20347] Updated weights for policy 0, policy_version 817780 (0.0011) [2023-12-27 11:17:26,818][20347] Updated weights for policy 0, policy_version 817790 (0.0010) [2023-12-27 11:17:27,001][20381] Updated weights for policy 1, policy_version 817275 (0.0005) [2023-12-27 11:17:27,061][20381] Updated weights for policy 1, policy_version 817285 (0.0005) [2023-12-27 11:17:27,115][20381] Updated weights for policy 1, policy_version 817295 (0.0008) [2023-12-27 11:17:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 418635776. Throughput: 0: 9754.3, 1: 9670.0. Samples: 418625344. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:27,363][19170] Avg episode reward: [(0, '980.600'), (1, '1000.000')] [2023-12-27 11:17:27,513][20347] Updated weights for policy 0, policy_version 817800 (0.0011) [2023-12-27 11:17:27,566][20347] Updated weights for policy 0, policy_version 817810 (0.0010) [2023-12-27 11:17:27,618][20347] Updated weights for policy 0, policy_version 817820 (0.0010) [2023-12-27 11:17:27,772][20381] Updated weights for policy 1, policy_version 817305 (0.0007) [2023-12-27 11:17:27,835][20381] Updated weights for policy 1, policy_version 817315 (0.0007) [2023-12-27 11:17:27,891][20381] Updated weights for policy 1, policy_version 817325 (0.0008) [2023-12-27 11:17:27,944][20381] Updated weights for policy 1, policy_version 817335 (0.0007) [2023-12-27 11:17:28,276][20347] Updated weights for policy 0, policy_version 817830 (0.0008) [2023-12-27 11:17:28,336][20347] Updated weights for policy 0, policy_version 817840 (0.0007) [2023-12-27 11:17:28,393][20347] Updated weights for policy 0, policy_version 817850 (0.0007) [2023-12-27 11:17:28,570][20381] Updated weights for policy 1, policy_version 817345 (0.0008) [2023-12-27 11:17:28,628][20381] Updated weights for policy 1, policy_version 817355 (0.0009) [2023-12-27 11:17:28,686][20381] Updated weights for policy 1, policy_version 817365 (0.0006) [2023-12-27 11:17:29,069][20347] Updated weights for policy 0, policy_version 817860 (0.0007) [2023-12-27 11:17:29,135][20347] Updated weights for policy 0, policy_version 817870 (0.0007) [2023-12-27 11:17:29,185][20347] Updated weights for policy 0, policy_version 817880 (0.0007) [2023-12-27 11:17:29,392][20381] Updated weights for policy 1, policy_version 817375 (0.0008) [2023-12-27 11:17:29,445][20381] Updated weights for policy 1, policy_version 817385 (0.0009) [2023-12-27 11:17:29,499][20381] Updated weights for policy 1, policy_version 817395 (0.0008) [2023-12-27 11:17:29,923][20347] Updated weights for policy 0, policy_version 817890 (0.0007) [2023-12-27 11:17:29,996][20347] Updated weights for policy 0, policy_version 817900 (0.0008) [2023-12-27 11:17:30,059][20347] Updated weights for policy 0, policy_version 817910 (0.0008) [2023-12-27 11:17:30,123][20347] Updated weights for policy 0, policy_version 817920 (0.0008) [2023-12-27 11:17:30,237][20381] Updated weights for policy 1, policy_version 817405 (0.0010) [2023-12-27 11:17:30,295][20381] Updated weights for policy 1, policy_version 817415 (0.0011) [2023-12-27 11:17:30,352][20381] Updated weights for policy 1, policy_version 817425 (0.0011) [2023-12-27 11:17:30,752][20347] Updated weights for policy 0, policy_version 817930 (0.0006) [2023-12-27 11:17:30,803][20347] Updated weights for policy 0, policy_version 817940 (0.0008) [2023-12-27 11:17:30,857][20347] Updated weights for policy 0, policy_version 817950 (0.0007) [2023-12-27 11:17:31,035][20381] Updated weights for policy 1, policy_version 817435 (0.0008) [2023-12-27 11:17:31,096][20381] Updated weights for policy 1, policy_version 817445 (0.0009) [2023-12-27 11:17:31,162][20381] Updated weights for policy 1, policy_version 817455 (0.0009) [2023-12-27 11:17:31,594][20347] Updated weights for policy 0, policy_version 817960 (0.0008) [2023-12-27 11:17:31,661][20347] Updated weights for policy 0, policy_version 817970 (0.0008) [2023-12-27 11:17:31,723][20347] Updated weights for policy 0, policy_version 817980 (0.0008) [2023-12-27 11:17:31,882][20381] Updated weights for policy 1, policy_version 817465 (0.0010) [2023-12-27 11:17:31,949][20381] Updated weights for policy 1, policy_version 817475 (0.0009) [2023-12-27 11:17:32,007][20381] Updated weights for policy 1, policy_version 817485 (0.0007) [2023-12-27 11:17:32,064][20381] Updated weights for policy 1, policy_version 817495 (0.0008) [2023-12-27 11:17:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 418734080. Throughput: 0: 9791.1, 1: 9704.2. Samples: 418745264. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:32,365][19170] Avg episode reward: [(0, '961.270'), (1, '1000.000')] [2023-12-27 11:17:32,482][20347] Updated weights for policy 0, policy_version 817990 (0.0008) [2023-12-27 11:17:32,546][20347] Updated weights for policy 0, policy_version 818000 (0.0008) [2023-12-27 11:17:32,610][20347] Updated weights for policy 0, policy_version 818010 (0.0008) [2023-12-27 11:17:32,792][20381] Updated weights for policy 1, policy_version 817505 (0.0008) [2023-12-27 11:17:32,845][20381] Updated weights for policy 1, policy_version 817515 (0.0007) [2023-12-27 11:17:32,909][20381] Updated weights for policy 1, policy_version 817525 (0.0008) [2023-12-27 11:17:33,295][20347] Updated weights for policy 0, policy_version 818020 (0.0008) [2023-12-27 11:17:33,345][20347] Updated weights for policy 0, policy_version 818030 (0.0007) [2023-12-27 11:17:33,397][20347] Updated weights for policy 0, policy_version 818040 (0.0007) [2023-12-27 11:17:33,530][20381] Updated weights for policy 1, policy_version 817535 (0.0008) [2023-12-27 11:17:33,586][20381] Updated weights for policy 1, policy_version 817545 (0.0006) [2023-12-27 11:17:33,646][20381] Updated weights for policy 1, policy_version 817555 (0.0005) [2023-12-27 11:17:34,037][20347] Updated weights for policy 0, policy_version 818050 (0.0007) [2023-12-27 11:17:34,088][20347] Updated weights for policy 0, policy_version 818060 (0.0006) [2023-12-27 11:17:34,152][20347] Updated weights for policy 0, policy_version 818070 (0.0008) [2023-12-27 11:17:34,213][20347] Updated weights for policy 0, policy_version 818080 (0.0007) [2023-12-27 11:17:34,271][20381] Updated weights for policy 1, policy_version 817565 (0.0008) [2023-12-27 11:17:34,335][20381] Updated weights for policy 1, policy_version 817575 (0.0008) [2023-12-27 11:17:34,393][20381] Updated weights for policy 1, policy_version 817585 (0.0008) [2023-12-27 11:17:34,947][20347] Updated weights for policy 0, policy_version 818090 (0.0007) [2023-12-27 11:17:35,000][20347] Updated weights for policy 0, policy_version 818100 (0.0009) [2023-12-27 11:17:35,050][20381] Updated weights for policy 1, policy_version 817595 (0.0008) [2023-12-27 11:17:35,060][20347] Updated weights for policy 0, policy_version 818110 (0.0007) [2023-12-27 11:17:35,114][20381] Updated weights for policy 1, policy_version 817605 (0.0008) [2023-12-27 11:17:35,167][20381] Updated weights for policy 1, policy_version 817615 (0.0007) [2023-12-27 11:17:35,804][20347] Updated weights for policy 0, policy_version 818120 (0.0007) [2023-12-27 11:17:35,855][20347] Updated weights for policy 0, policy_version 818130 (0.0008) [2023-12-27 11:17:35,876][20381] Updated weights for policy 1, policy_version 817625 (0.0008) [2023-12-27 11:17:35,916][20347] Updated weights for policy 0, policy_version 818140 (0.0007) [2023-12-27 11:17:35,929][20381] Updated weights for policy 1, policy_version 817635 (0.0006) [2023-12-27 11:17:35,978][20381] Updated weights for policy 1, policy_version 817645 (0.0008) [2023-12-27 11:17:36,038][20381] Updated weights for policy 1, policy_version 817655 (0.0008) [2023-12-27 11:17:36,614][20347] Updated weights for policy 0, policy_version 818150 (0.0007) [2023-12-27 11:17:36,671][20347] Updated weights for policy 0, policy_version 818160 (0.0007) [2023-12-27 11:17:36,729][20347] Updated weights for policy 0, policy_version 818170 (0.0006) [2023-12-27 11:17:36,771][20381] Updated weights for policy 1, policy_version 817665 (0.0007) [2023-12-27 11:17:36,825][20381] Updated weights for policy 1, policy_version 817675 (0.0008) [2023-12-27 11:17:36,881][20381] Updated weights for policy 1, policy_version 817685 (0.0008) [2023-12-27 11:17:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19522.0). Total num frames: 418832384. Throughput: 0: 9805.9, 1: 9732.5. Samples: 418806464. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:37,363][19170] Avg episode reward: [(0, '951.420'), (1, '1000.000')] [2023-12-27 11:17:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000817688_209354752.pth... [2023-12-27 11:17:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000816536_209059840.pth [2023-12-27 11:17:37,385][20347] Updated weights for policy 0, policy_version 818180 (0.0006) [2023-12-27 11:17:37,441][20347] Updated weights for policy 0, policy_version 818190 (0.0007) [2023-12-27 11:17:37,501][20347] Updated weights for policy 0, policy_version 818200 (0.0007) [2023-12-27 11:17:37,548][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000818208_209485824.pth... [2023-12-27 11:17:37,552][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000817056_209190912.pth [2023-12-27 11:17:37,570][20381] Updated weights for policy 1, policy_version 817695 (0.0006) [2023-12-27 11:17:37,639][20381] Updated weights for policy 1, policy_version 817705 (0.0007) [2023-12-27 11:17:37,702][20381] Updated weights for policy 1, policy_version 817715 (0.0008) [2023-12-27 11:17:38,180][20347] Updated weights for policy 0, policy_version 818210 (0.0008) [2023-12-27 11:17:38,251][20347] Updated weights for policy 0, policy_version 818220 (0.0006) [2023-12-27 11:17:38,313][20347] Updated weights for policy 0, policy_version 818230 (0.0006) [2023-12-27 11:17:38,318][20381] Updated weights for policy 1, policy_version 817725 (0.0007) [2023-12-27 11:17:38,374][20347] Updated weights for policy 0, policy_version 818240 (0.0008) [2023-12-27 11:17:38,376][20381] Updated weights for policy 1, policy_version 817735 (0.0007) [2023-12-27 11:17:38,438][20381] Updated weights for policy 1, policy_version 817745 (0.0008) [2023-12-27 11:17:39,068][20347] Updated weights for policy 0, policy_version 818250 (0.0007) [2023-12-27 11:17:39,123][20347] Updated weights for policy 0, policy_version 818260 (0.0008) [2023-12-27 11:17:39,147][20381] Updated weights for policy 1, policy_version 817755 (0.0009) [2023-12-27 11:17:39,191][20347] Updated weights for policy 0, policy_version 818270 (0.0009) [2023-12-27 11:17:39,201][20381] Updated weights for policy 1, policy_version 817765 (0.0006) [2023-12-27 11:17:39,269][20381] Updated weights for policy 1, policy_version 817775 (0.0007) [2023-12-27 11:17:39,919][20347] Updated weights for policy 0, policy_version 818280 (0.0007) [2023-12-27 11:17:39,934][20291] KL-divergence is very high: 199.4941 [2023-12-27 11:17:39,948][20291] KL-divergence is very high: 176.9906 [2023-12-27 11:17:39,990][20347] Updated weights for policy 0, policy_version 818290 (0.0007) [2023-12-27 11:17:40,046][20291] KL-divergence is very high: 198.5630 [2023-12-27 11:17:40,047][20381] Updated weights for policy 1, policy_version 817785 (0.0008) [2023-12-27 11:17:40,057][20347] Updated weights for policy 0, policy_version 818300 (0.0008) [2023-12-27 11:17:40,059][20291] KL-divergence is very high: 153.0008 [2023-12-27 11:17:40,105][20381] Updated weights for policy 1, policy_version 817795 (0.0007) [2023-12-27 11:17:40,169][20381] Updated weights for policy 1, policy_version 817805 (0.0008) [2023-12-27 11:17:40,236][20381] Updated weights for policy 1, policy_version 817815 (0.0008) [2023-12-27 11:17:40,786][20347] Updated weights for policy 0, policy_version 818310 (0.0007) [2023-12-27 11:17:40,844][20347] Updated weights for policy 0, policy_version 818320 (0.0008) [2023-12-27 11:17:40,905][20347] Updated weights for policy 0, policy_version 818330 (0.0007) [2023-12-27 11:17:40,935][20381] Updated weights for policy 1, policy_version 817825 (0.0006) [2023-12-27 11:17:40,993][20381] Updated weights for policy 1, policy_version 817835 (0.0008) [2023-12-27 11:17:41,054][20381] Updated weights for policy 1, policy_version 817845 (0.0008) [2023-12-27 11:17:41,675][20347] Updated weights for policy 0, policy_version 818340 (0.0007) [2023-12-27 11:17:41,687][20291] KL-divergence is very high: 141.9588 [2023-12-27 11:17:41,743][20347] Updated weights for policy 0, policy_version 818350 (0.0008) [2023-12-27 11:17:41,801][20381] Updated weights for policy 1, policy_version 817855 (0.0007) [2023-12-27 11:17:41,803][20347] Updated weights for policy 0, policy_version 818360 (0.0006) [2023-12-27 11:17:41,858][20381] Updated weights for policy 1, policy_version 817865 (0.0007) [2023-12-27 11:17:41,917][20381] Updated weights for policy 1, policy_version 817875 (0.0007) [2023-12-27 11:17:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 418930688. Throughput: 0: 9769.8, 1: 9781.2. Samples: 418923472. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:42,363][19170] Avg episode reward: [(0, '903.070'), (1, '1000.000')] [2023-12-27 11:17:42,561][20347] Updated weights for policy 0, policy_version 818370 (0.0006) [2023-12-27 11:17:42,625][20347] Updated weights for policy 0, policy_version 818380 (0.0008) [2023-12-27 11:17:42,656][20381] Updated weights for policy 1, policy_version 817885 (0.0008) [2023-12-27 11:17:42,692][20347] Updated weights for policy 0, policy_version 818390 (0.0008) [2023-12-27 11:17:42,713][20381] Updated weights for policy 1, policy_version 817895 (0.0007) [2023-12-27 11:17:42,752][20347] Updated weights for policy 0, policy_version 818400 (0.0008) [2023-12-27 11:17:42,775][20381] Updated weights for policy 1, policy_version 817905 (0.0007) [2023-12-27 11:17:43,482][20381] Updated weights for policy 1, policy_version 817915 (0.0007) [2023-12-27 11:17:43,507][20347] Updated weights for policy 0, policy_version 818410 (0.0006) [2023-12-27 11:17:43,548][20381] Updated weights for policy 1, policy_version 817925 (0.0006) [2023-12-27 11:17:43,560][20347] Updated weights for policy 0, policy_version 818420 (0.0006) [2023-12-27 11:17:43,606][20381] Updated weights for policy 1, policy_version 817935 (0.0006) [2023-12-27 11:17:43,623][20347] Updated weights for policy 0, policy_version 818430 (0.0006) [2023-12-27 11:17:44,197][20381] Updated weights for policy 1, policy_version 817945 (0.0007) [2023-12-27 11:17:44,254][20381] Updated weights for policy 1, policy_version 817955 (0.0006) [2023-12-27 11:17:44,313][20381] Updated weights for policy 1, policy_version 817965 (0.0006) [2023-12-27 11:17:44,335][20347] Updated weights for policy 0, policy_version 818440 (0.0007) [2023-12-27 11:17:44,377][20381] Updated weights for policy 1, policy_version 817975 (0.0006) [2023-12-27 11:17:44,391][20347] Updated weights for policy 0, policy_version 818450 (0.0008) [2023-12-27 11:17:44,446][20347] Updated weights for policy 0, policy_version 818460 (0.0007) [2023-12-27 11:17:45,007][20381] Updated weights for policy 1, policy_version 817985 (0.0007) [2023-12-27 11:17:45,069][20381] Updated weights for policy 1, policy_version 817995 (0.0007) [2023-12-27 11:17:45,127][20381] Updated weights for policy 1, policy_version 818005 (0.0008) [2023-12-27 11:17:45,188][20347] Updated weights for policy 0, policy_version 818470 (0.0007) [2023-12-27 11:17:45,215][20291] KL-divergence is very high: 204.8705 [2023-12-27 11:17:45,254][20347] Updated weights for policy 0, policy_version 818480 (0.0008) [2023-12-27 11:17:45,264][20291] KL-divergence is very high: 125.3802 [2023-12-27 11:17:45,316][20347] Updated weights for policy 0, policy_version 818490 (0.0008) [2023-12-27 11:17:45,802][20381] Updated weights for policy 1, policy_version 818015 (0.0008) [2023-12-27 11:17:45,857][20381] Updated weights for policy 1, policy_version 818025 (0.0008) [2023-12-27 11:17:45,923][20381] Updated weights for policy 1, policy_version 818035 (0.0008) [2023-12-27 11:17:46,047][20347] Updated weights for policy 0, policy_version 818500 (0.0007) [2023-12-27 11:17:46,106][20347] Updated weights for policy 0, policy_version 818510 (0.0007) [2023-12-27 11:17:46,162][20347] Updated weights for policy 0, policy_version 818520 (0.0008) [2023-12-27 11:17:46,647][20381] Updated weights for policy 1, policy_version 818045 (0.0008) [2023-12-27 11:17:46,711][20381] Updated weights for policy 1, policy_version 818055 (0.0008) [2023-12-27 11:17:46,773][20381] Updated weights for policy 1, policy_version 818065 (0.0008) [2023-12-27 11:17:46,836][20347] Updated weights for policy 0, policy_version 818530 (0.0007) [2023-12-27 11:17:46,897][20347] Updated weights for policy 0, policy_version 818540 (0.0008) [2023-12-27 11:17:46,956][20347] Updated weights for policy 0, policy_version 818550 (0.0008) [2023-12-27 11:17:47,022][20347] Updated weights for policy 0, policy_version 818560 (0.0008) [2023-12-27 11:17:47,363][19170] Fps is (10 sec: 19660.3, 60 sec: 19524.2, 300 sec: 19466.4). Total num frames: 419028992. Throughput: 0: 9721.4, 1: 9809.8. Samples: 419040352. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:47,363][19170] Avg episode reward: [(0, '873.950'), (1, '1000.000')] [2023-12-27 11:17:47,465][20381] Updated weights for policy 1, policy_version 818075 (0.0008) [2023-12-27 11:17:47,522][20381] Updated weights for policy 1, policy_version 818085 (0.0008) [2023-12-27 11:17:47,579][20381] Updated weights for policy 1, policy_version 818095 (0.0008) [2023-12-27 11:17:47,740][20347] Updated weights for policy 0, policy_version 818570 (0.0007) [2023-12-27 11:17:47,794][20347] Updated weights for policy 0, policy_version 818580 (0.0006) [2023-12-27 11:17:47,848][20347] Updated weights for policy 0, policy_version 818590 (0.0007) [2023-12-27 11:17:48,259][20381] Updated weights for policy 1, policy_version 818105 (0.0008) [2023-12-27 11:17:48,321][20381] Updated weights for policy 1, policy_version 818115 (0.0009) [2023-12-27 11:17:48,382][20381] Updated weights for policy 1, policy_version 818125 (0.0008) [2023-12-27 11:17:48,442][20381] Updated weights for policy 1, policy_version 818135 (0.0008) [2023-12-27 11:17:48,559][20347] Updated weights for policy 0, policy_version 818600 (0.0007) [2023-12-27 11:17:48,622][20347] Updated weights for policy 0, policy_version 818610 (0.0007) [2023-12-27 11:17:48,684][20347] Updated weights for policy 0, policy_version 818620 (0.0008) [2023-12-27 11:17:49,213][20381] Updated weights for policy 1, policy_version 818145 (0.0008) [2023-12-27 11:17:49,277][20381] Updated weights for policy 1, policy_version 818155 (0.0008) [2023-12-27 11:17:49,339][20381] Updated weights for policy 1, policy_version 818165 (0.0008) [2023-12-27 11:17:49,438][20347] Updated weights for policy 0, policy_version 818630 (0.0007) [2023-12-27 11:17:49,497][20347] Updated weights for policy 0, policy_version 818640 (0.0008) [2023-12-27 11:17:49,556][20347] Updated weights for policy 0, policy_version 818650 (0.0008) [2023-12-27 11:17:50,152][20381] Updated weights for policy 1, policy_version 818175 (0.0008) [2023-12-27 11:17:50,212][20381] Updated weights for policy 1, policy_version 818185 (0.0008) [2023-12-27 11:17:50,270][20347] Updated weights for policy 0, policy_version 818660 (0.0008) [2023-12-27 11:17:50,276][20381] Updated weights for policy 1, policy_version 818195 (0.0008) [2023-12-27 11:17:50,333][20347] Updated weights for policy 0, policy_version 818670 (0.0007) [2023-12-27 11:17:50,390][20347] Updated weights for policy 0, policy_version 818680 (0.0008) [2023-12-27 11:17:51,017][20381] Updated weights for policy 1, policy_version 818205 (0.0007) [2023-12-27 11:17:51,087][20381] Updated weights for policy 1, policy_version 818215 (0.0007) [2023-12-27 11:17:51,122][20347] Updated weights for policy 0, policy_version 818690 (0.0007) [2023-12-27 11:17:51,163][20381] Updated weights for policy 1, policy_version 818225 (0.0008) [2023-12-27 11:17:51,194][20347] Updated weights for policy 0, policy_version 818700 (0.0008) [2023-12-27 11:17:51,259][20347] Updated weights for policy 0, policy_version 818710 (0.0008) [2023-12-27 11:17:51,323][20347] Updated weights for policy 0, policy_version 818720 (0.0008) [2023-12-27 11:17:51,965][20381] Updated weights for policy 1, policy_version 818235 (0.0007) [2023-12-27 11:17:52,018][20381] Updated weights for policy 1, policy_version 818245 (0.0009) [2023-12-27 11:17:52,040][20347] Updated weights for policy 0, policy_version 818730 (0.0010) [2023-12-27 11:17:52,081][20381] Updated weights for policy 1, policy_version 818255 (0.0006) [2023-12-27 11:17:52,101][20347] Updated weights for policy 0, policy_version 818740 (0.0011) [2023-12-27 11:17:52,160][20347] Updated weights for policy 0, policy_version 818750 (0.0009) [2023-12-27 11:17:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 419127296. Throughput: 0: 9677.5, 1: 9783.3. Samples: 419097124. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:52,363][19170] Avg episode reward: [(0, '922.270'), (1, '1000.000')] [2023-12-27 11:17:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000818264_209502208.pth... [2023-12-27 11:17:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000818752_209625088.pth... [2023-12-27 11:17:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000817112_209207296.pth [2023-12-27 11:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000817600_209330176.pth [2023-12-27 11:17:52,800][20381] Updated weights for policy 1, policy_version 818265 (0.0007) [2023-12-27 11:17:52,869][20381] Updated weights for policy 1, policy_version 818275 (0.0008) [2023-12-27 11:17:52,929][20381] Updated weights for policy 1, policy_version 818285 (0.0007) [2023-12-27 11:17:52,931][20347] Updated weights for policy 0, policy_version 818760 (0.0008) [2023-12-27 11:17:52,987][20381] Updated weights for policy 1, policy_version 818295 (0.0007) [2023-12-27 11:17:52,989][20347] Updated weights for policy 0, policy_version 818770 (0.0007) [2023-12-27 11:17:53,037][20347] Updated weights for policy 0, policy_version 818780 (0.0007) [2023-12-27 11:17:53,707][20381] Updated weights for policy 1, policy_version 818305 (0.0007) [2023-12-27 11:17:53,727][20347] Updated weights for policy 0, policy_version 818790 (0.0007) [2023-12-27 11:17:53,759][20381] Updated weights for policy 1, policy_version 818315 (0.0006) [2023-12-27 11:17:53,784][20347] Updated weights for policy 0, policy_version 818800 (0.0008) [2023-12-27 11:17:53,815][20381] Updated weights for policy 1, policy_version 818325 (0.0008) [2023-12-27 11:17:53,848][20347] Updated weights for policy 0, policy_version 818810 (0.0008) [2023-12-27 11:17:54,531][20347] Updated weights for policy 0, policy_version 818820 (0.0008) [2023-12-27 11:17:54,542][20381] Updated weights for policy 1, policy_version 818335 (0.0007) [2023-12-27 11:17:54,584][20347] Updated weights for policy 0, policy_version 818830 (0.0005) [2023-12-27 11:17:54,602][20381] Updated weights for policy 1, policy_version 818345 (0.0007) [2023-12-27 11:17:54,642][20347] Updated weights for policy 0, policy_version 818840 (0.0007) [2023-12-27 11:17:54,662][20381] Updated weights for policy 1, policy_version 818355 (0.0006) [2023-12-27 11:17:55,332][20347] Updated weights for policy 0, policy_version 818850 (0.0008) [2023-12-27 11:17:55,369][20381] Updated weights for policy 1, policy_version 818365 (0.0007) [2023-12-27 11:17:55,392][20347] Updated weights for policy 0, policy_version 818860 (0.0007) [2023-12-27 11:17:55,427][20381] Updated weights for policy 1, policy_version 818375 (0.0008) [2023-12-27 11:17:55,452][20347] Updated weights for policy 0, policy_version 818870 (0.0008) [2023-12-27 11:17:55,486][20381] Updated weights for policy 1, policy_version 818385 (0.0006) [2023-12-27 11:17:55,512][20347] Updated weights for policy 0, policy_version 818880 (0.0007) [2023-12-27 11:17:56,077][20381] Updated weights for policy 1, policy_version 818395 (0.0007) [2023-12-27 11:17:56,128][20381] Updated weights for policy 1, policy_version 818405 (0.0007) [2023-12-27 11:17:56,177][20381] Updated weights for policy 1, policy_version 818415 (0.0008) [2023-12-27 11:17:56,253][20347] Updated weights for policy 0, policy_version 818890 (0.0007) [2023-12-27 11:17:56,283][20291] KL-divergence is very high: 193.3018 [2023-12-27 11:17:56,304][20347] Updated weights for policy 0, policy_version 818900 (0.0005) [2023-12-27 11:17:56,334][20291] KL-divergence is very high: 126.8495 [2023-12-27 11:17:56,363][20347] Updated weights for policy 0, policy_version 818910 (0.0005) [2023-12-27 11:17:56,807][20381] Updated weights for policy 1, policy_version 818425 (0.0007) [2023-12-27 11:17:56,852][20381] Updated weights for policy 1, policy_version 818435 (0.0008) [2023-12-27 11:17:56,904][20381] Updated weights for policy 1, policy_version 818445 (0.0006) [2023-12-27 11:17:56,919][20347] Updated weights for policy 0, policy_version 818920 (0.0006) [2023-12-27 11:17:56,955][20381] Updated weights for policy 1, policy_version 818455 (0.0007) [2023-12-27 11:17:56,971][20347] Updated weights for policy 0, policy_version 818930 (0.0005) [2023-12-27 11:17:57,024][20347] Updated weights for policy 0, policy_version 818940 (0.0007) [2023-12-27 11:17:57,363][19170] Fps is (10 sec: 19661.0, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 419225600. Throughput: 0: 9671.8, 1: 9773.2. Samples: 419212756. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:17:57,364][19170] Avg episode reward: [(0, '893.060'), (1, '1000.000')] [2023-12-27 11:17:57,581][20347] Updated weights for policy 0, policy_version 818950 (0.0006) [2023-12-27 11:17:57,640][20347] Updated weights for policy 0, policy_version 818960 (0.0005) [2023-12-27 11:17:57,696][20347] Updated weights for policy 0, policy_version 818970 (0.0005) [2023-12-27 11:17:57,710][20381] Updated weights for policy 1, policy_version 818465 (0.0006) [2023-12-27 11:17:57,768][20381] Updated weights for policy 1, policy_version 818475 (0.0007) [2023-12-27 11:17:57,819][20381] Updated weights for policy 1, policy_version 818485 (0.0007) [2023-12-27 11:17:58,305][20347] Updated weights for policy 0, policy_version 818980 (0.0007) [2023-12-27 11:17:58,365][20291] KL-divergence is very high: 195.9780 [2023-12-27 11:17:58,382][20347] Updated weights for policy 0, policy_version 818990 (0.0010) [2023-12-27 11:17:58,388][20291] KL-divergence is very high: 186.1928 [2023-12-27 11:17:58,444][20347] Updated weights for policy 0, policy_version 819000 (0.0008) [2023-12-27 11:17:58,569][20381] Updated weights for policy 1, policy_version 818495 (0.0007) [2023-12-27 11:17:58,635][20381] Updated weights for policy 1, policy_version 818505 (0.0008) [2023-12-27 11:17:58,699][20381] Updated weights for policy 1, policy_version 818515 (0.0008) [2023-12-27 11:17:59,183][20347] Updated weights for policy 0, policy_version 819010 (0.0008) [2023-12-27 11:17:59,197][20291] KL-divergence is very high: 254.7790 [2023-12-27 11:17:59,256][20291] KL-divergence is very high: 422.8864 [2023-12-27 11:17:59,267][20347] Updated weights for policy 0, policy_version 819022 (0.0008) [2023-12-27 11:17:59,322][20291] KL-divergence is very high: 151.0552 [2023-12-27 11:17:59,327][20347] Updated weights for policy 0, policy_version 819032 (0.0007) [2023-12-27 11:17:59,352][20291] KL-divergence is very high: 136.9028 [2023-12-27 11:17:59,463][20381] Updated weights for policy 1, policy_version 818525 (0.0008) [2023-12-27 11:17:59,517][20381] Updated weights for policy 1, policy_version 818535 (0.0007) [2023-12-27 11:17:59,575][20381] Updated weights for policy 1, policy_version 818545 (0.0007) [2023-12-27 11:18:00,056][20347] Updated weights for policy 0, policy_version 819042 (0.0010) [2023-12-27 11:18:00,117][20347] Updated weights for policy 0, policy_version 819052 (0.0011) [2023-12-27 11:18:00,133][20291] KL-divergence is very high: 103.6402 [2023-12-27 11:18:00,176][20347] Updated weights for policy 0, policy_version 819062 (0.0010) [2023-12-27 11:18:00,178][20381] Updated weights for policy 1, policy_version 818555 (0.0006) [2023-12-27 11:18:00,237][20347] Updated weights for policy 0, policy_version 819072 (0.0010) [2023-12-27 11:18:00,237][20381] Updated weights for policy 1, policy_version 818565 (0.0007) [2023-12-27 11:18:00,297][20381] Updated weights for policy 1, policy_version 818575 (0.0008) [2023-12-27 11:18:00,881][20347] Updated weights for policy 0, policy_version 819082 (0.0006) [2023-12-27 11:18:00,931][20347] Updated weights for policy 0, policy_version 819092 (0.0005) [2023-12-27 11:18:00,952][20381] Updated weights for policy 1, policy_version 818585 (0.0007) [2023-12-27 11:18:00,990][20347] Updated weights for policy 0, policy_version 819102 (0.0005) [2023-12-27 11:18:01,007][20381] Updated weights for policy 1, policy_version 818595 (0.0005) [2023-12-27 11:18:01,063][20381] Updated weights for policy 1, policy_version 818605 (0.0010) [2023-12-27 11:18:01,121][20381] Updated weights for policy 1, policy_version 818615 (0.0010) [2023-12-27 11:18:01,650][20347] Updated weights for policy 0, policy_version 819112 (0.0008) [2023-12-27 11:18:01,729][20347] Updated weights for policy 0, policy_version 819122 (0.0008) [2023-12-27 11:18:01,790][20347] Updated weights for policy 0, policy_version 819132 (0.0007) [2023-12-27 11:18:01,795][20291] KL-divergence is very high: 108.0564 [2023-12-27 11:18:01,812][20381] Updated weights for policy 1, policy_version 818625 (0.0006) [2023-12-27 11:18:01,865][20381] Updated weights for policy 1, policy_version 818635 (0.0008) [2023-12-27 11:18:01,920][20381] Updated weights for policy 1, policy_version 818645 (0.0008) [2023-12-27 11:18:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 419323904. Throughput: 0: 9782.0, 1: 9832.1. Samples: 419336144. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:02,363][19170] Avg episode reward: [(0, '854.090'), (1, '1000.000')] [2023-12-27 11:18:02,451][20347] Updated weights for policy 0, policy_version 819142 (0.0008) [2023-12-27 11:18:02,517][20347] Updated weights for policy 0, policy_version 819152 (0.0008) [2023-12-27 11:18:02,579][20347] Updated weights for policy 0, policy_version 819162 (0.0010) [2023-12-27 11:18:02,615][20381] Updated weights for policy 1, policy_version 818655 (0.0008) [2023-12-27 11:18:02,677][20381] Updated weights for policy 1, policy_version 818665 (0.0008) [2023-12-27 11:18:02,737][20381] Updated weights for policy 1, policy_version 818675 (0.0007) [2023-12-27 11:18:03,183][20347] Updated weights for policy 0, policy_version 819172 (0.0010) [2023-12-27 11:18:03,246][20347] Updated weights for policy 0, policy_version 819182 (0.0010) [2023-12-27 11:18:03,303][20347] Updated weights for policy 0, policy_version 819192 (0.0010) [2023-12-27 11:18:03,392][20381] Updated weights for policy 1, policy_version 818685 (0.0006) [2023-12-27 11:18:03,451][20381] Updated weights for policy 1, policy_version 818695 (0.0007) [2023-12-27 11:18:03,506][20381] Updated weights for policy 1, policy_version 818705 (0.0008) [2023-12-27 11:18:03,874][20347] Updated weights for policy 0, policy_version 819202 (0.0010) [2023-12-27 11:18:03,934][20347] Updated weights for policy 0, policy_version 819212 (0.0005) [2023-12-27 11:18:03,992][20347] Updated weights for policy 0, policy_version 819222 (0.0008) [2023-12-27 11:18:04,044][20347] Updated weights for policy 0, policy_version 819232 (0.0010) [2023-12-27 11:18:04,281][20381] Updated weights for policy 1, policy_version 818715 (0.0008) [2023-12-27 11:18:04,339][20381] Updated weights for policy 1, policy_version 818725 (0.0008) [2023-12-27 11:18:04,399][20381] Updated weights for policy 1, policy_version 818735 (0.0008) [2023-12-27 11:18:04,722][20347] Updated weights for policy 0, policy_version 819242 (0.0006) [2023-12-27 11:18:04,773][20347] Updated weights for policy 0, policy_version 819252 (0.0008) [2023-12-27 11:18:04,833][20347] Updated weights for policy 0, policy_version 819262 (0.0008) [2023-12-27 11:18:05,091][20381] Updated weights for policy 1, policy_version 818745 (0.0008) [2023-12-27 11:18:05,146][20381] Updated weights for policy 1, policy_version 818755 (0.0008) [2023-12-27 11:18:05,201][20381] Updated weights for policy 1, policy_version 818765 (0.0008) [2023-12-27 11:18:05,253][20381] Updated weights for policy 1, policy_version 818775 (0.0007) [2023-12-27 11:18:05,449][20347] Updated weights for policy 0, policy_version 819272 (0.0009) [2023-12-27 11:18:05,506][20347] Updated weights for policy 0, policy_version 819282 (0.0010) [2023-12-27 11:18:05,558][20347] Updated weights for policy 0, policy_version 819292 (0.0010) [2023-12-27 11:18:05,965][20381] Updated weights for policy 1, policy_version 818785 (0.0009) [2023-12-27 11:18:06,024][20381] Updated weights for policy 1, policy_version 818795 (0.0008) [2023-12-27 11:18:06,086][20381] Updated weights for policy 1, policy_version 818805 (0.0008) [2023-12-27 11:18:06,254][20347] Updated weights for policy 0, policy_version 819302 (0.0009) [2023-12-27 11:18:06,322][20347] Updated weights for policy 0, policy_version 819312 (0.0008) [2023-12-27 11:18:06,391][20347] Updated weights for policy 0, policy_version 819322 (0.0009) [2023-12-27 11:18:06,826][20381] Updated weights for policy 1, policy_version 818815 (0.0008) [2023-12-27 11:18:06,886][20381] Updated weights for policy 1, policy_version 818825 (0.0007) [2023-12-27 11:18:06,946][20381] Updated weights for policy 1, policy_version 818835 (0.0007) [2023-12-27 11:18:07,065][20347] Updated weights for policy 0, policy_version 819332 (0.0008) [2023-12-27 11:18:07,124][20347] Updated weights for policy 0, policy_version 819342 (0.0007) [2023-12-27 11:18:07,192][20347] Updated weights for policy 0, policy_version 819352 (0.0007) [2023-12-27 11:18:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 419430400. Throughput: 0: 9867.6, 1: 9843.7. Samples: 419398480. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:07,363][19170] Avg episode reward: [(0, '863.360'), (1, '1000.000')] [2023-12-27 11:18:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000818840_209649664.pth... [2023-12-27 11:18:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000819360_209780736.pth... [2023-12-27 11:18:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000818208_209485824.pth [2023-12-27 11:18:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000817688_209354752.pth [2023-12-27 11:18:07,687][20381] Updated weights for policy 1, policy_version 818845 (0.0010) [2023-12-27 11:18:07,743][20381] Updated weights for policy 1, policy_version 818855 (0.0011) [2023-12-27 11:18:07,806][20381] Updated weights for policy 1, policy_version 818865 (0.0011) [2023-12-27 11:18:07,884][20347] Updated weights for policy 0, policy_version 819362 (0.0006) [2023-12-27 11:18:07,942][20347] Updated weights for policy 0, policy_version 819372 (0.0008) [2023-12-27 11:18:08,006][20347] Updated weights for policy 0, policy_version 819382 (0.0007) [2023-12-27 11:18:08,062][20347] Updated weights for policy 0, policy_version 819392 (0.0007) [2023-12-27 11:18:08,533][20381] Updated weights for policy 1, policy_version 818875 (0.0008) [2023-12-27 11:18:08,596][20381] Updated weights for policy 1, policy_version 818885 (0.0008) [2023-12-27 11:18:08,662][20381] Updated weights for policy 1, policy_version 818895 (0.0008) [2023-12-27 11:18:08,858][20347] Updated weights for policy 0, policy_version 819402 (0.0008) [2023-12-27 11:18:08,908][20347] Updated weights for policy 0, policy_version 819412 (0.0008) [2023-12-27 11:18:08,968][20347] Updated weights for policy 0, policy_version 819422 (0.0008) [2023-12-27 11:18:09,446][20381] Updated weights for policy 1, policy_version 818905 (0.0007) [2023-12-27 11:18:09,517][20381] Updated weights for policy 1, policy_version 818915 (0.0008) [2023-12-27 11:18:09,587][20381] Updated weights for policy 1, policy_version 818925 (0.0008) [2023-12-27 11:18:09,648][20381] Updated weights for policy 1, policy_version 818935 (0.0007) [2023-12-27 11:18:09,730][20347] Updated weights for policy 0, policy_version 819432 (0.0007) [2023-12-27 11:18:09,795][20347] Updated weights for policy 0, policy_version 819442 (0.0007) [2023-12-27 11:18:09,862][20347] Updated weights for policy 0, policy_version 819452 (0.0007) [2023-12-27 11:18:10,366][20381] Updated weights for policy 1, policy_version 818945 (0.0009) [2023-12-27 11:18:10,429][20381] Updated weights for policy 1, policy_version 818955 (0.0009) [2023-12-27 11:18:10,495][20381] Updated weights for policy 1, policy_version 818965 (0.0008) [2023-12-27 11:18:10,583][20347] Updated weights for policy 0, policy_version 819462 (0.0009) [2023-12-27 11:18:10,643][20347] Updated weights for policy 0, policy_version 819472 (0.0007) [2023-12-27 11:18:10,701][20347] Updated weights for policy 0, policy_version 819482 (0.0007) [2023-12-27 11:18:11,205][20381] Updated weights for policy 1, policy_version 818975 (0.0007) [2023-12-27 11:18:11,273][20381] Updated weights for policy 1, policy_version 818985 (0.0008) [2023-12-27 11:18:11,339][20381] Updated weights for policy 1, policy_version 818995 (0.0008) [2023-12-27 11:18:11,501][20347] Updated weights for policy 0, policy_version 819492 (0.0008) [2023-12-27 11:18:11,563][20347] Updated weights for policy 0, policy_version 819502 (0.0008) [2023-12-27 11:18:11,623][20347] Updated weights for policy 0, policy_version 819512 (0.0008) [2023-12-27 11:18:12,107][20381] Updated weights for policy 1, policy_version 819005 (0.0009) [2023-12-27 11:18:12,170][20381] Updated weights for policy 1, policy_version 819015 (0.0008) [2023-12-27 11:18:12,230][20381] Updated weights for policy 1, policy_version 819025 (0.0009) [2023-12-27 11:18:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 419520512. Throughput: 0: 9870.7, 1: 9836.5. Samples: 419512172. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:12,363][19170] Avg episode reward: [(0, '872.770'), (1, '1000.000')] [2023-12-27 11:18:12,379][20347] Updated weights for policy 0, policy_version 819522 (0.0008) [2023-12-27 11:18:12,447][20347] Updated weights for policy 0, policy_version 819532 (0.0007) [2023-12-27 11:18:12,510][20347] Updated weights for policy 0, policy_version 819542 (0.0008) [2023-12-27 11:18:12,573][20347] Updated weights for policy 0, policy_version 819552 (0.0006) [2023-12-27 11:18:13,025][20381] Updated weights for policy 1, policy_version 819035 (0.0009) [2023-12-27 11:18:13,076][20381] Updated weights for policy 1, policy_version 819045 (0.0007) [2023-12-27 11:18:13,128][20381] Updated weights for policy 1, policy_version 819055 (0.0008) [2023-12-27 11:18:13,226][20347] Updated weights for policy 0, policy_version 819562 (0.0007) [2023-12-27 11:18:13,286][20347] Updated weights for policy 0, policy_version 819572 (0.0008) [2023-12-27 11:18:13,344][20347] Updated weights for policy 0, policy_version 819582 (0.0008) [2023-12-27 11:18:13,873][20381] Updated weights for policy 1, policy_version 819065 (0.0009) [2023-12-27 11:18:13,930][20381] Updated weights for policy 1, policy_version 819075 (0.0008) [2023-12-27 11:18:13,966][20347] Updated weights for policy 0, policy_version 819592 (0.0006) [2023-12-27 11:18:13,983][20381] Updated weights for policy 1, policy_version 819085 (0.0007) [2023-12-27 11:18:14,020][20347] Updated weights for policy 0, policy_version 819602 (0.0006) [2023-12-27 11:18:14,042][20381] Updated weights for policy 1, policy_version 819095 (0.0007) [2023-12-27 11:18:14,080][20347] Updated weights for policy 0, policy_version 819612 (0.0006) [2023-12-27 11:18:14,715][20347] Updated weights for policy 0, policy_version 819622 (0.0007) [2023-12-27 11:18:14,765][20381] Updated weights for policy 1, policy_version 819105 (0.0006) [2023-12-27 11:18:14,777][20347] Updated weights for policy 0, policy_version 819632 (0.0007) [2023-12-27 11:18:14,826][20381] Updated weights for policy 1, policy_version 819115 (0.0008) [2023-12-27 11:18:14,833][20347] Updated weights for policy 0, policy_version 819642 (0.0006) [2023-12-27 11:18:14,885][20381] Updated weights for policy 1, policy_version 819125 (0.0007) [2023-12-27 11:18:15,554][20347] Updated weights for policy 0, policy_version 819652 (0.0008) [2023-12-27 11:18:15,614][20381] Updated weights for policy 1, policy_version 819135 (0.0006) [2023-12-27 11:18:15,619][20347] Updated weights for policy 0, policy_version 819662 (0.0007) [2023-12-27 11:18:15,676][20381] Updated weights for policy 1, policy_version 819145 (0.0006) [2023-12-27 11:18:15,678][20347] Updated weights for policy 0, policy_version 819672 (0.0008) [2023-12-27 11:18:15,737][20381] Updated weights for policy 1, policy_version 819155 (0.0006) [2023-12-27 11:18:16,350][20381] Updated weights for policy 1, policy_version 819165 (0.0007) [2023-12-27 11:18:16,398][20347] Updated weights for policy 0, policy_version 819682 (0.0006) [2023-12-27 11:18:16,405][20381] Updated weights for policy 1, policy_version 819175 (0.0007) [2023-12-27 11:18:16,462][20347] Updated weights for policy 0, policy_version 819692 (0.0007) [2023-12-27 11:18:16,464][20381] Updated weights for policy 1, policy_version 819185 (0.0007) [2023-12-27 11:18:16,528][20347] Updated weights for policy 0, policy_version 819702 (0.0008) [2023-12-27 11:18:16,592][20347] Updated weights for policy 0, policy_version 819712 (0.0006) [2023-12-27 11:18:17,171][20381] Updated weights for policy 1, policy_version 819195 (0.0009) [2023-12-27 11:18:17,220][20381] Updated weights for policy 1, policy_version 819205 (0.0008) [2023-12-27 11:18:17,254][20347] Updated weights for policy 0, policy_version 819722 (0.0006) [2023-12-27 11:18:17,276][20381] Updated weights for policy 1, policy_version 819215 (0.0006) [2023-12-27 11:18:17,311][20347] Updated weights for policy 0, policy_version 819732 (0.0006) [2023-12-27 11:18:17,362][19170] Fps is (10 sec: 18841.9, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 419618816. Throughput: 0: 9870.2, 1: 9781.7. Samples: 419629600. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:17,363][19170] Avg episode reward: [(0, '882.750'), (1, '1000.000')] [2023-12-27 11:18:17,365][20347] Updated weights for policy 0, policy_version 819742 (0.0006) [2023-12-27 11:18:18,039][20347] Updated weights for policy 0, policy_version 819752 (0.0006) [2023-12-27 11:18:18,044][20381] Updated weights for policy 1, policy_version 819225 (0.0006) [2023-12-27 11:18:18,096][20347] Updated weights for policy 0, policy_version 819762 (0.0007) [2023-12-27 11:18:18,096][20381] Updated weights for policy 1, policy_version 819235 (0.0007) [2023-12-27 11:18:18,155][20381] Updated weights for policy 1, policy_version 819245 (0.0006) [2023-12-27 11:18:18,157][20347] Updated weights for policy 0, policy_version 819772 (0.0008) [2023-12-27 11:18:18,213][20381] Updated weights for policy 1, policy_version 819255 (0.0007) [2023-12-27 11:18:18,880][20347] Updated weights for policy 0, policy_version 819782 (0.0007) [2023-12-27 11:18:18,895][20381] Updated weights for policy 1, policy_version 819265 (0.0006) [2023-12-27 11:18:18,939][20347] Updated weights for policy 0, policy_version 819792 (0.0007) [2023-12-27 11:18:18,953][20381] Updated weights for policy 1, policy_version 819275 (0.0007) [2023-12-27 11:18:18,986][20347] Updated weights for policy 0, policy_version 819802 (0.0006) [2023-12-27 11:18:19,003][20381] Updated weights for policy 1, policy_version 819285 (0.0006) [2023-12-27 11:18:19,727][20347] Updated weights for policy 0, policy_version 819812 (0.0006) [2023-12-27 11:18:19,737][20381] Updated weights for policy 1, policy_version 819295 (0.0008) [2023-12-27 11:18:19,791][20347] Updated weights for policy 0, policy_version 819822 (0.0007) [2023-12-27 11:18:19,804][20381] Updated weights for policy 1, policy_version 819305 (0.0007) [2023-12-27 11:18:19,859][20347] Updated weights for policy 0, policy_version 819832 (0.0007) [2023-12-27 11:18:19,876][20381] Updated weights for policy 1, policy_version 819315 (0.0008) [2023-12-27 11:18:20,536][20347] Updated weights for policy 0, policy_version 819842 (0.0007) [2023-12-27 11:18:20,599][20347] Updated weights for policy 0, policy_version 819852 (0.0009) [2023-12-27 11:18:20,650][20381] Updated weights for policy 1, policy_version 819325 (0.0007) [2023-12-27 11:18:20,662][20347] Updated weights for policy 0, policy_version 819862 (0.0006) [2023-12-27 11:18:20,718][20381] Updated weights for policy 1, policy_version 819335 (0.0007) [2023-12-27 11:18:20,725][20347] Updated weights for policy 0, policy_version 819872 (0.0006) [2023-12-27 11:18:20,784][20381] Updated weights for policy 1, policy_version 819345 (0.0008) [2023-12-27 11:18:21,435][20347] Updated weights for policy 0, policy_version 819882 (0.0007) [2023-12-27 11:18:21,493][20347] Updated weights for policy 0, policy_version 819892 (0.0005) [2023-12-27 11:18:21,541][20381] Updated weights for policy 1, policy_version 819355 (0.0007) [2023-12-27 11:18:21,556][20347] Updated weights for policy 0, policy_version 819902 (0.0007) [2023-12-27 11:18:21,612][20381] Updated weights for policy 1, policy_version 819365 (0.0007) [2023-12-27 11:18:21,675][20381] Updated weights for policy 1, policy_version 819375 (0.0009) [2023-12-27 11:18:22,269][20347] Updated weights for policy 0, policy_version 819912 (0.0008) [2023-12-27 11:18:22,328][20347] Updated weights for policy 0, policy_version 819922 (0.0007) [2023-12-27 11:18:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 419708928. Throughput: 0: 9845.5, 1: 9710.6. Samples: 419686488. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:22,363][19170] Avg episode reward: [(0, '863.110'), (1, '1000.000')] [2023-12-27 11:18:22,374][20381] Updated weights for policy 1, policy_version 819385 (0.0009) [2023-12-27 11:18:22,393][20347] Updated weights for policy 0, policy_version 819932 (0.0008) [2023-12-27 11:18:22,418][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000819936_209928192.pth... [2023-12-27 11:18:22,421][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000818752_209625088.pth [2023-12-27 11:18:22,439][20381] Updated weights for policy 1, policy_version 819395 (0.0007) [2023-12-27 11:18:22,503][20381] Updated weights for policy 1, policy_version 819405 (0.0007) [2023-12-27 11:18:22,567][20381] Updated weights for policy 1, policy_version 819415 (0.0006) [2023-12-27 11:18:22,572][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000819416_209797120.pth... [2023-12-27 11:18:22,576][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000818264_209502208.pth [2023-12-27 11:18:23,161][20347] Updated weights for policy 0, policy_version 819942 (0.0009) [2023-12-27 11:18:23,218][20347] Updated weights for policy 0, policy_version 819952 (0.0008) [2023-12-27 11:18:23,276][20347] Updated weights for policy 0, policy_version 819962 (0.0007) [2023-12-27 11:18:23,286][20381] Updated weights for policy 1, policy_version 819425 (0.0007) [2023-12-27 11:18:23,343][20381] Updated weights for policy 1, policy_version 819435 (0.0006) [2023-12-27 11:18:23,396][20381] Updated weights for policy 1, policy_version 819445 (0.0006) [2023-12-27 11:18:23,999][20347] Updated weights for policy 0, policy_version 819972 (0.0007) [2023-12-27 11:18:24,060][20347] Updated weights for policy 0, policy_version 819982 (0.0007) [2023-12-27 11:18:24,119][20347] Updated weights for policy 0, policy_version 819992 (0.0007) [2023-12-27 11:18:24,128][20381] Updated weights for policy 1, policy_version 819455 (0.0008) [2023-12-27 11:18:24,186][20381] Updated weights for policy 1, policy_version 819465 (0.0008) [2023-12-27 11:18:24,249][20381] Updated weights for policy 1, policy_version 819475 (0.0008) [2023-12-27 11:18:24,778][20347] Updated weights for policy 0, policy_version 820002 (0.0006) [2023-12-27 11:18:24,830][20347] Updated weights for policy 0, policy_version 820012 (0.0009) [2023-12-27 11:18:24,884][20347] Updated weights for policy 0, policy_version 820022 (0.0009) [2023-12-27 11:18:24,930][20381] Updated weights for policy 1, policy_version 819485 (0.0007) [2023-12-27 11:18:24,953][20347] Updated weights for policy 0, policy_version 820032 (0.0010) [2023-12-27 11:18:24,987][20381] Updated weights for policy 1, policy_version 819495 (0.0006) [2023-12-27 11:18:25,045][20381] Updated weights for policy 1, policy_version 819505 (0.0007) [2023-12-27 11:18:25,656][20347] Updated weights for policy 0, policy_version 820042 (0.0007) [2023-12-27 11:18:25,694][20381] Updated weights for policy 1, policy_version 819515 (0.0006) [2023-12-27 11:18:25,717][20347] Updated weights for policy 0, policy_version 820052 (0.0007) [2023-12-27 11:18:25,755][20381] Updated weights for policy 1, policy_version 819525 (0.0007) [2023-12-27 11:18:25,783][20347] Updated weights for policy 0, policy_version 820062 (0.0009) [2023-12-27 11:18:25,819][20381] Updated weights for policy 1, policy_version 819535 (0.0007) [2023-12-27 11:18:26,506][20347] Updated weights for policy 0, policy_version 820072 (0.0009) [2023-12-27 11:18:26,539][20381] Updated weights for policy 1, policy_version 819545 (0.0007) [2023-12-27 11:18:26,567][20347] Updated weights for policy 0, policy_version 820082 (0.0007) [2023-12-27 11:18:26,596][20381] Updated weights for policy 1, policy_version 819555 (0.0006) [2023-12-27 11:18:26,629][20347] Updated weights for policy 0, policy_version 820092 (0.0011) [2023-12-27 11:18:26,650][20381] Updated weights for policy 1, policy_version 819565 (0.0006) [2023-12-27 11:18:26,702][20381] Updated weights for policy 1, policy_version 819575 (0.0006) [2023-12-27 11:18:27,254][20347] Updated weights for policy 0, policy_version 820102 (0.0010) [2023-12-27 11:18:27,310][20347] Updated weights for policy 0, policy_version 820112 (0.0010) [2023-12-27 11:18:27,360][20347] Updated weights for policy 0, policy_version 820122 (0.0010) [2023-12-27 11:18:27,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 419807232. Throughput: 0: 9849.4, 1: 9706.1. Samples: 419803468. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:27,363][19170] Avg episode reward: [(0, '864.130'), (1, '1000.000')] [2023-12-27 11:18:27,415][20381] Updated weights for policy 1, policy_version 819585 (0.0008) [2023-12-27 11:18:27,461][20381] Updated weights for policy 1, policy_version 819595 (0.0008) [2023-12-27 11:18:27,512][20381] Updated weights for policy 1, policy_version 819605 (0.0006) [2023-12-27 11:18:28,060][20347] Updated weights for policy 0, policy_version 820132 (0.0010) [2023-12-27 11:18:28,116][20347] Updated weights for policy 0, policy_version 820142 (0.0010) [2023-12-27 11:18:28,175][20381] Updated weights for policy 1, policy_version 819615 (0.0006) [2023-12-27 11:18:28,177][20347] Updated weights for policy 0, policy_version 820152 (0.0011) [2023-12-27 11:18:28,231][20381] Updated weights for policy 1, policy_version 819625 (0.0007) [2023-12-27 11:18:28,281][20381] Updated weights for policy 1, policy_version 819635 (0.0007) [2023-12-27 11:18:28,848][20347] Updated weights for policy 0, policy_version 820162 (0.0009) [2023-12-27 11:18:28,904][20347] Updated weights for policy 0, policy_version 820172 (0.0010) [2023-12-27 11:18:28,963][20347] Updated weights for policy 0, policy_version 820182 (0.0010) [2023-12-27 11:18:28,998][20381] Updated weights for policy 1, policy_version 819645 (0.0009) [2023-12-27 11:18:29,023][20347] Updated weights for policy 0, policy_version 820192 (0.0010) [2023-12-27 11:18:29,052][20381] Updated weights for policy 1, policy_version 819655 (0.0007) [2023-12-27 11:18:29,110][20381] Updated weights for policy 1, policy_version 819665 (0.0006) [2023-12-27 11:18:29,750][20347] Updated weights for policy 0, policy_version 820202 (0.0007) [2023-12-27 11:18:29,812][20347] Updated weights for policy 0, policy_version 820212 (0.0011) [2023-12-27 11:18:29,814][20381] Updated weights for policy 1, policy_version 819675 (0.0007) [2023-12-27 11:18:29,880][20381] Updated weights for policy 1, policy_version 819685 (0.0009) [2023-12-27 11:18:29,890][20347] Updated weights for policy 0, policy_version 820222 (0.0007) [2023-12-27 11:18:29,948][20381] Updated weights for policy 1, policy_version 819695 (0.0009) [2023-12-27 11:18:30,579][20347] Updated weights for policy 0, policy_version 820232 (0.0007) [2023-12-27 11:18:30,627][20347] Updated weights for policy 0, policy_version 820242 (0.0008) [2023-12-27 11:18:30,650][20381] Updated weights for policy 1, policy_version 819705 (0.0009) [2023-12-27 11:18:30,682][20347] Updated weights for policy 0, policy_version 820252 (0.0008) [2023-12-27 11:18:30,706][20381] Updated weights for policy 1, policy_version 819715 (0.0006) [2023-12-27 11:18:30,760][20381] Updated weights for policy 1, policy_version 819725 (0.0008) [2023-12-27 11:18:30,820][20381] Updated weights for policy 1, policy_version 819735 (0.0007) [2023-12-27 11:18:31,408][20347] Updated weights for policy 0, policy_version 820262 (0.0007) [2023-12-27 11:18:31,459][20347] Updated weights for policy 0, policy_version 820272 (0.0008) [2023-12-27 11:18:31,506][20347] Updated weights for policy 0, policy_version 820282 (0.0008) [2023-12-27 11:18:31,553][20381] Updated weights for policy 1, policy_version 819745 (0.0006) [2023-12-27 11:18:31,614][20381] Updated weights for policy 1, policy_version 819755 (0.0007) [2023-12-27 11:18:31,674][20381] Updated weights for policy 1, policy_version 819765 (0.0008) [2023-12-27 11:18:32,254][20347] Updated weights for policy 0, policy_version 820292 (0.0008) [2023-12-27 11:18:32,310][20347] Updated weights for policy 0, policy_version 820302 (0.0008) [2023-12-27 11:18:32,333][20381] Updated weights for policy 1, policy_version 819775 (0.0006) [2023-12-27 11:18:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 419905536. Throughput: 0: 9911.5, 1: 9688.8. Samples: 419922360. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:32,363][19170] Avg episode reward: [(0, '864.270'), (1, '1000.000')] [2023-12-27 11:18:32,369][20347] Updated weights for policy 0, policy_version 820312 (0.0008) [2023-12-27 11:18:32,402][20381] Updated weights for policy 1, policy_version 819785 (0.0009) [2023-12-27 11:18:32,458][20381] Updated weights for policy 1, policy_version 819795 (0.0008) [2023-12-27 11:18:33,063][20347] Updated weights for policy 0, policy_version 820322 (0.0008) [2023-12-27 11:18:33,123][20347] Updated weights for policy 0, policy_version 820332 (0.0007) [2023-12-27 11:18:33,156][20381] Updated weights for policy 1, policy_version 819805 (0.0007) [2023-12-27 11:18:33,174][20347] Updated weights for policy 0, policy_version 820342 (0.0008) [2023-12-27 11:18:33,212][20381] Updated weights for policy 1, policy_version 819815 (0.0006) [2023-12-27 11:18:33,233][20347] Updated weights for policy 0, policy_version 820352 (0.0007) [2023-12-27 11:18:33,261][20381] Updated weights for policy 1, policy_version 819825 (0.0007) [2023-12-27 11:18:33,953][20347] Updated weights for policy 0, policy_version 820362 (0.0007) [2023-12-27 11:18:34,007][20381] Updated weights for policy 1, policy_version 819835 (0.0008) [2023-12-27 11:18:34,008][20347] Updated weights for policy 0, policy_version 820372 (0.0006) [2023-12-27 11:18:34,055][20347] Updated weights for policy 0, policy_version 820382 (0.0006) [2023-12-27 11:18:34,058][20381] Updated weights for policy 1, policy_version 819845 (0.0008) [2023-12-27 11:18:34,114][20381] Updated weights for policy 1, policy_version 819855 (0.0008) [2023-12-27 11:18:34,759][20347] Updated weights for policy 0, policy_version 820392 (0.0009) [2023-12-27 11:18:34,816][20347] Updated weights for policy 0, policy_version 820402 (0.0009) [2023-12-27 11:18:34,867][20347] Updated weights for policy 0, policy_version 820412 (0.0010) [2023-12-27 11:18:34,892][20381] Updated weights for policy 1, policy_version 819865 (0.0008) [2023-12-27 11:18:34,948][20381] Updated weights for policy 1, policy_version 819875 (0.0007) [2023-12-27 11:18:35,005][20381] Updated weights for policy 1, policy_version 819885 (0.0007) [2023-12-27 11:18:35,057][20381] Updated weights for policy 1, policy_version 819895 (0.0008) [2023-12-27 11:18:35,598][20347] Updated weights for policy 0, policy_version 820422 (0.0010) [2023-12-27 11:18:35,661][20347] Updated weights for policy 0, policy_version 820432 (0.0008) [2023-12-27 11:18:35,721][20347] Updated weights for policy 0, policy_version 820442 (0.0008) [2023-12-27 11:18:35,763][20381] Updated weights for policy 1, policy_version 819905 (0.0007) [2023-12-27 11:18:35,812][20381] Updated weights for policy 1, policy_version 819915 (0.0008) [2023-12-27 11:18:35,871][20381] Updated weights for policy 1, policy_version 819925 (0.0007) [2023-12-27 11:18:36,430][20347] Updated weights for policy 0, policy_version 820452 (0.0010) [2023-12-27 11:18:36,491][20347] Updated weights for policy 0, policy_version 820462 (0.0011) [2023-12-27 11:18:36,555][20347] Updated weights for policy 0, policy_version 820472 (0.0010) [2023-12-27 11:18:36,645][20381] Updated weights for policy 1, policy_version 819935 (0.0008) [2023-12-27 11:18:36,717][20381] Updated weights for policy 1, policy_version 819945 (0.0008) [2023-12-27 11:18:36,774][20381] Updated weights for policy 1, policy_version 819955 (0.0007) [2023-12-27 11:18:37,264][20347] Updated weights for policy 0, policy_version 820482 (0.0011) [2023-12-27 11:18:37,322][20347] Updated weights for policy 0, policy_version 820492 (0.0011) [2023-12-27 11:18:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 420003840. Throughput: 0: 9927.0, 1: 9707.6. Samples: 419980680. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:37,363][19170] Avg episode reward: [(0, '873.670'), (1, '1000.000')] [2023-12-27 11:18:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000819960_209936384.pth... [2023-12-27 11:18:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000818840_209649664.pth [2023-12-27 11:18:37,387][20347] Updated weights for policy 0, policy_version 820502 (0.0010) [2023-12-27 11:18:37,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000820512_210075648.pth... [2023-12-27 11:18:37,448][20347] Updated weights for policy 0, policy_version 820512 (0.0011) [2023-12-27 11:18:37,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000819360_209780736.pth [2023-12-27 11:18:37,501][20381] Updated weights for policy 1, policy_version 819965 (0.0008) [2023-12-27 11:18:37,562][20381] Updated weights for policy 1, policy_version 819975 (0.0008) [2023-12-27 11:18:37,625][20381] Updated weights for policy 1, policy_version 819985 (0.0007) [2023-12-27 11:18:38,121][20291] KL-divergence is very high: 108.4088 [2023-12-27 11:18:38,147][20291] KL-divergence is very high: 273.4735 [2023-12-27 11:18:38,164][20291] KL-divergence is very high: 336.4380 [2023-12-27 11:18:38,169][20347] Updated weights for policy 0, policy_version 820522 (0.0011) [2023-12-27 11:18:38,170][20291] KL-divergence is very high: 506.9058 [2023-12-27 11:18:38,229][20347] Updated weights for policy 0, policy_version 820532 (0.0011) [2023-12-27 11:18:38,286][20291] KL-divergence is very high: 139.5712 [2023-12-27 11:18:38,286][20347] Updated weights for policy 0, policy_version 820542 (0.0010) [2023-12-27 11:18:38,383][20381] Updated weights for policy 1, policy_version 819995 (0.0008) [2023-12-27 11:18:38,437][20381] Updated weights for policy 1, policy_version 820005 (0.0008) [2023-12-27 11:18:38,501][20381] Updated weights for policy 1, policy_version 820015 (0.0007) [2023-12-27 11:18:39,008][20347] Updated weights for policy 0, policy_version 820552 (0.0010) [2023-12-27 11:18:39,068][20347] Updated weights for policy 0, policy_version 820562 (0.0011) [2023-12-27 11:18:39,128][20347] Updated weights for policy 0, policy_version 820572 (0.0010) [2023-12-27 11:18:39,247][20381] Updated weights for policy 1, policy_version 820025 (0.0008) [2023-12-27 11:18:39,311][20381] Updated weights for policy 1, policy_version 820035 (0.0008) [2023-12-27 11:18:39,382][20381] Updated weights for policy 1, policy_version 820045 (0.0009) [2023-12-27 11:18:39,450][20381] Updated weights for policy 1, policy_version 820055 (0.0008) [2023-12-27 11:18:39,918][20347] Updated weights for policy 0, policy_version 820582 (0.0010) [2023-12-27 11:18:39,989][20347] Updated weights for policy 0, policy_version 820592 (0.0010) [2023-12-27 11:18:40,050][20347] Updated weights for policy 0, policy_version 820602 (0.0011) [2023-12-27 11:18:40,267][20381] Updated weights for policy 1, policy_version 820065 (0.0008) [2023-12-27 11:18:40,323][20381] Updated weights for policy 1, policy_version 820075 (0.0008) [2023-12-27 11:18:40,387][20381] Updated weights for policy 1, policy_version 820085 (0.0008) [2023-12-27 11:18:40,799][20347] Updated weights for policy 0, policy_version 820612 (0.0010) [2023-12-27 11:18:40,853][20347] Updated weights for policy 0, policy_version 820622 (0.0010) [2023-12-27 11:18:40,908][20347] Updated weights for policy 0, policy_version 820632 (0.0007) [2023-12-27 11:18:41,215][20381] Updated weights for policy 1, policy_version 820095 (0.0008) [2023-12-27 11:18:41,278][20381] Updated weights for policy 1, policy_version 820105 (0.0008) [2023-12-27 11:18:41,338][20381] Updated weights for policy 1, policy_version 820115 (0.0007) [2023-12-27 11:18:41,625][20347] Updated weights for policy 0, policy_version 820642 (0.0008) [2023-12-27 11:18:41,690][20347] Updated weights for policy 0, policy_version 820652 (0.0008) [2023-12-27 11:18:41,761][20347] Updated weights for policy 0, policy_version 820662 (0.0007) [2023-12-27 11:18:41,819][20347] Updated weights for policy 0, policy_version 820672 (0.0008) [2023-12-27 11:18:42,104][20381] Updated weights for policy 1, policy_version 820125 (0.0009) [2023-12-27 11:18:42,156][20381] Updated weights for policy 1, policy_version 820135 (0.0008) [2023-12-27 11:18:42,211][20381] Updated weights for policy 1, policy_version 820145 (0.0007) [2023-12-27 11:18:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 420102144. Throughput: 0: 9915.0, 1: 9635.9. Samples: 420092540. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:42,363][19170] Avg episode reward: [(0, '883.040'), (1, '1000.000')] [2023-12-27 11:18:42,517][20347] Updated weights for policy 0, policy_version 820682 (0.0007) [2023-12-27 11:18:42,578][20347] Updated weights for policy 0, policy_version 820692 (0.0010) [2023-12-27 11:18:42,635][20347] Updated weights for policy 0, policy_version 820702 (0.0010) [2023-12-27 11:18:43,000][20381] Updated weights for policy 1, policy_version 820155 (0.0008) [2023-12-27 11:18:43,052][20381] Updated weights for policy 1, policy_version 820165 (0.0006) [2023-12-27 11:18:43,101][20381] Updated weights for policy 1, policy_version 820175 (0.0007) [2023-12-27 11:18:43,266][20347] Updated weights for policy 0, policy_version 820712 (0.0010) [2023-12-27 11:18:43,317][20347] Updated weights for policy 0, policy_version 820722 (0.0011) [2023-12-27 11:18:43,400][20347] Updated weights for policy 0, policy_version 820732 (0.0011) [2023-12-27 11:18:43,799][20381] Updated weights for policy 1, policy_version 820185 (0.0008) [2023-12-27 11:18:43,869][20381] Updated weights for policy 1, policy_version 820195 (0.0009) [2023-12-27 11:18:43,932][20381] Updated weights for policy 1, policy_version 820205 (0.0008) [2023-12-27 11:18:44,000][20381] Updated weights for policy 1, policy_version 820215 (0.0007) [2023-12-27 11:18:44,088][20347] Updated weights for policy 0, policy_version 820742 (0.0008) [2023-12-27 11:18:44,153][20347] Updated weights for policy 0, policy_version 820752 (0.0010) [2023-12-27 11:18:44,212][20347] Updated weights for policy 0, policy_version 820762 (0.0010) [2023-12-27 11:18:44,663][20381] Updated weights for policy 1, policy_version 820225 (0.0006) [2023-12-27 11:18:44,713][20381] Updated weights for policy 1, policy_version 820235 (0.0006) [2023-12-27 11:18:44,775][20381] Updated weights for policy 1, policy_version 820245 (0.0006) [2023-12-27 11:18:44,943][20347] Updated weights for policy 0, policy_version 820772 (0.0010) [2023-12-27 11:18:45,002][20347] Updated weights for policy 0, policy_version 820782 (0.0011) [2023-12-27 11:18:45,064][20347] Updated weights for policy 0, policy_version 820792 (0.0008) [2023-12-27 11:18:45,496][20381] Updated weights for policy 1, policy_version 820255 (0.0008) [2023-12-27 11:18:45,551][20381] Updated weights for policy 1, policy_version 820265 (0.0007) [2023-12-27 11:18:45,615][20381] Updated weights for policy 1, policy_version 820275 (0.0009) [2023-12-27 11:18:45,736][20347] Updated weights for policy 0, policy_version 820802 (0.0009) [2023-12-27 11:18:45,792][20347] Updated weights for policy 0, policy_version 820812 (0.0009) [2023-12-27 11:18:45,851][20347] Updated weights for policy 0, policy_version 820822 (0.0011) [2023-12-27 11:18:45,914][20347] Updated weights for policy 0, policy_version 820832 (0.0009) [2023-12-27 11:18:46,339][20381] Updated weights for policy 1, policy_version 820285 (0.0008) [2023-12-27 11:18:46,401][20381] Updated weights for policy 1, policy_version 820295 (0.0007) [2023-12-27 11:18:46,467][20381] Updated weights for policy 1, policy_version 820305 (0.0008) [2023-12-27 11:18:46,625][20347] Updated weights for policy 0, policy_version 820842 (0.0010) [2023-12-27 11:18:46,687][20347] Updated weights for policy 0, policy_version 820852 (0.0011) [2023-12-27 11:18:46,751][20347] Updated weights for policy 0, policy_version 820862 (0.0010) [2023-12-27 11:18:47,170][20381] Updated weights for policy 1, policy_version 820315 (0.0007) [2023-12-27 11:18:47,230][20381] Updated weights for policy 1, policy_version 820325 (0.0007) [2023-12-27 11:18:47,288][20381] Updated weights for policy 1, policy_version 820335 (0.0008) [2023-12-27 11:18:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 420200448. Throughput: 0: 9832.3, 1: 9568.2. Samples: 420209172. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:47,363][19170] Avg episode reward: [(0, '736.100'), (1, '1000.000')] [2023-12-27 11:18:47,461][20347] Updated weights for policy 0, policy_version 820872 (0.0011) [2023-12-27 11:18:47,517][20347] Updated weights for policy 0, policy_version 820882 (0.0010) [2023-12-27 11:18:47,577][20347] Updated weights for policy 0, policy_version 820892 (0.0010) [2023-12-27 11:18:48,001][20381] Updated weights for policy 1, policy_version 820345 (0.0008) [2023-12-27 11:18:48,061][20381] Updated weights for policy 1, policy_version 820355 (0.0007) [2023-12-27 11:18:48,123][20381] Updated weights for policy 1, policy_version 820365 (0.0008) [2023-12-27 11:18:48,185][20381] Updated weights for policy 1, policy_version 820375 (0.0007) [2023-12-27 11:18:48,271][20347] Updated weights for policy 0, policy_version 820902 (0.0010) [2023-12-27 11:18:48,328][20347] Updated weights for policy 0, policy_version 820912 (0.0010) [2023-12-27 11:18:48,391][20347] Updated weights for policy 0, policy_version 820922 (0.0009) [2023-12-27 11:18:48,899][20381] Updated weights for policy 1, policy_version 820385 (0.0007) [2023-12-27 11:18:48,952][20381] Updated weights for policy 1, policy_version 820395 (0.0007) [2023-12-27 11:18:49,005][20381] Updated weights for policy 1, policy_version 820405 (0.0008) [2023-12-27 11:18:49,103][20347] Updated weights for policy 0, policy_version 820932 (0.0010) [2023-12-27 11:18:49,166][20347] Updated weights for policy 0, policy_version 820942 (0.0011) [2023-12-27 11:18:49,230][20347] Updated weights for policy 0, policy_version 820952 (0.0010) [2023-12-27 11:18:49,778][20381] Updated weights for policy 1, policy_version 820415 (0.0007) [2023-12-27 11:18:49,846][20381] Updated weights for policy 1, policy_version 820425 (0.0008) [2023-12-27 11:18:49,905][20381] Updated weights for policy 1, policy_version 820435 (0.0007) [2023-12-27 11:18:50,056][20347] Updated weights for policy 0, policy_version 820962 (0.0010) [2023-12-27 11:18:50,115][20347] Updated weights for policy 0, policy_version 820972 (0.0011) [2023-12-27 11:18:50,168][20347] Updated weights for policy 0, policy_version 820982 (0.0010) [2023-12-27 11:18:50,233][20347] Updated weights for policy 0, policy_version 820992 (0.0011) [2023-12-27 11:18:50,632][20381] Updated weights for policy 1, policy_version 820445 (0.0008) [2023-12-27 11:18:50,691][20381] Updated weights for policy 1, policy_version 820455 (0.0007) [2023-12-27 11:18:50,748][20381] Updated weights for policy 1, policy_version 820465 (0.0008) [2023-12-27 11:18:50,983][20347] Updated weights for policy 0, policy_version 821002 (0.0011) [2023-12-27 11:18:51,051][20347] Updated weights for policy 0, policy_version 821012 (0.0011) [2023-12-27 11:18:51,115][20347] Updated weights for policy 0, policy_version 821022 (0.0011) [2023-12-27 11:18:51,519][20381] Updated weights for policy 1, policy_version 820475 (0.0008) [2023-12-27 11:18:51,582][20381] Updated weights for policy 1, policy_version 820485 (0.0009) [2023-12-27 11:18:51,649][20381] Updated weights for policy 1, policy_version 820495 (0.0008) [2023-12-27 11:18:51,888][20347] Updated weights for policy 0, policy_version 821032 (0.0009) [2023-12-27 11:18:51,951][20347] Updated weights for policy 0, policy_version 821042 (0.0008) [2023-12-27 11:18:52,015][20347] Updated weights for policy 0, policy_version 821053 (0.0008) [2023-12-27 11:18:52,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 420290560. Throughput: 0: 9749.4, 1: 9531.0. Samples: 420266096. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 11:18:52,363][19170] Avg episode reward: [(0, '648.530'), (1, '1000.000')] [2023-12-27 11:18:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000821056_210214912.pth... [2023-12-27 11:18:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000820504_210075648.pth... [2023-12-27 11:18:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000819936_209928192.pth [2023-12-27 11:18:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000819416_209797120.pth [2023-12-27 11:18:52,393][20381] Updated weights for policy 1, policy_version 820505 (0.0006) [2023-12-27 11:18:52,456][20381] Updated weights for policy 1, policy_version 820515 (0.0008) [2023-12-27 11:18:52,518][20381] Updated weights for policy 1, policy_version 820525 (0.0008) [2023-12-27 11:18:52,579][20381] Updated weights for policy 1, policy_version 820535 (0.0008) [2023-12-27 11:18:52,739][20347] Updated weights for policy 0, policy_version 821063 (0.0006) [2023-12-27 11:18:52,802][20347] Updated weights for policy 0, policy_version 821073 (0.0006) [2023-12-27 11:18:52,864][20347] Updated weights for policy 0, policy_version 821083 (0.0007) [2023-12-27 11:18:53,277][20381] Updated weights for policy 1, policy_version 820545 (0.0006) [2023-12-27 11:18:53,331][20381] Updated weights for policy 1, policy_version 820555 (0.0008) [2023-12-27 11:18:53,381][20381] Updated weights for policy 1, policy_version 820565 (0.0007) [2023-12-27 11:18:53,512][20347] Updated weights for policy 0, policy_version 821093 (0.0007) [2023-12-27 11:18:53,578][20347] Updated weights for policy 0, policy_version 821103 (0.0008) [2023-12-27 11:18:53,641][20347] Updated weights for policy 0, policy_version 821113 (0.0008) [2023-12-27 11:18:54,040][20381] Updated weights for policy 1, policy_version 820575 (0.0007) [2023-12-27 11:18:54,095][20381] Updated weights for policy 1, policy_version 820585 (0.0007) [2023-12-27 11:18:54,156][20381] Updated weights for policy 1, policy_version 820595 (0.0007) [2023-12-27 11:18:54,306][20347] Updated weights for policy 0, policy_version 821123 (0.0008) [2023-12-27 11:18:54,373][20347] Updated weights for policy 0, policy_version 821133 (0.0009) [2023-12-27 11:18:54,434][20347] Updated weights for policy 0, policy_version 821143 (0.0011) [2023-12-27 11:18:54,875][20381] Updated weights for policy 1, policy_version 820605 (0.0008) [2023-12-27 11:18:54,929][20381] Updated weights for policy 1, policy_version 820615 (0.0008) [2023-12-27 11:18:54,994][20381] Updated weights for policy 1, policy_version 820625 (0.0007) [2023-12-27 11:18:55,139][20347] Updated weights for policy 0, policy_version 821153 (0.0010) [2023-12-27 11:18:55,191][20347] Updated weights for policy 0, policy_version 821163 (0.0008) [2023-12-27 11:18:55,248][20347] Updated weights for policy 0, policy_version 821173 (0.0007) [2023-12-27 11:18:55,307][20347] Updated weights for policy 0, policy_version 821183 (0.0007) [2023-12-27 11:18:55,700][20381] Updated weights for policy 1, policy_version 820635 (0.0008) [2023-12-27 11:18:55,758][20381] Updated weights for policy 1, policy_version 820645 (0.0008) [2023-12-27 11:18:55,815][20381] Updated weights for policy 1, policy_version 820655 (0.0008) [2023-12-27 11:18:56,017][20347] Updated weights for policy 0, policy_version 821193 (0.0008) [2023-12-27 11:18:56,077][20347] Updated weights for policy 0, policy_version 821203 (0.0007) [2023-12-27 11:18:56,137][20347] Updated weights for policy 0, policy_version 821213 (0.0007) [2023-12-27 11:18:56,522][20381] Updated weights for policy 1, policy_version 820665 (0.0008) [2023-12-27 11:18:56,586][20381] Updated weights for policy 1, policy_version 820675 (0.0008) [2023-12-27 11:18:56,643][20381] Updated weights for policy 1, policy_version 820685 (0.0007) [2023-12-27 11:18:56,700][20381] Updated weights for policy 1, policy_version 820695 (0.0007) [2023-12-27 11:18:56,860][20347] Updated weights for policy 0, policy_version 821223 (0.0008) [2023-12-27 11:18:56,908][20347] Updated weights for policy 0, policy_version 821233 (0.0007) [2023-12-27 11:18:56,960][20347] Updated weights for policy 0, policy_version 821243 (0.0007) [2023-12-27 11:18:57,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 420388864. Throughput: 0: 9769.0, 1: 9563.9. Samples: 420382152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:18:57,363][19170] Avg episode reward: [(0, '620.050'), (1, '986.630')] [2023-12-27 11:18:57,402][20381] Updated weights for policy 1, policy_version 820705 (0.0008) [2023-12-27 11:18:57,462][20381] Updated weights for policy 1, policy_version 820715 (0.0008) [2023-12-27 11:18:57,520][20381] Updated weights for policy 1, policy_version 820725 (0.0007) [2023-12-27 11:18:57,634][20347] Updated weights for policy 0, policy_version 821253 (0.0007) [2023-12-27 11:18:57,652][20291] KL-divergence is very high: 101.2428 [2023-12-27 11:18:57,663][20291] KL-divergence is very high: 198.7182 [2023-12-27 11:18:57,680][20291] KL-divergence is very high: 320.6665 [2023-12-27 11:18:57,689][20347] Updated weights for policy 0, policy_version 821263 (0.0007) [2023-12-27 11:18:57,752][20347] Updated weights for policy 0, policy_version 821273 (0.0006) [2023-12-27 11:18:57,761][20291] KL-divergence is very high: 117.2491 [2023-12-27 11:18:58,221][20381] Updated weights for policy 1, policy_version 820735 (0.0008) [2023-12-27 11:18:58,284][20381] Updated weights for policy 1, policy_version 820745 (0.0008) [2023-12-27 11:18:58,354][20381] Updated weights for policy 1, policy_version 820755 (0.0008) [2023-12-27 11:18:58,404][20291] KL-divergence is very high: 117.4746 [2023-12-27 11:18:58,411][20291] KL-divergence is very high: 189.9037 [2023-12-27 11:18:58,419][20291] KL-divergence is very high: 153.3333 [2023-12-27 11:18:58,425][20347] Updated weights for policy 0, policy_version 821283 (0.0006) [2023-12-27 11:18:58,426][20291] KL-divergence is very high: 127.2321 [2023-12-27 11:18:58,433][20291] KL-divergence is very high: 137.0214 [2023-12-27 11:18:58,465][20291] KL-divergence is very high: 141.8155 [2023-12-27 11:18:58,471][20291] KL-divergence is very high: 148.6566 [2023-12-27 11:18:58,478][20291] KL-divergence is very high: 170.6997 [2023-12-27 11:18:58,485][20291] KL-divergence is very high: 148.8744 [2023-12-27 11:18:58,492][20347] Updated weights for policy 0, policy_version 821293 (0.0008) [2023-12-27 11:18:58,535][20291] KL-divergence is very high: 115.5080 [2023-12-27 11:18:58,542][20291] KL-divergence is very high: 123.6468 [2023-12-27 11:18:58,555][20347] Updated weights for policy 0, policy_version 821303 (0.0008) [2023-12-27 11:18:58,561][20291] KL-divergence is very high: 374.6944 [2023-12-27 11:18:58,568][20291] KL-divergence is very high: 332.9622 [2023-12-27 11:18:58,591][20291] KL-divergence is very high: 154.8665 [2023-12-27 11:18:59,162][20381] Updated weights for policy 1, policy_version 820765 (0.0007) [2023-12-27 11:18:59,225][20381] Updated weights for policy 1, policy_version 820775 (0.0009) [2023-12-27 11:18:59,292][20381] Updated weights for policy 1, policy_version 820785 (0.0008) [2023-12-27 11:18:59,386][20347] Updated weights for policy 0, policy_version 821313 (0.0008) [2023-12-27 11:18:59,451][20347] Updated weights for policy 0, policy_version 821323 (0.0009) [2023-12-27 11:18:59,508][20291] KL-divergence is very high: 136.8242 [2023-12-27 11:18:59,514][20291] KL-divergence is very high: 129.7977 [2023-12-27 11:18:59,515][20347] Updated weights for policy 0, policy_version 821333 (0.0009) [2023-12-27 11:18:59,520][20291] KL-divergence is very high: 111.5893 [2023-12-27 11:18:59,525][20291] KL-divergence is very high: 101.8524 [2023-12-27 11:18:59,572][20347] Updated weights for policy 0, policy_version 821343 (0.0010) [2023-12-27 11:19:00,118][20381] Updated weights for policy 1, policy_version 820795 (0.0007) [2023-12-27 11:19:00,180][20381] Updated weights for policy 1, policy_version 820805 (0.0006) [2023-12-27 11:19:00,243][20381] Updated weights for policy 1, policy_version 820815 (0.0008) [2023-12-27 11:19:00,299][20347] Updated weights for policy 0, policy_version 821353 (0.0009) [2023-12-27 11:19:00,366][20347] Updated weights for policy 0, policy_version 821363 (0.0009) [2023-12-27 11:19:00,402][20291] KL-divergence is very high: 168.0206 [2023-12-27 11:19:00,426][20347] Updated weights for policy 0, policy_version 821373 (0.0007) [2023-12-27 11:19:00,940][20381] Updated weights for policy 1, policy_version 820825 (0.0006) [2023-12-27 11:19:00,994][20381] Updated weights for policy 1, policy_version 820835 (0.0008) [2023-12-27 11:19:01,056][20381] Updated weights for policy 1, policy_version 820845 (0.0007) [2023-12-27 11:19:01,062][20347] Updated weights for policy 0, policy_version 821383 (0.0009) [2023-12-27 11:19:01,115][20347] Updated weights for policy 0, policy_version 821393 (0.0006) [2023-12-27 11:19:01,116][20381] Updated weights for policy 1, policy_version 820855 (0.0007) [2023-12-27 11:19:01,178][20347] Updated weights for policy 0, policy_version 821403 (0.0009) [2023-12-27 11:19:01,878][20381] Updated weights for policy 1, policy_version 820865 (0.0008) [2023-12-27 11:19:01,892][20347] Updated weights for policy 0, policy_version 821413 (0.0006) [2023-12-27 11:19:01,929][20381] Updated weights for policy 1, policy_version 820875 (0.0008) [2023-12-27 11:19:01,956][20347] Updated weights for policy 0, policy_version 821423 (0.0006) [2023-12-27 11:19:01,978][20381] Updated weights for policy 1, policy_version 820885 (0.0007) [2023-12-27 11:19:02,016][20347] Updated weights for policy 0, policy_version 821433 (0.0007) [2023-12-27 11:19:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 420487168. Throughput: 0: 9737.5, 1: 9544.1. Samples: 420497272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:02,363][19170] Avg episode reward: [(0, '473.800'), (1, '922.900')] [2023-12-27 11:19:02,704][20347] Updated weights for policy 0, policy_version 821443 (0.0007) [2023-12-27 11:19:02,724][20381] Updated weights for policy 1, policy_version 820895 (0.0009) [2023-12-27 11:19:02,759][20347] Updated weights for policy 0, policy_version 821453 (0.0009) [2023-12-27 11:19:02,781][20381] Updated weights for policy 1, policy_version 820905 (0.0006) [2023-12-27 11:19:02,815][20347] Updated weights for policy 0, policy_version 821463 (0.0007) [2023-12-27 11:19:02,839][20381] Updated weights for policy 1, policy_version 820915 (0.0008) [2023-12-27 11:19:03,466][20347] Updated weights for policy 0, policy_version 821473 (0.0007) [2023-12-27 11:19:03,507][20381] Updated weights for policy 1, policy_version 820925 (0.0007) [2023-12-27 11:19:03,526][20347] Updated weights for policy 0, policy_version 821483 (0.0005) [2023-12-27 11:19:03,557][20381] Updated weights for policy 1, policy_version 820935 (0.0008) [2023-12-27 11:19:03,586][20347] Updated weights for policy 0, policy_version 821493 (0.0005) [2023-12-27 11:19:03,616][20381] Updated weights for policy 1, policy_version 820945 (0.0008) [2023-12-27 11:19:03,639][20347] Updated weights for policy 0, policy_version 821503 (0.0007) [2023-12-27 11:19:04,279][20381] Updated weights for policy 1, policy_version 820955 (0.0007) [2023-12-27 11:19:04,327][20347] Updated weights for policy 0, policy_version 821513 (0.0007) [2023-12-27 11:19:04,339][20381] Updated weights for policy 1, policy_version 820965 (0.0008) [2023-12-27 11:19:04,392][20347] Updated weights for policy 0, policy_version 821523 (0.0008) [2023-12-27 11:19:04,400][20381] Updated weights for policy 1, policy_version 820975 (0.0008) [2023-12-27 11:19:04,444][20347] Updated weights for policy 0, policy_version 821533 (0.0008) [2023-12-27 11:19:05,105][20381] Updated weights for policy 1, policy_version 820985 (0.0008) [2023-12-27 11:19:05,170][20347] Updated weights for policy 0, policy_version 821543 (0.0006) [2023-12-27 11:19:05,173][20381] Updated weights for policy 1, policy_version 820995 (0.0008) [2023-12-27 11:19:05,230][20347] Updated weights for policy 0, policy_version 821553 (0.0006) [2023-12-27 11:19:05,239][20381] Updated weights for policy 1, policy_version 821005 (0.0008) [2023-12-27 11:19:05,287][20347] Updated weights for policy 0, policy_version 821563 (0.0006) [2023-12-27 11:19:05,300][20381] Updated weights for policy 1, policy_version 821015 (0.0008) [2023-12-27 11:19:05,965][20381] Updated weights for policy 1, policy_version 821025 (0.0007) [2023-12-27 11:19:05,987][20347] Updated weights for policy 0, policy_version 821573 (0.0007) [2023-12-27 11:19:06,016][20381] Updated weights for policy 1, policy_version 821035 (0.0005) [2023-12-27 11:19:06,043][20347] Updated weights for policy 0, policy_version 821583 (0.0007) [2023-12-27 11:19:06,068][20381] Updated weights for policy 1, policy_version 821045 (0.0007) [2023-12-27 11:19:06,107][20347] Updated weights for policy 0, policy_version 821593 (0.0007) [2023-12-27 11:19:06,819][20347] Updated weights for policy 0, policy_version 821603 (0.0007) [2023-12-27 11:19:06,834][20381] Updated weights for policy 1, policy_version 821055 (0.0007) [2023-12-27 11:19:06,877][20347] Updated weights for policy 0, policy_version 821613 (0.0007) [2023-12-27 11:19:06,891][20381] Updated weights for policy 1, policy_version 821065 (0.0008) [2023-12-27 11:19:06,935][20347] Updated weights for policy 0, policy_version 821623 (0.0006) [2023-12-27 11:19:06,949][20381] Updated weights for policy 1, policy_version 821075 (0.0008) [2023-12-27 11:19:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 420585472. Throughput: 0: 9768.1, 1: 9587.1. Samples: 420557472. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:07,363][19170] Avg episode reward: [(0, '551.910'), (1, '862.680')] [2023-12-27 11:19:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000821080_210223104.pth... [2023-12-27 11:19:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000821632_210362368.pth... [2023-12-27 11:19:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000820512_210075648.pth [2023-12-27 11:19:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000819960_209936384.pth [2023-12-27 11:19:07,611][20347] Updated weights for policy 0, policy_version 821633 (0.0006) [2023-12-27 11:19:07,660][20381] Updated weights for policy 1, policy_version 821085 (0.0007) [2023-12-27 11:19:07,674][20347] Updated weights for policy 0, policy_version 821643 (0.0008) [2023-12-27 11:19:07,719][20381] Updated weights for policy 1, policy_version 821095 (0.0006) [2023-12-27 11:19:07,732][20347] Updated weights for policy 0, policy_version 821653 (0.0006) [2023-12-27 11:19:07,779][20381] Updated weights for policy 1, policy_version 821105 (0.0007) [2023-12-27 11:19:07,785][20347] Updated weights for policy 0, policy_version 821663 (0.0006) [2023-12-27 11:19:08,446][20347] Updated weights for policy 0, policy_version 821673 (0.0009) [2023-12-27 11:19:08,482][20381] Updated weights for policy 1, policy_version 821115 (0.0006) [2023-12-27 11:19:08,508][20347] Updated weights for policy 0, policy_version 821683 (0.0010) [2023-12-27 11:19:08,547][20381] Updated weights for policy 1, policy_version 821125 (0.0006) [2023-12-27 11:19:08,563][20347] Updated weights for policy 0, policy_version 821693 (0.0008) [2023-12-27 11:19:08,603][20381] Updated weights for policy 1, policy_version 821135 (0.0007) [2023-12-27 11:19:09,290][20347] Updated weights for policy 0, policy_version 821703 (0.0008) [2023-12-27 11:19:09,330][20381] Updated weights for policy 1, policy_version 821145 (0.0008) [2023-12-27 11:19:09,355][20347] Updated weights for policy 0, policy_version 821713 (0.0009) [2023-12-27 11:19:09,401][20381] Updated weights for policy 1, policy_version 821155 (0.0009) [2023-12-27 11:19:09,421][20347] Updated weights for policy 0, policy_version 821723 (0.0009) [2023-12-27 11:19:09,470][20381] Updated weights for policy 1, policy_version 821165 (0.0008) [2023-12-27 11:19:09,525][20381] Updated weights for policy 1, policy_version 821175 (0.0008) [2023-12-27 11:19:10,173][20347] Updated weights for policy 0, policy_version 821733 (0.0007) [2023-12-27 11:19:10,243][20347] Updated weights for policy 0, policy_version 821743 (0.0008) [2023-12-27 11:19:10,288][20381] Updated weights for policy 1, policy_version 821185 (0.0009) [2023-12-27 11:19:10,308][20347] Updated weights for policy 0, policy_version 821753 (0.0007) [2023-12-27 11:19:10,357][20381] Updated weights for policy 1, policy_version 821195 (0.0007) [2023-12-27 11:19:10,422][20381] Updated weights for policy 1, policy_version 821205 (0.0008) [2023-12-27 11:19:11,023][20347] Updated weights for policy 0, policy_version 821763 (0.0007) [2023-12-27 11:19:11,088][20347] Updated weights for policy 0, policy_version 821773 (0.0009) [2023-12-27 11:19:11,146][20347] Updated weights for policy 0, policy_version 821783 (0.0009) [2023-12-27 11:19:11,185][20381] Updated weights for policy 1, policy_version 821215 (0.0008) [2023-12-27 11:19:11,246][20381] Updated weights for policy 1, policy_version 821225 (0.0009) [2023-12-27 11:19:11,304][20381] Updated weights for policy 1, policy_version 821235 (0.0007) [2023-12-27 11:19:11,933][20347] Updated weights for policy 0, policy_version 821793 (0.0008) [2023-12-27 11:19:11,998][20347] Updated weights for policy 0, policy_version 821803 (0.0008) [2023-12-27 11:19:12,060][20347] Updated weights for policy 0, policy_version 821813 (0.0007) [2023-12-27 11:19:12,076][20381] Updated weights for policy 1, policy_version 821245 (0.0007) [2023-12-27 11:19:12,124][20347] Updated weights for policy 0, policy_version 821823 (0.0007) [2023-12-27 11:19:12,138][20381] Updated weights for policy 1, policy_version 821255 (0.0006) [2023-12-27 11:19:12,201][20381] Updated weights for policy 1, policy_version 821265 (0.0007) [2023-12-27 11:19:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 420683776. Throughput: 0: 9759.6, 1: 9558.1. Samples: 420672768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:12,363][19170] Avg episode reward: [(0, '552.010'), (1, '843.220')] [2023-12-27 11:19:12,842][20347] Updated weights for policy 0, policy_version 821833 (0.0006) [2023-12-27 11:19:12,886][20381] Updated weights for policy 1, policy_version 821275 (0.0008) [2023-12-27 11:19:12,897][20347] Updated weights for policy 0, policy_version 821843 (0.0007) [2023-12-27 11:19:12,947][20381] Updated weights for policy 1, policy_version 821285 (0.0011) [2023-12-27 11:19:12,957][20347] Updated weights for policy 0, policy_version 821853 (0.0006) [2023-12-27 11:19:13,011][20381] Updated weights for policy 1, policy_version 821295 (0.0011) [2023-12-27 11:19:13,690][20347] Updated weights for policy 0, policy_version 821863 (0.0009) [2023-12-27 11:19:13,692][20381] Updated weights for policy 1, policy_version 821305 (0.0010) [2023-12-27 11:19:13,751][20347] Updated weights for policy 0, policy_version 821873 (0.0011) [2023-12-27 11:19:13,760][20381] Updated weights for policy 1, policy_version 821315 (0.0007) [2023-12-27 11:19:13,812][20347] Updated weights for policy 0, policy_version 821883 (0.0011) [2023-12-27 11:19:13,814][20381] Updated weights for policy 1, policy_version 821325 (0.0011) [2023-12-27 11:19:13,873][20381] Updated weights for policy 1, policy_version 821335 (0.0008) [2023-12-27 11:19:14,525][20347] Updated weights for policy 0, policy_version 821893 (0.0008) [2023-12-27 11:19:14,566][20381] Updated weights for policy 1, policy_version 821345 (0.0011) [2023-12-27 11:19:14,583][20347] Updated weights for policy 0, policy_version 821903 (0.0006) [2023-12-27 11:19:14,619][20381] Updated weights for policy 1, policy_version 821355 (0.0011) [2023-12-27 11:19:14,640][20347] Updated weights for policy 0, policy_version 821913 (0.0006) [2023-12-27 11:19:14,663][20291] KL-divergence is very high: 188.6598 [2023-12-27 11:19:14,676][20381] Updated weights for policy 1, policy_version 821365 (0.0010) [2023-12-27 11:19:15,328][20347] Updated weights for policy 0, policy_version 821923 (0.0006) [2023-12-27 11:19:15,374][20381] Updated weights for policy 1, policy_version 821375 (0.0009) [2023-12-27 11:19:15,395][20347] Updated weights for policy 0, policy_version 821933 (0.0008) [2023-12-27 11:19:15,431][20381] Updated weights for policy 1, policy_version 821385 (0.0011) [2023-12-27 11:19:15,457][20347] Updated weights for policy 0, policy_version 821943 (0.0006) [2023-12-27 11:19:15,489][20381] Updated weights for policy 1, policy_version 821395 (0.0011) [2023-12-27 11:19:16,167][20347] Updated weights for policy 0, policy_version 821953 (0.0005) [2023-12-27 11:19:16,215][20381] Updated weights for policy 1, policy_version 821405 (0.0011) [2023-12-27 11:19:16,232][20347] Updated weights for policy 0, policy_version 821963 (0.0006) [2023-12-27 11:19:16,275][20381] Updated weights for policy 1, policy_version 821415 (0.0011) [2023-12-27 11:19:16,292][20347] Updated weights for policy 0, policy_version 821973 (0.0005) [2023-12-27 11:19:16,335][20381] Updated weights for policy 1, policy_version 821425 (0.0011) [2023-12-27 11:19:16,350][20347] Updated weights for policy 0, policy_version 821983 (0.0007) [2023-12-27 11:19:16,952][20347] Updated weights for policy 0, policy_version 821993 (0.0008) [2023-12-27 11:19:17,012][20347] Updated weights for policy 0, policy_version 822003 (0.0007) [2023-12-27 11:19:17,067][20381] Updated weights for policy 1, policy_version 821435 (0.0010) [2023-12-27 11:19:17,074][20347] Updated weights for policy 0, policy_version 822013 (0.0007) [2023-12-27 11:19:17,129][20381] Updated weights for policy 1, policy_version 821445 (0.0006) [2023-12-27 11:19:17,193][20381] Updated weights for policy 1, policy_version 821455 (0.0008) [2023-12-27 11:19:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 420782080. Throughput: 0: 9735.5, 1: 9515.7. Samples: 420788660. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:17,363][19170] Avg episode reward: [(0, '601.380'), (1, '865.960')] [2023-12-27 11:19:17,734][20347] Updated weights for policy 0, policy_version 822023 (0.0007) [2023-12-27 11:19:17,793][20347] Updated weights for policy 0, policy_version 822033 (0.0007) [2023-12-27 11:19:17,843][20347] Updated weights for policy 0, policy_version 822043 (0.0008) [2023-12-27 11:19:17,881][20381] Updated weights for policy 1, policy_version 821465 (0.0009) [2023-12-27 11:19:17,942][20381] Updated weights for policy 1, policy_version 821475 (0.0007) [2023-12-27 11:19:17,994][20381] Updated weights for policy 1, policy_version 821485 (0.0008) [2023-12-27 11:19:18,055][20381] Updated weights for policy 1, policy_version 821495 (0.0008) [2023-12-27 11:19:18,526][20347] Updated weights for policy 0, policy_version 822053 (0.0007) [2023-12-27 11:19:18,586][20347] Updated weights for policy 0, policy_version 822063 (0.0008) [2023-12-27 11:19:18,597][20291] KL-divergence is very high: 104.3089 [2023-12-27 11:19:18,641][20347] Updated weights for policy 0, policy_version 822073 (0.0008) [2023-12-27 11:19:18,754][20381] Updated weights for policy 1, policy_version 821505 (0.0007) [2023-12-27 11:19:18,818][20381] Updated weights for policy 1, policy_version 821515 (0.0008) [2023-12-27 11:19:18,882][20381] Updated weights for policy 1, policy_version 821525 (0.0008) [2023-12-27 11:19:19,371][20347] Updated weights for policy 0, policy_version 822083 (0.0008) [2023-12-27 11:19:19,443][20347] Updated weights for policy 0, policy_version 822093 (0.0008) [2023-12-27 11:19:19,509][20347] Updated weights for policy 0, policy_version 822103 (0.0009) [2023-12-27 11:19:19,626][20381] Updated weights for policy 1, policy_version 821535 (0.0008) [2023-12-27 11:19:19,688][20381] Updated weights for policy 1, policy_version 821545 (0.0008) [2023-12-27 11:19:19,751][20381] Updated weights for policy 1, policy_version 821555 (0.0008) [2023-12-27 11:19:20,255][20347] Updated weights for policy 0, policy_version 822113 (0.0007) [2023-12-27 11:19:20,314][20347] Updated weights for policy 0, policy_version 822123 (0.0011) [2023-12-27 11:19:20,380][20347] Updated weights for policy 0, policy_version 822133 (0.0011) [2023-12-27 11:19:20,445][20347] Updated weights for policy 0, policy_version 822143 (0.0011) [2023-12-27 11:19:20,524][20381] Updated weights for policy 1, policy_version 821565 (0.0008) [2023-12-27 11:19:20,584][20381] Updated weights for policy 1, policy_version 821575 (0.0007) [2023-12-27 11:19:20,679][20381] Updated weights for policy 1, policy_version 821585 (0.0007) [2023-12-27 11:19:21,204][20347] Updated weights for policy 0, policy_version 822153 (0.0007) [2023-12-27 11:19:21,277][20347] Updated weights for policy 0, policy_version 822163 (0.0007) [2023-12-27 11:19:21,334][20347] Updated weights for policy 0, policy_version 822173 (0.0008) [2023-12-27 11:19:21,418][20381] Updated weights for policy 1, policy_version 821595 (0.0008) [2023-12-27 11:19:21,480][20381] Updated weights for policy 1, policy_version 821605 (0.0007) [2023-12-27 11:19:21,540][20381] Updated weights for policy 1, policy_version 821615 (0.0007) [2023-12-27 11:19:22,110][20347] Updated weights for policy 0, policy_version 822183 (0.0008) [2023-12-27 11:19:22,166][20347] Updated weights for policy 0, policy_version 822193 (0.0008) [2023-12-27 11:19:22,215][20347] Updated weights for policy 0, policy_version 822203 (0.0006) [2023-12-27 11:19:22,286][20381] Updated weights for policy 1, policy_version 821625 (0.0007) [2023-12-27 11:19:22,345][20381] Updated weights for policy 1, policy_version 821635 (0.0008) [2023-12-27 11:19:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 420872192. Throughput: 0: 9729.8, 1: 9527.1. Samples: 420847240. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:22,363][19170] Avg episode reward: [(0, '717.350'), (1, '893.530')] [2023-12-27 11:19:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000822208_210509824.pth... [2023-12-27 11:19:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000821056_210214912.pth [2023-12-27 11:19:22,413][20381] Updated weights for policy 1, policy_version 821645 (0.0008) [2023-12-27 11:19:22,479][20381] Updated weights for policy 1, policy_version 821655 (0.0008) [2023-12-27 11:19:22,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000821656_210370560.pth... [2023-12-27 11:19:22,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000820504_210075648.pth [2023-12-27 11:19:22,953][20347] Updated weights for policy 0, policy_version 822213 (0.0006) [2023-12-27 11:19:23,014][20347] Updated weights for policy 0, policy_version 822223 (0.0008) [2023-12-27 11:19:23,080][20347] Updated weights for policy 0, policy_version 822233 (0.0008) [2023-12-27 11:19:23,203][20381] Updated weights for policy 1, policy_version 821665 (0.0007) [2023-12-27 11:19:23,273][20381] Updated weights for policy 1, policy_version 821675 (0.0008) [2023-12-27 11:19:23,341][20381] Updated weights for policy 1, policy_version 821685 (0.0007) [2023-12-27 11:19:23,731][20347] Updated weights for policy 0, policy_version 822243 (0.0008) [2023-12-27 11:19:23,785][20347] Updated weights for policy 0, policy_version 822253 (0.0009) [2023-12-27 11:19:23,839][20347] Updated weights for policy 0, policy_version 822263 (0.0008) [2023-12-27 11:19:23,999][20381] Updated weights for policy 1, policy_version 821695 (0.0006) [2023-12-27 11:19:24,058][20381] Updated weights for policy 1, policy_version 821705 (0.0006) [2023-12-27 11:19:24,120][20381] Updated weights for policy 1, policy_version 821715 (0.0008) [2023-12-27 11:19:24,578][20347] Updated weights for policy 0, policy_version 822273 (0.0007) [2023-12-27 11:19:24,638][20347] Updated weights for policy 0, policy_version 822283 (0.0007) [2023-12-27 11:19:24,701][20347] Updated weights for policy 0, policy_version 822293 (0.0006) [2023-12-27 11:19:24,757][20381] Updated weights for policy 1, policy_version 821725 (0.0007) [2023-12-27 11:19:24,762][20347] Updated weights for policy 0, policy_version 822303 (0.0007) [2023-12-27 11:19:24,816][20381] Updated weights for policy 1, policy_version 821735 (0.0006) [2023-12-27 11:19:24,871][20381] Updated weights for policy 1, policy_version 821745 (0.0006) [2023-12-27 11:19:25,442][20347] Updated weights for policy 0, policy_version 822313 (0.0008) [2023-12-27 11:19:25,503][20347] Updated weights for policy 0, policy_version 822323 (0.0008) [2023-12-27 11:19:25,520][20381] Updated weights for policy 1, policy_version 821755 (0.0007) [2023-12-27 11:19:25,561][20347] Updated weights for policy 0, policy_version 822333 (0.0006) [2023-12-27 11:19:25,574][20381] Updated weights for policy 1, policy_version 821765 (0.0006) [2023-12-27 11:19:25,633][20381] Updated weights for policy 1, policy_version 821775 (0.0008) [2023-12-27 11:19:26,225][20347] Updated weights for policy 0, policy_version 822343 (0.0007) [2023-12-27 11:19:26,289][20347] Updated weights for policy 0, policy_version 822353 (0.0008) [2023-12-27 11:19:26,346][20347] Updated weights for policy 0, policy_version 822363 (0.0006) [2023-12-27 11:19:26,420][20381] Updated weights for policy 1, policy_version 821785 (0.0009) [2023-12-27 11:19:26,486][20381] Updated weights for policy 1, policy_version 821795 (0.0010) [2023-12-27 11:19:26,549][20381] Updated weights for policy 1, policy_version 821805 (0.0011) [2023-12-27 11:19:26,604][20381] Updated weights for policy 1, policy_version 821815 (0.0011) [2023-12-27 11:19:27,045][20347] Updated weights for policy 0, policy_version 822373 (0.0006) [2023-12-27 11:19:27,093][20347] Updated weights for policy 0, policy_version 822383 (0.0005) [2023-12-27 11:19:27,145][20347] Updated weights for policy 0, policy_version 822393 (0.0008) [2023-12-27 11:19:27,299][20381] Updated weights for policy 1, policy_version 821825 (0.0010) [2023-12-27 11:19:27,358][20381] Updated weights for policy 1, policy_version 821835 (0.0009) [2023-12-27 11:19:27,363][19170] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 420970496. Throughput: 0: 9745.9, 1: 9603.3. Samples: 420963256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:27,363][19170] Avg episode reward: [(0, '755.790'), (1, '870.040')] [2023-12-27 11:19:27,411][20381] Updated weights for policy 1, policy_version 821845 (0.0007) [2023-12-27 11:19:27,853][20347] Updated weights for policy 0, policy_version 822403 (0.0008) [2023-12-27 11:19:27,911][20347] Updated weights for policy 0, policy_version 822413 (0.0007) [2023-12-27 11:19:27,976][20347] Updated weights for policy 0, policy_version 822423 (0.0006) [2023-12-27 11:19:28,079][20381] Updated weights for policy 1, policy_version 821855 (0.0006) [2023-12-27 11:19:28,136][20381] Updated weights for policy 1, policy_version 821865 (0.0008) [2023-12-27 11:19:28,192][20381] Updated weights for policy 1, policy_version 821875 (0.0010) [2023-12-27 11:19:28,698][20347] Updated weights for policy 0, policy_version 822433 (0.0006) [2023-12-27 11:19:28,760][20347] Updated weights for policy 0, policy_version 822443 (0.0008) [2023-12-27 11:19:28,817][20347] Updated weights for policy 0, policy_version 822453 (0.0008) [2023-12-27 11:19:28,823][20381] Updated weights for policy 1, policy_version 821885 (0.0007) [2023-12-27 11:19:28,876][20347] Updated weights for policy 0, policy_version 822463 (0.0008) [2023-12-27 11:19:28,880][20381] Updated weights for policy 1, policy_version 821895 (0.0011) [2023-12-27 11:19:28,936][20381] Updated weights for policy 1, policy_version 821905 (0.0010) [2023-12-27 11:19:29,580][20347] Updated weights for policy 0, policy_version 822473 (0.0007) [2023-12-27 11:19:29,637][20347] Updated weights for policy 0, policy_version 822483 (0.0008) [2023-12-27 11:19:29,644][20381] Updated weights for policy 1, policy_version 821915 (0.0009) [2023-12-27 11:19:29,689][20347] Updated weights for policy 0, policy_version 822493 (0.0007) [2023-12-27 11:19:29,700][20381] Updated weights for policy 1, policy_version 821925 (0.0011) [2023-12-27 11:19:29,762][20381] Updated weights for policy 1, policy_version 821935 (0.0011) [2023-12-27 11:19:30,381][20347] Updated weights for policy 0, policy_version 822503 (0.0006) [2023-12-27 11:19:30,444][20347] Updated weights for policy 0, policy_version 822513 (0.0006) [2023-12-27 11:19:30,477][20381] Updated weights for policy 1, policy_version 821945 (0.0011) [2023-12-27 11:19:30,507][20347] Updated weights for policy 0, policy_version 822523 (0.0006) [2023-12-27 11:19:30,536][20381] Updated weights for policy 1, policy_version 821955 (0.0010) [2023-12-27 11:19:30,593][20381] Updated weights for policy 1, policy_version 821965 (0.0011) [2023-12-27 11:19:30,656][20381] Updated weights for policy 1, policy_version 821975 (0.0006) [2023-12-27 11:19:31,070][20347] Updated weights for policy 0, policy_version 822533 (0.0007) [2023-12-27 11:19:31,134][20347] Updated weights for policy 0, policy_version 822543 (0.0008) [2023-12-27 11:19:31,199][20347] Updated weights for policy 0, policy_version 822553 (0.0008) [2023-12-27 11:19:31,338][20381] Updated weights for policy 1, policy_version 821985 (0.0008) [2023-12-27 11:19:31,405][20381] Updated weights for policy 1, policy_version 821995 (0.0007) [2023-12-27 11:19:31,460][20381] Updated weights for policy 1, policy_version 822005 (0.0007) [2023-12-27 11:19:31,904][20347] Updated weights for policy 0, policy_version 822563 (0.0007) [2023-12-27 11:19:31,962][20347] Updated weights for policy 0, policy_version 822573 (0.0008) [2023-12-27 11:19:32,024][20347] Updated weights for policy 0, policy_version 822583 (0.0008) [2023-12-27 11:19:32,163][20381] Updated weights for policy 1, policy_version 822015 (0.0008) [2023-12-27 11:19:32,226][20381] Updated weights for policy 1, policy_version 822025 (0.0009) [2023-12-27 11:19:32,287][20381] Updated weights for policy 1, policy_version 822035 (0.0008) [2023-12-27 11:19:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 421076992. Throughput: 0: 9769.6, 1: 9664.4. Samples: 421083700. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:32,363][19170] Avg episode reward: [(0, '746.380'), (1, '906.790')] [2023-12-27 11:19:32,706][20347] Updated weights for policy 0, policy_version 822593 (0.0006) [2023-12-27 11:19:32,756][20347] Updated weights for policy 0, policy_version 822603 (0.0007) [2023-12-27 11:19:32,803][20347] Updated weights for policy 0, policy_version 822613 (0.0008) [2023-12-27 11:19:32,849][20347] Updated weights for policy 0, policy_version 822623 (0.0007) [2023-12-27 11:19:32,997][20381] Updated weights for policy 1, policy_version 822045 (0.0007) [2023-12-27 11:19:33,057][20381] Updated weights for policy 1, policy_version 822055 (0.0007) [2023-12-27 11:19:33,116][20381] Updated weights for policy 1, policy_version 822065 (0.0008) [2023-12-27 11:19:33,585][20347] Updated weights for policy 0, policy_version 822633 (0.0008) [2023-12-27 11:19:33,645][20347] Updated weights for policy 0, policy_version 822643 (0.0008) [2023-12-27 11:19:33,701][20347] Updated weights for policy 0, policy_version 822653 (0.0008) [2023-12-27 11:19:33,782][20381] Updated weights for policy 1, policy_version 822075 (0.0007) [2023-12-27 11:19:33,845][20381] Updated weights for policy 1, policy_version 822085 (0.0008) [2023-12-27 11:19:33,892][20381] Updated weights for policy 1, policy_version 822095 (0.0008) [2023-12-27 11:19:34,419][20347] Updated weights for policy 0, policy_version 822663 (0.0007) [2023-12-27 11:19:34,483][20347] Updated weights for policy 0, policy_version 822673 (0.0008) [2023-12-27 11:19:34,490][20291] KL-divergence is very high: 149.2573 [2023-12-27 11:19:34,497][20291] KL-divergence is very high: 150.3578 [2023-12-27 11:19:34,517][20291] KL-divergence is very high: 120.8142 [2023-12-27 11:19:34,549][20347] Updated weights for policy 0, policy_version 822683 (0.0007) [2023-12-27 11:19:34,628][20381] Updated weights for policy 1, policy_version 822105 (0.0007) [2023-12-27 11:19:34,679][20381] Updated weights for policy 1, policy_version 822115 (0.0008) [2023-12-27 11:19:34,734][20381] Updated weights for policy 1, policy_version 822125 (0.0008) [2023-12-27 11:19:34,790][20381] Updated weights for policy 1, policy_version 822135 (0.0008) [2023-12-27 11:19:35,253][20347] Updated weights for policy 0, policy_version 822693 (0.0007) [2023-12-27 11:19:35,307][20347] Updated weights for policy 0, policy_version 822703 (0.0006) [2023-12-27 11:19:35,361][20347] Updated weights for policy 0, policy_version 822713 (0.0006) [2023-12-27 11:19:35,491][20381] Updated weights for policy 1, policy_version 822145 (0.0007) [2023-12-27 11:19:35,549][20381] Updated weights for policy 1, policy_version 822155 (0.0008) [2023-12-27 11:19:35,602][20381] Updated weights for policy 1, policy_version 822165 (0.0008) [2023-12-27 11:19:36,014][20347] Updated weights for policy 0, policy_version 822723 (0.0006) [2023-12-27 11:19:36,069][20347] Updated weights for policy 0, policy_version 822733 (0.0007) [2023-12-27 11:19:36,134][20347] Updated weights for policy 0, policy_version 822743 (0.0008) [2023-12-27 11:19:36,369][20381] Updated weights for policy 1, policy_version 822175 (0.0008) [2023-12-27 11:19:36,428][20381] Updated weights for policy 1, policy_version 822185 (0.0008) [2023-12-27 11:19:36,483][20381] Updated weights for policy 1, policy_version 822195 (0.0008) [2023-12-27 11:19:36,834][20347] Updated weights for policy 0, policy_version 822753 (0.0008) [2023-12-27 11:19:36,884][20347] Updated weights for policy 0, policy_version 822763 (0.0008) [2023-12-27 11:19:36,938][20347] Updated weights for policy 0, policy_version 822773 (0.0006) [2023-12-27 11:19:37,000][20347] Updated weights for policy 0, policy_version 822783 (0.0008) [2023-12-27 11:19:37,237][20381] Updated weights for policy 1, policy_version 822205 (0.0008) [2023-12-27 11:19:37,294][20381] Updated weights for policy 1, policy_version 822215 (0.0008) [2023-12-27 11:19:37,349][20381] Updated weights for policy 1, policy_version 822225 (0.0007) [2023-12-27 11:19:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 421167104. Throughput: 0: 9792.4, 1: 9700.4. Samples: 421143272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:37,363][19170] Avg episode reward: [(0, '785.880'), (1, '968.100')] [2023-12-27 11:19:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000822784_210657280.pth... [2023-12-27 11:19:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000821632_210362368.pth [2023-12-27 11:19:37,385][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000822232_210518016.pth... [2023-12-27 11:19:37,389][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000821080_210223104.pth [2023-12-27 11:19:37,752][20347] Updated weights for policy 0, policy_version 822793 (0.0009) [2023-12-27 11:19:37,811][20347] Updated weights for policy 0, policy_version 822803 (0.0007) [2023-12-27 11:19:37,871][20347] Updated weights for policy 0, policy_version 822813 (0.0008) [2023-12-27 11:19:38,010][20381] Updated weights for policy 1, policy_version 822235 (0.0007) [2023-12-27 11:19:38,072][20381] Updated weights for policy 1, policy_version 822245 (0.0008) [2023-12-27 11:19:38,133][20381] Updated weights for policy 1, policy_version 822255 (0.0008) [2023-12-27 11:19:38,583][20347] Updated weights for policy 0, policy_version 822823 (0.0007) [2023-12-27 11:19:38,652][20347] Updated weights for policy 0, policy_version 822833 (0.0006) [2023-12-27 11:19:38,718][20347] Updated weights for policy 0, policy_version 822843 (0.0008) [2023-12-27 11:19:38,833][20381] Updated weights for policy 1, policy_version 822265 (0.0008) [2023-12-27 11:19:38,902][20381] Updated weights for policy 1, policy_version 822275 (0.0006) [2023-12-27 11:19:38,967][20381] Updated weights for policy 1, policy_version 822285 (0.0007) [2023-12-27 11:19:39,031][20381] Updated weights for policy 1, policy_version 822295 (0.0007) [2023-12-27 11:19:39,373][20347] Updated weights for policy 0, policy_version 822853 (0.0008) [2023-12-27 11:19:39,448][20347] Updated weights for policy 0, policy_version 822863 (0.0008) [2023-12-27 11:19:39,507][20347] Updated weights for policy 0, policy_version 822873 (0.0008) [2023-12-27 11:19:39,735][20381] Updated weights for policy 1, policy_version 822305 (0.0008) [2023-12-27 11:19:39,792][20381] Updated weights for policy 1, policy_version 822315 (0.0008) [2023-12-27 11:19:39,860][20381] Updated weights for policy 1, policy_version 822325 (0.0009) [2023-12-27 11:19:40,277][20347] Updated weights for policy 0, policy_version 822883 (0.0008) [2023-12-27 11:19:40,341][20347] Updated weights for policy 0, policy_version 822893 (0.0008) [2023-12-27 11:19:40,404][20347] Updated weights for policy 0, policy_version 822903 (0.0008) [2023-12-27 11:19:40,606][20381] Updated weights for policy 1, policy_version 822335 (0.0007) [2023-12-27 11:19:40,673][20381] Updated weights for policy 1, policy_version 822345 (0.0008) [2023-12-27 11:19:40,736][20381] Updated weights for policy 1, policy_version 822355 (0.0009) [2023-12-27 11:19:41,141][20347] Updated weights for policy 0, policy_version 822913 (0.0008) [2023-12-27 11:19:41,203][20347] Updated weights for policy 0, policy_version 822923 (0.0007) [2023-12-27 11:19:41,273][20347] Updated weights for policy 0, policy_version 822933 (0.0007) [2023-12-27 11:19:41,329][20347] Updated weights for policy 0, policy_version 822943 (0.0006) [2023-12-27 11:19:41,495][20381] Updated weights for policy 1, policy_version 822365 (0.0008) [2023-12-27 11:19:41,560][20381] Updated weights for policy 1, policy_version 822375 (0.0008) [2023-12-27 11:19:41,630][20381] Updated weights for policy 1, policy_version 822385 (0.0009) [2023-12-27 11:19:42,025][20347] Updated weights for policy 0, policy_version 822953 (0.0008) [2023-12-27 11:19:42,084][20347] Updated weights for policy 0, policy_version 822963 (0.0007) [2023-12-27 11:19:42,150][20347] Updated weights for policy 0, policy_version 822973 (0.0008) [2023-12-27 11:19:42,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 421265408. Throughput: 0: 9793.2, 1: 9677.6. Samples: 421258340. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:42,367][19170] Avg episode reward: [(0, '795.050'), (1, '991.420')] [2023-12-27 11:19:42,436][20381] Updated weights for policy 1, policy_version 822395 (0.0008) [2023-12-27 11:19:42,497][20381] Updated weights for policy 1, policy_version 822405 (0.0009) [2023-12-27 11:19:42,566][20381] Updated weights for policy 1, policy_version 822415 (0.0011) [2023-12-27 11:19:42,920][20347] Updated weights for policy 0, policy_version 822983 (0.0008) [2023-12-27 11:19:42,969][20347] Updated weights for policy 0, policy_version 822993 (0.0007) [2023-12-27 11:19:43,022][20347] Updated weights for policy 0, policy_version 823003 (0.0009) [2023-12-27 11:19:43,221][20381] Updated weights for policy 1, policy_version 822425 (0.0011) [2023-12-27 11:19:43,281][20381] Updated weights for policy 1, policy_version 822435 (0.0010) [2023-12-27 11:19:43,335][20381] Updated weights for policy 1, policy_version 822445 (0.0008) [2023-12-27 11:19:43,394][20381] Updated weights for policy 1, policy_version 822455 (0.0009) [2023-12-27 11:19:43,776][20347] Updated weights for policy 0, policy_version 823013 (0.0008) [2023-12-27 11:19:43,831][20347] Updated weights for policy 0, policy_version 823023 (0.0007) [2023-12-27 11:19:43,884][20347] Updated weights for policy 0, policy_version 823033 (0.0006) [2023-12-27 11:19:44,039][20381] Updated weights for policy 1, policy_version 822465 (0.0006) [2023-12-27 11:19:44,099][20381] Updated weights for policy 1, policy_version 822475 (0.0010) [2023-12-27 11:19:44,161][20381] Updated weights for policy 1, policy_version 822485 (0.0009) [2023-12-27 11:19:44,611][20347] Updated weights for policy 0, policy_version 823043 (0.0006) [2023-12-27 11:19:44,664][20347] Updated weights for policy 0, policy_version 823053 (0.0008) [2023-12-27 11:19:44,720][20347] Updated weights for policy 0, policy_version 823063 (0.0007) [2023-12-27 11:19:44,864][20381] Updated weights for policy 1, policy_version 822495 (0.0011) [2023-12-27 11:19:44,925][20381] Updated weights for policy 1, policy_version 822505 (0.0011) [2023-12-27 11:19:44,986][20381] Updated weights for policy 1, policy_version 822515 (0.0011) [2023-12-27 11:19:45,465][20347] Updated weights for policy 0, policy_version 823073 (0.0007) [2023-12-27 11:19:45,527][20347] Updated weights for policy 0, policy_version 823083 (0.0006) [2023-12-27 11:19:45,580][20347] Updated weights for policy 0, policy_version 823093 (0.0005) [2023-12-27 11:19:45,636][20347] Updated weights for policy 0, policy_version 823103 (0.0007) [2023-12-27 11:19:45,678][20381] Updated weights for policy 1, policy_version 822525 (0.0008) [2023-12-27 11:19:45,734][20381] Updated weights for policy 1, policy_version 822535 (0.0005) [2023-12-27 11:19:45,804][20381] Updated weights for policy 1, policy_version 822545 (0.0006) [2023-12-27 11:19:46,254][20347] Updated weights for policy 0, policy_version 823113 (0.0007) [2023-12-27 11:19:46,318][20347] Updated weights for policy 0, policy_version 823123 (0.0008) [2023-12-27 11:19:46,386][20347] Updated weights for policy 0, policy_version 823133 (0.0009) [2023-12-27 11:19:46,431][20381] Updated weights for policy 1, policy_version 822555 (0.0009) [2023-12-27 11:19:46,490][20381] Updated weights for policy 1, policy_version 822565 (0.0008) [2023-12-27 11:19:46,546][20381] Updated weights for policy 1, policy_version 822575 (0.0010) [2023-12-27 11:19:47,112][20347] Updated weights for policy 0, policy_version 823143 (0.0007) [2023-12-27 11:19:47,164][20347] Updated weights for policy 0, policy_version 823153 (0.0008) [2023-12-27 11:19:47,223][20347] Updated weights for policy 0, policy_version 823163 (0.0007) [2023-12-27 11:19:47,271][20381] Updated weights for policy 1, policy_version 822585 (0.0010) [2023-12-27 11:19:47,327][20381] Updated weights for policy 1, policy_version 822595 (0.0008) [2023-12-27 11:19:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.7). Total num frames: 421363712. Throughput: 0: 9763.4, 1: 9735.0. Samples: 421374700. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:47,363][19170] Avg episode reward: [(0, '834.870'), (1, '997.700')] [2023-12-27 11:19:47,382][20381] Updated weights for policy 1, policy_version 822605 (0.0008) [2023-12-27 11:19:47,442][20381] Updated weights for policy 1, policy_version 822615 (0.0007) [2023-12-27 11:19:47,962][20347] Updated weights for policy 0, policy_version 823173 (0.0008) [2023-12-27 11:19:48,016][20347] Updated weights for policy 0, policy_version 823183 (0.0008) [2023-12-27 11:19:48,078][20347] Updated weights for policy 0, policy_version 823193 (0.0008) [2023-12-27 11:19:48,173][20381] Updated weights for policy 1, policy_version 822625 (0.0010) [2023-12-27 11:19:48,234][20381] Updated weights for policy 1, policy_version 822635 (0.0011) [2023-12-27 11:19:48,294][20381] Updated weights for policy 1, policy_version 822645 (0.0011) [2023-12-27 11:19:48,843][20347] Updated weights for policy 0, policy_version 823203 (0.0008) [2023-12-27 11:19:48,903][20347] Updated weights for policy 0, policy_version 823213 (0.0008) [2023-12-27 11:19:48,958][20347] Updated weights for policy 0, policy_version 823223 (0.0008) [2023-12-27 11:19:49,073][20381] Updated weights for policy 1, policy_version 822655 (0.0011) [2023-12-27 11:19:49,137][20381] Updated weights for policy 1, policy_version 822665 (0.0008) [2023-12-27 11:19:49,188][20381] Updated weights for policy 1, policy_version 822675 (0.0010) [2023-12-27 11:19:49,655][20347] Updated weights for policy 0, policy_version 823233 (0.0008) [2023-12-27 11:19:49,718][20347] Updated weights for policy 0, policy_version 823243 (0.0006) [2023-12-27 11:19:49,786][20347] Updated weights for policy 0, policy_version 823253 (0.0006) [2023-12-27 11:19:49,854][20347] Updated weights for policy 0, policy_version 823263 (0.0008) [2023-12-27 11:19:49,940][20381] Updated weights for policy 1, policy_version 822685 (0.0010) [2023-12-27 11:19:50,001][20381] Updated weights for policy 1, policy_version 822695 (0.0009) [2023-12-27 11:19:50,061][20381] Updated weights for policy 1, policy_version 822705 (0.0008) [2023-12-27 11:19:50,522][20347] Updated weights for policy 0, policy_version 823273 (0.0007) [2023-12-27 11:19:50,600][20347] Updated weights for policy 0, policy_version 823283 (0.0008) [2023-12-27 11:19:50,662][20347] Updated weights for policy 0, policy_version 823293 (0.0008) [2023-12-27 11:19:50,781][20381] Updated weights for policy 1, policy_version 822715 (0.0010) [2023-12-27 11:19:50,840][20381] Updated weights for policy 1, policy_version 822725 (0.0011) [2023-12-27 11:19:50,901][20381] Updated weights for policy 1, policy_version 822735 (0.0011) [2023-12-27 11:19:51,458][20347] Updated weights for policy 0, policy_version 823303 (0.0008) [2023-12-27 11:19:51,518][20347] Updated weights for policy 0, policy_version 823313 (0.0008) [2023-12-27 11:19:51,582][20347] Updated weights for policy 0, policy_version 823323 (0.0008) [2023-12-27 11:19:51,614][20381] Updated weights for policy 1, policy_version 822745 (0.0010) [2023-12-27 11:19:51,678][20381] Updated weights for policy 1, policy_version 822755 (0.0006) [2023-12-27 11:19:51,746][20381] Updated weights for policy 1, policy_version 822765 (0.0008) [2023-12-27 11:19:51,803][20381] Updated weights for policy 1, policy_version 822775 (0.0009) [2023-12-27 11:19:52,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 421453824. Throughput: 0: 9726.7, 1: 9688.4. Samples: 421431152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:52,363][19170] Avg episode reward: [(0, '864.520'), (1, '1000.000')] [2023-12-27 11:19:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000822776_210657280.pth... [2023-12-27 11:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000821656_210370560.pth [2023-12-27 11:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000822776_210657280.pth [2023-12-27 11:19:52,392][20347] Updated weights for policy 0, policy_version 823333 (0.0008) [2023-12-27 11:19:52,455][20347] Updated weights for policy 0, policy_version 823343 (0.0008) [2023-12-27 11:19:52,477][20381] Updated weights for policy 1, policy_version 822785 (0.0008) [2023-12-27 11:19:52,519][20347] Updated weights for policy 0, policy_version 823353 (0.0008) [2023-12-27 11:19:52,530][20381] Updated weights for policy 1, policy_version 822795 (0.0009) [2023-12-27 11:19:52,562][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000823360_210804736.pth... [2023-12-27 11:19:52,566][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000822208_210509824.pth [2023-12-27 11:19:52,567][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000823360_210804736.pth [2023-12-27 11:19:52,589][20381] Updated weights for policy 1, policy_version 822805 (0.0011) [2023-12-27 11:19:53,209][20347] Updated weights for policy 0, policy_version 823363 (0.0008) [2023-12-27 11:19:53,262][20347] Updated weights for policy 0, policy_version 823373 (0.0008) [2023-12-27 11:19:53,317][20381] Updated weights for policy 1, policy_version 822815 (0.0009) [2023-12-27 11:19:53,317][20347] Updated weights for policy 0, policy_version 823383 (0.0006) [2023-12-27 11:19:53,375][20381] Updated weights for policy 1, policy_version 822825 (0.0007) [2023-12-27 11:19:53,428][20381] Updated weights for policy 1, policy_version 822835 (0.0008) [2023-12-27 11:19:54,039][20347] Updated weights for policy 0, policy_version 823393 (0.0006) [2023-12-27 11:19:54,098][20347] Updated weights for policy 0, policy_version 823403 (0.0008) [2023-12-27 11:19:54,119][20381] Updated weights for policy 1, policy_version 822845 (0.0009) [2023-12-27 11:19:54,158][20347] Updated weights for policy 0, policy_version 823413 (0.0007) [2023-12-27 11:19:54,184][20381] Updated weights for policy 1, policy_version 822855 (0.0006) [2023-12-27 11:19:54,218][20347] Updated weights for policy 0, policy_version 823423 (0.0007) [2023-12-27 11:19:54,250][20381] Updated weights for policy 1, policy_version 822865 (0.0006) [2023-12-27 11:19:54,925][20381] Updated weights for policy 1, policy_version 822875 (0.0007) [2023-12-27 11:19:54,930][20347] Updated weights for policy 0, policy_version 823433 (0.0008) [2023-12-27 11:19:54,987][20381] Updated weights for policy 1, policy_version 822885 (0.0008) [2023-12-27 11:19:54,993][20347] Updated weights for policy 0, policy_version 823443 (0.0007) [2023-12-27 11:19:55,049][20381] Updated weights for policy 1, policy_version 822895 (0.0011) [2023-12-27 11:19:55,055][20347] Updated weights for policy 0, policy_version 823453 (0.0006) [2023-12-27 11:19:55,728][20347] Updated weights for policy 0, policy_version 823463 (0.0007) [2023-12-27 11:19:55,751][20381] Updated weights for policy 1, policy_version 822905 (0.0007) [2023-12-27 11:19:55,791][20347] Updated weights for policy 0, policy_version 823473 (0.0007) [2023-12-27 11:19:55,801][20381] Updated weights for policy 1, policy_version 822915 (0.0005) [2023-12-27 11:19:55,851][20347] Updated weights for policy 0, policy_version 823483 (0.0007) [2023-12-27 11:19:55,858][20381] Updated weights for policy 1, policy_version 822925 (0.0006) [2023-12-27 11:19:55,916][20381] Updated weights for policy 1, policy_version 822935 (0.0007) [2023-12-27 11:19:56,585][20347] Updated weights for policy 0, policy_version 823493 (0.0006) [2023-12-27 11:19:56,644][20381] Updated weights for policy 1, policy_version 822945 (0.0007) [2023-12-27 11:19:56,646][20347] Updated weights for policy 0, policy_version 823503 (0.0008) [2023-12-27 11:19:56,701][20381] Updated weights for policy 1, policy_version 822955 (0.0007) [2023-12-27 11:19:56,709][20347] Updated weights for policy 0, policy_version 823513 (0.0007) [2023-12-27 11:19:56,761][20381] Updated weights for policy 1, policy_version 822965 (0.0007) [2023-12-27 11:19:57,282][20347] Updated weights for policy 0, policy_version 823523 (0.0006) [2023-12-27 11:19:57,349][20347] Updated weights for policy 0, policy_version 823533 (0.0007) [2023-12-27 11:19:57,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 421552128. Throughput: 0: 9691.3, 1: 9749.0. Samples: 421547580. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:19:57,363][19170] Avg episode reward: [(0, '775.460'), (1, '1000.000')] [2023-12-27 11:19:57,416][20347] Updated weights for policy 0, policy_version 823543 (0.0006) [2023-12-27 11:19:57,492][20381] Updated weights for policy 1, policy_version 822975 (0.0007) [2023-12-27 11:19:57,553][20381] Updated weights for policy 1, policy_version 822985 (0.0008) [2023-12-27 11:19:57,607][20381] Updated weights for policy 1, policy_version 822995 (0.0008) [2023-12-27 11:19:58,032][20347] Updated weights for policy 0, policy_version 823553 (0.0008) [2023-12-27 11:19:58,091][20347] Updated weights for policy 0, policy_version 823563 (0.0008) [2023-12-27 11:19:58,149][20347] Updated weights for policy 0, policy_version 823573 (0.0006) [2023-12-27 11:19:58,217][20347] Updated weights for policy 0, policy_version 823583 (0.0008) [2023-12-27 11:19:58,352][20381] Updated weights for policy 1, policy_version 823005 (0.0008) [2023-12-27 11:19:58,416][20381] Updated weights for policy 1, policy_version 823015 (0.0008) [2023-12-27 11:19:58,482][20381] Updated weights for policy 1, policy_version 823025 (0.0007) [2023-12-27 11:19:58,995][20347] Updated weights for policy 0, policy_version 823593 (0.0008) [2023-12-27 11:19:59,059][20347] Updated weights for policy 0, policy_version 823603 (0.0006) [2023-12-27 11:19:59,120][20347] Updated weights for policy 0, policy_version 823613 (0.0007) [2023-12-27 11:19:59,222][20381] Updated weights for policy 1, policy_version 823035 (0.0008) [2023-12-27 11:19:59,288][20381] Updated weights for policy 1, policy_version 823045 (0.0008) [2023-12-27 11:19:59,362][20381] Updated weights for policy 1, policy_version 823055 (0.0009) [2023-12-27 11:19:59,839][20347] Updated weights for policy 0, policy_version 823623 (0.0007) [2023-12-27 11:19:59,900][20347] Updated weights for policy 0, policy_version 823633 (0.0007) [2023-12-27 11:19:59,964][20347] Updated weights for policy 0, policy_version 823643 (0.0007) [2023-12-27 11:20:00,119][20381] Updated weights for policy 1, policy_version 823065 (0.0008) [2023-12-27 11:20:00,173][20381] Updated weights for policy 1, policy_version 823075 (0.0008) [2023-12-27 11:20:00,228][20381] Updated weights for policy 1, policy_version 823085 (0.0011) [2023-12-27 11:20:00,281][20381] Updated weights for policy 1, policy_version 823095 (0.0007) [2023-12-27 11:20:00,627][20347] Updated weights for policy 0, policy_version 823653 (0.0007) [2023-12-27 11:20:00,682][20347] Updated weights for policy 0, policy_version 823663 (0.0007) [2023-12-27 11:20:00,747][20347] Updated weights for policy 0, policy_version 823673 (0.0006) [2023-12-27 11:20:01,020][20381] Updated weights for policy 1, policy_version 823105 (0.0009) [2023-12-27 11:20:01,085][20381] Updated weights for policy 1, policy_version 823115 (0.0008) [2023-12-27 11:20:01,144][20381] Updated weights for policy 1, policy_version 823126 (0.0008) [2023-12-27 11:20:01,464][20347] Updated weights for policy 0, policy_version 823683 (0.0008) [2023-12-27 11:20:01,527][20347] Updated weights for policy 0, policy_version 823693 (0.0009) [2023-12-27 11:20:01,587][20347] Updated weights for policy 0, policy_version 823703 (0.0007) [2023-12-27 11:20:01,856][20381] Updated weights for policy 1, policy_version 823136 (0.0006) [2023-12-27 11:20:01,923][20381] Updated weights for policy 1, policy_version 823146 (0.0009) [2023-12-27 11:20:01,983][20381] Updated weights for policy 1, policy_version 823156 (0.0008) [2023-12-27 11:20:02,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 421650432. Throughput: 0: 9735.4, 1: 9697.3. Samples: 421663132. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:02,363][19170] Avg episode reward: [(0, '688.050'), (1, '1000.000')] [2023-12-27 11:20:02,372][20347] Updated weights for policy 0, policy_version 823713 (0.0008) [2023-12-27 11:20:02,433][20347] Updated weights for policy 0, policy_version 823723 (0.0008) [2023-12-27 11:20:02,499][20347] Updated weights for policy 0, policy_version 823733 (0.0008) [2023-12-27 11:20:02,557][20347] Updated weights for policy 0, policy_version 823743 (0.0008) [2023-12-27 11:20:02,708][20381] Updated weights for policy 1, policy_version 823166 (0.0008) [2023-12-27 11:20:02,756][20381] Updated weights for policy 1, policy_version 823176 (0.0008) [2023-12-27 11:20:02,813][20381] Updated weights for policy 1, policy_version 823186 (0.0009) [2023-12-27 11:20:03,225][20347] Updated weights for policy 0, policy_version 823753 (0.0008) [2023-12-27 11:20:03,283][20347] Updated weights for policy 0, policy_version 823763 (0.0008) [2023-12-27 11:20:03,345][20347] Updated weights for policy 0, policy_version 823773 (0.0008) [2023-12-27 11:20:03,550][20381] Updated weights for policy 1, policy_version 823196 (0.0007) [2023-12-27 11:20:03,607][20381] Updated weights for policy 1, policy_version 823206 (0.0006) [2023-12-27 11:20:03,665][20381] Updated weights for policy 1, policy_version 823216 (0.0006) [2023-12-27 11:20:04,088][20347] Updated weights for policy 0, policy_version 823783 (0.0008) [2023-12-27 11:20:04,154][20347] Updated weights for policy 0, policy_version 823793 (0.0008) [2023-12-27 11:20:04,217][20347] Updated weights for policy 0, policy_version 823803 (0.0009) [2023-12-27 11:20:04,347][20381] Updated weights for policy 1, policy_version 823226 (0.0007) [2023-12-27 11:20:04,410][20381] Updated weights for policy 1, policy_version 823236 (0.0008) [2023-12-27 11:20:04,473][20381] Updated weights for policy 1, policy_version 823246 (0.0006) [2023-12-27 11:20:04,539][20381] Updated weights for policy 1, policy_version 823256 (0.0009) [2023-12-27 11:20:04,918][20347] Updated weights for policy 0, policy_version 823813 (0.0008) [2023-12-27 11:20:04,977][20347] Updated weights for policy 0, policy_version 823823 (0.0007) [2023-12-27 11:20:05,036][20347] Updated weights for policy 0, policy_version 823833 (0.0006) [2023-12-27 11:20:05,219][20381] Updated weights for policy 1, policy_version 823266 (0.0006) [2023-12-27 11:20:05,275][20381] Updated weights for policy 1, policy_version 823276 (0.0005) [2023-12-27 11:20:05,332][20381] Updated weights for policy 1, policy_version 823286 (0.0007) [2023-12-27 11:20:05,674][20347] Updated weights for policy 0, policy_version 823843 (0.0007) [2023-12-27 11:20:05,731][20347] Updated weights for policy 0, policy_version 823853 (0.0007) [2023-12-27 11:20:05,783][20347] Updated weights for policy 0, policy_version 823863 (0.0007) [2023-12-27 11:20:06,054][20381] Updated weights for policy 1, policy_version 823296 (0.0008) [2023-12-27 11:20:06,129][20381] Updated weights for policy 1, policy_version 823306 (0.0008) [2023-12-27 11:20:06,178][20381] Updated weights for policy 1, policy_version 823316 (0.0008) [2023-12-27 11:20:06,450][20347] Updated weights for policy 0, policy_version 823873 (0.0007) [2023-12-27 11:20:06,514][20347] Updated weights for policy 0, policy_version 823883 (0.0006) [2023-12-27 11:20:06,581][20347] Updated weights for policy 0, policy_version 823893 (0.0007) [2023-12-27 11:20:06,648][20347] Updated weights for policy 0, policy_version 823903 (0.0008) [2023-12-27 11:20:06,921][20381] Updated weights for policy 1, policy_version 823326 (0.0008) [2023-12-27 11:20:06,985][20381] Updated weights for policy 1, policy_version 823336 (0.0007) [2023-12-27 11:20:07,044][20381] Updated weights for policy 1, policy_version 823346 (0.0008) [2023-12-27 11:20:07,334][20347] Updated weights for policy 0, policy_version 823913 (0.0008) [2023-12-27 11:20:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 421748736. Throughput: 0: 9725.4, 1: 9704.2. Samples: 421721572. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:07,363][19170] Avg episode reward: [(0, '591.150'), (1, '1000.000')] [2023-12-27 11:20:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000823352_210804736.pth... [2023-12-27 11:20:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000822232_210518016.pth [2023-12-27 11:20:07,398][20347] Updated weights for policy 0, policy_version 823923 (0.0008) [2023-12-27 11:20:07,458][20347] Updated weights for policy 0, policy_version 823933 (0.0008) [2023-12-27 11:20:07,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000823936_210952192.pth... [2023-12-27 11:20:07,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000822784_210657280.pth [2023-12-27 11:20:07,760][20381] Updated weights for policy 1, policy_version 823356 (0.0007) [2023-12-27 11:20:07,818][20381] Updated weights for policy 1, policy_version 823366 (0.0008) [2023-12-27 11:20:07,869][20381] Updated weights for policy 1, policy_version 823376 (0.0008) [2023-12-27 11:20:08,181][20347] Updated weights for policy 0, policy_version 823943 (0.0008) [2023-12-27 11:20:08,240][20347] Updated weights for policy 0, policy_version 823953 (0.0008) [2023-12-27 11:20:08,302][20347] Updated weights for policy 0, policy_version 823963 (0.0007) [2023-12-27 11:20:08,578][20381] Updated weights for policy 1, policy_version 823386 (0.0008) [2023-12-27 11:20:08,645][20381] Updated weights for policy 1, policy_version 823396 (0.0008) [2023-12-27 11:20:08,712][20381] Updated weights for policy 1, policy_version 823406 (0.0007) [2023-12-27 11:20:08,780][20381] Updated weights for policy 1, policy_version 823416 (0.0007) [2023-12-27 11:20:09,038][20347] Updated weights for policy 0, policy_version 823973 (0.0008) [2023-12-27 11:20:09,095][20347] Updated weights for policy 0, policy_version 823983 (0.0008) [2023-12-27 11:20:09,154][20347] Updated weights for policy 0, policy_version 823993 (0.0007) [2023-12-27 11:20:09,482][20381] Updated weights for policy 1, policy_version 823426 (0.0006) [2023-12-27 11:20:09,544][20381] Updated weights for policy 1, policy_version 823436 (0.0006) [2023-12-27 11:20:09,611][20381] Updated weights for policy 1, policy_version 823446 (0.0006) [2023-12-27 11:20:09,946][20347] Updated weights for policy 0, policy_version 824003 (0.0007) [2023-12-27 11:20:10,014][20347] Updated weights for policy 0, policy_version 824013 (0.0008) [2023-12-27 11:20:10,077][20347] Updated weights for policy 0, policy_version 824023 (0.0007) [2023-12-27 11:20:10,297][20381] Updated weights for policy 1, policy_version 823456 (0.0008) [2023-12-27 11:20:10,360][20381] Updated weights for policy 1, policy_version 823466 (0.0008) [2023-12-27 11:20:10,430][20381] Updated weights for policy 1, policy_version 823476 (0.0007) [2023-12-27 11:20:10,783][20347] Updated weights for policy 0, policy_version 824033 (0.0008) [2023-12-27 11:20:10,840][20347] Updated weights for policy 0, policy_version 824043 (0.0007) [2023-12-27 11:20:10,896][20347] Updated weights for policy 0, policy_version 824053 (0.0009) [2023-12-27 11:20:10,963][20347] Updated weights for policy 0, policy_version 824063 (0.0007) [2023-12-27 11:20:11,141][20381] Updated weights for policy 1, policy_version 823486 (0.0008) [2023-12-27 11:20:11,213][20381] Updated weights for policy 1, policy_version 823496 (0.0008) [2023-12-27 11:20:11,282][20381] Updated weights for policy 1, policy_version 823506 (0.0009) [2023-12-27 11:20:11,738][20347] Updated weights for policy 0, policy_version 824073 (0.0008) [2023-12-27 11:20:11,807][20347] Updated weights for policy 0, policy_version 824083 (0.0008) [2023-12-27 11:20:11,874][20347] Updated weights for policy 0, policy_version 824093 (0.0008) [2023-12-27 11:20:12,023][20381] Updated weights for policy 1, policy_version 823516 (0.0007) [2023-12-27 11:20:12,082][20381] Updated weights for policy 1, policy_version 823526 (0.0007) [2023-12-27 11:20:12,130][20381] Updated weights for policy 1, policy_version 823536 (0.0008) [2023-12-27 11:20:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 421847040. Throughput: 0: 9740.4, 1: 9692.2. Samples: 421837724. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:12,363][19170] Avg episode reward: [(0, '638.720'), (1, '1000.000')] [2023-12-27 11:20:12,608][20347] Updated weights for policy 0, policy_version 824103 (0.0008) [2023-12-27 11:20:12,665][20347] Updated weights for policy 0, policy_version 824113 (0.0007) [2023-12-27 11:20:12,736][20347] Updated weights for policy 0, policy_version 824123 (0.0008) [2023-12-27 11:20:12,819][20381] Updated weights for policy 1, policy_version 823546 (0.0007) [2023-12-27 11:20:12,887][20381] Updated weights for policy 1, policy_version 823556 (0.0006) [2023-12-27 11:20:12,949][20381] Updated weights for policy 1, policy_version 823566 (0.0006) [2023-12-27 11:20:13,000][20381] Updated weights for policy 1, policy_version 823576 (0.0008) [2023-12-27 11:20:13,411][20347] Updated weights for policy 0, policy_version 824133 (0.0008) [2023-12-27 11:20:13,467][20347] Updated weights for policy 0, policy_version 824143 (0.0007) [2023-12-27 11:20:13,530][20347] Updated weights for policy 0, policy_version 824153 (0.0007) [2023-12-27 11:20:13,726][20381] Updated weights for policy 1, policy_version 823586 (0.0008) [2023-12-27 11:20:13,786][20381] Updated weights for policy 1, policy_version 823596 (0.0008) [2023-12-27 11:20:13,852][20381] Updated weights for policy 1, policy_version 823606 (0.0007) [2023-12-27 11:20:14,225][20347] Updated weights for policy 0, policy_version 824163 (0.0007) [2023-12-27 11:20:14,278][20347] Updated weights for policy 0, policy_version 824173 (0.0008) [2023-12-27 11:20:14,334][20347] Updated weights for policy 0, policy_version 824183 (0.0007) [2023-12-27 11:20:14,589][20381] Updated weights for policy 1, policy_version 823616 (0.0007) [2023-12-27 11:20:14,654][20381] Updated weights for policy 1, policy_version 823626 (0.0008) [2023-12-27 11:20:14,711][20381] Updated weights for policy 1, policy_version 823636 (0.0008) [2023-12-27 11:20:15,036][20347] Updated weights for policy 0, policy_version 824193 (0.0007) [2023-12-27 11:20:15,097][20347] Updated weights for policy 0, policy_version 824203 (0.0008) [2023-12-27 11:20:15,163][20347] Updated weights for policy 0, policy_version 824213 (0.0008) [2023-12-27 11:20:15,224][20347] Updated weights for policy 0, policy_version 824223 (0.0006) [2023-12-27 11:20:15,506][20381] Updated weights for policy 1, policy_version 823646 (0.0008) [2023-12-27 11:20:15,570][20381] Updated weights for policy 1, policy_version 823656 (0.0008) [2023-12-27 11:20:15,635][20381] Updated weights for policy 1, policy_version 823666 (0.0008) [2023-12-27 11:20:15,937][20347] Updated weights for policy 0, policy_version 824233 (0.0008) [2023-12-27 11:20:15,995][20347] Updated weights for policy 0, policy_version 824243 (0.0008) [2023-12-27 11:20:16,051][20347] Updated weights for policy 0, policy_version 824253 (0.0008) [2023-12-27 11:20:16,252][20381] Updated weights for policy 1, policy_version 823676 (0.0008) [2023-12-27 11:20:16,312][20381] Updated weights for policy 1, policy_version 823686 (0.0008) [2023-12-27 11:20:16,366][20381] Updated weights for policy 1, policy_version 823696 (0.0008) [2023-12-27 11:20:16,777][20347] Updated weights for policy 0, policy_version 824263 (0.0007) [2023-12-27 11:20:16,827][20347] Updated weights for policy 0, policy_version 824273 (0.0007) [2023-12-27 11:20:16,883][20347] Updated weights for policy 0, policy_version 824283 (0.0007) [2023-12-27 11:20:17,049][20381] Updated weights for policy 1, policy_version 823706 (0.0007) [2023-12-27 11:20:17,114][20381] Updated weights for policy 1, policy_version 823716 (0.0008) [2023-12-27 11:20:17,172][20381] Updated weights for policy 1, policy_version 823726 (0.0008) [2023-12-27 11:20:17,231][20381] Updated weights for policy 1, policy_version 823736 (0.0008) [2023-12-27 11:20:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 421945344. Throughput: 0: 9673.9, 1: 9654.3. Samples: 421953468. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:17,363][19170] Avg episode reward: [(0, '688.150'), (1, '1000.000')] [2023-12-27 11:20:17,612][20347] Updated weights for policy 0, policy_version 824293 (0.0007) [2023-12-27 11:20:17,667][20347] Updated weights for policy 0, policy_version 824303 (0.0008) [2023-12-27 11:20:17,727][20347] Updated weights for policy 0, policy_version 824313 (0.0008) [2023-12-27 11:20:17,898][20381] Updated weights for policy 1, policy_version 823746 (0.0006) [2023-12-27 11:20:17,952][20381] Updated weights for policy 1, policy_version 823756 (0.0009) [2023-12-27 11:20:18,007][20381] Updated weights for policy 1, policy_version 823766 (0.0007) [2023-12-27 11:20:18,367][20347] Updated weights for policy 0, policy_version 824323 (0.0006) [2023-12-27 11:20:18,431][20347] Updated weights for policy 0, policy_version 824333 (0.0008) [2023-12-27 11:20:18,496][20347] Updated weights for policy 0, policy_version 824343 (0.0008) [2023-12-27 11:20:18,781][20381] Updated weights for policy 1, policy_version 823776 (0.0008) [2023-12-27 11:20:18,846][20381] Updated weights for policy 1, policy_version 823786 (0.0008) [2023-12-27 11:20:18,913][20381] Updated weights for policy 1, policy_version 823796 (0.0008) [2023-12-27 11:20:19,196][20347] Updated weights for policy 0, policy_version 824353 (0.0008) [2023-12-27 11:20:19,254][20347] Updated weights for policy 0, policy_version 824363 (0.0008) [2023-12-27 11:20:19,311][20347] Updated weights for policy 0, policy_version 824373 (0.0007) [2023-12-27 11:20:19,380][20347] Updated weights for policy 0, policy_version 824383 (0.0008) [2023-12-27 11:20:19,712][20381] Updated weights for policy 1, policy_version 823806 (0.0009) [2023-12-27 11:20:19,771][20381] Updated weights for policy 1, policy_version 823816 (0.0009) [2023-12-27 11:20:19,833][20381] Updated weights for policy 1, policy_version 823826 (0.0013) [2023-12-27 11:20:20,161][20347] Updated weights for policy 0, policy_version 824393 (0.0008) [2023-12-27 11:20:20,214][20347] Updated weights for policy 0, policy_version 824403 (0.0008) [2023-12-27 11:20:20,274][20347] Updated weights for policy 0, policy_version 824413 (0.0008) [2023-12-27 11:20:20,603][20381] Updated weights for policy 1, policy_version 823836 (0.0008) [2023-12-27 11:20:20,669][20381] Updated weights for policy 1, policy_version 823846 (0.0009) [2023-12-27 11:20:20,739][20381] Updated weights for policy 1, policy_version 823856 (0.0008) [2023-12-27 11:20:21,016][20347] Updated weights for policy 0, policy_version 824423 (0.0008) [2023-12-27 11:20:21,089][20347] Updated weights for policy 0, policy_version 824433 (0.0008) [2023-12-27 11:20:21,151][20347] Updated weights for policy 0, policy_version 824443 (0.0008) [2023-12-27 11:20:21,522][20381] Updated weights for policy 1, policy_version 823866 (0.0009) [2023-12-27 11:20:21,576][20381] Updated weights for policy 1, policy_version 823876 (0.0008) [2023-12-27 11:20:21,640][20381] Updated weights for policy 1, policy_version 823886 (0.0008) [2023-12-27 11:20:21,697][20381] Updated weights for policy 1, policy_version 823896 (0.0007) [2023-12-27 11:20:21,901][20347] Updated weights for policy 0, policy_version 824453 (0.0007) [2023-12-27 11:20:21,962][20347] Updated weights for policy 0, policy_version 824463 (0.0008) [2023-12-27 11:20:22,023][20347] Updated weights for policy 0, policy_version 824473 (0.0007) [2023-12-27 11:20:22,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 422035456. Throughput: 0: 9667.6, 1: 9616.6. Samples: 422011064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:22,363][19170] Avg episode reward: [(0, '726.570'), (1, '1000.000')] [2023-12-27 11:20:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000824480_211091456.pth... [2023-12-27 11:20:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000823360_210804736.pth [2023-12-27 11:20:22,437][20381] Updated weights for policy 1, policy_version 823906 (0.0008) [2023-12-27 11:20:22,501][20381] Updated weights for policy 1, policy_version 823916 (0.0008) [2023-12-27 11:20:22,568][20381] Updated weights for policy 1, policy_version 823926 (0.0008) [2023-12-27 11:20:22,579][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000823928_210952192.pth... [2023-12-27 11:20:22,584][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000822776_210657280.pth [2023-12-27 11:20:22,778][20347] Updated weights for policy 0, policy_version 824483 (0.0009) [2023-12-27 11:20:22,842][20347] Updated weights for policy 0, policy_version 824493 (0.0008) [2023-12-27 11:20:22,908][20347] Updated weights for policy 0, policy_version 824503 (0.0008) [2023-12-27 11:20:23,326][20381] Updated weights for policy 1, policy_version 823936 (0.0008) [2023-12-27 11:20:23,378][20381] Updated weights for policy 1, policy_version 823946 (0.0006) [2023-12-27 11:20:23,430][20381] Updated weights for policy 1, policy_version 823956 (0.0006) [2023-12-27 11:20:23,635][20347] Updated weights for policy 0, policy_version 824513 (0.0007) [2023-12-27 11:20:23,703][20347] Updated weights for policy 0, policy_version 824523 (0.0008) [2023-12-27 11:20:23,761][20347] Updated weights for policy 0, policy_version 824533 (0.0008) [2023-12-27 11:20:23,817][20347] Updated weights for policy 0, policy_version 824543 (0.0007) [2023-12-27 11:20:24,086][20381] Updated weights for policy 1, policy_version 823966 (0.0008) [2023-12-27 11:20:24,150][20381] Updated weights for policy 1, policy_version 823976 (0.0008) [2023-12-27 11:20:24,214][20381] Updated weights for policy 1, policy_version 823986 (0.0007) [2023-12-27 11:20:24,515][20347] Updated weights for policy 0, policy_version 824553 (0.0007) [2023-12-27 11:20:24,580][20347] Updated weights for policy 0, policy_version 824563 (0.0008) [2023-12-27 11:20:24,634][20347] Updated weights for policy 0, policy_version 824573 (0.0007) [2023-12-27 11:20:24,972][20381] Updated weights for policy 1, policy_version 823996 (0.0008) [2023-12-27 11:20:25,034][20381] Updated weights for policy 1, policy_version 824006 (0.0008) [2023-12-27 11:20:25,079][20381] Updated weights for policy 1, policy_version 824016 (0.0008) [2023-12-27 11:20:25,312][20347] Updated weights for policy 0, policy_version 824583 (0.0006) [2023-12-27 11:20:25,369][20347] Updated weights for policy 0, policy_version 824593 (0.0008) [2023-12-27 11:20:25,430][20347] Updated weights for policy 0, policy_version 824603 (0.0008) [2023-12-27 11:20:25,806][20381] Updated weights for policy 1, policy_version 824026 (0.0008) [2023-12-27 11:20:25,874][20381] Updated weights for policy 1, policy_version 824036 (0.0007) [2023-12-27 11:20:25,938][20381] Updated weights for policy 1, policy_version 824046 (0.0008) [2023-12-27 11:20:26,005][20381] Updated weights for policy 1, policy_version 824056 (0.0008) [2023-12-27 11:20:26,077][20347] Updated weights for policy 0, policy_version 824613 (0.0007) [2023-12-27 11:20:26,138][20347] Updated weights for policy 0, policy_version 824623 (0.0008) [2023-12-27 11:20:26,202][20347] Updated weights for policy 0, policy_version 824633 (0.0007) [2023-12-27 11:20:26,748][20381] Updated weights for policy 1, policy_version 824066 (0.0008) [2023-12-27 11:20:26,809][20381] Updated weights for policy 1, policy_version 824076 (0.0008) [2023-12-27 11:20:26,865][20381] Updated weights for policy 1, policy_version 824086 (0.0008) [2023-12-27 11:20:26,887][20347] Updated weights for policy 0, policy_version 824643 (0.0009) [2023-12-27 11:20:26,937][20347] Updated weights for policy 0, policy_version 824653 (0.0010) [2023-12-27 11:20:26,990][20347] Updated weights for policy 0, policy_version 824663 (0.0010) [2023-12-27 11:20:27,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 422133760. Throughput: 0: 9657.2, 1: 9611.2. Samples: 422125416. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:27,363][19170] Avg episode reward: [(0, '677.190'), (1, '1000.000')] [2023-12-27 11:20:27,562][20381] Updated weights for policy 1, policy_version 824096 (0.0008) [2023-12-27 11:20:27,622][20381] Updated weights for policy 1, policy_version 824106 (0.0008) [2023-12-27 11:20:27,669][20347] Updated weights for policy 0, policy_version 824673 (0.0011) [2023-12-27 11:20:27,679][20381] Updated weights for policy 1, policy_version 824116 (0.0007) [2023-12-27 11:20:27,726][20347] Updated weights for policy 0, policy_version 824683 (0.0010) [2023-12-27 11:20:27,782][20347] Updated weights for policy 0, policy_version 824693 (0.0010) [2023-12-27 11:20:27,834][20347] Updated weights for policy 0, policy_version 824703 (0.0010) [2023-12-27 11:20:28,390][20381] Updated weights for policy 1, policy_version 824126 (0.0007) [2023-12-27 11:20:28,454][20381] Updated weights for policy 1, policy_version 824136 (0.0008) [2023-12-27 11:20:28,507][20381] Updated weights for policy 1, policy_version 824146 (0.0008) [2023-12-27 11:20:28,519][20347] Updated weights for policy 0, policy_version 824713 (0.0007) [2023-12-27 11:20:28,576][20347] Updated weights for policy 0, policy_version 824723 (0.0007) [2023-12-27 11:20:28,635][20347] Updated weights for policy 0, policy_version 824733 (0.0008) [2023-12-27 11:20:29,271][20381] Updated weights for policy 1, policy_version 824156 (0.0008) [2023-12-27 11:20:29,338][20381] Updated weights for policy 1, policy_version 824166 (0.0007) [2023-12-27 11:20:29,338][20347] Updated weights for policy 0, policy_version 824743 (0.0009) [2023-12-27 11:20:29,360][20291] KL-divergence is very high: 235.5582 [2023-12-27 11:20:29,401][20347] Updated weights for policy 0, policy_version 824753 (0.0008) [2023-12-27 11:20:29,404][20381] Updated weights for policy 1, policy_version 824176 (0.0006) [2023-12-27 11:20:29,406][20291] KL-divergence is very high: 264.0870 [2023-12-27 11:20:29,453][20291] KL-divergence is very high: 137.5474 [2023-12-27 11:20:29,459][20347] Updated weights for policy 0, policy_version 824763 (0.0007) [2023-12-27 11:20:30,163][20381] Updated weights for policy 1, policy_version 824186 (0.0007) [2023-12-27 11:20:30,182][20347] Updated weights for policy 0, policy_version 824773 (0.0008) [2023-12-27 11:20:30,224][20381] Updated weights for policy 1, policy_version 824196 (0.0009) [2023-12-27 11:20:30,237][20347] Updated weights for policy 0, policy_version 824783 (0.0008) [2023-12-27 11:20:30,281][20381] Updated weights for policy 1, policy_version 824206 (0.0006) [2023-12-27 11:20:30,290][20347] Updated weights for policy 0, policy_version 824793 (0.0006) [2023-12-27 11:20:30,339][20381] Updated weights for policy 1, policy_version 824216 (0.0006) [2023-12-27 11:20:30,923][20347] Updated weights for policy 0, policy_version 824803 (0.0006) [2023-12-27 11:20:30,983][20347] Updated weights for policy 0, policy_version 824813 (0.0008) [2023-12-27 11:20:31,043][20347] Updated weights for policy 0, policy_version 824823 (0.0006) [2023-12-27 11:20:31,056][20381] Updated weights for policy 1, policy_version 824226 (0.0007) [2023-12-27 11:20:31,121][20381] Updated weights for policy 1, policy_version 824236 (0.0008) [2023-12-27 11:20:31,187][20381] Updated weights for policy 1, policy_version 824246 (0.0007) [2023-12-27 11:20:31,789][20347] Updated weights for policy 0, policy_version 824833 (0.0007) [2023-12-27 11:20:31,846][20347] Updated weights for policy 0, policy_version 824843 (0.0007) [2023-12-27 11:20:31,906][20347] Updated weights for policy 0, policy_version 824853 (0.0008) [2023-12-27 11:20:31,928][20381] Updated weights for policy 1, policy_version 824256 (0.0007) [2023-12-27 11:20:31,960][20347] Updated weights for policy 0, policy_version 824863 (0.0007) [2023-12-27 11:20:31,986][20381] Updated weights for policy 1, policy_version 824266 (0.0007) [2023-12-27 11:20:32,039][20381] Updated weights for policy 1, policy_version 824276 (0.0010) [2023-12-27 11:20:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19466.4). Total num frames: 422232064. Throughput: 0: 9719.3, 1: 9557.3. Samples: 422242148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:32,363][19170] Avg episode reward: [(0, '736.240'), (1, '1000.000')] [2023-12-27 11:20:32,607][20347] Updated weights for policy 0, policy_version 824873 (0.0007) [2023-12-27 11:20:32,609][20291] KL-divergence is very high: 180.8414 [2023-12-27 11:20:32,614][20291] KL-divergence is very high: 149.9685 [2023-12-27 11:20:32,623][20291] KL-divergence is very high: 125.2940 [2023-12-27 11:20:32,655][20347] Updated weights for policy 0, policy_version 824883 (0.0007) [2023-12-27 11:20:32,703][20347] Updated weights for policy 0, policy_version 824893 (0.0007) [2023-12-27 11:20:32,776][20381] Updated weights for policy 1, policy_version 824286 (0.0008) [2023-12-27 11:20:32,834][20381] Updated weights for policy 1, policy_version 824296 (0.0008) [2023-12-27 11:20:32,894][20381] Updated weights for policy 1, policy_version 824306 (0.0008) [2023-12-27 11:20:33,356][20347] Updated weights for policy 0, policy_version 824903 (0.0006) [2023-12-27 11:20:33,412][20347] Updated weights for policy 0, policy_version 824913 (0.0006) [2023-12-27 11:20:33,473][20347] Updated weights for policy 0, policy_version 824923 (0.0007) [2023-12-27 11:20:33,577][20381] Updated weights for policy 1, policy_version 824316 (0.0006) [2023-12-27 11:20:33,630][20381] Updated weights for policy 1, policy_version 824326 (0.0008) [2023-12-27 11:20:33,687][20381] Updated weights for policy 1, policy_version 824336 (0.0008) [2023-12-27 11:20:34,107][20347] Updated weights for policy 0, policy_version 824933 (0.0008) [2023-12-27 11:20:34,157][20347] Updated weights for policy 0, policy_version 824943 (0.0007) [2023-12-27 11:20:34,215][20347] Updated weights for policy 0, policy_version 824953 (0.0007) [2023-12-27 11:20:34,365][20381] Updated weights for policy 1, policy_version 824346 (0.0008) [2023-12-27 11:20:34,426][20381] Updated weights for policy 1, policy_version 824356 (0.0008) [2023-12-27 11:20:34,485][20381] Updated weights for policy 1, policy_version 824366 (0.0008) [2023-12-27 11:20:34,547][20381] Updated weights for policy 1, policy_version 824376 (0.0008) [2023-12-27 11:20:34,980][20347] Updated weights for policy 0, policy_version 824963 (0.0008) [2023-12-27 11:20:35,049][20347] Updated weights for policy 0, policy_version 824973 (0.0008) [2023-12-27 11:20:35,118][20347] Updated weights for policy 0, policy_version 824983 (0.0006) [2023-12-27 11:20:35,275][20381] Updated weights for policy 1, policy_version 824386 (0.0006) [2023-12-27 11:20:35,327][20381] Updated weights for policy 1, policy_version 824396 (0.0006) [2023-12-27 11:20:35,379][20381] Updated weights for policy 1, policy_version 824406 (0.0008) [2023-12-27 11:20:35,781][20347] Updated weights for policy 0, policy_version 824993 (0.0005) [2023-12-27 11:20:35,842][20347] Updated weights for policy 0, policy_version 825003 (0.0006) [2023-12-27 11:20:35,903][20347] Updated weights for policy 0, policy_version 825013 (0.0008) [2023-12-27 11:20:35,953][20347] Updated weights for policy 0, policy_version 825023 (0.0007) [2023-12-27 11:20:36,054][20381] Updated weights for policy 1, policy_version 824416 (0.0009) [2023-12-27 11:20:36,102][20381] Updated weights for policy 1, policy_version 824426 (0.0008) [2023-12-27 11:20:36,167][20381] Updated weights for policy 1, policy_version 824436 (0.0008) [2023-12-27 11:20:36,654][20347] Updated weights for policy 0, policy_version 825033 (0.0009) [2023-12-27 11:20:36,720][20347] Updated weights for policy 0, policy_version 825043 (0.0007) [2023-12-27 11:20:36,773][20347] Updated weights for policy 0, policy_version 825053 (0.0008) [2023-12-27 11:20:36,860][20381] Updated weights for policy 1, policy_version 824446 (0.0007) [2023-12-27 11:20:36,920][20381] Updated weights for policy 1, policy_version 824456 (0.0007) [2023-12-27 11:20:36,980][20381] Updated weights for policy 1, policy_version 824466 (0.0007) [2023-12-27 11:20:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 422330368. Throughput: 0: 9772.9, 1: 9587.6. Samples: 422302372. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:37,363][19170] Avg episode reward: [(0, '746.940'), (1, '1000.000')] [2023-12-27 11:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000824472_211091456.pth... [2023-12-27 11:20:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000825056_211238912.pth... [2023-12-27 11:20:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000823352_210804736.pth [2023-12-27 11:20:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000823936_210952192.pth [2023-12-27 11:20:37,493][20347] Updated weights for policy 0, policy_version 825063 (0.0008) [2023-12-27 11:20:37,553][20347] Updated weights for policy 0, policy_version 825073 (0.0008) [2023-12-27 11:20:37,606][20347] Updated weights for policy 0, policy_version 825083 (0.0007) [2023-12-27 11:20:37,683][20381] Updated weights for policy 1, policy_version 824476 (0.0007) [2023-12-27 11:20:37,738][20381] Updated weights for policy 1, policy_version 824486 (0.0009) [2023-12-27 11:20:37,793][20381] Updated weights for policy 1, policy_version 824496 (0.0008) [2023-12-27 11:20:38,290][20347] Updated weights for policy 0, policy_version 825093 (0.0006) [2023-12-27 11:20:38,352][20347] Updated weights for policy 0, policy_version 825103 (0.0007) [2023-12-27 11:20:38,412][20347] Updated weights for policy 0, policy_version 825113 (0.0007) [2023-12-27 11:20:38,560][20381] Updated weights for policy 1, policy_version 824506 (0.0008) [2023-12-27 11:20:38,627][20381] Updated weights for policy 1, policy_version 824516 (0.0009) [2023-12-27 11:20:38,689][20381] Updated weights for policy 1, policy_version 824526 (0.0008) [2023-12-27 11:20:38,757][20381] Updated weights for policy 1, policy_version 824536 (0.0008) [2023-12-27 11:20:39,056][20347] Updated weights for policy 0, policy_version 825123 (0.0007) [2023-12-27 11:20:39,104][20347] Updated weights for policy 0, policy_version 825133 (0.0005) [2023-12-27 11:20:39,162][20347] Updated weights for policy 0, policy_version 825143 (0.0005) [2023-12-27 11:20:39,515][20381] Updated weights for policy 1, policy_version 824546 (0.0008) [2023-12-27 11:20:39,573][20381] Updated weights for policy 1, policy_version 824556 (0.0008) [2023-12-27 11:20:39,631][20381] Updated weights for policy 1, policy_version 824566 (0.0008) [2023-12-27 11:20:39,936][20347] Updated weights for policy 0, policy_version 825153 (0.0006) [2023-12-27 11:20:40,003][20347] Updated weights for policy 0, policy_version 825163 (0.0007) [2023-12-27 11:20:40,067][20347] Updated weights for policy 0, policy_version 825173 (0.0008) [2023-12-27 11:20:40,133][20347] Updated weights for policy 0, policy_version 825183 (0.0006) [2023-12-27 11:20:40,425][20381] Updated weights for policy 1, policy_version 824576 (0.0008) [2023-12-27 11:20:40,487][20381] Updated weights for policy 1, policy_version 824586 (0.0008) [2023-12-27 11:20:40,556][20381] Updated weights for policy 1, policy_version 824596 (0.0007) [2023-12-27 11:20:40,854][20347] Updated weights for policy 0, policy_version 825193 (0.0008) [2023-12-27 11:20:40,914][20347] Updated weights for policy 0, policy_version 825203 (0.0008) [2023-12-27 11:20:40,973][20347] Updated weights for policy 0, policy_version 825213 (0.0007) [2023-12-27 11:20:41,273][20381] Updated weights for policy 1, policy_version 824606 (0.0007) [2023-12-27 11:20:41,333][20381] Updated weights for policy 1, policy_version 824616 (0.0007) [2023-12-27 11:20:41,401][20381] Updated weights for policy 1, policy_version 824626 (0.0008) [2023-12-27 11:20:41,766][20347] Updated weights for policy 0, policy_version 825223 (0.0008) [2023-12-27 11:20:41,831][20347] Updated weights for policy 0, policy_version 825233 (0.0009) [2023-12-27 11:20:41,885][20347] Updated weights for policy 0, policy_version 825243 (0.0009) [2023-12-27 11:20:42,133][20381] Updated weights for policy 1, policy_version 824636 (0.0007) [2023-12-27 11:20:42,194][20381] Updated weights for policy 1, policy_version 824646 (0.0007) [2023-12-27 11:20:42,257][20381] Updated weights for policy 1, policy_version 824656 (0.0006) [2023-12-27 11:20:42,363][19170] Fps is (10 sec: 19660.2, 60 sec: 19387.6, 300 sec: 19466.4). Total num frames: 422428672. Throughput: 0: 9822.1, 1: 9536.3. Samples: 422418716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 11:20:42,363][19170] Avg episode reward: [(0, '718.400'), (1, '1000.000')] [2023-12-27 11:20:42,721][20347] Updated weights for policy 0, policy_version 825253 (0.0008) [2023-12-27 11:20:42,771][20347] Updated weights for policy 0, policy_version 825263 (0.0006) [2023-12-27 11:20:42,829][20347] Updated weights for policy 0, policy_version 825273 (0.0006) [2023-12-27 11:20:42,971][20381] Updated weights for policy 1, policy_version 824666 (0.0008) [2023-12-27 11:20:43,031][20381] Updated weights for policy 1, policy_version 824676 (0.0008) [2023-12-27 11:20:43,094][20381] Updated weights for policy 1, policy_version 824686 (0.0008) [2023-12-27 11:20:43,158][20381] Updated weights for policy 1, policy_version 824696 (0.0007) [2023-12-27 11:20:43,507][20347] Updated weights for policy 0, policy_version 825283 (0.0008) [2023-12-27 11:20:43,578][20347] Updated weights for policy 0, policy_version 825293 (0.0008) [2023-12-27 11:20:43,641][20347] Updated weights for policy 0, policy_version 825303 (0.0006) [2023-12-27 11:20:43,839][20381] Updated weights for policy 1, policy_version 824706 (0.0007) [2023-12-27 11:20:43,896][20381] Updated weights for policy 1, policy_version 824716 (0.0008) [2023-12-27 11:20:43,945][20381] Updated weights for policy 1, policy_version 824726 (0.0008) [2023-12-27 11:20:44,274][20347] Updated weights for policy 0, policy_version 825313 (0.0006) [2023-12-27 11:20:44,340][20347] Updated weights for policy 0, policy_version 825323 (0.0007) [2023-12-27 11:20:44,400][20347] Updated weights for policy 0, policy_version 825333 (0.0008) [2023-12-27 11:20:44,458][20347] Updated weights for policy 0, policy_version 825343 (0.0007) [2023-12-27 11:20:44,625][20381] Updated weights for policy 1, policy_version 824736 (0.0006) [2023-12-27 11:20:44,684][20381] Updated weights for policy 1, policy_version 824746 (0.0008) [2023-12-27 11:20:44,749][20381] Updated weights for policy 1, policy_version 824756 (0.0008) [2023-12-27 11:20:45,188][20347] Updated weights for policy 0, policy_version 825353 (0.0008) [2023-12-27 11:20:45,254][20347] Updated weights for policy 0, policy_version 825363 (0.0008) [2023-12-27 11:20:45,320][20347] Updated weights for policy 0, policy_version 825373 (0.0008) [2023-12-27 11:20:45,487][20381] Updated weights for policy 1, policy_version 824766 (0.0008) [2023-12-27 11:20:45,553][20381] Updated weights for policy 1, policy_version 824776 (0.0008) [2023-12-27 11:20:45,615][20381] Updated weights for policy 1, policy_version 824786 (0.0007) [2023-12-27 11:20:46,021][20347] Updated weights for policy 0, policy_version 825383 (0.0008) [2023-12-27 11:20:46,084][20347] Updated weights for policy 0, policy_version 825393 (0.0008) [2023-12-27 11:20:46,138][20347] Updated weights for policy 0, policy_version 825403 (0.0006) [2023-12-27 11:20:46,302][20381] Updated weights for policy 1, policy_version 824796 (0.0008) [2023-12-27 11:20:46,366][20381] Updated weights for policy 1, policy_version 824806 (0.0007) [2023-12-27 11:20:46,422][20381] Updated weights for policy 1, policy_version 824816 (0.0008) [2023-12-27 11:20:46,799][20347] Updated weights for policy 0, policy_version 825413 (0.0008) [2023-12-27 11:20:46,852][20347] Updated weights for policy 0, policy_version 825423 (0.0008) [2023-12-27 11:20:46,910][20347] Updated weights for policy 0, policy_version 825433 (0.0008) [2023-12-27 11:20:47,145][20381] Updated weights for policy 1, policy_version 824826 (0.0008) [2023-12-27 11:20:47,204][20381] Updated weights for policy 1, policy_version 824836 (0.0008) [2023-12-27 11:20:47,259][20381] Updated weights for policy 1, policy_version 824846 (0.0008) [2023-12-27 11:20:47,318][20381] Updated weights for policy 1, policy_version 824856 (0.0008) [2023-12-27 11:20:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 422526976. Throughput: 0: 9776.0, 1: 9592.5. Samples: 422534712. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:20:47,363][19170] Avg episode reward: [(0, '716.360'), (1, '1000.000')] [2023-12-27 11:20:47,640][20347] Updated weights for policy 0, policy_version 825443 (0.0008) [2023-12-27 11:20:47,705][20347] Updated weights for policy 0, policy_version 825453 (0.0006) [2023-12-27 11:20:47,761][20347] Updated weights for policy 0, policy_version 825463 (0.0008) [2023-12-27 11:20:47,947][20381] Updated weights for policy 1, policy_version 824866 (0.0007) [2023-12-27 11:20:48,004][20381] Updated weights for policy 1, policy_version 824876 (0.0008) [2023-12-27 11:20:48,064][20381] Updated weights for policy 1, policy_version 824886 (0.0008) [2023-12-27 11:20:48,428][20347] Updated weights for policy 0, policy_version 825473 (0.0006) [2023-12-27 11:20:48,482][20347] Updated weights for policy 0, policy_version 825483 (0.0008) [2023-12-27 11:20:48,534][20347] Updated weights for policy 0, policy_version 825493 (0.0007) [2023-12-27 11:20:48,584][20347] Updated weights for policy 0, policy_version 825503 (0.0008) [2023-12-27 11:20:48,793][20381] Updated weights for policy 1, policy_version 824896 (0.0010) [2023-12-27 11:20:48,865][20381] Updated weights for policy 1, policy_version 824906 (0.0011) [2023-12-27 11:20:48,926][20381] Updated weights for policy 1, policy_version 824916 (0.0011) [2023-12-27 11:20:49,376][20347] Updated weights for policy 0, policy_version 825513 (0.0009) [2023-12-27 11:20:49,437][20347] Updated weights for policy 0, policy_version 825523 (0.0011) [2023-12-27 11:20:49,501][20347] Updated weights for policy 0, policy_version 825533 (0.0010) [2023-12-27 11:20:49,686][20381] Updated weights for policy 1, policy_version 824926 (0.0009) [2023-12-27 11:20:49,752][20381] Updated weights for policy 1, policy_version 824936 (0.0007) [2023-12-27 11:20:49,818][20381] Updated weights for policy 1, policy_version 824946 (0.0008) [2023-12-27 11:20:50,242][20347] Updated weights for policy 0, policy_version 825543 (0.0009) [2023-12-27 11:20:50,305][20347] Updated weights for policy 0, policy_version 825553 (0.0007) [2023-12-27 11:20:50,361][20347] Updated weights for policy 0, policy_version 825563 (0.0008) [2023-12-27 11:20:50,543][20381] Updated weights for policy 1, policy_version 824956 (0.0008) [2023-12-27 11:20:50,608][20381] Updated weights for policy 1, policy_version 824966 (0.0008) [2023-12-27 11:20:50,674][20381] Updated weights for policy 1, policy_version 824976 (0.0008) [2023-12-27 11:20:51,083][20347] Updated weights for policy 0, policy_version 825573 (0.0008) [2023-12-27 11:20:51,145][20347] Updated weights for policy 0, policy_version 825583 (0.0008) [2023-12-27 11:20:51,205][20347] Updated weights for policy 0, policy_version 825593 (0.0006) [2023-12-27 11:20:51,420][20381] Updated weights for policy 1, policy_version 824986 (0.0008) [2023-12-27 11:20:51,481][20381] Updated weights for policy 1, policy_version 824996 (0.0008) [2023-12-27 11:20:51,542][20381] Updated weights for policy 1, policy_version 825006 (0.0008) [2023-12-27 11:20:51,605][20381] Updated weights for policy 1, policy_version 825016 (0.0008) [2023-12-27 11:20:51,930][20347] Updated weights for policy 0, policy_version 825603 (0.0007) [2023-12-27 11:20:51,993][20347] Updated weights for policy 0, policy_version 825613 (0.0008) [2023-12-27 11:20:52,049][20347] Updated weights for policy 0, policy_version 825623 (0.0008) [2023-12-27 11:20:52,335][20381] Updated weights for policy 1, policy_version 825026 (0.0008) [2023-12-27 11:20:52,362][19170] Fps is (10 sec: 18842.0, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 422617088. Throughput: 0: 9771.6, 1: 9592.6. Samples: 422592960. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:20:52,363][19170] Avg episode reward: [(0, '657.110'), (1, '1000.000')] [2023-12-27 11:20:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000825632_211386368.pth... [2023-12-27 11:20:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000824480_211091456.pth [2023-12-27 11:20:52,405][20381] Updated weights for policy 1, policy_version 825036 (0.0009) [2023-12-27 11:20:52,467][20381] Updated weights for policy 1, policy_version 825046 (0.0007) [2023-12-27 11:20:52,476][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000825048_211238912.pth... [2023-12-27 11:20:52,481][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000823928_210952192.pth [2023-12-27 11:20:52,745][20347] Updated weights for policy 0, policy_version 825633 (0.0007) [2023-12-27 11:20:52,810][20347] Updated weights for policy 0, policy_version 825643 (0.0007) [2023-12-27 11:20:52,879][20347] Updated weights for policy 0, policy_version 825653 (0.0005) [2023-12-27 11:20:52,941][20347] Updated weights for policy 0, policy_version 825663 (0.0006) [2023-12-27 11:20:53,139][20381] Updated weights for policy 1, policy_version 825056 (0.0008) [2023-12-27 11:20:53,201][20381] Updated weights for policy 1, policy_version 825066 (0.0007) [2023-12-27 11:20:53,261][20381] Updated weights for policy 1, policy_version 825076 (0.0008) [2023-12-27 11:20:53,563][20347] Updated weights for policy 0, policy_version 825673 (0.0007) [2023-12-27 11:20:53,624][20347] Updated weights for policy 0, policy_version 825683 (0.0007) [2023-12-27 11:20:53,682][20347] Updated weights for policy 0, policy_version 825693 (0.0008) [2023-12-27 11:20:53,899][20381] Updated weights for policy 1, policy_version 825086 (0.0007) [2023-12-27 11:20:53,954][20381] Updated weights for policy 1, policy_version 825096 (0.0009) [2023-12-27 11:20:54,014][20381] Updated weights for policy 1, policy_version 825106 (0.0008) [2023-12-27 11:20:54,375][20347] Updated weights for policy 0, policy_version 825703 (0.0008) [2023-12-27 11:20:54,432][20347] Updated weights for policy 0, policy_version 825713 (0.0008) [2023-12-27 11:20:54,489][20347] Updated weights for policy 0, policy_version 825723 (0.0008) [2023-12-27 11:20:54,761][20381] Updated weights for policy 1, policy_version 825116 (0.0008) [2023-12-27 11:20:54,817][20381] Updated weights for policy 1, policy_version 825126 (0.0009) [2023-12-27 11:20:54,877][20381] Updated weights for policy 1, policy_version 825136 (0.0008) [2023-12-27 11:20:55,142][20347] Updated weights for policy 0, policy_version 825733 (0.0006) [2023-12-27 11:20:55,205][20347] Updated weights for policy 0, policy_version 825743 (0.0006) [2023-12-27 11:20:55,258][20347] Updated weights for policy 0, policy_version 825753 (0.0006) [2023-12-27 11:20:55,599][20381] Updated weights for policy 1, policy_version 825146 (0.0008) [2023-12-27 11:20:55,656][20381] Updated weights for policy 1, policy_version 825156 (0.0011) [2023-12-27 11:20:55,712][20381] Updated weights for policy 1, policy_version 825166 (0.0010) [2023-12-27 11:20:55,763][20381] Updated weights for policy 1, policy_version 825176 (0.0010) [2023-12-27 11:20:55,884][20347] Updated weights for policy 0, policy_version 825763 (0.0006) [2023-12-27 11:20:55,943][20347] Updated weights for policy 0, policy_version 825773 (0.0006) [2023-12-27 11:20:56,005][20347] Updated weights for policy 0, policy_version 825783 (0.0006) [2023-12-27 11:20:56,500][20381] Updated weights for policy 1, policy_version 825186 (0.0008) [2023-12-27 11:20:56,558][20381] Updated weights for policy 1, policy_version 825196 (0.0009) [2023-12-27 11:20:56,622][20381] Updated weights for policy 1, policy_version 825206 (0.0008) [2023-12-27 11:20:56,643][20347] Updated weights for policy 0, policy_version 825793 (0.0008) [2023-12-27 11:20:56,699][20347] Updated weights for policy 0, policy_version 825803 (0.0008) [2023-12-27 11:20:56,762][20347] Updated weights for policy 0, policy_version 825813 (0.0007) [2023-12-27 11:20:56,820][20347] Updated weights for policy 0, policy_version 825823 (0.0007) [2023-12-27 11:20:57,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 422715392. Throughput: 0: 9818.2, 1: 9598.3. Samples: 422711464. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:20:57,363][19170] Avg episode reward: [(0, '706.120'), (1, '1000.000')] [2023-12-27 11:20:57,411][20381] Updated weights for policy 1, policy_version 825216 (0.0010) [2023-12-27 11:20:57,461][20381] Updated weights for policy 1, policy_version 825226 (0.0011) [2023-12-27 11:20:57,513][20381] Updated weights for policy 1, policy_version 825236 (0.0011) [2023-12-27 11:20:57,548][20347] Updated weights for policy 0, policy_version 825833 (0.0007) [2023-12-27 11:20:57,604][20347] Updated weights for policy 0, policy_version 825843 (0.0007) [2023-12-27 11:20:57,655][20347] Updated weights for policy 0, policy_version 825853 (0.0007) [2023-12-27 11:20:58,254][20381] Updated weights for policy 1, policy_version 825246 (0.0011) [2023-12-27 11:20:58,312][20381] Updated weights for policy 1, policy_version 825256 (0.0010) [2023-12-27 11:20:58,380][20381] Updated weights for policy 1, policy_version 825266 (0.0009) [2023-12-27 11:20:58,391][20347] Updated weights for policy 0, policy_version 825863 (0.0007) [2023-12-27 11:20:58,453][20347] Updated weights for policy 0, policy_version 825873 (0.0007) [2023-12-27 11:20:58,517][20347] Updated weights for policy 0, policy_version 825883 (0.0008) [2023-12-27 11:20:59,127][20381] Updated weights for policy 1, policy_version 825276 (0.0008) [2023-12-27 11:20:59,179][20381] Updated weights for policy 1, policy_version 825286 (0.0008) [2023-12-27 11:20:59,245][20381] Updated weights for policy 1, policy_version 825296 (0.0008) [2023-12-27 11:20:59,305][20347] Updated weights for policy 0, policy_version 825893 (0.0007) [2023-12-27 11:20:59,379][20347] Updated weights for policy 0, policy_version 825903 (0.0008) [2023-12-27 11:20:59,444][20347] Updated weights for policy 0, policy_version 825913 (0.0007) [2023-12-27 11:20:59,923][20381] Updated weights for policy 1, policy_version 825306 (0.0007) [2023-12-27 11:20:59,979][20381] Updated weights for policy 1, policy_version 825316 (0.0007) [2023-12-27 11:21:00,041][20381] Updated weights for policy 1, policy_version 825326 (0.0007) [2023-12-27 11:21:00,104][20381] Updated weights for policy 1, policy_version 825336 (0.0007) [2023-12-27 11:21:00,167][20347] Updated weights for policy 0, policy_version 825923 (0.0008) [2023-12-27 11:21:00,231][20347] Updated weights for policy 0, policy_version 825933 (0.0008) [2023-12-27 11:21:00,291][20347] Updated weights for policy 0, policy_version 825943 (0.0008) [2023-12-27 11:21:00,803][20381] Updated weights for policy 1, policy_version 825346 (0.0008) [2023-12-27 11:21:00,861][20381] Updated weights for policy 1, policy_version 825356 (0.0006) [2023-12-27 11:21:00,917][20381] Updated weights for policy 1, policy_version 825366 (0.0008) [2023-12-27 11:21:01,040][20347] Updated weights for policy 0, policy_version 825953 (0.0008) [2023-12-27 11:21:01,098][20347] Updated weights for policy 0, policy_version 825963 (0.0008) [2023-12-27 11:21:01,158][20347] Updated weights for policy 0, policy_version 825973 (0.0008) [2023-12-27 11:21:01,214][20347] Updated weights for policy 0, policy_version 825983 (0.0008) [2023-12-27 11:21:01,624][20381] Updated weights for policy 1, policy_version 825376 (0.0008) [2023-12-27 11:21:01,687][20381] Updated weights for policy 1, policy_version 825386 (0.0008) [2023-12-27 11:21:01,747][20381] Updated weights for policy 1, policy_version 825396 (0.0009) [2023-12-27 11:21:01,978][20347] Updated weights for policy 0, policy_version 825993 (0.0007) [2023-12-27 11:21:02,036][20347] Updated weights for policy 0, policy_version 826003 (0.0008) [2023-12-27 11:21:02,091][20347] Updated weights for policy 0, policy_version 826013 (0.0007) [2023-12-27 11:21:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 422813696. Throughput: 0: 9786.2, 1: 9575.4. Samples: 422824740. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:02,363][19170] Avg episode reward: [(0, '784.510'), (1, '1000.000')] [2023-12-27 11:21:02,473][20381] Updated weights for policy 1, policy_version 825406 (0.0008) [2023-12-27 11:21:02,527][20381] Updated weights for policy 1, policy_version 825416 (0.0008) [2023-12-27 11:21:02,585][20381] Updated weights for policy 1, policy_version 825426 (0.0008) [2023-12-27 11:21:02,792][20347] Updated weights for policy 0, policy_version 826023 (0.0007) [2023-12-27 11:21:02,849][20347] Updated weights for policy 0, policy_version 826033 (0.0008) [2023-12-27 11:21:02,908][20347] Updated weights for policy 0, policy_version 826043 (0.0008) [2023-12-27 11:21:03,289][20381] Updated weights for policy 1, policy_version 825436 (0.0008) [2023-12-27 11:21:03,349][20381] Updated weights for policy 1, policy_version 825446 (0.0008) [2023-12-27 11:21:03,408][20381] Updated weights for policy 1, policy_version 825456 (0.0010) [2023-12-27 11:21:03,578][20347] Updated weights for policy 0, policy_version 826053 (0.0007) [2023-12-27 11:21:03,630][20347] Updated weights for policy 0, policy_version 826063 (0.0007) [2023-12-27 11:21:03,682][20347] Updated weights for policy 0, policy_version 826073 (0.0007) [2023-12-27 11:21:04,097][20381] Updated weights for policy 1, policy_version 825466 (0.0008) [2023-12-27 11:21:04,157][20381] Updated weights for policy 1, policy_version 825476 (0.0011) [2023-12-27 11:21:04,222][20381] Updated weights for policy 1, policy_version 825486 (0.0007) [2023-12-27 11:21:04,285][20381] Updated weights for policy 1, policy_version 825496 (0.0010) [2023-12-27 11:21:04,438][20347] Updated weights for policy 0, policy_version 826083 (0.0008) [2023-12-27 11:21:04,502][20347] Updated weights for policy 0, policy_version 826093 (0.0010) [2023-12-27 11:21:04,565][20347] Updated weights for policy 0, policy_version 826103 (0.0010) [2023-12-27 11:21:04,949][20381] Updated weights for policy 1, policy_version 825506 (0.0009) [2023-12-27 11:21:05,009][20381] Updated weights for policy 1, policy_version 825516 (0.0008) [2023-12-27 11:21:05,068][20381] Updated weights for policy 1, policy_version 825526 (0.0008) [2023-12-27 11:21:05,191][20347] Updated weights for policy 0, policy_version 826113 (0.0011) [2023-12-27 11:21:05,254][20347] Updated weights for policy 0, policy_version 826123 (0.0010) [2023-12-27 11:21:05,314][20347] Updated weights for policy 0, policy_version 826133 (0.0010) [2023-12-27 11:21:05,380][20347] Updated weights for policy 0, policy_version 826143 (0.0009) [2023-12-27 11:21:05,685][20381] Updated weights for policy 1, policy_version 825536 (0.0007) [2023-12-27 11:21:05,744][20381] Updated weights for policy 1, policy_version 825546 (0.0006) [2023-12-27 11:21:05,805][20381] Updated weights for policy 1, policy_version 825556 (0.0006) [2023-12-27 11:21:06,016][20347] Updated weights for policy 0, policy_version 826153 (0.0009) [2023-12-27 11:21:06,066][20347] Updated weights for policy 0, policy_version 826163 (0.0010) [2023-12-27 11:21:06,125][20347] Updated weights for policy 0, policy_version 826173 (0.0009) [2023-12-27 11:21:06,431][20381] Updated weights for policy 1, policy_version 825566 (0.0006) [2023-12-27 11:21:06,488][20381] Updated weights for policy 1, policy_version 825576 (0.0006) [2023-12-27 11:21:06,545][20381] Updated weights for policy 1, policy_version 825586 (0.0006) [2023-12-27 11:21:06,967][20347] Updated weights for policy 0, policy_version 826183 (0.0008) [2023-12-27 11:21:07,029][20347] Updated weights for policy 0, policy_version 826193 (0.0009) [2023-12-27 11:21:07,086][20347] Updated weights for policy 0, policy_version 826203 (0.0009) [2023-12-27 11:21:07,238][20381] Updated weights for policy 1, policy_version 825596 (0.0008) [2023-12-27 11:21:07,298][20381] Updated weights for policy 1, policy_version 825606 (0.0006) [2023-12-27 11:21:07,362][20381] Updated weights for policy 1, policy_version 825616 (0.0007) [2023-12-27 11:21:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 422912000. Throughput: 0: 9798.6, 1: 9605.9. Samples: 422884264. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:07,363][19170] Avg episode reward: [(0, '853.050'), (1, '1000.000')] [2023-12-27 11:21:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000826208_211533824.pth... [2023-12-27 11:21:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000825056_211238912.pth [2023-12-27 11:21:07,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000825624_211386368.pth... [2023-12-27 11:21:07,417][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000824472_211091456.pth [2023-12-27 11:21:07,822][20347] Updated weights for policy 0, policy_version 826213 (0.0009) [2023-12-27 11:21:07,876][20347] Updated weights for policy 0, policy_version 826223 (0.0008) [2023-12-27 11:21:07,929][20347] Updated weights for policy 0, policy_version 826233 (0.0007) [2023-12-27 11:21:08,026][20381] Updated weights for policy 1, policy_version 825626 (0.0008) [2023-12-27 11:21:08,086][20381] Updated weights for policy 1, policy_version 825636 (0.0006) [2023-12-27 11:21:08,147][20381] Updated weights for policy 1, policy_version 825646 (0.0006) [2023-12-27 11:21:08,206][20381] Updated weights for policy 1, policy_version 825656 (0.0007) [2023-12-27 11:21:08,657][20347] Updated weights for policy 0, policy_version 826243 (0.0009) [2023-12-27 11:21:08,715][20347] Updated weights for policy 0, policy_version 826253 (0.0007) [2023-12-27 11:21:08,770][20347] Updated weights for policy 0, policy_version 826263 (0.0007) [2023-12-27 11:21:08,857][20381] Updated weights for policy 1, policy_version 825666 (0.0007) [2023-12-27 11:21:08,918][20381] Updated weights for policy 1, policy_version 825676 (0.0008) [2023-12-27 11:21:08,976][20381] Updated weights for policy 1, policy_version 825686 (0.0007) [2023-12-27 11:21:09,539][20347] Updated weights for policy 0, policy_version 826273 (0.0008) [2023-12-27 11:21:09,602][20347] Updated weights for policy 0, policy_version 826283 (0.0007) [2023-12-27 11:21:09,662][20347] Updated weights for policy 0, policy_version 826293 (0.0007) [2023-12-27 11:21:09,720][20347] Updated weights for policy 0, policy_version 826303 (0.0007) [2023-12-27 11:21:09,748][20381] Updated weights for policy 1, policy_version 825696 (0.0008) [2023-12-27 11:21:09,813][20381] Updated weights for policy 1, policy_version 825706 (0.0008) [2023-12-27 11:21:09,873][20381] Updated weights for policy 1, policy_version 825716 (0.0008) [2023-12-27 11:21:10,441][20347] Updated weights for policy 0, policy_version 826313 (0.0007) [2023-12-27 11:21:10,505][20347] Updated weights for policy 0, policy_version 826323 (0.0007) [2023-12-27 11:21:10,574][20347] Updated weights for policy 0, policy_version 826333 (0.0007) [2023-12-27 11:21:10,637][20381] Updated weights for policy 1, policy_version 825726 (0.0009) [2023-12-27 11:21:10,703][20381] Updated weights for policy 1, policy_version 825736 (0.0008) [2023-12-27 11:21:10,762][20381] Updated weights for policy 1, policy_version 825746 (0.0009) [2023-12-27 11:21:11,171][20347] Updated weights for policy 0, policy_version 826343 (0.0008) [2023-12-27 11:21:11,235][20347] Updated weights for policy 0, policy_version 826353 (0.0008) [2023-12-27 11:21:11,291][20347] Updated weights for policy 0, policy_version 826363 (0.0008) [2023-12-27 11:21:11,563][20381] Updated weights for policy 1, policy_version 825756 (0.0008) [2023-12-27 11:21:11,630][20381] Updated weights for policy 1, policy_version 825766 (0.0008) [2023-12-27 11:21:11,694][20381] Updated weights for policy 1, policy_version 825776 (0.0006) [2023-12-27 11:21:12,141][20347] Updated weights for policy 0, policy_version 826373 (0.0007) [2023-12-27 11:21:12,199][20347] Updated weights for policy 0, policy_version 826383 (0.0006) [2023-12-27 11:21:12,255][20347] Updated weights for policy 0, policy_version 826393 (0.0008) [2023-12-27 11:21:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 423010304. Throughput: 0: 9799.0, 1: 9653.8. Samples: 423000796. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:12,363][19170] Avg episode reward: [(0, '862.920'), (1, '1000.000')] [2023-12-27 11:21:12,483][20381] Updated weights for policy 1, policy_version 825786 (0.0008) [2023-12-27 11:21:12,542][20381] Updated weights for policy 1, policy_version 825796 (0.0007) [2023-12-27 11:21:12,599][20381] Updated weights for policy 1, policy_version 825806 (0.0007) [2023-12-27 11:21:12,650][20381] Updated weights for policy 1, policy_version 825816 (0.0007) [2023-12-27 11:21:13,008][20347] Updated weights for policy 0, policy_version 826403 (0.0008) [2023-12-27 11:21:13,066][20347] Updated weights for policy 0, policy_version 826413 (0.0008) [2023-12-27 11:21:13,129][20347] Updated weights for policy 0, policy_version 826423 (0.0007) [2023-12-27 11:21:13,333][20381] Updated weights for policy 1, policy_version 825826 (0.0005) [2023-12-27 11:21:13,398][20381] Updated weights for policy 1, policy_version 825836 (0.0006) [2023-12-27 11:21:13,458][20381] Updated weights for policy 1, policy_version 825846 (0.0007) [2023-12-27 11:21:13,813][20347] Updated weights for policy 0, policy_version 826433 (0.0007) [2023-12-27 11:21:13,876][20347] Updated weights for policy 0, policy_version 826443 (0.0006) [2023-12-27 11:21:13,938][20347] Updated weights for policy 0, policy_version 826453 (0.0008) [2023-12-27 11:21:13,995][20347] Updated weights for policy 0, policy_version 826463 (0.0006) [2023-12-27 11:21:14,099][20381] Updated weights for policy 1, policy_version 825856 (0.0008) [2023-12-27 11:21:14,155][20381] Updated weights for policy 1, policy_version 825866 (0.0007) [2023-12-27 11:21:14,212][20381] Updated weights for policy 1, policy_version 825876 (0.0007) [2023-12-27 11:21:14,665][20347] Updated weights for policy 0, policy_version 826473 (0.0006) [2023-12-27 11:21:14,722][20347] Updated weights for policy 0, policy_version 826483 (0.0006) [2023-12-27 11:21:14,788][20347] Updated weights for policy 0, policy_version 826493 (0.0007) [2023-12-27 11:21:14,924][20381] Updated weights for policy 1, policy_version 825886 (0.0009) [2023-12-27 11:21:14,987][20381] Updated weights for policy 1, policy_version 825896 (0.0008) [2023-12-27 11:21:15,055][20381] Updated weights for policy 1, policy_version 825906 (0.0008) [2023-12-27 11:21:15,478][20347] Updated weights for policy 0, policy_version 826503 (0.0008) [2023-12-27 11:21:15,548][20347] Updated weights for policy 0, policy_version 826513 (0.0007) [2023-12-27 11:21:15,611][20347] Updated weights for policy 0, policy_version 826523 (0.0009) [2023-12-27 11:21:15,742][20381] Updated weights for policy 1, policy_version 825916 (0.0009) [2023-12-27 11:21:15,803][20381] Updated weights for policy 1, policy_version 825926 (0.0009) [2023-12-27 11:21:15,855][20381] Updated weights for policy 1, policy_version 825936 (0.0008) [2023-12-27 11:21:16,316][20347] Updated weights for policy 0, policy_version 826533 (0.0008) [2023-12-27 11:21:16,375][20347] Updated weights for policy 0, policy_version 826543 (0.0005) [2023-12-27 11:21:16,438][20347] Updated weights for policy 0, policy_version 826553 (0.0006) [2023-12-27 11:21:16,584][20381] Updated weights for policy 1, policy_version 825946 (0.0007) [2023-12-27 11:21:16,642][20381] Updated weights for policy 1, policy_version 825956 (0.0007) [2023-12-27 11:21:16,706][20381] Updated weights for policy 1, policy_version 825966 (0.0008) [2023-12-27 11:21:16,766][20381] Updated weights for policy 1, policy_version 825976 (0.0009) [2023-12-27 11:21:17,094][20347] Updated weights for policy 0, policy_version 826563 (0.0008) [2023-12-27 11:21:17,150][20347] Updated weights for policy 0, policy_version 826573 (0.0007) [2023-12-27 11:21:17,211][20347] Updated weights for policy 0, policy_version 826583 (0.0008) [2023-12-27 11:21:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 423108608. Throughput: 0: 9751.4, 1: 9690.6. Samples: 423117040. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:17,363][19170] Avg episode reward: [(0, '941.290'), (1, '1000.000')] [2023-12-27 11:21:17,489][20381] Updated weights for policy 1, policy_version 825986 (0.0006) [2023-12-27 11:21:17,548][20381] Updated weights for policy 1, policy_version 825996 (0.0007) [2023-12-27 11:21:17,611][20381] Updated weights for policy 1, policy_version 826006 (0.0007) [2023-12-27 11:21:17,884][20347] Updated weights for policy 0, policy_version 826593 (0.0008) [2023-12-27 11:21:17,946][20347] Updated weights for policy 0, policy_version 826603 (0.0008) [2023-12-27 11:21:18,009][20347] Updated weights for policy 0, policy_version 826613 (0.0008) [2023-12-27 11:21:18,063][20347] Updated weights for policy 0, policy_version 826623 (0.0008) [2023-12-27 11:21:18,303][20381] Updated weights for policy 1, policy_version 826016 (0.0008) [2023-12-27 11:21:18,367][20381] Updated weights for policy 1, policy_version 826026 (0.0007) [2023-12-27 11:21:18,428][20381] Updated weights for policy 1, policy_version 826036 (0.0008) [2023-12-27 11:21:18,797][20347] Updated weights for policy 0, policy_version 826633 (0.0006) [2023-12-27 11:21:18,863][20347] Updated weights for policy 0, policy_version 826643 (0.0007) [2023-12-27 11:21:18,931][20347] Updated weights for policy 0, policy_version 826653 (0.0006) [2023-12-27 11:21:19,155][20381] Updated weights for policy 1, policy_version 826046 (0.0007) [2023-12-27 11:21:19,220][20381] Updated weights for policy 1, policy_version 826056 (0.0007) [2023-12-27 11:21:19,285][20381] Updated weights for policy 1, policy_version 826066 (0.0009) [2023-12-27 11:21:19,582][20347] Updated weights for policy 0, policy_version 826663 (0.0007) [2023-12-27 11:21:19,646][20347] Updated weights for policy 0, policy_version 826673 (0.0009) [2023-12-27 11:21:19,714][20347] Updated weights for policy 0, policy_version 826683 (0.0009) [2023-12-27 11:21:20,111][20381] Updated weights for policy 1, policy_version 826076 (0.0007) [2023-12-27 11:21:20,177][20381] Updated weights for policy 1, policy_version 826086 (0.0007) [2023-12-27 11:21:20,243][20381] Updated weights for policy 1, policy_version 826096 (0.0008) [2023-12-27 11:21:20,417][20347] Updated weights for policy 0, policy_version 826693 (0.0007) [2023-12-27 11:21:20,480][20347] Updated weights for policy 0, policy_version 826703 (0.0007) [2023-12-27 11:21:20,542][20347] Updated weights for policy 0, policy_version 826713 (0.0008) [2023-12-27 11:21:20,954][20381] Updated weights for policy 1, policy_version 826106 (0.0008) [2023-12-27 11:21:21,016][20381] Updated weights for policy 1, policy_version 826116 (0.0008) [2023-12-27 11:21:21,080][20381] Updated weights for policy 1, policy_version 826126 (0.0008) [2023-12-27 11:21:21,132][20381] Updated weights for policy 1, policy_version 826136 (0.0007) [2023-12-27 11:21:21,275][20347] Updated weights for policy 0, policy_version 826723 (0.0009) [2023-12-27 11:21:21,338][20347] Updated weights for policy 0, policy_version 826733 (0.0006) [2023-12-27 11:21:21,415][20347] Updated weights for policy 0, policy_version 826743 (0.0008) [2023-12-27 11:21:21,916][20381] Updated weights for policy 1, policy_version 826146 (0.0009) [2023-12-27 11:21:21,980][20381] Updated weights for policy 1, policy_version 826156 (0.0009) [2023-12-27 11:21:22,043][20381] Updated weights for policy 1, policy_version 826166 (0.0009) [2023-12-27 11:21:22,091][20347] Updated weights for policy 0, policy_version 826753 (0.0007) [2023-12-27 11:21:22,153][20347] Updated weights for policy 0, policy_version 826763 (0.0006) [2023-12-27 11:21:22,216][20347] Updated weights for policy 0, policy_version 826773 (0.0006) [2023-12-27 11:21:22,278][20347] Updated weights for policy 0, policy_version 826783 (0.0007) [2023-12-27 11:21:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 423206912. Throughput: 0: 9718.8, 1: 9666.0. Samples: 423174684. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:22,363][19170] Avg episode reward: [(0, '921.990'), (1, '1000.000')] [2023-12-27 11:21:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000826168_211525632.pth... [2023-12-27 11:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000826784_211681280.pth... [2023-12-27 11:21:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000825048_211238912.pth [2023-12-27 11:21:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000825632_211386368.pth [2023-12-27 11:21:22,764][20381] Updated weights for policy 1, policy_version 826176 (0.0006) [2023-12-27 11:21:22,826][20381] Updated weights for policy 1, policy_version 826186 (0.0008) [2023-12-27 11:21:22,884][20381] Updated weights for policy 1, policy_version 826196 (0.0008) [2023-12-27 11:21:22,998][20347] Updated weights for policy 0, policy_version 826793 (0.0008) [2023-12-27 11:21:23,064][20347] Updated weights for policy 0, policy_version 826803 (0.0008) [2023-12-27 11:21:23,129][20347] Updated weights for policy 0, policy_version 826813 (0.0008) [2023-12-27 11:21:23,560][20381] Updated weights for policy 1, policy_version 826206 (0.0008) [2023-12-27 11:21:23,617][20381] Updated weights for policy 1, policy_version 826216 (0.0008) [2023-12-27 11:21:23,669][20381] Updated weights for policy 1, policy_version 826226 (0.0007) [2023-12-27 11:21:23,769][20291] KL-divergence is very high: 112.9856 [2023-12-27 11:21:23,802][20347] Updated weights for policy 0, policy_version 826823 (0.0009) [2023-12-27 11:21:23,809][20291] KL-divergence is very high: 130.4718 [2023-12-27 11:21:23,852][20347] Updated weights for policy 0, policy_version 826833 (0.0006) [2023-12-27 11:21:23,903][20347] Updated weights for policy 0, policy_version 826843 (0.0006) [2023-12-27 11:21:24,408][20381] Updated weights for policy 1, policy_version 826236 (0.0007) [2023-12-27 11:21:24,470][20381] Updated weights for policy 1, policy_version 826246 (0.0008) [2023-12-27 11:21:24,535][20381] Updated weights for policy 1, policy_version 826256 (0.0007) [2023-12-27 11:21:24,625][20347] Updated weights for policy 0, policy_version 826853 (0.0008) [2023-12-27 11:21:24,673][20347] Updated weights for policy 0, policy_version 826863 (0.0005) [2023-12-27 11:21:24,724][20347] Updated weights for policy 0, policy_version 826873 (0.0005) [2023-12-27 11:21:25,255][20381] Updated weights for policy 1, policy_version 826266 (0.0009) [2023-12-27 11:21:25,315][20381] Updated weights for policy 1, policy_version 826276 (0.0007) [2023-12-27 11:21:25,379][20381] Updated weights for policy 1, policy_version 826286 (0.0008) [2023-12-27 11:21:25,402][20347] Updated weights for policy 0, policy_version 826883 (0.0007) [2023-12-27 11:21:25,431][20381] Updated weights for policy 1, policy_version 826296 (0.0006) [2023-12-27 11:21:25,458][20347] Updated weights for policy 0, policy_version 826893 (0.0007) [2023-12-27 11:21:25,507][20347] Updated weights for policy 0, policy_version 826903 (0.0008) [2023-12-27 11:21:26,134][20381] Updated weights for policy 1, policy_version 826306 (0.0008) [2023-12-27 11:21:26,185][20381] Updated weights for policy 1, policy_version 826316 (0.0008) [2023-12-27 11:21:26,213][20347] Updated weights for policy 0, policy_version 826913 (0.0008) [2023-12-27 11:21:26,244][20381] Updated weights for policy 1, policy_version 826326 (0.0007) [2023-12-27 11:21:26,274][20347] Updated weights for policy 0, policy_version 826923 (0.0008) [2023-12-27 11:21:26,336][20347] Updated weights for policy 0, policy_version 826933 (0.0007) [2023-12-27 11:21:26,390][20347] Updated weights for policy 0, policy_version 826943 (0.0008) [2023-12-27 11:21:26,995][20381] Updated weights for policy 1, policy_version 826336 (0.0009) [2023-12-27 11:21:27,056][20381] Updated weights for policy 1, policy_version 826346 (0.0008) [2023-12-27 11:21:27,088][20347] Updated weights for policy 0, policy_version 826953 (0.0007) [2023-12-27 11:21:27,114][20381] Updated weights for policy 1, policy_version 826356 (0.0008) [2023-12-27 11:21:27,123][20291] KL-divergence is very high: 103.0448 [2023-12-27 11:21:27,148][20347] Updated weights for policy 0, policy_version 826963 (0.0006) [2023-12-27 11:21:27,208][20347] Updated weights for policy 0, policy_version 826973 (0.0008) [2023-12-27 11:21:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 423305216. Throughput: 0: 9728.1, 1: 9668.8. Samples: 423291572. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:27,363][19170] Avg episode reward: [(0, '814.430'), (1, '1000.000')] [2023-12-27 11:21:27,802][20381] Updated weights for policy 1, policy_version 826366 (0.0006) [2023-12-27 11:21:27,862][20381] Updated weights for policy 1, policy_version 826376 (0.0007) [2023-12-27 11:21:27,906][20347] Updated weights for policy 0, policy_version 826983 (0.0008) [2023-12-27 11:21:27,917][20291] KL-divergence is very high: 190.5574 [2023-12-27 11:21:27,919][20381] Updated weights for policy 1, policy_version 826386 (0.0009) [2023-12-27 11:21:27,959][20291] KL-divergence is very high: 101.7830 [2023-12-27 11:21:27,961][20347] Updated weights for policy 0, policy_version 826993 (0.0006) [2023-12-27 11:21:28,014][20347] Updated weights for policy 0, policy_version 827003 (0.0007) [2023-12-27 11:21:28,538][20381] Updated weights for policy 1, policy_version 826396 (0.0007) [2023-12-27 11:21:28,603][20381] Updated weights for policy 1, policy_version 826406 (0.0008) [2023-12-27 11:21:28,668][20381] Updated weights for policy 1, policy_version 826416 (0.0006) [2023-12-27 11:21:28,730][20347] Updated weights for policy 0, policy_version 827013 (0.0007) [2023-12-27 11:21:28,793][20347] Updated weights for policy 0, policy_version 827023 (0.0008) [2023-12-27 11:21:28,849][20347] Updated weights for policy 0, policy_version 827033 (0.0007) [2023-12-27 11:21:29,319][20381] Updated weights for policy 1, policy_version 826426 (0.0007) [2023-12-27 11:21:29,389][20381] Updated weights for policy 1, policy_version 826436 (0.0010) [2023-12-27 11:21:29,453][20381] Updated weights for policy 1, policy_version 826446 (0.0008) [2023-12-27 11:21:29,517][20381] Updated weights for policy 1, policy_version 826456 (0.0008) [2023-12-27 11:21:29,560][20347] Updated weights for policy 0, policy_version 827043 (0.0007) [2023-12-27 11:21:29,623][20347] Updated weights for policy 0, policy_version 827053 (0.0006) [2023-12-27 11:21:29,690][20347] Updated weights for policy 0, policy_version 827063 (0.0006) [2023-12-27 11:21:30,244][20381] Updated weights for policy 1, policy_version 826466 (0.0007) [2023-12-27 11:21:30,313][20381] Updated weights for policy 1, policy_version 826476 (0.0006) [2023-12-27 11:21:30,379][20381] Updated weights for policy 1, policy_version 826486 (0.0007) [2023-12-27 11:21:30,405][20347] Updated weights for policy 0, policy_version 827073 (0.0008) [2023-12-27 11:21:30,463][20347] Updated weights for policy 0, policy_version 827083 (0.0008) [2023-12-27 11:21:30,521][20347] Updated weights for policy 0, policy_version 827093 (0.0007) [2023-12-27 11:21:30,591][20347] Updated weights for policy 0, policy_version 827103 (0.0007) [2023-12-27 11:21:31,033][20381] Updated weights for policy 1, policy_version 826496 (0.0007) [2023-12-27 11:21:31,095][20381] Updated weights for policy 1, policy_version 826506 (0.0008) [2023-12-27 11:21:31,165][20381] Updated weights for policy 1, policy_version 826516 (0.0007) [2023-12-27 11:21:31,315][20347] Updated weights for policy 0, policy_version 827113 (0.0008) [2023-12-27 11:21:31,381][20347] Updated weights for policy 0, policy_version 827123 (0.0008) [2023-12-27 11:21:31,442][20347] Updated weights for policy 0, policy_version 827133 (0.0008) [2023-12-27 11:21:31,863][20381] Updated weights for policy 1, policy_version 826526 (0.0006) [2023-12-27 11:21:31,924][20381] Updated weights for policy 1, policy_version 826536 (0.0006) [2023-12-27 11:21:31,987][20381] Updated weights for policy 1, policy_version 826546 (0.0008) [2023-12-27 11:21:32,154][20347] Updated weights for policy 0, policy_version 827143 (0.0006) [2023-12-27 11:21:32,208][20347] Updated weights for policy 0, policy_version 827153 (0.0006) [2023-12-27 11:21:32,272][20347] Updated weights for policy 0, policy_version 827163 (0.0007) [2023-12-27 11:21:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 423403520. Throughput: 0: 9738.0, 1: 9694.5. Samples: 423409172. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:32,363][19170] Avg episode reward: [(0, '824.190'), (1, '1000.000')] [2023-12-27 11:21:32,647][20381] Updated weights for policy 1, policy_version 826556 (0.0007) [2023-12-27 11:21:32,703][20381] Updated weights for policy 1, policy_version 826566 (0.0008) [2023-12-27 11:21:32,758][20381] Updated weights for policy 1, policy_version 826576 (0.0008) [2023-12-27 11:21:32,943][20347] Updated weights for policy 0, policy_version 827173 (0.0008) [2023-12-27 11:21:32,998][20347] Updated weights for policy 0, policy_version 827183 (0.0006) [2023-12-27 11:21:33,051][20347] Updated weights for policy 0, policy_version 827193 (0.0008) [2023-12-27 11:21:33,429][20381] Updated weights for policy 1, policy_version 826586 (0.0008) [2023-12-27 11:21:33,488][20381] Updated weights for policy 1, policy_version 826596 (0.0007) [2023-12-27 11:21:33,547][20381] Updated weights for policy 1, policy_version 826606 (0.0008) [2023-12-27 11:21:33,597][20381] Updated weights for policy 1, policy_version 826616 (0.0008) [2023-12-27 11:21:33,747][20347] Updated weights for policy 0, policy_version 827203 (0.0008) [2023-12-27 11:21:33,800][20347] Updated weights for policy 0, policy_version 827213 (0.0008) [2023-12-27 11:21:33,856][20347] Updated weights for policy 0, policy_version 827223 (0.0007) [2023-12-27 11:21:34,320][20381] Updated weights for policy 1, policy_version 826626 (0.0008) [2023-12-27 11:21:34,379][20381] Updated weights for policy 1, policy_version 826636 (0.0007) [2023-12-27 11:21:34,442][20381] Updated weights for policy 1, policy_version 826646 (0.0007) [2023-12-27 11:21:34,627][20347] Updated weights for policy 0, policy_version 827233 (0.0007) [2023-12-27 11:21:34,694][20347] Updated weights for policy 0, policy_version 827243 (0.0008) [2023-12-27 11:21:34,751][20347] Updated weights for policy 0, policy_version 827253 (0.0007) [2023-12-27 11:21:34,805][20347] Updated weights for policy 0, policy_version 827263 (0.0008) [2023-12-27 11:21:35,123][20381] Updated weights for policy 1, policy_version 826656 (0.0006) [2023-12-27 11:21:35,177][20381] Updated weights for policy 1, policy_version 826666 (0.0007) [2023-12-27 11:21:35,241][20381] Updated weights for policy 1, policy_version 826676 (0.0008) [2023-12-27 11:21:35,519][20347] Updated weights for policy 0, policy_version 827273 (0.0007) [2023-12-27 11:21:35,572][20347] Updated weights for policy 0, policy_version 827283 (0.0007) [2023-12-27 11:21:35,622][20347] Updated weights for policy 0, policy_version 827293 (0.0008) [2023-12-27 11:21:35,939][20381] Updated weights for policy 1, policy_version 826686 (0.0008) [2023-12-27 11:21:35,997][20381] Updated weights for policy 1, policy_version 826696 (0.0006) [2023-12-27 11:21:36,060][20381] Updated weights for policy 1, policy_version 826706 (0.0006) [2023-12-27 11:21:36,332][20347] Updated weights for policy 0, policy_version 827303 (0.0008) [2023-12-27 11:21:36,398][20347] Updated weights for policy 0, policy_version 827313 (0.0008) [2023-12-27 11:21:36,467][20347] Updated weights for policy 0, policy_version 827323 (0.0008) [2023-12-27 11:21:36,767][20381] Updated weights for policy 1, policy_version 826716 (0.0007) [2023-12-27 11:21:36,823][20381] Updated weights for policy 1, policy_version 826726 (0.0008) [2023-12-27 11:21:36,885][20381] Updated weights for policy 1, policy_version 826736 (0.0008) [2023-12-27 11:21:37,140][20347] Updated weights for policy 0, policy_version 827333 (0.0008) [2023-12-27 11:21:37,190][20347] Updated weights for policy 0, policy_version 827343 (0.0008) [2023-12-27 11:21:37,250][20347] Updated weights for policy 0, policy_version 827353 (0.0009) [2023-12-27 11:21:37,363][19170] Fps is (10 sec: 19660.2, 60 sec: 19524.1, 300 sec: 19494.2). Total num frames: 423501824. Throughput: 0: 9755.8, 1: 9715.6. Samples: 423469180. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:37,364][19170] Avg episode reward: [(0, '873.160'), (1, '1000.000')] [2023-12-27 11:21:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000826744_211673088.pth... [2023-12-27 11:21:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000827360_211828736.pth... [2023-12-27 11:21:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000825624_211386368.pth [2023-12-27 11:21:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000826208_211533824.pth [2023-12-27 11:21:37,571][20381] Updated weights for policy 1, policy_version 826746 (0.0008) [2023-12-27 11:21:37,631][20381] Updated weights for policy 1, policy_version 826756 (0.0008) [2023-12-27 11:21:37,693][20381] Updated weights for policy 1, policy_version 826766 (0.0007) [2023-12-27 11:21:37,758][20381] Updated weights for policy 1, policy_version 826776 (0.0008) [2023-12-27 11:21:37,945][20347] Updated weights for policy 0, policy_version 827363 (0.0007) [2023-12-27 11:21:38,014][20347] Updated weights for policy 0, policy_version 827373 (0.0006) [2023-12-27 11:21:38,078][20347] Updated weights for policy 0, policy_version 827383 (0.0007) [2023-12-27 11:21:38,512][20381] Updated weights for policy 1, policy_version 826786 (0.0009) [2023-12-27 11:21:38,573][20381] Updated weights for policy 1, policy_version 826796 (0.0008) [2023-12-27 11:21:38,625][20381] Updated weights for policy 1, policy_version 826806 (0.0008) [2023-12-27 11:21:38,731][20347] Updated weights for policy 0, policy_version 827393 (0.0009) [2023-12-27 11:21:38,793][20347] Updated weights for policy 0, policy_version 827403 (0.0006) [2023-12-27 11:21:38,860][20347] Updated weights for policy 0, policy_version 827413 (0.0006) [2023-12-27 11:21:38,923][20347] Updated weights for policy 0, policy_version 827423 (0.0006) [2023-12-27 11:21:39,387][20381] Updated weights for policy 1, policy_version 826816 (0.0008) [2023-12-27 11:21:39,445][20381] Updated weights for policy 1, policy_version 826826 (0.0008) [2023-12-27 11:21:39,500][20381] Updated weights for policy 1, policy_version 826836 (0.0008) [2023-12-27 11:21:39,563][20347] Updated weights for policy 0, policy_version 827433 (0.0008) [2023-12-27 11:21:39,626][20347] Updated weights for policy 0, policy_version 827443 (0.0010) [2023-12-27 11:21:39,688][20347] Updated weights for policy 0, policy_version 827453 (0.0008) [2023-12-27 11:21:40,283][20381] Updated weights for policy 1, policy_version 826846 (0.0008) [2023-12-27 11:21:40,344][20381] Updated weights for policy 1, policy_version 826856 (0.0007) [2023-12-27 11:21:40,400][20347] Updated weights for policy 0, policy_version 827463 (0.0009) [2023-12-27 11:21:40,407][20381] Updated weights for policy 1, policy_version 826866 (0.0008) [2023-12-27 11:21:40,460][20347] Updated weights for policy 0, policy_version 827473 (0.0010) [2023-12-27 11:21:40,520][20347] Updated weights for policy 0, policy_version 827483 (0.0010) [2023-12-27 11:21:41,148][20381] Updated weights for policy 1, policy_version 826876 (0.0007) [2023-12-27 11:21:41,175][20347] Updated weights for policy 0, policy_version 827493 (0.0011) [2023-12-27 11:21:41,213][20381] Updated weights for policy 1, policy_version 826886 (0.0008) [2023-12-27 11:21:41,235][20347] Updated weights for policy 0, policy_version 827503 (0.0008) [2023-12-27 11:21:41,280][20381] Updated weights for policy 1, policy_version 826896 (0.0008) [2023-12-27 11:21:41,302][20347] Updated weights for policy 0, policy_version 827513 (0.0008) [2023-12-27 11:21:41,998][20381] Updated weights for policy 1, policy_version 826906 (0.0007) [2023-12-27 11:21:42,061][20381] Updated weights for policy 1, policy_version 826916 (0.0007) [2023-12-27 11:21:42,110][20347] Updated weights for policy 0, policy_version 827523 (0.0008) [2023-12-27 11:21:42,122][20381] Updated weights for policy 1, policy_version 826926 (0.0008) [2023-12-27 11:21:42,171][20347] Updated weights for policy 0, policy_version 827533 (0.0007) [2023-12-27 11:21:42,187][20381] Updated weights for policy 1, policy_version 826936 (0.0006) [2023-12-27 11:21:42,233][20347] Updated weights for policy 0, policy_version 827543 (0.0007) [2023-12-27 11:21:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.4, 300 sec: 19466.4). Total num frames: 423600128. Throughput: 0: 9750.0, 1: 9699.0. Samples: 423586672. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:42,363][19170] Avg episode reward: [(0, '941.230'), (1, '1000.000')] [2023-12-27 11:21:42,870][20381] Updated weights for policy 1, policy_version 826946 (0.0008) [2023-12-27 11:21:42,919][20347] Updated weights for policy 0, policy_version 827553 (0.0007) [2023-12-27 11:21:42,930][20381] Updated weights for policy 1, policy_version 826956 (0.0006) [2023-12-27 11:21:42,983][20347] Updated weights for policy 0, policy_version 827563 (0.0008) [2023-12-27 11:21:42,989][20381] Updated weights for policy 1, policy_version 826966 (0.0011) [2023-12-27 11:21:43,035][20347] Updated weights for policy 0, policy_version 827573 (0.0007) [2023-12-27 11:21:43,083][20347] Updated weights for policy 0, policy_version 827583 (0.0008) [2023-12-27 11:21:43,682][20381] Updated weights for policy 1, policy_version 826976 (0.0009) [2023-12-27 11:21:43,742][20381] Updated weights for policy 1, policy_version 826986 (0.0006) [2023-12-27 11:21:43,786][20347] Updated weights for policy 0, policy_version 827593 (0.0008) [2023-12-27 11:21:43,801][20381] Updated weights for policy 1, policy_version 826996 (0.0005) [2023-12-27 11:21:43,838][20347] Updated weights for policy 0, policy_version 827603 (0.0007) [2023-12-27 11:21:43,887][20347] Updated weights for policy 0, policy_version 827613 (0.0009) [2023-12-27 11:21:44,472][20381] Updated weights for policy 1, policy_version 827006 (0.0011) [2023-12-27 11:21:44,527][20381] Updated weights for policy 1, policy_version 827016 (0.0011) [2023-12-27 11:21:44,587][20381] Updated weights for policy 1, policy_version 827026 (0.0010) [2023-12-27 11:21:44,621][20347] Updated weights for policy 0, policy_version 827623 (0.0007) [2023-12-27 11:21:44,670][20347] Updated weights for policy 0, policy_version 827633 (0.0008) [2023-12-27 11:21:44,731][20347] Updated weights for policy 0, policy_version 827643 (0.0007) [2023-12-27 11:21:45,361][20381] Updated weights for policy 1, policy_version 827036 (0.0010) [2023-12-27 11:21:45,416][20381] Updated weights for policy 1, policy_version 827046 (0.0008) [2023-12-27 11:21:45,448][20347] Updated weights for policy 0, policy_version 827653 (0.0006) [2023-12-27 11:21:45,480][20381] Updated weights for policy 1, policy_version 827056 (0.0009) [2023-12-27 11:21:45,506][20347] Updated weights for policy 0, policy_version 827663 (0.0005) [2023-12-27 11:21:45,562][20347] Updated weights for policy 0, policy_version 827673 (0.0006) [2023-12-27 11:21:46,209][20347] Updated weights for policy 0, policy_version 827683 (0.0006) [2023-12-27 11:21:46,269][20381] Updated weights for policy 1, policy_version 827066 (0.0008) [2023-12-27 11:21:46,270][20347] Updated weights for policy 0, policy_version 827693 (0.0006) [2023-12-27 11:21:46,327][20347] Updated weights for policy 0, policy_version 827703 (0.0006) [2023-12-27 11:21:46,332][20381] Updated weights for policy 1, policy_version 827076 (0.0010) [2023-12-27 11:21:46,383][20381] Updated weights for policy 1, policy_version 827086 (0.0011) [2023-12-27 11:21:46,445][20381] Updated weights for policy 1, policy_version 827096 (0.0008) [2023-12-27 11:21:46,958][20347] Updated weights for policy 0, policy_version 827713 (0.0006) [2023-12-27 11:21:47,015][20347] Updated weights for policy 0, policy_version 827723 (0.0009) [2023-12-27 11:21:47,072][20347] Updated weights for policy 0, policy_version 827733 (0.0010) [2023-12-27 11:21:47,108][20381] Updated weights for policy 1, policy_version 827106 (0.0011) [2023-12-27 11:21:47,133][20347] Updated weights for policy 0, policy_version 827743 (0.0010) [2023-12-27 11:21:47,169][20381] Updated weights for policy 1, policy_version 827116 (0.0011) [2023-12-27 11:21:47,231][20381] Updated weights for policy 1, policy_version 827126 (0.0009) [2023-12-27 11:21:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19494.2). Total num frames: 423698432. Throughput: 0: 9836.5, 1: 9704.9. Samples: 423704104. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:47,363][19170] Avg episode reward: [(0, '921.660'), (1, '1000.000')] [2023-12-27 11:21:47,772][20347] Updated weights for policy 0, policy_version 827753 (0.0011) [2023-12-27 11:21:47,834][20347] Updated weights for policy 0, policy_version 827763 (0.0011) [2023-12-27 11:21:47,889][20347] Updated weights for policy 0, policy_version 827773 (0.0010) [2023-12-27 11:21:47,902][20381] Updated weights for policy 1, policy_version 827136 (0.0011) [2023-12-27 11:21:47,953][20381] Updated weights for policy 1, policy_version 827146 (0.0010) [2023-12-27 11:21:48,016][20381] Updated weights for policy 1, policy_version 827156 (0.0011) [2023-12-27 11:21:48,590][20347] Updated weights for policy 0, policy_version 827783 (0.0011) [2023-12-27 11:21:48,652][20347] Updated weights for policy 0, policy_version 827793 (0.0011) [2023-12-27 11:21:48,713][20347] Updated weights for policy 0, policy_version 827803 (0.0010) [2023-12-27 11:21:48,715][20381] Updated weights for policy 1, policy_version 827166 (0.0011) [2023-12-27 11:21:48,774][20381] Updated weights for policy 1, policy_version 827176 (0.0008) [2023-12-27 11:21:48,837][20381] Updated weights for policy 1, policy_version 827186 (0.0007) [2023-12-27 11:21:49,402][20347] Updated weights for policy 0, policy_version 827813 (0.0011) [2023-12-27 11:21:49,469][20347] Updated weights for policy 0, policy_version 827823 (0.0010) [2023-12-27 11:21:49,527][20347] Updated weights for policy 0, policy_version 827833 (0.0011) [2023-12-27 11:21:49,565][20381] Updated weights for policy 1, policy_version 827196 (0.0008) [2023-12-27 11:21:49,628][20381] Updated weights for policy 1, policy_version 827206 (0.0008) [2023-12-27 11:21:49,684][20381] Updated weights for policy 1, policy_version 827216 (0.0009) [2023-12-27 11:21:50,293][20347] Updated weights for policy 0, policy_version 827843 (0.0010) [2023-12-27 11:21:50,347][20347] Updated weights for policy 0, policy_version 827853 (0.0008) [2023-12-27 11:21:50,411][20347] Updated weights for policy 0, policy_version 827863 (0.0008) [2023-12-27 11:21:50,452][20381] Updated weights for policy 1, policy_version 827226 (0.0010) [2023-12-27 11:21:50,506][20381] Updated weights for policy 1, policy_version 827236 (0.0010) [2023-12-27 11:21:50,567][20381] Updated weights for policy 1, policy_version 827246 (0.0010) [2023-12-27 11:21:50,622][20381] Updated weights for policy 1, policy_version 827256 (0.0007) [2023-12-27 11:21:51,147][20347] Updated weights for policy 0, policy_version 827873 (0.0008) [2023-12-27 11:21:51,200][20347] Updated weights for policy 0, policy_version 827883 (0.0007) [2023-12-27 11:21:51,256][20347] Updated weights for policy 0, policy_version 827893 (0.0008) [2023-12-27 11:21:51,316][20347] Updated weights for policy 0, policy_version 827903 (0.0008) [2023-12-27 11:21:51,353][20381] Updated weights for policy 1, policy_version 827266 (0.0011) [2023-12-27 11:21:51,423][20381] Updated weights for policy 1, policy_version 827276 (0.0008) [2023-12-27 11:21:51,488][20381] Updated weights for policy 1, policy_version 827286 (0.0008) [2023-12-27 11:21:52,083][20347] Updated weights for policy 0, policy_version 827913 (0.0008) [2023-12-27 11:21:52,144][20347] Updated weights for policy 0, policy_version 827923 (0.0007) [2023-12-27 11:21:52,209][20347] Updated weights for policy 0, policy_version 827933 (0.0007) [2023-12-27 11:21:52,232][20381] Updated weights for policy 1, policy_version 827296 (0.0008) [2023-12-27 11:21:52,295][20381] Updated weights for policy 1, policy_version 827306 (0.0008) [2023-12-27 11:21:52,359][20381] Updated weights for policy 1, policy_version 827316 (0.0008) [2023-12-27 11:21:52,362][19170] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 423788544. Throughput: 0: 9834.6, 1: 9692.6. Samples: 423762992. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:52,363][19170] Avg episode reward: [(0, '872.680'), (1, '1000.000')] [2023-12-27 11:21:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000827936_211976192.pth... [2023-12-27 11:21:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000826784_211681280.pth [2023-12-27 11:21:52,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000827320_211820544.pth... [2023-12-27 11:21:52,388][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000826168_211525632.pth [2023-12-27 11:21:52,868][20347] Updated weights for policy 0, policy_version 827943 (0.0009) [2023-12-27 11:21:52,924][20347] Updated weights for policy 0, policy_version 827953 (0.0010) [2023-12-27 11:21:52,981][20347] Updated weights for policy 0, policy_version 827963 (0.0008) [2023-12-27 11:21:53,069][20381] Updated weights for policy 1, policy_version 827326 (0.0006) [2023-12-27 11:21:53,126][20381] Updated weights for policy 1, policy_version 827336 (0.0006) [2023-12-27 11:21:53,183][20381] Updated weights for policy 1, policy_version 827346 (0.0005) [2023-12-27 11:21:53,666][20347] Updated weights for policy 0, policy_version 827973 (0.0008) [2023-12-27 11:21:53,716][20347] Updated weights for policy 0, policy_version 827983 (0.0010) [2023-12-27 11:21:53,765][20347] Updated weights for policy 0, policy_version 827993 (0.0010) [2023-12-27 11:21:53,828][20381] Updated weights for policy 1, policy_version 827356 (0.0007) [2023-12-27 11:21:53,886][20381] Updated weights for policy 1, policy_version 827366 (0.0007) [2023-12-27 11:21:53,939][20381] Updated weights for policy 1, policy_version 827376 (0.0007) [2023-12-27 11:21:54,494][20347] Updated weights for policy 0, policy_version 828003 (0.0010) [2023-12-27 11:21:54,548][20347] Updated weights for policy 0, policy_version 828013 (0.0006) [2023-12-27 11:21:54,601][20347] Updated weights for policy 0, policy_version 828023 (0.0007) [2023-12-27 11:21:54,664][20381] Updated weights for policy 1, policy_version 827386 (0.0007) [2023-12-27 11:21:54,727][20381] Updated weights for policy 1, policy_version 827396 (0.0006) [2023-12-27 11:21:54,783][20381] Updated weights for policy 1, policy_version 827406 (0.0007) [2023-12-27 11:21:54,833][20381] Updated weights for policy 1, policy_version 827416 (0.0008) [2023-12-27 11:21:55,278][20347] Updated weights for policy 0, policy_version 828033 (0.0008) [2023-12-27 11:21:55,327][20347] Updated weights for policy 0, policy_version 828043 (0.0007) [2023-12-27 11:21:55,386][20347] Updated weights for policy 0, policy_version 828053 (0.0007) [2023-12-27 11:21:55,455][20347] Updated weights for policy 0, policy_version 828063 (0.0006) [2023-12-27 11:21:55,539][20381] Updated weights for policy 1, policy_version 827426 (0.0007) [2023-12-27 11:21:55,595][20381] Updated weights for policy 1, policy_version 827436 (0.0006) [2023-12-27 11:21:55,645][20381] Updated weights for policy 1, policy_version 827446 (0.0007) [2023-12-27 11:21:56,130][20347] Updated weights for policy 0, policy_version 828073 (0.0006) [2023-12-27 11:21:56,193][20347] Updated weights for policy 0, policy_version 828083 (0.0012) [2023-12-27 11:21:56,250][20347] Updated weights for policy 0, policy_version 828093 (0.0010) [2023-12-27 11:21:56,265][20381] Updated weights for policy 1, policy_version 827456 (0.0009) [2023-12-27 11:21:56,318][20381] Updated weights for policy 1, policy_version 827466 (0.0007) [2023-12-27 11:21:56,375][20381] Updated weights for policy 1, policy_version 827476 (0.0007) [2023-12-27 11:21:56,860][20347] Updated weights for policy 0, policy_version 828103 (0.0007) [2023-12-27 11:21:56,923][20347] Updated weights for policy 0, policy_version 828113 (0.0008) [2023-12-27 11:21:56,972][20347] Updated weights for policy 0, policy_version 828123 (0.0010) [2023-12-27 11:21:57,075][20381] Updated weights for policy 1, policy_version 827487 (0.0008) [2023-12-27 11:21:57,130][20381] Updated weights for policy 1, policy_version 827497 (0.0008) [2023-12-27 11:21:57,186][20381] Updated weights for policy 1, policy_version 827507 (0.0008) [2023-12-27 11:21:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 423895040. Throughput: 0: 9861.8, 1: 9710.3. Samples: 423881540. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:21:57,363][19170] Avg episode reward: [(0, '911.870'), (1, '1000.000')] [2023-12-27 11:21:57,636][20347] Updated weights for policy 0, policy_version 828133 (0.0010) [2023-12-27 11:21:57,692][20347] Updated weights for policy 0, policy_version 828143 (0.0008) [2023-12-27 11:21:57,751][20347] Updated weights for policy 0, policy_version 828153 (0.0010) [2023-12-27 11:21:57,876][20381] Updated weights for policy 1, policy_version 827517 (0.0007) [2023-12-27 11:21:57,939][20381] Updated weights for policy 1, policy_version 827527 (0.0010) [2023-12-27 11:21:57,991][20381] Updated weights for policy 1, policy_version 827537 (0.0010) [2023-12-27 11:21:58,437][20347] Updated weights for policy 0, policy_version 828163 (0.0010) [2023-12-27 11:21:58,499][20347] Updated weights for policy 0, policy_version 828173 (0.0008) [2023-12-27 11:21:58,563][20347] Updated weights for policy 0, policy_version 828183 (0.0008) [2023-12-27 11:21:58,834][20381] Updated weights for policy 1, policy_version 827547 (0.0007) [2023-12-27 11:21:58,906][20381] Updated weights for policy 1, policy_version 827557 (0.0010) [2023-12-27 11:21:58,977][20381] Updated weights for policy 1, policy_version 827567 (0.0014) [2023-12-27 11:21:59,386][20347] Updated weights for policy 0, policy_version 828193 (0.0007) [2023-12-27 11:21:59,453][20347] Updated weights for policy 0, policy_version 828203 (0.0008) [2023-12-27 11:21:59,517][20347] Updated weights for policy 0, policy_version 828213 (0.0008) [2023-12-27 11:21:59,570][20347] Updated weights for policy 0, policy_version 828223 (0.0008) [2023-12-27 11:21:59,759][20381] Updated weights for policy 1, policy_version 827577 (0.0009) [2023-12-27 11:21:59,807][20381] Updated weights for policy 1, policy_version 827587 (0.0007) [2023-12-27 11:21:59,866][20381] Updated weights for policy 1, policy_version 827597 (0.0008) [2023-12-27 11:21:59,929][20381] Updated weights for policy 1, policy_version 827607 (0.0008) [2023-12-27 11:22:00,261][20347] Updated weights for policy 0, policy_version 828233 (0.0007) [2023-12-27 11:22:00,318][20347] Updated weights for policy 0, policy_version 828243 (0.0007) [2023-12-27 11:22:00,380][20347] Updated weights for policy 0, policy_version 828253 (0.0006) [2023-12-27 11:22:00,729][20381] Updated weights for policy 1, policy_version 827617 (0.0007) [2023-12-27 11:22:00,788][20381] Updated weights for policy 1, policy_version 827627 (0.0007) [2023-12-27 11:22:00,847][20381] Updated weights for policy 1, policy_version 827637 (0.0007) [2023-12-27 11:22:01,090][20347] Updated weights for policy 0, policy_version 828263 (0.0007) [2023-12-27 11:22:01,148][20347] Updated weights for policy 0, policy_version 828273 (0.0005) [2023-12-27 11:22:01,213][20347] Updated weights for policy 0, policy_version 828283 (0.0009) [2023-12-27 11:22:01,643][20381] Updated weights for policy 1, policy_version 827647 (0.0008) [2023-12-27 11:22:01,703][20381] Updated weights for policy 1, policy_version 827657 (0.0008) [2023-12-27 11:22:01,769][20381] Updated weights for policy 1, policy_version 827667 (0.0008) [2023-12-27 11:22:01,934][20347] Updated weights for policy 0, policy_version 828293 (0.0009) [2023-12-27 11:22:01,995][20347] Updated weights for policy 0, policy_version 828303 (0.0007) [2023-12-27 11:22:02,013][20291] KL-divergence is very high: 138.1263 [2023-12-27 11:22:02,055][20347] Updated weights for policy 0, policy_version 828313 (0.0007) [2023-12-27 11:22:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 423985152. Throughput: 0: 9887.3, 1: 9635.9. Samples: 423995584. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:02,363][19170] Avg episode reward: [(0, '921.810'), (1, '1000.000')] [2023-12-27 11:22:02,557][20381] Updated weights for policy 1, policy_version 827677 (0.0008) [2023-12-27 11:22:02,609][20381] Updated weights for policy 1, policy_version 827687 (0.0007) [2023-12-27 11:22:02,661][20381] Updated weights for policy 1, policy_version 827697 (0.0005) [2023-12-27 11:22:02,756][20347] Updated weights for policy 0, policy_version 828323 (0.0006) [2023-12-27 11:22:02,812][20347] Updated weights for policy 0, policy_version 828333 (0.0006) [2023-12-27 11:22:02,866][20347] Updated weights for policy 0, policy_version 828343 (0.0006) [2023-12-27 11:22:03,375][20381] Updated weights for policy 1, policy_version 827707 (0.0006) [2023-12-27 11:22:03,432][20381] Updated weights for policy 1, policy_version 827717 (0.0008) [2023-12-27 11:22:03,485][20381] Updated weights for policy 1, policy_version 827727 (0.0007) [2023-12-27 11:22:03,490][20347] Updated weights for policy 0, policy_version 828353 (0.0008) [2023-12-27 11:22:03,543][20347] Updated weights for policy 0, policy_version 828363 (0.0007) [2023-12-27 11:22:03,596][20347] Updated weights for policy 0, policy_version 828373 (0.0007) [2023-12-27 11:22:03,646][20347] Updated weights for policy 0, policy_version 828383 (0.0007) [2023-12-27 11:22:04,214][20381] Updated weights for policy 1, policy_version 827737 (0.0007) [2023-12-27 11:22:04,282][20381] Updated weights for policy 1, policy_version 827747 (0.0008) [2023-12-27 11:22:04,344][20381] Updated weights for policy 1, policy_version 827757 (0.0007) [2023-12-27 11:22:04,354][20347] Updated weights for policy 0, policy_version 828393 (0.0007) [2023-12-27 11:22:04,405][20381] Updated weights for policy 1, policy_version 827767 (0.0008) [2023-12-27 11:22:04,411][20347] Updated weights for policy 0, policy_version 828403 (0.0006) [2023-12-27 11:22:04,473][20347] Updated weights for policy 0, policy_version 828413 (0.0007) [2023-12-27 11:22:05,134][20381] Updated weights for policy 1, policy_version 827777 (0.0008) [2023-12-27 11:22:05,176][20347] Updated weights for policy 0, policy_version 828423 (0.0009) [2023-12-27 11:22:05,197][20381] Updated weights for policy 1, policy_version 827787 (0.0007) [2023-12-27 11:22:05,236][20347] Updated weights for policy 0, policy_version 828433 (0.0007) [2023-12-27 11:22:05,253][20381] Updated weights for policy 1, policy_version 827797 (0.0009) [2023-12-27 11:22:05,295][20347] Updated weights for policy 0, policy_version 828443 (0.0008) [2023-12-27 11:22:05,962][20381] Updated weights for policy 1, policy_version 827807 (0.0008) [2023-12-27 11:22:05,975][20347] Updated weights for policy 0, policy_version 828453 (0.0007) [2023-12-27 11:22:06,023][20381] Updated weights for policy 1, policy_version 827817 (0.0006) [2023-12-27 11:22:06,032][20347] Updated weights for policy 0, policy_version 828463 (0.0008) [2023-12-27 11:22:06,077][20381] Updated weights for policy 1, policy_version 827827 (0.0008) [2023-12-27 11:22:06,088][20347] Updated weights for policy 0, policy_version 828473 (0.0008) [2023-12-27 11:22:06,808][20381] Updated weights for policy 1, policy_version 827837 (0.0007) [2023-12-27 11:22:06,842][20347] Updated weights for policy 0, policy_version 828483 (0.0008) [2023-12-27 11:22:06,871][20381] Updated weights for policy 1, policy_version 827847 (0.0007) [2023-12-27 11:22:06,903][20347] Updated weights for policy 0, policy_version 828493 (0.0007) [2023-12-27 11:22:06,935][20381] Updated weights for policy 1, policy_version 827857 (0.0008) [2023-12-27 11:22:06,966][20347] Updated weights for policy 0, policy_version 828503 (0.0007) [2023-12-27 11:22:07,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 424083456. Throughput: 0: 9916.4, 1: 9642.2. Samples: 424054824. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:07,363][19170] Avg episode reward: [(0, '853.310'), (1, '1000.000')] [2023-12-27 11:22:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000827864_211959808.pth... [2023-12-27 11:22:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000828512_212123648.pth... [2023-12-27 11:22:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000826744_211673088.pth [2023-12-27 11:22:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000827360_211828736.pth [2023-12-27 11:22:07,616][20347] Updated weights for policy 0, policy_version 828513 (0.0007) [2023-12-27 11:22:07,666][20381] Updated weights for policy 1, policy_version 827867 (0.0007) [2023-12-27 11:22:07,671][20347] Updated weights for policy 0, policy_version 828523 (0.0008) [2023-12-27 11:22:07,719][20381] Updated weights for policy 1, policy_version 827877 (0.0006) [2023-12-27 11:22:07,733][20347] Updated weights for policy 0, policy_version 828533 (0.0007) [2023-12-27 11:22:07,774][20381] Updated weights for policy 1, policy_version 827887 (0.0007) [2023-12-27 11:22:07,795][20347] Updated weights for policy 0, policy_version 828543 (0.0007) [2023-12-27 11:22:08,476][20381] Updated weights for policy 1, policy_version 827897 (0.0007) [2023-12-27 11:22:08,501][20347] Updated weights for policy 0, policy_version 828553 (0.0007) [2023-12-27 11:22:08,545][20381] Updated weights for policy 1, policy_version 827907 (0.0008) [2023-12-27 11:22:08,560][20347] Updated weights for policy 0, policy_version 828563 (0.0008) [2023-12-27 11:22:08,601][20381] Updated weights for policy 1, policy_version 827917 (0.0007) [2023-12-27 11:22:08,624][20347] Updated weights for policy 0, policy_version 828573 (0.0006) [2023-12-27 11:22:08,669][20381] Updated weights for policy 1, policy_version 827927 (0.0009) [2023-12-27 11:22:09,327][20347] Updated weights for policy 0, policy_version 828583 (0.0009) [2023-12-27 11:22:09,379][20381] Updated weights for policy 1, policy_version 827937 (0.0008) [2023-12-27 11:22:09,399][20347] Updated weights for policy 0, policy_version 828593 (0.0010) [2023-12-27 11:22:09,451][20381] Updated weights for policy 1, policy_version 827947 (0.0008) [2023-12-27 11:22:09,462][20347] Updated weights for policy 0, policy_version 828603 (0.0009) [2023-12-27 11:22:09,508][20381] Updated weights for policy 1, policy_version 827957 (0.0009) [2023-12-27 11:22:10,191][20381] Updated weights for policy 1, policy_version 827967 (0.0011) [2023-12-27 11:22:10,211][20347] Updated weights for policy 0, policy_version 828613 (0.0007) [2023-12-27 11:22:10,262][20381] Updated weights for policy 1, policy_version 827977 (0.0009) [2023-12-27 11:22:10,281][20347] Updated weights for policy 0, policy_version 828623 (0.0006) [2023-12-27 11:22:10,323][20381] Updated weights for policy 1, policy_version 827987 (0.0009) [2023-12-27 11:22:10,347][20347] Updated weights for policy 0, policy_version 828633 (0.0007) [2023-12-27 11:22:11,002][20347] Updated weights for policy 0, policy_version 828643 (0.0007) [2023-12-27 11:22:11,064][20347] Updated weights for policy 0, policy_version 828653 (0.0008) [2023-12-27 11:22:11,120][20347] Updated weights for policy 0, policy_version 828663 (0.0006) [2023-12-27 11:22:11,202][20381] Updated weights for policy 1, policy_version 827997 (0.0009) [2023-12-27 11:22:11,262][20381] Updated weights for policy 1, policy_version 828007 (0.0009) [2023-12-27 11:22:11,313][20381] Updated weights for policy 1, policy_version 828017 (0.0009) [2023-12-27 11:22:11,934][20347] Updated weights for policy 0, policy_version 828673 (0.0007) [2023-12-27 11:22:11,994][20347] Updated weights for policy 0, policy_version 828683 (0.0008) [2023-12-27 11:22:12,056][20347] Updated weights for policy 0, policy_version 828693 (0.0008) [2023-12-27 11:22:12,116][20347] Updated weights for policy 0, policy_version 828703 (0.0008) [2023-12-27 11:22:12,136][20381] Updated weights for policy 1, policy_version 828027 (0.0010) [2023-12-27 11:22:12,199][20381] Updated weights for policy 1, policy_version 828037 (0.0010) [2023-12-27 11:22:12,256][20381] Updated weights for policy 1, policy_version 828047 (0.0011) [2023-12-27 11:22:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 424181760. Throughput: 0: 9906.4, 1: 9610.1. Samples: 424169812. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:12,363][19170] Avg episode reward: [(0, '862.840'), (1, '1000.000')] [2023-12-27 11:22:12,894][20347] Updated weights for policy 0, policy_version 828713 (0.0007) [2023-12-27 11:22:12,951][20347] Updated weights for policy 0, policy_version 828723 (0.0008) [2023-12-27 11:22:12,983][20381] Updated weights for policy 1, policy_version 828057 (0.0011) [2023-12-27 11:22:13,011][20347] Updated weights for policy 0, policy_version 828733 (0.0008) [2023-12-27 11:22:13,042][20381] Updated weights for policy 1, policy_version 828067 (0.0006) [2023-12-27 11:22:13,100][20381] Updated weights for policy 1, policy_version 828077 (0.0007) [2023-12-27 11:22:13,156][20381] Updated weights for policy 1, policy_version 828087 (0.0008) [2023-12-27 11:22:13,744][20347] Updated weights for policy 0, policy_version 828743 (0.0007) [2023-12-27 11:22:13,762][20381] Updated weights for policy 1, policy_version 828097 (0.0007) [2023-12-27 11:22:13,794][20347] Updated weights for policy 0, policy_version 828753 (0.0007) [2023-12-27 11:22:13,819][20381] Updated weights for policy 1, policy_version 828107 (0.0006) [2023-12-27 11:22:13,848][20347] Updated weights for policy 0, policy_version 828763 (0.0006) [2023-12-27 11:22:13,880][20381] Updated weights for policy 1, policy_version 828117 (0.0006) [2023-12-27 11:22:14,523][20347] Updated weights for policy 0, policy_version 828773 (0.0006) [2023-12-27 11:22:14,529][20381] Updated weights for policy 1, policy_version 828127 (0.0007) [2023-12-27 11:22:14,583][20381] Updated weights for policy 1, policy_version 828137 (0.0005) [2023-12-27 11:22:14,584][20347] Updated weights for policy 0, policy_version 828783 (0.0008) [2023-12-27 11:22:14,636][20381] Updated weights for policy 1, policy_version 828147 (0.0007) [2023-12-27 11:22:14,647][20347] Updated weights for policy 0, policy_version 828793 (0.0008) [2023-12-27 11:22:15,392][20381] Updated weights for policy 1, policy_version 828157 (0.0006) [2023-12-27 11:22:15,403][20347] Updated weights for policy 0, policy_version 828803 (0.0008) [2023-12-27 11:22:15,448][20381] Updated weights for policy 1, policy_version 828167 (0.0007) [2023-12-27 11:22:15,459][20347] Updated weights for policy 0, policy_version 828813 (0.0008) [2023-12-27 11:22:15,509][20381] Updated weights for policy 1, policy_version 828177 (0.0008) [2023-12-27 11:22:15,516][20347] Updated weights for policy 0, policy_version 828823 (0.0010) [2023-12-27 11:22:16,220][20381] Updated weights for policy 1, policy_version 828187 (0.0008) [2023-12-27 11:22:16,221][20347] Updated weights for policy 0, policy_version 828833 (0.0011) [2023-12-27 11:22:16,267][20381] Updated weights for policy 1, policy_version 828197 (0.0007) [2023-12-27 11:22:16,280][20347] Updated weights for policy 0, policy_version 828843 (0.0010) [2023-12-27 11:22:16,318][20381] Updated weights for policy 1, policy_version 828207 (0.0007) [2023-12-27 11:22:16,334][20347] Updated weights for policy 0, policy_version 828853 (0.0008) [2023-12-27 11:22:16,387][20347] Updated weights for policy 0, policy_version 828863 (0.0008) [2023-12-27 11:22:17,054][20381] Updated weights for policy 1, policy_version 828217 (0.0008) [2023-12-27 11:22:17,081][20347] Updated weights for policy 0, policy_version 828873 (0.0007) [2023-12-27 11:22:17,119][20381] Updated weights for policy 1, policy_version 828227 (0.0008) [2023-12-27 11:22:17,145][20347] Updated weights for policy 0, policy_version 828883 (0.0006) [2023-12-27 11:22:17,177][20381] Updated weights for policy 1, policy_version 828237 (0.0009) [2023-12-27 11:22:17,204][20347] Updated weights for policy 0, policy_version 828893 (0.0006) [2023-12-27 11:22:17,231][20381] Updated weights for policy 1, policy_version 828247 (0.0007) [2023-12-27 11:22:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 424280064. Throughput: 0: 9870.8, 1: 9591.1. Samples: 424284956. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:17,363][19170] Avg episode reward: [(0, '882.700'), (1, '990.290')] [2023-12-27 11:22:17,876][20347] Updated weights for policy 0, policy_version 828903 (0.0007) [2023-12-27 11:22:17,930][20347] Updated weights for policy 0, policy_version 828913 (0.0007) [2023-12-27 11:22:17,950][20381] Updated weights for policy 1, policy_version 828257 (0.0008) [2023-12-27 11:22:17,987][20347] Updated weights for policy 0, policy_version 828923 (0.0006) [2023-12-27 11:22:18,010][20381] Updated weights for policy 1, policy_version 828267 (0.0006) [2023-12-27 11:22:18,072][20381] Updated weights for policy 1, policy_version 828277 (0.0008) [2023-12-27 11:22:18,679][20347] Updated weights for policy 0, policy_version 828933 (0.0008) [2023-12-27 11:22:18,740][20347] Updated weights for policy 0, policy_version 828943 (0.0011) [2023-12-27 11:22:18,802][20347] Updated weights for policy 0, policy_version 828953 (0.0011) [2023-12-27 11:22:18,819][20381] Updated weights for policy 1, policy_version 828287 (0.0007) [2023-12-27 11:22:18,878][20381] Updated weights for policy 1, policy_version 828297 (0.0007) [2023-12-27 11:22:18,937][20381] Updated weights for policy 1, policy_version 828307 (0.0009) [2023-12-27 11:22:19,543][20347] Updated weights for policy 0, policy_version 828963 (0.0010) [2023-12-27 11:22:19,607][20347] Updated weights for policy 0, policy_version 828973 (0.0008) [2023-12-27 11:22:19,652][20381] Updated weights for policy 1, policy_version 828317 (0.0007) [2023-12-27 11:22:19,668][20347] Updated weights for policy 0, policy_version 828983 (0.0008) [2023-12-27 11:22:19,712][20381] Updated weights for policy 1, policy_version 828327 (0.0008) [2023-12-27 11:22:19,775][20381] Updated weights for policy 1, policy_version 828337 (0.0008) [2023-12-27 11:22:20,485][20347] Updated weights for policy 0, policy_version 828993 (0.0008) [2023-12-27 11:22:20,545][20347] Updated weights for policy 0, policy_version 829003 (0.0008) [2023-12-27 11:22:20,591][20381] Updated weights for policy 1, policy_version 828347 (0.0007) [2023-12-27 11:22:20,619][20347] Updated weights for policy 0, policy_version 829013 (0.0008) [2023-12-27 11:22:20,657][20381] Updated weights for policy 1, policy_version 828357 (0.0008) [2023-12-27 11:22:20,684][20347] Updated weights for policy 0, policy_version 829023 (0.0008) [2023-12-27 11:22:20,733][20381] Updated weights for policy 1, policy_version 828367 (0.0008) [2023-12-27 11:22:21,474][20347] Updated weights for policy 0, policy_version 829033 (0.0007) [2023-12-27 11:22:21,495][20381] Updated weights for policy 1, policy_version 828377 (0.0008) [2023-12-27 11:22:21,537][20347] Updated weights for policy 0, policy_version 829043 (0.0008) [2023-12-27 11:22:21,555][20381] Updated weights for policy 1, policy_version 828387 (0.0007) [2023-12-27 11:22:21,601][20347] Updated weights for policy 0, policy_version 829053 (0.0008) [2023-12-27 11:22:21,623][20381] Updated weights for policy 1, policy_version 828397 (0.0008) [2023-12-27 11:22:21,681][20381] Updated weights for policy 1, policy_version 828407 (0.0008) [2023-12-27 11:22:22,358][20347] Updated weights for policy 0, policy_version 829063 (0.0010) [2023-12-27 11:22:22,362][19170] Fps is (10 sec: 18022.3, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 424361984. Throughput: 0: 9846.2, 1: 9545.1. Samples: 424341784. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:22,363][19170] Avg episode reward: [(0, '882.760'), (1, '990.290')] [2023-12-27 11:22:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000828408_212099072.pth... [2023-12-27 11:22:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000827320_211820544.pth [2023-12-27 11:22:22,420][20347] Updated weights for policy 0, policy_version 829073 (0.0008) [2023-12-27 11:22:22,441][20381] Updated weights for policy 1, policy_version 828417 (0.0007) [2023-12-27 11:22:22,483][20347] Updated weights for policy 0, policy_version 829083 (0.0008) [2023-12-27 11:22:22,501][20381] Updated weights for policy 1, policy_version 828427 (0.0007) [2023-12-27 11:22:22,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000829088_212271104.pth... [2023-12-27 11:22:22,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000827936_211976192.pth [2023-12-27 11:22:22,565][20381] Updated weights for policy 1, policy_version 828437 (0.0009) [2023-12-27 11:22:23,217][20347] Updated weights for policy 0, policy_version 829093 (0.0009) [2023-12-27 11:22:23,240][20381] Updated weights for policy 1, policy_version 828447 (0.0007) [2023-12-27 11:22:23,271][20347] Updated weights for policy 0, policy_version 829103 (0.0011) [2023-12-27 11:22:23,299][20381] Updated weights for policy 1, policy_version 828457 (0.0006) [2023-12-27 11:22:23,324][20347] Updated weights for policy 0, policy_version 829113 (0.0009) [2023-12-27 11:22:23,350][20381] Updated weights for policy 1, policy_version 828467 (0.0007) [2023-12-27 11:22:24,018][20347] Updated weights for policy 0, policy_version 829123 (0.0007) [2023-12-27 11:22:24,048][20381] Updated weights for policy 1, policy_version 828477 (0.0007) [2023-12-27 11:22:24,081][20347] Updated weights for policy 0, policy_version 829133 (0.0006) [2023-12-27 11:22:24,110][20381] Updated weights for policy 1, policy_version 828487 (0.0007) [2023-12-27 11:22:24,140][20347] Updated weights for policy 0, policy_version 829143 (0.0006) [2023-12-27 11:22:24,175][20381] Updated weights for policy 1, policy_version 828497 (0.0008) [2023-12-27 11:22:24,815][20347] Updated weights for policy 0, policy_version 829153 (0.0007) [2023-12-27 11:22:24,875][20347] Updated weights for policy 0, policy_version 829163 (0.0008) [2023-12-27 11:22:24,879][20381] Updated weights for policy 1, policy_version 828507 (0.0007) [2023-12-27 11:22:24,932][20347] Updated weights for policy 0, policy_version 829173 (0.0006) [2023-12-27 11:22:24,945][20381] Updated weights for policy 1, policy_version 828517 (0.0008) [2023-12-27 11:22:24,984][20347] Updated weights for policy 0, policy_version 829183 (0.0008) [2023-12-27 11:22:24,997][20381] Updated weights for policy 1, policy_version 828527 (0.0008) [2023-12-27 11:22:25,654][20347] Updated weights for policy 0, policy_version 829193 (0.0006) [2023-12-27 11:22:25,671][20381] Updated weights for policy 1, policy_version 828537 (0.0008) [2023-12-27 11:22:25,712][20347] Updated weights for policy 0, policy_version 829203 (0.0006) [2023-12-27 11:22:25,719][20381] Updated weights for policy 1, policy_version 828547 (0.0007) [2023-12-27 11:22:25,769][20347] Updated weights for policy 0, policy_version 829213 (0.0006) [2023-12-27 11:22:25,779][20381] Updated weights for policy 1, policy_version 828557 (0.0009) [2023-12-27 11:22:25,836][20381] Updated weights for policy 1, policy_version 828567 (0.0008) [2023-12-27 11:22:26,379][20347] Updated weights for policy 0, policy_version 829223 (0.0007) [2023-12-27 11:22:26,434][20347] Updated weights for policy 0, policy_version 829233 (0.0007) [2023-12-27 11:22:26,491][20347] Updated weights for policy 0, policy_version 829243 (0.0005) [2023-12-27 11:22:26,589][20381] Updated weights for policy 1, policy_version 828577 (0.0009) [2023-12-27 11:22:26,649][20381] Updated weights for policy 1, policy_version 828587 (0.0008) [2023-12-27 11:22:26,708][20381] Updated weights for policy 1, policy_version 828597 (0.0007) [2023-12-27 11:22:27,113][20347] Updated weights for policy 0, policy_version 829253 (0.0007) [2023-12-27 11:22:27,165][20347] Updated weights for policy 0, policy_version 829263 (0.0007) [2023-12-27 11:22:27,213][20347] Updated weights for policy 0, policy_version 829273 (0.0007) [2023-12-27 11:22:27,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 424468480. Throughput: 0: 9779.7, 1: 9543.1. Samples: 424456200. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:27,363][19170] Avg episode reward: [(0, '931.350'), (1, '1000.000')] [2023-12-27 11:22:27,500][20381] Updated weights for policy 1, policy_version 828607 (0.0008) [2023-12-27 11:22:27,562][20381] Updated weights for policy 1, policy_version 828617 (0.0007) [2023-12-27 11:22:27,630][20381] Updated weights for policy 1, policy_version 828627 (0.0007) [2023-12-27 11:22:27,916][20347] Updated weights for policy 0, policy_version 829283 (0.0007) [2023-12-27 11:22:27,976][20347] Updated weights for policy 0, policy_version 829293 (0.0007) [2023-12-27 11:22:28,032][20347] Updated weights for policy 0, policy_version 829303 (0.0007) [2023-12-27 11:22:28,362][20381] Updated weights for policy 1, policy_version 828637 (0.0007) [2023-12-27 11:22:28,421][20381] Updated weights for policy 1, policy_version 828647 (0.0008) [2023-12-27 11:22:28,482][20381] Updated weights for policy 1, policy_version 828657 (0.0008) [2023-12-27 11:22:28,732][20347] Updated weights for policy 0, policy_version 829313 (0.0007) [2023-12-27 11:22:28,798][20347] Updated weights for policy 0, policy_version 829323 (0.0007) [2023-12-27 11:22:28,856][20347] Updated weights for policy 0, policy_version 829333 (0.0008) [2023-12-27 11:22:28,916][20347] Updated weights for policy 0, policy_version 829343 (0.0007) [2023-12-27 11:22:29,182][20381] Updated weights for policy 1, policy_version 828667 (0.0008) [2023-12-27 11:22:29,245][20381] Updated weights for policy 1, policy_version 828677 (0.0009) [2023-12-27 11:22:29,293][20381] Updated weights for policy 1, policy_version 828687 (0.0006) [2023-12-27 11:22:29,582][20347] Updated weights for policy 0, policy_version 829353 (0.0008) [2023-12-27 11:22:29,642][20347] Updated weights for policy 0, policy_version 829363 (0.0007) [2023-12-27 11:22:29,704][20347] Updated weights for policy 0, policy_version 829373 (0.0008) [2023-12-27 11:22:30,039][20381] Updated weights for policy 1, policy_version 828697 (0.0008) [2023-12-27 11:22:30,098][20381] Updated weights for policy 1, policy_version 828707 (0.0007) [2023-12-27 11:22:30,157][20381] Updated weights for policy 1, policy_version 828717 (0.0006) [2023-12-27 11:22:30,213][20381] Updated weights for policy 1, policy_version 828727 (0.0008) [2023-12-27 11:22:30,467][20347] Updated weights for policy 0, policy_version 829383 (0.0008) [2023-12-27 11:22:30,526][20347] Updated weights for policy 0, policy_version 829393 (0.0008) [2023-12-27 11:22:30,587][20347] Updated weights for policy 0, policy_version 829403 (0.0008) [2023-12-27 11:22:30,883][20381] Updated weights for policy 1, policy_version 828737 (0.0008) [2023-12-27 11:22:30,940][20381] Updated weights for policy 1, policy_version 828747 (0.0008) [2023-12-27 11:22:30,990][20381] Updated weights for policy 1, policy_version 828757 (0.0008) [2023-12-27 11:22:31,300][20347] Updated weights for policy 0, policy_version 829413 (0.0007) [2023-12-27 11:22:31,367][20347] Updated weights for policy 0, policy_version 829423 (0.0008) [2023-12-27 11:22:31,435][20347] Updated weights for policy 0, policy_version 829433 (0.0009) [2023-12-27 11:22:31,825][20381] Updated weights for policy 1, policy_version 828767 (0.0009) [2023-12-27 11:22:31,886][20381] Updated weights for policy 1, policy_version 828777 (0.0008) [2023-12-27 11:22:31,947][20381] Updated weights for policy 1, policy_version 828787 (0.0007) [2023-12-27 11:22:32,203][20347] Updated weights for policy 0, policy_version 829443 (0.0007) [2023-12-27 11:22:32,267][20347] Updated weights for policy 0, policy_version 829453 (0.0007) [2023-12-27 11:22:32,328][20347] Updated weights for policy 0, policy_version 829463 (0.0007) [2023-12-27 11:22:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 424558592. Throughput: 0: 9773.3, 1: 9554.8. Samples: 424573872. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 11:22:32,364][19170] Avg episode reward: [(0, '941.310'), (1, '1000.000')] [2023-12-27 11:22:32,683][20381] Updated weights for policy 1, policy_version 828797 (0.0008) [2023-12-27 11:22:32,740][20381] Updated weights for policy 1, policy_version 828807 (0.0009) [2023-12-27 11:22:32,805][20381] Updated weights for policy 1, policy_version 828817 (0.0007) [2023-12-27 11:22:33,055][20347] Updated weights for policy 0, policy_version 829473 (0.0008) [2023-12-27 11:22:33,111][20347] Updated weights for policy 0, policy_version 829483 (0.0006) [2023-12-27 11:22:33,168][20347] Updated weights for policy 0, policy_version 829493 (0.0006) [2023-12-27 11:22:33,222][20347] Updated weights for policy 0, policy_version 829503 (0.0008) [2023-12-27 11:22:33,478][20381] Updated weights for policy 1, policy_version 828827 (0.0009) [2023-12-27 11:22:33,532][20381] Updated weights for policy 1, policy_version 828837 (0.0008) [2023-12-27 11:22:33,585][20381] Updated weights for policy 1, policy_version 828847 (0.0005) [2023-12-27 11:22:33,915][20347] Updated weights for policy 0, policy_version 829513 (0.0010) [2023-12-27 11:22:33,974][20347] Updated weights for policy 0, policy_version 829523 (0.0010) [2023-12-27 11:22:34,031][20347] Updated weights for policy 0, policy_version 829533 (0.0010) [2023-12-27 11:22:34,266][20381] Updated weights for policy 1, policy_version 828857 (0.0006) [2023-12-27 11:22:34,333][20381] Updated weights for policy 1, policy_version 828867 (0.0007) [2023-12-27 11:22:34,389][20381] Updated weights for policy 1, policy_version 828877 (0.0006) [2023-12-27 11:22:34,448][20381] Updated weights for policy 1, policy_version 828887 (0.0008) [2023-12-27 11:22:34,759][20347] Updated weights for policy 0, policy_version 829543 (0.0007) [2023-12-27 11:22:34,820][20347] Updated weights for policy 0, policy_version 829553 (0.0007) [2023-12-27 11:22:34,884][20347] Updated weights for policy 0, policy_version 829563 (0.0006) [2023-12-27 11:22:35,139][20381] Updated weights for policy 1, policy_version 828897 (0.0007) [2023-12-27 11:22:35,200][20381] Updated weights for policy 1, policy_version 828907 (0.0008) [2023-12-27 11:22:35,256][20381] Updated weights for policy 1, policy_version 828917 (0.0007) [2023-12-27 11:22:35,540][20347] Updated weights for policy 0, policy_version 829573 (0.0006) [2023-12-27 11:22:35,599][20347] Updated weights for policy 0, policy_version 829583 (0.0007) [2023-12-27 11:22:35,658][20347] Updated weights for policy 0, policy_version 829593 (0.0007) [2023-12-27 11:22:35,899][20381] Updated weights for policy 1, policy_version 828927 (0.0007) [2023-12-27 11:22:35,953][20381] Updated weights for policy 1, policy_version 828937 (0.0008) [2023-12-27 11:22:36,021][20381] Updated weights for policy 1, policy_version 828947 (0.0007) [2023-12-27 11:22:36,322][20347] Updated weights for policy 0, policy_version 829603 (0.0007) [2023-12-27 11:22:36,381][20347] Updated weights for policy 0, policy_version 829613 (0.0010) [2023-12-27 11:22:36,442][20347] Updated weights for policy 0, policy_version 829623 (0.0008) [2023-12-27 11:22:36,715][20381] Updated weights for policy 1, policy_version 828957 (0.0008) [2023-12-27 11:22:36,763][20381] Updated weights for policy 1, policy_version 828967 (0.0007) [2023-12-27 11:22:36,821][20381] Updated weights for policy 1, policy_version 828977 (0.0007) [2023-12-27 11:22:37,158][20347] Updated weights for policy 0, policy_version 829633 (0.0008) [2023-12-27 11:22:37,217][20347] Updated weights for policy 0, policy_version 829643 (0.0011) [2023-12-27 11:22:37,277][20347] Updated weights for policy 0, policy_version 829653 (0.0011) [2023-12-27 11:22:37,335][20347] Updated weights for policy 0, policy_version 829663 (0.0010) [2023-12-27 11:22:37,363][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 424665088. Throughput: 0: 9765.4, 1: 9570.3. Samples: 424633100. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:22:37,363][19170] Avg episode reward: [(0, '941.310'), (1, '990.250')] [2023-12-27 11:22:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000828984_212246528.pth... [2023-12-27 11:22:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000829664_212418560.pth... [2023-12-27 11:22:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000827864_211959808.pth [2023-12-27 11:22:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000828512_212123648.pth [2023-12-27 11:22:37,540][20381] Updated weights for policy 1, policy_version 828987 (0.0008) [2023-12-27 11:22:37,594][20381] Updated weights for policy 1, policy_version 828997 (0.0008) [2023-12-27 11:22:37,650][20381] Updated weights for policy 1, policy_version 829007 (0.0007) [2023-12-27 11:22:38,023][20347] Updated weights for policy 0, policy_version 829673 (0.0011) [2023-12-27 11:22:38,074][20347] Updated weights for policy 0, policy_version 829683 (0.0011) [2023-12-27 11:22:38,134][20347] Updated weights for policy 0, policy_version 829693 (0.0010) [2023-12-27 11:22:38,377][20381] Updated weights for policy 1, policy_version 829017 (0.0010) [2023-12-27 11:22:38,436][20381] Updated weights for policy 1, policy_version 829027 (0.0006) [2023-12-27 11:22:38,496][20381] Updated weights for policy 1, policy_version 829037 (0.0008) [2023-12-27 11:22:38,555][20381] Updated weights for policy 1, policy_version 829047 (0.0008) [2023-12-27 11:22:38,824][20347] Updated weights for policy 0, policy_version 829703 (0.0007) [2023-12-27 11:22:38,879][20347] Updated weights for policy 0, policy_version 829713 (0.0007) [2023-12-27 11:22:38,940][20347] Updated weights for policy 0, policy_version 829723 (0.0006) [2023-12-27 11:22:39,254][20381] Updated weights for policy 1, policy_version 829057 (0.0007) [2023-12-27 11:22:39,313][20381] Updated weights for policy 1, policy_version 829067 (0.0008) [2023-12-27 11:22:39,388][20381] Updated weights for policy 1, policy_version 829077 (0.0009) [2023-12-27 11:22:39,674][20347] Updated weights for policy 0, policy_version 829733 (0.0008) [2023-12-27 11:22:39,736][20347] Updated weights for policy 0, policy_version 829743 (0.0008) [2023-12-27 11:22:39,799][20347] Updated weights for policy 0, policy_version 829753 (0.0008) [2023-12-27 11:22:40,076][20381] Updated weights for policy 1, policy_version 829087 (0.0007) [2023-12-27 11:22:40,137][20381] Updated weights for policy 1, policy_version 829097 (0.0008) [2023-12-27 11:22:40,197][20381] Updated weights for policy 1, policy_version 829107 (0.0007) [2023-12-27 11:22:40,565][20347] Updated weights for policy 0, policy_version 829763 (0.0008) [2023-12-27 11:22:40,625][20347] Updated weights for policy 0, policy_version 829773 (0.0009) [2023-12-27 11:22:40,683][20347] Updated weights for policy 0, policy_version 829783 (0.0010) [2023-12-27 11:22:40,897][20381] Updated weights for policy 1, policy_version 829117 (0.0007) [2023-12-27 11:22:40,953][20381] Updated weights for policy 1, policy_version 829127 (0.0008) [2023-12-27 11:22:41,011][20381] Updated weights for policy 1, policy_version 829137 (0.0008) [2023-12-27 11:22:41,415][20347] Updated weights for policy 0, policy_version 829793 (0.0007) [2023-12-27 11:22:41,477][20347] Updated weights for policy 0, policy_version 829803 (0.0009) [2023-12-27 11:22:41,545][20347] Updated weights for policy 0, policy_version 829813 (0.0008) [2023-12-27 11:22:41,610][20347] Updated weights for policy 0, policy_version 829823 (0.0008) [2023-12-27 11:22:41,715][20381] Updated weights for policy 1, policy_version 829147 (0.0007) [2023-12-27 11:22:41,782][20381] Updated weights for policy 1, policy_version 829157 (0.0008) [2023-12-27 11:22:41,846][20381] Updated weights for policy 1, policy_version 829167 (0.0008) [2023-12-27 11:22:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 424755200. Throughput: 0: 9761.5, 1: 9562.4. Samples: 424751116. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:22:42,363][19170] Avg episode reward: [(0, '951.070'), (1, '990.250')] [2023-12-27 11:22:42,406][20347] Updated weights for policy 0, policy_version 829833 (0.0008) [2023-12-27 11:22:42,463][20347] Updated weights for policy 0, policy_version 829843 (0.0007) [2023-12-27 11:22:42,530][20347] Updated weights for policy 0, policy_version 829853 (0.0009) [2023-12-27 11:22:42,606][20381] Updated weights for policy 1, policy_version 829177 (0.0008) [2023-12-27 11:22:42,668][20381] Updated weights for policy 1, policy_version 829187 (0.0007) [2023-12-27 11:22:42,724][20381] Updated weights for policy 1, policy_version 829197 (0.0007) [2023-12-27 11:22:42,784][20381] Updated weights for policy 1, policy_version 829207 (0.0008) [2023-12-27 11:22:43,233][20347] Updated weights for policy 0, policy_version 829863 (0.0008) [2023-12-27 11:22:43,275][20291] KL-divergence is very high: 118.4924 [2023-12-27 11:22:43,286][20347] Updated weights for policy 0, policy_version 829873 (0.0007) [2023-12-27 11:22:43,337][20347] Updated weights for policy 0, policy_version 829883 (0.0008) [2023-12-27 11:22:43,492][20381] Updated weights for policy 1, policy_version 829217 (0.0008) [2023-12-27 11:22:43,549][20381] Updated weights for policy 1, policy_version 829227 (0.0006) [2023-12-27 11:22:43,613][20381] Updated weights for policy 1, policy_version 829237 (0.0007) [2023-12-27 11:22:43,998][20347] Updated weights for policy 0, policy_version 829893 (0.0008) [2023-12-27 11:22:44,056][20347] Updated weights for policy 0, policy_version 829903 (0.0008) [2023-12-27 11:22:44,117][20347] Updated weights for policy 0, policy_version 829913 (0.0008) [2023-12-27 11:22:44,311][20381] Updated weights for policy 1, policy_version 829247 (0.0008) [2023-12-27 11:22:44,375][20381] Updated weights for policy 1, policy_version 829257 (0.0006) [2023-12-27 11:22:44,445][20381] Updated weights for policy 1, policy_version 829267 (0.0008) [2023-12-27 11:22:44,881][20347] Updated weights for policy 0, policy_version 829923 (0.0008) [2023-12-27 11:22:44,943][20347] Updated weights for policy 0, policy_version 829933 (0.0007) [2023-12-27 11:22:45,001][20347] Updated weights for policy 0, policy_version 829943 (0.0008) [2023-12-27 11:22:45,093][20381] Updated weights for policy 1, policy_version 829277 (0.0008) [2023-12-27 11:22:45,154][20381] Updated weights for policy 1, policy_version 829287 (0.0007) [2023-12-27 11:22:45,210][20381] Updated weights for policy 1, policy_version 829297 (0.0008) [2023-12-27 11:22:45,742][20347] Updated weights for policy 0, policy_version 829953 (0.0006) [2023-12-27 11:22:45,784][20291] KL-divergence is very high: 294.5357 [2023-12-27 11:22:45,789][20291] KL-divergence is very high: 333.1334 [2023-12-27 11:22:45,800][20347] Updated weights for policy 0, policy_version 829963 (0.0007) [2023-12-27 11:22:45,829][20291] KL-divergence is very high: 193.2628 [2023-12-27 11:22:45,849][20347] Updated weights for policy 0, policy_version 829973 (0.0008) [2023-12-27 11:22:45,898][20381] Updated weights for policy 1, policy_version 829307 (0.0008) [2023-12-27 11:22:45,903][20347] Updated weights for policy 0, policy_version 829983 (0.0007) [2023-12-27 11:22:45,953][20381] Updated weights for policy 1, policy_version 829317 (0.0008) [2023-12-27 11:22:46,007][20381] Updated weights for policy 1, policy_version 829327 (0.0008) [2023-12-27 11:22:46,592][20347] Updated weights for policy 0, policy_version 829993 (0.0008) [2023-12-27 11:22:46,646][20347] Updated weights for policy 0, policy_version 830003 (0.0008) [2023-12-27 11:22:46,685][20381] Updated weights for policy 1, policy_version 829337 (0.0008) [2023-12-27 11:22:46,703][20347] Updated weights for policy 0, policy_version 830013 (0.0008) [2023-12-27 11:22:46,745][20381] Updated weights for policy 1, policy_version 829347 (0.0006) [2023-12-27 11:22:46,802][20381] Updated weights for policy 1, policy_version 829357 (0.0007) [2023-12-27 11:22:46,857][20381] Updated weights for policy 1, policy_version 829367 (0.0007) [2023-12-27 11:22:47,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 424853504. Throughput: 0: 9721.1, 1: 9646.4. Samples: 424867120. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:22:47,363][19170] Avg episode reward: [(0, '912.170'), (1, '1000.000')] [2023-12-27 11:22:47,434][20347] Updated weights for policy 0, policy_version 830023 (0.0007) [2023-12-27 11:22:47,494][20347] Updated weights for policy 0, policy_version 830033 (0.0008) [2023-12-27 11:22:47,520][20381] Updated weights for policy 1, policy_version 829377 (0.0007) [2023-12-27 11:22:47,556][20347] Updated weights for policy 0, policy_version 830043 (0.0008) [2023-12-27 11:22:47,588][20381] Updated weights for policy 1, policy_version 829387 (0.0007) [2023-12-27 11:22:47,641][20381] Updated weights for policy 1, policy_version 829397 (0.0008) [2023-12-27 11:22:48,238][20347] Updated weights for policy 0, policy_version 830053 (0.0007) [2023-12-27 11:22:48,305][20347] Updated weights for policy 0, policy_version 830063 (0.0006) [2023-12-27 11:22:48,364][20291] KL-divergence is very high: 105.2817 [2023-12-27 11:22:48,366][20381] Updated weights for policy 1, policy_version 829407 (0.0009) [2023-12-27 11:22:48,371][20347] Updated weights for policy 0, policy_version 830073 (0.0007) [2023-12-27 11:22:48,429][20381] Updated weights for policy 1, policy_version 829417 (0.0007) [2023-12-27 11:22:48,488][20381] Updated weights for policy 1, policy_version 829427 (0.0008) [2023-12-27 11:22:49,047][20347] Updated weights for policy 0, policy_version 830083 (0.0007) [2023-12-27 11:22:49,103][20347] Updated weights for policy 0, policy_version 830093 (0.0007) [2023-12-27 11:22:49,150][20347] Updated weights for policy 0, policy_version 830103 (0.0007) [2023-12-27 11:22:49,256][20381] Updated weights for policy 1, policy_version 829437 (0.0008) [2023-12-27 11:22:49,306][20381] Updated weights for policy 1, policy_version 829447 (0.0007) [2023-12-27 11:22:49,374][20381] Updated weights for policy 1, policy_version 829457 (0.0009) [2023-12-27 11:22:49,882][20347] Updated weights for policy 0, policy_version 830113 (0.0006) [2023-12-27 11:22:49,949][20347] Updated weights for policy 0, policy_version 830123 (0.0008) [2023-12-27 11:22:50,009][20291] KL-divergence is very high: 171.0866 [2023-12-27 11:22:50,014][20347] Updated weights for policy 0, policy_version 830133 (0.0008) [2023-12-27 11:22:50,026][20291] KL-divergence is very high: 101.0189 [2023-12-27 11:22:50,074][20347] Updated weights for policy 0, policy_version 830143 (0.0007) [2023-12-27 11:22:50,157][20381] Updated weights for policy 1, policy_version 829467 (0.0008) [2023-12-27 11:22:50,218][20381] Updated weights for policy 1, policy_version 829477 (0.0007) [2023-12-27 11:22:50,274][20381] Updated weights for policy 1, policy_version 829487 (0.0007) [2023-12-27 11:22:50,821][20291] KL-divergence is very high: 128.9064 [2023-12-27 11:22:50,833][20347] Updated weights for policy 0, policy_version 830153 (0.0008) [2023-12-27 11:22:50,898][20347] Updated weights for policy 0, policy_version 830163 (0.0008) [2023-12-27 11:22:50,960][20347] Updated weights for policy 0, policy_version 830173 (0.0007) [2023-12-27 11:22:50,975][20291] KL-divergence is very high: 189.3645 [2023-12-27 11:22:50,998][20381] Updated weights for policy 1, policy_version 829497 (0.0008) [2023-12-27 11:22:51,067][20381] Updated weights for policy 1, policy_version 829507 (0.0009) [2023-12-27 11:22:51,123][20381] Updated weights for policy 1, policy_version 829517 (0.0008) [2023-12-27 11:22:51,192][20381] Updated weights for policy 1, policy_version 829527 (0.0008) [2023-12-27 11:22:51,727][20347] Updated weights for policy 0, policy_version 830183 (0.0008) [2023-12-27 11:22:51,760][20291] KL-divergence is very high: 139.8797 [2023-12-27 11:22:51,791][20347] Updated weights for policy 0, policy_version 830193 (0.0007) [2023-12-27 11:22:51,811][20291] KL-divergence is very high: 110.7712 [2023-12-27 11:22:51,855][20347] Updated weights for policy 0, policy_version 830203 (0.0009) [2023-12-27 11:22:51,860][20291] KL-divergence is very high: 189.5356 [2023-12-27 11:22:51,872][20291] KL-divergence is very high: 138.1487 [2023-12-27 11:22:51,955][20381] Updated weights for policy 1, policy_version 829537 (0.0009) [2023-12-27 11:22:52,011][20381] Updated weights for policy 1, policy_version 829547 (0.0007) [2023-12-27 11:22:52,075][20381] Updated weights for policy 1, policy_version 829557 (0.0007) [2023-12-27 11:22:52,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 424951808. Throughput: 0: 9687.9, 1: 9645.3. Samples: 424924816. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:22:52,363][19170] Avg episode reward: [(0, '765.630'), (1, '990.270')] [2023-12-27 11:22:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000829560_212393984.pth... [2023-12-27 11:22:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000830208_212557824.pth... [2023-12-27 11:22:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000828408_212099072.pth [2023-12-27 11:22:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000829088_212271104.pth [2023-12-27 11:22:52,607][20291] KL-divergence is very high: 116.6072 [2023-12-27 11:22:52,617][20347] Updated weights for policy 0, policy_version 830213 (0.0007) [2023-12-27 11:22:52,682][20347] Updated weights for policy 0, policy_version 830223 (0.0007) [2023-12-27 11:22:52,738][20347] Updated weights for policy 0, policy_version 830233 (0.0008) [2023-12-27 11:22:52,818][20381] Updated weights for policy 1, policy_version 829567 (0.0008) [2023-12-27 11:22:52,875][20381] Updated weights for policy 1, policy_version 829577 (0.0007) [2023-12-27 11:22:52,937][20381] Updated weights for policy 1, policy_version 829587 (0.0007) [2023-12-27 11:22:53,334][20347] Updated weights for policy 0, policy_version 830243 (0.0006) [2023-12-27 11:22:53,397][20347] Updated weights for policy 0, policy_version 830253 (0.0009) [2023-12-27 11:22:53,452][20347] Updated weights for policy 0, policy_version 830263 (0.0010) [2023-12-27 11:22:53,597][20381] Updated weights for policy 1, policy_version 829597 (0.0007) [2023-12-27 11:22:53,659][20381] Updated weights for policy 1, policy_version 829607 (0.0007) [2023-12-27 11:22:53,718][20381] Updated weights for policy 1, policy_version 829617 (0.0006) [2023-12-27 11:22:54,138][20347] Updated weights for policy 0, policy_version 830273 (0.0006) [2023-12-27 11:22:54,191][20347] Updated weights for policy 0, policy_version 830283 (0.0007) [2023-12-27 11:22:54,245][20347] Updated weights for policy 0, policy_version 830293 (0.0006) [2023-12-27 11:22:54,293][20347] Updated weights for policy 0, policy_version 830303 (0.0007) [2023-12-27 11:22:54,351][20381] Updated weights for policy 1, policy_version 829627 (0.0006) [2023-12-27 11:22:54,408][20381] Updated weights for policy 1, policy_version 829637 (0.0008) [2023-12-27 11:22:54,459][20381] Updated weights for policy 1, policy_version 829647 (0.0007) [2023-12-27 11:22:54,956][20347] Updated weights for policy 0, policy_version 830313 (0.0008) [2023-12-27 11:22:55,017][20347] Updated weights for policy 0, policy_version 830323 (0.0007) [2023-12-27 11:22:55,079][20347] Updated weights for policy 0, policy_version 830333 (0.0005) [2023-12-27 11:22:55,192][20381] Updated weights for policy 1, policy_version 829657 (0.0008) [2023-12-27 11:22:55,258][20381] Updated weights for policy 1, policy_version 829667 (0.0008) [2023-12-27 11:22:55,322][20381] Updated weights for policy 1, policy_version 829677 (0.0008) [2023-12-27 11:22:55,384][20381] Updated weights for policy 1, policy_version 829687 (0.0009) [2023-12-27 11:22:55,683][20347] Updated weights for policy 0, policy_version 830343 (0.0007) [2023-12-27 11:22:55,743][20347] Updated weights for policy 0, policy_version 830353 (0.0007) [2023-12-27 11:22:55,792][20347] Updated weights for policy 0, policy_version 830363 (0.0007) [2023-12-27 11:22:56,011][20381] Updated weights for policy 1, policy_version 829697 (0.0006) [2023-12-27 11:22:56,065][20381] Updated weights for policy 1, policy_version 829707 (0.0006) [2023-12-27 11:22:56,114][20381] Updated weights for policy 1, policy_version 829717 (0.0006) [2023-12-27 11:22:56,521][20347] Updated weights for policy 0, policy_version 830373 (0.0007) [2023-12-27 11:22:56,583][20347] Updated weights for policy 0, policy_version 830383 (0.0008) [2023-12-27 11:22:56,638][20347] Updated weights for policy 0, policy_version 830393 (0.0008) [2023-12-27 11:22:56,823][20381] Updated weights for policy 1, policy_version 829727 (0.0007) [2023-12-27 11:22:56,881][20381] Updated weights for policy 1, policy_version 829737 (0.0009) [2023-12-27 11:22:56,936][20381] Updated weights for policy 1, policy_version 829747 (0.0007) [2023-12-27 11:22:57,256][20347] Updated weights for policy 0, policy_version 830403 (0.0007) [2023-12-27 11:22:57,308][20347] Updated weights for policy 0, policy_version 830413 (0.0007) [2023-12-27 11:22:57,361][20347] Updated weights for policy 0, policy_version 830423 (0.0006) [2023-12-27 11:22:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 425050112. Throughput: 0: 9697.2, 1: 9726.2. Samples: 425043860. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:22:57,363][19170] Avg episode reward: [(0, '726.720'), (1, '980.540')] [2023-12-27 11:22:57,693][20381] Updated weights for policy 1, policy_version 829757 (0.0007) [2023-12-27 11:22:57,756][20381] Updated weights for policy 1, policy_version 829767 (0.0008) [2023-12-27 11:22:57,815][20381] Updated weights for policy 1, policy_version 829777 (0.0008) [2023-12-27 11:22:58,060][20347] Updated weights for policy 0, policy_version 830433 (0.0006) [2023-12-27 11:22:58,120][20347] Updated weights for policy 0, policy_version 830443 (0.0006) [2023-12-27 11:22:58,188][20347] Updated weights for policy 0, policy_version 830453 (0.0007) [2023-12-27 11:22:58,256][20347] Updated weights for policy 0, policy_version 830463 (0.0008) [2023-12-27 11:22:58,570][20381] Updated weights for policy 1, policy_version 829787 (0.0008) [2023-12-27 11:22:58,636][20381] Updated weights for policy 1, policy_version 829797 (0.0008) [2023-12-27 11:22:58,701][20381] Updated weights for policy 1, policy_version 829807 (0.0008) [2023-12-27 11:22:59,053][20347] Updated weights for policy 0, policy_version 830473 (0.0008) [2023-12-27 11:22:59,109][20347] Updated weights for policy 0, policy_version 830483 (0.0005) [2023-12-27 11:22:59,162][20347] Updated weights for policy 0, policy_version 830493 (0.0006) [2023-12-27 11:22:59,420][20381] Updated weights for policy 1, policy_version 829817 (0.0008) [2023-12-27 11:22:59,483][20381] Updated weights for policy 1, policy_version 829827 (0.0008) [2023-12-27 11:22:59,542][20381] Updated weights for policy 1, policy_version 829837 (0.0008) [2023-12-27 11:22:59,605][20381] Updated weights for policy 1, policy_version 829847 (0.0008) [2023-12-27 11:22:59,867][20347] Updated weights for policy 0, policy_version 830503 (0.0008) [2023-12-27 11:22:59,926][20347] Updated weights for policy 0, policy_version 830513 (0.0008) [2023-12-27 11:22:59,995][20347] Updated weights for policy 0, policy_version 830523 (0.0010) [2023-12-27 11:23:00,303][20381] Updated weights for policy 1, policy_version 829857 (0.0010) [2023-12-27 11:23:00,371][20381] Updated weights for policy 1, policy_version 829867 (0.0011) [2023-12-27 11:23:00,440][20381] Updated weights for policy 1, policy_version 829877 (0.0008) [2023-12-27 11:23:00,684][20347] Updated weights for policy 0, policy_version 830533 (0.0010) [2023-12-27 11:23:00,747][20347] Updated weights for policy 0, policy_version 830543 (0.0006) [2023-12-27 11:23:00,803][20347] Updated weights for policy 0, policy_version 830553 (0.0010) [2023-12-27 11:23:01,091][20381] Updated weights for policy 1, policy_version 829887 (0.0008) [2023-12-27 11:23:01,155][20381] Updated weights for policy 1, policy_version 829897 (0.0008) [2023-12-27 11:23:01,220][20381] Updated weights for policy 1, policy_version 829907 (0.0008) [2023-12-27 11:23:01,518][20347] Updated weights for policy 0, policy_version 830563 (0.0009) [2023-12-27 11:23:01,572][20347] Updated weights for policy 0, policy_version 830573 (0.0008) [2023-12-27 11:23:01,639][20347] Updated weights for policy 0, policy_version 830583 (0.0011) [2023-12-27 11:23:01,983][20381] Updated weights for policy 1, policy_version 829917 (0.0007) [2023-12-27 11:23:02,043][20381] Updated weights for policy 1, policy_version 829927 (0.0007) [2023-12-27 11:23:02,097][20381] Updated weights for policy 1, policy_version 829937 (0.0008) [2023-12-27 11:23:02,359][20347] Updated weights for policy 0, policy_version 830593 (0.0010) [2023-12-27 11:23:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 425148416. Throughput: 0: 9755.7, 1: 9695.5. Samples: 425160264. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:02,363][19170] Avg episode reward: [(0, '795.200'), (1, '990.270')] [2023-12-27 11:23:02,428][20347] Updated weights for policy 0, policy_version 830603 (0.0009) [2023-12-27 11:23:02,492][20347] Updated weights for policy 0, policy_version 830613 (0.0011) [2023-12-27 11:23:02,555][20347] Updated weights for policy 0, policy_version 830623 (0.0011) [2023-12-27 11:23:02,794][20381] Updated weights for policy 1, policy_version 829947 (0.0008) [2023-12-27 11:23:02,855][20381] Updated weights for policy 1, policy_version 829957 (0.0007) [2023-12-27 11:23:02,914][20381] Updated weights for policy 1, policy_version 829967 (0.0008) [2023-12-27 11:23:03,246][20347] Updated weights for policy 0, policy_version 830633 (0.0009) [2023-12-27 11:23:03,303][20347] Updated weights for policy 0, policy_version 830643 (0.0008) [2023-12-27 11:23:03,358][20347] Updated weights for policy 0, policy_version 830653 (0.0007) [2023-12-27 11:23:03,633][20381] Updated weights for policy 1, policy_version 829977 (0.0007) [2023-12-27 11:23:03,690][20381] Updated weights for policy 1, policy_version 829987 (0.0008) [2023-12-27 11:23:03,750][20381] Updated weights for policy 1, policy_version 829997 (0.0007) [2023-12-27 11:23:03,807][20381] Updated weights for policy 1, policy_version 830007 (0.0007) [2023-12-27 11:23:04,065][20347] Updated weights for policy 0, policy_version 830663 (0.0010) [2023-12-27 11:23:04,129][20347] Updated weights for policy 0, policy_version 830673 (0.0007) [2023-12-27 11:23:04,193][20347] Updated weights for policy 0, policy_version 830683 (0.0008) [2023-12-27 11:23:04,552][20381] Updated weights for policy 1, policy_version 830017 (0.0007) [2023-12-27 11:23:04,607][20381] Updated weights for policy 1, policy_version 830027 (0.0007) [2023-12-27 11:23:04,661][20381] Updated weights for policy 1, policy_version 830037 (0.0007) [2023-12-27 11:23:04,884][20347] Updated weights for policy 0, policy_version 830693 (0.0007) [2023-12-27 11:23:04,940][20347] Updated weights for policy 0, policy_version 830703 (0.0006) [2023-12-27 11:23:04,997][20347] Updated weights for policy 0, policy_version 830713 (0.0010) [2023-12-27 11:23:05,401][20381] Updated weights for policy 1, policy_version 830047 (0.0008) [2023-12-27 11:23:05,464][20381] Updated weights for policy 1, policy_version 830057 (0.0008) [2023-12-27 11:23:05,527][20381] Updated weights for policy 1, policy_version 830067 (0.0007) [2023-12-27 11:23:05,649][20347] Updated weights for policy 0, policy_version 830723 (0.0009) [2023-12-27 11:23:05,710][20347] Updated weights for policy 0, policy_version 830733 (0.0011) [2023-12-27 11:23:05,767][20347] Updated weights for policy 0, policy_version 830743 (0.0009) [2023-12-27 11:23:06,255][20381] Updated weights for policy 1, policy_version 830077 (0.0008) [2023-12-27 11:23:06,315][20381] Updated weights for policy 1, policy_version 830087 (0.0007) [2023-12-27 11:23:06,382][20381] Updated weights for policy 1, policy_version 830097 (0.0006) [2023-12-27 11:23:06,427][20347] Updated weights for policy 0, policy_version 830753 (0.0009) [2023-12-27 11:23:06,488][20347] Updated weights for policy 0, policy_version 830763 (0.0011) [2023-12-27 11:23:06,551][20347] Updated weights for policy 0, policy_version 830773 (0.0011) [2023-12-27 11:23:06,611][20347] Updated weights for policy 0, policy_version 830783 (0.0010) [2023-12-27 11:23:07,030][20381] Updated weights for policy 1, policy_version 830107 (0.0008) [2023-12-27 11:23:07,087][20381] Updated weights for policy 1, policy_version 830117 (0.0009) [2023-12-27 11:23:07,147][20381] Updated weights for policy 1, policy_version 830127 (0.0009) [2023-12-27 11:23:07,274][20347] Updated weights for policy 0, policy_version 830793 (0.0010) [2023-12-27 11:23:07,335][20347] Updated weights for policy 0, policy_version 830803 (0.0011) [2023-12-27 11:23:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 425246720. Throughput: 0: 9777.4, 1: 9716.3. Samples: 425219000. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:07,363][19170] Avg episode reward: [(0, '785.450'), (1, '1000.000')] [2023-12-27 11:23:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000830136_212541440.pth... [2023-12-27 11:23:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000828984_212246528.pth [2023-12-27 11:23:07,393][20347] Updated weights for policy 0, policy_version 830813 (0.0010) [2023-12-27 11:23:07,409][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000830816_212713472.pth... [2023-12-27 11:23:07,412][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000829664_212418560.pth [2023-12-27 11:23:07,846][20381] Updated weights for policy 1, policy_version 830137 (0.0008) [2023-12-27 11:23:07,906][20381] Updated weights for policy 1, policy_version 830147 (0.0007) [2023-12-27 11:23:07,960][20381] Updated weights for policy 1, policy_version 830157 (0.0008) [2023-12-27 11:23:08,012][20381] Updated weights for policy 1, policy_version 830167 (0.0008) [2023-12-27 11:23:08,090][20347] Updated weights for policy 0, policy_version 830823 (0.0008) [2023-12-27 11:23:08,142][20347] Updated weights for policy 0, policy_version 830833 (0.0011) [2023-12-27 11:23:08,205][20347] Updated weights for policy 0, policy_version 830843 (0.0010) [2023-12-27 11:23:08,737][20381] Updated weights for policy 1, policy_version 830177 (0.0006) [2023-12-27 11:23:08,799][20381] Updated weights for policy 1, policy_version 830187 (0.0011) [2023-12-27 11:23:08,860][20381] Updated weights for policy 1, policy_version 830197 (0.0011) [2023-12-27 11:23:08,920][20347] Updated weights for policy 0, policy_version 830853 (0.0010) [2023-12-27 11:23:08,977][20347] Updated weights for policy 0, policy_version 830863 (0.0011) [2023-12-27 11:23:09,035][20347] Updated weights for policy 0, policy_version 830873 (0.0012) [2023-12-27 11:23:09,588][20381] Updated weights for policy 1, policy_version 830207 (0.0009) [2023-12-27 11:23:09,655][20381] Updated weights for policy 1, policy_version 830217 (0.0008) [2023-12-27 11:23:09,718][20381] Updated weights for policy 1, policy_version 830227 (0.0008) [2023-12-27 11:23:09,784][20347] Updated weights for policy 0, policy_version 830883 (0.0009) [2023-12-27 11:23:09,805][20291] KL-divergence is very high: 134.9140 [2023-12-27 11:23:09,854][20347] Updated weights for policy 0, policy_version 830893 (0.0008) [2023-12-27 11:23:09,860][20291] KL-divergence is very high: 131.2033 [2023-12-27 11:23:09,910][20291] KL-divergence is very high: 127.8997 [2023-12-27 11:23:09,915][20347] Updated weights for policy 0, policy_version 830903 (0.0008) [2023-12-27 11:23:09,950][20291] KL-divergence is very high: 112.5622 [2023-12-27 11:23:09,956][20291] KL-divergence is very high: 104.4662 [2023-12-27 11:23:09,962][20291] KL-divergence is very high: 113.4722 [2023-12-27 11:23:10,486][20381] Updated weights for policy 1, policy_version 830237 (0.0009) [2023-12-27 11:23:10,545][20381] Updated weights for policy 1, policy_version 830247 (0.0008) [2023-12-27 11:23:10,606][20381] Updated weights for policy 1, policy_version 830257 (0.0007) [2023-12-27 11:23:10,634][20347] Updated weights for policy 0, policy_version 830913 (0.0007) [2023-12-27 11:23:10,692][20347] Updated weights for policy 0, policy_version 830923 (0.0005) [2023-12-27 11:23:10,761][20347] Updated weights for policy 0, policy_version 830933 (0.0007) [2023-12-27 11:23:10,821][20347] Updated weights for policy 0, policy_version 830943 (0.0006) [2023-12-27 11:23:11,395][20381] Updated weights for policy 1, policy_version 830267 (0.0007) [2023-12-27 11:23:11,460][20381] Updated weights for policy 1, policy_version 830277 (0.0008) [2023-12-27 11:23:11,495][20347] Updated weights for policy 0, policy_version 830953 (0.0007) [2023-12-27 11:23:11,528][20381] Updated weights for policy 1, policy_version 830287 (0.0008) [2023-12-27 11:23:11,561][20347] Updated weights for policy 0, policy_version 830963 (0.0006) [2023-12-27 11:23:11,622][20347] Updated weights for policy 0, policy_version 830973 (0.0009) [2023-12-27 11:23:12,214][20381] Updated weights for policy 1, policy_version 830297 (0.0008) [2023-12-27 11:23:12,280][20381] Updated weights for policy 1, policy_version 830307 (0.0007) [2023-12-27 11:23:12,332][20347] Updated weights for policy 0, policy_version 830983 (0.0008) [2023-12-27 11:23:12,342][20381] Updated weights for policy 1, policy_version 830317 (0.0006) [2023-12-27 11:23:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 425336832. Throughput: 0: 9831.6, 1: 9722.3. Samples: 425336124. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:12,363][19170] Avg episode reward: [(0, '706.650'), (1, '1000.000')] [2023-12-27 11:23:12,403][20347] Updated weights for policy 0, policy_version 830993 (0.0010) [2023-12-27 11:23:12,407][20381] Updated weights for policy 1, policy_version 830327 (0.0008) [2023-12-27 11:23:12,463][20347] Updated weights for policy 0, policy_version 831003 (0.0008) [2023-12-27 11:23:13,124][20381] Updated weights for policy 1, policy_version 830337 (0.0008) [2023-12-27 11:23:13,189][20347] Updated weights for policy 0, policy_version 831013 (0.0007) [2023-12-27 11:23:13,191][20381] Updated weights for policy 1, policy_version 830347 (0.0007) [2023-12-27 11:23:13,196][20291] KL-divergence is very high: 125.2135 [2023-12-27 11:23:13,255][20381] Updated weights for policy 1, policy_version 830357 (0.0007) [2023-12-27 11:23:13,257][20347] Updated weights for policy 0, policy_version 831023 (0.0006) [2023-12-27 11:23:13,309][20347] Updated weights for policy 0, policy_version 831033 (0.0008) [2023-12-27 11:23:13,911][20381] Updated weights for policy 1, policy_version 830367 (0.0007) [2023-12-27 11:23:13,974][20381] Updated weights for policy 1, policy_version 830377 (0.0008) [2023-12-27 11:23:14,019][20347] Updated weights for policy 0, policy_version 831043 (0.0007) [2023-12-27 11:23:14,039][20381] Updated weights for policy 1, policy_version 830387 (0.0007) [2023-12-27 11:23:14,075][20347] Updated weights for policy 0, policy_version 831053 (0.0006) [2023-12-27 11:23:14,141][20347] Updated weights for policy 0, policy_version 831063 (0.0007) [2023-12-27 11:23:14,710][20381] Updated weights for policy 1, policy_version 830397 (0.0006) [2023-12-27 11:23:14,762][20381] Updated weights for policy 1, policy_version 830407 (0.0008) [2023-12-27 11:23:14,796][20347] Updated weights for policy 0, policy_version 831073 (0.0008) [2023-12-27 11:23:14,829][20381] Updated weights for policy 1, policy_version 830417 (0.0009) [2023-12-27 11:23:14,868][20347] Updated weights for policy 0, policy_version 831083 (0.0011) [2023-12-27 11:23:14,935][20347] Updated weights for policy 0, policy_version 831093 (0.0010) [2023-12-27 11:23:14,993][20347] Updated weights for policy 0, policy_version 831103 (0.0010) [2023-12-27 11:23:15,584][20381] Updated weights for policy 1, policy_version 830427 (0.0009) [2023-12-27 11:23:15,647][20381] Updated weights for policy 1, policy_version 830437 (0.0011) [2023-12-27 11:23:15,694][20347] Updated weights for policy 0, policy_version 831113 (0.0006) [2023-12-27 11:23:15,707][20381] Updated weights for policy 1, policy_version 830447 (0.0011) [2023-12-27 11:23:15,752][20347] Updated weights for policy 0, policy_version 831123 (0.0006) [2023-12-27 11:23:15,816][20347] Updated weights for policy 0, policy_version 831133 (0.0008) [2023-12-27 11:23:16,377][20381] Updated weights for policy 1, policy_version 830457 (0.0008) [2023-12-27 11:23:16,443][20381] Updated weights for policy 1, policy_version 830467 (0.0010) [2023-12-27 11:23:16,478][20347] Updated weights for policy 0, policy_version 831143 (0.0009) [2023-12-27 11:23:16,495][20291] KL-divergence is very high: 254.6975 [2023-12-27 11:23:16,507][20381] Updated weights for policy 1, policy_version 830477 (0.0008) [2023-12-27 11:23:16,539][20347] Updated weights for policy 0, policy_version 831153 (0.0011) [2023-12-27 11:23:16,563][20381] Updated weights for policy 1, policy_version 830487 (0.0009) [2023-12-27 11:23:16,597][20347] Updated weights for policy 0, policy_version 831163 (0.0011) [2023-12-27 11:23:17,249][20381] Updated weights for policy 1, policy_version 830497 (0.0007) [2023-12-27 11:23:17,308][20381] Updated weights for policy 1, policy_version 830507 (0.0006) [2023-12-27 11:23:17,321][20347] Updated weights for policy 0, policy_version 831173 (0.0010) [2023-12-27 11:23:17,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 425435136. Throughput: 0: 9806.3, 1: 9740.8. Samples: 425453492. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:17,363][19170] Avg episode reward: [(0, '745.070'), (1, '1000.000')] [2023-12-27 11:23:17,366][20381] Updated weights for policy 1, policy_version 830517 (0.0006) [2023-12-27 11:23:17,383][20347] Updated weights for policy 0, policy_version 831183 (0.0011) [2023-12-27 11:23:17,440][20347] Updated weights for policy 0, policy_version 831193 (0.0011) [2023-12-27 11:23:17,936][20381] Updated weights for policy 1, policy_version 830527 (0.0007) [2023-12-27 11:23:17,992][20381] Updated weights for policy 1, policy_version 830537 (0.0007) [2023-12-27 11:23:18,046][20381] Updated weights for policy 1, policy_version 830547 (0.0007) [2023-12-27 11:23:18,175][20347] Updated weights for policy 0, policy_version 831203 (0.0010) [2023-12-27 11:23:18,230][20347] Updated weights for policy 0, policy_version 831213 (0.0011) [2023-12-27 11:23:18,292][20347] Updated weights for policy 0, policy_version 831223 (0.0010) [2023-12-27 11:23:18,815][20381] Updated weights for policy 1, policy_version 830557 (0.0007) [2023-12-27 11:23:18,877][20381] Updated weights for policy 1, policy_version 830567 (0.0008) [2023-12-27 11:23:18,937][20381] Updated weights for policy 1, policy_version 830577 (0.0009) [2023-12-27 11:23:19,008][20347] Updated weights for policy 0, policy_version 831233 (0.0009) [2023-12-27 11:23:19,062][20347] Updated weights for policy 0, policy_version 831243 (0.0008) [2023-12-27 11:23:19,126][20347] Updated weights for policy 0, policy_version 831253 (0.0008) [2023-12-27 11:23:19,180][20347] Updated weights for policy 0, policy_version 831263 (0.0007) [2023-12-27 11:23:19,601][20381] Updated weights for policy 1, policy_version 830587 (0.0008) [2023-12-27 11:23:19,661][20381] Updated weights for policy 1, policy_version 830597 (0.0007) [2023-12-27 11:23:19,723][20381] Updated weights for policy 1, policy_version 830607 (0.0006) [2023-12-27 11:23:19,968][20347] Updated weights for policy 0, policy_version 831273 (0.0008) [2023-12-27 11:23:20,029][20347] Updated weights for policy 0, policy_version 831283 (0.0006) [2023-12-27 11:23:20,092][20347] Updated weights for policy 0, policy_version 831293 (0.0008) [2023-12-27 11:23:20,432][20381] Updated weights for policy 1, policy_version 830617 (0.0008) [2023-12-27 11:23:20,500][20381] Updated weights for policy 1, policy_version 830627 (0.0007) [2023-12-27 11:23:20,562][20381] Updated weights for policy 1, policy_version 830637 (0.0007) [2023-12-27 11:23:20,628][20381] Updated weights for policy 1, policy_version 830647 (0.0010) [2023-12-27 11:23:20,809][20347] Updated weights for policy 0, policy_version 831303 (0.0008) [2023-12-27 11:23:20,862][20347] Updated weights for policy 0, policy_version 831313 (0.0007) [2023-12-27 11:23:20,927][20347] Updated weights for policy 0, policy_version 831323 (0.0008) [2023-12-27 11:23:21,380][20381] Updated weights for policy 1, policy_version 830657 (0.0010) [2023-12-27 11:23:21,451][20381] Updated weights for policy 1, policy_version 830667 (0.0007) [2023-12-27 11:23:21,511][20381] Updated weights for policy 1, policy_version 830677 (0.0006) [2023-12-27 11:23:21,696][20347] Updated weights for policy 0, policy_version 831333 (0.0009) [2023-12-27 11:23:21,770][20347] Updated weights for policy 0, policy_version 831343 (0.0009) [2023-12-27 11:23:21,837][20347] Updated weights for policy 0, policy_version 831353 (0.0008) [2023-12-27 11:23:22,241][20381] Updated weights for policy 1, policy_version 830687 (0.0006) [2023-12-27 11:23:22,307][20381] Updated weights for policy 1, policy_version 830697 (0.0009) [2023-12-27 11:23:22,362][20381] Updated weights for policy 1, policy_version 830707 (0.0007) [2023-12-27 11:23:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 425533440. Throughput: 0: 9775.3, 1: 9762.9. Samples: 425512316. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:22,363][19170] Avg episode reward: [(0, '803.780'), (1, '1000.000')] [2023-12-27 11:23:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000831360_212852736.pth... [2023-12-27 11:23:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000830208_212557824.pth [2023-12-27 11:23:22,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000830712_212688896.pth... [2023-12-27 11:23:22,403][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000829560_212393984.pth [2023-12-27 11:23:22,582][20347] Updated weights for policy 0, policy_version 831363 (0.0008) [2023-12-27 11:23:22,641][20347] Updated weights for policy 0, policy_version 831373 (0.0008) [2023-12-27 11:23:22,698][20347] Updated weights for policy 0, policy_version 831383 (0.0008) [2023-12-27 11:23:23,073][20381] Updated weights for policy 1, policy_version 830717 (0.0008) [2023-12-27 11:23:23,132][20381] Updated weights for policy 1, policy_version 830727 (0.0007) [2023-12-27 11:23:23,195][20381] Updated weights for policy 1, policy_version 830737 (0.0008) [2023-12-27 11:23:23,389][20347] Updated weights for policy 0, policy_version 831393 (0.0008) [2023-12-27 11:23:23,445][20347] Updated weights for policy 0, policy_version 831403 (0.0006) [2023-12-27 11:23:23,500][20347] Updated weights for policy 0, policy_version 831413 (0.0006) [2023-12-27 11:23:23,556][20347] Updated weights for policy 0, policy_version 831423 (0.0006) [2023-12-27 11:23:23,922][20381] Updated weights for policy 1, policy_version 830747 (0.0008) [2023-12-27 11:23:23,980][20381] Updated weights for policy 1, policy_version 830757 (0.0007) [2023-12-27 11:23:24,044][20381] Updated weights for policy 1, policy_version 830767 (0.0008) [2023-12-27 11:23:24,157][20347] Updated weights for policy 0, policy_version 831433 (0.0008) [2023-12-27 11:23:24,215][20347] Updated weights for policy 0, policy_version 831443 (0.0009) [2023-12-27 11:23:24,275][20347] Updated weights for policy 0, policy_version 831453 (0.0007) [2023-12-27 11:23:24,752][20381] Updated weights for policy 1, policy_version 830777 (0.0008) [2023-12-27 11:23:24,811][20381] Updated weights for policy 1, policy_version 830787 (0.0008) [2023-12-27 11:23:24,863][20381] Updated weights for policy 1, policy_version 830797 (0.0007) [2023-12-27 11:23:24,915][20381] Updated weights for policy 1, policy_version 830807 (0.0007) [2023-12-27 11:23:24,942][20347] Updated weights for policy 0, policy_version 831463 (0.0009) [2023-12-27 11:23:24,991][20347] Updated weights for policy 0, policy_version 831473 (0.0010) [2023-12-27 11:23:25,047][20347] Updated weights for policy 0, policy_version 831483 (0.0006) [2023-12-27 11:23:25,657][20381] Updated weights for policy 1, policy_version 830817 (0.0008) [2023-12-27 11:23:25,704][20347] Updated weights for policy 0, policy_version 831493 (0.0008) [2023-12-27 11:23:25,717][20381] Updated weights for policy 1, policy_version 830827 (0.0007) [2023-12-27 11:23:25,766][20347] Updated weights for policy 0, policy_version 831503 (0.0010) [2023-12-27 11:23:25,779][20381] Updated weights for policy 1, policy_version 830837 (0.0008) [2023-12-27 11:23:25,824][20347] Updated weights for policy 0, policy_version 831513 (0.0008) [2023-12-27 11:23:26,432][20347] Updated weights for policy 0, policy_version 831523 (0.0009) [2023-12-27 11:23:26,482][20347] Updated weights for policy 0, policy_version 831533 (0.0006) [2023-12-27 11:23:26,521][20381] Updated weights for policy 1, policy_version 830847 (0.0008) [2023-12-27 11:23:26,543][20347] Updated weights for policy 0, policy_version 831543 (0.0005) [2023-12-27 11:23:26,567][20381] Updated weights for policy 1, policy_version 830857 (0.0007) [2023-12-27 11:23:26,622][20381] Updated weights for policy 1, policy_version 830867 (0.0008) [2023-12-27 11:23:27,203][20347] Updated weights for policy 0, policy_version 831553 (0.0007) [2023-12-27 11:23:27,265][20347] Updated weights for policy 0, policy_version 831563 (0.0008) [2023-12-27 11:23:27,312][20381] Updated weights for policy 1, policy_version 830877 (0.0007) [2023-12-27 11:23:27,327][20347] Updated weights for policy 0, policy_version 831573 (0.0008) [2023-12-27 11:23:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 425631744. Throughput: 0: 9791.2, 1: 9709.6. Samples: 425628652. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:27,363][19170] Avg episode reward: [(0, '814.110'), (1, '1000.000')] [2023-12-27 11:23:27,370][20381] Updated weights for policy 1, policy_version 830887 (0.0007) [2023-12-27 11:23:27,380][20347] Updated weights for policy 0, policy_version 831583 (0.0006) [2023-12-27 11:23:27,429][20381] Updated weights for policy 1, policy_version 830897 (0.0007) [2023-12-27 11:23:28,106][20347] Updated weights for policy 0, policy_version 831593 (0.0007) [2023-12-27 11:23:28,149][20381] Updated weights for policy 1, policy_version 830907 (0.0008) [2023-12-27 11:23:28,163][20347] Updated weights for policy 0, policy_version 831603 (0.0007) [2023-12-27 11:23:28,215][20381] Updated weights for policy 1, policy_version 830917 (0.0008) [2023-12-27 11:23:28,225][20347] Updated weights for policy 0, policy_version 831613 (0.0008) [2023-12-27 11:23:28,277][20381] Updated weights for policy 1, policy_version 830927 (0.0007) [2023-12-27 11:23:28,938][20381] Updated weights for policy 1, policy_version 830937 (0.0007) [2023-12-27 11:23:28,959][20347] Updated weights for policy 0, policy_version 831623 (0.0008) [2023-12-27 11:23:28,996][20381] Updated weights for policy 1, policy_version 830947 (0.0009) [2023-12-27 11:23:29,017][20347] Updated weights for policy 0, policy_version 831633 (0.0007) [2023-12-27 11:23:29,053][20381] Updated weights for policy 1, policy_version 830957 (0.0009) [2023-12-27 11:23:29,060][20292] KL-divergence is very high: 250.6992 [2023-12-27 11:23:29,070][20347] Updated weights for policy 0, policy_version 831643 (0.0007) [2023-12-27 11:23:29,115][20381] Updated weights for policy 1, policy_version 830967 (0.0008) [2023-12-27 11:23:29,757][20347] Updated weights for policy 0, policy_version 831653 (0.0006) [2023-12-27 11:23:29,806][20347] Updated weights for policy 0, policy_version 831663 (0.0006) [2023-12-27 11:23:29,866][20347] Updated weights for policy 0, policy_version 831673 (0.0007) [2023-12-27 11:23:29,907][20381] Updated weights for policy 1, policy_version 830977 (0.0006) [2023-12-27 11:23:29,971][20381] Updated weights for policy 1, policy_version 830987 (0.0008) [2023-12-27 11:23:30,043][20381] Updated weights for policy 1, policy_version 830997 (0.0006) [2023-12-27 11:23:30,613][20347] Updated weights for policy 0, policy_version 831683 (0.0008) [2023-12-27 11:23:30,675][20347] Updated weights for policy 0, policy_version 831693 (0.0008) [2023-12-27 11:23:30,710][20381] Updated weights for policy 1, policy_version 831007 (0.0007) [2023-12-27 11:23:30,711][20292] KL-divergence is very high: 155.6830 [2023-12-27 11:23:30,728][20347] Updated weights for policy 0, policy_version 831703 (0.0006) [2023-12-27 11:23:30,733][20292] KL-divergence is very high: 302.1854 [2023-12-27 11:23:30,777][20381] Updated weights for policy 1, policy_version 831017 (0.0007) [2023-12-27 11:23:30,834][20381] Updated weights for policy 1, policy_version 831027 (0.0009) [2023-12-27 11:23:30,855][20292] KL-divergence is very high: 280.7506 [2023-12-27 11:23:31,386][20347] Updated weights for policy 0, policy_version 831713 (0.0006) [2023-12-27 11:23:31,443][20347] Updated weights for policy 0, policy_version 831723 (0.0008) [2023-12-27 11:23:31,500][20347] Updated weights for policy 0, policy_version 831733 (0.0007) [2023-12-27 11:23:31,557][20347] Updated weights for policy 0, policy_version 831743 (0.0007) [2023-12-27 11:23:31,575][20381] Updated weights for policy 1, policy_version 831037 (0.0007) [2023-12-27 11:23:31,637][20381] Updated weights for policy 1, policy_version 831047 (0.0007) [2023-12-27 11:23:31,692][20381] Updated weights for policy 1, policy_version 831057 (0.0008) [2023-12-27 11:23:32,268][20347] Updated weights for policy 0, policy_version 831753 (0.0006) [2023-12-27 11:23:32,321][20347] Updated weights for policy 0, policy_version 831763 (0.0008) [2023-12-27 11:23:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 425730048. Throughput: 0: 9868.7, 1: 9689.2. Samples: 425747228. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:32,363][19170] Avg episode reward: [(0, '775.290'), (1, '980.510')] [2023-12-27 11:23:32,385][20347] Updated weights for policy 0, policy_version 831773 (0.0008) [2023-12-27 11:23:32,454][20381] Updated weights for policy 1, policy_version 831067 (0.0008) [2023-12-27 11:23:32,511][20381] Updated weights for policy 1, policy_version 831077 (0.0008) [2023-12-27 11:23:32,571][20381] Updated weights for policy 1, policy_version 831087 (0.0007) [2023-12-27 11:23:33,091][20347] Updated weights for policy 0, policy_version 831783 (0.0008) [2023-12-27 11:23:33,153][20347] Updated weights for policy 0, policy_version 831793 (0.0008) [2023-12-27 11:23:33,214][20347] Updated weights for policy 0, policy_version 831803 (0.0008) [2023-12-27 11:23:33,256][20381] Updated weights for policy 1, policy_version 831097 (0.0008) [2023-12-27 11:23:33,309][20381] Updated weights for policy 1, policy_version 831107 (0.0005) [2023-12-27 11:23:33,364][20381] Updated weights for policy 1, policy_version 831117 (0.0005) [2023-12-27 11:23:33,420][20381] Updated weights for policy 1, policy_version 831127 (0.0005) [2023-12-27 11:23:33,889][20347] Updated weights for policy 0, policy_version 831813 (0.0008) [2023-12-27 11:23:33,947][20347] Updated weights for policy 0, policy_version 831823 (0.0008) [2023-12-27 11:23:34,012][20347] Updated weights for policy 0, policy_version 831833 (0.0007) [2023-12-27 11:23:34,056][20381] Updated weights for policy 1, policy_version 831137 (0.0006) [2023-12-27 11:23:34,115][20381] Updated weights for policy 1, policy_version 831147 (0.0007) [2023-12-27 11:23:34,180][20381] Updated weights for policy 1, policy_version 831157 (0.0008) [2023-12-27 11:23:34,699][20347] Updated weights for policy 0, policy_version 831843 (0.0006) [2023-12-27 11:23:34,750][20347] Updated weights for policy 0, policy_version 831853 (0.0008) [2023-12-27 11:23:34,811][20347] Updated weights for policy 0, policy_version 831863 (0.0008) [2023-12-27 11:23:34,902][20381] Updated weights for policy 1, policy_version 831167 (0.0008) [2023-12-27 11:23:34,963][20381] Updated weights for policy 1, policy_version 831177 (0.0008) [2023-12-27 11:23:35,022][20381] Updated weights for policy 1, policy_version 831187 (0.0008) [2023-12-27 11:23:35,536][20347] Updated weights for policy 0, policy_version 831873 (0.0008) [2023-12-27 11:23:35,591][20347] Updated weights for policy 0, policy_version 831883 (0.0005) [2023-12-27 11:23:35,648][20347] Updated weights for policy 0, policy_version 831893 (0.0006) [2023-12-27 11:23:35,704][20347] Updated weights for policy 0, policy_version 831903 (0.0008) [2023-12-27 11:23:35,748][20381] Updated weights for policy 1, policy_version 831197 (0.0009) [2023-12-27 11:23:35,798][20381] Updated weights for policy 1, policy_version 831207 (0.0007) [2023-12-27 11:23:35,856][20381] Updated weights for policy 1, policy_version 831217 (0.0009) [2023-12-27 11:23:36,407][20347] Updated weights for policy 0, policy_version 831913 (0.0006) [2023-12-27 11:23:36,472][20347] Updated weights for policy 0, policy_version 831923 (0.0006) [2023-12-27 11:23:36,532][20347] Updated weights for policy 0, policy_version 831933 (0.0006) [2023-12-27 11:23:36,602][20381] Updated weights for policy 1, policy_version 831227 (0.0008) [2023-12-27 11:23:36,664][20381] Updated weights for policy 1, policy_version 831237 (0.0008) [2023-12-27 11:23:36,728][20381] Updated weights for policy 1, policy_version 831247 (0.0008) [2023-12-27 11:23:37,149][20347] Updated weights for policy 0, policy_version 831943 (0.0006) [2023-12-27 11:23:37,208][20347] Updated weights for policy 0, policy_version 831953 (0.0007) [2023-12-27 11:23:37,265][20347] Updated weights for policy 0, policy_version 831963 (0.0006) [2023-12-27 11:23:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 425836544. Throughput: 0: 9880.4, 1: 9716.9. Samples: 425806700. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:37,364][19170] Avg episode reward: [(0, '843.440'), (1, '961.000')] [2023-12-27 11:23:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000831256_212828160.pth... [2023-12-27 11:23:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000831968_213008384.pth... [2023-12-27 11:23:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000830136_212541440.pth [2023-12-27 11:23:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000830816_212713472.pth [2023-12-27 11:23:37,427][20381] Updated weights for policy 1, policy_version 831257 (0.0008) [2023-12-27 11:23:37,481][20381] Updated weights for policy 1, policy_version 831267 (0.0008) [2023-12-27 11:23:37,537][20381] Updated weights for policy 1, policy_version 831277 (0.0009) [2023-12-27 11:23:37,597][20381] Updated weights for policy 1, policy_version 831287 (0.0009) [2023-12-27 11:23:37,911][20347] Updated weights for policy 0, policy_version 831973 (0.0007) [2023-12-27 11:23:37,973][20347] Updated weights for policy 0, policy_version 831983 (0.0008) [2023-12-27 11:23:38,035][20347] Updated weights for policy 0, policy_version 831993 (0.0008) [2023-12-27 11:23:38,346][20381] Updated weights for policy 1, policy_version 831297 (0.0008) [2023-12-27 11:23:38,404][20381] Updated weights for policy 1, policy_version 831307 (0.0007) [2023-12-27 11:23:38,464][20381] Updated weights for policy 1, policy_version 831317 (0.0008) [2023-12-27 11:23:38,737][20347] Updated weights for policy 0, policy_version 832003 (0.0007) [2023-12-27 11:23:38,800][20347] Updated weights for policy 0, policy_version 832013 (0.0006) [2023-12-27 11:23:38,865][20347] Updated weights for policy 0, policy_version 832023 (0.0008) [2023-12-27 11:23:39,206][20381] Updated weights for policy 1, policy_version 831327 (0.0007) [2023-12-27 11:23:39,272][20381] Updated weights for policy 1, policy_version 831337 (0.0008) [2023-12-27 11:23:39,332][20381] Updated weights for policy 1, policy_version 831347 (0.0008) [2023-12-27 11:23:39,542][20347] Updated weights for policy 0, policy_version 832033 (0.0008) [2023-12-27 11:23:39,602][20347] Updated weights for policy 0, policy_version 832043 (0.0007) [2023-12-27 11:23:39,654][20347] Updated weights for policy 0, policy_version 832053 (0.0008) [2023-12-27 11:23:39,711][20347] Updated weights for policy 0, policy_version 832063 (0.0007) [2023-12-27 11:23:40,147][20381] Updated weights for policy 1, policy_version 831357 (0.0009) [2023-12-27 11:23:40,207][20381] Updated weights for policy 1, policy_version 831367 (0.0008) [2023-12-27 11:23:40,272][20381] Updated weights for policy 1, policy_version 831377 (0.0007) [2023-12-27 11:23:40,428][20347] Updated weights for policy 0, policy_version 832073 (0.0006) [2023-12-27 11:23:40,497][20347] Updated weights for policy 0, policy_version 832083 (0.0008) [2023-12-27 11:23:40,565][20347] Updated weights for policy 0, policy_version 832093 (0.0008) [2023-12-27 11:23:41,008][20381] Updated weights for policy 1, policy_version 831387 (0.0008) [2023-12-27 11:23:41,068][20381] Updated weights for policy 1, policy_version 831397 (0.0008) [2023-12-27 11:23:41,133][20381] Updated weights for policy 1, policy_version 831407 (0.0008) [2023-12-27 11:23:41,281][20347] Updated weights for policy 0, policy_version 832103 (0.0007) [2023-12-27 11:23:41,340][20347] Updated weights for policy 0, policy_version 832113 (0.0007) [2023-12-27 11:23:41,404][20347] Updated weights for policy 0, policy_version 832123 (0.0007) [2023-12-27 11:23:41,865][20381] Updated weights for policy 1, policy_version 831417 (0.0009) [2023-12-27 11:23:41,927][20381] Updated weights for policy 1, policy_version 831427 (0.0011) [2023-12-27 11:23:41,992][20381] Updated weights for policy 1, policy_version 831437 (0.0011) [2023-12-27 11:23:42,054][20381] Updated weights for policy 1, policy_version 831447 (0.0011) [2023-12-27 11:23:42,154][20347] Updated weights for policy 0, policy_version 832133 (0.0008) [2023-12-27 11:23:42,217][20347] Updated weights for policy 0, policy_version 832143 (0.0008) [2023-12-27 11:23:42,280][20347] Updated weights for policy 0, policy_version 832153 (0.0008) [2023-12-27 11:23:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 425934848. Throughput: 0: 9890.7, 1: 9630.8. Samples: 425922328. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:42,363][19170] Avg episode reward: [(0, '863.260'), (1, '980.490')] [2023-12-27 11:23:42,810][20381] Updated weights for policy 1, policy_version 831457 (0.0009) [2023-12-27 11:23:42,868][20381] Updated weights for policy 1, policy_version 831467 (0.0008) [2023-12-27 11:23:42,929][20381] Updated weights for policy 1, policy_version 831477 (0.0008) [2023-12-27 11:23:43,005][20347] Updated weights for policy 0, policy_version 832163 (0.0007) [2023-12-27 11:23:43,067][20347] Updated weights for policy 0, policy_version 832173 (0.0007) [2023-12-27 11:23:43,120][20347] Updated weights for policy 0, policy_version 832183 (0.0007) [2023-12-27 11:23:43,728][20381] Updated weights for policy 1, policy_version 831487 (0.0008) [2023-12-27 11:23:43,776][20381] Updated weights for policy 1, policy_version 831497 (0.0008) [2023-12-27 11:23:43,779][20347] Updated weights for policy 0, policy_version 832193 (0.0007) [2023-12-27 11:23:43,832][20381] Updated weights for policy 1, policy_version 831507 (0.0005) [2023-12-27 11:23:43,833][20347] Updated weights for policy 0, policy_version 832203 (0.0008) [2023-12-27 11:23:43,894][20347] Updated weights for policy 0, policy_version 832213 (0.0006) [2023-12-27 11:23:43,940][20291] KL-divergence is very high: 108.2650 [2023-12-27 11:23:43,959][20347] Updated weights for policy 0, policy_version 832223 (0.0006) [2023-12-27 11:23:44,484][20381] Updated weights for policy 1, policy_version 831517 (0.0007) [2023-12-27 11:23:44,546][20381] Updated weights for policy 1, policy_version 831527 (0.0008) [2023-12-27 11:23:44,602][20381] Updated weights for policy 1, policy_version 831537 (0.0008) [2023-12-27 11:23:44,656][20347] Updated weights for policy 0, policy_version 832233 (0.0006) [2023-12-27 11:23:44,704][20347] Updated weights for policy 0, policy_version 832243 (0.0007) [2023-12-27 11:23:44,751][20347] Updated weights for policy 0, policy_version 832253 (0.0006) [2023-12-27 11:23:45,260][20381] Updated weights for policy 1, policy_version 831547 (0.0007) [2023-12-27 11:23:45,324][20381] Updated weights for policy 1, policy_version 831557 (0.0008) [2023-12-27 11:23:45,390][20381] Updated weights for policy 1, policy_version 831567 (0.0008) [2023-12-27 11:23:45,514][20347] Updated weights for policy 0, policy_version 832263 (0.0006) [2023-12-27 11:23:45,574][20347] Updated weights for policy 0, policy_version 832273 (0.0006) [2023-12-27 11:23:45,632][20347] Updated weights for policy 0, policy_version 832283 (0.0006) [2023-12-27 11:23:46,156][20381] Updated weights for policy 1, policy_version 831577 (0.0008) [2023-12-27 11:23:46,203][20347] Updated weights for policy 0, policy_version 832293 (0.0005) [2023-12-27 11:23:46,212][20381] Updated weights for policy 1, policy_version 831587 (0.0007) [2023-12-27 11:23:46,259][20347] Updated weights for policy 0, policy_version 832303 (0.0007) [2023-12-27 11:23:46,273][20381] Updated weights for policy 1, policy_version 831597 (0.0005) [2023-12-27 11:23:46,315][20347] Updated weights for policy 0, policy_version 832313 (0.0007) [2023-12-27 11:23:46,341][20381] Updated weights for policy 1, policy_version 831607 (0.0007) [2023-12-27 11:23:47,014][20347] Updated weights for policy 0, policy_version 832323 (0.0007) [2023-12-27 11:23:47,032][20381] Updated weights for policy 1, policy_version 831617 (0.0006) [2023-12-27 11:23:47,072][20347] Updated weights for policy 0, policy_version 832333 (0.0006) [2023-12-27 11:23:47,089][20381] Updated weights for policy 1, policy_version 831627 (0.0006) [2023-12-27 11:23:47,126][20347] Updated weights for policy 0, policy_version 832343 (0.0007) [2023-12-27 11:23:47,154][20381] Updated weights for policy 1, policy_version 831637 (0.0008) [2023-12-27 11:23:47,362][19170] Fps is (10 sec: 19661.5, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 426033152. Throughput: 0: 9895.2, 1: 9649.9. Samples: 426039792. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:47,363][19170] Avg episode reward: [(0, '843.710'), (1, '990.260')] [2023-12-27 11:23:47,753][20291] KL-divergence is very high: 121.0458 [2023-12-27 11:23:47,759][20347] Updated weights for policy 0, policy_version 832353 (0.0007) [2023-12-27 11:23:47,819][20347] Updated weights for policy 0, policy_version 832363 (0.0005) [2023-12-27 11:23:47,870][20347] Updated weights for policy 0, policy_version 832373 (0.0007) [2023-12-27 11:23:47,876][20381] Updated weights for policy 1, policy_version 831647 (0.0007) [2023-12-27 11:23:47,922][20347] Updated weights for policy 0, policy_version 832383 (0.0006) [2023-12-27 11:23:47,932][20381] Updated weights for policy 1, policy_version 831657 (0.0006) [2023-12-27 11:23:47,999][20381] Updated weights for policy 1, policy_version 831667 (0.0008) [2023-12-27 11:23:48,640][20347] Updated weights for policy 0, policy_version 832393 (0.0008) [2023-12-27 11:23:48,650][20381] Updated weights for policy 1, policy_version 831677 (0.0007) [2023-12-27 11:23:48,695][20347] Updated weights for policy 0, policy_version 832403 (0.0006) [2023-12-27 11:23:48,712][20381] Updated weights for policy 1, policy_version 831687 (0.0006) [2023-12-27 11:23:48,757][20347] Updated weights for policy 0, policy_version 832413 (0.0006) [2023-12-27 11:23:48,775][20381] Updated weights for policy 1, policy_version 831697 (0.0006) [2023-12-27 11:23:49,512][20347] Updated weights for policy 0, policy_version 832423 (0.0008) [2023-12-27 11:23:49,549][20381] Updated weights for policy 1, policy_version 831707 (0.0007) [2023-12-27 11:23:49,568][20347] Updated weights for policy 0, policy_version 832433 (0.0008) [2023-12-27 11:23:49,608][20381] Updated weights for policy 1, policy_version 831717 (0.0007) [2023-12-27 11:23:49,624][20347] Updated weights for policy 0, policy_version 832443 (0.0008) [2023-12-27 11:23:49,668][20381] Updated weights for policy 1, policy_version 831727 (0.0007) [2023-12-27 11:23:50,406][20347] Updated weights for policy 0, policy_version 832453 (0.0007) [2023-12-27 11:23:50,419][20381] Updated weights for policy 1, policy_version 831737 (0.0008) [2023-12-27 11:23:50,466][20347] Updated weights for policy 0, policy_version 832463 (0.0007) [2023-12-27 11:23:50,485][20381] Updated weights for policy 1, policy_version 831747 (0.0006) [2023-12-27 11:23:50,531][20347] Updated weights for policy 0, policy_version 832473 (0.0008) [2023-12-27 11:23:50,558][20381] Updated weights for policy 1, policy_version 831757 (0.0008) [2023-12-27 11:23:50,625][20381] Updated weights for policy 1, policy_version 831767 (0.0009) [2023-12-27 11:23:51,314][20347] Updated weights for policy 0, policy_version 832483 (0.0008) [2023-12-27 11:23:51,375][20347] Updated weights for policy 0, policy_version 832493 (0.0008) [2023-12-27 11:23:51,398][20381] Updated weights for policy 1, policy_version 831777 (0.0008) [2023-12-27 11:23:51,436][20347] Updated weights for policy 0, policy_version 832503 (0.0008) [2023-12-27 11:23:51,463][20381] Updated weights for policy 1, policy_version 831787 (0.0008) [2023-12-27 11:23:51,532][20381] Updated weights for policy 1, policy_version 831797 (0.0008) [2023-12-27 11:23:52,188][20347] Updated weights for policy 0, policy_version 832513 (0.0008) [2023-12-27 11:23:52,252][20347] Updated weights for policy 0, policy_version 832523 (0.0006) [2023-12-27 11:23:52,293][20381] Updated weights for policy 1, policy_version 831807 (0.0007) [2023-12-27 11:23:52,311][20347] Updated weights for policy 0, policy_version 832533 (0.0008) [2023-12-27 11:23:52,357][20381] Updated weights for policy 1, policy_version 831817 (0.0007) [2023-12-27 11:23:52,362][19170] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 426115072. Throughput: 0: 9883.4, 1: 9644.1. Samples: 426097736. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:52,363][19170] Avg episode reward: [(0, '814.150'), (1, '1000.000')] [2023-12-27 11:23:52,378][20347] Updated weights for policy 0, policy_version 832543 (0.0007) [2023-12-27 11:23:52,382][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000832544_213155840.pth... [2023-12-27 11:23:52,386][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000831360_212852736.pth [2023-12-27 11:23:52,423][20381] Updated weights for policy 1, policy_version 831827 (0.0009) [2023-12-27 11:23:52,453][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000831832_212975616.pth... [2023-12-27 11:23:52,457][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000830712_212688896.pth [2023-12-27 11:23:53,047][20347] Updated weights for policy 0, policy_version 832553 (0.0008) [2023-12-27 11:23:53,104][20347] Updated weights for policy 0, policy_version 832563 (0.0009) [2023-12-27 11:23:53,156][20347] Updated weights for policy 0, policy_version 832573 (0.0008) [2023-12-27 11:23:53,174][20381] Updated weights for policy 1, policy_version 831837 (0.0008) [2023-12-27 11:23:53,236][20381] Updated weights for policy 1, policy_version 831847 (0.0008) [2023-12-27 11:23:53,294][20381] Updated weights for policy 1, policy_version 831857 (0.0008) [2023-12-27 11:23:53,923][20347] Updated weights for policy 0, policy_version 832583 (0.0006) [2023-12-27 11:23:53,924][20381] Updated weights for policy 1, policy_version 831867 (0.0008) [2023-12-27 11:23:53,977][20347] Updated weights for policy 0, policy_version 832593 (0.0006) [2023-12-27 11:23:53,982][20381] Updated weights for policy 1, policy_version 831877 (0.0008) [2023-12-27 11:23:54,031][20347] Updated weights for policy 0, policy_version 832603 (0.0007) [2023-12-27 11:23:54,039][20381] Updated weights for policy 1, policy_version 831887 (0.0006) [2023-12-27 11:23:54,729][20347] Updated weights for policy 0, policy_version 832613 (0.0007) [2023-12-27 11:23:54,736][20381] Updated weights for policy 1, policy_version 831897 (0.0008) [2023-12-27 11:23:54,786][20347] Updated weights for policy 0, policy_version 832623 (0.0006) [2023-12-27 11:23:54,795][20381] Updated weights for policy 1, policy_version 831907 (0.0008) [2023-12-27 11:23:54,839][20347] Updated weights for policy 0, policy_version 832633 (0.0005) [2023-12-27 11:23:54,853][20381] Updated weights for policy 1, policy_version 831917 (0.0007) [2023-12-27 11:23:54,913][20381] Updated weights for policy 1, policy_version 831927 (0.0007) [2023-12-27 11:23:55,488][20347] Updated weights for policy 0, policy_version 832643 (0.0006) [2023-12-27 11:23:55,577][20347] Updated weights for policy 0, policy_version 832653 (0.0006) [2023-12-27 11:23:55,633][20347] Updated weights for policy 0, policy_version 832663 (0.0008) [2023-12-27 11:23:55,640][20381] Updated weights for policy 1, policy_version 831937 (0.0006) [2023-12-27 11:23:55,700][20381] Updated weights for policy 1, policy_version 831947 (0.0006) [2023-12-27 11:23:55,760][20381] Updated weights for policy 1, policy_version 831957 (0.0008) [2023-12-27 11:23:56,316][20347] Updated weights for policy 0, policy_version 832673 (0.0008) [2023-12-27 11:23:56,372][20347] Updated weights for policy 0, policy_version 832683 (0.0006) [2023-12-27 11:23:56,406][20381] Updated weights for policy 1, policy_version 831967 (0.0008) [2023-12-27 11:23:56,429][20347] Updated weights for policy 0, policy_version 832693 (0.0005) [2023-12-27 11:23:56,460][20381] Updated weights for policy 1, policy_version 831977 (0.0008) [2023-12-27 11:23:56,485][20347] Updated weights for policy 0, policy_version 832703 (0.0007) [2023-12-27 11:23:56,515][20381] Updated weights for policy 1, policy_version 831987 (0.0008) [2023-12-27 11:23:57,152][20347] Updated weights for policy 0, policy_version 832713 (0.0008) [2023-12-27 11:23:57,209][20347] Updated weights for policy 0, policy_version 832723 (0.0006) [2023-12-27 11:23:57,240][20381] Updated weights for policy 1, policy_version 831997 (0.0008) [2023-12-27 11:23:57,266][20347] Updated weights for policy 0, policy_version 832733 (0.0006) [2023-12-27 11:23:57,295][20381] Updated weights for policy 1, policy_version 832007 (0.0006) [2023-12-27 11:23:57,353][20381] Updated weights for policy 1, policy_version 832017 (0.0008) [2023-12-27 11:23:57,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 426221568. Throughput: 0: 9848.7, 1: 9647.9. Samples: 426213472. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:23:57,363][19170] Avg episode reward: [(0, '794.530'), (1, '1000.000')] [2023-12-27 11:23:57,958][20347] Updated weights for policy 0, policy_version 832743 (0.0008) [2023-12-27 11:23:58,018][20347] Updated weights for policy 0, policy_version 832753 (0.0007) [2023-12-27 11:23:58,037][20381] Updated weights for policy 1, policy_version 832027 (0.0006) [2023-12-27 11:23:58,075][20347] Updated weights for policy 0, policy_version 832763 (0.0006) [2023-12-27 11:23:58,082][20291] KL-divergence is very high: 118.3390 [2023-12-27 11:23:58,095][20381] Updated weights for policy 1, policy_version 832037 (0.0007) [2023-12-27 11:23:58,099][20291] KL-divergence is very high: 198.0820 [2023-12-27 11:23:58,154][20381] Updated weights for policy 1, policy_version 832047 (0.0007) [2023-12-27 11:23:58,878][20347] Updated weights for policy 0, policy_version 832773 (0.0007) [2023-12-27 11:23:58,937][20347] Updated weights for policy 0, policy_version 832783 (0.0008) [2023-12-27 11:23:58,945][20381] Updated weights for policy 1, policy_version 832057 (0.0009) [2023-12-27 11:23:59,005][20347] Updated weights for policy 0, policy_version 832793 (0.0008) [2023-12-27 11:23:59,013][20381] Updated weights for policy 1, policy_version 832067 (0.0008) [2023-12-27 11:23:59,076][20381] Updated weights for policy 1, policy_version 832077 (0.0008) [2023-12-27 11:23:59,138][20381] Updated weights for policy 1, policy_version 832087 (0.0008) [2023-12-27 11:23:59,689][20347] Updated weights for policy 0, policy_version 832803 (0.0007) [2023-12-27 11:23:59,751][20347] Updated weights for policy 0, policy_version 832813 (0.0008) [2023-12-27 11:23:59,813][20347] Updated weights for policy 0, policy_version 832823 (0.0007) [2023-12-27 11:23:59,877][20381] Updated weights for policy 1, policy_version 832097 (0.0009) [2023-12-27 11:23:59,945][20381] Updated weights for policy 1, policy_version 832107 (0.0009) [2023-12-27 11:24:00,012][20381] Updated weights for policy 1, policy_version 832117 (0.0007) [2023-12-27 11:24:00,511][20347] Updated weights for policy 0, policy_version 832833 (0.0008) [2023-12-27 11:24:00,566][20347] Updated weights for policy 0, policy_version 832843 (0.0007) [2023-12-27 11:24:00,623][20347] Updated weights for policy 0, policy_version 832853 (0.0005) [2023-12-27 11:24:00,682][20347] Updated weights for policy 0, policy_version 832863 (0.0005) [2023-12-27 11:24:00,738][20381] Updated weights for policy 1, policy_version 832127 (0.0006) [2023-12-27 11:24:00,792][20381] Updated weights for policy 1, policy_version 832137 (0.0006) [2023-12-27 11:24:00,847][20381] Updated weights for policy 1, policy_version 832147 (0.0005) [2023-12-27 11:24:01,351][20347] Updated weights for policy 0, policy_version 832873 (0.0007) [2023-12-27 11:24:01,424][20347] Updated weights for policy 0, policy_version 832883 (0.0008) [2023-12-27 11:24:01,481][20347] Updated weights for policy 0, policy_version 832893 (0.0007) [2023-12-27 11:24:01,559][20381] Updated weights for policy 1, policy_version 832157 (0.0006) [2023-12-27 11:24:01,622][20381] Updated weights for policy 1, policy_version 832167 (0.0008) [2023-12-27 11:24:01,677][20381] Updated weights for policy 1, policy_version 832177 (0.0008) [2023-12-27 11:24:02,216][20347] Updated weights for policy 0, policy_version 832903 (0.0009) [2023-12-27 11:24:02,281][20347] Updated weights for policy 0, policy_version 832913 (0.0007) [2023-12-27 11:24:02,333][20347] Updated weights for policy 0, policy_version 832923 (0.0008) [2023-12-27 11:24:02,349][20381] Updated weights for policy 1, policy_version 832187 (0.0008) [2023-12-27 11:24:02,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 426311680. Throughput: 0: 9848.1, 1: 9620.1. Samples: 426329564. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:24:02,363][19170] Avg episode reward: [(0, '863.160'), (1, '1000.000')] [2023-12-27 11:24:02,411][20381] Updated weights for policy 1, policy_version 832197 (0.0006) [2023-12-27 11:24:02,472][20381] Updated weights for policy 1, policy_version 832207 (0.0007) [2023-12-27 11:24:03,043][20347] Updated weights for policy 0, policy_version 832933 (0.0007) [2023-12-27 11:24:03,098][20347] Updated weights for policy 0, policy_version 832943 (0.0008) [2023-12-27 11:24:03,149][20381] Updated weights for policy 1, policy_version 832217 (0.0008) [2023-12-27 11:24:03,151][20347] Updated weights for policy 0, policy_version 832953 (0.0008) [2023-12-27 11:24:03,203][20381] Updated weights for policy 1, policy_version 832227 (0.0007) [2023-12-27 11:24:03,259][20381] Updated weights for policy 1, policy_version 832237 (0.0008) [2023-12-27 11:24:03,309][20381] Updated weights for policy 1, policy_version 832247 (0.0008) [2023-12-27 11:24:03,872][20347] Updated weights for policy 0, policy_version 832963 (0.0006) [2023-12-27 11:24:03,931][20347] Updated weights for policy 0, policy_version 832973 (0.0008) [2023-12-27 11:24:03,992][20347] Updated weights for policy 0, policy_version 832983 (0.0008) [2023-12-27 11:24:04,032][20381] Updated weights for policy 1, policy_version 832257 (0.0007) [2023-12-27 11:24:04,086][20381] Updated weights for policy 1, policy_version 832267 (0.0008) [2023-12-27 11:24:04,141][20381] Updated weights for policy 1, policy_version 832277 (0.0008) [2023-12-27 11:24:04,734][20347] Updated weights for policy 0, policy_version 832993 (0.0006) [2023-12-27 11:24:04,786][20347] Updated weights for policy 0, policy_version 833003 (0.0007) [2023-12-27 11:24:04,792][20381] Updated weights for policy 1, policy_version 832287 (0.0006) [2023-12-27 11:24:04,839][20347] Updated weights for policy 0, policy_version 833013 (0.0006) [2023-12-27 11:24:04,852][20381] Updated weights for policy 1, policy_version 832297 (0.0006) [2023-12-27 11:24:04,896][20347] Updated weights for policy 0, policy_version 833023 (0.0006) [2023-12-27 11:24:04,913][20381] Updated weights for policy 1, policy_version 832307 (0.0007) [2023-12-27 11:24:05,559][20381] Updated weights for policy 1, policy_version 832317 (0.0007) [2023-12-27 11:24:05,589][20347] Updated weights for policy 0, policy_version 833033 (0.0007) [2023-12-27 11:24:05,617][20381] Updated weights for policy 1, policy_version 832327 (0.0010) [2023-12-27 11:24:05,650][20347] Updated weights for policy 0, policy_version 833043 (0.0006) [2023-12-27 11:24:05,676][20381] Updated weights for policy 1, policy_version 832337 (0.0010) [2023-12-27 11:24:05,711][20347] Updated weights for policy 0, policy_version 833053 (0.0006) [2023-12-27 11:24:06,400][20381] Updated weights for policy 1, policy_version 832347 (0.0010) [2023-12-27 11:24:06,437][20347] Updated weights for policy 0, policy_version 833063 (0.0007) [2023-12-27 11:24:06,467][20381] Updated weights for policy 1, policy_version 832357 (0.0007) [2023-12-27 11:24:06,500][20347] Updated weights for policy 0, policy_version 833073 (0.0007) [2023-12-27 11:24:06,530][20381] Updated weights for policy 1, policy_version 832367 (0.0006) [2023-12-27 11:24:06,561][20347] Updated weights for policy 0, policy_version 833083 (0.0006) [2023-12-27 11:24:07,249][20381] Updated weights for policy 1, policy_version 832377 (0.0007) [2023-12-27 11:24:07,312][20347] Updated weights for policy 0, policy_version 833093 (0.0008) [2023-12-27 11:24:07,312][20381] Updated weights for policy 1, policy_version 832387 (0.0009) [2023-12-27 11:24:07,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 426409984. Throughput: 0: 9858.2, 1: 9602.2. Samples: 426388028. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:24:07,363][19170] Avg episode reward: [(0, '902.200'), (1, '990.300')] [2023-12-27 11:24:07,368][20381] Updated weights for policy 1, policy_version 832397 (0.0010) [2023-12-27 11:24:07,370][20347] Updated weights for policy 0, policy_version 833103 (0.0008) [2023-12-27 11:24:07,424][20347] Updated weights for policy 0, policy_version 833113 (0.0007) [2023-12-27 11:24:07,426][20381] Updated weights for policy 1, policy_version 832407 (0.0007) [2023-12-27 11:24:07,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000832408_213123072.pth... [2023-12-27 11:24:07,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000831256_212828160.pth [2023-12-27 11:24:07,463][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000833120_213303296.pth... [2023-12-27 11:24:07,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000831968_213008384.pth [2023-12-27 11:24:08,137][20381] Updated weights for policy 1, policy_version 832417 (0.0007) [2023-12-27 11:24:08,150][20347] Updated weights for policy 0, policy_version 833123 (0.0007) [2023-12-27 11:24:08,198][20381] Updated weights for policy 1, policy_version 832427 (0.0007) [2023-12-27 11:24:08,208][20347] Updated weights for policy 0, policy_version 833133 (0.0006) [2023-12-27 11:24:08,257][20381] Updated weights for policy 1, policy_version 832437 (0.0006) [2023-12-27 11:24:08,264][20347] Updated weights for policy 0, policy_version 833143 (0.0007) [2023-12-27 11:24:08,971][20381] Updated weights for policy 1, policy_version 832447 (0.0007) [2023-12-27 11:24:09,037][20381] Updated weights for policy 1, policy_version 832457 (0.0008) [2023-12-27 11:24:09,066][20347] Updated weights for policy 0, policy_version 833153 (0.0008) [2023-12-27 11:24:09,098][20381] Updated weights for policy 1, policy_version 832467 (0.0009) [2023-12-27 11:24:09,118][20347] Updated weights for policy 0, policy_version 833163 (0.0006) [2023-12-27 11:24:09,178][20347] Updated weights for policy 0, policy_version 833173 (0.0008) [2023-12-27 11:24:09,249][20347] Updated weights for policy 0, policy_version 833183 (0.0008) [2023-12-27 11:24:09,861][20381] Updated weights for policy 1, policy_version 832477 (0.0008) [2023-12-27 11:24:09,925][20381] Updated weights for policy 1, policy_version 832487 (0.0009) [2023-12-27 11:24:09,988][20381] Updated weights for policy 1, policy_version 832497 (0.0007) [2023-12-27 11:24:10,083][20347] Updated weights for policy 0, policy_version 833193 (0.0007) [2023-12-27 11:24:10,151][20347] Updated weights for policy 0, policy_version 833203 (0.0007) [2023-12-27 11:24:10,212][20347] Updated weights for policy 0, policy_version 833213 (0.0008) [2023-12-27 11:24:10,746][20381] Updated weights for policy 1, policy_version 832507 (0.0008) [2023-12-27 11:24:10,801][20381] Updated weights for policy 1, policy_version 832517 (0.0008) [2023-12-27 11:24:10,855][20381] Updated weights for policy 1, policy_version 832527 (0.0007) [2023-12-27 11:24:10,902][20347] Updated weights for policy 0, policy_version 833223 (0.0008) [2023-12-27 11:24:10,959][20347] Updated weights for policy 0, policy_version 833233 (0.0008) [2023-12-27 11:24:11,020][20347] Updated weights for policy 0, policy_version 833243 (0.0008) [2023-12-27 11:24:11,605][20381] Updated weights for policy 1, policy_version 832537 (0.0006) [2023-12-27 11:24:11,668][20381] Updated weights for policy 1, policy_version 832547 (0.0007) [2023-12-27 11:24:11,740][20381] Updated weights for policy 1, policy_version 832557 (0.0009) [2023-12-27 11:24:11,803][20381] Updated weights for policy 1, policy_version 832567 (0.0008) [2023-12-27 11:24:11,854][20347] Updated weights for policy 0, policy_version 833253 (0.0008) [2023-12-27 11:24:11,914][20347] Updated weights for policy 0, policy_version 833263 (0.0008) [2023-12-27 11:24:11,976][20347] Updated weights for policy 0, policy_version 833273 (0.0007) [2023-12-27 11:24:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 426508288. Throughput: 0: 9782.6, 1: 9616.2. Samples: 426501604. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:24:12,363][19170] Avg episode reward: [(0, '853.300'), (1, '990.300')] [2023-12-27 11:24:12,523][20381] Updated weights for policy 1, policy_version 832577 (0.0008) [2023-12-27 11:24:12,585][20381] Updated weights for policy 1, policy_version 832587 (0.0008) [2023-12-27 11:24:12,647][20381] Updated weights for policy 1, policy_version 832597 (0.0008) [2023-12-27 11:24:12,707][20347] Updated weights for policy 0, policy_version 833283 (0.0007) [2023-12-27 11:24:12,768][20347] Updated weights for policy 0, policy_version 833293 (0.0009) [2023-12-27 11:24:12,832][20347] Updated weights for policy 0, policy_version 833303 (0.0007) [2023-12-27 11:24:13,385][20381] Updated weights for policy 1, policy_version 832607 (0.0008) [2023-12-27 11:24:13,447][20381] Updated weights for policy 1, policy_version 832617 (0.0007) [2023-12-27 11:24:13,501][20381] Updated weights for policy 1, policy_version 832627 (0.0008) [2023-12-27 11:24:13,521][20347] Updated weights for policy 0, policy_version 833313 (0.0007) [2023-12-27 11:24:13,586][20347] Updated weights for policy 0, policy_version 833323 (0.0006) [2023-12-27 11:24:13,641][20347] Updated weights for policy 0, policy_version 833333 (0.0006) [2023-12-27 11:24:13,697][20347] Updated weights for policy 0, policy_version 833343 (0.0006) [2023-12-27 11:24:14,163][20381] Updated weights for policy 1, policy_version 832637 (0.0006) [2023-12-27 11:24:14,228][20381] Updated weights for policy 1, policy_version 832647 (0.0008) [2023-12-27 11:24:14,288][20381] Updated weights for policy 1, policy_version 832657 (0.0007) [2023-12-27 11:24:14,378][20347] Updated weights for policy 0, policy_version 833353 (0.0008) [2023-12-27 11:24:14,431][20347] Updated weights for policy 0, policy_version 833363 (0.0007) [2023-12-27 11:24:14,491][20347] Updated weights for policy 0, policy_version 833373 (0.0007) [2023-12-27 11:24:14,915][20381] Updated weights for policy 1, policy_version 832667 (0.0008) [2023-12-27 11:24:14,960][20292] KL-divergence is very high: 139.8041 [2023-12-27 11:24:14,979][20381] Updated weights for policy 1, policy_version 832677 (0.0007) [2023-12-27 11:24:15,046][20381] Updated weights for policy 1, policy_version 832687 (0.0009) [2023-12-27 11:24:15,216][20347] Updated weights for policy 0, policy_version 833383 (0.0008) [2023-12-27 11:24:15,272][20347] Updated weights for policy 0, policy_version 833393 (0.0009) [2023-12-27 11:24:15,330][20347] Updated weights for policy 0, policy_version 833403 (0.0008) [2023-12-27 11:24:15,727][20381] Updated weights for policy 1, policy_version 832697 (0.0007) [2023-12-27 11:24:15,784][20381] Updated weights for policy 1, policy_version 832707 (0.0011) [2023-12-27 11:24:15,838][20381] Updated weights for policy 1, policy_version 832717 (0.0006) [2023-12-27 11:24:15,893][20381] Updated weights for policy 1, policy_version 832727 (0.0006) [2023-12-27 11:24:16,072][20347] Updated weights for policy 0, policy_version 833413 (0.0007) [2023-12-27 11:24:16,130][20347] Updated weights for policy 0, policy_version 833423 (0.0010) [2023-12-27 11:24:16,191][20347] Updated weights for policy 0, policy_version 833433 (0.0006) [2023-12-27 11:24:16,559][20381] Updated weights for policy 1, policy_version 832737 (0.0007) [2023-12-27 11:24:16,614][20381] Updated weights for policy 1, policy_version 832747 (0.0008) [2023-12-27 11:24:16,676][20381] Updated weights for policy 1, policy_version 832757 (0.0005) [2023-12-27 11:24:16,851][20347] Updated weights for policy 0, policy_version 833443 (0.0008) [2023-12-27 11:24:16,906][20347] Updated weights for policy 0, policy_version 833453 (0.0010) [2023-12-27 11:24:16,966][20347] Updated weights for policy 0, policy_version 833463 (0.0010) [2023-12-27 11:24:17,293][20381] Updated weights for policy 1, policy_version 832767 (0.0007) [2023-12-27 11:24:17,343][20381] Updated weights for policy 1, policy_version 832777 (0.0009) [2023-12-27 11:24:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 426606592. Throughput: 0: 9716.7, 1: 9671.1. Samples: 426619680. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:24:17,363][19170] Avg episode reward: [(0, '833.790'), (1, '990.270')] [2023-12-27 11:24:17,396][20381] Updated weights for policy 1, policy_version 832787 (0.0008) [2023-12-27 11:24:17,620][20347] Updated weights for policy 0, policy_version 833473 (0.0010) [2023-12-27 11:24:17,686][20347] Updated weights for policy 0, policy_version 833483 (0.0010) [2023-12-27 11:24:17,743][20347] Updated weights for policy 0, policy_version 833493 (0.0011) [2023-12-27 11:24:17,802][20347] Updated weights for policy 0, policy_version 833503 (0.0010) [2023-12-27 11:24:18,079][20381] Updated weights for policy 1, policy_version 832797 (0.0008) [2023-12-27 11:24:18,130][20381] Updated weights for policy 1, policy_version 832807 (0.0008) [2023-12-27 11:24:18,194][20381] Updated weights for policy 1, policy_version 832817 (0.0008) [2023-12-27 11:24:18,456][20347] Updated weights for policy 0, policy_version 833513 (0.0008) [2023-12-27 11:24:18,514][20347] Updated weights for policy 0, policy_version 833523 (0.0009) [2023-12-27 11:24:18,579][20347] Updated weights for policy 0, policy_version 833533 (0.0008) [2023-12-27 11:24:18,922][20381] Updated weights for policy 1, policy_version 832827 (0.0008) [2023-12-27 11:24:18,987][20381] Updated weights for policy 1, policy_version 832837 (0.0008) [2023-12-27 11:24:19,043][20381] Updated weights for policy 1, policy_version 832847 (0.0007) [2023-12-27 11:24:19,290][20347] Updated weights for policy 0, policy_version 833543 (0.0009) [2023-12-27 11:24:19,358][20347] Updated weights for policy 0, policy_version 833553 (0.0008) [2023-12-27 11:24:19,432][20347] Updated weights for policy 0, policy_version 833563 (0.0008) [2023-12-27 11:24:19,757][20381] Updated weights for policy 1, policy_version 832857 (0.0006) [2023-12-27 11:24:19,819][20381] Updated weights for policy 1, policy_version 832867 (0.0008) [2023-12-27 11:24:19,886][20381] Updated weights for policy 1, policy_version 832877 (0.0008) [2023-12-27 11:24:19,955][20381] Updated weights for policy 1, policy_version 832887 (0.0008) [2023-12-27 11:24:20,198][20347] Updated weights for policy 0, policy_version 833573 (0.0008) [2023-12-27 11:24:20,257][20347] Updated weights for policy 0, policy_version 833583 (0.0009) [2023-12-27 11:24:20,318][20347] Updated weights for policy 0, policy_version 833593 (0.0008) [2023-12-27 11:24:20,662][20381] Updated weights for policy 1, policy_version 832897 (0.0008) [2023-12-27 11:24:20,726][20381] Updated weights for policy 1, policy_version 832907 (0.0010) [2023-12-27 11:24:20,790][20381] Updated weights for policy 1, policy_version 832917 (0.0010) [2023-12-27 11:24:21,056][20347] Updated weights for policy 0, policy_version 833603 (0.0007) [2023-12-27 11:24:21,119][20347] Updated weights for policy 0, policy_version 833613 (0.0006) [2023-12-27 11:24:21,192][20347] Updated weights for policy 0, policy_version 833623 (0.0009) [2023-12-27 11:24:21,554][20381] Updated weights for policy 1, policy_version 832927 (0.0008) [2023-12-27 11:24:21,623][20381] Updated weights for policy 1, policy_version 832937 (0.0008) [2023-12-27 11:24:21,689][20381] Updated weights for policy 1, policy_version 832947 (0.0006) [2023-12-27 11:24:21,996][20347] Updated weights for policy 0, policy_version 833633 (0.0007) [2023-12-27 11:24:22,060][20347] Updated weights for policy 0, policy_version 833643 (0.0008) [2023-12-27 11:24:22,128][20347] Updated weights for policy 0, policy_version 833653 (0.0008) [2023-12-27 11:24:22,196][20347] Updated weights for policy 0, policy_version 833663 (0.0006) [2023-12-27 11:24:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 426704896. Throughput: 0: 9715.5, 1: 9652.8. Samples: 426678272. Policy #0 lag: (min: 36.0, avg: 55.1, max: 56.0) [2023-12-27 11:24:22,363][19170] Avg episode reward: [(0, '794.360'), (1, '990.270')] [2023-12-27 11:24:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000832952_213262336.pth... [2023-12-27 11:24:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000833664_213442560.pth... [2023-12-27 11:24:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000831832_212975616.pth [2023-12-27 11:24:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000832544_213155840.pth [2023-12-27 11:24:22,466][20381] Updated weights for policy 1, policy_version 832957 (0.0007) [2023-12-27 11:24:22,527][20381] Updated weights for policy 1, policy_version 832967 (0.0009) [2023-12-27 11:24:22,592][20381] Updated weights for policy 1, policy_version 832977 (0.0008) [2023-12-27 11:24:22,902][20347] Updated weights for policy 0, policy_version 833673 (0.0010) [2023-12-27 11:24:22,960][20347] Updated weights for policy 0, policy_version 833683 (0.0010) [2023-12-27 11:24:23,023][20347] Updated weights for policy 0, policy_version 833693 (0.0010) [2023-12-27 11:24:23,319][20381] Updated weights for policy 1, policy_version 832987 (0.0007) [2023-12-27 11:24:23,372][20381] Updated weights for policy 1, policy_version 832997 (0.0008) [2023-12-27 11:24:23,429][20381] Updated weights for policy 1, policy_version 833007 (0.0011) [2023-12-27 11:24:23,767][20347] Updated weights for policy 0, policy_version 833703 (0.0010) [2023-12-27 11:24:23,818][20347] Updated weights for policy 0, policy_version 833713 (0.0010) [2023-12-27 11:24:23,865][20347] Updated weights for policy 0, policy_version 833723 (0.0011) [2023-12-27 11:24:24,121][20381] Updated weights for policy 1, policy_version 833017 (0.0010) [2023-12-27 11:24:24,178][20381] Updated weights for policy 1, policy_version 833027 (0.0009) [2023-12-27 11:24:24,242][20381] Updated weights for policy 1, policy_version 833037 (0.0011) [2023-12-27 11:24:24,307][20381] Updated weights for policy 1, policy_version 833047 (0.0006) [2023-12-27 11:24:24,573][20347] Updated weights for policy 0, policy_version 833733 (0.0010) [2023-12-27 11:24:24,632][20347] Updated weights for policy 0, policy_version 833743 (0.0010) [2023-12-27 11:24:24,687][20347] Updated weights for policy 0, policy_version 833753 (0.0010) [2023-12-27 11:24:24,944][20381] Updated weights for policy 1, policy_version 833057 (0.0008) [2023-12-27 11:24:25,000][20381] Updated weights for policy 1, policy_version 833067 (0.0010) [2023-12-27 11:24:25,060][20381] Updated weights for policy 1, policy_version 833077 (0.0011) [2023-12-27 11:24:25,375][20347] Updated weights for policy 0, policy_version 833763 (0.0009) [2023-12-27 11:24:25,442][20347] Updated weights for policy 0, policy_version 833773 (0.0008) [2023-12-27 11:24:25,508][20347] Updated weights for policy 0, policy_version 833783 (0.0007) [2023-12-27 11:24:25,745][20381] Updated weights for policy 1, policy_version 833087 (0.0008) [2023-12-27 11:24:25,801][20381] Updated weights for policy 1, policy_version 833097 (0.0009) [2023-12-27 11:24:25,866][20381] Updated weights for policy 1, policy_version 833107 (0.0010) [2023-12-27 11:24:26,087][20347] Updated weights for policy 0, policy_version 833793 (0.0010) [2023-12-27 11:24:26,156][20347] Updated weights for policy 0, policy_version 833803 (0.0009) [2023-12-27 11:24:26,212][20347] Updated weights for policy 0, policy_version 833813 (0.0010) [2023-12-27 11:24:26,273][20347] Updated weights for policy 0, policy_version 833823 (0.0011) [2023-12-27 11:24:26,602][20381] Updated weights for policy 1, policy_version 833117 (0.0009) [2023-12-27 11:24:26,650][20381] Updated weights for policy 1, policy_version 833127 (0.0007) [2023-12-27 11:24:26,709][20381] Updated weights for policy 1, policy_version 833137 (0.0007) [2023-12-27 11:24:26,917][20347] Updated weights for policy 0, policy_version 833833 (0.0008) [2023-12-27 11:24:26,972][20347] Updated weights for policy 0, policy_version 833843 (0.0010) [2023-12-27 11:24:27,029][20347] Updated weights for policy 0, policy_version 833853 (0.0010) [2023-12-27 11:24:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19524.2, 300 sec: 19410.9). Total num frames: 426803200. Throughput: 0: 9674.7, 1: 9696.6. Samples: 426794040. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:27,363][19170] Avg episode reward: [(0, '804.470'), (1, '990.250')] [2023-12-27 11:24:27,455][20381] Updated weights for policy 1, policy_version 833147 (0.0007) [2023-12-27 11:24:27,507][20381] Updated weights for policy 1, policy_version 833157 (0.0008) [2023-12-27 11:24:27,559][20381] Updated weights for policy 1, policy_version 833167 (0.0007) [2023-12-27 11:24:27,674][20347] Updated weights for policy 0, policy_version 833863 (0.0009) [2023-12-27 11:24:27,725][20347] Updated weights for policy 0, policy_version 833873 (0.0010) [2023-12-27 11:24:27,783][20347] Updated weights for policy 0, policy_version 833883 (0.0009) [2023-12-27 11:24:28,224][20381] Updated weights for policy 1, policy_version 833177 (0.0007) [2023-12-27 11:24:28,280][20381] Updated weights for policy 1, policy_version 833187 (0.0008) [2023-12-27 11:24:28,343][20381] Updated weights for policy 1, policy_version 833197 (0.0007) [2023-12-27 11:24:28,401][20381] Updated weights for policy 1, policy_version 833207 (0.0008) [2023-12-27 11:24:28,449][20347] Updated weights for policy 0, policy_version 833893 (0.0007) [2023-12-27 11:24:28,511][20347] Updated weights for policy 0, policy_version 833903 (0.0011) [2023-12-27 11:24:28,571][20347] Updated weights for policy 0, policy_version 833913 (0.0010) [2023-12-27 11:24:29,159][20381] Updated weights for policy 1, policy_version 833217 (0.0008) [2023-12-27 11:24:29,211][20381] Updated weights for policy 1, policy_version 833227 (0.0008) [2023-12-27 11:24:29,275][20381] Updated weights for policy 1, policy_version 833237 (0.0009) [2023-12-27 11:24:29,281][20347] Updated weights for policy 0, policy_version 833923 (0.0008) [2023-12-27 11:24:29,346][20347] Updated weights for policy 0, policy_version 833933 (0.0008) [2023-12-27 11:24:29,413][20347] Updated weights for policy 0, policy_version 833943 (0.0008) [2023-12-27 11:24:30,104][20381] Updated weights for policy 1, policy_version 833247 (0.0007) [2023-12-27 11:24:30,145][20347] Updated weights for policy 0, policy_version 833953 (0.0009) [2023-12-27 11:24:30,163][20381] Updated weights for policy 1, policy_version 833257 (0.0006) [2023-12-27 11:24:30,202][20347] Updated weights for policy 0, policy_version 833963 (0.0008) [2023-12-27 11:24:30,227][20381] Updated weights for policy 1, policy_version 833267 (0.0006) [2023-12-27 11:24:30,260][20347] Updated weights for policy 0, policy_version 833973 (0.0009) [2023-12-27 11:24:30,325][20347] Updated weights for policy 0, policy_version 833983 (0.0010) [2023-12-27 11:24:30,959][20381] Updated weights for policy 1, policy_version 833277 (0.0007) [2023-12-27 11:24:31,015][20381] Updated weights for policy 1, policy_version 833287 (0.0008) [2023-12-27 11:24:31,052][20347] Updated weights for policy 0, policy_version 833993 (0.0011) [2023-12-27 11:24:31,076][20381] Updated weights for policy 1, policy_version 833297 (0.0007) [2023-12-27 11:24:31,108][20347] Updated weights for policy 0, policy_version 834003 (0.0011) [2023-12-27 11:24:31,176][20347] Updated weights for policy 0, policy_version 834013 (0.0011) [2023-12-27 11:24:31,881][20381] Updated weights for policy 1, policy_version 833307 (0.0007) [2023-12-27 11:24:31,935][20347] Updated weights for policy 0, policy_version 834023 (0.0010) [2023-12-27 11:24:31,938][20381] Updated weights for policy 1, policy_version 833317 (0.0006) [2023-12-27 11:24:31,989][20347] Updated weights for policy 0, policy_version 834033 (0.0011) [2023-12-27 11:24:31,995][20381] Updated weights for policy 1, policy_version 833327 (0.0006) [2023-12-27 11:24:32,050][20347] Updated weights for policy 0, policy_version 834043 (0.0010) [2023-12-27 11:24:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 426901504. Throughput: 0: 9671.9, 1: 9670.0. Samples: 426910180. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:32,363][19170] Avg episode reward: [(0, '784.870'), (1, '980.550')] [2023-12-27 11:24:32,718][20381] Updated weights for policy 1, policy_version 833337 (0.0006) [2023-12-27 11:24:32,762][20347] Updated weights for policy 0, policy_version 834053 (0.0011) [2023-12-27 11:24:32,779][20381] Updated weights for policy 1, policy_version 833347 (0.0006) [2023-12-27 11:24:32,813][20291] KL-divergence is very high: 285.6488 [2023-12-27 11:24:32,826][20347] Updated weights for policy 0, policy_version 834063 (0.0010) [2023-12-27 11:24:32,843][20381] Updated weights for policy 1, policy_version 833357 (0.0007) [2023-12-27 11:24:32,859][20291] KL-divergence is very high: 283.1844 [2023-12-27 11:24:32,882][20347] Updated weights for policy 0, policy_version 834073 (0.0009) [2023-12-27 11:24:32,902][20381] Updated weights for policy 1, policy_version 833367 (0.0009) [2023-12-27 11:24:32,906][20291] KL-divergence is very high: 149.9617 [2023-12-27 11:24:33,559][20347] Updated weights for policy 0, policy_version 834083 (0.0007) [2023-12-27 11:24:33,594][20381] Updated weights for policy 1, policy_version 833377 (0.0006) [2023-12-27 11:24:33,619][20347] Updated weights for policy 0, policy_version 834093 (0.0008) [2023-12-27 11:24:33,652][20381] Updated weights for policy 1, policy_version 833387 (0.0006) [2023-12-27 11:24:33,675][20347] Updated weights for policy 0, policy_version 834103 (0.0007) [2023-12-27 11:24:33,709][20381] Updated weights for policy 1, policy_version 833397 (0.0006) [2023-12-27 11:24:34,358][20381] Updated weights for policy 1, policy_version 833407 (0.0008) [2023-12-27 11:24:34,366][20347] Updated weights for policy 0, policy_version 834113 (0.0008) [2023-12-27 11:24:34,412][20381] Updated weights for policy 1, policy_version 833417 (0.0007) [2023-12-27 11:24:34,425][20347] Updated weights for policy 0, policy_version 834123 (0.0008) [2023-12-27 11:24:34,462][20381] Updated weights for policy 1, policy_version 833427 (0.0005) [2023-12-27 11:24:34,479][20347] Updated weights for policy 0, policy_version 834133 (0.0006) [2023-12-27 11:24:34,539][20347] Updated weights for policy 0, policy_version 834143 (0.0006) [2023-12-27 11:24:35,176][20347] Updated weights for policy 0, policy_version 834153 (0.0010) [2023-12-27 11:24:35,221][20381] Updated weights for policy 1, policy_version 833437 (0.0007) [2023-12-27 11:24:35,232][20347] Updated weights for policy 0, policy_version 834163 (0.0007) [2023-12-27 11:24:35,286][20381] Updated weights for policy 1, policy_version 833447 (0.0007) [2023-12-27 11:24:35,291][20347] Updated weights for policy 0, policy_version 834173 (0.0006) [2023-12-27 11:24:35,350][20381] Updated weights for policy 1, policy_version 833457 (0.0006) [2023-12-27 11:24:35,881][20347] Updated weights for policy 0, policy_version 834183 (0.0007) [2023-12-27 11:24:35,931][20347] Updated weights for policy 0, policy_version 834193 (0.0008) [2023-12-27 11:24:35,986][20347] Updated weights for policy 0, policy_version 834203 (0.0006) [2023-12-27 11:24:36,012][20381] Updated weights for policy 1, policy_version 833467 (0.0008) [2023-12-27 11:24:36,070][20381] Updated weights for policy 1, policy_version 833477 (0.0011) [2023-12-27 11:24:36,140][20381] Updated weights for policy 1, policy_version 833488 (0.0009) [2023-12-27 11:24:36,660][20347] Updated weights for policy 0, policy_version 834213 (0.0007) [2023-12-27 11:24:36,723][20347] Updated weights for policy 0, policy_version 834223 (0.0007) [2023-12-27 11:24:36,791][20347] Updated weights for policy 0, policy_version 834233 (0.0008) [2023-12-27 11:24:36,925][20381] Updated weights for policy 1, policy_version 833498 (0.0008) [2023-12-27 11:24:36,989][20381] Updated weights for policy 1, policy_version 833508 (0.0006) [2023-12-27 11:24:37,050][20381] Updated weights for policy 1, policy_version 833518 (0.0008) [2023-12-27 11:24:37,118][20381] Updated weights for policy 1, policy_version 833528 (0.0007) [2023-12-27 11:24:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 426999808. Throughput: 0: 9704.1, 1: 9686.3. Samples: 426970304. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:37,363][19170] Avg episode reward: [(0, '764.620'), (1, '980.580')] [2023-12-27 11:24:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000834240_213590016.pth... [2023-12-27 11:24:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000833528_213409792.pth... [2023-12-27 11:24:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000833120_213303296.pth [2023-12-27 11:24:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000832408_213123072.pth [2023-12-27 11:24:37,502][20347] Updated weights for policy 0, policy_version 834243 (0.0008) [2023-12-27 11:24:37,563][20347] Updated weights for policy 0, policy_version 834253 (0.0008) [2023-12-27 11:24:37,626][20347] Updated weights for policy 0, policy_version 834263 (0.0008) [2023-12-27 11:24:37,657][20291] KL-divergence is very high: 128.4384 [2023-12-27 11:24:37,669][20291] KL-divergence is very high: 274.9955 [2023-12-27 11:24:37,675][20291] KL-divergence is very high: 169.7048 [2023-12-27 11:24:37,793][20381] Updated weights for policy 1, policy_version 833538 (0.0008) [2023-12-27 11:24:37,848][20381] Updated weights for policy 1, policy_version 833548 (0.0006) [2023-12-27 11:24:37,904][20381] Updated weights for policy 1, policy_version 833558 (0.0007) [2023-12-27 11:24:38,396][20291] KL-divergence is very high: 117.1411 [2023-12-27 11:24:38,396][20347] Updated weights for policy 0, policy_version 834273 (0.0008) [2023-12-27 11:24:38,461][20347] Updated weights for policy 0, policy_version 834283 (0.0011) [2023-12-27 11:24:38,528][20347] Updated weights for policy 0, policy_version 834293 (0.0010) [2023-12-27 11:24:38,594][20347] Updated weights for policy 0, policy_version 834303 (0.0011) [2023-12-27 11:24:38,607][20381] Updated weights for policy 1, policy_version 833568 (0.0006) [2023-12-27 11:24:38,671][20381] Updated weights for policy 1, policy_version 833578 (0.0007) [2023-12-27 11:24:38,731][20381] Updated weights for policy 1, policy_version 833588 (0.0008) [2023-12-27 11:24:39,297][20347] Updated weights for policy 0, policy_version 834313 (0.0008) [2023-12-27 11:24:39,366][20347] Updated weights for policy 0, policy_version 834323 (0.0008) [2023-12-27 11:24:39,416][20381] Updated weights for policy 1, policy_version 833598 (0.0007) [2023-12-27 11:24:39,425][20347] Updated weights for policy 0, policy_version 834333 (0.0007) [2023-12-27 11:24:39,471][20381] Updated weights for policy 1, policy_version 833608 (0.0007) [2023-12-27 11:24:39,533][20381] Updated weights for policy 1, policy_version 833618 (0.0008) [2023-12-27 11:24:40,204][20347] Updated weights for policy 0, policy_version 834343 (0.0007) [2023-12-27 11:24:40,272][20347] Updated weights for policy 0, policy_version 834353 (0.0008) [2023-12-27 11:24:40,325][20381] Updated weights for policy 1, policy_version 833628 (0.0008) [2023-12-27 11:24:40,335][20347] Updated weights for policy 0, policy_version 834363 (0.0008) [2023-12-27 11:24:40,388][20381] Updated weights for policy 1, policy_version 833638 (0.0008) [2023-12-27 11:24:40,447][20381] Updated weights for policy 1, policy_version 833648 (0.0008) [2023-12-27 11:24:41,099][20347] Updated weights for policy 0, policy_version 834373 (0.0008) [2023-12-27 11:24:41,132][20381] Updated weights for policy 1, policy_version 833658 (0.0006) [2023-12-27 11:24:41,168][20347] Updated weights for policy 0, policy_version 834383 (0.0011) [2023-12-27 11:24:41,205][20381] Updated weights for policy 1, policy_version 833668 (0.0009) [2023-12-27 11:24:41,229][20347] Updated weights for policy 0, policy_version 834393 (0.0009) [2023-12-27 11:24:41,268][20381] Updated weights for policy 1, policy_version 833678 (0.0009) [2023-12-27 11:24:41,326][20381] Updated weights for policy 1, policy_version 833688 (0.0009) [2023-12-27 11:24:42,089][20347] Updated weights for policy 0, policy_version 834403 (0.0008) [2023-12-27 11:24:42,142][20381] Updated weights for policy 1, policy_version 833698 (0.0010) [2023-12-27 11:24:42,153][20347] Updated weights for policy 0, policy_version 834413 (0.0009) [2023-12-27 11:24:42,203][20381] Updated weights for policy 1, policy_version 833708 (0.0006) [2023-12-27 11:24:42,212][20347] Updated weights for policy 0, policy_version 834423 (0.0008) [2023-12-27 11:24:42,263][20381] Updated weights for policy 1, policy_version 833718 (0.0007) [2023-12-27 11:24:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 427098112. Throughput: 0: 9706.8, 1: 9689.7. Samples: 427086316. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:42,363][19170] Avg episode reward: [(0, '705.870'), (1, '980.560')] [2023-12-27 11:24:42,948][20347] Updated weights for policy 0, policy_version 834433 (0.0007) [2023-12-27 11:24:43,002][20347] Updated weights for policy 0, policy_version 834443 (0.0007) [2023-12-27 11:24:43,015][20381] Updated weights for policy 1, policy_version 833728 (0.0007) [2023-12-27 11:24:43,056][20347] Updated weights for policy 0, policy_version 834453 (0.0006) [2023-12-27 11:24:43,074][20381] Updated weights for policy 1, policy_version 833738 (0.0007) [2023-12-27 11:24:43,111][20347] Updated weights for policy 0, policy_version 834463 (0.0006) [2023-12-27 11:24:43,135][20381] Updated weights for policy 1, policy_version 833748 (0.0007) [2023-12-27 11:24:43,844][20381] Updated weights for policy 1, policy_version 833758 (0.0007) [2023-12-27 11:24:43,848][20347] Updated weights for policy 0, policy_version 834473 (0.0007) [2023-12-27 11:24:43,902][20381] Updated weights for policy 1, policy_version 833768 (0.0009) [2023-12-27 11:24:43,908][20347] Updated weights for policy 0, policy_version 834483 (0.0006) [2023-12-27 11:24:43,967][20347] Updated weights for policy 0, policy_version 834493 (0.0006) [2023-12-27 11:24:43,967][20381] Updated weights for policy 1, policy_version 833778 (0.0007) [2023-12-27 11:24:44,610][20347] Updated weights for policy 0, policy_version 834503 (0.0008) [2023-12-27 11:24:44,650][20381] Updated weights for policy 1, policy_version 833788 (0.0007) [2023-12-27 11:24:44,662][20347] Updated weights for policy 0, policy_version 834513 (0.0007) [2023-12-27 11:24:44,705][20381] Updated weights for policy 1, policy_version 833798 (0.0007) [2023-12-27 11:24:44,714][20347] Updated weights for policy 0, policy_version 834523 (0.0008) [2023-12-27 11:24:44,762][20381] Updated weights for policy 1, policy_version 833808 (0.0006) [2023-12-27 11:24:45,431][20347] Updated weights for policy 0, policy_version 834533 (0.0007) [2023-12-27 11:24:45,481][20381] Updated weights for policy 1, policy_version 833818 (0.0008) [2023-12-27 11:24:45,488][20347] Updated weights for policy 0, policy_version 834543 (0.0008) [2023-12-27 11:24:45,543][20381] Updated weights for policy 1, policy_version 833828 (0.0006) [2023-12-27 11:24:45,551][20347] Updated weights for policy 0, policy_version 834553 (0.0008) [2023-12-27 11:24:45,604][20381] Updated weights for policy 1, policy_version 833838 (0.0007) [2023-12-27 11:24:45,664][20381] Updated weights for policy 1, policy_version 833848 (0.0008) [2023-12-27 11:24:46,239][20347] Updated weights for policy 0, policy_version 834563 (0.0006) [2023-12-27 11:24:46,299][20347] Updated weights for policy 0, policy_version 834573 (0.0007) [2023-12-27 11:24:46,351][20347] Updated weights for policy 0, policy_version 834583 (0.0007) [2023-12-27 11:24:46,389][20381] Updated weights for policy 1, policy_version 833858 (0.0008) [2023-12-27 11:24:46,447][20381] Updated weights for policy 1, policy_version 833868 (0.0008) [2023-12-27 11:24:46,504][20381] Updated weights for policy 1, policy_version 833878 (0.0008) [2023-12-27 11:24:47,040][20347] Updated weights for policy 0, policy_version 834593 (0.0007) [2023-12-27 11:24:47,099][20347] Updated weights for policy 0, policy_version 834603 (0.0005) [2023-12-27 11:24:47,155][20347] Updated weights for policy 0, policy_version 834613 (0.0006) [2023-12-27 11:24:47,208][20347] Updated weights for policy 0, policy_version 834623 (0.0007) [2023-12-27 11:24:47,255][20381] Updated weights for policy 1, policy_version 833888 (0.0008) [2023-12-27 11:24:47,315][20381] Updated weights for policy 1, policy_version 833898 (0.0008) [2023-12-27 11:24:47,345][20292] KL-divergence is very high: 155.1273 [2023-12-27 11:24:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 427188224. Throughput: 0: 9688.0, 1: 9667.9. Samples: 427200580. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:47,363][19170] Avg episode reward: [(0, '804.050'), (1, '961.300')] [2023-12-27 11:24:47,366][20381] Updated weights for policy 1, policy_version 833908 (0.0008) [2023-12-27 11:24:47,903][20347] Updated weights for policy 0, policy_version 834633 (0.0007) [2023-12-27 11:24:47,965][20347] Updated weights for policy 0, policy_version 834643 (0.0007) [2023-12-27 11:24:48,028][20347] Updated weights for policy 0, policy_version 834653 (0.0007) [2023-12-27 11:24:48,079][20381] Updated weights for policy 1, policy_version 833918 (0.0007) [2023-12-27 11:24:48,142][20381] Updated weights for policy 1, policy_version 833928 (0.0008) [2023-12-27 11:24:48,205][20381] Updated weights for policy 1, policy_version 833938 (0.0008) [2023-12-27 11:24:48,794][20347] Updated weights for policy 0, policy_version 834663 (0.0007) [2023-12-27 11:24:48,857][20347] Updated weights for policy 0, policy_version 834673 (0.0008) [2023-12-27 11:24:48,921][20347] Updated weights for policy 0, policy_version 834683 (0.0008) [2023-12-27 11:24:48,977][20381] Updated weights for policy 1, policy_version 833948 (0.0007) [2023-12-27 11:24:49,043][20381] Updated weights for policy 1, policy_version 833958 (0.0007) [2023-12-27 11:24:49,094][20381] Updated weights for policy 1, policy_version 833968 (0.0008) [2023-12-27 11:24:49,660][20347] Updated weights for policy 0, policy_version 834693 (0.0008) [2023-12-27 11:24:49,718][20347] Updated weights for policy 0, policy_version 834703 (0.0007) [2023-12-27 11:24:49,775][20347] Updated weights for policy 0, policy_version 834713 (0.0008) [2023-12-27 11:24:49,890][20381] Updated weights for policy 1, policy_version 833978 (0.0008) [2023-12-27 11:24:49,960][20381] Updated weights for policy 1, policy_version 833988 (0.0008) [2023-12-27 11:24:50,026][20381] Updated weights for policy 1, policy_version 833998 (0.0008) [2023-12-27 11:24:50,087][20381] Updated weights for policy 1, policy_version 834008 (0.0008) [2023-12-27 11:24:50,486][20347] Updated weights for policy 0, policy_version 834723 (0.0008) [2023-12-27 11:24:50,539][20347] Updated weights for policy 0, policy_version 834733 (0.0010) [2023-12-27 11:24:50,609][20347] Updated weights for policy 0, policy_version 834743 (0.0009) [2023-12-27 11:24:50,854][20381] Updated weights for policy 1, policy_version 834018 (0.0008) [2023-12-27 11:24:50,920][20381] Updated weights for policy 1, policy_version 834028 (0.0009) [2023-12-27 11:24:50,988][20381] Updated weights for policy 1, policy_version 834038 (0.0007) [2023-12-27 11:24:51,320][20347] Updated weights for policy 0, policy_version 834753 (0.0007) [2023-12-27 11:24:51,391][20347] Updated weights for policy 0, policy_version 834763 (0.0007) [2023-12-27 11:24:51,460][20347] Updated weights for policy 0, policy_version 834773 (0.0008) [2023-12-27 11:24:51,528][20347] Updated weights for policy 0, policy_version 834783 (0.0007) [2023-12-27 11:24:51,742][20381] Updated weights for policy 1, policy_version 834048 (0.0007) [2023-12-27 11:24:51,804][20381] Updated weights for policy 1, policy_version 834058 (0.0009) [2023-12-27 11:24:51,869][20381] Updated weights for policy 1, policy_version 834068 (0.0008) [2023-12-27 11:24:52,293][20347] Updated weights for policy 0, policy_version 834793 (0.0008) [2023-12-27 11:24:52,362][20347] Updated weights for policy 0, policy_version 834803 (0.0008) [2023-12-27 11:24:52,362][19170] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 427278336. Throughput: 0: 9684.2, 1: 9610.8. Samples: 427256304. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:52,363][19170] Avg episode reward: [(0, '892.260'), (1, '951.520')] [2023-12-27 11:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000834072_213549056.pth... [2023-12-27 11:24:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000832952_213262336.pth [2023-12-27 11:24:52,429][20347] Updated weights for policy 0, policy_version 834813 (0.0008) [2023-12-27 11:24:52,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000834816_213737472.pth... [2023-12-27 11:24:52,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000833664_213442560.pth [2023-12-27 11:24:52,589][20381] Updated weights for policy 1, policy_version 834078 (0.0007) [2023-12-27 11:24:52,649][20381] Updated weights for policy 1, policy_version 834088 (0.0007) [2023-12-27 11:24:52,714][20381] Updated weights for policy 1, policy_version 834098 (0.0008) [2023-12-27 11:24:53,158][20347] Updated weights for policy 0, policy_version 834823 (0.0010) [2023-12-27 11:24:53,214][20347] Updated weights for policy 0, policy_version 834833 (0.0010) [2023-12-27 11:24:53,274][20347] Updated weights for policy 0, policy_version 834843 (0.0011) [2023-12-27 11:24:53,439][20381] Updated weights for policy 1, policy_version 834108 (0.0007) [2023-12-27 11:24:53,503][20381] Updated weights for policy 1, policy_version 834118 (0.0008) [2023-12-27 11:24:53,569][20381] Updated weights for policy 1, policy_version 834128 (0.0008) [2023-12-27 11:24:53,968][20347] Updated weights for policy 0, policy_version 834853 (0.0008) [2023-12-27 11:24:54,025][20347] Updated weights for policy 0, policy_version 834863 (0.0008) [2023-12-27 11:24:54,087][20347] Updated weights for policy 0, policy_version 834873 (0.0010) [2023-12-27 11:24:54,263][20381] Updated weights for policy 1, policy_version 834138 (0.0007) [2023-12-27 11:24:54,324][20381] Updated weights for policy 1, policy_version 834148 (0.0008) [2023-12-27 11:24:54,380][20381] Updated weights for policy 1, policy_version 834158 (0.0007) [2023-12-27 11:24:54,434][20381] Updated weights for policy 1, policy_version 834168 (0.0008) [2023-12-27 11:24:54,775][20347] Updated weights for policy 0, policy_version 834883 (0.0010) [2023-12-27 11:24:54,825][20347] Updated weights for policy 0, policy_version 834893 (0.0011) [2023-12-27 11:24:54,883][20347] Updated weights for policy 0, policy_version 834903 (0.0011) [2023-12-27 11:24:55,154][20381] Updated weights for policy 1, policy_version 834178 (0.0007) [2023-12-27 11:24:55,215][20381] Updated weights for policy 1, policy_version 834188 (0.0007) [2023-12-27 11:24:55,265][20381] Updated weights for policy 1, policy_version 834198 (0.0008) [2023-12-27 11:24:55,569][20347] Updated weights for policy 0, policy_version 834913 (0.0010) [2023-12-27 11:24:55,637][20347] Updated weights for policy 0, policy_version 834923 (0.0009) [2023-12-27 11:24:55,706][20347] Updated weights for policy 0, policy_version 834933 (0.0008) [2023-12-27 11:24:55,772][20347] Updated weights for policy 0, policy_version 834943 (0.0008) [2023-12-27 11:24:55,956][20381] Updated weights for policy 1, policy_version 834208 (0.0008) [2023-12-27 11:24:56,010][20381] Updated weights for policy 1, policy_version 834218 (0.0009) [2023-12-27 11:24:56,060][20381] Updated weights for policy 1, policy_version 834228 (0.0007) [2023-12-27 11:24:56,455][20347] Updated weights for policy 0, policy_version 834953 (0.0007) [2023-12-27 11:24:56,512][20347] Updated weights for policy 0, policy_version 834963 (0.0007) [2023-12-27 11:24:56,559][20347] Updated weights for policy 0, policy_version 834973 (0.0007) [2023-12-27 11:24:56,738][20381] Updated weights for policy 1, policy_version 834238 (0.0007) [2023-12-27 11:24:56,799][20381] Updated weights for policy 1, policy_version 834248 (0.0007) [2023-12-27 11:24:56,848][20381] Updated weights for policy 1, policy_version 834258 (0.0009) [2023-12-27 11:24:57,234][20347] Updated weights for policy 0, policy_version 834983 (0.0006) [2023-12-27 11:24:57,288][20347] Updated weights for policy 0, policy_version 834993 (0.0008) [2023-12-27 11:24:57,348][20347] Updated weights for policy 0, policy_version 835003 (0.0008) [2023-12-27 11:24:57,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 427376640. Throughput: 0: 9734.7, 1: 9600.7. Samples: 427371696. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:24:57,363][19170] Avg episode reward: [(0, '862.720'), (1, '941.480')] [2023-12-27 11:24:57,536][20381] Updated weights for policy 1, policy_version 834268 (0.0008) [2023-12-27 11:24:57,585][20381] Updated weights for policy 1, policy_version 834278 (0.0008) [2023-12-27 11:24:57,638][20381] Updated weights for policy 1, policy_version 834288 (0.0008) [2023-12-27 11:24:58,032][20347] Updated weights for policy 0, policy_version 835013 (0.0008) [2023-12-27 11:24:58,089][20347] Updated weights for policy 0, policy_version 835023 (0.0007) [2023-12-27 11:24:58,143][20347] Updated weights for policy 0, policy_version 835033 (0.0008) [2023-12-27 11:24:58,377][20381] Updated weights for policy 1, policy_version 834298 (0.0007) [2023-12-27 11:24:58,445][20381] Updated weights for policy 1, policy_version 834308 (0.0008) [2023-12-27 11:24:58,521][20381] Updated weights for policy 1, policy_version 834318 (0.0008) [2023-12-27 11:24:58,586][20381] Updated weights for policy 1, policy_version 834328 (0.0007) [2023-12-27 11:24:59,016][20347] Updated weights for policy 0, policy_version 835043 (0.0008) [2023-12-27 11:24:59,077][20347] Updated weights for policy 0, policy_version 835053 (0.0008) [2023-12-27 11:24:59,147][20347] Updated weights for policy 0, policy_version 835063 (0.0008) [2023-12-27 11:24:59,452][20381] Updated weights for policy 1, policy_version 834338 (0.0008) [2023-12-27 11:24:59,506][20381] Updated weights for policy 1, policy_version 834348 (0.0008) [2023-12-27 11:24:59,569][20381] Updated weights for policy 1, policy_version 834358 (0.0008) [2023-12-27 11:24:59,897][20347] Updated weights for policy 0, policy_version 835073 (0.0008) [2023-12-27 11:24:59,960][20347] Updated weights for policy 0, policy_version 835083 (0.0008) [2023-12-27 11:25:00,023][20347] Updated weights for policy 0, policy_version 835093 (0.0007) [2023-12-27 11:25:00,078][20347] Updated weights for policy 0, policy_version 835103 (0.0008) [2023-12-27 11:25:00,354][20381] Updated weights for policy 1, policy_version 834368 (0.0008) [2023-12-27 11:25:00,409][20381] Updated weights for policy 1, policy_version 834378 (0.0008) [2023-12-27 11:25:00,464][20381] Updated weights for policy 1, policy_version 834388 (0.0008) [2023-12-27 11:25:00,749][20347] Updated weights for policy 0, policy_version 835113 (0.0006) [2023-12-27 11:25:00,810][20347] Updated weights for policy 0, policy_version 835123 (0.0005) [2023-12-27 11:25:00,870][20347] Updated weights for policy 0, policy_version 835133 (0.0005) [2023-12-27 11:25:01,200][20381] Updated weights for policy 1, policy_version 834398 (0.0007) [2023-12-27 11:25:01,263][20381] Updated weights for policy 1, policy_version 834408 (0.0009) [2023-12-27 11:25:01,317][20381] Updated weights for policy 1, policy_version 834418 (0.0008) [2023-12-27 11:25:01,569][20347] Updated weights for policy 0, policy_version 835143 (0.0007) [2023-12-27 11:25:01,633][20347] Updated weights for policy 0, policy_version 835153 (0.0007) [2023-12-27 11:25:01,693][20347] Updated weights for policy 0, policy_version 835163 (0.0006) [2023-12-27 11:25:02,103][20381] Updated weights for policy 1, policy_version 834428 (0.0010) [2023-12-27 11:25:02,158][20381] Updated weights for policy 1, policy_version 834438 (0.0008) [2023-12-27 11:25:02,223][20381] Updated weights for policy 1, policy_version 834448 (0.0008) [2023-12-27 11:25:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 427474944. Throughput: 0: 9732.3, 1: 9503.3. Samples: 427485280. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:02,363][19170] Avg episode reward: [(0, '862.720'), (1, '922.060')] [2023-12-27 11:25:02,390][20347] Updated weights for policy 0, policy_version 835173 (0.0009) [2023-12-27 11:25:02,452][20347] Updated weights for policy 0, policy_version 835183 (0.0007) [2023-12-27 11:25:02,514][20347] Updated weights for policy 0, policy_version 835193 (0.0008) [2023-12-27 11:25:02,961][20381] Updated weights for policy 1, policy_version 834458 (0.0009) [2023-12-27 11:25:03,033][20381] Updated weights for policy 1, policy_version 834468 (0.0006) [2023-12-27 11:25:03,095][20381] Updated weights for policy 1, policy_version 834478 (0.0008) [2023-12-27 11:25:03,153][20381] Updated weights for policy 1, policy_version 834488 (0.0010) [2023-12-27 11:25:03,176][20347] Updated weights for policy 0, policy_version 835203 (0.0006) [2023-12-27 11:25:03,237][20347] Updated weights for policy 0, policy_version 835213 (0.0007) [2023-12-27 11:25:03,296][20347] Updated weights for policy 0, policy_version 835223 (0.0008) [2023-12-27 11:25:03,802][20381] Updated weights for policy 1, policy_version 834498 (0.0008) [2023-12-27 11:25:03,869][20381] Updated weights for policy 1, policy_version 834508 (0.0008) [2023-12-27 11:25:03,928][20381] Updated weights for policy 1, policy_version 834518 (0.0008) [2023-12-27 11:25:03,941][20347] Updated weights for policy 0, policy_version 835233 (0.0008) [2023-12-27 11:25:04,006][20347] Updated weights for policy 0, policy_version 835243 (0.0008) [2023-12-27 11:25:04,068][20347] Updated weights for policy 0, policy_version 835253 (0.0007) [2023-12-27 11:25:04,130][20347] Updated weights for policy 0, policy_version 835263 (0.0006) [2023-12-27 11:25:04,661][20381] Updated weights for policy 1, policy_version 834528 (0.0007) [2023-12-27 11:25:04,718][20381] Updated weights for policy 1, policy_version 834538 (0.0009) [2023-12-27 11:25:04,775][20381] Updated weights for policy 1, policy_version 834548 (0.0007) [2023-12-27 11:25:04,837][20347] Updated weights for policy 0, policy_version 835273 (0.0008) [2023-12-27 11:25:04,894][20347] Updated weights for policy 0, policy_version 835283 (0.0008) [2023-12-27 11:25:04,951][20347] Updated weights for policy 0, policy_version 835293 (0.0005) [2023-12-27 11:25:05,366][20381] Updated weights for policy 1, policy_version 834558 (0.0007) [2023-12-27 11:25:05,428][20381] Updated weights for policy 1, policy_version 834568 (0.0008) [2023-12-27 11:25:05,478][20381] Updated weights for policy 1, policy_version 834578 (0.0009) [2023-12-27 11:25:05,610][20347] Updated weights for policy 0, policy_version 835303 (0.0007) [2023-12-27 11:25:05,665][20347] Updated weights for policy 0, policy_version 835313 (0.0006) [2023-12-27 11:25:05,717][20347] Updated weights for policy 0, policy_version 835323 (0.0005) [2023-12-27 11:25:06,206][20381] Updated weights for policy 1, policy_version 834588 (0.0008) [2023-12-27 11:25:06,267][20381] Updated weights for policy 1, policy_version 834598 (0.0008) [2023-12-27 11:25:06,324][20381] Updated weights for policy 1, policy_version 834608 (0.0008) [2023-12-27 11:25:06,388][20347] Updated weights for policy 0, policy_version 835333 (0.0006) [2023-12-27 11:25:06,444][20347] Updated weights for policy 0, policy_version 835343 (0.0006) [2023-12-27 11:25:06,504][20347] Updated weights for policy 0, policy_version 835353 (0.0007) [2023-12-27 11:25:07,166][20381] Updated weights for policy 1, policy_version 834618 (0.0008) [2023-12-27 11:25:07,170][20347] Updated weights for policy 0, policy_version 835363 (0.0006) [2023-12-27 11:25:07,218][20381] Updated weights for policy 1, policy_version 834628 (0.0009) [2023-12-27 11:25:07,219][20347] Updated weights for policy 0, policy_version 835373 (0.0007) [2023-12-27 11:25:07,270][20347] Updated weights for policy 0, policy_version 835383 (0.0006) [2023-12-27 11:25:07,271][20381] Updated weights for policy 1, policy_version 834638 (0.0009) [2023-12-27 11:25:07,331][20381] Updated weights for policy 1, policy_version 834648 (0.0008) [2023-12-27 11:25:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 427581440. Throughput: 0: 9746.2, 1: 9537.2. Samples: 427546020. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:07,363][19170] Avg episode reward: [(0, '872.630'), (1, '922.160')] [2023-12-27 11:25:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000834648_213696512.pth... [2023-12-27 11:25:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000835392_213884928.pth... [2023-12-27 11:25:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000833528_213409792.pth [2023-12-27 11:25:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000834240_213590016.pth [2023-12-27 11:25:07,977][20347] Updated weights for policy 0, policy_version 835393 (0.0007) [2023-12-27 11:25:08,031][20347] Updated weights for policy 0, policy_version 835403 (0.0008) [2023-12-27 11:25:08,077][20347] Updated weights for policy 0, policy_version 835413 (0.0007) [2023-12-27 11:25:08,104][20381] Updated weights for policy 1, policy_version 834658 (0.0007) [2023-12-27 11:25:08,132][20347] Updated weights for policy 0, policy_version 835423 (0.0007) [2023-12-27 11:25:08,158][20381] Updated weights for policy 1, policy_version 834668 (0.0008) [2023-12-27 11:25:08,222][20381] Updated weights for policy 1, policy_version 834678 (0.0008) [2023-12-27 11:25:08,926][20347] Updated weights for policy 0, policy_version 835433 (0.0007) [2023-12-27 11:25:08,979][20381] Updated weights for policy 1, policy_version 834688 (0.0007) [2023-12-27 11:25:08,983][20347] Updated weights for policy 0, policy_version 835443 (0.0008) [2023-12-27 11:25:09,031][20381] Updated weights for policy 1, policy_version 834698 (0.0007) [2023-12-27 11:25:09,040][20347] Updated weights for policy 0, policy_version 835453 (0.0006) [2023-12-27 11:25:09,089][20381] Updated weights for policy 1, policy_version 834708 (0.0007) [2023-12-27 11:25:09,822][20347] Updated weights for policy 0, policy_version 835463 (0.0008) [2023-12-27 11:25:09,847][20381] Updated weights for policy 1, policy_version 834718 (0.0010) [2023-12-27 11:25:09,887][20347] Updated weights for policy 0, policy_version 835473 (0.0009) [2023-12-27 11:25:09,917][20381] Updated weights for policy 1, policy_version 834728 (0.0007) [2023-12-27 11:25:09,954][20347] Updated weights for policy 0, policy_version 835483 (0.0007) [2023-12-27 11:25:09,986][20381] Updated weights for policy 1, policy_version 834738 (0.0008) [2023-12-27 11:25:10,696][20381] Updated weights for policy 1, policy_version 834748 (0.0008) [2023-12-27 11:25:10,717][20347] Updated weights for policy 0, policy_version 835493 (0.0008) [2023-12-27 11:25:10,759][20381] Updated weights for policy 1, policy_version 834758 (0.0006) [2023-12-27 11:25:10,778][20347] Updated weights for policy 0, policy_version 835503 (0.0007) [2023-12-27 11:25:10,820][20381] Updated weights for policy 1, policy_version 834768 (0.0006) [2023-12-27 11:25:10,833][20347] Updated weights for policy 0, policy_version 835513 (0.0007) [2023-12-27 11:25:11,524][20381] Updated weights for policy 1, policy_version 834778 (0.0007) [2023-12-27 11:25:11,570][20347] Updated weights for policy 0, policy_version 835523 (0.0006) [2023-12-27 11:25:11,590][20381] Updated weights for policy 1, policy_version 834788 (0.0008) [2023-12-27 11:25:11,635][20347] Updated weights for policy 0, policy_version 835533 (0.0007) [2023-12-27 11:25:11,655][20381] Updated weights for policy 1, policy_version 834798 (0.0008) [2023-12-27 11:25:11,693][20347] Updated weights for policy 0, policy_version 835543 (0.0006) [2023-12-27 11:25:11,717][20381] Updated weights for policy 1, policy_version 834808 (0.0008) [2023-12-27 11:25:12,362][19170] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 427663360. Throughput: 0: 9756.6, 1: 9491.6. Samples: 427660208. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:12,363][19170] Avg episode reward: [(0, '882.280'), (1, '931.870')] [2023-12-27 11:25:12,437][20347] Updated weights for policy 0, policy_version 835553 (0.0008) [2023-12-27 11:25:12,467][20381] Updated weights for policy 1, policy_version 834818 (0.0009) [2023-12-27 11:25:12,498][20347] Updated weights for policy 0, policy_version 835563 (0.0007) [2023-12-27 11:25:12,538][20381] Updated weights for policy 1, policy_version 834828 (0.0007) [2023-12-27 11:25:12,555][20347] Updated weights for policy 0, policy_version 835573 (0.0007) [2023-12-27 11:25:12,592][20381] Updated weights for policy 1, policy_version 834838 (0.0011) [2023-12-27 11:25:12,612][20347] Updated weights for policy 0, policy_version 835583 (0.0006) [2023-12-27 11:25:13,257][20381] Updated weights for policy 1, policy_version 834848 (0.0010) [2023-12-27 11:25:13,314][20381] Updated weights for policy 1, policy_version 834858 (0.0011) [2023-12-27 11:25:13,331][20347] Updated weights for policy 0, policy_version 835593 (0.0007) [2023-12-27 11:25:13,359][20291] KL-divergence is very high: 104.1517 [2023-12-27 11:25:13,368][20381] Updated weights for policy 1, policy_version 834868 (0.0010) [2023-12-27 11:25:13,388][20347] Updated weights for policy 0, policy_version 835603 (0.0007) [2023-12-27 11:25:13,444][20347] Updated weights for policy 0, policy_version 835613 (0.0006) [2023-12-27 11:25:14,131][20347] Updated weights for policy 0, policy_version 835623 (0.0008) [2023-12-27 11:25:14,178][20381] Updated weights for policy 1, policy_version 834878 (0.0009) [2023-12-27 11:25:14,190][20347] Updated weights for policy 0, policy_version 835633 (0.0007) [2023-12-27 11:25:14,237][20381] Updated weights for policy 1, policy_version 834888 (0.0009) [2023-12-27 11:25:14,254][20347] Updated weights for policy 0, policy_version 835643 (0.0007) [2023-12-27 11:25:14,297][20381] Updated weights for policy 1, policy_version 834898 (0.0009) [2023-12-27 11:25:14,929][20347] Updated weights for policy 0, policy_version 835653 (0.0009) [2023-12-27 11:25:14,988][20347] Updated weights for policy 0, policy_version 835663 (0.0010) [2023-12-27 11:25:15,051][20347] Updated weights for policy 0, policy_version 835673 (0.0008) [2023-12-27 11:25:15,083][20381] Updated weights for policy 1, policy_version 834908 (0.0008) [2023-12-27 11:25:15,144][20381] Updated weights for policy 1, policy_version 834918 (0.0008) [2023-12-27 11:25:15,202][20381] Updated weights for policy 1, policy_version 834928 (0.0008) [2023-12-27 11:25:15,767][20347] Updated weights for policy 0, policy_version 835683 (0.0007) [2023-12-27 11:25:15,827][20347] Updated weights for policy 0, policy_version 835693 (0.0008) [2023-12-27 11:25:15,881][20347] Updated weights for policy 0, policy_version 835703 (0.0008) [2023-12-27 11:25:15,933][20381] Updated weights for policy 1, policy_version 834938 (0.0007) [2023-12-27 11:25:15,992][20381] Updated weights for policy 1, policy_version 834948 (0.0007) [2023-12-27 11:25:16,056][20381] Updated weights for policy 1, policy_version 834958 (0.0008) [2023-12-27 11:25:16,110][20381] Updated weights for policy 1, policy_version 834968 (0.0009) [2023-12-27 11:25:16,514][20347] Updated weights for policy 0, policy_version 835713 (0.0007) [2023-12-27 11:25:16,575][20347] Updated weights for policy 0, policy_version 835723 (0.0008) [2023-12-27 11:25:16,632][20347] Updated weights for policy 0, policy_version 835733 (0.0008) [2023-12-27 11:25:16,692][20347] Updated weights for policy 0, policy_version 835743 (0.0008) [2023-12-27 11:25:16,832][20381] Updated weights for policy 1, policy_version 834978 (0.0008) [2023-12-27 11:25:16,890][20381] Updated weights for policy 1, policy_version 834988 (0.0008) [2023-12-27 11:25:16,947][20381] Updated weights for policy 1, policy_version 834998 (0.0008) [2023-12-27 11:25:17,362][19170] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 427761664. Throughput: 0: 9749.2, 1: 9480.3. Samples: 427775508. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:17,363][19170] Avg episode reward: [(0, '921.620'), (1, '853.540')] [2023-12-27 11:25:17,403][20347] Updated weights for policy 0, policy_version 835753 (0.0007) [2023-12-27 11:25:17,464][20347] Updated weights for policy 0, policy_version 835763 (0.0008) [2023-12-27 11:25:17,518][20347] Updated weights for policy 0, policy_version 835773 (0.0008) [2023-12-27 11:25:17,591][20381] Updated weights for policy 1, policy_version 835008 (0.0008) [2023-12-27 11:25:17,642][20381] Updated weights for policy 1, policy_version 835018 (0.0009) [2023-12-27 11:25:17,699][20381] Updated weights for policy 1, policy_version 835028 (0.0007) [2023-12-27 11:25:18,236][20347] Updated weights for policy 0, policy_version 835783 (0.0008) [2023-12-27 11:25:18,285][20347] Updated weights for policy 0, policy_version 835793 (0.0007) [2023-12-27 11:25:18,351][20347] Updated weights for policy 0, policy_version 835803 (0.0008) [2023-12-27 11:25:18,383][20381] Updated weights for policy 1, policy_version 835038 (0.0008) [2023-12-27 11:25:18,439][20381] Updated weights for policy 1, policy_version 835048 (0.0009) [2023-12-27 11:25:18,499][20381] Updated weights for policy 1, policy_version 835058 (0.0009) [2023-12-27 11:25:19,068][20347] Updated weights for policy 0, policy_version 835813 (0.0007) [2023-12-27 11:25:19,104][20291] KL-divergence is very high: 257.3678 [2023-12-27 11:25:19,119][20347] Updated weights for policy 0, policy_version 835823 (0.0008) [2023-12-27 11:25:19,130][20291] KL-divergence is very high: 174.9808 [2023-12-27 11:25:19,146][20291] KL-divergence is very high: 219.4043 [2023-12-27 11:25:19,176][20347] Updated weights for policy 0, policy_version 835833 (0.0007) [2023-12-27 11:25:19,251][20381] Updated weights for policy 1, policy_version 835068 (0.0009) [2023-12-27 11:25:19,319][20381] Updated weights for policy 1, policy_version 835078 (0.0007) [2023-12-27 11:25:19,389][20381] Updated weights for policy 1, policy_version 835088 (0.0008) [2023-12-27 11:25:19,977][20347] Updated weights for policy 0, policy_version 835843 (0.0008) [2023-12-27 11:25:20,040][20347] Updated weights for policy 0, policy_version 835853 (0.0008) [2023-12-27 11:25:20,095][20347] Updated weights for policy 0, policy_version 835863 (0.0008) [2023-12-27 11:25:20,138][20381] Updated weights for policy 1, policy_version 835098 (0.0008) [2023-12-27 11:25:20,195][20381] Updated weights for policy 1, policy_version 835108 (0.0011) [2023-12-27 11:25:20,253][20381] Updated weights for policy 1, policy_version 835118 (0.0010) [2023-12-27 11:25:20,313][20381] Updated weights for policy 1, policy_version 835128 (0.0011) [2023-12-27 11:25:20,959][20347] Updated weights for policy 0, policy_version 835873 (0.0009) [2023-12-27 11:25:21,018][20347] Updated weights for policy 0, policy_version 835883 (0.0009) [2023-12-27 11:25:21,080][20347] Updated weights for policy 0, policy_version 835893 (0.0009) [2023-12-27 11:25:21,126][20381] Updated weights for policy 1, policy_version 835138 (0.0006) [2023-12-27 11:25:21,140][20347] Updated weights for policy 0, policy_version 835903 (0.0009) [2023-12-27 11:25:21,197][20381] Updated weights for policy 1, policy_version 835148 (0.0007) [2023-12-27 11:25:21,262][20381] Updated weights for policy 1, policy_version 835158 (0.0008) [2023-12-27 11:25:21,990][20381] Updated weights for policy 1, policy_version 835168 (0.0008) [2023-12-27 11:25:22,050][20381] Updated weights for policy 1, policy_version 835178 (0.0008) [2023-12-27 11:25:22,051][20347] Updated weights for policy 0, policy_version 835913 (0.0009) [2023-12-27 11:25:22,110][20381] Updated weights for policy 1, policy_version 835188 (0.0008) [2023-12-27 11:25:22,112][20347] Updated weights for policy 0, policy_version 835923 (0.0009) [2023-12-27 11:25:22,166][20347] Updated weights for policy 0, policy_version 835933 (0.0009) [2023-12-27 11:25:22,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 427859968. Throughput: 0: 9702.0, 1: 9471.8. Samples: 427833124. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:22,363][19170] Avg episode reward: [(0, '960.820'), (1, '716.690')] [2023-12-27 11:25:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000835192_213835776.pth... [2023-12-27 11:25:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000835936_214024192.pth... [2023-12-27 11:25:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000834072_213549056.pth [2023-12-27 11:25:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000834816_213737472.pth [2023-12-27 11:25:22,821][20381] Updated weights for policy 1, policy_version 835198 (0.0008) [2023-12-27 11:25:22,879][20381] Updated weights for policy 1, policy_version 835208 (0.0007) [2023-12-27 11:25:22,938][20381] Updated weights for policy 1, policy_version 835218 (0.0008) [2023-12-27 11:25:23,057][20347] Updated weights for policy 0, policy_version 835943 (0.0009) [2023-12-27 11:25:23,111][20347] Updated weights for policy 0, policy_version 835953 (0.0009) [2023-12-27 11:25:23,172][20347] Updated weights for policy 0, policy_version 835963 (0.0009) [2023-12-27 11:25:23,659][20381] Updated weights for policy 1, policy_version 835228 (0.0006) [2023-12-27 11:25:23,721][20381] Updated weights for policy 1, policy_version 835238 (0.0006) [2023-12-27 11:25:23,779][20381] Updated weights for policy 1, policy_version 835248 (0.0006) [2023-12-27 11:25:23,859][20347] Updated weights for policy 0, policy_version 835973 (0.0008) [2023-12-27 11:25:23,924][20347] Updated weights for policy 0, policy_version 835983 (0.0007) [2023-12-27 11:25:23,931][20291] KL-divergence is very high: 514.2557 [2023-12-27 11:25:23,980][20291] KL-divergence is very high: 119.0144 [2023-12-27 11:25:23,986][20347] Updated weights for policy 0, policy_version 835993 (0.0008) [2023-12-27 11:25:24,473][20381] Updated weights for policy 1, policy_version 835258 (0.0007) [2023-12-27 11:25:24,540][20381] Updated weights for policy 1, policy_version 835268 (0.0007) [2023-12-27 11:25:24,553][20292] KL-divergence is very high: 215.6931 [2023-12-27 11:25:24,602][20292] KL-divergence is very high: 239.2171 [2023-12-27 11:25:24,603][20381] Updated weights for policy 1, policy_version 835278 (0.0008) [2023-12-27 11:25:24,643][20292] KL-divergence is very high: 148.8219 [2023-12-27 11:25:24,654][20381] Updated weights for policy 1, policy_version 835288 (0.0007) [2023-12-27 11:25:24,666][20347] Updated weights for policy 0, policy_version 836003 (0.0007) [2023-12-27 11:25:24,725][20347] Updated weights for policy 0, policy_version 836013 (0.0006) [2023-12-27 11:25:24,781][20347] Updated weights for policy 0, policy_version 836023 (0.0005) [2023-12-27 11:25:25,294][20381] Updated weights for policy 1, policy_version 835298 (0.0011) [2023-12-27 11:25:25,343][20347] Updated weights for policy 0, policy_version 836033 (0.0006) [2023-12-27 11:25:25,354][20381] Updated weights for policy 1, policy_version 835308 (0.0010) [2023-12-27 11:25:25,395][20347] Updated weights for policy 0, policy_version 836043 (0.0007) [2023-12-27 11:25:25,420][20381] Updated weights for policy 1, policy_version 835318 (0.0011) [2023-12-27 11:25:25,446][20347] Updated weights for policy 0, policy_version 836053 (0.0006) [2023-12-27 11:25:25,504][20347] Updated weights for policy 0, policy_version 836063 (0.0007) [2023-12-27 11:25:26,071][20381] Updated weights for policy 1, policy_version 835328 (0.0010) [2023-12-27 11:25:26,130][20381] Updated weights for policy 1, policy_version 835338 (0.0010) [2023-12-27 11:25:26,152][20347] Updated weights for policy 0, policy_version 836073 (0.0007) [2023-12-27 11:25:26,190][20381] Updated weights for policy 1, policy_version 835348 (0.0010) [2023-12-27 11:25:26,217][20347] Updated weights for policy 0, policy_version 836083 (0.0006) [2023-12-27 11:25:26,275][20347] Updated weights for policy 0, policy_version 836093 (0.0006) [2023-12-27 11:25:26,871][20381] Updated weights for policy 1, policy_version 835358 (0.0010) [2023-12-27 11:25:26,882][20347] Updated weights for policy 0, policy_version 836103 (0.0009) [2023-12-27 11:25:26,924][20381] Updated weights for policy 1, policy_version 835368 (0.0011) [2023-12-27 11:25:26,934][20347] Updated weights for policy 0, policy_version 836113 (0.0010) [2023-12-27 11:25:26,977][20381] Updated weights for policy 1, policy_version 835378 (0.0010) [2023-12-27 11:25:26,991][20347] Updated weights for policy 0, policy_version 836123 (0.0010) [2023-12-27 11:25:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 427958272. Throughput: 0: 9646.2, 1: 9480.5. Samples: 427947020. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:27,363][19170] Avg episode reward: [(0, '960.730'), (1, '628.550')] [2023-12-27 11:25:27,642][20381] Updated weights for policy 1, policy_version 835388 (0.0011) [2023-12-27 11:25:27,672][20347] Updated weights for policy 0, policy_version 836133 (0.0010) [2023-12-27 11:25:27,701][20381] Updated weights for policy 1, policy_version 835398 (0.0011) [2023-12-27 11:25:27,730][20347] Updated weights for policy 0, policy_version 836143 (0.0010) [2023-12-27 11:25:27,761][20381] Updated weights for policy 1, policy_version 835408 (0.0011) [2023-12-27 11:25:27,785][20347] Updated weights for policy 0, policy_version 836153 (0.0010) [2023-12-27 11:25:28,464][20381] Updated weights for policy 1, policy_version 835418 (0.0010) [2023-12-27 11:25:28,514][20347] Updated weights for policy 0, policy_version 836163 (0.0010) [2023-12-27 11:25:28,522][20381] Updated weights for policy 1, policy_version 835428 (0.0010) [2023-12-27 11:25:28,575][20347] Updated weights for policy 0, policy_version 836173 (0.0011) [2023-12-27 11:25:28,583][20381] Updated weights for policy 1, policy_version 835438 (0.0010) [2023-12-27 11:25:28,635][20347] Updated weights for policy 0, policy_version 836183 (0.0009) [2023-12-27 11:25:28,647][20381] Updated weights for policy 1, policy_version 835448 (0.0011) [2023-12-27 11:25:29,366][20347] Updated weights for policy 0, policy_version 836193 (0.0008) [2023-12-27 11:25:29,388][20381] Updated weights for policy 1, policy_version 835458 (0.0013) [2023-12-27 11:25:29,432][20347] Updated weights for policy 0, policy_version 836203 (0.0007) [2023-12-27 11:25:29,439][20381] Updated weights for policy 1, policy_version 835468 (0.0008) [2023-12-27 11:25:29,489][20381] Updated weights for policy 1, policy_version 835478 (0.0006) [2023-12-27 11:25:29,498][20347] Updated weights for policy 0, policy_version 836213 (0.0007) [2023-12-27 11:25:29,559][20347] Updated weights for policy 0, policy_version 836223 (0.0008) [2023-12-27 11:25:30,210][20381] Updated weights for policy 1, policy_version 835488 (0.0008) [2023-12-27 11:25:30,259][20381] Updated weights for policy 1, policy_version 835498 (0.0009) [2023-12-27 11:25:30,320][20381] Updated weights for policy 1, policy_version 835508 (0.0006) [2023-12-27 11:25:30,333][20347] Updated weights for policy 0, policy_version 836233 (0.0007) [2023-12-27 11:25:30,392][20347] Updated weights for policy 0, policy_version 836243 (0.0008) [2023-12-27 11:25:30,442][20347] Updated weights for policy 0, policy_version 836253 (0.0008) [2023-12-27 11:25:30,970][20381] Updated weights for policy 1, policy_version 835518 (0.0007) [2023-12-27 11:25:31,027][20381] Updated weights for policy 1, policy_version 835528 (0.0008) [2023-12-27 11:25:31,087][20381] Updated weights for policy 1, policy_version 835538 (0.0007) [2023-12-27 11:25:31,154][20347] Updated weights for policy 0, policy_version 836263 (0.0008) [2023-12-27 11:25:31,216][20347] Updated weights for policy 0, policy_version 836273 (0.0007) [2023-12-27 11:25:31,279][20347] Updated weights for policy 0, policy_version 836283 (0.0008) [2023-12-27 11:25:31,836][20381] Updated weights for policy 1, policy_version 835548 (0.0008) [2023-12-27 11:25:31,843][20292] KL-divergence is very high: 152.7834 [2023-12-27 11:25:31,855][20292] KL-divergence is very high: 246.6346 [2023-12-27 11:25:31,861][20292] KL-divergence is very high: 291.8315 [2023-12-27 11:25:31,867][20292] KL-divergence is very high: 294.3770 [2023-12-27 11:25:31,875][20292] KL-divergence is very high: 105.2831 [2023-12-27 11:25:31,897][20381] Updated weights for policy 1, policy_version 835558 (0.0007) [2023-12-27 11:25:31,963][20381] Updated weights for policy 1, policy_version 835568 (0.0007) [2023-12-27 11:25:32,021][20347] Updated weights for policy 0, policy_version 836293 (0.0006) [2023-12-27 11:25:32,080][20347] Updated weights for policy 0, policy_version 836303 (0.0007) [2023-12-27 11:25:32,137][20347] Updated weights for policy 0, policy_version 836313 (0.0010) [2023-12-27 11:25:32,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 428056576. Throughput: 0: 9678.6, 1: 9537.3. Samples: 428065300. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:32,363][19170] Avg episode reward: [(0, '950.960'), (1, '564.090')] [2023-12-27 11:25:32,701][20381] Updated weights for policy 1, policy_version 835578 (0.0008) [2023-12-27 11:25:32,762][20381] Updated weights for policy 1, policy_version 835588 (0.0008) [2023-12-27 11:25:32,812][20347] Updated weights for policy 0, policy_version 836323 (0.0009) [2023-12-27 11:25:32,819][20381] Updated weights for policy 1, policy_version 835598 (0.0007) [2023-12-27 11:25:32,871][20381] Updated weights for policy 1, policy_version 835608 (0.0007) [2023-12-27 11:25:32,878][20347] Updated weights for policy 0, policy_version 836333 (0.0008) [2023-12-27 11:25:32,929][20347] Updated weights for policy 0, policy_version 836343 (0.0007) [2023-12-27 11:25:33,542][20347] Updated weights for policy 0, policy_version 836353 (0.0007) [2023-12-27 11:25:33,599][20381] Updated weights for policy 1, policy_version 835618 (0.0007) [2023-12-27 11:25:33,605][20347] Updated weights for policy 0, policy_version 836363 (0.0006) [2023-12-27 11:25:33,655][20381] Updated weights for policy 1, policy_version 835628 (0.0007) [2023-12-27 11:25:33,664][20347] Updated weights for policy 0, policy_version 836373 (0.0005) [2023-12-27 11:25:33,712][20381] Updated weights for policy 1, policy_version 835638 (0.0010) [2023-12-27 11:25:33,718][20347] Updated weights for policy 0, policy_version 836383 (0.0005) [2023-12-27 11:25:34,321][20347] Updated weights for policy 0, policy_version 836393 (0.0008) [2023-12-27 11:25:34,381][20347] Updated weights for policy 0, policy_version 836403 (0.0009) [2023-12-27 11:25:34,383][20381] Updated weights for policy 1, policy_version 835648 (0.0010) [2023-12-27 11:25:34,440][20347] Updated weights for policy 0, policy_version 836413 (0.0008) [2023-12-27 11:25:34,444][20381] Updated weights for policy 1, policy_version 835658 (0.0008) [2023-12-27 11:25:34,507][20381] Updated weights for policy 1, policy_version 835668 (0.0008) [2023-12-27 11:25:35,124][20347] Updated weights for policy 0, policy_version 836423 (0.0007) [2023-12-27 11:25:35,175][20347] Updated weights for policy 0, policy_version 836433 (0.0006) [2023-12-27 11:25:35,184][20381] Updated weights for policy 1, policy_version 835678 (0.0010) [2023-12-27 11:25:35,228][20347] Updated weights for policy 0, policy_version 836443 (0.0006) [2023-12-27 11:25:35,231][20381] Updated weights for policy 1, policy_version 835688 (0.0008) [2023-12-27 11:25:35,284][20381] Updated weights for policy 1, policy_version 835698 (0.0006) [2023-12-27 11:25:35,898][20347] Updated weights for policy 0, policy_version 836453 (0.0006) [2023-12-27 11:25:35,899][20381] Updated weights for policy 1, policy_version 835708 (0.0008) [2023-12-27 11:25:35,947][20381] Updated weights for policy 1, policy_version 835718 (0.0010) [2023-12-27 11:25:35,955][20347] Updated weights for policy 0, policy_version 836463 (0.0007) [2023-12-27 11:25:36,005][20381] Updated weights for policy 1, policy_version 835728 (0.0009) [2023-12-27 11:25:36,011][20347] Updated weights for policy 0, policy_version 836473 (0.0006) [2023-12-27 11:25:36,727][20347] Updated weights for policy 0, policy_version 836483 (0.0007) [2023-12-27 11:25:36,730][20381] Updated weights for policy 1, policy_version 835738 (0.0010) [2023-12-27 11:25:36,789][20381] Updated weights for policy 1, policy_version 835748 (0.0009) [2023-12-27 11:25:36,790][20347] Updated weights for policy 0, policy_version 836493 (0.0007) [2023-12-27 11:25:36,845][20347] Updated weights for policy 0, policy_version 836503 (0.0006) [2023-12-27 11:25:36,849][20381] Updated weights for policy 1, policy_version 835758 (0.0007) [2023-12-27 11:25:36,908][20381] Updated weights for policy 1, policy_version 835768 (0.0010) [2023-12-27 11:25:37,363][19170] Fps is (10 sec: 19660.2, 60 sec: 19251.1, 300 sec: 19410.9). Total num frames: 428154880. Throughput: 0: 9751.1, 1: 9587.3. Samples: 428126536. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:37,364][19170] Avg episode reward: [(0, '941.210'), (1, '582.850')] [2023-12-27 11:25:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000835768_213983232.pth... [2023-12-27 11:25:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000836512_214171648.pth... [2023-12-27 11:25:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000834648_213696512.pth [2023-12-27 11:25:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000835392_213884928.pth [2023-12-27 11:25:37,558][20347] Updated weights for policy 0, policy_version 836513 (0.0007) [2023-12-27 11:25:37,609][20347] Updated weights for policy 0, policy_version 836523 (0.0005) [2023-12-27 11:25:37,634][20381] Updated weights for policy 1, policy_version 835778 (0.0010) [2023-12-27 11:25:37,668][20347] Updated weights for policy 0, policy_version 836533 (0.0006) [2023-12-27 11:25:37,691][20381] Updated weights for policy 1, policy_version 835788 (0.0011) [2023-12-27 11:25:37,733][20347] Updated weights for policy 0, policy_version 836543 (0.0007) [2023-12-27 11:25:37,750][20381] Updated weights for policy 1, policy_version 835798 (0.0007) [2023-12-27 11:25:38,371][20347] Updated weights for policy 0, policy_version 836553 (0.0007) [2023-12-27 11:25:38,403][20381] Updated weights for policy 1, policy_version 835808 (0.0006) [2023-12-27 11:25:38,430][20347] Updated weights for policy 0, policy_version 836563 (0.0007) [2023-12-27 11:25:38,464][20381] Updated weights for policy 1, policy_version 835818 (0.0009) [2023-12-27 11:25:38,488][20347] Updated weights for policy 0, policy_version 836573 (0.0006) [2023-12-27 11:25:38,522][20381] Updated weights for policy 1, policy_version 835828 (0.0009) [2023-12-27 11:25:39,241][20381] Updated weights for policy 1, policy_version 835838 (0.0011) [2023-12-27 11:25:39,245][20347] Updated weights for policy 0, policy_version 836583 (0.0007) [2023-12-27 11:25:39,305][20381] Updated weights for policy 1, policy_version 835848 (0.0011) [2023-12-27 11:25:39,312][20347] Updated weights for policy 0, policy_version 836593 (0.0006) [2023-12-27 11:25:39,368][20381] Updated weights for policy 1, policy_version 835858 (0.0009) [2023-12-27 11:25:39,381][20347] Updated weights for policy 0, policy_version 836603 (0.0008) [2023-12-27 11:25:39,411][20291] KL-divergence is very high: 100.8877 [2023-12-27 11:25:40,100][20347] Updated weights for policy 0, policy_version 836613 (0.0007) [2023-12-27 11:25:40,142][20381] Updated weights for policy 1, policy_version 835868 (0.0010) [2023-12-27 11:25:40,167][20347] Updated weights for policy 0, policy_version 836623 (0.0008) [2023-12-27 11:25:40,203][20381] Updated weights for policy 1, policy_version 835878 (0.0011) [2023-12-27 11:25:40,226][20347] Updated weights for policy 0, policy_version 836633 (0.0007) [2023-12-27 11:25:40,264][20381] Updated weights for policy 1, policy_version 835888 (0.0011) [2023-12-27 11:25:40,904][20347] Updated weights for policy 0, policy_version 836643 (0.0007) [2023-12-27 11:25:40,974][20347] Updated weights for policy 0, policy_version 836653 (0.0009) [2023-12-27 11:25:40,986][20381] Updated weights for policy 1, policy_version 835898 (0.0011) [2023-12-27 11:25:41,043][20347] Updated weights for policy 0, policy_version 836663 (0.0007) [2023-12-27 11:25:41,045][20381] Updated weights for policy 1, policy_version 835908 (0.0011) [2023-12-27 11:25:41,109][20381] Updated weights for policy 1, policy_version 835918 (0.0011) [2023-12-27 11:25:41,180][20381] Updated weights for policy 1, policy_version 835928 (0.0007) [2023-12-27 11:25:41,843][20347] Updated weights for policy 0, policy_version 836673 (0.0007) [2023-12-27 11:25:41,904][20347] Updated weights for policy 0, policy_version 836683 (0.0007) [2023-12-27 11:25:41,962][20347] Updated weights for policy 0, policy_version 836693 (0.0006) [2023-12-27 11:25:41,977][20381] Updated weights for policy 1, policy_version 835938 (0.0008) [2023-12-27 11:25:42,024][20347] Updated weights for policy 0, policy_version 836703 (0.0007) [2023-12-27 11:25:42,034][20381] Updated weights for policy 1, policy_version 835948 (0.0008) [2023-12-27 11:25:42,095][20381] Updated weights for policy 1, policy_version 835958 (0.0008) [2023-12-27 11:25:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 428253184. Throughput: 0: 9769.8, 1: 9625.2. Samples: 428244472. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:42,363][19170] Avg episode reward: [(0, '882.360'), (1, '708.840')] [2023-12-27 11:25:42,767][20347] Updated weights for policy 0, policy_version 836713 (0.0007) [2023-12-27 11:25:42,829][20347] Updated weights for policy 0, policy_version 836723 (0.0007) [2023-12-27 11:25:42,894][20347] Updated weights for policy 0, policy_version 836733 (0.0007) [2023-12-27 11:25:42,902][20381] Updated weights for policy 1, policy_version 835968 (0.0008) [2023-12-27 11:25:42,961][20381] Updated weights for policy 1, policy_version 835978 (0.0008) [2023-12-27 11:25:43,012][20381] Updated weights for policy 1, policy_version 835988 (0.0008) [2023-12-27 11:25:43,566][20347] Updated weights for policy 0, policy_version 836743 (0.0006) [2023-12-27 11:25:43,621][20347] Updated weights for policy 0, policy_version 836753 (0.0006) [2023-12-27 11:25:43,674][20347] Updated weights for policy 0, policy_version 836763 (0.0008) [2023-12-27 11:25:43,709][20381] Updated weights for policy 1, policy_version 835998 (0.0008) [2023-12-27 11:25:43,763][20381] Updated weights for policy 1, policy_version 836008 (0.0008) [2023-12-27 11:25:43,816][20381] Updated weights for policy 1, policy_version 836018 (0.0008) [2023-12-27 11:25:44,387][20347] Updated weights for policy 0, policy_version 836773 (0.0007) [2023-12-27 11:25:44,453][20347] Updated weights for policy 0, policy_version 836783 (0.0007) [2023-12-27 11:25:44,515][20347] Updated weights for policy 0, policy_version 836793 (0.0007) [2023-12-27 11:25:44,520][20381] Updated weights for policy 1, policy_version 836028 (0.0007) [2023-12-27 11:25:44,574][20381] Updated weights for policy 1, policy_version 836038 (0.0006) [2023-12-27 11:25:44,620][20381] Updated weights for policy 1, policy_version 836048 (0.0007) [2023-12-27 11:25:45,231][20347] Updated weights for policy 0, policy_version 836803 (0.0007) [2023-12-27 11:25:45,288][20347] Updated weights for policy 0, policy_version 836813 (0.0008) [2023-12-27 11:25:45,325][20381] Updated weights for policy 1, policy_version 836058 (0.0007) [2023-12-27 11:25:45,342][20347] Updated weights for policy 0, policy_version 836823 (0.0007) [2023-12-27 11:25:45,387][20381] Updated weights for policy 1, policy_version 836068 (0.0006) [2023-12-27 11:25:45,441][20381] Updated weights for policy 1, policy_version 836078 (0.0007) [2023-12-27 11:25:45,496][20381] Updated weights for policy 1, policy_version 836088 (0.0006) [2023-12-27 11:25:46,007][20347] Updated weights for policy 0, policy_version 836833 (0.0006) [2023-12-27 11:25:46,057][20347] Updated weights for policy 0, policy_version 836843 (0.0007) [2023-12-27 11:25:46,112][20347] Updated weights for policy 0, policy_version 836853 (0.0007) [2023-12-27 11:25:46,166][20347] Updated weights for policy 0, policy_version 836863 (0.0007) [2023-12-27 11:25:46,241][20381] Updated weights for policy 1, policy_version 836098 (0.0008) [2023-12-27 11:25:46,297][20381] Updated weights for policy 1, policy_version 836108 (0.0008) [2023-12-27 11:25:46,346][20381] Updated weights for policy 1, policy_version 836118 (0.0006) [2023-12-27 11:25:46,856][20347] Updated weights for policy 0, policy_version 836873 (0.0007) [2023-12-27 11:25:46,920][20347] Updated weights for policy 0, policy_version 836883 (0.0007) [2023-12-27 11:25:46,982][20347] Updated weights for policy 0, policy_version 836893 (0.0007) [2023-12-27 11:25:47,019][20381] Updated weights for policy 1, policy_version 836128 (0.0008) [2023-12-27 11:25:47,078][20381] Updated weights for policy 1, policy_version 836138 (0.0006) [2023-12-27 11:25:47,142][20381] Updated weights for policy 1, policy_version 836148 (0.0006) [2023-12-27 11:25:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 428351488. Throughput: 0: 9783.1, 1: 9669.4. Samples: 428360644. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:47,363][19170] Avg episode reward: [(0, '921.520'), (1, '688.180')] [2023-12-27 11:25:47,716][20347] Updated weights for policy 0, policy_version 836903 (0.0008) [2023-12-27 11:25:47,758][20381] Updated weights for policy 1, policy_version 836158 (0.0007) [2023-12-27 11:25:47,775][20347] Updated weights for policy 0, policy_version 836913 (0.0008) [2023-12-27 11:25:47,806][20381] Updated weights for policy 1, policy_version 836168 (0.0007) [2023-12-27 11:25:47,834][20347] Updated weights for policy 0, policy_version 836923 (0.0008) [2023-12-27 11:25:47,861][20381] Updated weights for policy 1, policy_version 836178 (0.0009) [2023-12-27 11:25:48,575][20347] Updated weights for policy 0, policy_version 836933 (0.0009) [2023-12-27 11:25:48,584][20381] Updated weights for policy 1, policy_version 836188 (0.0007) [2023-12-27 11:25:48,641][20381] Updated weights for policy 1, policy_version 836198 (0.0008) [2023-12-27 11:25:48,644][20347] Updated weights for policy 0, policy_version 836943 (0.0006) [2023-12-27 11:25:48,701][20381] Updated weights for policy 1, policy_version 836208 (0.0008) [2023-12-27 11:25:48,704][20347] Updated weights for policy 0, policy_version 836953 (0.0008) [2023-12-27 11:25:49,366][20381] Updated weights for policy 1, policy_version 836218 (0.0008) [2023-12-27 11:25:49,420][20347] Updated weights for policy 0, policy_version 836963 (0.0006) [2023-12-27 11:25:49,433][20381] Updated weights for policy 1, policy_version 836228 (0.0009) [2023-12-27 11:25:49,476][20347] Updated weights for policy 0, policy_version 836973 (0.0007) [2023-12-27 11:25:49,495][20381] Updated weights for policy 1, policy_version 836238 (0.0008) [2023-12-27 11:25:49,532][20347] Updated weights for policy 0, policy_version 836983 (0.0006) [2023-12-27 11:25:49,558][20381] Updated weights for policy 1, policy_version 836248 (0.0008) [2023-12-27 11:25:50,254][20347] Updated weights for policy 0, policy_version 836993 (0.0007) [2023-12-27 11:25:50,275][20381] Updated weights for policy 1, policy_version 836258 (0.0008) [2023-12-27 11:25:50,318][20347] Updated weights for policy 0, policy_version 837003 (0.0006) [2023-12-27 11:25:50,329][20381] Updated weights for policy 1, policy_version 836268 (0.0008) [2023-12-27 11:25:50,386][20347] Updated weights for policy 0, policy_version 837013 (0.0007) [2023-12-27 11:25:50,388][20381] Updated weights for policy 1, policy_version 836278 (0.0008) [2023-12-27 11:25:50,447][20347] Updated weights for policy 0, policy_version 837023 (0.0008) [2023-12-27 11:25:51,090][20381] Updated weights for policy 1, policy_version 836288 (0.0008) [2023-12-27 11:25:51,152][20381] Updated weights for policy 1, policy_version 836298 (0.0008) [2023-12-27 11:25:51,199][20347] Updated weights for policy 0, policy_version 837033 (0.0008) [2023-12-27 11:25:51,220][20381] Updated weights for policy 1, policy_version 836308 (0.0009) [2023-12-27 11:25:51,267][20347] Updated weights for policy 0, policy_version 837043 (0.0008) [2023-12-27 11:25:51,327][20347] Updated weights for policy 0, policy_version 837053 (0.0008) [2023-12-27 11:25:52,034][20381] Updated weights for policy 1, policy_version 836318 (0.0009) [2023-12-27 11:25:52,098][20381] Updated weights for policy 1, policy_version 836328 (0.0008) [2023-12-27 11:25:52,152][20347] Updated weights for policy 0, policy_version 837063 (0.0008) [2023-12-27 11:25:52,158][20381] Updated weights for policy 1, policy_version 836338 (0.0006) [2023-12-27 11:25:52,210][20347] Updated weights for policy 0, policy_version 837073 (0.0008) [2023-12-27 11:25:52,278][20347] Updated weights for policy 0, policy_version 837083 (0.0007) [2023-12-27 11:25:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 428449792. Throughput: 0: 9740.0, 1: 9671.0. Samples: 428419516. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:52,364][19170] Avg episode reward: [(0, '941.150'), (1, '573.400')] [2023-12-27 11:25:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000836344_214130688.pth... [2023-12-27 11:25:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000837088_214319104.pth... [2023-12-27 11:25:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000835192_213835776.pth [2023-12-27 11:25:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000835936_214024192.pth [2023-12-27 11:25:52,816][20381] Updated weights for policy 1, policy_version 836348 (0.0008) [2023-12-27 11:25:52,877][20381] Updated weights for policy 1, policy_version 836358 (0.0009) [2023-12-27 11:25:52,933][20381] Updated weights for policy 1, policy_version 836368 (0.0008) [2023-12-27 11:25:53,004][20347] Updated weights for policy 0, policy_version 837093 (0.0007) [2023-12-27 11:25:53,055][20347] Updated weights for policy 0, policy_version 837103 (0.0010) [2023-12-27 11:25:53,109][20347] Updated weights for policy 0, policy_version 837113 (0.0010) [2023-12-27 11:25:53,673][20381] Updated weights for policy 1, policy_version 836378 (0.0008) [2023-12-27 11:25:53,732][20381] Updated weights for policy 1, policy_version 836388 (0.0008) [2023-12-27 11:25:53,795][20381] Updated weights for policy 1, policy_version 836398 (0.0006) [2023-12-27 11:25:53,825][20347] Updated weights for policy 0, policy_version 837123 (0.0010) [2023-12-27 11:25:53,849][20381] Updated weights for policy 1, policy_version 836408 (0.0005) [2023-12-27 11:25:53,878][20347] Updated weights for policy 0, policy_version 837133 (0.0008) [2023-12-27 11:25:53,930][20347] Updated weights for policy 0, policy_version 837143 (0.0007) [2023-12-27 11:25:54,521][20381] Updated weights for policy 1, policy_version 836418 (0.0007) [2023-12-27 11:25:54,577][20381] Updated weights for policy 1, policy_version 836428 (0.0007) [2023-12-27 11:25:54,635][20347] Updated weights for policy 0, policy_version 837153 (0.0011) [2023-12-27 11:25:54,641][20381] Updated weights for policy 1, policy_version 836438 (0.0007) [2023-12-27 11:25:54,692][20347] Updated weights for policy 0, policy_version 837163 (0.0008) [2023-12-27 11:25:54,750][20347] Updated weights for policy 0, policy_version 837173 (0.0008) [2023-12-27 11:25:54,808][20347] Updated weights for policy 0, policy_version 837183 (0.0009) [2023-12-27 11:25:55,312][20381] Updated weights for policy 1, policy_version 836448 (0.0008) [2023-12-27 11:25:55,376][20381] Updated weights for policy 1, policy_version 836458 (0.0011) [2023-12-27 11:25:55,433][20381] Updated weights for policy 1, policy_version 836468 (0.0011) [2023-12-27 11:25:55,459][20347] Updated weights for policy 0, policy_version 837193 (0.0006) [2023-12-27 11:25:55,514][20347] Updated weights for policy 0, policy_version 837203 (0.0008) [2023-12-27 11:25:55,576][20347] Updated weights for policy 0, policy_version 837213 (0.0006) [2023-12-27 11:25:56,093][20381] Updated weights for policy 1, policy_version 836478 (0.0009) [2023-12-27 11:25:56,150][20381] Updated weights for policy 1, policy_version 836488 (0.0008) [2023-12-27 11:25:56,177][20347] Updated weights for policy 0, policy_version 837223 (0.0007) [2023-12-27 11:25:56,210][20381] Updated weights for policy 1, policy_version 836498 (0.0006) [2023-12-27 11:25:56,233][20347] Updated weights for policy 0, policy_version 837233 (0.0007) [2023-12-27 11:25:56,284][20347] Updated weights for policy 0, policy_version 837243 (0.0007) [2023-12-27 11:25:56,886][20381] Updated weights for policy 1, policy_version 836508 (0.0006) [2023-12-27 11:25:56,942][20381] Updated weights for policy 1, policy_version 836518 (0.0007) [2023-12-27 11:25:56,965][20347] Updated weights for policy 0, policy_version 837253 (0.0008) [2023-12-27 11:25:56,998][20381] Updated weights for policy 1, policy_version 836528 (0.0007) [2023-12-27 11:25:57,022][20347] Updated weights for policy 0, policy_version 837263 (0.0008) [2023-12-27 11:25:57,078][20347] Updated weights for policy 0, policy_version 837273 (0.0007) [2023-12-27 11:25:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 428548096. Throughput: 0: 9746.7, 1: 9749.3. Samples: 428537524. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:25:57,363][19170] Avg episode reward: [(0, '941.170'), (1, '545.810')] [2023-12-27 11:25:57,712][20381] Updated weights for policy 1, policy_version 836538 (0.0006) [2023-12-27 11:25:57,771][20381] Updated weights for policy 1, policy_version 836548 (0.0008) [2023-12-27 11:25:57,776][20347] Updated weights for policy 0, policy_version 837283 (0.0008) [2023-12-27 11:25:57,824][20381] Updated weights for policy 1, policy_version 836558 (0.0008) [2023-12-27 11:25:57,841][20347] Updated weights for policy 0, policy_version 837293 (0.0008) [2023-12-27 11:25:57,878][20381] Updated weights for policy 1, policy_version 836568 (0.0005) [2023-12-27 11:25:57,900][20347] Updated weights for policy 0, policy_version 837303 (0.0006) [2023-12-27 11:25:58,622][20381] Updated weights for policy 1, policy_version 836578 (0.0007) [2023-12-27 11:25:58,640][20347] Updated weights for policy 0, policy_version 837313 (0.0007) [2023-12-27 11:25:58,689][20381] Updated weights for policy 1, policy_version 836588 (0.0007) [2023-12-27 11:25:58,706][20347] Updated weights for policy 0, policy_version 837323 (0.0008) [2023-12-27 11:25:58,757][20381] Updated weights for policy 1, policy_version 836598 (0.0008) [2023-12-27 11:25:58,770][20347] Updated weights for policy 0, policy_version 837333 (0.0008) [2023-12-27 11:25:58,840][20347] Updated weights for policy 0, policy_version 837343 (0.0008) [2023-12-27 11:25:59,514][20381] Updated weights for policy 1, policy_version 836608 (0.0008) [2023-12-27 11:25:59,567][20381] Updated weights for policy 1, policy_version 836618 (0.0008) [2023-12-27 11:25:59,624][20381] Updated weights for policy 1, policy_version 836628 (0.0007) [2023-12-27 11:25:59,645][20347] Updated weights for policy 0, policy_version 837353 (0.0007) [2023-12-27 11:25:59,708][20347] Updated weights for policy 0, policy_version 837363 (0.0008) [2023-12-27 11:25:59,764][20347] Updated weights for policy 0, policy_version 837373 (0.0007) [2023-12-27 11:26:00,358][20381] Updated weights for policy 1, policy_version 836638 (0.0006) [2023-12-27 11:26:00,422][20381] Updated weights for policy 1, policy_version 836648 (0.0008) [2023-12-27 11:26:00,478][20381] Updated weights for policy 1, policy_version 836658 (0.0007) [2023-12-27 11:26:00,492][20347] Updated weights for policy 0, policy_version 837383 (0.0008) [2023-12-27 11:26:00,497][20291] KL-divergence is very high: 424.2424 [2023-12-27 11:26:00,555][20347] Updated weights for policy 0, policy_version 837393 (0.0008) [2023-12-27 11:26:00,611][20347] Updated weights for policy 0, policy_version 837403 (0.0010) [2023-12-27 11:26:01,154][20381] Updated weights for policy 1, policy_version 836668 (0.0007) [2023-12-27 11:26:01,219][20381] Updated weights for policy 1, policy_version 836678 (0.0008) [2023-12-27 11:26:01,284][20381] Updated weights for policy 1, policy_version 836688 (0.0007) [2023-12-27 11:26:01,312][20347] Updated weights for policy 0, policy_version 837413 (0.0008) [2023-12-27 11:26:01,383][20347] Updated weights for policy 0, policy_version 837423 (0.0009) [2023-12-27 11:26:01,441][20347] Updated weights for policy 0, policy_version 837433 (0.0006) [2023-12-27 11:26:01,986][20381] Updated weights for policy 1, policy_version 836698 (0.0008) [2023-12-27 11:26:02,045][20381] Updated weights for policy 1, policy_version 836708 (0.0007) [2023-12-27 11:26:02,108][20381] Updated weights for policy 1, policy_version 836718 (0.0006) [2023-12-27 11:26:02,166][20347] Updated weights for policy 0, policy_version 837443 (0.0006) [2023-12-27 11:26:02,172][20381] Updated weights for policy 1, policy_version 836728 (0.0008) [2023-12-27 11:26:02,228][20347] Updated weights for policy 0, policy_version 837453 (0.0009) [2023-12-27 11:26:02,290][20347] Updated weights for policy 0, policy_version 837463 (0.0008) [2023-12-27 11:26:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 428646400. Throughput: 0: 9689.8, 1: 9779.0. Samples: 428651600. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:26:02,363][19170] Avg episode reward: [(0, '970.600'), (1, '597.820')] [2023-12-27 11:26:02,836][20381] Updated weights for policy 1, policy_version 836738 (0.0006) [2023-12-27 11:26:02,892][20381] Updated weights for policy 1, policy_version 836748 (0.0006) [2023-12-27 11:26:02,949][20381] Updated weights for policy 1, policy_version 836758 (0.0006) [2023-12-27 11:26:02,949][20347] Updated weights for policy 0, policy_version 837473 (0.0008) [2023-12-27 11:26:03,001][20347] Updated weights for policy 0, policy_version 837483 (0.0008) [2023-12-27 11:26:03,052][20347] Updated weights for policy 0, policy_version 837493 (0.0007) [2023-12-27 11:26:03,102][20347] Updated weights for policy 0, policy_version 837503 (0.0008) [2023-12-27 11:26:03,635][20381] Updated weights for policy 1, policy_version 836768 (0.0007) [2023-12-27 11:26:03,688][20381] Updated weights for policy 1, policy_version 836778 (0.0007) [2023-12-27 11:26:03,744][20381] Updated weights for policy 1, policy_version 836788 (0.0008) [2023-12-27 11:26:03,805][20347] Updated weights for policy 0, policy_version 837513 (0.0008) [2023-12-27 11:26:03,871][20347] Updated weights for policy 0, policy_version 837523 (0.0008) [2023-12-27 11:26:03,935][20347] Updated weights for policy 0, policy_version 837533 (0.0006) [2023-12-27 11:26:04,518][20381] Updated weights for policy 1, policy_version 836798 (0.0009) [2023-12-27 11:26:04,567][20381] Updated weights for policy 1, policy_version 836808 (0.0007) [2023-12-27 11:26:04,618][20381] Updated weights for policy 1, policy_version 836818 (0.0007) [2023-12-27 11:26:04,681][20347] Updated weights for policy 0, policy_version 837543 (0.0008) [2023-12-27 11:26:04,739][20347] Updated weights for policy 0, policy_version 837553 (0.0007) [2023-12-27 11:26:04,798][20347] Updated weights for policy 0, policy_version 837563 (0.0007) [2023-12-27 11:26:05,275][20381] Updated weights for policy 1, policy_version 836828 (0.0007) [2023-12-27 11:26:05,337][20381] Updated weights for policy 1, policy_version 836838 (0.0006) [2023-12-27 11:26:05,400][20381] Updated weights for policy 1, policy_version 836848 (0.0006) [2023-12-27 11:26:05,525][20347] Updated weights for policy 0, policy_version 837573 (0.0008) [2023-12-27 11:26:05,586][20347] Updated weights for policy 0, policy_version 837583 (0.0008) [2023-12-27 11:26:05,642][20347] Updated weights for policy 0, policy_version 837593 (0.0007) [2023-12-27 11:26:06,056][20381] Updated weights for policy 1, policy_version 836858 (0.0006) [2023-12-27 11:26:06,115][20381] Updated weights for policy 1, policy_version 836868 (0.0008) [2023-12-27 11:26:06,180][20381] Updated weights for policy 1, policy_version 836878 (0.0007) [2023-12-27 11:26:06,249][20381] Updated weights for policy 1, policy_version 836888 (0.0007) [2023-12-27 11:26:06,309][20347] Updated weights for policy 0, policy_version 837603 (0.0008) [2023-12-27 11:26:06,375][20347] Updated weights for policy 0, policy_version 837613 (0.0009) [2023-12-27 11:26:06,441][20347] Updated weights for policy 0, policy_version 837623 (0.0009) [2023-12-27 11:26:06,898][20381] Updated weights for policy 1, policy_version 836898 (0.0006) [2023-12-27 11:26:06,962][20381] Updated weights for policy 1, policy_version 836908 (0.0006) [2023-12-27 11:26:07,021][20381] Updated weights for policy 1, policy_version 836918 (0.0007) [2023-12-27 11:26:07,138][20347] Updated weights for policy 0, policy_version 837633 (0.0007) [2023-12-27 11:26:07,191][20347] Updated weights for policy 0, policy_version 837643 (0.0008) [2023-12-27 11:26:07,249][20347] Updated weights for policy 0, policy_version 837653 (0.0007) [2023-12-27 11:26:07,297][20347] Updated weights for policy 0, policy_version 837663 (0.0007) [2023-12-27 11:26:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 19387.6, 300 sec: 19438.6). Total num frames: 428744704. Throughput: 0: 9711.3, 1: 9797.6. Samples: 428711032. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:26:07,364][19170] Avg episode reward: [(0, '970.800'), (1, '588.400')] [2023-12-27 11:26:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000836920_214278144.pth... [2023-12-27 11:26:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000837664_214466560.pth... [2023-12-27 11:26:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000835768_213983232.pth [2023-12-27 11:26:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000836512_214171648.pth [2023-12-27 11:26:07,709][20381] Updated weights for policy 1, policy_version 836928 (0.0007) [2023-12-27 11:26:07,758][20381] Updated weights for policy 1, policy_version 836938 (0.0007) [2023-12-27 11:26:07,818][20381] Updated weights for policy 1, policy_version 836948 (0.0008) [2023-12-27 11:26:08,018][20347] Updated weights for policy 0, policy_version 837673 (0.0006) [2023-12-27 11:26:08,071][20347] Updated weights for policy 0, policy_version 837683 (0.0008) [2023-12-27 11:26:08,121][20347] Updated weights for policy 0, policy_version 837693 (0.0008) [2023-12-27 11:26:08,518][20381] Updated weights for policy 1, policy_version 836958 (0.0007) [2023-12-27 11:26:08,578][20381] Updated weights for policy 1, policy_version 836968 (0.0006) [2023-12-27 11:26:08,636][20381] Updated weights for policy 1, policy_version 836978 (0.0011) [2023-12-27 11:26:08,856][20347] Updated weights for policy 0, policy_version 837703 (0.0007) [2023-12-27 11:26:08,914][20347] Updated weights for policy 0, policy_version 837713 (0.0008) [2023-12-27 11:26:08,975][20347] Updated weights for policy 0, policy_version 837723 (0.0008) [2023-12-27 11:26:09,342][20381] Updated weights for policy 1, policy_version 836988 (0.0011) [2023-12-27 11:26:09,414][20381] Updated weights for policy 1, policy_version 836998 (0.0011) [2023-12-27 11:26:09,474][20381] Updated weights for policy 1, policy_version 837008 (0.0008) [2023-12-27 11:26:09,759][20347] Updated weights for policy 0, policy_version 837733 (0.0008) [2023-12-27 11:26:09,817][20347] Updated weights for policy 0, policy_version 837743 (0.0008) [2023-12-27 11:26:09,880][20347] Updated weights for policy 0, policy_version 837753 (0.0007) [2023-12-27 11:26:10,190][20381] Updated weights for policy 1, policy_version 837018 (0.0010) [2023-12-27 11:26:10,252][20381] Updated weights for policy 1, policy_version 837028 (0.0008) [2023-12-27 11:26:10,310][20381] Updated weights for policy 1, policy_version 837038 (0.0008) [2023-12-27 11:26:10,359][20381] Updated weights for policy 1, policy_version 837048 (0.0006) [2023-12-27 11:26:10,645][20347] Updated weights for policy 0, policy_version 837763 (0.0008) [2023-12-27 11:26:10,705][20347] Updated weights for policy 0, policy_version 837773 (0.0008) [2023-12-27 11:26:10,770][20347] Updated weights for policy 0, policy_version 837783 (0.0007) [2023-12-27 11:26:11,059][20381] Updated weights for policy 1, policy_version 837058 (0.0008) [2023-12-27 11:26:11,109][20381] Updated weights for policy 1, policy_version 837068 (0.0008) [2023-12-27 11:26:11,179][20381] Updated weights for policy 1, policy_version 837078 (0.0008) [2023-12-27 11:26:11,500][20347] Updated weights for policy 0, policy_version 837793 (0.0007) [2023-12-27 11:26:11,564][20347] Updated weights for policy 0, policy_version 837803 (0.0008) [2023-12-27 11:26:11,624][20347] Updated weights for policy 0, policy_version 837813 (0.0008) [2023-12-27 11:26:11,684][20347] Updated weights for policy 0, policy_version 837823 (0.0007) [2023-12-27 11:26:11,930][20381] Updated weights for policy 1, policy_version 837088 (0.0010) [2023-12-27 11:26:11,997][20381] Updated weights for policy 1, policy_version 837098 (0.0011) [2023-12-27 11:26:12,058][20381] Updated weights for policy 1, policy_version 837108 (0.0007) [2023-12-27 11:26:12,362][19170] Fps is (10 sec: 18841.4, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 428834816. Throughput: 0: 9756.3, 1: 9830.7. Samples: 428828440. Policy #0 lag: (min: 26.0, avg: 41.0, max: 58.0) [2023-12-27 11:26:12,363][19170] Avg episode reward: [(0, '980.650'), (1, '503.250')] [2023-12-27 11:26:12,464][20347] Updated weights for policy 0, policy_version 837833 (0.0008) [2023-12-27 11:26:12,531][20347] Updated weights for policy 0, policy_version 837843 (0.0007) [2023-12-27 11:26:12,591][20347] Updated weights for policy 0, policy_version 837853 (0.0007) [2023-12-27 11:26:12,808][20381] Updated weights for policy 1, policy_version 837118 (0.0006) [2023-12-27 11:26:12,873][20381] Updated weights for policy 1, policy_version 837128 (0.0008) [2023-12-27 11:26:12,931][20381] Updated weights for policy 1, policy_version 837138 (0.0008) [2023-12-27 11:26:13,261][20347] Updated weights for policy 0, policy_version 837863 (0.0007) [2023-12-27 11:26:13,318][20347] Updated weights for policy 0, policy_version 837873 (0.0008) [2023-12-27 11:26:13,375][20347] Updated weights for policy 0, policy_version 837883 (0.0008) [2023-12-27 11:26:13,620][20381] Updated weights for policy 1, policy_version 837148 (0.0008) [2023-12-27 11:26:13,680][20381] Updated weights for policy 1, policy_version 837158 (0.0008) [2023-12-27 11:26:13,738][20381] Updated weights for policy 1, policy_version 837168 (0.0007) [2023-12-27 11:26:14,032][20347] Updated weights for policy 0, policy_version 837893 (0.0007) [2023-12-27 11:26:14,092][20347] Updated weights for policy 0, policy_version 837903 (0.0009) [2023-12-27 11:26:14,157][20347] Updated weights for policy 0, policy_version 837913 (0.0009) [2023-12-27 11:26:14,471][20381] Updated weights for policy 1, policy_version 837178 (0.0008) [2023-12-27 11:26:14,531][20381] Updated weights for policy 1, policy_version 837188 (0.0008) [2023-12-27 11:26:14,584][20381] Updated weights for policy 1, policy_version 837198 (0.0008) [2023-12-27 11:26:14,641][20381] Updated weights for policy 1, policy_version 837208 (0.0007) [2023-12-27 11:26:14,889][20347] Updated weights for policy 0, policy_version 837923 (0.0009) [2023-12-27 11:26:14,948][20347] Updated weights for policy 0, policy_version 837933 (0.0008) [2023-12-27 11:26:15,013][20347] Updated weights for policy 0, policy_version 837943 (0.0008) [2023-12-27 11:26:15,361][20381] Updated weights for policy 1, policy_version 837218 (0.0007) [2023-12-27 11:26:15,419][20381] Updated weights for policy 1, policy_version 837228 (0.0008) [2023-12-27 11:26:15,482][20381] Updated weights for policy 1, policy_version 837238 (0.0008) [2023-12-27 11:26:15,658][20347] Updated weights for policy 0, policy_version 837953 (0.0008) [2023-12-27 11:26:15,717][20347] Updated weights for policy 0, policy_version 837963 (0.0007) [2023-12-27 11:26:15,785][20347] Updated weights for policy 0, policy_version 837973 (0.0008) [2023-12-27 11:26:15,857][20347] Updated weights for policy 0, policy_version 837983 (0.0006) [2023-12-27 11:26:16,155][20381] Updated weights for policy 1, policy_version 837248 (0.0008) [2023-12-27 11:26:16,206][20381] Updated weights for policy 1, policy_version 837258 (0.0006) [2023-12-27 11:26:16,262][20381] Updated weights for policy 1, policy_version 837268 (0.0006) [2023-12-27 11:26:16,532][20347] Updated weights for policy 0, policy_version 837993 (0.0008) [2023-12-27 11:26:16,588][20347] Updated weights for policy 0, policy_version 838003 (0.0008) [2023-12-27 11:26:16,646][20347] Updated weights for policy 0, policy_version 838013 (0.0007) [2023-12-27 11:26:16,952][20381] Updated weights for policy 1, policy_version 837278 (0.0008) [2023-12-27 11:26:17,006][20381] Updated weights for policy 1, policy_version 837288 (0.0008) [2023-12-27 11:26:17,057][20381] Updated weights for policy 1, policy_version 837298 (0.0008) [2023-12-27 11:26:17,358][20347] Updated weights for policy 0, policy_version 838023 (0.0008) [2023-12-27 11:26:17,362][19170] Fps is (10 sec: 18842.2, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 428933120. Throughput: 0: 9753.2, 1: 9816.8. Samples: 428945948. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '535.910')] [2023-12-27 11:26:17,418][20347] Updated weights for policy 0, policy_version 838033 (0.0008) [2023-12-27 11:26:17,477][20347] Updated weights for policy 0, policy_version 838043 (0.0006) [2023-12-27 11:26:17,732][20381] Updated weights for policy 1, policy_version 837308 (0.0007) [2023-12-27 11:26:17,784][20381] Updated weights for policy 1, policy_version 837318 (0.0006) [2023-12-27 11:26:17,839][20381] Updated weights for policy 1, policy_version 837328 (0.0006) [2023-12-27 11:26:18,130][20347] Updated weights for policy 0, policy_version 838053 (0.0007) [2023-12-27 11:26:18,191][20347] Updated weights for policy 0, policy_version 838063 (0.0006) [2023-12-27 11:26:18,256][20347] Updated weights for policy 0, policy_version 838073 (0.0007) [2023-12-27 11:26:18,510][20381] Updated weights for policy 1, policy_version 837338 (0.0008) [2023-12-27 11:26:18,564][20381] Updated weights for policy 1, policy_version 837348 (0.0009) [2023-12-27 11:26:18,625][20381] Updated weights for policy 1, policy_version 837358 (0.0009) [2023-12-27 11:26:18,684][20381] Updated weights for policy 1, policy_version 837368 (0.0009) [2023-12-27 11:26:18,862][20347] Updated weights for policy 0, policy_version 838083 (0.0006) [2023-12-27 11:26:18,925][20347] Updated weights for policy 0, policy_version 838093 (0.0006) [2023-12-27 11:26:18,984][20347] Updated weights for policy 0, policy_version 838103 (0.0007) [2023-12-27 11:26:19,558][20381] Updated weights for policy 1, policy_version 837378 (0.0010) [2023-12-27 11:26:19,622][20381] Updated weights for policy 1, policy_version 837388 (0.0010) [2023-12-27 11:26:19,679][20381] Updated weights for policy 1, policy_version 837398 (0.0010) [2023-12-27 11:26:19,683][20347] Updated weights for policy 0, policy_version 838113 (0.0008) [2023-12-27 11:26:19,743][20347] Updated weights for policy 0, policy_version 838123 (0.0009) [2023-12-27 11:26:19,802][20347] Updated weights for policy 0, policy_version 838133 (0.0008) [2023-12-27 11:26:19,865][20347] Updated weights for policy 0, policy_version 838143 (0.0008) [2023-12-27 11:26:20,592][20381] Updated weights for policy 1, policy_version 837408 (0.0009) [2023-12-27 11:26:20,652][20381] Updated weights for policy 1, policy_version 837418 (0.0009) [2023-12-27 11:26:20,715][20381] Updated weights for policy 1, policy_version 837428 (0.0008) [2023-12-27 11:26:20,718][20347] Updated weights for policy 0, policy_version 838153 (0.0009) [2023-12-27 11:26:20,778][20347] Updated weights for policy 0, policy_version 838163 (0.0009) [2023-12-27 11:26:20,836][20347] Updated weights for policy 0, policy_version 838173 (0.0009) [2023-12-27 11:26:21,512][20381] Updated weights for policy 1, policy_version 837438 (0.0009) [2023-12-27 11:26:21,565][20381] Updated weights for policy 1, policy_version 837448 (0.0009) [2023-12-27 11:26:21,626][20381] Updated weights for policy 1, policy_version 837458 (0.0007) [2023-12-27 11:26:21,768][20347] Updated weights for policy 0, policy_version 838183 (0.0009) [2023-12-27 11:26:21,833][20347] Updated weights for policy 0, policy_version 838193 (0.0009) [2023-12-27 11:26:21,898][20347] Updated weights for policy 0, policy_version 838203 (0.0010) [2023-12-27 11:26:22,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 429023232. Throughput: 0: 9715.2, 1: 9761.6. Samples: 429002992. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:22,363][19170] Avg episode reward: [(0, '990.220'), (1, '110.720')] [2023-12-27 11:26:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000838208_214605824.pth... [2023-12-27 11:26:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000837088_214319104.pth [2023-12-27 11:26:22,388][20381] Updated weights for policy 1, policy_version 837468 (0.0008) [2023-12-27 11:26:22,452][20381] Updated weights for policy 1, policy_version 837478 (0.0006) [2023-12-27 11:26:22,510][20381] Updated weights for policy 1, policy_version 837488 (0.0008) [2023-12-27 11:26:22,560][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000837496_214425600.pth... [2023-12-27 11:26:22,563][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000836344_214130688.pth [2023-12-27 11:26:22,620][20347] Updated weights for policy 0, policy_version 838213 (0.0007) [2023-12-27 11:26:22,687][20347] Updated weights for policy 0, policy_version 838223 (0.0007) [2023-12-27 11:26:22,752][20347] Updated weights for policy 0, policy_version 838233 (0.0008) [2023-12-27 11:26:23,214][20381] Updated weights for policy 1, policy_version 837498 (0.0009) [2023-12-27 11:26:23,276][20381] Updated weights for policy 1, policy_version 837508 (0.0008) [2023-12-27 11:26:23,336][20381] Updated weights for policy 1, policy_version 837518 (0.0008) [2023-12-27 11:26:23,397][20381] Updated weights for policy 1, policy_version 837528 (0.0008) [2023-12-27 11:26:23,459][20347] Updated weights for policy 0, policy_version 838243 (0.0007) [2023-12-27 11:26:23,519][20347] Updated weights for policy 0, policy_version 838253 (0.0009) [2023-12-27 11:26:23,579][20347] Updated weights for policy 0, policy_version 838263 (0.0009) [2023-12-27 11:26:24,109][20381] Updated weights for policy 1, policy_version 837538 (0.0007) [2023-12-27 11:26:24,171][20381] Updated weights for policy 1, policy_version 837548 (0.0008) [2023-12-27 11:26:24,232][20381] Updated weights for policy 1, policy_version 837558 (0.0008) [2023-12-27 11:26:24,257][20347] Updated weights for policy 0, policy_version 838273 (0.0007) [2023-12-27 11:26:24,314][20347] Updated weights for policy 0, policy_version 838283 (0.0008) [2023-12-27 11:26:24,371][20347] Updated weights for policy 0, policy_version 838293 (0.0008) [2023-12-27 11:26:24,424][20347] Updated weights for policy 0, policy_version 838303 (0.0008) [2023-12-27 11:26:24,904][20381] Updated weights for policy 1, policy_version 837568 (0.0008) [2023-12-27 11:26:24,954][20381] Updated weights for policy 1, policy_version 837578 (0.0008) [2023-12-27 11:26:25,013][20381] Updated weights for policy 1, policy_version 837588 (0.0008) [2023-12-27 11:26:25,157][20347] Updated weights for policy 0, policy_version 838313 (0.0007) [2023-12-27 11:26:25,218][20347] Updated weights for policy 0, policy_version 838323 (0.0006) [2023-12-27 11:26:25,279][20347] Updated weights for policy 0, policy_version 838333 (0.0006) [2023-12-27 11:26:25,744][20381] Updated weights for policy 1, policy_version 837598 (0.0007) [2023-12-27 11:26:25,801][20381] Updated weights for policy 1, policy_version 837608 (0.0008) [2023-12-27 11:26:25,858][20381] Updated weights for policy 1, policy_version 837618 (0.0008) [2023-12-27 11:26:25,923][20347] Updated weights for policy 0, policy_version 838343 (0.0008) [2023-12-27 11:26:25,983][20347] Updated weights for policy 0, policy_version 838353 (0.0007) [2023-12-27 11:26:26,042][20347] Updated weights for policy 0, policy_version 838363 (0.0008) [2023-12-27 11:26:26,566][20381] Updated weights for policy 1, policy_version 837628 (0.0008) [2023-12-27 11:26:26,630][20381] Updated weights for policy 1, policy_version 837638 (0.0008) [2023-12-27 11:26:26,685][20381] Updated weights for policy 1, policy_version 837648 (0.0007) [2023-12-27 11:26:26,773][20347] Updated weights for policy 0, policy_version 838373 (0.0007) [2023-12-27 11:26:26,835][20347] Updated weights for policy 0, policy_version 838383 (0.0007) [2023-12-27 11:26:26,888][20347] Updated weights for policy 0, policy_version 838393 (0.0009) [2023-12-27 11:26:27,321][20381] Updated weights for policy 1, policy_version 837658 (0.0008) [2023-12-27 11:26:27,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 429121536. Throughput: 0: 9635.1, 1: 9709.1. Samples: 429114960. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '193.690')] [2023-12-27 11:26:27,380][20381] Updated weights for policy 1, policy_version 837668 (0.0007) [2023-12-27 11:26:27,432][20381] Updated weights for policy 1, policy_version 837678 (0.0009) [2023-12-27 11:26:27,485][20381] Updated weights for policy 1, policy_version 837688 (0.0008) [2023-12-27 11:26:27,590][20347] Updated weights for policy 0, policy_version 838403 (0.0008) [2023-12-27 11:26:27,643][20347] Updated weights for policy 0, policy_version 838413 (0.0008) [2023-12-27 11:26:27,696][20347] Updated weights for policy 0, policy_version 838423 (0.0008) [2023-12-27 11:26:28,193][20381] Updated weights for policy 1, policy_version 837698 (0.0008) [2023-12-27 11:26:28,249][20381] Updated weights for policy 1, policy_version 837708 (0.0008) [2023-12-27 11:26:28,310][20381] Updated weights for policy 1, policy_version 837718 (0.0007) [2023-12-27 11:26:28,377][20347] Updated weights for policy 0, policy_version 838433 (0.0007) [2023-12-27 11:26:28,439][20347] Updated weights for policy 0, policy_version 838443 (0.0007) [2023-12-27 11:26:28,496][20347] Updated weights for policy 0, policy_version 838453 (0.0009) [2023-12-27 11:26:28,552][20347] Updated weights for policy 0, policy_version 838463 (0.0010) [2023-12-27 11:26:29,035][20381] Updated weights for policy 1, policy_version 837728 (0.0008) [2023-12-27 11:26:29,089][20381] Updated weights for policy 1, policy_version 837738 (0.0008) [2023-12-27 11:26:29,142][20381] Updated weights for policy 1, policy_version 837748 (0.0008) [2023-12-27 11:26:29,230][20347] Updated weights for policy 0, policy_version 838473 (0.0011) [2023-12-27 11:26:29,293][20347] Updated weights for policy 0, policy_version 838483 (0.0011) [2023-12-27 11:26:29,357][20347] Updated weights for policy 0, policy_version 838493 (0.0011) [2023-12-27 11:26:29,872][20381] Updated weights for policy 1, policy_version 837758 (0.0008) [2023-12-27 11:26:29,935][20381] Updated weights for policy 1, policy_version 837768 (0.0011) [2023-12-27 11:26:29,989][20381] Updated weights for policy 1, policy_version 837778 (0.0011) [2023-12-27 11:26:30,051][20347] Updated weights for policy 0, policy_version 838503 (0.0009) [2023-12-27 11:26:30,103][20347] Updated weights for policy 0, policy_version 838513 (0.0008) [2023-12-27 11:26:30,164][20347] Updated weights for policy 0, policy_version 838523 (0.0008) [2023-12-27 11:26:30,714][20381] Updated weights for policy 1, policy_version 837788 (0.0011) [2023-12-27 11:26:30,774][20381] Updated weights for policy 1, policy_version 837798 (0.0011) [2023-12-27 11:26:30,827][20381] Updated weights for policy 1, policy_version 837808 (0.0010) [2023-12-27 11:26:30,881][20347] Updated weights for policy 0, policy_version 838533 (0.0007) [2023-12-27 11:26:30,935][20347] Updated weights for policy 0, policy_version 838543 (0.0008) [2023-12-27 11:26:30,983][20347] Updated weights for policy 0, policy_version 838553 (0.0008) [2023-12-27 11:26:31,575][20381] Updated weights for policy 1, policy_version 837818 (0.0009) [2023-12-27 11:26:31,637][20381] Updated weights for policy 1, policy_version 837828 (0.0007) [2023-12-27 11:26:31,704][20381] Updated weights for policy 1, policy_version 837838 (0.0009) [2023-12-27 11:26:31,731][20347] Updated weights for policy 0, policy_version 838563 (0.0008) [2023-12-27 11:26:31,765][20381] Updated weights for policy 1, policy_version 837848 (0.0008) [2023-12-27 11:26:31,789][20347] Updated weights for policy 0, policy_version 838573 (0.0008) [2023-12-27 11:26:31,847][20347] Updated weights for policy 0, policy_version 838583 (0.0006) [2023-12-27 11:26:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 429219840. Throughput: 0: 9652.7, 1: 9727.5. Samples: 429232752. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:32,363][19170] Avg episode reward: [(0, '990.200'), (1, '285.670')] [2023-12-27 11:26:32,437][20381] Updated weights for policy 1, policy_version 837858 (0.0006) [2023-12-27 11:26:32,502][20381] Updated weights for policy 1, policy_version 837868 (0.0006) [2023-12-27 11:26:32,558][20381] Updated weights for policy 1, policy_version 837878 (0.0006) [2023-12-27 11:26:32,599][20347] Updated weights for policy 0, policy_version 838593 (0.0006) [2023-12-27 11:26:32,660][20347] Updated weights for policy 0, policy_version 838603 (0.0008) [2023-12-27 11:26:32,739][20347] Updated weights for policy 0, policy_version 838613 (0.0007) [2023-12-27 11:26:32,793][20291] KL-divergence is very high: 316.6906 [2023-12-27 11:26:32,799][20347] Updated weights for policy 0, policy_version 838623 (0.0008) [2023-12-27 11:26:33,193][20381] Updated weights for policy 1, policy_version 837888 (0.0008) [2023-12-27 11:26:33,248][20381] Updated weights for policy 1, policy_version 837898 (0.0008) [2023-12-27 11:26:33,298][20381] Updated weights for policy 1, policy_version 837908 (0.0007) [2023-12-27 11:26:33,447][20291] KL-divergence is very high: 226.5494 [2023-12-27 11:26:33,495][20291] KL-divergence is very high: 102.9935 [2023-12-27 11:26:33,495][20347] Updated weights for policy 0, policy_version 838633 (0.0008) [2023-12-27 11:26:33,535][20291] KL-divergence is very high: 201.2012 [2023-12-27 11:26:33,545][20347] Updated weights for policy 0, policy_version 838643 (0.0007) [2023-12-27 11:26:33,578][20291] KL-divergence is very high: 164.1778 [2023-12-27 11:26:33,602][20347] Updated weights for policy 0, policy_version 838653 (0.0007) [2023-12-27 11:26:33,970][20381] Updated weights for policy 1, policy_version 837918 (0.0007) [2023-12-27 11:26:34,017][20381] Updated weights for policy 1, policy_version 837928 (0.0008) [2023-12-27 11:26:34,070][20381] Updated weights for policy 1, policy_version 837938 (0.0008) [2023-12-27 11:26:34,317][20347] Updated weights for policy 0, policy_version 838663 (0.0008) [2023-12-27 11:26:34,381][20347] Updated weights for policy 0, policy_version 838673 (0.0007) [2023-12-27 11:26:34,452][20347] Updated weights for policy 0, policy_version 838683 (0.0006) [2023-12-27 11:26:34,840][20381] Updated weights for policy 1, policy_version 837948 (0.0008) [2023-12-27 11:26:34,889][20381] Updated weights for policy 1, policy_version 837958 (0.0008) [2023-12-27 11:26:34,938][20381] Updated weights for policy 1, policy_version 837968 (0.0008) [2023-12-27 11:26:35,082][20347] Updated weights for policy 0, policy_version 838693 (0.0006) [2023-12-27 11:26:35,139][20347] Updated weights for policy 0, policy_version 838703 (0.0007) [2023-12-27 11:26:35,194][20347] Updated weights for policy 0, policy_version 838713 (0.0008) [2023-12-27 11:26:35,636][20381] Updated weights for policy 1, policy_version 837978 (0.0007) [2023-12-27 11:26:35,701][20381] Updated weights for policy 1, policy_version 837988 (0.0005) [2023-12-27 11:26:35,751][20347] Updated weights for policy 0, policy_version 838723 (0.0006) [2023-12-27 11:26:35,768][20381] Updated weights for policy 1, policy_version 837998 (0.0005) [2023-12-27 11:26:35,805][20347] Updated weights for policy 0, policy_version 838733 (0.0007) [2023-12-27 11:26:35,829][20381] Updated weights for policy 1, policy_version 838008 (0.0006) [2023-12-27 11:26:35,868][20347] Updated weights for policy 0, policy_version 838743 (0.0008) [2023-12-27 11:26:36,517][20381] Updated weights for policy 1, policy_version 838018 (0.0008) [2023-12-27 11:26:36,555][20347] Updated weights for policy 0, policy_version 838753 (0.0007) [2023-12-27 11:26:36,577][20381] Updated weights for policy 1, policy_version 838028 (0.0008) [2023-12-27 11:26:36,619][20347] Updated weights for policy 0, policy_version 838763 (0.0006) [2023-12-27 11:26:36,638][20381] Updated weights for policy 1, policy_version 838038 (0.0008) [2023-12-27 11:26:36,681][20347] Updated weights for policy 0, policy_version 838773 (0.0006) [2023-12-27 11:26:36,745][20347] Updated weights for policy 0, policy_version 838783 (0.0007) [2023-12-27 11:26:37,336][20347] Updated weights for policy 0, policy_version 838793 (0.0006) [2023-12-27 11:26:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 429318144. Throughput: 0: 9676.6, 1: 9715.0. Samples: 429292136. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:37,363][19170] Avg episode reward: [(0, '951.240'), (1, '362.410')] [2023-12-27 11:26:37,397][20347] Updated weights for policy 0, policy_version 838803 (0.0006) [2023-12-27 11:26:37,402][20381] Updated weights for policy 1, policy_version 838048 (0.0007) [2023-12-27 11:26:37,456][20381] Updated weights for policy 1, policy_version 838058 (0.0006) [2023-12-27 11:26:37,461][20347] Updated weights for policy 0, policy_version 838813 (0.0007) [2023-12-27 11:26:37,479][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000838816_214761472.pth... [2023-12-27 11:26:37,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000837664_214466560.pth [2023-12-27 11:26:37,513][20381] Updated weights for policy 1, policy_version 838068 (0.0006) [2023-12-27 11:26:37,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000838072_214573056.pth... [2023-12-27 11:26:37,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000836920_214278144.pth [2023-12-27 11:26:38,121][20347] Updated weights for policy 0, policy_version 838823 (0.0007) [2023-12-27 11:26:38,176][20381] Updated weights for policy 1, policy_version 838078 (0.0007) [2023-12-27 11:26:38,190][20347] Updated weights for policy 0, policy_version 838833 (0.0008) [2023-12-27 11:26:38,232][20381] Updated weights for policy 1, policy_version 838088 (0.0006) [2023-12-27 11:26:38,252][20347] Updated weights for policy 0, policy_version 838843 (0.0008) [2023-12-27 11:26:38,290][20381] Updated weights for policy 1, policy_version 838098 (0.0006) [2023-12-27 11:26:38,975][20347] Updated weights for policy 0, policy_version 838853 (0.0008) [2023-12-27 11:26:39,001][20381] Updated weights for policy 1, policy_version 838108 (0.0008) [2023-12-27 11:26:39,034][20347] Updated weights for policy 0, policy_version 838863 (0.0006) [2023-12-27 11:26:39,077][20381] Updated weights for policy 1, policy_version 838118 (0.0006) [2023-12-27 11:26:39,082][20347] Updated weights for policy 0, policy_version 838873 (0.0006) [2023-12-27 11:26:39,128][20381] Updated weights for policy 1, policy_version 838128 (0.0006) [2023-12-27 11:26:39,842][20347] Updated weights for policy 0, policy_version 838883 (0.0006) [2023-12-27 11:26:39,898][20381] Updated weights for policy 1, policy_version 838138 (0.0008) [2023-12-27 11:26:39,904][20347] Updated weights for policy 0, policy_version 838893 (0.0008) [2023-12-27 11:26:39,960][20381] Updated weights for policy 1, policy_version 838148 (0.0007) [2023-12-27 11:26:39,968][20347] Updated weights for policy 0, policy_version 838903 (0.0008) [2023-12-27 11:26:40,028][20381] Updated weights for policy 1, policy_version 838158 (0.0007) [2023-12-27 11:26:40,092][20381] Updated weights for policy 1, policy_version 838168 (0.0007) [2023-12-27 11:26:40,684][20347] Updated weights for policy 0, policy_version 838913 (0.0008) [2023-12-27 11:26:40,753][20347] Updated weights for policy 0, policy_version 838923 (0.0010) [2023-12-27 11:26:40,799][20381] Updated weights for policy 1, policy_version 838178 (0.0008) [2023-12-27 11:26:40,820][20347] Updated weights for policy 0, policy_version 838933 (0.0011) [2023-12-27 11:26:40,852][20381] Updated weights for policy 1, policy_version 838188 (0.0005) [2023-12-27 11:26:40,883][20347] Updated weights for policy 0, policy_version 838943 (0.0010) [2023-12-27 11:26:40,914][20381] Updated weights for policy 1, policy_version 838198 (0.0007) [2023-12-27 11:26:41,584][20347] Updated weights for policy 0, policy_version 838953 (0.0008) [2023-12-27 11:26:41,655][20347] Updated weights for policy 0, policy_version 838963 (0.0008) [2023-12-27 11:26:41,723][20347] Updated weights for policy 0, policy_version 838973 (0.0008) [2023-12-27 11:26:41,761][20381] Updated weights for policy 1, policy_version 838208 (0.0007) [2023-12-27 11:26:41,829][20381] Updated weights for policy 1, policy_version 838218 (0.0006) [2023-12-27 11:26:41,895][20381] Updated weights for policy 1, policy_version 838228 (0.0009) [2023-12-27 11:26:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 429416448. Throughput: 0: 9727.2, 1: 9685.3. Samples: 429411088. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:42,364][19170] Avg episode reward: [(0, '941.380'), (1, '458.130')] [2023-12-27 11:26:42,488][20347] Updated weights for policy 0, policy_version 838983 (0.0008) [2023-12-27 11:26:42,553][20347] Updated weights for policy 0, policy_version 838993 (0.0008) [2023-12-27 11:26:42,590][20381] Updated weights for policy 1, policy_version 838238 (0.0007) [2023-12-27 11:26:42,616][20347] Updated weights for policy 0, policy_version 839003 (0.0008) [2023-12-27 11:26:42,650][20381] Updated weights for policy 1, policy_version 838248 (0.0008) [2023-12-27 11:26:42,709][20381] Updated weights for policy 1, policy_version 838258 (0.0008) [2023-12-27 11:26:43,253][20347] Updated weights for policy 0, policy_version 839013 (0.0007) [2023-12-27 11:26:43,305][20347] Updated weights for policy 0, policy_version 839023 (0.0006) [2023-12-27 11:26:43,363][20347] Updated weights for policy 0, policy_version 839033 (0.0005) [2023-12-27 11:26:43,422][20381] Updated weights for policy 1, policy_version 838268 (0.0006) [2023-12-27 11:26:43,476][20381] Updated weights for policy 1, policy_version 838278 (0.0007) [2023-12-27 11:26:43,530][20381] Updated weights for policy 1, policy_version 838288 (0.0007) [2023-12-27 11:26:44,010][20347] Updated weights for policy 0, policy_version 839043 (0.0007) [2023-12-27 11:26:44,072][20347] Updated weights for policy 0, policy_version 839053 (0.0008) [2023-12-27 11:26:44,132][20347] Updated weights for policy 0, policy_version 839063 (0.0008) [2023-12-27 11:26:44,220][20381] Updated weights for policy 1, policy_version 838298 (0.0007) [2023-12-27 11:26:44,273][20381] Updated weights for policy 1, policy_version 838308 (0.0008) [2023-12-27 11:26:44,323][20381] Updated weights for policy 1, policy_version 838318 (0.0008) [2023-12-27 11:26:44,380][20381] Updated weights for policy 1, policy_version 838328 (0.0007) [2023-12-27 11:26:44,760][20347] Updated weights for policy 0, policy_version 839073 (0.0007) [2023-12-27 11:26:44,824][20347] Updated weights for policy 0, policy_version 839083 (0.0008) [2023-12-27 11:26:44,893][20347] Updated weights for policy 0, policy_version 839093 (0.0008) [2023-12-27 11:26:44,961][20347] Updated weights for policy 0, policy_version 839103 (0.0007) [2023-12-27 11:26:45,197][20381] Updated weights for policy 1, policy_version 838338 (0.0007) [2023-12-27 11:26:45,256][20381] Updated weights for policy 1, policy_version 838348 (0.0007) [2023-12-27 11:26:45,316][20381] Updated weights for policy 1, policy_version 838358 (0.0008) [2023-12-27 11:26:45,657][20347] Updated weights for policy 0, policy_version 839113 (0.0008) [2023-12-27 11:26:45,715][20347] Updated weights for policy 0, policy_version 839123 (0.0007) [2023-12-27 11:26:45,764][20347] Updated weights for policy 0, policy_version 839133 (0.0007) [2023-12-27 11:26:46,003][20381] Updated weights for policy 1, policy_version 838368 (0.0006) [2023-12-27 11:26:46,060][20381] Updated weights for policy 1, policy_version 838378 (0.0006) [2023-12-27 11:26:46,119][20381] Updated weights for policy 1, policy_version 838388 (0.0005) [2023-12-27 11:26:46,509][20347] Updated weights for policy 0, policy_version 839143 (0.0006) [2023-12-27 11:26:46,561][20291] KL-divergence is very high: 647.3873 [2023-12-27 11:26:46,571][20291] KL-divergence is very high: 669.2674 [2023-12-27 11:26:46,571][20347] Updated weights for policy 0, policy_version 839153 (0.0007) [2023-12-27 11:26:46,603][20291] KL-divergence is very high: 214.6844 [2023-12-27 11:26:46,616][20291] KL-divergence is very high: 118.4956 [2023-12-27 11:26:46,627][20347] Updated weights for policy 0, policy_version 839163 (0.0006) [2023-12-27 11:26:46,711][20381] Updated weights for policy 1, policy_version 838398 (0.0007) [2023-12-27 11:26:46,767][20381] Updated weights for policy 1, policy_version 838408 (0.0006) [2023-12-27 11:26:46,824][20381] Updated weights for policy 1, policy_version 838418 (0.0007) [2023-12-27 11:26:47,271][20291] KL-divergence is very high: 715.8586 [2023-12-27 11:26:47,282][20291] KL-divergence is very high: 584.9985 [2023-12-27 11:26:47,292][20347] Updated weights for policy 0, policy_version 839173 (0.0007) [2023-12-27 11:26:47,302][20291] KL-divergence is very high: 257.9965 [2023-12-27 11:26:47,313][20291] KL-divergence is very high: 589.1133 [2023-12-27 11:26:47,327][20291] KL-divergence is very high: 454.1975 [2023-12-27 11:26:47,352][20347] Updated weights for policy 0, policy_version 839183 (0.0006) [2023-12-27 11:26:47,352][20291] KL-divergence is very high: 187.7776 [2023-12-27 11:26:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 429514752. Throughput: 0: 9791.5, 1: 9714.2. Samples: 429529356. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:47,363][19170] Avg episode reward: [(0, '883.020'), (1, '431.480')] [2023-12-27 11:26:47,364][20291] KL-divergence is very high: 138.7676 [2023-12-27 11:26:47,376][20291] KL-divergence is very high: 206.6211 [2023-12-27 11:26:47,382][20291] KL-divergence is very high: 120.1041 [2023-12-27 11:26:47,399][20291] KL-divergence is very high: 149.9291 [2023-12-27 11:26:47,409][20347] Updated weights for policy 0, policy_version 839193 (0.0007) [2023-12-27 11:26:47,410][20291] KL-divergence is very high: 439.4840 [2023-12-27 11:26:47,420][20291] KL-divergence is very high: 617.1982 [2023-12-27 11:26:47,440][20291] KL-divergence is very high: 163.2673 [2023-12-27 11:26:47,510][20381] Updated weights for policy 1, policy_version 838428 (0.0009) [2023-12-27 11:26:47,561][20381] Updated weights for policy 1, policy_version 838438 (0.0008) [2023-12-27 11:26:47,614][20381] Updated weights for policy 1, policy_version 838448 (0.0006) [2023-12-27 11:26:48,094][20347] Updated weights for policy 0, policy_version 839203 (0.0008) [2023-12-27 11:26:48,148][20347] Updated weights for policy 0, policy_version 839213 (0.0007) [2023-12-27 11:26:48,206][20347] Updated weights for policy 0, policy_version 839223 (0.0007) [2023-12-27 11:26:48,313][20381] Updated weights for policy 1, policy_version 838458 (0.0005) [2023-12-27 11:26:48,379][20381] Updated weights for policy 1, policy_version 838468 (0.0007) [2023-12-27 11:26:48,446][20381] Updated weights for policy 1, policy_version 838478 (0.0006) [2023-12-27 11:26:48,503][20381] Updated weights for policy 1, policy_version 838488 (0.0006) [2023-12-27 11:26:48,899][20347] Updated weights for policy 0, policy_version 839233 (0.0006) [2023-12-27 11:26:48,962][20347] Updated weights for policy 0, policy_version 839243 (0.0008) [2023-12-27 11:26:49,019][20347] Updated weights for policy 0, policy_version 839253 (0.0007) [2023-12-27 11:26:49,078][20347] Updated weights for policy 0, policy_version 839263 (0.0008) [2023-12-27 11:26:49,156][20381] Updated weights for policy 1, policy_version 838498 (0.0008) [2023-12-27 11:26:49,204][20381] Updated weights for policy 1, policy_version 838508 (0.0007) [2023-12-27 11:26:49,269][20381] Updated weights for policy 1, policy_version 838518 (0.0008) [2023-12-27 11:26:49,815][20347] Updated weights for policy 0, policy_version 839273 (0.0008) [2023-12-27 11:26:49,880][20347] Updated weights for policy 0, policy_version 839283 (0.0006) [2023-12-27 11:26:49,949][20347] Updated weights for policy 0, policy_version 839293 (0.0007) [2023-12-27 11:26:50,034][20381] Updated weights for policy 1, policy_version 838528 (0.0007) [2023-12-27 11:26:50,100][20381] Updated weights for policy 1, policy_version 838538 (0.0007) [2023-12-27 11:26:50,164][20381] Updated weights for policy 1, policy_version 838548 (0.0008) [2023-12-27 11:26:50,671][20347] Updated weights for policy 0, policy_version 839303 (0.0008) [2023-12-27 11:26:50,729][20347] Updated weights for policy 0, policy_version 839313 (0.0008) [2023-12-27 11:26:50,796][20347] Updated weights for policy 0, policy_version 839323 (0.0008) [2023-12-27 11:26:50,890][20381] Updated weights for policy 1, policy_version 838558 (0.0008) [2023-12-27 11:26:50,954][20381] Updated weights for policy 1, policy_version 838568 (0.0008) [2023-12-27 11:26:51,018][20381] Updated weights for policy 1, policy_version 838578 (0.0008) [2023-12-27 11:26:51,595][20347] Updated weights for policy 0, policy_version 839333 (0.0007) [2023-12-27 11:26:51,664][20347] Updated weights for policy 0, policy_version 839343 (0.0007) [2023-12-27 11:26:51,724][20381] Updated weights for policy 1, policy_version 838588 (0.0007) [2023-12-27 11:26:51,724][20347] Updated weights for policy 0, policy_version 839353 (0.0008) [2023-12-27 11:26:51,790][20381] Updated weights for policy 1, policy_version 838598 (0.0009) [2023-12-27 11:26:51,849][20381] Updated weights for policy 1, policy_version 838608 (0.0009) [2023-12-27 11:26:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 429613056. Throughput: 0: 9797.3, 1: 9695.1. Samples: 429588188. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:52,363][19170] Avg episode reward: [(0, '674.470'), (1, '416.750')] [2023-12-27 11:26:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000838616_214712320.pth... [2023-12-27 11:26:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000839360_214900736.pth... [2023-12-27 11:26:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000837496_214425600.pth [2023-12-27 11:26:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000838208_214605824.pth [2023-12-27 11:26:52,450][20347] Updated weights for policy 0, policy_version 839363 (0.0009) [2023-12-27 11:26:52,508][20347] Updated weights for policy 0, policy_version 839373 (0.0011) [2023-12-27 11:26:52,572][20347] Updated weights for policy 0, policy_version 839383 (0.0011) [2023-12-27 11:26:52,588][20381] Updated weights for policy 1, policy_version 838618 (0.0008) [2023-12-27 11:26:52,655][20381] Updated weights for policy 1, policy_version 838628 (0.0008) [2023-12-27 11:26:52,718][20381] Updated weights for policy 1, policy_version 838638 (0.0009) [2023-12-27 11:26:52,783][20381] Updated weights for policy 1, policy_version 838648 (0.0008) [2023-12-27 11:26:53,254][20347] Updated weights for policy 0, policy_version 839393 (0.0010) [2023-12-27 11:26:53,309][20347] Updated weights for policy 0, policy_version 839403 (0.0006) [2023-12-27 11:26:53,368][20347] Updated weights for policy 0, policy_version 839413 (0.0006) [2023-12-27 11:26:53,430][20347] Updated weights for policy 0, policy_version 839423 (0.0008) [2023-12-27 11:26:53,493][20381] Updated weights for policy 1, policy_version 838658 (0.0008) [2023-12-27 11:26:53,553][20381] Updated weights for policy 1, policy_version 838668 (0.0008) [2023-12-27 11:26:53,606][20381] Updated weights for policy 1, policy_version 838678 (0.0007) [2023-12-27 11:26:54,048][20347] Updated weights for policy 0, policy_version 839433 (0.0008) [2023-12-27 11:26:54,114][20347] Updated weights for policy 0, policy_version 839443 (0.0008) [2023-12-27 11:26:54,177][20347] Updated weights for policy 0, policy_version 839453 (0.0007) [2023-12-27 11:26:54,326][20381] Updated weights for policy 1, policy_version 838688 (0.0008) [2023-12-27 11:26:54,386][20381] Updated weights for policy 1, policy_version 838698 (0.0008) [2023-12-27 11:26:54,451][20381] Updated weights for policy 1, policy_version 838708 (0.0008) [2023-12-27 11:26:54,845][20347] Updated weights for policy 0, policy_version 839463 (0.0008) [2023-12-27 11:26:54,896][20347] Updated weights for policy 0, policy_version 839473 (0.0007) [2023-12-27 11:26:54,959][20347] Updated weights for policy 0, policy_version 839483 (0.0008) [2023-12-27 11:26:55,129][20381] Updated weights for policy 1, policy_version 838718 (0.0009) [2023-12-27 11:26:55,189][20381] Updated weights for policy 1, policy_version 838728 (0.0007) [2023-12-27 11:26:55,246][20381] Updated weights for policy 1, policy_version 838738 (0.0008) [2023-12-27 11:26:55,677][20347] Updated weights for policy 0, policy_version 839493 (0.0007) [2023-12-27 11:26:55,726][20347] Updated weights for policy 0, policy_version 839503 (0.0007) [2023-12-27 11:26:55,785][20347] Updated weights for policy 0, policy_version 839513 (0.0006) [2023-12-27 11:26:55,905][20381] Updated weights for policy 1, policy_version 838748 (0.0009) [2023-12-27 11:26:55,972][20381] Updated weights for policy 1, policy_version 838758 (0.0010) [2023-12-27 11:26:56,029][20381] Updated weights for policy 1, policy_version 838768 (0.0011) [2023-12-27 11:26:56,457][20347] Updated weights for policy 0, policy_version 839523 (0.0008) [2023-12-27 11:26:56,515][20347] Updated weights for policy 0, policy_version 839533 (0.0010) [2023-12-27 11:26:56,568][20347] Updated weights for policy 0, policy_version 839543 (0.0009) [2023-12-27 11:26:56,641][20381] Updated weights for policy 1, policy_version 838778 (0.0010) [2023-12-27 11:26:56,688][20381] Updated weights for policy 1, policy_version 838788 (0.0009) [2023-12-27 11:26:56,747][20381] Updated weights for policy 1, policy_version 838798 (0.0010) [2023-12-27 11:26:56,800][20381] Updated weights for policy 1, policy_version 838808 (0.0011) [2023-12-27 11:26:57,269][20347] Updated weights for policy 0, policy_version 839553 (0.0007) [2023-12-27 11:26:57,334][20347] Updated weights for policy 0, policy_version 839563 (0.0008) [2023-12-27 11:26:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19410.9). Total num frames: 429711360. Throughput: 0: 9821.5, 1: 9671.4. Samples: 429705620. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:26:57,363][19170] Avg episode reward: [(0, '162.050'), (1, '517.680')] [2023-12-27 11:26:57,392][20347] Updated weights for policy 0, policy_version 839573 (0.0008) [2023-12-27 11:26:57,412][20291] KL-divergence is very high: 117.5429 [2023-12-27 11:26:57,416][20291] KL-divergence is very high: 133.1625 [2023-12-27 11:26:57,427][20291] KL-divergence is very high: 132.4337 [2023-12-27 11:26:57,441][20347] Updated weights for policy 0, policy_version 839583 (0.0008) [2023-12-27 11:26:57,512][20381] Updated weights for policy 1, policy_version 838818 (0.0010) [2023-12-27 11:26:57,571][20381] Updated weights for policy 1, policy_version 838828 (0.0010) [2023-12-27 11:26:57,629][20381] Updated weights for policy 1, policy_version 838838 (0.0010) [2023-12-27 11:26:58,190][20347] Updated weights for policy 0, policy_version 839593 (0.0007) [2023-12-27 11:26:58,254][20347] Updated weights for policy 0, policy_version 839603 (0.0008) [2023-12-27 11:26:58,277][20381] Updated weights for policy 1, policy_version 838848 (0.0011) [2023-12-27 11:26:58,314][20347] Updated weights for policy 0, policy_version 839613 (0.0007) [2023-12-27 11:26:58,346][20381] Updated weights for policy 1, policy_version 838858 (0.0011) [2023-12-27 11:26:58,408][20381] Updated weights for policy 1, policy_version 838868 (0.0009) [2023-12-27 11:26:59,100][20347] Updated weights for policy 0, policy_version 839623 (0.0008) [2023-12-27 11:26:59,102][20291] KL-divergence is very high: 108.4102 [2023-12-27 11:26:59,156][20291] KL-divergence is very high: 102.2478 [2023-12-27 11:26:59,163][20347] Updated weights for policy 0, policy_version 839633 (0.0008) [2023-12-27 11:26:59,226][20347] Updated weights for policy 0, policy_version 839643 (0.0008) [2023-12-27 11:26:59,227][20381] Updated weights for policy 1, policy_version 838878 (0.0009) [2023-12-27 11:26:59,290][20381] Updated weights for policy 1, policy_version 838888 (0.0011) [2023-12-27 11:26:59,356][20381] Updated weights for policy 1, policy_version 838898 (0.0013) [2023-12-27 11:26:59,951][20347] Updated weights for policy 0, policy_version 839653 (0.0009) [2023-12-27 11:27:00,012][20347] Updated weights for policy 0, policy_version 839663 (0.0007) [2023-12-27 11:27:00,062][20381] Updated weights for policy 1, policy_version 838908 (0.0007) [2023-12-27 11:27:00,070][20347] Updated weights for policy 0, policy_version 839673 (0.0008) [2023-12-27 11:27:00,116][20381] Updated weights for policy 1, policy_version 838918 (0.0006) [2023-12-27 11:27:00,176][20381] Updated weights for policy 1, policy_version 838928 (0.0008) [2023-12-27 11:27:00,733][20347] Updated weights for policy 0, policy_version 839683 (0.0006) [2023-12-27 11:27:00,791][20347] Updated weights for policy 0, policy_version 839693 (0.0008) [2023-12-27 11:27:00,842][20381] Updated weights for policy 1, policy_version 838938 (0.0007) [2023-12-27 11:27:00,848][20347] Updated weights for policy 0, policy_version 839703 (0.0007) [2023-12-27 11:27:00,900][20381] Updated weights for policy 1, policy_version 838948 (0.0006) [2023-12-27 11:27:00,966][20381] Updated weights for policy 1, policy_version 838958 (0.0008) [2023-12-27 11:27:01,025][20381] Updated weights for policy 1, policy_version 838968 (0.0008) [2023-12-27 11:27:01,591][20347] Updated weights for policy 0, policy_version 839713 (0.0006) [2023-12-27 11:27:01,657][20347] Updated weights for policy 0, policy_version 839723 (0.0009) [2023-12-27 11:27:01,731][20347] Updated weights for policy 0, policy_version 839733 (0.0008) [2023-12-27 11:27:01,783][20381] Updated weights for policy 1, policy_version 838978 (0.0006) [2023-12-27 11:27:01,795][20347] Updated weights for policy 0, policy_version 839743 (0.0008) [2023-12-27 11:27:01,842][20381] Updated weights for policy 1, policy_version 838988 (0.0007) [2023-12-27 11:27:01,894][20381] Updated weights for policy 1, policy_version 838998 (0.0008) [2023-12-27 11:27:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 429809664. Throughput: 0: 9783.8, 1: 9680.6. Samples: 429821848. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:02,363][19170] Avg episode reward: [(0, '35.620'), (1, '618.100')] [2023-12-27 11:27:02,476][20347] Updated weights for policy 0, policy_version 839753 (0.0006) [2023-12-27 11:27:02,536][20347] Updated weights for policy 0, policy_version 839763 (0.0007) [2023-12-27 11:27:02,593][20347] Updated weights for policy 0, policy_version 839773 (0.0006) [2023-12-27 11:27:02,607][20381] Updated weights for policy 1, policy_version 839008 (0.0008) [2023-12-27 11:27:02,668][20381] Updated weights for policy 1, policy_version 839018 (0.0008) [2023-12-27 11:27:02,726][20381] Updated weights for policy 1, policy_version 839028 (0.0008) [2023-12-27 11:27:03,239][20347] Updated weights for policy 0, policy_version 839783 (0.0007) [2023-12-27 11:27:03,301][20347] Updated weights for policy 0, policy_version 839793 (0.0006) [2023-12-27 11:27:03,362][20347] Updated weights for policy 0, policy_version 839803 (0.0006) [2023-12-27 11:27:03,461][20381] Updated weights for policy 1, policy_version 839038 (0.0008) [2023-12-27 11:27:03,511][20381] Updated weights for policy 1, policy_version 839048 (0.0010) [2023-12-27 11:27:03,569][20381] Updated weights for policy 1, policy_version 839058 (0.0009) [2023-12-27 11:27:03,943][20347] Updated weights for policy 0, policy_version 839813 (0.0006) [2023-12-27 11:27:04,006][20347] Updated weights for policy 0, policy_version 839823 (0.0008) [2023-12-27 11:27:04,071][20347] Updated weights for policy 0, policy_version 839833 (0.0008) [2023-12-27 11:27:04,338][20381] Updated weights for policy 1, policy_version 839068 (0.0008) [2023-12-27 11:27:04,395][20381] Updated weights for policy 1, policy_version 839078 (0.0007) [2023-12-27 11:27:04,456][20381] Updated weights for policy 1, policy_version 839088 (0.0008) [2023-12-27 11:27:04,775][20347] Updated weights for policy 0, policy_version 839843 (0.0008) [2023-12-27 11:27:04,828][20347] Updated weights for policy 0, policy_version 839853 (0.0010) [2023-12-27 11:27:04,885][20347] Updated weights for policy 0, policy_version 839863 (0.0011) [2023-12-27 11:27:05,216][20381] Updated weights for policy 1, policy_version 839098 (0.0008) [2023-12-27 11:27:05,283][20381] Updated weights for policy 1, policy_version 839108 (0.0008) [2023-12-27 11:27:05,343][20381] Updated weights for policy 1, policy_version 839118 (0.0007) [2023-12-27 11:27:05,412][20381] Updated weights for policy 1, policy_version 839128 (0.0007) [2023-12-27 11:27:05,554][20347] Updated weights for policy 0, policy_version 839873 (0.0010) [2023-12-27 11:27:05,607][20347] Updated weights for policy 0, policy_version 839883 (0.0010) [2023-12-27 11:27:05,658][20347] Updated weights for policy 0, policy_version 839893 (0.0008) [2023-12-27 11:27:05,703][20347] Updated weights for policy 0, policy_version 839903 (0.0005) [2023-12-27 11:27:05,990][20381] Updated weights for policy 1, policy_version 839138 (0.0008) [2023-12-27 11:27:06,057][20381] Updated weights for policy 1, policy_version 839148 (0.0008) [2023-12-27 11:27:06,127][20381] Updated weights for policy 1, policy_version 839158 (0.0008) [2023-12-27 11:27:06,391][20347] Updated weights for policy 0, policy_version 839913 (0.0007) [2023-12-27 11:27:06,462][20347] Updated weights for policy 0, policy_version 839923 (0.0008) [2023-12-27 11:27:06,532][20347] Updated weights for policy 0, policy_version 839933 (0.0008) [2023-12-27 11:27:06,797][20381] Updated weights for policy 1, policy_version 839168 (0.0008) [2023-12-27 11:27:06,861][20381] Updated weights for policy 1, policy_version 839178 (0.0007) [2023-12-27 11:27:06,928][20381] Updated weights for policy 1, policy_version 839188 (0.0007) [2023-12-27 11:27:07,214][20347] Updated weights for policy 0, policy_version 839943 (0.0008) [2023-12-27 11:27:07,276][20347] Updated weights for policy 0, policy_version 839953 (0.0008) [2023-12-27 11:27:07,330][20347] Updated weights for policy 0, policy_version 839963 (0.0008) [2023-12-27 11:27:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19524.4, 300 sec: 19438.7). Total num frames: 429916160. Throughput: 0: 9814.9, 1: 9708.2. Samples: 429881528. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:07,363][19170] Avg episode reward: [(0, '33.350'), (1, '673.110')] [2023-12-27 11:27:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000839192_214859776.pth... [2023-12-27 11:27:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000839968_215056384.pth... [2023-12-27 11:27:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000838816_214761472.pth [2023-12-27 11:27:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000838072_214573056.pth [2023-12-27 11:27:07,634][20381] Updated weights for policy 1, policy_version 839198 (0.0008) [2023-12-27 11:27:07,695][20381] Updated weights for policy 1, policy_version 839208 (0.0010) [2023-12-27 11:27:07,753][20381] Updated weights for policy 1, policy_version 839218 (0.0010) [2023-12-27 11:27:07,977][20347] Updated weights for policy 0, policy_version 839973 (0.0006) [2023-12-27 11:27:08,039][20347] Updated weights for policy 0, policy_version 839983 (0.0007) [2023-12-27 11:27:08,098][20347] Updated weights for policy 0, policy_version 839993 (0.0007) [2023-12-27 11:27:08,462][20381] Updated weights for policy 1, policy_version 839228 (0.0007) [2023-12-27 11:27:08,521][20381] Updated weights for policy 1, policy_version 839238 (0.0009) [2023-12-27 11:27:08,590][20381] Updated weights for policy 1, policy_version 839248 (0.0009) [2023-12-27 11:27:08,770][20347] Updated weights for policy 0, policy_version 840003 (0.0007) [2023-12-27 11:27:08,831][20347] Updated weights for policy 0, policy_version 840013 (0.0008) [2023-12-27 11:27:08,886][20347] Updated weights for policy 0, policy_version 840023 (0.0007) [2023-12-27 11:27:09,288][20381] Updated weights for policy 1, policy_version 839258 (0.0008) [2023-12-27 11:27:09,351][20381] Updated weights for policy 1, policy_version 839268 (0.0007) [2023-12-27 11:27:09,422][20381] Updated weights for policy 1, policy_version 839278 (0.0008) [2023-12-27 11:27:09,488][20381] Updated weights for policy 1, policy_version 839288 (0.0008) [2023-12-27 11:27:09,636][20347] Updated weights for policy 0, policy_version 840033 (0.0008) [2023-12-27 11:27:09,696][20347] Updated weights for policy 0, policy_version 840043 (0.0008) [2023-12-27 11:27:09,757][20347] Updated weights for policy 0, policy_version 840053 (0.0008) [2023-12-27 11:27:09,828][20347] Updated weights for policy 0, policy_version 840063 (0.0008) [2023-12-27 11:27:10,276][20381] Updated weights for policy 1, policy_version 839298 (0.0008) [2023-12-27 11:27:10,337][20381] Updated weights for policy 1, policy_version 839308 (0.0008) [2023-12-27 11:27:10,395][20381] Updated weights for policy 1, policy_version 839318 (0.0008) [2023-12-27 11:27:10,661][20347] Updated weights for policy 0, policy_version 840073 (0.0008) [2023-12-27 11:27:10,721][20347] Updated weights for policy 0, policy_version 840083 (0.0011) [2023-12-27 11:27:10,774][20347] Updated weights for policy 0, policy_version 840093 (0.0011) [2023-12-27 11:27:11,169][20381] Updated weights for policy 1, policy_version 839328 (0.0008) [2023-12-27 11:27:11,240][20381] Updated weights for policy 1, policy_version 839338 (0.0008) [2023-12-27 11:27:11,304][20381] Updated weights for policy 1, policy_version 839348 (0.0006) [2023-12-27 11:27:11,582][20347] Updated weights for policy 0, policy_version 840103 (0.0009) [2023-12-27 11:27:11,654][20347] Updated weights for policy 0, policy_version 840113 (0.0008) [2023-12-27 11:27:11,720][20347] Updated weights for policy 0, policy_version 840123 (0.0008) [2023-12-27 11:27:12,081][20381] Updated weights for policy 1, policy_version 839358 (0.0007) [2023-12-27 11:27:12,146][20381] Updated weights for policy 1, policy_version 839368 (0.0008) [2023-12-27 11:27:12,209][20381] Updated weights for policy 1, policy_version 839378 (0.0009) [2023-12-27 11:27:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 430006272. Throughput: 0: 9874.9, 1: 9737.1. Samples: 429997500. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:12,363][19170] Avg episode reward: [(0, '49.030'), (1, '649.560')] [2023-12-27 11:27:12,460][20347] Updated weights for policy 0, policy_version 840133 (0.0008) [2023-12-27 11:27:12,514][20347] Updated weights for policy 0, policy_version 840143 (0.0007) [2023-12-27 11:27:12,572][20347] Updated weights for policy 0, policy_version 840153 (0.0009) [2023-12-27 11:27:12,920][20381] Updated weights for policy 1, policy_version 839388 (0.0008) [2023-12-27 11:27:12,979][20381] Updated weights for policy 1, policy_version 839398 (0.0007) [2023-12-27 11:27:13,029][20381] Updated weights for policy 1, policy_version 839408 (0.0008) [2023-12-27 11:27:13,273][20347] Updated weights for policy 0, policy_version 840163 (0.0007) [2023-12-27 11:27:13,324][20347] Updated weights for policy 0, policy_version 840173 (0.0006) [2023-12-27 11:27:13,376][20347] Updated weights for policy 0, policy_version 840183 (0.0007) [2023-12-27 11:27:13,714][20381] Updated weights for policy 1, policy_version 839418 (0.0008) [2023-12-27 11:27:13,772][20381] Updated weights for policy 1, policy_version 839428 (0.0005) [2023-12-27 11:27:13,842][20381] Updated weights for policy 1, policy_version 839438 (0.0007) [2023-12-27 11:27:13,894][20381] Updated weights for policy 1, policy_version 839448 (0.0008) [2023-12-27 11:27:14,061][20347] Updated weights for policy 0, policy_version 840193 (0.0007) [2023-12-27 11:27:14,118][20347] Updated weights for policy 0, policy_version 840203 (0.0006) [2023-12-27 11:27:14,171][20347] Updated weights for policy 0, policy_version 840213 (0.0006) [2023-12-27 11:27:14,234][20347] Updated weights for policy 0, policy_version 840223 (0.0008) [2023-12-27 11:27:14,545][20381] Updated weights for policy 1, policy_version 839458 (0.0007) [2023-12-27 11:27:14,606][20381] Updated weights for policy 1, policy_version 839468 (0.0007) [2023-12-27 11:27:14,663][20381] Updated weights for policy 1, policy_version 839478 (0.0008) [2023-12-27 11:27:14,933][20347] Updated weights for policy 0, policy_version 840233 (0.0007) [2023-12-27 11:27:15,003][20347] Updated weights for policy 0, policy_version 840243 (0.0007) [2023-12-27 11:27:15,063][20347] Updated weights for policy 0, policy_version 840253 (0.0008) [2023-12-27 11:27:15,463][20381] Updated weights for policy 1, policy_version 839488 (0.0008) [2023-12-27 11:27:15,511][20381] Updated weights for policy 1, policy_version 839498 (0.0008) [2023-12-27 11:27:15,575][20381] Updated weights for policy 1, policy_version 839508 (0.0006) [2023-12-27 11:27:15,778][20347] Updated weights for policy 0, policy_version 840263 (0.0007) [2023-12-27 11:27:15,848][20347] Updated weights for policy 0, policy_version 840273 (0.0009) [2023-12-27 11:27:15,916][20347] Updated weights for policy 0, policy_version 840283 (0.0007) [2023-12-27 11:27:16,285][20381] Updated weights for policy 1, policy_version 839518 (0.0008) [2023-12-27 11:27:16,339][20381] Updated weights for policy 1, policy_version 839528 (0.0008) [2023-12-27 11:27:16,391][20381] Updated weights for policy 1, policy_version 839538 (0.0008) [2023-12-27 11:27:16,547][20347] Updated weights for policy 0, policy_version 840293 (0.0008) [2023-12-27 11:27:16,603][20347] Updated weights for policy 0, policy_version 840303 (0.0008) [2023-12-27 11:27:16,659][20347] Updated weights for policy 0, policy_version 840313 (0.0008) [2023-12-27 11:27:17,108][20381] Updated weights for policy 1, policy_version 839548 (0.0009) [2023-12-27 11:27:17,162][20381] Updated weights for policy 1, policy_version 839558 (0.0010) [2023-12-27 11:27:17,222][20381] Updated weights for policy 1, policy_version 839568 (0.0010) [2023-12-27 11:27:17,342][20347] Updated weights for policy 0, policy_version 840323 (0.0007) [2023-12-27 11:27:17,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 430104576. Throughput: 0: 9861.7, 1: 9709.6. Samples: 430113460. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:17,363][19170] Avg episode reward: [(0, '45.980'), (1, '688.760')] [2023-12-27 11:27:17,401][20347] Updated weights for policy 0, policy_version 840333 (0.0009) [2023-12-27 11:27:17,460][20347] Updated weights for policy 0, policy_version 840343 (0.0010) [2023-12-27 11:27:17,844][20381] Updated weights for policy 1, policy_version 839578 (0.0010) [2023-12-27 11:27:17,904][20381] Updated weights for policy 1, policy_version 839588 (0.0006) [2023-12-27 11:27:17,964][20381] Updated weights for policy 1, policy_version 839598 (0.0005) [2023-12-27 11:27:18,018][20381] Updated weights for policy 1, policy_version 839608 (0.0006) [2023-12-27 11:27:18,160][20347] Updated weights for policy 0, policy_version 840353 (0.0011) [2023-12-27 11:27:18,222][20347] Updated weights for policy 0, policy_version 840363 (0.0010) [2023-12-27 11:27:18,274][20347] Updated weights for policy 0, policy_version 840373 (0.0010) [2023-12-27 11:27:18,322][20347] Updated weights for policy 0, policy_version 840383 (0.0010) [2023-12-27 11:27:18,679][20381] Updated weights for policy 1, policy_version 839618 (0.0011) [2023-12-27 11:27:18,743][20381] Updated weights for policy 1, policy_version 839628 (0.0011) [2023-12-27 11:27:18,801][20381] Updated weights for policy 1, policy_version 839638 (0.0011) [2023-12-27 11:27:19,018][20347] Updated weights for policy 0, policy_version 840393 (0.0009) [2023-12-27 11:27:19,070][20347] Updated weights for policy 0, policy_version 840403 (0.0006) [2023-12-27 11:27:19,120][20347] Updated weights for policy 0, policy_version 840413 (0.0006) [2023-12-27 11:27:19,535][20381] Updated weights for policy 1, policy_version 839648 (0.0008) [2023-12-27 11:27:19,596][20381] Updated weights for policy 1, policy_version 839658 (0.0007) [2023-12-27 11:27:19,661][20381] Updated weights for policy 1, policy_version 839668 (0.0009) [2023-12-27 11:27:19,864][20347] Updated weights for policy 0, policy_version 840423 (0.0008) [2023-12-27 11:27:19,925][20347] Updated weights for policy 0, policy_version 840433 (0.0008) [2023-12-27 11:27:19,985][20347] Updated weights for policy 0, policy_version 840443 (0.0008) [2023-12-27 11:27:20,454][20381] Updated weights for policy 1, policy_version 839678 (0.0007) [2023-12-27 11:27:20,520][20381] Updated weights for policy 1, policy_version 839688 (0.0009) [2023-12-27 11:27:20,587][20381] Updated weights for policy 1, policy_version 839698 (0.0008) [2023-12-27 11:27:20,703][20347] Updated weights for policy 0, policy_version 840453 (0.0008) [2023-12-27 11:27:20,766][20347] Updated weights for policy 0, policy_version 840463 (0.0008) [2023-12-27 11:27:20,827][20347] Updated weights for policy 0, policy_version 840473 (0.0008) [2023-12-27 11:27:21,336][20381] Updated weights for policy 1, policy_version 839708 (0.0008) [2023-12-27 11:27:21,410][20381] Updated weights for policy 1, policy_version 839718 (0.0008) [2023-12-27 11:27:21,475][20381] Updated weights for policy 1, policy_version 839728 (0.0006) [2023-12-27 11:27:21,586][20347] Updated weights for policy 0, policy_version 840483 (0.0008) [2023-12-27 11:27:21,656][20347] Updated weights for policy 0, policy_version 840493 (0.0008) [2023-12-27 11:27:21,720][20347] Updated weights for policy 0, policy_version 840503 (0.0008) [2023-12-27 11:27:22,229][20381] Updated weights for policy 1, policy_version 839738 (0.0007) [2023-12-27 11:27:22,296][20381] Updated weights for policy 1, policy_version 839748 (0.0009) [2023-12-27 11:27:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 430194688. Throughput: 0: 9860.4, 1: 9706.2. Samples: 430172636. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:22,363][19170] Avg episode reward: [(0, '49.520'), (1, '746.720')] [2023-12-27 11:27:22,366][20381] Updated weights for policy 1, policy_version 839758 (0.0008) [2023-12-27 11:27:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000840512_215195648.pth... [2023-12-27 11:27:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000839360_214900736.pth [2023-12-27 11:27:22,429][20381] Updated weights for policy 1, policy_version 839768 (0.0008) [2023-12-27 11:27:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000839768_215007232.pth... [2023-12-27 11:27:22,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000838616_214712320.pth [2023-12-27 11:27:22,471][20347] Updated weights for policy 0, policy_version 840513 (0.0007) [2023-12-27 11:27:22,533][20347] Updated weights for policy 0, policy_version 840523 (0.0007) [2023-12-27 11:27:22,595][20347] Updated weights for policy 0, policy_version 840533 (0.0007) [2023-12-27 11:27:22,664][20347] Updated weights for policy 0, policy_version 840543 (0.0007) [2023-12-27 11:27:23,155][20381] Updated weights for policy 1, policy_version 839778 (0.0008) [2023-12-27 11:27:23,209][20381] Updated weights for policy 1, policy_version 839788 (0.0009) [2023-12-27 11:27:23,259][20381] Updated weights for policy 1, policy_version 839798 (0.0009) [2023-12-27 11:27:23,294][20347] Updated weights for policy 0, policy_version 840553 (0.0007) [2023-12-27 11:27:23,344][20347] Updated weights for policy 0, policy_version 840563 (0.0008) [2023-12-27 11:27:23,397][20347] Updated weights for policy 0, policy_version 840573 (0.0008) [2023-12-27 11:27:24,007][20381] Updated weights for policy 1, policy_version 839808 (0.0007) [2023-12-27 11:27:24,071][20381] Updated weights for policy 1, policy_version 839818 (0.0008) [2023-12-27 11:27:24,104][20347] Updated weights for policy 0, policy_version 840583 (0.0007) [2023-12-27 11:27:24,132][20381] Updated weights for policy 1, policy_version 839828 (0.0006) [2023-12-27 11:27:24,165][20347] Updated weights for policy 0, policy_version 840593 (0.0006) [2023-12-27 11:27:24,225][20347] Updated weights for policy 0, policy_version 840603 (0.0008) [2023-12-27 11:27:24,837][20381] Updated weights for policy 1, policy_version 839838 (0.0006) [2023-12-27 11:27:24,896][20381] Updated weights for policy 1, policy_version 839848 (0.0006) [2023-12-27 11:27:24,937][20347] Updated weights for policy 0, policy_version 840613 (0.0006) [2023-12-27 11:27:24,960][20381] Updated weights for policy 1, policy_version 839858 (0.0008) [2023-12-27 11:27:24,994][20347] Updated weights for policy 0, policy_version 840623 (0.0008) [2023-12-27 11:27:25,057][20347] Updated weights for policy 0, policy_version 840633 (0.0008) [2023-12-27 11:27:25,633][20381] Updated weights for policy 1, policy_version 839868 (0.0008) [2023-12-27 11:27:25,689][20381] Updated weights for policy 1, policy_version 839878 (0.0008) [2023-12-27 11:27:25,742][20381] Updated weights for policy 1, policy_version 839888 (0.0007) [2023-12-27 11:27:25,766][20347] Updated weights for policy 0, policy_version 840643 (0.0006) [2023-12-27 11:27:25,828][20347] Updated weights for policy 0, policy_version 840653 (0.0006) [2023-12-27 11:27:25,887][20347] Updated weights for policy 0, policy_version 840663 (0.0008) [2023-12-27 11:27:26,452][20381] Updated weights for policy 1, policy_version 839898 (0.0007) [2023-12-27 11:27:26,500][20381] Updated weights for policy 1, policy_version 839908 (0.0005) [2023-12-27 11:27:26,552][20381] Updated weights for policy 1, policy_version 839918 (0.0008) [2023-12-27 11:27:26,569][20347] Updated weights for policy 0, policy_version 840673 (0.0008) [2023-12-27 11:27:26,613][20381] Updated weights for policy 1, policy_version 839928 (0.0008) [2023-12-27 11:27:26,624][20347] Updated weights for policy 0, policy_version 840683 (0.0007) [2023-12-27 11:27:26,673][20347] Updated weights for policy 0, policy_version 840693 (0.0005) [2023-12-27 11:27:26,726][20347] Updated weights for policy 0, policy_version 840703 (0.0005) [2023-12-27 11:27:27,236][20381] Updated weights for policy 1, policy_version 839938 (0.0008) [2023-12-27 11:27:27,300][20381] Updated weights for policy 1, policy_version 839948 (0.0006) [2023-12-27 11:27:27,358][20381] Updated weights for policy 1, policy_version 839958 (0.0006) [2023-12-27 11:27:27,359][20347] Updated weights for policy 0, policy_version 840713 (0.0005) [2023-12-27 11:27:27,362][19170] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 430292992. Throughput: 0: 9794.9, 1: 9686.3. Samples: 430287740. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:27,363][19170] Avg episode reward: [(0, '37.430'), (1, '688.260')] [2023-12-27 11:27:27,410][20347] Updated weights for policy 0, policy_version 840723 (0.0007) [2023-12-27 11:27:27,469][20347] Updated weights for policy 0, policy_version 840733 (0.0007) [2023-12-27 11:27:28,061][20381] Updated weights for policy 1, policy_version 839968 (0.0006) [2023-12-27 11:27:28,082][20347] Updated weights for policy 0, policy_version 840743 (0.0008) [2023-12-27 11:27:28,120][20381] Updated weights for policy 1, policy_version 839978 (0.0007) [2023-12-27 11:27:28,145][20347] Updated weights for policy 0, policy_version 840753 (0.0006) [2023-12-27 11:27:28,175][20381] Updated weights for policy 1, policy_version 839988 (0.0007) [2023-12-27 11:27:28,203][20347] Updated weights for policy 0, policy_version 840763 (0.0005) [2023-12-27 11:27:28,834][20381] Updated weights for policy 1, policy_version 839998 (0.0008) [2023-12-27 11:27:28,878][20347] Updated weights for policy 0, policy_version 840773 (0.0007) [2023-12-27 11:27:28,892][20381] Updated weights for policy 1, policy_version 840008 (0.0008) [2023-12-27 11:27:28,937][20347] Updated weights for policy 0, policy_version 840783 (0.0008) [2023-12-27 11:27:28,943][20381] Updated weights for policy 1, policy_version 840018 (0.0008) [2023-12-27 11:27:28,988][20347] Updated weights for policy 0, policy_version 840793 (0.0008) [2023-12-27 11:27:29,665][20381] Updated weights for policy 1, policy_version 840028 (0.0006) [2023-12-27 11:27:29,726][20347] Updated weights for policy 0, policy_version 840803 (0.0007) [2023-12-27 11:27:29,726][20381] Updated weights for policy 1, policy_version 840038 (0.0006) [2023-12-27 11:27:29,786][20381] Updated weights for policy 1, policy_version 840048 (0.0009) [2023-12-27 11:27:29,806][20347] Updated weights for policy 0, policy_version 840813 (0.0008) [2023-12-27 11:27:29,869][20347] Updated weights for policy 0, policy_version 840823 (0.0008) [2023-12-27 11:27:30,468][20381] Updated weights for policy 1, policy_version 840058 (0.0007) [2023-12-27 11:27:30,523][20381] Updated weights for policy 1, policy_version 840068 (0.0007) [2023-12-27 11:27:30,585][20381] Updated weights for policy 1, policy_version 840078 (0.0006) [2023-12-27 11:27:30,636][20347] Updated weights for policy 0, policy_version 840833 (0.0008) [2023-12-27 11:27:30,650][20381] Updated weights for policy 1, policy_version 840088 (0.0007) [2023-12-27 11:27:30,693][20347] Updated weights for policy 0, policy_version 840843 (0.0007) [2023-12-27 11:27:30,739][20347] Updated weights for policy 0, policy_version 840853 (0.0007) [2023-12-27 11:27:30,796][20347] Updated weights for policy 0, policy_version 840863 (0.0008) [2023-12-27 11:27:31,384][20381] Updated weights for policy 1, policy_version 840098 (0.0008) [2023-12-27 11:27:31,440][20381] Updated weights for policy 1, policy_version 840108 (0.0007) [2023-12-27 11:27:31,495][20381] Updated weights for policy 1, policy_version 840118 (0.0007) [2023-12-27 11:27:31,518][20347] Updated weights for policy 0, policy_version 840873 (0.0008) [2023-12-27 11:27:31,574][20347] Updated weights for policy 0, policy_version 840883 (0.0008) [2023-12-27 11:27:31,632][20347] Updated weights for policy 0, policy_version 840893 (0.0008) [2023-12-27 11:27:32,192][20381] Updated weights for policy 1, policy_version 840128 (0.0007) [2023-12-27 11:27:32,257][20381] Updated weights for policy 1, policy_version 840138 (0.0007) [2023-12-27 11:27:32,315][20381] Updated weights for policy 1, policy_version 840148 (0.0006) [2023-12-27 11:27:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 430399488. Throughput: 0: 9797.7, 1: 9722.8. Samples: 430407780. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:32,363][19170] Avg episode reward: [(0, '42.870'), (1, '727.530')] [2023-12-27 11:27:32,445][20347] Updated weights for policy 0, policy_version 840903 (0.0007) [2023-12-27 11:27:32,506][20347] Updated weights for policy 0, policy_version 840913 (0.0006) [2023-12-27 11:27:32,572][20347] Updated weights for policy 0, policy_version 840923 (0.0006) [2023-12-27 11:27:33,027][20381] Updated weights for policy 1, policy_version 840158 (0.0008) [2023-12-27 11:27:33,091][20381] Updated weights for policy 1, policy_version 840168 (0.0007) [2023-12-27 11:27:33,158][20381] Updated weights for policy 1, policy_version 840178 (0.0007) [2023-12-27 11:27:33,209][20347] Updated weights for policy 0, policy_version 840933 (0.0006) [2023-12-27 11:27:33,269][20347] Updated weights for policy 0, policy_version 840943 (0.0008) [2023-12-27 11:27:33,331][20347] Updated weights for policy 0, policy_version 840953 (0.0008) [2023-12-27 11:27:33,821][20381] Updated weights for policy 1, policy_version 840188 (0.0007) [2023-12-27 11:27:33,872][20381] Updated weights for policy 1, policy_version 840198 (0.0008) [2023-12-27 11:27:33,926][20381] Updated weights for policy 1, policy_version 840208 (0.0007) [2023-12-27 11:27:34,000][20347] Updated weights for policy 0, policy_version 840963 (0.0008) [2023-12-27 11:27:34,063][20347] Updated weights for policy 0, policy_version 840973 (0.0009) [2023-12-27 11:27:34,117][20347] Updated weights for policy 0, policy_version 840983 (0.0008) [2023-12-27 11:27:34,561][20381] Updated weights for policy 1, policy_version 840218 (0.0007) [2023-12-27 11:27:34,622][20381] Updated weights for policy 1, policy_version 840228 (0.0006) [2023-12-27 11:27:34,691][20381] Updated weights for policy 1, policy_version 840238 (0.0007) [2023-12-27 11:27:34,759][20381] Updated weights for policy 1, policy_version 840248 (0.0006) [2023-12-27 11:27:34,912][20347] Updated weights for policy 0, policy_version 840993 (0.0009) [2023-12-27 11:27:34,974][20347] Updated weights for policy 0, policy_version 841003 (0.0008) [2023-12-27 11:27:35,032][20347] Updated weights for policy 0, policy_version 841013 (0.0007) [2023-12-27 11:27:35,090][20347] Updated weights for policy 0, policy_version 841023 (0.0008) [2023-12-27 11:27:35,356][20381] Updated weights for policy 1, policy_version 840258 (0.0008) [2023-12-27 11:27:35,420][20381] Updated weights for policy 1, policy_version 840268 (0.0008) [2023-12-27 11:27:35,481][20381] Updated weights for policy 1, policy_version 840278 (0.0008) [2023-12-27 11:27:35,775][20347] Updated weights for policy 0, policy_version 841033 (0.0006) [2023-12-27 11:27:35,828][20347] Updated weights for policy 0, policy_version 841043 (0.0007) [2023-12-27 11:27:35,881][20347] Updated weights for policy 0, policy_version 841053 (0.0008) [2023-12-27 11:27:36,156][20381] Updated weights for policy 1, policy_version 840288 (0.0008) [2023-12-27 11:27:36,224][20381] Updated weights for policy 1, policy_version 840298 (0.0007) [2023-12-27 11:27:36,288][20381] Updated weights for policy 1, policy_version 840308 (0.0006) [2023-12-27 11:27:36,656][20347] Updated weights for policy 0, policy_version 841063 (0.0007) [2023-12-27 11:27:36,714][20347] Updated weights for policy 0, policy_version 841073 (0.0007) [2023-12-27 11:27:36,759][20347] Updated weights for policy 0, policy_version 841083 (0.0008) [2023-12-27 11:27:36,924][20381] Updated weights for policy 1, policy_version 840318 (0.0007) [2023-12-27 11:27:36,978][20381] Updated weights for policy 1, policy_version 840328 (0.0006) [2023-12-27 11:27:37,034][20381] Updated weights for policy 1, policy_version 840338 (0.0006) [2023-12-27 11:27:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 430497792. Throughput: 0: 9789.2, 1: 9763.4. Samples: 430468052. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:37,363][19170] Avg episode reward: [(0, '137.960'), (1, '718.300')] [2023-12-27 11:27:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000840344_215154688.pth... [2023-12-27 11:27:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000841088_215343104.pth... [2023-12-27 11:27:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000839192_214859776.pth [2023-12-27 11:27:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000839968_215056384.pth [2023-12-27 11:27:37,423][20347] Updated weights for policy 0, policy_version 841093 (0.0008) [2023-12-27 11:27:37,491][20347] Updated weights for policy 0, policy_version 841103 (0.0008) [2023-12-27 11:27:37,549][20347] Updated weights for policy 0, policy_version 841113 (0.0008) [2023-12-27 11:27:37,715][20381] Updated weights for policy 1, policy_version 840348 (0.0008) [2023-12-27 11:27:37,779][20381] Updated weights for policy 1, policy_version 840358 (0.0009) [2023-12-27 11:27:37,839][20381] Updated weights for policy 1, policy_version 840368 (0.0007) [2023-12-27 11:27:38,239][20347] Updated weights for policy 0, policy_version 841123 (0.0008) [2023-12-27 11:27:38,296][20347] Updated weights for policy 0, policy_version 841133 (0.0008) [2023-12-27 11:27:38,357][20347] Updated weights for policy 0, policy_version 841143 (0.0007) [2023-12-27 11:27:38,589][20381] Updated weights for policy 1, policy_version 840378 (0.0009) [2023-12-27 11:27:38,651][20381] Updated weights for policy 1, policy_version 840388 (0.0008) [2023-12-27 11:27:38,707][20381] Updated weights for policy 1, policy_version 840398 (0.0008) [2023-12-27 11:27:38,770][20381] Updated weights for policy 1, policy_version 840408 (0.0007) [2023-12-27 11:27:39,098][20347] Updated weights for policy 0, policy_version 841153 (0.0008) [2023-12-27 11:27:39,146][20347] Updated weights for policy 0, policy_version 841163 (0.0007) [2023-12-27 11:27:39,198][20347] Updated weights for policy 0, policy_version 841173 (0.0008) [2023-12-27 11:27:39,264][20347] Updated weights for policy 0, policy_version 841183 (0.0008) [2023-12-27 11:27:39,470][20381] Updated weights for policy 1, policy_version 840418 (0.0008) [2023-12-27 11:27:39,536][20381] Updated weights for policy 1, policy_version 840428 (0.0007) [2023-12-27 11:27:39,604][20381] Updated weights for policy 1, policy_version 840438 (0.0008) [2023-12-27 11:27:40,043][20347] Updated weights for policy 0, policy_version 841193 (0.0008) [2023-12-27 11:27:40,105][20347] Updated weights for policy 0, policy_version 841203 (0.0008) [2023-12-27 11:27:40,159][20347] Updated weights for policy 0, policy_version 841213 (0.0008) [2023-12-27 11:27:40,324][20381] Updated weights for policy 1, policy_version 840448 (0.0008) [2023-12-27 11:27:40,386][20381] Updated weights for policy 1, policy_version 840458 (0.0008) [2023-12-27 11:27:40,447][20381] Updated weights for policy 1, policy_version 840468 (0.0008) [2023-12-27 11:27:40,874][20347] Updated weights for policy 0, policy_version 841223 (0.0007) [2023-12-27 11:27:40,929][20347] Updated weights for policy 0, policy_version 841233 (0.0007) [2023-12-27 11:27:40,990][20347] Updated weights for policy 0, policy_version 841243 (0.0006) [2023-12-27 11:27:41,170][20381] Updated weights for policy 1, policy_version 840478 (0.0008) [2023-12-27 11:27:41,230][20381] Updated weights for policy 1, policy_version 840488 (0.0008) [2023-12-27 11:27:41,295][20381] Updated weights for policy 1, policy_version 840498 (0.0006) [2023-12-27 11:27:41,755][20347] Updated weights for policy 0, policy_version 841253 (0.0008) [2023-12-27 11:27:41,814][20347] Updated weights for policy 0, policy_version 841263 (0.0008) [2023-12-27 11:27:41,873][20347] Updated weights for policy 0, policy_version 841273 (0.0008) [2023-12-27 11:27:42,059][20381] Updated weights for policy 1, policy_version 840508 (0.0008) [2023-12-27 11:27:42,123][20381] Updated weights for policy 1, policy_version 840518 (0.0008) [2023-12-27 11:27:42,177][20381] Updated weights for policy 1, policy_version 840528 (0.0009) [2023-12-27 11:27:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 430596096. Throughput: 0: 9773.0, 1: 9779.3. Samples: 430585472. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:42,363][19170] Avg episode reward: [(0, '179.890'), (1, '776.560')] [2023-12-27 11:27:42,573][20347] Updated weights for policy 0, policy_version 841283 (0.0008) [2023-12-27 11:27:42,636][20347] Updated weights for policy 0, policy_version 841293 (0.0010) [2023-12-27 11:27:42,699][20347] Updated weights for policy 0, policy_version 841303 (0.0006) [2023-12-27 11:27:42,880][20381] Updated weights for policy 1, policy_version 840538 (0.0007) [2023-12-27 11:27:42,935][20381] Updated weights for policy 1, policy_version 840548 (0.0007) [2023-12-27 11:27:42,992][20381] Updated weights for policy 1, policy_version 840558 (0.0006) [2023-12-27 11:27:43,055][20381] Updated weights for policy 1, policy_version 840568 (0.0006) [2023-12-27 11:27:43,384][20347] Updated weights for policy 0, policy_version 841313 (0.0010) [2023-12-27 11:27:43,446][20347] Updated weights for policy 0, policy_version 841323 (0.0010) [2023-12-27 11:27:43,508][20347] Updated weights for policy 0, policy_version 841333 (0.0010) [2023-12-27 11:27:43,568][20347] Updated weights for policy 0, policy_version 841343 (0.0006) [2023-12-27 11:27:43,678][20381] Updated weights for policy 1, policy_version 840578 (0.0008) [2023-12-27 11:27:43,742][20381] Updated weights for policy 1, policy_version 840588 (0.0007) [2023-12-27 11:27:43,808][20381] Updated weights for policy 1, policy_version 840598 (0.0007) [2023-12-27 11:27:44,231][20347] Updated weights for policy 0, policy_version 841353 (0.0008) [2023-12-27 11:27:44,283][20347] Updated weights for policy 0, policy_version 841363 (0.0006) [2023-12-27 11:27:44,336][20347] Updated weights for policy 0, policy_version 841373 (0.0005) [2023-12-27 11:27:44,494][20381] Updated weights for policy 1, policy_version 840608 (0.0008) [2023-12-27 11:27:44,555][20381] Updated weights for policy 1, policy_version 840618 (0.0007) [2023-12-27 11:27:44,619][20381] Updated weights for policy 1, policy_version 840628 (0.0006) [2023-12-27 11:27:45,064][20347] Updated weights for policy 0, policy_version 841383 (0.0009) [2023-12-27 11:27:45,128][20347] Updated weights for policy 0, policy_version 841393 (0.0011) [2023-12-27 11:27:45,189][20347] Updated weights for policy 0, policy_version 841403 (0.0010) [2023-12-27 11:27:45,298][20381] Updated weights for policy 1, policy_version 840638 (0.0007) [2023-12-27 11:27:45,362][20381] Updated weights for policy 1, policy_version 840648 (0.0006) [2023-12-27 11:27:45,419][20381] Updated weights for policy 1, policy_version 840658 (0.0008) [2023-12-27 11:27:45,904][20347] Updated weights for policy 0, policy_version 841413 (0.0010) [2023-12-27 11:27:45,965][20347] Updated weights for policy 0, policy_version 841423 (0.0010) [2023-12-27 11:27:46,017][20347] Updated weights for policy 0, policy_version 841433 (0.0011) [2023-12-27 11:27:46,025][20381] Updated weights for policy 1, policy_version 840668 (0.0006) [2023-12-27 11:27:46,088][20381] Updated weights for policy 1, policy_version 840678 (0.0006) [2023-12-27 11:27:46,144][20381] Updated weights for policy 1, policy_version 840688 (0.0008) [2023-12-27 11:27:46,713][20347] Updated weights for policy 0, policy_version 841443 (0.0009) [2023-12-27 11:27:46,765][20347] Updated weights for policy 0, policy_version 841453 (0.0007) [2023-12-27 11:27:46,785][20381] Updated weights for policy 1, policy_version 840698 (0.0008) [2023-12-27 11:27:46,821][20347] Updated weights for policy 0, policy_version 841463 (0.0006) [2023-12-27 11:27:46,846][20381] Updated weights for policy 1, policy_version 840708 (0.0006) [2023-12-27 11:27:46,906][20381] Updated weights for policy 1, policy_version 840718 (0.0007) [2023-12-27 11:27:46,967][20381] Updated weights for policy 1, policy_version 840728 (0.0008) [2023-12-27 11:27:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 19660.7, 300 sec: 19466.4). Total num frames: 430694400. Throughput: 0: 9803.0, 1: 9816.2. Samples: 430704716. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:47,364][19170] Avg episode reward: [(0, '349.690'), (1, '863.600')] [2023-12-27 11:27:47,535][20347] Updated weights for policy 0, policy_version 841473 (0.0006) [2023-12-27 11:27:47,600][20347] Updated weights for policy 0, policy_version 841483 (0.0008) [2023-12-27 11:27:47,656][20347] Updated weights for policy 0, policy_version 841493 (0.0006) [2023-12-27 11:27:47,681][20381] Updated weights for policy 1, policy_version 840738 (0.0006) [2023-12-27 11:27:47,714][20347] Updated weights for policy 0, policy_version 841503 (0.0006) [2023-12-27 11:27:47,737][20381] Updated weights for policy 1, policy_version 840748 (0.0006) [2023-12-27 11:27:47,790][20381] Updated weights for policy 1, policy_version 840758 (0.0008) [2023-12-27 11:27:48,469][20347] Updated weights for policy 0, policy_version 841513 (0.0008) [2023-12-27 11:27:48,474][20381] Updated weights for policy 1, policy_version 840768 (0.0008) [2023-12-27 11:27:48,529][20347] Updated weights for policy 0, policy_version 841523 (0.0008) [2023-12-27 11:27:48,535][20381] Updated weights for policy 1, policy_version 840778 (0.0006) [2023-12-27 11:27:48,585][20347] Updated weights for policy 0, policy_version 841533 (0.0008) [2023-12-27 11:27:48,589][20381] Updated weights for policy 1, policy_version 840788 (0.0007) [2023-12-27 11:27:49,226][20381] Updated weights for policy 1, policy_version 840798 (0.0007) [2023-12-27 11:27:49,289][20381] Updated weights for policy 1, policy_version 840808 (0.0012) [2023-12-27 11:27:49,355][20381] Updated weights for policy 1, policy_version 840818 (0.0008) [2023-12-27 11:27:49,382][20347] Updated weights for policy 0, policy_version 841543 (0.0008) [2023-12-27 11:27:49,450][20347] Updated weights for policy 0, policy_version 841553 (0.0007) [2023-12-27 11:27:49,517][20347] Updated weights for policy 0, policy_version 841563 (0.0007) [2023-12-27 11:27:50,128][20381] Updated weights for policy 1, policy_version 840828 (0.0008) [2023-12-27 11:27:50,186][20381] Updated weights for policy 1, policy_version 840838 (0.0006) [2023-12-27 11:27:50,232][20347] Updated weights for policy 0, policy_version 841573 (0.0008) [2023-12-27 11:27:50,244][20381] Updated weights for policy 1, policy_version 840848 (0.0008) [2023-12-27 11:27:50,288][20347] Updated weights for policy 0, policy_version 841583 (0.0007) [2023-12-27 11:27:50,343][20347] Updated weights for policy 0, policy_version 841593 (0.0008) [2023-12-27 11:27:51,004][20381] Updated weights for policy 1, policy_version 840858 (0.0008) [2023-12-27 11:27:51,074][20381] Updated weights for policy 1, policy_version 840868 (0.0008) [2023-12-27 11:27:51,105][20347] Updated weights for policy 0, policy_version 841603 (0.0008) [2023-12-27 11:27:51,141][20381] Updated weights for policy 1, policy_version 840878 (0.0008) [2023-12-27 11:27:51,173][20347] Updated weights for policy 0, policy_version 841613 (0.0008) [2023-12-27 11:27:51,201][20381] Updated weights for policy 1, policy_version 840888 (0.0006) [2023-12-27 11:27:51,239][20347] Updated weights for policy 0, policy_version 841623 (0.0007) [2023-12-27 11:27:51,954][20381] Updated weights for policy 1, policy_version 840898 (0.0008) [2023-12-27 11:27:52,010][20381] Updated weights for policy 1, policy_version 840908 (0.0008) [2023-12-27 11:27:52,025][20347] Updated weights for policy 0, policy_version 841633 (0.0007) [2023-12-27 11:27:52,066][20381] Updated weights for policy 1, policy_version 840918 (0.0008) [2023-12-27 11:27:52,086][20347] Updated weights for policy 0, policy_version 841643 (0.0008) [2023-12-27 11:27:52,151][20347] Updated weights for policy 0, policy_version 841653 (0.0009) [2023-12-27 11:27:52,206][20347] Updated weights for policy 0, policy_version 841663 (0.0008) [2023-12-27 11:27:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19660.9, 300 sec: 19466.4). Total num frames: 430792704. Throughput: 0: 9730.3, 1: 9838.5. Samples: 430762124. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:52,363][19170] Avg episode reward: [(0, '57.250'), (1, '815.090')] [2023-12-27 11:27:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000840920_215302144.pth... [2023-12-27 11:27:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000841664_215490560.pth... [2023-12-27 11:27:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000839768_215007232.pth [2023-12-27 11:27:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000840512_215195648.pth [2023-12-27 11:27:52,827][20381] Updated weights for policy 1, policy_version 840928 (0.0007) [2023-12-27 11:27:52,884][20381] Updated weights for policy 1, policy_version 840938 (0.0007) [2023-12-27 11:27:52,893][20347] Updated weights for policy 0, policy_version 841673 (0.0007) [2023-12-27 11:27:52,945][20381] Updated weights for policy 1, policy_version 840948 (0.0006) [2023-12-27 11:27:52,954][20347] Updated weights for policy 0, policy_version 841683 (0.0006) [2023-12-27 11:27:53,009][20347] Updated weights for policy 0, policy_version 841693 (0.0008) [2023-12-27 11:27:53,677][20381] Updated weights for policy 1, policy_version 840958 (0.0006) [2023-12-27 11:27:53,687][20347] Updated weights for policy 0, policy_version 841703 (0.0008) [2023-12-27 11:27:53,731][20381] Updated weights for policy 1, policy_version 840968 (0.0006) [2023-12-27 11:27:53,745][20347] Updated weights for policy 0, policy_version 841713 (0.0007) [2023-12-27 11:27:53,785][20381] Updated weights for policy 1, policy_version 840978 (0.0005) [2023-12-27 11:27:53,803][20347] Updated weights for policy 0, policy_version 841723 (0.0007) [2023-12-27 11:27:54,514][20381] Updated weights for policy 1, policy_version 840988 (0.0007) [2023-12-27 11:27:54,516][20347] Updated weights for policy 0, policy_version 841733 (0.0007) [2023-12-27 11:27:54,568][20381] Updated weights for policy 1, policy_version 840998 (0.0008) [2023-12-27 11:27:54,573][20347] Updated weights for policy 0, policy_version 841743 (0.0006) [2023-12-27 11:27:54,619][20381] Updated weights for policy 1, policy_version 841008 (0.0008) [2023-12-27 11:27:54,633][20347] Updated weights for policy 0, policy_version 841753 (0.0005) [2023-12-27 11:27:55,320][20347] Updated weights for policy 0, policy_version 841763 (0.0007) [2023-12-27 11:27:55,344][20381] Updated weights for policy 1, policy_version 841018 (0.0007) [2023-12-27 11:27:55,377][20347] Updated weights for policy 0, policy_version 841773 (0.0006) [2023-12-27 11:27:55,401][20381] Updated weights for policy 1, policy_version 841028 (0.0007) [2023-12-27 11:27:55,439][20347] Updated weights for policy 0, policy_version 841783 (0.0008) [2023-12-27 11:27:55,463][20381] Updated weights for policy 1, policy_version 841038 (0.0009) [2023-12-27 11:27:55,520][20381] Updated weights for policy 1, policy_version 841048 (0.0008) [2023-12-27 11:27:56,135][20347] Updated weights for policy 0, policy_version 841793 (0.0008) [2023-12-27 11:27:56,189][20347] Updated weights for policy 0, policy_version 841803 (0.0007) [2023-12-27 11:27:56,200][20381] Updated weights for policy 1, policy_version 841058 (0.0005) [2023-12-27 11:27:56,246][20347] Updated weights for policy 0, policy_version 841813 (0.0007) [2023-12-27 11:27:56,248][20381] Updated weights for policy 1, policy_version 841068 (0.0007) [2023-12-27 11:27:56,299][20381] Updated weights for policy 1, policy_version 841078 (0.0008) [2023-12-27 11:27:56,305][20347] Updated weights for policy 0, policy_version 841823 (0.0007) [2023-12-27 11:27:56,983][20347] Updated weights for policy 0, policy_version 841833 (0.0010) [2023-12-27 11:27:57,016][20381] Updated weights for policy 1, policy_version 841088 (0.0006) [2023-12-27 11:27:57,037][20347] Updated weights for policy 0, policy_version 841843 (0.0011) [2023-12-27 11:27:57,077][20381] Updated weights for policy 1, policy_version 841098 (0.0005) [2023-12-27 11:27:57,093][20347] Updated weights for policy 0, policy_version 841853 (0.0010) [2023-12-27 11:27:57,131][20381] Updated weights for policy 1, policy_version 841108 (0.0006) [2023-12-27 11:27:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 430891008. Throughput: 0: 9718.9, 1: 9836.6. Samples: 430877496. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:27:57,363][19170] Avg episode reward: [(0, '326.350'), (1, '765.880')] [2023-12-27 11:27:57,794][20347] Updated weights for policy 0, policy_version 841863 (0.0007) [2023-12-27 11:27:57,851][20347] Updated weights for policy 0, policy_version 841873 (0.0009) [2023-12-27 11:27:57,852][20381] Updated weights for policy 1, policy_version 841118 (0.0007) [2023-12-27 11:27:57,903][20347] Updated weights for policy 0, policy_version 841883 (0.0010) [2023-12-27 11:27:57,913][20381] Updated weights for policy 1, policy_version 841128 (0.0006) [2023-12-27 11:27:57,974][20381] Updated weights for policy 1, policy_version 841138 (0.0007) [2023-12-27 11:27:58,644][20347] Updated weights for policy 0, policy_version 841893 (0.0007) [2023-12-27 11:27:58,711][20347] Updated weights for policy 0, policy_version 841903 (0.0006) [2023-12-27 11:27:58,737][20381] Updated weights for policy 1, policy_version 841148 (0.0007) [2023-12-27 11:27:58,789][20347] Updated weights for policy 0, policy_version 841913 (0.0009) [2023-12-27 11:27:58,808][20381] Updated weights for policy 1, policy_version 841158 (0.0007) [2023-12-27 11:27:58,879][20381] Updated weights for policy 1, policy_version 841168 (0.0007) [2023-12-27 11:27:59,674][20381] Updated weights for policy 1, policy_version 841178 (0.0008) [2023-12-27 11:27:59,676][20347] Updated weights for policy 0, policy_version 841923 (0.0007) [2023-12-27 11:27:59,734][20347] Updated weights for policy 0, policy_version 841933 (0.0007) [2023-12-27 11:27:59,737][20381] Updated weights for policy 1, policy_version 841188 (0.0008) [2023-12-27 11:27:59,792][20381] Updated weights for policy 1, policy_version 841198 (0.0008) [2023-12-27 11:27:59,797][20347] Updated weights for policy 0, policy_version 841943 (0.0011) [2023-12-27 11:27:59,853][20381] Updated weights for policy 1, policy_version 841208 (0.0009) [2023-12-27 11:28:00,450][20347] Updated weights for policy 0, policy_version 841953 (0.0009) [2023-12-27 11:28:00,515][20347] Updated weights for policy 0, policy_version 841963 (0.0007) [2023-12-27 11:28:00,581][20347] Updated weights for policy 0, policy_version 841973 (0.0010) [2023-12-27 11:28:00,637][20347] Updated weights for policy 0, policy_version 841983 (0.0009) [2023-12-27 11:28:00,639][20381] Updated weights for policy 1, policy_version 841218 (0.0010) [2023-12-27 11:28:00,698][20381] Updated weights for policy 1, policy_version 841228 (0.0011) [2023-12-27 11:28:00,762][20381] Updated weights for policy 1, policy_version 841238 (0.0011) [2023-12-27 11:28:01,295][20347] Updated weights for policy 0, policy_version 841993 (0.0008) [2023-12-27 11:28:01,349][20347] Updated weights for policy 0, policy_version 842003 (0.0007) [2023-12-27 11:28:01,414][20347] Updated weights for policy 0, policy_version 842013 (0.0009) [2023-12-27 11:28:01,503][20381] Updated weights for policy 1, policy_version 841248 (0.0011) [2023-12-27 11:28:01,561][20381] Updated weights for policy 1, policy_version 841258 (0.0009) [2023-12-27 11:28:01,636][20381] Updated weights for policy 1, policy_version 841268 (0.0009) [2023-12-27 11:28:02,209][20347] Updated weights for policy 0, policy_version 842023 (0.0007) [2023-12-27 11:28:02,273][20347] Updated weights for policy 0, policy_version 842033 (0.0007) [2023-12-27 11:28:02,334][20347] Updated weights for policy 0, policy_version 842043 (0.0006) [2023-12-27 11:28:02,362][19170] Fps is (10 sec: 18022.5, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 430972928. Throughput: 0: 9695.4, 1: 9788.6. Samples: 430990244. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 11:28:02,363][19170] Avg episode reward: [(0, '330.520'), (1, '756.990')] [2023-12-27 11:28:02,374][20381] Updated weights for policy 1, policy_version 841278 (0.0008) [2023-12-27 11:28:02,431][20381] Updated weights for policy 1, policy_version 841288 (0.0009) [2023-12-27 11:28:02,487][20381] Updated weights for policy 1, policy_version 841298 (0.0008) [2023-12-27 11:28:03,003][20347] Updated weights for policy 0, policy_version 842053 (0.0008) [2023-12-27 11:28:03,061][20347] Updated weights for policy 0, policy_version 842063 (0.0007) [2023-12-27 11:28:03,123][20347] Updated weights for policy 0, policy_version 842073 (0.0007) [2023-12-27 11:28:03,154][20381] Updated weights for policy 1, policy_version 841308 (0.0010) [2023-12-27 11:28:03,211][20381] Updated weights for policy 1, policy_version 841318 (0.0011) [2023-12-27 11:28:03,272][20381] Updated weights for policy 1, policy_version 841328 (0.0009) [2023-12-27 11:28:03,781][20347] Updated weights for policy 0, policy_version 842083 (0.0007) [2023-12-27 11:28:03,834][20347] Updated weights for policy 0, policy_version 842093 (0.0007) [2023-12-27 11:28:03,899][20347] Updated weights for policy 0, policy_version 842103 (0.0008) [2023-12-27 11:28:03,966][20381] Updated weights for policy 1, policy_version 841338 (0.0009) [2023-12-27 11:28:04,028][20381] Updated weights for policy 1, policy_version 841348 (0.0010) [2023-12-27 11:28:04,091][20381] Updated weights for policy 1, policy_version 841358 (0.0010) [2023-12-27 11:28:04,161][20381] Updated weights for policy 1, policy_version 841368 (0.0008) [2023-12-27 11:28:04,632][20347] Updated weights for policy 0, policy_version 842113 (0.0008) [2023-12-27 11:28:04,688][20347] Updated weights for policy 0, policy_version 842123 (0.0009) [2023-12-27 11:28:04,747][20347] Updated weights for policy 0, policy_version 842133 (0.0007) [2023-12-27 11:28:04,801][20347] Updated weights for policy 0, policy_version 842143 (0.0007) [2023-12-27 11:28:04,876][20381] Updated weights for policy 1, policy_version 841378 (0.0006) [2023-12-27 11:28:04,933][20381] Updated weights for policy 1, policy_version 841388 (0.0006) [2023-12-27 11:28:04,992][20381] Updated weights for policy 1, policy_version 841398 (0.0006) [2023-12-27 11:28:05,452][20347] Updated weights for policy 0, policy_version 842153 (0.0008) [2023-12-27 11:28:05,514][20347] Updated weights for policy 0, policy_version 842163 (0.0007) [2023-12-27 11:28:05,570][20347] Updated weights for policy 0, policy_version 842173 (0.0007) [2023-12-27 11:28:05,688][20381] Updated weights for policy 1, policy_version 841408 (0.0008) [2023-12-27 11:28:05,746][20381] Updated weights for policy 1, policy_version 841418 (0.0007) [2023-12-27 11:28:05,804][20381] Updated weights for policy 1, policy_version 841428 (0.0007) [2023-12-27 11:28:06,314][20347] Updated weights for policy 0, policy_version 842183 (0.0008) [2023-12-27 11:28:06,381][20347] Updated weights for policy 0, policy_version 842193 (0.0008) [2023-12-27 11:28:06,444][20347] Updated weights for policy 0, policy_version 842203 (0.0007) [2023-12-27 11:28:06,521][20381] Updated weights for policy 1, policy_version 841438 (0.0006) [2023-12-27 11:28:06,587][20381] Updated weights for policy 1, policy_version 841448 (0.0007) [2023-12-27 11:28:06,651][20381] Updated weights for policy 1, policy_version 841458 (0.0008) [2023-12-27 11:28:07,170][20347] Updated weights for policy 0, policy_version 842213 (0.0008) [2023-12-27 11:28:07,233][20347] Updated weights for policy 0, policy_version 842223 (0.0009) [2023-12-27 11:28:07,290][20381] Updated weights for policy 1, policy_version 841468 (0.0008) [2023-12-27 11:28:07,292][20347] Updated weights for policy 0, policy_version 842233 (0.0007) [2023-12-27 11:28:07,343][20381] Updated weights for policy 1, policy_version 841478 (0.0007) [2023-12-27 11:28:07,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 431079424. Throughput: 0: 9712.6, 1: 9787.0. Samples: 431050120. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:07,363][19170] Avg episode reward: [(0, '296.520'), (1, '718.510')] [2023-12-27 11:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000842240_215638016.pth... [2023-12-27 11:28:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000841088_215343104.pth [2023-12-27 11:28:07,410][20381] Updated weights for policy 1, policy_version 841488 (0.0009) [2023-12-27 11:28:07,459][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000841496_215449600.pth... [2023-12-27 11:28:07,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000840344_215154688.pth [2023-12-27 11:28:07,991][20347] Updated weights for policy 0, policy_version 842243 (0.0008) [2023-12-27 11:28:08,052][20347] Updated weights for policy 0, policy_version 842253 (0.0008) [2023-12-27 11:28:08,116][20347] Updated weights for policy 0, policy_version 842263 (0.0007) [2023-12-27 11:28:08,140][20381] Updated weights for policy 1, policy_version 841498 (0.0008) [2023-12-27 11:28:08,199][20381] Updated weights for policy 1, policy_version 841508 (0.0007) [2023-12-27 11:28:08,262][20381] Updated weights for policy 1, policy_version 841518 (0.0007) [2023-12-27 11:28:08,321][20381] Updated weights for policy 1, policy_version 841528 (0.0007) [2023-12-27 11:28:08,839][20347] Updated weights for policy 0, policy_version 842273 (0.0006) [2023-12-27 11:28:08,904][20347] Updated weights for policy 0, policy_version 842283 (0.0008) [2023-12-27 11:28:08,960][20347] Updated weights for policy 0, policy_version 842293 (0.0008) [2023-12-27 11:28:09,023][20347] Updated weights for policy 0, policy_version 842303 (0.0009) [2023-12-27 11:28:09,072][20381] Updated weights for policy 1, policy_version 841538 (0.0008) [2023-12-27 11:28:09,129][20381] Updated weights for policy 1, policy_version 841548 (0.0008) [2023-12-27 11:28:09,185][20381] Updated weights for policy 1, policy_version 841558 (0.0009) [2023-12-27 11:28:09,843][20347] Updated weights for policy 0, policy_version 842313 (0.0008) [2023-12-27 11:28:09,909][20381] Updated weights for policy 1, policy_version 841568 (0.0008) [2023-12-27 11:28:09,910][20347] Updated weights for policy 0, policy_version 842323 (0.0008) [2023-12-27 11:28:09,977][20347] Updated weights for policy 0, policy_version 842333 (0.0008) [2023-12-27 11:28:09,978][20381] Updated weights for policy 1, policy_version 841578 (0.0007) [2023-12-27 11:28:10,043][20381] Updated weights for policy 1, policy_version 841588 (0.0007) [2023-12-27 11:28:10,729][20381] Updated weights for policy 1, policy_version 841598 (0.0007) [2023-12-27 11:28:10,791][20381] Updated weights for policy 1, policy_version 841608 (0.0007) [2023-12-27 11:28:10,801][20347] Updated weights for policy 0, policy_version 842343 (0.0007) [2023-12-27 11:28:10,842][20381] Updated weights for policy 1, policy_version 841618 (0.0006) [2023-12-27 11:28:10,859][20347] Updated weights for policy 0, policy_version 842353 (0.0006) [2023-12-27 11:28:10,920][20347] Updated weights for policy 0, policy_version 842363 (0.0007) [2023-12-27 11:28:11,594][20381] Updated weights for policy 1, policy_version 841628 (0.0007) [2023-12-27 11:28:11,653][20381] Updated weights for policy 1, policy_version 841638 (0.0008) [2023-12-27 11:28:11,711][20381] Updated weights for policy 1, policy_version 841648 (0.0008) [2023-12-27 11:28:11,744][20347] Updated weights for policy 0, policy_version 842373 (0.0008) [2023-12-27 11:28:11,813][20347] Updated weights for policy 0, policy_version 842383 (0.0008) [2023-12-27 11:28:11,878][20347] Updated weights for policy 0, policy_version 842393 (0.0008) [2023-12-27 11:28:12,363][19170] Fps is (10 sec: 19659.7, 60 sec: 19387.6, 300 sec: 19438.6). Total num frames: 431169536. Throughput: 0: 9659.8, 1: 9823.2. Samples: 431164484. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:12,364][19170] Avg episode reward: [(0, '254.950'), (1, '681.760')] [2023-12-27 11:28:12,513][20381] Updated weights for policy 1, policy_version 841658 (0.0010) [2023-12-27 11:28:12,571][20381] Updated weights for policy 1, policy_version 841668 (0.0007) [2023-12-27 11:28:12,627][20347] Updated weights for policy 0, policy_version 842403 (0.0007) [2023-12-27 11:28:12,641][20381] Updated weights for policy 1, policy_version 841678 (0.0009) [2023-12-27 11:28:12,691][20347] Updated weights for policy 0, policy_version 842413 (0.0007) [2023-12-27 11:28:12,705][20381] Updated weights for policy 1, policy_version 841688 (0.0009) [2023-12-27 11:28:12,756][20347] Updated weights for policy 0, policy_version 842423 (0.0007) [2023-12-27 11:28:13,414][20381] Updated weights for policy 1, policy_version 841698 (0.0010) [2023-12-27 11:28:13,469][20381] Updated weights for policy 1, policy_version 841708 (0.0007) [2023-12-27 11:28:13,495][20347] Updated weights for policy 0, policy_version 842433 (0.0007) [2023-12-27 11:28:13,525][20381] Updated weights for policy 1, policy_version 841718 (0.0008) [2023-12-27 11:28:13,561][20347] Updated weights for policy 0, policy_version 842443 (0.0008) [2023-12-27 11:28:13,625][20347] Updated weights for policy 0, policy_version 842453 (0.0008) [2023-12-27 11:28:13,685][20347] Updated weights for policy 0, policy_version 842463 (0.0008) [2023-12-27 11:28:14,165][20381] Updated weights for policy 1, policy_version 841728 (0.0008) [2023-12-27 11:28:14,224][20381] Updated weights for policy 1, policy_version 841738 (0.0008) [2023-12-27 11:28:14,276][20381] Updated weights for policy 1, policy_version 841748 (0.0008) [2023-12-27 11:28:14,408][20347] Updated weights for policy 0, policy_version 842473 (0.0008) [2023-12-27 11:28:14,462][20347] Updated weights for policy 0, policy_version 842483 (0.0008) [2023-12-27 11:28:14,513][20347] Updated weights for policy 0, policy_version 842493 (0.0007) [2023-12-27 11:28:14,931][20381] Updated weights for policy 1, policy_version 841758 (0.0008) [2023-12-27 11:28:14,989][20381] Updated weights for policy 1, policy_version 841768 (0.0007) [2023-12-27 11:28:15,047][20381] Updated weights for policy 1, policy_version 841778 (0.0008) [2023-12-27 11:28:15,296][20347] Updated weights for policy 0, policy_version 842503 (0.0007) [2023-12-27 11:28:15,353][20347] Updated weights for policy 0, policy_version 842513 (0.0006) [2023-12-27 11:28:15,409][20347] Updated weights for policy 0, policy_version 842523 (0.0005) [2023-12-27 11:28:15,741][20381] Updated weights for policy 1, policy_version 841788 (0.0008) [2023-12-27 11:28:15,802][20381] Updated weights for policy 1, policy_version 841798 (0.0007) [2023-12-27 11:28:15,867][20381] Updated weights for policy 1, policy_version 841808 (0.0006) [2023-12-27 11:28:16,112][20347] Updated weights for policy 0, policy_version 842533 (0.0007) [2023-12-27 11:28:16,161][20347] Updated weights for policy 0, policy_version 842543 (0.0008) [2023-12-27 11:28:16,213][20347] Updated weights for policy 0, policy_version 842553 (0.0009) [2023-12-27 11:28:16,495][20381] Updated weights for policy 1, policy_version 841818 (0.0005) [2023-12-27 11:28:16,560][20381] Updated weights for policy 1, policy_version 841828 (0.0008) [2023-12-27 11:28:16,605][20381] Updated weights for policy 1, policy_version 841838 (0.0007) [2023-12-27 11:28:16,658][20381] Updated weights for policy 1, policy_version 841848 (0.0008) [2023-12-27 11:28:16,841][20347] Updated weights for policy 0, policy_version 842563 (0.0007) [2023-12-27 11:28:16,903][20347] Updated weights for policy 0, policy_version 842573 (0.0006) [2023-12-27 11:28:16,963][20347] Updated weights for policy 0, policy_version 842583 (0.0008) [2023-12-27 11:28:17,339][20381] Updated weights for policy 1, policy_version 841858 (0.0007) [2023-12-27 11:28:17,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19438.7). Total num frames: 431267840. Throughput: 0: 9593.8, 1: 9808.7. Samples: 431280892. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:17,363][19170] Avg episode reward: [(0, '276.200'), (1, '786.870')] [2023-12-27 11:28:17,392][20381] Updated weights for policy 1, policy_version 841868 (0.0008) [2023-12-27 11:28:17,437][20381] Updated weights for policy 1, policy_version 841878 (0.0005) [2023-12-27 11:28:17,604][20347] Updated weights for policy 0, policy_version 842593 (0.0008) [2023-12-27 11:28:17,665][20347] Updated weights for policy 0, policy_version 842603 (0.0010) [2023-12-27 11:28:17,719][20347] Updated weights for policy 0, policy_version 842613 (0.0010) [2023-12-27 11:28:17,770][20347] Updated weights for policy 0, policy_version 842623 (0.0010) [2023-12-27 11:28:18,101][20381] Updated weights for policy 1, policy_version 841888 (0.0007) [2023-12-27 11:28:18,158][20381] Updated weights for policy 1, policy_version 841898 (0.0007) [2023-12-27 11:28:18,212][20381] Updated weights for policy 1, policy_version 841908 (0.0007) [2023-12-27 11:28:18,458][20347] Updated weights for policy 0, policy_version 842633 (0.0011) [2023-12-27 11:28:18,514][20347] Updated weights for policy 0, policy_version 842643 (0.0010) [2023-12-27 11:28:18,569][20347] Updated weights for policy 0, policy_version 842653 (0.0010) [2023-12-27 11:28:18,916][20381] Updated weights for policy 1, policy_version 841918 (0.0007) [2023-12-27 11:28:18,977][20381] Updated weights for policy 1, policy_version 841928 (0.0008) [2023-12-27 11:28:19,035][20381] Updated weights for policy 1, policy_version 841938 (0.0007) [2023-12-27 11:28:19,277][20347] Updated weights for policy 0, policy_version 842663 (0.0009) [2023-12-27 11:28:19,346][20347] Updated weights for policy 0, policy_version 842673 (0.0008) [2023-12-27 11:28:19,422][20347] Updated weights for policy 0, policy_version 842683 (0.0008) [2023-12-27 11:28:19,758][20381] Updated weights for policy 1, policy_version 841948 (0.0008) [2023-12-27 11:28:19,823][20381] Updated weights for policy 1, policy_version 841958 (0.0009) [2023-12-27 11:28:19,889][20381] Updated weights for policy 1, policy_version 841968 (0.0009) [2023-12-27 11:28:20,151][20347] Updated weights for policy 0, policy_version 842693 (0.0008) [2023-12-27 11:28:20,211][20347] Updated weights for policy 0, policy_version 842703 (0.0008) [2023-12-27 11:28:20,270][20347] Updated weights for policy 0, policy_version 842713 (0.0008) [2023-12-27 11:28:20,580][20381] Updated weights for policy 1, policy_version 841978 (0.0009) [2023-12-27 11:28:20,643][20381] Updated weights for policy 1, policy_version 841988 (0.0007) [2023-12-27 11:28:20,705][20381] Updated weights for policy 1, policy_version 841998 (0.0006) [2023-12-27 11:28:20,765][20381] Updated weights for policy 1, policy_version 842008 (0.0008) [2023-12-27 11:28:21,043][20347] Updated weights for policy 0, policy_version 842723 (0.0008) [2023-12-27 11:28:21,113][20347] Updated weights for policy 0, policy_version 842733 (0.0008) [2023-12-27 11:28:21,185][20347] Updated weights for policy 0, policy_version 842743 (0.0008) [2023-12-27 11:28:21,518][20381] Updated weights for policy 1, policy_version 842018 (0.0007) [2023-12-27 11:28:21,577][20381] Updated weights for policy 1, policy_version 842028 (0.0007) [2023-12-27 11:28:21,642][20381] Updated weights for policy 1, policy_version 842038 (0.0007) [2023-12-27 11:28:21,957][20347] Updated weights for policy 0, policy_version 842753 (0.0008) [2023-12-27 11:28:22,010][20347] Updated weights for policy 0, policy_version 842763 (0.0008) [2023-12-27 11:28:22,036][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-27 11:28:22,362][19170] Fps is (10 sec: 19661.8, 60 sec: 19524.3, 300 sec: 19438.6). Total num frames: 431366144. Throughput: 0: 9609.8, 1: 9764.6. Samples: 431339900. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:22,363][19170] Avg episode reward: [(0, '283.480'), (1, '835.240')] [2023-12-27 11:28:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000842768_215777280.pth... [2023-12-27 11:28:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000841664_215490560.pth [2023-12-27 11:28:22,401][20381] Updated weights for policy 1, policy_version 842048 (0.0008) [2023-12-27 11:28:22,450][20381] Updated weights for policy 1, policy_version 842058 (0.0007) [2023-12-27 11:28:22,509][20381] Updated weights for policy 1, policy_version 842068 (0.0008) [2023-12-27 11:28:22,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000842072_215597056.pth... [2023-12-27 11:28:22,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000840920_215302144.pth [2023-12-27 11:28:22,852][20347] Updated weights for policy 0, policy_version 842773 (0.0009) [2023-12-27 11:28:22,911][20347] Updated weights for policy 0, policy_version 842783 (0.0008) [2023-12-27 11:28:22,968][20347] Updated weights for policy 0, policy_version 842793 (0.0008) [2023-12-27 11:28:23,136][20381] Updated weights for policy 1, policy_version 842078 (0.0007) [2023-12-27 11:28:23,198][20381] Updated weights for policy 1, policy_version 842088 (0.0007) [2023-12-27 11:28:23,254][20381] Updated weights for policy 1, policy_version 842098 (0.0007) [2023-12-27 11:28:23,715][20347] Updated weights for policy 0, policy_version 842803 (0.0008) [2023-12-27 11:28:23,766][20347] Updated weights for policy 0, policy_version 842813 (0.0009) [2023-12-27 11:28:23,810][20381] Updated weights for policy 1, policy_version 842108 (0.0006) [2023-12-27 11:28:23,817][20347] Updated weights for policy 0, policy_version 842823 (0.0010) [2023-12-27 11:28:23,864][20381] Updated weights for policy 1, policy_version 842118 (0.0005) [2023-12-27 11:28:23,922][20381] Updated weights for policy 1, policy_version 842128 (0.0005) [2023-12-27 11:28:24,481][20347] Updated weights for policy 0, policy_version 842833 (0.0010) [2023-12-27 11:28:24,518][20381] Updated weights for policy 1, policy_version 842138 (0.0007) [2023-12-27 11:28:24,529][20347] Updated weights for policy 0, policy_version 842843 (0.0010) [2023-12-27 11:28:24,569][20381] Updated weights for policy 1, policy_version 842148 (0.0007) [2023-12-27 11:28:24,574][20347] Updated weights for policy 0, policy_version 842853 (0.0010) [2023-12-27 11:28:24,620][20347] Updated weights for policy 0, policy_version 842863 (0.0010) [2023-12-27 11:28:24,621][20381] Updated weights for policy 1, policy_version 842158 (0.0005) [2023-12-27 11:28:24,672][20381] Updated weights for policy 1, policy_version 842168 (0.0008) [2023-12-27 11:28:25,310][20381] Updated weights for policy 1, policy_version 842178 (0.0007) [2023-12-27 11:28:25,326][20347] Updated weights for policy 0, policy_version 842873 (0.0006) [2023-12-27 11:28:25,369][20381] Updated weights for policy 1, policy_version 842188 (0.0007) [2023-12-27 11:28:25,372][20347] Updated weights for policy 0, policy_version 842883 (0.0006) [2023-12-27 11:28:25,418][20347] Updated weights for policy 0, policy_version 842893 (0.0007) [2023-12-27 11:28:25,426][20381] Updated weights for policy 1, policy_version 842198 (0.0008) [2023-12-27 11:28:26,008][20347] Updated weights for policy 0, policy_version 842903 (0.0007) [2023-12-27 11:28:26,066][20347] Updated weights for policy 0, policy_version 842913 (0.0006) [2023-12-27 11:28:26,122][20347] Updated weights for policy 0, policy_version 842923 (0.0008) [2023-12-27 11:28:26,123][20381] Updated weights for policy 1, policy_version 842208 (0.0006) [2023-12-27 11:28:26,183][20381] Updated weights for policy 1, policy_version 842218 (0.0007) [2023-12-27 11:28:26,234][20381] Updated weights for policy 1, policy_version 842228 (0.0008) [2023-12-27 11:28:26,812][20347] Updated weights for policy 0, policy_version 842933 (0.0008) [2023-12-27 11:28:26,860][20347] Updated weights for policy 0, policy_version 842943 (0.0007) [2023-12-27 11:28:26,904][20381] Updated weights for policy 1, policy_version 842238 (0.0007) [2023-12-27 11:28:26,910][20347] Updated weights for policy 0, policy_version 842953 (0.0008) [2023-12-27 11:28:26,962][20381] Updated weights for policy 1, policy_version 842248 (0.0005) [2023-12-27 11:28:27,026][20381] Updated weights for policy 1, policy_version 842258 (0.0005) [2023-12-27 11:28:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 431472640. Throughput: 0: 9634.9, 1: 9840.2. Samples: 431461852. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:27,363][19170] Avg episode reward: [(0, '324.990'), (1, '844.890')] [2023-12-27 11:28:27,593][20381] Updated weights for policy 1, policy_version 842268 (0.0005) [2023-12-27 11:28:27,647][20381] Updated weights for policy 1, policy_version 842278 (0.0005) [2023-12-27 11:28:27,660][20347] Updated weights for policy 0, policy_version 842963 (0.0006) [2023-12-27 11:28:27,703][20381] Updated weights for policy 1, policy_version 842288 (0.0005) [2023-12-27 11:28:27,718][20347] Updated weights for policy 0, policy_version 842973 (0.0007) [2023-12-27 11:28:27,771][20347] Updated weights for policy 0, policy_version 842983 (0.0008) [2023-12-27 11:28:28,336][20381] Updated weights for policy 1, policy_version 842298 (0.0009) [2023-12-27 11:28:28,395][20381] Updated weights for policy 1, policy_version 842308 (0.0011) [2023-12-27 11:28:28,448][20381] Updated weights for policy 1, policy_version 842318 (0.0011) [2023-12-27 11:28:28,453][20347] Updated weights for policy 0, policy_version 842993 (0.0007) [2023-12-27 11:28:28,504][20381] Updated weights for policy 1, policy_version 842328 (0.0010) [2023-12-27 11:28:28,504][20347] Updated weights for policy 0, policy_version 843003 (0.0005) [2023-12-27 11:28:28,557][20347] Updated weights for policy 0, policy_version 843013 (0.0007) [2023-12-27 11:28:28,604][20347] Updated weights for policy 0, policy_version 843023 (0.0006) [2023-12-27 11:28:29,193][20381] Updated weights for policy 1, policy_version 842338 (0.0010) [2023-12-27 11:28:29,221][20347] Updated weights for policy 0, policy_version 843033 (0.0008) [2023-12-27 11:28:29,252][20381] Updated weights for policy 1, policy_version 842348 (0.0010) [2023-12-27 11:28:29,282][20347] Updated weights for policy 0, policy_version 843043 (0.0007) [2023-12-27 11:28:29,307][20381] Updated weights for policy 1, policy_version 842358 (0.0010) [2023-12-27 11:28:29,345][20347] Updated weights for policy 0, policy_version 843053 (0.0006) [2023-12-27 11:28:30,057][20381] Updated weights for policy 1, policy_version 842368 (0.0008) [2023-12-27 11:28:30,114][20381] Updated weights for policy 1, policy_version 842378 (0.0008) [2023-12-27 11:28:30,142][20347] Updated weights for policy 0, policy_version 843063 (0.0007) [2023-12-27 11:28:30,182][20381] Updated weights for policy 1, policy_version 842388 (0.0008) [2023-12-27 11:28:30,198][20347] Updated weights for policy 0, policy_version 843073 (0.0008) [2023-12-27 11:28:30,251][20347] Updated weights for policy 0, policy_version 843083 (0.0008) [2023-12-27 11:28:30,851][20381] Updated weights for policy 1, policy_version 842398 (0.0008) [2023-12-27 11:28:30,909][20381] Updated weights for policy 1, policy_version 842408 (0.0007) [2023-12-27 11:28:30,968][20381] Updated weights for policy 1, policy_version 842418 (0.0006) [2023-12-27 11:28:30,995][20347] Updated weights for policy 0, policy_version 843093 (0.0008) [2023-12-27 11:28:31,059][20347] Updated weights for policy 0, policy_version 843103 (0.0008) [2023-12-27 11:28:31,121][20347] Updated weights for policy 0, policy_version 843113 (0.0007) [2023-12-27 11:28:31,735][20381] Updated weights for policy 1, policy_version 842428 (0.0008) [2023-12-27 11:28:31,796][20381] Updated weights for policy 1, policy_version 842438 (0.0010) [2023-12-27 11:28:31,802][20347] Updated weights for policy 0, policy_version 843123 (0.0006) [2023-12-27 11:28:31,852][20381] Updated weights for policy 1, policy_version 842448 (0.0008) [2023-12-27 11:28:31,855][20347] Updated weights for policy 0, policy_version 843133 (0.0007) [2023-12-27 11:28:31,915][20347] Updated weights for policy 0, policy_version 843143 (0.0006) [2023-12-27 11:28:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 431570944. Throughput: 0: 9651.8, 1: 9853.6. Samples: 431582456. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:32,363][19170] Avg episode reward: [(0, '387.260'), (1, '582.830')] [2023-12-27 11:28:32,484][20381] Updated weights for policy 1, policy_version 842458 (0.0008) [2023-12-27 11:28:32,549][20381] Updated weights for policy 1, policy_version 842468 (0.0009) [2023-12-27 11:28:32,611][20381] Updated weights for policy 1, policy_version 842478 (0.0008) [2023-12-27 11:28:32,619][20347] Updated weights for policy 0, policy_version 843153 (0.0006) [2023-12-27 11:28:32,669][20381] Updated weights for policy 1, policy_version 842488 (0.0008) [2023-12-27 11:28:32,691][20347] Updated weights for policy 0, policy_version 843163 (0.0011) [2023-12-27 11:28:32,756][20347] Updated weights for policy 0, policy_version 843173 (0.0010) [2023-12-27 11:28:32,834][20347] Updated weights for policy 0, policy_version 843183 (0.0011) [2023-12-27 11:28:33,303][20381] Updated weights for policy 1, policy_version 842498 (0.0006) [2023-12-27 11:28:33,358][20381] Updated weights for policy 1, policy_version 842508 (0.0009) [2023-12-27 11:28:33,403][20381] Updated weights for policy 1, policy_version 842518 (0.0008) [2023-12-27 11:28:33,497][20347] Updated weights for policy 0, policy_version 843193 (0.0006) [2023-12-27 11:28:33,554][20347] Updated weights for policy 0, policy_version 843203 (0.0010) [2023-12-27 11:28:33,599][20347] Updated weights for policy 0, policy_version 843213 (0.0010) [2023-12-27 11:28:34,040][20381] Updated weights for policy 1, policy_version 842528 (0.0006) [2023-12-27 11:28:34,091][20381] Updated weights for policy 1, policy_version 842538 (0.0006) [2023-12-27 11:28:34,146][20381] Updated weights for policy 1, policy_version 842548 (0.0009) [2023-12-27 11:28:34,212][20347] Updated weights for policy 0, policy_version 843223 (0.0008) [2023-12-27 11:28:34,282][20347] Updated weights for policy 0, policy_version 843233 (0.0009) [2023-12-27 11:28:34,358][20347] Updated weights for policy 0, policy_version 843243 (0.0010) [2023-12-27 11:28:34,835][20381] Updated weights for policy 1, policy_version 842558 (0.0010) [2023-12-27 11:28:34,886][20381] Updated weights for policy 1, policy_version 842568 (0.0010) [2023-12-27 11:28:34,930][20381] Updated weights for policy 1, policy_version 842578 (0.0010) [2023-12-27 11:28:35,096][20347] Updated weights for policy 0, policy_version 843253 (0.0008) [2023-12-27 11:28:35,153][20347] Updated weights for policy 0, policy_version 843263 (0.0010) [2023-12-27 11:28:35,205][20347] Updated weights for policy 0, policy_version 843273 (0.0008) [2023-12-27 11:28:35,595][20381] Updated weights for policy 1, policy_version 842588 (0.0010) [2023-12-27 11:28:35,659][20381] Updated weights for policy 1, policy_version 842598 (0.0010) [2023-12-27 11:28:35,723][20381] Updated weights for policy 1, policy_version 842608 (0.0011) [2023-12-27 11:28:35,819][20347] Updated weights for policy 0, policy_version 843283 (0.0006) [2023-12-27 11:28:35,867][20347] Updated weights for policy 0, policy_version 843293 (0.0007) [2023-12-27 11:28:35,919][20347] Updated weights for policy 0, policy_version 843303 (0.0008) [2023-12-27 11:28:36,402][20381] Updated weights for policy 1, policy_version 842618 (0.0011) [2023-12-27 11:28:36,465][20381] Updated weights for policy 1, policy_version 842628 (0.0011) [2023-12-27 11:28:36,521][20381] Updated weights for policy 1, policy_version 842638 (0.0011) [2023-12-27 11:28:36,579][20381] Updated weights for policy 1, policy_version 842648 (0.0009) [2023-12-27 11:28:36,633][20347] Updated weights for policy 0, policy_version 843313 (0.0007) [2023-12-27 11:28:36,695][20347] Updated weights for policy 0, policy_version 843323 (0.0007) [2023-12-27 11:28:36,757][20347] Updated weights for policy 0, policy_version 843333 (0.0008) [2023-12-27 11:28:36,819][20347] Updated weights for policy 0, policy_version 843343 (0.0008) [2023-12-27 11:28:37,259][20381] Updated weights for policy 1, policy_version 842658 (0.0011) [2023-12-27 11:28:37,311][20381] Updated weights for policy 1, policy_version 842668 (0.0010) [2023-12-27 11:28:37,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19524.2, 300 sec: 19438.6). Total num frames: 431669248. Throughput: 0: 9693.1, 1: 9906.2. Samples: 431644096. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:37,363][19170] Avg episode reward: [(0, '479.990'), (1, '698.660')] [2023-12-27 11:28:37,369][20381] Updated weights for policy 1, policy_version 842678 (0.0011) [2023-12-27 11:28:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000843344_215924736.pth... [2023-12-27 11:28:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000842240_215638016.pth [2023-12-27 11:28:37,378][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000842680_215752704.pth... [2023-12-27 11:28:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000841496_215449600.pth [2023-12-27 11:28:37,496][20347] Updated weights for policy 0, policy_version 843353 (0.0007) [2023-12-27 11:28:37,547][20347] Updated weights for policy 0, policy_version 843363 (0.0007) [2023-12-27 11:28:37,604][20347] Updated weights for policy 0, policy_version 843373 (0.0007) [2023-12-27 11:28:37,983][20381] Updated weights for policy 1, policy_version 842688 (0.0010) [2023-12-27 11:28:38,049][20381] Updated weights for policy 1, policy_version 842698 (0.0010) [2023-12-27 11:28:38,108][20381] Updated weights for policy 1, policy_version 842708 (0.0010) [2023-12-27 11:28:38,326][20347] Updated weights for policy 0, policy_version 843383 (0.0007) [2023-12-27 11:28:38,393][20347] Updated weights for policy 0, policy_version 843393 (0.0006) [2023-12-27 11:28:38,453][20347] Updated weights for policy 0, policy_version 843403 (0.0006) [2023-12-27 11:28:38,797][20381] Updated weights for policy 1, policy_version 842718 (0.0009) [2023-12-27 11:28:38,862][20381] Updated weights for policy 1, policy_version 842728 (0.0007) [2023-12-27 11:28:38,919][20381] Updated weights for policy 1, policy_version 842738 (0.0006) [2023-12-27 11:28:39,061][20347] Updated weights for policy 0, policy_version 843413 (0.0007) [2023-12-27 11:28:39,114][20347] Updated weights for policy 0, policy_version 843423 (0.0007) [2023-12-27 11:28:39,166][20347] Updated weights for policy 0, policy_version 843433 (0.0007) [2023-12-27 11:28:39,533][20381] Updated weights for policy 1, policy_version 842748 (0.0007) [2023-12-27 11:28:39,592][20381] Updated weights for policy 1, policy_version 842758 (0.0008) [2023-12-27 11:28:39,640][20381] Updated weights for policy 1, policy_version 842768 (0.0008) [2023-12-27 11:28:39,919][20347] Updated weights for policy 0, policy_version 843443 (0.0008) [2023-12-27 11:28:39,976][20347] Updated weights for policy 0, policy_version 843453 (0.0008) [2023-12-27 11:28:40,032][20347] Updated weights for policy 0, policy_version 843463 (0.0008) [2023-12-27 11:28:40,366][20381] Updated weights for policy 1, policy_version 842778 (0.0007) [2023-12-27 11:28:40,421][20381] Updated weights for policy 1, policy_version 842788 (0.0008) [2023-12-27 11:28:40,478][20381] Updated weights for policy 1, policy_version 842798 (0.0008) [2023-12-27 11:28:40,535][20381] Updated weights for policy 1, policy_version 842808 (0.0008) [2023-12-27 11:28:40,674][20347] Updated weights for policy 0, policy_version 843473 (0.0007) [2023-12-27 11:28:40,726][20347] Updated weights for policy 0, policy_version 843483 (0.0008) [2023-12-27 11:28:40,784][20347] Updated weights for policy 0, policy_version 843493 (0.0008) [2023-12-27 11:28:40,839][20347] Updated weights for policy 0, policy_version 843503 (0.0007) [2023-12-27 11:28:41,249][20381] Updated weights for policy 1, policy_version 842818 (0.0008) [2023-12-27 11:28:41,306][20381] Updated weights for policy 1, policy_version 842828 (0.0008) [2023-12-27 11:28:41,377][20381] Updated weights for policy 1, policy_version 842838 (0.0008) [2023-12-27 11:28:41,625][20347] Updated weights for policy 0, policy_version 843513 (0.0008) [2023-12-27 11:28:41,691][20347] Updated weights for policy 0, policy_version 843523 (0.0008) [2023-12-27 11:28:41,760][20347] Updated weights for policy 0, policy_version 843533 (0.0007) [2023-12-27 11:28:42,132][20381] Updated weights for policy 1, policy_version 842848 (0.0006) [2023-12-27 11:28:42,183][20381] Updated weights for policy 1, policy_version 842858 (0.0005) [2023-12-27 11:28:42,243][20381] Updated weights for policy 1, policy_version 842868 (0.0006) [2023-12-27 11:28:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 19660.8, 300 sec: 19466.4). Total num frames: 431775744. Throughput: 0: 9758.4, 1: 9980.0. Samples: 431765724. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:42,363][19170] Avg episode reward: [(0, '287.820'), (1, '757.440')] [2023-12-27 11:28:42,417][20347] Updated weights for policy 0, policy_version 843543 (0.0007) [2023-12-27 11:28:42,475][20347] Updated weights for policy 0, policy_version 843553 (0.0008) [2023-12-27 11:28:42,533][20347] Updated weights for policy 0, policy_version 843563 (0.0008) [2023-12-27 11:28:42,978][20381] Updated weights for policy 1, policy_version 842878 (0.0008) [2023-12-27 11:28:43,040][20381] Updated weights for policy 1, policy_version 842888 (0.0008) [2023-12-27 11:28:43,089][20381] Updated weights for policy 1, policy_version 842898 (0.0006) [2023-12-27 11:28:43,227][20347] Updated weights for policy 0, policy_version 843573 (0.0007) [2023-12-27 11:28:43,279][20347] Updated weights for policy 0, policy_version 843583 (0.0006) [2023-12-27 11:28:43,335][20347] Updated weights for policy 0, policy_version 843593 (0.0010) [2023-12-27 11:28:43,704][20381] Updated weights for policy 1, policy_version 842908 (0.0006) [2023-12-27 11:28:43,762][20381] Updated weights for policy 1, policy_version 842918 (0.0008) [2023-12-27 11:28:43,809][20381] Updated weights for policy 1, policy_version 842928 (0.0007) [2023-12-27 11:28:44,004][20347] Updated weights for policy 0, policy_version 843603 (0.0010) [2023-12-27 11:28:44,063][20347] Updated weights for policy 0, policy_version 843613 (0.0010) [2023-12-27 11:28:44,117][20347] Updated weights for policy 0, policy_version 843623 (0.0010) [2023-12-27 11:28:44,524][20381] Updated weights for policy 1, policy_version 842938 (0.0007) [2023-12-27 11:28:44,571][20381] Updated weights for policy 1, policy_version 842948 (0.0007) [2023-12-27 11:28:44,619][20381] Updated weights for policy 1, policy_version 842958 (0.0007) [2023-12-27 11:28:44,677][20381] Updated weights for policy 1, policy_version 842968 (0.0007) [2023-12-27 11:28:44,783][20347] Updated weights for policy 0, policy_version 843633 (0.0010) [2023-12-27 11:28:44,852][20347] Updated weights for policy 0, policy_version 843643 (0.0010) [2023-12-27 11:28:44,915][20347] Updated weights for policy 0, policy_version 843653 (0.0011) [2023-12-27 11:28:44,979][20347] Updated weights for policy 0, policy_version 843663 (0.0010) [2023-12-27 11:28:45,402][20381] Updated weights for policy 1, policy_version 842978 (0.0008) [2023-12-27 11:28:45,460][20381] Updated weights for policy 1, policy_version 842988 (0.0007) [2023-12-27 11:28:45,521][20381] Updated weights for policy 1, policy_version 842998 (0.0008) [2023-12-27 11:28:45,633][20347] Updated weights for policy 0, policy_version 843673 (0.0010) [2023-12-27 11:28:45,692][20347] Updated weights for policy 0, policy_version 843683 (0.0010) [2023-12-27 11:28:45,751][20347] Updated weights for policy 0, policy_version 843693 (0.0010) [2023-12-27 11:28:46,272][20381] Updated weights for policy 1, policy_version 843008 (0.0007) [2023-12-27 11:28:46,327][20381] Updated weights for policy 1, policy_version 843018 (0.0007) [2023-12-27 11:28:46,397][20381] Updated weights for policy 1, policy_version 843028 (0.0008) [2023-12-27 11:28:46,438][20347] Updated weights for policy 0, policy_version 843703 (0.0010) [2023-12-27 11:28:46,501][20347] Updated weights for policy 0, policy_version 843713 (0.0006) [2023-12-27 11:28:46,559][20347] Updated weights for policy 0, policy_version 843723 (0.0010) [2023-12-27 11:28:47,104][20381] Updated weights for policy 1, policy_version 843038 (0.0007) [2023-12-27 11:28:47,167][20381] Updated weights for policy 1, policy_version 843048 (0.0009) [2023-12-27 11:28:47,185][20347] Updated weights for policy 0, policy_version 843733 (0.0008) [2023-12-27 11:28:47,227][20381] Updated weights for policy 1, policy_version 843058 (0.0008) [2023-12-27 11:28:47,239][20347] Updated weights for policy 0, policy_version 843743 (0.0005) [2023-12-27 11:28:47,244][20292] KL-divergence is very high: 107.5238 [2023-12-27 11:28:47,288][20347] Updated weights for policy 0, policy_version 843753 (0.0009) [2023-12-27 11:28:47,363][19170] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 431882240. Throughput: 0: 9834.5, 1: 10066.4. Samples: 431885788. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:47,364][19170] Avg episode reward: [(0, '436.520'), (1, '767.270')] [2023-12-27 11:28:47,847][20381] Updated weights for policy 1, policy_version 843068 (0.0009) [2023-12-27 11:28:47,906][20347] Updated weights for policy 0, policy_version 843763 (0.0009) [2023-12-27 11:28:47,906][20381] Updated weights for policy 1, policy_version 843078 (0.0010) [2023-12-27 11:28:47,963][20347] Updated weights for policy 0, policy_version 843773 (0.0006) [2023-12-27 11:28:47,970][20381] Updated weights for policy 1, policy_version 843088 (0.0005) [2023-12-27 11:28:48,021][20347] Updated weights for policy 0, policy_version 843783 (0.0006) [2023-12-27 11:28:48,519][20381] Updated weights for policy 1, policy_version 843098 (0.0006) [2023-12-27 11:28:48,560][20347] Updated weights for policy 0, policy_version 843793 (0.0005) [2023-12-27 11:28:48,574][20381] Updated weights for policy 1, policy_version 843108 (0.0009) [2023-12-27 11:28:48,616][20347] Updated weights for policy 0, policy_version 843803 (0.0010) [2023-12-27 11:28:48,622][20381] Updated weights for policy 1, policy_version 843118 (0.0010) [2023-12-27 11:28:48,674][20347] Updated weights for policy 0, policy_version 843813 (0.0008) [2023-12-27 11:28:48,683][20381] Updated weights for policy 1, policy_version 843128 (0.0010) [2023-12-27 11:28:48,730][20347] Updated weights for policy 0, policy_version 843823 (0.0010) [2023-12-27 11:28:49,389][20381] Updated weights for policy 1, policy_version 843138 (0.0010) [2023-12-27 11:28:49,421][20347] Updated weights for policy 0, policy_version 843833 (0.0008) [2023-12-27 11:28:49,443][20381] Updated weights for policy 1, policy_version 843148 (0.0010) [2023-12-27 11:28:49,486][20347] Updated weights for policy 0, policy_version 843843 (0.0008) [2023-12-27 11:28:49,501][20381] Updated weights for policy 1, policy_version 843158 (0.0010) [2023-12-27 11:28:49,549][20347] Updated weights for policy 0, policy_version 843853 (0.0007) [2023-12-27 11:28:50,215][20347] Updated weights for policy 0, policy_version 843863 (0.0010) [2023-12-27 11:28:50,239][20381] Updated weights for policy 1, policy_version 843168 (0.0007) [2023-12-27 11:28:50,288][20347] Updated weights for policy 0, policy_version 843873 (0.0011) [2023-12-27 11:28:50,301][20381] Updated weights for policy 1, policy_version 843178 (0.0007) [2023-12-27 11:28:50,357][20347] Updated weights for policy 0, policy_version 843883 (0.0010) [2023-12-27 11:28:50,364][20381] Updated weights for policy 1, policy_version 843188 (0.0007) [2023-12-27 11:28:51,053][20347] Updated weights for policy 0, policy_version 843893 (0.0009) [2023-12-27 11:28:51,072][20381] Updated weights for policy 1, policy_version 843198 (0.0009) [2023-12-27 11:28:51,115][20347] Updated weights for policy 0, policy_version 843903 (0.0007) [2023-12-27 11:28:51,133][20381] Updated weights for policy 1, policy_version 843208 (0.0007) [2023-12-27 11:28:51,176][20347] Updated weights for policy 0, policy_version 843913 (0.0007) [2023-12-27 11:28:51,201][20381] Updated weights for policy 1, policy_version 843218 (0.0008) [2023-12-27 11:28:51,904][20347] Updated weights for policy 0, policy_version 843923 (0.0008) [2023-12-27 11:28:51,965][20347] Updated weights for policy 0, policy_version 843933 (0.0007) [2023-12-27 11:28:52,014][20381] Updated weights for policy 1, policy_version 843228 (0.0008) [2023-12-27 11:28:52,033][20347] Updated weights for policy 0, policy_version 843943 (0.0007) [2023-12-27 11:28:52,077][20381] Updated weights for policy 1, policy_version 843238 (0.0008) [2023-12-27 11:28:52,144][20381] Updated weights for policy 1, policy_version 843248 (0.0007) [2023-12-27 11:28:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 431980544. Throughput: 0: 9897.0, 1: 10109.0. Samples: 431950392. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:52,363][19170] Avg episode reward: [(0, '253.640'), (1, '739.890')] [2023-12-27 11:28:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000843256_215900160.pth... [2023-12-27 11:28:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000843952_216080384.pth... [2023-12-27 11:28:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000842072_215597056.pth [2023-12-27 11:28:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000842768_215777280.pth [2023-12-27 11:28:52,710][20347] Updated weights for policy 0, policy_version 843953 (0.0007) [2023-12-27 11:28:52,769][20347] Updated weights for policy 0, policy_version 843963 (0.0008) [2023-12-27 11:28:52,830][20347] Updated weights for policy 0, policy_version 843973 (0.0007) [2023-12-27 11:28:52,842][20381] Updated weights for policy 1, policy_version 843258 (0.0008) [2023-12-27 11:28:52,886][20347] Updated weights for policy 0, policy_version 843983 (0.0007) [2023-12-27 11:28:52,901][20381] Updated weights for policy 1, policy_version 843268 (0.0008) [2023-12-27 11:28:52,984][20381] Updated weights for policy 1, policy_version 843278 (0.0008) [2023-12-27 11:28:53,051][20381] Updated weights for policy 1, policy_version 843288 (0.0007) [2023-12-27 11:28:53,604][20347] Updated weights for policy 0, policy_version 843993 (0.0006) [2023-12-27 11:28:53,662][20347] Updated weights for policy 0, policy_version 844003 (0.0006) [2023-12-27 11:28:53,690][20381] Updated weights for policy 1, policy_version 843298 (0.0007) [2023-12-27 11:28:53,718][20347] Updated weights for policy 0, policy_version 844013 (0.0008) [2023-12-27 11:28:53,748][20381] Updated weights for policy 1, policy_version 843308 (0.0007) [2023-12-27 11:28:53,813][20381] Updated weights for policy 1, policy_version 843318 (0.0007) [2023-12-27 11:28:54,396][20347] Updated weights for policy 0, policy_version 844023 (0.0006) [2023-12-27 11:28:54,450][20347] Updated weights for policy 0, policy_version 844033 (0.0008) [2023-12-27 11:28:54,512][20347] Updated weights for policy 0, policy_version 844043 (0.0006) [2023-12-27 11:28:54,521][20381] Updated weights for policy 1, policy_version 843328 (0.0006) [2023-12-27 11:28:54,580][20381] Updated weights for policy 1, policy_version 843338 (0.0006) [2023-12-27 11:28:54,640][20381] Updated weights for policy 1, policy_version 843348 (0.0007) [2023-12-27 11:28:55,275][20347] Updated weights for policy 0, policy_version 844053 (0.0006) [2023-12-27 11:28:55,295][20381] Updated weights for policy 1, policy_version 843358 (0.0007) [2023-12-27 11:28:55,327][20347] Updated weights for policy 0, policy_version 844063 (0.0005) [2023-12-27 11:28:55,358][20381] Updated weights for policy 1, policy_version 843368 (0.0007) [2023-12-27 11:28:55,387][20347] Updated weights for policy 0, policy_version 844073 (0.0008) [2023-12-27 11:28:55,415][20381] Updated weights for policy 1, policy_version 843378 (0.0007) [2023-12-27 11:28:56,028][20347] Updated weights for policy 0, policy_version 844083 (0.0010) [2023-12-27 11:28:56,087][20347] Updated weights for policy 0, policy_version 844093 (0.0007) [2023-12-27 11:28:56,105][20381] Updated weights for policy 1, policy_version 843388 (0.0007) [2023-12-27 11:28:56,147][20347] Updated weights for policy 0, policy_version 844103 (0.0007) [2023-12-27 11:28:56,170][20381] Updated weights for policy 1, policy_version 843398 (0.0008) [2023-12-27 11:28:56,230][20381] Updated weights for policy 1, policy_version 843408 (0.0008) [2023-12-27 11:28:56,782][20347] Updated weights for policy 0, policy_version 844113 (0.0006) [2023-12-27 11:28:56,848][20347] Updated weights for policy 0, policy_version 844123 (0.0006) [2023-12-27 11:28:56,907][20347] Updated weights for policy 0, policy_version 844133 (0.0009) [2023-12-27 11:28:56,927][20381] Updated weights for policy 1, policy_version 843418 (0.0007) [2023-12-27 11:28:56,962][20347] Updated weights for policy 0, policy_version 844143 (0.0007) [2023-12-27 11:28:56,981][20381] Updated weights for policy 1, policy_version 843428 (0.0006) [2023-12-27 11:28:57,034][20381] Updated weights for policy 1, policy_version 843438 (0.0007) [2023-12-27 11:28:57,081][20381] Updated weights for policy 1, policy_version 843448 (0.0007) [2023-12-27 11:28:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 19549.7). Total num frames: 432078848. Throughput: 0: 9962.9, 1: 10113.8. Samples: 432067924. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:28:57,363][19170] Avg episode reward: [(0, '310.220'), (1, '596.410')] [2023-12-27 11:28:57,651][20347] Updated weights for policy 0, policy_version 844153 (0.0008) [2023-12-27 11:28:57,707][20347] Updated weights for policy 0, policy_version 844163 (0.0007) [2023-12-27 11:28:57,748][20381] Updated weights for policy 1, policy_version 843458 (0.0007) [2023-12-27 11:28:57,763][20347] Updated weights for policy 0, policy_version 844173 (0.0007) [2023-12-27 11:28:57,798][20381] Updated weights for policy 1, policy_version 843468 (0.0008) [2023-12-27 11:28:57,856][20381] Updated weights for policy 1, policy_version 843478 (0.0008) [2023-12-27 11:28:58,389][20347] Updated weights for policy 0, policy_version 844183 (0.0009) [2023-12-27 11:28:58,458][20347] Updated weights for policy 0, policy_version 844193 (0.0010) [2023-12-27 11:28:58,520][20347] Updated weights for policy 0, policy_version 844203 (0.0010) [2023-12-27 11:28:58,649][20381] Updated weights for policy 1, policy_version 843488 (0.0008) [2023-12-27 11:28:58,709][20381] Updated weights for policy 1, policy_version 843498 (0.0008) [2023-12-27 11:28:58,771][20381] Updated weights for policy 1, policy_version 843508 (0.0009) [2023-12-27 11:28:59,331][20347] Updated weights for policy 0, policy_version 844213 (0.0009) [2023-12-27 11:28:59,396][20347] Updated weights for policy 0, policy_version 844223 (0.0008) [2023-12-27 11:28:59,449][20347] Updated weights for policy 0, policy_version 844233 (0.0007) [2023-12-27 11:28:59,621][20381] Updated weights for policy 1, policy_version 843518 (0.0007) [2023-12-27 11:28:59,680][20381] Updated weights for policy 1, policy_version 843528 (0.0008) [2023-12-27 11:28:59,732][20381] Updated weights for policy 1, policy_version 843538 (0.0007) [2023-12-27 11:29:00,159][20347] Updated weights for policy 0, policy_version 844243 (0.0006) [2023-12-27 11:29:00,213][20347] Updated weights for policy 0, policy_version 844253 (0.0006) [2023-12-27 11:29:00,271][20347] Updated weights for policy 0, policy_version 844263 (0.0005) [2023-12-27 11:29:00,456][20381] Updated weights for policy 1, policy_version 843548 (0.0008) [2023-12-27 11:29:00,506][20381] Updated weights for policy 1, policy_version 843558 (0.0008) [2023-12-27 11:29:00,555][20381] Updated weights for policy 1, policy_version 843568 (0.0008) [2023-12-27 11:29:00,898][20347] Updated weights for policy 0, policy_version 844273 (0.0006) [2023-12-27 11:29:00,974][20347] Updated weights for policy 0, policy_version 844283 (0.0009) [2023-12-27 11:29:01,039][20347] Updated weights for policy 0, policy_version 844293 (0.0009) [2023-12-27 11:29:01,096][20347] Updated weights for policy 0, policy_version 844303 (0.0008) [2023-12-27 11:29:01,235][20381] Updated weights for policy 1, policy_version 843578 (0.0007) [2023-12-27 11:29:01,295][20381] Updated weights for policy 1, policy_version 843588 (0.0007) [2023-12-27 11:29:01,360][20381] Updated weights for policy 1, policy_version 843598 (0.0006) [2023-12-27 11:29:01,423][20381] Updated weights for policy 1, policy_version 843608 (0.0008) [2023-12-27 11:29:01,865][20347] Updated weights for policy 0, policy_version 844313 (0.0008) [2023-12-27 11:29:01,928][20347] Updated weights for policy 0, policy_version 844323 (0.0006) [2023-12-27 11:29:01,991][20347] Updated weights for policy 0, policy_version 844333 (0.0008) [2023-12-27 11:29:02,134][20381] Updated weights for policy 1, policy_version 843618 (0.0008) [2023-12-27 11:29:02,187][20381] Updated weights for policy 1, policy_version 843628 (0.0008) [2023-12-27 11:29:02,237][20381] Updated weights for policy 1, policy_version 843638 (0.0008) [2023-12-27 11:29:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 19549.7). Total num frames: 432177152. Throughput: 0: 10023.5, 1: 10059.5. Samples: 432184624. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:02,363][19170] Avg episode reward: [(0, '175.530'), (1, '613.490')] [2023-12-27 11:29:02,614][20347] Updated weights for policy 0, policy_version 844343 (0.0008) [2023-12-27 11:29:02,666][20347] Updated weights for policy 0, policy_version 844353 (0.0008) [2023-12-27 11:29:02,717][20347] Updated weights for policy 0, policy_version 844363 (0.0006) [2023-12-27 11:29:02,921][20381] Updated weights for policy 1, policy_version 843648 (0.0008) [2023-12-27 11:29:02,973][20381] Updated weights for policy 1, policy_version 843658 (0.0008) [2023-12-27 11:29:03,019][20381] Updated weights for policy 1, policy_version 843668 (0.0008) [2023-12-27 11:29:03,335][20347] Updated weights for policy 0, policy_version 844373 (0.0008) [2023-12-27 11:29:03,388][20347] Updated weights for policy 0, policy_version 844383 (0.0005) [2023-12-27 11:29:03,440][20347] Updated weights for policy 0, policy_version 844393 (0.0007) [2023-12-27 11:29:03,667][20381] Updated weights for policy 1, policy_version 843678 (0.0006) [2023-12-27 11:29:03,715][20381] Updated weights for policy 1, policy_version 843688 (0.0005) [2023-12-27 11:29:03,762][20381] Updated weights for policy 1, policy_version 843698 (0.0008) [2023-12-27 11:29:04,054][20347] Updated weights for policy 0, policy_version 844403 (0.0010) [2023-12-27 11:29:04,118][20347] Updated weights for policy 0, policy_version 844413 (0.0011) [2023-12-27 11:29:04,177][20347] Updated weights for policy 0, policy_version 844423 (0.0011) [2023-12-27 11:29:04,411][20381] Updated weights for policy 1, policy_version 843708 (0.0009) [2023-12-27 11:29:04,468][20381] Updated weights for policy 1, policy_version 843718 (0.0010) [2023-12-27 11:29:04,519][20381] Updated weights for policy 1, policy_version 843728 (0.0010) [2023-12-27 11:29:04,900][20347] Updated weights for policy 0, policy_version 844433 (0.0010) [2023-12-27 11:29:04,955][20347] Updated weights for policy 0, policy_version 844443 (0.0010) [2023-12-27 11:29:05,016][20347] Updated weights for policy 0, policy_version 844453 (0.0010) [2023-12-27 11:29:05,078][20347] Updated weights for policy 0, policy_version 844463 (0.0010) [2023-12-27 11:29:05,222][20381] Updated weights for policy 1, policy_version 843738 (0.0010) [2023-12-27 11:29:05,269][20381] Updated weights for policy 1, policy_version 843748 (0.0010) [2023-12-27 11:29:05,328][20381] Updated weights for policy 1, policy_version 843758 (0.0010) [2023-12-27 11:29:05,379][20381] Updated weights for policy 1, policy_version 843768 (0.0010) [2023-12-27 11:29:05,754][20347] Updated weights for policy 0, policy_version 844473 (0.0010) [2023-12-27 11:29:05,813][20347] Updated weights for policy 0, policy_version 844483 (0.0010) [2023-12-27 11:29:05,874][20347] Updated weights for policy 0, policy_version 844493 (0.0010) [2023-12-27 11:29:06,076][20381] Updated weights for policy 1, policy_version 843778 (0.0008) [2023-12-27 11:29:06,138][20381] Updated weights for policy 1, policy_version 843788 (0.0007) [2023-12-27 11:29:06,191][20381] Updated weights for policy 1, policy_version 843798 (0.0009) [2023-12-27 11:29:06,542][20347] Updated weights for policy 0, policy_version 844503 (0.0007) [2023-12-27 11:29:06,597][20347] Updated weights for policy 0, policy_version 844513 (0.0006) [2023-12-27 11:29:06,661][20347] Updated weights for policy 0, policy_version 844523 (0.0006) [2023-12-27 11:29:06,891][20381] Updated weights for policy 1, policy_version 843808 (0.0006) [2023-12-27 11:29:06,953][20381] Updated weights for policy 1, policy_version 843818 (0.0006) [2023-12-27 11:29:07,015][20381] Updated weights for policy 1, policy_version 843828 (0.0006) [2023-12-27 11:29:07,307][20347] Updated weights for policy 0, policy_version 844533 (0.0008) [2023-12-27 11:29:07,361][20347] Updated weights for policy 0, policy_version 844543 (0.0010) [2023-12-27 11:29:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19933.9, 300 sec: 19549.7). Total num frames: 432275456. Throughput: 0: 10069.8, 1: 10116.3. Samples: 432248272. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:07,363][19170] Avg episode reward: [(0, '207.690'), (1, '688.460')] [2023-12-27 11:29:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000843832_216047616.pth... [2023-12-27 11:29:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000842680_215752704.pth [2023-12-27 11:29:07,414][20347] Updated weights for policy 0, policy_version 844553 (0.0010) [2023-12-27 11:29:07,456][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000844560_216236032.pth... [2023-12-27 11:29:07,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000843344_215924736.pth [2023-12-27 11:29:07,616][20381] Updated weights for policy 1, policy_version 843838 (0.0009) [2023-12-27 11:29:07,672][20381] Updated weights for policy 1, policy_version 843848 (0.0011) [2023-12-27 11:29:07,731][20381] Updated weights for policy 1, policy_version 843858 (0.0011) [2023-12-27 11:29:08,041][20347] Updated weights for policy 0, policy_version 844563 (0.0006) [2023-12-27 11:29:08,093][20347] Updated weights for policy 0, policy_version 844573 (0.0010) [2023-12-27 11:29:08,178][20347] Updated weights for policy 0, policy_version 844583 (0.0010) [2023-12-27 11:29:08,453][20381] Updated weights for policy 1, policy_version 843868 (0.0010) [2023-12-27 11:29:08,511][20381] Updated weights for policy 1, policy_version 843878 (0.0007) [2023-12-27 11:29:08,570][20381] Updated weights for policy 1, policy_version 843888 (0.0007) [2023-12-27 11:29:08,826][20347] Updated weights for policy 0, policy_version 844593 (0.0009) [2023-12-27 11:29:08,902][20347] Updated weights for policy 0, policy_version 844603 (0.0009) [2023-12-27 11:29:08,974][20347] Updated weights for policy 0, policy_version 844613 (0.0009) [2023-12-27 11:29:09,035][20347] Updated weights for policy 0, policy_version 844623 (0.0009) [2023-12-27 11:29:09,263][20381] Updated weights for policy 1, policy_version 843898 (0.0008) [2023-12-27 11:29:09,322][20381] Updated weights for policy 1, policy_version 843908 (0.0011) [2023-12-27 11:29:09,389][20381] Updated weights for policy 1, policy_version 843918 (0.0010) [2023-12-27 11:29:09,455][20381] Updated weights for policy 1, policy_version 843928 (0.0006) [2023-12-27 11:29:09,736][20347] Updated weights for policy 0, policy_version 844633 (0.0008) [2023-12-27 11:29:09,792][20347] Updated weights for policy 0, policy_version 844643 (0.0007) [2023-12-27 11:29:09,856][20347] Updated weights for policy 0, policy_version 844653 (0.0008) [2023-12-27 11:29:10,140][20381] Updated weights for policy 1, policy_version 843938 (0.0011) [2023-12-27 11:29:10,200][20381] Updated weights for policy 1, policy_version 843948 (0.0010) [2023-12-27 11:29:10,254][20381] Updated weights for policy 1, policy_version 843958 (0.0011) [2023-12-27 11:29:10,543][20347] Updated weights for policy 0, policy_version 844663 (0.0006) [2023-12-27 11:29:10,602][20347] Updated weights for policy 0, policy_version 844673 (0.0006) [2023-12-27 11:29:10,652][20347] Updated weights for policy 0, policy_version 844683 (0.0005) [2023-12-27 11:29:10,951][20381] Updated weights for policy 1, policy_version 843968 (0.0010) [2023-12-27 11:29:11,009][20381] Updated weights for policy 1, policy_version 843978 (0.0007) [2023-12-27 11:29:11,080][20381] Updated weights for policy 1, policy_version 843988 (0.0011) [2023-12-27 11:29:11,236][20347] Updated weights for policy 0, policy_version 844693 (0.0006) [2023-12-27 11:29:11,303][20347] Updated weights for policy 0, policy_version 844703 (0.0008) [2023-12-27 11:29:11,357][20347] Updated weights for policy 0, policy_version 844713 (0.0009) [2023-12-27 11:29:11,738][20381] Updated weights for policy 1, policy_version 843998 (0.0008) [2023-12-27 11:29:11,798][20381] Updated weights for policy 1, policy_version 844008 (0.0009) [2023-12-27 11:29:11,855][20381] Updated weights for policy 1, policy_version 844018 (0.0006) [2023-12-27 11:29:12,094][20347] Updated weights for policy 0, policy_version 844723 (0.0008) [2023-12-27 11:29:12,155][20347] Updated weights for policy 0, policy_version 844733 (0.0008) [2023-12-27 11:29:12,213][20347] Updated weights for policy 0, policy_version 844743 (0.0008) [2023-12-27 11:29:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20207.1, 300 sec: 19577.5). Total num frames: 432381952. Throughput: 0: 10127.3, 1: 10064.4. Samples: 432370480. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:12,363][19170] Avg episode reward: [(0, '347.570'), (1, '667.600')] [2023-12-27 11:29:12,554][20381] Updated weights for policy 1, policy_version 844028 (0.0008) [2023-12-27 11:29:12,604][20381] Updated weights for policy 1, policy_version 844038 (0.0008) [2023-12-27 11:29:12,657][20381] Updated weights for policy 1, policy_version 844048 (0.0008) [2023-12-27 11:29:12,927][20347] Updated weights for policy 0, policy_version 844753 (0.0008) [2023-12-27 11:29:12,987][20347] Updated weights for policy 0, policy_version 844763 (0.0007) [2023-12-27 11:29:13,041][20347] Updated weights for policy 0, policy_version 844773 (0.0008) [2023-12-27 11:29:13,102][20347] Updated weights for policy 0, policy_version 844783 (0.0008) [2023-12-27 11:29:13,304][20381] Updated weights for policy 1, policy_version 844058 (0.0007) [2023-12-27 11:29:13,359][20381] Updated weights for policy 1, policy_version 844068 (0.0006) [2023-12-27 11:29:13,436][20381] Updated weights for policy 1, policy_version 844078 (0.0006) [2023-12-27 11:29:13,491][20381] Updated weights for policy 1, policy_version 844088 (0.0007) [2023-12-27 11:29:13,839][20347] Updated weights for policy 0, policy_version 844793 (0.0006) [2023-12-27 11:29:13,885][20347] Updated weights for policy 0, policy_version 844803 (0.0005) [2023-12-27 11:29:13,934][20347] Updated weights for policy 0, policy_version 844813 (0.0005) [2023-12-27 11:29:14,126][20381] Updated weights for policy 1, policy_version 844098 (0.0010) [2023-12-27 11:29:14,171][20381] Updated weights for policy 1, policy_version 844108 (0.0010) [2023-12-27 11:29:14,224][20381] Updated weights for policy 1, policy_version 844118 (0.0010) [2023-12-27 11:29:14,523][20347] Updated weights for policy 0, policy_version 844823 (0.0007) [2023-12-27 11:29:14,573][20347] Updated weights for policy 0, policy_version 844833 (0.0007) [2023-12-27 11:29:14,626][20347] Updated weights for policy 0, policy_version 844843 (0.0007) [2023-12-27 11:29:14,917][20381] Updated weights for policy 1, policy_version 844128 (0.0011) [2023-12-27 11:29:14,977][20381] Updated weights for policy 1, policy_version 844138 (0.0011) [2023-12-27 11:29:15,040][20381] Updated weights for policy 1, policy_version 844148 (0.0011) [2023-12-27 11:29:15,306][20347] Updated weights for policy 0, policy_version 844853 (0.0006) [2023-12-27 11:29:15,371][20347] Updated weights for policy 0, policy_version 844863 (0.0007) [2023-12-27 11:29:15,424][20347] Updated weights for policy 0, policy_version 844873 (0.0007) [2023-12-27 11:29:15,719][20381] Updated weights for policy 1, policy_version 844158 (0.0011) [2023-12-27 11:29:15,777][20381] Updated weights for policy 1, policy_version 844168 (0.0010) [2023-12-27 11:29:15,833][20381] Updated weights for policy 1, policy_version 844178 (0.0009) [2023-12-27 11:29:16,098][20347] Updated weights for policy 0, policy_version 844883 (0.0007) [2023-12-27 11:29:16,157][20347] Updated weights for policy 0, policy_version 844893 (0.0006) [2023-12-27 11:29:16,207][20347] Updated weights for policy 0, policy_version 844903 (0.0008) [2023-12-27 11:29:16,455][20381] Updated weights for policy 1, policy_version 844188 (0.0005) [2023-12-27 11:29:16,509][20381] Updated weights for policy 1, policy_version 844198 (0.0005) [2023-12-27 11:29:16,575][20381] Updated weights for policy 1, policy_version 844208 (0.0006) [2023-12-27 11:29:16,936][20347] Updated weights for policy 0, policy_version 844913 (0.0008) [2023-12-27 11:29:16,982][20347] Updated weights for policy 0, policy_version 844923 (0.0007) [2023-12-27 11:29:17,041][20347] Updated weights for policy 0, policy_version 844933 (0.0008) [2023-12-27 11:29:17,090][20347] Updated weights for policy 0, policy_version 844943 (0.0007) [2023-12-27 11:29:17,120][20381] Updated weights for policy 1, policy_version 844218 (0.0006) [2023-12-27 11:29:17,166][20381] Updated weights for policy 1, policy_version 844228 (0.0008) [2023-12-27 11:29:17,218][20381] Updated weights for policy 1, policy_version 844238 (0.0008) [2023-12-27 11:29:17,271][20381] Updated weights for policy 1, policy_version 844248 (0.0008) [2023-12-27 11:29:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 19605.3). Total num frames: 432488448. Throughput: 0: 10143.6, 1: 10107.2. Samples: 432493744. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:17,363][19170] Avg episode reward: [(0, '254.850'), (1, '724.830')] [2023-12-27 11:29:17,787][20347] Updated weights for policy 0, policy_version 844953 (0.0007) [2023-12-27 11:29:17,841][20347] Updated weights for policy 0, policy_version 844963 (0.0007) [2023-12-27 11:29:17,896][20347] Updated weights for policy 0, policy_version 844973 (0.0008) [2023-12-27 11:29:18,000][20381] Updated weights for policy 1, policy_version 844258 (0.0008) [2023-12-27 11:29:18,059][20381] Updated weights for policy 1, policy_version 844268 (0.0008) [2023-12-27 11:29:18,116][20381] Updated weights for policy 1, policy_version 844278 (0.0008) [2023-12-27 11:29:18,563][20347] Updated weights for policy 0, policy_version 844983 (0.0009) [2023-12-27 11:29:18,630][20347] Updated weights for policy 0, policy_version 844993 (0.0009) [2023-12-27 11:29:18,696][20347] Updated weights for policy 0, policy_version 845003 (0.0008) [2023-12-27 11:29:18,843][20381] Updated weights for policy 1, policy_version 844288 (0.0008) [2023-12-27 11:29:18,896][20381] Updated weights for policy 1, policy_version 844298 (0.0007) [2023-12-27 11:29:18,952][20381] Updated weights for policy 1, policy_version 844308 (0.0006) [2023-12-27 11:29:19,360][20347] Updated weights for policy 0, policy_version 845013 (0.0012) [2023-12-27 11:29:19,425][20347] Updated weights for policy 0, policy_version 845023 (0.0009) [2023-12-27 11:29:19,490][20347] Updated weights for policy 0, policy_version 845033 (0.0009) [2023-12-27 11:29:19,640][20381] Updated weights for policy 1, policy_version 844318 (0.0008) [2023-12-27 11:29:19,705][20381] Updated weights for policy 1, policy_version 844328 (0.0008) [2023-12-27 11:29:19,766][20381] Updated weights for policy 1, policy_version 844338 (0.0008) [2023-12-27 11:29:20,223][20347] Updated weights for policy 0, policy_version 845043 (0.0008) [2023-12-27 11:29:20,292][20347] Updated weights for policy 0, policy_version 845053 (0.0009) [2023-12-27 11:29:20,359][20347] Updated weights for policy 0, policy_version 845063 (0.0010) [2023-12-27 11:29:20,512][20381] Updated weights for policy 1, policy_version 844348 (0.0007) [2023-12-27 11:29:20,577][20381] Updated weights for policy 1, policy_version 844358 (0.0008) [2023-12-27 11:29:20,633][20381] Updated weights for policy 1, policy_version 844368 (0.0009) [2023-12-27 11:29:21,079][20347] Updated weights for policy 0, policy_version 845073 (0.0008) [2023-12-27 11:29:21,138][20347] Updated weights for policy 0, policy_version 845083 (0.0009) [2023-12-27 11:29:21,203][20347] Updated weights for policy 0, policy_version 845093 (0.0007) [2023-12-27 11:29:21,267][20347] Updated weights for policy 0, policy_version 845103 (0.0008) [2023-12-27 11:29:21,362][20381] Updated weights for policy 1, policy_version 844378 (0.0009) [2023-12-27 11:29:21,439][20381] Updated weights for policy 1, policy_version 844388 (0.0008) [2023-12-27 11:29:21,502][20381] Updated weights for policy 1, policy_version 844398 (0.0008) [2023-12-27 11:29:21,564][20381] Updated weights for policy 1, policy_version 844408 (0.0008) [2023-12-27 11:29:22,062][20347] Updated weights for policy 0, policy_version 845113 (0.0007) [2023-12-27 11:29:22,125][20347] Updated weights for policy 0, policy_version 845123 (0.0008) [2023-12-27 11:29:22,187][20347] Updated weights for policy 0, policy_version 845133 (0.0008) [2023-12-27 11:29:22,364][19170] Fps is (10 sec: 19658.6, 60 sec: 20206.6, 300 sec: 19577.4). Total num frames: 432578560. Throughput: 0: 10150.0, 1: 10046.0. Samples: 432552936. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:22,364][19170] Avg episode reward: [(0, '173.700'), (1, '663.520')] [2023-12-27 11:29:22,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000845136_216383488.pth... [2023-12-27 11:29:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000843952_216080384.pth [2023-12-27 11:29:22,381][20381] Updated weights for policy 1, policy_version 844418 (0.0009) [2023-12-27 11:29:22,457][20381] Updated weights for policy 1, policy_version 844428 (0.0008) [2023-12-27 11:29:22,524][20381] Updated weights for policy 1, policy_version 844438 (0.0008) [2023-12-27 11:29:22,539][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000844440_216203264.pth... [2023-12-27 11:29:22,543][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000843256_215900160.pth [2023-12-27 11:29:22,982][20347] Updated weights for policy 0, policy_version 845143 (0.0008) [2023-12-27 11:29:23,044][20347] Updated weights for policy 0, policy_version 845153 (0.0007) [2023-12-27 11:29:23,106][20347] Updated weights for policy 0, policy_version 845163 (0.0008) [2023-12-27 11:29:23,273][20381] Updated weights for policy 1, policy_version 844448 (0.0008) [2023-12-27 11:29:23,331][20381] Updated weights for policy 1, policy_version 844458 (0.0008) [2023-12-27 11:29:23,389][20381] Updated weights for policy 1, policy_version 844468 (0.0008) [2023-12-27 11:29:23,792][20347] Updated weights for policy 0, policy_version 845173 (0.0006) [2023-12-27 11:29:23,855][20347] Updated weights for policy 0, policy_version 845183 (0.0006) [2023-12-27 11:29:23,911][20347] Updated weights for policy 0, policy_version 845193 (0.0006) [2023-12-27 11:29:24,178][20381] Updated weights for policy 1, policy_version 844478 (0.0008) [2023-12-27 11:29:24,235][20381] Updated weights for policy 1, policy_version 844488 (0.0008) [2023-12-27 11:29:24,293][20381] Updated weights for policy 1, policy_version 844498 (0.0008) [2023-12-27 11:29:24,581][20347] Updated weights for policy 0, policy_version 845203 (0.0007) [2023-12-27 11:29:24,643][20347] Updated weights for policy 0, policy_version 845213 (0.0008) [2023-12-27 11:29:24,699][20347] Updated weights for policy 0, policy_version 845223 (0.0007) [2023-12-27 11:29:25,001][20381] Updated weights for policy 1, policy_version 844508 (0.0006) [2023-12-27 11:29:25,062][20381] Updated weights for policy 1, policy_version 844518 (0.0007) [2023-12-27 11:29:25,121][20381] Updated weights for policy 1, policy_version 844528 (0.0008) [2023-12-27 11:29:25,454][20347] Updated weights for policy 0, policy_version 845233 (0.0008) [2023-12-27 11:29:25,511][20347] Updated weights for policy 0, policy_version 845243 (0.0007) [2023-12-27 11:29:25,565][20347] Updated weights for policy 0, policy_version 845253 (0.0008) [2023-12-27 11:29:25,622][20347] Updated weights for policy 0, policy_version 845263 (0.0008) [2023-12-27 11:29:25,840][20381] Updated weights for policy 1, policy_version 844538 (0.0008) [2023-12-27 11:29:25,895][20381] Updated weights for policy 1, policy_version 844548 (0.0008) [2023-12-27 11:29:25,946][20381] Updated weights for policy 1, policy_version 844558 (0.0008) [2023-12-27 11:29:25,997][20381] Updated weights for policy 1, policy_version 844568 (0.0007) [2023-12-27 11:29:26,391][20347] Updated weights for policy 0, policy_version 845273 (0.0007) [2023-12-27 11:29:26,453][20347] Updated weights for policy 0, policy_version 845283 (0.0007) [2023-12-27 11:29:26,510][20347] Updated weights for policy 0, policy_version 845293 (0.0007) [2023-12-27 11:29:26,744][20381] Updated weights for policy 1, policy_version 844578 (0.0009) [2023-12-27 11:29:26,801][20381] Updated weights for policy 1, policy_version 844588 (0.0008) [2023-12-27 11:29:26,859][20381] Updated weights for policy 1, policy_version 844598 (0.0008) [2023-12-27 11:29:27,192][20347] Updated weights for policy 0, policy_version 845303 (0.0006) [2023-12-27 11:29:27,251][20347] Updated weights for policy 0, policy_version 845313 (0.0007) [2023-12-27 11:29:27,313][20347] Updated weights for policy 0, policy_version 845323 (0.0008) [2023-12-27 11:29:27,362][19170] Fps is (10 sec: 18841.3, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 432676864. Throughput: 0: 10055.6, 1: 9934.7. Samples: 432665288. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:27,363][19170] Avg episode reward: [(0, '92.190'), (1, '615.820')] [2023-12-27 11:29:27,582][20381] Updated weights for policy 1, policy_version 844608 (0.0008) [2023-12-27 11:29:27,640][20381] Updated weights for policy 1, policy_version 844618 (0.0007) [2023-12-27 11:29:27,699][20381] Updated weights for policy 1, policy_version 844628 (0.0008) [2023-12-27 11:29:28,011][20347] Updated weights for policy 0, policy_version 845333 (0.0008) [2023-12-27 11:29:28,064][20347] Updated weights for policy 0, policy_version 845343 (0.0009) [2023-12-27 11:29:28,124][20347] Updated weights for policy 0, policy_version 845353 (0.0008) [2023-12-27 11:29:28,410][20381] Updated weights for policy 1, policy_version 844638 (0.0008) [2023-12-27 11:29:28,467][20381] Updated weights for policy 1, policy_version 844648 (0.0008) [2023-12-27 11:29:28,529][20381] Updated weights for policy 1, policy_version 844658 (0.0008) [2023-12-27 11:29:28,855][20347] Updated weights for policy 0, policy_version 845363 (0.0008) [2023-12-27 11:29:28,917][20347] Updated weights for policy 0, policy_version 845373 (0.0007) [2023-12-27 11:29:28,977][20347] Updated weights for policy 0, policy_version 845383 (0.0006) [2023-12-27 11:29:29,302][20381] Updated weights for policy 1, policy_version 844668 (0.0008) [2023-12-27 11:29:29,371][20381] Updated weights for policy 1, policy_version 844678 (0.0008) [2023-12-27 11:29:29,439][20381] Updated weights for policy 1, policy_version 844688 (0.0008) [2023-12-27 11:29:29,663][20347] Updated weights for policy 0, policy_version 845393 (0.0006) [2023-12-27 11:29:29,719][20347] Updated weights for policy 0, policy_version 845403 (0.0007) [2023-12-27 11:29:29,779][20347] Updated weights for policy 0, policy_version 845413 (0.0006) [2023-12-27 11:29:29,846][20347] Updated weights for policy 0, policy_version 845423 (0.0007) [2023-12-27 11:29:30,199][20381] Updated weights for policy 1, policy_version 844698 (0.0008) [2023-12-27 11:29:30,262][20381] Updated weights for policy 1, policy_version 844708 (0.0008) [2023-12-27 11:29:30,331][20381] Updated weights for policy 1, policy_version 844718 (0.0007) [2023-12-27 11:29:30,387][20381] Updated weights for policy 1, policy_version 844728 (0.0007) [2023-12-27 11:29:30,533][20347] Updated weights for policy 0, policy_version 845433 (0.0008) [2023-12-27 11:29:30,581][20347] Updated weights for policy 0, policy_version 845443 (0.0007) [2023-12-27 11:29:30,639][20347] Updated weights for policy 0, policy_version 845453 (0.0008) [2023-12-27 11:29:31,090][20381] Updated weights for policy 1, policy_version 844738 (0.0008) [2023-12-27 11:29:31,155][20381] Updated weights for policy 1, policy_version 844748 (0.0008) [2023-12-27 11:29:31,223][20381] Updated weights for policy 1, policy_version 844758 (0.0007) [2023-12-27 11:29:31,337][20347] Updated weights for policy 0, policy_version 845463 (0.0007) [2023-12-27 11:29:31,408][20347] Updated weights for policy 0, policy_version 845473 (0.0007) [2023-12-27 11:29:31,470][20347] Updated weights for policy 0, policy_version 845483 (0.0006) [2023-12-27 11:29:31,999][20381] Updated weights for policy 1, policy_version 844768 (0.0008) [2023-12-27 11:29:32,052][20381] Updated weights for policy 1, policy_version 844778 (0.0007) [2023-12-27 11:29:32,103][20347] Updated weights for policy 0, policy_version 845493 (0.0007) [2023-12-27 11:29:32,118][20381] Updated weights for policy 1, policy_version 844788 (0.0007) [2023-12-27 11:29:32,160][20347] Updated weights for policy 0, policy_version 845503 (0.0008) [2023-12-27 11:29:32,224][20347] Updated weights for policy 0, policy_version 845513 (0.0008) [2023-12-27 11:29:32,362][19170] Fps is (10 sec: 19663.0, 60 sec: 20070.4, 300 sec: 19577.5). Total num frames: 432775168. Throughput: 0: 10034.4, 1: 9866.3. Samples: 432781320. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:32,363][19170] Avg episode reward: [(0, '100.390'), (1, '655.450')] [2023-12-27 11:29:32,857][20381] Updated weights for policy 1, policy_version 844798 (0.0007) [2023-12-27 11:29:32,907][20381] Updated weights for policy 1, policy_version 844808 (0.0008) [2023-12-27 11:29:32,952][20347] Updated weights for policy 0, policy_version 845523 (0.0007) [2023-12-27 11:29:32,962][20381] Updated weights for policy 1, policy_version 844818 (0.0007) [2023-12-27 11:29:33,011][20347] Updated weights for policy 0, policy_version 845533 (0.0009) [2023-12-27 11:29:33,064][20347] Updated weights for policy 0, policy_version 845543 (0.0009) [2023-12-27 11:29:33,826][20381] Updated weights for policy 1, policy_version 844828 (0.0008) [2023-12-27 11:29:33,880][20381] Updated weights for policy 1, policy_version 844838 (0.0009) [2023-12-27 11:29:33,926][20347] Updated weights for policy 0, policy_version 845553 (0.0009) [2023-12-27 11:29:33,938][20381] Updated weights for policy 1, policy_version 844848 (0.0009) [2023-12-27 11:29:33,980][20347] Updated weights for policy 0, policy_version 845563 (0.0009) [2023-12-27 11:29:34,033][20347] Updated weights for policy 0, policy_version 845573 (0.0009) [2023-12-27 11:29:34,086][20347] Updated weights for policy 0, policy_version 845583 (0.0009) [2023-12-27 11:29:34,866][20381] Updated weights for policy 1, policy_version 844858 (0.0009) [2023-12-27 11:29:34,916][20381] Updated weights for policy 1, policy_version 844868 (0.0009) [2023-12-27 11:29:34,970][20381] Updated weights for policy 1, policy_version 844878 (0.0009) [2023-12-27 11:29:35,007][20347] Updated weights for policy 0, policy_version 845593 (0.0009) [2023-12-27 11:29:35,026][20381] Updated weights for policy 1, policy_version 844888 (0.0009) [2023-12-27 11:29:35,070][20347] Updated weights for policy 0, policy_version 845604 (0.0009) [2023-12-27 11:29:35,119][20347] Updated weights for policy 0, policy_version 845614 (0.0009) [2023-12-27 11:29:35,925][20381] Updated weights for policy 1, policy_version 844898 (0.0009) [2023-12-27 11:29:35,987][20381] Updated weights for policy 1, policy_version 844908 (0.0009) [2023-12-27 11:29:36,012][20347] Updated weights for policy 0, policy_version 845624 (0.0010) [2023-12-27 11:29:36,044][20381] Updated weights for policy 1, policy_version 844918 (0.0006) [2023-12-27 11:29:36,069][20347] Updated weights for policy 0, policy_version 845634 (0.0010) [2023-12-27 11:29:36,146][20347] Updated weights for policy 0, policy_version 845646 (0.0008) [2023-12-27 11:29:36,801][20381] Updated weights for policy 1, policy_version 844928 (0.0007) [2023-12-27 11:29:36,864][20381] Updated weights for policy 1, policy_version 844938 (0.0007) [2023-12-27 11:29:36,878][20347] Updated weights for policy 0, policy_version 845656 (0.0008) [2023-12-27 11:29:36,922][20381] Updated weights for policy 1, policy_version 844948 (0.0006) [2023-12-27 11:29:36,938][20347] Updated weights for policy 0, policy_version 845666 (0.0006) [2023-12-27 11:29:37,003][20347] Updated weights for policy 0, policy_version 845676 (0.0008) [2023-12-27 11:29:37,363][19170] Fps is (10 sec: 18022.1, 60 sec: 19797.3, 300 sec: 19521.9). Total num frames: 432857088. Throughput: 0: 9873.9, 1: 9717.7. Samples: 432832016. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:37,364][19170] Avg episode reward: [(0, '114.510'), (1, '690.960')] [2023-12-27 11:29:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000844952_216334336.pth... [2023-12-27 11:29:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000845680_216522752.pth... [2023-12-27 11:29:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000844560_216236032.pth [2023-12-27 11:29:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000843832_216047616.pth [2023-12-27 11:29:37,635][20347] Updated weights for policy 0, policy_version 845686 (0.0007) [2023-12-27 11:29:37,669][20381] Updated weights for policy 1, policy_version 844958 (0.0007) [2023-12-27 11:29:37,692][20347] Updated weights for policy 0, policy_version 845696 (0.0006) [2023-12-27 11:29:37,725][20381] Updated weights for policy 1, policy_version 844968 (0.0008) [2023-12-27 11:29:37,748][20347] Updated weights for policy 0, policy_version 845706 (0.0006) [2023-12-27 11:29:37,781][20381] Updated weights for policy 1, policy_version 844978 (0.0007) [2023-12-27 11:29:38,488][20347] Updated weights for policy 0, policy_version 845716 (0.0007) [2023-12-27 11:29:38,530][20381] Updated weights for policy 1, policy_version 844988 (0.0007) [2023-12-27 11:29:38,552][20347] Updated weights for policy 0, policy_version 845726 (0.0007) [2023-12-27 11:29:38,590][20381] Updated weights for policy 1, policy_version 844998 (0.0007) [2023-12-27 11:29:38,611][20347] Updated weights for policy 0, policy_version 845736 (0.0006) [2023-12-27 11:29:38,652][20381] Updated weights for policy 1, policy_version 845008 (0.0007) [2023-12-27 11:29:39,331][20347] Updated weights for policy 0, policy_version 845746 (0.0007) [2023-12-27 11:29:39,397][20347] Updated weights for policy 0, policy_version 845756 (0.0008) [2023-12-27 11:29:39,403][20381] Updated weights for policy 1, policy_version 845018 (0.0009) [2023-12-27 11:29:39,453][20347] Updated weights for policy 0, policy_version 845766 (0.0008) [2023-12-27 11:29:39,459][20381] Updated weights for policy 1, policy_version 845028 (0.0008) [2023-12-27 11:29:39,507][20347] Updated weights for policy 0, policy_version 845776 (0.0007) [2023-12-27 11:29:39,525][20381] Updated weights for policy 1, policy_version 845038 (0.0008) [2023-12-27 11:29:39,589][20381] Updated weights for policy 1, policy_version 845048 (0.0008) [2023-12-27 11:29:40,254][20347] Updated weights for policy 0, policy_version 845786 (0.0008) [2023-12-27 11:29:40,318][20347] Updated weights for policy 0, policy_version 845796 (0.0008) [2023-12-27 11:29:40,385][20347] Updated weights for policy 0, policy_version 845806 (0.0008) [2023-12-27 11:29:40,398][20381] Updated weights for policy 1, policy_version 845058 (0.0009) [2023-12-27 11:29:40,456][20381] Updated weights for policy 1, policy_version 845068 (0.0007) [2023-12-27 11:29:40,513][20381] Updated weights for policy 1, policy_version 845078 (0.0009) [2023-12-27 11:29:41,091][20347] Updated weights for policy 0, policy_version 845816 (0.0007) [2023-12-27 11:29:41,149][20347] Updated weights for policy 0, policy_version 845826 (0.0008) [2023-12-27 11:29:41,206][20347] Updated weights for policy 0, policy_version 845836 (0.0007) [2023-12-27 11:29:41,248][20381] Updated weights for policy 1, policy_version 845088 (0.0008) [2023-12-27 11:29:41,304][20381] Updated weights for policy 1, policy_version 845098 (0.0008) [2023-12-27 11:29:41,363][20381] Updated weights for policy 1, policy_version 845108 (0.0008) [2023-12-27 11:29:41,966][20347] Updated weights for policy 0, policy_version 845846 (0.0007) [2023-12-27 11:29:42,018][20347] Updated weights for policy 0, policy_version 845856 (0.0008) [2023-12-27 11:29:42,086][20347] Updated weights for policy 0, policy_version 845866 (0.0009) [2023-12-27 11:29:42,132][20381] Updated weights for policy 1, policy_version 845118 (0.0008) [2023-12-27 11:29:42,192][20381] Updated weights for policy 1, policy_version 845128 (0.0008) [2023-12-27 11:29:42,253][20381] Updated weights for policy 1, policy_version 845138 (0.0008) [2023-12-27 11:29:42,363][19170] Fps is (10 sec: 18022.5, 60 sec: 19660.8, 300 sec: 19549.7). Total num frames: 432955392. Throughput: 0: 9830.9, 1: 9631.7. Samples: 432943736. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:42,363][19170] Avg episode reward: [(0, '137.250'), (1, '711.700')] [2023-12-27 11:29:42,850][20347] Updated weights for policy 0, policy_version 845876 (0.0008) [2023-12-27 11:29:42,911][20347] Updated weights for policy 0, policy_version 845886 (0.0008) [2023-12-27 11:29:42,971][20347] Updated weights for policy 0, policy_version 845896 (0.0007) [2023-12-27 11:29:43,061][20381] Updated weights for policy 1, policy_version 845148 (0.0007) [2023-12-27 11:29:43,113][20381] Updated weights for policy 1, policy_version 845158 (0.0006) [2023-12-27 11:29:43,158][20381] Updated weights for policy 1, policy_version 845168 (0.0007) [2023-12-27 11:29:43,638][20347] Updated weights for policy 0, policy_version 845906 (0.0007) [2023-12-27 11:29:43,705][20347] Updated weights for policy 0, policy_version 845916 (0.0005) [2023-12-27 11:29:43,766][20347] Updated weights for policy 0, policy_version 845926 (0.0005) [2023-12-27 11:29:43,825][20347] Updated weights for policy 0, policy_version 845936 (0.0005) [2023-12-27 11:29:43,877][20381] Updated weights for policy 1, policy_version 845178 (0.0008) [2023-12-27 11:29:43,943][20381] Updated weights for policy 1, policy_version 845188 (0.0008) [2023-12-27 11:29:44,003][20381] Updated weights for policy 1, policy_version 845198 (0.0008) [2023-12-27 11:29:44,062][20381] Updated weights for policy 1, policy_version 845208 (0.0005) [2023-12-27 11:29:44,392][20347] Updated weights for policy 0, policy_version 845946 (0.0010) [2023-12-27 11:29:44,448][20347] Updated weights for policy 0, policy_version 845956 (0.0010) [2023-12-27 11:29:44,513][20347] Updated weights for policy 0, policy_version 845966 (0.0007) [2023-12-27 11:29:44,726][20381] Updated weights for policy 1, policy_version 845218 (0.0007) [2023-12-27 11:29:44,789][20381] Updated weights for policy 1, policy_version 845228 (0.0008) [2023-12-27 11:29:44,841][20381] Updated weights for policy 1, policy_version 845238 (0.0007) [2023-12-27 11:29:45,214][20347] Updated weights for policy 0, policy_version 845976 (0.0009) [2023-12-27 11:29:45,282][20347] Updated weights for policy 0, policy_version 845986 (0.0008) [2023-12-27 11:29:45,342][20347] Updated weights for policy 0, policy_version 845996 (0.0008) [2023-12-27 11:29:45,510][20381] Updated weights for policy 1, policy_version 845248 (0.0008) [2023-12-27 11:29:45,574][20381] Updated weights for policy 1, policy_version 845258 (0.0008) [2023-12-27 11:29:45,631][20381] Updated weights for policy 1, policy_version 845268 (0.0008) [2023-12-27 11:29:46,061][20347] Updated weights for policy 0, policy_version 846006 (0.0008) [2023-12-27 11:29:46,121][20347] Updated weights for policy 0, policy_version 846016 (0.0010) [2023-12-27 11:29:46,180][20347] Updated weights for policy 0, policy_version 846026 (0.0011) [2023-12-27 11:29:46,308][20381] Updated weights for policy 1, policy_version 845278 (0.0009) [2023-12-27 11:29:46,370][20381] Updated weights for policy 1, policy_version 845288 (0.0008) [2023-12-27 11:29:46,435][20381] Updated weights for policy 1, policy_version 845298 (0.0007) [2023-12-27 11:29:46,862][20347] Updated weights for policy 0, policy_version 846036 (0.0010) [2023-12-27 11:29:46,928][20347] Updated weights for policy 0, policy_version 846046 (0.0011) [2023-12-27 11:29:46,994][20347] Updated weights for policy 0, policy_version 846056 (0.0009) [2023-12-27 11:29:47,117][20381] Updated weights for policy 1, policy_version 845308 (0.0007) [2023-12-27 11:29:47,173][20381] Updated weights for policy 1, policy_version 845318 (0.0008) [2023-12-27 11:29:47,225][20381] Updated weights for policy 1, policy_version 845328 (0.0009) [2023-12-27 11:29:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 433053696. Throughput: 0: 9817.3, 1: 9653.7. Samples: 433060816. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:47,363][19170] Avg episode reward: [(0, '246.000'), (1, '735.370')] [2023-12-27 11:29:47,643][20347] Updated weights for policy 0, policy_version 846066 (0.0006) [2023-12-27 11:29:47,707][20347] Updated weights for policy 0, policy_version 846076 (0.0008) [2023-12-27 11:29:47,761][20347] Updated weights for policy 0, policy_version 846086 (0.0007) [2023-12-27 11:29:47,825][20347] Updated weights for policy 0, policy_version 846096 (0.0008) [2023-12-27 11:29:47,896][20381] Updated weights for policy 1, policy_version 845338 (0.0008) [2023-12-27 11:29:47,951][20381] Updated weights for policy 1, policy_version 845348 (0.0009) [2023-12-27 11:29:48,006][20381] Updated weights for policy 1, policy_version 845358 (0.0008) [2023-12-27 11:29:48,062][20381] Updated weights for policy 1, policy_version 845368 (0.0008) [2023-12-27 11:29:48,522][20347] Updated weights for policy 0, policy_version 846106 (0.0008) [2023-12-27 11:29:48,591][20347] Updated weights for policy 0, policy_version 846116 (0.0007) [2023-12-27 11:29:48,649][20347] Updated weights for policy 0, policy_version 846126 (0.0007) [2023-12-27 11:29:48,783][20381] Updated weights for policy 1, policy_version 845378 (0.0007) [2023-12-27 11:29:48,846][20381] Updated weights for policy 1, policy_version 845388 (0.0008) [2023-12-27 11:29:48,901][20381] Updated weights for policy 1, policy_version 845398 (0.0008) [2023-12-27 11:29:49,373][20347] Updated weights for policy 0, policy_version 846136 (0.0008) [2023-12-27 11:29:49,427][20347] Updated weights for policy 0, policy_version 846146 (0.0008) [2023-12-27 11:29:49,485][20347] Updated weights for policy 0, policy_version 846156 (0.0008) [2023-12-27 11:29:49,644][20381] Updated weights for policy 1, policy_version 845408 (0.0008) [2023-12-27 11:29:49,706][20381] Updated weights for policy 1, policy_version 845418 (0.0009) [2023-12-27 11:29:49,766][20381] Updated weights for policy 1, policy_version 845428 (0.0008) [2023-12-27 11:29:50,199][20347] Updated weights for policy 0, policy_version 846166 (0.0009) [2023-12-27 11:29:50,258][20347] Updated weights for policy 0, policy_version 846176 (0.0010) [2023-12-27 11:29:50,322][20347] Updated weights for policy 0, policy_version 846186 (0.0010) [2023-12-27 11:29:50,540][20381] Updated weights for policy 1, policy_version 845438 (0.0008) [2023-12-27 11:29:50,609][20381] Updated weights for policy 1, policy_version 845448 (0.0008) [2023-12-27 11:29:50,673][20381] Updated weights for policy 1, policy_version 845458 (0.0008) [2023-12-27 11:29:51,042][20347] Updated weights for policy 0, policy_version 846196 (0.0010) [2023-12-27 11:29:51,099][20347] Updated weights for policy 0, policy_version 846206 (0.0008) [2023-12-27 11:29:51,167][20347] Updated weights for policy 0, policy_version 846216 (0.0013) [2023-12-27 11:29:51,434][20381] Updated weights for policy 1, policy_version 845468 (0.0008) [2023-12-27 11:29:51,492][20381] Updated weights for policy 1, policy_version 845478 (0.0008) [2023-12-27 11:29:51,564][20381] Updated weights for policy 1, policy_version 845488 (0.0007) [2023-12-27 11:29:51,904][20347] Updated weights for policy 0, policy_version 846226 (0.0007) [2023-12-27 11:29:51,964][20347] Updated weights for policy 0, policy_version 846236 (0.0006) [2023-12-27 11:29:52,018][20347] Updated weights for policy 0, policy_version 846246 (0.0006) [2023-12-27 11:29:52,081][20347] Updated weights for policy 0, policy_version 846256 (0.0009) [2023-12-27 11:29:52,228][20381] Updated weights for policy 1, policy_version 845498 (0.0007) [2023-12-27 11:29:52,294][20381] Updated weights for policy 1, policy_version 845508 (0.0008) [2023-12-27 11:29:52,360][20381] Updated weights for policy 1, policy_version 845518 (0.0008) [2023-12-27 11:29:52,363][19170] Fps is (10 sec: 18841.3, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 433143808. Throughput: 0: 9760.5, 1: 9594.1. Samples: 433119232. Policy #0 lag: (min: 23.0, avg: 26.6, max: 55.0) [2023-12-27 11:29:52,363][19170] Avg episode reward: [(0, '87.430'), (1, '824.840')] [2023-12-27 11:29:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000846256_216670208.pth... [2023-12-27 11:29:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000845136_216383488.pth [2023-12-27 11:29:52,427][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000845528_216481792.pth... [2023-12-27 11:29:52,429][20381] Updated weights for policy 1, policy_version 845528 (0.0007) [2023-12-27 11:29:52,431][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000844440_216203264.pth [2023-12-27 11:29:52,794][20347] Updated weights for policy 0, policy_version 846266 (0.0008) [2023-12-27 11:29:52,862][20347] Updated weights for policy 0, policy_version 846276 (0.0009) [2023-12-27 11:29:52,919][20347] Updated weights for policy 0, policy_version 846286 (0.0008) [2023-12-27 11:29:53,054][20381] Updated weights for policy 1, policy_version 845538 (0.0008) [2023-12-27 11:29:53,121][20381] Updated weights for policy 1, policy_version 845548 (0.0011) [2023-12-27 11:29:53,182][20381] Updated weights for policy 1, policy_version 845558 (0.0011) [2023-12-27 11:29:53,609][20347] Updated weights for policy 0, policy_version 846296 (0.0007) [2023-12-27 11:29:53,671][20347] Updated weights for policy 0, policy_version 846306 (0.0006) [2023-12-27 11:29:53,732][20347] Updated weights for policy 0, policy_version 846316 (0.0006) [2023-12-27 11:29:53,890][20381] Updated weights for policy 1, policy_version 845568 (0.0011) [2023-12-27 11:29:53,944][20381] Updated weights for policy 1, policy_version 845578 (0.0011) [2023-12-27 11:29:54,008][20381] Updated weights for policy 1, policy_version 845588 (0.0011) [2023-12-27 11:29:54,380][20347] Updated weights for policy 0, policy_version 846326 (0.0009) [2023-12-27 11:29:54,461][20347] Updated weights for policy 0, policy_version 846336 (0.0009) [2023-12-27 11:29:54,528][20347] Updated weights for policy 0, policy_version 846346 (0.0008) [2023-12-27 11:29:54,730][20381] Updated weights for policy 1, policy_version 845598 (0.0011) [2023-12-27 11:29:54,788][20381] Updated weights for policy 1, policy_version 845608 (0.0008) [2023-12-27 11:29:54,839][20381] Updated weights for policy 1, policy_version 845618 (0.0008) [2023-12-27 11:29:55,215][20347] Updated weights for policy 0, policy_version 846356 (0.0007) [2023-12-27 11:29:55,276][20347] Updated weights for policy 0, policy_version 846366 (0.0007) [2023-12-27 11:29:55,338][20347] Updated weights for policy 0, policy_version 846376 (0.0007) [2023-12-27 11:29:55,556][20381] Updated weights for policy 1, policy_version 845628 (0.0009) [2023-12-27 11:29:55,619][20381] Updated weights for policy 1, policy_version 845638 (0.0008) [2023-12-27 11:29:55,667][20381] Updated weights for policy 1, policy_version 845648 (0.0009) [2023-12-27 11:29:56,023][20347] Updated weights for policy 0, policy_version 846386 (0.0007) [2023-12-27 11:29:56,076][20347] Updated weights for policy 0, policy_version 846396 (0.0009) [2023-12-27 11:29:56,139][20347] Updated weights for policy 0, policy_version 846406 (0.0008) [2023-12-27 11:29:56,203][20347] Updated weights for policy 0, policy_version 846416 (0.0009) [2023-12-27 11:29:56,403][20381] Updated weights for policy 1, policy_version 845658 (0.0007) [2023-12-27 11:29:56,461][20381] Updated weights for policy 1, policy_version 845668 (0.0007) [2023-12-27 11:29:56,520][20381] Updated weights for policy 1, policy_version 845678 (0.0008) [2023-12-27 11:29:56,578][20381] Updated weights for policy 1, policy_version 845688 (0.0008) [2023-12-27 11:29:56,994][20347] Updated weights for policy 0, policy_version 846426 (0.0008) [2023-12-27 11:29:57,055][20347] Updated weights for policy 0, policy_version 846436 (0.0008) [2023-12-27 11:29:57,107][20347] Updated weights for policy 0, policy_version 846446 (0.0007) [2023-12-27 11:29:57,305][20381] Updated weights for policy 1, policy_version 845698 (0.0007) [2023-12-27 11:29:57,360][20381] Updated weights for policy 1, policy_version 845708 (0.0006) [2023-12-27 11:29:57,362][19170] Fps is (10 sec: 18841.4, 60 sec: 19387.7, 300 sec: 19549.7). Total num frames: 433242112. Throughput: 0: 9698.9, 1: 9545.3. Samples: 433236472. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:29:57,363][19170] Avg episode reward: [(0, '153.290'), (1, '893.040')] [2023-12-27 11:29:57,414][20381] Updated weights for policy 1, policy_version 845718 (0.0008) [2023-12-27 11:29:57,837][20347] Updated weights for policy 0, policy_version 846456 (0.0008) [2023-12-27 11:29:57,888][20347] Updated weights for policy 0, policy_version 846466 (0.0007) [2023-12-27 11:29:57,946][20347] Updated weights for policy 0, policy_version 846476 (0.0007) [2023-12-27 11:29:58,093][20381] Updated weights for policy 1, policy_version 845728 (0.0006) [2023-12-27 11:29:58,157][20381] Updated weights for policy 1, policy_version 845738 (0.0006) [2023-12-27 11:29:58,219][20381] Updated weights for policy 1, policy_version 845748 (0.0008) [2023-12-27 11:29:58,773][20347] Updated weights for policy 0, policy_version 846486 (0.0008) [2023-12-27 11:29:58,842][20347] Updated weights for policy 0, policy_version 846496 (0.0009) [2023-12-27 11:29:58,910][20347] Updated weights for policy 0, policy_version 846506 (0.0008) [2023-12-27 11:29:59,065][20381] Updated weights for policy 1, policy_version 845758 (0.0008) [2023-12-27 11:29:59,122][20381] Updated weights for policy 1, policy_version 845768 (0.0008) [2023-12-27 11:29:59,185][20381] Updated weights for policy 1, policy_version 845778 (0.0009) [2023-12-27 11:29:59,818][20347] Updated weights for policy 0, policy_version 846516 (0.0010) [2023-12-27 11:29:59,911][20347] Updated weights for policy 0, policy_version 846528 (0.0013) [2023-12-27 11:29:59,983][20347] Updated weights for policy 0, policy_version 846538 (0.0008) [2023-12-27 11:29:59,989][20381] Updated weights for policy 1, policy_version 845788 (0.0008) [2023-12-27 11:30:00,046][20381] Updated weights for policy 1, policy_version 845798 (0.0009) [2023-12-27 11:30:00,104][20381] Updated weights for policy 1, policy_version 845808 (0.0008) [2023-12-27 11:30:00,676][20347] Updated weights for policy 0, policy_version 846548 (0.0008) [2023-12-27 11:30:00,737][20347] Updated weights for policy 0, policy_version 846558 (0.0006) [2023-12-27 11:30:00,795][20347] Updated weights for policy 0, policy_version 846568 (0.0007) [2023-12-27 11:30:00,820][20381] Updated weights for policy 1, policy_version 845818 (0.0011) [2023-12-27 11:30:00,877][20381] Updated weights for policy 1, policy_version 845828 (0.0011) [2023-12-27 11:30:00,929][20381] Updated weights for policy 1, policy_version 845838 (0.0010) [2023-12-27 11:30:00,981][20381] Updated weights for policy 1, policy_version 845848 (0.0010) [2023-12-27 11:30:01,533][20347] Updated weights for policy 0, policy_version 846578 (0.0007) [2023-12-27 11:30:01,586][20347] Updated weights for policy 0, policy_version 846588 (0.0006) [2023-12-27 11:30:01,654][20347] Updated weights for policy 0, policy_version 846598 (0.0007) [2023-12-27 11:30:01,719][20347] Updated weights for policy 0, policy_version 846608 (0.0007) [2023-12-27 11:30:01,746][20381] Updated weights for policy 1, policy_version 845858 (0.0009) [2023-12-27 11:30:01,806][20381] Updated weights for policy 1, policy_version 845868 (0.0008) [2023-12-27 11:30:01,864][20381] Updated weights for policy 1, policy_version 845878 (0.0008) [2023-12-27 11:30:02,362][19170] Fps is (10 sec: 18842.0, 60 sec: 19251.2, 300 sec: 19494.2). Total num frames: 433332224. Throughput: 0: 9564.2, 1: 9378.5. Samples: 433346164. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:02,363][19170] Avg episode reward: [(0, '225.470'), (1, '854.570')] [2023-12-27 11:30:02,469][20347] Updated weights for policy 0, policy_version 846618 (0.0008) [2023-12-27 11:30:02,528][20347] Updated weights for policy 0, policy_version 846628 (0.0007) [2023-12-27 11:30:02,581][20347] Updated weights for policy 0, policy_version 846638 (0.0007) [2023-12-27 11:30:02,582][20381] Updated weights for policy 1, policy_version 845888 (0.0006) [2023-12-27 11:30:02,639][20381] Updated weights for policy 1, policy_version 845898 (0.0008) [2023-12-27 11:30:02,693][20381] Updated weights for policy 1, policy_version 845908 (0.0007) [2023-12-27 11:30:03,264][20347] Updated weights for policy 0, policy_version 846648 (0.0006) [2023-12-27 11:30:03,326][20347] Updated weights for policy 0, policy_version 846658 (0.0007) [2023-12-27 11:30:03,379][20347] Updated weights for policy 0, policy_version 846668 (0.0008) [2023-12-27 11:30:03,417][20381] Updated weights for policy 1, policy_version 845918 (0.0008) [2023-12-27 11:30:03,474][20381] Updated weights for policy 1, policy_version 845928 (0.0008) [2023-12-27 11:30:03,531][20381] Updated weights for policy 1, policy_version 845938 (0.0008) [2023-12-27 11:30:04,105][20347] Updated weights for policy 0, policy_version 846678 (0.0007) [2023-12-27 11:30:04,168][20347] Updated weights for policy 0, policy_version 846688 (0.0007) [2023-12-27 11:30:04,235][20347] Updated weights for policy 0, policy_version 846698 (0.0009) [2023-12-27 11:30:04,261][20381] Updated weights for policy 1, policy_version 845948 (0.0007) [2023-12-27 11:30:04,329][20381] Updated weights for policy 1, policy_version 845958 (0.0009) [2023-12-27 11:30:04,395][20381] Updated weights for policy 1, policy_version 845968 (0.0008) [2023-12-27 11:30:04,889][20347] Updated weights for policy 0, policy_version 846708 (0.0008) [2023-12-27 11:30:04,939][20347] Updated weights for policy 0, policy_version 846718 (0.0008) [2023-12-27 11:30:04,989][20347] Updated weights for policy 0, policy_version 846728 (0.0008) [2023-12-27 11:30:05,041][20381] Updated weights for policy 1, policy_version 845978 (0.0008) [2023-12-27 11:30:05,091][20381] Updated weights for policy 1, policy_version 845988 (0.0005) [2023-12-27 11:30:05,149][20381] Updated weights for policy 1, policy_version 845998 (0.0008) [2023-12-27 11:30:05,211][20381] Updated weights for policy 1, policy_version 846008 (0.0008) [2023-12-27 11:30:05,726][20347] Updated weights for policy 0, policy_version 846738 (0.0008) [2023-12-27 11:30:05,783][20347] Updated weights for policy 0, policy_version 846748 (0.0008) [2023-12-27 11:30:05,841][20347] Updated weights for policy 0, policy_version 846758 (0.0008) [2023-12-27 11:30:05,848][20381] Updated weights for policy 1, policy_version 846018 (0.0006) [2023-12-27 11:30:05,896][20347] Updated weights for policy 0, policy_version 846768 (0.0008) [2023-12-27 11:30:05,899][20381] Updated weights for policy 1, policy_version 846028 (0.0007) [2023-12-27 11:30:05,956][20381] Updated weights for policy 1, policy_version 846038 (0.0007) [2023-12-27 11:30:06,684][20347] Updated weights for policy 0, policy_version 846778 (0.0008) [2023-12-27 11:30:06,725][20381] Updated weights for policy 1, policy_version 846048 (0.0007) [2023-12-27 11:30:06,751][20347] Updated weights for policy 0, policy_version 846788 (0.0008) [2023-12-27 11:30:06,783][20381] Updated weights for policy 1, policy_version 846058 (0.0007) [2023-12-27 11:30:06,820][20347] Updated weights for policy 0, policy_version 846798 (0.0007) [2023-12-27 11:30:06,843][20381] Updated weights for policy 1, policy_version 846068 (0.0008) [2023-12-27 11:30:07,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19251.2, 300 sec: 19549.7). Total num frames: 433430528. Throughput: 0: 9554.3, 1: 9397.6. Samples: 433405748. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:07,363][19170] Avg episode reward: [(0, '270.210'), (1, '864.740')] [2023-12-27 11:30:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000846072_216621056.pth... [2023-12-27 11:30:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000846800_216809472.pth... [2023-12-27 11:30:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000844952_216334336.pth [2023-12-27 11:30:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000845680_216522752.pth [2023-12-27 11:30:07,525][20347] Updated weights for policy 0, policy_version 846808 (0.0007) [2023-12-27 11:30:07,556][20381] Updated weights for policy 1, policy_version 846078 (0.0007) [2023-12-27 11:30:07,582][20347] Updated weights for policy 0, policy_version 846818 (0.0007) [2023-12-27 11:30:07,628][20381] Updated weights for policy 1, policy_version 846088 (0.0006) [2023-12-27 11:30:07,634][20347] Updated weights for policy 0, policy_version 846828 (0.0007) [2023-12-27 11:30:07,694][20381] Updated weights for policy 1, policy_version 846098 (0.0006) [2023-12-27 11:30:08,357][20381] Updated weights for policy 1, policy_version 846108 (0.0008) [2023-12-27 11:30:08,389][20347] Updated weights for policy 0, policy_version 846838 (0.0006) [2023-12-27 11:30:08,431][20381] Updated weights for policy 1, policy_version 846118 (0.0009) [2023-12-27 11:30:08,460][20347] Updated weights for policy 0, policy_version 846848 (0.0007) [2023-12-27 11:30:08,498][20381] Updated weights for policy 1, policy_version 846128 (0.0009) [2023-12-27 11:30:08,525][20347] Updated weights for policy 0, policy_version 846858 (0.0006) [2023-12-27 11:30:09,208][20381] Updated weights for policy 1, policy_version 846138 (0.0008) [2023-12-27 11:30:09,244][20347] Updated weights for policy 0, policy_version 846868 (0.0007) [2023-12-27 11:30:09,269][20381] Updated weights for policy 1, policy_version 846148 (0.0009) [2023-12-27 11:30:09,305][20347] Updated weights for policy 0, policy_version 846878 (0.0007) [2023-12-27 11:30:09,331][20381] Updated weights for policy 1, policy_version 846158 (0.0006) [2023-12-27 11:30:09,375][20347] Updated weights for policy 0, policy_version 846888 (0.0007) [2023-12-27 11:30:09,402][20381] Updated weights for policy 1, policy_version 846168 (0.0008) [2023-12-27 11:30:10,123][20347] Updated weights for policy 0, policy_version 846898 (0.0009) [2023-12-27 11:30:10,140][20381] Updated weights for policy 1, policy_version 846178 (0.0007) [2023-12-27 11:30:10,182][20347] Updated weights for policy 0, policy_version 846908 (0.0009) [2023-12-27 11:30:10,200][20381] Updated weights for policy 1, policy_version 846188 (0.0008) [2023-12-27 11:30:10,246][20347] Updated weights for policy 0, policy_version 846918 (0.0007) [2023-12-27 11:30:10,260][20381] Updated weights for policy 1, policy_version 846198 (0.0006) [2023-12-27 11:30:10,302][20347] Updated weights for policy 0, policy_version 846928 (0.0008) [2023-12-27 11:30:10,996][20381] Updated weights for policy 1, policy_version 846208 (0.0007) [2023-12-27 11:30:11,009][20347] Updated weights for policy 0, policy_version 846938 (0.0009) [2023-12-27 11:30:11,062][20381] Updated weights for policy 1, policy_version 846218 (0.0008) [2023-12-27 11:30:11,088][20347] Updated weights for policy 0, policy_version 846948 (0.0008) [2023-12-27 11:30:11,121][20381] Updated weights for policy 1, policy_version 846228 (0.0008) [2023-12-27 11:30:11,149][20347] Updated weights for policy 0, policy_version 846958 (0.0008) [2023-12-27 11:30:11,901][20381] Updated weights for policy 1, policy_version 846238 (0.0008) [2023-12-27 11:30:11,955][20347] Updated weights for policy 0, policy_version 846968 (0.0007) [2023-12-27 11:30:11,966][20381] Updated weights for policy 1, policy_version 846248 (0.0007) [2023-12-27 11:30:12,020][20347] Updated weights for policy 0, policy_version 846978 (0.0007) [2023-12-27 11:30:12,034][20381] Updated weights for policy 1, policy_version 846258 (0.0006) [2023-12-27 11:30:12,078][20347] Updated weights for policy 0, policy_version 846988 (0.0009) [2023-12-27 11:30:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19549.7). Total num frames: 433528832. Throughput: 0: 9559.0, 1: 9439.2. Samples: 433520204. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:12,363][19170] Avg episode reward: [(0, '199.450'), (1, '913.040')] [2023-12-27 11:30:12,705][20381] Updated weights for policy 1, policy_version 846268 (0.0007) [2023-12-27 11:30:12,764][20381] Updated weights for policy 1, policy_version 846278 (0.0008) [2023-12-27 11:30:12,835][20381] Updated weights for policy 1, policy_version 846288 (0.0008) [2023-12-27 11:30:12,856][20347] Updated weights for policy 0, policy_version 846998 (0.0008) [2023-12-27 11:30:12,921][20347] Updated weights for policy 0, policy_version 847008 (0.0007) [2023-12-27 11:30:12,986][20347] Updated weights for policy 0, policy_version 847018 (0.0008) [2023-12-27 11:30:13,500][20381] Updated weights for policy 1, policy_version 846298 (0.0006) [2023-12-27 11:30:13,563][20381] Updated weights for policy 1, policy_version 846308 (0.0005) [2023-12-27 11:30:13,616][20381] Updated weights for policy 1, policy_version 846318 (0.0005) [2023-12-27 11:30:13,667][20381] Updated weights for policy 1, policy_version 846328 (0.0005) [2023-12-27 11:30:13,686][20347] Updated weights for policy 0, policy_version 847028 (0.0008) [2023-12-27 11:30:13,735][20347] Updated weights for policy 0, policy_version 847038 (0.0007) [2023-12-27 11:30:13,791][20347] Updated weights for policy 0, policy_version 847048 (0.0008) [2023-12-27 11:30:14,256][20381] Updated weights for policy 1, policy_version 846338 (0.0008) [2023-12-27 11:30:14,306][20381] Updated weights for policy 1, policy_version 846348 (0.0008) [2023-12-27 11:30:14,368][20381] Updated weights for policy 1, policy_version 846358 (0.0007) [2023-12-27 11:30:14,515][20347] Updated weights for policy 0, policy_version 847058 (0.0007) [2023-12-27 11:30:14,572][20347] Updated weights for policy 0, policy_version 847068 (0.0005) [2023-12-27 11:30:14,630][20347] Updated weights for policy 0, policy_version 847078 (0.0007) [2023-12-27 11:30:14,680][20347] Updated weights for policy 0, policy_version 847088 (0.0008) [2023-12-27 11:30:15,064][20381] Updated weights for policy 1, policy_version 846368 (0.0007) [2023-12-27 11:30:15,130][20381] Updated weights for policy 1, policy_version 846378 (0.0009) [2023-12-27 11:30:15,194][20381] Updated weights for policy 1, policy_version 846388 (0.0007) [2023-12-27 11:30:15,432][20347] Updated weights for policy 0, policy_version 847098 (0.0008) [2023-12-27 11:30:15,492][20347] Updated weights for policy 0, policy_version 847108 (0.0008) [2023-12-27 11:30:15,551][20347] Updated weights for policy 0, policy_version 847118 (0.0009) [2023-12-27 11:30:15,899][20381] Updated weights for policy 1, policy_version 846398 (0.0008) [2023-12-27 11:30:15,987][20381] Updated weights for policy 1, policy_version 846412 (0.0008) [2023-12-27 11:30:16,045][20381] Updated weights for policy 1, policy_version 846422 (0.0006) [2023-12-27 11:30:16,333][20347] Updated weights for policy 0, policy_version 847128 (0.0008) [2023-12-27 11:30:16,391][20347] Updated weights for policy 0, policy_version 847138 (0.0008) [2023-12-27 11:30:16,441][20347] Updated weights for policy 0, policy_version 847148 (0.0007) [2023-12-27 11:30:16,778][20381] Updated weights for policy 1, policy_version 846432 (0.0008) [2023-12-27 11:30:16,842][20381] Updated weights for policy 1, policy_version 846442 (0.0008) [2023-12-27 11:30:16,907][20381] Updated weights for policy 1, policy_version 846452 (0.0007) [2023-12-27 11:30:17,150][20347] Updated weights for policy 0, policy_version 847158 (0.0009) [2023-12-27 11:30:17,212][20347] Updated weights for policy 0, policy_version 847168 (0.0007) [2023-12-27 11:30:17,274][20347] Updated weights for policy 0, policy_version 847178 (0.0008) [2023-12-27 11:30:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 18978.1, 300 sec: 19549.7). Total num frames: 433627136. Throughput: 0: 9476.4, 1: 9513.8. Samples: 433635880. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:17,363][19170] Avg episode reward: [(0, '181.320'), (1, '912.480')] [2023-12-27 11:30:17,615][20381] Updated weights for policy 1, policy_version 846462 (0.0008) [2023-12-27 11:30:17,673][20381] Updated weights for policy 1, policy_version 846472 (0.0008) [2023-12-27 11:30:17,727][20381] Updated weights for policy 1, policy_version 846482 (0.0008) [2023-12-27 11:30:18,037][20347] Updated weights for policy 0, policy_version 847188 (0.0008) [2023-12-27 11:30:18,141][20347] Updated weights for policy 0, policy_version 847201 (0.0007) [2023-12-27 11:30:18,203][20347] Updated weights for policy 0, policy_version 847211 (0.0006) [2023-12-27 11:30:18,437][20381] Updated weights for policy 1, policy_version 846492 (0.0007) [2023-12-27 11:30:18,495][20381] Updated weights for policy 1, policy_version 846502 (0.0007) [2023-12-27 11:30:18,549][20381] Updated weights for policy 1, policy_version 846512 (0.0007) [2023-12-27 11:30:18,897][20347] Updated weights for policy 0, policy_version 847221 (0.0006) [2023-12-27 11:30:18,964][20347] Updated weights for policy 0, policy_version 847231 (0.0007) [2023-12-27 11:30:19,024][20347] Updated weights for policy 0, policy_version 847241 (0.0006) [2023-12-27 11:30:19,231][20381] Updated weights for policy 1, policy_version 846522 (0.0008) [2023-12-27 11:30:19,296][20381] Updated weights for policy 1, policy_version 846532 (0.0007) [2023-12-27 11:30:19,367][20381] Updated weights for policy 1, policy_version 846542 (0.0008) [2023-12-27 11:30:19,438][20381] Updated weights for policy 1, policy_version 846552 (0.0008) [2023-12-27 11:30:19,685][20347] Updated weights for policy 0, policy_version 847251 (0.0008) [2023-12-27 11:30:19,749][20347] Updated weights for policy 0, policy_version 847261 (0.0007) [2023-12-27 11:30:19,814][20347] Updated weights for policy 0, policy_version 847271 (0.0008) [2023-12-27 11:30:20,164][20381] Updated weights for policy 1, policy_version 846562 (0.0008) [2023-12-27 11:30:20,226][20381] Updated weights for policy 1, policy_version 846572 (0.0009) [2023-12-27 11:30:20,292][20381] Updated weights for policy 1, policy_version 846582 (0.0008) [2023-12-27 11:30:20,505][20347] Updated weights for policy 0, policy_version 847281 (0.0007) [2023-12-27 11:30:20,563][20347] Updated weights for policy 0, policy_version 847291 (0.0008) [2023-12-27 11:30:20,632][20347] Updated weights for policy 0, policy_version 847301 (0.0008) [2023-12-27 11:30:20,696][20347] Updated weights for policy 0, policy_version 847311 (0.0007) [2023-12-27 11:30:21,041][20381] Updated weights for policy 1, policy_version 846592 (0.0008) [2023-12-27 11:30:21,112][20381] Updated weights for policy 1, policy_version 846602 (0.0007) [2023-12-27 11:30:21,182][20381] Updated weights for policy 1, policy_version 846612 (0.0008) [2023-12-27 11:30:21,416][20347] Updated weights for policy 0, policy_version 847321 (0.0008) [2023-12-27 11:30:21,484][20347] Updated weights for policy 0, policy_version 847331 (0.0007) [2023-12-27 11:30:21,540][20347] Updated weights for policy 0, policy_version 847341 (0.0007) [2023-12-27 11:30:22,016][20381] Updated weights for policy 1, policy_version 846622 (0.0008) [2023-12-27 11:30:22,085][20381] Updated weights for policy 1, policy_version 846632 (0.0008) [2023-12-27 11:30:22,150][20381] Updated weights for policy 1, policy_version 846642 (0.0008) [2023-12-27 11:30:22,257][20347] Updated weights for policy 0, policy_version 847351 (0.0007) [2023-12-27 11:30:22,327][20347] Updated weights for policy 0, policy_version 847361 (0.0008) [2023-12-27 11:30:22,363][19170] Fps is (10 sec: 18841.2, 60 sec: 18978.5, 300 sec: 19521.9). Total num frames: 433717248. Throughput: 0: 9540.2, 1: 9602.2. Samples: 433693420. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:22,363][19170] Avg episode reward: [(0, '240.000'), (1, '912.670')] [2023-12-27 11:30:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000846648_216768512.pth... [2023-12-27 11:30:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000845528_216481792.pth [2023-12-27 11:30:22,399][20347] Updated weights for policy 0, policy_version 847371 (0.0007) [2023-12-27 11:30:22,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000847376_216956928.pth... [2023-12-27 11:30:22,434][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000846256_216670208.pth [2023-12-27 11:30:22,949][20381] Updated weights for policy 1, policy_version 846652 (0.0008) [2023-12-27 11:30:23,011][20381] Updated weights for policy 1, policy_version 846662 (0.0008) [2023-12-27 11:30:23,071][20381] Updated weights for policy 1, policy_version 846672 (0.0008) [2023-12-27 11:30:23,160][20347] Updated weights for policy 0, policy_version 847381 (0.0008) [2023-12-27 11:30:23,238][20347] Updated weights for policy 0, policy_version 847391 (0.0007) [2023-12-27 11:30:23,304][20347] Updated weights for policy 0, policy_version 847401 (0.0008) [2023-12-27 11:30:23,797][20381] Updated weights for policy 1, policy_version 846682 (0.0008) [2023-12-27 11:30:23,861][20381] Updated weights for policy 1, policy_version 846692 (0.0008) [2023-12-27 11:30:23,929][20381] Updated weights for policy 1, policy_version 846702 (0.0008) [2023-12-27 11:30:23,981][20347] Updated weights for policy 0, policy_version 847411 (0.0007) [2023-12-27 11:30:23,993][20381] Updated weights for policy 1, policy_version 846712 (0.0007) [2023-12-27 11:30:24,048][20347] Updated weights for policy 0, policy_version 847421 (0.0009) [2023-12-27 11:30:24,120][20347] Updated weights for policy 0, policy_version 847431 (0.0009) [2023-12-27 11:30:24,719][20381] Updated weights for policy 1, policy_version 846722 (0.0007) [2023-12-27 11:30:24,781][20381] Updated weights for policy 1, policy_version 846732 (0.0007) [2023-12-27 11:30:24,846][20381] Updated weights for policy 1, policy_version 846742 (0.0008) [2023-12-27 11:30:24,878][20347] Updated weights for policy 0, policy_version 847442 (0.0009) [2023-12-27 11:30:24,937][20347] Updated weights for policy 0, policy_version 847452 (0.0008) [2023-12-27 11:30:25,004][20347] Updated weights for policy 0, policy_version 847462 (0.0008) [2023-12-27 11:30:25,061][20347] Updated weights for policy 0, policy_version 847472 (0.0008) [2023-12-27 11:30:25,556][20381] Updated weights for policy 1, policy_version 846752 (0.0008) [2023-12-27 11:30:25,624][20381] Updated weights for policy 1, policy_version 846762 (0.0006) [2023-12-27 11:30:25,687][20381] Updated weights for policy 1, policy_version 846772 (0.0008) [2023-12-27 11:30:25,823][20347] Updated weights for policy 0, policy_version 847484 (0.0013) [2023-12-27 11:30:25,904][20347] Updated weights for policy 0, policy_version 847495 (0.0008) [2023-12-27 11:30:26,418][20381] Updated weights for policy 1, policy_version 846782 (0.0014) [2023-12-27 11:30:26,488][20381] Updated weights for policy 1, policy_version 846792 (0.0007) [2023-12-27 11:30:26,555][20381] Updated weights for policy 1, policy_version 846802 (0.0008) [2023-12-27 11:30:26,664][20347] Updated weights for policy 0, policy_version 847505 (0.0008) [2023-12-27 11:30:26,723][20347] Updated weights for policy 0, policy_version 847515 (0.0007) [2023-12-27 11:30:26,787][20347] Updated weights for policy 0, policy_version 847525 (0.0007) [2023-12-27 11:30:26,845][20347] Updated weights for policy 0, policy_version 847535 (0.0008) [2023-12-27 11:30:27,241][20381] Updated weights for policy 1, policy_version 846812 (0.0008) [2023-12-27 11:30:27,305][20381] Updated weights for policy 1, policy_version 846822 (0.0008) [2023-12-27 11:30:27,362][19170] Fps is (10 sec: 18022.6, 60 sec: 18841.6, 300 sec: 19494.2). Total num frames: 433807360. Throughput: 0: 9518.9, 1: 9613.8. Samples: 433804704. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:27,363][19170] Avg episode reward: [(0, '214.090'), (1, '941.730')] [2023-12-27 11:30:27,371][20381] Updated weights for policy 1, policy_version 846832 (0.0008) [2023-12-27 11:30:27,554][20347] Updated weights for policy 0, policy_version 847545 (0.0007) [2023-12-27 11:30:27,613][20347] Updated weights for policy 0, policy_version 847555 (0.0008) [2023-12-27 11:30:27,678][20347] Updated weights for policy 0, policy_version 847565 (0.0008) [2023-12-27 11:30:28,068][20381] Updated weights for policy 1, policy_version 846842 (0.0008) [2023-12-27 11:30:28,124][20381] Updated weights for policy 1, policy_version 846852 (0.0008) [2023-12-27 11:30:28,182][20381] Updated weights for policy 1, policy_version 846862 (0.0008) [2023-12-27 11:30:28,240][20381] Updated weights for policy 1, policy_version 846872 (0.0008) [2023-12-27 11:30:28,437][20347] Updated weights for policy 0, policy_version 847575 (0.0007) [2023-12-27 11:30:28,504][20347] Updated weights for policy 0, policy_version 847585 (0.0007) [2023-12-27 11:30:28,567][20347] Updated weights for policy 0, policy_version 847595 (0.0008) [2023-12-27 11:30:29,022][20381] Updated weights for policy 1, policy_version 846882 (0.0008) [2023-12-27 11:30:29,082][20381] Updated weights for policy 1, policy_version 846892 (0.0006) [2023-12-27 11:30:29,138][20381] Updated weights for policy 1, policy_version 846902 (0.0007) [2023-12-27 11:30:29,284][20347] Updated weights for policy 0, policy_version 847605 (0.0007) [2023-12-27 11:30:29,347][20347] Updated weights for policy 0, policy_version 847615 (0.0008) [2023-12-27 11:30:29,411][20347] Updated weights for policy 0, policy_version 847625 (0.0008) [2023-12-27 11:30:29,930][20381] Updated weights for policy 1, policy_version 846912 (0.0008) [2023-12-27 11:30:30,004][20381] Updated weights for policy 1, policy_version 846922 (0.0008) [2023-12-27 11:30:30,070][20381] Updated weights for policy 1, policy_version 846932 (0.0008) [2023-12-27 11:30:30,169][20347] Updated weights for policy 0, policy_version 847635 (0.0008) [2023-12-27 11:30:30,236][20347] Updated weights for policy 0, policy_version 847645 (0.0008) [2023-12-27 11:30:30,303][20347] Updated weights for policy 0, policy_version 847655 (0.0007) [2023-12-27 11:30:30,820][20381] Updated weights for policy 1, policy_version 846942 (0.0008) [2023-12-27 11:30:30,880][20381] Updated weights for policy 1, policy_version 846952 (0.0008) [2023-12-27 11:30:30,937][20381] Updated weights for policy 1, policy_version 846962 (0.0007) [2023-12-27 11:30:30,982][20347] Updated weights for policy 0, policy_version 847665 (0.0007) [2023-12-27 11:30:31,046][20347] Updated weights for policy 0, policy_version 847675 (0.0008) [2023-12-27 11:30:31,112][20347] Updated weights for policy 0, policy_version 847685 (0.0007) [2023-12-27 11:30:31,183][20347] Updated weights for policy 0, policy_version 847695 (0.0008) [2023-12-27 11:30:31,671][20381] Updated weights for policy 1, policy_version 846972 (0.0007) [2023-12-27 11:30:31,739][20381] Updated weights for policy 1, policy_version 846982 (0.0009) [2023-12-27 11:30:31,811][20381] Updated weights for policy 1, policy_version 846992 (0.0010) [2023-12-27 11:30:31,957][20347] Updated weights for policy 0, policy_version 847705 (0.0007) [2023-12-27 11:30:32,022][20347] Updated weights for policy 0, policy_version 847715 (0.0008) [2023-12-27 11:30:32,087][20347] Updated weights for policy 0, policy_version 847725 (0.0008) [2023-12-27 11:30:32,362][19170] Fps is (10 sec: 18841.9, 60 sec: 18841.6, 300 sec: 19494.2). Total num frames: 433905664. Throughput: 0: 9448.4, 1: 9542.2. Samples: 433915396. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:32,363][19170] Avg episode reward: [(0, '269.880'), (1, '970.750')] [2023-12-27 11:30:32,553][20381] Updated weights for policy 1, policy_version 847002 (0.0008) [2023-12-27 11:30:32,609][20381] Updated weights for policy 1, policy_version 847012 (0.0008) [2023-12-27 11:30:32,670][20381] Updated weights for policy 1, policy_version 847022 (0.0008) [2023-12-27 11:30:32,738][20381] Updated weights for policy 1, policy_version 847032 (0.0008) [2023-12-27 11:30:32,788][20347] Updated weights for policy 0, policy_version 847735 (0.0007) [2023-12-27 11:30:32,845][20347] Updated weights for policy 0, policy_version 847745 (0.0008) [2023-12-27 11:30:32,904][20347] Updated weights for policy 0, policy_version 847755 (0.0007) [2023-12-27 11:30:33,490][20381] Updated weights for policy 1, policy_version 847042 (0.0011) [2023-12-27 11:30:33,555][20381] Updated weights for policy 1, policy_version 847052 (0.0011) [2023-12-27 11:30:33,615][20381] Updated weights for policy 1, policy_version 847062 (0.0010) [2023-12-27 11:30:33,694][20347] Updated weights for policy 0, policy_version 847765 (0.0008) [2023-12-27 11:30:33,755][20347] Updated weights for policy 0, policy_version 847775 (0.0007) [2023-12-27 11:30:33,825][20347] Updated weights for policy 0, policy_version 847785 (0.0008) [2023-12-27 11:30:34,343][20381] Updated weights for policy 1, policy_version 847072 (0.0011) [2023-12-27 11:30:34,401][20381] Updated weights for policy 1, policy_version 847082 (0.0011) [2023-12-27 11:30:34,466][20381] Updated weights for policy 1, policy_version 847092 (0.0009) [2023-12-27 11:30:34,630][20347] Updated weights for policy 0, policy_version 847795 (0.0009) [2023-12-27 11:30:34,700][20347] Updated weights for policy 0, policy_version 847805 (0.0008) [2023-12-27 11:30:34,764][20347] Updated weights for policy 0, policy_version 847815 (0.0007) [2023-12-27 11:30:35,210][20381] Updated weights for policy 1, policy_version 847102 (0.0008) [2023-12-27 11:30:35,263][20381] Updated weights for policy 1, policy_version 847112 (0.0008) [2023-12-27 11:30:35,315][20381] Updated weights for policy 1, policy_version 847122 (0.0008) [2023-12-27 11:30:35,377][20347] Updated weights for policy 0, policy_version 847825 (0.0007) [2023-12-27 11:30:35,435][20347] Updated weights for policy 0, policy_version 847835 (0.0006) [2023-12-27 11:30:35,494][20347] Updated weights for policy 0, policy_version 847845 (0.0009) [2023-12-27 11:30:35,556][20347] Updated weights for policy 0, policy_version 847855 (0.0006) [2023-12-27 11:30:35,944][20381] Updated weights for policy 1, policy_version 847132 (0.0009) [2023-12-27 11:30:36,006][20381] Updated weights for policy 1, policy_version 847142 (0.0006) [2023-12-27 11:30:36,067][20381] Updated weights for policy 1, policy_version 847152 (0.0007) [2023-12-27 11:30:36,231][20347] Updated weights for policy 0, policy_version 847865 (0.0007) [2023-12-27 11:30:36,297][20347] Updated weights for policy 0, policy_version 847875 (0.0008) [2023-12-27 11:30:36,358][20347] Updated weights for policy 0, policy_version 847885 (0.0008) [2023-12-27 11:30:36,741][20381] Updated weights for policy 1, policy_version 847162 (0.0008) [2023-12-27 11:30:36,804][20381] Updated weights for policy 1, policy_version 847172 (0.0008) [2023-12-27 11:30:36,860][20381] Updated weights for policy 1, policy_version 847182 (0.0008) [2023-12-27 11:30:36,922][20381] Updated weights for policy 1, policy_version 847192 (0.0007) [2023-12-27 11:30:37,094][20347] Updated weights for policy 0, policy_version 847895 (0.0006) [2023-12-27 11:30:37,155][20347] Updated weights for policy 0, policy_version 847905 (0.0008) [2023-12-27 11:30:37,213][20347] Updated weights for policy 0, policy_version 847915 (0.0008) [2023-12-27 11:30:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19494.2). Total num frames: 434003968. Throughput: 0: 9419.1, 1: 9515.6. Samples: 433971288. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:37,363][19170] Avg episode reward: [(0, '285.870'), (1, '990.240')] [2023-12-27 11:30:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000847920_217096192.pth... [2023-12-27 11:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000847192_216907776.pth... [2023-12-27 11:30:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000846072_216621056.pth [2023-12-27 11:30:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000846800_216809472.pth [2023-12-27 11:30:37,590][20381] Updated weights for policy 1, policy_version 847202 (0.0008) [2023-12-27 11:30:37,652][20381] Updated weights for policy 1, policy_version 847212 (0.0007) [2023-12-27 11:30:37,712][20381] Updated weights for policy 1, policy_version 847222 (0.0008) [2023-12-27 11:30:37,936][20347] Updated weights for policy 0, policy_version 847925 (0.0008) [2023-12-27 11:30:37,998][20347] Updated weights for policy 0, policy_version 847935 (0.0007) [2023-12-27 11:30:38,053][20347] Updated weights for policy 0, policy_version 847945 (0.0008) [2023-12-27 11:30:38,440][20381] Updated weights for policy 1, policy_version 847232 (0.0009) [2023-12-27 11:30:38,507][20381] Updated weights for policy 1, policy_version 847242 (0.0009) [2023-12-27 11:30:38,571][20381] Updated weights for policy 1, policy_version 847252 (0.0009) [2023-12-27 11:30:38,839][20347] Updated weights for policy 0, policy_version 847955 (0.0008) [2023-12-27 11:30:38,893][20347] Updated weights for policy 0, policy_version 847965 (0.0009) [2023-12-27 11:30:38,951][20347] Updated weights for policy 0, policy_version 847975 (0.0009) [2023-12-27 11:30:39,469][20381] Updated weights for policy 1, policy_version 847262 (0.0008) [2023-12-27 11:30:39,529][20381] Updated weights for policy 1, policy_version 847272 (0.0008) [2023-12-27 11:30:39,596][20381] Updated weights for policy 1, policy_version 847282 (0.0008) [2023-12-27 11:30:39,764][20347] Updated weights for policy 0, policy_version 847985 (0.0009) [2023-12-27 11:30:39,828][20347] Updated weights for policy 0, policy_version 847995 (0.0010) [2023-12-27 11:30:39,894][20347] Updated weights for policy 0, policy_version 848005 (0.0010) [2023-12-27 11:30:39,964][20347] Updated weights for policy 0, policy_version 848015 (0.0010) [2023-12-27 11:30:40,381][20381] Updated weights for policy 1, policy_version 847292 (0.0008) [2023-12-27 11:30:40,434][20381] Updated weights for policy 1, policy_version 847302 (0.0008) [2023-12-27 11:30:40,488][20381] Updated weights for policy 1, policy_version 847312 (0.0008) [2023-12-27 11:30:40,698][20347] Updated weights for policy 0, policy_version 848025 (0.0010) [2023-12-27 11:30:40,755][20347] Updated weights for policy 0, policy_version 848035 (0.0009) [2023-12-27 11:30:40,810][20347] Updated weights for policy 0, policy_version 848045 (0.0009) [2023-12-27 11:30:41,214][20381] Updated weights for policy 1, policy_version 847322 (0.0008) [2023-12-27 11:30:41,275][20381] Updated weights for policy 1, policy_version 847332 (0.0009) [2023-12-27 11:30:41,330][20381] Updated weights for policy 1, policy_version 847342 (0.0010) [2023-12-27 11:30:41,403][20381] Updated weights for policy 1, policy_version 847352 (0.0010) [2023-12-27 11:30:41,561][20347] Updated weights for policy 0, policy_version 848055 (0.0008) [2023-12-27 11:30:41,625][20347] Updated weights for policy 0, policy_version 848065 (0.0007) [2023-12-27 11:30:41,689][20347] Updated weights for policy 0, policy_version 848075 (0.0008) [2023-12-27 11:30:42,204][20381] Updated weights for policy 1, policy_version 847362 (0.0008) [2023-12-27 11:30:42,272][20381] Updated weights for policy 1, policy_version 847372 (0.0008) [2023-12-27 11:30:42,330][20381] Updated weights for policy 1, policy_version 847382 (0.0008) [2023-12-27 11:30:42,362][19170] Fps is (10 sec: 18841.5, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 434094080. Throughput: 0: 9369.3, 1: 9491.6. Samples: 434085216. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:42,363][19170] Avg episode reward: [(0, '346.050'), (1, '961.200')] [2023-12-27 11:30:42,404][20347] Updated weights for policy 0, policy_version 848085 (0.0007) [2023-12-27 11:30:42,477][20347] Updated weights for policy 0, policy_version 848095 (0.0008) [2023-12-27 11:30:42,541][20347] Updated weights for policy 0, policy_version 848105 (0.0007) [2023-12-27 11:30:43,106][20381] Updated weights for policy 1, policy_version 847392 (0.0007) [2023-12-27 11:30:43,161][20381] Updated weights for policy 1, policy_version 847402 (0.0007) [2023-12-27 11:30:43,218][20381] Updated weights for policy 1, policy_version 847412 (0.0006) [2023-12-27 11:30:43,262][20347] Updated weights for policy 0, policy_version 848115 (0.0009) [2023-12-27 11:30:43,311][20347] Updated weights for policy 0, policy_version 848125 (0.0010) [2023-12-27 11:30:43,368][20347] Updated weights for policy 0, policy_version 848135 (0.0010) [2023-12-27 11:30:43,914][20381] Updated weights for policy 1, policy_version 847422 (0.0006) [2023-12-27 11:30:43,976][20381] Updated weights for policy 1, policy_version 847432 (0.0006) [2023-12-27 11:30:44,038][20381] Updated weights for policy 1, policy_version 847442 (0.0007) [2023-12-27 11:30:44,062][20347] Updated weights for policy 0, policy_version 848145 (0.0010) [2023-12-27 11:30:44,116][20347] Updated weights for policy 0, policy_version 848155 (0.0010) [2023-12-27 11:30:44,179][20347] Updated weights for policy 0, policy_version 848165 (0.0010) [2023-12-27 11:30:44,228][20347] Updated weights for policy 0, policy_version 848175 (0.0008) [2023-12-27 11:30:44,698][20381] Updated weights for policy 1, policy_version 847452 (0.0009) [2023-12-27 11:30:44,763][20381] Updated weights for policy 1, policy_version 847462 (0.0008) [2023-12-27 11:30:44,827][20381] Updated weights for policy 1, policy_version 847472 (0.0008) [2023-12-27 11:30:44,931][20347] Updated weights for policy 0, policy_version 848185 (0.0010) [2023-12-27 11:30:44,994][20347] Updated weights for policy 0, policy_version 848195 (0.0010) [2023-12-27 11:30:45,058][20347] Updated weights for policy 0, policy_version 848205 (0.0011) [2023-12-27 11:30:45,560][20381] Updated weights for policy 1, policy_version 847482 (0.0008) [2023-12-27 11:30:45,613][20381] Updated weights for policy 1, policy_version 847492 (0.0009) [2023-12-27 11:30:45,670][20381] Updated weights for policy 1, policy_version 847502 (0.0008) [2023-12-27 11:30:45,732][20381] Updated weights for policy 1, policy_version 847512 (0.0007) [2023-12-27 11:30:45,767][20347] Updated weights for policy 0, policy_version 848215 (0.0008) [2023-12-27 11:30:45,823][20347] Updated weights for policy 0, policy_version 848225 (0.0008) [2023-12-27 11:30:45,882][20347] Updated weights for policy 0, policy_version 848235 (0.0008) [2023-12-27 11:30:46,389][20381] Updated weights for policy 1, policy_version 847522 (0.0007) [2023-12-27 11:30:46,446][20381] Updated weights for policy 1, policy_version 847532 (0.0007) [2023-12-27 11:30:46,498][20381] Updated weights for policy 1, policy_version 847542 (0.0008) [2023-12-27 11:30:46,624][20347] Updated weights for policy 0, policy_version 848245 (0.0007) [2023-12-27 11:30:46,681][20347] Updated weights for policy 0, policy_version 848255 (0.0008) [2023-12-27 11:30:46,729][20347] Updated weights for policy 0, policy_version 848265 (0.0008) [2023-12-27 11:30:47,203][20381] Updated weights for policy 1, policy_version 847552 (0.0008) [2023-12-27 11:30:47,263][20381] Updated weights for policy 1, policy_version 847562 (0.0007) [2023-12-27 11:30:47,325][20381] Updated weights for policy 1, policy_version 847572 (0.0006) [2023-12-27 11:30:47,362][19170] Fps is (10 sec: 18841.8, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 434192384. Throughput: 0: 9452.7, 1: 9534.2. Samples: 434200576. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:47,363][19170] Avg episode reward: [(0, '330.850'), (1, '942.460')] [2023-12-27 11:30:47,441][20347] Updated weights for policy 0, policy_version 848275 (0.0008) [2023-12-27 11:30:47,497][20347] Updated weights for policy 0, policy_version 848285 (0.0008) [2023-12-27 11:30:47,555][20347] Updated weights for policy 0, policy_version 848295 (0.0007) [2023-12-27 11:30:47,941][20381] Updated weights for policy 1, policy_version 847582 (0.0007) [2023-12-27 11:30:47,995][20381] Updated weights for policy 1, policy_version 847592 (0.0008) [2023-12-27 11:30:48,056][20381] Updated weights for policy 1, policy_version 847602 (0.0008) [2023-12-27 11:30:48,274][20347] Updated weights for policy 0, policy_version 848305 (0.0007) [2023-12-27 11:30:48,337][20347] Updated weights for policy 0, policy_version 848315 (0.0007) [2023-12-27 11:30:48,396][20347] Updated weights for policy 0, policy_version 848325 (0.0008) [2023-12-27 11:30:48,453][20347] Updated weights for policy 0, policy_version 848335 (0.0007) [2023-12-27 11:30:48,742][20381] Updated weights for policy 1, policy_version 847612 (0.0008) [2023-12-27 11:30:48,806][20381] Updated weights for policy 1, policy_version 847622 (0.0007) [2023-12-27 11:30:48,867][20381] Updated weights for policy 1, policy_version 847632 (0.0007) [2023-12-27 11:30:49,153][20347] Updated weights for policy 0, policy_version 848345 (0.0006) [2023-12-27 11:30:49,218][20347] Updated weights for policy 0, policy_version 848355 (0.0006) [2023-12-27 11:30:49,278][20347] Updated weights for policy 0, policy_version 848365 (0.0010) [2023-12-27 11:30:49,548][20381] Updated weights for policy 1, policy_version 847642 (0.0007) [2023-12-27 11:30:49,602][20381] Updated weights for policy 1, policy_version 847652 (0.0009) [2023-12-27 11:30:49,656][20381] Updated weights for policy 1, policy_version 847662 (0.0009) [2023-12-27 11:30:49,714][20381] Updated weights for policy 1, policy_version 847672 (0.0009) [2023-12-27 11:30:50,086][20347] Updated weights for policy 0, policy_version 848375 (0.0008) [2023-12-27 11:30:50,150][20347] Updated weights for policy 0, policy_version 848385 (0.0006) [2023-12-27 11:30:50,215][20347] Updated weights for policy 0, policy_version 848395 (0.0006) [2023-12-27 11:30:50,531][20381] Updated weights for policy 1, policy_version 847682 (0.0009) [2023-12-27 11:30:50,594][20381] Updated weights for policy 1, policy_version 847692 (0.0008) [2023-12-27 11:30:50,657][20381] Updated weights for policy 1, policy_version 847702 (0.0006) [2023-12-27 11:30:50,928][20347] Updated weights for policy 0, policy_version 848405 (0.0007) [2023-12-27 11:30:50,990][20347] Updated weights for policy 0, policy_version 848415 (0.0006) [2023-12-27 11:30:51,060][20347] Updated weights for policy 0, policy_version 848425 (0.0008) [2023-12-27 11:30:51,389][20381] Updated weights for policy 1, policy_version 847712 (0.0008) [2023-12-27 11:30:51,458][20381] Updated weights for policy 1, policy_version 847722 (0.0007) [2023-12-27 11:30:51,523][20381] Updated weights for policy 1, policy_version 847732 (0.0008) [2023-12-27 11:30:51,813][20347] Updated weights for policy 0, policy_version 848435 (0.0007) [2023-12-27 11:30:51,874][20347] Updated weights for policy 0, policy_version 848445 (0.0007) [2023-12-27 11:30:51,936][20347] Updated weights for policy 0, policy_version 848455 (0.0007) [2023-12-27 11:30:52,270][20381] Updated weights for policy 1, policy_version 847742 (0.0010) [2023-12-27 11:30:52,334][20381] Updated weights for policy 1, policy_version 847752 (0.0010) [2023-12-27 11:30:52,363][19170] Fps is (10 sec: 18840.5, 60 sec: 18978.0, 300 sec: 19438.6). Total num frames: 434282496. Throughput: 0: 9417.9, 1: 9526.8. Samples: 434258272. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:52,364][19170] Avg episode reward: [(0, '209.850'), (1, '864.430')] [2023-12-27 11:30:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000848464_217235456.pth... [2023-12-27 11:30:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000847376_216956928.pth [2023-12-27 11:30:52,400][20381] Updated weights for policy 1, policy_version 847762 (0.0009) [2023-12-27 11:30:52,437][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000847768_217055232.pth... [2023-12-27 11:30:52,441][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000846648_216768512.pth [2023-12-27 11:30:52,617][20347] Updated weights for policy 0, policy_version 848465 (0.0007) [2023-12-27 11:30:52,678][20347] Updated weights for policy 0, policy_version 848475 (0.0008) [2023-12-27 11:30:52,737][20347] Updated weights for policy 0, policy_version 848485 (0.0006) [2023-12-27 11:30:52,799][20347] Updated weights for policy 0, policy_version 848495 (0.0009) [2023-12-27 11:30:53,072][20381] Updated weights for policy 1, policy_version 847772 (0.0009) [2023-12-27 11:30:53,133][20381] Updated weights for policy 1, policy_version 847782 (0.0008) [2023-12-27 11:30:53,198][20381] Updated weights for policy 1, policy_version 847792 (0.0007) [2023-12-27 11:30:53,536][20347] Updated weights for policy 0, policy_version 848505 (0.0006) [2023-12-27 11:30:53,596][20347] Updated weights for policy 0, policy_version 848515 (0.0008) [2023-12-27 11:30:53,645][20347] Updated weights for policy 0, policy_version 848525 (0.0008) [2023-12-27 11:30:53,818][20381] Updated weights for policy 1, policy_version 847802 (0.0006) [2023-12-27 11:30:53,885][20381] Updated weights for policy 1, policy_version 847812 (0.0007) [2023-12-27 11:30:53,946][20381] Updated weights for policy 1, policy_version 847822 (0.0008) [2023-12-27 11:30:54,005][20381] Updated weights for policy 1, policy_version 847832 (0.0008) [2023-12-27 11:30:54,261][20347] Updated weights for policy 0, policy_version 848535 (0.0007) [2023-12-27 11:30:54,314][20347] Updated weights for policy 0, policy_version 848545 (0.0008) [2023-12-27 11:30:54,371][20347] Updated weights for policy 0, policy_version 848555 (0.0008) [2023-12-27 11:30:54,732][20381] Updated weights for policy 1, policy_version 847842 (0.0008) [2023-12-27 11:30:54,796][20381] Updated weights for policy 1, policy_version 847852 (0.0008) [2023-12-27 11:30:54,850][20381] Updated weights for policy 1, policy_version 847862 (0.0008) [2023-12-27 11:30:55,107][20347] Updated weights for policy 0, policy_version 848565 (0.0007) [2023-12-27 11:30:55,161][20347] Updated weights for policy 0, policy_version 848575 (0.0008) [2023-12-27 11:30:55,220][20347] Updated weights for policy 0, policy_version 848585 (0.0007) [2023-12-27 11:30:55,562][20381] Updated weights for policy 1, policy_version 847872 (0.0007) [2023-12-27 11:30:55,618][20381] Updated weights for policy 1, policy_version 847882 (0.0007) [2023-12-27 11:30:55,663][20381] Updated weights for policy 1, policy_version 847892 (0.0006) [2023-12-27 11:30:55,940][20347] Updated weights for policy 0, policy_version 848595 (0.0008) [2023-12-27 11:30:55,998][20347] Updated weights for policy 0, policy_version 848605 (0.0008) [2023-12-27 11:30:56,060][20347] Updated weights for policy 0, policy_version 848615 (0.0008) [2023-12-27 11:30:56,368][20381] Updated weights for policy 1, policy_version 847902 (0.0008) [2023-12-27 11:30:56,431][20381] Updated weights for policy 1, policy_version 847912 (0.0008) [2023-12-27 11:30:56,490][20381] Updated weights for policy 1, policy_version 847922 (0.0008) [2023-12-27 11:30:56,852][20347] Updated weights for policy 0, policy_version 848625 (0.0007) [2023-12-27 11:30:56,911][20347] Updated weights for policy 0, policy_version 848635 (0.0007) [2023-12-27 11:30:56,975][20347] Updated weights for policy 0, policy_version 848645 (0.0007) [2023-12-27 11:30:57,036][20347] Updated weights for policy 0, policy_version 848655 (0.0007) [2023-12-27 11:30:57,213][20381] Updated weights for policy 1, policy_version 847932 (0.0006) [2023-12-27 11:30:57,270][20381] Updated weights for policy 1, policy_version 847942 (0.0006) [2023-12-27 11:30:57,327][20381] Updated weights for policy 1, policy_version 847952 (0.0007) [2023-12-27 11:30:57,362][19170] Fps is (10 sec: 18841.4, 60 sec: 18978.2, 300 sec: 19438.6). Total num frames: 434380800. Throughput: 0: 9462.8, 1: 9552.6. Samples: 434375900. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:30:57,363][19170] Avg episode reward: [(0, '308.020'), (1, '805.990')] [2023-12-27 11:30:57,738][20347] Updated weights for policy 0, policy_version 848665 (0.0008) [2023-12-27 11:30:57,788][20347] Updated weights for policy 0, policy_version 848675 (0.0008) [2023-12-27 11:30:57,835][20347] Updated weights for policy 0, policy_version 848685 (0.0007) [2023-12-27 11:30:57,935][20381] Updated weights for policy 1, policy_version 847962 (0.0006) [2023-12-27 11:30:57,994][20381] Updated weights for policy 1, policy_version 847972 (0.0008) [2023-12-27 11:30:58,054][20381] Updated weights for policy 1, policy_version 847982 (0.0008) [2023-12-27 11:30:58,115][20381] Updated weights for policy 1, policy_version 847992 (0.0006) [2023-12-27 11:30:58,596][20347] Updated weights for policy 0, policy_version 848695 (0.0008) [2023-12-27 11:30:58,664][20347] Updated weights for policy 0, policy_version 848705 (0.0007) [2023-12-27 11:30:58,731][20347] Updated weights for policy 0, policy_version 848715 (0.0008) [2023-12-27 11:30:58,892][20381] Updated weights for policy 1, policy_version 848002 (0.0008) [2023-12-27 11:30:58,969][20381] Updated weights for policy 1, policy_version 848012 (0.0010) [2023-12-27 11:30:59,036][20381] Updated weights for policy 1, policy_version 848022 (0.0008) [2023-12-27 11:30:59,693][20347] Updated weights for policy 0, policy_version 848725 (0.0007) [2023-12-27 11:30:59,752][20347] Updated weights for policy 0, policy_version 848735 (0.0008) [2023-12-27 11:30:59,811][20347] Updated weights for policy 0, policy_version 848745 (0.0007) [2023-12-27 11:30:59,896][20381] Updated weights for policy 1, policy_version 848032 (0.0007) [2023-12-27 11:30:59,964][20381] Updated weights for policy 1, policy_version 848042 (0.0008) [2023-12-27 11:31:00,022][20381] Updated weights for policy 1, policy_version 848052 (0.0008) [2023-12-27 11:31:00,553][20347] Updated weights for policy 0, policy_version 848755 (0.0008) [2023-12-27 11:31:00,603][20347] Updated weights for policy 0, policy_version 848765 (0.0008) [2023-12-27 11:31:00,665][20347] Updated weights for policy 0, policy_version 848775 (0.0008) [2023-12-27 11:31:00,748][20381] Updated weights for policy 1, policy_version 848062 (0.0008) [2023-12-27 11:31:00,802][20381] Updated weights for policy 1, policy_version 848072 (0.0008) [2023-12-27 11:31:00,850][20381] Updated weights for policy 1, policy_version 848082 (0.0008) [2023-12-27 11:31:01,381][20347] Updated weights for policy 0, policy_version 848785 (0.0007) [2023-12-27 11:31:01,441][20347] Updated weights for policy 0, policy_version 848795 (0.0008) [2023-12-27 11:31:01,499][20347] Updated weights for policy 0, policy_version 848805 (0.0008) [2023-12-27 11:31:01,559][20347] Updated weights for policy 0, policy_version 848815 (0.0008) [2023-12-27 11:31:01,577][20381] Updated weights for policy 1, policy_version 848092 (0.0006) [2023-12-27 11:31:01,641][20381] Updated weights for policy 1, policy_version 848102 (0.0009) [2023-12-27 11:31:01,698][20381] Updated weights for policy 1, policy_version 848112 (0.0008) [2023-12-27 11:31:02,283][20347] Updated weights for policy 0, policy_version 848825 (0.0007) [2023-12-27 11:31:02,346][20347] Updated weights for policy 0, policy_version 848835 (0.0007) [2023-12-27 11:31:02,362][19170] Fps is (10 sec: 18842.7, 60 sec: 18978.1, 300 sec: 19410.9). Total num frames: 434470912. Throughput: 0: 9420.1, 1: 9502.6. Samples: 434487400. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:02,363][19170] Avg episode reward: [(0, '284.440'), (1, '854.610')] [2023-12-27 11:31:02,406][20347] Updated weights for policy 0, policy_version 848845 (0.0007) [2023-12-27 11:31:02,425][20381] Updated weights for policy 1, policy_version 848122 (0.0008) [2023-12-27 11:31:02,484][20381] Updated weights for policy 1, policy_version 848132 (0.0006) [2023-12-27 11:31:02,543][20381] Updated weights for policy 1, policy_version 848142 (0.0008) [2023-12-27 11:31:02,603][20381] Updated weights for policy 1, policy_version 848152 (0.0008) [2023-12-27 11:31:03,096][20347] Updated weights for policy 0, policy_version 848855 (0.0007) [2023-12-27 11:31:03,151][20347] Updated weights for policy 0, policy_version 848865 (0.0007) [2023-12-27 11:31:03,207][20347] Updated weights for policy 0, policy_version 848875 (0.0008) [2023-12-27 11:31:03,307][20381] Updated weights for policy 1, policy_version 848162 (0.0008) [2023-12-27 11:31:03,362][20381] Updated weights for policy 1, policy_version 848172 (0.0008) [2023-12-27 11:31:03,412][20381] Updated weights for policy 1, policy_version 848182 (0.0008) [2023-12-27 11:31:03,928][20347] Updated weights for policy 0, policy_version 848885 (0.0007) [2023-12-27 11:31:03,991][20347] Updated weights for policy 0, policy_version 848895 (0.0007) [2023-12-27 11:31:04,094][20347] Updated weights for policy 0, policy_version 848908 (0.0017) [2023-12-27 11:31:04,152][20381] Updated weights for policy 1, policy_version 848192 (0.0009) [2023-12-27 11:31:04,219][20381] Updated weights for policy 1, policy_version 848202 (0.0008) [2023-12-27 11:31:04,282][20381] Updated weights for policy 1, policy_version 848212 (0.0008) [2023-12-27 11:31:04,819][20347] Updated weights for policy 0, policy_version 848918 (0.0007) [2023-12-27 11:31:04,876][20347] Updated weights for policy 0, policy_version 848928 (0.0008) [2023-12-27 11:31:04,935][20347] Updated weights for policy 0, policy_version 848938 (0.0007) [2023-12-27 11:31:04,971][20381] Updated weights for policy 1, policy_version 848222 (0.0008) [2023-12-27 11:31:05,033][20381] Updated weights for policy 1, policy_version 848232 (0.0009) [2023-12-27 11:31:05,090][20381] Updated weights for policy 1, policy_version 848242 (0.0008) [2023-12-27 11:31:05,591][20347] Updated weights for policy 0, policy_version 848948 (0.0008) [2023-12-27 11:31:05,652][20347] Updated weights for policy 0, policy_version 848958 (0.0007) [2023-12-27 11:31:05,711][20347] Updated weights for policy 0, policy_version 848968 (0.0007) [2023-12-27 11:31:05,806][20381] Updated weights for policy 1, policy_version 848252 (0.0007) [2023-12-27 11:31:05,861][20381] Updated weights for policy 1, policy_version 848262 (0.0009) [2023-12-27 11:31:05,907][20381] Updated weights for policy 1, policy_version 848272 (0.0009) [2023-12-27 11:31:06,378][20347] Updated weights for policy 0, policy_version 848978 (0.0007) [2023-12-27 11:31:06,443][20347] Updated weights for policy 0, policy_version 848988 (0.0006) [2023-12-27 11:31:06,511][20347] Updated weights for policy 0, policy_version 848998 (0.0006) [2023-12-27 11:31:06,573][20347] Updated weights for policy 0, policy_version 849008 (0.0006) [2023-12-27 11:31:06,677][20381] Updated weights for policy 1, policy_version 848282 (0.0009) [2023-12-27 11:31:06,737][20381] Updated weights for policy 1, policy_version 848292 (0.0008) [2023-12-27 11:31:06,798][20381] Updated weights for policy 1, policy_version 848302 (0.0008) [2023-12-27 11:31:06,857][20381] Updated weights for policy 1, policy_version 848312 (0.0008) [2023-12-27 11:31:07,206][20347] Updated weights for policy 0, policy_version 849018 (0.0006) [2023-12-27 11:31:07,274][20347] Updated weights for policy 0, policy_version 849028 (0.0007) [2023-12-27 11:31:07,330][20347] Updated weights for policy 0, policy_version 849038 (0.0008) [2023-12-27 11:31:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19114.7, 300 sec: 19466.4). Total num frames: 434577408. Throughput: 0: 9427.3, 1: 9503.9. Samples: 434545320. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:07,363][19170] Avg episode reward: [(0, '274.010'), (1, '834.560')] [2023-12-27 11:31:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000848312_217194496.pth... [2023-12-27 11:31:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000849040_217382912.pth... [2023-12-27 11:31:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000847192_216907776.pth [2023-12-27 11:31:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000847920_217096192.pth [2023-12-27 11:31:07,549][20381] Updated weights for policy 1, policy_version 848322 (0.0008) [2023-12-27 11:31:07,600][20381] Updated weights for policy 1, policy_version 848332 (0.0008) [2023-12-27 11:31:07,647][20381] Updated weights for policy 1, policy_version 848342 (0.0008) [2023-12-27 11:31:08,033][20347] Updated weights for policy 0, policy_version 849048 (0.0008) [2023-12-27 11:31:08,087][20347] Updated weights for policy 0, policy_version 849058 (0.0008) [2023-12-27 11:31:08,139][20347] Updated weights for policy 0, policy_version 849068 (0.0009) [2023-12-27 11:31:08,380][20381] Updated weights for policy 1, policy_version 848352 (0.0008) [2023-12-27 11:31:08,442][20381] Updated weights for policy 1, policy_version 848362 (0.0008) [2023-12-27 11:31:08,498][20381] Updated weights for policy 1, policy_version 848372 (0.0008) [2023-12-27 11:31:08,887][20347] Updated weights for policy 0, policy_version 849078 (0.0008) [2023-12-27 11:31:08,943][20347] Updated weights for policy 0, policy_version 849088 (0.0008) [2023-12-27 11:31:09,001][20347] Updated weights for policy 0, policy_version 849098 (0.0008) [2023-12-27 11:31:09,231][20381] Updated weights for policy 1, policy_version 848382 (0.0007) [2023-12-27 11:31:09,292][20381] Updated weights for policy 1, policy_version 848392 (0.0008) [2023-12-27 11:31:09,355][20381] Updated weights for policy 1, policy_version 848402 (0.0009) [2023-12-27 11:31:09,770][20347] Updated weights for policy 0, policy_version 849108 (0.0008) [2023-12-27 11:31:09,840][20347] Updated weights for policy 0, policy_version 849118 (0.0008) [2023-12-27 11:31:09,903][20347] Updated weights for policy 0, policy_version 849128 (0.0007) [2023-12-27 11:31:10,189][20381] Updated weights for policy 1, policy_version 848412 (0.0008) [2023-12-27 11:31:10,250][20381] Updated weights for policy 1, policy_version 848422 (0.0008) [2023-12-27 11:31:10,313][20381] Updated weights for policy 1, policy_version 848432 (0.0008) [2023-12-27 11:31:10,664][20347] Updated weights for policy 0, policy_version 849138 (0.0007) [2023-12-27 11:31:10,718][20347] Updated weights for policy 0, policy_version 849148 (0.0008) [2023-12-27 11:31:10,782][20347] Updated weights for policy 0, policy_version 849158 (0.0007) [2023-12-27 11:31:10,848][20347] Updated weights for policy 0, policy_version 849168 (0.0009) [2023-12-27 11:31:11,010][20381] Updated weights for policy 1, policy_version 848442 (0.0008) [2023-12-27 11:31:11,074][20381] Updated weights for policy 1, policy_version 848452 (0.0008) [2023-12-27 11:31:11,126][20381] Updated weights for policy 1, policy_version 848462 (0.0007) [2023-12-27 11:31:11,200][20381] Updated weights for policy 1, policy_version 848472 (0.0008) [2023-12-27 11:31:11,634][20347] Updated weights for policy 0, policy_version 849178 (0.0008) [2023-12-27 11:31:11,700][20347] Updated weights for policy 0, policy_version 849188 (0.0008) [2023-12-27 11:31:11,777][20347] Updated weights for policy 0, policy_version 849198 (0.0008) [2023-12-27 11:31:12,008][20381] Updated weights for policy 1, policy_version 848482 (0.0007) [2023-12-27 11:31:12,074][20381] Updated weights for policy 1, policy_version 848492 (0.0008) [2023-12-27 11:31:12,139][20381] Updated weights for policy 1, policy_version 848502 (0.0008) [2023-12-27 11:31:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 18978.1, 300 sec: 19438.6). Total num frames: 434667520. Throughput: 0: 9499.7, 1: 9526.1. Samples: 434660868. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:12,363][19170] Avg episode reward: [(0, '385.040'), (1, '864.070')] [2023-12-27 11:31:12,540][20347] Updated weights for policy 0, policy_version 849208 (0.0008) [2023-12-27 11:31:12,597][20347] Updated weights for policy 0, policy_version 849218 (0.0007) [2023-12-27 11:31:12,654][20347] Updated weights for policy 0, policy_version 849228 (0.0006) [2023-12-27 11:31:12,893][20381] Updated weights for policy 1, policy_version 848512 (0.0008) [2023-12-27 11:31:12,955][20381] Updated weights for policy 1, policy_version 848522 (0.0008) [2023-12-27 11:31:13,016][20381] Updated weights for policy 1, policy_version 848532 (0.0008) [2023-12-27 11:31:13,354][20347] Updated weights for policy 0, policy_version 849238 (0.0008) [2023-12-27 11:31:13,406][20347] Updated weights for policy 0, policy_version 849248 (0.0007) [2023-12-27 11:31:13,470][20347] Updated weights for policy 0, policy_version 849258 (0.0008) [2023-12-27 11:31:13,729][20381] Updated weights for policy 1, policy_version 848542 (0.0008) [2023-12-27 11:31:13,794][20381] Updated weights for policy 1, policy_version 848552 (0.0009) [2023-12-27 11:31:13,846][20381] Updated weights for policy 1, policy_version 848562 (0.0009) [2023-12-27 11:31:14,160][20347] Updated weights for policy 0, policy_version 849268 (0.0008) [2023-12-27 11:31:14,224][20347] Updated weights for policy 0, policy_version 849278 (0.0008) [2023-12-27 11:31:14,272][20347] Updated weights for policy 0, policy_version 849288 (0.0008) [2023-12-27 11:31:14,546][20381] Updated weights for policy 1, policy_version 848572 (0.0009) [2023-12-27 11:31:14,605][20381] Updated weights for policy 1, policy_version 848582 (0.0008) [2023-12-27 11:31:14,662][20381] Updated weights for policy 1, policy_version 848592 (0.0009) [2023-12-27 11:31:15,006][20347] Updated weights for policy 0, policy_version 849298 (0.0008) [2023-12-27 11:31:15,076][20347] Updated weights for policy 0, policy_version 849308 (0.0008) [2023-12-27 11:31:15,145][20347] Updated weights for policy 0, policy_version 849318 (0.0008) [2023-12-27 11:31:15,211][20347] Updated weights for policy 0, policy_version 849328 (0.0009) [2023-12-27 11:31:15,383][20381] Updated weights for policy 1, policy_version 848602 (0.0008) [2023-12-27 11:31:15,446][20381] Updated weights for policy 1, policy_version 848612 (0.0006) [2023-12-27 11:31:15,506][20381] Updated weights for policy 1, policy_version 848622 (0.0008) [2023-12-27 11:31:15,564][20381] Updated weights for policy 1, policy_version 848632 (0.0008) [2023-12-27 11:31:15,908][20347] Updated weights for policy 0, policy_version 849338 (0.0008) [2023-12-27 11:31:15,966][20347] Updated weights for policy 0, policy_version 849348 (0.0008) [2023-12-27 11:31:16,025][20347] Updated weights for policy 0, policy_version 849358 (0.0007) [2023-12-27 11:31:16,258][20381] Updated weights for policy 1, policy_version 848642 (0.0006) [2023-12-27 11:31:16,322][20381] Updated weights for policy 1, policy_version 848652 (0.0006) [2023-12-27 11:31:16,383][20381] Updated weights for policy 1, policy_version 848662 (0.0006) [2023-12-27 11:31:16,733][20347] Updated weights for policy 0, policy_version 849368 (0.0008) [2023-12-27 11:31:16,795][20347] Updated weights for policy 0, policy_version 849378 (0.0007) [2023-12-27 11:31:16,855][20347] Updated weights for policy 0, policy_version 849388 (0.0008) [2023-12-27 11:31:17,041][20381] Updated weights for policy 1, policy_version 848672 (0.0008) [2023-12-27 11:31:17,105][20381] Updated weights for policy 1, policy_version 848682 (0.0008) [2023-12-27 11:31:17,175][20381] Updated weights for policy 1, policy_version 848692 (0.0009) [2023-12-27 11:31:17,363][19170] Fps is (10 sec: 18840.9, 60 sec: 18978.1, 300 sec: 19466.4). Total num frames: 434765824. Throughput: 0: 9527.8, 1: 9581.6. Samples: 434775328. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:17,364][19170] Avg episode reward: [(0, '285.770'), (1, '873.760')] [2023-12-27 11:31:17,585][20347] Updated weights for policy 0, policy_version 849398 (0.0008) [2023-12-27 11:31:17,649][20347] Updated weights for policy 0, policy_version 849408 (0.0007) [2023-12-27 11:31:17,734][20347] Updated weights for policy 0, policy_version 849418 (0.0008) [2023-12-27 11:31:17,888][20381] Updated weights for policy 1, policy_version 848702 (0.0008) [2023-12-27 11:31:17,938][20381] Updated weights for policy 1, policy_version 848712 (0.0008) [2023-12-27 11:31:17,985][20381] Updated weights for policy 1, policy_version 848722 (0.0007) [2023-12-27 11:31:18,423][20347] Updated weights for policy 0, policy_version 849428 (0.0006) [2023-12-27 11:31:18,489][20347] Updated weights for policy 0, policy_version 849438 (0.0010) [2023-12-27 11:31:18,546][20347] Updated weights for policy 0, policy_version 849448 (0.0008) [2023-12-27 11:31:18,703][20381] Updated weights for policy 1, policy_version 848732 (0.0008) [2023-12-27 11:31:18,760][20381] Updated weights for policy 1, policy_version 848742 (0.0008) [2023-12-27 11:31:18,821][20381] Updated weights for policy 1, policy_version 848752 (0.0009) [2023-12-27 11:31:19,213][20347] Updated weights for policy 0, policy_version 849458 (0.0007) [2023-12-27 11:31:19,283][20347] Updated weights for policy 0, policy_version 849468 (0.0006) [2023-12-27 11:31:19,349][20347] Updated weights for policy 0, policy_version 849478 (0.0009) [2023-12-27 11:31:19,419][20347] Updated weights for policy 0, policy_version 849488 (0.0008) [2023-12-27 11:31:19,493][20381] Updated weights for policy 1, policy_version 848762 (0.0010) [2023-12-27 11:31:19,556][20381] Updated weights for policy 1, policy_version 848772 (0.0007) [2023-12-27 11:31:19,614][20381] Updated weights for policy 1, policy_version 848782 (0.0008) [2023-12-27 11:31:19,671][20381] Updated weights for policy 1, policy_version 848792 (0.0008) [2023-12-27 11:31:20,170][20347] Updated weights for policy 0, policy_version 849498 (0.0008) [2023-12-27 11:31:20,232][20347] Updated weights for policy 0, policy_version 849508 (0.0007) [2023-12-27 11:31:20,291][20347] Updated weights for policy 0, policy_version 849518 (0.0008) [2023-12-27 11:31:20,426][20381] Updated weights for policy 1, policy_version 848802 (0.0007) [2023-12-27 11:31:20,487][20381] Updated weights for policy 1, policy_version 848812 (0.0008) [2023-12-27 11:31:20,553][20381] Updated weights for policy 1, policy_version 848822 (0.0008) [2023-12-27 11:31:21,058][20347] Updated weights for policy 0, policy_version 849528 (0.0010) [2023-12-27 11:31:21,127][20347] Updated weights for policy 0, policy_version 849538 (0.0011) [2023-12-27 11:31:21,196][20347] Updated weights for policy 0, policy_version 849548 (0.0010) [2023-12-27 11:31:21,314][20381] Updated weights for policy 1, policy_version 848832 (0.0007) [2023-12-27 11:31:21,393][20381] Updated weights for policy 1, policy_version 848842 (0.0009) [2023-12-27 11:31:21,457][20381] Updated weights for policy 1, policy_version 848852 (0.0008) [2023-12-27 11:31:21,929][20347] Updated weights for policy 0, policy_version 849558 (0.0010) [2023-12-27 11:31:21,988][20347] Updated weights for policy 0, policy_version 849568 (0.0010) [2023-12-27 11:31:22,050][20347] Updated weights for policy 0, policy_version 849578 (0.0011) [2023-12-27 11:31:22,228][20381] Updated weights for policy 1, policy_version 848862 (0.0008) [2023-12-27 11:31:22,294][20381] Updated weights for policy 1, policy_version 848872 (0.0008) [2023-12-27 11:31:22,359][20381] Updated weights for policy 1, policy_version 848882 (0.0008) [2023-12-27 11:31:22,362][19170] Fps is (10 sec: 18841.7, 60 sec: 18978.2, 300 sec: 19438.6). Total num frames: 434855936. Throughput: 0: 9550.0, 1: 9605.7. Samples: 434833296. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:22,363][19170] Avg episode reward: [(0, '252.140'), (1, '805.660')] [2023-12-27 11:31:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000849584_217522176.pth... [2023-12-27 11:31:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000848464_217235456.pth [2023-12-27 11:31:22,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000848888_217341952.pth... [2023-12-27 11:31:22,401][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000847768_217055232.pth [2023-12-27 11:31:22,771][20347] Updated weights for policy 0, policy_version 849588 (0.0008) [2023-12-27 11:31:22,830][20347] Updated weights for policy 0, policy_version 849598 (0.0007) [2023-12-27 11:31:22,889][20347] Updated weights for policy 0, policy_version 849608 (0.0007) [2023-12-27 11:31:23,159][20381] Updated weights for policy 1, policy_version 848892 (0.0008) [2023-12-27 11:31:23,211][20381] Updated weights for policy 1, policy_version 848902 (0.0008) [2023-12-27 11:31:23,264][20381] Updated weights for policy 1, policy_version 848912 (0.0010) [2023-12-27 11:31:23,557][20347] Updated weights for policy 0, policy_version 849618 (0.0006) [2023-12-27 11:31:23,616][20347] Updated weights for policy 0, policy_version 849628 (0.0007) [2023-12-27 11:31:23,677][20347] Updated weights for policy 0, policy_version 849638 (0.0008) [2023-12-27 11:31:23,731][20347] Updated weights for policy 0, policy_version 849648 (0.0008) [2023-12-27 11:31:24,002][20381] Updated weights for policy 1, policy_version 848922 (0.0008) [2023-12-27 11:31:24,064][20381] Updated weights for policy 1, policy_version 848932 (0.0008) [2023-12-27 11:31:24,114][20381] Updated weights for policy 1, policy_version 848942 (0.0008) [2023-12-27 11:31:24,168][20381] Updated weights for policy 1, policy_version 848952 (0.0007) [2023-12-27 11:31:24,429][20347] Updated weights for policy 0, policy_version 849658 (0.0008) [2023-12-27 11:31:24,485][20347] Updated weights for policy 0, policy_version 849668 (0.0009) [2023-12-27 11:31:24,538][20347] Updated weights for policy 0, policy_version 849678 (0.0009) [2023-12-27 11:31:24,863][20381] Updated weights for policy 1, policy_version 848962 (0.0009) [2023-12-27 11:31:24,914][20381] Updated weights for policy 1, policy_version 848972 (0.0007) [2023-12-27 11:31:24,977][20381] Updated weights for policy 1, policy_version 848982 (0.0007) [2023-12-27 11:31:25,232][20347] Updated weights for policy 0, policy_version 849688 (0.0010) [2023-12-27 11:31:25,291][20347] Updated weights for policy 0, policy_version 849698 (0.0009) [2023-12-27 11:31:25,352][20347] Updated weights for policy 0, policy_version 849708 (0.0010) [2023-12-27 11:31:25,726][20381] Updated weights for policy 1, policy_version 848992 (0.0007) [2023-12-27 11:31:25,788][20381] Updated weights for policy 1, policy_version 849002 (0.0008) [2023-12-27 11:31:25,849][20381] Updated weights for policy 1, policy_version 849012 (0.0008) [2023-12-27 11:31:26,030][20347] Updated weights for policy 0, policy_version 849718 (0.0010) [2023-12-27 11:31:26,085][20347] Updated weights for policy 0, policy_version 849728 (0.0010) [2023-12-27 11:31:26,138][20347] Updated weights for policy 0, policy_version 849738 (0.0010) [2023-12-27 11:31:26,514][20381] Updated weights for policy 1, policy_version 849022 (0.0007) [2023-12-27 11:31:26,574][20381] Updated weights for policy 1, policy_version 849032 (0.0007) [2023-12-27 11:31:26,630][20381] Updated weights for policy 1, policy_version 849042 (0.0008) [2023-12-27 11:31:26,812][20347] Updated weights for policy 0, policy_version 849748 (0.0010) [2023-12-27 11:31:26,870][20347] Updated weights for policy 0, policy_version 849758 (0.0010) [2023-12-27 11:31:26,929][20347] Updated weights for policy 0, policy_version 849768 (0.0010) [2023-12-27 11:31:27,292][20381] Updated weights for policy 1, policy_version 849052 (0.0007) [2023-12-27 11:31:27,350][20381] Updated weights for policy 1, policy_version 849062 (0.0006) [2023-12-27 11:31:27,362][19170] Fps is (10 sec: 18842.3, 60 sec: 19114.7, 300 sec: 19438.6). Total num frames: 434954240. Throughput: 0: 9573.0, 1: 9587.8. Samples: 434947452. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:27,363][19170] Avg episode reward: [(0, '323.550'), (1, '835.030')] [2023-12-27 11:31:27,410][20381] Updated weights for policy 1, policy_version 849072 (0.0006) [2023-12-27 11:31:27,595][20347] Updated weights for policy 0, policy_version 849778 (0.0009) [2023-12-27 11:31:27,654][20347] Updated weights for policy 0, policy_version 849788 (0.0006) [2023-12-27 11:31:27,708][20347] Updated weights for policy 0, policy_version 849798 (0.0008) [2023-12-27 11:31:27,769][20347] Updated weights for policy 0, policy_version 849808 (0.0008) [2023-12-27 11:31:28,097][20381] Updated weights for policy 1, policy_version 849082 (0.0007) [2023-12-27 11:31:28,155][20381] Updated weights for policy 1, policy_version 849092 (0.0008) [2023-12-27 11:31:28,211][20381] Updated weights for policy 1, policy_version 849102 (0.0007) [2023-12-27 11:31:28,270][20381] Updated weights for policy 1, policy_version 849112 (0.0008) [2023-12-27 11:31:28,451][20347] Updated weights for policy 0, policy_version 849818 (0.0007) [2023-12-27 11:31:28,518][20347] Updated weights for policy 0, policy_version 849828 (0.0006) [2023-12-27 11:31:28,584][20347] Updated weights for policy 0, policy_version 849838 (0.0006) [2023-12-27 11:31:28,997][20381] Updated weights for policy 1, policy_version 849122 (0.0008) [2023-12-27 11:31:29,053][20381] Updated weights for policy 1, policy_version 849132 (0.0008) [2023-12-27 11:31:29,110][20381] Updated weights for policy 1, policy_version 849142 (0.0008) [2023-12-27 11:31:29,187][20347] Updated weights for policy 0, policy_version 849848 (0.0006) [2023-12-27 11:31:29,256][20347] Updated weights for policy 0, policy_version 849858 (0.0007) [2023-12-27 11:31:29,311][20347] Updated weights for policy 0, policy_version 849868 (0.0008) [2023-12-27 11:31:29,921][20381] Updated weights for policy 1, policy_version 849152 (0.0008) [2023-12-27 11:31:29,978][20381] Updated weights for policy 1, policy_version 849162 (0.0008) [2023-12-27 11:31:30,038][20381] Updated weights for policy 1, policy_version 849172 (0.0008) [2023-12-27 11:31:30,047][20347] Updated weights for policy 0, policy_version 849878 (0.0008) [2023-12-27 11:31:30,107][20347] Updated weights for policy 0, policy_version 849888 (0.0007) [2023-12-27 11:31:30,161][20347] Updated weights for policy 0, policy_version 849898 (0.0006) [2023-12-27 11:31:30,757][20381] Updated weights for policy 1, policy_version 849182 (0.0007) [2023-12-27 11:31:30,811][20381] Updated weights for policy 1, policy_version 849192 (0.0006) [2023-12-27 11:31:30,873][20381] Updated weights for policy 1, policy_version 849202 (0.0005) [2023-12-27 11:31:30,886][20347] Updated weights for policy 0, policy_version 849908 (0.0006) [2023-12-27 11:31:30,933][20347] Updated weights for policy 0, policy_version 849918 (0.0006) [2023-12-27 11:31:31,002][20347] Updated weights for policy 0, policy_version 849928 (0.0007) [2023-12-27 11:31:31,594][20381] Updated weights for policy 1, policy_version 849212 (0.0006) [2023-12-27 11:31:31,666][20381] Updated weights for policy 1, policy_version 849222 (0.0008) [2023-12-27 11:31:31,710][20347] Updated weights for policy 0, policy_version 849938 (0.0007) [2023-12-27 11:31:31,728][20381] Updated weights for policy 1, policy_version 849232 (0.0007) [2023-12-27 11:31:31,775][20347] Updated weights for policy 0, policy_version 849948 (0.0007) [2023-12-27 11:31:31,836][20347] Updated weights for policy 0, policy_version 849958 (0.0006) [2023-12-27 11:31:31,896][20347] Updated weights for policy 0, policy_version 849968 (0.0007) [2023-12-27 11:31:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19438.7). Total num frames: 435052544. Throughput: 0: 9622.8, 1: 9590.8. Samples: 435065192. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:32,363][19170] Avg episode reward: [(0, '339.420'), (1, '863.880')] [2023-12-27 11:31:32,475][20381] Updated weights for policy 1, policy_version 849242 (0.0008) [2023-12-27 11:31:32,543][20381] Updated weights for policy 1, policy_version 849252 (0.0007) [2023-12-27 11:31:32,610][20381] Updated weights for policy 1, policy_version 849262 (0.0008) [2023-12-27 11:31:32,615][20347] Updated weights for policy 0, policy_version 849978 (0.0006) [2023-12-27 11:31:32,671][20381] Updated weights for policy 1, policy_version 849272 (0.0007) [2023-12-27 11:31:32,674][20347] Updated weights for policy 0, policy_version 849988 (0.0007) [2023-12-27 11:31:32,733][20347] Updated weights for policy 0, policy_version 849998 (0.0008) [2023-12-27 11:31:33,258][20381] Updated weights for policy 1, policy_version 849282 (0.0006) [2023-12-27 11:31:33,304][20381] Updated weights for policy 1, policy_version 849292 (0.0008) [2023-12-27 11:31:33,362][20381] Updated weights for policy 1, policy_version 849302 (0.0008) [2023-12-27 11:31:33,406][20347] Updated weights for policy 0, policy_version 850008 (0.0007) [2023-12-27 11:31:33,466][20347] Updated weights for policy 0, policy_version 850018 (0.0008) [2023-12-27 11:31:33,528][20347] Updated weights for policy 0, policy_version 850028 (0.0007) [2023-12-27 11:31:34,045][20381] Updated weights for policy 1, policy_version 849312 (0.0007) [2023-12-27 11:31:34,104][20381] Updated weights for policy 1, policy_version 849322 (0.0007) [2023-12-27 11:31:34,163][20381] Updated weights for policy 1, policy_version 849332 (0.0008) [2023-12-27 11:31:34,243][20347] Updated weights for policy 0, policy_version 850038 (0.0008) [2023-12-27 11:31:34,303][20347] Updated weights for policy 0, policy_version 850048 (0.0008) [2023-12-27 11:31:34,371][20347] Updated weights for policy 0, policy_version 850058 (0.0008) [2023-12-27 11:31:34,944][20381] Updated weights for policy 1, policy_version 849342 (0.0006) [2023-12-27 11:31:35,003][20381] Updated weights for policy 1, policy_version 849352 (0.0006) [2023-12-27 11:31:35,065][20381] Updated weights for policy 1, policy_version 849362 (0.0008) [2023-12-27 11:31:35,131][20347] Updated weights for policy 0, policy_version 850068 (0.0008) [2023-12-27 11:31:35,187][20347] Updated weights for policy 0, policy_version 850078 (0.0007) [2023-12-27 11:31:35,249][20347] Updated weights for policy 0, policy_version 850088 (0.0007) [2023-12-27 11:31:35,715][20381] Updated weights for policy 1, policy_version 849372 (0.0008) [2023-12-27 11:31:35,767][20381] Updated weights for policy 1, policy_version 849382 (0.0008) [2023-12-27 11:31:35,773][20292] KL-divergence is very high: 117.0085 [2023-12-27 11:31:35,829][20381] Updated weights for policy 1, policy_version 849392 (0.0008) [2023-12-27 11:31:35,865][20347] Updated weights for policy 0, policy_version 850098 (0.0007) [2023-12-27 11:31:35,927][20347] Updated weights for policy 0, policy_version 850108 (0.0008) [2023-12-27 11:31:35,989][20347] Updated weights for policy 0, policy_version 850118 (0.0008) [2023-12-27 11:31:36,045][20347] Updated weights for policy 0, policy_version 850128 (0.0009) [2023-12-27 11:31:36,608][20381] Updated weights for policy 1, policy_version 849402 (0.0008) [2023-12-27 11:31:36,673][20381] Updated weights for policy 1, policy_version 849412 (0.0008) [2023-12-27 11:31:36,735][20381] Updated weights for policy 1, policy_version 849422 (0.0008) [2023-12-27 11:31:36,788][20347] Updated weights for policy 0, policy_version 850138 (0.0005) [2023-12-27 11:31:36,794][20381] Updated weights for policy 1, policy_version 849432 (0.0008) [2023-12-27 11:31:36,847][20347] Updated weights for policy 0, policy_version 850148 (0.0008) [2023-12-27 11:31:36,916][20347] Updated weights for policy 0, policy_version 850158 (0.0007) [2023-12-27 11:31:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19114.6, 300 sec: 19438.6). Total num frames: 435150848. Throughput: 0: 9637.9, 1: 9605.0. Samples: 435124192. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:37,364][19170] Avg episode reward: [(0, '306.280'), (1, '883.840')] [2023-12-27 11:31:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000849432_217481216.pth... [2023-12-27 11:31:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000850160_217669632.pth... [2023-12-27 11:31:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000848312_217194496.pth [2023-12-27 11:31:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000849040_217382912.pth [2023-12-27 11:31:37,538][20381] Updated weights for policy 1, policy_version 849442 (0.0006) [2023-12-27 11:31:37,568][20347] Updated weights for policy 0, policy_version 850168 (0.0006) [2023-12-27 11:31:37,590][20381] Updated weights for policy 1, policy_version 849452 (0.0007) [2023-12-27 11:31:37,621][20347] Updated weights for policy 0, policy_version 850178 (0.0006) [2023-12-27 11:31:37,650][20381] Updated weights for policy 1, policy_version 849462 (0.0008) [2023-12-27 11:31:37,677][20347] Updated weights for policy 0, policy_version 850188 (0.0007) [2023-12-27 11:31:38,358][20381] Updated weights for policy 1, policy_version 849472 (0.0008) [2023-12-27 11:31:38,368][20347] Updated weights for policy 0, policy_version 850198 (0.0008) [2023-12-27 11:31:38,421][20381] Updated weights for policy 1, policy_version 849482 (0.0007) [2023-12-27 11:31:38,431][20347] Updated weights for policy 0, policy_version 850208 (0.0007) [2023-12-27 11:31:38,484][20381] Updated weights for policy 1, policy_version 849492 (0.0006) [2023-12-27 11:31:38,490][20347] Updated weights for policy 0, policy_version 850218 (0.0007) [2023-12-27 11:31:39,168][20381] Updated weights for policy 1, policy_version 849502 (0.0006) [2023-12-27 11:31:39,198][20347] Updated weights for policy 0, policy_version 850228 (0.0006) [2023-12-27 11:31:39,228][20381] Updated weights for policy 1, policy_version 849512 (0.0007) [2023-12-27 11:31:39,264][20347] Updated weights for policy 0, policy_version 850238 (0.0007) [2023-12-27 11:31:39,290][20381] Updated weights for policy 1, policy_version 849522 (0.0007) [2023-12-27 11:31:39,325][20347] Updated weights for policy 0, policy_version 850248 (0.0007) [2023-12-27 11:31:40,006][20381] Updated weights for policy 1, policy_version 849532 (0.0008) [2023-12-27 11:31:40,066][20347] Updated weights for policy 0, policy_version 850258 (0.0008) [2023-12-27 11:31:40,072][20381] Updated weights for policy 1, policy_version 849542 (0.0007) [2023-12-27 11:31:40,132][20347] Updated weights for policy 0, policy_version 850268 (0.0011) [2023-12-27 11:31:40,135][20381] Updated weights for policy 1, policy_version 849552 (0.0008) [2023-12-27 11:31:40,201][20347] Updated weights for policy 0, policy_version 850278 (0.0010) [2023-12-27 11:31:40,267][20347] Updated weights for policy 0, policy_version 850288 (0.0009) [2023-12-27 11:31:40,881][20381] Updated weights for policy 1, policy_version 849562 (0.0008) [2023-12-27 11:31:40,946][20381] Updated weights for policy 1, policy_version 849572 (0.0008) [2023-12-27 11:31:40,960][20347] Updated weights for policy 0, policy_version 850298 (0.0006) [2023-12-27 11:31:41,005][20381] Updated weights for policy 1, policy_version 849582 (0.0008) [2023-12-27 11:31:41,019][20347] Updated weights for policy 0, policy_version 850308 (0.0007) [2023-12-27 11:31:41,070][20381] Updated weights for policy 1, policy_version 849592 (0.0007) [2023-12-27 11:31:41,090][20347] Updated weights for policy 0, policy_version 850318 (0.0010) [2023-12-27 11:31:41,837][20381] Updated weights for policy 1, policy_version 849602 (0.0007) [2023-12-27 11:31:41,856][20347] Updated weights for policy 0, policy_version 850328 (0.0009) [2023-12-27 11:31:41,896][20381] Updated weights for policy 1, policy_version 849612 (0.0006) [2023-12-27 11:31:41,916][20347] Updated weights for policy 0, policy_version 850338 (0.0007) [2023-12-27 11:31:41,962][20381] Updated weights for policy 1, policy_version 849622 (0.0006) [2023-12-27 11:31:41,975][20347] Updated weights for policy 0, policy_version 850348 (0.0007) [2023-12-27 11:31:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19251.2, 300 sec: 19438.6). Total num frames: 435249152. Throughput: 0: 9650.5, 1: 9568.1. Samples: 435240736. Policy #0 lag: (min: 18.0, avg: 20.5, max: 41.0) [2023-12-27 11:31:42,363][19170] Avg episode reward: [(0, '287.740'), (1, '893.670')] [2023-12-27 11:31:42,644][20347] Updated weights for policy 0, policy_version 850358 (0.0007) [2023-12-27 11:31:42,649][20292] KL-divergence is very high: 102.4094 [2023-12-27 11:31:42,674][20381] Updated weights for policy 1, policy_version 849632 (0.0007) [2023-12-27 11:31:42,703][20347] Updated weights for policy 0, policy_version 850368 (0.0006) [2023-12-27 11:31:42,736][20381] Updated weights for policy 1, policy_version 849642 (0.0007) [2023-12-27 11:31:42,755][20347] Updated weights for policy 0, policy_version 850378 (0.0006) [2023-12-27 11:31:42,792][20381] Updated weights for policy 1, policy_version 849652 (0.0006) [2023-12-27 11:31:43,476][20347] Updated weights for policy 0, policy_version 850388 (0.0006) [2023-12-27 11:31:43,510][20381] Updated weights for policy 1, policy_version 849662 (0.0007) [2023-12-27 11:31:43,527][20347] Updated weights for policy 0, policy_version 850398 (0.0006) [2023-12-27 11:31:43,564][20381] Updated weights for policy 1, policy_version 849672 (0.0006) [2023-12-27 11:31:43,582][20347] Updated weights for policy 0, policy_version 850408 (0.0006) [2023-12-27 11:31:43,620][20381] Updated weights for policy 1, policy_version 849682 (0.0007) [2023-12-27 11:31:44,240][20347] Updated weights for policy 0, policy_version 850418 (0.0009) [2023-12-27 11:31:44,295][20347] Updated weights for policy 0, policy_version 850428 (0.0006) [2023-12-27 11:31:44,356][20347] Updated weights for policy 0, policy_version 850438 (0.0007) [2023-12-27 11:31:44,366][20381] Updated weights for policy 1, policy_version 849692 (0.0008) [2023-12-27 11:31:44,407][20347] Updated weights for policy 0, policy_version 850448 (0.0006) [2023-12-27 11:31:44,421][20381] Updated weights for policy 1, policy_version 849702 (0.0006) [2023-12-27 11:31:44,482][20381] Updated weights for policy 1, policy_version 849712 (0.0005) [2023-12-27 11:31:45,110][20347] Updated weights for policy 0, policy_version 850458 (0.0009) [2023-12-27 11:31:45,173][20347] Updated weights for policy 0, policy_version 850468 (0.0008) [2023-12-27 11:31:45,192][20381] Updated weights for policy 1, policy_version 849722 (0.0006) [2023-12-27 11:31:45,235][20347] Updated weights for policy 0, policy_version 850478 (0.0008) [2023-12-27 11:31:45,253][20381] Updated weights for policy 1, policy_version 849732 (0.0007) [2023-12-27 11:31:45,315][20381] Updated weights for policy 1, policy_version 849742 (0.0008) [2023-12-27 11:31:45,370][20381] Updated weights for policy 1, policy_version 849752 (0.0008) [2023-12-27 11:31:45,947][20347] Updated weights for policy 0, policy_version 850488 (0.0009) [2023-12-27 11:31:46,004][20347] Updated weights for policy 0, policy_version 850498 (0.0007) [2023-12-27 11:31:46,062][20347] Updated weights for policy 0, policy_version 850508 (0.0008) [2023-12-27 11:31:46,103][20381] Updated weights for policy 1, policy_version 849762 (0.0007) [2023-12-27 11:31:46,162][20381] Updated weights for policy 1, policy_version 849772 (0.0008) [2023-12-27 11:31:46,222][20381] Updated weights for policy 1, policy_version 849782 (0.0008) [2023-12-27 11:31:46,763][20347] Updated weights for policy 0, policy_version 850518 (0.0007) [2023-12-27 11:31:46,821][20347] Updated weights for policy 0, policy_version 850528 (0.0008) [2023-12-27 11:31:46,903][20347] Updated weights for policy 0, policy_version 850538 (0.0007) [2023-12-27 11:31:46,945][20381] Updated weights for policy 1, policy_version 849792 (0.0007) [2023-12-27 11:31:47,004][20381] Updated weights for policy 1, policy_version 849802 (0.0007) [2023-12-27 11:31:47,056][20381] Updated weights for policy 1, policy_version 849812 (0.0007) [2023-12-27 11:31:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19251.2, 300 sec: 19438.7). Total num frames: 435347456. Throughput: 0: 9757.6, 1: 9568.8. Samples: 435357088. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:31:47,363][19170] Avg episode reward: [(0, '266.390'), (1, '931.910')] [2023-12-27 11:31:47,575][20347] Updated weights for policy 0, policy_version 850548 (0.0007) [2023-12-27 11:31:47,636][20347] Updated weights for policy 0, policy_version 850558 (0.0008) [2023-12-27 11:31:47,697][20347] Updated weights for policy 0, policy_version 850568 (0.0007) [2023-12-27 11:31:47,770][20381] Updated weights for policy 1, policy_version 849822 (0.0007) [2023-12-27 11:31:47,832][20381] Updated weights for policy 1, policy_version 849832 (0.0007) [2023-12-27 11:31:47,904][20381] Updated weights for policy 1, policy_version 849842 (0.0008) [2023-12-27 11:31:47,935][20292] KL-divergence is very high: 321.9836 [2023-12-27 11:31:48,398][20347] Updated weights for policy 0, policy_version 850578 (0.0008) [2023-12-27 11:31:48,462][20347] Updated weights for policy 0, policy_version 850588 (0.0008) [2023-12-27 11:31:48,523][20347] Updated weights for policy 0, policy_version 850598 (0.0008) [2023-12-27 11:31:48,588][20347] Updated weights for policy 0, policy_version 850608 (0.0007) [2023-12-27 11:31:48,591][20292] KL-divergence is very high: 297.3244 [2023-12-27 11:31:48,610][20381] Updated weights for policy 1, policy_version 849852 (0.0008) [2023-12-27 11:31:48,672][20381] Updated weights for policy 1, policy_version 849862 (0.0007) [2023-12-27 11:31:48,692][20292] KL-divergence is very high: 234.3394 [2023-12-27 11:31:48,737][20381] Updated weights for policy 1, policy_version 849872 (0.0007) [2023-12-27 11:31:48,745][20292] KL-divergence is very high: 194.1976 [2023-12-27 11:31:49,304][20347] Updated weights for policy 0, policy_version 850618 (0.0008) [2023-12-27 11:31:49,375][20347] Updated weights for policy 0, policy_version 850628 (0.0007) [2023-12-27 11:31:49,434][20347] Updated weights for policy 0, policy_version 850638 (0.0008) [2023-12-27 11:31:49,486][20381] Updated weights for policy 1, policy_version 849882 (0.0008) [2023-12-27 11:31:49,547][20381] Updated weights for policy 1, policy_version 849892 (0.0010) [2023-12-27 11:31:49,606][20381] Updated weights for policy 1, policy_version 849902 (0.0008) [2023-12-27 11:31:49,665][20381] Updated weights for policy 1, policy_version 849912 (0.0007) [2023-12-27 11:31:50,242][20347] Updated weights for policy 0, policy_version 850648 (0.0010) [2023-12-27 11:31:50,299][20347] Updated weights for policy 0, policy_version 850658 (0.0011) [2023-12-27 11:31:50,361][20347] Updated weights for policy 0, policy_version 850668 (0.0011) [2023-12-27 11:31:50,386][20381] Updated weights for policy 1, policy_version 849922 (0.0007) [2023-12-27 11:31:50,444][20381] Updated weights for policy 1, policy_version 849932 (0.0008) [2023-12-27 11:31:50,493][20381] Updated weights for policy 1, policy_version 849942 (0.0007) [2023-12-27 11:31:51,147][20347] Updated weights for policy 0, policy_version 850678 (0.0009) [2023-12-27 11:31:51,163][20381] Updated weights for policy 1, policy_version 849952 (0.0008) [2023-12-27 11:31:51,211][20347] Updated weights for policy 0, policy_version 850688 (0.0008) [2023-12-27 11:31:51,232][20381] Updated weights for policy 1, policy_version 849962 (0.0007) [2023-12-27 11:31:51,282][20347] Updated weights for policy 0, policy_version 850698 (0.0008) [2023-12-27 11:31:51,297][20381] Updated weights for policy 1, policy_version 849972 (0.0007) [2023-12-27 11:31:52,052][20381] Updated weights for policy 1, policy_version 849982 (0.0008) [2023-12-27 11:31:52,073][20347] Updated weights for policy 0, policy_version 850709 (0.0008) [2023-12-27 11:31:52,110][20381] Updated weights for policy 1, policy_version 849992 (0.0011) [2023-12-27 11:31:52,134][20347] Updated weights for policy 0, policy_version 850719 (0.0008) [2023-12-27 11:31:52,174][20381] Updated weights for policy 1, policy_version 850002 (0.0011) [2023-12-27 11:31:52,198][20347] Updated weights for policy 0, policy_version 850729 (0.0008) [2023-12-27 11:31:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.9, 300 sec: 19438.6). Total num frames: 435445760. Throughput: 0: 9747.9, 1: 9560.1. Samples: 435414180. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:31:52,363][19170] Avg episode reward: [(0, '362.080'), (1, '922.750')] [2023-12-27 11:31:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000850008_217628672.pth... [2023-12-27 11:31:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000850736_217817088.pth... [2023-12-27 11:31:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000848888_217341952.pth [2023-12-27 11:31:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000849584_217522176.pth [2023-12-27 11:31:52,888][20381] Updated weights for policy 1, policy_version 850012 (0.0011) [2023-12-27 11:31:52,942][20381] Updated weights for policy 1, policy_version 850022 (0.0009) [2023-12-27 11:31:52,949][20347] Updated weights for policy 0, policy_version 850739 (0.0008) [2023-12-27 11:31:52,996][20381] Updated weights for policy 1, policy_version 850032 (0.0011) [2023-12-27 11:31:53,005][20347] Updated weights for policy 0, policy_version 850749 (0.0006) [2023-12-27 11:31:53,057][20347] Updated weights for policy 0, policy_version 850759 (0.0008) [2023-12-27 11:31:53,653][20381] Updated weights for policy 1, policy_version 850042 (0.0010) [2023-12-27 11:31:53,714][20381] Updated weights for policy 1, policy_version 850052 (0.0008) [2023-12-27 11:31:53,714][20292] KL-divergence is very high: 110.2057 [2023-12-27 11:31:53,772][20381] Updated weights for policy 1, policy_version 850062 (0.0005) [2023-12-27 11:31:53,797][20347] Updated weights for policy 0, policy_version 850769 (0.0007) [2023-12-27 11:31:53,829][20381] Updated weights for policy 1, policy_version 850072 (0.0007) [2023-12-27 11:31:53,856][20347] Updated weights for policy 0, policy_version 850779 (0.0007) [2023-12-27 11:31:53,909][20347] Updated weights for policy 0, policy_version 850789 (0.0008) [2023-12-27 11:31:53,964][20347] Updated weights for policy 0, policy_version 850799 (0.0008) [2023-12-27 11:31:54,489][20381] Updated weights for policy 1, policy_version 850082 (0.0008) [2023-12-27 11:31:54,549][20381] Updated weights for policy 1, policy_version 850092 (0.0009) [2023-12-27 11:31:54,566][20292] KL-divergence is very high: 139.6963 [2023-12-27 11:31:54,608][20381] Updated weights for policy 1, policy_version 850102 (0.0007) [2023-12-27 11:31:54,680][20347] Updated weights for policy 0, policy_version 850809 (0.0008) [2023-12-27 11:31:54,742][20347] Updated weights for policy 0, policy_version 850819 (0.0006) [2023-12-27 11:31:54,803][20347] Updated weights for policy 0, policy_version 850829 (0.0006) [2023-12-27 11:31:55,318][20381] Updated weights for policy 1, policy_version 850112 (0.0008) [2023-12-27 11:31:55,376][20381] Updated weights for policy 1, policy_version 850122 (0.0007) [2023-12-27 11:31:55,430][20381] Updated weights for policy 1, policy_version 850132 (0.0007) [2023-12-27 11:31:55,450][20347] Updated weights for policy 0, policy_version 850839 (0.0007) [2023-12-27 11:31:55,504][20347] Updated weights for policy 0, policy_version 850849 (0.0008) [2023-12-27 11:31:55,566][20347] Updated weights for policy 0, policy_version 850859 (0.0008) [2023-12-27 11:31:56,102][20381] Updated weights for policy 1, policy_version 850142 (0.0007) [2023-12-27 11:31:56,160][20381] Updated weights for policy 1, policy_version 850152 (0.0006) [2023-12-27 11:31:56,217][20381] Updated weights for policy 1, policy_version 850162 (0.0008) [2023-12-27 11:31:56,281][20347] Updated weights for policy 0, policy_version 850869 (0.0008) [2023-12-27 11:31:56,335][20347] Updated weights for policy 0, policy_version 850879 (0.0006) [2023-12-27 11:31:56,402][20347] Updated weights for policy 0, policy_version 850889 (0.0008) [2023-12-27 11:31:56,879][20381] Updated weights for policy 1, policy_version 850172 (0.0008) [2023-12-27 11:31:56,938][20381] Updated weights for policy 1, policy_version 850182 (0.0007) [2023-12-27 11:31:56,993][20381] Updated weights for policy 1, policy_version 850192 (0.0008) [2023-12-27 11:31:57,095][20347] Updated weights for policy 0, policy_version 850899 (0.0007) [2023-12-27 11:31:57,146][20347] Updated weights for policy 0, policy_version 850909 (0.0007) [2023-12-27 11:31:57,195][20347] Updated weights for policy 0, policy_version 850919 (0.0007) [2023-12-27 11:31:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 435544064. Throughput: 0: 9691.8, 1: 9643.7. Samples: 435530968. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:31:57,363][19170] Avg episode reward: [(0, '318.510'), (1, '903.290')] [2023-12-27 11:31:57,677][20381] Updated weights for policy 1, policy_version 850202 (0.0007) [2023-12-27 11:31:57,732][20381] Updated weights for policy 1, policy_version 850212 (0.0007) [2023-12-27 11:31:57,793][20381] Updated weights for policy 1, policy_version 850222 (0.0007) [2023-12-27 11:31:57,853][20381] Updated weights for policy 1, policy_version 850232 (0.0008) [2023-12-27 11:31:57,898][20347] Updated weights for policy 0, policy_version 850929 (0.0008) [2023-12-27 11:31:57,956][20347] Updated weights for policy 0, policy_version 850939 (0.0006) [2023-12-27 11:31:58,010][20347] Updated weights for policy 0, policy_version 850949 (0.0007) [2023-12-27 11:31:58,072][20347] Updated weights for policy 0, policy_version 850959 (0.0007) [2023-12-27 11:31:58,600][20381] Updated weights for policy 1, policy_version 850242 (0.0008) [2023-12-27 11:31:58,667][20381] Updated weights for policy 1, policy_version 850252 (0.0009) [2023-12-27 11:31:58,734][20381] Updated weights for policy 1, policy_version 850262 (0.0008) [2023-12-27 11:31:58,875][20347] Updated weights for policy 0, policy_version 850969 (0.0009) [2023-12-27 11:31:58,948][20347] Updated weights for policy 0, policy_version 850979 (0.0008) [2023-12-27 11:31:59,010][20347] Updated weights for policy 0, policy_version 850989 (0.0008) [2023-12-27 11:31:59,566][20381] Updated weights for policy 1, policy_version 850272 (0.0007) [2023-12-27 11:31:59,627][20381] Updated weights for policy 1, policy_version 850282 (0.0008) [2023-12-27 11:31:59,690][20381] Updated weights for policy 1, policy_version 850292 (0.0007) [2023-12-27 11:31:59,794][20347] Updated weights for policy 0, policy_version 850999 (0.0009) [2023-12-27 11:31:59,859][20347] Updated weights for policy 0, policy_version 851009 (0.0010) [2023-12-27 11:31:59,920][20347] Updated weights for policy 0, policy_version 851019 (0.0010) [2023-12-27 11:32:00,404][20381] Updated weights for policy 1, policy_version 850302 (0.0008) [2023-12-27 11:32:00,462][20381] Updated weights for policy 1, policy_version 850312 (0.0008) [2023-12-27 11:32:00,522][20381] Updated weights for policy 1, policy_version 850322 (0.0007) [2023-12-27 11:32:00,613][20347] Updated weights for policy 0, policy_version 851029 (0.0008) [2023-12-27 11:32:00,670][20347] Updated weights for policy 0, policy_version 851039 (0.0006) [2023-12-27 11:32:00,736][20347] Updated weights for policy 0, policy_version 851049 (0.0008) [2023-12-27 11:32:01,272][20381] Updated weights for policy 1, policy_version 850332 (0.0007) [2023-12-27 11:32:01,334][20381] Updated weights for policy 1, policy_version 850342 (0.0006) [2023-12-27 11:32:01,397][20381] Updated weights for policy 1, policy_version 850352 (0.0008) [2023-12-27 11:32:01,429][20347] Updated weights for policy 0, policy_version 851059 (0.0010) [2023-12-27 11:32:01,495][20347] Updated weights for policy 0, policy_version 851069 (0.0007) [2023-12-27 11:32:01,563][20347] Updated weights for policy 0, policy_version 851079 (0.0008) [2023-12-27 11:32:02,101][20381] Updated weights for policy 1, policy_version 850362 (0.0007) [2023-12-27 11:32:02,156][20381] Updated weights for policy 1, policy_version 850372 (0.0006) [2023-12-27 11:32:02,221][20381] Updated weights for policy 1, policy_version 850382 (0.0009) [2023-12-27 11:32:02,285][20381] Updated weights for policy 1, policy_version 850392 (0.0008) [2023-12-27 11:32:02,338][20347] Updated weights for policy 0, policy_version 851089 (0.0008) [2023-12-27 11:32:02,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 435634176. Throughput: 0: 9688.8, 1: 9625.7. Samples: 435644472. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:02,363][19170] Avg episode reward: [(0, '333.250'), (1, '941.540')] [2023-12-27 11:32:02,406][20347] Updated weights for policy 0, policy_version 851099 (0.0007) [2023-12-27 11:32:02,467][20347] Updated weights for policy 0, policy_version 851109 (0.0006) [2023-12-27 11:32:02,523][20347] Updated weights for policy 0, policy_version 851119 (0.0006) [2023-12-27 11:32:03,059][20381] Updated weights for policy 1, policy_version 850402 (0.0008) [2023-12-27 11:32:03,110][20381] Updated weights for policy 1, policy_version 850412 (0.0007) [2023-12-27 11:32:03,160][20381] Updated weights for policy 1, policy_version 850422 (0.0006) [2023-12-27 11:32:03,173][20347] Updated weights for policy 0, policy_version 851129 (0.0007) [2023-12-27 11:32:03,226][20347] Updated weights for policy 0, policy_version 851139 (0.0007) [2023-12-27 11:32:03,275][20347] Updated weights for policy 0, policy_version 851149 (0.0008) [2023-12-27 11:32:03,855][20381] Updated weights for policy 1, policy_version 850432 (0.0009) [2023-12-27 11:32:03,909][20381] Updated weights for policy 1, policy_version 850442 (0.0007) [2023-12-27 11:32:03,968][20381] Updated weights for policy 1, policy_version 850452 (0.0008) [2023-12-27 11:32:03,989][20347] Updated weights for policy 0, policy_version 851159 (0.0006) [2023-12-27 11:32:04,038][20347] Updated weights for policy 0, policy_version 851169 (0.0008) [2023-12-27 11:32:04,094][20347] Updated weights for policy 0, policy_version 851179 (0.0009) [2023-12-27 11:32:04,717][20381] Updated weights for policy 1, policy_version 850462 (0.0008) [2023-12-27 11:32:04,776][20381] Updated weights for policy 1, policy_version 850472 (0.0007) [2023-12-27 11:32:04,795][20347] Updated weights for policy 0, policy_version 851189 (0.0008) [2023-12-27 11:32:04,832][20381] Updated weights for policy 1, policy_version 850482 (0.0006) [2023-12-27 11:32:04,853][20347] Updated weights for policy 0, policy_version 851199 (0.0008) [2023-12-27 11:32:04,904][20347] Updated weights for policy 0, policy_version 851209 (0.0008) [2023-12-27 11:32:05,434][20381] Updated weights for policy 1, policy_version 850492 (0.0006) [2023-12-27 11:32:05,493][20381] Updated weights for policy 1, policy_version 850502 (0.0009) [2023-12-27 11:32:05,553][20381] Updated weights for policy 1, policy_version 850512 (0.0008) [2023-12-27 11:32:05,645][20347] Updated weights for policy 0, policy_version 851219 (0.0007) [2023-12-27 11:32:05,707][20347] Updated weights for policy 0, policy_version 851229 (0.0007) [2023-12-27 11:32:05,762][20347] Updated weights for policy 0, policy_version 851239 (0.0008) [2023-12-27 11:32:06,167][20381] Updated weights for policy 1, policy_version 850522 (0.0008) [2023-12-27 11:32:06,227][20381] Updated weights for policy 1, policy_version 850532 (0.0008) [2023-12-27 11:32:06,292][20381] Updated weights for policy 1, policy_version 850542 (0.0008) [2023-12-27 11:32:06,358][20381] Updated weights for policy 1, policy_version 850552 (0.0007) [2023-12-27 11:32:06,462][20347] Updated weights for policy 0, policy_version 851249 (0.0008) [2023-12-27 11:32:06,519][20347] Updated weights for policy 0, policy_version 851259 (0.0007) [2023-12-27 11:32:06,580][20347] Updated weights for policy 0, policy_version 851269 (0.0007) [2023-12-27 11:32:06,638][20347] Updated weights for policy 0, policy_version 851279 (0.0008) [2023-12-27 11:32:07,097][20381] Updated weights for policy 1, policy_version 850562 (0.0007) [2023-12-27 11:32:07,159][20381] Updated weights for policy 1, policy_version 850572 (0.0008) [2023-12-27 11:32:07,220][20381] Updated weights for policy 1, policy_version 850582 (0.0007) [2023-12-27 11:32:07,348][20347] Updated weights for policy 0, policy_version 851289 (0.0009) [2023-12-27 11:32:07,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19410.9). Total num frames: 435732480. Throughput: 0: 9703.5, 1: 9649.8. Samples: 435704200. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:07,363][19170] Avg episode reward: [(0, '385.200'), (1, '980.490')] [2023-12-27 11:32:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000850584_217776128.pth... [2023-12-27 11:32:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000849432_217481216.pth [2023-12-27 11:32:07,407][20347] Updated weights for policy 0, policy_version 851299 (0.0008) [2023-12-27 11:32:07,469][20347] Updated weights for policy 0, policy_version 851309 (0.0008) [2023-12-27 11:32:07,488][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000851312_217964544.pth... [2023-12-27 11:32:07,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000850160_217669632.pth [2023-12-27 11:32:07,880][20381] Updated weights for policy 1, policy_version 850592 (0.0010) [2023-12-27 11:32:07,940][20381] Updated weights for policy 1, policy_version 850602 (0.0010) [2023-12-27 11:32:08,000][20381] Updated weights for policy 1, policy_version 850612 (0.0011) [2023-12-27 11:32:08,190][20347] Updated weights for policy 0, policy_version 851319 (0.0007) [2023-12-27 11:32:08,248][20347] Updated weights for policy 0, policy_version 851329 (0.0007) [2023-12-27 11:32:08,305][20347] Updated weights for policy 0, policy_version 851339 (0.0007) [2023-12-27 11:32:08,703][20381] Updated weights for policy 1, policy_version 850622 (0.0009) [2023-12-27 11:32:08,766][20381] Updated weights for policy 1, policy_version 850632 (0.0010) [2023-12-27 11:32:08,827][20381] Updated weights for policy 1, policy_version 850642 (0.0011) [2023-12-27 11:32:09,033][20347] Updated weights for policy 0, policy_version 851349 (0.0008) [2023-12-27 11:32:09,090][20347] Updated weights for policy 0, policy_version 851359 (0.0007) [2023-12-27 11:32:09,146][20347] Updated weights for policy 0, policy_version 851369 (0.0006) [2023-12-27 11:32:09,553][20381] Updated weights for policy 1, policy_version 850652 (0.0009) [2023-12-27 11:32:09,616][20381] Updated weights for policy 1, policy_version 850662 (0.0007) [2023-12-27 11:32:09,672][20381] Updated weights for policy 1, policy_version 850672 (0.0007) [2023-12-27 11:32:10,016][20347] Updated weights for policy 0, policy_version 851379 (0.0009) [2023-12-27 11:32:10,081][20347] Updated weights for policy 0, policy_version 851389 (0.0006) [2023-12-27 11:32:10,139][20347] Updated weights for policy 0, policy_version 851399 (0.0007) [2023-12-27 11:32:10,455][20381] Updated weights for policy 1, policy_version 850682 (0.0008) [2023-12-27 11:32:10,523][20381] Updated weights for policy 1, policy_version 850692 (0.0007) [2023-12-27 11:32:10,589][20381] Updated weights for policy 1, policy_version 850702 (0.0007) [2023-12-27 11:32:10,651][20381] Updated weights for policy 1, policy_version 850712 (0.0008) [2023-12-27 11:32:10,827][20347] Updated weights for policy 0, policy_version 851409 (0.0007) [2023-12-27 11:32:10,884][20347] Updated weights for policy 0, policy_version 851419 (0.0008) [2023-12-27 11:32:10,940][20347] Updated weights for policy 0, policy_version 851429 (0.0007) [2023-12-27 11:32:11,000][20347] Updated weights for policy 0, policy_version 851439 (0.0008) [2023-12-27 11:32:11,349][20381] Updated weights for policy 1, policy_version 850722 (0.0008) [2023-12-27 11:32:11,417][20381] Updated weights for policy 1, policy_version 850732 (0.0008) [2023-12-27 11:32:11,480][20381] Updated weights for policy 1, policy_version 850742 (0.0008) [2023-12-27 11:32:11,760][20347] Updated weights for policy 0, policy_version 851449 (0.0008) [2023-12-27 11:32:11,823][20347] Updated weights for policy 0, policy_version 851459 (0.0008) [2023-12-27 11:32:11,881][20347] Updated weights for policy 0, policy_version 851469 (0.0010) [2023-12-27 11:32:12,224][20381] Updated weights for policy 1, policy_version 850752 (0.0008) [2023-12-27 11:32:12,288][20381] Updated weights for policy 1, policy_version 850762 (0.0009) [2023-12-27 11:32:12,351][20381] Updated weights for policy 1, policy_version 850772 (0.0008) [2023-12-27 11:32:12,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19383.1). Total num frames: 435822592. Throughput: 0: 9679.9, 1: 9701.2. Samples: 435819604. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:12,363][19170] Avg episode reward: [(0, '311.680'), (1, '990.280')] [2023-12-27 11:32:12,602][20347] Updated weights for policy 0, policy_version 851479 (0.0008) [2023-12-27 11:32:12,664][20347] Updated weights for policy 0, policy_version 851489 (0.0008) [2023-12-27 11:32:12,721][20347] Updated weights for policy 0, policy_version 851499 (0.0008) [2023-12-27 11:32:13,105][20381] Updated weights for policy 1, policy_version 850782 (0.0009) [2023-12-27 11:32:13,166][20381] Updated weights for policy 1, policy_version 850792 (0.0008) [2023-12-27 11:32:13,227][20381] Updated weights for policy 1, policy_version 850802 (0.0008) [2023-12-27 11:32:13,405][20347] Updated weights for policy 0, policy_version 851509 (0.0006) [2023-12-27 11:32:13,467][20347] Updated weights for policy 0, policy_version 851519 (0.0006) [2023-12-27 11:32:13,535][20347] Updated weights for policy 0, policy_version 851529 (0.0007) [2023-12-27 11:32:13,949][20381] Updated weights for policy 1, policy_version 850812 (0.0007) [2023-12-27 11:32:14,013][20381] Updated weights for policy 1, policy_version 850822 (0.0005) [2023-12-27 11:32:14,070][20381] Updated weights for policy 1, policy_version 850832 (0.0006) [2023-12-27 11:32:14,207][20347] Updated weights for policy 0, policy_version 851539 (0.0008) [2023-12-27 11:32:14,268][20347] Updated weights for policy 0, policy_version 851549 (0.0008) [2023-12-27 11:32:14,329][20347] Updated weights for policy 0, policy_version 851559 (0.0006) [2023-12-27 11:32:14,735][20381] Updated weights for policy 1, policy_version 850842 (0.0007) [2023-12-27 11:32:14,802][20381] Updated weights for policy 1, policy_version 850852 (0.0008) [2023-12-27 11:32:14,857][20381] Updated weights for policy 1, policy_version 850862 (0.0007) [2023-12-27 11:32:14,917][20381] Updated weights for policy 1, policy_version 850872 (0.0008) [2023-12-27 11:32:15,028][20347] Updated weights for policy 0, policy_version 851569 (0.0007) [2023-12-27 11:32:15,092][20347] Updated weights for policy 0, policy_version 851579 (0.0007) [2023-12-27 11:32:15,153][20347] Updated weights for policy 0, policy_version 851589 (0.0008) [2023-12-27 11:32:15,216][20347] Updated weights for policy 0, policy_version 851599 (0.0008) [2023-12-27 11:32:15,633][20381] Updated weights for policy 1, policy_version 850882 (0.0006) [2023-12-27 11:32:15,688][20381] Updated weights for policy 1, policy_version 850892 (0.0006) [2023-12-27 11:32:15,750][20381] Updated weights for policy 1, policy_version 850902 (0.0008) [2023-12-27 11:32:15,892][20347] Updated weights for policy 0, policy_version 851609 (0.0008) [2023-12-27 11:32:15,949][20347] Updated weights for policy 0, policy_version 851619 (0.0007) [2023-12-27 11:32:16,003][20347] Updated weights for policy 0, policy_version 851629 (0.0008) [2023-12-27 11:32:16,448][20381] Updated weights for policy 1, policy_version 850912 (0.0008) [2023-12-27 11:32:16,502][20381] Updated weights for policy 1, policy_version 850922 (0.0007) [2023-12-27 11:32:16,562][20381] Updated weights for policy 1, policy_version 850932 (0.0008) [2023-12-27 11:32:16,686][20347] Updated weights for policy 0, policy_version 851639 (0.0010) [2023-12-27 11:32:16,750][20347] Updated weights for policy 0, policy_version 851649 (0.0010) [2023-12-27 11:32:16,814][20347] Updated weights for policy 0, policy_version 851659 (0.0009) [2023-12-27 11:32:17,310][20381] Updated weights for policy 1, policy_version 850942 (0.0007) [2023-12-27 11:32:17,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19251.3, 300 sec: 19410.9). Total num frames: 435920896. Throughput: 0: 9661.7, 1: 9689.4. Samples: 435935992. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:17,363][19170] Avg episode reward: [(0, '359.590'), (1, '990.280')] [2023-12-27 11:32:17,366][20381] Updated weights for policy 1, policy_version 850952 (0.0008) [2023-12-27 11:32:17,426][20381] Updated weights for policy 1, policy_version 850962 (0.0008) [2023-12-27 11:32:17,524][20347] Updated weights for policy 0, policy_version 851669 (0.0007) [2023-12-27 11:32:17,583][20347] Updated weights for policy 0, policy_version 851679 (0.0006) [2023-12-27 11:32:17,644][20347] Updated weights for policy 0, policy_version 851689 (0.0008) [2023-12-27 11:32:18,165][20381] Updated weights for policy 1, policy_version 850972 (0.0008) [2023-12-27 11:32:18,217][20381] Updated weights for policy 1, policy_version 850982 (0.0008) [2023-12-27 11:32:18,276][20381] Updated weights for policy 1, policy_version 850992 (0.0008) [2023-12-27 11:32:18,347][20347] Updated weights for policy 0, policy_version 851699 (0.0008) [2023-12-27 11:32:18,398][20347] Updated weights for policy 0, policy_version 851709 (0.0007) [2023-12-27 11:32:18,457][20347] Updated weights for policy 0, policy_version 851719 (0.0008) [2023-12-27 11:32:19,057][20381] Updated weights for policy 1, policy_version 851002 (0.0008) [2023-12-27 11:32:19,117][20381] Updated weights for policy 1, policy_version 851012 (0.0007) [2023-12-27 11:32:19,178][20381] Updated weights for policy 1, policy_version 851022 (0.0008) [2023-12-27 11:32:19,186][20347] Updated weights for policy 0, policy_version 851729 (0.0007) [2023-12-27 11:32:19,240][20381] Updated weights for policy 1, policy_version 851032 (0.0008) [2023-12-27 11:32:19,247][20347] Updated weights for policy 0, policy_version 851739 (0.0009) [2023-12-27 11:32:19,308][20347] Updated weights for policy 0, policy_version 851749 (0.0008) [2023-12-27 11:32:19,382][20347] Updated weights for policy 0, policy_version 851759 (0.0008) [2023-12-27 11:32:19,987][20381] Updated weights for policy 1, policy_version 851042 (0.0008) [2023-12-27 11:32:20,047][20381] Updated weights for policy 1, policy_version 851052 (0.0007) [2023-12-27 11:32:20,114][20381] Updated weights for policy 1, policy_version 851062 (0.0008) [2023-12-27 11:32:20,143][20347] Updated weights for policy 0, policy_version 851769 (0.0008) [2023-12-27 11:32:20,208][20347] Updated weights for policy 0, policy_version 851779 (0.0008) [2023-12-27 11:32:20,264][20347] Updated weights for policy 0, policy_version 851789 (0.0008) [2023-12-27 11:32:20,874][20381] Updated weights for policy 1, policy_version 851072 (0.0008) [2023-12-27 11:32:20,934][20381] Updated weights for policy 1, policy_version 851082 (0.0008) [2023-12-27 11:32:20,994][20381] Updated weights for policy 1, policy_version 851092 (0.0008) [2023-12-27 11:32:21,036][20347] Updated weights for policy 0, policy_version 851799 (0.0008) [2023-12-27 11:32:21,092][20347] Updated weights for policy 0, policy_version 851809 (0.0006) [2023-12-27 11:32:21,160][20347] Updated weights for policy 0, policy_version 851819 (0.0007) [2023-12-27 11:32:21,807][20381] Updated weights for policy 1, policy_version 851102 (0.0008) [2023-12-27 11:32:21,868][20381] Updated weights for policy 1, policy_version 851112 (0.0008) [2023-12-27 11:32:21,910][20347] Updated weights for policy 0, policy_version 851830 (0.0007) [2023-12-27 11:32:21,936][20381] Updated weights for policy 1, policy_version 851122 (0.0008) [2023-12-27 11:32:21,975][20347] Updated weights for policy 0, policy_version 851840 (0.0007) [2023-12-27 11:32:22,037][20347] Updated weights for policy 0, policy_version 851850 (0.0008) [2023-12-27 11:32:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 436019200. Throughput: 0: 9651.9, 1: 9640.6. Samples: 435992356. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:22,363][19170] Avg episode reward: [(0, '358.340'), (1, '980.490')] [2023-12-27 11:32:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000851128_217915392.pth... [2023-12-27 11:32:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000851856_218103808.pth... [2023-12-27 11:32:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000850008_217628672.pth [2023-12-27 11:32:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000850736_217817088.pth [2023-12-27 11:32:22,699][20381] Updated weights for policy 1, policy_version 851132 (0.0008) [2023-12-27 11:32:22,767][20381] Updated weights for policy 1, policy_version 851142 (0.0008) [2023-12-27 11:32:22,796][20347] Updated weights for policy 0, policy_version 851860 (0.0007) [2023-12-27 11:32:22,826][20381] Updated weights for policy 1, policy_version 851152 (0.0009) [2023-12-27 11:32:22,857][20347] Updated weights for policy 0, policy_version 851870 (0.0008) [2023-12-27 11:32:22,919][20347] Updated weights for policy 0, policy_version 851880 (0.0008) [2023-12-27 11:32:23,573][20381] Updated weights for policy 1, policy_version 851162 (0.0007) [2023-12-27 11:32:23,602][20347] Updated weights for policy 0, policy_version 851890 (0.0011) [2023-12-27 11:32:23,639][20381] Updated weights for policy 1, policy_version 851172 (0.0006) [2023-12-27 11:32:23,656][20347] Updated weights for policy 0, policy_version 851900 (0.0011) [2023-12-27 11:32:23,700][20381] Updated weights for policy 1, policy_version 851182 (0.0006) [2023-12-27 11:32:23,720][20347] Updated weights for policy 0, policy_version 851910 (0.0008) [2023-12-27 11:32:23,761][20381] Updated weights for policy 1, policy_version 851192 (0.0006) [2023-12-27 11:32:23,781][20347] Updated weights for policy 0, policy_version 851920 (0.0006) [2023-12-27 11:32:24,414][20347] Updated weights for policy 0, policy_version 851930 (0.0007) [2023-12-27 11:32:24,445][20381] Updated weights for policy 1, policy_version 851202 (0.0008) [2023-12-27 11:32:24,474][20347] Updated weights for policy 0, policy_version 851940 (0.0008) [2023-12-27 11:32:24,507][20381] Updated weights for policy 1, policy_version 851212 (0.0007) [2023-12-27 11:32:24,536][20347] Updated weights for policy 0, policy_version 851950 (0.0006) [2023-12-27 11:32:24,558][20381] Updated weights for policy 1, policy_version 851222 (0.0006) [2023-12-27 11:32:25,218][20347] Updated weights for policy 0, policy_version 851960 (0.0007) [2023-12-27 11:32:25,267][20347] Updated weights for policy 0, policy_version 851970 (0.0007) [2023-12-27 11:32:25,281][20381] Updated weights for policy 1, policy_version 851232 (0.0008) [2023-12-27 11:32:25,318][20347] Updated weights for policy 0, policy_version 851980 (0.0006) [2023-12-27 11:32:25,334][20381] Updated weights for policy 1, policy_version 851242 (0.0007) [2023-12-27 11:32:25,398][20381] Updated weights for policy 1, policy_version 851252 (0.0007) [2023-12-27 11:32:25,902][20347] Updated weights for policy 0, policy_version 851990 (0.0009) [2023-12-27 11:32:25,951][20347] Updated weights for policy 0, policy_version 852000 (0.0007) [2023-12-27 11:32:26,001][20347] Updated weights for policy 0, policy_version 852010 (0.0008) [2023-12-27 11:32:26,111][20381] Updated weights for policy 1, policy_version 851262 (0.0008) [2023-12-27 11:32:26,167][20381] Updated weights for policy 1, policy_version 851272 (0.0006) [2023-12-27 11:32:26,224][20381] Updated weights for policy 1, policy_version 851282 (0.0006) [2023-12-27 11:32:26,776][20347] Updated weights for policy 0, policy_version 852020 (0.0009) [2023-12-27 11:32:26,832][20381] Updated weights for policy 1, policy_version 851292 (0.0006) [2023-12-27 11:32:26,834][20347] Updated weights for policy 0, policy_version 852030 (0.0006) [2023-12-27 11:32:26,886][20381] Updated weights for policy 1, policy_version 851302 (0.0006) [2023-12-27 11:32:26,891][20347] Updated weights for policy 0, policy_version 852040 (0.0006) [2023-12-27 11:32:26,934][20381] Updated weights for policy 1, policy_version 851312 (0.0005) [2023-12-27 11:32:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 436117504. Throughput: 0: 9651.5, 1: 9617.3. Samples: 436107828. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:27,363][19170] Avg episode reward: [(0, '407.460'), (1, '980.490')] [2023-12-27 11:32:27,570][20347] Updated weights for policy 0, policy_version 852050 (0.0007) [2023-12-27 11:32:27,610][20381] Updated weights for policy 1, policy_version 851322 (0.0006) [2023-12-27 11:32:27,621][20347] Updated weights for policy 0, policy_version 852060 (0.0006) [2023-12-27 11:32:27,665][20381] Updated weights for policy 1, policy_version 851332 (0.0008) [2023-12-27 11:32:27,671][20347] Updated weights for policy 0, policy_version 852070 (0.0005) [2023-12-27 11:32:27,711][20381] Updated weights for policy 1, policy_version 851342 (0.0006) [2023-12-27 11:32:27,725][20347] Updated weights for policy 0, policy_version 852080 (0.0006) [2023-12-27 11:32:27,766][20381] Updated weights for policy 1, policy_version 851352 (0.0006) [2023-12-27 11:32:28,384][20347] Updated weights for policy 0, policy_version 852090 (0.0008) [2023-12-27 11:32:28,432][20347] Updated weights for policy 0, policy_version 852100 (0.0007) [2023-12-27 11:32:28,489][20347] Updated weights for policy 0, policy_version 852110 (0.0007) [2023-12-27 11:32:28,492][20381] Updated weights for policy 1, policy_version 851362 (0.0008) [2023-12-27 11:32:28,550][20381] Updated weights for policy 1, policy_version 851372 (0.0008) [2023-12-27 11:32:28,605][20381] Updated weights for policy 1, policy_version 851382 (0.0008) [2023-12-27 11:32:29,195][20347] Updated weights for policy 0, policy_version 852120 (0.0006) [2023-12-27 11:32:29,257][20347] Updated weights for policy 0, policy_version 852130 (0.0006) [2023-12-27 11:32:29,311][20347] Updated weights for policy 0, policy_version 852140 (0.0007) [2023-12-27 11:32:29,375][20381] Updated weights for policy 1, policy_version 851392 (0.0008) [2023-12-27 11:32:29,437][20381] Updated weights for policy 1, policy_version 851402 (0.0008) [2023-12-27 11:32:29,500][20381] Updated weights for policy 1, policy_version 851412 (0.0008) [2023-12-27 11:32:29,988][20347] Updated weights for policy 0, policy_version 852150 (0.0008) [2023-12-27 11:32:30,055][20347] Updated weights for policy 0, policy_version 852160 (0.0008) [2023-12-27 11:32:30,116][20347] Updated weights for policy 0, policy_version 852170 (0.0008) [2023-12-27 11:32:30,249][20381] Updated weights for policy 1, policy_version 851422 (0.0007) [2023-12-27 11:32:30,311][20381] Updated weights for policy 1, policy_version 851432 (0.0008) [2023-12-27 11:32:30,366][20381] Updated weights for policy 1, policy_version 851442 (0.0008) [2023-12-27 11:32:30,779][20347] Updated weights for policy 0, policy_version 852180 (0.0007) [2023-12-27 11:32:30,835][20347] Updated weights for policy 0, policy_version 852190 (0.0008) [2023-12-27 11:32:30,880][20347] Updated weights for policy 0, policy_version 852200 (0.0007) [2023-12-27 11:32:31,091][20381] Updated weights for policy 1, policy_version 851452 (0.0007) [2023-12-27 11:32:31,159][20381] Updated weights for policy 1, policy_version 851462 (0.0008) [2023-12-27 11:32:31,219][20381] Updated weights for policy 1, policy_version 851472 (0.0008) [2023-12-27 11:32:31,672][20347] Updated weights for policy 0, policy_version 852210 (0.0006) [2023-12-27 11:32:31,739][20347] Updated weights for policy 0, policy_version 852220 (0.0008) [2023-12-27 11:32:31,809][20347] Updated weights for policy 0, policy_version 852230 (0.0007) [2023-12-27 11:32:31,872][20347] Updated weights for policy 0, policy_version 852240 (0.0006) [2023-12-27 11:32:31,916][20381] Updated weights for policy 1, policy_version 851482 (0.0008) [2023-12-27 11:32:31,983][20381] Updated weights for policy 1, policy_version 851492 (0.0007) [2023-12-27 11:32:32,045][20381] Updated weights for policy 1, policy_version 851502 (0.0008) [2023-12-27 11:32:32,108][20381] Updated weights for policy 1, policy_version 851512 (0.0008) [2023-12-27 11:32:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 436215808. Throughput: 0: 9672.5, 1: 9642.8. Samples: 436226280. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:32,363][19170] Avg episode reward: [(0, '375.600'), (1, '1000.000')] [2023-12-27 11:32:32,588][20347] Updated weights for policy 0, policy_version 852250 (0.0008) [2023-12-27 11:32:32,650][20347] Updated weights for policy 0, policy_version 852260 (0.0008) [2023-12-27 11:32:32,715][20347] Updated weights for policy 0, policy_version 852270 (0.0008) [2023-12-27 11:32:32,870][20381] Updated weights for policy 1, policy_version 851522 (0.0010) [2023-12-27 11:32:32,930][20381] Updated weights for policy 1, policy_version 851532 (0.0010) [2023-12-27 11:32:32,993][20381] Updated weights for policy 1, policy_version 851542 (0.0011) [2023-12-27 11:32:33,443][20347] Updated weights for policy 0, policy_version 852280 (0.0007) [2023-12-27 11:32:33,503][20347] Updated weights for policy 0, policy_version 852290 (0.0008) [2023-12-27 11:32:33,562][20347] Updated weights for policy 0, policy_version 852300 (0.0008) [2023-12-27 11:32:33,680][20381] Updated weights for policy 1, policy_version 851552 (0.0010) [2023-12-27 11:32:33,739][20381] Updated weights for policy 1, policy_version 851562 (0.0010) [2023-12-27 11:32:33,791][20381] Updated weights for policy 1, policy_version 851572 (0.0010) [2023-12-27 11:32:34,294][20347] Updated weights for policy 0, policy_version 852310 (0.0008) [2023-12-27 11:32:34,355][20347] Updated weights for policy 0, policy_version 852320 (0.0008) [2023-12-27 11:32:34,418][20347] Updated weights for policy 0, policy_version 852330 (0.0007) [2023-12-27 11:32:34,500][20381] Updated weights for policy 1, policy_version 851582 (0.0010) [2023-12-27 11:32:34,564][20381] Updated weights for policy 1, policy_version 851592 (0.0009) [2023-12-27 11:32:34,627][20381] Updated weights for policy 1, policy_version 851602 (0.0008) [2023-12-27 11:32:35,133][20347] Updated weights for policy 0, policy_version 852340 (0.0007) [2023-12-27 11:32:35,193][20347] Updated weights for policy 0, policy_version 852350 (0.0008) [2023-12-27 11:32:35,249][20347] Updated weights for policy 0, policy_version 852360 (0.0008) [2023-12-27 11:32:35,322][20381] Updated weights for policy 1, policy_version 851612 (0.0008) [2023-12-27 11:32:35,351][20292] KL-divergence is very high: 136.2079 [2023-12-27 11:32:35,369][20381] Updated weights for policy 1, policy_version 851622 (0.0008) [2023-12-27 11:32:35,417][20381] Updated weights for policy 1, policy_version 851632 (0.0005) [2023-12-27 11:32:35,428][20292] KL-divergence is very high: 103.0103 [2023-12-27 11:32:35,925][20347] Updated weights for policy 0, policy_version 852370 (0.0008) [2023-12-27 11:32:35,992][20347] Updated weights for policy 0, policy_version 852380 (0.0008) [2023-12-27 11:32:36,059][20347] Updated weights for policy 0, policy_version 852390 (0.0008) [2023-12-27 11:32:36,121][20381] Updated weights for policy 1, policy_version 851642 (0.0007) [2023-12-27 11:32:36,123][20347] Updated weights for policy 0, policy_version 852400 (0.0007) [2023-12-27 11:32:36,183][20381] Updated weights for policy 1, policy_version 851652 (0.0008) [2023-12-27 11:32:36,246][20381] Updated weights for policy 1, policy_version 851662 (0.0008) [2023-12-27 11:32:36,310][20381] Updated weights for policy 1, policy_version 851672 (0.0008) [2023-12-27 11:32:36,800][20347] Updated weights for policy 0, policy_version 852410 (0.0005) [2023-12-27 11:32:36,861][20347] Updated weights for policy 0, policy_version 852420 (0.0005) [2023-12-27 11:32:36,918][20347] Updated weights for policy 0, policy_version 852430 (0.0005) [2023-12-27 11:32:37,040][20381] Updated weights for policy 1, policy_version 851682 (0.0007) [2023-12-27 11:32:37,097][20381] Updated weights for policy 1, policy_version 851692 (0.0006) [2023-12-27 11:32:37,150][20381] Updated weights for policy 1, policy_version 851702 (0.0007) [2023-12-27 11:32:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 436314112. Throughput: 0: 9675.2, 1: 9652.8. Samples: 436283940. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:37,363][19170] Avg episode reward: [(0, '342.700'), (1, '990.250')] [2023-12-27 11:32:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000851704_218062848.pth... [2023-12-27 11:32:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000852432_218251264.pth... [2023-12-27 11:32:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000850584_217776128.pth [2023-12-27 11:32:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000851312_217964544.pth [2023-12-27 11:32:37,516][20347] Updated weights for policy 0, policy_version 852440 (0.0008) [2023-12-27 11:32:37,573][20347] Updated weights for policy 0, policy_version 852450 (0.0008) [2023-12-27 11:32:37,634][20347] Updated weights for policy 0, policy_version 852460 (0.0006) [2023-12-27 11:32:37,870][20381] Updated weights for policy 1, policy_version 851712 (0.0007) [2023-12-27 11:32:37,931][20381] Updated weights for policy 1, policy_version 851722 (0.0007) [2023-12-27 11:32:37,983][20381] Updated weights for policy 1, policy_version 851732 (0.0008) [2023-12-27 11:32:38,342][20347] Updated weights for policy 0, policy_version 852470 (0.0008) [2023-12-27 11:32:38,401][20347] Updated weights for policy 0, policy_version 852480 (0.0008) [2023-12-27 11:32:38,465][20347] Updated weights for policy 0, policy_version 852490 (0.0008) [2023-12-27 11:32:38,636][20381] Updated weights for policy 1, policy_version 851742 (0.0008) [2023-12-27 11:32:38,699][20381] Updated weights for policy 1, policy_version 851752 (0.0007) [2023-12-27 11:32:38,761][20381] Updated weights for policy 1, policy_version 851762 (0.0008) [2023-12-27 11:32:39,236][20347] Updated weights for policy 0, policy_version 852500 (0.0009) [2023-12-27 11:32:39,300][20347] Updated weights for policy 0, policy_version 852510 (0.0009) [2023-12-27 11:32:39,372][20347] Updated weights for policy 0, policy_version 852520 (0.0009) [2023-12-27 11:32:39,464][20381] Updated weights for policy 1, policy_version 851772 (0.0008) [2023-12-27 11:32:39,524][20381] Updated weights for policy 1, policy_version 851782 (0.0008) [2023-12-27 11:32:39,595][20381] Updated weights for policy 1, policy_version 851792 (0.0008) [2023-12-27 11:32:40,126][20347] Updated weights for policy 0, policy_version 852530 (0.0008) [2023-12-27 11:32:40,186][20347] Updated weights for policy 0, policy_version 852540 (0.0007) [2023-12-27 11:32:40,250][20347] Updated weights for policy 0, policy_version 852550 (0.0007) [2023-12-27 11:32:40,312][20347] Updated weights for policy 0, policy_version 852560 (0.0008) [2023-12-27 11:32:40,363][20381] Updated weights for policy 1, policy_version 851802 (0.0008) [2023-12-27 11:32:40,426][20381] Updated weights for policy 1, policy_version 851812 (0.0011) [2023-12-27 11:32:40,487][20381] Updated weights for policy 1, policy_version 851822 (0.0011) [2023-12-27 11:32:40,552][20381] Updated weights for policy 1, policy_version 851832 (0.0011) [2023-12-27 11:32:41,036][20347] Updated weights for policy 0, policy_version 852570 (0.0008) [2023-12-27 11:32:41,110][20347] Updated weights for policy 0, policy_version 852580 (0.0008) [2023-12-27 11:32:41,179][20347] Updated weights for policy 0, policy_version 852590 (0.0007) [2023-12-27 11:32:41,274][20381] Updated weights for policy 1, policy_version 851842 (0.0008) [2023-12-27 11:32:41,338][20381] Updated weights for policy 1, policy_version 851852 (0.0008) [2023-12-27 11:32:41,403][20381] Updated weights for policy 1, policy_version 851862 (0.0008) [2023-12-27 11:32:41,934][20347] Updated weights for policy 0, policy_version 852600 (0.0008) [2023-12-27 11:32:42,002][20347] Updated weights for policy 0, policy_version 852610 (0.0008) [2023-12-27 11:32:42,072][20347] Updated weights for policy 0, policy_version 852620 (0.0008) [2023-12-27 11:32:42,084][20381] Updated weights for policy 1, policy_version 851872 (0.0008) [2023-12-27 11:32:42,145][20381] Updated weights for policy 1, policy_version 851882 (0.0007) [2023-12-27 11:32:42,208][20381] Updated weights for policy 1, policy_version 851892 (0.0007) [2023-12-27 11:32:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 436412416. Throughput: 0: 9710.9, 1: 9609.2. Samples: 436400376. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:42,363][19170] Avg episode reward: [(0, '299.960'), (1, '980.530')] [2023-12-27 11:32:42,796][20347] Updated weights for policy 0, policy_version 852630 (0.0009) [2023-12-27 11:32:42,858][20347] Updated weights for policy 0, policy_version 852640 (0.0009) [2023-12-27 11:32:42,922][20347] Updated weights for policy 0, policy_version 852650 (0.0010) [2023-12-27 11:32:42,935][20381] Updated weights for policy 1, policy_version 851902 (0.0009) [2023-12-27 11:32:42,995][20381] Updated weights for policy 1, policy_version 851912 (0.0007) [2023-12-27 11:32:43,047][20381] Updated weights for policy 1, policy_version 851922 (0.0008) [2023-12-27 11:32:43,622][20347] Updated weights for policy 0, policy_version 852660 (0.0009) [2023-12-27 11:32:43,682][20347] Updated weights for policy 0, policy_version 852670 (0.0007) [2023-12-27 11:32:43,742][20347] Updated weights for policy 0, policy_version 852680 (0.0008) [2023-12-27 11:32:43,775][20381] Updated weights for policy 1, policy_version 851932 (0.0007) [2023-12-27 11:32:43,835][20381] Updated weights for policy 1, policy_version 851942 (0.0008) [2023-12-27 11:32:43,891][20381] Updated weights for policy 1, policy_version 851952 (0.0007) [2023-12-27 11:32:44,452][20347] Updated weights for policy 0, policy_version 852690 (0.0006) [2023-12-27 11:32:44,512][20347] Updated weights for policy 0, policy_version 852700 (0.0008) [2023-12-27 11:32:44,563][20347] Updated weights for policy 0, policy_version 852710 (0.0007) [2023-12-27 11:32:44,622][20381] Updated weights for policy 1, policy_version 851962 (0.0007) [2023-12-27 11:32:44,623][20347] Updated weights for policy 0, policy_version 852720 (0.0007) [2023-12-27 11:32:44,678][20381] Updated weights for policy 1, policy_version 851972 (0.0008) [2023-12-27 11:32:44,730][20381] Updated weights for policy 1, policy_version 851982 (0.0008) [2023-12-27 11:32:44,790][20381] Updated weights for policy 1, policy_version 851992 (0.0008) [2023-12-27 11:32:45,376][20347] Updated weights for policy 0, policy_version 852730 (0.0011) [2023-12-27 11:32:45,437][20347] Updated weights for policy 0, policy_version 852740 (0.0008) [2023-12-27 11:32:45,496][20347] Updated weights for policy 0, policy_version 852750 (0.0009) [2023-12-27 11:32:45,519][20381] Updated weights for policy 1, policy_version 852002 (0.0006) [2023-12-27 11:32:45,580][20381] Updated weights for policy 1, policy_version 852012 (0.0008) [2023-12-27 11:32:45,644][20381] Updated weights for policy 1, policy_version 852022 (0.0007) [2023-12-27 11:32:46,167][20347] Updated weights for policy 0, policy_version 852760 (0.0010) [2023-12-27 11:32:46,220][20347] Updated weights for policy 0, policy_version 852770 (0.0010) [2023-12-27 11:32:46,276][20347] Updated weights for policy 0, policy_version 852780 (0.0008) [2023-12-27 11:32:46,325][20381] Updated weights for policy 1, policy_version 852032 (0.0010) [2023-12-27 11:32:46,385][20381] Updated weights for policy 1, policy_version 852042 (0.0010) [2023-12-27 11:32:46,449][20381] Updated weights for policy 1, policy_version 852052 (0.0010) [2023-12-27 11:32:46,954][20347] Updated weights for policy 0, policy_version 852790 (0.0010) [2023-12-27 11:32:47,012][20347] Updated weights for policy 0, policy_version 852800 (0.0010) [2023-12-27 11:32:47,077][20347] Updated weights for policy 0, policy_version 852810 (0.0010) [2023-12-27 11:32:47,103][20381] Updated weights for policy 1, policy_version 852062 (0.0011) [2023-12-27 11:32:47,164][20381] Updated weights for policy 1, policy_version 852072 (0.0011) [2023-12-27 11:32:47,217][20381] Updated weights for policy 1, policy_version 852082 (0.0011) [2023-12-27 11:32:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 436510720. Throughput: 0: 9732.9, 1: 9646.5. Samples: 436516544. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:47,363][19170] Avg episode reward: [(0, '314.810'), (1, '970.790')] [2023-12-27 11:32:47,715][20347] Updated weights for policy 0, policy_version 852820 (0.0009) [2023-12-27 11:32:47,769][20347] Updated weights for policy 0, policy_version 852830 (0.0009) [2023-12-27 11:32:47,832][20347] Updated weights for policy 0, policy_version 852840 (0.0009) [2023-12-27 11:32:47,833][20381] Updated weights for policy 1, policy_version 852092 (0.0009) [2023-12-27 11:32:47,898][20381] Updated weights for policy 1, policy_version 852102 (0.0006) [2023-12-27 11:32:47,961][20381] Updated weights for policy 1, policy_version 852112 (0.0009) [2023-12-27 11:32:48,511][20347] Updated weights for policy 0, policy_version 852850 (0.0009) [2023-12-27 11:32:48,574][20347] Updated weights for policy 0, policy_version 852860 (0.0007) [2023-12-27 11:32:48,635][20347] Updated weights for policy 0, policy_version 852870 (0.0011) [2023-12-27 11:32:48,683][20381] Updated weights for policy 1, policy_version 852122 (0.0010) [2023-12-27 11:32:48,702][20347] Updated weights for policy 0, policy_version 852880 (0.0009) [2023-12-27 11:32:48,751][20381] Updated weights for policy 1, policy_version 852132 (0.0009) [2023-12-27 11:32:48,816][20381] Updated weights for policy 1, policy_version 852142 (0.0008) [2023-12-27 11:32:48,883][20381] Updated weights for policy 1, policy_version 852152 (0.0011) [2023-12-27 11:32:49,385][20347] Updated weights for policy 0, policy_version 852890 (0.0009) [2023-12-27 11:32:49,452][20347] Updated weights for policy 0, policy_version 852900 (0.0008) [2023-12-27 11:32:49,509][20347] Updated weights for policy 0, policy_version 852910 (0.0010) [2023-12-27 11:32:49,570][20381] Updated weights for policy 1, policy_version 852162 (0.0008) [2023-12-27 11:32:49,632][20381] Updated weights for policy 1, policy_version 852172 (0.0008) [2023-12-27 11:32:49,691][20381] Updated weights for policy 1, policy_version 852182 (0.0008) [2023-12-27 11:32:50,208][20347] Updated weights for policy 0, policy_version 852920 (0.0007) [2023-12-27 11:32:50,270][20347] Updated weights for policy 0, policy_version 852930 (0.0006) [2023-12-27 11:32:50,336][20347] Updated weights for policy 0, policy_version 852940 (0.0007) [2023-12-27 11:32:50,417][20381] Updated weights for policy 1, policy_version 852192 (0.0008) [2023-12-27 11:32:50,477][20381] Updated weights for policy 1, policy_version 852202 (0.0006) [2023-12-27 11:32:50,546][20381] Updated weights for policy 1, policy_version 852212 (0.0006) [2023-12-27 11:32:51,003][20347] Updated weights for policy 0, policy_version 852950 (0.0008) [2023-12-27 11:32:51,073][20347] Updated weights for policy 0, policy_version 852960 (0.0007) [2023-12-27 11:32:51,138][20347] Updated weights for policy 0, policy_version 852970 (0.0006) [2023-12-27 11:32:51,211][20381] Updated weights for policy 1, policy_version 852222 (0.0007) [2023-12-27 11:32:51,276][20381] Updated weights for policy 1, policy_version 852232 (0.0009) [2023-12-27 11:32:51,326][20381] Updated weights for policy 1, policy_version 852242 (0.0007) [2023-12-27 11:32:51,816][20347] Updated weights for policy 0, policy_version 852980 (0.0008) [2023-12-27 11:32:51,887][20347] Updated weights for policy 0, policy_version 852990 (0.0007) [2023-12-27 11:32:51,950][20347] Updated weights for policy 0, policy_version 853000 (0.0007) [2023-12-27 11:32:52,081][20381] Updated weights for policy 1, policy_version 852252 (0.0008) [2023-12-27 11:32:52,138][20381] Updated weights for policy 1, policy_version 852262 (0.0008) [2023-12-27 11:32:52,188][20381] Updated weights for policy 1, policy_version 852272 (0.0010) [2023-12-27 11:32:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19383.1). Total num frames: 436609024. Throughput: 0: 9737.3, 1: 9647.3. Samples: 436576508. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:52,363][19170] Avg episode reward: [(0, '410.810'), (1, '971.440')] [2023-12-27 11:32:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000852280_218210304.pth... [2023-12-27 11:32:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000853008_218398720.pth... [2023-12-27 11:32:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000851128_217915392.pth [2023-12-27 11:32:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000851856_218103808.pth [2023-12-27 11:32:52,610][20347] Updated weights for policy 0, policy_version 853010 (0.0007) [2023-12-27 11:32:52,672][20347] Updated weights for policy 0, policy_version 853020 (0.0007) [2023-12-27 11:32:52,736][20347] Updated weights for policy 0, policy_version 853030 (0.0006) [2023-12-27 11:32:52,794][20347] Updated weights for policy 0, policy_version 853040 (0.0007) [2023-12-27 11:32:53,022][20381] Updated weights for policy 1, policy_version 852282 (0.0010) [2023-12-27 11:32:53,078][20381] Updated weights for policy 1, policy_version 852292 (0.0007) [2023-12-27 11:32:53,135][20381] Updated weights for policy 1, policy_version 852302 (0.0007) [2023-12-27 11:32:53,196][20381] Updated weights for policy 1, policy_version 852312 (0.0006) [2023-12-27 11:32:53,461][20347] Updated weights for policy 0, policy_version 853050 (0.0007) [2023-12-27 11:32:53,512][20347] Updated weights for policy 0, policy_version 853060 (0.0008) [2023-12-27 11:32:53,571][20347] Updated weights for policy 0, policy_version 853070 (0.0008) [2023-12-27 11:32:53,897][20381] Updated weights for policy 1, policy_version 852322 (0.0007) [2023-12-27 11:32:53,966][20381] Updated weights for policy 1, policy_version 852332 (0.0007) [2023-12-27 11:32:54,029][20381] Updated weights for policy 1, policy_version 852342 (0.0007) [2023-12-27 11:32:54,287][20347] Updated weights for policy 0, policy_version 853080 (0.0008) [2023-12-27 11:32:54,345][20347] Updated weights for policy 0, policy_version 853090 (0.0008) [2023-12-27 11:32:54,404][20347] Updated weights for policy 0, policy_version 853100 (0.0007) [2023-12-27 11:32:54,691][20381] Updated weights for policy 1, policy_version 852352 (0.0008) [2023-12-27 11:32:54,757][20381] Updated weights for policy 1, policy_version 852362 (0.0008) [2023-12-27 11:32:54,820][20381] Updated weights for policy 1, policy_version 852372 (0.0007) [2023-12-27 11:32:55,097][20347] Updated weights for policy 0, policy_version 853110 (0.0007) [2023-12-27 11:32:55,157][20347] Updated weights for policy 0, policy_version 853120 (0.0007) [2023-12-27 11:32:55,206][20347] Updated weights for policy 0, policy_version 853130 (0.0006) [2023-12-27 11:32:55,483][20381] Updated weights for policy 1, policy_version 852382 (0.0007) [2023-12-27 11:32:55,537][20381] Updated weights for policy 1, policy_version 852392 (0.0008) [2023-12-27 11:32:55,591][20381] Updated weights for policy 1, policy_version 852402 (0.0007) [2023-12-27 11:32:55,926][20347] Updated weights for policy 0, policy_version 853140 (0.0007) [2023-12-27 11:32:55,983][20347] Updated weights for policy 0, policy_version 853150 (0.0008) [2023-12-27 11:32:56,037][20347] Updated weights for policy 0, policy_version 853160 (0.0008) [2023-12-27 11:32:56,233][20381] Updated weights for policy 1, policy_version 852412 (0.0008) [2023-12-27 11:32:56,293][20381] Updated weights for policy 1, policy_version 852422 (0.0008) [2023-12-27 11:32:56,354][20381] Updated weights for policy 1, policy_version 852432 (0.0007) [2023-12-27 11:32:56,751][20347] Updated weights for policy 0, policy_version 853170 (0.0008) [2023-12-27 11:32:56,809][20347] Updated weights for policy 0, policy_version 853180 (0.0010) [2023-12-27 11:32:56,869][20347] Updated weights for policy 0, policy_version 853190 (0.0010) [2023-12-27 11:32:56,929][20347] Updated weights for policy 0, policy_version 853200 (0.0010) [2023-12-27 11:32:57,075][20381] Updated weights for policy 1, policy_version 852442 (0.0007) [2023-12-27 11:32:57,126][20381] Updated weights for policy 1, policy_version 852452 (0.0008) [2023-12-27 11:32:57,183][20381] Updated weights for policy 1, policy_version 852462 (0.0008) [2023-12-27 11:32:57,244][20381] Updated weights for policy 1, policy_version 852472 (0.0008) [2023-12-27 11:32:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19438.6). Total num frames: 436707328. Throughput: 0: 9801.4, 1: 9668.9. Samples: 436695764. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:32:57,363][19170] Avg episode reward: [(0, '389.800'), (1, '971.410')] [2023-12-27 11:32:57,563][20347] Updated weights for policy 0, policy_version 853210 (0.0010) [2023-12-27 11:32:57,616][20347] Updated weights for policy 0, policy_version 853220 (0.0011) [2023-12-27 11:32:57,672][20347] Updated weights for policy 0, policy_version 853230 (0.0011) [2023-12-27 11:32:57,928][20381] Updated weights for policy 1, policy_version 852482 (0.0008) [2023-12-27 11:32:57,986][20381] Updated weights for policy 1, policy_version 852492 (0.0008) [2023-12-27 11:32:58,040][20381] Updated weights for policy 1, policy_version 852502 (0.0007) [2023-12-27 11:32:58,364][20347] Updated weights for policy 0, policy_version 853240 (0.0009) [2023-12-27 11:32:58,427][20347] Updated weights for policy 0, policy_version 853250 (0.0008) [2023-12-27 11:32:58,489][20347] Updated weights for policy 0, policy_version 853260 (0.0007) [2023-12-27 11:32:58,843][20381] Updated weights for policy 1, policy_version 852512 (0.0008) [2023-12-27 11:32:58,923][20381] Updated weights for policy 1, policy_version 852522 (0.0008) [2023-12-27 11:32:58,979][20381] Updated weights for policy 1, policy_version 852532 (0.0008) [2023-12-27 11:32:59,218][20347] Updated weights for policy 0, policy_version 853270 (0.0008) [2023-12-27 11:32:59,281][20347] Updated weights for policy 0, policy_version 853280 (0.0009) [2023-12-27 11:32:59,350][20347] Updated weights for policy 0, policy_version 853290 (0.0008) [2023-12-27 11:32:59,710][20381] Updated weights for policy 1, policy_version 852542 (0.0007) [2023-12-27 11:32:59,767][20381] Updated weights for policy 1, policy_version 852552 (0.0008) [2023-12-27 11:32:59,828][20381] Updated weights for policy 1, policy_version 852562 (0.0008) [2023-12-27 11:33:00,080][20347] Updated weights for policy 0, policy_version 853300 (0.0007) [2023-12-27 11:33:00,136][20347] Updated weights for policy 0, policy_version 853310 (0.0007) [2023-12-27 11:33:00,201][20347] Updated weights for policy 0, policy_version 853320 (0.0008) [2023-12-27 11:33:00,526][20381] Updated weights for policy 1, policy_version 852572 (0.0009) [2023-12-27 11:33:00,584][20381] Updated weights for policy 1, policy_version 852582 (0.0005) [2023-12-27 11:33:00,644][20381] Updated weights for policy 1, policy_version 852592 (0.0006) [2023-12-27 11:33:00,802][20347] Updated weights for policy 0, policy_version 853330 (0.0008) [2023-12-27 11:33:00,861][20347] Updated weights for policy 0, policy_version 853340 (0.0006) [2023-12-27 11:33:00,913][20347] Updated weights for policy 0, policy_version 853350 (0.0005) [2023-12-27 11:33:00,965][20347] Updated weights for policy 0, policy_version 853360 (0.0008) [2023-12-27 11:33:01,264][20381] Updated weights for policy 1, policy_version 852602 (0.0006) [2023-12-27 11:33:01,325][20381] Updated weights for policy 1, policy_version 852612 (0.0007) [2023-12-27 11:33:01,402][20381] Updated weights for policy 1, policy_version 852622 (0.0008) [2023-12-27 11:33:01,470][20381] Updated weights for policy 1, policy_version 852632 (0.0009) [2023-12-27 11:33:01,676][20347] Updated weights for policy 0, policy_version 853370 (0.0007) [2023-12-27 11:33:01,740][20347] Updated weights for policy 0, policy_version 853380 (0.0007) [2023-12-27 11:33:01,808][20347] Updated weights for policy 0, policy_version 853390 (0.0009) [2023-12-27 11:33:02,120][20381] Updated weights for policy 1, policy_version 852642 (0.0008) [2023-12-27 11:33:02,173][20381] Updated weights for policy 1, policy_version 852652 (0.0008) [2023-12-27 11:33:02,232][20381] Updated weights for policy 1, policy_version 852662 (0.0008) [2023-12-27 11:33:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 436805632. Throughput: 0: 9816.4, 1: 9686.0. Samples: 436813604. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:02,363][19170] Avg episode reward: [(0, '403.610'), (1, '951.810')] [2023-12-27 11:33:02,539][20347] Updated weights for policy 0, policy_version 853400 (0.0008) [2023-12-27 11:33:02,590][20347] Updated weights for policy 0, policy_version 853410 (0.0009) [2023-12-27 11:33:02,647][20347] Updated weights for policy 0, policy_version 853420 (0.0009) [2023-12-27 11:33:02,918][20381] Updated weights for policy 1, policy_version 852672 (0.0007) [2023-12-27 11:33:02,988][20381] Updated weights for policy 1, policy_version 852682 (0.0007) [2023-12-27 11:33:03,061][20381] Updated weights for policy 1, policy_version 852692 (0.0010) [2023-12-27 11:33:03,363][20347] Updated weights for policy 0, policy_version 853430 (0.0008) [2023-12-27 11:33:03,427][20347] Updated weights for policy 0, policy_version 853440 (0.0007) [2023-12-27 11:33:03,478][20347] Updated weights for policy 0, policy_version 853450 (0.0006) [2023-12-27 11:33:03,762][20381] Updated weights for policy 1, policy_version 852702 (0.0009) [2023-12-27 11:33:03,815][20381] Updated weights for policy 1, policy_version 852712 (0.0008) [2023-12-27 11:33:03,883][20381] Updated weights for policy 1, policy_version 852722 (0.0008) [2023-12-27 11:33:04,183][20347] Updated weights for policy 0, policy_version 853460 (0.0007) [2023-12-27 11:33:04,243][20347] Updated weights for policy 0, policy_version 853470 (0.0008) [2023-12-27 11:33:04,297][20347] Updated weights for policy 0, policy_version 853480 (0.0009) [2023-12-27 11:33:04,577][20381] Updated weights for policy 1, policy_version 852732 (0.0007) [2023-12-27 11:33:04,632][20381] Updated weights for policy 1, policy_version 852742 (0.0007) [2023-12-27 11:33:04,679][20381] Updated weights for policy 1, policy_version 852752 (0.0008) [2023-12-27 11:33:05,012][20347] Updated weights for policy 0, policy_version 853490 (0.0007) [2023-12-27 11:33:05,073][20347] Updated weights for policy 0, policy_version 853500 (0.0008) [2023-12-27 11:33:05,125][20347] Updated weights for policy 0, policy_version 853510 (0.0007) [2023-12-27 11:33:05,192][20347] Updated weights for policy 0, policy_version 853520 (0.0008) [2023-12-27 11:33:05,406][20381] Updated weights for policy 1, policy_version 852762 (0.0008) [2023-12-27 11:33:05,460][20381] Updated weights for policy 1, policy_version 852772 (0.0007) [2023-12-27 11:33:05,513][20381] Updated weights for policy 1, policy_version 852782 (0.0007) [2023-12-27 11:33:05,568][20381] Updated weights for policy 1, policy_version 852792 (0.0008) [2023-12-27 11:33:05,875][20347] Updated weights for policy 0, policy_version 853530 (0.0008) [2023-12-27 11:33:05,935][20347] Updated weights for policy 0, policy_version 853540 (0.0007) [2023-12-27 11:33:05,992][20347] Updated weights for policy 0, policy_version 853550 (0.0008) [2023-12-27 11:33:06,311][20381] Updated weights for policy 1, policy_version 852802 (0.0008) [2023-12-27 11:33:06,380][20381] Updated weights for policy 1, policy_version 852812 (0.0008) [2023-12-27 11:33:06,450][20381] Updated weights for policy 1, policy_version 852822 (0.0008) [2023-12-27 11:33:06,731][20347] Updated weights for policy 0, policy_version 853560 (0.0008) [2023-12-27 11:33:06,788][20347] Updated weights for policy 0, policy_version 853570 (0.0007) [2023-12-27 11:33:06,848][20347] Updated weights for policy 0, policy_version 853580 (0.0007) [2023-12-27 11:33:07,174][20381] Updated weights for policy 1, policy_version 852832 (0.0008) [2023-12-27 11:33:07,230][20381] Updated weights for policy 1, policy_version 852842 (0.0007) [2023-12-27 11:33:07,292][20381] Updated weights for policy 1, policy_version 852852 (0.0007) [2023-12-27 11:33:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19524.3, 300 sec: 19438.7). Total num frames: 436903936. Throughput: 0: 9835.4, 1: 9720.6. Samples: 436872376. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:07,363][19170] Avg episode reward: [(0, '361.810'), (1, '951.870')] [2023-12-27 11:33:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000852856_218357760.pth... [2023-12-27 11:33:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000853584_218546176.pth... [2023-12-27 11:33:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000851704_218062848.pth [2023-12-27 11:33:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000852432_218251264.pth [2023-12-27 11:33:07,566][20347] Updated weights for policy 0, policy_version 853590 (0.0008) [2023-12-27 11:33:07,623][20347] Updated weights for policy 0, policy_version 853600 (0.0008) [2023-12-27 11:33:07,682][20347] Updated weights for policy 0, policy_version 853610 (0.0007) [2023-12-27 11:33:08,014][20381] Updated weights for policy 1, policy_version 852862 (0.0007) [2023-12-27 11:33:08,068][20381] Updated weights for policy 1, policy_version 852872 (0.0008) [2023-12-27 11:33:08,129][20381] Updated weights for policy 1, policy_version 852882 (0.0007) [2023-12-27 11:33:08,387][20347] Updated weights for policy 0, policy_version 853620 (0.0007) [2023-12-27 11:33:08,455][20347] Updated weights for policy 0, policy_version 853630 (0.0006) [2023-12-27 11:33:08,522][20347] Updated weights for policy 0, policy_version 853640 (0.0006) [2023-12-27 11:33:08,812][20381] Updated weights for policy 1, policy_version 852892 (0.0008) [2023-12-27 11:33:08,877][20381] Updated weights for policy 1, policy_version 852902 (0.0007) [2023-12-27 11:33:08,938][20381] Updated weights for policy 1, policy_version 852912 (0.0008) [2023-12-27 11:33:09,204][20347] Updated weights for policy 0, policy_version 853650 (0.0006) [2023-12-27 11:33:09,269][20347] Updated weights for policy 0, policy_version 853660 (0.0010) [2023-12-27 11:33:09,329][20347] Updated weights for policy 0, policy_version 853670 (0.0010) [2023-12-27 11:33:09,396][20347] Updated weights for policy 0, policy_version 853680 (0.0010) [2023-12-27 11:33:09,659][20381] Updated weights for policy 1, policy_version 852922 (0.0008) [2023-12-27 11:33:09,716][20381] Updated weights for policy 1, policy_version 852932 (0.0008) [2023-12-27 11:33:09,779][20381] Updated weights for policy 1, policy_version 852942 (0.0008) [2023-12-27 11:33:09,845][20381] Updated weights for policy 1, policy_version 852952 (0.0008) [2023-12-27 11:33:10,138][20347] Updated weights for policy 0, policy_version 853690 (0.0006) [2023-12-27 11:33:10,205][20347] Updated weights for policy 0, policy_version 853700 (0.0007) [2023-12-27 11:33:10,263][20347] Updated weights for policy 0, policy_version 853710 (0.0008) [2023-12-27 11:33:10,569][20381] Updated weights for policy 1, policy_version 852962 (0.0008) [2023-12-27 11:33:10,631][20381] Updated weights for policy 1, policy_version 852972 (0.0007) [2023-12-27 11:33:10,695][20381] Updated weights for policy 1, policy_version 852982 (0.0006) [2023-12-27 11:33:11,006][20347] Updated weights for policy 0, policy_version 853720 (0.0007) [2023-12-27 11:33:11,068][20347] Updated weights for policy 0, policy_version 853730 (0.0008) [2023-12-27 11:33:11,123][20347] Updated weights for policy 0, policy_version 853740 (0.0007) [2023-12-27 11:33:11,396][20381] Updated weights for policy 1, policy_version 852992 (0.0009) [2023-12-27 11:33:11,463][20381] Updated weights for policy 1, policy_version 853002 (0.0008) [2023-12-27 11:33:11,529][20381] Updated weights for policy 1, policy_version 853012 (0.0007) [2023-12-27 11:33:11,974][20347] Updated weights for policy 0, policy_version 853750 (0.0008) [2023-12-27 11:33:12,040][20347] Updated weights for policy 0, policy_version 853760 (0.0008) [2023-12-27 11:33:12,103][20347] Updated weights for policy 0, policy_version 853770 (0.0008) [2023-12-27 11:33:12,263][20381] Updated weights for policy 1, policy_version 853022 (0.0007) [2023-12-27 11:33:12,327][20381] Updated weights for policy 1, policy_version 853032 (0.0007) [2023-12-27 11:33:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 436994048. Throughput: 0: 9806.0, 1: 9759.8. Samples: 436988292. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:12,363][19170] Avg episode reward: [(0, '341.370'), (1, '941.610')] [2023-12-27 11:33:12,405][20381] Updated weights for policy 1, policy_version 853042 (0.0010) [2023-12-27 11:33:12,789][20347] Updated weights for policy 0, policy_version 853780 (0.0007) [2023-12-27 11:33:12,843][20347] Updated weights for policy 0, policy_version 853790 (0.0007) [2023-12-27 11:33:12,901][20347] Updated weights for policy 0, policy_version 853800 (0.0008) [2023-12-27 11:33:13,047][20381] Updated weights for policy 1, policy_version 853052 (0.0007) [2023-12-27 11:33:13,100][20381] Updated weights for policy 1, policy_version 853062 (0.0007) [2023-12-27 11:33:13,150][20292] KL-divergence is very high: 407.8127 [2023-12-27 11:33:13,157][20381] Updated weights for policy 1, policy_version 853072 (0.0008) [2023-12-27 11:33:13,566][20347] Updated weights for policy 0, policy_version 853810 (0.0008) [2023-12-27 11:33:13,620][20347] Updated weights for policy 0, policy_version 853820 (0.0008) [2023-12-27 11:33:13,680][20347] Updated weights for policy 0, policy_version 853830 (0.0007) [2023-12-27 11:33:13,748][20347] Updated weights for policy 0, policy_version 853840 (0.0008) [2023-12-27 11:33:13,836][20381] Updated weights for policy 1, policy_version 853082 (0.0007) [2023-12-27 11:33:13,889][20381] Updated weights for policy 1, policy_version 853092 (0.0005) [2023-12-27 11:33:13,942][20381] Updated weights for policy 1, policy_version 853102 (0.0006) [2023-12-27 11:33:13,999][20381] Updated weights for policy 1, policy_version 853112 (0.0007) [2023-12-27 11:33:14,478][20347] Updated weights for policy 0, policy_version 853850 (0.0007) [2023-12-27 11:33:14,535][20347] Updated weights for policy 0, policy_version 853860 (0.0008) [2023-12-27 11:33:14,588][20347] Updated weights for policy 0, policy_version 853870 (0.0008) [2023-12-27 11:33:14,629][20381] Updated weights for policy 1, policy_version 853122 (0.0006) [2023-12-27 11:33:14,684][20381] Updated weights for policy 1, policy_version 853132 (0.0006) [2023-12-27 11:33:14,748][20381] Updated weights for policy 1, policy_version 853142 (0.0007) [2023-12-27 11:33:15,358][20347] Updated weights for policy 0, policy_version 853880 (0.0007) [2023-12-27 11:33:15,417][20347] Updated weights for policy 0, policy_version 853890 (0.0009) [2023-12-27 11:33:15,423][20381] Updated weights for policy 1, policy_version 853152 (0.0009) [2023-12-27 11:33:15,475][20347] Updated weights for policy 0, policy_version 853900 (0.0006) [2023-12-27 11:33:15,482][20381] Updated weights for policy 1, policy_version 853162 (0.0008) [2023-12-27 11:33:15,546][20381] Updated weights for policy 1, policy_version 853172 (0.0008) [2023-12-27 11:33:16,165][20347] Updated weights for policy 0, policy_version 853910 (0.0007) [2023-12-27 11:33:16,226][20347] Updated weights for policy 0, policy_version 853920 (0.0007) [2023-12-27 11:33:16,263][20381] Updated weights for policy 1, policy_version 853182 (0.0007) [2023-12-27 11:33:16,293][20347] Updated weights for policy 0, policy_version 853930 (0.0007) [2023-12-27 11:33:16,320][20381] Updated weights for policy 1, policy_version 853192 (0.0009) [2023-12-27 11:33:16,375][20381] Updated weights for policy 1, policy_version 853202 (0.0008) [2023-12-27 11:33:16,982][20347] Updated weights for policy 0, policy_version 853940 (0.0007) [2023-12-27 11:33:17,042][20347] Updated weights for policy 0, policy_version 853950 (0.0008) [2023-12-27 11:33:17,093][20347] Updated weights for policy 0, policy_version 853960 (0.0007) [2023-12-27 11:33:17,094][20381] Updated weights for policy 1, policy_version 853212 (0.0007) [2023-12-27 11:33:17,152][20381] Updated weights for policy 1, policy_version 853222 (0.0009) [2023-12-27 11:33:17,207][20381] Updated weights for policy 1, policy_version 853232 (0.0007) [2023-12-27 11:33:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 19438.7). Total num frames: 437100544. Throughput: 0: 9742.1, 1: 9783.1. Samples: 437104916. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:17,363][19170] Avg episode reward: [(0, '348.640'), (1, '932.580')] [2023-12-27 11:33:17,813][20347] Updated weights for policy 0, policy_version 853970 (0.0009) [2023-12-27 11:33:17,873][20347] Updated weights for policy 0, policy_version 853980 (0.0008) [2023-12-27 11:33:17,919][20347] Updated weights for policy 0, policy_version 853990 (0.0008) [2023-12-27 11:33:17,928][20381] Updated weights for policy 1, policy_version 853242 (0.0008) [2023-12-27 11:33:17,982][20381] Updated weights for policy 1, policy_version 853252 (0.0007) [2023-12-27 11:33:17,983][20347] Updated weights for policy 0, policy_version 854000 (0.0007) [2023-12-27 11:33:18,035][20381] Updated weights for policy 1, policy_version 853262 (0.0007) [2023-12-27 11:33:18,089][20381] Updated weights for policy 1, policy_version 853272 (0.0008) [2023-12-27 11:33:18,734][20347] Updated weights for policy 0, policy_version 854010 (0.0008) [2023-12-27 11:33:18,790][20347] Updated weights for policy 0, policy_version 854020 (0.0008) [2023-12-27 11:33:18,854][20347] Updated weights for policy 0, policy_version 854030 (0.0007) [2023-12-27 11:33:18,942][20381] Updated weights for policy 1, policy_version 853282 (0.0007) [2023-12-27 11:33:19,003][20381] Updated weights for policy 1, policy_version 853292 (0.0009) [2023-12-27 11:33:19,060][20381] Updated weights for policy 1, policy_version 853302 (0.0007) [2023-12-27 11:33:19,615][20347] Updated weights for policy 0, policy_version 854040 (0.0009) [2023-12-27 11:33:19,680][20347] Updated weights for policy 0, policy_version 854050 (0.0007) [2023-12-27 11:33:19,738][20347] Updated weights for policy 0, policy_version 854060 (0.0007) [2023-12-27 11:33:19,781][20381] Updated weights for policy 1, policy_version 853312 (0.0008) [2023-12-27 11:33:19,845][20381] Updated weights for policy 1, policy_version 853322 (0.0008) [2023-12-27 11:33:19,904][20381] Updated weights for policy 1, policy_version 853332 (0.0006) [2023-12-27 11:33:20,458][20347] Updated weights for policy 0, policy_version 854070 (0.0006) [2023-12-27 11:33:20,511][20347] Updated weights for policy 0, policy_version 854080 (0.0007) [2023-12-27 11:33:20,562][20347] Updated weights for policy 0, policy_version 854090 (0.0007) [2023-12-27 11:33:20,678][20381] Updated weights for policy 1, policy_version 853342 (0.0008) [2023-12-27 11:33:20,744][20381] Updated weights for policy 1, policy_version 853352 (0.0009) [2023-12-27 11:33:20,801][20381] Updated weights for policy 1, policy_version 853362 (0.0008) [2023-12-27 11:33:21,311][20347] Updated weights for policy 0, policy_version 854100 (0.0009) [2023-12-27 11:33:21,381][20347] Updated weights for policy 0, policy_version 854110 (0.0008) [2023-12-27 11:33:21,448][20347] Updated weights for policy 0, policy_version 854120 (0.0009) [2023-12-27 11:33:21,574][20381] Updated weights for policy 1, policy_version 853372 (0.0007) [2023-12-27 11:33:21,638][20381] Updated weights for policy 1, policy_version 853382 (0.0007) [2023-12-27 11:33:21,703][20381] Updated weights for policy 1, policy_version 853392 (0.0007) [2023-12-27 11:33:22,187][20347] Updated weights for policy 0, policy_version 854130 (0.0007) [2023-12-27 11:33:22,248][20347] Updated weights for policy 0, policy_version 854140 (0.0008) [2023-12-27 11:33:22,307][20347] Updated weights for policy 0, policy_version 854150 (0.0008) [2023-12-27 11:33:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 19355.3). Total num frames: 437182464. Throughput: 0: 9735.6, 1: 9762.7. Samples: 437161368. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:22,363][19170] Avg episode reward: [(0, '382.670'), (1, '942.760')] [2023-12-27 11:33:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000853400_218497024.pth... [2023-12-27 11:33:22,370][20347] Updated weights for policy 0, policy_version 854160 (0.0008) [2023-12-27 11:33:22,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000854160_218693632.pth... [2023-12-27 11:33:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000852280_218210304.pth [2023-12-27 11:33:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000853008_218398720.pth [2023-12-27 11:33:22,433][20381] Updated weights for policy 1, policy_version 853402 (0.0008) [2023-12-27 11:33:22,496][20381] Updated weights for policy 1, policy_version 853412 (0.0008) [2023-12-27 11:33:22,553][20381] Updated weights for policy 1, policy_version 853422 (0.0008) [2023-12-27 11:33:22,614][20381] Updated weights for policy 1, policy_version 853432 (0.0007) [2023-12-27 11:33:23,081][20347] Updated weights for policy 0, policy_version 854170 (0.0008) [2023-12-27 11:33:23,143][20347] Updated weights for policy 0, policy_version 854180 (0.0008) [2023-12-27 11:33:23,201][20347] Updated weights for policy 0, policy_version 854190 (0.0008) [2023-12-27 11:33:23,334][20381] Updated weights for policy 1, policy_version 853442 (0.0008) [2023-12-27 11:33:23,393][20381] Updated weights for policy 1, policy_version 853452 (0.0008) [2023-12-27 11:33:23,462][20381] Updated weights for policy 1, policy_version 853462 (0.0006) [2023-12-27 11:33:23,894][20347] Updated weights for policy 0, policy_version 854200 (0.0008) [2023-12-27 11:33:23,950][20347] Updated weights for policy 0, policy_version 854210 (0.0008) [2023-12-27 11:33:24,014][20347] Updated weights for policy 0, policy_version 854220 (0.0007) [2023-12-27 11:33:24,135][20381] Updated weights for policy 1, policy_version 853472 (0.0007) [2023-12-27 11:33:24,194][20381] Updated weights for policy 1, policy_version 853482 (0.0008) [2023-12-27 11:33:24,249][20381] Updated weights for policy 1, policy_version 853492 (0.0008) [2023-12-27 11:33:24,747][20347] Updated weights for policy 0, policy_version 854230 (0.0008) [2023-12-27 11:33:24,798][20347] Updated weights for policy 0, policy_version 854240 (0.0007) [2023-12-27 11:33:24,843][20347] Updated weights for policy 0, policy_version 854250 (0.0007) [2023-12-27 11:33:24,916][20381] Updated weights for policy 1, policy_version 853502 (0.0006) [2023-12-27 11:33:24,971][20381] Updated weights for policy 1, policy_version 853512 (0.0005) [2023-12-27 11:33:25,020][20381] Updated weights for policy 1, policy_version 853522 (0.0006) [2023-12-27 11:33:25,435][20347] Updated weights for policy 0, policy_version 854260 (0.0007) [2023-12-27 11:33:25,481][20347] Updated weights for policy 0, policy_version 854270 (0.0007) [2023-12-27 11:33:25,527][20347] Updated weights for policy 0, policy_version 854280 (0.0007) [2023-12-27 11:33:25,648][20381] Updated weights for policy 1, policy_version 853532 (0.0010) [2023-12-27 11:33:25,701][20381] Updated weights for policy 1, policy_version 853542 (0.0006) [2023-12-27 11:33:25,752][20381] Updated weights for policy 1, policy_version 853552 (0.0005) [2023-12-27 11:33:26,266][20347] Updated weights for policy 0, policy_version 854290 (0.0007) [2023-12-27 11:33:26,315][20347] Updated weights for policy 0, policy_version 854300 (0.0007) [2023-12-27 11:33:26,369][20347] Updated weights for policy 0, policy_version 854310 (0.0008) [2023-12-27 11:33:26,415][20347] Updated weights for policy 0, policy_version 854320 (0.0008) [2023-12-27 11:33:26,426][20381] Updated weights for policy 1, policy_version 853562 (0.0006) [2023-12-27 11:33:26,473][20381] Updated weights for policy 1, policy_version 853572 (0.0006) [2023-12-27 11:33:26,515][20381] Updated weights for policy 1, policy_version 853582 (0.0008) [2023-12-27 11:33:26,560][20381] Updated weights for policy 1, policy_version 853592 (0.0008) [2023-12-27 11:33:27,101][20347] Updated weights for policy 0, policy_version 854330 (0.0007) [2023-12-27 11:33:27,154][20347] Updated weights for policy 0, policy_version 854340 (0.0007) [2023-12-27 11:33:27,201][20347] Updated weights for policy 0, policy_version 854350 (0.0008) [2023-12-27 11:33:27,270][20381] Updated weights for policy 1, policy_version 853602 (0.0008) [2023-12-27 11:33:27,321][20381] Updated weights for policy 1, policy_version 853612 (0.0007) [2023-12-27 11:33:27,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 437288960. Throughput: 0: 9750.1, 1: 9785.4. Samples: 437279472. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:27,363][19170] Avg episode reward: [(0, '428.340'), (1, '932.980')] [2023-12-27 11:33:27,367][20381] Updated weights for policy 1, policy_version 853622 (0.0008) [2023-12-27 11:33:27,931][20347] Updated weights for policy 0, policy_version 854360 (0.0007) [2023-12-27 11:33:27,982][20381] Updated weights for policy 1, policy_version 853632 (0.0006) [2023-12-27 11:33:27,986][20347] Updated weights for policy 0, policy_version 854370 (0.0008) [2023-12-27 11:33:28,035][20381] Updated weights for policy 1, policy_version 853642 (0.0006) [2023-12-27 11:33:28,043][20347] Updated weights for policy 0, policy_version 854380 (0.0006) [2023-12-27 11:33:28,093][20381] Updated weights for policy 1, policy_version 853652 (0.0006) [2023-12-27 11:33:28,727][20381] Updated weights for policy 1, policy_version 853662 (0.0007) [2023-12-27 11:33:28,752][20347] Updated weights for policy 0, policy_version 854390 (0.0006) [2023-12-27 11:33:28,786][20381] Updated weights for policy 1, policy_version 853672 (0.0006) [2023-12-27 11:33:28,811][20347] Updated weights for policy 0, policy_version 854400 (0.0006) [2023-12-27 11:33:28,843][20381] Updated weights for policy 1, policy_version 853682 (0.0005) [2023-12-27 11:33:28,867][20347] Updated weights for policy 0, policy_version 854410 (0.0007) [2023-12-27 11:33:29,480][20381] Updated weights for policy 1, policy_version 853692 (0.0006) [2023-12-27 11:33:29,488][20347] Updated weights for policy 0, policy_version 854420 (0.0007) [2023-12-27 11:33:29,535][20381] Updated weights for policy 1, policy_version 853702 (0.0006) [2023-12-27 11:33:29,547][20347] Updated weights for policy 0, policy_version 854430 (0.0006) [2023-12-27 11:33:29,594][20381] Updated weights for policy 1, policy_version 853712 (0.0005) [2023-12-27 11:33:29,604][20347] Updated weights for policy 0, policy_version 854440 (0.0005) [2023-12-27 11:33:30,302][20347] Updated weights for policy 0, policy_version 854450 (0.0006) [2023-12-27 11:33:30,328][20381] Updated weights for policy 1, policy_version 853722 (0.0006) [2023-12-27 11:33:30,367][20347] Updated weights for policy 0, policy_version 854460 (0.0008) [2023-12-27 11:33:30,388][20381] Updated weights for policy 1, policy_version 853732 (0.0005) [2023-12-27 11:33:30,421][20347] Updated weights for policy 0, policy_version 854470 (0.0008) [2023-12-27 11:33:30,435][20381] Updated weights for policy 1, policy_version 853742 (0.0005) [2023-12-27 11:33:30,479][20347] Updated weights for policy 0, policy_version 854480 (0.0008) [2023-12-27 11:33:30,484][20381] Updated weights for policy 1, policy_version 853752 (0.0006) [2023-12-27 11:33:31,139][20347] Updated weights for policy 0, policy_version 854490 (0.0010) [2023-12-27 11:33:31,173][20381] Updated weights for policy 1, policy_version 853762 (0.0006) [2023-12-27 11:33:31,195][20347] Updated weights for policy 0, policy_version 854500 (0.0007) [2023-12-27 11:33:31,238][20381] Updated weights for policy 1, policy_version 853772 (0.0007) [2023-12-27 11:33:31,251][20347] Updated weights for policy 0, policy_version 854510 (0.0006) [2023-12-27 11:33:31,297][20381] Updated weights for policy 1, policy_version 853782 (0.0007) [2023-12-27 11:33:31,949][20347] Updated weights for policy 0, policy_version 854520 (0.0009) [2023-12-27 11:33:32,006][20381] Updated weights for policy 1, policy_version 853792 (0.0007) [2023-12-27 11:33:32,011][20347] Updated weights for policy 0, policy_version 854530 (0.0011) [2023-12-27 11:33:32,030][20292] KL-divergence is very high: 185.4007 [2023-12-27 11:33:32,036][20292] KL-divergence is very high: 160.2407 [2023-12-27 11:33:32,065][20381] Updated weights for policy 1, policy_version 853802 (0.0006) [2023-12-27 11:33:32,067][20347] Updated weights for policy 0, policy_version 854540 (0.0010) [2023-12-27 11:33:32,119][20381] Updated weights for policy 1, policy_version 853812 (0.0007) [2023-12-27 11:33:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 437395456. Throughput: 0: 9814.2, 1: 9857.0. Samples: 437401748. Policy #0 lag: (min: 22.0, avg: 26.2, max: 54.0) [2023-12-27 11:33:32,363][19170] Avg episode reward: [(0, '553.890'), (1, '923.420')] [2023-12-27 11:33:32,725][20347] Updated weights for policy 0, policy_version 854550 (0.0010) [2023-12-27 11:33:32,734][20292] KL-divergence is very high: 523.9915 [2023-12-27 11:33:32,760][20381] Updated weights for policy 1, policy_version 853822 (0.0006) [2023-12-27 11:33:32,781][20347] Updated weights for policy 0, policy_version 854560 (0.0010) [2023-12-27 11:33:32,783][20292] KL-divergence is very high: 175.9862 [2023-12-27 11:33:32,818][20381] Updated weights for policy 1, policy_version 853832 (0.0005) [2023-12-27 11:33:32,837][20347] Updated weights for policy 0, policy_version 854570 (0.0010) [2023-12-27 11:33:32,877][20381] Updated weights for policy 1, policy_version 853842 (0.0006) [2023-12-27 11:33:33,519][20347] Updated weights for policy 0, policy_version 854580 (0.0010) [2023-12-27 11:33:33,527][20292] KL-divergence is very high: 118.4874 [2023-12-27 11:33:33,552][20381] Updated weights for policy 1, policy_version 853852 (0.0009) [2023-12-27 11:33:33,579][20347] Updated weights for policy 0, policy_version 854590 (0.0011) [2023-12-27 11:33:33,609][20381] Updated weights for policy 1, policy_version 853862 (0.0011) [2023-12-27 11:33:33,637][20347] Updated weights for policy 0, policy_version 854600 (0.0010) [2023-12-27 11:33:33,666][20381] Updated weights for policy 1, policy_version 853872 (0.0010) [2023-12-27 11:33:34,302][20347] Updated weights for policy 0, policy_version 854610 (0.0010) [2023-12-27 11:33:34,331][20381] Updated weights for policy 1, policy_version 853882 (0.0010) [2023-12-27 11:33:34,361][20347] Updated weights for policy 0, policy_version 854620 (0.0010) [2023-12-27 11:33:34,396][20381] Updated weights for policy 1, policy_version 853892 (0.0009) [2023-12-27 11:33:34,425][20347] Updated weights for policy 0, policy_version 854630 (0.0010) [2023-12-27 11:33:34,458][20381] Updated weights for policy 1, policy_version 853902 (0.0008) [2023-12-27 11:33:34,486][20347] Updated weights for policy 0, policy_version 854640 (0.0008) [2023-12-27 11:33:34,523][20381] Updated weights for policy 1, policy_version 853912 (0.0007) [2023-12-27 11:33:35,140][20347] Updated weights for policy 0, policy_version 854650 (0.0005) [2023-12-27 11:33:35,198][20347] Updated weights for policy 0, policy_version 854660 (0.0005) [2023-12-27 11:33:35,200][20381] Updated weights for policy 1, policy_version 853922 (0.0010) [2023-12-27 11:33:35,252][20381] Updated weights for policy 1, policy_version 853932 (0.0007) [2023-12-27 11:33:35,262][20347] Updated weights for policy 0, policy_version 854670 (0.0006) [2023-12-27 11:33:35,309][20381] Updated weights for policy 1, policy_version 853942 (0.0007) [2023-12-27 11:33:35,827][20347] Updated weights for policy 0, policy_version 854680 (0.0008) [2023-12-27 11:33:35,887][20347] Updated weights for policy 0, policy_version 854690 (0.0006) [2023-12-27 11:33:35,943][20381] Updated weights for policy 1, policy_version 853952 (0.0007) [2023-12-27 11:33:35,950][20347] Updated weights for policy 0, policy_version 854700 (0.0006) [2023-12-27 11:33:35,996][20381] Updated weights for policy 1, policy_version 853962 (0.0006) [2023-12-27 11:33:36,054][20381] Updated weights for policy 1, policy_version 853972 (0.0007) [2023-12-27 11:33:36,626][20347] Updated weights for policy 0, policy_version 854710 (0.0009) [2023-12-27 11:33:36,676][20347] Updated weights for policy 0, policy_version 854720 (0.0008) [2023-12-27 11:33:36,727][20347] Updated weights for policy 0, policy_version 854730 (0.0007) [2023-12-27 11:33:36,735][20381] Updated weights for policy 1, policy_version 853982 (0.0007) [2023-12-27 11:33:36,791][20381] Updated weights for policy 1, policy_version 853992 (0.0009) [2023-12-27 11:33:36,843][20381] Updated weights for policy 1, policy_version 854002 (0.0008) [2023-12-27 11:33:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 19660.7, 300 sec: 19383.1). Total num frames: 437493760. Throughput: 0: 9829.1, 1: 9875.0. Samples: 437463196. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:33:37,364][19170] Avg episode reward: [(0, '503.800'), (1, '942.330')] [2023-12-27 11:33:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000854008_218652672.pth... [2023-12-27 11:33:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000854736_218841088.pth... [2023-12-27 11:33:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000853584_218546176.pth [2023-12-27 11:33:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000852856_218357760.pth [2023-12-27 11:33:37,416][20347] Updated weights for policy 0, policy_version 854740 (0.0007) [2023-12-27 11:33:37,463][20381] Updated weights for policy 1, policy_version 854012 (0.0010) [2023-12-27 11:33:37,477][20347] Updated weights for policy 0, policy_version 854750 (0.0007) [2023-12-27 11:33:37,508][20292] KL-divergence is very high: 103.7488 [2023-12-27 11:33:37,523][20381] Updated weights for policy 1, policy_version 854022 (0.0007) [2023-12-27 11:33:37,532][20347] Updated weights for policy 0, policy_version 854760 (0.0005) [2023-12-27 11:33:37,551][20292] KL-divergence is very high: 112.8476 [2023-12-27 11:33:37,580][20381] Updated weights for policy 1, policy_version 854032 (0.0005) [2023-12-27 11:33:38,120][20381] Updated weights for policy 1, policy_version 854042 (0.0005) [2023-12-27 11:33:38,184][20381] Updated weights for policy 1, policy_version 854052 (0.0008) [2023-12-27 11:33:38,209][20347] Updated weights for policy 0, policy_version 854770 (0.0007) [2023-12-27 11:33:38,242][20381] Updated weights for policy 1, policy_version 854062 (0.0009) [2023-12-27 11:33:38,265][20347] Updated weights for policy 0, policy_version 854780 (0.0005) [2023-12-27 11:33:38,297][20381] Updated weights for policy 1, policy_version 854072 (0.0007) [2023-12-27 11:33:38,325][20347] Updated weights for policy 0, policy_version 854790 (0.0006) [2023-12-27 11:33:38,385][20347] Updated weights for policy 0, policy_version 854800 (0.0008) [2023-12-27 11:33:38,964][20347] Updated weights for policy 0, policy_version 854810 (0.0008) [2023-12-27 11:33:39,020][20347] Updated weights for policy 0, policy_version 854820 (0.0006) [2023-12-27 11:33:39,022][20381] Updated weights for policy 1, policy_version 854082 (0.0006) [2023-12-27 11:33:39,079][20347] Updated weights for policy 0, policy_version 854830 (0.0005) [2023-12-27 11:33:39,082][20381] Updated weights for policy 1, policy_version 854092 (0.0006) [2023-12-27 11:33:39,132][20381] Updated weights for policy 1, policy_version 854102 (0.0007) [2023-12-27 11:33:39,780][20347] Updated weights for policy 0, policy_version 854840 (0.0007) [2023-12-27 11:33:39,839][20347] Updated weights for policy 0, policy_version 854850 (0.0007) [2023-12-27 11:33:39,881][20381] Updated weights for policy 1, policy_version 854112 (0.0008) [2023-12-27 11:33:39,902][20347] Updated weights for policy 0, policy_version 854860 (0.0007) [2023-12-27 11:33:39,944][20381] Updated weights for policy 1, policy_version 854122 (0.0008) [2023-12-27 11:33:39,995][20381] Updated weights for policy 1, policy_version 854132 (0.0007) [2023-12-27 11:33:40,630][20347] Updated weights for policy 0, policy_version 854870 (0.0008) [2023-12-27 11:33:40,638][20381] Updated weights for policy 1, policy_version 854142 (0.0007) [2023-12-27 11:33:40,684][20381] Updated weights for policy 1, policy_version 854152 (0.0010) [2023-12-27 11:33:40,690][20347] Updated weights for policy 0, policy_version 854880 (0.0007) [2023-12-27 11:33:40,731][20381] Updated weights for policy 1, policy_version 854162 (0.0011) [2023-12-27 11:33:40,748][20347] Updated weights for policy 0, policy_version 854890 (0.0006) [2023-12-27 11:33:41,450][20381] Updated weights for policy 1, policy_version 854172 (0.0011) [2023-12-27 11:33:41,483][20347] Updated weights for policy 0, policy_version 854900 (0.0008) [2023-12-27 11:33:41,501][20381] Updated weights for policy 1, policy_version 854182 (0.0011) [2023-12-27 11:33:41,536][20347] Updated weights for policy 0, policy_version 854910 (0.0010) [2023-12-27 11:33:41,554][20381] Updated weights for policy 1, policy_version 854192 (0.0011) [2023-12-27 11:33:41,589][20347] Updated weights for policy 0, policy_version 854920 (0.0011) [2023-12-27 11:33:42,345][20381] Updated weights for policy 1, policy_version 854202 (0.0010) [2023-12-27 11:33:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19660.8, 300 sec: 19355.3). Total num frames: 437592064. Throughput: 0: 9886.1, 1: 9948.1. Samples: 437588304. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:33:42,363][19170] Avg episode reward: [(0, '525.720'), (1, '961.180')] [2023-12-27 11:33:42,371][20347] Updated weights for policy 0, policy_version 854930 (0.0010) [2023-12-27 11:33:42,407][20381] Updated weights for policy 1, policy_version 854212 (0.0007) [2023-12-27 11:33:42,426][20347] Updated weights for policy 0, policy_version 854940 (0.0008) [2023-12-27 11:33:42,462][20381] Updated weights for policy 1, policy_version 854222 (0.0006) [2023-12-27 11:33:42,482][20347] Updated weights for policy 0, policy_version 854950 (0.0008) [2023-12-27 11:33:42,515][20381] Updated weights for policy 1, policy_version 854232 (0.0007) [2023-12-27 11:33:42,537][20347] Updated weights for policy 0, policy_version 854960 (0.0008) [2023-12-27 11:33:43,180][20347] Updated weights for policy 0, policy_version 854970 (0.0010) [2023-12-27 11:33:43,186][20381] Updated weights for policy 1, policy_version 854242 (0.0005) [2023-12-27 11:33:43,232][20381] Updated weights for policy 1, policy_version 854252 (0.0006) [2023-12-27 11:33:43,240][20347] Updated weights for policy 0, policy_version 854980 (0.0010) [2023-12-27 11:33:43,286][20381] Updated weights for policy 1, policy_version 854262 (0.0006) [2023-12-27 11:33:43,292][20347] Updated weights for policy 0, policy_version 854990 (0.0010) [2023-12-27 11:33:43,928][20381] Updated weights for policy 1, policy_version 854272 (0.0009) [2023-12-27 11:33:43,964][20347] Updated weights for policy 0, policy_version 855000 (0.0009) [2023-12-27 11:33:43,975][20381] Updated weights for policy 1, policy_version 854282 (0.0010) [2023-12-27 11:33:44,025][20347] Updated weights for policy 0, policy_version 855010 (0.0010) [2023-12-27 11:33:44,030][20381] Updated weights for policy 1, policy_version 854292 (0.0010) [2023-12-27 11:33:44,086][20347] Updated weights for policy 0, policy_version 855020 (0.0010) [2023-12-27 11:33:44,664][20381] Updated weights for policy 1, policy_version 854302 (0.0008) [2023-12-27 11:33:44,698][20347] Updated weights for policy 0, policy_version 855030 (0.0008) [2023-12-27 11:33:44,720][20381] Updated weights for policy 1, policy_version 854312 (0.0007) [2023-12-27 11:33:44,754][20347] Updated weights for policy 0, policy_version 855040 (0.0008) [2023-12-27 11:33:44,774][20381] Updated weights for policy 1, policy_version 854322 (0.0007) [2023-12-27 11:33:44,814][20347] Updated weights for policy 0, policy_version 855050 (0.0007) [2023-12-27 11:33:45,453][20347] Updated weights for policy 0, policy_version 855060 (0.0006) [2023-12-27 11:33:45,488][20381] Updated weights for policy 1, policy_version 854332 (0.0007) [2023-12-27 11:33:45,509][20347] Updated weights for policy 0, policy_version 855070 (0.0007) [2023-12-27 11:33:45,544][20381] Updated weights for policy 1, policy_version 854342 (0.0006) [2023-12-27 11:33:45,566][20347] Updated weights for policy 0, policy_version 855080 (0.0005) [2023-12-27 11:33:45,593][20381] Updated weights for policy 1, policy_version 854352 (0.0005) [2023-12-27 11:33:46,122][20347] Updated weights for policy 0, policy_version 855090 (0.0006) [2023-12-27 11:33:46,170][20381] Updated weights for policy 1, policy_version 854362 (0.0007) [2023-12-27 11:33:46,184][20347] Updated weights for policy 0, policy_version 855100 (0.0005) [2023-12-27 11:33:46,224][20381] Updated weights for policy 1, policy_version 854372 (0.0007) [2023-12-27 11:33:46,233][20347] Updated weights for policy 0, policy_version 855110 (0.0005) [2023-12-27 11:33:46,279][20381] Updated weights for policy 1, policy_version 854382 (0.0007) [2023-12-27 11:33:46,282][20347] Updated weights for policy 0, policy_version 855120 (0.0005) [2023-12-27 11:33:46,339][20381] Updated weights for policy 1, policy_version 854392 (0.0007) [2023-12-27 11:33:46,882][20347] Updated weights for policy 0, policy_version 855130 (0.0007) [2023-12-27 11:33:46,906][20381] Updated weights for policy 1, policy_version 854402 (0.0005) [2023-12-27 11:33:46,939][20347] Updated weights for policy 0, policy_version 855140 (0.0007) [2023-12-27 11:33:46,964][20381] Updated weights for policy 1, policy_version 854412 (0.0008) [2023-12-27 11:33:46,985][20347] Updated weights for policy 0, policy_version 855150 (0.0007) [2023-12-27 11:33:47,020][20381] Updated weights for policy 1, policy_version 854422 (0.0007) [2023-12-27 11:33:47,362][19170] Fps is (10 sec: 21300.0, 60 sec: 19933.9, 300 sec: 19410.9). Total num frames: 437706752. Throughput: 0: 9961.6, 1: 10062.9. Samples: 437714704. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:33:47,363][19170] Avg episode reward: [(0, '509.750'), (1, '941.750')] [2023-12-27 11:33:47,645][20347] Updated weights for policy 0, policy_version 855160 (0.0007) [2023-12-27 11:33:47,701][20347] Updated weights for policy 0, policy_version 855170 (0.0008) [2023-12-27 11:33:47,709][20381] Updated weights for policy 1, policy_version 854432 (0.0006) [2023-12-27 11:33:47,754][20347] Updated weights for policy 0, policy_version 855180 (0.0008) [2023-12-27 11:33:47,764][20381] Updated weights for policy 1, policy_version 854442 (0.0005) [2023-12-27 11:33:47,817][20381] Updated weights for policy 1, policy_version 854452 (0.0006) [2023-12-27 11:33:48,459][20347] Updated weights for policy 0, policy_version 855190 (0.0007) [2023-12-27 11:33:48,481][20381] Updated weights for policy 1, policy_version 854462 (0.0006) [2023-12-27 11:33:48,516][20347] Updated weights for policy 0, policy_version 855200 (0.0007) [2023-12-27 11:33:48,541][20381] Updated weights for policy 1, policy_version 854472 (0.0007) [2023-12-27 11:33:48,574][20347] Updated weights for policy 0, policy_version 855210 (0.0007) [2023-12-27 11:33:48,599][20381] Updated weights for policy 1, policy_version 854482 (0.0006) [2023-12-27 11:33:49,248][20347] Updated weights for policy 0, policy_version 855220 (0.0006) [2023-12-27 11:33:49,266][20381] Updated weights for policy 1, policy_version 854492 (0.0007) [2023-12-27 11:33:49,310][20347] Updated weights for policy 0, policy_version 855230 (0.0007) [2023-12-27 11:33:49,318][20381] Updated weights for policy 1, policy_version 854502 (0.0006) [2023-12-27 11:33:49,373][20347] Updated weights for policy 0, policy_version 855240 (0.0008) [2023-12-27 11:33:49,382][20381] Updated weights for policy 1, policy_version 854512 (0.0007) [2023-12-27 11:33:50,083][20347] Updated weights for policy 0, policy_version 855250 (0.0007) [2023-12-27 11:33:50,083][20381] Updated weights for policy 1, policy_version 854522 (0.0006) [2023-12-27 11:33:50,145][20381] Updated weights for policy 1, policy_version 854532 (0.0006) [2023-12-27 11:33:50,146][20347] Updated weights for policy 0, policy_version 855260 (0.0007) [2023-12-27 11:33:50,202][20381] Updated weights for policy 1, policy_version 854542 (0.0006) [2023-12-27 11:33:50,203][20347] Updated weights for policy 0, policy_version 855270 (0.0007) [2023-12-27 11:33:50,261][20347] Updated weights for policy 0, policy_version 855280 (0.0007) [2023-12-27 11:33:50,262][20381] Updated weights for policy 1, policy_version 854552 (0.0006) [2023-12-27 11:33:50,862][20347] Updated weights for policy 0, policy_version 855290 (0.0007) [2023-12-27 11:33:50,921][20347] Updated weights for policy 0, policy_version 855300 (0.0006) [2023-12-27 11:33:50,937][20292] KL-divergence is very high: 126.1732 [2023-12-27 11:33:50,941][20381] Updated weights for policy 1, policy_version 854562 (0.0008) [2023-12-27 11:33:50,978][20347] Updated weights for policy 0, policy_version 855310 (0.0006) [2023-12-27 11:33:50,986][20292] KL-divergence is very high: 103.2132 [2023-12-27 11:33:51,005][20381] Updated weights for policy 1, policy_version 854572 (0.0007) [2023-12-27 11:33:51,071][20381] Updated weights for policy 1, policy_version 854582 (0.0007) [2023-12-27 11:33:51,633][20347] Updated weights for policy 0, policy_version 855320 (0.0007) [2023-12-27 11:33:51,690][20347] Updated weights for policy 0, policy_version 855330 (0.0007) [2023-12-27 11:33:51,754][20347] Updated weights for policy 0, policy_version 855340 (0.0008) [2023-12-27 11:33:51,838][20381] Updated weights for policy 1, policy_version 854592 (0.0009) [2023-12-27 11:33:51,891][20381] Updated weights for policy 1, policy_version 854602 (0.0010) [2023-12-27 11:33:51,943][20381] Updated weights for policy 1, policy_version 854612 (0.0011) [2023-12-27 11:33:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 19933.8, 300 sec: 19410.9). Total num frames: 437805056. Throughput: 0: 9991.1, 1: 10096.1. Samples: 437776300. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:33:52,363][19170] Avg episode reward: [(0, '482.240'), (1, '699.950')] [2023-12-27 11:33:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000854616_218808320.pth... [2023-12-27 11:33:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000855344_218996736.pth... [2023-12-27 11:33:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000853400_218497024.pth [2023-12-27 11:33:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000854160_218693632.pth [2023-12-27 11:33:52,482][20347] Updated weights for policy 0, policy_version 855350 (0.0007) [2023-12-27 11:33:52,539][20347] Updated weights for policy 0, policy_version 855360 (0.0007) [2023-12-27 11:33:52,598][20347] Updated weights for policy 0, policy_version 855370 (0.0007) [2023-12-27 11:33:52,627][20381] Updated weights for policy 1, policy_version 854622 (0.0008) [2023-12-27 11:33:52,688][20381] Updated weights for policy 1, policy_version 854632 (0.0007) [2023-12-27 11:33:52,752][20381] Updated weights for policy 1, policy_version 854642 (0.0008) [2023-12-27 11:33:53,239][20347] Updated weights for policy 0, policy_version 855380 (0.0008) [2023-12-27 11:33:53,297][20347] Updated weights for policy 0, policy_version 855390 (0.0011) [2023-12-27 11:33:53,360][20347] Updated weights for policy 0, policy_version 855400 (0.0010) [2023-12-27 11:33:53,417][20381] Updated weights for policy 1, policy_version 854652 (0.0007) [2023-12-27 11:33:53,476][20381] Updated weights for policy 1, policy_version 854662 (0.0005) [2023-12-27 11:33:53,533][20381] Updated weights for policy 1, policy_version 854672 (0.0005) [2023-12-27 11:33:53,970][20347] Updated weights for policy 0, policy_version 855410 (0.0010) [2023-12-27 11:33:54,024][20347] Updated weights for policy 0, policy_version 855420 (0.0010) [2023-12-27 11:33:54,076][20347] Updated weights for policy 0, policy_version 855430 (0.0010) [2023-12-27 11:33:54,103][20381] Updated weights for policy 1, policy_version 854682 (0.0007) [2023-12-27 11:33:54,132][20347] Updated weights for policy 0, policy_version 855440 (0.0010) [2023-12-27 11:33:54,159][20381] Updated weights for policy 1, policy_version 854692 (0.0006) [2023-12-27 11:33:54,214][20381] Updated weights for policy 1, policy_version 854702 (0.0008) [2023-12-27 11:33:54,265][20381] Updated weights for policy 1, policy_version 854712 (0.0010) [2023-12-27 11:33:54,783][20347] Updated weights for policy 0, policy_version 855450 (0.0010) [2023-12-27 11:33:54,835][20347] Updated weights for policy 0, policy_version 855460 (0.0009) [2023-12-27 11:33:54,890][20347] Updated weights for policy 0, policy_version 855470 (0.0005) [2023-12-27 11:33:54,943][20381] Updated weights for policy 1, policy_version 854722 (0.0008) [2023-12-27 11:33:55,006][20381] Updated weights for policy 1, policy_version 854732 (0.0009) [2023-12-27 11:33:55,070][20381] Updated weights for policy 1, policy_version 854742 (0.0008) [2023-12-27 11:33:55,523][20347] Updated weights for policy 0, policy_version 855480 (0.0009) [2023-12-27 11:33:55,587][20347] Updated weights for policy 0, policy_version 855490 (0.0010) [2023-12-27 11:33:55,645][20347] Updated weights for policy 0, policy_version 855500 (0.0010) [2023-12-27 11:33:55,696][20381] Updated weights for policy 1, policy_version 854752 (0.0007) [2023-12-27 11:33:55,754][20381] Updated weights for policy 1, policy_version 854762 (0.0008) [2023-12-27 11:33:55,808][20381] Updated weights for policy 1, policy_version 854772 (0.0007) [2023-12-27 11:33:56,286][20347] Updated weights for policy 0, policy_version 855510 (0.0009) [2023-12-27 11:33:56,335][20347] Updated weights for policy 0, policy_version 855520 (0.0009) [2023-12-27 11:33:56,395][20347] Updated weights for policy 0, policy_version 855530 (0.0010) [2023-12-27 11:33:56,457][20381] Updated weights for policy 1, policy_version 854782 (0.0007) [2023-12-27 11:33:56,519][20381] Updated weights for policy 1, policy_version 854792 (0.0007) [2023-12-27 11:33:56,576][20381] Updated weights for policy 1, policy_version 854802 (0.0007) [2023-12-27 11:33:57,018][20347] Updated weights for policy 0, policy_version 855540 (0.0007) [2023-12-27 11:33:57,078][20347] Updated weights for policy 0, policy_version 855550 (0.0006) [2023-12-27 11:33:57,134][20347] Updated weights for policy 0, policy_version 855560 (0.0005) [2023-12-27 11:33:57,248][20381] Updated weights for policy 1, policy_version 854812 (0.0007) [2023-12-27 11:33:57,301][20381] Updated weights for policy 1, policy_version 854822 (0.0006) [2023-12-27 11:33:57,356][20381] Updated weights for policy 1, policy_version 854832 (0.0008) [2023-12-27 11:33:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20070.4, 300 sec: 19438.6). Total num frames: 437911552. Throughput: 0: 10107.5, 1: 10171.4. Samples: 437900840. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:33:57,363][19170] Avg episode reward: [(0, '522.600'), (1, '709.180')] [2023-12-27 11:33:57,675][20347] Updated weights for policy 0, policy_version 855570 (0.0005) [2023-12-27 11:33:57,734][20347] Updated weights for policy 0, policy_version 855580 (0.0006) [2023-12-27 11:33:57,789][20347] Updated weights for policy 0, policy_version 855590 (0.0005) [2023-12-27 11:33:57,845][20347] Updated weights for policy 0, policy_version 855600 (0.0005) [2023-12-27 11:33:58,080][20381] Updated weights for policy 1, policy_version 854842 (0.0009) [2023-12-27 11:33:58,141][20381] Updated weights for policy 1, policy_version 854852 (0.0007) [2023-12-27 11:33:58,201][20381] Updated weights for policy 1, policy_version 854862 (0.0007) [2023-12-27 11:33:58,265][20381] Updated weights for policy 1, policy_version 854872 (0.0008) [2023-12-27 11:33:58,504][20347] Updated weights for policy 0, policy_version 855610 (0.0010) [2023-12-27 11:33:58,561][20347] Updated weights for policy 0, policy_version 855620 (0.0010) [2023-12-27 11:33:58,618][20347] Updated weights for policy 0, policy_version 855630 (0.0010) [2023-12-27 11:33:59,058][20381] Updated weights for policy 1, policy_version 854882 (0.0008) [2023-12-27 11:33:59,105][20381] Updated weights for policy 1, policy_version 854892 (0.0007) [2023-12-27 11:33:59,156][20381] Updated weights for policy 1, policy_version 854902 (0.0008) [2023-12-27 11:33:59,394][20347] Updated weights for policy 0, policy_version 855640 (0.0008) [2023-12-27 11:33:59,445][20347] Updated weights for policy 0, policy_version 855650 (0.0009) [2023-12-27 11:33:59,493][20347] Updated weights for policy 0, policy_version 855660 (0.0007) [2023-12-27 11:33:59,952][20381] Updated weights for policy 1, policy_version 854912 (0.0007) [2023-12-27 11:34:00,005][20381] Updated weights for policy 1, policy_version 854922 (0.0009) [2023-12-27 11:34:00,057][20381] Updated weights for policy 1, policy_version 854932 (0.0008) [2023-12-27 11:34:00,223][20347] Updated weights for policy 0, policy_version 855670 (0.0006) [2023-12-27 11:34:00,276][20347] Updated weights for policy 0, policy_version 855680 (0.0010) [2023-12-27 11:34:00,338][20347] Updated weights for policy 0, policy_version 855690 (0.0010) [2023-12-27 11:34:00,783][20381] Updated weights for policy 1, policy_version 854942 (0.0007) [2023-12-27 11:34:00,835][20381] Updated weights for policy 1, policy_version 854952 (0.0005) [2023-12-27 11:34:00,894][20381] Updated weights for policy 1, policy_version 854962 (0.0007) [2023-12-27 11:34:00,988][20347] Updated weights for policy 0, policy_version 855700 (0.0010) [2023-12-27 11:34:01,045][20347] Updated weights for policy 0, policy_version 855710 (0.0010) [2023-12-27 11:34:01,103][20347] Updated weights for policy 0, policy_version 855720 (0.0008) [2023-12-27 11:34:01,594][20381] Updated weights for policy 1, policy_version 854972 (0.0007) [2023-12-27 11:34:01,660][20381] Updated weights for policy 1, policy_version 854982 (0.0008) [2023-12-27 11:34:01,718][20381] Updated weights for policy 1, policy_version 854992 (0.0009) [2023-12-27 11:34:01,825][20347] Updated weights for policy 0, policy_version 855730 (0.0009) [2023-12-27 11:34:01,887][20347] Updated weights for policy 0, policy_version 855740 (0.0007) [2023-12-27 11:34:01,949][20347] Updated weights for policy 0, policy_version 855750 (0.0008) [2023-12-27 11:34:02,011][20347] Updated weights for policy 0, policy_version 855760 (0.0007) [2023-12-27 11:34:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20070.4, 300 sec: 19438.6). Total num frames: 438009856. Throughput: 0: 10223.3, 1: 10148.2. Samples: 438021632. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:02,363][19170] Avg episode reward: [(0, '601.500'), (1, '795.310')] [2023-12-27 11:34:02,396][20381] Updated weights for policy 1, policy_version 855002 (0.0008) [2023-12-27 11:34:02,454][20381] Updated weights for policy 1, policy_version 855012 (0.0008) [2023-12-27 11:34:02,510][20381] Updated weights for policy 1, policy_version 855022 (0.0008) [2023-12-27 11:34:02,557][20381] Updated weights for policy 1, policy_version 855032 (0.0007) [2023-12-27 11:34:02,687][20347] Updated weights for policy 0, policy_version 855770 (0.0010) [2023-12-27 11:34:02,750][20347] Updated weights for policy 0, policy_version 855780 (0.0011) [2023-12-27 11:34:02,808][20347] Updated weights for policy 0, policy_version 855790 (0.0010) [2023-12-27 11:34:03,256][20381] Updated weights for policy 1, policy_version 855042 (0.0007) [2023-12-27 11:34:03,307][20381] Updated weights for policy 1, policy_version 855052 (0.0007) [2023-12-27 11:34:03,355][20381] Updated weights for policy 1, policy_version 855062 (0.0007) [2023-12-27 11:34:03,413][20347] Updated weights for policy 0, policy_version 855800 (0.0010) [2023-12-27 11:34:03,470][20347] Updated weights for policy 0, policy_version 855810 (0.0010) [2023-12-27 11:34:03,524][20347] Updated weights for policy 0, policy_version 855820 (0.0010) [2023-12-27 11:34:04,060][20381] Updated weights for policy 1, policy_version 855072 (0.0007) [2023-12-27 11:34:04,112][20381] Updated weights for policy 1, policy_version 855082 (0.0007) [2023-12-27 11:34:04,173][20381] Updated weights for policy 1, policy_version 855092 (0.0008) [2023-12-27 11:34:04,221][20347] Updated weights for policy 0, policy_version 855830 (0.0010) [2023-12-27 11:34:04,281][20347] Updated weights for policy 0, policy_version 855840 (0.0010) [2023-12-27 11:34:04,331][20347] Updated weights for policy 0, policy_version 855850 (0.0010) [2023-12-27 11:34:04,881][20381] Updated weights for policy 1, policy_version 855102 (0.0007) [2023-12-27 11:34:04,938][20381] Updated weights for policy 1, policy_version 855112 (0.0008) [2023-12-27 11:34:04,984][20381] Updated weights for policy 1, policy_version 855122 (0.0007) [2023-12-27 11:34:05,016][20347] Updated weights for policy 0, policy_version 855860 (0.0010) [2023-12-27 11:34:05,060][20347] Updated weights for policy 0, policy_version 855870 (0.0007) [2023-12-27 11:34:05,107][20347] Updated weights for policy 0, policy_version 855880 (0.0007) [2023-12-27 11:34:05,694][20381] Updated weights for policy 1, policy_version 855132 (0.0008) [2023-12-27 11:34:05,749][20381] Updated weights for policy 1, policy_version 855142 (0.0007) [2023-12-27 11:34:05,799][20381] Updated weights for policy 1, policy_version 855152 (0.0007) [2023-12-27 11:34:05,815][20347] Updated weights for policy 0, policy_version 855890 (0.0010) [2023-12-27 11:34:05,864][20347] Updated weights for policy 0, policy_version 855900 (0.0010) [2023-12-27 11:34:05,912][20347] Updated weights for policy 0, policy_version 855910 (0.0010) [2023-12-27 11:34:05,967][20347] Updated weights for policy 0, policy_version 855920 (0.0010) [2023-12-27 11:34:06,524][20381] Updated weights for policy 1, policy_version 855162 (0.0006) [2023-12-27 11:34:06,587][20381] Updated weights for policy 1, policy_version 855172 (0.0008) [2023-12-27 11:34:06,646][20381] Updated weights for policy 1, policy_version 855182 (0.0008) [2023-12-27 11:34:06,687][20347] Updated weights for policy 0, policy_version 855930 (0.0011) [2023-12-27 11:34:06,704][20381] Updated weights for policy 1, policy_version 855192 (0.0007) [2023-12-27 11:34:06,750][20347] Updated weights for policy 0, policy_version 855940 (0.0010) [2023-12-27 11:34:06,803][20347] Updated weights for policy 0, policy_version 855950 (0.0010) [2023-12-27 11:34:07,359][20381] Updated weights for policy 1, policy_version 855202 (0.0007) [2023-12-27 11:34:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 19410.9). Total num frames: 438108160. Throughput: 0: 10281.8, 1: 10179.7. Samples: 438082132. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:07,363][19170] Avg episode reward: [(0, '660.300'), (1, '853.900')] [2023-12-27 11:34:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000855952_219152384.pth... [2023-12-27 11:34:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000854736_218841088.pth [2023-12-27 11:34:07,413][20381] Updated weights for policy 1, policy_version 855212 (0.0008) [2023-12-27 11:34:07,458][20381] Updated weights for policy 1, policy_version 855222 (0.0008) [2023-12-27 11:34:07,469][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000855224_218963968.pth... [2023-12-27 11:34:07,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000854008_218652672.pth [2023-12-27 11:34:07,484][20347] Updated weights for policy 0, policy_version 855960 (0.0006) [2023-12-27 11:34:07,549][20347] Updated weights for policy 0, policy_version 855970 (0.0006) [2023-12-27 11:34:07,604][20347] Updated weights for policy 0, policy_version 855980 (0.0005) [2023-12-27 11:34:08,137][20347] Updated weights for policy 0, policy_version 855990 (0.0005) [2023-12-27 11:34:08,194][20381] Updated weights for policy 1, policy_version 855232 (0.0006) [2023-12-27 11:34:08,196][20347] Updated weights for policy 0, policy_version 856000 (0.0009) [2023-12-27 11:34:08,245][20381] Updated weights for policy 1, policy_version 855242 (0.0006) [2023-12-27 11:34:08,251][20347] Updated weights for policy 0, policy_version 856010 (0.0010) [2023-12-27 11:34:08,301][20381] Updated weights for policy 1, policy_version 855252 (0.0006) [2023-12-27 11:34:08,921][20347] Updated weights for policy 0, policy_version 856020 (0.0008) [2023-12-27 11:34:08,983][20347] Updated weights for policy 0, policy_version 856030 (0.0010) [2023-12-27 11:34:09,015][20381] Updated weights for policy 1, policy_version 855262 (0.0007) [2023-12-27 11:34:09,043][20347] Updated weights for policy 0, policy_version 856040 (0.0010) [2023-12-27 11:34:09,071][20381] Updated weights for policy 1, policy_version 855272 (0.0005) [2023-12-27 11:34:09,123][20381] Updated weights for policy 1, policy_version 855282 (0.0006) [2023-12-27 11:34:09,771][20347] Updated weights for policy 0, policy_version 856050 (0.0009) [2023-12-27 11:34:09,837][20347] Updated weights for policy 0, policy_version 856060 (0.0008) [2023-12-27 11:34:09,845][20381] Updated weights for policy 1, policy_version 855292 (0.0007) [2023-12-27 11:34:09,898][20347] Updated weights for policy 0, policy_version 856070 (0.0007) [2023-12-27 11:34:09,899][20381] Updated weights for policy 1, policy_version 855302 (0.0006) [2023-12-27 11:34:09,958][20381] Updated weights for policy 1, policy_version 855312 (0.0009) [2023-12-27 11:34:09,960][20347] Updated weights for policy 0, policy_version 856080 (0.0008) [2023-12-27 11:34:10,676][20381] Updated weights for policy 1, policy_version 855322 (0.0008) [2023-12-27 11:34:10,707][20347] Updated weights for policy 0, policy_version 856090 (0.0006) [2023-12-27 11:34:10,743][20381] Updated weights for policy 1, policy_version 855332 (0.0008) [2023-12-27 11:34:10,757][20347] Updated weights for policy 0, policy_version 856100 (0.0007) [2023-12-27 11:34:10,796][20381] Updated weights for policy 1, policy_version 855342 (0.0006) [2023-12-27 11:34:10,809][20347] Updated weights for policy 0, policy_version 856110 (0.0005) [2023-12-27 11:34:10,852][20381] Updated weights for policy 1, policy_version 855352 (0.0007) [2023-12-27 11:34:11,519][20347] Updated weights for policy 0, policy_version 856120 (0.0005) [2023-12-27 11:34:11,523][20381] Updated weights for policy 1, policy_version 855362 (0.0006) [2023-12-27 11:34:11,582][20347] Updated weights for policy 0, policy_version 856130 (0.0007) [2023-12-27 11:34:11,583][20381] Updated weights for policy 1, policy_version 855372 (0.0006) [2023-12-27 11:34:11,646][20381] Updated weights for policy 1, policy_version 855382 (0.0008) [2023-12-27 11:34:11,647][20347] Updated weights for policy 0, policy_version 856140 (0.0007) [2023-12-27 11:34:12,315][20381] Updated weights for policy 1, policy_version 855392 (0.0010) [2023-12-27 11:34:12,337][20347] Updated weights for policy 0, policy_version 856150 (0.0009) [2023-12-27 11:34:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 19383.1). Total num frames: 438206464. Throughput: 0: 10330.6, 1: 10178.9. Samples: 438202400. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:12,363][19170] Avg episode reward: [(0, '568.780'), (1, '785.850')] [2023-12-27 11:34:12,382][20381] Updated weights for policy 1, policy_version 855402 (0.0010) [2023-12-27 11:34:12,399][20347] Updated weights for policy 0, policy_version 856160 (0.0009) [2023-12-27 11:34:12,439][20381] Updated weights for policy 1, policy_version 855412 (0.0007) [2023-12-27 11:34:12,457][20347] Updated weights for policy 0, policy_version 856170 (0.0007) [2023-12-27 11:34:13,094][20381] Updated weights for policy 1, policy_version 855422 (0.0010) [2023-12-27 11:34:13,117][20347] Updated weights for policy 0, policy_version 856180 (0.0006) [2023-12-27 11:34:13,152][20381] Updated weights for policy 1, policy_version 855432 (0.0008) [2023-12-27 11:34:13,174][20347] Updated weights for policy 0, policy_version 856190 (0.0006) [2023-12-27 11:34:13,204][20381] Updated weights for policy 1, policy_version 855442 (0.0010) [2023-12-27 11:34:13,221][20347] Updated weights for policy 0, policy_version 856200 (0.0009) [2023-12-27 11:34:13,850][20347] Updated weights for policy 0, policy_version 856210 (0.0010) [2023-12-27 11:34:13,904][20381] Updated weights for policy 1, policy_version 855452 (0.0008) [2023-12-27 11:34:13,905][20347] Updated weights for policy 0, policy_version 856220 (0.0007) [2023-12-27 11:34:13,959][20347] Updated weights for policy 0, policy_version 856230 (0.0007) [2023-12-27 11:34:13,964][20381] Updated weights for policy 1, policy_version 855462 (0.0005) [2023-12-27 11:34:14,013][20347] Updated weights for policy 0, policy_version 856240 (0.0007) [2023-12-27 11:34:14,029][20381] Updated weights for policy 1, policy_version 855472 (0.0005) [2023-12-27 11:34:14,601][20381] Updated weights for policy 1, policy_version 855482 (0.0006) [2023-12-27 11:34:14,650][20381] Updated weights for policy 1, policy_version 855492 (0.0011) [2023-12-27 11:34:14,704][20347] Updated weights for policy 0, policy_version 856250 (0.0005) [2023-12-27 11:34:14,709][20381] Updated weights for policy 1, policy_version 855502 (0.0010) [2023-12-27 11:34:14,758][20347] Updated weights for policy 0, policy_version 856260 (0.0006) [2023-12-27 11:34:14,772][20381] Updated weights for policy 1, policy_version 855512 (0.0011) [2023-12-27 11:34:14,821][20347] Updated weights for policy 0, policy_version 856270 (0.0009) [2023-12-27 11:34:15,420][20381] Updated weights for policy 1, policy_version 855522 (0.0007) [2023-12-27 11:34:15,472][20381] Updated weights for policy 1, policy_version 855532 (0.0008) [2023-12-27 11:34:15,527][20381] Updated weights for policy 1, policy_version 855542 (0.0010) [2023-12-27 11:34:15,537][20347] Updated weights for policy 0, policy_version 856280 (0.0008) [2023-12-27 11:34:15,588][20347] Updated weights for policy 0, policy_version 856290 (0.0007) [2023-12-27 11:34:15,636][20347] Updated weights for policy 0, policy_version 856300 (0.0008) [2023-12-27 11:34:16,214][20381] Updated weights for policy 1, policy_version 855552 (0.0010) [2023-12-27 11:34:16,267][20381] Updated weights for policy 1, policy_version 855562 (0.0010) [2023-12-27 11:34:16,329][20381] Updated weights for policy 1, policy_version 855572 (0.0005) [2023-12-27 11:34:16,347][20347] Updated weights for policy 0, policy_version 856310 (0.0007) [2023-12-27 11:34:16,403][20347] Updated weights for policy 0, policy_version 856320 (0.0008) [2023-12-27 11:34:16,458][20347] Updated weights for policy 0, policy_version 856330 (0.0007) [2023-12-27 11:34:16,936][20381] Updated weights for policy 1, policy_version 855582 (0.0005) [2023-12-27 11:34:16,986][20381] Updated weights for policy 1, policy_version 855592 (0.0008) [2023-12-27 11:34:17,044][20381] Updated weights for policy 1, policy_version 855602 (0.0007) [2023-12-27 11:34:17,051][20347] Updated weights for policy 0, policy_version 856340 (0.0005) [2023-12-27 11:34:17,115][20347] Updated weights for policy 0, policy_version 856350 (0.0006) [2023-12-27 11:34:17,167][20347] Updated weights for policy 0, policy_version 856360 (0.0005) [2023-12-27 11:34:17,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20343.4, 300 sec: 19466.5). Total num frames: 438321152. Throughput: 0: 10351.8, 1: 10212.1. Samples: 438327128. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:17,364][19170] Avg episode reward: [(0, '487.220'), (1, '776.360')] [2023-12-27 11:34:17,623][20381] Updated weights for policy 1, policy_version 855612 (0.0006) [2023-12-27 11:34:17,679][20381] Updated weights for policy 1, policy_version 855622 (0.0008) [2023-12-27 11:34:17,734][20381] Updated weights for policy 1, policy_version 855632 (0.0007) [2023-12-27 11:34:17,794][20347] Updated weights for policy 0, policy_version 856370 (0.0006) [2023-12-27 11:34:17,855][20347] Updated weights for policy 0, policy_version 856380 (0.0008) [2023-12-27 11:34:17,916][20347] Updated weights for policy 0, policy_version 856390 (0.0008) [2023-12-27 11:34:17,976][20347] Updated weights for policy 0, policy_version 856400 (0.0007) [2023-12-27 11:34:18,374][20381] Updated weights for policy 1, policy_version 855642 (0.0008) [2023-12-27 11:34:18,434][20381] Updated weights for policy 1, policy_version 855652 (0.0008) [2023-12-27 11:34:18,484][20381] Updated weights for policy 1, policy_version 855662 (0.0007) [2023-12-27 11:34:18,540][20381] Updated weights for policy 1, policy_version 855672 (0.0008) [2023-12-27 11:34:18,665][20347] Updated weights for policy 0, policy_version 856410 (0.0008) [2023-12-27 11:34:18,720][20347] Updated weights for policy 0, policy_version 856420 (0.0007) [2023-12-27 11:34:18,779][20347] Updated weights for policy 0, policy_version 856430 (0.0007) [2023-12-27 11:34:19,247][20381] Updated weights for policy 1, policy_version 855682 (0.0008) [2023-12-27 11:34:19,299][20381] Updated weights for policy 1, policy_version 855692 (0.0007) [2023-12-27 11:34:19,319][20292] KL-divergence is very high: 163.9912 [2023-12-27 11:34:19,363][20381] Updated weights for policy 1, policy_version 855702 (0.0007) [2023-12-27 11:34:19,544][20347] Updated weights for policy 0, policy_version 856440 (0.0008) [2023-12-27 11:34:19,606][20347] Updated weights for policy 0, policy_version 856450 (0.0008) [2023-12-27 11:34:19,666][20347] Updated weights for policy 0, policy_version 856460 (0.0008) [2023-12-27 11:34:20,179][20381] Updated weights for policy 1, policy_version 855712 (0.0008) [2023-12-27 11:34:20,237][20381] Updated weights for policy 1, policy_version 855722 (0.0008) [2023-12-27 11:34:20,288][20381] Updated weights for policy 1, policy_version 855732 (0.0008) [2023-12-27 11:34:20,355][20347] Updated weights for policy 0, policy_version 856470 (0.0006) [2023-12-27 11:34:20,407][20347] Updated weights for policy 0, policy_version 856480 (0.0007) [2023-12-27 11:34:20,462][20347] Updated weights for policy 0, policy_version 856490 (0.0005) [2023-12-27 11:34:21,047][20381] Updated weights for policy 1, policy_version 855742 (0.0008) [2023-12-27 11:34:21,103][20381] Updated weights for policy 1, policy_version 855752 (0.0007) [2023-12-27 11:34:21,135][20347] Updated weights for policy 0, policy_version 856500 (0.0005) [2023-12-27 11:34:21,169][20381] Updated weights for policy 1, policy_version 855762 (0.0009) [2023-12-27 11:34:21,202][20347] Updated weights for policy 0, policy_version 856510 (0.0008) [2023-12-27 11:34:21,261][20347] Updated weights for policy 0, policy_version 856520 (0.0007) [2023-12-27 11:34:21,900][20381] Updated weights for policy 1, policy_version 855772 (0.0007) [2023-12-27 11:34:21,953][20381] Updated weights for policy 1, policy_version 855782 (0.0008) [2023-12-27 11:34:21,976][20347] Updated weights for policy 0, policy_version 856530 (0.0008) [2023-12-27 11:34:22,015][20381] Updated weights for policy 1, policy_version 855792 (0.0008) [2023-12-27 11:34:22,029][20347] Updated weights for policy 0, policy_version 856540 (0.0006) [2023-12-27 11:34:22,083][20347] Updated weights for policy 0, policy_version 856550 (0.0007) [2023-12-27 11:34:22,135][20347] Updated weights for policy 0, policy_version 856560 (0.0007) [2023-12-27 11:34:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 19466.4). Total num frames: 438419456. Throughput: 0: 10331.6, 1: 10209.6. Samples: 438387544. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:22,363][19170] Avg episode reward: [(0, '715.540'), (1, '777.430')] [2023-12-27 11:34:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000856560_219308032.pth... [2023-12-27 11:34:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000855800_219111424.pth... [2023-12-27 11:34:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000855344_218996736.pth [2023-12-27 11:34:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000854616_218808320.pth [2023-12-27 11:34:22,653][20381] Updated weights for policy 1, policy_version 855802 (0.0006) [2023-12-27 11:34:22,710][20381] Updated weights for policy 1, policy_version 855812 (0.0008) [2023-12-27 11:34:22,765][20381] Updated weights for policy 1, policy_version 855822 (0.0009) [2023-12-27 11:34:22,817][20381] Updated weights for policy 1, policy_version 855832 (0.0007) [2023-12-27 11:34:22,854][20347] Updated weights for policy 0, policy_version 856570 (0.0006) [2023-12-27 11:34:22,910][20347] Updated weights for policy 0, policy_version 856580 (0.0007) [2023-12-27 11:34:22,970][20347] Updated weights for policy 0, policy_version 856590 (0.0006) [2023-12-27 11:34:23,546][20381] Updated weights for policy 1, policy_version 855842 (0.0005) [2023-12-27 11:34:23,601][20381] Updated weights for policy 1, policy_version 855852 (0.0005) [2023-12-27 11:34:23,632][20347] Updated weights for policy 0, policy_version 856600 (0.0006) [2023-12-27 11:34:23,656][20381] Updated weights for policy 1, policy_version 855862 (0.0007) [2023-12-27 11:34:23,685][20347] Updated weights for policy 0, policy_version 856610 (0.0007) [2023-12-27 11:34:23,738][20347] Updated weights for policy 0, policy_version 856620 (0.0008) [2023-12-27 11:34:24,318][20381] Updated weights for policy 1, policy_version 855872 (0.0007) [2023-12-27 11:34:24,368][20381] Updated weights for policy 1, policy_version 855882 (0.0008) [2023-12-27 11:34:24,405][20347] Updated weights for policy 0, policy_version 856630 (0.0007) [2023-12-27 11:34:24,415][20381] Updated weights for policy 1, policy_version 855892 (0.0006) [2023-12-27 11:34:24,455][20347] Updated weights for policy 0, policy_version 856640 (0.0006) [2023-12-27 11:34:24,516][20347] Updated weights for policy 0, policy_version 856651 (0.0007) [2023-12-27 11:34:25,122][20347] Updated weights for policy 0, policy_version 856661 (0.0005) [2023-12-27 11:34:25,161][20381] Updated weights for policy 1, policy_version 855902 (0.0007) [2023-12-27 11:34:25,180][20347] Updated weights for policy 0, policy_version 856671 (0.0007) [2023-12-27 11:34:25,226][20381] Updated weights for policy 1, policy_version 855912 (0.0006) [2023-12-27 11:34:25,237][20347] Updated weights for policy 0, policy_version 856681 (0.0005) [2023-12-27 11:34:25,290][20381] Updated weights for policy 1, policy_version 855922 (0.0007) [2023-12-27 11:34:25,870][20347] Updated weights for policy 0, policy_version 856691 (0.0008) [2023-12-27 11:34:25,930][20347] Updated weights for policy 0, policy_version 856701 (0.0010) [2023-12-27 11:34:25,952][20381] Updated weights for policy 1, policy_version 855932 (0.0008) [2023-12-27 11:34:25,985][20347] Updated weights for policy 0, policy_version 856711 (0.0010) [2023-12-27 11:34:26,005][20381] Updated weights for policy 1, policy_version 855942 (0.0005) [2023-12-27 11:34:26,062][20381] Updated weights for policy 1, policy_version 855952 (0.0006) [2023-12-27 11:34:26,107][20292] KL-divergence is very high: 230.8541 [2023-12-27 11:34:26,602][20347] Updated weights for policy 0, policy_version 856721 (0.0010) [2023-12-27 11:34:26,662][20347] Updated weights for policy 0, policy_version 856731 (0.0005) [2023-12-27 11:34:26,712][20347] Updated weights for policy 0, policy_version 856741 (0.0009) [2023-12-27 11:34:26,763][20347] Updated weights for policy 0, policy_version 856751 (0.0010) [2023-12-27 11:34:26,767][20381] Updated weights for policy 1, policy_version 855962 (0.0008) [2023-12-27 11:34:26,822][20381] Updated weights for policy 1, policy_version 855972 (0.0007) [2023-12-27 11:34:26,866][20381] Updated weights for policy 1, policy_version 855982 (0.0005) [2023-12-27 11:34:26,920][20381] Updated weights for policy 1, policy_version 855992 (0.0005) [2023-12-27 11:34:27,357][20347] Updated weights for policy 0, policy_version 856761 (0.0010) [2023-12-27 11:34:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 19466.4). Total num frames: 438517760. Throughput: 0: 10330.5, 1: 10125.4. Samples: 438508820. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:27,363][19170] Avg episode reward: [(0, '806.980'), (1, '659.660')] [2023-12-27 11:34:27,402][20347] Updated weights for policy 0, policy_version 856771 (0.0010) [2023-12-27 11:34:27,463][20347] Updated weights for policy 0, policy_version 856781 (0.0011) [2023-12-27 11:34:27,485][20381] Updated weights for policy 1, policy_version 856002 (0.0009) [2023-12-27 11:34:27,539][20381] Updated weights for policy 1, policy_version 856012 (0.0007) [2023-12-27 11:34:27,596][20381] Updated weights for policy 1, policy_version 856022 (0.0007) [2023-12-27 11:34:28,138][20347] Updated weights for policy 0, policy_version 856791 (0.0010) [2023-12-27 11:34:28,192][20347] Updated weights for policy 0, policy_version 856801 (0.0010) [2023-12-27 11:34:28,247][20347] Updated weights for policy 0, policy_version 856811 (0.0010) [2023-12-27 11:34:28,257][20381] Updated weights for policy 1, policy_version 856032 (0.0008) [2023-12-27 11:34:28,310][20381] Updated weights for policy 1, policy_version 856042 (0.0008) [2023-12-27 11:34:28,369][20381] Updated weights for policy 1, policy_version 856052 (0.0007) [2023-12-27 11:34:28,985][20347] Updated weights for policy 0, policy_version 856821 (0.0008) [2023-12-27 11:34:28,991][20381] Updated weights for policy 1, policy_version 856062 (0.0005) [2023-12-27 11:34:29,035][20381] Updated weights for policy 1, policy_version 856072 (0.0005) [2023-12-27 11:34:29,041][20347] Updated weights for policy 0, policy_version 856831 (0.0005) [2023-12-27 11:34:29,088][20381] Updated weights for policy 1, policy_version 856082 (0.0007) [2023-12-27 11:34:29,098][20347] Updated weights for policy 0, policy_version 856841 (0.0005) [2023-12-27 11:34:29,741][20347] Updated weights for policy 0, policy_version 856851 (0.0006) [2023-12-27 11:34:29,774][20381] Updated weights for policy 1, policy_version 856092 (0.0009) [2023-12-27 11:34:29,798][20347] Updated weights for policy 0, policy_version 856861 (0.0006) [2023-12-27 11:34:29,829][20381] Updated weights for policy 1, policy_version 856102 (0.0010) [2023-12-27 11:34:29,857][20347] Updated weights for policy 0, policy_version 856871 (0.0007) [2023-12-27 11:34:29,886][20381] Updated weights for policy 1, policy_version 856112 (0.0008) [2023-12-27 11:34:30,509][20381] Updated weights for policy 1, policy_version 856122 (0.0008) [2023-12-27 11:34:30,559][20381] Updated weights for policy 1, policy_version 856132 (0.0006) [2023-12-27 11:34:30,559][20347] Updated weights for policy 0, policy_version 856881 (0.0006) [2023-12-27 11:34:30,605][20381] Updated weights for policy 1, policy_version 856142 (0.0005) [2023-12-27 11:34:30,607][20347] Updated weights for policy 0, policy_version 856891 (0.0007) [2023-12-27 11:34:30,654][20381] Updated weights for policy 1, policy_version 856152 (0.0005) [2023-12-27 11:34:30,670][20347] Updated weights for policy 0, policy_version 856901 (0.0008) [2023-12-27 11:34:30,730][20347] Updated weights for policy 0, policy_version 856911 (0.0007) [2023-12-27 11:34:31,313][20381] Updated weights for policy 1, policy_version 856162 (0.0009) [2023-12-27 11:34:31,372][20381] Updated weights for policy 1, policy_version 856172 (0.0009) [2023-12-27 11:34:31,436][20381] Updated weights for policy 1, policy_version 856182 (0.0009) [2023-12-27 11:34:31,453][20347] Updated weights for policy 0, policy_version 856921 (0.0006) [2023-12-27 11:34:31,516][20347] Updated weights for policy 0, policy_version 856931 (0.0005) [2023-12-27 11:34:31,570][20347] Updated weights for policy 0, policy_version 856941 (0.0008) [2023-12-27 11:34:32,100][20381] Updated weights for policy 1, policy_version 856192 (0.0006) [2023-12-27 11:34:32,151][20381] Updated weights for policy 1, policy_version 856202 (0.0010) [2023-12-27 11:34:32,200][20381] Updated weights for policy 1, policy_version 856212 (0.0011) [2023-12-27 11:34:32,244][20347] Updated weights for policy 0, policy_version 856951 (0.0010) [2023-12-27 11:34:32,303][20347] Updated weights for policy 0, policy_version 856961 (0.0010) [2023-12-27 11:34:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 19549.7). Total num frames: 438624256. Throughput: 0: 10298.3, 1: 10148.4. Samples: 438634808. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:32,363][19170] Avg episode reward: [(0, '838.860'), (1, '643.200')] [2023-12-27 11:34:32,372][20347] Updated weights for policy 0, policy_version 856971 (0.0009) [2023-12-27 11:34:32,883][20381] Updated weights for policy 1, policy_version 856222 (0.0010) [2023-12-27 11:34:32,931][20381] Updated weights for policy 1, policy_version 856232 (0.0010) [2023-12-27 11:34:32,986][20381] Updated weights for policy 1, policy_version 856242 (0.0010) [2023-12-27 11:34:33,042][20347] Updated weights for policy 0, policy_version 856981 (0.0007) [2023-12-27 11:34:33,092][20347] Updated weights for policy 0, policy_version 856991 (0.0007) [2023-12-27 11:34:33,142][20347] Updated weights for policy 0, policy_version 857001 (0.0007) [2023-12-27 11:34:33,611][20381] Updated weights for policy 1, policy_version 856252 (0.0008) [2023-12-27 11:34:33,664][20381] Updated weights for policy 1, policy_version 856262 (0.0005) [2023-12-27 11:34:33,711][20381] Updated weights for policy 1, policy_version 856272 (0.0010) [2023-12-27 11:34:33,791][20347] Updated weights for policy 0, policy_version 857011 (0.0005) [2023-12-27 11:34:33,852][20347] Updated weights for policy 0, policy_version 857021 (0.0006) [2023-12-27 11:34:33,916][20347] Updated weights for policy 0, policy_version 857031 (0.0007) [2023-12-27 11:34:34,374][20381] Updated weights for policy 1, policy_version 856282 (0.0010) [2023-12-27 11:34:34,433][20381] Updated weights for policy 1, policy_version 856292 (0.0011) [2023-12-27 11:34:34,488][20381] Updated weights for policy 1, policy_version 856302 (0.0011) [2023-12-27 11:34:34,527][20347] Updated weights for policy 0, policy_version 857041 (0.0006) [2023-12-27 11:34:34,551][20381] Updated weights for policy 1, policy_version 856312 (0.0009) [2023-12-27 11:34:34,581][20347] Updated weights for policy 0, policy_version 857051 (0.0008) [2023-12-27 11:34:34,633][20347] Updated weights for policy 0, policy_version 857061 (0.0008) [2023-12-27 11:34:34,691][20347] Updated weights for policy 0, policy_version 857071 (0.0007) [2023-12-27 11:34:35,110][20381] Updated weights for policy 1, policy_version 856322 (0.0005) [2023-12-27 11:34:35,171][20381] Updated weights for policy 1, policy_version 856332 (0.0005) [2023-12-27 11:34:35,232][20381] Updated weights for policy 1, policy_version 856342 (0.0005) [2023-12-27 11:34:35,396][20347] Updated weights for policy 0, policy_version 857081 (0.0008) [2023-12-27 11:34:35,443][20347] Updated weights for policy 0, policy_version 857091 (0.0009) [2023-12-27 11:34:35,492][20347] Updated weights for policy 0, policy_version 857101 (0.0006) [2023-12-27 11:34:35,802][20381] Updated weights for policy 1, policy_version 856352 (0.0005) [2023-12-27 11:34:35,865][20381] Updated weights for policy 1, policy_version 856362 (0.0008) [2023-12-27 11:34:35,915][20381] Updated weights for policy 1, policy_version 856372 (0.0008) [2023-12-27 11:34:36,208][20347] Updated weights for policy 0, policy_version 857111 (0.0008) [2023-12-27 11:34:36,275][20347] Updated weights for policy 0, policy_version 857121 (0.0007) [2023-12-27 11:34:36,338][20347] Updated weights for policy 0, policy_version 857131 (0.0007) [2023-12-27 11:34:36,600][20381] Updated weights for policy 1, policy_version 856382 (0.0011) [2023-12-27 11:34:36,654][20381] Updated weights for policy 1, policy_version 856392 (0.0011) [2023-12-27 11:34:36,714][20381] Updated weights for policy 1, policy_version 856402 (0.0011) [2023-12-27 11:34:37,019][20347] Updated weights for policy 0, policy_version 857141 (0.0006) [2023-12-27 11:34:37,077][20347] Updated weights for policy 0, policy_version 857151 (0.0008) [2023-12-27 11:34:37,133][20347] Updated weights for policy 0, policy_version 857161 (0.0008) [2023-12-27 11:34:37,362][20381] Updated weights for policy 1, policy_version 856412 (0.0011) [2023-12-27 11:34:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 19577.5). Total num frames: 438730752. Throughput: 0: 10300.7, 1: 10191.9. Samples: 438698460. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:37,363][19170] Avg episode reward: [(0, '866.500'), (1, '661.270')] [2023-12-27 11:34:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000857168_219463680.pth... [2023-12-27 11:34:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000855952_219152384.pth [2023-12-27 11:34:37,418][20381] Updated weights for policy 1, policy_version 856422 (0.0011) [2023-12-27 11:34:37,479][20381] Updated weights for policy 1, policy_version 856432 (0.0009) [2023-12-27 11:34:37,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000856440_219275264.pth... [2023-12-27 11:34:37,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000855224_218963968.pth [2023-12-27 11:34:37,836][20347] Updated weights for policy 0, policy_version 857171 (0.0007) [2023-12-27 11:34:37,888][20347] Updated weights for policy 0, policy_version 857181 (0.0008) [2023-12-27 11:34:37,933][20347] Updated weights for policy 0, policy_version 857191 (0.0007) [2023-12-27 11:34:38,142][20381] Updated weights for policy 1, policy_version 856442 (0.0011) [2023-12-27 11:34:38,204][20381] Updated weights for policy 1, policy_version 856452 (0.0011) [2023-12-27 11:34:38,269][20381] Updated weights for policy 1, policy_version 856462 (0.0010) [2023-12-27 11:34:38,334][20381] Updated weights for policy 1, policy_version 856472 (0.0010) [2023-12-27 11:34:38,648][20347] Updated weights for policy 0, policy_version 857201 (0.0006) [2023-12-27 11:34:38,705][20347] Updated weights for policy 0, policy_version 857211 (0.0008) [2023-12-27 11:34:38,756][20347] Updated weights for policy 0, policy_version 857221 (0.0007) [2023-12-27 11:34:38,810][20347] Updated weights for policy 0, policy_version 857231 (0.0007) [2023-12-27 11:34:38,998][20381] Updated weights for policy 1, policy_version 856482 (0.0011) [2023-12-27 11:34:39,052][20381] Updated weights for policy 1, policy_version 856492 (0.0010) [2023-12-27 11:34:39,100][20381] Updated weights for policy 1, policy_version 856502 (0.0010) [2023-12-27 11:34:39,515][20347] Updated weights for policy 0, policy_version 857241 (0.0007) [2023-12-27 11:34:39,561][20347] Updated weights for policy 0, policy_version 857251 (0.0008) [2023-12-27 11:34:39,621][20347] Updated weights for policy 0, policy_version 857261 (0.0007) [2023-12-27 11:34:39,815][20381] Updated weights for policy 1, policy_version 856512 (0.0009) [2023-12-27 11:34:39,879][20381] Updated weights for policy 1, policy_version 856522 (0.0008) [2023-12-27 11:34:39,937][20381] Updated weights for policy 1, policy_version 856532 (0.0009) [2023-12-27 11:34:40,393][20347] Updated weights for policy 0, policy_version 857271 (0.0008) [2023-12-27 11:34:40,443][20347] Updated weights for policy 0, policy_version 857281 (0.0008) [2023-12-27 11:34:40,498][20347] Updated weights for policy 0, policy_version 857291 (0.0006) [2023-12-27 11:34:40,658][20381] Updated weights for policy 1, policy_version 856542 (0.0008) [2023-12-27 11:34:40,721][20381] Updated weights for policy 1, policy_version 856552 (0.0009) [2023-12-27 11:34:40,780][20381] Updated weights for policy 1, policy_version 856562 (0.0009) [2023-12-27 11:34:41,213][20347] Updated weights for policy 0, policy_version 857301 (0.0006) [2023-12-27 11:34:41,274][20347] Updated weights for policy 0, policy_version 857311 (0.0008) [2023-12-27 11:34:41,338][20347] Updated weights for policy 0, policy_version 857321 (0.0007) [2023-12-27 11:34:41,501][20381] Updated weights for policy 1, policy_version 856572 (0.0009) [2023-12-27 11:34:41,557][20381] Updated weights for policy 1, policy_version 856582 (0.0007) [2023-12-27 11:34:41,581][20292] KL-divergence is very high: 123.6645 [2023-12-27 11:34:41,619][20381] Updated weights for policy 1, policy_version 856592 (0.0007) [2023-12-27 11:34:42,064][20347] Updated weights for policy 0, policy_version 857331 (0.0008) [2023-12-27 11:34:42,128][20347] Updated weights for policy 0, policy_version 857341 (0.0008) [2023-12-27 11:34:42,183][20347] Updated weights for policy 0, policy_version 857351 (0.0007) [2023-12-27 11:34:42,300][20381] Updated weights for policy 1, policy_version 856602 (0.0007) [2023-12-27 11:34:42,339][20292] KL-divergence is very high: 227.7817 [2023-12-27 11:34:42,345][20292] KL-divergence is very high: 293.8724 [2023-12-27 11:34:42,351][20292] KL-divergence is very high: 219.5975 [2023-12-27 11:34:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 19577.5). Total num frames: 438829056. Throughput: 0: 10237.0, 1: 10192.9. Samples: 438820184. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:42,363][19170] Avg episode reward: [(0, '872.830'), (1, '730.990')] [2023-12-27 11:34:42,365][20381] Updated weights for policy 1, policy_version 856612 (0.0008) [2023-12-27 11:34:42,391][20292] KL-divergence is very high: 162.7984 [2023-12-27 11:34:42,428][20381] Updated weights for policy 1, policy_version 856622 (0.0008) [2023-12-27 11:34:42,490][20381] Updated weights for policy 1, policy_version 856632 (0.0005) [2023-12-27 11:34:42,852][20347] Updated weights for policy 0, policy_version 857361 (0.0005) [2023-12-27 11:34:42,916][20347] Updated weights for policy 0, policy_version 857371 (0.0008) [2023-12-27 11:34:42,979][20347] Updated weights for policy 0, policy_version 857381 (0.0008) [2023-12-27 11:34:43,028][20347] Updated weights for policy 0, policy_version 857391 (0.0007) [2023-12-27 11:34:43,100][20292] KL-divergence is very high: 101.6256 [2023-12-27 11:34:43,136][20381] Updated weights for policy 1, policy_version 856642 (0.0008) [2023-12-27 11:34:43,145][20292] KL-divergence is very high: 153.4440 [2023-12-27 11:34:43,189][20292] KL-divergence is very high: 103.3622 [2023-12-27 11:34:43,189][20381] Updated weights for policy 1, policy_version 856652 (0.0007) [2023-12-27 11:34:43,239][20381] Updated weights for policy 1, policy_version 856662 (0.0008) [2023-12-27 11:34:43,737][20347] Updated weights for policy 0, policy_version 857401 (0.0008) [2023-12-27 11:34:43,793][20347] Updated weights for policy 0, policy_version 857411 (0.0008) [2023-12-27 11:34:43,834][20381] Updated weights for policy 1, policy_version 856672 (0.0007) [2023-12-27 11:34:43,843][20347] Updated weights for policy 0, policy_version 857421 (0.0006) [2023-12-27 11:34:43,884][20381] Updated weights for policy 1, policy_version 856682 (0.0007) [2023-12-27 11:34:43,933][20381] Updated weights for policy 1, policy_version 856692 (0.0006) [2023-12-27 11:34:44,511][20347] Updated weights for policy 0, policy_version 857431 (0.0007) [2023-12-27 11:34:44,566][20347] Updated weights for policy 0, policy_version 857441 (0.0006) [2023-12-27 11:34:44,575][20381] Updated weights for policy 1, policy_version 856702 (0.0008) [2023-12-27 11:34:44,620][20381] Updated weights for policy 1, policy_version 856712 (0.0008) [2023-12-27 11:34:44,622][20347] Updated weights for policy 0, policy_version 857451 (0.0005) [2023-12-27 11:34:44,684][20381] Updated weights for policy 1, policy_version 856722 (0.0005) [2023-12-27 11:34:45,265][20347] Updated weights for policy 0, policy_version 857461 (0.0006) [2023-12-27 11:34:45,317][20347] Updated weights for policy 0, policy_version 857471 (0.0008) [2023-12-27 11:34:45,370][20347] Updated weights for policy 0, policy_version 857481 (0.0007) [2023-12-27 11:34:45,404][20381] Updated weights for policy 1, policy_version 856732 (0.0006) [2023-12-27 11:34:45,456][20381] Updated weights for policy 1, policy_version 856742 (0.0008) [2023-12-27 11:34:45,509][20381] Updated weights for policy 1, policy_version 856752 (0.0007) [2023-12-27 11:34:46,054][20347] Updated weights for policy 0, policy_version 857491 (0.0008) [2023-12-27 11:34:46,114][20347] Updated weights for policy 0, policy_version 857501 (0.0008) [2023-12-27 11:34:46,163][20347] Updated weights for policy 0, policy_version 857511 (0.0008) [2023-12-27 11:34:46,220][20381] Updated weights for policy 1, policy_version 856762 (0.0008) [2023-12-27 11:34:46,269][20381] Updated weights for policy 1, policy_version 856772 (0.0008) [2023-12-27 11:34:46,325][20381] Updated weights for policy 1, policy_version 856782 (0.0007) [2023-12-27 11:34:46,383][20381] Updated weights for policy 1, policy_version 856792 (0.0006) [2023-12-27 11:34:46,874][20347] Updated weights for policy 0, policy_version 857521 (0.0009) [2023-12-27 11:34:46,942][20347] Updated weights for policy 0, policy_version 857531 (0.0007) [2023-12-27 11:34:47,003][20347] Updated weights for policy 0, policy_version 857541 (0.0008) [2023-12-27 11:34:47,017][20381] Updated weights for policy 1, policy_version 856802 (0.0011) [2023-12-27 11:34:47,059][20347] Updated weights for policy 0, policy_version 857551 (0.0005) [2023-12-27 11:34:47,065][20381] Updated weights for policy 1, policy_version 856812 (0.0010) [2023-12-27 11:34:47,121][20381] Updated weights for policy 1, policy_version 856822 (0.0007) [2023-12-27 11:34:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 19633.0). Total num frames: 438935552. Throughput: 0: 10189.2, 1: 10282.5. Samples: 438942864. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:47,364][19170] Avg episode reward: [(0, '872.910'), (1, '654.370')] [2023-12-27 11:34:47,755][20347] Updated weights for policy 0, policy_version 857561 (0.0008) [2023-12-27 11:34:47,766][20381] Updated weights for policy 1, policy_version 856832 (0.0007) [2023-12-27 11:34:47,822][20347] Updated weights for policy 0, policy_version 857571 (0.0008) [2023-12-27 11:34:47,831][20381] Updated weights for policy 1, policy_version 856842 (0.0006) [2023-12-27 11:34:47,880][20347] Updated weights for policy 0, policy_version 857581 (0.0007) [2023-12-27 11:34:47,891][20381] Updated weights for policy 1, policy_version 856852 (0.0005) [2023-12-27 11:34:48,503][20381] Updated weights for policy 1, policy_version 856862 (0.0007) [2023-12-27 11:34:48,553][20381] Updated weights for policy 1, policy_version 856872 (0.0007) [2023-12-27 11:34:48,593][20347] Updated weights for policy 0, policy_version 857591 (0.0008) [2023-12-27 11:34:48,608][20381] Updated weights for policy 1, policy_version 856882 (0.0006) [2023-12-27 11:34:48,655][20347] Updated weights for policy 0, policy_version 857601 (0.0006) [2023-12-27 11:34:48,720][20347] Updated weights for policy 0, policy_version 857611 (0.0009) [2023-12-27 11:34:49,246][20381] Updated weights for policy 1, policy_version 856892 (0.0007) [2023-12-27 11:34:49,293][20381] Updated weights for policy 1, policy_version 856902 (0.0008) [2023-12-27 11:34:49,353][20381] Updated weights for policy 1, policy_version 856912 (0.0008) [2023-12-27 11:34:49,434][20347] Updated weights for policy 0, policy_version 857621 (0.0007) [2023-12-27 11:34:49,495][20347] Updated weights for policy 0, policy_version 857631 (0.0006) [2023-12-27 11:34:49,557][20347] Updated weights for policy 0, policy_version 857641 (0.0005) [2023-12-27 11:34:50,109][20381] Updated weights for policy 1, policy_version 856922 (0.0006) [2023-12-27 11:34:50,164][20381] Updated weights for policy 1, policy_version 856932 (0.0007) [2023-12-27 11:34:50,205][20347] Updated weights for policy 0, policy_version 857651 (0.0007) [2023-12-27 11:34:50,215][20381] Updated weights for policy 1, policy_version 856942 (0.0007) [2023-12-27 11:34:50,260][20347] Updated weights for policy 0, policy_version 857661 (0.0007) [2023-12-27 11:34:50,267][20381] Updated weights for policy 1, policy_version 856952 (0.0007) [2023-12-27 11:34:50,325][20347] Updated weights for policy 0, policy_version 857671 (0.0007) [2023-12-27 11:34:50,937][20381] Updated weights for policy 1, policy_version 856962 (0.0007) [2023-12-27 11:34:50,986][20381] Updated weights for policy 1, policy_version 856972 (0.0007) [2023-12-27 11:34:51,052][20381] Updated weights for policy 1, policy_version 856982 (0.0008) [2023-12-27 11:34:51,057][20347] Updated weights for policy 0, policy_version 857681 (0.0008) [2023-12-27 11:34:51,112][20347] Updated weights for policy 0, policy_version 857691 (0.0008) [2023-12-27 11:34:51,180][20347] Updated weights for policy 0, policy_version 857701 (0.0008) [2023-12-27 11:34:51,239][20347] Updated weights for policy 0, policy_version 857711 (0.0008) [2023-12-27 11:34:51,761][20381] Updated weights for policy 1, policy_version 856992 (0.0008) [2023-12-27 11:34:51,824][20381] Updated weights for policy 1, policy_version 857002 (0.0008) [2023-12-27 11:34:51,885][20381] Updated weights for policy 1, policy_version 857012 (0.0009) [2023-12-27 11:34:51,982][20347] Updated weights for policy 0, policy_version 857721 (0.0007) [2023-12-27 11:34:52,043][20347] Updated weights for policy 0, policy_version 857731 (0.0006) [2023-12-27 11:34:52,101][20347] Updated weights for policy 0, policy_version 857741 (0.0007) [2023-12-27 11:34:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 19633.0). Total num frames: 439033856. Throughput: 0: 10162.4, 1: 10329.7. Samples: 439004276. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:52,363][19170] Avg episode reward: [(0, '863.080'), (1, '632.490')] [2023-12-27 11:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000857016_219422720.pth... [2023-12-27 11:34:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000857744_219611136.pth... [2023-12-27 11:34:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000856560_219308032.pth [2023-12-27 11:34:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000855800_219111424.pth [2023-12-27 11:34:52,616][20381] Updated weights for policy 1, policy_version 857022 (0.0006) [2023-12-27 11:34:52,670][20381] Updated weights for policy 1, policy_version 857032 (0.0009) [2023-12-27 11:34:52,721][20381] Updated weights for policy 1, policy_version 857042 (0.0011) [2023-12-27 11:34:52,770][20347] Updated weights for policy 0, policy_version 857751 (0.0006) [2023-12-27 11:34:52,826][20347] Updated weights for policy 0, policy_version 857761 (0.0005) [2023-12-27 11:34:52,879][20347] Updated weights for policy 0, policy_version 857771 (0.0006) [2023-12-27 11:34:53,403][20381] Updated weights for policy 1, policy_version 857052 (0.0010) [2023-12-27 11:34:53,455][20381] Updated weights for policy 1, policy_version 857062 (0.0010) [2023-12-27 11:34:53,507][20381] Updated weights for policy 1, policy_version 857072 (0.0010) [2023-12-27 11:34:53,521][20347] Updated weights for policy 0, policy_version 857781 (0.0008) [2023-12-27 11:34:53,576][20347] Updated weights for policy 0, policy_version 857791 (0.0010) [2023-12-27 11:34:53,622][20347] Updated weights for policy 0, policy_version 857801 (0.0006) [2023-12-27 11:34:54,112][20381] Updated weights for policy 1, policy_version 857082 (0.0010) [2023-12-27 11:34:54,176][20381] Updated weights for policy 1, policy_version 857092 (0.0010) [2023-12-27 11:34:54,190][20347] Updated weights for policy 0, policy_version 857811 (0.0009) [2023-12-27 11:34:54,226][20381] Updated weights for policy 1, policy_version 857102 (0.0011) [2023-12-27 11:34:54,256][20347] Updated weights for policy 0, policy_version 857821 (0.0005) [2023-12-27 11:34:54,281][20381] Updated weights for policy 1, policy_version 857112 (0.0010) [2023-12-27 11:34:54,306][20347] Updated weights for policy 0, policy_version 857831 (0.0010) [2023-12-27 11:34:54,931][20381] Updated weights for policy 1, policy_version 857122 (0.0005) [2023-12-27 11:34:54,940][20347] Updated weights for policy 0, policy_version 857841 (0.0008) [2023-12-27 11:34:54,990][20381] Updated weights for policy 1, policy_version 857132 (0.0005) [2023-12-27 11:34:55,001][20347] Updated weights for policy 0, policy_version 857851 (0.0007) [2023-12-27 11:34:55,046][20381] Updated weights for policy 1, policy_version 857142 (0.0009) [2023-12-27 11:34:55,066][20347] Updated weights for policy 0, policy_version 857861 (0.0009) [2023-12-27 11:34:55,123][20347] Updated weights for policy 0, policy_version 857871 (0.0010) [2023-12-27 11:34:55,693][20381] Updated weights for policy 1, policy_version 857152 (0.0008) [2023-12-27 11:34:55,746][20381] Updated weights for policy 1, policy_version 857162 (0.0007) [2023-12-27 11:34:55,752][20347] Updated weights for policy 0, policy_version 857881 (0.0010) [2023-12-27 11:34:55,795][20381] Updated weights for policy 1, policy_version 857172 (0.0005) [2023-12-27 11:34:55,809][20347] Updated weights for policy 0, policy_version 857891 (0.0010) [2023-12-27 11:34:55,872][20347] Updated weights for policy 0, policy_version 857901 (0.0010) [2023-12-27 11:34:56,340][20381] Updated weights for policy 1, policy_version 857182 (0.0007) [2023-12-27 11:34:56,391][20381] Updated weights for policy 1, policy_version 857192 (0.0005) [2023-12-27 11:34:56,432][20381] Updated weights for policy 1, policy_version 857202 (0.0005) [2023-12-27 11:34:56,535][20347] Updated weights for policy 0, policy_version 857911 (0.0007) [2023-12-27 11:34:56,588][20347] Updated weights for policy 0, policy_version 857921 (0.0008) [2023-12-27 11:34:56,633][20347] Updated weights for policy 0, policy_version 857931 (0.0010) [2023-12-27 11:34:57,018][20381] Updated weights for policy 1, policy_version 857212 (0.0006) [2023-12-27 11:34:57,063][20381] Updated weights for policy 1, policy_version 857222 (0.0006) [2023-12-27 11:34:57,116][20381] Updated weights for policy 1, policy_version 857232 (0.0005) [2023-12-27 11:34:57,307][20347] Updated weights for policy 0, policy_version 857941 (0.0010) [2023-12-27 11:34:57,357][20347] Updated weights for policy 0, policy_version 857951 (0.0010) [2023-12-27 11:34:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 19688.6). Total num frames: 439140352. Throughput: 0: 10193.1, 1: 10416.6. Samples: 439129836. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:34:57,363][19170] Avg episode reward: [(0, '872.830'), (1, '643.860')] [2023-12-27 11:34:57,405][20347] Updated weights for policy 0, policy_version 857961 (0.0010) [2023-12-27 11:34:57,747][20381] Updated weights for policy 1, policy_version 857242 (0.0005) [2023-12-27 11:34:57,800][20381] Updated weights for policy 1, policy_version 857252 (0.0005) [2023-12-27 11:34:57,850][20381] Updated weights for policy 1, policy_version 857262 (0.0005) [2023-12-27 11:34:57,899][20381] Updated weights for policy 1, policy_version 857272 (0.0006) [2023-12-27 11:34:58,090][20347] Updated weights for policy 0, policy_version 857971 (0.0010) [2023-12-27 11:34:58,144][20347] Updated weights for policy 0, policy_version 857981 (0.0010) [2023-12-27 11:34:58,198][20347] Updated weights for policy 0, policy_version 857991 (0.0010) [2023-12-27 11:34:58,584][20381] Updated weights for policy 1, policy_version 857282 (0.0006) [2023-12-27 11:34:58,651][20381] Updated weights for policy 1, policy_version 857292 (0.0007) [2023-12-27 11:34:58,714][20381] Updated weights for policy 1, policy_version 857302 (0.0008) [2023-12-27 11:34:59,010][20347] Updated weights for policy 0, policy_version 858001 (0.0010) [2023-12-27 11:34:59,072][20347] Updated weights for policy 0, policy_version 858011 (0.0008) [2023-12-27 11:34:59,132][20347] Updated weights for policy 0, policy_version 858021 (0.0007) [2023-12-27 11:34:59,192][20347] Updated weights for policy 0, policy_version 858031 (0.0007) [2023-12-27 11:34:59,498][20381] Updated weights for policy 1, policy_version 857312 (0.0008) [2023-12-27 11:34:59,549][20381] Updated weights for policy 1, policy_version 857322 (0.0008) [2023-12-27 11:34:59,607][20381] Updated weights for policy 1, policy_version 857332 (0.0008) [2023-12-27 11:34:59,922][20347] Updated weights for policy 0, policy_version 858041 (0.0009) [2023-12-27 11:34:59,987][20347] Updated weights for policy 0, policy_version 858051 (0.0010) [2023-12-27 11:35:00,052][20347] Updated weights for policy 0, policy_version 858061 (0.0010) [2023-12-27 11:35:00,290][20381] Updated weights for policy 1, policy_version 857342 (0.0009) [2023-12-27 11:35:00,342][20381] Updated weights for policy 1, policy_version 857352 (0.0008) [2023-12-27 11:35:00,396][20381] Updated weights for policy 1, policy_version 857362 (0.0006) [2023-12-27 11:35:00,696][20347] Updated weights for policy 0, policy_version 858071 (0.0010) [2023-12-27 11:35:00,753][20347] Updated weights for policy 0, policy_version 858081 (0.0010) [2023-12-27 11:35:00,811][20347] Updated weights for policy 0, policy_version 858091 (0.0010) [2023-12-27 11:35:01,064][20381] Updated weights for policy 1, policy_version 857372 (0.0007) [2023-12-27 11:35:01,127][20381] Updated weights for policy 1, policy_version 857382 (0.0009) [2023-12-27 11:35:01,183][20381] Updated weights for policy 1, policy_version 857392 (0.0008) [2023-12-27 11:35:01,526][20347] Updated weights for policy 0, policy_version 858101 (0.0009) [2023-12-27 11:35:01,579][20347] Updated weights for policy 0, policy_version 858111 (0.0008) [2023-12-27 11:35:01,638][20347] Updated weights for policy 0, policy_version 858121 (0.0009) [2023-12-27 11:35:01,930][20381] Updated weights for policy 1, policy_version 857402 (0.0007) [2023-12-27 11:35:01,990][20292] KL-divergence is very high: 261.2793 [2023-12-27 11:35:01,997][20381] Updated weights for policy 1, policy_version 857412 (0.0005) [2023-12-27 11:35:01,997][20292] KL-divergence is very high: 253.2280 [2023-12-27 11:35:02,054][20381] Updated weights for policy 1, policy_version 857422 (0.0006) [2023-12-27 11:35:02,116][20381] Updated weights for policy 1, policy_version 857432 (0.0005) [2023-12-27 11:35:02,311][20347] Updated weights for policy 0, policy_version 858131 (0.0006) [2023-12-27 11:35:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 19688.6). Total num frames: 439238656. Throughput: 0: 10150.7, 1: 10389.4. Samples: 439251428. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:35:02,363][19170] Avg episode reward: [(0, '892.540'), (1, '662.260')] [2023-12-27 11:35:02,378][20347] Updated weights for policy 0, policy_version 858141 (0.0007) [2023-12-27 11:35:02,437][20347] Updated weights for policy 0, policy_version 858151 (0.0007) [2023-12-27 11:35:02,691][20381] Updated weights for policy 1, policy_version 857442 (0.0009) [2023-12-27 11:35:02,753][20381] Updated weights for policy 1, policy_version 857452 (0.0007) [2023-12-27 11:35:02,819][20381] Updated weights for policy 1, policy_version 857462 (0.0007) [2023-12-27 11:35:03,037][20347] Updated weights for policy 0, policy_version 858161 (0.0006) [2023-12-27 11:35:03,098][20347] Updated weights for policy 0, policy_version 858171 (0.0008) [2023-12-27 11:35:03,156][20347] Updated weights for policy 0, policy_version 858181 (0.0005) [2023-12-27 11:35:03,212][20347] Updated weights for policy 0, policy_version 858191 (0.0005) [2023-12-27 11:35:03,532][20381] Updated weights for policy 1, policy_version 857472 (0.0007) [2023-12-27 11:35:03,587][20381] Updated weights for policy 1, policy_version 857482 (0.0007) [2023-12-27 11:35:03,636][20381] Updated weights for policy 1, policy_version 857492 (0.0007) [2023-12-27 11:35:03,735][20347] Updated weights for policy 0, policy_version 858201 (0.0006) [2023-12-27 11:35:03,789][20347] Updated weights for policy 0, policy_version 858211 (0.0007) [2023-12-27 11:35:03,847][20347] Updated weights for policy 0, policy_version 858221 (0.0007) [2023-12-27 11:35:04,373][20381] Updated weights for policy 1, policy_version 857502 (0.0007) [2023-12-27 11:35:04,437][20381] Updated weights for policy 1, policy_version 857512 (0.0007) [2023-12-27 11:35:04,507][20347] Updated weights for policy 0, policy_version 858231 (0.0007) [2023-12-27 11:35:04,508][20381] Updated weights for policy 1, policy_version 857522 (0.0006) [2023-12-27 11:35:04,561][20347] Updated weights for policy 0, policy_version 858241 (0.0007) [2023-12-27 11:35:04,625][20347] Updated weights for policy 0, policy_version 858251 (0.0008) [2023-12-27 11:35:05,146][20381] Updated weights for policy 1, policy_version 857532 (0.0007) [2023-12-27 11:35:05,191][20381] Updated weights for policy 1, policy_version 857542 (0.0007) [2023-12-27 11:35:05,242][20381] Updated weights for policy 1, policy_version 857552 (0.0008) [2023-12-27 11:35:05,317][20347] Updated weights for policy 0, policy_version 858261 (0.0007) [2023-12-27 11:35:05,367][20347] Updated weights for policy 0, policy_version 858271 (0.0007) [2023-12-27 11:35:05,419][20347] Updated weights for policy 0, policy_version 858281 (0.0007) [2023-12-27 11:35:05,976][20381] Updated weights for policy 1, policy_version 857562 (0.0008) [2023-12-27 11:35:06,036][20381] Updated weights for policy 1, policy_version 857572 (0.0008) [2023-12-27 11:35:06,058][20347] Updated weights for policy 0, policy_version 858291 (0.0005) [2023-12-27 11:35:06,088][20381] Updated weights for policy 1, policy_version 857582 (0.0006) [2023-12-27 11:35:06,119][20347] Updated weights for policy 0, policy_version 858301 (0.0007) [2023-12-27 11:35:06,158][20381] Updated weights for policy 1, policy_version 857592 (0.0009) [2023-12-27 11:35:06,175][20347] Updated weights for policy 0, policy_version 858311 (0.0007) [2023-12-27 11:35:06,765][20381] Updated weights for policy 1, policy_version 857602 (0.0006) [2023-12-27 11:35:06,819][20381] Updated weights for policy 1, policy_version 857612 (0.0005) [2023-12-27 11:35:06,871][20381] Updated weights for policy 1, policy_version 857622 (0.0005) [2023-12-27 11:35:06,934][20347] Updated weights for policy 0, policy_version 858321 (0.0008) [2023-12-27 11:35:06,993][20347] Updated weights for policy 0, policy_version 858331 (0.0008) [2023-12-27 11:35:07,054][20347] Updated weights for policy 0, policy_version 858341 (0.0008) [2023-12-27 11:35:07,110][20347] Updated weights for policy 0, policy_version 858351 (0.0007) [2023-12-27 11:35:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.4, 300 sec: 19716.3). Total num frames: 439345152. Throughput: 0: 10219.5, 1: 10368.8. Samples: 439314020. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:35:07,364][19170] Avg episode reward: [(0, '931.670'), (1, '785.850')] [2023-12-27 11:35:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000857624_219578368.pth... [2023-12-27 11:35:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000858352_219766784.pth... [2023-12-27 11:35:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000856440_219275264.pth [2023-12-27 11:35:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000857168_219463680.pth [2023-12-27 11:35:07,539][20381] Updated weights for policy 1, policy_version 857632 (0.0007) [2023-12-27 11:35:07,587][20381] Updated weights for policy 1, policy_version 857642 (0.0007) [2023-12-27 11:35:07,639][20381] Updated weights for policy 1, policy_version 857652 (0.0008) [2023-12-27 11:35:07,733][20347] Updated weights for policy 0, policy_version 858361 (0.0008) [2023-12-27 11:35:07,789][20347] Updated weights for policy 0, policy_version 858371 (0.0007) [2023-12-27 11:35:07,841][20347] Updated weights for policy 0, policy_version 858381 (0.0008) [2023-12-27 11:35:08,306][20381] Updated weights for policy 1, policy_version 857662 (0.0008) [2023-12-27 11:35:08,368][20381] Updated weights for policy 1, policy_version 857672 (0.0008) [2023-12-27 11:35:08,416][20381] Updated weights for policy 1, policy_version 857682 (0.0008) [2023-12-27 11:35:08,536][20347] Updated weights for policy 0, policy_version 858391 (0.0007) [2023-12-27 11:35:08,593][20347] Updated weights for policy 0, policy_version 858401 (0.0007) [2023-12-27 11:35:08,649][20347] Updated weights for policy 0, policy_version 858411 (0.0007) [2023-12-27 11:35:09,151][20381] Updated weights for policy 1, policy_version 857692 (0.0008) [2023-12-27 11:35:09,205][20381] Updated weights for policy 1, policy_version 857702 (0.0008) [2023-12-27 11:35:09,269][20381] Updated weights for policy 1, policy_version 857712 (0.0007) [2023-12-27 11:35:09,320][20347] Updated weights for policy 0, policy_version 858421 (0.0009) [2023-12-27 11:35:09,380][20347] Updated weights for policy 0, policy_version 858431 (0.0007) [2023-12-27 11:35:09,447][20347] Updated weights for policy 0, policy_version 858441 (0.0008) [2023-12-27 11:35:09,991][20381] Updated weights for policy 1, policy_version 857722 (0.0007) [2023-12-27 11:35:10,044][20381] Updated weights for policy 1, policy_version 857732 (0.0009) [2023-12-27 11:35:10,114][20381] Updated weights for policy 1, policy_version 857742 (0.0006) [2023-12-27 11:35:10,174][20347] Updated weights for policy 0, policy_version 858451 (0.0007) [2023-12-27 11:35:10,180][20381] Updated weights for policy 1, policy_version 857752 (0.0006) [2023-12-27 11:35:10,227][20347] Updated weights for policy 0, policy_version 858461 (0.0006) [2023-12-27 11:35:10,285][20347] Updated weights for policy 0, policy_version 858471 (0.0006) [2023-12-27 11:35:10,763][20381] Updated weights for policy 1, policy_version 857762 (0.0008) [2023-12-27 11:35:10,824][20381] Updated weights for policy 1, policy_version 857772 (0.0007) [2023-12-27 11:35:10,882][20381] Updated weights for policy 1, policy_version 857782 (0.0007) [2023-12-27 11:35:11,010][20347] Updated weights for policy 0, policy_version 858481 (0.0008) [2023-12-27 11:35:11,080][20347] Updated weights for policy 0, policy_version 858491 (0.0008) [2023-12-27 11:35:11,147][20347] Updated weights for policy 0, policy_version 858501 (0.0008) [2023-12-27 11:35:11,211][20347] Updated weights for policy 0, policy_version 858511 (0.0007) [2023-12-27 11:35:11,572][20381] Updated weights for policy 1, policy_version 857792 (0.0007) [2023-12-27 11:35:11,637][20381] Updated weights for policy 1, policy_version 857802 (0.0008) [2023-12-27 11:35:11,698][20381] Updated weights for policy 1, policy_version 857812 (0.0007) [2023-12-27 11:35:11,980][20347] Updated weights for policy 0, policy_version 858521 (0.0007) [2023-12-27 11:35:12,041][20347] Updated weights for policy 0, policy_version 858531 (0.0008) [2023-12-27 11:35:12,096][20347] Updated weights for policy 0, policy_version 858541 (0.0008) [2023-12-27 11:35:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 19716.3). Total num frames: 439443456. Throughput: 0: 10172.5, 1: 10421.3. Samples: 439435544. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:35:12,363][19170] Avg episode reward: [(0, '912.080'), (1, '863.570')] [2023-12-27 11:35:12,415][20381] Updated weights for policy 1, policy_version 857822 (0.0006) [2023-12-27 11:35:12,476][20381] Updated weights for policy 1, policy_version 857832 (0.0007) [2023-12-27 11:35:12,532][20381] Updated weights for policy 1, policy_version 857842 (0.0008) [2023-12-27 11:35:12,816][20347] Updated weights for policy 0, policy_version 858551 (0.0008) [2023-12-27 11:35:12,872][20347] Updated weights for policy 0, policy_version 858561 (0.0009) [2023-12-27 11:35:12,927][20347] Updated weights for policy 0, policy_version 858571 (0.0007) [2023-12-27 11:35:13,170][20381] Updated weights for policy 1, policy_version 857852 (0.0007) [2023-12-27 11:35:13,222][20381] Updated weights for policy 1, policy_version 857862 (0.0007) [2023-12-27 11:35:13,278][20381] Updated weights for policy 1, policy_version 857872 (0.0008) [2023-12-27 11:35:13,653][20347] Updated weights for policy 0, policy_version 858581 (0.0007) [2023-12-27 11:35:13,705][20347] Updated weights for policy 0, policy_version 858591 (0.0007) [2023-12-27 11:35:13,755][20347] Updated weights for policy 0, policy_version 858601 (0.0005) [2023-12-27 11:35:13,972][20381] Updated weights for policy 1, policy_version 857882 (0.0008) [2023-12-27 11:35:14,031][20381] Updated weights for policy 1, policy_version 857892 (0.0009) [2023-12-27 11:35:14,084][20381] Updated weights for policy 1, policy_version 857902 (0.0008) [2023-12-27 11:35:14,146][20381] Updated weights for policy 1, policy_version 857912 (0.0008) [2023-12-27 11:35:14,411][20347] Updated weights for policy 0, policy_version 858611 (0.0007) [2023-12-27 11:35:14,462][20347] Updated weights for policy 0, policy_version 858621 (0.0008) [2023-12-27 11:35:14,518][20347] Updated weights for policy 0, policy_version 858631 (0.0008) [2023-12-27 11:35:14,795][20381] Updated weights for policy 1, policy_version 857922 (0.0007) [2023-12-27 11:35:14,847][20381] Updated weights for policy 1, policy_version 857932 (0.0008) [2023-12-27 11:35:14,902][20381] Updated weights for policy 1, policy_version 857942 (0.0008) [2023-12-27 11:35:15,220][20347] Updated weights for policy 0, policy_version 858641 (0.0006) [2023-12-27 11:35:15,273][20347] Updated weights for policy 0, policy_version 858651 (0.0009) [2023-12-27 11:35:15,327][20347] Updated weights for policy 0, policy_version 858661 (0.0009) [2023-12-27 11:35:15,376][20347] Updated weights for policy 0, policy_version 858671 (0.0009) [2023-12-27 11:35:15,670][20381] Updated weights for policy 1, policy_version 857952 (0.0007) [2023-12-27 11:35:15,717][20381] Updated weights for policy 1, policy_version 857962 (0.0007) [2023-12-27 11:35:15,775][20381] Updated weights for policy 1, policy_version 857972 (0.0007) [2023-12-27 11:35:16,111][20347] Updated weights for policy 0, policy_version 858681 (0.0010) [2023-12-27 11:35:16,163][20347] Updated weights for policy 0, policy_version 858691 (0.0010) [2023-12-27 11:35:16,218][20347] Updated weights for policy 0, policy_version 858701 (0.0006) [2023-12-27 11:35:16,516][20381] Updated weights for policy 1, policy_version 857982 (0.0008) [2023-12-27 11:35:16,575][20381] Updated weights for policy 1, policy_version 857992 (0.0007) [2023-12-27 11:35:16,632][20381] Updated weights for policy 1, policy_version 858002 (0.0005) [2023-12-27 11:35:16,791][20347] Updated weights for policy 0, policy_version 858711 (0.0005) [2023-12-27 11:35:16,855][20347] Updated weights for policy 0, policy_version 858721 (0.0005) [2023-12-27 11:35:16,917][20347] Updated weights for policy 0, policy_version 858731 (0.0006) [2023-12-27 11:35:17,224][20381] Updated weights for policy 1, policy_version 858012 (0.0008) [2023-12-27 11:35:17,268][20381] Updated weights for policy 1, policy_version 858022 (0.0009) [2023-12-27 11:35:17,326][20381] Updated weights for policy 1, policy_version 858032 (0.0005) [2023-12-27 11:35:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.6, 300 sec: 19744.1). Total num frames: 439541760. Throughput: 0: 10139.7, 1: 10348.0. Samples: 439556752. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:35:17,363][19170] Avg episode reward: [(0, '912.020'), (1, '873.530')] [2023-12-27 11:35:17,490][20347] Updated weights for policy 0, policy_version 858741 (0.0008) [2023-12-27 11:35:17,544][20347] Updated weights for policy 0, policy_version 858751 (0.0005) [2023-12-27 11:35:17,605][20347] Updated weights for policy 0, policy_version 858761 (0.0005) [2023-12-27 11:35:18,000][20381] Updated weights for policy 1, policy_version 858042 (0.0010) [2023-12-27 11:35:18,065][20381] Updated weights for policy 1, policy_version 858052 (0.0011) [2023-12-27 11:35:18,126][20381] Updated weights for policy 1, policy_version 858062 (0.0011) [2023-12-27 11:35:18,186][20381] Updated weights for policy 1, policy_version 858072 (0.0010) [2023-12-27 11:35:18,221][20347] Updated weights for policy 0, policy_version 858771 (0.0007) [2023-12-27 11:35:18,281][20347] Updated weights for policy 0, policy_version 858781 (0.0009) [2023-12-27 11:35:18,335][20347] Updated weights for policy 0, policy_version 858791 (0.0010) [2023-12-27 11:35:18,880][20381] Updated weights for policy 1, policy_version 858082 (0.0009) [2023-12-27 11:35:18,928][20381] Updated weights for policy 1, policy_version 858092 (0.0007) [2023-12-27 11:35:18,986][20381] Updated weights for policy 1, policy_version 858102 (0.0007) [2023-12-27 11:35:19,003][20347] Updated weights for policy 0, policy_version 858801 (0.0009) [2023-12-27 11:35:19,059][20347] Updated weights for policy 0, policy_version 858811 (0.0007) [2023-12-27 11:35:19,114][20347] Updated weights for policy 0, policy_version 858821 (0.0008) [2023-12-27 11:35:19,164][20347] Updated weights for policy 0, policy_version 858831 (0.0007) [2023-12-27 11:35:19,730][20381] Updated weights for policy 1, policy_version 858112 (0.0007) [2023-12-27 11:35:19,798][20381] Updated weights for policy 1, policy_version 858122 (0.0007) [2023-12-27 11:35:19,862][20381] Updated weights for policy 1, policy_version 858132 (0.0007) [2023-12-27 11:35:19,891][20347] Updated weights for policy 0, policy_version 858841 (0.0007) [2023-12-27 11:35:19,952][20347] Updated weights for policy 0, policy_version 858851 (0.0008) [2023-12-27 11:35:20,006][20347] Updated weights for policy 0, policy_version 858861 (0.0008) [2023-12-27 11:35:20,577][20381] Updated weights for policy 1, policy_version 858142 (0.0008) [2023-12-27 11:35:20,640][20381] Updated weights for policy 1, policy_version 858152 (0.0008) [2023-12-27 11:35:20,698][20381] Updated weights for policy 1, policy_version 858162 (0.0007) [2023-12-27 11:35:20,711][20347] Updated weights for policy 0, policy_version 858871 (0.0007) [2023-12-27 11:35:20,772][20347] Updated weights for policy 0, policy_version 858881 (0.0006) [2023-12-27 11:35:20,832][20347] Updated weights for policy 0, policy_version 858891 (0.0008) [2023-12-27 11:35:21,466][20381] Updated weights for policy 1, policy_version 858172 (0.0006) [2023-12-27 11:35:21,506][20347] Updated weights for policy 0, policy_version 858901 (0.0007) [2023-12-27 11:35:21,519][20381] Updated weights for policy 1, policy_version 858182 (0.0007) [2023-12-27 11:35:21,565][20347] Updated weights for policy 0, policy_version 858911 (0.0006) [2023-12-27 11:35:21,573][20381] Updated weights for policy 1, policy_version 858192 (0.0007) [2023-12-27 11:35:21,630][20347] Updated weights for policy 0, policy_version 858921 (0.0007) [2023-12-27 11:35:22,257][20347] Updated weights for policy 0, policy_version 858931 (0.0007) [2023-12-27 11:35:22,308][20381] Updated weights for policy 1, policy_version 858202 (0.0007) [2023-12-27 11:35:22,314][20347] Updated weights for policy 0, policy_version 858941 (0.0007) [2023-12-27 11:35:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 19771.9). Total num frames: 439640064. Throughput: 0: 10148.4, 1: 10264.1. Samples: 439617024. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 11:35:22,363][19170] Avg episode reward: [(0, '931.640'), (1, '883.370')] [2023-12-27 11:35:22,373][20381] Updated weights for policy 1, policy_version 858212 (0.0008) [2023-12-27 11:35:22,384][20347] Updated weights for policy 0, policy_version 858951 (0.0007) [2023-12-27 11:35:22,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000858960_219922432.pth... [2023-12-27 11:35:22,435][20381] Updated weights for policy 1, policy_version 858222 (0.0007) [2023-12-27 11:35:22,437][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000857744_219611136.pth [2023-12-27 11:35:22,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000858232_219734016.pth... [2023-12-27 11:35:22,496][20381] Updated weights for policy 1, policy_version 858232 (0.0007) [2023-12-27 11:35:22,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000857016_219422720.pth [2023-12-27 11:35:23,089][20347] Updated weights for policy 0, policy_version 858961 (0.0007) [2023-12-27 11:35:23,145][20347] Updated weights for policy 0, policy_version 858971 (0.0007) [2023-12-27 11:35:23,200][20381] Updated weights for policy 1, policy_version 858242 (0.0008) [2023-12-27 11:35:23,203][20347] Updated weights for policy 0, policy_version 858981 (0.0005) [2023-12-27 11:35:23,253][20347] Updated weights for policy 0, policy_version 858991 (0.0005) [2023-12-27 11:35:23,257][20381] Updated weights for policy 1, policy_version 858252 (0.0007) [2023-12-27 11:35:23,320][20381] Updated weights for policy 1, policy_version 858262 (0.0007) [2023-12-27 11:35:23,841][20347] Updated weights for policy 0, policy_version 859001 (0.0005) [2023-12-27 11:35:23,894][20347] Updated weights for policy 0, policy_version 859011 (0.0007) [2023-12-27 11:35:23,950][20347] Updated weights for policy 0, policy_version 859021 (0.0005) [2023-12-27 11:35:24,001][20381] Updated weights for policy 1, policy_version 858272 (0.0009) [2023-12-27 11:35:24,054][20381] Updated weights for policy 1, policy_version 858282 (0.0008) [2023-12-27 11:35:24,104][20381] Updated weights for policy 1, policy_version 858292 (0.0007) [2023-12-27 11:35:24,552][20347] Updated weights for policy 0, policy_version 859031 (0.0009) [2023-12-27 11:35:24,617][20347] Updated weights for policy 0, policy_version 859041 (0.0010) [2023-12-27 11:35:24,674][20347] Updated weights for policy 0, policy_version 859051 (0.0010) [2023-12-27 11:35:24,837][20381] Updated weights for policy 1, policy_version 858302 (0.0008) [2023-12-27 11:35:24,890][20381] Updated weights for policy 1, policy_version 858312 (0.0006) [2023-12-27 11:35:24,947][20381] Updated weights for policy 1, policy_version 858322 (0.0006) [2023-12-27 11:35:25,328][20347] Updated weights for policy 0, policy_version 859061 (0.0008) [2023-12-27 11:35:25,374][20347] Updated weights for policy 0, policy_version 859071 (0.0007) [2023-12-27 11:35:25,423][20347] Updated weights for policy 0, policy_version 859081 (0.0008) [2023-12-27 11:35:25,629][20381] Updated weights for policy 1, policy_version 858332 (0.0008) [2023-12-27 11:35:25,682][20381] Updated weights for policy 1, policy_version 858342 (0.0008) [2023-12-27 11:35:25,734][20381] Updated weights for policy 1, policy_version 858352 (0.0008) [2023-12-27 11:35:26,086][20347] Updated weights for policy 0, policy_version 859091 (0.0007) [2023-12-27 11:35:26,137][20347] Updated weights for policy 0, policy_version 859101 (0.0007) [2023-12-27 11:35:26,183][20347] Updated weights for policy 0, policy_version 859111 (0.0005) [2023-12-27 11:35:26,491][20381] Updated weights for policy 1, policy_version 858362 (0.0007) [2023-12-27 11:35:26,536][20381] Updated weights for policy 1, policy_version 858372 (0.0007) [2023-12-27 11:35:26,585][20381] Updated weights for policy 1, policy_version 858382 (0.0008) [2023-12-27 11:35:26,638][20381] Updated weights for policy 1, policy_version 858392 (0.0007) [2023-12-27 11:35:26,745][20347] Updated weights for policy 0, policy_version 859121 (0.0005) [2023-12-27 11:35:26,791][20347] Updated weights for policy 0, policy_version 859131 (0.0006) [2023-12-27 11:35:26,851][20347] Updated weights for policy 0, policy_version 859141 (0.0005) [2023-12-27 11:35:26,904][20347] Updated weights for policy 0, policy_version 859151 (0.0007) [2023-12-27 11:35:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 19799.6). Total num frames: 439746560. Throughput: 0: 10223.2, 1: 10190.8. Samples: 439738816. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:27,363][19170] Avg episode reward: [(0, '951.310'), (1, '892.950')] [2023-12-27 11:35:27,384][20381] Updated weights for policy 1, policy_version 858402 (0.0008) [2023-12-27 11:35:27,441][20381] Updated weights for policy 1, policy_version 858412 (0.0007) [2023-12-27 11:35:27,502][20381] Updated weights for policy 1, policy_version 858422 (0.0008) [2023-12-27 11:35:27,520][20347] Updated weights for policy 0, policy_version 859161 (0.0006) [2023-12-27 11:35:27,590][20347] Updated weights for policy 0, policy_version 859171 (0.0006) [2023-12-27 11:35:27,649][20347] Updated weights for policy 0, policy_version 859181 (0.0006) [2023-12-27 11:35:28,211][20381] Updated weights for policy 1, policy_version 858432 (0.0006) [2023-12-27 11:35:28,213][20347] Updated weights for policy 0, policy_version 859191 (0.0006) [2023-12-27 11:35:28,256][20381] Updated weights for policy 1, policy_version 858442 (0.0007) [2023-12-27 11:35:28,270][20347] Updated weights for policy 0, policy_version 859201 (0.0007) [2023-12-27 11:35:28,298][20381] Updated weights for policy 1, policy_version 858452 (0.0006) [2023-12-27 11:35:28,327][20347] Updated weights for policy 0, policy_version 859211 (0.0007) [2023-12-27 11:35:29,002][20347] Updated weights for policy 0, policy_version 859221 (0.0007) [2023-12-27 11:35:29,014][20381] Updated weights for policy 1, policy_version 858462 (0.0007) [2023-12-27 11:35:29,049][20347] Updated weights for policy 0, policy_version 859231 (0.0008) [2023-12-27 11:35:29,068][20381] Updated weights for policy 1, policy_version 858472 (0.0006) [2023-12-27 11:35:29,099][20347] Updated weights for policy 0, policy_version 859241 (0.0006) [2023-12-27 11:35:29,125][20381] Updated weights for policy 1, policy_version 858482 (0.0007) [2023-12-27 11:35:29,789][20347] Updated weights for policy 0, policy_version 859251 (0.0006) [2023-12-27 11:35:29,847][20381] Updated weights for policy 1, policy_version 858492 (0.0007) [2023-12-27 11:35:29,857][20347] Updated weights for policy 0, policy_version 859261 (0.0006) [2023-12-27 11:35:29,908][20381] Updated weights for policy 1, policy_version 858502 (0.0007) [2023-12-27 11:35:29,911][20347] Updated weights for policy 0, policy_version 859271 (0.0005) [2023-12-27 11:35:29,972][20381] Updated weights for policy 1, policy_version 858512 (0.0008) [2023-12-27 11:35:30,592][20347] Updated weights for policy 0, policy_version 859281 (0.0008) [2023-12-27 11:35:30,634][20381] Updated weights for policy 1, policy_version 858522 (0.0007) [2023-12-27 11:35:30,655][20347] Updated weights for policy 0, policy_version 859291 (0.0008) [2023-12-27 11:35:30,684][20381] Updated weights for policy 1, policy_version 858532 (0.0006) [2023-12-27 11:35:30,709][20347] Updated weights for policy 0, policy_version 859301 (0.0008) [2023-12-27 11:35:30,732][20381] Updated weights for policy 1, policy_version 858542 (0.0008) [2023-12-27 11:35:30,763][20347] Updated weights for policy 0, policy_version 859311 (0.0007) [2023-12-27 11:35:30,786][20381] Updated weights for policy 1, policy_version 858552 (0.0006) [2023-12-27 11:35:31,436][20381] Updated weights for policy 1, policy_version 858562 (0.0007) [2023-12-27 11:35:31,485][20381] Updated weights for policy 1, policy_version 858572 (0.0006) [2023-12-27 11:35:31,489][20347] Updated weights for policy 0, policy_version 859321 (0.0008) [2023-12-27 11:35:31,526][20292] KL-divergence is very high: 127.7134 [2023-12-27 11:35:31,533][20381] Updated weights for policy 1, policy_version 858582 (0.0006) [2023-12-27 11:35:31,536][20347] Updated weights for policy 0, policy_version 859331 (0.0008) [2023-12-27 11:35:31,590][20347] Updated weights for policy 0, policy_version 859341 (0.0008) [2023-12-27 11:35:32,212][20381] Updated weights for policy 1, policy_version 858592 (0.0007) [2023-12-27 11:35:32,270][20381] Updated weights for policy 1, policy_version 858602 (0.0008) [2023-12-27 11:35:32,325][20381] Updated weights for policy 1, policy_version 858612 (0.0008) [2023-12-27 11:35:32,325][20347] Updated weights for policy 0, policy_version 859351 (0.0007) [2023-12-27 11:35:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 19827.4). Total num frames: 439853056. Throughput: 0: 10281.6, 1: 10129.4. Samples: 439861352. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:32,363][19170] Avg episode reward: [(0, '951.280'), (1, '873.660')] [2023-12-27 11:35:32,390][20347] Updated weights for policy 0, policy_version 859361 (0.0007) [2023-12-27 11:35:32,443][20347] Updated weights for policy 0, policy_version 859371 (0.0008) [2023-12-27 11:35:32,949][20381] Updated weights for policy 1, policy_version 858622 (0.0008) [2023-12-27 11:35:33,008][20381] Updated weights for policy 1, policy_version 858632 (0.0008) [2023-12-27 11:35:33,076][20381] Updated weights for policy 1, policy_version 858642 (0.0008) [2023-12-27 11:35:33,128][20347] Updated weights for policy 0, policy_version 859381 (0.0008) [2023-12-27 11:35:33,186][20347] Updated weights for policy 0, policy_version 859391 (0.0008) [2023-12-27 11:35:33,235][20347] Updated weights for policy 0, policy_version 859401 (0.0009) [2023-12-27 11:35:33,705][20381] Updated weights for policy 1, policy_version 858652 (0.0007) [2023-12-27 11:35:33,756][20381] Updated weights for policy 1, policy_version 858662 (0.0005) [2023-12-27 11:35:33,811][20381] Updated weights for policy 1, policy_version 858672 (0.0005) [2023-12-27 11:35:33,898][20347] Updated weights for policy 0, policy_version 859411 (0.0008) [2023-12-27 11:35:33,956][20347] Updated weights for policy 0, policy_version 859421 (0.0005) [2023-12-27 11:35:34,006][20347] Updated weights for policy 0, policy_version 859431 (0.0005) [2023-12-27 11:35:34,412][20381] Updated weights for policy 1, policy_version 858682 (0.0006) [2023-12-27 11:35:34,475][20381] Updated weights for policy 1, policy_version 858692 (0.0008) [2023-12-27 11:35:34,537][20381] Updated weights for policy 1, policy_version 858702 (0.0008) [2023-12-27 11:35:34,595][20381] Updated weights for policy 1, policy_version 858712 (0.0008) [2023-12-27 11:35:34,665][20347] Updated weights for policy 0, policy_version 859441 (0.0005) [2023-12-27 11:35:34,719][20347] Updated weights for policy 0, policy_version 859451 (0.0005) [2023-12-27 11:35:34,782][20347] Updated weights for policy 0, policy_version 859461 (0.0006) [2023-12-27 11:35:34,851][20347] Updated weights for policy 0, policy_version 859471 (0.0006) [2023-12-27 11:35:35,223][20381] Updated weights for policy 1, policy_version 858722 (0.0007) [2023-12-27 11:35:35,267][20381] Updated weights for policy 1, policy_version 858732 (0.0008) [2023-12-27 11:35:35,313][20381] Updated weights for policy 1, policy_version 858742 (0.0008) [2023-12-27 11:35:35,418][20347] Updated weights for policy 0, policy_version 859481 (0.0008) [2023-12-27 11:35:35,469][20347] Updated weights for policy 0, policy_version 859491 (0.0008) [2023-12-27 11:35:35,517][20347] Updated weights for policy 0, policy_version 859501 (0.0008) [2023-12-27 11:35:35,989][20381] Updated weights for policy 1, policy_version 858752 (0.0006) [2023-12-27 11:35:36,035][20381] Updated weights for policy 1, policy_version 858762 (0.0005) [2023-12-27 11:35:36,086][20381] Updated weights for policy 1, policy_version 858772 (0.0005) [2023-12-27 11:35:36,176][20347] Updated weights for policy 0, policy_version 859511 (0.0008) [2023-12-27 11:35:36,233][20347] Updated weights for policy 0, policy_version 859521 (0.0008) [2023-12-27 11:35:36,296][20347] Updated weights for policy 0, policy_version 859531 (0.0007) [2023-12-27 11:35:36,792][20381] Updated weights for policy 1, policy_version 858782 (0.0008) [2023-12-27 11:35:36,859][20381] Updated weights for policy 1, policy_version 858792 (0.0008) [2023-12-27 11:35:36,919][20381] Updated weights for policy 1, policy_version 858802 (0.0008) [2023-12-27 11:35:36,937][20347] Updated weights for policy 0, policy_version 859541 (0.0007) [2023-12-27 11:35:36,996][20347] Updated weights for policy 0, policy_version 859551 (0.0006) [2023-12-27 11:35:37,050][20347] Updated weights for policy 0, policy_version 859561 (0.0006) [2023-12-27 11:35:37,363][19170] Fps is (10 sec: 21298.3, 60 sec: 20479.8, 300 sec: 19882.9). Total num frames: 439959552. Throughput: 0: 10309.7, 1: 10153.7. Samples: 439925136. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:37,364][19170] Avg episode reward: [(0, '980.500'), (1, '844.540')] [2023-12-27 11:35:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000858808_219881472.pth... [2023-12-27 11:35:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000859568_220078080.pth... [2023-12-27 11:35:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000857624_219578368.pth [2023-12-27 11:35:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000858352_219766784.pth [2023-12-27 11:35:37,531][20381] Updated weights for policy 1, policy_version 858812 (0.0005) [2023-12-27 11:35:37,594][20381] Updated weights for policy 1, policy_version 858822 (0.0005) [2023-12-27 11:35:37,596][20347] Updated weights for policy 0, policy_version 859571 (0.0006) [2023-12-27 11:35:37,646][20381] Updated weights for policy 1, policy_version 858832 (0.0009) [2023-12-27 11:35:37,647][20347] Updated weights for policy 0, policy_version 859581 (0.0006) [2023-12-27 11:35:37,697][20347] Updated weights for policy 0, policy_version 859591 (0.0006) [2023-12-27 11:35:38,279][20381] Updated weights for policy 1, policy_version 858842 (0.0011) [2023-12-27 11:35:38,326][20381] Updated weights for policy 1, policy_version 858852 (0.0007) [2023-12-27 11:35:38,382][20381] Updated weights for policy 1, policy_version 858862 (0.0007) [2023-12-27 11:35:38,436][20381] Updated weights for policy 1, policy_version 858872 (0.0007) [2023-12-27 11:35:38,451][20347] Updated weights for policy 0, policy_version 859601 (0.0007) [2023-12-27 11:35:38,511][20347] Updated weights for policy 0, policy_version 859611 (0.0008) [2023-12-27 11:35:38,570][20347] Updated weights for policy 0, policy_version 859621 (0.0008) [2023-12-27 11:35:38,630][20347] Updated weights for policy 0, policy_version 859631 (0.0007) [2023-12-27 11:35:39,184][20381] Updated weights for policy 1, policy_version 858882 (0.0008) [2023-12-27 11:35:39,251][20381] Updated weights for policy 1, policy_version 858892 (0.0007) [2023-12-27 11:35:39,296][20347] Updated weights for policy 0, policy_version 859641 (0.0009) [2023-12-27 11:35:39,316][20381] Updated weights for policy 1, policy_version 858902 (0.0007) [2023-12-27 11:35:39,354][20347] Updated weights for policy 0, policy_version 859651 (0.0008) [2023-12-27 11:35:39,426][20347] Updated weights for policy 0, policy_version 859661 (0.0008) [2023-12-27 11:35:40,038][20381] Updated weights for policy 1, policy_version 858912 (0.0006) [2023-12-27 11:35:40,099][20381] Updated weights for policy 1, policy_version 858922 (0.0008) [2023-12-27 11:35:40,156][20381] Updated weights for policy 1, policy_version 858932 (0.0008) [2023-12-27 11:35:40,171][20347] Updated weights for policy 0, policy_version 859671 (0.0007) [2023-12-27 11:35:40,225][20347] Updated weights for policy 0, policy_version 859681 (0.0008) [2023-12-27 11:35:40,282][20347] Updated weights for policy 0, policy_version 859691 (0.0007) [2023-12-27 11:35:40,877][20381] Updated weights for policy 1, policy_version 858942 (0.0010) [2023-12-27 11:35:40,932][20381] Updated weights for policy 1, policy_version 858952 (0.0010) [2023-12-27 11:35:40,994][20381] Updated weights for policy 1, policy_version 858962 (0.0011) [2023-12-27 11:35:41,016][20347] Updated weights for policy 0, policy_version 859701 (0.0008) [2023-12-27 11:35:41,084][20347] Updated weights for policy 0, policy_version 859711 (0.0007) [2023-12-27 11:35:41,178][20347] Updated weights for policy 0, policy_version 859721 (0.0007) [2023-12-27 11:35:41,761][20381] Updated weights for policy 1, policy_version 858972 (0.0010) [2023-12-27 11:35:41,807][20347] Updated weights for policy 0, policy_version 859731 (0.0007) [2023-12-27 11:35:41,814][20381] Updated weights for policy 1, policy_version 858982 (0.0007) [2023-12-27 11:35:41,868][20347] Updated weights for policy 0, policy_version 859741 (0.0009) [2023-12-27 11:35:41,870][20381] Updated weights for policy 1, policy_version 858992 (0.0007) [2023-12-27 11:35:41,929][20347] Updated weights for policy 0, policy_version 859751 (0.0007) [2023-12-27 11:35:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 19882.9). Total num frames: 440057856. Throughput: 0: 10312.2, 1: 10100.1. Samples: 440048392. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:42,363][19170] Avg episode reward: [(0, '980.510'), (1, '883.560')] [2023-12-27 11:35:42,585][20347] Updated weights for policy 0, policy_version 859761 (0.0005) [2023-12-27 11:35:42,600][20381] Updated weights for policy 1, policy_version 859002 (0.0009) [2023-12-27 11:35:42,643][20347] Updated weights for policy 0, policy_version 859771 (0.0007) [2023-12-27 11:35:42,659][20381] Updated weights for policy 1, policy_version 859012 (0.0010) [2023-12-27 11:35:42,705][20347] Updated weights for policy 0, policy_version 859781 (0.0007) [2023-12-27 11:35:42,709][20381] Updated weights for policy 1, policy_version 859022 (0.0010) [2023-12-27 11:35:42,762][20347] Updated weights for policy 0, policy_version 859791 (0.0007) [2023-12-27 11:35:42,763][20381] Updated weights for policy 1, policy_version 859032 (0.0007) [2023-12-27 11:35:43,418][20381] Updated weights for policy 1, policy_version 859042 (0.0008) [2023-12-27 11:35:43,473][20381] Updated weights for policy 1, policy_version 859052 (0.0005) [2023-12-27 11:35:43,482][20347] Updated weights for policy 0, policy_version 859801 (0.0007) [2023-12-27 11:35:43,532][20381] Updated weights for policy 1, policy_version 859062 (0.0007) [2023-12-27 11:35:43,538][20347] Updated weights for policy 0, policy_version 859811 (0.0007) [2023-12-27 11:35:43,590][20347] Updated weights for policy 0, policy_version 859821 (0.0007) [2023-12-27 11:35:44,151][20381] Updated weights for policy 1, policy_version 859072 (0.0008) [2023-12-27 11:35:44,222][20381] Updated weights for policy 1, policy_version 859082 (0.0006) [2023-12-27 11:35:44,283][20381] Updated weights for policy 1, policy_version 859092 (0.0006) [2023-12-27 11:35:44,298][20347] Updated weights for policy 0, policy_version 859831 (0.0009) [2023-12-27 11:35:44,351][20347] Updated weights for policy 0, policy_version 859841 (0.0007) [2023-12-27 11:35:44,398][20347] Updated weights for policy 0, policy_version 859851 (0.0006) [2023-12-27 11:35:44,915][20381] Updated weights for policy 1, policy_version 859102 (0.0007) [2023-12-27 11:35:44,980][20381] Updated weights for policy 1, policy_version 859112 (0.0006) [2023-12-27 11:35:45,037][20347] Updated weights for policy 0, policy_version 859861 (0.0006) [2023-12-27 11:35:45,044][20381] Updated weights for policy 1, policy_version 859122 (0.0007) [2023-12-27 11:35:45,099][20347] Updated weights for policy 0, policy_version 859871 (0.0006) [2023-12-27 11:35:45,166][20347] Updated weights for policy 0, policy_version 859881 (0.0007) [2023-12-27 11:35:45,734][20381] Updated weights for policy 1, policy_version 859132 (0.0007) [2023-12-27 11:35:45,737][20347] Updated weights for policy 0, policy_version 859891 (0.0007) [2023-12-27 11:35:45,791][20381] Updated weights for policy 1, policy_version 859142 (0.0006) [2023-12-27 11:35:45,797][20347] Updated weights for policy 0, policy_version 859901 (0.0007) [2023-12-27 11:35:45,845][20381] Updated weights for policy 1, policy_version 859152 (0.0006) [2023-12-27 11:35:45,847][20347] Updated weights for policy 0, policy_version 859911 (0.0005) [2023-12-27 11:35:46,484][20347] Updated weights for policy 0, policy_version 859921 (0.0005) [2023-12-27 11:35:46,540][20347] Updated weights for policy 0, policy_version 859931 (0.0006) [2023-12-27 11:35:46,542][20381] Updated weights for policy 1, policy_version 859162 (0.0006) [2023-12-27 11:35:46,590][20381] Updated weights for policy 1, policy_version 859172 (0.0008) [2023-12-27 11:35:46,604][20347] Updated weights for policy 0, policy_version 859941 (0.0005) [2023-12-27 11:35:46,639][20381] Updated weights for policy 1, policy_version 859182 (0.0007) [2023-12-27 11:35:46,664][20347] Updated weights for policy 0, policy_version 859951 (0.0005) [2023-12-27 11:35:46,684][20381] Updated weights for policy 1, policy_version 859192 (0.0007) [2023-12-27 11:35:47,254][20347] Updated weights for policy 0, policy_version 859961 (0.0007) [2023-12-27 11:35:47,310][20347] Updated weights for policy 0, policy_version 859971 (0.0006) [2023-12-27 11:35:47,362][19170] Fps is (10 sec: 19661.9, 60 sec: 20343.6, 300 sec: 19910.8). Total num frames: 440156160. Throughput: 0: 10390.7, 1: 10077.4. Samples: 440172492. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:47,363][19170] Avg episode reward: [(0, '990.280'), (1, '893.250')] [2023-12-27 11:35:47,366][20347] Updated weights for policy 0, policy_version 859981 (0.0006) [2023-12-27 11:35:47,415][20381] Updated weights for policy 1, policy_version 859202 (0.0008) [2023-12-27 11:35:47,475][20381] Updated weights for policy 1, policy_version 859212 (0.0006) [2023-12-27 11:35:47,538][20381] Updated weights for policy 1, policy_version 859222 (0.0005) [2023-12-27 11:35:48,027][20347] Updated weights for policy 0, policy_version 859991 (0.0007) [2023-12-27 11:35:48,070][20347] Updated weights for policy 0, policy_version 860001 (0.0007) [2023-12-27 11:35:48,114][20347] Updated weights for policy 0, policy_version 860011 (0.0007) [2023-12-27 11:35:48,161][20381] Updated weights for policy 1, policy_version 859232 (0.0010) [2023-12-27 11:35:48,214][20381] Updated weights for policy 1, policy_version 859242 (0.0006) [2023-12-27 11:35:48,263][20381] Updated weights for policy 1, policy_version 859252 (0.0005) [2023-12-27 11:35:48,840][20347] Updated weights for policy 0, policy_version 860021 (0.0008) [2023-12-27 11:35:48,901][20347] Updated weights for policy 0, policy_version 860031 (0.0007) [2023-12-27 11:35:48,919][20381] Updated weights for policy 1, policy_version 859262 (0.0008) [2023-12-27 11:35:48,954][20347] Updated weights for policy 0, policy_version 860041 (0.0008) [2023-12-27 11:35:48,968][20381] Updated weights for policy 1, policy_version 859272 (0.0011) [2023-12-27 11:35:49,018][20381] Updated weights for policy 1, policy_version 859282 (0.0009) [2023-12-27 11:35:49,697][20347] Updated weights for policy 0, policy_version 860051 (0.0008) [2023-12-27 11:35:49,753][20381] Updated weights for policy 1, policy_version 859292 (0.0006) [2023-12-27 11:35:49,759][20347] Updated weights for policy 0, policy_version 860061 (0.0008) [2023-12-27 11:35:49,805][20381] Updated weights for policy 1, policy_version 859302 (0.0007) [2023-12-27 11:35:49,823][20347] Updated weights for policy 0, policy_version 860071 (0.0006) [2023-12-27 11:35:49,871][20381] Updated weights for policy 1, policy_version 859312 (0.0010) [2023-12-27 11:35:50,533][20347] Updated weights for policy 0, policy_version 860081 (0.0007) [2023-12-27 11:35:50,537][20381] Updated weights for policy 1, policy_version 859322 (0.0008) [2023-12-27 11:35:50,592][20347] Updated weights for policy 0, policy_version 860091 (0.0008) [2023-12-27 11:35:50,599][20381] Updated weights for policy 1, policy_version 859332 (0.0009) [2023-12-27 11:35:50,644][20347] Updated weights for policy 0, policy_version 860101 (0.0008) [2023-12-27 11:35:50,657][20381] Updated weights for policy 1, policy_version 859342 (0.0011) [2023-12-27 11:35:50,692][20347] Updated weights for policy 0, policy_version 860111 (0.0008) [2023-12-27 11:35:50,712][20381] Updated weights for policy 1, policy_version 859352 (0.0011) [2023-12-27 11:35:51,410][20381] Updated weights for policy 1, policy_version 859362 (0.0008) [2023-12-27 11:35:51,432][20347] Updated weights for policy 0, policy_version 860121 (0.0008) [2023-12-27 11:35:51,475][20381] Updated weights for policy 1, policy_version 859372 (0.0007) [2023-12-27 11:35:51,491][20347] Updated weights for policy 0, policy_version 860131 (0.0008) [2023-12-27 11:35:51,540][20381] Updated weights for policy 1, policy_version 859382 (0.0009) [2023-12-27 11:35:51,550][20347] Updated weights for policy 0, policy_version 860141 (0.0007) [2023-12-27 11:35:52,251][20381] Updated weights for policy 1, policy_version 859392 (0.0006) [2023-12-27 11:35:52,276][20347] Updated weights for policy 0, policy_version 860151 (0.0007) [2023-12-27 11:35:52,312][20381] Updated weights for policy 1, policy_version 859402 (0.0007) [2023-12-27 11:35:52,337][20347] Updated weights for policy 0, policy_version 860161 (0.0007) [2023-12-27 11:35:52,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 19910.7). Total num frames: 440254464. Throughput: 0: 10315.0, 1: 10105.3. Samples: 440232928. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:52,363][19170] Avg episode reward: [(0, '980.540'), (1, '834.880')] [2023-12-27 11:35:52,377][20381] Updated weights for policy 1, policy_version 859412 (0.0007) [2023-12-27 11:35:52,395][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000859416_220037120.pth... [2023-12-27 11:35:52,398][20347] Updated weights for policy 0, policy_version 860171 (0.0007) [2023-12-27 11:35:52,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000858232_219734016.pth [2023-12-27 11:35:52,427][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000860176_220233728.pth... [2023-12-27 11:35:52,430][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000858960_219922432.pth [2023-12-27 11:35:53,076][20381] Updated weights for policy 1, policy_version 859422 (0.0006) [2023-12-27 11:35:53,085][20347] Updated weights for policy 0, policy_version 860181 (0.0009) [2023-12-27 11:35:53,130][20347] Updated weights for policy 0, policy_version 860191 (0.0007) [2023-12-27 11:35:53,132][20381] Updated weights for policy 1, policy_version 859432 (0.0007) [2023-12-27 11:35:53,185][20381] Updated weights for policy 1, policy_version 859442 (0.0006) [2023-12-27 11:35:53,188][20347] Updated weights for policy 0, policy_version 860201 (0.0007) [2023-12-27 11:35:53,819][20381] Updated weights for policy 1, policy_version 859452 (0.0007) [2023-12-27 11:35:53,855][20347] Updated weights for policy 0, policy_version 860211 (0.0006) [2023-12-27 11:35:53,870][20381] Updated weights for policy 1, policy_version 859462 (0.0007) [2023-12-27 11:35:53,914][20347] Updated weights for policy 0, policy_version 860221 (0.0006) [2023-12-27 11:35:53,929][20381] Updated weights for policy 1, policy_version 859472 (0.0008) [2023-12-27 11:35:53,968][20347] Updated weights for policy 0, policy_version 860231 (0.0010) [2023-12-27 11:35:54,631][20381] Updated weights for policy 1, policy_version 859482 (0.0010) [2023-12-27 11:35:54,652][20347] Updated weights for policy 0, policy_version 860241 (0.0011) [2023-12-27 11:35:54,691][20381] Updated weights for policy 1, policy_version 859492 (0.0011) [2023-12-27 11:35:54,701][20347] Updated weights for policy 0, policy_version 860251 (0.0010) [2023-12-27 11:35:54,744][20381] Updated weights for policy 1, policy_version 859502 (0.0007) [2023-12-27 11:35:54,754][20347] Updated weights for policy 0, policy_version 860261 (0.0010) [2023-12-27 11:35:54,800][20381] Updated weights for policy 1, policy_version 859512 (0.0009) [2023-12-27 11:35:54,811][20347] Updated weights for policy 0, policy_version 860271 (0.0010) [2023-12-27 11:35:55,485][20347] Updated weights for policy 0, policy_version 860281 (0.0007) [2023-12-27 11:35:55,513][20381] Updated weights for policy 1, policy_version 859522 (0.0011) [2023-12-27 11:35:55,549][20347] Updated weights for policy 0, policy_version 860291 (0.0006) [2023-12-27 11:35:55,571][20381] Updated weights for policy 1, policy_version 859532 (0.0011) [2023-12-27 11:35:55,615][20347] Updated weights for policy 0, policy_version 860301 (0.0007) [2023-12-27 11:35:55,632][20381] Updated weights for policy 1, policy_version 859542 (0.0008) [2023-12-27 11:35:56,192][20347] Updated weights for policy 0, policy_version 860311 (0.0006) [2023-12-27 11:35:56,250][20347] Updated weights for policy 0, policy_version 860321 (0.0006) [2023-12-27 11:35:56,303][20347] Updated weights for policy 0, policy_version 860331 (0.0006) [2023-12-27 11:35:56,380][20381] Updated weights for policy 1, policy_version 859552 (0.0008) [2023-12-27 11:35:56,442][20381] Updated weights for policy 1, policy_version 859562 (0.0008) [2023-12-27 11:35:56,503][20381] Updated weights for policy 1, policy_version 859572 (0.0008) [2023-12-27 11:35:56,908][20347] Updated weights for policy 0, policy_version 860341 (0.0006) [2023-12-27 11:35:56,967][20347] Updated weights for policy 0, policy_version 860351 (0.0007) [2023-12-27 11:35:57,027][20347] Updated weights for policy 0, policy_version 860361 (0.0007) [2023-12-27 11:35:57,218][20381] Updated weights for policy 1, policy_version 859582 (0.0009) [2023-12-27 11:35:57,278][20381] Updated weights for policy 1, policy_version 859592 (0.0008) [2023-12-27 11:35:57,336][20381] Updated weights for policy 1, policy_version 859602 (0.0008) [2023-12-27 11:35:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 19966.3). Total num frames: 440360960. Throughput: 0: 10333.2, 1: 10061.7. Samples: 440353316. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:35:57,363][19170] Avg episode reward: [(0, '980.540'), (1, '834.700')] [2023-12-27 11:35:57,681][20347] Updated weights for policy 0, policy_version 860371 (0.0008) [2023-12-27 11:35:57,740][20347] Updated weights for policy 0, policy_version 860381 (0.0006) [2023-12-27 11:35:57,802][20347] Updated weights for policy 0, policy_version 860391 (0.0007) [2023-12-27 11:35:58,038][20381] Updated weights for policy 1, policy_version 859612 (0.0009) [2023-12-27 11:35:58,093][20381] Updated weights for policy 1, policy_version 859622 (0.0010) [2023-12-27 11:35:58,158][20381] Updated weights for policy 1, policy_version 859632 (0.0006) [2023-12-27 11:35:58,545][20347] Updated weights for policy 0, policy_version 860401 (0.0008) [2023-12-27 11:35:58,616][20347] Updated weights for policy 0, policy_version 860411 (0.0009) [2023-12-27 11:35:58,682][20347] Updated weights for policy 0, policy_version 860421 (0.0008) [2023-12-27 11:35:58,748][20347] Updated weights for policy 0, policy_version 860431 (0.0011) [2023-12-27 11:35:58,946][20381] Updated weights for policy 1, policy_version 859642 (0.0008) [2023-12-27 11:35:59,020][20381] Updated weights for policy 1, policy_version 859652 (0.0008) [2023-12-27 11:35:59,079][20381] Updated weights for policy 1, policy_version 859662 (0.0007) [2023-12-27 11:35:59,148][20381] Updated weights for policy 1, policy_version 859672 (0.0008) [2023-12-27 11:35:59,669][20347] Updated weights for policy 0, policy_version 860441 (0.0008) [2023-12-27 11:35:59,731][20347] Updated weights for policy 0, policy_version 860451 (0.0008) [2023-12-27 11:35:59,787][20347] Updated weights for policy 0, policy_version 860461 (0.0008) [2023-12-27 11:35:59,935][20381] Updated weights for policy 1, policy_version 859682 (0.0008) [2023-12-27 11:35:59,996][20381] Updated weights for policy 1, policy_version 859692 (0.0008) [2023-12-27 11:36:00,054][20381] Updated weights for policy 1, policy_version 859702 (0.0008) [2023-12-27 11:36:00,482][20347] Updated weights for policy 0, policy_version 860471 (0.0007) [2023-12-27 11:36:00,536][20347] Updated weights for policy 0, policy_version 860481 (0.0007) [2023-12-27 11:36:00,597][20347] Updated weights for policy 0, policy_version 860491 (0.0007) [2023-12-27 11:36:00,723][20381] Updated weights for policy 1, policy_version 859712 (0.0008) [2023-12-27 11:36:00,769][20381] Updated weights for policy 1, policy_version 859722 (0.0005) [2023-12-27 11:36:00,823][20381] Updated weights for policy 1, policy_version 859732 (0.0005) [2023-12-27 11:36:01,336][20347] Updated weights for policy 0, policy_version 860501 (0.0008) [2023-12-27 11:36:01,403][20347] Updated weights for policy 0, policy_version 860511 (0.0009) [2023-12-27 11:36:01,462][20347] Updated weights for policy 0, policy_version 860521 (0.0008) [2023-12-27 11:36:01,487][20381] Updated weights for policy 1, policy_version 859742 (0.0005) [2023-12-27 11:36:01,532][20381] Updated weights for policy 1, policy_version 859752 (0.0005) [2023-12-27 11:36:01,587][20381] Updated weights for policy 1, policy_version 859762 (0.0005) [2023-12-27 11:36:02,192][20347] Updated weights for policy 0, policy_version 860531 (0.0008) [2023-12-27 11:36:02,247][20347] Updated weights for policy 0, policy_version 860541 (0.0007) [2023-12-27 11:36:02,275][20381] Updated weights for policy 1, policy_version 859772 (0.0006) [2023-12-27 11:36:02,305][20347] Updated weights for policy 0, policy_version 860551 (0.0009) [2023-12-27 11:36:02,333][20381] Updated weights for policy 1, policy_version 859782 (0.0009) [2023-12-27 11:36:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 19938.5). Total num frames: 440459264. Throughput: 0: 10269.2, 1: 10003.9. Samples: 440469044. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.200')] [2023-12-27 11:36:02,393][20381] Updated weights for policy 1, policy_version 859792 (0.0008) [2023-12-27 11:36:02,959][20347] Updated weights for policy 0, policy_version 860561 (0.0008) [2023-12-27 11:36:03,018][20347] Updated weights for policy 0, policy_version 860571 (0.0006) [2023-12-27 11:36:03,075][20347] Updated weights for policy 0, policy_version 860581 (0.0006) [2023-12-27 11:36:03,131][20347] Updated weights for policy 0, policy_version 860591 (0.0007) [2023-12-27 11:36:03,156][20381] Updated weights for policy 1, policy_version 859802 (0.0008) [2023-12-27 11:36:03,214][20381] Updated weights for policy 1, policy_version 859812 (0.0006) [2023-12-27 11:36:03,269][20381] Updated weights for policy 1, policy_version 859822 (0.0006) [2023-12-27 11:36:03,320][20381] Updated weights for policy 1, policy_version 859832 (0.0007) [2023-12-27 11:36:03,831][20347] Updated weights for policy 0, policy_version 860601 (0.0006) [2023-12-27 11:36:03,894][20347] Updated weights for policy 0, policy_version 860611 (0.0008) [2023-12-27 11:36:03,923][20381] Updated weights for policy 1, policy_version 859842 (0.0011) [2023-12-27 11:36:03,951][20347] Updated weights for policy 0, policy_version 860621 (0.0008) [2023-12-27 11:36:03,973][20381] Updated weights for policy 1, policy_version 859852 (0.0011) [2023-12-27 11:36:04,026][20381] Updated weights for policy 1, policy_version 859862 (0.0011) [2023-12-27 11:36:04,702][20347] Updated weights for policy 0, policy_version 860631 (0.0008) [2023-12-27 11:36:04,761][20347] Updated weights for policy 0, policy_version 860641 (0.0008) [2023-12-27 11:36:04,772][20381] Updated weights for policy 1, policy_version 859872 (0.0008) [2023-12-27 11:36:04,816][20347] Updated weights for policy 0, policy_version 860651 (0.0007) [2023-12-27 11:36:04,831][20381] Updated weights for policy 1, policy_version 859882 (0.0009) [2023-12-27 11:36:04,890][20381] Updated weights for policy 1, policy_version 859892 (0.0010) [2023-12-27 11:36:05,443][20347] Updated weights for policy 0, policy_version 860661 (0.0006) [2023-12-27 11:36:05,497][20347] Updated weights for policy 0, policy_version 860671 (0.0006) [2023-12-27 11:36:05,555][20347] Updated weights for policy 0, policy_version 860681 (0.0008) [2023-12-27 11:36:05,614][20381] Updated weights for policy 1, policy_version 859902 (0.0009) [2023-12-27 11:36:05,667][20381] Updated weights for policy 1, policy_version 859912 (0.0008) [2023-12-27 11:36:05,726][20381] Updated weights for policy 1, policy_version 859922 (0.0008) [2023-12-27 11:36:06,189][20347] Updated weights for policy 0, policy_version 860691 (0.0007) [2023-12-27 11:36:06,246][20347] Updated weights for policy 0, policy_version 860701 (0.0008) [2023-12-27 11:36:06,300][20347] Updated weights for policy 0, policy_version 860711 (0.0008) [2023-12-27 11:36:06,449][20381] Updated weights for policy 1, policy_version 859932 (0.0008) [2023-12-27 11:36:06,505][20381] Updated weights for policy 1, policy_version 859942 (0.0008) [2023-12-27 11:36:06,566][20381] Updated weights for policy 1, policy_version 859952 (0.0008) [2023-12-27 11:36:07,019][20347] Updated weights for policy 0, policy_version 860721 (0.0006) [2023-12-27 11:36:07,080][20347] Updated weights for policy 0, policy_version 860731 (0.0006) [2023-12-27 11:36:07,132][20347] Updated weights for policy 0, policy_version 860741 (0.0007) [2023-12-27 11:36:07,193][20347] Updated weights for policy 0, policy_version 860751 (0.0007) [2023-12-27 11:36:07,219][20381] Updated weights for policy 1, policy_version 859962 (0.0007) [2023-12-27 11:36:07,284][20381] Updated weights for policy 1, policy_version 859972 (0.0008) [2023-12-27 11:36:07,341][20381] Updated weights for policy 1, policy_version 859982 (0.0008) [2023-12-27 11:36:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 19966.3). Total num frames: 440557568. Throughput: 0: 10241.7, 1: 10020.1. Samples: 440528804. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:07,363][19170] Avg episode reward: [(0, '990.290'), (1, '883.080')] [2023-12-27 11:36:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000860752_220381184.pth... [2023-12-27 11:36:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000859568_220078080.pth [2023-12-27 11:36:07,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000859992_220184576.pth... [2023-12-27 11:36:07,396][20381] Updated weights for policy 1, policy_version 859992 (0.0007) [2023-12-27 11:36:07,400][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000858808_219881472.pth [2023-12-27 11:36:07,821][20347] Updated weights for policy 0, policy_version 860761 (0.0010) [2023-12-27 11:36:07,878][20347] Updated weights for policy 0, policy_version 860771 (0.0010) [2023-12-27 11:36:07,932][20347] Updated weights for policy 0, policy_version 860781 (0.0006) [2023-12-27 11:36:08,123][20381] Updated weights for policy 1, policy_version 860002 (0.0008) [2023-12-27 11:36:08,178][20381] Updated weights for policy 1, policy_version 860012 (0.0009) [2023-12-27 11:36:08,230][20381] Updated weights for policy 1, policy_version 860022 (0.0008) [2023-12-27 11:36:08,553][20347] Updated weights for policy 0, policy_version 860791 (0.0006) [2023-12-27 11:36:08,620][20347] Updated weights for policy 0, policy_version 860801 (0.0007) [2023-12-27 11:36:08,680][20347] Updated weights for policy 0, policy_version 860811 (0.0008) [2023-12-27 11:36:08,969][20381] Updated weights for policy 1, policy_version 860032 (0.0008) [2023-12-27 11:36:09,026][20381] Updated weights for policy 1, policy_version 860042 (0.0009) [2023-12-27 11:36:09,072][20381] Updated weights for policy 1, policy_version 860052 (0.0010) [2023-12-27 11:36:09,238][20347] Updated weights for policy 0, policy_version 860821 (0.0008) [2023-12-27 11:36:09,304][20347] Updated weights for policy 0, policy_version 860831 (0.0010) [2023-12-27 11:36:09,365][20347] Updated weights for policy 0, policy_version 860841 (0.0009) [2023-12-27 11:36:09,811][20381] Updated weights for policy 1, policy_version 860062 (0.0006) [2023-12-27 11:36:09,883][20381] Updated weights for policy 1, policy_version 860072 (0.0009) [2023-12-27 11:36:09,947][20381] Updated weights for policy 1, policy_version 860082 (0.0008) [2023-12-27 11:36:10,104][20347] Updated weights for policy 0, policy_version 860851 (0.0009) [2023-12-27 11:36:10,168][20347] Updated weights for policy 0, policy_version 860861 (0.0007) [2023-12-27 11:36:10,225][20347] Updated weights for policy 0, policy_version 860871 (0.0008) [2023-12-27 11:36:10,591][20381] Updated weights for policy 1, policy_version 860092 (0.0007) [2023-12-27 11:36:10,639][20381] Updated weights for policy 1, policy_version 860102 (0.0008) [2023-12-27 11:36:10,688][20381] Updated weights for policy 1, policy_version 860112 (0.0009) [2023-12-27 11:36:10,903][20347] Updated weights for policy 0, policy_version 860881 (0.0007) [2023-12-27 11:36:10,954][20347] Updated weights for policy 0, policy_version 860891 (0.0007) [2023-12-27 11:36:11,002][20347] Updated weights for policy 0, policy_version 860901 (0.0006) [2023-12-27 11:36:11,069][20347] Updated weights for policy 0, policy_version 860911 (0.0007) [2023-12-27 11:36:11,481][20381] Updated weights for policy 1, policy_version 860122 (0.0009) [2023-12-27 11:36:11,539][20381] Updated weights for policy 1, policy_version 860132 (0.0009) [2023-12-27 11:36:11,590][20381] Updated weights for policy 1, policy_version 860142 (0.0008) [2023-12-27 11:36:11,658][20381] Updated weights for policy 1, policy_version 860152 (0.0008) [2023-12-27 11:36:11,773][20347] Updated weights for policy 0, policy_version 860921 (0.0008) [2023-12-27 11:36:11,833][20347] Updated weights for policy 0, policy_version 860931 (0.0008) [2023-12-27 11:36:11,884][20347] Updated weights for policy 0, policy_version 860941 (0.0008) [2023-12-27 11:36:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 19966.3). Total num frames: 440655872. Throughput: 0: 10241.0, 1: 10024.0. Samples: 440650740. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:12,363][19170] Avg episode reward: [(0, '990.290'), (1, '892.760')] [2023-12-27 11:36:12,421][20381] Updated weights for policy 1, policy_version 860162 (0.0008) [2023-12-27 11:36:12,478][20381] Updated weights for policy 1, policy_version 860172 (0.0008) [2023-12-27 11:36:12,537][20381] Updated weights for policy 1, policy_version 860182 (0.0008) [2023-12-27 11:36:12,577][20347] Updated weights for policy 0, policy_version 860951 (0.0006) [2023-12-27 11:36:12,633][20347] Updated weights for policy 0, policy_version 860961 (0.0005) [2023-12-27 11:36:12,681][20347] Updated weights for policy 0, policy_version 860971 (0.0006) [2023-12-27 11:36:13,260][20381] Updated weights for policy 1, policy_version 860192 (0.0007) [2023-12-27 11:36:13,314][20347] Updated weights for policy 0, policy_version 860981 (0.0006) [2023-12-27 11:36:13,315][20381] Updated weights for policy 1, policy_version 860202 (0.0007) [2023-12-27 11:36:13,365][20381] Updated weights for policy 1, policy_version 860212 (0.0005) [2023-12-27 11:36:13,367][20347] Updated weights for policy 0, policy_version 860991 (0.0005) [2023-12-27 11:36:13,419][20347] Updated weights for policy 0, policy_version 861001 (0.0006) [2023-12-27 11:36:14,011][20381] Updated weights for policy 1, policy_version 860222 (0.0006) [2023-12-27 11:36:14,020][20347] Updated weights for policy 0, policy_version 861011 (0.0006) [2023-12-27 11:36:14,069][20381] Updated weights for policy 1, policy_version 860232 (0.0005) [2023-12-27 11:36:14,081][20347] Updated weights for policy 0, policy_version 861021 (0.0006) [2023-12-27 11:36:14,120][20381] Updated weights for policy 1, policy_version 860242 (0.0007) [2023-12-27 11:36:14,140][20347] Updated weights for policy 0, policy_version 861031 (0.0007) [2023-12-27 11:36:14,727][20381] Updated weights for policy 1, policy_version 860252 (0.0008) [2023-12-27 11:36:14,754][20347] Updated weights for policy 0, policy_version 861041 (0.0007) [2023-12-27 11:36:14,784][20381] Updated weights for policy 1, policy_version 860262 (0.0010) [2023-12-27 11:36:14,824][20347] Updated weights for policy 0, policy_version 861051 (0.0006) [2023-12-27 11:36:14,841][20381] Updated weights for policy 1, policy_version 860272 (0.0008) [2023-12-27 11:36:14,885][20347] Updated weights for policy 0, policy_version 861061 (0.0006) [2023-12-27 11:36:14,938][20347] Updated weights for policy 0, policy_version 861071 (0.0007) [2023-12-27 11:36:15,483][20381] Updated weights for policy 1, policy_version 860282 (0.0007) [2023-12-27 11:36:15,531][20381] Updated weights for policy 1, policy_version 860292 (0.0008) [2023-12-27 11:36:15,590][20381] Updated weights for policy 1, policy_version 860302 (0.0007) [2023-12-27 11:36:15,627][20347] Updated weights for policy 0, policy_version 861081 (0.0007) [2023-12-27 11:36:15,648][20381] Updated weights for policy 1, policy_version 860312 (0.0009) [2023-12-27 11:36:15,679][20347] Updated weights for policy 0, policy_version 861091 (0.0006) [2023-12-27 11:36:15,729][20347] Updated weights for policy 0, policy_version 861101 (0.0005) [2023-12-27 11:36:16,350][20381] Updated weights for policy 1, policy_version 860322 (0.0010) [2023-12-27 11:36:16,354][20347] Updated weights for policy 0, policy_version 861111 (0.0005) [2023-12-27 11:36:16,400][20381] Updated weights for policy 1, policy_version 860332 (0.0008) [2023-12-27 11:36:16,414][20347] Updated weights for policy 0, policy_version 861121 (0.0005) [2023-12-27 11:36:16,447][20381] Updated weights for policy 1, policy_version 860342 (0.0008) [2023-12-27 11:36:16,471][20347] Updated weights for policy 0, policy_version 861131 (0.0007) [2023-12-27 11:36:17,066][20347] Updated weights for policy 0, policy_version 861141 (0.0011) [2023-12-27 11:36:17,079][20381] Updated weights for policy 1, policy_version 860352 (0.0007) [2023-12-27 11:36:17,124][20347] Updated weights for policy 0, policy_version 861151 (0.0010) [2023-12-27 11:36:17,130][20381] Updated weights for policy 1, policy_version 860362 (0.0006) [2023-12-27 11:36:17,178][20347] Updated weights for policy 0, policy_version 861161 (0.0006) [2023-12-27 11:36:17,187][20381] Updated weights for policy 1, policy_version 860372 (0.0006) [2023-12-27 11:36:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20049.6). Total num frames: 440770560. Throughput: 0: 10268.9, 1: 10057.0. Samples: 440776016. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:17,363][19170] Avg episode reward: [(0, '990.240'), (1, '824.730')] [2023-12-27 11:36:17,844][20381] Updated weights for policy 1, policy_version 860382 (0.0007) [2023-12-27 11:36:17,849][20347] Updated weights for policy 0, policy_version 861171 (0.0008) [2023-12-27 11:36:17,908][20381] Updated weights for policy 1, policy_version 860392 (0.0007) [2023-12-27 11:36:17,919][20347] Updated weights for policy 0, policy_version 861181 (0.0008) [2023-12-27 11:36:17,960][20381] Updated weights for policy 1, policy_version 860402 (0.0008) [2023-12-27 11:36:17,978][20347] Updated weights for policy 0, policy_version 861191 (0.0008) [2023-12-27 11:36:18,630][20381] Updated weights for policy 1, policy_version 860412 (0.0008) [2023-12-27 11:36:18,635][20347] Updated weights for policy 0, policy_version 861201 (0.0008) [2023-12-27 11:36:18,693][20381] Updated weights for policy 1, policy_version 860422 (0.0010) [2023-12-27 11:36:18,701][20347] Updated weights for policy 0, policy_version 861211 (0.0011) [2023-12-27 11:36:18,758][20381] Updated weights for policy 1, policy_version 860432 (0.0008) [2023-12-27 11:36:18,764][20347] Updated weights for policy 0, policy_version 861221 (0.0010) [2023-12-27 11:36:18,826][20347] Updated weights for policy 0, policy_version 861231 (0.0010) [2023-12-27 11:36:19,490][20381] Updated weights for policy 1, policy_version 860442 (0.0006) [2023-12-27 11:36:19,510][20347] Updated weights for policy 0, policy_version 861241 (0.0011) [2023-12-27 11:36:19,547][20381] Updated weights for policy 1, policy_version 860452 (0.0007) [2023-12-27 11:36:19,569][20347] Updated weights for policy 0, policy_version 861251 (0.0011) [2023-12-27 11:36:19,598][20381] Updated weights for policy 1, policy_version 860462 (0.0005) [2023-12-27 11:36:19,625][20347] Updated weights for policy 0, policy_version 861261 (0.0011) [2023-12-27 11:36:19,650][20381] Updated weights for policy 1, policy_version 860472 (0.0006) [2023-12-27 11:36:20,324][20347] Updated weights for policy 0, policy_version 861271 (0.0007) [2023-12-27 11:36:20,389][20347] Updated weights for policy 0, policy_version 861281 (0.0008) [2023-12-27 11:36:20,434][20381] Updated weights for policy 1, policy_version 860482 (0.0006) [2023-12-27 11:36:20,453][20347] Updated weights for policy 0, policy_version 861291 (0.0011) [2023-12-27 11:36:20,504][20381] Updated weights for policy 1, policy_version 860492 (0.0006) [2023-12-27 11:36:20,579][20381] Updated weights for policy 1, policy_version 860502 (0.0007) [2023-12-27 11:36:21,161][20347] Updated weights for policy 0, policy_version 861301 (0.0008) [2023-12-27 11:36:21,191][20381] Updated weights for policy 1, policy_version 860512 (0.0007) [2023-12-27 11:36:21,221][20347] Updated weights for policy 0, policy_version 861311 (0.0006) [2023-12-27 11:36:21,256][20381] Updated weights for policy 1, policy_version 860522 (0.0007) [2023-12-27 11:36:21,285][20347] Updated weights for policy 0, policy_version 861321 (0.0007) [2023-12-27 11:36:21,316][20381] Updated weights for policy 1, policy_version 860532 (0.0008) [2023-12-27 11:36:21,950][20347] Updated weights for policy 0, policy_version 861331 (0.0007) [2023-12-27 11:36:22,011][20347] Updated weights for policy 0, policy_version 861341 (0.0009) [2023-12-27 11:36:22,072][20347] Updated weights for policy 0, policy_version 861351 (0.0010) [2023-12-27 11:36:22,077][20381] Updated weights for policy 1, policy_version 860542 (0.0007) [2023-12-27 11:36:22,131][20381] Updated weights for policy 1, policy_version 860552 (0.0006) [2023-12-27 11:36:22,178][20381] Updated weights for policy 1, policy_version 860562 (0.0007) [2023-12-27 11:36:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20049.6). Total num frames: 440868864. Throughput: 0: 10240.5, 1: 9985.1. Samples: 440835280. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:22,363][19170] Avg episode reward: [(0, '951.260'), (1, '775.970')] [2023-12-27 11:36:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000860568_220332032.pth... [2023-12-27 11:36:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000861360_220536832.pth... [2023-12-27 11:36:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000859416_220037120.pth [2023-12-27 11:36:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000860176_220233728.pth [2023-12-27 11:36:22,708][20347] Updated weights for policy 0, policy_version 861361 (0.0010) [2023-12-27 11:36:22,765][20347] Updated weights for policy 0, policy_version 861371 (0.0006) [2023-12-27 11:36:22,821][20347] Updated weights for policy 0, policy_version 861381 (0.0006) [2023-12-27 11:36:22,874][20381] Updated weights for policy 1, policy_version 860572 (0.0007) [2023-12-27 11:36:22,880][20347] Updated weights for policy 0, policy_version 861391 (0.0010) [2023-12-27 11:36:22,926][20381] Updated weights for policy 1, policy_version 860582 (0.0007) [2023-12-27 11:36:22,973][20381] Updated weights for policy 1, policy_version 860592 (0.0007) [2023-12-27 11:36:23,548][20347] Updated weights for policy 0, policy_version 861401 (0.0010) [2023-12-27 11:36:23,602][20347] Updated weights for policy 0, policy_version 861411 (0.0010) [2023-12-27 11:36:23,659][20347] Updated weights for policy 0, policy_version 861421 (0.0009) [2023-12-27 11:36:23,671][20381] Updated weights for policy 1, policy_version 860602 (0.0007) [2023-12-27 11:36:23,721][20381] Updated weights for policy 1, policy_version 860612 (0.0010) [2023-12-27 11:36:23,769][20381] Updated weights for policy 1, policy_version 860622 (0.0010) [2023-12-27 11:36:23,826][20381] Updated weights for policy 1, policy_version 860632 (0.0006) [2023-12-27 11:36:24,333][20347] Updated weights for policy 0, policy_version 861431 (0.0005) [2023-12-27 11:36:24,386][20347] Updated weights for policy 0, policy_version 861441 (0.0009) [2023-12-27 11:36:24,436][20347] Updated weights for policy 0, policy_version 861451 (0.0010) [2023-12-27 11:36:24,478][20381] Updated weights for policy 1, policy_version 860642 (0.0009) [2023-12-27 11:36:24,533][20381] Updated weights for policy 1, policy_version 860652 (0.0006) [2023-12-27 11:36:24,587][20381] Updated weights for policy 1, policy_version 860662 (0.0008) [2023-12-27 11:36:25,144][20347] Updated weights for policy 0, policy_version 861461 (0.0010) [2023-12-27 11:36:25,194][20347] Updated weights for policy 0, policy_version 861471 (0.0010) [2023-12-27 11:36:25,249][20347] Updated weights for policy 0, policy_version 861481 (0.0010) [2023-12-27 11:36:25,274][20381] Updated weights for policy 1, policy_version 860672 (0.0010) [2023-12-27 11:36:25,324][20381] Updated weights for policy 1, policy_version 860682 (0.0010) [2023-12-27 11:36:25,370][20381] Updated weights for policy 1, policy_version 860692 (0.0010) [2023-12-27 11:36:25,910][20347] Updated weights for policy 0, policy_version 861491 (0.0009) [2023-12-27 11:36:25,970][20347] Updated weights for policy 0, policy_version 861501 (0.0005) [2023-12-27 11:36:26,018][20347] Updated weights for policy 0, policy_version 861511 (0.0006) [2023-12-27 11:36:26,069][20381] Updated weights for policy 1, policy_version 860702 (0.0008) [2023-12-27 11:36:26,120][20381] Updated weights for policy 1, policy_version 860712 (0.0006) [2023-12-27 11:36:26,173][20381] Updated weights for policy 1, policy_version 860722 (0.0005) [2023-12-27 11:36:26,620][20347] Updated weights for policy 0, policy_version 861521 (0.0006) [2023-12-27 11:36:26,678][20347] Updated weights for policy 0, policy_version 861531 (0.0007) [2023-12-27 11:36:26,728][20347] Updated weights for policy 0, policy_version 861541 (0.0007) [2023-12-27 11:36:26,785][20347] Updated weights for policy 0, policy_version 861551 (0.0007) [2023-12-27 11:36:26,836][20381] Updated weights for policy 1, policy_version 860732 (0.0008) [2023-12-27 11:36:26,889][20381] Updated weights for policy 1, policy_version 860742 (0.0005) [2023-12-27 11:36:26,941][20381] Updated weights for policy 1, policy_version 860752 (0.0005) [2023-12-27 11:36:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20049.6). Total num frames: 440967168. Throughput: 0: 10222.2, 1: 9990.1. Samples: 440957940. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:27,363][19170] Avg episode reward: [(0, '960.880'), (1, '775.640')] [2023-12-27 11:36:27,477][20347] Updated weights for policy 0, policy_version 861561 (0.0006) [2023-12-27 11:36:27,512][20381] Updated weights for policy 1, policy_version 860762 (0.0006) [2023-12-27 11:36:27,529][20347] Updated weights for policy 0, policy_version 861571 (0.0005) [2023-12-27 11:36:27,567][20381] Updated weights for policy 1, policy_version 860773 (0.0007) [2023-12-27 11:36:27,586][20347] Updated weights for policy 0, policy_version 861581 (0.0006) [2023-12-27 11:36:27,621][20381] Updated weights for policy 1, policy_version 860783 (0.0005) [2023-12-27 11:36:28,104][20347] Updated weights for policy 0, policy_version 861591 (0.0007) [2023-12-27 11:36:28,146][20381] Updated weights for policy 1, policy_version 860793 (0.0005) [2023-12-27 11:36:28,164][20347] Updated weights for policy 0, policy_version 861601 (0.0008) [2023-12-27 11:36:28,194][20381] Updated weights for policy 1, policy_version 860803 (0.0005) [2023-12-27 11:36:28,209][20347] Updated weights for policy 0, policy_version 861611 (0.0006) [2023-12-27 11:36:28,250][20381] Updated weights for policy 1, policy_version 860813 (0.0009) [2023-12-27 11:36:28,308][20381] Updated weights for policy 1, policy_version 860823 (0.0008) [2023-12-27 11:36:28,915][20347] Updated weights for policy 0, policy_version 861621 (0.0006) [2023-12-27 11:36:28,969][20381] Updated weights for policy 1, policy_version 860833 (0.0010) [2023-12-27 11:36:28,971][20347] Updated weights for policy 0, policy_version 861631 (0.0006) [2023-12-27 11:36:29,023][20347] Updated weights for policy 0, policy_version 861641 (0.0005) [2023-12-27 11:36:29,025][20381] Updated weights for policy 1, policy_version 860843 (0.0011) [2023-12-27 11:36:29,077][20381] Updated weights for policy 1, policy_version 860853 (0.0010) [2023-12-27 11:36:29,737][20347] Updated weights for policy 0, policy_version 861651 (0.0007) [2023-12-27 11:36:29,748][20381] Updated weights for policy 1, policy_version 860863 (0.0007) [2023-12-27 11:36:29,789][20347] Updated weights for policy 0, policy_version 861661 (0.0009) [2023-12-27 11:36:29,800][20381] Updated weights for policy 1, policy_version 860873 (0.0008) [2023-12-27 11:36:29,846][20347] Updated weights for policy 0, policy_version 861671 (0.0008) [2023-12-27 11:36:29,863][20381] Updated weights for policy 1, policy_version 860883 (0.0007) [2023-12-27 11:36:30,557][20381] Updated weights for policy 1, policy_version 860893 (0.0007) [2023-12-27 11:36:30,558][20347] Updated weights for policy 0, policy_version 861681 (0.0006) [2023-12-27 11:36:30,611][20381] Updated weights for policy 1, policy_version 860903 (0.0007) [2023-12-27 11:36:30,617][20347] Updated weights for policy 0, policy_version 861691 (0.0011) [2023-12-27 11:36:30,663][20381] Updated weights for policy 1, policy_version 860913 (0.0005) [2023-12-27 11:36:30,672][20347] Updated weights for policy 0, policy_version 861701 (0.0010) [2023-12-27 11:36:30,727][20347] Updated weights for policy 0, policy_version 861711 (0.0010) [2023-12-27 11:36:31,388][20381] Updated weights for policy 1, policy_version 860923 (0.0006) [2023-12-27 11:36:31,441][20381] Updated weights for policy 1, policy_version 860933 (0.0008) [2023-12-27 11:36:31,460][20347] Updated weights for policy 0, policy_version 861721 (0.0007) [2023-12-27 11:36:31,497][20381] Updated weights for policy 1, policy_version 860943 (0.0010) [2023-12-27 11:36:31,521][20347] Updated weights for policy 0, policy_version 861731 (0.0008) [2023-12-27 11:36:31,578][20347] Updated weights for policy 0, policy_version 861741 (0.0006) [2023-12-27 11:36:32,157][20381] Updated weights for policy 1, policy_version 860953 (0.0008) [2023-12-27 11:36:32,211][20381] Updated weights for policy 1, policy_version 860963 (0.0007) [2023-12-27 11:36:32,269][20347] Updated weights for policy 0, policy_version 861751 (0.0007) [2023-12-27 11:36:32,273][20381] Updated weights for policy 1, policy_version 860973 (0.0006) [2023-12-27 11:36:32,328][20381] Updated weights for policy 1, policy_version 860983 (0.0008) [2023-12-27 11:36:32,329][20347] Updated weights for policy 0, policy_version 861761 (0.0005) [2023-12-27 11:36:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20077.3). Total num frames: 441073664. Throughput: 0: 10199.3, 1: 10060.8. Samples: 441084196. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:32,363][19170] Avg episode reward: [(0, '911.940'), (1, '843.930')] [2023-12-27 11:36:32,393][20347] Updated weights for policy 0, policy_version 861771 (0.0006) [2023-12-27 11:36:32,986][20347] Updated weights for policy 0, policy_version 861781 (0.0007) [2023-12-27 11:36:32,990][20381] Updated weights for policy 1, policy_version 860993 (0.0005) [2023-12-27 11:36:33,042][20347] Updated weights for policy 0, policy_version 861791 (0.0008) [2023-12-27 11:36:33,052][20381] Updated weights for policy 1, policy_version 861003 (0.0005) [2023-12-27 11:36:33,098][20347] Updated weights for policy 0, policy_version 861801 (0.0008) [2023-12-27 11:36:33,102][20381] Updated weights for policy 1, policy_version 861013 (0.0005) [2023-12-27 11:36:33,697][20381] Updated weights for policy 1, policy_version 861023 (0.0008) [2023-12-27 11:36:33,751][20381] Updated weights for policy 1, policy_version 861033 (0.0008) [2023-12-27 11:36:33,803][20347] Updated weights for policy 0, policy_version 861811 (0.0007) [2023-12-27 11:36:33,807][20381] Updated weights for policy 1, policy_version 861043 (0.0005) [2023-12-27 11:36:33,846][20347] Updated weights for policy 0, policy_version 861821 (0.0005) [2023-12-27 11:36:33,893][20347] Updated weights for policy 0, policy_version 861831 (0.0006) [2023-12-27 11:36:34,470][20381] Updated weights for policy 1, policy_version 861053 (0.0007) [2023-12-27 11:36:34,529][20381] Updated weights for policy 1, policy_version 861063 (0.0006) [2023-12-27 11:36:34,576][20347] Updated weights for policy 0, policy_version 861841 (0.0007) [2023-12-27 11:36:34,585][20381] Updated weights for policy 1, policy_version 861073 (0.0007) [2023-12-27 11:36:34,639][20347] Updated weights for policy 0, policy_version 861851 (0.0006) [2023-12-27 11:36:34,698][20347] Updated weights for policy 0, policy_version 861861 (0.0008) [2023-12-27 11:36:34,755][20347] Updated weights for policy 0, policy_version 861871 (0.0009) [2023-12-27 11:36:35,171][20381] Updated weights for policy 1, policy_version 861083 (0.0007) [2023-12-27 11:36:35,228][20381] Updated weights for policy 1, policy_version 861093 (0.0011) [2023-12-27 11:36:35,287][20381] Updated weights for policy 1, policy_version 861103 (0.0010) [2023-12-27 11:36:35,456][20347] Updated weights for policy 0, policy_version 861881 (0.0006) [2023-12-27 11:36:35,512][20347] Updated weights for policy 0, policy_version 861891 (0.0008) [2023-12-27 11:36:35,567][20347] Updated weights for policy 0, policy_version 861901 (0.0010) [2023-12-27 11:36:35,907][20381] Updated weights for policy 1, policy_version 861113 (0.0010) [2023-12-27 11:36:35,970][20381] Updated weights for policy 1, policy_version 861123 (0.0010) [2023-12-27 11:36:36,028][20381] Updated weights for policy 1, policy_version 861133 (0.0006) [2023-12-27 11:36:36,084][20381] Updated weights for policy 1, policy_version 861143 (0.0005) [2023-12-27 11:36:36,157][20347] Updated weights for policy 0, policy_version 861911 (0.0008) [2023-12-27 11:36:36,225][20347] Updated weights for policy 0, policy_version 861921 (0.0010) [2023-12-27 11:36:36,293][20347] Updated weights for policy 0, policy_version 861931 (0.0011) [2023-12-27 11:36:36,739][20381] Updated weights for policy 1, policy_version 861153 (0.0007) [2023-12-27 11:36:36,797][20381] Updated weights for policy 1, policy_version 861163 (0.0008) [2023-12-27 11:36:36,856][20381] Updated weights for policy 1, policy_version 861173 (0.0007) [2023-12-27 11:36:36,982][20347] Updated weights for policy 0, policy_version 861941 (0.0011) [2023-12-27 11:36:37,035][20347] Updated weights for policy 0, policy_version 861951 (0.0009) [2023-12-27 11:36:37,095][20347] Updated weights for policy 0, policy_version 861961 (0.0007) [2023-12-27 11:36:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.7, 300 sec: 20105.1). Total num frames: 441180160. Throughput: 0: 10228.7, 1: 10094.4. Samples: 441147460. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:37,363][19170] Avg episode reward: [(0, '844.010'), (1, '902.510')] [2023-12-27 11:36:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000861176_220487680.pth... [2023-12-27 11:36:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000861968_220692480.pth... [2023-12-27 11:36:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000859992_220184576.pth [2023-12-27 11:36:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000860752_220381184.pth [2023-12-27 11:36:37,546][20381] Updated weights for policy 1, policy_version 861183 (0.0007) [2023-12-27 11:36:37,595][20381] Updated weights for policy 1, policy_version 861193 (0.0008) [2023-12-27 11:36:37,650][20381] Updated weights for policy 1, policy_version 861203 (0.0007) [2023-12-27 11:36:37,769][20347] Updated weights for policy 0, policy_version 861971 (0.0009) [2023-12-27 11:36:37,801][20291] KL-divergence is very high: 318.1891 [2023-12-27 11:36:37,812][20291] KL-divergence is very high: 389.0057 [2023-12-27 11:36:37,822][20347] Updated weights for policy 0, policy_version 861981 (0.0007) [2023-12-27 11:36:37,847][20291] KL-divergence is very high: 143.6546 [2023-12-27 11:36:37,863][20291] KL-divergence is very high: 212.2075 [2023-12-27 11:36:37,878][20347] Updated weights for policy 0, policy_version 861991 (0.0005) [2023-12-27 11:36:37,888][20291] KL-divergence is very high: 166.3016 [2023-12-27 11:36:37,903][20291] KL-divergence is very high: 167.1541 [2023-12-27 11:36:38,390][20381] Updated weights for policy 1, policy_version 861213 (0.0006) [2023-12-27 11:36:38,452][20381] Updated weights for policy 1, policy_version 861223 (0.0006) [2023-12-27 11:36:38,487][20347] Updated weights for policy 0, policy_version 862001 (0.0007) [2023-12-27 11:36:38,512][20381] Updated weights for policy 1, policy_version 861233 (0.0006) [2023-12-27 11:36:38,547][20347] Updated weights for policy 0, policy_version 862011 (0.0005) [2023-12-27 11:36:38,598][20347] Updated weights for policy 0, policy_version 862021 (0.0007) [2023-12-27 11:36:38,658][20347] Updated weights for policy 0, policy_version 862031 (0.0008) [2023-12-27 11:36:39,110][20381] Updated weights for policy 1, policy_version 861243 (0.0007) [2023-12-27 11:36:39,170][20381] Updated weights for policy 1, policy_version 861253 (0.0008) [2023-12-27 11:36:39,229][20381] Updated weights for policy 1, policy_version 861263 (0.0008) [2023-12-27 11:36:39,346][20347] Updated weights for policy 0, policy_version 862041 (0.0007) [2023-12-27 11:36:39,414][20347] Updated weights for policy 0, policy_version 862051 (0.0008) [2023-12-27 11:36:39,479][20347] Updated weights for policy 0, policy_version 862061 (0.0008) [2023-12-27 11:36:39,484][20291] KL-divergence is very high: 316.4848 [2023-12-27 11:36:39,491][20291] KL-divergence is very high: 258.8196 [2023-12-27 11:36:39,998][20381] Updated weights for policy 1, policy_version 861273 (0.0008) [2023-12-27 11:36:40,058][20381] Updated weights for policy 1, policy_version 861283 (0.0006) [2023-12-27 11:36:40,120][20381] Updated weights for policy 1, policy_version 861293 (0.0006) [2023-12-27 11:36:40,138][20291] KL-divergence is very high: 411.2769 [2023-12-27 11:36:40,159][20347] Updated weights for policy 0, policy_version 862071 (0.0007) [2023-12-27 11:36:40,179][20381] Updated weights for policy 1, policy_version 861303 (0.0006) [2023-12-27 11:36:40,186][20291] KL-divergence is very high: 174.6624 [2023-12-27 11:36:40,221][20347] Updated weights for policy 0, policy_version 862081 (0.0008) [2023-12-27 11:36:40,232][20291] KL-divergence is very high: 137.8048 [2023-12-27 11:36:40,283][20347] Updated weights for policy 0, policy_version 862091 (0.0007) [2023-12-27 11:36:40,778][20381] Updated weights for policy 1, policy_version 861313 (0.0010) [2023-12-27 11:36:40,839][20381] Updated weights for policy 1, policy_version 861323 (0.0011) [2023-12-27 11:36:40,894][20381] Updated weights for policy 1, policy_version 861333 (0.0009) [2023-12-27 11:36:41,015][20347] Updated weights for policy 0, policy_version 862101 (0.0006) [2023-12-27 11:36:41,081][20347] Updated weights for policy 0, policy_version 862111 (0.0008) [2023-12-27 11:36:41,145][20347] Updated weights for policy 0, policy_version 862121 (0.0007) [2023-12-27 11:36:41,654][20381] Updated weights for policy 1, policy_version 861343 (0.0009) [2023-12-27 11:36:41,718][20381] Updated weights for policy 1, policy_version 861353 (0.0008) [2023-12-27 11:36:41,788][20381] Updated weights for policy 1, policy_version 861363 (0.0008) [2023-12-27 11:36:41,937][20347] Updated weights for policy 0, policy_version 862131 (0.0008) [2023-12-27 11:36:41,999][20347] Updated weights for policy 0, policy_version 862141 (0.0008) [2023-12-27 11:36:42,059][20347] Updated weights for policy 0, policy_version 862151 (0.0008) [2023-12-27 11:36:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20105.1). Total num frames: 441278464. Throughput: 0: 10254.7, 1: 10131.5. Samples: 441270696. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:42,363][19170] Avg episode reward: [(0, '853.510'), (1, '902.460')] [2023-12-27 11:36:42,438][20381] Updated weights for policy 1, policy_version 861373 (0.0008) [2023-12-27 11:36:42,501][20381] Updated weights for policy 1, policy_version 861383 (0.0008) [2023-12-27 11:36:42,561][20381] Updated weights for policy 1, policy_version 861393 (0.0006) [2023-12-27 11:36:42,829][20347] Updated weights for policy 0, policy_version 862161 (0.0008) [2023-12-27 11:36:42,881][20347] Updated weights for policy 0, policy_version 862171 (0.0009) [2023-12-27 11:36:42,938][20347] Updated weights for policy 0, policy_version 862181 (0.0008) [2023-12-27 11:36:42,995][20347] Updated weights for policy 0, policy_version 862191 (0.0009) [2023-12-27 11:36:43,113][20381] Updated weights for policy 1, policy_version 861403 (0.0006) [2023-12-27 11:36:43,172][20381] Updated weights for policy 1, policy_version 861413 (0.0007) [2023-12-27 11:36:43,228][20381] Updated weights for policy 1, policy_version 861423 (0.0008) [2023-12-27 11:36:43,702][20347] Updated weights for policy 0, policy_version 862201 (0.0007) [2023-12-27 11:36:43,762][20347] Updated weights for policy 0, policy_version 862211 (0.0007) [2023-12-27 11:36:43,818][20347] Updated weights for policy 0, policy_version 862221 (0.0006) [2023-12-27 11:36:43,888][20381] Updated weights for policy 1, policy_version 861433 (0.0009) [2023-12-27 11:36:43,936][20381] Updated weights for policy 1, policy_version 861443 (0.0009) [2023-12-27 11:36:43,988][20381] Updated weights for policy 1, policy_version 861453 (0.0008) [2023-12-27 11:36:44,038][20381] Updated weights for policy 1, policy_version 861463 (0.0008) [2023-12-27 11:36:44,492][20347] Updated weights for policy 0, policy_version 862231 (0.0009) [2023-12-27 11:36:44,554][20347] Updated weights for policy 0, policy_version 862241 (0.0010) [2023-12-27 11:36:44,608][20347] Updated weights for policy 0, policy_version 862251 (0.0010) [2023-12-27 11:36:44,634][20381] Updated weights for policy 1, policy_version 861473 (0.0008) [2023-12-27 11:36:44,684][20381] Updated weights for policy 1, policy_version 861483 (0.0007) [2023-12-27 11:36:44,728][20381] Updated weights for policy 1, policy_version 861493 (0.0007) [2023-12-27 11:36:45,282][20347] Updated weights for policy 0, policy_version 862261 (0.0010) [2023-12-27 11:36:45,343][20347] Updated weights for policy 0, policy_version 862271 (0.0010) [2023-12-27 11:36:45,399][20347] Updated weights for policy 0, policy_version 862281 (0.0010) [2023-12-27 11:36:45,431][20381] Updated weights for policy 1, policy_version 861503 (0.0006) [2023-12-27 11:36:45,488][20381] Updated weights for policy 1, policy_version 861513 (0.0007) [2023-12-27 11:36:45,537][20381] Updated weights for policy 1, policy_version 861523 (0.0008) [2023-12-27 11:36:46,051][20347] Updated weights for policy 0, policy_version 862291 (0.0009) [2023-12-27 11:36:46,100][20347] Updated weights for policy 0, policy_version 862301 (0.0007) [2023-12-27 11:36:46,151][20347] Updated weights for policy 0, policy_version 862311 (0.0007) [2023-12-27 11:36:46,210][20381] Updated weights for policy 1, policy_version 861533 (0.0009) [2023-12-27 11:36:46,266][20381] Updated weights for policy 1, policy_version 861543 (0.0011) [2023-12-27 11:36:46,319][20381] Updated weights for policy 1, policy_version 861553 (0.0010) [2023-12-27 11:36:46,808][20347] Updated weights for policy 0, policy_version 862321 (0.0007) [2023-12-27 11:36:46,859][20347] Updated weights for policy 0, policy_version 862331 (0.0006) [2023-12-27 11:36:46,906][20381] Updated weights for policy 1, policy_version 861563 (0.0005) [2023-12-27 11:36:46,916][20347] Updated weights for policy 0, policy_version 862341 (0.0005) [2023-12-27 11:36:46,968][20381] Updated weights for policy 1, policy_version 861573 (0.0010) [2023-12-27 11:36:46,974][20347] Updated weights for policy 0, policy_version 862351 (0.0005) [2023-12-27 11:36:47,021][20381] Updated weights for policy 1, policy_version 861583 (0.0011) [2023-12-27 11:36:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20132.9). Total num frames: 441384960. Throughput: 0: 10298.4, 1: 10260.0. Samples: 441394172. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:47,363][19170] Avg episode reward: [(0, '745.230'), (1, '844.000')] [2023-12-27 11:36:47,617][20381] Updated weights for policy 1, policy_version 861593 (0.0010) [2023-12-27 11:36:47,663][20381] Updated weights for policy 1, policy_version 861603 (0.0005) [2023-12-27 11:36:47,683][20347] Updated weights for policy 0, policy_version 862361 (0.0006) [2023-12-27 11:36:47,713][20381] Updated weights for policy 1, policy_version 861613 (0.0005) [2023-12-27 11:36:47,744][20347] Updated weights for policy 0, policy_version 862371 (0.0006) [2023-12-27 11:36:47,774][20381] Updated weights for policy 1, policy_version 861623 (0.0005) [2023-12-27 11:36:47,799][20347] Updated weights for policy 0, policy_version 862381 (0.0006) [2023-12-27 11:36:48,335][20381] Updated weights for policy 1, policy_version 861633 (0.0006) [2023-12-27 11:36:48,388][20381] Updated weights for policy 1, policy_version 861643 (0.0011) [2023-12-27 11:36:48,446][20381] Updated weights for policy 1, policy_version 861653 (0.0011) [2023-12-27 11:36:48,447][20347] Updated weights for policy 0, policy_version 862391 (0.0006) [2023-12-27 11:36:48,505][20347] Updated weights for policy 0, policy_version 862401 (0.0006) [2023-12-27 11:36:48,570][20347] Updated weights for policy 0, policy_version 862411 (0.0007) [2023-12-27 11:36:49,101][20381] Updated weights for policy 1, policy_version 861663 (0.0008) [2023-12-27 11:36:49,158][20381] Updated weights for policy 1, policy_version 861673 (0.0008) [2023-12-27 11:36:49,218][20381] Updated weights for policy 1, policy_version 861683 (0.0008) [2023-12-27 11:36:49,234][20347] Updated weights for policy 0, policy_version 862421 (0.0008) [2023-12-27 11:36:49,292][20347] Updated weights for policy 0, policy_version 862431 (0.0007) [2023-12-27 11:36:49,355][20347] Updated weights for policy 0, policy_version 862441 (0.0008) [2023-12-27 11:36:49,924][20381] Updated weights for policy 1, policy_version 861693 (0.0007) [2023-12-27 11:36:49,983][20381] Updated weights for policy 1, policy_version 861703 (0.0007) [2023-12-27 11:36:50,037][20381] Updated weights for policy 1, policy_version 861713 (0.0008) [2023-12-27 11:36:50,038][20347] Updated weights for policy 0, policy_version 862451 (0.0007) [2023-12-27 11:36:50,101][20347] Updated weights for policy 0, policy_version 862461 (0.0010) [2023-12-27 11:36:50,197][20347] Updated weights for policy 0, policy_version 862471 (0.0010) [2023-12-27 11:36:50,686][20381] Updated weights for policy 1, policy_version 861723 (0.0008) [2023-12-27 11:36:50,740][20381] Updated weights for policy 1, policy_version 861733 (0.0006) [2023-12-27 11:36:50,802][20381] Updated weights for policy 1, policy_version 861743 (0.0007) [2023-12-27 11:36:50,806][20347] Updated weights for policy 0, policy_version 862481 (0.0010) [2023-12-27 11:36:50,857][20347] Updated weights for policy 0, policy_version 862491 (0.0008) [2023-12-27 11:36:50,913][20347] Updated weights for policy 0, policy_version 862501 (0.0008) [2023-12-27 11:36:50,968][20347] Updated weights for policy 0, policy_version 862511 (0.0008) [2023-12-27 11:36:51,522][20381] Updated weights for policy 1, policy_version 861753 (0.0008) [2023-12-27 11:36:51,580][20381] Updated weights for policy 1, policy_version 861763 (0.0007) [2023-12-27 11:36:51,646][20381] Updated weights for policy 1, policy_version 861773 (0.0006) [2023-12-27 11:36:51,707][20381] Updated weights for policy 1, policy_version 861783 (0.0006) [2023-12-27 11:36:51,709][20347] Updated weights for policy 0, policy_version 862521 (0.0007) [2023-12-27 11:36:51,777][20347] Updated weights for policy 0, policy_version 862531 (0.0008) [2023-12-27 11:36:51,839][20347] Updated weights for policy 0, policy_version 862541 (0.0007) [2023-12-27 11:36:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20132.9). Total num frames: 441483264. Throughput: 0: 10306.5, 1: 10318.9. Samples: 441456948. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:52,363][19170] Avg episode reward: [(0, '745.120'), (1, '767.010')] [2023-12-27 11:36:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000862544_220839936.pth... [2023-12-27 11:36:52,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000861360_220536832.pth [2023-12-27 11:36:52,428][20381] Updated weights for policy 1, policy_version 861793 (0.0007) [2023-12-27 11:36:52,487][20381] Updated weights for policy 1, policy_version 861803 (0.0006) [2023-12-27 11:36:52,523][20347] Updated weights for policy 0, policy_version 862551 (0.0006) [2023-12-27 11:36:52,541][20381] Updated weights for policy 1, policy_version 861813 (0.0007) [2023-12-27 11:36:52,559][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000861816_220651520.pth... [2023-12-27 11:36:52,563][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000860568_220332032.pth [2023-12-27 11:36:52,581][20347] Updated weights for policy 0, policy_version 862561 (0.0008) [2023-12-27 11:36:52,634][20347] Updated weights for policy 0, policy_version 862571 (0.0010) [2023-12-27 11:36:53,228][20381] Updated weights for policy 1, policy_version 861823 (0.0007) [2023-12-27 11:36:53,280][20347] Updated weights for policy 0, policy_version 862581 (0.0010) [2023-12-27 11:36:53,286][20381] Updated weights for policy 1, policy_version 861833 (0.0008) [2023-12-27 11:36:53,336][20381] Updated weights for policy 1, policy_version 861843 (0.0008) [2023-12-27 11:36:53,338][20347] Updated weights for policy 0, policy_version 862591 (0.0010) [2023-12-27 11:36:53,396][20347] Updated weights for policy 0, policy_version 862601 (0.0010) [2023-12-27 11:36:53,939][20381] Updated weights for policy 1, policy_version 861853 (0.0007) [2023-12-27 11:36:54,009][20381] Updated weights for policy 1, policy_version 861863 (0.0007) [2023-12-27 11:36:54,067][20381] Updated weights for policy 1, policy_version 861873 (0.0010) [2023-12-27 11:36:54,073][20347] Updated weights for policy 0, policy_version 862611 (0.0009) [2023-12-27 11:36:54,133][20347] Updated weights for policy 0, policy_version 862621 (0.0007) [2023-12-27 11:36:54,197][20347] Updated weights for policy 0, policy_version 862631 (0.0008) [2023-12-27 11:36:54,687][20381] Updated weights for policy 1, policy_version 861883 (0.0008) [2023-12-27 11:36:54,742][20381] Updated weights for policy 1, policy_version 861893 (0.0005) [2023-12-27 11:36:54,790][20381] Updated weights for policy 1, policy_version 861903 (0.0005) [2023-12-27 11:36:54,871][20347] Updated weights for policy 0, policy_version 862641 (0.0007) [2023-12-27 11:36:54,920][20347] Updated weights for policy 0, policy_version 862651 (0.0010) [2023-12-27 11:36:54,980][20347] Updated weights for policy 0, policy_version 862661 (0.0007) [2023-12-27 11:36:55,038][20347] Updated weights for policy 0, policy_version 862671 (0.0006) [2023-12-27 11:36:55,449][20381] Updated weights for policy 1, policy_version 861913 (0.0007) [2023-12-27 11:36:55,497][20381] Updated weights for policy 1, policy_version 861923 (0.0010) [2023-12-27 11:36:55,553][20381] Updated weights for policy 1, policy_version 861933 (0.0006) [2023-12-27 11:36:55,608][20381] Updated weights for policy 1, policy_version 861943 (0.0006) [2023-12-27 11:36:55,653][20347] Updated weights for policy 0, policy_version 862681 (0.0010) [2023-12-27 11:36:55,708][20347] Updated weights for policy 0, policy_version 862691 (0.0010) [2023-12-27 11:36:55,764][20347] Updated weights for policy 0, policy_version 862701 (0.0009) [2023-12-27 11:36:56,265][20381] Updated weights for policy 1, policy_version 861953 (0.0008) [2023-12-27 11:36:56,319][20381] Updated weights for policy 1, policy_version 861963 (0.0008) [2023-12-27 11:36:56,401][20381] Updated weights for policy 1, policy_version 861973 (0.0007) [2023-12-27 11:36:56,428][20347] Updated weights for policy 0, policy_version 862711 (0.0008) [2023-12-27 11:36:56,489][20347] Updated weights for policy 0, policy_version 862721 (0.0007) [2023-12-27 11:36:56,546][20347] Updated weights for policy 0, policy_version 862731 (0.0008) [2023-12-27 11:36:57,100][20381] Updated weights for policy 1, policy_version 861983 (0.0007) [2023-12-27 11:36:57,159][20381] Updated weights for policy 1, policy_version 861993 (0.0009) [2023-12-27 11:36:57,216][20381] Updated weights for policy 1, policy_version 862003 (0.0008) [2023-12-27 11:36:57,234][20347] Updated weights for policy 0, policy_version 862741 (0.0007) [2023-12-27 11:36:57,293][20347] Updated weights for policy 0, policy_version 862751 (0.0006) [2023-12-27 11:36:57,360][20347] Updated weights for policy 0, policy_version 862761 (0.0006) [2023-12-27 11:36:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 441589760. Throughput: 0: 10265.6, 1: 10416.9. Samples: 441581452. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:36:57,363][19170] Avg episode reward: [(0, '676.530'), (1, '767.370')] [2023-12-27 11:36:57,938][20381] Updated weights for policy 1, policy_version 862013 (0.0008) [2023-12-27 11:36:57,995][20381] Updated weights for policy 1, policy_version 862023 (0.0007) [2023-12-27 11:36:57,996][20347] Updated weights for policy 0, policy_version 862771 (0.0006) [2023-12-27 11:36:58,054][20381] Updated weights for policy 1, policy_version 862033 (0.0007) [2023-12-27 11:36:58,055][20347] Updated weights for policy 0, policy_version 862781 (0.0008) [2023-12-27 11:36:58,110][20347] Updated weights for policy 0, policy_version 862791 (0.0008) [2023-12-27 11:36:58,804][20381] Updated weights for policy 1, policy_version 862043 (0.0006) [2023-12-27 11:36:58,882][20381] Updated weights for policy 1, policy_version 862053 (0.0009) [2023-12-27 11:36:58,906][20347] Updated weights for policy 0, policy_version 862801 (0.0007) [2023-12-27 11:36:58,953][20381] Updated weights for policy 1, policy_version 862063 (0.0009) [2023-12-27 11:36:58,973][20347] Updated weights for policy 0, policy_version 862811 (0.0007) [2023-12-27 11:36:59,041][20347] Updated weights for policy 0, policy_version 862821 (0.0009) [2023-12-27 11:36:59,104][20347] Updated weights for policy 0, policy_version 862831 (0.0007) [2023-12-27 11:36:59,799][20381] Updated weights for policy 1, policy_version 862073 (0.0007) [2023-12-27 11:36:59,800][20347] Updated weights for policy 0, policy_version 862841 (0.0007) [2023-12-27 11:36:59,855][20381] Updated weights for policy 1, policy_version 862083 (0.0008) [2023-12-27 11:36:59,858][20347] Updated weights for policy 0, policy_version 862851 (0.0008) [2023-12-27 11:36:59,914][20381] Updated weights for policy 1, policy_version 862093 (0.0006) [2023-12-27 11:36:59,918][20347] Updated weights for policy 0, policy_version 862861 (0.0007) [2023-12-27 11:36:59,983][20381] Updated weights for policy 1, policy_version 862103 (0.0009) [2023-12-27 11:37:00,673][20347] Updated weights for policy 0, policy_version 862871 (0.0006) [2023-12-27 11:37:00,703][20381] Updated weights for policy 1, policy_version 862113 (0.0007) [2023-12-27 11:37:00,737][20347] Updated weights for policy 0, policy_version 862881 (0.0006) [2023-12-27 11:37:00,761][20381] Updated weights for policy 1, policy_version 862123 (0.0007) [2023-12-27 11:37:00,796][20347] Updated weights for policy 0, policy_version 862891 (0.0006) [2023-12-27 11:37:00,819][20381] Updated weights for policy 1, policy_version 862133 (0.0008) [2023-12-27 11:37:01,477][20347] Updated weights for policy 0, policy_version 862901 (0.0007) [2023-12-27 11:37:01,537][20347] Updated weights for policy 0, policy_version 862911 (0.0007) [2023-12-27 11:37:01,539][20381] Updated weights for policy 1, policy_version 862143 (0.0007) [2023-12-27 11:37:01,595][20381] Updated weights for policy 1, policy_version 862153 (0.0007) [2023-12-27 11:37:01,597][20347] Updated weights for policy 0, policy_version 862921 (0.0008) [2023-12-27 11:37:01,654][20381] Updated weights for policy 1, policy_version 862163 (0.0008) [2023-12-27 11:37:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20160.7). Total num frames: 441679872. Throughput: 0: 10123.4, 1: 10319.1. Samples: 441695928. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:37:02,363][19170] Avg episode reward: [(0, '656.500'), (1, '730.200')] [2023-12-27 11:37:02,371][20381] Updated weights for policy 1, policy_version 862173 (0.0009) [2023-12-27 11:37:02,372][20347] Updated weights for policy 0, policy_version 862931 (0.0008) [2023-12-27 11:37:02,433][20381] Updated weights for policy 1, policy_version 862183 (0.0007) [2023-12-27 11:37:02,435][20347] Updated weights for policy 0, policy_version 862941 (0.0007) [2023-12-27 11:37:02,494][20347] Updated weights for policy 0, policy_version 862951 (0.0006) [2023-12-27 11:37:02,495][20381] Updated weights for policy 1, policy_version 862193 (0.0006) [2023-12-27 11:37:03,205][20381] Updated weights for policy 1, policy_version 862203 (0.0007) [2023-12-27 11:37:03,218][20347] Updated weights for policy 0, policy_version 862961 (0.0006) [2023-12-27 11:37:03,261][20381] Updated weights for policy 1, policy_version 862213 (0.0007) [2023-12-27 11:37:03,279][20347] Updated weights for policy 0, policy_version 862971 (0.0006) [2023-12-27 11:37:03,323][20381] Updated weights for policy 1, policy_version 862223 (0.0006) [2023-12-27 11:37:03,337][20347] Updated weights for policy 0, policy_version 862981 (0.0008) [2023-12-27 11:37:03,408][20347] Updated weights for policy 0, policy_version 862991 (0.0008) [2023-12-27 11:37:04,006][20381] Updated weights for policy 1, policy_version 862233 (0.0008) [2023-12-27 11:37:04,070][20381] Updated weights for policy 1, policy_version 862243 (0.0008) [2023-12-27 11:37:04,124][20347] Updated weights for policy 0, policy_version 863001 (0.0007) [2023-12-27 11:37:04,140][20381] Updated weights for policy 1, policy_version 862253 (0.0008) [2023-12-27 11:37:04,181][20347] Updated weights for policy 0, policy_version 863011 (0.0007) [2023-12-27 11:37:04,207][20381] Updated weights for policy 1, policy_version 862263 (0.0008) [2023-12-27 11:37:04,247][20347] Updated weights for policy 0, policy_version 863021 (0.0008) [2023-12-27 11:37:04,931][20381] Updated weights for policy 1, policy_version 862273 (0.0007) [2023-12-27 11:37:04,937][20347] Updated weights for policy 0, policy_version 863031 (0.0006) [2023-12-27 11:37:04,991][20381] Updated weights for policy 1, policy_version 862283 (0.0007) [2023-12-27 11:37:04,993][20347] Updated weights for policy 0, policy_version 863041 (0.0006) [2023-12-27 11:37:05,049][20347] Updated weights for policy 0, policy_version 863051 (0.0006) [2023-12-27 11:37:05,054][20381] Updated weights for policy 1, policy_version 862293 (0.0008) [2023-12-27 11:37:05,712][20347] Updated weights for policy 0, policy_version 863061 (0.0008) [2023-12-27 11:37:05,775][20347] Updated weights for policy 0, policy_version 863071 (0.0007) [2023-12-27 11:37:05,797][20381] Updated weights for policy 1, policy_version 862303 (0.0006) [2023-12-27 11:37:05,834][20347] Updated weights for policy 0, policy_version 863081 (0.0007) [2023-12-27 11:37:05,854][20381] Updated weights for policy 1, policy_version 862313 (0.0006) [2023-12-27 11:37:05,909][20381] Updated weights for policy 1, policy_version 862323 (0.0007) [2023-12-27 11:37:06,586][20347] Updated weights for policy 0, policy_version 863091 (0.0006) [2023-12-27 11:37:06,627][20381] Updated weights for policy 1, policy_version 862333 (0.0009) [2023-12-27 11:37:06,650][20347] Updated weights for policy 0, policy_version 863101 (0.0007) [2023-12-27 11:37:06,692][20381] Updated weights for policy 1, policy_version 862343 (0.0007) [2023-12-27 11:37:06,716][20347] Updated weights for policy 0, policy_version 863111 (0.0007) [2023-12-27 11:37:06,755][20381] Updated weights for policy 1, policy_version 862353 (0.0007) [2023-12-27 11:37:07,362][19170] Fps is (10 sec: 18841.4, 60 sec: 20343.4, 300 sec: 20188.4). Total num frames: 441778176. Throughput: 0: 10104.5, 1: 10305.2. Samples: 441753720. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:37:07,363][19170] Avg episode reward: [(0, '705.860'), (1, '797.910')] [2023-12-27 11:37:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000862360_220790784.pth... [2023-12-27 11:37:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000863120_220987392.pth... [2023-12-27 11:37:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000861176_220487680.pth [2023-12-27 11:37:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000861968_220692480.pth [2023-12-27 11:37:07,435][20381] Updated weights for policy 1, policy_version 862363 (0.0008) [2023-12-27 11:37:07,457][20347] Updated weights for policy 0, policy_version 863121 (0.0007) [2023-12-27 11:37:07,497][20381] Updated weights for policy 1, policy_version 862373 (0.0008) [2023-12-27 11:37:07,511][20347] Updated weights for policy 0, policy_version 863131 (0.0006) [2023-12-27 11:37:07,558][20381] Updated weights for policy 1, policy_version 862383 (0.0005) [2023-12-27 11:37:07,573][20347] Updated weights for policy 0, policy_version 863141 (0.0007) [2023-12-27 11:37:07,631][20347] Updated weights for policy 0, policy_version 863151 (0.0007) [2023-12-27 11:37:08,130][20381] Updated weights for policy 1, policy_version 862393 (0.0006) [2023-12-27 11:37:08,170][20292] KL-divergence is very high: 100.6383 [2023-12-27 11:37:08,189][20381] Updated weights for policy 1, policy_version 862403 (0.0007) [2023-12-27 11:37:08,254][20381] Updated weights for policy 1, policy_version 862413 (0.0008) [2023-12-27 11:37:08,312][20381] Updated weights for policy 1, policy_version 862423 (0.0009) [2023-12-27 11:37:08,430][20347] Updated weights for policy 0, policy_version 863161 (0.0009) [2023-12-27 11:37:08,497][20347] Updated weights for policy 0, policy_version 863171 (0.0008) [2023-12-27 11:37:08,562][20347] Updated weights for policy 0, policy_version 863181 (0.0008) [2023-12-27 11:37:08,942][20381] Updated weights for policy 1, policy_version 862433 (0.0006) [2023-12-27 11:37:09,004][20381] Updated weights for policy 1, policy_version 862443 (0.0007) [2023-12-27 11:37:09,061][20381] Updated weights for policy 1, policy_version 862453 (0.0005) [2023-12-27 11:37:09,253][20347] Updated weights for policy 0, policy_version 863191 (0.0009) [2023-12-27 11:37:09,317][20347] Updated weights for policy 0, policy_version 863201 (0.0008) [2023-12-27 11:37:09,319][20291] KL-divergence is very high: 259.1429 [2023-12-27 11:37:09,326][20291] KL-divergence is very high: 257.9880 [2023-12-27 11:37:09,333][20291] KL-divergence is very high: 169.4728 [2023-12-27 11:37:09,348][20291] KL-divergence is very high: 241.1395 [2023-12-27 11:37:09,376][20291] KL-divergence is very high: 293.9063 [2023-12-27 11:37:09,382][20291] KL-divergence is very high: 230.2140 [2023-12-27 11:37:09,389][20347] Updated weights for policy 0, policy_version 863211 (0.0007) [2023-12-27 11:37:09,404][20291] KL-divergence is very high: 199.9523 [2023-12-27 11:37:09,722][20381] Updated weights for policy 1, policy_version 862463 (0.0006) [2023-12-27 11:37:09,794][20381] Updated weights for policy 1, policy_version 862473 (0.0007) [2023-12-27 11:37:09,865][20381] Updated weights for policy 1, policy_version 862483 (0.0008) [2023-12-27 11:37:10,066][20291] KL-divergence is very high: 169.5641 [2023-12-27 11:37:10,073][20291] KL-divergence is very high: 127.0391 [2023-12-27 11:37:10,096][20347] Updated weights for policy 0, policy_version 863221 (0.0007) [2023-12-27 11:37:10,155][20347] Updated weights for policy 0, policy_version 863231 (0.0008) [2023-12-27 11:37:10,217][20347] Updated weights for policy 0, policy_version 863241 (0.0007) [2023-12-27 11:37:10,609][20381] Updated weights for policy 1, policy_version 862493 (0.0008) [2023-12-27 11:37:10,673][20381] Updated weights for policy 1, policy_version 862503 (0.0008) [2023-12-27 11:37:10,729][20381] Updated weights for policy 1, policy_version 862513 (0.0007) [2023-12-27 11:37:10,885][20347] Updated weights for policy 0, policy_version 863251 (0.0007) [2023-12-27 11:37:10,946][20347] Updated weights for policy 0, policy_version 863261 (0.0008) [2023-12-27 11:37:11,005][20347] Updated weights for policy 0, policy_version 863271 (0.0008) [2023-12-27 11:37:11,488][20381] Updated weights for policy 1, policy_version 862523 (0.0008) [2023-12-27 11:37:11,543][20381] Updated weights for policy 1, policy_version 862533 (0.0008) [2023-12-27 11:37:11,604][20381] Updated weights for policy 1, policy_version 862543 (0.0008) [2023-12-27 11:37:11,777][20347] Updated weights for policy 0, policy_version 863281 (0.0008) [2023-12-27 11:37:11,847][20347] Updated weights for policy 0, policy_version 863291 (0.0008) [2023-12-27 11:37:11,914][20347] Updated weights for policy 0, policy_version 863301 (0.0007) [2023-12-27 11:37:11,973][20347] Updated weights for policy 0, policy_version 863311 (0.0008) [2023-12-27 11:37:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20188.4). Total num frames: 441876480. Throughput: 0: 10008.5, 1: 10284.4. Samples: 441871124. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:37:12,363][19170] Avg episode reward: [(0, '696.550'), (1, '681.110')] [2023-12-27 11:37:12,390][20381] Updated weights for policy 1, policy_version 862553 (0.0008) [2023-12-27 11:37:12,443][20381] Updated weights for policy 1, policy_version 862563 (0.0006) [2023-12-27 11:37:12,495][20381] Updated weights for policy 1, policy_version 862573 (0.0009) [2023-12-27 11:37:12,551][20381] Updated weights for policy 1, policy_version 862583 (0.0008) [2023-12-27 11:37:12,704][20347] Updated weights for policy 0, policy_version 863321 (0.0007) [2023-12-27 11:37:12,771][20347] Updated weights for policy 0, policy_version 863331 (0.0008) [2023-12-27 11:37:12,837][20347] Updated weights for policy 0, policy_version 863341 (0.0008) [2023-12-27 11:37:13,292][20381] Updated weights for policy 1, policy_version 862593 (0.0007) [2023-12-27 11:37:13,353][20381] Updated weights for policy 1, policy_version 862603 (0.0006) [2023-12-27 11:37:13,411][20381] Updated weights for policy 1, policy_version 862613 (0.0008) [2023-12-27 11:37:13,548][20347] Updated weights for policy 0, policy_version 863351 (0.0008) [2023-12-27 11:37:13,607][20347] Updated weights for policy 0, policy_version 863361 (0.0007) [2023-12-27 11:37:13,665][20347] Updated weights for policy 0, policy_version 863371 (0.0006) [2023-12-27 11:37:14,073][20381] Updated weights for policy 1, policy_version 862623 (0.0007) [2023-12-27 11:37:14,133][20381] Updated weights for policy 1, policy_version 862633 (0.0008) [2023-12-27 11:37:14,193][20381] Updated weights for policy 1, policy_version 862643 (0.0008) [2023-12-27 11:37:14,255][20347] Updated weights for policy 0, policy_version 863381 (0.0006) [2023-12-27 11:37:14,316][20347] Updated weights for policy 0, policy_version 863391 (0.0008) [2023-12-27 11:37:14,373][20347] Updated weights for policy 0, policy_version 863401 (0.0007) [2023-12-27 11:37:14,884][20381] Updated weights for policy 1, policy_version 862653 (0.0008) [2023-12-27 11:37:14,945][20381] Updated weights for policy 1, policy_version 862663 (0.0008) [2023-12-27 11:37:15,010][20381] Updated weights for policy 1, policy_version 862673 (0.0008) [2023-12-27 11:37:15,047][20347] Updated weights for policy 0, policy_version 863411 (0.0006) [2023-12-27 11:37:15,106][20347] Updated weights for policy 0, policy_version 863421 (0.0007) [2023-12-27 11:37:15,170][20347] Updated weights for policy 0, policy_version 863431 (0.0007) [2023-12-27 11:37:15,806][20381] Updated weights for policy 1, policy_version 862683 (0.0008) [2023-12-27 11:37:15,866][20381] Updated weights for policy 1, policy_version 862693 (0.0008) [2023-12-27 11:37:15,895][20347] Updated weights for policy 0, policy_version 863441 (0.0007) [2023-12-27 11:37:15,924][20381] Updated weights for policy 1, policy_version 862703 (0.0006) [2023-12-27 11:37:15,953][20347] Updated weights for policy 0, policy_version 863451 (0.0008) [2023-12-27 11:37:16,014][20347] Updated weights for policy 0, policy_version 863461 (0.0008) [2023-12-27 11:37:16,071][20347] Updated weights for policy 0, policy_version 863471 (0.0008) [2023-12-27 11:37:16,550][20381] Updated weights for policy 1, policy_version 862713 (0.0006) [2023-12-27 11:37:16,602][20381] Updated weights for policy 1, policy_version 862723 (0.0008) [2023-12-27 11:37:16,659][20381] Updated weights for policy 1, policy_version 862733 (0.0008) [2023-12-27 11:37:16,721][20381] Updated weights for policy 1, policy_version 862743 (0.0008) [2023-12-27 11:37:16,773][20347] Updated weights for policy 0, policy_version 863481 (0.0009) [2023-12-27 11:37:16,823][20347] Updated weights for policy 0, policy_version 863491 (0.0008) [2023-12-27 11:37:16,880][20347] Updated weights for policy 0, policy_version 863501 (0.0008) [2023-12-27 11:37:17,347][20292] KL-divergence is very high: 172.9539 [2023-12-27 11:37:17,352][20292] KL-divergence is very high: 163.1302 [2023-12-27 11:37:17,357][20381] Updated weights for policy 1, policy_version 862753 (0.0006) [2023-12-27 11:37:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 20188.4). Total num frames: 441974784. Throughput: 0: 9918.7, 1: 10165.6. Samples: 441987988. Policy #0 lag: (min: 22.0, avg: 28.4, max: 54.0) [2023-12-27 11:37:17,363][19170] Avg episode reward: [(0, '770.470'), (1, '737.340')] [2023-12-27 11:37:17,405][20381] Updated weights for policy 1, policy_version 862763 (0.0007) [2023-12-27 11:37:17,459][20381] Updated weights for policy 1, policy_version 862773 (0.0008) [2023-12-27 11:37:17,583][20347] Updated weights for policy 0, policy_version 863511 (0.0008) [2023-12-27 11:37:17,637][20347] Updated weights for policy 0, policy_version 863521 (0.0008) [2023-12-27 11:37:17,690][20347] Updated weights for policy 0, policy_version 863531 (0.0007) [2023-12-27 11:37:18,099][20381] Updated weights for policy 1, policy_version 862783 (0.0008) [2023-12-27 11:37:18,155][20381] Updated weights for policy 1, policy_version 862793 (0.0008) [2023-12-27 11:37:18,212][20381] Updated weights for policy 1, policy_version 862803 (0.0007) [2023-12-27 11:37:18,465][20347] Updated weights for policy 0, policy_version 863541 (0.0008) [2023-12-27 11:37:18,538][20347] Updated weights for policy 0, policy_version 863551 (0.0008) [2023-12-27 11:37:18,601][20347] Updated weights for policy 0, policy_version 863561 (0.0009) [2023-12-27 11:37:18,979][20381] Updated weights for policy 1, policy_version 862813 (0.0007) [2023-12-27 11:37:19,042][20381] Updated weights for policy 1, policy_version 862823 (0.0008) [2023-12-27 11:37:19,102][20381] Updated weights for policy 1, policy_version 862833 (0.0007) [2023-12-27 11:37:19,275][20347] Updated weights for policy 0, policy_version 863571 (0.0008) [2023-12-27 11:37:19,335][20347] Updated weights for policy 0, policy_version 863581 (0.0008) [2023-12-27 11:37:19,402][20347] Updated weights for policy 0, policy_version 863591 (0.0007) [2023-12-27 11:37:19,864][20381] Updated weights for policy 1, policy_version 862843 (0.0008) [2023-12-27 11:37:19,935][20381] Updated weights for policy 1, policy_version 862853 (0.0009) [2023-12-27 11:37:19,995][20381] Updated weights for policy 1, policy_version 862863 (0.0006) [2023-12-27 11:37:20,138][20347] Updated weights for policy 0, policy_version 863601 (0.0006) [2023-12-27 11:37:20,197][20347] Updated weights for policy 0, policy_version 863611 (0.0007) [2023-12-27 11:37:20,254][20347] Updated weights for policy 0, policy_version 863621 (0.0008) [2023-12-27 11:37:20,327][20347] Updated weights for policy 0, policy_version 863631 (0.0008) [2023-12-27 11:37:20,759][20381] Updated weights for policy 1, policy_version 862873 (0.0008) [2023-12-27 11:37:20,832][20381] Updated weights for policy 1, policy_version 862883 (0.0008) [2023-12-27 11:37:20,902][20381] Updated weights for policy 1, policy_version 862893 (0.0008) [2023-12-27 11:37:20,965][20381] Updated weights for policy 1, policy_version 862903 (0.0008) [2023-12-27 11:37:21,063][20347] Updated weights for policy 0, policy_version 863641 (0.0007) [2023-12-27 11:37:21,132][20347] Updated weights for policy 0, policy_version 863651 (0.0007) [2023-12-27 11:37:21,198][20347] Updated weights for policy 0, policy_version 863661 (0.0007) [2023-12-27 11:37:21,680][20381] Updated weights for policy 1, policy_version 862913 (0.0007) [2023-12-27 11:37:21,753][20381] Updated weights for policy 1, policy_version 862923 (0.0008) [2023-12-27 11:37:21,820][20381] Updated weights for policy 1, policy_version 862933 (0.0008) [2023-12-27 11:37:21,995][20347] Updated weights for policy 0, policy_version 863671 (0.0008) [2023-12-27 11:37:22,059][20347] Updated weights for policy 0, policy_version 863681 (0.0007) [2023-12-27 11:37:22,125][20347] Updated weights for policy 0, policy_version 863691 (0.0007) [2023-12-27 11:37:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20188.4). Total num frames: 442073088. Throughput: 0: 9886.0, 1: 10083.8. Samples: 442046104. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:22,363][19170] Avg episode reward: [(0, '814.000'), (1, '607.810')] [2023-12-27 11:37:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000862936_220938240.pth... [2023-12-27 11:37:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000863696_221134848.pth... [2023-12-27 11:37:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000861816_220651520.pth [2023-12-27 11:37:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000862544_220839936.pth [2023-12-27 11:37:22,554][20381] Updated weights for policy 1, policy_version 862943 (0.0008) [2023-12-27 11:37:22,615][20381] Updated weights for policy 1, policy_version 862953 (0.0007) [2023-12-27 11:37:22,676][20381] Updated weights for policy 1, policy_version 862963 (0.0007) [2023-12-27 11:37:22,834][20347] Updated weights for policy 0, policy_version 863701 (0.0006) [2023-12-27 11:37:22,894][20347] Updated weights for policy 0, policy_version 863711 (0.0009) [2023-12-27 11:37:22,948][20347] Updated weights for policy 0, policy_version 863721 (0.0008) [2023-12-27 11:37:23,335][20381] Updated weights for policy 1, policy_version 862973 (0.0007) [2023-12-27 11:37:23,390][20381] Updated weights for policy 1, policy_version 862983 (0.0006) [2023-12-27 11:37:23,450][20381] Updated weights for policy 1, policy_version 862993 (0.0008) [2023-12-27 11:37:23,634][20347] Updated weights for policy 0, policy_version 863731 (0.0009) [2023-12-27 11:37:23,697][20347] Updated weights for policy 0, policy_version 863741 (0.0005) [2023-12-27 11:37:23,756][20347] Updated weights for policy 0, policy_version 863751 (0.0007) [2023-12-27 11:37:24,142][20381] Updated weights for policy 1, policy_version 863003 (0.0007) [2023-12-27 11:37:24,201][20381] Updated weights for policy 1, policy_version 863013 (0.0008) [2023-12-27 11:37:24,260][20381] Updated weights for policy 1, policy_version 863023 (0.0007) [2023-12-27 11:37:24,416][20347] Updated weights for policy 0, policy_version 863761 (0.0008) [2023-12-27 11:37:24,473][20347] Updated weights for policy 0, policy_version 863771 (0.0007) [2023-12-27 11:37:24,530][20347] Updated weights for policy 0, policy_version 863781 (0.0007) [2023-12-27 11:37:24,581][20347] Updated weights for policy 0, policy_version 863791 (0.0008) [2023-12-27 11:37:24,956][20381] Updated weights for policy 1, policy_version 863033 (0.0008) [2023-12-27 11:37:25,012][20381] Updated weights for policy 1, policy_version 863043 (0.0008) [2023-12-27 11:37:25,069][20381] Updated weights for policy 1, policy_version 863053 (0.0007) [2023-12-27 11:37:25,119][20381] Updated weights for policy 1, policy_version 863063 (0.0008) [2023-12-27 11:37:25,277][20347] Updated weights for policy 0, policy_version 863801 (0.0006) [2023-12-27 11:37:25,327][20347] Updated weights for policy 0, policy_version 863811 (0.0007) [2023-12-27 11:37:25,381][20347] Updated weights for policy 0, policy_version 863821 (0.0008) [2023-12-27 11:37:25,856][20381] Updated weights for policy 1, policy_version 863073 (0.0008) [2023-12-27 11:37:25,910][20381] Updated weights for policy 1, policy_version 863083 (0.0008) [2023-12-27 11:37:25,963][20381] Updated weights for policy 1, policy_version 863093 (0.0008) [2023-12-27 11:37:26,079][20347] Updated weights for policy 0, policy_version 863831 (0.0008) [2023-12-27 11:37:26,137][20347] Updated weights for policy 0, policy_version 863841 (0.0008) [2023-12-27 11:37:26,187][20347] Updated weights for policy 0, policy_version 863851 (0.0007) [2023-12-27 11:37:26,600][20381] Updated weights for policy 1, policy_version 863103 (0.0006) [2023-12-27 11:37:26,661][20381] Updated weights for policy 1, policy_version 863113 (0.0006) [2023-12-27 11:37:26,720][20381] Updated weights for policy 1, policy_version 863123 (0.0008) [2023-12-27 11:37:26,873][20347] Updated weights for policy 0, policy_version 863861 (0.0007) [2023-12-27 11:37:26,927][20347] Updated weights for policy 0, policy_version 863871 (0.0009) [2023-12-27 11:37:26,982][20347] Updated weights for policy 0, policy_version 863881 (0.0010) [2023-12-27 11:37:27,310][20381] Updated weights for policy 1, policy_version 863133 (0.0007) [2023-12-27 11:37:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20188.4). Total num frames: 442171392. Throughput: 0: 9804.1, 1: 10015.7. Samples: 442162588. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:27,364][19170] Avg episode reward: [(0, '669.730'), (1, '693.080')] [2023-12-27 11:37:27,374][20381] Updated weights for policy 1, policy_version 863143 (0.0007) [2023-12-27 11:37:27,435][20381] Updated weights for policy 1, policy_version 863153 (0.0006) [2023-12-27 11:37:27,689][20347] Updated weights for policy 0, policy_version 863891 (0.0008) [2023-12-27 11:37:27,740][20347] Updated weights for policy 0, policy_version 863901 (0.0006) [2023-12-27 11:37:27,795][20347] Updated weights for policy 0, policy_version 863911 (0.0008) [2023-12-27 11:37:28,114][20381] Updated weights for policy 1, policy_version 863163 (0.0007) [2023-12-27 11:37:28,175][20381] Updated weights for policy 1, policy_version 863173 (0.0008) [2023-12-27 11:37:28,231][20381] Updated weights for policy 1, policy_version 863183 (0.0007) [2023-12-27 11:37:28,497][20347] Updated weights for policy 0, policy_version 863921 (0.0008) [2023-12-27 11:37:28,552][20347] Updated weights for policy 0, policy_version 863931 (0.0009) [2023-12-27 11:37:28,612][20347] Updated weights for policy 0, policy_version 863941 (0.0007) [2023-12-27 11:37:28,674][20347] Updated weights for policy 0, policy_version 863951 (0.0007) [2023-12-27 11:37:28,910][20381] Updated weights for policy 1, policy_version 863193 (0.0008) [2023-12-27 11:37:28,967][20381] Updated weights for policy 1, policy_version 863203 (0.0007) [2023-12-27 11:37:29,016][20381] Updated weights for policy 1, policy_version 863213 (0.0008) [2023-12-27 11:37:29,073][20381] Updated weights for policy 1, policy_version 863223 (0.0008) [2023-12-27 11:37:29,397][20347] Updated weights for policy 0, policy_version 863961 (0.0008) [2023-12-27 11:37:29,458][20347] Updated weights for policy 0, policy_version 863971 (0.0005) [2023-12-27 11:37:29,519][20347] Updated weights for policy 0, policy_version 863981 (0.0007) [2023-12-27 11:37:29,839][20381] Updated weights for policy 1, policy_version 863233 (0.0010) [2023-12-27 11:37:29,896][20381] Updated weights for policy 1, policy_version 863243 (0.0008) [2023-12-27 11:37:29,956][20381] Updated weights for policy 1, policy_version 863253 (0.0008) [2023-12-27 11:37:30,202][20347] Updated weights for policy 0, policy_version 863991 (0.0008) [2023-12-27 11:37:30,263][20347] Updated weights for policy 0, policy_version 864001 (0.0011) [2023-12-27 11:37:30,320][20347] Updated weights for policy 0, policy_version 864011 (0.0009) [2023-12-27 11:37:30,670][20381] Updated weights for policy 1, policy_version 863263 (0.0011) [2023-12-27 11:37:30,724][20381] Updated weights for policy 1, policy_version 863273 (0.0010) [2023-12-27 11:37:30,779][20381] Updated weights for policy 1, policy_version 863283 (0.0010) [2023-12-27 11:37:30,998][20347] Updated weights for policy 0, policy_version 864021 (0.0008) [2023-12-27 11:37:31,060][20347] Updated weights for policy 0, policy_version 864031 (0.0007) [2023-12-27 11:37:31,110][20347] Updated weights for policy 0, policy_version 864041 (0.0009) [2023-12-27 11:37:31,505][20381] Updated weights for policy 1, policy_version 863293 (0.0010) [2023-12-27 11:37:31,555][20381] Updated weights for policy 1, policy_version 863303 (0.0011) [2023-12-27 11:37:31,609][20381] Updated weights for policy 1, policy_version 863313 (0.0011) [2023-12-27 11:37:31,870][20347] Updated weights for policy 0, policy_version 864051 (0.0010) [2023-12-27 11:37:31,928][20347] Updated weights for policy 0, policy_version 864061 (0.0007) [2023-12-27 11:37:31,989][20347] Updated weights for policy 0, policy_version 864071 (0.0008) [2023-12-27 11:37:32,340][20381] Updated weights for policy 1, policy_version 863323 (0.0011) [2023-12-27 11:37:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19933.8, 300 sec: 20188.4). Total num frames: 442269696. Throughput: 0: 9789.2, 1: 9933.2. Samples: 442281684. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:32,363][19170] Avg episode reward: [(0, '322.350'), (1, '669.370')] [2023-12-27 11:37:32,410][20381] Updated weights for policy 1, policy_version 863333 (0.0008) [2023-12-27 11:37:32,472][20381] Updated weights for policy 1, policy_version 863343 (0.0007) [2023-12-27 11:37:32,749][20347] Updated weights for policy 0, policy_version 864081 (0.0008) [2023-12-27 11:37:32,819][20347] Updated weights for policy 0, policy_version 864091 (0.0007) [2023-12-27 11:37:32,884][20347] Updated weights for policy 0, policy_version 864101 (0.0006) [2023-12-27 11:37:32,944][20347] Updated weights for policy 0, policy_version 864111 (0.0006) [2023-12-27 11:37:33,157][20381] Updated weights for policy 1, policy_version 863353 (0.0007) [2023-12-27 11:37:33,211][20381] Updated weights for policy 1, policy_version 863363 (0.0008) [2023-12-27 11:37:33,260][20381] Updated weights for policy 1, policy_version 863373 (0.0006) [2023-12-27 11:37:33,318][20381] Updated weights for policy 1, policy_version 863383 (0.0005) [2023-12-27 11:37:33,546][20347] Updated weights for policy 0, policy_version 864121 (0.0008) [2023-12-27 11:37:33,598][20347] Updated weights for policy 0, policy_version 864131 (0.0007) [2023-12-27 11:37:33,647][20347] Updated weights for policy 0, policy_version 864141 (0.0010) [2023-12-27 11:37:33,871][20381] Updated weights for policy 1, policy_version 863393 (0.0008) [2023-12-27 11:37:33,924][20381] Updated weights for policy 1, policy_version 863403 (0.0007) [2023-12-27 11:37:33,976][20381] Updated weights for policy 1, policy_version 863413 (0.0008) [2023-12-27 11:37:34,329][20347] Updated weights for policy 0, policy_version 864151 (0.0007) [2023-12-27 11:37:34,392][20347] Updated weights for policy 0, policy_version 864161 (0.0009) [2023-12-27 11:37:34,454][20347] Updated weights for policy 0, policy_version 864172 (0.0010) [2023-12-27 11:37:34,698][20381] Updated weights for policy 1, policy_version 863423 (0.0008) [2023-12-27 11:37:34,752][20381] Updated weights for policy 1, policy_version 863433 (0.0007) [2023-12-27 11:37:34,808][20381] Updated weights for policy 1, policy_version 863443 (0.0008) [2023-12-27 11:37:35,186][20347] Updated weights for policy 0, policy_version 864182 (0.0010) [2023-12-27 11:37:35,245][20347] Updated weights for policy 0, policy_version 864192 (0.0010) [2023-12-27 11:37:35,300][20347] Updated weights for policy 0, policy_version 864202 (0.0010) [2023-12-27 11:37:35,525][20381] Updated weights for policy 1, policy_version 863453 (0.0008) [2023-12-27 11:37:35,581][20381] Updated weights for policy 1, policy_version 863463 (0.0008) [2023-12-27 11:37:35,645][20381] Updated weights for policy 1, policy_version 863473 (0.0007) [2023-12-27 11:37:36,037][20347] Updated weights for policy 0, policy_version 864212 (0.0009) [2023-12-27 11:37:36,100][20347] Updated weights for policy 0, policy_version 864222 (0.0008) [2023-12-27 11:37:36,160][20347] Updated weights for policy 0, policy_version 864232 (0.0007) [2023-12-27 11:37:36,324][20381] Updated weights for policy 1, policy_version 863483 (0.0007) [2023-12-27 11:37:36,381][20381] Updated weights for policy 1, policy_version 863493 (0.0008) [2023-12-27 11:37:36,442][20381] Updated weights for policy 1, policy_version 863503 (0.0009) [2023-12-27 11:37:36,790][20347] Updated weights for policy 0, policy_version 864242 (0.0007) [2023-12-27 11:37:36,842][20347] Updated weights for policy 0, policy_version 864252 (0.0006) [2023-12-27 11:37:36,906][20347] Updated weights for policy 0, policy_version 864262 (0.0006) [2023-12-27 11:37:36,977][20347] Updated weights for policy 0, policy_version 864272 (0.0007) [2023-12-27 11:37:37,130][20381] Updated weights for policy 1, policy_version 863513 (0.0009) [2023-12-27 11:37:37,187][20381] Updated weights for policy 1, policy_version 863523 (0.0008) [2023-12-27 11:37:37,250][20381] Updated weights for policy 1, policy_version 863533 (0.0008) [2023-12-27 11:37:37,307][20381] Updated weights for policy 1, policy_version 863543 (0.0008) [2023-12-27 11:37:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 19933.8, 300 sec: 20216.2). Total num frames: 442376192. Throughput: 0: 9783.0, 1: 9888.5. Samples: 442342168. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:37,364][19170] Avg episode reward: [(0, '341.260'), (1, '688.650')] [2023-12-27 11:37:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000863544_221093888.pth... [2023-12-27 11:37:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000864272_221282304.pth... [2023-12-27 11:37:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000862360_220790784.pth [2023-12-27 11:37:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000863120_220987392.pth [2023-12-27 11:37:37,631][20347] Updated weights for policy 0, policy_version 864282 (0.0006) [2023-12-27 11:37:37,693][20347] Updated weights for policy 0, policy_version 864292 (0.0006) [2023-12-27 11:37:37,753][20347] Updated weights for policy 0, policy_version 864302 (0.0006) [2023-12-27 11:37:38,009][20381] Updated weights for policy 1, policy_version 863553 (0.0007) [2023-12-27 11:37:38,061][20381] Updated weights for policy 1, policy_version 863563 (0.0007) [2023-12-27 11:37:38,122][20381] Updated weights for policy 1, policy_version 863573 (0.0008) [2023-12-27 11:37:38,456][20347] Updated weights for policy 0, policy_version 864312 (0.0010) [2023-12-27 11:37:38,520][20347] Updated weights for policy 0, policy_version 864322 (0.0011) [2023-12-27 11:37:38,592][20347] Updated weights for policy 0, policy_version 864332 (0.0011) [2023-12-27 11:37:38,862][20381] Updated weights for policy 1, policy_version 863583 (0.0008) [2023-12-27 11:37:38,929][20381] Updated weights for policy 1, policy_version 863593 (0.0007) [2023-12-27 11:37:38,993][20381] Updated weights for policy 1, policy_version 863603 (0.0008) [2023-12-27 11:37:39,313][20347] Updated weights for policy 0, policy_version 864342 (0.0010) [2023-12-27 11:37:39,375][20347] Updated weights for policy 0, policy_version 864352 (0.0010) [2023-12-27 11:37:39,441][20347] Updated weights for policy 0, policy_version 864362 (0.0010) [2023-12-27 11:37:39,704][20381] Updated weights for policy 1, policy_version 863613 (0.0009) [2023-12-27 11:37:39,765][20381] Updated weights for policy 1, policy_version 863623 (0.0010) [2023-12-27 11:37:39,830][20381] Updated weights for policy 1, policy_version 863633 (0.0011) [2023-12-27 11:37:40,257][20347] Updated weights for policy 0, policy_version 864372 (0.0010) [2023-12-27 11:37:40,313][20347] Updated weights for policy 0, policy_version 864382 (0.0009) [2023-12-27 11:37:40,374][20347] Updated weights for policy 0, policy_version 864392 (0.0009) [2023-12-27 11:37:40,603][20381] Updated weights for policy 1, policy_version 863643 (0.0009) [2023-12-27 11:37:40,657][20381] Updated weights for policy 1, policy_version 863653 (0.0010) [2023-12-27 11:37:40,716][20381] Updated weights for policy 1, policy_version 863663 (0.0009) [2023-12-27 11:37:41,202][20347] Updated weights for policy 0, policy_version 864402 (0.0008) [2023-12-27 11:37:41,268][20347] Updated weights for policy 0, policy_version 864412 (0.0008) [2023-12-27 11:37:41,330][20347] Updated weights for policy 0, policy_version 864422 (0.0008) [2023-12-27 11:37:41,398][20347] Updated weights for policy 0, policy_version 864432 (0.0008) [2023-12-27 11:37:41,511][20381] Updated weights for policy 1, policy_version 863673 (0.0009) [2023-12-27 11:37:41,573][20381] Updated weights for policy 1, policy_version 863683 (0.0011) [2023-12-27 11:37:41,636][20381] Updated weights for policy 1, policy_version 863693 (0.0010) [2023-12-27 11:37:41,700][20381] Updated weights for policy 1, policy_version 863703 (0.0012) [2023-12-27 11:37:42,073][20347] Updated weights for policy 0, policy_version 864442 (0.0007) [2023-12-27 11:37:42,133][20347] Updated weights for policy 0, policy_version 864452 (0.0008) [2023-12-27 11:37:42,194][20347] Updated weights for policy 0, policy_version 864462 (0.0008) [2023-12-27 11:37:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19797.3, 300 sec: 20188.4). Total num frames: 442466304. Throughput: 0: 9688.6, 1: 9777.7. Samples: 442457436. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:42,363][19170] Avg episode reward: [(0, '648.650'), (1, '796.130')] [2023-12-27 11:37:42,502][20381] Updated weights for policy 1, policy_version 863713 (0.0008) [2023-12-27 11:37:42,567][20381] Updated weights for policy 1, policy_version 863723 (0.0007) [2023-12-27 11:37:42,628][20381] Updated weights for policy 1, policy_version 863733 (0.0008) [2023-12-27 11:37:42,914][20347] Updated weights for policy 0, policy_version 864472 (0.0008) [2023-12-27 11:37:42,975][20347] Updated weights for policy 0, policy_version 864482 (0.0008) [2023-12-27 11:37:43,040][20347] Updated weights for policy 0, policy_version 864492 (0.0008) [2023-12-27 11:37:43,269][20381] Updated weights for policy 1, policy_version 863743 (0.0007) [2023-12-27 11:37:43,328][20381] Updated weights for policy 1, policy_version 863753 (0.0006) [2023-12-27 11:37:43,384][20381] Updated weights for policy 1, policy_version 863763 (0.0008) [2023-12-27 11:37:43,761][20347] Updated weights for policy 0, policy_version 864502 (0.0007) [2023-12-27 11:37:43,812][20347] Updated weights for policy 0, policy_version 864512 (0.0008) [2023-12-27 11:37:43,863][20347] Updated weights for policy 0, policy_version 864522 (0.0007) [2023-12-27 11:37:44,076][20381] Updated weights for policy 1, policy_version 863773 (0.0009) [2023-12-27 11:37:44,129][20381] Updated weights for policy 1, policy_version 863783 (0.0008) [2023-12-27 11:37:44,182][20381] Updated weights for policy 1, policy_version 863793 (0.0007) [2023-12-27 11:37:44,612][20347] Updated weights for policy 0, policy_version 864532 (0.0008) [2023-12-27 11:37:44,670][20347] Updated weights for policy 0, policy_version 864542 (0.0007) [2023-12-27 11:37:44,731][20347] Updated weights for policy 0, policy_version 864552 (0.0008) [2023-12-27 11:37:44,934][20381] Updated weights for policy 1, policy_version 863803 (0.0009) [2023-12-27 11:37:44,996][20381] Updated weights for policy 1, policy_version 863813 (0.0011) [2023-12-27 11:37:45,060][20381] Updated weights for policy 1, policy_version 863823 (0.0011) [2023-12-27 11:37:45,477][20347] Updated weights for policy 0, policy_version 864562 (0.0007) [2023-12-27 11:37:45,529][20347] Updated weights for policy 0, policy_version 864572 (0.0007) [2023-12-27 11:37:45,585][20347] Updated weights for policy 0, policy_version 864582 (0.0007) [2023-12-27 11:37:45,633][20347] Updated weights for policy 0, policy_version 864592 (0.0007) [2023-12-27 11:37:45,781][20381] Updated weights for policy 1, policy_version 863833 (0.0011) [2023-12-27 11:37:45,841][20381] Updated weights for policy 1, policy_version 863843 (0.0010) [2023-12-27 11:37:45,895][20381] Updated weights for policy 1, policy_version 863853 (0.0010) [2023-12-27 11:37:45,952][20381] Updated weights for policy 1, policy_version 863863 (0.0011) [2023-12-27 11:37:46,370][20347] Updated weights for policy 0, policy_version 864602 (0.0008) [2023-12-27 11:37:46,430][20347] Updated weights for policy 0, policy_version 864612 (0.0008) [2023-12-27 11:37:46,490][20347] Updated weights for policy 0, policy_version 864622 (0.0007) [2023-12-27 11:37:46,631][20381] Updated weights for policy 1, policy_version 863873 (0.0010) [2023-12-27 11:37:46,687][20381] Updated weights for policy 1, policy_version 863883 (0.0011) [2023-12-27 11:37:46,751][20381] Updated weights for policy 1, policy_version 863893 (0.0010) [2023-12-27 11:37:47,211][20347] Updated weights for policy 0, policy_version 864632 (0.0008) [2023-12-27 11:37:47,267][20291] KL-divergence is very high: 147.7921 [2023-12-27 11:37:47,273][20291] KL-divergence is very high: 163.9635 [2023-12-27 11:37:47,273][20347] Updated weights for policy 0, policy_version 864642 (0.0007) [2023-12-27 11:37:47,278][20291] KL-divergence is very high: 169.5149 [2023-12-27 11:37:47,285][20291] KL-divergence is very high: 170.3137 [2023-12-27 11:37:47,303][20291] KL-divergence is very high: 167.6539 [2023-12-27 11:37:47,315][20291] KL-divergence is very high: 146.9037 [2023-12-27 11:37:47,323][20291] KL-divergence is very high: 131.2276 [2023-12-27 11:37:47,330][20291] KL-divergence is very high: 111.6611 [2023-12-27 11:37:47,337][20347] Updated weights for policy 0, policy_version 864652 (0.0007) [2023-12-27 11:37:47,362][19170] Fps is (10 sec: 18842.2, 60 sec: 19660.8, 300 sec: 20188.4). Total num frames: 442564608. Throughput: 0: 9683.1, 1: 9800.8. Samples: 442572704. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:47,363][19170] Avg episode reward: [(0, '773.010'), (1, '816.020')] [2023-12-27 11:37:47,502][20381] Updated weights for policy 1, policy_version 863903 (0.0011) [2023-12-27 11:37:47,562][20381] Updated weights for policy 1, policy_version 863913 (0.0008) [2023-12-27 11:37:47,618][20381] Updated weights for policy 1, policy_version 863923 (0.0007) [2023-12-27 11:37:48,048][20347] Updated weights for policy 0, policy_version 864662 (0.0008) [2023-12-27 11:37:48,106][20347] Updated weights for policy 0, policy_version 864672 (0.0007) [2023-12-27 11:37:48,157][20347] Updated weights for policy 0, policy_version 864682 (0.0007) [2023-12-27 11:37:48,330][20381] Updated weights for policy 1, policy_version 863933 (0.0009) [2023-12-27 11:37:48,402][20381] Updated weights for policy 1, policy_version 863943 (0.0010) [2023-12-27 11:37:48,470][20381] Updated weights for policy 1, policy_version 863953 (0.0011) [2023-12-27 11:37:48,932][20347] Updated weights for policy 0, policy_version 864692 (0.0007) [2023-12-27 11:37:48,990][20347] Updated weights for policy 0, policy_version 864702 (0.0008) [2023-12-27 11:37:49,046][20347] Updated weights for policy 0, policy_version 864712 (0.0007) [2023-12-27 11:37:49,152][20381] Updated weights for policy 1, policy_version 863963 (0.0010) [2023-12-27 11:37:49,218][20381] Updated weights for policy 1, policy_version 863973 (0.0010) [2023-12-27 11:37:49,281][20381] Updated weights for policy 1, policy_version 863983 (0.0009) [2023-12-27 11:37:49,774][20347] Updated weights for policy 0, policy_version 864722 (0.0008) [2023-12-27 11:37:49,839][20347] Updated weights for policy 0, policy_version 864732 (0.0008) [2023-12-27 11:37:49,898][20347] Updated weights for policy 0, policy_version 864742 (0.0007) [2023-12-27 11:37:49,970][20347] Updated weights for policy 0, policy_version 864752 (0.0008) [2023-12-27 11:37:50,074][20381] Updated weights for policy 1, policy_version 863993 (0.0010) [2023-12-27 11:37:50,138][20381] Updated weights for policy 1, policy_version 864003 (0.0008) [2023-12-27 11:37:50,198][20381] Updated weights for policy 1, policy_version 864013 (0.0008) [2023-12-27 11:37:50,264][20381] Updated weights for policy 1, policy_version 864023 (0.0008) [2023-12-27 11:37:50,751][20347] Updated weights for policy 0, policy_version 864762 (0.0008) [2023-12-27 11:37:50,810][20347] Updated weights for policy 0, policy_version 864772 (0.0008) [2023-12-27 11:37:50,870][20347] Updated weights for policy 0, policy_version 864782 (0.0007) [2023-12-27 11:37:50,964][20381] Updated weights for policy 1, policy_version 864033 (0.0008) [2023-12-27 11:37:51,033][20381] Updated weights for policy 1, policy_version 864043 (0.0008) [2023-12-27 11:37:51,083][20381] Updated weights for policy 1, policy_version 864053 (0.0008) [2023-12-27 11:37:51,567][20347] Updated weights for policy 0, policy_version 864792 (0.0007) [2023-12-27 11:37:51,633][20347] Updated weights for policy 0, policy_version 864802 (0.0008) [2023-12-27 11:37:51,689][20347] Updated weights for policy 0, policy_version 864812 (0.0006) [2023-12-27 11:37:51,844][20381] Updated weights for policy 1, policy_version 864063 (0.0008) [2023-12-27 11:37:51,892][20381] Updated weights for policy 1, policy_version 864073 (0.0008) [2023-12-27 11:37:51,946][20381] Updated weights for policy 1, policy_version 864083 (0.0008) [2023-12-27 11:37:52,343][20347] Updated weights for policy 0, policy_version 864822 (0.0007) [2023-12-27 11:37:52,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19524.2, 300 sec: 20160.6). Total num frames: 442654720. Throughput: 0: 9665.0, 1: 9779.6. Samples: 442628724. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:52,363][19170] Avg episode reward: [(0, '606.480'), (1, '825.570')] [2023-12-27 11:37:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000864088_221233152.pth... [2023-12-27 11:37:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000862936_220938240.pth [2023-12-27 11:37:52,416][20347] Updated weights for policy 0, policy_version 864833 (0.0008) [2023-12-27 11:37:52,469][20347] Updated weights for policy 0, policy_version 864843 (0.0007) [2023-12-27 11:37:52,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000864848_221429760.pth... [2023-12-27 11:37:52,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000863696_221134848.pth [2023-12-27 11:37:52,700][20381] Updated weights for policy 1, policy_version 864093 (0.0009) [2023-12-27 11:37:52,755][20381] Updated weights for policy 1, policy_version 864103 (0.0010) [2023-12-27 11:37:52,813][20381] Updated weights for policy 1, policy_version 864113 (0.0010) [2023-12-27 11:37:53,168][20347] Updated weights for policy 0, policy_version 864853 (0.0007) [2023-12-27 11:37:53,225][20347] Updated weights for policy 0, policy_version 864863 (0.0007) [2023-12-27 11:37:53,291][20347] Updated weights for policy 0, policy_version 864873 (0.0007) [2023-12-27 11:37:53,498][20381] Updated weights for policy 1, policy_version 864123 (0.0008) [2023-12-27 11:37:53,554][20381] Updated weights for policy 1, policy_version 864133 (0.0009) [2023-12-27 11:37:53,613][20381] Updated weights for policy 1, policy_version 864143 (0.0007) [2023-12-27 11:37:53,950][20347] Updated weights for policy 0, policy_version 864883 (0.0008) [2023-12-27 11:37:54,016][20347] Updated weights for policy 0, policy_version 864893 (0.0007) [2023-12-27 11:37:54,073][20347] Updated weights for policy 0, policy_version 864903 (0.0008) [2023-12-27 11:37:54,278][20381] Updated weights for policy 1, policy_version 864153 (0.0008) [2023-12-27 11:37:54,339][20381] Updated weights for policy 1, policy_version 864163 (0.0008) [2023-12-27 11:37:54,393][20381] Updated weights for policy 1, policy_version 864173 (0.0008) [2023-12-27 11:37:54,454][20381] Updated weights for policy 1, policy_version 864183 (0.0008) [2023-12-27 11:37:54,766][20347] Updated weights for policy 0, policy_version 864913 (0.0008) [2023-12-27 11:37:54,823][20347] Updated weights for policy 0, policy_version 864923 (0.0008) [2023-12-27 11:37:54,884][20347] Updated weights for policy 0, policy_version 864933 (0.0008) [2023-12-27 11:37:54,944][20347] Updated weights for policy 0, policy_version 864943 (0.0008) [2023-12-27 11:37:55,161][20381] Updated weights for policy 1, policy_version 864193 (0.0007) [2023-12-27 11:37:55,218][20381] Updated weights for policy 1, policy_version 864203 (0.0008) [2023-12-27 11:37:55,271][20381] Updated weights for policy 1, policy_version 864213 (0.0008) [2023-12-27 11:37:55,625][20347] Updated weights for policy 0, policy_version 864953 (0.0008) [2023-12-27 11:37:55,678][20347] Updated weights for policy 0, policy_version 864963 (0.0008) [2023-12-27 11:37:55,735][20347] Updated weights for policy 0, policy_version 864973 (0.0007) [2023-12-27 11:37:55,973][20381] Updated weights for policy 1, policy_version 864223 (0.0008) [2023-12-27 11:37:56,028][20381] Updated weights for policy 1, policy_version 864233 (0.0007) [2023-12-27 11:37:56,081][20381] Updated weights for policy 1, policy_version 864243 (0.0008) [2023-12-27 11:37:56,436][20347] Updated weights for policy 0, policy_version 864983 (0.0008) [2023-12-27 11:37:56,496][20347] Updated weights for policy 0, policy_version 864993 (0.0008) [2023-12-27 11:37:56,550][20347] Updated weights for policy 0, policy_version 865003 (0.0007) [2023-12-27 11:37:56,756][20381] Updated weights for policy 1, policy_version 864253 (0.0007) [2023-12-27 11:37:56,805][20381] Updated weights for policy 1, policy_version 864263 (0.0008) [2023-12-27 11:37:56,869][20381] Updated weights for policy 1, policy_version 864273 (0.0009) [2023-12-27 11:37:57,195][20347] Updated weights for policy 0, policy_version 865013 (0.0006) [2023-12-27 11:37:57,259][20347] Updated weights for policy 0, policy_version 865023 (0.0006) [2023-12-27 11:37:57,325][20347] Updated weights for policy 0, policy_version 865033 (0.0006) [2023-12-27 11:37:57,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 20160.7). Total num frames: 442753024. Throughput: 0: 9710.5, 1: 9753.3. Samples: 442746992. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:37:57,363][19170] Avg episode reward: [(0, '616.520'), (1, '863.700')] [2023-12-27 11:37:57,536][20381] Updated weights for policy 1, policy_version 864283 (0.0008) [2023-12-27 11:37:57,593][20381] Updated weights for policy 1, policy_version 864293 (0.0006) [2023-12-27 11:37:57,651][20381] Updated weights for policy 1, policy_version 864303 (0.0007) [2023-12-27 11:37:57,894][20347] Updated weights for policy 0, policy_version 865043 (0.0007) [2023-12-27 11:37:57,952][20347] Updated weights for policy 0, policy_version 865053 (0.0007) [2023-12-27 11:37:58,008][20347] Updated weights for policy 0, policy_version 865063 (0.0007) [2023-12-27 11:37:58,364][20381] Updated weights for policy 1, policy_version 864313 (0.0007) [2023-12-27 11:37:58,430][20381] Updated weights for policy 1, policy_version 864323 (0.0007) [2023-12-27 11:37:58,495][20381] Updated weights for policy 1, policy_version 864333 (0.0008) [2023-12-27 11:37:58,561][20381] Updated weights for policy 1, policy_version 864343 (0.0007) [2023-12-27 11:37:58,806][20347] Updated weights for policy 0, policy_version 865073 (0.0008) [2023-12-27 11:37:58,879][20347] Updated weights for policy 0, policy_version 865083 (0.0008) [2023-12-27 11:37:58,956][20347] Updated weights for policy 0, policy_version 865093 (0.0008) [2023-12-27 11:37:59,020][20347] Updated weights for policy 0, policy_version 865103 (0.0008) [2023-12-27 11:37:59,400][20381] Updated weights for policy 1, policy_version 864353 (0.0008) [2023-12-27 11:37:59,467][20381] Updated weights for policy 1, policy_version 864363 (0.0007) [2023-12-27 11:37:59,531][20381] Updated weights for policy 1, policy_version 864373 (0.0008) [2023-12-27 11:37:59,815][20347] Updated weights for policy 0, policy_version 865113 (0.0010) [2023-12-27 11:37:59,878][20347] Updated weights for policy 0, policy_version 865123 (0.0008) [2023-12-27 11:37:59,939][20347] Updated weights for policy 0, policy_version 865133 (0.0007) [2023-12-27 11:38:00,247][20381] Updated weights for policy 1, policy_version 864383 (0.0008) [2023-12-27 11:38:00,309][20381] Updated weights for policy 1, policy_version 864393 (0.0007) [2023-12-27 11:38:00,375][20381] Updated weights for policy 1, policy_version 864403 (0.0007) [2023-12-27 11:38:00,666][20347] Updated weights for policy 0, policy_version 865143 (0.0007) [2023-12-27 11:38:00,729][20347] Updated weights for policy 0, policy_version 865153 (0.0006) [2023-12-27 11:38:00,794][20347] Updated weights for policy 0, policy_version 865163 (0.0010) [2023-12-27 11:38:00,984][20381] Updated weights for policy 1, policy_version 864413 (0.0006) [2023-12-27 11:38:01,054][20381] Updated weights for policy 1, policy_version 864423 (0.0007) [2023-12-27 11:38:01,113][20381] Updated weights for policy 1, policy_version 864433 (0.0007) [2023-12-27 11:38:01,491][20347] Updated weights for policy 0, policy_version 865173 (0.0009) [2023-12-27 11:38:01,550][20347] Updated weights for policy 0, policy_version 865183 (0.0006) [2023-12-27 11:38:01,614][20347] Updated weights for policy 0, policy_version 865193 (0.0007) [2023-12-27 11:38:01,838][20381] Updated weights for policy 1, policy_version 864443 (0.0007) [2023-12-27 11:38:01,891][20381] Updated weights for policy 1, policy_version 864453 (0.0007) [2023-12-27 11:38:01,941][20381] Updated weights for policy 1, policy_version 864463 (0.0007) [2023-12-27 11:38:02,312][20347] Updated weights for policy 0, policy_version 865203 (0.0009) [2023-12-27 11:38:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19524.3, 300 sec: 20160.7). Total num frames: 442851328. Throughput: 0: 9719.5, 1: 9741.7. Samples: 442863740. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:02,363][19170] Avg episode reward: [(0, '628.730'), (1, '853.700')] [2023-12-27 11:38:02,378][20347] Updated weights for policy 0, policy_version 865213 (0.0008) [2023-12-27 11:38:02,438][20347] Updated weights for policy 0, policy_version 865223 (0.0008) [2023-12-27 11:38:02,689][20381] Updated weights for policy 1, policy_version 864473 (0.0008) [2023-12-27 11:38:02,746][20381] Updated weights for policy 1, policy_version 864483 (0.0008) [2023-12-27 11:38:02,803][20381] Updated weights for policy 1, policy_version 864493 (0.0007) [2023-12-27 11:38:02,857][20381] Updated weights for policy 1, policy_version 864503 (0.0005) [2023-12-27 11:38:03,147][20347] Updated weights for policy 0, policy_version 865233 (0.0008) [2023-12-27 11:38:03,204][20347] Updated weights for policy 0, policy_version 865243 (0.0010) [2023-12-27 11:38:03,256][20347] Updated weights for policy 0, policy_version 865253 (0.0010) [2023-12-27 11:38:03,313][20347] Updated weights for policy 0, policy_version 865263 (0.0010) [2023-12-27 11:38:03,522][20381] Updated weights for policy 1, policy_version 864513 (0.0007) [2023-12-27 11:38:03,577][20381] Updated weights for policy 1, policy_version 864523 (0.0007) [2023-12-27 11:38:03,631][20381] Updated weights for policy 1, policy_version 864533 (0.0007) [2023-12-27 11:38:03,992][20347] Updated weights for policy 0, policy_version 865273 (0.0010) [2023-12-27 11:38:04,049][20347] Updated weights for policy 0, policy_version 865283 (0.0011) [2023-12-27 11:38:04,105][20347] Updated weights for policy 0, policy_version 865293 (0.0011) [2023-12-27 11:38:04,347][20381] Updated weights for policy 1, policy_version 864543 (0.0007) [2023-12-27 11:38:04,413][20381] Updated weights for policy 1, policy_version 864553 (0.0007) [2023-12-27 11:38:04,474][20381] Updated weights for policy 1, policy_version 864563 (0.0008) [2023-12-27 11:38:04,778][20347] Updated weights for policy 0, policy_version 865303 (0.0009) [2023-12-27 11:38:04,844][20347] Updated weights for policy 0, policy_version 865313 (0.0008) [2023-12-27 11:38:04,906][20347] Updated weights for policy 0, policy_version 865323 (0.0011) [2023-12-27 11:38:05,206][20381] Updated weights for policy 1, policy_version 864573 (0.0008) [2023-12-27 11:38:05,267][20381] Updated weights for policy 1, policy_version 864583 (0.0008) [2023-12-27 11:38:05,328][20381] Updated weights for policy 1, policy_version 864593 (0.0008) [2023-12-27 11:38:05,571][20347] Updated weights for policy 0, policy_version 865333 (0.0008) [2023-12-27 11:38:05,624][20347] Updated weights for policy 0, policy_version 865343 (0.0008) [2023-12-27 11:38:05,682][20347] Updated weights for policy 0, policy_version 865353 (0.0006) [2023-12-27 11:38:06,081][20381] Updated weights for policy 1, policy_version 864603 (0.0007) [2023-12-27 11:38:06,151][20381] Updated weights for policy 1, policy_version 864613 (0.0008) [2023-12-27 11:38:06,210][20381] Updated weights for policy 1, policy_version 864623 (0.0009) [2023-12-27 11:38:06,410][20347] Updated weights for policy 0, policy_version 865363 (0.0007) [2023-12-27 11:38:06,476][20347] Updated weights for policy 0, policy_version 865373 (0.0008) [2023-12-27 11:38:06,545][20347] Updated weights for policy 0, policy_version 865383 (0.0008) [2023-12-27 11:38:06,904][20381] Updated weights for policy 1, policy_version 864633 (0.0008) [2023-12-27 11:38:06,962][20381] Updated weights for policy 1, policy_version 864643 (0.0008) [2023-12-27 11:38:07,018][20381] Updated weights for policy 1, policy_version 864653 (0.0008) [2023-12-27 11:38:07,082][20381] Updated weights for policy 1, policy_version 864663 (0.0008) [2023-12-27 11:38:07,242][20347] Updated weights for policy 0, policy_version 865393 (0.0008) [2023-12-27 11:38:07,306][20347] Updated weights for policy 0, policy_version 865403 (0.0008) [2023-12-27 11:38:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19524.3, 300 sec: 20188.4). Total num frames: 442949632. Throughput: 0: 9737.6, 1: 9760.2. Samples: 442923504. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:07,363][19170] Avg episode reward: [(0, '589.320'), (1, '833.970')] [2023-12-27 11:38:07,366][20347] Updated weights for policy 0, policy_version 865413 (0.0007) [2023-12-27 11:38:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000864664_221380608.pth... [2023-12-27 11:38:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000863544_221093888.pth [2023-12-27 11:38:07,424][20347] Updated weights for policy 0, policy_version 865423 (0.0007) [2023-12-27 11:38:07,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000865424_221577216.pth... [2023-12-27 11:38:07,435][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000864272_221282304.pth [2023-12-27 11:38:07,767][20381] Updated weights for policy 1, policy_version 864673 (0.0008) [2023-12-27 11:38:07,820][20381] Updated weights for policy 1, policy_version 864683 (0.0009) [2023-12-27 11:38:07,874][20381] Updated weights for policy 1, policy_version 864693 (0.0006) [2023-12-27 11:38:08,131][20347] Updated weights for policy 0, policy_version 865433 (0.0008) [2023-12-27 11:38:08,190][20347] Updated weights for policy 0, policy_version 865443 (0.0007) [2023-12-27 11:38:08,252][20347] Updated weights for policy 0, policy_version 865453 (0.0008) [2023-12-27 11:38:08,634][20381] Updated weights for policy 1, policy_version 864703 (0.0008) [2023-12-27 11:38:08,696][20381] Updated weights for policy 1, policy_version 864713 (0.0008) [2023-12-27 11:38:08,754][20381] Updated weights for policy 1, policy_version 864723 (0.0008) [2023-12-27 11:38:08,980][20347] Updated weights for policy 0, policy_version 865463 (0.0007) [2023-12-27 11:38:09,040][20347] Updated weights for policy 0, policy_version 865473 (0.0007) [2023-12-27 11:38:09,103][20347] Updated weights for policy 0, policy_version 865483 (0.0009) [2023-12-27 11:38:09,489][20381] Updated weights for policy 1, policy_version 864733 (0.0008) [2023-12-27 11:38:09,552][20381] Updated weights for policy 1, policy_version 864743 (0.0008) [2023-12-27 11:38:09,612][20381] Updated weights for policy 1, policy_version 864753 (0.0006) [2023-12-27 11:38:09,866][20347] Updated weights for policy 0, policy_version 865493 (0.0007) [2023-12-27 11:38:09,933][20347] Updated weights for policy 0, policy_version 865503 (0.0008) [2023-12-27 11:38:09,999][20347] Updated weights for policy 0, policy_version 865513 (0.0007) [2023-12-27 11:38:10,347][20381] Updated weights for policy 1, policy_version 864763 (0.0009) [2023-12-27 11:38:10,412][20381] Updated weights for policy 1, policy_version 864773 (0.0011) [2023-12-27 11:38:10,481][20381] Updated weights for policy 1, policy_version 864783 (0.0010) [2023-12-27 11:38:10,720][20347] Updated weights for policy 0, policy_version 865523 (0.0008) [2023-12-27 11:38:10,780][20347] Updated weights for policy 0, policy_version 865533 (0.0007) [2023-12-27 11:38:10,844][20347] Updated weights for policy 0, policy_version 865543 (0.0008) [2023-12-27 11:38:11,172][20381] Updated weights for policy 1, policy_version 864793 (0.0007) [2023-12-27 11:38:11,242][20381] Updated weights for policy 1, policy_version 864803 (0.0007) [2023-12-27 11:38:11,304][20381] Updated weights for policy 1, policy_version 864813 (0.0007) [2023-12-27 11:38:11,366][20381] Updated weights for policy 1, policy_version 864823 (0.0007) [2023-12-27 11:38:11,586][20347] Updated weights for policy 0, policy_version 865553 (0.0007) [2023-12-27 11:38:11,658][20347] Updated weights for policy 0, policy_version 865563 (0.0008) [2023-12-27 11:38:11,732][20347] Updated weights for policy 0, policy_version 865573 (0.0009) [2023-12-27 11:38:11,797][20347] Updated weights for policy 0, policy_version 865583 (0.0010) [2023-12-27 11:38:12,070][20381] Updated weights for policy 1, policy_version 864833 (0.0009) [2023-12-27 11:38:12,128][20381] Updated weights for policy 1, policy_version 864843 (0.0008) [2023-12-27 11:38:12,190][20381] Updated weights for policy 1, policy_version 864853 (0.0008) [2023-12-27 11:38:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 20160.6). Total num frames: 443047936. Throughput: 0: 9716.2, 1: 9751.6. Samples: 443038636. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:12,363][19170] Avg episode reward: [(0, '636.930'), (1, '833.960')] [2023-12-27 11:38:12,549][20347] Updated weights for policy 0, policy_version 865593 (0.0008) [2023-12-27 11:38:12,610][20347] Updated weights for policy 0, policy_version 865603 (0.0008) [2023-12-27 11:38:12,670][20347] Updated weights for policy 0, policy_version 865613 (0.0011) [2023-12-27 11:38:12,900][20381] Updated weights for policy 1, policy_version 864863 (0.0009) [2023-12-27 11:38:12,964][20381] Updated weights for policy 1, policy_version 864873 (0.0008) [2023-12-27 11:38:13,026][20381] Updated weights for policy 1, policy_version 864883 (0.0009) [2023-12-27 11:38:13,398][20347] Updated weights for policy 0, policy_version 865623 (0.0007) [2023-12-27 11:38:13,448][20347] Updated weights for policy 0, policy_version 865633 (0.0008) [2023-12-27 11:38:13,502][20347] Updated weights for policy 0, policy_version 865643 (0.0008) [2023-12-27 11:38:13,687][20381] Updated weights for policy 1, policy_version 864893 (0.0008) [2023-12-27 11:38:13,743][20381] Updated weights for policy 1, policy_version 864903 (0.0005) [2023-12-27 11:38:13,798][20381] Updated weights for policy 1, policy_version 864913 (0.0007) [2023-12-27 11:38:14,265][20347] Updated weights for policy 0, policy_version 865653 (0.0008) [2023-12-27 11:38:14,324][20347] Updated weights for policy 0, policy_version 865663 (0.0007) [2023-12-27 11:38:14,377][20347] Updated weights for policy 0, policy_version 865673 (0.0008) [2023-12-27 11:38:14,450][20381] Updated weights for policy 1, policy_version 864923 (0.0008) [2023-12-27 11:38:14,507][20381] Updated weights for policy 1, policy_version 864933 (0.0007) [2023-12-27 11:38:14,571][20381] Updated weights for policy 1, policy_version 864943 (0.0007) [2023-12-27 11:38:15,120][20347] Updated weights for policy 0, policy_version 865683 (0.0006) [2023-12-27 11:38:15,187][20347] Updated weights for policy 0, policy_version 865693 (0.0008) [2023-12-27 11:38:15,199][20381] Updated weights for policy 1, policy_version 864953 (0.0006) [2023-12-27 11:38:15,247][20347] Updated weights for policy 0, policy_version 865703 (0.0008) [2023-12-27 11:38:15,263][20381] Updated weights for policy 1, policy_version 864963 (0.0006) [2023-12-27 11:38:15,327][20381] Updated weights for policy 1, policy_version 864973 (0.0008) [2023-12-27 11:38:15,393][20381] Updated weights for policy 1, policy_version 864983 (0.0008) [2023-12-27 11:38:15,956][20347] Updated weights for policy 0, policy_version 865713 (0.0007) [2023-12-27 11:38:16,019][20347] Updated weights for policy 0, policy_version 865723 (0.0007) [2023-12-27 11:38:16,050][20381] Updated weights for policy 1, policy_version 864993 (0.0006) [2023-12-27 11:38:16,071][20347] Updated weights for policy 0, policy_version 865733 (0.0007) [2023-12-27 11:38:16,112][20381] Updated weights for policy 1, policy_version 865003 (0.0007) [2023-12-27 11:38:16,129][20347] Updated weights for policy 0, policy_version 865743 (0.0009) [2023-12-27 11:38:16,165][20381] Updated weights for policy 1, policy_version 865013 (0.0007) [2023-12-27 11:38:16,819][20347] Updated weights for policy 0, policy_version 865753 (0.0007) [2023-12-27 11:38:16,880][20347] Updated weights for policy 0, policy_version 865763 (0.0007) [2023-12-27 11:38:16,886][20381] Updated weights for policy 1, policy_version 865023 (0.0007) [2023-12-27 11:38:16,936][20347] Updated weights for policy 0, policy_version 865773 (0.0006) [2023-12-27 11:38:16,939][20381] Updated weights for policy 1, policy_version 865033 (0.0006) [2023-12-27 11:38:16,996][20381] Updated weights for policy 1, policy_version 865043 (0.0006) [2023-12-27 11:38:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 20216.2). Total num frames: 443146240. Throughput: 0: 9682.4, 1: 9743.8. Samples: 443155860. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:17,363][19170] Avg episode reward: [(0, '735.360'), (1, '824.200')] [2023-12-27 11:38:17,678][20347] Updated weights for policy 0, policy_version 865783 (0.0008) [2023-12-27 11:38:17,684][20381] Updated weights for policy 1, policy_version 865053 (0.0007) [2023-12-27 11:38:17,742][20381] Updated weights for policy 1, policy_version 865063 (0.0006) [2023-12-27 11:38:17,744][20347] Updated weights for policy 0, policy_version 865793 (0.0007) [2023-12-27 11:38:17,799][20347] Updated weights for policy 0, policy_version 865803 (0.0008) [2023-12-27 11:38:17,801][20381] Updated weights for policy 1, policy_version 865073 (0.0007) [2023-12-27 11:38:18,440][20381] Updated weights for policy 1, policy_version 865083 (0.0007) [2023-12-27 11:38:18,499][20381] Updated weights for policy 1, policy_version 865093 (0.0009) [2023-12-27 11:38:18,528][20347] Updated weights for policy 0, policy_version 865813 (0.0006) [2023-12-27 11:38:18,565][20381] Updated weights for policy 1, policy_version 865103 (0.0008) [2023-12-27 11:38:18,590][20347] Updated weights for policy 0, policy_version 865823 (0.0006) [2023-12-27 11:38:18,653][20347] Updated weights for policy 0, policy_version 865833 (0.0007) [2023-12-27 11:38:19,308][20347] Updated weights for policy 0, policy_version 865843 (0.0008) [2023-12-27 11:38:19,329][20381] Updated weights for policy 1, policy_version 865113 (0.0007) [2023-12-27 11:38:19,372][20347] Updated weights for policy 0, policy_version 865853 (0.0008) [2023-12-27 11:38:19,397][20381] Updated weights for policy 1, policy_version 865123 (0.0007) [2023-12-27 11:38:19,434][20347] Updated weights for policy 0, policy_version 865863 (0.0007) [2023-12-27 11:38:19,468][20381] Updated weights for policy 1, policy_version 865133 (0.0006) [2023-12-27 11:38:19,530][20381] Updated weights for policy 1, policy_version 865143 (0.0009) [2023-12-27 11:38:20,176][20347] Updated weights for policy 0, policy_version 865873 (0.0008) [2023-12-27 11:38:20,245][20347] Updated weights for policy 0, policy_version 865883 (0.0008) [2023-12-27 11:38:20,249][20381] Updated weights for policy 1, policy_version 865153 (0.0007) [2023-12-27 11:38:20,310][20347] Updated weights for policy 0, policy_version 865893 (0.0008) [2023-12-27 11:38:20,322][20381] Updated weights for policy 1, policy_version 865163 (0.0007) [2023-12-27 11:38:20,373][20347] Updated weights for policy 0, policy_version 865903 (0.0007) [2023-12-27 11:38:20,381][20381] Updated weights for policy 1, policy_version 865173 (0.0008) [2023-12-27 11:38:21,108][20381] Updated weights for policy 1, policy_version 865183 (0.0009) [2023-12-27 11:38:21,128][20347] Updated weights for policy 0, policy_version 865913 (0.0010) [2023-12-27 11:38:21,179][20381] Updated weights for policy 1, policy_version 865193 (0.0009) [2023-12-27 11:38:21,188][20347] Updated weights for policy 0, policy_version 865923 (0.0008) [2023-12-27 11:38:21,242][20381] Updated weights for policy 1, policy_version 865203 (0.0008) [2023-12-27 11:38:21,250][20347] Updated weights for policy 0, policy_version 865933 (0.0009) [2023-12-27 11:38:22,028][20347] Updated weights for policy 0, policy_version 865943 (0.0007) [2023-12-27 11:38:22,033][20381] Updated weights for policy 1, policy_version 865213 (0.0008) [2023-12-27 11:38:22,091][20347] Updated weights for policy 0, policy_version 865953 (0.0006) [2023-12-27 11:38:22,097][20381] Updated weights for policy 1, policy_version 865223 (0.0008) [2023-12-27 11:38:22,152][20347] Updated weights for policy 0, policy_version 865963 (0.0007) [2023-12-27 11:38:22,155][20381] Updated weights for policy 1, policy_version 865233 (0.0008) [2023-12-27 11:38:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 20188.4). Total num frames: 443244544. Throughput: 0: 9654.1, 1: 9724.8. Samples: 443214216. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:22,363][19170] Avg episode reward: [(0, '666.740'), (1, '795.120')] [2023-12-27 11:38:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000865240_221528064.pth... [2023-12-27 11:38:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000865968_221716480.pth... [2023-12-27 11:38:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000864088_221233152.pth [2023-12-27 11:38:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000864848_221429760.pth [2023-12-27 11:38:22,890][20381] Updated weights for policy 1, policy_version 865243 (0.0008) [2023-12-27 11:38:22,895][20347] Updated weights for policy 0, policy_version 865973 (0.0008) [2023-12-27 11:38:22,950][20381] Updated weights for policy 1, policy_version 865253 (0.0007) [2023-12-27 11:38:22,953][20347] Updated weights for policy 0, policy_version 865983 (0.0006) [2023-12-27 11:38:23,009][20381] Updated weights for policy 1, policy_version 865263 (0.0009) [2023-12-27 11:38:23,014][20347] Updated weights for policy 0, policy_version 865993 (0.0006) [2023-12-27 11:38:23,715][20381] Updated weights for policy 1, policy_version 865273 (0.0010) [2023-12-27 11:38:23,720][20347] Updated weights for policy 0, policy_version 866003 (0.0006) [2023-12-27 11:38:23,769][20381] Updated weights for policy 1, policy_version 865283 (0.0011) [2023-12-27 11:38:23,775][20347] Updated weights for policy 0, policy_version 866013 (0.0007) [2023-12-27 11:38:23,819][20381] Updated weights for policy 1, policy_version 865293 (0.0010) [2023-12-27 11:38:23,827][20347] Updated weights for policy 0, policy_version 866023 (0.0008) [2023-12-27 11:38:23,872][20381] Updated weights for policy 1, policy_version 865303 (0.0008) [2023-12-27 11:38:24,488][20347] Updated weights for policy 0, policy_version 866033 (0.0007) [2023-12-27 11:38:24,551][20347] Updated weights for policy 0, policy_version 866043 (0.0007) [2023-12-27 11:38:24,577][20381] Updated weights for policy 1, policy_version 865313 (0.0011) [2023-12-27 11:38:24,605][20347] Updated weights for policy 0, policy_version 866053 (0.0007) [2023-12-27 11:38:24,641][20381] Updated weights for policy 1, policy_version 865323 (0.0011) [2023-12-27 11:38:24,668][20347] Updated weights for policy 0, policy_version 866063 (0.0007) [2023-12-27 11:38:24,700][20381] Updated weights for policy 1, policy_version 865333 (0.0010) [2023-12-27 11:38:25,335][20347] Updated weights for policy 0, policy_version 866073 (0.0008) [2023-12-27 11:38:25,391][20347] Updated weights for policy 0, policy_version 866083 (0.0006) [2023-12-27 11:38:25,400][20381] Updated weights for policy 1, policy_version 865343 (0.0011) [2023-12-27 11:38:25,441][20347] Updated weights for policy 0, policy_version 866093 (0.0007) [2023-12-27 11:38:25,452][20381] Updated weights for policy 1, policy_version 865353 (0.0009) [2023-12-27 11:38:25,515][20381] Updated weights for policy 1, policy_version 865363 (0.0010) [2023-12-27 11:38:26,088][20347] Updated weights for policy 0, policy_version 866103 (0.0006) [2023-12-27 11:38:26,096][20381] Updated weights for policy 1, policy_version 865373 (0.0008) [2023-12-27 11:38:26,138][20347] Updated weights for policy 0, policy_version 866113 (0.0005) [2023-12-27 11:38:26,144][20381] Updated weights for policy 1, policy_version 865383 (0.0009) [2023-12-27 11:38:26,190][20347] Updated weights for policy 0, policy_version 866123 (0.0005) [2023-12-27 11:38:26,193][20381] Updated weights for policy 1, policy_version 865393 (0.0010) [2023-12-27 11:38:26,880][20347] Updated weights for policy 0, policy_version 866133 (0.0009) [2023-12-27 11:38:26,882][20381] Updated weights for policy 1, policy_version 865403 (0.0010) [2023-12-27 11:38:26,936][20347] Updated weights for policy 0, policy_version 866143 (0.0010) [2023-12-27 11:38:26,939][20381] Updated weights for policy 1, policy_version 865413 (0.0010) [2023-12-27 11:38:26,986][20347] Updated weights for policy 0, policy_version 866153 (0.0010) [2023-12-27 11:38:26,998][20381] Updated weights for policy 1, policy_version 865423 (0.0010) [2023-12-27 11:38:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 20160.7). Total num frames: 443342848. Throughput: 0: 9671.8, 1: 9745.0. Samples: 443331192. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:27,363][19170] Avg episode reward: [(0, '676.370'), (1, '853.870')] [2023-12-27 11:38:27,652][20381] Updated weights for policy 1, policy_version 865433 (0.0010) [2023-12-27 11:38:27,675][20347] Updated weights for policy 0, policy_version 866163 (0.0010) [2023-12-27 11:38:27,706][20381] Updated weights for policy 1, policy_version 865443 (0.0005) [2023-12-27 11:38:27,734][20347] Updated weights for policy 0, policy_version 866173 (0.0010) [2023-12-27 11:38:27,761][20381] Updated weights for policy 1, policy_version 865453 (0.0009) [2023-12-27 11:38:27,790][20347] Updated weights for policy 0, policy_version 866183 (0.0010) [2023-12-27 11:38:27,820][20381] Updated weights for policy 1, policy_version 865463 (0.0011) [2023-12-27 11:38:28,449][20347] Updated weights for policy 0, policy_version 866193 (0.0010) [2023-12-27 11:38:28,455][20381] Updated weights for policy 1, policy_version 865473 (0.0007) [2023-12-27 11:38:28,501][20347] Updated weights for policy 0, policy_version 866203 (0.0010) [2023-12-27 11:38:28,511][20381] Updated weights for policy 1, policy_version 865483 (0.0005) [2023-12-27 11:38:28,551][20347] Updated weights for policy 0, policy_version 866213 (0.0010) [2023-12-27 11:38:28,561][20381] Updated weights for policy 1, policy_version 865493 (0.0005) [2023-12-27 11:38:28,612][20347] Updated weights for policy 0, policy_version 866223 (0.0011) [2023-12-27 11:38:29,245][20381] Updated weights for policy 1, policy_version 865503 (0.0008) [2023-12-27 11:38:29,306][20381] Updated weights for policy 1, policy_version 865513 (0.0007) [2023-12-27 11:38:29,327][20347] Updated weights for policy 0, policy_version 866233 (0.0008) [2023-12-27 11:38:29,371][20381] Updated weights for policy 1, policy_version 865523 (0.0008) [2023-12-27 11:38:29,388][20347] Updated weights for policy 0, policy_version 866243 (0.0008) [2023-12-27 11:38:29,444][20347] Updated weights for policy 0, policy_version 866253 (0.0008) [2023-12-27 11:38:30,101][20381] Updated weights for policy 1, policy_version 865533 (0.0006) [2023-12-27 11:38:30,153][20381] Updated weights for policy 1, policy_version 865543 (0.0007) [2023-12-27 11:38:30,208][20381] Updated weights for policy 1, policy_version 865553 (0.0008) [2023-12-27 11:38:30,234][20347] Updated weights for policy 0, policy_version 866263 (0.0008) [2023-12-27 11:38:30,294][20347] Updated weights for policy 0, policy_version 866273 (0.0008) [2023-12-27 11:38:30,349][20347] Updated weights for policy 0, policy_version 866283 (0.0007) [2023-12-27 11:38:30,896][20381] Updated weights for policy 1, policy_version 865563 (0.0007) [2023-12-27 11:38:30,949][20381] Updated weights for policy 1, policy_version 865573 (0.0008) [2023-12-27 11:38:31,004][20381] Updated weights for policy 1, policy_version 865583 (0.0008) [2023-12-27 11:38:31,036][20347] Updated weights for policy 0, policy_version 866293 (0.0008) [2023-12-27 11:38:31,090][20347] Updated weights for policy 0, policy_version 866303 (0.0007) [2023-12-27 11:38:31,155][20347] Updated weights for policy 0, policy_version 866313 (0.0008) [2023-12-27 11:38:31,784][20381] Updated weights for policy 1, policy_version 865593 (0.0006) [2023-12-27 11:38:31,842][20381] Updated weights for policy 1, policy_version 865603 (0.0008) [2023-12-27 11:38:31,864][20347] Updated weights for policy 0, policy_version 866323 (0.0007) [2023-12-27 11:38:31,900][20381] Updated weights for policy 1, policy_version 865613 (0.0008) [2023-12-27 11:38:31,920][20347] Updated weights for policy 0, policy_version 866333 (0.0006) [2023-12-27 11:38:31,957][20381] Updated weights for policy 1, policy_version 865623 (0.0007) [2023-12-27 11:38:31,981][20347] Updated weights for policy 0, policy_version 866343 (0.0005) [2023-12-27 11:38:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 20160.7). Total num frames: 443441152. Throughput: 0: 9718.1, 1: 9791.9. Samples: 443450652. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:32,363][19170] Avg episode reward: [(0, '744.880'), (1, '931.970')] [2023-12-27 11:38:32,700][20347] Updated weights for policy 0, policy_version 866353 (0.0006) [2023-12-27 11:38:32,733][20381] Updated weights for policy 1, policy_version 865633 (0.0006) [2023-12-27 11:38:32,759][20347] Updated weights for policy 0, policy_version 866363 (0.0006) [2023-12-27 11:38:32,796][20381] Updated weights for policy 1, policy_version 865643 (0.0006) [2023-12-27 11:38:32,825][20347] Updated weights for policy 0, policy_version 866373 (0.0009) [2023-12-27 11:38:32,852][20381] Updated weights for policy 1, policy_version 865653 (0.0006) [2023-12-27 11:38:32,891][20347] Updated weights for policy 0, policy_version 866383 (0.0007) [2023-12-27 11:38:33,535][20381] Updated weights for policy 1, policy_version 865663 (0.0006) [2023-12-27 11:38:33,592][20381] Updated weights for policy 1, policy_version 865673 (0.0005) [2023-12-27 11:38:33,615][20347] Updated weights for policy 0, policy_version 866393 (0.0010) [2023-12-27 11:38:33,649][20381] Updated weights for policy 1, policy_version 865683 (0.0008) [2023-12-27 11:38:33,674][20347] Updated weights for policy 0, policy_version 866403 (0.0010) [2023-12-27 11:38:33,726][20347] Updated weights for policy 0, policy_version 866413 (0.0010) [2023-12-27 11:38:34,406][20381] Updated weights for policy 1, policy_version 865693 (0.0007) [2023-12-27 11:38:34,409][20347] Updated weights for policy 0, policy_version 866423 (0.0010) [2023-12-27 11:38:34,465][20381] Updated weights for policy 1, policy_version 865703 (0.0007) [2023-12-27 11:38:34,472][20347] Updated weights for policy 0, policy_version 866433 (0.0009) [2023-12-27 11:38:34,523][20381] Updated weights for policy 1, policy_version 865713 (0.0009) [2023-12-27 11:38:34,530][20347] Updated weights for policy 0, policy_version 866443 (0.0009) [2023-12-27 11:38:35,250][20347] Updated weights for policy 0, policy_version 866453 (0.0009) [2023-12-27 11:38:35,281][20381] Updated weights for policy 1, policy_version 865723 (0.0008) [2023-12-27 11:38:35,318][20347] Updated weights for policy 0, policy_version 866463 (0.0008) [2023-12-27 11:38:35,329][20292] KL-divergence is very high: 121.5559 [2023-12-27 11:38:35,339][20381] Updated weights for policy 1, policy_version 865733 (0.0006) [2023-12-27 11:38:35,372][20347] Updated weights for policy 0, policy_version 866473 (0.0009) [2023-12-27 11:38:35,394][20381] Updated weights for policy 1, policy_version 865743 (0.0006) [2023-12-27 11:38:36,132][20347] Updated weights for policy 0, policy_version 866483 (0.0008) [2023-12-27 11:38:36,151][20381] Updated weights for policy 1, policy_version 865753 (0.0007) [2023-12-27 11:38:36,203][20347] Updated weights for policy 0, policy_version 866493 (0.0009) [2023-12-27 11:38:36,227][20381] Updated weights for policy 1, policy_version 865763 (0.0009) [2023-12-27 11:38:36,261][20347] Updated weights for policy 0, policy_version 866503 (0.0009) [2023-12-27 11:38:36,286][20381] Updated weights for policy 1, policy_version 865773 (0.0007) [2023-12-27 11:38:36,348][20381] Updated weights for policy 1, policy_version 865783 (0.0008) [2023-12-27 11:38:36,990][20347] Updated weights for policy 0, policy_version 866513 (0.0008) [2023-12-27 11:38:37,045][20347] Updated weights for policy 0, policy_version 866523 (0.0010) [2023-12-27 11:38:37,065][20381] Updated weights for policy 1, policy_version 865793 (0.0006) [2023-12-27 11:38:37,104][20347] Updated weights for policy 0, policy_version 866533 (0.0010) [2023-12-27 11:38:37,124][20381] Updated weights for policy 1, policy_version 865803 (0.0005) [2023-12-27 11:38:37,159][20347] Updated weights for policy 0, policy_version 866543 (0.0010) [2023-12-27 11:38:37,177][20381] Updated weights for policy 1, policy_version 865813 (0.0006) [2023-12-27 11:38:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 20160.6). Total num frames: 443539456. Throughput: 0: 9732.5, 1: 9807.9. Samples: 443508040. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:37,363][19170] Avg episode reward: [(0, '755.070'), (1, '853.010')] [2023-12-27 11:38:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000865816_221675520.pth... [2023-12-27 11:38:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000866544_221863936.pth... [2023-12-27 11:38:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000865424_221577216.pth [2023-12-27 11:38:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000864664_221380608.pth [2023-12-27 11:38:37,860][20381] Updated weights for policy 1, policy_version 865823 (0.0006) [2023-12-27 11:38:37,882][20347] Updated weights for policy 0, policy_version 866553 (0.0011) [2023-12-27 11:38:37,920][20381] Updated weights for policy 1, policy_version 865833 (0.0007) [2023-12-27 11:38:37,933][20347] Updated weights for policy 0, policy_version 866563 (0.0010) [2023-12-27 11:38:37,980][20381] Updated weights for policy 1, policy_version 865843 (0.0009) [2023-12-27 11:38:37,985][20347] Updated weights for policy 0, policy_version 866573 (0.0009) [2023-12-27 11:38:38,715][20381] Updated weights for policy 1, policy_version 865853 (0.0007) [2023-12-27 11:38:38,737][20347] Updated weights for policy 0, policy_version 866583 (0.0008) [2023-12-27 11:38:38,783][20381] Updated weights for policy 1, policy_version 865863 (0.0009) [2023-12-27 11:38:38,801][20347] Updated weights for policy 0, policy_version 866593 (0.0011) [2023-12-27 11:38:38,846][20381] Updated weights for policy 1, policy_version 865873 (0.0007) [2023-12-27 11:38:38,864][20347] Updated weights for policy 0, policy_version 866603 (0.0010) [2023-12-27 11:38:39,548][20381] Updated weights for policy 1, policy_version 865883 (0.0007) [2023-12-27 11:38:39,573][20347] Updated weights for policy 0, policy_version 866613 (0.0006) [2023-12-27 11:38:39,617][20381] Updated weights for policy 1, policy_version 865893 (0.0008) [2023-12-27 11:38:39,640][20347] Updated weights for policy 0, policy_version 866623 (0.0007) [2023-12-27 11:38:39,681][20381] Updated weights for policy 1, policy_version 865903 (0.0008) [2023-12-27 11:38:39,700][20347] Updated weights for policy 0, policy_version 866633 (0.0007) [2023-12-27 11:38:40,498][20381] Updated weights for policy 1, policy_version 865913 (0.0008) [2023-12-27 11:38:40,501][20347] Updated weights for policy 0, policy_version 866643 (0.0007) [2023-12-27 11:38:40,557][20347] Updated weights for policy 0, policy_version 866653 (0.0008) [2023-12-27 11:38:40,558][20381] Updated weights for policy 1, policy_version 865923 (0.0008) [2023-12-27 11:38:40,617][20347] Updated weights for policy 0, policy_version 866663 (0.0007) [2023-12-27 11:38:40,618][20381] Updated weights for policy 1, policy_version 865933 (0.0007) [2023-12-27 11:38:40,679][20381] Updated weights for policy 1, policy_version 865943 (0.0007) [2023-12-27 11:38:41,306][20347] Updated weights for policy 0, policy_version 866673 (0.0006) [2023-12-27 11:38:41,374][20347] Updated weights for policy 0, policy_version 866683 (0.0007) [2023-12-27 11:38:41,451][20347] Updated weights for policy 0, policy_version 866693 (0.0009) [2023-12-27 11:38:41,451][20381] Updated weights for policy 1, policy_version 865953 (0.0008) [2023-12-27 11:38:41,517][20347] Updated weights for policy 0, policy_version 866703 (0.0007) [2023-12-27 11:38:41,519][20381] Updated weights for policy 1, policy_version 865963 (0.0007) [2023-12-27 11:38:41,580][20381] Updated weights for policy 1, policy_version 865973 (0.0007) [2023-12-27 11:38:42,216][20347] Updated weights for policy 0, policy_version 866713 (0.0008) [2023-12-27 11:38:42,281][20347] Updated weights for policy 0, policy_version 866723 (0.0008) [2023-12-27 11:38:42,338][20381] Updated weights for policy 1, policy_version 865983 (0.0008) [2023-12-27 11:38:42,344][20347] Updated weights for policy 0, policy_version 866733 (0.0007) [2023-12-27 11:38:42,362][19170] Fps is (10 sec: 18022.2, 60 sec: 19251.2, 300 sec: 20049.6). Total num frames: 443621376. Throughput: 0: 9677.1, 1: 9753.8. Samples: 443621384. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:42,363][19170] Avg episode reward: [(0, '647.260'), (1, '824.560')] [2023-12-27 11:38:42,401][20381] Updated weights for policy 1, policy_version 865993 (0.0008) [2023-12-27 11:38:42,451][20381] Updated weights for policy 1, policy_version 866003 (0.0008) [2023-12-27 11:38:43,068][20347] Updated weights for policy 0, policy_version 866743 (0.0008) [2023-12-27 11:38:43,127][20347] Updated weights for policy 0, policy_version 866753 (0.0006) [2023-12-27 11:38:43,180][20381] Updated weights for policy 1, policy_version 866013 (0.0007) [2023-12-27 11:38:43,186][20347] Updated weights for policy 0, policy_version 866763 (0.0006) [2023-12-27 11:38:43,247][20381] Updated weights for policy 1, policy_version 866023 (0.0007) [2023-12-27 11:38:43,310][20381] Updated weights for policy 1, policy_version 866033 (0.0007) [2023-12-27 11:38:43,870][20347] Updated weights for policy 0, policy_version 866773 (0.0006) [2023-12-27 11:38:43,936][20347] Updated weights for policy 0, policy_version 866783 (0.0008) [2023-12-27 11:38:43,996][20347] Updated weights for policy 0, policy_version 866793 (0.0007) [2023-12-27 11:38:44,004][20381] Updated weights for policy 1, policy_version 866043 (0.0008) [2023-12-27 11:38:44,071][20381] Updated weights for policy 1, policy_version 866053 (0.0006) [2023-12-27 11:38:44,131][20381] Updated weights for policy 1, policy_version 866063 (0.0007) [2023-12-27 11:38:44,607][20347] Updated weights for policy 0, policy_version 866803 (0.0007) [2023-12-27 11:38:44,663][20347] Updated weights for policy 0, policy_version 866813 (0.0008) [2023-12-27 11:38:44,719][20347] Updated weights for policy 0, policy_version 866823 (0.0008) [2023-12-27 11:38:44,808][20381] Updated weights for policy 1, policy_version 866073 (0.0008) [2023-12-27 11:38:44,872][20381] Updated weights for policy 1, policy_version 866083 (0.0007) [2023-12-27 11:38:44,934][20381] Updated weights for policy 1, policy_version 866093 (0.0009) [2023-12-27 11:38:45,000][20381] Updated weights for policy 1, policy_version 866103 (0.0007) [2023-12-27 11:38:45,444][20347] Updated weights for policy 0, policy_version 866833 (0.0007) [2023-12-27 11:38:45,502][20347] Updated weights for policy 0, policy_version 866843 (0.0008) [2023-12-27 11:38:45,563][20347] Updated weights for policy 0, policy_version 866853 (0.0008) [2023-12-27 11:38:45,622][20347] Updated weights for policy 0, policy_version 866863 (0.0009) [2023-12-27 11:38:45,717][20292] KL-divergence is very high: 183.0332 [2023-12-27 11:38:45,723][20381] Updated weights for policy 1, policy_version 866113 (0.0008) [2023-12-27 11:38:45,785][20381] Updated weights for policy 1, policy_version 866123 (0.0006) [2023-12-27 11:38:45,846][20381] Updated weights for policy 1, policy_version 866133 (0.0006) [2023-12-27 11:38:46,248][20347] Updated weights for policy 0, policy_version 866873 (0.0007) [2023-12-27 11:38:46,309][20347] Updated weights for policy 0, policy_version 866883 (0.0007) [2023-12-27 11:38:46,367][20347] Updated weights for policy 0, policy_version 866893 (0.0008) [2023-12-27 11:38:46,495][20381] Updated weights for policy 1, policy_version 866143 (0.0007) [2023-12-27 11:38:46,556][20381] Updated weights for policy 1, policy_version 866153 (0.0007) [2023-12-27 11:38:46,615][20381] Updated weights for policy 1, policy_version 866163 (0.0006) [2023-12-27 11:38:47,083][20347] Updated weights for policy 0, policy_version 866903 (0.0008) [2023-12-27 11:38:47,145][20347] Updated weights for policy 0, policy_version 866913 (0.0007) [2023-12-27 11:38:47,211][20347] Updated weights for policy 0, policy_version 866923 (0.0006) [2023-12-27 11:38:47,265][20381] Updated weights for policy 1, policy_version 866173 (0.0006) [2023-12-27 11:38:47,319][20381] Updated weights for policy 1, policy_version 866183 (0.0006) [2023-12-27 11:38:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 20077.4). Total num frames: 443727872. Throughput: 0: 9703.9, 1: 9767.1. Samples: 443739936. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:47,363][19170] Avg episode reward: [(0, '627.530'), (1, '795.430')] [2023-12-27 11:38:47,370][20381] Updated weights for policy 1, policy_version 866193 (0.0006) [2023-12-27 11:38:47,905][20347] Updated weights for policy 0, policy_version 866933 (0.0006) [2023-12-27 11:38:47,961][20347] Updated weights for policy 0, policy_version 866943 (0.0007) [2023-12-27 11:38:48,004][20381] Updated weights for policy 1, policy_version 866203 (0.0006) [2023-12-27 11:38:48,021][20347] Updated weights for policy 0, policy_version 866953 (0.0007) [2023-12-27 11:38:48,050][20381] Updated weights for policy 1, policy_version 866213 (0.0005) [2023-12-27 11:38:48,095][20381] Updated weights for policy 1, policy_version 866223 (0.0008) [2023-12-27 11:38:48,724][20347] Updated weights for policy 0, policy_version 866963 (0.0006) [2023-12-27 11:38:48,776][20347] Updated weights for policy 0, policy_version 866973 (0.0006) [2023-12-27 11:38:48,834][20347] Updated weights for policy 0, policy_version 866983 (0.0007) [2023-12-27 11:38:48,872][20381] Updated weights for policy 1, policy_version 866233 (0.0009) [2023-12-27 11:38:48,933][20381] Updated weights for policy 1, policy_version 866243 (0.0008) [2023-12-27 11:38:48,986][20381] Updated weights for policy 1, policy_version 866253 (0.0008) [2023-12-27 11:38:49,040][20381] Updated weights for policy 1, policy_version 866263 (0.0008) [2023-12-27 11:38:49,586][20347] Updated weights for policy 0, policy_version 866993 (0.0008) [2023-12-27 11:38:49,647][20347] Updated weights for policy 0, policy_version 867003 (0.0007) [2023-12-27 11:38:49,706][20347] Updated weights for policy 0, policy_version 867013 (0.0007) [2023-12-27 11:38:49,773][20347] Updated weights for policy 0, policy_version 867023 (0.0008) [2023-12-27 11:38:49,845][20381] Updated weights for policy 1, policy_version 866273 (0.0010) [2023-12-27 11:38:49,907][20381] Updated weights for policy 1, policy_version 866283 (0.0008) [2023-12-27 11:38:49,973][20381] Updated weights for policy 1, policy_version 866293 (0.0009) [2023-12-27 11:38:50,556][20347] Updated weights for policy 0, policy_version 867033 (0.0007) [2023-12-27 11:38:50,615][20347] Updated weights for policy 0, policy_version 867043 (0.0007) [2023-12-27 11:38:50,671][20347] Updated weights for policy 0, policy_version 867053 (0.0007) [2023-12-27 11:38:50,724][20381] Updated weights for policy 1, policy_version 866303 (0.0009) [2023-12-27 11:38:50,786][20381] Updated weights for policy 1, policy_version 866313 (0.0011) [2023-12-27 11:38:50,856][20381] Updated weights for policy 1, policy_version 866323 (0.0010) [2023-12-27 11:38:51,477][20347] Updated weights for policy 0, policy_version 867063 (0.0009) [2023-12-27 11:38:51,543][20347] Updated weights for policy 0, policy_version 867073 (0.0009) [2023-12-27 11:38:51,598][20381] Updated weights for policy 1, policy_version 866333 (0.0007) [2023-12-27 11:38:51,609][20347] Updated weights for policy 0, policy_version 867083 (0.0007) [2023-12-27 11:38:51,669][20381] Updated weights for policy 1, policy_version 866343 (0.0008) [2023-12-27 11:38:51,747][20381] Updated weights for policy 1, policy_version 866353 (0.0008) [2023-12-27 11:38:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 20021.8). Total num frames: 443817984. Throughput: 0: 9668.4, 1: 9741.5. Samples: 443796948. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:52,363][19170] Avg episode reward: [(0, '754.800'), (1, '873.460')] [2023-12-27 11:38:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000866360_221814784.pth... [2023-12-27 11:38:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000865240_221528064.pth [2023-12-27 11:38:52,380][20347] Updated weights for policy 0, policy_version 867093 (0.0007) [2023-12-27 11:38:52,419][20381] Updated weights for policy 1, policy_version 866363 (0.0007) [2023-12-27 11:38:52,441][20347] Updated weights for policy 0, policy_version 867103 (0.0007) [2023-12-27 11:38:52,475][20292] KL-divergence is very high: 194.4602 [2023-12-27 11:38:52,484][20381] Updated weights for policy 1, policy_version 866373 (0.0007) [2023-12-27 11:38:52,503][20347] Updated weights for policy 0, policy_version 867113 (0.0007) [2023-12-27 11:38:52,543][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000867120_222011392.pth... [2023-12-27 11:38:52,547][20381] Updated weights for policy 1, policy_version 866383 (0.0010) [2023-12-27 11:38:52,547][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000865968_221716480.pth [2023-12-27 11:38:53,205][20381] Updated weights for policy 1, policy_version 866393 (0.0008) [2023-12-27 11:38:53,264][20381] Updated weights for policy 1, policy_version 866403 (0.0007) [2023-12-27 11:38:53,266][20347] Updated weights for policy 0, policy_version 867123 (0.0008) [2023-12-27 11:38:53,319][20381] Updated weights for policy 1, policy_version 866413 (0.0005) [2023-12-27 11:38:53,325][20347] Updated weights for policy 0, policy_version 867133 (0.0007) [2023-12-27 11:38:53,380][20381] Updated weights for policy 1, policy_version 866423 (0.0006) [2023-12-27 11:38:53,382][20347] Updated weights for policy 0, policy_version 867143 (0.0007) [2023-12-27 11:38:54,068][20381] Updated weights for policy 1, policy_version 866433 (0.0008) [2023-12-27 11:38:54,117][20347] Updated weights for policy 0, policy_version 867153 (0.0007) [2023-12-27 11:38:54,125][20381] Updated weights for policy 1, policy_version 866443 (0.0008) [2023-12-27 11:38:54,173][20347] Updated weights for policy 0, policy_version 867163 (0.0005) [2023-12-27 11:38:54,189][20381] Updated weights for policy 1, policy_version 866453 (0.0007) [2023-12-27 11:38:54,234][20347] Updated weights for policy 0, policy_version 867173 (0.0006) [2023-12-27 11:38:54,295][20347] Updated weights for policy 0, policy_version 867183 (0.0008) [2023-12-27 11:38:54,897][20381] Updated weights for policy 1, policy_version 866463 (0.0007) [2023-12-27 11:38:54,958][20381] Updated weights for policy 1, policy_version 866473 (0.0006) [2023-12-27 11:38:55,018][20347] Updated weights for policy 0, policy_version 867193 (0.0008) [2023-12-27 11:38:55,022][20381] Updated weights for policy 1, policy_version 866483 (0.0007) [2023-12-27 11:38:55,075][20347] Updated weights for policy 0, policy_version 867203 (0.0005) [2023-12-27 11:38:55,126][20347] Updated weights for policy 0, policy_version 867213 (0.0006) [2023-12-27 11:38:55,716][20381] Updated weights for policy 1, policy_version 866493 (0.0008) [2023-12-27 11:38:55,770][20381] Updated weights for policy 1, policy_version 866503 (0.0008) [2023-12-27 11:38:55,796][20347] Updated weights for policy 0, policy_version 867223 (0.0007) [2023-12-27 11:38:55,822][20381] Updated weights for policy 1, policy_version 866513 (0.0007) [2023-12-27 11:38:55,860][20347] Updated weights for policy 0, policy_version 867233 (0.0007) [2023-12-27 11:38:55,921][20347] Updated weights for policy 0, policy_version 867243 (0.0008) [2023-12-27 11:38:56,457][20381] Updated weights for policy 1, policy_version 866523 (0.0007) [2023-12-27 11:38:56,516][20381] Updated weights for policy 1, policy_version 866533 (0.0006) [2023-12-27 11:38:56,577][20381] Updated weights for policy 1, policy_version 866543 (0.0007) [2023-12-27 11:38:56,653][20347] Updated weights for policy 0, policy_version 867253 (0.0008) [2023-12-27 11:38:56,717][20347] Updated weights for policy 0, policy_version 867263 (0.0007) [2023-12-27 11:38:56,773][20347] Updated weights for policy 0, policy_version 867273 (0.0006) [2023-12-27 11:38:57,279][20381] Updated weights for policy 1, policy_version 866553 (0.0008) [2023-12-27 11:38:57,341][20381] Updated weights for policy 1, policy_version 866563 (0.0007) [2023-12-27 11:38:57,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19387.7, 300 sec: 20021.8). Total num frames: 443916288. Throughput: 0: 9631.7, 1: 9764.8. Samples: 443911476. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:38:57,363][19170] Avg episode reward: [(0, '656.550'), (1, '931.910')] [2023-12-27 11:38:57,405][20381] Updated weights for policy 1, policy_version 866573 (0.0006) [2023-12-27 11:38:57,427][20347] Updated weights for policy 0, policy_version 867283 (0.0007) [2023-12-27 11:38:57,461][20381] Updated weights for policy 1, policy_version 866583 (0.0008) [2023-12-27 11:38:57,486][20347] Updated weights for policy 0, policy_version 867293 (0.0005) [2023-12-27 11:38:57,503][20291] KL-divergence is very high: 160.6014 [2023-12-27 11:38:57,542][20347] Updated weights for policy 0, policy_version 867303 (0.0008) [2023-12-27 11:38:58,095][20381] Updated weights for policy 1, policy_version 866593 (0.0008) [2023-12-27 11:38:58,152][20381] Updated weights for policy 1, policy_version 866603 (0.0008) [2023-12-27 11:38:58,212][20381] Updated weights for policy 1, policy_version 866613 (0.0008) [2023-12-27 11:38:58,238][20347] Updated weights for policy 0, policy_version 867313 (0.0006) [2023-12-27 11:38:58,301][20347] Updated weights for policy 0, policy_version 867323 (0.0008) [2023-12-27 11:38:58,364][20347] Updated weights for policy 0, policy_version 867333 (0.0009) [2023-12-27 11:38:58,433][20347] Updated weights for policy 0, policy_version 867343 (0.0008) [2023-12-27 11:38:59,115][20381] Updated weights for policy 1, policy_version 866623 (0.0008) [2023-12-27 11:38:59,175][20381] Updated weights for policy 1, policy_version 866633 (0.0008) [2023-12-27 11:38:59,241][20381] Updated weights for policy 1, policy_version 866643 (0.0007) [2023-12-27 11:38:59,244][20347] Updated weights for policy 0, policy_version 867355 (0.0008) [2023-12-27 11:38:59,311][20347] Updated weights for policy 0, policy_version 867365 (0.0008) [2023-12-27 11:38:59,383][20347] Updated weights for policy 0, policy_version 867375 (0.0008) [2023-12-27 11:39:00,042][20381] Updated weights for policy 1, policy_version 866653 (0.0008) [2023-12-27 11:39:00,089][20347] Updated weights for policy 0, policy_version 867385 (0.0008) [2023-12-27 11:39:00,110][20381] Updated weights for policy 1, policy_version 866663 (0.0008) [2023-12-27 11:39:00,157][20347] Updated weights for policy 0, policy_version 867395 (0.0010) [2023-12-27 11:39:00,173][20381] Updated weights for policy 1, policy_version 866673 (0.0008) [2023-12-27 11:39:00,216][20347] Updated weights for policy 0, policy_version 867405 (0.0007) [2023-12-27 11:39:00,903][20381] Updated weights for policy 1, policy_version 866683 (0.0008) [2023-12-27 11:39:00,921][20347] Updated weights for policy 0, policy_version 867415 (0.0008) [2023-12-27 11:39:00,963][20381] Updated weights for policy 1, policy_version 866693 (0.0006) [2023-12-27 11:39:00,980][20347] Updated weights for policy 0, policy_version 867425 (0.0009) [2023-12-27 11:39:01,024][20381] Updated weights for policy 1, policy_version 866703 (0.0006) [2023-12-27 11:39:01,040][20347] Updated weights for policy 0, policy_version 867435 (0.0010) [2023-12-27 11:39:01,761][20381] Updated weights for policy 1, policy_version 866713 (0.0007) [2023-12-27 11:39:01,768][20347] Updated weights for policy 0, policy_version 867445 (0.0010) [2023-12-27 11:39:01,824][20381] Updated weights for policy 1, policy_version 866723 (0.0007) [2023-12-27 11:39:01,826][20347] Updated weights for policy 0, policy_version 867455 (0.0006) [2023-12-27 11:39:01,880][20381] Updated weights for policy 1, policy_version 866733 (0.0007) [2023-12-27 11:39:01,883][20347] Updated weights for policy 0, policy_version 867465 (0.0006) [2023-12-27 11:39:01,939][20381] Updated weights for policy 1, policy_version 866743 (0.0006) [2023-12-27 11:39:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 20021.8). Total num frames: 444014592. Throughput: 0: 9637.8, 1: 9665.2. Samples: 444024496. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:39:02,363][19170] Avg episode reward: [(0, '656.280'), (1, '892.880')] [2023-12-27 11:39:02,574][20347] Updated weights for policy 0, policy_version 867475 (0.0007) [2023-12-27 11:39:02,638][20347] Updated weights for policy 0, policy_version 867485 (0.0008) [2023-12-27 11:39:02,695][20347] Updated weights for policy 0, policy_version 867495 (0.0009) [2023-12-27 11:39:02,718][20381] Updated weights for policy 1, policy_version 866753 (0.0007) [2023-12-27 11:39:02,777][20381] Updated weights for policy 1, policy_version 866763 (0.0008) [2023-12-27 11:39:02,838][20381] Updated weights for policy 1, policy_version 866773 (0.0007) [2023-12-27 11:39:03,400][20347] Updated weights for policy 0, policy_version 867505 (0.0006) [2023-12-27 11:39:03,452][20291] KL-divergence is very high: 117.6603 [2023-12-27 11:39:03,468][20347] Updated weights for policy 0, policy_version 867515 (0.0007) [2023-12-27 11:39:03,518][20347] Updated weights for policy 0, policy_version 867525 (0.0010) [2023-12-27 11:39:03,542][20381] Updated weights for policy 1, policy_version 866783 (0.0006) [2023-12-27 11:39:03,558][20291] KL-divergence is very high: 196.3196 [2023-12-27 11:39:03,576][20347] Updated weights for policy 0, policy_version 867535 (0.0010) [2023-12-27 11:39:03,610][20381] Updated weights for policy 1, policy_version 866793 (0.0007) [2023-12-27 11:39:03,670][20381] Updated weights for policy 1, policy_version 866803 (0.0007) [2023-12-27 11:39:04,357][20347] Updated weights for policy 0, policy_version 867545 (0.0009) [2023-12-27 11:39:04,426][20347] Updated weights for policy 0, policy_version 867555 (0.0008) [2023-12-27 11:39:04,458][20381] Updated weights for policy 1, policy_version 866813 (0.0008) [2023-12-27 11:39:04,497][20347] Updated weights for policy 0, policy_version 867565 (0.0008) [2023-12-27 11:39:04,521][20381] Updated weights for policy 1, policy_version 866823 (0.0008) [2023-12-27 11:39:04,580][20381] Updated weights for policy 1, policy_version 866833 (0.0008) [2023-12-27 11:39:05,203][20347] Updated weights for policy 0, policy_version 867575 (0.0007) [2023-12-27 11:39:05,254][20347] Updated weights for policy 0, policy_version 867585 (0.0006) [2023-12-27 11:39:05,285][20381] Updated weights for policy 1, policy_version 866843 (0.0007) [2023-12-27 11:39:05,316][20347] Updated weights for policy 0, policy_version 867595 (0.0007) [2023-12-27 11:39:05,350][20381] Updated weights for policy 1, policy_version 866853 (0.0006) [2023-12-27 11:39:05,412][20381] Updated weights for policy 1, policy_version 866863 (0.0007) [2023-12-27 11:39:05,954][20347] Updated weights for policy 0, policy_version 867605 (0.0008) [2023-12-27 11:39:06,012][20347] Updated weights for policy 0, policy_version 867615 (0.0007) [2023-12-27 11:39:06,078][20347] Updated weights for policy 0, policy_version 867625 (0.0008) [2023-12-27 11:39:06,138][20381] Updated weights for policy 1, policy_version 866873 (0.0009) [2023-12-27 11:39:06,204][20381] Updated weights for policy 1, policy_version 866883 (0.0008) [2023-12-27 11:39:06,229][20292] KL-divergence is very high: 477.2484 [2023-12-27 11:39:06,266][20381] Updated weights for policy 1, policy_version 866893 (0.0008) [2023-12-27 11:39:06,322][20381] Updated weights for policy 1, policy_version 866903 (0.0008) [2023-12-27 11:39:06,816][20347] Updated weights for policy 0, policy_version 867635 (0.0008) [2023-12-27 11:39:06,877][20347] Updated weights for policy 0, policy_version 867645 (0.0009) [2023-12-27 11:39:06,938][20347] Updated weights for policy 0, policy_version 867655 (0.0008) [2023-12-27 11:39:07,115][20381] Updated weights for policy 1, policy_version 866913 (0.0006) [2023-12-27 11:39:07,180][20381] Updated weights for policy 1, policy_version 866923 (0.0006) [2023-12-27 11:39:07,240][20381] Updated weights for policy 1, policy_version 866933 (0.0008) [2023-12-27 11:39:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.8, 300 sec: 20021.8). Total num frames: 444112896. Throughput: 0: 9638.9, 1: 9629.3. Samples: 444081280. Policy #0 lag: (min: 4.0, avg: 13.7, max: 36.0) [2023-12-27 11:39:07,363][19170] Avg episode reward: [(0, '715.410'), (1, '883.080')] [2023-12-27 11:39:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000866936_221962240.pth... [2023-12-27 11:39:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000867664_222150656.pth... [2023-12-27 11:39:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000865816_221675520.pth [2023-12-27 11:39:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000866544_221863936.pth [2023-12-27 11:39:07,630][20347] Updated weights for policy 0, policy_version 867665 (0.0008) [2023-12-27 11:39:07,688][20347] Updated weights for policy 0, policy_version 867675 (0.0007) [2023-12-27 11:39:07,738][20291] KL-divergence is very high: 103.4148 [2023-12-27 11:39:07,740][20347] Updated weights for policy 0, policy_version 867685 (0.0008) [2023-12-27 11:39:07,792][20347] Updated weights for policy 0, policy_version 867695 (0.0008) [2023-12-27 11:39:07,935][20381] Updated weights for policy 1, policy_version 866943 (0.0007) [2023-12-27 11:39:07,989][20381] Updated weights for policy 1, policy_version 866953 (0.0007) [2023-12-27 11:39:08,047][20381] Updated weights for policy 1, policy_version 866963 (0.0008) [2023-12-27 11:39:08,548][20347] Updated weights for policy 0, policy_version 867705 (0.0008) [2023-12-27 11:39:08,608][20347] Updated weights for policy 0, policy_version 867715 (0.0007) [2023-12-27 11:39:08,673][20347] Updated weights for policy 0, policy_version 867725 (0.0008) [2023-12-27 11:39:08,765][20381] Updated weights for policy 1, policy_version 866973 (0.0007) [2023-12-27 11:39:08,823][20381] Updated weights for policy 1, policy_version 866983 (0.0009) [2023-12-27 11:39:08,881][20381] Updated weights for policy 1, policy_version 866993 (0.0011) [2023-12-27 11:39:09,449][20347] Updated weights for policy 0, policy_version 867735 (0.0008) [2023-12-27 11:39:09,511][20347] Updated weights for policy 0, policy_version 867745 (0.0008) [2023-12-27 11:39:09,575][20347] Updated weights for policy 0, policy_version 867755 (0.0008) [2023-12-27 11:39:09,688][20381] Updated weights for policy 1, policy_version 867003 (0.0010) [2023-12-27 11:39:09,746][20381] Updated weights for policy 1, policy_version 867013 (0.0008) [2023-12-27 11:39:09,808][20381] Updated weights for policy 1, policy_version 867023 (0.0009) [2023-12-27 11:39:10,282][20347] Updated weights for policy 0, policy_version 867765 (0.0007) [2023-12-27 11:39:10,342][20347] Updated weights for policy 0, policy_version 867775 (0.0007) [2023-12-27 11:39:10,400][20347] Updated weights for policy 0, policy_version 867785 (0.0008) [2023-12-27 11:39:10,660][20381] Updated weights for policy 1, policy_version 867033 (0.0009) [2023-12-27 11:39:10,721][20381] Updated weights for policy 1, policy_version 867043 (0.0009) [2023-12-27 11:39:10,774][20381] Updated weights for policy 1, policy_version 867053 (0.0008) [2023-12-27 11:39:10,832][20381] Updated weights for policy 1, policy_version 867063 (0.0008) [2023-12-27 11:39:11,131][20347] Updated weights for policy 0, policy_version 867795 (0.0008) [2023-12-27 11:39:11,196][20347] Updated weights for policy 0, policy_version 867805 (0.0007) [2023-12-27 11:39:11,250][20347] Updated weights for policy 0, policy_version 867815 (0.0008) [2023-12-27 11:39:11,657][20381] Updated weights for policy 1, policy_version 867073 (0.0009) [2023-12-27 11:39:11,724][20381] Updated weights for policy 1, policy_version 867083 (0.0009) [2023-12-27 11:39:11,795][20381] Updated weights for policy 1, policy_version 867093 (0.0007) [2023-12-27 11:39:12,029][20347] Updated weights for policy 0, policy_version 867825 (0.0008) [2023-12-27 11:39:12,099][20347] Updated weights for policy 0, policy_version 867835 (0.0007) [2023-12-27 11:39:12,160][20347] Updated weights for policy 0, policy_version 867845 (0.0007) [2023-12-27 11:39:12,224][20347] Updated weights for policy 0, policy_version 867855 (0.0008) [2023-12-27 11:39:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19251.2, 300 sec: 19938.5). Total num frames: 444203008. Throughput: 0: 9630.9, 1: 9558.5. Samples: 444194716. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:12,363][19170] Avg episode reward: [(0, '734.880'), (1, '922.260')] [2023-12-27 11:39:12,455][20381] Updated weights for policy 1, policy_version 867103 (0.0006) [2023-12-27 11:39:12,518][20381] Updated weights for policy 1, policy_version 867113 (0.0008) [2023-12-27 11:39:12,579][20381] Updated weights for policy 1, policy_version 867123 (0.0008) [2023-12-27 11:39:12,953][20347] Updated weights for policy 0, policy_version 867865 (0.0007) [2023-12-27 11:39:13,017][20347] Updated weights for policy 0, policy_version 867875 (0.0007) [2023-12-27 11:39:13,076][20347] Updated weights for policy 0, policy_version 867885 (0.0007) [2023-12-27 11:39:13,257][20381] Updated weights for policy 1, policy_version 867133 (0.0008) [2023-12-27 11:39:13,311][20381] Updated weights for policy 1, policy_version 867143 (0.0008) [2023-12-27 11:39:13,364][20381] Updated weights for policy 1, policy_version 867153 (0.0007) [2023-12-27 11:39:13,827][20347] Updated weights for policy 0, policy_version 867895 (0.0007) [2023-12-27 11:39:13,880][20347] Updated weights for policy 0, policy_version 867905 (0.0006) [2023-12-27 11:39:13,947][20347] Updated weights for policy 0, policy_version 867915 (0.0007) [2023-12-27 11:39:14,066][20381] Updated weights for policy 1, policy_version 867163 (0.0006) [2023-12-27 11:39:14,117][20381] Updated weights for policy 1, policy_version 867173 (0.0008) [2023-12-27 11:39:14,173][20381] Updated weights for policy 1, policy_version 867183 (0.0009) [2023-12-27 11:39:14,678][20347] Updated weights for policy 0, policy_version 867925 (0.0008) [2023-12-27 11:39:14,737][20347] Updated weights for policy 0, policy_version 867935 (0.0007) [2023-12-27 11:39:14,801][20347] Updated weights for policy 0, policy_version 867945 (0.0008) [2023-12-27 11:39:14,907][20381] Updated weights for policy 1, policy_version 867193 (0.0007) [2023-12-27 11:39:14,971][20381] Updated weights for policy 1, policy_version 867203 (0.0007) [2023-12-27 11:39:15,038][20381] Updated weights for policy 1, policy_version 867213 (0.0008) [2023-12-27 11:39:15,099][20381] Updated weights for policy 1, policy_version 867223 (0.0008) [2023-12-27 11:39:15,556][20347] Updated weights for policy 0, policy_version 867955 (0.0008) [2023-12-27 11:39:15,622][20347] Updated weights for policy 0, policy_version 867965 (0.0008) [2023-12-27 11:39:15,687][20347] Updated weights for policy 0, policy_version 867975 (0.0008) [2023-12-27 11:39:15,836][20381] Updated weights for policy 1, policy_version 867233 (0.0007) [2023-12-27 11:39:15,893][20381] Updated weights for policy 1, policy_version 867243 (0.0008) [2023-12-27 11:39:15,956][20381] Updated weights for policy 1, policy_version 867253 (0.0008) [2023-12-27 11:39:16,408][20347] Updated weights for policy 0, policy_version 867985 (0.0007) [2023-12-27 11:39:16,469][20347] Updated weights for policy 0, policy_version 867995 (0.0010) [2023-12-27 11:39:16,530][20347] Updated weights for policy 0, policy_version 868005 (0.0010) [2023-12-27 11:39:16,589][20347] Updated weights for policy 0, policy_version 868015 (0.0010) [2023-12-27 11:39:16,667][20381] Updated weights for policy 1, policy_version 867263 (0.0008) [2023-12-27 11:39:16,721][20381] Updated weights for policy 1, policy_version 867273 (0.0007) [2023-12-27 11:39:16,776][20381] Updated weights for policy 1, policy_version 867283 (0.0006) [2023-12-27 11:39:17,257][20347] Updated weights for policy 0, policy_version 868025 (0.0007) [2023-12-27 11:39:17,314][20347] Updated weights for policy 0, policy_version 868035 (0.0008) [2023-12-27 11:39:17,362][19170] Fps is (10 sec: 18022.6, 60 sec: 19114.7, 300 sec: 19910.7). Total num frames: 444293120. Throughput: 0: 9547.9, 1: 9519.4. Samples: 444308680. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:17,363][19170] Avg episode reward: [(0, '646.130'), (1, '902.560')] [2023-12-27 11:39:17,372][20347] Updated weights for policy 0, policy_version 868045 (0.0005) [2023-12-27 11:39:17,489][20381] Updated weights for policy 1, policy_version 867293 (0.0006) [2023-12-27 11:39:17,549][20381] Updated weights for policy 1, policy_version 867303 (0.0008) [2023-12-27 11:39:17,613][20381] Updated weights for policy 1, policy_version 867313 (0.0007) [2023-12-27 11:39:18,051][20347] Updated weights for policy 0, policy_version 868055 (0.0006) [2023-12-27 11:39:18,107][20347] Updated weights for policy 0, policy_version 868065 (0.0008) [2023-12-27 11:39:18,168][20347] Updated weights for policy 0, policy_version 868075 (0.0009) [2023-12-27 11:39:18,312][20381] Updated weights for policy 1, policy_version 867323 (0.0007) [2023-12-27 11:39:18,376][20381] Updated weights for policy 1, policy_version 867333 (0.0008) [2023-12-27 11:39:18,440][20381] Updated weights for policy 1, policy_version 867343 (0.0008) [2023-12-27 11:39:18,864][20347] Updated weights for policy 0, policy_version 868085 (0.0007) [2023-12-27 11:39:18,918][20347] Updated weights for policy 0, policy_version 868095 (0.0007) [2023-12-27 11:39:18,976][20347] Updated weights for policy 0, policy_version 868105 (0.0007) [2023-12-27 11:39:19,167][20381] Updated weights for policy 1, policy_version 867353 (0.0007) [2023-12-27 11:39:19,231][20381] Updated weights for policy 1, policy_version 867363 (0.0007) [2023-12-27 11:39:19,293][20381] Updated weights for policy 1, policy_version 867373 (0.0008) [2023-12-27 11:39:19,373][20381] Updated weights for policy 1, policy_version 867383 (0.0007) [2023-12-27 11:39:19,736][20347] Updated weights for policy 0, policy_version 868115 (0.0008) [2023-12-27 11:39:19,797][20347] Updated weights for policy 0, policy_version 868125 (0.0008) [2023-12-27 11:39:19,866][20347] Updated weights for policy 0, policy_version 868135 (0.0008) [2023-12-27 11:39:20,103][20381] Updated weights for policy 1, policy_version 867393 (0.0008) [2023-12-27 11:39:20,167][20381] Updated weights for policy 1, policy_version 867403 (0.0007) [2023-12-27 11:39:20,232][20381] Updated weights for policy 1, policy_version 867413 (0.0006) [2023-12-27 11:39:20,652][20347] Updated weights for policy 0, policy_version 868145 (0.0009) [2023-12-27 11:39:20,719][20347] Updated weights for policy 0, policy_version 868155 (0.0008) [2023-12-27 11:39:20,785][20347] Updated weights for policy 0, policy_version 868165 (0.0008) [2023-12-27 11:39:20,850][20347] Updated weights for policy 0, policy_version 868175 (0.0007) [2023-12-27 11:39:20,936][20381] Updated weights for policy 1, policy_version 867423 (0.0007) [2023-12-27 11:39:21,005][20381] Updated weights for policy 1, policy_version 867433 (0.0006) [2023-12-27 11:39:21,072][20381] Updated weights for policy 1, policy_version 867443 (0.0008) [2023-12-27 11:39:21,715][20347] Updated weights for policy 0, policy_version 868185 (0.0009) [2023-12-27 11:39:21,781][20347] Updated weights for policy 0, policy_version 868195 (0.0007) [2023-12-27 11:39:21,846][20381] Updated weights for policy 1, policy_version 867453 (0.0007) [2023-12-27 11:39:21,847][20347] Updated weights for policy 0, policy_version 868205 (0.0008) [2023-12-27 11:39:21,908][20381] Updated weights for policy 1, policy_version 867463 (0.0006) [2023-12-27 11:39:21,965][20381] Updated weights for policy 1, policy_version 867473 (0.0007) [2023-12-27 11:39:22,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 19910.7). Total num frames: 444391424. Throughput: 0: 9555.7, 1: 9510.0. Samples: 444365996. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:22,363][19170] Avg episode reward: [(0, '695.200'), (1, '902.500')] [2023-12-27 11:39:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000867480_222101504.pth... [2023-12-27 11:39:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000868208_222289920.pth... [2023-12-27 11:39:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000866360_221814784.pth [2023-12-27 11:39:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000867120_222011392.pth [2023-12-27 11:39:22,649][20347] Updated weights for policy 0, policy_version 868215 (0.0007) [2023-12-27 11:39:22,719][20347] Updated weights for policy 0, policy_version 868225 (0.0007) [2023-12-27 11:39:22,732][20381] Updated weights for policy 1, policy_version 867483 (0.0007) [2023-12-27 11:39:22,776][20347] Updated weights for policy 0, policy_version 868235 (0.0008) [2023-12-27 11:39:22,796][20381] Updated weights for policy 1, policy_version 867493 (0.0008) [2023-12-27 11:39:22,858][20381] Updated weights for policy 1, policy_version 867503 (0.0008) [2023-12-27 11:39:23,491][20347] Updated weights for policy 0, policy_version 868245 (0.0007) [2023-12-27 11:39:23,544][20347] Updated weights for policy 0, policy_version 868255 (0.0005) [2023-12-27 11:39:23,583][20381] Updated weights for policy 1, policy_version 867513 (0.0008) [2023-12-27 11:39:23,612][20347] Updated weights for policy 0, policy_version 868265 (0.0007) [2023-12-27 11:39:23,642][20381] Updated weights for policy 1, policy_version 867523 (0.0007) [2023-12-27 11:39:23,695][20381] Updated weights for policy 1, policy_version 867533 (0.0008) [2023-12-27 11:39:23,748][20381] Updated weights for policy 1, policy_version 867543 (0.0007) [2023-12-27 11:39:24,267][20347] Updated weights for policy 0, policy_version 868275 (0.0006) [2023-12-27 11:39:24,329][20347] Updated weights for policy 0, policy_version 868285 (0.0007) [2023-12-27 11:39:24,386][20347] Updated weights for policy 0, policy_version 868295 (0.0006) [2023-12-27 11:39:24,486][20381] Updated weights for policy 1, policy_version 867553 (0.0008) [2023-12-27 11:39:24,556][20381] Updated weights for policy 1, policy_version 867563 (0.0009) [2023-12-27 11:39:24,615][20381] Updated weights for policy 1, policy_version 867573 (0.0008) [2023-12-27 11:39:25,014][20347] Updated weights for policy 0, policy_version 868305 (0.0007) [2023-12-27 11:39:25,076][20347] Updated weights for policy 0, policy_version 868315 (0.0008) [2023-12-27 11:39:25,134][20347] Updated weights for policy 0, policy_version 868325 (0.0008) [2023-12-27 11:39:25,188][20347] Updated weights for policy 0, policy_version 868335 (0.0007) [2023-12-27 11:39:25,331][20381] Updated weights for policy 1, policy_version 867583 (0.0008) [2023-12-27 11:39:25,389][20381] Updated weights for policy 1, policy_version 867593 (0.0008) [2023-12-27 11:39:25,443][20381] Updated weights for policy 1, policy_version 867603 (0.0008) [2023-12-27 11:39:25,878][20347] Updated weights for policy 0, policy_version 868345 (0.0008) [2023-12-27 11:39:25,931][20347] Updated weights for policy 0, policy_version 868355 (0.0007) [2023-12-27 11:39:25,991][20347] Updated weights for policy 0, policy_version 868365 (0.0008) [2023-12-27 11:39:26,199][20381] Updated weights for policy 1, policy_version 867613 (0.0008) [2023-12-27 11:39:26,255][20381] Updated weights for policy 1, policy_version 867623 (0.0006) [2023-12-27 11:39:26,313][20381] Updated weights for policy 1, policy_version 867633 (0.0008) [2023-12-27 11:39:26,633][20347] Updated weights for policy 0, policy_version 868375 (0.0007) [2023-12-27 11:39:26,693][20347] Updated weights for policy 0, policy_version 868385 (0.0006) [2023-12-27 11:39:26,740][20347] Updated weights for policy 0, policy_version 868395 (0.0007) [2023-12-27 11:39:26,976][20381] Updated weights for policy 1, policy_version 867643 (0.0007) [2023-12-27 11:39:27,036][20381] Updated weights for policy 1, policy_version 867653 (0.0008) [2023-12-27 11:39:27,094][20381] Updated weights for policy 1, policy_version 867663 (0.0007) [2023-12-27 11:39:27,322][20347] Updated weights for policy 0, policy_version 868405 (0.0008) [2023-12-27 11:39:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19114.7, 300 sec: 19883.0). Total num frames: 444489728. Throughput: 0: 9555.7, 1: 9509.2. Samples: 444479304. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:27,363][19170] Avg episode reward: [(0, '794.380'), (1, '902.650')] [2023-12-27 11:39:27,392][20347] Updated weights for policy 0, policy_version 868415 (0.0006) [2023-12-27 11:39:27,485][20347] Updated weights for policy 0, policy_version 868425 (0.0006) [2023-12-27 11:39:27,823][20381] Updated weights for policy 1, policy_version 867673 (0.0008) [2023-12-27 11:39:27,875][20381] Updated weights for policy 1, policy_version 867683 (0.0005) [2023-12-27 11:39:27,933][20381] Updated weights for policy 1, policy_version 867693 (0.0006) [2023-12-27 11:39:27,993][20381] Updated weights for policy 1, policy_version 867703 (0.0006) [2023-12-27 11:39:28,079][20347] Updated weights for policy 0, policy_version 868435 (0.0007) [2023-12-27 11:39:28,134][20347] Updated weights for policy 0, policy_version 868445 (0.0010) [2023-12-27 11:39:28,196][20347] Updated weights for policy 0, policy_version 868455 (0.0010) [2023-12-27 11:39:28,647][20381] Updated weights for policy 1, policy_version 867713 (0.0008) [2023-12-27 11:39:28,703][20381] Updated weights for policy 1, policy_version 867723 (0.0008) [2023-12-27 11:39:28,762][20381] Updated weights for policy 1, policy_version 867733 (0.0008) [2023-12-27 11:39:28,876][20347] Updated weights for policy 0, policy_version 868465 (0.0010) [2023-12-27 11:39:28,936][20347] Updated weights for policy 0, policy_version 868475 (0.0008) [2023-12-27 11:39:28,994][20347] Updated weights for policy 0, policy_version 868485 (0.0008) [2023-12-27 11:39:29,051][20347] Updated weights for policy 0, policy_version 868495 (0.0010) [2023-12-27 11:39:29,519][20381] Updated weights for policy 1, policy_version 867743 (0.0010) [2023-12-27 11:39:29,579][20381] Updated weights for policy 1, policy_version 867753 (0.0007) [2023-12-27 11:39:29,640][20381] Updated weights for policy 1, policy_version 867763 (0.0011) [2023-12-27 11:39:29,722][20347] Updated weights for policy 0, policy_version 868505 (0.0011) [2023-12-27 11:39:29,782][20347] Updated weights for policy 0, policy_version 868515 (0.0009) [2023-12-27 11:39:29,851][20347] Updated weights for policy 0, policy_version 868525 (0.0007) [2023-12-27 11:39:30,316][20381] Updated weights for policy 1, policy_version 867773 (0.0011) [2023-12-27 11:39:30,376][20381] Updated weights for policy 1, policy_version 867783 (0.0011) [2023-12-27 11:39:30,436][20381] Updated weights for policy 1, policy_version 867793 (0.0006) [2023-12-27 11:39:30,541][20347] Updated weights for policy 0, policy_version 868535 (0.0009) [2023-12-27 11:39:30,598][20347] Updated weights for policy 0, policy_version 868545 (0.0008) [2023-12-27 11:39:30,647][20347] Updated weights for policy 0, policy_version 868555 (0.0008) [2023-12-27 11:39:31,027][20381] Updated weights for policy 1, policy_version 867803 (0.0006) [2023-12-27 11:39:31,092][20381] Updated weights for policy 1, policy_version 867813 (0.0010) [2023-12-27 11:39:31,156][20381] Updated weights for policy 1, policy_version 867823 (0.0007) [2023-12-27 11:39:31,416][20347] Updated weights for policy 0, policy_version 868565 (0.0008) [2023-12-27 11:39:31,478][20347] Updated weights for policy 0, policy_version 868575 (0.0008) [2023-12-27 11:39:31,536][20347] Updated weights for policy 0, policy_version 868585 (0.0008) [2023-12-27 11:39:31,844][20381] Updated weights for policy 1, policy_version 867833 (0.0008) [2023-12-27 11:39:31,901][20381] Updated weights for policy 1, policy_version 867843 (0.0008) [2023-12-27 11:39:31,951][20381] Updated weights for policy 1, policy_version 867853 (0.0011) [2023-12-27 11:39:32,008][20381] Updated weights for policy 1, policy_version 867863 (0.0007) [2023-12-27 11:39:32,193][20347] Updated weights for policy 0, policy_version 868595 (0.0007) [2023-12-27 11:39:32,240][20347] Updated weights for policy 0, policy_version 868605 (0.0007) [2023-12-27 11:39:32,294][20347] Updated weights for policy 0, policy_version 868615 (0.0009) [2023-12-27 11:39:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19251.2, 300 sec: 19883.0). Total num frames: 444596224. Throughput: 0: 9600.4, 1: 9549.9. Samples: 444601704. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:32,363][19170] Avg episode reward: [(0, '716.450'), (1, '892.900')] [2023-12-27 11:39:32,730][20381] Updated weights for policy 1, policy_version 867873 (0.0009) [2023-12-27 11:39:32,777][20381] Updated weights for policy 1, policy_version 867883 (0.0008) [2023-12-27 11:39:32,835][20381] Updated weights for policy 1, policy_version 867893 (0.0008) [2023-12-27 11:39:33,040][20347] Updated weights for policy 0, policy_version 868625 (0.0007) [2023-12-27 11:39:33,096][20347] Updated weights for policy 0, policy_version 868635 (0.0008) [2023-12-27 11:39:33,147][20347] Updated weights for policy 0, policy_version 868645 (0.0006) [2023-12-27 11:39:33,205][20347] Updated weights for policy 0, policy_version 868655 (0.0007) [2023-12-27 11:39:33,442][20381] Updated weights for policy 1, policy_version 867903 (0.0006) [2023-12-27 11:39:33,497][20381] Updated weights for policy 1, policy_version 867913 (0.0006) [2023-12-27 11:39:33,550][20381] Updated weights for policy 1, policy_version 867923 (0.0006) [2023-12-27 11:39:33,922][20347] Updated weights for policy 0, policy_version 868665 (0.0009) [2023-12-27 11:39:33,980][20347] Updated weights for policy 0, policy_version 868675 (0.0009) [2023-12-27 11:39:34,034][20347] Updated weights for policy 0, policy_version 868685 (0.0009) [2023-12-27 11:39:34,307][20381] Updated weights for policy 1, policy_version 867933 (0.0007) [2023-12-27 11:39:34,373][20381] Updated weights for policy 1, policy_version 867943 (0.0008) [2023-12-27 11:39:34,437][20381] Updated weights for policy 1, policy_version 867953 (0.0008) [2023-12-27 11:39:34,817][20347] Updated weights for policy 0, policy_version 868695 (0.0007) [2023-12-27 11:39:34,875][20347] Updated weights for policy 0, policy_version 868705 (0.0006) [2023-12-27 11:39:34,935][20347] Updated weights for policy 0, policy_version 868715 (0.0007) [2023-12-27 11:39:35,108][20381] Updated weights for policy 1, policy_version 867963 (0.0011) [2023-12-27 11:39:35,172][20381] Updated weights for policy 1, policy_version 867973 (0.0011) [2023-12-27 11:39:35,236][20381] Updated weights for policy 1, policy_version 867983 (0.0010) [2023-12-27 11:39:35,603][20347] Updated weights for policy 0, policy_version 868725 (0.0007) [2023-12-27 11:39:35,653][20347] Updated weights for policy 0, policy_version 868735 (0.0008) [2023-12-27 11:39:35,709][20347] Updated weights for policy 0, policy_version 868745 (0.0007) [2023-12-27 11:39:35,894][20381] Updated weights for policy 1, policy_version 867993 (0.0011) [2023-12-27 11:39:35,948][20381] Updated weights for policy 1, policy_version 868003 (0.0011) [2023-12-27 11:39:36,007][20381] Updated weights for policy 1, policy_version 868013 (0.0011) [2023-12-27 11:39:36,064][20381] Updated weights for policy 1, policy_version 868023 (0.0011) [2023-12-27 11:39:36,462][20347] Updated weights for policy 0, policy_version 868755 (0.0008) [2023-12-27 11:39:36,529][20347] Updated weights for policy 0, policy_version 868765 (0.0008) [2023-12-27 11:39:36,598][20347] Updated weights for policy 0, policy_version 868775 (0.0008) [2023-12-27 11:39:36,780][20381] Updated weights for policy 1, policy_version 868033 (0.0009) [2023-12-27 11:39:36,841][20381] Updated weights for policy 1, policy_version 868043 (0.0008) [2023-12-27 11:39:36,902][20381] Updated weights for policy 1, policy_version 868053 (0.0010) [2023-12-27 11:39:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19114.6, 300 sec: 19855.2). Total num frames: 444686336. Throughput: 0: 9614.3, 1: 9590.0. Samples: 444661144. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:37,363][19170] Avg episode reward: [(0, '735.010'), (1, '922.020')] [2023-12-27 11:39:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000868056_222248960.pth... [2023-12-27 11:39:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000868784_222437376.pth... [2023-12-27 11:39:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000866936_221962240.pth [2023-12-27 11:39:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000867664_222150656.pth [2023-12-27 11:39:37,409][20347] Updated weights for policy 0, policy_version 868785 (0.0008) [2023-12-27 11:39:37,465][20347] Updated weights for policy 0, policy_version 868795 (0.0008) [2023-12-27 11:39:37,526][20347] Updated weights for policy 0, policy_version 868805 (0.0008) [2023-12-27 11:39:37,584][20347] Updated weights for policy 0, policy_version 868815 (0.0008) [2023-12-27 11:39:37,594][20381] Updated weights for policy 1, policy_version 868063 (0.0007) [2023-12-27 11:39:37,660][20381] Updated weights for policy 1, policy_version 868073 (0.0009) [2023-12-27 11:39:37,718][20381] Updated weights for policy 1, policy_version 868083 (0.0008) [2023-12-27 11:39:38,306][20347] Updated weights for policy 0, policy_version 868825 (0.0007) [2023-12-27 11:39:38,375][20347] Updated weights for policy 0, policy_version 868835 (0.0007) [2023-12-27 11:39:38,390][20381] Updated weights for policy 1, policy_version 868093 (0.0008) [2023-12-27 11:39:38,436][20347] Updated weights for policy 0, policy_version 868845 (0.0006) [2023-12-27 11:39:38,450][20381] Updated weights for policy 1, policy_version 868103 (0.0008) [2023-12-27 11:39:38,517][20381] Updated weights for policy 1, policy_version 868113 (0.0008) [2023-12-27 11:39:39,161][20347] Updated weights for policy 0, policy_version 868855 (0.0008) [2023-12-27 11:39:39,222][20381] Updated weights for policy 1, policy_version 868123 (0.0008) [2023-12-27 11:39:39,232][20347] Updated weights for policy 0, policy_version 868865 (0.0006) [2023-12-27 11:39:39,280][20381] Updated weights for policy 1, policy_version 868133 (0.0007) [2023-12-27 11:39:39,293][20347] Updated weights for policy 0, policy_version 868875 (0.0006) [2023-12-27 11:39:39,337][20381] Updated weights for policy 1, policy_version 868143 (0.0006) [2023-12-27 11:39:40,086][20347] Updated weights for policy 0, policy_version 868885 (0.0007) [2023-12-27 11:39:40,110][20381] Updated weights for policy 1, policy_version 868153 (0.0007) [2023-12-27 11:39:40,149][20347] Updated weights for policy 0, policy_version 868895 (0.0010) [2023-12-27 11:39:40,172][20381] Updated weights for policy 1, policy_version 868163 (0.0008) [2023-12-27 11:39:40,206][20347] Updated weights for policy 0, policy_version 868905 (0.0006) [2023-12-27 11:39:40,239][20381] Updated weights for policy 1, policy_version 868173 (0.0008) [2023-12-27 11:39:40,298][20381] Updated weights for policy 1, policy_version 868183 (0.0008) [2023-12-27 11:39:40,961][20347] Updated weights for policy 0, policy_version 868915 (0.0009) [2023-12-27 11:39:41,028][20347] Updated weights for policy 0, policy_version 868925 (0.0007) [2023-12-27 11:39:41,060][20381] Updated weights for policy 1, policy_version 868193 (0.0007) [2023-12-27 11:39:41,099][20347] Updated weights for policy 0, policy_version 868935 (0.0008) [2023-12-27 11:39:41,123][20381] Updated weights for policy 1, policy_version 868203 (0.0007) [2023-12-27 11:39:41,191][20381] Updated weights for policy 1, policy_version 868213 (0.0008) [2023-12-27 11:39:41,901][20347] Updated weights for policy 0, policy_version 868945 (0.0008) [2023-12-27 11:39:41,944][20381] Updated weights for policy 1, policy_version 868223 (0.0006) [2023-12-27 11:39:41,962][20347] Updated weights for policy 0, policy_version 868955 (0.0007) [2023-12-27 11:39:42,011][20381] Updated weights for policy 1, policy_version 868233 (0.0008) [2023-12-27 11:39:42,030][20347] Updated weights for policy 0, policy_version 868965 (0.0008) [2023-12-27 11:39:42,077][20381] Updated weights for policy 1, policy_version 868243 (0.0006) [2023-12-27 11:39:42,090][20347] Updated weights for policy 0, policy_version 868975 (0.0008) [2023-12-27 11:39:42,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19387.7, 300 sec: 19827.4). Total num frames: 444784640. Throughput: 0: 9619.1, 1: 9553.8. Samples: 444774256. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:42,363][19170] Avg episode reward: [(0, '774.280'), (1, '970.860')] [2023-12-27 11:39:42,772][20381] Updated weights for policy 1, policy_version 868253 (0.0008) [2023-12-27 11:39:42,834][20381] Updated weights for policy 1, policy_version 868263 (0.0008) [2023-12-27 11:39:42,842][20347] Updated weights for policy 0, policy_version 868985 (0.0006) [2023-12-27 11:39:42,894][20381] Updated weights for policy 1, policy_version 868273 (0.0008) [2023-12-27 11:39:42,899][20347] Updated weights for policy 0, policy_version 868995 (0.0006) [2023-12-27 11:39:42,956][20347] Updated weights for policy 0, policy_version 869005 (0.0007) [2023-12-27 11:39:43,614][20347] Updated weights for policy 0, policy_version 869015 (0.0008) [2023-12-27 11:39:43,638][20381] Updated weights for policy 1, policy_version 868283 (0.0006) [2023-12-27 11:39:43,680][20347] Updated weights for policy 0, policy_version 869025 (0.0007) [2023-12-27 11:39:43,707][20381] Updated weights for policy 1, policy_version 868293 (0.0007) [2023-12-27 11:39:43,738][20347] Updated weights for policy 0, policy_version 869035 (0.0007) [2023-12-27 11:39:43,772][20381] Updated weights for policy 1, policy_version 868303 (0.0008) [2023-12-27 11:39:44,392][20347] Updated weights for policy 0, policy_version 869045 (0.0008) [2023-12-27 11:39:44,457][20347] Updated weights for policy 0, policy_version 869055 (0.0006) [2023-12-27 11:39:44,471][20381] Updated weights for policy 1, policy_version 868313 (0.0008) [2023-12-27 11:39:44,519][20347] Updated weights for policy 0, policy_version 869065 (0.0006) [2023-12-27 11:39:44,521][20381] Updated weights for policy 1, policy_version 868323 (0.0006) [2023-12-27 11:39:44,579][20381] Updated weights for policy 1, policy_version 868333 (0.0008) [2023-12-27 11:39:44,617][20292] KL-divergence is very high: 164.1736 [2023-12-27 11:39:44,642][20381] Updated weights for policy 1, policy_version 868343 (0.0007) [2023-12-27 11:39:45,195][20347] Updated weights for policy 0, policy_version 869075 (0.0007) [2023-12-27 11:39:45,261][20347] Updated weights for policy 0, policy_version 869085 (0.0008) [2023-12-27 11:39:45,329][20347] Updated weights for policy 0, policy_version 869095 (0.0008) [2023-12-27 11:39:45,405][20381] Updated weights for policy 1, policy_version 868353 (0.0007) [2023-12-27 11:39:45,468][20381] Updated weights for policy 1, policy_version 868363 (0.0008) [2023-12-27 11:39:45,527][20381] Updated weights for policy 1, policy_version 868373 (0.0006) [2023-12-27 11:39:45,945][20347] Updated weights for policy 0, policy_version 869105 (0.0006) [2023-12-27 11:39:46,005][20347] Updated weights for policy 0, policy_version 869115 (0.0008) [2023-12-27 11:39:46,067][20347] Updated weights for policy 0, policy_version 869125 (0.0008) [2023-12-27 11:39:46,127][20347] Updated weights for policy 0, policy_version 869135 (0.0007) [2023-12-27 11:39:46,189][20381] Updated weights for policy 1, policy_version 868383 (0.0007) [2023-12-27 11:39:46,244][20381] Updated weights for policy 1, policy_version 868393 (0.0008) [2023-12-27 11:39:46,297][20381] Updated weights for policy 1, policy_version 868403 (0.0008) [2023-12-27 11:39:46,810][20347] Updated weights for policy 0, policy_version 869145 (0.0007) [2023-12-27 11:39:46,869][20347] Updated weights for policy 0, policy_version 869155 (0.0009) [2023-12-27 11:39:46,925][20347] Updated weights for policy 0, policy_version 869165 (0.0007) [2023-12-27 11:39:47,050][20381] Updated weights for policy 1, policy_version 868413 (0.0008) [2023-12-27 11:39:47,111][20381] Updated weights for policy 1, policy_version 868423 (0.0008) [2023-12-27 11:39:47,173][20381] Updated weights for policy 1, policy_version 868433 (0.0008) [2023-12-27 11:39:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19827.4). Total num frames: 444882944. Throughput: 0: 9661.4, 1: 9587.4. Samples: 444890692. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:47,363][19170] Avg episode reward: [(0, '764.530'), (1, '941.920')] [2023-12-27 11:39:47,539][20347] Updated weights for policy 0, policy_version 869175 (0.0006) [2023-12-27 11:39:47,599][20347] Updated weights for policy 0, policy_version 869185 (0.0007) [2023-12-27 11:39:47,662][20347] Updated weights for policy 0, policy_version 869195 (0.0008) [2023-12-27 11:39:47,899][20381] Updated weights for policy 1, policy_version 868443 (0.0009) [2023-12-27 11:39:47,956][20381] Updated weights for policy 1, policy_version 868453 (0.0008) [2023-12-27 11:39:48,018][20381] Updated weights for policy 1, policy_version 868463 (0.0008) [2023-12-27 11:39:48,337][20347] Updated weights for policy 0, policy_version 869205 (0.0008) [2023-12-27 11:39:48,404][20347] Updated weights for policy 0, policy_version 869215 (0.0008) [2023-12-27 11:39:48,466][20347] Updated weights for policy 0, policy_version 869225 (0.0008) [2023-12-27 11:39:48,761][20381] Updated weights for policy 1, policy_version 868473 (0.0008) [2023-12-27 11:39:48,827][20381] Updated weights for policy 1, policy_version 868483 (0.0008) [2023-12-27 11:39:48,890][20381] Updated weights for policy 1, policy_version 868493 (0.0007) [2023-12-27 11:39:48,953][20381] Updated weights for policy 1, policy_version 868503 (0.0007) [2023-12-27 11:39:49,193][20347] Updated weights for policy 0, policy_version 869235 (0.0008) [2023-12-27 11:39:49,262][20347] Updated weights for policy 0, policy_version 869245 (0.0007) [2023-12-27 11:39:49,333][20347] Updated weights for policy 0, policy_version 869255 (0.0008) [2023-12-27 11:39:49,672][20381] Updated weights for policy 1, policy_version 868513 (0.0007) [2023-12-27 11:39:49,735][20381] Updated weights for policy 1, policy_version 868523 (0.0008) [2023-12-27 11:39:49,790][20381] Updated weights for policy 1, policy_version 868533 (0.0007) [2023-12-27 11:39:50,066][20347] Updated weights for policy 0, policy_version 869265 (0.0008) [2023-12-27 11:39:50,123][20347] Updated weights for policy 0, policy_version 869275 (0.0006) [2023-12-27 11:39:50,184][20347] Updated weights for policy 0, policy_version 869285 (0.0007) [2023-12-27 11:39:50,251][20347] Updated weights for policy 0, policy_version 869295 (0.0007) [2023-12-27 11:39:50,570][20381] Updated weights for policy 1, policy_version 868543 (0.0008) [2023-12-27 11:39:50,636][20381] Updated weights for policy 1, policy_version 868553 (0.0007) [2023-12-27 11:39:50,663][20292] KL-divergence is very high: 106.7225 [2023-12-27 11:39:50,698][20381] Updated weights for policy 1, policy_version 868563 (0.0007) [2023-12-27 11:39:51,028][20347] Updated weights for policy 0, policy_version 869305 (0.0007) [2023-12-27 11:39:51,093][20347] Updated weights for policy 0, policy_version 869315 (0.0007) [2023-12-27 11:39:51,155][20347] Updated weights for policy 0, policy_version 869325 (0.0007) [2023-12-27 11:39:51,409][20292] KL-divergence is very high: 107.1559 [2023-12-27 11:39:51,420][20381] Updated weights for policy 1, policy_version 868573 (0.0008) [2023-12-27 11:39:51,435][20292] KL-divergence is very high: 100.3132 [2023-12-27 11:39:51,461][20292] KL-divergence is very high: 110.4109 [2023-12-27 11:39:51,487][20381] Updated weights for policy 1, policy_version 868583 (0.0008) [2023-12-27 11:39:51,534][20292] KL-divergence is very high: 101.4332 [2023-12-27 11:39:51,546][20381] Updated weights for policy 1, policy_version 868593 (0.0007) [2023-12-27 11:39:51,584][20292] KL-divergence is very high: 116.4567 [2023-12-27 11:39:51,899][20347] Updated weights for policy 0, policy_version 869335 (0.0008) [2023-12-27 11:39:51,965][20347] Updated weights for policy 0, policy_version 869345 (0.0007) [2023-12-27 11:39:52,025][20347] Updated weights for policy 0, policy_version 869355 (0.0006) [2023-12-27 11:39:52,277][20381] Updated weights for policy 1, policy_version 868603 (0.0007) [2023-12-27 11:39:52,339][20381] Updated weights for policy 1, policy_version 868613 (0.0008) [2023-12-27 11:39:52,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19771.9). Total num frames: 444973056. Throughput: 0: 9687.2, 1: 9595.8. Samples: 444949016. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:52,363][19170] Avg episode reward: [(0, '754.360'), (1, '901.840')] [2023-12-27 11:39:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000869360_222584832.pth... [2023-12-27 11:39:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000868208_222289920.pth [2023-12-27 11:39:52,377][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000869360_222584832.pth [2023-12-27 11:39:52,407][20381] Updated weights for policy 1, policy_version 868623 (0.0008) [2023-12-27 11:39:52,465][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000868632_222396416.pth... [2023-12-27 11:39:52,469][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000867480_222101504.pth [2023-12-27 11:39:52,470][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000868632_222396416.pth [2023-12-27 11:39:52,712][20347] Updated weights for policy 0, policy_version 869365 (0.0007) [2023-12-27 11:39:52,772][20347] Updated weights for policy 0, policy_version 869375 (0.0008) [2023-12-27 11:39:52,837][20347] Updated weights for policy 0, policy_version 869385 (0.0008) [2023-12-27 11:39:53,122][20381] Updated weights for policy 1, policy_version 868633 (0.0008) [2023-12-27 11:39:53,178][20381] Updated weights for policy 1, policy_version 868643 (0.0008) [2023-12-27 11:39:53,238][20381] Updated weights for policy 1, policy_version 868653 (0.0007) [2023-12-27 11:39:53,303][20381] Updated weights for policy 1, policy_version 868663 (0.0005) [2023-12-27 11:39:53,556][20347] Updated weights for policy 0, policy_version 869395 (0.0008) [2023-12-27 11:39:53,606][20347] Updated weights for policy 0, policy_version 869405 (0.0008) [2023-12-27 11:39:53,664][20347] Updated weights for policy 0, policy_version 869415 (0.0007) [2023-12-27 11:39:53,974][20381] Updated weights for policy 1, policy_version 868673 (0.0006) [2023-12-27 11:39:54,029][20381] Updated weights for policy 1, policy_version 868683 (0.0006) [2023-12-27 11:39:54,083][20381] Updated weights for policy 1, policy_version 868693 (0.0006) [2023-12-27 11:39:54,246][20347] Updated weights for policy 0, policy_version 869425 (0.0006) [2023-12-27 11:39:54,305][20347] Updated weights for policy 0, policy_version 869435 (0.0008) [2023-12-27 11:39:54,368][20347] Updated weights for policy 0, policy_version 869445 (0.0008) [2023-12-27 11:39:54,427][20347] Updated weights for policy 0, policy_version 869455 (0.0008) [2023-12-27 11:39:54,762][20381] Updated weights for policy 1, policy_version 868703 (0.0006) [2023-12-27 11:39:54,824][20381] Updated weights for policy 1, policy_version 868713 (0.0006) [2023-12-27 11:39:54,879][20381] Updated weights for policy 1, policy_version 868723 (0.0007) [2023-12-27 11:39:55,178][20347] Updated weights for policy 0, policy_version 869465 (0.0008) [2023-12-27 11:39:55,235][20347] Updated weights for policy 0, policy_version 869475 (0.0007) [2023-12-27 11:39:55,293][20347] Updated weights for policy 0, policy_version 869485 (0.0008) [2023-12-27 11:39:55,544][20381] Updated weights for policy 1, policy_version 868733 (0.0006) [2023-12-27 11:39:55,603][20381] Updated weights for policy 1, policy_version 868743 (0.0006) [2023-12-27 11:39:55,663][20381] Updated weights for policy 1, policy_version 868753 (0.0007) [2023-12-27 11:39:56,038][20347] Updated weights for policy 0, policy_version 869495 (0.0009) [2023-12-27 11:39:56,086][20347] Updated weights for policy 0, policy_version 869505 (0.0008) [2023-12-27 11:39:56,151][20347] Updated weights for policy 0, policy_version 869515 (0.0010) [2023-12-27 11:39:56,316][20381] Updated weights for policy 1, policy_version 868763 (0.0008) [2023-12-27 11:39:56,373][20381] Updated weights for policy 1, policy_version 868773 (0.0007) [2023-12-27 11:39:56,435][20381] Updated weights for policy 1, policy_version 868783 (0.0006) [2023-12-27 11:39:56,906][20347] Updated weights for policy 0, policy_version 869525 (0.0008) [2023-12-27 11:39:56,960][20347] Updated weights for policy 0, policy_version 869535 (0.0008) [2023-12-27 11:39:57,015][20347] Updated weights for policy 0, policy_version 869545 (0.0008) [2023-12-27 11:39:57,088][20381] Updated weights for policy 1, policy_version 868793 (0.0008) [2023-12-27 11:39:57,146][20381] Updated weights for policy 1, policy_version 868803 (0.0006) [2023-12-27 11:39:57,206][20381] Updated weights for policy 1, policy_version 868813 (0.0007) [2023-12-27 11:39:57,266][20381] Updated weights for policy 1, policy_version 868823 (0.0008) [2023-12-27 11:39:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19799.6). Total num frames: 445079552. Throughput: 0: 9687.4, 1: 9674.1. Samples: 445065984. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:39:57,363][19170] Avg episode reward: [(0, '725.090'), (1, '770.700')] [2023-12-27 11:39:57,731][20347] Updated weights for policy 0, policy_version 869555 (0.0007) [2023-12-27 11:39:57,787][20347] Updated weights for policy 0, policy_version 869565 (0.0006) [2023-12-27 11:39:57,845][20347] Updated weights for policy 0, policy_version 869575 (0.0007) [2023-12-27 11:39:57,940][20381] Updated weights for policy 1, policy_version 868833 (0.0010) [2023-12-27 11:39:58,001][20381] Updated weights for policy 1, policy_version 868843 (0.0011) [2023-12-27 11:39:58,052][20381] Updated weights for policy 1, policy_version 868853 (0.0007) [2023-12-27 11:39:58,618][20347] Updated weights for policy 0, policy_version 869585 (0.0008) [2023-12-27 11:39:58,690][20347] Updated weights for policy 0, policy_version 869595 (0.0008) [2023-12-27 11:39:58,756][20347] Updated weights for policy 0, policy_version 869605 (0.0008) [2023-12-27 11:39:58,825][20347] Updated weights for policy 0, policy_version 869615 (0.0008) [2023-12-27 11:39:58,939][20381] Updated weights for policy 1, policy_version 868863 (0.0010) [2023-12-27 11:39:59,007][20381] Updated weights for policy 1, policy_version 868873 (0.0010) [2023-12-27 11:39:59,070][20381] Updated weights for policy 1, policy_version 868883 (0.0008) [2023-12-27 11:39:59,636][20347] Updated weights for policy 0, policy_version 869625 (0.0008) [2023-12-27 11:39:59,689][20347] Updated weights for policy 0, policy_version 869635 (0.0008) [2023-12-27 11:39:59,752][20347] Updated weights for policy 0, policy_version 869645 (0.0008) [2023-12-27 11:39:59,772][20381] Updated weights for policy 1, policy_version 868893 (0.0008) [2023-12-27 11:39:59,832][20381] Updated weights for policy 1, policy_version 868903 (0.0007) [2023-12-27 11:39:59,888][20381] Updated weights for policy 1, policy_version 868913 (0.0007) [2023-12-27 11:40:00,536][20347] Updated weights for policy 0, policy_version 869655 (0.0008) [2023-12-27 11:40:00,565][20381] Updated weights for policy 1, policy_version 868923 (0.0008) [2023-12-27 11:40:00,595][20347] Updated weights for policy 0, policy_version 869665 (0.0006) [2023-12-27 11:40:00,622][20381] Updated weights for policy 1, policy_version 868933 (0.0006) [2023-12-27 11:40:00,653][20347] Updated weights for policy 0, policy_version 869675 (0.0005) [2023-12-27 11:40:00,680][20381] Updated weights for policy 1, policy_version 868943 (0.0008) [2023-12-27 11:40:01,360][20347] Updated weights for policy 0, policy_version 869685 (0.0007) [2023-12-27 11:40:01,371][20381] Updated weights for policy 1, policy_version 868953 (0.0007) [2023-12-27 11:40:01,432][20347] Updated weights for policy 0, policy_version 869695 (0.0008) [2023-12-27 11:40:01,437][20381] Updated weights for policy 1, policy_version 868963 (0.0008) [2023-12-27 11:40:01,495][20347] Updated weights for policy 0, policy_version 869705 (0.0008) [2023-12-27 11:40:01,502][20381] Updated weights for policy 1, policy_version 868973 (0.0008) [2023-12-27 11:40:01,561][20381] Updated weights for policy 1, policy_version 868983 (0.0007) [2023-12-27 11:40:02,236][20347] Updated weights for policy 0, policy_version 869715 (0.0008) [2023-12-27 11:40:02,303][20347] Updated weights for policy 0, policy_version 869725 (0.0008) [2023-12-27 11:40:02,328][20381] Updated weights for policy 1, policy_version 868993 (0.0007) [2023-12-27 11:40:02,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19114.6, 300 sec: 19716.4). Total num frames: 445161472. Throughput: 0: 9672.4, 1: 9670.3. Samples: 445179104. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:02,363][19170] Avg episode reward: [(0, '764.530'), (1, '258.440')] [2023-12-27 11:40:02,370][20347] Updated weights for policy 0, policy_version 869735 (0.0008) [2023-12-27 11:40:02,399][20381] Updated weights for policy 1, policy_version 869003 (0.0009) [2023-12-27 11:40:02,464][20381] Updated weights for policy 1, policy_version 869013 (0.0007) [2023-12-27 11:40:03,106][20347] Updated weights for policy 0, policy_version 869745 (0.0007) [2023-12-27 11:40:03,163][20347] Updated weights for policy 0, policy_version 869755 (0.0007) [2023-12-27 11:40:03,167][20381] Updated weights for policy 1, policy_version 869023 (0.0006) [2023-12-27 11:40:03,225][20381] Updated weights for policy 1, policy_version 869033 (0.0007) [2023-12-27 11:40:03,225][20347] Updated weights for policy 0, policy_version 869765 (0.0006) [2023-12-27 11:40:03,277][20381] Updated weights for policy 1, policy_version 869043 (0.0007) [2023-12-27 11:40:03,288][20347] Updated weights for policy 0, policy_version 869775 (0.0006) [2023-12-27 11:40:03,895][20347] Updated weights for policy 0, policy_version 869785 (0.0008) [2023-12-27 11:40:03,957][20347] Updated weights for policy 0, policy_version 869795 (0.0008) [2023-12-27 11:40:03,981][20381] Updated weights for policy 1, policy_version 869053 (0.0008) [2023-12-27 11:40:04,016][20347] Updated weights for policy 0, policy_version 869805 (0.0008) [2023-12-27 11:40:04,047][20381] Updated weights for policy 1, policy_version 869063 (0.0008) [2023-12-27 11:40:04,109][20381] Updated weights for policy 1, policy_version 869073 (0.0008) [2023-12-27 11:40:04,735][20347] Updated weights for policy 0, policy_version 869815 (0.0009) [2023-12-27 11:40:04,794][20347] Updated weights for policy 0, policy_version 869825 (0.0010) [2023-12-27 11:40:04,804][20381] Updated weights for policy 1, policy_version 869083 (0.0008) [2023-12-27 11:40:04,847][20347] Updated weights for policy 0, policy_version 869835 (0.0006) [2023-12-27 11:40:04,857][20381] Updated weights for policy 1, policy_version 869093 (0.0005) [2023-12-27 11:40:04,912][20381] Updated weights for policy 1, policy_version 869103 (0.0007) [2023-12-27 11:40:05,526][20347] Updated weights for policy 0, policy_version 869845 (0.0006) [2023-12-27 11:40:05,577][20347] Updated weights for policy 0, policy_version 869855 (0.0008) [2023-12-27 11:40:05,607][20381] Updated weights for policy 1, policy_version 869113 (0.0008) [2023-12-27 11:40:05,631][20347] Updated weights for policy 0, policy_version 869865 (0.0008) [2023-12-27 11:40:05,660][20381] Updated weights for policy 1, policy_version 869123 (0.0007) [2023-12-27 11:40:05,717][20381] Updated weights for policy 1, policy_version 869133 (0.0007) [2023-12-27 11:40:05,774][20381] Updated weights for policy 1, policy_version 869143 (0.0008) [2023-12-27 11:40:06,404][20347] Updated weights for policy 0, policy_version 869875 (0.0008) [2023-12-27 11:40:06,465][20347] Updated weights for policy 0, policy_version 869885 (0.0008) [2023-12-27 11:40:06,525][20347] Updated weights for policy 0, policy_version 869895 (0.0007) [2023-12-27 11:40:06,535][20381] Updated weights for policy 1, policy_version 869153 (0.0007) [2023-12-27 11:40:06,600][20381] Updated weights for policy 1, policy_version 869163 (0.0006) [2023-12-27 11:40:06,663][20381] Updated weights for policy 1, policy_version 869173 (0.0008) [2023-12-27 11:40:07,219][20347] Updated weights for policy 0, policy_version 869905 (0.0006) [2023-12-27 11:40:07,286][20347] Updated weights for policy 0, policy_version 869915 (0.0008) [2023-12-27 11:40:07,357][20347] Updated weights for policy 0, policy_version 869925 (0.0008) [2023-12-27 11:40:07,363][19170] Fps is (10 sec: 18022.3, 60 sec: 19114.6, 300 sec: 19716.3). Total num frames: 445259776. Throughput: 0: 9684.6, 1: 9694.5. Samples: 445238056. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:07,363][19170] Avg episode reward: [(0, '814.000'), (1, '168.710')] [2023-12-27 11:40:07,419][20381] Updated weights for policy 1, policy_version 869183 (0.0007) [2023-12-27 11:40:07,421][20347] Updated weights for policy 0, policy_version 869935 (0.0008) [2023-12-27 11:40:07,424][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000869936_222732288.pth... [2023-12-27 11:40:07,428][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000868784_222437376.pth [2023-12-27 11:40:07,485][20381] Updated weights for policy 1, policy_version 869193 (0.0007) [2023-12-27 11:40:07,548][20381] Updated weights for policy 1, policy_version 869203 (0.0008) [2023-12-27 11:40:07,577][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000869208_222543872.pth... [2023-12-27 11:40:07,580][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000868056_222248960.pth [2023-12-27 11:40:08,141][20347] Updated weights for policy 0, policy_version 869945 (0.0007) [2023-12-27 11:40:08,196][20347] Updated weights for policy 0, policy_version 869955 (0.0006) [2023-12-27 11:40:08,246][20381] Updated weights for policy 1, policy_version 869213 (0.0007) [2023-12-27 11:40:08,265][20347] Updated weights for policy 0, policy_version 869965 (0.0006) [2023-12-27 11:40:08,307][20381] Updated weights for policy 1, policy_version 869223 (0.0007) [2023-12-27 11:40:08,370][20381] Updated weights for policy 1, policy_version 869233 (0.0007) [2023-12-27 11:40:08,928][20347] Updated weights for policy 0, policy_version 869975 (0.0007) [2023-12-27 11:40:08,983][20347] Updated weights for policy 0, policy_version 869985 (0.0008) [2023-12-27 11:40:09,048][20347] Updated weights for policy 0, policy_version 869995 (0.0008) [2023-12-27 11:40:09,069][20381] Updated weights for policy 1, policy_version 869243 (0.0009) [2023-12-27 11:40:09,132][20381] Updated weights for policy 1, policy_version 869253 (0.0008) [2023-12-27 11:40:09,192][20381] Updated weights for policy 1, policy_version 869263 (0.0006) [2023-12-27 11:40:09,859][20347] Updated weights for policy 0, policy_version 870005 (0.0008) [2023-12-27 11:40:09,915][20381] Updated weights for policy 1, policy_version 869273 (0.0008) [2023-12-27 11:40:09,931][20347] Updated weights for policy 0, policy_version 870015 (0.0007) [2023-12-27 11:40:09,978][20381] Updated weights for policy 1, policy_version 869283 (0.0009) [2023-12-27 11:40:09,994][20347] Updated weights for policy 0, policy_version 870025 (0.0008) [2023-12-27 11:40:10,044][20381] Updated weights for policy 1, policy_version 869293 (0.0008) [2023-12-27 11:40:10,109][20381] Updated weights for policy 1, policy_version 869303 (0.0007) [2023-12-27 11:40:10,753][20347] Updated weights for policy 0, policy_version 870035 (0.0008) [2023-12-27 11:40:10,809][20347] Updated weights for policy 0, policy_version 870045 (0.0007) [2023-12-27 11:40:10,842][20381] Updated weights for policy 1, policy_version 869313 (0.0007) [2023-12-27 11:40:10,872][20347] Updated weights for policy 0, policy_version 870055 (0.0007) [2023-12-27 11:40:10,900][20381] Updated weights for policy 1, policy_version 869323 (0.0006) [2023-12-27 11:40:10,956][20381] Updated weights for policy 1, policy_version 869333 (0.0006) [2023-12-27 11:40:11,706][20347] Updated weights for policy 0, policy_version 870065 (0.0007) [2023-12-27 11:40:11,729][20381] Updated weights for policy 1, policy_version 869343 (0.0006) [2023-12-27 11:40:11,781][20347] Updated weights for policy 0, policy_version 870075 (0.0008) [2023-12-27 11:40:11,800][20381] Updated weights for policy 1, policy_version 869353 (0.0008) [2023-12-27 11:40:11,840][20347] Updated weights for policy 0, policy_version 870085 (0.0007) [2023-12-27 11:40:11,864][20381] Updated weights for policy 1, policy_version 869363 (0.0007) [2023-12-27 11:40:11,904][20347] Updated weights for policy 0, policy_version 870095 (0.0007) [2023-12-27 11:40:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19251.2, 300 sec: 19716.3). Total num frames: 445358080. Throughput: 0: 9668.2, 1: 9700.9. Samples: 445350912. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:12,363][19170] Avg episode reward: [(0, '715.610'), (1, '303.110')] [2023-12-27 11:40:12,593][20381] Updated weights for policy 1, policy_version 869373 (0.0007) [2023-12-27 11:40:12,648][20347] Updated weights for policy 0, policy_version 870105 (0.0006) [2023-12-27 11:40:12,658][20381] Updated weights for policy 1, policy_version 869383 (0.0008) [2023-12-27 11:40:12,708][20347] Updated weights for policy 0, policy_version 870115 (0.0008) [2023-12-27 11:40:12,717][20381] Updated weights for policy 1, policy_version 869393 (0.0008) [2023-12-27 11:40:12,758][20347] Updated weights for policy 0, policy_version 870125 (0.0008) [2023-12-27 11:40:13,427][20381] Updated weights for policy 1, policy_version 869403 (0.0008) [2023-12-27 11:40:13,480][20347] Updated weights for policy 0, policy_version 870135 (0.0007) [2023-12-27 11:40:13,480][20381] Updated weights for policy 1, policy_version 869413 (0.0006) [2023-12-27 11:40:13,539][20381] Updated weights for policy 1, policy_version 869423 (0.0005) [2023-12-27 11:40:13,541][20347] Updated weights for policy 0, policy_version 870145 (0.0008) [2023-12-27 11:40:13,606][20347] Updated weights for policy 0, policy_version 870155 (0.0008) [2023-12-27 11:40:14,224][20381] Updated weights for policy 1, policy_version 869433 (0.0006) [2023-12-27 11:40:14,278][20381] Updated weights for policy 1, policy_version 869443 (0.0008) [2023-12-27 11:40:14,330][20347] Updated weights for policy 0, policy_version 870165 (0.0008) [2023-12-27 11:40:14,332][20381] Updated weights for policy 1, policy_version 869453 (0.0006) [2023-12-27 11:40:14,388][20381] Updated weights for policy 1, policy_version 869463 (0.0006) [2023-12-27 11:40:14,389][20347] Updated weights for policy 0, policy_version 870175 (0.0008) [2023-12-27 11:40:14,450][20347] Updated weights for policy 0, policy_version 870185 (0.0008) [2023-12-27 11:40:15,174][20381] Updated weights for policy 1, policy_version 869473 (0.0010) [2023-12-27 11:40:15,205][20347] Updated weights for policy 0, policy_version 870195 (0.0007) [2023-12-27 11:40:15,238][20381] Updated weights for policy 1, policy_version 869483 (0.0011) [2023-12-27 11:40:15,271][20347] Updated weights for policy 0, policy_version 870205 (0.0007) [2023-12-27 11:40:15,298][20381] Updated weights for policy 1, policy_version 869493 (0.0011) [2023-12-27 11:40:15,333][20347] Updated weights for policy 0, policy_version 870215 (0.0007) [2023-12-27 11:40:15,988][20381] Updated weights for policy 1, policy_version 869503 (0.0008) [2023-12-27 11:40:15,999][20347] Updated weights for policy 0, policy_version 870225 (0.0008) [2023-12-27 11:40:16,044][20381] Updated weights for policy 1, policy_version 869513 (0.0008) [2023-12-27 11:40:16,059][20347] Updated weights for policy 0, policy_version 870235 (0.0005) [2023-12-27 11:40:16,099][20381] Updated weights for policy 1, policy_version 869523 (0.0007) [2023-12-27 11:40:16,118][20347] Updated weights for policy 0, policy_version 870245 (0.0005) [2023-12-27 11:40:16,174][20347] Updated weights for policy 0, policy_version 870255 (0.0005) [2023-12-27 11:40:16,754][20347] Updated weights for policy 0, policy_version 870265 (0.0010) [2023-12-27 11:40:16,808][20347] Updated weights for policy 0, policy_version 870275 (0.0011) [2023-12-27 11:40:16,848][20381] Updated weights for policy 1, policy_version 869533 (0.0009) [2023-12-27 11:40:16,861][20347] Updated weights for policy 0, policy_version 870285 (0.0010) [2023-12-27 11:40:16,908][20381] Updated weights for policy 1, policy_version 869543 (0.0010) [2023-12-27 11:40:16,967][20381] Updated weights for policy 1, policy_version 869553 (0.0010) [2023-12-27 11:40:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19716.3). Total num frames: 445456384. Throughput: 0: 9594.0, 1: 9641.7. Samples: 445467308. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:17,363][19170] Avg episode reward: [(0, '773.890'), (1, '410.050')] [2023-12-27 11:40:17,500][20347] Updated weights for policy 0, policy_version 870295 (0.0010) [2023-12-27 11:40:17,552][20347] Updated weights for policy 0, policy_version 870305 (0.0010) [2023-12-27 11:40:17,589][20381] Updated weights for policy 1, policy_version 869563 (0.0009) [2023-12-27 11:40:17,606][20347] Updated weights for policy 0, policy_version 870315 (0.0010) [2023-12-27 11:40:17,643][20381] Updated weights for policy 1, policy_version 869573 (0.0010) [2023-12-27 11:40:17,697][20381] Updated weights for policy 1, policy_version 869583 (0.0010) [2023-12-27 11:40:18,303][20347] Updated weights for policy 0, policy_version 870325 (0.0008) [2023-12-27 11:40:18,365][20347] Updated weights for policy 0, policy_version 870335 (0.0008) [2023-12-27 11:40:18,424][20381] Updated weights for policy 1, policy_version 869593 (0.0010) [2023-12-27 11:40:18,427][20347] Updated weights for policy 0, policy_version 870345 (0.0010) [2023-12-27 11:40:18,486][20381] Updated weights for policy 1, policy_version 869603 (0.0011) [2023-12-27 11:40:18,544][20381] Updated weights for policy 1, policy_version 869613 (0.0011) [2023-12-27 11:40:18,612][20381] Updated weights for policy 1, policy_version 869623 (0.0011) [2023-12-27 11:40:19,114][20347] Updated weights for policy 0, policy_version 870355 (0.0010) [2023-12-27 11:40:19,168][20347] Updated weights for policy 0, policy_version 870365 (0.0008) [2023-12-27 11:40:19,228][20347] Updated weights for policy 0, policy_version 870375 (0.0007) [2023-12-27 11:40:19,330][20381] Updated weights for policy 1, policy_version 869633 (0.0010) [2023-12-27 11:40:19,397][20381] Updated weights for policy 1, policy_version 869643 (0.0008) [2023-12-27 11:40:19,468][20381] Updated weights for policy 1, policy_version 869653 (0.0008) [2023-12-27 11:40:20,040][20347] Updated weights for policy 0, policy_version 870385 (0.0008) [2023-12-27 11:40:20,103][20347] Updated weights for policy 0, policy_version 870395 (0.0009) [2023-12-27 11:40:20,159][20347] Updated weights for policy 0, policy_version 870405 (0.0009) [2023-12-27 11:40:20,217][20347] Updated weights for policy 0, policy_version 870415 (0.0009) [2023-12-27 11:40:20,285][20381] Updated weights for policy 1, policy_version 869663 (0.0008) [2023-12-27 11:40:20,352][20381] Updated weights for policy 1, policy_version 869673 (0.0007) [2023-12-27 11:40:20,407][20381] Updated weights for policy 1, policy_version 869683 (0.0006) [2023-12-27 11:40:21,092][20347] Updated weights for policy 0, policy_version 870425 (0.0010) [2023-12-27 11:40:21,125][20381] Updated weights for policy 1, policy_version 869693 (0.0007) [2023-12-27 11:40:21,157][20347] Updated weights for policy 0, policy_version 870435 (0.0008) [2023-12-27 11:40:21,194][20381] Updated weights for policy 1, policy_version 869703 (0.0008) [2023-12-27 11:40:21,227][20347] Updated weights for policy 0, policy_version 870445 (0.0008) [2023-12-27 11:40:21,263][20381] Updated weights for policy 1, policy_version 869713 (0.0008) [2023-12-27 11:40:21,991][20347] Updated weights for policy 0, policy_version 870455 (0.0006) [2023-12-27 11:40:22,034][20381] Updated weights for policy 1, policy_version 869723 (0.0008) [2023-12-27 11:40:22,057][20347] Updated weights for policy 0, policy_version 870465 (0.0007) [2023-12-27 11:40:22,094][20381] Updated weights for policy 1, policy_version 869733 (0.0012) [2023-12-27 11:40:22,121][20347] Updated weights for policy 0, policy_version 870475 (0.0008) [2023-12-27 11:40:22,159][20381] Updated weights for policy 1, policy_version 869743 (0.0011) [2023-12-27 11:40:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19387.7, 300 sec: 19688.6). Total num frames: 445554688. Throughput: 0: 9608.0, 1: 9597.7. Samples: 445525400. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:22,363][19170] Avg episode reward: [(0, '793.790'), (1, '547.390')] [2023-12-27 11:40:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000869752_222683136.pth... [2023-12-27 11:40:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000870480_222871552.pth... [2023-12-27 11:40:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000868632_222396416.pth [2023-12-27 11:40:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000869360_222584832.pth [2023-12-27 11:40:22,877][20381] Updated weights for policy 1, policy_version 869753 (0.0009) [2023-12-27 11:40:22,885][20347] Updated weights for policy 0, policy_version 870485 (0.0007) [2023-12-27 11:40:22,946][20381] Updated weights for policy 1, policy_version 869763 (0.0010) [2023-12-27 11:40:22,946][20347] Updated weights for policy 0, policy_version 870495 (0.0008) [2023-12-27 11:40:23,000][20347] Updated weights for policy 0, policy_version 870505 (0.0006) [2023-12-27 11:40:23,009][20381] Updated weights for policy 1, policy_version 869773 (0.0010) [2023-12-27 11:40:23,067][20381] Updated weights for policy 1, policy_version 869783 (0.0009) [2023-12-27 11:40:23,744][20381] Updated weights for policy 1, policy_version 869793 (0.0011) [2023-12-27 11:40:23,772][20347] Updated weights for policy 0, policy_version 870515 (0.0007) [2023-12-27 11:40:23,808][20381] Updated weights for policy 1, policy_version 869803 (0.0011) [2023-12-27 11:40:23,830][20347] Updated weights for policy 0, policy_version 870525 (0.0009) [2023-12-27 11:40:23,867][20381] Updated weights for policy 1, policy_version 869813 (0.0010) [2023-12-27 11:40:23,894][20347] Updated weights for policy 0, policy_version 870535 (0.0010) [2023-12-27 11:40:24,562][20381] Updated weights for policy 1, policy_version 869823 (0.0008) [2023-12-27 11:40:24,622][20347] Updated weights for policy 0, policy_version 870545 (0.0008) [2023-12-27 11:40:24,626][20381] Updated weights for policy 1, policy_version 869833 (0.0009) [2023-12-27 11:40:24,666][20291] KL-divergence is very high: 264.0093 [2023-12-27 11:40:24,679][20291] KL-divergence is very high: 284.2832 [2023-12-27 11:40:24,680][20381] Updated weights for policy 1, policy_version 869843 (0.0008) [2023-12-27 11:40:24,685][20347] Updated weights for policy 0, policy_version 870555 (0.0008) [2023-12-27 11:40:24,716][20291] KL-divergence is very high: 232.8759 [2023-12-27 11:40:24,728][20291] KL-divergence is very high: 186.9685 [2023-12-27 11:40:24,747][20347] Updated weights for policy 0, policy_version 870565 (0.0008) [2023-12-27 11:40:24,814][20347] Updated weights for policy 0, policy_version 870575 (0.0006) [2023-12-27 11:40:25,373][20381] Updated weights for policy 1, policy_version 869853 (0.0008) [2023-12-27 11:40:25,434][20381] Updated weights for policy 1, policy_version 869863 (0.0008) [2023-12-27 11:40:25,487][20347] Updated weights for policy 0, policy_version 870585 (0.0006) [2023-12-27 11:40:25,492][20381] Updated weights for policy 1, policy_version 869873 (0.0006) [2023-12-27 11:40:25,544][20347] Updated weights for policy 0, policy_version 870595 (0.0006) [2023-12-27 11:40:25,604][20347] Updated weights for policy 0, policy_version 870605 (0.0007) [2023-12-27 11:40:26,196][20381] Updated weights for policy 1, policy_version 869883 (0.0007) [2023-12-27 11:40:26,253][20381] Updated weights for policy 1, policy_version 869893 (0.0008) [2023-12-27 11:40:26,296][20347] Updated weights for policy 0, policy_version 870615 (0.0006) [2023-12-27 11:40:26,313][20381] Updated weights for policy 1, policy_version 869903 (0.0007) [2023-12-27 11:40:26,348][20347] Updated weights for policy 0, policy_version 870625 (0.0008) [2023-12-27 11:40:26,397][20347] Updated weights for policy 0, policy_version 870635 (0.0007) [2023-12-27 11:40:27,020][20381] Updated weights for policy 1, policy_version 869913 (0.0006) [2023-12-27 11:40:27,069][20381] Updated weights for policy 1, policy_version 869923 (0.0006) [2023-12-27 11:40:27,083][20347] Updated weights for policy 0, policy_version 870645 (0.0008) [2023-12-27 11:40:27,116][20381] Updated weights for policy 1, policy_version 869933 (0.0005) [2023-12-27 11:40:27,131][20347] Updated weights for policy 0, policy_version 870655 (0.0007) [2023-12-27 11:40:27,167][20381] Updated weights for policy 1, policy_version 869943 (0.0007) [2023-12-27 11:40:27,190][20347] Updated weights for policy 0, policy_version 870665 (0.0007) [2023-12-27 11:40:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.8, 300 sec: 19660.8). Total num frames: 445652992. Throughput: 0: 9581.4, 1: 9609.9. Samples: 445637860. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:27,363][19170] Avg episode reward: [(0, '833.430'), (1, '646.420')] [2023-12-27 11:40:27,856][20347] Updated weights for policy 0, policy_version 870675 (0.0007) [2023-12-27 11:40:27,860][20381] Updated weights for policy 1, policy_version 869953 (0.0006) [2023-12-27 11:40:27,912][20347] Updated weights for policy 0, policy_version 870685 (0.0008) [2023-12-27 11:40:27,922][20381] Updated weights for policy 1, policy_version 869963 (0.0005) [2023-12-27 11:40:27,965][20347] Updated weights for policy 0, policy_version 870695 (0.0009) [2023-12-27 11:40:27,975][20381] Updated weights for policy 1, policy_version 869973 (0.0008) [2023-12-27 11:40:28,650][20381] Updated weights for policy 1, policy_version 869983 (0.0007) [2023-12-27 11:40:28,692][20347] Updated weights for policy 0, policy_version 870705 (0.0007) [2023-12-27 11:40:28,714][20381] Updated weights for policy 1, policy_version 869993 (0.0007) [2023-12-27 11:40:28,755][20347] Updated weights for policy 0, policy_version 870715 (0.0008) [2023-12-27 11:40:28,774][20381] Updated weights for policy 1, policy_version 870003 (0.0006) [2023-12-27 11:40:28,815][20347] Updated weights for policy 0, policy_version 870725 (0.0007) [2023-12-27 11:40:28,866][20347] Updated weights for policy 0, policy_version 870735 (0.0008) [2023-12-27 11:40:29,505][20381] Updated weights for policy 1, policy_version 870013 (0.0007) [2023-12-27 11:40:29,569][20381] Updated weights for policy 1, policy_version 870023 (0.0007) [2023-12-27 11:40:29,616][20347] Updated weights for policy 0, policy_version 870745 (0.0007) [2023-12-27 11:40:29,626][20381] Updated weights for policy 1, policy_version 870033 (0.0006) [2023-12-27 11:40:29,674][20347] Updated weights for policy 0, policy_version 870755 (0.0007) [2023-12-27 11:40:29,734][20347] Updated weights for policy 0, policy_version 870765 (0.0006) [2023-12-27 11:40:30,389][20347] Updated weights for policy 0, policy_version 870775 (0.0007) [2023-12-27 11:40:30,392][20381] Updated weights for policy 1, policy_version 870043 (0.0007) [2023-12-27 11:40:30,449][20347] Updated weights for policy 0, policy_version 870785 (0.0007) [2023-12-27 11:40:30,455][20381] Updated weights for policy 1, policy_version 870053 (0.0008) [2023-12-27 11:40:30,506][20347] Updated weights for policy 0, policy_version 870795 (0.0008) [2023-12-27 11:40:30,514][20381] Updated weights for policy 1, policy_version 870063 (0.0008) [2023-12-27 11:40:31,222][20381] Updated weights for policy 1, policy_version 870073 (0.0007) [2023-12-27 11:40:31,225][20347] Updated weights for policy 0, policy_version 870805 (0.0007) [2023-12-27 11:40:31,283][20381] Updated weights for policy 1, policy_version 870083 (0.0007) [2023-12-27 11:40:31,284][20347] Updated weights for policy 0, policy_version 870815 (0.0008) [2023-12-27 11:40:31,333][20347] Updated weights for policy 0, policy_version 870825 (0.0007) [2023-12-27 11:40:31,335][20381] Updated weights for policy 1, policy_version 870093 (0.0006) [2023-12-27 11:40:31,403][20381] Updated weights for policy 1, policy_version 870103 (0.0008) [2023-12-27 11:40:32,056][20347] Updated weights for policy 0, policy_version 870835 (0.0007) [2023-12-27 11:40:32,098][20381] Updated weights for policy 1, policy_version 870113 (0.0008) [2023-12-27 11:40:32,118][20347] Updated weights for policy 0, policy_version 870845 (0.0006) [2023-12-27 11:40:32,161][20381] Updated weights for policy 1, policy_version 870123 (0.0007) [2023-12-27 11:40:32,182][20347] Updated weights for policy 0, policy_version 870855 (0.0007) [2023-12-27 11:40:32,220][20381] Updated weights for policy 1, policy_version 870133 (0.0006) [2023-12-27 11:40:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19251.2, 300 sec: 19633.1). Total num frames: 445751296. Throughput: 0: 9585.6, 1: 9652.0. Samples: 445756384. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:32,363][19170] Avg episode reward: [(0, '862.610'), (1, '685.530')] [2023-12-27 11:40:32,886][20347] Updated weights for policy 0, policy_version 870865 (0.0007) [2023-12-27 11:40:32,907][20381] Updated weights for policy 1, policy_version 870143 (0.0008) [2023-12-27 11:40:32,942][20347] Updated weights for policy 0, policy_version 870875 (0.0006) [2023-12-27 11:40:32,962][20381] Updated weights for policy 1, policy_version 870153 (0.0007) [2023-12-27 11:40:32,994][20347] Updated weights for policy 0, policy_version 870885 (0.0005) [2023-12-27 11:40:33,015][20381] Updated weights for policy 1, policy_version 870163 (0.0008) [2023-12-27 11:40:33,051][20347] Updated weights for policy 0, policy_version 870895 (0.0006) [2023-12-27 11:40:33,731][20381] Updated weights for policy 1, policy_version 870173 (0.0009) [2023-12-27 11:40:33,766][20347] Updated weights for policy 0, policy_version 870905 (0.0008) [2023-12-27 11:40:33,785][20381] Updated weights for policy 1, policy_version 870183 (0.0011) [2023-12-27 11:40:33,824][20347] Updated weights for policy 0, policy_version 870915 (0.0007) [2023-12-27 11:40:33,845][20381] Updated weights for policy 1, policy_version 870193 (0.0011) [2023-12-27 11:40:33,884][20347] Updated weights for policy 0, policy_version 870925 (0.0008) [2023-12-27 11:40:34,600][20381] Updated weights for policy 1, policy_version 870203 (0.0010) [2023-12-27 11:40:34,603][20347] Updated weights for policy 0, policy_version 870935 (0.0007) [2023-12-27 11:40:34,662][20381] Updated weights for policy 1, policy_version 870213 (0.0008) [2023-12-27 11:40:34,662][20347] Updated weights for policy 0, policy_version 870945 (0.0009) [2023-12-27 11:40:34,722][20347] Updated weights for policy 0, policy_version 870955 (0.0008) [2023-12-27 11:40:34,727][20381] Updated weights for policy 1, policy_version 870223 (0.0007) [2023-12-27 11:40:35,378][20381] Updated weights for policy 1, policy_version 870233 (0.0009) [2023-12-27 11:40:35,436][20381] Updated weights for policy 1, policy_version 870243 (0.0011) [2023-12-27 11:40:35,459][20347] Updated weights for policy 0, policy_version 870965 (0.0008) [2023-12-27 11:40:35,497][20381] Updated weights for policy 1, policy_version 870253 (0.0011) [2023-12-27 11:40:35,521][20347] Updated weights for policy 0, policy_version 870975 (0.0008) [2023-12-27 11:40:35,553][20381] Updated weights for policy 1, policy_version 870263 (0.0006) [2023-12-27 11:40:35,581][20347] Updated weights for policy 0, policy_version 870985 (0.0008) [2023-12-27 11:40:36,131][20381] Updated weights for policy 1, policy_version 870273 (0.0009) [2023-12-27 11:40:36,198][20381] Updated weights for policy 1, policy_version 870283 (0.0006) [2023-12-27 11:40:36,264][20381] Updated weights for policy 1, policy_version 870293 (0.0010) [2023-12-27 11:40:36,309][20347] Updated weights for policy 0, policy_version 870995 (0.0007) [2023-12-27 11:40:36,370][20347] Updated weights for policy 0, policy_version 871005 (0.0007) [2023-12-27 11:40:36,437][20347] Updated weights for policy 0, policy_version 871015 (0.0007) [2023-12-27 11:40:37,007][20381] Updated weights for policy 1, policy_version 870303 (0.0009) [2023-12-27 11:40:37,063][20347] Updated weights for policy 0, policy_version 871025 (0.0007) [2023-12-27 11:40:37,071][20381] Updated weights for policy 1, policy_version 870313 (0.0008) [2023-12-27 11:40:37,123][20347] Updated weights for policy 0, policy_version 871035 (0.0007) [2023-12-27 11:40:37,134][20381] Updated weights for policy 1, policy_version 870323 (0.0008) [2023-12-27 11:40:37,182][20347] Updated weights for policy 0, policy_version 871045 (0.0007) [2023-12-27 11:40:37,233][20347] Updated weights for policy 0, policy_version 871055 (0.0006) [2023-12-27 11:40:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19387.7, 300 sec: 19633.0). Total num frames: 445849600. Throughput: 0: 9564.5, 1: 9672.5. Samples: 445814684. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:37,363][19170] Avg episode reward: [(0, '911.740'), (1, '825.550')] [2023-12-27 11:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000871056_223019008.pth... [2023-12-27 11:40:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000870328_222830592.pth... [2023-12-27 11:40:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000869208_222543872.pth [2023-12-27 11:40:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000869936_222732288.pth [2023-12-27 11:40:37,865][20381] Updated weights for policy 1, policy_version 870333 (0.0007) [2023-12-27 11:40:37,907][20347] Updated weights for policy 0, policy_version 871065 (0.0006) [2023-12-27 11:40:37,916][20381] Updated weights for policy 1, policy_version 870343 (0.0007) [2023-12-27 11:40:37,955][20347] Updated weights for policy 0, policy_version 871075 (0.0007) [2023-12-27 11:40:37,978][20381] Updated weights for policy 1, policy_version 870353 (0.0008) [2023-12-27 11:40:38,012][20347] Updated weights for policy 0, policy_version 871085 (0.0008) [2023-12-27 11:40:38,680][20381] Updated weights for policy 1, policy_version 870363 (0.0007) [2023-12-27 11:40:38,741][20381] Updated weights for policy 1, policy_version 870373 (0.0006) [2023-12-27 11:40:38,749][20347] Updated weights for policy 0, policy_version 871095 (0.0008) [2023-12-27 11:40:38,800][20381] Updated weights for policy 1, policy_version 870383 (0.0006) [2023-12-27 11:40:38,809][20347] Updated weights for policy 0, policy_version 871105 (0.0008) [2023-12-27 11:40:38,870][20347] Updated weights for policy 0, policy_version 871115 (0.0007) [2023-12-27 11:40:39,490][20381] Updated weights for policy 1, policy_version 870393 (0.0007) [2023-12-27 11:40:39,553][20381] Updated weights for policy 1, policy_version 870403 (0.0009) [2023-12-27 11:40:39,614][20347] Updated weights for policy 0, policy_version 871125 (0.0008) [2023-12-27 11:40:39,620][20381] Updated weights for policy 1, policy_version 870413 (0.0007) [2023-12-27 11:40:39,671][20347] Updated weights for policy 0, policy_version 871135 (0.0006) [2023-12-27 11:40:39,689][20381] Updated weights for policy 1, policy_version 870423 (0.0009) [2023-12-27 11:40:39,732][20347] Updated weights for policy 0, policy_version 871145 (0.0008) [2023-12-27 11:40:40,367][20381] Updated weights for policy 1, policy_version 870433 (0.0007) [2023-12-27 11:40:40,433][20381] Updated weights for policy 1, policy_version 870443 (0.0007) [2023-12-27 11:40:40,500][20381] Updated weights for policy 1, policy_version 870453 (0.0007) [2023-12-27 11:40:40,524][20347] Updated weights for policy 0, policy_version 871155 (0.0008) [2023-12-27 11:40:40,583][20347] Updated weights for policy 0, policy_version 871165 (0.0008) [2023-12-27 11:40:40,646][20347] Updated weights for policy 0, policy_version 871175 (0.0008) [2023-12-27 11:40:41,220][20381] Updated weights for policy 1, policy_version 870463 (0.0007) [2023-12-27 11:40:41,287][20381] Updated weights for policy 1, policy_version 870473 (0.0007) [2023-12-27 11:40:41,341][20381] Updated weights for policy 1, policy_version 870483 (0.0008) [2023-12-27 11:40:41,453][20347] Updated weights for policy 0, policy_version 871185 (0.0008) [2023-12-27 11:40:41,519][20347] Updated weights for policy 0, policy_version 871195 (0.0008) [2023-12-27 11:40:41,584][20347] Updated weights for policy 0, policy_version 871205 (0.0008) [2023-12-27 11:40:41,653][20347] Updated weights for policy 0, policy_version 871215 (0.0008) [2023-12-27 11:40:42,156][20381] Updated weights for policy 1, policy_version 870493 (0.0008) [2023-12-27 11:40:42,222][20381] Updated weights for policy 1, policy_version 870503 (0.0008) [2023-12-27 11:40:42,287][20381] Updated weights for policy 1, policy_version 870513 (0.0008) [2023-12-27 11:40:42,348][20347] Updated weights for policy 0, policy_version 871225 (0.0007) [2023-12-27 11:40:42,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19251.2, 300 sec: 19605.3). Total num frames: 445939712. Throughput: 0: 9545.4, 1: 9684.9. Samples: 445931348. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:42,363][19170] Avg episode reward: [(0, '950.960'), (1, '874.120')] [2023-12-27 11:40:42,416][20347] Updated weights for policy 0, policy_version 871235 (0.0008) [2023-12-27 11:40:42,483][20347] Updated weights for policy 0, policy_version 871245 (0.0007) [2023-12-27 11:40:43,009][20381] Updated weights for policy 1, policy_version 870523 (0.0007) [2023-12-27 11:40:43,061][20381] Updated weights for policy 1, policy_version 870533 (0.0007) [2023-12-27 11:40:43,111][20381] Updated weights for policy 1, policy_version 870543 (0.0007) [2023-12-27 11:40:43,234][20347] Updated weights for policy 0, policy_version 871255 (0.0008) [2023-12-27 11:40:43,285][20347] Updated weights for policy 0, policy_version 871265 (0.0008) [2023-12-27 11:40:43,342][20347] Updated weights for policy 0, policy_version 871275 (0.0007) [2023-12-27 11:40:43,849][20381] Updated weights for policy 1, policy_version 870553 (0.0007) [2023-12-27 11:40:43,908][20381] Updated weights for policy 1, policy_version 870563 (0.0007) [2023-12-27 11:40:43,961][20381] Updated weights for policy 1, policy_version 870573 (0.0008) [2023-12-27 11:40:43,983][20347] Updated weights for policy 0, policy_version 871285 (0.0005) [2023-12-27 11:40:44,022][20381] Updated weights for policy 1, policy_version 870583 (0.0008) [2023-12-27 11:40:44,041][20347] Updated weights for policy 0, policy_version 871295 (0.0006) [2023-12-27 11:40:44,104][20347] Updated weights for policy 0, policy_version 871305 (0.0008) [2023-12-27 11:40:44,700][20381] Updated weights for policy 1, policy_version 870593 (0.0008) [2023-12-27 11:40:44,745][20347] Updated weights for policy 0, policy_version 871315 (0.0008) [2023-12-27 11:40:44,762][20381] Updated weights for policy 1, policy_version 870603 (0.0008) [2023-12-27 11:40:44,771][20292] KL-divergence is very high: 340.8169 [2023-12-27 11:40:44,814][20347] Updated weights for policy 0, policy_version 871326 (0.0008) [2023-12-27 11:40:44,816][20292] KL-divergence is very high: 284.6229 [2023-12-27 11:40:44,820][20381] Updated weights for policy 1, policy_version 870613 (0.0006) [2023-12-27 11:40:44,877][20347] Updated weights for policy 0, policy_version 871336 (0.0006) [2023-12-27 11:40:45,544][20381] Updated weights for policy 1, policy_version 870623 (0.0007) [2023-12-27 11:40:45,553][20347] Updated weights for policy 0, policy_version 871346 (0.0008) [2023-12-27 11:40:45,602][20381] Updated weights for policy 1, policy_version 870633 (0.0006) [2023-12-27 11:40:45,616][20347] Updated weights for policy 0, policy_version 871356 (0.0007) [2023-12-27 11:40:45,654][20381] Updated weights for policy 1, policy_version 870643 (0.0006) [2023-12-27 11:40:45,676][20347] Updated weights for policy 0, policy_version 871366 (0.0006) [2023-12-27 11:40:45,732][20347] Updated weights for policy 0, policy_version 871376 (0.0006) [2023-12-27 11:40:46,320][20381] Updated weights for policy 1, policy_version 870653 (0.0007) [2023-12-27 11:40:46,333][20347] Updated weights for policy 0, policy_version 871386 (0.0008) [2023-12-27 11:40:46,377][20381] Updated weights for policy 1, policy_version 870663 (0.0007) [2023-12-27 11:40:46,383][20347] Updated weights for policy 0, policy_version 871396 (0.0006) [2023-12-27 11:40:46,434][20381] Updated weights for policy 1, policy_version 870673 (0.0007) [2023-12-27 11:40:46,436][20347] Updated weights for policy 0, policy_version 871406 (0.0006) [2023-12-27 11:40:47,066][20347] Updated weights for policy 0, policy_version 871416 (0.0006) [2023-12-27 11:40:47,121][20347] Updated weights for policy 0, policy_version 871426 (0.0006) [2023-12-27 11:40:47,170][20381] Updated weights for policy 1, policy_version 870683 (0.0008) [2023-12-27 11:40:47,184][20347] Updated weights for policy 0, policy_version 871436 (0.0007) [2023-12-27 11:40:47,229][20381] Updated weights for policy 1, policy_version 870694 (0.0009) [2023-12-27 11:40:47,293][20381] Updated weights for policy 1, policy_version 870704 (0.0007) [2023-12-27 11:40:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19387.8, 300 sec: 19633.0). Total num frames: 446046208. Throughput: 0: 9678.7, 1: 9671.8. Samples: 446049872. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:47,363][19170] Avg episode reward: [(0, '882.680'), (1, '873.550')] [2023-12-27 11:40:47,871][20347] Updated weights for policy 0, policy_version 871446 (0.0006) [2023-12-27 11:40:47,929][20347] Updated weights for policy 0, policy_version 871456 (0.0006) [2023-12-27 11:40:47,933][20381] Updated weights for policy 1, policy_version 870714 (0.0006) [2023-12-27 11:40:47,989][20347] Updated weights for policy 0, policy_version 871466 (0.0008) [2023-12-27 11:40:47,994][20381] Updated weights for policy 1, policy_version 870724 (0.0006) [2023-12-27 11:40:48,053][20381] Updated weights for policy 1, policy_version 870734 (0.0006) [2023-12-27 11:40:48,114][20381] Updated weights for policy 1, policy_version 870744 (0.0006) [2023-12-27 11:40:48,670][20347] Updated weights for policy 0, policy_version 871476 (0.0008) [2023-12-27 11:40:48,728][20347] Updated weights for policy 0, policy_version 871486 (0.0007) [2023-12-27 11:40:48,771][20381] Updated weights for policy 1, policy_version 870754 (0.0011) [2023-12-27 11:40:48,783][20347] Updated weights for policy 0, policy_version 871496 (0.0006) [2023-12-27 11:40:48,824][20381] Updated weights for policy 1, policy_version 870764 (0.0011) [2023-12-27 11:40:48,883][20381] Updated weights for policy 1, policy_version 870774 (0.0011) [2023-12-27 11:40:49,478][20347] Updated weights for policy 0, policy_version 871506 (0.0006) [2023-12-27 11:40:49,542][20347] Updated weights for policy 0, policy_version 871516 (0.0007) [2023-12-27 11:40:49,547][20381] Updated weights for policy 1, policy_version 870784 (0.0007) [2023-12-27 11:40:49,599][20347] Updated weights for policy 0, policy_version 871526 (0.0006) [2023-12-27 11:40:49,605][20381] Updated weights for policy 1, policy_version 870794 (0.0006) [2023-12-27 11:40:49,661][20347] Updated weights for policy 0, policy_version 871536 (0.0007) [2023-12-27 11:40:49,668][20381] Updated weights for policy 1, policy_version 870804 (0.0007) [2023-12-27 11:40:50,388][20347] Updated weights for policy 0, policy_version 871546 (0.0007) [2023-12-27 11:40:50,410][20381] Updated weights for policy 1, policy_version 870814 (0.0008) [2023-12-27 11:40:50,452][20347] Updated weights for policy 0, policy_version 871556 (0.0006) [2023-12-27 11:40:50,469][20381] Updated weights for policy 1, policy_version 870824 (0.0006) [2023-12-27 11:40:50,511][20347] Updated weights for policy 0, policy_version 871566 (0.0007) [2023-12-27 11:40:50,521][20381] Updated weights for policy 1, policy_version 870834 (0.0006) [2023-12-27 11:40:51,252][20381] Updated weights for policy 1, policy_version 870844 (0.0008) [2023-12-27 11:40:51,258][20347] Updated weights for policy 0, policy_version 871576 (0.0007) [2023-12-27 11:40:51,313][20347] Updated weights for policy 0, policy_version 871586 (0.0007) [2023-12-27 11:40:51,315][20381] Updated weights for policy 1, policy_version 870854 (0.0008) [2023-12-27 11:40:51,373][20381] Updated weights for policy 1, policy_version 870864 (0.0008) [2023-12-27 11:40:51,376][20347] Updated weights for policy 0, policy_version 871596 (0.0007) [2023-12-27 11:40:52,156][20347] Updated weights for policy 0, policy_version 871606 (0.0008) [2023-12-27 11:40:52,181][20381] Updated weights for policy 1, policy_version 870874 (0.0008) [2023-12-27 11:40:52,209][20347] Updated weights for policy 0, policy_version 871616 (0.0007) [2023-12-27 11:40:52,236][20381] Updated weights for policy 1, policy_version 870884 (0.0008) [2023-12-27 11:40:52,270][20347] Updated weights for policy 0, policy_version 871626 (0.0007) [2023-12-27 11:40:52,296][20381] Updated weights for policy 1, policy_version 870894 (0.0008) [2023-12-27 11:40:52,359][20381] Updated weights for policy 1, policy_version 870904 (0.0008) [2023-12-27 11:40:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 19524.2, 300 sec: 19605.2). Total num frames: 446144512. Throughput: 0: 9672.9, 1: 9700.6. Samples: 446109864. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:52,363][19170] Avg episode reward: [(0, '882.820'), (1, '931.840')] [2023-12-27 11:40:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000870904_222978048.pth... [2023-12-27 11:40:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000871632_223166464.pth... [2023-12-27 11:40:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000869752_222683136.pth [2023-12-27 11:40:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000870480_222871552.pth [2023-12-27 11:40:53,030][20347] Updated weights for policy 0, policy_version 871636 (0.0007) [2023-12-27 11:40:53,083][20347] Updated weights for policy 0, policy_version 871646 (0.0008) [2023-12-27 11:40:53,104][20381] Updated weights for policy 1, policy_version 870914 (0.0007) [2023-12-27 11:40:53,136][20347] Updated weights for policy 0, policy_version 871656 (0.0008) [2023-12-27 11:40:53,163][20381] Updated weights for policy 1, policy_version 870924 (0.0008) [2023-12-27 11:40:53,219][20381] Updated weights for policy 1, policy_version 870934 (0.0007) [2023-12-27 11:40:53,839][20347] Updated weights for policy 0, policy_version 871666 (0.0008) [2023-12-27 11:40:53,899][20347] Updated weights for policy 0, policy_version 871676 (0.0008) [2023-12-27 11:40:53,935][20381] Updated weights for policy 1, policy_version 870944 (0.0007) [2023-12-27 11:40:53,952][20347] Updated weights for policy 0, policy_version 871686 (0.0006) [2023-12-27 11:40:53,998][20381] Updated weights for policy 1, policy_version 870954 (0.0006) [2023-12-27 11:40:54,011][20347] Updated weights for policy 0, policy_version 871696 (0.0008) [2023-12-27 11:40:54,052][20381] Updated weights for policy 1, policy_version 870964 (0.0006) [2023-12-27 11:40:54,704][20381] Updated weights for policy 1, policy_version 870974 (0.0006) [2023-12-27 11:40:54,736][20347] Updated weights for policy 0, policy_version 871706 (0.0007) [2023-12-27 11:40:54,762][20381] Updated weights for policy 1, policy_version 870984 (0.0007) [2023-12-27 11:40:54,796][20347] Updated weights for policy 0, policy_version 871716 (0.0006) [2023-12-27 11:40:54,825][20381] Updated weights for policy 1, policy_version 870994 (0.0008) [2023-12-27 11:40:54,850][20347] Updated weights for policy 0, policy_version 871726 (0.0007) [2023-12-27 11:40:55,514][20381] Updated weights for policy 1, policy_version 871004 (0.0006) [2023-12-27 11:40:55,535][20347] Updated weights for policy 0, policy_version 871736 (0.0006) [2023-12-27 11:40:55,576][20381] Updated weights for policy 1, policy_version 871014 (0.0007) [2023-12-27 11:40:55,594][20347] Updated weights for policy 0, policy_version 871746 (0.0006) [2023-12-27 11:40:55,639][20381] Updated weights for policy 1, policy_version 871024 (0.0008) [2023-12-27 11:40:55,649][20347] Updated weights for policy 0, policy_version 871756 (0.0007) [2023-12-27 11:40:56,370][20347] Updated weights for policy 0, policy_version 871766 (0.0007) [2023-12-27 11:40:56,371][20381] Updated weights for policy 1, policy_version 871034 (0.0008) [2023-12-27 11:40:56,430][20347] Updated weights for policy 0, policy_version 871776 (0.0006) [2023-12-27 11:40:56,432][20381] Updated weights for policy 1, policy_version 871044 (0.0011) [2023-12-27 11:40:56,487][20381] Updated weights for policy 1, policy_version 871054 (0.0011) [2023-12-27 11:40:56,488][20347] Updated weights for policy 0, policy_version 871786 (0.0006) [2023-12-27 11:40:56,546][20381] Updated weights for policy 1, policy_version 871064 (0.0011) [2023-12-27 11:40:57,123][20347] Updated weights for policy 0, policy_version 871796 (0.0006) [2023-12-27 11:40:57,176][20347] Updated weights for policy 0, policy_version 871806 (0.0005) [2023-12-27 11:40:57,234][20347] Updated weights for policy 0, policy_version 871816 (0.0006) [2023-12-27 11:40:57,258][20381] Updated weights for policy 1, policy_version 871074 (0.0007) [2023-12-27 11:40:57,319][20381] Updated weights for policy 1, policy_version 871084 (0.0009) [2023-12-27 11:40:57,362][19170] Fps is (10 sec: 18841.4, 60 sec: 19251.2, 300 sec: 19577.5). Total num frames: 446234624. Throughput: 0: 9711.8, 1: 9722.3. Samples: 446225448. Policy #0 lag: (min: 25.0, avg: 40.7, max: 57.0) [2023-12-27 11:40:57,363][19170] Avg episode reward: [(0, '882.740'), (1, '892.890')] [2023-12-27 11:40:57,379][20381] Updated weights for policy 1, policy_version 871094 (0.0011) [2023-12-27 11:40:57,914][20347] Updated weights for policy 0, policy_version 871826 (0.0008) [2023-12-27 11:40:57,973][20347] Updated weights for policy 0, policy_version 871836 (0.0008) [2023-12-27 11:40:58,028][20347] Updated weights for policy 0, policy_version 871846 (0.0007) [2023-12-27 11:40:58,071][20381] Updated weights for policy 1, policy_version 871104 (0.0011) [2023-12-27 11:40:58,080][20347] Updated weights for policy 0, policy_version 871856 (0.0006) [2023-12-27 11:40:58,129][20381] Updated weights for policy 1, policy_version 871114 (0.0011) [2023-12-27 11:40:58,191][20381] Updated weights for policy 1, policy_version 871124 (0.0011) [2023-12-27 11:40:58,816][20347] Updated weights for policy 0, policy_version 871866 (0.0008) [2023-12-27 11:40:58,887][20347] Updated weights for policy 0, policy_version 871876 (0.0007) [2023-12-27 11:40:58,953][20347] Updated weights for policy 0, policy_version 871886 (0.0007) [2023-12-27 11:40:58,987][20381] Updated weights for policy 1, policy_version 871134 (0.0012) [2023-12-27 11:40:59,054][20381] Updated weights for policy 1, policy_version 871144 (0.0010) [2023-12-27 11:40:59,115][20381] Updated weights for policy 1, policy_version 871154 (0.0009) [2023-12-27 11:40:59,720][20347] Updated weights for policy 0, policy_version 871896 (0.0007) [2023-12-27 11:40:59,780][20347] Updated weights for policy 0, policy_version 871906 (0.0008) [2023-12-27 11:40:59,838][20381] Updated weights for policy 1, policy_version 871164 (0.0008) [2023-12-27 11:40:59,840][20347] Updated weights for policy 0, policy_version 871916 (0.0008) [2023-12-27 11:40:59,897][20381] Updated weights for policy 1, policy_version 871174 (0.0007) [2023-12-27 11:40:59,960][20381] Updated weights for policy 1, policy_version 871184 (0.0008) [2023-12-27 11:41:00,564][20347] Updated weights for policy 0, policy_version 871926 (0.0006) [2023-12-27 11:41:00,618][20347] Updated weights for policy 0, policy_version 871936 (0.0005) [2023-12-27 11:41:00,667][20347] Updated weights for policy 0, policy_version 871946 (0.0007) [2023-12-27 11:41:00,701][20381] Updated weights for policy 1, policy_version 871194 (0.0009) [2023-12-27 11:41:00,761][20381] Updated weights for policy 1, policy_version 871204 (0.0011) [2023-12-27 11:41:00,821][20381] Updated weights for policy 1, policy_version 871214 (0.0011) [2023-12-27 11:41:00,873][20381] Updated weights for policy 1, policy_version 871224 (0.0010) [2023-12-27 11:41:01,321][20347] Updated weights for policy 0, policy_version 871956 (0.0007) [2023-12-27 11:41:01,386][20347] Updated weights for policy 0, policy_version 871966 (0.0008) [2023-12-27 11:41:01,438][20347] Updated weights for policy 0, policy_version 871976 (0.0008) [2023-12-27 11:41:01,580][20381] Updated weights for policy 1, policy_version 871234 (0.0008) [2023-12-27 11:41:01,646][20381] Updated weights for policy 1, policy_version 871244 (0.0008) [2023-12-27 11:41:01,707][20381] Updated weights for policy 1, policy_version 871254 (0.0007) [2023-12-27 11:41:02,139][20347] Updated weights for policy 0, policy_version 871986 (0.0008) [2023-12-27 11:41:02,202][20347] Updated weights for policy 0, policy_version 871996 (0.0008) [2023-12-27 11:41:02,265][20347] Updated weights for policy 0, policy_version 872006 (0.0007) [2023-12-27 11:41:02,324][20347] Updated weights for policy 0, policy_version 872016 (0.0007) [2023-12-27 11:41:02,347][20381] Updated weights for policy 1, policy_version 871264 (0.0008) [2023-12-27 11:41:02,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19524.3, 300 sec: 19577.5). Total num frames: 446332928. Throughput: 0: 9722.5, 1: 9718.7. Samples: 446342160. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:02,363][19170] Avg episode reward: [(0, '853.110'), (1, '922.110')] [2023-12-27 11:41:02,409][20381] Updated weights for policy 1, policy_version 871274 (0.0006) [2023-12-27 11:41:02,468][20381] Updated weights for policy 1, policy_version 871284 (0.0006) [2023-12-27 11:41:03,045][20347] Updated weights for policy 0, policy_version 872026 (0.0005) [2023-12-27 11:41:03,102][20347] Updated weights for policy 0, policy_version 872036 (0.0006) [2023-12-27 11:41:03,104][20381] Updated weights for policy 1, policy_version 871294 (0.0008) [2023-12-27 11:41:03,159][20381] Updated weights for policy 1, policy_version 871304 (0.0006) [2023-12-27 11:41:03,161][20347] Updated weights for policy 0, policy_version 872046 (0.0006) [2023-12-27 11:41:03,220][20381] Updated weights for policy 1, policy_version 871314 (0.0007) [2023-12-27 11:41:03,829][20347] Updated weights for policy 0, policy_version 872056 (0.0007) [2023-12-27 11:41:03,885][20347] Updated weights for policy 0, policy_version 872066 (0.0008) [2023-12-27 11:41:03,918][20381] Updated weights for policy 1, policy_version 871324 (0.0007) [2023-12-27 11:41:03,934][20347] Updated weights for policy 0, policy_version 872076 (0.0007) [2023-12-27 11:41:03,985][20381] Updated weights for policy 1, policy_version 871334 (0.0006) [2023-12-27 11:41:04,053][20381] Updated weights for policy 1, policy_version 871344 (0.0007) [2023-12-27 11:41:04,643][20347] Updated weights for policy 0, policy_version 872086 (0.0007) [2023-12-27 11:41:04,705][20347] Updated weights for policy 0, policy_version 872096 (0.0007) [2023-12-27 11:41:04,770][20347] Updated weights for policy 0, policy_version 872106 (0.0008) [2023-12-27 11:41:04,807][20381] Updated weights for policy 1, policy_version 871354 (0.0008) [2023-12-27 11:41:04,870][20381] Updated weights for policy 1, policy_version 871364 (0.0008) [2023-12-27 11:41:04,930][20381] Updated weights for policy 1, policy_version 871374 (0.0008) [2023-12-27 11:41:04,992][20381] Updated weights for policy 1, policy_version 871384 (0.0008) [2023-12-27 11:41:05,495][20347] Updated weights for policy 0, policy_version 872116 (0.0006) [2023-12-27 11:41:05,552][20347] Updated weights for policy 0, policy_version 872126 (0.0008) [2023-12-27 11:41:05,606][20347] Updated weights for policy 0, policy_version 872136 (0.0007) [2023-12-27 11:41:05,619][20381] Updated weights for policy 1, policy_version 871394 (0.0006) [2023-12-27 11:41:05,677][20381] Updated weights for policy 1, policy_version 871404 (0.0006) [2023-12-27 11:41:05,737][20381] Updated weights for policy 1, policy_version 871414 (0.0007) [2023-12-27 11:41:06,384][20347] Updated weights for policy 0, policy_version 872146 (0.0006) [2023-12-27 11:41:06,444][20347] Updated weights for policy 0, policy_version 872156 (0.0008) [2023-12-27 11:41:06,449][20381] Updated weights for policy 1, policy_version 871424 (0.0007) [2023-12-27 11:41:06,505][20347] Updated weights for policy 0, policy_version 872166 (0.0008) [2023-12-27 11:41:06,506][20381] Updated weights for policy 1, policy_version 871434 (0.0006) [2023-12-27 11:41:06,558][20347] Updated weights for policy 0, policy_version 872176 (0.0008) [2023-12-27 11:41:06,563][20381] Updated weights for policy 1, policy_version 871444 (0.0008) [2023-12-27 11:41:07,310][20347] Updated weights for policy 0, policy_version 872186 (0.0007) [2023-12-27 11:41:07,311][20381] Updated weights for policy 1, policy_version 871454 (0.0007) [2023-12-27 11:41:07,362][19170] Fps is (10 sec: 18841.9, 60 sec: 19387.8, 300 sec: 19549.7). Total num frames: 446423040. Throughput: 0: 9715.7, 1: 9750.9. Samples: 446401392. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:07,363][19170] Avg episode reward: [(0, '852.810'), (1, '961.150')] [2023-12-27 11:41:07,363][20347] Updated weights for policy 0, policy_version 872196 (0.0006) [2023-12-27 11:41:07,369][20381] Updated weights for policy 1, policy_version 871464 (0.0007) [2023-12-27 11:41:07,420][20347] Updated weights for policy 0, policy_version 872206 (0.0006) [2023-12-27 11:41:07,422][20381] Updated weights for policy 1, policy_version 871474 (0.0007) [2023-12-27 11:41:07,432][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000872208_223313920.pth... [2023-12-27 11:41:07,437][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000871056_223019008.pth [2023-12-27 11:41:07,458][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000871480_223125504.pth... [2023-12-27 11:41:07,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000870328_222830592.pth [2023-12-27 11:41:08,132][20347] Updated weights for policy 0, policy_version 872216 (0.0008) [2023-12-27 11:41:08,172][20381] Updated weights for policy 1, policy_version 871484 (0.0008) [2023-12-27 11:41:08,191][20347] Updated weights for policy 0, policy_version 872226 (0.0006) [2023-12-27 11:41:08,229][20381] Updated weights for policy 1, policy_version 871494 (0.0006) [2023-12-27 11:41:08,247][20347] Updated weights for policy 0, policy_version 872236 (0.0006) [2023-12-27 11:41:08,282][20381] Updated weights for policy 1, policy_version 871504 (0.0007) [2023-12-27 11:41:08,977][20347] Updated weights for policy 0, policy_version 872246 (0.0007) [2023-12-27 11:41:09,006][20381] Updated weights for policy 1, policy_version 871514 (0.0008) [2023-12-27 11:41:09,038][20347] Updated weights for policy 0, policy_version 872256 (0.0007) [2023-12-27 11:41:09,072][20381] Updated weights for policy 1, policy_version 871524 (0.0006) [2023-12-27 11:41:09,098][20347] Updated weights for policy 0, policy_version 872266 (0.0007) [2023-12-27 11:41:09,136][20381] Updated weights for policy 1, policy_version 871534 (0.0007) [2023-12-27 11:41:09,189][20381] Updated weights for policy 1, policy_version 871544 (0.0005) [2023-12-27 11:41:09,838][20381] Updated weights for policy 1, policy_version 871554 (0.0008) [2023-12-27 11:41:09,852][20347] Updated weights for policy 0, policy_version 872276 (0.0008) [2023-12-27 11:41:09,908][20381] Updated weights for policy 1, policy_version 871564 (0.0007) [2023-12-27 11:41:09,921][20347] Updated weights for policy 0, policy_version 872286 (0.0008) [2023-12-27 11:41:09,975][20381] Updated weights for policy 1, policy_version 871574 (0.0009) [2023-12-27 11:41:09,982][20347] Updated weights for policy 0, policy_version 872296 (0.0008) [2023-12-27 11:41:10,664][20381] Updated weights for policy 1, policy_version 871584 (0.0006) [2023-12-27 11:41:10,713][20381] Updated weights for policy 1, policy_version 871594 (0.0005) [2023-12-27 11:41:10,726][20347] Updated weights for policy 0, policy_version 872306 (0.0007) [2023-12-27 11:41:10,770][20381] Updated weights for policy 1, policy_version 871604 (0.0005) [2023-12-27 11:41:10,779][20347] Updated weights for policy 0, policy_version 872316 (0.0007) [2023-12-27 11:41:10,827][20347] Updated weights for policy 0, policy_version 872326 (0.0008) [2023-12-27 11:41:10,884][20347] Updated weights for policy 0, policy_version 872336 (0.0010) [2023-12-27 11:41:11,450][20381] Updated weights for policy 1, policy_version 871614 (0.0008) [2023-12-27 11:41:11,510][20381] Updated weights for policy 1, policy_version 871624 (0.0009) [2023-12-27 11:41:11,575][20381] Updated weights for policy 1, policy_version 871634 (0.0009) [2023-12-27 11:41:11,723][20347] Updated weights for policy 0, policy_version 872346 (0.0007) [2023-12-27 11:41:11,791][20347] Updated weights for policy 0, policy_version 872356 (0.0007) [2023-12-27 11:41:11,859][20347] Updated weights for policy 0, policy_version 872366 (0.0008) [2023-12-27 11:41:12,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 446521344. Throughput: 0: 9743.1, 1: 9792.2. Samples: 446516948. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:12,363][19170] Avg episode reward: [(0, '872.580'), (1, '951.420')] [2023-12-27 11:41:12,395][20381] Updated weights for policy 1, policy_version 871644 (0.0008) [2023-12-27 11:41:12,465][20381] Updated weights for policy 1, policy_version 871654 (0.0009) [2023-12-27 11:41:12,530][20381] Updated weights for policy 1, policy_version 871664 (0.0008) [2023-12-27 11:41:12,554][20347] Updated weights for policy 0, policy_version 872376 (0.0007) [2023-12-27 11:41:12,609][20347] Updated weights for policy 0, policy_version 872386 (0.0006) [2023-12-27 11:41:12,676][20347] Updated weights for policy 0, policy_version 872396 (0.0008) [2023-12-27 11:41:13,289][20381] Updated weights for policy 1, policy_version 871674 (0.0007) [2023-12-27 11:41:13,353][20381] Updated weights for policy 1, policy_version 871684 (0.0006) [2023-12-27 11:41:13,365][20347] Updated weights for policy 0, policy_version 872406 (0.0006) [2023-12-27 11:41:13,405][20381] Updated weights for policy 1, policy_version 871694 (0.0006) [2023-12-27 11:41:13,425][20347] Updated weights for policy 0, policy_version 872416 (0.0006) [2023-12-27 11:41:13,464][20381] Updated weights for policy 1, policy_version 871704 (0.0006) [2023-12-27 11:41:13,482][20347] Updated weights for policy 0, policy_version 872426 (0.0006) [2023-12-27 11:41:14,041][20381] Updated weights for policy 1, policy_version 871714 (0.0007) [2023-12-27 11:41:14,094][20381] Updated weights for policy 1, policy_version 871724 (0.0008) [2023-12-27 11:41:14,138][20347] Updated weights for policy 0, policy_version 872436 (0.0006) [2023-12-27 11:41:14,145][20381] Updated weights for policy 1, policy_version 871734 (0.0007) [2023-12-27 11:41:14,202][20347] Updated weights for policy 0, policy_version 872446 (0.0007) [2023-12-27 11:41:14,255][20347] Updated weights for policy 0, policy_version 872456 (0.0006) [2023-12-27 11:41:14,821][20381] Updated weights for policy 1, policy_version 871744 (0.0008) [2023-12-27 11:41:14,869][20381] Updated weights for policy 1, policy_version 871754 (0.0007) [2023-12-27 11:41:14,928][20381] Updated weights for policy 1, policy_version 871764 (0.0008) [2023-12-27 11:41:15,018][20347] Updated weights for policy 0, policy_version 872466 (0.0008) [2023-12-27 11:41:15,082][20347] Updated weights for policy 0, policy_version 872476 (0.0006) [2023-12-27 11:41:15,146][20347] Updated weights for policy 0, policy_version 872486 (0.0008) [2023-12-27 11:41:15,213][20347] Updated weights for policy 0, policy_version 872496 (0.0006) [2023-12-27 11:41:15,726][20381] Updated weights for policy 1, policy_version 871774 (0.0009) [2023-12-27 11:41:15,785][20381] Updated weights for policy 1, policy_version 871784 (0.0007) [2023-12-27 11:41:15,838][20347] Updated weights for policy 0, policy_version 872506 (0.0005) [2023-12-27 11:41:15,842][20381] Updated weights for policy 1, policy_version 871794 (0.0008) [2023-12-27 11:41:15,896][20347] Updated weights for policy 0, policy_version 872516 (0.0006) [2023-12-27 11:41:15,954][20291] KL-divergence is very high: 150.4423 [2023-12-27 11:41:15,954][20347] Updated weights for policy 0, policy_version 872526 (0.0007) [2023-12-27 11:41:15,961][20291] KL-divergence is very high: 157.0100 [2023-12-27 11:41:16,479][20381] Updated weights for policy 1, policy_version 871804 (0.0007) [2023-12-27 11:41:16,538][20381] Updated weights for policy 1, policy_version 871814 (0.0006) [2023-12-27 11:41:16,602][20381] Updated weights for policy 1, policy_version 871824 (0.0008) [2023-12-27 11:41:16,633][20291] KL-divergence is very high: 169.5243 [2023-12-27 11:41:16,654][20347] Updated weights for policy 0, policy_version 872536 (0.0007) [2023-12-27 11:41:16,684][20291] KL-divergence is very high: 133.2075 [2023-12-27 11:41:16,711][20347] Updated weights for policy 0, policy_version 872546 (0.0008) [2023-12-27 11:41:16,725][20291] KL-divergence is very high: 109.7134 [2023-12-27 11:41:16,761][20347] Updated weights for policy 0, policy_version 872556 (0.0008) [2023-12-27 11:41:16,768][20291] KL-divergence is very high: 109.6724 [2023-12-27 11:41:17,265][20381] Updated weights for policy 1, policy_version 871834 (0.0008) [2023-12-27 11:41:17,315][20381] Updated weights for policy 1, policy_version 871844 (0.0008) [2023-12-27 11:41:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 446619648. Throughput: 0: 9726.7, 1: 9782.7. Samples: 446634312. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:17,363][19170] Avg episode reward: [(0, '823.470'), (1, '951.430')] [2023-12-27 11:41:17,370][20381] Updated weights for policy 1, policy_version 871854 (0.0008) [2023-12-27 11:41:17,431][20381] Updated weights for policy 1, policy_version 871864 (0.0006) [2023-12-27 11:41:17,452][20347] Updated weights for policy 0, policy_version 872566 (0.0007) [2023-12-27 11:41:17,511][20347] Updated weights for policy 0, policy_version 872576 (0.0008) [2023-12-27 11:41:17,567][20347] Updated weights for policy 0, policy_version 872586 (0.0007) [2023-12-27 11:41:18,104][20381] Updated weights for policy 1, policy_version 871874 (0.0008) [2023-12-27 11:41:18,164][20381] Updated weights for policy 1, policy_version 871884 (0.0008) [2023-12-27 11:41:18,226][20381] Updated weights for policy 1, policy_version 871894 (0.0008) [2023-12-27 11:41:18,243][20347] Updated weights for policy 0, policy_version 872596 (0.0007) [2023-12-27 11:41:18,304][20347] Updated weights for policy 0, policy_version 872606 (0.0008) [2023-12-27 11:41:18,369][20347] Updated weights for policy 0, policy_version 872616 (0.0008) [2023-12-27 11:41:18,964][20381] Updated weights for policy 1, policy_version 871904 (0.0007) [2023-12-27 11:41:19,014][20381] Updated weights for policy 1, policy_version 871914 (0.0008) [2023-12-27 11:41:19,065][20381] Updated weights for policy 1, policy_version 871924 (0.0008) [2023-12-27 11:41:19,072][20347] Updated weights for policy 0, policy_version 872626 (0.0008) [2023-12-27 11:41:19,123][20347] Updated weights for policy 0, policy_version 872636 (0.0008) [2023-12-27 11:41:19,167][20347] Updated weights for policy 0, policy_version 872646 (0.0008) [2023-12-27 11:41:19,221][20347] Updated weights for policy 0, policy_version 872656 (0.0007) [2023-12-27 11:41:19,868][20381] Updated weights for policy 1, policy_version 871934 (0.0007) [2023-12-27 11:41:19,936][20381] Updated weights for policy 1, policy_version 871944 (0.0008) [2023-12-27 11:41:19,985][20347] Updated weights for policy 0, policy_version 872666 (0.0007) [2023-12-27 11:41:19,997][20381] Updated weights for policy 1, policy_version 871954 (0.0008) [2023-12-27 11:41:20,046][20347] Updated weights for policy 0, policy_version 872676 (0.0007) [2023-12-27 11:41:20,111][20347] Updated weights for policy 0, policy_version 872686 (0.0007) [2023-12-27 11:41:20,750][20381] Updated weights for policy 1, policy_version 871964 (0.0008) [2023-12-27 11:41:20,807][20381] Updated weights for policy 1, policy_version 871974 (0.0008) [2023-12-27 11:41:20,840][20347] Updated weights for policy 0, policy_version 872696 (0.0007) [2023-12-27 11:41:20,867][20381] Updated weights for policy 1, policy_version 871984 (0.0008) [2023-12-27 11:41:20,895][20347] Updated weights for policy 0, policy_version 872706 (0.0006) [2023-12-27 11:41:20,953][20347] Updated weights for policy 0, policy_version 872716 (0.0008) [2023-12-27 11:41:21,656][20347] Updated weights for policy 0, policy_version 872726 (0.0008) [2023-12-27 11:41:21,662][20381] Updated weights for policy 1, policy_version 871994 (0.0008) [2023-12-27 11:41:21,716][20347] Updated weights for policy 0, policy_version 872736 (0.0008) [2023-12-27 11:41:21,736][20381] Updated weights for policy 1, policy_version 872004 (0.0006) [2023-12-27 11:41:21,783][20347] Updated weights for policy 0, policy_version 872746 (0.0008) [2023-12-27 11:41:21,801][20381] Updated weights for policy 1, policy_version 872014 (0.0007) [2023-12-27 11:41:21,867][20381] Updated weights for policy 1, policy_version 872024 (0.0008) [2023-12-27 11:41:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19387.7, 300 sec: 19494.2). Total num frames: 446717952. Throughput: 0: 9740.4, 1: 9760.6. Samples: 446692232. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:22,363][19170] Avg episode reward: [(0, '862.550'), (1, '941.620')] [2023-12-27 11:41:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000872752_223453184.pth... [2023-12-27 11:41:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000872024_223264768.pth... [2023-12-27 11:41:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000870904_222978048.pth [2023-12-27 11:41:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000871632_223166464.pth [2023-12-27 11:41:22,547][20381] Updated weights for policy 1, policy_version 872034 (0.0007) [2023-12-27 11:41:22,621][20381] Updated weights for policy 1, policy_version 872044 (0.0008) [2023-12-27 11:41:22,638][20347] Updated weights for policy 0, policy_version 872756 (0.0008) [2023-12-27 11:41:22,685][20381] Updated weights for policy 1, policy_version 872054 (0.0008) [2023-12-27 11:41:22,698][20347] Updated weights for policy 0, policy_version 872766 (0.0008) [2023-12-27 11:41:22,764][20347] Updated weights for policy 0, policy_version 872776 (0.0008) [2023-12-27 11:41:22,810][20291] KL-divergence is very high: 147.9375 [2023-12-27 11:41:23,428][20381] Updated weights for policy 1, policy_version 872064 (0.0008) [2023-12-27 11:41:23,478][20381] Updated weights for policy 1, policy_version 872074 (0.0007) [2023-12-27 11:41:23,501][20291] KL-divergence is very high: 138.7866 [2023-12-27 11:41:23,503][20347] Updated weights for policy 0, policy_version 872786 (0.0008) [2023-12-27 11:41:23,532][20381] Updated weights for policy 1, policy_version 872084 (0.0006) [2023-12-27 11:41:23,555][20347] Updated weights for policy 0, policy_version 872796 (0.0006) [2023-12-27 11:41:23,621][20347] Updated weights for policy 0, policy_version 872806 (0.0009) [2023-12-27 11:41:23,678][20347] Updated weights for policy 0, policy_version 872816 (0.0007) [2023-12-27 11:41:24,275][20381] Updated weights for policy 1, policy_version 872094 (0.0007) [2023-12-27 11:41:24,337][20381] Updated weights for policy 1, policy_version 872104 (0.0008) [2023-12-27 11:41:24,393][20381] Updated weights for policy 1, policy_version 872114 (0.0006) [2023-12-27 11:41:24,395][20347] Updated weights for policy 0, policy_version 872826 (0.0007) [2023-12-27 11:41:24,457][20347] Updated weights for policy 0, policy_version 872836 (0.0008) [2023-12-27 11:41:24,518][20347] Updated weights for policy 0, policy_version 872846 (0.0007) [2023-12-27 11:41:24,981][20381] Updated weights for policy 1, policy_version 872124 (0.0006) [2023-12-27 11:41:25,036][20381] Updated weights for policy 1, policy_version 872134 (0.0006) [2023-12-27 11:41:25,094][20381] Updated weights for policy 1, policy_version 872144 (0.0007) [2023-12-27 11:41:25,200][20347] Updated weights for policy 0, policy_version 872856 (0.0008) [2023-12-27 11:41:25,255][20347] Updated weights for policy 0, policy_version 872866 (0.0009) [2023-12-27 11:41:25,292][20291] KL-divergence is very high: 130.7212 [2023-12-27 11:41:25,318][20347] Updated weights for policy 0, policy_version 872876 (0.0008) [2023-12-27 11:41:25,732][20381] Updated weights for policy 1, policy_version 872154 (0.0008) [2023-12-27 11:41:25,798][20381] Updated weights for policy 1, policy_version 872164 (0.0006) [2023-12-27 11:41:25,863][20381] Updated weights for policy 1, policy_version 872174 (0.0006) [2023-12-27 11:41:25,929][20381] Updated weights for policy 1, policy_version 872184 (0.0006) [2023-12-27 11:41:26,041][20347] Updated weights for policy 0, policy_version 872886 (0.0008) [2023-12-27 11:41:26,091][20347] Updated weights for policy 0, policy_version 872896 (0.0007) [2023-12-27 11:41:26,142][20347] Updated weights for policy 0, policy_version 872906 (0.0007) [2023-12-27 11:41:26,547][20381] Updated weights for policy 1, policy_version 872194 (0.0008) [2023-12-27 11:41:26,603][20381] Updated weights for policy 1, policy_version 872204 (0.0008) [2023-12-27 11:41:26,666][20381] Updated weights for policy 1, policy_version 872214 (0.0008) [2023-12-27 11:41:26,875][20347] Updated weights for policy 0, policy_version 872916 (0.0007) [2023-12-27 11:41:26,934][20347] Updated weights for policy 0, policy_version 872926 (0.0006) [2023-12-27 11:41:26,991][20347] Updated weights for policy 0, policy_version 872936 (0.0006) [2023-12-27 11:41:27,306][20381] Updated weights for policy 1, policy_version 872224 (0.0007) [2023-12-27 11:41:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19387.7, 300 sec: 19466.4). Total num frames: 446816256. Throughput: 0: 9735.1, 1: 9761.0. Samples: 446808672. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:27,363][19170] Avg episode reward: [(0, '892.070'), (1, '922.210')] [2023-12-27 11:41:27,365][20381] Updated weights for policy 1, policy_version 872234 (0.0006) [2023-12-27 11:41:27,418][20381] Updated weights for policy 1, policy_version 872244 (0.0008) [2023-12-27 11:41:27,671][20347] Updated weights for policy 0, policy_version 872946 (0.0007) [2023-12-27 11:41:27,732][20347] Updated weights for policy 0, policy_version 872956 (0.0008) [2023-12-27 11:41:27,795][20347] Updated weights for policy 0, policy_version 872966 (0.0007) [2023-12-27 11:41:27,851][20347] Updated weights for policy 0, policy_version 872976 (0.0007) [2023-12-27 11:41:28,121][20381] Updated weights for policy 1, policy_version 872254 (0.0008) [2023-12-27 11:41:28,177][20381] Updated weights for policy 1, policy_version 872264 (0.0008) [2023-12-27 11:41:28,234][20381] Updated weights for policy 1, policy_version 872274 (0.0008) [2023-12-27 11:41:28,583][20347] Updated weights for policy 0, policy_version 872986 (0.0006) [2023-12-27 11:41:28,649][20347] Updated weights for policy 0, policy_version 872996 (0.0008) [2023-12-27 11:41:28,710][20347] Updated weights for policy 0, policy_version 873006 (0.0007) [2023-12-27 11:41:28,918][20381] Updated weights for policy 1, policy_version 872284 (0.0007) [2023-12-27 11:41:28,978][20381] Updated weights for policy 1, policy_version 872294 (0.0006) [2023-12-27 11:41:29,039][20381] Updated weights for policy 1, policy_version 872304 (0.0006) [2023-12-27 11:41:29,455][20347] Updated weights for policy 0, policy_version 873016 (0.0008) [2023-12-27 11:41:29,513][20347] Updated weights for policy 0, policy_version 873026 (0.0006) [2023-12-27 11:41:29,534][20291] KL-divergence is very high: 112.8835 [2023-12-27 11:41:29,572][20347] Updated weights for policy 0, policy_version 873036 (0.0007) [2023-12-27 11:41:29,750][20381] Updated weights for policy 1, policy_version 872314 (0.0008) [2023-12-27 11:41:29,808][20381] Updated weights for policy 1, policy_version 872324 (0.0008) [2023-12-27 11:41:29,869][20381] Updated weights for policy 1, policy_version 872334 (0.0008) [2023-12-27 11:41:29,933][20381] Updated weights for policy 1, policy_version 872344 (0.0007) [2023-12-27 11:41:30,266][20347] Updated weights for policy 0, policy_version 873046 (0.0009) [2023-12-27 11:41:30,329][20347] Updated weights for policy 0, policy_version 873056 (0.0008) [2023-12-27 11:41:30,389][20347] Updated weights for policy 0, policy_version 873066 (0.0008) [2023-12-27 11:41:30,658][20381] Updated weights for policy 1, policy_version 872354 (0.0008) [2023-12-27 11:41:30,718][20381] Updated weights for policy 1, policy_version 872364 (0.0008) [2023-12-27 11:41:30,774][20381] Updated weights for policy 1, policy_version 872374 (0.0008) [2023-12-27 11:41:31,029][20347] Updated weights for policy 0, policy_version 873076 (0.0007) [2023-12-27 11:41:31,090][20347] Updated weights for policy 0, policy_version 873086 (0.0007) [2023-12-27 11:41:31,157][20347] Updated weights for policy 0, policy_version 873096 (0.0008) [2023-12-27 11:41:31,532][20381] Updated weights for policy 1, policy_version 872384 (0.0007) [2023-12-27 11:41:31,586][20381] Updated weights for policy 1, policy_version 872394 (0.0008) [2023-12-27 11:41:31,647][20381] Updated weights for policy 1, policy_version 872404 (0.0007) [2023-12-27 11:41:31,855][20347] Updated weights for policy 0, policy_version 873106 (0.0008) [2023-12-27 11:41:31,908][20347] Updated weights for policy 0, policy_version 873116 (0.0008) [2023-12-27 11:41:31,966][20347] Updated weights for policy 0, policy_version 873126 (0.0008) [2023-12-27 11:41:32,020][20347] Updated weights for policy 0, policy_version 873136 (0.0007) [2023-12-27 11:41:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19387.7, 300 sec: 19438.6). Total num frames: 446914560. Throughput: 0: 9684.7, 1: 9799.7. Samples: 446926672. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:32,363][19170] Avg episode reward: [(0, '892.340'), (1, '951.350')] [2023-12-27 11:41:32,380][20381] Updated weights for policy 1, policy_version 872414 (0.0007) [2023-12-27 11:41:32,454][20381] Updated weights for policy 1, policy_version 872424 (0.0008) [2023-12-27 11:41:32,519][20381] Updated weights for policy 1, policy_version 872434 (0.0009) [2023-12-27 11:41:32,781][20347] Updated weights for policy 0, policy_version 873146 (0.0007) [2023-12-27 11:41:32,840][20347] Updated weights for policy 0, policy_version 873156 (0.0007) [2023-12-27 11:41:32,898][20347] Updated weights for policy 0, policy_version 873166 (0.0008) [2023-12-27 11:41:33,162][20381] Updated weights for policy 1, policy_version 872444 (0.0006) [2023-12-27 11:41:33,210][20381] Updated weights for policy 1, policy_version 872454 (0.0006) [2023-12-27 11:41:33,268][20381] Updated weights for policy 1, policy_version 872464 (0.0005) [2023-12-27 11:41:33,546][20347] Updated weights for policy 0, policy_version 873176 (0.0007) [2023-12-27 11:41:33,594][20347] Updated weights for policy 0, policy_version 873186 (0.0005) [2023-12-27 11:41:33,644][20347] Updated weights for policy 0, policy_version 873196 (0.0006) [2023-12-27 11:41:33,915][20381] Updated weights for policy 1, policy_version 872474 (0.0008) [2023-12-27 11:41:33,968][20381] Updated weights for policy 1, policy_version 872484 (0.0005) [2023-12-27 11:41:34,026][20381] Updated weights for policy 1, policy_version 872494 (0.0009) [2023-12-27 11:41:34,084][20381] Updated weights for policy 1, policy_version 872504 (0.0007) [2023-12-27 11:41:34,327][20347] Updated weights for policy 0, policy_version 873206 (0.0008) [2023-12-27 11:41:34,390][20347] Updated weights for policy 0, policy_version 873216 (0.0007) [2023-12-27 11:41:34,449][20347] Updated weights for policy 0, policy_version 873226 (0.0008) [2023-12-27 11:41:34,792][20381] Updated weights for policy 1, policy_version 872514 (0.0005) [2023-12-27 11:41:34,843][20381] Updated weights for policy 1, policy_version 872524 (0.0005) [2023-12-27 11:41:34,902][20381] Updated weights for policy 1, policy_version 872534 (0.0006) [2023-12-27 11:41:35,193][20347] Updated weights for policy 0, policy_version 873236 (0.0007) [2023-12-27 11:41:35,250][20347] Updated weights for policy 0, policy_version 873246 (0.0008) [2023-12-27 11:41:35,301][20347] Updated weights for policy 0, policy_version 873256 (0.0008) [2023-12-27 11:41:35,523][20381] Updated weights for policy 1, policy_version 872544 (0.0007) [2023-12-27 11:41:35,574][20381] Updated weights for policy 1, policy_version 872554 (0.0005) [2023-12-27 11:41:35,631][20381] Updated weights for policy 1, policy_version 872564 (0.0006) [2023-12-27 11:41:35,992][20347] Updated weights for policy 0, policy_version 873266 (0.0008) [2023-12-27 11:41:36,042][20347] Updated weights for policy 0, policy_version 873276 (0.0007) [2023-12-27 11:41:36,094][20347] Updated weights for policy 0, policy_version 873286 (0.0007) [2023-12-27 11:41:36,158][20347] Updated weights for policy 0, policy_version 873296 (0.0007) [2023-12-27 11:41:36,351][20381] Updated weights for policy 1, policy_version 872574 (0.0008) [2023-12-27 11:41:36,418][20381] Updated weights for policy 1, policy_version 872584 (0.0008) [2023-12-27 11:41:36,487][20381] Updated weights for policy 1, policy_version 872594 (0.0008) [2023-12-27 11:41:36,864][20347] Updated weights for policy 0, policy_version 873306 (0.0006) [2023-12-27 11:41:36,923][20347] Updated weights for policy 0, policy_version 873316 (0.0007) [2023-12-27 11:41:36,979][20347] Updated weights for policy 0, policy_version 873326 (0.0009) [2023-12-27 11:41:37,141][20381] Updated weights for policy 1, policy_version 872604 (0.0006) [2023-12-27 11:41:37,204][20381] Updated weights for policy 1, policy_version 872614 (0.0006) [2023-12-27 11:41:37,261][20381] Updated weights for policy 1, policy_version 872624 (0.0007) [2023-12-27 11:41:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 19524.3, 300 sec: 19466.4). Total num frames: 447021056. Throughput: 0: 9701.2, 1: 9814.9. Samples: 446988084. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:37,363][19170] Avg episode reward: [(0, '882.740'), (1, '990.320')] [2023-12-27 11:41:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000872632_223420416.pth... [2023-12-27 11:41:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000873328_223600640.pth... [2023-12-27 11:41:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000871480_223125504.pth [2023-12-27 11:41:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000872208_223313920.pth [2023-12-27 11:41:37,733][20347] Updated weights for policy 0, policy_version 873336 (0.0008) [2023-12-27 11:41:37,787][20347] Updated weights for policy 0, policy_version 873346 (0.0008) [2023-12-27 11:41:37,848][20347] Updated weights for policy 0, policy_version 873356 (0.0008) [2023-12-27 11:41:37,926][20381] Updated weights for policy 1, policy_version 872634 (0.0008) [2023-12-27 11:41:37,988][20381] Updated weights for policy 1, policy_version 872644 (0.0008) [2023-12-27 11:41:38,050][20381] Updated weights for policy 1, policy_version 872654 (0.0007) [2023-12-27 11:41:38,116][20381] Updated weights for policy 1, policy_version 872664 (0.0008) [2023-12-27 11:41:38,665][20347] Updated weights for policy 0, policy_version 873366 (0.0008) [2023-12-27 11:41:38,729][20347] Updated weights for policy 0, policy_version 873376 (0.0007) [2023-12-27 11:41:38,795][20347] Updated weights for policy 0, policy_version 873386 (0.0006) [2023-12-27 11:41:38,803][20381] Updated weights for policy 1, policy_version 872674 (0.0008) [2023-12-27 11:41:38,857][20381] Updated weights for policy 1, policy_version 872684 (0.0008) [2023-12-27 11:41:38,914][20381] Updated weights for policy 1, policy_version 872694 (0.0007) [2023-12-27 11:41:39,521][20347] Updated weights for policy 0, policy_version 873396 (0.0006) [2023-12-27 11:41:39,582][20347] Updated weights for policy 0, policy_version 873406 (0.0006) [2023-12-27 11:41:39,647][20347] Updated weights for policy 0, policy_version 873416 (0.0007) [2023-12-27 11:41:39,685][20381] Updated weights for policy 1, policy_version 872704 (0.0007) [2023-12-27 11:41:39,748][20381] Updated weights for policy 1, policy_version 872714 (0.0007) [2023-12-27 11:41:39,801][20381] Updated weights for policy 1, policy_version 872724 (0.0008) [2023-12-27 11:41:40,303][20347] Updated weights for policy 0, policy_version 873426 (0.0008) [2023-12-27 11:41:40,372][20347] Updated weights for policy 0, policy_version 873436 (0.0007) [2023-12-27 11:41:40,433][20347] Updated weights for policy 0, policy_version 873446 (0.0008) [2023-12-27 11:41:40,501][20347] Updated weights for policy 0, policy_version 873456 (0.0009) [2023-12-27 11:41:40,552][20381] Updated weights for policy 1, policy_version 872734 (0.0009) [2023-12-27 11:41:40,618][20381] Updated weights for policy 1, policy_version 872744 (0.0009) [2023-12-27 11:41:40,682][20381] Updated weights for policy 1, policy_version 872754 (0.0009) [2023-12-27 11:41:41,142][20347] Updated weights for policy 0, policy_version 873466 (0.0006) [2023-12-27 11:41:41,211][20347] Updated weights for policy 0, policy_version 873476 (0.0008) [2023-12-27 11:41:41,276][20347] Updated weights for policy 0, policy_version 873486 (0.0007) [2023-12-27 11:41:41,445][20381] Updated weights for policy 1, policy_version 872764 (0.0008) [2023-12-27 11:41:41,513][20381] Updated weights for policy 1, policy_version 872774 (0.0008) [2023-12-27 11:41:41,572][20381] Updated weights for policy 1, policy_version 872784 (0.0008) [2023-12-27 11:41:42,008][20347] Updated weights for policy 0, policy_version 873496 (0.0007) [2023-12-27 11:41:42,070][20347] Updated weights for policy 0, policy_version 873506 (0.0007) [2023-12-27 11:41:42,127][20347] Updated weights for policy 0, policy_version 873516 (0.0007) [2023-12-27 11:41:42,356][20381] Updated weights for policy 1, policy_version 872794 (0.0008) [2023-12-27 11:41:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 447111168. Throughput: 0: 9705.4, 1: 9838.3. Samples: 447104916. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:42,363][19170] Avg episode reward: [(0, '931.480'), (1, '990.320')] [2023-12-27 11:41:42,430][20381] Updated weights for policy 1, policy_version 872804 (0.0009) [2023-12-27 11:41:42,491][20381] Updated weights for policy 1, policy_version 872814 (0.0008) [2023-12-27 11:41:42,551][20381] Updated weights for policy 1, policy_version 872824 (0.0008) [2023-12-27 11:41:42,894][20347] Updated weights for policy 0, policy_version 873526 (0.0007) [2023-12-27 11:41:42,953][20347] Updated weights for policy 0, policy_version 873536 (0.0008) [2023-12-27 11:41:43,016][20347] Updated weights for policy 0, policy_version 873546 (0.0008) [2023-12-27 11:41:43,300][20381] Updated weights for policy 1, policy_version 872834 (0.0008) [2023-12-27 11:41:43,359][20381] Updated weights for policy 1, policy_version 872844 (0.0008) [2023-12-27 11:41:43,415][20381] Updated weights for policy 1, policy_version 872854 (0.0009) [2023-12-27 11:41:43,712][20347] Updated weights for policy 0, policy_version 873556 (0.0008) [2023-12-27 11:41:43,765][20347] Updated weights for policy 0, policy_version 873566 (0.0007) [2023-12-27 11:41:43,817][20347] Updated weights for policy 0, policy_version 873576 (0.0007) [2023-12-27 11:41:44,105][20381] Updated weights for policy 1, policy_version 872864 (0.0006) [2023-12-27 11:41:44,168][20381] Updated weights for policy 1, policy_version 872874 (0.0010) [2023-12-27 11:41:44,221][20381] Updated weights for policy 1, policy_version 872884 (0.0009) [2023-12-27 11:41:44,535][20347] Updated weights for policy 0, policy_version 873586 (0.0008) [2023-12-27 11:41:44,594][20347] Updated weights for policy 0, policy_version 873596 (0.0006) [2023-12-27 11:41:44,646][20347] Updated weights for policy 0, policy_version 873606 (0.0008) [2023-12-27 11:41:44,702][20347] Updated weights for policy 0, policy_version 873616 (0.0008) [2023-12-27 11:41:44,912][20381] Updated weights for policy 1, policy_version 872894 (0.0009) [2023-12-27 11:41:44,975][20381] Updated weights for policy 1, policy_version 872904 (0.0011) [2023-12-27 11:41:45,035][20381] Updated weights for policy 1, policy_version 872914 (0.0009) [2023-12-27 11:41:45,425][20347] Updated weights for policy 0, policy_version 873626 (0.0006) [2023-12-27 11:41:45,476][20347] Updated weights for policy 0, policy_version 873636 (0.0007) [2023-12-27 11:41:45,528][20347] Updated weights for policy 0, policy_version 873646 (0.0008) [2023-12-27 11:41:45,781][20381] Updated weights for policy 1, policy_version 872924 (0.0008) [2023-12-27 11:41:45,850][20381] Updated weights for policy 1, policy_version 872934 (0.0006) [2023-12-27 11:41:45,919][20381] Updated weights for policy 1, policy_version 872944 (0.0006) [2023-12-27 11:41:46,227][20347] Updated weights for policy 0, policy_version 873656 (0.0006) [2023-12-27 11:41:46,278][20347] Updated weights for policy 0, policy_version 873666 (0.0008) [2023-12-27 11:41:46,331][20347] Updated weights for policy 0, policy_version 873676 (0.0008) [2023-12-27 11:41:46,547][20381] Updated weights for policy 1, policy_version 872954 (0.0007) [2023-12-27 11:41:46,610][20381] Updated weights for policy 1, policy_version 872964 (0.0008) [2023-12-27 11:41:46,675][20381] Updated weights for policy 1, policy_version 872974 (0.0006) [2023-12-27 11:41:46,733][20381] Updated weights for policy 1, policy_version 872984 (0.0007) [2023-12-27 11:41:47,002][20347] Updated weights for policy 0, policy_version 873686 (0.0008) [2023-12-27 11:41:47,060][20347] Updated weights for policy 0, policy_version 873696 (0.0008) [2023-12-27 11:41:47,120][20347] Updated weights for policy 0, policy_version 873706 (0.0008) [2023-12-27 11:41:47,362][19170] Fps is (10 sec: 18841.8, 60 sec: 19387.7, 300 sec: 19410.9). Total num frames: 447209472. Throughput: 0: 9691.1, 1: 9839.0. Samples: 447221016. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:47,363][19170] Avg episode reward: [(0, '911.870'), (1, '941.620')] [2023-12-27 11:41:47,364][20381] Updated weights for policy 1, policy_version 872994 (0.0006) [2023-12-27 11:41:47,429][20381] Updated weights for policy 1, policy_version 873004 (0.0006) [2023-12-27 11:41:47,487][20381] Updated weights for policy 1, policy_version 873014 (0.0008) [2023-12-27 11:41:47,791][20347] Updated weights for policy 0, policy_version 873716 (0.0008) [2023-12-27 11:41:47,848][20347] Updated weights for policy 0, policy_version 873726 (0.0008) [2023-12-27 11:41:47,903][20347] Updated weights for policy 0, policy_version 873736 (0.0007) [2023-12-27 11:41:48,131][20381] Updated weights for policy 1, policy_version 873024 (0.0007) [2023-12-27 11:41:48,194][20381] Updated weights for policy 1, policy_version 873034 (0.0008) [2023-12-27 11:41:48,233][20292] KL-divergence is very high: 194.1077 [2023-12-27 11:41:48,250][20381] Updated weights for policy 1, policy_version 873044 (0.0008) [2023-12-27 11:41:48,603][20347] Updated weights for policy 0, policy_version 873746 (0.0006) [2023-12-27 11:41:48,664][20347] Updated weights for policy 0, policy_version 873756 (0.0008) [2023-12-27 11:41:48,722][20347] Updated weights for policy 0, policy_version 873766 (0.0007) [2023-12-27 11:41:48,787][20347] Updated weights for policy 0, policy_version 873776 (0.0008) [2023-12-27 11:41:48,967][20381] Updated weights for policy 1, policy_version 873054 (0.0007) [2023-12-27 11:41:49,029][20381] Updated weights for policy 1, policy_version 873064 (0.0007) [2023-12-27 11:41:49,089][20381] Updated weights for policy 1, policy_version 873074 (0.0008) [2023-12-27 11:41:49,544][20347] Updated weights for policy 0, policy_version 873786 (0.0008) [2023-12-27 11:41:49,607][20347] Updated weights for policy 0, policy_version 873796 (0.0008) [2023-12-27 11:41:49,671][20347] Updated weights for policy 0, policy_version 873806 (0.0007) [2023-12-27 11:41:49,802][20381] Updated weights for policy 1, policy_version 873084 (0.0007) [2023-12-27 11:41:49,864][20381] Updated weights for policy 1, policy_version 873094 (0.0008) [2023-12-27 11:41:49,924][20381] Updated weights for policy 1, policy_version 873104 (0.0008) [2023-12-27 11:41:50,372][20347] Updated weights for policy 0, policy_version 873816 (0.0007) [2023-12-27 11:41:50,433][20347] Updated weights for policy 0, policy_version 873826 (0.0008) [2023-12-27 11:41:50,493][20347] Updated weights for policy 0, policy_version 873836 (0.0008) [2023-12-27 11:41:50,679][20381] Updated weights for policy 1, policy_version 873114 (0.0007) [2023-12-27 11:41:50,743][20381] Updated weights for policy 1, policy_version 873124 (0.0007) [2023-12-27 11:41:50,806][20381] Updated weights for policy 1, policy_version 873134 (0.0007) [2023-12-27 11:41:50,865][20381] Updated weights for policy 1, policy_version 873144 (0.0008) [2023-12-27 11:41:51,254][20347] Updated weights for policy 0, policy_version 873846 (0.0007) [2023-12-27 11:41:51,317][20347] Updated weights for policy 0, policy_version 873856 (0.0008) [2023-12-27 11:41:51,391][20347] Updated weights for policy 0, policy_version 873866 (0.0008) [2023-12-27 11:41:51,631][20381] Updated weights for policy 1, policy_version 873154 (0.0008) [2023-12-27 11:41:51,696][20381] Updated weights for policy 1, policy_version 873164 (0.0006) [2023-12-27 11:41:51,766][20381] Updated weights for policy 1, policy_version 873174 (0.0008) [2023-12-27 11:41:52,102][20347] Updated weights for policy 0, policy_version 873876 (0.0007) [2023-12-27 11:41:52,161][20347] Updated weights for policy 0, policy_version 873886 (0.0008) [2023-12-27 11:41:52,225][20347] Updated weights for policy 0, policy_version 873896 (0.0007) [2023-12-27 11:41:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.8, 300 sec: 19383.1). Total num frames: 447307776. Throughput: 0: 9684.9, 1: 9822.1. Samples: 447279208. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:52,363][19170] Avg episode reward: [(0, '921.530'), (1, '961.020')] [2023-12-27 11:41:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000873176_223559680.pth... [2023-12-27 11:41:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000873904_223748096.pth... [2023-12-27 11:41:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000872024_223264768.pth [2023-12-27 11:41:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000872752_223453184.pth [2023-12-27 11:41:52,517][20381] Updated weights for policy 1, policy_version 873184 (0.0006) [2023-12-27 11:41:52,579][20381] Updated weights for policy 1, policy_version 873194 (0.0007) [2023-12-27 11:41:52,643][20381] Updated weights for policy 1, policy_version 873204 (0.0008) [2023-12-27 11:41:52,938][20347] Updated weights for policy 0, policy_version 873906 (0.0007) [2023-12-27 11:41:52,998][20347] Updated weights for policy 0, policy_version 873916 (0.0007) [2023-12-27 11:41:53,059][20347] Updated weights for policy 0, policy_version 873926 (0.0008) [2023-12-27 11:41:53,113][20347] Updated weights for policy 0, policy_version 873936 (0.0008) [2023-12-27 11:41:53,296][20381] Updated weights for policy 1, policy_version 873214 (0.0007) [2023-12-27 11:41:53,356][20381] Updated weights for policy 1, policy_version 873224 (0.0006) [2023-12-27 11:41:53,418][20381] Updated weights for policy 1, policy_version 873234 (0.0006) [2023-12-27 11:41:53,773][20347] Updated weights for policy 0, policy_version 873946 (0.0008) [2023-12-27 11:41:53,827][20347] Updated weights for policy 0, policy_version 873956 (0.0008) [2023-12-27 11:41:53,884][20347] Updated weights for policy 0, policy_version 873966 (0.0008) [2023-12-27 11:41:54,019][20381] Updated weights for policy 1, policy_version 873244 (0.0007) [2023-12-27 11:41:54,070][20381] Updated weights for policy 1, policy_version 873254 (0.0010) [2023-12-27 11:41:54,123][20381] Updated weights for policy 1, policy_version 873264 (0.0007) [2023-12-27 11:41:54,581][20347] Updated weights for policy 0, policy_version 873976 (0.0007) [2023-12-27 11:41:54,631][20347] Updated weights for policy 0, policy_version 873986 (0.0007) [2023-12-27 11:41:54,682][20347] Updated weights for policy 0, policy_version 873996 (0.0007) [2023-12-27 11:41:54,808][20381] Updated weights for policy 1, policy_version 873274 (0.0008) [2023-12-27 11:41:54,851][20381] Updated weights for policy 1, policy_version 873284 (0.0008) [2023-12-27 11:41:54,899][20381] Updated weights for policy 1, policy_version 873294 (0.0007) [2023-12-27 11:41:54,947][20381] Updated weights for policy 1, policy_version 873304 (0.0007) [2023-12-27 11:41:55,339][20347] Updated weights for policy 0, policy_version 874006 (0.0008) [2023-12-27 11:41:55,400][20347] Updated weights for policy 0, policy_version 874016 (0.0008) [2023-12-27 11:41:55,462][20347] Updated weights for policy 0, policy_version 874026 (0.0008) [2023-12-27 11:41:55,632][20381] Updated weights for policy 1, policy_version 873314 (0.0010) [2023-12-27 11:41:55,685][20381] Updated weights for policy 1, policy_version 873324 (0.0010) [2023-12-27 11:41:55,739][20381] Updated weights for policy 1, policy_version 873334 (0.0010) [2023-12-27 11:41:56,141][20347] Updated weights for policy 0, policy_version 874036 (0.0009) [2023-12-27 11:41:56,196][20347] Updated weights for policy 0, policy_version 874046 (0.0010) [2023-12-27 11:41:56,251][20347] Updated weights for policy 0, policy_version 874056 (0.0006) [2023-12-27 11:41:56,420][20381] Updated weights for policy 1, policy_version 873344 (0.0008) [2023-12-27 11:41:56,478][20381] Updated weights for policy 1, policy_version 873354 (0.0010) [2023-12-27 11:41:56,532][20381] Updated weights for policy 1, policy_version 873364 (0.0011) [2023-12-27 11:41:56,900][20347] Updated weights for policy 0, policy_version 874066 (0.0005) [2023-12-27 11:41:56,965][20347] Updated weights for policy 0, policy_version 874076 (0.0005) [2023-12-27 11:41:57,026][20347] Updated weights for policy 0, policy_version 874086 (0.0005) [2023-12-27 11:41:57,079][20347] Updated weights for policy 0, policy_version 874096 (0.0007) [2023-12-27 11:41:57,168][20381] Updated weights for policy 1, policy_version 873374 (0.0007) [2023-12-27 11:41:57,226][20381] Updated weights for policy 1, policy_version 873384 (0.0008) [2023-12-27 11:41:57,278][20381] Updated weights for policy 1, policy_version 873394 (0.0007) [2023-12-27 11:41:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19660.8, 300 sec: 19438.6). Total num frames: 447414272. Throughput: 0: 9760.8, 1: 9840.0. Samples: 447398984. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:41:57,363][19170] Avg episode reward: [(0, '941.130'), (1, '951.280')] [2023-12-27 11:41:57,688][20347] Updated weights for policy 0, policy_version 874106 (0.0008) [2023-12-27 11:41:57,749][20347] Updated weights for policy 0, policy_version 874116 (0.0007) [2023-12-27 11:41:57,813][20347] Updated weights for policy 0, policy_version 874126 (0.0010) [2023-12-27 11:41:57,923][20381] Updated weights for policy 1, policy_version 873404 (0.0007) [2023-12-27 11:41:57,979][20381] Updated weights for policy 1, policy_version 873414 (0.0011) [2023-12-27 11:41:58,042][20381] Updated weights for policy 1, policy_version 873424 (0.0011) [2023-12-27 11:41:58,577][20347] Updated weights for policy 0, policy_version 874136 (0.0009) [2023-12-27 11:41:58,641][20347] Updated weights for policy 0, policy_version 874146 (0.0009) [2023-12-27 11:41:58,711][20347] Updated weights for policy 0, policy_version 874156 (0.0011) [2023-12-27 11:41:58,841][20381] Updated weights for policy 1, policy_version 873434 (0.0010) [2023-12-27 11:41:58,913][20381] Updated weights for policy 1, policy_version 873444 (0.0009) [2023-12-27 11:41:58,982][20381] Updated weights for policy 1, policy_version 873454 (0.0008) [2023-12-27 11:41:59,056][20381] Updated weights for policy 1, policy_version 873464 (0.0008) [2023-12-27 11:41:59,571][20347] Updated weights for policy 0, policy_version 874166 (0.0008) [2023-12-27 11:41:59,631][20347] Updated weights for policy 0, policy_version 874176 (0.0007) [2023-12-27 11:41:59,695][20347] Updated weights for policy 0, policy_version 874186 (0.0007) [2023-12-27 11:41:59,891][20381] Updated weights for policy 1, policy_version 873474 (0.0008) [2023-12-27 11:41:59,956][20381] Updated weights for policy 1, policy_version 873484 (0.0008) [2023-12-27 11:42:00,016][20381] Updated weights for policy 1, policy_version 873494 (0.0007) [2023-12-27 11:42:00,444][20347] Updated weights for policy 0, policy_version 874196 (0.0007) [2023-12-27 11:42:00,499][20347] Updated weights for policy 0, policy_version 874206 (0.0008) [2023-12-27 11:42:00,554][20347] Updated weights for policy 0, policy_version 874216 (0.0008) [2023-12-27 11:42:00,750][20381] Updated weights for policy 1, policy_version 873504 (0.0008) [2023-12-27 11:42:00,803][20381] Updated weights for policy 1, policy_version 873514 (0.0008) [2023-12-27 11:42:00,856][20381] Updated weights for policy 1, policy_version 873524 (0.0008) [2023-12-27 11:42:01,266][20347] Updated weights for policy 0, policy_version 874226 (0.0006) [2023-12-27 11:42:01,321][20347] Updated weights for policy 0, policy_version 874236 (0.0006) [2023-12-27 11:42:01,388][20347] Updated weights for policy 0, policy_version 874246 (0.0009) [2023-12-27 11:42:01,460][20347] Updated weights for policy 0, policy_version 874256 (0.0008) [2023-12-27 11:42:01,595][20381] Updated weights for policy 1, policy_version 873534 (0.0007) [2023-12-27 11:42:01,658][20381] Updated weights for policy 1, policy_version 873544 (0.0007) [2023-12-27 11:42:01,719][20381] Updated weights for policy 1, policy_version 873554 (0.0006) [2023-12-27 11:42:02,219][20347] Updated weights for policy 0, policy_version 874266 (0.0010) [2023-12-27 11:42:02,279][20347] Updated weights for policy 0, policy_version 874276 (0.0010) [2023-12-27 11:42:02,338][20347] Updated weights for policy 0, policy_version 874286 (0.0010) [2023-12-27 11:42:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 447504384. Throughput: 0: 9719.5, 1: 9788.3. Samples: 447512160. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:02,363][19170] Avg episode reward: [(0, '970.600'), (1, '961.040')] [2023-12-27 11:42:02,551][20381] Updated weights for policy 1, policy_version 873564 (0.0009) [2023-12-27 11:42:02,607][20381] Updated weights for policy 1, policy_version 873574 (0.0008) [2023-12-27 11:42:02,659][20381] Updated weights for policy 1, policy_version 873584 (0.0008) [2023-12-27 11:42:03,046][20347] Updated weights for policy 0, policy_version 874296 (0.0010) [2023-12-27 11:42:03,108][20347] Updated weights for policy 0, policy_version 874306 (0.0010) [2023-12-27 11:42:03,170][20347] Updated weights for policy 0, policy_version 874316 (0.0010) [2023-12-27 11:42:03,317][20381] Updated weights for policy 1, policy_version 873594 (0.0010) [2023-12-27 11:42:03,373][20381] Updated weights for policy 1, policy_version 873604 (0.0010) [2023-12-27 11:42:03,432][20381] Updated weights for policy 1, policy_version 873614 (0.0010) [2023-12-27 11:42:03,490][20381] Updated weights for policy 1, policy_version 873624 (0.0005) [2023-12-27 11:42:03,856][20347] Updated weights for policy 0, policy_version 874326 (0.0010) [2023-12-27 11:42:03,921][20347] Updated weights for policy 0, policy_version 874336 (0.0010) [2023-12-27 11:42:03,986][20347] Updated weights for policy 0, policy_version 874346 (0.0009) [2023-12-27 11:42:04,143][20381] Updated weights for policy 1, policy_version 873634 (0.0008) [2023-12-27 11:42:04,207][20381] Updated weights for policy 1, policy_version 873644 (0.0008) [2023-12-27 11:42:04,272][20381] Updated weights for policy 1, policy_version 873654 (0.0008) [2023-12-27 11:42:04,657][20347] Updated weights for policy 0, policy_version 874356 (0.0010) [2023-12-27 11:42:04,719][20347] Updated weights for policy 0, policy_version 874366 (0.0010) [2023-12-27 11:42:04,781][20347] Updated weights for policy 0, policy_version 874376 (0.0011) [2023-12-27 11:42:04,983][20381] Updated weights for policy 1, policy_version 873664 (0.0008) [2023-12-27 11:42:05,040][20381] Updated weights for policy 1, policy_version 873674 (0.0006) [2023-12-27 11:42:05,091][20381] Updated weights for policy 1, policy_version 873684 (0.0007) [2023-12-27 11:42:05,471][20347] Updated weights for policy 0, policy_version 874386 (0.0010) [2023-12-27 11:42:05,556][20347] Updated weights for policy 0, policy_version 874396 (0.0010) [2023-12-27 11:42:05,607][20347] Updated weights for policy 0, policy_version 874406 (0.0010) [2023-12-27 11:42:05,661][20347] Updated weights for policy 0, policy_version 874416 (0.0010) [2023-12-27 11:42:05,762][20381] Updated weights for policy 1, policy_version 873694 (0.0007) [2023-12-27 11:42:05,818][20381] Updated weights for policy 1, policy_version 873704 (0.0007) [2023-12-27 11:42:05,872][20381] Updated weights for policy 1, policy_version 873714 (0.0007) [2023-12-27 11:42:06,348][20347] Updated weights for policy 0, policy_version 874426 (0.0010) [2023-12-27 11:42:06,411][20347] Updated weights for policy 0, policy_version 874436 (0.0009) [2023-12-27 11:42:06,471][20347] Updated weights for policy 0, policy_version 874446 (0.0008) [2023-12-27 11:42:06,567][20381] Updated weights for policy 1, policy_version 873724 (0.0009) [2023-12-27 11:42:06,629][20381] Updated weights for policy 1, policy_version 873734 (0.0007) [2023-12-27 11:42:06,689][20381] Updated weights for policy 1, policy_version 873744 (0.0010) [2023-12-27 11:42:07,247][20347] Updated weights for policy 0, policy_version 874456 (0.0009) [2023-12-27 11:42:07,307][20347] Updated weights for policy 0, policy_version 874466 (0.0008) [2023-12-27 11:42:07,342][20381] Updated weights for policy 1, policy_version 873754 (0.0007) [2023-12-27 11:42:07,362][19170] Fps is (10 sec: 18022.3, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 447594496. Throughput: 0: 9716.9, 1: 9822.3. Samples: 447571496. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:07,363][19170] Avg episode reward: [(0, '980.370'), (1, '970.730')] [2023-12-27 11:42:07,366][20347] Updated weights for policy 0, policy_version 874476 (0.0006) [2023-12-27 11:42:07,388][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000874480_223895552.pth... [2023-12-27 11:42:07,392][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000873328_223600640.pth [2023-12-27 11:42:07,405][20381] Updated weights for policy 1, policy_version 873764 (0.0010) [2023-12-27 11:42:07,470][20381] Updated weights for policy 1, policy_version 873774 (0.0010) [2023-12-27 11:42:07,532][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000873784_223715328.pth... [2023-12-27 11:42:07,535][20381] Updated weights for policy 1, policy_version 873784 (0.0011) [2023-12-27 11:42:07,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000872632_223420416.pth [2023-12-27 11:42:08,054][20347] Updated weights for policy 0, policy_version 874486 (0.0007) [2023-12-27 11:42:08,121][20347] Updated weights for policy 0, policy_version 874496 (0.0008) [2023-12-27 11:42:08,185][20347] Updated weights for policy 0, policy_version 874506 (0.0006) [2023-12-27 11:42:08,234][20381] Updated weights for policy 1, policy_version 873794 (0.0009) [2023-12-27 11:42:08,292][20381] Updated weights for policy 1, policy_version 873804 (0.0008) [2023-12-27 11:42:08,360][20381] Updated weights for policy 1, policy_version 873814 (0.0007) [2023-12-27 11:42:08,920][20347] Updated weights for policy 0, policy_version 874516 (0.0006) [2023-12-27 11:42:08,956][20291] KL-divergence is very high: 164.4034 [2023-12-27 11:42:08,975][20347] Updated weights for policy 0, policy_version 874526 (0.0007) [2023-12-27 11:42:09,004][20291] KL-divergence is very high: 179.2509 [2023-12-27 11:42:09,008][20381] Updated weights for policy 1, policy_version 873824 (0.0007) [2023-12-27 11:42:09,033][20347] Updated weights for policy 0, policy_version 874536 (0.0007) [2023-12-27 11:42:09,074][20381] Updated weights for policy 1, policy_version 873834 (0.0006) [2023-12-27 11:42:09,132][20381] Updated weights for policy 1, policy_version 873844 (0.0007) [2023-12-27 11:42:09,772][20347] Updated weights for policy 0, policy_version 874546 (0.0006) [2023-12-27 11:42:09,842][20347] Updated weights for policy 0, policy_version 874556 (0.0008) [2023-12-27 11:42:09,861][20291] KL-divergence is very high: 181.7079 [2023-12-27 11:42:09,895][20381] Updated weights for policy 1, policy_version 873854 (0.0006) [2023-12-27 11:42:09,901][20347] Updated weights for policy 0, policy_version 874566 (0.0006) [2023-12-27 11:42:09,908][20291] KL-divergence is very high: 203.2539 [2023-12-27 11:42:09,953][20291] KL-divergence is very high: 138.9952 [2023-12-27 11:42:09,961][20347] Updated weights for policy 0, policy_version 874576 (0.0008) [2023-12-27 11:42:09,961][20381] Updated weights for policy 1, policy_version 873864 (0.0008) [2023-12-27 11:42:10,025][20381] Updated weights for policy 1, policy_version 873874 (0.0007) [2023-12-27 11:42:10,606][20291] KL-divergence is very high: 133.1732 [2023-12-27 11:42:10,648][20291] KL-divergence is very high: 127.4236 [2023-12-27 11:42:10,658][20347] Updated weights for policy 0, policy_version 874586 (0.0008) [2023-12-27 11:42:10,716][20347] Updated weights for policy 0, policy_version 874596 (0.0008) [2023-12-27 11:42:10,762][20381] Updated weights for policy 1, policy_version 873884 (0.0006) [2023-12-27 11:42:10,783][20347] Updated weights for policy 0, policy_version 874606 (0.0007) [2023-12-27 11:42:10,848][20381] Updated weights for policy 1, policy_version 873894 (0.0007) [2023-12-27 11:42:10,903][20381] Updated weights for policy 1, policy_version 873904 (0.0007) [2023-12-27 11:42:11,583][20347] Updated weights for policy 0, policy_version 874616 (0.0010) [2023-12-27 11:42:11,640][20381] Updated weights for policy 1, policy_version 873914 (0.0008) [2023-12-27 11:42:11,645][20347] Updated weights for policy 0, policy_version 874626 (0.0009) [2023-12-27 11:42:11,704][20381] Updated weights for policy 1, policy_version 873924 (0.0007) [2023-12-27 11:42:11,705][20347] Updated weights for policy 0, policy_version 874636 (0.0006) [2023-12-27 11:42:11,770][20381] Updated weights for policy 1, policy_version 873934 (0.0008) [2023-12-27 11:42:11,838][20381] Updated weights for policy 1, policy_version 873944 (0.0007) [2023-12-27 11:42:12,362][19170] Fps is (10 sec: 18841.3, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 447692800. Throughput: 0: 9727.8, 1: 9816.6. Samples: 447688172. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:12,363][19170] Avg episode reward: [(0, '912.010'), (1, '951.240')] [2023-12-27 11:42:12,446][20347] Updated weights for policy 0, policy_version 874646 (0.0006) [2023-12-27 11:42:12,507][20347] Updated weights for policy 0, policy_version 874656 (0.0008) [2023-12-27 11:42:12,567][20347] Updated weights for policy 0, policy_version 874666 (0.0007) [2023-12-27 11:42:12,569][20381] Updated weights for policy 1, policy_version 873954 (0.0006) [2023-12-27 11:42:12,635][20381] Updated weights for policy 1, policy_version 873964 (0.0008) [2023-12-27 11:42:12,693][20381] Updated weights for policy 1, policy_version 873974 (0.0008) [2023-12-27 11:42:13,329][20347] Updated weights for policy 0, policy_version 874676 (0.0007) [2023-12-27 11:42:13,391][20347] Updated weights for policy 0, policy_version 874686 (0.0009) [2023-12-27 11:42:13,416][20381] Updated weights for policy 1, policy_version 873984 (0.0008) [2023-12-27 11:42:13,445][20347] Updated weights for policy 0, policy_version 874696 (0.0009) [2023-12-27 11:42:13,470][20381] Updated weights for policy 1, policy_version 873994 (0.0009) [2023-12-27 11:42:13,527][20381] Updated weights for policy 1, policy_version 874004 (0.0009) [2023-12-27 11:42:14,176][20347] Updated weights for policy 0, policy_version 874706 (0.0009) [2023-12-27 11:42:14,224][20347] Updated weights for policy 0, policy_version 874716 (0.0007) [2023-12-27 11:42:14,282][20347] Updated weights for policy 0, policy_version 874726 (0.0006) [2023-12-27 11:42:14,288][20381] Updated weights for policy 1, policy_version 874014 (0.0009) [2023-12-27 11:42:14,340][20347] Updated weights for policy 0, policy_version 874736 (0.0006) [2023-12-27 11:42:14,345][20381] Updated weights for policy 1, policy_version 874024 (0.0007) [2023-12-27 11:42:14,404][20381] Updated weights for policy 1, policy_version 874034 (0.0009) [2023-12-27 11:42:15,040][20347] Updated weights for policy 0, policy_version 874746 (0.0007) [2023-12-27 11:42:15,101][20347] Updated weights for policy 0, policy_version 874756 (0.0007) [2023-12-27 11:42:15,126][20381] Updated weights for policy 1, policy_version 874044 (0.0009) [2023-12-27 11:42:15,163][20347] Updated weights for policy 0, policy_version 874766 (0.0008) [2023-12-27 11:42:15,189][20381] Updated weights for policy 1, policy_version 874054 (0.0007) [2023-12-27 11:42:15,255][20381] Updated weights for policy 1, policy_version 874064 (0.0007) [2023-12-27 11:42:15,876][20347] Updated weights for policy 0, policy_version 874776 (0.0007) [2023-12-27 11:42:15,927][20347] Updated weights for policy 0, policy_version 874786 (0.0007) [2023-12-27 11:42:15,967][20381] Updated weights for policy 1, policy_version 874074 (0.0007) [2023-12-27 11:42:15,983][20347] Updated weights for policy 0, policy_version 874796 (0.0007) [2023-12-27 11:42:16,014][20381] Updated weights for policy 1, policy_version 874084 (0.0005) [2023-12-27 11:42:16,072][20381] Updated weights for policy 1, policy_version 874094 (0.0007) [2023-12-27 11:42:16,125][20381] Updated weights for policy 1, policy_version 874104 (0.0008) [2023-12-27 11:42:16,661][20347] Updated weights for policy 0, policy_version 874806 (0.0006) [2023-12-27 11:42:16,715][20347] Updated weights for policy 0, policy_version 874816 (0.0007) [2023-12-27 11:42:16,767][20347] Updated weights for policy 0, policy_version 874826 (0.0005) [2023-12-27 11:42:16,783][20381] Updated weights for policy 1, policy_version 874114 (0.0008) [2023-12-27 11:42:16,837][20381] Updated weights for policy 1, policy_version 874124 (0.0007) [2023-12-27 11:42:16,886][20381] Updated weights for policy 1, policy_version 874134 (0.0008) [2023-12-27 11:42:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 447791104. Throughput: 0: 9717.8, 1: 9760.8. Samples: 447803208. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:17,363][19170] Avg episode reward: [(0, '941.360'), (1, '951.260')] [2023-12-27 11:42:17,435][20347] Updated weights for policy 0, policy_version 874836 (0.0007) [2023-12-27 11:42:17,494][20347] Updated weights for policy 0, policy_version 874846 (0.0007) [2023-12-27 11:42:17,559][20347] Updated weights for policy 0, policy_version 874856 (0.0007) [2023-12-27 11:42:17,597][20381] Updated weights for policy 1, policy_version 874144 (0.0007) [2023-12-27 11:42:17,654][20381] Updated weights for policy 1, policy_version 874154 (0.0008) [2023-12-27 11:42:17,716][20381] Updated weights for policy 1, policy_version 874164 (0.0008) [2023-12-27 11:42:18,268][20347] Updated weights for policy 0, policy_version 874866 (0.0007) [2023-12-27 11:42:18,326][20347] Updated weights for policy 0, policy_version 874876 (0.0007) [2023-12-27 11:42:18,367][20381] Updated weights for policy 1, policy_version 874174 (0.0009) [2023-12-27 11:42:18,392][20347] Updated weights for policy 0, policy_version 874886 (0.0007) [2023-12-27 11:42:18,431][20381] Updated weights for policy 1, policy_version 874184 (0.0008) [2023-12-27 11:42:18,453][20347] Updated weights for policy 0, policy_version 874896 (0.0006) [2023-12-27 11:42:18,491][20381] Updated weights for policy 1, policy_version 874194 (0.0007) [2023-12-27 11:42:19,158][20347] Updated weights for policy 0, policy_version 874906 (0.0006) [2023-12-27 11:42:19,197][20381] Updated weights for policy 1, policy_version 874204 (0.0007) [2023-12-27 11:42:19,214][20347] Updated weights for policy 0, policy_version 874916 (0.0008) [2023-12-27 11:42:19,261][20381] Updated weights for policy 1, policy_version 874214 (0.0008) [2023-12-27 11:42:19,269][20347] Updated weights for policy 0, policy_version 874926 (0.0007) [2023-12-27 11:42:19,315][20381] Updated weights for policy 1, policy_version 874224 (0.0006) [2023-12-27 11:42:20,002][20347] Updated weights for policy 0, policy_version 874936 (0.0008) [2023-12-27 11:42:20,063][20347] Updated weights for policy 0, policy_version 874946 (0.0008) [2023-12-27 11:42:20,069][20381] Updated weights for policy 1, policy_version 874234 (0.0008) [2023-12-27 11:42:20,127][20347] Updated weights for policy 0, policy_version 874956 (0.0006) [2023-12-27 11:42:20,132][20381] Updated weights for policy 1, policy_version 874244 (0.0009) [2023-12-27 11:42:20,197][20381] Updated weights for policy 1, policy_version 874254 (0.0008) [2023-12-27 11:42:20,256][20381] Updated weights for policy 1, policy_version 874264 (0.0008) [2023-12-27 11:42:20,818][20347] Updated weights for policy 0, policy_version 874966 (0.0009) [2023-12-27 11:42:20,878][20347] Updated weights for policy 0, policy_version 874976 (0.0010) [2023-12-27 11:42:20,941][20347] Updated weights for policy 0, policy_version 874986 (0.0009) [2023-12-27 11:42:20,994][20381] Updated weights for policy 1, policy_version 874274 (0.0007) [2023-12-27 11:42:21,056][20381] Updated weights for policy 1, policy_version 874284 (0.0008) [2023-12-27 11:42:21,119][20381] Updated weights for policy 1, policy_version 874294 (0.0008) [2023-12-27 11:42:21,701][20347] Updated weights for policy 0, policy_version 874996 (0.0008) [2023-12-27 11:42:21,772][20347] Updated weights for policy 0, policy_version 875006 (0.0009) [2023-12-27 11:42:21,825][20347] Updated weights for policy 0, policy_version 875016 (0.0007) [2023-12-27 11:42:21,912][20381] Updated weights for policy 1, policy_version 874304 (0.0008) [2023-12-27 11:42:21,980][20381] Updated weights for policy 1, policy_version 874314 (0.0008) [2023-12-27 11:42:22,039][20381] Updated weights for policy 1, policy_version 874324 (0.0008) [2023-12-27 11:42:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.3, 300 sec: 19383.1). Total num frames: 447889408. Throughput: 0: 9686.1, 1: 9721.2. Samples: 447861416. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:22,363][19170] Avg episode reward: [(0, '921.840'), (1, '922.010')] [2023-12-27 11:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000874328_223854592.pth... [2023-12-27 11:42:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000875024_224034816.pth... [2023-12-27 11:42:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000873176_223559680.pth [2023-12-27 11:42:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000873904_223748096.pth [2023-12-27 11:42:22,571][20347] Updated weights for policy 0, policy_version 875026 (0.0008) [2023-12-27 11:42:22,630][20347] Updated weights for policy 0, policy_version 875036 (0.0008) [2023-12-27 11:42:22,690][20347] Updated weights for policy 0, policy_version 875046 (0.0009) [2023-12-27 11:42:22,756][20347] Updated weights for policy 0, policy_version 875056 (0.0007) [2023-12-27 11:42:22,812][20381] Updated weights for policy 1, policy_version 874334 (0.0007) [2023-12-27 11:42:22,875][20381] Updated weights for policy 1, policy_version 874344 (0.0006) [2023-12-27 11:42:22,935][20381] Updated weights for policy 1, policy_version 874354 (0.0006) [2023-12-27 11:42:23,506][20347] Updated weights for policy 0, policy_version 875066 (0.0008) [2023-12-27 11:42:23,571][20347] Updated weights for policy 0, policy_version 875076 (0.0009) [2023-12-27 11:42:23,602][20381] Updated weights for policy 1, policy_version 874364 (0.0006) [2023-12-27 11:42:23,631][20347] Updated weights for policy 0, policy_version 875086 (0.0007) [2023-12-27 11:42:23,662][20381] Updated weights for policy 1, policy_version 874374 (0.0007) [2023-12-27 11:42:23,729][20381] Updated weights for policy 1, policy_version 874384 (0.0008) [2023-12-27 11:42:24,312][20347] Updated weights for policy 0, policy_version 875096 (0.0008) [2023-12-27 11:42:24,379][20347] Updated weights for policy 0, policy_version 875106 (0.0007) [2023-12-27 11:42:24,445][20347] Updated weights for policy 0, policy_version 875116 (0.0006) [2023-12-27 11:42:24,455][20381] Updated weights for policy 1, policy_version 874394 (0.0007) [2023-12-27 11:42:24,515][20381] Updated weights for policy 1, policy_version 874404 (0.0008) [2023-12-27 11:42:24,570][20381] Updated weights for policy 1, policy_version 874414 (0.0007) [2023-12-27 11:42:24,624][20381] Updated weights for policy 1, policy_version 874424 (0.0008) [2023-12-27 11:42:25,074][20347] Updated weights for policy 0, policy_version 875126 (0.0006) [2023-12-27 11:42:25,135][20347] Updated weights for policy 0, policy_version 875136 (0.0005) [2023-12-27 11:42:25,199][20347] Updated weights for policy 0, policy_version 875146 (0.0006) [2023-12-27 11:42:25,341][20381] Updated weights for policy 1, policy_version 874434 (0.0009) [2023-12-27 11:42:25,402][20381] Updated weights for policy 1, policy_version 874444 (0.0008) [2023-12-27 11:42:25,455][20381] Updated weights for policy 1, policy_version 874454 (0.0006) [2023-12-27 11:42:25,825][20347] Updated weights for policy 0, policy_version 875156 (0.0007) [2023-12-27 11:42:25,890][20347] Updated weights for policy 0, policy_version 875166 (0.0007) [2023-12-27 11:42:25,951][20347] Updated weights for policy 0, policy_version 875176 (0.0008) [2023-12-27 11:42:26,178][20381] Updated weights for policy 1, policy_version 874464 (0.0008) [2023-12-27 11:42:26,240][20381] Updated weights for policy 1, policy_version 874474 (0.0007) [2023-12-27 11:42:26,299][20381] Updated weights for policy 1, policy_version 874484 (0.0008) [2023-12-27 11:42:26,612][20347] Updated weights for policy 0, policy_version 875186 (0.0007) [2023-12-27 11:42:26,667][20347] Updated weights for policy 0, policy_version 875196 (0.0007) [2023-12-27 11:42:26,727][20347] Updated weights for policy 0, policy_version 875206 (0.0008) [2023-12-27 11:42:26,782][20347] Updated weights for policy 0, policy_version 875216 (0.0007) [2023-12-27 11:42:26,984][20381] Updated weights for policy 1, policy_version 874494 (0.0008) [2023-12-27 11:42:27,040][20381] Updated weights for policy 1, policy_version 874504 (0.0008) [2023-12-27 11:42:27,096][20381] Updated weights for policy 1, policy_version 874514 (0.0007) [2023-12-27 11:42:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19524.2, 300 sec: 19383.1). Total num frames: 447987712. Throughput: 0: 9691.0, 1: 9686.3. Samples: 447976896. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:27,363][19170] Avg episode reward: [(0, '872.980'), (1, '961.020')] [2023-12-27 11:42:27,471][20347] Updated weights for policy 0, policy_version 875226 (0.0006) [2023-12-27 11:42:27,522][20347] Updated weights for policy 0, policy_version 875236 (0.0008) [2023-12-27 11:42:27,565][20347] Updated weights for policy 0, policy_version 875246 (0.0007) [2023-12-27 11:42:27,811][20381] Updated weights for policy 1, policy_version 874524 (0.0007) [2023-12-27 11:42:27,875][20381] Updated weights for policy 1, policy_version 874534 (0.0005) [2023-12-27 11:42:27,927][20381] Updated weights for policy 1, policy_version 874544 (0.0007) [2023-12-27 11:42:28,202][20347] Updated weights for policy 0, policy_version 875256 (0.0009) [2023-12-27 11:42:28,247][20347] Updated weights for policy 0, policy_version 875266 (0.0010) [2023-12-27 11:42:28,296][20347] Updated weights for policy 0, policy_version 875276 (0.0010) [2023-12-27 11:42:28,599][20381] Updated weights for policy 1, policy_version 874554 (0.0006) [2023-12-27 11:42:28,660][20381] Updated weights for policy 1, policy_version 874564 (0.0005) [2023-12-27 11:42:28,718][20381] Updated weights for policy 1, policy_version 874574 (0.0005) [2023-12-27 11:42:28,780][20381] Updated weights for policy 1, policy_version 874584 (0.0007) [2023-12-27 11:42:28,996][20347] Updated weights for policy 0, policy_version 875286 (0.0010) [2023-12-27 11:42:29,056][20347] Updated weights for policy 0, policy_version 875296 (0.0010) [2023-12-27 11:42:29,079][20291] KL-divergence is very high: 120.5851 [2023-12-27 11:42:29,117][20347] Updated weights for policy 0, policy_version 875306 (0.0007) [2023-12-27 11:42:29,464][20381] Updated weights for policy 1, policy_version 874594 (0.0008) [2023-12-27 11:42:29,511][20381] Updated weights for policy 1, policy_version 874604 (0.0008) [2023-12-27 11:42:29,569][20381] Updated weights for policy 1, policy_version 874614 (0.0008) [2023-12-27 11:42:29,780][20347] Updated weights for policy 0, policy_version 875316 (0.0009) [2023-12-27 11:42:29,845][20347] Updated weights for policy 0, policy_version 875326 (0.0007) [2023-12-27 11:42:29,907][20347] Updated weights for policy 0, policy_version 875336 (0.0007) [2023-12-27 11:42:30,289][20381] Updated weights for policy 1, policy_version 874624 (0.0008) [2023-12-27 11:42:30,352][20381] Updated weights for policy 1, policy_version 874634 (0.0009) [2023-12-27 11:42:30,412][20381] Updated weights for policy 1, policy_version 874644 (0.0007) [2023-12-27 11:42:30,490][20347] Updated weights for policy 0, policy_version 875346 (0.0007) [2023-12-27 11:42:30,552][20347] Updated weights for policy 0, policy_version 875356 (0.0007) [2023-12-27 11:42:30,602][20347] Updated weights for policy 0, policy_version 875366 (0.0008) [2023-12-27 11:42:30,650][20347] Updated weights for policy 0, policy_version 875376 (0.0007) [2023-12-27 11:42:31,091][20381] Updated weights for policy 1, policy_version 874654 (0.0008) [2023-12-27 11:42:31,157][20381] Updated weights for policy 1, policy_version 874664 (0.0007) [2023-12-27 11:42:31,218][20381] Updated weights for policy 1, policy_version 874674 (0.0008) [2023-12-27 11:42:31,351][20347] Updated weights for policy 0, policy_version 875386 (0.0008) [2023-12-27 11:42:31,417][20347] Updated weights for policy 0, policy_version 875396 (0.0008) [2023-12-27 11:42:31,480][20347] Updated weights for policy 0, policy_version 875406 (0.0008) [2023-12-27 11:42:31,937][20381] Updated weights for policy 1, policy_version 874684 (0.0008) [2023-12-27 11:42:31,987][20381] Updated weights for policy 1, policy_version 874694 (0.0009) [2023-12-27 11:42:32,034][20381] Updated weights for policy 1, policy_version 874704 (0.0008) [2023-12-27 11:42:32,169][20347] Updated weights for policy 0, policy_version 875416 (0.0005) [2023-12-27 11:42:32,229][20347] Updated weights for policy 0, policy_version 875426 (0.0006) [2023-12-27 11:42:32,296][20347] Updated weights for policy 0, policy_version 875436 (0.0007) [2023-12-27 11:42:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 19660.8, 300 sec: 19383.1). Total num frames: 448094208. Throughput: 0: 9775.6, 1: 9728.0. Samples: 448098676. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:32,363][19170] Avg episode reward: [(0, '892.560'), (1, '1000.000')] [2023-12-27 11:42:32,725][20381] Updated weights for policy 1, policy_version 874715 (0.0007) [2023-12-27 11:42:32,784][20381] Updated weights for policy 1, policy_version 874725 (0.0006) [2023-12-27 11:42:32,849][20381] Updated weights for policy 1, policy_version 874735 (0.0007) [2023-12-27 11:42:32,944][20347] Updated weights for policy 0, policy_version 875446 (0.0008) [2023-12-27 11:42:32,994][20347] Updated weights for policy 0, policy_version 875456 (0.0007) [2023-12-27 11:42:33,040][20347] Updated weights for policy 0, policy_version 875466 (0.0007) [2023-12-27 11:42:33,511][20381] Updated weights for policy 1, policy_version 874745 (0.0008) [2023-12-27 11:42:33,561][20381] Updated weights for policy 1, policy_version 874755 (0.0006) [2023-12-27 11:42:33,613][20381] Updated weights for policy 1, policy_version 874765 (0.0008) [2023-12-27 11:42:33,671][20381] Updated weights for policy 1, policy_version 874775 (0.0008) [2023-12-27 11:42:33,735][20347] Updated weights for policy 0, policy_version 875476 (0.0007) [2023-12-27 11:42:33,791][20347] Updated weights for policy 0, policy_version 875486 (0.0007) [2023-12-27 11:42:33,849][20347] Updated weights for policy 0, policy_version 875496 (0.0007) [2023-12-27 11:42:34,368][20381] Updated weights for policy 1, policy_version 874785 (0.0009) [2023-12-27 11:42:34,423][20381] Updated weights for policy 1, policy_version 874795 (0.0007) [2023-12-27 11:42:34,486][20381] Updated weights for policy 1, policy_version 874805 (0.0008) [2023-12-27 11:42:34,522][20347] Updated weights for policy 0, policy_version 875506 (0.0008) [2023-12-27 11:42:34,578][20347] Updated weights for policy 0, policy_version 875516 (0.0010) [2023-12-27 11:42:34,628][20347] Updated weights for policy 0, policy_version 875526 (0.0010) [2023-12-27 11:42:34,691][20347] Updated weights for policy 0, policy_version 875536 (0.0011) [2023-12-27 11:42:35,113][20381] Updated weights for policy 1, policy_version 874815 (0.0006) [2023-12-27 11:42:35,168][20381] Updated weights for policy 1, policy_version 874825 (0.0006) [2023-12-27 11:42:35,215][20381] Updated weights for policy 1, policy_version 874835 (0.0010) [2023-12-27 11:42:35,334][20347] Updated weights for policy 0, policy_version 875546 (0.0010) [2023-12-27 11:42:35,385][20347] Updated weights for policy 0, policy_version 875556 (0.0010) [2023-12-27 11:42:35,442][20347] Updated weights for policy 0, policy_version 875566 (0.0010) [2023-12-27 11:42:35,799][20381] Updated weights for policy 1, policy_version 874845 (0.0007) [2023-12-27 11:42:35,854][20381] Updated weights for policy 1, policy_version 874855 (0.0005) [2023-12-27 11:42:35,906][20381] Updated weights for policy 1, policy_version 874865 (0.0005) [2023-12-27 11:42:36,116][20347] Updated weights for policy 0, policy_version 875576 (0.0009) [2023-12-27 11:42:36,171][20347] Updated weights for policy 0, policy_version 875586 (0.0010) [2023-12-27 11:42:36,220][20347] Updated weights for policy 0, policy_version 875596 (0.0010) [2023-12-27 11:42:36,612][20381] Updated weights for policy 1, policy_version 874875 (0.0006) [2023-12-27 11:42:36,670][20381] Updated weights for policy 1, policy_version 874885 (0.0007) [2023-12-27 11:42:36,726][20381] Updated weights for policy 1, policy_version 874895 (0.0007) [2023-12-27 11:42:36,906][20347] Updated weights for policy 0, policy_version 875606 (0.0007) [2023-12-27 11:42:36,957][20347] Updated weights for policy 0, policy_version 875616 (0.0005) [2023-12-27 11:42:37,007][20347] Updated weights for policy 0, policy_version 875626 (0.0005) [2023-12-27 11:42:37,362][20381] Updated weights for policy 1, policy_version 874905 (0.0006) [2023-12-27 11:42:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19524.3, 300 sec: 19410.9). Total num frames: 448192512. Throughput: 0: 9812.4, 1: 9785.1. Samples: 448161092. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:37,363][19170] Avg episode reward: [(0, '843.390'), (1, '990.260')] [2023-12-27 11:42:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000875632_224190464.pth... [2023-12-27 11:42:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000874480_223895552.pth [2023-12-27 11:42:37,418][20381] Updated weights for policy 1, policy_version 874915 (0.0009) [2023-12-27 11:42:37,473][20381] Updated weights for policy 1, policy_version 874925 (0.0008) [2023-12-27 11:42:37,525][20381] Updated weights for policy 1, policy_version 874935 (0.0007) [2023-12-27 11:42:37,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000874936_224010240.pth... [2023-12-27 11:42:37,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000873784_223715328.pth [2023-12-27 11:42:37,578][20347] Updated weights for policy 0, policy_version 875636 (0.0005) [2023-12-27 11:42:37,630][20347] Updated weights for policy 0, policy_version 875646 (0.0005) [2023-12-27 11:42:37,686][20347] Updated weights for policy 0, policy_version 875656 (0.0010) [2023-12-27 11:42:38,182][20381] Updated weights for policy 1, policy_version 874945 (0.0005) [2023-12-27 11:42:38,239][20381] Updated weights for policy 1, policy_version 874955 (0.0007) [2023-12-27 11:42:38,305][20381] Updated weights for policy 1, policy_version 874965 (0.0008) [2023-12-27 11:42:38,307][20347] Updated weights for policy 0, policy_version 875666 (0.0009) [2023-12-27 11:42:38,378][20347] Updated weights for policy 0, policy_version 875676 (0.0008) [2023-12-27 11:42:38,427][20347] Updated weights for policy 0, policy_version 875686 (0.0008) [2023-12-27 11:42:38,482][20347] Updated weights for policy 0, policy_version 875696 (0.0007) [2023-12-27 11:42:38,933][20381] Updated weights for policy 1, policy_version 874975 (0.0008) [2023-12-27 11:42:38,978][20381] Updated weights for policy 1, policy_version 874985 (0.0007) [2023-12-27 11:42:39,033][20381] Updated weights for policy 1, policy_version 874995 (0.0008) [2023-12-27 11:42:39,135][20347] Updated weights for policy 0, policy_version 875706 (0.0005) [2023-12-27 11:42:39,195][20347] Updated weights for policy 0, policy_version 875716 (0.0009) [2023-12-27 11:42:39,267][20347] Updated weights for policy 0, policy_version 875726 (0.0010) [2023-12-27 11:42:39,747][20381] Updated weights for policy 1, policy_version 875005 (0.0007) [2023-12-27 11:42:39,805][20381] Updated weights for policy 1, policy_version 875015 (0.0007) [2023-12-27 11:42:39,874][20381] Updated weights for policy 1, policy_version 875025 (0.0008) [2023-12-27 11:42:39,972][20347] Updated weights for policy 0, policy_version 875736 (0.0008) [2023-12-27 11:42:40,037][20347] Updated weights for policy 0, policy_version 875746 (0.0008) [2023-12-27 11:42:40,104][20347] Updated weights for policy 0, policy_version 875756 (0.0009) [2023-12-27 11:42:40,559][20381] Updated weights for policy 1, policy_version 875035 (0.0007) [2023-12-27 11:42:40,615][20381] Updated weights for policy 1, policy_version 875045 (0.0006) [2023-12-27 11:42:40,666][20381] Updated weights for policy 1, policy_version 875055 (0.0009) [2023-12-27 11:42:40,754][20347] Updated weights for policy 0, policy_version 875766 (0.0007) [2023-12-27 11:42:40,805][20347] Updated weights for policy 0, policy_version 875776 (0.0006) [2023-12-27 11:42:40,858][20347] Updated weights for policy 0, policy_version 875786 (0.0005) [2023-12-27 11:42:41,358][20381] Updated weights for policy 1, policy_version 875065 (0.0010) [2023-12-27 11:42:41,419][20381] Updated weights for policy 1, policy_version 875075 (0.0007) [2023-12-27 11:42:41,484][20381] Updated weights for policy 1, policy_version 875085 (0.0008) [2023-12-27 11:42:41,543][20381] Updated weights for policy 1, policy_version 875095 (0.0007) [2023-12-27 11:42:41,585][20347] Updated weights for policy 0, policy_version 875796 (0.0008) [2023-12-27 11:42:41,644][20347] Updated weights for policy 0, policy_version 875806 (0.0009) [2023-12-27 11:42:41,702][20347] Updated weights for policy 0, policy_version 875816 (0.0008) [2023-12-27 11:42:42,245][20381] Updated weights for policy 1, policy_version 875105 (0.0008) [2023-12-27 11:42:42,307][20381] Updated weights for policy 1, policy_version 875115 (0.0007) [2023-12-27 11:42:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 19410.9). Total num frames: 448290816. Throughput: 0: 9909.0, 1: 9830.4. Samples: 448287256. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:42,363][19170] Avg episode reward: [(0, '872.750'), (1, '961.080')] [2023-12-27 11:42:42,369][20381] Updated weights for policy 1, policy_version 875125 (0.0007) [2023-12-27 11:42:42,443][20347] Updated weights for policy 0, policy_version 875826 (0.0009) [2023-12-27 11:42:42,496][20347] Updated weights for policy 0, policy_version 875836 (0.0008) [2023-12-27 11:42:42,552][20347] Updated weights for policy 0, policy_version 875846 (0.0006) [2023-12-27 11:42:42,621][20347] Updated weights for policy 0, policy_version 875856 (0.0008) [2023-12-27 11:42:42,969][20381] Updated weights for policy 1, policy_version 875135 (0.0008) [2023-12-27 11:42:43,026][20381] Updated weights for policy 1, policy_version 875145 (0.0008) [2023-12-27 11:42:43,078][20381] Updated weights for policy 1, policy_version 875155 (0.0008) [2023-12-27 11:42:43,369][20347] Updated weights for policy 0, policy_version 875866 (0.0005) [2023-12-27 11:42:43,423][20347] Updated weights for policy 0, policy_version 875876 (0.0005) [2023-12-27 11:42:43,468][20347] Updated weights for policy 0, policy_version 875886 (0.0007) [2023-12-27 11:42:43,661][20381] Updated weights for policy 1, policy_version 875165 (0.0006) [2023-12-27 11:42:43,707][20381] Updated weights for policy 1, policy_version 875175 (0.0008) [2023-12-27 11:42:43,758][20381] Updated weights for policy 1, policy_version 875185 (0.0008) [2023-12-27 11:42:44,134][20347] Updated weights for policy 0, policy_version 875896 (0.0007) [2023-12-27 11:42:44,184][20347] Updated weights for policy 0, policy_version 875906 (0.0006) [2023-12-27 11:42:44,238][20347] Updated weights for policy 0, policy_version 875916 (0.0007) [2023-12-27 11:42:44,448][20381] Updated weights for policy 1, policy_version 875195 (0.0008) [2023-12-27 11:42:44,501][20381] Updated weights for policy 1, policy_version 875205 (0.0008) [2023-12-27 11:42:44,556][20381] Updated weights for policy 1, policy_version 875215 (0.0007) [2023-12-27 11:42:44,856][20347] Updated weights for policy 0, policy_version 875926 (0.0007) [2023-12-27 11:42:44,914][20347] Updated weights for policy 0, policy_version 875936 (0.0008) [2023-12-27 11:42:44,969][20347] Updated weights for policy 0, policy_version 875946 (0.0007) [2023-12-27 11:42:45,244][20381] Updated weights for policy 1, policy_version 875225 (0.0006) [2023-12-27 11:42:45,302][20381] Updated weights for policy 1, policy_version 875235 (0.0006) [2023-12-27 11:42:45,363][20381] Updated weights for policy 1, policy_version 875245 (0.0007) [2023-12-27 11:42:45,427][20381] Updated weights for policy 1, policy_version 875255 (0.0005) [2023-12-27 11:42:45,692][20347] Updated weights for policy 0, policy_version 875956 (0.0008) [2023-12-27 11:42:45,749][20347] Updated weights for policy 0, policy_version 875966 (0.0006) [2023-12-27 11:42:45,803][20347] Updated weights for policy 0, policy_version 875976 (0.0008) [2023-12-27 11:42:45,968][20381] Updated weights for policy 1, policy_version 875265 (0.0007) [2023-12-27 11:42:46,015][20381] Updated weights for policy 1, policy_version 875275 (0.0008) [2023-12-27 11:42:46,070][20381] Updated weights for policy 1, policy_version 875285 (0.0007) [2023-12-27 11:42:46,483][20347] Updated weights for policy 0, policy_version 875986 (0.0008) [2023-12-27 11:42:46,536][20347] Updated weights for policy 0, policy_version 875996 (0.0009) [2023-12-27 11:42:46,599][20347] Updated weights for policy 0, policy_version 876006 (0.0008) [2023-12-27 11:42:46,662][20347] Updated weights for policy 0, policy_version 876016 (0.0009) [2023-12-27 11:42:46,712][20381] Updated weights for policy 1, policy_version 875295 (0.0006) [2023-12-27 11:42:46,782][20381] Updated weights for policy 1, policy_version 875305 (0.0005) [2023-12-27 11:42:46,841][20381] Updated weights for policy 1, policy_version 875315 (0.0005) [2023-12-27 11:42:47,332][20347] Updated weights for policy 0, policy_version 876026 (0.0006) [2023-12-27 11:42:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19797.3, 300 sec: 19466.4). Total num frames: 448397312. Throughput: 0: 9962.6, 1: 10004.9. Samples: 448410700. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:47,363][19170] Avg episode reward: [(0, '872.630'), (1, '970.820')] [2023-12-27 11:42:47,391][20347] Updated weights for policy 0, policy_version 876036 (0.0008) [2023-12-27 11:42:47,441][20381] Updated weights for policy 1, policy_version 875325 (0.0008) [2023-12-27 11:42:47,444][20347] Updated weights for policy 0, policy_version 876046 (0.0008) [2023-12-27 11:42:47,498][20381] Updated weights for policy 1, policy_version 875335 (0.0010) [2023-12-27 11:42:47,549][20381] Updated weights for policy 1, policy_version 875345 (0.0010) [2023-12-27 11:42:48,057][20347] Updated weights for policy 0, policy_version 876056 (0.0009) [2023-12-27 11:42:48,115][20347] Updated weights for policy 0, policy_version 876066 (0.0010) [2023-12-27 11:42:48,175][20347] Updated weights for policy 0, policy_version 876076 (0.0010) [2023-12-27 11:42:48,222][20381] Updated weights for policy 1, policy_version 875355 (0.0008) [2023-12-27 11:42:48,274][20381] Updated weights for policy 1, policy_version 875365 (0.0006) [2023-12-27 11:42:48,332][20381] Updated weights for policy 1, policy_version 875375 (0.0007) [2023-12-27 11:42:48,842][20347] Updated weights for policy 0, policy_version 876086 (0.0010) [2023-12-27 11:42:48,900][20347] Updated weights for policy 0, policy_version 876096 (0.0010) [2023-12-27 11:42:48,955][20347] Updated weights for policy 0, policy_version 876106 (0.0009) [2023-12-27 11:42:49,006][20381] Updated weights for policy 1, policy_version 875385 (0.0007) [2023-12-27 11:42:49,068][20381] Updated weights for policy 1, policy_version 875395 (0.0006) [2023-12-27 11:42:49,130][20381] Updated weights for policy 1, policy_version 875405 (0.0007) [2023-12-27 11:42:49,192][20381] Updated weights for policy 1, policy_version 875415 (0.0006) [2023-12-27 11:42:49,671][20347] Updated weights for policy 0, policy_version 876116 (0.0010) [2023-12-27 11:42:49,729][20347] Updated weights for policy 0, policy_version 876126 (0.0010) [2023-12-27 11:42:49,788][20347] Updated weights for policy 0, policy_version 876136 (0.0010) [2023-12-27 11:42:49,950][20381] Updated weights for policy 1, policy_version 875425 (0.0007) [2023-12-27 11:42:50,008][20381] Updated weights for policy 1, policy_version 875435 (0.0007) [2023-12-27 11:42:50,064][20381] Updated weights for policy 1, policy_version 875445 (0.0008) [2023-12-27 11:42:50,445][20347] Updated weights for policy 0, policy_version 876146 (0.0011) [2023-12-27 11:42:50,506][20347] Updated weights for policy 0, policy_version 876156 (0.0010) [2023-12-27 11:42:50,563][20347] Updated weights for policy 0, policy_version 876166 (0.0010) [2023-12-27 11:42:50,625][20347] Updated weights for policy 0, policy_version 876176 (0.0010) [2023-12-27 11:42:50,770][20381] Updated weights for policy 1, policy_version 875455 (0.0007) [2023-12-27 11:42:50,837][20381] Updated weights for policy 1, policy_version 875465 (0.0008) [2023-12-27 11:42:50,898][20381] Updated weights for policy 1, policy_version 875475 (0.0007) [2023-12-27 11:42:51,315][20347] Updated weights for policy 0, policy_version 876186 (0.0011) [2023-12-27 11:42:51,379][20347] Updated weights for policy 0, policy_version 876196 (0.0009) [2023-12-27 11:42:51,440][20347] Updated weights for policy 0, policy_version 876206 (0.0008) [2023-12-27 11:42:51,557][20381] Updated weights for policy 1, policy_version 875485 (0.0007) [2023-12-27 11:42:51,617][20381] Updated weights for policy 1, policy_version 875495 (0.0008) [2023-12-27 11:42:51,676][20381] Updated weights for policy 1, policy_version 875505 (0.0007) [2023-12-27 11:42:52,132][20347] Updated weights for policy 0, policy_version 876216 (0.0007) [2023-12-27 11:42:52,183][20347] Updated weights for policy 0, policy_version 876226 (0.0007) [2023-12-27 11:42:52,234][20347] Updated weights for policy 0, policy_version 876236 (0.0007) [2023-12-27 11:42:52,329][20381] Updated weights for policy 1, policy_version 875515 (0.0008) [2023-12-27 11:42:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 19933.8, 300 sec: 19494.2). Total num frames: 448503808. Throughput: 0: 10013.7, 1: 10003.7. Samples: 448472284. Policy #0 lag: (min: 31.0, avg: 33.1, max: 63.0) [2023-12-27 11:42:52,363][19170] Avg episode reward: [(0, '902.020'), (1, '1000.000')] [2023-12-27 11:42:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000876240_224346112.pth... [2023-12-27 11:42:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000875024_224034816.pth [2023-12-27 11:42:52,398][20381] Updated weights for policy 1, policy_version 875525 (0.0008) [2023-12-27 11:42:52,452][20381] Updated weights for policy 1, policy_version 875535 (0.0007) [2023-12-27 11:42:52,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000875544_224165888.pth... [2023-12-27 11:42:52,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000874328_223854592.pth [2023-12-27 11:42:52,952][20347] Updated weights for policy 0, policy_version 876246 (0.0010) [2023-12-27 11:42:53,011][20347] Updated weights for policy 0, policy_version 876256 (0.0010) [2023-12-27 11:42:53,067][20347] Updated weights for policy 0, policy_version 876266 (0.0010) [2023-12-27 11:42:53,154][20381] Updated weights for policy 1, policy_version 875545 (0.0007) [2023-12-27 11:42:53,212][20381] Updated weights for policy 1, policy_version 875555 (0.0005) [2023-12-27 11:42:53,260][20381] Updated weights for policy 1, policy_version 875565 (0.0006) [2023-12-27 11:42:53,305][20381] Updated weights for policy 1, policy_version 875575 (0.0005) [2023-12-27 11:42:53,812][20347] Updated weights for policy 0, policy_version 876276 (0.0009) [2023-12-27 11:42:53,879][20347] Updated weights for policy 0, policy_version 876286 (0.0008) [2023-12-27 11:42:53,934][20347] Updated weights for policy 0, policy_version 876296 (0.0009) [2023-12-27 11:42:53,960][20381] Updated weights for policy 1, policy_version 875585 (0.0010) [2023-12-27 11:42:54,019][20381] Updated weights for policy 1, policy_version 875595 (0.0011) [2023-12-27 11:42:54,071][20381] Updated weights for policy 1, policy_version 875605 (0.0010) [2023-12-27 11:42:54,642][20347] Updated weights for policy 0, policy_version 876306 (0.0010) [2023-12-27 11:42:54,697][20347] Updated weights for policy 0, policy_version 876316 (0.0007) [2023-12-27 11:42:54,736][20381] Updated weights for policy 1, policy_version 875615 (0.0011) [2023-12-27 11:42:54,757][20347] Updated weights for policy 0, policy_version 876326 (0.0006) [2023-12-27 11:42:54,789][20381] Updated weights for policy 1, policy_version 875625 (0.0010) [2023-12-27 11:42:54,813][20347] Updated weights for policy 0, policy_version 876336 (0.0006) [2023-12-27 11:42:54,842][20381] Updated weights for policy 1, policy_version 875635 (0.0008) [2023-12-27 11:42:55,446][20381] Updated weights for policy 1, policy_version 875645 (0.0007) [2023-12-27 11:42:55,505][20381] Updated weights for policy 1, policy_version 875655 (0.0005) [2023-12-27 11:42:55,516][20347] Updated weights for policy 0, policy_version 876346 (0.0008) [2023-12-27 11:42:55,561][20381] Updated weights for policy 1, policy_version 875665 (0.0005) [2023-12-27 11:42:55,566][20347] Updated weights for policy 0, policy_version 876356 (0.0007) [2023-12-27 11:42:55,617][20347] Updated weights for policy 0, policy_version 876366 (0.0009) [2023-12-27 11:42:56,088][20381] Updated weights for policy 1, policy_version 875675 (0.0005) [2023-12-27 11:42:56,145][20381] Updated weights for policy 1, policy_version 875685 (0.0006) [2023-12-27 11:42:56,202][20381] Updated weights for policy 1, policy_version 875695 (0.0007) [2023-12-27 11:42:56,380][20347] Updated weights for policy 0, policy_version 876376 (0.0008) [2023-12-27 11:42:56,431][20347] Updated weights for policy 0, policy_version 876386 (0.0007) [2023-12-27 11:42:56,476][20347] Updated weights for policy 0, policy_version 876396 (0.0009) [2023-12-27 11:42:56,737][20381] Updated weights for policy 1, policy_version 875705 (0.0005) [2023-12-27 11:42:56,788][20381] Updated weights for policy 1, policy_version 875715 (0.0005) [2023-12-27 11:42:56,838][20381] Updated weights for policy 1, policy_version 875725 (0.0005) [2023-12-27 11:42:56,858][20292] KL-divergence is very high: 442.4249 [2023-12-27 11:42:56,895][20381] Updated weights for policy 1, policy_version 875735 (0.0007) [2023-12-27 11:42:57,166][20347] Updated weights for policy 0, policy_version 876406 (0.0009) [2023-12-27 11:42:57,220][20347] Updated weights for policy 0, policy_version 876416 (0.0007) [2023-12-27 11:42:57,264][20347] Updated weights for policy 0, policy_version 876426 (0.0006) [2023-12-27 11:42:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 19933.9, 300 sec: 19522.0). Total num frames: 448610304. Throughput: 0: 10054.9, 1: 10120.5. Samples: 448596064. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:42:57,363][19170] Avg episode reward: [(0, '833.720'), (1, '990.260')] [2023-12-27 11:42:57,548][20381] Updated weights for policy 1, policy_version 875745 (0.0008) [2023-12-27 11:42:57,606][20381] Updated weights for policy 1, policy_version 875755 (0.0008) [2023-12-27 11:42:57,667][20381] Updated weights for policy 1, policy_version 875765 (0.0007) [2023-12-27 11:42:57,938][20347] Updated weights for policy 0, policy_version 876436 (0.0007) [2023-12-27 11:42:57,979][20347] Updated weights for policy 0, policy_version 876446 (0.0007) [2023-12-27 11:42:58,031][20347] Updated weights for policy 0, policy_version 876456 (0.0006) [2023-12-27 11:42:58,340][20381] Updated weights for policy 1, policy_version 875775 (0.0007) [2023-12-27 11:42:58,408][20381] Updated weights for policy 1, policy_version 875785 (0.0009) [2023-12-27 11:42:58,470][20381] Updated weights for policy 1, policy_version 875795 (0.0007) [2023-12-27 11:42:58,730][20347] Updated weights for policy 0, policy_version 876466 (0.0008) [2023-12-27 11:42:58,799][20347] Updated weights for policy 0, policy_version 876476 (0.0008) [2023-12-27 11:42:58,868][20347] Updated weights for policy 0, policy_version 876486 (0.0008) [2023-12-27 11:42:58,940][20347] Updated weights for policy 0, policy_version 876496 (0.0007) [2023-12-27 11:42:59,261][20381] Updated weights for policy 1, policy_version 875805 (0.0008) [2023-12-27 11:42:59,322][20381] Updated weights for policy 1, policy_version 875815 (0.0008) [2023-12-27 11:42:59,390][20381] Updated weights for policy 1, policy_version 875825 (0.0008) [2023-12-27 11:42:59,634][20347] Updated weights for policy 0, policy_version 876506 (0.0006) [2023-12-27 11:42:59,687][20347] Updated weights for policy 0, policy_version 876516 (0.0006) [2023-12-27 11:42:59,742][20347] Updated weights for policy 0, policy_version 876526 (0.0006) [2023-12-27 11:43:00,111][20381] Updated weights for policy 1, policy_version 875835 (0.0007) [2023-12-27 11:43:00,174][20381] Updated weights for policy 1, policy_version 875845 (0.0008) [2023-12-27 11:43:00,236][20381] Updated weights for policy 1, policy_version 875855 (0.0008) [2023-12-27 11:43:00,400][20347] Updated weights for policy 0, policy_version 876536 (0.0007) [2023-12-27 11:43:00,457][20347] Updated weights for policy 0, policy_version 876546 (0.0007) [2023-12-27 11:43:00,512][20347] Updated weights for policy 0, policy_version 876556 (0.0007) [2023-12-27 11:43:00,513][20291] KL-divergence is very high: 428.1640 [2023-12-27 11:43:00,939][20381] Updated weights for policy 1, policy_version 875865 (0.0007) [2023-12-27 11:43:00,993][20381] Updated weights for policy 1, policy_version 875875 (0.0007) [2023-12-27 11:43:01,055][20381] Updated weights for policy 1, policy_version 875885 (0.0008) [2023-12-27 11:43:01,114][20381] Updated weights for policy 1, policy_version 875895 (0.0008) [2023-12-27 11:43:01,145][20291] KL-divergence is very high: 337.9955 [2023-12-27 11:43:01,175][20347] Updated weights for policy 0, policy_version 876566 (0.0007) [2023-12-27 11:43:01,191][20291] KL-divergence is very high: 132.5387 [2023-12-27 11:43:01,227][20347] Updated weights for policy 0, policy_version 876576 (0.0008) [2023-12-27 11:43:01,233][20291] KL-divergence is very high: 170.8347 [2023-12-27 11:43:01,280][20291] KL-divergence is very high: 162.3616 [2023-12-27 11:43:01,286][20347] Updated weights for policy 0, policy_version 876586 (0.0008) [2023-12-27 11:43:01,821][20381] Updated weights for policy 1, policy_version 875905 (0.0008) [2023-12-27 11:43:01,885][20381] Updated weights for policy 1, policy_version 875915 (0.0007) [2023-12-27 11:43:01,944][20381] Updated weights for policy 1, policy_version 875925 (0.0007) [2023-12-27 11:43:02,004][20347] Updated weights for policy 0, policy_version 876596 (0.0008) [2023-12-27 11:43:02,065][20347] Updated weights for policy 0, policy_version 876606 (0.0008) [2023-12-27 11:43:02,125][20347] Updated weights for policy 0, policy_version 876616 (0.0007) [2023-12-27 11:43:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20070.4, 300 sec: 19522.0). Total num frames: 448708608. Throughput: 0: 10091.6, 1: 10197.9. Samples: 448716232. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:02,363][19170] Avg episode reward: [(0, '813.940'), (1, '990.260')] [2023-12-27 11:43:02,578][20381] Updated weights for policy 1, policy_version 875935 (0.0008) [2023-12-27 11:43:02,640][20381] Updated weights for policy 1, policy_version 875945 (0.0008) [2023-12-27 11:43:02,696][20381] Updated weights for policy 1, policy_version 875955 (0.0008) [2023-12-27 11:43:02,842][20347] Updated weights for policy 0, policy_version 876626 (0.0008) [2023-12-27 11:43:02,899][20347] Updated weights for policy 0, policy_version 876636 (0.0008) [2023-12-27 11:43:02,956][20347] Updated weights for policy 0, policy_version 876646 (0.0009) [2023-12-27 11:43:03,012][20347] Updated weights for policy 0, policy_version 876656 (0.0010) [2023-12-27 11:43:03,377][20381] Updated weights for policy 1, policy_version 875965 (0.0008) [2023-12-27 11:43:03,421][20381] Updated weights for policy 1, policy_version 875975 (0.0008) [2023-12-27 11:43:03,480][20381] Updated weights for policy 1, policy_version 875985 (0.0007) [2023-12-27 11:43:03,628][20347] Updated weights for policy 0, policy_version 876666 (0.0005) [2023-12-27 11:43:03,680][20347] Updated weights for policy 0, policy_version 876676 (0.0010) [2023-12-27 11:43:03,728][20347] Updated weights for policy 0, policy_version 876686 (0.0010) [2023-12-27 11:43:04,246][20381] Updated weights for policy 1, policy_version 875995 (0.0007) [2023-12-27 11:43:04,307][20381] Updated weights for policy 1, policy_version 876005 (0.0007) [2023-12-27 11:43:04,363][20381] Updated weights for policy 1, policy_version 876015 (0.0006) [2023-12-27 11:43:04,396][20347] Updated weights for policy 0, policy_version 876696 (0.0008) [2023-12-27 11:43:04,457][20347] Updated weights for policy 0, policy_version 876706 (0.0007) [2023-12-27 11:43:04,507][20347] Updated weights for policy 0, policy_version 876716 (0.0008) [2023-12-27 11:43:04,955][20381] Updated weights for policy 1, policy_version 876025 (0.0006) [2023-12-27 11:43:05,013][20381] Updated weights for policy 1, policy_version 876035 (0.0008) [2023-12-27 11:43:05,061][20381] Updated weights for policy 1, policy_version 876045 (0.0007) [2023-12-27 11:43:05,109][20381] Updated weights for policy 1, policy_version 876055 (0.0008) [2023-12-27 11:43:05,228][20347] Updated weights for policy 0, policy_version 876726 (0.0007) [2023-12-27 11:43:05,280][20347] Updated weights for policy 0, policy_version 876736 (0.0007) [2023-12-27 11:43:05,324][20347] Updated weights for policy 0, policy_version 876746 (0.0007) [2023-12-27 11:43:05,858][20381] Updated weights for policy 1, policy_version 876065 (0.0007) [2023-12-27 11:43:05,906][20381] Updated weights for policy 1, policy_version 876075 (0.0007) [2023-12-27 11:43:05,928][20347] Updated weights for policy 0, policy_version 876756 (0.0007) [2023-12-27 11:43:05,959][20381] Updated weights for policy 1, policy_version 876085 (0.0006) [2023-12-27 11:43:05,982][20347] Updated weights for policy 0, policy_version 876766 (0.0006) [2023-12-27 11:43:06,042][20347] Updated weights for policy 0, policy_version 876776 (0.0005) [2023-12-27 11:43:06,681][20381] Updated weights for policy 1, policy_version 876095 (0.0008) [2023-12-27 11:43:06,710][20347] Updated weights for policy 0, policy_version 876786 (0.0006) [2023-12-27 11:43:06,733][20381] Updated weights for policy 1, policy_version 876105 (0.0006) [2023-12-27 11:43:06,768][20347] Updated weights for policy 0, policy_version 876796 (0.0010) [2023-12-27 11:43:06,792][20381] Updated weights for policy 1, policy_version 876115 (0.0006) [2023-12-27 11:43:06,813][20347] Updated weights for policy 0, policy_version 876806 (0.0010) [2023-12-27 11:43:06,859][20347] Updated weights for policy 0, policy_version 876816 (0.0010) [2023-12-27 11:43:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 19521.9). Total num frames: 448806912. Throughput: 0: 10139.7, 1: 10221.4. Samples: 448777668. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:07,363][19170] Avg episode reward: [(0, '941.130'), (1, '1000.000')] [2023-12-27 11:43:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000876120_224313344.pth... [2023-12-27 11:43:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000876816_224493568.pth... [2023-12-27 11:43:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000874936_224010240.pth [2023-12-27 11:43:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000875632_224190464.pth [2023-12-27 11:43:07,476][20381] Updated weights for policy 1, policy_version 876125 (0.0007) [2023-12-27 11:43:07,516][20347] Updated weights for policy 0, policy_version 876826 (0.0008) [2023-12-27 11:43:07,530][20381] Updated weights for policy 1, policy_version 876135 (0.0006) [2023-12-27 11:43:07,570][20347] Updated weights for policy 0, policy_version 876836 (0.0008) [2023-12-27 11:43:07,584][20381] Updated weights for policy 1, policy_version 876145 (0.0006) [2023-12-27 11:43:07,621][20347] Updated weights for policy 0, policy_version 876846 (0.0006) [2023-12-27 11:43:08,203][20347] Updated weights for policy 0, policy_version 876856 (0.0006) [2023-12-27 11:43:08,257][20347] Updated weights for policy 0, policy_version 876866 (0.0010) [2023-12-27 11:43:08,304][20347] Updated weights for policy 0, policy_version 876876 (0.0009) [2023-12-27 11:43:08,335][20381] Updated weights for policy 1, policy_version 876155 (0.0007) [2023-12-27 11:43:08,392][20381] Updated weights for policy 1, policy_version 876165 (0.0009) [2023-12-27 11:43:08,455][20381] Updated weights for policy 1, policy_version 876175 (0.0007) [2023-12-27 11:43:08,992][20347] Updated weights for policy 0, policy_version 876886 (0.0008) [2023-12-27 11:43:09,058][20347] Updated weights for policy 0, policy_version 876896 (0.0007) [2023-12-27 11:43:09,118][20347] Updated weights for policy 0, policy_version 876906 (0.0011) [2023-12-27 11:43:09,182][20381] Updated weights for policy 1, policy_version 876185 (0.0007) [2023-12-27 11:43:09,242][20381] Updated weights for policy 1, policy_version 876195 (0.0008) [2023-12-27 11:43:09,309][20381] Updated weights for policy 1, policy_version 876205 (0.0007) [2023-12-27 11:43:09,374][20381] Updated weights for policy 1, policy_version 876215 (0.0008) [2023-12-27 11:43:09,891][20347] Updated weights for policy 0, policy_version 876916 (0.0009) [2023-12-27 11:43:09,957][20347] Updated weights for policy 0, policy_version 876926 (0.0007) [2023-12-27 11:43:10,025][20347] Updated weights for policy 0, policy_version 876936 (0.0006) [2023-12-27 11:43:10,062][20381] Updated weights for policy 1, policy_version 876225 (0.0007) [2023-12-27 11:43:10,126][20381] Updated weights for policy 1, policy_version 876235 (0.0007) [2023-12-27 11:43:10,194][20381] Updated weights for policy 1, policy_version 876245 (0.0007) [2023-12-27 11:43:10,685][20347] Updated weights for policy 0, policy_version 876946 (0.0006) [2023-12-27 11:43:10,743][20347] Updated weights for policy 0, policy_version 876956 (0.0008) [2023-12-27 11:43:10,798][20347] Updated weights for policy 0, policy_version 876966 (0.0007) [2023-12-27 11:43:10,852][20347] Updated weights for policy 0, policy_version 876976 (0.0007) [2023-12-27 11:43:10,875][20381] Updated weights for policy 1, policy_version 876255 (0.0008) [2023-12-27 11:43:10,924][20381] Updated weights for policy 1, policy_version 876265 (0.0008) [2023-12-27 11:43:10,988][20381] Updated weights for policy 1, policy_version 876275 (0.0008) [2023-12-27 11:43:11,545][20347] Updated weights for policy 0, policy_version 876986 (0.0008) [2023-12-27 11:43:11,607][20347] Updated weights for policy 0, policy_version 876996 (0.0008) [2023-12-27 11:43:11,675][20347] Updated weights for policy 0, policy_version 877006 (0.0008) [2023-12-27 11:43:11,805][20381] Updated weights for policy 1, policy_version 876285 (0.0008) [2023-12-27 11:43:11,861][20381] Updated weights for policy 1, policy_version 876295 (0.0008) [2023-12-27 11:43:11,915][20381] Updated weights for policy 1, policy_version 876305 (0.0007) [2023-12-27 11:43:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 19521.9). Total num frames: 448905216. Throughput: 0: 10240.8, 1: 10249.2. Samples: 448898944. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:12,363][19170] Avg episode reward: [(0, '931.670'), (1, '980.540')] [2023-12-27 11:43:12,400][20347] Updated weights for policy 0, policy_version 877016 (0.0008) [2023-12-27 11:43:12,456][20347] Updated weights for policy 0, policy_version 877026 (0.0009) [2023-12-27 11:43:12,511][20347] Updated weights for policy 0, policy_version 877036 (0.0008) [2023-12-27 11:43:12,576][20381] Updated weights for policy 1, policy_version 876315 (0.0007) [2023-12-27 11:43:12,638][20381] Updated weights for policy 1, policy_version 876325 (0.0007) [2023-12-27 11:43:12,695][20381] Updated weights for policy 1, policy_version 876335 (0.0008) [2023-12-27 11:43:13,244][20347] Updated weights for policy 0, policy_version 877046 (0.0009) [2023-12-27 11:43:13,276][20381] Updated weights for policy 1, policy_version 876345 (0.0008) [2023-12-27 11:43:13,299][20347] Updated weights for policy 0, policy_version 877056 (0.0010) [2023-12-27 11:43:13,326][20381] Updated weights for policy 1, policy_version 876355 (0.0009) [2023-12-27 11:43:13,357][20347] Updated weights for policy 0, policy_version 877066 (0.0010) [2023-12-27 11:43:13,380][20381] Updated weights for policy 1, policy_version 876365 (0.0008) [2023-12-27 11:43:13,428][20381] Updated weights for policy 1, policy_version 876375 (0.0008) [2023-12-27 11:43:13,971][20381] Updated weights for policy 1, policy_version 876385 (0.0005) [2023-12-27 11:43:14,015][20347] Updated weights for policy 0, policy_version 877076 (0.0010) [2023-12-27 11:43:14,023][20381] Updated weights for policy 1, policy_version 876395 (0.0007) [2023-12-27 11:43:14,071][20347] Updated weights for policy 0, policy_version 877086 (0.0010) [2023-12-27 11:43:14,072][20381] Updated weights for policy 1, policy_version 876405 (0.0011) [2023-12-27 11:43:14,124][20347] Updated weights for policy 0, policy_version 877096 (0.0010) [2023-12-27 11:43:14,719][20381] Updated weights for policy 1, policy_version 876415 (0.0007) [2023-12-27 11:43:14,778][20347] Updated weights for policy 0, policy_version 877106 (0.0010) [2023-12-27 11:43:14,783][20381] Updated weights for policy 1, policy_version 876425 (0.0009) [2023-12-27 11:43:14,838][20347] Updated weights for policy 0, policy_version 877116 (0.0010) [2023-12-27 11:43:14,841][20381] Updated weights for policy 1, policy_version 876435 (0.0011) [2023-12-27 11:43:14,898][20347] Updated weights for policy 0, policy_version 877126 (0.0007) [2023-12-27 11:43:14,962][20347] Updated weights for policy 0, policy_version 877136 (0.0008) [2023-12-27 11:43:15,526][20381] Updated weights for policy 1, policy_version 876445 (0.0011) [2023-12-27 11:43:15,588][20381] Updated weights for policy 1, policy_version 876455 (0.0011) [2023-12-27 11:43:15,645][20381] Updated weights for policy 1, policy_version 876465 (0.0011) [2023-12-27 11:43:15,646][20347] Updated weights for policy 0, policy_version 877146 (0.0010) [2023-12-27 11:43:15,701][20347] Updated weights for policy 0, policy_version 877156 (0.0010) [2023-12-27 11:43:15,762][20347] Updated weights for policy 0, policy_version 877166 (0.0010) [2023-12-27 11:43:16,314][20381] Updated weights for policy 1, policy_version 876475 (0.0011) [2023-12-27 11:43:16,363][20347] Updated weights for policy 0, policy_version 877176 (0.0006) [2023-12-27 11:43:16,368][20381] Updated weights for policy 1, policy_version 876485 (0.0011) [2023-12-27 11:43:16,418][20381] Updated weights for policy 1, policy_version 876495 (0.0010) [2023-12-27 11:43:16,421][20347] Updated weights for policy 0, policy_version 877186 (0.0005) [2023-12-27 11:43:16,475][20347] Updated weights for policy 0, policy_version 877196 (0.0005) [2023-12-27 11:43:17,003][20381] Updated weights for policy 1, policy_version 876505 (0.0010) [2023-12-27 11:43:17,040][20347] Updated weights for policy 0, policy_version 877206 (0.0006) [2023-12-27 11:43:17,060][20381] Updated weights for policy 1, policy_version 876515 (0.0007) [2023-12-27 11:43:17,089][20347] Updated weights for policy 0, policy_version 877216 (0.0006) [2023-12-27 11:43:17,118][20381] Updated weights for policy 1, policy_version 876525 (0.0008) [2023-12-27 11:43:17,150][20347] Updated weights for policy 0, policy_version 877226 (0.0005) [2023-12-27 11:43:17,171][20381] Updated weights for policy 1, policy_version 876535 (0.0008) [2023-12-27 11:43:17,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 19577.5). Total num frames: 449019904. Throughput: 0: 10245.5, 1: 10353.9. Samples: 449025652. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:17,364][19170] Avg episode reward: [(0, '912.500'), (1, '970.780')] [2023-12-27 11:43:17,721][20347] Updated weights for policy 0, policy_version 877236 (0.0007) [2023-12-27 11:43:17,781][20347] Updated weights for policy 0, policy_version 877246 (0.0005) [2023-12-27 11:43:17,842][20347] Updated weights for policy 0, policy_version 877256 (0.0005) [2023-12-27 11:43:17,903][20381] Updated weights for policy 1, policy_version 876545 (0.0007) [2023-12-27 11:43:17,956][20381] Updated weights for policy 1, policy_version 876555 (0.0007) [2023-12-27 11:43:18,015][20381] Updated weights for policy 1, policy_version 876565 (0.0010) [2023-12-27 11:43:18,401][20347] Updated weights for policy 0, policy_version 877266 (0.0006) [2023-12-27 11:43:18,459][20347] Updated weights for policy 0, policy_version 877276 (0.0010) [2023-12-27 11:43:18,521][20347] Updated weights for policy 0, policy_version 877286 (0.0011) [2023-12-27 11:43:18,576][20347] Updated weights for policy 0, policy_version 877296 (0.0010) [2023-12-27 11:43:18,747][20381] Updated weights for policy 1, policy_version 876575 (0.0007) [2023-12-27 11:43:18,803][20381] Updated weights for policy 1, policy_version 876585 (0.0008) [2023-12-27 11:43:18,858][20381] Updated weights for policy 1, policy_version 876595 (0.0008) [2023-12-27 11:43:19,261][20347] Updated weights for policy 0, policy_version 877306 (0.0009) [2023-12-27 11:43:19,322][20347] Updated weights for policy 0, policy_version 877316 (0.0008) [2023-12-27 11:43:19,386][20347] Updated weights for policy 0, policy_version 877326 (0.0008) [2023-12-27 11:43:19,500][20381] Updated weights for policy 1, policy_version 876605 (0.0007) [2023-12-27 11:43:19,558][20381] Updated weights for policy 1, policy_version 876615 (0.0008) [2023-12-27 11:43:19,615][20381] Updated weights for policy 1, policy_version 876625 (0.0007) [2023-12-27 11:43:20,130][20347] Updated weights for policy 0, policy_version 877336 (0.0010) [2023-12-27 11:43:20,194][20347] Updated weights for policy 0, policy_version 877346 (0.0011) [2023-12-27 11:43:20,248][20347] Updated weights for policy 0, policy_version 877356 (0.0008) [2023-12-27 11:43:20,298][20381] Updated weights for policy 1, policy_version 876635 (0.0006) [2023-12-27 11:43:20,358][20381] Updated weights for policy 1, policy_version 876645 (0.0006) [2023-12-27 11:43:20,411][20381] Updated weights for policy 1, policy_version 876655 (0.0006) [2023-12-27 11:43:20,887][20347] Updated weights for policy 0, policy_version 877366 (0.0006) [2023-12-27 11:43:20,950][20347] Updated weights for policy 0, policy_version 877376 (0.0005) [2023-12-27 11:43:21,010][20347] Updated weights for policy 0, policy_version 877386 (0.0009) [2023-12-27 11:43:21,029][20381] Updated weights for policy 1, policy_version 876665 (0.0006) [2023-12-27 11:43:21,092][20381] Updated weights for policy 1, policy_version 876675 (0.0007) [2023-12-27 11:43:21,150][20381] Updated weights for policy 1, policy_version 876685 (0.0007) [2023-12-27 11:43:21,211][20381] Updated weights for policy 1, policy_version 876695 (0.0006) [2023-12-27 11:43:21,685][20347] Updated weights for policy 0, policy_version 877396 (0.0010) [2023-12-27 11:43:21,748][20347] Updated weights for policy 0, policy_version 877406 (0.0008) [2023-12-27 11:43:21,811][20347] Updated weights for policy 0, policy_version 877416 (0.0008) [2023-12-27 11:43:21,884][20381] Updated weights for policy 1, policy_version 876705 (0.0007) [2023-12-27 11:43:21,947][20381] Updated weights for policy 1, policy_version 876715 (0.0009) [2023-12-27 11:43:22,008][20381] Updated weights for policy 1, policy_version 876725 (0.0007) [2023-12-27 11:43:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 19577.5). Total num frames: 449118208. Throughput: 0: 10279.6, 1: 10318.2. Samples: 449087996. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:22,363][19170] Avg episode reward: [(0, '922.650'), (1, '990.240')] [2023-12-27 11:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000876728_224468992.pth... [2023-12-27 11:43:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000877424_224649216.pth... [2023-12-27 11:43:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000875544_224165888.pth [2023-12-27 11:43:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000876240_224346112.pth [2023-12-27 11:43:22,458][20347] Updated weights for policy 0, policy_version 877426 (0.0006) [2023-12-27 11:43:22,517][20347] Updated weights for policy 0, policy_version 877436 (0.0008) [2023-12-27 11:43:22,556][20291] KL-divergence is very high: 120.9035 [2023-12-27 11:43:22,580][20347] Updated weights for policy 0, policy_version 877446 (0.0006) [2023-12-27 11:43:22,640][20347] Updated weights for policy 0, policy_version 877456 (0.0006) [2023-12-27 11:43:22,681][20381] Updated weights for policy 1, policy_version 876735 (0.0008) [2023-12-27 11:43:22,741][20381] Updated weights for policy 1, policy_version 876745 (0.0008) [2023-12-27 11:43:22,802][20381] Updated weights for policy 1, policy_version 876755 (0.0008) [2023-12-27 11:43:23,341][20347] Updated weights for policy 0, policy_version 877466 (0.0006) [2023-12-27 11:43:23,395][20347] Updated weights for policy 0, policy_version 877476 (0.0009) [2023-12-27 11:43:23,420][20381] Updated weights for policy 1, policy_version 876765 (0.0006) [2023-12-27 11:43:23,441][20347] Updated weights for policy 0, policy_version 877486 (0.0010) [2023-12-27 11:43:23,475][20381] Updated weights for policy 1, policy_version 876775 (0.0006) [2023-12-27 11:43:23,525][20381] Updated weights for policy 1, policy_version 876785 (0.0007) [2023-12-27 11:43:24,130][20347] Updated weights for policy 0, policy_version 877496 (0.0010) [2023-12-27 11:43:24,193][20347] Updated weights for policy 0, policy_version 877506 (0.0008) [2023-12-27 11:43:24,196][20381] Updated weights for policy 1, policy_version 876795 (0.0008) [2023-12-27 11:43:24,251][20347] Updated weights for policy 0, policy_version 877516 (0.0008) [2023-12-27 11:43:24,256][20381] Updated weights for policy 1, policy_version 876805 (0.0006) [2023-12-27 11:43:24,314][20381] Updated weights for policy 1, policy_version 876815 (0.0005) [2023-12-27 11:43:24,914][20347] Updated weights for policy 0, policy_version 877526 (0.0006) [2023-12-27 11:43:24,930][20381] Updated weights for policy 1, policy_version 876825 (0.0007) [2023-12-27 11:43:24,969][20347] Updated weights for policy 0, policy_version 877536 (0.0005) [2023-12-27 11:43:24,986][20381] Updated weights for policy 1, policy_version 876835 (0.0007) [2023-12-27 11:43:25,035][20347] Updated weights for policy 0, policy_version 877546 (0.0006) [2023-12-27 11:43:25,044][20381] Updated weights for policy 1, policy_version 876845 (0.0007) [2023-12-27 11:43:25,103][20381] Updated weights for policy 1, policy_version 876855 (0.0006) [2023-12-27 11:43:25,690][20347] Updated weights for policy 0, policy_version 877556 (0.0006) [2023-12-27 11:43:25,696][20381] Updated weights for policy 1, policy_version 876865 (0.0010) [2023-12-27 11:43:25,741][20347] Updated weights for policy 0, policy_version 877566 (0.0005) [2023-12-27 11:43:25,751][20381] Updated weights for policy 1, policy_version 876875 (0.0006) [2023-12-27 11:43:25,800][20347] Updated weights for policy 0, policy_version 877576 (0.0005) [2023-12-27 11:43:25,802][20381] Updated weights for policy 1, policy_version 876885 (0.0005) [2023-12-27 11:43:26,365][20381] Updated weights for policy 1, policy_version 876895 (0.0009) [2023-12-27 11:43:26,416][20381] Updated weights for policy 1, policy_version 876905 (0.0010) [2023-12-27 11:43:26,453][20347] Updated weights for policy 0, policy_version 877586 (0.0007) [2023-12-27 11:43:26,466][20381] Updated weights for policy 1, policy_version 876915 (0.0007) [2023-12-27 11:43:26,502][20347] Updated weights for policy 0, policy_version 877596 (0.0005) [2023-12-27 11:43:26,547][20347] Updated weights for policy 0, policy_version 877606 (0.0008) [2023-12-27 11:43:26,600][20347] Updated weights for policy 0, policy_version 877616 (0.0007) [2023-12-27 11:43:27,142][20381] Updated weights for policy 1, policy_version 876925 (0.0008) [2023-12-27 11:43:27,190][20381] Updated weights for policy 1, policy_version 876935 (0.0010) [2023-12-27 11:43:27,201][20347] Updated weights for policy 0, policy_version 877626 (0.0005) [2023-12-27 11:43:27,240][20381] Updated weights for policy 1, policy_version 876945 (0.0010) [2023-12-27 11:43:27,251][20347] Updated weights for policy 0, policy_version 877636 (0.0005) [2023-12-27 11:43:27,304][20347] Updated weights for policy 0, policy_version 877646 (0.0006) [2023-12-27 11:43:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 19633.0). Total num frames: 449232896. Throughput: 0: 10226.5, 1: 10368.1. Samples: 449214012. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:27,363][19170] Avg episode reward: [(0, '912.350'), (1, '1000.000')] [2023-12-27 11:43:27,866][20347] Updated weights for policy 0, policy_version 877656 (0.0006) [2023-12-27 11:43:27,898][20381] Updated weights for policy 1, policy_version 876955 (0.0010) [2023-12-27 11:43:27,917][20347] Updated weights for policy 0, policy_version 877666 (0.0005) [2023-12-27 11:43:27,952][20381] Updated weights for policy 1, policy_version 876965 (0.0010) [2023-12-27 11:43:27,970][20347] Updated weights for policy 0, policy_version 877676 (0.0005) [2023-12-27 11:43:28,004][20381] Updated weights for policy 1, policy_version 876975 (0.0010) [2023-12-27 11:43:28,519][20347] Updated weights for policy 0, policy_version 877686 (0.0007) [2023-12-27 11:43:28,571][20347] Updated weights for policy 0, policy_version 877696 (0.0008) [2023-12-27 11:43:28,623][20347] Updated weights for policy 0, policy_version 877706 (0.0007) [2023-12-27 11:43:28,667][20381] Updated weights for policy 1, policy_version 876985 (0.0009) [2023-12-27 11:43:28,714][20381] Updated weights for policy 1, policy_version 876995 (0.0009) [2023-12-27 11:43:28,776][20381] Updated weights for policy 1, policy_version 877005 (0.0009) [2023-12-27 11:43:28,837][20381] Updated weights for policy 1, policy_version 877015 (0.0008) [2023-12-27 11:43:29,373][20347] Updated weights for policy 0, policy_version 877716 (0.0008) [2023-12-27 11:43:29,420][20347] Updated weights for policy 0, policy_version 877726 (0.0008) [2023-12-27 11:43:29,469][20347] Updated weights for policy 0, policy_version 877736 (0.0008) [2023-12-27 11:43:29,515][20381] Updated weights for policy 1, policy_version 877025 (0.0007) [2023-12-27 11:43:29,568][20381] Updated weights for policy 1, policy_version 877035 (0.0005) [2023-12-27 11:43:29,620][20381] Updated weights for policy 1, policy_version 877045 (0.0005) [2023-12-27 11:43:30,206][20347] Updated weights for policy 0, policy_version 877746 (0.0008) [2023-12-27 11:43:30,261][20347] Updated weights for policy 0, policy_version 877756 (0.0007) [2023-12-27 11:43:30,296][20381] Updated weights for policy 1, policy_version 877055 (0.0006) [2023-12-27 11:43:30,313][20347] Updated weights for policy 0, policy_version 877766 (0.0005) [2023-12-27 11:43:30,351][20381] Updated weights for policy 1, policy_version 877065 (0.0006) [2023-12-27 11:43:30,367][20347] Updated weights for policy 0, policy_version 877776 (0.0005) [2023-12-27 11:43:30,409][20381] Updated weights for policy 1, policy_version 877075 (0.0008) [2023-12-27 11:43:31,044][20347] Updated weights for policy 0, policy_version 877786 (0.0008) [2023-12-27 11:43:31,080][20381] Updated weights for policy 1, policy_version 877085 (0.0007) [2023-12-27 11:43:31,108][20347] Updated weights for policy 0, policy_version 877796 (0.0007) [2023-12-27 11:43:31,147][20381] Updated weights for policy 1, policy_version 877095 (0.0007) [2023-12-27 11:43:31,169][20347] Updated weights for policy 0, policy_version 877806 (0.0007) [2023-12-27 11:43:31,205][20381] Updated weights for policy 1, policy_version 877105 (0.0006) [2023-12-27 11:43:31,860][20381] Updated weights for policy 1, policy_version 877115 (0.0007) [2023-12-27 11:43:31,918][20381] Updated weights for policy 1, policy_version 877125 (0.0008) [2023-12-27 11:43:31,920][20347] Updated weights for policy 0, policy_version 877816 (0.0006) [2023-12-27 11:43:31,974][20381] Updated weights for policy 1, policy_version 877135 (0.0006) [2023-12-27 11:43:31,980][20347] Updated weights for policy 0, policy_version 877826 (0.0007) [2023-12-27 11:43:32,038][20347] Updated weights for policy 0, policy_version 877836 (0.0007) [2023-12-27 11:43:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 19633.0). Total num frames: 449331200. Throughput: 0: 10308.3, 1: 10336.8. Samples: 449339728. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:32,363][19170] Avg episode reward: [(0, '922.000'), (1, '1000.000')] [2023-12-27 11:43:32,604][20381] Updated weights for policy 1, policy_version 877145 (0.0006) [2023-12-27 11:43:32,662][20381] Updated weights for policy 1, policy_version 877155 (0.0007) [2023-12-27 11:43:32,677][20347] Updated weights for policy 0, policy_version 877846 (0.0005) [2023-12-27 11:43:32,720][20381] Updated weights for policy 1, policy_version 877165 (0.0007) [2023-12-27 11:43:32,725][20347] Updated weights for policy 0, policy_version 877856 (0.0007) [2023-12-27 11:43:32,773][20381] Updated weights for policy 1, policy_version 877175 (0.0005) [2023-12-27 11:43:32,778][20347] Updated weights for policy 0, policy_version 877866 (0.0007) [2023-12-27 11:43:33,434][20381] Updated weights for policy 1, policy_version 877185 (0.0006) [2023-12-27 11:43:33,434][20347] Updated weights for policy 0, policy_version 877876 (0.0008) [2023-12-27 11:43:33,489][20347] Updated weights for policy 0, policy_version 877886 (0.0010) [2023-12-27 11:43:33,490][20381] Updated weights for policy 1, policy_version 877195 (0.0007) [2023-12-27 11:43:33,542][20347] Updated weights for policy 0, policy_version 877896 (0.0009) [2023-12-27 11:43:33,544][20381] Updated weights for policy 1, policy_version 877205 (0.0008) [2023-12-27 11:43:34,102][20381] Updated weights for policy 1, policy_version 877215 (0.0008) [2023-12-27 11:43:34,105][20347] Updated weights for policy 0, policy_version 877906 (0.0010) [2023-12-27 11:43:34,164][20347] Updated weights for policy 0, policy_version 877916 (0.0010) [2023-12-27 11:43:34,166][20381] Updated weights for policy 1, policy_version 877225 (0.0007) [2023-12-27 11:43:34,222][20381] Updated weights for policy 1, policy_version 877235 (0.0008) [2023-12-27 11:43:34,226][20347] Updated weights for policy 0, policy_version 877926 (0.0009) [2023-12-27 11:43:34,294][20347] Updated weights for policy 0, policy_version 877936 (0.0009) [2023-12-27 11:43:34,912][20347] Updated weights for policy 0, policy_version 877946 (0.0010) [2023-12-27 11:43:34,948][20381] Updated weights for policy 1, policy_version 877245 (0.0006) [2023-12-27 11:43:34,958][20347] Updated weights for policy 0, policy_version 877956 (0.0010) [2023-12-27 11:43:35,001][20381] Updated weights for policy 1, policy_version 877255 (0.0005) [2023-12-27 11:43:35,004][20347] Updated weights for policy 0, policy_version 877966 (0.0010) [2023-12-27 11:43:35,056][20381] Updated weights for policy 1, policy_version 877265 (0.0007) [2023-12-27 11:43:35,594][20347] Updated weights for policy 0, policy_version 877976 (0.0006) [2023-12-27 11:43:35,595][20291] KL-divergence is very high: 106.2837 [2023-12-27 11:43:35,605][20291] KL-divergence is very high: 205.4686 [2023-12-27 11:43:35,624][20291] KL-divergence is very high: 171.9130 [2023-12-27 11:43:35,645][20347] Updated weights for policy 0, policy_version 877986 (0.0007) [2023-12-27 11:43:35,694][20347] Updated weights for policy 0, policy_version 877996 (0.0010) [2023-12-27 11:43:35,696][20381] Updated weights for policy 1, policy_version 877275 (0.0007) [2023-12-27 11:43:35,746][20381] Updated weights for policy 1, policy_version 877285 (0.0006) [2023-12-27 11:43:35,804][20381] Updated weights for policy 1, policy_version 877295 (0.0007) [2023-12-27 11:43:36,342][20347] Updated weights for policy 0, policy_version 878006 (0.0008) [2023-12-27 11:43:36,402][20347] Updated weights for policy 0, policy_version 878016 (0.0008) [2023-12-27 11:43:36,461][20347] Updated weights for policy 0, policy_version 878026 (0.0008) [2023-12-27 11:43:36,528][20381] Updated weights for policy 1, policy_version 877305 (0.0007) [2023-12-27 11:43:36,587][20381] Updated weights for policy 1, policy_version 877315 (0.0011) [2023-12-27 11:43:36,641][20381] Updated weights for policy 1, policy_version 877325 (0.0011) [2023-12-27 11:43:36,703][20381] Updated weights for policy 1, policy_version 877335 (0.0011) [2023-12-27 11:43:37,148][20347] Updated weights for policy 0, policy_version 878036 (0.0006) [2023-12-27 11:43:37,200][20347] Updated weights for policy 0, policy_version 878046 (0.0007) [2023-12-27 11:43:37,254][20347] Updated weights for policy 0, policy_version 878056 (0.0008) [2023-12-27 11:43:37,315][20381] Updated weights for policy 1, policy_version 877345 (0.0010) [2023-12-27 11:43:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 19716.3). Total num frames: 449437696. Throughput: 0: 10331.0, 1: 10389.4. Samples: 449404696. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:37,363][19170] Avg episode reward: [(0, '883.540'), (1, '1000.000')] [2023-12-27 11:43:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000878064_224813056.pth... [2023-12-27 11:43:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000876816_224493568.pth [2023-12-27 11:43:37,379][20381] Updated weights for policy 1, policy_version 877355 (0.0011) [2023-12-27 11:43:37,432][20381] Updated weights for policy 1, policy_version 877365 (0.0010) [2023-12-27 11:43:37,448][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000877368_224632832.pth... [2023-12-27 11:43:37,451][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000876120_224313344.pth [2023-12-27 11:43:37,926][20347] Updated weights for policy 0, policy_version 878066 (0.0008) [2023-12-27 11:43:37,994][20347] Updated weights for policy 0, policy_version 878076 (0.0010) [2023-12-27 11:43:38,040][20381] Updated weights for policy 1, policy_version 877375 (0.0010) [2023-12-27 11:43:38,053][20347] Updated weights for policy 0, policy_version 878086 (0.0010) [2023-12-27 11:43:38,096][20381] Updated weights for policy 1, policy_version 877385 (0.0011) [2023-12-27 11:43:38,114][20347] Updated weights for policy 0, policy_version 878096 (0.0010) [2023-12-27 11:43:38,151][20381] Updated weights for policy 1, policy_version 877395 (0.0010) [2023-12-27 11:43:38,783][20347] Updated weights for policy 0, policy_version 878106 (0.0010) [2023-12-27 11:43:38,818][20381] Updated weights for policy 1, policy_version 877405 (0.0011) [2023-12-27 11:43:38,845][20347] Updated weights for policy 0, policy_version 878116 (0.0011) [2023-12-27 11:43:38,872][20381] Updated weights for policy 1, policy_version 877415 (0.0010) [2023-12-27 11:43:38,894][20347] Updated weights for policy 0, policy_version 878126 (0.0010) [2023-12-27 11:43:38,931][20381] Updated weights for policy 1, policy_version 877425 (0.0011) [2023-12-27 11:43:39,639][20347] Updated weights for policy 0, policy_version 878136 (0.0008) [2023-12-27 11:43:39,699][20347] Updated weights for policy 0, policy_version 878146 (0.0008) [2023-12-27 11:43:39,708][20381] Updated weights for policy 1, policy_version 877435 (0.0009) [2023-12-27 11:43:39,760][20347] Updated weights for policy 0, policy_version 878156 (0.0007) [2023-12-27 11:43:39,763][20381] Updated weights for policy 1, policy_version 877445 (0.0005) [2023-12-27 11:43:39,819][20381] Updated weights for policy 1, policy_version 877455 (0.0006) [2023-12-27 11:43:40,451][20381] Updated weights for policy 1, policy_version 877465 (0.0007) [2023-12-27 11:43:40,464][20347] Updated weights for policy 0, policy_version 878166 (0.0008) [2023-12-27 11:43:40,509][20381] Updated weights for policy 1, policy_version 877475 (0.0006) [2023-12-27 11:43:40,527][20347] Updated weights for policy 0, policy_version 878176 (0.0007) [2023-12-27 11:43:40,563][20381] Updated weights for policy 1, policy_version 877485 (0.0005) [2023-12-27 11:43:40,584][20347] Updated weights for policy 0, policy_version 878186 (0.0007) [2023-12-27 11:43:40,624][20381] Updated weights for policy 1, policy_version 877495 (0.0006) [2023-12-27 11:43:41,282][20347] Updated weights for policy 0, policy_version 878196 (0.0006) [2023-12-27 11:43:41,346][20381] Updated weights for policy 1, policy_version 877505 (0.0008) [2023-12-27 11:43:41,347][20347] Updated weights for policy 0, policy_version 878206 (0.0007) [2023-12-27 11:43:41,418][20347] Updated weights for policy 0, policy_version 878216 (0.0008) [2023-12-27 11:43:41,420][20381] Updated weights for policy 1, policy_version 877515 (0.0008) [2023-12-27 11:43:41,479][20381] Updated weights for policy 1, policy_version 877525 (0.0006) [2023-12-27 11:43:42,122][20347] Updated weights for policy 0, policy_version 878226 (0.0007) [2023-12-27 11:43:42,183][20347] Updated weights for policy 0, policy_version 878236 (0.0009) [2023-12-27 11:43:42,183][20381] Updated weights for policy 1, policy_version 877535 (0.0005) [2023-12-27 11:43:42,242][20347] Updated weights for policy 0, policy_version 878246 (0.0008) [2023-12-27 11:43:42,245][20381] Updated weights for policy 1, policy_version 877545 (0.0007) [2023-12-27 11:43:42,306][20381] Updated weights for policy 1, policy_version 877555 (0.0008) [2023-12-27 11:43:42,306][20347] Updated weights for policy 0, policy_version 878256 (0.0011) [2023-12-27 11:43:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 19716.3). Total num frames: 449544192. Throughput: 0: 10394.8, 1: 10338.0. Samples: 449529036. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:42,363][19170] Avg episode reward: [(0, '883.380'), (1, '1000.000')] [2023-12-27 11:43:42,996][20347] Updated weights for policy 0, policy_version 878266 (0.0006) [2023-12-27 11:43:43,003][20381] Updated weights for policy 1, policy_version 877565 (0.0007) [2023-12-27 11:43:43,045][20347] Updated weights for policy 0, policy_version 878276 (0.0005) [2023-12-27 11:43:43,054][20381] Updated weights for policy 1, policy_version 877575 (0.0007) [2023-12-27 11:43:43,098][20347] Updated weights for policy 0, policy_version 878286 (0.0005) [2023-12-27 11:43:43,099][20381] Updated weights for policy 1, policy_version 877585 (0.0005) [2023-12-27 11:43:43,680][20347] Updated weights for policy 0, policy_version 878296 (0.0006) [2023-12-27 11:43:43,731][20347] Updated weights for policy 0, policy_version 878306 (0.0005) [2023-12-27 11:43:43,745][20381] Updated weights for policy 1, policy_version 877595 (0.0007) [2023-12-27 11:43:43,787][20347] Updated weights for policy 0, policy_version 878316 (0.0005) [2023-12-27 11:43:43,795][20381] Updated weights for policy 1, policy_version 877605 (0.0007) [2023-12-27 11:43:43,847][20381] Updated weights for policy 1, policy_version 877615 (0.0007) [2023-12-27 11:43:44,386][20347] Updated weights for policy 0, policy_version 878326 (0.0007) [2023-12-27 11:43:44,435][20347] Updated weights for policy 0, policy_version 878336 (0.0008) [2023-12-27 11:43:44,482][20347] Updated weights for policy 0, policy_version 878346 (0.0008) [2023-12-27 11:43:44,526][20381] Updated weights for policy 1, policy_version 877625 (0.0008) [2023-12-27 11:43:44,586][20381] Updated weights for policy 1, policy_version 877635 (0.0007) [2023-12-27 11:43:44,640][20381] Updated weights for policy 1, policy_version 877645 (0.0005) [2023-12-27 11:43:44,696][20381] Updated weights for policy 1, policy_version 877655 (0.0008) [2023-12-27 11:43:45,278][20347] Updated weights for policy 0, policy_version 878356 (0.0009) [2023-12-27 11:43:45,336][20347] Updated weights for policy 0, policy_version 878366 (0.0010) [2023-12-27 11:43:45,374][20381] Updated weights for policy 1, policy_version 877665 (0.0011) [2023-12-27 11:43:45,393][20347] Updated weights for policy 0, policy_version 878376 (0.0010) [2023-12-27 11:43:45,427][20381] Updated weights for policy 1, policy_version 877675 (0.0011) [2023-12-27 11:43:45,486][20381] Updated weights for policy 1, policy_version 877685 (0.0011) [2023-12-27 11:43:46,039][20347] Updated weights for policy 0, policy_version 878386 (0.0008) [2023-12-27 11:43:46,089][20347] Updated weights for policy 0, policy_version 878396 (0.0010) [2023-12-27 11:43:46,148][20347] Updated weights for policy 0, policy_version 878406 (0.0011) [2023-12-27 11:43:46,184][20381] Updated weights for policy 1, policy_version 877695 (0.0011) [2023-12-27 11:43:46,206][20347] Updated weights for policy 0, policy_version 878416 (0.0010) [2023-12-27 11:43:46,246][20381] Updated weights for policy 1, policy_version 877705 (0.0011) [2023-12-27 11:43:46,311][20381] Updated weights for policy 1, policy_version 877715 (0.0011) [2023-12-27 11:43:46,880][20347] Updated weights for policy 0, policy_version 878426 (0.0010) [2023-12-27 11:43:46,937][20347] Updated weights for policy 0, policy_version 878436 (0.0010) [2023-12-27 11:43:46,983][20381] Updated weights for policy 1, policy_version 877725 (0.0010) [2023-12-27 11:43:46,999][20347] Updated weights for policy 0, policy_version 878446 (0.0010) [2023-12-27 11:43:47,037][20381] Updated weights for policy 1, policy_version 877735 (0.0011) [2023-12-27 11:43:47,092][20381] Updated weights for policy 1, policy_version 877745 (0.0010) [2023-12-27 11:43:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 19744.1). Total num frames: 449642496. Throughput: 0: 10428.0, 1: 10347.0. Samples: 449651108. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:47,363][19170] Avg episode reward: [(0, '902.270'), (1, '1000.000')] [2023-12-27 11:43:47,653][20347] Updated weights for policy 0, policy_version 878456 (0.0006) [2023-12-27 11:43:47,708][20347] Updated weights for policy 0, policy_version 878466 (0.0008) [2023-12-27 11:43:47,715][20291] KL-divergence is very high: 213.7579 [2023-12-27 11:43:47,727][20291] KL-divergence is very high: 161.4356 [2023-12-27 11:43:47,766][20347] Updated weights for policy 0, policy_version 878476 (0.0010) [2023-12-27 11:43:47,773][20381] Updated weights for policy 1, policy_version 877755 (0.0011) [2023-12-27 11:43:47,828][20381] Updated weights for policy 1, policy_version 877765 (0.0011) [2023-12-27 11:43:47,883][20381] Updated weights for policy 1, policy_version 877775 (0.0010) [2023-12-27 11:43:48,436][20347] Updated weights for policy 0, policy_version 878486 (0.0010) [2023-12-27 11:43:48,487][20347] Updated weights for policy 0, policy_version 878496 (0.0010) [2023-12-27 11:43:48,542][20347] Updated weights for policy 0, policy_version 878506 (0.0010) [2023-12-27 11:43:48,581][20381] Updated weights for policy 1, policy_version 877785 (0.0010) [2023-12-27 11:43:48,634][20381] Updated weights for policy 1, policy_version 877795 (0.0011) [2023-12-27 11:43:48,689][20381] Updated weights for policy 1, policy_version 877805 (0.0011) [2023-12-27 11:43:48,748][20381] Updated weights for policy 1, policy_version 877815 (0.0011) [2023-12-27 11:43:49,249][20347] Updated weights for policy 0, policy_version 878516 (0.0009) [2023-12-27 11:43:49,307][20347] Updated weights for policy 0, policy_version 878526 (0.0007) [2023-12-27 11:43:49,374][20347] Updated weights for policy 0, policy_version 878536 (0.0009) [2023-12-27 11:43:49,466][20381] Updated weights for policy 1, policy_version 877825 (0.0009) [2023-12-27 11:43:49,527][20381] Updated weights for policy 1, policy_version 877835 (0.0008) [2023-12-27 11:43:49,592][20381] Updated weights for policy 1, policy_version 877845 (0.0009) [2023-12-27 11:43:50,071][20347] Updated weights for policy 0, policy_version 878546 (0.0009) [2023-12-27 11:43:50,135][20347] Updated weights for policy 0, policy_version 878556 (0.0011) [2023-12-27 11:43:50,197][20347] Updated weights for policy 0, policy_version 878566 (0.0011) [2023-12-27 11:43:50,252][20347] Updated weights for policy 0, policy_version 878576 (0.0010) [2023-12-27 11:43:50,266][20381] Updated weights for policy 1, policy_version 877855 (0.0009) [2023-12-27 11:43:50,293][20292] KL-divergence is very high: 224.6817 [2023-12-27 11:43:50,330][20381] Updated weights for policy 1, policy_version 877865 (0.0005) [2023-12-27 11:43:50,392][20381] Updated weights for policy 1, policy_version 877875 (0.0006) [2023-12-27 11:43:50,921][20347] Updated weights for policy 0, policy_version 878586 (0.0010) [2023-12-27 11:43:50,983][20347] Updated weights for policy 0, policy_version 878596 (0.0010) [2023-12-27 11:43:51,046][20347] Updated weights for policy 0, policy_version 878606 (0.0010) [2023-12-27 11:43:51,065][20381] Updated weights for policy 1, policy_version 877885 (0.0011) [2023-12-27 11:43:51,129][20381] Updated weights for policy 1, policy_version 877895 (0.0010) [2023-12-27 11:43:51,194][20381] Updated weights for policy 1, policy_version 877905 (0.0009) [2023-12-27 11:43:51,748][20347] Updated weights for policy 0, policy_version 878616 (0.0009) [2023-12-27 11:43:51,808][20347] Updated weights for policy 0, policy_version 878626 (0.0009) [2023-12-27 11:43:51,862][20347] Updated weights for policy 0, policy_version 878636 (0.0009) [2023-12-27 11:43:51,921][20381] Updated weights for policy 1, policy_version 877915 (0.0007) [2023-12-27 11:43:51,982][20381] Updated weights for policy 1, policy_version 877925 (0.0007) [2023-12-27 11:43:52,045][20381] Updated weights for policy 1, policy_version 877935 (0.0008) [2023-12-27 11:43:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 19744.1). Total num frames: 449740800. Throughput: 0: 10421.3, 1: 10331.2. Samples: 449711528. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:52,363][19170] Avg episode reward: [(0, '902.290'), (1, '990.250')] [2023-12-27 11:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000877944_224780288.pth... [2023-12-27 11:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000878640_224960512.pth... [2023-12-27 11:43:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000876728_224468992.pth [2023-12-27 11:43:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000877424_224649216.pth [2023-12-27 11:43:52,597][20347] Updated weights for policy 0, policy_version 878646 (0.0007) [2023-12-27 11:43:52,657][20347] Updated weights for policy 0, policy_version 878656 (0.0008) [2023-12-27 11:43:52,716][20347] Updated weights for policy 0, policy_version 878666 (0.0009) [2023-12-27 11:43:52,762][20381] Updated weights for policy 1, policy_version 877945 (0.0007) [2023-12-27 11:43:52,821][20381] Updated weights for policy 1, policy_version 877955 (0.0011) [2023-12-27 11:43:52,879][20381] Updated weights for policy 1, policy_version 877965 (0.0010) [2023-12-27 11:43:52,937][20381] Updated weights for policy 1, policy_version 877975 (0.0011) [2023-12-27 11:43:53,307][20347] Updated weights for policy 0, policy_version 878676 (0.0009) [2023-12-27 11:43:53,361][20347] Updated weights for policy 0, policy_version 878686 (0.0006) [2023-12-27 11:43:53,416][20347] Updated weights for policy 0, policy_version 878696 (0.0005) [2023-12-27 11:43:53,574][20381] Updated weights for policy 1, policy_version 877985 (0.0011) [2023-12-27 11:43:53,624][20381] Updated weights for policy 1, policy_version 877995 (0.0009) [2023-12-27 11:43:53,681][20381] Updated weights for policy 1, policy_version 878005 (0.0009) [2023-12-27 11:43:53,938][20347] Updated weights for policy 0, policy_version 878706 (0.0006) [2023-12-27 11:43:54,003][20347] Updated weights for policy 0, policy_version 878716 (0.0009) [2023-12-27 11:43:54,062][20347] Updated weights for policy 0, policy_version 878726 (0.0007) [2023-12-27 11:43:54,126][20347] Updated weights for policy 0, policy_version 878736 (0.0007) [2023-12-27 11:43:54,405][20381] Updated weights for policy 1, policy_version 878015 (0.0006) [2023-12-27 11:43:54,460][20381] Updated weights for policy 1, policy_version 878025 (0.0007) [2023-12-27 11:43:54,524][20381] Updated weights for policy 1, policy_version 878035 (0.0009) [2023-12-27 11:43:54,735][20347] Updated weights for policy 0, policy_version 878746 (0.0009) [2023-12-27 11:43:54,796][20347] Updated weights for policy 0, policy_version 878756 (0.0008) [2023-12-27 11:43:54,842][20347] Updated weights for policy 0, policy_version 878766 (0.0008) [2023-12-27 11:43:55,116][20381] Updated weights for policy 1, policy_version 878045 (0.0008) [2023-12-27 11:43:55,179][20381] Updated weights for policy 1, policy_version 878055 (0.0008) [2023-12-27 11:43:55,245][20381] Updated weights for policy 1, policy_version 878065 (0.0007) [2023-12-27 11:43:55,563][20347] Updated weights for policy 0, policy_version 878776 (0.0008) [2023-12-27 11:43:55,616][20347] Updated weights for policy 0, policy_version 878786 (0.0008) [2023-12-27 11:43:55,672][20347] Updated weights for policy 0, policy_version 878796 (0.0007) [2023-12-27 11:43:55,859][20381] Updated weights for policy 1, policy_version 878075 (0.0007) [2023-12-27 11:43:55,919][20381] Updated weights for policy 1, policy_version 878085 (0.0007) [2023-12-27 11:43:55,982][20381] Updated weights for policy 1, policy_version 878095 (0.0007) [2023-12-27 11:43:56,413][20347] Updated weights for policy 0, policy_version 878806 (0.0007) [2023-12-27 11:43:56,470][20347] Updated weights for policy 0, policy_version 878816 (0.0007) [2023-12-27 11:43:56,528][20347] Updated weights for policy 0, policy_version 878826 (0.0008) [2023-12-27 11:43:56,618][20381] Updated weights for policy 1, policy_version 878105 (0.0007) [2023-12-27 11:43:56,669][20381] Updated weights for policy 1, policy_version 878115 (0.0007) [2023-12-27 11:43:56,720][20381] Updated weights for policy 1, policy_version 878125 (0.0008) [2023-12-27 11:43:56,778][20381] Updated weights for policy 1, policy_version 878135 (0.0007) [2023-12-27 11:43:57,184][20347] Updated weights for policy 0, policy_version 878836 (0.0006) [2023-12-27 11:43:57,233][20347] Updated weights for policy 0, policy_version 878846 (0.0007) [2023-12-27 11:43:57,277][20347] Updated weights for policy 0, policy_version 878856 (0.0007) [2023-12-27 11:43:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 19771.9). Total num frames: 449847296. Throughput: 0: 10402.6, 1: 10401.3. Samples: 449835116. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:43:57,363][19170] Avg episode reward: [(0, '882.630'), (1, '980.540')] [2023-12-27 11:43:57,400][20381] Updated weights for policy 1, policy_version 878145 (0.0010) [2023-12-27 11:43:57,454][20381] Updated weights for policy 1, policy_version 878155 (0.0010) [2023-12-27 11:43:57,509][20381] Updated weights for policy 1, policy_version 878165 (0.0010) [2023-12-27 11:43:57,915][20347] Updated weights for policy 0, policy_version 878866 (0.0007) [2023-12-27 11:43:57,964][20347] Updated weights for policy 0, policy_version 878876 (0.0007) [2023-12-27 11:43:58,011][20347] Updated weights for policy 0, policy_version 878886 (0.0005) [2023-12-27 11:43:58,067][20347] Updated weights for policy 0, policy_version 878896 (0.0005) [2023-12-27 11:43:58,102][20381] Updated weights for policy 1, policy_version 878175 (0.0010) [2023-12-27 11:43:58,156][20381] Updated weights for policy 1, policy_version 878185 (0.0010) [2023-12-27 11:43:58,218][20381] Updated weights for policy 1, policy_version 878195 (0.0010) [2023-12-27 11:43:58,708][20347] Updated weights for policy 0, policy_version 878906 (0.0011) [2023-12-27 11:43:58,771][20347] Updated weights for policy 0, policy_version 878916 (0.0009) [2023-12-27 11:43:58,835][20347] Updated weights for policy 0, policy_version 878926 (0.0008) [2023-12-27 11:43:58,963][20381] Updated weights for policy 1, policy_version 878205 (0.0009) [2023-12-27 11:43:59,022][20381] Updated weights for policy 1, policy_version 878215 (0.0008) [2023-12-27 11:43:59,074][20381] Updated weights for policy 1, policy_version 878225 (0.0005) [2023-12-27 11:43:59,633][20347] Updated weights for policy 0, policy_version 878936 (0.0008) [2023-12-27 11:43:59,696][20347] Updated weights for policy 0, policy_version 878946 (0.0008) [2023-12-27 11:43:59,703][20381] Updated weights for policy 1, policy_version 878235 (0.0005) [2023-12-27 11:43:59,750][20347] Updated weights for policy 0, policy_version 878956 (0.0007) [2023-12-27 11:43:59,761][20381] Updated weights for policy 1, policy_version 878245 (0.0006) [2023-12-27 11:43:59,768][20292] KL-divergence is very high: 232.4514 [2023-12-27 11:43:59,821][20381] Updated weights for policy 1, policy_version 878255 (0.0009) [2023-12-27 11:44:00,442][20381] Updated weights for policy 1, policy_version 878265 (0.0008) [2023-12-27 11:44:00,479][20347] Updated weights for policy 0, policy_version 878966 (0.0007) [2023-12-27 11:44:00,496][20381] Updated weights for policy 1, policy_version 878275 (0.0010) [2023-12-27 11:44:00,526][20347] Updated weights for policy 0, policy_version 878976 (0.0005) [2023-12-27 11:44:00,549][20381] Updated weights for policy 1, policy_version 878285 (0.0010) [2023-12-27 11:44:00,580][20347] Updated weights for policy 0, policy_version 878986 (0.0005) [2023-12-27 11:44:00,607][20381] Updated weights for policy 1, policy_version 878295 (0.0010) [2023-12-27 11:44:01,161][20347] Updated weights for policy 0, policy_version 878996 (0.0006) [2023-12-27 11:44:01,222][20347] Updated weights for policy 0, policy_version 879006 (0.0008) [2023-12-27 11:44:01,249][20381] Updated weights for policy 1, policy_version 878305 (0.0008) [2023-12-27 11:44:01,282][20347] Updated weights for policy 0, policy_version 879016 (0.0007) [2023-12-27 11:44:01,307][20381] Updated weights for policy 1, policy_version 878315 (0.0007) [2023-12-27 11:44:01,367][20381] Updated weights for policy 1, policy_version 878325 (0.0008) [2023-12-27 11:44:01,974][20347] Updated weights for policy 0, policy_version 879026 (0.0007) [2023-12-27 11:44:02,036][20347] Updated weights for policy 0, policy_version 879036 (0.0006) [2023-12-27 11:44:02,072][20381] Updated weights for policy 1, policy_version 878335 (0.0006) [2023-12-27 11:44:02,093][20347] Updated weights for policy 0, policy_version 879046 (0.0007) [2023-12-27 11:44:02,135][20381] Updated weights for policy 1, policy_version 878345 (0.0007) [2023-12-27 11:44:02,152][20347] Updated weights for policy 0, policy_version 879056 (0.0005) [2023-12-27 11:44:02,193][20381] Updated weights for policy 1, policy_version 878355 (0.0007) [2023-12-27 11:44:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 19799.6). Total num frames: 449953792. Throughput: 0: 10384.2, 1: 10365.7. Samples: 449959396. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:02,363][19170] Avg episode reward: [(0, '853.170'), (1, '970.880')] [2023-12-27 11:44:02,846][20381] Updated weights for policy 1, policy_version 878365 (0.0007) [2023-12-27 11:44:02,851][20347] Updated weights for policy 0, policy_version 879066 (0.0008) [2023-12-27 11:44:02,893][20381] Updated weights for policy 1, policy_version 878375 (0.0005) [2023-12-27 11:44:02,906][20347] Updated weights for policy 0, policy_version 879076 (0.0007) [2023-12-27 11:44:02,941][20381] Updated weights for policy 1, policy_version 878385 (0.0005) [2023-12-27 11:44:02,960][20347] Updated weights for policy 0, policy_version 879086 (0.0006) [2023-12-27 11:44:03,567][20347] Updated weights for policy 0, policy_version 879096 (0.0007) [2023-12-27 11:44:03,626][20347] Updated weights for policy 0, policy_version 879106 (0.0005) [2023-12-27 11:44:03,665][20381] Updated weights for policy 1, policy_version 878395 (0.0007) [2023-12-27 11:44:03,684][20347] Updated weights for policy 0, policy_version 879116 (0.0005) [2023-12-27 11:44:03,718][20381] Updated weights for policy 1, policy_version 878405 (0.0008) [2023-12-27 11:44:03,770][20381] Updated weights for policy 1, policy_version 878415 (0.0008) [2023-12-27 11:44:04,319][20347] Updated weights for policy 0, policy_version 879126 (0.0007) [2023-12-27 11:44:04,335][20291] KL-divergence is very high: 104.5781 [2023-12-27 11:44:04,386][20347] Updated weights for policy 0, policy_version 879136 (0.0008) [2023-12-27 11:44:04,439][20347] Updated weights for policy 0, policy_version 879146 (0.0007) [2023-12-27 11:44:04,502][20381] Updated weights for policy 1, policy_version 878425 (0.0008) [2023-12-27 11:44:04,556][20381] Updated weights for policy 1, policy_version 878435 (0.0008) [2023-12-27 11:44:04,614][20381] Updated weights for policy 1, policy_version 878445 (0.0007) [2023-12-27 11:44:04,664][20381] Updated weights for policy 1, policy_version 878455 (0.0009) [2023-12-27 11:44:05,113][20347] Updated weights for policy 0, policy_version 879156 (0.0007) [2023-12-27 11:44:05,185][20347] Updated weights for policy 0, policy_version 879166 (0.0009) [2023-12-27 11:44:05,250][20347] Updated weights for policy 0, policy_version 879176 (0.0009) [2023-12-27 11:44:05,323][20381] Updated weights for policy 1, policy_version 878465 (0.0006) [2023-12-27 11:44:05,373][20381] Updated weights for policy 1, policy_version 878475 (0.0008) [2023-12-27 11:44:05,426][20381] Updated weights for policy 1, policy_version 878485 (0.0007) [2023-12-27 11:44:05,894][20347] Updated weights for policy 0, policy_version 879186 (0.0007) [2023-12-27 11:44:05,953][20347] Updated weights for policy 0, policy_version 879196 (0.0007) [2023-12-27 11:44:06,010][20347] Updated weights for policy 0, policy_version 879206 (0.0007) [2023-12-27 11:44:06,072][20347] Updated weights for policy 0, policy_version 879216 (0.0007) [2023-12-27 11:44:06,132][20381] Updated weights for policy 1, policy_version 878495 (0.0010) [2023-12-27 11:44:06,193][20381] Updated weights for policy 1, policy_version 878505 (0.0011) [2023-12-27 11:44:06,247][20381] Updated weights for policy 1, policy_version 878515 (0.0011) [2023-12-27 11:44:06,790][20347] Updated weights for policy 0, policy_version 879226 (0.0007) [2023-12-27 11:44:06,845][20347] Updated weights for policy 0, policy_version 879236 (0.0007) [2023-12-27 11:44:06,901][20347] Updated weights for policy 0, policy_version 879246 (0.0006) [2023-12-27 11:44:06,976][20381] Updated weights for policy 1, policy_version 878525 (0.0011) [2023-12-27 11:44:07,025][20381] Updated weights for policy 1, policy_version 878535 (0.0010) [2023-12-27 11:44:07,087][20381] Updated weights for policy 1, policy_version 878545 (0.0011) [2023-12-27 11:44:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 19827.4). Total num frames: 450052096. Throughput: 0: 10370.3, 1: 10375.3. Samples: 450021548. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:07,363][19170] Avg episode reward: [(0, '823.810'), (1, '980.590')] [2023-12-27 11:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000878552_224935936.pth... [2023-12-27 11:44:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000879248_225116160.pth... [2023-12-27 11:44:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000877368_224632832.pth [2023-12-27 11:44:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000878064_224813056.pth [2023-12-27 11:44:07,592][20347] Updated weights for policy 0, policy_version 879256 (0.0007) [2023-12-27 11:44:07,649][20347] Updated weights for policy 0, policy_version 879266 (0.0007) [2023-12-27 11:44:07,705][20347] Updated weights for policy 0, policy_version 879276 (0.0007) [2023-12-27 11:44:07,752][20381] Updated weights for policy 1, policy_version 878555 (0.0010) [2023-12-27 11:44:07,807][20381] Updated weights for policy 1, policy_version 878565 (0.0010) [2023-12-27 11:44:07,861][20381] Updated weights for policy 1, policy_version 878575 (0.0008) [2023-12-27 11:44:08,377][20347] Updated weights for policy 0, policy_version 879286 (0.0008) [2023-12-27 11:44:08,401][20291] KL-divergence is very high: 173.0934 [2023-12-27 11:44:08,412][20291] KL-divergence is very high: 135.2749 [2023-12-27 11:44:08,432][20347] Updated weights for policy 0, policy_version 879296 (0.0007) [2023-12-27 11:44:08,494][20347] Updated weights for policy 0, policy_version 879306 (0.0007) [2023-12-27 11:44:08,506][20381] Updated weights for policy 1, policy_version 878585 (0.0010) [2023-12-27 11:44:08,553][20381] Updated weights for policy 1, policy_version 878595 (0.0007) [2023-12-27 11:44:08,604][20381] Updated weights for policy 1, policy_version 878605 (0.0006) [2023-12-27 11:44:08,652][20381] Updated weights for policy 1, policy_version 878615 (0.0006) [2023-12-27 11:44:09,179][20347] Updated weights for policy 0, policy_version 879316 (0.0007) [2023-12-27 11:44:09,250][20347] Updated weights for policy 0, policy_version 879326 (0.0008) [2023-12-27 11:44:09,286][20381] Updated weights for policy 1, policy_version 878625 (0.0008) [2023-12-27 11:44:09,311][20347] Updated weights for policy 0, policy_version 879336 (0.0007) [2023-12-27 11:44:09,346][20381] Updated weights for policy 1, policy_version 878635 (0.0006) [2023-12-27 11:44:09,410][20381] Updated weights for policy 1, policy_version 878645 (0.0008) [2023-12-27 11:44:10,054][20347] Updated weights for policy 0, policy_version 879346 (0.0006) [2023-12-27 11:44:10,096][20381] Updated weights for policy 1, policy_version 878655 (0.0008) [2023-12-27 11:44:10,115][20347] Updated weights for policy 0, policy_version 879356 (0.0007) [2023-12-27 11:44:10,162][20381] Updated weights for policy 1, policy_version 878665 (0.0006) [2023-12-27 11:44:10,174][20347] Updated weights for policy 0, policy_version 879366 (0.0007) [2023-12-27 11:44:10,213][20381] Updated weights for policy 1, policy_version 878675 (0.0006) [2023-12-27 11:44:10,222][20347] Updated weights for policy 0, policy_version 879376 (0.0006) [2023-12-27 11:44:10,875][20381] Updated weights for policy 1, policy_version 878685 (0.0006) [2023-12-27 11:44:10,933][20381] Updated weights for policy 1, policy_version 878695 (0.0007) [2023-12-27 11:44:10,972][20347] Updated weights for policy 0, policy_version 879386 (0.0006) [2023-12-27 11:44:10,995][20381] Updated weights for policy 1, policy_version 878705 (0.0006) [2023-12-27 11:44:11,038][20347] Updated weights for policy 0, policy_version 879396 (0.0008) [2023-12-27 11:44:11,103][20347] Updated weights for policy 0, policy_version 879406 (0.0008) [2023-12-27 11:44:11,754][20381] Updated weights for policy 1, policy_version 878715 (0.0008) [2023-12-27 11:44:11,812][20381] Updated weights for policy 1, policy_version 878725 (0.0007) [2023-12-27 11:44:11,851][20347] Updated weights for policy 0, policy_version 879416 (0.0010) [2023-12-27 11:44:11,871][20381] Updated weights for policy 1, policy_version 878735 (0.0008) [2023-12-27 11:44:11,908][20347] Updated weights for policy 0, policy_version 879426 (0.0010) [2023-12-27 11:44:11,958][20347] Updated weights for policy 0, policy_version 879436 (0.0010) [2023-12-27 11:44:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 19855.2). Total num frames: 450150400. Throughput: 0: 10311.7, 1: 10303.3. Samples: 450141684. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:12,363][19170] Avg episode reward: [(0, '814.590'), (1, '980.510')] [2023-12-27 11:44:12,594][20381] Updated weights for policy 1, policy_version 878745 (0.0009) [2023-12-27 11:44:12,627][20347] Updated weights for policy 0, policy_version 879446 (0.0011) [2023-12-27 11:44:12,651][20381] Updated weights for policy 1, policy_version 878755 (0.0011) [2023-12-27 11:44:12,680][20347] Updated weights for policy 0, policy_version 879456 (0.0010) [2023-12-27 11:44:12,706][20381] Updated weights for policy 1, policy_version 878765 (0.0010) [2023-12-27 11:44:12,739][20347] Updated weights for policy 0, policy_version 879466 (0.0010) [2023-12-27 11:44:12,765][20381] Updated weights for policy 1, policy_version 878775 (0.0011) [2023-12-27 11:44:13,429][20347] Updated weights for policy 0, policy_version 879476 (0.0007) [2023-12-27 11:44:13,455][20381] Updated weights for policy 1, policy_version 878785 (0.0009) [2023-12-27 11:44:13,487][20347] Updated weights for policy 0, policy_version 879486 (0.0007) [2023-12-27 11:44:13,511][20381] Updated weights for policy 1, policy_version 878795 (0.0006) [2023-12-27 11:44:13,544][20347] Updated weights for policy 0, policy_version 879496 (0.0008) [2023-12-27 11:44:13,567][20381] Updated weights for policy 1, policy_version 878805 (0.0005) [2023-12-27 11:44:14,186][20381] Updated weights for policy 1, policy_version 878815 (0.0007) [2023-12-27 11:44:14,245][20381] Updated weights for policy 1, policy_version 878825 (0.0007) [2023-12-27 11:44:14,266][20347] Updated weights for policy 0, policy_version 879506 (0.0007) [2023-12-27 11:44:14,308][20381] Updated weights for policy 1, policy_version 878835 (0.0006) [2023-12-27 11:44:14,331][20347] Updated weights for policy 0, policy_version 879516 (0.0007) [2023-12-27 11:44:14,393][20347] Updated weights for policy 0, policy_version 879526 (0.0008) [2023-12-27 11:44:14,456][20347] Updated weights for policy 0, policy_version 879536 (0.0007) [2023-12-27 11:44:14,996][20381] Updated weights for policy 1, policy_version 878845 (0.0007) [2023-12-27 11:44:15,061][20381] Updated weights for policy 1, policy_version 878855 (0.0007) [2023-12-27 11:44:15,127][20381] Updated weights for policy 1, policy_version 878865 (0.0007) [2023-12-27 11:44:15,166][20291] KL-divergence is very high: 391.4725 [2023-12-27 11:44:15,223][20347] Updated weights for policy 0, policy_version 879546 (0.0008) [2023-12-27 11:44:15,284][20347] Updated weights for policy 0, policy_version 879556 (0.0008) [2023-12-27 11:44:15,337][20347] Updated weights for policy 0, policy_version 879566 (0.0008) [2023-12-27 11:44:15,805][20381] Updated weights for policy 1, policy_version 878875 (0.0007) [2023-12-27 11:44:15,868][20381] Updated weights for policy 1, policy_version 878885 (0.0008) [2023-12-27 11:44:15,927][20381] Updated weights for policy 1, policy_version 878895 (0.0008) [2023-12-27 11:44:16,058][20347] Updated weights for policy 0, policy_version 879576 (0.0007) [2023-12-27 11:44:16,116][20347] Updated weights for policy 0, policy_version 879586 (0.0009) [2023-12-27 11:44:16,168][20347] Updated weights for policy 0, policy_version 879596 (0.0008) [2023-12-27 11:44:16,648][20381] Updated weights for policy 1, policy_version 878905 (0.0008) [2023-12-27 11:44:16,699][20381] Updated weights for policy 1, policy_version 878915 (0.0009) [2023-12-27 11:44:16,756][20381] Updated weights for policy 1, policy_version 878925 (0.0009) [2023-12-27 11:44:16,816][20381] Updated weights for policy 1, policy_version 878935 (0.0009) [2023-12-27 11:44:16,952][20347] Updated weights for policy 0, policy_version 879606 (0.0008) [2023-12-27 11:44:17,024][20347] Updated weights for policy 0, policy_version 879616 (0.0008) [2023-12-27 11:44:17,085][20347] Updated weights for policy 0, policy_version 879626 (0.0007) [2023-12-27 11:44:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 19855.2). Total num frames: 450248704. Throughput: 0: 10182.6, 1: 10213.8. Samples: 450257564. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:17,363][19170] Avg episode reward: [(0, '785.950'), (1, '970.830')] [2023-12-27 11:44:17,587][20381] Updated weights for policy 1, policy_version 878945 (0.0006) [2023-12-27 11:44:17,642][20381] Updated weights for policy 1, policy_version 878955 (0.0007) [2023-12-27 11:44:17,700][20381] Updated weights for policy 1, policy_version 878965 (0.0008) [2023-12-27 11:44:17,781][20347] Updated weights for policy 0, policy_version 879636 (0.0007) [2023-12-27 11:44:17,833][20347] Updated weights for policy 0, policy_version 879646 (0.0007) [2023-12-27 11:44:17,894][20347] Updated weights for policy 0, policy_version 879656 (0.0007) [2023-12-27 11:44:18,374][20381] Updated weights for policy 1, policy_version 878975 (0.0006) [2023-12-27 11:44:18,434][20381] Updated weights for policy 1, policy_version 878985 (0.0006) [2023-12-27 11:44:18,494][20381] Updated weights for policy 1, policy_version 878995 (0.0006) [2023-12-27 11:44:18,548][20347] Updated weights for policy 0, policy_version 879666 (0.0006) [2023-12-27 11:44:18,612][20347] Updated weights for policy 0, policy_version 879676 (0.0008) [2023-12-27 11:44:18,675][20347] Updated weights for policy 0, policy_version 879686 (0.0008) [2023-12-27 11:44:18,691][20291] KL-divergence is very high: 140.1357 [2023-12-27 11:44:18,697][20291] KL-divergence is very high: 113.5943 [2023-12-27 11:44:18,730][20347] Updated weights for policy 0, policy_version 879696 (0.0008) [2023-12-27 11:44:19,172][20381] Updated weights for policy 1, policy_version 879005 (0.0008) [2023-12-27 11:44:19,224][20381] Updated weights for policy 1, policy_version 879015 (0.0010) [2023-12-27 11:44:19,280][20381] Updated weights for policy 1, policy_version 879025 (0.0008) [2023-12-27 11:44:19,472][20347] Updated weights for policy 0, policy_version 879706 (0.0008) [2023-12-27 11:44:19,531][20347] Updated weights for policy 0, policy_version 879716 (0.0008) [2023-12-27 11:44:19,533][20291] KL-divergence is very high: 108.9647 [2023-12-27 11:44:19,580][20291] KL-divergence is very high: 102.3587 [2023-12-27 11:44:19,594][20347] Updated weights for policy 0, policy_version 879726 (0.0008) [2023-12-27 11:44:20,057][20381] Updated weights for policy 1, policy_version 879035 (0.0008) [2023-12-27 11:44:20,107][20381] Updated weights for policy 1, policy_version 879045 (0.0009) [2023-12-27 11:44:20,165][20381] Updated weights for policy 1, policy_version 879055 (0.0009) [2023-12-27 11:44:20,480][20347] Updated weights for policy 0, policy_version 879736 (0.0009) [2023-12-27 11:44:20,530][20347] Updated weights for policy 0, policy_version 879746 (0.0010) [2023-12-27 11:44:20,586][20347] Updated weights for policy 0, policy_version 879756 (0.0009) [2023-12-27 11:44:20,995][20381] Updated weights for policy 1, policy_version 879065 (0.0009) [2023-12-27 11:44:21,068][20381] Updated weights for policy 1, policy_version 879075 (0.0008) [2023-12-27 11:44:21,141][20381] Updated weights for policy 1, policy_version 879085 (0.0007) [2023-12-27 11:44:21,204][20381] Updated weights for policy 1, policy_version 879095 (0.0006) [2023-12-27 11:44:21,450][20347] Updated weights for policy 0, policy_version 879766 (0.0008) [2023-12-27 11:44:21,509][20347] Updated weights for policy 0, policy_version 879776 (0.0008) [2023-12-27 11:44:21,569][20347] Updated weights for policy 0, policy_version 879786 (0.0007) [2023-12-27 11:44:21,979][20381] Updated weights for policy 1, policy_version 879105 (0.0008) [2023-12-27 11:44:22,041][20381] Updated weights for policy 1, policy_version 879115 (0.0007) [2023-12-27 11:44:22,107][20381] Updated weights for policy 1, policy_version 879125 (0.0007) [2023-12-27 11:44:22,263][20347] Updated weights for policy 0, policy_version 879796 (0.0006) [2023-12-27 11:44:22,331][20347] Updated weights for policy 0, policy_version 879806 (0.0007) [2023-12-27 11:44:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20343.5, 300 sec: 19827.4). Total num frames: 450338816. Throughput: 0: 10093.0, 1: 10148.1. Samples: 450315544. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:22,363][19170] Avg episode reward: [(0, '824.030'), (1, '980.670')] [2023-12-27 11:44:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000879128_225083392.pth... [2023-12-27 11:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000877944_224780288.pth [2023-12-27 11:44:22,399][20347] Updated weights for policy 0, policy_version 879816 (0.0008) [2023-12-27 11:44:22,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000879824_225263616.pth... [2023-12-27 11:44:22,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000878640_224960512.pth [2023-12-27 11:44:22,825][20381] Updated weights for policy 1, policy_version 879135 (0.0008) [2023-12-27 11:44:22,892][20381] Updated weights for policy 1, policy_version 879145 (0.0007) [2023-12-27 11:44:22,952][20381] Updated weights for policy 1, policy_version 879155 (0.0008) [2023-12-27 11:44:23,057][20347] Updated weights for policy 0, policy_version 879826 (0.0008) [2023-12-27 11:44:23,110][20347] Updated weights for policy 0, policy_version 879836 (0.0009) [2023-12-27 11:44:23,162][20347] Updated weights for policy 0, policy_version 879846 (0.0007) [2023-12-27 11:44:23,216][20347] Updated weights for policy 0, policy_version 879856 (0.0007) [2023-12-27 11:44:23,614][20381] Updated weights for policy 1, policy_version 879165 (0.0008) [2023-12-27 11:44:23,672][20381] Updated weights for policy 1, policy_version 879175 (0.0008) [2023-12-27 11:44:23,730][20381] Updated weights for policy 1, policy_version 879185 (0.0007) [2023-12-27 11:44:23,832][20347] Updated weights for policy 0, policy_version 879866 (0.0008) [2023-12-27 11:44:23,847][20291] KL-divergence is very high: 149.1353 [2023-12-27 11:44:23,859][20291] KL-divergence is very high: 141.6240 [2023-12-27 11:44:23,888][20347] Updated weights for policy 0, policy_version 879876 (0.0006) [2023-12-27 11:44:23,942][20347] Updated weights for policy 0, policy_version 879886 (0.0005) [2023-12-27 11:44:24,473][20381] Updated weights for policy 1, policy_version 879195 (0.0007) [2023-12-27 11:44:24,478][20291] KL-divergence is very high: 162.5817 [2023-12-27 11:44:24,515][20347] Updated weights for policy 0, policy_version 879896 (0.0006) [2023-12-27 11:44:24,528][20291] KL-divergence is very high: 407.9873 [2023-12-27 11:44:24,535][20381] Updated weights for policy 1, policy_version 879205 (0.0008) [2023-12-27 11:44:24,574][20347] Updated weights for policy 0, policy_version 879906 (0.0007) [2023-12-27 11:44:24,575][20291] KL-divergence is very high: 269.5084 [2023-12-27 11:44:24,593][20381] Updated weights for policy 1, policy_version 879215 (0.0008) [2023-12-27 11:44:24,621][20291] KL-divergence is very high: 103.4739 [2023-12-27 11:44:24,634][20347] Updated weights for policy 0, policy_version 879916 (0.0007) [2023-12-27 11:44:25,220][20381] Updated weights for policy 1, policy_version 879225 (0.0007) [2023-12-27 11:44:25,273][20381] Updated weights for policy 1, policy_version 879235 (0.0005) [2023-12-27 11:44:25,335][20381] Updated weights for policy 1, policy_version 879245 (0.0006) [2023-12-27 11:44:25,335][20347] Updated weights for policy 0, policy_version 879926 (0.0007) [2023-12-27 11:44:25,336][20292] KL-divergence is very high: 217.0433 [2023-12-27 11:44:25,384][20347] Updated weights for policy 0, policy_version 879936 (0.0008) [2023-12-27 11:44:25,390][20381] Updated weights for policy 1, policy_version 879255 (0.0010) [2023-12-27 11:44:25,430][20347] Updated weights for policy 0, policy_version 879946 (0.0007) [2023-12-27 11:44:26,055][20381] Updated weights for policy 1, policy_version 879265 (0.0006) [2023-12-27 11:44:26,109][20381] Updated weights for policy 1, policy_version 879275 (0.0006) [2023-12-27 11:44:26,135][20347] Updated weights for policy 0, policy_version 879956 (0.0007) [2023-12-27 11:44:26,172][20381] Updated weights for policy 1, policy_version 879285 (0.0005) [2023-12-27 11:44:26,187][20347] Updated weights for policy 0, policy_version 879966 (0.0007) [2023-12-27 11:44:26,242][20347] Updated weights for policy 0, policy_version 879976 (0.0007) [2023-12-27 11:44:26,817][20381] Updated weights for policy 1, policy_version 879295 (0.0007) [2023-12-27 11:44:26,874][20347] Updated weights for policy 0, policy_version 879986 (0.0005) [2023-12-27 11:44:26,879][20381] Updated weights for policy 1, policy_version 879305 (0.0008) [2023-12-27 11:44:26,929][20347] Updated weights for policy 0, policy_version 879996 (0.0005) [2023-12-27 11:44:26,934][20381] Updated weights for policy 1, policy_version 879315 (0.0007) [2023-12-27 11:44:26,981][20347] Updated weights for policy 0, policy_version 880006 (0.0006) [2023-12-27 11:44:27,038][20347] Updated weights for policy 0, policy_version 880016 (0.0008) [2023-12-27 11:44:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 19827.4). Total num frames: 450445312. Throughput: 0: 10041.9, 1: 10070.9. Samples: 450434108. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:27,363][19170] Avg episode reward: [(0, '863.500'), (1, '952.120')] [2023-12-27 11:44:27,507][20381] Updated weights for policy 1, policy_version 879325 (0.0006) [2023-12-27 11:44:27,558][20381] Updated weights for policy 1, policy_version 879335 (0.0007) [2023-12-27 11:44:27,606][20381] Updated weights for policy 1, policy_version 879345 (0.0008) [2023-12-27 11:44:27,767][20347] Updated weights for policy 0, policy_version 880026 (0.0008) [2023-12-27 11:44:27,821][20347] Updated weights for policy 0, policy_version 880036 (0.0007) [2023-12-27 11:44:27,871][20347] Updated weights for policy 0, policy_version 880046 (0.0007) [2023-12-27 11:44:28,318][20381] Updated weights for policy 1, policy_version 879355 (0.0008) [2023-12-27 11:44:28,371][20381] Updated weights for policy 1, policy_version 879365 (0.0008) [2023-12-27 11:44:28,424][20381] Updated weights for policy 1, policy_version 879375 (0.0008) [2023-12-27 11:44:28,451][20347] Updated weights for policy 0, policy_version 880056 (0.0006) [2023-12-27 11:44:28,508][20347] Updated weights for policy 0, policy_version 880066 (0.0008) [2023-12-27 11:44:28,567][20347] Updated weights for policy 0, policy_version 880076 (0.0009) [2023-12-27 11:44:29,156][20347] Updated weights for policy 0, policy_version 880086 (0.0006) [2023-12-27 11:44:29,175][20381] Updated weights for policy 1, policy_version 879385 (0.0007) [2023-12-27 11:44:29,212][20347] Updated weights for policy 0, policy_version 880096 (0.0006) [2023-12-27 11:44:29,228][20381] Updated weights for policy 1, policy_version 879395 (0.0008) [2023-12-27 11:44:29,279][20347] Updated weights for policy 0, policy_version 880106 (0.0007) [2023-12-27 11:44:29,301][20381] Updated weights for policy 1, policy_version 879405 (0.0009) [2023-12-27 11:44:29,373][20381] Updated weights for policy 1, policy_version 879415 (0.0009) [2023-12-27 11:44:29,892][20347] Updated weights for policy 0, policy_version 880116 (0.0006) [2023-12-27 11:44:29,957][20347] Updated weights for policy 0, policy_version 880126 (0.0007) [2023-12-27 11:44:30,020][20347] Updated weights for policy 0, policy_version 880136 (0.0008) [2023-12-27 11:44:30,100][20381] Updated weights for policy 1, policy_version 879425 (0.0006) [2023-12-27 11:44:30,154][20381] Updated weights for policy 1, policy_version 879435 (0.0005) [2023-12-27 11:44:30,218][20381] Updated weights for policy 1, policy_version 879445 (0.0006) [2023-12-27 11:44:30,710][20347] Updated weights for policy 0, policy_version 880146 (0.0007) [2023-12-27 11:44:30,771][20347] Updated weights for policy 0, policy_version 880156 (0.0008) [2023-12-27 11:44:30,799][20381] Updated weights for policy 1, policy_version 879455 (0.0006) [2023-12-27 11:44:30,825][20347] Updated weights for policy 0, policy_version 880166 (0.0006) [2023-12-27 11:44:30,853][20381] Updated weights for policy 1, policy_version 879465 (0.0006) [2023-12-27 11:44:30,885][20347] Updated weights for policy 0, policy_version 880176 (0.0007) [2023-12-27 11:44:30,902][20381] Updated weights for policy 1, policy_version 879475 (0.0006) [2023-12-27 11:44:31,596][20381] Updated weights for policy 1, policy_version 879485 (0.0006) [2023-12-27 11:44:31,623][20347] Updated weights for policy 0, policy_version 880186 (0.0007) [2023-12-27 11:44:31,661][20381] Updated weights for policy 1, policy_version 879495 (0.0007) [2023-12-27 11:44:31,687][20347] Updated weights for policy 0, policy_version 880196 (0.0008) [2023-12-27 11:44:31,722][20381] Updated weights for policy 1, policy_version 879505 (0.0006) [2023-12-27 11:44:31,756][20347] Updated weights for policy 0, policy_version 880206 (0.0008) [2023-12-27 11:44:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 19855.2). Total num frames: 450543616. Throughput: 0: 10060.2, 1: 10082.4. Samples: 450557524. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:32,363][19170] Avg episode reward: [(0, '834.120'), (1, '932.670')] [2023-12-27 11:44:32,420][20381] Updated weights for policy 1, policy_version 879515 (0.0007) [2023-12-27 11:44:32,435][20347] Updated weights for policy 0, policy_version 880216 (0.0006) [2023-12-27 11:44:32,480][20381] Updated weights for policy 1, policy_version 879525 (0.0006) [2023-12-27 11:44:32,492][20347] Updated weights for policy 0, policy_version 880226 (0.0006) [2023-12-27 11:44:32,538][20381] Updated weights for policy 1, policy_version 879535 (0.0005) [2023-12-27 11:44:32,547][20347] Updated weights for policy 0, policy_version 880236 (0.0009) [2023-12-27 11:44:33,168][20381] Updated weights for policy 1, policy_version 879545 (0.0006) [2023-12-27 11:44:33,226][20381] Updated weights for policy 1, policy_version 879555 (0.0006) [2023-12-27 11:44:33,246][20347] Updated weights for policy 0, policy_version 880246 (0.0011) [2023-12-27 11:44:33,287][20381] Updated weights for policy 1, policy_version 879565 (0.0006) [2023-12-27 11:44:33,304][20347] Updated weights for policy 0, policy_version 880256 (0.0010) [2023-12-27 11:44:33,345][20381] Updated weights for policy 1, policy_version 879575 (0.0006) [2023-12-27 11:44:33,364][20347] Updated weights for policy 0, policy_version 880266 (0.0010) [2023-12-27 11:44:33,951][20381] Updated weights for policy 1, policy_version 879585 (0.0010) [2023-12-27 11:44:34,004][20381] Updated weights for policy 1, policy_version 879595 (0.0011) [2023-12-27 11:44:34,040][20347] Updated weights for policy 0, policy_version 880276 (0.0010) [2023-12-27 11:44:34,058][20381] Updated weights for policy 1, policy_version 879605 (0.0011) [2023-12-27 11:44:34,100][20347] Updated weights for policy 0, policy_version 880286 (0.0010) [2023-12-27 11:44:34,167][20347] Updated weights for policy 0, policy_version 880296 (0.0009) [2023-12-27 11:44:34,803][20347] Updated weights for policy 0, policy_version 880306 (0.0009) [2023-12-27 11:44:34,833][20381] Updated weights for policy 1, policy_version 879615 (0.0009) [2023-12-27 11:44:34,867][20347] Updated weights for policy 0, policy_version 880316 (0.0007) [2023-12-27 11:44:34,892][20381] Updated weights for policy 1, policy_version 879625 (0.0008) [2023-12-27 11:44:34,928][20347] Updated weights for policy 0, policy_version 880326 (0.0010) [2023-12-27 11:44:34,941][20381] Updated weights for policy 1, policy_version 879635 (0.0006) [2023-12-27 11:44:34,986][20347] Updated weights for policy 0, policy_version 880336 (0.0010) [2023-12-27 11:44:35,521][20381] Updated weights for policy 1, policy_version 879645 (0.0006) [2023-12-27 11:44:35,563][20347] Updated weights for policy 0, policy_version 880346 (0.0010) [2023-12-27 11:44:35,578][20381] Updated weights for policy 1, policy_version 879655 (0.0006) [2023-12-27 11:44:35,637][20381] Updated weights for policy 1, policy_version 879665 (0.0007) [2023-12-27 11:44:35,643][20347] Updated weights for policy 0, policy_version 880356 (0.0010) [2023-12-27 11:44:35,693][20347] Updated weights for policy 0, policy_version 880366 (0.0010) [2023-12-27 11:44:36,274][20381] Updated weights for policy 1, policy_version 879675 (0.0005) [2023-12-27 11:44:36,338][20381] Updated weights for policy 1, policy_version 879685 (0.0008) [2023-12-27 11:44:36,362][20347] Updated weights for policy 0, policy_version 880376 (0.0010) [2023-12-27 11:44:36,397][20381] Updated weights for policy 1, policy_version 879695 (0.0006) [2023-12-27 11:44:36,425][20347] Updated weights for policy 0, policy_version 880386 (0.0010) [2023-12-27 11:44:36,486][20347] Updated weights for policy 0, policy_version 880396 (0.0010) [2023-12-27 11:44:36,998][20381] Updated weights for policy 1, policy_version 879705 (0.0006) [2023-12-27 11:44:37,062][20381] Updated weights for policy 1, policy_version 879715 (0.0007) [2023-12-27 11:44:37,128][20381] Updated weights for policy 1, policy_version 879725 (0.0007) [2023-12-27 11:44:37,191][20381] Updated weights for policy 1, policy_version 879735 (0.0007) [2023-12-27 11:44:37,198][20347] Updated weights for policy 0, policy_version 880406 (0.0010) [2023-12-27 11:44:37,252][20347] Updated weights for policy 0, policy_version 880416 (0.0011) [2023-12-27 11:44:37,312][20347] Updated weights for policy 0, policy_version 880426 (0.0010) [2023-12-27 11:44:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 19910.7). Total num frames: 450658304. Throughput: 0: 10064.5, 1: 10115.0. Samples: 450619608. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:37,363][19170] Avg episode reward: [(0, '833.790'), (1, '961.110')] [2023-12-27 11:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000879736_225239040.pth... [2023-12-27 11:44:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000880432_225419264.pth... [2023-12-27 11:44:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000878552_224935936.pth [2023-12-27 11:44:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000879248_225116160.pth [2023-12-27 11:44:37,844][20381] Updated weights for policy 1, policy_version 879745 (0.0006) [2023-12-27 11:44:37,903][20381] Updated weights for policy 1, policy_version 879755 (0.0005) [2023-12-27 11:44:37,961][20381] Updated weights for policy 1, policy_version 879765 (0.0010) [2023-12-27 11:44:37,974][20347] Updated weights for policy 0, policy_version 880436 (0.0010) [2023-12-27 11:44:38,034][20347] Updated weights for policy 0, policy_version 880446 (0.0010) [2023-12-27 11:44:38,091][20347] Updated weights for policy 0, policy_version 880456 (0.0010) [2023-12-27 11:44:38,562][20381] Updated weights for policy 1, policy_version 879775 (0.0011) [2023-12-27 11:44:38,628][20381] Updated weights for policy 1, policy_version 879785 (0.0011) [2023-12-27 11:44:38,683][20381] Updated weights for policy 1, policy_version 879795 (0.0010) [2023-12-27 11:44:38,720][20347] Updated weights for policy 0, policy_version 880466 (0.0009) [2023-12-27 11:44:38,779][20347] Updated weights for policy 0, policy_version 880476 (0.0007) [2023-12-27 11:44:38,838][20347] Updated weights for policy 0, policy_version 880486 (0.0010) [2023-12-27 11:44:38,897][20347] Updated weights for policy 0, policy_version 880496 (0.0010) [2023-12-27 11:44:39,355][20381] Updated weights for policy 1, policy_version 879805 (0.0008) [2023-12-27 11:44:39,425][20381] Updated weights for policy 1, policy_version 879815 (0.0008) [2023-12-27 11:44:39,502][20381] Updated weights for policy 1, policy_version 879825 (0.0008) [2023-12-27 11:44:39,596][20347] Updated weights for policy 0, policy_version 880506 (0.0008) [2023-12-27 11:44:39,659][20347] Updated weights for policy 0, policy_version 880516 (0.0008) [2023-12-27 11:44:39,721][20347] Updated weights for policy 0, policy_version 880526 (0.0007) [2023-12-27 11:44:40,237][20381] Updated weights for policy 1, policy_version 879835 (0.0009) [2023-12-27 11:44:40,297][20381] Updated weights for policy 1, policy_version 879845 (0.0008) [2023-12-27 11:44:40,377][20381] Updated weights for policy 1, policy_version 879855 (0.0006) [2023-12-27 11:44:40,443][20347] Updated weights for policy 0, policy_version 880536 (0.0006) [2023-12-27 11:44:40,498][20347] Updated weights for policy 0, policy_version 880546 (0.0008) [2023-12-27 11:44:40,557][20347] Updated weights for policy 0, policy_version 880556 (0.0008) [2023-12-27 11:44:41,016][20381] Updated weights for policy 1, policy_version 879865 (0.0008) [2023-12-27 11:44:41,077][20381] Updated weights for policy 1, policy_version 879875 (0.0008) [2023-12-27 11:44:41,138][20381] Updated weights for policy 1, policy_version 879885 (0.0007) [2023-12-27 11:44:41,198][20381] Updated weights for policy 1, policy_version 879895 (0.0009) [2023-12-27 11:44:41,332][20347] Updated weights for policy 0, policy_version 880566 (0.0008) [2023-12-27 11:44:41,395][20347] Updated weights for policy 0, policy_version 880576 (0.0009) [2023-12-27 11:44:41,452][20347] Updated weights for policy 0, policy_version 880586 (0.0009) [2023-12-27 11:44:41,992][20381] Updated weights for policy 1, policy_version 879905 (0.0009) [2023-12-27 11:44:42,056][20381] Updated weights for policy 1, policy_version 879915 (0.0008) [2023-12-27 11:44:42,124][20381] Updated weights for policy 1, policy_version 879925 (0.0008) [2023-12-27 11:44:42,256][20347] Updated weights for policy 0, policy_version 880596 (0.0008) [2023-12-27 11:44:42,321][20347] Updated weights for policy 0, policy_version 880606 (0.0007) [2023-12-27 11:44:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 19883.0). Total num frames: 450748416. Throughput: 0: 10042.6, 1: 10143.0. Samples: 450743468. Policy #0 lag: (min: 18.0, avg: 31.6, max: 50.0) [2023-12-27 11:44:42,363][19170] Avg episode reward: [(0, '882.690'), (1, '990.250')] [2023-12-27 11:44:42,394][20347] Updated weights for policy 0, policy_version 880616 (0.0008) [2023-12-27 11:44:42,864][20381] Updated weights for policy 1, policy_version 879935 (0.0007) [2023-12-27 11:44:42,931][20381] Updated weights for policy 1, policy_version 879945 (0.0008) [2023-12-27 11:44:42,995][20381] Updated weights for policy 1, policy_version 879955 (0.0009) [2023-12-27 11:44:43,042][20347] Updated weights for policy 0, policy_version 880626 (0.0008) [2023-12-27 11:44:43,091][20347] Updated weights for policy 0, policy_version 880636 (0.0008) [2023-12-27 11:44:43,134][20347] Updated weights for policy 0, policy_version 880646 (0.0008) [2023-12-27 11:44:43,189][20347] Updated weights for policy 0, policy_version 880656 (0.0007) [2023-12-27 11:44:43,651][20381] Updated weights for policy 1, policy_version 879965 (0.0008) [2023-12-27 11:44:43,710][20381] Updated weights for policy 1, policy_version 879975 (0.0008) [2023-12-27 11:44:43,767][20381] Updated weights for policy 1, policy_version 879985 (0.0008) [2023-12-27 11:44:43,839][20347] Updated weights for policy 0, policy_version 880666 (0.0007) [2023-12-27 11:44:43,902][20347] Updated weights for policy 0, policy_version 880676 (0.0008) [2023-12-27 11:44:43,966][20347] Updated weights for policy 0, policy_version 880686 (0.0008) [2023-12-27 11:44:44,363][20381] Updated weights for policy 1, policy_version 879995 (0.0007) [2023-12-27 11:44:44,411][20381] Updated weights for policy 1, policy_version 880005 (0.0008) [2023-12-27 11:44:44,475][20381] Updated weights for policy 1, policy_version 880015 (0.0008) [2023-12-27 11:44:44,667][20347] Updated weights for policy 0, policy_version 880696 (0.0007) [2023-12-27 11:44:44,716][20347] Updated weights for policy 0, policy_version 880706 (0.0008) [2023-12-27 11:44:44,765][20347] Updated weights for policy 0, policy_version 880716 (0.0009) [2023-12-27 11:44:45,220][20381] Updated weights for policy 1, policy_version 880025 (0.0007) [2023-12-27 11:44:45,277][20381] Updated weights for policy 1, policy_version 880035 (0.0006) [2023-12-27 11:44:45,333][20381] Updated weights for policy 1, policy_version 880045 (0.0006) [2023-12-27 11:44:45,393][20381] Updated weights for policy 1, policy_version 880055 (0.0007) [2023-12-27 11:44:45,407][20347] Updated weights for policy 0, policy_version 880726 (0.0009) [2023-12-27 11:44:45,471][20347] Updated weights for policy 0, policy_version 880736 (0.0010) [2023-12-27 11:44:45,538][20347] Updated weights for policy 0, policy_version 880746 (0.0010) [2023-12-27 11:44:45,989][20381] Updated weights for policy 1, policy_version 880065 (0.0007) [2023-12-27 11:44:46,039][20381] Updated weights for policy 1, policy_version 880075 (0.0007) [2023-12-27 11:44:46,086][20381] Updated weights for policy 1, policy_version 880085 (0.0007) [2023-12-27 11:44:46,150][20347] Updated weights for policy 0, policy_version 880756 (0.0010) [2023-12-27 11:44:46,219][20347] Updated weights for policy 0, policy_version 880766 (0.0011) [2023-12-27 11:44:46,283][20347] Updated weights for policy 0, policy_version 880776 (0.0010) [2023-12-27 11:44:46,739][20381] Updated weights for policy 1, policy_version 880095 (0.0006) [2023-12-27 11:44:46,796][20381] Updated weights for policy 1, policy_version 880105 (0.0005) [2023-12-27 11:44:46,849][20381] Updated weights for policy 1, policy_version 880115 (0.0005) [2023-12-27 11:44:46,871][20347] Updated weights for policy 0, policy_version 880786 (0.0006) [2023-12-27 11:44:46,928][20347] Updated weights for policy 0, policy_version 880796 (0.0008) [2023-12-27 11:44:46,986][20347] Updated weights for policy 0, policy_version 880806 (0.0007) [2023-12-27 11:44:47,047][20347] Updated weights for policy 0, policy_version 880816 (0.0006) [2023-12-27 11:44:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 19938.5). Total num frames: 450854912. Throughput: 0: 10034.1, 1: 10111.3. Samples: 450865940. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:44:47,363][19170] Avg episode reward: [(0, '921.960'), (1, '1000.000')] [2023-12-27 11:44:47,489][20381] Updated weights for policy 1, policy_version 880125 (0.0007) [2023-12-27 11:44:47,556][20381] Updated weights for policy 1, policy_version 880135 (0.0008) [2023-12-27 11:44:47,602][20381] Updated weights for policy 1, policy_version 880145 (0.0008) [2023-12-27 11:44:47,672][20347] Updated weights for policy 0, policy_version 880826 (0.0007) [2023-12-27 11:44:47,732][20347] Updated weights for policy 0, policy_version 880836 (0.0007) [2023-12-27 11:44:47,779][20347] Updated weights for policy 0, policy_version 880846 (0.0008) [2023-12-27 11:44:48,343][20381] Updated weights for policy 1, policy_version 880155 (0.0008) [2023-12-27 11:44:48,403][20381] Updated weights for policy 1, policy_version 880165 (0.0008) [2023-12-27 11:44:48,411][20347] Updated weights for policy 0, policy_version 880856 (0.0006) [2023-12-27 11:44:48,448][20291] KL-divergence is very high: 427.9784 [2023-12-27 11:44:48,467][20381] Updated weights for policy 1, policy_version 880175 (0.0009) [2023-12-27 11:44:48,473][20347] Updated weights for policy 0, policy_version 880866 (0.0006) [2023-12-27 11:44:48,536][20347] Updated weights for policy 0, policy_version 880876 (0.0007) [2023-12-27 11:44:49,081][20381] Updated weights for policy 1, policy_version 880185 (0.0008) [2023-12-27 11:44:49,149][20381] Updated weights for policy 1, policy_version 880195 (0.0006) [2023-12-27 11:44:49,210][20381] Updated weights for policy 1, policy_version 880205 (0.0010) [2023-12-27 11:44:49,256][20347] Updated weights for policy 0, policy_version 880886 (0.0008) [2023-12-27 11:44:49,278][20381] Updated weights for policy 1, policy_version 880215 (0.0008) [2023-12-27 11:44:49,314][20347] Updated weights for policy 0, policy_version 880896 (0.0008) [2023-12-27 11:44:49,380][20347] Updated weights for policy 0, policy_version 880906 (0.0008) [2023-12-27 11:44:50,010][20381] Updated weights for policy 1, policy_version 880225 (0.0007) [2023-12-27 11:44:50,066][20381] Updated weights for policy 1, policy_version 880235 (0.0008) [2023-12-27 11:44:50,097][20347] Updated weights for policy 0, policy_version 880916 (0.0008) [2023-12-27 11:44:50,124][20381] Updated weights for policy 1, policy_version 880245 (0.0008) [2023-12-27 11:44:50,157][20347] Updated weights for policy 0, policy_version 880926 (0.0010) [2023-12-27 11:44:50,216][20347] Updated weights for policy 0, policy_version 880936 (0.0009) [2023-12-27 11:44:50,812][20381] Updated weights for policy 1, policy_version 880255 (0.0005) [2023-12-27 11:44:50,870][20381] Updated weights for policy 1, policy_version 880265 (0.0006) [2023-12-27 11:44:50,881][20347] Updated weights for policy 0, policy_version 880946 (0.0009) [2023-12-27 11:44:50,925][20381] Updated weights for policy 1, policy_version 880275 (0.0006) [2023-12-27 11:44:50,925][20291] KL-divergence is very high: 100.9395 [2023-12-27 11:44:50,944][20347] Updated weights for policy 0, policy_version 880956 (0.0010) [2023-12-27 11:44:51,011][20347] Updated weights for policy 0, policy_version 880966 (0.0009) [2023-12-27 11:44:51,082][20347] Updated weights for policy 0, policy_version 880976 (0.0007) [2023-12-27 11:44:51,621][20381] Updated weights for policy 1, policy_version 880285 (0.0007) [2023-12-27 11:44:51,686][20381] Updated weights for policy 1, policy_version 880295 (0.0006) [2023-12-27 11:44:51,756][20381] Updated weights for policy 1, policy_version 880305 (0.0007) [2023-12-27 11:44:51,828][20347] Updated weights for policy 0, policy_version 880986 (0.0008) [2023-12-27 11:44:51,894][20347] Updated weights for policy 0, policy_version 880996 (0.0008) [2023-12-27 11:44:51,956][20347] Updated weights for policy 0, policy_version 881006 (0.0007) [2023-12-27 11:44:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 19910.7). Total num frames: 450953216. Throughput: 0: 10006.5, 1: 10094.3. Samples: 450926088. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:44:52,363][19170] Avg episode reward: [(0, '902.500'), (1, '990.940')] [2023-12-27 11:44:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000880312_225386496.pth... [2023-12-27 11:44:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000881008_225566720.pth... [2023-12-27 11:44:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000879128_225083392.pth [2023-12-27 11:44:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000879824_225263616.pth [2023-12-27 11:44:52,452][20381] Updated weights for policy 1, policy_version 880315 (0.0006) [2023-12-27 11:44:52,506][20381] Updated weights for policy 1, policy_version 880325 (0.0008) [2023-12-27 11:44:52,561][20381] Updated weights for policy 1, policy_version 880335 (0.0006) [2023-12-27 11:44:52,563][20347] Updated weights for policy 0, policy_version 881016 (0.0006) [2023-12-27 11:44:52,617][20347] Updated weights for policy 0, policy_version 881026 (0.0010) [2023-12-27 11:44:52,672][20347] Updated weights for policy 0, policy_version 881036 (0.0010) [2023-12-27 11:44:53,169][20381] Updated weights for policy 1, policy_version 880345 (0.0007) [2023-12-27 11:44:53,220][20381] Updated weights for policy 1, policy_version 880355 (0.0007) [2023-12-27 11:44:53,273][20381] Updated weights for policy 1, policy_version 880365 (0.0007) [2023-12-27 11:44:53,330][20381] Updated weights for policy 1, policy_version 880375 (0.0006) [2023-12-27 11:44:53,352][20347] Updated weights for policy 0, policy_version 881046 (0.0007) [2023-12-27 11:44:53,414][20347] Updated weights for policy 0, policy_version 881056 (0.0008) [2023-12-27 11:44:53,477][20347] Updated weights for policy 0, policy_version 881066 (0.0008) [2023-12-27 11:44:54,030][20381] Updated weights for policy 1, policy_version 880385 (0.0006) [2023-12-27 11:44:54,031][20347] Updated weights for policy 0, policy_version 881076 (0.0008) [2023-12-27 11:44:54,087][20381] Updated weights for policy 1, policy_version 880395 (0.0005) [2023-12-27 11:44:54,099][20347] Updated weights for policy 0, policy_version 881086 (0.0008) [2023-12-27 11:44:54,149][20381] Updated weights for policy 1, policy_version 880405 (0.0006) [2023-12-27 11:44:54,152][20347] Updated weights for policy 0, policy_version 881096 (0.0010) [2023-12-27 11:44:54,815][20347] Updated weights for policy 0, policy_version 881106 (0.0010) [2023-12-27 11:44:54,862][20381] Updated weights for policy 1, policy_version 880415 (0.0006) [2023-12-27 11:44:54,877][20347] Updated weights for policy 0, policy_version 881116 (0.0010) [2023-12-27 11:44:54,921][20381] Updated weights for policy 1, policy_version 880425 (0.0006) [2023-12-27 11:44:54,932][20347] Updated weights for policy 0, policy_version 881126 (0.0010) [2023-12-27 11:44:54,976][20381] Updated weights for policy 1, policy_version 880435 (0.0008) [2023-12-27 11:44:54,992][20347] Updated weights for policy 0, policy_version 881136 (0.0009) [2023-12-27 11:44:55,597][20381] Updated weights for policy 1, policy_version 880445 (0.0007) [2023-12-27 11:44:55,643][20381] Updated weights for policy 1, policy_version 880455 (0.0006) [2023-12-27 11:44:55,667][20347] Updated weights for policy 0, policy_version 881146 (0.0010) [2023-12-27 11:44:55,692][20381] Updated weights for policy 1, policy_version 880465 (0.0005) [2023-12-27 11:44:55,721][20347] Updated weights for policy 0, policy_version 881156 (0.0010) [2023-12-27 11:44:55,779][20347] Updated weights for policy 0, policy_version 881166 (0.0010) [2023-12-27 11:44:56,219][20381] Updated weights for policy 1, policy_version 880475 (0.0005) [2023-12-27 11:44:56,268][20381] Updated weights for policy 1, policy_version 880485 (0.0007) [2023-12-27 11:44:56,314][20381] Updated weights for policy 1, policy_version 880495 (0.0010) [2023-12-27 11:44:56,414][20347] Updated weights for policy 0, policy_version 881176 (0.0006) [2023-12-27 11:44:56,465][20347] Updated weights for policy 0, policy_version 881186 (0.0005) [2023-12-27 11:44:56,516][20347] Updated weights for policy 0, policy_version 881196 (0.0008) [2023-12-27 11:44:56,974][20381] Updated weights for policy 1, policy_version 880505 (0.0010) [2023-12-27 11:44:57,033][20381] Updated weights for policy 1, policy_version 880515 (0.0008) [2023-12-27 11:44:57,086][20381] Updated weights for policy 1, policy_version 880525 (0.0005) [2023-12-27 11:44:57,141][20381] Updated weights for policy 1, policy_version 880535 (0.0009) [2023-12-27 11:44:57,172][20347] Updated weights for policy 0, policy_version 881206 (0.0010) [2023-12-27 11:44:57,226][20347] Updated weights for policy 0, policy_version 881216 (0.0009) [2023-12-27 11:44:57,284][20347] Updated weights for policy 0, policy_version 881226 (0.0010) [2023-12-27 11:44:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20021.8). Total num frames: 451067904. Throughput: 0: 10062.8, 1: 10138.3. Samples: 451050732. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:44:57,363][19170] Avg episode reward: [(0, '912.330'), (1, '981.260')] [2023-12-27 11:44:57,764][20381] Updated weights for policy 1, policy_version 880545 (0.0008) [2023-12-27 11:44:57,810][20381] Updated weights for policy 1, policy_version 880555 (0.0006) [2023-12-27 11:44:57,856][20381] Updated weights for policy 1, policy_version 880565 (0.0009) [2023-12-27 11:44:57,897][20347] Updated weights for policy 0, policy_version 881236 (0.0010) [2023-12-27 11:44:57,957][20347] Updated weights for policy 0, policy_version 881246 (0.0010) [2023-12-27 11:44:58,014][20347] Updated weights for policy 0, policy_version 881256 (0.0010) [2023-12-27 11:44:58,605][20381] Updated weights for policy 1, policy_version 880575 (0.0008) [2023-12-27 11:44:58,665][20381] Updated weights for policy 1, policy_version 880585 (0.0008) [2023-12-27 11:44:58,728][20381] Updated weights for policy 1, policy_version 880595 (0.0007) [2023-12-27 11:44:58,757][20347] Updated weights for policy 0, policy_version 881266 (0.0010) [2023-12-27 11:44:58,826][20347] Updated weights for policy 0, policy_version 881277 (0.0007) [2023-12-27 11:44:58,887][20347] Updated weights for policy 0, policy_version 881287 (0.0007) [2023-12-27 11:44:59,483][20381] Updated weights for policy 1, policy_version 880605 (0.0006) [2023-12-27 11:44:59,530][20381] Updated weights for policy 1, policy_version 880615 (0.0005) [2023-12-27 11:44:59,558][20347] Updated weights for policy 0, policy_version 881297 (0.0008) [2023-12-27 11:44:59,583][20381] Updated weights for policy 1, policy_version 880625 (0.0006) [2023-12-27 11:44:59,613][20291] KL-divergence is very high: 210.0036 [2023-12-27 11:44:59,618][20347] Updated weights for policy 0, policy_version 881307 (0.0008) [2023-12-27 11:44:59,663][20291] KL-divergence is very high: 178.1416 [2023-12-27 11:44:59,684][20347] Updated weights for policy 0, policy_version 881317 (0.0008) [2023-12-27 11:44:59,744][20347] Updated weights for policy 0, policy_version 881327 (0.0008) [2023-12-27 11:45:00,267][20381] Updated weights for policy 1, policy_version 880635 (0.0008) [2023-12-27 11:45:00,327][20381] Updated weights for policy 1, policy_version 880645 (0.0008) [2023-12-27 11:45:00,389][20381] Updated weights for policy 1, policy_version 880655 (0.0007) [2023-12-27 11:45:00,479][20347] Updated weights for policy 0, policy_version 881337 (0.0007) [2023-12-27 11:45:00,530][20347] Updated weights for policy 0, policy_version 881347 (0.0007) [2023-12-27 11:45:00,582][20347] Updated weights for policy 0, policy_version 881357 (0.0007) [2023-12-27 11:45:01,092][20381] Updated weights for policy 1, policy_version 880665 (0.0007) [2023-12-27 11:45:01,158][20381] Updated weights for policy 1, policy_version 880675 (0.0008) [2023-12-27 11:45:01,222][20381] Updated weights for policy 1, policy_version 880685 (0.0007) [2023-12-27 11:45:01,284][20381] Updated weights for policy 1, policy_version 880695 (0.0008) [2023-12-27 11:45:01,352][20347] Updated weights for policy 0, policy_version 881367 (0.0008) [2023-12-27 11:45:01,375][20291] KL-divergence is very high: 157.2099 [2023-12-27 11:45:01,393][20291] KL-divergence is very high: 151.6209 [2023-12-27 11:45:01,417][20347] Updated weights for policy 0, policy_version 881377 (0.0009) [2023-12-27 11:45:01,422][20291] KL-divergence is very high: 174.3997 [2023-12-27 11:45:01,470][20347] Updated weights for policy 0, policy_version 881387 (0.0009) [2023-12-27 11:45:02,004][20381] Updated weights for policy 1, policy_version 880705 (0.0008) [2023-12-27 11:45:02,060][20381] Updated weights for policy 1, policy_version 880715 (0.0008) [2023-12-27 11:45:02,123][20381] Updated weights for policy 1, policy_version 880725 (0.0008) [2023-12-27 11:45:02,197][20347] Updated weights for policy 0, policy_version 881397 (0.0010) [2023-12-27 11:45:02,253][20347] Updated weights for policy 0, policy_version 881407 (0.0010) [2023-12-27 11:45:02,311][20347] Updated weights for policy 0, policy_version 881417 (0.0010) [2023-12-27 11:45:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20207.0, 300 sec: 20021.8). Total num frames: 451166208. Throughput: 0: 10143.3, 1: 10148.9. Samples: 451170712. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:02,363][19170] Avg episode reward: [(0, '902.380'), (1, '980.670')] [2023-12-27 11:45:02,851][20381] Updated weights for policy 1, policy_version 880735 (0.0008) [2023-12-27 11:45:02,908][20381] Updated weights for policy 1, policy_version 880745 (0.0006) [2023-12-27 11:45:02,973][20381] Updated weights for policy 1, policy_version 880755 (0.0006) [2023-12-27 11:45:02,987][20347] Updated weights for policy 0, policy_version 881427 (0.0009) [2023-12-27 11:45:03,047][20347] Updated weights for policy 0, policy_version 881437 (0.0009) [2023-12-27 11:45:03,102][20347] Updated weights for policy 0, policy_version 881447 (0.0006) [2023-12-27 11:45:03,592][20381] Updated weights for policy 1, policy_version 880765 (0.0006) [2023-12-27 11:45:03,641][20381] Updated weights for policy 1, policy_version 880775 (0.0005) [2023-12-27 11:45:03,652][20347] Updated weights for policy 0, policy_version 881457 (0.0010) [2023-12-27 11:45:03,695][20381] Updated weights for policy 1, policy_version 880785 (0.0005) [2023-12-27 11:45:03,706][20347] Updated weights for policy 0, policy_version 881467 (0.0005) [2023-12-27 11:45:03,755][20347] Updated weights for policy 0, policy_version 881477 (0.0007) [2023-12-27 11:45:03,800][20347] Updated weights for policy 0, policy_version 881487 (0.0010) [2023-12-27 11:45:04,413][20381] Updated weights for policy 1, policy_version 880795 (0.0006) [2023-12-27 11:45:04,469][20381] Updated weights for policy 1, policy_version 880805 (0.0007) [2023-12-27 11:45:04,498][20347] Updated weights for policy 0, policy_version 881497 (0.0011) [2023-12-27 11:45:04,524][20381] Updated weights for policy 1, policy_version 880815 (0.0008) [2023-12-27 11:45:04,559][20347] Updated weights for policy 0, policy_version 881507 (0.0010) [2023-12-27 11:45:04,622][20347] Updated weights for policy 0, policy_version 881517 (0.0010) [2023-12-27 11:45:05,090][20381] Updated weights for policy 1, policy_version 880825 (0.0008) [2023-12-27 11:45:05,151][20381] Updated weights for policy 1, policy_version 880835 (0.0005) [2023-12-27 11:45:05,196][20381] Updated weights for policy 1, policy_version 880845 (0.0008) [2023-12-27 11:45:05,237][20347] Updated weights for policy 0, policy_version 881527 (0.0007) [2023-12-27 11:45:05,250][20381] Updated weights for policy 1, policy_version 880855 (0.0006) [2023-12-27 11:45:05,293][20347] Updated weights for policy 0, policy_version 881537 (0.0007) [2023-12-27 11:45:05,351][20347] Updated weights for policy 0, policy_version 881547 (0.0007) [2023-12-27 11:45:05,936][20347] Updated weights for policy 0, policy_version 881557 (0.0008) [2023-12-27 11:45:05,947][20381] Updated weights for policy 1, policy_version 880865 (0.0010) [2023-12-27 11:45:05,993][20381] Updated weights for policy 1, policy_version 880875 (0.0011) [2023-12-27 11:45:05,993][20347] Updated weights for policy 0, policy_version 881567 (0.0007) [2023-12-27 11:45:06,044][20381] Updated weights for policy 1, policy_version 880885 (0.0009) [2023-12-27 11:45:06,050][20347] Updated weights for policy 0, policy_version 881577 (0.0007) [2023-12-27 11:45:06,742][20347] Updated weights for policy 0, policy_version 881587 (0.0006) [2023-12-27 11:45:06,769][20381] Updated weights for policy 1, policy_version 880895 (0.0009) [2023-12-27 11:45:06,791][20292] KL-divergence is very high: 125.7333 [2023-12-27 11:45:06,801][20347] Updated weights for policy 0, policy_version 881597 (0.0006) [2023-12-27 11:45:06,825][20381] Updated weights for policy 1, policy_version 880905 (0.0011) [2023-12-27 11:45:06,850][20347] Updated weights for policy 0, policy_version 881607 (0.0005) [2023-12-27 11:45:06,882][20381] Updated weights for policy 1, policy_version 880915 (0.0011) [2023-12-27 11:45:06,883][20292] KL-divergence is very high: 158.4933 [2023-12-27 11:45:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20021.8). Total num frames: 451264512. Throughput: 0: 10221.1, 1: 10217.5. Samples: 451235284. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:07,363][19170] Avg episode reward: [(0, '883.010'), (1, '980.700')] [2023-12-27 11:45:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000880920_225542144.pth... [2023-12-27 11:45:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000881616_225722368.pth... [2023-12-27 11:45:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000879736_225239040.pth [2023-12-27 11:45:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000880432_225419264.pth [2023-12-27 11:45:07,500][20347] Updated weights for policy 0, policy_version 881617 (0.0005) [2023-12-27 11:45:07,553][20347] Updated weights for policy 0, policy_version 881627 (0.0007) [2023-12-27 11:45:07,576][20381] Updated weights for policy 1, policy_version 880925 (0.0011) [2023-12-27 11:45:07,605][20347] Updated weights for policy 0, policy_version 881637 (0.0008) [2023-12-27 11:45:07,625][20381] Updated weights for policy 1, policy_version 880935 (0.0011) [2023-12-27 11:45:07,660][20347] Updated weights for policy 0, policy_version 881647 (0.0008) [2023-12-27 11:45:07,681][20381] Updated weights for policy 1, policy_version 880945 (0.0011) [2023-12-27 11:45:08,327][20291] KL-divergence is very high: 107.3116 [2023-12-27 11:45:08,334][20347] Updated weights for policy 0, policy_version 881657 (0.0009) [2023-12-27 11:45:08,389][20381] Updated weights for policy 1, policy_version 880955 (0.0009) [2023-12-27 11:45:08,400][20347] Updated weights for policy 0, policy_version 881667 (0.0009) [2023-12-27 11:45:08,447][20381] Updated weights for policy 1, policy_version 880965 (0.0008) [2023-12-27 11:45:08,458][20347] Updated weights for policy 0, policy_version 881677 (0.0006) [2023-12-27 11:45:08,507][20381] Updated weights for policy 1, policy_version 880975 (0.0008) [2023-12-27 11:45:09,115][20347] Updated weights for policy 0, policy_version 881687 (0.0007) [2023-12-27 11:45:09,162][20347] Updated weights for policy 0, policy_version 881697 (0.0007) [2023-12-27 11:45:09,215][20381] Updated weights for policy 1, policy_version 880985 (0.0009) [2023-12-27 11:45:09,229][20347] Updated weights for policy 0, policy_version 881707 (0.0007) [2023-12-27 11:45:09,230][20291] KL-divergence is very high: 145.7772 [2023-12-27 11:45:09,283][20381] Updated weights for policy 1, policy_version 880995 (0.0009) [2023-12-27 11:45:09,348][20381] Updated weights for policy 1, policy_version 881005 (0.0008) [2023-12-27 11:45:09,411][20381] Updated weights for policy 1, policy_version 881015 (0.0011) [2023-12-27 11:45:09,980][20347] Updated weights for policy 0, policy_version 881717 (0.0008) [2023-12-27 11:45:10,045][20347] Updated weights for policy 0, policy_version 881727 (0.0008) [2023-12-27 11:45:10,110][20347] Updated weights for policy 0, policy_version 881737 (0.0008) [2023-12-27 11:45:10,156][20381] Updated weights for policy 1, policy_version 881025 (0.0009) [2023-12-27 11:45:10,223][20381] Updated weights for policy 1, policy_version 881035 (0.0008) [2023-12-27 11:45:10,283][20381] Updated weights for policy 1, policy_version 881045 (0.0008) [2023-12-27 11:45:10,821][20347] Updated weights for policy 0, policy_version 881747 (0.0006) [2023-12-27 11:45:10,874][20347] Updated weights for policy 0, policy_version 881757 (0.0010) [2023-12-27 11:45:10,922][20347] Updated weights for policy 0, policy_version 881767 (0.0010) [2023-12-27 11:45:10,983][20381] Updated weights for policy 1, policy_version 881055 (0.0009) [2023-12-27 11:45:11,042][20381] Updated weights for policy 1, policy_version 881065 (0.0008) [2023-12-27 11:45:11,105][20381] Updated weights for policy 1, policy_version 881076 (0.0008) [2023-12-27 11:45:11,624][20347] Updated weights for policy 0, policy_version 881777 (0.0010) [2023-12-27 11:45:11,688][20347] Updated weights for policy 0, policy_version 881787 (0.0009) [2023-12-27 11:45:11,752][20347] Updated weights for policy 0, policy_version 881797 (0.0008) [2023-12-27 11:45:11,809][20347] Updated weights for policy 0, policy_version 881807 (0.0008) [2023-12-27 11:45:11,834][20381] Updated weights for policy 1, policy_version 881086 (0.0008) [2023-12-27 11:45:11,889][20381] Updated weights for policy 1, policy_version 881096 (0.0008) [2023-12-27 11:45:11,939][20381] Updated weights for policy 1, policy_version 881106 (0.0007) [2023-12-27 11:45:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20021.8). Total num frames: 451362816. Throughput: 0: 10252.9, 1: 10195.3. Samples: 451354276. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:12,363][19170] Avg episode reward: [(0, '785.860'), (1, '980.660')] [2023-12-27 11:45:12,555][20347] Updated weights for policy 0, policy_version 881817 (0.0008) [2023-12-27 11:45:12,612][20347] Updated weights for policy 0, policy_version 881827 (0.0008) [2023-12-27 11:45:12,675][20347] Updated weights for policy 0, policy_version 881837 (0.0007) [2023-12-27 11:45:12,706][20381] Updated weights for policy 1, policy_version 881116 (0.0007) [2023-12-27 11:45:12,768][20381] Updated weights for policy 1, policy_version 881126 (0.0007) [2023-12-27 11:45:12,824][20381] Updated weights for policy 1, policy_version 881136 (0.0008) [2023-12-27 11:45:13,332][20347] Updated weights for policy 0, policy_version 881847 (0.0006) [2023-12-27 11:45:13,394][20347] Updated weights for policy 0, policy_version 881857 (0.0008) [2023-12-27 11:45:13,441][20347] Updated weights for policy 0, policy_version 881867 (0.0007) [2023-12-27 11:45:13,547][20381] Updated weights for policy 1, policy_version 881146 (0.0008) [2023-12-27 11:45:13,596][20381] Updated weights for policy 1, policy_version 881156 (0.0010) [2023-12-27 11:45:13,644][20381] Updated weights for policy 1, policy_version 881166 (0.0010) [2023-12-27 11:45:13,693][20381] Updated weights for policy 1, policy_version 881176 (0.0010) [2023-12-27 11:45:14,101][20347] Updated weights for policy 0, policy_version 881877 (0.0006) [2023-12-27 11:45:14,159][20347] Updated weights for policy 0, policy_version 881887 (0.0005) [2023-12-27 11:45:14,218][20347] Updated weights for policy 0, policy_version 881897 (0.0005) [2023-12-27 11:45:14,262][20381] Updated weights for policy 1, policy_version 881186 (0.0011) [2023-12-27 11:45:14,318][20381] Updated weights for policy 1, policy_version 881196 (0.0011) [2023-12-27 11:45:14,367][20381] Updated weights for policy 1, policy_version 881206 (0.0010) [2023-12-27 11:45:14,826][20347] Updated weights for policy 0, policy_version 881907 (0.0006) [2023-12-27 11:45:14,886][20347] Updated weights for policy 0, policy_version 881917 (0.0008) [2023-12-27 11:45:14,953][20347] Updated weights for policy 0, policy_version 881927 (0.0006) [2023-12-27 11:45:15,109][20381] Updated weights for policy 1, policy_version 881216 (0.0011) [2023-12-27 11:45:15,172][20381] Updated weights for policy 1, policy_version 881226 (0.0011) [2023-12-27 11:45:15,239][20381] Updated weights for policy 1, policy_version 881236 (0.0011) [2023-12-27 11:45:15,580][20347] Updated weights for policy 0, policy_version 881937 (0.0006) [2023-12-27 11:45:15,640][20347] Updated weights for policy 0, policy_version 881947 (0.0008) [2023-12-27 11:45:15,704][20347] Updated weights for policy 0, policy_version 881957 (0.0009) [2023-12-27 11:45:15,762][20347] Updated weights for policy 0, policy_version 881967 (0.0007) [2023-12-27 11:45:15,884][20381] Updated weights for policy 1, policy_version 881246 (0.0011) [2023-12-27 11:45:15,942][20381] Updated weights for policy 1, policy_version 881256 (0.0010) [2023-12-27 11:45:16,003][20381] Updated weights for policy 1, policy_version 881266 (0.0010) [2023-12-27 11:45:16,480][20347] Updated weights for policy 0, policy_version 881977 (0.0008) [2023-12-27 11:45:16,533][20347] Updated weights for policy 0, policy_version 881987 (0.0007) [2023-12-27 11:45:16,567][20381] Updated weights for policy 1, policy_version 881276 (0.0008) [2023-12-27 11:45:16,584][20347] Updated weights for policy 0, policy_version 881997 (0.0007) [2023-12-27 11:45:16,612][20381] Updated weights for policy 1, policy_version 881286 (0.0010) [2023-12-27 11:45:16,669][20381] Updated weights for policy 1, policy_version 881296 (0.0008) [2023-12-27 11:45:17,244][20347] Updated weights for policy 0, policy_version 882007 (0.0005) [2023-12-27 11:45:17,264][20291] KL-divergence is very high: 152.7593 [2023-12-27 11:45:17,274][20291] KL-divergence is very high: 132.7896 [2023-12-27 11:45:17,291][20347] Updated weights for policy 0, policy_version 882017 (0.0005) [2023-12-27 11:45:17,309][20381] Updated weights for policy 1, policy_version 881306 (0.0010) [2023-12-27 11:45:17,343][20347] Updated weights for policy 0, policy_version 882027 (0.0007) [2023-12-27 11:45:17,363][20381] Updated weights for policy 1, policy_version 881316 (0.0010) [2023-12-27 11:45:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20206.9, 300 sec: 20021.8). Total num frames: 451461120. Throughput: 0: 10225.2, 1: 10220.5. Samples: 451477584. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:17,363][19170] Avg episode reward: [(0, '795.830'), (1, '980.660')] [2023-12-27 11:45:17,423][20381] Updated weights for policy 1, policy_version 881326 (0.0010) [2023-12-27 11:45:17,481][20381] Updated weights for policy 1, policy_version 881336 (0.0010) [2023-12-27 11:45:17,919][20347] Updated weights for policy 0, policy_version 882037 (0.0006) [2023-12-27 11:45:17,947][20291] KL-divergence is very high: 138.3354 [2023-12-27 11:45:17,984][20347] Updated weights for policy 0, policy_version 882047 (0.0006) [2023-12-27 11:45:18,041][20381] Updated weights for policy 1, policy_version 881346 (0.0011) [2023-12-27 11:45:18,050][20347] Updated weights for policy 0, policy_version 882057 (0.0006) [2023-12-27 11:45:18,058][20291] KL-divergence is very high: 110.2139 [2023-12-27 11:45:18,099][20381] Updated weights for policy 1, policy_version 881356 (0.0010) [2023-12-27 11:45:18,151][20381] Updated weights for policy 1, policy_version 881366 (0.0010) [2023-12-27 11:45:18,686][20347] Updated weights for policy 0, policy_version 882067 (0.0006) [2023-12-27 11:45:18,717][20291] KL-divergence is very high: 457.4545 [2023-12-27 11:45:18,723][20291] KL-divergence is very high: 108.4180 [2023-12-27 11:45:18,745][20347] Updated weights for policy 0, policy_version 882077 (0.0008) [2023-12-27 11:45:18,761][20291] KL-divergence is very high: 170.2877 [2023-12-27 11:45:18,797][20347] Updated weights for policy 0, policy_version 882087 (0.0007) [2023-12-27 11:45:18,864][20381] Updated weights for policy 1, policy_version 881376 (0.0008) [2023-12-27 11:45:18,921][20381] Updated weights for policy 1, policy_version 881386 (0.0009) [2023-12-27 11:45:18,967][20381] Updated weights for policy 1, policy_version 881396 (0.0007) [2023-12-27 11:45:19,452][20347] Updated weights for policy 0, policy_version 882097 (0.0009) [2023-12-27 11:45:19,522][20347] Updated weights for policy 0, policy_version 882107 (0.0008) [2023-12-27 11:45:19,584][20347] Updated weights for policy 0, policy_version 882117 (0.0006) [2023-12-27 11:45:19,646][20347] Updated weights for policy 0, policy_version 882127 (0.0007) [2023-12-27 11:45:19,777][20381] Updated weights for policy 1, policy_version 881406 (0.0008) [2023-12-27 11:45:19,836][20381] Updated weights for policy 1, policy_version 881416 (0.0008) [2023-12-27 11:45:19,905][20381] Updated weights for policy 1, policy_version 881426 (0.0008) [2023-12-27 11:45:20,316][20347] Updated weights for policy 0, policy_version 882137 (0.0007) [2023-12-27 11:45:20,353][20291] KL-divergence is very high: 133.3124 [2023-12-27 11:45:20,377][20347] Updated weights for policy 0, policy_version 882147 (0.0007) [2023-12-27 11:45:20,430][20347] Updated weights for policy 0, policy_version 882157 (0.0008) [2023-12-27 11:45:20,623][20381] Updated weights for policy 1, policy_version 881436 (0.0008) [2023-12-27 11:45:20,686][20381] Updated weights for policy 1, policy_version 881446 (0.0007) [2023-12-27 11:45:20,739][20381] Updated weights for policy 1, policy_version 881456 (0.0008) [2023-12-27 11:45:21,221][20347] Updated weights for policy 0, policy_version 882167 (0.0007) [2023-12-27 11:45:21,283][20347] Updated weights for policy 0, policy_version 882177 (0.0008) [2023-12-27 11:45:21,333][20347] Updated weights for policy 0, policy_version 882187 (0.0007) [2023-12-27 11:45:21,441][20381] Updated weights for policy 1, policy_version 881466 (0.0007) [2023-12-27 11:45:21,498][20381] Updated weights for policy 1, policy_version 881476 (0.0008) [2023-12-27 11:45:21,558][20381] Updated weights for policy 1, policy_version 881486 (0.0008) [2023-12-27 11:45:21,612][20381] Updated weights for policy 1, policy_version 881496 (0.0008) [2023-12-27 11:45:22,096][20347] Updated weights for policy 0, policy_version 882197 (0.0008) [2023-12-27 11:45:22,153][20347] Updated weights for policy 0, policy_version 882207 (0.0008) [2023-12-27 11:45:22,202][20347] Updated weights for policy 0, policy_version 882217 (0.0008) [2023-12-27 11:45:22,218][20291] KL-divergence is very high: 101.9693 [2023-12-27 11:45:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20049.6). Total num frames: 451567616. Throughput: 0: 10248.3, 1: 10184.8. Samples: 451539100. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:22,363][19170] Avg episode reward: [(0, '795.500'), (1, '1000.000')] [2023-12-27 11:45:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000882224_225878016.pth... [2023-12-27 11:45:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000881008_225566720.pth [2023-12-27 11:45:22,382][20381] Updated weights for policy 1, policy_version 881506 (0.0009) [2023-12-27 11:45:22,435][20381] Updated weights for policy 1, policy_version 881516 (0.0008) [2023-12-27 11:45:22,494][20381] Updated weights for policy 1, policy_version 881526 (0.0007) [2023-12-27 11:45:22,503][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000881528_225697792.pth... [2023-12-27 11:45:22,506][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000880312_225386496.pth [2023-12-27 11:45:22,940][20347] Updated weights for policy 0, policy_version 882227 (0.0007) [2023-12-27 11:45:23,003][20347] Updated weights for policy 0, policy_version 882237 (0.0006) [2023-12-27 11:45:23,062][20347] Updated weights for policy 0, policy_version 882247 (0.0005) [2023-12-27 11:45:23,231][20381] Updated weights for policy 1, policy_version 881536 (0.0008) [2023-12-27 11:45:23,282][20381] Updated weights for policy 1, policy_version 881546 (0.0007) [2023-12-27 11:45:23,337][20381] Updated weights for policy 1, policy_version 881556 (0.0008) [2023-12-27 11:45:23,683][20347] Updated weights for policy 0, policy_version 882257 (0.0006) [2023-12-27 11:45:23,741][20347] Updated weights for policy 0, policy_version 882267 (0.0006) [2023-12-27 11:45:23,796][20347] Updated weights for policy 0, policy_version 882277 (0.0008) [2023-12-27 11:45:23,850][20347] Updated weights for policy 0, policy_version 882287 (0.0008) [2023-12-27 11:45:24,037][20381] Updated weights for policy 1, policy_version 881566 (0.0008) [2023-12-27 11:45:24,088][20381] Updated weights for policy 1, policy_version 881576 (0.0008) [2023-12-27 11:45:24,132][20381] Updated weights for policy 1, policy_version 881586 (0.0008) [2023-12-27 11:45:24,529][20347] Updated weights for policy 0, policy_version 882297 (0.0005) [2023-12-27 11:45:24,584][20347] Updated weights for policy 0, policy_version 882307 (0.0007) [2023-12-27 11:45:24,637][20347] Updated weights for policy 0, policy_version 882317 (0.0007) [2023-12-27 11:45:24,880][20381] Updated weights for policy 1, policy_version 881596 (0.0008) [2023-12-27 11:45:24,933][20381] Updated weights for policy 1, policy_version 881606 (0.0008) [2023-12-27 11:45:24,982][20381] Updated weights for policy 1, policy_version 881616 (0.0008) [2023-12-27 11:45:25,362][20347] Updated weights for policy 0, policy_version 882327 (0.0007) [2023-12-27 11:45:25,413][20347] Updated weights for policy 0, policy_version 882337 (0.0008) [2023-12-27 11:45:25,464][20347] Updated weights for policy 0, policy_version 882347 (0.0007) [2023-12-27 11:45:25,672][20381] Updated weights for policy 1, policy_version 881626 (0.0008) [2023-12-27 11:45:25,734][20381] Updated weights for policy 1, policy_version 881636 (0.0008) [2023-12-27 11:45:25,786][20381] Updated weights for policy 1, policy_version 881646 (0.0007) [2023-12-27 11:45:25,847][20381] Updated weights for policy 1, policy_version 881656 (0.0008) [2023-12-27 11:45:26,168][20347] Updated weights for policy 0, policy_version 882357 (0.0007) [2023-12-27 11:45:26,221][20347] Updated weights for policy 0, policy_version 882367 (0.0008) [2023-12-27 11:45:26,274][20347] Updated weights for policy 0, policy_version 882377 (0.0007) [2023-12-27 11:45:26,539][20381] Updated weights for policy 1, policy_version 881666 (0.0008) [2023-12-27 11:45:26,592][20381] Updated weights for policy 1, policy_version 881676 (0.0008) [2023-12-27 11:45:26,643][20381] Updated weights for policy 1, policy_version 881686 (0.0007) [2023-12-27 11:45:27,000][20347] Updated weights for policy 0, policy_version 882387 (0.0008) [2023-12-27 11:45:27,066][20347] Updated weights for policy 0, policy_version 882397 (0.0008) [2023-12-27 11:45:27,120][20347] Updated weights for policy 0, policy_version 882407 (0.0007) [2023-12-27 11:45:27,133][20291] KL-divergence is very high: 118.9645 [2023-12-27 11:45:27,138][20291] KL-divergence is very high: 108.6634 [2023-12-27 11:45:27,279][20381] Updated weights for policy 1, policy_version 881696 (0.0006) [2023-12-27 11:45:27,338][20381] Updated weights for policy 1, policy_version 881706 (0.0007) [2023-12-27 11:45:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20049.6). Total num frames: 451665920. Throughput: 0: 10217.0, 1: 10093.5. Samples: 451657440. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:27,363][19170] Avg episode reward: [(0, '853.890'), (1, '1000.000')] [2023-12-27 11:45:27,393][20381] Updated weights for policy 1, policy_version 881716 (0.0008) [2023-12-27 11:45:27,814][20347] Updated weights for policy 0, policy_version 882417 (0.0007) [2023-12-27 11:45:27,862][20347] Updated weights for policy 0, policy_version 882427 (0.0008) [2023-12-27 11:45:27,911][20347] Updated weights for policy 0, policy_version 882437 (0.0008) [2023-12-27 11:45:27,968][20347] Updated weights for policy 0, policy_version 882447 (0.0008) [2023-12-27 11:45:28,032][20381] Updated weights for policy 1, policy_version 881726 (0.0008) [2023-12-27 11:45:28,081][20381] Updated weights for policy 1, policy_version 881736 (0.0008) [2023-12-27 11:45:28,130][20381] Updated weights for policy 1, policy_version 881746 (0.0008) [2023-12-27 11:45:28,673][20347] Updated weights for policy 0, policy_version 882457 (0.0008) [2023-12-27 11:45:28,735][20347] Updated weights for policy 0, policy_version 882467 (0.0007) [2023-12-27 11:45:28,792][20347] Updated weights for policy 0, policy_version 882477 (0.0008) [2023-12-27 11:45:28,918][20381] Updated weights for policy 1, policy_version 881756 (0.0008) [2023-12-27 11:45:28,970][20381] Updated weights for policy 1, policy_version 881766 (0.0008) [2023-12-27 11:45:29,023][20381] Updated weights for policy 1, policy_version 881776 (0.0008) [2023-12-27 11:45:29,473][20347] Updated weights for policy 0, policy_version 882487 (0.0008) [2023-12-27 11:45:29,527][20347] Updated weights for policy 0, policy_version 882497 (0.0008) [2023-12-27 11:45:29,580][20347] Updated weights for policy 0, policy_version 882507 (0.0008) [2023-12-27 11:45:29,768][20381] Updated weights for policy 1, policy_version 881786 (0.0007) [2023-12-27 11:45:29,834][20381] Updated weights for policy 1, policy_version 881796 (0.0008) [2023-12-27 11:45:29,897][20381] Updated weights for policy 1, policy_version 881806 (0.0008) [2023-12-27 11:45:29,963][20381] Updated weights for policy 1, policy_version 881816 (0.0008) [2023-12-27 11:45:30,384][20347] Updated weights for policy 0, policy_version 882517 (0.0008) [2023-12-27 11:45:30,446][20347] Updated weights for policy 0, policy_version 882527 (0.0008) [2023-12-27 11:45:30,492][20291] KL-divergence is very high: 162.4773 [2023-12-27 11:45:30,505][20347] Updated weights for policy 0, policy_version 882537 (0.0008) [2023-12-27 11:45:30,535][20291] KL-divergence is very high: 129.6967 [2023-12-27 11:45:30,710][20381] Updated weights for policy 1, policy_version 881826 (0.0010) [2023-12-27 11:45:30,769][20381] Updated weights for policy 1, policy_version 881836 (0.0006) [2023-12-27 11:45:30,823][20381] Updated weights for policy 1, policy_version 881846 (0.0006) [2023-12-27 11:45:31,190][20347] Updated weights for policy 0, policy_version 882547 (0.0007) [2023-12-27 11:45:31,248][20347] Updated weights for policy 0, policy_version 882557 (0.0009) [2023-12-27 11:45:31,314][20347] Updated weights for policy 0, policy_version 882567 (0.0007) [2023-12-27 11:45:31,511][20381] Updated weights for policy 1, policy_version 881856 (0.0006) [2023-12-27 11:45:31,566][20381] Updated weights for policy 1, policy_version 881866 (0.0006) [2023-12-27 11:45:31,629][20381] Updated weights for policy 1, policy_version 881876 (0.0007) [2023-12-27 11:45:31,993][20347] Updated weights for policy 0, policy_version 882577 (0.0008) [2023-12-27 11:45:32,050][20347] Updated weights for policy 0, policy_version 882587 (0.0006) [2023-12-27 11:45:32,110][20347] Updated weights for policy 0, policy_version 882597 (0.0007) [2023-12-27 11:45:32,170][20347] Updated weights for policy 0, policy_version 882607 (0.0008) [2023-12-27 11:45:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20049.6). Total num frames: 451764224. Throughput: 0: 10171.2, 1: 10048.7. Samples: 451775836. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:32,363][19170] Avg episode reward: [(0, '807.420'), (1, '1000.000')] [2023-12-27 11:45:32,386][20381] Updated weights for policy 1, policy_version 881886 (0.0012) [2023-12-27 11:45:32,451][20381] Updated weights for policy 1, policy_version 881896 (0.0007) [2023-12-27 11:45:32,514][20381] Updated weights for policy 1, policy_version 881906 (0.0007) [2023-12-27 11:45:32,916][20347] Updated weights for policy 0, policy_version 882617 (0.0008) [2023-12-27 11:45:32,975][20347] Updated weights for policy 0, policy_version 882627 (0.0005) [2023-12-27 11:45:33,038][20347] Updated weights for policy 0, policy_version 882637 (0.0006) [2023-12-27 11:45:33,195][20381] Updated weights for policy 1, policy_version 881916 (0.0008) [2023-12-27 11:45:33,249][20381] Updated weights for policy 1, policy_version 881926 (0.0008) [2023-12-27 11:45:33,298][20381] Updated weights for policy 1, policy_version 881936 (0.0008) [2023-12-27 11:45:33,692][20347] Updated weights for policy 0, policy_version 882647 (0.0008) [2023-12-27 11:45:33,744][20347] Updated weights for policy 0, policy_version 882657 (0.0007) [2023-12-27 11:45:33,797][20347] Updated weights for policy 0, policy_version 882667 (0.0006) [2023-12-27 11:45:33,977][20381] Updated weights for policy 1, policy_version 881946 (0.0007) [2023-12-27 11:45:34,027][20381] Updated weights for policy 1, policy_version 881956 (0.0006) [2023-12-27 11:45:34,080][20381] Updated weights for policy 1, policy_version 881966 (0.0007) [2023-12-27 11:45:34,131][20381] Updated weights for policy 1, policy_version 881976 (0.0008) [2023-12-27 11:45:34,543][20347] Updated weights for policy 0, policy_version 882677 (0.0007) [2023-12-27 11:45:34,610][20347] Updated weights for policy 0, policy_version 882687 (0.0008) [2023-12-27 11:45:34,673][20347] Updated weights for policy 0, policy_version 882697 (0.0008) [2023-12-27 11:45:34,797][20381] Updated weights for policy 1, policy_version 881986 (0.0006) [2023-12-27 11:45:34,861][20381] Updated weights for policy 1, policy_version 881996 (0.0009) [2023-12-27 11:45:34,922][20381] Updated weights for policy 1, policy_version 882006 (0.0011) [2023-12-27 11:45:35,335][20347] Updated weights for policy 0, policy_version 882707 (0.0008) [2023-12-27 11:45:35,388][20347] Updated weights for policy 0, policy_version 882717 (0.0007) [2023-12-27 11:45:35,451][20347] Updated weights for policy 0, policy_version 882727 (0.0005) [2023-12-27 11:45:35,595][20381] Updated weights for policy 1, policy_version 882016 (0.0011) [2023-12-27 11:45:35,650][20381] Updated weights for policy 1, policy_version 882026 (0.0010) [2023-12-27 11:45:35,698][20381] Updated weights for policy 1, policy_version 882036 (0.0010) [2023-12-27 11:45:36,009][20347] Updated weights for policy 0, policy_version 882737 (0.0005) [2023-12-27 11:45:36,064][20347] Updated weights for policy 0, policy_version 882747 (0.0008) [2023-12-27 11:45:36,089][20291] KL-divergence is very high: 112.6353 [2023-12-27 11:45:36,123][20347] Updated weights for policy 0, policy_version 882757 (0.0007) [2023-12-27 11:45:36,140][20291] KL-divergence is very high: 147.5347 [2023-12-27 11:45:36,173][20347] Updated weights for policy 0, policy_version 882767 (0.0008) [2023-12-27 11:45:36,347][20381] Updated weights for policy 1, policy_version 882046 (0.0010) [2023-12-27 11:45:36,403][20381] Updated weights for policy 1, policy_version 882056 (0.0010) [2023-12-27 11:45:36,462][20381] Updated weights for policy 1, policy_version 882066 (0.0011) [2023-12-27 11:45:36,921][20347] Updated weights for policy 0, policy_version 882777 (0.0006) [2023-12-27 11:45:36,980][20347] Updated weights for policy 0, policy_version 882787 (0.0005) [2023-12-27 11:45:37,031][20347] Updated weights for policy 0, policy_version 882797 (0.0005) [2023-12-27 11:45:37,102][20381] Updated weights for policy 1, policy_version 882076 (0.0011) [2023-12-27 11:45:37,161][20381] Updated weights for policy 1, policy_version 882086 (0.0008) [2023-12-27 11:45:37,216][20381] Updated weights for policy 1, policy_version 882096 (0.0008) [2023-12-27 11:45:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20105.1). Total num frames: 451870720. Throughput: 0: 10140.9, 1: 10065.6. Samples: 451835380. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:37,363][19170] Avg episode reward: [(0, '742.390'), (1, '990.300')] [2023-12-27 11:45:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000882104_225845248.pth... [2023-12-27 11:45:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000882800_226025472.pth... [2023-12-27 11:45:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000881616_225722368.pth [2023-12-27 11:45:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000880920_225542144.pth [2023-12-27 11:45:37,647][20347] Updated weights for policy 0, policy_version 882807 (0.0009) [2023-12-27 11:45:37,709][20347] Updated weights for policy 0, policy_version 882817 (0.0006) [2023-12-27 11:45:37,770][20347] Updated weights for policy 0, policy_version 882827 (0.0009) [2023-12-27 11:45:37,919][20381] Updated weights for policy 1, policy_version 882106 (0.0009) [2023-12-27 11:45:37,985][20381] Updated weights for policy 1, policy_version 882116 (0.0010) [2023-12-27 11:45:38,040][20381] Updated weights for policy 1, policy_version 882126 (0.0010) [2023-12-27 11:45:38,101][20381] Updated weights for policy 1, policy_version 882136 (0.0010) [2023-12-27 11:45:38,376][20347] Updated weights for policy 0, policy_version 882837 (0.0010) [2023-12-27 11:45:38,434][20347] Updated weights for policy 0, policy_version 882847 (0.0008) [2023-12-27 11:45:38,494][20347] Updated weights for policy 0, policy_version 882857 (0.0006) [2023-12-27 11:45:38,761][20381] Updated weights for policy 1, policy_version 882146 (0.0009) [2023-12-27 11:45:38,821][20381] Updated weights for policy 1, policy_version 882156 (0.0011) [2023-12-27 11:45:38,879][20381] Updated weights for policy 1, policy_version 882166 (0.0007) [2023-12-27 11:45:39,071][20347] Updated weights for policy 0, policy_version 882867 (0.0008) [2023-12-27 11:45:39,127][20347] Updated weights for policy 0, policy_version 882877 (0.0008) [2023-12-27 11:45:39,183][20347] Updated weights for policy 0, policy_version 882887 (0.0007) [2023-12-27 11:45:39,660][20381] Updated weights for policy 1, policy_version 882176 (0.0008) [2023-12-27 11:45:39,725][20381] Updated weights for policy 1, policy_version 882186 (0.0008) [2023-12-27 11:45:39,786][20381] Updated weights for policy 1, policy_version 882196 (0.0007) [2023-12-27 11:45:39,927][20347] Updated weights for policy 0, policy_version 882897 (0.0008) [2023-12-27 11:45:39,998][20347] Updated weights for policy 0, policy_version 882907 (0.0008) [2023-12-27 11:45:40,062][20347] Updated weights for policy 0, policy_version 882917 (0.0007) [2023-12-27 11:45:40,124][20347] Updated weights for policy 0, policy_version 882927 (0.0008) [2023-12-27 11:45:40,533][20381] Updated weights for policy 1, policy_version 882206 (0.0008) [2023-12-27 11:45:40,595][20381] Updated weights for policy 1, policy_version 882216 (0.0009) [2023-12-27 11:45:40,652][20381] Updated weights for policy 1, policy_version 882226 (0.0007) [2023-12-27 11:45:40,898][20347] Updated weights for policy 0, policy_version 882937 (0.0008) [2023-12-27 11:45:40,947][20347] Updated weights for policy 0, policy_version 882947 (0.0008) [2023-12-27 11:45:41,003][20347] Updated weights for policy 0, policy_version 882957 (0.0007) [2023-12-27 11:45:41,270][20381] Updated weights for policy 1, policy_version 882236 (0.0007) [2023-12-27 11:45:41,332][20381] Updated weights for policy 1, policy_version 882246 (0.0010) [2023-12-27 11:45:41,406][20381] Updated weights for policy 1, policy_version 882256 (0.0009) [2023-12-27 11:45:41,752][20347] Updated weights for policy 0, policy_version 882967 (0.0008) [2023-12-27 11:45:41,814][20347] Updated weights for policy 0, policy_version 882977 (0.0007) [2023-12-27 11:45:41,868][20347] Updated weights for policy 0, policy_version 882987 (0.0008) [2023-12-27 11:45:42,144][20381] Updated weights for policy 1, policy_version 882266 (0.0007) [2023-12-27 11:45:42,204][20381] Updated weights for policy 1, policy_version 882276 (0.0009) [2023-12-27 11:45:42,266][20381] Updated weights for policy 1, policy_version 882286 (0.0008) [2023-12-27 11:45:42,326][20381] Updated weights for policy 1, policy_version 882296 (0.0008) [2023-12-27 11:45:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20077.3). Total num frames: 451969024. Throughput: 0: 10151.6, 1: 10003.3. Samples: 451957700. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:42,363][19170] Avg episode reward: [(0, '743.530'), (1, '1000.000')] [2023-12-27 11:45:42,645][20347] Updated weights for policy 0, policy_version 882997 (0.0008) [2023-12-27 11:45:42,701][20347] Updated weights for policy 0, policy_version 883007 (0.0007) [2023-12-27 11:45:42,759][20347] Updated weights for policy 0, policy_version 883017 (0.0005) [2023-12-27 11:45:43,052][20381] Updated weights for policy 1, policy_version 882306 (0.0007) [2023-12-27 11:45:43,113][20381] Updated weights for policy 1, policy_version 882316 (0.0008) [2023-12-27 11:45:43,158][20381] Updated weights for policy 1, policy_version 882326 (0.0007) [2023-12-27 11:45:43,390][20347] Updated weights for policy 0, policy_version 883027 (0.0008) [2023-12-27 11:45:43,441][20347] Updated weights for policy 0, policy_version 883037 (0.0005) [2023-12-27 11:45:43,494][20347] Updated weights for policy 0, policy_version 883047 (0.0005) [2023-12-27 11:45:43,857][20381] Updated weights for policy 1, policy_version 882336 (0.0007) [2023-12-27 11:45:43,912][20381] Updated weights for policy 1, policy_version 882346 (0.0008) [2023-12-27 11:45:43,968][20381] Updated weights for policy 1, policy_version 882356 (0.0010) [2023-12-27 11:45:44,122][20347] Updated weights for policy 0, policy_version 883057 (0.0008) [2023-12-27 11:45:44,169][20347] Updated weights for policy 0, policy_version 883067 (0.0008) [2023-12-27 11:45:44,228][20347] Updated weights for policy 0, policy_version 883077 (0.0010) [2023-12-27 11:45:44,279][20347] Updated weights for policy 0, policy_version 883087 (0.0010) [2023-12-27 11:45:44,672][20381] Updated weights for policy 1, policy_version 882366 (0.0010) [2023-12-27 11:45:44,725][20381] Updated weights for policy 1, policy_version 882376 (0.0010) [2023-12-27 11:45:44,783][20381] Updated weights for policy 1, policy_version 882386 (0.0009) [2023-12-27 11:45:44,972][20347] Updated weights for policy 0, policy_version 883097 (0.0008) [2023-12-27 11:45:45,031][20347] Updated weights for policy 0, policy_version 883107 (0.0007) [2023-12-27 11:45:45,091][20347] Updated weights for policy 0, policy_version 883117 (0.0006) [2023-12-27 11:45:45,510][20381] Updated weights for policy 1, policy_version 882396 (0.0007) [2023-12-27 11:45:45,561][20381] Updated weights for policy 1, policy_version 882406 (0.0008) [2023-12-27 11:45:45,624][20381] Updated weights for policy 1, policy_version 882416 (0.0007) [2023-12-27 11:45:45,743][20347] Updated weights for policy 0, policy_version 883127 (0.0007) [2023-12-27 11:45:45,794][20347] Updated weights for policy 0, policy_version 883137 (0.0008) [2023-12-27 11:45:45,843][20347] Updated weights for policy 0, policy_version 883147 (0.0006) [2023-12-27 11:45:46,267][20381] Updated weights for policy 1, policy_version 882426 (0.0008) [2023-12-27 11:45:46,331][20381] Updated weights for policy 1, policy_version 882436 (0.0006) [2023-12-27 11:45:46,397][20381] Updated weights for policy 1, policy_version 882446 (0.0006) [2023-12-27 11:45:46,467][20381] Updated weights for policy 1, policy_version 882456 (0.0008) [2023-12-27 11:45:46,527][20347] Updated weights for policy 0, policy_version 883157 (0.0007) [2023-12-27 11:45:46,577][20347] Updated weights for policy 0, policy_version 883167 (0.0006) [2023-12-27 11:45:46,625][20347] Updated weights for policy 0, policy_version 883177 (0.0008) [2023-12-27 11:45:47,132][20381] Updated weights for policy 1, policy_version 882466 (0.0008) [2023-12-27 11:45:47,192][20381] Updated weights for policy 1, policy_version 882476 (0.0008) [2023-12-27 11:45:47,242][20347] Updated weights for policy 0, policy_version 883187 (0.0007) [2023-12-27 11:45:47,247][20381] Updated weights for policy 1, policy_version 882486 (0.0007) [2023-12-27 11:45:47,290][20347] Updated weights for policy 0, policy_version 883197 (0.0010) [2023-12-27 11:45:47,337][20347] Updated weights for policy 0, policy_version 883207 (0.0010) [2023-12-27 11:45:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20077.3). Total num frames: 452067328. Throughput: 0: 10165.0, 1: 9996.4. Samples: 452077976. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:47,363][19170] Avg episode reward: [(0, '765.900'), (1, '1000.000')] [2023-12-27 11:45:47,946][20347] Updated weights for policy 0, policy_version 883217 (0.0006) [2023-12-27 11:45:47,977][20381] Updated weights for policy 1, policy_version 882496 (0.0008) [2023-12-27 11:45:47,998][20347] Updated weights for policy 0, policy_version 883227 (0.0007) [2023-12-27 11:45:48,028][20381] Updated weights for policy 1, policy_version 882506 (0.0008) [2023-12-27 11:45:48,058][20347] Updated weights for policy 0, policy_version 883237 (0.0005) [2023-12-27 11:45:48,081][20381] Updated weights for policy 1, policy_version 882516 (0.0008) [2023-12-27 11:45:48,115][20347] Updated weights for policy 0, policy_version 883247 (0.0006) [2023-12-27 11:45:48,749][20347] Updated weights for policy 0, policy_version 883257 (0.0007) [2023-12-27 11:45:48,807][20347] Updated weights for policy 0, policy_version 883267 (0.0007) [2023-12-27 11:45:48,808][20381] Updated weights for policy 1, policy_version 882526 (0.0008) [2023-12-27 11:45:48,864][20347] Updated weights for policy 0, policy_version 883277 (0.0006) [2023-12-27 11:45:48,866][20381] Updated weights for policy 1, policy_version 882536 (0.0007) [2023-12-27 11:45:48,929][20381] Updated weights for policy 1, policy_version 882546 (0.0007) [2023-12-27 11:45:49,550][20347] Updated weights for policy 0, policy_version 883287 (0.0008) [2023-12-27 11:45:49,611][20347] Updated weights for policy 0, policy_version 883297 (0.0008) [2023-12-27 11:45:49,671][20347] Updated weights for policy 0, policy_version 883307 (0.0006) [2023-12-27 11:45:49,680][20381] Updated weights for policy 1, policy_version 882556 (0.0009) [2023-12-27 11:45:49,743][20381] Updated weights for policy 1, policy_version 882566 (0.0011) [2023-12-27 11:45:49,802][20381] Updated weights for policy 1, policy_version 882576 (0.0011) [2023-12-27 11:45:50,363][20347] Updated weights for policy 0, policy_version 883317 (0.0007) [2023-12-27 11:45:50,426][20347] Updated weights for policy 0, policy_version 883327 (0.0009) [2023-12-27 11:45:50,445][20381] Updated weights for policy 1, policy_version 882586 (0.0010) [2023-12-27 11:45:50,482][20347] Updated weights for policy 0, policy_version 883337 (0.0006) [2023-12-27 11:45:50,492][20381] Updated weights for policy 1, policy_version 882596 (0.0006) [2023-12-27 11:45:50,550][20381] Updated weights for policy 1, policy_version 882606 (0.0006) [2023-12-27 11:45:50,619][20381] Updated weights for policy 1, policy_version 882616 (0.0008) [2023-12-27 11:45:51,277][20347] Updated weights for policy 0, policy_version 883347 (0.0007) [2023-12-27 11:45:51,311][20291] KL-divergence is very high: 252.6011 [2023-12-27 11:45:51,322][20381] Updated weights for policy 1, policy_version 882626 (0.0007) [2023-12-27 11:45:51,330][20291] KL-divergence is very high: 110.1900 [2023-12-27 11:45:51,335][20347] Updated weights for policy 0, policy_version 883357 (0.0007) [2023-12-27 11:45:51,385][20381] Updated weights for policy 1, policy_version 882636 (0.0008) [2023-12-27 11:45:51,403][20347] Updated weights for policy 0, policy_version 883367 (0.0009) [2023-12-27 11:45:51,415][20291] KL-divergence is very high: 141.7115 [2023-12-27 11:45:51,445][20381] Updated weights for policy 1, policy_version 882646 (0.0008) [2023-12-27 11:45:52,098][20347] Updated weights for policy 0, policy_version 883377 (0.0007) [2023-12-27 11:45:52,145][20347] Updated weights for policy 0, policy_version 883387 (0.0006) [2023-12-27 11:45:52,161][20381] Updated weights for policy 1, policy_version 882656 (0.0006) [2023-12-27 11:45:52,200][20347] Updated weights for policy 0, policy_version 883397 (0.0007) [2023-12-27 11:45:52,218][20381] Updated weights for policy 1, policy_version 882666 (0.0006) [2023-12-27 11:45:52,265][20347] Updated weights for policy 0, policy_version 883407 (0.0007) [2023-12-27 11:45:52,279][20381] Updated weights for policy 1, policy_version 882676 (0.0007) [2023-12-27 11:45:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20132.9). Total num frames: 452173824. Throughput: 0: 10147.8, 1: 9925.4. Samples: 452138576. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:52,363][19170] Avg episode reward: [(0, '747.050'), (1, '1000.000')] [2023-12-27 11:45:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000882680_225992704.pth... [2023-12-27 11:45:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000883408_226181120.pth... [2023-12-27 11:45:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000882224_225878016.pth [2023-12-27 11:45:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000881528_225697792.pth [2023-12-27 11:45:52,966][20381] Updated weights for policy 1, policy_version 882686 (0.0007) [2023-12-27 11:45:52,988][20347] Updated weights for policy 0, policy_version 883417 (0.0008) [2023-12-27 11:45:53,009][20381] Updated weights for policy 1, policy_version 882696 (0.0008) [2023-12-27 11:45:53,048][20347] Updated weights for policy 0, policy_version 883427 (0.0008) [2023-12-27 11:45:53,054][20381] Updated weights for policy 1, policy_version 882706 (0.0007) [2023-12-27 11:45:53,102][20347] Updated weights for policy 0, policy_version 883437 (0.0006) [2023-12-27 11:45:53,669][20381] Updated weights for policy 1, policy_version 882716 (0.0007) [2023-12-27 11:45:53,714][20381] Updated weights for policy 1, policy_version 882726 (0.0005) [2023-12-27 11:45:53,760][20381] Updated weights for policy 1, policy_version 882736 (0.0005) [2023-12-27 11:45:53,811][20347] Updated weights for policy 0, policy_version 883447 (0.0007) [2023-12-27 11:45:53,868][20347] Updated weights for policy 0, policy_version 883457 (0.0007) [2023-12-27 11:45:53,931][20347] Updated weights for policy 0, policy_version 883467 (0.0007) [2023-12-27 11:45:54,374][20381] Updated weights for policy 1, policy_version 882746 (0.0006) [2023-12-27 11:45:54,421][20381] Updated weights for policy 1, policy_version 882756 (0.0007) [2023-12-27 11:45:54,467][20381] Updated weights for policy 1, policy_version 882766 (0.0005) [2023-12-27 11:45:54,514][20381] Updated weights for policy 1, policy_version 882776 (0.0007) [2023-12-27 11:45:54,635][20347] Updated weights for policy 0, policy_version 883477 (0.0007) [2023-12-27 11:45:54,686][20347] Updated weights for policy 0, policy_version 883487 (0.0008) [2023-12-27 11:45:54,737][20347] Updated weights for policy 0, policy_version 883497 (0.0007) [2023-12-27 11:45:55,204][20381] Updated weights for policy 1, policy_version 882786 (0.0007) [2023-12-27 11:45:55,260][20381] Updated weights for policy 1, policy_version 882796 (0.0007) [2023-12-27 11:45:55,317][20381] Updated weights for policy 1, policy_version 882806 (0.0007) [2023-12-27 11:45:55,509][20347] Updated weights for policy 0, policy_version 883507 (0.0007) [2023-12-27 11:45:55,568][20347] Updated weights for policy 0, policy_version 883517 (0.0008) [2023-12-27 11:45:55,624][20347] Updated weights for policy 0, policy_version 883527 (0.0008) [2023-12-27 11:45:56,008][20381] Updated weights for policy 1, policy_version 882816 (0.0008) [2023-12-27 11:45:56,059][20381] Updated weights for policy 1, policy_version 882826 (0.0008) [2023-12-27 11:45:56,109][20381] Updated weights for policy 1, policy_version 882836 (0.0007) [2023-12-27 11:45:56,314][20347] Updated weights for policy 0, policy_version 883537 (0.0007) [2023-12-27 11:45:56,364][20347] Updated weights for policy 0, policy_version 883547 (0.0008) [2023-12-27 11:45:56,420][20347] Updated weights for policy 0, policy_version 883557 (0.0007) [2023-12-27 11:45:56,476][20347] Updated weights for policy 0, policy_version 883567 (0.0007) [2023-12-27 11:45:56,833][20381] Updated weights for policy 1, policy_version 882846 (0.0008) [2023-12-27 11:45:56,889][20381] Updated weights for policy 1, policy_version 882856 (0.0006) [2023-12-27 11:45:56,940][20381] Updated weights for policy 1, policy_version 882866 (0.0006) [2023-12-27 11:45:57,101][20347] Updated weights for policy 0, policy_version 883577 (0.0005) [2023-12-27 11:45:57,157][20347] Updated weights for policy 0, policy_version 883587 (0.0005) [2023-12-27 11:45:57,203][20347] Updated weights for policy 0, policy_version 883597 (0.0005) [2023-12-27 11:45:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20070.4, 300 sec: 20132.9). Total num frames: 452272128. Throughput: 0: 10067.8, 1: 10030.2. Samples: 452258688. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:45:57,363][19170] Avg episode reward: [(0, '718.610'), (1, '1000.000')] [2023-12-27 11:45:57,531][20381] Updated weights for policy 1, policy_version 882876 (0.0010) [2023-12-27 11:45:57,592][20381] Updated weights for policy 1, policy_version 882886 (0.0010) [2023-12-27 11:45:57,652][20381] Updated weights for policy 1, policy_version 882896 (0.0010) [2023-12-27 11:45:57,751][20347] Updated weights for policy 0, policy_version 883607 (0.0009) [2023-12-27 11:45:57,799][20347] Updated weights for policy 0, policy_version 883617 (0.0008) [2023-12-27 11:45:57,846][20347] Updated weights for policy 0, policy_version 883627 (0.0008) [2023-12-27 11:45:58,330][20381] Updated weights for policy 1, policy_version 882906 (0.0010) [2023-12-27 11:45:58,395][20381] Updated weights for policy 1, policy_version 882916 (0.0008) [2023-12-27 11:45:58,465][20381] Updated weights for policy 1, policy_version 882926 (0.0008) [2023-12-27 11:45:58,475][20347] Updated weights for policy 0, policy_version 883637 (0.0006) [2023-12-27 11:45:58,526][20381] Updated weights for policy 1, policy_version 882936 (0.0008) [2023-12-27 11:45:58,543][20347] Updated weights for policy 0, policy_version 883647 (0.0008) [2023-12-27 11:45:58,609][20347] Updated weights for policy 0, policy_version 883657 (0.0006) [2023-12-27 11:45:59,333][20381] Updated weights for policy 1, policy_version 882946 (0.0008) [2023-12-27 11:45:59,385][20347] Updated weights for policy 0, policy_version 883667 (0.0008) [2023-12-27 11:45:59,422][20381] Updated weights for policy 1, policy_version 882956 (0.0008) [2023-12-27 11:45:59,451][20347] Updated weights for policy 0, policy_version 883677 (0.0008) [2023-12-27 11:45:59,477][20381] Updated weights for policy 1, policy_version 882966 (0.0008) [2023-12-27 11:45:59,506][20347] Updated weights for policy 0, policy_version 883687 (0.0007) [2023-12-27 11:46:00,193][20381] Updated weights for policy 1, policy_version 882976 (0.0008) [2023-12-27 11:46:00,249][20381] Updated weights for policy 1, policy_version 882986 (0.0008) [2023-12-27 11:46:00,288][20347] Updated weights for policy 0, policy_version 883697 (0.0006) [2023-12-27 11:46:00,301][20381] Updated weights for policy 1, policy_version 882996 (0.0007) [2023-12-27 11:46:00,335][20347] Updated weights for policy 0, policy_version 883707 (0.0007) [2023-12-27 11:46:00,391][20347] Updated weights for policy 0, policy_version 883717 (0.0008) [2023-12-27 11:46:00,454][20347] Updated weights for policy 0, policy_version 883727 (0.0008) [2023-12-27 11:46:00,933][20381] Updated weights for policy 1, policy_version 883006 (0.0007) [2023-12-27 11:46:00,979][20381] Updated weights for policy 1, policy_version 883016 (0.0005) [2023-12-27 11:46:01,047][20381] Updated weights for policy 1, policy_version 883026 (0.0008) [2023-12-27 11:46:01,114][20347] Updated weights for policy 0, policy_version 883737 (0.0007) [2023-12-27 11:46:01,183][20347] Updated weights for policy 0, policy_version 883747 (0.0007) [2023-12-27 11:46:01,244][20347] Updated weights for policy 0, policy_version 883757 (0.0007) [2023-12-27 11:46:01,759][20381] Updated weights for policy 1, policy_version 883036 (0.0007) [2023-12-27 11:46:01,827][20381] Updated weights for policy 1, policy_version 883046 (0.0007) [2023-12-27 11:46:01,891][20381] Updated weights for policy 1, policy_version 883056 (0.0008) [2023-12-27 11:46:02,002][20347] Updated weights for policy 0, policy_version 883767 (0.0007) [2023-12-27 11:46:02,061][20347] Updated weights for policy 0, policy_version 883777 (0.0007) [2023-12-27 11:46:02,118][20347] Updated weights for policy 0, policy_version 883787 (0.0008) [2023-12-27 11:46:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20070.4, 300 sec: 20160.6). Total num frames: 452370432. Throughput: 0: 10073.0, 1: 9963.8. Samples: 452379240. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:02,363][19170] Avg episode reward: [(0, '738.500'), (1, '1000.000')] [2023-12-27 11:46:02,542][20381] Updated weights for policy 1, policy_version 883066 (0.0008) [2023-12-27 11:46:02,595][20381] Updated weights for policy 1, policy_version 883076 (0.0005) [2023-12-27 11:46:02,654][20381] Updated weights for policy 1, policy_version 883086 (0.0005) [2023-12-27 11:46:02,706][20381] Updated weights for policy 1, policy_version 883096 (0.0006) [2023-12-27 11:46:02,817][20347] Updated weights for policy 0, policy_version 883797 (0.0008) [2023-12-27 11:46:02,876][20347] Updated weights for policy 0, policy_version 883807 (0.0008) [2023-12-27 11:46:02,932][20347] Updated weights for policy 0, policy_version 883817 (0.0007) [2023-12-27 11:46:03,309][20381] Updated weights for policy 1, policy_version 883106 (0.0010) [2023-12-27 11:46:03,366][20381] Updated weights for policy 1, policy_version 883116 (0.0010) [2023-12-27 11:46:03,426][20381] Updated weights for policy 1, policy_version 883126 (0.0007) [2023-12-27 11:46:03,666][20347] Updated weights for policy 0, policy_version 883827 (0.0007) [2023-12-27 11:46:03,727][20347] Updated weights for policy 0, policy_version 883837 (0.0007) [2023-12-27 11:46:03,780][20347] Updated weights for policy 0, policy_version 883847 (0.0009) [2023-12-27 11:46:04,017][20381] Updated weights for policy 1, policy_version 883136 (0.0008) [2023-12-27 11:46:04,072][20381] Updated weights for policy 1, policy_version 883146 (0.0008) [2023-12-27 11:46:04,131][20381] Updated weights for policy 1, policy_version 883156 (0.0008) [2023-12-27 11:46:04,511][20347] Updated weights for policy 0, policy_version 883857 (0.0008) [2023-12-27 11:46:04,566][20347] Updated weights for policy 0, policy_version 883867 (0.0008) [2023-12-27 11:46:04,616][20347] Updated weights for policy 0, policy_version 883877 (0.0008) [2023-12-27 11:46:04,665][20347] Updated weights for policy 0, policy_version 883887 (0.0008) [2023-12-27 11:46:04,878][20381] Updated weights for policy 1, policy_version 883166 (0.0010) [2023-12-27 11:46:04,926][20381] Updated weights for policy 1, policy_version 883176 (0.0010) [2023-12-27 11:46:04,983][20381] Updated weights for policy 1, policy_version 883186 (0.0010) [2023-12-27 11:46:05,366][20347] Updated weights for policy 0, policy_version 883897 (0.0010) [2023-12-27 11:46:05,420][20347] Updated weights for policy 0, policy_version 883907 (0.0005) [2023-12-27 11:46:05,472][20347] Updated weights for policy 0, policy_version 883917 (0.0009) [2023-12-27 11:46:05,655][20381] Updated weights for policy 1, policy_version 883196 (0.0009) [2023-12-27 11:46:05,704][20381] Updated weights for policy 1, policy_version 883206 (0.0010) [2023-12-27 11:46:05,756][20381] Updated weights for policy 1, policy_version 883216 (0.0010) [2023-12-27 11:46:06,145][20347] Updated weights for policy 0, policy_version 883927 (0.0009) [2023-12-27 11:46:06,203][20347] Updated weights for policy 0, policy_version 883937 (0.0008) [2023-12-27 11:46:06,269][20347] Updated weights for policy 0, policy_version 883947 (0.0008) [2023-12-27 11:46:06,389][20381] Updated weights for policy 1, policy_version 883226 (0.0009) [2023-12-27 11:46:06,449][20381] Updated weights for policy 1, policy_version 883236 (0.0008) [2023-12-27 11:46:06,504][20381] Updated weights for policy 1, policy_version 883246 (0.0008) [2023-12-27 11:46:06,553][20381] Updated weights for policy 1, policy_version 883256 (0.0008) [2023-12-27 11:46:07,056][20347] Updated weights for policy 0, policy_version 883957 (0.0007) [2023-12-27 11:46:07,110][20347] Updated weights for policy 0, policy_version 883967 (0.0005) [2023-12-27 11:46:07,176][20347] Updated weights for policy 0, policy_version 883977 (0.0008) [2023-12-27 11:46:07,299][20381] Updated weights for policy 1, policy_version 883266 (0.0005) [2023-12-27 11:46:07,353][20381] Updated weights for policy 1, policy_version 883276 (0.0005) [2023-12-27 11:46:07,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20070.5, 300 sec: 20160.7). Total num frames: 452468736. Throughput: 0: 10015.3, 1: 10013.1. Samples: 452440376. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:07,363][19170] Avg episode reward: [(0, '807.480'), (1, '990.290')] [2023-12-27 11:46:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000883984_226328576.pth... [2023-12-27 11:46:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000882800_226025472.pth [2023-12-27 11:46:07,411][20381] Updated weights for policy 1, policy_version 883286 (0.0005) [2023-12-27 11:46:07,422][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000883288_226148352.pth... [2023-12-27 11:46:07,425][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000882104_225845248.pth [2023-12-27 11:46:07,747][20347] Updated weights for policy 0, policy_version 883987 (0.0009) [2023-12-27 11:46:07,806][20347] Updated weights for policy 0, policy_version 883997 (0.0008) [2023-12-27 11:46:07,858][20347] Updated weights for policy 0, policy_version 884007 (0.0010) [2023-12-27 11:46:07,917][20381] Updated weights for policy 1, policy_version 883296 (0.0005) [2023-12-27 11:46:07,977][20381] Updated weights for policy 1, policy_version 883306 (0.0006) [2023-12-27 11:46:08,033][20381] Updated weights for policy 1, policy_version 883316 (0.0007) [2023-12-27 11:46:08,493][20347] Updated weights for policy 0, policy_version 884017 (0.0010) [2023-12-27 11:46:08,555][20347] Updated weights for policy 0, policy_version 884027 (0.0005) [2023-12-27 11:46:08,617][20347] Updated weights for policy 0, policy_version 884037 (0.0010) [2023-12-27 11:46:08,674][20347] Updated weights for policy 0, policy_version 884047 (0.0009) [2023-12-27 11:46:08,747][20381] Updated weights for policy 1, policy_version 883326 (0.0006) [2023-12-27 11:46:08,811][20381] Updated weights for policy 1, policy_version 883336 (0.0007) [2023-12-27 11:46:08,865][20381] Updated weights for policy 1, policy_version 883346 (0.0007) [2023-12-27 11:46:09,387][20347] Updated weights for policy 0, policy_version 884057 (0.0008) [2023-12-27 11:46:09,453][20347] Updated weights for policy 0, policy_version 884067 (0.0008) [2023-12-27 11:46:09,500][20381] Updated weights for policy 1, policy_version 883356 (0.0007) [2023-12-27 11:46:09,512][20347] Updated weights for policy 0, policy_version 884077 (0.0008) [2023-12-27 11:46:09,561][20381] Updated weights for policy 1, policy_version 883366 (0.0006) [2023-12-27 11:46:09,631][20381] Updated weights for policy 1, policy_version 883376 (0.0007) [2023-12-27 11:46:10,299][20347] Updated weights for policy 0, policy_version 884087 (0.0008) [2023-12-27 11:46:10,363][20347] Updated weights for policy 0, policy_version 884097 (0.0007) [2023-12-27 11:46:10,420][20381] Updated weights for policy 1, policy_version 883386 (0.0009) [2023-12-27 11:46:10,428][20347] Updated weights for policy 0, policy_version 884107 (0.0008) [2023-12-27 11:46:10,480][20381] Updated weights for policy 1, policy_version 883396 (0.0008) [2023-12-27 11:46:10,543][20381] Updated weights for policy 1, policy_version 883406 (0.0008) [2023-12-27 11:46:10,603][20381] Updated weights for policy 1, policy_version 883416 (0.0008) [2023-12-27 11:46:11,236][20347] Updated weights for policy 0, policy_version 884117 (0.0007) [2023-12-27 11:46:11,301][20347] Updated weights for policy 0, policy_version 884127 (0.0007) [2023-12-27 11:46:11,367][20347] Updated weights for policy 0, policy_version 884137 (0.0007) [2023-12-27 11:46:11,457][20381] Updated weights for policy 1, policy_version 883426 (0.0009) [2023-12-27 11:46:11,522][20381] Updated weights for policy 1, policy_version 883436 (0.0010) [2023-12-27 11:46:11,588][20381] Updated weights for policy 1, policy_version 883446 (0.0008) [2023-12-27 11:46:12,195][20347] Updated weights for policy 0, policy_version 884147 (0.0008) [2023-12-27 11:46:12,266][20347] Updated weights for policy 0, policy_version 884157 (0.0008) [2023-12-27 11:46:12,329][20347] Updated weights for policy 0, policy_version 884167 (0.0007) [2023-12-27 11:46:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19933.9, 300 sec: 20132.9). Total num frames: 452558848. Throughput: 0: 10003.7, 1: 10053.3. Samples: 452560008. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:12,363][19170] Avg episode reward: [(0, '787.150'), (1, '990.290')] [2023-12-27 11:46:12,400][20381] Updated weights for policy 1, policy_version 883456 (0.0009) [2023-12-27 11:46:12,462][20381] Updated weights for policy 1, policy_version 883466 (0.0009) [2023-12-27 11:46:12,529][20381] Updated weights for policy 1, policy_version 883476 (0.0008) [2023-12-27 11:46:13,089][20347] Updated weights for policy 0, policy_version 884177 (0.0008) [2023-12-27 11:46:13,145][20347] Updated weights for policy 0, policy_version 884187 (0.0007) [2023-12-27 11:46:13,199][20347] Updated weights for policy 0, policy_version 884197 (0.0008) [2023-12-27 11:46:13,264][20347] Updated weights for policy 0, policy_version 884207 (0.0008) [2023-12-27 11:46:13,313][20381] Updated weights for policy 1, policy_version 883486 (0.0009) [2023-12-27 11:46:13,375][20381] Updated weights for policy 1, policy_version 883496 (0.0010) [2023-12-27 11:46:13,441][20381] Updated weights for policy 1, policy_version 883506 (0.0006) [2023-12-27 11:46:13,991][20347] Updated weights for policy 0, policy_version 884217 (0.0006) [2023-12-27 11:46:14,053][20347] Updated weights for policy 0, policy_version 884227 (0.0007) [2023-12-27 11:46:14,106][20381] Updated weights for policy 1, policy_version 883516 (0.0008) [2023-12-27 11:46:14,116][20347] Updated weights for policy 0, policy_version 884237 (0.0007) [2023-12-27 11:46:14,162][20381] Updated weights for policy 1, policy_version 883526 (0.0010) [2023-12-27 11:46:14,222][20381] Updated weights for policy 1, policy_version 883536 (0.0010) [2023-12-27 11:46:14,882][20347] Updated weights for policy 0, policy_version 884247 (0.0007) [2023-12-27 11:46:14,943][20381] Updated weights for policy 1, policy_version 883546 (0.0010) [2023-12-27 11:46:14,948][20347] Updated weights for policy 0, policy_version 884257 (0.0008) [2023-12-27 11:46:15,007][20381] Updated weights for policy 1, policy_version 883556 (0.0007) [2023-12-27 11:46:15,018][20347] Updated weights for policy 0, policy_version 884267 (0.0007) [2023-12-27 11:46:15,070][20381] Updated weights for policy 1, policy_version 883566 (0.0006) [2023-12-27 11:46:15,132][20381] Updated weights for policy 1, policy_version 883576 (0.0008) [2023-12-27 11:46:15,790][20381] Updated weights for policy 1, policy_version 883586 (0.0007) [2023-12-27 11:46:15,799][20347] Updated weights for policy 0, policy_version 884277 (0.0007) [2023-12-27 11:46:15,847][20381] Updated weights for policy 1, policy_version 883596 (0.0006) [2023-12-27 11:46:15,855][20347] Updated weights for policy 0, policy_version 884287 (0.0006) [2023-12-27 11:46:15,900][20381] Updated weights for policy 1, policy_version 883606 (0.0008) [2023-12-27 11:46:15,907][20347] Updated weights for policy 0, policy_version 884297 (0.0006) [2023-12-27 11:46:16,580][20347] Updated weights for policy 0, policy_version 884307 (0.0007) [2023-12-27 11:46:16,604][20381] Updated weights for policy 1, policy_version 883616 (0.0009) [2023-12-27 11:46:16,632][20347] Updated weights for policy 0, policy_version 884317 (0.0007) [2023-12-27 11:46:16,653][20381] Updated weights for policy 1, policy_version 883626 (0.0005) [2023-12-27 11:46:16,691][20347] Updated weights for policy 0, policy_version 884327 (0.0007) [2023-12-27 11:46:16,701][20381] Updated weights for policy 1, policy_version 883636 (0.0006) [2023-12-27 11:46:17,326][20347] Updated weights for policy 0, policy_version 884337 (0.0008) [2023-12-27 11:46:17,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19933.9, 300 sec: 20132.9). Total num frames: 452657152. Throughput: 0: 9928.2, 1: 9998.2. Samples: 452672524. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:17,363][19170] Avg episode reward: [(0, '903.550'), (1, '990.290')] [2023-12-27 11:46:17,382][20347] Updated weights for policy 0, policy_version 884347 (0.0011) [2023-12-27 11:46:17,394][20381] Updated weights for policy 1, policy_version 883646 (0.0009) [2023-12-27 11:46:17,433][20347] Updated weights for policy 0, policy_version 884357 (0.0011) [2023-12-27 11:46:17,459][20381] Updated weights for policy 1, policy_version 883656 (0.0011) [2023-12-27 11:46:17,492][20347] Updated weights for policy 0, policy_version 884367 (0.0010) [2023-12-27 11:46:17,520][20381] Updated weights for policy 1, policy_version 883666 (0.0010) [2023-12-27 11:46:18,154][20381] Updated weights for policy 1, policy_version 883676 (0.0011) [2023-12-27 11:46:18,196][20347] Updated weights for policy 0, policy_version 884377 (0.0008) [2023-12-27 11:46:18,207][20381] Updated weights for policy 1, policy_version 883686 (0.0006) [2023-12-27 11:46:18,254][20347] Updated weights for policy 0, policy_version 884387 (0.0009) [2023-12-27 11:46:18,263][20381] Updated weights for policy 1, policy_version 883696 (0.0009) [2023-12-27 11:46:18,320][20347] Updated weights for policy 0, policy_version 884397 (0.0010) [2023-12-27 11:46:18,988][20381] Updated weights for policy 1, policy_version 883706 (0.0008) [2023-12-27 11:46:19,021][20347] Updated weights for policy 0, policy_version 884407 (0.0008) [2023-12-27 11:46:19,046][20381] Updated weights for policy 1, policy_version 883716 (0.0008) [2023-12-27 11:46:19,070][20347] Updated weights for policy 0, policy_version 884417 (0.0007) [2023-12-27 11:46:19,109][20381] Updated weights for policy 1, policy_version 883726 (0.0008) [2023-12-27 11:46:19,126][20347] Updated weights for policy 0, policy_version 884427 (0.0008) [2023-12-27 11:46:19,168][20381] Updated weights for policy 1, policy_version 883736 (0.0008) [2023-12-27 11:46:19,997][20381] Updated weights for policy 1, policy_version 883746 (0.0008) [2023-12-27 11:46:20,033][20347] Updated weights for policy 0, policy_version 884437 (0.0008) [2023-12-27 11:46:20,057][20381] Updated weights for policy 1, policy_version 883756 (0.0007) [2023-12-27 11:46:20,092][20347] Updated weights for policy 0, policy_version 884447 (0.0008) [2023-12-27 11:46:20,119][20381] Updated weights for policy 1, policy_version 883766 (0.0008) [2023-12-27 11:46:20,150][20347] Updated weights for policy 0, policy_version 884457 (0.0008) [2023-12-27 11:46:20,909][20381] Updated weights for policy 1, policy_version 883776 (0.0008) [2023-12-27 11:46:20,918][20347] Updated weights for policy 0, policy_version 884467 (0.0008) [2023-12-27 11:46:20,972][20381] Updated weights for policy 1, policy_version 883786 (0.0009) [2023-12-27 11:46:20,976][20347] Updated weights for policy 0, policy_version 884477 (0.0008) [2023-12-27 11:46:21,038][20381] Updated weights for policy 1, policy_version 883796 (0.0009) [2023-12-27 11:46:21,043][20347] Updated weights for policy 0, policy_version 884487 (0.0008) [2023-12-27 11:46:21,851][20347] Updated weights for policy 0, policy_version 884497 (0.0008) [2023-12-27 11:46:21,858][20381] Updated weights for policy 1, policy_version 883806 (0.0007) [2023-12-27 11:46:21,912][20347] Updated weights for policy 0, policy_version 884507 (0.0008) [2023-12-27 11:46:21,924][20381] Updated weights for policy 1, policy_version 883816 (0.0007) [2023-12-27 11:46:21,973][20347] Updated weights for policy 0, policy_version 884517 (0.0008) [2023-12-27 11:46:21,994][20381] Updated weights for policy 1, policy_version 883826 (0.0007) [2023-12-27 11:46:22,034][20347] Updated weights for policy 0, policy_version 884527 (0.0008) [2023-12-27 11:46:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 20132.9). Total num frames: 452755456. Throughput: 0: 9913.0, 1: 9956.4. Samples: 452729508. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:22,363][19170] Avg episode reward: [(0, '913.110'), (1, '980.530')] [2023-12-27 11:46:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000883832_226287616.pth... [2023-12-27 11:46:22,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000884528_226467840.pth... [2023-12-27 11:46:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000883408_226181120.pth [2023-12-27 11:46:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000882680_225992704.pth [2023-12-27 11:46:22,718][20381] Updated weights for policy 1, policy_version 883836 (0.0008) [2023-12-27 11:46:22,785][20381] Updated weights for policy 1, policy_version 883846 (0.0007) [2023-12-27 11:46:22,848][20381] Updated weights for policy 1, policy_version 883856 (0.0008) [2023-12-27 11:46:22,873][20347] Updated weights for policy 0, policy_version 884537 (0.0008) [2023-12-27 11:46:22,933][20347] Updated weights for policy 0, policy_version 884547 (0.0008) [2023-12-27 11:46:22,990][20347] Updated weights for policy 0, policy_version 884557 (0.0008) [2023-12-27 11:46:23,514][20381] Updated weights for policy 1, policy_version 883866 (0.0007) [2023-12-27 11:46:23,571][20381] Updated weights for policy 1, policy_version 883876 (0.0011) [2023-12-27 11:46:23,626][20381] Updated weights for policy 1, policy_version 883886 (0.0011) [2023-12-27 11:46:23,684][20381] Updated weights for policy 1, policy_version 883896 (0.0011) [2023-12-27 11:46:23,724][20347] Updated weights for policy 0, policy_version 884567 (0.0008) [2023-12-27 11:46:23,783][20347] Updated weights for policy 0, policy_version 884577 (0.0007) [2023-12-27 11:46:23,845][20347] Updated weights for policy 0, policy_version 884587 (0.0007) [2023-12-27 11:46:24,329][20381] Updated weights for policy 1, policy_version 883906 (0.0008) [2023-12-27 11:46:24,390][20381] Updated weights for policy 1, policy_version 883916 (0.0011) [2023-12-27 11:46:24,455][20381] Updated weights for policy 1, policy_version 883926 (0.0009) [2023-12-27 11:46:24,575][20347] Updated weights for policy 0, policy_version 884597 (0.0008) [2023-12-27 11:46:24,642][20347] Updated weights for policy 0, policy_version 884607 (0.0007) [2023-12-27 11:46:24,706][20347] Updated weights for policy 0, policy_version 884617 (0.0008) [2023-12-27 11:46:25,128][20381] Updated weights for policy 1, policy_version 883936 (0.0006) [2023-12-27 11:46:25,178][20381] Updated weights for policy 1, policy_version 883946 (0.0007) [2023-12-27 11:46:25,239][20381] Updated weights for policy 1, policy_version 883956 (0.0007) [2023-12-27 11:46:25,443][20347] Updated weights for policy 0, policy_version 884627 (0.0007) [2023-12-27 11:46:25,499][20347] Updated weights for policy 0, policy_version 884637 (0.0008) [2023-12-27 11:46:25,555][20347] Updated weights for policy 0, policy_version 884647 (0.0008) [2023-12-27 11:46:25,886][20381] Updated weights for policy 1, policy_version 883966 (0.0007) [2023-12-27 11:46:25,943][20381] Updated weights for policy 1, policy_version 883976 (0.0006) [2023-12-27 11:46:26,001][20381] Updated weights for policy 1, policy_version 883986 (0.0006) [2023-12-27 11:46:26,358][20347] Updated weights for policy 0, policy_version 884657 (0.0008) [2023-12-27 11:46:26,409][20347] Updated weights for policy 0, policy_version 884667 (0.0006) [2023-12-27 11:46:26,467][20347] Updated weights for policy 0, policy_version 884677 (0.0008) [2023-12-27 11:46:26,531][20347] Updated weights for policy 0, policy_version 884687 (0.0007) [2023-12-27 11:46:26,572][20381] Updated weights for policy 1, policy_version 883996 (0.0007) [2023-12-27 11:46:26,633][20381] Updated weights for policy 1, policy_version 884006 (0.0007) [2023-12-27 11:46:26,697][20381] Updated weights for policy 1, policy_version 884016 (0.0008) [2023-12-27 11:46:27,189][20347] Updated weights for policy 0, policy_version 884697 (0.0008) [2023-12-27 11:46:27,240][20347] Updated weights for policy 0, policy_version 884707 (0.0007) [2023-12-27 11:46:27,300][20347] Updated weights for policy 0, policy_version 884717 (0.0006) [2023-12-27 11:46:27,351][20381] Updated weights for policy 1, policy_version 884026 (0.0008) [2023-12-27 11:46:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 20132.9). Total num frames: 452853760. Throughput: 0: 9748.4, 1: 9918.1. Samples: 452842688. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:27,363][19170] Avg episode reward: [(0, '902.730'), (1, '970.820')] [2023-12-27 11:46:27,411][20381] Updated weights for policy 1, policy_version 884036 (0.0008) [2023-12-27 11:46:27,478][20381] Updated weights for policy 1, policy_version 884046 (0.0008) [2023-12-27 11:46:27,533][20381] Updated weights for policy 1, policy_version 884056 (0.0008) [2023-12-27 11:46:27,901][20347] Updated weights for policy 0, policy_version 884727 (0.0007) [2023-12-27 11:46:27,950][20347] Updated weights for policy 0, policy_version 884737 (0.0007) [2023-12-27 11:46:28,011][20347] Updated weights for policy 0, policy_version 884747 (0.0008) [2023-12-27 11:46:28,210][20381] Updated weights for policy 1, policy_version 884066 (0.0011) [2023-12-27 11:46:28,265][20381] Updated weights for policy 1, policy_version 884076 (0.0010) [2023-12-27 11:46:28,329][20381] Updated weights for policy 1, policy_version 884086 (0.0010) [2023-12-27 11:46:28,688][20347] Updated weights for policy 0, policy_version 884757 (0.0008) [2023-12-27 11:46:28,752][20347] Updated weights for policy 0, policy_version 884767 (0.0006) [2023-12-27 11:46:28,815][20347] Updated weights for policy 0, policy_version 884777 (0.0006) [2023-12-27 11:46:29,054][20381] Updated weights for policy 1, policy_version 884096 (0.0011) [2023-12-27 11:46:29,114][20381] Updated weights for policy 1, policy_version 884106 (0.0011) [2023-12-27 11:46:29,174][20381] Updated weights for policy 1, policy_version 884116 (0.0010) [2023-12-27 11:46:29,545][20347] Updated weights for policy 0, policy_version 884787 (0.0007) [2023-12-27 11:46:29,604][20347] Updated weights for policy 0, policy_version 884797 (0.0007) [2023-12-27 11:46:29,656][20347] Updated weights for policy 0, policy_version 884807 (0.0008) [2023-12-27 11:46:29,963][20381] Updated weights for policy 1, policy_version 884126 (0.0009) [2023-12-27 11:46:30,023][20381] Updated weights for policy 1, policy_version 884136 (0.0008) [2023-12-27 11:46:30,085][20381] Updated weights for policy 1, policy_version 884146 (0.0008) [2023-12-27 11:46:30,403][20347] Updated weights for policy 0, policy_version 884817 (0.0007) [2023-12-27 11:46:30,462][20347] Updated weights for policy 0, policy_version 884827 (0.0008) [2023-12-27 11:46:30,525][20347] Updated weights for policy 0, policy_version 884837 (0.0008) [2023-12-27 11:46:30,579][20347] Updated weights for policy 0, policy_version 884847 (0.0007) [2023-12-27 11:46:30,813][20381] Updated weights for policy 1, policy_version 884156 (0.0007) [2023-12-27 11:46:30,877][20381] Updated weights for policy 1, policy_version 884166 (0.0007) [2023-12-27 11:46:30,932][20381] Updated weights for policy 1, policy_version 884176 (0.0005) [2023-12-27 11:46:31,296][20347] Updated weights for policy 0, policy_version 884857 (0.0007) [2023-12-27 11:46:31,358][20347] Updated weights for policy 0, policy_version 884867 (0.0007) [2023-12-27 11:46:31,438][20347] Updated weights for policy 0, policy_version 884877 (0.0009) [2023-12-27 11:46:31,692][20381] Updated weights for policy 1, policy_version 884186 (0.0007) [2023-12-27 11:46:31,764][20381] Updated weights for policy 1, policy_version 884196 (0.0008) [2023-12-27 11:46:31,818][20381] Updated weights for policy 1, policy_version 884206 (0.0008) [2023-12-27 11:46:31,879][20381] Updated weights for policy 1, policy_version 884216 (0.0008) [2023-12-27 11:46:32,188][20347] Updated weights for policy 0, policy_version 884887 (0.0009) [2023-12-27 11:46:32,243][20347] Updated weights for policy 0, policy_version 884897 (0.0009) [2023-12-27 11:46:32,304][20347] Updated weights for policy 0, policy_version 884907 (0.0008) [2023-12-27 11:46:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19797.4, 300 sec: 20105.1). Total num frames: 452952064. Throughput: 0: 9694.3, 1: 9907.8. Samples: 452960072. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:32,363][19170] Avg episode reward: [(0, '883.290'), (1, '990.290')] [2023-12-27 11:46:32,618][20381] Updated weights for policy 1, policy_version 884226 (0.0009) [2023-12-27 11:46:32,677][20381] Updated weights for policy 1, policy_version 884236 (0.0008) [2023-12-27 11:46:32,736][20381] Updated weights for policy 1, policy_version 884246 (0.0008) [2023-12-27 11:46:33,073][20347] Updated weights for policy 0, policy_version 884917 (0.0009) [2023-12-27 11:46:33,131][20347] Updated weights for policy 0, policy_version 884927 (0.0007) [2023-12-27 11:46:33,192][20347] Updated weights for policy 0, policy_version 884937 (0.0007) [2023-12-27 11:46:33,472][20381] Updated weights for policy 1, policy_version 884256 (0.0007) [2023-12-27 11:46:33,519][20381] Updated weights for policy 1, policy_version 884266 (0.0006) [2023-12-27 11:46:33,573][20381] Updated weights for policy 1, policy_version 884276 (0.0008) [2023-12-27 11:46:33,876][20347] Updated weights for policy 0, policy_version 884947 (0.0007) [2023-12-27 11:46:33,935][20347] Updated weights for policy 0, policy_version 884957 (0.0006) [2023-12-27 11:46:33,984][20347] Updated weights for policy 0, policy_version 884967 (0.0008) [2023-12-27 11:46:34,299][20381] Updated weights for policy 1, policy_version 884286 (0.0008) [2023-12-27 11:46:34,366][20381] Updated weights for policy 1, policy_version 884296 (0.0008) [2023-12-27 11:46:34,429][20381] Updated weights for policy 1, policy_version 884306 (0.0008) [2023-12-27 11:46:34,678][20347] Updated weights for policy 0, policy_version 884977 (0.0007) [2023-12-27 11:46:34,738][20347] Updated weights for policy 0, policy_version 884987 (0.0007) [2023-12-27 11:46:34,801][20347] Updated weights for policy 0, policy_version 884997 (0.0006) [2023-12-27 11:46:34,864][20347] Updated weights for policy 0, policy_version 885007 (0.0007) [2023-12-27 11:46:35,167][20381] Updated weights for policy 1, policy_version 884316 (0.0008) [2023-12-27 11:46:35,228][20381] Updated weights for policy 1, policy_version 884326 (0.0008) [2023-12-27 11:46:35,284][20381] Updated weights for policy 1, policy_version 884336 (0.0008) [2023-12-27 11:46:35,505][20347] Updated weights for policy 0, policy_version 885017 (0.0008) [2023-12-27 11:46:35,572][20347] Updated weights for policy 0, policy_version 885027 (0.0007) [2023-12-27 11:46:35,629][20347] Updated weights for policy 0, policy_version 885037 (0.0008) [2023-12-27 11:46:35,982][20381] Updated weights for policy 1, policy_version 884346 (0.0008) [2023-12-27 11:46:36,036][20381] Updated weights for policy 1, policy_version 884356 (0.0008) [2023-12-27 11:46:36,090][20381] Updated weights for policy 1, policy_version 884366 (0.0010) [2023-12-27 11:46:36,149][20381] Updated weights for policy 1, policy_version 884376 (0.0009) [2023-12-27 11:46:36,380][20347] Updated weights for policy 0, policy_version 885047 (0.0008) [2023-12-27 11:46:36,442][20347] Updated weights for policy 0, policy_version 885057 (0.0007) [2023-12-27 11:46:36,507][20347] Updated weights for policy 0, policy_version 885067 (0.0007) [2023-12-27 11:46:36,893][20381] Updated weights for policy 1, policy_version 884386 (0.0006) [2023-12-27 11:46:36,948][20381] Updated weights for policy 1, policy_version 884396 (0.0006) [2023-12-27 11:46:37,013][20381] Updated weights for policy 1, policy_version 884406 (0.0008) [2023-12-27 11:46:37,254][20347] Updated weights for policy 0, policy_version 885077 (0.0008) [2023-12-27 11:46:37,313][20347] Updated weights for policy 0, policy_version 885087 (0.0008) [2023-12-27 11:46:37,362][19170] Fps is (10 sec: 18841.5, 60 sec: 19524.3, 300 sec: 20105.1). Total num frames: 453042176. Throughput: 0: 9649.0, 1: 9890.1. Samples: 453017836. Policy #0 lag: (min: 31.0, avg: 32.7, max: 52.0) [2023-12-27 11:46:37,363][19170] Avg episode reward: [(0, '883.240'), (1, '1000.000')] [2023-12-27 11:46:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000884408_226435072.pth... [2023-12-27 11:46:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000883288_226148352.pth [2023-12-27 11:46:37,373][20347] Updated weights for policy 0, policy_version 885097 (0.0007) [2023-12-27 11:46:37,411][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000885104_226615296.pth... [2023-12-27 11:46:37,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000883984_226328576.pth [2023-12-27 11:46:37,697][20381] Updated weights for policy 1, policy_version 884416 (0.0009) [2023-12-27 11:46:37,758][20381] Updated weights for policy 1, policy_version 884426 (0.0007) [2023-12-27 11:46:37,814][20381] Updated weights for policy 1, policy_version 884436 (0.0008) [2023-12-27 11:46:37,993][20347] Updated weights for policy 0, policy_version 885107 (0.0006) [2023-12-27 11:46:38,047][20347] Updated weights for policy 0, policy_version 885117 (0.0007) [2023-12-27 11:46:38,099][20347] Updated weights for policy 0, policy_version 885127 (0.0008) [2023-12-27 11:46:38,540][20381] Updated weights for policy 1, policy_version 884446 (0.0008) [2023-12-27 11:46:38,598][20381] Updated weights for policy 1, policy_version 884456 (0.0010) [2023-12-27 11:46:38,666][20381] Updated weights for policy 1, policy_version 884466 (0.0006) [2023-12-27 11:46:38,818][20347] Updated weights for policy 0, policy_version 885137 (0.0007) [2023-12-27 11:46:38,885][20347] Updated weights for policy 0, policy_version 885147 (0.0008) [2023-12-27 11:46:38,948][20347] Updated weights for policy 0, policy_version 885157 (0.0009) [2023-12-27 11:46:39,013][20347] Updated weights for policy 0, policy_version 885167 (0.0008) [2023-12-27 11:46:39,345][20381] Updated weights for policy 1, policy_version 884476 (0.0007) [2023-12-27 11:46:39,417][20381] Updated weights for policy 1, policy_version 884486 (0.0008) [2023-12-27 11:46:39,483][20381] Updated weights for policy 1, policy_version 884496 (0.0008) [2023-12-27 11:46:39,812][20347] Updated weights for policy 0, policy_version 885177 (0.0007) [2023-12-27 11:46:39,879][20347] Updated weights for policy 0, policy_version 885187 (0.0008) [2023-12-27 11:46:39,945][20347] Updated weights for policy 0, policy_version 885197 (0.0007) [2023-12-27 11:46:40,282][20381] Updated weights for policy 1, policy_version 884506 (0.0008) [2023-12-27 11:46:40,352][20381] Updated weights for policy 1, policy_version 884516 (0.0008) [2023-12-27 11:46:40,418][20381] Updated weights for policy 1, policy_version 884526 (0.0008) [2023-12-27 11:46:40,477][20381] Updated weights for policy 1, policy_version 884536 (0.0008) [2023-12-27 11:46:40,676][20347] Updated weights for policy 0, policy_version 885207 (0.0007) [2023-12-27 11:46:40,739][20347] Updated weights for policy 0, policy_version 885217 (0.0007) [2023-12-27 11:46:40,798][20347] Updated weights for policy 0, policy_version 885227 (0.0008) [2023-12-27 11:46:41,307][20381] Updated weights for policy 1, policy_version 884546 (0.0008) [2023-12-27 11:46:41,374][20381] Updated weights for policy 1, policy_version 884556 (0.0007) [2023-12-27 11:46:41,453][20381] Updated weights for policy 1, policy_version 884567 (0.0008) [2023-12-27 11:46:41,594][20347] Updated weights for policy 0, policy_version 885237 (0.0008) [2023-12-27 11:46:41,663][20347] Updated weights for policy 0, policy_version 885247 (0.0008) [2023-12-27 11:46:41,734][20347] Updated weights for policy 0, policy_version 885257 (0.0007) [2023-12-27 11:46:42,277][20381] Updated weights for policy 1, policy_version 884577 (0.0008) [2023-12-27 11:46:42,341][20381] Updated weights for policy 1, policy_version 884587 (0.0007) [2023-12-27 11:46:42,362][19170] Fps is (10 sec: 18022.3, 60 sec: 19387.7, 300 sec: 20077.3). Total num frames: 453132288. Throughput: 0: 9649.7, 1: 9773.2. Samples: 453132716. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:46:42,363][19170] Avg episode reward: [(0, '893.100'), (1, '1000.000')] [2023-12-27 11:46:42,422][20381] Updated weights for policy 1, policy_version 884597 (0.0008) [2023-12-27 11:46:42,567][20347] Updated weights for policy 0, policy_version 885267 (0.0009) [2023-12-27 11:46:42,627][20347] Updated weights for policy 0, policy_version 885277 (0.0008) [2023-12-27 11:46:42,692][20347] Updated weights for policy 0, policy_version 885287 (0.0008) [2023-12-27 11:46:43,211][20381] Updated weights for policy 1, policy_version 884607 (0.0008) [2023-12-27 11:46:43,269][20381] Updated weights for policy 1, policy_version 884617 (0.0008) [2023-12-27 11:46:43,335][20381] Updated weights for policy 1, policy_version 884627 (0.0008) [2023-12-27 11:46:43,403][20347] Updated weights for policy 0, policy_version 885297 (0.0008) [2023-12-27 11:46:43,468][20347] Updated weights for policy 0, policy_version 885307 (0.0007) [2023-12-27 11:46:43,525][20347] Updated weights for policy 0, policy_version 885317 (0.0007) [2023-12-27 11:46:43,582][20347] Updated weights for policy 0, policy_version 885327 (0.0007) [2023-12-27 11:46:44,085][20381] Updated weights for policy 1, policy_version 884637 (0.0008) [2023-12-27 11:46:44,150][20381] Updated weights for policy 1, policy_version 884647 (0.0008) [2023-12-27 11:46:44,210][20381] Updated weights for policy 1, policy_version 884657 (0.0007) [2023-12-27 11:46:44,297][20347] Updated weights for policy 0, policy_version 885337 (0.0008) [2023-12-27 11:46:44,348][20347] Updated weights for policy 0, policy_version 885347 (0.0008) [2023-12-27 11:46:44,400][20347] Updated weights for policy 0, policy_version 885357 (0.0007) [2023-12-27 11:46:44,926][20381] Updated weights for policy 1, policy_version 884667 (0.0008) [2023-12-27 11:46:44,991][20381] Updated weights for policy 1, policy_version 884677 (0.0009) [2023-12-27 11:46:45,055][20381] Updated weights for policy 1, policy_version 884687 (0.0007) [2023-12-27 11:46:45,195][20347] Updated weights for policy 0, policy_version 885367 (0.0008) [2023-12-27 11:46:45,259][20347] Updated weights for policy 0, policy_version 885377 (0.0008) [2023-12-27 11:46:45,319][20347] Updated weights for policy 0, policy_version 885387 (0.0008) [2023-12-27 11:46:45,773][20381] Updated weights for policy 1, policy_version 884697 (0.0008) [2023-12-27 11:46:45,837][20381] Updated weights for policy 1, policy_version 884707 (0.0007) [2023-12-27 11:46:45,901][20381] Updated weights for policy 1, policy_version 884717 (0.0007) [2023-12-27 11:46:45,967][20381] Updated weights for policy 1, policy_version 884727 (0.0008) [2023-12-27 11:46:46,045][20347] Updated weights for policy 0, policy_version 885397 (0.0008) [2023-12-27 11:46:46,110][20347] Updated weights for policy 0, policy_version 885407 (0.0008) [2023-12-27 11:46:46,174][20347] Updated weights for policy 0, policy_version 885417 (0.0008) [2023-12-27 11:46:46,580][20381] Updated weights for policy 1, policy_version 884737 (0.0008) [2023-12-27 11:46:46,637][20381] Updated weights for policy 1, policy_version 884747 (0.0006) [2023-12-27 11:46:46,692][20381] Updated weights for policy 1, policy_version 884757 (0.0007) [2023-12-27 11:46:46,848][20347] Updated weights for policy 0, policy_version 885427 (0.0007) [2023-12-27 11:46:46,915][20347] Updated weights for policy 0, policy_version 885437 (0.0007) [2023-12-27 11:46:46,967][20347] Updated weights for policy 0, policy_version 885447 (0.0008) [2023-12-27 11:46:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19387.8, 300 sec: 20077.3). Total num frames: 453230592. Throughput: 0: 9526.2, 1: 9704.6. Samples: 453244628. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:46:47,363][19170] Avg episode reward: [(0, '863.890'), (1, '1000.000')] [2023-12-27 11:46:47,369][20381] Updated weights for policy 1, policy_version 884767 (0.0008) [2023-12-27 11:46:47,428][20381] Updated weights for policy 1, policy_version 884777 (0.0008) [2023-12-27 11:46:47,490][20381] Updated weights for policy 1, policy_version 884787 (0.0008) [2023-12-27 11:46:47,675][20347] Updated weights for policy 0, policy_version 885457 (0.0008) [2023-12-27 11:46:47,742][20347] Updated weights for policy 0, policy_version 885467 (0.0007) [2023-12-27 11:46:47,798][20347] Updated weights for policy 0, policy_version 885477 (0.0007) [2023-12-27 11:46:47,864][20347] Updated weights for policy 0, policy_version 885487 (0.0008) [2023-12-27 11:46:48,134][20381] Updated weights for policy 1, policy_version 884797 (0.0008) [2023-12-27 11:46:48,188][20381] Updated weights for policy 1, policy_version 884807 (0.0007) [2023-12-27 11:46:48,247][20381] Updated weights for policy 1, policy_version 884817 (0.0007) [2023-12-27 11:46:48,623][20347] Updated weights for policy 0, policy_version 885497 (0.0008) [2023-12-27 11:46:48,687][20347] Updated weights for policy 0, policy_version 885507 (0.0008) [2023-12-27 11:46:48,753][20347] Updated weights for policy 0, policy_version 885517 (0.0008) [2023-12-27 11:46:48,962][20381] Updated weights for policy 1, policy_version 884827 (0.0007) [2023-12-27 11:46:49,028][20381] Updated weights for policy 1, policy_version 884837 (0.0006) [2023-12-27 11:46:49,088][20381] Updated weights for policy 1, policy_version 884847 (0.0006) [2023-12-27 11:46:49,511][20347] Updated weights for policy 0, policy_version 885527 (0.0008) [2023-12-27 11:46:49,564][20347] Updated weights for policy 0, policy_version 885537 (0.0007) [2023-12-27 11:46:49,618][20347] Updated weights for policy 0, policy_version 885547 (0.0008) [2023-12-27 11:46:49,763][20381] Updated weights for policy 1, policy_version 884857 (0.0006) [2023-12-27 11:46:49,829][20381] Updated weights for policy 1, policy_version 884867 (0.0008) [2023-12-27 11:46:49,888][20381] Updated weights for policy 1, policy_version 884877 (0.0008) [2023-12-27 11:46:49,971][20381] Updated weights for policy 1, policy_version 884887 (0.0008) [2023-12-27 11:46:50,386][20347] Updated weights for policy 0, policy_version 885557 (0.0007) [2023-12-27 11:46:50,454][20347] Updated weights for policy 0, policy_version 885567 (0.0008) [2023-12-27 11:46:50,519][20347] Updated weights for policy 0, policy_version 885577 (0.0008) [2023-12-27 11:46:50,652][20381] Updated weights for policy 1, policy_version 884897 (0.0007) [2023-12-27 11:46:50,713][20381] Updated weights for policy 1, policy_version 884907 (0.0008) [2023-12-27 11:46:50,774][20381] Updated weights for policy 1, policy_version 884917 (0.0008) [2023-12-27 11:46:51,268][20347] Updated weights for policy 0, policy_version 885587 (0.0008) [2023-12-27 11:46:51,334][20347] Updated weights for policy 0, policy_version 885597 (0.0008) [2023-12-27 11:46:51,410][20347] Updated weights for policy 0, policy_version 885607 (0.0008) [2023-12-27 11:46:51,569][20381] Updated weights for policy 1, policy_version 884927 (0.0008) [2023-12-27 11:46:51,632][20381] Updated weights for policy 1, policy_version 884937 (0.0007) [2023-12-27 11:46:51,691][20381] Updated weights for policy 1, policy_version 884947 (0.0010) [2023-12-27 11:46:52,174][20347] Updated weights for policy 0, policy_version 885617 (0.0007) [2023-12-27 11:46:52,235][20347] Updated weights for policy 0, policy_version 885627 (0.0008) [2023-12-27 11:46:52,301][20347] Updated weights for policy 0, policy_version 885637 (0.0008) [2023-12-27 11:46:52,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19114.7, 300 sec: 20021.8). Total num frames: 453320704. Throughput: 0: 9488.2, 1: 9646.0. Samples: 453301416. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:46:52,363][19170] Avg episode reward: [(0, '815.250'), (1, '1000.000')] [2023-12-27 11:46:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000884952_226574336.pth... [2023-12-27 11:46:52,372][20347] Updated weights for policy 0, policy_version 885647 (0.0008) [2023-12-27 11:46:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000883832_226287616.pth [2023-12-27 11:46:52,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000885648_226754560.pth... [2023-12-27 11:46:52,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000884528_226467840.pth [2023-12-27 11:46:52,482][20381] Updated weights for policy 1, policy_version 884957 (0.0011) [2023-12-27 11:46:52,534][20381] Updated weights for policy 1, policy_version 884967 (0.0010) [2023-12-27 11:46:52,595][20381] Updated weights for policy 1, policy_version 884977 (0.0007) [2023-12-27 11:46:53,134][20347] Updated weights for policy 0, policy_version 885657 (0.0008) [2023-12-27 11:46:53,195][20347] Updated weights for policy 0, policy_version 885667 (0.0008) [2023-12-27 11:46:53,250][20347] Updated weights for policy 0, policy_version 885677 (0.0007) [2023-12-27 11:46:53,319][20381] Updated weights for policy 1, policy_version 884987 (0.0008) [2023-12-27 11:46:53,380][20381] Updated weights for policy 1, policy_version 884997 (0.0009) [2023-12-27 11:46:53,434][20381] Updated weights for policy 1, policy_version 885007 (0.0011) [2023-12-27 11:46:53,961][20347] Updated weights for policy 0, policy_version 885687 (0.0007) [2023-12-27 11:46:54,018][20347] Updated weights for policy 0, policy_version 885697 (0.0008) [2023-12-27 11:46:54,075][20347] Updated weights for policy 0, policy_version 885707 (0.0008) [2023-12-27 11:46:54,121][20381] Updated weights for policy 1, policy_version 885017 (0.0010) [2023-12-27 11:46:54,182][20381] Updated weights for policy 1, policy_version 885027 (0.0008) [2023-12-27 11:46:54,246][20381] Updated weights for policy 1, policy_version 885037 (0.0008) [2023-12-27 11:46:54,308][20381] Updated weights for policy 1, policy_version 885047 (0.0007) [2023-12-27 11:46:54,311][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 11:46:54,769][20347] Updated weights for policy 0, policy_version 885717 (0.0007) [2023-12-27 11:46:54,834][20347] Updated weights for policy 0, policy_version 885727 (0.0009) [2023-12-27 11:46:54,892][20347] Updated weights for policy 0, policy_version 885737 (0.0008) [2023-12-27 11:46:54,963][20381] Updated weights for policy 1, policy_version 885057 (0.0007) [2023-12-27 11:46:55,027][20381] Updated weights for policy 1, policy_version 885067 (0.0007) [2023-12-27 11:46:55,083][20381] Updated weights for policy 1, policy_version 885077 (0.0007) [2023-12-27 11:46:55,583][20347] Updated weights for policy 0, policy_version 885747 (0.0006) [2023-12-27 11:46:55,650][20347] Updated weights for policy 0, policy_version 885757 (0.0006) [2023-12-27 11:46:55,716][20347] Updated weights for policy 0, policy_version 885767 (0.0006) [2023-12-27 11:46:55,764][20381] Updated weights for policy 1, policy_version 885087 (0.0009) [2023-12-27 11:46:55,822][20381] Updated weights for policy 1, policy_version 885097 (0.0008) [2023-12-27 11:46:55,890][20381] Updated weights for policy 1, policy_version 885107 (0.0009) [2023-12-27 11:46:56,377][20347] Updated weights for policy 0, policy_version 885777 (0.0008) [2023-12-27 11:46:56,441][20347] Updated weights for policy 0, policy_version 885787 (0.0007) [2023-12-27 11:46:56,505][20347] Updated weights for policy 0, policy_version 885797 (0.0006) [2023-12-27 11:46:56,553][20381] Updated weights for policy 1, policy_version 885117 (0.0007) [2023-12-27 11:46:56,564][20347] Updated weights for policy 0, policy_version 885807 (0.0006) [2023-12-27 11:46:56,611][20381] Updated weights for policy 1, policy_version 885127 (0.0006) [2023-12-27 11:46:56,674][20381] Updated weights for policy 1, policy_version 885137 (0.0006) [2023-12-27 11:46:57,221][20347] Updated weights for policy 0, policy_version 885817 (0.0007) [2023-12-27 11:46:57,278][20347] Updated weights for policy 0, policy_version 885827 (0.0008) [2023-12-27 11:46:57,304][20381] Updated weights for policy 1, policy_version 885147 (0.0006) [2023-12-27 11:46:57,335][20347] Updated weights for policy 0, policy_version 885837 (0.0007) [2023-12-27 11:46:57,359][20381] Updated weights for policy 1, policy_version 885157 (0.0006) [2023-12-27 11:46:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19251.3, 300 sec: 20077.3). Total num frames: 453427200. Throughput: 0: 9442.6, 1: 9594.7. Samples: 453416684. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:46:57,363][19170] Avg episode reward: [(0, '903.210'), (1, '1000.000')] [2023-12-27 11:46:57,422][20381] Updated weights for policy 1, policy_version 885167 (0.0007) [2023-12-27 11:46:58,072][20347] Updated weights for policy 0, policy_version 885847 (0.0008) [2023-12-27 11:46:58,112][20381] Updated weights for policy 1, policy_version 885177 (0.0006) [2023-12-27 11:46:58,137][20347] Updated weights for policy 0, policy_version 885857 (0.0008) [2023-12-27 11:46:58,173][20381] Updated weights for policy 1, policy_version 885187 (0.0008) [2023-12-27 11:46:58,199][20347] Updated weights for policy 0, policy_version 885867 (0.0008) [2023-12-27 11:46:58,234][20381] Updated weights for policy 1, policy_version 885197 (0.0008) [2023-12-27 11:46:58,304][20381] Updated weights for policy 1, policy_version 885207 (0.0008) [2023-12-27 11:46:59,031][20347] Updated weights for policy 0, policy_version 885877 (0.0009) [2023-12-27 11:46:59,090][20347] Updated weights for policy 0, policy_version 885887 (0.0010) [2023-12-27 11:46:59,155][20347] Updated weights for policy 0, policy_version 885897 (0.0010) [2023-12-27 11:46:59,164][20381] Updated weights for policy 1, policy_version 885217 (0.0008) [2023-12-27 11:46:59,228][20381] Updated weights for policy 1, policy_version 885227 (0.0010) [2023-12-27 11:46:59,302][20381] Updated weights for policy 1, policy_version 885237 (0.0008) [2023-12-27 11:47:00,073][20347] Updated weights for policy 0, policy_version 885907 (0.0008) [2023-12-27 11:47:00,136][20347] Updated weights for policy 0, policy_version 885917 (0.0008) [2023-12-27 11:47:00,191][20347] Updated weights for policy 0, policy_version 885927 (0.0006) [2023-12-27 11:47:00,197][20381] Updated weights for policy 1, policy_version 885247 (0.0010) [2023-12-27 11:47:00,259][20381] Updated weights for policy 1, policy_version 885257 (0.0010) [2023-12-27 11:47:00,320][20381] Updated weights for policy 1, policy_version 885267 (0.0008) [2023-12-27 11:47:00,871][20347] Updated weights for policy 0, policy_version 885937 (0.0007) [2023-12-27 11:47:00,944][20347] Updated weights for policy 0, policy_version 885947 (0.0009) [2023-12-27 11:47:01,014][20347] Updated weights for policy 0, policy_version 885957 (0.0008) [2023-12-27 11:47:01,036][20381] Updated weights for policy 1, policy_version 885277 (0.0007) [2023-12-27 11:47:01,093][20347] Updated weights for policy 0, policy_version 885967 (0.0009) [2023-12-27 11:47:01,120][20381] Updated weights for policy 1, policy_version 885289 (0.0010) [2023-12-27 11:47:01,192][20381] Updated weights for policy 1, policy_version 885299 (0.0010) [2023-12-27 11:47:01,946][20347] Updated weights for policy 0, policy_version 885977 (0.0008) [2023-12-27 11:47:02,014][20347] Updated weights for policy 0, policy_version 885987 (0.0008) [2023-12-27 11:47:02,084][20347] Updated weights for policy 0, policy_version 885997 (0.0008) [2023-12-27 11:47:02,126][20381] Updated weights for policy 1, policy_version 885309 (0.0008) [2023-12-27 11:47:02,190][20381] Updated weights for policy 1, policy_version 885319 (0.0008) [2023-12-27 11:47:02,252][20381] Updated weights for policy 1, policy_version 885329 (0.0008) [2023-12-27 11:47:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19114.7, 300 sec: 20077.3). Total num frames: 453517312. Throughput: 0: 9377.3, 1: 9517.0. Samples: 453522764. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:02,363][19170] Avg episode reward: [(0, '863.630'), (1, '1000.000')] [2023-12-27 11:47:02,981][20347] Updated weights for policy 0, policy_version 886007 (0.0008) [2023-12-27 11:47:03,040][20347] Updated weights for policy 0, policy_version 886017 (0.0006) [2023-12-27 11:47:03,081][20381] Updated weights for policy 1, policy_version 885339 (0.0009) [2023-12-27 11:47:03,104][20347] Updated weights for policy 0, policy_version 886027 (0.0008) [2023-12-27 11:47:03,140][20381] Updated weights for policy 1, policy_version 885349 (0.0005) [2023-12-27 11:47:03,206][20381] Updated weights for policy 1, policy_version 885359 (0.0008) [2023-12-27 11:47:03,896][20347] Updated weights for policy 0, policy_version 886037 (0.0007) [2023-12-27 11:47:03,935][20381] Updated weights for policy 1, policy_version 885369 (0.0007) [2023-12-27 11:47:03,962][20347] Updated weights for policy 0, policy_version 886047 (0.0008) [2023-12-27 11:47:04,000][20381] Updated weights for policy 1, policy_version 885379 (0.0009) [2023-12-27 11:47:04,029][20347] Updated weights for policy 0, policy_version 886057 (0.0007) [2023-12-27 11:47:04,056][20381] Updated weights for policy 1, policy_version 885389 (0.0009) [2023-12-27 11:47:04,069][20291] KL-divergence is very high: 144.6350 [2023-12-27 11:47:04,120][20381] Updated weights for policy 1, policy_version 885399 (0.0009) [2023-12-27 11:47:04,771][20291] KL-divergence is very high: 207.2549 [2023-12-27 11:47:04,789][20347] Updated weights for policy 0, policy_version 886067 (0.0007) [2023-12-27 11:47:04,850][20347] Updated weights for policy 0, policy_version 886078 (0.0008) [2023-12-27 11:47:04,856][20291] KL-divergence is very high: 131.1418 [2023-12-27 11:47:04,868][20381] Updated weights for policy 1, policy_version 885409 (0.0007) [2023-12-27 11:47:04,900][20291] KL-divergence is very high: 122.0685 [2023-12-27 11:47:04,907][20347] Updated weights for policy 0, policy_version 886088 (0.0007) [2023-12-27 11:47:04,916][20381] Updated weights for policy 1, policy_version 885419 (0.0007) [2023-12-27 11:47:04,970][20381] Updated weights for policy 1, policy_version 885429 (0.0007) [2023-12-27 11:47:05,624][20381] Updated weights for policy 1, policy_version 885439 (0.0007) [2023-12-27 11:47:05,681][20381] Updated weights for policy 1, policy_version 885449 (0.0008) [2023-12-27 11:47:05,684][20291] KL-divergence is very high: 148.9493 [2023-12-27 11:47:05,698][20347] Updated weights for policy 0, policy_version 886098 (0.0007) [2023-12-27 11:47:05,745][20381] Updated weights for policy 1, policy_version 885459 (0.0007) [2023-12-27 11:47:05,755][20347] Updated weights for policy 0, policy_version 886108 (0.0007) [2023-12-27 11:47:05,810][20347] Updated weights for policy 0, policy_version 886118 (0.0007) [2023-12-27 11:47:05,871][20347] Updated weights for policy 0, policy_version 886128 (0.0008) [2023-12-27 11:47:06,428][20381] Updated weights for policy 1, policy_version 885469 (0.0007) [2023-12-27 11:47:06,497][20381] Updated weights for policy 1, policy_version 885479 (0.0007) [2023-12-27 11:47:06,558][20381] Updated weights for policy 1, policy_version 885489 (0.0008) [2023-12-27 11:47:06,648][20347] Updated weights for policy 0, policy_version 886138 (0.0007) [2023-12-27 11:47:06,710][20347] Updated weights for policy 0, policy_version 886148 (0.0008) [2023-12-27 11:47:06,766][20347] Updated weights for policy 0, policy_version 886158 (0.0008) [2023-12-27 11:47:07,243][20381] Updated weights for policy 1, policy_version 885499 (0.0008) [2023-12-27 11:47:07,298][20381] Updated weights for policy 1, policy_version 885509 (0.0006) [2023-12-27 11:47:07,355][20381] Updated weights for policy 1, policy_version 885519 (0.0008) [2023-12-27 11:47:07,362][19170] Fps is (10 sec: 17203.0, 60 sec: 18841.6, 300 sec: 20021.8). Total num frames: 453599232. Throughput: 0: 9337.6, 1: 9495.8. Samples: 453577012. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:07,363][19170] Avg episode reward: [(0, '814.630'), (1, '1000.000')] [2023-12-27 11:47:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000886160_226885632.pth... [2023-12-27 11:47:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000885104_226615296.pth [2023-12-27 11:47:07,415][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000885528_226721792.pth... [2023-12-27 11:47:07,419][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000884408_226435072.pth [2023-12-27 11:47:07,548][20347] Updated weights for policy 0, policy_version 886168 (0.0006) [2023-12-27 11:47:07,611][20347] Updated weights for policy 0, policy_version 886178 (0.0006) [2023-12-27 11:47:07,670][20347] Updated weights for policy 0, policy_version 886188 (0.0006) [2023-12-27 11:47:08,064][20381] Updated weights for policy 1, policy_version 885529 (0.0006) [2023-12-27 11:47:08,122][20381] Updated weights for policy 1, policy_version 885539 (0.0008) [2023-12-27 11:47:08,188][20381] Updated weights for policy 1, policy_version 885549 (0.0008) [2023-12-27 11:47:08,229][20347] Updated weights for policy 0, policy_version 886198 (0.0007) [2023-12-27 11:47:08,253][20381] Updated weights for policy 1, policy_version 885559 (0.0009) [2023-12-27 11:47:08,291][20347] Updated weights for policy 0, policy_version 886208 (0.0006) [2023-12-27 11:47:08,360][20347] Updated weights for policy 0, policy_version 886218 (0.0009) [2023-12-27 11:47:09,033][20381] Updated weights for policy 1, policy_version 885569 (0.0007) [2023-12-27 11:47:09,037][20347] Updated weights for policy 0, policy_version 886228 (0.0009) [2023-12-27 11:47:09,097][20347] Updated weights for policy 0, policy_version 886238 (0.0010) [2023-12-27 11:47:09,099][20381] Updated weights for policy 1, policy_version 885579 (0.0008) [2023-12-27 11:47:09,154][20347] Updated weights for policy 0, policy_version 886248 (0.0011) [2023-12-27 11:47:09,154][20381] Updated weights for policy 1, policy_version 885589 (0.0008) [2023-12-27 11:47:09,905][20347] Updated weights for policy 0, policy_version 886258 (0.0010) [2023-12-27 11:47:09,978][20347] Updated weights for policy 0, policy_version 886268 (0.0007) [2023-12-27 11:47:09,984][20381] Updated weights for policy 1, policy_version 885599 (0.0007) [2023-12-27 11:47:10,046][20347] Updated weights for policy 0, policy_version 886278 (0.0007) [2023-12-27 11:47:10,048][20381] Updated weights for policy 1, policy_version 885609 (0.0007) [2023-12-27 11:47:10,110][20347] Updated weights for policy 0, policy_version 886288 (0.0010) [2023-12-27 11:47:10,112][20381] Updated weights for policy 1, policy_version 885619 (0.0009) [2023-12-27 11:47:10,830][20347] Updated weights for policy 0, policy_version 886298 (0.0007) [2023-12-27 11:47:10,893][20381] Updated weights for policy 1, policy_version 885629 (0.0006) [2023-12-27 11:47:10,893][20347] Updated weights for policy 0, policy_version 886308 (0.0008) [2023-12-27 11:47:10,954][20381] Updated weights for policy 1, policy_version 885639 (0.0008) [2023-12-27 11:47:10,982][20347] Updated weights for policy 0, policy_version 886318 (0.0007) [2023-12-27 11:47:11,019][20381] Updated weights for policy 1, policy_version 885649 (0.0008) [2023-12-27 11:47:11,714][20347] Updated weights for policy 0, policy_version 886328 (0.0008) [2023-12-27 11:47:11,787][20347] Updated weights for policy 0, policy_version 886338 (0.0009) [2023-12-27 11:47:11,818][20381] Updated weights for policy 1, policy_version 885659 (0.0008) [2023-12-27 11:47:11,852][20347] Updated weights for policy 0, policy_version 886348 (0.0008) [2023-12-27 11:47:11,884][20381] Updated weights for policy 1, policy_version 885669 (0.0010) [2023-12-27 11:47:11,942][20381] Updated weights for policy 1, policy_version 885679 (0.0010) [2023-12-27 11:47:12,365][19170] Fps is (10 sec: 18016.8, 60 sec: 18977.2, 300 sec: 20021.6). Total num frames: 453697536. Throughput: 0: 9398.8, 1: 9450.4. Samples: 453690960. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:12,366][19170] Avg episode reward: [(0, '873.230'), (1, '1000.000')] [2023-12-27 11:47:12,635][20347] Updated weights for policy 0, policy_version 886358 (0.0008) [2023-12-27 11:47:12,693][20381] Updated weights for policy 1, policy_version 885689 (0.0010) [2023-12-27 11:47:12,703][20347] Updated weights for policy 0, policy_version 886368 (0.0007) [2023-12-27 11:47:12,755][20381] Updated weights for policy 1, policy_version 885699 (0.0006) [2023-12-27 11:47:12,772][20347] Updated weights for policy 0, policy_version 886378 (0.0006) [2023-12-27 11:47:12,820][20381] Updated weights for policy 1, policy_version 885709 (0.0006) [2023-12-27 11:47:12,886][20381] Updated weights for policy 1, policy_version 885719 (0.0008) [2023-12-27 11:47:13,535][20347] Updated weights for policy 0, policy_version 886388 (0.0009) [2023-12-27 11:47:13,566][20381] Updated weights for policy 1, policy_version 885729 (0.0007) [2023-12-27 11:47:13,604][20347] Updated weights for policy 0, policy_version 886398 (0.0008) [2023-12-27 11:47:13,632][20381] Updated weights for policy 1, policy_version 885739 (0.0008) [2023-12-27 11:47:13,664][20347] Updated weights for policy 0, policy_version 886408 (0.0008) [2023-12-27 11:47:13,690][20381] Updated weights for policy 1, policy_version 885749 (0.0008) [2023-12-27 11:47:14,390][20347] Updated weights for policy 0, policy_version 886418 (0.0009) [2023-12-27 11:47:14,408][20381] Updated weights for policy 1, policy_version 885759 (0.0007) [2023-12-27 11:47:14,451][20347] Updated weights for policy 0, policy_version 886428 (0.0007) [2023-12-27 11:47:14,465][20381] Updated weights for policy 1, policy_version 885769 (0.0006) [2023-12-27 11:47:14,510][20347] Updated weights for policy 0, policy_version 886438 (0.0007) [2023-12-27 11:47:14,524][20381] Updated weights for policy 1, policy_version 885779 (0.0006) [2023-12-27 11:47:14,565][20347] Updated weights for policy 0, policy_version 886448 (0.0006) [2023-12-27 11:47:15,221][20381] Updated weights for policy 1, policy_version 885789 (0.0006) [2023-12-27 11:47:15,285][20381] Updated weights for policy 1, policy_version 885799 (0.0009) [2023-12-27 11:47:15,312][20347] Updated weights for policy 0, policy_version 886458 (0.0008) [2023-12-27 11:47:15,350][20381] Updated weights for policy 1, policy_version 885809 (0.0008) [2023-12-27 11:47:15,376][20347] Updated weights for policy 0, policy_version 886468 (0.0007) [2023-12-27 11:47:15,441][20347] Updated weights for policy 0, policy_version 886478 (0.0008) [2023-12-27 11:47:16,081][20381] Updated weights for policy 1, policy_version 885819 (0.0007) [2023-12-27 11:47:16,137][20381] Updated weights for policy 1, policy_version 885829 (0.0011) [2023-12-27 11:47:16,178][20347] Updated weights for policy 0, policy_version 886488 (0.0007) [2023-12-27 11:47:16,197][20381] Updated weights for policy 1, policy_version 885839 (0.0009) [2023-12-27 11:47:16,230][20347] Updated weights for policy 0, policy_version 886498 (0.0008) [2023-12-27 11:47:16,289][20347] Updated weights for policy 0, policy_version 886508 (0.0006) [2023-12-27 11:47:16,835][20381] Updated weights for policy 1, policy_version 885849 (0.0006) [2023-12-27 11:47:16,896][20381] Updated weights for policy 1, policy_version 885859 (0.0009) [2023-12-27 11:47:16,953][20381] Updated weights for policy 1, policy_version 885869 (0.0007) [2023-12-27 11:47:16,964][20347] Updated weights for policy 0, policy_version 886518 (0.0006) [2023-12-27 11:47:17,012][20381] Updated weights for policy 1, policy_version 885879 (0.0010) [2023-12-27 11:47:17,021][20347] Updated weights for policy 0, policy_version 886528 (0.0006) [2023-12-27 11:47:17,079][20347] Updated weights for policy 0, policy_version 886538 (0.0008) [2023-12-27 11:47:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 18978.1, 300 sec: 20021.8). Total num frames: 453795840. Throughput: 0: 9321.3, 1: 9433.9. Samples: 453804056. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:17,363][19170] Avg episode reward: [(0, '931.650'), (1, '1000.000')] [2023-12-27 11:47:17,728][20381] Updated weights for policy 1, policy_version 885889 (0.0008) [2023-12-27 11:47:17,747][20347] Updated weights for policy 0, policy_version 886548 (0.0007) [2023-12-27 11:47:17,787][20381] Updated weights for policy 1, policy_version 885899 (0.0008) [2023-12-27 11:47:17,803][20347] Updated weights for policy 0, policy_version 886558 (0.0007) [2023-12-27 11:47:17,842][20381] Updated weights for policy 1, policy_version 885909 (0.0008) [2023-12-27 11:47:17,859][20347] Updated weights for policy 0, policy_version 886568 (0.0007) [2023-12-27 11:47:18,580][20347] Updated weights for policy 0, policy_version 886578 (0.0007) [2023-12-27 11:47:18,583][20381] Updated weights for policy 1, policy_version 885919 (0.0008) [2023-12-27 11:47:18,643][20381] Updated weights for policy 1, policy_version 885929 (0.0008) [2023-12-27 11:47:18,643][20347] Updated weights for policy 0, policy_version 886588 (0.0007) [2023-12-27 11:47:18,700][20381] Updated weights for policy 1, policy_version 885939 (0.0008) [2023-12-27 11:47:18,710][20347] Updated weights for policy 0, policy_version 886598 (0.0008) [2023-12-27 11:47:18,777][20347] Updated weights for policy 0, policy_version 886608 (0.0009) [2023-12-27 11:47:19,503][20381] Updated weights for policy 1, policy_version 885949 (0.0008) [2023-12-27 11:47:19,544][20347] Updated weights for policy 0, policy_version 886618 (0.0007) [2023-12-27 11:47:19,561][20381] Updated weights for policy 1, policy_version 885959 (0.0008) [2023-12-27 11:47:19,600][20347] Updated weights for policy 0, policy_version 886628 (0.0008) [2023-12-27 11:47:19,630][20381] Updated weights for policy 1, policy_version 885969 (0.0008) [2023-12-27 11:47:19,657][20347] Updated weights for policy 0, policy_version 886638 (0.0007) [2023-12-27 11:47:20,371][20381] Updated weights for policy 1, policy_version 885979 (0.0009) [2023-12-27 11:47:20,407][20347] Updated weights for policy 0, policy_version 886648 (0.0008) [2023-12-27 11:47:20,439][20381] Updated weights for policy 1, policy_version 885989 (0.0006) [2023-12-27 11:47:20,471][20347] Updated weights for policy 0, policy_version 886658 (0.0008) [2023-12-27 11:47:20,500][20381] Updated weights for policy 1, policy_version 885999 (0.0007) [2023-12-27 11:47:20,528][20347] Updated weights for policy 0, policy_version 886668 (0.0008) [2023-12-27 11:47:21,251][20381] Updated weights for policy 1, policy_version 886009 (0.0007) [2023-12-27 11:47:21,317][20381] Updated weights for policy 1, policy_version 886019 (0.0008) [2023-12-27 11:47:21,320][20347] Updated weights for policy 0, policy_version 886678 (0.0009) [2023-12-27 11:47:21,390][20381] Updated weights for policy 1, policy_version 886029 (0.0008) [2023-12-27 11:47:21,396][20347] Updated weights for policy 0, policy_version 886688 (0.0010) [2023-12-27 11:47:21,455][20381] Updated weights for policy 1, policy_version 886039 (0.0007) [2023-12-27 11:47:21,462][20347] Updated weights for policy 0, policy_version 886698 (0.0010) [2023-12-27 11:47:22,185][20347] Updated weights for policy 0, policy_version 886708 (0.0008) [2023-12-27 11:47:22,239][20381] Updated weights for policy 1, policy_version 886049 (0.0007) [2023-12-27 11:47:22,244][20347] Updated weights for policy 0, policy_version 886718 (0.0006) [2023-12-27 11:47:22,301][20381] Updated weights for policy 1, policy_version 886059 (0.0008) [2023-12-27 11:47:22,306][20347] Updated weights for policy 0, policy_version 886728 (0.0006) [2023-12-27 11:47:22,362][19170] Fps is (10 sec: 18847.2, 60 sec: 18841.6, 300 sec: 19994.0). Total num frames: 453885952. Throughput: 0: 9300.7, 1: 9428.5. Samples: 453860652. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:22,363][19170] Avg episode reward: [(0, '912.300'), (1, '1000.000')] [2023-12-27 11:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000886736_227033088.pth... [2023-12-27 11:47:22,371][20381] Updated weights for policy 1, policy_version 886069 (0.0008) [2023-12-27 11:47:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000885648_226754560.pth [2023-12-27 11:47:22,387][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000886072_226861056.pth... [2023-12-27 11:47:22,391][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000884952_226574336.pth [2023-12-27 11:47:23,031][20347] Updated weights for policy 0, policy_version 886738 (0.0007) [2023-12-27 11:47:23,087][20347] Updated weights for policy 0, policy_version 886748 (0.0010) [2023-12-27 11:47:23,101][20381] Updated weights for policy 1, policy_version 886079 (0.0007) [2023-12-27 11:47:23,137][20347] Updated weights for policy 0, policy_version 886758 (0.0010) [2023-12-27 11:47:23,151][20381] Updated weights for policy 1, policy_version 886089 (0.0005) [2023-12-27 11:47:23,186][20347] Updated weights for policy 0, policy_version 886768 (0.0010) [2023-12-27 11:47:23,201][20381] Updated weights for policy 1, policy_version 886099 (0.0006) [2023-12-27 11:47:23,845][20347] Updated weights for policy 0, policy_version 886778 (0.0007) [2023-12-27 11:47:23,898][20347] Updated weights for policy 0, policy_version 886788 (0.0007) [2023-12-27 11:47:23,950][20347] Updated weights for policy 0, policy_version 886798 (0.0008) [2023-12-27 11:47:23,959][20381] Updated weights for policy 1, policy_version 886109 (0.0007) [2023-12-27 11:47:24,016][20381] Updated weights for policy 1, policy_version 886119 (0.0009) [2023-12-27 11:47:24,070][20381] Updated weights for policy 1, policy_version 886129 (0.0007) [2023-12-27 11:47:24,613][20347] Updated weights for policy 0, policy_version 886808 (0.0006) [2023-12-27 11:47:24,681][20347] Updated weights for policy 0, policy_version 886818 (0.0008) [2023-12-27 11:47:24,745][20347] Updated weights for policy 0, policy_version 886828 (0.0008) [2023-12-27 11:47:24,817][20381] Updated weights for policy 1, policy_version 886139 (0.0007) [2023-12-27 11:47:24,882][20381] Updated weights for policy 1, policy_version 886149 (0.0009) [2023-12-27 11:47:24,938][20381] Updated weights for policy 1, policy_version 886159 (0.0007) [2023-12-27 11:47:25,393][20347] Updated weights for policy 0, policy_version 886838 (0.0009) [2023-12-27 11:47:25,458][20347] Updated weights for policy 0, policy_version 886848 (0.0008) [2023-12-27 11:47:25,528][20347] Updated weights for policy 0, policy_version 886858 (0.0006) [2023-12-27 11:47:25,627][20381] Updated weights for policy 1, policy_version 886169 (0.0007) [2023-12-27 11:47:25,688][20381] Updated weights for policy 1, policy_version 886179 (0.0005) [2023-12-27 11:47:25,747][20381] Updated weights for policy 1, policy_version 886189 (0.0006) [2023-12-27 11:47:25,813][20381] Updated weights for policy 1, policy_version 886199 (0.0007) [2023-12-27 11:47:26,242][20347] Updated weights for policy 0, policy_version 886868 (0.0007) [2023-12-27 11:47:26,296][20347] Updated weights for policy 0, policy_version 886878 (0.0007) [2023-12-27 11:47:26,352][20347] Updated weights for policy 0, policy_version 886888 (0.0008) [2023-12-27 11:47:26,538][20381] Updated weights for policy 1, policy_version 886209 (0.0007) [2023-12-27 11:47:26,600][20381] Updated weights for policy 1, policy_version 886219 (0.0005) [2023-12-27 11:47:26,651][20381] Updated weights for policy 1, policy_version 886229 (0.0006) [2023-12-27 11:47:26,993][20347] Updated weights for policy 0, policy_version 886898 (0.0007) [2023-12-27 11:47:27,044][20347] Updated weights for policy 0, policy_version 886908 (0.0006) [2023-12-27 11:47:27,100][20347] Updated weights for policy 0, policy_version 886918 (0.0006) [2023-12-27 11:47:27,152][20347] Updated weights for policy 0, policy_version 886928 (0.0008) [2023-12-27 11:47:27,314][20381] Updated weights for policy 1, policy_version 886239 (0.0007) [2023-12-27 11:47:27,362][19170] Fps is (10 sec: 18841.5, 60 sec: 18841.6, 300 sec: 19966.3). Total num frames: 453984256. Throughput: 0: 9320.1, 1: 9410.7. Samples: 453975600. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:27,363][19170] Avg episode reward: [(0, '912.240'), (1, '1000.000')] [2023-12-27 11:47:27,375][20381] Updated weights for policy 1, policy_version 886249 (0.0008) [2023-12-27 11:47:27,423][20381] Updated weights for policy 1, policy_version 886259 (0.0007) [2023-12-27 11:47:27,792][20347] Updated weights for policy 0, policy_version 886938 (0.0009) [2023-12-27 11:47:27,848][20347] Updated weights for policy 0, policy_version 886948 (0.0008) [2023-12-27 11:47:27,909][20347] Updated weights for policy 0, policy_version 886958 (0.0007) [2023-12-27 11:47:28,115][20381] Updated weights for policy 1, policy_version 886269 (0.0008) [2023-12-27 11:47:28,171][20381] Updated weights for policy 1, policy_version 886279 (0.0007) [2023-12-27 11:47:28,221][20381] Updated weights for policy 1, policy_version 886289 (0.0007) [2023-12-27 11:47:28,620][20347] Updated weights for policy 0, policy_version 886968 (0.0007) [2023-12-27 11:47:28,681][20347] Updated weights for policy 0, policy_version 886978 (0.0008) [2023-12-27 11:47:28,740][20347] Updated weights for policy 0, policy_version 886988 (0.0007) [2023-12-27 11:47:28,960][20381] Updated weights for policy 1, policy_version 886299 (0.0008) [2023-12-27 11:47:29,023][20381] Updated weights for policy 1, policy_version 886309 (0.0008) [2023-12-27 11:47:29,084][20381] Updated weights for policy 1, policy_version 886319 (0.0007) [2023-12-27 11:47:29,486][20347] Updated weights for policy 0, policy_version 886998 (0.0006) [2023-12-27 11:47:29,546][20347] Updated weights for policy 0, policy_version 887008 (0.0006) [2023-12-27 11:47:29,604][20347] Updated weights for policy 0, policy_version 887018 (0.0006) [2023-12-27 11:47:29,859][20381] Updated weights for policy 1, policy_version 886329 (0.0008) [2023-12-27 11:47:29,930][20381] Updated weights for policy 1, policy_version 886339 (0.0008) [2023-12-27 11:47:29,994][20381] Updated weights for policy 1, policy_version 886349 (0.0006) [2023-12-27 11:47:30,056][20381] Updated weights for policy 1, policy_version 886359 (0.0006) [2023-12-27 11:47:30,350][20347] Updated weights for policy 0, policy_version 887028 (0.0008) [2023-12-27 11:47:30,406][20347] Updated weights for policy 0, policy_version 887038 (0.0008) [2023-12-27 11:47:30,462][20347] Updated weights for policy 0, policy_version 887048 (0.0008) [2023-12-27 11:47:30,678][20381] Updated weights for policy 1, policy_version 886369 (0.0006) [2023-12-27 11:47:30,740][20381] Updated weights for policy 1, policy_version 886379 (0.0006) [2023-12-27 11:47:30,808][20381] Updated weights for policy 1, policy_version 886389 (0.0008) [2023-12-27 11:47:31,134][20347] Updated weights for policy 0, policy_version 887058 (0.0008) [2023-12-27 11:47:31,202][20347] Updated weights for policy 0, policy_version 887068 (0.0009) [2023-12-27 11:47:31,264][20347] Updated weights for policy 0, policy_version 887078 (0.0010) [2023-12-27 11:47:31,330][20347] Updated weights for policy 0, policy_version 887088 (0.0010) [2023-12-27 11:47:31,563][20381] Updated weights for policy 1, policy_version 886399 (0.0008) [2023-12-27 11:47:31,623][20381] Updated weights for policy 1, policy_version 886409 (0.0010) [2023-12-27 11:47:31,683][20381] Updated weights for policy 1, policy_version 886419 (0.0007) [2023-12-27 11:47:32,061][20347] Updated weights for policy 0, policy_version 887098 (0.0009) [2023-12-27 11:47:32,124][20347] Updated weights for policy 0, policy_version 887108 (0.0009) [2023-12-27 11:47:32,188][20347] Updated weights for policy 0, policy_version 887118 (0.0009) [2023-12-27 11:47:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 18841.6, 300 sec: 19966.3). Total num frames: 454082560. Throughput: 0: 9418.2, 1: 9458.9. Samples: 454094100. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:32,363][19170] Avg episode reward: [(0, '814.950'), (1, '1000.000')] [2023-12-27 11:47:32,393][20381] Updated weights for policy 1, policy_version 886429 (0.0008) [2023-12-27 11:47:32,456][20381] Updated weights for policy 1, policy_version 886439 (0.0006) [2023-12-27 11:47:32,513][20381] Updated weights for policy 1, policy_version 886449 (0.0007) [2023-12-27 11:47:32,863][20347] Updated weights for policy 0, policy_version 887128 (0.0008) [2023-12-27 11:47:32,926][20347] Updated weights for policy 0, policy_version 887138 (0.0010) [2023-12-27 11:47:32,986][20347] Updated weights for policy 0, policy_version 887148 (0.0010) [2023-12-27 11:47:33,182][20381] Updated weights for policy 1, policy_version 886459 (0.0008) [2023-12-27 11:47:33,241][20381] Updated weights for policy 1, policy_version 886469 (0.0006) [2023-12-27 11:47:33,297][20381] Updated weights for policy 1, policy_version 886479 (0.0007) [2023-12-27 11:47:33,655][20347] Updated weights for policy 0, policy_version 887158 (0.0010) [2023-12-27 11:47:33,708][20347] Updated weights for policy 0, policy_version 887168 (0.0010) [2023-12-27 11:47:33,762][20347] Updated weights for policy 0, policy_version 887178 (0.0010) [2023-12-27 11:47:33,966][20381] Updated weights for policy 1, policy_version 886489 (0.0008) [2023-12-27 11:47:34,020][20381] Updated weights for policy 1, policy_version 886499 (0.0007) [2023-12-27 11:47:34,085][20381] Updated weights for policy 1, policy_version 886509 (0.0006) [2023-12-27 11:47:34,140][20381] Updated weights for policy 1, policy_version 886519 (0.0006) [2023-12-27 11:47:34,443][20347] Updated weights for policy 0, policy_version 887188 (0.0010) [2023-12-27 11:47:34,506][20347] Updated weights for policy 0, policy_version 887198 (0.0009) [2023-12-27 11:47:34,568][20347] Updated weights for policy 0, policy_version 887208 (0.0008) [2023-12-27 11:47:34,785][20381] Updated weights for policy 1, policy_version 886529 (0.0006) [2023-12-27 11:47:34,839][20381] Updated weights for policy 1, policy_version 886539 (0.0006) [2023-12-27 11:47:34,899][20381] Updated weights for policy 1, policy_version 886549 (0.0007) [2023-12-27 11:47:35,282][20347] Updated weights for policy 0, policy_version 887218 (0.0009) [2023-12-27 11:47:35,341][20347] Updated weights for policy 0, policy_version 887228 (0.0010) [2023-12-27 11:47:35,400][20347] Updated weights for policy 0, policy_version 887238 (0.0010) [2023-12-27 11:47:35,457][20347] Updated weights for policy 0, policy_version 887248 (0.0010) [2023-12-27 11:47:35,491][20381] Updated weights for policy 1, policy_version 886559 (0.0006) [2023-12-27 11:47:35,548][20381] Updated weights for policy 1, policy_version 886569 (0.0009) [2023-12-27 11:47:35,600][20381] Updated weights for policy 1, policy_version 886579 (0.0008) [2023-12-27 11:47:36,101][20347] Updated weights for policy 0, policy_version 887258 (0.0010) [2023-12-27 11:47:36,167][20347] Updated weights for policy 0, policy_version 887268 (0.0010) [2023-12-27 11:47:36,239][20347] Updated weights for policy 0, policy_version 887278 (0.0008) [2023-12-27 11:47:36,286][20381] Updated weights for policy 1, policy_version 886589 (0.0007) [2023-12-27 11:47:36,352][20381] Updated weights for policy 1, policy_version 886599 (0.0007) [2023-12-27 11:47:36,416][20381] Updated weights for policy 1, policy_version 886609 (0.0007) [2023-12-27 11:47:36,928][20347] Updated weights for policy 0, policy_version 887288 (0.0008) [2023-12-27 11:47:36,988][20347] Updated weights for policy 0, policy_version 887298 (0.0007) [2023-12-27 11:47:37,042][20347] Updated weights for policy 0, policy_version 887308 (0.0007) [2023-12-27 11:47:37,117][20381] Updated weights for policy 1, policy_version 886619 (0.0007) [2023-12-27 11:47:37,176][20381] Updated weights for policy 1, policy_version 886629 (0.0008) [2023-12-27 11:47:37,237][20381] Updated weights for policy 1, policy_version 886639 (0.0007) [2023-12-27 11:47:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 19114.7, 300 sec: 19994.0). Total num frames: 454189056. Throughput: 0: 9472.3, 1: 9503.7. Samples: 454155336. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:37,363][19170] Avg episode reward: [(0, '824.690'), (1, '1000.000')] [2023-12-27 11:47:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000886648_227008512.pth... [2023-12-27 11:47:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000887312_227180544.pth... [2023-12-27 11:47:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000886160_226885632.pth [2023-12-27 11:47:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000885528_226721792.pth [2023-12-27 11:47:37,711][20347] Updated weights for policy 0, policy_version 887318 (0.0010) [2023-12-27 11:47:37,774][20347] Updated weights for policy 0, policy_version 887328 (0.0011) [2023-12-27 11:47:37,829][20347] Updated weights for policy 0, policy_version 887338 (0.0008) [2023-12-27 11:47:37,961][20381] Updated weights for policy 1, policy_version 886649 (0.0009) [2023-12-27 11:47:38,022][20381] Updated weights for policy 1, policy_version 886659 (0.0007) [2023-12-27 11:47:38,079][20381] Updated weights for policy 1, policy_version 886669 (0.0009) [2023-12-27 11:47:38,139][20381] Updated weights for policy 1, policy_version 886679 (0.0008) [2023-12-27 11:47:38,527][20347] Updated weights for policy 0, policy_version 887348 (0.0010) [2023-12-27 11:47:38,579][20347] Updated weights for policy 0, policy_version 887358 (0.0008) [2023-12-27 11:47:38,638][20347] Updated weights for policy 0, policy_version 887368 (0.0009) [2023-12-27 11:47:38,825][20381] Updated weights for policy 1, policy_version 886689 (0.0008) [2023-12-27 11:47:38,889][20381] Updated weights for policy 1, policy_version 886699 (0.0008) [2023-12-27 11:47:38,951][20381] Updated weights for policy 1, policy_version 886709 (0.0008) [2023-12-27 11:47:39,434][20347] Updated weights for policy 0, policy_version 887378 (0.0007) [2023-12-27 11:47:39,494][20347] Updated weights for policy 0, policy_version 887388 (0.0006) [2023-12-27 11:47:39,557][20347] Updated weights for policy 0, policy_version 887398 (0.0007) [2023-12-27 11:47:39,623][20347] Updated weights for policy 0, policy_version 887408 (0.0007) [2023-12-27 11:47:39,681][20381] Updated weights for policy 1, policy_version 886719 (0.0008) [2023-12-27 11:47:39,743][20381] Updated weights for policy 1, policy_version 886729 (0.0008) [2023-12-27 11:47:39,806][20381] Updated weights for policy 1, policy_version 886739 (0.0008) [2023-12-27 11:47:40,331][20347] Updated weights for policy 0, policy_version 887418 (0.0008) [2023-12-27 11:47:40,399][20347] Updated weights for policy 0, policy_version 887428 (0.0008) [2023-12-27 11:47:40,466][20347] Updated weights for policy 0, policy_version 887438 (0.0008) [2023-12-27 11:47:40,542][20381] Updated weights for policy 1, policy_version 886749 (0.0008) [2023-12-27 11:47:40,600][20381] Updated weights for policy 1, policy_version 886759 (0.0008) [2023-12-27 11:47:40,660][20381] Updated weights for policy 1, policy_version 886769 (0.0008) [2023-12-27 11:47:41,105][20347] Updated weights for policy 0, policy_version 887448 (0.0007) [2023-12-27 11:47:41,168][20347] Updated weights for policy 0, policy_version 887458 (0.0007) [2023-12-27 11:47:41,232][20347] Updated weights for policy 0, policy_version 887468 (0.0008) [2023-12-27 11:47:41,462][20381] Updated weights for policy 1, policy_version 886779 (0.0007) [2023-12-27 11:47:41,522][20381] Updated weights for policy 1, policy_version 886789 (0.0008) [2023-12-27 11:47:41,578][20381] Updated weights for policy 1, policy_version 886799 (0.0008) [2023-12-27 11:47:41,978][20347] Updated weights for policy 0, policy_version 887478 (0.0008) [2023-12-27 11:47:42,031][20291] KL-divergence is very high: 115.2122 [2023-12-27 11:47:42,044][20347] Updated weights for policy 0, policy_version 887488 (0.0007) [2023-12-27 11:47:42,108][20347] Updated weights for policy 0, policy_version 887498 (0.0007) [2023-12-27 11:47:42,326][20381] Updated weights for policy 1, policy_version 886809 (0.0008) [2023-12-27 11:47:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 19114.7, 300 sec: 19938.5). Total num frames: 454279168. Throughput: 0: 9519.3, 1: 9504.5. Samples: 454272756. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:42,363][19170] Avg episode reward: [(0, '843.950'), (1, '1000.000')] [2023-12-27 11:47:42,397][20381] Updated weights for policy 1, policy_version 886819 (0.0007) [2023-12-27 11:47:42,459][20381] Updated weights for policy 1, policy_version 886829 (0.0007) [2023-12-27 11:47:42,524][20381] Updated weights for policy 1, policy_version 886839 (0.0006) [2023-12-27 11:47:42,832][20347] Updated weights for policy 0, policy_version 887508 (0.0007) [2023-12-27 11:47:42,897][20347] Updated weights for policy 0, policy_version 887518 (0.0006) [2023-12-27 11:47:42,960][20347] Updated weights for policy 0, policy_version 887528 (0.0006) [2023-12-27 11:47:43,156][20381] Updated weights for policy 1, policy_version 886849 (0.0006) [2023-12-27 11:47:43,218][20381] Updated weights for policy 1, policy_version 886859 (0.0006) [2023-12-27 11:47:43,276][20381] Updated weights for policy 1, policy_version 886869 (0.0006) [2023-12-27 11:47:43,647][20347] Updated weights for policy 0, policy_version 887538 (0.0008) [2023-12-27 11:47:43,704][20347] Updated weights for policy 0, policy_version 887548 (0.0007) [2023-12-27 11:47:43,735][20291] KL-divergence is very high: 104.8493 [2023-12-27 11:47:43,748][20291] KL-divergence is very high: 147.9717 [2023-12-27 11:47:43,758][20347] Updated weights for policy 0, policy_version 887558 (0.0006) [2023-12-27 11:47:43,815][20347] Updated weights for policy 0, policy_version 887568 (0.0006) [2023-12-27 11:47:43,954][20381] Updated weights for policy 1, policy_version 886879 (0.0006) [2023-12-27 11:47:44,022][20381] Updated weights for policy 1, policy_version 886889 (0.0008) [2023-12-27 11:47:44,088][20381] Updated weights for policy 1, policy_version 886899 (0.0008) [2023-12-27 11:47:44,504][20347] Updated weights for policy 0, policy_version 887578 (0.0005) [2023-12-27 11:47:44,564][20347] Updated weights for policy 0, policy_version 887588 (0.0006) [2023-12-27 11:47:44,615][20347] Updated weights for policy 0, policy_version 887598 (0.0007) [2023-12-27 11:47:44,761][20381] Updated weights for policy 1, policy_version 886909 (0.0009) [2023-12-27 11:47:44,826][20381] Updated weights for policy 1, policy_version 886919 (0.0007) [2023-12-27 11:47:44,882][20381] Updated weights for policy 1, policy_version 886929 (0.0010) [2023-12-27 11:47:45,303][20347] Updated weights for policy 0, policy_version 887608 (0.0008) [2023-12-27 11:47:45,353][20347] Updated weights for policy 0, policy_version 887618 (0.0008) [2023-12-27 11:47:45,409][20347] Updated weights for policy 0, policy_version 887628 (0.0008) [2023-12-27 11:47:45,585][20381] Updated weights for policy 1, policy_version 886939 (0.0009) [2023-12-27 11:47:45,638][20381] Updated weights for policy 1, policy_version 886949 (0.0007) [2023-12-27 11:47:45,695][20381] Updated weights for policy 1, policy_version 886959 (0.0008) [2023-12-27 11:47:46,233][20347] Updated weights for policy 0, policy_version 887638 (0.0008) [2023-12-27 11:47:46,293][20347] Updated weights for policy 0, policy_version 887648 (0.0008) [2023-12-27 11:47:46,364][20347] Updated weights for policy 0, policy_version 887658 (0.0008) [2023-12-27 11:47:46,421][20381] Updated weights for policy 1, policy_version 886969 (0.0007) [2023-12-27 11:47:46,482][20381] Updated weights for policy 1, policy_version 886979 (0.0008) [2023-12-27 11:47:46,538][20381] Updated weights for policy 1, policy_version 886989 (0.0007) [2023-12-27 11:47:46,596][20381] Updated weights for policy 1, policy_version 886999 (0.0008) [2023-12-27 11:47:47,082][20347] Updated weights for policy 0, policy_version 887668 (0.0009) [2023-12-27 11:47:47,141][20347] Updated weights for policy 0, policy_version 887678 (0.0008) [2023-12-27 11:47:47,206][20347] Updated weights for policy 0, policy_version 887688 (0.0008) [2023-12-27 11:47:47,323][20381] Updated weights for policy 1, policy_version 887009 (0.0007) [2023-12-27 11:47:47,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19114.6, 300 sec: 19910.7). Total num frames: 454377472. Throughput: 0: 9615.7, 1: 9632.0. Samples: 454388912. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:47,363][19170] Avg episode reward: [(0, '863.280'), (1, '1000.000')] [2023-12-27 11:47:47,385][20381] Updated weights for policy 1, policy_version 887019 (0.0008) [2023-12-27 11:47:47,447][20381] Updated weights for policy 1, policy_version 887029 (0.0008) [2023-12-27 11:47:47,875][20347] Updated weights for policy 0, policy_version 887698 (0.0006) [2023-12-27 11:47:47,935][20347] Updated weights for policy 0, policy_version 887708 (0.0007) [2023-12-27 11:47:47,995][20347] Updated weights for policy 0, policy_version 887718 (0.0006) [2023-12-27 11:47:48,053][20347] Updated weights for policy 0, policy_version 887728 (0.0006) [2023-12-27 11:47:48,151][20381] Updated weights for policy 1, policy_version 887039 (0.0007) [2023-12-27 11:47:48,212][20381] Updated weights for policy 1, policy_version 887049 (0.0007) [2023-12-27 11:47:48,275][20381] Updated weights for policy 1, policy_version 887059 (0.0007) [2023-12-27 11:47:48,732][20347] Updated weights for policy 0, policy_version 887738 (0.0006) [2023-12-27 11:47:48,786][20347] Updated weights for policy 0, policy_version 887748 (0.0007) [2023-12-27 11:47:48,844][20347] Updated weights for policy 0, policy_version 887758 (0.0006) [2023-12-27 11:47:49,006][20381] Updated weights for policy 1, policy_version 887069 (0.0006) [2023-12-27 11:47:49,067][20381] Updated weights for policy 1, policy_version 887079 (0.0005) [2023-12-27 11:47:49,117][20381] Updated weights for policy 1, policy_version 887089 (0.0006) [2023-12-27 11:47:49,541][20347] Updated weights for policy 0, policy_version 887768 (0.0008) [2023-12-27 11:47:49,564][20291] KL-divergence is very high: 157.4724 [2023-12-27 11:47:49,608][20347] Updated weights for policy 0, policy_version 887778 (0.0008) [2023-12-27 11:47:49,615][20291] KL-divergence is very high: 146.7583 [2023-12-27 11:47:49,680][20347] Updated weights for policy 0, policy_version 887788 (0.0010) [2023-12-27 11:47:49,863][20381] Updated weights for policy 1, policy_version 887099 (0.0009) [2023-12-27 11:47:49,932][20381] Updated weights for policy 1, policy_version 887109 (0.0009) [2023-12-27 11:47:50,003][20381] Updated weights for policy 1, policy_version 887119 (0.0009) [2023-12-27 11:47:50,498][20347] Updated weights for policy 0, policy_version 887798 (0.0009) [2023-12-27 11:47:50,557][20347] Updated weights for policy 0, policy_version 887808 (0.0008) [2023-12-27 11:47:50,627][20347] Updated weights for policy 0, policy_version 887818 (0.0008) [2023-12-27 11:47:50,774][20381] Updated weights for policy 1, policy_version 887129 (0.0007) [2023-12-27 11:47:50,839][20381] Updated weights for policy 1, policy_version 887139 (0.0008) [2023-12-27 11:47:50,904][20381] Updated weights for policy 1, policy_version 887149 (0.0008) [2023-12-27 11:47:50,975][20381] Updated weights for policy 1, policy_version 887159 (0.0008) [2023-12-27 11:47:51,539][20347] Updated weights for policy 0, policy_version 887828 (0.0008) [2023-12-27 11:47:51,603][20347] Updated weights for policy 0, policy_version 887838 (0.0007) [2023-12-27 11:47:51,660][20347] Updated weights for policy 0, policy_version 887848 (0.0006) [2023-12-27 11:47:51,810][20381] Updated weights for policy 1, policy_version 887169 (0.0008) [2023-12-27 11:47:51,877][20381] Updated weights for policy 1, policy_version 887179 (0.0008) [2023-12-27 11:47:51,941][20381] Updated weights for policy 1, policy_version 887189 (0.0008) [2023-12-27 11:47:52,378][19170] Fps is (10 sec: 18812.0, 60 sec: 19109.7, 300 sec: 19854.1). Total num frames: 454467584. Throughput: 0: 9652.4, 1: 9636.6. Samples: 454445316. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:52,379][19170] Avg episode reward: [(0, '834.060'), (1, '1000.000')] [2023-12-27 11:47:52,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000887192_227147776.pth... [2023-12-27 11:47:52,386][20347] Updated weights for policy 0, policy_version 887858 (0.0007) [2023-12-27 11:47:52,389][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000886072_226861056.pth [2023-12-27 11:47:52,453][20347] Updated weights for policy 0, policy_version 887868 (0.0008) [2023-12-27 11:47:52,513][20347] Updated weights for policy 0, policy_version 887878 (0.0007) [2023-12-27 11:47:52,574][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000887888_227328000.pth... [2023-12-27 11:47:52,577][20347] Updated weights for policy 0, policy_version 887888 (0.0008) [2023-12-27 11:47:52,577][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000886736_227033088.pth [2023-12-27 11:47:52,693][20381] Updated weights for policy 1, policy_version 887199 (0.0008) [2023-12-27 11:47:52,753][20381] Updated weights for policy 1, policy_version 887209 (0.0008) [2023-12-27 11:47:52,816][20381] Updated weights for policy 1, policy_version 887219 (0.0008) [2023-12-27 11:47:53,303][20347] Updated weights for policy 0, policy_version 887898 (0.0008) [2023-12-27 11:47:53,360][20291] KL-divergence is very high: 235.7626 [2023-12-27 11:47:53,360][20347] Updated weights for policy 0, policy_version 887908 (0.0007) [2023-12-27 11:47:53,378][20291] KL-divergence is very high: 218.2881 [2023-12-27 11:47:53,391][20291] KL-divergence is very high: 177.9851 [2023-12-27 11:47:53,426][20347] Updated weights for policy 0, policy_version 887918 (0.0007) [2023-12-27 11:47:53,439][20381] Updated weights for policy 1, policy_version 887229 (0.0008) [2023-12-27 11:47:53,498][20381] Updated weights for policy 1, policy_version 887239 (0.0007) [2023-12-27 11:47:53,558][20381] Updated weights for policy 1, policy_version 887249 (0.0008) [2023-12-27 11:47:54,143][20347] Updated weights for policy 0, policy_version 887928 (0.0008) [2023-12-27 11:47:54,194][20347] Updated weights for policy 0, policy_version 887938 (0.0008) [2023-12-27 11:47:54,245][20347] Updated weights for policy 0, policy_version 887948 (0.0008) [2023-12-27 11:47:54,254][20381] Updated weights for policy 1, policy_version 887259 (0.0007) [2023-12-27 11:47:54,307][20381] Updated weights for policy 1, policy_version 887269 (0.0007) [2023-12-27 11:47:54,361][20381] Updated weights for policy 1, policy_version 887279 (0.0008) [2023-12-27 11:47:54,954][20347] Updated weights for policy 0, policy_version 887958 (0.0006) [2023-12-27 11:47:55,017][20347] Updated weights for policy 0, policy_version 887968 (0.0009) [2023-12-27 11:47:55,082][20347] Updated weights for policy 0, policy_version 887978 (0.0007) [2023-12-27 11:47:55,093][20381] Updated weights for policy 1, policy_version 887289 (0.0007) [2023-12-27 11:47:55,142][20381] Updated weights for policy 1, policy_version 887299 (0.0006) [2023-12-27 11:47:55,200][20381] Updated weights for policy 1, policy_version 887309 (0.0008) [2023-12-27 11:47:55,257][20381] Updated weights for policy 1, policy_version 887319 (0.0008) [2023-12-27 11:47:55,779][20347] Updated weights for policy 0, policy_version 887988 (0.0007) [2023-12-27 11:47:55,824][20347] Updated weights for policy 0, policy_version 887998 (0.0007) [2023-12-27 11:47:55,877][20347] Updated weights for policy 0, policy_version 888008 (0.0007) [2023-12-27 11:47:55,889][20381] Updated weights for policy 1, policy_version 887329 (0.0006) [2023-12-27 11:47:55,946][20381] Updated weights for policy 1, policy_version 887339 (0.0007) [2023-12-27 11:47:55,994][20381] Updated weights for policy 1, policy_version 887349 (0.0008) [2023-12-27 11:47:56,592][20347] Updated weights for policy 0, policy_version 888018 (0.0006) [2023-12-27 11:47:56,646][20347] Updated weights for policy 0, policy_version 888028 (0.0008) [2023-12-27 11:47:56,692][20381] Updated weights for policy 1, policy_version 887359 (0.0006) [2023-12-27 11:47:56,700][20347] Updated weights for policy 0, policy_version 888038 (0.0007) [2023-12-27 11:47:56,749][20347] Updated weights for policy 0, policy_version 888048 (0.0005) [2023-12-27 11:47:56,751][20381] Updated weights for policy 1, policy_version 887369 (0.0011) [2023-12-27 11:47:56,799][20381] Updated weights for policy 1, policy_version 887379 (0.0006) [2023-12-27 11:47:57,362][19170] Fps is (10 sec: 18841.7, 60 sec: 18978.1, 300 sec: 19855.2). Total num frames: 454565888. Throughput: 0: 9637.8, 1: 9653.7. Samples: 454559020. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:47:57,363][19170] Avg episode reward: [(0, '804.820'), (1, '1000.000')] [2023-12-27 11:47:57,455][20347] Updated weights for policy 0, policy_version 888058 (0.0007) [2023-12-27 11:47:57,469][20381] Updated weights for policy 1, policy_version 887389 (0.0010) [2023-12-27 11:47:57,509][20347] Updated weights for policy 0, policy_version 888068 (0.0005) [2023-12-27 11:47:57,529][20381] Updated weights for policy 1, policy_version 887399 (0.0011) [2023-12-27 11:47:57,565][20347] Updated weights for policy 0, policy_version 888078 (0.0005) [2023-12-27 11:47:57,585][20381] Updated weights for policy 1, policy_version 887409 (0.0011) [2023-12-27 11:47:58,305][20381] Updated weights for policy 1, policy_version 887419 (0.0009) [2023-12-27 11:47:58,310][20347] Updated weights for policy 0, policy_version 888088 (0.0008) [2023-12-27 11:47:58,311][20291] KL-divergence is very high: 100.7155 [2023-12-27 11:47:58,318][20291] KL-divergence is very high: 112.1102 [2023-12-27 11:47:58,370][20381] Updated weights for policy 1, policy_version 887429 (0.0007) [2023-12-27 11:47:58,377][20347] Updated weights for policy 0, policy_version 888098 (0.0008) [2023-12-27 11:47:58,433][20381] Updated weights for policy 1, policy_version 887439 (0.0007) [2023-12-27 11:47:58,445][20347] Updated weights for policy 0, policy_version 888108 (0.0007) [2023-12-27 11:47:59,178][20347] Updated weights for policy 0, policy_version 888118 (0.0010) [2023-12-27 11:47:59,222][20381] Updated weights for policy 1, policy_version 887449 (0.0008) [2023-12-27 11:47:59,235][20347] Updated weights for policy 0, policy_version 888128 (0.0009) [2023-12-27 11:47:59,286][20381] Updated weights for policy 1, policy_version 887459 (0.0008) [2023-12-27 11:47:59,295][20347] Updated weights for policy 0, policy_version 888138 (0.0010) [2023-12-27 11:47:59,351][20381] Updated weights for policy 1, policy_version 887469 (0.0010) [2023-12-27 11:47:59,409][20381] Updated weights for policy 1, policy_version 887479 (0.0008) [2023-12-27 11:48:00,029][20347] Updated weights for policy 0, policy_version 888148 (0.0009) [2023-12-27 11:48:00,090][20347] Updated weights for policy 0, policy_version 888158 (0.0006) [2023-12-27 11:48:00,119][20381] Updated weights for policy 1, policy_version 887489 (0.0007) [2023-12-27 11:48:00,151][20347] Updated weights for policy 0, policy_version 888168 (0.0007) [2023-12-27 11:48:00,180][20381] Updated weights for policy 1, policy_version 887499 (0.0006) [2023-12-27 11:48:00,237][20381] Updated weights for policy 1, policy_version 887509 (0.0007) [2023-12-27 11:48:00,754][20347] Updated weights for policy 0, policy_version 888178 (0.0007) [2023-12-27 11:48:00,808][20347] Updated weights for policy 0, policy_version 888188 (0.0008) [2023-12-27 11:48:00,864][20347] Updated weights for policy 0, policy_version 888198 (0.0008) [2023-12-27 11:48:00,909][20381] Updated weights for policy 1, policy_version 887519 (0.0008) [2023-12-27 11:48:00,924][20347] Updated weights for policy 0, policy_version 888208 (0.0005) [2023-12-27 11:48:00,969][20381] Updated weights for policy 1, policy_version 887529 (0.0008) [2023-12-27 11:48:01,034][20381] Updated weights for policy 1, policy_version 887539 (0.0007) [2023-12-27 11:48:01,654][20347] Updated weights for policy 0, policy_version 888218 (0.0009) [2023-12-27 11:48:01,715][20347] Updated weights for policy 0, policy_version 888228 (0.0008) [2023-12-27 11:48:01,779][20347] Updated weights for policy 0, policy_version 888238 (0.0008) [2023-12-27 11:48:01,864][20381] Updated weights for policy 1, policy_version 887549 (0.0006) [2023-12-27 11:48:01,925][20381] Updated weights for policy 1, policy_version 887559 (0.0006) [2023-12-27 11:48:01,991][20381] Updated weights for policy 1, policy_version 887569 (0.0006) [2023-12-27 11:48:02,362][19170] Fps is (10 sec: 19691.8, 60 sec: 19114.7, 300 sec: 19855.2). Total num frames: 454664192. Throughput: 0: 9706.6, 1: 9655.6. Samples: 454675352. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:02,363][19170] Avg episode reward: [(0, '804.830'), (1, '1000.000')] [2023-12-27 11:48:02,532][20347] Updated weights for policy 0, policy_version 888248 (0.0008) [2023-12-27 11:48:02,592][20347] Updated weights for policy 0, policy_version 888258 (0.0007) [2023-12-27 11:48:02,647][20347] Updated weights for policy 0, policy_version 888268 (0.0009) [2023-12-27 11:48:02,652][20381] Updated weights for policy 1, policy_version 887579 (0.0006) [2023-12-27 11:48:02,706][20381] Updated weights for policy 1, policy_version 887589 (0.0006) [2023-12-27 11:48:02,763][20381] Updated weights for policy 1, policy_version 887599 (0.0008) [2023-12-27 11:48:03,378][20347] Updated weights for policy 0, policy_version 888278 (0.0007) [2023-12-27 11:48:03,432][20347] Updated weights for policy 0, policy_version 888288 (0.0007) [2023-12-27 11:48:03,481][20381] Updated weights for policy 1, policy_version 887609 (0.0008) [2023-12-27 11:48:03,488][20347] Updated weights for policy 0, policy_version 888298 (0.0006) [2023-12-27 11:48:03,539][20381] Updated weights for policy 1, policy_version 887619 (0.0007) [2023-12-27 11:48:03,591][20381] Updated weights for policy 1, policy_version 887629 (0.0007) [2023-12-27 11:48:03,648][20381] Updated weights for policy 1, policy_version 887639 (0.0008) [2023-12-27 11:48:04,189][20347] Updated weights for policy 0, policy_version 888308 (0.0007) [2023-12-27 11:48:04,254][20347] Updated weights for policy 0, policy_version 888318 (0.0008) [2023-12-27 11:48:04,317][20347] Updated weights for policy 0, policy_version 888328 (0.0008) [2023-12-27 11:48:04,392][20381] Updated weights for policy 1, policy_version 887649 (0.0007) [2023-12-27 11:48:04,454][20381] Updated weights for policy 1, policy_version 887659 (0.0008) [2023-12-27 11:48:04,516][20381] Updated weights for policy 1, policy_version 887669 (0.0008) [2023-12-27 11:48:05,024][20347] Updated weights for policy 0, policy_version 888338 (0.0007) [2023-12-27 11:48:05,081][20347] Updated weights for policy 0, policy_version 888348 (0.0006) [2023-12-27 11:48:05,138][20347] Updated weights for policy 0, policy_version 888358 (0.0006) [2023-12-27 11:48:05,196][20347] Updated weights for policy 0, policy_version 888368 (0.0006) [2023-12-27 11:48:05,224][20381] Updated weights for policy 1, policy_version 887679 (0.0009) [2023-12-27 11:48:05,271][20381] Updated weights for policy 1, policy_version 887689 (0.0008) [2023-12-27 11:48:05,319][20381] Updated weights for policy 1, policy_version 887699 (0.0008) [2023-12-27 11:48:05,863][20347] Updated weights for policy 0, policy_version 888378 (0.0007) [2023-12-27 11:48:05,912][20347] Updated weights for policy 0, policy_version 888388 (0.0008) [2023-12-27 11:48:05,938][20381] Updated weights for policy 1, policy_version 887709 (0.0006) [2023-12-27 11:48:05,968][20347] Updated weights for policy 0, policy_version 888398 (0.0007) [2023-12-27 11:48:05,988][20381] Updated weights for policy 1, policy_version 887719 (0.0006) [2023-12-27 11:48:06,070][20381] Updated weights for policy 1, policy_version 887731 (0.0017) [2023-12-27 11:48:06,736][20347] Updated weights for policy 0, policy_version 888408 (0.0008) [2023-12-27 11:48:06,736][20291] KL-divergence is very high: 137.5264 [2023-12-27 11:48:06,782][20291] KL-divergence is very high: 158.4812 [2023-12-27 11:48:06,795][20347] Updated weights for policy 0, policy_version 888418 (0.0007) [2023-12-27 11:48:06,802][20381] Updated weights for policy 1, policy_version 887741 (0.0007) [2023-12-27 11:48:06,852][20347] Updated weights for policy 0, policy_version 888428 (0.0007) [2023-12-27 11:48:06,868][20381] Updated weights for policy 1, policy_version 887751 (0.0006) [2023-12-27 11:48:06,934][20381] Updated weights for policy 1, policy_version 887761 (0.0006) [2023-12-27 11:48:07,363][19170] Fps is (10 sec: 19659.8, 60 sec: 19387.6, 300 sec: 19855.2). Total num frames: 454762496. Throughput: 0: 9719.2, 1: 9674.9. Samples: 454733392. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:07,364][19170] Avg episode reward: [(0, '726.860'), (1, '1000.000')] [2023-12-27 11:48:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000887768_227295232.pth... [2023-12-27 11:48:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000888432_227467264.pth... [2023-12-27 11:48:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000887312_227180544.pth [2023-12-27 11:48:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000886648_227008512.pth [2023-12-27 11:48:07,481][20347] Updated weights for policy 0, policy_version 888438 (0.0006) [2023-12-27 11:48:07,523][20381] Updated weights for policy 1, policy_version 887771 (0.0008) [2023-12-27 11:48:07,543][20347] Updated weights for policy 0, policy_version 888448 (0.0007) [2023-12-27 11:48:07,573][20381] Updated weights for policy 1, policy_version 887781 (0.0011) [2023-12-27 11:48:07,598][20347] Updated weights for policy 0, policy_version 888458 (0.0006) [2023-12-27 11:48:07,631][20381] Updated weights for policy 1, policy_version 887791 (0.0010) [2023-12-27 11:48:08,275][20381] Updated weights for policy 1, policy_version 887801 (0.0008) [2023-12-27 11:48:08,325][20347] Updated weights for policy 0, policy_version 888468 (0.0007) [2023-12-27 11:48:08,337][20381] Updated weights for policy 1, policy_version 887811 (0.0009) [2023-12-27 11:48:08,389][20347] Updated weights for policy 0, policy_version 888478 (0.0008) [2023-12-27 11:48:08,401][20381] Updated weights for policy 1, policy_version 887821 (0.0008) [2023-12-27 11:48:08,453][20347] Updated weights for policy 0, policy_version 888488 (0.0009) [2023-12-27 11:48:08,466][20381] Updated weights for policy 1, policy_version 887831 (0.0009) [2023-12-27 11:48:09,130][20347] Updated weights for policy 0, policy_version 888498 (0.0006) [2023-12-27 11:48:09,190][20347] Updated weights for policy 0, policy_version 888508 (0.0007) [2023-12-27 11:48:09,223][20381] Updated weights for policy 1, policy_version 887841 (0.0009) [2023-12-27 11:48:09,256][20347] Updated weights for policy 0, policy_version 888518 (0.0008) [2023-12-27 11:48:09,288][20381] Updated weights for policy 1, policy_version 887851 (0.0009) [2023-12-27 11:48:09,319][20347] Updated weights for policy 0, policy_version 888528 (0.0007) [2023-12-27 11:48:09,353][20381] Updated weights for policy 1, policy_version 887861 (0.0011) [2023-12-27 11:48:10,114][20347] Updated weights for policy 0, policy_version 888538 (0.0008) [2023-12-27 11:48:10,158][20381] Updated weights for policy 1, policy_version 887871 (0.0009) [2023-12-27 11:48:10,178][20347] Updated weights for policy 0, policy_version 888548 (0.0008) [2023-12-27 11:48:10,222][20381] Updated weights for policy 1, policy_version 887881 (0.0009) [2023-12-27 11:48:10,242][20347] Updated weights for policy 0, policy_version 888558 (0.0008) [2023-12-27 11:48:10,283][20381] Updated weights for policy 1, policy_version 887891 (0.0009) [2023-12-27 11:48:10,991][20347] Updated weights for policy 0, policy_version 888568 (0.0007) [2023-12-27 11:48:11,015][20381] Updated weights for policy 1, policy_version 887901 (0.0009) [2023-12-27 11:48:11,067][20347] Updated weights for policy 0, policy_version 888578 (0.0007) [2023-12-27 11:48:11,087][20381] Updated weights for policy 1, policy_version 887911 (0.0010) [2023-12-27 11:48:11,126][20347] Updated weights for policy 0, policy_version 888588 (0.0008) [2023-12-27 11:48:11,154][20381] Updated weights for policy 1, policy_version 887921 (0.0011) [2023-12-27 11:48:11,847][20291] KL-divergence is very high: 101.2606 [2023-12-27 11:48:11,875][20347] Updated weights for policy 0, policy_version 888598 (0.0009) [2023-12-27 11:48:11,888][20291] KL-divergence is very high: 193.0044 [2023-12-27 11:48:11,901][20291] KL-divergence is very high: 179.4144 [2023-12-27 11:48:11,928][20381] Updated weights for policy 1, policy_version 887931 (0.0009) [2023-12-27 11:48:11,946][20347] Updated weights for policy 0, policy_version 888608 (0.0007) [2023-12-27 11:48:11,992][20381] Updated weights for policy 1, policy_version 887941 (0.0006) [2023-12-27 11:48:12,006][20347] Updated weights for policy 0, policy_version 888618 (0.0006) [2023-12-27 11:48:12,047][20381] Updated weights for policy 1, policy_version 887951 (0.0007) [2023-12-27 11:48:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19388.7, 300 sec: 19799.7). Total num frames: 454860800. Throughput: 0: 9697.1, 1: 9737.1. Samples: 454850144. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:12,363][19170] Avg episode reward: [(0, '746.740'), (1, '1000.000')] [2023-12-27 11:48:12,766][20347] Updated weights for policy 0, policy_version 888628 (0.0007) [2023-12-27 11:48:12,808][20381] Updated weights for policy 1, policy_version 887961 (0.0007) [2023-12-27 11:48:12,827][20347] Updated weights for policy 0, policy_version 888638 (0.0006) [2023-12-27 11:48:12,871][20381] Updated weights for policy 1, policy_version 887971 (0.0008) [2023-12-27 11:48:12,887][20347] Updated weights for policy 0, policy_version 888648 (0.0007) [2023-12-27 11:48:12,935][20381] Updated weights for policy 1, policy_version 887981 (0.0008) [2023-12-27 11:48:12,997][20381] Updated weights for policy 1, policy_version 887991 (0.0008) [2023-12-27 11:48:13,575][20347] Updated weights for policy 0, policy_version 888658 (0.0008) [2023-12-27 11:48:13,634][20347] Updated weights for policy 0, policy_version 888668 (0.0007) [2023-12-27 11:48:13,674][20291] KL-divergence is very high: 144.7533 [2023-12-27 11:48:13,683][20381] Updated weights for policy 1, policy_version 888001 (0.0008) [2023-12-27 11:48:13,694][20347] Updated weights for policy 0, policy_version 888678 (0.0007) [2023-12-27 11:48:13,723][20291] KL-divergence is very high: 112.3550 [2023-12-27 11:48:13,740][20381] Updated weights for policy 1, policy_version 888011 (0.0006) [2023-12-27 11:48:13,752][20347] Updated weights for policy 0, policy_version 888688 (0.0007) [2023-12-27 11:48:13,796][20381] Updated weights for policy 1, policy_version 888021 (0.0007) [2023-12-27 11:48:14,474][20347] Updated weights for policy 0, policy_version 888698 (0.0006) [2023-12-27 11:48:14,487][20381] Updated weights for policy 1, policy_version 888031 (0.0007) [2023-12-27 11:48:14,531][20347] Updated weights for policy 0, policy_version 888708 (0.0007) [2023-12-27 11:48:14,553][20381] Updated weights for policy 1, policy_version 888041 (0.0008) [2023-12-27 11:48:14,584][20347] Updated weights for policy 0, policy_version 888718 (0.0007) [2023-12-27 11:48:14,604][20381] Updated weights for policy 1, policy_version 888051 (0.0006) [2023-12-27 11:48:15,332][20347] Updated weights for policy 0, policy_version 888728 (0.0008) [2023-12-27 11:48:15,348][20381] Updated weights for policy 1, policy_version 888061 (0.0007) [2023-12-27 11:48:15,390][20347] Updated weights for policy 0, policy_version 888738 (0.0006) [2023-12-27 11:48:15,404][20381] Updated weights for policy 1, policy_version 888071 (0.0008) [2023-12-27 11:48:15,453][20347] Updated weights for policy 0, policy_version 888748 (0.0007) [2023-12-27 11:48:15,457][20381] Updated weights for policy 1, policy_version 888081 (0.0008) [2023-12-27 11:48:16,137][20381] Updated weights for policy 1, policy_version 888091 (0.0008) [2023-12-27 11:48:16,153][20347] Updated weights for policy 0, policy_version 888758 (0.0006) [2023-12-27 11:48:16,192][20381] Updated weights for policy 1, policy_version 888101 (0.0009) [2023-12-27 11:48:16,211][20347] Updated weights for policy 0, policy_version 888768 (0.0006) [2023-12-27 11:48:16,251][20381] Updated weights for policy 1, policy_version 888111 (0.0007) [2023-12-27 11:48:16,277][20347] Updated weights for policy 0, policy_version 888778 (0.0007) [2023-12-27 11:48:16,875][20381] Updated weights for policy 1, policy_version 888121 (0.0006) [2023-12-27 11:48:16,936][20381] Updated weights for policy 1, policy_version 888131 (0.0008) [2023-12-27 11:48:16,964][20347] Updated weights for policy 0, policy_version 888788 (0.0006) [2023-12-27 11:48:17,000][20381] Updated weights for policy 1, policy_version 888141 (0.0008) [2023-12-27 11:48:17,024][20347] Updated weights for policy 0, policy_version 888798 (0.0009) [2023-12-27 11:48:17,064][20381] Updated weights for policy 1, policy_version 888151 (0.0008) [2023-12-27 11:48:17,087][20347] Updated weights for policy 0, policy_version 888808 (0.0009) [2023-12-27 11:48:17,362][19170] Fps is (10 sec: 19661.6, 60 sec: 19387.7, 300 sec: 19799.6). Total num frames: 454959104. Throughput: 0: 9638.9, 1: 9753.1. Samples: 454966740. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:17,363][19170] Avg episode reward: [(0, '853.800'), (1, '1000.000')] [2023-12-27 11:48:17,746][20381] Updated weights for policy 1, policy_version 888161 (0.0006) [2023-12-27 11:48:17,794][20347] Updated weights for policy 0, policy_version 888818 (0.0008) [2023-12-27 11:48:17,806][20381] Updated weights for policy 1, policy_version 888171 (0.0005) [2023-12-27 11:48:17,859][20347] Updated weights for policy 0, policy_version 888828 (0.0007) [2023-12-27 11:48:17,864][20381] Updated weights for policy 1, policy_version 888181 (0.0006) [2023-12-27 11:48:17,912][20347] Updated weights for policy 0, policy_version 888838 (0.0007) [2023-12-27 11:48:17,966][20347] Updated weights for policy 0, policy_version 888848 (0.0005) [2023-12-27 11:48:18,516][20381] Updated weights for policy 1, policy_version 888191 (0.0006) [2023-12-27 11:48:18,574][20381] Updated weights for policy 1, policy_version 888201 (0.0006) [2023-12-27 11:48:18,626][20347] Updated weights for policy 0, policy_version 888858 (0.0006) [2023-12-27 11:48:18,631][20381] Updated weights for policy 1, policy_version 888211 (0.0007) [2023-12-27 11:48:18,676][20347] Updated weights for policy 0, policy_version 888868 (0.0006) [2023-12-27 11:48:18,734][20347] Updated weights for policy 0, policy_version 888878 (0.0007) [2023-12-27 11:48:19,304][20381] Updated weights for policy 1, policy_version 888221 (0.0007) [2023-12-27 11:48:19,375][20381] Updated weights for policy 1, policy_version 888231 (0.0008) [2023-12-27 11:48:19,437][20381] Updated weights for policy 1, policy_version 888241 (0.0008) [2023-12-27 11:48:19,504][20347] Updated weights for policy 0, policy_version 888888 (0.0007) [2023-12-27 11:48:19,571][20347] Updated weights for policy 0, policy_version 888898 (0.0008) [2023-12-27 11:48:19,634][20347] Updated weights for policy 0, policy_version 888908 (0.0008) [2023-12-27 11:48:20,057][20381] Updated weights for policy 1, policy_version 888251 (0.0007) [2023-12-27 11:48:20,114][20381] Updated weights for policy 1, policy_version 888261 (0.0008) [2023-12-27 11:48:20,177][20381] Updated weights for policy 1, policy_version 888271 (0.0008) [2023-12-27 11:48:20,370][20347] Updated weights for policy 0, policy_version 888918 (0.0008) [2023-12-27 11:48:20,432][20347] Updated weights for policy 0, policy_version 888928 (0.0008) [2023-12-27 11:48:20,497][20347] Updated weights for policy 0, policy_version 888938 (0.0007) [2023-12-27 11:48:20,919][20381] Updated weights for policy 1, policy_version 888281 (0.0008) [2023-12-27 11:48:20,969][20381] Updated weights for policy 1, policy_version 888291 (0.0008) [2023-12-27 11:48:21,024][20381] Updated weights for policy 1, policy_version 888301 (0.0008) [2023-12-27 11:48:21,094][20381] Updated weights for policy 1, policy_version 888311 (0.0008) [2023-12-27 11:48:21,195][20347] Updated weights for policy 0, policy_version 888948 (0.0008) [2023-12-27 11:48:21,260][20347] Updated weights for policy 0, policy_version 888958 (0.0010) [2023-12-27 11:48:21,321][20347] Updated weights for policy 0, policy_version 888968 (0.0011) [2023-12-27 11:48:21,877][20381] Updated weights for policy 1, policy_version 888321 (0.0007) [2023-12-27 11:48:21,937][20381] Updated weights for policy 1, policy_version 888331 (0.0007) [2023-12-27 11:48:21,995][20381] Updated weights for policy 1, policy_version 888341 (0.0008) [2023-12-27 11:48:22,068][20347] Updated weights for policy 0, policy_version 888978 (0.0011) [2023-12-27 11:48:22,128][20347] Updated weights for policy 0, policy_version 888988 (0.0011) [2023-12-27 11:48:22,178][20347] Updated weights for policy 0, policy_version 888998 (0.0010) [2023-12-27 11:48:22,228][20347] Updated weights for policy 0, policy_version 889008 (0.0009) [2023-12-27 11:48:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19524.3, 300 sec: 19744.1). Total num frames: 455057408. Throughput: 0: 9613.7, 1: 9749.8. Samples: 455026692. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:22,363][19170] Avg episode reward: [(0, '912.190'), (1, '1000.000')] [2023-12-27 11:48:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000888344_227442688.pth... [2023-12-27 11:48:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000887192_227147776.pth [2023-12-27 11:48:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000889008_227614720.pth... [2023-12-27 11:48:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000887888_227328000.pth [2023-12-27 11:48:22,737][20381] Updated weights for policy 1, policy_version 888351 (0.0008) [2023-12-27 11:48:22,795][20381] Updated weights for policy 1, policy_version 888361 (0.0008) [2023-12-27 11:48:22,858][20381] Updated weights for policy 1, policy_version 888371 (0.0008) [2023-12-27 11:48:22,954][20291] KL-divergence is very high: 315.9630 [2023-12-27 11:48:22,963][20347] Updated weights for policy 0, policy_version 889018 (0.0007) [2023-12-27 11:48:23,023][20347] Updated weights for policy 0, policy_version 889028 (0.0007) [2023-12-27 11:48:23,050][20291] KL-divergence is very high: 147.3726 [2023-12-27 11:48:23,085][20347] Updated weights for policy 0, policy_version 889038 (0.0006) [2023-12-27 11:48:23,518][20381] Updated weights for policy 1, policy_version 888381 (0.0009) [2023-12-27 11:48:23,572][20381] Updated weights for policy 1, policy_version 888391 (0.0007) [2023-12-27 11:48:23,620][20381] Updated weights for policy 1, policy_version 888401 (0.0007) [2023-12-27 11:48:23,705][20347] Updated weights for policy 0, policy_version 889048 (0.0010) [2023-12-27 11:48:23,750][20347] Updated weights for policy 0, policy_version 889058 (0.0010) [2023-12-27 11:48:23,793][20347] Updated weights for policy 0, policy_version 889068 (0.0010) [2023-12-27 11:48:24,221][20381] Updated weights for policy 1, policy_version 888411 (0.0007) [2023-12-27 11:48:24,274][20381] Updated weights for policy 1, policy_version 888421 (0.0006) [2023-12-27 11:48:24,326][20381] Updated weights for policy 1, policy_version 888431 (0.0006) [2023-12-27 11:48:24,438][20291] KL-divergence is very high: 255.4405 [2023-12-27 11:48:24,465][20347] Updated weights for policy 0, policy_version 889078 (0.0007) [2023-12-27 11:48:24,479][20291] KL-divergence is very high: 204.0953 [2023-12-27 11:48:24,510][20347] Updated weights for policy 0, policy_version 889088 (0.0005) [2023-12-27 11:48:24,556][20291] KL-divergence is very high: 126.9785 [2023-12-27 11:48:24,560][20347] Updated weights for policy 0, policy_version 889098 (0.0005) [2023-12-27 11:48:24,897][20381] Updated weights for policy 1, policy_version 888441 (0.0010) [2023-12-27 11:48:24,952][20381] Updated weights for policy 1, policy_version 888451 (0.0006) [2023-12-27 11:48:25,004][20381] Updated weights for policy 1, policy_version 888461 (0.0007) [2023-12-27 11:48:25,058][20381] Updated weights for policy 1, policy_version 888471 (0.0005) [2023-12-27 11:48:25,200][20347] Updated weights for policy 0, policy_version 889108 (0.0008) [2023-12-27 11:48:25,258][20347] Updated weights for policy 0, policy_version 889118 (0.0010) [2023-12-27 11:48:25,312][20347] Updated weights for policy 0, policy_version 889128 (0.0010) [2023-12-27 11:48:25,334][20291] KL-divergence is very high: 107.4203 [2023-12-27 11:48:25,641][20381] Updated weights for policy 1, policy_version 888481 (0.0006) [2023-12-27 11:48:25,689][20381] Updated weights for policy 1, policy_version 888491 (0.0007) [2023-12-27 11:48:25,743][20381] Updated weights for policy 1, policy_version 888501 (0.0007) [2023-12-27 11:48:25,945][20347] Updated weights for policy 0, policy_version 889138 (0.0007) [2023-12-27 11:48:26,004][20347] Updated weights for policy 0, policy_version 889148 (0.0007) [2023-12-27 11:48:26,050][20291] KL-divergence is very high: 142.9576 [2023-12-27 11:48:26,060][20347] Updated weights for policy 0, policy_version 889158 (0.0006) [2023-12-27 11:48:26,111][20347] Updated weights for policy 0, policy_version 889168 (0.0006) [2023-12-27 11:48:26,387][20381] Updated weights for policy 1, policy_version 888511 (0.0007) [2023-12-27 11:48:26,453][20381] Updated weights for policy 1, policy_version 888521 (0.0007) [2023-12-27 11:48:26,508][20381] Updated weights for policy 1, policy_version 888531 (0.0007) [2023-12-27 11:48:26,773][20347] Updated weights for policy 0, policy_version 889178 (0.0010) [2023-12-27 11:48:26,833][20347] Updated weights for policy 0, policy_version 889188 (0.0005) [2023-12-27 11:48:26,891][20347] Updated weights for policy 0, policy_version 889198 (0.0006) [2023-12-27 11:48:27,091][20381] Updated weights for policy 1, policy_version 888541 (0.0011) [2023-12-27 11:48:27,139][20381] Updated weights for policy 1, policy_version 888551 (0.0010) [2023-12-27 11:48:27,183][20381] Updated weights for policy 1, policy_version 888561 (0.0010) [2023-12-27 11:48:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 19660.8, 300 sec: 19771.9). Total num frames: 455163904. Throughput: 0: 9663.4, 1: 9841.6. Samples: 455150480. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:27,363][19170] Avg episode reward: [(0, '824.460'), (1, '1000.000')] [2023-12-27 11:48:27,493][20347] Updated weights for policy 0, policy_version 889208 (0.0005) [2023-12-27 11:48:27,539][20347] Updated weights for policy 0, policy_version 889218 (0.0005) [2023-12-27 11:48:27,585][20347] Updated weights for policy 0, policy_version 889228 (0.0005) [2023-12-27 11:48:27,847][20381] Updated weights for policy 1, policy_version 888571 (0.0009) [2023-12-27 11:48:27,898][20381] Updated weights for policy 1, policy_version 888581 (0.0008) [2023-12-27 11:48:27,942][20381] Updated weights for policy 1, policy_version 888591 (0.0010) [2023-12-27 11:48:28,132][20347] Updated weights for policy 0, policy_version 889238 (0.0005) [2023-12-27 11:48:28,184][20347] Updated weights for policy 0, policy_version 889248 (0.0005) [2023-12-27 11:48:28,238][20347] Updated weights for policy 0, policy_version 889258 (0.0005) [2023-12-27 11:48:28,537][20381] Updated weights for policy 1, policy_version 888601 (0.0009) [2023-12-27 11:48:28,596][20381] Updated weights for policy 1, policy_version 888611 (0.0005) [2023-12-27 11:48:28,652][20381] Updated weights for policy 1, policy_version 888621 (0.0006) [2023-12-27 11:48:28,711][20381] Updated weights for policy 1, policy_version 888631 (0.0011) [2023-12-27 11:48:28,842][20347] Updated weights for policy 0, policy_version 889268 (0.0006) [2023-12-27 11:48:28,903][20347] Updated weights for policy 0, policy_version 889278 (0.0007) [2023-12-27 11:48:28,951][20291] KL-divergence is very high: 169.0373 [2023-12-27 11:48:28,957][20291] KL-divergence is very high: 631.5233 [2023-12-27 11:48:28,962][20347] Updated weights for policy 0, policy_version 889288 (0.0007) [2023-12-27 11:48:29,001][20291] KL-divergence is very high: 105.5314 [2023-12-27 11:48:29,007][20291] KL-divergence is very high: 367.7544 [2023-12-27 11:48:29,387][20381] Updated weights for policy 1, policy_version 888641 (0.0008) [2023-12-27 11:48:29,443][20381] Updated weights for policy 1, policy_version 888651 (0.0008) [2023-12-27 11:48:29,504][20381] Updated weights for policy 1, policy_version 888661 (0.0008) [2023-12-27 11:48:29,636][20347] Updated weights for policy 0, policy_version 889298 (0.0007) [2023-12-27 11:48:29,690][20347] Updated weights for policy 0, policy_version 889308 (0.0005) [2023-12-27 11:48:29,741][20347] Updated weights for policy 0, policy_version 889318 (0.0005) [2023-12-27 11:48:29,807][20347] Updated weights for policy 0, policy_version 889328 (0.0005) [2023-12-27 11:48:30,203][20381] Updated weights for policy 1, policy_version 888671 (0.0008) [2023-12-27 11:48:30,256][20381] Updated weights for policy 1, policy_version 888681 (0.0006) [2023-12-27 11:48:30,313][20381] Updated weights for policy 1, policy_version 888691 (0.0006) [2023-12-27 11:48:30,483][20291] KL-divergence is very high: 197.0308 [2023-12-27 11:48:30,503][20347] Updated weights for policy 0, policy_version 889338 (0.0008) [2023-12-27 11:48:30,549][20347] Updated weights for policy 0, policy_version 889348 (0.0007) [2023-12-27 11:48:30,598][20347] Updated weights for policy 0, policy_version 889358 (0.0008) [2023-12-27 11:48:30,976][20381] Updated weights for policy 1, policy_version 888701 (0.0007) [2023-12-27 11:48:31,025][20381] Updated weights for policy 1, policy_version 888711 (0.0007) [2023-12-27 11:48:31,090][20381] Updated weights for policy 1, policy_version 888721 (0.0009) [2023-12-27 11:48:31,361][20347] Updated weights for policy 0, policy_version 889368 (0.0008) [2023-12-27 11:48:31,427][20347] Updated weights for policy 0, policy_version 889378 (0.0007) [2023-12-27 11:48:31,493][20347] Updated weights for policy 0, policy_version 889388 (0.0007) [2023-12-27 11:48:31,812][20381] Updated weights for policy 1, policy_version 888731 (0.0008) [2023-12-27 11:48:31,868][20381] Updated weights for policy 1, policy_version 888741 (0.0008) [2023-12-27 11:48:31,929][20381] Updated weights for policy 1, policy_version 888751 (0.0008) [2023-12-27 11:48:32,177][20347] Updated weights for policy 0, policy_version 889398 (0.0007) [2023-12-27 11:48:32,238][20347] Updated weights for policy 0, policy_version 889408 (0.0007) [2023-12-27 11:48:32,299][20347] Updated weights for policy 0, policy_version 889418 (0.0006) [2023-12-27 11:48:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 19797.4, 300 sec: 19771.9). Total num frames: 455270400. Throughput: 0: 9810.2, 1: 9930.1. Samples: 455277228. Policy #0 lag: (min: 31.0, avg: 47.1, max: 63.0) [2023-12-27 11:48:32,363][19170] Avg episode reward: [(0, '893.000'), (1, '1000.000')] [2023-12-27 11:48:32,613][20381] Updated weights for policy 1, policy_version 888761 (0.0007) [2023-12-27 11:48:32,674][20381] Updated weights for policy 1, policy_version 888771 (0.0008) [2023-12-27 11:48:32,737][20381] Updated weights for policy 1, policy_version 888781 (0.0009) [2023-12-27 11:48:32,799][20381] Updated weights for policy 1, policy_version 888791 (0.0011) [2023-12-27 11:48:32,905][20347] Updated weights for policy 0, policy_version 889428 (0.0008) [2023-12-27 11:48:32,970][20347] Updated weights for policy 0, policy_version 889438 (0.0008) [2023-12-27 11:48:33,030][20347] Updated weights for policy 0, policy_version 889448 (0.0006) [2023-12-27 11:48:33,444][20381] Updated weights for policy 1, policy_version 888801 (0.0010) [2023-12-27 11:48:33,496][20381] Updated weights for policy 1, policy_version 888811 (0.0010) [2023-12-27 11:48:33,544][20381] Updated weights for policy 1, policy_version 888821 (0.0010) [2023-12-27 11:48:33,666][20347] Updated weights for policy 0, policy_version 889458 (0.0005) [2023-12-27 11:48:33,718][20347] Updated weights for policy 0, policy_version 889468 (0.0007) [2023-12-27 11:48:33,767][20347] Updated weights for policy 0, policy_version 889478 (0.0007) [2023-12-27 11:48:33,822][20347] Updated weights for policy 0, policy_version 889488 (0.0007) [2023-12-27 11:48:34,228][20381] Updated weights for policy 1, policy_version 888831 (0.0008) [2023-12-27 11:48:34,291][20381] Updated weights for policy 1, policy_version 888841 (0.0009) [2023-12-27 11:48:34,350][20381] Updated weights for policy 1, policy_version 888851 (0.0006) [2023-12-27 11:48:34,483][20347] Updated weights for policy 0, policy_version 889498 (0.0010) [2023-12-27 11:48:34,545][20347] Updated weights for policy 0, policy_version 889508 (0.0010) [2023-12-27 11:48:34,602][20347] Updated weights for policy 0, policy_version 889518 (0.0010) [2023-12-27 11:48:35,032][20381] Updated weights for policy 1, policy_version 888861 (0.0008) [2023-12-27 11:48:35,093][20381] Updated weights for policy 1, policy_version 888871 (0.0011) [2023-12-27 11:48:35,142][20381] Updated weights for policy 1, policy_version 888881 (0.0010) [2023-12-27 11:48:35,262][20347] Updated weights for policy 0, policy_version 889528 (0.0010) [2023-12-27 11:48:35,316][20347] Updated weights for policy 0, policy_version 889538 (0.0010) [2023-12-27 11:48:35,367][20347] Updated weights for policy 0, policy_version 889548 (0.0010) [2023-12-27 11:48:35,806][20381] Updated weights for policy 1, policy_version 888891 (0.0011) [2023-12-27 11:48:35,863][20381] Updated weights for policy 1, policy_version 888901 (0.0010) [2023-12-27 11:48:35,921][20381] Updated weights for policy 1, policy_version 888911 (0.0008) [2023-12-27 11:48:36,035][20347] Updated weights for policy 0, policy_version 889558 (0.0010) [2023-12-27 11:48:36,082][20347] Updated weights for policy 0, policy_version 889568 (0.0010) [2023-12-27 11:48:36,143][20347] Updated weights for policy 0, policy_version 889578 (0.0006) [2023-12-27 11:48:36,590][20381] Updated weights for policy 1, policy_version 888921 (0.0006) [2023-12-27 11:48:36,650][20381] Updated weights for policy 1, policy_version 888931 (0.0011) [2023-12-27 11:48:36,709][20381] Updated weights for policy 1, policy_version 888941 (0.0011) [2023-12-27 11:48:36,764][20381] Updated weights for policy 1, policy_version 888951 (0.0011) [2023-12-27 11:48:36,831][20347] Updated weights for policy 0, policy_version 889588 (0.0006) [2023-12-27 11:48:36,877][20347] Updated weights for policy 0, policy_version 889598 (0.0007) [2023-12-27 11:48:36,927][20347] Updated weights for policy 0, policy_version 889608 (0.0008) [2023-12-27 11:48:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 19660.8, 300 sec: 19744.1). Total num frames: 455368704. Throughput: 0: 9885.0, 1: 9984.4. Samples: 455339124. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:48:37,363][19170] Avg episode reward: [(0, '971.100'), (1, '1000.000')] [2023-12-27 11:48:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000888952_227598336.pth... [2023-12-27 11:48:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000889616_227770368.pth... [2023-12-27 11:48:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000888432_227467264.pth [2023-12-27 11:48:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000887768_227295232.pth [2023-12-27 11:48:37,454][20381] Updated weights for policy 1, policy_version 888961 (0.0011) [2023-12-27 11:48:37,510][20381] Updated weights for policy 1, policy_version 888971 (0.0010) [2023-12-27 11:48:37,562][20381] Updated weights for policy 1, policy_version 888981 (0.0011) [2023-12-27 11:48:37,633][20347] Updated weights for policy 0, policy_version 889618 (0.0007) [2023-12-27 11:48:37,679][20347] Updated weights for policy 0, policy_version 889628 (0.0010) [2023-12-27 11:48:37,730][20347] Updated weights for policy 0, policy_version 889638 (0.0010) [2023-12-27 11:48:37,790][20347] Updated weights for policy 0, policy_version 889648 (0.0010) [2023-12-27 11:48:38,234][20381] Updated weights for policy 1, policy_version 888991 (0.0009) [2023-12-27 11:48:38,289][20381] Updated weights for policy 1, policy_version 889001 (0.0007) [2023-12-27 11:48:38,348][20381] Updated weights for policy 1, policy_version 889011 (0.0007) [2023-12-27 11:48:38,450][20347] Updated weights for policy 0, policy_version 889658 (0.0008) [2023-12-27 11:48:38,500][20347] Updated weights for policy 0, policy_version 889668 (0.0007) [2023-12-27 11:48:38,553][20347] Updated weights for policy 0, policy_version 889678 (0.0008) [2023-12-27 11:48:38,980][20381] Updated weights for policy 1, policy_version 889021 (0.0007) [2023-12-27 11:48:39,035][20381] Updated weights for policy 1, policy_version 889031 (0.0007) [2023-12-27 11:48:39,089][20381] Updated weights for policy 1, policy_version 889041 (0.0007) [2023-12-27 11:48:39,244][20347] Updated weights for policy 0, policy_version 889688 (0.0008) [2023-12-27 11:48:39,304][20347] Updated weights for policy 0, policy_version 889698 (0.0008) [2023-12-27 11:48:39,363][20347] Updated weights for policy 0, policy_version 889708 (0.0008) [2023-12-27 11:48:39,816][20381] Updated weights for policy 1, policy_version 889051 (0.0008) [2023-12-27 11:48:39,887][20381] Updated weights for policy 1, policy_version 889061 (0.0008) [2023-12-27 11:48:39,955][20381] Updated weights for policy 1, policy_version 889071 (0.0007) [2023-12-27 11:48:40,115][20347] Updated weights for policy 0, policy_version 889718 (0.0009) [2023-12-27 11:48:40,172][20347] Updated weights for policy 0, policy_version 889728 (0.0008) [2023-12-27 11:48:40,230][20347] Updated weights for policy 0, policy_version 889738 (0.0008) [2023-12-27 11:48:40,579][20381] Updated weights for policy 1, policy_version 889081 (0.0006) [2023-12-27 11:48:40,629][20381] Updated weights for policy 1, policy_version 889091 (0.0006) [2023-12-27 11:48:40,685][20381] Updated weights for policy 1, policy_version 889101 (0.0006) [2023-12-27 11:48:40,744][20381] Updated weights for policy 1, policy_version 889111 (0.0006) [2023-12-27 11:48:40,897][20347] Updated weights for policy 0, policy_version 889748 (0.0007) [2023-12-27 11:48:40,950][20347] Updated weights for policy 0, policy_version 889758 (0.0005) [2023-12-27 11:48:41,001][20347] Updated weights for policy 0, policy_version 889768 (0.0005) [2023-12-27 11:48:41,465][20381] Updated weights for policy 1, policy_version 889121 (0.0008) [2023-12-27 11:48:41,530][20381] Updated weights for policy 1, policy_version 889131 (0.0008) [2023-12-27 11:48:41,592][20381] Updated weights for policy 1, policy_version 889141 (0.0008) [2023-12-27 11:48:41,704][20347] Updated weights for policy 0, policy_version 889778 (0.0007) [2023-12-27 11:48:41,766][20347] Updated weights for policy 0, policy_version 889788 (0.0008) [2023-12-27 11:48:41,831][20347] Updated weights for policy 0, policy_version 889798 (0.0007) [2023-12-27 11:48:41,888][20347] Updated weights for policy 0, policy_version 889808 (0.0009) [2023-12-27 11:48:42,313][20381] Updated weights for policy 1, policy_version 889151 (0.0008) [2023-12-27 11:48:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 19744.1). Total num frames: 455467008. Throughput: 0: 9991.3, 1: 10081.0. Samples: 455462272. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:48:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:48:42,380][20381] Updated weights for policy 1, policy_version 889161 (0.0008) [2023-12-27 11:48:42,438][20381] Updated weights for policy 1, policy_version 889171 (0.0007) [2023-12-27 11:48:42,630][20347] Updated weights for policy 0, policy_version 889818 (0.0010) [2023-12-27 11:48:42,695][20347] Updated weights for policy 0, policy_version 889828 (0.0010) [2023-12-27 11:48:42,750][20347] Updated weights for policy 0, policy_version 889838 (0.0010) [2023-12-27 11:48:43,108][20381] Updated weights for policy 1, policy_version 889181 (0.0008) [2023-12-27 11:48:43,168][20381] Updated weights for policy 1, policy_version 889191 (0.0009) [2023-12-27 11:48:43,227][20381] Updated weights for policy 1, policy_version 889201 (0.0008) [2023-12-27 11:48:43,414][20347] Updated weights for policy 0, policy_version 889848 (0.0010) [2023-12-27 11:48:43,466][20347] Updated weights for policy 0, policy_version 889858 (0.0010) [2023-12-27 11:48:43,523][20347] Updated weights for policy 0, policy_version 889868 (0.0010) [2023-12-27 11:48:43,870][20381] Updated weights for policy 1, policy_version 889211 (0.0007) [2023-12-27 11:48:43,933][20381] Updated weights for policy 1, policy_version 889221 (0.0005) [2023-12-27 11:48:43,990][20381] Updated weights for policy 1, policy_version 889231 (0.0006) [2023-12-27 11:48:44,094][20347] Updated weights for policy 0, policy_version 889878 (0.0008) [2023-12-27 11:48:44,140][20347] Updated weights for policy 0, policy_version 889888 (0.0008) [2023-12-27 11:48:44,202][20347] Updated weights for policy 0, policy_version 889898 (0.0006) [2023-12-27 11:48:44,648][20381] Updated weights for policy 1, policy_version 889241 (0.0007) [2023-12-27 11:48:44,694][20381] Updated weights for policy 1, policy_version 889251 (0.0008) [2023-12-27 11:48:44,742][20381] Updated weights for policy 1, policy_version 889261 (0.0007) [2023-12-27 11:48:44,759][20347] Updated weights for policy 0, policy_version 889908 (0.0007) [2023-12-27 11:48:44,809][20381] Updated weights for policy 1, policy_version 889271 (0.0006) [2023-12-27 11:48:44,822][20347] Updated weights for policy 0, policy_version 889918 (0.0008) [2023-12-27 11:48:44,875][20347] Updated weights for policy 0, policy_version 889928 (0.0007) [2023-12-27 11:48:45,503][20381] Updated weights for policy 1, policy_version 889281 (0.0008) [2023-12-27 11:48:45,563][20381] Updated weights for policy 1, policy_version 889291 (0.0008) [2023-12-27 11:48:45,609][20347] Updated weights for policy 0, policy_version 889938 (0.0007) [2023-12-27 11:48:45,616][20381] Updated weights for policy 1, policy_version 889301 (0.0008) [2023-12-27 11:48:45,652][20347] Updated weights for policy 0, policy_version 889948 (0.0006) [2023-12-27 11:48:45,697][20347] Updated weights for policy 0, policy_version 889958 (0.0007) [2023-12-27 11:48:45,741][20347] Updated weights for policy 0, policy_version 889968 (0.0007) [2023-12-27 11:48:46,258][20381] Updated weights for policy 1, policy_version 889311 (0.0011) [2023-12-27 11:48:46,299][20381] Updated weights for policy 1, policy_version 889321 (0.0010) [2023-12-27 11:48:46,352][20381] Updated weights for policy 1, policy_version 889331 (0.0011) [2023-12-27 11:48:46,383][20347] Updated weights for policy 0, policy_version 889978 (0.0005) [2023-12-27 11:48:46,428][20347] Updated weights for policy 0, policy_version 889988 (0.0005) [2023-12-27 11:48:46,482][20347] Updated weights for policy 0, policy_version 889998 (0.0007) [2023-12-27 11:48:47,008][20381] Updated weights for policy 1, policy_version 889341 (0.0008) [2023-12-27 11:48:47,059][20381] Updated weights for policy 1, policy_version 889351 (0.0005) [2023-12-27 11:48:47,107][20381] Updated weights for policy 1, policy_version 889361 (0.0005) [2023-12-27 11:48:47,117][20347] Updated weights for policy 0, policy_version 890008 (0.0009) [2023-12-27 11:48:47,177][20347] Updated weights for policy 0, policy_version 890018 (0.0007) [2023-12-27 11:48:47,229][20347] Updated weights for policy 0, policy_version 890028 (0.0010) [2023-12-27 11:48:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20070.4, 300 sec: 19799.6). Total num frames: 455581696. Throughput: 0: 10095.8, 1: 10166.2. Samples: 455587144. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:48:47,363][19170] Avg episode reward: [(0, '998.820'), (1, '1000.000')] [2023-12-27 11:48:47,722][20381] Updated weights for policy 1, policy_version 889371 (0.0007) [2023-12-27 11:48:47,773][20381] Updated weights for policy 1, policy_version 889381 (0.0007) [2023-12-27 11:48:47,824][20381] Updated weights for policy 1, policy_version 889391 (0.0010) [2023-12-27 11:48:47,897][20347] Updated weights for policy 0, policy_version 890038 (0.0009) [2023-12-27 11:48:47,961][20347] Updated weights for policy 0, policy_version 890048 (0.0010) [2023-12-27 11:48:48,016][20347] Updated weights for policy 0, policy_version 890058 (0.0010) [2023-12-27 11:48:48,535][20381] Updated weights for policy 1, policy_version 889401 (0.0011) [2023-12-27 11:48:48,584][20381] Updated weights for policy 1, policy_version 889411 (0.0007) [2023-12-27 11:48:48,635][20381] Updated weights for policy 1, policy_version 889421 (0.0009) [2023-12-27 11:48:48,669][20347] Updated weights for policy 0, policy_version 890068 (0.0010) [2023-12-27 11:48:48,695][20381] Updated weights for policy 1, policy_version 889431 (0.0010) [2023-12-27 11:48:48,729][20347] Updated weights for policy 0, policy_version 890078 (0.0011) [2023-12-27 11:48:48,785][20347] Updated weights for policy 0, policy_version 890088 (0.0010) [2023-12-27 11:48:49,447][20381] Updated weights for policy 1, policy_version 889441 (0.0009) [2023-12-27 11:48:49,508][20381] Updated weights for policy 1, policy_version 889451 (0.0007) [2023-12-27 11:48:49,533][20347] Updated weights for policy 0, policy_version 890098 (0.0010) [2023-12-27 11:48:49,568][20381] Updated weights for policy 1, policy_version 889461 (0.0008) [2023-12-27 11:48:49,586][20347] Updated weights for policy 0, policy_version 890108 (0.0007) [2023-12-27 11:48:49,644][20347] Updated weights for policy 0, policy_version 890118 (0.0008) [2023-12-27 11:48:49,698][20347] Updated weights for policy 0, policy_version 890128 (0.0007) [2023-12-27 11:48:50,294][20381] Updated weights for policy 1, policy_version 889471 (0.0010) [2023-12-27 11:48:50,350][20381] Updated weights for policy 1, policy_version 889481 (0.0010) [2023-12-27 11:48:50,410][20381] Updated weights for policy 1, policy_version 889491 (0.0009) [2023-12-27 11:48:50,415][20347] Updated weights for policy 0, policy_version 890138 (0.0006) [2023-12-27 11:48:50,477][20347] Updated weights for policy 0, policy_version 890148 (0.0008) [2023-12-27 11:48:50,529][20347] Updated weights for policy 0, policy_version 890158 (0.0007) [2023-12-27 11:48:51,116][20381] Updated weights for policy 1, policy_version 889501 (0.0011) [2023-12-27 11:48:51,176][20381] Updated weights for policy 1, policy_version 889511 (0.0007) [2023-12-27 11:48:51,227][20381] Updated weights for policy 1, policy_version 889521 (0.0009) [2023-12-27 11:48:51,247][20347] Updated weights for policy 0, policy_version 890168 (0.0007) [2023-12-27 11:48:51,310][20347] Updated weights for policy 0, policy_version 890178 (0.0008) [2023-12-27 11:48:51,380][20347] Updated weights for policy 0, policy_version 890188 (0.0008) [2023-12-27 11:48:51,878][20381] Updated weights for policy 1, policy_version 889531 (0.0008) [2023-12-27 11:48:51,927][20381] Updated weights for policy 1, policy_version 889541 (0.0007) [2023-12-27 11:48:51,987][20381] Updated weights for policy 1, policy_version 889551 (0.0008) [2023-12-27 11:48:52,056][20347] Updated weights for policy 0, policy_version 890198 (0.0006) [2023-12-27 11:48:52,116][20347] Updated weights for policy 0, policy_version 890208 (0.0006) [2023-12-27 11:48:52,175][20347] Updated weights for policy 0, policy_version 890218 (0.0007) [2023-12-27 11:48:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20212.2, 300 sec: 19771.9). Total num frames: 455680000. Throughput: 0: 10116.0, 1: 10183.1. Samples: 455646844. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:48:52,363][19170] Avg episode reward: [(0, '998.820'), (1, '1000.000')] [2023-12-27 11:48:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000889560_227753984.pth... [2023-12-27 11:48:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000890224_227926016.pth... [2023-12-27 11:48:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000888344_227442688.pth [2023-12-27 11:48:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000889008_227614720.pth [2023-12-27 11:48:52,761][20381] Updated weights for policy 1, policy_version 889561 (0.0007) [2023-12-27 11:48:52,811][20381] Updated weights for policy 1, policy_version 889571 (0.0007) [2023-12-27 11:48:52,851][20347] Updated weights for policy 0, policy_version 890228 (0.0006) [2023-12-27 11:48:52,872][20381] Updated weights for policy 1, policy_version 889581 (0.0007) [2023-12-27 11:48:52,903][20347] Updated weights for policy 0, policy_version 890238 (0.0007) [2023-12-27 11:48:52,920][20381] Updated weights for policy 1, policy_version 889591 (0.0006) [2023-12-27 11:48:52,954][20347] Updated weights for policy 0, policy_version 890248 (0.0006) [2023-12-27 11:48:53,578][20381] Updated weights for policy 1, policy_version 889601 (0.0007) [2023-12-27 11:48:53,638][20381] Updated weights for policy 1, policy_version 889611 (0.0009) [2023-12-27 11:48:53,668][20347] Updated weights for policy 0, policy_version 890258 (0.0007) [2023-12-27 11:48:53,692][20381] Updated weights for policy 1, policy_version 889621 (0.0010) [2023-12-27 11:48:53,713][20347] Updated weights for policy 0, policy_version 890268 (0.0010) [2023-12-27 11:48:53,760][20347] Updated weights for policy 0, policy_version 890278 (0.0010) [2023-12-27 11:48:53,817][20347] Updated weights for policy 0, policy_version 890288 (0.0010) [2023-12-27 11:48:54,266][20381] Updated weights for policy 1, policy_version 889631 (0.0007) [2023-12-27 11:48:54,314][20381] Updated weights for policy 1, policy_version 889641 (0.0009) [2023-12-27 11:48:54,375][20381] Updated weights for policy 1, policy_version 889651 (0.0008) [2023-12-27 11:48:54,453][20347] Updated weights for policy 0, policy_version 890298 (0.0009) [2023-12-27 11:48:54,510][20347] Updated weights for policy 0, policy_version 890308 (0.0010) [2023-12-27 11:48:54,567][20347] Updated weights for policy 0, policy_version 890318 (0.0010) [2023-12-27 11:48:55,043][20381] Updated weights for policy 1, policy_version 889661 (0.0010) [2023-12-27 11:48:55,096][20381] Updated weights for policy 1, policy_version 889671 (0.0010) [2023-12-27 11:48:55,148][20381] Updated weights for policy 1, policy_version 889681 (0.0010) [2023-12-27 11:48:55,175][20347] Updated weights for policy 0, policy_version 890328 (0.0010) [2023-12-27 11:48:55,222][20347] Updated weights for policy 0, policy_version 890338 (0.0005) [2023-12-27 11:48:55,281][20347] Updated weights for policy 0, policy_version 890348 (0.0005) [2023-12-27 11:48:55,821][20381] Updated weights for policy 1, policy_version 889691 (0.0009) [2023-12-27 11:48:55,878][20381] Updated weights for policy 1, policy_version 889701 (0.0011) [2023-12-27 11:48:55,896][20347] Updated weights for policy 0, policy_version 890358 (0.0008) [2023-12-27 11:48:55,936][20381] Updated weights for policy 1, policy_version 889711 (0.0011) [2023-12-27 11:48:55,947][20347] Updated weights for policy 0, policy_version 890368 (0.0009) [2023-12-27 11:48:56,005][20347] Updated weights for policy 0, policy_version 890378 (0.0006) [2023-12-27 11:48:56,621][20381] Updated weights for policy 1, policy_version 889721 (0.0011) [2023-12-27 11:48:56,650][20347] Updated weights for policy 0, policy_version 890388 (0.0010) [2023-12-27 11:48:56,685][20381] Updated weights for policy 1, policy_version 889731 (0.0009) [2023-12-27 11:48:56,702][20347] Updated weights for policy 0, policy_version 890398 (0.0010) [2023-12-27 11:48:56,744][20381] Updated weights for policy 1, policy_version 889741 (0.0011) [2023-12-27 11:48:56,755][20347] Updated weights for policy 0, policy_version 890408 (0.0010) [2023-12-27 11:48:56,800][20381] Updated weights for policy 1, policy_version 889751 (0.0011) [2023-12-27 11:48:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20206.9, 300 sec: 19744.1). Total num frames: 455778304. Throughput: 0: 10229.7, 1: 10231.7. Samples: 455770908. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:48:57,410][20347] Updated weights for policy 0, policy_version 890418 (0.0009) [2023-12-27 11:48:57,458][20347] Updated weights for policy 0, policy_version 890428 (0.0007) [2023-12-27 11:48:57,503][20381] Updated weights for policy 1, policy_version 889761 (0.0010) [2023-12-27 11:48:57,509][20347] Updated weights for policy 0, policy_version 890438 (0.0005) [2023-12-27 11:48:57,555][20381] Updated weights for policy 1, policy_version 889771 (0.0010) [2023-12-27 11:48:57,565][20347] Updated weights for policy 0, policy_version 890448 (0.0006) [2023-12-27 11:48:57,601][20381] Updated weights for policy 1, policy_version 889781 (0.0010) [2023-12-27 11:48:58,222][20347] Updated weights for policy 0, policy_version 890458 (0.0007) [2023-12-27 11:48:58,252][20381] Updated weights for policy 1, policy_version 889791 (0.0009) [2023-12-27 11:48:58,278][20347] Updated weights for policy 0, policy_version 890468 (0.0006) [2023-12-27 11:48:58,311][20381] Updated weights for policy 1, policy_version 889801 (0.0008) [2023-12-27 11:48:58,340][20347] Updated weights for policy 0, policy_version 890478 (0.0007) [2023-12-27 11:48:58,377][20381] Updated weights for policy 1, policy_version 889811 (0.0009) [2023-12-27 11:48:59,066][20347] Updated weights for policy 0, policy_version 890488 (0.0008) [2023-12-27 11:48:59,111][20347] Updated weights for policy 0, policy_version 890498 (0.0009) [2023-12-27 11:48:59,165][20347] Updated weights for policy 0, policy_version 890508 (0.0009) [2023-12-27 11:48:59,170][20381] Updated weights for policy 1, policy_version 889821 (0.0008) [2023-12-27 11:48:59,240][20381] Updated weights for policy 1, policy_version 889831 (0.0009) [2023-12-27 11:48:59,301][20381] Updated weights for policy 1, policy_version 889841 (0.0008) [2023-12-27 11:48:59,911][20347] Updated weights for policy 0, policy_version 890518 (0.0008) [2023-12-27 11:48:59,973][20347] Updated weights for policy 0, policy_version 890528 (0.0008) [2023-12-27 11:48:59,976][20381] Updated weights for policy 1, policy_version 889851 (0.0006) [2023-12-27 11:49:00,033][20381] Updated weights for policy 1, policy_version 889861 (0.0005) [2023-12-27 11:49:00,037][20347] Updated weights for policy 0, policy_version 890538 (0.0008) [2023-12-27 11:49:00,093][20381] Updated weights for policy 1, policy_version 889871 (0.0006) [2023-12-27 11:49:00,641][20347] Updated weights for policy 0, policy_version 890548 (0.0006) [2023-12-27 11:49:00,700][20347] Updated weights for policy 0, policy_version 890558 (0.0007) [2023-12-27 11:49:00,714][20381] Updated weights for policy 1, policy_version 889881 (0.0008) [2023-12-27 11:49:00,759][20347] Updated weights for policy 0, policy_version 890568 (0.0006) [2023-12-27 11:49:00,771][20381] Updated weights for policy 1, policy_version 889891 (0.0007) [2023-12-27 11:49:00,832][20381] Updated weights for policy 1, policy_version 889901 (0.0008) [2023-12-27 11:49:00,894][20381] Updated weights for policy 1, policy_version 889911 (0.0008) [2023-12-27 11:49:01,381][20347] Updated weights for policy 0, policy_version 890578 (0.0006) [2023-12-27 11:49:01,438][20347] Updated weights for policy 0, policy_version 890588 (0.0007) [2023-12-27 11:49:01,494][20347] Updated weights for policy 0, policy_version 890598 (0.0007) [2023-12-27 11:49:01,549][20347] Updated weights for policy 0, policy_version 890608 (0.0007) [2023-12-27 11:49:01,661][20381] Updated weights for policy 1, policy_version 889921 (0.0008) [2023-12-27 11:49:01,728][20381] Updated weights for policy 1, policy_version 889931 (0.0008) [2023-12-27 11:49:01,778][20381] Updated weights for policy 1, policy_version 889941 (0.0008) [2023-12-27 11:49:02,296][20347] Updated weights for policy 0, policy_version 890618 (0.0008) [2023-12-27 11:49:02,349][20347] Updated weights for policy 0, policy_version 890628 (0.0008) [2023-12-27 11:49:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 19744.1). Total num frames: 455876608. Throughput: 0: 10318.5, 1: 10242.0. Samples: 455891960. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:02,363][19170] Avg episode reward: [(0, '961.320'), (1, '1000.000')] [2023-12-27 11:49:02,418][20347] Updated weights for policy 0, policy_version 890638 (0.0008) [2023-12-27 11:49:02,468][20381] Updated weights for policy 1, policy_version 889951 (0.0008) [2023-12-27 11:49:02,527][20381] Updated weights for policy 1, policy_version 889961 (0.0008) [2023-12-27 11:49:02,588][20381] Updated weights for policy 1, policy_version 889971 (0.0007) [2023-12-27 11:49:03,097][20291] KL-divergence is very high: 144.3287 [2023-12-27 11:49:03,124][20347] Updated weights for policy 0, policy_version 890648 (0.0008) [2023-12-27 11:49:03,177][20347] Updated weights for policy 0, policy_version 890658 (0.0008) [2023-12-27 11:49:03,178][20381] Updated weights for policy 1, policy_version 889981 (0.0007) [2023-12-27 11:49:03,225][20347] Updated weights for policy 0, policy_version 890668 (0.0006) [2023-12-27 11:49:03,230][20381] Updated weights for policy 1, policy_version 889991 (0.0007) [2023-12-27 11:49:03,287][20381] Updated weights for policy 1, policy_version 890001 (0.0007) [2023-12-27 11:49:03,916][20347] Updated weights for policy 0, policy_version 890678 (0.0007) [2023-12-27 11:49:03,933][20381] Updated weights for policy 1, policy_version 890011 (0.0007) [2023-12-27 11:49:03,953][20291] KL-divergence is very high: 608.1182 [2023-12-27 11:49:03,977][20347] Updated weights for policy 0, policy_version 890688 (0.0006) [2023-12-27 11:49:03,991][20381] Updated weights for policy 1, policy_version 890021 (0.0005) [2023-12-27 11:49:04,003][20291] KL-divergence is very high: 125.0313 [2023-12-27 11:49:04,037][20347] Updated weights for policy 0, policy_version 890698 (0.0006) [2023-12-27 11:49:04,043][20381] Updated weights for policy 1, policy_version 890031 (0.0005) [2023-12-27 11:49:04,703][20381] Updated weights for policy 1, policy_version 890041 (0.0006) [2023-12-27 11:49:04,732][20347] Updated weights for policy 0, policy_version 890708 (0.0006) [2023-12-27 11:49:04,749][20381] Updated weights for policy 1, policy_version 890051 (0.0007) [2023-12-27 11:49:04,788][20347] Updated weights for policy 0, policy_version 890718 (0.0008) [2023-12-27 11:49:04,798][20381] Updated weights for policy 1, policy_version 890061 (0.0006) [2023-12-27 11:49:04,840][20347] Updated weights for policy 0, policy_version 890728 (0.0006) [2023-12-27 11:49:04,847][20381] Updated weights for policy 1, policy_version 890071 (0.0006) [2023-12-27 11:49:05,485][20347] Updated weights for policy 0, policy_version 890738 (0.0008) [2023-12-27 11:49:05,487][20381] Updated weights for policy 1, policy_version 890081 (0.0008) [2023-12-27 11:49:05,536][20347] Updated weights for policy 0, policy_version 890748 (0.0006) [2023-12-27 11:49:05,541][20291] KL-divergence is very high: 263.9995 [2023-12-27 11:49:05,543][20381] Updated weights for policy 1, policy_version 890091 (0.0007) [2023-12-27 11:49:05,546][20291] KL-divergence is very high: 130.6066 [2023-12-27 11:49:05,578][20291] KL-divergence is very high: 215.5795 [2023-12-27 11:49:05,584][20291] KL-divergence is very high: 104.1379 [2023-12-27 11:49:05,585][20347] Updated weights for policy 0, policy_version 890758 (0.0009) [2023-12-27 11:49:05,593][20381] Updated weights for policy 1, policy_version 890101 (0.0009) [2023-12-27 11:49:05,642][20347] Updated weights for policy 0, policy_version 890768 (0.0007) [2023-12-27 11:49:06,216][20381] Updated weights for policy 1, policy_version 890111 (0.0007) [2023-12-27 11:49:06,274][20381] Updated weights for policy 1, policy_version 890121 (0.0008) [2023-12-27 11:49:06,334][20381] Updated weights for policy 1, policy_version 890131 (0.0008) [2023-12-27 11:49:06,356][20347] Updated weights for policy 0, policy_version 890778 (0.0006) [2023-12-27 11:49:06,378][20291] KL-divergence is very high: 176.4431 [2023-12-27 11:49:06,412][20347] Updated weights for policy 0, policy_version 890788 (0.0008) [2023-12-27 11:49:06,427][20291] KL-divergence is very high: 181.7650 [2023-12-27 11:49:06,472][20347] Updated weights for policy 0, policy_version 890798 (0.0007) [2023-12-27 11:49:06,474][20291] KL-divergence is very high: 274.6253 [2023-12-27 11:49:07,060][20381] Updated weights for policy 1, policy_version 890141 (0.0007) [2023-12-27 11:49:07,109][20381] Updated weights for policy 1, policy_version 890151 (0.0007) [2023-12-27 11:49:07,129][20347] Updated weights for policy 0, policy_version 890808 (0.0006) [2023-12-27 11:49:07,156][20381] Updated weights for policy 1, policy_version 890161 (0.0007) [2023-12-27 11:49:07,183][20347] Updated weights for policy 0, policy_version 890818 (0.0005) [2023-12-27 11:49:07,240][20347] Updated weights for policy 0, policy_version 890828 (0.0005) [2023-12-27 11:49:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.2, 300 sec: 19799.7). Total num frames: 455991296. Throughput: 0: 10338.0, 1: 10265.5. Samples: 455953848. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:07,363][19170] Avg episode reward: [(0, '922.550'), (1, '1000.000')] [2023-12-27 11:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000890832_228081664.pth... [2023-12-27 11:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000890168_227909632.pth... [2023-12-27 11:49:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000889616_227770368.pth [2023-12-27 11:49:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000888952_227598336.pth [2023-12-27 11:49:07,808][20381] Updated weights for policy 1, policy_version 890171 (0.0007) [2023-12-27 11:49:07,862][20381] Updated weights for policy 1, policy_version 890181 (0.0006) [2023-12-27 11:49:07,877][20347] Updated weights for policy 0, policy_version 890838 (0.0006) [2023-12-27 11:49:07,921][20381] Updated weights for policy 1, policy_version 890191 (0.0009) [2023-12-27 11:49:07,931][20347] Updated weights for policy 0, policy_version 890848 (0.0006) [2023-12-27 11:49:07,990][20347] Updated weights for policy 0, policy_version 890858 (0.0006) [2023-12-27 11:49:08,613][20381] Updated weights for policy 1, policy_version 890201 (0.0008) [2023-12-27 11:49:08,669][20381] Updated weights for policy 1, policy_version 890211 (0.0008) [2023-12-27 11:49:08,701][20347] Updated weights for policy 0, policy_version 890868 (0.0006) [2023-12-27 11:49:08,726][20381] Updated weights for policy 1, policy_version 890221 (0.0006) [2023-12-27 11:49:08,758][20347] Updated weights for policy 0, policy_version 890878 (0.0006) [2023-12-27 11:49:08,776][20381] Updated weights for policy 1, policy_version 890231 (0.0005) [2023-12-27 11:49:08,813][20347] Updated weights for policy 0, policy_version 890888 (0.0006) [2023-12-27 11:49:09,508][20347] Updated weights for policy 0, policy_version 890898 (0.0007) [2023-12-27 11:49:09,547][20381] Updated weights for policy 1, policy_version 890241 (0.0006) [2023-12-27 11:49:09,571][20347] Updated weights for policy 0, policy_version 890908 (0.0006) [2023-12-27 11:49:09,611][20381] Updated weights for policy 1, policy_version 890251 (0.0005) [2023-12-27 11:49:09,635][20347] Updated weights for policy 0, policy_version 890918 (0.0006) [2023-12-27 11:49:09,674][20381] Updated weights for policy 1, policy_version 890261 (0.0008) [2023-12-27 11:49:09,695][20347] Updated weights for policy 0, policy_version 890928 (0.0006) [2023-12-27 11:49:10,306][20381] Updated weights for policy 1, policy_version 890271 (0.0006) [2023-12-27 11:49:10,369][20381] Updated weights for policy 1, policy_version 890281 (0.0008) [2023-12-27 11:49:10,426][20347] Updated weights for policy 0, policy_version 890938 (0.0006) [2023-12-27 11:49:10,435][20381] Updated weights for policy 1, policy_version 890291 (0.0008) [2023-12-27 11:49:10,475][20347] Updated weights for policy 0, policy_version 890948 (0.0005) [2023-12-27 11:49:10,533][20347] Updated weights for policy 0, policy_version 890958 (0.0008) [2023-12-27 11:49:11,111][20381] Updated weights for policy 1, policy_version 890301 (0.0007) [2023-12-27 11:49:11,181][20381] Updated weights for policy 1, policy_version 890311 (0.0009) [2023-12-27 11:49:11,241][20381] Updated weights for policy 1, policy_version 890321 (0.0008) [2023-12-27 11:49:11,286][20347] Updated weights for policy 0, policy_version 890968 (0.0008) [2023-12-27 11:49:11,352][20347] Updated weights for policy 0, policy_version 890978 (0.0008) [2023-12-27 11:49:11,421][20347] Updated weights for policy 0, policy_version 890988 (0.0008) [2023-12-27 11:49:12,035][20381] Updated weights for policy 1, policy_version 890331 (0.0007) [2023-12-27 11:49:12,091][20381] Updated weights for policy 1, policy_version 890341 (0.0008) [2023-12-27 11:49:12,143][20381] Updated weights for policy 1, policy_version 890351 (0.0007) [2023-12-27 11:49:12,157][20347] Updated weights for policy 0, policy_version 890998 (0.0007) [2023-12-27 11:49:12,214][20347] Updated weights for policy 0, policy_version 891008 (0.0006) [2023-12-27 11:49:12,271][20347] Updated weights for policy 0, policy_version 891018 (0.0007) [2023-12-27 11:49:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 19799.6). Total num frames: 456089600. Throughput: 0: 10333.2, 1: 10247.6. Samples: 456076616. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:12,363][19170] Avg episode reward: [(0, '951.480'), (1, '1000.000')] [2023-12-27 11:49:12,895][20381] Updated weights for policy 1, policy_version 890361 (0.0006) [2023-12-27 11:49:12,955][20381] Updated weights for policy 1, policy_version 890371 (0.0008) [2023-12-27 11:49:12,977][20347] Updated weights for policy 0, policy_version 891028 (0.0007) [2023-12-27 11:49:13,009][20381] Updated weights for policy 1, policy_version 890381 (0.0006) [2023-12-27 11:49:13,034][20347] Updated weights for policy 0, policy_version 891038 (0.0005) [2023-12-27 11:49:13,065][20381] Updated weights for policy 1, policy_version 890391 (0.0008) [2023-12-27 11:49:13,090][20347] Updated weights for policy 0, policy_version 891048 (0.0005) [2023-12-27 11:49:13,644][20347] Updated weights for policy 0, policy_version 891058 (0.0007) [2023-12-27 11:49:13,702][20347] Updated weights for policy 0, policy_version 891068 (0.0010) [2023-12-27 11:49:13,754][20347] Updated weights for policy 0, policy_version 891078 (0.0008) [2023-12-27 11:49:13,761][20381] Updated weights for policy 1, policy_version 890401 (0.0010) [2023-12-27 11:49:13,808][20347] Updated weights for policy 0, policy_version 891088 (0.0010) [2023-12-27 11:49:13,823][20381] Updated weights for policy 1, policy_version 890411 (0.0010) [2023-12-27 11:49:13,877][20381] Updated weights for policy 1, policy_version 890421 (0.0010) [2023-12-27 11:49:14,418][20347] Updated weights for policy 0, policy_version 891098 (0.0010) [2023-12-27 11:49:14,470][20347] Updated weights for policy 0, policy_version 891108 (0.0010) [2023-12-27 11:49:14,525][20347] Updated weights for policy 0, policy_version 891118 (0.0005) [2023-12-27 11:49:14,554][20381] Updated weights for policy 1, policy_version 890431 (0.0010) [2023-12-27 11:49:14,616][20381] Updated weights for policy 1, policy_version 890441 (0.0011) [2023-12-27 11:49:14,674][20381] Updated weights for policy 1, policy_version 890451 (0.0010) [2023-12-27 11:49:15,240][20347] Updated weights for policy 0, policy_version 891128 (0.0010) [2023-12-27 11:49:15,328][20347] Updated weights for policy 0, policy_version 891140 (0.0010) [2023-12-27 11:49:15,380][20381] Updated weights for policy 1, policy_version 890461 (0.0011) [2023-12-27 11:49:15,395][20347] Updated weights for policy 0, policy_version 891150 (0.0011) [2023-12-27 11:49:15,434][20381] Updated weights for policy 1, policy_version 890471 (0.0010) [2023-12-27 11:49:15,489][20381] Updated weights for policy 1, policy_version 890481 (0.0011) [2023-12-27 11:49:16,077][20347] Updated weights for policy 0, policy_version 891160 (0.0006) [2023-12-27 11:49:16,130][20347] Updated weights for policy 0, policy_version 891170 (0.0006) [2023-12-27 11:49:16,132][20381] Updated weights for policy 1, policy_version 890491 (0.0009) [2023-12-27 11:49:16,184][20347] Updated weights for policy 0, policy_version 891180 (0.0008) [2023-12-27 11:49:16,186][20381] Updated weights for policy 1, policy_version 890501 (0.0005) [2023-12-27 11:49:16,239][20381] Updated weights for policy 1, policy_version 890511 (0.0005) [2023-12-27 11:49:16,766][20381] Updated weights for policy 1, policy_version 890521 (0.0005) [2023-12-27 11:49:16,827][20381] Updated weights for policy 1, policy_version 890531 (0.0005) [2023-12-27 11:49:16,827][20347] Updated weights for policy 0, policy_version 891190 (0.0010) [2023-12-27 11:49:16,876][20381] Updated weights for policy 1, policy_version 890541 (0.0005) [2023-12-27 11:49:16,890][20347] Updated weights for policy 0, policy_version 891200 (0.0009) [2023-12-27 11:49:16,933][20381] Updated weights for policy 1, policy_version 890551 (0.0006) [2023-12-27 11:49:16,956][20347] Updated weights for policy 0, policy_version 891210 (0.0007) [2023-12-27 11:49:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 19827.4). Total num frames: 456187904. Throughput: 0: 10278.7, 1: 10216.9. Samples: 456199532. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:49:17,552][20381] Updated weights for policy 1, policy_version 890561 (0.0010) [2023-12-27 11:49:17,583][20347] Updated weights for policy 0, policy_version 891220 (0.0008) [2023-12-27 11:49:17,606][20381] Updated weights for policy 1, policy_version 890571 (0.0010) [2023-12-27 11:49:17,645][20347] Updated weights for policy 0, policy_version 891230 (0.0006) [2023-12-27 11:49:17,664][20381] Updated weights for policy 1, policy_version 890581 (0.0010) [2023-12-27 11:49:17,700][20347] Updated weights for policy 0, policy_version 891240 (0.0006) [2023-12-27 11:49:18,307][20347] Updated weights for policy 0, policy_version 891250 (0.0006) [2023-12-27 11:49:18,364][20381] Updated weights for policy 1, policy_version 890591 (0.0010) [2023-12-27 11:49:18,370][20347] Updated weights for policy 0, policy_version 891260 (0.0008) [2023-12-27 11:49:18,427][20347] Updated weights for policy 0, policy_version 891270 (0.0008) [2023-12-27 11:49:18,430][20381] Updated weights for policy 1, policy_version 890601 (0.0009) [2023-12-27 11:49:18,480][20347] Updated weights for policy 0, policy_version 891280 (0.0006) [2023-12-27 11:49:18,485][20381] Updated weights for policy 1, policy_version 890611 (0.0010) [2023-12-27 11:49:19,139][20347] Updated weights for policy 0, policy_version 891290 (0.0009) [2023-12-27 11:49:19,167][20381] Updated weights for policy 1, policy_version 890621 (0.0009) [2023-12-27 11:49:19,195][20347] Updated weights for policy 0, policy_version 891300 (0.0010) [2023-12-27 11:49:19,229][20381] Updated weights for policy 1, policy_version 890631 (0.0009) [2023-12-27 11:49:19,258][20347] Updated weights for policy 0, policy_version 891310 (0.0007) [2023-12-27 11:49:19,299][20381] Updated weights for policy 1, policy_version 890641 (0.0008) [2023-12-27 11:49:19,957][20347] Updated weights for policy 0, policy_version 891320 (0.0008) [2023-12-27 11:49:20,009][20381] Updated weights for policy 1, policy_version 890651 (0.0008) [2023-12-27 11:49:20,019][20347] Updated weights for policy 0, policy_version 891330 (0.0007) [2023-12-27 11:49:20,072][20381] Updated weights for policy 1, policy_version 890661 (0.0006) [2023-12-27 11:49:20,086][20347] Updated weights for policy 0, policy_version 891340 (0.0007) [2023-12-27 11:49:20,133][20381] Updated weights for policy 1, policy_version 890671 (0.0006) [2023-12-27 11:49:20,797][20347] Updated weights for policy 0, policy_version 891350 (0.0007) [2023-12-27 11:49:20,818][20381] Updated weights for policy 1, policy_version 890681 (0.0008) [2023-12-27 11:49:20,856][20347] Updated weights for policy 0, policy_version 891360 (0.0008) [2023-12-27 11:49:20,886][20381] Updated weights for policy 1, policy_version 890691 (0.0007) [2023-12-27 11:49:20,912][20347] Updated weights for policy 0, policy_version 891370 (0.0006) [2023-12-27 11:49:20,945][20381] Updated weights for policy 1, policy_version 890701 (0.0006) [2023-12-27 11:49:21,003][20381] Updated weights for policy 1, policy_version 890711 (0.0007) [2023-12-27 11:49:21,709][20347] Updated weights for policy 0, policy_version 891380 (0.0006) [2023-12-27 11:49:21,776][20347] Updated weights for policy 0, policy_version 891390 (0.0008) [2023-12-27 11:49:21,797][20381] Updated weights for policy 1, policy_version 890721 (0.0006) [2023-12-27 11:49:21,834][20347] Updated weights for policy 0, policy_version 891400 (0.0006) [2023-12-27 11:49:21,861][20381] Updated weights for policy 1, policy_version 890731 (0.0007) [2023-12-27 11:49:21,926][20381] Updated weights for policy 1, policy_version 890741 (0.0008) [2023-12-27 11:49:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 19799.6). Total num frames: 456286208. Throughput: 0: 10267.4, 1: 10201.8. Samples: 456260244. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 11:49:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000891408_228229120.pth... [2023-12-27 11:49:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000890744_228057088.pth... [2023-12-27 11:49:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000889560_227753984.pth [2023-12-27 11:49:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000890224_227926016.pth [2023-12-27 11:49:22,583][20347] Updated weights for policy 0, policy_version 891410 (0.0007) [2023-12-27 11:49:22,619][20381] Updated weights for policy 1, policy_version 890751 (0.0006) [2023-12-27 11:49:22,645][20347] Updated weights for policy 0, policy_version 891420 (0.0007) [2023-12-27 11:49:22,689][20381] Updated weights for policy 1, policy_version 890761 (0.0006) [2023-12-27 11:49:22,707][20347] Updated weights for policy 0, policy_version 891430 (0.0006) [2023-12-27 11:49:22,750][20381] Updated weights for policy 1, policy_version 890771 (0.0008) [2023-12-27 11:49:22,764][20347] Updated weights for policy 0, policy_version 891440 (0.0007) [2023-12-27 11:49:23,381][20381] Updated weights for policy 1, policy_version 890781 (0.0007) [2023-12-27 11:49:23,436][20381] Updated weights for policy 1, policy_version 890791 (0.0005) [2023-12-27 11:49:23,488][20381] Updated weights for policy 1, policy_version 890801 (0.0005) [2023-12-27 11:49:23,512][20347] Updated weights for policy 0, policy_version 891450 (0.0006) [2023-12-27 11:49:23,559][20347] Updated weights for policy 0, policy_version 891460 (0.0008) [2023-12-27 11:49:23,618][20347] Updated weights for policy 0, policy_version 891470 (0.0007) [2023-12-27 11:49:24,032][20381] Updated weights for policy 1, policy_version 890811 (0.0007) [2023-12-27 11:49:24,086][20381] Updated weights for policy 1, policy_version 890821 (0.0010) [2023-12-27 11:49:24,138][20381] Updated weights for policy 1, policy_version 890831 (0.0010) [2023-12-27 11:49:24,275][20347] Updated weights for policy 0, policy_version 891480 (0.0006) [2023-12-27 11:49:24,327][20347] Updated weights for policy 0, policy_version 891490 (0.0006) [2023-12-27 11:49:24,379][20347] Updated weights for policy 0, policy_version 891500 (0.0008) [2023-12-27 11:49:24,849][20381] Updated weights for policy 1, policy_version 890841 (0.0010) [2023-12-27 11:49:24,907][20381] Updated weights for policy 1, policy_version 890851 (0.0005) [2023-12-27 11:49:24,964][20347] Updated weights for policy 0, policy_version 891510 (0.0009) [2023-12-27 11:49:24,967][20381] Updated weights for policy 1, policy_version 890861 (0.0005) [2023-12-27 11:49:25,019][20347] Updated weights for policy 0, policy_version 891520 (0.0010) [2023-12-27 11:49:25,027][20381] Updated weights for policy 1, policy_version 890871 (0.0005) [2023-12-27 11:49:25,077][20347] Updated weights for policy 0, policy_version 891530 (0.0010) [2023-12-27 11:49:25,625][20381] Updated weights for policy 1, policy_version 890881 (0.0010) [2023-12-27 11:49:25,683][20381] Updated weights for policy 1, policy_version 890891 (0.0010) [2023-12-27 11:49:25,737][20381] Updated weights for policy 1, policy_version 890901 (0.0010) [2023-12-27 11:49:25,752][20347] Updated weights for policy 0, policy_version 891540 (0.0010) [2023-12-27 11:49:25,796][20347] Updated weights for policy 0, policy_version 891550 (0.0007) [2023-12-27 11:49:25,858][20347] Updated weights for policy 0, policy_version 891560 (0.0005) [2023-12-27 11:49:26,346][20381] Updated weights for policy 1, policy_version 890911 (0.0009) [2023-12-27 11:49:26,403][20381] Updated weights for policy 1, policy_version 890921 (0.0005) [2023-12-27 11:49:26,449][20347] Updated weights for policy 0, policy_version 891570 (0.0005) [2023-12-27 11:49:26,461][20381] Updated weights for policy 1, policy_version 890931 (0.0006) [2023-12-27 11:49:26,501][20347] Updated weights for policy 0, policy_version 891580 (0.0005) [2023-12-27 11:49:26,548][20347] Updated weights for policy 0, policy_version 891590 (0.0008) [2023-12-27 11:49:26,597][20347] Updated weights for policy 0, policy_version 891600 (0.0009) [2023-12-27 11:49:27,066][20381] Updated weights for policy 1, policy_version 890941 (0.0008) [2023-12-27 11:49:27,116][20381] Updated weights for policy 1, policy_version 890951 (0.0010) [2023-12-27 11:49:27,169][20381] Updated weights for policy 1, policy_version 890961 (0.0008) [2023-12-27 11:49:27,275][20347] Updated weights for policy 0, policy_version 891610 (0.0008) [2023-12-27 11:49:27,330][20347] Updated weights for policy 0, policy_version 891620 (0.0008) [2023-12-27 11:49:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 19827.4). Total num frames: 456392704. Throughput: 0: 10236.3, 1: 10224.3. Samples: 456382996. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '1000.000')] [2023-12-27 11:49:27,386][20347] Updated weights for policy 0, policy_version 891630 (0.0008) [2023-12-27 11:49:27,921][20381] Updated weights for policy 1, policy_version 890971 (0.0008) [2023-12-27 11:49:27,967][20381] Updated weights for policy 1, policy_version 890981 (0.0008) [2023-12-27 11:49:27,974][20347] Updated weights for policy 0, policy_version 891640 (0.0005) [2023-12-27 11:49:28,018][20381] Updated weights for policy 1, policy_version 890991 (0.0008) [2023-12-27 11:49:28,025][20347] Updated weights for policy 0, policy_version 891650 (0.0005) [2023-12-27 11:49:28,079][20347] Updated weights for policy 0, policy_version 891660 (0.0005) [2023-12-27 11:49:28,670][20347] Updated weights for policy 0, policy_version 891670 (0.0006) [2023-12-27 11:49:28,715][20347] Updated weights for policy 0, policy_version 891680 (0.0007) [2023-12-27 11:49:28,753][20381] Updated weights for policy 1, policy_version 891001 (0.0008) [2023-12-27 11:49:28,768][20347] Updated weights for policy 0, policy_version 891690 (0.0007) [2023-12-27 11:49:28,815][20381] Updated weights for policy 1, policy_version 891011 (0.0007) [2023-12-27 11:49:28,866][20381] Updated weights for policy 1, policy_version 891021 (0.0007) [2023-12-27 11:49:28,917][20381] Updated weights for policy 1, policy_version 891031 (0.0007) [2023-12-27 11:49:29,449][20347] Updated weights for policy 0, policy_version 891700 (0.0006) [2023-12-27 11:49:29,510][20347] Updated weights for policy 0, policy_version 891710 (0.0007) [2023-12-27 11:49:29,556][20347] Updated weights for policy 0, policy_version 891720 (0.0008) [2023-12-27 11:49:29,636][20381] Updated weights for policy 1, policy_version 891041 (0.0007) [2023-12-27 11:49:29,691][20381] Updated weights for policy 1, policy_version 891051 (0.0008) [2023-12-27 11:49:29,746][20381] Updated weights for policy 1, policy_version 891061 (0.0008) [2023-12-27 11:49:30,173][20347] Updated weights for policy 0, policy_version 891730 (0.0006) [2023-12-27 11:49:30,228][20347] Updated weights for policy 0, policy_version 891740 (0.0007) [2023-12-27 11:49:30,288][20347] Updated weights for policy 0, policy_version 891750 (0.0007) [2023-12-27 11:49:30,343][20347] Updated weights for policy 0, policy_version 891760 (0.0007) [2023-12-27 11:49:30,511][20381] Updated weights for policy 1, policy_version 891071 (0.0007) [2023-12-27 11:49:30,565][20381] Updated weights for policy 1, policy_version 891081 (0.0008) [2023-12-27 11:49:30,622][20381] Updated weights for policy 1, policy_version 891091 (0.0007) [2023-12-27 11:49:31,003][20347] Updated weights for policy 0, policy_version 891770 (0.0005) [2023-12-27 11:49:31,070][20347] Updated weights for policy 0, policy_version 891780 (0.0008) [2023-12-27 11:49:31,116][20347] Updated weights for policy 0, policy_version 891790 (0.0006) [2023-12-27 11:49:31,322][20381] Updated weights for policy 1, policy_version 891101 (0.0008) [2023-12-27 11:49:31,393][20381] Updated weights for policy 1, policy_version 891111 (0.0008) [2023-12-27 11:49:31,458][20381] Updated weights for policy 1, policy_version 891121 (0.0008) [2023-12-27 11:49:31,779][20347] Updated weights for policy 0, policy_version 891800 (0.0007) [2023-12-27 11:49:31,797][20291] KL-divergence is very high: 145.3289 [2023-12-27 11:49:31,844][20347] Updated weights for policy 0, policy_version 891810 (0.0008) [2023-12-27 11:49:31,907][20347] Updated weights for policy 0, policy_version 891820 (0.0009) [2023-12-27 11:49:32,113][20381] Updated weights for policy 1, policy_version 891131 (0.0009) [2023-12-27 11:49:32,163][20381] Updated weights for policy 1, policy_version 891141 (0.0011) [2023-12-27 11:49:32,217][20381] Updated weights for policy 1, policy_version 891151 (0.0010) [2023-12-27 11:49:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 19799.6). Total num frames: 456499200. Throughput: 0: 10269.0, 1: 10172.3. Samples: 456507000. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:32,363][19170] Avg episode reward: [(0, '980.620'), (1, '1000.000')] [2023-12-27 11:49:32,590][20347] Updated weights for policy 0, policy_version 891830 (0.0008) [2023-12-27 11:49:32,644][20347] Updated weights for policy 0, policy_version 891840 (0.0007) [2023-12-27 11:49:32,699][20347] Updated weights for policy 0, policy_version 891850 (0.0007) [2023-12-27 11:49:32,955][20381] Updated weights for policy 1, policy_version 891161 (0.0011) [2023-12-27 11:49:33,006][20381] Updated weights for policy 1, policy_version 891171 (0.0010) [2023-12-27 11:49:33,052][20381] Updated weights for policy 1, policy_version 891181 (0.0010) [2023-12-27 11:49:33,106][20381] Updated weights for policy 1, policy_version 891191 (0.0009) [2023-12-27 11:49:33,276][20347] Updated weights for policy 0, policy_version 891860 (0.0008) [2023-12-27 11:49:33,325][20347] Updated weights for policy 0, policy_version 891870 (0.0007) [2023-12-27 11:49:33,375][20347] Updated weights for policy 0, policy_version 891880 (0.0007) [2023-12-27 11:49:33,737][20381] Updated weights for policy 1, policy_version 891201 (0.0008) [2023-12-27 11:49:33,778][20381] Updated weights for policy 1, policy_version 891211 (0.0008) [2023-12-27 11:49:33,830][20381] Updated weights for policy 1, policy_version 891221 (0.0007) [2023-12-27 11:49:34,045][20347] Updated weights for policy 0, policy_version 891890 (0.0007) [2023-12-27 11:49:34,104][20347] Updated weights for policy 0, policy_version 891900 (0.0008) [2023-12-27 11:49:34,165][20347] Updated weights for policy 0, policy_version 891910 (0.0007) [2023-12-27 11:49:34,226][20347] Updated weights for policy 0, policy_version 891920 (0.0007) [2023-12-27 11:49:34,555][20381] Updated weights for policy 1, policy_version 891231 (0.0007) [2023-12-27 11:49:34,613][20381] Updated weights for policy 1, policy_version 891241 (0.0009) [2023-12-27 11:49:34,673][20381] Updated weights for policy 1, policy_version 891251 (0.0007) [2023-12-27 11:49:34,827][20347] Updated weights for policy 0, policy_version 891930 (0.0007) [2023-12-27 11:49:34,891][20347] Updated weights for policy 0, policy_version 891940 (0.0008) [2023-12-27 11:49:34,948][20347] Updated weights for policy 0, policy_version 891950 (0.0006) [2023-12-27 11:49:35,304][20381] Updated weights for policy 1, policy_version 891261 (0.0006) [2023-12-27 11:49:35,366][20381] Updated weights for policy 1, policy_version 891271 (0.0006) [2023-12-27 11:49:35,430][20381] Updated weights for policy 1, policy_version 891281 (0.0006) [2023-12-27 11:49:35,619][20347] Updated weights for policy 0, policy_version 891960 (0.0008) [2023-12-27 11:49:35,680][20347] Updated weights for policy 0, policy_version 891970 (0.0006) [2023-12-27 11:49:35,731][20347] Updated weights for policy 0, policy_version 891980 (0.0008) [2023-12-27 11:49:36,105][20381] Updated weights for policy 1, policy_version 891291 (0.0007) [2023-12-27 11:49:36,168][20381] Updated weights for policy 1, policy_version 891301 (0.0007) [2023-12-27 11:49:36,221][20381] Updated weights for policy 1, policy_version 891311 (0.0008) [2023-12-27 11:49:36,318][20347] Updated weights for policy 0, policy_version 891990 (0.0007) [2023-12-27 11:49:36,370][20291] KL-divergence is very high: 264.7317 [2023-12-27 11:49:36,375][20347] Updated weights for policy 0, policy_version 892000 (0.0006) [2023-12-27 11:49:36,390][20291] KL-divergence is very high: 158.4985 [2023-12-27 11:49:36,422][20291] KL-divergence is very high: 117.6846 [2023-12-27 11:49:36,438][20347] Updated weights for policy 0, policy_version 892010 (0.0005) [2023-12-27 11:49:36,949][20381] Updated weights for policy 1, policy_version 891321 (0.0008) [2023-12-27 11:49:37,010][20381] Updated weights for policy 1, policy_version 891331 (0.0008) [2023-12-27 11:49:37,063][20347] Updated weights for policy 0, policy_version 892020 (0.0007) [2023-12-27 11:49:37,068][20381] Updated weights for policy 1, policy_version 891341 (0.0008) [2023-12-27 11:49:37,114][20381] Updated weights for policy 1, policy_version 891351 (0.0006) [2023-12-27 11:49:37,118][20347] Updated weights for policy 0, policy_version 892030 (0.0007) [2023-12-27 11:49:37,183][20347] Updated weights for policy 0, policy_version 892040 (0.0006) [2023-12-27 11:49:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 19855.2). Total num frames: 456605696. Throughput: 0: 10319.8, 1: 10197.2. Samples: 456570108. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:37,363][19170] Avg episode reward: [(0, '980.710'), (1, '1000.000')] [2023-12-27 11:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000891352_228212736.pth... [2023-12-27 11:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000892048_228392960.pth... [2023-12-27 11:49:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000890168_227909632.pth [2023-12-27 11:49:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000890832_228081664.pth [2023-12-27 11:49:37,745][20381] Updated weights for policy 1, policy_version 891361 (0.0008) [2023-12-27 11:49:37,801][20381] Updated weights for policy 1, policy_version 891371 (0.0008) [2023-12-27 11:49:37,827][20347] Updated weights for policy 0, policy_version 892050 (0.0005) [2023-12-27 11:49:37,855][20381] Updated weights for policy 1, policy_version 891381 (0.0007) [2023-12-27 11:49:37,875][20347] Updated weights for policy 0, policy_version 892060 (0.0005) [2023-12-27 11:49:37,932][20347] Updated weights for policy 0, policy_version 892070 (0.0007) [2023-12-27 11:49:37,980][20347] Updated weights for policy 0, policy_version 892080 (0.0007) [2023-12-27 11:49:38,530][20381] Updated weights for policy 1, policy_version 891391 (0.0007) [2023-12-27 11:49:38,587][20381] Updated weights for policy 1, policy_version 891401 (0.0007) [2023-12-27 11:49:38,647][20381] Updated weights for policy 1, policy_version 891411 (0.0007) [2023-12-27 11:49:38,710][20347] Updated weights for policy 0, policy_version 892090 (0.0007) [2023-12-27 11:49:38,774][20347] Updated weights for policy 0, policy_version 892100 (0.0007) [2023-12-27 11:49:38,831][20347] Updated weights for policy 0, policy_version 892110 (0.0008) [2023-12-27 11:49:39,332][20381] Updated weights for policy 1, policy_version 891421 (0.0007) [2023-12-27 11:49:39,397][20381] Updated weights for policy 1, policy_version 891431 (0.0006) [2023-12-27 11:49:39,462][20381] Updated weights for policy 1, policy_version 891441 (0.0007) [2023-12-27 11:49:39,598][20347] Updated weights for policy 0, policy_version 892120 (0.0006) [2023-12-27 11:49:39,659][20347] Updated weights for policy 0, policy_version 892130 (0.0006) [2023-12-27 11:49:39,718][20347] Updated weights for policy 0, policy_version 892140 (0.0010) [2023-12-27 11:49:40,169][20381] Updated weights for policy 1, policy_version 891451 (0.0007) [2023-12-27 11:49:40,238][20381] Updated weights for policy 1, policy_version 891461 (0.0006) [2023-12-27 11:49:40,303][20381] Updated weights for policy 1, policy_version 891471 (0.0007) [2023-12-27 11:49:40,370][20347] Updated weights for policy 0, policy_version 892150 (0.0008) [2023-12-27 11:49:40,428][20347] Updated weights for policy 0, policy_version 892160 (0.0006) [2023-12-27 11:49:40,491][20347] Updated weights for policy 0, policy_version 892170 (0.0008) [2023-12-27 11:49:40,920][20381] Updated weights for policy 1, policy_version 891481 (0.0007) [2023-12-27 11:49:40,982][20381] Updated weights for policy 1, policy_version 891491 (0.0008) [2023-12-27 11:49:41,051][20381] Updated weights for policy 1, policy_version 891501 (0.0008) [2023-12-27 11:49:41,109][20381] Updated weights for policy 1, policy_version 891511 (0.0007) [2023-12-27 11:49:41,153][20347] Updated weights for policy 0, policy_version 892180 (0.0008) [2023-12-27 11:49:41,218][20347] Updated weights for policy 0, policy_version 892190 (0.0007) [2023-12-27 11:49:41,278][20347] Updated weights for policy 0, policy_version 892200 (0.0008) [2023-12-27 11:49:41,834][20381] Updated weights for policy 1, policy_version 891521 (0.0008) [2023-12-27 11:49:41,891][20381] Updated weights for policy 1, policy_version 891531 (0.0005) [2023-12-27 11:49:41,949][20381] Updated weights for policy 1, policy_version 891541 (0.0005) [2023-12-27 11:49:42,060][20347] Updated weights for policy 0, policy_version 892210 (0.0009) [2023-12-27 11:49:42,116][20347] Updated weights for policy 0, policy_version 892220 (0.0008) [2023-12-27 11:49:42,178][20347] Updated weights for policy 0, policy_version 892230 (0.0008) [2023-12-27 11:49:42,240][20347] Updated weights for policy 0, policy_version 892240 (0.0006) [2023-12-27 11:49:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 19827.4). Total num frames: 456704000. Throughput: 0: 10315.2, 1: 10162.1. Samples: 456692388. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:42,363][19170] Avg episode reward: [(0, '970.840'), (1, '1000.000')] [2023-12-27 11:49:42,650][20381] Updated weights for policy 1, policy_version 891551 (0.0007) [2023-12-27 11:49:42,706][20381] Updated weights for policy 1, policy_version 891561 (0.0007) [2023-12-27 11:49:42,756][20381] Updated weights for policy 1, policy_version 891571 (0.0007) [2023-12-27 11:49:42,889][20347] Updated weights for policy 0, policy_version 892250 (0.0010) [2023-12-27 11:49:42,943][20347] Updated weights for policy 0, policy_version 892260 (0.0010) [2023-12-27 11:49:43,004][20347] Updated weights for policy 0, policy_version 892270 (0.0010) [2023-12-27 11:49:43,448][20381] Updated weights for policy 1, policy_version 891581 (0.0008) [2023-12-27 11:49:43,493][20381] Updated weights for policy 1, policy_version 891591 (0.0009) [2023-12-27 11:49:43,544][20381] Updated weights for policy 1, policy_version 891601 (0.0005) [2023-12-27 11:49:43,642][20347] Updated weights for policy 0, policy_version 892280 (0.0010) [2023-12-27 11:49:43,722][20347] Updated weights for policy 0, policy_version 892290 (0.0008) [2023-12-27 11:49:43,772][20347] Updated weights for policy 0, policy_version 892300 (0.0005) [2023-12-27 11:49:44,136][20381] Updated weights for policy 1, policy_version 891611 (0.0005) [2023-12-27 11:49:44,192][20381] Updated weights for policy 1, policy_version 891621 (0.0005) [2023-12-27 11:49:44,248][20381] Updated weights for policy 1, policy_version 891631 (0.0007) [2023-12-27 11:49:44,300][20347] Updated weights for policy 0, policy_version 892310 (0.0005) [2023-12-27 11:49:44,358][20347] Updated weights for policy 0, policy_version 892320 (0.0005) [2023-12-27 11:49:44,420][20347] Updated weights for policy 0, policy_version 892330 (0.0005) [2023-12-27 11:49:44,807][20381] Updated weights for policy 1, policy_version 891641 (0.0008) [2023-12-27 11:49:44,872][20381] Updated weights for policy 1, policy_version 891651 (0.0006) [2023-12-27 11:49:44,938][20381] Updated weights for policy 1, policy_version 891661 (0.0008) [2023-12-27 11:49:45,001][20381] Updated weights for policy 1, policy_version 891671 (0.0011) [2023-12-27 11:49:45,023][20347] Updated weights for policy 0, policy_version 892340 (0.0009) [2023-12-27 11:49:45,091][20347] Updated weights for policy 0, policy_version 892350 (0.0010) [2023-12-27 11:49:45,157][20347] Updated weights for policy 0, policy_version 892360 (0.0010) [2023-12-27 11:49:45,588][20381] Updated weights for policy 1, policy_version 891681 (0.0010) [2023-12-27 11:49:45,643][20381] Updated weights for policy 1, policy_version 891691 (0.0010) [2023-12-27 11:49:45,698][20381] Updated weights for policy 1, policy_version 891701 (0.0010) [2023-12-27 11:49:45,796][20347] Updated weights for policy 0, policy_version 892370 (0.0010) [2023-12-27 11:49:45,853][20347] Updated weights for policy 0, policy_version 892380 (0.0010) [2023-12-27 11:49:45,920][20347] Updated weights for policy 0, policy_version 892390 (0.0010) [2023-12-27 11:49:45,974][20347] Updated weights for policy 0, policy_version 892400 (0.0010) [2023-12-27 11:49:46,371][20381] Updated weights for policy 1, policy_version 891711 (0.0007) [2023-12-27 11:49:46,431][20381] Updated weights for policy 1, policy_version 891721 (0.0007) [2023-12-27 11:49:46,486][20381] Updated weights for policy 1, policy_version 891731 (0.0007) [2023-12-27 11:49:46,634][20347] Updated weights for policy 0, policy_version 892410 (0.0010) [2023-12-27 11:49:46,682][20347] Updated weights for policy 0, policy_version 892420 (0.0010) [2023-12-27 11:49:46,736][20347] Updated weights for policy 0, policy_version 892430 (0.0010) [2023-12-27 11:49:47,109][20381] Updated weights for policy 1, policy_version 891741 (0.0008) [2023-12-27 11:49:47,160][20381] Updated weights for policy 1, policy_version 891751 (0.0007) [2023-12-27 11:49:47,213][20381] Updated weights for policy 1, policy_version 891761 (0.0009) [2023-12-27 11:49:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 19855.2). Total num frames: 456810496. Throughput: 0: 10342.2, 1: 10266.8. Samples: 456819364. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:47,363][19170] Avg episode reward: [(0, '883.570'), (1, '1000.000')] [2023-12-27 11:49:47,392][20347] Updated weights for policy 0, policy_version 892440 (0.0010) [2023-12-27 11:49:47,440][20291] KL-divergence is very high: 145.2227 [2023-12-27 11:49:47,444][20347] Updated weights for policy 0, policy_version 892450 (0.0010) [2023-12-27 11:49:47,492][20347] Updated weights for policy 0, policy_version 892460 (0.0010) [2023-12-27 11:49:47,841][20381] Updated weights for policy 1, policy_version 891771 (0.0009) [2023-12-27 11:49:47,894][20381] Updated weights for policy 1, policy_version 891781 (0.0007) [2023-12-27 11:49:47,945][20381] Updated weights for policy 1, policy_version 891791 (0.0010) [2023-12-27 11:49:48,173][20347] Updated weights for policy 0, policy_version 892470 (0.0010) [2023-12-27 11:49:48,225][20347] Updated weights for policy 0, policy_version 892480 (0.0010) [2023-12-27 11:49:48,282][20347] Updated weights for policy 0, policy_version 892490 (0.0010) [2023-12-27 11:49:48,617][20381] Updated weights for policy 1, policy_version 891801 (0.0010) [2023-12-27 11:49:48,683][20381] Updated weights for policy 1, policy_version 891811 (0.0011) [2023-12-27 11:49:48,739][20381] Updated weights for policy 1, policy_version 891821 (0.0011) [2023-12-27 11:49:48,799][20381] Updated weights for policy 1, policy_version 891831 (0.0011) [2023-12-27 11:49:48,980][20347] Updated weights for policy 0, policy_version 892500 (0.0010) [2023-12-27 11:49:49,039][20347] Updated weights for policy 0, policy_version 892510 (0.0010) [2023-12-27 11:49:49,096][20347] Updated weights for policy 0, policy_version 892520 (0.0010) [2023-12-27 11:49:49,538][20381] Updated weights for policy 1, policy_version 891841 (0.0011) [2023-12-27 11:49:49,590][20381] Updated weights for policy 1, policy_version 891851 (0.0009) [2023-12-27 11:49:49,647][20381] Updated weights for policy 1, policy_version 891861 (0.0007) [2023-12-27 11:49:49,794][20347] Updated weights for policy 0, policy_version 892530 (0.0010) [2023-12-27 11:49:49,856][20347] Updated weights for policy 0, policy_version 892540 (0.0011) [2023-12-27 11:49:49,929][20347] Updated weights for policy 0, policy_version 892550 (0.0008) [2023-12-27 11:49:49,987][20347] Updated weights for policy 0, policy_version 892560 (0.0010) [2023-12-27 11:49:50,370][20381] Updated weights for policy 1, policy_version 891871 (0.0009) [2023-12-27 11:49:50,420][20381] Updated weights for policy 1, policy_version 891881 (0.0008) [2023-12-27 11:49:50,471][20381] Updated weights for policy 1, policy_version 891891 (0.0008) [2023-12-27 11:49:50,677][20347] Updated weights for policy 0, policy_version 892570 (0.0010) [2023-12-27 11:49:50,735][20347] Updated weights for policy 0, policy_version 892580 (0.0007) [2023-12-27 11:49:50,787][20347] Updated weights for policy 0, policy_version 892590 (0.0008) [2023-12-27 11:49:51,221][20381] Updated weights for policy 1, policy_version 891901 (0.0008) [2023-12-27 11:49:51,282][20381] Updated weights for policy 1, policy_version 891911 (0.0008) [2023-12-27 11:49:51,335][20381] Updated weights for policy 1, policy_version 891921 (0.0006) [2023-12-27 11:49:51,487][20347] Updated weights for policy 0, policy_version 892600 (0.0007) [2023-12-27 11:49:51,544][20347] Updated weights for policy 0, policy_version 892610 (0.0008) [2023-12-27 11:49:51,596][20347] Updated weights for policy 0, policy_version 892620 (0.0009) [2023-12-27 11:49:52,047][20381] Updated weights for policy 1, policy_version 891931 (0.0009) [2023-12-27 11:49:52,106][20381] Updated weights for policy 1, policy_version 891941 (0.0008) [2023-12-27 11:49:52,160][20381] Updated weights for policy 1, policy_version 891951 (0.0008) [2023-12-27 11:49:52,361][20347] Updated weights for policy 0, policy_version 892630 (0.0009) [2023-12-27 11:49:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 19799.6). Total num frames: 456908800. Throughput: 0: 10352.9, 1: 10228.5. Samples: 456880012. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:52,363][19170] Avg episode reward: [(0, '864.460'), (1, '1000.000')] [2023-12-27 11:49:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000891960_228368384.pth... [2023-12-27 11:49:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000890744_228057088.pth [2023-12-27 11:49:52,421][20347] Updated weights for policy 0, policy_version 892640 (0.0008) [2023-12-27 11:49:52,489][20347] Updated weights for policy 0, policy_version 892650 (0.0008) [2023-12-27 11:49:52,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000892656_228548608.pth... [2023-12-27 11:49:52,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000891408_228229120.pth [2023-12-27 11:49:52,870][20381] Updated weights for policy 1, policy_version 891961 (0.0008) [2023-12-27 11:49:52,926][20381] Updated weights for policy 1, policy_version 891971 (0.0008) [2023-12-27 11:49:52,986][20381] Updated weights for policy 1, policy_version 891981 (0.0008) [2023-12-27 11:49:53,042][20381] Updated weights for policy 1, policy_version 891991 (0.0007) [2023-12-27 11:49:53,193][20347] Updated weights for policy 0, policy_version 892660 (0.0007) [2023-12-27 11:49:53,257][20347] Updated weights for policy 0, policy_version 892670 (0.0007) [2023-12-27 11:49:53,315][20347] Updated weights for policy 0, policy_version 892680 (0.0007) [2023-12-27 11:49:53,709][20381] Updated weights for policy 1, policy_version 892001 (0.0008) [2023-12-27 11:49:53,768][20381] Updated weights for policy 1, policy_version 892011 (0.0007) [2023-12-27 11:49:53,830][20381] Updated weights for policy 1, policy_version 892021 (0.0008) [2023-12-27 11:49:53,984][20347] Updated weights for policy 0, policy_version 892690 (0.0007) [2023-12-27 11:49:54,031][20347] Updated weights for policy 0, policy_version 892700 (0.0007) [2023-12-27 11:49:54,074][20347] Updated weights for policy 0, policy_version 892710 (0.0007) [2023-12-27 11:49:54,122][20347] Updated weights for policy 0, policy_version 892720 (0.0006) [2023-12-27 11:49:54,473][20381] Updated weights for policy 1, policy_version 892031 (0.0006) [2023-12-27 11:49:54,526][20381] Updated weights for policy 1, policy_version 892041 (0.0006) [2023-12-27 11:49:54,579][20381] Updated weights for policy 1, policy_version 892051 (0.0007) [2023-12-27 11:49:54,837][20347] Updated weights for policy 0, policy_version 892730 (0.0010) [2023-12-27 11:49:54,881][20347] Updated weights for policy 0, policy_version 892740 (0.0007) [2023-12-27 11:49:54,927][20347] Updated weights for policy 0, policy_version 892750 (0.0005) [2023-12-27 11:49:55,259][20381] Updated weights for policy 1, policy_version 892061 (0.0007) [2023-12-27 11:49:55,315][20381] Updated weights for policy 1, policy_version 892071 (0.0007) [2023-12-27 11:49:55,372][20381] Updated weights for policy 1, policy_version 892081 (0.0007) [2023-12-27 11:49:55,608][20347] Updated weights for policy 0, policy_version 892760 (0.0009) [2023-12-27 11:49:55,655][20347] Updated weights for policy 0, policy_version 892770 (0.0009) [2023-12-27 11:49:55,715][20347] Updated weights for policy 0, policy_version 892780 (0.0010) [2023-12-27 11:49:55,993][20381] Updated weights for policy 1, policy_version 892091 (0.0006) [2023-12-27 11:49:56,051][20381] Updated weights for policy 1, policy_version 892101 (0.0005) [2023-12-27 11:49:56,110][20381] Updated weights for policy 1, policy_version 892111 (0.0007) [2023-12-27 11:49:56,340][20347] Updated weights for policy 0, policy_version 892790 (0.0007) [2023-12-27 11:49:56,383][20347] Updated weights for policy 0, policy_version 892800 (0.0005) [2023-12-27 11:49:56,433][20347] Updated weights for policy 0, policy_version 892810 (0.0010) [2023-12-27 11:49:56,803][20381] Updated weights for policy 1, policy_version 892121 (0.0008) [2023-12-27 11:49:56,862][20381] Updated weights for policy 1, policy_version 892131 (0.0009) [2023-12-27 11:49:56,906][20381] Updated weights for policy 1, policy_version 892141 (0.0007) [2023-12-27 11:49:56,958][20381] Updated weights for policy 1, policy_version 892151 (0.0008) [2023-12-27 11:49:57,027][20347] Updated weights for policy 0, policy_version 892820 (0.0009) [2023-12-27 11:49:57,083][20347] Updated weights for policy 0, policy_version 892830 (0.0010) [2023-12-27 11:49:57,143][20347] Updated weights for policy 0, policy_version 892840 (0.0010) [2023-12-27 11:49:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 19827.4). Total num frames: 457015296. Throughput: 0: 10319.1, 1: 10212.9. Samples: 457000556. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:49:57,363][19170] Avg episode reward: [(0, '686.000'), (1, '1000.000')] [2023-12-27 11:49:57,680][20381] Updated weights for policy 1, policy_version 892161 (0.0007) [2023-12-27 11:49:57,733][20381] Updated weights for policy 1, policy_version 892171 (0.0007) [2023-12-27 11:49:57,773][20347] Updated weights for policy 0, policy_version 892850 (0.0010) [2023-12-27 11:49:57,780][20291] KL-divergence is very high: 172.3966 [2023-12-27 11:49:57,785][20381] Updated weights for policy 1, policy_version 892181 (0.0006) [2023-12-27 11:49:57,786][20291] KL-divergence is very high: 113.7396 [2023-12-27 11:49:57,826][20291] KL-divergence is very high: 227.2647 [2023-12-27 11:49:57,832][20347] Updated weights for policy 0, policy_version 892860 (0.0010) [2023-12-27 11:49:57,833][20291] KL-divergence is very high: 145.5818 [2023-12-27 11:49:57,879][20347] Updated weights for policy 0, policy_version 892870 (0.0008) [2023-12-27 11:49:57,903][20291] KL-divergence is very high: 122.0998 [2023-12-27 11:49:57,925][20347] Updated weights for policy 0, policy_version 892880 (0.0005) [2023-12-27 11:49:58,528][20381] Updated weights for policy 1, policy_version 892191 (0.0007) [2023-12-27 11:49:58,598][20381] Updated weights for policy 1, policy_version 892201 (0.0008) [2023-12-27 11:49:58,640][20347] Updated weights for policy 0, policy_version 892890 (0.0010) [2023-12-27 11:49:58,657][20381] Updated weights for policy 1, policy_version 892211 (0.0007) [2023-12-27 11:49:58,703][20347] Updated weights for policy 0, policy_version 892900 (0.0009) [2023-12-27 11:49:58,771][20347] Updated weights for policy 0, policy_version 892910 (0.0008) [2023-12-27 11:49:59,482][20381] Updated weights for policy 1, policy_version 892221 (0.0008) [2023-12-27 11:49:59,509][20347] Updated weights for policy 0, policy_version 892920 (0.0006) [2023-12-27 11:49:59,545][20381] Updated weights for policy 1, policy_version 892231 (0.0008) [2023-12-27 11:49:59,567][20347] Updated weights for policy 0, policy_version 892930 (0.0006) [2023-12-27 11:49:59,601][20381] Updated weights for policy 1, policy_version 892241 (0.0007) [2023-12-27 11:49:59,634][20347] Updated weights for policy 0, policy_version 892940 (0.0007) [2023-12-27 11:50:00,273][20347] Updated weights for policy 0, policy_version 892950 (0.0009) [2023-12-27 11:50:00,302][20381] Updated weights for policy 1, policy_version 892251 (0.0007) [2023-12-27 11:50:00,332][20347] Updated weights for policy 0, policy_version 892960 (0.0011) [2023-12-27 11:50:00,363][20381] Updated weights for policy 1, policy_version 892261 (0.0009) [2023-12-27 11:50:00,394][20347] Updated weights for policy 0, policy_version 892970 (0.0010) [2023-12-27 11:50:00,422][20381] Updated weights for policy 1, policy_version 892271 (0.0005) [2023-12-27 11:50:00,979][20347] Updated weights for policy 0, policy_version 892980 (0.0009) [2023-12-27 11:50:00,988][20381] Updated weights for policy 1, policy_version 892281 (0.0006) [2023-12-27 11:50:01,030][20347] Updated weights for policy 0, policy_version 892990 (0.0010) [2023-12-27 11:50:01,045][20381] Updated weights for policy 1, policy_version 892291 (0.0006) [2023-12-27 11:50:01,092][20347] Updated weights for policy 0, policy_version 893000 (0.0010) [2023-12-27 11:50:01,101][20381] Updated weights for policy 1, policy_version 892301 (0.0006) [2023-12-27 11:50:01,163][20381] Updated weights for policy 1, policy_version 892311 (0.0006) [2023-12-27 11:50:01,735][20347] Updated weights for policy 0, policy_version 893010 (0.0010) [2023-12-27 11:50:01,799][20347] Updated weights for policy 0, policy_version 893020 (0.0006) [2023-12-27 11:50:01,839][20381] Updated weights for policy 1, policy_version 892321 (0.0008) [2023-12-27 11:50:01,857][20347] Updated weights for policy 0, policy_version 893030 (0.0007) [2023-12-27 11:50:01,902][20381] Updated weights for policy 1, policy_version 892331 (0.0007) [2023-12-27 11:50:01,914][20347] Updated weights for policy 0, policy_version 893040 (0.0007) [2023-12-27 11:50:01,957][20381] Updated weights for policy 1, policy_version 892341 (0.0008) [2023-12-27 11:50:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 19827.4). Total num frames: 457113600. Throughput: 0: 10373.7, 1: 10160.7. Samples: 457123580. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:50:02,363][19170] Avg episode reward: [(0, '581.510'), (1, '1000.000')] [2023-12-27 11:50:02,589][20381] Updated weights for policy 1, policy_version 892351 (0.0007) [2023-12-27 11:50:02,627][20347] Updated weights for policy 0, policy_version 893050 (0.0006) [2023-12-27 11:50:02,644][20381] Updated weights for policy 1, policy_version 892361 (0.0006) [2023-12-27 11:50:02,658][20291] KL-divergence is very high: 153.4987 [2023-12-27 11:50:02,676][20291] KL-divergence is very high: 171.8086 [2023-12-27 11:50:02,681][20347] Updated weights for policy 0, policy_version 893060 (0.0007) [2023-12-27 11:50:02,682][20291] KL-divergence is very high: 198.5969 [2023-12-27 11:50:02,687][20291] KL-divergence is very high: 118.8523 [2023-12-27 11:50:02,698][20381] Updated weights for policy 1, policy_version 892371 (0.0007) [2023-12-27 11:50:02,734][20347] Updated weights for policy 0, policy_version 893070 (0.0006) [2023-12-27 11:50:03,354][20381] Updated weights for policy 1, policy_version 892382 (0.0007) [2023-12-27 11:50:03,405][20381] Updated weights for policy 1, policy_version 892392 (0.0005) [2023-12-27 11:50:03,436][20347] Updated weights for policy 0, policy_version 893080 (0.0006) [2023-12-27 11:50:03,451][20381] Updated weights for policy 1, policy_version 892402 (0.0006) [2023-12-27 11:50:03,490][20347] Updated weights for policy 0, policy_version 893090 (0.0005) [2023-12-27 11:50:03,536][20347] Updated weights for policy 0, policy_version 893100 (0.0005) [2023-12-27 11:50:04,051][20381] Updated weights for policy 1, policy_version 892412 (0.0006) [2023-12-27 11:50:04,107][20381] Updated weights for policy 1, policy_version 892422 (0.0006) [2023-12-27 11:50:04,165][20347] Updated weights for policy 0, policy_version 893110 (0.0008) [2023-12-27 11:50:04,167][20381] Updated weights for policy 1, policy_version 892432 (0.0008) [2023-12-27 11:50:04,226][20347] Updated weights for policy 0, policy_version 893120 (0.0010) [2023-12-27 11:50:04,289][20347] Updated weights for policy 0, policy_version 893130 (0.0009) [2023-12-27 11:50:04,838][20381] Updated weights for policy 1, policy_version 892442 (0.0008) [2023-12-27 11:50:04,898][20381] Updated weights for policy 1, policy_version 892452 (0.0010) [2023-12-27 11:50:04,937][20347] Updated weights for policy 0, policy_version 893140 (0.0006) [2023-12-27 11:50:04,957][20381] Updated weights for policy 1, policy_version 892462 (0.0010) [2023-12-27 11:50:04,986][20347] Updated weights for policy 0, policy_version 893150 (0.0010) [2023-12-27 11:50:05,012][20381] Updated weights for policy 1, policy_version 892472 (0.0010) [2023-12-27 11:50:05,036][20347] Updated weights for policy 0, policy_version 893160 (0.0010) [2023-12-27 11:50:05,694][20381] Updated weights for policy 1, policy_version 892482 (0.0011) [2023-12-27 11:50:05,723][20347] Updated weights for policy 0, policy_version 893170 (0.0010) [2023-12-27 11:50:05,758][20381] Updated weights for policy 1, policy_version 892492 (0.0010) [2023-12-27 11:50:05,782][20347] Updated weights for policy 0, policy_version 893180 (0.0010) [2023-12-27 11:50:05,825][20381] Updated weights for policy 1, policy_version 892502 (0.0010) [2023-12-27 11:50:05,833][20347] Updated weights for policy 0, policy_version 893190 (0.0008) [2023-12-27 11:50:05,885][20347] Updated weights for policy 0, policy_version 893200 (0.0005) [2023-12-27 11:50:06,513][20381] Updated weights for policy 1, policy_version 892512 (0.0010) [2023-12-27 11:50:06,515][20347] Updated weights for policy 0, policy_version 893210 (0.0009) [2023-12-27 11:50:06,566][20381] Updated weights for policy 1, policy_version 892522 (0.0011) [2023-12-27 11:50:06,573][20347] Updated weights for policy 0, policy_version 893220 (0.0008) [2023-12-27 11:50:06,629][20381] Updated weights for policy 1, policy_version 892532 (0.0011) [2023-12-27 11:50:06,630][20347] Updated weights for policy 0, policy_version 893230 (0.0008) [2023-12-27 11:50:07,317][20347] Updated weights for policy 0, policy_version 893240 (0.0010) [2023-12-27 11:50:07,325][20381] Updated weights for policy 1, policy_version 892542 (0.0011) [2023-12-27 11:50:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 19827.4). Total num frames: 457211904. Throughput: 0: 10369.4, 1: 10211.6. Samples: 457186384. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:50:07,363][19170] Avg episode reward: [(0, '625.740'), (1, '1000.000')] [2023-12-27 11:50:07,371][20347] Updated weights for policy 0, policy_version 893250 (0.0010) [2023-12-27 11:50:07,380][20381] Updated weights for policy 1, policy_version 892552 (0.0010) [2023-12-27 11:50:07,426][20347] Updated weights for policy 0, policy_version 893260 (0.0010) [2023-12-27 11:50:07,437][20381] Updated weights for policy 1, policy_version 892562 (0.0011) [2023-12-27 11:50:07,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000893264_228704256.pth... [2023-12-27 11:50:07,454][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000892048_228392960.pth [2023-12-27 11:50:07,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000892568_228524032.pth... [2023-12-27 11:50:07,476][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000891352_228212736.pth [2023-12-27 11:50:08,048][20381] Updated weights for policy 1, policy_version 892572 (0.0011) [2023-12-27 11:50:08,103][20381] Updated weights for policy 1, policy_version 892582 (0.0010) [2023-12-27 11:50:08,112][20347] Updated weights for policy 0, policy_version 893270 (0.0010) [2023-12-27 11:50:08,149][20381] Updated weights for policy 1, policy_version 892592 (0.0011) [2023-12-27 11:50:08,164][20347] Updated weights for policy 0, policy_version 893280 (0.0010) [2023-12-27 11:50:08,216][20347] Updated weights for policy 0, policy_version 893290 (0.0010) [2023-12-27 11:50:08,815][20381] Updated weights for policy 1, policy_version 892602 (0.0010) [2023-12-27 11:50:08,881][20381] Updated weights for policy 1, policy_version 892612 (0.0008) [2023-12-27 11:50:08,909][20347] Updated weights for policy 0, policy_version 893300 (0.0010) [2023-12-27 11:50:08,940][20381] Updated weights for policy 1, policy_version 892622 (0.0005) [2023-12-27 11:50:08,959][20347] Updated weights for policy 0, policy_version 893310 (0.0010) [2023-12-27 11:50:08,999][20381] Updated weights for policy 1, policy_version 892632 (0.0006) [2023-12-27 11:50:09,006][20347] Updated weights for policy 0, policy_version 893320 (0.0010) [2023-12-27 11:50:09,640][20381] Updated weights for policy 1, policy_version 892642 (0.0006) [2023-12-27 11:50:09,701][20381] Updated weights for policy 1, policy_version 892652 (0.0008) [2023-12-27 11:50:09,759][20381] Updated weights for policy 1, policy_version 892662 (0.0008) [2023-12-27 11:50:09,814][20347] Updated weights for policy 0, policy_version 893330 (0.0010) [2023-12-27 11:50:09,878][20347] Updated weights for policy 0, policy_version 893340 (0.0007) [2023-12-27 11:50:09,940][20347] Updated weights for policy 0, policy_version 893350 (0.0008) [2023-12-27 11:50:10,001][20347] Updated weights for policy 0, policy_version 893360 (0.0009) [2023-12-27 11:50:10,454][20381] Updated weights for policy 1, policy_version 892672 (0.0008) [2023-12-27 11:50:10,510][20381] Updated weights for policy 1, policy_version 892682 (0.0008) [2023-12-27 11:50:10,570][20381] Updated weights for policy 1, policy_version 892692 (0.0008) [2023-12-27 11:50:10,696][20347] Updated weights for policy 0, policy_version 893370 (0.0008) [2023-12-27 11:50:10,757][20347] Updated weights for policy 0, policy_version 893380 (0.0008) [2023-12-27 11:50:10,809][20347] Updated weights for policy 0, policy_version 893390 (0.0007) [2023-12-27 11:50:11,281][20381] Updated weights for policy 1, policy_version 892702 (0.0010) [2023-12-27 11:50:11,337][20381] Updated weights for policy 1, policy_version 892712 (0.0011) [2023-12-27 11:50:11,405][20381] Updated weights for policy 1, policy_version 892722 (0.0007) [2023-12-27 11:50:11,563][20347] Updated weights for policy 0, policy_version 893400 (0.0008) [2023-12-27 11:50:11,629][20347] Updated weights for policy 0, policy_version 893410 (0.0008) [2023-12-27 11:50:11,705][20347] Updated weights for policy 0, policy_version 893420 (0.0011) [2023-12-27 11:50:12,131][20381] Updated weights for policy 1, policy_version 892732 (0.0008) [2023-12-27 11:50:12,181][20381] Updated weights for policy 1, policy_version 892742 (0.0011) [2023-12-27 11:50:12,227][20381] Updated weights for policy 1, policy_version 892752 (0.0010) [2023-12-27 11:50:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 19855.2). Total num frames: 457318400. Throughput: 0: 10380.7, 1: 10183.7. Samples: 457308396. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:50:12,363][19170] Avg episode reward: [(0, '686.380'), (1, '1000.000')] [2023-12-27 11:50:12,405][20347] Updated weights for policy 0, policy_version 893430 (0.0008) [2023-12-27 11:50:12,468][20347] Updated weights for policy 0, policy_version 893440 (0.0005) [2023-12-27 11:50:12,522][20347] Updated weights for policy 0, policy_version 893450 (0.0005) [2023-12-27 11:50:12,923][20381] Updated weights for policy 1, policy_version 892762 (0.0009) [2023-12-27 11:50:12,985][20381] Updated weights for policy 1, policy_version 892772 (0.0011) [2023-12-27 11:50:13,048][20381] Updated weights for policy 1, policy_version 892782 (0.0007) [2023-12-27 11:50:13,097][20381] Updated weights for policy 1, policy_version 892792 (0.0005) [2023-12-27 11:50:13,156][20347] Updated weights for policy 0, policy_version 893460 (0.0005) [2023-12-27 11:50:13,207][20347] Updated weights for policy 0, policy_version 893470 (0.0005) [2023-12-27 11:50:13,257][20347] Updated weights for policy 0, policy_version 893480 (0.0005) [2023-12-27 11:50:13,683][20381] Updated weights for policy 1, policy_version 892802 (0.0007) [2023-12-27 11:50:13,745][20381] Updated weights for policy 1, policy_version 892812 (0.0006) [2023-12-27 11:50:13,797][20381] Updated weights for policy 1, policy_version 892822 (0.0007) [2023-12-27 11:50:13,881][20347] Updated weights for policy 0, policy_version 893490 (0.0005) [2023-12-27 11:50:13,926][20347] Updated weights for policy 0, policy_version 893500 (0.0008) [2023-12-27 11:50:13,978][20347] Updated weights for policy 0, policy_version 893510 (0.0007) [2023-12-27 11:50:14,032][20347] Updated weights for policy 0, policy_version 893520 (0.0006) [2023-12-27 11:50:14,471][20381] Updated weights for policy 1, policy_version 892832 (0.0006) [2023-12-27 11:50:14,519][20381] Updated weights for policy 1, policy_version 892842 (0.0005) [2023-12-27 11:50:14,575][20381] Updated weights for policy 1, policy_version 892852 (0.0005) [2023-12-27 11:50:14,677][20347] Updated weights for policy 0, policy_version 893530 (0.0008) [2023-12-27 11:50:14,728][20347] Updated weights for policy 0, policy_version 893540 (0.0007) [2023-12-27 11:50:14,794][20347] Updated weights for policy 0, policy_version 893550 (0.0008) [2023-12-27 11:50:15,205][20381] Updated weights for policy 1, policy_version 892862 (0.0007) [2023-12-27 11:50:15,263][20381] Updated weights for policy 1, policy_version 892872 (0.0008) [2023-12-27 11:50:15,320][20381] Updated weights for policy 1, policy_version 892882 (0.0010) [2023-12-27 11:50:15,450][20347] Updated weights for policy 0, policy_version 893560 (0.0007) [2023-12-27 11:50:15,511][20347] Updated weights for policy 0, policy_version 893570 (0.0007) [2023-12-27 11:50:15,572][20347] Updated weights for policy 0, policy_version 893580 (0.0007) [2023-12-27 11:50:16,041][20381] Updated weights for policy 1, policy_version 892892 (0.0009) [2023-12-27 11:50:16,097][20381] Updated weights for policy 1, policy_version 892902 (0.0008) [2023-12-27 11:50:16,155][20381] Updated weights for policy 1, policy_version 892912 (0.0010) [2023-12-27 11:50:16,245][20347] Updated weights for policy 0, policy_version 893590 (0.0007) [2023-12-27 11:50:16,295][20347] Updated weights for policy 0, policy_version 893600 (0.0007) [2023-12-27 11:50:16,343][20347] Updated weights for policy 0, policy_version 893610 (0.0007) [2023-12-27 11:50:16,841][20381] Updated weights for policy 1, policy_version 892922 (0.0011) [2023-12-27 11:50:16,892][20381] Updated weights for policy 1, policy_version 892932 (0.0010) [2023-12-27 11:50:16,949][20381] Updated weights for policy 1, policy_version 892942 (0.0010) [2023-12-27 11:50:17,008][20381] Updated weights for policy 1, policy_version 892952 (0.0008) [2023-12-27 11:50:17,060][20347] Updated weights for policy 0, policy_version 893620 (0.0008) [2023-12-27 11:50:17,115][20347] Updated weights for policy 0, policy_version 893630 (0.0008) [2023-12-27 11:50:17,168][20347] Updated weights for policy 0, policy_version 893640 (0.0007) [2023-12-27 11:50:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 19855.2). Total num frames: 457424896. Throughput: 0: 10314.0, 1: 10235.4. Samples: 457431724. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:50:17,363][19170] Avg episode reward: [(0, '642.820'), (1, '1000.000')] [2023-12-27 11:50:17,660][20381] Updated weights for policy 1, policy_version 892962 (0.0006) [2023-12-27 11:50:17,720][20381] Updated weights for policy 1, policy_version 892972 (0.0008) [2023-12-27 11:50:17,776][20381] Updated weights for policy 1, policy_version 892982 (0.0007) [2023-12-27 11:50:17,894][20347] Updated weights for policy 0, policy_version 893650 (0.0008) [2023-12-27 11:50:17,948][20347] Updated weights for policy 0, policy_version 893661 (0.0008) [2023-12-27 11:50:17,996][20347] Updated weights for policy 0, policy_version 893671 (0.0007) [2023-12-27 11:50:18,351][20381] Updated weights for policy 1, policy_version 892992 (0.0010) [2023-12-27 11:50:18,412][20381] Updated weights for policy 1, policy_version 893002 (0.0007) [2023-12-27 11:50:18,471][20381] Updated weights for policy 1, policy_version 893012 (0.0005) [2023-12-27 11:50:18,688][20347] Updated weights for policy 0, policy_version 893681 (0.0006) [2023-12-27 11:50:18,750][20347] Updated weights for policy 0, policy_version 893691 (0.0007) [2023-12-27 11:50:18,809][20347] Updated weights for policy 0, policy_version 893701 (0.0007) [2023-12-27 11:50:18,868][20347] Updated weights for policy 0, policy_version 893711 (0.0008) [2023-12-27 11:50:19,126][20381] Updated weights for policy 1, policy_version 893022 (0.0009) [2023-12-27 11:50:19,176][20381] Updated weights for policy 1, policy_version 893032 (0.0011) [2023-12-27 11:50:19,247][20381] Updated weights for policy 1, policy_version 893042 (0.0010) [2023-12-27 11:50:19,574][20347] Updated weights for policy 0, policy_version 893721 (0.0008) [2023-12-27 11:50:19,632][20347] Updated weights for policy 0, policy_version 893731 (0.0007) [2023-12-27 11:50:19,687][20347] Updated weights for policy 0, policy_version 893741 (0.0007) [2023-12-27 11:50:20,037][20381] Updated weights for policy 1, policy_version 893052 (0.0010) [2023-12-27 11:50:20,098][20381] Updated weights for policy 1, policy_version 893062 (0.0011) [2023-12-27 11:50:20,156][20381] Updated weights for policy 1, policy_version 893072 (0.0011) [2023-12-27 11:50:20,386][20347] Updated weights for policy 0, policy_version 893751 (0.0006) [2023-12-27 11:50:20,457][20347] Updated weights for policy 0, policy_version 893761 (0.0007) [2023-12-27 11:50:20,524][20347] Updated weights for policy 0, policy_version 893771 (0.0008) [2023-12-27 11:50:20,881][20381] Updated weights for policy 1, policy_version 893082 (0.0011) [2023-12-27 11:50:20,943][20381] Updated weights for policy 1, policy_version 893092 (0.0011) [2023-12-27 11:50:20,997][20381] Updated weights for policy 1, policy_version 893102 (0.0011) [2023-12-27 11:50:21,056][20381] Updated weights for policy 1, policy_version 893112 (0.0009) [2023-12-27 11:50:21,234][20347] Updated weights for policy 0, policy_version 893781 (0.0009) [2023-12-27 11:50:21,304][20347] Updated weights for policy 0, policy_version 893791 (0.0008) [2023-12-27 11:50:21,380][20347] Updated weights for policy 0, policy_version 893801 (0.0009) [2023-12-27 11:50:21,886][20381] Updated weights for policy 1, policy_version 893122 (0.0007) [2023-12-27 11:50:21,946][20381] Updated weights for policy 1, policy_version 893132 (0.0008) [2023-12-27 11:50:22,006][20381] Updated weights for policy 1, policy_version 893142 (0.0007) [2023-12-27 11:50:22,075][20347] Updated weights for policy 0, policy_version 893811 (0.0006) [2023-12-27 11:50:22,142][20347] Updated weights for policy 0, policy_version 893821 (0.0008) [2023-12-27 11:50:22,211][20347] Updated weights for policy 0, policy_version 893831 (0.0008) [2023-12-27 11:50:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 19855.2). Total num frames: 457523200. Throughput: 0: 10245.2, 1: 10235.1. Samples: 457491724. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-12-27 11:50:22,363][19170] Avg episode reward: [(0, '558.980'), (1, '1000.000')] [2023-12-27 11:50:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000893144_228671488.pth... [2023-12-27 11:50:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000893840_228851712.pth... [2023-12-27 11:50:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000891960_228368384.pth [2023-12-27 11:50:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000892656_228548608.pth [2023-12-27 11:50:22,770][20381] Updated weights for policy 1, policy_version 893152 (0.0007) [2023-12-27 11:50:22,827][20381] Updated weights for policy 1, policy_version 893162 (0.0008) [2023-12-27 11:50:22,887][20381] Updated weights for policy 1, policy_version 893172 (0.0007) [2023-12-27 11:50:22,935][20347] Updated weights for policy 0, policy_version 893841 (0.0009) [2023-12-27 11:50:22,994][20347] Updated weights for policy 0, policy_version 893851 (0.0007) [2023-12-27 11:50:23,046][20347] Updated weights for policy 0, policy_version 893861 (0.0009) [2023-12-27 11:50:23,106][20347] Updated weights for policy 0, policy_version 893871 (0.0007) [2023-12-27 11:50:23,540][20381] Updated weights for policy 1, policy_version 893182 (0.0008) [2023-12-27 11:50:23,593][20381] Updated weights for policy 1, policy_version 893192 (0.0009) [2023-12-27 11:50:23,659][20381] Updated weights for policy 1, policy_version 893202 (0.0007) [2023-12-27 11:50:23,878][20347] Updated weights for policy 0, policy_version 893881 (0.0006) [2023-12-27 11:50:23,935][20347] Updated weights for policy 0, policy_version 893891 (0.0008) [2023-12-27 11:50:23,992][20347] Updated weights for policy 0, policy_version 893901 (0.0007) [2023-12-27 11:50:24,388][20381] Updated weights for policy 1, policy_version 893212 (0.0007) [2023-12-27 11:50:24,446][20381] Updated weights for policy 1, policy_version 893222 (0.0008) [2023-12-27 11:50:24,502][20381] Updated weights for policy 1, policy_version 893232 (0.0007) [2023-12-27 11:50:24,620][20347] Updated weights for policy 0, policy_version 893911 (0.0009) [2023-12-27 11:50:24,678][20347] Updated weights for policy 0, policy_version 893921 (0.0010) [2023-12-27 11:50:24,733][20347] Updated weights for policy 0, policy_version 893931 (0.0010) [2023-12-27 11:50:25,223][20381] Updated weights for policy 1, policy_version 893242 (0.0007) [2023-12-27 11:50:25,267][20381] Updated weights for policy 1, policy_version 893252 (0.0007) [2023-12-27 11:50:25,323][20381] Updated weights for policy 1, policy_version 893262 (0.0008) [2023-12-27 11:50:25,382][20381] Updated weights for policy 1, policy_version 893272 (0.0008) [2023-12-27 11:50:25,390][20347] Updated weights for policy 0, policy_version 893941 (0.0010) [2023-12-27 11:50:25,451][20347] Updated weights for policy 0, policy_version 893951 (0.0010) [2023-12-27 11:50:25,509][20347] Updated weights for policy 0, policy_version 893961 (0.0010) [2023-12-27 11:50:25,994][20381] Updated weights for policy 1, policy_version 893282 (0.0008) [2023-12-27 11:50:26,037][20381] Updated weights for policy 1, policy_version 893292 (0.0005) [2023-12-27 11:50:26,083][20381] Updated weights for policy 1, policy_version 893302 (0.0005) [2023-12-27 11:50:26,170][20347] Updated weights for policy 0, policy_version 893971 (0.0009) [2023-12-27 11:50:26,219][20347] Updated weights for policy 0, policy_version 893981 (0.0005) [2023-12-27 11:50:26,282][20347] Updated weights for policy 0, policy_version 893991 (0.0005) [2023-12-27 11:50:26,764][20381] Updated weights for policy 1, policy_version 893312 (0.0009) [2023-12-27 11:50:26,815][20381] Updated weights for policy 1, policy_version 893322 (0.0010) [2023-12-27 11:50:26,870][20381] Updated weights for policy 1, policy_version 893332 (0.0005) [2023-12-27 11:50:26,872][20347] Updated weights for policy 0, policy_version 894001 (0.0005) [2023-12-27 11:50:26,928][20347] Updated weights for policy 0, policy_version 894011 (0.0009) [2023-12-27 11:50:26,985][20347] Updated weights for policy 0, policy_version 894021 (0.0010) [2023-12-27 11:50:27,050][20347] Updated weights for policy 0, policy_version 894031 (0.0010) [2023-12-27 11:50:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 19855.2). Total num frames: 457621504. Throughput: 0: 10182.0, 1: 10211.7. Samples: 457610100. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:27,363][19170] Avg episode reward: [(0, '638.490'), (1, '1000.000')] [2023-12-27 11:50:27,426][20381] Updated weights for policy 1, policy_version 893342 (0.0007) [2023-12-27 11:50:27,489][20381] Updated weights for policy 1, policy_version 893352 (0.0007) [2023-12-27 11:50:27,543][20381] Updated weights for policy 1, policy_version 893362 (0.0007) [2023-12-27 11:50:27,711][20347] Updated weights for policy 0, policy_version 894041 (0.0010) [2023-12-27 11:50:27,768][20347] Updated weights for policy 0, policy_version 894051 (0.0008) [2023-12-27 11:50:27,827][20347] Updated weights for policy 0, policy_version 894061 (0.0009) [2023-12-27 11:50:28,149][20381] Updated weights for policy 1, policy_version 893372 (0.0005) [2023-12-27 11:50:28,213][20381] Updated weights for policy 1, policy_version 893382 (0.0007) [2023-12-27 11:50:28,267][20381] Updated weights for policy 1, policy_version 893392 (0.0005) [2023-12-27 11:50:28,419][20347] Updated weights for policy 0, policy_version 894071 (0.0007) [2023-12-27 11:50:28,477][20347] Updated weights for policy 0, policy_version 894081 (0.0010) [2023-12-27 11:50:28,532][20347] Updated weights for policy 0, policy_version 894091 (0.0010) [2023-12-27 11:50:28,835][20381] Updated weights for policy 1, policy_version 893402 (0.0007) [2023-12-27 11:50:28,891][20381] Updated weights for policy 1, policy_version 893412 (0.0009) [2023-12-27 11:50:28,947][20381] Updated weights for policy 1, policy_version 893422 (0.0006) [2023-12-27 11:50:28,994][20381] Updated weights for policy 1, policy_version 893432 (0.0008) [2023-12-27 11:50:29,206][20347] Updated weights for policy 0, policy_version 894101 (0.0010) [2023-12-27 11:50:29,269][20347] Updated weights for policy 0, policy_version 894111 (0.0007) [2023-12-27 11:50:29,333][20347] Updated weights for policy 0, policy_version 894121 (0.0007) [2023-12-27 11:50:29,641][20381] Updated weights for policy 1, policy_version 893442 (0.0008) [2023-12-27 11:50:29,702][20381] Updated weights for policy 1, policy_version 893452 (0.0008) [2023-12-27 11:50:29,766][20381] Updated weights for policy 1, policy_version 893462 (0.0008) [2023-12-27 11:50:30,005][20347] Updated weights for policy 0, policy_version 894131 (0.0008) [2023-12-27 11:50:30,062][20347] Updated weights for policy 0, policy_version 894141 (0.0008) [2023-12-27 11:50:30,110][20347] Updated weights for policy 0, policy_version 894151 (0.0009) [2023-12-27 11:50:30,347][20381] Updated weights for policy 1, policy_version 893472 (0.0006) [2023-12-27 11:50:30,400][20381] Updated weights for policy 1, policy_version 893482 (0.0006) [2023-12-27 11:50:30,461][20381] Updated weights for policy 1, policy_version 893492 (0.0008) [2023-12-27 11:50:30,777][20347] Updated weights for policy 0, policy_version 894161 (0.0008) [2023-12-27 11:50:30,838][20347] Updated weights for policy 0, policy_version 894171 (0.0006) [2023-12-27 11:50:30,895][20347] Updated weights for policy 0, policy_version 894181 (0.0010) [2023-12-27 11:50:30,946][20347] Updated weights for policy 0, policy_version 894191 (0.0010) [2023-12-27 11:50:31,063][20381] Updated weights for policy 1, policy_version 893502 (0.0007) [2023-12-27 11:50:31,120][20381] Updated weights for policy 1, policy_version 893512 (0.0008) [2023-12-27 11:50:31,181][20381] Updated weights for policy 1, policy_version 893522 (0.0008) [2023-12-27 11:50:31,626][20347] Updated weights for policy 0, policy_version 894201 (0.0007) [2023-12-27 11:50:31,686][20347] Updated weights for policy 0, policy_version 894211 (0.0006) [2023-12-27 11:50:31,744][20347] Updated weights for policy 0, policy_version 894221 (0.0006) [2023-12-27 11:50:31,931][20381] Updated weights for policy 1, policy_version 893532 (0.0007) [2023-12-27 11:50:31,980][20381] Updated weights for policy 1, policy_version 893542 (0.0007) [2023-12-27 11:50:32,032][20381] Updated weights for policy 1, policy_version 893552 (0.0009) [2023-12-27 11:50:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 19855.2). Total num frames: 457728000. Throughput: 0: 10209.3, 1: 10245.4. Samples: 457739828. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:32,363][19170] Avg episode reward: [(0, '674.780'), (1, '1000.000')] [2023-12-27 11:50:32,365][20347] Updated weights for policy 0, policy_version 894231 (0.0007) [2023-12-27 11:50:32,420][20347] Updated weights for policy 0, policy_version 894241 (0.0008) [2023-12-27 11:50:32,475][20347] Updated weights for policy 0, policy_version 894251 (0.0008) [2023-12-27 11:50:32,664][20381] Updated weights for policy 1, policy_version 893562 (0.0006) [2023-12-27 11:50:32,719][20381] Updated weights for policy 1, policy_version 893572 (0.0008) [2023-12-27 11:50:32,778][20381] Updated weights for policy 1, policy_version 893582 (0.0008) [2023-12-27 11:50:32,842][20381] Updated weights for policy 1, policy_version 893592 (0.0008) [2023-12-27 11:50:33,133][20347] Updated weights for policy 0, policy_version 894261 (0.0007) [2023-12-27 11:50:33,193][20347] Updated weights for policy 0, policy_version 894271 (0.0008) [2023-12-27 11:50:33,238][20347] Updated weights for policy 0, policy_version 894281 (0.0009) [2023-12-27 11:50:33,468][20381] Updated weights for policy 1, policy_version 893602 (0.0007) [2023-12-27 11:50:33,522][20381] Updated weights for policy 1, policy_version 893612 (0.0007) [2023-12-27 11:50:33,579][20381] Updated weights for policy 1, policy_version 893622 (0.0007) [2023-12-27 11:50:33,888][20347] Updated weights for policy 0, policy_version 894291 (0.0007) [2023-12-27 11:50:33,943][20347] Updated weights for policy 0, policy_version 894301 (0.0010) [2023-12-27 11:50:33,994][20347] Updated weights for policy 0, policy_version 894311 (0.0010) [2023-12-27 11:50:34,279][20381] Updated weights for policy 1, policy_version 893632 (0.0007) [2023-12-27 11:50:34,344][20381] Updated weights for policy 1, policy_version 893642 (0.0007) [2023-12-27 11:50:34,401][20381] Updated weights for policy 1, policy_version 893652 (0.0008) [2023-12-27 11:50:34,650][20347] Updated weights for policy 0, policy_version 894321 (0.0010) [2023-12-27 11:50:34,699][20347] Updated weights for policy 0, policy_version 894331 (0.0010) [2023-12-27 11:50:34,744][20347] Updated weights for policy 0, policy_version 894341 (0.0010) [2023-12-27 11:50:34,790][20347] Updated weights for policy 0, policy_version 894351 (0.0010) [2023-12-27 11:50:35,087][20381] Updated weights for policy 1, policy_version 893662 (0.0008) [2023-12-27 11:50:35,136][20381] Updated weights for policy 1, policy_version 893672 (0.0010) [2023-12-27 11:50:35,183][20381] Updated weights for policy 1, policy_version 893682 (0.0010) [2023-12-27 11:50:35,419][20347] Updated weights for policy 0, policy_version 894361 (0.0007) [2023-12-27 11:50:35,472][20347] Updated weights for policy 0, policy_version 894371 (0.0008) [2023-12-27 11:50:35,526][20347] Updated weights for policy 0, policy_version 894381 (0.0008) [2023-12-27 11:50:35,825][20381] Updated weights for policy 1, policy_version 893692 (0.0010) [2023-12-27 11:50:35,880][20381] Updated weights for policy 1, policy_version 893702 (0.0010) [2023-12-27 11:50:35,932][20381] Updated weights for policy 1, policy_version 893712 (0.0010) [2023-12-27 11:50:36,220][20347] Updated weights for policy 0, policy_version 894391 (0.0008) [2023-12-27 11:50:36,285][20347] Updated weights for policy 0, policy_version 894401 (0.0009) [2023-12-27 11:50:36,350][20347] Updated weights for policy 0, policy_version 894411 (0.0007) [2023-12-27 11:50:36,656][20381] Updated weights for policy 1, policy_version 893722 (0.0010) [2023-12-27 11:50:36,719][20381] Updated weights for policy 1, policy_version 893732 (0.0008) [2023-12-27 11:50:36,774][20381] Updated weights for policy 1, policy_version 893742 (0.0008) [2023-12-27 11:50:36,833][20381] Updated weights for policy 1, policy_version 893752 (0.0006) [2023-12-27 11:50:37,008][20347] Updated weights for policy 0, policy_version 894421 (0.0008) [2023-12-27 11:50:37,068][20347] Updated weights for policy 0, policy_version 894431 (0.0008) [2023-12-27 11:50:37,118][20347] Updated weights for policy 0, policy_version 894441 (0.0007) [2023-12-27 11:50:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 19883.0). Total num frames: 457834496. Throughput: 0: 10245.3, 1: 10250.0. Samples: 457802304. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:37,363][19170] Avg episode reward: [(0, '665.710'), (1, '1000.000')] [2023-12-27 11:50:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000893752_228827136.pth... [2023-12-27 11:50:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000894448_229007360.pth... [2023-12-27 11:50:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000892568_228524032.pth [2023-12-27 11:50:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000893264_228704256.pth [2023-12-27 11:50:37,505][20381] Updated weights for policy 1, policy_version 893762 (0.0006) [2023-12-27 11:50:37,562][20381] Updated weights for policy 1, policy_version 893772 (0.0008) [2023-12-27 11:50:37,612][20381] Updated weights for policy 1, policy_version 893782 (0.0007) [2023-12-27 11:50:37,789][20347] Updated weights for policy 0, policy_version 894451 (0.0007) [2023-12-27 11:50:37,849][20347] Updated weights for policy 0, policy_version 894461 (0.0005) [2023-12-27 11:50:37,904][20347] Updated weights for policy 0, policy_version 894471 (0.0005) [2023-12-27 11:50:38,274][20381] Updated weights for policy 1, policy_version 893792 (0.0007) [2023-12-27 11:50:38,335][20381] Updated weights for policy 1, policy_version 893802 (0.0007) [2023-12-27 11:50:38,401][20381] Updated weights for policy 1, policy_version 893812 (0.0008) [2023-12-27 11:50:38,518][20347] Updated weights for policy 0, policy_version 894481 (0.0006) [2023-12-27 11:50:38,575][20347] Updated weights for policy 0, policy_version 894491 (0.0008) [2023-12-27 11:50:38,638][20347] Updated weights for policy 0, policy_version 894501 (0.0008) [2023-12-27 11:50:38,697][20347] Updated weights for policy 0, policy_version 894511 (0.0008) [2023-12-27 11:50:39,045][20381] Updated weights for policy 1, policy_version 893822 (0.0009) [2023-12-27 11:50:39,104][20381] Updated weights for policy 1, policy_version 893832 (0.0010) [2023-12-27 11:50:39,163][20381] Updated weights for policy 1, policy_version 893842 (0.0010) [2023-12-27 11:50:39,456][20347] Updated weights for policy 0, policy_version 894521 (0.0008) [2023-12-27 11:50:39,513][20347] Updated weights for policy 0, policy_version 894531 (0.0008) [2023-12-27 11:50:39,573][20347] Updated weights for policy 0, policy_version 894541 (0.0008) [2023-12-27 11:50:39,863][20381] Updated weights for policy 1, policy_version 893852 (0.0010) [2023-12-27 11:50:39,928][20381] Updated weights for policy 1, policy_version 893862 (0.0007) [2023-12-27 11:50:39,987][20381] Updated weights for policy 1, policy_version 893872 (0.0007) [2023-12-27 11:50:40,297][20347] Updated weights for policy 0, policy_version 894551 (0.0008) [2023-12-27 11:50:40,352][20347] Updated weights for policy 0, policy_version 894561 (0.0008) [2023-12-27 11:50:40,415][20347] Updated weights for policy 0, policy_version 894571 (0.0008) [2023-12-27 11:50:40,691][20381] Updated weights for policy 1, policy_version 893882 (0.0007) [2023-12-27 11:50:40,747][20381] Updated weights for policy 1, policy_version 893892 (0.0008) [2023-12-27 11:50:40,801][20381] Updated weights for policy 1, policy_version 893902 (0.0006) [2023-12-27 11:50:40,855][20381] Updated weights for policy 1, policy_version 893912 (0.0005) [2023-12-27 11:50:41,163][20347] Updated weights for policy 0, policy_version 894581 (0.0008) [2023-12-27 11:50:41,223][20347] Updated weights for policy 0, policy_version 894591 (0.0008) [2023-12-27 11:50:41,287][20347] Updated weights for policy 0, policy_version 894601 (0.0008) [2023-12-27 11:50:41,575][20381] Updated weights for policy 1, policy_version 893922 (0.0008) [2023-12-27 11:50:41,642][20381] Updated weights for policy 1, policy_version 893932 (0.0009) [2023-12-27 11:50:41,693][20381] Updated weights for policy 1, policy_version 893942 (0.0006) [2023-12-27 11:50:41,999][20347] Updated weights for policy 0, policy_version 894611 (0.0007) [2023-12-27 11:50:42,051][20347] Updated weights for policy 0, policy_version 894621 (0.0008) [2023-12-27 11:50:42,114][20347] Updated weights for policy 0, policy_version 894631 (0.0008) [2023-12-27 11:50:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 19883.0). Total num frames: 457932800. Throughput: 0: 10264.2, 1: 10253.7. Samples: 457923860. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:42,363][19170] Avg episode reward: [(0, '700.730'), (1, '1000.000')] [2023-12-27 11:50:42,433][20381] Updated weights for policy 1, policy_version 893952 (0.0008) [2023-12-27 11:50:42,493][20381] Updated weights for policy 1, policy_version 893962 (0.0008) [2023-12-27 11:50:42,560][20381] Updated weights for policy 1, policy_version 893972 (0.0008) [2023-12-27 11:50:42,874][20347] Updated weights for policy 0, policy_version 894641 (0.0008) [2023-12-27 11:50:42,929][20347] Updated weights for policy 0, policy_version 894651 (0.0007) [2023-12-27 11:50:42,985][20347] Updated weights for policy 0, policy_version 894661 (0.0008) [2023-12-27 11:50:43,044][20347] Updated weights for policy 0, policy_version 894671 (0.0007) [2023-12-27 11:50:43,150][20381] Updated weights for policy 1, policy_version 893982 (0.0006) [2023-12-27 11:50:43,205][20381] Updated weights for policy 1, policy_version 893992 (0.0005) [2023-12-27 11:50:43,249][20381] Updated weights for policy 1, policy_version 894002 (0.0005) [2023-12-27 11:50:43,773][20347] Updated weights for policy 0, policy_version 894681 (0.0010) [2023-12-27 11:50:43,818][20381] Updated weights for policy 1, policy_version 894012 (0.0005) [2023-12-27 11:50:43,831][20347] Updated weights for policy 0, policy_version 894691 (0.0009) [2023-12-27 11:50:43,873][20381] Updated weights for policy 1, policy_version 894022 (0.0006) [2023-12-27 11:50:43,892][20347] Updated weights for policy 0, policy_version 894701 (0.0008) [2023-12-27 11:50:43,932][20381] Updated weights for policy 1, policy_version 894032 (0.0006) [2023-12-27 11:50:44,554][20347] Updated weights for policy 0, policy_version 894711 (0.0007) [2023-12-27 11:50:44,554][20381] Updated weights for policy 1, policy_version 894042 (0.0008) [2023-12-27 11:50:44,610][20381] Updated weights for policy 1, policy_version 894052 (0.0008) [2023-12-27 11:50:44,612][20347] Updated weights for policy 0, policy_version 894721 (0.0009) [2023-12-27 11:50:44,655][20381] Updated weights for policy 1, policy_version 894062 (0.0005) [2023-12-27 11:50:44,659][20347] Updated weights for policy 0, policy_version 894731 (0.0009) [2023-12-27 11:50:44,699][20381] Updated weights for policy 1, policy_version 894072 (0.0006) [2023-12-27 11:50:45,313][20381] Updated weights for policy 1, policy_version 894082 (0.0008) [2023-12-27 11:50:45,325][20347] Updated weights for policy 0, policy_version 894741 (0.0006) [2023-12-27 11:50:45,372][20381] Updated weights for policy 1, policy_version 894092 (0.0009) [2023-12-27 11:50:45,383][20347] Updated weights for policy 0, policy_version 894751 (0.0009) [2023-12-27 11:50:45,430][20381] Updated weights for policy 1, policy_version 894102 (0.0007) [2023-12-27 11:50:45,441][20347] Updated weights for policy 0, policy_version 894761 (0.0008) [2023-12-27 11:50:45,999][20347] Updated weights for policy 0, policy_version 894771 (0.0009) [2023-12-27 11:50:46,054][20347] Updated weights for policy 0, policy_version 894781 (0.0010) [2023-12-27 11:50:46,113][20347] Updated weights for policy 0, policy_version 894791 (0.0010) [2023-12-27 11:50:46,120][20381] Updated weights for policy 1, policy_version 894112 (0.0009) [2023-12-27 11:50:46,174][20381] Updated weights for policy 1, policy_version 894122 (0.0007) [2023-12-27 11:50:46,228][20381] Updated weights for policy 1, policy_version 894132 (0.0007) [2023-12-27 11:50:46,771][20347] Updated weights for policy 0, policy_version 894801 (0.0009) [2023-12-27 11:50:46,829][20347] Updated weights for policy 0, policy_version 894811 (0.0005) [2023-12-27 11:50:46,891][20347] Updated weights for policy 0, policy_version 894821 (0.0005) [2023-12-27 11:50:46,910][20381] Updated weights for policy 1, policy_version 894142 (0.0007) [2023-12-27 11:50:46,945][20347] Updated weights for policy 0, policy_version 894831 (0.0005) [2023-12-27 11:50:46,967][20381] Updated weights for policy 1, policy_version 894152 (0.0008) [2023-12-27 11:50:47,019][20381] Updated weights for policy 1, policy_version 894162 (0.0008) [2023-12-27 11:50:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 19883.0). Total num frames: 458039296. Throughput: 0: 10206.8, 1: 10334.2. Samples: 458047920. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:47,363][19170] Avg episode reward: [(0, '758.110'), (1, '1000.000')] [2023-12-27 11:50:47,495][20347] Updated weights for policy 0, policy_version 894841 (0.0010) [2023-12-27 11:50:47,550][20347] Updated weights for policy 0, policy_version 894851 (0.0010) [2023-12-27 11:50:47,601][20347] Updated weights for policy 0, policy_version 894861 (0.0010) [2023-12-27 11:50:47,714][20381] Updated weights for policy 1, policy_version 894172 (0.0009) [2023-12-27 11:50:47,762][20381] Updated weights for policy 1, policy_version 894182 (0.0005) [2023-12-27 11:50:47,815][20381] Updated weights for policy 1, policy_version 894192 (0.0005) [2023-12-27 11:50:48,351][20347] Updated weights for policy 0, policy_version 894871 (0.0009) [2023-12-27 11:50:48,418][20347] Updated weights for policy 0, policy_version 894881 (0.0006) [2023-12-27 11:50:48,480][20347] Updated weights for policy 0, policy_version 894891 (0.0006) [2023-12-27 11:50:48,505][20381] Updated weights for policy 1, policy_version 894202 (0.0006) [2023-12-27 11:50:48,558][20381] Updated weights for policy 1, policy_version 894212 (0.0011) [2023-12-27 11:50:48,608][20381] Updated weights for policy 1, policy_version 894222 (0.0010) [2023-12-27 11:50:48,666][20381] Updated weights for policy 1, policy_version 894232 (0.0007) [2023-12-27 11:50:49,146][20347] Updated weights for policy 0, policy_version 894901 (0.0006) [2023-12-27 11:50:49,206][20347] Updated weights for policy 0, policy_version 894911 (0.0007) [2023-12-27 11:50:49,272][20347] Updated weights for policy 0, policy_version 894921 (0.0008) [2023-12-27 11:50:49,303][20381] Updated weights for policy 1, policy_version 894242 (0.0011) [2023-12-27 11:50:49,370][20381] Updated weights for policy 1, policy_version 894252 (0.0009) [2023-12-27 11:50:49,432][20381] Updated weights for policy 1, policy_version 894262 (0.0007) [2023-12-27 11:50:49,992][20347] Updated weights for policy 0, policy_version 894931 (0.0008) [2023-12-27 11:50:50,050][20347] Updated weights for policy 0, policy_version 894941 (0.0008) [2023-12-27 11:50:50,090][20381] Updated weights for policy 1, policy_version 894272 (0.0008) [2023-12-27 11:50:50,111][20347] Updated weights for policy 0, policy_version 894951 (0.0008) [2023-12-27 11:50:50,156][20381] Updated weights for policy 1, policy_version 894282 (0.0007) [2023-12-27 11:50:50,214][20381] Updated weights for policy 1, policy_version 894292 (0.0006) [2023-12-27 11:50:50,792][20347] Updated weights for policy 0, policy_version 894961 (0.0007) [2023-12-27 11:50:50,843][20381] Updated weights for policy 1, policy_version 894302 (0.0008) [2023-12-27 11:50:50,852][20347] Updated weights for policy 0, policy_version 894971 (0.0007) [2023-12-27 11:50:50,896][20381] Updated weights for policy 1, policy_version 894312 (0.0005) [2023-12-27 11:50:50,909][20347] Updated weights for policy 0, policy_version 894981 (0.0007) [2023-12-27 11:50:50,945][20381] Updated weights for policy 1, policy_version 894322 (0.0005) [2023-12-27 11:50:50,965][20347] Updated weights for policy 0, policy_version 894991 (0.0007) [2023-12-27 11:50:51,673][20347] Updated weights for policy 0, policy_version 895001 (0.0010) [2023-12-27 11:50:51,706][20381] Updated weights for policy 1, policy_version 894332 (0.0006) [2023-12-27 11:50:51,735][20347] Updated weights for policy 0, policy_version 895011 (0.0009) [2023-12-27 11:50:51,775][20381] Updated weights for policy 1, policy_version 894342 (0.0009) [2023-12-27 11:50:51,802][20347] Updated weights for policy 0, policy_version 895021 (0.0010) [2023-12-27 11:50:51,833][20381] Updated weights for policy 1, policy_version 894352 (0.0007) [2023-12-27 11:50:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 19883.0). Total num frames: 458137600. Throughput: 0: 10185.9, 1: 10327.0. Samples: 458109468. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:52,363][19170] Avg episode reward: [(0, '769.440'), (1, '1000.000')] [2023-12-27 11:50:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000894360_228982784.pth... [2023-12-27 11:50:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000895024_229154816.pth... [2023-12-27 11:50:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000893144_228671488.pth [2023-12-27 11:50:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000893840_228851712.pth [2023-12-27 11:50:52,473][20347] Updated weights for policy 0, policy_version 895031 (0.0006) [2023-12-27 11:50:52,532][20347] Updated weights for policy 0, policy_version 895041 (0.0006) [2023-12-27 11:50:52,568][20381] Updated weights for policy 1, policy_version 894362 (0.0008) [2023-12-27 11:50:52,590][20347] Updated weights for policy 0, policy_version 895051 (0.0006) [2023-12-27 11:50:52,591][20291] KL-divergence is very high: 259.5827 [2023-12-27 11:50:52,622][20381] Updated weights for policy 1, policy_version 894372 (0.0006) [2023-12-27 11:50:52,686][20381] Updated weights for policy 1, policy_version 894382 (0.0008) [2023-12-27 11:50:52,736][20381] Updated weights for policy 1, policy_version 894392 (0.0009) [2023-12-27 11:50:53,216][20291] KL-divergence is very high: 210.6395 [2023-12-27 11:50:53,244][20347] Updated weights for policy 0, policy_version 895061 (0.0005) [2023-12-27 11:50:53,290][20347] Updated weights for policy 0, policy_version 895071 (0.0005) [2023-12-27 11:50:53,338][20347] Updated weights for policy 0, policy_version 895081 (0.0005) [2023-12-27 11:50:53,447][20381] Updated weights for policy 1, policy_version 894402 (0.0007) [2023-12-27 11:50:53,506][20381] Updated weights for policy 1, policy_version 894412 (0.0008) [2023-12-27 11:50:53,561][20381] Updated weights for policy 1, policy_version 894422 (0.0007) [2023-12-27 11:50:53,985][20347] Updated weights for policy 0, policy_version 895091 (0.0006) [2023-12-27 11:50:54,027][20347] Updated weights for policy 0, policy_version 895101 (0.0006) [2023-12-27 11:50:54,076][20347] Updated weights for policy 0, policy_version 895111 (0.0005) [2023-12-27 11:50:54,220][20381] Updated weights for policy 1, policy_version 894432 (0.0007) [2023-12-27 11:50:54,276][20381] Updated weights for policy 1, policy_version 894442 (0.0011) [2023-12-27 11:50:54,335][20381] Updated weights for policy 1, policy_version 894452 (0.0011) [2023-12-27 11:50:54,802][20347] Updated weights for policy 0, policy_version 895121 (0.0009) [2023-12-27 11:50:54,865][20347] Updated weights for policy 0, policy_version 895131 (0.0008) [2023-12-27 11:50:54,925][20347] Updated weights for policy 0, policy_version 895141 (0.0008) [2023-12-27 11:50:54,987][20347] Updated weights for policy 0, policy_version 895151 (0.0007) [2023-12-27 11:50:55,012][20381] Updated weights for policy 1, policy_version 894462 (0.0011) [2023-12-27 11:50:55,061][20381] Updated weights for policy 1, policy_version 894472 (0.0010) [2023-12-27 11:50:55,125][20381] Updated weights for policy 1, policy_version 894482 (0.0009) [2023-12-27 11:50:55,624][20347] Updated weights for policy 0, policy_version 895161 (0.0008) [2023-12-27 11:50:55,685][20347] Updated weights for policy 0, policy_version 895171 (0.0006) [2023-12-27 11:50:55,733][20347] Updated weights for policy 0, policy_version 895181 (0.0008) [2023-12-27 11:50:55,832][20381] Updated weights for policy 1, policy_version 894492 (0.0008) [2023-12-27 11:50:55,893][20381] Updated weights for policy 1, policy_version 894502 (0.0007) [2023-12-27 11:50:55,950][20381] Updated weights for policy 1, policy_version 894512 (0.0008) [2023-12-27 11:50:56,453][20347] Updated weights for policy 0, policy_version 895191 (0.0008) [2023-12-27 11:50:56,497][20347] Updated weights for policy 0, policy_version 895201 (0.0007) [2023-12-27 11:50:56,545][20381] Updated weights for policy 1, policy_version 894522 (0.0007) [2023-12-27 11:50:56,547][20347] Updated weights for policy 0, policy_version 895211 (0.0007) [2023-12-27 11:50:56,593][20381] Updated weights for policy 1, policy_version 894532 (0.0005) [2023-12-27 11:50:56,641][20381] Updated weights for policy 1, policy_version 894542 (0.0005) [2023-12-27 11:50:56,690][20381] Updated weights for policy 1, policy_version 894552 (0.0005) [2023-12-27 11:50:57,280][20347] Updated weights for policy 0, policy_version 895221 (0.0007) [2023-12-27 11:50:57,297][20381] Updated weights for policy 1, policy_version 894562 (0.0005) [2023-12-27 11:50:57,328][20347] Updated weights for policy 0, policy_version 895231 (0.0006) [2023-12-27 11:50:57,345][20381] Updated weights for policy 1, policy_version 894572 (0.0007) [2023-12-27 11:50:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 19882.9). Total num frames: 458235904. Throughput: 0: 10209.6, 1: 10304.2. Samples: 458231516. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:50:57,363][19170] Avg episode reward: [(0, '759.570'), (1, '1000.000')] [2023-12-27 11:50:57,376][20347] Updated weights for policy 0, policy_version 895241 (0.0005) [2023-12-27 11:50:57,392][20381] Updated weights for policy 1, policy_version 894582 (0.0007) [2023-12-27 11:50:57,973][20347] Updated weights for policy 0, policy_version 895251 (0.0006) [2023-12-27 11:50:58,027][20347] Updated weights for policy 0, policy_version 895261 (0.0008) [2023-12-27 11:50:58,088][20347] Updated weights for policy 0, policy_version 895271 (0.0008) [2023-12-27 11:50:58,150][20381] Updated weights for policy 1, policy_version 894592 (0.0007) [2023-12-27 11:50:58,211][20381] Updated weights for policy 1, policy_version 894602 (0.0008) [2023-12-27 11:50:58,276][20381] Updated weights for policy 1, policy_version 894612 (0.0009) [2023-12-27 11:50:58,815][20347] Updated weights for policy 0, policy_version 895281 (0.0008) [2023-12-27 11:50:58,878][20347] Updated weights for policy 0, policy_version 895291 (0.0007) [2023-12-27 11:50:58,941][20347] Updated weights for policy 0, policy_version 895301 (0.0008) [2023-12-27 11:50:58,999][20347] Updated weights for policy 0, policy_version 895311 (0.0007) [2023-12-27 11:50:59,067][20381] Updated weights for policy 1, policy_version 894622 (0.0007) [2023-12-27 11:50:59,130][20381] Updated weights for policy 1, policy_version 894632 (0.0008) [2023-12-27 11:50:59,188][20381] Updated weights for policy 1, policy_version 894642 (0.0008) [2023-12-27 11:50:59,756][20347] Updated weights for policy 0, policy_version 895321 (0.0007) [2023-12-27 11:50:59,813][20347] Updated weights for policy 0, policy_version 895331 (0.0006) [2023-12-27 11:50:59,878][20347] Updated weights for policy 0, policy_version 895341 (0.0010) [2023-12-27 11:50:59,888][20381] Updated weights for policy 1, policy_version 894652 (0.0008) [2023-12-27 11:50:59,946][20381] Updated weights for policy 1, policy_version 894662 (0.0008) [2023-12-27 11:50:59,991][20381] Updated weights for policy 1, policy_version 894672 (0.0007) [2023-12-27 11:51:00,539][20347] Updated weights for policy 0, policy_version 895351 (0.0010) [2023-12-27 11:51:00,599][20347] Updated weights for policy 0, policy_version 895361 (0.0005) [2023-12-27 11:51:00,635][20381] Updated weights for policy 1, policy_version 894682 (0.0008) [2023-12-27 11:51:00,661][20347] Updated weights for policy 0, policy_version 895371 (0.0006) [2023-12-27 11:51:00,685][20381] Updated weights for policy 1, policy_version 894692 (0.0006) [2023-12-27 11:51:00,729][20381] Updated weights for policy 1, policy_version 894702 (0.0007) [2023-12-27 11:51:00,782][20381] Updated weights for policy 1, policy_version 894712 (0.0006) [2023-12-27 11:51:01,301][20347] Updated weights for policy 0, policy_version 895381 (0.0010) [2023-12-27 11:51:01,356][20347] Updated weights for policy 0, policy_version 895391 (0.0010) [2023-12-27 11:51:01,419][20347] Updated weights for policy 0, policy_version 895401 (0.0009) [2023-12-27 11:51:01,453][20381] Updated weights for policy 1, policy_version 894722 (0.0006) [2023-12-27 11:51:01,506][20381] Updated weights for policy 1, policy_version 894732 (0.0008) [2023-12-27 11:51:01,565][20381] Updated weights for policy 1, policy_version 894742 (0.0008) [2023-12-27 11:51:02,145][20347] Updated weights for policy 0, policy_version 895411 (0.0007) [2023-12-27 11:51:02,201][20347] Updated weights for policy 0, policy_version 895421 (0.0008) [2023-12-27 11:51:02,231][20381] Updated weights for policy 1, policy_version 894752 (0.0007) [2023-12-27 11:51:02,261][20347] Updated weights for policy 0, policy_version 895431 (0.0006) [2023-12-27 11:51:02,291][20381] Updated weights for policy 1, policy_version 894762 (0.0009) [2023-12-27 11:51:02,339][20381] Updated weights for policy 1, policy_version 894772 (0.0008) [2023-12-27 11:51:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 19938.5). Total num frames: 458350592. Throughput: 0: 10175.6, 1: 10294.1. Samples: 458352860. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:02,363][19170] Avg episode reward: [(0, '776.480'), (1, '1000.000')] [2023-12-27 11:51:02,966][20347] Updated weights for policy 0, policy_version 895441 (0.0007) [2023-12-27 11:51:03,014][20347] Updated weights for policy 0, policy_version 895451 (0.0010) [2023-12-27 11:51:03,065][20347] Updated weights for policy 0, policy_version 895461 (0.0010) [2023-12-27 11:51:03,094][20381] Updated weights for policy 1, policy_version 894782 (0.0006) [2023-12-27 11:51:03,114][20347] Updated weights for policy 0, policy_version 895471 (0.0010) [2023-12-27 11:51:03,150][20381] Updated weights for policy 1, policy_version 894792 (0.0006) [2023-12-27 11:51:03,205][20381] Updated weights for policy 1, policy_version 894802 (0.0008) [2023-12-27 11:51:03,785][20347] Updated weights for policy 0, policy_version 895481 (0.0010) [2023-12-27 11:51:03,841][20347] Updated weights for policy 0, policy_version 895491 (0.0009) [2023-12-27 11:51:03,899][20347] Updated weights for policy 0, policy_version 895501 (0.0011) [2023-12-27 11:51:03,923][20381] Updated weights for policy 1, policy_version 894812 (0.0007) [2023-12-27 11:51:03,985][20381] Updated weights for policy 1, policy_version 894822 (0.0007) [2023-12-27 11:51:04,047][20381] Updated weights for policy 1, policy_version 894832 (0.0007) [2023-12-27 11:51:04,589][20347] Updated weights for policy 0, policy_version 895511 (0.0011) [2023-12-27 11:51:04,644][20347] Updated weights for policy 0, policy_version 895521 (0.0010) [2023-12-27 11:51:04,700][20347] Updated weights for policy 0, policy_version 895531 (0.0010) [2023-12-27 11:51:04,758][20381] Updated weights for policy 1, policy_version 894842 (0.0010) [2023-12-27 11:51:04,811][20381] Updated weights for policy 1, policy_version 894852 (0.0009) [2023-12-27 11:51:04,867][20381] Updated weights for policy 1, policy_version 894862 (0.0008) [2023-12-27 11:51:04,921][20381] Updated weights for policy 1, policy_version 894872 (0.0007) [2023-12-27 11:51:05,347][20347] Updated weights for policy 0, policy_version 895541 (0.0005) [2023-12-27 11:51:05,409][20347] Updated weights for policy 0, policy_version 895551 (0.0010) [2023-12-27 11:51:05,467][20347] Updated weights for policy 0, policy_version 895561 (0.0010) [2023-12-27 11:51:05,647][20381] Updated weights for policy 1, policy_version 894882 (0.0007) [2023-12-27 11:51:05,701][20381] Updated weights for policy 1, policy_version 894892 (0.0007) [2023-12-27 11:51:05,757][20381] Updated weights for policy 1, policy_version 894902 (0.0009) [2023-12-27 11:51:06,042][20347] Updated weights for policy 0, policy_version 895571 (0.0009) [2023-12-27 11:51:06,090][20347] Updated weights for policy 0, policy_version 895581 (0.0007) [2023-12-27 11:51:06,145][20347] Updated weights for policy 0, policy_version 895591 (0.0008) [2023-12-27 11:51:06,468][20381] Updated weights for policy 1, policy_version 894912 (0.0007) [2023-12-27 11:51:06,529][20381] Updated weights for policy 1, policy_version 894922 (0.0008) [2023-12-27 11:51:06,591][20381] Updated weights for policy 1, policy_version 894932 (0.0007) [2023-12-27 11:51:06,875][20347] Updated weights for policy 0, policy_version 895601 (0.0007) [2023-12-27 11:51:06,932][20347] Updated weights for policy 0, policy_version 895611 (0.0005) [2023-12-27 11:51:06,987][20347] Updated weights for policy 0, policy_version 895621 (0.0006) [2023-12-27 11:51:07,045][20347] Updated weights for policy 0, policy_version 895631 (0.0008) [2023-12-27 11:51:07,309][20381] Updated weights for policy 1, policy_version 894942 (0.0010) [2023-12-27 11:51:07,361][20381] Updated weights for policy 1, policy_version 894952 (0.0008) [2023-12-27 11:51:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 19938.5). Total num frames: 458440704. Throughput: 0: 10211.0, 1: 10254.1. Samples: 458412656. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:07,363][19170] Avg episode reward: [(0, '690.380'), (1, '1000.000')] [2023-12-27 11:51:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000895632_229310464.pth... [2023-12-27 11:51:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000894448_229007360.pth [2023-12-27 11:51:07,406][20381] Updated weights for policy 1, policy_version 894962 (0.0010) [2023-12-27 11:51:07,434][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000894968_229138432.pth... [2023-12-27 11:51:07,437][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000893752_228827136.pth [2023-12-27 11:51:07,729][20347] Updated weights for policy 0, policy_version 895641 (0.0007) [2023-12-27 11:51:07,786][20347] Updated weights for policy 0, policy_version 895651 (0.0007) [2023-12-27 11:51:07,846][20347] Updated weights for policy 0, policy_version 895661 (0.0007) [2023-12-27 11:51:08,120][20381] Updated weights for policy 1, policy_version 894972 (0.0011) [2023-12-27 11:51:08,172][20381] Updated weights for policy 1, policy_version 894982 (0.0011) [2023-12-27 11:51:08,225][20381] Updated weights for policy 1, policy_version 894992 (0.0011) [2023-12-27 11:51:08,540][20347] Updated weights for policy 0, policy_version 895671 (0.0008) [2023-12-27 11:51:08,600][20347] Updated weights for policy 0, policy_version 895681 (0.0007) [2023-12-27 11:51:08,649][20347] Updated weights for policy 0, policy_version 895691 (0.0007) [2023-12-27 11:51:08,935][20381] Updated weights for policy 1, policy_version 895002 (0.0009) [2023-12-27 11:51:08,997][20381] Updated weights for policy 1, policy_version 895012 (0.0008) [2023-12-27 11:51:09,058][20381] Updated weights for policy 1, policy_version 895022 (0.0008) [2023-12-27 11:51:09,110][20381] Updated weights for policy 1, policy_version 895032 (0.0008) [2023-12-27 11:51:09,283][20347] Updated weights for policy 0, policy_version 895701 (0.0007) [2023-12-27 11:51:09,343][20347] Updated weights for policy 0, policy_version 895711 (0.0007) [2023-12-27 11:51:09,411][20347] Updated weights for policy 0, policy_version 895721 (0.0008) [2023-12-27 11:51:09,765][20381] Updated weights for policy 1, policy_version 895042 (0.0007) [2023-12-27 11:51:09,827][20381] Updated weights for policy 1, policy_version 895052 (0.0008) [2023-12-27 11:51:09,890][20381] Updated weights for policy 1, policy_version 895062 (0.0007) [2023-12-27 11:51:10,124][20347] Updated weights for policy 0, policy_version 895731 (0.0007) [2023-12-27 11:51:10,181][20347] Updated weights for policy 0, policy_version 895741 (0.0007) [2023-12-27 11:51:10,240][20347] Updated weights for policy 0, policy_version 895751 (0.0007) [2023-12-27 11:51:10,583][20381] Updated weights for policy 1, policy_version 895072 (0.0008) [2023-12-27 11:51:10,640][20381] Updated weights for policy 1, policy_version 895082 (0.0008) [2023-12-27 11:51:10,695][20381] Updated weights for policy 1, policy_version 895092 (0.0008) [2023-12-27 11:51:10,907][20347] Updated weights for policy 0, policy_version 895761 (0.0007) [2023-12-27 11:51:10,970][20347] Updated weights for policy 0, policy_version 895771 (0.0008) [2023-12-27 11:51:11,040][20347] Updated weights for policy 0, policy_version 895781 (0.0008) [2023-12-27 11:51:11,103][20347] Updated weights for policy 0, policy_version 895791 (0.0008) [2023-12-27 11:51:11,447][20381] Updated weights for policy 1, policy_version 895102 (0.0007) [2023-12-27 11:51:11,507][20381] Updated weights for policy 1, policy_version 895112 (0.0010) [2023-12-27 11:51:11,568][20381] Updated weights for policy 1, policy_version 895122 (0.0005) [2023-12-27 11:51:11,847][20347] Updated weights for policy 0, policy_version 895801 (0.0008) [2023-12-27 11:51:11,901][20347] Updated weights for policy 0, policy_version 895811 (0.0007) [2023-12-27 11:51:11,958][20347] Updated weights for policy 0, policy_version 895821 (0.0008) [2023-12-27 11:51:12,206][20381] Updated weights for policy 1, policy_version 895132 (0.0009) [2023-12-27 11:51:12,264][20381] Updated weights for policy 1, policy_version 895142 (0.0010) [2023-12-27 11:51:12,325][20381] Updated weights for policy 1, policy_version 895152 (0.0011) [2023-12-27 11:51:12,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20343.4, 300 sec: 19938.5). Total num frames: 458539008. Throughput: 0: 10248.3, 1: 10257.2. Samples: 458532848. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:12,363][19170] Avg episode reward: [(0, '757.840'), (1, '1000.000')] [2023-12-27 11:51:12,672][20347] Updated weights for policy 0, policy_version 895831 (0.0010) [2023-12-27 11:51:12,734][20347] Updated weights for policy 0, policy_version 895841 (0.0010) [2023-12-27 11:51:12,796][20347] Updated weights for policy 0, policy_version 895851 (0.0010) [2023-12-27 11:51:13,000][20381] Updated weights for policy 1, policy_version 895162 (0.0009) [2023-12-27 11:51:13,057][20381] Updated weights for policy 1, policy_version 895172 (0.0007) [2023-12-27 11:51:13,104][20381] Updated weights for policy 1, policy_version 895182 (0.0006) [2023-12-27 11:51:13,155][20381] Updated weights for policy 1, policy_version 895192 (0.0006) [2023-12-27 11:51:13,519][20347] Updated weights for policy 0, policy_version 895861 (0.0008) [2023-12-27 11:51:13,581][20347] Updated weights for policy 0, policy_version 895871 (0.0007) [2023-12-27 11:51:13,645][20347] Updated weights for policy 0, policy_version 895881 (0.0010) [2023-12-27 11:51:13,795][20381] Updated weights for policy 1, policy_version 895202 (0.0005) [2023-12-27 11:51:13,853][20381] Updated weights for policy 1, policy_version 895212 (0.0007) [2023-12-27 11:51:13,911][20381] Updated weights for policy 1, policy_version 895222 (0.0007) [2023-12-27 11:51:14,314][20347] Updated weights for policy 0, policy_version 895891 (0.0010) [2023-12-27 11:51:14,370][20347] Updated weights for policy 0, policy_version 895901 (0.0008) [2023-12-27 11:51:14,417][20347] Updated weights for policy 0, policy_version 895911 (0.0005) [2023-12-27 11:51:14,583][20381] Updated weights for policy 1, policy_version 895232 (0.0008) [2023-12-27 11:51:14,632][20381] Updated weights for policy 1, policy_version 895242 (0.0008) [2023-12-27 11:51:14,680][20381] Updated weights for policy 1, policy_version 895252 (0.0007) [2023-12-27 11:51:15,071][20347] Updated weights for policy 0, policy_version 895921 (0.0006) [2023-12-27 11:51:15,127][20347] Updated weights for policy 0, policy_version 895931 (0.0008) [2023-12-27 11:51:15,177][20347] Updated weights for policy 0, policy_version 895941 (0.0007) [2023-12-27 11:51:15,227][20347] Updated weights for policy 0, policy_version 895951 (0.0008) [2023-12-27 11:51:15,441][20381] Updated weights for policy 1, policy_version 895262 (0.0007) [2023-12-27 11:51:15,497][20381] Updated weights for policy 1, policy_version 895272 (0.0008) [2023-12-27 11:51:15,552][20381] Updated weights for policy 1, policy_version 895282 (0.0008) [2023-12-27 11:51:15,945][20347] Updated weights for policy 0, policy_version 895961 (0.0008) [2023-12-27 11:51:15,997][20347] Updated weights for policy 0, policy_version 895971 (0.0007) [2023-12-27 11:51:16,053][20347] Updated weights for policy 0, policy_version 895981 (0.0008) [2023-12-27 11:51:16,216][20381] Updated weights for policy 1, policy_version 895292 (0.0007) [2023-12-27 11:51:16,271][20381] Updated weights for policy 1, policy_version 895302 (0.0007) [2023-12-27 11:51:16,319][20381] Updated weights for policy 1, policy_version 895312 (0.0007) [2023-12-27 11:51:16,714][20347] Updated weights for policy 0, policy_version 895991 (0.0008) [2023-12-27 11:51:16,765][20347] Updated weights for policy 0, policy_version 896001 (0.0008) [2023-12-27 11:51:16,812][20347] Updated weights for policy 0, policy_version 896011 (0.0007) [2023-12-27 11:51:17,009][20381] Updated weights for policy 1, policy_version 895322 (0.0007) [2023-12-27 11:51:17,065][20381] Updated weights for policy 1, policy_version 895332 (0.0008) [2023-12-27 11:51:17,120][20381] Updated weights for policy 1, policy_version 895342 (0.0007) [2023-12-27 11:51:17,177][20381] Updated weights for policy 1, policy_version 895352 (0.0007) [2023-12-27 11:51:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 19966.3). Total num frames: 458645504. Throughput: 0: 10160.7, 1: 10156.2. Samples: 458654088. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:17,363][19170] Avg episode reward: [(0, '825.220'), (1, '1000.000')] [2023-12-27 11:51:17,541][20347] Updated weights for policy 0, policy_version 896021 (0.0008) [2023-12-27 11:51:17,593][20347] Updated weights for policy 0, policy_version 896031 (0.0010) [2023-12-27 11:51:17,642][20347] Updated weights for policy 0, policy_version 896041 (0.0010) [2023-12-27 11:51:17,832][20381] Updated weights for policy 1, policy_version 895362 (0.0006) [2023-12-27 11:51:17,884][20381] Updated weights for policy 1, policy_version 895372 (0.0009) [2023-12-27 11:51:17,934][20381] Updated weights for policy 1, policy_version 895382 (0.0009) [2023-12-27 11:51:18,257][20347] Updated weights for policy 0, policy_version 896051 (0.0009) [2023-12-27 11:51:18,319][20347] Updated weights for policy 0, policy_version 896061 (0.0009) [2023-12-27 11:51:18,383][20347] Updated weights for policy 0, policy_version 896071 (0.0007) [2023-12-27 11:51:18,635][20381] Updated weights for policy 1, policy_version 895392 (0.0011) [2023-12-27 11:51:18,687][20381] Updated weights for policy 1, policy_version 895402 (0.0011) [2023-12-27 11:51:18,738][20381] Updated weights for policy 1, policy_version 895412 (0.0011) [2023-12-27 11:51:18,987][20347] Updated weights for policy 0, policy_version 896081 (0.0008) [2023-12-27 11:51:19,038][20347] Updated weights for policy 0, policy_version 896091 (0.0007) [2023-12-27 11:51:19,097][20347] Updated weights for policy 0, policy_version 896101 (0.0008) [2023-12-27 11:51:19,142][20347] Updated weights for policy 0, policy_version 896111 (0.0007) [2023-12-27 11:51:19,429][20381] Updated weights for policy 1, policy_version 895422 (0.0010) [2023-12-27 11:51:19,488][20381] Updated weights for policy 1, policy_version 895432 (0.0010) [2023-12-27 11:51:19,545][20381] Updated weights for policy 1, policy_version 895442 (0.0009) [2023-12-27 11:51:19,857][20347] Updated weights for policy 0, policy_version 896121 (0.0007) [2023-12-27 11:51:19,907][20347] Updated weights for policy 0, policy_version 896131 (0.0008) [2023-12-27 11:51:19,967][20347] Updated weights for policy 0, policy_version 896141 (0.0007) [2023-12-27 11:51:20,206][20381] Updated weights for policy 1, policy_version 895452 (0.0006) [2023-12-27 11:51:20,262][20381] Updated weights for policy 1, policy_version 895462 (0.0008) [2023-12-27 11:51:20,320][20381] Updated weights for policy 1, policy_version 895472 (0.0008) [2023-12-27 11:51:20,684][20347] Updated weights for policy 0, policy_version 896151 (0.0008) [2023-12-27 11:51:20,740][20347] Updated weights for policy 0, policy_version 896161 (0.0007) [2023-12-27 11:51:20,801][20347] Updated weights for policy 0, policy_version 896171 (0.0009) [2023-12-27 11:51:20,996][20381] Updated weights for policy 1, policy_version 895482 (0.0007) [2023-12-27 11:51:21,064][20381] Updated weights for policy 1, policy_version 895492 (0.0007) [2023-12-27 11:51:21,126][20381] Updated weights for policy 1, policy_version 895502 (0.0009) [2023-12-27 11:51:21,184][20381] Updated weights for policy 1, policy_version 895512 (0.0008) [2023-12-27 11:51:21,594][20347] Updated weights for policy 0, policy_version 896181 (0.0008) [2023-12-27 11:51:21,662][20347] Updated weights for policy 0, policy_version 896191 (0.0008) [2023-12-27 11:51:21,727][20347] Updated weights for policy 0, policy_version 896201 (0.0008) [2023-12-27 11:51:21,895][20381] Updated weights for policy 1, policy_version 895522 (0.0008) [2023-12-27 11:51:21,955][20381] Updated weights for policy 1, policy_version 895532 (0.0007) [2023-12-27 11:51:22,012][20381] Updated weights for policy 1, policy_version 895542 (0.0007) [2023-12-27 11:51:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 19966.3). Total num frames: 458743808. Throughput: 0: 10145.2, 1: 10160.0. Samples: 458716036. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:22,363][19170] Avg episode reward: [(0, '854.550'), (1, '1000.000')] [2023-12-27 11:51:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000895544_229285888.pth... [2023-12-27 11:51:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000896208_229457920.pth... [2023-12-27 11:51:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000894360_228982784.pth [2023-12-27 11:51:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000895024_229154816.pth [2023-12-27 11:51:22,426][20347] Updated weights for policy 0, policy_version 896211 (0.0008) [2023-12-27 11:51:22,482][20347] Updated weights for policy 0, policy_version 896221 (0.0007) [2023-12-27 11:51:22,538][20347] Updated weights for policy 0, policy_version 896231 (0.0008) [2023-12-27 11:51:22,730][20381] Updated weights for policy 1, policy_version 895552 (0.0008) [2023-12-27 11:51:22,779][20381] Updated weights for policy 1, policy_version 895562 (0.0007) [2023-12-27 11:51:22,835][20381] Updated weights for policy 1, policy_version 895572 (0.0008) [2023-12-27 11:51:23,245][20347] Updated weights for policy 0, policy_version 896241 (0.0007) [2023-12-27 11:51:23,294][20347] Updated weights for policy 0, policy_version 896251 (0.0008) [2023-12-27 11:51:23,353][20347] Updated weights for policy 0, policy_version 896261 (0.0006) [2023-12-27 11:51:23,403][20347] Updated weights for policy 0, policy_version 896271 (0.0006) [2023-12-27 11:51:23,501][20381] Updated weights for policy 1, policy_version 895582 (0.0007) [2023-12-27 11:51:23,557][20381] Updated weights for policy 1, policy_version 895592 (0.0007) [2023-12-27 11:51:23,610][20381] Updated weights for policy 1, policy_version 895602 (0.0008) [2023-12-27 11:51:24,096][20347] Updated weights for policy 0, policy_version 896281 (0.0007) [2023-12-27 11:51:24,144][20347] Updated weights for policy 0, policy_version 896291 (0.0007) [2023-12-27 11:51:24,192][20347] Updated weights for policy 0, policy_version 896301 (0.0007) [2023-12-27 11:51:24,301][20381] Updated weights for policy 1, policy_version 895612 (0.0008) [2023-12-27 11:51:24,353][20381] Updated weights for policy 1, policy_version 895622 (0.0010) [2023-12-27 11:51:24,408][20381] Updated weights for policy 1, policy_version 895632 (0.0011) [2023-12-27 11:51:24,905][20347] Updated weights for policy 0, policy_version 896311 (0.0007) [2023-12-27 11:51:24,962][20347] Updated weights for policy 0, policy_version 896321 (0.0008) [2023-12-27 11:51:25,019][20347] Updated weights for policy 0, policy_version 896331 (0.0007) [2023-12-27 11:51:25,025][20291] KL-divergence is very high: 120.4514 [2023-12-27 11:51:25,040][20291] KL-divergence is very high: 132.4977 [2023-12-27 11:51:25,098][20381] Updated weights for policy 1, policy_version 895642 (0.0010) [2023-12-27 11:51:25,153][20381] Updated weights for policy 1, policy_version 895652 (0.0011) [2023-12-27 11:51:25,206][20381] Updated weights for policy 1, policy_version 895662 (0.0010) [2023-12-27 11:51:25,254][20381] Updated weights for policy 1, policy_version 895672 (0.0010) [2023-12-27 11:51:25,699][20291] KL-divergence is very high: 138.1352 [2023-12-27 11:51:25,720][20347] Updated weights for policy 0, policy_version 896341 (0.0007) [2023-12-27 11:51:25,746][20291] KL-divergence is very high: 116.8933 [2023-12-27 11:51:25,753][20291] KL-divergence is very high: 127.7458 [2023-12-27 11:51:25,785][20347] Updated weights for policy 0, policy_version 896351 (0.0008) [2023-12-27 11:51:25,850][20347] Updated weights for policy 0, policy_version 896361 (0.0007) [2023-12-27 11:51:25,969][20381] Updated weights for policy 1, policy_version 895682 (0.0011) [2023-12-27 11:51:26,019][20381] Updated weights for policy 1, policy_version 895692 (0.0010) [2023-12-27 11:51:26,076][20381] Updated weights for policy 1, policy_version 895702 (0.0011) [2023-12-27 11:51:26,541][20347] Updated weights for policy 0, policy_version 896371 (0.0007) [2023-12-27 11:51:26,599][20347] Updated weights for policy 0, policy_version 896381 (0.0007) [2023-12-27 11:51:26,654][20347] Updated weights for policy 0, policy_version 896391 (0.0008) [2023-12-27 11:51:26,769][20381] Updated weights for policy 1, policy_version 895712 (0.0011) [2023-12-27 11:51:26,824][20381] Updated weights for policy 1, policy_version 895722 (0.0010) [2023-12-27 11:51:26,879][20381] Updated weights for policy 1, policy_version 895732 (0.0010) [2023-12-27 11:51:27,359][20347] Updated weights for policy 0, policy_version 896401 (0.0007) [2023-12-27 11:51:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 19966.3). Total num frames: 458842112. Throughput: 0: 10100.3, 1: 10137.0. Samples: 458834540. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:27,363][19170] Avg episode reward: [(0, '727.150'), (1, '1000.000')] [2023-12-27 11:51:27,413][20347] Updated weights for policy 0, policy_version 896411 (0.0005) [2023-12-27 11:51:27,470][20347] Updated weights for policy 0, policy_version 896421 (0.0007) [2023-12-27 11:51:27,522][20347] Updated weights for policy 0, policy_version 896431 (0.0006) [2023-12-27 11:51:27,563][20381] Updated weights for policy 1, policy_version 895742 (0.0010) [2023-12-27 11:51:27,611][20381] Updated weights for policy 1, policy_version 895752 (0.0010) [2023-12-27 11:51:27,659][20381] Updated weights for policy 1, policy_version 895762 (0.0010) [2023-12-27 11:51:28,195][20347] Updated weights for policy 0, policy_version 896441 (0.0007) [2023-12-27 11:51:28,256][20347] Updated weights for policy 0, policy_version 896451 (0.0007) [2023-12-27 11:51:28,320][20347] Updated weights for policy 0, policy_version 896461 (0.0008) [2023-12-27 11:51:28,352][20381] Updated weights for policy 1, policy_version 895772 (0.0010) [2023-12-27 11:51:28,399][20381] Updated weights for policy 1, policy_version 895782 (0.0006) [2023-12-27 11:51:28,448][20381] Updated weights for policy 1, policy_version 895792 (0.0005) [2023-12-27 11:51:29,038][20381] Updated weights for policy 1, policy_version 895802 (0.0005) [2023-12-27 11:51:29,058][20347] Updated weights for policy 0, policy_version 896471 (0.0008) [2023-12-27 11:51:29,089][20381] Updated weights for policy 1, policy_version 895812 (0.0005) [2023-12-27 11:51:29,114][20347] Updated weights for policy 0, policy_version 896481 (0.0007) [2023-12-27 11:51:29,141][20381] Updated weights for policy 1, policy_version 895822 (0.0005) [2023-12-27 11:51:29,166][20347] Updated weights for policy 0, policy_version 896491 (0.0008) [2023-12-27 11:51:29,194][20381] Updated weights for policy 1, policy_version 895832 (0.0005) [2023-12-27 11:51:29,818][20381] Updated weights for policy 1, policy_version 895842 (0.0011) [2023-12-27 11:51:29,830][20347] Updated weights for policy 0, policy_version 896501 (0.0008) [2023-12-27 11:51:29,879][20381] Updated weights for policy 1, policy_version 895852 (0.0009) [2023-12-27 11:51:29,890][20347] Updated weights for policy 0, policy_version 896511 (0.0007) [2023-12-27 11:51:29,941][20381] Updated weights for policy 1, policy_version 895862 (0.0007) [2023-12-27 11:51:29,953][20347] Updated weights for policy 0, policy_version 896521 (0.0007) [2023-12-27 11:51:30,606][20381] Updated weights for policy 1, policy_version 895872 (0.0006) [2023-12-27 11:51:30,650][20347] Updated weights for policy 0, policy_version 896531 (0.0007) [2023-12-27 11:51:30,660][20381] Updated weights for policy 1, policy_version 895882 (0.0006) [2023-12-27 11:51:30,698][20347] Updated weights for policy 0, policy_version 896541 (0.0007) [2023-12-27 11:51:30,708][20381] Updated weights for policy 1, policy_version 895892 (0.0006) [2023-12-27 11:51:30,752][20347] Updated weights for policy 0, policy_version 896551 (0.0006) [2023-12-27 11:51:31,390][20381] Updated weights for policy 1, policy_version 895902 (0.0007) [2023-12-27 11:51:31,450][20381] Updated weights for policy 1, policy_version 895912 (0.0007) [2023-12-27 11:51:31,478][20347] Updated weights for policy 0, policy_version 896561 (0.0007) [2023-12-27 11:51:31,509][20381] Updated weights for policy 1, policy_version 895922 (0.0006) [2023-12-27 11:51:31,538][20347] Updated weights for policy 0, policy_version 896571 (0.0009) [2023-12-27 11:51:31,590][20347] Updated weights for policy 0, policy_version 896581 (0.0010) [2023-12-27 11:51:31,649][20347] Updated weights for policy 0, policy_version 896591 (0.0008) [2023-12-27 11:51:32,156][20381] Updated weights for policy 1, policy_version 895932 (0.0008) [2023-12-27 11:51:32,208][20381] Updated weights for policy 1, policy_version 895942 (0.0007) [2023-12-27 11:51:32,267][20381] Updated weights for policy 1, policy_version 895952 (0.0007) [2023-12-27 11:51:32,346][20347] Updated weights for policy 0, policy_version 896601 (0.0008) [2023-12-27 11:51:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20021.8). Total num frames: 458948608. Throughput: 0: 10072.7, 1: 10147.8. Samples: 458957844. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:32,363][19170] Avg episode reward: [(0, '728.260'), (1, '1000.000')] [2023-12-27 11:51:32,406][20347] Updated weights for policy 0, policy_version 896611 (0.0008) [2023-12-27 11:51:32,444][20291] KL-divergence is very high: 201.4953 [2023-12-27 11:51:32,450][20291] KL-divergence is very high: 188.7517 [2023-12-27 11:51:32,461][20291] KL-divergence is very high: 107.2992 [2023-12-27 11:51:32,468][20347] Updated weights for policy 0, policy_version 896621 (0.0008) [2023-12-27 11:51:32,920][20381] Updated weights for policy 1, policy_version 895962 (0.0006) [2023-12-27 11:51:32,976][20381] Updated weights for policy 1, policy_version 895972 (0.0005) [2023-12-27 11:51:33,021][20381] Updated weights for policy 1, policy_version 895982 (0.0005) [2023-12-27 11:51:33,065][20381] Updated weights for policy 1, policy_version 895992 (0.0007) [2023-12-27 11:51:33,075][20347] Updated weights for policy 0, policy_version 896631 (0.0007) [2023-12-27 11:51:33,125][20347] Updated weights for policy 0, policy_version 896641 (0.0005) [2023-12-27 11:51:33,174][20347] Updated weights for policy 0, policy_version 896651 (0.0005) [2023-12-27 11:51:33,746][20347] Updated weights for policy 0, policy_version 896661 (0.0006) [2023-12-27 11:51:33,764][20381] Updated weights for policy 1, policy_version 896002 (0.0006) [2023-12-27 11:51:33,802][20347] Updated weights for policy 0, policy_version 896671 (0.0006) [2023-12-27 11:51:33,819][20381] Updated weights for policy 1, policy_version 896012 (0.0005) [2023-12-27 11:51:33,857][20347] Updated weights for policy 0, policy_version 896681 (0.0008) [2023-12-27 11:51:33,874][20381] Updated weights for policy 1, policy_version 896022 (0.0006) [2023-12-27 11:51:34,559][20347] Updated weights for policy 0, policy_version 896691 (0.0006) [2023-12-27 11:51:34,616][20347] Updated weights for policy 0, policy_version 896701 (0.0008) [2023-12-27 11:51:34,618][20381] Updated weights for policy 1, policy_version 896032 (0.0007) [2023-12-27 11:51:34,668][20381] Updated weights for policy 1, policy_version 896042 (0.0005) [2023-12-27 11:51:34,672][20347] Updated weights for policy 0, policy_version 896711 (0.0008) [2023-12-27 11:51:34,725][20381] Updated weights for policy 1, policy_version 896052 (0.0006) [2023-12-27 11:51:35,258][20347] Updated weights for policy 0, policy_version 896721 (0.0007) [2023-12-27 11:51:35,313][20347] Updated weights for policy 0, policy_version 896731 (0.0007) [2023-12-27 11:51:35,364][20347] Updated weights for policy 0, policy_version 896741 (0.0007) [2023-12-27 11:51:35,420][20381] Updated weights for policy 1, policy_version 896062 (0.0007) [2023-12-27 11:51:35,422][20347] Updated weights for policy 0, policy_version 896751 (0.0007) [2023-12-27 11:51:35,470][20381] Updated weights for policy 1, policy_version 896072 (0.0007) [2023-12-27 11:51:35,523][20381] Updated weights for policy 1, policy_version 896082 (0.0008) [2023-12-27 11:51:36,112][20347] Updated weights for policy 0, policy_version 896761 (0.0007) [2023-12-27 11:51:36,176][20347] Updated weights for policy 0, policy_version 896771 (0.0008) [2023-12-27 11:51:36,241][20347] Updated weights for policy 0, policy_version 896781 (0.0007) [2023-12-27 11:51:36,254][20381] Updated weights for policy 1, policy_version 896092 (0.0007) [2023-12-27 11:51:36,312][20381] Updated weights for policy 1, policy_version 896102 (0.0006) [2023-12-27 11:51:36,375][20381] Updated weights for policy 1, policy_version 896112 (0.0007) [2023-12-27 11:51:36,924][20381] Updated weights for policy 1, policy_version 896122 (0.0006) [2023-12-27 11:51:36,983][20381] Updated weights for policy 1, policy_version 896132 (0.0010) [2023-12-27 11:51:37,004][20347] Updated weights for policy 0, policy_version 896791 (0.0006) [2023-12-27 11:51:37,029][20381] Updated weights for policy 1, policy_version 896142 (0.0011) [2023-12-27 11:51:37,061][20347] Updated weights for policy 0, policy_version 896801 (0.0006) [2023-12-27 11:51:37,082][20381] Updated weights for policy 1, policy_version 896152 (0.0011) [2023-12-27 11:51:37,116][20347] Updated weights for policy 0, policy_version 896811 (0.0006) [2023-12-27 11:51:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20077.3). Total num frames: 459055104. Throughput: 0: 10134.5, 1: 10112.7. Samples: 459020592. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:37,363][19170] Avg episode reward: [(0, '776.960'), (1, '1000.000')] [2023-12-27 11:51:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000896816_229613568.pth... [2023-12-27 11:51:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000896152_229441536.pth... [2023-12-27 11:51:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000895632_229310464.pth [2023-12-27 11:51:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000894968_229138432.pth [2023-12-27 11:51:37,707][20381] Updated weights for policy 1, policy_version 896162 (0.0010) [2023-12-27 11:51:37,753][20381] Updated weights for policy 1, policy_version 896172 (0.0005) [2023-12-27 11:51:37,798][20381] Updated weights for policy 1, policy_version 896182 (0.0009) [2023-12-27 11:51:37,802][20347] Updated weights for policy 0, policy_version 896821 (0.0007) [2023-12-27 11:51:37,860][20347] Updated weights for policy 0, policy_version 896831 (0.0006) [2023-12-27 11:51:37,914][20347] Updated weights for policy 0, policy_version 896841 (0.0008) [2023-12-27 11:51:38,495][20381] Updated weights for policy 1, policy_version 896192 (0.0006) [2023-12-27 11:51:38,545][20381] Updated weights for policy 1, policy_version 896202 (0.0010) [2023-12-27 11:51:38,550][20347] Updated weights for policy 0, policy_version 896851 (0.0008) [2023-12-27 11:51:38,598][20347] Updated weights for policy 0, policy_version 896861 (0.0006) [2023-12-27 11:51:38,611][20381] Updated weights for policy 1, policy_version 896212 (0.0008) [2023-12-27 11:51:38,652][20347] Updated weights for policy 0, policy_version 896871 (0.0007) [2023-12-27 11:51:39,235][20381] Updated weights for policy 1, policy_version 896222 (0.0008) [2023-12-27 11:51:39,292][20381] Updated weights for policy 1, policy_version 896232 (0.0007) [2023-12-27 11:51:39,327][20347] Updated weights for policy 0, policy_version 896881 (0.0007) [2023-12-27 11:51:39,352][20381] Updated weights for policy 1, policy_version 896242 (0.0006) [2023-12-27 11:51:39,393][20347] Updated weights for policy 0, policy_version 896891 (0.0008) [2023-12-27 11:51:39,458][20347] Updated weights for policy 0, policy_version 896901 (0.0007) [2023-12-27 11:51:39,518][20347] Updated weights for policy 0, policy_version 896911 (0.0007) [2023-12-27 11:51:40,109][20381] Updated weights for policy 1, policy_version 896252 (0.0013) [2023-12-27 11:51:40,166][20381] Updated weights for policy 1, policy_version 896262 (0.0011) [2023-12-27 11:51:40,208][20347] Updated weights for policy 0, policy_version 896921 (0.0007) [2023-12-27 11:51:40,223][20381] Updated weights for policy 1, policy_version 896272 (0.0011) [2023-12-27 11:51:40,262][20347] Updated weights for policy 0, policy_version 896931 (0.0006) [2023-12-27 11:51:40,322][20347] Updated weights for policy 0, policy_version 896941 (0.0007) [2023-12-27 11:51:40,917][20381] Updated weights for policy 1, policy_version 896282 (0.0011) [2023-12-27 11:51:40,965][20381] Updated weights for policy 1, policy_version 896292 (0.0009) [2023-12-27 11:51:41,021][20381] Updated weights for policy 1, policy_version 896302 (0.0006) [2023-12-27 11:51:41,037][20347] Updated weights for policy 0, policy_version 896951 (0.0007) [2023-12-27 11:51:41,087][20381] Updated weights for policy 1, policy_version 896312 (0.0008) [2023-12-27 11:51:41,104][20347] Updated weights for policy 0, policy_version 896961 (0.0008) [2023-12-27 11:51:41,166][20347] Updated weights for policy 0, policy_version 896971 (0.0008) [2023-12-27 11:51:41,896][20381] Updated weights for policy 1, policy_version 896322 (0.0006) [2023-12-27 11:51:41,920][20347] Updated weights for policy 0, policy_version 896981 (0.0007) [2023-12-27 11:51:41,957][20381] Updated weights for policy 1, policy_version 896332 (0.0007) [2023-12-27 11:51:41,976][20347] Updated weights for policy 0, policy_version 896991 (0.0007) [2023-12-27 11:51:42,014][20381] Updated weights for policy 1, policy_version 896342 (0.0005) [2023-12-27 11:51:42,033][20347] Updated weights for policy 0, policy_version 897001 (0.0007) [2023-12-27 11:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20077.3). Total num frames: 459153408. Throughput: 0: 10112.6, 1: 10147.7. Samples: 459143232. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:42,363][19170] Avg episode reward: [(0, '805.770'), (1, '1000.000')] [2023-12-27 11:51:42,657][20381] Updated weights for policy 1, policy_version 896352 (0.0007) [2023-12-27 11:51:42,685][20347] Updated weights for policy 0, policy_version 897011 (0.0008) [2023-12-27 11:51:42,715][20381] Updated weights for policy 1, policy_version 896362 (0.0008) [2023-12-27 11:51:42,740][20347] Updated weights for policy 0, policy_version 897021 (0.0007) [2023-12-27 11:51:42,772][20381] Updated weights for policy 1, policy_version 896372 (0.0006) [2023-12-27 11:51:42,799][20347] Updated weights for policy 0, policy_version 897031 (0.0006) [2023-12-27 11:51:43,449][20381] Updated weights for policy 1, policy_version 896382 (0.0007) [2023-12-27 11:51:43,483][20347] Updated weights for policy 0, policy_version 897041 (0.0009) [2023-12-27 11:51:43,499][20381] Updated weights for policy 1, policy_version 896392 (0.0006) [2023-12-27 11:51:43,540][20347] Updated weights for policy 0, policy_version 897051 (0.0010) [2023-12-27 11:51:43,561][20381] Updated weights for policy 1, policy_version 896402 (0.0006) [2023-12-27 11:51:43,596][20347] Updated weights for policy 0, policy_version 897061 (0.0010) [2023-12-27 11:51:43,646][20347] Updated weights for policy 0, policy_version 897071 (0.0010) [2023-12-27 11:51:44,167][20381] Updated weights for policy 1, policy_version 896412 (0.0007) [2023-12-27 11:51:44,218][20381] Updated weights for policy 1, policy_version 896422 (0.0007) [2023-12-27 11:51:44,261][20347] Updated weights for policy 0, policy_version 897081 (0.0008) [2023-12-27 11:51:44,274][20381] Updated weights for policy 1, policy_version 896432 (0.0010) [2023-12-27 11:51:44,318][20347] Updated weights for policy 0, policy_version 897091 (0.0006) [2023-12-27 11:51:44,380][20347] Updated weights for policy 0, policy_version 897101 (0.0006) [2023-12-27 11:51:44,910][20381] Updated weights for policy 1, policy_version 896442 (0.0008) [2023-12-27 11:51:44,961][20381] Updated weights for policy 1, policy_version 896452 (0.0008) [2023-12-27 11:51:45,017][20381] Updated weights for policy 1, policy_version 896462 (0.0008) [2023-12-27 11:51:45,042][20347] Updated weights for policy 0, policy_version 897111 (0.0007) [2023-12-27 11:51:45,079][20381] Updated weights for policy 1, policy_version 896472 (0.0006) [2023-12-27 11:51:45,095][20347] Updated weights for policy 0, policy_version 897121 (0.0006) [2023-12-27 11:51:45,157][20347] Updated weights for policy 0, policy_version 897131 (0.0006) [2023-12-27 11:51:45,770][20347] Updated weights for policy 0, policy_version 897141 (0.0006) [2023-12-27 11:51:45,819][20347] Updated weights for policy 0, policy_version 897151 (0.0005) [2023-12-27 11:51:45,821][20381] Updated weights for policy 1, policy_version 896482 (0.0005) [2023-12-27 11:51:45,868][20347] Updated weights for policy 0, policy_version 897161 (0.0006) [2023-12-27 11:51:45,870][20381] Updated weights for policy 1, policy_version 896492 (0.0005) [2023-12-27 11:51:45,923][20381] Updated weights for policy 1, policy_version 896502 (0.0006) [2023-12-27 11:51:46,566][20347] Updated weights for policy 0, policy_version 897171 (0.0007) [2023-12-27 11:51:46,609][20381] Updated weights for policy 1, policy_version 896512 (0.0006) [2023-12-27 11:51:46,619][20347] Updated weights for policy 0, policy_version 897181 (0.0010) [2023-12-27 11:51:46,660][20381] Updated weights for policy 1, policy_version 896522 (0.0005) [2023-12-27 11:51:46,676][20347] Updated weights for policy 0, policy_version 897191 (0.0010) [2023-12-27 11:51:46,711][20381] Updated weights for policy 1, policy_version 896532 (0.0005) [2023-12-27 11:51:47,249][20347] Updated weights for policy 0, policy_version 897201 (0.0010) [2023-12-27 11:51:47,305][20347] Updated weights for policy 0, policy_version 897211 (0.0005) [2023-12-27 11:51:47,317][20381] Updated weights for policy 1, policy_version 896542 (0.0006) [2023-12-27 11:51:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20105.1). Total num frames: 459251712. Throughput: 0: 10169.3, 1: 10174.9. Samples: 459268352. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:47,363][19170] Avg episode reward: [(0, '835.170'), (1, '1000.000')] [2023-12-27 11:51:47,365][20347] Updated weights for policy 0, policy_version 897221 (0.0006) [2023-12-27 11:51:47,374][20381] Updated weights for policy 1, policy_version 896552 (0.0005) [2023-12-27 11:51:47,420][20347] Updated weights for policy 0, policy_version 897231 (0.0010) [2023-12-27 11:51:47,422][20381] Updated weights for policy 1, policy_version 896562 (0.0005) [2023-12-27 11:51:48,018][20381] Updated weights for policy 1, policy_version 896572 (0.0006) [2023-12-27 11:51:48,064][20347] Updated weights for policy 0, policy_version 897241 (0.0011) [2023-12-27 11:51:48,071][20381] Updated weights for policy 1, policy_version 896582 (0.0009) [2023-12-27 11:51:48,122][20347] Updated weights for policy 0, policy_version 897251 (0.0010) [2023-12-27 11:51:48,123][20381] Updated weights for policy 1, policy_version 896592 (0.0005) [2023-12-27 11:51:48,177][20347] Updated weights for policy 0, policy_version 897261 (0.0010) [2023-12-27 11:51:48,806][20381] Updated weights for policy 1, policy_version 896602 (0.0005) [2023-12-27 11:51:48,861][20347] Updated weights for policy 0, policy_version 897271 (0.0007) [2023-12-27 11:51:48,869][20381] Updated weights for policy 1, policy_version 896612 (0.0008) [2023-12-27 11:51:48,913][20347] Updated weights for policy 0, policy_version 897281 (0.0006) [2023-12-27 11:51:48,928][20381] Updated weights for policy 1, policy_version 896622 (0.0008) [2023-12-27 11:51:48,971][20347] Updated weights for policy 0, policy_version 897291 (0.0005) [2023-12-27 11:51:48,990][20381] Updated weights for policy 1, policy_version 896632 (0.0007) [2023-12-27 11:51:49,646][20347] Updated weights for policy 0, policy_version 897301 (0.0007) [2023-12-27 11:51:49,667][20381] Updated weights for policy 1, policy_version 896642 (0.0007) [2023-12-27 11:51:49,692][20347] Updated weights for policy 0, policy_version 897311 (0.0006) [2023-12-27 11:51:49,726][20381] Updated weights for policy 1, policy_version 896652 (0.0006) [2023-12-27 11:51:49,739][20347] Updated weights for policy 0, policy_version 897321 (0.0007) [2023-12-27 11:51:49,791][20381] Updated weights for policy 1, policy_version 896662 (0.0007) [2023-12-27 11:51:50,416][20347] Updated weights for policy 0, policy_version 897331 (0.0007) [2023-12-27 11:51:50,465][20347] Updated weights for policy 0, policy_version 897341 (0.0007) [2023-12-27 11:51:50,512][20347] Updated weights for policy 0, policy_version 897351 (0.0006) [2023-12-27 11:51:50,528][20381] Updated weights for policy 1, policy_version 896672 (0.0008) [2023-12-27 11:51:50,587][20381] Updated weights for policy 1, policy_version 896682 (0.0008) [2023-12-27 11:51:50,641][20381] Updated weights for policy 1, policy_version 896692 (0.0007) [2023-12-27 11:51:51,160][20347] Updated weights for policy 0, policy_version 897361 (0.0009) [2023-12-27 11:51:51,216][20347] Updated weights for policy 0, policy_version 897371 (0.0006) [2023-12-27 11:51:51,279][20347] Updated weights for policy 0, policy_version 897381 (0.0008) [2023-12-27 11:51:51,339][20347] Updated weights for policy 0, policy_version 897391 (0.0007) [2023-12-27 11:51:51,389][20381] Updated weights for policy 1, policy_version 896702 (0.0009) [2023-12-27 11:51:51,457][20381] Updated weights for policy 1, policy_version 896712 (0.0007) [2023-12-27 11:51:51,524][20381] Updated weights for policy 1, policy_version 896722 (0.0007) [2023-12-27 11:51:52,064][20347] Updated weights for policy 0, policy_version 897401 (0.0007) [2023-12-27 11:51:52,124][20347] Updated weights for policy 0, policy_version 897411 (0.0006) [2023-12-27 11:51:52,188][20347] Updated weights for policy 0, policy_version 897421 (0.0007) [2023-12-27 11:51:52,192][20381] Updated weights for policy 1, policy_version 896732 (0.0007) [2023-12-27 11:51:52,249][20381] Updated weights for policy 1, policy_version 896742 (0.0007) [2023-12-27 11:51:52,315][20381] Updated weights for policy 1, policy_version 896752 (0.0006) [2023-12-27 11:51:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20105.1). Total num frames: 459358208. Throughput: 0: 10176.0, 1: 10220.5. Samples: 459330500. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:52,363][19170] Avg episode reward: [(0, '824.970'), (1, '1000.000')] [2023-12-27 11:51:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000897424_229769216.pth... [2023-12-27 11:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000896760_229597184.pth... [2023-12-27 11:51:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000896208_229457920.pth [2023-12-27 11:51:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000895544_229285888.pth [2023-12-27 11:51:52,867][20347] Updated weights for policy 0, policy_version 897431 (0.0008) [2023-12-27 11:51:52,922][20347] Updated weights for policy 0, policy_version 897441 (0.0007) [2023-12-27 11:51:52,977][20347] Updated weights for policy 0, policy_version 897451 (0.0008) [2023-12-27 11:51:52,983][20381] Updated weights for policy 1, policy_version 896762 (0.0008) [2023-12-27 11:51:53,037][20381] Updated weights for policy 1, policy_version 896772 (0.0007) [2023-12-27 11:51:53,102][20381] Updated weights for policy 1, policy_version 896782 (0.0007) [2023-12-27 11:51:53,155][20381] Updated weights for policy 1, policy_version 896792 (0.0005) [2023-12-27 11:51:53,594][20347] Updated weights for policy 0, policy_version 897461 (0.0006) [2023-12-27 11:51:53,640][20347] Updated weights for policy 0, policy_version 897471 (0.0007) [2023-12-27 11:51:53,685][20347] Updated weights for policy 0, policy_version 897481 (0.0007) [2023-12-27 11:51:53,804][20381] Updated weights for policy 1, policy_version 896802 (0.0008) [2023-12-27 11:51:53,864][20381] Updated weights for policy 1, policy_version 896812 (0.0008) [2023-12-27 11:51:53,923][20381] Updated weights for policy 1, policy_version 896822 (0.0007) [2023-12-27 11:51:54,374][20347] Updated weights for policy 0, policy_version 897491 (0.0007) [2023-12-27 11:51:54,432][20347] Updated weights for policy 0, policy_version 897501 (0.0008) [2023-12-27 11:51:54,492][20347] Updated weights for policy 0, policy_version 897511 (0.0007) [2023-12-27 11:51:54,636][20381] Updated weights for policy 1, policy_version 896832 (0.0006) [2023-12-27 11:51:54,686][20381] Updated weights for policy 1, policy_version 896842 (0.0005) [2023-12-27 11:51:54,740][20381] Updated weights for policy 1, policy_version 896852 (0.0005) [2023-12-27 11:51:55,150][20347] Updated weights for policy 0, policy_version 897521 (0.0007) [2023-12-27 11:51:55,202][20347] Updated weights for policy 0, policy_version 897531 (0.0005) [2023-12-27 11:51:55,263][20347] Updated weights for policy 0, policy_version 897541 (0.0006) [2023-12-27 11:51:55,320][20347] Updated weights for policy 0, policy_version 897551 (0.0007) [2023-12-27 11:51:55,337][20381] Updated weights for policy 1, policy_version 896862 (0.0007) [2023-12-27 11:51:55,383][20381] Updated weights for policy 1, policy_version 896872 (0.0008) [2023-12-27 11:51:55,441][20381] Updated weights for policy 1, policy_version 896882 (0.0009) [2023-12-27 11:51:55,867][20347] Updated weights for policy 0, policy_version 897561 (0.0007) [2023-12-27 11:51:55,924][20347] Updated weights for policy 0, policy_version 897571 (0.0007) [2023-12-27 11:51:55,978][20347] Updated weights for policy 0, policy_version 897581 (0.0007) [2023-12-27 11:51:56,140][20381] Updated weights for policy 1, policy_version 896892 (0.0009) [2023-12-27 11:51:56,199][20381] Updated weights for policy 1, policy_version 896902 (0.0011) [2023-12-27 11:51:56,246][20381] Updated weights for policy 1, policy_version 896912 (0.0010) [2023-12-27 11:51:56,570][20347] Updated weights for policy 0, policy_version 897591 (0.0006) [2023-12-27 11:51:56,622][20347] Updated weights for policy 0, policy_version 897601 (0.0006) [2023-12-27 11:51:56,675][20347] Updated weights for policy 0, policy_version 897611 (0.0007) [2023-12-27 11:51:56,865][20381] Updated weights for policy 1, policy_version 896922 (0.0009) [2023-12-27 11:51:56,916][20381] Updated weights for policy 1, policy_version 896932 (0.0010) [2023-12-27 11:51:56,964][20381] Updated weights for policy 1, policy_version 896942 (0.0010) [2023-12-27 11:51:57,012][20381] Updated weights for policy 1, policy_version 896952 (0.0010) [2023-12-27 11:51:57,343][20347] Updated weights for policy 0, policy_version 897621 (0.0008) [2023-12-27 11:51:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20160.7). Total num frames: 459464704. Throughput: 0: 10222.4, 1: 10254.9. Samples: 459454324. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:51:57,363][19170] Avg episode reward: [(0, '873.350'), (1, '1000.000')] [2023-12-27 11:51:57,390][20347] Updated weights for policy 0, policy_version 897631 (0.0006) [2023-12-27 11:51:57,445][20347] Updated weights for policy 0, policy_version 897641 (0.0005) [2023-12-27 11:51:57,710][20381] Updated weights for policy 1, policy_version 896962 (0.0011) [2023-12-27 11:51:57,768][20381] Updated weights for policy 1, policy_version 896972 (0.0011) [2023-12-27 11:51:57,829][20381] Updated weights for policy 1, policy_version 896982 (0.0011) [2023-12-27 11:51:58,043][20347] Updated weights for policy 0, policy_version 897651 (0.0006) [2023-12-27 11:51:58,098][20347] Updated weights for policy 0, policy_version 897661 (0.0007) [2023-12-27 11:51:58,153][20347] Updated weights for policy 0, policy_version 897671 (0.0007) [2023-12-27 11:51:58,575][20381] Updated weights for policy 1, policy_version 896992 (0.0011) [2023-12-27 11:51:58,639][20381] Updated weights for policy 1, policy_version 897002 (0.0011) [2023-12-27 11:51:58,697][20381] Updated weights for policy 1, policy_version 897012 (0.0009) [2023-12-27 11:51:59,019][20347] Updated weights for policy 0, policy_version 897681 (0.0008) [2023-12-27 11:51:59,067][20347] Updated weights for policy 0, policy_version 897691 (0.0009) [2023-12-27 11:51:59,115][20347] Updated weights for policy 0, policy_version 897701 (0.0008) [2023-12-27 11:51:59,163][20347] Updated weights for policy 0, policy_version 897711 (0.0008) [2023-12-27 11:51:59,478][20381] Updated weights for policy 1, policy_version 897022 (0.0008) [2023-12-27 11:51:59,531][20381] Updated weights for policy 1, policy_version 897032 (0.0007) [2023-12-27 11:51:59,589][20381] Updated weights for policy 1, policy_version 897042 (0.0006) [2023-12-27 11:51:59,921][20347] Updated weights for policy 0, policy_version 897721 (0.0007) [2023-12-27 11:51:59,987][20347] Updated weights for policy 0, policy_version 897731 (0.0007) [2023-12-27 11:52:00,049][20347] Updated weights for policy 0, policy_version 897741 (0.0009) [2023-12-27 11:52:00,270][20381] Updated weights for policy 1, policy_version 897052 (0.0007) [2023-12-27 11:52:00,329][20381] Updated weights for policy 1, policy_version 897062 (0.0006) [2023-12-27 11:52:00,382][20381] Updated weights for policy 1, policy_version 897072 (0.0007) [2023-12-27 11:52:00,765][20347] Updated weights for policy 0, policy_version 897751 (0.0008) [2023-12-27 11:52:00,812][20347] Updated weights for policy 0, policy_version 897761 (0.0009) [2023-12-27 11:52:00,860][20347] Updated weights for policy 0, policy_version 897771 (0.0007) [2023-12-27 11:52:01,022][20381] Updated weights for policy 1, policy_version 897082 (0.0007) [2023-12-27 11:52:01,084][20381] Updated weights for policy 1, policy_version 897092 (0.0008) [2023-12-27 11:52:01,149][20381] Updated weights for policy 1, policy_version 897102 (0.0009) [2023-12-27 11:52:01,213][20381] Updated weights for policy 1, policy_version 897112 (0.0007) [2023-12-27 11:52:01,605][20347] Updated weights for policy 0, policy_version 897781 (0.0008) [2023-12-27 11:52:01,666][20347] Updated weights for policy 0, policy_version 897791 (0.0008) [2023-12-27 11:52:01,731][20347] Updated weights for policy 0, policy_version 897801 (0.0007) [2023-12-27 11:52:01,930][20381] Updated weights for policy 1, policy_version 897122 (0.0008) [2023-12-27 11:52:01,984][20381] Updated weights for policy 1, policy_version 897132 (0.0008) [2023-12-27 11:52:02,047][20381] Updated weights for policy 1, policy_version 897142 (0.0007) [2023-12-27 11:52:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20216.2). Total num frames: 459563008. Throughput: 0: 10216.4, 1: 10215.0. Samples: 459573504. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:52:02,363][19170] Avg episode reward: [(0, '961.170'), (1, '1000.000')] [2023-12-27 11:52:02,370][20347] Updated weights for policy 0, policy_version 897811 (0.0008) [2023-12-27 11:52:02,435][20347] Updated weights for policy 0, policy_version 897821 (0.0008) [2023-12-27 11:52:02,494][20347] Updated weights for policy 0, policy_version 897831 (0.0007) [2023-12-27 11:52:02,747][20381] Updated weights for policy 1, policy_version 897152 (0.0007) [2023-12-27 11:52:02,796][20381] Updated weights for policy 1, policy_version 897162 (0.0007) [2023-12-27 11:52:02,849][20381] Updated weights for policy 1, policy_version 897172 (0.0007) [2023-12-27 11:52:03,144][20347] Updated weights for policy 0, policy_version 897841 (0.0006) [2023-12-27 11:52:03,201][20347] Updated weights for policy 0, policy_version 897851 (0.0010) [2023-12-27 11:52:03,252][20347] Updated weights for policy 0, policy_version 897861 (0.0010) [2023-12-27 11:52:03,296][20347] Updated weights for policy 0, policy_version 897871 (0.0010) [2023-12-27 11:52:03,496][20381] Updated weights for policy 1, policy_version 897182 (0.0006) [2023-12-27 11:52:03,553][20381] Updated weights for policy 1, policy_version 897192 (0.0007) [2023-12-27 11:52:03,615][20381] Updated weights for policy 1, policy_version 897202 (0.0005) [2023-12-27 11:52:03,947][20347] Updated weights for policy 0, policy_version 897881 (0.0007) [2023-12-27 11:52:04,003][20347] Updated weights for policy 0, policy_version 897891 (0.0007) [2023-12-27 11:52:04,065][20347] Updated weights for policy 0, policy_version 897901 (0.0007) [2023-12-27 11:52:04,258][20381] Updated weights for policy 1, policy_version 897212 (0.0005) [2023-12-27 11:52:04,317][20381] Updated weights for policy 1, policy_version 897222 (0.0005) [2023-12-27 11:52:04,379][20381] Updated weights for policy 1, policy_version 897232 (0.0006) [2023-12-27 11:52:04,702][20347] Updated weights for policy 0, policy_version 897911 (0.0005) [2023-12-27 11:52:04,755][20347] Updated weights for policy 0, policy_version 897921 (0.0005) [2023-12-27 11:52:04,806][20347] Updated weights for policy 0, policy_version 897931 (0.0005) [2023-12-27 11:52:04,975][20381] Updated weights for policy 1, policy_version 897242 (0.0009) [2023-12-27 11:52:05,022][20381] Updated weights for policy 1, policy_version 897252 (0.0005) [2023-12-27 11:52:05,068][20381] Updated weights for policy 1, policy_version 897262 (0.0005) [2023-12-27 11:52:05,123][20381] Updated weights for policy 1, policy_version 897272 (0.0006) [2023-12-27 11:52:05,466][20347] Updated weights for policy 0, policy_version 897941 (0.0007) [2023-12-27 11:52:05,525][20347] Updated weights for policy 0, policy_version 897951 (0.0008) [2023-12-27 11:52:05,577][20347] Updated weights for policy 0, policy_version 897961 (0.0009) [2023-12-27 11:52:05,701][20381] Updated weights for policy 1, policy_version 897282 (0.0008) [2023-12-27 11:52:05,762][20381] Updated weights for policy 1, policy_version 897292 (0.0008) [2023-12-27 11:52:05,823][20381] Updated weights for policy 1, policy_version 897302 (0.0008) [2023-12-27 11:52:06,272][20347] Updated weights for policy 0, policy_version 897971 (0.0008) [2023-12-27 11:52:06,334][20347] Updated weights for policy 0, policy_version 897981 (0.0008) [2023-12-27 11:52:06,395][20347] Updated weights for policy 0, policy_version 897991 (0.0008) [2023-12-27 11:52:06,557][20381] Updated weights for policy 1, policy_version 897312 (0.0008) [2023-12-27 11:52:06,609][20381] Updated weights for policy 1, policy_version 897322 (0.0008) [2023-12-27 11:52:06,662][20381] Updated weights for policy 1, policy_version 897332 (0.0007) [2023-12-27 11:52:07,029][20347] Updated weights for policy 0, policy_version 898001 (0.0008) [2023-12-27 11:52:07,091][20347] Updated weights for policy 0, policy_version 898011 (0.0008) [2023-12-27 11:52:07,152][20347] Updated weights for policy 0, policy_version 898021 (0.0008) [2023-12-27 11:52:07,207][20347] Updated weights for policy 0, policy_version 898031 (0.0008) [2023-12-27 11:52:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20479.9, 300 sec: 20244.2). Total num frames: 459669504. Throughput: 0: 10240.0, 1: 10251.0. Samples: 459638136. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:52:07,364][19170] Avg episode reward: [(0, '912.810'), (1, '1000.000')] [2023-12-27 11:52:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000898032_229924864.pth... [2023-12-27 11:52:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000896816_229613568.pth [2023-12-27 11:52:07,403][20381] Updated weights for policy 1, policy_version 897342 (0.0009) [2023-12-27 11:52:07,453][20381] Updated weights for policy 1, policy_version 897352 (0.0010) [2023-12-27 11:52:07,505][20381] Updated weights for policy 1, policy_version 897362 (0.0010) [2023-12-27 11:52:07,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000897368_229752832.pth... [2023-12-27 11:52:07,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000896152_229441536.pth [2023-12-27 11:52:07,815][20347] Updated weights for policy 0, policy_version 898041 (0.0010) [2023-12-27 11:52:07,869][20347] Updated weights for policy 0, policy_version 898051 (0.0010) [2023-12-27 11:52:07,928][20347] Updated weights for policy 0, policy_version 898061 (0.0007) [2023-12-27 11:52:08,125][20381] Updated weights for policy 1, policy_version 897372 (0.0008) [2023-12-27 11:52:08,177][20381] Updated weights for policy 1, policy_version 897382 (0.0006) [2023-12-27 11:52:08,223][20381] Updated weights for policy 1, policy_version 897392 (0.0006) [2023-12-27 11:52:08,593][20347] Updated weights for policy 0, policy_version 898071 (0.0010) [2023-12-27 11:52:08,648][20347] Updated weights for policy 0, policy_version 898081 (0.0010) [2023-12-27 11:52:08,710][20347] Updated weights for policy 0, policy_version 898091 (0.0011) [2023-12-27 11:52:08,866][20381] Updated weights for policy 1, policy_version 897402 (0.0009) [2023-12-27 11:52:08,918][20381] Updated weights for policy 1, policy_version 897412 (0.0008) [2023-12-27 11:52:08,973][20381] Updated weights for policy 1, policy_version 897422 (0.0007) [2023-12-27 11:52:09,025][20381] Updated weights for policy 1, policy_version 897432 (0.0008) [2023-12-27 11:52:09,345][20347] Updated weights for policy 0, policy_version 898101 (0.0010) [2023-12-27 11:52:09,416][20347] Updated weights for policy 0, policy_version 898111 (0.0009) [2023-12-27 11:52:09,470][20347] Updated weights for policy 0, policy_version 898121 (0.0006) [2023-12-27 11:52:09,750][20381] Updated weights for policy 1, policy_version 897442 (0.0007) [2023-12-27 11:52:09,804][20381] Updated weights for policy 1, policy_version 897452 (0.0007) [2023-12-27 11:52:09,867][20381] Updated weights for policy 1, policy_version 897462 (0.0008) [2023-12-27 11:52:10,180][20347] Updated weights for policy 0, policy_version 898131 (0.0008) [2023-12-27 11:52:10,236][20347] Updated weights for policy 0, policy_version 898141 (0.0007) [2023-12-27 11:52:10,292][20347] Updated weights for policy 0, policy_version 898151 (0.0008) [2023-12-27 11:52:10,554][20381] Updated weights for policy 1, policy_version 897472 (0.0008) [2023-12-27 11:52:10,611][20381] Updated weights for policy 1, policy_version 897482 (0.0007) [2023-12-27 11:52:10,665][20381] Updated weights for policy 1, policy_version 897492 (0.0006) [2023-12-27 11:52:11,068][20347] Updated weights for policy 0, policy_version 898161 (0.0008) [2023-12-27 11:52:11,143][20347] Updated weights for policy 0, policy_version 898171 (0.0010) [2023-12-27 11:52:11,200][20347] Updated weights for policy 0, policy_version 898181 (0.0009) [2023-12-27 11:52:11,264][20347] Updated weights for policy 0, policy_version 898191 (0.0009) [2023-12-27 11:52:11,360][20381] Updated weights for policy 1, policy_version 897502 (0.0007) [2023-12-27 11:52:11,423][20381] Updated weights for policy 1, policy_version 897512 (0.0007) [2023-12-27 11:52:11,479][20381] Updated weights for policy 1, policy_version 897522 (0.0008) [2023-12-27 11:52:11,991][20347] Updated weights for policy 0, policy_version 898201 (0.0008) [2023-12-27 11:52:12,050][20347] Updated weights for policy 0, policy_version 898211 (0.0008) [2023-12-27 11:52:12,107][20347] Updated weights for policy 0, policy_version 898221 (0.0007) [2023-12-27 11:52:12,187][20381] Updated weights for policy 1, policy_version 897532 (0.0007) [2023-12-27 11:52:12,251][20381] Updated weights for policy 1, policy_version 897542 (0.0006) [2023-12-27 11:52:12,313][20381] Updated weights for policy 1, policy_version 897552 (0.0008) [2023-12-27 11:52:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 459776000. Throughput: 0: 10306.2, 1: 10290.6. Samples: 459761396. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:52:12,363][19170] Avg episode reward: [(0, '864.330'), (1, '1000.000')] [2023-12-27 11:52:12,788][20347] Updated weights for policy 0, policy_version 898231 (0.0008) [2023-12-27 11:52:12,846][20347] Updated weights for policy 0, policy_version 898241 (0.0008) [2023-12-27 11:52:12,929][20347] Updated weights for policy 0, policy_version 898251 (0.0008) [2023-12-27 11:52:12,982][20381] Updated weights for policy 1, policy_version 897562 (0.0008) [2023-12-27 11:52:13,049][20381] Updated weights for policy 1, policy_version 897572 (0.0007) [2023-12-27 11:52:13,109][20381] Updated weights for policy 1, policy_version 897582 (0.0008) [2023-12-27 11:52:13,161][20381] Updated weights for policy 1, policy_version 897592 (0.0008) [2023-12-27 11:52:13,540][20347] Updated weights for policy 0, policy_version 898261 (0.0008) [2023-12-27 11:52:13,600][20347] Updated weights for policy 0, policy_version 898271 (0.0008) [2023-12-27 11:52:13,663][20347] Updated weights for policy 0, policy_version 898281 (0.0008) [2023-12-27 11:52:13,877][20381] Updated weights for policy 1, policy_version 897602 (0.0005) [2023-12-27 11:52:13,890][20292] KL-divergence is very high: 496.4001 [2023-12-27 11:52:13,936][20381] Updated weights for policy 1, policy_version 897612 (0.0006) [2023-12-27 11:52:13,936][20292] KL-divergence is very high: 115.6005 [2023-12-27 11:52:14,004][20381] Updated weights for policy 1, policy_version 897622 (0.0011) [2023-12-27 11:52:14,334][20347] Updated weights for policy 0, policy_version 898291 (0.0008) [2023-12-27 11:52:14,388][20347] Updated weights for policy 0, policy_version 898301 (0.0007) [2023-12-27 11:52:14,442][20347] Updated weights for policy 0, policy_version 898311 (0.0007) [2023-12-27 11:52:14,652][20381] Updated weights for policy 1, policy_version 897632 (0.0010) [2023-12-27 11:52:14,701][20381] Updated weights for policy 1, policy_version 897642 (0.0010) [2023-12-27 11:52:14,746][20381] Updated weights for policy 1, policy_version 897652 (0.0010) [2023-12-27 11:52:15,088][20347] Updated weights for policy 0, policy_version 898321 (0.0007) [2023-12-27 11:52:15,149][20347] Updated weights for policy 0, policy_version 898331 (0.0007) [2023-12-27 11:52:15,216][20347] Updated weights for policy 0, policy_version 898341 (0.0009) [2023-12-27 11:52:15,282][20347] Updated weights for policy 0, policy_version 898351 (0.0006) [2023-12-27 11:52:15,434][20381] Updated weights for policy 1, policy_version 897662 (0.0010) [2023-12-27 11:52:15,488][20381] Updated weights for policy 1, policy_version 897672 (0.0010) [2023-12-27 11:52:15,543][20381] Updated weights for policy 1, policy_version 897682 (0.0010) [2023-12-27 11:52:15,853][20347] Updated weights for policy 0, policy_version 898361 (0.0008) [2023-12-27 11:52:15,901][20347] Updated weights for policy 0, policy_version 898371 (0.0005) [2023-12-27 11:52:15,955][20347] Updated weights for policy 0, policy_version 898381 (0.0005) [2023-12-27 11:52:16,227][20381] Updated weights for policy 1, policy_version 897692 (0.0009) [2023-12-27 11:52:16,274][20381] Updated weights for policy 1, policy_version 897702 (0.0007) [2023-12-27 11:52:16,323][20381] Updated weights for policy 1, policy_version 897712 (0.0009) [2023-12-27 11:52:16,551][20347] Updated weights for policy 0, policy_version 898391 (0.0007) [2023-12-27 11:52:16,611][20347] Updated weights for policy 0, policy_version 898401 (0.0007) [2023-12-27 11:52:16,670][20347] Updated weights for policy 0, policy_version 898411 (0.0008) [2023-12-27 11:52:16,974][20381] Updated weights for policy 1, policy_version 897722 (0.0008) [2023-12-27 11:52:17,029][20381] Updated weights for policy 1, policy_version 897732 (0.0005) [2023-12-27 11:52:17,084][20381] Updated weights for policy 1, policy_version 897742 (0.0005) [2023-12-27 11:52:17,140][20381] Updated weights for policy 1, policy_version 897752 (0.0007) [2023-12-27 11:52:17,311][20347] Updated weights for policy 0, policy_version 898421 (0.0008) [2023-12-27 11:52:17,359][20347] Updated weights for policy 0, policy_version 898431 (0.0005) [2023-12-27 11:52:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 459874304. Throughput: 0: 10374.6, 1: 10252.7. Samples: 459886072. Policy #0 lag: (min: 29.0, avg: 43.7, max: 61.0) [2023-12-27 11:52:17,363][19170] Avg episode reward: [(0, '854.470'), (1, '990.370')] [2023-12-27 11:52:17,412][20347] Updated weights for policy 0, policy_version 898441 (0.0005) [2023-12-27 11:52:17,750][20381] Updated weights for policy 1, policy_version 897762 (0.0011) [2023-12-27 11:52:17,817][20381] Updated weights for policy 1, policy_version 897772 (0.0010) [2023-12-27 11:52:17,884][20381] Updated weights for policy 1, policy_version 897782 (0.0011) [2023-12-27 11:52:17,991][20347] Updated weights for policy 0, policy_version 898451 (0.0006) [2023-12-27 11:52:18,044][20347] Updated weights for policy 0, policy_version 898461 (0.0008) [2023-12-27 11:52:18,101][20347] Updated weights for policy 0, policy_version 898471 (0.0007) [2023-12-27 11:52:18,546][20381] Updated weights for policy 1, policy_version 897792 (0.0010) [2023-12-27 11:52:18,630][20381] Updated weights for policy 1, policy_version 897802 (0.0010) [2023-12-27 11:52:18,691][20381] Updated weights for policy 1, policy_version 897812 (0.0011) [2023-12-27 11:52:18,830][20347] Updated weights for policy 0, policy_version 898481 (0.0008) [2023-12-27 11:52:18,883][20347] Updated weights for policy 0, policy_version 898491 (0.0011) [2023-12-27 11:52:18,941][20347] Updated weights for policy 0, policy_version 898501 (0.0010) [2023-12-27 11:52:19,003][20347] Updated weights for policy 0, policy_version 898511 (0.0010) [2023-12-27 11:52:19,347][20381] Updated weights for policy 1, policy_version 897822 (0.0011) [2023-12-27 11:52:19,382][20292] KL-divergence is very high: 199.1179 [2023-12-27 11:52:19,416][20381] Updated weights for policy 1, policy_version 897832 (0.0011) [2023-12-27 11:52:19,479][20381] Updated weights for policy 1, policy_version 897842 (0.0010) [2023-12-27 11:52:19,709][20347] Updated weights for policy 0, policy_version 898521 (0.0008) [2023-12-27 11:52:19,772][20347] Updated weights for policy 0, policy_version 898531 (0.0006) [2023-12-27 11:52:19,839][20347] Updated weights for policy 0, policy_version 898541 (0.0007) [2023-12-27 11:52:20,225][20381] Updated weights for policy 1, policy_version 897852 (0.0011) [2023-12-27 11:52:20,286][20381] Updated weights for policy 1, policy_version 897862 (0.0008) [2023-12-27 11:52:20,349][20381] Updated weights for policy 1, policy_version 897872 (0.0007) [2023-12-27 11:52:20,521][20347] Updated weights for policy 0, policy_version 898551 (0.0008) [2023-12-27 11:52:20,577][20347] Updated weights for policy 0, policy_version 898561 (0.0008) [2023-12-27 11:52:20,637][20347] Updated weights for policy 0, policy_version 898571 (0.0008) [2023-12-27 11:52:20,968][20381] Updated weights for policy 1, policy_version 897882 (0.0008) [2023-12-27 11:52:21,032][20381] Updated weights for policy 1, policy_version 897892 (0.0006) [2023-12-27 11:52:21,098][20381] Updated weights for policy 1, policy_version 897902 (0.0008) [2023-12-27 11:52:21,164][20381] Updated weights for policy 1, policy_version 897912 (0.0008) [2023-12-27 11:52:21,434][20347] Updated weights for policy 0, policy_version 898581 (0.0008) [2023-12-27 11:52:21,494][20347] Updated weights for policy 0, policy_version 898591 (0.0007) [2023-12-27 11:52:21,556][20347] Updated weights for policy 0, policy_version 898601 (0.0008) [2023-12-27 11:52:21,891][20381] Updated weights for policy 1, policy_version 897922 (0.0006) [2023-12-27 11:52:21,955][20381] Updated weights for policy 1, policy_version 897932 (0.0006) [2023-12-27 11:52:22,017][20381] Updated weights for policy 1, policy_version 897942 (0.0007) [2023-12-27 11:52:22,281][20347] Updated weights for policy 0, policy_version 898611 (0.0007) [2023-12-27 11:52:22,344][20347] Updated weights for policy 0, policy_version 898621 (0.0008) [2023-12-27 11:52:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 459972608. Throughput: 0: 10331.7, 1: 10251.3. Samples: 459946828. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:22,363][19170] Avg episode reward: [(0, '893.050'), (1, '971.170')] [2023-12-27 11:52:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000897944_229900288.pth... [2023-12-27 11:52:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000896760_229597184.pth [2023-12-27 11:52:22,404][20347] Updated weights for policy 0, policy_version 898631 (0.0008) [2023-12-27 11:52:22,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000898640_230080512.pth... [2023-12-27 11:52:22,469][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000897424_229769216.pth [2023-12-27 11:52:22,702][20381] Updated weights for policy 1, policy_version 897952 (0.0008) [2023-12-27 11:52:22,753][20381] Updated weights for policy 1, policy_version 897962 (0.0010) [2023-12-27 11:52:22,804][20381] Updated weights for policy 1, policy_version 897972 (0.0009) [2023-12-27 11:52:23,025][20347] Updated weights for policy 0, policy_version 898641 (0.0008) [2023-12-27 11:52:23,055][20291] KL-divergence is very high: 115.5377 [2023-12-27 11:52:23,068][20291] KL-divergence is very high: 107.8892 [2023-12-27 11:52:23,086][20347] Updated weights for policy 0, policy_version 898651 (0.0007) [2023-12-27 11:52:23,143][20347] Updated weights for policy 0, policy_version 898661 (0.0007) [2023-12-27 11:52:23,201][20347] Updated weights for policy 0, policy_version 898671 (0.0010) [2023-12-27 11:52:23,581][20381] Updated weights for policy 1, policy_version 897982 (0.0009) [2023-12-27 11:52:23,628][20381] Updated weights for policy 1, policy_version 897992 (0.0010) [2023-12-27 11:52:23,675][20381] Updated weights for policy 1, policy_version 898002 (0.0009) [2023-12-27 11:52:23,797][20347] Updated weights for policy 0, policy_version 898681 (0.0010) [2023-12-27 11:52:23,855][20347] Updated weights for policy 0, policy_version 898691 (0.0010) [2023-12-27 11:52:23,909][20347] Updated weights for policy 0, policy_version 898701 (0.0010) [2023-12-27 11:52:24,420][20381] Updated weights for policy 1, policy_version 898012 (0.0008) [2023-12-27 11:52:24,472][20381] Updated weights for policy 1, policy_version 898022 (0.0008) [2023-12-27 11:52:24,522][20381] Updated weights for policy 1, policy_version 898032 (0.0007) [2023-12-27 11:52:24,542][20347] Updated weights for policy 0, policy_version 898711 (0.0010) [2023-12-27 11:52:24,598][20347] Updated weights for policy 0, policy_version 898721 (0.0010) [2023-12-27 11:52:24,652][20347] Updated weights for policy 0, policy_version 898731 (0.0010) [2023-12-27 11:52:25,249][20381] Updated weights for policy 1, policy_version 898042 (0.0006) [2023-12-27 11:52:25,303][20381] Updated weights for policy 1, policy_version 898052 (0.0007) [2023-12-27 11:52:25,314][20347] Updated weights for policy 0, policy_version 898741 (0.0010) [2023-12-27 11:52:25,359][20381] Updated weights for policy 1, policy_version 898062 (0.0006) [2023-12-27 11:52:25,368][20347] Updated weights for policy 0, policy_version 898751 (0.0010) [2023-12-27 11:52:25,411][20381] Updated weights for policy 1, policy_version 898072 (0.0005) [2023-12-27 11:52:25,416][20347] Updated weights for policy 0, policy_version 898761 (0.0010) [2023-12-27 11:52:26,088][20381] Updated weights for policy 1, policy_version 898082 (0.0007) [2023-12-27 11:52:26,100][20347] Updated weights for policy 0, policy_version 898771 (0.0010) [2023-12-27 11:52:26,136][20381] Updated weights for policy 1, policy_version 898092 (0.0006) [2023-12-27 11:52:26,155][20347] Updated weights for policy 0, policy_version 898781 (0.0010) [2023-12-27 11:52:26,188][20381] Updated weights for policy 1, policy_version 898102 (0.0008) [2023-12-27 11:52:26,206][20347] Updated weights for policy 0, policy_version 898791 (0.0010) [2023-12-27 11:52:26,842][20347] Updated weights for policy 0, policy_version 898801 (0.0007) [2023-12-27 11:52:26,897][20347] Updated weights for policy 0, policy_version 898811 (0.0010) [2023-12-27 11:52:26,914][20381] Updated weights for policy 1, policy_version 898112 (0.0006) [2023-12-27 11:52:26,946][20347] Updated weights for policy 0, policy_version 898821 (0.0010) [2023-12-27 11:52:26,966][20381] Updated weights for policy 1, policy_version 898122 (0.0005) [2023-12-27 11:52:27,002][20347] Updated weights for policy 0, policy_version 898831 (0.0010) [2023-12-27 11:52:27,021][20381] Updated weights for policy 1, policy_version 898132 (0.0007) [2023-12-27 11:52:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20327.3). Total num frames: 460079104. Throughput: 0: 10350.1, 1: 10172.6. Samples: 460066752. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:27,363][19170] Avg episode reward: [(0, '844.240'), (1, '971.170')] [2023-12-27 11:52:27,689][20347] Updated weights for policy 0, policy_version 898841 (0.0010) [2023-12-27 11:52:27,734][20347] Updated weights for policy 0, policy_version 898851 (0.0008) [2023-12-27 11:52:27,736][20381] Updated weights for policy 1, policy_version 898142 (0.0008) [2023-12-27 11:52:27,775][20292] KL-divergence is very high: 102.4521 [2023-12-27 11:52:27,778][20347] Updated weights for policy 0, policy_version 898861 (0.0010) [2023-12-27 11:52:27,786][20381] Updated weights for policy 1, policy_version 898152 (0.0008) [2023-12-27 11:52:27,838][20381] Updated weights for policy 1, policy_version 898162 (0.0008) [2023-12-27 11:52:28,386][20347] Updated weights for policy 0, policy_version 898871 (0.0010) [2023-12-27 11:52:28,441][20347] Updated weights for policy 0, policy_version 898881 (0.0010) [2023-12-27 11:52:28,496][20347] Updated weights for policy 0, policy_version 898891 (0.0010) [2023-12-27 11:52:28,655][20381] Updated weights for policy 1, policy_version 898172 (0.0008) [2023-12-27 11:52:28,710][20381] Updated weights for policy 1, policy_version 898182 (0.0007) [2023-12-27 11:52:28,766][20381] Updated weights for policy 1, policy_version 898192 (0.0007) [2023-12-27 11:52:29,163][20347] Updated weights for policy 0, policy_version 898901 (0.0011) [2023-12-27 11:52:29,221][20347] Updated weights for policy 0, policy_version 898911 (0.0010) [2023-12-27 11:52:29,282][20347] Updated weights for policy 0, policy_version 898921 (0.0008) [2023-12-27 11:52:29,520][20381] Updated weights for policy 1, policy_version 898202 (0.0007) [2023-12-27 11:52:29,583][20381] Updated weights for policy 1, policy_version 898212 (0.0007) [2023-12-27 11:52:29,645][20381] Updated weights for policy 1, policy_version 898222 (0.0007) [2023-12-27 11:52:29,710][20381] Updated weights for policy 1, policy_version 898232 (0.0009) [2023-12-27 11:52:29,908][20347] Updated weights for policy 0, policy_version 898931 (0.0007) [2023-12-27 11:52:29,976][20347] Updated weights for policy 0, policy_version 898941 (0.0007) [2023-12-27 11:52:30,044][20347] Updated weights for policy 0, policy_version 898951 (0.0006) [2023-12-27 11:52:30,443][20381] Updated weights for policy 1, policy_version 898242 (0.0007) [2023-12-27 11:52:30,489][20381] Updated weights for policy 1, policy_version 898252 (0.0005) [2023-12-27 11:52:30,534][20381] Updated weights for policy 1, policy_version 898262 (0.0008) [2023-12-27 11:52:30,594][20347] Updated weights for policy 0, policy_version 898961 (0.0008) [2023-12-27 11:52:30,649][20347] Updated weights for policy 0, policy_version 898971 (0.0007) [2023-12-27 11:52:30,696][20347] Updated weights for policy 0, policy_version 898981 (0.0008) [2023-12-27 11:52:30,739][20347] Updated weights for policy 0, policy_version 898991 (0.0008) [2023-12-27 11:52:31,115][20381] Updated weights for policy 1, policy_version 898272 (0.0008) [2023-12-27 11:52:31,175][20381] Updated weights for policy 1, policy_version 898282 (0.0008) [2023-12-27 11:52:31,226][20381] Updated weights for policy 1, policy_version 898292 (0.0008) [2023-12-27 11:52:31,494][20347] Updated weights for policy 0, policy_version 899001 (0.0009) [2023-12-27 11:52:31,542][20347] Updated weights for policy 0, policy_version 899011 (0.0008) [2023-12-27 11:52:31,591][20347] Updated weights for policy 0, policy_version 899021 (0.0009) [2023-12-27 11:52:31,945][20381] Updated weights for policy 1, policy_version 898302 (0.0009) [2023-12-27 11:52:32,001][20381] Updated weights for policy 1, policy_version 898312 (0.0010) [2023-12-27 11:52:32,053][20381] Updated weights for policy 1, policy_version 898322 (0.0009) [2023-12-27 11:52:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 460177408. Throughput: 0: 10355.3, 1: 10103.0. Samples: 460188976. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:32,363][19170] Avg episode reward: [(0, '863.790'), (1, '991.050')] [2023-12-27 11:52:32,381][20347] Updated weights for policy 0, policy_version 899031 (0.0007) [2023-12-27 11:52:32,449][20347] Updated weights for policy 0, policy_version 899041 (0.0010) [2023-12-27 11:52:32,506][20347] Updated weights for policy 0, policy_version 899051 (0.0009) [2023-12-27 11:52:32,816][20381] Updated weights for policy 1, policy_version 898332 (0.0010) [2023-12-27 11:52:32,873][20381] Updated weights for policy 1, policy_version 898342 (0.0008) [2023-12-27 11:52:32,930][20381] Updated weights for policy 1, policy_version 898352 (0.0008) [2023-12-27 11:52:33,075][20347] Updated weights for policy 0, policy_version 899061 (0.0008) [2023-12-27 11:52:33,122][20291] KL-divergence is very high: 107.0867 [2023-12-27 11:52:33,134][20347] Updated weights for policy 0, policy_version 899071 (0.0008) [2023-12-27 11:52:33,171][20291] KL-divergence is very high: 102.3558 [2023-12-27 11:52:33,197][20347] Updated weights for policy 0, policy_version 899081 (0.0008) [2023-12-27 11:52:33,637][20381] Updated weights for policy 1, policy_version 898362 (0.0008) [2023-12-27 11:52:33,693][20381] Updated weights for policy 1, policy_version 898372 (0.0005) [2023-12-27 11:52:33,751][20381] Updated weights for policy 1, policy_version 898382 (0.0005) [2023-12-27 11:52:33,794][20381] Updated weights for policy 1, policy_version 898392 (0.0005) [2023-12-27 11:52:33,850][20347] Updated weights for policy 0, policy_version 899091 (0.0006) [2023-12-27 11:52:33,903][20347] Updated weights for policy 0, policy_version 899101 (0.0006) [2023-12-27 11:52:33,958][20347] Updated weights for policy 0, policy_version 899111 (0.0006) [2023-12-27 11:52:34,459][20381] Updated weights for policy 1, policy_version 898402 (0.0008) [2023-12-27 11:52:34,512][20381] Updated weights for policy 1, policy_version 898412 (0.0008) [2023-12-27 11:52:34,562][20381] Updated weights for policy 1, policy_version 898422 (0.0007) [2023-12-27 11:52:34,593][20347] Updated weights for policy 0, policy_version 899121 (0.0007) [2023-12-27 11:52:34,654][20347] Updated weights for policy 0, policy_version 899131 (0.0007) [2023-12-27 11:52:34,711][20347] Updated weights for policy 0, policy_version 899141 (0.0007) [2023-12-27 11:52:34,772][20347] Updated weights for policy 0, policy_version 899151 (0.0007) [2023-12-27 11:52:35,262][20381] Updated weights for policy 1, policy_version 898432 (0.0007) [2023-12-27 11:52:35,315][20381] Updated weights for policy 1, policy_version 898442 (0.0007) [2023-12-27 11:52:35,366][20381] Updated weights for policy 1, policy_version 898452 (0.0005) [2023-12-27 11:52:35,447][20347] Updated weights for policy 0, policy_version 899161 (0.0008) [2023-12-27 11:52:35,500][20347] Updated weights for policy 0, policy_version 899171 (0.0008) [2023-12-27 11:52:35,556][20347] Updated weights for policy 0, policy_version 899181 (0.0007) [2023-12-27 11:52:35,943][20381] Updated weights for policy 1, policy_version 898462 (0.0006) [2023-12-27 11:52:35,998][20381] Updated weights for policy 1, policy_version 898472 (0.0005) [2023-12-27 11:52:36,047][20381] Updated weights for policy 1, policy_version 898482 (0.0005) [2023-12-27 11:52:36,229][20347] Updated weights for policy 0, policy_version 899191 (0.0010) [2023-12-27 11:52:36,292][20347] Updated weights for policy 0, policy_version 899201 (0.0011) [2023-12-27 11:52:36,352][20347] Updated weights for policy 0, policy_version 899211 (0.0010) [2023-12-27 11:52:36,729][20381] Updated weights for policy 1, policy_version 898492 (0.0007) [2023-12-27 11:52:36,788][20381] Updated weights for policy 1, policy_version 898502 (0.0009) [2023-12-27 11:52:36,848][20381] Updated weights for policy 1, policy_version 898512 (0.0009) [2023-12-27 11:52:36,999][20347] Updated weights for policy 0, policy_version 899221 (0.0008) [2023-12-27 11:52:37,061][20347] Updated weights for policy 0, policy_version 899231 (0.0005) [2023-12-27 11:52:37,122][20347] Updated weights for policy 0, policy_version 899241 (0.0009) [2023-12-27 11:52:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 460283904. Throughput: 0: 10375.9, 1: 10087.2. Samples: 460251340. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:37,363][19170] Avg episode reward: [(0, '892.810'), (1, '990.300')] [2023-12-27 11:52:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000898520_230047744.pth... [2023-12-27 11:52:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000899248_230236160.pth... [2023-12-27 11:52:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000897368_229752832.pth [2023-12-27 11:52:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000898032_229924864.pth [2023-12-27 11:52:37,496][20381] Updated weights for policy 1, policy_version 898522 (0.0010) [2023-12-27 11:52:37,557][20381] Updated weights for policy 1, policy_version 898532 (0.0006) [2023-12-27 11:52:37,613][20381] Updated weights for policy 1, policy_version 898542 (0.0007) [2023-12-27 11:52:37,668][20381] Updated weights for policy 1, policy_version 898552 (0.0007) [2023-12-27 11:52:37,777][20347] Updated weights for policy 0, policy_version 899251 (0.0010) [2023-12-27 11:52:37,828][20347] Updated weights for policy 0, policy_version 899261 (0.0010) [2023-12-27 11:52:37,877][20347] Updated weights for policy 0, policy_version 899271 (0.0010) [2023-12-27 11:52:38,315][20381] Updated weights for policy 1, policy_version 898562 (0.0006) [2023-12-27 11:52:38,377][20381] Updated weights for policy 1, policy_version 898572 (0.0008) [2023-12-27 11:52:38,430][20381] Updated weights for policy 1, policy_version 898582 (0.0009) [2023-12-27 11:52:38,565][20347] Updated weights for policy 0, policy_version 899281 (0.0007) [2023-12-27 11:52:38,627][20347] Updated weights for policy 0, policy_version 899291 (0.0010) [2023-12-27 11:52:38,691][20347] Updated weights for policy 0, policy_version 899301 (0.0010) [2023-12-27 11:52:38,746][20347] Updated weights for policy 0, policy_version 899311 (0.0010) [2023-12-27 11:52:39,088][20381] Updated weights for policy 1, policy_version 898592 (0.0008) [2023-12-27 11:52:39,144][20381] Updated weights for policy 1, policy_version 898602 (0.0007) [2023-12-27 11:52:39,203][20381] Updated weights for policy 1, policy_version 898612 (0.0008) [2023-12-27 11:52:39,429][20347] Updated weights for policy 0, policy_version 899321 (0.0008) [2023-12-27 11:52:39,491][20347] Updated weights for policy 0, policy_version 899331 (0.0008) [2023-12-27 11:52:39,547][20347] Updated weights for policy 0, policy_version 899341 (0.0008) [2023-12-27 11:52:39,842][20381] Updated weights for policy 1, policy_version 898622 (0.0008) [2023-12-27 11:52:39,901][20381] Updated weights for policy 1, policy_version 898632 (0.0008) [2023-12-27 11:52:39,970][20381] Updated weights for policy 1, policy_version 898642 (0.0007) [2023-12-27 11:52:40,297][20347] Updated weights for policy 0, policy_version 899351 (0.0010) [2023-12-27 11:52:40,356][20347] Updated weights for policy 0, policy_version 899361 (0.0011) [2023-12-27 11:52:40,412][20347] Updated weights for policy 0, policy_version 899371 (0.0011) [2023-12-27 11:52:40,703][20381] Updated weights for policy 1, policy_version 898652 (0.0008) [2023-12-27 11:52:40,754][20381] Updated weights for policy 1, policy_version 898662 (0.0008) [2023-12-27 11:52:40,813][20381] Updated weights for policy 1, policy_version 898672 (0.0008) [2023-12-27 11:52:41,061][20347] Updated weights for policy 0, policy_version 899381 (0.0009) [2023-12-27 11:52:41,116][20347] Updated weights for policy 0, policy_version 899391 (0.0008) [2023-12-27 11:52:41,183][20347] Updated weights for policy 0, policy_version 899401 (0.0007) [2023-12-27 11:52:41,525][20381] Updated weights for policy 1, policy_version 898682 (0.0007) [2023-12-27 11:52:41,586][20381] Updated weights for policy 1, policy_version 898692 (0.0008) [2023-12-27 11:52:41,646][20381] Updated weights for policy 1, policy_version 898702 (0.0008) [2023-12-27 11:52:41,707][20381] Updated weights for policy 1, policy_version 898712 (0.0009) [2023-12-27 11:52:41,922][20347] Updated weights for policy 0, policy_version 899411 (0.0006) [2023-12-27 11:52:41,981][20347] Updated weights for policy 0, policy_version 899421 (0.0007) [2023-12-27 11:52:42,040][20347] Updated weights for policy 0, policy_version 899431 (0.0006) [2023-12-27 11:52:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 460382208. Throughput: 0: 10324.8, 1: 10134.7. Samples: 460375000. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:42,363][19170] Avg episode reward: [(0, '882.830'), (1, '971.620')] [2023-12-27 11:52:42,404][20292] KL-divergence is very high: 267.3633 [2023-12-27 11:52:42,416][20381] Updated weights for policy 1, policy_version 898722 (0.0007) [2023-12-27 11:52:42,470][20381] Updated weights for policy 1, policy_version 898732 (0.0005) [2023-12-27 11:52:42,529][20381] Updated weights for policy 1, policy_version 898742 (0.0005) [2023-12-27 11:52:42,700][20347] Updated weights for policy 0, policy_version 899441 (0.0006) [2023-12-27 11:52:42,750][20347] Updated weights for policy 0, policy_version 899451 (0.0005) [2023-12-27 11:52:42,810][20347] Updated weights for policy 0, policy_version 899461 (0.0007) [2023-12-27 11:52:42,864][20347] Updated weights for policy 0, policy_version 899471 (0.0007) [2023-12-27 11:52:43,163][20381] Updated weights for policy 1, policy_version 898752 (0.0006) [2023-12-27 11:52:43,230][20381] Updated weights for policy 1, policy_version 898762 (0.0008) [2023-12-27 11:52:43,290][20381] Updated weights for policy 1, policy_version 898772 (0.0008) [2023-12-27 11:52:43,450][20347] Updated weights for policy 0, policy_version 899481 (0.0008) [2023-12-27 11:52:43,498][20347] Updated weights for policy 0, policy_version 899491 (0.0007) [2023-12-27 11:52:43,545][20347] Updated weights for policy 0, policy_version 899501 (0.0007) [2023-12-27 11:52:43,937][20381] Updated weights for policy 1, policy_version 898782 (0.0007) [2023-12-27 11:52:43,991][20381] Updated weights for policy 1, policy_version 898792 (0.0008) [2023-12-27 11:52:44,047][20381] Updated weights for policy 1, policy_version 898802 (0.0008) [2023-12-27 11:52:44,263][20347] Updated weights for policy 0, policy_version 899511 (0.0006) [2023-12-27 11:52:44,328][20347] Updated weights for policy 0, policy_version 899521 (0.0006) [2023-12-27 11:52:44,390][20347] Updated weights for policy 0, policy_version 899531 (0.0005) [2023-12-27 11:52:44,776][20381] Updated weights for policy 1, policy_version 898812 (0.0007) [2023-12-27 11:52:44,840][20381] Updated weights for policy 1, policy_version 898822 (0.0006) [2023-12-27 11:52:44,901][20381] Updated weights for policy 1, policy_version 898832 (0.0010) [2023-12-27 11:52:44,945][20347] Updated weights for policy 0, policy_version 899541 (0.0007) [2023-12-27 11:52:45,004][20347] Updated weights for policy 0, policy_version 899551 (0.0008) [2023-12-27 11:52:45,064][20347] Updated weights for policy 0, policy_version 899561 (0.0008) [2023-12-27 11:52:45,485][20381] Updated weights for policy 1, policy_version 898842 (0.0009) [2023-12-27 11:52:45,532][20381] Updated weights for policy 1, policy_version 898852 (0.0005) [2023-12-27 11:52:45,577][20381] Updated weights for policy 1, policy_version 898862 (0.0005) [2023-12-27 11:52:45,636][20381] Updated weights for policy 1, policy_version 898872 (0.0005) [2023-12-27 11:52:45,766][20347] Updated weights for policy 0, policy_version 899571 (0.0008) [2023-12-27 11:52:45,818][20347] Updated weights for policy 0, policy_version 899581 (0.0005) [2023-12-27 11:52:45,865][20347] Updated weights for policy 0, policy_version 899591 (0.0009) [2023-12-27 11:52:46,233][20381] Updated weights for policy 1, policy_version 898882 (0.0011) [2023-12-27 11:52:46,295][20381] Updated weights for policy 1, policy_version 898892 (0.0011) [2023-12-27 11:52:46,359][20381] Updated weights for policy 1, policy_version 898902 (0.0011) [2023-12-27 11:52:46,519][20347] Updated weights for policy 0, policy_version 899601 (0.0007) [2023-12-27 11:52:46,573][20347] Updated weights for policy 0, policy_version 899611 (0.0006) [2023-12-27 11:52:46,623][20347] Updated weights for policy 0, policy_version 899621 (0.0007) [2023-12-27 11:52:46,673][20347] Updated weights for policy 0, policy_version 899631 (0.0008) [2023-12-27 11:52:46,976][20381] Updated weights for policy 1, policy_version 898912 (0.0010) [2023-12-27 11:52:47,027][20381] Updated weights for policy 1, policy_version 898922 (0.0007) [2023-12-27 11:52:47,073][20381] Updated weights for policy 1, policy_version 898932 (0.0007) [2023-12-27 11:52:47,301][20347] Updated weights for policy 0, policy_version 899641 (0.0006) [2023-12-27 11:52:47,347][20347] Updated weights for policy 0, policy_version 899651 (0.0005) [2023-12-27 11:52:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20411.7). Total num frames: 460488704. Throughput: 0: 10393.6, 1: 10202.0. Samples: 460500308. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:47,363][19170] Avg episode reward: [(0, '814.280'), (1, '952.140')] [2023-12-27 11:52:47,393][20347] Updated weights for policy 0, policy_version 899661 (0.0005) [2023-12-27 11:52:47,748][20381] Updated weights for policy 1, policy_version 898942 (0.0007) [2023-12-27 11:52:47,799][20381] Updated weights for policy 1, policy_version 898952 (0.0007) [2023-12-27 11:52:47,856][20381] Updated weights for policy 1, policy_version 898962 (0.0005) [2023-12-27 11:52:47,950][20347] Updated weights for policy 0, policy_version 899671 (0.0009) [2023-12-27 11:52:48,009][20347] Updated weights for policy 0, policy_version 899681 (0.0010) [2023-12-27 11:52:48,064][20347] Updated weights for policy 0, policy_version 899691 (0.0011) [2023-12-27 11:52:48,519][20381] Updated weights for policy 1, policy_version 898972 (0.0008) [2023-12-27 11:52:48,577][20381] Updated weights for policy 1, policy_version 898982 (0.0008) [2023-12-27 11:52:48,642][20381] Updated weights for policy 1, policy_version 898992 (0.0009) [2023-12-27 11:52:48,706][20347] Updated weights for policy 0, policy_version 899701 (0.0009) [2023-12-27 11:52:48,728][20291] KL-divergence is very high: 300.2915 [2023-12-27 11:52:48,734][20291] KL-divergence is very high: 312.7106 [2023-12-27 11:52:48,747][20291] KL-divergence is very high: 105.9058 [2023-12-27 11:52:48,774][20347] Updated weights for policy 0, policy_version 899711 (0.0007) [2023-12-27 11:52:48,837][20347] Updated weights for policy 0, policy_version 899721 (0.0006) [2023-12-27 11:52:48,849][20291] KL-divergence is very high: 160.3332 [2023-12-27 11:52:49,365][20381] Updated weights for policy 1, policy_version 899002 (0.0007) [2023-12-27 11:52:49,407][20291] KL-divergence is very high: 139.2500 [2023-12-27 11:52:49,422][20347] Updated weights for policy 0, policy_version 899731 (0.0006) [2023-12-27 11:52:49,427][20381] Updated weights for policy 1, policy_version 899012 (0.0008) [2023-12-27 11:52:49,475][20347] Updated weights for policy 0, policy_version 899741 (0.0006) [2023-12-27 11:52:49,477][20381] Updated weights for policy 1, policy_version 899022 (0.0006) [2023-12-27 11:52:49,530][20347] Updated weights for policy 0, policy_version 899751 (0.0007) [2023-12-27 11:52:49,532][20381] Updated weights for policy 1, policy_version 899032 (0.0006) [2023-12-27 11:52:49,536][20291] KL-divergence is very high: 117.3976 [2023-12-27 11:52:50,248][20381] Updated weights for policy 1, policy_version 899042 (0.0008) [2023-12-27 11:52:50,305][20381] Updated weights for policy 1, policy_version 899052 (0.0008) [2023-12-27 11:52:50,315][20347] Updated weights for policy 0, policy_version 899761 (0.0007) [2023-12-27 11:52:50,363][20381] Updated weights for policy 1, policy_version 899062 (0.0006) [2023-12-27 11:52:50,378][20347] Updated weights for policy 0, policy_version 899771 (0.0006) [2023-12-27 11:52:50,438][20347] Updated weights for policy 0, policy_version 899781 (0.0008) [2023-12-27 11:52:50,498][20347] Updated weights for policy 0, policy_version 899791 (0.0008) [2023-12-27 11:52:51,022][20381] Updated weights for policy 1, policy_version 899072 (0.0009) [2023-12-27 11:52:51,086][20381] Updated weights for policy 1, policy_version 899082 (0.0008) [2023-12-27 11:52:51,144][20381] Updated weights for policy 1, policy_version 899092 (0.0007) [2023-12-27 11:52:51,211][20347] Updated weights for policy 0, policy_version 899801 (0.0007) [2023-12-27 11:52:51,270][20347] Updated weights for policy 0, policy_version 899811 (0.0006) [2023-12-27 11:52:51,333][20347] Updated weights for policy 0, policy_version 899821 (0.0006) [2023-12-27 11:52:51,837][20381] Updated weights for policy 1, policy_version 899102 (0.0008) [2023-12-27 11:52:51,890][20381] Updated weights for policy 1, policy_version 899112 (0.0007) [2023-12-27 11:52:51,951][20381] Updated weights for policy 1, policy_version 899122 (0.0008) [2023-12-27 11:52:52,069][20347] Updated weights for policy 0, policy_version 899831 (0.0009) [2023-12-27 11:52:52,126][20347] Updated weights for policy 0, policy_version 899841 (0.0008) [2023-12-27 11:52:52,182][20347] Updated weights for policy 0, policy_version 899851 (0.0008) [2023-12-27 11:52:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 460595200. Throughput: 0: 10406.7, 1: 10163.6. Samples: 460563796. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:52,363][19170] Avg episode reward: [(0, '794.820'), (1, '961.070')] [2023-12-27 11:52:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000899128_230203392.pth... [2023-12-27 11:52:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000899856_230391808.pth... [2023-12-27 11:52:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000897944_229900288.pth [2023-12-27 11:52:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000898640_230080512.pth [2023-12-27 11:52:52,645][20381] Updated weights for policy 1, policy_version 899132 (0.0010) [2023-12-27 11:52:52,681][20292] KL-divergence is very high: 576.4800 [2023-12-27 11:52:52,704][20381] Updated weights for policy 1, policy_version 899142 (0.0010) [2023-12-27 11:52:52,764][20381] Updated weights for policy 1, policy_version 899152 (0.0011) [2023-12-27 11:52:52,940][20347] Updated weights for policy 0, policy_version 899861 (0.0008) [2023-12-27 11:52:52,998][20347] Updated weights for policy 0, policy_version 899871 (0.0007) [2023-12-27 11:52:53,060][20347] Updated weights for policy 0, policy_version 899881 (0.0008) [2023-12-27 11:52:53,433][20381] Updated weights for policy 1, policy_version 899162 (0.0011) [2023-12-27 11:52:53,480][20381] Updated weights for policy 1, policy_version 899172 (0.0011) [2023-12-27 11:52:53,495][20292] KL-divergence is very high: 513.5930 [2023-12-27 11:52:53,526][20381] Updated weights for policy 1, policy_version 899182 (0.0010) [2023-12-27 11:52:53,581][20381] Updated weights for policy 1, policy_version 899192 (0.0010) [2023-12-27 11:52:53,768][20347] Updated weights for policy 0, policy_version 899891 (0.0008) [2023-12-27 11:52:53,819][20347] Updated weights for policy 0, policy_version 899901 (0.0010) [2023-12-27 11:52:53,868][20347] Updated weights for policy 0, policy_version 899911 (0.0010) [2023-12-27 11:52:54,262][20381] Updated weights for policy 1, policy_version 899202 (0.0010) [2023-12-27 11:52:54,311][20381] Updated weights for policy 1, policy_version 899212 (0.0010) [2023-12-27 11:52:54,369][20381] Updated weights for policy 1, policy_version 899222 (0.0011) [2023-12-27 11:52:54,529][20347] Updated weights for policy 0, policy_version 899921 (0.0010) [2023-12-27 11:52:54,575][20347] Updated weights for policy 0, policy_version 899931 (0.0007) [2023-12-27 11:52:54,624][20347] Updated weights for policy 0, policy_version 899941 (0.0007) [2023-12-27 11:52:54,670][20347] Updated weights for policy 0, policy_version 899951 (0.0007) [2023-12-27 11:52:55,029][20381] Updated weights for policy 1, policy_version 899232 (0.0009) [2023-12-27 11:52:55,087][20381] Updated weights for policy 1, policy_version 899242 (0.0011) [2023-12-27 11:52:55,149][20381] Updated weights for policy 1, policy_version 899252 (0.0011) [2023-12-27 11:52:55,388][20347] Updated weights for policy 0, policy_version 899961 (0.0007) [2023-12-27 11:52:55,449][20347] Updated weights for policy 0, policy_version 899971 (0.0007) [2023-12-27 11:52:55,496][20347] Updated weights for policy 0, policy_version 899981 (0.0007) [2023-12-27 11:52:55,803][20381] Updated weights for policy 1, policy_version 899262 (0.0007) [2023-12-27 11:52:55,834][20292] KL-divergence is very high: 523.1697 [2023-12-27 11:52:55,849][20381] Updated weights for policy 1, policy_version 899272 (0.0008) [2023-12-27 11:52:55,901][20381] Updated weights for policy 1, policy_version 899282 (0.0010) [2023-12-27 11:52:56,227][20347] Updated weights for policy 0, policy_version 899991 (0.0008) [2023-12-27 11:52:56,293][20347] Updated weights for policy 0, policy_version 900001 (0.0008) [2023-12-27 11:52:56,350][20347] Updated weights for policy 0, policy_version 900011 (0.0007) [2023-12-27 11:52:56,446][20381] Updated weights for policy 1, policy_version 899292 (0.0006) [2023-12-27 11:52:56,501][20381] Updated weights for policy 1, policy_version 899302 (0.0005) [2023-12-27 11:52:56,550][20381] Updated weights for policy 1, policy_version 899312 (0.0006) [2023-12-27 11:52:56,951][20347] Updated weights for policy 0, policy_version 900021 (0.0008) [2023-12-27 11:52:57,002][20347] Updated weights for policy 0, policy_version 900031 (0.0010) [2023-12-27 11:52:57,062][20347] Updated weights for policy 0, policy_version 900041 (0.0010) [2023-12-27 11:52:57,208][20381] Updated weights for policy 1, policy_version 899322 (0.0010) [2023-12-27 11:52:57,259][20381] Updated weights for policy 1, policy_version 899332 (0.0006) [2023-12-27 11:52:57,317][20381] Updated weights for policy 1, policy_version 899342 (0.0007) [2023-12-27 11:52:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 460693504. Throughput: 0: 10333.8, 1: 10180.7. Samples: 460684548. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:52:57,363][19170] Avg episode reward: [(0, '765.540'), (1, '961.190')] [2023-12-27 11:52:57,375][20381] Updated weights for policy 1, policy_version 899352 (0.0010) [2023-12-27 11:52:57,677][20347] Updated weights for policy 0, policy_version 900051 (0.0010) [2023-12-27 11:52:57,727][20347] Updated weights for policy 0, policy_version 900061 (0.0009) [2023-12-27 11:52:57,780][20347] Updated weights for policy 0, policy_version 900071 (0.0010) [2023-12-27 11:52:58,047][20381] Updated weights for policy 1, policy_version 899362 (0.0010) [2023-12-27 11:52:58,110][20381] Updated weights for policy 1, policy_version 899372 (0.0010) [2023-12-27 11:52:58,167][20381] Updated weights for policy 1, policy_version 899382 (0.0010) [2023-12-27 11:52:58,495][20347] Updated weights for policy 0, policy_version 900081 (0.0010) [2023-12-27 11:52:58,554][20347] Updated weights for policy 0, policy_version 900091 (0.0008) [2023-12-27 11:52:58,622][20347] Updated weights for policy 0, policy_version 900101 (0.0008) [2023-12-27 11:52:58,688][20347] Updated weights for policy 0, policy_version 900111 (0.0008) [2023-12-27 11:52:58,956][20381] Updated weights for policy 1, policy_version 899392 (0.0008) [2023-12-27 11:52:58,980][20292] KL-divergence is very high: 187.2805 [2023-12-27 11:52:58,993][20292] KL-divergence is very high: 153.4380 [2023-12-27 11:52:59,025][20381] Updated weights for policy 1, policy_version 899402 (0.0008) [2023-12-27 11:52:59,075][20381] Updated weights for policy 1, policy_version 899412 (0.0008) [2023-12-27 11:52:59,541][20347] Updated weights for policy 0, policy_version 900121 (0.0007) [2023-12-27 11:52:59,594][20347] Updated weights for policy 0, policy_version 900131 (0.0005) [2023-12-27 11:52:59,643][20347] Updated weights for policy 0, policy_version 900141 (0.0005) [2023-12-27 11:52:59,833][20381] Updated weights for policy 1, policy_version 899422 (0.0006) [2023-12-27 11:52:59,892][20381] Updated weights for policy 1, policy_version 899432 (0.0007) [2023-12-27 11:52:59,956][20381] Updated weights for policy 1, policy_version 899442 (0.0008) [2023-12-27 11:53:00,346][20347] Updated weights for policy 0, policy_version 900151 (0.0005) [2023-12-27 11:53:00,408][20347] Updated weights for policy 0, policy_version 900161 (0.0005) [2023-12-27 11:53:00,463][20347] Updated weights for policy 0, policy_version 900171 (0.0007) [2023-12-27 11:53:00,558][20381] Updated weights for policy 1, policy_version 899452 (0.0007) [2023-12-27 11:53:00,610][20381] Updated weights for policy 1, policy_version 899462 (0.0005) [2023-12-27 11:53:00,656][20381] Updated weights for policy 1, policy_version 899472 (0.0005) [2023-12-27 11:53:01,110][20347] Updated weights for policy 0, policy_version 900181 (0.0009) [2023-12-27 11:53:01,171][20347] Updated weights for policy 0, policy_version 900191 (0.0011) [2023-12-27 11:53:01,222][20347] Updated weights for policy 0, policy_version 900201 (0.0007) [2023-12-27 11:53:01,371][20381] Updated weights for policy 1, policy_version 899482 (0.0006) [2023-12-27 11:53:01,434][20381] Updated weights for policy 1, policy_version 899492 (0.0008) [2023-12-27 11:53:01,493][20381] Updated weights for policy 1, policy_version 899502 (0.0008) [2023-12-27 11:53:01,554][20381] Updated weights for policy 1, policy_version 899512 (0.0008) [2023-12-27 11:53:01,892][20347] Updated weights for policy 0, policy_version 900211 (0.0008) [2023-12-27 11:53:01,958][20347] Updated weights for policy 0, policy_version 900221 (0.0005) [2023-12-27 11:53:02,020][20347] Updated weights for policy 0, policy_version 900231 (0.0007) [2023-12-27 11:53:02,282][20381] Updated weights for policy 1, policy_version 899522 (0.0008) [2023-12-27 11:53:02,346][20381] Updated weights for policy 1, policy_version 899532 (0.0007) [2023-12-27 11:53:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 460791808. Throughput: 0: 10288.0, 1: 10149.4. Samples: 460805756. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:02,363][19170] Avg episode reward: [(0, '804.860'), (1, '952.960')] [2023-12-27 11:53:02,413][20381] Updated weights for policy 1, policy_version 899542 (0.0007) [2023-12-27 11:53:02,682][20347] Updated weights for policy 0, policy_version 900241 (0.0010) [2023-12-27 11:53:02,737][20347] Updated weights for policy 0, policy_version 900251 (0.0010) [2023-12-27 11:53:02,788][20347] Updated weights for policy 0, policy_version 900261 (0.0010) [2023-12-27 11:53:02,840][20347] Updated weights for policy 0, policy_version 900271 (0.0010) [2023-12-27 11:53:03,121][20381] Updated weights for policy 1, policy_version 899552 (0.0007) [2023-12-27 11:53:03,175][20381] Updated weights for policy 1, policy_version 899562 (0.0007) [2023-12-27 11:53:03,234][20381] Updated weights for policy 1, policy_version 899572 (0.0007) [2023-12-27 11:53:03,501][20347] Updated weights for policy 0, policy_version 900281 (0.0007) [2023-12-27 11:53:03,557][20347] Updated weights for policy 0, policy_version 900291 (0.0006) [2023-12-27 11:53:03,604][20347] Updated weights for policy 0, policy_version 900301 (0.0007) [2023-12-27 11:53:03,886][20381] Updated weights for policy 1, policy_version 899582 (0.0007) [2023-12-27 11:53:03,941][20381] Updated weights for policy 1, policy_version 899592 (0.0008) [2023-12-27 11:53:03,998][20381] Updated weights for policy 1, policy_version 899602 (0.0008) [2023-12-27 11:53:04,339][20347] Updated weights for policy 0, policy_version 900311 (0.0010) [2023-12-27 11:53:04,396][20347] Updated weights for policy 0, policy_version 900321 (0.0009) [2023-12-27 11:53:04,453][20347] Updated weights for policy 0, policy_version 900331 (0.0009) [2023-12-27 11:53:04,725][20381] Updated weights for policy 1, policy_version 899612 (0.0008) [2023-12-27 11:53:04,777][20381] Updated weights for policy 1, policy_version 899622 (0.0008) [2023-12-27 11:53:04,833][20381] Updated weights for policy 1, policy_version 899632 (0.0007) [2023-12-27 11:53:05,111][20347] Updated weights for policy 0, policy_version 900341 (0.0008) [2023-12-27 11:53:05,169][20347] Updated weights for policy 0, policy_version 900351 (0.0005) [2023-12-27 11:53:05,225][20347] Updated weights for policy 0, policy_version 900361 (0.0006) [2023-12-27 11:53:05,531][20381] Updated weights for policy 1, policy_version 899642 (0.0007) [2023-12-27 11:53:05,577][20381] Updated weights for policy 1, policy_version 899652 (0.0008) [2023-12-27 11:53:05,621][20381] Updated weights for policy 1, policy_version 899662 (0.0007) [2023-12-27 11:53:05,669][20381] Updated weights for policy 1, policy_version 899672 (0.0007) [2023-12-27 11:53:05,792][20347] Updated weights for policy 0, policy_version 900371 (0.0005) [2023-12-27 11:53:05,845][20347] Updated weights for policy 0, policy_version 900381 (0.0005) [2023-12-27 11:53:05,894][20347] Updated weights for policy 0, policy_version 900391 (0.0005) [2023-12-27 11:53:06,418][20381] Updated weights for policy 1, policy_version 899682 (0.0008) [2023-12-27 11:53:06,478][20347] Updated weights for policy 0, policy_version 900401 (0.0005) [2023-12-27 11:53:06,478][20381] Updated weights for policy 1, policy_version 899692 (0.0009) [2023-12-27 11:53:06,541][20381] Updated weights for policy 1, policy_version 899702 (0.0007) [2023-12-27 11:53:06,546][20347] Updated weights for policy 0, policy_version 900411 (0.0006) [2023-12-27 11:53:06,615][20347] Updated weights for policy 0, policy_version 900421 (0.0007) [2023-12-27 11:53:06,683][20347] Updated weights for policy 0, policy_version 900431 (0.0011) [2023-12-27 11:53:07,243][20381] Updated weights for policy 1, policy_version 899712 (0.0009) [2023-12-27 11:53:07,268][20347] Updated weights for policy 0, policy_version 900441 (0.0010) [2023-12-27 11:53:07,282][20292] KL-divergence is very high: 382.1321 [2023-12-27 11:53:07,302][20381] Updated weights for policy 1, policy_version 899722 (0.0010) [2023-12-27 11:53:07,307][20292] KL-divergence is very high: 118.8052 [2023-12-27 11:53:07,324][20347] Updated weights for policy 0, policy_version 900451 (0.0010) [2023-12-27 11:53:07,333][20292] KL-divergence is very high: 163.4821 [2023-12-27 11:53:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.6, 300 sec: 20438.3). Total num frames: 460890112. Throughput: 0: 10287.7, 1: 10153.2. Samples: 460866668. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:07,363][19170] Avg episode reward: [(0, '882.850'), (1, '962.610')] [2023-12-27 11:53:07,363][20381] Updated weights for policy 1, policy_version 899732 (0.0009) [2023-12-27 11:53:07,386][20347] Updated weights for policy 0, policy_version 900461 (0.0010) [2023-12-27 11:53:07,387][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000899736_230359040.pth... [2023-12-27 11:53:07,390][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000898520_230047744.pth [2023-12-27 11:53:07,401][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000900464_230547456.pth... [2023-12-27 11:53:07,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000899248_230236160.pth [2023-12-27 11:53:07,959][20381] Updated weights for policy 1, policy_version 899742 (0.0005) [2023-12-27 11:53:08,017][20381] Updated weights for policy 1, policy_version 899752 (0.0007) [2023-12-27 11:53:08,035][20347] Updated weights for policy 0, policy_version 900471 (0.0010) [2023-12-27 11:53:08,068][20381] Updated weights for policy 1, policy_version 899762 (0.0008) [2023-12-27 11:53:08,093][20347] Updated weights for policy 0, policy_version 900481 (0.0010) [2023-12-27 11:53:08,160][20347] Updated weights for policy 0, policy_version 900491 (0.0010) [2023-12-27 11:53:08,738][20381] Updated weights for policy 1, policy_version 899772 (0.0008) [2023-12-27 11:53:08,787][20381] Updated weights for policy 1, policy_version 899782 (0.0010) [2023-12-27 11:53:08,839][20381] Updated weights for policy 1, policy_version 899792 (0.0010) [2023-12-27 11:53:08,859][20347] Updated weights for policy 0, policy_version 900501 (0.0009) [2023-12-27 11:53:08,909][20347] Updated weights for policy 0, policy_version 900511 (0.0007) [2023-12-27 11:53:08,960][20347] Updated weights for policy 0, policy_version 900521 (0.0007) [2023-12-27 11:53:09,537][20381] Updated weights for policy 1, policy_version 899802 (0.0011) [2023-12-27 11:53:09,604][20381] Updated weights for policy 1, policy_version 899812 (0.0011) [2023-12-27 11:53:09,673][20381] Updated weights for policy 1, policy_version 899822 (0.0011) [2023-12-27 11:53:09,707][20347] Updated weights for policy 0, policy_version 900531 (0.0007) [2023-12-27 11:53:09,738][20381] Updated weights for policy 1, policy_version 899832 (0.0011) [2023-12-27 11:53:09,767][20347] Updated weights for policy 0, policy_version 900541 (0.0006) [2023-12-27 11:53:09,827][20347] Updated weights for policy 0, policy_version 900551 (0.0007) [2023-12-27 11:53:10,437][20381] Updated weights for policy 1, policy_version 899842 (0.0008) [2023-12-27 11:53:10,495][20381] Updated weights for policy 1, policy_version 899852 (0.0006) [2023-12-27 11:53:10,520][20347] Updated weights for policy 0, policy_version 900561 (0.0010) [2023-12-27 11:53:10,560][20381] Updated weights for policy 1, policy_version 899862 (0.0008) [2023-12-27 11:53:10,578][20347] Updated weights for policy 0, policy_version 900571 (0.0010) [2023-12-27 11:53:10,633][20347] Updated weights for policy 0, policy_version 900581 (0.0010) [2023-12-27 11:53:10,686][20347] Updated weights for policy 0, policy_version 900591 (0.0010) [2023-12-27 11:53:11,192][20381] Updated weights for policy 1, policy_version 899872 (0.0010) [2023-12-27 11:53:11,259][20381] Updated weights for policy 1, policy_version 899882 (0.0010) [2023-12-27 11:53:11,320][20381] Updated weights for policy 1, policy_version 899892 (0.0011) [2023-12-27 11:53:11,323][20347] Updated weights for policy 0, policy_version 900601 (0.0007) [2023-12-27 11:53:11,400][20347] Updated weights for policy 0, policy_version 900611 (0.0010) [2023-12-27 11:53:11,463][20347] Updated weights for policy 0, policy_version 900621 (0.0011) [2023-12-27 11:53:12,096][20381] Updated weights for policy 1, policy_version 899902 (0.0009) [2023-12-27 11:53:12,152][20381] Updated weights for policy 1, policy_version 899912 (0.0007) [2023-12-27 11:53:12,176][20347] Updated weights for policy 0, policy_version 900631 (0.0008) [2023-12-27 11:53:12,206][20381] Updated weights for policy 1, policy_version 899922 (0.0007) [2023-12-27 11:53:12,233][20347] Updated weights for policy 0, policy_version 900641 (0.0007) [2023-12-27 11:53:12,300][20347] Updated weights for policy 0, policy_version 900651 (0.0007) [2023-12-27 11:53:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 461004800. Throughput: 0: 10343.6, 1: 10191.5. Samples: 460990832. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:12,363][19170] Avg episode reward: [(0, '892.670'), (1, '970.910')] [2023-12-27 11:53:12,946][20347] Updated weights for policy 0, policy_version 900661 (0.0007) [2023-12-27 11:53:12,953][20381] Updated weights for policy 1, policy_version 899932 (0.0006) [2023-12-27 11:53:12,999][20347] Updated weights for policy 0, policy_version 900671 (0.0006) [2023-12-27 11:53:13,008][20381] Updated weights for policy 1, policy_version 899942 (0.0007) [2023-12-27 11:53:13,048][20347] Updated weights for policy 0, policy_version 900681 (0.0005) [2023-12-27 11:53:13,061][20381] Updated weights for policy 1, policy_version 899952 (0.0006) [2023-12-27 11:53:13,706][20381] Updated weights for policy 1, policy_version 899962 (0.0006) [2023-12-27 11:53:13,733][20347] Updated weights for policy 0, policy_version 900691 (0.0006) [2023-12-27 11:53:13,762][20381] Updated weights for policy 1, policy_version 899972 (0.0005) [2023-12-27 11:53:13,782][20347] Updated weights for policy 0, policy_version 900701 (0.0006) [2023-12-27 11:53:13,818][20381] Updated weights for policy 1, policy_version 899982 (0.0005) [2023-12-27 11:53:13,840][20347] Updated weights for policy 0, policy_version 900711 (0.0006) [2023-12-27 11:53:13,872][20381] Updated weights for policy 1, policy_version 899992 (0.0005) [2023-12-27 11:53:14,513][20347] Updated weights for policy 0, policy_version 900721 (0.0006) [2023-12-27 11:53:14,553][20381] Updated weights for policy 1, policy_version 900002 (0.0006) [2023-12-27 11:53:14,567][20347] Updated weights for policy 0, policy_version 900731 (0.0006) [2023-12-27 11:53:14,608][20381] Updated weights for policy 1, policy_version 900012 (0.0007) [2023-12-27 11:53:14,623][20347] Updated weights for policy 0, policy_version 900741 (0.0008) [2023-12-27 11:53:14,663][20381] Updated weights for policy 1, policy_version 900022 (0.0009) [2023-12-27 11:53:14,679][20347] Updated weights for policy 0, policy_version 900751 (0.0009) [2023-12-27 11:53:15,337][20347] Updated weights for policy 0, policy_version 900761 (0.0007) [2023-12-27 11:53:15,397][20347] Updated weights for policy 0, policy_version 900771 (0.0008) [2023-12-27 11:53:15,421][20381] Updated weights for policy 1, policy_version 900032 (0.0006) [2023-12-27 11:53:15,446][20347] Updated weights for policy 0, policy_version 900781 (0.0006) [2023-12-27 11:53:15,481][20381] Updated weights for policy 1, policy_version 900042 (0.0007) [2023-12-27 11:53:15,539][20381] Updated weights for policy 1, policy_version 900052 (0.0008) [2023-12-27 11:53:16,103][20347] Updated weights for policy 0, policy_version 900791 (0.0007) [2023-12-27 11:53:16,149][20347] Updated weights for policy 0, policy_version 900801 (0.0008) [2023-12-27 11:53:16,207][20347] Updated weights for policy 0, policy_version 900811 (0.0008) [2023-12-27 11:53:16,232][20381] Updated weights for policy 1, policy_version 900062 (0.0006) [2023-12-27 11:53:16,287][20381] Updated weights for policy 1, policy_version 900072 (0.0007) [2023-12-27 11:53:16,336][20381] Updated weights for policy 1, policy_version 900082 (0.0008) [2023-12-27 11:53:16,872][20347] Updated weights for policy 0, policy_version 900821 (0.0006) [2023-12-27 11:53:16,928][20347] Updated weights for policy 0, policy_version 900831 (0.0007) [2023-12-27 11:53:16,976][20381] Updated weights for policy 1, policy_version 900092 (0.0008) [2023-12-27 11:53:16,988][20347] Updated weights for policy 0, policy_version 900841 (0.0010) [2023-12-27 11:53:17,031][20381] Updated weights for policy 1, policy_version 900102 (0.0008) [2023-12-27 11:53:17,086][20381] Updated weights for policy 1, policy_version 900112 (0.0007) [2023-12-27 11:53:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 461103104. Throughput: 0: 10296.5, 1: 10193.8. Samples: 461111044. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:17,364][19170] Avg episode reward: [(0, '863.510'), (1, '912.540')] [2023-12-27 11:53:17,675][20347] Updated weights for policy 0, policy_version 900851 (0.0009) [2023-12-27 11:53:17,738][20347] Updated weights for policy 0, policy_version 900861 (0.0008) [2023-12-27 11:53:17,765][20381] Updated weights for policy 1, policy_version 900122 (0.0008) [2023-12-27 11:53:17,794][20347] Updated weights for policy 0, policy_version 900871 (0.0007) [2023-12-27 11:53:17,818][20381] Updated weights for policy 1, policy_version 900132 (0.0010) [2023-12-27 11:53:17,875][20381] Updated weights for policy 1, policy_version 900142 (0.0010) [2023-12-27 11:53:17,932][20381] Updated weights for policy 1, policy_version 900152 (0.0010) [2023-12-27 11:53:18,390][20347] Updated weights for policy 0, policy_version 900881 (0.0006) [2023-12-27 11:53:18,447][20347] Updated weights for policy 0, policy_version 900891 (0.0008) [2023-12-27 11:53:18,507][20347] Updated weights for policy 0, policy_version 900901 (0.0007) [2023-12-27 11:53:18,561][20381] Updated weights for policy 1, policy_version 900162 (0.0006) [2023-12-27 11:53:18,567][20347] Updated weights for policy 0, policy_version 900911 (0.0007) [2023-12-27 11:53:18,619][20381] Updated weights for policy 1, policy_version 900172 (0.0007) [2023-12-27 11:53:18,671][20381] Updated weights for policy 1, policy_version 900182 (0.0007) [2023-12-27 11:53:19,186][20347] Updated weights for policy 0, policy_version 900921 (0.0008) [2023-12-27 11:53:19,245][20347] Updated weights for policy 0, policy_version 900931 (0.0007) [2023-12-27 11:53:19,250][20381] Updated weights for policy 1, policy_version 900193 (0.0012) [2023-12-27 11:53:19,304][20347] Updated weights for policy 0, policy_version 900941 (0.0008) [2023-12-27 11:53:19,310][20381] Updated weights for policy 1, policy_version 900203 (0.0006) [2023-12-27 11:53:19,379][20381] Updated weights for policy 1, policy_version 900213 (0.0007) [2023-12-27 11:53:20,041][20381] Updated weights for policy 1, policy_version 900223 (0.0006) [2023-12-27 11:53:20,101][20381] Updated weights for policy 1, policy_version 900233 (0.0006) [2023-12-27 11:53:20,106][20347] Updated weights for policy 0, policy_version 900951 (0.0008) [2023-12-27 11:53:20,166][20381] Updated weights for policy 1, policy_version 900243 (0.0006) [2023-12-27 11:53:20,170][20347] Updated weights for policy 0, policy_version 900961 (0.0008) [2023-12-27 11:53:20,226][20347] Updated weights for policy 0, policy_version 900971 (0.0008) [2023-12-27 11:53:20,818][20381] Updated weights for policy 1, policy_version 900253 (0.0006) [2023-12-27 11:53:20,869][20381] Updated weights for policy 1, policy_version 900263 (0.0007) [2023-12-27 11:53:20,920][20381] Updated weights for policy 1, policy_version 900273 (0.0007) [2023-12-27 11:53:20,984][20347] Updated weights for policy 0, policy_version 900981 (0.0008) [2023-12-27 11:53:21,050][20347] Updated weights for policy 0, policy_version 900991 (0.0008) [2023-12-27 11:53:21,111][20347] Updated weights for policy 0, policy_version 901001 (0.0007) [2023-12-27 11:53:21,781][20381] Updated weights for policy 1, policy_version 900283 (0.0007) [2023-12-27 11:53:21,809][20347] Updated weights for policy 0, policy_version 901011 (0.0006) [2023-12-27 11:53:21,831][20381] Updated weights for policy 1, policy_version 900293 (0.0009) [2023-12-27 11:53:21,866][20347] Updated weights for policy 0, policy_version 901021 (0.0007) [2023-12-27 11:53:21,879][20381] Updated weights for policy 1, policy_version 900303 (0.0009) [2023-12-27 11:53:21,920][20347] Updated weights for policy 0, policy_version 901031 (0.0007) [2023-12-27 11:53:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 461201408. Throughput: 0: 10276.1, 1: 10232.4. Samples: 461174224. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:22,363][19170] Avg episode reward: [(0, '912.310'), (1, '865.440')] [2023-12-27 11:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000900312_230506496.pth... [2023-12-27 11:53:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000901040_230694912.pth... [2023-12-27 11:53:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000899128_230203392.pth [2023-12-27 11:53:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000899856_230391808.pth [2023-12-27 11:53:22,633][20347] Updated weights for policy 0, policy_version 901041 (0.0008) [2023-12-27 11:53:22,653][20381] Updated weights for policy 1, policy_version 900313 (0.0007) [2023-12-27 11:53:22,696][20347] Updated weights for policy 0, policy_version 901051 (0.0010) [2023-12-27 11:53:22,712][20381] Updated weights for policy 1, policy_version 900323 (0.0009) [2023-12-27 11:53:22,752][20347] Updated weights for policy 0, policy_version 901061 (0.0010) [2023-12-27 11:53:22,770][20381] Updated weights for policy 1, policy_version 900333 (0.0006) [2023-12-27 11:53:22,812][20347] Updated weights for policy 0, policy_version 901071 (0.0010) [2023-12-27 11:53:22,830][20381] Updated weights for policy 1, policy_version 900343 (0.0008) [2023-12-27 11:53:23,482][20381] Updated weights for policy 1, policy_version 900353 (0.0005) [2023-12-27 11:53:23,490][20347] Updated weights for policy 0, policy_version 901081 (0.0010) [2023-12-27 11:53:23,544][20381] Updated weights for policy 1, policy_version 900363 (0.0005) [2023-12-27 11:53:23,547][20347] Updated weights for policy 0, policy_version 901091 (0.0010) [2023-12-27 11:53:23,604][20381] Updated weights for policy 1, policy_version 900373 (0.0005) [2023-12-27 11:53:23,605][20347] Updated weights for policy 0, policy_version 901101 (0.0010) [2023-12-27 11:53:24,183][20381] Updated weights for policy 1, policy_version 900383 (0.0005) [2023-12-27 11:53:24,239][20381] Updated weights for policy 1, policy_version 900393 (0.0005) [2023-12-27 11:53:24,271][20347] Updated weights for policy 0, policy_version 901111 (0.0010) [2023-12-27 11:53:24,292][20381] Updated weights for policy 1, policy_version 900403 (0.0005) [2023-12-27 11:53:24,333][20347] Updated weights for policy 0, policy_version 901121 (0.0010) [2023-12-27 11:53:24,391][20347] Updated weights for policy 0, policy_version 901131 (0.0009) [2023-12-27 11:53:24,866][20381] Updated weights for policy 1, policy_version 900413 (0.0006) [2023-12-27 11:53:24,920][20381] Updated weights for policy 1, policy_version 900423 (0.0006) [2023-12-27 11:53:24,986][20381] Updated weights for policy 1, policy_version 900433 (0.0006) [2023-12-27 11:53:25,041][20347] Updated weights for policy 0, policy_version 901141 (0.0008) [2023-12-27 11:53:25,087][20347] Updated weights for policy 0, policy_version 901151 (0.0010) [2023-12-27 11:53:25,138][20347] Updated weights for policy 0, policy_version 901161 (0.0010) [2023-12-27 11:53:25,596][20381] Updated weights for policy 1, policy_version 900443 (0.0006) [2023-12-27 11:53:25,652][20381] Updated weights for policy 1, policy_version 900453 (0.0007) [2023-12-27 11:53:25,709][20381] Updated weights for policy 1, policy_version 900463 (0.0007) [2023-12-27 11:53:25,821][20347] Updated weights for policy 0, policy_version 901171 (0.0010) [2023-12-27 11:53:25,882][20347] Updated weights for policy 0, policy_version 901181 (0.0010) [2023-12-27 11:53:25,931][20347] Updated weights for policy 0, policy_version 901191 (0.0008) [2023-12-27 11:53:26,355][20381] Updated weights for policy 1, policy_version 900473 (0.0008) [2023-12-27 11:53:26,413][20381] Updated weights for policy 1, policy_version 900483 (0.0008) [2023-12-27 11:53:26,462][20381] Updated weights for policy 1, policy_version 900493 (0.0005) [2023-12-27 11:53:26,511][20381] Updated weights for policy 1, policy_version 900503 (0.0006) [2023-12-27 11:53:26,541][20347] Updated weights for policy 0, policy_version 901201 (0.0007) [2023-12-27 11:53:26,600][20347] Updated weights for policy 0, policy_version 901211 (0.0006) [2023-12-27 11:53:26,657][20347] Updated weights for policy 0, policy_version 901221 (0.0005) [2023-12-27 11:53:26,702][20347] Updated weights for policy 0, policy_version 901231 (0.0005) [2023-12-27 11:53:27,174][20381] Updated weights for policy 1, policy_version 900513 (0.0010) [2023-12-27 11:53:27,233][20381] Updated weights for policy 1, policy_version 900523 (0.0009) [2023-12-27 11:53:27,286][20381] Updated weights for policy 1, policy_version 900533 (0.0005) [2023-12-27 11:53:27,323][20347] Updated weights for policy 0, policy_version 901241 (0.0007) [2023-12-27 11:53:27,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 461307904. Throughput: 0: 10244.9, 1: 10216.7. Samples: 461295776. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:27,364][19170] Avg episode reward: [(0, '902.720'), (1, '893.670')] [2023-12-27 11:53:27,373][20347] Updated weights for policy 0, policy_version 901251 (0.0007) [2023-12-27 11:53:27,416][20347] Updated weights for policy 0, policy_version 901261 (0.0007) [2023-12-27 11:53:27,950][20381] Updated weights for policy 1, policy_version 900543 (0.0009) [2023-12-27 11:53:27,996][20381] Updated weights for policy 1, policy_version 900553 (0.0010) [2023-12-27 11:53:28,052][20381] Updated weights for policy 1, policy_version 900563 (0.0011) [2023-12-27 11:53:28,112][20347] Updated weights for policy 0, policy_version 901271 (0.0008) [2023-12-27 11:53:28,174][20347] Updated weights for policy 0, policy_version 901281 (0.0008) [2023-12-27 11:53:28,228][20347] Updated weights for policy 0, policy_version 901291 (0.0005) [2023-12-27 11:53:28,688][20381] Updated weights for policy 1, policy_version 900573 (0.0011) [2023-12-27 11:53:28,743][20381] Updated weights for policy 1, policy_version 900583 (0.0010) [2023-12-27 11:53:28,806][20381] Updated weights for policy 1, policy_version 900593 (0.0011) [2023-12-27 11:53:28,878][20347] Updated weights for policy 0, policy_version 901301 (0.0006) [2023-12-27 11:53:28,928][20347] Updated weights for policy 0, policy_version 901311 (0.0007) [2023-12-27 11:53:28,977][20347] Updated weights for policy 0, policy_version 901321 (0.0007) [2023-12-27 11:53:29,529][20381] Updated weights for policy 1, policy_version 900603 (0.0010) [2023-12-27 11:53:29,587][20381] Updated weights for policy 1, policy_version 900613 (0.0009) [2023-12-27 11:53:29,647][20381] Updated weights for policy 1, policy_version 900623 (0.0007) [2023-12-27 11:53:29,761][20347] Updated weights for policy 0, policy_version 901331 (0.0007) [2023-12-27 11:53:29,819][20347] Updated weights for policy 0, policy_version 901341 (0.0007) [2023-12-27 11:53:29,884][20347] Updated weights for policy 0, policy_version 901351 (0.0006) [2023-12-27 11:53:30,348][20381] Updated weights for policy 1, policy_version 900633 (0.0008) [2023-12-27 11:53:30,405][20381] Updated weights for policy 1, policy_version 900643 (0.0005) [2023-12-27 11:53:30,461][20381] Updated weights for policy 1, policy_version 900653 (0.0009) [2023-12-27 11:53:30,508][20347] Updated weights for policy 0, policy_version 901361 (0.0007) [2023-12-27 11:53:30,520][20381] Updated weights for policy 1, policy_version 900663 (0.0010) [2023-12-27 11:53:30,558][20347] Updated weights for policy 0, policy_version 901371 (0.0008) [2023-12-27 11:53:30,606][20347] Updated weights for policy 0, policy_version 901381 (0.0007) [2023-12-27 11:53:30,657][20347] Updated weights for policy 0, policy_version 901391 (0.0007) [2023-12-27 11:53:31,154][20381] Updated weights for policy 1, policy_version 900673 (0.0008) [2023-12-27 11:53:31,208][20381] Updated weights for policy 1, policy_version 900683 (0.0006) [2023-12-27 11:53:31,265][20381] Updated weights for policy 1, policy_version 900693 (0.0011) [2023-12-27 11:53:31,346][20347] Updated weights for policy 0, policy_version 901401 (0.0005) [2023-12-27 11:53:31,411][20347] Updated weights for policy 0, policy_version 901411 (0.0007) [2023-12-27 11:53:31,463][20347] Updated weights for policy 0, policy_version 901421 (0.0007) [2023-12-27 11:53:32,026][20381] Updated weights for policy 1, policy_version 900703 (0.0009) [2023-12-27 11:53:32,086][20381] Updated weights for policy 1, policy_version 900713 (0.0007) [2023-12-27 11:53:32,123][20347] Updated weights for policy 0, policy_version 901431 (0.0006) [2023-12-27 11:53:32,143][20381] Updated weights for policy 1, policy_version 900723 (0.0008) [2023-12-27 11:53:32,177][20347] Updated weights for policy 0, policy_version 901441 (0.0005) [2023-12-27 11:53:32,229][20347] Updated weights for policy 0, policy_version 901451 (0.0007) [2023-12-27 11:53:32,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 461414400. Throughput: 0: 10250.3, 1: 10190.9. Samples: 461420164. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:32,363][19170] Avg episode reward: [(0, '941.770'), (1, '912.680')] [2023-12-27 11:53:32,849][20381] Updated weights for policy 1, policy_version 900733 (0.0006) [2023-12-27 11:53:32,899][20347] Updated weights for policy 0, policy_version 901461 (0.0006) [2023-12-27 11:53:32,904][20381] Updated weights for policy 1, policy_version 900743 (0.0007) [2023-12-27 11:53:32,943][20347] Updated weights for policy 0, policy_version 901471 (0.0006) [2023-12-27 11:53:32,958][20381] Updated weights for policy 1, policy_version 900753 (0.0008) [2023-12-27 11:53:32,995][20347] Updated weights for policy 0, policy_version 901481 (0.0009) [2023-12-27 11:53:33,563][20381] Updated weights for policy 1, policy_version 900763 (0.0007) [2023-12-27 11:53:33,609][20381] Updated weights for policy 1, policy_version 900773 (0.0005) [2023-12-27 11:53:33,655][20381] Updated weights for policy 1, policy_version 900783 (0.0005) [2023-12-27 11:53:33,730][20347] Updated weights for policy 0, policy_version 901491 (0.0008) [2023-12-27 11:53:33,781][20347] Updated weights for policy 0, policy_version 901501 (0.0007) [2023-12-27 11:53:33,836][20347] Updated weights for policy 0, policy_version 901511 (0.0007) [2023-12-27 11:53:34,195][20381] Updated weights for policy 1, policy_version 900793 (0.0006) [2023-12-27 11:53:34,247][20381] Updated weights for policy 1, policy_version 900803 (0.0008) [2023-12-27 11:53:34,305][20381] Updated weights for policy 1, policy_version 900813 (0.0008) [2023-12-27 11:53:34,354][20381] Updated weights for policy 1, policy_version 900823 (0.0008) [2023-12-27 11:53:34,519][20347] Updated weights for policy 0, policy_version 901521 (0.0008) [2023-12-27 11:53:34,569][20347] Updated weights for policy 0, policy_version 901531 (0.0007) [2023-12-27 11:53:34,617][20347] Updated weights for policy 0, policy_version 901541 (0.0007) [2023-12-27 11:53:34,666][20347] Updated weights for policy 0, policy_version 901551 (0.0007) [2023-12-27 11:53:35,067][20381] Updated weights for policy 1, policy_version 900833 (0.0006) [2023-12-27 11:53:35,117][20381] Updated weights for policy 1, policy_version 900843 (0.0009) [2023-12-27 11:53:35,165][20381] Updated weights for policy 1, policy_version 900853 (0.0010) [2023-12-27 11:53:35,391][20347] Updated weights for policy 0, policy_version 901561 (0.0007) [2023-12-27 11:53:35,444][20347] Updated weights for policy 0, policy_version 901571 (0.0008) [2023-12-27 11:53:35,496][20347] Updated weights for policy 0, policy_version 901581 (0.0007) [2023-12-27 11:53:35,808][20381] Updated weights for policy 1, policy_version 900863 (0.0006) [2023-12-27 11:53:35,854][20381] Updated weights for policy 1, policy_version 900873 (0.0006) [2023-12-27 11:53:35,899][20381] Updated weights for policy 1, policy_version 900883 (0.0005) [2023-12-27 11:53:36,218][20347] Updated weights for policy 0, policy_version 901591 (0.0007) [2023-12-27 11:53:36,277][20347] Updated weights for policy 0, policy_version 901601 (0.0008) [2023-12-27 11:53:36,336][20347] Updated weights for policy 0, policy_version 901611 (0.0007) [2023-12-27 11:53:36,630][20381] Updated weights for policy 1, policy_version 900893 (0.0008) [2023-12-27 11:53:36,694][20381] Updated weights for policy 1, policy_version 900903 (0.0011) [2023-12-27 11:53:36,754][20381] Updated weights for policy 1, policy_version 900913 (0.0011) [2023-12-27 11:53:37,055][20347] Updated weights for policy 0, policy_version 901621 (0.0009) [2023-12-27 11:53:37,115][20347] Updated weights for policy 0, policy_version 901631 (0.0008) [2023-12-27 11:53:37,179][20347] Updated weights for policy 0, policy_version 901641 (0.0010) [2023-12-27 11:53:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 461512704. Throughput: 0: 10195.6, 1: 10236.9. Samples: 461483264. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:37,364][19170] Avg episode reward: [(0, '961.140'), (1, '895.270')] [2023-12-27 11:53:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000900920_230662144.pth... [2023-12-27 11:53:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000901648_230850560.pth... [2023-12-27 11:53:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000899736_230359040.pth [2023-12-27 11:53:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000900464_230547456.pth [2023-12-27 11:53:37,461][20381] Updated weights for policy 1, policy_version 900923 (0.0010) [2023-12-27 11:53:37,514][20381] Updated weights for policy 1, policy_version 900933 (0.0007) [2023-12-27 11:53:37,570][20381] Updated weights for policy 1, policy_version 900943 (0.0007) [2023-12-27 11:53:37,849][20347] Updated weights for policy 0, policy_version 901651 (0.0011) [2023-12-27 11:53:37,907][20347] Updated weights for policy 0, policy_version 901661 (0.0010) [2023-12-27 11:53:37,962][20347] Updated weights for policy 0, policy_version 901671 (0.0010) [2023-12-27 11:53:38,286][20381] Updated weights for policy 1, policy_version 900953 (0.0008) [2023-12-27 11:53:38,341][20381] Updated weights for policy 1, policy_version 900963 (0.0007) [2023-12-27 11:53:38,397][20381] Updated weights for policy 1, policy_version 900973 (0.0006) [2023-12-27 11:53:38,451][20381] Updated weights for policy 1, policy_version 900983 (0.0007) [2023-12-27 11:53:38,682][20347] Updated weights for policy 0, policy_version 901681 (0.0010) [2023-12-27 11:53:38,748][20347] Updated weights for policy 0, policy_version 901691 (0.0008) [2023-12-27 11:53:38,811][20347] Updated weights for policy 0, policy_version 901701 (0.0008) [2023-12-27 11:53:38,869][20347] Updated weights for policy 0, policy_version 901711 (0.0008) [2023-12-27 11:53:39,143][20381] Updated weights for policy 1, policy_version 900993 (0.0005) [2023-12-27 11:53:39,203][20381] Updated weights for policy 1, policy_version 901003 (0.0009) [2023-12-27 11:53:39,273][20381] Updated weights for policy 1, policy_version 901013 (0.0010) [2023-12-27 11:53:39,579][20347] Updated weights for policy 0, policy_version 901721 (0.0009) [2023-12-27 11:53:39,652][20347] Updated weights for policy 0, policy_version 901731 (0.0008) [2023-12-27 11:53:39,724][20347] Updated weights for policy 0, policy_version 901741 (0.0008) [2023-12-27 11:53:39,998][20381] Updated weights for policy 1, policy_version 901023 (0.0011) [2023-12-27 11:53:40,058][20381] Updated weights for policy 1, policy_version 901033 (0.0011) [2023-12-27 11:53:40,121][20381] Updated weights for policy 1, policy_version 901043 (0.0011) [2023-12-27 11:53:40,429][20347] Updated weights for policy 0, policy_version 901751 (0.0008) [2023-12-27 11:53:40,484][20347] Updated weights for policy 0, policy_version 901761 (0.0007) [2023-12-27 11:53:40,532][20347] Updated weights for policy 0, policy_version 901771 (0.0007) [2023-12-27 11:53:40,813][20381] Updated weights for policy 1, policy_version 901053 (0.0008) [2023-12-27 11:53:40,871][20381] Updated weights for policy 1, policy_version 901063 (0.0005) [2023-12-27 11:53:40,925][20381] Updated weights for policy 1, policy_version 901073 (0.0005) [2023-12-27 11:53:41,261][20347] Updated weights for policy 0, policy_version 901781 (0.0007) [2023-12-27 11:53:41,323][20347] Updated weights for policy 0, policy_version 901791 (0.0007) [2023-12-27 11:53:41,391][20347] Updated weights for policy 0, policy_version 901801 (0.0009) [2023-12-27 11:53:41,589][20381] Updated weights for policy 1, policy_version 901083 (0.0006) [2023-12-27 11:53:41,656][20381] Updated weights for policy 1, policy_version 901093 (0.0008) [2023-12-27 11:53:41,715][20381] Updated weights for policy 1, policy_version 901103 (0.0007) [2023-12-27 11:53:42,116][20347] Updated weights for policy 0, policy_version 901811 (0.0008) [2023-12-27 11:53:42,164][20347] Updated weights for policy 0, policy_version 901821 (0.0009) [2023-12-27 11:53:42,220][20347] Updated weights for policy 0, policy_version 901831 (0.0007) [2023-12-27 11:53:42,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 461611008. Throughput: 0: 10207.8, 1: 10185.2. Samples: 461602236. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:42,363][19170] Avg episode reward: [(0, '961.180'), (1, '895.900')] [2023-12-27 11:53:42,405][20381] Updated weights for policy 1, policy_version 901113 (0.0007) [2023-12-27 11:53:42,469][20381] Updated weights for policy 1, policy_version 901123 (0.0008) [2023-12-27 11:53:42,526][20381] Updated weights for policy 1, policy_version 901133 (0.0008) [2023-12-27 11:53:42,579][20381] Updated weights for policy 1, policy_version 901143 (0.0008) [2023-12-27 11:53:42,857][20347] Updated weights for policy 0, policy_version 901841 (0.0007) [2023-12-27 11:53:42,918][20347] Updated weights for policy 0, policy_version 901851 (0.0007) [2023-12-27 11:53:42,971][20347] Updated weights for policy 0, policy_version 901861 (0.0007) [2023-12-27 11:53:43,027][20347] Updated weights for policy 0, policy_version 901871 (0.0008) [2023-12-27 11:53:43,187][20381] Updated weights for policy 1, policy_version 901153 (0.0005) [2023-12-27 11:53:43,235][20381] Updated weights for policy 1, policy_version 901163 (0.0005) [2023-12-27 11:53:43,292][20381] Updated weights for policy 1, policy_version 901173 (0.0005) [2023-12-27 11:53:43,766][20347] Updated weights for policy 0, policy_version 901881 (0.0009) [2023-12-27 11:53:43,827][20347] Updated weights for policy 0, policy_version 901891 (0.0007) [2023-12-27 11:53:43,839][20381] Updated weights for policy 1, policy_version 901183 (0.0005) [2023-12-27 11:53:43,889][20347] Updated weights for policy 0, policy_version 901901 (0.0007) [2023-12-27 11:53:43,893][20381] Updated weights for policy 1, policy_version 901193 (0.0006) [2023-12-27 11:53:43,951][20381] Updated weights for policy 1, policy_version 901203 (0.0006) [2023-12-27 11:53:44,539][20381] Updated weights for policy 1, policy_version 901213 (0.0006) [2023-12-27 11:53:44,600][20381] Updated weights for policy 1, policy_version 901223 (0.0006) [2023-12-27 11:53:44,641][20347] Updated weights for policy 0, policy_version 901911 (0.0006) [2023-12-27 11:53:44,656][20381] Updated weights for policy 1, policy_version 901233 (0.0006) [2023-12-27 11:53:44,709][20347] Updated weights for policy 0, policy_version 901921 (0.0005) [2023-12-27 11:53:44,779][20347] Updated weights for policy 0, policy_version 901931 (0.0006) [2023-12-27 11:53:45,318][20381] Updated weights for policy 1, policy_version 901243 (0.0006) [2023-12-27 11:53:45,356][20292] KL-divergence is very high: 226.2261 [2023-12-27 11:53:45,385][20381] Updated weights for policy 1, policy_version 901253 (0.0008) [2023-12-27 11:53:45,422][20347] Updated weights for policy 0, policy_version 901941 (0.0007) [2023-12-27 11:53:45,439][20381] Updated weights for policy 1, policy_version 901263 (0.0007) [2023-12-27 11:53:45,490][20347] Updated weights for policy 0, policy_version 901951 (0.0007) [2023-12-27 11:53:45,552][20347] Updated weights for policy 0, policy_version 901961 (0.0005) [2023-12-27 11:53:46,104][20381] Updated weights for policy 1, policy_version 901273 (0.0008) [2023-12-27 11:53:46,146][20292] KL-divergence is very high: 136.0018 [2023-12-27 11:53:46,158][20381] Updated weights for policy 1, policy_version 901283 (0.0010) [2023-12-27 11:53:46,174][20292] KL-divergence is very high: 106.4521 [2023-12-27 11:53:46,204][20347] Updated weights for policy 0, policy_version 901971 (0.0006) [2023-12-27 11:53:46,216][20381] Updated weights for policy 1, policy_version 901293 (0.0010) [2023-12-27 11:53:46,222][20292] KL-divergence is very high: 159.2766 [2023-12-27 11:53:46,238][20292] KL-divergence is very high: 171.7607 [2023-12-27 11:53:46,258][20347] Updated weights for policy 0, policy_version 901981 (0.0005) [2023-12-27 11:53:46,271][20381] Updated weights for policy 1, policy_version 901303 (0.0010) [2023-12-27 11:53:46,310][20347] Updated weights for policy 0, policy_version 901991 (0.0006) [2023-12-27 11:53:46,963][20381] Updated weights for policy 1, policy_version 901313 (0.0009) [2023-12-27 11:53:46,999][20347] Updated weights for policy 0, policy_version 902001 (0.0007) [2023-12-27 11:53:47,026][20381] Updated weights for policy 1, policy_version 901323 (0.0007) [2023-12-27 11:53:47,057][20347] Updated weights for policy 0, policy_version 902011 (0.0006) [2023-12-27 11:53:47,079][20381] Updated weights for policy 1, policy_version 901333 (0.0009) [2023-12-27 11:53:47,115][20347] Updated weights for policy 0, policy_version 902021 (0.0007) [2023-12-27 11:53:47,166][20347] Updated weights for policy 0, policy_version 902031 (0.0007) [2023-12-27 11:53:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 461717504. Throughput: 0: 10181.0, 1: 10266.4. Samples: 461725892. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:47,364][19170] Avg episode reward: [(0, '990.260'), (1, '884.600')] [2023-12-27 11:53:47,678][20292] KL-divergence is very high: 120.7518 [2023-12-27 11:53:47,681][20381] Updated weights for policy 1, policy_version 901343 (0.0007) [2023-12-27 11:53:47,695][20292] KL-divergence is very high: 103.0839 [2023-12-27 11:53:47,736][20292] KL-divergence is very high: 357.9378 [2023-12-27 11:53:47,737][20381] Updated weights for policy 1, policy_version 901353 (0.0008) [2023-12-27 11:53:47,763][20292] KL-divergence is very high: 120.8373 [2023-12-27 11:53:47,794][20381] Updated weights for policy 1, policy_version 901363 (0.0006) [2023-12-27 11:53:47,872][20347] Updated weights for policy 0, policy_version 902041 (0.0007) [2023-12-27 11:53:47,930][20347] Updated weights for policy 0, policy_version 902051 (0.0010) [2023-12-27 11:53:47,984][20347] Updated weights for policy 0, policy_version 902061 (0.0007) [2023-12-27 11:53:48,427][20292] KL-divergence is very high: 144.0722 [2023-12-27 11:53:48,443][20381] Updated weights for policy 1, policy_version 901373 (0.0005) [2023-12-27 11:53:48,475][20292] KL-divergence is very high: 151.2198 [2023-12-27 11:53:48,506][20381] Updated weights for policy 1, policy_version 901383 (0.0005) [2023-12-27 11:53:48,526][20292] KL-divergence is very high: 394.2057 [2023-12-27 11:53:48,569][20381] Updated weights for policy 1, policy_version 901393 (0.0008) [2023-12-27 11:53:48,575][20292] KL-divergence is very high: 362.0617 [2023-12-27 11:53:48,646][20347] Updated weights for policy 0, policy_version 902071 (0.0010) [2023-12-27 11:53:48,701][20347] Updated weights for policy 0, policy_version 902081 (0.0010) [2023-12-27 11:53:48,759][20347] Updated weights for policy 0, policy_version 902091 (0.0010) [2023-12-27 11:53:49,226][20381] Updated weights for policy 1, policy_version 901403 (0.0007) [2023-12-27 11:53:49,261][20292] KL-divergence is very high: 125.9359 [2023-12-27 11:53:49,304][20381] Updated weights for policy 1, policy_version 901413 (0.0008) [2023-12-27 11:53:49,311][20292] KL-divergence is very high: 138.6549 [2023-12-27 11:53:49,363][20292] KL-divergence is very high: 177.2354 [2023-12-27 11:53:49,369][20381] Updated weights for policy 1, policy_version 901423 (0.0008) [2023-12-27 11:53:49,408][20292] KL-divergence is very high: 107.9410 [2023-12-27 11:53:49,468][20347] Updated weights for policy 0, policy_version 902101 (0.0009) [2023-12-27 11:53:49,524][20347] Updated weights for policy 0, policy_version 902111 (0.0007) [2023-12-27 11:53:49,584][20347] Updated weights for policy 0, policy_version 902121 (0.0007) [2023-12-27 11:53:50,042][20381] Updated weights for policy 1, policy_version 901433 (0.0008) [2023-12-27 11:53:50,101][20381] Updated weights for policy 1, policy_version 901443 (0.0008) [2023-12-27 11:53:50,103][20292] KL-divergence is very high: 142.4849 [2023-12-27 11:53:50,169][20381] Updated weights for policy 1, policy_version 901453 (0.0009) [2023-12-27 11:53:50,237][20381] Updated weights for policy 1, policy_version 901463 (0.0005) [2023-12-27 11:53:50,291][20347] Updated weights for policy 0, policy_version 902131 (0.0007) [2023-12-27 11:53:50,351][20347] Updated weights for policy 0, policy_version 902141 (0.0006) [2023-12-27 11:53:50,399][20347] Updated weights for policy 0, policy_version 902151 (0.0010) [2023-12-27 11:53:50,909][20381] Updated weights for policy 1, policy_version 901473 (0.0007) [2023-12-27 11:53:50,962][20381] Updated weights for policy 1, policy_version 901483 (0.0007) [2023-12-27 11:53:51,018][20381] Updated weights for policy 1, policy_version 901493 (0.0006) [2023-12-27 11:53:51,087][20347] Updated weights for policy 0, policy_version 902161 (0.0009) [2023-12-27 11:53:51,151][20347] Updated weights for policy 0, policy_version 902171 (0.0008) [2023-12-27 11:53:51,210][20347] Updated weights for policy 0, policy_version 902181 (0.0008) [2023-12-27 11:53:51,272][20347] Updated weights for policy 0, policy_version 902191 (0.0008) [2023-12-27 11:53:51,718][20381] Updated weights for policy 1, policy_version 901503 (0.0009) [2023-12-27 11:53:51,788][20381] Updated weights for policy 1, policy_version 901513 (0.0006) [2023-12-27 11:53:51,854][20381] Updated weights for policy 1, policy_version 901523 (0.0007) [2023-12-27 11:53:52,019][20347] Updated weights for policy 0, policy_version 902201 (0.0008) [2023-12-27 11:53:52,079][20347] Updated weights for policy 0, policy_version 902211 (0.0008) [2023-12-27 11:53:52,138][20347] Updated weights for policy 0, policy_version 902221 (0.0007) [2023-12-27 11:53:52,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 461815808. Throughput: 0: 10166.6, 1: 10285.4. Samples: 461787012. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:52,363][19170] Avg episode reward: [(0, '990.290'), (1, '670.620')] [2023-12-27 11:53:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000901528_230817792.pth... [2023-12-27 11:53:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000902224_230998016.pth... [2023-12-27 11:53:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000900312_230506496.pth [2023-12-27 11:53:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000901040_230694912.pth [2023-12-27 11:53:52,519][20381] Updated weights for policy 1, policy_version 901533 (0.0008) [2023-12-27 11:53:52,574][20381] Updated weights for policy 1, policy_version 901543 (0.0007) [2023-12-27 11:53:52,626][20381] Updated weights for policy 1, policy_version 901553 (0.0007) [2023-12-27 11:53:52,890][20347] Updated weights for policy 0, policy_version 902231 (0.0008) [2023-12-27 11:53:52,921][20291] KL-divergence is very high: 129.7222 [2023-12-27 11:53:52,944][20347] Updated weights for policy 0, policy_version 902241 (0.0008) [2023-12-27 11:53:52,994][20347] Updated weights for policy 0, policy_version 902251 (0.0008) [2023-12-27 11:53:53,319][20381] Updated weights for policy 1, policy_version 901563 (0.0007) [2023-12-27 11:53:53,375][20381] Updated weights for policy 1, policy_version 901573 (0.0008) [2023-12-27 11:53:53,434][20381] Updated weights for policy 1, policy_version 901583 (0.0008) [2023-12-27 11:53:53,719][20347] Updated weights for policy 0, policy_version 902261 (0.0008) [2023-12-27 11:53:53,768][20347] Updated weights for policy 0, policy_version 902271 (0.0009) [2023-12-27 11:53:53,824][20347] Updated weights for policy 0, policy_version 902281 (0.0008) [2023-12-27 11:53:54,014][20381] Updated weights for policy 1, policy_version 901593 (0.0008) [2023-12-27 11:53:54,070][20381] Updated weights for policy 1, policy_version 901603 (0.0005) [2023-12-27 11:53:54,131][20381] Updated weights for policy 1, policy_version 901613 (0.0005) [2023-12-27 11:53:54,186][20381] Updated weights for policy 1, policy_version 901623 (0.0006) [2023-12-27 11:53:54,598][20347] Updated weights for policy 0, policy_version 902291 (0.0007) [2023-12-27 11:53:54,648][20347] Updated weights for policy 0, policy_version 902301 (0.0008) [2023-12-27 11:53:54,675][20292] KL-divergence is very high: 130.0269 [2023-12-27 11:53:54,695][20347] Updated weights for policy 0, policy_version 902311 (0.0008) [2023-12-27 11:53:54,720][20381] Updated weights for policy 1, policy_version 901633 (0.0005) [2023-12-27 11:53:54,776][20381] Updated weights for policy 1, policy_version 901643 (0.0005) [2023-12-27 11:53:54,832][20381] Updated weights for policy 1, policy_version 901653 (0.0005) [2023-12-27 11:53:55,364][20292] KL-divergence is very high: 105.9524 [2023-12-27 11:53:55,370][20381] Updated weights for policy 1, policy_version 901663 (0.0005) [2023-12-27 11:53:55,419][20381] Updated weights for policy 1, policy_version 901673 (0.0006) [2023-12-27 11:53:55,432][20347] Updated weights for policy 0, policy_version 902321 (0.0008) [2023-12-27 11:53:55,478][20381] Updated weights for policy 1, policy_version 901683 (0.0005) [2023-12-27 11:53:55,494][20347] Updated weights for policy 0, policy_version 902331 (0.0008) [2023-12-27 11:53:55,556][20347] Updated weights for policy 0, policy_version 902341 (0.0008) [2023-12-27 11:53:55,618][20347] Updated weights for policy 0, policy_version 902351 (0.0008) [2023-12-27 11:53:56,025][20381] Updated weights for policy 1, policy_version 901693 (0.0006) [2023-12-27 11:53:56,077][20381] Updated weights for policy 1, policy_version 901703 (0.0008) [2023-12-27 11:53:56,132][20381] Updated weights for policy 1, policy_version 901713 (0.0008) [2023-12-27 11:53:56,271][20347] Updated weights for policy 0, policy_version 902361 (0.0005) [2023-12-27 11:53:56,316][20347] Updated weights for policy 0, policy_version 902371 (0.0005) [2023-12-27 11:53:56,368][20347] Updated weights for policy 0, policy_version 902381 (0.0006) [2023-12-27 11:53:56,799][20381] Updated weights for policy 1, policy_version 901723 (0.0008) [2023-12-27 11:53:56,846][20381] Updated weights for policy 1, policy_version 901733 (0.0005) [2023-12-27 11:53:56,905][20381] Updated weights for policy 1, policy_version 901743 (0.0005) [2023-12-27 11:53:56,986][20347] Updated weights for policy 0, policy_version 902391 (0.0007) [2023-12-27 11:53:57,044][20347] Updated weights for policy 0, policy_version 902401 (0.0007) [2023-12-27 11:53:57,105][20347] Updated weights for policy 0, policy_version 902411 (0.0006) [2023-12-27 11:53:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 461922304. Throughput: 0: 10032.6, 1: 10400.4. Samples: 461910320. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:53:57,363][19170] Avg episode reward: [(0, '980.650'), (1, '573.810')] [2023-12-27 11:53:57,536][20381] Updated weights for policy 1, policy_version 901753 (0.0006) [2023-12-27 11:53:57,587][20381] Updated weights for policy 1, policy_version 901763 (0.0007) [2023-12-27 11:53:57,636][20381] Updated weights for policy 1, policy_version 901773 (0.0007) [2023-12-27 11:53:57,680][20381] Updated weights for policy 1, policy_version 901783 (0.0005) [2023-12-27 11:53:57,737][20347] Updated weights for policy 0, policy_version 902421 (0.0009) [2023-12-27 11:53:57,796][20347] Updated weights for policy 0, policy_version 902431 (0.0010) [2023-12-27 11:53:57,853][20347] Updated weights for policy 0, policy_version 902441 (0.0010) [2023-12-27 11:53:58,314][20381] Updated weights for policy 1, policy_version 901793 (0.0009) [2023-12-27 11:53:58,378][20381] Updated weights for policy 1, policy_version 901803 (0.0008) [2023-12-27 11:53:58,442][20381] Updated weights for policy 1, policy_version 901813 (0.0008) [2023-12-27 11:53:58,448][20292] KL-divergence is very high: 153.9472 [2023-12-27 11:53:58,554][20347] Updated weights for policy 0, policy_version 902451 (0.0010) [2023-12-27 11:53:58,619][20291] KL-divergence is very high: 187.0277 [2023-12-27 11:53:58,626][20347] Updated weights for policy 0, policy_version 902461 (0.0008) [2023-12-27 11:53:58,632][20291] KL-divergence is very high: 181.8213 [2023-12-27 11:53:58,689][20347] Updated weights for policy 0, policy_version 902471 (0.0007) [2023-12-27 11:53:59,185][20292] KL-divergence is very high: 142.2660 [2023-12-27 11:53:59,222][20381] Updated weights for policy 1, policy_version 901823 (0.0007) [2023-12-27 11:53:59,286][20381] Updated weights for policy 1, policy_version 901833 (0.0008) [2023-12-27 11:53:59,345][20381] Updated weights for policy 1, policy_version 901843 (0.0008) [2023-12-27 11:53:59,361][20292] KL-divergence is very high: 138.5136 [2023-12-27 11:53:59,503][20347] Updated weights for policy 0, policy_version 902481 (0.0009) [2023-12-27 11:53:59,564][20347] Updated weights for policy 0, policy_version 902491 (0.0010) [2023-12-27 11:53:59,617][20347] Updated weights for policy 0, policy_version 902501 (0.0010) [2023-12-27 11:53:59,673][20347] Updated weights for policy 0, policy_version 902511 (0.0010) [2023-12-27 11:53:59,978][20381] Updated weights for policy 1, policy_version 901853 (0.0007) [2023-12-27 11:54:00,041][20381] Updated weights for policy 1, policy_version 901863 (0.0007) [2023-12-27 11:54:00,094][20381] Updated weights for policy 1, policy_version 901873 (0.0008) [2023-12-27 11:54:00,357][20347] Updated weights for policy 0, policy_version 902521 (0.0006) [2023-12-27 11:54:00,420][20347] Updated weights for policy 0, policy_version 902531 (0.0008) [2023-12-27 11:54:00,484][20347] Updated weights for policy 0, policy_version 902541 (0.0006) [2023-12-27 11:54:00,761][20381] Updated weights for policy 1, policy_version 901883 (0.0007) [2023-12-27 11:54:00,784][20292] KL-divergence is very high: 116.4310 [2023-12-27 11:54:00,816][20381] Updated weights for policy 1, policy_version 901893 (0.0007) [2023-12-27 11:54:00,828][20292] KL-divergence is very high: 103.7202 [2023-12-27 11:54:00,875][20381] Updated weights for policy 1, policy_version 901903 (0.0011) [2023-12-27 11:54:01,101][20347] Updated weights for policy 0, policy_version 902551 (0.0005) [2023-12-27 11:54:01,166][20347] Updated weights for policy 0, policy_version 902561 (0.0008) [2023-12-27 11:54:01,226][20347] Updated weights for policy 0, policy_version 902571 (0.0010) [2023-12-27 11:54:01,557][20381] Updated weights for policy 1, policy_version 901913 (0.0011) [2023-12-27 11:54:01,606][20381] Updated weights for policy 1, policy_version 901923 (0.0010) [2023-12-27 11:54:01,669][20381] Updated weights for policy 1, policy_version 901933 (0.0009) [2023-12-27 11:54:01,720][20381] Updated weights for policy 1, policy_version 901943 (0.0008) [2023-12-27 11:54:01,938][20347] Updated weights for policy 0, policy_version 902581 (0.0010) [2023-12-27 11:54:01,995][20347] Updated weights for policy 0, policy_version 902591 (0.0010) [2023-12-27 11:54:02,053][20347] Updated weights for policy 0, policy_version 902601 (0.0010) [2023-12-27 11:54:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 462020608. Throughput: 0: 10012.7, 1: 10466.2. Samples: 462032592. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:54:02,363][19170] Avg episode reward: [(0, '990.240'), (1, '53.760')] [2023-12-27 11:54:02,410][20381] Updated weights for policy 1, policy_version 901953 (0.0007) [2023-12-27 11:54:02,469][20381] Updated weights for policy 1, policy_version 901963 (0.0010) [2023-12-27 11:54:02,528][20381] Updated weights for policy 1, policy_version 901973 (0.0011) [2023-12-27 11:54:02,783][20347] Updated weights for policy 0, policy_version 902611 (0.0009) [2023-12-27 11:54:02,842][20347] Updated weights for policy 0, policy_version 902621 (0.0005) [2023-12-27 11:54:02,901][20347] Updated weights for policy 0, policy_version 902631 (0.0010) [2023-12-27 11:54:03,156][20381] Updated weights for policy 1, policy_version 901983 (0.0007) [2023-12-27 11:54:03,214][20381] Updated weights for policy 1, policy_version 901993 (0.0005) [2023-12-27 11:54:03,269][20381] Updated weights for policy 1, policy_version 902003 (0.0005) [2023-12-27 11:54:03,547][20347] Updated weights for policy 0, policy_version 902641 (0.0010) [2023-12-27 11:54:03,600][20347] Updated weights for policy 0, policy_version 902651 (0.0010) [2023-12-27 11:54:03,655][20347] Updated weights for policy 0, policy_version 902661 (0.0010) [2023-12-27 11:54:03,700][20347] Updated weights for policy 0, policy_version 902671 (0.0010) [2023-12-27 11:54:03,804][20381] Updated weights for policy 1, policy_version 902013 (0.0006) [2023-12-27 11:54:03,868][20381] Updated weights for policy 1, policy_version 902023 (0.0007) [2023-12-27 11:54:03,924][20381] Updated weights for policy 1, policy_version 902033 (0.0007) [2023-12-27 11:54:04,402][20347] Updated weights for policy 0, policy_version 902681 (0.0011) [2023-12-27 11:54:04,459][20347] Updated weights for policy 0, policy_version 902691 (0.0011) [2023-12-27 11:54:04,513][20347] Updated weights for policy 0, policy_version 902701 (0.0010) [2023-12-27 11:54:04,612][20381] Updated weights for policy 1, policy_version 902043 (0.0007) [2023-12-27 11:54:04,667][20381] Updated weights for policy 1, policy_version 902053 (0.0008) [2023-12-27 11:54:04,717][20381] Updated weights for policy 1, policy_version 902063 (0.0008) [2023-12-27 11:54:05,175][20347] Updated weights for policy 0, policy_version 902711 (0.0010) [2023-12-27 11:54:05,237][20347] Updated weights for policy 0, policy_version 902721 (0.0010) [2023-12-27 11:54:05,284][20347] Updated weights for policy 0, policy_version 902731 (0.0010) [2023-12-27 11:54:05,406][20381] Updated weights for policy 1, policy_version 902073 (0.0007) [2023-12-27 11:54:05,471][20381] Updated weights for policy 1, policy_version 902083 (0.0007) [2023-12-27 11:54:05,520][20381] Updated weights for policy 1, policy_version 902093 (0.0010) [2023-12-27 11:54:05,571][20381] Updated weights for policy 1, policy_version 902103 (0.0010) [2023-12-27 11:54:05,953][20347] Updated weights for policy 0, policy_version 902741 (0.0009) [2023-12-27 11:54:06,007][20347] Updated weights for policy 0, policy_version 902751 (0.0010) [2023-12-27 11:54:06,058][20347] Updated weights for policy 0, policy_version 902761 (0.0010) [2023-12-27 11:54:06,177][20381] Updated weights for policy 1, policy_version 902113 (0.0009) [2023-12-27 11:54:06,236][20381] Updated weights for policy 1, policy_version 902123 (0.0011) [2023-12-27 11:54:06,298][20381] Updated weights for policy 1, policy_version 902133 (0.0011) [2023-12-27 11:54:06,790][20347] Updated weights for policy 0, policy_version 902771 (0.0009) [2023-12-27 11:54:06,841][20347] Updated weights for policy 0, policy_version 902781 (0.0008) [2023-12-27 11:54:06,889][20347] Updated weights for policy 0, policy_version 902791 (0.0007) [2023-12-27 11:54:06,998][20381] Updated weights for policy 1, policy_version 902143 (0.0010) [2023-12-27 11:54:07,065][20381] Updated weights for policy 1, policy_version 902153 (0.0011) [2023-12-27 11:54:07,120][20381] Updated weights for policy 1, policy_version 902163 (0.0011) [2023-12-27 11:54:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 462127104. Throughput: 0: 9996.8, 1: 10466.9. Samples: 462095092. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:54:07,363][19170] Avg episode reward: [(0, '961.110'), (1, '89.960')] [2023-12-27 11:54:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000902168_230981632.pth... [2023-12-27 11:54:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000902800_231145472.pth... [2023-12-27 11:54:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000900920_230662144.pth [2023-12-27 11:54:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000901648_230850560.pth [2023-12-27 11:54:07,600][20347] Updated weights for policy 0, policy_version 902801 (0.0007) [2023-12-27 11:54:07,660][20347] Updated weights for policy 0, policy_version 902811 (0.0010) [2023-12-27 11:54:07,717][20347] Updated weights for policy 0, policy_version 902821 (0.0011) [2023-12-27 11:54:07,773][20347] Updated weights for policy 0, policy_version 902831 (0.0010) [2023-12-27 11:54:07,787][20381] Updated weights for policy 1, policy_version 902173 (0.0011) [2023-12-27 11:54:07,849][20381] Updated weights for policy 1, policy_version 902183 (0.0007) [2023-12-27 11:54:07,897][20381] Updated weights for policy 1, policy_version 902193 (0.0008) [2023-12-27 11:54:08,455][20347] Updated weights for policy 0, policy_version 902841 (0.0010) [2023-12-27 11:54:08,507][20347] Updated weights for policy 0, policy_version 902851 (0.0010) [2023-12-27 11:54:08,564][20347] Updated weights for policy 0, policy_version 902861 (0.0011) [2023-12-27 11:54:08,573][20381] Updated weights for policy 1, policy_version 902203 (0.0007) [2023-12-27 11:54:08,621][20381] Updated weights for policy 1, policy_version 902213 (0.0007) [2023-12-27 11:54:08,672][20381] Updated weights for policy 1, policy_version 902223 (0.0008) [2023-12-27 11:54:09,231][20347] Updated weights for policy 0, policy_version 902871 (0.0010) [2023-12-27 11:54:09,288][20347] Updated weights for policy 0, policy_version 902881 (0.0010) [2023-12-27 11:54:09,304][20381] Updated weights for policy 1, policy_version 902233 (0.0008) [2023-12-27 11:54:09,340][20347] Updated weights for policy 0, policy_version 902891 (0.0010) [2023-12-27 11:54:09,370][20381] Updated weights for policy 1, policy_version 902243 (0.0009) [2023-12-27 11:54:09,433][20381] Updated weights for policy 1, policy_version 902253 (0.0008) [2023-12-27 11:54:09,495][20381] Updated weights for policy 1, policy_version 902263 (0.0006) [2023-12-27 11:54:10,079][20347] Updated weights for policy 0, policy_version 902901 (0.0010) [2023-12-27 11:54:10,138][20347] Updated weights for policy 0, policy_version 902911 (0.0008) [2023-12-27 11:54:10,173][20381] Updated weights for policy 1, policy_version 902273 (0.0008) [2023-12-27 11:54:10,200][20347] Updated weights for policy 0, policy_version 902921 (0.0006) [2023-12-27 11:54:10,239][20381] Updated weights for policy 1, policy_version 902283 (0.0007) [2023-12-27 11:54:10,301][20381] Updated weights for policy 1, policy_version 902293 (0.0008) [2023-12-27 11:54:10,827][20347] Updated weights for policy 0, policy_version 902931 (0.0011) [2023-12-27 11:54:10,876][20347] Updated weights for policy 0, policy_version 902941 (0.0010) [2023-12-27 11:54:10,931][20347] Updated weights for policy 0, policy_version 902951 (0.0010) [2023-12-27 11:54:10,974][20381] Updated weights for policy 1, policy_version 902303 (0.0006) [2023-12-27 11:54:11,026][20381] Updated weights for policy 1, policy_version 902313 (0.0007) [2023-12-27 11:54:11,088][20381] Updated weights for policy 1, policy_version 902323 (0.0006) [2023-12-27 11:54:11,696][20347] Updated weights for policy 0, policy_version 902961 (0.0010) [2023-12-27 11:54:11,765][20347] Updated weights for policy 0, policy_version 902971 (0.0010) [2023-12-27 11:54:11,823][20347] Updated weights for policy 0, policy_version 902981 (0.0010) [2023-12-27 11:54:11,883][20347] Updated weights for policy 0, policy_version 902991 (0.0011) [2023-12-27 11:54:11,910][20381] Updated weights for policy 1, policy_version 902333 (0.0007) [2023-12-27 11:54:11,965][20381] Updated weights for policy 1, policy_version 902343 (0.0006) [2023-12-27 11:54:12,027][20381] Updated weights for policy 1, policy_version 902353 (0.0008) [2023-12-27 11:54:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 462225408. Throughput: 0: 10010.3, 1: 10456.3. Samples: 462216772. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 11:54:12,363][19170] Avg episode reward: [(0, '951.370'), (1, '75.450')] [2023-12-27 11:54:12,586][20347] Updated weights for policy 0, policy_version 903001 (0.0007) [2023-12-27 11:54:12,647][20347] Updated weights for policy 0, policy_version 903011 (0.0007) [2023-12-27 11:54:12,694][20381] Updated weights for policy 1, policy_version 902363 (0.0007) [2023-12-27 11:54:12,705][20347] Updated weights for policy 0, policy_version 903021 (0.0009) [2023-12-27 11:54:12,758][20381] Updated weights for policy 1, policy_version 902373 (0.0006) [2023-12-27 11:54:12,823][20381] Updated weights for policy 1, policy_version 902383 (0.0006) [2023-12-27 11:54:13,326][20347] Updated weights for policy 0, policy_version 903031 (0.0007) [2023-12-27 11:54:13,374][20347] Updated weights for policy 0, policy_version 903041 (0.0007) [2023-12-27 11:54:13,428][20347] Updated weights for policy 0, policy_version 903051 (0.0007) [2023-12-27 11:54:13,510][20381] Updated weights for policy 1, policy_version 902393 (0.0007) [2023-12-27 11:54:13,561][20381] Updated weights for policy 1, policy_version 902403 (0.0010) [2023-12-27 11:54:13,610][20381] Updated weights for policy 1, policy_version 902413 (0.0010) [2023-12-27 11:54:13,670][20381] Updated weights for policy 1, policy_version 902423 (0.0010) [2023-12-27 11:54:14,133][20347] Updated weights for policy 0, policy_version 903061 (0.0007) [2023-12-27 11:54:14,195][20347] Updated weights for policy 0, policy_version 903071 (0.0005) [2023-12-27 11:54:14,255][20347] Updated weights for policy 0, policy_version 903081 (0.0006) [2023-12-27 11:54:14,299][20381] Updated weights for policy 1, policy_version 902433 (0.0006) [2023-12-27 11:54:14,361][20381] Updated weights for policy 1, policy_version 902443 (0.0006) [2023-12-27 11:54:14,415][20381] Updated weights for policy 1, policy_version 902453 (0.0007) [2023-12-27 11:54:14,926][20347] Updated weights for policy 0, policy_version 903091 (0.0008) [2023-12-27 11:54:14,985][20347] Updated weights for policy 0, policy_version 903101 (0.0008) [2023-12-27 11:54:15,044][20347] Updated weights for policy 0, policy_version 903111 (0.0008) [2023-12-27 11:54:15,062][20381] Updated weights for policy 1, policy_version 902463 (0.0007) [2023-12-27 11:54:15,113][20381] Updated weights for policy 1, policy_version 902473 (0.0006) [2023-12-27 11:54:15,167][20381] Updated weights for policy 1, policy_version 902483 (0.0008) [2023-12-27 11:54:15,708][20347] Updated weights for policy 0, policy_version 903121 (0.0007) [2023-12-27 11:54:15,765][20347] Updated weights for policy 0, policy_version 903131 (0.0007) [2023-12-27 11:54:15,811][20381] Updated weights for policy 1, policy_version 902493 (0.0008) [2023-12-27 11:54:15,822][20347] Updated weights for policy 0, policy_version 903141 (0.0007) [2023-12-27 11:54:15,869][20381] Updated weights for policy 1, policy_version 902503 (0.0008) [2023-12-27 11:54:15,881][20347] Updated weights for policy 0, policy_version 903151 (0.0005) [2023-12-27 11:54:15,922][20381] Updated weights for policy 1, policy_version 902513 (0.0007) [2023-12-27 11:54:16,532][20347] Updated weights for policy 0, policy_version 903161 (0.0005) [2023-12-27 11:54:16,587][20381] Updated weights for policy 1, policy_version 902523 (0.0007) [2023-12-27 11:54:16,589][20347] Updated weights for policy 0, policy_version 903171 (0.0006) [2023-12-27 11:54:16,644][20381] Updated weights for policy 1, policy_version 902533 (0.0006) [2023-12-27 11:54:16,645][20347] Updated weights for policy 0, policy_version 903181 (0.0005) [2023-12-27 11:54:16,701][20381] Updated weights for policy 1, policy_version 902543 (0.0007) [2023-12-27 11:54:17,296][20381] Updated weights for policy 1, policy_version 902553 (0.0006) [2023-12-27 11:54:17,335][20347] Updated weights for policy 0, policy_version 903191 (0.0007) [2023-12-27 11:54:17,352][20381] Updated weights for policy 1, policy_version 902563 (0.0005) [2023-12-27 11:54:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.6, 300 sec: 20466.1). Total num frames: 462323712. Throughput: 0: 9964.9, 1: 10486.8. Samples: 462340488. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:17,363][19170] Avg episode reward: [(0, '942.230'), (1, '48.810')] [2023-12-27 11:54:17,393][20347] Updated weights for policy 0, policy_version 903201 (0.0008) [2023-12-27 11:54:17,407][20381] Updated weights for policy 1, policy_version 902573 (0.0006) [2023-12-27 11:54:17,443][20347] Updated weights for policy 0, policy_version 903211 (0.0006) [2023-12-27 11:54:17,457][20381] Updated weights for policy 1, policy_version 902583 (0.0006) [2023-12-27 11:54:18,062][20381] Updated weights for policy 1, policy_version 902593 (0.0006) [2023-12-27 11:54:18,115][20381] Updated weights for policy 1, policy_version 902603 (0.0007) [2023-12-27 11:54:18,163][20347] Updated weights for policy 0, policy_version 903221 (0.0006) [2023-12-27 11:54:18,179][20381] Updated weights for policy 1, policy_version 902613 (0.0011) [2023-12-27 11:54:18,223][20347] Updated weights for policy 0, policy_version 903231 (0.0006) [2023-12-27 11:54:18,278][20347] Updated weights for policy 0, policy_version 903241 (0.0007) [2023-12-27 11:54:18,878][20381] Updated weights for policy 1, policy_version 902623 (0.0008) [2023-12-27 11:54:18,941][20381] Updated weights for policy 1, policy_version 902633 (0.0006) [2023-12-27 11:54:18,991][20347] Updated weights for policy 0, policy_version 903251 (0.0008) [2023-12-27 11:54:19,003][20381] Updated weights for policy 1, policy_version 902643 (0.0007) [2023-12-27 11:54:19,047][20347] Updated weights for policy 0, policy_version 903261 (0.0007) [2023-12-27 11:54:19,103][20347] Updated weights for policy 0, policy_version 903271 (0.0010) [2023-12-27 11:54:19,652][20381] Updated weights for policy 1, policy_version 902653 (0.0008) [2023-12-27 11:54:19,704][20381] Updated weights for policy 1, policy_version 902663 (0.0006) [2023-12-27 11:54:19,755][20347] Updated weights for policy 0, policy_version 903281 (0.0006) [2023-12-27 11:54:19,763][20381] Updated weights for policy 1, policy_version 902673 (0.0005) [2023-12-27 11:54:19,813][20347] Updated weights for policy 0, policy_version 903291 (0.0007) [2023-12-27 11:54:19,878][20347] Updated weights for policy 0, policy_version 903301 (0.0009) [2023-12-27 11:54:19,940][20347] Updated weights for policy 0, policy_version 903311 (0.0007) [2023-12-27 11:54:20,463][20381] Updated weights for policy 1, policy_version 902683 (0.0006) [2023-12-27 11:54:20,509][20381] Updated weights for policy 1, policy_version 902693 (0.0008) [2023-12-27 11:54:20,569][20381] Updated weights for policy 1, policy_version 902703 (0.0006) [2023-12-27 11:54:20,655][20347] Updated weights for policy 0, policy_version 903321 (0.0007) [2023-12-27 11:54:20,706][20347] Updated weights for policy 0, policy_version 903331 (0.0007) [2023-12-27 11:54:20,755][20347] Updated weights for policy 0, policy_version 903341 (0.0007) [2023-12-27 11:54:21,306][20381] Updated weights for policy 1, policy_version 902713 (0.0008) [2023-12-27 11:54:21,373][20381] Updated weights for policy 1, policy_version 902723 (0.0006) [2023-12-27 11:54:21,435][20381] Updated weights for policy 1, policy_version 902733 (0.0007) [2023-12-27 11:54:21,497][20381] Updated weights for policy 1, policy_version 902743 (0.0005) [2023-12-27 11:54:21,582][20347] Updated weights for policy 0, policy_version 903351 (0.0007) [2023-12-27 11:54:21,644][20347] Updated weights for policy 0, policy_version 903361 (0.0008) [2023-12-27 11:54:21,716][20347] Updated weights for policy 0, policy_version 903371 (0.0007) [2023-12-27 11:54:22,220][20381] Updated weights for policy 1, policy_version 902753 (0.0010) [2023-12-27 11:54:22,276][20381] Updated weights for policy 1, policy_version 902763 (0.0009) [2023-12-27 11:54:22,342][20381] Updated weights for policy 1, policy_version 902773 (0.0011) [2023-12-27 11:54:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 462422016. Throughput: 0: 9944.8, 1: 10441.5. Samples: 462400648. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:22,363][19170] Avg episode reward: [(0, '951.990'), (1, '37.710')] [2023-12-27 11:54:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000902776_231137280.pth... [2023-12-27 11:54:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000901528_230817792.pth [2023-12-27 11:54:22,379][20347] Updated weights for policy 0, policy_version 903381 (0.0009) [2023-12-27 11:54:22,437][20347] Updated weights for policy 0, policy_version 903391 (0.0006) [2023-12-27 11:54:22,498][20347] Updated weights for policy 0, policy_version 903401 (0.0005) [2023-12-27 11:54:22,541][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000903408_231301120.pth... [2023-12-27 11:54:22,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000902224_230998016.pth [2023-12-27 11:54:23,068][20381] Updated weights for policy 1, policy_version 902783 (0.0008) [2023-12-27 11:54:23,113][20347] Updated weights for policy 0, policy_version 903411 (0.0006) [2023-12-27 11:54:23,128][20381] Updated weights for policy 1, policy_version 902793 (0.0008) [2023-12-27 11:54:23,171][20347] Updated weights for policy 0, policy_version 903421 (0.0007) [2023-12-27 11:54:23,187][20381] Updated weights for policy 1, policy_version 902803 (0.0006) [2023-12-27 11:54:23,222][20347] Updated weights for policy 0, policy_version 903431 (0.0006) [2023-12-27 11:54:23,838][20381] Updated weights for policy 1, policy_version 902813 (0.0007) [2023-12-27 11:54:23,887][20381] Updated weights for policy 1, policy_version 902823 (0.0008) [2023-12-27 11:54:23,931][20381] Updated weights for policy 1, policy_version 902833 (0.0007) [2023-12-27 11:54:23,953][20347] Updated weights for policy 0, policy_version 903441 (0.0009) [2023-12-27 11:54:24,012][20347] Updated weights for policy 0, policy_version 903451 (0.0011) [2023-12-27 11:54:24,077][20347] Updated weights for policy 0, policy_version 903461 (0.0010) [2023-12-27 11:54:24,140][20347] Updated weights for policy 0, policy_version 903471 (0.0011) [2023-12-27 11:54:24,547][20381] Updated weights for policy 1, policy_version 902843 (0.0007) [2023-12-27 11:54:24,607][20381] Updated weights for policy 1, policy_version 902853 (0.0009) [2023-12-27 11:54:24,651][20381] Updated weights for policy 1, policy_version 902863 (0.0010) [2023-12-27 11:54:24,753][20347] Updated weights for policy 0, policy_version 903481 (0.0006) [2023-12-27 11:54:24,819][20347] Updated weights for policy 0, policy_version 903491 (0.0005) [2023-12-27 11:54:24,879][20347] Updated weights for policy 0, policy_version 903501 (0.0005) [2023-12-27 11:54:25,270][20381] Updated weights for policy 1, policy_version 902873 (0.0005) [2023-12-27 11:54:25,319][20381] Updated weights for policy 1, policy_version 902883 (0.0005) [2023-12-27 11:54:25,375][20381] Updated weights for policy 1, policy_version 902893 (0.0005) [2023-12-27 11:54:25,442][20381] Updated weights for policy 1, policy_version 902903 (0.0006) [2023-12-27 11:54:25,465][20347] Updated weights for policy 0, policy_version 903511 (0.0007) [2023-12-27 11:54:25,521][20347] Updated weights for policy 0, policy_version 903521 (0.0007) [2023-12-27 11:54:25,573][20347] Updated weights for policy 0, policy_version 903531 (0.0008) [2023-12-27 11:54:26,041][20381] Updated weights for policy 1, policy_version 902913 (0.0010) [2023-12-27 11:54:26,093][20381] Updated weights for policy 1, policy_version 902923 (0.0010) [2023-12-27 11:54:26,139][20381] Updated weights for policy 1, policy_version 902933 (0.0010) [2023-12-27 11:54:26,212][20347] Updated weights for policy 0, policy_version 903541 (0.0007) [2023-12-27 11:54:26,271][20347] Updated weights for policy 0, policy_version 903551 (0.0007) [2023-12-27 11:54:26,328][20347] Updated weights for policy 0, policy_version 903561 (0.0007) [2023-12-27 11:54:26,832][20381] Updated weights for policy 1, policy_version 902943 (0.0011) [2023-12-27 11:54:26,880][20381] Updated weights for policy 1, policy_version 902953 (0.0010) [2023-12-27 11:54:26,904][20347] Updated weights for policy 0, policy_version 903571 (0.0005) [2023-12-27 11:54:26,935][20381] Updated weights for policy 1, policy_version 902963 (0.0010) [2023-12-27 11:54:26,958][20347] Updated weights for policy 0, policy_version 903581 (0.0005) [2023-12-27 11:54:27,010][20347] Updated weights for policy 0, policy_version 903591 (0.0007) [2023-12-27 11:54:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 462536704. Throughput: 0: 10006.6, 1: 10476.8. Samples: 462523988. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:27,363][19170] Avg episode reward: [(0, '970.810'), (1, '39.720')] [2023-12-27 11:54:27,608][20347] Updated weights for policy 0, policy_version 903601 (0.0005) [2023-12-27 11:54:27,636][20381] Updated weights for policy 1, policy_version 902973 (0.0009) [2023-12-27 11:54:27,664][20347] Updated weights for policy 0, policy_version 903611 (0.0007) [2023-12-27 11:54:27,687][20381] Updated weights for policy 1, policy_version 902983 (0.0010) [2023-12-27 11:54:27,721][20347] Updated weights for policy 0, policy_version 903621 (0.0005) [2023-12-27 11:54:27,744][20381] Updated weights for policy 1, policy_version 902993 (0.0010) [2023-12-27 11:54:27,776][20347] Updated weights for policy 0, policy_version 903631 (0.0006) [2023-12-27 11:54:28,360][20347] Updated weights for policy 0, policy_version 903641 (0.0008) [2023-12-27 11:54:28,383][20381] Updated weights for policy 1, policy_version 903003 (0.0010) [2023-12-27 11:54:28,416][20347] Updated weights for policy 0, policy_version 903651 (0.0007) [2023-12-27 11:54:28,438][20381] Updated weights for policy 1, policy_version 903013 (0.0008) [2023-12-27 11:54:28,473][20347] Updated weights for policy 0, policy_version 903661 (0.0006) [2023-12-27 11:54:28,493][20381] Updated weights for policy 1, policy_version 903023 (0.0008) [2023-12-27 11:54:29,150][20347] Updated weights for policy 0, policy_version 903671 (0.0007) [2023-12-27 11:54:29,205][20347] Updated weights for policy 0, policy_version 903681 (0.0007) [2023-12-27 11:54:29,206][20381] Updated weights for policy 1, policy_version 903033 (0.0008) [2023-12-27 11:54:29,268][20347] Updated weights for policy 0, policy_version 903691 (0.0008) [2023-12-27 11:54:29,272][20381] Updated weights for policy 1, policy_version 903043 (0.0007) [2023-12-27 11:54:29,332][20381] Updated weights for policy 1, policy_version 903053 (0.0007) [2023-12-27 11:54:29,391][20381] Updated weights for policy 1, policy_version 903063 (0.0008) [2023-12-27 11:54:30,026][20347] Updated weights for policy 0, policy_version 903701 (0.0007) [2023-12-27 11:54:30,090][20347] Updated weights for policy 0, policy_version 903711 (0.0007) [2023-12-27 11:54:30,112][20381] Updated weights for policy 1, policy_version 903073 (0.0008) [2023-12-27 11:54:30,153][20347] Updated weights for policy 0, policy_version 903721 (0.0006) [2023-12-27 11:54:30,167][20381] Updated weights for policy 1, policy_version 903083 (0.0007) [2023-12-27 11:54:30,227][20381] Updated weights for policy 1, policy_version 903093 (0.0006) [2023-12-27 11:54:30,703][20347] Updated weights for policy 0, policy_version 903731 (0.0006) [2023-12-27 11:54:30,752][20347] Updated weights for policy 0, policy_version 903741 (0.0006) [2023-12-27 11:54:30,804][20347] Updated weights for policy 0, policy_version 903751 (0.0007) [2023-12-27 11:54:30,859][20381] Updated weights for policy 1, policy_version 903103 (0.0005) [2023-12-27 11:54:30,918][20381] Updated weights for policy 1, policy_version 903113 (0.0006) [2023-12-27 11:54:30,973][20381] Updated weights for policy 1, policy_version 903123 (0.0005) [2023-12-27 11:54:31,533][20347] Updated weights for policy 0, policy_version 903761 (0.0008) [2023-12-27 11:54:31,591][20347] Updated weights for policy 0, policy_version 903771 (0.0011) [2023-12-27 11:54:31,620][20381] Updated weights for policy 1, policy_version 903133 (0.0006) [2023-12-27 11:54:31,658][20347] Updated weights for policy 0, policy_version 903781 (0.0010) [2023-12-27 11:54:31,682][20381] Updated weights for policy 1, policy_version 903143 (0.0009) [2023-12-27 11:54:31,732][20347] Updated weights for policy 0, policy_version 903791 (0.0008) [2023-12-27 11:54:31,746][20381] Updated weights for policy 1, policy_version 903153 (0.0008) [2023-12-27 11:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 462635008. Throughput: 0: 10096.1, 1: 10409.8. Samples: 462648656. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:32,363][19170] Avg episode reward: [(0, '912.600'), (1, '37.020')] [2023-12-27 11:54:32,433][20347] Updated weights for policy 0, policy_version 903801 (0.0009) [2023-12-27 11:54:32,482][20347] Updated weights for policy 0, policy_version 903811 (0.0009) [2023-12-27 11:54:32,536][20381] Updated weights for policy 1, policy_version 903163 (0.0008) [2023-12-27 11:54:32,536][20347] Updated weights for policy 0, policy_version 903821 (0.0010) [2023-12-27 11:54:32,594][20381] Updated weights for policy 1, policy_version 903173 (0.0011) [2023-12-27 11:54:32,652][20381] Updated weights for policy 1, policy_version 903183 (0.0010) [2023-12-27 11:54:33,168][20347] Updated weights for policy 0, policy_version 903831 (0.0007) [2023-12-27 11:54:33,225][20347] Updated weights for policy 0, policy_version 903841 (0.0007) [2023-12-27 11:54:33,276][20347] Updated weights for policy 0, policy_version 903851 (0.0010) [2023-12-27 11:54:33,280][20381] Updated weights for policy 1, policy_version 903193 (0.0011) [2023-12-27 11:54:33,331][20381] Updated weights for policy 1, policy_version 903203 (0.0010) [2023-12-27 11:54:33,392][20381] Updated weights for policy 1, policy_version 903213 (0.0010) [2023-12-27 11:54:33,448][20381] Updated weights for policy 1, policy_version 903223 (0.0010) [2023-12-27 11:54:33,857][20347] Updated weights for policy 0, policy_version 903861 (0.0010) [2023-12-27 11:54:33,917][20347] Updated weights for policy 0, policy_version 903871 (0.0010) [2023-12-27 11:54:33,967][20347] Updated weights for policy 0, policy_version 903881 (0.0007) [2023-12-27 11:54:34,129][20381] Updated weights for policy 1, policy_version 903233 (0.0011) [2023-12-27 11:54:34,194][20381] Updated weights for policy 1, policy_version 903243 (0.0011) [2023-12-27 11:54:34,249][20381] Updated weights for policy 1, policy_version 903253 (0.0011) [2023-12-27 11:54:34,631][20347] Updated weights for policy 0, policy_version 903891 (0.0006) [2023-12-27 11:54:34,687][20347] Updated weights for policy 0, policy_version 903901 (0.0007) [2023-12-27 11:54:34,746][20347] Updated weights for policy 0, policy_version 903911 (0.0007) [2023-12-27 11:54:34,929][20381] Updated weights for policy 1, policy_version 903263 (0.0010) [2023-12-27 11:54:34,990][20381] Updated weights for policy 1, policy_version 903273 (0.0010) [2023-12-27 11:54:35,046][20381] Updated weights for policy 1, policy_version 903283 (0.0010) [2023-12-27 11:54:35,369][20347] Updated weights for policy 0, policy_version 903921 (0.0007) [2023-12-27 11:54:35,423][20347] Updated weights for policy 0, policy_version 903931 (0.0005) [2023-12-27 11:54:35,472][20347] Updated weights for policy 0, policy_version 903941 (0.0008) [2023-12-27 11:54:35,523][20347] Updated weights for policy 0, policy_version 903951 (0.0008) [2023-12-27 11:54:35,742][20381] Updated weights for policy 1, policy_version 903293 (0.0008) [2023-12-27 11:54:35,797][20381] Updated weights for policy 1, policy_version 903303 (0.0005) [2023-12-27 11:54:35,854][20381] Updated weights for policy 1, policy_version 903313 (0.0005) [2023-12-27 11:54:36,233][20347] Updated weights for policy 0, policy_version 903961 (0.0007) [2023-12-27 11:54:36,298][20347] Updated weights for policy 0, policy_version 903971 (0.0006) [2023-12-27 11:54:36,358][20347] Updated weights for policy 0, policy_version 903981 (0.0005) [2023-12-27 11:54:36,471][20381] Updated weights for policy 1, policy_version 903323 (0.0007) [2023-12-27 11:54:36,529][20381] Updated weights for policy 1, policy_version 903333 (0.0005) [2023-12-27 11:54:36,587][20381] Updated weights for policy 1, policy_version 903343 (0.0005) [2023-12-27 11:54:37,010][20347] Updated weights for policy 0, policy_version 903991 (0.0009) [2023-12-27 11:54:37,075][20347] Updated weights for policy 0, policy_version 904001 (0.0010) [2023-12-27 11:54:37,139][20347] Updated weights for policy 0, policy_version 904011 (0.0010) [2023-12-27 11:54:37,236][20381] Updated weights for policy 1, policy_version 903353 (0.0006) [2023-12-27 11:54:37,289][20381] Updated weights for policy 1, policy_version 903363 (0.0007) [2023-12-27 11:54:37,342][20381] Updated weights for policy 1, policy_version 903373 (0.0008) [2023-12-27 11:54:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 462741504. Throughput: 0: 10133.1, 1: 10394.3. Samples: 462710744. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:37,363][19170] Avg episode reward: [(0, '902.830'), (1, '37.670')] [2023-12-27 11:54:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000904016_231456768.pth... [2023-12-27 11:54:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000902800_231145472.pth [2023-12-27 11:54:37,404][20381] Updated weights for policy 1, policy_version 903383 (0.0007) [2023-12-27 11:54:37,410][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000903384_231292928.pth... [2023-12-27 11:54:37,414][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000902168_230981632.pth [2023-12-27 11:54:37,803][20347] Updated weights for policy 0, policy_version 904021 (0.0010) [2023-12-27 11:54:37,857][20347] Updated weights for policy 0, policy_version 904031 (0.0006) [2023-12-27 11:54:37,918][20347] Updated weights for policy 0, policy_version 904041 (0.0005) [2023-12-27 11:54:38,073][20381] Updated weights for policy 1, policy_version 903393 (0.0007) [2023-12-27 11:54:38,135][20381] Updated weights for policy 1, policy_version 903403 (0.0007) [2023-12-27 11:54:38,191][20381] Updated weights for policy 1, policy_version 903413 (0.0007) [2023-12-27 11:54:38,523][20347] Updated weights for policy 0, policy_version 904051 (0.0006) [2023-12-27 11:54:38,590][20347] Updated weights for policy 0, policy_version 904061 (0.0008) [2023-12-27 11:54:38,646][20347] Updated weights for policy 0, policy_version 904071 (0.0008) [2023-12-27 11:54:38,909][20381] Updated weights for policy 1, policy_version 903423 (0.0009) [2023-12-27 11:54:38,969][20381] Updated weights for policy 1, policy_version 903433 (0.0011) [2023-12-27 11:54:39,031][20381] Updated weights for policy 1, policy_version 903443 (0.0011) [2023-12-27 11:54:39,374][20347] Updated weights for policy 0, policy_version 904081 (0.0008) [2023-12-27 11:54:39,445][20347] Updated weights for policy 0, policy_version 904091 (0.0009) [2023-12-27 11:54:39,512][20347] Updated weights for policy 0, policy_version 904101 (0.0011) [2023-12-27 11:54:39,512][20291] KL-divergence is very high: 195.4544 [2023-12-27 11:54:39,575][20347] Updated weights for policy 0, policy_version 904111 (0.0010) [2023-12-27 11:54:39,742][20381] Updated weights for policy 1, policy_version 903453 (0.0008) [2023-12-27 11:54:39,797][20381] Updated weights for policy 1, policy_version 903463 (0.0006) [2023-12-27 11:54:39,867][20381] Updated weights for policy 1, policy_version 903473 (0.0009) [2023-12-27 11:54:40,294][20347] Updated weights for policy 0, policy_version 904121 (0.0007) [2023-12-27 11:54:40,356][20347] Updated weights for policy 0, policy_version 904131 (0.0008) [2023-12-27 11:54:40,415][20347] Updated weights for policy 0, policy_version 904141 (0.0008) [2023-12-27 11:54:40,520][20381] Updated weights for policy 1, policy_version 903483 (0.0011) [2023-12-27 11:54:40,574][20381] Updated weights for policy 1, policy_version 903493 (0.0011) [2023-12-27 11:54:40,632][20381] Updated weights for policy 1, policy_version 903503 (0.0008) [2023-12-27 11:54:41,080][20347] Updated weights for policy 0, policy_version 904151 (0.0009) [2023-12-27 11:54:41,142][20347] Updated weights for policy 0, policy_version 904161 (0.0008) [2023-12-27 11:54:41,202][20347] Updated weights for policy 0, policy_version 904171 (0.0008) [2023-12-27 11:54:41,338][20381] Updated weights for policy 1, policy_version 903513 (0.0006) [2023-12-27 11:54:41,405][20381] Updated weights for policy 1, policy_version 903523 (0.0014) [2023-12-27 11:54:41,459][20381] Updated weights for policy 1, policy_version 903533 (0.0009) [2023-12-27 11:54:41,523][20381] Updated weights for policy 1, policy_version 903543 (0.0006) [2023-12-27 11:54:41,973][20347] Updated weights for policy 0, policy_version 904181 (0.0008) [2023-12-27 11:54:42,032][20347] Updated weights for policy 0, policy_version 904191 (0.0008) [2023-12-27 11:54:42,094][20347] Updated weights for policy 0, policy_version 904201 (0.0008) [2023-12-27 11:54:42,200][20381] Updated weights for policy 1, policy_version 903553 (0.0008) [2023-12-27 11:54:42,255][20381] Updated weights for policy 1, policy_version 903563 (0.0008) [2023-12-27 11:54:42,319][20381] Updated weights for policy 1, policy_version 903573 (0.0007) [2023-12-27 11:54:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 462848000. Throughput: 0: 10216.6, 1: 10303.9. Samples: 462833744. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:42,363][19170] Avg episode reward: [(0, '932.010'), (1, '45.460')] [2023-12-27 11:54:42,790][20347] Updated weights for policy 0, policy_version 904211 (0.0007) [2023-12-27 11:54:42,856][20347] Updated weights for policy 0, policy_version 904221 (0.0008) [2023-12-27 11:54:42,916][20347] Updated weights for policy 0, policy_version 904231 (0.0008) [2023-12-27 11:54:43,006][20381] Updated weights for policy 1, policy_version 903583 (0.0008) [2023-12-27 11:54:43,055][20381] Updated weights for policy 1, policy_version 903593 (0.0008) [2023-12-27 11:54:43,099][20381] Updated weights for policy 1, policy_version 903603 (0.0007) [2023-12-27 11:54:43,548][20347] Updated weights for policy 0, policy_version 904241 (0.0007) [2023-12-27 11:54:43,614][20347] Updated weights for policy 0, policy_version 904251 (0.0005) [2023-12-27 11:54:43,673][20347] Updated weights for policy 0, policy_version 904261 (0.0005) [2023-12-27 11:54:43,739][20347] Updated weights for policy 0, policy_version 904271 (0.0005) [2023-12-27 11:54:43,831][20381] Updated weights for policy 1, policy_version 903613 (0.0008) [2023-12-27 11:54:43,884][20381] Updated weights for policy 1, policy_version 903623 (0.0008) [2023-12-27 11:54:43,938][20381] Updated weights for policy 1, policy_version 903633 (0.0008) [2023-12-27 11:54:44,314][20347] Updated weights for policy 0, policy_version 904281 (0.0007) [2023-12-27 11:54:44,360][20347] Updated weights for policy 0, policy_version 904291 (0.0009) [2023-12-27 11:54:44,424][20347] Updated weights for policy 0, policy_version 904301 (0.0008) [2023-12-27 11:54:44,607][20381] Updated weights for policy 1, policy_version 903643 (0.0007) [2023-12-27 11:54:44,670][20381] Updated weights for policy 1, policy_version 903653 (0.0007) [2023-12-27 11:54:44,731][20381] Updated weights for policy 1, policy_version 903663 (0.0009) [2023-12-27 11:54:45,144][20347] Updated weights for policy 0, policy_version 904311 (0.0007) [2023-12-27 11:54:45,210][20347] Updated weights for policy 0, policy_version 904321 (0.0007) [2023-12-27 11:54:45,275][20347] Updated weights for policy 0, policy_version 904331 (0.0008) [2023-12-27 11:54:45,416][20381] Updated weights for policy 1, policy_version 903673 (0.0008) [2023-12-27 11:54:45,476][20381] Updated weights for policy 1, policy_version 903683 (0.0007) [2023-12-27 11:54:45,524][20381] Updated weights for policy 1, policy_version 903693 (0.0007) [2023-12-27 11:54:45,580][20381] Updated weights for policy 1, policy_version 903703 (0.0005) [2023-12-27 11:54:45,917][20347] Updated weights for policy 0, policy_version 904341 (0.0007) [2023-12-27 11:54:45,977][20347] Updated weights for policy 0, policy_version 904351 (0.0005) [2023-12-27 11:54:46,041][20347] Updated weights for policy 0, policy_version 904361 (0.0005) [2023-12-27 11:54:46,235][20381] Updated weights for policy 1, policy_version 903713 (0.0007) [2023-12-27 11:54:46,294][20381] Updated weights for policy 1, policy_version 903723 (0.0005) [2023-12-27 11:54:46,357][20381] Updated weights for policy 1, policy_version 903733 (0.0010) [2023-12-27 11:54:46,643][20347] Updated weights for policy 0, policy_version 904371 (0.0006) [2023-12-27 11:54:46,693][20347] Updated weights for policy 0, policy_version 904381 (0.0006) [2023-12-27 11:54:46,742][20347] Updated weights for policy 0, policy_version 904391 (0.0005) [2023-12-27 11:54:46,975][20381] Updated weights for policy 1, policy_version 903743 (0.0008) [2023-12-27 11:54:47,028][20381] Updated weights for policy 1, policy_version 903753 (0.0007) [2023-12-27 11:54:47,118][20381] Updated weights for policy 1, policy_version 903763 (0.0006) [2023-12-27 11:54:47,355][20347] Updated weights for policy 0, policy_version 904401 (0.0005) [2023-12-27 11:54:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 462946304. Throughput: 0: 10266.3, 1: 10278.8. Samples: 462957124. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:47,364][19170] Avg episode reward: [(0, '902.500'), (1, '42.420')] [2023-12-27 11:54:47,415][20347] Updated weights for policy 0, policy_version 904411 (0.0005) [2023-12-27 11:54:47,466][20347] Updated weights for policy 0, policy_version 904421 (0.0005) [2023-12-27 11:54:47,518][20347] Updated weights for policy 0, policy_version 904431 (0.0005) [2023-12-27 11:54:47,718][20381] Updated weights for policy 1, policy_version 903773 (0.0007) [2023-12-27 11:54:47,771][20381] Updated weights for policy 1, policy_version 903783 (0.0008) [2023-12-27 11:54:47,825][20381] Updated weights for policy 1, policy_version 903793 (0.0008) [2023-12-27 11:54:48,160][20347] Updated weights for policy 0, policy_version 904441 (0.0007) [2023-12-27 11:54:48,214][20347] Updated weights for policy 0, policy_version 904451 (0.0007) [2023-12-27 11:54:48,260][20347] Updated weights for policy 0, policy_version 904461 (0.0007) [2023-12-27 11:54:48,548][20381] Updated weights for policy 1, policy_version 903803 (0.0007) [2023-12-27 11:54:48,605][20381] Updated weights for policy 1, policy_version 903813 (0.0007) [2023-12-27 11:54:48,662][20381] Updated weights for policy 1, policy_version 903823 (0.0008) [2023-12-27 11:54:48,986][20347] Updated weights for policy 0, policy_version 904471 (0.0006) [2023-12-27 11:54:49,045][20347] Updated weights for policy 0, policy_version 904481 (0.0005) [2023-12-27 11:54:49,095][20347] Updated weights for policy 0, policy_version 904491 (0.0006) [2023-12-27 11:54:49,377][20381] Updated weights for policy 1, policy_version 903833 (0.0007) [2023-12-27 11:54:49,434][20381] Updated weights for policy 1, policy_version 903843 (0.0008) [2023-12-27 11:54:49,494][20381] Updated weights for policy 1, policy_version 903853 (0.0007) [2023-12-27 11:54:49,554][20381] Updated weights for policy 1, policy_version 903863 (0.0008) [2023-12-27 11:54:49,757][20347] Updated weights for policy 0, policy_version 904501 (0.0006) [2023-12-27 11:54:49,816][20347] Updated weights for policy 0, policy_version 904511 (0.0006) [2023-12-27 11:54:49,886][20347] Updated weights for policy 0, policy_version 904521 (0.0008) [2023-12-27 11:54:50,232][20381] Updated weights for policy 1, policy_version 903873 (0.0008) [2023-12-27 11:54:50,289][20381] Updated weights for policy 1, policy_version 903883 (0.0008) [2023-12-27 11:54:50,348][20381] Updated weights for policy 1, policy_version 903893 (0.0007) [2023-12-27 11:54:50,616][20347] Updated weights for policy 0, policy_version 904531 (0.0008) [2023-12-27 11:54:50,674][20347] Updated weights for policy 0, policy_version 904541 (0.0006) [2023-12-27 11:54:50,736][20347] Updated weights for policy 0, policy_version 904551 (0.0008) [2023-12-27 11:54:51,044][20381] Updated weights for policy 1, policy_version 903903 (0.0008) [2023-12-27 11:54:51,111][20381] Updated weights for policy 1, policy_version 903913 (0.0008) [2023-12-27 11:54:51,167][20381] Updated weights for policy 1, policy_version 903923 (0.0008) [2023-12-27 11:54:51,461][20347] Updated weights for policy 0, policy_version 904561 (0.0008) [2023-12-27 11:54:51,519][20347] Updated weights for policy 0, policy_version 904571 (0.0007) [2023-12-27 11:54:51,572][20347] Updated weights for policy 0, policy_version 904581 (0.0008) [2023-12-27 11:54:51,631][20347] Updated weights for policy 0, policy_version 904591 (0.0008) [2023-12-27 11:54:51,952][20381] Updated weights for policy 1, policy_version 903933 (0.0008) [2023-12-27 11:54:52,011][20381] Updated weights for policy 1, policy_version 903943 (0.0008) [2023-12-27 11:54:52,065][20381] Updated weights for policy 1, policy_version 903953 (0.0008) [2023-12-27 11:54:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20438.4). Total num frames: 463044608. Throughput: 0: 10272.2, 1: 10243.8. Samples: 463018308. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:52,363][19170] Avg episode reward: [(0, '863.480'), (1, '38.350')] [2023-12-27 11:54:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000903960_231440384.pth... [2023-12-27 11:54:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000902776_231137280.pth [2023-12-27 11:54:52,375][20347] Updated weights for policy 0, policy_version 904601 (0.0008) [2023-12-27 11:54:52,426][20347] Updated weights for policy 0, policy_version 904611 (0.0007) [2023-12-27 11:54:52,477][20347] Updated weights for policy 0, policy_version 904621 (0.0007) [2023-12-27 11:54:52,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000904624_231612416.pth... [2023-12-27 11:54:52,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000903408_231301120.pth [2023-12-27 11:54:52,764][20381] Updated weights for policy 1, policy_version 903963 (0.0008) [2023-12-27 11:54:52,823][20381] Updated weights for policy 1, policy_version 903973 (0.0008) [2023-12-27 11:54:52,884][20381] Updated weights for policy 1, policy_version 903983 (0.0008) [2023-12-27 11:54:53,185][20347] Updated weights for policy 0, policy_version 904631 (0.0007) [2023-12-27 11:54:53,242][20347] Updated weights for policy 0, policy_version 904641 (0.0006) [2023-12-27 11:54:53,306][20347] Updated weights for policy 0, policy_version 904651 (0.0005) [2023-12-27 11:54:53,551][20381] Updated weights for policy 1, policy_version 903993 (0.0008) [2023-12-27 11:54:53,620][20381] Updated weights for policy 1, policy_version 904003 (0.0005) [2023-12-27 11:54:53,685][20381] Updated weights for policy 1, policy_version 904013 (0.0009) [2023-12-27 11:54:53,746][20381] Updated weights for policy 1, policy_version 904023 (0.0010) [2023-12-27 11:54:53,846][20347] Updated weights for policy 0, policy_version 904661 (0.0005) [2023-12-27 11:54:53,894][20347] Updated weights for policy 0, policy_version 904671 (0.0006) [2023-12-27 11:54:53,942][20347] Updated weights for policy 0, policy_version 904681 (0.0005) [2023-12-27 11:54:54,272][20381] Updated weights for policy 1, policy_version 904033 (0.0009) [2023-12-27 11:54:54,325][20381] Updated weights for policy 1, policy_version 904043 (0.0008) [2023-12-27 11:54:54,379][20381] Updated weights for policy 1, policy_version 904053 (0.0008) [2023-12-27 11:54:54,638][20347] Updated weights for policy 0, policy_version 904691 (0.0008) [2023-12-27 11:54:54,686][20347] Updated weights for policy 0, policy_version 904701 (0.0007) [2023-12-27 11:54:54,727][20347] Updated weights for policy 0, policy_version 904711 (0.0007) [2023-12-27 11:54:55,050][20381] Updated weights for policy 1, policy_version 904063 (0.0008) [2023-12-27 11:54:55,119][20381] Updated weights for policy 1, policy_version 904073 (0.0008) [2023-12-27 11:54:55,185][20381] Updated weights for policy 1, policy_version 904083 (0.0008) [2023-12-27 11:54:55,482][20347] Updated weights for policy 0, policy_version 904721 (0.0007) [2023-12-27 11:54:55,532][20347] Updated weights for policy 0, policy_version 904731 (0.0007) [2023-12-27 11:54:55,585][20347] Updated weights for policy 0, policy_version 904741 (0.0008) [2023-12-27 11:54:55,636][20347] Updated weights for policy 0, policy_version 904751 (0.0007) [2023-12-27 11:54:55,787][20381] Updated weights for policy 1, policy_version 904093 (0.0008) [2023-12-27 11:54:55,851][20381] Updated weights for policy 1, policy_version 904103 (0.0008) [2023-12-27 11:54:55,910][20381] Updated weights for policy 1, policy_version 904113 (0.0005) [2023-12-27 11:54:56,349][20347] Updated weights for policy 0, policy_version 904761 (0.0010) [2023-12-27 11:54:56,406][20347] Updated weights for policy 0, policy_version 904771 (0.0010) [2023-12-27 11:54:56,460][20347] Updated weights for policy 0, policy_version 904781 (0.0010) [2023-12-27 11:54:56,467][20381] Updated weights for policy 1, policy_version 904123 (0.0006) [2023-12-27 11:54:56,522][20381] Updated weights for policy 1, policy_version 904133 (0.0007) [2023-12-27 11:54:56,584][20381] Updated weights for policy 1, policy_version 904143 (0.0008) [2023-12-27 11:54:57,053][20347] Updated weights for policy 0, policy_version 904791 (0.0007) [2023-12-27 11:54:57,100][20347] Updated weights for policy 0, policy_version 904801 (0.0005) [2023-12-27 11:54:57,150][20347] Updated weights for policy 0, policy_version 904811 (0.0005) [2023-12-27 11:54:57,164][20381] Updated weights for policy 1, policy_version 904153 (0.0005) [2023-12-27 11:54:57,225][20381] Updated weights for policy 1, policy_version 904163 (0.0006) [2023-12-27 11:54:57,281][20381] Updated weights for policy 1, policy_version 904173 (0.0006) [2023-12-27 11:54:57,336][20381] Updated weights for policy 1, policy_version 904183 (0.0005) [2023-12-27 11:54:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 463159296. Throughput: 0: 10266.3, 1: 10273.3. Samples: 463141056. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:54:57,363][19170] Avg episode reward: [(0, '902.930'), (1, '47.300')] [2023-12-27 11:54:57,696][20347] Updated weights for policy 0, policy_version 904821 (0.0005) [2023-12-27 11:54:57,747][20347] Updated weights for policy 0, policy_version 904831 (0.0005) [2023-12-27 11:54:57,809][20347] Updated weights for policy 0, policy_version 904841 (0.0006) [2023-12-27 11:54:57,961][20381] Updated weights for policy 1, policy_version 904193 (0.0009) [2023-12-27 11:54:58,007][20381] Updated weights for policy 1, policy_version 904203 (0.0008) [2023-12-27 11:54:58,052][20381] Updated weights for policy 1, policy_version 904213 (0.0008) [2023-12-27 11:54:58,383][20347] Updated weights for policy 0, policy_version 904851 (0.0006) [2023-12-27 11:54:58,452][20347] Updated weights for policy 0, policy_version 904861 (0.0009) [2023-12-27 11:54:58,515][20347] Updated weights for policy 0, policy_version 904871 (0.0009) [2023-12-27 11:54:58,808][20381] Updated weights for policy 1, policy_version 904223 (0.0009) [2023-12-27 11:54:58,880][20381] Updated weights for policy 1, policy_version 904233 (0.0008) [2023-12-27 11:54:58,945][20381] Updated weights for policy 1, policy_version 904243 (0.0009) [2023-12-27 11:54:59,350][20347] Updated weights for policy 0, policy_version 904881 (0.0008) [2023-12-27 11:54:59,431][20347] Updated weights for policy 0, policy_version 904891 (0.0008) [2023-12-27 11:54:59,483][20347] Updated weights for policy 0, policy_version 904901 (0.0007) [2023-12-27 11:54:59,539][20347] Updated weights for policy 0, policy_version 904911 (0.0008) [2023-12-27 11:54:59,590][20381] Updated weights for policy 1, policy_version 904253 (0.0007) [2023-12-27 11:54:59,640][20381] Updated weights for policy 1, policy_version 904263 (0.0007) [2023-12-27 11:54:59,687][20381] Updated weights for policy 1, policy_version 904273 (0.0006) [2023-12-27 11:55:00,205][20347] Updated weights for policy 0, policy_version 904921 (0.0008) [2023-12-27 11:55:00,258][20347] Updated weights for policy 0, policy_version 904931 (0.0008) [2023-12-27 11:55:00,316][20347] Updated weights for policy 0, policy_version 904941 (0.0007) [2023-12-27 11:55:00,415][20381] Updated weights for policy 1, policy_version 904283 (0.0007) [2023-12-27 11:55:00,471][20381] Updated weights for policy 1, policy_version 904293 (0.0006) [2023-12-27 11:55:00,520][20381] Updated weights for policy 1, policy_version 904303 (0.0007) [2023-12-27 11:55:00,995][20347] Updated weights for policy 0, policy_version 904951 (0.0008) [2023-12-27 11:55:01,053][20347] Updated weights for policy 0, policy_version 904961 (0.0007) [2023-12-27 11:55:01,110][20347] Updated weights for policy 0, policy_version 904971 (0.0006) [2023-12-27 11:55:01,220][20381] Updated weights for policy 1, policy_version 904313 (0.0007) [2023-12-27 11:55:01,283][20381] Updated weights for policy 1, policy_version 904323 (0.0008) [2023-12-27 11:55:01,338][20381] Updated weights for policy 1, policy_version 904333 (0.0007) [2023-12-27 11:55:01,402][20381] Updated weights for policy 1, policy_version 904343 (0.0008) [2023-12-27 11:55:01,840][20347] Updated weights for policy 0, policy_version 904981 (0.0009) [2023-12-27 11:55:01,896][20347] Updated weights for policy 0, policy_version 904991 (0.0010) [2023-12-27 11:55:01,961][20347] Updated weights for policy 0, policy_version 905001 (0.0011) [2023-12-27 11:55:02,103][20381] Updated weights for policy 1, policy_version 904353 (0.0008) [2023-12-27 11:55:02,145][20381] Updated weights for policy 1, policy_version 904363 (0.0007) [2023-12-27 11:55:02,194][20381] Updated weights for policy 1, policy_version 904373 (0.0008) [2023-12-27 11:55:02,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 463257600. Throughput: 0: 10285.5, 1: 10241.8. Samples: 463264220. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:02,363][19170] Avg episode reward: [(0, '922.570'), (1, '64.620')] [2023-12-27 11:55:02,671][20347] Updated weights for policy 0, policy_version 905011 (0.0011) [2023-12-27 11:55:02,729][20347] Updated weights for policy 0, policy_version 905021 (0.0008) [2023-12-27 11:55:02,783][20347] Updated weights for policy 0, policy_version 905031 (0.0005) [2023-12-27 11:55:02,917][20381] Updated weights for policy 1, policy_version 904383 (0.0010) [2023-12-27 11:55:02,966][20381] Updated weights for policy 1, policy_version 904393 (0.0010) [2023-12-27 11:55:03,019][20381] Updated weights for policy 1, policy_version 904403 (0.0009) [2023-12-27 11:55:03,398][20347] Updated weights for policy 0, policy_version 905041 (0.0006) [2023-12-27 11:55:03,458][20347] Updated weights for policy 0, policy_version 905051 (0.0010) [2023-12-27 11:55:03,517][20347] Updated weights for policy 0, policy_version 905061 (0.0010) [2023-12-27 11:55:03,577][20347] Updated weights for policy 0, policy_version 905071 (0.0010) [2023-12-27 11:55:03,619][20381] Updated weights for policy 1, policy_version 904413 (0.0007) [2023-12-27 11:55:03,673][20381] Updated weights for policy 1, policy_version 904423 (0.0005) [2023-12-27 11:55:03,730][20381] Updated weights for policy 1, policy_version 904433 (0.0009) [2023-12-27 11:55:04,236][20347] Updated weights for policy 0, policy_version 905081 (0.0010) [2023-12-27 11:55:04,305][20347] Updated weights for policy 0, policy_version 905091 (0.0010) [2023-12-27 11:55:04,350][20381] Updated weights for policy 1, policy_version 904443 (0.0010) [2023-12-27 11:55:04,365][20347] Updated weights for policy 0, policy_version 905101 (0.0011) [2023-12-27 11:55:04,404][20381] Updated weights for policy 1, policy_version 904453 (0.0007) [2023-12-27 11:55:04,459][20381] Updated weights for policy 1, policy_version 904463 (0.0008) [2023-12-27 11:55:04,967][20347] Updated weights for policy 0, policy_version 905111 (0.0007) [2023-12-27 11:55:05,014][20347] Updated weights for policy 0, policy_version 905121 (0.0006) [2023-12-27 11:55:05,061][20347] Updated weights for policy 0, policy_version 905131 (0.0008) [2023-12-27 11:55:05,165][20381] Updated weights for policy 1, policy_version 904473 (0.0008) [2023-12-27 11:55:05,220][20381] Updated weights for policy 1, policy_version 904483 (0.0007) [2023-12-27 11:55:05,272][20381] Updated weights for policy 1, policy_version 904493 (0.0008) [2023-12-27 11:55:05,324][20381] Updated weights for policy 1, policy_version 904503 (0.0008) [2023-12-27 11:55:05,610][20347] Updated weights for policy 0, policy_version 905141 (0.0006) [2023-12-27 11:55:05,657][20347] Updated weights for policy 0, policy_version 905151 (0.0007) [2023-12-27 11:55:05,707][20347] Updated weights for policy 0, policy_version 905161 (0.0007) [2023-12-27 11:55:06,067][20381] Updated weights for policy 1, policy_version 904513 (0.0007) [2023-12-27 11:55:06,125][20381] Updated weights for policy 1, policy_version 904523 (0.0008) [2023-12-27 11:55:06,183][20381] Updated weights for policy 1, policy_version 904533 (0.0008) [2023-12-27 11:55:06,458][20347] Updated weights for policy 0, policy_version 905171 (0.0008) [2023-12-27 11:55:06,513][20347] Updated weights for policy 0, policy_version 905181 (0.0008) [2023-12-27 11:55:06,541][20291] KL-divergence is very high: 123.4682 [2023-12-27 11:55:06,569][20347] Updated weights for policy 0, policy_version 905191 (0.0008) [2023-12-27 11:55:06,587][20291] KL-divergence is very high: 172.3609 [2023-12-27 11:55:06,908][20381] Updated weights for policy 1, policy_version 904543 (0.0008) [2023-12-27 11:55:06,964][20381] Updated weights for policy 1, policy_version 904553 (0.0005) [2023-12-27 11:55:07,025][20381] Updated weights for policy 1, policy_version 904563 (0.0006) [2023-12-27 11:55:07,215][20347] Updated weights for policy 0, policy_version 905201 (0.0008) [2023-12-27 11:55:07,282][20347] Updated weights for policy 0, policy_version 905211 (0.0006) [2023-12-27 11:55:07,351][20347] Updated weights for policy 0, policy_version 905221 (0.0006) [2023-12-27 11:55:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 463355904. Throughput: 0: 10351.0, 1: 10265.7. Samples: 463328400. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:07,363][19170] Avg episode reward: [(0, '970.840'), (1, '81.980')] [2023-12-27 11:55:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000904568_231596032.pth... [2023-12-27 11:55:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000903384_231292928.pth [2023-12-27 11:55:07,414][20347] Updated weights for policy 0, policy_version 905231 (0.0010) [2023-12-27 11:55:07,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000905232_231768064.pth... [2023-12-27 11:55:07,424][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000904016_231456768.pth [2023-12-27 11:55:07,643][20381] Updated weights for policy 1, policy_version 904573 (0.0008) [2023-12-27 11:55:07,710][20381] Updated weights for policy 1, policy_version 904583 (0.0007) [2023-12-27 11:55:07,757][20381] Updated weights for policy 1, policy_version 904593 (0.0007) [2023-12-27 11:55:08,028][20347] Updated weights for policy 0, policy_version 905241 (0.0006) [2023-12-27 11:55:08,079][20347] Updated weights for policy 0, policy_version 905251 (0.0005) [2023-12-27 11:55:08,136][20347] Updated weights for policy 0, policy_version 905261 (0.0006) [2023-12-27 11:55:08,433][20381] Updated weights for policy 1, policy_version 904603 (0.0007) [2023-12-27 11:55:08,483][20381] Updated weights for policy 1, policy_version 904613 (0.0006) [2023-12-27 11:55:08,538][20381] Updated weights for policy 1, policy_version 904623 (0.0005) [2023-12-27 11:55:08,747][20347] Updated weights for policy 0, policy_version 905271 (0.0007) [2023-12-27 11:55:08,800][20347] Updated weights for policy 0, policy_version 905281 (0.0007) [2023-12-27 11:55:08,852][20347] Updated weights for policy 0, policy_version 905291 (0.0007) [2023-12-27 11:55:09,104][20381] Updated weights for policy 1, policy_version 904633 (0.0005) [2023-12-27 11:55:09,163][20381] Updated weights for policy 1, policy_version 904643 (0.0005) [2023-12-27 11:55:09,230][20381] Updated weights for policy 1, policy_version 904653 (0.0007) [2023-12-27 11:55:09,291][20381] Updated weights for policy 1, policy_version 904663 (0.0007) [2023-12-27 11:55:09,645][20347] Updated weights for policy 0, policy_version 905301 (0.0008) [2023-12-27 11:55:09,704][20347] Updated weights for policy 0, policy_version 905311 (0.0008) [2023-12-27 11:55:09,763][20291] KL-divergence is very high: 127.6127 [2023-12-27 11:55:09,764][20347] Updated weights for policy 0, policy_version 905321 (0.0008) [2023-12-27 11:55:09,979][20381] Updated weights for policy 1, policy_version 904673 (0.0006) [2023-12-27 11:55:10,028][20381] Updated weights for policy 1, policy_version 904683 (0.0007) [2023-12-27 11:55:10,080][20381] Updated weights for policy 1, policy_version 904693 (0.0006) [2023-12-27 11:55:10,495][20347] Updated weights for policy 0, policy_version 905331 (0.0007) [2023-12-27 11:55:10,556][20347] Updated weights for policy 0, policy_version 905341 (0.0007) [2023-12-27 11:55:10,613][20347] Updated weights for policy 0, policy_version 905351 (0.0007) [2023-12-27 11:55:10,797][20381] Updated weights for policy 1, policy_version 904703 (0.0008) [2023-12-27 11:55:10,862][20381] Updated weights for policy 1, policy_version 904713 (0.0005) [2023-12-27 11:55:10,921][20381] Updated weights for policy 1, policy_version 904723 (0.0006) [2023-12-27 11:55:11,287][20347] Updated weights for policy 0, policy_version 905361 (0.0007) [2023-12-27 11:55:11,353][20347] Updated weights for policy 0, policy_version 905371 (0.0008) [2023-12-27 11:55:11,429][20347] Updated weights for policy 0, policy_version 905381 (0.0009) [2023-12-27 11:55:11,495][20347] Updated weights for policy 0, policy_version 905391 (0.0008) [2023-12-27 11:55:11,579][20381] Updated weights for policy 1, policy_version 904733 (0.0007) [2023-12-27 11:55:11,645][20381] Updated weights for policy 1, policy_version 904743 (0.0007) [2023-12-27 11:55:11,701][20381] Updated weights for policy 1, policy_version 904753 (0.0008) [2023-12-27 11:55:12,137][20347] Updated weights for policy 0, policy_version 905401 (0.0008) [2023-12-27 11:55:12,198][20347] Updated weights for policy 0, policy_version 905411 (0.0007) [2023-12-27 11:55:12,257][20347] Updated weights for policy 0, policy_version 905421 (0.0007) [2023-12-27 11:55:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 463462400. Throughput: 0: 10371.0, 1: 10261.3. Samples: 463452444. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:12,363][19170] Avg episode reward: [(0, '893.240'), (1, '231.870')] [2023-12-27 11:55:12,481][20381] Updated weights for policy 1, policy_version 904763 (0.0009) [2023-12-27 11:55:12,540][20381] Updated weights for policy 1, policy_version 904773 (0.0007) [2023-12-27 11:55:12,598][20381] Updated weights for policy 1, policy_version 904783 (0.0008) [2023-12-27 11:55:12,989][20347] Updated weights for policy 0, policy_version 905431 (0.0008) [2023-12-27 11:55:13,048][20347] Updated weights for policy 0, policy_version 905441 (0.0008) [2023-12-27 11:55:13,053][20291] KL-divergence is very high: 102.4411 [2023-12-27 11:55:13,105][20347] Updated weights for policy 0, policy_version 905451 (0.0008) [2023-12-27 11:55:13,240][20381] Updated weights for policy 1, policy_version 904793 (0.0008) [2023-12-27 11:55:13,298][20381] Updated weights for policy 1, policy_version 904803 (0.0007) [2023-12-27 11:55:13,357][20381] Updated weights for policy 1, policy_version 904813 (0.0007) [2023-12-27 11:55:13,406][20381] Updated weights for policy 1, policy_version 904823 (0.0008) [2023-12-27 11:55:13,822][20347] Updated weights for policy 0, policy_version 905461 (0.0007) [2023-12-27 11:55:13,872][20347] Updated weights for policy 0, policy_version 905471 (0.0005) [2023-12-27 11:55:13,923][20347] Updated weights for policy 0, policy_version 905481 (0.0005) [2023-12-27 11:55:14,113][20381] Updated weights for policy 1, policy_version 904833 (0.0008) [2023-12-27 11:55:14,164][20381] Updated weights for policy 1, policy_version 904843 (0.0008) [2023-12-27 11:55:14,218][20381] Updated weights for policy 1, policy_version 904853 (0.0008) [2023-12-27 11:55:14,520][20347] Updated weights for policy 0, policy_version 905491 (0.0006) [2023-12-27 11:55:14,574][20347] Updated weights for policy 0, policy_version 905501 (0.0008) [2023-12-27 11:55:14,625][20347] Updated weights for policy 0, policy_version 905511 (0.0007) [2023-12-27 11:55:14,917][20381] Updated weights for policy 1, policy_version 904863 (0.0008) [2023-12-27 11:55:14,983][20381] Updated weights for policy 1, policy_version 904873 (0.0008) [2023-12-27 11:55:15,043][20381] Updated weights for policy 1, policy_version 904883 (0.0008) [2023-12-27 11:55:15,355][20347] Updated weights for policy 0, policy_version 905521 (0.0008) [2023-12-27 11:55:15,423][20347] Updated weights for policy 0, policy_version 905531 (0.0008) [2023-12-27 11:55:15,490][20347] Updated weights for policy 0, policy_version 905541 (0.0008) [2023-12-27 11:55:15,554][20347] Updated weights for policy 0, policy_version 905551 (0.0008) [2023-12-27 11:55:15,724][20381] Updated weights for policy 1, policy_version 904893 (0.0007) [2023-12-27 11:55:15,795][20381] Updated weights for policy 1, policy_version 904903 (0.0008) [2023-12-27 11:55:15,858][20381] Updated weights for policy 1, policy_version 904913 (0.0007) [2023-12-27 11:55:16,183][20347] Updated weights for policy 0, policy_version 905561 (0.0008) [2023-12-27 11:55:16,246][20347] Updated weights for policy 0, policy_version 905571 (0.0008) [2023-12-27 11:55:16,305][20347] Updated weights for policy 0, policy_version 905581 (0.0007) [2023-12-27 11:55:16,543][20381] Updated weights for policy 1, policy_version 904923 (0.0007) [2023-12-27 11:55:16,600][20381] Updated weights for policy 1, policy_version 904933 (0.0008) [2023-12-27 11:55:16,661][20381] Updated weights for policy 1, policy_version 904943 (0.0008) [2023-12-27 11:55:16,979][20347] Updated weights for policy 0, policy_version 905591 (0.0008) [2023-12-27 11:55:17,024][20347] Updated weights for policy 0, policy_version 905601 (0.0005) [2023-12-27 11:55:17,073][20347] Updated weights for policy 0, policy_version 905611 (0.0007) [2023-12-27 11:55:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 463560704. Throughput: 0: 10303.8, 1: 10222.9. Samples: 463572360. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:17,363][19170] Avg episode reward: [(0, '805.700'), (1, '303.630')] [2023-12-27 11:55:17,387][20381] Updated weights for policy 1, policy_version 904953 (0.0007) [2023-12-27 11:55:17,438][20381] Updated weights for policy 1, policy_version 904963 (0.0009) [2023-12-27 11:55:17,489][20381] Updated weights for policy 1, policy_version 904973 (0.0010) [2023-12-27 11:55:17,546][20381] Updated weights for policy 1, policy_version 904983 (0.0008) [2023-12-27 11:55:17,719][20347] Updated weights for policy 0, policy_version 905621 (0.0008) [2023-12-27 11:55:17,779][20347] Updated weights for policy 0, policy_version 905631 (0.0007) [2023-12-27 11:55:17,832][20347] Updated weights for policy 0, policy_version 905641 (0.0007) [2023-12-27 11:55:18,244][20381] Updated weights for policy 1, policy_version 904993 (0.0008) [2023-12-27 11:55:18,313][20381] Updated weights for policy 1, policy_version 905003 (0.0008) [2023-12-27 11:55:18,384][20381] Updated weights for policy 1, policy_version 905013 (0.0008) [2023-12-27 11:55:18,493][20347] Updated weights for policy 0, policy_version 905651 (0.0008) [2023-12-27 11:55:18,554][20347] Updated weights for policy 0, policy_version 905661 (0.0010) [2023-12-27 11:55:18,617][20347] Updated weights for policy 0, policy_version 905671 (0.0006) [2023-12-27 11:55:19,083][20381] Updated weights for policy 1, policy_version 905023 (0.0008) [2023-12-27 11:55:19,136][20381] Updated weights for policy 1, policy_version 905033 (0.0009) [2023-12-27 11:55:19,184][20381] Updated weights for policy 1, policy_version 905043 (0.0008) [2023-12-27 11:55:19,205][20347] Updated weights for policy 0, policy_version 905681 (0.0008) [2023-12-27 11:55:19,276][20347] Updated weights for policy 0, policy_version 905691 (0.0008) [2023-12-27 11:55:19,338][20347] Updated weights for policy 0, policy_version 905701 (0.0006) [2023-12-27 11:55:19,392][20347] Updated weights for policy 0, policy_version 905711 (0.0010) [2023-12-27 11:55:20,006][20381] Updated weights for policy 1, policy_version 905053 (0.0008) [2023-12-27 11:55:20,064][20381] Updated weights for policy 1, policy_version 905063 (0.0007) [2023-12-27 11:55:20,095][20347] Updated weights for policy 0, policy_version 905721 (0.0008) [2023-12-27 11:55:20,126][20381] Updated weights for policy 1, policy_version 905073 (0.0009) [2023-12-27 11:55:20,157][20347] Updated weights for policy 0, policy_version 905731 (0.0008) [2023-12-27 11:55:20,219][20347] Updated weights for policy 0, policy_version 905741 (0.0007) [2023-12-27 11:55:20,851][20381] Updated weights for policy 1, policy_version 905083 (0.0008) [2023-12-27 11:55:20,892][20347] Updated weights for policy 0, policy_version 905751 (0.0006) [2023-12-27 11:55:20,913][20381] Updated weights for policy 1, policy_version 905093 (0.0008) [2023-12-27 11:55:20,950][20347] Updated weights for policy 0, policy_version 905761 (0.0009) [2023-12-27 11:55:20,971][20381] Updated weights for policy 1, policy_version 905103 (0.0007) [2023-12-27 11:55:21,008][20347] Updated weights for policy 0, policy_version 905771 (0.0009) [2023-12-27 11:55:21,713][20381] Updated weights for policy 1, policy_version 905113 (0.0007) [2023-12-27 11:55:21,740][20347] Updated weights for policy 0, policy_version 905781 (0.0008) [2023-12-27 11:55:21,776][20381] Updated weights for policy 1, policy_version 905123 (0.0008) [2023-12-27 11:55:21,803][20347] Updated weights for policy 0, policy_version 905791 (0.0007) [2023-12-27 11:55:21,832][20381] Updated weights for policy 1, policy_version 905133 (0.0008) [2023-12-27 11:55:21,865][20347] Updated weights for policy 0, policy_version 905801 (0.0007) [2023-12-27 11:55:21,893][20381] Updated weights for policy 1, policy_version 905143 (0.0007) [2023-12-27 11:55:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 463659008. Throughput: 0: 10296.3, 1: 10182.2. Samples: 463632280. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:22,363][19170] Avg episode reward: [(0, '824.570'), (1, '264.080')] [2023-12-27 11:55:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000905144_231743488.pth... [2023-12-27 11:55:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000905808_231915520.pth... [2023-12-27 11:55:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000904624_231612416.pth [2023-12-27 11:55:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000903960_231440384.pth [2023-12-27 11:55:22,510][20347] Updated weights for policy 0, policy_version 905811 (0.0007) [2023-12-27 11:55:22,573][20347] Updated weights for policy 0, policy_version 905821 (0.0007) [2023-12-27 11:55:22,637][20347] Updated weights for policy 0, policy_version 905831 (0.0007) [2023-12-27 11:55:22,663][20381] Updated weights for policy 1, policy_version 905153 (0.0006) [2023-12-27 11:55:22,722][20381] Updated weights for policy 1, policy_version 905163 (0.0007) [2023-12-27 11:55:22,788][20381] Updated weights for policy 1, policy_version 905173 (0.0007) [2023-12-27 11:55:23,249][20347] Updated weights for policy 0, policy_version 905841 (0.0006) [2023-12-27 11:55:23,300][20347] Updated weights for policy 0, policy_version 905851 (0.0010) [2023-12-27 11:55:23,348][20347] Updated weights for policy 0, policy_version 905861 (0.0010) [2023-12-27 11:55:23,406][20347] Updated weights for policy 0, policy_version 905871 (0.0010) [2023-12-27 11:55:23,491][20381] Updated weights for policy 1, policy_version 905183 (0.0007) [2023-12-27 11:55:23,552][20381] Updated weights for policy 1, policy_version 905193 (0.0009) [2023-12-27 11:55:23,600][20381] Updated weights for policy 1, policy_version 905203 (0.0007) [2023-12-27 11:55:24,023][20347] Updated weights for policy 0, policy_version 905881 (0.0010) [2023-12-27 11:55:24,083][20347] Updated weights for policy 0, policy_version 905891 (0.0010) [2023-12-27 11:55:24,141][20347] Updated weights for policy 0, policy_version 905901 (0.0010) [2023-12-27 11:55:24,334][20381] Updated weights for policy 1, policy_version 905213 (0.0007) [2023-12-27 11:55:24,397][20381] Updated weights for policy 1, policy_version 905223 (0.0006) [2023-12-27 11:55:24,451][20381] Updated weights for policy 1, policy_version 905233 (0.0007) [2023-12-27 11:55:24,776][20347] Updated weights for policy 0, policy_version 905911 (0.0010) [2023-12-27 11:55:24,828][20347] Updated weights for policy 0, policy_version 905921 (0.0010) [2023-12-27 11:55:24,882][20347] Updated weights for policy 0, policy_version 905931 (0.0010) [2023-12-27 11:55:25,152][20381] Updated weights for policy 1, policy_version 905243 (0.0008) [2023-12-27 11:55:25,202][20381] Updated weights for policy 1, policy_version 905253 (0.0009) [2023-12-27 11:55:25,255][20381] Updated weights for policy 1, policy_version 905263 (0.0008) [2023-12-27 11:55:25,495][20347] Updated weights for policy 0, policy_version 905941 (0.0008) [2023-12-27 11:55:25,557][20347] Updated weights for policy 0, policy_version 905951 (0.0006) [2023-12-27 11:55:25,616][20347] Updated weights for policy 0, policy_version 905961 (0.0005) [2023-12-27 11:55:25,978][20381] Updated weights for policy 1, policy_version 905273 (0.0008) [2023-12-27 11:55:26,038][20381] Updated weights for policy 1, policy_version 905283 (0.0007) [2023-12-27 11:55:26,101][20381] Updated weights for policy 1, policy_version 905293 (0.0008) [2023-12-27 11:55:26,159][20381] Updated weights for policy 1, policy_version 905303 (0.0007) [2023-12-27 11:55:26,250][20347] Updated weights for policy 0, policy_version 905971 (0.0010) [2023-12-27 11:55:26,304][20347] Updated weights for policy 0, policy_version 905981 (0.0010) [2023-12-27 11:55:26,355][20347] Updated weights for policy 0, policy_version 905991 (0.0010) [2023-12-27 11:55:26,861][20381] Updated weights for policy 1, policy_version 905313 (0.0009) [2023-12-27 11:55:26,910][20381] Updated weights for policy 1, policy_version 905323 (0.0009) [2023-12-27 11:55:26,942][20347] Updated weights for policy 0, policy_version 906001 (0.0010) [2023-12-27 11:55:26,961][20381] Updated weights for policy 1, policy_version 905333 (0.0007) [2023-12-27 11:55:27,004][20347] Updated weights for policy 0, policy_version 906011 (0.0009) [2023-12-27 11:55:27,056][20347] Updated weights for policy 0, policy_version 906021 (0.0010) [2023-12-27 11:55:27,107][20347] Updated weights for policy 0, policy_version 906031 (0.0010) [2023-12-27 11:55:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 463765504. Throughput: 0: 10337.4, 1: 10096.2. Samples: 463753256. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:27,363][19170] Avg episode reward: [(0, '707.040'), (1, '57.520')] [2023-12-27 11:55:27,632][20381] Updated weights for policy 1, policy_version 905343 (0.0008) [2023-12-27 11:55:27,675][20381] Updated weights for policy 1, policy_version 905353 (0.0007) [2023-12-27 11:55:27,723][20381] Updated weights for policy 1, policy_version 905363 (0.0007) [2023-12-27 11:55:27,772][20347] Updated weights for policy 0, policy_version 906041 (0.0010) [2023-12-27 11:55:27,820][20347] Updated weights for policy 0, policy_version 906051 (0.0010) [2023-12-27 11:55:27,867][20347] Updated weights for policy 0, policy_version 906061 (0.0010) [2023-12-27 11:55:28,374][20381] Updated weights for policy 1, policy_version 905373 (0.0009) [2023-12-27 11:55:28,436][20381] Updated weights for policy 1, policy_version 905383 (0.0011) [2023-12-27 11:55:28,458][20347] Updated weights for policy 0, policy_version 906071 (0.0007) [2023-12-27 11:55:28,491][20381] Updated weights for policy 1, policy_version 905393 (0.0010) [2023-12-27 11:55:28,516][20347] Updated weights for policy 0, policy_version 906081 (0.0010) [2023-12-27 11:55:28,577][20347] Updated weights for policy 0, policy_version 906091 (0.0008) [2023-12-27 11:55:29,189][20381] Updated weights for policy 1, policy_version 905403 (0.0011) [2023-12-27 11:55:29,212][20347] Updated weights for policy 0, policy_version 906101 (0.0011) [2023-12-27 11:55:29,252][20381] Updated weights for policy 1, policy_version 905413 (0.0009) [2023-12-27 11:55:29,278][20347] Updated weights for policy 0, policy_version 906111 (0.0009) [2023-12-27 11:55:29,312][20381] Updated weights for policy 1, policy_version 905423 (0.0010) [2023-12-27 11:55:29,331][20347] Updated weights for policy 0, policy_version 906121 (0.0010) [2023-12-27 11:55:29,979][20381] Updated weights for policy 1, policy_version 905433 (0.0011) [2023-12-27 11:55:30,013][20347] Updated weights for policy 0, policy_version 906131 (0.0011) [2023-12-27 11:55:30,037][20381] Updated weights for policy 1, policy_version 905443 (0.0008) [2023-12-27 11:55:30,064][20347] Updated weights for policy 0, policy_version 906141 (0.0007) [2023-12-27 11:55:30,099][20381] Updated weights for policy 1, policy_version 905453 (0.0007) [2023-12-27 11:55:30,122][20347] Updated weights for policy 0, policy_version 906151 (0.0008) [2023-12-27 11:55:30,149][20381] Updated weights for policy 1, policy_version 905463 (0.0008) [2023-12-27 11:55:30,744][20381] Updated weights for policy 1, policy_version 905473 (0.0008) [2023-12-27 11:55:30,793][20381] Updated weights for policy 1, policy_version 905483 (0.0008) [2023-12-27 11:55:30,838][20381] Updated weights for policy 1, policy_version 905493 (0.0008) [2023-12-27 11:55:30,863][20347] Updated weights for policy 0, policy_version 906161 (0.0010) [2023-12-27 11:55:30,911][20347] Updated weights for policy 0, policy_version 906171 (0.0006) [2023-12-27 11:55:30,965][20347] Updated weights for policy 0, policy_version 906181 (0.0007) [2023-12-27 11:55:31,022][20347] Updated weights for policy 0, policy_version 906191 (0.0010) [2023-12-27 11:55:31,542][20381] Updated weights for policy 1, policy_version 905503 (0.0009) [2023-12-27 11:55:31,587][20381] Updated weights for policy 1, policy_version 905513 (0.0010) [2023-12-27 11:55:31,651][20381] Updated weights for policy 1, policy_version 905523 (0.0010) [2023-12-27 11:55:31,756][20347] Updated weights for policy 0, policy_version 906201 (0.0008) [2023-12-27 11:55:31,814][20347] Updated weights for policy 0, policy_version 906211 (0.0008) [2023-12-27 11:55:31,871][20347] Updated weights for policy 0, policy_version 906221 (0.0006) [2023-12-27 11:55:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 463863808. Throughput: 0: 10336.2, 1: 10118.2. Samples: 463877568. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:32,363][19170] Avg episode reward: [(0, '736.410'), (1, '75.750')] [2023-12-27 11:55:32,363][20381] Updated weights for policy 1, policy_version 905533 (0.0008) [2023-12-27 11:55:32,417][20381] Updated weights for policy 1, policy_version 905543 (0.0008) [2023-12-27 11:55:32,466][20381] Updated weights for policy 1, policy_version 905553 (0.0006) [2023-12-27 11:55:32,504][20347] Updated weights for policy 0, policy_version 906231 (0.0006) [2023-12-27 11:55:32,562][20347] Updated weights for policy 0, policy_version 906241 (0.0007) [2023-12-27 11:55:32,616][20347] Updated weights for policy 0, policy_version 906251 (0.0007) [2023-12-27 11:55:33,127][20381] Updated weights for policy 1, policy_version 905563 (0.0005) [2023-12-27 11:55:33,177][20381] Updated weights for policy 1, policy_version 905573 (0.0005) [2023-12-27 11:55:33,233][20381] Updated weights for policy 1, policy_version 905583 (0.0005) [2023-12-27 11:55:33,272][20347] Updated weights for policy 0, policy_version 906261 (0.0008) [2023-12-27 11:55:33,325][20347] Updated weights for policy 0, policy_version 906271 (0.0007) [2023-12-27 11:55:33,369][20347] Updated weights for policy 0, policy_version 906281 (0.0005) [2023-12-27 11:55:33,842][20381] Updated weights for policy 1, policy_version 905593 (0.0005) [2023-12-27 11:55:33,903][20381] Updated weights for policy 1, policy_version 905603 (0.0008) [2023-12-27 11:55:33,961][20381] Updated weights for policy 1, policy_version 905613 (0.0008) [2023-12-27 11:55:33,980][20347] Updated weights for policy 0, policy_version 906291 (0.0005) [2023-12-27 11:55:34,016][20381] Updated weights for policy 1, policy_version 905623 (0.0007) [2023-12-27 11:55:34,028][20347] Updated weights for policy 0, policy_version 906301 (0.0005) [2023-12-27 11:55:34,078][20347] Updated weights for policy 0, policy_version 906311 (0.0006) [2023-12-27 11:55:34,681][20381] Updated weights for policy 1, policy_version 905633 (0.0006) [2023-12-27 11:55:34,740][20347] Updated weights for policy 0, policy_version 906321 (0.0008) [2023-12-27 11:55:34,745][20381] Updated weights for policy 1, policy_version 905643 (0.0006) [2023-12-27 11:55:34,800][20347] Updated weights for policy 0, policy_version 906331 (0.0005) [2023-12-27 11:55:34,805][20381] Updated weights for policy 1, policy_version 905653 (0.0006) [2023-12-27 11:55:34,854][20347] Updated weights for policy 0, policy_version 906341 (0.0007) [2023-12-27 11:55:34,898][20347] Updated weights for policy 0, policy_version 906351 (0.0007) [2023-12-27 11:55:35,421][20381] Updated weights for policy 1, policy_version 905663 (0.0006) [2023-12-27 11:55:35,476][20381] Updated weights for policy 1, policy_version 905673 (0.0006) [2023-12-27 11:55:35,532][20381] Updated weights for policy 1, policy_version 905683 (0.0005) [2023-12-27 11:55:35,608][20347] Updated weights for policy 0, policy_version 906361 (0.0009) [2023-12-27 11:55:35,656][20347] Updated weights for policy 0, policy_version 906371 (0.0008) [2023-12-27 11:55:35,704][20347] Updated weights for policy 0, policy_version 906381 (0.0007) [2023-12-27 11:55:36,139][20381] Updated weights for policy 1, policy_version 905693 (0.0007) [2023-12-27 11:55:36,194][20381] Updated weights for policy 1, policy_version 905703 (0.0010) [2023-12-27 11:55:36,256][20381] Updated weights for policy 1, policy_version 905713 (0.0007) [2023-12-27 11:55:36,362][20347] Updated weights for policy 0, policy_version 906391 (0.0008) [2023-12-27 11:55:36,421][20347] Updated weights for policy 0, policy_version 906401 (0.0008) [2023-12-27 11:55:36,484][20347] Updated weights for policy 0, policy_version 906411 (0.0007) [2023-12-27 11:55:36,975][20381] Updated weights for policy 1, policy_version 905723 (0.0009) [2023-12-27 11:55:37,032][20381] Updated weights for policy 1, policy_version 905733 (0.0006) [2023-12-27 11:55:37,089][20381] Updated weights for policy 1, policy_version 905743 (0.0005) [2023-12-27 11:55:37,174][20347] Updated weights for policy 0, policy_version 906421 (0.0006) [2023-12-27 11:55:37,231][20347] Updated weights for policy 0, policy_version 906431 (0.0006) [2023-12-27 11:55:37,295][20347] Updated weights for policy 0, policy_version 906441 (0.0007) [2023-12-27 11:55:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 463978496. Throughput: 0: 10367.2, 1: 10150.6. Samples: 463941612. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:37,364][19170] Avg episode reward: [(0, '902.420'), (1, '168.480')] [2023-12-27 11:55:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000905752_231899136.pth... [2023-12-27 11:55:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000906448_232079360.pth... [2023-12-27 11:55:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000904568_231596032.pth [2023-12-27 11:55:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000905232_231768064.pth [2023-12-27 11:55:37,692][20381] Updated weights for policy 1, policy_version 905753 (0.0005) [2023-12-27 11:55:37,748][20381] Updated weights for policy 1, policy_version 905763 (0.0008) [2023-12-27 11:55:37,804][20381] Updated weights for policy 1, policy_version 905773 (0.0006) [2023-12-27 11:55:37,864][20381] Updated weights for policy 1, policy_version 905783 (0.0007) [2023-12-27 11:55:37,944][20347] Updated weights for policy 0, policy_version 906451 (0.0009) [2023-12-27 11:55:37,999][20347] Updated weights for policy 0, policy_version 906461 (0.0010) [2023-12-27 11:55:38,049][20347] Updated weights for policy 0, policy_version 906471 (0.0010) [2023-12-27 11:55:38,598][20381] Updated weights for policy 1, policy_version 905793 (0.0008) [2023-12-27 11:55:38,659][20381] Updated weights for policy 1, policy_version 905803 (0.0008) [2023-12-27 11:55:38,660][20347] Updated weights for policy 0, policy_version 906481 (0.0010) [2023-12-27 11:55:38,723][20347] Updated weights for policy 0, policy_version 906491 (0.0006) [2023-12-27 11:55:38,723][20381] Updated weights for policy 1, policy_version 905813 (0.0008) [2023-12-27 11:55:38,789][20347] Updated weights for policy 0, policy_version 906501 (0.0006) [2023-12-27 11:55:38,854][20347] Updated weights for policy 0, policy_version 906511 (0.0008) [2023-12-27 11:55:39,427][20381] Updated weights for policy 1, policy_version 905823 (0.0008) [2023-12-27 11:55:39,490][20381] Updated weights for policy 1, policy_version 905833 (0.0008) [2023-12-27 11:55:39,529][20347] Updated weights for policy 0, policy_version 906521 (0.0008) [2023-12-27 11:55:39,547][20381] Updated weights for policy 1, policy_version 905843 (0.0008) [2023-12-27 11:55:39,586][20347] Updated weights for policy 0, policy_version 906531 (0.0007) [2023-12-27 11:55:39,639][20347] Updated weights for policy 0, policy_version 906541 (0.0008) [2023-12-27 11:55:40,307][20381] Updated weights for policy 1, policy_version 905853 (0.0008) [2023-12-27 11:55:40,359][20347] Updated weights for policy 0, policy_version 906551 (0.0008) [2023-12-27 11:55:40,366][20381] Updated weights for policy 1, policy_version 905863 (0.0006) [2023-12-27 11:55:40,411][20347] Updated weights for policy 0, policy_version 906561 (0.0007) [2023-12-27 11:55:40,414][20381] Updated weights for policy 1, policy_version 905873 (0.0006) [2023-12-27 11:55:40,467][20347] Updated weights for policy 0, policy_version 906571 (0.0005) [2023-12-27 11:55:41,078][20381] Updated weights for policy 1, policy_version 905883 (0.0007) [2023-12-27 11:55:41,147][20381] Updated weights for policy 1, policy_version 905893 (0.0008) [2023-12-27 11:55:41,167][20347] Updated weights for policy 0, policy_version 906581 (0.0007) [2023-12-27 11:55:41,212][20381] Updated weights for policy 1, policy_version 905903 (0.0008) [2023-12-27 11:55:41,222][20347] Updated weights for policy 0, policy_version 906591 (0.0007) [2023-12-27 11:55:41,279][20347] Updated weights for policy 0, policy_version 906601 (0.0008) [2023-12-27 11:55:41,945][20381] Updated weights for policy 1, policy_version 905913 (0.0008) [2023-12-27 11:55:42,000][20381] Updated weights for policy 1, policy_version 905923 (0.0008) [2023-12-27 11:55:42,045][20347] Updated weights for policy 0, policy_version 906611 (0.0007) [2023-12-27 11:55:42,060][20381] Updated weights for policy 1, policy_version 905933 (0.0010) [2023-12-27 11:55:42,102][20347] Updated weights for policy 0, policy_version 906621 (0.0006) [2023-12-27 11:55:42,112][20381] Updated weights for policy 1, policy_version 905943 (0.0010) [2023-12-27 11:55:42,157][20347] Updated weights for policy 0, policy_version 906631 (0.0007) [2023-12-27 11:55:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 464076800. Throughput: 0: 10412.9, 1: 10115.0. Samples: 464064812. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:42,363][19170] Avg episode reward: [(0, '931.860'), (1, '67.960')] [2023-12-27 11:55:42,746][20381] Updated weights for policy 1, policy_version 905953 (0.0008) [2023-12-27 11:55:42,802][20381] Updated weights for policy 1, policy_version 905963 (0.0008) [2023-12-27 11:55:42,859][20381] Updated weights for policy 1, policy_version 905973 (0.0010) [2023-12-27 11:55:42,929][20347] Updated weights for policy 0, policy_version 906641 (0.0007) [2023-12-27 11:55:42,986][20347] Updated weights for policy 0, policy_version 906651 (0.0007) [2023-12-27 11:55:43,045][20347] Updated weights for policy 0, policy_version 906661 (0.0008) [2023-12-27 11:55:43,104][20347] Updated weights for policy 0, policy_version 906671 (0.0008) [2023-12-27 11:55:43,443][20381] Updated weights for policy 1, policy_version 905983 (0.0005) [2023-12-27 11:55:43,489][20381] Updated weights for policy 1, policy_version 905993 (0.0005) [2023-12-27 11:55:43,535][20381] Updated weights for policy 1, policy_version 906003 (0.0005) [2023-12-27 11:55:43,866][20347] Updated weights for policy 0, policy_version 906681 (0.0009) [2023-12-27 11:55:43,930][20347] Updated weights for policy 0, policy_version 906691 (0.0008) [2023-12-27 11:55:43,990][20347] Updated weights for policy 0, policy_version 906701 (0.0008) [2023-12-27 11:55:44,080][20381] Updated weights for policy 1, policy_version 906013 (0.0005) [2023-12-27 11:55:44,142][20381] Updated weights for policy 1, policy_version 906023 (0.0005) [2023-12-27 11:55:44,195][20381] Updated weights for policy 1, policy_version 906033 (0.0005) [2023-12-27 11:55:44,640][20347] Updated weights for policy 0, policy_version 906711 (0.0007) [2023-12-27 11:55:44,698][20347] Updated weights for policy 0, policy_version 906721 (0.0007) [2023-12-27 11:55:44,736][20381] Updated weights for policy 1, policy_version 906043 (0.0005) [2023-12-27 11:55:44,760][20347] Updated weights for policy 0, policy_version 906731 (0.0007) [2023-12-27 11:55:44,794][20381] Updated weights for policy 1, policy_version 906053 (0.0008) [2023-12-27 11:55:44,853][20381] Updated weights for policy 1, policy_version 906063 (0.0011) [2023-12-27 11:55:45,461][20347] Updated weights for policy 0, policy_version 906741 (0.0007) [2023-12-27 11:55:45,514][20347] Updated weights for policy 0, policy_version 906751 (0.0008) [2023-12-27 11:55:45,544][20381] Updated weights for policy 1, policy_version 906073 (0.0009) [2023-12-27 11:55:45,565][20347] Updated weights for policy 0, policy_version 906761 (0.0007) [2023-12-27 11:55:45,594][20381] Updated weights for policy 1, policy_version 906083 (0.0006) [2023-12-27 11:55:45,647][20381] Updated weights for policy 1, policy_version 906093 (0.0008) [2023-12-27 11:55:45,706][20381] Updated weights for policy 1, policy_version 906103 (0.0008) [2023-12-27 11:55:46,316][20347] Updated weights for policy 0, policy_version 906771 (0.0007) [2023-12-27 11:55:46,316][20381] Updated weights for policy 1, policy_version 906113 (0.0008) [2023-12-27 11:55:46,364][20347] Updated weights for policy 0, policy_version 906781 (0.0006) [2023-12-27 11:55:46,365][20381] Updated weights for policy 1, policy_version 906123 (0.0008) [2023-12-27 11:55:46,414][20381] Updated weights for policy 1, policy_version 906133 (0.0006) [2023-12-27 11:55:46,420][20347] Updated weights for policy 0, policy_version 906791 (0.0006) [2023-12-27 11:55:47,061][20347] Updated weights for policy 0, policy_version 906801 (0.0008) [2023-12-27 11:55:47,114][20347] Updated weights for policy 0, policy_version 906811 (0.0008) [2023-12-27 11:55:47,144][20381] Updated weights for policy 1, policy_version 906143 (0.0006) [2023-12-27 11:55:47,168][20347] Updated weights for policy 0, policy_version 906821 (0.0007) [2023-12-27 11:55:47,199][20381] Updated weights for policy 1, policy_version 906153 (0.0006) [2023-12-27 11:55:47,220][20347] Updated weights for policy 0, policy_version 906831 (0.0006) [2023-12-27 11:55:47,256][20381] Updated weights for policy 1, policy_version 906163 (0.0007) [2023-12-27 11:55:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 464183296. Throughput: 0: 10331.7, 1: 10196.9. Samples: 464188004. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:47,363][19170] Avg episode reward: [(0, '814.540'), (1, '72.340')] [2023-12-27 11:55:47,807][20347] Updated weights for policy 0, policy_version 906841 (0.0006) [2023-12-27 11:55:47,867][20347] Updated weights for policy 0, policy_version 906851 (0.0007) [2023-12-27 11:55:47,913][20347] Updated weights for policy 0, policy_version 906861 (0.0007) [2023-12-27 11:55:47,991][20381] Updated weights for policy 1, policy_version 906173 (0.0008) [2023-12-27 11:55:48,056][20381] Updated weights for policy 1, policy_version 906183 (0.0007) [2023-12-27 11:55:48,114][20381] Updated weights for policy 1, policy_version 906193 (0.0008) [2023-12-27 11:55:48,585][20347] Updated weights for policy 0, policy_version 906871 (0.0008) [2023-12-27 11:55:48,598][20291] KL-divergence is very high: 478.2333 [2023-12-27 11:55:48,610][20291] KL-divergence is very high: 321.2295 [2023-12-27 11:55:48,616][20291] KL-divergence is very high: 238.8897 [2023-12-27 11:55:48,622][20291] KL-divergence is very high: 271.5061 [2023-12-27 11:55:48,645][20347] Updated weights for policy 0, policy_version 906881 (0.0008) [2023-12-27 11:55:48,705][20347] Updated weights for policy 0, policy_version 906891 (0.0008) [2023-12-27 11:55:48,705][20291] KL-divergence is very high: 105.6125 [2023-12-27 11:55:48,800][20381] Updated weights for policy 1, policy_version 906203 (0.0006) [2023-12-27 11:55:48,853][20381] Updated weights for policy 1, policy_version 906213 (0.0008) [2023-12-27 11:55:48,907][20381] Updated weights for policy 1, policy_version 906223 (0.0008) [2023-12-27 11:55:49,370][20347] Updated weights for policy 0, policy_version 906901 (0.0008) [2023-12-27 11:55:49,425][20347] Updated weights for policy 0, policy_version 906911 (0.0008) [2023-12-27 11:55:49,480][20347] Updated weights for policy 0, policy_version 906921 (0.0008) [2023-12-27 11:55:49,604][20381] Updated weights for policy 1, policy_version 906233 (0.0008) [2023-12-27 11:55:49,671][20381] Updated weights for policy 1, policy_version 906243 (0.0006) [2023-12-27 11:55:49,740][20381] Updated weights for policy 1, policy_version 906253 (0.0006) [2023-12-27 11:55:49,800][20381] Updated weights for policy 1, policy_version 906263 (0.0006) [2023-12-27 11:55:50,273][20347] Updated weights for policy 0, policy_version 906931 (0.0009) [2023-12-27 11:55:50,332][20347] Updated weights for policy 0, policy_version 906941 (0.0010) [2023-12-27 11:55:50,387][20347] Updated weights for policy 0, policy_version 906951 (0.0010) [2023-12-27 11:55:50,467][20381] Updated weights for policy 1, policy_version 906273 (0.0006) [2023-12-27 11:55:50,512][20381] Updated weights for policy 1, policy_version 906283 (0.0006) [2023-12-27 11:55:50,563][20381] Updated weights for policy 1, policy_version 906293 (0.0005) [2023-12-27 11:55:51,101][20347] Updated weights for policy 0, policy_version 906961 (0.0010) [2023-12-27 11:55:51,165][20347] Updated weights for policy 0, policy_version 906971 (0.0010) [2023-12-27 11:55:51,220][20347] Updated weights for policy 0, policy_version 906981 (0.0010) [2023-12-27 11:55:51,231][20381] Updated weights for policy 1, policy_version 906303 (0.0009) [2023-12-27 11:55:51,276][20347] Updated weights for policy 0, policy_version 906991 (0.0010) [2023-12-27 11:55:51,290][20381] Updated weights for policy 1, policy_version 906313 (0.0008) [2023-12-27 11:55:51,343][20381] Updated weights for policy 1, policy_version 906323 (0.0011) [2023-12-27 11:55:51,939][20347] Updated weights for policy 0, policy_version 907001 (0.0007) [2023-12-27 11:55:51,988][20347] Updated weights for policy 0, policy_version 907011 (0.0007) [2023-12-27 11:55:52,043][20347] Updated weights for policy 0, policy_version 907021 (0.0007) [2023-12-27 11:55:52,045][20381] Updated weights for policy 1, policy_version 906333 (0.0010) [2023-12-27 11:55:52,108][20381] Updated weights for policy 1, policy_version 906343 (0.0011) [2023-12-27 11:55:52,167][20381] Updated weights for policy 1, policy_version 906353 (0.0011) [2023-12-27 11:55:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 464281600. Throughput: 0: 10304.1, 1: 10146.9. Samples: 464248700. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:52,363][19170] Avg episode reward: [(0, '804.470'), (1, '177.220')] [2023-12-27 11:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000906360_232054784.pth... [2023-12-27 11:55:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000907024_232226816.pth... [2023-12-27 11:55:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000905144_231743488.pth [2023-12-27 11:55:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000905808_231915520.pth [2023-12-27 11:55:52,695][20347] Updated weights for policy 0, policy_version 907031 (0.0008) [2023-12-27 11:55:52,756][20347] Updated weights for policy 0, policy_version 907041 (0.0007) [2023-12-27 11:55:52,805][20347] Updated weights for policy 0, policy_version 907051 (0.0007) [2023-12-27 11:55:52,840][20381] Updated weights for policy 1, policy_version 906363 (0.0009) [2023-12-27 11:55:52,893][20381] Updated weights for policy 1, policy_version 906373 (0.0005) [2023-12-27 11:55:52,939][20381] Updated weights for policy 1, policy_version 906383 (0.0005) [2023-12-27 11:55:53,435][20347] Updated weights for policy 0, policy_version 907061 (0.0007) [2023-12-27 11:55:53,498][20347] Updated weights for policy 0, policy_version 907071 (0.0007) [2023-12-27 11:55:53,559][20347] Updated weights for policy 0, policy_version 907081 (0.0007) [2023-12-27 11:55:53,566][20381] Updated weights for policy 1, policy_version 906393 (0.0006) [2023-12-27 11:55:53,637][20381] Updated weights for policy 1, policy_version 906403 (0.0010) [2023-12-27 11:55:53,701][20381] Updated weights for policy 1, policy_version 906413 (0.0010) [2023-12-27 11:55:53,750][20381] Updated weights for policy 1, policy_version 906423 (0.0010) [2023-12-27 11:55:54,124][20347] Updated weights for policy 0, policy_version 907091 (0.0008) [2023-12-27 11:55:54,179][20347] Updated weights for policy 0, policy_version 907101 (0.0010) [2023-12-27 11:55:54,225][20347] Updated weights for policy 0, policy_version 907111 (0.0009) [2023-12-27 11:55:54,373][20381] Updated weights for policy 1, policy_version 906433 (0.0010) [2023-12-27 11:55:54,423][20381] Updated weights for policy 1, policy_version 906443 (0.0006) [2023-12-27 11:55:54,482][20381] Updated weights for policy 1, policy_version 906453 (0.0009) [2023-12-27 11:55:54,799][20347] Updated weights for policy 0, policy_version 907121 (0.0006) [2023-12-27 11:55:54,863][20347] Updated weights for policy 0, policy_version 907131 (0.0005) [2023-12-27 11:55:54,923][20347] Updated weights for policy 0, policy_version 907141 (0.0007) [2023-12-27 11:55:54,985][20347] Updated weights for policy 0, policy_version 907151 (0.0008) [2023-12-27 11:55:55,167][20381] Updated weights for policy 1, policy_version 906463 (0.0010) [2023-12-27 11:55:55,219][20381] Updated weights for policy 1, policy_version 906473 (0.0010) [2023-12-27 11:55:55,270][20381] Updated weights for policy 1, policy_version 906483 (0.0010) [2023-12-27 11:55:55,556][20347] Updated weights for policy 0, policy_version 907161 (0.0005) [2023-12-27 11:55:55,622][20347] Updated weights for policy 0, policy_version 907171 (0.0005) [2023-12-27 11:55:55,675][20347] Updated weights for policy 0, policy_version 907181 (0.0005) [2023-12-27 11:55:55,927][20381] Updated weights for policy 1, policy_version 906493 (0.0010) [2023-12-27 11:55:55,976][20381] Updated weights for policy 1, policy_version 906503 (0.0010) [2023-12-27 11:55:56,034][20381] Updated weights for policy 1, policy_version 906513 (0.0011) [2023-12-27 11:55:56,291][20347] Updated weights for policy 0, policy_version 907191 (0.0007) [2023-12-27 11:55:56,344][20347] Updated weights for policy 0, policy_version 907201 (0.0007) [2023-12-27 11:55:56,395][20347] Updated weights for policy 0, policy_version 907211 (0.0007) [2023-12-27 11:55:56,744][20381] Updated weights for policy 1, policy_version 906523 (0.0009) [2023-12-27 11:55:56,799][20381] Updated weights for policy 1, policy_version 906533 (0.0005) [2023-12-27 11:55:56,845][20381] Updated weights for policy 1, policy_version 906543 (0.0005) [2023-12-27 11:55:57,003][20347] Updated weights for policy 0, policy_version 907221 (0.0006) [2023-12-27 11:55:57,061][20347] Updated weights for policy 0, policy_version 907231 (0.0005) [2023-12-27 11:55:57,114][20347] Updated weights for policy 0, policy_version 907241 (0.0005) [2023-12-27 11:55:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 464388096. Throughput: 0: 10372.4, 1: 10165.9. Samples: 464376668. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:55:57,363][19170] Avg episode reward: [(0, '941.290'), (1, '223.160')] [2023-12-27 11:55:57,521][20381] Updated weights for policy 1, policy_version 906553 (0.0006) [2023-12-27 11:55:57,577][20381] Updated weights for policy 1, policy_version 906563 (0.0006) [2023-12-27 11:55:57,622][20381] Updated weights for policy 1, policy_version 906573 (0.0005) [2023-12-27 11:55:57,640][20347] Updated weights for policy 0, policy_version 907251 (0.0006) [2023-12-27 11:55:57,672][20381] Updated weights for policy 1, policy_version 906583 (0.0005) [2023-12-27 11:55:57,685][20347] Updated weights for policy 0, policy_version 907261 (0.0005) [2023-12-27 11:55:57,732][20347] Updated weights for policy 0, policy_version 907271 (0.0008) [2023-12-27 11:55:58,224][20381] Updated weights for policy 1, policy_version 906593 (0.0010) [2023-12-27 11:55:58,283][20381] Updated weights for policy 1, policy_version 906603 (0.0011) [2023-12-27 11:55:58,349][20381] Updated weights for policy 1, policy_version 906613 (0.0010) [2023-12-27 11:55:58,458][20347] Updated weights for policy 0, policy_version 907281 (0.0007) [2023-12-27 11:55:58,520][20347] Updated weights for policy 0, policy_version 907291 (0.0007) [2023-12-27 11:55:58,586][20347] Updated weights for policy 0, policy_version 907301 (0.0007) [2023-12-27 11:55:58,649][20347] Updated weights for policy 0, policy_version 907311 (0.0008) [2023-12-27 11:55:59,143][20381] Updated weights for policy 1, policy_version 906623 (0.0006) [2023-12-27 11:55:59,210][20381] Updated weights for policy 1, policy_version 906633 (0.0006) [2023-12-27 11:55:59,278][20381] Updated weights for policy 1, policy_version 906643 (0.0008) [2023-12-27 11:55:59,451][20347] Updated weights for policy 0, policy_version 907321 (0.0008) [2023-12-27 11:55:59,499][20347] Updated weights for policy 0, policy_version 907331 (0.0008) [2023-12-27 11:55:59,554][20347] Updated weights for policy 0, policy_version 907341 (0.0008) [2023-12-27 11:55:59,969][20381] Updated weights for policy 1, policy_version 906653 (0.0009) [2023-12-27 11:56:00,029][20381] Updated weights for policy 1, policy_version 906663 (0.0011) [2023-12-27 11:56:00,093][20381] Updated weights for policy 1, policy_version 906673 (0.0011) [2023-12-27 11:56:00,242][20347] Updated weights for policy 0, policy_version 907351 (0.0006) [2023-12-27 11:56:00,294][20347] Updated weights for policy 0, policy_version 907361 (0.0007) [2023-12-27 11:56:00,352][20347] Updated weights for policy 0, policy_version 907371 (0.0007) [2023-12-27 11:56:00,741][20381] Updated weights for policy 1, policy_version 906683 (0.0010) [2023-12-27 11:56:00,792][20381] Updated weights for policy 1, policy_version 906693 (0.0010) [2023-12-27 11:56:00,854][20381] Updated weights for policy 1, policy_version 906703 (0.0011) [2023-12-27 11:56:01,094][20347] Updated weights for policy 0, policy_version 907381 (0.0007) [2023-12-27 11:56:01,163][20347] Updated weights for policy 0, policy_version 907391 (0.0008) [2023-12-27 11:56:01,227][20347] Updated weights for policy 0, policy_version 907401 (0.0008) [2023-12-27 11:56:01,564][20381] Updated weights for policy 1, policy_version 906713 (0.0011) [2023-12-27 11:56:01,620][20381] Updated weights for policy 1, policy_version 906723 (0.0010) [2023-12-27 11:56:01,678][20381] Updated weights for policy 1, policy_version 906733 (0.0007) [2023-12-27 11:56:01,744][20381] Updated weights for policy 1, policy_version 906743 (0.0007) [2023-12-27 11:56:01,930][20347] Updated weights for policy 0, policy_version 907411 (0.0009) [2023-12-27 11:56:01,988][20347] Updated weights for policy 0, policy_version 907421 (0.0010) [2023-12-27 11:56:02,044][20347] Updated weights for policy 0, policy_version 907431 (0.0010) [2023-12-27 11:56:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 464486400. Throughput: 0: 10362.3, 1: 10205.9. Samples: 464497932. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:56:02,363][19170] Avg episode reward: [(0, '951.270'), (1, '297.550')] [2023-12-27 11:56:02,393][20381] Updated weights for policy 1, policy_version 906753 (0.0008) [2023-12-27 11:56:02,448][20381] Updated weights for policy 1, policy_version 906763 (0.0007) [2023-12-27 11:56:02,498][20381] Updated weights for policy 1, policy_version 906773 (0.0007) [2023-12-27 11:56:02,711][20347] Updated weights for policy 0, policy_version 907441 (0.0010) [2023-12-27 11:56:02,760][20347] Updated weights for policy 0, policy_version 907451 (0.0008) [2023-12-27 11:56:02,813][20347] Updated weights for policy 0, policy_version 907461 (0.0008) [2023-12-27 11:56:02,862][20347] Updated weights for policy 0, policy_version 907471 (0.0010) [2023-12-27 11:56:03,211][20381] Updated weights for policy 1, policy_version 906783 (0.0007) [2023-12-27 11:56:03,266][20381] Updated weights for policy 1, policy_version 906793 (0.0007) [2023-12-27 11:56:03,320][20381] Updated weights for policy 1, policy_version 906803 (0.0008) [2023-12-27 11:56:03,519][20347] Updated weights for policy 0, policy_version 907481 (0.0010) [2023-12-27 11:56:03,568][20347] Updated weights for policy 0, policy_version 907491 (0.0010) [2023-12-27 11:56:03,613][20347] Updated weights for policy 0, policy_version 907501 (0.0010) [2023-12-27 11:56:03,957][20381] Updated weights for policy 1, policy_version 906813 (0.0007) [2023-12-27 11:56:04,014][20381] Updated weights for policy 1, policy_version 906823 (0.0006) [2023-12-27 11:56:04,078][20381] Updated weights for policy 1, policy_version 906833 (0.0006) [2023-12-27 11:56:04,362][20347] Updated weights for policy 0, policy_version 907511 (0.0009) [2023-12-27 11:56:04,419][20347] Updated weights for policy 0, policy_version 907521 (0.0008) [2023-12-27 11:56:04,479][20347] Updated weights for policy 0, policy_version 907531 (0.0008) [2023-12-27 11:56:04,720][20381] Updated weights for policy 1, policy_version 906843 (0.0006) [2023-12-27 11:56:04,769][20381] Updated weights for policy 1, policy_version 906853 (0.0007) [2023-12-27 11:56:04,824][20381] Updated weights for policy 1, policy_version 906863 (0.0008) [2023-12-27 11:56:05,150][20347] Updated weights for policy 0, policy_version 907541 (0.0007) [2023-12-27 11:56:05,210][20347] Updated weights for policy 0, policy_version 907551 (0.0007) [2023-12-27 11:56:05,259][20347] Updated weights for policy 0, policy_version 907561 (0.0005) [2023-12-27 11:56:05,514][20381] Updated weights for policy 1, policy_version 906873 (0.0007) [2023-12-27 11:56:05,578][20381] Updated weights for policy 1, policy_version 906883 (0.0006) [2023-12-27 11:56:05,629][20381] Updated weights for policy 1, policy_version 906893 (0.0006) [2023-12-27 11:56:05,677][20381] Updated weights for policy 1, policy_version 906903 (0.0008) [2023-12-27 11:56:05,896][20347] Updated weights for policy 0, policy_version 907571 (0.0006) [2023-12-27 11:56:05,951][20347] Updated weights for policy 0, policy_version 907581 (0.0007) [2023-12-27 11:56:06,003][20347] Updated weights for policy 0, policy_version 907591 (0.0007) [2023-12-27 11:56:06,398][20381] Updated weights for policy 1, policy_version 906913 (0.0007) [2023-12-27 11:56:06,453][20381] Updated weights for policy 1, policy_version 906923 (0.0008) [2023-12-27 11:56:06,514][20381] Updated weights for policy 1, policy_version 906933 (0.0007) [2023-12-27 11:56:06,744][20347] Updated weights for policy 0, policy_version 907601 (0.0007) [2023-12-27 11:56:06,792][20347] Updated weights for policy 0, policy_version 907611 (0.0007) [2023-12-27 11:56:06,851][20347] Updated weights for policy 0, policy_version 907621 (0.0009) [2023-12-27 11:56:06,911][20347] Updated weights for policy 0, policy_version 907631 (0.0005) [2023-12-27 11:56:07,209][20381] Updated weights for policy 1, policy_version 906943 (0.0008) [2023-12-27 11:56:07,276][20381] Updated weights for policy 1, policy_version 906953 (0.0008) [2023-12-27 11:56:07,339][20381] Updated weights for policy 1, policy_version 906963 (0.0008) [2023-12-27 11:56:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 464584704. Throughput: 0: 10342.0, 1: 10263.6. Samples: 464559528. Policy #0 lag: (min: 29.0, avg: 30.6, max: 61.0) [2023-12-27 11:56:07,363][19170] Avg episode reward: [(0, '961.140'), (1, '316.100')] [2023-12-27 11:56:07,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000907632_232382464.pth... [2023-12-27 11:56:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000906968_232210432.pth... [2023-12-27 11:56:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000906448_232079360.pth [2023-12-27 11:56:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000905752_231899136.pth [2023-12-27 11:56:07,501][20347] Updated weights for policy 0, policy_version 907641 (0.0006) [2023-12-27 11:56:07,550][20347] Updated weights for policy 0, policy_version 907651 (0.0007) [2023-12-27 11:56:07,607][20347] Updated weights for policy 0, policy_version 907661 (0.0007) [2023-12-27 11:56:08,027][20381] Updated weights for policy 1, policy_version 906973 (0.0007) [2023-12-27 11:56:08,095][20381] Updated weights for policy 1, policy_version 906983 (0.0005) [2023-12-27 11:56:08,155][20381] Updated weights for policy 1, policy_version 906993 (0.0005) [2023-12-27 11:56:08,281][20347] Updated weights for policy 0, policy_version 907671 (0.0008) [2023-12-27 11:56:08,348][20347] Updated weights for policy 0, policy_version 907681 (0.0007) [2023-12-27 11:56:08,404][20347] Updated weights for policy 0, policy_version 907691 (0.0008) [2023-12-27 11:56:08,775][20381] Updated weights for policy 1, policy_version 907003 (0.0006) [2023-12-27 11:56:08,838][20381] Updated weights for policy 1, policy_version 907013 (0.0008) [2023-12-27 11:56:08,898][20381] Updated weights for policy 1, policy_version 907023 (0.0008) [2023-12-27 11:56:09,090][20347] Updated weights for policy 0, policy_version 907701 (0.0007) [2023-12-27 11:56:09,145][20347] Updated weights for policy 0, policy_version 907711 (0.0006) [2023-12-27 11:56:09,200][20347] Updated weights for policy 0, policy_version 907721 (0.0005) [2023-12-27 11:56:09,619][20381] Updated weights for policy 1, policy_version 907033 (0.0007) [2023-12-27 11:56:09,683][20381] Updated weights for policy 1, policy_version 907043 (0.0005) [2023-12-27 11:56:09,749][20381] Updated weights for policy 1, policy_version 907053 (0.0006) [2023-12-27 11:56:09,812][20381] Updated weights for policy 1, policy_version 907063 (0.0006) [2023-12-27 11:56:09,965][20347] Updated weights for policy 0, policy_version 907731 (0.0008) [2023-12-27 11:56:10,027][20347] Updated weights for policy 0, policy_version 907741 (0.0008) [2023-12-27 11:56:10,085][20347] Updated weights for policy 0, policy_version 907751 (0.0007) [2023-12-27 11:56:10,452][20381] Updated weights for policy 1, policy_version 907073 (0.0006) [2023-12-27 11:56:10,501][20381] Updated weights for policy 1, policy_version 907083 (0.0006) [2023-12-27 11:56:10,564][20381] Updated weights for policy 1, policy_version 907093 (0.0006) [2023-12-27 11:56:10,727][20347] Updated weights for policy 0, policy_version 907761 (0.0008) [2023-12-27 11:56:10,791][20347] Updated weights for policy 0, policy_version 907771 (0.0007) [2023-12-27 11:56:10,836][20347] Updated weights for policy 0, policy_version 907781 (0.0007) [2023-12-27 11:56:10,897][20347] Updated weights for policy 0, policy_version 907791 (0.0005) [2023-12-27 11:56:11,196][20381] Updated weights for policy 1, policy_version 907103 (0.0007) [2023-12-27 11:56:11,251][20381] Updated weights for policy 1, policy_version 907113 (0.0008) [2023-12-27 11:56:11,314][20381] Updated weights for policy 1, policy_version 907123 (0.0008) [2023-12-27 11:56:11,623][20347] Updated weights for policy 0, policy_version 907801 (0.0008) [2023-12-27 11:56:11,692][20347] Updated weights for policy 0, policy_version 907811 (0.0007) [2023-12-27 11:56:11,755][20347] Updated weights for policy 0, policy_version 907821 (0.0007) [2023-12-27 11:56:12,156][20381] Updated weights for policy 1, policy_version 907133 (0.0008) [2023-12-27 11:56:12,215][20381] Updated weights for policy 1, policy_version 907143 (0.0007) [2023-12-27 11:56:12,279][20381] Updated weights for policy 1, policy_version 907153 (0.0008) [2023-12-27 11:56:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 464691200. Throughput: 0: 10299.7, 1: 10339.0. Samples: 464682000. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:12,363][19170] Avg episode reward: [(0, '980.620'), (1, '428.710')] [2023-12-27 11:56:12,455][20347] Updated weights for policy 0, policy_version 907831 (0.0007) [2023-12-27 11:56:12,510][20347] Updated weights for policy 0, policy_version 907841 (0.0008) [2023-12-27 11:56:12,569][20347] Updated weights for policy 0, policy_version 907851 (0.0008) [2023-12-27 11:56:12,955][20381] Updated weights for policy 1, policy_version 907163 (0.0007) [2023-12-27 11:56:13,019][20381] Updated weights for policy 1, policy_version 907173 (0.0008) [2023-12-27 11:56:13,080][20381] Updated weights for policy 1, policy_version 907183 (0.0007) [2023-12-27 11:56:13,221][20347] Updated weights for policy 0, policy_version 907861 (0.0010) [2023-12-27 11:56:13,276][20347] Updated weights for policy 0, policy_version 907871 (0.0010) [2023-12-27 11:56:13,324][20347] Updated weights for policy 0, policy_version 907881 (0.0010) [2023-12-27 11:56:13,707][20381] Updated weights for policy 1, policy_version 907193 (0.0007) [2023-12-27 11:56:13,767][20381] Updated weights for policy 1, policy_version 907203 (0.0006) [2023-12-27 11:56:13,821][20381] Updated weights for policy 1, policy_version 907213 (0.0007) [2023-12-27 11:56:13,879][20381] Updated weights for policy 1, policy_version 907223 (0.0007) [2023-12-27 11:56:14,010][20347] Updated weights for policy 0, policy_version 907891 (0.0010) [2023-12-27 11:56:14,056][20347] Updated weights for policy 0, policy_version 907901 (0.0010) [2023-12-27 11:56:14,112][20347] Updated weights for policy 0, policy_version 907911 (0.0010) [2023-12-27 11:56:14,546][20381] Updated weights for policy 1, policy_version 907233 (0.0008) [2023-12-27 11:56:14,606][20381] Updated weights for policy 1, policy_version 907243 (0.0011) [2023-12-27 11:56:14,666][20381] Updated weights for policy 1, policy_version 907253 (0.0009) [2023-12-27 11:56:14,802][20347] Updated weights for policy 0, policy_version 907921 (0.0010) [2023-12-27 11:56:14,856][20347] Updated weights for policy 0, policy_version 907931 (0.0009) [2023-12-27 11:56:14,909][20347] Updated weights for policy 0, policy_version 907941 (0.0010) [2023-12-27 11:56:14,963][20347] Updated weights for policy 0, policy_version 907951 (0.0011) [2023-12-27 11:56:15,319][20381] Updated weights for policy 1, policy_version 907263 (0.0009) [2023-12-27 11:56:15,378][20381] Updated weights for policy 1, policy_version 907273 (0.0008) [2023-12-27 11:56:15,430][20381] Updated weights for policy 1, policy_version 907283 (0.0007) [2023-12-27 11:56:15,646][20347] Updated weights for policy 0, policy_version 907961 (0.0006) [2023-12-27 11:56:15,694][20347] Updated weights for policy 0, policy_version 907971 (0.0008) [2023-12-27 11:56:15,744][20347] Updated weights for policy 0, policy_version 907981 (0.0010) [2023-12-27 11:56:16,072][20381] Updated weights for policy 1, policy_version 907293 (0.0007) [2023-12-27 11:56:16,130][20381] Updated weights for policy 1, policy_version 907303 (0.0007) [2023-12-27 11:56:16,181][20381] Updated weights for policy 1, policy_version 907313 (0.0005) [2023-12-27 11:56:16,420][20347] Updated weights for policy 0, policy_version 907991 (0.0009) [2023-12-27 11:56:16,469][20347] Updated weights for policy 0, policy_version 908001 (0.0007) [2023-12-27 11:56:16,515][20347] Updated weights for policy 0, policy_version 908011 (0.0008) [2023-12-27 11:56:16,806][20381] Updated weights for policy 1, policy_version 907323 (0.0008) [2023-12-27 11:56:16,858][20381] Updated weights for policy 1, policy_version 907333 (0.0008) [2023-12-27 11:56:16,911][20381] Updated weights for policy 1, policy_version 907343 (0.0008) [2023-12-27 11:56:17,247][20347] Updated weights for policy 0, policy_version 908021 (0.0008) [2023-12-27 11:56:17,295][20347] Updated weights for policy 0, policy_version 908031 (0.0008) [2023-12-27 11:56:17,346][20347] Updated weights for policy 0, policy_version 908041 (0.0007) [2023-12-27 11:56:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 464789504. Throughput: 0: 10259.3, 1: 10323.5. Samples: 464803796. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:17,364][19170] Avg episode reward: [(0, '961.270'), (1, '342.410')] [2023-12-27 11:56:17,579][20381] Updated weights for policy 1, policy_version 907353 (0.0008) [2023-12-27 11:56:17,634][20381] Updated weights for policy 1, policy_version 907363 (0.0008) [2023-12-27 11:56:17,689][20381] Updated weights for policy 1, policy_version 907373 (0.0008) [2023-12-27 11:56:17,739][20381] Updated weights for policy 1, policy_version 907383 (0.0008) [2023-12-27 11:56:18,010][20347] Updated weights for policy 0, policy_version 908051 (0.0007) [2023-12-27 11:56:18,058][20347] Updated weights for policy 0, policy_version 908061 (0.0005) [2023-12-27 11:56:18,106][20347] Updated weights for policy 0, policy_version 908071 (0.0006) [2023-12-27 11:56:18,446][20381] Updated weights for policy 1, policy_version 907393 (0.0006) [2023-12-27 11:56:18,501][20381] Updated weights for policy 1, policy_version 907403 (0.0006) [2023-12-27 11:56:18,568][20381] Updated weights for policy 1, policy_version 907413 (0.0007) [2023-12-27 11:56:18,815][20347] Updated weights for policy 0, policy_version 908081 (0.0007) [2023-12-27 11:56:18,877][20347] Updated weights for policy 0, policy_version 908091 (0.0008) [2023-12-27 11:56:18,891][20291] KL-divergence is very high: 297.2561 [2023-12-27 11:56:18,939][20291] KL-divergence is very high: 112.0066 [2023-12-27 11:56:18,939][20347] Updated weights for policy 0, policy_version 908101 (0.0008) [2023-12-27 11:56:18,975][20291] KL-divergence is very high: 105.9911 [2023-12-27 11:56:18,986][20291] KL-divergence is very high: 171.8918 [2023-12-27 11:56:18,998][20347] Updated weights for policy 0, policy_version 908111 (0.0008) [2023-12-27 11:56:19,199][20381] Updated weights for policy 1, policy_version 907423 (0.0006) [2023-12-27 11:56:19,263][20381] Updated weights for policy 1, policy_version 907433 (0.0007) [2023-12-27 11:56:19,323][20381] Updated weights for policy 1, policy_version 907443 (0.0008) [2023-12-27 11:56:19,724][20347] Updated weights for policy 0, policy_version 908121 (0.0008) [2023-12-27 11:56:19,790][20347] Updated weights for policy 0, policy_version 908131 (0.0007) [2023-12-27 11:56:19,854][20347] Updated weights for policy 0, policy_version 908141 (0.0008) [2023-12-27 11:56:20,086][20381] Updated weights for policy 1, policy_version 907453 (0.0008) [2023-12-27 11:56:20,149][20381] Updated weights for policy 1, policy_version 907463 (0.0009) [2023-12-27 11:56:20,208][20381] Updated weights for policy 1, policy_version 907473 (0.0008) [2023-12-27 11:56:20,588][20347] Updated weights for policy 0, policy_version 908151 (0.0008) [2023-12-27 11:56:20,652][20347] Updated weights for policy 0, policy_version 908161 (0.0007) [2023-12-27 11:56:20,709][20347] Updated weights for policy 0, policy_version 908171 (0.0007) [2023-12-27 11:56:20,939][20381] Updated weights for policy 1, policy_version 907483 (0.0008) [2023-12-27 11:56:20,996][20381] Updated weights for policy 1, policy_version 907493 (0.0008) [2023-12-27 11:56:21,060][20381] Updated weights for policy 1, policy_version 907503 (0.0008) [2023-12-27 11:56:21,419][20347] Updated weights for policy 0, policy_version 908181 (0.0008) [2023-12-27 11:56:21,478][20347] Updated weights for policy 0, policy_version 908191 (0.0008) [2023-12-27 11:56:21,537][20347] Updated weights for policy 0, policy_version 908201 (0.0008) [2023-12-27 11:56:21,885][20381] Updated weights for policy 1, policy_version 907513 (0.0008) [2023-12-27 11:56:21,943][20381] Updated weights for policy 1, policy_version 907523 (0.0008) [2023-12-27 11:56:22,007][20381] Updated weights for policy 1, policy_version 907533 (0.0009) [2023-12-27 11:56:22,068][20381] Updated weights for policy 1, policy_version 907543 (0.0008) [2023-12-27 11:56:22,201][20347] Updated weights for policy 0, policy_version 908211 (0.0008) [2023-12-27 11:56:22,251][20347] Updated weights for policy 0, policy_version 908221 (0.0006) [2023-12-27 11:56:22,312][20347] Updated weights for policy 0, policy_version 908231 (0.0007) [2023-12-27 11:56:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 464887808. Throughput: 0: 10199.3, 1: 10273.2. Samples: 464862872. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:22,363][19170] Avg episode reward: [(0, '961.290'), (1, '199.450')] [2023-12-27 11:56:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000908240_232538112.pth... [2023-12-27 11:56:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000907544_232357888.pth... [2023-12-27 11:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000906360_232054784.pth [2023-12-27 11:56:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000907024_232226816.pth [2023-12-27 11:56:22,795][20381] Updated weights for policy 1, policy_version 907553 (0.0007) [2023-12-27 11:56:22,855][20381] Updated weights for policy 1, policy_version 907563 (0.0008) [2023-12-27 11:56:22,913][20381] Updated weights for policy 1, policy_version 907573 (0.0008) [2023-12-27 11:56:23,007][20347] Updated weights for policy 0, policy_version 908241 (0.0008) [2023-12-27 11:56:23,060][20347] Updated weights for policy 0, policy_version 908251 (0.0008) [2023-12-27 11:56:23,113][20347] Updated weights for policy 0, policy_version 908261 (0.0007) [2023-12-27 11:56:23,172][20347] Updated weights for policy 0, policy_version 908271 (0.0006) [2023-12-27 11:56:23,587][20381] Updated weights for policy 1, policy_version 907583 (0.0008) [2023-12-27 11:56:23,647][20381] Updated weights for policy 1, policy_version 907593 (0.0007) [2023-12-27 11:56:23,704][20381] Updated weights for policy 1, policy_version 907603 (0.0007) [2023-12-27 11:56:23,796][20347] Updated weights for policy 0, policy_version 908281 (0.0007) [2023-12-27 11:56:23,846][20347] Updated weights for policy 0, policy_version 908291 (0.0007) [2023-12-27 11:56:23,894][20347] Updated weights for policy 0, policy_version 908301 (0.0007) [2023-12-27 11:56:24,397][20381] Updated weights for policy 1, policy_version 907613 (0.0007) [2023-12-27 11:56:24,453][20381] Updated weights for policy 1, policy_version 907623 (0.0008) [2023-12-27 11:56:24,500][20381] Updated weights for policy 1, policy_version 907633 (0.0008) [2023-12-27 11:56:24,601][20347] Updated weights for policy 0, policy_version 908311 (0.0009) [2023-12-27 11:56:24,648][20347] Updated weights for policy 0, policy_version 908321 (0.0009) [2023-12-27 11:56:24,715][20347] Updated weights for policy 0, policy_version 908331 (0.0008) [2023-12-27 11:56:25,193][20381] Updated weights for policy 1, policy_version 907643 (0.0008) [2023-12-27 11:56:25,251][20381] Updated weights for policy 1, policy_version 907653 (0.0008) [2023-12-27 11:56:25,309][20381] Updated weights for policy 1, policy_version 907663 (0.0007) [2023-12-27 11:56:25,323][20347] Updated weights for policy 0, policy_version 908341 (0.0008) [2023-12-27 11:56:25,389][20347] Updated weights for policy 0, policy_version 908351 (0.0007) [2023-12-27 11:56:25,456][20347] Updated weights for policy 0, policy_version 908361 (0.0008) [2023-12-27 11:56:26,022][20381] Updated weights for policy 1, policy_version 907673 (0.0006) [2023-12-27 11:56:26,069][20347] Updated weights for policy 0, policy_version 908371 (0.0007) [2023-12-27 11:56:26,081][20381] Updated weights for policy 1, policy_version 907683 (0.0008) [2023-12-27 11:56:26,124][20347] Updated weights for policy 0, policy_version 908381 (0.0005) [2023-12-27 11:56:26,134][20381] Updated weights for policy 1, policy_version 907693 (0.0008) [2023-12-27 11:56:26,175][20347] Updated weights for policy 0, policy_version 908391 (0.0006) [2023-12-27 11:56:26,188][20381] Updated weights for policy 1, policy_version 907703 (0.0006) [2023-12-27 11:56:26,802][20347] Updated weights for policy 0, policy_version 908401 (0.0007) [2023-12-27 11:56:26,805][20381] Updated weights for policy 1, policy_version 907713 (0.0007) [2023-12-27 11:56:26,848][20347] Updated weights for policy 0, policy_version 908411 (0.0008) [2023-12-27 11:56:26,858][20381] Updated weights for policy 1, policy_version 907723 (0.0006) [2023-12-27 11:56:26,889][20347] Updated weights for policy 0, policy_version 908421 (0.0006) [2023-12-27 11:56:26,909][20381] Updated weights for policy 1, policy_version 907733 (0.0006) [2023-12-27 11:56:26,934][20347] Updated weights for policy 0, policy_version 908431 (0.0006) [2023-12-27 11:56:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 464994304. Throughput: 0: 10203.8, 1: 10198.6. Samples: 464982920. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:27,363][19170] Avg episode reward: [(0, '922.370'), (1, '178.260')] [2023-12-27 11:56:27,464][20381] Updated weights for policy 1, policy_version 907743 (0.0007) [2023-12-27 11:56:27,520][20381] Updated weights for policy 1, policy_version 907753 (0.0008) [2023-12-27 11:56:27,579][20381] Updated weights for policy 1, policy_version 907763 (0.0008) [2023-12-27 11:56:27,652][20347] Updated weights for policy 0, policy_version 908441 (0.0007) [2023-12-27 11:56:27,695][20347] Updated weights for policy 0, policy_version 908451 (0.0007) [2023-12-27 11:56:27,744][20347] Updated weights for policy 0, policy_version 908461 (0.0007) [2023-12-27 11:56:28,251][20381] Updated weights for policy 1, policy_version 907773 (0.0008) [2023-12-27 11:56:28,310][20381] Updated weights for policy 1, policy_version 907783 (0.0008) [2023-12-27 11:56:28,368][20381] Updated weights for policy 1, policy_version 907793 (0.0008) [2023-12-27 11:56:28,433][20347] Updated weights for policy 0, policy_version 908471 (0.0007) [2023-12-27 11:56:28,482][20347] Updated weights for policy 0, policy_version 908481 (0.0007) [2023-12-27 11:56:28,534][20347] Updated weights for policy 0, policy_version 908491 (0.0008) [2023-12-27 11:56:29,065][20381] Updated weights for policy 1, policy_version 907803 (0.0007) [2023-12-27 11:56:29,119][20381] Updated weights for policy 1, policy_version 907813 (0.0008) [2023-12-27 11:56:29,175][20381] Updated weights for policy 1, policy_version 907823 (0.0008) [2023-12-27 11:56:29,255][20347] Updated weights for policy 0, policy_version 908501 (0.0008) [2023-12-27 11:56:29,312][20347] Updated weights for policy 0, policy_version 908511 (0.0010) [2023-12-27 11:56:29,375][20347] Updated weights for policy 0, policy_version 908521 (0.0009) [2023-12-27 11:56:29,912][20381] Updated weights for policy 1, policy_version 907833 (0.0008) [2023-12-27 11:56:29,978][20381] Updated weights for policy 1, policy_version 907843 (0.0008) [2023-12-27 11:56:30,031][20381] Updated weights for policy 1, policy_version 907853 (0.0008) [2023-12-27 11:56:30,034][20347] Updated weights for policy 0, policy_version 908531 (0.0007) [2023-12-27 11:56:30,087][20381] Updated weights for policy 1, policy_version 907863 (0.0008) [2023-12-27 11:56:30,093][20347] Updated weights for policy 0, policy_version 908541 (0.0006) [2023-12-27 11:56:30,155][20347] Updated weights for policy 0, policy_version 908551 (0.0008) [2023-12-27 11:56:30,815][20381] Updated weights for policy 1, policy_version 907873 (0.0007) [2023-12-27 11:56:30,841][20347] Updated weights for policy 0, policy_version 908561 (0.0008) [2023-12-27 11:56:30,876][20381] Updated weights for policy 1, policy_version 907883 (0.0006) [2023-12-27 11:56:30,902][20347] Updated weights for policy 0, policy_version 908571 (0.0010) [2023-12-27 11:56:30,933][20381] Updated weights for policy 1, policy_version 907893 (0.0005) [2023-12-27 11:56:30,960][20347] Updated weights for policy 0, policy_version 908581 (0.0010) [2023-12-27 11:56:31,006][20347] Updated weights for policy 0, policy_version 908591 (0.0010) [2023-12-27 11:56:31,597][20381] Updated weights for policy 1, policy_version 907903 (0.0007) [2023-12-27 11:56:31,664][20381] Updated weights for policy 1, policy_version 907913 (0.0008) [2023-12-27 11:56:31,730][20381] Updated weights for policy 1, policy_version 907923 (0.0007) [2023-12-27 11:56:31,762][20347] Updated weights for policy 0, policy_version 908601 (0.0009) [2023-12-27 11:56:31,820][20347] Updated weights for policy 0, policy_version 908611 (0.0006) [2023-12-27 11:56:31,879][20347] Updated weights for policy 0, policy_version 908621 (0.0009) [2023-12-27 11:56:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 465092608. Throughput: 0: 10272.4, 1: 10120.4. Samples: 465105684. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:32,363][19170] Avg episode reward: [(0, '941.730'), (1, '136.040')] [2023-12-27 11:56:32,459][20381] Updated weights for policy 1, policy_version 907933 (0.0007) [2023-12-27 11:56:32,480][20347] Updated weights for policy 0, policy_version 908631 (0.0009) [2023-12-27 11:56:32,523][20381] Updated weights for policy 1, policy_version 907943 (0.0007) [2023-12-27 11:56:32,537][20347] Updated weights for policy 0, policy_version 908641 (0.0011) [2023-12-27 11:56:32,582][20381] Updated weights for policy 1, policy_version 907953 (0.0007) [2023-12-27 11:56:32,595][20347] Updated weights for policy 0, policy_version 908651 (0.0008) [2023-12-27 11:56:33,158][20347] Updated weights for policy 0, policy_version 908661 (0.0006) [2023-12-27 11:56:33,217][20347] Updated weights for policy 0, policy_version 908671 (0.0005) [2023-12-27 11:56:33,267][20347] Updated weights for policy 0, policy_version 908681 (0.0005) [2023-12-27 11:56:33,288][20381] Updated weights for policy 1, policy_version 907963 (0.0007) [2023-12-27 11:56:33,343][20381] Updated weights for policy 1, policy_version 907973 (0.0007) [2023-12-27 11:56:33,390][20381] Updated weights for policy 1, policy_version 907983 (0.0007) [2023-12-27 11:56:33,876][20347] Updated weights for policy 0, policy_version 908691 (0.0007) [2023-12-27 11:56:33,936][20347] Updated weights for policy 0, policy_version 908701 (0.0008) [2023-12-27 11:56:34,002][20347] Updated weights for policy 0, policy_version 908711 (0.0008) [2023-12-27 11:56:34,020][20381] Updated weights for policy 1, policy_version 907993 (0.0008) [2023-12-27 11:56:34,077][20381] Updated weights for policy 1, policy_version 908003 (0.0006) [2023-12-27 11:56:34,132][20381] Updated weights for policy 1, policy_version 908013 (0.0006) [2023-12-27 11:56:34,197][20381] Updated weights for policy 1, policy_version 908023 (0.0009) [2023-12-27 11:56:34,760][20347] Updated weights for policy 0, policy_version 908721 (0.0009) [2023-12-27 11:56:34,806][20381] Updated weights for policy 1, policy_version 908033 (0.0009) [2023-12-27 11:56:34,821][20347] Updated weights for policy 0, policy_version 908731 (0.0009) [2023-12-27 11:56:34,866][20381] Updated weights for policy 1, policy_version 908043 (0.0005) [2023-12-27 11:56:34,875][20347] Updated weights for policy 0, policy_version 908741 (0.0007) [2023-12-27 11:56:34,921][20381] Updated weights for policy 1, policy_version 908053 (0.0005) [2023-12-27 11:56:34,927][20347] Updated weights for policy 0, policy_version 908751 (0.0008) [2023-12-27 11:56:35,442][20381] Updated weights for policy 1, policy_version 908063 (0.0006) [2023-12-27 11:56:35,497][20381] Updated weights for policy 1, policy_version 908073 (0.0010) [2023-12-27 11:56:35,556][20381] Updated weights for policy 1, policy_version 908083 (0.0010) [2023-12-27 11:56:35,621][20347] Updated weights for policy 0, policy_version 908761 (0.0005) [2023-12-27 11:56:35,675][20347] Updated weights for policy 0, policy_version 908771 (0.0007) [2023-12-27 11:56:35,723][20347] Updated weights for policy 0, policy_version 908781 (0.0006) [2023-12-27 11:56:36,259][20381] Updated weights for policy 1, policy_version 908093 (0.0010) [2023-12-27 11:56:36,319][20381] Updated weights for policy 1, policy_version 908103 (0.0011) [2023-12-27 11:56:36,375][20381] Updated weights for policy 1, policy_version 908113 (0.0011) [2023-12-27 11:56:36,391][20347] Updated weights for policy 0, policy_version 908791 (0.0009) [2023-12-27 11:56:36,453][20347] Updated weights for policy 0, policy_version 908801 (0.0010) [2023-12-27 11:56:36,511][20347] Updated weights for policy 0, policy_version 908811 (0.0010) [2023-12-27 11:56:37,072][20381] Updated weights for policy 1, policy_version 908123 (0.0011) [2023-12-27 11:56:37,132][20381] Updated weights for policy 1, policy_version 908133 (0.0010) [2023-12-27 11:56:37,165][20347] Updated weights for policy 0, policy_version 908821 (0.0009) [2023-12-27 11:56:37,197][20381] Updated weights for policy 1, policy_version 908143 (0.0007) [2023-12-27 11:56:37,217][20347] Updated weights for policy 0, policy_version 908831 (0.0005) [2023-12-27 11:56:37,264][20347] Updated weights for policy 0, policy_version 908841 (0.0006) [2023-12-27 11:56:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 465207296. Throughput: 0: 10285.3, 1: 10178.3. Samples: 465169564. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:37,363][19170] Avg episode reward: [(0, '990.320'), (1, '329.010')] [2023-12-27 11:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000908152_232513536.pth... [2023-12-27 11:56:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000908848_232693760.pth... [2023-12-27 11:56:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000906968_232210432.pth [2023-12-27 11:56:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000907632_232382464.pth [2023-12-27 11:56:37,879][20347] Updated weights for policy 0, policy_version 908851 (0.0006) [2023-12-27 11:56:37,899][20381] Updated weights for policy 1, policy_version 908153 (0.0010) [2023-12-27 11:56:37,928][20347] Updated weights for policy 0, policy_version 908861 (0.0010) [2023-12-27 11:56:37,956][20381] Updated weights for policy 1, policy_version 908163 (0.0006) [2023-12-27 11:56:37,975][20347] Updated weights for policy 0, policy_version 908871 (0.0010) [2023-12-27 11:56:38,010][20381] Updated weights for policy 1, policy_version 908173 (0.0005) [2023-12-27 11:56:38,059][20381] Updated weights for policy 1, policy_version 908183 (0.0005) [2023-12-27 11:56:38,663][20347] Updated weights for policy 0, policy_version 908881 (0.0009) [2023-12-27 11:56:38,718][20347] Updated weights for policy 0, policy_version 908891 (0.0007) [2023-12-27 11:56:38,768][20381] Updated weights for policy 1, policy_version 908193 (0.0007) [2023-12-27 11:56:38,774][20347] Updated weights for policy 0, policy_version 908901 (0.0006) [2023-12-27 11:56:38,823][20381] Updated weights for policy 1, policy_version 908203 (0.0006) [2023-12-27 11:56:38,828][20347] Updated weights for policy 0, policy_version 908911 (0.0006) [2023-12-27 11:56:38,877][20381] Updated weights for policy 1, policy_version 908213 (0.0007) [2023-12-27 11:56:39,532][20347] Updated weights for policy 0, policy_version 908921 (0.0008) [2023-12-27 11:56:39,584][20347] Updated weights for policy 0, policy_version 908931 (0.0007) [2023-12-27 11:56:39,600][20381] Updated weights for policy 1, policy_version 908223 (0.0008) [2023-12-27 11:56:39,645][20347] Updated weights for policy 0, policy_version 908941 (0.0006) [2023-12-27 11:56:39,661][20381] Updated weights for policy 1, policy_version 908233 (0.0006) [2023-12-27 11:56:39,725][20381] Updated weights for policy 1, policy_version 908243 (0.0006) [2023-12-27 11:56:40,370][20347] Updated weights for policy 0, policy_version 908951 (0.0007) [2023-12-27 11:56:40,404][20381] Updated weights for policy 1, policy_version 908253 (0.0007) [2023-12-27 11:56:40,427][20347] Updated weights for policy 0, policy_version 908961 (0.0008) [2023-12-27 11:56:40,462][20381] Updated weights for policy 1, policy_version 908263 (0.0008) [2023-12-27 11:56:40,479][20347] Updated weights for policy 0, policy_version 908971 (0.0008) [2023-12-27 11:56:40,519][20381] Updated weights for policy 1, policy_version 908273 (0.0006) [2023-12-27 11:56:41,178][20381] Updated weights for policy 1, policy_version 908283 (0.0007) [2023-12-27 11:56:41,231][20381] Updated weights for policy 1, policy_version 908293 (0.0009) [2023-12-27 11:56:41,259][20347] Updated weights for policy 0, policy_version 908981 (0.0007) [2023-12-27 11:56:41,288][20381] Updated weights for policy 1, policy_version 908303 (0.0008) [2023-12-27 11:56:41,318][20347] Updated weights for policy 0, policy_version 908991 (0.0007) [2023-12-27 11:56:41,385][20347] Updated weights for policy 0, policy_version 909001 (0.0008) [2023-12-27 11:56:42,113][20381] Updated weights for policy 1, policy_version 908313 (0.0008) [2023-12-27 11:56:42,137][20347] Updated weights for policy 0, policy_version 909011 (0.0012) [2023-12-27 11:56:42,170][20381] Updated weights for policy 1, policy_version 908323 (0.0005) [2023-12-27 11:56:42,194][20347] Updated weights for policy 0, policy_version 909021 (0.0010) [2023-12-27 11:56:42,227][20381] Updated weights for policy 1, policy_version 908333 (0.0006) [2023-12-27 11:56:42,252][20347] Updated weights for policy 0, policy_version 909031 (0.0011) [2023-12-27 11:56:42,291][20381] Updated weights for policy 1, policy_version 908343 (0.0008) [2023-12-27 11:56:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 465305600. Throughput: 0: 10194.9, 1: 10138.9. Samples: 465291684. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:42,363][19170] Avg episode reward: [(0, '990.320'), (1, '188.730')] [2023-12-27 11:56:43,005][20347] Updated weights for policy 0, policy_version 909041 (0.0009) [2023-12-27 11:56:43,012][20381] Updated weights for policy 1, policy_version 908353 (0.0008) [2023-12-27 11:56:43,058][20347] Updated weights for policy 0, policy_version 909051 (0.0005) [2023-12-27 11:56:43,075][20381] Updated weights for policy 1, policy_version 908363 (0.0008) [2023-12-27 11:56:43,110][20347] Updated weights for policy 0, policy_version 909061 (0.0005) [2023-12-27 11:56:43,139][20381] Updated weights for policy 1, policy_version 908373 (0.0008) [2023-12-27 11:56:43,163][20347] Updated weights for policy 0, policy_version 909071 (0.0005) [2023-12-27 11:56:43,704][20347] Updated weights for policy 0, policy_version 909081 (0.0005) [2023-12-27 11:56:43,751][20347] Updated weights for policy 0, policy_version 909091 (0.0006) [2023-12-27 11:56:43,801][20347] Updated weights for policy 0, policy_version 909101 (0.0008) [2023-12-27 11:56:43,885][20381] Updated weights for policy 1, policy_version 908383 (0.0008) [2023-12-27 11:56:43,940][20381] Updated weights for policy 1, policy_version 908393 (0.0008) [2023-12-27 11:56:43,989][20381] Updated weights for policy 1, policy_version 908403 (0.0008) [2023-12-27 11:56:44,449][20347] Updated weights for policy 0, policy_version 909111 (0.0007) [2023-12-27 11:56:44,514][20347] Updated weights for policy 0, policy_version 909121 (0.0008) [2023-12-27 11:56:44,563][20381] Updated weights for policy 1, policy_version 908413 (0.0006) [2023-12-27 11:56:44,572][20347] Updated weights for policy 0, policy_version 909131 (0.0006) [2023-12-27 11:56:44,611][20381] Updated weights for policy 1, policy_version 908423 (0.0008) [2023-12-27 11:56:44,659][20381] Updated weights for policy 1, policy_version 908433 (0.0010) [2023-12-27 11:56:45,288][20347] Updated weights for policy 0, policy_version 909141 (0.0007) [2023-12-27 11:56:45,304][20291] KL-divergence is very high: 108.5141 [2023-12-27 11:56:45,345][20347] Updated weights for policy 0, policy_version 909151 (0.0007) [2023-12-27 11:56:45,365][20381] Updated weights for policy 1, policy_version 908443 (0.0010) [2023-12-27 11:56:45,400][20347] Updated weights for policy 0, policy_version 909161 (0.0006) [2023-12-27 11:56:45,416][20381] Updated weights for policy 1, policy_version 908453 (0.0010) [2023-12-27 11:56:45,461][20381] Updated weights for policy 1, policy_version 908463 (0.0010) [2023-12-27 11:56:46,077][20347] Updated weights for policy 0, policy_version 909171 (0.0006) [2023-12-27 11:56:46,130][20347] Updated weights for policy 0, policy_version 909181 (0.0008) [2023-12-27 11:56:46,132][20381] Updated weights for policy 1, policy_version 908473 (0.0010) [2023-12-27 11:56:46,187][20347] Updated weights for policy 0, policy_version 909191 (0.0006) [2023-12-27 11:56:46,193][20381] Updated weights for policy 1, policy_version 908483 (0.0006) [2023-12-27 11:56:46,248][20381] Updated weights for policy 1, policy_version 908493 (0.0008) [2023-12-27 11:56:46,305][20381] Updated weights for policy 1, policy_version 908503 (0.0008) [2023-12-27 11:56:46,915][20381] Updated weights for policy 1, policy_version 908513 (0.0006) [2023-12-27 11:56:46,933][20347] Updated weights for policy 0, policy_version 909201 (0.0006) [2023-12-27 11:56:46,971][20381] Updated weights for policy 1, policy_version 908523 (0.0005) [2023-12-27 11:56:46,986][20347] Updated weights for policy 0, policy_version 909211 (0.0008) [2023-12-27 11:56:47,027][20381] Updated weights for policy 1, policy_version 908533 (0.0005) [2023-12-27 11:56:47,036][20347] Updated weights for policy 0, policy_version 909221 (0.0007) [2023-12-27 11:56:47,084][20347] Updated weights for policy 0, policy_version 909231 (0.0009) [2023-12-27 11:56:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 465403904. Throughput: 0: 10200.3, 1: 10138.4. Samples: 465413172. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:47,363][19170] Avg episode reward: [(0, '990.260'), (1, '221.460')] [2023-12-27 11:56:47,603][20381] Updated weights for policy 1, policy_version 908543 (0.0007) [2023-12-27 11:56:47,650][20381] Updated weights for policy 1, policy_version 908553 (0.0007) [2023-12-27 11:56:47,705][20381] Updated weights for policy 1, policy_version 908563 (0.0008) [2023-12-27 11:56:47,736][20347] Updated weights for policy 0, policy_version 909241 (0.0006) [2023-12-27 11:56:47,789][20347] Updated weights for policy 0, policy_version 909251 (0.0005) [2023-12-27 11:56:47,842][20347] Updated weights for policy 0, policy_version 909261 (0.0005) [2023-12-27 11:56:48,407][20381] Updated weights for policy 1, policy_version 908573 (0.0007) [2023-12-27 11:56:48,430][20347] Updated weights for policy 0, policy_version 909271 (0.0008) [2023-12-27 11:56:48,462][20381] Updated weights for policy 1, policy_version 908583 (0.0007) [2023-12-27 11:56:48,483][20347] Updated weights for policy 0, policy_version 909281 (0.0007) [2023-12-27 11:56:48,519][20381] Updated weights for policy 1, policy_version 908593 (0.0006) [2023-12-27 11:56:48,536][20347] Updated weights for policy 0, policy_version 909291 (0.0006) [2023-12-27 11:56:49,223][20347] Updated weights for policy 0, policy_version 909301 (0.0006) [2023-12-27 11:56:49,235][20381] Updated weights for policy 1, policy_version 908603 (0.0007) [2023-12-27 11:56:49,287][20347] Updated weights for policy 0, policy_version 909311 (0.0006) [2023-12-27 11:56:49,299][20381] Updated weights for policy 1, policy_version 908613 (0.0007) [2023-12-27 11:56:49,339][20347] Updated weights for policy 0, policy_version 909321 (0.0007) [2023-12-27 11:56:49,369][20381] Updated weights for policy 1, policy_version 908623 (0.0008) [2023-12-27 11:56:50,060][20381] Updated weights for policy 1, policy_version 908633 (0.0006) [2023-12-27 11:56:50,069][20347] Updated weights for policy 0, policy_version 909331 (0.0007) [2023-12-27 11:56:50,119][20381] Updated weights for policy 1, policy_version 908643 (0.0008) [2023-12-27 11:56:50,131][20347] Updated weights for policy 0, policy_version 909341 (0.0008) [2023-12-27 11:56:50,175][20381] Updated weights for policy 1, policy_version 908653 (0.0010) [2023-12-27 11:56:50,186][20347] Updated weights for policy 0, policy_version 909351 (0.0010) [2023-12-27 11:56:50,228][20381] Updated weights for policy 1, policy_version 908663 (0.0010) [2023-12-27 11:56:50,892][20381] Updated weights for policy 1, policy_version 908673 (0.0011) [2023-12-27 11:56:50,903][20347] Updated weights for policy 0, policy_version 909361 (0.0010) [2023-12-27 11:56:50,944][20381] Updated weights for policy 1, policy_version 908683 (0.0010) [2023-12-27 11:56:50,961][20347] Updated weights for policy 0, policy_version 909371 (0.0006) [2023-12-27 11:56:50,997][20381] Updated weights for policy 1, policy_version 908693 (0.0011) [2023-12-27 11:56:51,018][20347] Updated weights for policy 0, policy_version 909381 (0.0006) [2023-12-27 11:56:51,083][20347] Updated weights for policy 0, policy_version 909391 (0.0008) [2023-12-27 11:56:51,761][20381] Updated weights for policy 1, policy_version 908703 (0.0009) [2023-12-27 11:56:51,820][20381] Updated weights for policy 1, policy_version 908713 (0.0008) [2023-12-27 11:56:51,835][20347] Updated weights for policy 0, policy_version 909401 (0.0010) [2023-12-27 11:56:51,877][20381] Updated weights for policy 1, policy_version 908723 (0.0008) [2023-12-27 11:56:51,899][20347] Updated weights for policy 0, policy_version 909411 (0.0011) [2023-12-27 11:56:51,964][20347] Updated weights for policy 0, policy_version 909421 (0.0009) [2023-12-27 11:56:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 465502208. Throughput: 0: 10228.5, 1: 10110.7. Samples: 465474796. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:52,363][19170] Avg episode reward: [(0, '970.870'), (1, '133.190')] [2023-12-27 11:56:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000909424_232841216.pth... [2023-12-27 11:56:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000908728_232660992.pth... [2023-12-27 11:56:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000907544_232357888.pth [2023-12-27 11:56:52,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000908240_232538112.pth [2023-12-27 11:56:52,593][20381] Updated weights for policy 1, policy_version 908733 (0.0008) [2023-12-27 11:56:52,633][20347] Updated weights for policy 0, policy_version 909431 (0.0007) [2023-12-27 11:56:52,643][20381] Updated weights for policy 1, policy_version 908743 (0.0005) [2023-12-27 11:56:52,687][20347] Updated weights for policy 0, policy_version 909441 (0.0005) [2023-12-27 11:56:52,700][20381] Updated weights for policy 1, policy_version 908753 (0.0006) [2023-12-27 11:56:52,744][20347] Updated weights for policy 0, policy_version 909451 (0.0006) [2023-12-27 11:56:53,282][20381] Updated weights for policy 1, policy_version 908763 (0.0006) [2023-12-27 11:56:53,338][20381] Updated weights for policy 1, policy_version 908773 (0.0006) [2023-12-27 11:56:53,396][20381] Updated weights for policy 1, policy_version 908783 (0.0008) [2023-12-27 11:56:53,443][20347] Updated weights for policy 0, policy_version 909461 (0.0007) [2023-12-27 11:56:53,502][20347] Updated weights for policy 0, policy_version 909471 (0.0008) [2023-12-27 11:56:53,554][20347] Updated weights for policy 0, policy_version 909481 (0.0008) [2023-12-27 11:56:54,004][20381] Updated weights for policy 1, policy_version 908793 (0.0007) [2023-12-27 11:56:54,063][20381] Updated weights for policy 1, policy_version 908803 (0.0011) [2023-12-27 11:56:54,121][20381] Updated weights for policy 1, policy_version 908813 (0.0010) [2023-12-27 11:56:54,174][20381] Updated weights for policy 1, policy_version 908823 (0.0010) [2023-12-27 11:56:54,187][20347] Updated weights for policy 0, policy_version 909491 (0.0008) [2023-12-27 11:56:54,242][20347] Updated weights for policy 0, policy_version 909501 (0.0010) [2023-12-27 11:56:54,299][20347] Updated weights for policy 0, policy_version 909511 (0.0008) [2023-12-27 11:56:54,838][20381] Updated weights for policy 1, policy_version 908833 (0.0010) [2023-12-27 11:56:54,886][20381] Updated weights for policy 1, policy_version 908843 (0.0010) [2023-12-27 11:56:54,930][20347] Updated weights for policy 0, policy_version 909521 (0.0008) [2023-12-27 11:56:54,941][20381] Updated weights for policy 1, policy_version 908853 (0.0011) [2023-12-27 11:56:54,982][20347] Updated weights for policy 0, policy_version 909531 (0.0007) [2023-12-27 11:56:55,028][20347] Updated weights for policy 0, policy_version 909541 (0.0007) [2023-12-27 11:56:55,077][20347] Updated weights for policy 0, policy_version 909551 (0.0008) [2023-12-27 11:56:55,645][20381] Updated weights for policy 1, policy_version 908863 (0.0010) [2023-12-27 11:56:55,703][20381] Updated weights for policy 1, policy_version 908873 (0.0010) [2023-12-27 11:56:55,759][20381] Updated weights for policy 1, policy_version 908883 (0.0011) [2023-12-27 11:56:55,789][20347] Updated weights for policy 0, policy_version 909561 (0.0005) [2023-12-27 11:56:55,836][20347] Updated weights for policy 0, policy_version 909571 (0.0007) [2023-12-27 11:56:55,885][20347] Updated weights for policy 0, policy_version 909581 (0.0005) [2023-12-27 11:56:56,409][20381] Updated weights for policy 1, policy_version 908893 (0.0010) [2023-12-27 11:56:56,462][20381] Updated weights for policy 1, policy_version 908903 (0.0010) [2023-12-27 11:56:56,526][20381] Updated weights for policy 1, policy_version 908913 (0.0006) [2023-12-27 11:56:56,558][20347] Updated weights for policy 0, policy_version 909591 (0.0007) [2023-12-27 11:56:56,602][20347] Updated weights for policy 0, policy_version 909601 (0.0008) [2023-12-27 11:56:56,653][20347] Updated weights for policy 0, policy_version 909611 (0.0008) [2023-12-27 11:56:57,087][20381] Updated weights for policy 1, policy_version 908923 (0.0006) [2023-12-27 11:56:57,154][20381] Updated weights for policy 1, policy_version 908933 (0.0008) [2023-12-27 11:56:57,214][20381] Updated weights for policy 1, policy_version 908943 (0.0009) [2023-12-27 11:56:57,265][20347] Updated weights for policy 0, policy_version 909621 (0.0006) [2023-12-27 11:56:57,323][20347] Updated weights for policy 0, policy_version 909631 (0.0007) [2023-12-27 11:56:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 465608704. Throughput: 0: 10192.1, 1: 10128.9. Samples: 465596452. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:56:57,363][19170] Avg episode reward: [(0, '970.960'), (1, '238.640')] [2023-12-27 11:56:57,384][20347] Updated weights for policy 0, policy_version 909641 (0.0006) [2023-12-27 11:56:57,856][20381] Updated weights for policy 1, policy_version 908953 (0.0006) [2023-12-27 11:56:57,907][20381] Updated weights for policy 1, policy_version 908963 (0.0005) [2023-12-27 11:56:57,966][20381] Updated weights for policy 1, policy_version 908973 (0.0005) [2023-12-27 11:56:58,022][20381] Updated weights for policy 1, policy_version 908983 (0.0007) [2023-12-27 11:56:58,060][20347] Updated weights for policy 0, policy_version 909651 (0.0007) [2023-12-27 11:56:58,111][20347] Updated weights for policy 0, policy_version 909661 (0.0007) [2023-12-27 11:56:58,168][20347] Updated weights for policy 0, policy_version 909671 (0.0007) [2023-12-27 11:56:58,760][20381] Updated weights for policy 1, policy_version 908993 (0.0008) [2023-12-27 11:56:58,827][20381] Updated weights for policy 1, policy_version 909003 (0.0007) [2023-12-27 11:56:58,894][20381] Updated weights for policy 1, policy_version 909013 (0.0007) [2023-12-27 11:56:59,018][20347] Updated weights for policy 0, policy_version 909681 (0.0008) [2023-12-27 11:56:59,075][20347] Updated weights for policy 0, policy_version 909691 (0.0008) [2023-12-27 11:56:59,126][20347] Updated weights for policy 0, policy_version 909701 (0.0007) [2023-12-27 11:56:59,170][20347] Updated weights for policy 0, policy_version 909711 (0.0007) [2023-12-27 11:56:59,675][20381] Updated weights for policy 1, policy_version 909023 (0.0008) [2023-12-27 11:56:59,729][20381] Updated weights for policy 1, policy_version 909033 (0.0007) [2023-12-27 11:56:59,791][20381] Updated weights for policy 1, policy_version 909043 (0.0008) [2023-12-27 11:56:59,921][20347] Updated weights for policy 0, policy_version 909721 (0.0008) [2023-12-27 11:56:59,984][20347] Updated weights for policy 0, policy_version 909731 (0.0008) [2023-12-27 11:57:00,042][20347] Updated weights for policy 0, policy_version 909741 (0.0007) [2023-12-27 11:57:00,516][20381] Updated weights for policy 1, policy_version 909053 (0.0007) [2023-12-27 11:57:00,564][20381] Updated weights for policy 1, policy_version 909063 (0.0005) [2023-12-27 11:57:00,614][20381] Updated weights for policy 1, policy_version 909073 (0.0007) [2023-12-27 11:57:00,678][20347] Updated weights for policy 0, policy_version 909751 (0.0009) [2023-12-27 11:57:00,733][20347] Updated weights for policy 0, policy_version 909761 (0.0009) [2023-12-27 11:57:00,784][20347] Updated weights for policy 0, policy_version 909771 (0.0008) [2023-12-27 11:57:01,233][20381] Updated weights for policy 1, policy_version 909083 (0.0006) [2023-12-27 11:57:01,295][20381] Updated weights for policy 1, policy_version 909093 (0.0006) [2023-12-27 11:57:01,361][20381] Updated weights for policy 1, policy_version 909103 (0.0007) [2023-12-27 11:57:01,519][20347] Updated weights for policy 0, policy_version 909781 (0.0008) [2023-12-27 11:57:01,578][20347] Updated weights for policy 0, policy_version 909791 (0.0008) [2023-12-27 11:57:01,643][20347] Updated weights for policy 0, policy_version 909801 (0.0008) [2023-12-27 11:57:02,083][20381] Updated weights for policy 1, policy_version 909113 (0.0007) [2023-12-27 11:57:02,141][20381] Updated weights for policy 1, policy_version 909123 (0.0006) [2023-12-27 11:57:02,201][20381] Updated weights for policy 1, policy_version 909133 (0.0007) [2023-12-27 11:57:02,224][20347] Updated weights for policy 0, policy_version 909811 (0.0006) [2023-12-27 11:57:02,263][20381] Updated weights for policy 1, policy_version 909143 (0.0007) [2023-12-27 11:57:02,289][20347] Updated weights for policy 0, policy_version 909821 (0.0007) [2023-12-27 11:57:02,338][20347] Updated weights for policy 0, policy_version 909831 (0.0008) [2023-12-27 11:57:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 465707008. Throughput: 0: 10193.0, 1: 10122.9. Samples: 465718012. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:02,363][19170] Avg episode reward: [(0, '990.350'), (1, '224.080')] [2023-12-27 11:57:02,914][20381] Updated weights for policy 1, policy_version 909153 (0.0006) [2023-12-27 11:57:02,969][20381] Updated weights for policy 1, policy_version 909163 (0.0006) [2023-12-27 11:57:03,013][20381] Updated weights for policy 1, policy_version 909173 (0.0005) [2023-12-27 11:57:03,048][20347] Updated weights for policy 0, policy_version 909841 (0.0008) [2023-12-27 11:57:03,103][20347] Updated weights for policy 0, policy_version 909851 (0.0007) [2023-12-27 11:57:03,153][20347] Updated weights for policy 0, policy_version 909861 (0.0008) [2023-12-27 11:57:03,204][20347] Updated weights for policy 0, policy_version 909871 (0.0007) [2023-12-27 11:57:03,605][20381] Updated weights for policy 1, policy_version 909183 (0.0009) [2023-12-27 11:57:03,659][20381] Updated weights for policy 1, policy_version 909193 (0.0010) [2023-12-27 11:57:03,710][20381] Updated weights for policy 1, policy_version 909203 (0.0010) [2023-12-27 11:57:03,944][20347] Updated weights for policy 0, policy_version 909881 (0.0007) [2023-12-27 11:57:04,001][20347] Updated weights for policy 0, policy_version 909891 (0.0007) [2023-12-27 11:57:04,063][20347] Updated weights for policy 0, policy_version 909901 (0.0008) [2023-12-27 11:57:04,375][20381] Updated weights for policy 1, policy_version 909213 (0.0008) [2023-12-27 11:57:04,438][20381] Updated weights for policy 1, policy_version 909223 (0.0007) [2023-12-27 11:57:04,491][20381] Updated weights for policy 1, policy_version 909233 (0.0009) [2023-12-27 11:57:04,759][20347] Updated weights for policy 0, policy_version 909911 (0.0008) [2023-12-27 11:57:04,821][20347] Updated weights for policy 0, policy_version 909921 (0.0007) [2023-12-27 11:57:04,881][20347] Updated weights for policy 0, policy_version 909931 (0.0008) [2023-12-27 11:57:05,055][20381] Updated weights for policy 1, policy_version 909243 (0.0008) [2023-12-27 11:57:05,109][20381] Updated weights for policy 1, policy_version 909253 (0.0009) [2023-12-27 11:57:05,162][20381] Updated weights for policy 1, policy_version 909263 (0.0009) [2023-12-27 11:57:05,600][20347] Updated weights for policy 0, policy_version 909941 (0.0007) [2023-12-27 11:57:05,651][20347] Updated weights for policy 0, policy_version 909951 (0.0007) [2023-12-27 11:57:05,709][20347] Updated weights for policy 0, policy_version 909961 (0.0008) [2023-12-27 11:57:05,747][20381] Updated weights for policy 1, policy_version 909273 (0.0010) [2023-12-27 11:57:05,805][20381] Updated weights for policy 1, policy_version 909283 (0.0007) [2023-12-27 11:57:05,857][20381] Updated weights for policy 1, policy_version 909293 (0.0008) [2023-12-27 11:57:05,915][20381] Updated weights for policy 1, policy_version 909303 (0.0008) [2023-12-27 11:57:06,429][20347] Updated weights for policy 0, policy_version 909971 (0.0007) [2023-12-27 11:57:06,488][20347] Updated weights for policy 0, policy_version 909981 (0.0008) [2023-12-27 11:57:06,550][20347] Updated weights for policy 0, policy_version 909991 (0.0007) [2023-12-27 11:57:06,624][20381] Updated weights for policy 1, policy_version 909313 (0.0007) [2023-12-27 11:57:06,685][20381] Updated weights for policy 1, policy_version 909323 (0.0008) [2023-12-27 11:57:06,739][20381] Updated weights for policy 1, policy_version 909333 (0.0008) [2023-12-27 11:57:07,279][20347] Updated weights for policy 0, policy_version 910001 (0.0007) [2023-12-27 11:57:07,340][20347] Updated weights for policy 0, policy_version 910011 (0.0007) [2023-12-27 11:57:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 465805312. Throughput: 0: 10199.9, 1: 10202.8. Samples: 465780996. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '273.850')] [2023-12-27 11:57:07,365][20381] Updated weights for policy 1, policy_version 909343 (0.0007) [2023-12-27 11:57:07,398][20347] Updated weights for policy 0, policy_version 910021 (0.0006) [2023-12-27 11:57:07,427][20381] Updated weights for policy 1, policy_version 909353 (0.0006) [2023-12-27 11:57:07,456][20347] Updated weights for policy 0, policy_version 910031 (0.0005) [2023-12-27 11:57:07,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000910032_232996864.pth... [2023-12-27 11:57:07,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000908848_232693760.pth [2023-12-27 11:57:07,486][20381] Updated weights for policy 1, policy_version 909363 (0.0007) [2023-12-27 11:57:07,514][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000909368_232824832.pth... [2023-12-27 11:57:07,517][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000908152_232513536.pth [2023-12-27 11:57:08,150][20347] Updated weights for policy 0, policy_version 910041 (0.0007) [2023-12-27 11:57:08,185][20381] Updated weights for policy 1, policy_version 909373 (0.0006) [2023-12-27 11:57:08,206][20347] Updated weights for policy 0, policy_version 910051 (0.0006) [2023-12-27 11:57:08,241][20381] Updated weights for policy 1, policy_version 909383 (0.0005) [2023-12-27 11:57:08,258][20347] Updated weights for policy 0, policy_version 910061 (0.0007) [2023-12-27 11:57:08,298][20381] Updated weights for policy 1, policy_version 909393 (0.0006) [2023-12-27 11:57:08,960][20347] Updated weights for policy 0, policy_version 910071 (0.0007) [2023-12-27 11:57:08,994][20381] Updated weights for policy 1, policy_version 909403 (0.0009) [2023-12-27 11:57:09,015][20347] Updated weights for policy 0, policy_version 910081 (0.0008) [2023-12-27 11:57:09,047][20381] Updated weights for policy 1, policy_version 909413 (0.0007) [2023-12-27 11:57:09,073][20347] Updated weights for policy 0, policy_version 910091 (0.0007) [2023-12-27 11:57:09,111][20381] Updated weights for policy 1, policy_version 909423 (0.0008) [2023-12-27 11:57:09,795][20347] Updated weights for policy 0, policy_version 910101 (0.0008) [2023-12-27 11:57:09,804][20381] Updated weights for policy 1, policy_version 909433 (0.0009) [2023-12-27 11:57:09,858][20347] Updated weights for policy 0, policy_version 910111 (0.0007) [2023-12-27 11:57:09,869][20381] Updated weights for policy 1, policy_version 909443 (0.0009) [2023-12-27 11:57:09,923][20347] Updated weights for policy 0, policy_version 910121 (0.0008) [2023-12-27 11:57:09,930][20381] Updated weights for policy 1, policy_version 909453 (0.0008) [2023-12-27 11:57:09,986][20381] Updated weights for policy 1, policy_version 909463 (0.0007) [2023-12-27 11:57:10,603][20347] Updated weights for policy 0, policy_version 910131 (0.0007) [2023-12-27 11:57:10,650][20381] Updated weights for policy 1, policy_version 909473 (0.0008) [2023-12-27 11:57:10,667][20347] Updated weights for policy 0, policy_version 910141 (0.0006) [2023-12-27 11:57:10,710][20381] Updated weights for policy 1, policy_version 909483 (0.0007) [2023-12-27 11:57:10,728][20347] Updated weights for policy 0, policy_version 910151 (0.0005) [2023-12-27 11:57:10,769][20381] Updated weights for policy 1, policy_version 909493 (0.0011) [2023-12-27 11:57:11,423][20347] Updated weights for policy 0, policy_version 910161 (0.0008) [2023-12-27 11:57:11,474][20347] Updated weights for policy 0, policy_version 910171 (0.0007) [2023-12-27 11:57:11,479][20381] Updated weights for policy 1, policy_version 909503 (0.0008) [2023-12-27 11:57:11,527][20347] Updated weights for policy 0, policy_version 910181 (0.0006) [2023-12-27 11:57:11,537][20381] Updated weights for policy 1, policy_version 909513 (0.0006) [2023-12-27 11:57:11,582][20347] Updated weights for policy 0, policy_version 910191 (0.0006) [2023-12-27 11:57:11,598][20381] Updated weights for policy 1, policy_version 909523 (0.0007) [2023-12-27 11:57:12,333][20381] Updated weights for policy 1, policy_version 909533 (0.0010) [2023-12-27 11:57:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 465903616. Throughput: 0: 10123.9, 1: 10285.0. Samples: 465901320. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '262.980')] [2023-12-27 11:57:12,398][20381] Updated weights for policy 1, policy_version 909543 (0.0009) [2023-12-27 11:57:12,402][20347] Updated weights for policy 0, policy_version 910201 (0.0008) [2023-12-27 11:57:12,453][20381] Updated weights for policy 1, policy_version 909553 (0.0007) [2023-12-27 11:57:12,455][20347] Updated weights for policy 0, policy_version 910211 (0.0008) [2023-12-27 11:57:12,503][20347] Updated weights for policy 0, policy_version 910221 (0.0008) [2023-12-27 11:57:13,201][20381] Updated weights for policy 1, policy_version 909563 (0.0008) [2023-12-27 11:57:13,223][20347] Updated weights for policy 0, policy_version 910231 (0.0006) [2023-12-27 11:57:13,261][20381] Updated weights for policy 1, policy_version 909573 (0.0008) [2023-12-27 11:57:13,277][20347] Updated weights for policy 0, policy_version 910241 (0.0008) [2023-12-27 11:57:13,324][20381] Updated weights for policy 1, policy_version 909583 (0.0008) [2023-12-27 11:57:13,331][20347] Updated weights for policy 0, policy_version 910251 (0.0005) [2023-12-27 11:57:13,966][20347] Updated weights for policy 0, policy_version 910261 (0.0007) [2023-12-27 11:57:14,019][20347] Updated weights for policy 0, policy_version 910271 (0.0008) [2023-12-27 11:57:14,042][20381] Updated weights for policy 1, policy_version 909593 (0.0008) [2023-12-27 11:57:14,070][20347] Updated weights for policy 0, policy_version 910281 (0.0007) [2023-12-27 11:57:14,096][20381] Updated weights for policy 1, policy_version 909603 (0.0005) [2023-12-27 11:57:14,156][20381] Updated weights for policy 1, policy_version 909613 (0.0005) [2023-12-27 11:57:14,205][20381] Updated weights for policy 1, policy_version 909623 (0.0005) [2023-12-27 11:57:14,693][20347] Updated weights for policy 0, policy_version 910291 (0.0007) [2023-12-27 11:57:14,754][20347] Updated weights for policy 0, policy_version 910301 (0.0009) [2023-12-27 11:57:14,801][20381] Updated weights for policy 1, policy_version 909633 (0.0008) [2023-12-27 11:57:14,810][20347] Updated weights for policy 0, policy_version 910311 (0.0008) [2023-12-27 11:57:14,856][20381] Updated weights for policy 1, policy_version 909643 (0.0006) [2023-12-27 11:57:14,915][20381] Updated weights for policy 1, policy_version 909653 (0.0009) [2023-12-27 11:57:15,522][20347] Updated weights for policy 0, policy_version 910321 (0.0006) [2023-12-27 11:57:15,553][20381] Updated weights for policy 1, policy_version 909663 (0.0007) [2023-12-27 11:57:15,583][20347] Updated weights for policy 0, policy_version 910331 (0.0007) [2023-12-27 11:57:15,611][20381] Updated weights for policy 1, policy_version 909673 (0.0005) [2023-12-27 11:57:15,637][20347] Updated weights for policy 0, policy_version 910341 (0.0006) [2023-12-27 11:57:15,665][20381] Updated weights for policy 1, policy_version 909683 (0.0006) [2023-12-27 11:57:15,692][20347] Updated weights for policy 0, policy_version 910351 (0.0007) [2023-12-27 11:57:16,283][20381] Updated weights for policy 1, policy_version 909693 (0.0007) [2023-12-27 11:57:16,336][20381] Updated weights for policy 1, policy_version 909703 (0.0007) [2023-12-27 11:57:16,394][20381] Updated weights for policy 1, policy_version 909713 (0.0007) [2023-12-27 11:57:16,417][20347] Updated weights for policy 0, policy_version 910361 (0.0010) [2023-12-27 11:57:16,471][20347] Updated weights for policy 0, policy_version 910371 (0.0008) [2023-12-27 11:57:16,526][20347] Updated weights for policy 0, policy_version 910381 (0.0008) [2023-12-27 11:57:17,060][20381] Updated weights for policy 1, policy_version 909723 (0.0008) [2023-12-27 11:57:17,123][20381] Updated weights for policy 1, policy_version 909733 (0.0005) [2023-12-27 11:57:17,176][20381] Updated weights for policy 1, policy_version 909743 (0.0005) [2023-12-27 11:57:17,250][20347] Updated weights for policy 0, policy_version 910391 (0.0007) [2023-12-27 11:57:17,298][20347] Updated weights for policy 0, policy_version 910401 (0.0009) [2023-12-27 11:57:17,348][20347] Updated weights for policy 0, policy_version 910411 (0.0009) [2023-12-27 11:57:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 466010112. Throughput: 0: 10089.3, 1: 10272.6. Samples: 466021972. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '255.910')] [2023-12-27 11:57:17,721][20381] Updated weights for policy 1, policy_version 909753 (0.0005) [2023-12-27 11:57:17,766][20381] Updated weights for policy 1, policy_version 909763 (0.0005) [2023-12-27 11:57:17,812][20381] Updated weights for policy 1, policy_version 909773 (0.0005) [2023-12-27 11:57:17,855][20381] Updated weights for policy 1, policy_version 909783 (0.0005) [2023-12-27 11:57:18,082][20347] Updated weights for policy 0, policy_version 910421 (0.0008) [2023-12-27 11:57:18,129][20347] Updated weights for policy 0, policy_version 910431 (0.0007) [2023-12-27 11:57:18,175][20347] Updated weights for policy 0, policy_version 910441 (0.0007) [2023-12-27 11:57:18,526][20381] Updated weights for policy 1, policy_version 909793 (0.0007) [2023-12-27 11:57:18,581][20381] Updated weights for policy 1, policy_version 909803 (0.0008) [2023-12-27 11:57:18,644][20381] Updated weights for policy 1, policy_version 909813 (0.0008) [2023-12-27 11:57:18,921][20347] Updated weights for policy 0, policy_version 910451 (0.0008) [2023-12-27 11:57:18,969][20347] Updated weights for policy 0, policy_version 910461 (0.0007) [2023-12-27 11:57:19,013][20347] Updated weights for policy 0, policy_version 910471 (0.0005) [2023-12-27 11:57:19,366][20381] Updated weights for policy 1, policy_version 909823 (0.0008) [2023-12-27 11:57:19,425][20381] Updated weights for policy 1, policy_version 909833 (0.0007) [2023-12-27 11:57:19,483][20381] Updated weights for policy 1, policy_version 909843 (0.0007) [2023-12-27 11:57:19,652][20347] Updated weights for policy 0, policy_version 910481 (0.0006) [2023-12-27 11:57:19,724][20347] Updated weights for policy 0, policy_version 910491 (0.0009) [2023-12-27 11:57:19,793][20347] Updated weights for policy 0, policy_version 910501 (0.0008) [2023-12-27 11:57:19,857][20347] Updated weights for policy 0, policy_version 910511 (0.0008) [2023-12-27 11:57:20,194][20381] Updated weights for policy 1, policy_version 909853 (0.0010) [2023-12-27 11:57:20,257][20381] Updated weights for policy 1, policy_version 909863 (0.0011) [2023-12-27 11:57:20,315][20381] Updated weights for policy 1, policy_version 909873 (0.0011) [2023-12-27 11:57:20,565][20347] Updated weights for policy 0, policy_version 910521 (0.0007) [2023-12-27 11:57:20,629][20347] Updated weights for policy 0, policy_version 910531 (0.0006) [2023-12-27 11:57:20,684][20347] Updated weights for policy 0, policy_version 910541 (0.0007) [2023-12-27 11:57:21,032][20381] Updated weights for policy 1, policy_version 909883 (0.0010) [2023-12-27 11:57:21,095][20381] Updated weights for policy 1, policy_version 909893 (0.0007) [2023-12-27 11:57:21,167][20381] Updated weights for policy 1, policy_version 909903 (0.0008) [2023-12-27 11:57:21,394][20347] Updated weights for policy 0, policy_version 910551 (0.0008) [2023-12-27 11:57:21,463][20347] Updated weights for policy 0, policy_version 910561 (0.0007) [2023-12-27 11:57:21,518][20347] Updated weights for policy 0, policy_version 910571 (0.0007) [2023-12-27 11:57:21,863][20381] Updated weights for policy 1, policy_version 909913 (0.0007) [2023-12-27 11:57:21,918][20381] Updated weights for policy 1, policy_version 909923 (0.0005) [2023-12-27 11:57:21,976][20381] Updated weights for policy 1, policy_version 909933 (0.0006) [2023-12-27 11:57:22,032][20381] Updated weights for policy 1, policy_version 909943 (0.0009) [2023-12-27 11:57:22,182][20347] Updated weights for policy 0, policy_version 910581 (0.0009) [2023-12-27 11:57:22,241][20347] Updated weights for policy 0, policy_version 910591 (0.0007) [2023-12-27 11:57:22,300][20347] Updated weights for policy 0, policy_version 910601 (0.0009) [2023-12-27 11:57:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 466116608. Throughput: 0: 10048.1, 1: 10253.8. Samples: 466083148. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '210.250')] [2023-12-27 11:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000909944_232972288.pth... [2023-12-27 11:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000910608_233144320.pth... [2023-12-27 11:57:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000908728_232660992.pth [2023-12-27 11:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000909424_232841216.pth [2023-12-27 11:57:22,744][20381] Updated weights for policy 1, policy_version 909953 (0.0008) [2023-12-27 11:57:22,799][20381] Updated weights for policy 1, policy_version 909963 (0.0008) [2023-12-27 11:57:22,854][20381] Updated weights for policy 1, policy_version 909973 (0.0008) [2023-12-27 11:57:23,006][20347] Updated weights for policy 0, policy_version 910611 (0.0008) [2023-12-27 11:57:23,063][20347] Updated weights for policy 0, policy_version 910621 (0.0007) [2023-12-27 11:57:23,115][20347] Updated weights for policy 0, policy_version 910631 (0.0008) [2023-12-27 11:57:23,520][20381] Updated weights for policy 1, policy_version 909983 (0.0006) [2023-12-27 11:57:23,569][20381] Updated weights for policy 1, policy_version 909993 (0.0006) [2023-12-27 11:57:23,615][20381] Updated weights for policy 1, policy_version 910003 (0.0005) [2023-12-27 11:57:23,753][20347] Updated weights for policy 0, policy_version 910641 (0.0007) [2023-12-27 11:57:23,808][20347] Updated weights for policy 0, policy_version 910651 (0.0005) [2023-12-27 11:57:23,866][20347] Updated weights for policy 0, policy_version 910661 (0.0007) [2023-12-27 11:57:23,921][20347] Updated weights for policy 0, policy_version 910671 (0.0007) [2023-12-27 11:57:24,292][20381] Updated weights for policy 1, policy_version 910013 (0.0006) [2023-12-27 11:57:24,342][20381] Updated weights for policy 1, policy_version 910023 (0.0007) [2023-12-27 11:57:24,388][20381] Updated weights for policy 1, policy_version 910033 (0.0007) [2023-12-27 11:57:24,591][20347] Updated weights for policy 0, policy_version 910681 (0.0008) [2023-12-27 11:57:24,642][20347] Updated weights for policy 0, policy_version 910691 (0.0008) [2023-12-27 11:57:24,702][20347] Updated weights for policy 0, policy_version 910701 (0.0007) [2023-12-27 11:57:25,048][20381] Updated weights for policy 1, policy_version 910043 (0.0007) [2023-12-27 11:57:25,106][20381] Updated weights for policy 1, policy_version 910053 (0.0006) [2023-12-27 11:57:25,162][20381] Updated weights for policy 1, policy_version 910063 (0.0005) [2023-12-27 11:57:25,409][20347] Updated weights for policy 0, policy_version 910711 (0.0009) [2023-12-27 11:57:25,460][20347] Updated weights for policy 0, policy_version 910721 (0.0010) [2023-12-27 11:57:25,509][20347] Updated weights for policy 0, policy_version 910731 (0.0010) [2023-12-27 11:57:25,802][20381] Updated weights for policy 1, policy_version 910073 (0.0007) [2023-12-27 11:57:25,849][20381] Updated weights for policy 1, policy_version 910083 (0.0008) [2023-12-27 11:57:25,896][20381] Updated weights for policy 1, policy_version 910093 (0.0010) [2023-12-27 11:57:25,945][20381] Updated weights for policy 1, policy_version 910103 (0.0009) [2023-12-27 11:57:26,123][20347] Updated weights for policy 0, policy_version 910741 (0.0010) [2023-12-27 11:57:26,168][20347] Updated weights for policy 0, policy_version 910751 (0.0010) [2023-12-27 11:57:26,223][20347] Updated weights for policy 0, policy_version 910761 (0.0010) [2023-12-27 11:57:26,684][20381] Updated weights for policy 1, policy_version 910113 (0.0009) [2023-12-27 11:57:26,728][20381] Updated weights for policy 1, policy_version 910123 (0.0008) [2023-12-27 11:57:26,777][20381] Updated weights for policy 1, policy_version 910133 (0.0008) [2023-12-27 11:57:26,838][20347] Updated weights for policy 0, policy_version 910771 (0.0009) [2023-12-27 11:57:26,891][20347] Updated weights for policy 0, policy_version 910781 (0.0005) [2023-12-27 11:57:26,942][20347] Updated weights for policy 0, policy_version 910791 (0.0005) [2023-12-27 11:57:27,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 466214912. Throughput: 0: 10042.1, 1: 10251.8. Samples: 466204908. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:27,363][19170] Avg episode reward: [(0, '990.320'), (1, '129.390')] [2023-12-27 11:57:27,491][20347] Updated weights for policy 0, policy_version 910801 (0.0006) [2023-12-27 11:57:27,546][20347] Updated weights for policy 0, policy_version 910811 (0.0011) [2023-12-27 11:57:27,551][20381] Updated weights for policy 1, policy_version 910143 (0.0008) [2023-12-27 11:57:27,599][20347] Updated weights for policy 0, policy_version 910821 (0.0007) [2023-12-27 11:57:27,607][20381] Updated weights for policy 1, policy_version 910153 (0.0007) [2023-12-27 11:57:27,660][20347] Updated weights for policy 0, policy_version 910831 (0.0010) [2023-12-27 11:57:27,666][20381] Updated weights for policy 1, policy_version 910163 (0.0006) [2023-12-27 11:57:28,292][20347] Updated weights for policy 0, policy_version 910841 (0.0006) [2023-12-27 11:57:28,359][20347] Updated weights for policy 0, policy_version 910851 (0.0006) [2023-12-27 11:57:28,368][20381] Updated weights for policy 1, policy_version 910173 (0.0007) [2023-12-27 11:57:28,415][20347] Updated weights for policy 0, policy_version 910861 (0.0008) [2023-12-27 11:57:28,420][20381] Updated weights for policy 1, policy_version 910183 (0.0006) [2023-12-27 11:57:28,475][20381] Updated weights for policy 1, policy_version 910193 (0.0006) [2023-12-27 11:57:29,063][20347] Updated weights for policy 0, policy_version 910871 (0.0007) [2023-12-27 11:57:29,110][20347] Updated weights for policy 0, policy_version 910881 (0.0005) [2023-12-27 11:57:29,158][20347] Updated weights for policy 0, policy_version 910891 (0.0006) [2023-12-27 11:57:29,171][20381] Updated weights for policy 1, policy_version 910203 (0.0006) [2023-12-27 11:57:29,233][20381] Updated weights for policy 1, policy_version 910213 (0.0008) [2023-12-27 11:57:29,292][20381] Updated weights for policy 1, policy_version 910223 (0.0006) [2023-12-27 11:57:29,817][20347] Updated weights for policy 0, policy_version 910901 (0.0006) [2023-12-27 11:57:29,882][20347] Updated weights for policy 0, policy_version 910911 (0.0007) [2023-12-27 11:57:29,939][20347] Updated weights for policy 0, policy_version 910921 (0.0006) [2023-12-27 11:57:30,071][20381] Updated weights for policy 1, policy_version 910233 (0.0009) [2023-12-27 11:57:30,131][20381] Updated weights for policy 1, policy_version 910243 (0.0008) [2023-12-27 11:57:30,188][20381] Updated weights for policy 1, policy_version 910253 (0.0008) [2023-12-27 11:57:30,238][20381] Updated weights for policy 1, policy_version 910263 (0.0008) [2023-12-27 11:57:30,572][20347] Updated weights for policy 0, policy_version 910931 (0.0008) [2023-12-27 11:57:30,620][20347] Updated weights for policy 0, policy_version 910941 (0.0007) [2023-12-27 11:57:30,666][20347] Updated weights for policy 0, policy_version 910951 (0.0008) [2023-12-27 11:57:30,877][20381] Updated weights for policy 1, policy_version 910273 (0.0009) [2023-12-27 11:57:30,922][20381] Updated weights for policy 1, policy_version 910283 (0.0008) [2023-12-27 11:57:30,970][20381] Updated weights for policy 1, policy_version 910293 (0.0007) [2023-12-27 11:57:31,333][20347] Updated weights for policy 0, policy_version 910961 (0.0007) [2023-12-27 11:57:31,401][20347] Updated weights for policy 0, policy_version 910971 (0.0007) [2023-12-27 11:57:31,453][20347] Updated weights for policy 0, policy_version 910981 (0.0011) [2023-12-27 11:57:31,506][20347] Updated weights for policy 0, policy_version 910991 (0.0010) [2023-12-27 11:57:31,751][20381] Updated weights for policy 1, policy_version 910303 (0.0008) [2023-12-27 11:57:31,809][20381] Updated weights for policy 1, policy_version 910313 (0.0008) [2023-12-27 11:57:31,867][20381] Updated weights for policy 1, policy_version 910323 (0.0008) [2023-12-27 11:57:32,192][20347] Updated weights for policy 0, policy_version 911001 (0.0010) [2023-12-27 11:57:32,250][20347] Updated weights for policy 0, policy_version 911011 (0.0008) [2023-12-27 11:57:32,314][20347] Updated weights for policy 0, policy_version 911021 (0.0008) [2023-12-27 11:57:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 466321408. Throughput: 0: 10147.4, 1: 10196.2. Samples: 466328632. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:32,363][19170] Avg episode reward: [(0, '990.320'), (1, '148.380')] [2023-12-27 11:57:32,593][20381] Updated weights for policy 1, policy_version 910333 (0.0007) [2023-12-27 11:57:32,643][20381] Updated weights for policy 1, policy_version 910343 (0.0008) [2023-12-27 11:57:32,697][20381] Updated weights for policy 1, policy_version 910353 (0.0007) [2023-12-27 11:57:33,028][20347] Updated weights for policy 0, policy_version 911031 (0.0007) [2023-12-27 11:57:33,085][20347] Updated weights for policy 0, policy_version 911041 (0.0006) [2023-12-27 11:57:33,137][20347] Updated weights for policy 0, policy_version 911051 (0.0008) [2023-12-27 11:57:33,359][20381] Updated weights for policy 1, policy_version 910363 (0.0007) [2023-12-27 11:57:33,411][20381] Updated weights for policy 1, policy_version 910373 (0.0007) [2023-12-27 11:57:33,471][20381] Updated weights for policy 1, policy_version 910383 (0.0007) [2023-12-27 11:57:33,746][20347] Updated weights for policy 0, policy_version 911061 (0.0009) [2023-12-27 11:57:33,792][20347] Updated weights for policy 0, policy_version 911071 (0.0010) [2023-12-27 11:57:33,842][20347] Updated weights for policy 0, policy_version 911081 (0.0010) [2023-12-27 11:57:34,123][20381] Updated weights for policy 1, policy_version 910393 (0.0005) [2023-12-27 11:57:34,188][20381] Updated weights for policy 1, policy_version 910403 (0.0008) [2023-12-27 11:57:34,247][20381] Updated weights for policy 1, policy_version 910413 (0.0007) [2023-12-27 11:57:34,309][20381] Updated weights for policy 1, policy_version 910423 (0.0008) [2023-12-27 11:57:34,491][20347] Updated weights for policy 0, policy_version 911091 (0.0009) [2023-12-27 11:57:34,552][20347] Updated weights for policy 0, policy_version 911101 (0.0007) [2023-12-27 11:57:34,610][20347] Updated weights for policy 0, policy_version 911111 (0.0009) [2023-12-27 11:57:35,047][20381] Updated weights for policy 1, policy_version 910433 (0.0008) [2023-12-27 11:57:35,095][20381] Updated weights for policy 1, policy_version 910443 (0.0006) [2023-12-27 11:57:35,142][20381] Updated weights for policy 1, policy_version 910453 (0.0007) [2023-12-27 11:57:35,186][20347] Updated weights for policy 0, policy_version 911121 (0.0009) [2023-12-27 11:57:35,228][20347] Updated weights for policy 0, policy_version 911131 (0.0010) [2023-12-27 11:57:35,281][20347] Updated weights for policy 0, policy_version 911141 (0.0010) [2023-12-27 11:57:35,324][20347] Updated weights for policy 0, policy_version 911151 (0.0008) [2023-12-27 11:57:35,845][20381] Updated weights for policy 1, policy_version 910463 (0.0007) [2023-12-27 11:57:35,899][20381] Updated weights for policy 1, policy_version 910473 (0.0007) [2023-12-27 11:57:35,954][20381] Updated weights for policy 1, policy_version 910483 (0.0007) [2023-12-27 11:57:36,006][20347] Updated weights for policy 0, policy_version 911161 (0.0009) [2023-12-27 11:57:36,064][20347] Updated weights for policy 0, policy_version 911171 (0.0010) [2023-12-27 11:57:36,126][20347] Updated weights for policy 0, policy_version 911181 (0.0010) [2023-12-27 11:57:36,657][20381] Updated weights for policy 1, policy_version 910493 (0.0007) [2023-12-27 11:57:36,709][20381] Updated weights for policy 1, policy_version 910503 (0.0007) [2023-12-27 11:57:36,763][20381] Updated weights for policy 1, policy_version 910513 (0.0007) [2023-12-27 11:57:36,860][20347] Updated weights for policy 0, policy_version 911191 (0.0009) [2023-12-27 11:57:36,925][20347] Updated weights for policy 0, policy_version 911201 (0.0010) [2023-12-27 11:57:36,989][20347] Updated weights for policy 0, policy_version 911211 (0.0010) [2023-12-27 11:57:37,354][20381] Updated weights for policy 1, policy_version 910523 (0.0008) [2023-12-27 11:57:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 466419712. Throughput: 0: 10162.1, 1: 10198.8. Samples: 466391032. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '197.530')] [2023-12-27 11:57:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000911216_233299968.pth... [2023-12-27 11:57:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000910032_232996864.pth [2023-12-27 11:57:37,416][20381] Updated weights for policy 1, policy_version 910533 (0.0007) [2023-12-27 11:57:37,474][20381] Updated weights for policy 1, policy_version 910543 (0.0008) [2023-12-27 11:57:37,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000910552_233127936.pth... [2023-12-27 11:57:37,522][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000909368_232824832.pth [2023-12-27 11:57:37,645][20347] Updated weights for policy 0, policy_version 911221 (0.0010) [2023-12-27 11:57:37,703][20347] Updated weights for policy 0, policy_version 911231 (0.0008) [2023-12-27 11:57:37,757][20347] Updated weights for policy 0, policy_version 911241 (0.0007) [2023-12-27 11:57:38,101][20381] Updated weights for policy 1, policy_version 910553 (0.0008) [2023-12-27 11:57:38,150][20381] Updated weights for policy 1, policy_version 910563 (0.0009) [2023-12-27 11:57:38,195][20381] Updated weights for policy 1, policy_version 910573 (0.0007) [2023-12-27 11:57:38,252][20381] Updated weights for policy 1, policy_version 910583 (0.0006) [2023-12-27 11:57:38,380][20347] Updated weights for policy 0, policy_version 911251 (0.0007) [2023-12-27 11:57:38,439][20347] Updated weights for policy 0, policy_version 911261 (0.0010) [2023-12-27 11:57:38,499][20347] Updated weights for policy 0, policy_version 911271 (0.0011) [2023-12-27 11:57:38,985][20381] Updated weights for policy 1, policy_version 910593 (0.0007) [2023-12-27 11:57:39,047][20381] Updated weights for policy 1, policy_version 910603 (0.0007) [2023-12-27 11:57:39,108][20381] Updated weights for policy 1, policy_version 910613 (0.0008) [2023-12-27 11:57:39,195][20347] Updated weights for policy 0, policy_version 911281 (0.0010) [2023-12-27 11:57:39,267][20347] Updated weights for policy 0, policy_version 911291 (0.0009) [2023-12-27 11:57:39,338][20347] Updated weights for policy 0, policy_version 911301 (0.0008) [2023-12-27 11:57:39,413][20347] Updated weights for policy 0, policy_version 911311 (0.0009) [2023-12-27 11:57:39,824][20381] Updated weights for policy 1, policy_version 910623 (0.0007) [2023-12-27 11:57:39,885][20381] Updated weights for policy 1, policy_version 910633 (0.0006) [2023-12-27 11:57:39,948][20381] Updated weights for policy 1, policy_version 910643 (0.0006) [2023-12-27 11:57:40,152][20347] Updated weights for policy 0, policy_version 911321 (0.0008) [2023-12-27 11:57:40,211][20347] Updated weights for policy 0, policy_version 911331 (0.0008) [2023-12-27 11:57:40,264][20347] Updated weights for policy 0, policy_version 911341 (0.0008) [2023-12-27 11:57:40,623][20381] Updated weights for policy 1, policy_version 910653 (0.0007) [2023-12-27 11:57:40,677][20381] Updated weights for policy 1, policy_version 910663 (0.0007) [2023-12-27 11:57:40,731][20381] Updated weights for policy 1, policy_version 910673 (0.0005) [2023-12-27 11:57:40,966][20347] Updated weights for policy 0, policy_version 911351 (0.0008) [2023-12-27 11:57:41,026][20347] Updated weights for policy 0, policy_version 911361 (0.0008) [2023-12-27 11:57:41,084][20347] Updated weights for policy 0, policy_version 911371 (0.0008) [2023-12-27 11:57:41,393][20381] Updated weights for policy 1, policy_version 910683 (0.0005) [2023-12-27 11:57:41,456][20381] Updated weights for policy 1, policy_version 910693 (0.0007) [2023-12-27 11:57:41,509][20381] Updated weights for policy 1, policy_version 910703 (0.0007) [2023-12-27 11:57:41,841][20347] Updated weights for policy 0, policy_version 911381 (0.0008) [2023-12-27 11:57:41,894][20347] Updated weights for policy 0, policy_version 911391 (0.0007) [2023-12-27 11:57:41,951][20347] Updated weights for policy 0, policy_version 911401 (0.0007) [2023-12-27 11:57:42,270][20381] Updated weights for policy 1, policy_version 910713 (0.0007) [2023-12-27 11:57:42,330][20381] Updated weights for policy 1, policy_version 910723 (0.0010) [2023-12-27 11:57:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 466518016. Throughput: 0: 10156.0, 1: 10205.3. Samples: 466512708. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '276.890')] [2023-12-27 11:57:42,393][20381] Updated weights for policy 1, policy_version 910733 (0.0010) [2023-12-27 11:57:42,442][20381] Updated weights for policy 1, policy_version 910743 (0.0010) [2023-12-27 11:57:42,685][20347] Updated weights for policy 0, policy_version 911411 (0.0008) [2023-12-27 11:57:42,736][20347] Updated weights for policy 0, policy_version 911421 (0.0008) [2023-12-27 11:57:42,786][20347] Updated weights for policy 0, policy_version 911431 (0.0008) [2023-12-27 11:57:43,096][20381] Updated weights for policy 1, policy_version 910753 (0.0006) [2023-12-27 11:57:43,158][20381] Updated weights for policy 1, policy_version 910763 (0.0005) [2023-12-27 11:57:43,217][20381] Updated weights for policy 1, policy_version 910773 (0.0005) [2023-12-27 11:57:43,563][20347] Updated weights for policy 0, policy_version 911441 (0.0008) [2023-12-27 11:57:43,618][20347] Updated weights for policy 0, policy_version 911451 (0.0007) [2023-12-27 11:57:43,673][20347] Updated weights for policy 0, policy_version 911461 (0.0006) [2023-12-27 11:57:43,720][20381] Updated weights for policy 1, policy_version 910783 (0.0009) [2023-12-27 11:57:43,722][20347] Updated weights for policy 0, policy_version 911471 (0.0006) [2023-12-27 11:57:43,769][20381] Updated weights for policy 1, policy_version 910793 (0.0010) [2023-12-27 11:57:43,814][20381] Updated weights for policy 1, policy_version 910803 (0.0010) [2023-12-27 11:57:44,419][20347] Updated weights for policy 0, policy_version 911481 (0.0007) [2023-12-27 11:57:44,460][20381] Updated weights for policy 1, policy_version 910813 (0.0008) [2023-12-27 11:57:44,477][20347] Updated weights for policy 0, policy_version 911491 (0.0006) [2023-12-27 11:57:44,520][20381] Updated weights for policy 1, policy_version 910823 (0.0009) [2023-12-27 11:57:44,534][20347] Updated weights for policy 0, policy_version 911501 (0.0005) [2023-12-27 11:57:44,570][20381] Updated weights for policy 1, policy_version 910833 (0.0011) [2023-12-27 11:57:45,215][20347] Updated weights for policy 0, policy_version 911511 (0.0009) [2023-12-27 11:57:45,268][20347] Updated weights for policy 0, policy_version 911521 (0.0010) [2023-12-27 11:57:45,290][20381] Updated weights for policy 1, policy_version 910843 (0.0011) [2023-12-27 11:57:45,324][20347] Updated weights for policy 0, policy_version 911531 (0.0008) [2023-12-27 11:57:45,350][20381] Updated weights for policy 1, policy_version 910853 (0.0011) [2023-12-27 11:57:45,404][20381] Updated weights for policy 1, policy_version 910863 (0.0006) [2023-12-27 11:57:45,991][20347] Updated weights for policy 0, policy_version 911541 (0.0009) [2023-12-27 11:57:46,042][20347] Updated weights for policy 0, policy_version 911551 (0.0007) [2023-12-27 11:57:46,057][20381] Updated weights for policy 1, policy_version 910873 (0.0006) [2023-12-27 11:57:46,103][20347] Updated weights for policy 0, policy_version 911561 (0.0006) [2023-12-27 11:57:46,109][20381] Updated weights for policy 1, policy_version 910883 (0.0010) [2023-12-27 11:57:46,161][20381] Updated weights for policy 1, policy_version 910893 (0.0010) [2023-12-27 11:57:46,214][20381] Updated weights for policy 1, policy_version 910903 (0.0010) [2023-12-27 11:57:46,809][20347] Updated weights for policy 0, policy_version 911571 (0.0006) [2023-12-27 11:57:46,857][20347] Updated weights for policy 0, policy_version 911581 (0.0008) [2023-12-27 11:57:46,888][20381] Updated weights for policy 1, policy_version 910913 (0.0008) [2023-12-27 11:57:46,908][20347] Updated weights for policy 0, policy_version 911591 (0.0006) [2023-12-27 11:57:46,943][20381] Updated weights for policy 1, policy_version 910923 (0.0007) [2023-12-27 11:57:47,003][20381] Updated weights for policy 1, policy_version 910933 (0.0006) [2023-12-27 11:57:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 466624512. Throughput: 0: 10115.0, 1: 10240.1. Samples: 466633992. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '233.970')] [2023-12-27 11:57:47,536][20347] Updated weights for policy 0, policy_version 911601 (0.0006) [2023-12-27 11:57:47,593][20347] Updated weights for policy 0, policy_version 911611 (0.0006) [2023-12-27 11:57:47,623][20381] Updated weights for policy 1, policy_version 910943 (0.0007) [2023-12-27 11:57:47,649][20347] Updated weights for policy 0, policy_version 911621 (0.0005) [2023-12-27 11:57:47,668][20381] Updated weights for policy 1, policy_version 910953 (0.0006) [2023-12-27 11:57:47,698][20347] Updated weights for policy 0, policy_version 911631 (0.0006) [2023-12-27 11:57:47,719][20381] Updated weights for policy 1, policy_version 910963 (0.0006) [2023-12-27 11:57:48,358][20347] Updated weights for policy 0, policy_version 911641 (0.0009) [2023-12-27 11:57:48,409][20347] Updated weights for policy 0, policy_version 911651 (0.0006) [2023-12-27 11:57:48,413][20381] Updated weights for policy 1, policy_version 910973 (0.0007) [2023-12-27 11:57:48,466][20347] Updated weights for policy 0, policy_version 911661 (0.0006) [2023-12-27 11:57:48,468][20381] Updated weights for policy 1, policy_version 910983 (0.0006) [2023-12-27 11:57:48,526][20381] Updated weights for policy 1, policy_version 910993 (0.0007) [2023-12-27 11:57:49,161][20381] Updated weights for policy 1, policy_version 911003 (0.0007) [2023-12-27 11:57:49,201][20347] Updated weights for policy 0, policy_version 911671 (0.0006) [2023-12-27 11:57:49,229][20381] Updated weights for policy 1, policy_version 911014 (0.0008) [2023-12-27 11:57:49,264][20347] Updated weights for policy 0, policy_version 911681 (0.0008) [2023-12-27 11:57:49,289][20381] Updated weights for policy 1, policy_version 911024 (0.0008) [2023-12-27 11:57:49,326][20347] Updated weights for policy 0, policy_version 911691 (0.0006) [2023-12-27 11:57:49,995][20347] Updated weights for policy 0, policy_version 911701 (0.0008) [2023-12-27 11:57:49,997][20381] Updated weights for policy 1, policy_version 911034 (0.0009) [2023-12-27 11:57:50,044][20347] Updated weights for policy 0, policy_version 911711 (0.0007) [2023-12-27 11:57:50,057][20381] Updated weights for policy 1, policy_version 911044 (0.0011) [2023-12-27 11:57:50,097][20347] Updated weights for policy 0, policy_version 911721 (0.0006) [2023-12-27 11:57:50,117][20381] Updated weights for policy 1, policy_version 911054 (0.0011) [2023-12-27 11:57:50,172][20381] Updated weights for policy 1, policy_version 911064 (0.0011) [2023-12-27 11:57:50,845][20347] Updated weights for policy 0, policy_version 911731 (0.0005) [2023-12-27 11:57:50,851][20381] Updated weights for policy 1, policy_version 911074 (0.0011) [2023-12-27 11:57:50,908][20347] Updated weights for policy 0, policy_version 911741 (0.0006) [2023-12-27 11:57:50,913][20381] Updated weights for policy 1, policy_version 911084 (0.0011) [2023-12-27 11:57:50,962][20381] Updated weights for policy 1, policy_version 911094 (0.0010) [2023-12-27 11:57:50,970][20347] Updated weights for policy 0, policy_version 911751 (0.0006) [2023-12-27 11:57:51,682][20347] Updated weights for policy 0, policy_version 911761 (0.0007) [2023-12-27 11:57:51,682][20381] Updated weights for policy 1, policy_version 911104 (0.0010) [2023-12-27 11:57:51,745][20347] Updated weights for policy 0, policy_version 911771 (0.0009) [2023-12-27 11:57:51,748][20381] Updated weights for policy 1, policy_version 911114 (0.0011) [2023-12-27 11:57:51,806][20381] Updated weights for policy 1, policy_version 911124 (0.0011) [2023-12-27 11:57:51,808][20347] Updated weights for policy 0, policy_version 911781 (0.0006) [2023-12-27 11:57:51,869][20347] Updated weights for policy 0, policy_version 911791 (0.0007) [2023-12-27 11:57:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 466722816. Throughput: 0: 10145.1, 1: 10180.9. Samples: 466695668. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:52,363][19170] Avg episode reward: [(0, '990.400'), (1, '366.420')] [2023-12-27 11:57:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000911128_233275392.pth... [2023-12-27 11:57:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000911792_233447424.pth... [2023-12-27 11:57:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000909944_232972288.pth [2023-12-27 11:57:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000910608_233144320.pth [2023-12-27 11:57:52,561][20347] Updated weights for policy 0, policy_version 911801 (0.0009) [2023-12-27 11:57:52,605][20381] Updated weights for policy 1, policy_version 911134 (0.0011) [2023-12-27 11:57:52,626][20347] Updated weights for policy 0, policy_version 911811 (0.0010) [2023-12-27 11:57:52,658][20381] Updated weights for policy 1, policy_version 911144 (0.0007) [2023-12-27 11:57:52,685][20347] Updated weights for policy 0, policy_version 911821 (0.0009) [2023-12-27 11:57:52,717][20381] Updated weights for policy 1, policy_version 911154 (0.0007) [2023-12-27 11:57:53,292][20381] Updated weights for policy 1, policy_version 911164 (0.0009) [2023-12-27 11:57:53,347][20381] Updated weights for policy 1, policy_version 911174 (0.0006) [2023-12-27 11:57:53,354][20347] Updated weights for policy 0, policy_version 911831 (0.0010) [2023-12-27 11:57:53,396][20381] Updated weights for policy 1, policy_version 911184 (0.0005) [2023-12-27 11:57:53,405][20347] Updated weights for policy 0, policy_version 911841 (0.0009) [2023-12-27 11:57:53,453][20347] Updated weights for policy 0, policy_version 911851 (0.0007) [2023-12-27 11:57:54,034][20381] Updated weights for policy 1, policy_version 911194 (0.0006) [2023-12-27 11:57:54,086][20381] Updated weights for policy 1, policy_version 911204 (0.0010) [2023-12-27 11:57:54,148][20381] Updated weights for policy 1, policy_version 911214 (0.0010) [2023-12-27 11:57:54,177][20347] Updated weights for policy 0, policy_version 911861 (0.0008) [2023-12-27 11:57:54,201][20381] Updated weights for policy 1, policy_version 911224 (0.0007) [2023-12-27 11:57:54,229][20347] Updated weights for policy 0, policy_version 911871 (0.0010) [2023-12-27 11:57:54,285][20347] Updated weights for policy 0, policy_version 911881 (0.0007) [2023-12-27 11:57:54,828][20381] Updated weights for policy 1, policy_version 911234 (0.0005) [2023-12-27 11:57:54,885][20381] Updated weights for policy 1, policy_version 911244 (0.0008) [2023-12-27 11:57:54,934][20347] Updated weights for policy 0, policy_version 911891 (0.0007) [2023-12-27 11:57:54,942][20381] Updated weights for policy 1, policy_version 911254 (0.0008) [2023-12-27 11:57:54,999][20347] Updated weights for policy 0, policy_version 911901 (0.0007) [2023-12-27 11:57:55,055][20347] Updated weights for policy 0, policy_version 911911 (0.0008) [2023-12-27 11:57:55,548][20381] Updated weights for policy 1, policy_version 911264 (0.0006) [2023-12-27 11:57:55,599][20381] Updated weights for policy 1, policy_version 911274 (0.0005) [2023-12-27 11:57:55,645][20381] Updated weights for policy 1, policy_version 911284 (0.0006) [2023-12-27 11:57:55,682][20347] Updated weights for policy 0, policy_version 911921 (0.0008) [2023-12-27 11:57:55,738][20347] Updated weights for policy 0, policy_version 911931 (0.0005) [2023-12-27 11:57:55,800][20347] Updated weights for policy 0, policy_version 911941 (0.0007) [2023-12-27 11:57:55,858][20347] Updated weights for policy 0, policy_version 911951 (0.0008) [2023-12-27 11:57:56,264][20381] Updated weights for policy 1, policy_version 911294 (0.0009) [2023-12-27 11:57:56,313][20381] Updated weights for policy 1, policy_version 911304 (0.0010) [2023-12-27 11:57:56,371][20381] Updated weights for policy 1, policy_version 911314 (0.0010) [2023-12-27 11:57:56,450][20347] Updated weights for policy 0, policy_version 911961 (0.0006) [2023-12-27 11:57:56,500][20347] Updated weights for policy 0, policy_version 911971 (0.0007) [2023-12-27 11:57:56,546][20347] Updated weights for policy 0, policy_version 911981 (0.0007) [2023-12-27 11:57:57,067][20381] Updated weights for policy 1, policy_version 911324 (0.0010) [2023-12-27 11:57:57,113][20381] Updated weights for policy 1, policy_version 911334 (0.0007) [2023-12-27 11:57:57,171][20381] Updated weights for policy 1, policy_version 911344 (0.0005) [2023-12-27 11:57:57,267][20347] Updated weights for policy 0, policy_version 911991 (0.0007) [2023-12-27 11:57:57,324][20347] Updated weights for policy 0, policy_version 912001 (0.0008) [2023-12-27 11:57:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 466829312. Throughput: 0: 10191.8, 1: 10236.1. Samples: 466820576. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:57:57,363][19170] Avg episode reward: [(0, '990.400'), (1, '361.830')] [2023-12-27 11:57:57,384][20347] Updated weights for policy 0, policy_version 912011 (0.0007) [2023-12-27 11:57:57,812][20381] Updated weights for policy 1, policy_version 911354 (0.0009) [2023-12-27 11:57:57,856][20381] Updated weights for policy 1, policy_version 911364 (0.0010) [2023-12-27 11:57:57,910][20381] Updated weights for policy 1, policy_version 911374 (0.0010) [2023-12-27 11:57:57,969][20381] Updated weights for policy 1, policy_version 911384 (0.0010) [2023-12-27 11:57:58,000][20347] Updated weights for policy 0, policy_version 912021 (0.0007) [2023-12-27 11:57:58,055][20347] Updated weights for policy 0, policy_version 912031 (0.0007) [2023-12-27 11:57:58,117][20347] Updated weights for policy 0, policy_version 912041 (0.0007) [2023-12-27 11:57:58,718][20381] Updated weights for policy 1, policy_version 911394 (0.0008) [2023-12-27 11:57:58,779][20381] Updated weights for policy 1, policy_version 911404 (0.0007) [2023-12-27 11:57:58,850][20381] Updated weights for policy 1, policy_version 911414 (0.0008) [2023-12-27 11:57:58,897][20347] Updated weights for policy 0, policy_version 912051 (0.0007) [2023-12-27 11:57:58,967][20347] Updated weights for policy 0, policy_version 912061 (0.0008) [2023-12-27 11:57:59,022][20347] Updated weights for policy 0, policy_version 912071 (0.0007) [2023-12-27 11:57:59,592][20381] Updated weights for policy 1, policy_version 911424 (0.0010) [2023-12-27 11:57:59,654][20381] Updated weights for policy 1, policy_version 911434 (0.0005) [2023-12-27 11:57:59,716][20381] Updated weights for policy 1, policy_version 911444 (0.0009) [2023-12-27 11:57:59,797][20347] Updated weights for policy 0, policy_version 912081 (0.0007) [2023-12-27 11:57:59,861][20347] Updated weights for policy 0, policy_version 912091 (0.0008) [2023-12-27 11:57:59,920][20347] Updated weights for policy 0, policy_version 912101 (0.0008) [2023-12-27 11:57:59,979][20347] Updated weights for policy 0, policy_version 912111 (0.0008) [2023-12-27 11:58:00,348][20381] Updated weights for policy 1, policy_version 911454 (0.0008) [2023-12-27 11:58:00,402][20381] Updated weights for policy 1, policy_version 911464 (0.0006) [2023-12-27 11:58:00,452][20381] Updated weights for policy 1, policy_version 911474 (0.0005) [2023-12-27 11:58:00,706][20347] Updated weights for policy 0, policy_version 912121 (0.0007) [2023-12-27 11:58:00,754][20347] Updated weights for policy 0, policy_version 912131 (0.0008) [2023-12-27 11:58:00,797][20347] Updated weights for policy 0, policy_version 912141 (0.0006) [2023-12-27 11:58:01,121][20381] Updated weights for policy 1, policy_version 911484 (0.0008) [2023-12-27 11:58:01,187][20381] Updated weights for policy 1, policy_version 911494 (0.0007) [2023-12-27 11:58:01,248][20381] Updated weights for policy 1, policy_version 911504 (0.0006) [2023-12-27 11:58:01,536][20347] Updated weights for policy 0, policy_version 912151 (0.0006) [2023-12-27 11:58:01,595][20347] Updated weights for policy 0, policy_version 912161 (0.0008) [2023-12-27 11:58:01,658][20347] Updated weights for policy 0, policy_version 912171 (0.0008) [2023-12-27 11:58:01,961][20381] Updated weights for policy 1, policy_version 911514 (0.0008) [2023-12-27 11:58:02,010][20381] Updated weights for policy 1, policy_version 911524 (0.0007) [2023-12-27 11:58:02,059][20381] Updated weights for policy 1, policy_version 911534 (0.0009) [2023-12-27 11:58:02,113][20381] Updated weights for policy 1, policy_version 911544 (0.0011) [2023-12-27 11:58:02,362][20347] Updated weights for policy 0, policy_version 912181 (0.0008) [2023-12-27 11:58:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 466927616. Throughput: 0: 10163.4, 1: 10214.5. Samples: 466938976. Policy #0 lag: (min: 8.0, avg: 32.6, max: 40.0) [2023-12-27 11:58:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '328.500')] [2023-12-27 11:58:02,424][20347] Updated weights for policy 0, policy_version 912191 (0.0006) [2023-12-27 11:58:02,482][20347] Updated weights for policy 0, policy_version 912201 (0.0007) [2023-12-27 11:58:02,824][20381] Updated weights for policy 1, policy_version 911554 (0.0011) [2023-12-27 11:58:02,869][20381] Updated weights for policy 1, policy_version 911564 (0.0010) [2023-12-27 11:58:02,915][20381] Updated weights for policy 1, policy_version 911574 (0.0010) [2023-12-27 11:58:03,120][20347] Updated weights for policy 0, policy_version 912211 (0.0007) [2023-12-27 11:58:03,168][20347] Updated weights for policy 0, policy_version 912221 (0.0006) [2023-12-27 11:58:03,228][20347] Updated weights for policy 0, policy_version 912231 (0.0006) [2023-12-27 11:58:03,534][20381] Updated weights for policy 1, policy_version 911584 (0.0006) [2023-12-27 11:58:03,581][20381] Updated weights for policy 1, policy_version 911594 (0.0007) [2023-12-27 11:58:03,631][20381] Updated weights for policy 1, policy_version 911604 (0.0008) [2023-12-27 11:58:03,856][20347] Updated weights for policy 0, policy_version 912241 (0.0007) [2023-12-27 11:58:03,913][20347] Updated weights for policy 0, policy_version 912251 (0.0006) [2023-12-27 11:58:03,974][20347] Updated weights for policy 0, policy_version 912261 (0.0006) [2023-12-27 11:58:04,038][20347] Updated weights for policy 0, policy_version 912271 (0.0008) [2023-12-27 11:58:04,327][20381] Updated weights for policy 1, policy_version 911614 (0.0007) [2023-12-27 11:58:04,387][20381] Updated weights for policy 1, policy_version 911624 (0.0008) [2023-12-27 11:58:04,444][20381] Updated weights for policy 1, policy_version 911634 (0.0007) [2023-12-27 11:58:04,695][20347] Updated weights for policy 0, policy_version 912281 (0.0010) [2023-12-27 11:58:04,740][20347] Updated weights for policy 0, policy_version 912291 (0.0010) [2023-12-27 11:58:04,796][20347] Updated weights for policy 0, policy_version 912301 (0.0009) [2023-12-27 11:58:05,120][20381] Updated weights for policy 1, policy_version 911644 (0.0006) [2023-12-27 11:58:05,180][20381] Updated weights for policy 1, policy_version 911654 (0.0005) [2023-12-27 11:58:05,229][20381] Updated weights for policy 1, policy_version 911664 (0.0005) [2023-12-27 11:58:05,529][20347] Updated weights for policy 0, policy_version 912311 (0.0008) [2023-12-27 11:58:05,589][20347] Updated weights for policy 0, policy_version 912321 (0.0006) [2023-12-27 11:58:05,641][20347] Updated weights for policy 0, policy_version 912331 (0.0006) [2023-12-27 11:58:05,847][20381] Updated weights for policy 1, policy_version 911674 (0.0006) [2023-12-27 11:58:05,895][20381] Updated weights for policy 1, policy_version 911684 (0.0007) [2023-12-27 11:58:05,939][20381] Updated weights for policy 1, policy_version 911694 (0.0007) [2023-12-27 11:58:05,985][20381] Updated weights for policy 1, policy_version 911704 (0.0008) [2023-12-27 11:58:06,272][20347] Updated weights for policy 0, policy_version 912341 (0.0008) [2023-12-27 11:58:06,331][20347] Updated weights for policy 0, policy_version 912351 (0.0007) [2023-12-27 11:58:06,391][20347] Updated weights for policy 0, policy_version 912361 (0.0007) [2023-12-27 11:58:06,691][20381] Updated weights for policy 1, policy_version 911714 (0.0006) [2023-12-27 11:58:06,752][20381] Updated weights for policy 1, policy_version 911724 (0.0006) [2023-12-27 11:58:06,813][20381] Updated weights for policy 1, policy_version 911734 (0.0008) [2023-12-27 11:58:07,120][20347] Updated weights for policy 0, policy_version 912371 (0.0007) [2023-12-27 11:58:07,175][20347] Updated weights for policy 0, policy_version 912381 (0.0007) [2023-12-27 11:58:07,227][20347] Updated weights for policy 0, policy_version 912391 (0.0008) [2023-12-27 11:58:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 467034112. Throughput: 0: 10202.9, 1: 10219.8. Samples: 467002164. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '531.890')] [2023-12-27 11:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000911736_233431040.pth... [2023-12-27 11:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000912400_233603072.pth... [2023-12-27 11:58:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000910552_233127936.pth [2023-12-27 11:58:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000911216_233299968.pth [2023-12-27 11:58:07,461][20381] Updated weights for policy 1, policy_version 911744 (0.0007) [2023-12-27 11:58:07,513][20381] Updated weights for policy 1, policy_version 911754 (0.0008) [2023-12-27 11:58:07,566][20381] Updated weights for policy 1, policy_version 911764 (0.0008) [2023-12-27 11:58:07,913][20347] Updated weights for policy 0, policy_version 912401 (0.0008) [2023-12-27 11:58:07,968][20347] Updated weights for policy 0, policy_version 912411 (0.0006) [2023-12-27 11:58:08,029][20347] Updated weights for policy 0, policy_version 912421 (0.0007) [2023-12-27 11:58:08,090][20347] Updated weights for policy 0, policy_version 912431 (0.0007) [2023-12-27 11:58:08,241][20381] Updated weights for policy 1, policy_version 911774 (0.0008) [2023-12-27 11:58:08,286][20381] Updated weights for policy 1, policy_version 911784 (0.0005) [2023-12-27 11:58:08,337][20381] Updated weights for policy 1, policy_version 911794 (0.0007) [2023-12-27 11:58:08,789][20347] Updated weights for policy 0, policy_version 912441 (0.0008) [2023-12-27 11:58:08,838][20347] Updated weights for policy 0, policy_version 912451 (0.0008) [2023-12-27 11:58:08,891][20347] Updated weights for policy 0, policy_version 912461 (0.0008) [2023-12-27 11:58:09,023][20381] Updated weights for policy 1, policy_version 911804 (0.0008) [2023-12-27 11:58:09,075][20381] Updated weights for policy 1, policy_version 911814 (0.0008) [2023-12-27 11:58:09,130][20381] Updated weights for policy 1, policy_version 911824 (0.0007) [2023-12-27 11:58:09,633][20347] Updated weights for policy 0, policy_version 912471 (0.0008) [2023-12-27 11:58:09,686][20347] Updated weights for policy 0, policy_version 912481 (0.0010) [2023-12-27 11:58:09,739][20347] Updated weights for policy 0, policy_version 912491 (0.0010) [2023-12-27 11:58:09,815][20381] Updated weights for policy 1, policy_version 911834 (0.0006) [2023-12-27 11:58:09,881][20381] Updated weights for policy 1, policy_version 911844 (0.0008) [2023-12-27 11:58:09,946][20381] Updated weights for policy 1, policy_version 911854 (0.0008) [2023-12-27 11:58:10,000][20381] Updated weights for policy 1, policy_version 911864 (0.0006) [2023-12-27 11:58:10,449][20347] Updated weights for policy 0, policy_version 912501 (0.0008) [2023-12-27 11:58:10,510][20347] Updated weights for policy 0, policy_version 912511 (0.0008) [2023-12-27 11:58:10,563][20347] Updated weights for policy 0, policy_version 912521 (0.0007) [2023-12-27 11:58:10,665][20381] Updated weights for policy 1, policy_version 911874 (0.0008) [2023-12-27 11:58:10,723][20381] Updated weights for policy 1, policy_version 911884 (0.0008) [2023-12-27 11:58:10,782][20381] Updated weights for policy 1, policy_version 911894 (0.0008) [2023-12-27 11:58:11,303][20347] Updated weights for policy 0, policy_version 912531 (0.0007) [2023-12-27 11:58:11,364][20347] Updated weights for policy 0, policy_version 912541 (0.0007) [2023-12-27 11:58:11,434][20347] Updated weights for policy 0, policy_version 912551 (0.0008) [2023-12-27 11:58:11,494][20381] Updated weights for policy 1, policy_version 911904 (0.0007) [2023-12-27 11:58:11,544][20381] Updated weights for policy 1, policy_version 911914 (0.0008) [2023-12-27 11:58:11,596][20381] Updated weights for policy 1, policy_version 911924 (0.0007) [2023-12-27 11:58:12,144][20347] Updated weights for policy 0, policy_version 912561 (0.0007) [2023-12-27 11:58:12,197][20347] Updated weights for policy 0, policy_version 912571 (0.0008) [2023-12-27 11:58:12,254][20347] Updated weights for policy 0, policy_version 912581 (0.0008) [2023-12-27 11:58:12,316][20347] Updated weights for policy 0, policy_version 912591 (0.0008) [2023-12-27 11:58:12,361][20381] Updated weights for policy 1, policy_version 911934 (0.0008) [2023-12-27 11:58:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 467132416. Throughput: 0: 10156.3, 1: 10241.6. Samples: 467122812. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '568.080')] [2023-12-27 11:58:12,413][20381] Updated weights for policy 1, policy_version 911944 (0.0008) [2023-12-27 11:58:12,464][20381] Updated weights for policy 1, policy_version 911954 (0.0008) [2023-12-27 11:58:13,017][20347] Updated weights for policy 0, policy_version 912601 (0.0007) [2023-12-27 11:58:13,062][20347] Updated weights for policy 0, policy_version 912611 (0.0009) [2023-12-27 11:58:13,111][20381] Updated weights for policy 1, policy_version 911964 (0.0008) [2023-12-27 11:58:13,116][20347] Updated weights for policy 0, policy_version 912621 (0.0005) [2023-12-27 11:58:13,169][20381] Updated weights for policy 1, policy_version 911974 (0.0008) [2023-12-27 11:58:13,230][20381] Updated weights for policy 1, policy_version 911984 (0.0010) [2023-12-27 11:58:13,670][20347] Updated weights for policy 0, policy_version 912631 (0.0009) [2023-12-27 11:58:13,725][20347] Updated weights for policy 0, policy_version 912641 (0.0010) [2023-12-27 11:58:13,784][20347] Updated weights for policy 0, policy_version 912651 (0.0011) [2023-12-27 11:58:13,883][20381] Updated weights for policy 1, policy_version 911994 (0.0010) [2023-12-27 11:58:13,943][20381] Updated weights for policy 1, policy_version 912004 (0.0007) [2023-12-27 11:58:13,988][20381] Updated weights for policy 1, policy_version 912014 (0.0010) [2023-12-27 11:58:14,033][20381] Updated weights for policy 1, policy_version 912024 (0.0010) [2023-12-27 11:58:14,504][20347] Updated weights for policy 0, policy_version 912661 (0.0010) [2023-12-27 11:58:14,561][20347] Updated weights for policy 0, policy_version 912671 (0.0010) [2023-12-27 11:58:14,616][20347] Updated weights for policy 0, policy_version 912681 (0.0010) [2023-12-27 11:58:14,648][20381] Updated weights for policy 1, policy_version 912034 (0.0010) [2023-12-27 11:58:14,697][20381] Updated weights for policy 1, policy_version 912044 (0.0006) [2023-12-27 11:58:14,751][20381] Updated weights for policy 1, policy_version 912054 (0.0005) [2023-12-27 11:58:15,318][20347] Updated weights for policy 0, policy_version 912691 (0.0010) [2023-12-27 11:58:15,377][20347] Updated weights for policy 0, policy_version 912701 (0.0008) [2023-12-27 11:58:15,422][20381] Updated weights for policy 1, policy_version 912064 (0.0008) [2023-12-27 11:58:15,428][20347] Updated weights for policy 0, policy_version 912711 (0.0005) [2023-12-27 11:58:15,478][20381] Updated weights for policy 1, policy_version 912074 (0.0005) [2023-12-27 11:58:15,526][20381] Updated weights for policy 1, policy_version 912084 (0.0007) [2023-12-27 11:58:16,080][20347] Updated weights for policy 0, policy_version 912721 (0.0006) [2023-12-27 11:58:16,133][20347] Updated weights for policy 0, policy_version 912731 (0.0007) [2023-12-27 11:58:16,169][20381] Updated weights for policy 1, policy_version 912094 (0.0006) [2023-12-27 11:58:16,186][20347] Updated weights for policy 0, policy_version 912741 (0.0006) [2023-12-27 11:58:16,233][20381] Updated weights for policy 1, policy_version 912104 (0.0008) [2023-12-27 11:58:16,242][20347] Updated weights for policy 0, policy_version 912751 (0.0007) [2023-12-27 11:58:16,291][20381] Updated weights for policy 1, policy_version 912114 (0.0006) [2023-12-27 11:58:16,964][20347] Updated weights for policy 0, policy_version 912761 (0.0007) [2023-12-27 11:58:17,017][20381] Updated weights for policy 1, policy_version 912124 (0.0007) [2023-12-27 11:58:17,028][20347] Updated weights for policy 0, policy_version 912771 (0.0007) [2023-12-27 11:58:17,075][20381] Updated weights for policy 1, policy_version 912134 (0.0008) [2023-12-27 11:58:17,088][20347] Updated weights for policy 0, policy_version 912781 (0.0006) [2023-12-27 11:58:17,132][20381] Updated weights for policy 1, policy_version 912144 (0.0011) [2023-12-27 11:58:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 467238912. Throughput: 0: 10060.6, 1: 10305.0. Samples: 467245084. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '527.250')] [2023-12-27 11:58:17,760][20347] Updated weights for policy 0, policy_version 912791 (0.0007) [2023-12-27 11:58:17,824][20381] Updated weights for policy 1, policy_version 912154 (0.0011) [2023-12-27 11:58:17,825][20347] Updated weights for policy 0, policy_version 912801 (0.0007) [2023-12-27 11:58:17,880][20381] Updated weights for policy 1, policy_version 912164 (0.0011) [2023-12-27 11:58:17,887][20347] Updated weights for policy 0, policy_version 912811 (0.0007) [2023-12-27 11:58:17,924][20381] Updated weights for policy 1, policy_version 912174 (0.0010) [2023-12-27 11:58:17,971][20381] Updated weights for policy 1, policy_version 912184 (0.0010) [2023-12-27 11:58:18,594][20347] Updated weights for policy 0, policy_version 912821 (0.0008) [2023-12-27 11:58:18,649][20347] Updated weights for policy 0, policy_version 912831 (0.0010) [2023-12-27 11:58:18,650][20381] Updated weights for policy 1, policy_version 912194 (0.0011) [2023-12-27 11:58:18,706][20381] Updated weights for policy 1, policy_version 912204 (0.0011) [2023-12-27 11:58:18,710][20347] Updated weights for policy 0, policy_version 912841 (0.0011) [2023-12-27 11:58:18,759][20381] Updated weights for policy 1, policy_version 912214 (0.0011) [2023-12-27 11:58:19,412][20347] Updated weights for policy 0, policy_version 912851 (0.0010) [2023-12-27 11:58:19,475][20381] Updated weights for policy 1, policy_version 912224 (0.0012) [2023-12-27 11:58:19,479][20347] Updated weights for policy 0, policy_version 912861 (0.0006) [2023-12-27 11:58:19,533][20381] Updated weights for policy 1, policy_version 912234 (0.0009) [2023-12-27 11:58:19,544][20347] Updated weights for policy 0, policy_version 912871 (0.0006) [2023-12-27 11:58:19,591][20381] Updated weights for policy 1, policy_version 912244 (0.0008) [2023-12-27 11:58:20,214][20347] Updated weights for policy 0, policy_version 912881 (0.0006) [2023-12-27 11:58:20,279][20347] Updated weights for policy 0, policy_version 912891 (0.0007) [2023-12-27 11:58:20,282][20381] Updated weights for policy 1, policy_version 912254 (0.0008) [2023-12-27 11:58:20,339][20381] Updated weights for policy 1, policy_version 912264 (0.0007) [2023-12-27 11:58:20,346][20347] Updated weights for policy 0, policy_version 912901 (0.0008) [2023-12-27 11:58:20,400][20381] Updated weights for policy 1, policy_version 912274 (0.0006) [2023-12-27 11:58:20,408][20347] Updated weights for policy 0, policy_version 912911 (0.0008) [2023-12-27 11:58:21,076][20347] Updated weights for policy 0, policy_version 912921 (0.0010) [2023-12-27 11:58:21,085][20381] Updated weights for policy 1, policy_version 912284 (0.0007) [2023-12-27 11:58:21,141][20347] Updated weights for policy 0, policy_version 912931 (0.0008) [2023-12-27 11:58:21,153][20381] Updated weights for policy 1, policy_version 912294 (0.0008) [2023-12-27 11:58:21,207][20347] Updated weights for policy 0, policy_version 912941 (0.0011) [2023-12-27 11:58:21,210][20381] Updated weights for policy 1, policy_version 912304 (0.0008) [2023-12-27 11:58:21,972][20381] Updated weights for policy 1, policy_version 912314 (0.0008) [2023-12-27 11:58:21,989][20347] Updated weights for policy 0, policy_version 912951 (0.0009) [2023-12-27 11:58:22,036][20381] Updated weights for policy 1, policy_version 912324 (0.0007) [2023-12-27 11:58:22,046][20347] Updated weights for policy 0, policy_version 912961 (0.0006) [2023-12-27 11:58:22,093][20381] Updated weights for policy 1, policy_version 912334 (0.0007) [2023-12-27 11:58:22,106][20347] Updated weights for policy 0, policy_version 912971 (0.0006) [2023-12-27 11:58:22,158][20381] Updated weights for policy 1, policy_version 912344 (0.0007) [2023-12-27 11:58:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 467337216. Throughput: 0: 9990.5, 1: 10320.9. Samples: 467305048. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:22,363][19170] Avg episode reward: [(0, '990.260'), (1, '617.980')] [2023-12-27 11:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000912344_233586688.pth... [2023-12-27 11:58:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000912976_233750528.pth... [2023-12-27 11:58:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000911792_233447424.pth [2023-12-27 11:58:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000911128_233275392.pth [2023-12-27 11:58:22,761][20347] Updated weights for policy 0, policy_version 912981 (0.0007) [2023-12-27 11:58:22,815][20347] Updated weights for policy 0, policy_version 912991 (0.0008) [2023-12-27 11:58:22,869][20381] Updated weights for policy 1, policy_version 912354 (0.0007) [2023-12-27 11:58:22,871][20347] Updated weights for policy 0, policy_version 913001 (0.0007) [2023-12-27 11:58:22,957][20381] Updated weights for policy 1, policy_version 912364 (0.0008) [2023-12-27 11:58:23,024][20381] Updated weights for policy 1, policy_version 912374 (0.0008) [2023-12-27 11:58:23,547][20347] Updated weights for policy 0, policy_version 913011 (0.0009) [2023-12-27 11:58:23,592][20347] Updated weights for policy 0, policy_version 913021 (0.0006) [2023-12-27 11:58:23,648][20347] Updated weights for policy 0, policy_version 913031 (0.0005) [2023-12-27 11:58:23,698][20381] Updated weights for policy 1, policy_version 912384 (0.0007) [2023-12-27 11:58:23,750][20381] Updated weights for policy 1, policy_version 912394 (0.0008) [2023-12-27 11:58:23,807][20381] Updated weights for policy 1, policy_version 912404 (0.0008) [2023-12-27 11:58:24,213][20347] Updated weights for policy 0, policy_version 913041 (0.0005) [2023-12-27 11:58:24,266][20347] Updated weights for policy 0, policy_version 913051 (0.0008) [2023-12-27 11:58:24,328][20347] Updated weights for policy 0, policy_version 913061 (0.0010) [2023-12-27 11:58:24,392][20347] Updated weights for policy 0, policy_version 913071 (0.0010) [2023-12-27 11:58:24,418][20381] Updated weights for policy 1, policy_version 912414 (0.0006) [2023-12-27 11:58:24,466][20381] Updated weights for policy 1, policy_version 912424 (0.0007) [2023-12-27 11:58:24,513][20381] Updated weights for policy 1, policy_version 912434 (0.0007) [2023-12-27 11:58:25,019][20347] Updated weights for policy 0, policy_version 913081 (0.0008) [2023-12-27 11:58:25,067][20347] Updated weights for policy 0, policy_version 913091 (0.0006) [2023-12-27 11:58:25,118][20347] Updated weights for policy 0, policy_version 913101 (0.0010) [2023-12-27 11:58:25,181][20381] Updated weights for policy 1, policy_version 912444 (0.0008) [2023-12-27 11:58:25,237][20381] Updated weights for policy 1, policy_version 912454 (0.0009) [2023-12-27 11:58:25,293][20381] Updated weights for policy 1, policy_version 912464 (0.0008) [2023-12-27 11:58:25,766][20347] Updated weights for policy 0, policy_version 913111 (0.0009) [2023-12-27 11:58:25,817][20347] Updated weights for policy 0, policy_version 913121 (0.0010) [2023-12-27 11:58:25,875][20347] Updated weights for policy 0, policy_version 913131 (0.0010) [2023-12-27 11:58:25,889][20381] Updated weights for policy 1, policy_version 912474 (0.0006) [2023-12-27 11:58:25,945][20381] Updated weights for policy 1, policy_version 912484 (0.0008) [2023-12-27 11:58:26,004][20381] Updated weights for policy 1, policy_version 912494 (0.0010) [2023-12-27 11:58:26,053][20381] Updated weights for policy 1, policy_version 912504 (0.0010) [2023-12-27 11:58:26,522][20347] Updated weights for policy 0, policy_version 913141 (0.0010) [2023-12-27 11:58:26,573][20347] Updated weights for policy 0, policy_version 913151 (0.0010) [2023-12-27 11:58:26,624][20347] Updated weights for policy 0, policy_version 913161 (0.0010) [2023-12-27 11:58:26,671][20381] Updated weights for policy 1, policy_version 912514 (0.0008) [2023-12-27 11:58:26,727][20381] Updated weights for policy 1, policy_version 912524 (0.0007) [2023-12-27 11:58:26,777][20381] Updated weights for policy 1, policy_version 912534 (0.0007) [2023-12-27 11:58:27,312][20347] Updated weights for policy 0, policy_version 913171 (0.0010) [2023-12-27 11:58:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 467435520. Throughput: 0: 10054.8, 1: 10295.8. Samples: 467428484. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:27,363][20347] Updated weights for policy 0, policy_version 913181 (0.0010) [2023-12-27 11:58:27,363][19170] Avg episode reward: [(0, '951.810'), (1, '729.820')] [2023-12-27 11:58:27,409][20381] Updated weights for policy 1, policy_version 912544 (0.0005) [2023-12-27 11:58:27,411][20347] Updated weights for policy 0, policy_version 913191 (0.0010) [2023-12-27 11:58:27,461][20381] Updated weights for policy 1, policy_version 912554 (0.0006) [2023-12-27 11:58:27,517][20381] Updated weights for policy 1, policy_version 912564 (0.0006) [2023-12-27 11:58:28,010][20347] Updated weights for policy 0, policy_version 913201 (0.0010) [2023-12-27 11:58:28,063][20347] Updated weights for policy 0, policy_version 913211 (0.0005) [2023-12-27 11:58:28,119][20347] Updated weights for policy 0, policy_version 913221 (0.0005) [2023-12-27 11:58:28,128][20381] Updated weights for policy 1, policy_version 912574 (0.0006) [2023-12-27 11:58:28,169][20347] Updated weights for policy 0, policy_version 913231 (0.0009) [2023-12-27 11:58:28,183][20381] Updated weights for policy 1, policy_version 912584 (0.0010) [2023-12-27 11:58:28,236][20381] Updated weights for policy 1, policy_version 912594 (0.0010) [2023-12-27 11:58:28,840][20347] Updated weights for policy 0, policy_version 913241 (0.0010) [2023-12-27 11:58:28,900][20347] Updated weights for policy 0, policy_version 913251 (0.0010) [2023-12-27 11:58:28,905][20381] Updated weights for policy 1, policy_version 912604 (0.0009) [2023-12-27 11:58:28,956][20347] Updated weights for policy 0, policy_version 913261 (0.0010) [2023-12-27 11:58:28,961][20381] Updated weights for policy 1, policy_version 912614 (0.0006) [2023-12-27 11:58:29,012][20381] Updated weights for policy 1, policy_version 912624 (0.0007) [2023-12-27 11:58:29,621][20347] Updated weights for policy 0, policy_version 913271 (0.0007) [2023-12-27 11:58:29,686][20347] Updated weights for policy 0, policy_version 913281 (0.0007) [2023-12-27 11:58:29,706][20381] Updated weights for policy 1, policy_version 912634 (0.0007) [2023-12-27 11:58:29,742][20347] Updated weights for policy 0, policy_version 913291 (0.0006) [2023-12-27 11:58:29,758][20381] Updated weights for policy 1, policy_version 912644 (0.0007) [2023-12-27 11:58:29,807][20381] Updated weights for policy 1, policy_version 912654 (0.0007) [2023-12-27 11:58:29,870][20381] Updated weights for policy 1, policy_version 912664 (0.0008) [2023-12-27 11:58:30,389][20347] Updated weights for policy 0, policy_version 913301 (0.0007) [2023-12-27 11:58:30,441][20347] Updated weights for policy 0, policy_version 913311 (0.0007) [2023-12-27 11:58:30,493][20347] Updated weights for policy 0, policy_version 913321 (0.0007) [2023-12-27 11:58:30,585][20381] Updated weights for policy 1, policy_version 912674 (0.0005) [2023-12-27 11:58:30,635][20381] Updated weights for policy 1, policy_version 912684 (0.0006) [2023-12-27 11:58:30,683][20381] Updated weights for policy 1, policy_version 912694 (0.0009) [2023-12-27 11:58:31,108][20347] Updated weights for policy 0, policy_version 913331 (0.0007) [2023-12-27 11:58:31,173][20347] Updated weights for policy 0, policy_version 913341 (0.0008) [2023-12-27 11:58:31,225][20347] Updated weights for policy 0, policy_version 913351 (0.0007) [2023-12-27 11:58:31,323][20381] Updated weights for policy 1, policy_version 912704 (0.0008) [2023-12-27 11:58:31,388][20381] Updated weights for policy 1, policy_version 912714 (0.0010) [2023-12-27 11:58:31,447][20381] Updated weights for policy 1, policy_version 912724 (0.0007) [2023-12-27 11:58:31,963][20347] Updated weights for policy 0, policy_version 913361 (0.0006) [2023-12-27 11:58:32,030][20347] Updated weights for policy 0, policy_version 913371 (0.0007) [2023-12-27 11:58:32,092][20347] Updated weights for policy 0, policy_version 913381 (0.0006) [2023-12-27 11:58:32,110][20381] Updated weights for policy 1, policy_version 912734 (0.0008) [2023-12-27 11:58:32,157][20347] Updated weights for policy 0, policy_version 913391 (0.0005) [2023-12-27 11:58:32,158][20381] Updated weights for policy 1, policy_version 912744 (0.0007) [2023-12-27 11:58:32,211][20381] Updated weights for policy 1, policy_version 912754 (0.0005) [2023-12-27 11:58:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 467550208. Throughput: 0: 10149.7, 1: 10309.5. Samples: 467554656. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:32,363][19170] Avg episode reward: [(0, '932.590'), (1, '656.000')] [2023-12-27 11:58:32,862][20347] Updated weights for policy 0, policy_version 913401 (0.0007) [2023-12-27 11:58:32,908][20347] Updated weights for policy 0, policy_version 913411 (0.0007) [2023-12-27 11:58:32,931][20381] Updated weights for policy 1, policy_version 912764 (0.0007) [2023-12-27 11:58:32,970][20347] Updated weights for policy 0, policy_version 913421 (0.0007) [2023-12-27 11:58:32,976][20381] Updated weights for policy 1, policy_version 912774 (0.0010) [2023-12-27 11:58:33,024][20381] Updated weights for policy 1, policy_version 912784 (0.0010) [2023-12-27 11:58:33,643][20347] Updated weights for policy 0, policy_version 913431 (0.0006) [2023-12-27 11:58:33,689][20347] Updated weights for policy 0, policy_version 913441 (0.0007) [2023-12-27 11:58:33,729][20381] Updated weights for policy 1, policy_version 912794 (0.0010) [2023-12-27 11:58:33,738][20347] Updated weights for policy 0, policy_version 913451 (0.0007) [2023-12-27 11:58:33,777][20381] Updated weights for policy 1, policy_version 912804 (0.0010) [2023-12-27 11:58:33,822][20381] Updated weights for policy 1, policy_version 912814 (0.0010) [2023-12-27 11:58:33,867][20381] Updated weights for policy 1, policy_version 912824 (0.0010) [2023-12-27 11:58:34,454][20347] Updated weights for policy 0, policy_version 913461 (0.0006) [2023-12-27 11:58:34,508][20347] Updated weights for policy 0, policy_version 913471 (0.0007) [2023-12-27 11:58:34,561][20347] Updated weights for policy 0, policy_version 913481 (0.0007) [2023-12-27 11:58:34,626][20381] Updated weights for policy 1, policy_version 912834 (0.0007) [2023-12-27 11:58:34,681][20381] Updated weights for policy 1, policy_version 912844 (0.0008) [2023-12-27 11:58:34,730][20381] Updated weights for policy 1, policy_version 912854 (0.0008) [2023-12-27 11:58:35,284][20347] Updated weights for policy 0, policy_version 913491 (0.0006) [2023-12-27 11:58:35,340][20347] Updated weights for policy 0, policy_version 913501 (0.0007) [2023-12-27 11:58:35,342][20381] Updated weights for policy 1, policy_version 912864 (0.0006) [2023-12-27 11:58:35,392][20347] Updated weights for policy 0, policy_version 913511 (0.0006) [2023-12-27 11:58:35,402][20381] Updated weights for policy 1, policy_version 912874 (0.0005) [2023-12-27 11:58:35,456][20381] Updated weights for policy 1, policy_version 912884 (0.0007) [2023-12-27 11:58:36,067][20381] Updated weights for policy 1, policy_version 912894 (0.0006) [2023-12-27 11:58:36,113][20347] Updated weights for policy 0, policy_version 913521 (0.0006) [2023-12-27 11:58:36,129][20381] Updated weights for policy 1, policy_version 912904 (0.0007) [2023-12-27 11:58:36,177][20347] Updated weights for policy 0, policy_version 913531 (0.0008) [2023-12-27 11:58:36,188][20381] Updated weights for policy 1, policy_version 912914 (0.0006) [2023-12-27 11:58:36,232][20347] Updated weights for policy 0, policy_version 913541 (0.0007) [2023-12-27 11:58:36,281][20347] Updated weights for policy 0, policy_version 913551 (0.0008) [2023-12-27 11:58:36,866][20381] Updated weights for policy 1, policy_version 912924 (0.0006) [2023-12-27 11:58:36,920][20381] Updated weights for policy 1, policy_version 912934 (0.0008) [2023-12-27 11:58:36,978][20381] Updated weights for policy 1, policy_version 912944 (0.0008) [2023-12-27 11:58:36,993][20347] Updated weights for policy 0, policy_version 913561 (0.0010) [2023-12-27 11:58:37,046][20347] Updated weights for policy 0, policy_version 913571 (0.0010) [2023-12-27 11:58:37,101][20347] Updated weights for policy 0, policy_version 913581 (0.0010) [2023-12-27 11:58:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 467648512. Throughput: 0: 10132.1, 1: 10314.5. Samples: 467615772. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:37,364][19170] Avg episode reward: [(0, '961.250'), (1, '665.710')] [2023-12-27 11:58:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000912952_233742336.pth... [2023-12-27 11:58:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000913584_233906176.pth... [2023-12-27 11:58:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000912400_233603072.pth [2023-12-27 11:58:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000911736_233431040.pth [2023-12-27 11:58:37,698][20381] Updated weights for policy 1, policy_version 912954 (0.0009) [2023-12-27 11:58:37,755][20381] Updated weights for policy 1, policy_version 912964 (0.0007) [2023-12-27 11:58:37,778][20347] Updated weights for policy 0, policy_version 913591 (0.0007) [2023-12-27 11:58:37,812][20381] Updated weights for policy 1, policy_version 912974 (0.0007) [2023-12-27 11:58:37,835][20347] Updated weights for policy 0, policy_version 913601 (0.0008) [2023-12-27 11:58:37,863][20381] Updated weights for policy 1, policy_version 912984 (0.0006) [2023-12-27 11:58:37,891][20347] Updated weights for policy 0, policy_version 913611 (0.0010) [2023-12-27 11:58:38,536][20347] Updated weights for policy 0, policy_version 913621 (0.0008) [2023-12-27 11:58:38,590][20347] Updated weights for policy 0, policy_version 913631 (0.0006) [2023-12-27 11:58:38,591][20381] Updated weights for policy 1, policy_version 912994 (0.0006) [2023-12-27 11:58:38,648][20381] Updated weights for policy 1, policy_version 913004 (0.0007) [2023-12-27 11:58:38,650][20347] Updated weights for policy 0, policy_version 913641 (0.0006) [2023-12-27 11:58:38,706][20381] Updated weights for policy 1, policy_version 913014 (0.0006) [2023-12-27 11:58:39,337][20347] Updated weights for policy 0, policy_version 913651 (0.0007) [2023-12-27 11:58:39,405][20347] Updated weights for policy 0, policy_version 913661 (0.0008) [2023-12-27 11:58:39,428][20381] Updated weights for policy 1, policy_version 913024 (0.0008) [2023-12-27 11:58:39,465][20347] Updated weights for policy 0, policy_version 913671 (0.0007) [2023-12-27 11:58:39,491][20381] Updated weights for policy 1, policy_version 913034 (0.0008) [2023-12-27 11:58:39,556][20381] Updated weights for policy 1, policy_version 913044 (0.0008) [2023-12-27 11:58:40,135][20347] Updated weights for policy 0, policy_version 913681 (0.0007) [2023-12-27 11:58:40,198][20347] Updated weights for policy 0, policy_version 913691 (0.0008) [2023-12-27 11:58:40,264][20347] Updated weights for policy 0, policy_version 913701 (0.0008) [2023-12-27 11:58:40,306][20381] Updated weights for policy 1, policy_version 913054 (0.0007) [2023-12-27 11:58:40,321][20347] Updated weights for policy 0, policy_version 913711 (0.0007) [2023-12-27 11:58:40,364][20381] Updated weights for policy 1, policy_version 913064 (0.0007) [2023-12-27 11:58:40,426][20381] Updated weights for policy 1, policy_version 913074 (0.0008) [2023-12-27 11:58:41,014][20347] Updated weights for policy 0, policy_version 913721 (0.0007) [2023-12-27 11:58:41,082][20347] Updated weights for policy 0, policy_version 913731 (0.0008) [2023-12-27 11:58:41,128][20381] Updated weights for policy 1, policy_version 913084 (0.0008) [2023-12-27 11:58:41,152][20347] Updated weights for policy 0, policy_version 913741 (0.0008) [2023-12-27 11:58:41,189][20381] Updated weights for policy 1, policy_version 913094 (0.0007) [2023-12-27 11:58:41,253][20381] Updated weights for policy 1, policy_version 913104 (0.0006) [2023-12-27 11:58:41,860][20347] Updated weights for policy 0, policy_version 913751 (0.0006) [2023-12-27 11:58:41,922][20347] Updated weights for policy 0, policy_version 913761 (0.0006) [2023-12-27 11:58:41,976][20347] Updated weights for policy 0, policy_version 913771 (0.0008) [2023-12-27 11:58:42,058][20381] Updated weights for policy 1, policy_version 913114 (0.0008) [2023-12-27 11:58:42,119][20381] Updated weights for policy 1, policy_version 913124 (0.0008) [2023-12-27 11:58:42,183][20381] Updated weights for policy 1, policy_version 913134 (0.0008) [2023-12-27 11:58:42,245][20381] Updated weights for policy 1, policy_version 913144 (0.0008) [2023-12-27 11:58:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 467746816. Throughput: 0: 10117.6, 1: 10204.3. Samples: 467735064. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:42,363][19170] Avg episode reward: [(0, '951.700'), (1, '648.080')] [2023-12-27 11:58:42,660][20347] Updated weights for policy 0, policy_version 913781 (0.0008) [2023-12-27 11:58:42,721][20347] Updated weights for policy 0, policy_version 913791 (0.0007) [2023-12-27 11:58:42,775][20347] Updated weights for policy 0, policy_version 913801 (0.0008) [2023-12-27 11:58:43,001][20381] Updated weights for policy 1, policy_version 913154 (0.0011) [2023-12-27 11:58:43,058][20381] Updated weights for policy 1, policy_version 913164 (0.0010) [2023-12-27 11:58:43,110][20381] Updated weights for policy 1, policy_version 913174 (0.0010) [2023-12-27 11:58:43,418][20347] Updated weights for policy 0, policy_version 913811 (0.0007) [2023-12-27 11:58:43,472][20347] Updated weights for policy 0, policy_version 913821 (0.0008) [2023-12-27 11:58:43,527][20347] Updated weights for policy 0, policy_version 913831 (0.0007) [2023-12-27 11:58:43,792][20381] Updated weights for policy 1, policy_version 913184 (0.0007) [2023-12-27 11:58:43,849][20381] Updated weights for policy 1, policy_version 913194 (0.0007) [2023-12-27 11:58:43,913][20381] Updated weights for policy 1, policy_version 913204 (0.0007) [2023-12-27 11:58:44,171][20347] Updated weights for policy 0, policy_version 913841 (0.0007) [2023-12-27 11:58:44,234][20347] Updated weights for policy 0, policy_version 913851 (0.0007) [2023-12-27 11:58:44,292][20347] Updated weights for policy 0, policy_version 913862 (0.0010) [2023-12-27 11:58:44,341][20347] Updated weights for policy 0, policy_version 913872 (0.0009) [2023-12-27 11:58:44,553][20381] Updated weights for policy 1, policy_version 913214 (0.0006) [2023-12-27 11:58:44,604][20381] Updated weights for policy 1, policy_version 913224 (0.0007) [2023-12-27 11:58:44,651][20381] Updated weights for policy 1, policy_version 913234 (0.0007) [2023-12-27 11:58:45,042][20347] Updated weights for policy 0, policy_version 913882 (0.0010) [2023-12-27 11:58:45,098][20347] Updated weights for policy 0, policy_version 913892 (0.0010) [2023-12-27 11:58:45,151][20347] Updated weights for policy 0, policy_version 913902 (0.0010) [2023-12-27 11:58:45,366][20381] Updated weights for policy 1, policy_version 913244 (0.0007) [2023-12-27 11:58:45,424][20381] Updated weights for policy 1, policy_version 913254 (0.0008) [2023-12-27 11:58:45,475][20381] Updated weights for policy 1, policy_version 913264 (0.0006) [2023-12-27 11:58:45,798][20347] Updated weights for policy 0, policy_version 913912 (0.0006) [2023-12-27 11:58:45,812][20291] KL-divergence is very high: 144.6219 [2023-12-27 11:58:45,824][20291] KL-divergence is very high: 137.7185 [2023-12-27 11:58:45,857][20347] Updated weights for policy 0, policy_version 913922 (0.0006) [2023-12-27 11:58:45,923][20347] Updated weights for policy 0, policy_version 913932 (0.0005) [2023-12-27 11:58:46,107][20381] Updated weights for policy 1, policy_version 913274 (0.0006) [2023-12-27 11:58:46,153][20381] Updated weights for policy 1, policy_version 913284 (0.0009) [2023-12-27 11:58:46,201][20381] Updated weights for policy 1, policy_version 913294 (0.0007) [2023-12-27 11:58:46,252][20381] Updated weights for policy 1, policy_version 913304 (0.0005) [2023-12-27 11:58:46,508][20347] Updated weights for policy 0, policy_version 913942 (0.0006) [2023-12-27 11:58:46,564][20347] Updated weights for policy 0, policy_version 913952 (0.0007) [2023-12-27 11:58:46,620][20347] Updated weights for policy 0, policy_version 913962 (0.0007) [2023-12-27 11:58:46,934][20381] Updated weights for policy 1, policy_version 913314 (0.0011) [2023-12-27 11:58:47,004][20381] Updated weights for policy 1, policy_version 913324 (0.0011) [2023-12-27 11:58:47,068][20381] Updated weights for policy 1, policy_version 913334 (0.0011) [2023-12-27 11:58:47,327][20347] Updated weights for policy 0, policy_version 913972 (0.0007) [2023-12-27 11:58:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 467845120. Throughput: 0: 10201.2, 1: 10211.7. Samples: 467857556. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:47,363][19170] Avg episode reward: [(0, '961.630'), (1, '446.010')] [2023-12-27 11:58:47,383][20347] Updated weights for policy 0, policy_version 913982 (0.0007) [2023-12-27 11:58:47,442][20347] Updated weights for policy 0, policy_version 913992 (0.0007) [2023-12-27 11:58:47,761][20381] Updated weights for policy 1, policy_version 913344 (0.0011) [2023-12-27 11:58:47,811][20381] Updated weights for policy 1, policy_version 913354 (0.0010) [2023-12-27 11:58:47,857][20381] Updated weights for policy 1, policy_version 913364 (0.0011) [2023-12-27 11:58:48,149][20347] Updated weights for policy 0, policy_version 914002 (0.0007) [2023-12-27 11:58:48,204][20347] Updated weights for policy 0, policy_version 914012 (0.0007) [2023-12-27 11:58:48,256][20347] Updated weights for policy 0, policy_version 914022 (0.0008) [2023-12-27 11:58:48,305][20347] Updated weights for policy 0, policy_version 914032 (0.0007) [2023-12-27 11:58:48,618][20381] Updated weights for policy 1, policy_version 913374 (0.0009) [2023-12-27 11:58:48,674][20381] Updated weights for policy 1, policy_version 913384 (0.0008) [2023-12-27 11:58:48,728][20381] Updated weights for policy 1, policy_version 913394 (0.0009) [2023-12-27 11:58:48,965][20347] Updated weights for policy 0, policy_version 914042 (0.0008) [2023-12-27 11:58:49,019][20347] Updated weights for policy 0, policy_version 914052 (0.0007) [2023-12-27 11:58:49,075][20347] Updated weights for policy 0, policy_version 914062 (0.0007) [2023-12-27 11:58:49,477][20381] Updated weights for policy 1, policy_version 913404 (0.0007) [2023-12-27 11:58:49,529][20381] Updated weights for policy 1, policy_version 913414 (0.0009) [2023-12-27 11:58:49,578][20381] Updated weights for policy 1, policy_version 913424 (0.0005) [2023-12-27 11:58:49,759][20347] Updated weights for policy 0, policy_version 914072 (0.0006) [2023-12-27 11:58:49,820][20347] Updated weights for policy 0, policy_version 914082 (0.0007) [2023-12-27 11:58:49,886][20347] Updated weights for policy 0, policy_version 914092 (0.0008) [2023-12-27 11:58:50,223][20381] Updated weights for policy 1, policy_version 913434 (0.0006) [2023-12-27 11:58:50,276][20381] Updated weights for policy 1, policy_version 913444 (0.0011) [2023-12-27 11:58:50,334][20381] Updated weights for policy 1, policy_version 913454 (0.0011) [2023-12-27 11:58:50,392][20381] Updated weights for policy 1, policy_version 913464 (0.0011) [2023-12-27 11:58:50,557][20347] Updated weights for policy 0, policy_version 914102 (0.0008) [2023-12-27 11:58:50,620][20347] Updated weights for policy 0, policy_version 914112 (0.0010) [2023-12-27 11:58:50,672][20347] Updated weights for policy 0, policy_version 914122 (0.0010) [2023-12-27 11:58:51,099][20381] Updated weights for policy 1, policy_version 913474 (0.0011) [2023-12-27 11:58:51,160][20381] Updated weights for policy 1, policy_version 913484 (0.0010) [2023-12-27 11:58:51,219][20381] Updated weights for policy 1, policy_version 913494 (0.0006) [2023-12-27 11:58:51,411][20347] Updated weights for policy 0, policy_version 914132 (0.0009) [2023-12-27 11:58:51,470][20347] Updated weights for policy 0, policy_version 914142 (0.0008) [2023-12-27 11:58:51,526][20347] Updated weights for policy 0, policy_version 914152 (0.0007) [2023-12-27 11:58:51,944][20381] Updated weights for policy 1, policy_version 913504 (0.0008) [2023-12-27 11:58:52,000][20381] Updated weights for policy 1, policy_version 913514 (0.0008) [2023-12-27 11:58:52,056][20381] Updated weights for policy 1, policy_version 913524 (0.0009) [2023-12-27 11:58:52,188][20347] Updated weights for policy 0, policy_version 914162 (0.0009) [2023-12-27 11:58:52,256][20347] Updated weights for policy 0, policy_version 914172 (0.0007) [2023-12-27 11:58:52,316][20347] Updated weights for policy 0, policy_version 914182 (0.0010) [2023-12-27 11:58:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 467943424. Throughput: 0: 10176.4, 1: 10171.7. Samples: 467917832. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:52,363][19170] Avg episode reward: [(0, '980.770'), (1, '352.320')] [2023-12-27 11:58:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000913528_233889792.pth... [2023-12-27 11:58:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000912344_233586688.pth [2023-12-27 11:58:52,378][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000914192_234061824.pth... [2023-12-27 11:58:52,379][20347] Updated weights for policy 0, policy_version 914192 (0.0010) [2023-12-27 11:58:52,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000912976_233750528.pth [2023-12-27 11:58:52,776][20381] Updated weights for policy 1, policy_version 913534 (0.0009) [2023-12-27 11:58:52,829][20381] Updated weights for policy 1, policy_version 913544 (0.0009) [2023-12-27 11:58:52,881][20381] Updated weights for policy 1, policy_version 913554 (0.0008) [2023-12-27 11:58:52,975][20291] KL-divergence is very high: 353.2996 [2023-12-27 11:58:52,977][20347] Updated weights for policy 0, policy_version 914202 (0.0011) [2023-12-27 11:58:53,028][20347] Updated weights for policy 0, policy_version 914212 (0.0010) [2023-12-27 11:58:53,073][20347] Updated weights for policy 0, policy_version 914222 (0.0010) [2023-12-27 11:58:53,510][20381] Updated weights for policy 1, policy_version 913564 (0.0007) [2023-12-27 11:58:53,555][20381] Updated weights for policy 1, policy_version 913574 (0.0005) [2023-12-27 11:58:53,606][20381] Updated weights for policy 1, policy_version 913584 (0.0006) [2023-12-27 11:58:53,724][20347] Updated weights for policy 0, policy_version 914232 (0.0008) [2023-12-27 11:58:53,772][20347] Updated weights for policy 0, policy_version 914242 (0.0008) [2023-12-27 11:58:53,828][20347] Updated weights for policy 0, policy_version 914252 (0.0007) [2023-12-27 11:58:54,201][20381] Updated weights for policy 1, policy_version 913594 (0.0006) [2023-12-27 11:58:54,266][20381] Updated weights for policy 1, policy_version 913604 (0.0008) [2023-12-27 11:58:54,327][20381] Updated weights for policy 1, policy_version 913614 (0.0008) [2023-12-27 11:58:54,390][20381] Updated weights for policy 1, policy_version 913624 (0.0008) [2023-12-27 11:58:54,480][20347] Updated weights for policy 0, policy_version 914262 (0.0007) [2023-12-27 11:58:54,534][20347] Updated weights for policy 0, policy_version 914272 (0.0008) [2023-12-27 11:58:54,589][20347] Updated weights for policy 0, policy_version 914282 (0.0007) [2023-12-27 11:58:55,028][20381] Updated weights for policy 1, policy_version 913634 (0.0007) [2023-12-27 11:58:55,072][20381] Updated weights for policy 1, policy_version 913644 (0.0009) [2023-12-27 11:58:55,120][20381] Updated weights for policy 1, policy_version 913654 (0.0007) [2023-12-27 11:58:55,191][20347] Updated weights for policy 0, policy_version 914292 (0.0007) [2023-12-27 11:58:55,246][20347] Updated weights for policy 0, policy_version 914302 (0.0008) [2023-12-27 11:58:55,292][20347] Updated weights for policy 0, policy_version 914312 (0.0008) [2023-12-27 11:58:55,795][20381] Updated weights for policy 1, policy_version 913664 (0.0008) [2023-12-27 11:58:55,848][20381] Updated weights for policy 1, policy_version 913674 (0.0008) [2023-12-27 11:58:55,896][20381] Updated weights for policy 1, policy_version 913684 (0.0009) [2023-12-27 11:58:55,973][20347] Updated weights for policy 0, policy_version 914322 (0.0008) [2023-12-27 11:58:56,026][20347] Updated weights for policy 0, policy_version 914332 (0.0006) [2023-12-27 11:58:56,091][20347] Updated weights for policy 0, policy_version 914342 (0.0006) [2023-12-27 11:58:56,156][20347] Updated weights for policy 0, policy_version 914352 (0.0005) [2023-12-27 11:58:56,613][20381] Updated weights for policy 1, policy_version 913694 (0.0008) [2023-12-27 11:58:56,659][20381] Updated weights for policy 1, policy_version 913704 (0.0007) [2023-12-27 11:58:56,700][20381] Updated weights for policy 1, policy_version 913714 (0.0007) [2023-12-27 11:58:56,769][20347] Updated weights for policy 0, policy_version 914362 (0.0010) [2023-12-27 11:58:56,820][20347] Updated weights for policy 0, policy_version 914372 (0.0008) [2023-12-27 11:58:56,881][20347] Updated weights for policy 0, policy_version 914382 (0.0005) [2023-12-27 11:58:57,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 468049920. Throughput: 0: 10260.7, 1: 10167.7. Samples: 468042092. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:58:57,364][19170] Avg episode reward: [(0, '970.850'), (1, '558.530')] [2023-12-27 11:58:57,400][20381] Updated weights for policy 1, policy_version 913724 (0.0009) [2023-12-27 11:58:57,413][20347] Updated weights for policy 0, policy_version 914392 (0.0009) [2023-12-27 11:58:57,459][20381] Updated weights for policy 1, policy_version 913734 (0.0006) [2023-12-27 11:58:57,468][20347] Updated weights for policy 0, policy_version 914402 (0.0009) [2023-12-27 11:58:57,519][20347] Updated weights for policy 0, policy_version 914412 (0.0010) [2023-12-27 11:58:57,519][20381] Updated weights for policy 1, policy_version 913744 (0.0005) [2023-12-27 11:58:58,129][20347] Updated weights for policy 0, policy_version 914422 (0.0009) [2023-12-27 11:58:58,164][20381] Updated weights for policy 1, policy_version 913754 (0.0008) [2023-12-27 11:58:58,190][20347] Updated weights for policy 0, policy_version 914432 (0.0007) [2023-12-27 11:58:58,227][20381] Updated weights for policy 1, policy_version 913764 (0.0011) [2023-12-27 11:58:58,253][20347] Updated weights for policy 0, policy_version 914442 (0.0006) [2023-12-27 11:58:58,288][20381] Updated weights for policy 1, policy_version 913774 (0.0011) [2023-12-27 11:58:58,353][20381] Updated weights for policy 1, policy_version 913784 (0.0010) [2023-12-27 11:58:59,020][20347] Updated weights for policy 0, policy_version 914452 (0.0006) [2023-12-27 11:58:59,081][20347] Updated weights for policy 0, policy_version 914462 (0.0007) [2023-12-27 11:58:59,093][20381] Updated weights for policy 1, policy_version 913794 (0.0007) [2023-12-27 11:58:59,145][20347] Updated weights for policy 0, policy_version 914472 (0.0007) [2023-12-27 11:58:59,154][20381] Updated weights for policy 1, policy_version 913804 (0.0009) [2023-12-27 11:58:59,220][20381] Updated weights for policy 1, policy_version 913814 (0.0010) [2023-12-27 11:58:59,894][20347] Updated weights for policy 0, policy_version 914482 (0.0008) [2023-12-27 11:58:59,954][20347] Updated weights for policy 0, policy_version 914492 (0.0008) [2023-12-27 11:58:59,973][20381] Updated weights for policy 1, policy_version 913824 (0.0007) [2023-12-27 11:59:00,011][20347] Updated weights for policy 0, policy_version 914502 (0.0007) [2023-12-27 11:59:00,040][20381] Updated weights for policy 1, policy_version 913834 (0.0006) [2023-12-27 11:59:00,063][20347] Updated weights for policy 0, policy_version 914512 (0.0007) [2023-12-27 11:59:00,103][20381] Updated weights for policy 1, policy_version 913844 (0.0006) [2023-12-27 11:59:00,665][20381] Updated weights for policy 1, policy_version 913854 (0.0005) [2023-12-27 11:59:00,710][20381] Updated weights for policy 1, policy_version 913864 (0.0005) [2023-12-27 11:59:00,764][20381] Updated weights for policy 1, policy_version 913874 (0.0008) [2023-12-27 11:59:00,777][20347] Updated weights for policy 0, policy_version 914522 (0.0007) [2023-12-27 11:59:00,831][20347] Updated weights for policy 0, policy_version 914532 (0.0007) [2023-12-27 11:59:00,876][20347] Updated weights for policy 0, policy_version 914542 (0.0008) [2023-12-27 11:59:01,433][20381] Updated weights for policy 1, policy_version 913884 (0.0008) [2023-12-27 11:59:01,485][20381] Updated weights for policy 1, policy_version 913894 (0.0008) [2023-12-27 11:59:01,549][20381] Updated weights for policy 1, policy_version 913904 (0.0008) [2023-12-27 11:59:01,624][20347] Updated weights for policy 0, policy_version 914552 (0.0008) [2023-12-27 11:59:01,689][20347] Updated weights for policy 0, policy_version 914562 (0.0007) [2023-12-27 11:59:01,756][20347] Updated weights for policy 0, policy_version 914572 (0.0007) [2023-12-27 11:59:02,212][20381] Updated weights for policy 1, policy_version 913914 (0.0008) [2023-12-27 11:59:02,276][20381] Updated weights for policy 1, policy_version 913924 (0.0009) [2023-12-27 11:59:02,339][20381] Updated weights for policy 1, policy_version 913934 (0.0006) [2023-12-27 11:59:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 468148224. Throughput: 0: 10269.3, 1: 10162.3. Samples: 468164504. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:02,363][19170] Avg episode reward: [(0, '980.560'), (1, '293.290')] [2023-12-27 11:59:02,402][20381] Updated weights for policy 1, policy_version 913944 (0.0010) [2023-12-27 11:59:02,442][20347] Updated weights for policy 0, policy_version 914582 (0.0007) [2023-12-27 11:59:02,500][20347] Updated weights for policy 0, policy_version 914592 (0.0007) [2023-12-27 11:59:02,559][20347] Updated weights for policy 0, policy_version 914602 (0.0007) [2023-12-27 11:59:03,085][20381] Updated weights for policy 1, policy_version 913954 (0.0010) [2023-12-27 11:59:03,142][20381] Updated weights for policy 1, policy_version 913964 (0.0010) [2023-12-27 11:59:03,196][20381] Updated weights for policy 1, policy_version 913974 (0.0007) [2023-12-27 11:59:03,247][20347] Updated weights for policy 0, policy_version 914612 (0.0008) [2023-12-27 11:59:03,296][20347] Updated weights for policy 0, policy_version 914622 (0.0008) [2023-12-27 11:59:03,348][20347] Updated weights for policy 0, policy_version 914632 (0.0010) [2023-12-27 11:59:03,809][20381] Updated weights for policy 1, policy_version 913984 (0.0007) [2023-12-27 11:59:03,868][20381] Updated weights for policy 1, policy_version 913994 (0.0009) [2023-12-27 11:59:03,915][20347] Updated weights for policy 0, policy_version 914642 (0.0010) [2023-12-27 11:59:03,920][20381] Updated weights for policy 1, policy_version 914004 (0.0008) [2023-12-27 11:59:03,974][20347] Updated weights for policy 0, policy_version 914652 (0.0007) [2023-12-27 11:59:04,029][20347] Updated weights for policy 0, policy_version 914662 (0.0005) [2023-12-27 11:59:04,081][20347] Updated weights for policy 0, policy_version 914672 (0.0010) [2023-12-27 11:59:04,625][20381] Updated weights for policy 1, policy_version 914014 (0.0009) [2023-12-27 11:59:04,687][20381] Updated weights for policy 1, policy_version 914024 (0.0011) [2023-12-27 11:59:04,742][20381] Updated weights for policy 1, policy_version 914034 (0.0011) [2023-12-27 11:59:04,747][20347] Updated weights for policy 0, policy_version 914682 (0.0010) [2023-12-27 11:59:04,806][20347] Updated weights for policy 0, policy_version 914692 (0.0010) [2023-12-27 11:59:04,868][20347] Updated weights for policy 0, policy_version 914702 (0.0010) [2023-12-27 11:59:05,414][20381] Updated weights for policy 1, policy_version 914044 (0.0011) [2023-12-27 11:59:05,459][20381] Updated weights for policy 1, policy_version 914054 (0.0010) [2023-12-27 11:59:05,501][20381] Updated weights for policy 1, policy_version 914064 (0.0010) [2023-12-27 11:59:05,523][20347] Updated weights for policy 0, policy_version 914712 (0.0008) [2023-12-27 11:59:05,576][20347] Updated weights for policy 0, policy_version 914722 (0.0007) [2023-12-27 11:59:05,623][20347] Updated weights for policy 0, policy_version 914732 (0.0007) [2023-12-27 11:59:06,214][20381] Updated weights for policy 1, policy_version 914074 (0.0010) [2023-12-27 11:59:06,242][20347] Updated weights for policy 0, policy_version 914742 (0.0007) [2023-12-27 11:59:06,271][20381] Updated weights for policy 1, policy_version 914084 (0.0011) [2023-12-27 11:59:06,304][20347] Updated weights for policy 0, policy_version 914752 (0.0008) [2023-12-27 11:59:06,331][20381] Updated weights for policy 1, policy_version 914094 (0.0010) [2023-12-27 11:59:06,366][20347] Updated weights for policy 0, policy_version 914762 (0.0006) [2023-12-27 11:59:06,394][20381] Updated weights for policy 1, policy_version 914104 (0.0011) [2023-12-27 11:59:07,007][20347] Updated weights for policy 0, policy_version 914772 (0.0005) [2023-12-27 11:59:07,053][20347] Updated weights for policy 0, policy_version 914782 (0.0007) [2023-12-27 11:59:07,062][20381] Updated weights for policy 1, policy_version 914114 (0.0005) [2023-12-27 11:59:07,109][20347] Updated weights for policy 0, policy_version 914792 (0.0006) [2023-12-27 11:59:07,115][20381] Updated weights for policy 1, policy_version 914124 (0.0007) [2023-12-27 11:59:07,171][20381] Updated weights for policy 1, policy_version 914134 (0.0011) [2023-12-27 11:59:07,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 468262912. Throughput: 0: 10319.4, 1: 10164.0. Samples: 468226800. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '297.500')] [2023-12-27 11:59:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000914136_234045440.pth... [2023-12-27 11:59:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000914800_234217472.pth... [2023-12-27 11:59:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000912952_233742336.pth [2023-12-27 11:59:07,386][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000913584_233906176.pth [2023-12-27 11:59:07,723][20347] Updated weights for policy 0, policy_version 914802 (0.0005) [2023-12-27 11:59:07,765][20381] Updated weights for policy 1, policy_version 914144 (0.0007) [2023-12-27 11:59:07,777][20347] Updated weights for policy 0, policy_version 914812 (0.0006) [2023-12-27 11:59:07,829][20381] Updated weights for policy 1, policy_version 914154 (0.0008) [2023-12-27 11:59:07,830][20347] Updated weights for policy 0, policy_version 914822 (0.0005) [2023-12-27 11:59:07,884][20347] Updated weights for policy 0, policy_version 914832 (0.0005) [2023-12-27 11:59:07,893][20381] Updated weights for policy 1, policy_version 914164 (0.0008) [2023-12-27 11:59:08,555][20381] Updated weights for policy 1, policy_version 914174 (0.0008) [2023-12-27 11:59:08,556][20347] Updated weights for policy 0, policy_version 914842 (0.0007) [2023-12-27 11:59:08,608][20381] Updated weights for policy 1, policy_version 914184 (0.0008) [2023-12-27 11:59:08,614][20347] Updated weights for policy 0, policy_version 914852 (0.0006) [2023-12-27 11:59:08,667][20381] Updated weights for policy 1, policy_version 914194 (0.0008) [2023-12-27 11:59:08,671][20347] Updated weights for policy 0, policy_version 914862 (0.0007) [2023-12-27 11:59:09,302][20347] Updated weights for policy 0, policy_version 914872 (0.0007) [2023-12-27 11:59:09,371][20347] Updated weights for policy 0, policy_version 914882 (0.0008) [2023-12-27 11:59:09,412][20381] Updated weights for policy 1, policy_version 914204 (0.0008) [2023-12-27 11:59:09,436][20347] Updated weights for policy 0, policy_version 914892 (0.0008) [2023-12-27 11:59:09,471][20381] Updated weights for policy 1, policy_version 914214 (0.0006) [2023-12-27 11:59:09,534][20381] Updated weights for policy 1, policy_version 914224 (0.0008) [2023-12-27 11:59:10,089][20347] Updated weights for policy 0, policy_version 914902 (0.0008) [2023-12-27 11:59:10,146][20347] Updated weights for policy 0, policy_version 914912 (0.0007) [2023-12-27 11:59:10,206][20347] Updated weights for policy 0, policy_version 914922 (0.0008) [2023-12-27 11:59:10,260][20381] Updated weights for policy 1, policy_version 914234 (0.0008) [2023-12-27 11:59:10,322][20381] Updated weights for policy 1, policy_version 914244 (0.0008) [2023-12-27 11:59:10,383][20381] Updated weights for policy 1, policy_version 914254 (0.0008) [2023-12-27 11:59:10,444][20381] Updated weights for policy 1, policy_version 914264 (0.0007) [2023-12-27 11:59:10,837][20347] Updated weights for policy 0, policy_version 914932 (0.0007) [2023-12-27 11:59:10,897][20347] Updated weights for policy 0, policy_version 914942 (0.0006) [2023-12-27 11:59:10,949][20347] Updated weights for policy 0, policy_version 914952 (0.0006) [2023-12-27 11:59:11,161][20381] Updated weights for policy 1, policy_version 914274 (0.0008) [2023-12-27 11:59:11,218][20381] Updated weights for policy 1, policy_version 914284 (0.0007) [2023-12-27 11:59:11,286][20381] Updated weights for policy 1, policy_version 914294 (0.0007) [2023-12-27 11:59:11,714][20347] Updated weights for policy 0, policy_version 914962 (0.0006) [2023-12-27 11:59:11,774][20347] Updated weights for policy 0, policy_version 914972 (0.0007) [2023-12-27 11:59:11,831][20347] Updated weights for policy 0, policy_version 914982 (0.0006) [2023-12-27 11:59:11,881][20347] Updated weights for policy 0, policy_version 914992 (0.0006) [2023-12-27 11:59:12,039][20381] Updated weights for policy 1, policy_version 914304 (0.0007) [2023-12-27 11:59:12,101][20381] Updated weights for policy 1, policy_version 914314 (0.0008) [2023-12-27 11:59:12,154][20381] Updated weights for policy 1, policy_version 914324 (0.0008) [2023-12-27 11:59:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 468361216. Throughput: 0: 10363.9, 1: 10141.0. Samples: 468351204. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:12,363][19170] Avg episode reward: [(0, '990.410'), (1, '367.400')] [2023-12-27 11:59:12,573][20347] Updated weights for policy 0, policy_version 915002 (0.0008) [2023-12-27 11:59:12,624][20347] Updated weights for policy 0, policy_version 915012 (0.0008) [2023-12-27 11:59:12,674][20347] Updated weights for policy 0, policy_version 915022 (0.0008) [2023-12-27 11:59:12,861][20381] Updated weights for policy 1, policy_version 914334 (0.0006) [2023-12-27 11:59:12,915][20381] Updated weights for policy 1, policy_version 914344 (0.0007) [2023-12-27 11:59:12,974][20381] Updated weights for policy 1, policy_version 914354 (0.0008) [2023-12-27 11:59:13,408][20347] Updated weights for policy 0, policy_version 915032 (0.0008) [2023-12-27 11:59:13,469][20347] Updated weights for policy 0, policy_version 915042 (0.0008) [2023-12-27 11:59:13,515][20347] Updated weights for policy 0, policy_version 915052 (0.0007) [2023-12-27 11:59:13,657][20381] Updated weights for policy 1, policy_version 914364 (0.0008) [2023-12-27 11:59:13,721][20381] Updated weights for policy 1, policy_version 914374 (0.0008) [2023-12-27 11:59:13,785][20381] Updated weights for policy 1, policy_version 914384 (0.0009) [2023-12-27 11:59:14,104][20347] Updated weights for policy 0, policy_version 915062 (0.0008) [2023-12-27 11:59:14,160][20347] Updated weights for policy 0, policy_version 915072 (0.0008) [2023-12-27 11:59:14,205][20347] Updated weights for policy 0, policy_version 915082 (0.0008) [2023-12-27 11:59:14,501][20381] Updated weights for policy 1, policy_version 914394 (0.0007) [2023-12-27 11:59:14,560][20381] Updated weights for policy 1, policy_version 914404 (0.0008) [2023-12-27 11:59:14,617][20381] Updated weights for policy 1, policy_version 914414 (0.0007) [2023-12-27 11:59:14,675][20381] Updated weights for policy 1, policy_version 914424 (0.0008) [2023-12-27 11:59:14,823][20347] Updated weights for policy 0, policy_version 915092 (0.0008) [2023-12-27 11:59:14,878][20347] Updated weights for policy 0, policy_version 915102 (0.0008) [2023-12-27 11:59:14,941][20347] Updated weights for policy 0, policy_version 915112 (0.0008) [2023-12-27 11:59:15,426][20381] Updated weights for policy 1, policy_version 914434 (0.0008) [2023-12-27 11:59:15,475][20381] Updated weights for policy 1, policy_version 914444 (0.0008) [2023-12-27 11:59:15,528][20381] Updated weights for policy 1, policy_version 914454 (0.0008) [2023-12-27 11:59:15,663][20347] Updated weights for policy 0, policy_version 915122 (0.0008) [2023-12-27 11:59:15,711][20347] Updated weights for policy 0, policy_version 915132 (0.0007) [2023-12-27 11:59:15,762][20347] Updated weights for policy 0, policy_version 915142 (0.0009) [2023-12-27 11:59:15,815][20347] Updated weights for policy 0, policy_version 915152 (0.0008) [2023-12-27 11:59:16,232][20381] Updated weights for policy 1, policy_version 914464 (0.0010) [2023-12-27 11:59:16,285][20381] Updated weights for policy 1, policy_version 914474 (0.0010) [2023-12-27 11:59:16,335][20381] Updated weights for policy 1, policy_version 914484 (0.0006) [2023-12-27 11:59:16,456][20347] Updated weights for policy 0, policy_version 915162 (0.0005) [2023-12-27 11:59:16,517][20347] Updated weights for policy 0, policy_version 915172 (0.0005) [2023-12-27 11:59:16,577][20347] Updated weights for policy 0, policy_version 915182 (0.0006) [2023-12-27 11:59:16,984][20381] Updated weights for policy 1, policy_version 914494 (0.0008) [2023-12-27 11:59:17,042][20381] Updated weights for policy 1, policy_version 914504 (0.0010) [2023-12-27 11:59:17,104][20381] Updated weights for policy 1, policy_version 914514 (0.0011) [2023-12-27 11:59:17,247][20347] Updated weights for policy 0, policy_version 915192 (0.0010) [2023-12-27 11:59:17,309][20347] Updated weights for policy 0, policy_version 915202 (0.0010) [2023-12-27 11:59:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 468459520. Throughput: 0: 10351.7, 1: 10060.5. Samples: 468473208. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:17,363][19170] Avg episode reward: [(0, '990.410'), (1, '266.660')] [2023-12-27 11:59:17,370][20347] Updated weights for policy 0, policy_version 915212 (0.0010) [2023-12-27 11:59:17,785][20381] Updated weights for policy 1, policy_version 914524 (0.0009) [2023-12-27 11:59:17,845][20381] Updated weights for policy 1, policy_version 914534 (0.0006) [2023-12-27 11:59:17,897][20381] Updated weights for policy 1, policy_version 914544 (0.0006) [2023-12-27 11:59:17,962][20347] Updated weights for policy 0, policy_version 915222 (0.0010) [2023-12-27 11:59:18,018][20347] Updated weights for policy 0, policy_version 915232 (0.0010) [2023-12-27 11:59:18,077][20347] Updated weights for policy 0, policy_version 915242 (0.0010) [2023-12-27 11:59:18,551][20381] Updated weights for policy 1, policy_version 914554 (0.0007) [2023-12-27 11:59:18,619][20381] Updated weights for policy 1, policy_version 914564 (0.0006) [2023-12-27 11:59:18,687][20381] Updated weights for policy 1, policy_version 914574 (0.0006) [2023-12-27 11:59:18,739][20381] Updated weights for policy 1, policy_version 914584 (0.0007) [2023-12-27 11:59:18,800][20347] Updated weights for policy 0, policy_version 915252 (0.0010) [2023-12-27 11:59:18,853][20347] Updated weights for policy 0, policy_version 915262 (0.0010) [2023-12-27 11:59:18,911][20347] Updated weights for policy 0, policy_version 915272 (0.0010) [2023-12-27 11:59:19,467][20381] Updated weights for policy 1, policy_version 914594 (0.0008) [2023-12-27 11:59:19,537][20381] Updated weights for policy 1, policy_version 914604 (0.0007) [2023-12-27 11:59:19,607][20381] Updated weights for policy 1, policy_version 914614 (0.0007) [2023-12-27 11:59:19,616][20347] Updated weights for policy 0, policy_version 915282 (0.0010) [2023-12-27 11:59:19,671][20347] Updated weights for policy 0, policy_version 915292 (0.0009) [2023-12-27 11:59:19,734][20347] Updated weights for policy 0, policy_version 915302 (0.0011) [2023-12-27 11:59:19,791][20347] Updated weights for policy 0, policy_version 915312 (0.0010) [2023-12-27 11:59:20,321][20381] Updated weights for policy 1, policy_version 914624 (0.0007) [2023-12-27 11:59:20,370][20381] Updated weights for policy 1, policy_version 914634 (0.0007) [2023-12-27 11:59:20,425][20381] Updated weights for policy 1, policy_version 914644 (0.0007) [2023-12-27 11:59:20,486][20347] Updated weights for policy 0, policy_version 915322 (0.0010) [2023-12-27 11:59:20,537][20347] Updated weights for policy 0, policy_version 915332 (0.0010) [2023-12-27 11:59:20,600][20347] Updated weights for policy 0, policy_version 915342 (0.0010) [2023-12-27 11:59:21,086][20381] Updated weights for policy 1, policy_version 914654 (0.0006) [2023-12-27 11:59:21,154][20381] Updated weights for policy 1, policy_version 914664 (0.0008) [2023-12-27 11:59:21,219][20381] Updated weights for policy 1, policy_version 914674 (0.0007) [2023-12-27 11:59:21,320][20347] Updated weights for policy 0, policy_version 915352 (0.0008) [2023-12-27 11:59:21,389][20347] Updated weights for policy 0, policy_version 915362 (0.0008) [2023-12-27 11:59:21,458][20347] Updated weights for policy 0, policy_version 915372 (0.0008) [2023-12-27 11:59:21,943][20381] Updated weights for policy 1, policy_version 914684 (0.0009) [2023-12-27 11:59:22,007][20381] Updated weights for policy 1, policy_version 914694 (0.0011) [2023-12-27 11:59:22,067][20381] Updated weights for policy 1, policy_version 914704 (0.0011) [2023-12-27 11:59:22,126][20347] Updated weights for policy 0, policy_version 915382 (0.0006) [2023-12-27 11:59:22,192][20347] Updated weights for policy 0, policy_version 915392 (0.0008) [2023-12-27 11:59:22,270][20347] Updated weights for policy 0, policy_version 915402 (0.0010) [2023-12-27 11:59:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 468566016. Throughput: 0: 10352.7, 1: 10038.8. Samples: 468533384. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '296.240')] [2023-12-27 11:59:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000914712_234192896.pth... [2023-12-27 11:59:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000915408_234373120.pth... [2023-12-27 11:59:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000913528_233889792.pth [2023-12-27 11:59:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000914192_234061824.pth [2023-12-27 11:59:22,747][20381] Updated weights for policy 1, policy_version 914714 (0.0010) [2023-12-27 11:59:22,810][20381] Updated weights for policy 1, policy_version 914724 (0.0011) [2023-12-27 11:59:22,873][20381] Updated weights for policy 1, policy_version 914734 (0.0011) [2023-12-27 11:59:22,931][20381] Updated weights for policy 1, policy_version 914744 (0.0010) [2023-12-27 11:59:22,950][20347] Updated weights for policy 0, policy_version 915412 (0.0010) [2023-12-27 11:59:23,015][20347] Updated weights for policy 0, policy_version 915422 (0.0008) [2023-12-27 11:59:23,068][20347] Updated weights for policy 0, policy_version 915432 (0.0010) [2023-12-27 11:59:23,561][20381] Updated weights for policy 1, policy_version 914754 (0.0008) [2023-12-27 11:59:23,625][20381] Updated weights for policy 1, policy_version 914764 (0.0008) [2023-12-27 11:59:23,670][20347] Updated weights for policy 0, policy_version 915442 (0.0010) [2023-12-27 11:59:23,689][20381] Updated weights for policy 1, policy_version 914774 (0.0008) [2023-12-27 11:59:23,724][20347] Updated weights for policy 0, policy_version 915452 (0.0010) [2023-12-27 11:59:23,773][20347] Updated weights for policy 0, policy_version 915462 (0.0011) [2023-12-27 11:59:23,828][20347] Updated weights for policy 0, policy_version 915472 (0.0010) [2023-12-27 11:59:24,232][20381] Updated weights for policy 1, policy_version 914784 (0.0010) [2023-12-27 11:59:24,291][20381] Updated weights for policy 1, policy_version 914794 (0.0010) [2023-12-27 11:59:24,347][20381] Updated weights for policy 1, policy_version 914804 (0.0011) [2023-12-27 11:59:24,491][20347] Updated weights for policy 0, policy_version 915482 (0.0005) [2023-12-27 11:59:24,552][20347] Updated weights for policy 0, policy_version 915492 (0.0005) [2023-12-27 11:59:24,608][20347] Updated weights for policy 0, policy_version 915502 (0.0005) [2023-12-27 11:59:25,044][20381] Updated weights for policy 1, policy_version 914814 (0.0011) [2023-12-27 11:59:25,102][20381] Updated weights for policy 1, policy_version 914824 (0.0010) [2023-12-27 11:59:25,121][20347] Updated weights for policy 0, policy_version 915512 (0.0009) [2023-12-27 11:59:25,162][20381] Updated weights for policy 1, policy_version 914834 (0.0010) [2023-12-27 11:59:25,171][20347] Updated weights for policy 0, policy_version 915522 (0.0006) [2023-12-27 11:59:25,222][20347] Updated weights for policy 0, policy_version 915532 (0.0005) [2023-12-27 11:59:25,767][20347] Updated weights for policy 0, policy_version 915542 (0.0008) [2023-12-27 11:59:25,822][20347] Updated weights for policy 0, policy_version 915552 (0.0010) [2023-12-27 11:59:25,832][20381] Updated weights for policy 1, policy_version 914844 (0.0010) [2023-12-27 11:59:25,877][20347] Updated weights for policy 0, policy_version 915562 (0.0010) [2023-12-27 11:59:25,881][20381] Updated weights for policy 1, policy_version 914854 (0.0010) [2023-12-27 11:59:25,929][20381] Updated weights for policy 1, policy_version 914864 (0.0010) [2023-12-27 11:59:26,541][20381] Updated weights for policy 1, policy_version 914874 (0.0010) [2023-12-27 11:59:26,548][20347] Updated weights for policy 0, policy_version 915572 (0.0010) [2023-12-27 11:59:26,595][20381] Updated weights for policy 1, policy_version 914884 (0.0010) [2023-12-27 11:59:26,599][20347] Updated weights for policy 0, policy_version 915582 (0.0010) [2023-12-27 11:59:26,640][20381] Updated weights for policy 1, policy_version 914894 (0.0010) [2023-12-27 11:59:26,657][20347] Updated weights for policy 0, policy_version 915592 (0.0010) [2023-12-27 11:59:26,685][20381] Updated weights for policy 1, policy_version 914904 (0.0010) [2023-12-27 11:59:27,319][20347] Updated weights for policy 0, policy_version 915602 (0.0010) [2023-12-27 11:59:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 468664320. Throughput: 0: 10439.1, 1: 10111.1. Samples: 468659824. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '309.360')] [2023-12-27 11:59:27,368][20347] Updated weights for policy 0, policy_version 915612 (0.0006) [2023-12-27 11:59:27,369][20381] Updated weights for policy 1, policy_version 914914 (0.0010) [2023-12-27 11:59:27,422][20347] Updated weights for policy 0, policy_version 915622 (0.0005) [2023-12-27 11:59:27,424][20381] Updated weights for policy 1, policy_version 914924 (0.0010) [2023-12-27 11:59:27,475][20381] Updated weights for policy 1, policy_version 914934 (0.0010) [2023-12-27 11:59:27,486][20347] Updated weights for policy 0, policy_version 915632 (0.0006) [2023-12-27 11:59:28,153][20347] Updated weights for policy 0, policy_version 915642 (0.0007) [2023-12-27 11:59:28,153][20381] Updated weights for policy 1, policy_version 914944 (0.0010) [2023-12-27 11:59:28,207][20347] Updated weights for policy 0, policy_version 915652 (0.0006) [2023-12-27 11:59:28,212][20381] Updated weights for policy 1, policy_version 914954 (0.0010) [2023-12-27 11:59:28,262][20347] Updated weights for policy 0, policy_version 915662 (0.0005) [2023-12-27 11:59:28,270][20381] Updated weights for policy 1, policy_version 914964 (0.0010) [2023-12-27 11:59:28,886][20381] Updated weights for policy 1, policy_version 914974 (0.0010) [2023-12-27 11:59:28,910][20347] Updated weights for policy 0, policy_version 915672 (0.0009) [2023-12-27 11:59:28,940][20381] Updated weights for policy 1, policy_version 914984 (0.0010) [2023-12-27 11:59:28,964][20347] Updated weights for policy 0, policy_version 915682 (0.0010) [2023-12-27 11:59:28,991][20381] Updated weights for policy 1, policy_version 914994 (0.0010) [2023-12-27 11:59:29,018][20347] Updated weights for policy 0, policy_version 915692 (0.0010) [2023-12-27 11:59:29,602][20381] Updated weights for policy 1, policy_version 915004 (0.0009) [2023-12-27 11:59:29,654][20381] Updated weights for policy 1, policy_version 915014 (0.0010) [2023-12-27 11:59:29,714][20347] Updated weights for policy 0, policy_version 915702 (0.0009) [2023-12-27 11:59:29,715][20381] Updated weights for policy 1, policy_version 915024 (0.0008) [2023-12-27 11:59:29,774][20347] Updated weights for policy 0, policy_version 915712 (0.0007) [2023-12-27 11:59:29,841][20347] Updated weights for policy 0, policy_version 915722 (0.0009) [2023-12-27 11:59:30,418][20381] Updated weights for policy 1, policy_version 915034 (0.0008) [2023-12-27 11:59:30,468][20381] Updated weights for policy 1, policy_version 915044 (0.0008) [2023-12-27 11:59:30,518][20381] Updated weights for policy 1, policy_version 915054 (0.0005) [2023-12-27 11:59:30,565][20347] Updated weights for policy 0, policy_version 915732 (0.0010) [2023-12-27 11:59:30,576][20381] Updated weights for policy 1, policy_version 915064 (0.0008) [2023-12-27 11:59:30,616][20347] Updated weights for policy 0, policy_version 915742 (0.0010) [2023-12-27 11:59:30,663][20347] Updated weights for policy 0, policy_version 915752 (0.0010) [2023-12-27 11:59:31,206][20381] Updated weights for policy 1, policy_version 915074 (0.0008) [2023-12-27 11:59:31,263][20381] Updated weights for policy 1, policy_version 915084 (0.0009) [2023-12-27 11:59:31,324][20381] Updated weights for policy 1, policy_version 915094 (0.0006) [2023-12-27 11:59:31,345][20347] Updated weights for policy 0, policy_version 915762 (0.0010) [2023-12-27 11:59:31,414][20347] Updated weights for policy 0, policy_version 915772 (0.0008) [2023-12-27 11:59:31,470][20347] Updated weights for policy 0, policy_version 915782 (0.0009) [2023-12-27 11:59:31,530][20347] Updated weights for policy 0, policy_version 915792 (0.0008) [2023-12-27 11:59:31,939][20381] Updated weights for policy 1, policy_version 915104 (0.0006) [2023-12-27 11:59:32,002][20381] Updated weights for policy 1, policy_version 915114 (0.0007) [2023-12-27 11:59:32,059][20381] Updated weights for policy 1, policy_version 915124 (0.0007) [2023-12-27 11:59:32,280][20347] Updated weights for policy 0, policy_version 915802 (0.0010) [2023-12-27 11:59:32,340][20347] Updated weights for policy 0, policy_version 915812 (0.0008) [2023-12-27 11:59:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 468770816. Throughput: 0: 10418.6, 1: 10199.9. Samples: 468785388. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '360.570')] [2023-12-27 11:59:32,390][20347] Updated weights for policy 0, policy_version 915822 (0.0011) [2023-12-27 11:59:32,659][20381] Updated weights for policy 1, policy_version 915134 (0.0006) [2023-12-27 11:59:32,712][20381] Updated weights for policy 1, policy_version 915144 (0.0005) [2023-12-27 11:59:32,766][20381] Updated weights for policy 1, policy_version 915154 (0.0005) [2023-12-27 11:59:33,117][20347] Updated weights for policy 0, policy_version 915832 (0.0008) [2023-12-27 11:59:33,165][20347] Updated weights for policy 0, policy_version 915842 (0.0007) [2023-12-27 11:59:33,219][20347] Updated weights for policy 0, policy_version 915852 (0.0006) [2023-12-27 11:59:33,321][20381] Updated weights for policy 1, policy_version 915164 (0.0006) [2023-12-27 11:59:33,378][20381] Updated weights for policy 1, policy_version 915174 (0.0008) [2023-12-27 11:59:33,437][20381] Updated weights for policy 1, policy_version 915184 (0.0008) [2023-12-27 11:59:33,890][20347] Updated weights for policy 0, policy_version 915862 (0.0008) [2023-12-27 11:59:33,948][20347] Updated weights for policy 0, policy_version 915872 (0.0008) [2023-12-27 11:59:34,005][20347] Updated weights for policy 0, policy_version 915882 (0.0009) [2023-12-27 11:59:34,085][20381] Updated weights for policy 1, policy_version 915194 (0.0009) [2023-12-27 11:59:34,146][20381] Updated weights for policy 1, policy_version 915204 (0.0009) [2023-12-27 11:59:34,206][20381] Updated weights for policy 1, policy_version 915214 (0.0007) [2023-12-27 11:59:34,268][20381] Updated weights for policy 1, policy_version 915224 (0.0008) [2023-12-27 11:59:34,708][20347] Updated weights for policy 0, policy_version 915892 (0.0007) [2023-12-27 11:59:34,774][20347] Updated weights for policy 0, policy_version 915902 (0.0008) [2023-12-27 11:59:34,836][20347] Updated weights for policy 0, policy_version 915912 (0.0007) [2023-12-27 11:59:34,979][20381] Updated weights for policy 1, policy_version 915234 (0.0011) [2023-12-27 11:59:35,037][20381] Updated weights for policy 1, policy_version 915244 (0.0010) [2023-12-27 11:59:35,093][20381] Updated weights for policy 1, policy_version 915254 (0.0010) [2023-12-27 11:59:35,455][20347] Updated weights for policy 0, policy_version 915922 (0.0009) [2023-12-27 11:59:35,513][20347] Updated weights for policy 0, policy_version 915932 (0.0010) [2023-12-27 11:59:35,562][20347] Updated weights for policy 0, policy_version 915942 (0.0009) [2023-12-27 11:59:35,611][20347] Updated weights for policy 0, policy_version 915952 (0.0008) [2023-12-27 11:59:35,762][20381] Updated weights for policy 1, policy_version 915264 (0.0007) [2023-12-27 11:59:35,820][20381] Updated weights for policy 1, policy_version 915274 (0.0005) [2023-12-27 11:59:35,883][20381] Updated weights for policy 1, policy_version 915284 (0.0007) [2023-12-27 11:59:36,205][20347] Updated weights for policy 0, policy_version 915962 (0.0007) [2023-12-27 11:59:36,240][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000009 [2023-12-27 11:59:36,632][20381] Updated weights for policy 1, policy_version 915294 (0.0009) [2023-12-27 11:59:36,694][20381] Updated weights for policy 1, policy_version 915304 (0.0008) [2023-12-27 11:59:36,757][20381] Updated weights for policy 1, policy_version 915314 (0.0008) [2023-12-27 11:59:36,883][20347] Updated weights for policy 0, policy_version 915972 (0.0007) [2023-12-27 11:59:36,945][20347] Updated weights for policy 0, policy_version 915982 (0.0008) [2023-12-27 11:59:37,000][20347] Updated weights for policy 0, policy_version 915992 (0.0008) [2023-12-27 11:59:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20438.3). Total num frames: 468877312. Throughput: 0: 10408.2, 1: 10257.0. Samples: 468847764. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '335.060')] [2023-12-27 11:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000915320_234348544.pth... [2023-12-27 11:59:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000916000_234528768.pth... [2023-12-27 11:59:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000914136_234045440.pth [2023-12-27 11:59:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000914800_234217472.pth [2023-12-27 11:59:37,426][20381] Updated weights for policy 1, policy_version 915324 (0.0009) [2023-12-27 11:59:37,475][20381] Updated weights for policy 1, policy_version 915334 (0.0008) [2023-12-27 11:59:37,533][20381] Updated weights for policy 1, policy_version 915344 (0.0008) [2023-12-27 11:59:37,614][20347] Updated weights for policy 0, policy_version 916002 (0.0007) [2023-12-27 11:59:37,668][20347] Updated weights for policy 0, policy_version 916012 (0.0006) [2023-12-27 11:59:37,723][20347] Updated weights for policy 0, policy_version 916022 (0.0005) [2023-12-27 11:59:37,782][20347] Updated weights for policy 0, policy_version 916032 (0.0006) [2023-12-27 11:59:38,253][20381] Updated weights for policy 1, policy_version 915354 (0.0008) [2023-12-27 11:59:38,302][20381] Updated weights for policy 1, policy_version 915364 (0.0011) [2023-12-27 11:59:38,366][20381] Updated weights for policy 1, policy_version 915374 (0.0009) [2023-12-27 11:59:38,392][20347] Updated weights for policy 0, policy_version 916042 (0.0008) [2023-12-27 11:59:38,428][20381] Updated weights for policy 1, policy_version 915384 (0.0006) [2023-12-27 11:59:38,451][20347] Updated weights for policy 0, policy_version 916052 (0.0008) [2023-12-27 11:59:38,511][20347] Updated weights for policy 0, policy_version 916062 (0.0007) [2023-12-27 11:59:39,028][20381] Updated weights for policy 1, policy_version 915394 (0.0007) [2023-12-27 11:59:39,082][20381] Updated weights for policy 1, policy_version 915404 (0.0005) [2023-12-27 11:59:39,141][20381] Updated weights for policy 1, policy_version 915414 (0.0005) [2023-12-27 11:59:39,217][20347] Updated weights for policy 0, policy_version 916072 (0.0006) [2023-12-27 11:59:39,274][20347] Updated weights for policy 0, policy_version 916082 (0.0007) [2023-12-27 11:59:39,328][20347] Updated weights for policy 0, policy_version 916092 (0.0008) [2023-12-27 11:59:39,842][20381] Updated weights for policy 1, policy_version 915424 (0.0007) [2023-12-27 11:59:39,901][20381] Updated weights for policy 1, policy_version 915434 (0.0008) [2023-12-27 11:59:39,963][20381] Updated weights for policy 1, policy_version 915444 (0.0007) [2023-12-27 11:59:40,034][20347] Updated weights for policy 0, policy_version 916102 (0.0008) [2023-12-27 11:59:40,088][20347] Updated weights for policy 0, policy_version 916112 (0.0007) [2023-12-27 11:59:40,147][20347] Updated weights for policy 0, policy_version 916122 (0.0009) [2023-12-27 11:59:40,618][20381] Updated weights for policy 1, policy_version 915454 (0.0007) [2023-12-27 11:59:40,682][20381] Updated weights for policy 1, policy_version 915464 (0.0007) [2023-12-27 11:59:40,749][20381] Updated weights for policy 1, policy_version 915474 (0.0008) [2023-12-27 11:59:40,803][20347] Updated weights for policy 0, policy_version 916132 (0.0006) [2023-12-27 11:59:40,863][20347] Updated weights for policy 0, policy_version 916142 (0.0005) [2023-12-27 11:59:40,922][20347] Updated weights for policy 0, policy_version 916152 (0.0005) [2023-12-27 11:59:41,470][20381] Updated weights for policy 1, policy_version 915484 (0.0007) [2023-12-27 11:59:41,523][20381] Updated weights for policy 1, policy_version 915494 (0.0005) [2023-12-27 11:59:41,582][20381] Updated weights for policy 1, policy_version 915504 (0.0008) [2023-12-27 11:59:41,614][20347] Updated weights for policy 0, policy_version 916162 (0.0006) [2023-12-27 11:59:41,679][20347] Updated weights for policy 0, policy_version 916172 (0.0011) [2023-12-27 11:59:41,744][20347] Updated weights for policy 0, policy_version 916182 (0.0010) [2023-12-27 11:59:41,806][20347] Updated weights for policy 0, policy_version 916192 (0.0009) [2023-12-27 11:59:42,278][20381] Updated weights for policy 1, policy_version 915514 (0.0007) [2023-12-27 11:59:42,333][20381] Updated weights for policy 1, policy_version 915524 (0.0007) [2023-12-27 11:59:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 468975616. Throughput: 0: 10442.2, 1: 10230.5. Samples: 468972364. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '321.690')] [2023-12-27 11:59:42,401][20381] Updated weights for policy 1, policy_version 915534 (0.0008) [2023-12-27 11:59:42,464][20381] Updated weights for policy 1, policy_version 915544 (0.0007) [2023-12-27 11:59:42,541][20347] Updated weights for policy 0, policy_version 916202 (0.0005) [2023-12-27 11:59:42,599][20347] Updated weights for policy 0, policy_version 916212 (0.0009) [2023-12-27 11:59:42,656][20347] Updated weights for policy 0, policy_version 916222 (0.0008) [2023-12-27 11:59:43,133][20381] Updated weights for policy 1, policy_version 915554 (0.0007) [2023-12-27 11:59:43,192][20381] Updated weights for policy 1, policy_version 915564 (0.0006) [2023-12-27 11:59:43,249][20381] Updated weights for policy 1, policy_version 915574 (0.0008) [2023-12-27 11:59:43,305][20347] Updated weights for policy 0, policy_version 916232 (0.0010) [2023-12-27 11:59:43,362][20347] Updated weights for policy 0, policy_version 916242 (0.0009) [2023-12-27 11:59:43,414][20347] Updated weights for policy 0, policy_version 916252 (0.0008) [2023-12-27 11:59:43,943][20381] Updated weights for policy 1, policy_version 915584 (0.0006) [2023-12-27 11:59:43,995][20381] Updated weights for policy 1, policy_version 915594 (0.0006) [2023-12-27 11:59:44,036][20347] Updated weights for policy 0, policy_version 916262 (0.0008) [2023-12-27 11:59:44,044][20381] Updated weights for policy 1, policy_version 915604 (0.0006) [2023-12-27 11:59:44,100][20347] Updated weights for policy 0, policy_version 916272 (0.0010) [2023-12-27 11:59:44,171][20347] Updated weights for policy 0, policy_version 916282 (0.0010) [2023-12-27 11:59:44,714][20381] Updated weights for policy 1, policy_version 915614 (0.0008) [2023-12-27 11:59:44,774][20381] Updated weights for policy 1, policy_version 915624 (0.0008) [2023-12-27 11:59:44,810][20347] Updated weights for policy 0, policy_version 916292 (0.0008) [2023-12-27 11:59:44,839][20381] Updated weights for policy 1, policy_version 915634 (0.0008) [2023-12-27 11:59:44,873][20347] Updated weights for policy 0, policy_version 916302 (0.0007) [2023-12-27 11:59:44,938][20347] Updated weights for policy 0, policy_version 916312 (0.0006) [2023-12-27 11:59:45,501][20381] Updated weights for policy 1, policy_version 915644 (0.0008) [2023-12-27 11:59:45,560][20381] Updated weights for policy 1, policy_version 915654 (0.0008) [2023-12-27 11:59:45,609][20381] Updated weights for policy 1, policy_version 915664 (0.0007) [2023-12-27 11:59:45,626][20347] Updated weights for policy 0, policy_version 916322 (0.0006) [2023-12-27 11:59:45,687][20347] Updated weights for policy 0, policy_version 916332 (0.0007) [2023-12-27 11:59:45,738][20347] Updated weights for policy 0, policy_version 916342 (0.0005) [2023-12-27 11:59:45,787][20347] Updated weights for policy 0, policy_version 916352 (0.0005) [2023-12-27 11:59:46,327][20381] Updated weights for policy 1, policy_version 915674 (0.0008) [2023-12-27 11:59:46,349][20347] Updated weights for policy 0, policy_version 916362 (0.0006) [2023-12-27 11:59:46,379][20381] Updated weights for policy 1, policy_version 915684 (0.0010) [2023-12-27 11:59:46,396][20347] Updated weights for policy 0, policy_version 916372 (0.0010) [2023-12-27 11:59:46,430][20381] Updated weights for policy 1, policy_version 915694 (0.0010) [2023-12-27 11:59:46,437][20347] Updated weights for policy 0, policy_version 916382 (0.0010) [2023-12-27 11:59:46,484][20381] Updated weights for policy 1, policy_version 915704 (0.0010) [2023-12-27 11:59:47,072][20347] Updated weights for policy 0, policy_version 916392 (0.0010) [2023-12-27 11:59:47,104][20381] Updated weights for policy 1, policy_version 915714 (0.0009) [2023-12-27 11:59:47,127][20347] Updated weights for policy 0, policy_version 916402 (0.0009) [2023-12-27 11:59:47,162][20381] Updated weights for policy 1, policy_version 915724 (0.0010) [2023-12-27 11:59:47,182][20347] Updated weights for policy 0, policy_version 916412 (0.0010) [2023-12-27 11:59:47,214][20381] Updated weights for policy 1, policy_version 915734 (0.0010) [2023-12-27 11:59:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 469090304. Throughput: 0: 10470.2, 1: 10236.0. Samples: 469096284. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '399.490')] [2023-12-27 11:59:47,800][20381] Updated weights for policy 1, policy_version 915744 (0.0010) [2023-12-27 11:59:47,845][20347] Updated weights for policy 0, policy_version 916422 (0.0010) [2023-12-27 11:59:47,858][20381] Updated weights for policy 1, policy_version 915754 (0.0007) [2023-12-27 11:59:47,898][20347] Updated weights for policy 0, policy_version 916432 (0.0010) [2023-12-27 11:59:47,913][20381] Updated weights for policy 1, policy_version 915764 (0.0007) [2023-12-27 11:59:47,943][20347] Updated weights for policy 0, policy_version 916442 (0.0010) [2023-12-27 11:59:48,564][20381] Updated weights for policy 1, policy_version 915774 (0.0011) [2023-12-27 11:59:48,613][20381] Updated weights for policy 1, policy_version 915784 (0.0010) [2023-12-27 11:59:48,640][20347] Updated weights for policy 0, policy_version 916452 (0.0009) [2023-12-27 11:59:48,660][20381] Updated weights for policy 1, policy_version 915794 (0.0011) [2023-12-27 11:59:48,697][20347] Updated weights for policy 0, policy_version 916462 (0.0007) [2023-12-27 11:59:48,759][20347] Updated weights for policy 0, policy_version 916472 (0.0008) [2023-12-27 11:59:49,381][20381] Updated weights for policy 1, policy_version 915804 (0.0008) [2023-12-27 11:59:49,441][20381] Updated weights for policy 1, policy_version 915814 (0.0011) [2023-12-27 11:59:49,495][20347] Updated weights for policy 0, policy_version 916482 (0.0007) [2023-12-27 11:59:49,496][20381] Updated weights for policy 1, policy_version 915824 (0.0011) [2023-12-27 11:59:49,553][20347] Updated weights for policy 0, policy_version 916492 (0.0007) [2023-12-27 11:59:49,612][20347] Updated weights for policy 0, policy_version 916502 (0.0007) [2023-12-27 11:59:49,670][20347] Updated weights for policy 0, policy_version 916512 (0.0007) [2023-12-27 11:59:50,178][20381] Updated weights for policy 1, policy_version 915834 (0.0010) [2023-12-27 11:59:50,242][20381] Updated weights for policy 1, policy_version 915844 (0.0007) [2023-12-27 11:59:50,303][20381] Updated weights for policy 1, policy_version 915854 (0.0008) [2023-12-27 11:59:50,368][20381] Updated weights for policy 1, policy_version 915864 (0.0007) [2023-12-27 11:59:50,406][20347] Updated weights for policy 0, policy_version 916522 (0.0008) [2023-12-27 11:59:50,467][20347] Updated weights for policy 0, policy_version 916532 (0.0008) [2023-12-27 11:59:50,522][20347] Updated weights for policy 0, policy_version 916542 (0.0008) [2023-12-27 11:59:51,086][20381] Updated weights for policy 1, policy_version 915874 (0.0008) [2023-12-27 11:59:51,145][20381] Updated weights for policy 1, policy_version 915884 (0.0008) [2023-12-27 11:59:51,207][20381] Updated weights for policy 1, policy_version 915894 (0.0007) [2023-12-27 11:59:51,318][20347] Updated weights for policy 0, policy_version 916552 (0.0008) [2023-12-27 11:59:51,399][20347] Updated weights for policy 0, policy_version 916562 (0.0008) [2023-12-27 11:59:51,455][20347] Updated weights for policy 0, policy_version 916572 (0.0008) [2023-12-27 11:59:51,961][20381] Updated weights for policy 1, policy_version 915904 (0.0006) [2023-12-27 11:59:52,025][20381] Updated weights for policy 1, policy_version 915914 (0.0008) [2023-12-27 11:59:52,082][20381] Updated weights for policy 1, policy_version 915924 (0.0007) [2023-12-27 11:59:52,135][20347] Updated weights for policy 0, policy_version 916582 (0.0006) [2023-12-27 11:59:52,189][20347] Updated weights for policy 0, policy_version 916592 (0.0007) [2023-12-27 11:59:52,245][20347] Updated weights for policy 0, policy_version 916602 (0.0008) [2023-12-27 11:59:52,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20438.3). Total num frames: 469188608. Throughput: 0: 10423.5, 1: 10258.2. Samples: 469157480. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '403.480')] [2023-12-27 11:59:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000915928_234504192.pth... [2023-12-27 11:59:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000916608_234684416.pth... [2023-12-27 11:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000914712_234192896.pth [2023-12-27 11:59:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000915928_234504192.pth [2023-12-27 11:59:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000915408_234373120.pth [2023-12-27 11:59:52,377][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000916608_234684416.pth [2023-12-27 11:59:52,791][20381] Updated weights for policy 1, policy_version 915934 (0.0007) [2023-12-27 11:59:52,851][20381] Updated weights for policy 1, policy_version 915944 (0.0008) [2023-12-27 11:59:52,913][20347] Updated weights for policy 0, policy_version 916612 (0.0007) [2023-12-27 11:59:52,915][20381] Updated weights for policy 1, policy_version 915954 (0.0007) [2023-12-27 11:59:52,972][20347] Updated weights for policy 0, policy_version 916622 (0.0006) [2023-12-27 11:59:53,035][20347] Updated weights for policy 0, policy_version 916632 (0.0008) [2023-12-27 11:59:53,526][20381] Updated weights for policy 1, policy_version 915964 (0.0008) [2023-12-27 11:59:53,583][20381] Updated weights for policy 1, policy_version 915974 (0.0010) [2023-12-27 11:59:53,644][20381] Updated weights for policy 1, policy_version 915984 (0.0007) [2023-12-27 11:59:53,653][20347] Updated weights for policy 0, policy_version 916642 (0.0007) [2023-12-27 11:59:53,707][20347] Updated weights for policy 0, policy_version 916652 (0.0006) [2023-12-27 11:59:53,763][20347] Updated weights for policy 0, policy_version 916662 (0.0005) [2023-12-27 11:59:53,821][20347] Updated weights for policy 0, policy_version 916672 (0.0006) [2023-12-27 11:59:54,325][20381] Updated weights for policy 1, policy_version 915994 (0.0008) [2023-12-27 11:59:54,385][20381] Updated weights for policy 1, policy_version 916004 (0.0008) [2023-12-27 11:59:54,439][20347] Updated weights for policy 0, policy_version 916682 (0.0007) [2023-12-27 11:59:54,444][20381] Updated weights for policy 1, policy_version 916014 (0.0006) [2023-12-27 11:59:54,495][20347] Updated weights for policy 0, policy_version 916692 (0.0007) [2023-12-27 11:59:54,500][20381] Updated weights for policy 1, policy_version 916024 (0.0005) [2023-12-27 11:59:54,555][20347] Updated weights for policy 0, policy_version 916702 (0.0007) [2023-12-27 11:59:55,088][20381] Updated weights for policy 1, policy_version 916034 (0.0008) [2023-12-27 11:59:55,135][20381] Updated weights for policy 1, policy_version 916044 (0.0009) [2023-12-27 11:59:55,187][20381] Updated weights for policy 1, policy_version 916054 (0.0009) [2023-12-27 11:59:55,210][20347] Updated weights for policy 0, policy_version 916712 (0.0006) [2023-12-27 11:59:55,253][20347] Updated weights for policy 0, policy_version 916722 (0.0007) [2023-12-27 11:59:55,306][20347] Updated weights for policy 0, policy_version 916732 (0.0008) [2023-12-27 11:59:55,884][20381] Updated weights for policy 1, policy_version 916064 (0.0007) [2023-12-27 11:59:55,938][20381] Updated weights for policy 1, policy_version 916074 (0.0008) [2023-12-27 11:59:55,996][20381] Updated weights for policy 1, policy_version 916084 (0.0007) [2023-12-27 11:59:56,009][20347] Updated weights for policy 0, policy_version 916742 (0.0006) [2023-12-27 11:59:56,059][20347] Updated weights for policy 0, policy_version 916752 (0.0006) [2023-12-27 11:59:56,114][20347] Updated weights for policy 0, policy_version 916762 (0.0007) [2023-12-27 11:59:56,645][20381] Updated weights for policy 1, policy_version 916094 (0.0006) [2023-12-27 11:59:56,697][20381] Updated weights for policy 1, policy_version 916104 (0.0007) [2023-12-27 11:59:56,716][20347] Updated weights for policy 0, policy_version 916772 (0.0008) [2023-12-27 11:59:56,748][20381] Updated weights for policy 1, policy_version 916114 (0.0009) [2023-12-27 11:59:56,770][20347] Updated weights for policy 0, policy_version 916782 (0.0010) [2023-12-27 11:59:56,824][20347] Updated weights for policy 0, policy_version 916792 (0.0010) [2023-12-27 11:59:57,350][20381] Updated weights for policy 1, policy_version 916124 (0.0008) [2023-12-27 11:59:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 469286912. Throughput: 0: 10342.1, 1: 10287.5. Samples: 469279532. Policy #0 lag: (min: 1.0, avg: 22.3, max: 33.0) [2023-12-27 11:59:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '503.950')] [2023-12-27 11:59:57,403][20381] Updated weights for policy 1, policy_version 916134 (0.0007) [2023-12-27 11:59:57,454][20381] Updated weights for policy 1, policy_version 916144 (0.0008) [2023-12-27 11:59:57,498][20347] Updated weights for policy 0, policy_version 916802 (0.0010) [2023-12-27 11:59:57,543][20347] Updated weights for policy 0, policy_version 916812 (0.0010) [2023-12-27 11:59:57,595][20347] Updated weights for policy 0, policy_version 916822 (0.0010) [2023-12-27 11:59:57,639][20347] Updated weights for policy 0, policy_version 916832 (0.0010) [2023-12-27 11:59:58,184][20381] Updated weights for policy 1, policy_version 916154 (0.0007) [2023-12-27 11:59:58,248][20381] Updated weights for policy 1, policy_version 916164 (0.0007) [2023-12-27 11:59:58,268][20347] Updated weights for policy 0, policy_version 916842 (0.0008) [2023-12-27 11:59:58,308][20381] Updated weights for policy 1, policy_version 916174 (0.0006) [2023-12-27 11:59:58,326][20347] Updated weights for policy 0, policy_version 916853 (0.0008) [2023-12-27 11:59:58,378][20381] Updated weights for policy 1, policy_version 916184 (0.0008) [2023-12-27 11:59:58,394][20347] Updated weights for policy 0, policy_version 916863 (0.0009) [2023-12-27 11:59:59,087][20381] Updated weights for policy 1, policy_version 916194 (0.0005) [2023-12-27 11:59:59,142][20381] Updated weights for policy 1, policy_version 916204 (0.0005) [2023-12-27 11:59:59,158][20347] Updated weights for policy 0, policy_version 916873 (0.0007) [2023-12-27 11:59:59,199][20381] Updated weights for policy 1, policy_version 916214 (0.0006) [2023-12-27 11:59:59,211][20347] Updated weights for policy 0, policy_version 916883 (0.0009) [2023-12-27 11:59:59,273][20347] Updated weights for policy 0, policy_version 916893 (0.0007) [2023-12-27 11:59:59,865][20381] Updated weights for policy 1, policy_version 916224 (0.0007) [2023-12-27 11:59:59,926][20381] Updated weights for policy 1, policy_version 916234 (0.0008) [2023-12-27 11:59:59,931][20347] Updated weights for policy 0, policy_version 916903 (0.0009) [2023-12-27 11:59:59,986][20381] Updated weights for policy 1, policy_version 916244 (0.0007) [2023-12-27 11:59:59,988][20347] Updated weights for policy 0, policy_version 916913 (0.0011) [2023-12-27 12:00:00,039][20347] Updated weights for policy 0, policy_version 916923 (0.0010) [2023-12-27 12:00:00,634][20381] Updated weights for policy 1, policy_version 916254 (0.0006) [2023-12-27 12:00:00,692][20381] Updated weights for policy 1, policy_version 916264 (0.0007) [2023-12-27 12:00:00,757][20381] Updated weights for policy 1, policy_version 916274 (0.0010) [2023-12-27 12:00:00,763][20347] Updated weights for policy 0, policy_version 916933 (0.0007) [2023-12-27 12:00:00,814][20347] Updated weights for policy 0, policy_version 916943 (0.0007) [2023-12-27 12:00:00,867][20347] Updated weights for policy 0, policy_version 916953 (0.0007) [2023-12-27 12:00:01,472][20381] Updated weights for policy 1, policy_version 916284 (0.0011) [2023-12-27 12:00:01,529][20381] Updated weights for policy 1, policy_version 916294 (0.0009) [2023-12-27 12:00:01,572][20347] Updated weights for policy 0, policy_version 916963 (0.0008) [2023-12-27 12:00:01,584][20381] Updated weights for policy 1, policy_version 916304 (0.0008) [2023-12-27 12:00:01,627][20347] Updated weights for policy 0, policy_version 916973 (0.0010) [2023-12-27 12:00:01,680][20347] Updated weights for policy 0, policy_version 916983 (0.0009) [2023-12-27 12:00:02,302][20381] Updated weights for policy 1, policy_version 916314 (0.0006) [2023-12-27 12:00:02,361][20381] Updated weights for policy 1, policy_version 916324 (0.0006) [2023-12-27 12:00:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 469385216. Throughput: 0: 10323.6, 1: 10314.1. Samples: 469401908. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:02,363][20347] Updated weights for policy 0, policy_version 916993 (0.0011) [2023-12-27 12:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '399.450')] [2023-12-27 12:00:02,422][20381] Updated weights for policy 1, policy_version 916334 (0.0008) [2023-12-27 12:00:02,430][20347] Updated weights for policy 0, policy_version 917003 (0.0009) [2023-12-27 12:00:02,477][20381] Updated weights for policy 1, policy_version 916344 (0.0008) [2023-12-27 12:00:02,491][20347] Updated weights for policy 0, policy_version 917013 (0.0009) [2023-12-27 12:00:02,555][20347] Updated weights for policy 0, policy_version 917023 (0.0008) [2023-12-27 12:00:03,181][20381] Updated weights for policy 1, policy_version 916354 (0.0007) [2023-12-27 12:00:03,212][20347] Updated weights for policy 0, policy_version 917033 (0.0010) [2023-12-27 12:00:03,238][20381] Updated weights for policy 1, policy_version 916364 (0.0005) [2023-12-27 12:00:03,269][20347] Updated weights for policy 0, policy_version 917043 (0.0010) [2023-12-27 12:00:03,289][20381] Updated weights for policy 1, policy_version 916374 (0.0005) [2023-12-27 12:00:03,326][20347] Updated weights for policy 0, policy_version 917053 (0.0010) [2023-12-27 12:00:03,935][20381] Updated weights for policy 1, policy_version 916384 (0.0006) [2023-12-27 12:00:03,994][20381] Updated weights for policy 1, policy_version 916394 (0.0008) [2023-12-27 12:00:04,007][20347] Updated weights for policy 0, policy_version 917063 (0.0007) [2023-12-27 12:00:04,048][20381] Updated weights for policy 1, policy_version 916404 (0.0006) [2023-12-27 12:00:04,065][20347] Updated weights for policy 0, policy_version 917073 (0.0008) [2023-12-27 12:00:04,131][20347] Updated weights for policy 0, policy_version 917083 (0.0008) [2023-12-27 12:00:04,745][20381] Updated weights for policy 1, policy_version 916414 (0.0008) [2023-12-27 12:00:04,796][20347] Updated weights for policy 0, policy_version 917093 (0.0011) [2023-12-27 12:00:04,806][20381] Updated weights for policy 1, policy_version 916424 (0.0007) [2023-12-27 12:00:04,851][20347] Updated weights for policy 0, policy_version 917103 (0.0010) [2023-12-27 12:00:04,863][20381] Updated weights for policy 1, policy_version 916434 (0.0006) [2023-12-27 12:00:04,899][20347] Updated weights for policy 0, policy_version 917113 (0.0010) [2023-12-27 12:00:05,519][20381] Updated weights for policy 1, policy_version 916444 (0.0008) [2023-12-27 12:00:05,530][20347] Updated weights for policy 0, policy_version 917123 (0.0009) [2023-12-27 12:00:05,573][20291] KL-divergence is very high: 316.8469 [2023-12-27 12:00:05,575][20381] Updated weights for policy 1, policy_version 916454 (0.0011) [2023-12-27 12:00:05,592][20347] Updated weights for policy 0, policy_version 917133 (0.0008) [2023-12-27 12:00:05,630][20381] Updated weights for policy 1, policy_version 916464 (0.0010) [2023-12-27 12:00:05,642][20347] Updated weights for policy 0, policy_version 917143 (0.0010) [2023-12-27 12:00:06,277][20381] Updated weights for policy 1, policy_version 916474 (0.0006) [2023-12-27 12:00:06,326][20381] Updated weights for policy 1, policy_version 916484 (0.0008) [2023-12-27 12:00:06,353][20347] Updated weights for policy 0, policy_version 917153 (0.0010) [2023-12-27 12:00:06,385][20381] Updated weights for policy 1, policy_version 916494 (0.0007) [2023-12-27 12:00:06,416][20347] Updated weights for policy 0, policy_version 917163 (0.0010) [2023-12-27 12:00:06,449][20381] Updated weights for policy 1, policy_version 916504 (0.0006) [2023-12-27 12:00:06,480][20347] Updated weights for policy 0, policy_version 917173 (0.0011) [2023-12-27 12:00:06,544][20347] Updated weights for policy 0, policy_version 917183 (0.0009) [2023-12-27 12:00:07,103][20381] Updated weights for policy 1, policy_version 916514 (0.0005) [2023-12-27 12:00:07,153][20381] Updated weights for policy 1, policy_version 916524 (0.0005) [2023-12-27 12:00:07,203][20381] Updated weights for policy 1, policy_version 916534 (0.0005) [2023-12-27 12:00:07,207][20347] Updated weights for policy 0, policy_version 917193 (0.0009) [2023-12-27 12:00:07,281][20347] Updated weights for policy 0, policy_version 917203 (0.0010) [2023-12-27 12:00:07,342][20347] Updated weights for policy 0, policy_version 917213 (0.0010) [2023-12-27 12:00:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 469499904. Throughput: 0: 10334.4, 1: 10321.8. Samples: 469462912. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:07,363][19170] Avg episode reward: [(0, '991.460'), (1, '260.060')] [2023-12-27 12:00:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000916536_234659840.pth... [2023-12-27 12:00:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000917216_234840064.pth... [2023-12-27 12:00:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000915320_234348544.pth [2023-12-27 12:00:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000916000_234528768.pth [2023-12-27 12:00:07,799][20381] Updated weights for policy 1, policy_version 916544 (0.0007) [2023-12-27 12:00:07,853][20381] Updated weights for policy 1, policy_version 916554 (0.0008) [2023-12-27 12:00:07,901][20381] Updated weights for policy 1, policy_version 916564 (0.0007) [2023-12-27 12:00:07,996][20347] Updated weights for policy 0, policy_version 917223 (0.0010) [2023-12-27 12:00:08,052][20347] Updated weights for policy 0, policy_version 917233 (0.0010) [2023-12-27 12:00:08,107][20347] Updated weights for policy 0, policy_version 917243 (0.0010) [2023-12-27 12:00:08,507][20381] Updated weights for policy 1, policy_version 916574 (0.0006) [2023-12-27 12:00:08,563][20381] Updated weights for policy 1, policy_version 916584 (0.0007) [2023-12-27 12:00:08,614][20381] Updated weights for policy 1, policy_version 916594 (0.0008) [2023-12-27 12:00:08,720][20347] Updated weights for policy 0, policy_version 917253 (0.0010) [2023-12-27 12:00:08,775][20347] Updated weights for policy 0, policy_version 917263 (0.0010) [2023-12-27 12:00:08,831][20347] Updated weights for policy 0, policy_version 917273 (0.0010) [2023-12-27 12:00:09,304][20381] Updated weights for policy 1, policy_version 916604 (0.0007) [2023-12-27 12:00:09,364][20381] Updated weights for policy 1, policy_version 916614 (0.0008) [2023-12-27 12:00:09,427][20381] Updated weights for policy 1, policy_version 916624 (0.0009) [2023-12-27 12:00:09,550][20347] Updated weights for policy 0, policy_version 917283 (0.0009) [2023-12-27 12:00:09,609][20347] Updated weights for policy 0, policy_version 917293 (0.0008) [2023-12-27 12:00:09,661][20347] Updated weights for policy 0, policy_version 917303 (0.0007) [2023-12-27 12:00:10,155][20381] Updated weights for policy 1, policy_version 916634 (0.0009) [2023-12-27 12:00:10,220][20381] Updated weights for policy 1, policy_version 916644 (0.0008) [2023-12-27 12:00:10,286][20381] Updated weights for policy 1, policy_version 916654 (0.0008) [2023-12-27 12:00:10,346][20347] Updated weights for policy 0, policy_version 917313 (0.0007) [2023-12-27 12:00:10,352][20381] Updated weights for policy 1, policy_version 916664 (0.0008) [2023-12-27 12:00:10,405][20347] Updated weights for policy 0, policy_version 917323 (0.0006) [2023-12-27 12:00:10,469][20347] Updated weights for policy 0, policy_version 917333 (0.0006) [2023-12-27 12:00:10,527][20347] Updated weights for policy 0, policy_version 917343 (0.0005) [2023-12-27 12:00:10,995][20381] Updated weights for policy 1, policy_version 916674 (0.0005) [2023-12-27 12:00:11,060][20381] Updated weights for policy 1, policy_version 916685 (0.0009) [2023-12-27 12:00:11,121][20381] Updated weights for policy 1, policy_version 916695 (0.0010) [2023-12-27 12:00:11,169][20347] Updated weights for policy 0, policy_version 917353 (0.0008) [2023-12-27 12:00:11,231][20347] Updated weights for policy 0, policy_version 917363 (0.0008) [2023-12-27 12:00:11,296][20347] Updated weights for policy 0, policy_version 917373 (0.0007) [2023-12-27 12:00:11,922][20381] Updated weights for policy 1, policy_version 916705 (0.0008) [2023-12-27 12:00:11,983][20381] Updated weights for policy 1, policy_version 916715 (0.0008) [2023-12-27 12:00:12,006][20347] Updated weights for policy 0, policy_version 917383 (0.0008) [2023-12-27 12:00:12,046][20381] Updated weights for policy 1, policy_version 916725 (0.0006) [2023-12-27 12:00:12,070][20347] Updated weights for policy 0, policy_version 917393 (0.0007) [2023-12-27 12:00:12,126][20347] Updated weights for policy 0, policy_version 917403 (0.0008) [2023-12-27 12:00:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 469598208. Throughput: 0: 10293.6, 1: 10324.3. Samples: 469587628. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:12,363][19170] Avg episode reward: [(0, '991.460'), (1, '277.320')] [2023-12-27 12:00:12,720][20381] Updated weights for policy 1, policy_version 916735 (0.0008) [2023-12-27 12:00:12,776][20381] Updated weights for policy 1, policy_version 916745 (0.0008) [2023-12-27 12:00:12,816][20347] Updated weights for policy 0, policy_version 917413 (0.0007) [2023-12-27 12:00:12,834][20381] Updated weights for policy 1, policy_version 916755 (0.0007) [2023-12-27 12:00:12,873][20347] Updated weights for policy 0, policy_version 917423 (0.0007) [2023-12-27 12:00:12,925][20347] Updated weights for policy 0, policy_version 917433 (0.0008) [2023-12-27 12:00:13,508][20381] Updated weights for policy 1, policy_version 916765 (0.0006) [2023-12-27 12:00:13,558][20381] Updated weights for policy 1, policy_version 916775 (0.0008) [2023-12-27 12:00:13,614][20347] Updated weights for policy 0, policy_version 917443 (0.0008) [2023-12-27 12:00:13,617][20381] Updated weights for policy 1, policy_version 916785 (0.0007) [2023-12-27 12:00:13,668][20347] Updated weights for policy 0, policy_version 917453 (0.0007) [2023-12-27 12:00:13,724][20347] Updated weights for policy 0, policy_version 917463 (0.0007) [2023-12-27 12:00:14,236][20381] Updated weights for policy 1, policy_version 916795 (0.0006) [2023-12-27 12:00:14,286][20381] Updated weights for policy 1, policy_version 916805 (0.0006) [2023-12-27 12:00:14,332][20381] Updated weights for policy 1, policy_version 916815 (0.0007) [2023-12-27 12:00:14,436][20347] Updated weights for policy 0, policy_version 917473 (0.0008) [2023-12-27 12:00:14,490][20347] Updated weights for policy 0, policy_version 917483 (0.0007) [2023-12-27 12:00:14,551][20347] Updated weights for policy 0, policy_version 917493 (0.0008) [2023-12-27 12:00:14,602][20347] Updated weights for policy 0, policy_version 917503 (0.0006) [2023-12-27 12:00:14,931][20381] Updated weights for policy 1, policy_version 916825 (0.0007) [2023-12-27 12:00:14,993][20381] Updated weights for policy 1, policy_version 916835 (0.0008) [2023-12-27 12:00:15,054][20381] Updated weights for policy 1, policy_version 916845 (0.0009) [2023-12-27 12:00:15,119][20381] Updated weights for policy 1, policy_version 916855 (0.0010) [2023-12-27 12:00:15,245][20347] Updated weights for policy 0, policy_version 917513 (0.0005) [2023-12-27 12:00:15,298][20347] Updated weights for policy 0, policy_version 917523 (0.0005) [2023-12-27 12:00:15,346][20347] Updated weights for policy 0, policy_version 917533 (0.0005) [2023-12-27 12:00:15,758][20381] Updated weights for policy 1, policy_version 916865 (0.0008) [2023-12-27 12:00:15,806][20381] Updated weights for policy 1, policy_version 916875 (0.0008) [2023-12-27 12:00:15,863][20381] Updated weights for policy 1, policy_version 916885 (0.0008) [2023-12-27 12:00:15,976][20347] Updated weights for policy 0, policy_version 917543 (0.0009) [2023-12-27 12:00:16,023][20347] Updated weights for policy 0, policy_version 917553 (0.0006) [2023-12-27 12:00:16,081][20347] Updated weights for policy 0, policy_version 917563 (0.0007) [2023-12-27 12:00:16,507][20381] Updated weights for policy 1, policy_version 916895 (0.0010) [2023-12-27 12:00:16,571][20381] Updated weights for policy 1, policy_version 916905 (0.0010) [2023-12-27 12:00:16,626][20381] Updated weights for policy 1, policy_version 916915 (0.0010) [2023-12-27 12:00:16,789][20347] Updated weights for policy 0, policy_version 917573 (0.0008) [2023-12-27 12:00:16,846][20347] Updated weights for policy 0, policy_version 917583 (0.0008) [2023-12-27 12:00:16,906][20347] Updated weights for policy 0, policy_version 917593 (0.0008) [2023-12-27 12:00:17,251][20381] Updated weights for policy 1, policy_version 916925 (0.0008) [2023-12-27 12:00:17,304][20381] Updated weights for policy 1, policy_version 916935 (0.0007) [2023-12-27 12:00:17,360][20381] Updated weights for policy 1, policy_version 916945 (0.0005) [2023-12-27 12:00:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 469696512. Throughput: 0: 10268.2, 1: 10291.6. Samples: 469710576. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '337.820')] [2023-12-27 12:00:17,672][20347] Updated weights for policy 0, policy_version 917603 (0.0008) [2023-12-27 12:00:17,729][20347] Updated weights for policy 0, policy_version 917613 (0.0007) [2023-12-27 12:00:17,794][20347] Updated weights for policy 0, policy_version 917623 (0.0007) [2023-12-27 12:00:18,005][20381] Updated weights for policy 1, policy_version 916955 (0.0005) [2023-12-27 12:00:18,069][20381] Updated weights for policy 1, policy_version 916965 (0.0005) [2023-12-27 12:00:18,127][20381] Updated weights for policy 1, policy_version 916975 (0.0005) [2023-12-27 12:00:18,492][20347] Updated weights for policy 0, policy_version 917633 (0.0008) [2023-12-27 12:00:18,552][20347] Updated weights for policy 0, policy_version 917643 (0.0007) [2023-12-27 12:00:18,611][20347] Updated weights for policy 0, policy_version 917653 (0.0008) [2023-12-27 12:00:18,675][20347] Updated weights for policy 0, policy_version 917663 (0.0008) [2023-12-27 12:00:18,750][20381] Updated weights for policy 1, policy_version 916985 (0.0007) [2023-12-27 12:00:18,803][20381] Updated weights for policy 1, policy_version 916995 (0.0007) [2023-12-27 12:00:18,859][20381] Updated weights for policy 1, policy_version 917005 (0.0008) [2023-12-27 12:00:18,908][20381] Updated weights for policy 1, policy_version 917015 (0.0008) [2023-12-27 12:00:19,383][20347] Updated weights for policy 0, policy_version 917673 (0.0008) [2023-12-27 12:00:19,445][20347] Updated weights for policy 0, policy_version 917683 (0.0008) [2023-12-27 12:00:19,501][20347] Updated weights for policy 0, policy_version 917693 (0.0008) [2023-12-27 12:00:19,607][20381] Updated weights for policy 1, policy_version 917025 (0.0008) [2023-12-27 12:00:19,669][20381] Updated weights for policy 1, policy_version 917035 (0.0007) [2023-12-27 12:00:19,730][20381] Updated weights for policy 1, policy_version 917045 (0.0009) [2023-12-27 12:00:20,290][20347] Updated weights for policy 0, policy_version 917703 (0.0008) [2023-12-27 12:00:20,358][20347] Updated weights for policy 0, policy_version 917713 (0.0008) [2023-12-27 12:00:20,420][20381] Updated weights for policy 1, policy_version 917055 (0.0008) [2023-12-27 12:00:20,421][20347] Updated weights for policy 0, policy_version 917723 (0.0006) [2023-12-27 12:00:20,477][20381] Updated weights for policy 1, policy_version 917065 (0.0008) [2023-12-27 12:00:20,533][20381] Updated weights for policy 1, policy_version 917075 (0.0007) [2023-12-27 12:00:21,123][20347] Updated weights for policy 0, policy_version 917733 (0.0006) [2023-12-27 12:00:21,187][20347] Updated weights for policy 0, policy_version 917743 (0.0006) [2023-12-27 12:00:21,246][20347] Updated weights for policy 0, policy_version 917753 (0.0008) [2023-12-27 12:00:21,259][20381] Updated weights for policy 1, policy_version 917085 (0.0007) [2023-12-27 12:00:21,323][20381] Updated weights for policy 1, policy_version 917095 (0.0008) [2023-12-27 12:00:21,381][20381] Updated weights for policy 1, policy_version 917105 (0.0009) [2023-12-27 12:00:21,975][20347] Updated weights for policy 0, policy_version 917763 (0.0008) [2023-12-27 12:00:22,040][20347] Updated weights for policy 0, policy_version 917773 (0.0008) [2023-12-27 12:00:22,080][20381] Updated weights for policy 1, policy_version 917115 (0.0007) [2023-12-27 12:00:22,103][20347] Updated weights for policy 0, policy_version 917783 (0.0008) [2023-12-27 12:00:22,140][20381] Updated weights for policy 1, policy_version 917125 (0.0006) [2023-12-27 12:00:22,203][20381] Updated weights for policy 1, policy_version 917135 (0.0008) [2023-12-27 12:00:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 469803008. Throughput: 0: 10237.5, 1: 10253.7. Samples: 469769868. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '420.980')] [2023-12-27 12:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000917144_234815488.pth... [2023-12-27 12:00:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000917792_234987520.pth... [2023-12-27 12:00:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000916608_234684416.pth [2023-12-27 12:00:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000915928_234504192.pth [2023-12-27 12:00:22,895][20347] Updated weights for policy 0, policy_version 917793 (0.0007) [2023-12-27 12:00:22,945][20381] Updated weights for policy 1, policy_version 917145 (0.0007) [2023-12-27 12:00:22,958][20347] Updated weights for policy 0, policy_version 917803 (0.0007) [2023-12-27 12:00:23,005][20381] Updated weights for policy 1, policy_version 917155 (0.0006) [2023-12-27 12:00:23,022][20347] Updated weights for policy 0, policy_version 917813 (0.0010) [2023-12-27 12:00:23,066][20381] Updated weights for policy 1, policy_version 917165 (0.0006) [2023-12-27 12:00:23,086][20347] Updated weights for policy 0, policy_version 917823 (0.0010) [2023-12-27 12:00:23,126][20381] Updated weights for policy 1, policy_version 917175 (0.0006) [2023-12-27 12:00:23,691][20381] Updated weights for policy 1, policy_version 917185 (0.0005) [2023-12-27 12:00:23,693][20347] Updated weights for policy 0, policy_version 917833 (0.0007) [2023-12-27 12:00:23,741][20381] Updated weights for policy 1, policy_version 917195 (0.0007) [2023-12-27 12:00:23,744][20347] Updated weights for policy 0, policy_version 917843 (0.0005) [2023-12-27 12:00:23,795][20347] Updated weights for policy 0, policy_version 917853 (0.0005) [2023-12-27 12:00:23,795][20381] Updated weights for policy 1, policy_version 917205 (0.0007) [2023-12-27 12:00:24,401][20347] Updated weights for policy 0, policy_version 917863 (0.0007) [2023-12-27 12:00:24,463][20347] Updated weights for policy 0, policy_version 917873 (0.0008) [2023-12-27 12:00:24,499][20381] Updated weights for policy 1, policy_version 917215 (0.0007) [2023-12-27 12:00:24,520][20347] Updated weights for policy 0, policy_version 917883 (0.0008) [2023-12-27 12:00:24,556][20381] Updated weights for policy 1, policy_version 917225 (0.0006) [2023-12-27 12:00:24,611][20381] Updated weights for policy 1, policy_version 917235 (0.0007) [2023-12-27 12:00:25,179][20347] Updated weights for policy 0, policy_version 917893 (0.0007) [2023-12-27 12:00:25,203][20381] Updated weights for policy 1, policy_version 917245 (0.0007) [2023-12-27 12:00:25,227][20347] Updated weights for policy 0, policy_version 917903 (0.0005) [2023-12-27 12:00:25,258][20381] Updated weights for policy 1, policy_version 917255 (0.0005) [2023-12-27 12:00:25,278][20347] Updated weights for policy 0, policy_version 917913 (0.0006) [2023-12-27 12:00:25,322][20381] Updated weights for policy 1, policy_version 917265 (0.0007) [2023-12-27 12:00:25,951][20347] Updated weights for policy 0, policy_version 917923 (0.0009) [2023-12-27 12:00:25,996][20381] Updated weights for policy 1, policy_version 917275 (0.0008) [2023-12-27 12:00:26,006][20347] Updated weights for policy 0, policy_version 917933 (0.0009) [2023-12-27 12:00:26,051][20381] Updated weights for policy 1, policy_version 917285 (0.0007) [2023-12-27 12:00:26,068][20347] Updated weights for policy 0, policy_version 917943 (0.0008) [2023-12-27 12:00:26,118][20381] Updated weights for policy 1, policy_version 917295 (0.0007) [2023-12-27 12:00:26,763][20347] Updated weights for policy 0, policy_version 917953 (0.0008) [2023-12-27 12:00:26,799][20381] Updated weights for policy 1, policy_version 917305 (0.0008) [2023-12-27 12:00:26,815][20347] Updated weights for policy 0, policy_version 917963 (0.0005) [2023-12-27 12:00:26,858][20381] Updated weights for policy 1, policy_version 917315 (0.0009) [2023-12-27 12:00:26,869][20347] Updated weights for policy 0, policy_version 917973 (0.0005) [2023-12-27 12:00:26,911][20381] Updated weights for policy 1, policy_version 917325 (0.0010) [2023-12-27 12:00:26,922][20347] Updated weights for policy 0, policy_version 917983 (0.0005) [2023-12-27 12:00:26,957][20381] Updated weights for policy 1, policy_version 917335 (0.0007) [2023-12-27 12:00:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 469901312. Throughput: 0: 10147.0, 1: 10279.3. Samples: 469891544. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:27,363][19170] Avg episode reward: [(0, '990.370'), (1, '331.130')] [2023-12-27 12:00:27,530][20347] Updated weights for policy 0, policy_version 917993 (0.0007) [2023-12-27 12:00:27,545][20381] Updated weights for policy 1, policy_version 917345 (0.0005) [2023-12-27 12:00:27,573][20347] Updated weights for policy 0, policy_version 918003 (0.0006) [2023-12-27 12:00:27,590][20381] Updated weights for policy 1, policy_version 917355 (0.0010) [2023-12-27 12:00:27,622][20347] Updated weights for policy 0, policy_version 918013 (0.0007) [2023-12-27 12:00:27,645][20381] Updated weights for policy 1, policy_version 917365 (0.0011) [2023-12-27 12:00:28,270][20347] Updated weights for policy 0, policy_version 918023 (0.0009) [2023-12-27 12:00:28,314][20381] Updated weights for policy 1, policy_version 917375 (0.0011) [2023-12-27 12:00:28,332][20347] Updated weights for policy 0, policy_version 918033 (0.0010) [2023-12-27 12:00:28,378][20381] Updated weights for policy 1, policy_version 917385 (0.0012) [2023-12-27 12:00:28,392][20347] Updated weights for policy 0, policy_version 918043 (0.0008) [2023-12-27 12:00:28,438][20381] Updated weights for policy 1, policy_version 917395 (0.0006) [2023-12-27 12:00:29,034][20347] Updated weights for policy 0, policy_version 918053 (0.0008) [2023-12-27 12:00:29,038][20381] Updated weights for policy 1, policy_version 917405 (0.0008) [2023-12-27 12:00:29,088][20291] KL-divergence is very high: 170.9500 [2023-12-27 12:00:29,089][20347] Updated weights for policy 0, policy_version 918063 (0.0008) [2023-12-27 12:00:29,102][20381] Updated weights for policy 1, policy_version 917415 (0.0010) [2023-12-27 12:00:29,149][20347] Updated weights for policy 0, policy_version 918073 (0.0007) [2023-12-27 12:00:29,166][20381] Updated weights for policy 1, policy_version 917425 (0.0011) [2023-12-27 12:00:29,806][20347] Updated weights for policy 0, policy_version 918083 (0.0009) [2023-12-27 12:00:29,864][20347] Updated weights for policy 0, policy_version 918093 (0.0008) [2023-12-27 12:00:29,880][20381] Updated weights for policy 1, policy_version 917435 (0.0011) [2023-12-27 12:00:29,916][20347] Updated weights for policy 0, policy_version 918103 (0.0006) [2023-12-27 12:00:29,940][20381] Updated weights for policy 1, policy_version 917445 (0.0011) [2023-12-27 12:00:29,999][20381] Updated weights for policy 1, policy_version 917455 (0.0008) [2023-12-27 12:00:30,555][20347] Updated weights for policy 0, policy_version 918113 (0.0008) [2023-12-27 12:00:30,605][20347] Updated weights for policy 0, policy_version 918123 (0.0007) [2023-12-27 12:00:30,658][20347] Updated weights for policy 0, policy_version 918133 (0.0008) [2023-12-27 12:00:30,674][20381] Updated weights for policy 1, policy_version 917465 (0.0008) [2023-12-27 12:00:30,711][20347] Updated weights for policy 0, policy_version 918143 (0.0007) [2023-12-27 12:00:30,727][20381] Updated weights for policy 1, policy_version 917475 (0.0005) [2023-12-27 12:00:30,788][20381] Updated weights for policy 1, policy_version 917485 (0.0005) [2023-12-27 12:00:30,845][20381] Updated weights for policy 1, policy_version 917495 (0.0007) [2023-12-27 12:00:31,534][20347] Updated weights for policy 0, policy_version 918153 (0.0007) [2023-12-27 12:00:31,559][20381] Updated weights for policy 1, policy_version 917505 (0.0007) [2023-12-27 12:00:31,589][20347] Updated weights for policy 0, policy_version 918163 (0.0005) [2023-12-27 12:00:31,620][20381] Updated weights for policy 1, policy_version 917515 (0.0007) [2023-12-27 12:00:31,658][20347] Updated weights for policy 0, policy_version 918173 (0.0008) [2023-12-27 12:00:31,689][20381] Updated weights for policy 1, policy_version 917525 (0.0006) [2023-12-27 12:00:32,292][20347] Updated weights for policy 0, policy_version 918183 (0.0009) [2023-12-27 12:00:32,351][20347] Updated weights for policy 0, policy_version 918193 (0.0010) [2023-12-27 12:00:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 469999616. Throughput: 0: 10131.1, 1: 10302.5. Samples: 470015800. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:32,363][19170] Avg episode reward: [(0, '980.910'), (1, '419.630')] [2023-12-27 12:00:32,386][20381] Updated weights for policy 1, policy_version 917535 (0.0009) [2023-12-27 12:00:32,417][20347] Updated weights for policy 0, policy_version 918203 (0.0008) [2023-12-27 12:00:32,443][20381] Updated weights for policy 1, policy_version 917545 (0.0008) [2023-12-27 12:00:32,502][20381] Updated weights for policy 1, policy_version 917555 (0.0008) [2023-12-27 12:00:33,093][20347] Updated weights for policy 0, policy_version 918213 (0.0007) [2023-12-27 12:00:33,153][20347] Updated weights for policy 0, policy_version 918223 (0.0007) [2023-12-27 12:00:33,208][20381] Updated weights for policy 1, policy_version 917565 (0.0009) [2023-12-27 12:00:33,210][20347] Updated weights for policy 0, policy_version 918233 (0.0005) [2023-12-27 12:00:33,258][20381] Updated weights for policy 1, policy_version 917575 (0.0010) [2023-12-27 12:00:33,312][20381] Updated weights for policy 1, policy_version 917585 (0.0006) [2023-12-27 12:00:33,905][20347] Updated weights for policy 0, policy_version 918243 (0.0007) [2023-12-27 12:00:33,966][20347] Updated weights for policy 0, policy_version 918253 (0.0006) [2023-12-27 12:00:33,975][20381] Updated weights for policy 1, policy_version 917595 (0.0007) [2023-12-27 12:00:34,025][20347] Updated weights for policy 0, policy_version 918263 (0.0005) [2023-12-27 12:00:34,028][20381] Updated weights for policy 1, policy_version 917605 (0.0008) [2023-12-27 12:00:34,076][20381] Updated weights for policy 1, policy_version 917615 (0.0008) [2023-12-27 12:00:34,704][20347] Updated weights for policy 0, policy_version 918273 (0.0005) [2023-12-27 12:00:34,760][20347] Updated weights for policy 0, policy_version 918283 (0.0007) [2023-12-27 12:00:34,797][20381] Updated weights for policy 1, policy_version 917625 (0.0008) [2023-12-27 12:00:34,825][20347] Updated weights for policy 0, policy_version 918293 (0.0008) [2023-12-27 12:00:34,856][20381] Updated weights for policy 1, policy_version 917635 (0.0005) [2023-12-27 12:00:34,861][20291] KL-divergence is very high: 278.5672 [2023-12-27 12:00:34,877][20347] Updated weights for policy 0, policy_version 918303 (0.0008) [2023-12-27 12:00:34,913][20381] Updated weights for policy 1, policy_version 917645 (0.0007) [2023-12-27 12:00:34,960][20381] Updated weights for policy 1, policy_version 917655 (0.0007) [2023-12-27 12:00:35,559][20347] Updated weights for policy 0, policy_version 918313 (0.0008) [2023-12-27 12:00:35,619][20347] Updated weights for policy 0, policy_version 918323 (0.0007) [2023-12-27 12:00:35,656][20381] Updated weights for policy 1, policy_version 917665 (0.0007) [2023-12-27 12:00:35,677][20347] Updated weights for policy 0, policy_version 918333 (0.0008) [2023-12-27 12:00:35,716][20381] Updated weights for policy 1, policy_version 917675 (0.0007) [2023-12-27 12:00:35,778][20381] Updated weights for policy 1, policy_version 917685 (0.0007) [2023-12-27 12:00:36,428][20347] Updated weights for policy 0, policy_version 918343 (0.0006) [2023-12-27 12:00:36,485][20381] Updated weights for policy 1, policy_version 917695 (0.0010) [2023-12-27 12:00:36,489][20347] Updated weights for policy 0, policy_version 918353 (0.0007) [2023-12-27 12:00:36,545][20381] Updated weights for policy 1, policy_version 917705 (0.0011) [2023-12-27 12:00:36,550][20347] Updated weights for policy 0, policy_version 918363 (0.0008) [2023-12-27 12:00:36,598][20381] Updated weights for policy 1, policy_version 917715 (0.0010) [2023-12-27 12:00:37,249][20347] Updated weights for policy 0, policy_version 918373 (0.0007) [2023-12-27 12:00:37,269][20381] Updated weights for policy 1, policy_version 917725 (0.0008) [2023-12-27 12:00:37,311][20347] Updated weights for policy 0, policy_version 918383 (0.0007) [2023-12-27 12:00:37,314][20381] Updated weights for policy 1, policy_version 917735 (0.0006) [2023-12-27 12:00:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 470097920. Throughput: 0: 10158.9, 1: 10261.4. Samples: 470076388. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:37,363][19170] Avg episode reward: [(0, '971.160'), (1, '436.820')] [2023-12-27 12:00:37,364][20381] Updated weights for policy 1, policy_version 917745 (0.0008) [2023-12-27 12:00:37,366][20347] Updated weights for policy 0, policy_version 918393 (0.0005) [2023-12-27 12:00:37,404][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000917752_234971136.pth... [2023-12-27 12:00:37,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000918400_235143168.pth... [2023-12-27 12:00:37,407][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000916536_234659840.pth [2023-12-27 12:00:37,410][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000917216_234840064.pth [2023-12-27 12:00:38,006][20347] Updated weights for policy 0, policy_version 918403 (0.0006) [2023-12-27 12:00:38,074][20347] Updated weights for policy 0, policy_version 918413 (0.0007) [2023-12-27 12:00:38,094][20381] Updated weights for policy 1, policy_version 917755 (0.0009) [2023-12-27 12:00:38,137][20347] Updated weights for policy 0, policy_version 918423 (0.0005) [2023-12-27 12:00:38,151][20381] Updated weights for policy 1, policy_version 917765 (0.0008) [2023-12-27 12:00:38,202][20381] Updated weights for policy 1, policy_version 917775 (0.0008) [2023-12-27 12:00:38,809][20347] Updated weights for policy 0, policy_version 918433 (0.0007) [2023-12-27 12:00:38,817][20381] Updated weights for policy 1, policy_version 917785 (0.0005) [2023-12-27 12:00:38,865][20347] Updated weights for policy 0, policy_version 918443 (0.0007) [2023-12-27 12:00:38,878][20381] Updated weights for policy 1, policy_version 917795 (0.0006) [2023-12-27 12:00:38,927][20347] Updated weights for policy 0, policy_version 918453 (0.0009) [2023-12-27 12:00:38,934][20381] Updated weights for policy 1, policy_version 917805 (0.0007) [2023-12-27 12:00:38,981][20347] Updated weights for policy 0, policy_version 918463 (0.0006) [2023-12-27 12:00:38,986][20381] Updated weights for policy 1, policy_version 917815 (0.0006) [2023-12-27 12:00:39,696][20381] Updated weights for policy 1, policy_version 917825 (0.0008) [2023-12-27 12:00:39,754][20347] Updated weights for policy 0, policy_version 918473 (0.0007) [2023-12-27 12:00:39,757][20381] Updated weights for policy 1, policy_version 917835 (0.0008) [2023-12-27 12:00:39,815][20381] Updated weights for policy 1, policy_version 917845 (0.0006) [2023-12-27 12:00:39,817][20347] Updated weights for policy 0, policy_version 918483 (0.0007) [2023-12-27 12:00:39,871][20347] Updated weights for policy 0, policy_version 918493 (0.0006) [2023-12-27 12:00:40,464][20381] Updated weights for policy 1, policy_version 917855 (0.0006) [2023-12-27 12:00:40,521][20381] Updated weights for policy 1, policy_version 917865 (0.0006) [2023-12-27 12:00:40,572][20347] Updated weights for policy 0, policy_version 918503 (0.0006) [2023-12-27 12:00:40,588][20381] Updated weights for policy 1, policy_version 917875 (0.0006) [2023-12-27 12:00:40,626][20347] Updated weights for policy 0, policy_version 918513 (0.0005) [2023-12-27 12:00:40,676][20347] Updated weights for policy 0, policy_version 918523 (0.0006) [2023-12-27 12:00:41,285][20381] Updated weights for policy 1, policy_version 917885 (0.0008) [2023-12-27 12:00:41,347][20381] Updated weights for policy 1, policy_version 917895 (0.0007) [2023-12-27 12:00:41,384][20347] Updated weights for policy 0, policy_version 918533 (0.0008) [2023-12-27 12:00:41,412][20381] Updated weights for policy 1, policy_version 917905 (0.0009) [2023-12-27 12:00:41,453][20347] Updated weights for policy 0, policy_version 918543 (0.0007) [2023-12-27 12:00:41,514][20347] Updated weights for policy 0, policy_version 918553 (0.0006) [2023-12-27 12:00:42,171][20381] Updated weights for policy 1, policy_version 917915 (0.0007) [2023-12-27 12:00:42,203][20347] Updated weights for policy 0, policy_version 918563 (0.0006) [2023-12-27 12:00:42,230][20381] Updated weights for policy 1, policy_version 917925 (0.0006) [2023-12-27 12:00:42,264][20347] Updated weights for policy 0, policy_version 918573 (0.0008) [2023-12-27 12:00:42,292][20381] Updated weights for policy 1, policy_version 917935 (0.0007) [2023-12-27 12:00:42,326][20347] Updated weights for policy 0, policy_version 918583 (0.0007) [2023-12-27 12:00:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 470204416. Throughput: 0: 10129.7, 1: 10259.3. Samples: 470197036. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:42,363][19170] Avg episode reward: [(0, '971.040'), (1, '488.930')] [2023-12-27 12:00:42,965][20381] Updated weights for policy 1, policy_version 917945 (0.0007) [2023-12-27 12:00:43,014][20381] Updated weights for policy 1, policy_version 917955 (0.0009) [2023-12-27 12:00:43,023][20347] Updated weights for policy 0, policy_version 918593 (0.0008) [2023-12-27 12:00:43,070][20381] Updated weights for policy 1, policy_version 917965 (0.0007) [2023-12-27 12:00:43,087][20347] Updated weights for policy 0, policy_version 918603 (0.0007) [2023-12-27 12:00:43,124][20381] Updated weights for policy 1, policy_version 917975 (0.0006) [2023-12-27 12:00:43,141][20347] Updated weights for policy 0, policy_version 918613 (0.0006) [2023-12-27 12:00:43,201][20347] Updated weights for policy 0, policy_version 918623 (0.0007) [2023-12-27 12:00:43,817][20347] Updated weights for policy 0, policy_version 918633 (0.0007) [2023-12-27 12:00:43,829][20381] Updated weights for policy 1, policy_version 917985 (0.0005) [2023-12-27 12:00:43,868][20347] Updated weights for policy 0, policy_version 918643 (0.0008) [2023-12-27 12:00:43,877][20381] Updated weights for policy 1, policy_version 917995 (0.0005) [2023-12-27 12:00:43,917][20347] Updated weights for policy 0, policy_version 918653 (0.0008) [2023-12-27 12:00:43,926][20381] Updated weights for policy 1, policy_version 918005 (0.0005) [2023-12-27 12:00:44,564][20381] Updated weights for policy 1, policy_version 918015 (0.0009) [2023-12-27 12:00:44,602][20347] Updated weights for policy 0, policy_version 918663 (0.0006) [2023-12-27 12:00:44,616][20381] Updated weights for policy 1, policy_version 918025 (0.0010) [2023-12-27 12:00:44,661][20347] Updated weights for policy 0, policy_version 918673 (0.0005) [2023-12-27 12:00:44,675][20381] Updated weights for policy 1, policy_version 918035 (0.0011) [2023-12-27 12:00:44,713][20347] Updated weights for policy 0, policy_version 918683 (0.0007) [2023-12-27 12:00:45,323][20347] Updated weights for policy 0, policy_version 918693 (0.0006) [2023-12-27 12:00:45,377][20347] Updated weights for policy 0, policy_version 918703 (0.0007) [2023-12-27 12:00:45,412][20381] Updated weights for policy 1, policy_version 918045 (0.0010) [2023-12-27 12:00:45,431][20347] Updated weights for policy 0, policy_version 918713 (0.0006) [2023-12-27 12:00:45,466][20381] Updated weights for policy 1, policy_version 918055 (0.0010) [2023-12-27 12:00:45,524][20381] Updated weights for policy 1, policy_version 918065 (0.0010) [2023-12-27 12:00:46,130][20347] Updated weights for policy 0, policy_version 918723 (0.0006) [2023-12-27 12:00:46,183][20347] Updated weights for policy 0, policy_version 918733 (0.0007) [2023-12-27 12:00:46,209][20381] Updated weights for policy 1, policy_version 918075 (0.0011) [2023-12-27 12:00:46,233][20347] Updated weights for policy 0, policy_version 918743 (0.0006) [2023-12-27 12:00:46,258][20381] Updated weights for policy 1, policy_version 918085 (0.0010) [2023-12-27 12:00:46,306][20381] Updated weights for policy 1, policy_version 918095 (0.0010) [2023-12-27 12:00:46,907][20347] Updated weights for policy 0, policy_version 918753 (0.0006) [2023-12-27 12:00:46,964][20347] Updated weights for policy 0, policy_version 918763 (0.0007) [2023-12-27 12:00:46,998][20381] Updated weights for policy 1, policy_version 918105 (0.0010) [2023-12-27 12:00:47,019][20347] Updated weights for policy 0, policy_version 918773 (0.0006) [2023-12-27 12:00:47,048][20381] Updated weights for policy 1, policy_version 918115 (0.0010) [2023-12-27 12:00:47,075][20347] Updated weights for policy 0, policy_version 918783 (0.0005) [2023-12-27 12:00:47,106][20381] Updated weights for policy 1, policy_version 918125 (0.0010) [2023-12-27 12:00:47,151][20381] Updated weights for policy 1, policy_version 918135 (0.0010) [2023-12-27 12:00:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 470310912. Throughput: 0: 10129.2, 1: 10239.6. Samples: 470318500. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:47,363][19170] Avg episode reward: [(0, '990.420'), (1, '520.230')] [2023-12-27 12:00:47,780][20347] Updated weights for policy 0, policy_version 918793 (0.0006) [2023-12-27 12:00:47,814][20381] Updated weights for policy 1, policy_version 918145 (0.0009) [2023-12-27 12:00:47,839][20347] Updated weights for policy 0, policy_version 918803 (0.0006) [2023-12-27 12:00:47,871][20381] Updated weights for policy 1, policy_version 918155 (0.0006) [2023-12-27 12:00:47,899][20347] Updated weights for policy 0, policy_version 918813 (0.0008) [2023-12-27 12:00:47,928][20381] Updated weights for policy 1, policy_version 918165 (0.0006) [2023-12-27 12:00:48,517][20381] Updated weights for policy 1, policy_version 918175 (0.0007) [2023-12-27 12:00:48,576][20381] Updated weights for policy 1, policy_version 918185 (0.0007) [2023-12-27 12:00:48,622][20347] Updated weights for policy 0, policy_version 918823 (0.0007) [2023-12-27 12:00:48,631][20381] Updated weights for policy 1, policy_version 918195 (0.0007) [2023-12-27 12:00:48,681][20347] Updated weights for policy 0, policy_version 918833 (0.0006) [2023-12-27 12:00:48,740][20347] Updated weights for policy 0, policy_version 918843 (0.0007) [2023-12-27 12:00:49,355][20381] Updated weights for policy 1, policy_version 918205 (0.0006) [2023-12-27 12:00:49,420][20381] Updated weights for policy 1, policy_version 918215 (0.0008) [2023-12-27 12:00:49,444][20347] Updated weights for policy 0, policy_version 918853 (0.0008) [2023-12-27 12:00:49,465][20291] KL-divergence is very high: 152.0162 [2023-12-27 12:00:49,471][20291] KL-divergence is very high: 112.6007 [2023-12-27 12:00:49,478][20381] Updated weights for policy 1, policy_version 918225 (0.0010) [2023-12-27 12:00:49,495][20347] Updated weights for policy 0, policy_version 918863 (0.0007) [2023-12-27 12:00:49,553][20347] Updated weights for policy 0, policy_version 918873 (0.0006) [2023-12-27 12:00:50,197][20381] Updated weights for policy 1, policy_version 918235 (0.0009) [2023-12-27 12:00:50,248][20291] KL-divergence is very high: 426.5657 [2023-12-27 12:00:50,256][20381] Updated weights for policy 1, policy_version 918245 (0.0008) [2023-12-27 12:00:50,266][20347] Updated weights for policy 0, policy_version 918883 (0.0006) [2023-12-27 12:00:50,296][20291] KL-divergence is very high: 368.9599 [2023-12-27 12:00:50,310][20381] Updated weights for policy 1, policy_version 918255 (0.0006) [2023-12-27 12:00:50,328][20347] Updated weights for policy 0, policy_version 918893 (0.0006) [2023-12-27 12:00:50,346][20291] KL-divergence is very high: 193.3216 [2023-12-27 12:00:50,388][20347] Updated weights for policy 0, policy_version 918903 (0.0007) [2023-12-27 12:00:51,033][20381] Updated weights for policy 1, policy_version 918265 (0.0006) [2023-12-27 12:00:51,082][20347] Updated weights for policy 0, policy_version 918913 (0.0008) [2023-12-27 12:00:51,098][20381] Updated weights for policy 1, policy_version 918275 (0.0008) [2023-12-27 12:00:51,143][20347] Updated weights for policy 0, policy_version 918923 (0.0008) [2023-12-27 12:00:51,163][20381] Updated weights for policy 1, policy_version 918285 (0.0007) [2023-12-27 12:00:51,207][20347] Updated weights for policy 0, policy_version 918933 (0.0008) [2023-12-27 12:00:51,221][20381] Updated weights for policy 1, policy_version 918295 (0.0007) [2023-12-27 12:00:51,277][20347] Updated weights for policy 0, policy_version 918943 (0.0008) [2023-12-27 12:00:51,907][20381] Updated weights for policy 1, policy_version 918305 (0.0008) [2023-12-27 12:00:51,988][20381] Updated weights for policy 1, policy_version 918315 (0.0007) [2023-12-27 12:00:52,015][20347] Updated weights for policy 0, policy_version 918953 (0.0006) [2023-12-27 12:00:52,048][20381] Updated weights for policy 1, policy_version 918325 (0.0006) [2023-12-27 12:00:52,075][20347] Updated weights for policy 0, policy_version 918963 (0.0005) [2023-12-27 12:00:52,131][20347] Updated weights for policy 0, policy_version 918973 (0.0007) [2023-12-27 12:00:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 470409216. Throughput: 0: 10095.1, 1: 10263.6. Samples: 470379056. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:52,363][19170] Avg episode reward: [(0, '961.230'), (1, '399.070')] [2023-12-27 12:00:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000918328_235118592.pth... [2023-12-27 12:00:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000918976_235290624.pth... [2023-12-27 12:00:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000917144_234815488.pth [2023-12-27 12:00:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000917792_234987520.pth [2023-12-27 12:00:52,735][20381] Updated weights for policy 1, policy_version 918335 (0.0008) [2023-12-27 12:00:52,787][20381] Updated weights for policy 1, policy_version 918345 (0.0006) [2023-12-27 12:00:52,801][20347] Updated weights for policy 0, policy_version 918983 (0.0007) [2023-12-27 12:00:52,843][20381] Updated weights for policy 1, policy_version 918355 (0.0005) [2023-12-27 12:00:52,849][20347] Updated weights for policy 0, policy_version 918993 (0.0007) [2023-12-27 12:00:52,898][20347] Updated weights for policy 0, policy_version 919003 (0.0008) [2023-12-27 12:00:53,397][20381] Updated weights for policy 1, policy_version 918365 (0.0005) [2023-12-27 12:00:53,452][20381] Updated weights for policy 1, policy_version 918375 (0.0009) [2023-12-27 12:00:53,497][20381] Updated weights for policy 1, policy_version 918385 (0.0010) [2023-12-27 12:00:53,671][20347] Updated weights for policy 0, policy_version 919013 (0.0007) [2023-12-27 12:00:53,724][20347] Updated weights for policy 0, policy_version 919023 (0.0007) [2023-12-27 12:00:53,784][20347] Updated weights for policy 0, policy_version 919034 (0.0009) [2023-12-27 12:00:54,149][20381] Updated weights for policy 1, policy_version 918395 (0.0010) [2023-12-27 12:00:54,206][20381] Updated weights for policy 1, policy_version 918405 (0.0011) [2023-12-27 12:00:54,264][20381] Updated weights for policy 1, policy_version 918415 (0.0010) [2023-12-27 12:00:54,508][20347] Updated weights for policy 0, policy_version 919044 (0.0007) [2023-12-27 12:00:54,558][20347] Updated weights for policy 0, policy_version 919054 (0.0007) [2023-12-27 12:00:54,612][20347] Updated weights for policy 0, policy_version 919064 (0.0007) [2023-12-27 12:00:54,941][20381] Updated weights for policy 1, policy_version 918425 (0.0010) [2023-12-27 12:00:55,006][20381] Updated weights for policy 1, policy_version 918435 (0.0008) [2023-12-27 12:00:55,059][20381] Updated weights for policy 1, policy_version 918445 (0.0008) [2023-12-27 12:00:55,116][20381] Updated weights for policy 1, policy_version 918455 (0.0008) [2023-12-27 12:00:55,265][20347] Updated weights for policy 0, policy_version 919074 (0.0007) [2023-12-27 12:00:55,320][20347] Updated weights for policy 0, policy_version 919084 (0.0007) [2023-12-27 12:00:55,373][20347] Updated weights for policy 0, policy_version 919094 (0.0008) [2023-12-27 12:00:55,424][20347] Updated weights for policy 0, policy_version 919104 (0.0008) [2023-12-27 12:00:55,765][20381] Updated weights for policy 1, policy_version 918465 (0.0007) [2023-12-27 12:00:55,809][20381] Updated weights for policy 1, policy_version 918475 (0.0008) [2023-12-27 12:00:55,854][20381] Updated weights for policy 1, policy_version 918485 (0.0008) [2023-12-27 12:00:56,016][20347] Updated weights for policy 0, policy_version 919114 (0.0009) [2023-12-27 12:00:56,065][20347] Updated weights for policy 0, policy_version 919124 (0.0006) [2023-12-27 12:00:56,121][20347] Updated weights for policy 0, policy_version 919134 (0.0006) [2023-12-27 12:00:56,494][20381] Updated weights for policy 1, policy_version 918495 (0.0008) [2023-12-27 12:00:56,550][20381] Updated weights for policy 1, policy_version 918505 (0.0006) [2023-12-27 12:00:56,603][20381] Updated weights for policy 1, policy_version 918515 (0.0005) [2023-12-27 12:00:56,748][20347] Updated weights for policy 0, policy_version 919144 (0.0009) [2023-12-27 12:00:56,798][20347] Updated weights for policy 0, policy_version 919154 (0.0010) [2023-12-27 12:00:56,852][20347] Updated weights for policy 0, policy_version 919164 (0.0006) [2023-12-27 12:00:57,229][20381] Updated weights for policy 1, policy_version 918525 (0.0008) [2023-12-27 12:00:57,277][20381] Updated weights for policy 1, policy_version 918535 (0.0010) [2023-12-27 12:00:57,328][20381] Updated weights for policy 1, policy_version 918545 (0.0010) [2023-12-27 12:00:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 470507520. Throughput: 0: 10046.0, 1: 10267.9. Samples: 470501752. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:00:57,363][19170] Avg episode reward: [(0, '951.530'), (1, '384.490')] [2023-12-27 12:00:57,463][20347] Updated weights for policy 0, policy_version 919174 (0.0009) [2023-12-27 12:00:57,519][20347] Updated weights for policy 0, policy_version 919184 (0.0010) [2023-12-27 12:00:57,563][20347] Updated weights for policy 0, policy_version 919194 (0.0010) [2023-12-27 12:00:58,007][20381] Updated weights for policy 1, policy_version 918555 (0.0009) [2023-12-27 12:00:58,053][20381] Updated weights for policy 1, policy_version 918565 (0.0007) [2023-12-27 12:00:58,101][20381] Updated weights for policy 1, policy_version 918575 (0.0007) [2023-12-27 12:00:58,220][20347] Updated weights for policy 0, policy_version 919204 (0.0009) [2023-12-27 12:00:58,262][20291] KL-divergence is very high: 192.6930 [2023-12-27 12:00:58,280][20347] Updated weights for policy 0, policy_version 919214 (0.0009) [2023-12-27 12:00:58,346][20347] Updated weights for policy 0, policy_version 919225 (0.0008) [2023-12-27 12:00:58,937][20381] Updated weights for policy 1, policy_version 918585 (0.0007) [2023-12-27 12:00:58,997][20381] Updated weights for policy 1, policy_version 918595 (0.0009) [2023-12-27 12:00:59,056][20381] Updated weights for policy 1, policy_version 918605 (0.0006) [2023-12-27 12:00:59,119][20347] Updated weights for policy 0, policy_version 919235 (0.0007) [2023-12-27 12:00:59,122][20381] Updated weights for policy 1, policy_version 918615 (0.0006) [2023-12-27 12:00:59,173][20347] Updated weights for policy 0, policy_version 919245 (0.0008) [2023-12-27 12:00:59,183][20291] KL-divergence is very high: 364.7598 [2023-12-27 12:00:59,231][20347] Updated weights for policy 0, policy_version 919255 (0.0008) [2023-12-27 12:00:59,284][20291] KL-divergence is very high: 141.7169 [2023-12-27 12:00:59,831][20381] Updated weights for policy 1, policy_version 918625 (0.0010) [2023-12-27 12:00:59,895][20381] Updated weights for policy 1, policy_version 918635 (0.0011) [2023-12-27 12:00:59,936][20347] Updated weights for policy 0, policy_version 919265 (0.0008) [2023-12-27 12:00:59,961][20381] Updated weights for policy 1, policy_version 918645 (0.0009) [2023-12-27 12:00:59,995][20347] Updated weights for policy 0, policy_version 919275 (0.0008) [2023-12-27 12:01:00,050][20347] Updated weights for policy 0, policy_version 919285 (0.0007) [2023-12-27 12:01:00,104][20347] Updated weights for policy 0, policy_version 919295 (0.0007) [2023-12-27 12:01:00,626][20381] Updated weights for policy 1, policy_version 918655 (0.0007) [2023-12-27 12:01:00,673][20381] Updated weights for policy 1, policy_version 918665 (0.0006) [2023-12-27 12:01:00,725][20381] Updated weights for policy 1, policy_version 918675 (0.0007) [2023-12-27 12:01:00,757][20347] Updated weights for policy 0, policy_version 919305 (0.0006) [2023-12-27 12:01:00,810][20347] Updated weights for policy 0, policy_version 919315 (0.0005) [2023-12-27 12:01:00,862][20347] Updated weights for policy 0, policy_version 919325 (0.0005) [2023-12-27 12:01:01,478][20381] Updated weights for policy 1, policy_version 918685 (0.0007) [2023-12-27 12:01:01,536][20381] Updated weights for policy 1, policy_version 918695 (0.0008) [2023-12-27 12:01:01,538][20347] Updated weights for policy 0, policy_version 919335 (0.0006) [2023-12-27 12:01:01,592][20347] Updated weights for policy 0, policy_version 919345 (0.0005) [2023-12-27 12:01:01,594][20381] Updated weights for policy 1, policy_version 918705 (0.0008) [2023-12-27 12:01:01,648][20347] Updated weights for policy 0, policy_version 919355 (0.0010) [2023-12-27 12:01:02,177][20381] Updated weights for policy 1, policy_version 918715 (0.0007) [2023-12-27 12:01:02,227][20381] Updated weights for policy 1, policy_version 918725 (0.0005) [2023-12-27 12:01:02,288][20381] Updated weights for policy 1, policy_version 918735 (0.0006) [2023-12-27 12:01:02,349][20347] Updated weights for policy 0, policy_version 919365 (0.0007) [2023-12-27 12:01:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 470614016. Throughput: 0: 10071.6, 1: 10229.9. Samples: 470624140. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:02,363][19170] Avg episode reward: [(0, '922.400'), (1, '311.020')] [2023-12-27 12:01:02,406][20347] Updated weights for policy 0, policy_version 919375 (0.0008) [2023-12-27 12:01:02,462][20347] Updated weights for policy 0, policy_version 919385 (0.0006) [2023-12-27 12:01:02,966][20381] Updated weights for policy 1, policy_version 918745 (0.0007) [2023-12-27 12:01:03,025][20381] Updated weights for policy 1, policy_version 918755 (0.0011) [2023-12-27 12:01:03,085][20381] Updated weights for policy 1, policy_version 918765 (0.0011) [2023-12-27 12:01:03,152][20381] Updated weights for policy 1, policy_version 918775 (0.0010) [2023-12-27 12:01:03,162][20347] Updated weights for policy 0, policy_version 919395 (0.0006) [2023-12-27 12:01:03,215][20347] Updated weights for policy 0, policy_version 919405 (0.0007) [2023-12-27 12:01:03,261][20347] Updated weights for policy 0, policy_version 919415 (0.0007) [2023-12-27 12:01:03,781][20381] Updated weights for policy 1, policy_version 918785 (0.0010) [2023-12-27 12:01:03,826][20381] Updated weights for policy 1, policy_version 918795 (0.0010) [2023-12-27 12:01:03,887][20381] Updated weights for policy 1, policy_version 918805 (0.0011) [2023-12-27 12:01:03,964][20347] Updated weights for policy 0, policy_version 919425 (0.0007) [2023-12-27 12:01:04,018][20347] Updated weights for policy 0, policy_version 919435 (0.0006) [2023-12-27 12:01:04,074][20347] Updated weights for policy 0, policy_version 919445 (0.0008) [2023-12-27 12:01:04,134][20347] Updated weights for policy 0, policy_version 919455 (0.0008) [2023-12-27 12:01:04,576][20381] Updated weights for policy 1, policy_version 918815 (0.0007) [2023-12-27 12:01:04,646][20381] Updated weights for policy 1, policy_version 918825 (0.0008) [2023-12-27 12:01:04,700][20381] Updated weights for policy 1, policy_version 918835 (0.0009) [2023-12-27 12:01:04,827][20347] Updated weights for policy 0, policy_version 919465 (0.0006) [2023-12-27 12:01:04,884][20347] Updated weights for policy 0, policy_version 919475 (0.0006) [2023-12-27 12:01:04,934][20347] Updated weights for policy 0, policy_version 919485 (0.0005) [2023-12-27 12:01:05,349][20381] Updated weights for policy 1, policy_version 918845 (0.0009) [2023-12-27 12:01:05,398][20381] Updated weights for policy 1, policy_version 918855 (0.0009) [2023-12-27 12:01:05,450][20381] Updated weights for policy 1, policy_version 918865 (0.0008) [2023-12-27 12:01:05,527][20347] Updated weights for policy 0, policy_version 919495 (0.0006) [2023-12-27 12:01:05,575][20347] Updated weights for policy 0, policy_version 919505 (0.0005) [2023-12-27 12:01:05,626][20347] Updated weights for policy 0, policy_version 919515 (0.0008) [2023-12-27 12:01:06,163][20381] Updated weights for policy 1, policy_version 918875 (0.0007) [2023-12-27 12:01:06,226][20381] Updated weights for policy 1, policy_version 918885 (0.0007) [2023-12-27 12:01:06,236][20347] Updated weights for policy 0, policy_version 919525 (0.0010) [2023-12-27 12:01:06,277][20381] Updated weights for policy 1, policy_version 918895 (0.0006) [2023-12-27 12:01:06,300][20347] Updated weights for policy 0, policy_version 919535 (0.0011) [2023-12-27 12:01:06,364][20347] Updated weights for policy 0, policy_version 919545 (0.0011) [2023-12-27 12:01:06,956][20381] Updated weights for policy 1, policy_version 918905 (0.0007) [2023-12-27 12:01:07,005][20381] Updated weights for policy 1, policy_version 918915 (0.0007) [2023-12-27 12:01:07,054][20381] Updated weights for policy 1, policy_version 918925 (0.0008) [2023-12-27 12:01:07,054][20347] Updated weights for policy 0, policy_version 919555 (0.0011) [2023-12-27 12:01:07,107][20381] Updated weights for policy 1, policy_version 918935 (0.0009) [2023-12-27 12:01:07,109][20347] Updated weights for policy 0, policy_version 919565 (0.0010) [2023-12-27 12:01:07,171][20347] Updated weights for policy 0, policy_version 919575 (0.0010) [2023-12-27 12:01:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 470720512. Throughput: 0: 10108.1, 1: 10230.6. Samples: 470685108. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:07,363][19170] Avg episode reward: [(0, '941.750'), (1, '450.970')] [2023-12-27 12:01:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000919584_235446272.pth... [2023-12-27 12:01:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000918936_235274240.pth... [2023-12-27 12:01:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000918400_235143168.pth [2023-12-27 12:01:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000917752_234971136.pth [2023-12-27 12:01:07,741][20347] Updated weights for policy 0, policy_version 919585 (0.0010) [2023-12-27 12:01:07,790][20291] KL-divergence is very high: 128.7003 [2023-12-27 12:01:07,796][20347] Updated weights for policy 0, policy_version 919595 (0.0006) [2023-12-27 12:01:07,812][20291] KL-divergence is very high: 185.5573 [2023-12-27 12:01:07,849][20347] Updated weights for policy 0, policy_version 919605 (0.0006) [2023-12-27 12:01:07,863][20381] Updated weights for policy 1, policy_version 918945 (0.0007) [2023-12-27 12:01:07,906][20347] Updated weights for policy 0, policy_version 919615 (0.0008) [2023-12-27 12:01:07,918][20381] Updated weights for policy 1, policy_version 918955 (0.0006) [2023-12-27 12:01:07,978][20381] Updated weights for policy 1, policy_version 918965 (0.0005) [2023-12-27 12:01:08,613][20347] Updated weights for policy 0, policy_version 919625 (0.0008) [2023-12-27 12:01:08,657][20381] Updated weights for policy 1, policy_version 918975 (0.0007) [2023-12-27 12:01:08,671][20347] Updated weights for policy 0, policy_version 919635 (0.0007) [2023-12-27 12:01:08,706][20381] Updated weights for policy 1, policy_version 918985 (0.0005) [2023-12-27 12:01:08,732][20347] Updated weights for policy 0, policy_version 919645 (0.0008) [2023-12-27 12:01:08,767][20381] Updated weights for policy 1, policy_version 918995 (0.0009) [2023-12-27 12:01:09,452][20381] Updated weights for policy 1, policy_version 919005 (0.0008) [2023-12-27 12:01:09,515][20381] Updated weights for policy 1, policy_version 919015 (0.0008) [2023-12-27 12:01:09,535][20347] Updated weights for policy 0, policy_version 919655 (0.0007) [2023-12-27 12:01:09,582][20381] Updated weights for policy 1, policy_version 919025 (0.0008) [2023-12-27 12:01:09,597][20347] Updated weights for policy 0, policy_version 919665 (0.0008) [2023-12-27 12:01:09,659][20347] Updated weights for policy 0, policy_version 919675 (0.0007) [2023-12-27 12:01:10,272][20381] Updated weights for policy 1, policy_version 919035 (0.0008) [2023-12-27 12:01:10,335][20381] Updated weights for policy 1, policy_version 919045 (0.0007) [2023-12-27 12:01:10,357][20347] Updated weights for policy 0, policy_version 919685 (0.0007) [2023-12-27 12:01:10,394][20381] Updated weights for policy 1, policy_version 919055 (0.0006) [2023-12-27 12:01:10,417][20347] Updated weights for policy 0, policy_version 919695 (0.0007) [2023-12-27 12:01:10,469][20347] Updated weights for policy 0, policy_version 919705 (0.0007) [2023-12-27 12:01:11,036][20381] Updated weights for policy 1, policy_version 919065 (0.0007) [2023-12-27 12:01:11,101][20381] Updated weights for policy 1, policy_version 919075 (0.0008) [2023-12-27 12:01:11,167][20381] Updated weights for policy 1, policy_version 919085 (0.0008) [2023-12-27 12:01:11,184][20347] Updated weights for policy 0, policy_version 919715 (0.0008) [2023-12-27 12:01:11,220][20381] Updated weights for policy 1, policy_version 919095 (0.0008) [2023-12-27 12:01:11,248][20347] Updated weights for policy 0, policy_version 919725 (0.0007) [2023-12-27 12:01:11,307][20347] Updated weights for policy 0, policy_version 919735 (0.0008) [2023-12-27 12:01:12,027][20347] Updated weights for policy 0, policy_version 919745 (0.0007) [2023-12-27 12:01:12,048][20381] Updated weights for policy 1, policy_version 919105 (0.0007) [2023-12-27 12:01:12,075][20347] Updated weights for policy 0, policy_version 919755 (0.0007) [2023-12-27 12:01:12,100][20381] Updated weights for policy 1, policy_version 919115 (0.0005) [2023-12-27 12:01:12,124][20347] Updated weights for policy 0, policy_version 919765 (0.0007) [2023-12-27 12:01:12,150][20381] Updated weights for policy 1, policy_version 919125 (0.0006) [2023-12-27 12:01:12,181][20347] Updated weights for policy 0, policy_version 919775 (0.0006) [2023-12-27 12:01:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 470818816. Throughput: 0: 10137.8, 1: 10190.1. Samples: 470806300. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:12,363][19170] Avg episode reward: [(0, '970.800'), (1, '388.330')] [2023-12-27 12:01:12,845][20347] Updated weights for policy 0, policy_version 919785 (0.0009) [2023-12-27 12:01:12,875][20381] Updated weights for policy 1, policy_version 919135 (0.0006) [2023-12-27 12:01:12,907][20347] Updated weights for policy 0, policy_version 919795 (0.0011) [2023-12-27 12:01:12,933][20381] Updated weights for policy 1, policy_version 919145 (0.0009) [2023-12-27 12:01:12,963][20347] Updated weights for policy 0, policy_version 919805 (0.0010) [2023-12-27 12:01:12,992][20381] Updated weights for policy 1, policy_version 919155 (0.0009) [2023-12-27 12:01:13,552][20381] Updated weights for policy 1, policy_version 919165 (0.0007) [2023-12-27 12:01:13,573][20347] Updated weights for policy 0, policy_version 919815 (0.0007) [2023-12-27 12:01:13,575][20291] KL-divergence is very high: 168.8000 [2023-12-27 12:01:13,589][20291] KL-divergence is very high: 765.4549 [2023-12-27 12:01:13,606][20381] Updated weights for policy 1, policy_version 919175 (0.0005) [2023-12-27 12:01:13,621][20291] KL-divergence is very high: 855.6111 [2023-12-27 12:01:13,632][20347] Updated weights for policy 0, policy_version 919825 (0.0005) [2023-12-27 12:01:13,632][20291] KL-divergence is very high: 860.6685 [2023-12-27 12:01:13,658][20381] Updated weights for policy 1, policy_version 919185 (0.0005) [2023-12-27 12:01:13,667][20291] KL-divergence is very high: 211.0503 [2023-12-27 12:01:13,678][20291] KL-divergence is very high: 152.4272 [2023-12-27 12:01:13,688][20347] Updated weights for policy 0, policy_version 919835 (0.0005) [2023-12-27 12:01:13,689][20291] KL-divergence is very high: 117.8553 [2023-12-27 12:01:14,184][20381] Updated weights for policy 1, policy_version 919195 (0.0007) [2023-12-27 12:01:14,233][20381] Updated weights for policy 1, policy_version 919205 (0.0010) [2023-12-27 12:01:14,257][20291] KL-divergence is very high: 204.5067 [2023-12-27 12:01:14,282][20381] Updated weights for policy 1, policy_version 919215 (0.0010) [2023-12-27 12:01:14,283][20347] Updated weights for policy 0, policy_version 919845 (0.0008) [2023-12-27 12:01:14,301][20291] KL-divergence is very high: 179.7530 [2023-12-27 12:01:14,337][20347] Updated weights for policy 0, policy_version 919855 (0.0010) [2023-12-27 12:01:14,345][20291] KL-divergence is very high: 171.3177 [2023-12-27 12:01:14,389][20291] KL-divergence is very high: 214.7339 [2023-12-27 12:01:14,394][20347] Updated weights for policy 0, policy_version 919865 (0.0005) [2023-12-27 12:01:14,965][20381] Updated weights for policy 1, policy_version 919225 (0.0010) [2023-12-27 12:01:15,026][20381] Updated weights for policy 1, policy_version 919235 (0.0006) [2023-12-27 12:01:15,082][20381] Updated weights for policy 1, policy_version 919245 (0.0006) [2023-12-27 12:01:15,090][20347] Updated weights for policy 0, policy_version 919875 (0.0007) [2023-12-27 12:01:15,142][20381] Updated weights for policy 1, policy_version 919255 (0.0009) [2023-12-27 12:01:15,152][20347] Updated weights for policy 0, policy_version 919885 (0.0007) [2023-12-27 12:01:15,211][20347] Updated weights for policy 0, policy_version 919895 (0.0008) [2023-12-27 12:01:15,817][20381] Updated weights for policy 1, policy_version 919265 (0.0011) [2023-12-27 12:01:15,852][20347] Updated weights for policy 0, policy_version 919905 (0.0007) [2023-12-27 12:01:15,884][20381] Updated weights for policy 1, policy_version 919275 (0.0011) [2023-12-27 12:01:15,901][20347] Updated weights for policy 0, policy_version 919915 (0.0005) [2023-12-27 12:01:15,935][20381] Updated weights for policy 1, policy_version 919285 (0.0011) [2023-12-27 12:01:15,952][20347] Updated weights for policy 0, policy_version 919925 (0.0005) [2023-12-27 12:01:15,953][20291] KL-divergence is very high: 716.8456 [2023-12-27 12:01:16,000][20347] Updated weights for policy 0, policy_version 919935 (0.0006) [2023-12-27 12:01:16,524][20291] KL-divergence is very high: 158.5231 [2023-12-27 12:01:16,570][20347] Updated weights for policy 0, policy_version 919945 (0.0007) [2023-12-27 12:01:16,619][20291] KL-divergence is very high: 179.6792 [2023-12-27 12:01:16,622][20381] Updated weights for policy 1, policy_version 919295 (0.0011) [2023-12-27 12:01:16,631][20347] Updated weights for policy 0, policy_version 919955 (0.0010) [2023-12-27 12:01:16,642][20291] KL-divergence is very high: 115.7298 [2023-12-27 12:01:16,664][20291] KL-divergence is very high: 125.1931 [2023-12-27 12:01:16,667][20381] Updated weights for policy 1, policy_version 919305 (0.0010) [2023-12-27 12:01:16,686][20347] Updated weights for policy 0, policy_version 919965 (0.0010) [2023-12-27 12:01:16,719][20381] Updated weights for policy 1, policy_version 919315 (0.0011) [2023-12-27 12:01:17,203][20291] KL-divergence is very high: 164.2656 [2023-12-27 12:01:17,236][20347] Updated weights for policy 0, policy_version 919975 (0.0007) [2023-12-27 12:01:17,247][20291] KL-divergence is very high: 263.4914 [2023-12-27 12:01:17,292][20347] Updated weights for policy 0, policy_version 919985 (0.0005) [2023-12-27 12:01:17,293][20291] KL-divergence is very high: 256.9469 [2023-12-27 12:01:17,349][20347] Updated weights for policy 0, policy_version 919995 (0.0006) [2023-12-27 12:01:17,349][20291] KL-divergence is very high: 178.1223 [2023-12-27 12:01:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 470917120. Throughput: 0: 10196.6, 1: 10200.1. Samples: 470933648. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:17,363][19170] Avg episode reward: [(0, '875.150'), (1, '447.410')] [2023-12-27 12:01:17,409][20381] Updated weights for policy 1, policy_version 919325 (0.0010) [2023-12-27 12:01:17,465][20381] Updated weights for policy 1, policy_version 919335 (0.0011) [2023-12-27 12:01:17,523][20381] Updated weights for policy 1, policy_version 919345 (0.0011) [2023-12-27 12:01:17,914][20347] Updated weights for policy 0, policy_version 920005 (0.0006) [2023-12-27 12:01:17,950][20291] KL-divergence is very high: 214.9821 [2023-12-27 12:01:17,971][20291] KL-divergence is very high: 105.9238 [2023-12-27 12:01:17,979][20347] Updated weights for policy 0, policy_version 920015 (0.0007) [2023-12-27 12:01:18,005][20291] KL-divergence is very high: 120.6711 [2023-12-27 12:01:18,041][20347] Updated weights for policy 0, policy_version 920025 (0.0008) [2023-12-27 12:01:18,254][20381] Updated weights for policy 1, policy_version 919355 (0.0010) [2023-12-27 12:01:18,306][20381] Updated weights for policy 1, policy_version 919365 (0.0010) [2023-12-27 12:01:18,362][20381] Updated weights for policy 1, policy_version 919375 (0.0010) [2023-12-27 12:01:18,681][20347] Updated weights for policy 0, policy_version 920035 (0.0007) [2023-12-27 12:01:18,748][20347] Updated weights for policy 0, policy_version 920045 (0.0007) [2023-12-27 12:01:18,811][20347] Updated weights for policy 0, policy_version 920055 (0.0006) [2023-12-27 12:01:19,048][20381] Updated weights for policy 1, policy_version 919385 (0.0011) [2023-12-27 12:01:19,102][20381] Updated weights for policy 1, policy_version 919395 (0.0009) [2023-12-27 12:01:19,151][20381] Updated weights for policy 1, policy_version 919405 (0.0009) [2023-12-27 12:01:19,198][20381] Updated weights for policy 1, policy_version 919415 (0.0010) [2023-12-27 12:01:19,399][20347] Updated weights for policy 0, policy_version 920065 (0.0005) [2023-12-27 12:01:19,446][20347] Updated weights for policy 0, policy_version 920075 (0.0005) [2023-12-27 12:01:19,503][20347] Updated weights for policy 0, policy_version 920085 (0.0006) [2023-12-27 12:01:19,565][20347] Updated weights for policy 0, policy_version 920095 (0.0007) [2023-12-27 12:01:19,966][20381] Updated weights for policy 1, policy_version 919425 (0.0011) [2023-12-27 12:01:20,021][20381] Updated weights for policy 1, policy_version 919435 (0.0011) [2023-12-27 12:01:20,085][20381] Updated weights for policy 1, policy_version 919445 (0.0011) [2023-12-27 12:01:20,215][20291] KL-divergence is very high: 135.3727 [2023-12-27 12:01:20,222][20291] KL-divergence is very high: 140.5552 [2023-12-27 12:01:20,228][20347] Updated weights for policy 0, policy_version 920105 (0.0009) [2023-12-27 12:01:20,236][20291] KL-divergence is very high: 136.6947 [2023-12-27 12:01:20,295][20347] Updated weights for policy 0, policy_version 920115 (0.0008) [2023-12-27 12:01:20,360][20347] Updated weights for policy 0, policy_version 920125 (0.0008) [2023-12-27 12:01:20,370][20291] KL-divergence is very high: 112.2809 [2023-12-27 12:01:20,796][20381] Updated weights for policy 1, policy_version 919455 (0.0011) [2023-12-27 12:01:20,852][20381] Updated weights for policy 1, policy_version 919465 (0.0011) [2023-12-27 12:01:20,902][20381] Updated weights for policy 1, policy_version 919475 (0.0010) [2023-12-27 12:01:21,012][20347] Updated weights for policy 0, policy_version 920135 (0.0009) [2023-12-27 12:01:21,075][20347] Updated weights for policy 0, policy_version 920145 (0.0010) [2023-12-27 12:01:21,135][20347] Updated weights for policy 0, policy_version 920155 (0.0008) [2023-12-27 12:01:21,706][20381] Updated weights for policy 1, policy_version 919485 (0.0009) [2023-12-27 12:01:21,773][20381] Updated weights for policy 1, policy_version 919495 (0.0006) [2023-12-27 12:01:21,836][20381] Updated weights for policy 1, policy_version 919505 (0.0006) [2023-12-27 12:01:21,858][20347] Updated weights for policy 0, policy_version 920165 (0.0008) [2023-12-27 12:01:21,918][20347] Updated weights for policy 0, policy_version 920175 (0.0009) [2023-12-27 12:01:21,981][20347] Updated weights for policy 0, policy_version 920185 (0.0008) [2023-12-27 12:01:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 471023616. Throughput: 0: 10251.2, 1: 10186.1. Samples: 470996068. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:22,363][19170] Avg episode reward: [(0, '845.790'), (1, '591.100')] [2023-12-27 12:01:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000919512_235421696.pth... [2023-12-27 12:01:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000920192_235601920.pth... [2023-12-27 12:01:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000918328_235118592.pth [2023-12-27 12:01:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000918976_235290624.pth [2023-12-27 12:01:22,509][20381] Updated weights for policy 1, policy_version 919515 (0.0007) [2023-12-27 12:01:22,562][20381] Updated weights for policy 1, policy_version 919525 (0.0011) [2023-12-27 12:01:22,623][20381] Updated weights for policy 1, policy_version 919535 (0.0011) [2023-12-27 12:01:22,721][20347] Updated weights for policy 0, policy_version 920195 (0.0007) [2023-12-27 12:01:22,781][20347] Updated weights for policy 0, policy_version 920205 (0.0009) [2023-12-27 12:01:22,839][20347] Updated weights for policy 0, policy_version 920215 (0.0010) [2023-12-27 12:01:23,312][20381] Updated weights for policy 1, policy_version 919545 (0.0010) [2023-12-27 12:01:23,373][20381] Updated weights for policy 1, policy_version 919555 (0.0006) [2023-12-27 12:01:23,434][20381] Updated weights for policy 1, policy_version 919565 (0.0007) [2023-12-27 12:01:23,435][20347] Updated weights for policy 0, policy_version 920225 (0.0010) [2023-12-27 12:01:23,489][20381] Updated weights for policy 1, policy_version 919575 (0.0011) [2023-12-27 12:01:23,491][20347] Updated weights for policy 0, policy_version 920235 (0.0005) [2023-12-27 12:01:23,551][20347] Updated weights for policy 0, policy_version 920245 (0.0005) [2023-12-27 12:01:23,603][20347] Updated weights for policy 0, policy_version 920255 (0.0008) [2023-12-27 12:01:24,089][20381] Updated weights for policy 1, policy_version 919585 (0.0006) [2023-12-27 12:01:24,141][20381] Updated weights for policy 1, policy_version 919595 (0.0007) [2023-12-27 12:01:24,198][20381] Updated weights for policy 1, policy_version 919605 (0.0009) [2023-12-27 12:01:24,225][20347] Updated weights for policy 0, policy_version 920265 (0.0005) [2023-12-27 12:01:24,277][20347] Updated weights for policy 0, policy_version 920275 (0.0006) [2023-12-27 12:01:24,330][20347] Updated weights for policy 0, policy_version 920285 (0.0005) [2023-12-27 12:01:24,884][20381] Updated weights for policy 1, policy_version 919615 (0.0007) [2023-12-27 12:01:24,886][20347] Updated weights for policy 0, policy_version 920295 (0.0009) [2023-12-27 12:01:24,938][20347] Updated weights for policy 0, policy_version 920305 (0.0005) [2023-12-27 12:01:24,940][20381] Updated weights for policy 1, policy_version 919625 (0.0007) [2023-12-27 12:01:24,994][20347] Updated weights for policy 0, policy_version 920315 (0.0007) [2023-12-27 12:01:24,997][20381] Updated weights for policy 1, policy_version 919635 (0.0006) [2023-12-27 12:01:25,623][20347] Updated weights for policy 0, policy_version 920325 (0.0006) [2023-12-27 12:01:25,653][20291] KL-divergence is very high: 134.6304 [2023-12-27 12:01:25,664][20291] KL-divergence is very high: 122.8686 [2023-12-27 12:01:25,680][20347] Updated weights for policy 0, policy_version 920335 (0.0009) [2023-12-27 12:01:25,704][20381] Updated weights for policy 1, policy_version 919645 (0.0006) [2023-12-27 12:01:25,739][20347] Updated weights for policy 0, policy_version 920345 (0.0010) [2023-12-27 12:01:25,757][20381] Updated weights for policy 1, policy_version 919655 (0.0006) [2023-12-27 12:01:25,819][20381] Updated weights for policy 1, policy_version 919665 (0.0007) [2023-12-27 12:01:26,337][20347] Updated weights for policy 0, policy_version 920355 (0.0008) [2023-12-27 12:01:26,392][20347] Updated weights for policy 0, policy_version 920365 (0.0005) [2023-12-27 12:01:26,448][20347] Updated weights for policy 0, policy_version 920375 (0.0005) [2023-12-27 12:01:26,534][20381] Updated weights for policy 1, policy_version 919675 (0.0007) [2023-12-27 12:01:26,585][20381] Updated weights for policy 1, policy_version 919685 (0.0005) [2023-12-27 12:01:26,631][20381] Updated weights for policy 1, policy_version 919695 (0.0005) [2023-12-27 12:01:26,945][20347] Updated weights for policy 0, policy_version 920385 (0.0005) [2023-12-27 12:01:26,995][20347] Updated weights for policy 0, policy_version 920395 (0.0005) [2023-12-27 12:01:27,045][20347] Updated weights for policy 0, policy_version 920405 (0.0005) [2023-12-27 12:01:27,093][20347] Updated weights for policy 0, policy_version 920415 (0.0005) [2023-12-27 12:01:27,184][20381] Updated weights for policy 1, policy_version 919705 (0.0005) [2023-12-27 12:01:27,237][20381] Updated weights for policy 1, policy_version 919715 (0.0005) [2023-12-27 12:01:27,284][20381] Updated weights for policy 1, policy_version 919725 (0.0007) [2023-12-27 12:01:27,341][20381] Updated weights for policy 1, policy_version 919735 (0.0006) [2023-12-27 12:01:27,363][19170] Fps is (10 sec: 22117.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 471138304. Throughput: 0: 10351.0, 1: 10153.6. Samples: 471119744. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:27,364][19170] Avg episode reward: [(0, '809.070'), (1, '616.260')] [2023-12-27 12:01:27,620][20291] KL-divergence is very high: 375.5768 [2023-12-27 12:01:27,625][20291] KL-divergence is very high: 776.2113 [2023-12-27 12:01:27,635][20291] KL-divergence is very high: 349.4622 [2023-12-27 12:01:27,635][20347] Updated weights for policy 0, policy_version 920425 (0.0008) [2023-12-27 12:01:27,659][20291] KL-divergence is very high: 100.9339 [2023-12-27 12:01:27,663][20291] KL-divergence is very high: 256.1832 [2023-12-27 12:01:27,682][20347] Updated weights for policy 0, policy_version 920435 (0.0008) [2023-12-27 12:01:27,731][20347] Updated weights for policy 0, policy_version 920445 (0.0005) [2023-12-27 12:01:27,920][20381] Updated weights for policy 1, policy_version 919745 (0.0006) [2023-12-27 12:01:27,972][20381] Updated weights for policy 1, policy_version 919755 (0.0006) [2023-12-27 12:01:28,036][20381] Updated weights for policy 1, policy_version 919765 (0.0006) [2023-12-27 12:01:28,351][20347] Updated weights for policy 0, policy_version 920455 (0.0007) [2023-12-27 12:01:28,403][20347] Updated weights for policy 0, policy_version 920465 (0.0006) [2023-12-27 12:01:28,460][20347] Updated weights for policy 0, policy_version 920475 (0.0007) [2023-12-27 12:01:28,579][20381] Updated weights for policy 1, policy_version 919775 (0.0006) [2023-12-27 12:01:28,638][20381] Updated weights for policy 1, policy_version 919785 (0.0008) [2023-12-27 12:01:28,688][20381] Updated weights for policy 1, policy_version 919795 (0.0009) [2023-12-27 12:01:29,074][20347] Updated weights for policy 0, policy_version 920485 (0.0008) [2023-12-27 12:01:29,121][20347] Updated weights for policy 0, policy_version 920495 (0.0007) [2023-12-27 12:01:29,169][20347] Updated weights for policy 0, policy_version 920505 (0.0007) [2023-12-27 12:01:29,403][20381] Updated weights for policy 1, policy_version 919805 (0.0008) [2023-12-27 12:01:29,459][20381] Updated weights for policy 1, policy_version 919815 (0.0007) [2023-12-27 12:01:29,516][20381] Updated weights for policy 1, policy_version 919825 (0.0007) [2023-12-27 12:01:29,810][20347] Updated weights for policy 0, policy_version 920515 (0.0006) [2023-12-27 12:01:29,871][20347] Updated weights for policy 0, policy_version 920525 (0.0008) [2023-12-27 12:01:29,936][20347] Updated weights for policy 0, policy_version 920535 (0.0009) [2023-12-27 12:01:30,149][20381] Updated weights for policy 1, policy_version 919835 (0.0007) [2023-12-27 12:01:30,205][20381] Updated weights for policy 1, policy_version 919845 (0.0008) [2023-12-27 12:01:30,262][20381] Updated weights for policy 1, policy_version 919855 (0.0006) [2023-12-27 12:01:30,612][20347] Updated weights for policy 0, policy_version 920545 (0.0008) [2023-12-27 12:01:30,663][20347] Updated weights for policy 0, policy_version 920555 (0.0008) [2023-12-27 12:01:30,711][20347] Updated weights for policy 0, policy_version 920565 (0.0007) [2023-12-27 12:01:30,763][20347] Updated weights for policy 0, policy_version 920575 (0.0007) [2023-12-27 12:01:30,898][20381] Updated weights for policy 1, policy_version 919865 (0.0006) [2023-12-27 12:01:30,962][20381] Updated weights for policy 1, policy_version 919875 (0.0008) [2023-12-27 12:01:31,026][20381] Updated weights for policy 1, policy_version 919885 (0.0009) [2023-12-27 12:01:31,093][20381] Updated weights for policy 1, policy_version 919895 (0.0007) [2023-12-27 12:01:31,428][20347] Updated weights for policy 0, policy_version 920585 (0.0008) [2023-12-27 12:01:31,480][20347] Updated weights for policy 0, policy_version 920595 (0.0008) [2023-12-27 12:01:31,535][20347] Updated weights for policy 0, policy_version 920605 (0.0008) [2023-12-27 12:01:31,762][20381] Updated weights for policy 1, policy_version 919905 (0.0007) [2023-12-27 12:01:31,821][20381] Updated weights for policy 1, policy_version 919915 (0.0007) [2023-12-27 12:01:31,882][20381] Updated weights for policy 1, policy_version 919925 (0.0008) [2023-12-27 12:01:32,263][20347] Updated weights for policy 0, policy_version 920615 (0.0008) [2023-12-27 12:01:32,317][20347] Updated weights for policy 0, policy_version 920625 (0.0009) [2023-12-27 12:01:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 471236608. Throughput: 0: 10477.4, 1: 10265.3. Samples: 471251924. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:32,363][19170] Avg episode reward: [(0, '734.260'), (1, '667.050')] [2023-12-27 12:01:32,376][20347] Updated weights for policy 0, policy_version 920635 (0.0008) [2023-12-27 12:01:32,561][20381] Updated weights for policy 1, policy_version 919935 (0.0006) [2023-12-27 12:01:32,620][20381] Updated weights for policy 1, policy_version 919945 (0.0005) [2023-12-27 12:01:32,678][20381] Updated weights for policy 1, policy_version 919955 (0.0006) [2023-12-27 12:01:33,104][20347] Updated weights for policy 0, policy_version 920645 (0.0008) [2023-12-27 12:01:33,162][20347] Updated weights for policy 0, policy_version 920655 (0.0007) [2023-12-27 12:01:33,217][20347] Updated weights for policy 0, policy_version 920665 (0.0007) [2023-12-27 12:01:33,349][20381] Updated weights for policy 1, policy_version 919965 (0.0007) [2023-12-27 12:01:33,404][20381] Updated weights for policy 1, policy_version 919975 (0.0008) [2023-12-27 12:01:33,465][20381] Updated weights for policy 1, policy_version 919985 (0.0008) [2023-12-27 12:01:33,948][20347] Updated weights for policy 0, policy_version 920675 (0.0007) [2023-12-27 12:01:33,998][20347] Updated weights for policy 0, policy_version 920685 (0.0007) [2023-12-27 12:01:34,052][20347] Updated weights for policy 0, policy_version 920695 (0.0007) [2023-12-27 12:01:34,122][20381] Updated weights for policy 1, policy_version 919995 (0.0009) [2023-12-27 12:01:34,190][20381] Updated weights for policy 1, policy_version 920005 (0.0010) [2023-12-27 12:01:34,257][20381] Updated weights for policy 1, policy_version 920015 (0.0011) [2023-12-27 12:01:34,682][20347] Updated weights for policy 0, policy_version 920705 (0.0007) [2023-12-27 12:01:34,740][20347] Updated weights for policy 0, policy_version 920715 (0.0005) [2023-12-27 12:01:34,795][20347] Updated weights for policy 0, policy_version 920725 (0.0007) [2023-12-27 12:01:34,847][20347] Updated weights for policy 0, policy_version 920735 (0.0005) [2023-12-27 12:01:34,965][20381] Updated weights for policy 1, policy_version 920025 (0.0011) [2023-12-27 12:01:35,023][20381] Updated weights for policy 1, policy_version 920035 (0.0010) [2023-12-27 12:01:35,078][20381] Updated weights for policy 1, policy_version 920045 (0.0010) [2023-12-27 12:01:35,123][20381] Updated weights for policy 1, policy_version 920055 (0.0009) [2023-12-27 12:01:35,471][20347] Updated weights for policy 0, policy_version 920745 (0.0006) [2023-12-27 12:01:35,527][20347] Updated weights for policy 0, policy_version 920755 (0.0006) [2023-12-27 12:01:35,616][20347] Updated weights for policy 0, policy_version 920765 (0.0006) [2023-12-27 12:01:35,792][20381] Updated weights for policy 1, policy_version 920065 (0.0010) [2023-12-27 12:01:35,833][20381] Updated weights for policy 1, policy_version 920075 (0.0010) [2023-12-27 12:01:35,878][20381] Updated weights for policy 1, policy_version 920085 (0.0010) [2023-12-27 12:01:36,172][20347] Updated weights for policy 0, policy_version 920775 (0.0009) [2023-12-27 12:01:36,234][20347] Updated weights for policy 0, policy_version 920785 (0.0009) [2023-12-27 12:01:36,296][20347] Updated weights for policy 0, policy_version 920795 (0.0006) [2023-12-27 12:01:36,611][20381] Updated weights for policy 1, policy_version 920095 (0.0009) [2023-12-27 12:01:36,675][20381] Updated weights for policy 1, policy_version 920105 (0.0008) [2023-12-27 12:01:36,732][20381] Updated weights for policy 1, policy_version 920115 (0.0005) [2023-12-27 12:01:36,980][20347] Updated weights for policy 0, policy_version 920805 (0.0009) [2023-12-27 12:01:37,029][20347] Updated weights for policy 0, policy_version 920815 (0.0010) [2023-12-27 12:01:37,087][20347] Updated weights for policy 0, policy_version 920825 (0.0010) [2023-12-27 12:01:37,328][20381] Updated weights for policy 1, policy_version 920125 (0.0006) [2023-12-27 12:01:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 471343104. Throughput: 0: 10506.6, 1: 10244.4. Samples: 471312848. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:37,363][19170] Avg episode reward: [(0, '827.660'), (1, '631.740')] [2023-12-27 12:01:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000920832_235765760.pth... [2023-12-27 12:01:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000919584_235446272.pth [2023-12-27 12:01:37,388][20381] Updated weights for policy 1, policy_version 920135 (0.0006) [2023-12-27 12:01:37,456][20381] Updated weights for policy 1, policy_version 920145 (0.0007) [2023-12-27 12:01:37,499][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000920152_235585536.pth... [2023-12-27 12:01:37,503][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000918936_235274240.pth [2023-12-27 12:01:37,726][20347] Updated weights for policy 0, policy_version 920835 (0.0009) [2023-12-27 12:01:37,776][20347] Updated weights for policy 0, policy_version 920845 (0.0005) [2023-12-27 12:01:37,827][20347] Updated weights for policy 0, policy_version 920855 (0.0007) [2023-12-27 12:01:38,155][20381] Updated weights for policy 1, policy_version 920155 (0.0008) [2023-12-27 12:01:38,213][20381] Updated weights for policy 1, policy_version 920165 (0.0008) [2023-12-27 12:01:38,260][20381] Updated weights for policy 1, policy_version 920176 (0.0006) [2023-12-27 12:01:38,507][20347] Updated weights for policy 0, policy_version 920865 (0.0008) [2023-12-27 12:01:38,566][20347] Updated weights for policy 0, policy_version 920875 (0.0008) [2023-12-27 12:01:38,627][20347] Updated weights for policy 0, policy_version 920885 (0.0008) [2023-12-27 12:01:38,686][20347] Updated weights for policy 0, policy_version 920895 (0.0007) [2023-12-27 12:01:39,000][20381] Updated weights for policy 1, policy_version 920186 (0.0007) [2023-12-27 12:01:39,052][20381] Updated weights for policy 1, policy_version 920196 (0.0008) [2023-12-27 12:01:39,105][20381] Updated weights for policy 1, policy_version 920206 (0.0007) [2023-12-27 12:01:39,152][20381] Updated weights for policy 1, policy_version 920216 (0.0008) [2023-12-27 12:01:39,303][20347] Updated weights for policy 0, policy_version 920905 (0.0008) [2023-12-27 12:01:39,360][20347] Updated weights for policy 0, policy_version 920915 (0.0008) [2023-12-27 12:01:39,429][20347] Updated weights for policy 0, policy_version 920925 (0.0008) [2023-12-27 12:01:39,922][20381] Updated weights for policy 1, policy_version 920226 (0.0009) [2023-12-27 12:01:39,989][20381] Updated weights for policy 1, policy_version 920236 (0.0007) [2023-12-27 12:01:40,051][20381] Updated weights for policy 1, policy_version 920246 (0.0007) [2023-12-27 12:01:40,164][20347] Updated weights for policy 0, policy_version 920935 (0.0006) [2023-12-27 12:01:40,225][20347] Updated weights for policy 0, policy_version 920945 (0.0007) [2023-12-27 12:01:40,284][20347] Updated weights for policy 0, policy_version 920955 (0.0007) [2023-12-27 12:01:40,729][20381] Updated weights for policy 1, policy_version 920256 (0.0006) [2023-12-27 12:01:40,782][20381] Updated weights for policy 1, policy_version 920266 (0.0007) [2023-12-27 12:01:40,838][20381] Updated weights for policy 1, policy_version 920276 (0.0008) [2023-12-27 12:01:40,990][20347] Updated weights for policy 0, policy_version 920965 (0.0008) [2023-12-27 12:01:41,050][20347] Updated weights for policy 0, policy_version 920975 (0.0008) [2023-12-27 12:01:41,118][20347] Updated weights for policy 0, policy_version 920985 (0.0008) [2023-12-27 12:01:41,589][20381] Updated weights for policy 1, policy_version 920286 (0.0010) [2023-12-27 12:01:41,659][20381] Updated weights for policy 1, policy_version 920296 (0.0010) [2023-12-27 12:01:41,723][20381] Updated weights for policy 1, policy_version 920306 (0.0008) [2023-12-27 12:01:41,829][20347] Updated weights for policy 0, policy_version 920995 (0.0007) [2023-12-27 12:01:41,890][20347] Updated weights for policy 0, policy_version 921005 (0.0006) [2023-12-27 12:01:41,936][20347] Updated weights for policy 0, policy_version 921015 (0.0005) [2023-12-27 12:01:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 471441408. Throughput: 0: 10562.1, 1: 10183.5. Samples: 471435308. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:42,363][19170] Avg episode reward: [(0, '894.140'), (1, '759.280')] [2023-12-27 12:01:42,497][20381] Updated weights for policy 1, policy_version 920316 (0.0009) [2023-12-27 12:01:42,522][20347] Updated weights for policy 0, policy_version 921025 (0.0006) [2023-12-27 12:01:42,553][20381] Updated weights for policy 1, policy_version 920326 (0.0011) [2023-12-27 12:01:42,578][20347] Updated weights for policy 0, policy_version 921035 (0.0006) [2023-12-27 12:01:42,606][20381] Updated weights for policy 1, policy_version 920336 (0.0011) [2023-12-27 12:01:42,639][20347] Updated weights for policy 0, policy_version 921045 (0.0007) [2023-12-27 12:01:42,692][20347] Updated weights for policy 0, policy_version 921055 (0.0006) [2023-12-27 12:01:43,266][20347] Updated weights for policy 0, policy_version 921065 (0.0006) [2023-12-27 12:01:43,283][20381] Updated weights for policy 1, policy_version 920346 (0.0010) [2023-12-27 12:01:43,325][20347] Updated weights for policy 0, policy_version 921075 (0.0006) [2023-12-27 12:01:43,333][20381] Updated weights for policy 1, policy_version 920356 (0.0007) [2023-12-27 12:01:43,380][20347] Updated weights for policy 0, policy_version 921085 (0.0007) [2023-12-27 12:01:43,390][20381] Updated weights for policy 1, policy_version 920366 (0.0006) [2023-12-27 12:01:43,446][20381] Updated weights for policy 1, policy_version 920376 (0.0008) [2023-12-27 12:01:44,023][20347] Updated weights for policy 0, policy_version 921095 (0.0006) [2023-12-27 12:01:44,085][20347] Updated weights for policy 0, policy_version 921105 (0.0008) [2023-12-27 12:01:44,128][20381] Updated weights for policy 1, policy_version 920386 (0.0009) [2023-12-27 12:01:44,143][20347] Updated weights for policy 0, policy_version 921115 (0.0010) [2023-12-27 12:01:44,190][20381] Updated weights for policy 1, policy_version 920396 (0.0007) [2023-12-27 12:01:44,249][20381] Updated weights for policy 1, policy_version 920406 (0.0008) [2023-12-27 12:01:44,806][20347] Updated weights for policy 0, policy_version 921125 (0.0011) [2023-12-27 12:01:44,868][20347] Updated weights for policy 0, policy_version 921135 (0.0008) [2023-12-27 12:01:44,933][20347] Updated weights for policy 0, policy_version 921145 (0.0007) [2023-12-27 12:01:44,954][20381] Updated weights for policy 1, policy_version 920416 (0.0007) [2023-12-27 12:01:45,013][20381] Updated weights for policy 1, policy_version 920426 (0.0007) [2023-12-27 12:01:45,071][20381] Updated weights for policy 1, policy_version 920436 (0.0008) [2023-12-27 12:01:45,560][20347] Updated weights for policy 0, policy_version 921155 (0.0006) [2023-12-27 12:01:45,606][20347] Updated weights for policy 0, policy_version 921165 (0.0005) [2023-12-27 12:01:45,654][20347] Updated weights for policy 0, policy_version 921175 (0.0005) [2023-12-27 12:01:45,816][20381] Updated weights for policy 1, policy_version 920446 (0.0008) [2023-12-27 12:01:45,875][20381] Updated weights for policy 1, policy_version 920456 (0.0008) [2023-12-27 12:01:45,936][20381] Updated weights for policy 1, policy_version 920466 (0.0008) [2023-12-27 12:01:46,180][20347] Updated weights for policy 0, policy_version 921185 (0.0005) [2023-12-27 12:01:46,228][20347] Updated weights for policy 0, policy_version 921195 (0.0005) [2023-12-27 12:01:46,287][20347] Updated weights for policy 0, policy_version 921205 (0.0005) [2023-12-27 12:01:46,337][20347] Updated weights for policy 0, policy_version 921215 (0.0005) [2023-12-27 12:01:46,626][20381] Updated weights for policy 1, policy_version 920476 (0.0007) [2023-12-27 12:01:46,689][20381] Updated weights for policy 1, policy_version 920486 (0.0008) [2023-12-27 12:01:46,748][20381] Updated weights for policy 1, policy_version 920496 (0.0007) [2023-12-27 12:01:46,972][20347] Updated weights for policy 0, policy_version 921225 (0.0009) [2023-12-27 12:01:47,030][20347] Updated weights for policy 0, policy_version 921235 (0.0007) [2023-12-27 12:01:47,083][20347] Updated weights for policy 0, policy_version 921245 (0.0005) [2023-12-27 12:01:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 471547904. Throughput: 0: 10651.5, 1: 10135.0. Samples: 471559536. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:47,363][19170] Avg episode reward: [(0, '835.960'), (1, '668.290')] [2023-12-27 12:01:47,405][20381] Updated weights for policy 1, policy_version 920506 (0.0006) [2023-12-27 12:01:47,456][20381] Updated weights for policy 1, policy_version 920516 (0.0005) [2023-12-27 12:01:47,505][20381] Updated weights for policy 1, policy_version 920526 (0.0005) [2023-12-27 12:01:47,560][20381] Updated weights for policy 1, policy_version 920536 (0.0005) [2023-12-27 12:01:47,645][20291] KL-divergence is very high: 118.7872 [2023-12-27 12:01:47,673][20347] Updated weights for policy 0, policy_version 921255 (0.0007) [2023-12-27 12:01:47,729][20347] Updated weights for policy 0, policy_version 921265 (0.0008) [2023-12-27 12:01:47,776][20347] Updated weights for policy 0, policy_version 921275 (0.0007) [2023-12-27 12:01:48,210][20381] Updated weights for policy 1, policy_version 920546 (0.0007) [2023-12-27 12:01:48,271][20381] Updated weights for policy 1, policy_version 920556 (0.0008) [2023-12-27 12:01:48,327][20381] Updated weights for policy 1, policy_version 920566 (0.0007) [2023-12-27 12:01:48,450][20347] Updated weights for policy 0, policy_version 921285 (0.0007) [2023-12-27 12:01:48,508][20347] Updated weights for policy 0, policy_version 921295 (0.0007) [2023-12-27 12:01:48,568][20347] Updated weights for policy 0, policy_version 921305 (0.0008) [2023-12-27 12:01:48,994][20381] Updated weights for policy 1, policy_version 920576 (0.0007) [2023-12-27 12:01:49,047][20381] Updated weights for policy 1, policy_version 920586 (0.0005) [2023-12-27 12:01:49,104][20381] Updated weights for policy 1, policy_version 920596 (0.0007) [2023-12-27 12:01:49,283][20347] Updated weights for policy 0, policy_version 921315 (0.0008) [2023-12-27 12:01:49,343][20347] Updated weights for policy 0, policy_version 921325 (0.0008) [2023-12-27 12:01:49,404][20347] Updated weights for policy 0, policy_version 921335 (0.0007) [2023-12-27 12:01:49,786][20381] Updated weights for policy 1, policy_version 920606 (0.0009) [2023-12-27 12:01:49,852][20381] Updated weights for policy 1, policy_version 920616 (0.0011) [2023-12-27 12:01:49,914][20381] Updated weights for policy 1, policy_version 920626 (0.0010) [2023-12-27 12:01:50,142][20347] Updated weights for policy 0, policy_version 921345 (0.0007) [2023-12-27 12:01:50,193][20347] Updated weights for policy 0, policy_version 921355 (0.0006) [2023-12-27 12:01:50,243][20347] Updated weights for policy 0, policy_version 921365 (0.0007) [2023-12-27 12:01:50,303][20347] Updated weights for policy 0, policy_version 921375 (0.0008) [2023-12-27 12:01:50,553][20381] Updated weights for policy 1, policy_version 920636 (0.0009) [2023-12-27 12:01:50,616][20381] Updated weights for policy 1, policy_version 920646 (0.0010) [2023-12-27 12:01:50,667][20381] Updated weights for policy 1, policy_version 920656 (0.0009) [2023-12-27 12:01:51,056][20347] Updated weights for policy 0, policy_version 921385 (0.0007) [2023-12-27 12:01:51,120][20347] Updated weights for policy 0, policy_version 921395 (0.0008) [2023-12-27 12:01:51,180][20347] Updated weights for policy 0, policy_version 921405 (0.0007) [2023-12-27 12:01:51,406][20381] Updated weights for policy 1, policy_version 920666 (0.0008) [2023-12-27 12:01:51,470][20381] Updated weights for policy 1, policy_version 920676 (0.0007) [2023-12-27 12:01:51,523][20381] Updated weights for policy 1, policy_version 920686 (0.0008) [2023-12-27 12:01:51,569][20381] Updated weights for policy 1, policy_version 920696 (0.0007) [2023-12-27 12:01:51,849][20347] Updated weights for policy 0, policy_version 921415 (0.0006) [2023-12-27 12:01:51,908][20347] Updated weights for policy 0, policy_version 921425 (0.0006) [2023-12-27 12:01:51,969][20347] Updated weights for policy 0, policy_version 921435 (0.0005) [2023-12-27 12:01:52,262][20381] Updated weights for policy 1, policy_version 920706 (0.0007) [2023-12-27 12:01:52,323][20381] Updated weights for policy 1, policy_version 920716 (0.0007) [2023-12-27 12:01:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 471646208. Throughput: 0: 10661.8, 1: 10139.7. Samples: 471621172. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:01:52,363][19170] Avg episode reward: [(0, '711.930'), (1, '624.750')] [2023-12-27 12:01:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000921440_235921408.pth... [2023-12-27 12:01:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000920192_235601920.pth [2023-12-27 12:01:52,384][20381] Updated weights for policy 1, policy_version 920726 (0.0008) [2023-12-27 12:01:52,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000920728_235732992.pth... [2023-12-27 12:01:52,401][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000919512_235421696.pth [2023-12-27 12:01:52,679][20347] Updated weights for policy 0, policy_version 921445 (0.0006) [2023-12-27 12:01:52,747][20347] Updated weights for policy 0, policy_version 921455 (0.0006) [2023-12-27 12:01:52,811][20347] Updated weights for policy 0, policy_version 921465 (0.0006) [2023-12-27 12:01:53,064][20381] Updated weights for policy 1, policy_version 920736 (0.0006) [2023-12-27 12:01:53,128][20381] Updated weights for policy 1, policy_version 920746 (0.0005) [2023-12-27 12:01:53,180][20381] Updated weights for policy 1, policy_version 920756 (0.0005) [2023-12-27 12:01:53,453][20347] Updated weights for policy 0, policy_version 921475 (0.0007) [2023-12-27 12:01:53,511][20347] Updated weights for policy 0, policy_version 921485 (0.0006) [2023-12-27 12:01:53,569][20347] Updated weights for policy 0, policy_version 921495 (0.0005) [2023-12-27 12:01:53,767][20381] Updated weights for policy 1, policy_version 920766 (0.0007) [2023-12-27 12:01:53,822][20381] Updated weights for policy 1, policy_version 920776 (0.0008) [2023-12-27 12:01:53,879][20381] Updated weights for policy 1, policy_version 920786 (0.0009) [2023-12-27 12:01:54,108][20347] Updated weights for policy 0, policy_version 921505 (0.0005) [2023-12-27 12:01:54,168][20347] Updated weights for policy 0, policy_version 921515 (0.0006) [2023-12-27 12:01:54,232][20347] Updated weights for policy 0, policy_version 921525 (0.0005) [2023-12-27 12:01:54,284][20347] Updated weights for policy 0, policy_version 921535 (0.0005) [2023-12-27 12:01:54,485][20381] Updated weights for policy 1, policy_version 920796 (0.0009) [2023-12-27 12:01:54,543][20381] Updated weights for policy 1, policy_version 920806 (0.0008) [2023-12-27 12:01:54,588][20381] Updated weights for policy 1, policy_version 920816 (0.0007) [2023-12-27 12:01:54,887][20347] Updated weights for policy 0, policy_version 921545 (0.0005) [2023-12-27 12:01:54,943][20347] Updated weights for policy 0, policy_version 921555 (0.0005) [2023-12-27 12:01:55,003][20347] Updated weights for policy 0, policy_version 921565 (0.0005) [2023-12-27 12:01:55,353][20381] Updated weights for policy 1, policy_version 920826 (0.0008) [2023-12-27 12:01:55,403][20381] Updated weights for policy 1, policy_version 920836 (0.0008) [2023-12-27 12:01:55,448][20381] Updated weights for policy 1, policy_version 920846 (0.0008) [2023-12-27 12:01:55,497][20381] Updated weights for policy 1, policy_version 920856 (0.0008) [2023-12-27 12:01:55,588][20347] Updated weights for policy 0, policy_version 921575 (0.0007) [2023-12-27 12:01:55,646][20347] Updated weights for policy 0, policy_version 921585 (0.0007) [2023-12-27 12:01:55,697][20347] Updated weights for policy 0, policy_version 921595 (0.0007) [2023-12-27 12:01:56,206][20381] Updated weights for policy 1, policy_version 920866 (0.0008) [2023-12-27 12:01:56,253][20381] Updated weights for policy 1, policy_version 920876 (0.0007) [2023-12-27 12:01:56,308][20381] Updated weights for policy 1, policy_version 920886 (0.0008) [2023-12-27 12:01:56,390][20347] Updated weights for policy 0, policy_version 921605 (0.0008) [2023-12-27 12:01:56,448][20347] Updated weights for policy 0, policy_version 921615 (0.0008) [2023-12-27 12:01:56,502][20347] Updated weights for policy 0, policy_version 921625 (0.0008) [2023-12-27 12:01:56,507][20291] KL-divergence is very high: 183.1033 [2023-12-27 12:01:56,517][20291] KL-divergence is very high: 176.8692 [2023-12-27 12:01:56,532][20291] KL-divergence is very high: 217.4023 [2023-12-27 12:01:56,965][20381] Updated weights for policy 1, policy_version 920896 (0.0006) [2023-12-27 12:01:57,019][20381] Updated weights for policy 1, policy_version 920906 (0.0005) [2023-12-27 12:01:57,073][20381] Updated weights for policy 1, policy_version 920916 (0.0005) [2023-12-27 12:01:57,207][20291] KL-divergence is very high: 161.8675 [2023-12-27 12:01:57,224][20347] Updated weights for policy 0, policy_version 921635 (0.0007) [2023-12-27 12:01:57,278][20347] Updated weights for policy 0, policy_version 921645 (0.0008) [2023-12-27 12:01:57,338][20347] Updated weights for policy 0, policy_version 921655 (0.0007) [2023-12-27 12:01:57,344][20291] KL-divergence is very high: 144.4031 [2023-12-27 12:01:57,361][20291] KL-divergence is very high: 137.9517 [2023-12-27 12:01:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 471752704. Throughput: 0: 10699.9, 1: 10191.0. Samples: 471746392. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:01:57,363][19170] Avg episode reward: [(0, '566.170'), (1, '667.320')] [2023-12-27 12:01:57,693][20381] Updated weights for policy 1, policy_version 920926 (0.0006) [2023-12-27 12:01:57,744][20381] Updated weights for policy 1, policy_version 920936 (0.0007) [2023-12-27 12:01:57,795][20381] Updated weights for policy 1, policy_version 920946 (0.0007) [2023-12-27 12:01:58,023][20347] Updated weights for policy 0, policy_version 921665 (0.0008) [2023-12-27 12:01:58,076][20347] Updated weights for policy 0, policy_version 921675 (0.0007) [2023-12-27 12:01:58,123][20347] Updated weights for policy 0, policy_version 921685 (0.0007) [2023-12-27 12:01:58,166][20347] Updated weights for policy 0, policy_version 921695 (0.0007) [2023-12-27 12:01:58,500][20381] Updated weights for policy 1, policy_version 920956 (0.0008) [2023-12-27 12:01:58,561][20381] Updated weights for policy 1, policy_version 920966 (0.0008) [2023-12-27 12:01:58,636][20381] Updated weights for policy 1, policy_version 920976 (0.0008) [2023-12-27 12:01:59,001][20347] Updated weights for policy 0, policy_version 921705 (0.0007) [2023-12-27 12:01:59,057][20347] Updated weights for policy 0, policy_version 921715 (0.0008) [2023-12-27 12:01:59,122][20347] Updated weights for policy 0, policy_version 921725 (0.0008) [2023-12-27 12:01:59,322][20381] Updated weights for policy 1, policy_version 920986 (0.0009) [2023-12-27 12:01:59,391][20381] Updated weights for policy 1, policy_version 920996 (0.0008) [2023-12-27 12:01:59,449][20381] Updated weights for policy 1, policy_version 921006 (0.0006) [2023-12-27 12:01:59,500][20381] Updated weights for policy 1, policy_version 921016 (0.0007) [2023-12-27 12:01:59,901][20347] Updated weights for policy 0, policy_version 921735 (0.0008) [2023-12-27 12:01:59,970][20347] Updated weights for policy 0, policy_version 921745 (0.0008) [2023-12-27 12:02:00,037][20347] Updated weights for policy 0, policy_version 921755 (0.0008) [2023-12-27 12:02:00,172][20381] Updated weights for policy 1, policy_version 921026 (0.0007) [2023-12-27 12:02:00,230][20381] Updated weights for policy 1, policy_version 921036 (0.0007) [2023-12-27 12:02:00,286][20381] Updated weights for policy 1, policy_version 921046 (0.0006) [2023-12-27 12:02:00,761][20347] Updated weights for policy 0, policy_version 921765 (0.0006) [2023-12-27 12:02:00,814][20347] Updated weights for policy 0, policy_version 921775 (0.0006) [2023-12-27 12:02:00,865][20347] Updated weights for policy 0, policy_version 921785 (0.0007) [2023-12-27 12:02:00,866][20381] Updated weights for policy 1, policy_version 921056 (0.0005) [2023-12-27 12:02:00,915][20381] Updated weights for policy 1, policy_version 921066 (0.0007) [2023-12-27 12:02:00,961][20381] Updated weights for policy 1, policy_version 921076 (0.0008) [2023-12-27 12:02:01,567][20347] Updated weights for policy 0, policy_version 921795 (0.0007) [2023-12-27 12:02:01,629][20347] Updated weights for policy 0, policy_version 921805 (0.0007) [2023-12-27 12:02:01,674][20381] Updated weights for policy 1, policy_version 921086 (0.0007) [2023-12-27 12:02:01,690][20347] Updated weights for policy 0, policy_version 921815 (0.0008) [2023-12-27 12:02:01,731][20381] Updated weights for policy 1, policy_version 921096 (0.0006) [2023-12-27 12:02:01,797][20381] Updated weights for policy 1, policy_version 921106 (0.0007) [2023-12-27 12:02:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 471851008. Throughput: 0: 10536.8, 1: 10169.2. Samples: 471865416. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:02,363][19170] Avg episode reward: [(0, '493.870'), (1, '621.370')] [2023-12-27 12:02:02,433][20347] Updated weights for policy 0, policy_version 921825 (0.0008) [2023-12-27 12:02:02,490][20347] Updated weights for policy 0, policy_version 921835 (0.0008) [2023-12-27 12:02:02,516][20381] Updated weights for policy 1, policy_version 921116 (0.0009) [2023-12-27 12:02:02,543][20347] Updated weights for policy 0, policy_version 921845 (0.0007) [2023-12-27 12:02:02,566][20381] Updated weights for policy 1, policy_version 921126 (0.0007) [2023-12-27 12:02:02,600][20347] Updated weights for policy 0, policy_version 921855 (0.0007) [2023-12-27 12:02:02,623][20381] Updated weights for policy 1, policy_version 921136 (0.0006) [2023-12-27 12:02:03,229][20381] Updated weights for policy 1, policy_version 921146 (0.0005) [2023-12-27 12:02:03,285][20381] Updated weights for policy 1, policy_version 921156 (0.0007) [2023-12-27 12:02:03,308][20347] Updated weights for policy 0, policy_version 921865 (0.0009) [2023-12-27 12:02:03,329][20381] Updated weights for policy 1, policy_version 921166 (0.0010) [2023-12-27 12:02:03,353][20347] Updated weights for policy 0, policy_version 921875 (0.0005) [2023-12-27 12:02:03,380][20381] Updated weights for policy 1, policy_version 921176 (0.0010) [2023-12-27 12:02:03,409][20347] Updated weights for policy 0, policy_version 921885 (0.0007) [2023-12-27 12:02:04,007][20381] Updated weights for policy 1, policy_version 921186 (0.0011) [2023-12-27 12:02:04,066][20381] Updated weights for policy 1, policy_version 921196 (0.0010) [2023-12-27 12:02:04,105][20347] Updated weights for policy 0, policy_version 921895 (0.0009) [2023-12-27 12:02:04,123][20381] Updated weights for policy 1, policy_version 921206 (0.0011) [2023-12-27 12:02:04,164][20347] Updated weights for policy 0, policy_version 921905 (0.0010) [2023-12-27 12:02:04,223][20347] Updated weights for policy 0, policy_version 921915 (0.0010) [2023-12-27 12:02:04,792][20381] Updated weights for policy 1, policy_version 921216 (0.0007) [2023-12-27 12:02:04,858][20381] Updated weights for policy 1, policy_version 921226 (0.0005) [2023-12-27 12:02:04,886][20347] Updated weights for policy 0, policy_version 921925 (0.0008) [2023-12-27 12:02:04,918][20381] Updated weights for policy 1, policy_version 921236 (0.0006) [2023-12-27 12:02:04,935][20347] Updated weights for policy 0, policy_version 921935 (0.0009) [2023-12-27 12:02:04,986][20347] Updated weights for policy 0, policy_version 921945 (0.0007) [2023-12-27 12:02:05,548][20381] Updated weights for policy 1, policy_version 921246 (0.0008) [2023-12-27 12:02:05,605][20381] Updated weights for policy 1, policy_version 921256 (0.0010) [2023-12-27 12:02:05,660][20381] Updated weights for policy 1, policy_version 921266 (0.0010) [2023-12-27 12:02:05,670][20347] Updated weights for policy 0, policy_version 921955 (0.0007) [2023-12-27 12:02:05,717][20347] Updated weights for policy 0, policy_version 921965 (0.0008) [2023-12-27 12:02:05,764][20347] Updated weights for policy 0, policy_version 921975 (0.0007) [2023-12-27 12:02:06,338][20381] Updated weights for policy 1, policy_version 921276 (0.0010) [2023-12-27 12:02:06,397][20381] Updated weights for policy 1, policy_version 921286 (0.0010) [2023-12-27 12:02:06,458][20381] Updated weights for policy 1, policy_version 921296 (0.0010) [2023-12-27 12:02:06,515][20347] Updated weights for policy 0, policy_version 921985 (0.0007) [2023-12-27 12:02:06,570][20347] Updated weights for policy 0, policy_version 921995 (0.0007) [2023-12-27 12:02:06,633][20347] Updated weights for policy 0, policy_version 922005 (0.0007) [2023-12-27 12:02:06,694][20347] Updated weights for policy 0, policy_version 922015 (0.0005) [2023-12-27 12:02:07,126][20381] Updated weights for policy 1, policy_version 921306 (0.0010) [2023-12-27 12:02:07,176][20381] Updated weights for policy 1, policy_version 921316 (0.0008) [2023-12-27 12:02:07,229][20381] Updated weights for policy 1, policy_version 921326 (0.0008) [2023-12-27 12:02:07,284][20381] Updated weights for policy 1, policy_version 921336 (0.0008) [2023-12-27 12:02:07,311][20347] Updated weights for policy 0, policy_version 922025 (0.0007) [2023-12-27 12:02:07,360][20347] Updated weights for policy 0, policy_version 922035 (0.0007) [2023-12-27 12:02:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 471957504. Throughput: 0: 10466.9, 1: 10240.0. Samples: 471927876. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:07,363][19170] Avg episode reward: [(0, '496.130'), (1, '730.530')] [2023-12-27 12:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000921336_235888640.pth... [2023-12-27 12:02:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000920152_235585536.pth [2023-12-27 12:02:07,408][20347] Updated weights for policy 0, policy_version 922045 (0.0009) [2023-12-27 12:02:07,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000922048_236077056.pth... [2023-12-27 12:02:07,423][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000920832_235765760.pth [2023-12-27 12:02:07,914][20381] Updated weights for policy 1, policy_version 921346 (0.0008) [2023-12-27 12:02:07,963][20381] Updated weights for policy 1, policy_version 921356 (0.0008) [2023-12-27 12:02:08,023][20381] Updated weights for policy 1, policy_version 921366 (0.0007) [2023-12-27 12:02:08,157][20347] Updated weights for policy 0, policy_version 922055 (0.0006) [2023-12-27 12:02:08,211][20347] Updated weights for policy 0, policy_version 922065 (0.0005) [2023-12-27 12:02:08,268][20347] Updated weights for policy 0, policy_version 922075 (0.0005) [2023-12-27 12:02:08,709][20381] Updated weights for policy 1, policy_version 921376 (0.0007) [2023-12-27 12:02:08,772][20381] Updated weights for policy 1, policy_version 921386 (0.0008) [2023-12-27 12:02:08,831][20381] Updated weights for policy 1, policy_version 921396 (0.0008) [2023-12-27 12:02:08,847][20347] Updated weights for policy 0, policy_version 922085 (0.0006) [2023-12-27 12:02:08,899][20347] Updated weights for policy 0, policy_version 922095 (0.0006) [2023-12-27 12:02:08,954][20347] Updated weights for policy 0, policy_version 922105 (0.0006) [2023-12-27 12:02:09,542][20381] Updated weights for policy 1, policy_version 921406 (0.0007) [2023-12-27 12:02:09,605][20381] Updated weights for policy 1, policy_version 921416 (0.0005) [2023-12-27 12:02:09,670][20381] Updated weights for policy 1, policy_version 921426 (0.0008) [2023-12-27 12:02:09,693][20347] Updated weights for policy 0, policy_version 922115 (0.0008) [2023-12-27 12:02:09,742][20347] Updated weights for policy 0, policy_version 922125 (0.0007) [2023-12-27 12:02:09,799][20347] Updated weights for policy 0, policy_version 922135 (0.0007) [2023-12-27 12:02:10,310][20381] Updated weights for policy 1, policy_version 921436 (0.0006) [2023-12-27 12:02:10,372][20381] Updated weights for policy 1, policy_version 921446 (0.0006) [2023-12-27 12:02:10,436][20381] Updated weights for policy 1, policy_version 921456 (0.0008) [2023-12-27 12:02:10,530][20347] Updated weights for policy 0, policy_version 922145 (0.0008) [2023-12-27 12:02:10,577][20347] Updated weights for policy 0, policy_version 922155 (0.0005) [2023-12-27 12:02:10,625][20347] Updated weights for policy 0, policy_version 922165 (0.0008) [2023-12-27 12:02:10,676][20347] Updated weights for policy 0, policy_version 922175 (0.0008) [2023-12-27 12:02:11,021][20381] Updated weights for policy 1, policy_version 921466 (0.0007) [2023-12-27 12:02:11,084][20381] Updated weights for policy 1, policy_version 921476 (0.0008) [2023-12-27 12:02:11,149][20381] Updated weights for policy 1, policy_version 921486 (0.0008) [2023-12-27 12:02:11,209][20381] Updated weights for policy 1, policy_version 921496 (0.0007) [2023-12-27 12:02:11,402][20347] Updated weights for policy 0, policy_version 922185 (0.0009) [2023-12-27 12:02:11,465][20347] Updated weights for policy 0, policy_version 922195 (0.0006) [2023-12-27 12:02:11,530][20347] Updated weights for policy 0, policy_version 922205 (0.0008) [2023-12-27 12:02:12,012][20381] Updated weights for policy 1, policy_version 921506 (0.0010) [2023-12-27 12:02:12,069][20381] Updated weights for policy 1, policy_version 921516 (0.0010) [2023-12-27 12:02:12,126][20381] Updated weights for policy 1, policy_version 921526 (0.0008) [2023-12-27 12:02:12,285][20347] Updated weights for policy 0, policy_version 922215 (0.0008) [2023-12-27 12:02:12,360][20347] Updated weights for policy 0, policy_version 922225 (0.0008) [2023-12-27 12:02:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 472055808. Throughput: 0: 10386.4, 1: 10294.8. Samples: 472050396. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:12,363][19170] Avg episode reward: [(0, '464.540'), (1, '663.540')] [2023-12-27 12:02:12,423][20347] Updated weights for policy 0, policy_version 922235 (0.0007) [2023-12-27 12:02:12,792][20381] Updated weights for policy 1, policy_version 921536 (0.0007) [2023-12-27 12:02:12,848][20381] Updated weights for policy 1, policy_version 921546 (0.0008) [2023-12-27 12:02:12,914][20381] Updated weights for policy 1, policy_version 921556 (0.0008) [2023-12-27 12:02:13,112][20347] Updated weights for policy 0, policy_version 922245 (0.0007) [2023-12-27 12:02:13,176][20347] Updated weights for policy 0, policy_version 922255 (0.0007) [2023-12-27 12:02:13,231][20347] Updated weights for policy 0, policy_version 922265 (0.0008) [2023-12-27 12:02:13,530][20381] Updated weights for policy 1, policy_version 921566 (0.0007) [2023-12-27 12:02:13,581][20381] Updated weights for policy 1, policy_version 921576 (0.0006) [2023-12-27 12:02:13,635][20381] Updated weights for policy 1, policy_version 921586 (0.0007) [2023-12-27 12:02:13,928][20347] Updated weights for policy 0, policy_version 922275 (0.0007) [2023-12-27 12:02:13,973][20347] Updated weights for policy 0, policy_version 922285 (0.0007) [2023-12-27 12:02:14,030][20347] Updated weights for policy 0, policy_version 922295 (0.0007) [2023-12-27 12:02:14,231][20381] Updated weights for policy 1, policy_version 921596 (0.0007) [2023-12-27 12:02:14,277][20381] Updated weights for policy 1, policy_version 921606 (0.0008) [2023-12-27 12:02:14,323][20381] Updated weights for policy 1, policy_version 921616 (0.0007) [2023-12-27 12:02:14,723][20347] Updated weights for policy 0, policy_version 922305 (0.0008) [2023-12-27 12:02:14,777][20347] Updated weights for policy 0, policy_version 922315 (0.0008) [2023-12-27 12:02:14,843][20347] Updated weights for policy 0, policy_version 922325 (0.0008) [2023-12-27 12:02:14,907][20347] Updated weights for policy 0, policy_version 922335 (0.0011) [2023-12-27 12:02:15,075][20381] Updated weights for policy 1, policy_version 921626 (0.0008) [2023-12-27 12:02:15,139][20381] Updated weights for policy 1, policy_version 921636 (0.0009) [2023-12-27 12:02:15,205][20381] Updated weights for policy 1, policy_version 921646 (0.0008) [2023-12-27 12:02:15,267][20381] Updated weights for policy 1, policy_version 921656 (0.0008) [2023-12-27 12:02:15,554][20347] Updated weights for policy 0, policy_version 922345 (0.0006) [2023-12-27 12:02:15,614][20347] Updated weights for policy 0, policy_version 922355 (0.0005) [2023-12-27 12:02:15,663][20347] Updated weights for policy 0, policy_version 922365 (0.0005) [2023-12-27 12:02:15,912][20381] Updated weights for policy 1, policy_version 921666 (0.0005) [2023-12-27 12:02:15,975][20381] Updated weights for policy 1, policy_version 921676 (0.0005) [2023-12-27 12:02:16,031][20381] Updated weights for policy 1, policy_version 921686 (0.0007) [2023-12-27 12:02:16,234][20347] Updated weights for policy 0, policy_version 922375 (0.0009) [2023-12-27 12:02:16,286][20347] Updated weights for policy 0, policy_version 922385 (0.0010) [2023-12-27 12:02:16,348][20347] Updated weights for policy 0, policy_version 922395 (0.0010) [2023-12-27 12:02:16,674][20381] Updated weights for policy 1, policy_version 921696 (0.0008) [2023-12-27 12:02:16,732][20381] Updated weights for policy 1, policy_version 921706 (0.0008) [2023-12-27 12:02:16,781][20381] Updated weights for policy 1, policy_version 921716 (0.0008) [2023-12-27 12:02:17,004][20347] Updated weights for policy 0, policy_version 922405 (0.0008) [2023-12-27 12:02:17,060][20347] Updated weights for policy 0, policy_version 922415 (0.0005) [2023-12-27 12:02:17,106][20347] Updated weights for policy 0, policy_version 922425 (0.0006) [2023-12-27 12:02:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 472162304. Throughput: 0: 10252.2, 1: 10214.5. Samples: 472172932. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:17,364][19170] Avg episode reward: [(0, '134.830'), (1, '608.250')] [2023-12-27 12:02:17,503][20381] Updated weights for policy 1, policy_version 921726 (0.0007) [2023-12-27 12:02:17,576][20381] Updated weights for policy 1, policy_version 921736 (0.0008) [2023-12-27 12:02:17,637][20381] Updated weights for policy 1, policy_version 921746 (0.0007) [2023-12-27 12:02:17,741][20347] Updated weights for policy 0, policy_version 922435 (0.0007) [2023-12-27 12:02:17,799][20347] Updated weights for policy 0, policy_version 922445 (0.0005) [2023-12-27 12:02:17,857][20347] Updated weights for policy 0, policy_version 922455 (0.0006) [2023-12-27 12:02:18,302][20381] Updated weights for policy 1, policy_version 921756 (0.0007) [2023-12-27 12:02:18,363][20381] Updated weights for policy 1, policy_version 921766 (0.0007) [2023-12-27 12:02:18,427][20381] Updated weights for policy 1, policy_version 921776 (0.0008) [2023-12-27 12:02:18,550][20347] Updated weights for policy 0, policy_version 922465 (0.0008) [2023-12-27 12:02:18,618][20347] Updated weights for policy 0, policy_version 922475 (0.0011) [2023-12-27 12:02:18,681][20347] Updated weights for policy 0, policy_version 922485 (0.0010) [2023-12-27 12:02:18,741][20347] Updated weights for policy 0, policy_version 922495 (0.0011) [2023-12-27 12:02:19,144][20381] Updated weights for policy 1, policy_version 921786 (0.0007) [2023-12-27 12:02:19,190][20381] Updated weights for policy 1, policy_version 921796 (0.0005) [2023-12-27 12:02:19,250][20381] Updated weights for policy 1, policy_version 921806 (0.0007) [2023-12-27 12:02:19,304][20381] Updated weights for policy 1, policy_version 921816 (0.0009) [2023-12-27 12:02:19,408][20347] Updated weights for policy 0, policy_version 922505 (0.0010) [2023-12-27 12:02:19,458][20347] Updated weights for policy 0, policy_version 922515 (0.0010) [2023-12-27 12:02:19,521][20347] Updated weights for policy 0, policy_version 922525 (0.0008) [2023-12-27 12:02:20,083][20381] Updated weights for policy 1, policy_version 921826 (0.0006) [2023-12-27 12:02:20,146][20381] Updated weights for policy 1, policy_version 921836 (0.0008) [2023-12-27 12:02:20,214][20381] Updated weights for policy 1, policy_version 921846 (0.0006) [2023-12-27 12:02:20,245][20347] Updated weights for policy 0, policy_version 922535 (0.0007) [2023-12-27 12:02:20,305][20347] Updated weights for policy 0, policy_version 922545 (0.0008) [2023-12-27 12:02:20,364][20347] Updated weights for policy 0, policy_version 922555 (0.0008) [2023-12-27 12:02:20,827][20381] Updated weights for policy 1, policy_version 921856 (0.0008) [2023-12-27 12:02:20,886][20381] Updated weights for policy 1, policy_version 921866 (0.0008) [2023-12-27 12:02:20,948][20381] Updated weights for policy 1, policy_version 921876 (0.0008) [2023-12-27 12:02:21,157][20347] Updated weights for policy 0, policy_version 922565 (0.0008) [2023-12-27 12:02:21,225][20347] Updated weights for policy 0, policy_version 922575 (0.0007) [2023-12-27 12:02:21,293][20347] Updated weights for policy 0, policy_version 922585 (0.0008) [2023-12-27 12:02:21,665][20381] Updated weights for policy 1, policy_version 921886 (0.0006) [2023-12-27 12:02:21,733][20381] Updated weights for policy 1, policy_version 921896 (0.0008) [2023-12-27 12:02:21,803][20381] Updated weights for policy 1, policy_version 921906 (0.0007) [2023-12-27 12:02:21,976][20347] Updated weights for policy 0, policy_version 922595 (0.0007) [2023-12-27 12:02:22,034][20347] Updated weights for policy 0, policy_version 922605 (0.0007) [2023-12-27 12:02:22,095][20347] Updated weights for policy 0, policy_version 922615 (0.0008) [2023-12-27 12:02:22,364][19170] Fps is (10 sec: 20476.9, 60 sec: 20616.0, 300 sec: 20493.8). Total num frames: 472260608. Throughput: 0: 10244.4, 1: 10196.7. Samples: 472232732. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:22,365][19170] Avg episode reward: [(0, '115.250'), (1, '584.760')] [2023-12-27 12:02:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000922624_236224512.pth... [2023-12-27 12:02:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000921440_235921408.pth [2023-12-27 12:02:22,387][20381] Updated weights for policy 1, policy_version 921916 (0.0007) [2023-12-27 12:02:22,442][20381] Updated weights for policy 1, policy_version 921926 (0.0008) [2023-12-27 12:02:22,490][20381] Updated weights for policy 1, policy_version 921936 (0.0006) [2023-12-27 12:02:22,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000921944_236044288.pth... [2023-12-27 12:02:22,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000920728_235732992.pth [2023-12-27 12:02:22,835][20347] Updated weights for policy 0, policy_version 922625 (0.0008) [2023-12-27 12:02:22,890][20347] Updated weights for policy 0, policy_version 922635 (0.0005) [2023-12-27 12:02:22,943][20347] Updated weights for policy 0, policy_version 922645 (0.0005) [2023-12-27 12:02:23,000][20347] Updated weights for policy 0, policy_version 922655 (0.0005) [2023-12-27 12:02:23,105][20381] Updated weights for policy 1, policy_version 921946 (0.0006) [2023-12-27 12:02:23,151][20381] Updated weights for policy 1, policy_version 921956 (0.0007) [2023-12-27 12:02:23,201][20381] Updated weights for policy 1, policy_version 921966 (0.0007) [2023-12-27 12:02:23,248][20381] Updated weights for policy 1, policy_version 921976 (0.0008) [2023-12-27 12:02:23,580][20347] Updated weights for policy 0, policy_version 922665 (0.0006) [2023-12-27 12:02:23,625][20347] Updated weights for policy 0, policy_version 922675 (0.0006) [2023-12-27 12:02:23,669][20347] Updated weights for policy 0, policy_version 922685 (0.0005) [2023-12-27 12:02:23,969][20381] Updated weights for policy 1, policy_version 921986 (0.0007) [2023-12-27 12:02:24,024][20381] Updated weights for policy 1, policy_version 921996 (0.0005) [2023-12-27 12:02:24,070][20381] Updated weights for policy 1, policy_version 922006 (0.0006) [2023-12-27 12:02:24,312][20347] Updated weights for policy 0, policy_version 922695 (0.0009) [2023-12-27 12:02:24,359][20347] Updated weights for policy 0, policy_version 922705 (0.0009) [2023-12-27 12:02:24,420][20347] Updated weights for policy 0, policy_version 922715 (0.0010) [2023-12-27 12:02:24,758][20381] Updated weights for policy 1, policy_version 922016 (0.0007) [2023-12-27 12:02:24,820][20381] Updated weights for policy 1, policy_version 922026 (0.0008) [2023-12-27 12:02:24,887][20381] Updated weights for policy 1, policy_version 922036 (0.0007) [2023-12-27 12:02:25,116][20347] Updated weights for policy 0, policy_version 922725 (0.0010) [2023-12-27 12:02:25,171][20347] Updated weights for policy 0, policy_version 922735 (0.0010) [2023-12-27 12:02:25,224][20347] Updated weights for policy 0, policy_version 922745 (0.0010) [2023-12-27 12:02:25,548][20381] Updated weights for policy 1, policy_version 922046 (0.0008) [2023-12-27 12:02:25,605][20381] Updated weights for policy 1, policy_version 922056 (0.0008) [2023-12-27 12:02:25,662][20381] Updated weights for policy 1, policy_version 922066 (0.0008) [2023-12-27 12:02:25,804][20347] Updated weights for policy 0, policy_version 922755 (0.0007) [2023-12-27 12:02:25,862][20347] Updated weights for policy 0, policy_version 922765 (0.0005) [2023-12-27 12:02:25,920][20347] Updated weights for policy 0, policy_version 922775 (0.0005) [2023-12-27 12:02:26,348][20381] Updated weights for policy 1, policy_version 922076 (0.0008) [2023-12-27 12:02:26,411][20381] Updated weights for policy 1, policy_version 922086 (0.0008) [2023-12-27 12:02:26,475][20381] Updated weights for policy 1, policy_version 922096 (0.0008) [2023-12-27 12:02:26,515][20347] Updated weights for policy 0, policy_version 922785 (0.0010) [2023-12-27 12:02:26,571][20347] Updated weights for policy 0, policy_version 922795 (0.0005) [2023-12-27 12:02:26,629][20347] Updated weights for policy 0, policy_version 922805 (0.0005) [2023-12-27 12:02:26,686][20347] Updated weights for policy 0, policy_version 922815 (0.0006) [2023-12-27 12:02:27,202][20347] Updated weights for policy 0, policy_version 922825 (0.0005) [2023-12-27 12:02:27,206][20381] Updated weights for policy 1, policy_version 922106 (0.0008) [2023-12-27 12:02:27,251][20347] Updated weights for policy 0, policy_version 922835 (0.0008) [2023-12-27 12:02:27,259][20381] Updated weights for policy 1, policy_version 922116 (0.0008) [2023-12-27 12:02:27,300][20347] Updated weights for policy 0, policy_version 922845 (0.0010) [2023-12-27 12:02:27,310][20381] Updated weights for policy 1, policy_version 922126 (0.0006) [2023-12-27 12:02:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 472367104. Throughput: 0: 10221.0, 1: 10246.4. Samples: 472356340. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:27,363][19170] Avg episode reward: [(0, '129.510'), (1, '601.030')] [2023-12-27 12:02:27,373][20381] Updated weights for policy 1, policy_version 922136 (0.0007) [2023-12-27 12:02:27,848][20347] Updated weights for policy 0, policy_version 922855 (0.0007) [2023-12-27 12:02:27,907][20347] Updated weights for policy 0, policy_version 922865 (0.0005) [2023-12-27 12:02:27,956][20347] Updated weights for policy 0, policy_version 922875 (0.0006) [2023-12-27 12:02:28,109][20381] Updated weights for policy 1, policy_version 922146 (0.0009) [2023-12-27 12:02:28,165][20381] Updated weights for policy 1, policy_version 922156 (0.0008) [2023-12-27 12:02:28,225][20381] Updated weights for policy 1, policy_version 922166 (0.0008) [2023-12-27 12:02:28,492][20347] Updated weights for policy 0, policy_version 922885 (0.0005) [2023-12-27 12:02:28,554][20347] Updated weights for policy 0, policy_version 922895 (0.0009) [2023-12-27 12:02:28,617][20347] Updated weights for policy 0, policy_version 922905 (0.0010) [2023-12-27 12:02:28,955][20381] Updated weights for policy 1, policy_version 922176 (0.0006) [2023-12-27 12:02:29,013][20381] Updated weights for policy 1, policy_version 922186 (0.0007) [2023-12-27 12:02:29,063][20381] Updated weights for policy 1, policy_version 922196 (0.0007) [2023-12-27 12:02:29,232][20347] Updated weights for policy 0, policy_version 922915 (0.0010) [2023-12-27 12:02:29,294][20347] Updated weights for policy 0, policy_version 922925 (0.0007) [2023-12-27 12:02:29,353][20347] Updated weights for policy 0, policy_version 922935 (0.0008) [2023-12-27 12:02:29,822][20381] Updated weights for policy 1, policy_version 922206 (0.0007) [2023-12-27 12:02:29,887][20381] Updated weights for policy 1, policy_version 922216 (0.0009) [2023-12-27 12:02:29,946][20381] Updated weights for policy 1, policy_version 922226 (0.0009) [2023-12-27 12:02:30,066][20347] Updated weights for policy 0, policy_version 922945 (0.0008) [2023-12-27 12:02:30,125][20347] Updated weights for policy 0, policy_version 922955 (0.0006) [2023-12-27 12:02:30,180][20347] Updated weights for policy 0, policy_version 922965 (0.0008) [2023-12-27 12:02:30,232][20347] Updated weights for policy 0, policy_version 922975 (0.0005) [2023-12-27 12:02:30,702][20381] Updated weights for policy 1, policy_version 922236 (0.0008) [2023-12-27 12:02:30,750][20381] Updated weights for policy 1, policy_version 922246 (0.0007) [2023-12-27 12:02:30,801][20381] Updated weights for policy 1, policy_version 922256 (0.0007) [2023-12-27 12:02:30,848][20347] Updated weights for policy 0, policy_version 922985 (0.0010) [2023-12-27 12:02:30,900][20347] Updated weights for policy 0, policy_version 922995 (0.0007) [2023-12-27 12:02:30,954][20347] Updated weights for policy 0, policy_version 923005 (0.0010) [2023-12-27 12:02:31,509][20381] Updated weights for policy 1, policy_version 922266 (0.0006) [2023-12-27 12:02:31,563][20381] Updated weights for policy 1, policy_version 922276 (0.0006) [2023-12-27 12:02:31,618][20381] Updated weights for policy 1, policy_version 922286 (0.0010) [2023-12-27 12:02:31,663][20347] Updated weights for policy 0, policy_version 923016 (0.0009) [2023-12-27 12:02:31,682][20381] Updated weights for policy 1, policy_version 922296 (0.0008) [2023-12-27 12:02:31,725][20347] Updated weights for policy 0, policy_version 923026 (0.0006) [2023-12-27 12:02:31,790][20347] Updated weights for policy 0, policy_version 923036 (0.0008) [2023-12-27 12:02:32,295][20381] Updated weights for policy 1, policy_version 922306 (0.0008) [2023-12-27 12:02:32,352][20381] Updated weights for policy 1, policy_version 922316 (0.0007) [2023-12-27 12:02:32,362][19170] Fps is (10 sec: 20483.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 472465408. Throughput: 0: 10269.4, 1: 10236.5. Samples: 472482304. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:32,363][19170] Avg episode reward: [(0, '40.720'), (1, '654.990')] [2023-12-27 12:02:32,373][20347] Updated weights for policy 0, policy_version 923046 (0.0006) [2023-12-27 12:02:32,409][20381] Updated weights for policy 1, policy_version 922326 (0.0006) [2023-12-27 12:02:32,419][20347] Updated weights for policy 0, policy_version 923056 (0.0005) [2023-12-27 12:02:32,462][20347] Updated weights for policy 0, policy_version 923066 (0.0007) [2023-12-27 12:02:33,007][20381] Updated weights for policy 1, policy_version 922336 (0.0005) [2023-12-27 12:02:33,070][20381] Updated weights for policy 1, policy_version 922346 (0.0005) [2023-12-27 12:02:33,123][20381] Updated weights for policy 1, policy_version 922356 (0.0005) [2023-12-27 12:02:33,246][20347] Updated weights for policy 0, policy_version 923076 (0.0006) [2023-12-27 12:02:33,300][20347] Updated weights for policy 0, policy_version 923086 (0.0005) [2023-12-27 12:02:33,351][20347] Updated weights for policy 0, policy_version 923096 (0.0005) [2023-12-27 12:02:33,720][20381] Updated weights for policy 1, policy_version 922366 (0.0007) [2023-12-27 12:02:33,777][20381] Updated weights for policy 1, policy_version 922376 (0.0008) [2023-12-27 12:02:33,840][20381] Updated weights for policy 1, policy_version 922386 (0.0007) [2023-12-27 12:02:33,990][20347] Updated weights for policy 0, policy_version 923106 (0.0006) [2023-12-27 12:02:34,056][20347] Updated weights for policy 0, policy_version 923116 (0.0011) [2023-12-27 12:02:34,110][20347] Updated weights for policy 0, policy_version 923126 (0.0010) [2023-12-27 12:02:34,166][20347] Updated weights for policy 0, policy_version 923136 (0.0009) [2023-12-27 12:02:34,542][20381] Updated weights for policy 1, policy_version 922396 (0.0008) [2023-12-27 12:02:34,601][20381] Updated weights for policy 1, policy_version 922406 (0.0008) [2023-12-27 12:02:34,653][20381] Updated weights for policy 1, policy_version 922416 (0.0008) [2023-12-27 12:02:34,889][20347] Updated weights for policy 0, policy_version 923146 (0.0010) [2023-12-27 12:02:34,941][20347] Updated weights for policy 0, policy_version 923156 (0.0010) [2023-12-27 12:02:34,990][20347] Updated weights for policy 0, policy_version 923166 (0.0010) [2023-12-27 12:02:35,345][20381] Updated weights for policy 1, policy_version 922426 (0.0008) [2023-12-27 12:02:35,402][20381] Updated weights for policy 1, policy_version 922436 (0.0010) [2023-12-27 12:02:35,464][20381] Updated weights for policy 1, policy_version 922446 (0.0010) [2023-12-27 12:02:35,519][20381] Updated weights for policy 1, policy_version 922456 (0.0010) [2023-12-27 12:02:35,702][20347] Updated weights for policy 0, policy_version 923176 (0.0010) [2023-12-27 12:02:35,757][20347] Updated weights for policy 0, policy_version 923186 (0.0010) [2023-12-27 12:02:35,802][20347] Updated weights for policy 0, policy_version 923196 (0.0010) [2023-12-27 12:02:36,148][20381] Updated weights for policy 1, policy_version 922466 (0.0007) [2023-12-27 12:02:36,204][20381] Updated weights for policy 1, policy_version 922476 (0.0008) [2023-12-27 12:02:36,253][20381] Updated weights for policy 1, policy_version 922486 (0.0008) [2023-12-27 12:02:36,460][20347] Updated weights for policy 0, policy_version 923206 (0.0009) [2023-12-27 12:02:36,524][20347] Updated weights for policy 0, policy_version 923216 (0.0009) [2023-12-27 12:02:36,588][20347] Updated weights for policy 0, policy_version 923226 (0.0010) [2023-12-27 12:02:36,953][20381] Updated weights for policy 1, policy_version 922496 (0.0008) [2023-12-27 12:02:37,005][20381] Updated weights for policy 1, policy_version 922506 (0.0007) [2023-12-27 12:02:37,057][20381] Updated weights for policy 1, policy_version 922516 (0.0008) [2023-12-27 12:02:37,207][20347] Updated weights for policy 0, policy_version 923236 (0.0009) [2023-12-27 12:02:37,256][20347] Updated weights for policy 0, policy_version 923246 (0.0010) [2023-12-27 12:02:37,312][20347] Updated weights for policy 0, policy_version 923256 (0.0010) [2023-12-27 12:02:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 472580096. Throughput: 0: 10258.4, 1: 10255.1. Samples: 472544280. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:37,363][19170] Avg episode reward: [(0, '42.810'), (1, '586.800')] [2023-12-27 12:02:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000922520_236191744.pth... [2023-12-27 12:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000923264_236388352.pth... [2023-12-27 12:02:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000921336_235888640.pth [2023-12-27 12:02:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000922048_236077056.pth [2023-12-27 12:02:37,714][20381] Updated weights for policy 1, policy_version 922526 (0.0007) [2023-12-27 12:02:37,772][20381] Updated weights for policy 1, policy_version 922536 (0.0008) [2023-12-27 12:02:37,832][20381] Updated weights for policy 1, policy_version 922546 (0.0006) [2023-12-27 12:02:37,971][20347] Updated weights for policy 0, policy_version 923266 (0.0009) [2023-12-27 12:02:38,025][20347] Updated weights for policy 0, policy_version 923276 (0.0005) [2023-12-27 12:02:38,077][20347] Updated weights for policy 0, policy_version 923286 (0.0005) [2023-12-27 12:02:38,133][20347] Updated weights for policy 0, policy_version 923296 (0.0006) [2023-12-27 12:02:38,537][20381] Updated weights for policy 1, policy_version 922556 (0.0007) [2023-12-27 12:02:38,597][20381] Updated weights for policy 1, policy_version 922566 (0.0008) [2023-12-27 12:02:38,654][20381] Updated weights for policy 1, policy_version 922576 (0.0008) [2023-12-27 12:02:38,829][20347] Updated weights for policy 0, policy_version 923306 (0.0005) [2023-12-27 12:02:38,875][20347] Updated weights for policy 0, policy_version 923316 (0.0006) [2023-12-27 12:02:38,920][20347] Updated weights for policy 0, policy_version 923326 (0.0005) [2023-12-27 12:02:39,372][20381] Updated weights for policy 1, policy_version 922586 (0.0008) [2023-12-27 12:02:39,442][20381] Updated weights for policy 1, policy_version 922596 (0.0009) [2023-12-27 12:02:39,505][20381] Updated weights for policy 1, policy_version 922606 (0.0011) [2023-12-27 12:02:39,559][20381] Updated weights for policy 1, policy_version 922616 (0.0010) [2023-12-27 12:02:39,636][20347] Updated weights for policy 0, policy_version 923336 (0.0008) [2023-12-27 12:02:39,705][20347] Updated weights for policy 0, policy_version 923346 (0.0008) [2023-12-27 12:02:39,770][20347] Updated weights for policy 0, policy_version 923356 (0.0008) [2023-12-27 12:02:40,322][20381] Updated weights for policy 1, policy_version 922626 (0.0008) [2023-12-27 12:02:40,377][20381] Updated weights for policy 1, policy_version 922636 (0.0008) [2023-12-27 12:02:40,431][20347] Updated weights for policy 0, policy_version 923366 (0.0006) [2023-12-27 12:02:40,437][20381] Updated weights for policy 1, policy_version 922646 (0.0011) [2023-12-27 12:02:40,489][20347] Updated weights for policy 0, policy_version 923376 (0.0007) [2023-12-27 12:02:40,548][20347] Updated weights for policy 0, policy_version 923386 (0.0007) [2023-12-27 12:02:41,147][20381] Updated weights for policy 1, policy_version 922656 (0.0009) [2023-12-27 12:02:41,206][20381] Updated weights for policy 1, policy_version 922666 (0.0008) [2023-12-27 12:02:41,227][20347] Updated weights for policy 0, policy_version 923396 (0.0006) [2023-12-27 12:02:41,277][20381] Updated weights for policy 1, policy_version 922676 (0.0008) [2023-12-27 12:02:41,283][20347] Updated weights for policy 0, policy_version 923406 (0.0008) [2023-12-27 12:02:41,344][20347] Updated weights for policy 0, policy_version 923416 (0.0007) [2023-12-27 12:02:42,024][20381] Updated weights for policy 1, policy_version 922686 (0.0008) [2023-12-27 12:02:42,079][20381] Updated weights for policy 1, policy_version 922696 (0.0008) [2023-12-27 12:02:42,097][20347] Updated weights for policy 0, policy_version 923426 (0.0008) [2023-12-27 12:02:42,134][20381] Updated weights for policy 1, policy_version 922706 (0.0006) [2023-12-27 12:02:42,155][20347] Updated weights for policy 0, policy_version 923436 (0.0005) [2023-12-27 12:02:42,214][20347] Updated weights for policy 0, policy_version 923446 (0.0006) [2023-12-27 12:02:42,274][20347] Updated weights for policy 0, policy_version 923456 (0.0008) [2023-12-27 12:02:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 472678400. Throughput: 0: 10221.4, 1: 10205.4. Samples: 472665600. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:42,363][19170] Avg episode reward: [(0, '45.120'), (1, '415.830')] [2023-12-27 12:02:42,793][20381] Updated weights for policy 1, policy_version 922716 (0.0007) [2023-12-27 12:02:42,847][20381] Updated weights for policy 1, policy_version 922726 (0.0007) [2023-12-27 12:02:42,898][20381] Updated weights for policy 1, policy_version 922736 (0.0008) [2023-12-27 12:02:42,993][20347] Updated weights for policy 0, policy_version 923466 (0.0005) [2023-12-27 12:02:43,051][20347] Updated weights for policy 0, policy_version 923476 (0.0005) [2023-12-27 12:02:43,106][20347] Updated weights for policy 0, policy_version 923486 (0.0010) [2023-12-27 12:02:43,606][20381] Updated weights for policy 1, policy_version 922746 (0.0009) [2023-12-27 12:02:43,662][20381] Updated weights for policy 1, policy_version 922756 (0.0008) [2023-12-27 12:02:43,707][20381] Updated weights for policy 1, policy_version 922766 (0.0008) [2023-12-27 12:02:43,747][20347] Updated weights for policy 0, policy_version 923496 (0.0007) [2023-12-27 12:02:43,763][20381] Updated weights for policy 1, policy_version 922776 (0.0008) [2023-12-27 12:02:43,808][20347] Updated weights for policy 0, policy_version 923506 (0.0007) [2023-12-27 12:02:43,869][20347] Updated weights for policy 0, policy_version 923516 (0.0010) [2023-12-27 12:02:44,470][20347] Updated weights for policy 0, policy_version 923526 (0.0007) [2023-12-27 12:02:44,499][20381] Updated weights for policy 1, policy_version 922786 (0.0006) [2023-12-27 12:02:44,524][20347] Updated weights for policy 0, policy_version 923536 (0.0006) [2023-12-27 12:02:44,551][20381] Updated weights for policy 1, policy_version 922796 (0.0006) [2023-12-27 12:02:44,576][20347] Updated weights for policy 0, policy_version 923546 (0.0005) [2023-12-27 12:02:44,608][20381] Updated weights for policy 1, policy_version 922806 (0.0006) [2023-12-27 12:02:45,234][20347] Updated weights for policy 0, policy_version 923556 (0.0007) [2023-12-27 12:02:45,296][20347] Updated weights for policy 0, policy_version 923566 (0.0007) [2023-12-27 12:02:45,343][20381] Updated weights for policy 1, policy_version 922816 (0.0007) [2023-12-27 12:02:45,359][20347] Updated weights for policy 0, policy_version 923576 (0.0007) [2023-12-27 12:02:45,399][20381] Updated weights for policy 1, policy_version 922826 (0.0007) [2023-12-27 12:02:45,453][20381] Updated weights for policy 1, policy_version 922836 (0.0009) [2023-12-27 12:02:46,013][20347] Updated weights for policy 0, policy_version 923586 (0.0007) [2023-12-27 12:02:46,051][20381] Updated weights for policy 1, policy_version 922846 (0.0007) [2023-12-27 12:02:46,075][20347] Updated weights for policy 0, policy_version 923596 (0.0008) [2023-12-27 12:02:46,098][20381] Updated weights for policy 1, policy_version 922856 (0.0005) [2023-12-27 12:02:46,142][20347] Updated weights for policy 0, policy_version 923606 (0.0008) [2023-12-27 12:02:46,152][20381] Updated weights for policy 1, policy_version 922866 (0.0005) [2023-12-27 12:02:46,203][20347] Updated weights for policy 0, policy_version 923616 (0.0007) [2023-12-27 12:02:46,721][20381] Updated weights for policy 1, policy_version 922876 (0.0005) [2023-12-27 12:02:46,780][20381] Updated weights for policy 1, policy_version 922886 (0.0009) [2023-12-27 12:02:46,829][20347] Updated weights for policy 0, policy_version 923626 (0.0009) [2023-12-27 12:02:46,836][20381] Updated weights for policy 1, policy_version 922896 (0.0011) [2023-12-27 12:02:46,880][20347] Updated weights for policy 0, policy_version 923636 (0.0008) [2023-12-27 12:02:46,930][20347] Updated weights for policy 0, policy_version 923646 (0.0007) [2023-12-27 12:02:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 472776704. Throughput: 0: 10336.8, 1: 10202.2. Samples: 472789672. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:47,363][19170] Avg episode reward: [(0, '96.590'), (1, '510.750')] [2023-12-27 12:02:47,516][20381] Updated weights for policy 1, policy_version 922906 (0.0010) [2023-12-27 12:02:47,568][20381] Updated weights for policy 1, policy_version 922916 (0.0006) [2023-12-27 12:02:47,573][20347] Updated weights for policy 0, policy_version 923656 (0.0006) [2023-12-27 12:02:47,616][20381] Updated weights for policy 1, policy_version 922926 (0.0007) [2023-12-27 12:02:47,624][20347] Updated weights for policy 0, policy_version 923666 (0.0009) [2023-12-27 12:02:47,666][20381] Updated weights for policy 1, policy_version 922936 (0.0010) [2023-12-27 12:02:47,672][20347] Updated weights for policy 0, policy_version 923676 (0.0007) [2023-12-27 12:02:48,302][20347] Updated weights for policy 0, policy_version 923686 (0.0006) [2023-12-27 12:02:48,371][20347] Updated weights for policy 0, policy_version 923696 (0.0007) [2023-12-27 12:02:48,384][20381] Updated weights for policy 1, policy_version 922951 (0.0007) [2023-12-27 12:02:48,429][20347] Updated weights for policy 0, policy_version 923706 (0.0006) [2023-12-27 12:02:48,443][20381] Updated weights for policy 1, policy_version 922961 (0.0006) [2023-12-27 12:02:49,109][20347] Updated weights for policy 0, policy_version 923716 (0.0006) [2023-12-27 12:02:49,161][20347] Updated weights for policy 0, policy_version 923726 (0.0008) [2023-12-27 12:02:49,190][20381] Updated weights for policy 1, policy_version 922971 (0.0007) [2023-12-27 12:02:49,215][20347] Updated weights for policy 0, policy_version 923736 (0.0008) [2023-12-27 12:02:49,257][20381] Updated weights for policy 1, policy_version 922981 (0.0007) [2023-12-27 12:02:49,318][20381] Updated weights for policy 1, policy_version 922991 (0.0007) [2023-12-27 12:02:49,956][20347] Updated weights for policy 0, policy_version 923746 (0.0008) [2023-12-27 12:02:50,015][20347] Updated weights for policy 0, policy_version 923756 (0.0007) [2023-12-27 12:02:50,043][20381] Updated weights for policy 1, policy_version 923001 (0.0008) [2023-12-27 12:02:50,072][20347] Updated weights for policy 0, policy_version 923766 (0.0008) [2023-12-27 12:02:50,102][20381] Updated weights for policy 1, policy_version 923011 (0.0008) [2023-12-27 12:02:50,128][20347] Updated weights for policy 0, policy_version 923776 (0.0006) [2023-12-27 12:02:50,171][20381] Updated weights for policy 1, policy_version 923021 (0.0008) [2023-12-27 12:02:50,230][20381] Updated weights for policy 1, policy_version 923031 (0.0008) [2023-12-27 12:02:50,808][20347] Updated weights for policy 0, policy_version 923786 (0.0008) [2023-12-27 12:02:50,867][20347] Updated weights for policy 0, policy_version 923796 (0.0008) [2023-12-27 12:02:50,927][20347] Updated weights for policy 0, policy_version 923806 (0.0008) [2023-12-27 12:02:50,946][20381] Updated weights for policy 1, policy_version 923041 (0.0008) [2023-12-27 12:02:51,003][20381] Updated weights for policy 1, policy_version 923051 (0.0008) [2023-12-27 12:02:51,073][20381] Updated weights for policy 1, policy_version 923061 (0.0008) [2023-12-27 12:02:51,591][20347] Updated weights for policy 0, policy_version 923816 (0.0007) [2023-12-27 12:02:51,656][20347] Updated weights for policy 0, policy_version 923826 (0.0008) [2023-12-27 12:02:51,706][20347] Updated weights for policy 0, policy_version 923836 (0.0008) [2023-12-27 12:02:51,805][20381] Updated weights for policy 1, policy_version 923071 (0.0007) [2023-12-27 12:02:51,871][20381] Updated weights for policy 1, policy_version 923081 (0.0007) [2023-12-27 12:02:51,930][20381] Updated weights for policy 1, policy_version 923091 (0.0008) [2023-12-27 12:02:52,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 472875008. Throughput: 0: 10361.4, 1: 10119.9. Samples: 472849536. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:52,363][19170] Avg episode reward: [(0, '148.630'), (1, '510.860')] [2023-12-27 12:02:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000923096_236339200.pth... [2023-12-27 12:02:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000921944_236044288.pth [2023-12-27 12:02:52,407][20347] Updated weights for policy 0, policy_version 923846 (0.0007) [2023-12-27 12:02:52,464][20347] Updated weights for policy 0, policy_version 923856 (0.0007) [2023-12-27 12:02:52,515][20347] Updated weights for policy 0, policy_version 923866 (0.0008) [2023-12-27 12:02:52,545][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000923872_236544000.pth... [2023-12-27 12:02:52,548][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000922624_236224512.pth [2023-12-27 12:02:52,567][20381] Updated weights for policy 1, policy_version 923101 (0.0007) [2023-12-27 12:02:52,631][20381] Updated weights for policy 1, policy_version 923111 (0.0008) [2023-12-27 12:02:52,695][20381] Updated weights for policy 1, policy_version 923121 (0.0008) [2023-12-27 12:02:53,157][20347] Updated weights for policy 0, policy_version 923876 (0.0007) [2023-12-27 12:02:53,211][20347] Updated weights for policy 0, policy_version 923886 (0.0007) [2023-12-27 12:02:53,272][20347] Updated weights for policy 0, policy_version 923896 (0.0008) [2023-12-27 12:02:53,375][20381] Updated weights for policy 1, policy_version 923131 (0.0009) [2023-12-27 12:02:53,424][20381] Updated weights for policy 1, policy_version 923141 (0.0008) [2023-12-27 12:02:53,477][20381] Updated weights for policy 1, policy_version 923151 (0.0007) [2023-12-27 12:02:53,960][20347] Updated weights for policy 0, policy_version 923906 (0.0007) [2023-12-27 12:02:54,019][20347] Updated weights for policy 0, policy_version 923916 (0.0005) [2023-12-27 12:02:54,080][20347] Updated weights for policy 0, policy_version 923926 (0.0007) [2023-12-27 12:02:54,141][20347] Updated weights for policy 0, policy_version 923936 (0.0008) [2023-12-27 12:02:54,157][20381] Updated weights for policy 1, policy_version 923161 (0.0008) [2023-12-27 12:02:54,216][20381] Updated weights for policy 1, policy_version 923171 (0.0008) [2023-12-27 12:02:54,272][20381] Updated weights for policy 1, policy_version 923181 (0.0008) [2023-12-27 12:02:54,317][20381] Updated weights for policy 1, policy_version 923191 (0.0007) [2023-12-27 12:02:54,722][20347] Updated weights for policy 0, policy_version 923946 (0.0005) [2023-12-27 12:02:54,771][20347] Updated weights for policy 0, policy_version 923956 (0.0005) [2023-12-27 12:02:54,829][20347] Updated weights for policy 0, policy_version 923966 (0.0005) [2023-12-27 12:02:55,068][20381] Updated weights for policy 1, policy_version 923201 (0.0008) [2023-12-27 12:02:55,132][20381] Updated weights for policy 1, policy_version 923211 (0.0007) [2023-12-27 12:02:55,184][20381] Updated weights for policy 1, policy_version 923221 (0.0008) [2023-12-27 12:02:55,439][20347] Updated weights for policy 0, policy_version 923976 (0.0007) [2023-12-27 12:02:55,493][20347] Updated weights for policy 0, policy_version 923986 (0.0009) [2023-12-27 12:02:55,547][20347] Updated weights for policy 0, policy_version 923996 (0.0008) [2023-12-27 12:02:55,866][20381] Updated weights for policy 1, policy_version 923231 (0.0007) [2023-12-27 12:02:55,919][20381] Updated weights for policy 1, policy_version 923241 (0.0008) [2023-12-27 12:02:55,970][20381] Updated weights for policy 1, policy_version 923251 (0.0008) [2023-12-27 12:02:56,262][20347] Updated weights for policy 0, policy_version 924006 (0.0007) [2023-12-27 12:02:56,320][20347] Updated weights for policy 0, policy_version 924016 (0.0008) [2023-12-27 12:02:56,377][20347] Updated weights for policy 0, policy_version 924026 (0.0007) [2023-12-27 12:02:56,672][20381] Updated weights for policy 1, policy_version 923261 (0.0007) [2023-12-27 12:02:56,731][20381] Updated weights for policy 1, policy_version 923271 (0.0008) [2023-12-27 12:02:56,786][20381] Updated weights for policy 1, policy_version 923281 (0.0008) [2023-12-27 12:02:57,052][20347] Updated weights for policy 0, policy_version 924036 (0.0007) [2023-12-27 12:02:57,100][20347] Updated weights for policy 0, policy_version 924046 (0.0007) [2023-12-27 12:02:57,143][20347] Updated weights for policy 0, policy_version 924056 (0.0007) [2023-12-27 12:02:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 472981504. Throughput: 0: 10412.9, 1: 10057.0. Samples: 472971540. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:02:57,363][19170] Avg episode reward: [(0, '324.050'), (1, '472.200')] [2023-12-27 12:02:57,500][20381] Updated weights for policy 1, policy_version 923291 (0.0008) [2023-12-27 12:02:57,546][20381] Updated weights for policy 1, policy_version 923301 (0.0009) [2023-12-27 12:02:57,592][20381] Updated weights for policy 1, policy_version 923311 (0.0010) [2023-12-27 12:02:57,756][20347] Updated weights for policy 0, policy_version 924066 (0.0007) [2023-12-27 12:02:57,803][20347] Updated weights for policy 0, policy_version 924076 (0.0007) [2023-12-27 12:02:57,849][20347] Updated weights for policy 0, policy_version 924086 (0.0007) [2023-12-27 12:02:57,894][20347] Updated weights for policy 0, policy_version 924096 (0.0007) [2023-12-27 12:02:58,315][20381] Updated weights for policy 1, policy_version 923321 (0.0009) [2023-12-27 12:02:58,384][20381] Updated weights for policy 1, policy_version 923331 (0.0008) [2023-12-27 12:02:58,446][20381] Updated weights for policy 1, policy_version 923341 (0.0008) [2023-12-27 12:02:58,508][20381] Updated weights for policy 1, policy_version 923351 (0.0008) [2023-12-27 12:02:58,674][20347] Updated weights for policy 0, policy_version 924106 (0.0008) [2023-12-27 12:02:58,733][20347] Updated weights for policy 0, policy_version 924116 (0.0007) [2023-12-27 12:02:58,796][20347] Updated weights for policy 0, policy_version 924126 (0.0008) [2023-12-27 12:02:59,228][20381] Updated weights for policy 1, policy_version 923361 (0.0008) [2023-12-27 12:02:59,292][20381] Updated weights for policy 1, policy_version 923371 (0.0008) [2023-12-27 12:02:59,354][20381] Updated weights for policy 1, policy_version 923381 (0.0008) [2023-12-27 12:02:59,515][20347] Updated weights for policy 0, policy_version 924136 (0.0008) [2023-12-27 12:02:59,565][20347] Updated weights for policy 0, policy_version 924146 (0.0008) [2023-12-27 12:02:59,620][20347] Updated weights for policy 0, policy_version 924156 (0.0006) [2023-12-27 12:03:00,057][20381] Updated weights for policy 1, policy_version 923391 (0.0007) [2023-12-27 12:03:00,124][20381] Updated weights for policy 1, policy_version 923401 (0.0008) [2023-12-27 12:03:00,178][20381] Updated weights for policy 1, policy_version 923411 (0.0007) [2023-12-27 12:03:00,334][20347] Updated weights for policy 0, policy_version 924166 (0.0007) [2023-12-27 12:03:00,390][20347] Updated weights for policy 0, policy_version 924176 (0.0007) [2023-12-27 12:03:00,442][20347] Updated weights for policy 0, policy_version 924186 (0.0007) [2023-12-27 12:03:00,897][20381] Updated weights for policy 1, policy_version 923421 (0.0008) [2023-12-27 12:03:00,948][20381] Updated weights for policy 1, policy_version 923431 (0.0008) [2023-12-27 12:03:01,000][20381] Updated weights for policy 1, policy_version 923441 (0.0007) [2023-12-27 12:03:01,163][20347] Updated weights for policy 0, policy_version 924196 (0.0008) [2023-12-27 12:03:01,226][20347] Updated weights for policy 0, policy_version 924206 (0.0005) [2023-12-27 12:03:01,292][20347] Updated weights for policy 0, policy_version 924216 (0.0008) [2023-12-27 12:03:01,821][20381] Updated weights for policy 1, policy_version 923451 (0.0008) [2023-12-27 12:03:01,880][20381] Updated weights for policy 1, policy_version 923461 (0.0007) [2023-12-27 12:03:01,901][20347] Updated weights for policy 0, policy_version 924226 (0.0007) [2023-12-27 12:03:01,941][20381] Updated weights for policy 1, policy_version 923471 (0.0008) [2023-12-27 12:03:01,965][20347] Updated weights for policy 0, policy_version 924236 (0.0005) [2023-12-27 12:03:02,019][20347] Updated weights for policy 0, policy_version 924246 (0.0006) [2023-12-27 12:03:02,074][20347] Updated weights for policy 0, policy_version 924256 (0.0008) [2023-12-27 12:03:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 473079808. Throughput: 0: 10407.1, 1: 9977.6. Samples: 473090236. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:02,363][19170] Avg episode reward: [(0, '316.750'), (1, '448.070')] [2023-12-27 12:03:02,614][20381] Updated weights for policy 1, policy_version 923481 (0.0007) [2023-12-27 12:03:02,674][20381] Updated weights for policy 1, policy_version 923491 (0.0008) [2023-12-27 12:03:02,735][20381] Updated weights for policy 1, policy_version 923501 (0.0010) [2023-12-27 12:03:02,760][20347] Updated weights for policy 0, policy_version 924266 (0.0010) [2023-12-27 12:03:02,794][20381] Updated weights for policy 1, policy_version 923511 (0.0010) [2023-12-27 12:03:02,813][20347] Updated weights for policy 0, policy_version 924276 (0.0010) [2023-12-27 12:03:02,870][20347] Updated weights for policy 0, policy_version 924286 (0.0010) [2023-12-27 12:03:03,420][20381] Updated weights for policy 1, policy_version 923521 (0.0009) [2023-12-27 12:03:03,465][20381] Updated weights for policy 1, policy_version 923531 (0.0007) [2023-12-27 12:03:03,518][20381] Updated weights for policy 1, policy_version 923541 (0.0007) [2023-12-27 12:03:03,535][20347] Updated weights for policy 0, policy_version 924296 (0.0010) [2023-12-27 12:03:03,585][20347] Updated weights for policy 0, policy_version 924306 (0.0010) [2023-12-27 12:03:03,638][20347] Updated weights for policy 0, policy_version 924316 (0.0009) [2023-12-27 12:03:04,276][20381] Updated weights for policy 1, policy_version 923551 (0.0007) [2023-12-27 12:03:04,330][20381] Updated weights for policy 1, policy_version 923561 (0.0008) [2023-12-27 12:03:04,336][20347] Updated weights for policy 0, policy_version 924326 (0.0007) [2023-12-27 12:03:04,388][20381] Updated weights for policy 1, policy_version 923571 (0.0008) [2023-12-27 12:03:04,393][20347] Updated weights for policy 0, policy_version 924336 (0.0006) [2023-12-27 12:03:04,454][20347] Updated weights for policy 0, policy_version 924346 (0.0010) [2023-12-27 12:03:05,028][20381] Updated weights for policy 1, policy_version 923581 (0.0008) [2023-12-27 12:03:05,079][20347] Updated weights for policy 0, policy_version 924356 (0.0008) [2023-12-27 12:03:05,081][20381] Updated weights for policy 1, policy_version 923591 (0.0007) [2023-12-27 12:03:05,129][20347] Updated weights for policy 0, policy_version 924366 (0.0006) [2023-12-27 12:03:05,136][20381] Updated weights for policy 1, policy_version 923601 (0.0008) [2023-12-27 12:03:05,179][20347] Updated weights for policy 0, policy_version 924376 (0.0008) [2023-12-27 12:03:05,742][20381] Updated weights for policy 1, policy_version 923611 (0.0006) [2023-12-27 12:03:05,800][20381] Updated weights for policy 1, policy_version 923621 (0.0008) [2023-12-27 12:03:05,851][20347] Updated weights for policy 0, policy_version 924386 (0.0007) [2023-12-27 12:03:05,856][20381] Updated weights for policy 1, policy_version 923631 (0.0007) [2023-12-27 12:03:05,903][20347] Updated weights for policy 0, policy_version 924396 (0.0006) [2023-12-27 12:03:05,924][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 12:03:06,572][20381] Updated weights for policy 1, policy_version 923641 (0.0005) [2023-12-27 12:03:06,635][20381] Updated weights for policy 1, policy_version 923651 (0.0007) [2023-12-27 12:03:06,641][20347] Updated weights for policy 0, policy_version 924406 (0.0008) [2023-12-27 12:03:06,689][20381] Updated weights for policy 1, policy_version 923661 (0.0006) [2023-12-27 12:03:06,698][20347] Updated weights for policy 0, policy_version 924416 (0.0006) [2023-12-27 12:03:06,745][20381] Updated weights for policy 1, policy_version 923671 (0.0007) [2023-12-27 12:03:06,752][20347] Updated weights for policy 0, policy_version 924426 (0.0006) [2023-12-27 12:03:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 473178112. Throughput: 0: 10429.9, 1: 10004.4. Samples: 473152244. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:07,363][19170] Avg episode reward: [(0, '300.380'), (1, '383.990')] [2023-12-27 12:03:07,378][20347] Updated weights for policy 0, policy_version 924436 (0.0008) [2023-12-27 12:03:07,411][20381] Updated weights for policy 1, policy_version 923681 (0.0007) [2023-12-27 12:03:07,432][20347] Updated weights for policy 0, policy_version 924446 (0.0006) [2023-12-27 12:03:07,463][20381] Updated weights for policy 1, policy_version 923691 (0.0005) [2023-12-27 12:03:07,477][20347] Updated weights for policy 0, policy_version 924456 (0.0007) [2023-12-27 12:03:07,516][20381] Updated weights for policy 1, policy_version 923701 (0.0005) [2023-12-27 12:03:07,517][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000924464_236699648.pth... [2023-12-27 12:03:07,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000923264_236388352.pth [2023-12-27 12:03:07,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000923704_236494848.pth... [2023-12-27 12:03:07,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000922520_236191744.pth [2023-12-27 12:03:08,087][20381] Updated weights for policy 1, policy_version 923711 (0.0007) [2023-12-27 12:03:08,100][20347] Updated weights for policy 0, policy_version 924466 (0.0007) [2023-12-27 12:03:08,141][20381] Updated weights for policy 1, policy_version 923721 (0.0008) [2023-12-27 12:03:08,159][20347] Updated weights for policy 0, policy_version 924476 (0.0005) [2023-12-27 12:03:08,197][20381] Updated weights for policy 1, policy_version 923731 (0.0007) [2023-12-27 12:03:08,218][20347] Updated weights for policy 0, policy_version 924486 (0.0005) [2023-12-27 12:03:08,279][20347] Updated weights for policy 0, policy_version 924496 (0.0007) [2023-12-27 12:03:08,847][20347] Updated weights for policy 0, policy_version 924506 (0.0011) [2023-12-27 12:03:08,887][20381] Updated weights for policy 1, policy_version 923741 (0.0007) [2023-12-27 12:03:08,901][20347] Updated weights for policy 0, policy_version 924516 (0.0011) [2023-12-27 12:03:08,940][20381] Updated weights for policy 1, policy_version 923751 (0.0006) [2023-12-27 12:03:08,946][20347] Updated weights for policy 0, policy_version 924526 (0.0010) [2023-12-27 12:03:08,997][20381] Updated weights for policy 1, policy_version 923761 (0.0006) [2023-12-27 12:03:09,675][20347] Updated weights for policy 0, policy_version 924536 (0.0008) [2023-12-27 12:03:09,742][20381] Updated weights for policy 1, policy_version 923771 (0.0007) [2023-12-27 12:03:09,746][20347] Updated weights for policy 0, policy_version 924546 (0.0006) [2023-12-27 12:03:09,803][20381] Updated weights for policy 1, policy_version 923781 (0.0006) [2023-12-27 12:03:09,807][20347] Updated weights for policy 0, policy_version 924556 (0.0006) [2023-12-27 12:03:09,870][20381] Updated weights for policy 1, policy_version 923791 (0.0007) [2023-12-27 12:03:10,484][20347] Updated weights for policy 0, policy_version 924566 (0.0010) [2023-12-27 12:03:10,540][20347] Updated weights for policy 0, policy_version 924576 (0.0010) [2023-12-27 12:03:10,582][20381] Updated weights for policy 1, policy_version 923801 (0.0011) [2023-12-27 12:03:10,596][20347] Updated weights for policy 0, policy_version 924586 (0.0010) [2023-12-27 12:03:10,639][20381] Updated weights for policy 1, policy_version 923811 (0.0011) [2023-12-27 12:03:10,695][20381] Updated weights for policy 1, policy_version 923821 (0.0011) [2023-12-27 12:03:10,755][20381] Updated weights for policy 1, policy_version 923831 (0.0011) [2023-12-27 12:03:11,346][20347] Updated weights for policy 0, policy_version 924596 (0.0010) [2023-12-27 12:03:11,415][20347] Updated weights for policy 0, policy_version 924606 (0.0011) [2023-12-27 12:03:11,467][20347] Updated weights for policy 0, policy_version 924616 (0.0006) [2023-12-27 12:03:11,503][20381] Updated weights for policy 1, policy_version 923841 (0.0008) [2023-12-27 12:03:11,567][20381] Updated weights for policy 1, policy_version 923851 (0.0008) [2023-12-27 12:03:11,627][20381] Updated weights for policy 1, policy_version 923861 (0.0008) [2023-12-27 12:03:12,109][20347] Updated weights for policy 0, policy_version 924626 (0.0008) [2023-12-27 12:03:12,165][20347] Updated weights for policy 0, policy_version 924636 (0.0007) [2023-12-27 12:03:12,221][20347] Updated weights for policy 0, policy_version 924646 (0.0008) [2023-12-27 12:03:12,285][20347] Updated weights for policy 0, policy_version 924656 (0.0008) [2023-12-27 12:03:12,360][20381] Updated weights for policy 1, policy_version 923871 (0.0008) [2023-12-27 12:03:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 473284608. Throughput: 0: 10454.2, 1: 9995.0. Samples: 473276556. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:12,363][19170] Avg episode reward: [(0, '450.200'), (1, '515.840')] [2023-12-27 12:03:12,424][20381] Updated weights for policy 1, policy_version 923881 (0.0009) [2023-12-27 12:03:12,472][20381] Updated weights for policy 1, policy_version 923891 (0.0008) [2023-12-27 12:03:12,973][20347] Updated weights for policy 0, policy_version 924666 (0.0007) [2023-12-27 12:03:13,023][20347] Updated weights for policy 0, policy_version 924676 (0.0005) [2023-12-27 12:03:13,073][20347] Updated weights for policy 0, policy_version 924686 (0.0007) [2023-12-27 12:03:13,196][20381] Updated weights for policy 1, policy_version 923901 (0.0007) [2023-12-27 12:03:13,250][20381] Updated weights for policy 1, policy_version 923911 (0.0008) [2023-12-27 12:03:13,307][20381] Updated weights for policy 1, policy_version 923921 (0.0007) [2023-12-27 12:03:13,744][20347] Updated weights for policy 0, policy_version 924696 (0.0007) [2023-12-27 12:03:13,793][20347] Updated weights for policy 0, policy_version 924706 (0.0007) [2023-12-27 12:03:13,840][20347] Updated weights for policy 0, policy_version 924716 (0.0007) [2023-12-27 12:03:13,892][20381] Updated weights for policy 1, policy_version 923931 (0.0006) [2023-12-27 12:03:13,940][20381] Updated weights for policy 1, policy_version 923941 (0.0007) [2023-12-27 12:03:14,000][20381] Updated weights for policy 1, policy_version 923951 (0.0008) [2023-12-27 12:03:14,544][20347] Updated weights for policy 0, policy_version 924726 (0.0008) [2023-12-27 12:03:14,602][20347] Updated weights for policy 0, policy_version 924736 (0.0008) [2023-12-27 12:03:14,656][20347] Updated weights for policy 0, policy_version 924746 (0.0008) [2023-12-27 12:03:14,686][20381] Updated weights for policy 1, policy_version 923961 (0.0007) [2023-12-27 12:03:14,736][20381] Updated weights for policy 1, policy_version 923971 (0.0005) [2023-12-27 12:03:14,799][20381] Updated weights for policy 1, policy_version 923981 (0.0009) [2023-12-27 12:03:14,856][20381] Updated weights for policy 1, policy_version 923991 (0.0006) [2023-12-27 12:03:15,382][20347] Updated weights for policy 0, policy_version 924756 (0.0008) [2023-12-27 12:03:15,438][20347] Updated weights for policy 0, policy_version 924766 (0.0007) [2023-12-27 12:03:15,466][20381] Updated weights for policy 1, policy_version 924001 (0.0007) [2023-12-27 12:03:15,493][20347] Updated weights for policy 0, policy_version 924776 (0.0005) [2023-12-27 12:03:15,518][20381] Updated weights for policy 1, policy_version 924011 (0.0006) [2023-12-27 12:03:15,568][20381] Updated weights for policy 1, policy_version 924021 (0.0007) [2023-12-27 12:03:16,195][20347] Updated weights for policy 0, policy_version 924786 (0.0006) [2023-12-27 12:03:16,237][20381] Updated weights for policy 1, policy_version 924031 (0.0010) [2023-12-27 12:03:16,250][20347] Updated weights for policy 0, policy_version 924796 (0.0007) [2023-12-27 12:03:16,292][20381] Updated weights for policy 1, policy_version 924041 (0.0010) [2023-12-27 12:03:16,305][20347] Updated weights for policy 0, policy_version 924806 (0.0008) [2023-12-27 12:03:16,351][20381] Updated weights for policy 1, policy_version 924051 (0.0006) [2023-12-27 12:03:16,354][20347] Updated weights for policy 0, policy_version 924816 (0.0005) [2023-12-27 12:03:16,923][20347] Updated weights for policy 0, policy_version 924826 (0.0005) [2023-12-27 12:03:16,979][20381] Updated weights for policy 1, policy_version 924061 (0.0006) [2023-12-27 12:03:16,984][20347] Updated weights for policy 0, policy_version 924836 (0.0005) [2023-12-27 12:03:17,028][20347] Updated weights for policy 0, policy_version 924846 (0.0005) [2023-12-27 12:03:17,038][20381] Updated weights for policy 1, policy_version 924071 (0.0006) [2023-12-27 12:03:17,092][20381] Updated weights for policy 1, policy_version 924081 (0.0007) [2023-12-27 12:03:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 473391104. Throughput: 0: 10326.1, 1: 10067.6. Samples: 473400024. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:17,363][19170] Avg episode reward: [(0, '268.900'), (1, '494.780')] [2023-12-27 12:03:17,568][20347] Updated weights for policy 0, policy_version 924856 (0.0009) [2023-12-27 12:03:17,622][20347] Updated weights for policy 0, policy_version 924866 (0.0010) [2023-12-27 12:03:17,683][20347] Updated weights for policy 0, policy_version 924876 (0.0010) [2023-12-27 12:03:17,791][20381] Updated weights for policy 1, policy_version 924091 (0.0008) [2023-12-27 12:03:17,840][20381] Updated weights for policy 1, policy_version 924101 (0.0007) [2023-12-27 12:03:17,897][20381] Updated weights for policy 1, policy_version 924111 (0.0007) [2023-12-27 12:03:18,361][20347] Updated weights for policy 0, policy_version 924886 (0.0009) [2023-12-27 12:03:18,423][20347] Updated weights for policy 0, policy_version 924896 (0.0008) [2023-12-27 12:03:18,489][20347] Updated weights for policy 0, policy_version 924906 (0.0008) [2023-12-27 12:03:18,604][20381] Updated weights for policy 1, policy_version 924121 (0.0007) [2023-12-27 12:03:18,667][20381] Updated weights for policy 1, policy_version 924131 (0.0008) [2023-12-27 12:03:18,735][20381] Updated weights for policy 1, policy_version 924141 (0.0008) [2023-12-27 12:03:18,793][20381] Updated weights for policy 1, policy_version 924151 (0.0007) [2023-12-27 12:03:19,154][20347] Updated weights for policy 0, policy_version 924916 (0.0008) [2023-12-27 12:03:19,212][20347] Updated weights for policy 0, policy_version 924926 (0.0008) [2023-12-27 12:03:19,277][20347] Updated weights for policy 0, policy_version 924936 (0.0012) [2023-12-27 12:03:19,451][20381] Updated weights for policy 1, policy_version 924161 (0.0007) [2023-12-27 12:03:19,517][20381] Updated weights for policy 1, policy_version 924171 (0.0007) [2023-12-27 12:03:19,579][20381] Updated weights for policy 1, policy_version 924181 (0.0008) [2023-12-27 12:03:19,977][20347] Updated weights for policy 0, policy_version 924946 (0.0009) [2023-12-27 12:03:20,035][20347] Updated weights for policy 0, policy_version 924956 (0.0009) [2023-12-27 12:03:20,099][20347] Updated weights for policy 0, policy_version 924966 (0.0006) [2023-12-27 12:03:20,154][20347] Updated weights for policy 0, policy_version 924976 (0.0006) [2023-12-27 12:03:20,411][20381] Updated weights for policy 1, policy_version 924191 (0.0008) [2023-12-27 12:03:20,467][20381] Updated weights for policy 1, policy_version 924201 (0.0008) [2023-12-27 12:03:20,524][20381] Updated weights for policy 1, policy_version 924211 (0.0008) [2023-12-27 12:03:20,738][20347] Updated weights for policy 0, policy_version 924986 (0.0007) [2023-12-27 12:03:20,794][20347] Updated weights for policy 0, policy_version 924996 (0.0008) [2023-12-27 12:03:20,855][20347] Updated weights for policy 0, policy_version 925006 (0.0009) [2023-12-27 12:03:21,206][20381] Updated weights for policy 1, policy_version 924221 (0.0008) [2023-12-27 12:03:21,264][20381] Updated weights for policy 1, policy_version 924231 (0.0006) [2023-12-27 12:03:21,324][20381] Updated weights for policy 1, policy_version 924241 (0.0007) [2023-12-27 12:03:21,650][20347] Updated weights for policy 0, policy_version 925016 (0.0009) [2023-12-27 12:03:21,712][20347] Updated weights for policy 0, policy_version 925026 (0.0009) [2023-12-27 12:03:21,774][20347] Updated weights for policy 0, policy_version 925036 (0.0010) [2023-12-27 12:03:22,063][20381] Updated weights for policy 1, policy_version 924251 (0.0008) [2023-12-27 12:03:22,127][20381] Updated weights for policy 1, policy_version 924261 (0.0007) [2023-12-27 12:03:22,187][20381] Updated weights for policy 1, policy_version 924271 (0.0008) [2023-12-27 12:03:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.5, 300 sec: 20521.6). Total num frames: 473489408. Throughput: 0: 10348.4, 1: 10009.5. Samples: 473460388. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:22,363][19170] Avg episode reward: [(0, '119.580'), (1, '405.230')] [2023-12-27 12:03:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000924280_236642304.pth... [2023-12-27 12:03:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000925040_236847104.pth... [2023-12-27 12:03:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000923096_236339200.pth [2023-12-27 12:03:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000923872_236544000.pth [2023-12-27 12:03:22,441][20347] Updated weights for policy 0, policy_version 925046 (0.0007) [2023-12-27 12:03:22,501][20347] Updated weights for policy 0, policy_version 925056 (0.0007) [2023-12-27 12:03:22,561][20347] Updated weights for policy 0, policy_version 925066 (0.0008) [2023-12-27 12:03:22,898][20381] Updated weights for policy 1, policy_version 924281 (0.0008) [2023-12-27 12:03:22,950][20381] Updated weights for policy 1, policy_version 924291 (0.0007) [2023-12-27 12:03:23,014][20381] Updated weights for policy 1, policy_version 924301 (0.0008) [2023-12-27 12:03:23,077][20381] Updated weights for policy 1, policy_version 924311 (0.0009) [2023-12-27 12:03:23,205][20347] Updated weights for policy 0, policy_version 925076 (0.0007) [2023-12-27 12:03:23,256][20347] Updated weights for policy 0, policy_version 925086 (0.0005) [2023-12-27 12:03:23,307][20347] Updated weights for policy 0, policy_version 925096 (0.0005) [2023-12-27 12:03:23,819][20381] Updated weights for policy 1, policy_version 924321 (0.0009) [2023-12-27 12:03:23,864][20347] Updated weights for policy 0, policy_version 925106 (0.0005) [2023-12-27 12:03:23,869][20381] Updated weights for policy 1, policy_version 924331 (0.0007) [2023-12-27 12:03:23,924][20381] Updated weights for policy 1, policy_version 924341 (0.0007) [2023-12-27 12:03:23,927][20347] Updated weights for policy 0, policy_version 925116 (0.0005) [2023-12-27 12:03:23,991][20347] Updated weights for policy 0, policy_version 925126 (0.0005) [2023-12-27 12:03:24,044][20347] Updated weights for policy 0, policy_version 925136 (0.0005) [2023-12-27 12:03:24,567][20347] Updated weights for policy 0, policy_version 925146 (0.0005) [2023-12-27 12:03:24,608][20381] Updated weights for policy 1, policy_version 924351 (0.0007) [2023-12-27 12:03:24,626][20347] Updated weights for policy 0, policy_version 925156 (0.0005) [2023-12-27 12:03:24,655][20381] Updated weights for policy 1, policy_version 924361 (0.0008) [2023-12-27 12:03:24,683][20347] Updated weights for policy 0, policy_version 925166 (0.0005) [2023-12-27 12:03:24,701][20381] Updated weights for policy 1, policy_version 924371 (0.0008) [2023-12-27 12:03:25,242][20347] Updated weights for policy 0, policy_version 925176 (0.0005) [2023-12-27 12:03:25,295][20347] Updated weights for policy 0, policy_version 925186 (0.0005) [2023-12-27 12:03:25,348][20347] Updated weights for policy 0, policy_version 925196 (0.0010) [2023-12-27 12:03:25,449][20381] Updated weights for policy 1, policy_version 924381 (0.0007) [2023-12-27 12:03:25,496][20381] Updated weights for policy 1, policy_version 924391 (0.0007) [2023-12-27 12:03:25,539][20381] Updated weights for policy 1, policy_version 924401 (0.0007) [2023-12-27 12:03:25,979][20347] Updated weights for policy 0, policy_version 925206 (0.0007) [2023-12-27 12:03:26,040][20347] Updated weights for policy 0, policy_version 925216 (0.0008) [2023-12-27 12:03:26,098][20347] Updated weights for policy 0, policy_version 925226 (0.0010) [2023-12-27 12:03:26,277][20381] Updated weights for policy 1, policy_version 924411 (0.0007) [2023-12-27 12:03:26,336][20381] Updated weights for policy 1, policy_version 924421 (0.0007) [2023-12-27 12:03:26,394][20381] Updated weights for policy 1, policy_version 924431 (0.0007) [2023-12-27 12:03:26,751][20347] Updated weights for policy 0, policy_version 925236 (0.0010) [2023-12-27 12:03:26,805][20347] Updated weights for policy 0, policy_version 925246 (0.0010) [2023-12-27 12:03:26,856][20347] Updated weights for policy 0, policy_version 925256 (0.0010) [2023-12-27 12:03:27,091][20381] Updated weights for policy 1, policy_version 924441 (0.0007) [2023-12-27 12:03:27,146][20381] Updated weights for policy 1, policy_version 924451 (0.0007) [2023-12-27 12:03:27,210][20381] Updated weights for policy 1, policy_version 924461 (0.0008) [2023-12-27 12:03:27,263][20381] Updated weights for policy 1, policy_version 924471 (0.0008) [2023-12-27 12:03:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 473595904. Throughput: 0: 10435.5, 1: 9968.5. Samples: 473583784. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:27,363][19170] Avg episode reward: [(0, '291.620'), (1, '394.490')] [2023-12-27 12:03:27,439][20347] Updated weights for policy 0, policy_version 925266 (0.0010) [2023-12-27 12:03:27,494][20347] Updated weights for policy 0, policy_version 925276 (0.0010) [2023-12-27 12:03:27,548][20347] Updated weights for policy 0, policy_version 925286 (0.0010) [2023-12-27 12:03:27,596][20347] Updated weights for policy 0, policy_version 925296 (0.0010) [2023-12-27 12:03:27,980][20381] Updated weights for policy 1, policy_version 924481 (0.0007) [2023-12-27 12:03:28,026][20381] Updated weights for policy 1, policy_version 924491 (0.0007) [2023-12-27 12:03:28,076][20381] Updated weights for policy 1, policy_version 924501 (0.0007) [2023-12-27 12:03:28,284][20347] Updated weights for policy 0, policy_version 925306 (0.0010) [2023-12-27 12:03:28,345][20347] Updated weights for policy 0, policy_version 925316 (0.0010) [2023-12-27 12:03:28,396][20347] Updated weights for policy 0, policy_version 925326 (0.0010) [2023-12-27 12:03:28,777][20381] Updated weights for policy 1, policy_version 924511 (0.0007) [2023-12-27 12:03:28,831][20381] Updated weights for policy 1, policy_version 924521 (0.0007) [2023-12-27 12:03:28,887][20381] Updated weights for policy 1, policy_version 924531 (0.0007) [2023-12-27 12:03:29,076][20347] Updated weights for policy 0, policy_version 925336 (0.0009) [2023-12-27 12:03:29,131][20347] Updated weights for policy 0, policy_version 925346 (0.0005) [2023-12-27 12:03:29,180][20347] Updated weights for policy 0, policy_version 925356 (0.0005) [2023-12-27 12:03:29,599][20381] Updated weights for policy 1, policy_version 924541 (0.0007) [2023-12-27 12:03:29,647][20381] Updated weights for policy 1, policy_version 924551 (0.0007) [2023-12-27 12:03:29,704][20381] Updated weights for policy 1, policy_version 924561 (0.0006) [2023-12-27 12:03:29,794][20347] Updated weights for policy 0, policy_version 925366 (0.0007) [2023-12-27 12:03:29,851][20347] Updated weights for policy 0, policy_version 925376 (0.0008) [2023-12-27 12:03:29,900][20347] Updated weights for policy 0, policy_version 925386 (0.0008) [2023-12-27 12:03:30,351][20381] Updated weights for policy 1, policy_version 924571 (0.0008) [2023-12-27 12:03:30,415][20381] Updated weights for policy 1, policy_version 924581 (0.0008) [2023-12-27 12:03:30,471][20381] Updated weights for policy 1, policy_version 924591 (0.0007) [2023-12-27 12:03:30,564][20347] Updated weights for policy 0, policy_version 925396 (0.0007) [2023-12-27 12:03:30,616][20347] Updated weights for policy 0, policy_version 925406 (0.0005) [2023-12-27 12:03:30,669][20347] Updated weights for policy 0, policy_version 925416 (0.0006) [2023-12-27 12:03:31,231][20381] Updated weights for policy 1, policy_version 924601 (0.0008) [2023-12-27 12:03:31,302][20381] Updated weights for policy 1, policy_version 924611 (0.0008) [2023-12-27 12:03:31,322][20347] Updated weights for policy 0, policy_version 925426 (0.0006) [2023-12-27 12:03:31,364][20381] Updated weights for policy 1, policy_version 924621 (0.0008) [2023-12-27 12:03:31,396][20347] Updated weights for policy 0, policy_version 925436 (0.0008) [2023-12-27 12:03:31,445][20381] Updated weights for policy 1, policy_version 924631 (0.0007) [2023-12-27 12:03:31,457][20347] Updated weights for policy 0, policy_version 925446 (0.0008) [2023-12-27 12:03:31,520][20347] Updated weights for policy 0, policy_version 925456 (0.0008) [2023-12-27 12:03:32,131][20381] Updated weights for policy 1, policy_version 924641 (0.0006) [2023-12-27 12:03:32,187][20381] Updated weights for policy 1, policy_version 924651 (0.0005) [2023-12-27 12:03:32,233][20381] Updated weights for policy 1, policy_version 924661 (0.0006) [2023-12-27 12:03:32,259][20347] Updated weights for policy 0, policy_version 925466 (0.0009) [2023-12-27 12:03:32,316][20347] Updated weights for policy 0, policy_version 925476 (0.0008) [2023-12-27 12:03:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 473694208. Throughput: 0: 10450.8, 1: 9912.4. Samples: 473706016. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:32,363][19170] Avg episode reward: [(0, '213.260'), (1, '438.830')] [2023-12-27 12:03:32,383][20347] Updated weights for policy 0, policy_version 925486 (0.0007) [2023-12-27 12:03:32,881][20381] Updated weights for policy 1, policy_version 924671 (0.0007) [2023-12-27 12:03:32,931][20381] Updated weights for policy 1, policy_version 924681 (0.0009) [2023-12-27 12:03:32,983][20381] Updated weights for policy 1, policy_version 924691 (0.0009) [2023-12-27 12:03:33,022][20347] Updated weights for policy 0, policy_version 925496 (0.0006) [2023-12-27 12:03:33,083][20347] Updated weights for policy 0, policy_version 925506 (0.0007) [2023-12-27 12:03:33,135][20347] Updated weights for policy 0, policy_version 925516 (0.0007) [2023-12-27 12:03:33,633][20381] Updated weights for policy 1, policy_version 924701 (0.0006) [2023-12-27 12:03:33,692][20381] Updated weights for policy 1, policy_version 924711 (0.0005) [2023-12-27 12:03:33,735][20381] Updated weights for policy 1, policy_version 924721 (0.0005) [2023-12-27 12:03:33,797][20347] Updated weights for policy 0, policy_version 925526 (0.0006) [2023-12-27 12:03:33,849][20347] Updated weights for policy 0, policy_version 925536 (0.0006) [2023-12-27 12:03:33,900][20347] Updated weights for policy 0, policy_version 925546 (0.0006) [2023-12-27 12:03:34,394][20381] Updated weights for policy 1, policy_version 924731 (0.0006) [2023-12-27 12:03:34,456][20381] Updated weights for policy 1, policy_version 924741 (0.0008) [2023-12-27 12:03:34,510][20381] Updated weights for policy 1, policy_version 924751 (0.0008) [2023-12-27 12:03:34,594][20347] Updated weights for policy 0, policy_version 925556 (0.0008) [2023-12-27 12:03:34,646][20347] Updated weights for policy 0, policy_version 925566 (0.0008) [2023-12-27 12:03:34,697][20347] Updated weights for policy 0, policy_version 925576 (0.0006) [2023-12-27 12:03:35,229][20381] Updated weights for policy 1, policy_version 924761 (0.0008) [2023-12-27 12:03:35,276][20381] Updated weights for policy 1, policy_version 924771 (0.0007) [2023-12-27 12:03:35,329][20381] Updated weights for policy 1, policy_version 924781 (0.0007) [2023-12-27 12:03:35,377][20347] Updated weights for policy 0, policy_version 925586 (0.0007) [2023-12-27 12:03:35,379][20381] Updated weights for policy 1, policy_version 924791 (0.0008) [2023-12-27 12:03:35,424][20347] Updated weights for policy 0, policy_version 925596 (0.0006) [2023-12-27 12:03:35,474][20347] Updated weights for policy 0, policy_version 925606 (0.0008) [2023-12-27 12:03:35,534][20347] Updated weights for policy 0, policy_version 925616 (0.0008) [2023-12-27 12:03:36,044][20381] Updated weights for policy 1, policy_version 924801 (0.0006) [2023-12-27 12:03:36,103][20381] Updated weights for policy 1, policy_version 924811 (0.0006) [2023-12-27 12:03:36,170][20381] Updated weights for policy 1, policy_version 924821 (0.0008) [2023-12-27 12:03:36,268][20347] Updated weights for policy 0, policy_version 925626 (0.0006) [2023-12-27 12:03:36,331][20347] Updated weights for policy 0, policy_version 925636 (0.0007) [2023-12-27 12:03:36,387][20347] Updated weights for policy 0, policy_version 925646 (0.0006) [2023-12-27 12:03:36,885][20381] Updated weights for policy 1, policy_version 924831 (0.0008) [2023-12-27 12:03:36,940][20381] Updated weights for policy 1, policy_version 924841 (0.0008) [2023-12-27 12:03:36,988][20347] Updated weights for policy 0, policy_version 925656 (0.0006) [2023-12-27 12:03:36,994][20381] Updated weights for policy 1, policy_version 924851 (0.0007) [2023-12-27 12:03:37,035][20347] Updated weights for policy 0, policy_version 925666 (0.0006) [2023-12-27 12:03:37,084][20347] Updated weights for policy 0, policy_version 925677 (0.0009) [2023-12-27 12:03:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 473800704. Throughput: 0: 10436.2, 1: 9971.5. Samples: 473767880. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:37,363][19170] Avg episode reward: [(0, '180.520'), (1, '468.560')] [2023-12-27 12:03:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000924856_236789760.pth... [2023-12-27 12:03:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000925680_237010944.pth... [2023-12-27 12:03:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000924464_236699648.pth [2023-12-27 12:03:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000923704_236494848.pth [2023-12-27 12:03:37,683][20381] Updated weights for policy 1, policy_version 924861 (0.0007) [2023-12-27 12:03:37,718][20347] Updated weights for policy 0, policy_version 925687 (0.0009) [2023-12-27 12:03:37,739][20381] Updated weights for policy 1, policy_version 924871 (0.0006) [2023-12-27 12:03:37,777][20347] Updated weights for policy 0, policy_version 925697 (0.0010) [2023-12-27 12:03:37,786][20381] Updated weights for policy 1, policy_version 924881 (0.0007) [2023-12-27 12:03:37,833][20347] Updated weights for policy 0, policy_version 925707 (0.0010) [2023-12-27 12:03:38,394][20381] Updated weights for policy 1, policy_version 924891 (0.0007) [2023-12-27 12:03:38,460][20381] Updated weights for policy 1, policy_version 924901 (0.0006) [2023-12-27 12:03:38,511][20347] Updated weights for policy 0, policy_version 925717 (0.0008) [2023-12-27 12:03:38,519][20381] Updated weights for policy 1, policy_version 924911 (0.0006) [2023-12-27 12:03:38,570][20347] Updated weights for policy 0, policy_version 925727 (0.0010) [2023-12-27 12:03:38,623][20347] Updated weights for policy 0, policy_version 925737 (0.0010) [2023-12-27 12:03:39,186][20381] Updated weights for policy 1, policy_version 924921 (0.0006) [2023-12-27 12:03:39,245][20381] Updated weights for policy 1, policy_version 924931 (0.0010) [2023-12-27 12:03:39,298][20347] Updated weights for policy 0, policy_version 925747 (0.0009) [2023-12-27 12:03:39,302][20381] Updated weights for policy 1, policy_version 924941 (0.0012) [2023-12-27 12:03:39,359][20347] Updated weights for policy 0, policy_version 925757 (0.0007) [2023-12-27 12:03:39,364][20381] Updated weights for policy 1, policy_version 924951 (0.0009) [2023-12-27 12:03:39,435][20347] Updated weights for policy 0, policy_version 925767 (0.0009) [2023-12-27 12:03:40,107][20347] Updated weights for policy 0, policy_version 925777 (0.0008) [2023-12-27 12:03:40,139][20381] Updated weights for policy 1, policy_version 924961 (0.0008) [2023-12-27 12:03:40,170][20347] Updated weights for policy 0, policy_version 925787 (0.0007) [2023-12-27 12:03:40,198][20381] Updated weights for policy 1, policy_version 924971 (0.0008) [2023-12-27 12:03:40,225][20347] Updated weights for policy 0, policy_version 925797 (0.0007) [2023-12-27 12:03:40,253][20381] Updated weights for policy 1, policy_version 924981 (0.0008) [2023-12-27 12:03:40,276][20347] Updated weights for policy 0, policy_version 925807 (0.0006) [2023-12-27 12:03:40,968][20347] Updated weights for policy 0, policy_version 925817 (0.0007) [2023-12-27 12:03:41,012][20381] Updated weights for policy 1, policy_version 924991 (0.0007) [2023-12-27 12:03:41,023][20347] Updated weights for policy 0, policy_version 925827 (0.0007) [2023-12-27 12:03:41,077][20381] Updated weights for policy 1, policy_version 925001 (0.0007) [2023-12-27 12:03:41,086][20347] Updated weights for policy 0, policy_version 925837 (0.0008) [2023-12-27 12:03:41,135][20381] Updated weights for policy 1, policy_version 925011 (0.0007) [2023-12-27 12:03:41,833][20347] Updated weights for policy 0, policy_version 925847 (0.0007) [2023-12-27 12:03:41,893][20347] Updated weights for policy 0, policy_version 925857 (0.0006) [2023-12-27 12:03:41,949][20347] Updated weights for policy 0, policy_version 925867 (0.0006) [2023-12-27 12:03:41,953][20381] Updated weights for policy 1, policy_version 925021 (0.0009) [2023-12-27 12:03:42,021][20381] Updated weights for policy 1, policy_version 925031 (0.0009) [2023-12-27 12:03:42,092][20381] Updated weights for policy 1, policy_version 925041 (0.0009) [2023-12-27 12:03:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 473899008. Throughput: 0: 10417.8, 1: 9975.4. Samples: 473889240. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:42,363][19170] Avg episode reward: [(0, '202.530'), (1, '349.410')] [2023-12-27 12:03:42,582][20347] Updated weights for policy 0, policy_version 925877 (0.0007) [2023-12-27 12:03:42,642][20347] Updated weights for policy 0, policy_version 925887 (0.0008) [2023-12-27 12:03:42,700][20347] Updated weights for policy 0, policy_version 925897 (0.0007) [2023-12-27 12:03:42,814][20381] Updated weights for policy 1, policy_version 925051 (0.0009) [2023-12-27 12:03:42,872][20381] Updated weights for policy 1, policy_version 925061 (0.0009) [2023-12-27 12:03:42,930][20381] Updated weights for policy 1, policy_version 925071 (0.0008) [2023-12-27 12:03:43,371][20347] Updated weights for policy 0, policy_version 925907 (0.0007) [2023-12-27 12:03:43,426][20347] Updated weights for policy 0, policy_version 925917 (0.0008) [2023-12-27 12:03:43,481][20347] Updated weights for policy 0, policy_version 925927 (0.0007) [2023-12-27 12:03:43,599][20381] Updated weights for policy 1, policy_version 925081 (0.0008) [2023-12-27 12:03:43,657][20381] Updated weights for policy 1, policy_version 925091 (0.0007) [2023-12-27 12:03:43,715][20381] Updated weights for policy 1, policy_version 925101 (0.0008) [2023-12-27 12:03:43,767][20381] Updated weights for policy 1, policy_version 925111 (0.0008) [2023-12-27 12:03:44,146][20347] Updated weights for policy 0, policy_version 925937 (0.0008) [2023-12-27 12:03:44,198][20347] Updated weights for policy 0, policy_version 925947 (0.0008) [2023-12-27 12:03:44,253][20347] Updated weights for policy 0, policy_version 925957 (0.0006) [2023-12-27 12:03:44,304][20347] Updated weights for policy 0, policy_version 925967 (0.0007) [2023-12-27 12:03:44,444][20381] Updated weights for policy 1, policy_version 925121 (0.0007) [2023-12-27 12:03:44,499][20381] Updated weights for policy 1, policy_version 925131 (0.0007) [2023-12-27 12:03:44,558][20381] Updated weights for policy 1, policy_version 925141 (0.0007) [2023-12-27 12:03:45,024][20347] Updated weights for policy 0, policy_version 925977 (0.0006) [2023-12-27 12:03:45,078][20347] Updated weights for policy 0, policy_version 925987 (0.0007) [2023-12-27 12:03:45,127][20347] Updated weights for policy 0, policy_version 925997 (0.0007) [2023-12-27 12:03:45,289][20381] Updated weights for policy 1, policy_version 925151 (0.0008) [2023-12-27 12:03:45,338][20381] Updated weights for policy 1, policy_version 925161 (0.0008) [2023-12-27 12:03:45,384][20381] Updated weights for policy 1, policy_version 925171 (0.0007) [2023-12-27 12:03:45,813][20347] Updated weights for policy 0, policy_version 926007 (0.0007) [2023-12-27 12:03:45,862][20347] Updated weights for policy 0, policy_version 926017 (0.0008) [2023-12-27 12:03:45,923][20347] Updated weights for policy 0, policy_version 926027 (0.0008) [2023-12-27 12:03:45,972][20381] Updated weights for policy 1, policy_version 925181 (0.0006) [2023-12-27 12:03:46,027][20381] Updated weights for policy 1, policy_version 925191 (0.0006) [2023-12-27 12:03:46,080][20381] Updated weights for policy 1, policy_version 925201 (0.0007) [2023-12-27 12:03:46,601][20347] Updated weights for policy 0, policy_version 926037 (0.0008) [2023-12-27 12:03:46,660][20347] Updated weights for policy 0, policy_version 926047 (0.0005) [2023-12-27 12:03:46,721][20347] Updated weights for policy 0, policy_version 926057 (0.0007) [2023-12-27 12:03:46,762][20381] Updated weights for policy 1, policy_version 925211 (0.0006) [2023-12-27 12:03:46,818][20381] Updated weights for policy 1, policy_version 925221 (0.0008) [2023-12-27 12:03:46,867][20381] Updated weights for policy 1, policy_version 925231 (0.0007) [2023-12-27 12:03:47,331][20347] Updated weights for policy 0, policy_version 926067 (0.0007) [2023-12-27 12:03:47,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20343.3, 300 sec: 20521.6). Total num frames: 473997312. Throughput: 0: 10417.7, 1: 10028.4. Samples: 474010316. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-12-27 12:03:47,364][19170] Avg episode reward: [(0, '131.980'), (1, '507.750')] [2023-12-27 12:03:47,382][20347] Updated weights for policy 0, policy_version 926077 (0.0008) [2023-12-27 12:03:47,428][20347] Updated weights for policy 0, policy_version 926087 (0.0007) [2023-12-27 12:03:47,529][20381] Updated weights for policy 1, policy_version 925241 (0.0008) [2023-12-27 12:03:47,581][20381] Updated weights for policy 1, policy_version 925251 (0.0008) [2023-12-27 12:03:47,636][20381] Updated weights for policy 1, policy_version 925261 (0.0007) [2023-12-27 12:03:47,692][20381] Updated weights for policy 1, policy_version 925271 (0.0008) [2023-12-27 12:03:48,093][20347] Updated weights for policy 0, policy_version 926097 (0.0008) [2023-12-27 12:03:48,143][20347] Updated weights for policy 0, policy_version 926107 (0.0005) [2023-12-27 12:03:48,204][20347] Updated weights for policy 0, policy_version 926117 (0.0006) [2023-12-27 12:03:48,260][20347] Updated weights for policy 0, policy_version 926127 (0.0010) [2023-12-27 12:03:48,471][20381] Updated weights for policy 1, policy_version 925281 (0.0008) [2023-12-27 12:03:48,530][20381] Updated weights for policy 1, policy_version 925291 (0.0007) [2023-12-27 12:03:48,588][20381] Updated weights for policy 1, policy_version 925301 (0.0008) [2023-12-27 12:03:48,936][20347] Updated weights for policy 0, policy_version 926137 (0.0010) [2023-12-27 12:03:48,989][20347] Updated weights for policy 0, policy_version 926147 (0.0010) [2023-12-27 12:03:49,041][20347] Updated weights for policy 0, policy_version 926157 (0.0008) [2023-12-27 12:03:49,299][20381] Updated weights for policy 1, policy_version 925311 (0.0006) [2023-12-27 12:03:49,366][20381] Updated weights for policy 1, policy_version 925321 (0.0007) [2023-12-27 12:03:49,432][20381] Updated weights for policy 1, policy_version 925331 (0.0009) [2023-12-27 12:03:49,733][20347] Updated weights for policy 0, policy_version 926167 (0.0008) [2023-12-27 12:03:49,796][20347] Updated weights for policy 0, policy_version 926177 (0.0008) [2023-12-27 12:03:49,863][20347] Updated weights for policy 0, policy_version 926187 (0.0008) [2023-12-27 12:03:50,080][20381] Updated weights for policy 1, policy_version 925341 (0.0008) [2023-12-27 12:03:50,142][20381] Updated weights for policy 1, policy_version 925351 (0.0006) [2023-12-27 12:03:50,204][20381] Updated weights for policy 1, policy_version 925361 (0.0009) [2023-12-27 12:03:50,481][20347] Updated weights for policy 0, policy_version 926197 (0.0007) [2023-12-27 12:03:50,530][20347] Updated weights for policy 0, policy_version 926207 (0.0007) [2023-12-27 12:03:50,587][20347] Updated weights for policy 0, policy_version 926217 (0.0007) [2023-12-27 12:03:50,848][20381] Updated weights for policy 1, policy_version 925371 (0.0006) [2023-12-27 12:03:50,913][20381] Updated weights for policy 1, policy_version 925381 (0.0008) [2023-12-27 12:03:50,978][20381] Updated weights for policy 1, policy_version 925391 (0.0008) [2023-12-27 12:03:51,342][20347] Updated weights for policy 0, policy_version 926227 (0.0009) [2023-12-27 12:03:51,408][20347] Updated weights for policy 0, policy_version 926237 (0.0008) [2023-12-27 12:03:51,461][20347] Updated weights for policy 0, policy_version 926247 (0.0008) [2023-12-27 12:03:51,708][20381] Updated weights for policy 1, policy_version 925401 (0.0008) [2023-12-27 12:03:51,775][20381] Updated weights for policy 1, policy_version 925411 (0.0008) [2023-12-27 12:03:51,842][20381] Updated weights for policy 1, policy_version 925421 (0.0008) [2023-12-27 12:03:51,898][20381] Updated weights for policy 1, policy_version 925431 (0.0008) [2023-12-27 12:03:52,237][20347] Updated weights for policy 0, policy_version 926257 (0.0008) [2023-12-27 12:03:52,294][20347] Updated weights for policy 0, policy_version 926267 (0.0008) [2023-12-27 12:03:52,341][20347] Updated weights for policy 0, policy_version 926277 (0.0006) [2023-12-27 12:03:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 474095616. Throughput: 0: 10411.1, 1: 10000.8. Samples: 474070780. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:03:52,363][19170] Avg episode reward: [(0, '238.820'), (1, '434.810')] [2023-12-27 12:03:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000925432_236937216.pth... [2023-12-27 12:03:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000924280_236642304.pth [2023-12-27 12:03:52,402][20347] Updated weights for policy 0, policy_version 926287 (0.0008) [2023-12-27 12:03:52,411][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000926288_237166592.pth... [2023-12-27 12:03:52,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000925040_236847104.pth [2023-12-27 12:03:52,617][20381] Updated weights for policy 1, policy_version 925441 (0.0007) [2023-12-27 12:03:52,676][20381] Updated weights for policy 1, policy_version 925451 (0.0007) [2023-12-27 12:03:52,729][20381] Updated weights for policy 1, policy_version 925461 (0.0008) [2023-12-27 12:03:53,066][20347] Updated weights for policy 0, policy_version 926297 (0.0008) [2023-12-27 12:03:53,130][20347] Updated weights for policy 0, policy_version 926307 (0.0007) [2023-12-27 12:03:53,190][20347] Updated weights for policy 0, policy_version 926317 (0.0008) [2023-12-27 12:03:53,445][20381] Updated weights for policy 1, policy_version 925471 (0.0007) [2023-12-27 12:03:53,494][20381] Updated weights for policy 1, policy_version 925481 (0.0007) [2023-12-27 12:03:53,543][20381] Updated weights for policy 1, policy_version 925491 (0.0005) [2023-12-27 12:03:53,904][20347] Updated weights for policy 0, policy_version 926327 (0.0008) [2023-12-27 12:03:53,961][20347] Updated weights for policy 0, policy_version 926337 (0.0006) [2023-12-27 12:03:54,023][20347] Updated weights for policy 0, policy_version 926347 (0.0007) [2023-12-27 12:03:54,194][20381] Updated weights for policy 1, policy_version 925501 (0.0007) [2023-12-27 12:03:54,251][20381] Updated weights for policy 1, policy_version 925511 (0.0007) [2023-12-27 12:03:54,308][20381] Updated weights for policy 1, policy_version 925521 (0.0008) [2023-12-27 12:03:54,664][20347] Updated weights for policy 0, policy_version 926357 (0.0008) [2023-12-27 12:03:54,709][20347] Updated weights for policy 0, policy_version 926367 (0.0007) [2023-12-27 12:03:54,766][20347] Updated weights for policy 0, policy_version 926378 (0.0007) [2023-12-27 12:03:54,939][20381] Updated weights for policy 1, policy_version 925531 (0.0008) [2023-12-27 12:03:54,996][20381] Updated weights for policy 1, policy_version 925541 (0.0007) [2023-12-27 12:03:55,046][20381] Updated weights for policy 1, policy_version 925551 (0.0008) [2023-12-27 12:03:55,418][20347] Updated weights for policy 0, policy_version 926388 (0.0008) [2023-12-27 12:03:55,472][20347] Updated weights for policy 0, policy_version 926398 (0.0007) [2023-12-27 12:03:55,531][20347] Updated weights for policy 0, policy_version 926408 (0.0007) [2023-12-27 12:03:55,788][20381] Updated weights for policy 1, policy_version 925561 (0.0008) [2023-12-27 12:03:55,841][20381] Updated weights for policy 1, policy_version 925571 (0.0008) [2023-12-27 12:03:55,896][20381] Updated weights for policy 1, policy_version 925581 (0.0007) [2023-12-27 12:03:55,946][20381] Updated weights for policy 1, policy_version 925591 (0.0008) [2023-12-27 12:03:56,175][20347] Updated weights for policy 0, policy_version 926418 (0.0008) [2023-12-27 12:03:56,234][20347] Updated weights for policy 0, policy_version 926428 (0.0008) [2023-12-27 12:03:56,297][20347] Updated weights for policy 0, policy_version 926438 (0.0008) [2023-12-27 12:03:56,353][20347] Updated weights for policy 0, policy_version 926448 (0.0008) [2023-12-27 12:03:56,662][20381] Updated weights for policy 1, policy_version 925601 (0.0008) [2023-12-27 12:03:56,714][20381] Updated weights for policy 1, policy_version 925611 (0.0008) [2023-12-27 12:03:56,771][20381] Updated weights for policy 1, policy_version 925621 (0.0008) [2023-12-27 12:03:57,023][20347] Updated weights for policy 0, policy_version 926458 (0.0007) [2023-12-27 12:03:57,067][20347] Updated weights for policy 0, policy_version 926468 (0.0007) [2023-12-27 12:03:57,111][20347] Updated weights for policy 0, policy_version 926478 (0.0007) [2023-12-27 12:03:57,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 474202112. Throughput: 0: 10378.3, 1: 9959.1. Samples: 474191740. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:03:57,363][19170] Avg episode reward: [(0, '184.510'), (1, '354.130')] [2023-12-27 12:03:57,484][20381] Updated weights for policy 1, policy_version 925631 (0.0007) [2023-12-27 12:03:57,539][20381] Updated weights for policy 1, policy_version 925641 (0.0007) [2023-12-27 12:03:57,594][20381] Updated weights for policy 1, policy_version 925651 (0.0007) [2023-12-27 12:03:57,748][20347] Updated weights for policy 0, policy_version 926488 (0.0006) [2023-12-27 12:03:57,792][20347] Updated weights for policy 0, policy_version 926498 (0.0007) [2023-12-27 12:03:57,836][20347] Updated weights for policy 0, policy_version 926508 (0.0007) [2023-12-27 12:03:58,288][20381] Updated weights for policy 1, policy_version 925661 (0.0008) [2023-12-27 12:03:58,368][20381] Updated weights for policy 1, policy_version 925671 (0.0008) [2023-12-27 12:03:58,431][20381] Updated weights for policy 1, policy_version 925681 (0.0007) [2023-12-27 12:03:58,578][20347] Updated weights for policy 0, policy_version 926518 (0.0007) [2023-12-27 12:03:58,648][20347] Updated weights for policy 0, policy_version 926528 (0.0008) [2023-12-27 12:03:58,715][20347] Updated weights for policy 0, policy_version 926538 (0.0008) [2023-12-27 12:03:59,206][20381] Updated weights for policy 1, policy_version 925691 (0.0008) [2023-12-27 12:03:59,292][20381] Updated weights for policy 1, policy_version 925701 (0.0007) [2023-12-27 12:03:59,360][20381] Updated weights for policy 1, policy_version 925711 (0.0007) [2023-12-27 12:03:59,588][20347] Updated weights for policy 0, policy_version 926548 (0.0008) [2023-12-27 12:03:59,649][20347] Updated weights for policy 0, policy_version 926558 (0.0009) [2023-12-27 12:03:59,713][20347] Updated weights for policy 0, policy_version 926568 (0.0007) [2023-12-27 12:03:59,936][20381] Updated weights for policy 1, policy_version 925721 (0.0008) [2023-12-27 12:04:00,000][20381] Updated weights for policy 1, policy_version 925731 (0.0009) [2023-12-27 12:04:00,062][20381] Updated weights for policy 1, policy_version 925741 (0.0006) [2023-12-27 12:04:00,128][20381] Updated weights for policy 1, policy_version 925751 (0.0007) [2023-12-27 12:04:00,490][20347] Updated weights for policy 0, policy_version 926578 (0.0006) [2023-12-27 12:04:00,551][20347] Updated weights for policy 0, policy_version 926588 (0.0007) [2023-12-27 12:04:00,609][20347] Updated weights for policy 0, policy_version 926598 (0.0007) [2023-12-27 12:04:00,671][20347] Updated weights for policy 0, policy_version 926608 (0.0008) [2023-12-27 12:04:00,747][20381] Updated weights for policy 1, policy_version 925761 (0.0008) [2023-12-27 12:04:00,803][20381] Updated weights for policy 1, policy_version 925771 (0.0008) [2023-12-27 12:04:00,858][20381] Updated weights for policy 1, policy_version 925781 (0.0008) [2023-12-27 12:04:01,340][20347] Updated weights for policy 0, policy_version 926618 (0.0007) [2023-12-27 12:04:01,400][20347] Updated weights for policy 0, policy_version 926628 (0.0009) [2023-12-27 12:04:01,457][20347] Updated weights for policy 0, policy_version 926638 (0.0007) [2023-12-27 12:04:01,571][20381] Updated weights for policy 1, policy_version 925791 (0.0010) [2023-12-27 12:04:01,634][20381] Updated weights for policy 1, policy_version 925801 (0.0009) [2023-12-27 12:04:01,694][20381] Updated weights for policy 1, policy_version 925811 (0.0008) [2023-12-27 12:04:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 474292224. Throughput: 0: 10246.2, 1: 9854.5. Samples: 474304556. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:02,363][19170] Avg episode reward: [(0, '296.100'), (1, '464.590')] [2023-12-27 12:04:02,520][20347] Updated weights for policy 0, policy_version 926648 (0.0010) [2023-12-27 12:04:02,579][20347] Updated weights for policy 0, policy_version 926658 (0.0009) [2023-12-27 12:04:02,642][20347] Updated weights for policy 0, policy_version 926668 (0.0008) [2023-12-27 12:04:02,651][20381] Updated weights for policy 1, policy_version 925821 (0.0007) [2023-12-27 12:04:02,712][20381] Updated weights for policy 1, policy_version 925831 (0.0006) [2023-12-27 12:04:02,775][20381] Updated weights for policy 1, policy_version 925841 (0.0006) [2023-12-27 12:04:03,313][20347] Updated weights for policy 0, policy_version 926678 (0.0007) [2023-12-27 12:04:03,323][20381] Updated weights for policy 1, policy_version 925851 (0.0005) [2023-12-27 12:04:03,358][20347] Updated weights for policy 0, policy_version 926688 (0.0007) [2023-12-27 12:04:03,373][20381] Updated weights for policy 1, policy_version 925861 (0.0008) [2023-12-27 12:04:03,407][20347] Updated weights for policy 0, policy_version 926698 (0.0005) [2023-12-27 12:04:03,421][20381] Updated weights for policy 1, policy_version 925871 (0.0007) [2023-12-27 12:04:04,005][20381] Updated weights for policy 1, policy_version 925881 (0.0007) [2023-12-27 12:04:04,042][20347] Updated weights for policy 0, policy_version 926708 (0.0007) [2023-12-27 12:04:04,064][20381] Updated weights for policy 1, policy_version 925891 (0.0008) [2023-12-27 12:04:04,103][20347] Updated weights for policy 0, policy_version 926718 (0.0006) [2023-12-27 12:04:04,119][20381] Updated weights for policy 1, policy_version 925901 (0.0007) [2023-12-27 12:04:04,184][20347] Updated weights for policy 0, policy_version 926728 (0.0006) [2023-12-27 12:04:04,186][20381] Updated weights for policy 1, policy_version 925911 (0.0008) [2023-12-27 12:04:04,802][20347] Updated weights for policy 0, policy_version 926738 (0.0007) [2023-12-27 12:04:04,861][20347] Updated weights for policy 0, policy_version 926748 (0.0008) [2023-12-27 12:04:04,890][20381] Updated weights for policy 1, policy_version 925921 (0.0007) [2023-12-27 12:04:04,917][20347] Updated weights for policy 0, policy_version 926758 (0.0006) [2023-12-27 12:04:04,948][20381] Updated weights for policy 1, policy_version 925931 (0.0007) [2023-12-27 12:04:04,976][20347] Updated weights for policy 0, policy_version 926768 (0.0005) [2023-12-27 12:04:05,007][20381] Updated weights for policy 1, policy_version 925941 (0.0008) [2023-12-27 12:04:05,620][20347] Updated weights for policy 0, policy_version 926778 (0.0005) [2023-12-27 12:04:05,670][20381] Updated weights for policy 1, policy_version 925951 (0.0008) [2023-12-27 12:04:05,675][20347] Updated weights for policy 0, policy_version 926788 (0.0005) [2023-12-27 12:04:05,724][20381] Updated weights for policy 1, policy_version 925961 (0.0008) [2023-12-27 12:04:05,729][20347] Updated weights for policy 0, policy_version 926798 (0.0005) [2023-12-27 12:04:05,785][20381] Updated weights for policy 1, policy_version 925971 (0.0006) [2023-12-27 12:04:06,284][20347] Updated weights for policy 0, policy_version 926808 (0.0006) [2023-12-27 12:04:06,346][20347] Updated weights for policy 0, policy_version 926818 (0.0006) [2023-12-27 12:04:06,407][20347] Updated weights for policy 0, policy_version 926828 (0.0007) [2023-12-27 12:04:06,421][20381] Updated weights for policy 1, policy_version 925981 (0.0008) [2023-12-27 12:04:06,483][20381] Updated weights for policy 1, policy_version 925991 (0.0006) [2023-12-27 12:04:06,551][20381] Updated weights for policy 1, policy_version 926001 (0.0006) [2023-12-27 12:04:07,051][20347] Updated weights for policy 0, policy_version 926838 (0.0008) [2023-12-27 12:04:07,102][20347] Updated weights for policy 0, policy_version 926848 (0.0010) [2023-12-27 12:04:07,157][20347] Updated weights for policy 0, policy_version 926858 (0.0007) [2023-12-27 12:04:07,174][20381] Updated weights for policy 1, policy_version 926011 (0.0007) [2023-12-27 12:04:07,231][20381] Updated weights for policy 1, policy_version 926021 (0.0011) [2023-12-27 12:04:07,289][20381] Updated weights for policy 1, policy_version 926031 (0.0011) [2023-12-27 12:04:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 474406912. Throughput: 0: 10235.5, 1: 9932.4. Samples: 474367948. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:07,364][19170] Avg episode reward: [(0, '145.200'), (1, '355.230')] [2023-12-27 12:04:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000926864_237314048.pth... [2023-12-27 12:04:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000926040_237092864.pth... [2023-12-27 12:04:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000924856_236789760.pth [2023-12-27 12:04:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000925680_237010944.pth [2023-12-27 12:04:07,884][20347] Updated weights for policy 0, policy_version 926868 (0.0006) [2023-12-27 12:04:07,938][20347] Updated weights for policy 0, policy_version 926878 (0.0007) [2023-12-27 12:04:07,965][20381] Updated weights for policy 1, policy_version 926041 (0.0010) [2023-12-27 12:04:07,996][20347] Updated weights for policy 0, policy_version 926888 (0.0007) [2023-12-27 12:04:08,029][20381] Updated weights for policy 1, policy_version 926051 (0.0009) [2023-12-27 12:04:08,092][20381] Updated weights for policy 1, policy_version 926061 (0.0005) [2023-12-27 12:04:08,139][20381] Updated weights for policy 1, policy_version 926071 (0.0007) [2023-12-27 12:04:08,695][20347] Updated weights for policy 0, policy_version 926898 (0.0008) [2023-12-27 12:04:08,759][20347] Updated weights for policy 0, policy_version 926908 (0.0008) [2023-12-27 12:04:08,809][20381] Updated weights for policy 1, policy_version 926081 (0.0011) [2023-12-27 12:04:08,816][20347] Updated weights for policy 0, policy_version 926918 (0.0006) [2023-12-27 12:04:08,871][20381] Updated weights for policy 1, policy_version 926091 (0.0011) [2023-12-27 12:04:08,876][20347] Updated weights for policy 0, policy_version 926928 (0.0008) [2023-12-27 12:04:08,933][20381] Updated weights for policy 1, policy_version 926101 (0.0011) [2023-12-27 12:04:09,701][20347] Updated weights for policy 0, policy_version 926938 (0.0007) [2023-12-27 12:04:09,760][20347] Updated weights for policy 0, policy_version 926948 (0.0008) [2023-12-27 12:04:09,809][20381] Updated weights for policy 1, policy_version 926111 (0.0011) [2023-12-27 12:04:09,835][20347] Updated weights for policy 0, policy_version 926958 (0.0008) [2023-12-27 12:04:09,886][20381] Updated weights for policy 1, policy_version 926121 (0.0011) [2023-12-27 12:04:09,962][20381] Updated weights for policy 1, policy_version 926131 (0.0010) [2023-12-27 12:04:10,580][20347] Updated weights for policy 0, policy_version 926968 (0.0007) [2023-12-27 12:04:10,638][20347] Updated weights for policy 0, policy_version 926978 (0.0008) [2023-12-27 12:04:10,694][20347] Updated weights for policy 0, policy_version 926988 (0.0008) [2023-12-27 12:04:10,698][20381] Updated weights for policy 1, policy_version 926141 (0.0009) [2023-12-27 12:04:10,761][20381] Updated weights for policy 1, policy_version 926151 (0.0006) [2023-12-27 12:04:10,823][20381] Updated weights for policy 1, policy_version 926161 (0.0006) [2023-12-27 12:04:11,479][20347] Updated weights for policy 0, policy_version 926998 (0.0008) [2023-12-27 12:04:11,542][20347] Updated weights for policy 0, policy_version 927008 (0.0008) [2023-12-27 12:04:11,582][20381] Updated weights for policy 1, policy_version 926171 (0.0008) [2023-12-27 12:04:11,604][20347] Updated weights for policy 0, policy_version 927018 (0.0006) [2023-12-27 12:04:11,656][20381] Updated weights for policy 1, policy_version 926181 (0.0007) [2023-12-27 12:04:11,724][20381] Updated weights for policy 1, policy_version 926191 (0.0009) [2023-12-27 12:04:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 20438.3). Total num frames: 474488832. Throughput: 0: 10100.2, 1: 9949.3. Samples: 474486008. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:12,363][19170] Avg episode reward: [(0, '248.790'), (1, '401.880')] [2023-12-27 12:04:12,370][20347] Updated weights for policy 0, policy_version 927028 (0.0009) [2023-12-27 12:04:12,439][20347] Updated weights for policy 0, policy_version 927038 (0.0009) [2023-12-27 12:04:12,484][20381] Updated weights for policy 1, policy_version 926201 (0.0008) [2023-12-27 12:04:12,498][20347] Updated weights for policy 0, policy_version 927048 (0.0007) [2023-12-27 12:04:12,546][20381] Updated weights for policy 1, policy_version 926211 (0.0007) [2023-12-27 12:04:12,610][20381] Updated weights for policy 1, policy_version 926221 (0.0008) [2023-12-27 12:04:12,675][20381] Updated weights for policy 1, policy_version 926231 (0.0008) [2023-12-27 12:04:13,249][20347] Updated weights for policy 0, policy_version 927058 (0.0007) [2023-12-27 12:04:13,298][20347] Updated weights for policy 0, policy_version 927068 (0.0005) [2023-12-27 12:04:13,354][20347] Updated weights for policy 0, policy_version 927078 (0.0005) [2023-12-27 12:04:13,358][20381] Updated weights for policy 1, policy_version 926241 (0.0007) [2023-12-27 12:04:13,410][20347] Updated weights for policy 0, policy_version 927088 (0.0005) [2023-12-27 12:04:13,417][20381] Updated weights for policy 1, policy_version 926251 (0.0007) [2023-12-27 12:04:13,474][20381] Updated weights for policy 1, policy_version 926261 (0.0006) [2023-12-27 12:04:13,969][20347] Updated weights for policy 0, policy_version 927098 (0.0005) [2023-12-27 12:04:14,023][20347] Updated weights for policy 0, policy_version 927108 (0.0005) [2023-12-27 12:04:14,074][20381] Updated weights for policy 1, policy_version 926271 (0.0006) [2023-12-27 12:04:14,079][20347] Updated weights for policy 0, policy_version 927118 (0.0006) [2023-12-27 12:04:14,121][20381] Updated weights for policy 1, policy_version 926281 (0.0007) [2023-12-27 12:04:14,169][20381] Updated weights for policy 1, policy_version 926291 (0.0006) [2023-12-27 12:04:14,602][20347] Updated weights for policy 0, policy_version 927128 (0.0006) [2023-12-27 12:04:14,650][20347] Updated weights for policy 0, policy_version 927138 (0.0005) [2023-12-27 12:04:14,699][20347] Updated weights for policy 0, policy_version 927148 (0.0007) [2023-12-27 12:04:14,779][20381] Updated weights for policy 1, policy_version 926301 (0.0009) [2023-12-27 12:04:14,841][20381] Updated weights for policy 1, policy_version 926311 (0.0008) [2023-12-27 12:04:14,904][20381] Updated weights for policy 1, policy_version 926321 (0.0008) [2023-12-27 12:04:15,398][20347] Updated weights for policy 0, policy_version 927158 (0.0008) [2023-12-27 12:04:15,451][20347] Updated weights for policy 0, policy_version 927168 (0.0008) [2023-12-27 12:04:15,502][20347] Updated weights for policy 0, policy_version 927178 (0.0008) [2023-12-27 12:04:15,615][20381] Updated weights for policy 1, policy_version 926331 (0.0008) [2023-12-27 12:04:15,672][20381] Updated weights for policy 1, policy_version 926341 (0.0006) [2023-12-27 12:04:15,727][20381] Updated weights for policy 1, policy_version 926351 (0.0007) [2023-12-27 12:04:16,229][20347] Updated weights for policy 0, policy_version 927188 (0.0008) [2023-12-27 12:04:16,281][20347] Updated weights for policy 0, policy_version 927198 (0.0008) [2023-12-27 12:04:16,333][20347] Updated weights for policy 0, policy_version 927208 (0.0007) [2023-12-27 12:04:16,348][20381] Updated weights for policy 1, policy_version 926361 (0.0007) [2023-12-27 12:04:16,403][20381] Updated weights for policy 1, policy_version 926371 (0.0005) [2023-12-27 12:04:16,457][20381] Updated weights for policy 1, policy_version 926381 (0.0007) [2023-12-27 12:04:16,509][20381] Updated weights for policy 1, policy_version 926391 (0.0008) [2023-12-27 12:04:17,053][20381] Updated weights for policy 1, policy_version 926401 (0.0008) [2023-12-27 12:04:17,096][20347] Updated weights for policy 0, policy_version 927218 (0.0008) [2023-12-27 12:04:17,113][20381] Updated weights for policy 1, policy_version 926411 (0.0007) [2023-12-27 12:04:17,145][20347] Updated weights for policy 0, policy_version 927228 (0.0006) [2023-12-27 12:04:17,169][20381] Updated weights for policy 1, policy_version 926421 (0.0006) [2023-12-27 12:04:17,194][20347] Updated weights for policy 0, policy_version 927238 (0.0006) [2023-12-27 12:04:17,248][20347] Updated weights for policy 0, policy_version 927248 (0.0007) [2023-12-27 12:04:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 474603520. Throughput: 0: 10035.7, 1: 10018.5. Samples: 474608456. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:17,363][19170] Avg episode reward: [(0, '323.360'), (1, '520.190')] [2023-12-27 12:04:17,814][20381] Updated weights for policy 1, policy_version 926431 (0.0008) [2023-12-27 12:04:17,874][20381] Updated weights for policy 1, policy_version 926441 (0.0008) [2023-12-27 12:04:17,935][20381] Updated weights for policy 1, policy_version 926451 (0.0007) [2023-12-27 12:04:17,945][20347] Updated weights for policy 0, policy_version 927258 (0.0005) [2023-12-27 12:04:18,003][20347] Updated weights for policy 0, policy_version 927268 (0.0006) [2023-12-27 12:04:18,060][20347] Updated weights for policy 0, policy_version 927278 (0.0007) [2023-12-27 12:04:18,574][20381] Updated weights for policy 1, policy_version 926461 (0.0006) [2023-12-27 12:04:18,636][20381] Updated weights for policy 1, policy_version 926471 (0.0005) [2023-12-27 12:04:18,701][20381] Updated weights for policy 1, policy_version 926481 (0.0007) [2023-12-27 12:04:18,758][20347] Updated weights for policy 0, policy_version 927288 (0.0008) [2023-12-27 12:04:18,816][20347] Updated weights for policy 0, policy_version 927298 (0.0006) [2023-12-27 12:04:18,877][20347] Updated weights for policy 0, policy_version 927308 (0.0008) [2023-12-27 12:04:19,251][20381] Updated weights for policy 1, policy_version 926491 (0.0008) [2023-12-27 12:04:19,312][20381] Updated weights for policy 1, policy_version 926501 (0.0008) [2023-12-27 12:04:19,373][20381] Updated weights for policy 1, policy_version 926511 (0.0008) [2023-12-27 12:04:19,578][20347] Updated weights for policy 0, policy_version 927318 (0.0008) [2023-12-27 12:04:19,631][20347] Updated weights for policy 0, policy_version 927328 (0.0007) [2023-12-27 12:04:19,695][20347] Updated weights for policy 0, policy_version 927338 (0.0007) [2023-12-27 12:04:20,143][20381] Updated weights for policy 1, policy_version 926521 (0.0008) [2023-12-27 12:04:20,207][20381] Updated weights for policy 1, policy_version 926531 (0.0006) [2023-12-27 12:04:20,268][20381] Updated weights for policy 1, policy_version 926541 (0.0006) [2023-12-27 12:04:20,331][20381] Updated weights for policy 1, policy_version 926551 (0.0008) [2023-12-27 12:04:20,472][20347] Updated weights for policy 0, policy_version 927348 (0.0007) [2023-12-27 12:04:20,520][20347] Updated weights for policy 0, policy_version 927358 (0.0005) [2023-12-27 12:04:20,569][20347] Updated weights for policy 0, policy_version 927368 (0.0006) [2023-12-27 12:04:21,030][20381] Updated weights for policy 1, policy_version 926561 (0.0007) [2023-12-27 12:04:21,094][20381] Updated weights for policy 1, policy_version 926571 (0.0008) [2023-12-27 12:04:21,161][20381] Updated weights for policy 1, policy_version 926581 (0.0007) [2023-12-27 12:04:21,307][20347] Updated weights for policy 0, policy_version 927378 (0.0007) [2023-12-27 12:04:21,371][20347] Updated weights for policy 0, policy_version 927388 (0.0008) [2023-12-27 12:04:21,439][20347] Updated weights for policy 0, policy_version 927398 (0.0009) [2023-12-27 12:04:21,495][20347] Updated weights for policy 0, policy_version 927408 (0.0007) [2023-12-27 12:04:21,876][20381] Updated weights for policy 1, policy_version 926591 (0.0007) [2023-12-27 12:04:21,937][20381] Updated weights for policy 1, policy_version 926601 (0.0006) [2023-12-27 12:04:21,999][20381] Updated weights for policy 1, policy_version 926611 (0.0008) [2023-12-27 12:04:22,214][20347] Updated weights for policy 0, policy_version 927418 (0.0006) [2023-12-27 12:04:22,277][20347] Updated weights for policy 0, policy_version 927428 (0.0007) [2023-12-27 12:04:22,340][20347] Updated weights for policy 0, policy_version 927438 (0.0007) [2023-12-27 12:04:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 474701824. Throughput: 0: 10014.6, 1: 10026.3. Samples: 474669724. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:22,363][19170] Avg episode reward: [(0, '230.610'), (1, '589.680')] [2023-12-27 12:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000926616_237240320.pth... [2023-12-27 12:04:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000927440_237461504.pth... [2023-12-27 12:04:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000925432_236937216.pth [2023-12-27 12:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000926288_237166592.pth [2023-12-27 12:04:22,766][20381] Updated weights for policy 1, policy_version 926621 (0.0007) [2023-12-27 12:04:22,822][20381] Updated weights for policy 1, policy_version 926631 (0.0008) [2023-12-27 12:04:22,871][20381] Updated weights for policy 1, policy_version 926641 (0.0007) [2023-12-27 12:04:22,951][20347] Updated weights for policy 0, policy_version 927448 (0.0008) [2023-12-27 12:04:23,014][20347] Updated weights for policy 0, policy_version 927458 (0.0005) [2023-12-27 12:04:23,072][20347] Updated weights for policy 0, policy_version 927468 (0.0006) [2023-12-27 12:04:23,499][20381] Updated weights for policy 1, policy_version 926651 (0.0006) [2023-12-27 12:04:23,554][20381] Updated weights for policy 1, policy_version 926661 (0.0008) [2023-12-27 12:04:23,609][20381] Updated weights for policy 1, policy_version 926671 (0.0008) [2023-12-27 12:04:23,707][20347] Updated weights for policy 0, policy_version 927478 (0.0007) [2023-12-27 12:04:23,761][20347] Updated weights for policy 0, policy_version 927488 (0.0010) [2023-12-27 12:04:23,815][20347] Updated weights for policy 0, policy_version 927498 (0.0010) [2023-12-27 12:04:24,325][20381] Updated weights for policy 1, policy_version 926681 (0.0007) [2023-12-27 12:04:24,389][20381] Updated weights for policy 1, policy_version 926691 (0.0008) [2023-12-27 12:04:24,437][20381] Updated weights for policy 1, policy_version 926701 (0.0007) [2023-12-27 12:04:24,468][20347] Updated weights for policy 0, policy_version 927508 (0.0009) [2023-12-27 12:04:24,496][20381] Updated weights for policy 1, policy_version 926711 (0.0007) [2023-12-27 12:04:24,521][20347] Updated weights for policy 0, policy_version 927518 (0.0005) [2023-12-27 12:04:24,572][20347] Updated weights for policy 0, policy_version 927528 (0.0006) [2023-12-27 12:04:25,079][20381] Updated weights for policy 1, policy_version 926721 (0.0006) [2023-12-27 12:04:25,097][20347] Updated weights for policy 0, policy_version 927538 (0.0006) [2023-12-27 12:04:25,124][20381] Updated weights for policy 1, policy_version 926731 (0.0006) [2023-12-27 12:04:25,149][20347] Updated weights for policy 0, policy_version 927548 (0.0009) [2023-12-27 12:04:25,169][20381] Updated weights for policy 1, policy_version 926741 (0.0006) [2023-12-27 12:04:25,209][20347] Updated weights for policy 0, policy_version 927558 (0.0009) [2023-12-27 12:04:25,257][20347] Updated weights for policy 0, policy_version 927568 (0.0009) [2023-12-27 12:04:25,842][20381] Updated weights for policy 1, policy_version 926751 (0.0009) [2023-12-27 12:04:25,901][20381] Updated weights for policy 1, policy_version 926761 (0.0006) [2023-12-27 12:04:25,945][20347] Updated weights for policy 0, policy_version 927578 (0.0005) [2023-12-27 12:04:25,952][20381] Updated weights for policy 1, policy_version 926771 (0.0005) [2023-12-27 12:04:26,007][20347] Updated weights for policy 0, policy_version 927588 (0.0007) [2023-12-27 12:04:26,062][20347] Updated weights for policy 0, policy_version 927598 (0.0008) [2023-12-27 12:04:26,551][20381] Updated weights for policy 1, policy_version 926781 (0.0008) [2023-12-27 12:04:26,599][20381] Updated weights for policy 1, policy_version 926791 (0.0010) [2023-12-27 12:04:26,648][20381] Updated weights for policy 1, policy_version 926801 (0.0008) [2023-12-27 12:04:26,665][20347] Updated weights for policy 0, policy_version 927608 (0.0009) [2023-12-27 12:04:26,720][20347] Updated weights for policy 0, policy_version 927618 (0.0008) [2023-12-27 12:04:26,783][20347] Updated weights for policy 0, policy_version 927628 (0.0005) [2023-12-27 12:04:27,276][20381] Updated weights for policy 1, policy_version 926811 (0.0006) [2023-12-27 12:04:27,333][20381] Updated weights for policy 1, policy_version 926821 (0.0007) [2023-12-27 12:04:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 20438.3). Total num frames: 474800128. Throughput: 0: 10037.5, 1: 10059.8. Samples: 474793616. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:27,363][19170] Avg episode reward: [(0, '397.750'), (1, '713.930')] [2023-12-27 12:04:27,387][20381] Updated weights for policy 1, policy_version 926831 (0.0007) [2023-12-27 12:04:27,404][20347] Updated weights for policy 0, policy_version 927638 (0.0005) [2023-12-27 12:04:27,450][20347] Updated weights for policy 0, policy_version 927648 (0.0006) [2023-12-27 12:04:27,498][20347] Updated weights for policy 0, policy_version 927658 (0.0008) [2023-12-27 12:04:28,008][20381] Updated weights for policy 1, policy_version 926841 (0.0007) [2023-12-27 12:04:28,063][20381] Updated weights for policy 1, policy_version 926851 (0.0005) [2023-12-27 12:04:28,124][20381] Updated weights for policy 1, policy_version 926861 (0.0006) [2023-12-27 12:04:28,131][20347] Updated weights for policy 0, policy_version 927668 (0.0008) [2023-12-27 12:04:28,174][20381] Updated weights for policy 1, policy_version 926871 (0.0005) [2023-12-27 12:04:28,188][20347] Updated weights for policy 0, policy_version 927678 (0.0008) [2023-12-27 12:04:28,237][20347] Updated weights for policy 0, policy_version 927688 (0.0008) [2023-12-27 12:04:28,844][20381] Updated weights for policy 1, policy_version 926881 (0.0007) [2023-12-27 12:04:28,903][20381] Updated weights for policy 1, policy_version 926891 (0.0007) [2023-12-27 12:04:28,956][20347] Updated weights for policy 0, policy_version 927698 (0.0007) [2023-12-27 12:04:28,958][20381] Updated weights for policy 1, policy_version 926901 (0.0005) [2023-12-27 12:04:29,009][20347] Updated weights for policy 0, policy_version 927708 (0.0005) [2023-12-27 12:04:29,063][20347] Updated weights for policy 0, policy_version 927718 (0.0006) [2023-12-27 12:04:29,111][20347] Updated weights for policy 0, policy_version 927728 (0.0007) [2023-12-27 12:04:29,666][20381] Updated weights for policy 1, policy_version 926911 (0.0006) [2023-12-27 12:04:29,725][20381] Updated weights for policy 1, policy_version 926921 (0.0005) [2023-12-27 12:04:29,786][20381] Updated weights for policy 1, policy_version 926931 (0.0005) [2023-12-27 12:04:29,800][20347] Updated weights for policy 0, policy_version 927738 (0.0007) [2023-12-27 12:04:29,863][20347] Updated weights for policy 0, policy_version 927748 (0.0006) [2023-12-27 12:04:29,916][20347] Updated weights for policy 0, policy_version 927758 (0.0007) [2023-12-27 12:04:30,394][20381] Updated weights for policy 1, policy_version 926941 (0.0008) [2023-12-27 12:04:30,453][20381] Updated weights for policy 1, policy_version 926951 (0.0010) [2023-12-27 12:04:30,509][20381] Updated weights for policy 1, policy_version 926961 (0.0011) [2023-12-27 12:04:30,561][20347] Updated weights for policy 0, policy_version 927768 (0.0006) [2023-12-27 12:04:30,616][20347] Updated weights for policy 0, policy_version 927778 (0.0007) [2023-12-27 12:04:30,672][20347] Updated weights for policy 0, policy_version 927788 (0.0007) [2023-12-27 12:04:31,097][20381] Updated weights for policy 1, policy_version 926971 (0.0010) [2023-12-27 12:04:31,162][20381] Updated weights for policy 1, policy_version 926981 (0.0007) [2023-12-27 12:04:31,217][20381] Updated weights for policy 1, policy_version 926991 (0.0008) [2023-12-27 12:04:31,253][20347] Updated weights for policy 0, policy_version 927798 (0.0006) [2023-12-27 12:04:31,306][20347] Updated weights for policy 0, policy_version 927808 (0.0007) [2023-12-27 12:04:31,385][20347] Updated weights for policy 0, policy_version 927818 (0.0007) [2023-12-27 12:04:31,948][20381] Updated weights for policy 1, policy_version 927001 (0.0007) [2023-12-27 12:04:31,997][20347] Updated weights for policy 0, policy_version 927828 (0.0006) [2023-12-27 12:04:32,002][20381] Updated weights for policy 1, policy_version 927011 (0.0009) [2023-12-27 12:04:32,051][20381] Updated weights for policy 1, policy_version 927021 (0.0008) [2023-12-27 12:04:32,052][20347] Updated weights for policy 0, policy_version 927838 (0.0007) [2023-12-27 12:04:32,104][20381] Updated weights for policy 1, policy_version 927031 (0.0008) [2023-12-27 12:04:32,105][20347] Updated weights for policy 0, policy_version 927848 (0.0009) [2023-12-27 12:04:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 474914816. Throughput: 0: 10111.2, 1: 10140.9. Samples: 474921652. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:32,363][19170] Avg episode reward: [(0, '275.660'), (1, '723.620')] [2023-12-27 12:04:32,745][20347] Updated weights for policy 0, policy_version 927858 (0.0009) [2023-12-27 12:04:32,775][20381] Updated weights for policy 1, policy_version 927041 (0.0007) [2023-12-27 12:04:32,803][20347] Updated weights for policy 0, policy_version 927868 (0.0007) [2023-12-27 12:04:32,827][20381] Updated weights for policy 1, policy_version 927051 (0.0007) [2023-12-27 12:04:32,855][20347] Updated weights for policy 0, policy_version 927878 (0.0005) [2023-12-27 12:04:32,882][20381] Updated weights for policy 1, policy_version 927061 (0.0008) [2023-12-27 12:04:32,903][20347] Updated weights for policy 0, policy_version 927888 (0.0006) [2023-12-27 12:04:33,532][20381] Updated weights for policy 1, policy_version 927071 (0.0009) [2023-12-27 12:04:33,555][20347] Updated weights for policy 0, policy_version 927898 (0.0010) [2023-12-27 12:04:33,578][20381] Updated weights for policy 1, policy_version 927081 (0.0007) [2023-12-27 12:04:33,612][20347] Updated weights for policy 0, policy_version 927908 (0.0010) [2023-12-27 12:04:33,633][20381] Updated weights for policy 1, policy_version 927091 (0.0006) [2023-12-27 12:04:33,656][20347] Updated weights for policy 0, policy_version 927918 (0.0010) [2023-12-27 12:04:34,295][20347] Updated weights for policy 0, policy_version 927928 (0.0009) [2023-12-27 12:04:34,325][20381] Updated weights for policy 1, policy_version 927101 (0.0008) [2023-12-27 12:04:34,353][20347] Updated weights for policy 0, policy_version 927938 (0.0008) [2023-12-27 12:04:34,392][20381] Updated weights for policy 1, policy_version 927111 (0.0011) [2023-12-27 12:04:34,415][20347] Updated weights for policy 0, policy_version 927948 (0.0009) [2023-12-27 12:04:34,452][20381] Updated weights for policy 1, policy_version 927121 (0.0011) [2023-12-27 12:04:35,136][20381] Updated weights for policy 1, policy_version 927131 (0.0009) [2023-12-27 12:04:35,141][20347] Updated weights for policy 0, policy_version 927958 (0.0008) [2023-12-27 12:04:35,191][20347] Updated weights for policy 0, policy_version 927968 (0.0006) [2023-12-27 12:04:35,195][20381] Updated weights for policy 1, policy_version 927141 (0.0008) [2023-12-27 12:04:35,247][20347] Updated weights for policy 0, policy_version 927978 (0.0006) [2023-12-27 12:04:35,257][20381] Updated weights for policy 1, policy_version 927151 (0.0007) [2023-12-27 12:04:35,968][20381] Updated weights for policy 1, policy_version 927161 (0.0008) [2023-12-27 12:04:35,974][20347] Updated weights for policy 0, policy_version 927988 (0.0006) [2023-12-27 12:04:36,023][20381] Updated weights for policy 1, policy_version 927171 (0.0007) [2023-12-27 12:04:36,028][20347] Updated weights for policy 0, policy_version 927998 (0.0005) [2023-12-27 12:04:36,072][20381] Updated weights for policy 1, policy_version 927181 (0.0006) [2023-12-27 12:04:36,078][20347] Updated weights for policy 0, policy_version 928008 (0.0005) [2023-12-27 12:04:36,129][20381] Updated weights for policy 1, policy_version 927191 (0.0006) [2023-12-27 12:04:36,730][20347] Updated weights for policy 0, policy_version 928018 (0.0006) [2023-12-27 12:04:36,777][20347] Updated weights for policy 0, policy_version 928028 (0.0005) [2023-12-27 12:04:36,829][20347] Updated weights for policy 0, policy_version 928038 (0.0005) [2023-12-27 12:04:36,881][20347] Updated weights for policy 0, policy_version 928048 (0.0005) [2023-12-27 12:04:36,883][20381] Updated weights for policy 1, policy_version 927201 (0.0007) [2023-12-27 12:04:36,933][20381] Updated weights for policy 1, policy_version 927211 (0.0009) [2023-12-27 12:04:36,979][20381] Updated weights for policy 1, policy_version 927221 (0.0008) [2023-12-27 12:04:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 475013120. Throughput: 0: 10123.5, 1: 10169.9. Samples: 474983980. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:37,363][19170] Avg episode reward: [(0, '336.890'), (1, '614.230')] [2023-12-27 12:04:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000928048_237617152.pth... [2023-12-27 12:04:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000927224_237395968.pth... [2023-12-27 12:04:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000926040_237092864.pth [2023-12-27 12:04:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000926864_237314048.pth [2023-12-27 12:04:37,510][20347] Updated weights for policy 0, policy_version 928058 (0.0008) [2023-12-27 12:04:37,561][20347] Updated weights for policy 0, policy_version 928068 (0.0008) [2023-12-27 12:04:37,611][20347] Updated weights for policy 0, policy_version 928078 (0.0008) [2023-12-27 12:04:37,737][20381] Updated weights for policy 1, policy_version 927231 (0.0008) [2023-12-27 12:04:37,798][20381] Updated weights for policy 1, policy_version 927241 (0.0008) [2023-12-27 12:04:37,847][20381] Updated weights for policy 1, policy_version 927251 (0.0008) [2023-12-27 12:04:38,312][20347] Updated weights for policy 0, policy_version 928088 (0.0007) [2023-12-27 12:04:38,374][20347] Updated weights for policy 0, policy_version 928098 (0.0008) [2023-12-27 12:04:38,434][20347] Updated weights for policy 0, policy_version 928108 (0.0006) [2023-12-27 12:04:38,555][20381] Updated weights for policy 1, policy_version 927261 (0.0006) [2023-12-27 12:04:38,613][20381] Updated weights for policy 1, policy_version 927271 (0.0009) [2023-12-27 12:04:38,671][20381] Updated weights for policy 1, policy_version 927281 (0.0007) [2023-12-27 12:04:39,145][20347] Updated weights for policy 0, policy_version 928118 (0.0007) [2023-12-27 12:04:39,198][20347] Updated weights for policy 0, policy_version 928128 (0.0007) [2023-12-27 12:04:39,248][20381] Updated weights for policy 1, policy_version 927291 (0.0007) [2023-12-27 12:04:39,257][20347] Updated weights for policy 0, policy_version 928138 (0.0008) [2023-12-27 12:04:39,310][20381] Updated weights for policy 1, policy_version 927301 (0.0009) [2023-12-27 12:04:39,379][20381] Updated weights for policy 1, policy_version 927311 (0.0011) [2023-12-27 12:04:39,964][20347] Updated weights for policy 0, policy_version 928148 (0.0007) [2023-12-27 12:04:40,024][20347] Updated weights for policy 0, policy_version 928158 (0.0008) [2023-12-27 12:04:40,090][20381] Updated weights for policy 1, policy_version 927321 (0.0008) [2023-12-27 12:04:40,092][20347] Updated weights for policy 0, policy_version 928168 (0.0008) [2023-12-27 12:04:40,152][20381] Updated weights for policy 1, policy_version 927331 (0.0006) [2023-12-27 12:04:40,211][20381] Updated weights for policy 1, policy_version 927341 (0.0006) [2023-12-27 12:04:40,265][20381] Updated weights for policy 1, policy_version 927351 (0.0005) [2023-12-27 12:04:40,830][20347] Updated weights for policy 0, policy_version 928178 (0.0006) [2023-12-27 12:04:40,889][20381] Updated weights for policy 1, policy_version 927361 (0.0006) [2023-12-27 12:04:40,891][20347] Updated weights for policy 0, policy_version 928188 (0.0006) [2023-12-27 12:04:40,942][20381] Updated weights for policy 1, policy_version 927371 (0.0007) [2023-12-27 12:04:40,953][20347] Updated weights for policy 0, policy_version 928198 (0.0006) [2023-12-27 12:04:40,998][20381] Updated weights for policy 1, policy_version 927381 (0.0007) [2023-12-27 12:04:41,019][20347] Updated weights for policy 0, policy_version 928208 (0.0007) [2023-12-27 12:04:41,772][20347] Updated weights for policy 0, policy_version 928218 (0.0008) [2023-12-27 12:04:41,797][20381] Updated weights for policy 1, policy_version 927391 (0.0007) [2023-12-27 12:04:41,835][20347] Updated weights for policy 0, policy_version 928228 (0.0008) [2023-12-27 12:04:41,863][20381] Updated weights for policy 1, policy_version 927401 (0.0007) [2023-12-27 12:04:41,891][20347] Updated weights for policy 0, policy_version 928238 (0.0008) [2023-12-27 12:04:41,925][20381] Updated weights for policy 1, policy_version 927411 (0.0007) [2023-12-27 12:04:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 475111424. Throughput: 0: 10100.9, 1: 10176.5. Samples: 475104224. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:42,363][19170] Avg episode reward: [(0, '219.730'), (1, '612.040')] [2023-12-27 12:04:42,574][20347] Updated weights for policy 0, policy_version 928248 (0.0008) [2023-12-27 12:04:42,632][20381] Updated weights for policy 1, policy_version 927421 (0.0006) [2023-12-27 12:04:42,633][20347] Updated weights for policy 0, policy_version 928258 (0.0008) [2023-12-27 12:04:42,688][20347] Updated weights for policy 0, policy_version 928268 (0.0008) [2023-12-27 12:04:42,696][20381] Updated weights for policy 1, policy_version 927431 (0.0006) [2023-12-27 12:04:42,749][20381] Updated weights for policy 1, policy_version 927441 (0.0006) [2023-12-27 12:04:43,360][20347] Updated weights for policy 0, policy_version 928278 (0.0008) [2023-12-27 12:04:43,384][20381] Updated weights for policy 1, policy_version 927451 (0.0006) [2023-12-27 12:04:43,419][20347] Updated weights for policy 0, policy_version 928288 (0.0005) [2023-12-27 12:04:43,436][20381] Updated weights for policy 1, policy_version 927461 (0.0005) [2023-12-27 12:04:43,468][20347] Updated weights for policy 0, policy_version 928298 (0.0006) [2023-12-27 12:04:43,490][20381] Updated weights for policy 1, policy_version 927471 (0.0006) [2023-12-27 12:04:44,039][20381] Updated weights for policy 1, policy_version 927481 (0.0005) [2023-12-27 12:04:44,098][20381] Updated weights for policy 1, policy_version 927491 (0.0006) [2023-12-27 12:04:44,159][20381] Updated weights for policy 1, policy_version 927501 (0.0008) [2023-12-27 12:04:44,161][20347] Updated weights for policy 0, policy_version 928308 (0.0006) [2023-12-27 12:04:44,211][20347] Updated weights for policy 0, policy_version 928318 (0.0005) [2023-12-27 12:04:44,220][20381] Updated weights for policy 1, policy_version 927511 (0.0007) [2023-12-27 12:04:44,270][20347] Updated weights for policy 0, policy_version 928328 (0.0005) [2023-12-27 12:04:44,793][20381] Updated weights for policy 1, policy_version 927521 (0.0007) [2023-12-27 12:04:44,812][20347] Updated weights for policy 0, policy_version 928338 (0.0006) [2023-12-27 12:04:44,855][20381] Updated weights for policy 1, policy_version 927531 (0.0007) [2023-12-27 12:04:44,875][20347] Updated weights for policy 0, policy_version 928348 (0.0007) [2023-12-27 12:04:44,919][20381] Updated weights for policy 1, policy_version 927541 (0.0007) [2023-12-27 12:04:44,945][20347] Updated weights for policy 0, policy_version 928358 (0.0006) [2023-12-27 12:04:45,010][20347] Updated weights for policy 0, policy_version 928368 (0.0007) [2023-12-27 12:04:45,605][20381] Updated weights for policy 1, policy_version 927551 (0.0009) [2023-12-27 12:04:45,665][20381] Updated weights for policy 1, policy_version 927561 (0.0006) [2023-12-27 12:04:45,680][20347] Updated weights for policy 0, policy_version 928378 (0.0008) [2023-12-27 12:04:45,720][20381] Updated weights for policy 1, policy_version 927571 (0.0005) [2023-12-27 12:04:45,735][20347] Updated weights for policy 0, policy_version 928388 (0.0007) [2023-12-27 12:04:45,802][20347] Updated weights for policy 0, policy_version 928398 (0.0008) [2023-12-27 12:04:46,329][20381] Updated weights for policy 1, policy_version 927581 (0.0007) [2023-12-27 12:04:46,390][20381] Updated weights for policy 1, policy_version 927591 (0.0008) [2023-12-27 12:04:46,447][20381] Updated weights for policy 1, policy_version 927601 (0.0008) [2023-12-27 12:04:46,514][20347] Updated weights for policy 0, policy_version 928408 (0.0007) [2023-12-27 12:04:46,574][20347] Updated weights for policy 0, policy_version 928418 (0.0009) [2023-12-27 12:04:46,630][20347] Updated weights for policy 0, policy_version 928428 (0.0007) [2023-12-27 12:04:47,120][20381] Updated weights for policy 1, policy_version 927611 (0.0007) [2023-12-27 12:04:47,171][20381] Updated weights for policy 1, policy_version 927621 (0.0006) [2023-12-27 12:04:47,228][20381] Updated weights for policy 1, policy_version 927631 (0.0006) [2023-12-27 12:04:47,301][20347] Updated weights for policy 0, policy_version 928438 (0.0008) [2023-12-27 12:04:47,350][20347] Updated weights for policy 0, policy_version 928448 (0.0009) [2023-12-27 12:04:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 475217920. Throughput: 0: 10226.6, 1: 10321.0. Samples: 475229200. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:47,364][19170] Avg episode reward: [(0, '163.170'), (1, '538.130')] [2023-12-27 12:04:47,395][20347] Updated weights for policy 0, policy_version 928458 (0.0005) [2023-12-27 12:04:47,888][20381] Updated weights for policy 1, policy_version 927641 (0.0005) [2023-12-27 12:04:47,956][20381] Updated weights for policy 1, policy_version 927651 (0.0006) [2023-12-27 12:04:48,006][20347] Updated weights for policy 0, policy_version 928468 (0.0005) [2023-12-27 12:04:48,016][20381] Updated weights for policy 1, policy_version 927661 (0.0006) [2023-12-27 12:04:48,054][20347] Updated weights for policy 0, policy_version 928478 (0.0006) [2023-12-27 12:04:48,078][20381] Updated weights for policy 1, policy_version 927671 (0.0006) [2023-12-27 12:04:48,107][20347] Updated weights for policy 0, policy_version 928488 (0.0007) [2023-12-27 12:04:48,691][20381] Updated weights for policy 1, policy_version 927681 (0.0010) [2023-12-27 12:04:48,749][20381] Updated weights for policy 1, policy_version 927691 (0.0010) [2023-12-27 12:04:48,807][20381] Updated weights for policy 1, policy_version 927701 (0.0010) [2023-12-27 12:04:48,812][20347] Updated weights for policy 0, policy_version 928498 (0.0007) [2023-12-27 12:04:48,868][20347] Updated weights for policy 0, policy_version 928508 (0.0007) [2023-12-27 12:04:48,925][20347] Updated weights for policy 0, policy_version 928518 (0.0008) [2023-12-27 12:04:48,986][20347] Updated weights for policy 0, policy_version 928528 (0.0008) [2023-12-27 12:04:49,481][20381] Updated weights for policy 1, policy_version 927711 (0.0010) [2023-12-27 12:04:49,542][20381] Updated weights for policy 1, policy_version 927721 (0.0010) [2023-12-27 12:04:49,597][20381] Updated weights for policy 1, policy_version 927731 (0.0010) [2023-12-27 12:04:49,725][20347] Updated weights for policy 0, policy_version 928538 (0.0007) [2023-12-27 12:04:49,785][20347] Updated weights for policy 0, policy_version 928548 (0.0005) [2023-12-27 12:04:49,850][20347] Updated weights for policy 0, policy_version 928558 (0.0009) [2023-12-27 12:04:50,322][20381] Updated weights for policy 1, policy_version 927741 (0.0007) [2023-12-27 12:04:50,381][20381] Updated weights for policy 1, policy_version 927751 (0.0006) [2023-12-27 12:04:50,439][20381] Updated weights for policy 1, policy_version 927761 (0.0005) [2023-12-27 12:04:50,523][20347] Updated weights for policy 0, policy_version 928568 (0.0006) [2023-12-27 12:04:50,569][20347] Updated weights for policy 0, policy_version 928578 (0.0006) [2023-12-27 12:04:50,633][20347] Updated weights for policy 0, policy_version 928588 (0.0008) [2023-12-27 12:04:51,081][20381] Updated weights for policy 1, policy_version 927771 (0.0005) [2023-12-27 12:04:51,147][20381] Updated weights for policy 1, policy_version 927781 (0.0006) [2023-12-27 12:04:51,201][20381] Updated weights for policy 1, policy_version 927791 (0.0006) [2023-12-27 12:04:51,342][20347] Updated weights for policy 0, policy_version 928598 (0.0007) [2023-12-27 12:04:51,405][20347] Updated weights for policy 0, policy_version 928608 (0.0007) [2023-12-27 12:04:51,456][20347] Updated weights for policy 0, policy_version 928618 (0.0007) [2023-12-27 12:04:51,907][20381] Updated weights for policy 1, policy_version 927801 (0.0008) [2023-12-27 12:04:51,970][20381] Updated weights for policy 1, policy_version 927811 (0.0010) [2023-12-27 12:04:52,025][20381] Updated weights for policy 1, policy_version 927821 (0.0008) [2023-12-27 12:04:52,090][20381] Updated weights for policy 1, policy_version 927831 (0.0008) [2023-12-27 12:04:52,178][20347] Updated weights for policy 0, policy_version 928628 (0.0008) [2023-12-27 12:04:52,239][20347] Updated weights for policy 0, policy_version 928638 (0.0010) [2023-12-27 12:04:52,302][20347] Updated weights for policy 0, policy_version 928648 (0.0009) [2023-12-27 12:04:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 475324416. Throughput: 0: 10223.1, 1: 10280.8. Samples: 475290624. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:52,363][19170] Avg episode reward: [(0, '292.210'), (1, '535.480')] [2023-12-27 12:04:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000927832_237551616.pth... [2023-12-27 12:04:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000926616_237240320.pth [2023-12-27 12:04:52,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000928656_237772800.pth... [2023-12-27 12:04:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000927440_237461504.pth [2023-12-27 12:04:52,760][20381] Updated weights for policy 1, policy_version 927841 (0.0006) [2023-12-27 12:04:52,821][20381] Updated weights for policy 1, policy_version 927851 (0.0006) [2023-12-27 12:04:52,888][20381] Updated weights for policy 1, policy_version 927861 (0.0008) [2023-12-27 12:04:52,975][20347] Updated weights for policy 0, policy_version 928658 (0.0010) [2023-12-27 12:04:53,032][20347] Updated weights for policy 0, policy_version 928668 (0.0008) [2023-12-27 12:04:53,091][20347] Updated weights for policy 0, policy_version 928678 (0.0008) [2023-12-27 12:04:53,143][20347] Updated weights for policy 0, policy_version 928688 (0.0006) [2023-12-27 12:04:53,502][20381] Updated weights for policy 1, policy_version 927871 (0.0006) [2023-12-27 12:04:53,550][20381] Updated weights for policy 1, policy_version 927881 (0.0005) [2023-12-27 12:04:53,607][20381] Updated weights for policy 1, policy_version 927891 (0.0005) [2023-12-27 12:04:53,776][20347] Updated weights for policy 0, policy_version 928698 (0.0005) [2023-12-27 12:04:53,829][20347] Updated weights for policy 0, policy_version 928708 (0.0005) [2023-12-27 12:04:53,891][20347] Updated weights for policy 0, policy_version 928718 (0.0007) [2023-12-27 12:04:54,153][20381] Updated weights for policy 1, policy_version 927901 (0.0006) [2023-12-27 12:04:54,206][20381] Updated weights for policy 1, policy_version 927911 (0.0005) [2023-12-27 12:04:54,260][20381] Updated weights for policy 1, policy_version 927921 (0.0009) [2023-12-27 12:04:54,432][20347] Updated weights for policy 0, policy_version 928728 (0.0005) [2023-12-27 12:04:54,489][20347] Updated weights for policy 0, policy_version 928738 (0.0006) [2023-12-27 12:04:54,541][20347] Updated weights for policy 0, policy_version 928748 (0.0006) [2023-12-27 12:04:54,942][20381] Updated weights for policy 1, policy_version 927931 (0.0007) [2023-12-27 12:04:54,993][20381] Updated weights for policy 1, policy_version 927941 (0.0008) [2023-12-27 12:04:55,047][20381] Updated weights for policy 1, policy_version 927951 (0.0007) [2023-12-27 12:04:55,100][20347] Updated weights for policy 0, policy_version 928758 (0.0005) [2023-12-27 12:04:55,152][20347] Updated weights for policy 0, policy_version 928768 (0.0005) [2023-12-27 12:04:55,205][20347] Updated weights for policy 0, policy_version 928778 (0.0006) [2023-12-27 12:04:55,784][20347] Updated weights for policy 0, policy_version 928788 (0.0007) [2023-12-27 12:04:55,786][20381] Updated weights for policy 1, policy_version 927961 (0.0008) [2023-12-27 12:04:55,842][20381] Updated weights for policy 1, policy_version 927971 (0.0006) [2023-12-27 12:04:55,844][20347] Updated weights for policy 0, policy_version 928798 (0.0005) [2023-12-27 12:04:55,897][20381] Updated weights for policy 1, policy_version 927981 (0.0006) [2023-12-27 12:04:55,899][20347] Updated weights for policy 0, policy_version 928808 (0.0006) [2023-12-27 12:04:55,956][20381] Updated weights for policy 1, policy_version 927991 (0.0007) [2023-12-27 12:04:56,558][20347] Updated weights for policy 0, policy_version 928818 (0.0006) [2023-12-27 12:04:56,613][20347] Updated weights for policy 0, policy_version 928828 (0.0005) [2023-12-27 12:04:56,642][20381] Updated weights for policy 1, policy_version 928001 (0.0007) [2023-12-27 12:04:56,669][20347] Updated weights for policy 0, policy_version 928838 (0.0007) [2023-12-27 12:04:56,696][20381] Updated weights for policy 1, policy_version 928011 (0.0005) [2023-12-27 12:04:56,720][20347] Updated weights for policy 0, policy_version 928848 (0.0010) [2023-12-27 12:04:56,745][20381] Updated weights for policy 1, policy_version 928021 (0.0006) [2023-12-27 12:04:57,354][20347] Updated weights for policy 0, policy_version 928858 (0.0010) [2023-12-27 12:04:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 475422720. Throughput: 0: 10342.8, 1: 10362.1. Samples: 475417732. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:04:57,363][19170] Avg episode reward: [(0, '360.100'), (1, '649.090')] [2023-12-27 12:04:57,380][20381] Updated weights for policy 1, policy_version 928031 (0.0006) [2023-12-27 12:04:57,403][20347] Updated weights for policy 0, policy_version 928868 (0.0010) [2023-12-27 12:04:57,427][20381] Updated weights for policy 1, policy_version 928041 (0.0005) [2023-12-27 12:04:57,451][20347] Updated weights for policy 0, policy_version 928878 (0.0010) [2023-12-27 12:04:57,479][20381] Updated weights for policy 1, policy_version 928051 (0.0005) [2023-12-27 12:04:58,118][20347] Updated weights for policy 0, policy_version 928888 (0.0008) [2023-12-27 12:04:58,138][20381] Updated weights for policy 1, policy_version 928061 (0.0006) [2023-12-27 12:04:58,170][20347] Updated weights for policy 0, policy_version 928898 (0.0006) [2023-12-27 12:04:58,195][20381] Updated weights for policy 1, policy_version 928071 (0.0006) [2023-12-27 12:04:58,229][20347] Updated weights for policy 0, policy_version 928908 (0.0008) [2023-12-27 12:04:58,251][20381] Updated weights for policy 1, policy_version 928081 (0.0007) [2023-12-27 12:04:58,996][20347] Updated weights for policy 0, policy_version 928918 (0.0007) [2023-12-27 12:04:59,041][20381] Updated weights for policy 1, policy_version 928091 (0.0007) [2023-12-27 12:04:59,056][20347] Updated weights for policy 0, policy_version 928928 (0.0008) [2023-12-27 12:04:59,102][20381] Updated weights for policy 1, policy_version 928101 (0.0008) [2023-12-27 12:04:59,116][20347] Updated weights for policy 0, policy_version 928938 (0.0008) [2023-12-27 12:04:59,164][20381] Updated weights for policy 1, policy_version 928111 (0.0006) [2023-12-27 12:04:59,886][20347] Updated weights for policy 0, policy_version 928948 (0.0008) [2023-12-27 12:04:59,925][20381] Updated weights for policy 1, policy_version 928121 (0.0008) [2023-12-27 12:04:59,948][20347] Updated weights for policy 0, policy_version 928958 (0.0008) [2023-12-27 12:04:59,988][20381] Updated weights for policy 1, policy_version 928131 (0.0007) [2023-12-27 12:05:00,008][20347] Updated weights for policy 0, policy_version 928968 (0.0008) [2023-12-27 12:05:00,050][20381] Updated weights for policy 1, policy_version 928141 (0.0006) [2023-12-27 12:05:00,105][20381] Updated weights for policy 1, policy_version 928151 (0.0008) [2023-12-27 12:05:00,668][20347] Updated weights for policy 0, policy_version 928978 (0.0007) [2023-12-27 12:05:00,726][20347] Updated weights for policy 0, policy_version 928988 (0.0010) [2023-12-27 12:05:00,731][20381] Updated weights for policy 1, policy_version 928161 (0.0005) [2023-12-27 12:05:00,781][20347] Updated weights for policy 0, policy_version 928998 (0.0010) [2023-12-27 12:05:00,787][20381] Updated weights for policy 1, policy_version 928171 (0.0005) [2023-12-27 12:05:00,835][20347] Updated weights for policy 0, policy_version 929008 (0.0010) [2023-12-27 12:05:00,842][20381] Updated weights for policy 1, policy_version 928181 (0.0006) [2023-12-27 12:05:01,488][20381] Updated weights for policy 1, policy_version 928191 (0.0007) [2023-12-27 12:05:01,537][20347] Updated weights for policy 0, policy_version 929018 (0.0010) [2023-12-27 12:05:01,547][20381] Updated weights for policy 1, policy_version 928201 (0.0009) [2023-12-27 12:05:01,589][20347] Updated weights for policy 0, policy_version 929028 (0.0010) [2023-12-27 12:05:01,610][20381] Updated weights for policy 1, policy_version 928211 (0.0006) [2023-12-27 12:05:01,660][20347] Updated weights for policy 0, policy_version 929039 (0.0010) [2023-12-27 12:05:02,286][20347] Updated weights for policy 0, policy_version 929049 (0.0008) [2023-12-27 12:05:02,345][20347] Updated weights for policy 0, policy_version 929059 (0.0008) [2023-12-27 12:05:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 475521024. Throughput: 0: 10340.6, 1: 10301.2. Samples: 475537340. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:02,363][19170] Avg episode reward: [(0, '392.140'), (1, '704.670')] [2023-12-27 12:05:02,377][20381] Updated weights for policy 1, policy_version 928221 (0.0008) [2023-12-27 12:05:02,409][20347] Updated weights for policy 0, policy_version 929069 (0.0007) [2023-12-27 12:05:02,433][20381] Updated weights for policy 1, policy_version 928231 (0.0006) [2023-12-27 12:05:02,489][20381] Updated weights for policy 1, policy_version 928241 (0.0008) [2023-12-27 12:05:03,022][20347] Updated weights for policy 0, policy_version 929079 (0.0005) [2023-12-27 12:05:03,070][20347] Updated weights for policy 0, policy_version 929089 (0.0005) [2023-12-27 12:05:03,117][20347] Updated weights for policy 0, policy_version 929099 (0.0010) [2023-12-27 12:05:03,244][20381] Updated weights for policy 1, policy_version 928251 (0.0008) [2023-12-27 12:05:03,291][20381] Updated weights for policy 1, policy_version 928261 (0.0007) [2023-12-27 12:05:03,344][20381] Updated weights for policy 1, policy_version 928271 (0.0006) [2023-12-27 12:05:03,784][20347] Updated weights for policy 0, policy_version 929109 (0.0010) [2023-12-27 12:05:03,841][20347] Updated weights for policy 0, policy_version 929119 (0.0010) [2023-12-27 12:05:03,902][20347] Updated weights for policy 0, policy_version 929129 (0.0007) [2023-12-27 12:05:04,023][20381] Updated weights for policy 1, policy_version 928281 (0.0005) [2023-12-27 12:05:04,089][20381] Updated weights for policy 1, policy_version 928291 (0.0008) [2023-12-27 12:05:04,157][20381] Updated weights for policy 1, policy_version 928301 (0.0008) [2023-12-27 12:05:04,221][20381] Updated weights for policy 1, policy_version 928311 (0.0007) [2023-12-27 12:05:04,586][20347] Updated weights for policy 0, policy_version 929139 (0.0007) [2023-12-27 12:05:04,642][20347] Updated weights for policy 0, policy_version 929149 (0.0005) [2023-12-27 12:05:04,702][20347] Updated weights for policy 0, policy_version 929159 (0.0006) [2023-12-27 12:05:04,925][20381] Updated weights for policy 1, policy_version 928321 (0.0006) [2023-12-27 12:05:04,982][20381] Updated weights for policy 1, policy_version 928331 (0.0006) [2023-12-27 12:05:05,047][20381] Updated weights for policy 1, policy_version 928341 (0.0008) [2023-12-27 12:05:05,338][20347] Updated weights for policy 0, policy_version 929169 (0.0010) [2023-12-27 12:05:05,394][20347] Updated weights for policy 0, policy_version 929179 (0.0008) [2023-12-27 12:05:05,459][20347] Updated weights for policy 0, policy_version 929189 (0.0008) [2023-12-27 12:05:05,519][20347] Updated weights for policy 0, policy_version 929199 (0.0010) [2023-12-27 12:05:05,712][20381] Updated weights for policy 1, policy_version 928351 (0.0007) [2023-12-27 12:05:05,775][20381] Updated weights for policy 1, policy_version 928361 (0.0006) [2023-12-27 12:05:05,834][20381] Updated weights for policy 1, policy_version 928371 (0.0006) [2023-12-27 12:05:06,210][20347] Updated weights for policy 0, policy_version 929209 (0.0008) [2023-12-27 12:05:06,279][20347] Updated weights for policy 0, policy_version 929219 (0.0008) [2023-12-27 12:05:06,343][20347] Updated weights for policy 0, policy_version 929229 (0.0008) [2023-12-27 12:05:06,460][20381] Updated weights for policy 1, policy_version 928381 (0.0008) [2023-12-27 12:05:06,510][20381] Updated weights for policy 1, policy_version 928391 (0.0008) [2023-12-27 12:05:06,567][20381] Updated weights for policy 1, policy_version 928401 (0.0006) [2023-12-27 12:05:07,081][20347] Updated weights for policy 0, policy_version 929239 (0.0007) [2023-12-27 12:05:07,132][20347] Updated weights for policy 0, policy_version 929249 (0.0006) [2023-12-27 12:05:07,180][20347] Updated weights for policy 0, policy_version 929259 (0.0007) [2023-12-27 12:05:07,254][20381] Updated weights for policy 1, policy_version 928411 (0.0010) [2023-12-27 12:05:07,304][20381] Updated weights for policy 1, policy_version 928421 (0.0008) [2023-12-27 12:05:07,356][20381] Updated weights for policy 1, policy_version 928431 (0.0005) [2023-12-27 12:05:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 475627520. Throughput: 0: 10394.4, 1: 10251.6. Samples: 475598796. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:07,363][19170] Avg episode reward: [(0, '344.150'), (1, '735.010')] [2023-12-27 12:05:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000929264_237928448.pth... [2023-12-27 12:05:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000928048_237617152.pth [2023-12-27 12:05:07,402][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000928440_237707264.pth... [2023-12-27 12:05:07,406][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000927224_237395968.pth [2023-12-27 12:05:07,905][20347] Updated weights for policy 0, policy_version 929269 (0.0007) [2023-12-27 12:05:07,973][20347] Updated weights for policy 0, policy_version 929279 (0.0006) [2023-12-27 12:05:07,991][20381] Updated weights for policy 1, policy_version 928441 (0.0006) [2023-12-27 12:05:08,032][20347] Updated weights for policy 0, policy_version 929289 (0.0006) [2023-12-27 12:05:08,046][20381] Updated weights for policy 1, policy_version 928451 (0.0008) [2023-12-27 12:05:08,104][20381] Updated weights for policy 1, policy_version 928461 (0.0008) [2023-12-27 12:05:08,166][20381] Updated weights for policy 1, policy_version 928471 (0.0007) [2023-12-27 12:05:08,587][20347] Updated weights for policy 0, policy_version 929299 (0.0005) [2023-12-27 12:05:08,638][20347] Updated weights for policy 0, policy_version 929309 (0.0007) [2023-12-27 12:05:08,700][20347] Updated weights for policy 0, policy_version 929319 (0.0008) [2023-12-27 12:05:08,864][20381] Updated weights for policy 1, policy_version 928481 (0.0008) [2023-12-27 12:05:08,920][20381] Updated weights for policy 1, policy_version 928491 (0.0007) [2023-12-27 12:05:08,979][20381] Updated weights for policy 1, policy_version 928501 (0.0007) [2023-12-27 12:05:09,419][20347] Updated weights for policy 0, policy_version 929329 (0.0008) [2023-12-27 12:05:09,475][20347] Updated weights for policy 0, policy_version 929339 (0.0008) [2023-12-27 12:05:09,538][20347] Updated weights for policy 0, policy_version 929349 (0.0008) [2023-12-27 12:05:09,591][20347] Updated weights for policy 0, policy_version 929359 (0.0008) [2023-12-27 12:05:09,697][20381] Updated weights for policy 1, policy_version 928511 (0.0007) [2023-12-27 12:05:09,751][20381] Updated weights for policy 1, policy_version 928521 (0.0007) [2023-12-27 12:05:09,807][20381] Updated weights for policy 1, policy_version 928531 (0.0007) [2023-12-27 12:05:10,391][20347] Updated weights for policy 0, policy_version 929369 (0.0008) [2023-12-27 12:05:10,448][20347] Updated weights for policy 0, policy_version 929379 (0.0008) [2023-12-27 12:05:10,508][20347] Updated weights for policy 0, policy_version 929389 (0.0006) [2023-12-27 12:05:10,513][20381] Updated weights for policy 1, policy_version 928541 (0.0009) [2023-12-27 12:05:10,563][20381] Updated weights for policy 1, policy_version 928551 (0.0006) [2023-12-27 12:05:10,618][20381] Updated weights for policy 1, policy_version 928561 (0.0005) [2023-12-27 12:05:11,180][20347] Updated weights for policy 0, policy_version 929399 (0.0008) [2023-12-27 12:05:11,243][20347] Updated weights for policy 0, policy_version 929409 (0.0008) [2023-12-27 12:05:11,307][20347] Updated weights for policy 0, policy_version 929419 (0.0008) [2023-12-27 12:05:11,374][20381] Updated weights for policy 1, policy_version 928571 (0.0007) [2023-12-27 12:05:11,432][20381] Updated weights for policy 1, policy_version 928581 (0.0008) [2023-12-27 12:05:11,495][20381] Updated weights for policy 1, policy_version 928591 (0.0009) [2023-12-27 12:05:12,050][20347] Updated weights for policy 0, policy_version 929429 (0.0007) [2023-12-27 12:05:12,114][20347] Updated weights for policy 0, policy_version 929439 (0.0008) [2023-12-27 12:05:12,175][20347] Updated weights for policy 0, policy_version 929449 (0.0007) [2023-12-27 12:05:12,224][20381] Updated weights for policy 1, policy_version 928601 (0.0011) [2023-12-27 12:05:12,292][20381] Updated weights for policy 1, policy_version 928611 (0.0011) [2023-12-27 12:05:12,355][20381] Updated weights for policy 1, policy_version 928621 (0.0008) [2023-12-27 12:05:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 475725824. Throughput: 0: 10321.1, 1: 10242.2. Samples: 475718968. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:12,363][19170] Avg episode reward: [(0, '440.670'), (1, '736.330')] [2023-12-27 12:05:12,420][20381] Updated weights for policy 1, policy_version 928631 (0.0011) [2023-12-27 12:05:12,883][20347] Updated weights for policy 0, policy_version 929459 (0.0007) [2023-12-27 12:05:12,938][20347] Updated weights for policy 0, policy_version 929469 (0.0010) [2023-12-27 12:05:12,991][20347] Updated weights for policy 0, policy_version 929479 (0.0009) [2023-12-27 12:05:13,072][20381] Updated weights for policy 1, policy_version 928641 (0.0006) [2023-12-27 12:05:13,133][20381] Updated weights for policy 1, policy_version 928651 (0.0008) [2023-12-27 12:05:13,200][20381] Updated weights for policy 1, policy_version 928661 (0.0007) [2023-12-27 12:05:13,695][20347] Updated weights for policy 0, policy_version 929489 (0.0007) [2023-12-27 12:05:13,747][20347] Updated weights for policy 0, policy_version 929499 (0.0008) [2023-12-27 12:05:13,798][20347] Updated weights for policy 0, policy_version 929509 (0.0007) [2023-12-27 12:05:13,846][20381] Updated weights for policy 1, policy_version 928671 (0.0009) [2023-12-27 12:05:13,849][20347] Updated weights for policy 0, policy_version 929519 (0.0007) [2023-12-27 12:05:13,907][20381] Updated weights for policy 1, policy_version 928681 (0.0008) [2023-12-27 12:05:13,965][20381] Updated weights for policy 1, policy_version 928691 (0.0008) [2023-12-27 12:05:14,559][20347] Updated weights for policy 0, policy_version 929529 (0.0007) [2023-12-27 12:05:14,579][20381] Updated weights for policy 1, policy_version 928701 (0.0009) [2023-12-27 12:05:14,614][20347] Updated weights for policy 0, policy_version 929539 (0.0005) [2023-12-27 12:05:14,638][20381] Updated weights for policy 1, policy_version 928711 (0.0011) [2023-12-27 12:05:14,662][20347] Updated weights for policy 0, policy_version 929549 (0.0005) [2023-12-27 12:05:14,690][20381] Updated weights for policy 1, policy_version 928721 (0.0010) [2023-12-27 12:05:15,395][20347] Updated weights for policy 0, policy_version 929559 (0.0006) [2023-12-27 12:05:15,401][20381] Updated weights for policy 1, policy_version 928731 (0.0010) [2023-12-27 12:05:15,443][20347] Updated weights for policy 0, policy_version 929569 (0.0005) [2023-12-27 12:05:15,456][20381] Updated weights for policy 1, policy_version 928741 (0.0011) [2023-12-27 12:05:15,492][20347] Updated weights for policy 0, policy_version 929579 (0.0005) [2023-12-27 12:05:15,508][20381] Updated weights for policy 1, policy_version 928751 (0.0010) [2023-12-27 12:05:16,091][20347] Updated weights for policy 0, policy_version 929589 (0.0005) [2023-12-27 12:05:16,137][20347] Updated weights for policy 0, policy_version 929599 (0.0005) [2023-12-27 12:05:16,183][20347] Updated weights for policy 0, policy_version 929609 (0.0005) [2023-12-27 12:05:16,193][20381] Updated weights for policy 1, policy_version 928761 (0.0010) [2023-12-27 12:05:16,246][20381] Updated weights for policy 1, policy_version 928771 (0.0011) [2023-12-27 12:05:16,295][20381] Updated weights for policy 1, policy_version 928781 (0.0010) [2023-12-27 12:05:16,343][20381] Updated weights for policy 1, policy_version 928791 (0.0010) [2023-12-27 12:05:16,804][20347] Updated weights for policy 0, policy_version 929619 (0.0005) [2023-12-27 12:05:16,863][20347] Updated weights for policy 0, policy_version 929629 (0.0005) [2023-12-27 12:05:16,920][20347] Updated weights for policy 0, policy_version 929639 (0.0005) [2023-12-27 12:05:16,920][20381] Updated weights for policy 1, policy_version 928801 (0.0006) [2023-12-27 12:05:16,966][20381] Updated weights for policy 1, policy_version 928811 (0.0005) [2023-12-27 12:05:17,009][20381] Updated weights for policy 1, policy_version 928821 (0.0005) [2023-12-27 12:05:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 475832320. Throughput: 0: 10256.7, 1: 10216.9. Samples: 475842964. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:17,363][19170] Avg episode reward: [(0, '569.110'), (1, '750.190')] [2023-12-27 12:05:17,500][20347] Updated weights for policy 0, policy_version 929649 (0.0007) [2023-12-27 12:05:17,551][20347] Updated weights for policy 0, policy_version 929659 (0.0008) [2023-12-27 12:05:17,596][20347] Updated weights for policy 0, policy_version 929669 (0.0010) [2023-12-27 12:05:17,647][20347] Updated weights for policy 0, policy_version 929679 (0.0010) [2023-12-27 12:05:17,652][20381] Updated weights for policy 1, policy_version 928831 (0.0007) [2023-12-27 12:05:17,706][20381] Updated weights for policy 1, policy_version 928841 (0.0005) [2023-12-27 12:05:17,757][20381] Updated weights for policy 1, policy_version 928851 (0.0011) [2023-12-27 12:05:18,346][20347] Updated weights for policy 0, policy_version 929689 (0.0009) [2023-12-27 12:05:18,381][20381] Updated weights for policy 1, policy_version 928861 (0.0008) [2023-12-27 12:05:18,406][20347] Updated weights for policy 0, policy_version 929699 (0.0007) [2023-12-27 12:05:18,431][20381] Updated weights for policy 1, policy_version 928871 (0.0005) [2023-12-27 12:05:18,465][20347] Updated weights for policy 0, policy_version 929709 (0.0008) [2023-12-27 12:05:18,480][20381] Updated weights for policy 1, policy_version 928881 (0.0006) [2023-12-27 12:05:19,125][20381] Updated weights for policy 1, policy_version 928891 (0.0007) [2023-12-27 12:05:19,146][20347] Updated weights for policy 0, policy_version 929719 (0.0006) [2023-12-27 12:05:19,175][20381] Updated weights for policy 1, policy_version 928901 (0.0005) [2023-12-27 12:05:19,207][20347] Updated weights for policy 0, policy_version 929729 (0.0007) [2023-12-27 12:05:19,236][20381] Updated weights for policy 1, policy_version 928911 (0.0008) [2023-12-27 12:05:19,274][20347] Updated weights for policy 0, policy_version 929739 (0.0008) [2023-12-27 12:05:19,972][20381] Updated weights for policy 1, policy_version 928921 (0.0007) [2023-12-27 12:05:19,974][20347] Updated weights for policy 0, policy_version 929749 (0.0009) [2023-12-27 12:05:20,031][20347] Updated weights for policy 0, policy_version 929759 (0.0008) [2023-12-27 12:05:20,035][20381] Updated weights for policy 1, policy_version 928931 (0.0008) [2023-12-27 12:05:20,093][20347] Updated weights for policy 0, policy_version 929769 (0.0008) [2023-12-27 12:05:20,099][20381] Updated weights for policy 1, policy_version 928941 (0.0008) [2023-12-27 12:05:20,164][20381] Updated weights for policy 1, policy_version 928951 (0.0008) [2023-12-27 12:05:20,834][20347] Updated weights for policy 0, policy_version 929779 (0.0010) [2023-12-27 12:05:20,887][20347] Updated weights for policy 0, policy_version 929789 (0.0010) [2023-12-27 12:05:20,900][20381] Updated weights for policy 1, policy_version 928961 (0.0011) [2023-12-27 12:05:20,947][20347] Updated weights for policy 0, policy_version 929799 (0.0010) [2023-12-27 12:05:20,961][20381] Updated weights for policy 1, policy_version 928971 (0.0011) [2023-12-27 12:05:21,019][20381] Updated weights for policy 1, policy_version 928981 (0.0011) [2023-12-27 12:05:21,693][20347] Updated weights for policy 0, policy_version 929809 (0.0010) [2023-12-27 12:05:21,757][20347] Updated weights for policy 0, policy_version 929819 (0.0009) [2023-12-27 12:05:21,802][20347] Updated weights for policy 0, policy_version 929829 (0.0008) [2023-12-27 12:05:21,847][20381] Updated weights for policy 1, policy_version 928991 (0.0009) [2023-12-27 12:05:21,854][20347] Updated weights for policy 0, policy_version 929839 (0.0006) [2023-12-27 12:05:21,906][20381] Updated weights for policy 1, policy_version 929001 (0.0008) [2023-12-27 12:05:21,959][20381] Updated weights for policy 1, policy_version 929011 (0.0008) [2023-12-27 12:05:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 475930624. Throughput: 0: 10241.2, 1: 10218.4. Samples: 475904664. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:22,363][19170] Avg episode reward: [(0, '557.910'), (1, '746.250')] [2023-12-27 12:05:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000929016_237854720.pth... [2023-12-27 12:05:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000929840_238075904.pth... [2023-12-27 12:05:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000928656_237772800.pth [2023-12-27 12:05:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000927832_237551616.pth [2023-12-27 12:05:22,579][20347] Updated weights for policy 0, policy_version 929849 (0.0006) [2023-12-27 12:05:22,645][20381] Updated weights for policy 1, policy_version 929021 (0.0009) [2023-12-27 12:05:22,647][20347] Updated weights for policy 0, policy_version 929859 (0.0008) [2023-12-27 12:05:22,704][20381] Updated weights for policy 1, policy_version 929031 (0.0007) [2023-12-27 12:05:22,712][20347] Updated weights for policy 0, policy_version 929869 (0.0009) [2023-12-27 12:05:22,763][20381] Updated weights for policy 1, policy_version 929041 (0.0010) [2023-12-27 12:05:23,293][20347] Updated weights for policy 0, policy_version 929879 (0.0007) [2023-12-27 12:05:23,341][20347] Updated weights for policy 0, policy_version 929889 (0.0007) [2023-12-27 12:05:23,397][20347] Updated weights for policy 0, policy_version 929899 (0.0006) [2023-12-27 12:05:23,406][20381] Updated weights for policy 1, policy_version 929051 (0.0006) [2023-12-27 12:05:23,459][20381] Updated weights for policy 1, policy_version 929061 (0.0006) [2023-12-27 12:05:23,508][20381] Updated weights for policy 1, policy_version 929071 (0.0005) [2023-12-27 12:05:24,055][20347] Updated weights for policy 0, policy_version 929909 (0.0007) [2023-12-27 12:05:24,113][20347] Updated weights for policy 0, policy_version 929919 (0.0010) [2023-12-27 12:05:24,148][20381] Updated weights for policy 1, policy_version 929081 (0.0006) [2023-12-27 12:05:24,172][20347] Updated weights for policy 0, policy_version 929929 (0.0010) [2023-12-27 12:05:24,209][20381] Updated weights for policy 1, policy_version 929091 (0.0009) [2023-12-27 12:05:24,269][20381] Updated weights for policy 1, policy_version 929101 (0.0006) [2023-12-27 12:05:24,329][20381] Updated weights for policy 1, policy_version 929111 (0.0005) [2023-12-27 12:05:24,782][20347] Updated weights for policy 0, policy_version 929939 (0.0010) [2023-12-27 12:05:24,827][20347] Updated weights for policy 0, policy_version 929949 (0.0007) [2023-12-27 12:05:24,872][20347] Updated weights for policy 0, policy_version 929959 (0.0007) [2023-12-27 12:05:24,929][20381] Updated weights for policy 1, policy_version 929121 (0.0006) [2023-12-27 12:05:24,979][20381] Updated weights for policy 1, policy_version 929131 (0.0008) [2023-12-27 12:05:25,028][20381] Updated weights for policy 1, policy_version 929141 (0.0008) [2023-12-27 12:05:25,564][20347] Updated weights for policy 0, policy_version 929969 (0.0007) [2023-12-27 12:05:25,618][20381] Updated weights for policy 1, policy_version 929151 (0.0007) [2023-12-27 12:05:25,622][20347] Updated weights for policy 0, policy_version 929979 (0.0010) [2023-12-27 12:05:25,676][20381] Updated weights for policy 1, policy_version 929161 (0.0008) [2023-12-27 12:05:25,683][20347] Updated weights for policy 0, policy_version 929989 (0.0010) [2023-12-27 12:05:25,733][20381] Updated weights for policy 1, policy_version 929171 (0.0008) [2023-12-27 12:05:25,741][20347] Updated weights for policy 0, policy_version 929999 (0.0010) [2023-12-27 12:05:26,378][20347] Updated weights for policy 0, policy_version 930009 (0.0010) [2023-12-27 12:05:26,416][20381] Updated weights for policy 1, policy_version 929181 (0.0007) [2023-12-27 12:05:26,422][20347] Updated weights for policy 0, policy_version 930019 (0.0009) [2023-12-27 12:05:26,471][20381] Updated weights for policy 1, policy_version 929191 (0.0008) [2023-12-27 12:05:26,480][20347] Updated weights for policy 0, policy_version 930029 (0.0005) [2023-12-27 12:05:26,514][20381] Updated weights for policy 1, policy_version 929201 (0.0008) [2023-12-27 12:05:27,100][20347] Updated weights for policy 0, policy_version 930039 (0.0006) [2023-12-27 12:05:27,147][20381] Updated weights for policy 1, policy_version 929211 (0.0008) [2023-12-27 12:05:27,152][20347] Updated weights for policy 0, policy_version 930049 (0.0005) [2023-12-27 12:05:27,203][20347] Updated weights for policy 0, policy_version 930059 (0.0006) [2023-12-27 12:05:27,215][20381] Updated weights for policy 1, policy_version 929221 (0.0005) [2023-12-27 12:05:27,266][20381] Updated weights for policy 1, policy_version 929231 (0.0005) [2023-12-27 12:05:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 476045312. Throughput: 0: 10271.7, 1: 10261.1. Samples: 476028200. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:27,363][19170] Avg episode reward: [(0, '728.150'), (1, '752.390')] [2023-12-27 12:05:27,804][20347] Updated weights for policy 0, policy_version 930069 (0.0005) [2023-12-27 12:05:27,814][20381] Updated weights for policy 1, policy_version 929241 (0.0006) [2023-12-27 12:05:27,859][20347] Updated weights for policy 0, policy_version 930079 (0.0006) [2023-12-27 12:05:27,860][20381] Updated weights for policy 1, policy_version 929251 (0.0005) [2023-12-27 12:05:27,908][20381] Updated weights for policy 1, policy_version 929261 (0.0005) [2023-12-27 12:05:27,912][20347] Updated weights for policy 0, policy_version 930089 (0.0005) [2023-12-27 12:05:27,961][20381] Updated weights for policy 1, policy_version 929271 (0.0005) [2023-12-27 12:05:28,464][20347] Updated weights for policy 0, policy_version 930099 (0.0006) [2023-12-27 12:05:28,525][20347] Updated weights for policy 0, policy_version 930109 (0.0007) [2023-12-27 12:05:28,576][20347] Updated weights for policy 0, policy_version 930119 (0.0006) [2023-12-27 12:05:28,597][20381] Updated weights for policy 1, policy_version 929281 (0.0008) [2023-12-27 12:05:28,658][20381] Updated weights for policy 1, policy_version 929291 (0.0008) [2023-12-27 12:05:28,724][20381] Updated weights for policy 1, policy_version 929301 (0.0008) [2023-12-27 12:05:29,206][20347] Updated weights for policy 0, policy_version 930129 (0.0006) [2023-12-27 12:05:29,265][20347] Updated weights for policy 0, policy_version 930139 (0.0008) [2023-12-27 12:05:29,322][20347] Updated weights for policy 0, policy_version 930149 (0.0008) [2023-12-27 12:05:29,383][20381] Updated weights for policy 1, policy_version 929311 (0.0008) [2023-12-27 12:05:29,389][20347] Updated weights for policy 0, policy_version 930159 (0.0008) [2023-12-27 12:05:29,442][20381] Updated weights for policy 1, policy_version 929321 (0.0007) [2023-12-27 12:05:29,499][20381] Updated weights for policy 1, policy_version 929331 (0.0008) [2023-12-27 12:05:30,084][20347] Updated weights for policy 0, policy_version 930169 (0.0006) [2023-12-27 12:05:30,135][20347] Updated weights for policy 0, policy_version 930179 (0.0007) [2023-12-27 12:05:30,193][20347] Updated weights for policy 0, policy_version 930189 (0.0010) [2023-12-27 12:05:30,244][20381] Updated weights for policy 1, policy_version 929341 (0.0007) [2023-12-27 12:05:30,304][20381] Updated weights for policy 1, policy_version 929351 (0.0007) [2023-12-27 12:05:30,361][20381] Updated weights for policy 1, policy_version 929361 (0.0008) [2023-12-27 12:05:30,837][20347] Updated weights for policy 0, policy_version 930199 (0.0010) [2023-12-27 12:05:30,900][20347] Updated weights for policy 0, policy_version 930209 (0.0010) [2023-12-27 12:05:30,950][20347] Updated weights for policy 0, policy_version 930219 (0.0010) [2023-12-27 12:05:30,992][20381] Updated weights for policy 1, policy_version 929371 (0.0008) [2023-12-27 12:05:31,048][20381] Updated weights for policy 1, policy_version 929381 (0.0010) [2023-12-27 12:05:31,105][20381] Updated weights for policy 1, policy_version 929391 (0.0007) [2023-12-27 12:05:31,617][20347] Updated weights for policy 0, policy_version 930229 (0.0009) [2023-12-27 12:05:31,678][20347] Updated weights for policy 0, policy_version 930239 (0.0006) [2023-12-27 12:05:31,736][20347] Updated weights for policy 0, policy_version 930249 (0.0006) [2023-12-27 12:05:31,835][20381] Updated weights for policy 1, policy_version 929401 (0.0010) [2023-12-27 12:05:31,883][20381] Updated weights for policy 1, policy_version 929411 (0.0010) [2023-12-27 12:05:31,935][20381] Updated weights for policy 1, policy_version 929421 (0.0010) [2023-12-27 12:05:31,984][20381] Updated weights for policy 1, policy_version 929431 (0.0010) [2023-12-27 12:05:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 476143616. Throughput: 0: 10355.7, 1: 10241.2. Samples: 476156056. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:32,363][19170] Avg episode reward: [(0, '744.770'), (1, '722.920')] [2023-12-27 12:05:32,418][20347] Updated weights for policy 0, policy_version 930259 (0.0008) [2023-12-27 12:05:32,472][20347] Updated weights for policy 0, policy_version 930269 (0.0008) [2023-12-27 12:05:32,526][20347] Updated weights for policy 0, policy_version 930279 (0.0007) [2023-12-27 12:05:32,685][20381] Updated weights for policy 1, policy_version 929441 (0.0008) [2023-12-27 12:05:32,747][20381] Updated weights for policy 1, policy_version 929451 (0.0008) [2023-12-27 12:05:32,806][20381] Updated weights for policy 1, policy_version 929461 (0.0008) [2023-12-27 12:05:33,175][20347] Updated weights for policy 0, policy_version 930289 (0.0007) [2023-12-27 12:05:33,223][20347] Updated weights for policy 0, policy_version 930299 (0.0007) [2023-12-27 12:05:33,273][20347] Updated weights for policy 0, policy_version 930309 (0.0007) [2023-12-27 12:05:33,320][20347] Updated weights for policy 0, policy_version 930319 (0.0007) [2023-12-27 12:05:33,452][20381] Updated weights for policy 1, policy_version 929471 (0.0006) [2023-12-27 12:05:33,505][20381] Updated weights for policy 1, policy_version 929481 (0.0008) [2023-12-27 12:05:33,552][20381] Updated weights for policy 1, policy_version 929491 (0.0010) [2023-12-27 12:05:34,032][20347] Updated weights for policy 0, policy_version 930329 (0.0006) [2023-12-27 12:05:34,106][20347] Updated weights for policy 0, policy_version 930339 (0.0007) [2023-12-27 12:05:34,164][20347] Updated weights for policy 0, policy_version 930349 (0.0008) [2023-12-27 12:05:34,178][20381] Updated weights for policy 1, policy_version 929501 (0.0013) [2023-12-27 12:05:34,235][20381] Updated weights for policy 1, policy_version 929511 (0.0009) [2023-12-27 12:05:34,318][20381] Updated weights for policy 1, policy_version 929521 (0.0008) [2023-12-27 12:05:34,830][20347] Updated weights for policy 0, policy_version 930359 (0.0007) [2023-12-27 12:05:34,889][20347] Updated weights for policy 0, policy_version 930369 (0.0007) [2023-12-27 12:05:34,948][20347] Updated weights for policy 0, policy_version 930379 (0.0008) [2023-12-27 12:05:34,994][20381] Updated weights for policy 1, policy_version 929531 (0.0009) [2023-12-27 12:05:35,053][20381] Updated weights for policy 1, policy_version 929541 (0.0008) [2023-12-27 12:05:35,102][20381] Updated weights for policy 1, policy_version 929551 (0.0008) [2023-12-27 12:05:35,644][20347] Updated weights for policy 0, policy_version 930389 (0.0008) [2023-12-27 12:05:35,706][20347] Updated weights for policy 0, policy_version 930399 (0.0009) [2023-12-27 12:05:35,764][20347] Updated weights for policy 0, policy_version 930409 (0.0007) [2023-12-27 12:05:35,793][20381] Updated weights for policy 1, policy_version 929561 (0.0008) [2023-12-27 12:05:35,854][20381] Updated weights for policy 1, policy_version 929571 (0.0006) [2023-12-27 12:05:35,908][20381] Updated weights for policy 1, policy_version 929581 (0.0008) [2023-12-27 12:05:35,966][20381] Updated weights for policy 1, policy_version 929592 (0.0008) [2023-12-27 12:05:36,470][20347] Updated weights for policy 0, policy_version 930419 (0.0008) [2023-12-27 12:05:36,533][20347] Updated weights for policy 0, policy_version 930429 (0.0008) [2023-12-27 12:05:36,589][20347] Updated weights for policy 0, policy_version 930439 (0.0008) [2023-12-27 12:05:36,678][20381] Updated weights for policy 1, policy_version 929602 (0.0008) [2023-12-27 12:05:36,733][20381] Updated weights for policy 1, policy_version 929612 (0.0007) [2023-12-27 12:05:36,789][20381] Updated weights for policy 1, policy_version 929622 (0.0007) [2023-12-27 12:05:37,279][20347] Updated weights for policy 0, policy_version 930449 (0.0008) [2023-12-27 12:05:37,328][20347] Updated weights for policy 0, policy_version 930459 (0.0010) [2023-12-27 12:05:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 476241920. Throughput: 0: 10347.3, 1: 10238.6. Samples: 476216988. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:37,363][19170] Avg episode reward: [(0, '787.750'), (1, '620.360')] [2023-12-27 12:05:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000929624_238010368.pth... [2023-12-27 12:05:37,377][20347] Updated weights for policy 0, policy_version 930469 (0.0009) [2023-12-27 12:05:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000928440_237707264.pth [2023-12-27 12:05:37,429][20347] Updated weights for policy 0, policy_version 930479 (0.0006) [2023-12-27 12:05:37,433][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000930480_238239744.pth... [2023-12-27 12:05:37,437][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000929264_237928448.pth [2023-12-27 12:05:37,491][20381] Updated weights for policy 1, policy_version 929632 (0.0006) [2023-12-27 12:05:37,548][20381] Updated weights for policy 1, policy_version 929642 (0.0006) [2023-12-27 12:05:37,610][20381] Updated weights for policy 1, policy_version 929652 (0.0008) [2023-12-27 12:05:38,096][20347] Updated weights for policy 0, policy_version 930489 (0.0010) [2023-12-27 12:05:38,160][20347] Updated weights for policy 0, policy_version 930499 (0.0009) [2023-12-27 12:05:38,234][20347] Updated weights for policy 0, policy_version 930509 (0.0010) [2023-12-27 12:05:38,360][20381] Updated weights for policy 1, policy_version 929662 (0.0011) [2023-12-27 12:05:38,425][20381] Updated weights for policy 1, policy_version 929672 (0.0009) [2023-12-27 12:05:38,488][20381] Updated weights for policy 1, policy_version 929682 (0.0008) [2023-12-27 12:05:38,945][20347] Updated weights for policy 0, policy_version 930519 (0.0011) [2023-12-27 12:05:39,008][20347] Updated weights for policy 0, policy_version 930529 (0.0010) [2023-12-27 12:05:39,070][20347] Updated weights for policy 0, policy_version 930539 (0.0011) [2023-12-27 12:05:39,196][20381] Updated weights for policy 1, policy_version 929692 (0.0009) [2023-12-27 12:05:39,265][20381] Updated weights for policy 1, policy_version 929702 (0.0010) [2023-12-27 12:05:39,345][20381] Updated weights for policy 1, policy_version 929712 (0.0008) [2023-12-27 12:05:39,894][20347] Updated weights for policy 0, policy_version 930549 (0.0011) [2023-12-27 12:05:39,954][20347] Updated weights for policy 0, policy_version 930559 (0.0009) [2023-12-27 12:05:40,020][20347] Updated weights for policy 0, policy_version 930569 (0.0008) [2023-12-27 12:05:40,171][20381] Updated weights for policy 1, policy_version 929722 (0.0010) [2023-12-27 12:05:40,232][20381] Updated weights for policy 1, policy_version 929732 (0.0008) [2023-12-27 12:05:40,290][20381] Updated weights for policy 1, policy_version 929742 (0.0007) [2023-12-27 12:05:40,352][20381] Updated weights for policy 1, policy_version 929752 (0.0008) [2023-12-27 12:05:40,795][20347] Updated weights for policy 0, policy_version 930579 (0.0008) [2023-12-27 12:05:40,856][20347] Updated weights for policy 0, policy_version 930589 (0.0010) [2023-12-27 12:05:40,921][20347] Updated weights for policy 0, policy_version 930599 (0.0011) [2023-12-27 12:05:41,074][20381] Updated weights for policy 1, policy_version 929762 (0.0008) [2023-12-27 12:05:41,132][20381] Updated weights for policy 1, policy_version 929772 (0.0006) [2023-12-27 12:05:41,200][20381] Updated weights for policy 1, policy_version 929782 (0.0008) [2023-12-27 12:05:41,664][20347] Updated weights for policy 0, policy_version 930609 (0.0010) [2023-12-27 12:05:41,727][20347] Updated weights for policy 0, policy_version 930619 (0.0008) [2023-12-27 12:05:41,801][20347] Updated weights for policy 0, policy_version 930629 (0.0008) [2023-12-27 12:05:41,865][20347] Updated weights for policy 0, policy_version 930639 (0.0009) [2023-12-27 12:05:42,002][20381] Updated weights for policy 1, policy_version 929792 (0.0009) [2023-12-27 12:05:42,066][20381] Updated weights for policy 1, policy_version 929802 (0.0008) [2023-12-27 12:05:42,126][20381] Updated weights for policy 1, policy_version 929812 (0.0008) [2023-12-27 12:05:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 476340224. Throughput: 0: 10194.4, 1: 10131.1. Samples: 476332380. Policy #0 lag: (min: 25.0, avg: 40.9, max: 57.0) [2023-12-27 12:05:42,363][19170] Avg episode reward: [(0, '719.010'), (1, '679.990')] [2023-12-27 12:05:42,502][20347] Updated weights for policy 0, policy_version 930649 (0.0011) [2023-12-27 12:05:42,563][20347] Updated weights for policy 0, policy_version 930659 (0.0011) [2023-12-27 12:05:42,634][20347] Updated weights for policy 0, policy_version 930669 (0.0011) [2023-12-27 12:05:42,844][20381] Updated weights for policy 1, policy_version 929822 (0.0008) [2023-12-27 12:05:42,897][20381] Updated weights for policy 1, policy_version 929832 (0.0007) [2023-12-27 12:05:42,957][20381] Updated weights for policy 1, policy_version 929842 (0.0007) [2023-12-27 12:05:43,285][20347] Updated weights for policy 0, policy_version 930679 (0.0011) [2023-12-27 12:05:43,343][20347] Updated weights for policy 0, policy_version 930689 (0.0010) [2023-12-27 12:05:43,392][20347] Updated weights for policy 0, policy_version 930699 (0.0010) [2023-12-27 12:05:43,584][20381] Updated weights for policy 1, policy_version 929852 (0.0006) [2023-12-27 12:05:43,640][20381] Updated weights for policy 1, policy_version 929862 (0.0005) [2023-12-27 12:05:43,703][20381] Updated weights for policy 1, policy_version 929872 (0.0006) [2023-12-27 12:05:44,090][20347] Updated weights for policy 0, policy_version 930709 (0.0010) [2023-12-27 12:05:44,151][20347] Updated weights for policy 0, policy_version 930719 (0.0010) [2023-12-27 12:05:44,208][20347] Updated weights for policy 0, policy_version 930729 (0.0007) [2023-12-27 12:05:44,319][20381] Updated weights for policy 1, policy_version 929882 (0.0008) [2023-12-27 12:05:44,383][20381] Updated weights for policy 1, policy_version 929892 (0.0008) [2023-12-27 12:05:44,441][20381] Updated weights for policy 1, policy_version 929902 (0.0011) [2023-12-27 12:05:44,494][20381] Updated weights for policy 1, policy_version 929912 (0.0008) [2023-12-27 12:05:44,901][20347] Updated weights for policy 0, policy_version 930739 (0.0011) [2023-12-27 12:05:44,959][20347] Updated weights for policy 0, policy_version 930749 (0.0011) [2023-12-27 12:05:45,019][20347] Updated weights for policy 0, policy_version 930759 (0.0011) [2023-12-27 12:05:45,254][20381] Updated weights for policy 1, policy_version 929922 (0.0011) [2023-12-27 12:05:45,315][20381] Updated weights for policy 1, policy_version 929932 (0.0011) [2023-12-27 12:05:45,378][20381] Updated weights for policy 1, policy_version 929942 (0.0011) [2023-12-27 12:05:45,743][20347] Updated weights for policy 0, policy_version 930769 (0.0010) [2023-12-27 12:05:45,799][20347] Updated weights for policy 0, policy_version 930779 (0.0007) [2023-12-27 12:05:45,856][20347] Updated weights for policy 0, policy_version 930789 (0.0005) [2023-12-27 12:05:45,908][20347] Updated weights for policy 0, policy_version 930799 (0.0005) [2023-12-27 12:05:46,071][20381] Updated weights for policy 1, policy_version 929952 (0.0010) [2023-12-27 12:05:46,131][20381] Updated weights for policy 1, policy_version 929962 (0.0010) [2023-12-27 12:05:46,179][20381] Updated weights for policy 1, policy_version 929972 (0.0010) [2023-12-27 12:05:46,459][20347] Updated weights for policy 0, policy_version 930809 (0.0006) [2023-12-27 12:05:46,506][20347] Updated weights for policy 0, policy_version 930819 (0.0010) [2023-12-27 12:05:46,555][20347] Updated weights for policy 0, policy_version 930829 (0.0010) [2023-12-27 12:05:46,768][20381] Updated weights for policy 1, policy_version 929982 (0.0010) [2023-12-27 12:05:46,817][20381] Updated weights for policy 1, policy_version 929992 (0.0010) [2023-12-27 12:05:46,862][20381] Updated weights for policy 1, policy_version 930002 (0.0010) [2023-12-27 12:05:47,230][20347] Updated weights for policy 0, policy_version 930839 (0.0008) [2023-12-27 12:05:47,280][20347] Updated weights for policy 0, policy_version 930849 (0.0006) [2023-12-27 12:05:47,328][20347] Updated weights for policy 0, policy_version 930859 (0.0009) [2023-12-27 12:05:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 476446720. Throughput: 0: 10222.6, 1: 10153.5. Samples: 476454264. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:05:47,363][19170] Avg episode reward: [(0, '759.240'), (1, '745.550')] [2023-12-27 12:05:47,531][20381] Updated weights for policy 1, policy_version 930012 (0.0008) [2023-12-27 12:05:47,593][20381] Updated weights for policy 1, policy_version 930022 (0.0005) [2023-12-27 12:05:47,652][20381] Updated weights for policy 1, policy_version 930032 (0.0005) [2023-12-27 12:05:47,898][20347] Updated weights for policy 0, policy_version 930869 (0.0009) [2023-12-27 12:05:47,946][20347] Updated weights for policy 0, policy_version 930879 (0.0007) [2023-12-27 12:05:47,995][20347] Updated weights for policy 0, policy_version 930889 (0.0008) [2023-12-27 12:05:48,255][20381] Updated weights for policy 1, policy_version 930042 (0.0006) [2023-12-27 12:05:48,327][20381] Updated weights for policy 1, policy_version 930052 (0.0007) [2023-12-27 12:05:48,391][20381] Updated weights for policy 1, policy_version 930062 (0.0008) [2023-12-27 12:05:48,449][20381] Updated weights for policy 1, policy_version 930072 (0.0007) [2023-12-27 12:05:48,676][20347] Updated weights for policy 0, policy_version 930899 (0.0008) [2023-12-27 12:05:48,740][20347] Updated weights for policy 0, policy_version 930909 (0.0010) [2023-12-27 12:05:48,801][20347] Updated weights for policy 0, policy_version 930919 (0.0008) [2023-12-27 12:05:49,073][20381] Updated weights for policy 1, policy_version 930082 (0.0005) [2023-12-27 12:05:49,127][20381] Updated weights for policy 1, policy_version 930092 (0.0005) [2023-12-27 12:05:49,187][20381] Updated weights for policy 1, policy_version 930102 (0.0006) [2023-12-27 12:05:49,504][20347] Updated weights for policy 0, policy_version 930929 (0.0010) [2023-12-27 12:05:49,560][20347] Updated weights for policy 0, policy_version 930939 (0.0005) [2023-12-27 12:05:49,623][20347] Updated weights for policy 0, policy_version 930949 (0.0008) [2023-12-27 12:05:49,685][20347] Updated weights for policy 0, policy_version 930959 (0.0007) [2023-12-27 12:05:49,869][20381] Updated weights for policy 1, policy_version 930112 (0.0007) [2023-12-27 12:05:49,948][20381] Updated weights for policy 1, policy_version 930122 (0.0008) [2023-12-27 12:05:50,006][20381] Updated weights for policy 1, policy_version 930132 (0.0011) [2023-12-27 12:05:50,317][20347] Updated weights for policy 0, policy_version 930969 (0.0006) [2023-12-27 12:05:50,376][20347] Updated weights for policy 0, policy_version 930979 (0.0007) [2023-12-27 12:05:50,441][20347] Updated weights for policy 0, policy_version 930989 (0.0007) [2023-12-27 12:05:50,730][20381] Updated weights for policy 1, policy_version 930142 (0.0010) [2023-12-27 12:05:50,795][20381] Updated weights for policy 1, policy_version 930152 (0.0007) [2023-12-27 12:05:50,855][20381] Updated weights for policy 1, policy_version 930162 (0.0007) [2023-12-27 12:05:51,133][20347] Updated weights for policy 0, policy_version 930999 (0.0007) [2023-12-27 12:05:51,190][20347] Updated weights for policy 0, policy_version 931009 (0.0007) [2023-12-27 12:05:51,250][20347] Updated weights for policy 0, policy_version 931019 (0.0008) [2023-12-27 12:05:51,576][20381] Updated weights for policy 1, policy_version 930172 (0.0008) [2023-12-27 12:05:51,641][20381] Updated weights for policy 1, policy_version 930182 (0.0009) [2023-12-27 12:05:51,707][20381] Updated weights for policy 1, policy_version 930192 (0.0009) [2023-12-27 12:05:51,915][20347] Updated weights for policy 0, policy_version 931029 (0.0007) [2023-12-27 12:05:51,973][20347] Updated weights for policy 0, policy_version 931039 (0.0008) [2023-12-27 12:05:52,030][20347] Updated weights for policy 0, policy_version 931049 (0.0007) [2023-12-27 12:05:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 476545024. Throughput: 0: 10214.5, 1: 10199.1. Samples: 476517408. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:05:52,363][19170] Avg episode reward: [(0, '814.130'), (1, '695.190')] [2023-12-27 12:05:52,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000930200_238157824.pth... [2023-12-27 12:05:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000931056_238387200.pth... [2023-12-27 12:05:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000929016_237854720.pth [2023-12-27 12:05:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000929840_238075904.pth [2023-12-27 12:05:52,425][20381] Updated weights for policy 1, policy_version 930202 (0.0009) [2023-12-27 12:05:52,477][20381] Updated weights for policy 1, policy_version 930212 (0.0010) [2023-12-27 12:05:52,534][20381] Updated weights for policy 1, policy_version 930222 (0.0011) [2023-12-27 12:05:52,593][20381] Updated weights for policy 1, policy_version 930232 (0.0011) [2023-12-27 12:05:52,736][20347] Updated weights for policy 0, policy_version 931059 (0.0008) [2023-12-27 12:05:52,793][20347] Updated weights for policy 0, policy_version 931069 (0.0008) [2023-12-27 12:05:52,848][20347] Updated weights for policy 0, policy_version 931079 (0.0007) [2023-12-27 12:05:53,300][20381] Updated weights for policy 1, policy_version 930242 (0.0011) [2023-12-27 12:05:53,361][20381] Updated weights for policy 1, policy_version 930252 (0.0011) [2023-12-27 12:05:53,414][20381] Updated weights for policy 1, policy_version 930262 (0.0010) [2023-12-27 12:05:53,528][20347] Updated weights for policy 0, policy_version 931089 (0.0007) [2023-12-27 12:05:53,589][20347] Updated weights for policy 0, policy_version 931099 (0.0009) [2023-12-27 12:05:53,652][20347] Updated weights for policy 0, policy_version 931109 (0.0010) [2023-12-27 12:05:53,716][20347] Updated weights for policy 0, policy_version 931119 (0.0010) [2023-12-27 12:05:54,001][20381] Updated weights for policy 1, policy_version 930272 (0.0011) [2023-12-27 12:05:54,055][20381] Updated weights for policy 1, policy_version 930282 (0.0008) [2023-12-27 12:05:54,110][20381] Updated weights for policy 1, policy_version 930292 (0.0011) [2023-12-27 12:05:54,277][20347] Updated weights for policy 0, policy_version 931129 (0.0006) [2023-12-27 12:05:54,331][20347] Updated weights for policy 0, policy_version 931139 (0.0005) [2023-12-27 12:05:54,381][20347] Updated weights for policy 0, policy_version 931149 (0.0005) [2023-12-27 12:05:54,778][20381] Updated weights for policy 1, policy_version 930302 (0.0011) [2023-12-27 12:05:54,831][20381] Updated weights for policy 1, policy_version 930312 (0.0011) [2023-12-27 12:05:54,888][20381] Updated weights for policy 1, policy_version 930322 (0.0010) [2023-12-27 12:05:54,986][20347] Updated weights for policy 0, policy_version 931159 (0.0009) [2023-12-27 12:05:55,040][20347] Updated weights for policy 0, policy_version 931169 (0.0010) [2023-12-27 12:05:55,093][20347] Updated weights for policy 0, policy_version 931179 (0.0010) [2023-12-27 12:05:55,499][20381] Updated weights for policy 1, policy_version 930332 (0.0009) [2023-12-27 12:05:55,548][20381] Updated weights for policy 1, policy_version 930342 (0.0008) [2023-12-27 12:05:55,602][20381] Updated weights for policy 1, policy_version 930352 (0.0010) [2023-12-27 12:05:55,731][20347] Updated weights for policy 0, policy_version 931189 (0.0008) [2023-12-27 12:05:55,792][20347] Updated weights for policy 0, policy_version 931199 (0.0005) [2023-12-27 12:05:55,850][20347] Updated weights for policy 0, policy_version 931209 (0.0005) [2023-12-27 12:05:56,251][20381] Updated weights for policy 1, policy_version 930362 (0.0011) [2023-12-27 12:05:56,306][20381] Updated weights for policy 1, policy_version 930372 (0.0010) [2023-12-27 12:05:56,362][20381] Updated weights for policy 1, policy_version 930382 (0.0010) [2023-12-27 12:05:56,397][20347] Updated weights for policy 0, policy_version 931219 (0.0007) [2023-12-27 12:05:56,414][20381] Updated weights for policy 1, policy_version 930392 (0.0010) [2023-12-27 12:05:56,460][20347] Updated weights for policy 0, policy_version 931229 (0.0010) [2023-12-27 12:05:56,515][20347] Updated weights for policy 0, policy_version 931239 (0.0009) [2023-12-27 12:05:57,046][20347] Updated weights for policy 0, policy_version 931249 (0.0006) [2023-12-27 12:05:57,077][20381] Updated weights for policy 1, policy_version 930402 (0.0011) [2023-12-27 12:05:57,106][20347] Updated weights for policy 0, policy_version 931259 (0.0008) [2023-12-27 12:05:57,125][20381] Updated weights for policy 1, policy_version 930412 (0.0010) [2023-12-27 12:05:57,159][20347] Updated weights for policy 0, policy_version 931269 (0.0010) [2023-12-27 12:05:57,170][20381] Updated weights for policy 1, policy_version 930422 (0.0010) [2023-12-27 12:05:57,213][20347] Updated weights for policy 0, policy_version 931279 (0.0010) [2023-12-27 12:05:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 476659712. Throughput: 0: 10292.1, 1: 10222.0. Samples: 476642100. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:05:57,363][19170] Avg episode reward: [(0, '824.180'), (1, '705.860')] [2023-12-27 12:05:57,859][20381] Updated weights for policy 1, policy_version 930432 (0.0010) [2023-12-27 12:05:57,865][20347] Updated weights for policy 0, policy_version 931289 (0.0010) [2023-12-27 12:05:57,907][20381] Updated weights for policy 1, policy_version 930442 (0.0010) [2023-12-27 12:05:57,916][20347] Updated weights for policy 0, policy_version 931299 (0.0010) [2023-12-27 12:05:57,955][20381] Updated weights for policy 1, policy_version 930452 (0.0010) [2023-12-27 12:05:57,970][20347] Updated weights for policy 0, policy_version 931309 (0.0010) [2023-12-27 12:05:58,714][20381] Updated weights for policy 1, policy_version 930462 (0.0009) [2023-12-27 12:05:58,768][20347] Updated weights for policy 0, policy_version 931319 (0.0010) [2023-12-27 12:05:58,782][20381] Updated weights for policy 1, policy_version 930472 (0.0008) [2023-12-27 12:05:58,833][20347] Updated weights for policy 0, policy_version 931329 (0.0010) [2023-12-27 12:05:58,848][20381] Updated weights for policy 1, policy_version 930482 (0.0007) [2023-12-27 12:05:58,896][20347] Updated weights for policy 0, policy_version 931339 (0.0008) [2023-12-27 12:05:59,687][20347] Updated weights for policy 0, policy_version 931349 (0.0009) [2023-12-27 12:05:59,698][20381] Updated weights for policy 1, policy_version 930492 (0.0009) [2023-12-27 12:05:59,736][20347] Updated weights for policy 0, policy_version 931359 (0.0009) [2023-12-27 12:05:59,749][20381] Updated weights for policy 1, policy_version 930502 (0.0008) [2023-12-27 12:05:59,795][20347] Updated weights for policy 0, policy_version 931369 (0.0009) [2023-12-27 12:05:59,801][20381] Updated weights for policy 1, policy_version 930512 (0.0006) [2023-12-27 12:06:00,464][20347] Updated weights for policy 0, policy_version 931379 (0.0010) [2023-12-27 12:06:00,522][20347] Updated weights for policy 0, policy_version 931389 (0.0010) [2023-12-27 12:06:00,555][20381] Updated weights for policy 1, policy_version 930522 (0.0008) [2023-12-27 12:06:00,576][20347] Updated weights for policy 0, policy_version 931399 (0.0010) [2023-12-27 12:06:00,614][20381] Updated weights for policy 1, policy_version 930532 (0.0006) [2023-12-27 12:06:00,680][20381] Updated weights for policy 1, policy_version 930542 (0.0008) [2023-12-27 12:06:00,736][20381] Updated weights for policy 1, policy_version 930552 (0.0007) [2023-12-27 12:06:01,178][20347] Updated weights for policy 0, policy_version 931409 (0.0007) [2023-12-27 12:06:01,234][20347] Updated weights for policy 0, policy_version 931419 (0.0010) [2023-12-27 12:06:01,296][20347] Updated weights for policy 0, policy_version 931429 (0.0009) [2023-12-27 12:06:01,364][20347] Updated weights for policy 0, policy_version 931439 (0.0007) [2023-12-27 12:06:01,465][20381] Updated weights for policy 1, policy_version 930562 (0.0008) [2023-12-27 12:06:01,518][20381] Updated weights for policy 1, policy_version 930572 (0.0008) [2023-12-27 12:06:01,574][20381] Updated weights for policy 1, policy_version 930582 (0.0010) [2023-12-27 12:06:02,023][20347] Updated weights for policy 0, policy_version 931449 (0.0006) [2023-12-27 12:06:02,083][20347] Updated weights for policy 0, policy_version 931459 (0.0008) [2023-12-27 12:06:02,138][20347] Updated weights for policy 0, policy_version 931469 (0.0008) [2023-12-27 12:06:02,349][20381] Updated weights for policy 1, policy_version 930592 (0.0007) [2023-12-27 12:06:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 476749824. Throughput: 0: 10320.5, 1: 10094.5. Samples: 476761640. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:02,363][19170] Avg episode reward: [(0, '912.350'), (1, '754.850')] [2023-12-27 12:06:02,411][20381] Updated weights for policy 1, policy_version 930602 (0.0006) [2023-12-27 12:06:02,467][20381] Updated weights for policy 1, policy_version 930612 (0.0007) [2023-12-27 12:06:02,845][20347] Updated weights for policy 0, policy_version 931479 (0.0007) [2023-12-27 12:06:02,904][20347] Updated weights for policy 0, policy_version 931489 (0.0007) [2023-12-27 12:06:02,956][20347] Updated weights for policy 0, policy_version 931499 (0.0007) [2023-12-27 12:06:03,045][20381] Updated weights for policy 1, policy_version 930622 (0.0006) [2023-12-27 12:06:03,100][20381] Updated weights for policy 1, policy_version 930632 (0.0005) [2023-12-27 12:06:03,154][20381] Updated weights for policy 1, policy_version 930642 (0.0006) [2023-12-27 12:06:03,652][20347] Updated weights for policy 0, policy_version 931509 (0.0008) [2023-12-27 12:06:03,695][20347] Updated weights for policy 0, policy_version 931519 (0.0007) [2023-12-27 12:06:03,748][20347] Updated weights for policy 0, policy_version 931529 (0.0005) [2023-12-27 12:06:03,759][20381] Updated weights for policy 1, policy_version 930652 (0.0007) [2023-12-27 12:06:03,827][20381] Updated weights for policy 1, policy_version 930662 (0.0007) [2023-12-27 12:06:03,892][20381] Updated weights for policy 1, policy_version 930672 (0.0009) [2023-12-27 12:06:04,587][20347] Updated weights for policy 0, policy_version 931539 (0.0006) [2023-12-27 12:06:04,642][20347] Updated weights for policy 0, policy_version 931549 (0.0007) [2023-12-27 12:06:04,697][20347] Updated weights for policy 0, policy_version 931559 (0.0006) [2023-12-27 12:06:04,728][20381] Updated weights for policy 1, policy_version 930682 (0.0009) [2023-12-27 12:06:04,786][20381] Updated weights for policy 1, policy_version 930692 (0.0008) [2023-12-27 12:06:04,848][20381] Updated weights for policy 1, policy_version 930702 (0.0007) [2023-12-27 12:06:04,909][20381] Updated weights for policy 1, policy_version 930712 (0.0007) [2023-12-27 12:06:05,342][20347] Updated weights for policy 0, policy_version 931569 (0.0010) [2023-12-27 12:06:05,407][20347] Updated weights for policy 0, policy_version 931579 (0.0007) [2023-12-27 12:06:05,470][20347] Updated weights for policy 0, policy_version 931589 (0.0010) [2023-12-27 12:06:05,536][20347] Updated weights for policy 0, policy_version 931599 (0.0009) [2023-12-27 12:06:05,564][20381] Updated weights for policy 1, policy_version 930722 (0.0008) [2023-12-27 12:06:05,628][20381] Updated weights for policy 1, policy_version 930732 (0.0009) [2023-12-27 12:06:05,687][20381] Updated weights for policy 1, policy_version 930742 (0.0008) [2023-12-27 12:06:06,215][20347] Updated weights for policy 0, policy_version 931609 (0.0011) [2023-12-27 12:06:06,278][20347] Updated weights for policy 0, policy_version 931619 (0.0010) [2023-12-27 12:06:06,283][20381] Updated weights for policy 1, policy_version 930752 (0.0008) [2023-12-27 12:06:06,342][20381] Updated weights for policy 1, policy_version 930762 (0.0005) [2023-12-27 12:06:06,346][20347] Updated weights for policy 0, policy_version 931629 (0.0011) [2023-12-27 12:06:06,403][20381] Updated weights for policy 1, policy_version 930772 (0.0008) [2023-12-27 12:06:07,022][20347] Updated weights for policy 0, policy_version 931639 (0.0011) [2023-12-27 12:06:07,034][20381] Updated weights for policy 1, policy_version 930782 (0.0007) [2023-12-27 12:06:07,080][20347] Updated weights for policy 0, policy_version 931649 (0.0010) [2023-12-27 12:06:07,082][20381] Updated weights for policy 1, policy_version 930792 (0.0005) [2023-12-27 12:06:07,135][20381] Updated weights for policy 1, policy_version 930802 (0.0006) [2023-12-27 12:06:07,136][20347] Updated weights for policy 0, policy_version 931659 (0.0011) [2023-12-27 12:06:07,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 476856320. Throughput: 0: 10275.9, 1: 10080.1. Samples: 476820684. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:07,363][19170] Avg episode reward: [(0, '961.070'), (1, '801.780')] [2023-12-27 12:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000931664_238542848.pth... [2023-12-27 12:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000930808_238313472.pth... [2023-12-27 12:06:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000929624_238010368.pth [2023-12-27 12:06:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000930480_238239744.pth [2023-12-27 12:06:07,809][20347] Updated weights for policy 0, policy_version 931669 (0.0010) [2023-12-27 12:06:07,816][20381] Updated weights for policy 1, policy_version 930812 (0.0008) [2023-12-27 12:06:07,863][20347] Updated weights for policy 0, policy_version 931679 (0.0010) [2023-12-27 12:06:07,869][20381] Updated weights for policy 1, policy_version 930822 (0.0011) [2023-12-27 12:06:07,918][20347] Updated weights for policy 0, policy_version 931689 (0.0011) [2023-12-27 12:06:07,919][20381] Updated weights for policy 1, policy_version 930832 (0.0011) [2023-12-27 12:06:08,573][20347] Updated weights for policy 0, policy_version 931699 (0.0011) [2023-12-27 12:06:08,584][20381] Updated weights for policy 1, policy_version 930842 (0.0010) [2023-12-27 12:06:08,633][20347] Updated weights for policy 0, policy_version 931709 (0.0010) [2023-12-27 12:06:08,642][20381] Updated weights for policy 1, policy_version 930852 (0.0007) [2023-12-27 12:06:08,692][20347] Updated weights for policy 0, policy_version 931719 (0.0010) [2023-12-27 12:06:08,700][20381] Updated weights for policy 1, policy_version 930862 (0.0009) [2023-12-27 12:06:08,762][20381] Updated weights for policy 1, policy_version 930872 (0.0005) [2023-12-27 12:06:09,396][20381] Updated weights for policy 1, policy_version 930882 (0.0009) [2023-12-27 12:06:09,445][20347] Updated weights for policy 0, policy_version 931729 (0.0010) [2023-12-27 12:06:09,461][20381] Updated weights for policy 1, policy_version 930892 (0.0008) [2023-12-27 12:06:09,499][20347] Updated weights for policy 0, policy_version 931739 (0.0010) [2023-12-27 12:06:09,521][20381] Updated weights for policy 1, policy_version 930902 (0.0006) [2023-12-27 12:06:09,558][20347] Updated weights for policy 0, policy_version 931749 (0.0010) [2023-12-27 12:06:09,616][20347] Updated weights for policy 0, policy_version 931759 (0.0010) [2023-12-27 12:06:10,251][20381] Updated weights for policy 1, policy_version 930912 (0.0010) [2023-12-27 12:06:10,312][20381] Updated weights for policy 1, policy_version 930922 (0.0009) [2023-12-27 12:06:10,322][20347] Updated weights for policy 0, policy_version 931769 (0.0010) [2023-12-27 12:06:10,375][20381] Updated weights for policy 1, policy_version 930932 (0.0011) [2023-12-27 12:06:10,379][20347] Updated weights for policy 0, policy_version 931779 (0.0006) [2023-12-27 12:06:10,443][20347] Updated weights for policy 0, policy_version 931789 (0.0008) [2023-12-27 12:06:11,038][20381] Updated weights for policy 1, policy_version 930942 (0.0009) [2023-12-27 12:06:11,090][20347] Updated weights for policy 0, policy_version 931799 (0.0011) [2023-12-27 12:06:11,103][20381] Updated weights for policy 1, policy_version 930952 (0.0006) [2023-12-27 12:06:11,155][20347] Updated weights for policy 0, policy_version 931809 (0.0009) [2023-12-27 12:06:11,162][20381] Updated weights for policy 1, policy_version 930962 (0.0007) [2023-12-27 12:06:11,220][20347] Updated weights for policy 0, policy_version 931819 (0.0010) [2023-12-27 12:06:11,930][20381] Updated weights for policy 1, policy_version 930972 (0.0007) [2023-12-27 12:06:11,944][20347] Updated weights for policy 0, policy_version 931829 (0.0011) [2023-12-27 12:06:11,980][20381] Updated weights for policy 1, policy_version 930982 (0.0006) [2023-12-27 12:06:12,010][20347] Updated weights for policy 0, policy_version 931839 (0.0011) [2023-12-27 12:06:12,023][20381] Updated weights for policy 1, policy_version 930992 (0.0006) [2023-12-27 12:06:12,070][20347] Updated weights for policy 0, policy_version 931849 (0.0010) [2023-12-27 12:06:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 476954624. Throughput: 0: 10252.9, 1: 10096.4. Samples: 476943916. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:12,363][19170] Avg episode reward: [(0, '931.540'), (1, '787.310')] [2023-12-27 12:06:12,735][20347] Updated weights for policy 0, policy_version 931859 (0.0010) [2023-12-27 12:06:12,797][20347] Updated weights for policy 0, policy_version 931869 (0.0010) [2023-12-27 12:06:12,802][20381] Updated weights for policy 1, policy_version 931002 (0.0007) [2023-12-27 12:06:12,852][20347] Updated weights for policy 0, policy_version 931879 (0.0010) [2023-12-27 12:06:12,854][20381] Updated weights for policy 1, policy_version 931012 (0.0007) [2023-12-27 12:06:12,905][20381] Updated weights for policy 1, policy_version 931022 (0.0007) [2023-12-27 12:06:12,964][20381] Updated weights for policy 1, policy_version 931032 (0.0006) [2023-12-27 12:06:13,533][20347] Updated weights for policy 0, policy_version 931889 (0.0010) [2023-12-27 12:06:13,559][20381] Updated weights for policy 1, policy_version 931042 (0.0007) [2023-12-27 12:06:13,589][20347] Updated weights for policy 0, policy_version 931899 (0.0010) [2023-12-27 12:06:13,616][20381] Updated weights for policy 1, policy_version 931052 (0.0011) [2023-12-27 12:06:13,646][20347] Updated weights for policy 0, policy_version 931909 (0.0010) [2023-12-27 12:06:13,670][20381] Updated weights for policy 1, policy_version 931062 (0.0007) [2023-12-27 12:06:13,706][20347] Updated weights for policy 0, policy_version 931919 (0.0007) [2023-12-27 12:06:14,328][20381] Updated weights for policy 1, policy_version 931072 (0.0009) [2023-12-27 12:06:14,371][20347] Updated weights for policy 0, policy_version 931929 (0.0007) [2023-12-27 12:06:14,385][20381] Updated weights for policy 1, policy_version 931082 (0.0007) [2023-12-27 12:06:14,431][20347] Updated weights for policy 0, policy_version 931939 (0.0005) [2023-12-27 12:06:14,441][20381] Updated weights for policy 1, policy_version 931092 (0.0007) [2023-12-27 12:06:14,488][20347] Updated weights for policy 0, policy_version 931949 (0.0006) [2023-12-27 12:06:15,108][20347] Updated weights for policy 0, policy_version 931959 (0.0008) [2023-12-27 12:06:15,140][20381] Updated weights for policy 1, policy_version 931102 (0.0011) [2023-12-27 12:06:15,170][20347] Updated weights for policy 0, policy_version 931969 (0.0007) [2023-12-27 12:06:15,191][20381] Updated weights for policy 1, policy_version 931112 (0.0010) [2023-12-27 12:06:15,240][20347] Updated weights for policy 0, policy_version 931979 (0.0007) [2023-12-27 12:06:15,243][20381] Updated weights for policy 1, policy_version 931122 (0.0010) [2023-12-27 12:06:15,878][20347] Updated weights for policy 0, policy_version 931989 (0.0009) [2023-12-27 12:06:15,936][20381] Updated weights for policy 1, policy_version 931132 (0.0007) [2023-12-27 12:06:15,941][20347] Updated weights for policy 0, policy_version 931999 (0.0010) [2023-12-27 12:06:15,996][20381] Updated weights for policy 1, policy_version 931142 (0.0011) [2023-12-27 12:06:15,997][20347] Updated weights for policy 0, policy_version 932009 (0.0010) [2023-12-27 12:06:16,050][20381] Updated weights for policy 1, policy_version 931152 (0.0006) [2023-12-27 12:06:16,577][20347] Updated weights for policy 0, policy_version 932019 (0.0010) [2023-12-27 12:06:16,630][20347] Updated weights for policy 0, policy_version 932029 (0.0010) [2023-12-27 12:06:16,688][20347] Updated weights for policy 0, policy_version 932039 (0.0010) [2023-12-27 12:06:16,696][20381] Updated weights for policy 1, policy_version 931162 (0.0006) [2023-12-27 12:06:16,750][20381] Updated weights for policy 1, policy_version 931172 (0.0010) [2023-12-27 12:06:16,805][20381] Updated weights for policy 1, policy_version 931182 (0.0010) [2023-12-27 12:06:16,864][20381] Updated weights for policy 1, policy_version 931192 (0.0006) [2023-12-27 12:06:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 477052928. Throughput: 0: 10184.1, 1: 10077.5. Samples: 477067828. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:17,363][19170] Avg episode reward: [(0, '912.040'), (1, '703.050')] [2023-12-27 12:06:17,375][20347] Updated weights for policy 0, policy_version 932049 (0.0010) [2023-12-27 12:06:17,427][20347] Updated weights for policy 0, policy_version 932059 (0.0010) [2023-12-27 12:06:17,473][20381] Updated weights for policy 1, policy_version 931202 (0.0007) [2023-12-27 12:06:17,482][20347] Updated weights for policy 0, policy_version 932069 (0.0009) [2023-12-27 12:06:17,522][20381] Updated weights for policy 1, policy_version 931212 (0.0008) [2023-12-27 12:06:17,535][20347] Updated weights for policy 0, policy_version 932079 (0.0009) [2023-12-27 12:06:17,570][20381] Updated weights for policy 1, policy_version 931222 (0.0009) [2023-12-27 12:06:18,200][20381] Updated weights for policy 1, policy_version 931232 (0.0011) [2023-12-27 12:06:18,241][20347] Updated weights for policy 0, policy_version 932089 (0.0010) [2023-12-27 12:06:18,258][20381] Updated weights for policy 1, policy_version 931242 (0.0010) [2023-12-27 12:06:18,302][20347] Updated weights for policy 0, policy_version 932099 (0.0010) [2023-12-27 12:06:18,308][20381] Updated weights for policy 1, policy_version 931252 (0.0006) [2023-12-27 12:06:18,366][20347] Updated weights for policy 0, policy_version 932109 (0.0010) [2023-12-27 12:06:18,973][20381] Updated weights for policy 1, policy_version 931262 (0.0007) [2023-12-27 12:06:19,027][20381] Updated weights for policy 1, policy_version 931272 (0.0008) [2023-12-27 12:06:19,085][20381] Updated weights for policy 1, policy_version 931282 (0.0007) [2023-12-27 12:06:19,095][20347] Updated weights for policy 0, policy_version 932119 (0.0008) [2023-12-27 12:06:19,152][20347] Updated weights for policy 0, policy_version 932129 (0.0007) [2023-12-27 12:06:19,212][20347] Updated weights for policy 0, policy_version 932139 (0.0006) [2023-12-27 12:06:19,754][20381] Updated weights for policy 1, policy_version 931292 (0.0006) [2023-12-27 12:06:19,815][20381] Updated weights for policy 1, policy_version 931302 (0.0008) [2023-12-27 12:06:19,876][20381] Updated weights for policy 1, policy_version 931312 (0.0007) [2023-12-27 12:06:19,991][20347] Updated weights for policy 0, policy_version 932149 (0.0007) [2023-12-27 12:06:20,059][20347] Updated weights for policy 0, policy_version 932159 (0.0009) [2023-12-27 12:06:20,127][20347] Updated weights for policy 0, policy_version 932169 (0.0007) [2023-12-27 12:06:20,590][20381] Updated weights for policy 1, policy_version 931322 (0.0007) [2023-12-27 12:06:20,645][20381] Updated weights for policy 1, policy_version 931332 (0.0007) [2023-12-27 12:06:20,712][20381] Updated weights for policy 1, policy_version 931342 (0.0006) [2023-12-27 12:06:20,771][20381] Updated weights for policy 1, policy_version 931352 (0.0008) [2023-12-27 12:06:20,817][20347] Updated weights for policy 0, policy_version 932179 (0.0008) [2023-12-27 12:06:20,882][20347] Updated weights for policy 0, policy_version 932189 (0.0010) [2023-12-27 12:06:20,944][20347] Updated weights for policy 0, policy_version 932199 (0.0010) [2023-12-27 12:06:21,491][20381] Updated weights for policy 1, policy_version 931362 (0.0008) [2023-12-27 12:06:21,554][20381] Updated weights for policy 1, policy_version 931372 (0.0007) [2023-12-27 12:06:21,613][20381] Updated weights for policy 1, policy_version 931382 (0.0007) [2023-12-27 12:06:21,698][20347] Updated weights for policy 0, policy_version 932209 (0.0010) [2023-12-27 12:06:21,769][20347] Updated weights for policy 0, policy_version 932219 (0.0009) [2023-12-27 12:06:21,836][20347] Updated weights for policy 0, policy_version 932229 (0.0008) [2023-12-27 12:06:21,901][20347] Updated weights for policy 0, policy_version 932239 (0.0008) [2023-12-27 12:06:22,338][20381] Updated weights for policy 1, policy_version 931392 (0.0010) [2023-12-27 12:06:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20382.8). Total num frames: 477151232. Throughput: 0: 10166.3, 1: 10086.8. Samples: 477128380. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:22,363][19170] Avg episode reward: [(0, '931.520'), (1, '735.240')] [2023-12-27 12:06:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000932240_238690304.pth... [2023-12-27 12:06:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000931056_238387200.pth [2023-12-27 12:06:22,404][20381] Updated weights for policy 1, policy_version 931402 (0.0011) [2023-12-27 12:06:22,459][20381] Updated weights for policy 1, policy_version 931412 (0.0007) [2023-12-27 12:06:22,483][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000931416_238469120.pth... [2023-12-27 12:06:22,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000930200_238157824.pth [2023-12-27 12:06:22,555][20347] Updated weights for policy 0, policy_version 932249 (0.0007) [2023-12-27 12:06:22,620][20347] Updated weights for policy 0, policy_version 932259 (0.0011) [2023-12-27 12:06:22,683][20347] Updated weights for policy 0, policy_version 932269 (0.0010) [2023-12-27 12:06:23,104][20381] Updated weights for policy 1, policy_version 931422 (0.0006) [2023-12-27 12:06:23,165][20381] Updated weights for policy 1, policy_version 931432 (0.0005) [2023-12-27 12:06:23,224][20381] Updated weights for policy 1, policy_version 931442 (0.0005) [2023-12-27 12:06:23,354][20347] Updated weights for policy 0, policy_version 932279 (0.0010) [2023-12-27 12:06:23,399][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000009 [2023-12-27 12:06:23,795][20381] Updated weights for policy 1, policy_version 931452 (0.0005) [2023-12-27 12:06:23,847][20381] Updated weights for policy 1, policy_version 931462 (0.0005) [2023-12-27 12:06:23,907][20381] Updated weights for policy 1, policy_version 931472 (0.0005) [2023-12-27 12:06:24,073][20347] Updated weights for policy 0, policy_version 932289 (0.0010) [2023-12-27 12:06:24,124][20347] Updated weights for policy 0, policy_version 932299 (0.0010) [2023-12-27 12:06:24,179][20347] Updated weights for policy 0, policy_version 932309 (0.0007) [2023-12-27 12:06:24,243][20347] Updated weights for policy 0, policy_version 932319 (0.0006) [2023-12-27 12:06:24,434][20381] Updated weights for policy 1, policy_version 931482 (0.0005) [2023-12-27 12:06:24,487][20381] Updated weights for policy 1, policy_version 931492 (0.0005) [2023-12-27 12:06:24,546][20381] Updated weights for policy 1, policy_version 931502 (0.0005) [2023-12-27 12:06:24,593][20381] Updated weights for policy 1, policy_version 931512 (0.0005) [2023-12-27 12:06:24,895][20347] Updated weights for policy 0, policy_version 932329 (0.0007) [2023-12-27 12:06:24,956][20347] Updated weights for policy 0, policy_version 932339 (0.0007) [2023-12-27 12:06:25,016][20347] Updated weights for policy 0, policy_version 932349 (0.0007) [2023-12-27 12:06:25,225][20381] Updated weights for policy 1, policy_version 931522 (0.0010) [2023-12-27 12:06:25,282][20381] Updated weights for policy 1, policy_version 931532 (0.0010) [2023-12-27 12:06:25,341][20381] Updated weights for policy 1, policy_version 931542 (0.0011) [2023-12-27 12:06:25,653][20347] Updated weights for policy 0, policy_version 932359 (0.0008) [2023-12-27 12:06:25,703][20347] Updated weights for policy 0, policy_version 932369 (0.0009) [2023-12-27 12:06:25,756][20347] Updated weights for policy 0, policy_version 932379 (0.0006) [2023-12-27 12:06:25,949][20381] Updated weights for policy 1, policy_version 931552 (0.0007) [2023-12-27 12:06:26,012][20381] Updated weights for policy 1, policy_version 931562 (0.0005) [2023-12-27 12:06:26,073][20381] Updated weights for policy 1, policy_version 931572 (0.0007) [2023-12-27 12:06:26,379][20347] Updated weights for policy 0, policy_version 932389 (0.0006) [2023-12-27 12:06:26,432][20347] Updated weights for policy 0, policy_version 932399 (0.0007) [2023-12-27 12:06:26,487][20347] Updated weights for policy 0, policy_version 932409 (0.0007) [2023-12-27 12:06:26,731][20381] Updated weights for policy 1, policy_version 931582 (0.0010) [2023-12-27 12:06:26,775][20381] Updated weights for policy 1, policy_version 931592 (0.0010) [2023-12-27 12:06:26,830][20381] Updated weights for policy 1, policy_version 931602 (0.0010) [2023-12-27 12:06:27,147][20347] Updated weights for policy 0, policy_version 932419 (0.0007) [2023-12-27 12:06:27,206][20347] Updated weights for policy 0, policy_version 932429 (0.0008) [2023-12-27 12:06:27,263][20347] Updated weights for policy 0, policy_version 932439 (0.0007) [2023-12-27 12:06:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 477265920. Throughput: 0: 10231.4, 1: 10228.2. Samples: 477253060. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:27,363][19170] Avg episode reward: [(0, '902.150'), (1, '813.670')] [2023-12-27 12:06:27,466][20381] Updated weights for policy 1, policy_version 931612 (0.0008) [2023-12-27 12:06:27,514][20381] Updated weights for policy 1, policy_version 931622 (0.0005) [2023-12-27 12:06:27,563][20381] Updated weights for policy 1, policy_version 931632 (0.0006) [2023-12-27 12:06:27,932][20347] Updated weights for policy 0, policy_version 932449 (0.0008) [2023-12-27 12:06:27,977][20347] Updated weights for policy 0, policy_version 932459 (0.0006) [2023-12-27 12:06:28,035][20347] Updated weights for policy 0, policy_version 932469 (0.0005) [2023-12-27 12:06:28,085][20347] Updated weights for policy 0, policy_version 932479 (0.0005) [2023-12-27 12:06:28,144][20381] Updated weights for policy 1, policy_version 931642 (0.0010) [2023-12-27 12:06:28,202][20381] Updated weights for policy 1, policy_version 931652 (0.0007) [2023-12-27 12:06:28,261][20381] Updated weights for policy 1, policy_version 931662 (0.0009) [2023-12-27 12:06:28,315][20381] Updated weights for policy 1, policy_version 931672 (0.0010) [2023-12-27 12:06:28,649][20347] Updated weights for policy 0, policy_version 932489 (0.0008) [2023-12-27 12:06:28,704][20347] Updated weights for policy 0, policy_version 932499 (0.0010) [2023-12-27 12:06:28,752][20347] Updated weights for policy 0, policy_version 932509 (0.0010) [2023-12-27 12:06:28,980][20381] Updated weights for policy 1, policy_version 931682 (0.0010) [2023-12-27 12:06:29,039][20381] Updated weights for policy 1, policy_version 931692 (0.0010) [2023-12-27 12:06:29,094][20381] Updated weights for policy 1, policy_version 931702 (0.0009) [2023-12-27 12:06:29,452][20347] Updated weights for policy 0, policy_version 932519 (0.0008) [2023-12-27 12:06:29,503][20347] Updated weights for policy 0, policy_version 932529 (0.0008) [2023-12-27 12:06:29,557][20347] Updated weights for policy 0, policy_version 932539 (0.0008) [2023-12-27 12:06:29,760][20381] Updated weights for policy 1, policy_version 931712 (0.0006) [2023-12-27 12:06:29,820][20381] Updated weights for policy 1, policy_version 931722 (0.0008) [2023-12-27 12:06:29,891][20381] Updated weights for policy 1, policy_version 931732 (0.0010) [2023-12-27 12:06:30,246][20347] Updated weights for policy 0, policy_version 932549 (0.0006) [2023-12-27 12:06:30,304][20347] Updated weights for policy 0, policy_version 932559 (0.0005) [2023-12-27 12:06:30,352][20347] Updated weights for policy 0, policy_version 932569 (0.0006) [2023-12-27 12:06:30,562][20381] Updated weights for policy 1, policy_version 931742 (0.0009) [2023-12-27 12:06:30,624][20381] Updated weights for policy 1, policy_version 931752 (0.0009) [2023-12-27 12:06:30,678][20381] Updated weights for policy 1, policy_version 931762 (0.0007) [2023-12-27 12:06:30,962][20347] Updated weights for policy 0, policy_version 932579 (0.0007) [2023-12-27 12:06:31,017][20347] Updated weights for policy 0, policy_version 932589 (0.0008) [2023-12-27 12:06:31,078][20347] Updated weights for policy 0, policy_version 932599 (0.0008) [2023-12-27 12:06:31,372][20381] Updated weights for policy 1, policy_version 931772 (0.0008) [2023-12-27 12:06:31,429][20381] Updated weights for policy 1, policy_version 931782 (0.0007) [2023-12-27 12:06:31,477][20381] Updated weights for policy 1, policy_version 931792 (0.0007) [2023-12-27 12:06:31,830][20347] Updated weights for policy 0, policy_version 932609 (0.0008) [2023-12-27 12:06:31,885][20347] Updated weights for policy 0, policy_version 932619 (0.0008) [2023-12-27 12:06:31,944][20347] Updated weights for policy 0, policy_version 932629 (0.0008) [2023-12-27 12:06:32,006][20347] Updated weights for policy 0, policy_version 932639 (0.0007) [2023-12-27 12:06:32,150][20381] Updated weights for policy 1, policy_version 931802 (0.0007) [2023-12-27 12:06:32,197][20381] Updated weights for policy 1, policy_version 931812 (0.0005) [2023-12-27 12:06:32,260][20381] Updated weights for policy 1, policy_version 931822 (0.0007) [2023-12-27 12:06:32,315][20381] Updated weights for policy 1, policy_version 931832 (0.0008) [2023-12-27 12:06:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 477372416. Throughput: 0: 10267.6, 1: 10271.7. Samples: 477378536. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:32,363][19170] Avg episode reward: [(0, '805.800'), (1, '853.000')] [2023-12-27 12:06:32,685][20347] Updated weights for policy 0, policy_version 932649 (0.0007) [2023-12-27 12:06:32,742][20347] Updated weights for policy 0, policy_version 932659 (0.0006) [2023-12-27 12:06:32,793][20347] Updated weights for policy 0, policy_version 932669 (0.0007) [2023-12-27 12:06:33,021][20381] Updated weights for policy 1, policy_version 931842 (0.0005) [2023-12-27 12:06:33,078][20381] Updated weights for policy 1, policy_version 931852 (0.0005) [2023-12-27 12:06:33,145][20381] Updated weights for policy 1, policy_version 931862 (0.0006) [2023-12-27 12:06:33,484][20347] Updated weights for policy 0, policy_version 932679 (0.0007) [2023-12-27 12:06:33,543][20347] Updated weights for policy 0, policy_version 932689 (0.0006) [2023-12-27 12:06:33,606][20347] Updated weights for policy 0, policy_version 932699 (0.0007) [2023-12-27 12:06:33,700][20381] Updated weights for policy 1, policy_version 931872 (0.0006) [2023-12-27 12:06:33,755][20381] Updated weights for policy 1, policy_version 931882 (0.0006) [2023-12-27 12:06:33,806][20381] Updated weights for policy 1, policy_version 931892 (0.0010) [2023-12-27 12:06:34,293][20347] Updated weights for policy 0, policy_version 932709 (0.0008) [2023-12-27 12:06:34,356][20347] Updated weights for policy 0, policy_version 932719 (0.0006) [2023-12-27 12:06:34,423][20347] Updated weights for policy 0, policy_version 932729 (0.0005) [2023-12-27 12:06:34,514][20381] Updated weights for policy 1, policy_version 931902 (0.0008) [2023-12-27 12:06:34,575][20381] Updated weights for policy 1, policy_version 931912 (0.0008) [2023-12-27 12:06:34,631][20381] Updated weights for policy 1, policy_version 931922 (0.0007) [2023-12-27 12:06:35,038][20347] Updated weights for policy 0, policy_version 932739 (0.0006) [2023-12-27 12:06:35,096][20347] Updated weights for policy 0, policy_version 932749 (0.0007) [2023-12-27 12:06:35,142][20347] Updated weights for policy 0, policy_version 932759 (0.0008) [2023-12-27 12:06:35,281][20381] Updated weights for policy 1, policy_version 931932 (0.0007) [2023-12-27 12:06:35,345][20381] Updated weights for policy 1, policy_version 931942 (0.0006) [2023-12-27 12:06:35,401][20381] Updated weights for policy 1, policy_version 931952 (0.0008) [2023-12-27 12:06:35,849][20347] Updated weights for policy 0, policy_version 932769 (0.0007) [2023-12-27 12:06:35,906][20347] Updated weights for policy 0, policy_version 932779 (0.0008) [2023-12-27 12:06:35,954][20347] Updated weights for policy 0, policy_version 932789 (0.0006) [2023-12-27 12:06:36,010][20347] Updated weights for policy 0, policy_version 932799 (0.0007) [2023-12-27 12:06:36,088][20381] Updated weights for policy 1, policy_version 931962 (0.0008) [2023-12-27 12:06:36,152][20381] Updated weights for policy 1, policy_version 931972 (0.0008) [2023-12-27 12:06:36,217][20381] Updated weights for policy 1, policy_version 931982 (0.0006) [2023-12-27 12:06:36,280][20381] Updated weights for policy 1, policy_version 931992 (0.0006) [2023-12-27 12:06:36,701][20347] Updated weights for policy 0, policy_version 932809 (0.0009) [2023-12-27 12:06:36,763][20347] Updated weights for policy 0, policy_version 932819 (0.0010) [2023-12-27 12:06:36,828][20347] Updated weights for policy 0, policy_version 932829 (0.0009) [2023-12-27 12:06:36,941][20381] Updated weights for policy 1, policy_version 932002 (0.0006) [2023-12-27 12:06:36,992][20381] Updated weights for policy 1, policy_version 932012 (0.0005) [2023-12-27 12:06:37,043][20381] Updated weights for policy 1, policy_version 932022 (0.0007) [2023-12-27 12:06:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 477470720. Throughput: 0: 10252.1, 1: 10267.3. Samples: 477440780. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:37,364][19170] Avg episode reward: [(0, '767.950'), (1, '813.990')] [2023-12-27 12:06:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000932024_238624768.pth... [2023-12-27 12:06:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000932832_238845952.pth... [2023-12-27 12:06:37,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000930808_238313472.pth [2023-12-27 12:06:37,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000931664_238542848.pth [2023-12-27 12:06:37,513][20347] Updated weights for policy 0, policy_version 932839 (0.0008) [2023-12-27 12:06:37,575][20347] Updated weights for policy 0, policy_version 932849 (0.0008) [2023-12-27 12:06:37,633][20347] Updated weights for policy 0, policy_version 932859 (0.0009) [2023-12-27 12:06:37,748][20381] Updated weights for policy 1, policy_version 932032 (0.0008) [2023-12-27 12:06:37,798][20381] Updated weights for policy 1, policy_version 932042 (0.0008) [2023-12-27 12:06:37,855][20381] Updated weights for policy 1, policy_version 932052 (0.0008) [2023-12-27 12:06:38,292][20347] Updated weights for policy 0, policy_version 932869 (0.0010) [2023-12-27 12:06:38,349][20347] Updated weights for policy 0, policy_version 932879 (0.0009) [2023-12-27 12:06:38,417][20347] Updated weights for policy 0, policy_version 932889 (0.0010) [2023-12-27 12:06:38,528][20381] Updated weights for policy 1, policy_version 932062 (0.0006) [2023-12-27 12:06:38,589][20381] Updated weights for policy 1, policy_version 932072 (0.0007) [2023-12-27 12:06:38,648][20381] Updated weights for policy 1, policy_version 932082 (0.0011) [2023-12-27 12:06:39,053][20347] Updated weights for policy 0, policy_version 932899 (0.0007) [2023-12-27 12:06:39,111][20347] Updated weights for policy 0, policy_version 932909 (0.0010) [2023-12-27 12:06:39,163][20347] Updated weights for policy 0, policy_version 932919 (0.0011) [2023-12-27 12:06:39,254][20381] Updated weights for policy 1, policy_version 932092 (0.0010) [2023-12-27 12:06:39,314][20381] Updated weights for policy 1, policy_version 932102 (0.0008) [2023-12-27 12:06:39,374][20381] Updated weights for policy 1, policy_version 932112 (0.0009) [2023-12-27 12:06:39,993][20347] Updated weights for policy 0, policy_version 932929 (0.0010) [2023-12-27 12:06:40,053][20347] Updated weights for policy 0, policy_version 932939 (0.0008) [2023-12-27 12:06:40,113][20347] Updated weights for policy 0, policy_version 932949 (0.0008) [2023-12-27 12:06:40,173][20347] Updated weights for policy 0, policy_version 932959 (0.0008) [2023-12-27 12:06:40,184][20381] Updated weights for policy 1, policy_version 932122 (0.0008) [2023-12-27 12:06:40,243][20381] Updated weights for policy 1, policy_version 932132 (0.0007) [2023-12-27 12:06:40,299][20381] Updated weights for policy 1, policy_version 932142 (0.0009) [2023-12-27 12:06:40,353][20381] Updated weights for policy 1, policy_version 932152 (0.0008) [2023-12-27 12:06:40,849][20347] Updated weights for policy 0, policy_version 932969 (0.0006) [2023-12-27 12:06:40,910][20347] Updated weights for policy 0, policy_version 932979 (0.0006) [2023-12-27 12:06:40,966][20347] Updated weights for policy 0, policy_version 932989 (0.0008) [2023-12-27 12:06:41,044][20381] Updated weights for policy 1, policy_version 932162 (0.0007) [2023-12-27 12:06:41,105][20381] Updated weights for policy 1, policy_version 932172 (0.0008) [2023-12-27 12:06:41,172][20381] Updated weights for policy 1, policy_version 932182 (0.0007) [2023-12-27 12:06:41,635][20347] Updated weights for policy 0, policy_version 932999 (0.0009) [2023-12-27 12:06:41,701][20347] Updated weights for policy 0, policy_version 933009 (0.0007) [2023-12-27 12:06:41,769][20347] Updated weights for policy 0, policy_version 933019 (0.0008) [2023-12-27 12:06:41,934][20381] Updated weights for policy 1, policy_version 932192 (0.0008) [2023-12-27 12:06:41,995][20381] Updated weights for policy 1, policy_version 932202 (0.0009) [2023-12-27 12:06:42,053][20381] Updated weights for policy 1, policy_version 932212 (0.0006) [2023-12-27 12:06:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 477569024. Throughput: 0: 10172.6, 1: 10236.9. Samples: 477560524. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:42,363][19170] Avg episode reward: [(0, '785.810'), (1, '804.150')] [2023-12-27 12:06:42,430][20347] Updated weights for policy 0, policy_version 933029 (0.0007) [2023-12-27 12:06:42,480][20347] Updated weights for policy 0, policy_version 933039 (0.0005) [2023-12-27 12:06:42,533][20347] Updated weights for policy 0, policy_version 933049 (0.0006) [2023-12-27 12:06:42,740][20381] Updated weights for policy 1, policy_version 932222 (0.0007) [2023-12-27 12:06:42,793][20381] Updated weights for policy 1, policy_version 932232 (0.0008) [2023-12-27 12:06:42,843][20381] Updated weights for policy 1, policy_version 932242 (0.0008) [2023-12-27 12:06:43,172][20347] Updated weights for policy 0, policy_version 933059 (0.0006) [2023-12-27 12:06:43,232][20347] Updated weights for policy 0, policy_version 933069 (0.0007) [2023-12-27 12:06:43,289][20347] Updated weights for policy 0, policy_version 933079 (0.0008) [2023-12-27 12:06:43,572][20381] Updated weights for policy 1, policy_version 932252 (0.0006) [2023-12-27 12:06:43,619][20381] Updated weights for policy 1, policy_version 932262 (0.0005) [2023-12-27 12:06:43,671][20381] Updated weights for policy 1, policy_version 932272 (0.0007) [2023-12-27 12:06:43,986][20347] Updated weights for policy 0, policy_version 933089 (0.0008) [2023-12-27 12:06:44,041][20347] Updated weights for policy 0, policy_version 933099 (0.0008) [2023-12-27 12:06:44,101][20347] Updated weights for policy 0, policy_version 933109 (0.0009) [2023-12-27 12:06:44,155][20347] Updated weights for policy 0, policy_version 933119 (0.0010) [2023-12-27 12:06:44,359][20381] Updated weights for policy 1, policy_version 932282 (0.0007) [2023-12-27 12:06:44,415][20381] Updated weights for policy 1, policy_version 932292 (0.0010) [2023-12-27 12:06:44,464][20381] Updated weights for policy 1, policy_version 932302 (0.0010) [2023-12-27 12:06:44,510][20381] Updated weights for policy 1, policy_version 932312 (0.0011) [2023-12-27 12:06:44,855][20347] Updated weights for policy 0, policy_version 933129 (0.0008) [2023-12-27 12:06:44,913][20347] Updated weights for policy 0, policy_version 933139 (0.0007) [2023-12-27 12:06:44,960][20347] Updated weights for policy 0, policy_version 933149 (0.0006) [2023-12-27 12:06:45,265][20381] Updated weights for policy 1, policy_version 932322 (0.0008) [2023-12-27 12:06:45,322][20381] Updated weights for policy 1, policy_version 932332 (0.0009) [2023-12-27 12:06:45,374][20381] Updated weights for policy 1, policy_version 932342 (0.0008) [2023-12-27 12:06:45,680][20347] Updated weights for policy 0, policy_version 933159 (0.0009) [2023-12-27 12:06:45,725][20347] Updated weights for policy 0, policy_version 933169 (0.0007) [2023-12-27 12:06:45,773][20347] Updated weights for policy 0, policy_version 933179 (0.0009) [2023-12-27 12:06:46,022][20381] Updated weights for policy 1, policy_version 932352 (0.0009) [2023-12-27 12:06:46,077][20381] Updated weights for policy 1, policy_version 932362 (0.0011) [2023-12-27 12:06:46,127][20381] Updated weights for policy 1, policy_version 932372 (0.0008) [2023-12-27 12:06:46,348][20347] Updated weights for policy 0, policy_version 933189 (0.0008) [2023-12-27 12:06:46,404][20347] Updated weights for policy 0, policy_version 933199 (0.0006) [2023-12-27 12:06:46,457][20347] Updated weights for policy 0, policy_version 933209 (0.0010) [2023-12-27 12:06:46,803][20381] Updated weights for policy 1, policy_version 932382 (0.0008) [2023-12-27 12:06:46,852][20381] Updated weights for policy 1, policy_version 932392 (0.0010) [2023-12-27 12:06:46,907][20381] Updated weights for policy 1, policy_version 932402 (0.0010) [2023-12-27 12:06:47,105][20347] Updated weights for policy 0, policy_version 933219 (0.0010) [2023-12-27 12:06:47,160][20347] Updated weights for policy 0, policy_version 933229 (0.0007) [2023-12-27 12:06:47,206][20347] Updated weights for policy 0, policy_version 933239 (0.0007) [2023-12-27 12:06:47,363][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 477675520. Throughput: 0: 10172.1, 1: 10313.1. Samples: 477683476. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:47,364][19170] Avg episode reward: [(0, '805.090'), (1, '814.000')] [2023-12-27 12:06:47,523][20381] Updated weights for policy 1, policy_version 932412 (0.0010) [2023-12-27 12:06:47,568][20381] Updated weights for policy 1, policy_version 932422 (0.0010) [2023-12-27 12:06:47,621][20381] Updated weights for policy 1, policy_version 932432 (0.0011) [2023-12-27 12:06:47,904][20347] Updated weights for policy 0, policy_version 933249 (0.0007) [2023-12-27 12:06:47,967][20347] Updated weights for policy 0, policy_version 933259 (0.0007) [2023-12-27 12:06:48,025][20347] Updated weights for policy 0, policy_version 933269 (0.0007) [2023-12-27 12:06:48,083][20347] Updated weights for policy 0, policy_version 933279 (0.0008) [2023-12-27 12:06:48,319][20381] Updated weights for policy 1, policy_version 932442 (0.0010) [2023-12-27 12:06:48,384][20381] Updated weights for policy 1, policy_version 932452 (0.0011) [2023-12-27 12:06:48,440][20381] Updated weights for policy 1, policy_version 932462 (0.0008) [2023-12-27 12:06:48,496][20381] Updated weights for policy 1, policy_version 932472 (0.0008) [2023-12-27 12:06:48,715][20347] Updated weights for policy 0, policy_version 933289 (0.0005) [2023-12-27 12:06:48,769][20347] Updated weights for policy 0, policy_version 933299 (0.0008) [2023-12-27 12:06:48,827][20347] Updated weights for policy 0, policy_version 933309 (0.0007) [2023-12-27 12:06:49,186][20381] Updated weights for policy 1, policy_version 932482 (0.0010) [2023-12-27 12:06:49,244][20381] Updated weights for policy 1, policy_version 932492 (0.0009) [2023-12-27 12:06:49,299][20381] Updated weights for policy 1, policy_version 932502 (0.0006) [2023-12-27 12:06:49,476][20347] Updated weights for policy 0, policy_version 933319 (0.0006) [2023-12-27 12:06:49,536][20347] Updated weights for policy 0, policy_version 933329 (0.0007) [2023-12-27 12:06:49,594][20347] Updated weights for policy 0, policy_version 933339 (0.0008) [2023-12-27 12:06:50,025][20381] Updated weights for policy 1, policy_version 932512 (0.0009) [2023-12-27 12:06:50,082][20381] Updated weights for policy 1, policy_version 932522 (0.0011) [2023-12-27 12:06:50,139][20381] Updated weights for policy 1, policy_version 932532 (0.0011) [2023-12-27 12:06:50,289][20347] Updated weights for policy 0, policy_version 933349 (0.0007) [2023-12-27 12:06:50,348][20347] Updated weights for policy 0, policy_version 933359 (0.0008) [2023-12-27 12:06:50,406][20347] Updated weights for policy 0, policy_version 933369 (0.0007) [2023-12-27 12:06:50,781][20381] Updated weights for policy 1, policy_version 932542 (0.0009) [2023-12-27 12:06:50,834][20381] Updated weights for policy 1, policy_version 932552 (0.0005) [2023-12-27 12:06:50,893][20381] Updated weights for policy 1, policy_version 932562 (0.0012) [2023-12-27 12:06:51,143][20347] Updated weights for policy 0, policy_version 933379 (0.0007) [2023-12-27 12:06:51,209][20347] Updated weights for policy 0, policy_version 933389 (0.0007) [2023-12-27 12:06:51,271][20347] Updated weights for policy 0, policy_version 933399 (0.0007) [2023-12-27 12:06:51,621][20381] Updated weights for policy 1, policy_version 932572 (0.0011) [2023-12-27 12:06:51,691][20381] Updated weights for policy 1, policy_version 932582 (0.0010) [2023-12-27 12:06:51,758][20381] Updated weights for policy 1, policy_version 932592 (0.0008) [2023-12-27 12:06:51,979][20347] Updated weights for policy 0, policy_version 933409 (0.0008) [2023-12-27 12:06:52,038][20347] Updated weights for policy 0, policy_version 933419 (0.0006) [2023-12-27 12:06:52,103][20347] Updated weights for policy 0, policy_version 933429 (0.0006) [2023-12-27 12:06:52,155][20347] Updated weights for policy 0, policy_version 933439 (0.0007) [2023-12-27 12:06:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 477773824. Throughput: 0: 10219.5, 1: 10308.9. Samples: 477744460. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:52,363][19170] Avg episode reward: [(0, '881.990'), (1, '784.720')] [2023-12-27 12:06:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000932600_238772224.pth... [2023-12-27 12:06:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000933440_239001600.pth... [2023-12-27 12:06:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000931416_238469120.pth [2023-12-27 12:06:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000932240_238690304.pth [2023-12-27 12:06:52,416][20381] Updated weights for policy 1, policy_version 932602 (0.0008) [2023-12-27 12:06:52,465][20381] Updated weights for policy 1, policy_version 932612 (0.0007) [2023-12-27 12:06:52,514][20381] Updated weights for policy 1, policy_version 932622 (0.0007) [2023-12-27 12:06:52,568][20381] Updated weights for policy 1, policy_version 932632 (0.0009) [2023-12-27 12:06:52,825][20347] Updated weights for policy 0, policy_version 933449 (0.0008) [2023-12-27 12:06:52,890][20347] Updated weights for policy 0, policy_version 933459 (0.0008) [2023-12-27 12:06:52,955][20347] Updated weights for policy 0, policy_version 933469 (0.0008) [2023-12-27 12:06:53,247][20381] Updated weights for policy 1, policy_version 932642 (0.0008) [2023-12-27 12:06:53,293][20381] Updated weights for policy 1, policy_version 932652 (0.0008) [2023-12-27 12:06:53,344][20381] Updated weights for policy 1, policy_version 932662 (0.0009) [2023-12-27 12:06:53,606][20347] Updated weights for policy 0, policy_version 933479 (0.0007) [2023-12-27 12:06:53,661][20347] Updated weights for policy 0, policy_version 933489 (0.0008) [2023-12-27 12:06:53,716][20347] Updated weights for policy 0, policy_version 933499 (0.0008) [2023-12-27 12:06:54,044][20381] Updated weights for policy 1, policy_version 932672 (0.0010) [2023-12-27 12:06:54,106][20381] Updated weights for policy 1, policy_version 932682 (0.0010) [2023-12-27 12:06:54,164][20381] Updated weights for policy 1, policy_version 932692 (0.0011) [2023-12-27 12:06:54,404][20347] Updated weights for policy 0, policy_version 933509 (0.0007) [2023-12-27 12:06:54,468][20347] Updated weights for policy 0, policy_version 933519 (0.0008) [2023-12-27 12:06:54,529][20347] Updated weights for policy 0, policy_version 933529 (0.0007) [2023-12-27 12:06:54,813][20381] Updated weights for policy 1, policy_version 932702 (0.0011) [2023-12-27 12:06:54,866][20381] Updated weights for policy 1, policy_version 932712 (0.0011) [2023-12-27 12:06:54,921][20381] Updated weights for policy 1, policy_version 932722 (0.0011) [2023-12-27 12:06:55,227][20347] Updated weights for policy 0, policy_version 933539 (0.0007) [2023-12-27 12:06:55,278][20347] Updated weights for policy 0, policy_version 933549 (0.0007) [2023-12-27 12:06:55,329][20347] Updated weights for policy 0, policy_version 933559 (0.0007) [2023-12-27 12:06:55,596][20381] Updated weights for policy 1, policy_version 932732 (0.0010) [2023-12-27 12:06:55,644][20381] Updated weights for policy 1, policy_version 932742 (0.0010) [2023-12-27 12:06:55,698][20381] Updated weights for policy 1, policy_version 932752 (0.0010) [2023-12-27 12:06:56,001][20347] Updated weights for policy 0, policy_version 933569 (0.0007) [2023-12-27 12:06:56,068][20347] Updated weights for policy 0, policy_version 933579 (0.0008) [2023-12-27 12:06:56,131][20347] Updated weights for policy 0, policy_version 933589 (0.0007) [2023-12-27 12:06:56,189][20347] Updated weights for policy 0, policy_version 933599 (0.0007) [2023-12-27 12:06:56,349][20381] Updated weights for policy 1, policy_version 932762 (0.0010) [2023-12-27 12:06:56,406][20381] Updated weights for policy 1, policy_version 932772 (0.0010) [2023-12-27 12:06:56,463][20381] Updated weights for policy 1, policy_version 932782 (0.0006) [2023-12-27 12:06:56,520][20381] Updated weights for policy 1, policy_version 932792 (0.0007) [2023-12-27 12:06:56,885][20347] Updated weights for policy 0, policy_version 933609 (0.0008) [2023-12-27 12:06:56,946][20347] Updated weights for policy 0, policy_version 933619 (0.0007) [2023-12-27 12:06:57,005][20347] Updated weights for policy 0, policy_version 933629 (0.0007) [2023-12-27 12:06:57,158][20381] Updated weights for policy 1, policy_version 932802 (0.0010) [2023-12-27 12:06:57,212][20381] Updated weights for policy 1, policy_version 932812 (0.0010) [2023-12-27 12:06:57,263][20381] Updated weights for policy 1, policy_version 932822 (0.0010) [2023-12-27 12:06:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 477880320. Throughput: 0: 10207.5, 1: 10293.2. Samples: 477866444. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:06:57,363][19170] Avg episode reward: [(0, '912.360'), (1, '804.260')] [2023-12-27 12:06:57,675][20347] Updated weights for policy 0, policy_version 933639 (0.0007) [2023-12-27 12:06:57,724][20347] Updated weights for policy 0, policy_version 933649 (0.0007) [2023-12-27 12:06:57,774][20347] Updated weights for policy 0, policy_version 933659 (0.0007) [2023-12-27 12:06:57,921][20381] Updated weights for policy 1, policy_version 932832 (0.0006) [2023-12-27 12:06:57,966][20381] Updated weights for policy 1, policy_version 932842 (0.0005) [2023-12-27 12:06:58,021][20381] Updated weights for policy 1, policy_version 932852 (0.0005) [2023-12-27 12:06:58,586][20347] Updated weights for policy 0, policy_version 933669 (0.0007) [2023-12-27 12:06:58,650][20347] Updated weights for policy 0, policy_version 933679 (0.0008) [2023-12-27 12:06:58,653][20381] Updated weights for policy 1, policy_version 932862 (0.0006) [2023-12-27 12:06:58,710][20381] Updated weights for policy 1, policy_version 932872 (0.0011) [2023-12-27 12:06:58,715][20347] Updated weights for policy 0, policy_version 933689 (0.0008) [2023-12-27 12:06:58,772][20381] Updated weights for policy 1, policy_version 932882 (0.0008) [2023-12-27 12:06:59,444][20347] Updated weights for policy 0, policy_version 933699 (0.0008) [2023-12-27 12:06:59,500][20347] Updated weights for policy 0, policy_version 933709 (0.0008) [2023-12-27 12:06:59,524][20381] Updated weights for policy 1, policy_version 932892 (0.0009) [2023-12-27 12:06:59,560][20347] Updated weights for policy 0, policy_version 933719 (0.0007) [2023-12-27 12:06:59,573][20381] Updated weights for policy 1, policy_version 932902 (0.0007) [2023-12-27 12:06:59,630][20381] Updated weights for policy 1, policy_version 932912 (0.0006) [2023-12-27 12:07:00,261][20347] Updated weights for policy 0, policy_version 933729 (0.0007) [2023-12-27 12:07:00,283][20381] Updated weights for policy 1, policy_version 932922 (0.0008) [2023-12-27 12:07:00,320][20347] Updated weights for policy 0, policy_version 933739 (0.0008) [2023-12-27 12:07:00,346][20381] Updated weights for policy 1, policy_version 932932 (0.0008) [2023-12-27 12:07:00,377][20347] Updated weights for policy 0, policy_version 933749 (0.0009) [2023-12-27 12:07:00,405][20381] Updated weights for policy 1, policy_version 932942 (0.0005) [2023-12-27 12:07:00,430][20347] Updated weights for policy 0, policy_version 933759 (0.0008) [2023-12-27 12:07:00,461][20381] Updated weights for policy 1, policy_version 932952 (0.0006) [2023-12-27 12:07:01,012][20347] Updated weights for policy 0, policy_version 933769 (0.0006) [2023-12-27 12:07:01,069][20347] Updated weights for policy 0, policy_version 933779 (0.0010) [2023-12-27 12:07:01,122][20347] Updated weights for policy 0, policy_version 933789 (0.0010) [2023-12-27 12:07:01,160][20381] Updated weights for policy 1, policy_version 932962 (0.0006) [2023-12-27 12:07:01,219][20381] Updated weights for policy 1, policy_version 932972 (0.0008) [2023-12-27 12:07:01,278][20381] Updated weights for policy 1, policy_version 932982 (0.0007) [2023-12-27 12:07:01,797][20347] Updated weights for policy 0, policy_version 933799 (0.0008) [2023-12-27 12:07:01,860][20347] Updated weights for policy 0, policy_version 933809 (0.0005) [2023-12-27 12:07:01,922][20347] Updated weights for policy 0, policy_version 933819 (0.0006) [2023-12-27 12:07:02,048][20381] Updated weights for policy 1, policy_version 932992 (0.0007) [2023-12-27 12:07:02,105][20381] Updated weights for policy 1, policy_version 933002 (0.0008) [2023-12-27 12:07:02,161][20381] Updated weights for policy 1, policy_version 933012 (0.0007) [2023-12-27 12:07:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 477978624. Throughput: 0: 10163.2, 1: 10270.4. Samples: 477987340. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:02,363][19170] Avg episode reward: [(0, '912.150'), (1, '862.930')] [2023-12-27 12:07:02,522][20347] Updated weights for policy 0, policy_version 933829 (0.0007) [2023-12-27 12:07:02,579][20347] Updated weights for policy 0, policy_version 933839 (0.0009) [2023-12-27 12:07:02,637][20347] Updated weights for policy 0, policy_version 933849 (0.0006) [2023-12-27 12:07:02,836][20381] Updated weights for policy 1, policy_version 933022 (0.0009) [2023-12-27 12:07:02,897][20381] Updated weights for policy 1, policy_version 933032 (0.0008) [2023-12-27 12:07:02,951][20381] Updated weights for policy 1, policy_version 933042 (0.0007) [2023-12-27 12:07:03,276][20347] Updated weights for policy 0, policy_version 933859 (0.0007) [2023-12-27 12:07:03,324][20347] Updated weights for policy 0, policy_version 933869 (0.0010) [2023-12-27 12:07:03,382][20347] Updated weights for policy 0, policy_version 933879 (0.0010) [2023-12-27 12:07:03,614][20381] Updated weights for policy 1, policy_version 933052 (0.0007) [2023-12-27 12:07:03,674][20381] Updated weights for policy 1, policy_version 933062 (0.0006) [2023-12-27 12:07:03,725][20381] Updated weights for policy 1, policy_version 933072 (0.0008) [2023-12-27 12:07:04,046][20347] Updated weights for policy 0, policy_version 933889 (0.0010) [2023-12-27 12:07:04,108][20347] Updated weights for policy 0, policy_version 933899 (0.0006) [2023-12-27 12:07:04,169][20347] Updated weights for policy 0, policy_version 933909 (0.0006) [2023-12-27 12:07:04,230][20347] Updated weights for policy 0, policy_version 933919 (0.0008) [2023-12-27 12:07:04,460][20381] Updated weights for policy 1, policy_version 933082 (0.0007) [2023-12-27 12:07:04,517][20381] Updated weights for policy 1, policy_version 933092 (0.0008) [2023-12-27 12:07:04,578][20381] Updated weights for policy 1, policy_version 933102 (0.0007) [2023-12-27 12:07:04,638][20381] Updated weights for policy 1, policy_version 933112 (0.0007) [2023-12-27 12:07:04,907][20347] Updated weights for policy 0, policy_version 933929 (0.0007) [2023-12-27 12:07:04,968][20347] Updated weights for policy 0, policy_version 933939 (0.0008) [2023-12-27 12:07:05,022][20347] Updated weights for policy 0, policy_version 933949 (0.0007) [2023-12-27 12:07:05,308][20381] Updated weights for policy 1, policy_version 933122 (0.0007) [2023-12-27 12:07:05,355][20381] Updated weights for policy 1, policy_version 933132 (0.0008) [2023-12-27 12:07:05,405][20381] Updated weights for policy 1, policy_version 933142 (0.0007) [2023-12-27 12:07:05,723][20347] Updated weights for policy 0, policy_version 933959 (0.0008) [2023-12-27 12:07:05,776][20347] Updated weights for policy 0, policy_version 933969 (0.0007) [2023-12-27 12:07:05,830][20347] Updated weights for policy 0, policy_version 933979 (0.0006) [2023-12-27 12:07:06,044][20381] Updated weights for policy 1, policy_version 933152 (0.0007) [2023-12-27 12:07:06,113][20381] Updated weights for policy 1, policy_version 933162 (0.0007) [2023-12-27 12:07:06,183][20381] Updated weights for policy 1, policy_version 933172 (0.0007) [2023-12-27 12:07:06,536][20347] Updated weights for policy 0, policy_version 933989 (0.0007) [2023-12-27 12:07:06,600][20347] Updated weights for policy 0, policy_version 933999 (0.0008) [2023-12-27 12:07:06,670][20347] Updated weights for policy 0, policy_version 934009 (0.0008) [2023-12-27 12:07:06,842][20381] Updated weights for policy 1, policy_version 933182 (0.0007) [2023-12-27 12:07:06,903][20381] Updated weights for policy 1, policy_version 933192 (0.0008) [2023-12-27 12:07:06,964][20381] Updated weights for policy 1, policy_version 933202 (0.0007) [2023-12-27 12:07:07,310][20347] Updated weights for policy 0, policy_version 934019 (0.0008) [2023-12-27 12:07:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 478076928. Throughput: 0: 10209.5, 1: 10254.2. Samples: 478049248. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:07,363][19170] Avg episode reward: [(0, '902.230'), (1, '891.420')] [2023-12-27 12:07:07,367][20347] Updated weights for policy 0, policy_version 934029 (0.0008) [2023-12-27 12:07:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000933208_238927872.pth... [2023-12-27 12:07:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000932024_238624768.pth [2023-12-27 12:07:07,426][20347] Updated weights for policy 0, policy_version 934039 (0.0008) [2023-12-27 12:07:07,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000934048_239157248.pth... [2023-12-27 12:07:07,478][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000932832_238845952.pth [2023-12-27 12:07:07,652][20381] Updated weights for policy 1, policy_version 933212 (0.0007) [2023-12-27 12:07:07,704][20381] Updated weights for policy 1, policy_version 933222 (0.0008) [2023-12-27 12:07:07,758][20381] Updated weights for policy 1, policy_version 933232 (0.0008) [2023-12-27 12:07:08,044][20347] Updated weights for policy 0, policy_version 934049 (0.0008) [2023-12-27 12:07:08,095][20347] Updated weights for policy 0, policy_version 934059 (0.0009) [2023-12-27 12:07:08,142][20347] Updated weights for policy 0, policy_version 934069 (0.0008) [2023-12-27 12:07:08,194][20347] Updated weights for policy 0, policy_version 934079 (0.0007) [2023-12-27 12:07:08,406][20381] Updated weights for policy 1, policy_version 933242 (0.0008) [2023-12-27 12:07:08,456][20381] Updated weights for policy 1, policy_version 933252 (0.0006) [2023-12-27 12:07:08,506][20381] Updated weights for policy 1, policy_version 933262 (0.0007) [2023-12-27 12:07:08,555][20381] Updated weights for policy 1, policy_version 933272 (0.0008) [2023-12-27 12:07:08,943][20347] Updated weights for policy 0, policy_version 934089 (0.0010) [2023-12-27 12:07:08,998][20347] Updated weights for policy 0, policy_version 934099 (0.0006) [2023-12-27 12:07:09,051][20347] Updated weights for policy 0, policy_version 934109 (0.0005) [2023-12-27 12:07:09,290][20381] Updated weights for policy 1, policy_version 933282 (0.0007) [2023-12-27 12:07:09,349][20381] Updated weights for policy 1, policy_version 933292 (0.0008) [2023-12-27 12:07:09,417][20381] Updated weights for policy 1, policy_version 933302 (0.0008) [2023-12-27 12:07:09,696][20347] Updated weights for policy 0, policy_version 934119 (0.0007) [2023-12-27 12:07:09,754][20347] Updated weights for policy 0, policy_version 934129 (0.0005) [2023-12-27 12:07:09,824][20347] Updated weights for policy 0, policy_version 934139 (0.0006) [2023-12-27 12:07:10,174][20381] Updated weights for policy 1, policy_version 933312 (0.0007) [2023-12-27 12:07:10,237][20381] Updated weights for policy 1, policy_version 933322 (0.0009) [2023-12-27 12:07:10,301][20381] Updated weights for policy 1, policy_version 933332 (0.0009) [2023-12-27 12:07:10,391][20347] Updated weights for policy 0, policy_version 934149 (0.0007) [2023-12-27 12:07:10,450][20347] Updated weights for policy 0, policy_version 934159 (0.0008) [2023-12-27 12:07:10,515][20347] Updated weights for policy 0, policy_version 934169 (0.0007) [2023-12-27 12:07:10,954][20381] Updated weights for policy 1, policy_version 933342 (0.0006) [2023-12-27 12:07:11,019][20381] Updated weights for policy 1, policy_version 933352 (0.0006) [2023-12-27 12:07:11,083][20381] Updated weights for policy 1, policy_version 933362 (0.0008) [2023-12-27 12:07:11,305][20347] Updated weights for policy 0, policy_version 934179 (0.0006) [2023-12-27 12:07:11,371][20347] Updated weights for policy 0, policy_version 934189 (0.0008) [2023-12-27 12:07:11,431][20347] Updated weights for policy 0, policy_version 934199 (0.0008) [2023-12-27 12:07:11,762][20381] Updated weights for policy 1, policy_version 933372 (0.0007) [2023-12-27 12:07:11,823][20381] Updated weights for policy 1, policy_version 933382 (0.0008) [2023-12-27 12:07:11,886][20381] Updated weights for policy 1, policy_version 933392 (0.0006) [2023-12-27 12:07:12,192][20347] Updated weights for policy 0, policy_version 934209 (0.0008) [2023-12-27 12:07:12,252][20347] Updated weights for policy 0, policy_version 934219 (0.0007) [2023-12-27 12:07:12,317][20347] Updated weights for policy 0, policy_version 934229 (0.0008) [2023-12-27 12:07:12,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20382.8). Total num frames: 478175232. Throughput: 0: 10212.1, 1: 10186.4. Samples: 478170996. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:12,363][19170] Avg episode reward: [(0, '902.410'), (1, '909.850')] [2023-12-27 12:07:12,389][20347] Updated weights for policy 0, policy_version 934239 (0.0008) [2023-12-27 12:07:12,541][20381] Updated weights for policy 1, policy_version 933402 (0.0006) [2023-12-27 12:07:12,601][20381] Updated weights for policy 1, policy_version 933412 (0.0007) [2023-12-27 12:07:12,658][20381] Updated weights for policy 1, policy_version 933422 (0.0007) [2023-12-27 12:07:12,715][20381] Updated weights for policy 1, policy_version 933432 (0.0006) [2023-12-27 12:07:13,041][20347] Updated weights for policy 0, policy_version 934249 (0.0008) [2023-12-27 12:07:13,094][20347] Updated weights for policy 0, policy_version 934259 (0.0006) [2023-12-27 12:07:13,142][20347] Updated weights for policy 0, policy_version 934269 (0.0005) [2023-12-27 12:07:13,433][20381] Updated weights for policy 1, policy_version 933442 (0.0008) [2023-12-27 12:07:13,486][20381] Updated weights for policy 1, policy_version 933452 (0.0008) [2023-12-27 12:07:13,545][20381] Updated weights for policy 1, policy_version 933462 (0.0008) [2023-12-27 12:07:13,740][20347] Updated weights for policy 0, policy_version 934279 (0.0007) [2023-12-27 12:07:13,797][20347] Updated weights for policy 0, policy_version 934289 (0.0008) [2023-12-27 12:07:13,858][20347] Updated weights for policy 0, policy_version 934299 (0.0007) [2023-12-27 12:07:14,279][20381] Updated weights for policy 1, policy_version 933472 (0.0008) [2023-12-27 12:07:14,332][20381] Updated weights for policy 1, policy_version 933482 (0.0009) [2023-12-27 12:07:14,381][20381] Updated weights for policy 1, policy_version 933492 (0.0009) [2023-12-27 12:07:14,492][20347] Updated weights for policy 0, policy_version 934309 (0.0007) [2023-12-27 12:07:14,563][20347] Updated weights for policy 0, policy_version 934319 (0.0007) [2023-12-27 12:07:14,623][20347] Updated weights for policy 0, policy_version 934329 (0.0008) [2023-12-27 12:07:15,135][20381] Updated weights for policy 1, policy_version 933502 (0.0006) [2023-12-27 12:07:15,201][20381] Updated weights for policy 1, policy_version 933512 (0.0006) [2023-12-27 12:07:15,264][20381] Updated weights for policy 1, policy_version 933522 (0.0008) [2023-12-27 12:07:15,303][20347] Updated weights for policy 0, policy_version 934339 (0.0009) [2023-12-27 12:07:15,356][20347] Updated weights for policy 0, policy_version 934349 (0.0011) [2023-12-27 12:07:15,404][20347] Updated weights for policy 0, policy_version 934359 (0.0010) [2023-12-27 12:07:15,907][20381] Updated weights for policy 1, policy_version 933532 (0.0009) [2023-12-27 12:07:15,973][20381] Updated weights for policy 1, policy_version 933542 (0.0005) [2023-12-27 12:07:16,035][20381] Updated weights for policy 1, policy_version 933552 (0.0007) [2023-12-27 12:07:16,075][20347] Updated weights for policy 0, policy_version 934369 (0.0010) [2023-12-27 12:07:16,124][20347] Updated weights for policy 0, policy_version 934379 (0.0010) [2023-12-27 12:07:16,175][20347] Updated weights for policy 0, policy_version 934389 (0.0010) [2023-12-27 12:07:16,236][20347] Updated weights for policy 0, policy_version 934399 (0.0010) [2023-12-27 12:07:16,566][20381] Updated weights for policy 1, policy_version 933562 (0.0009) [2023-12-27 12:07:16,621][20381] Updated weights for policy 1, policy_version 933572 (0.0011) [2023-12-27 12:07:16,672][20381] Updated weights for policy 1, policy_version 933582 (0.0010) [2023-12-27 12:07:16,723][20381] Updated weights for policy 1, policy_version 933592 (0.0007) [2023-12-27 12:07:16,860][20291] KL-divergence is very high: 201.9662 [2023-12-27 12:07:16,866][20347] Updated weights for policy 0, policy_version 934409 (0.0008) [2023-12-27 12:07:16,883][20291] KL-divergence is very high: 461.3503 [2023-12-27 12:07:16,907][20291] KL-divergence is very high: 474.2783 [2023-12-27 12:07:16,924][20347] Updated weights for policy 0, policy_version 934419 (0.0008) [2023-12-27 12:07:16,930][20291] KL-divergence is very high: 381.8126 [2023-12-27 12:07:16,952][20291] KL-divergence is very high: 191.5716 [2023-12-27 12:07:16,978][20347] Updated weights for policy 0, policy_version 934429 (0.0007) [2023-12-27 12:07:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20410.7). Total num frames: 478281728. Throughput: 0: 10186.2, 1: 10163.5. Samples: 478294272. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:17,363][19170] Avg episode reward: [(0, '921.900'), (1, '891.450')] [2023-12-27 12:07:17,379][20381] Updated weights for policy 1, policy_version 933602 (0.0006) [2023-12-27 12:07:17,430][20381] Updated weights for policy 1, policy_version 933612 (0.0005) [2023-12-27 12:07:17,490][20381] Updated weights for policy 1, policy_version 933622 (0.0006) [2023-12-27 12:07:17,607][20347] Updated weights for policy 0, policy_version 934439 (0.0006) [2023-12-27 12:07:17,664][20347] Updated weights for policy 0, policy_version 934449 (0.0007) [2023-12-27 12:07:17,721][20347] Updated weights for policy 0, policy_version 934459 (0.0006) [2023-12-27 12:07:18,177][20381] Updated weights for policy 1, policy_version 933632 (0.0008) [2023-12-27 12:07:18,239][20381] Updated weights for policy 1, policy_version 933642 (0.0007) [2023-12-27 12:07:18,251][20347] Updated weights for policy 0, policy_version 934469 (0.0005) [2023-12-27 12:07:18,299][20381] Updated weights for policy 1, policy_version 933652 (0.0008) [2023-12-27 12:07:18,314][20347] Updated weights for policy 0, policy_version 934479 (0.0006) [2023-12-27 12:07:18,383][20347] Updated weights for policy 0, policy_version 934489 (0.0007) [2023-12-27 12:07:18,973][20347] Updated weights for policy 0, policy_version 934499 (0.0009) [2023-12-27 12:07:19,037][20347] Updated weights for policy 0, policy_version 934509 (0.0008) [2023-12-27 12:07:19,068][20381] Updated weights for policy 1, policy_version 933662 (0.0006) [2023-12-27 12:07:19,099][20347] Updated weights for policy 0, policy_version 934519 (0.0008) [2023-12-27 12:07:19,122][20381] Updated weights for policy 1, policy_version 933672 (0.0007) [2023-12-27 12:07:19,176][20381] Updated weights for policy 1, policy_version 933682 (0.0006) [2023-12-27 12:07:19,705][20347] Updated weights for policy 0, policy_version 934529 (0.0006) [2023-12-27 12:07:19,761][20347] Updated weights for policy 0, policy_version 934539 (0.0007) [2023-12-27 12:07:19,818][20347] Updated weights for policy 0, policy_version 934549 (0.0008) [2023-12-27 12:07:19,880][20347] Updated weights for policy 0, policy_version 934559 (0.0006) [2023-12-27 12:07:19,944][20381] Updated weights for policy 1, policy_version 933692 (0.0008) [2023-12-27 12:07:20,002][20381] Updated weights for policy 1, policy_version 933702 (0.0009) [2023-12-27 12:07:20,061][20381] Updated weights for policy 1, policy_version 933712 (0.0009) [2023-12-27 12:07:20,537][20347] Updated weights for policy 0, policy_version 934569 (0.0007) [2023-12-27 12:07:20,595][20347] Updated weights for policy 0, policy_version 934579 (0.0008) [2023-12-27 12:07:20,651][20347] Updated weights for policy 0, policy_version 934589 (0.0009) [2023-12-27 12:07:20,717][20381] Updated weights for policy 1, policy_version 933722 (0.0007) [2023-12-27 12:07:20,775][20381] Updated weights for policy 1, policy_version 933732 (0.0008) [2023-12-27 12:07:20,832][20381] Updated weights for policy 1, policy_version 933742 (0.0008) [2023-12-27 12:07:20,886][20381] Updated weights for policy 1, policy_version 933752 (0.0008) [2023-12-27 12:07:21,391][20347] Updated weights for policy 0, policy_version 934599 (0.0008) [2023-12-27 12:07:21,460][20347] Updated weights for policy 0, policy_version 934609 (0.0006) [2023-12-27 12:07:21,522][20347] Updated weights for policy 0, policy_version 934619 (0.0006) [2023-12-27 12:07:21,650][20381] Updated weights for policy 1, policy_version 933762 (0.0007) [2023-12-27 12:07:21,725][20381] Updated weights for policy 1, policy_version 933772 (0.0007) [2023-12-27 12:07:21,791][20381] Updated weights for policy 1, policy_version 933782 (0.0007) [2023-12-27 12:07:22,193][20347] Updated weights for policy 0, policy_version 934629 (0.0007) [2023-12-27 12:07:22,240][20347] Updated weights for policy 0, policy_version 934639 (0.0006) [2023-12-27 12:07:22,307][20347] Updated weights for policy 0, policy_version 934649 (0.0006) [2023-12-27 12:07:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 478388224. Throughput: 0: 10246.4, 1: 10112.2. Samples: 478356916. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:22,363][19170] Avg episode reward: [(0, '922.210'), (1, '872.860')] [2023-12-27 12:07:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000933784_239075328.pth... [2023-12-27 12:07:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000934656_239312896.pth... [2023-12-27 12:07:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000932600_238772224.pth [2023-12-27 12:07:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000933440_239001600.pth [2023-12-27 12:07:22,483][20381] Updated weights for policy 1, policy_version 933792 (0.0008) [2023-12-27 12:07:22,538][20381] Updated weights for policy 1, policy_version 933802 (0.0008) [2023-12-27 12:07:22,594][20381] Updated weights for policy 1, policy_version 933812 (0.0008) [2023-12-27 12:07:22,936][20347] Updated weights for policy 0, policy_version 934659 (0.0007) [2023-12-27 12:07:22,995][20347] Updated weights for policy 0, policy_version 934669 (0.0005) [2023-12-27 12:07:23,051][20347] Updated weights for policy 0, policy_version 934679 (0.0005) [2023-12-27 12:07:23,306][20381] Updated weights for policy 1, policy_version 933822 (0.0009) [2023-12-27 12:07:23,364][20381] Updated weights for policy 1, policy_version 933832 (0.0010) [2023-12-27 12:07:23,422][20381] Updated weights for policy 1, policy_version 933842 (0.0010) [2023-12-27 12:07:23,653][20347] Updated weights for policy 0, policy_version 934689 (0.0006) [2023-12-27 12:07:23,709][20347] Updated weights for policy 0, policy_version 934699 (0.0008) [2023-12-27 12:07:23,761][20347] Updated weights for policy 0, policy_version 934709 (0.0009) [2023-12-27 12:07:23,806][20347] Updated weights for policy 0, policy_version 934719 (0.0009) [2023-12-27 12:07:24,047][20381] Updated weights for policy 1, policy_version 933852 (0.0008) [2023-12-27 12:07:24,098][20381] Updated weights for policy 1, policy_version 933862 (0.0006) [2023-12-27 12:07:24,151][20381] Updated weights for policy 1, policy_version 933872 (0.0006) [2023-12-27 12:07:24,496][20347] Updated weights for policy 0, policy_version 934729 (0.0007) [2023-12-27 12:07:24,552][20347] Updated weights for policy 0, policy_version 934739 (0.0007) [2023-12-27 12:07:24,606][20347] Updated weights for policy 0, policy_version 934749 (0.0007) [2023-12-27 12:07:24,758][20381] Updated weights for policy 1, policy_version 933882 (0.0006) [2023-12-27 12:07:24,823][20381] Updated weights for policy 1, policy_version 933892 (0.0005) [2023-12-27 12:07:24,884][20381] Updated weights for policy 1, policy_version 933902 (0.0006) [2023-12-27 12:07:24,934][20381] Updated weights for policy 1, policy_version 933912 (0.0010) [2023-12-27 12:07:25,289][20347] Updated weights for policy 0, policy_version 934759 (0.0007) [2023-12-27 12:07:25,335][20347] Updated weights for policy 0, policy_version 934769 (0.0008) [2023-12-27 12:07:25,384][20347] Updated weights for policy 0, policy_version 934779 (0.0007) [2023-12-27 12:07:25,578][20381] Updated weights for policy 1, policy_version 933922 (0.0008) [2023-12-27 12:07:25,623][20381] Updated weights for policy 1, policy_version 933932 (0.0008) [2023-12-27 12:07:25,682][20381] Updated weights for policy 1, policy_version 933942 (0.0008) [2023-12-27 12:07:26,045][20347] Updated weights for policy 0, policy_version 934789 (0.0006) [2023-12-27 12:07:26,100][20347] Updated weights for policy 0, policy_version 934799 (0.0007) [2023-12-27 12:07:26,158][20347] Updated weights for policy 0, policy_version 934809 (0.0005) [2023-12-27 12:07:26,381][20381] Updated weights for policy 1, policy_version 933952 (0.0008) [2023-12-27 12:07:26,428][20381] Updated weights for policy 1, policy_version 933962 (0.0008) [2023-12-27 12:07:26,481][20381] Updated weights for policy 1, policy_version 933972 (0.0008) [2023-12-27 12:07:26,788][20347] Updated weights for policy 0, policy_version 934819 (0.0006) [2023-12-27 12:07:26,835][20347] Updated weights for policy 0, policy_version 934829 (0.0005) [2023-12-27 12:07:26,876][20291] KL-divergence is very high: 737.5903 [2023-12-27 12:07:26,881][20347] Updated weights for policy 0, policy_version 934839 (0.0005) [2023-12-27 12:07:26,881][20291] KL-divergence is very high: 703.4501 [2023-12-27 12:07:26,913][20291] KL-divergence is very high: 119.1914 [2023-12-27 12:07:27,115][20381] Updated weights for policy 1, policy_version 933982 (0.0006) [2023-12-27 12:07:27,157][20381] Updated weights for policy 1, policy_version 933992 (0.0005) [2023-12-27 12:07:27,203][20381] Updated weights for policy 1, policy_version 934002 (0.0005) [2023-12-27 12:07:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 478494720. Throughput: 0: 10308.5, 1: 10143.9. Samples: 478480888. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:27,364][19170] Avg episode reward: [(0, '893.010'), (1, '872.760')] [2023-12-27 12:07:27,529][20347] Updated weights for policy 0, policy_version 934849 (0.0006) [2023-12-27 12:07:27,589][20347] Updated weights for policy 0, policy_version 934859 (0.0005) [2023-12-27 12:07:27,649][20347] Updated weights for policy 0, policy_version 934869 (0.0005) [2023-12-27 12:07:27,710][20347] Updated weights for policy 0, policy_version 934879 (0.0005) [2023-12-27 12:07:27,802][20381] Updated weights for policy 1, policy_version 934012 (0.0005) [2023-12-27 12:07:27,858][20381] Updated weights for policy 1, policy_version 934022 (0.0007) [2023-12-27 12:07:27,905][20381] Updated weights for policy 1, policy_version 934032 (0.0010) [2023-12-27 12:07:28,291][20347] Updated weights for policy 0, policy_version 934889 (0.0005) [2023-12-27 12:07:28,355][20347] Updated weights for policy 0, policy_version 934899 (0.0007) [2023-12-27 12:07:28,407][20347] Updated weights for policy 0, policy_version 934909 (0.0008) [2023-12-27 12:07:28,560][20381] Updated weights for policy 1, policy_version 934042 (0.0010) [2023-12-27 12:07:28,617][20381] Updated weights for policy 1, policy_version 934052 (0.0011) [2023-12-27 12:07:28,683][20381] Updated weights for policy 1, policy_version 934062 (0.0011) [2023-12-27 12:07:28,738][20381] Updated weights for policy 1, policy_version 934072 (0.0011) [2023-12-27 12:07:29,084][20347] Updated weights for policy 0, policy_version 934919 (0.0008) [2023-12-27 12:07:29,135][20347] Updated weights for policy 0, policy_version 934929 (0.0009) [2023-12-27 12:07:29,194][20347] Updated weights for policy 0, policy_version 934939 (0.0008) [2023-12-27 12:07:29,322][20381] Updated weights for policy 1, policy_version 934082 (0.0008) [2023-12-27 12:07:29,384][20381] Updated weights for policy 1, policy_version 934092 (0.0009) [2023-12-27 12:07:29,445][20381] Updated weights for policy 1, policy_version 934102 (0.0006) [2023-12-27 12:07:29,957][20347] Updated weights for policy 0, policy_version 934949 (0.0007) [2023-12-27 12:07:30,014][20347] Updated weights for policy 0, policy_version 934959 (0.0008) [2023-12-27 12:07:30,071][20347] Updated weights for policy 0, policy_version 934969 (0.0007) [2023-12-27 12:07:30,103][20381] Updated weights for policy 1, policy_version 934112 (0.0008) [2023-12-27 12:07:30,161][20381] Updated weights for policy 1, policy_version 934122 (0.0008) [2023-12-27 12:07:30,226][20381] Updated weights for policy 1, policy_version 934132 (0.0008) [2023-12-27 12:07:30,748][20347] Updated weights for policy 0, policy_version 934979 (0.0006) [2023-12-27 12:07:30,808][20347] Updated weights for policy 0, policy_version 934989 (0.0008) [2023-12-27 12:07:30,864][20347] Updated weights for policy 0, policy_version 934999 (0.0007) [2023-12-27 12:07:30,875][20381] Updated weights for policy 1, policy_version 934142 (0.0005) [2023-12-27 12:07:30,927][20381] Updated weights for policy 1, policy_version 934152 (0.0007) [2023-12-27 12:07:30,977][20381] Updated weights for policy 1, policy_version 934162 (0.0010) [2023-12-27 12:07:31,565][20347] Updated weights for policy 0, policy_version 935009 (0.0007) [2023-12-27 12:07:31,629][20347] Updated weights for policy 0, policy_version 935019 (0.0010) [2023-12-27 12:07:31,634][20381] Updated weights for policy 1, policy_version 934172 (0.0009) [2023-12-27 12:07:31,692][20347] Updated weights for policy 0, policy_version 935029 (0.0007) [2023-12-27 12:07:31,693][20381] Updated weights for policy 1, policy_version 934182 (0.0007) [2023-12-27 12:07:31,755][20347] Updated weights for policy 0, policy_version 935039 (0.0007) [2023-12-27 12:07:31,761][20381] Updated weights for policy 1, policy_version 934192 (0.0007) [2023-12-27 12:07:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20382.8). Total num frames: 478593024. Throughput: 0: 10285.9, 1: 10247.7. Samples: 478607484. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:32,363][19170] Avg episode reward: [(0, '902.470'), (1, '872.720')] [2023-12-27 12:07:32,400][20381] Updated weights for policy 1, policy_version 934202 (0.0008) [2023-12-27 12:07:32,459][20381] Updated weights for policy 1, policy_version 934212 (0.0006) [2023-12-27 12:07:32,495][20347] Updated weights for policy 0, policy_version 935049 (0.0007) [2023-12-27 12:07:32,521][20381] Updated weights for policy 1, policy_version 934222 (0.0005) [2023-12-27 12:07:32,558][20347] Updated weights for policy 0, policy_version 935059 (0.0007) [2023-12-27 12:07:32,577][20381] Updated weights for policy 1, policy_version 934232 (0.0005) [2023-12-27 12:07:32,612][20347] Updated weights for policy 0, policy_version 935069 (0.0007) [2023-12-27 12:07:33,168][20381] Updated weights for policy 1, policy_version 934242 (0.0010) [2023-12-27 12:07:33,215][20381] Updated weights for policy 1, policy_version 934252 (0.0008) [2023-12-27 12:07:33,270][20347] Updated weights for policy 0, policy_version 935079 (0.0006) [2023-12-27 12:07:33,271][20381] Updated weights for policy 1, policy_version 934262 (0.0006) [2023-12-27 12:07:33,324][20347] Updated weights for policy 0, policy_version 935089 (0.0005) [2023-12-27 12:07:33,382][20347] Updated weights for policy 0, policy_version 935099 (0.0005) [2023-12-27 12:07:33,877][20381] Updated weights for policy 1, policy_version 934272 (0.0008) [2023-12-27 12:07:33,940][20381] Updated weights for policy 1, policy_version 934282 (0.0006) [2023-12-27 12:07:33,952][20347] Updated weights for policy 0, policy_version 935109 (0.0005) [2023-12-27 12:07:34,002][20381] Updated weights for policy 1, policy_version 934292 (0.0005) [2023-12-27 12:07:34,006][20347] Updated weights for policy 0, policy_version 935119 (0.0005) [2023-12-27 12:07:34,063][20347] Updated weights for policy 0, policy_version 935129 (0.0005) [2023-12-27 12:07:34,657][20381] Updated weights for policy 1, policy_version 934302 (0.0009) [2023-12-27 12:07:34,721][20381] Updated weights for policy 1, policy_version 934312 (0.0011) [2023-12-27 12:07:34,723][20347] Updated weights for policy 0, policy_version 935139 (0.0008) [2023-12-27 12:07:34,777][20381] Updated weights for policy 1, policy_version 934322 (0.0010) [2023-12-27 12:07:34,779][20347] Updated weights for policy 0, policy_version 935149 (0.0006) [2023-12-27 12:07:34,833][20347] Updated weights for policy 0, policy_version 935159 (0.0010) [2023-12-27 12:07:35,429][20347] Updated weights for policy 0, policy_version 935169 (0.0010) [2023-12-27 12:07:35,471][20381] Updated weights for policy 1, policy_version 934332 (0.0008) [2023-12-27 12:07:35,491][20347] Updated weights for policy 0, policy_version 935179 (0.0008) [2023-12-27 12:07:35,527][20381] Updated weights for policy 1, policy_version 934342 (0.0006) [2023-12-27 12:07:35,544][20347] Updated weights for policy 0, policy_version 935189 (0.0010) [2023-12-27 12:07:35,576][20381] Updated weights for policy 1, policy_version 934352 (0.0005) [2023-12-27 12:07:35,599][20347] Updated weights for policy 0, policy_version 935199 (0.0008) [2023-12-27 12:07:36,289][20381] Updated weights for policy 1, policy_version 934362 (0.0010) [2023-12-27 12:07:36,302][20347] Updated weights for policy 0, policy_version 935209 (0.0007) [2023-12-27 12:07:36,347][20381] Updated weights for policy 1, policy_version 934372 (0.0011) [2023-12-27 12:07:36,364][20347] Updated weights for policy 0, policy_version 935219 (0.0006) [2023-12-27 12:07:36,405][20381] Updated weights for policy 1, policy_version 934382 (0.0011) [2023-12-27 12:07:36,425][20347] Updated weights for policy 0, policy_version 935229 (0.0006) [2023-12-27 12:07:36,465][20381] Updated weights for policy 1, policy_version 934392 (0.0011) [2023-12-27 12:07:37,024][20347] Updated weights for policy 0, policy_version 935239 (0.0005) [2023-12-27 12:07:37,090][20347] Updated weights for policy 0, policy_version 935249 (0.0005) [2023-12-27 12:07:37,146][20347] Updated weights for policy 0, policy_version 935259 (0.0006) [2023-12-27 12:07:37,153][20381] Updated weights for policy 1, policy_version 934402 (0.0011) [2023-12-27 12:07:37,210][20381] Updated weights for policy 1, policy_version 934412 (0.0011) [2023-12-27 12:07:37,257][20381] Updated weights for policy 1, policy_version 934422 (0.0007) [2023-12-27 12:07:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 478707712. Throughput: 0: 10313.5, 1: 10298.3. Samples: 478671992. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 12:07:37,363][19170] Avg episode reward: [(0, '824.730'), (1, '882.540')] [2023-12-27 12:07:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000934424_239239168.pth... [2023-12-27 12:07:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000935264_239468544.pth... [2023-12-27 12:07:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000933208_238927872.pth [2023-12-27 12:07:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000934048_239157248.pth [2023-12-27 12:07:37,771][20347] Updated weights for policy 0, policy_version 935269 (0.0008) [2023-12-27 12:07:37,824][20347] Updated weights for policy 0, policy_version 935279 (0.0007) [2023-12-27 12:07:37,878][20347] Updated weights for policy 0, policy_version 935289 (0.0007) [2023-12-27 12:07:37,957][20381] Updated weights for policy 1, policy_version 934432 (0.0010) [2023-12-27 12:07:38,010][20381] Updated weights for policy 1, policy_version 934442 (0.0011) [2023-12-27 12:07:38,061][20381] Updated weights for policy 1, policy_version 934452 (0.0010) [2023-12-27 12:07:38,532][20347] Updated weights for policy 0, policy_version 935299 (0.0007) [2023-12-27 12:07:38,584][20347] Updated weights for policy 0, policy_version 935309 (0.0009) [2023-12-27 12:07:38,643][20347] Updated weights for policy 0, policy_version 935319 (0.0010) [2023-12-27 12:07:38,720][20381] Updated weights for policy 1, policy_version 934462 (0.0008) [2023-12-27 12:07:38,775][20381] Updated weights for policy 1, policy_version 934472 (0.0008) [2023-12-27 12:07:38,841][20381] Updated weights for policy 1, policy_version 934482 (0.0007) [2023-12-27 12:07:39,279][20347] Updated weights for policy 0, policy_version 935329 (0.0010) [2023-12-27 12:07:39,349][20347] Updated weights for policy 0, policy_version 935339 (0.0007) [2023-12-27 12:07:39,422][20347] Updated weights for policy 0, policy_version 935349 (0.0008) [2023-12-27 12:07:39,485][20347] Updated weights for policy 0, policy_version 935359 (0.0007) [2023-12-27 12:07:39,513][20381] Updated weights for policy 1, policy_version 934492 (0.0006) [2023-12-27 12:07:39,572][20381] Updated weights for policy 1, policy_version 934502 (0.0007) [2023-12-27 12:07:39,628][20381] Updated weights for policy 1, policy_version 934512 (0.0008) [2023-12-27 12:07:40,162][20347] Updated weights for policy 0, policy_version 935369 (0.0006) [2023-12-27 12:07:40,225][20347] Updated weights for policy 0, policy_version 935379 (0.0006) [2023-12-27 12:07:40,284][20347] Updated weights for policy 0, policy_version 935389 (0.0006) [2023-12-27 12:07:40,376][20381] Updated weights for policy 1, policy_version 934522 (0.0009) [2023-12-27 12:07:40,426][20381] Updated weights for policy 1, policy_version 934532 (0.0009) [2023-12-27 12:07:40,482][20381] Updated weights for policy 1, policy_version 934542 (0.0006) [2023-12-27 12:07:40,544][20381] Updated weights for policy 1, policy_version 934552 (0.0011) [2023-12-27 12:07:40,878][20347] Updated weights for policy 0, policy_version 935399 (0.0007) [2023-12-27 12:07:40,933][20347] Updated weights for policy 0, policy_version 935409 (0.0008) [2023-12-27 12:07:40,997][20347] Updated weights for policy 0, policy_version 935419 (0.0007) [2023-12-27 12:07:41,208][20381] Updated weights for policy 1, policy_version 934562 (0.0011) [2023-12-27 12:07:41,268][20381] Updated weights for policy 1, policy_version 934572 (0.0010) [2023-12-27 12:07:41,324][20381] Updated weights for policy 1, policy_version 934582 (0.0010) [2023-12-27 12:07:41,782][20347] Updated weights for policy 0, policy_version 935429 (0.0008) [2023-12-27 12:07:41,841][20347] Updated weights for policy 0, policy_version 935439 (0.0008) [2023-12-27 12:07:41,894][20347] Updated weights for policy 0, policy_version 935449 (0.0008) [2023-12-27 12:07:42,049][20381] Updated weights for policy 1, policy_version 934592 (0.0009) [2023-12-27 12:07:42,099][20381] Updated weights for policy 1, policy_version 934602 (0.0008) [2023-12-27 12:07:42,152][20381] Updated weights for policy 1, policy_version 934612 (0.0007) [2023-12-27 12:07:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 478806016. Throughput: 0: 10395.8, 1: 10260.3. Samples: 478795968. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:07:42,363][19170] Avg episode reward: [(0, '883.050'), (1, '872.860')] [2023-12-27 12:07:42,646][20347] Updated weights for policy 0, policy_version 935459 (0.0008) [2023-12-27 12:07:42,707][20347] Updated weights for policy 0, policy_version 935469 (0.0008) [2023-12-27 12:07:42,771][20347] Updated weights for policy 0, policy_version 935479 (0.0008) [2023-12-27 12:07:42,837][20381] Updated weights for policy 1, policy_version 934622 (0.0008) [2023-12-27 12:07:42,897][20381] Updated weights for policy 1, policy_version 934632 (0.0005) [2023-12-27 12:07:42,958][20381] Updated weights for policy 1, policy_version 934642 (0.0006) [2023-12-27 12:07:43,436][20347] Updated weights for policy 0, policy_version 935489 (0.0007) [2023-12-27 12:07:43,492][20347] Updated weights for policy 0, policy_version 935499 (0.0008) [2023-12-27 12:07:43,542][20347] Updated weights for policy 0, policy_version 935509 (0.0007) [2023-12-27 12:07:43,548][20381] Updated weights for policy 1, policy_version 934652 (0.0006) [2023-12-27 12:07:43,590][20347] Updated weights for policy 0, policy_version 935519 (0.0005) [2023-12-27 12:07:43,602][20381] Updated weights for policy 1, policy_version 934662 (0.0007) [2023-12-27 12:07:43,664][20381] Updated weights for policy 1, policy_version 934672 (0.0009) [2023-12-27 12:07:44,206][20347] Updated weights for policy 0, policy_version 935529 (0.0005) [2023-12-27 12:07:44,269][20347] Updated weights for policy 0, policy_version 935539 (0.0006) [2023-12-27 12:07:44,317][20347] Updated weights for policy 0, policy_version 935549 (0.0008) [2023-12-27 12:07:44,365][20381] Updated weights for policy 1, policy_version 934682 (0.0009) [2023-12-27 12:07:44,411][20381] Updated weights for policy 1, policy_version 934692 (0.0007) [2023-12-27 12:07:44,466][20381] Updated weights for policy 1, policy_version 934702 (0.0007) [2023-12-27 12:07:44,525][20381] Updated weights for policy 1, policy_version 934712 (0.0008) [2023-12-27 12:07:44,955][20347] Updated weights for policy 0, policy_version 935559 (0.0008) [2023-12-27 12:07:45,019][20347] Updated weights for policy 0, policy_version 935569 (0.0009) [2023-12-27 12:07:45,079][20347] Updated weights for policy 0, policy_version 935579 (0.0008) [2023-12-27 12:07:45,278][20381] Updated weights for policy 1, policy_version 934722 (0.0006) [2023-12-27 12:07:45,344][20381] Updated weights for policy 1, policy_version 934732 (0.0006) [2023-12-27 12:07:45,413][20381] Updated weights for policy 1, policy_version 934742 (0.0006) [2023-12-27 12:07:45,795][20347] Updated weights for policy 0, policy_version 935589 (0.0007) [2023-12-27 12:07:45,852][20347] Updated weights for policy 0, policy_version 935599 (0.0006) [2023-12-27 12:07:45,906][20347] Updated weights for policy 0, policy_version 935609 (0.0007) [2023-12-27 12:07:46,034][20381] Updated weights for policy 1, policy_version 934752 (0.0010) [2023-12-27 12:07:46,089][20381] Updated weights for policy 1, policy_version 934762 (0.0011) [2023-12-27 12:07:46,141][20381] Updated weights for policy 1, policy_version 934772 (0.0010) [2023-12-27 12:07:46,539][20347] Updated weights for policy 0, policy_version 935619 (0.0008) [2023-12-27 12:07:46,585][20347] Updated weights for policy 0, policy_version 935629 (0.0010) [2023-12-27 12:07:46,633][20347] Updated weights for policy 0, policy_version 935639 (0.0010) [2023-12-27 12:07:46,742][20381] Updated weights for policy 1, policy_version 934782 (0.0007) [2023-12-27 12:07:46,802][20381] Updated weights for policy 1, policy_version 934792 (0.0005) [2023-12-27 12:07:46,857][20381] Updated weights for policy 1, policy_version 934802 (0.0005) [2023-12-27 12:07:47,331][20347] Updated weights for policy 0, policy_version 935649 (0.0010) [2023-12-27 12:07:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 478904320. Throughput: 0: 10403.5, 1: 10316.2. Samples: 478919728. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:07:47,363][19170] Avg episode reward: [(0, '941.270'), (1, '872.870')] [2023-12-27 12:07:47,389][20347] Updated weights for policy 0, policy_version 935659 (0.0005) [2023-12-27 12:07:47,422][20381] Updated weights for policy 1, policy_version 934812 (0.0007) [2023-12-27 12:07:47,436][20347] Updated weights for policy 0, policy_version 935669 (0.0005) [2023-12-27 12:07:47,474][20381] Updated weights for policy 1, policy_version 934822 (0.0010) [2023-12-27 12:07:47,485][20347] Updated weights for policy 0, policy_version 935679 (0.0006) [2023-12-27 12:07:47,526][20381] Updated weights for policy 1, policy_version 934832 (0.0010) [2023-12-27 12:07:48,097][20347] Updated weights for policy 0, policy_version 935689 (0.0010) [2023-12-27 12:07:48,139][20381] Updated weights for policy 1, policy_version 934842 (0.0011) [2023-12-27 12:07:48,152][20347] Updated weights for policy 0, policy_version 935699 (0.0010) [2023-12-27 12:07:48,190][20381] Updated weights for policy 1, policy_version 934852 (0.0010) [2023-12-27 12:07:48,204][20347] Updated weights for policy 0, policy_version 935709 (0.0010) [2023-12-27 12:07:48,243][20381] Updated weights for policy 1, policy_version 934862 (0.0010) [2023-12-27 12:07:48,294][20381] Updated weights for policy 1, policy_version 934872 (0.0010) [2023-12-27 12:07:48,889][20347] Updated weights for policy 0, policy_version 935719 (0.0009) [2023-12-27 12:07:48,944][20347] Updated weights for policy 0, policy_version 935729 (0.0009) [2023-12-27 12:07:48,960][20381] Updated weights for policy 1, policy_version 934882 (0.0011) [2023-12-27 12:07:49,000][20347] Updated weights for policy 0, policy_version 935739 (0.0005) [2023-12-27 12:07:49,019][20381] Updated weights for policy 1, policy_version 934892 (0.0008) [2023-12-27 12:07:49,079][20381] Updated weights for policy 1, policy_version 934902 (0.0007) [2023-12-27 12:07:49,719][20347] Updated weights for policy 0, policy_version 935749 (0.0006) [2023-12-27 12:07:49,777][20381] Updated weights for policy 1, policy_version 934912 (0.0006) [2023-12-27 12:07:49,780][20347] Updated weights for policy 0, policy_version 935759 (0.0008) [2023-12-27 12:07:49,833][20381] Updated weights for policy 1, policy_version 934922 (0.0007) [2023-12-27 12:07:49,842][20347] Updated weights for policy 0, policy_version 935769 (0.0008) [2023-12-27 12:07:49,899][20381] Updated weights for policy 1, policy_version 934932 (0.0007) [2023-12-27 12:07:50,532][20381] Updated weights for policy 1, policy_version 934942 (0.0007) [2023-12-27 12:07:50,536][20347] Updated weights for policy 0, policy_version 935779 (0.0008) [2023-12-27 12:07:50,592][20381] Updated weights for policy 1, policy_version 934952 (0.0007) [2023-12-27 12:07:50,606][20347] Updated weights for policy 0, policy_version 935789 (0.0010) [2023-12-27 12:07:50,649][20381] Updated weights for policy 1, policy_version 934962 (0.0006) [2023-12-27 12:07:50,665][20347] Updated weights for policy 0, policy_version 935799 (0.0010) [2023-12-27 12:07:51,372][20347] Updated weights for policy 0, policy_version 935809 (0.0010) [2023-12-27 12:07:51,432][20347] Updated weights for policy 0, policy_version 935819 (0.0008) [2023-12-27 12:07:51,433][20381] Updated weights for policy 1, policy_version 934972 (0.0008) [2023-12-27 12:07:51,490][20347] Updated weights for policy 0, policy_version 935829 (0.0006) [2023-12-27 12:07:51,492][20381] Updated weights for policy 1, policy_version 934982 (0.0010) [2023-12-27 12:07:51,548][20381] Updated weights for policy 1, policy_version 934992 (0.0011) [2023-12-27 12:07:51,553][20347] Updated weights for policy 0, policy_version 935839 (0.0005) [2023-12-27 12:07:52,168][20381] Updated weights for policy 1, policy_version 935002 (0.0010) [2023-12-27 12:07:52,204][20292] KL-divergence is very high: 148.0852 [2023-12-27 12:07:52,222][20292] KL-divergence is very high: 151.6029 [2023-12-27 12:07:52,226][20381] Updated weights for policy 1, policy_version 935012 (0.0007) [2023-12-27 12:07:52,233][20292] KL-divergence is very high: 120.8864 [2023-12-27 12:07:52,261][20347] Updated weights for policy 0, policy_version 935849 (0.0006) [2023-12-27 12:07:52,285][20381] Updated weights for policy 1, policy_version 935022 (0.0008) [2023-12-27 12:07:52,322][20347] Updated weights for policy 0, policy_version 935859 (0.0007) [2023-12-27 12:07:52,333][20381] Updated weights for policy 1, policy_version 935032 (0.0005) [2023-12-27 12:07:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 479010816. Throughput: 0: 10385.1, 1: 10334.0. Samples: 478981604. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:07:52,363][19170] Avg episode reward: [(0, '941.510'), (1, '853.200')] [2023-12-27 12:07:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000935032_239394816.pth... [2023-12-27 12:07:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000933784_239075328.pth [2023-12-27 12:07:52,386][20347] Updated weights for policy 0, policy_version 935869 (0.0008) [2023-12-27 12:07:52,405][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000935872_239624192.pth... [2023-12-27 12:07:52,410][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000934656_239312896.pth [2023-12-27 12:07:52,995][20381] Updated weights for policy 1, policy_version 935042 (0.0009) [2023-12-27 12:07:53,022][20347] Updated weights for policy 0, policy_version 935879 (0.0006) [2023-12-27 12:07:53,051][20381] Updated weights for policy 1, policy_version 935052 (0.0011) [2023-12-27 12:07:53,077][20347] Updated weights for policy 0, policy_version 935889 (0.0005) [2023-12-27 12:07:53,114][20381] Updated weights for policy 1, policy_version 935062 (0.0011) [2023-12-27 12:07:53,130][20347] Updated weights for policy 0, policy_version 935899 (0.0005) [2023-12-27 12:07:53,762][20381] Updated weights for policy 1, policy_version 935072 (0.0011) [2023-12-27 12:07:53,787][20347] Updated weights for policy 0, policy_version 935909 (0.0006) [2023-12-27 12:07:53,817][20381] Updated weights for policy 1, policy_version 935082 (0.0011) [2023-12-27 12:07:53,841][20347] Updated weights for policy 0, policy_version 935919 (0.0006) [2023-12-27 12:07:53,876][20381] Updated weights for policy 1, policy_version 935092 (0.0011) [2023-12-27 12:07:53,896][20347] Updated weights for policy 0, policy_version 935929 (0.0005) [2023-12-27 12:07:54,504][20347] Updated weights for policy 0, policy_version 935939 (0.0006) [2023-12-27 12:07:54,528][20381] Updated weights for policy 1, policy_version 935102 (0.0011) [2023-12-27 12:07:54,559][20347] Updated weights for policy 0, policy_version 935949 (0.0005) [2023-12-27 12:07:54,584][20381] Updated weights for policy 1, policy_version 935112 (0.0010) [2023-12-27 12:07:54,615][20347] Updated weights for policy 0, policy_version 935959 (0.0005) [2023-12-27 12:07:54,639][20381] Updated weights for policy 1, policy_version 935122 (0.0011) [2023-12-27 12:07:55,286][20347] Updated weights for policy 0, policy_version 935969 (0.0006) [2023-12-27 12:07:55,302][20381] Updated weights for policy 1, policy_version 935132 (0.0011) [2023-12-27 12:07:55,343][20347] Updated weights for policy 0, policy_version 935979 (0.0005) [2023-12-27 12:07:55,350][20381] Updated weights for policy 1, policy_version 935142 (0.0010) [2023-12-27 12:07:55,392][20347] Updated weights for policy 0, policy_version 935989 (0.0007) [2023-12-27 12:07:55,399][20381] Updated weights for policy 1, policy_version 935152 (0.0011) [2023-12-27 12:07:55,438][20347] Updated weights for policy 0, policy_version 935999 (0.0006) [2023-12-27 12:07:56,058][20381] Updated weights for policy 1, policy_version 935162 (0.0009) [2023-12-27 12:07:56,101][20347] Updated weights for policy 0, policy_version 936009 (0.0008) [2023-12-27 12:07:56,107][20381] Updated weights for policy 1, policy_version 935172 (0.0005) [2023-12-27 12:07:56,154][20347] Updated weights for policy 0, policy_version 936019 (0.0009) [2023-12-27 12:07:56,170][20381] Updated weights for policy 1, policy_version 935182 (0.0006) [2023-12-27 12:07:56,206][20347] Updated weights for policy 0, policy_version 936029 (0.0009) [2023-12-27 12:07:56,224][20381] Updated weights for policy 1, policy_version 935192 (0.0010) [2023-12-27 12:07:56,791][20347] Updated weights for policy 0, policy_version 936039 (0.0009) [2023-12-27 12:07:56,846][20347] Updated weights for policy 0, policy_version 936049 (0.0010) [2023-12-27 12:07:56,872][20381] Updated weights for policy 1, policy_version 935202 (0.0010) [2023-12-27 12:07:56,892][20347] Updated weights for policy 0, policy_version 936059 (0.0010) [2023-12-27 12:07:56,915][20292] KL-divergence is very high: 104.5454 [2023-12-27 12:07:56,921][20381] Updated weights for policy 1, policy_version 935212 (0.0010) [2023-12-27 12:07:56,969][20381] Updated weights for policy 1, policy_version 935222 (0.0010) [2023-12-27 12:07:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 479117312. Throughput: 0: 10415.1, 1: 10370.4. Samples: 479106344. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:07:57,363][19170] Avg episode reward: [(0, '970.860'), (1, '804.140')] [2023-12-27 12:07:57,563][20347] Updated weights for policy 0, policy_version 936069 (0.0010) [2023-12-27 12:07:57,621][20347] Updated weights for policy 0, policy_version 936079 (0.0010) [2023-12-27 12:07:57,649][20381] Updated weights for policy 1, policy_version 935232 (0.0006) [2023-12-27 12:07:57,678][20347] Updated weights for policy 0, policy_version 936089 (0.0010) [2023-12-27 12:07:57,714][20381] Updated weights for policy 1, policy_version 935242 (0.0006) [2023-12-27 12:07:57,763][20381] Updated weights for policy 1, policy_version 935252 (0.0007) [2023-12-27 12:07:58,320][20347] Updated weights for policy 0, policy_version 936099 (0.0010) [2023-12-27 12:07:58,387][20347] Updated weights for policy 0, policy_version 936109 (0.0011) [2023-12-27 12:07:58,449][20347] Updated weights for policy 0, policy_version 936119 (0.0008) [2023-12-27 12:07:58,452][20381] Updated weights for policy 1, policy_version 935262 (0.0008) [2023-12-27 12:07:58,510][20381] Updated weights for policy 1, policy_version 935272 (0.0009) [2023-12-27 12:07:58,572][20381] Updated weights for policy 1, policy_version 935282 (0.0007) [2023-12-27 12:07:59,234][20347] Updated weights for policy 0, policy_version 936129 (0.0008) [2023-12-27 12:07:59,307][20347] Updated weights for policy 0, policy_version 936139 (0.0008) [2023-12-27 12:07:59,337][20381] Updated weights for policy 1, policy_version 935292 (0.0007) [2023-12-27 12:07:59,359][20347] Updated weights for policy 0, policy_version 936149 (0.0006) [2023-12-27 12:07:59,400][20381] Updated weights for policy 1, policy_version 935302 (0.0007) [2023-12-27 12:07:59,420][20347] Updated weights for policy 0, policy_version 936159 (0.0007) [2023-12-27 12:07:59,453][20381] Updated weights for policy 1, policy_version 935312 (0.0006) [2023-12-27 12:08:00,149][20381] Updated weights for policy 1, policy_version 935322 (0.0006) [2023-12-27 12:08:00,155][20347] Updated weights for policy 0, policy_version 936169 (0.0006) [2023-12-27 12:08:00,210][20347] Updated weights for policy 0, policy_version 936179 (0.0006) [2023-12-27 12:08:00,212][20381] Updated weights for policy 1, policy_version 935332 (0.0007) [2023-12-27 12:08:00,268][20347] Updated weights for policy 0, policy_version 936189 (0.0005) [2023-12-27 12:08:00,275][20381] Updated weights for policy 1, policy_version 935342 (0.0007) [2023-12-27 12:08:00,342][20381] Updated weights for policy 1, policy_version 935352 (0.0008) [2023-12-27 12:08:00,926][20347] Updated weights for policy 0, policy_version 936199 (0.0007) [2023-12-27 12:08:00,977][20347] Updated weights for policy 0, policy_version 936209 (0.0007) [2023-12-27 12:08:01,034][20347] Updated weights for policy 0, policy_version 936219 (0.0007) [2023-12-27 12:08:01,039][20381] Updated weights for policy 1, policy_version 935362 (0.0006) [2023-12-27 12:08:01,101][20381] Updated weights for policy 1, policy_version 935372 (0.0007) [2023-12-27 12:08:01,170][20381] Updated weights for policy 1, policy_version 935382 (0.0007) [2023-12-27 12:08:01,756][20347] Updated weights for policy 0, policy_version 936229 (0.0008) [2023-12-27 12:08:01,812][20347] Updated weights for policy 0, policy_version 936239 (0.0007) [2023-12-27 12:08:01,831][20381] Updated weights for policy 1, policy_version 935392 (0.0006) [2023-12-27 12:08:01,873][20347] Updated weights for policy 0, policy_version 936249 (0.0007) [2023-12-27 12:08:01,889][20381] Updated weights for policy 1, policy_version 935402 (0.0008) [2023-12-27 12:08:01,944][20381] Updated weights for policy 1, policy_version 935412 (0.0007) [2023-12-27 12:08:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 479215616. Throughput: 0: 10396.5, 1: 10326.0. Samples: 479226784. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:02,363][19170] Avg episode reward: [(0, '921.860'), (1, '755.310')] [2023-12-27 12:08:02,576][20381] Updated weights for policy 1, policy_version 935422 (0.0008) [2023-12-27 12:08:02,601][20347] Updated weights for policy 0, policy_version 936259 (0.0006) [2023-12-27 12:08:02,632][20381] Updated weights for policy 1, policy_version 935432 (0.0006) [2023-12-27 12:08:02,656][20347] Updated weights for policy 0, policy_version 936269 (0.0005) [2023-12-27 12:08:02,691][20381] Updated weights for policy 1, policy_version 935442 (0.0007) [2023-12-27 12:08:02,715][20347] Updated weights for policy 0, policy_version 936279 (0.0005) [2023-12-27 12:08:03,283][20347] Updated weights for policy 0, policy_version 936289 (0.0007) [2023-12-27 12:08:03,339][20347] Updated weights for policy 0, policy_version 936299 (0.0007) [2023-12-27 12:08:03,392][20347] Updated weights for policy 0, policy_version 936309 (0.0007) [2023-12-27 12:08:03,412][20381] Updated weights for policy 1, policy_version 935452 (0.0006) [2023-12-27 12:08:03,441][20347] Updated weights for policy 0, policy_version 936319 (0.0005) [2023-12-27 12:08:03,467][20381] Updated weights for policy 1, policy_version 935462 (0.0008) [2023-12-27 12:08:03,522][20381] Updated weights for policy 1, policy_version 935472 (0.0008) [2023-12-27 12:08:04,054][20347] Updated weights for policy 0, policy_version 936329 (0.0007) [2023-12-27 12:08:04,114][20347] Updated weights for policy 0, policy_version 936339 (0.0006) [2023-12-27 12:08:04,174][20347] Updated weights for policy 0, policy_version 936349 (0.0008) [2023-12-27 12:08:04,221][20381] Updated weights for policy 1, policy_version 935482 (0.0007) [2023-12-27 12:08:04,282][20381] Updated weights for policy 1, policy_version 935492 (0.0009) [2023-12-27 12:08:04,347][20381] Updated weights for policy 1, policy_version 935502 (0.0008) [2023-12-27 12:08:04,403][20381] Updated weights for policy 1, policy_version 935512 (0.0008) [2023-12-27 12:08:04,825][20347] Updated weights for policy 0, policy_version 936359 (0.0010) [2023-12-27 12:08:04,885][20347] Updated weights for policy 0, policy_version 936369 (0.0010) [2023-12-27 12:08:04,942][20347] Updated weights for policy 0, policy_version 936379 (0.0010) [2023-12-27 12:08:05,105][20381] Updated weights for policy 1, policy_version 935522 (0.0008) [2023-12-27 12:08:05,163][20381] Updated weights for policy 1, policy_version 935532 (0.0007) [2023-12-27 12:08:05,214][20381] Updated weights for policy 1, policy_version 935542 (0.0007) [2023-12-27 12:08:05,595][20347] Updated weights for policy 0, policy_version 936389 (0.0011) [2023-12-27 12:08:05,658][20347] Updated weights for policy 0, policy_version 936399 (0.0011) [2023-12-27 12:08:05,718][20347] Updated weights for policy 0, policy_version 936409 (0.0011) [2023-12-27 12:08:05,896][20381] Updated weights for policy 1, policy_version 935552 (0.0007) [2023-12-27 12:08:05,952][20381] Updated weights for policy 1, policy_version 935562 (0.0007) [2023-12-27 12:08:06,012][20381] Updated weights for policy 1, policy_version 935572 (0.0008) [2023-12-27 12:08:06,395][20347] Updated weights for policy 0, policy_version 936419 (0.0010) [2023-12-27 12:08:06,467][20347] Updated weights for policy 0, policy_version 936429 (0.0008) [2023-12-27 12:08:06,529][20347] Updated weights for policy 0, policy_version 936439 (0.0008) [2023-12-27 12:08:06,725][20381] Updated weights for policy 1, policy_version 935582 (0.0008) [2023-12-27 12:08:06,784][20381] Updated weights for policy 1, policy_version 935592 (0.0008) [2023-12-27 12:08:06,848][20381] Updated weights for policy 1, policy_version 935602 (0.0008) [2023-12-27 12:08:07,207][20347] Updated weights for policy 0, policy_version 936449 (0.0008) [2023-12-27 12:08:07,271][20347] Updated weights for policy 0, policy_version 936459 (0.0005) [2023-12-27 12:08:07,333][20347] Updated weights for policy 0, policy_version 936469 (0.0007) [2023-12-27 12:08:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 479313920. Throughput: 0: 10371.2, 1: 10344.3. Samples: 479289108. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:07,363][19170] Avg episode reward: [(0, '912.360'), (1, '814.180')] [2023-12-27 12:08:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000935608_239542272.pth... [2023-12-27 12:08:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000934424_239239168.pth [2023-12-27 12:08:07,391][20347] Updated weights for policy 0, policy_version 936479 (0.0008) [2023-12-27 12:08:07,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000936480_239779840.pth... [2023-12-27 12:08:07,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000935264_239468544.pth [2023-12-27 12:08:07,537][20381] Updated weights for policy 1, policy_version 935612 (0.0008) [2023-12-27 12:08:07,596][20381] Updated weights for policy 1, policy_version 935622 (0.0007) [2023-12-27 12:08:07,657][20381] Updated weights for policy 1, policy_version 935632 (0.0007) [2023-12-27 12:08:08,027][20347] Updated weights for policy 0, policy_version 936489 (0.0007) [2023-12-27 12:08:08,074][20347] Updated weights for policy 0, policy_version 936499 (0.0008) [2023-12-27 12:08:08,122][20347] Updated weights for policy 0, policy_version 936509 (0.0008) [2023-12-27 12:08:08,343][20381] Updated weights for policy 1, policy_version 935642 (0.0008) [2023-12-27 12:08:08,400][20381] Updated weights for policy 1, policy_version 935652 (0.0008) [2023-12-27 12:08:08,455][20381] Updated weights for policy 1, policy_version 935662 (0.0007) [2023-12-27 12:08:08,515][20381] Updated weights for policy 1, policy_version 935672 (0.0008) [2023-12-27 12:08:08,790][20347] Updated weights for policy 0, policy_version 936519 (0.0006) [2023-12-27 12:08:08,854][20347] Updated weights for policy 0, policy_version 936529 (0.0007) [2023-12-27 12:08:08,913][20347] Updated weights for policy 0, policy_version 936539 (0.0008) [2023-12-27 12:08:09,263][20381] Updated weights for policy 1, policy_version 935682 (0.0009) [2023-12-27 12:08:09,328][20381] Updated weights for policy 1, policy_version 935692 (0.0005) [2023-12-27 12:08:09,395][20381] Updated weights for policy 1, policy_version 935702 (0.0008) [2023-12-27 12:08:09,541][20347] Updated weights for policy 0, policy_version 936549 (0.0007) [2023-12-27 12:08:09,597][20347] Updated weights for policy 0, policy_version 936559 (0.0007) [2023-12-27 12:08:09,661][20347] Updated weights for policy 0, policy_version 936569 (0.0007) [2023-12-27 12:08:10,086][20381] Updated weights for policy 1, policy_version 935712 (0.0008) [2023-12-27 12:08:10,135][20381] Updated weights for policy 1, policy_version 935722 (0.0007) [2023-12-27 12:08:10,188][20381] Updated weights for policy 1, policy_version 935732 (0.0008) [2023-12-27 12:08:10,372][20347] Updated weights for policy 0, policy_version 936579 (0.0008) [2023-12-27 12:08:10,432][20347] Updated weights for policy 0, policy_version 936589 (0.0008) [2023-12-27 12:08:10,489][20347] Updated weights for policy 0, policy_version 936599 (0.0008) [2023-12-27 12:08:10,925][20381] Updated weights for policy 1, policy_version 935742 (0.0008) [2023-12-27 12:08:10,974][20381] Updated weights for policy 1, policy_version 935752 (0.0008) [2023-12-27 12:08:11,023][20381] Updated weights for policy 1, policy_version 935762 (0.0008) [2023-12-27 12:08:11,128][20347] Updated weights for policy 0, policy_version 936609 (0.0007) [2023-12-27 12:08:11,200][20347] Updated weights for policy 0, policy_version 936619 (0.0007) [2023-12-27 12:08:11,261][20347] Updated weights for policy 0, policy_version 936629 (0.0008) [2023-12-27 12:08:11,318][20347] Updated weights for policy 0, policy_version 936639 (0.0008) [2023-12-27 12:08:11,768][20381] Updated weights for policy 1, policy_version 935772 (0.0008) [2023-12-27 12:08:11,828][20381] Updated weights for policy 1, policy_version 935782 (0.0008) [2023-12-27 12:08:11,889][20381] Updated weights for policy 1, policy_version 935792 (0.0008) [2023-12-27 12:08:12,098][20347] Updated weights for policy 0, policy_version 936649 (0.0008) [2023-12-27 12:08:12,158][20347] Updated weights for policy 0, policy_version 936659 (0.0008) [2023-12-27 12:08:12,227][20347] Updated weights for policy 0, policy_version 936669 (0.0007) [2023-12-27 12:08:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20438.4). Total num frames: 479420416. Throughput: 0: 10363.7, 1: 10277.9. Samples: 479409756. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:12,363][19170] Avg episode reward: [(0, '990.330'), (1, '843.480')] [2023-12-27 12:08:12,585][20381] Updated weights for policy 1, policy_version 935802 (0.0008) [2023-12-27 12:08:12,649][20381] Updated weights for policy 1, policy_version 935812 (0.0005) [2023-12-27 12:08:12,696][20381] Updated weights for policy 1, policy_version 935822 (0.0007) [2023-12-27 12:08:12,746][20381] Updated weights for policy 1, policy_version 935832 (0.0008) [2023-12-27 12:08:12,968][20347] Updated weights for policy 0, policy_version 936679 (0.0008) [2023-12-27 12:08:13,016][20347] Updated weights for policy 0, policy_version 936689 (0.0005) [2023-12-27 12:08:13,067][20347] Updated weights for policy 0, policy_version 936699 (0.0005) [2023-12-27 12:08:13,343][20381] Updated weights for policy 1, policy_version 935842 (0.0007) [2023-12-27 12:08:13,392][20381] Updated weights for policy 1, policy_version 935852 (0.0007) [2023-12-27 12:08:13,438][20381] Updated weights for policy 1, policy_version 935862 (0.0007) [2023-12-27 12:08:13,690][20347] Updated weights for policy 0, policy_version 936709 (0.0005) [2023-12-27 12:08:13,752][20347] Updated weights for policy 0, policy_version 936719 (0.0005) [2023-12-27 12:08:13,814][20347] Updated weights for policy 0, policy_version 936729 (0.0005) [2023-12-27 12:08:14,201][20381] Updated weights for policy 1, policy_version 935872 (0.0007) [2023-12-27 12:08:14,261][20381] Updated weights for policy 1, policy_version 935882 (0.0008) [2023-12-27 12:08:14,324][20381] Updated weights for policy 1, policy_version 935892 (0.0007) [2023-12-27 12:08:14,396][20347] Updated weights for policy 0, policy_version 936739 (0.0006) [2023-12-27 12:08:14,453][20347] Updated weights for policy 0, policy_version 936749 (0.0005) [2023-12-27 12:08:14,510][20347] Updated weights for policy 0, policy_version 936759 (0.0007) [2023-12-27 12:08:15,050][20381] Updated weights for policy 1, policy_version 935902 (0.0009) [2023-12-27 12:08:15,107][20381] Updated weights for policy 1, policy_version 935912 (0.0011) [2023-12-27 12:08:15,163][20347] Updated weights for policy 0, policy_version 936769 (0.0006) [2023-12-27 12:08:15,166][20381] Updated weights for policy 1, policy_version 935922 (0.0011) [2023-12-27 12:08:15,221][20347] Updated weights for policy 0, policy_version 936779 (0.0006) [2023-12-27 12:08:15,276][20347] Updated weights for policy 0, policy_version 936789 (0.0008) [2023-12-27 12:08:15,331][20347] Updated weights for policy 0, policy_version 936799 (0.0007) [2023-12-27 12:08:15,865][20381] Updated weights for policy 1, policy_version 935932 (0.0011) [2023-12-27 12:08:15,917][20381] Updated weights for policy 1, policy_version 935942 (0.0011) [2023-12-27 12:08:15,966][20381] Updated weights for policy 1, policy_version 935952 (0.0011) [2023-12-27 12:08:16,036][20347] Updated weights for policy 0, policy_version 936809 (0.0009) [2023-12-27 12:08:16,099][20347] Updated weights for policy 0, policy_version 936819 (0.0010) [2023-12-27 12:08:16,164][20347] Updated weights for policy 0, policy_version 936829 (0.0006) [2023-12-27 12:08:16,633][20381] Updated weights for policy 1, policy_version 935962 (0.0009) [2023-12-27 12:08:16,686][20381] Updated weights for policy 1, policy_version 935972 (0.0009) [2023-12-27 12:08:16,742][20381] Updated weights for policy 1, policy_version 935982 (0.0005) [2023-12-27 12:08:16,748][20347] Updated weights for policy 0, policy_version 936839 (0.0006) [2023-12-27 12:08:16,796][20381] Updated weights for policy 1, policy_version 935992 (0.0008) [2023-12-27 12:08:16,804][20347] Updated weights for policy 0, policy_version 936849 (0.0008) [2023-12-27 12:08:16,856][20347] Updated weights for policy 0, policy_version 936859 (0.0007) [2023-12-27 12:08:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 479518720. Throughput: 0: 10366.6, 1: 10189.4. Samples: 479532504. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:17,363][19170] Avg episode reward: [(0, '980.650'), (1, '794.550')] [2023-12-27 12:08:17,372][20381] Updated weights for policy 1, policy_version 936002 (0.0008) [2023-12-27 12:08:17,433][20381] Updated weights for policy 1, policy_version 936012 (0.0008) [2023-12-27 12:08:17,497][20381] Updated weights for policy 1, policy_version 936022 (0.0008) [2023-12-27 12:08:17,578][20347] Updated weights for policy 0, policy_version 936869 (0.0006) [2023-12-27 12:08:17,637][20347] Updated weights for policy 0, policy_version 936879 (0.0006) [2023-12-27 12:08:17,693][20347] Updated weights for policy 0, policy_version 936889 (0.0008) [2023-12-27 12:08:18,050][20381] Updated weights for policy 1, policy_version 936032 (0.0009) [2023-12-27 12:08:18,098][20381] Updated weights for policy 1, policy_version 936042 (0.0010) [2023-12-27 12:08:18,163][20381] Updated weights for policy 1, policy_version 936052 (0.0010) [2023-12-27 12:08:18,372][20347] Updated weights for policy 0, policy_version 936899 (0.0009) [2023-12-27 12:08:18,426][20347] Updated weights for policy 0, policy_version 936909 (0.0008) [2023-12-27 12:08:18,482][20347] Updated weights for policy 0, policy_version 936919 (0.0008) [2023-12-27 12:08:18,850][20381] Updated weights for policy 1, policy_version 936062 (0.0011) [2023-12-27 12:08:18,906][20381] Updated weights for policy 1, policy_version 936072 (0.0011) [2023-12-27 12:08:18,956][20381] Updated weights for policy 1, policy_version 936082 (0.0011) [2023-12-27 12:08:19,187][20347] Updated weights for policy 0, policy_version 936929 (0.0008) [2023-12-27 12:08:19,252][20347] Updated weights for policy 0, policy_version 936939 (0.0009) [2023-12-27 12:08:19,316][20347] Updated weights for policy 0, policy_version 936949 (0.0007) [2023-12-27 12:08:19,386][20347] Updated weights for policy 0, policy_version 936959 (0.0008) [2023-12-27 12:08:19,721][20381] Updated weights for policy 1, policy_version 936092 (0.0011) [2023-12-27 12:08:19,784][20381] Updated weights for policy 1, policy_version 936102 (0.0011) [2023-12-27 12:08:19,849][20381] Updated weights for policy 1, policy_version 936112 (0.0012) [2023-12-27 12:08:20,043][20347] Updated weights for policy 0, policy_version 936969 (0.0007) [2023-12-27 12:08:20,107][20347] Updated weights for policy 0, policy_version 936979 (0.0006) [2023-12-27 12:08:20,162][20347] Updated weights for policy 0, policy_version 936989 (0.0007) [2023-12-27 12:08:20,501][20381] Updated weights for policy 1, policy_version 936122 (0.0011) [2023-12-27 12:08:20,558][20381] Updated weights for policy 1, policy_version 936132 (0.0011) [2023-12-27 12:08:20,625][20381] Updated weights for policy 1, policy_version 936142 (0.0011) [2023-12-27 12:08:20,689][20381] Updated weights for policy 1, policy_version 936152 (0.0011) [2023-12-27 12:08:20,873][20347] Updated weights for policy 0, policy_version 936999 (0.0007) [2023-12-27 12:08:20,933][20347] Updated weights for policy 0, policy_version 937009 (0.0007) [2023-12-27 12:08:20,982][20347] Updated weights for policy 0, policy_version 937019 (0.0006) [2023-12-27 12:08:21,400][20381] Updated weights for policy 1, policy_version 936162 (0.0010) [2023-12-27 12:08:21,466][20381] Updated weights for policy 1, policy_version 936172 (0.0009) [2023-12-27 12:08:21,528][20381] Updated weights for policy 1, policy_version 936182 (0.0010) [2023-12-27 12:08:21,754][20347] Updated weights for policy 0, policy_version 937029 (0.0006) [2023-12-27 12:08:21,813][20347] Updated weights for policy 0, policy_version 937039 (0.0007) [2023-12-27 12:08:21,874][20347] Updated weights for policy 0, policy_version 937049 (0.0007) [2023-12-27 12:08:22,192][20381] Updated weights for policy 1, policy_version 936192 (0.0009) [2023-12-27 12:08:22,251][20381] Updated weights for policy 1, policy_version 936202 (0.0008) [2023-12-27 12:08:22,338][20381] Updated weights for policy 1, policy_version 936212 (0.0008) [2023-12-27 12:08:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 20410.6). Total num frames: 479617024. Throughput: 0: 10314.9, 1: 10154.6. Samples: 479593116. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:22,363][19170] Avg episode reward: [(0, '980.650'), (1, '814.090')] [2023-12-27 12:08:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000936216_239697920.pth... [2023-12-27 12:08:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000937056_239927296.pth... [2023-12-27 12:08:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000935872_239624192.pth [2023-12-27 12:08:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000935032_239394816.pth [2023-12-27 12:08:22,596][20347] Updated weights for policy 0, policy_version 937059 (0.0006) [2023-12-27 12:08:22,663][20347] Updated weights for policy 0, policy_version 937069 (0.0009) [2023-12-27 12:08:22,735][20347] Updated weights for policy 0, policy_version 937079 (0.0008) [2023-12-27 12:08:23,004][20381] Updated weights for policy 1, policy_version 936222 (0.0007) [2023-12-27 12:08:23,055][20381] Updated weights for policy 1, policy_version 936232 (0.0006) [2023-12-27 12:08:23,109][20381] Updated weights for policy 1, policy_version 936242 (0.0006) [2023-12-27 12:08:23,382][20347] Updated weights for policy 0, policy_version 937089 (0.0007) [2023-12-27 12:08:23,441][20347] Updated weights for policy 0, policy_version 937099 (0.0008) [2023-12-27 12:08:23,489][20347] Updated weights for policy 0, policy_version 937109 (0.0005) [2023-12-27 12:08:23,550][20347] Updated weights for policy 0, policy_version 937119 (0.0005) [2023-12-27 12:08:23,668][20381] Updated weights for policy 1, policy_version 936252 (0.0006) [2023-12-27 12:08:23,730][20381] Updated weights for policy 1, policy_version 936262 (0.0007) [2023-12-27 12:08:23,781][20381] Updated weights for policy 1, policy_version 936272 (0.0007) [2023-12-27 12:08:24,175][20347] Updated weights for policy 0, policy_version 937129 (0.0005) [2023-12-27 12:08:24,223][20347] Updated weights for policy 0, policy_version 937139 (0.0007) [2023-12-27 12:08:24,270][20347] Updated weights for policy 0, policy_version 937149 (0.0008) [2023-12-27 12:08:24,406][20381] Updated weights for policy 1, policy_version 936282 (0.0007) [2023-12-27 12:08:24,461][20381] Updated weights for policy 1, policy_version 936292 (0.0007) [2023-12-27 12:08:24,511][20381] Updated weights for policy 1, policy_version 936302 (0.0010) [2023-12-27 12:08:24,559][20381] Updated weights for policy 1, policy_version 936312 (0.0010) [2023-12-27 12:08:24,853][20347] Updated weights for policy 0, policy_version 937159 (0.0007) [2023-12-27 12:08:24,906][20347] Updated weights for policy 0, policy_version 937169 (0.0005) [2023-12-27 12:08:24,962][20347] Updated weights for policy 0, policy_version 937179 (0.0005) [2023-12-27 12:08:25,339][20381] Updated weights for policy 1, policy_version 936322 (0.0010) [2023-12-27 12:08:25,389][20381] Updated weights for policy 1, policy_version 936332 (0.0008) [2023-12-27 12:08:25,434][20381] Updated weights for policy 1, policy_version 936342 (0.0009) [2023-12-27 12:08:25,472][20347] Updated weights for policy 0, policy_version 937189 (0.0005) [2023-12-27 12:08:25,535][20347] Updated weights for policy 0, policy_version 937199 (0.0010) [2023-12-27 12:08:25,593][20347] Updated weights for policy 0, policy_version 937209 (0.0010) [2023-12-27 12:08:26,051][20381] Updated weights for policy 1, policy_version 936352 (0.0008) [2023-12-27 12:08:26,104][20381] Updated weights for policy 1, policy_version 936362 (0.0008) [2023-12-27 12:08:26,161][20381] Updated weights for policy 1, policy_version 936372 (0.0008) [2023-12-27 12:08:26,180][20347] Updated weights for policy 0, policy_version 937219 (0.0010) [2023-12-27 12:08:26,236][20347] Updated weights for policy 0, policy_version 937229 (0.0009) [2023-12-27 12:08:26,286][20347] Updated weights for policy 0, policy_version 937239 (0.0006) [2023-12-27 12:08:26,829][20347] Updated weights for policy 0, policy_version 937249 (0.0006) [2023-12-27 12:08:26,878][20347] Updated weights for policy 0, policy_version 937259 (0.0007) [2023-12-27 12:08:26,899][20381] Updated weights for policy 1, policy_version 936382 (0.0007) [2023-12-27 12:08:26,926][20347] Updated weights for policy 0, policy_version 937269 (0.0007) [2023-12-27 12:08:26,957][20381] Updated weights for policy 1, policy_version 936392 (0.0005) [2023-12-27 12:08:26,981][20347] Updated weights for policy 0, policy_version 937279 (0.0008) [2023-12-27 12:08:27,010][20381] Updated weights for policy 1, policy_version 936402 (0.0006) [2023-12-27 12:08:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 479731712. Throughput: 0: 10313.8, 1: 10199.3. Samples: 479719056. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.460')] [2023-12-27 12:08:27,588][20381] Updated weights for policy 1, policy_version 936412 (0.0007) [2023-12-27 12:08:27,641][20381] Updated weights for policy 1, policy_version 936422 (0.0006) [2023-12-27 12:08:27,689][20381] Updated weights for policy 1, policy_version 936432 (0.0005) [2023-12-27 12:08:27,719][20347] Updated weights for policy 0, policy_version 937289 (0.0008) [2023-12-27 12:08:27,770][20347] Updated weights for policy 0, policy_version 937299 (0.0007) [2023-12-27 12:08:27,819][20347] Updated weights for policy 0, policy_version 937309 (0.0007) [2023-12-27 12:08:28,358][20381] Updated weights for policy 1, policy_version 936442 (0.0006) [2023-12-27 12:08:28,420][20381] Updated weights for policy 1, policy_version 936452 (0.0008) [2023-12-27 12:08:28,466][20381] Updated weights for policy 1, policy_version 936462 (0.0007) [2023-12-27 12:08:28,512][20347] Updated weights for policy 0, policy_version 937319 (0.0007) [2023-12-27 12:08:28,518][20381] Updated weights for policy 1, policy_version 936472 (0.0007) [2023-12-27 12:08:28,564][20347] Updated weights for policy 0, policy_version 937329 (0.0007) [2023-12-27 12:08:28,618][20347] Updated weights for policy 0, policy_version 937339 (0.0008) [2023-12-27 12:08:29,173][20381] Updated weights for policy 1, policy_version 936482 (0.0007) [2023-12-27 12:08:29,229][20381] Updated weights for policy 1, policy_version 936492 (0.0008) [2023-12-27 12:08:29,291][20381] Updated weights for policy 1, policy_version 936502 (0.0007) [2023-12-27 12:08:29,377][20347] Updated weights for policy 0, policy_version 937349 (0.0009) [2023-12-27 12:08:29,436][20347] Updated weights for policy 0, policy_version 937359 (0.0007) [2023-12-27 12:08:29,486][20347] Updated weights for policy 0, policy_version 937369 (0.0008) [2023-12-27 12:08:30,031][20381] Updated weights for policy 1, policy_version 936512 (0.0009) [2023-12-27 12:08:30,095][20381] Updated weights for policy 1, policy_version 936522 (0.0008) [2023-12-27 12:08:30,156][20381] Updated weights for policy 1, policy_version 936532 (0.0008) [2023-12-27 12:08:30,166][20347] Updated weights for policy 0, policy_version 937379 (0.0008) [2023-12-27 12:08:30,233][20347] Updated weights for policy 0, policy_version 937389 (0.0010) [2023-12-27 12:08:30,298][20347] Updated weights for policy 0, policy_version 937399 (0.0009) [2023-12-27 12:08:30,792][20381] Updated weights for policy 1, policy_version 936542 (0.0007) [2023-12-27 12:08:30,846][20381] Updated weights for policy 1, policy_version 936552 (0.0006) [2023-12-27 12:08:30,894][20381] Updated weights for policy 1, policy_version 936562 (0.0006) [2023-12-27 12:08:30,962][20347] Updated weights for policy 0, policy_version 937409 (0.0010) [2023-12-27 12:08:31,012][20347] Updated weights for policy 0, policy_version 937419 (0.0006) [2023-12-27 12:08:31,085][20347] Updated weights for policy 0, policy_version 937429 (0.0008) [2023-12-27 12:08:31,144][20347] Updated weights for policy 0, policy_version 937439 (0.0006) [2023-12-27 12:08:31,569][20381] Updated weights for policy 1, policy_version 936572 (0.0008) [2023-12-27 12:08:31,633][20381] Updated weights for policy 1, policy_version 936582 (0.0007) [2023-12-27 12:08:31,693][20381] Updated weights for policy 1, policy_version 936592 (0.0008) [2023-12-27 12:08:31,814][20347] Updated weights for policy 0, policy_version 937449 (0.0008) [2023-12-27 12:08:31,864][20347] Updated weights for policy 0, policy_version 937459 (0.0006) [2023-12-27 12:08:31,910][20347] Updated weights for policy 0, policy_version 937469 (0.0008) [2023-12-27 12:08:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 479830016. Throughput: 0: 10332.6, 1: 10163.6. Samples: 479842056. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '911.970')] [2023-12-27 12:08:32,406][20381] Updated weights for policy 1, policy_version 936602 (0.0008) [2023-12-27 12:08:32,465][20381] Updated weights for policy 1, policy_version 936612 (0.0006) [2023-12-27 12:08:32,521][20381] Updated weights for policy 1, policy_version 936622 (0.0008) [2023-12-27 12:08:32,573][20381] Updated weights for policy 1, policy_version 936632 (0.0008) [2023-12-27 12:08:32,614][20347] Updated weights for policy 0, policy_version 937479 (0.0007) [2023-12-27 12:08:32,673][20347] Updated weights for policy 0, policy_version 937489 (0.0008) [2023-12-27 12:08:32,722][20347] Updated weights for policy 0, policy_version 937499 (0.0008) [2023-12-27 12:08:33,234][20381] Updated weights for policy 1, policy_version 936642 (0.0008) [2023-12-27 12:08:33,279][20381] Updated weights for policy 1, policy_version 936652 (0.0008) [2023-12-27 12:08:33,328][20381] Updated weights for policy 1, policy_version 936662 (0.0007) [2023-12-27 12:08:33,414][20347] Updated weights for policy 0, policy_version 937509 (0.0007) [2023-12-27 12:08:33,477][20347] Updated weights for policy 0, policy_version 937519 (0.0008) [2023-12-27 12:08:33,526][20347] Updated weights for policy 0, policy_version 937529 (0.0007) [2023-12-27 12:08:34,048][20381] Updated weights for policy 1, policy_version 936672 (0.0008) [2023-12-27 12:08:34,103][20381] Updated weights for policy 1, policy_version 936682 (0.0009) [2023-12-27 12:08:34,153][20347] Updated weights for policy 0, policy_version 937539 (0.0008) [2023-12-27 12:08:34,173][20381] Updated weights for policy 1, policy_version 936692 (0.0008) [2023-12-27 12:08:34,213][20347] Updated weights for policy 0, policy_version 937549 (0.0007) [2023-12-27 12:08:34,264][20347] Updated weights for policy 0, policy_version 937559 (0.0008) [2023-12-27 12:08:34,777][20381] Updated weights for policy 1, policy_version 936702 (0.0009) [2023-12-27 12:08:34,832][20381] Updated weights for policy 1, policy_version 936712 (0.0010) [2023-12-27 12:08:34,880][20381] Updated weights for policy 1, policy_version 936722 (0.0010) [2023-12-27 12:08:34,908][20347] Updated weights for policy 0, policy_version 937569 (0.0008) [2023-12-27 12:08:34,963][20347] Updated weights for policy 0, policy_version 937579 (0.0005) [2023-12-27 12:08:35,017][20347] Updated weights for policy 0, policy_version 937589 (0.0007) [2023-12-27 12:08:35,061][20347] Updated weights for policy 0, policy_version 937599 (0.0007) [2023-12-27 12:08:35,524][20381] Updated weights for policy 1, policy_version 936732 (0.0010) [2023-12-27 12:08:35,569][20381] Updated weights for policy 1, policy_version 936742 (0.0010) [2023-12-27 12:08:35,623][20381] Updated weights for policy 1, policy_version 936752 (0.0010) [2023-12-27 12:08:35,690][20347] Updated weights for policy 0, policy_version 937609 (0.0007) [2023-12-27 12:08:35,743][20347] Updated weights for policy 0, policy_version 937619 (0.0005) [2023-12-27 12:08:35,799][20347] Updated weights for policy 0, policy_version 937629 (0.0006) [2023-12-27 12:08:36,306][20381] Updated weights for policy 1, policy_version 936762 (0.0007) [2023-12-27 12:08:36,356][20381] Updated weights for policy 1, policy_version 936772 (0.0007) [2023-12-27 12:08:36,405][20381] Updated weights for policy 1, policy_version 936782 (0.0008) [2023-12-27 12:08:36,462][20381] Updated weights for policy 1, policy_version 936792 (0.0009) [2023-12-27 12:08:36,464][20347] Updated weights for policy 0, policy_version 937639 (0.0008) [2023-12-27 12:08:36,529][20347] Updated weights for policy 0, policy_version 937649 (0.0011) [2023-12-27 12:08:36,593][20347] Updated weights for policy 0, policy_version 937659 (0.0011) [2023-12-27 12:08:37,169][20381] Updated weights for policy 1, policy_version 936802 (0.0007) [2023-12-27 12:08:37,225][20347] Updated weights for policy 0, policy_version 937669 (0.0008) [2023-12-27 12:08:37,234][20381] Updated weights for policy 1, policy_version 936812 (0.0007) [2023-12-27 12:08:37,284][20347] Updated weights for policy 0, policy_version 937679 (0.0007) [2023-12-27 12:08:37,294][20381] Updated weights for policy 1, policy_version 936822 (0.0006) [2023-12-27 12:08:37,336][20347] Updated weights for policy 0, policy_version 937689 (0.0009) [2023-12-27 12:08:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 479936512. Throughput: 0: 10365.7, 1: 10154.4. Samples: 479905012. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:37,363][19170] Avg episode reward: [(0, '980.550'), (1, '921.830')] [2023-12-27 12:08:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000936824_239853568.pth... [2023-12-27 12:08:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000937696_240091136.pth... [2023-12-27 12:08:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000935608_239542272.pth [2023-12-27 12:08:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000936480_239779840.pth [2023-12-27 12:08:37,935][20381] Updated weights for policy 1, policy_version 936832 (0.0007) [2023-12-27 12:08:37,989][20381] Updated weights for policy 1, policy_version 936842 (0.0007) [2023-12-27 12:08:38,031][20347] Updated weights for policy 0, policy_version 937699 (0.0010) [2023-12-27 12:08:38,042][20381] Updated weights for policy 1, policy_version 936852 (0.0007) [2023-12-27 12:08:38,082][20347] Updated weights for policy 0, policy_version 937709 (0.0009) [2023-12-27 12:08:38,141][20347] Updated weights for policy 0, policy_version 937719 (0.0005) [2023-12-27 12:08:38,694][20381] Updated weights for policy 1, policy_version 936862 (0.0007) [2023-12-27 12:08:38,755][20381] Updated weights for policy 1, policy_version 936872 (0.0008) [2023-12-27 12:08:38,816][20381] Updated weights for policy 1, policy_version 936882 (0.0008) [2023-12-27 12:08:38,853][20347] Updated weights for policy 0, policy_version 937729 (0.0006) [2023-12-27 12:08:38,913][20347] Updated weights for policy 0, policy_version 937739 (0.0010) [2023-12-27 12:08:38,972][20347] Updated weights for policy 0, policy_version 937749 (0.0011) [2023-12-27 12:08:39,031][20347] Updated weights for policy 0, policy_version 937759 (0.0010) [2023-12-27 12:08:39,487][20381] Updated weights for policy 1, policy_version 936892 (0.0009) [2023-12-27 12:08:39,547][20381] Updated weights for policy 1, policy_version 936902 (0.0007) [2023-12-27 12:08:39,606][20381] Updated weights for policy 1, policy_version 936912 (0.0008) [2023-12-27 12:08:39,725][20347] Updated weights for policy 0, policy_version 937769 (0.0010) [2023-12-27 12:08:39,784][20347] Updated weights for policy 0, policy_version 937779 (0.0010) [2023-12-27 12:08:39,853][20347] Updated weights for policy 0, policy_version 937789 (0.0010) [2023-12-27 12:08:40,307][20381] Updated weights for policy 1, policy_version 936922 (0.0008) [2023-12-27 12:08:40,370][20381] Updated weights for policy 1, policy_version 936932 (0.0008) [2023-12-27 12:08:40,432][20381] Updated weights for policy 1, policy_version 936942 (0.0007) [2023-12-27 12:08:40,492][20381] Updated weights for policy 1, policy_version 936952 (0.0007) [2023-12-27 12:08:40,554][20347] Updated weights for policy 0, policy_version 937799 (0.0011) [2023-12-27 12:08:40,613][20347] Updated weights for policy 0, policy_version 937809 (0.0010) [2023-12-27 12:08:40,669][20347] Updated weights for policy 0, policy_version 937819 (0.0010) [2023-12-27 12:08:41,125][20381] Updated weights for policy 1, policy_version 936962 (0.0007) [2023-12-27 12:08:41,193][20381] Updated weights for policy 1, policy_version 936972 (0.0008) [2023-12-27 12:08:41,255][20381] Updated weights for policy 1, policy_version 936982 (0.0006) [2023-12-27 12:08:41,367][20347] Updated weights for policy 0, policy_version 937829 (0.0010) [2023-12-27 12:08:41,431][20347] Updated weights for policy 0, policy_version 937839 (0.0009) [2023-12-27 12:08:41,489][20347] Updated weights for policy 0, policy_version 937849 (0.0008) [2023-12-27 12:08:42,007][20381] Updated weights for policy 1, policy_version 936992 (0.0007) [2023-12-27 12:08:42,057][20381] Updated weights for policy 1, policy_version 937002 (0.0008) [2023-12-27 12:08:42,111][20381] Updated weights for policy 1, policy_version 937012 (0.0008) [2023-12-27 12:08:42,186][20347] Updated weights for policy 0, policy_version 937859 (0.0008) [2023-12-27 12:08:42,251][20347] Updated weights for policy 0, policy_version 937869 (0.0008) [2023-12-27 12:08:42,324][20347] Updated weights for policy 0, policy_version 937879 (0.0007) [2023-12-27 12:08:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 480034816. Throughput: 0: 10342.6, 1: 10153.0. Samples: 480028644. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:42,363][19170] Avg episode reward: [(0, '990.300'), (1, '941.290')] [2023-12-27 12:08:42,820][20381] Updated weights for policy 1, policy_version 937022 (0.0008) [2023-12-27 12:08:42,871][20292] KL-divergence is very high: 119.1204 [2023-12-27 12:08:42,877][20381] Updated weights for policy 1, policy_version 937032 (0.0008) [2023-12-27 12:08:42,930][20381] Updated weights for policy 1, policy_version 937042 (0.0007) [2023-12-27 12:08:43,063][20347] Updated weights for policy 0, policy_version 937889 (0.0008) [2023-12-27 12:08:43,109][20347] Updated weights for policy 0, policy_version 937899 (0.0008) [2023-12-27 12:08:43,171][20347] Updated weights for policy 0, policy_version 937909 (0.0006) [2023-12-27 12:08:43,226][20347] Updated weights for policy 0, policy_version 937919 (0.0005) [2023-12-27 12:08:43,674][20381] Updated weights for policy 1, policy_version 937052 (0.0008) [2023-12-27 12:08:43,728][20381] Updated weights for policy 1, policy_version 937062 (0.0008) [2023-12-27 12:08:43,788][20381] Updated weights for policy 1, policy_version 937072 (0.0007) [2023-12-27 12:08:43,824][20347] Updated weights for policy 0, policy_version 937929 (0.0006) [2023-12-27 12:08:43,873][20347] Updated weights for policy 0, policy_version 937939 (0.0005) [2023-12-27 12:08:43,928][20347] Updated weights for policy 0, policy_version 937949 (0.0005) [2023-12-27 12:08:44,494][20381] Updated weights for policy 1, policy_version 937082 (0.0008) [2023-12-27 12:08:44,512][20347] Updated weights for policy 0, policy_version 937959 (0.0009) [2023-12-27 12:08:44,550][20381] Updated weights for policy 1, policy_version 937092 (0.0011) [2023-12-27 12:08:44,567][20347] Updated weights for policy 0, policy_version 937969 (0.0010) [2023-12-27 12:08:44,605][20381] Updated weights for policy 1, policy_version 937102 (0.0010) [2023-12-27 12:08:44,623][20347] Updated weights for policy 0, policy_version 937979 (0.0010) [2023-12-27 12:08:44,665][20381] Updated weights for policy 1, policy_version 937112 (0.0008) [2023-12-27 12:08:45,253][20347] Updated weights for policy 0, policy_version 937989 (0.0005) [2023-12-27 12:08:45,316][20347] Updated weights for policy 0, policy_version 937999 (0.0007) [2023-12-27 12:08:45,376][20347] Updated weights for policy 0, policy_version 938009 (0.0006) [2023-12-27 12:08:45,376][20381] Updated weights for policy 1, policy_version 937122 (0.0008) [2023-12-27 12:08:45,433][20381] Updated weights for policy 1, policy_version 937132 (0.0009) [2023-12-27 12:08:45,490][20381] Updated weights for policy 1, policy_version 937142 (0.0009) [2023-12-27 12:08:45,970][20347] Updated weights for policy 0, policy_version 938019 (0.0006) [2023-12-27 12:08:46,026][20347] Updated weights for policy 0, policy_version 938029 (0.0005) [2023-12-27 12:08:46,087][20347] Updated weights for policy 0, policy_version 938039 (0.0007) [2023-12-27 12:08:46,187][20381] Updated weights for policy 1, policy_version 937152 (0.0008) [2023-12-27 12:08:46,245][20381] Updated weights for policy 1, policy_version 937162 (0.0008) [2023-12-27 12:08:46,304][20381] Updated weights for policy 1, policy_version 937172 (0.0007) [2023-12-27 12:08:46,678][20347] Updated weights for policy 0, policy_version 938049 (0.0007) [2023-12-27 12:08:46,737][20347] Updated weights for policy 0, policy_version 938059 (0.0008) [2023-12-27 12:08:46,794][20347] Updated weights for policy 0, policy_version 938069 (0.0007) [2023-12-27 12:08:46,849][20347] Updated weights for policy 0, policy_version 938079 (0.0009) [2023-12-27 12:08:46,989][20381] Updated weights for policy 1, policy_version 937182 (0.0008) [2023-12-27 12:08:47,043][20381] Updated weights for policy 1, policy_version 937192 (0.0008) [2023-12-27 12:08:47,107][20381] Updated weights for policy 1, policy_version 937202 (0.0009) [2023-12-27 12:08:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 480141312. Throughput: 0: 10398.2, 1: 10127.8. Samples: 480150456. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.530')] [2023-12-27 12:08:47,524][20347] Updated weights for policy 0, policy_version 938089 (0.0010) [2023-12-27 12:08:47,570][20347] Updated weights for policy 0, policy_version 938099 (0.0011) [2023-12-27 12:08:47,616][20347] Updated weights for policy 0, policy_version 938109 (0.0010) [2023-12-27 12:08:47,725][20381] Updated weights for policy 1, policy_version 937212 (0.0007) [2023-12-27 12:08:47,786][20381] Updated weights for policy 1, policy_version 937222 (0.0005) [2023-12-27 12:08:47,852][20381] Updated weights for policy 1, policy_version 937232 (0.0005) [2023-12-27 12:08:48,248][20347] Updated weights for policy 0, policy_version 938119 (0.0010) [2023-12-27 12:08:48,306][20347] Updated weights for policy 0, policy_version 938129 (0.0010) [2023-12-27 12:08:48,373][20347] Updated weights for policy 0, policy_version 938139 (0.0009) [2023-12-27 12:08:48,436][20381] Updated weights for policy 1, policy_version 937242 (0.0005) [2023-12-27 12:08:48,505][20381] Updated weights for policy 1, policy_version 937252 (0.0005) [2023-12-27 12:08:48,564][20381] Updated weights for policy 1, policy_version 937262 (0.0006) [2023-12-27 12:08:48,607][20292] KL-divergence is very high: 120.5118 [2023-12-27 12:08:48,618][20292] KL-divergence is very high: 229.6086 [2023-12-27 12:08:48,621][20381] Updated weights for policy 1, policy_version 937272 (0.0007) [2023-12-27 12:08:49,060][20347] Updated weights for policy 0, policy_version 938149 (0.0010) [2023-12-27 12:08:49,116][20347] Updated weights for policy 0, policy_version 938159 (0.0008) [2023-12-27 12:08:49,171][20347] Updated weights for policy 0, policy_version 938169 (0.0007) [2023-12-27 12:08:49,266][20381] Updated weights for policy 1, policy_version 937282 (0.0008) [2023-12-27 12:08:49,317][20381] Updated weights for policy 1, policy_version 937292 (0.0006) [2023-12-27 12:08:49,385][20381] Updated weights for policy 1, policy_version 937302 (0.0008) [2023-12-27 12:08:49,870][20347] Updated weights for policy 0, policy_version 938179 (0.0007) [2023-12-27 12:08:49,927][20347] Updated weights for policy 0, policy_version 938189 (0.0008) [2023-12-27 12:08:49,985][20347] Updated weights for policy 0, policy_version 938199 (0.0007) [2023-12-27 12:08:50,158][20381] Updated weights for policy 1, policy_version 937312 (0.0008) [2023-12-27 12:08:50,215][20381] Updated weights for policy 1, policy_version 937322 (0.0007) [2023-12-27 12:08:50,275][20381] Updated weights for policy 1, policy_version 937332 (0.0008) [2023-12-27 12:08:50,656][20347] Updated weights for policy 0, policy_version 938209 (0.0007) [2023-12-27 12:08:50,712][20347] Updated weights for policy 0, policy_version 938219 (0.0007) [2023-12-27 12:08:50,769][20347] Updated weights for policy 0, policy_version 938229 (0.0006) [2023-12-27 12:08:50,829][20347] Updated weights for policy 0, policy_version 938239 (0.0006) [2023-12-27 12:08:50,983][20381] Updated weights for policy 1, policy_version 937342 (0.0008) [2023-12-27 12:08:51,045][20381] Updated weights for policy 1, policy_version 937352 (0.0008) [2023-12-27 12:08:51,106][20381] Updated weights for policy 1, policy_version 937362 (0.0007) [2023-12-27 12:08:51,524][20347] Updated weights for policy 0, policy_version 938249 (0.0007) [2023-12-27 12:08:51,581][20347] Updated weights for policy 0, policy_version 938259 (0.0007) [2023-12-27 12:08:51,649][20347] Updated weights for policy 0, policy_version 938269 (0.0007) [2023-12-27 12:08:51,856][20381] Updated weights for policy 1, policy_version 937372 (0.0009) [2023-12-27 12:08:51,915][20381] Updated weights for policy 1, policy_version 937382 (0.0011) [2023-12-27 12:08:51,974][20381] Updated weights for policy 1, policy_version 937392 (0.0011) [2023-12-27 12:08:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 480239616. Throughput: 0: 10369.1, 1: 10165.6. Samples: 480213172. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:52,363][19170] Avg episode reward: [(0, '990.250'), (1, '902.150')] [2023-12-27 12:08:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000937400_240001024.pth... [2023-12-27 12:08:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000936216_239697920.pth [2023-12-27 12:08:52,376][20347] Updated weights for policy 0, policy_version 938279 (0.0007) [2023-12-27 12:08:52,447][20347] Updated weights for policy 0, policy_version 938289 (0.0007) [2023-12-27 12:08:52,503][20347] Updated weights for policy 0, policy_version 938299 (0.0007) [2023-12-27 12:08:52,531][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000938304_240246784.pth... [2023-12-27 12:08:52,534][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000937056_239927296.pth [2023-12-27 12:08:52,682][20381] Updated weights for policy 1, policy_version 937402 (0.0010) [2023-12-27 12:08:52,737][20381] Updated weights for policy 1, policy_version 937412 (0.0008) [2023-12-27 12:08:52,796][20381] Updated weights for policy 1, policy_version 937422 (0.0008) [2023-12-27 12:08:52,855][20381] Updated weights for policy 1, policy_version 937432 (0.0009) [2023-12-27 12:08:53,205][20347] Updated weights for policy 0, policy_version 938309 (0.0008) [2023-12-27 12:08:53,257][20347] Updated weights for policy 0, policy_version 938319 (0.0008) [2023-12-27 12:08:53,310][20347] Updated weights for policy 0, policy_version 938329 (0.0006) [2023-12-27 12:08:53,553][20381] Updated weights for policy 1, policy_version 937442 (0.0006) [2023-12-27 12:08:53,604][20381] Updated weights for policy 1, policy_version 937452 (0.0005) [2023-12-27 12:08:53,655][20381] Updated weights for policy 1, policy_version 937462 (0.0007) [2023-12-27 12:08:53,988][20347] Updated weights for policy 0, policy_version 938339 (0.0007) [2023-12-27 12:08:54,040][20347] Updated weights for policy 0, policy_version 938349 (0.0007) [2023-12-27 12:08:54,102][20347] Updated weights for policy 0, policy_version 938359 (0.0007) [2023-12-27 12:08:54,381][20381] Updated weights for policy 1, policy_version 937472 (0.0007) [2023-12-27 12:08:54,429][20381] Updated weights for policy 1, policy_version 937482 (0.0008) [2023-12-27 12:08:54,478][20381] Updated weights for policy 1, policy_version 937492 (0.0008) [2023-12-27 12:08:54,719][20347] Updated weights for policy 0, policy_version 938369 (0.0005) [2023-12-27 12:08:54,771][20347] Updated weights for policy 0, policy_version 938379 (0.0007) [2023-12-27 12:08:54,828][20347] Updated weights for policy 0, policy_version 938389 (0.0008) [2023-12-27 12:08:54,886][20347] Updated weights for policy 0, policy_version 938399 (0.0007) [2023-12-27 12:08:55,188][20381] Updated weights for policy 1, policy_version 937502 (0.0006) [2023-12-27 12:08:55,225][20292] KL-divergence is very high: 123.4093 [2023-12-27 12:08:55,240][20381] Updated weights for policy 1, policy_version 937512 (0.0005) [2023-12-27 12:08:55,241][20292] KL-divergence is very high: 175.3506 [2023-12-27 12:08:55,265][20292] KL-divergence is very high: 116.1427 [2023-12-27 12:08:55,287][20381] Updated weights for policy 1, policy_version 937522 (0.0006) [2023-12-27 12:08:55,541][20347] Updated weights for policy 0, policy_version 938409 (0.0007) [2023-12-27 12:08:55,595][20347] Updated weights for policy 0, policy_version 938419 (0.0008) [2023-12-27 12:08:55,645][20347] Updated weights for policy 0, policy_version 938429 (0.0007) [2023-12-27 12:08:55,906][20381] Updated weights for policy 1, policy_version 937532 (0.0008) [2023-12-27 12:08:55,961][20381] Updated weights for policy 1, policy_version 937542 (0.0007) [2023-12-27 12:08:56,018][20381] Updated weights for policy 1, policy_version 937552 (0.0008) [2023-12-27 12:08:56,349][20347] Updated weights for policy 0, policy_version 938439 (0.0007) [2023-12-27 12:08:56,400][20347] Updated weights for policy 0, policy_version 938449 (0.0007) [2023-12-27 12:08:56,450][20347] Updated weights for policy 0, policy_version 938459 (0.0007) [2023-12-27 12:08:56,710][20381] Updated weights for policy 1, policy_version 937562 (0.0008) [2023-12-27 12:08:56,755][20381] Updated weights for policy 1, policy_version 937572 (0.0007) [2023-12-27 12:08:56,806][20381] Updated weights for policy 1, policy_version 937582 (0.0008) [2023-12-27 12:08:56,865][20381] Updated weights for policy 1, policy_version 937592 (0.0008) [2023-12-27 12:08:57,123][20347] Updated weights for policy 0, policy_version 938469 (0.0007) [2023-12-27 12:08:57,175][20347] Updated weights for policy 0, policy_version 938479 (0.0008) [2023-12-27 12:08:57,227][20347] Updated weights for policy 0, policy_version 938489 (0.0007) [2023-12-27 12:08:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 480346112. Throughput: 0: 10337.0, 1: 10199.3. Samples: 480333888. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:08:57,363][19170] Avg episode reward: [(0, '970.760'), (1, '892.330')] [2023-12-27 12:08:57,575][20381] Updated weights for policy 1, policy_version 937602 (0.0007) [2023-12-27 12:08:57,624][20381] Updated weights for policy 1, policy_version 937612 (0.0005) [2023-12-27 12:08:57,674][20381] Updated weights for policy 1, policy_version 937622 (0.0005) [2023-12-27 12:08:57,970][20347] Updated weights for policy 0, policy_version 938499 (0.0008) [2023-12-27 12:08:58,018][20347] Updated weights for policy 0, policy_version 938509 (0.0007) [2023-12-27 12:08:58,063][20347] Updated weights for policy 0, policy_version 938519 (0.0007) [2023-12-27 12:08:58,303][20381] Updated weights for policy 1, policy_version 937632 (0.0005) [2023-12-27 12:08:58,368][20381] Updated weights for policy 1, policy_version 937642 (0.0008) [2023-12-27 12:08:58,431][20381] Updated weights for policy 1, policy_version 937652 (0.0008) [2023-12-27 12:08:58,914][20347] Updated weights for policy 0, policy_version 938529 (0.0008) [2023-12-27 12:08:58,978][20347] Updated weights for policy 0, policy_version 938539 (0.0008) [2023-12-27 12:08:59,041][20347] Updated weights for policy 0, policy_version 938549 (0.0007) [2023-12-27 12:08:59,102][20347] Updated weights for policy 0, policy_version 938559 (0.0007) [2023-12-27 12:08:59,255][20381] Updated weights for policy 1, policy_version 937662 (0.0008) [2023-12-27 12:08:59,323][20381] Updated weights for policy 1, policy_version 937672 (0.0008) [2023-12-27 12:08:59,385][20381] Updated weights for policy 1, policy_version 937682 (0.0007) [2023-12-27 12:08:59,831][20347] Updated weights for policy 0, policy_version 938569 (0.0007) [2023-12-27 12:08:59,892][20347] Updated weights for policy 0, policy_version 938579 (0.0007) [2023-12-27 12:08:59,956][20347] Updated weights for policy 0, policy_version 938589 (0.0007) [2023-12-27 12:09:00,046][20381] Updated weights for policy 1, policy_version 937692 (0.0007) [2023-12-27 12:09:00,115][20381] Updated weights for policy 1, policy_version 937702 (0.0006) [2023-12-27 12:09:00,175][20381] Updated weights for policy 1, policy_version 937712 (0.0006) [2023-12-27 12:09:00,647][20347] Updated weights for policy 0, policy_version 938599 (0.0005) [2023-12-27 12:09:00,663][20381] Updated weights for policy 1, policy_version 937722 (0.0005) [2023-12-27 12:09:00,698][20347] Updated weights for policy 0, policy_version 938609 (0.0006) [2023-12-27 12:09:00,714][20381] Updated weights for policy 1, policy_version 937732 (0.0005) [2023-12-27 12:09:00,756][20347] Updated weights for policy 0, policy_version 938619 (0.0010) [2023-12-27 12:09:00,761][20381] Updated weights for policy 1, policy_version 937742 (0.0005) [2023-12-27 12:09:00,809][20381] Updated weights for policy 1, policy_version 937752 (0.0005) [2023-12-27 12:09:01,443][20381] Updated weights for policy 1, policy_version 937762 (0.0008) [2023-12-27 12:09:01,492][20381] Updated weights for policy 1, policy_version 937772 (0.0009) [2023-12-27 12:09:01,494][20347] Updated weights for policy 0, policy_version 938629 (0.0008) [2023-12-27 12:09:01,547][20347] Updated weights for policy 0, policy_version 938639 (0.0005) [2023-12-27 12:09:01,549][20381] Updated weights for policy 1, policy_version 937782 (0.0010) [2023-12-27 12:09:01,609][20347] Updated weights for policy 0, policy_version 938649 (0.0007) [2023-12-27 12:09:02,248][20381] Updated weights for policy 1, policy_version 937792 (0.0010) [2023-12-27 12:09:02,313][20381] Updated weights for policy 1, policy_version 937802 (0.0006) [2023-12-27 12:09:02,315][20347] Updated weights for policy 0, policy_version 938659 (0.0008) [2023-12-27 12:09:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20438.4). Total num frames: 480436224. Throughput: 0: 10256.6, 1: 10245.0. Samples: 480455080. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:02,363][19170] Avg episode reward: [(0, '961.030'), (1, '902.120')] [2023-12-27 12:09:02,382][20381] Updated weights for policy 1, policy_version 937812 (0.0010) [2023-12-27 12:09:02,383][20347] Updated weights for policy 0, policy_version 938669 (0.0006) [2023-12-27 12:09:02,435][20347] Updated weights for policy 0, policy_version 938679 (0.0006) [2023-12-27 12:09:03,022][20347] Updated weights for policy 0, policy_version 938689 (0.0005) [2023-12-27 12:09:03,028][20381] Updated weights for policy 1, policy_version 937822 (0.0007) [2023-12-27 12:09:03,073][20347] Updated weights for policy 0, policy_version 938699 (0.0006) [2023-12-27 12:09:03,076][20381] Updated weights for policy 1, policy_version 937832 (0.0006) [2023-12-27 12:09:03,124][20381] Updated weights for policy 1, policy_version 937842 (0.0010) [2023-12-27 12:09:03,128][20347] Updated weights for policy 0, policy_version 938709 (0.0007) [2023-12-27 12:09:03,181][20347] Updated weights for policy 0, policy_version 938719 (0.0005) [2023-12-27 12:09:03,696][20381] Updated weights for policy 1, policy_version 937852 (0.0008) [2023-12-27 12:09:03,717][20347] Updated weights for policy 0, policy_version 938729 (0.0005) [2023-12-27 12:09:03,744][20381] Updated weights for policy 1, policy_version 937862 (0.0005) [2023-12-27 12:09:03,775][20347] Updated weights for policy 0, policy_version 938739 (0.0006) [2023-12-27 12:09:03,792][20381] Updated weights for policy 1, policy_version 937872 (0.0005) [2023-12-27 12:09:03,823][20347] Updated weights for policy 0, policy_version 938749 (0.0006) [2023-12-27 12:09:04,429][20347] Updated weights for policy 0, policy_version 938759 (0.0006) [2023-12-27 12:09:04,448][20381] Updated weights for policy 1, policy_version 937882 (0.0009) [2023-12-27 12:09:04,488][20347] Updated weights for policy 0, policy_version 938769 (0.0007) [2023-12-27 12:09:04,495][20381] Updated weights for policy 1, policy_version 937892 (0.0006) [2023-12-27 12:09:04,545][20347] Updated weights for policy 0, policy_version 938779 (0.0005) [2023-12-27 12:09:04,551][20381] Updated weights for policy 1, policy_version 937902 (0.0008) [2023-12-27 12:09:04,612][20381] Updated weights for policy 1, policy_version 937912 (0.0007) [2023-12-27 12:09:05,201][20347] Updated weights for policy 0, policy_version 938789 (0.0006) [2023-12-27 12:09:05,246][20381] Updated weights for policy 1, policy_version 937922 (0.0007) [2023-12-27 12:09:05,249][20347] Updated weights for policy 0, policy_version 938799 (0.0008) [2023-12-27 12:09:05,297][20347] Updated weights for policy 0, policy_version 938809 (0.0008) [2023-12-27 12:09:05,310][20381] Updated weights for policy 1, policy_version 937932 (0.0007) [2023-12-27 12:09:05,371][20381] Updated weights for policy 1, policy_version 937942 (0.0006) [2023-12-27 12:09:05,925][20381] Updated weights for policy 1, policy_version 937952 (0.0006) [2023-12-27 12:09:05,960][20347] Updated weights for policy 0, policy_version 938819 (0.0006) [2023-12-27 12:09:05,984][20381] Updated weights for policy 1, policy_version 937962 (0.0006) [2023-12-27 12:09:06,015][20347] Updated weights for policy 0, policy_version 938829 (0.0007) [2023-12-27 12:09:06,036][20381] Updated weights for policy 1, policy_version 937972 (0.0006) [2023-12-27 12:09:06,069][20347] Updated weights for policy 0, policy_version 938839 (0.0006) [2023-12-27 12:09:06,672][20381] Updated weights for policy 1, policy_version 937982 (0.0006) [2023-12-27 12:09:06,731][20381] Updated weights for policy 1, policy_version 937992 (0.0008) [2023-12-27 12:09:06,790][20381] Updated weights for policy 1, policy_version 938002 (0.0008) [2023-12-27 12:09:06,791][20347] Updated weights for policy 0, policy_version 938849 (0.0008) [2023-12-27 12:09:06,853][20347] Updated weights for policy 0, policy_version 938859 (0.0008) [2023-12-27 12:09:06,914][20347] Updated weights for policy 0, policy_version 938869 (0.0008) [2023-12-27 12:09:06,980][20347] Updated weights for policy 0, policy_version 938879 (0.0007) [2023-12-27 12:09:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 480550912. Throughput: 0: 10353.2, 1: 10284.2. Samples: 480521800. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:07,363][19170] Avg episode reward: [(0, '941.500'), (1, '951.080')] [2023-12-27 12:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000938008_240156672.pth... [2023-12-27 12:09:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000938880_240394240.pth... [2023-12-27 12:09:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000936824_239853568.pth [2023-12-27 12:09:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000937696_240091136.pth [2023-12-27 12:09:07,475][20381] Updated weights for policy 1, policy_version 938012 (0.0006) [2023-12-27 12:09:07,534][20381] Updated weights for policy 1, policy_version 938022 (0.0008) [2023-12-27 12:09:07,600][20381] Updated weights for policy 1, policy_version 938032 (0.0008) [2023-12-27 12:09:07,668][20347] Updated weights for policy 0, policy_version 938889 (0.0006) [2023-12-27 12:09:07,715][20347] Updated weights for policy 0, policy_version 938899 (0.0008) [2023-12-27 12:09:07,774][20347] Updated weights for policy 0, policy_version 938909 (0.0008) [2023-12-27 12:09:08,282][20381] Updated weights for policy 1, policy_version 938042 (0.0009) [2023-12-27 12:09:08,339][20381] Updated weights for policy 1, policy_version 938052 (0.0007) [2023-12-27 12:09:08,397][20381] Updated weights for policy 1, policy_version 938062 (0.0008) [2023-12-27 12:09:08,463][20381] Updated weights for policy 1, policy_version 938072 (0.0008) [2023-12-27 12:09:08,511][20347] Updated weights for policy 0, policy_version 938919 (0.0008) [2023-12-27 12:09:08,567][20347] Updated weights for policy 0, policy_version 938929 (0.0009) [2023-12-27 12:09:08,632][20347] Updated weights for policy 0, policy_version 938939 (0.0006) [2023-12-27 12:09:09,148][20381] Updated weights for policy 1, policy_version 938082 (0.0010) [2023-12-27 12:09:09,203][20381] Updated weights for policy 1, policy_version 938092 (0.0010) [2023-12-27 12:09:09,268][20347] Updated weights for policy 0, policy_version 938949 (0.0007) [2023-12-27 12:09:09,282][20381] Updated weights for policy 1, policy_version 938102 (0.0008) [2023-12-27 12:09:09,321][20347] Updated weights for policy 0, policy_version 938959 (0.0010) [2023-12-27 12:09:09,386][20347] Updated weights for policy 0, policy_version 938969 (0.0010) [2023-12-27 12:09:09,994][20381] Updated weights for policy 1, policy_version 938112 (0.0008) [2023-12-27 12:09:10,041][20381] Updated weights for policy 1, policy_version 938122 (0.0008) [2023-12-27 12:09:10,101][20381] Updated weights for policy 1, policy_version 938132 (0.0008) [2023-12-27 12:09:10,144][20347] Updated weights for policy 0, policy_version 938979 (0.0009) [2023-12-27 12:09:10,194][20347] Updated weights for policy 0, policy_version 938989 (0.0010) [2023-12-27 12:09:10,247][20347] Updated weights for policy 0, policy_version 938999 (0.0011) [2023-12-27 12:09:10,800][20381] Updated weights for policy 1, policy_version 938142 (0.0006) [2023-12-27 12:09:10,855][20381] Updated weights for policy 1, policy_version 938152 (0.0006) [2023-12-27 12:09:10,908][20381] Updated weights for policy 1, policy_version 938162 (0.0009) [2023-12-27 12:09:10,913][20347] Updated weights for policy 0, policy_version 939009 (0.0010) [2023-12-27 12:09:10,965][20347] Updated weights for policy 0, policy_version 939019 (0.0005) [2023-12-27 12:09:11,025][20347] Updated weights for policy 0, policy_version 939029 (0.0009) [2023-12-27 12:09:11,092][20347] Updated weights for policy 0, policy_version 939039 (0.0009) [2023-12-27 12:09:11,656][20381] Updated weights for policy 1, policy_version 938172 (0.0009) [2023-12-27 12:09:11,714][20381] Updated weights for policy 1, policy_version 938182 (0.0009) [2023-12-27 12:09:11,777][20347] Updated weights for policy 0, policy_version 939049 (0.0009) [2023-12-27 12:09:11,782][20381] Updated weights for policy 1, policy_version 938192 (0.0008) [2023-12-27 12:09:11,832][20347] Updated weights for policy 0, policy_version 939059 (0.0006) [2023-12-27 12:09:11,896][20347] Updated weights for policy 0, policy_version 939069 (0.0007) [2023-12-27 12:09:12,365][19170] Fps is (10 sec: 21293.3, 60 sec: 20479.0, 300 sec: 20493.7). Total num frames: 480649216. Throughput: 0: 10274.1, 1: 10257.3. Samples: 480643028. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:12,366][19170] Avg episode reward: [(0, '931.820'), (1, '941.330')] [2023-12-27 12:09:12,504][20381] Updated weights for policy 1, policy_version 938202 (0.0008) [2023-12-27 12:09:12,568][20381] Updated weights for policy 1, policy_version 938212 (0.0007) [2023-12-27 12:09:12,591][20347] Updated weights for policy 0, policy_version 939079 (0.0006) [2023-12-27 12:09:12,627][20381] Updated weights for policy 1, policy_version 938222 (0.0007) [2023-12-27 12:09:12,653][20347] Updated weights for policy 0, policy_version 939089 (0.0006) [2023-12-27 12:09:12,683][20381] Updated weights for policy 1, policy_version 938232 (0.0006) [2023-12-27 12:09:12,709][20347] Updated weights for policy 0, policy_version 939099 (0.0006) [2023-12-27 12:09:13,293][20381] Updated weights for policy 1, policy_version 938242 (0.0005) [2023-12-27 12:09:13,353][20381] Updated weights for policy 1, policy_version 938252 (0.0006) [2023-12-27 12:09:13,371][20347] Updated weights for policy 0, policy_version 939109 (0.0009) [2023-12-27 12:09:13,407][20381] Updated weights for policy 1, policy_version 938262 (0.0005) [2023-12-27 12:09:13,420][20347] Updated weights for policy 0, policy_version 939119 (0.0010) [2023-12-27 12:09:13,467][20347] Updated weights for policy 0, policy_version 939129 (0.0010) [2023-12-27 12:09:13,959][20381] Updated weights for policy 1, policy_version 938272 (0.0005) [2023-12-27 12:09:14,015][20381] Updated weights for policy 1, policy_version 938282 (0.0005) [2023-12-27 12:09:14,077][20381] Updated weights for policy 1, policy_version 938292 (0.0005) [2023-12-27 12:09:14,160][20347] Updated weights for policy 0, policy_version 939139 (0.0010) [2023-12-27 12:09:14,208][20347] Updated weights for policy 0, policy_version 939149 (0.0010) [2023-12-27 12:09:14,256][20347] Updated weights for policy 0, policy_version 939159 (0.0010) [2023-12-27 12:09:14,685][20381] Updated weights for policy 1, policy_version 938302 (0.0007) [2023-12-27 12:09:14,729][20381] Updated weights for policy 1, policy_version 938312 (0.0007) [2023-12-27 12:09:14,785][20381] Updated weights for policy 1, policy_version 938322 (0.0008) [2023-12-27 12:09:14,912][20347] Updated weights for policy 0, policy_version 939169 (0.0010) [2023-12-27 12:09:14,971][20347] Updated weights for policy 0, policy_version 939179 (0.0010) [2023-12-27 12:09:15,032][20347] Updated weights for policy 0, policy_version 939189 (0.0010) [2023-12-27 12:09:15,090][20347] Updated weights for policy 0, policy_version 939199 (0.0007) [2023-12-27 12:09:15,501][20381] Updated weights for policy 1, policy_version 938332 (0.0007) [2023-12-27 12:09:15,547][20381] Updated weights for policy 1, policy_version 938342 (0.0006) [2023-12-27 12:09:15,593][20381] Updated weights for policy 1, policy_version 938352 (0.0005) [2023-12-27 12:09:15,648][20347] Updated weights for policy 0, policy_version 939209 (0.0006) [2023-12-27 12:09:15,710][20347] Updated weights for policy 0, policy_version 939219 (0.0007) [2023-12-27 12:09:15,770][20347] Updated weights for policy 0, policy_version 939229 (0.0007) [2023-12-27 12:09:16,262][20381] Updated weights for policy 1, policy_version 938362 (0.0008) [2023-12-27 12:09:16,325][20381] Updated weights for policy 1, policy_version 938372 (0.0010) [2023-12-27 12:09:16,369][20347] Updated weights for policy 0, policy_version 939239 (0.0009) [2023-12-27 12:09:16,380][20381] Updated weights for policy 1, policy_version 938382 (0.0010) [2023-12-27 12:09:16,426][20347] Updated weights for policy 0, policy_version 939249 (0.0010) [2023-12-27 12:09:16,433][20381] Updated weights for policy 1, policy_version 938392 (0.0007) [2023-12-27 12:09:16,484][20347] Updated weights for policy 0, policy_version 939259 (0.0010) [2023-12-27 12:09:17,052][20381] Updated weights for policy 1, policy_version 938402 (0.0008) [2023-12-27 12:09:17,102][20381] Updated weights for policy 1, policy_version 938412 (0.0008) [2023-12-27 12:09:17,143][20347] Updated weights for policy 0, policy_version 939269 (0.0008) [2023-12-27 12:09:17,149][20381] Updated weights for policy 1, policy_version 938422 (0.0006) [2023-12-27 12:09:17,195][20347] Updated weights for policy 0, policy_version 939279 (0.0007) [2023-12-27 12:09:17,253][20347] Updated weights for policy 0, policy_version 939289 (0.0007) [2023-12-27 12:09:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 480763904. Throughput: 0: 10323.8, 1: 10289.2. Samples: 480769644. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:17,364][19170] Avg episode reward: [(0, '912.270'), (1, '941.320')] [2023-12-27 12:09:17,877][20381] Updated weights for policy 1, policy_version 938432 (0.0007) [2023-12-27 12:09:17,900][20347] Updated weights for policy 0, policy_version 939299 (0.0007) [2023-12-27 12:09:17,933][20381] Updated weights for policy 1, policy_version 938442 (0.0008) [2023-12-27 12:09:17,961][20347] Updated weights for policy 0, policy_version 939309 (0.0006) [2023-12-27 12:09:17,990][20381] Updated weights for policy 1, policy_version 938452 (0.0006) [2023-12-27 12:09:18,018][20347] Updated weights for policy 0, policy_version 939319 (0.0006) [2023-12-27 12:09:18,660][20381] Updated weights for policy 1, policy_version 938462 (0.0007) [2023-12-27 12:09:18,722][20381] Updated weights for policy 1, policy_version 938472 (0.0007) [2023-12-27 12:09:18,724][20347] Updated weights for policy 0, policy_version 939329 (0.0007) [2023-12-27 12:09:18,782][20381] Updated weights for policy 1, policy_version 938482 (0.0006) [2023-12-27 12:09:18,784][20347] Updated weights for policy 0, policy_version 939339 (0.0006) [2023-12-27 12:09:18,839][20347] Updated weights for policy 0, policy_version 939349 (0.0007) [2023-12-27 12:09:18,905][20347] Updated weights for policy 0, policy_version 939359 (0.0008) [2023-12-27 12:09:19,498][20381] Updated weights for policy 1, policy_version 938492 (0.0008) [2023-12-27 12:09:19,563][20381] Updated weights for policy 1, policy_version 938502 (0.0009) [2023-12-27 12:09:19,567][20347] Updated weights for policy 0, policy_version 939369 (0.0006) [2023-12-27 12:09:19,628][20347] Updated weights for policy 0, policy_version 939379 (0.0006) [2023-12-27 12:09:19,629][20381] Updated weights for policy 1, policy_version 938512 (0.0006) [2023-12-27 12:09:19,681][20347] Updated weights for policy 0, policy_version 939389 (0.0005) [2023-12-27 12:09:20,349][20381] Updated weights for policy 1, policy_version 938522 (0.0007) [2023-12-27 12:09:20,382][20347] Updated weights for policy 0, policy_version 939399 (0.0007) [2023-12-27 12:09:20,408][20381] Updated weights for policy 1, policy_version 938532 (0.0007) [2023-12-27 12:09:20,437][20347] Updated weights for policy 0, policy_version 939409 (0.0007) [2023-12-27 12:09:20,461][20292] KL-divergence is very high: 145.1214 [2023-12-27 12:09:20,466][20381] Updated weights for policy 1, policy_version 938542 (0.0006) [2023-12-27 12:09:20,487][20347] Updated weights for policy 0, policy_version 939419 (0.0005) [2023-12-27 12:09:20,520][20381] Updated weights for policy 1, policy_version 938552 (0.0006) [2023-12-27 12:09:21,200][20347] Updated weights for policy 0, policy_version 939429 (0.0007) [2023-12-27 12:09:21,266][20347] Updated weights for policy 0, policy_version 939439 (0.0007) [2023-12-27 12:09:21,286][20381] Updated weights for policy 1, policy_version 938562 (0.0008) [2023-12-27 12:09:21,323][20347] Updated weights for policy 0, policy_version 939449 (0.0007) [2023-12-27 12:09:21,350][20381] Updated weights for policy 1, policy_version 938572 (0.0007) [2023-12-27 12:09:21,418][20381] Updated weights for policy 1, policy_version 938582 (0.0009) [2023-12-27 12:09:22,050][20347] Updated weights for policy 0, policy_version 939459 (0.0008) [2023-12-27 12:09:22,121][20347] Updated weights for policy 0, policy_version 939469 (0.0010) [2023-12-27 12:09:22,152][20381] Updated weights for policy 1, policy_version 938592 (0.0006) [2023-12-27 12:09:22,179][20347] Updated weights for policy 0, policy_version 939479 (0.0011) [2023-12-27 12:09:22,210][20381] Updated weights for policy 1, policy_version 938602 (0.0008) [2023-12-27 12:09:22,273][20381] Updated weights for policy 1, policy_version 938612 (0.0008) [2023-12-27 12:09:22,362][19170] Fps is (10 sec: 21305.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 480862208. Throughput: 0: 10291.0, 1: 10254.2. Samples: 480829548. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:22,363][19170] Avg episode reward: [(0, '912.280'), (1, '902.140')] [2023-12-27 12:09:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000938616_240312320.pth... [2023-12-27 12:09:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000939488_240549888.pth... [2023-12-27 12:09:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000937400_240001024.pth [2023-12-27 12:09:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000938304_240246784.pth [2023-12-27 12:09:22,906][20347] Updated weights for policy 0, policy_version 939489 (0.0010) [2023-12-27 12:09:22,908][20381] Updated weights for policy 1, policy_version 938622 (0.0007) [2023-12-27 12:09:22,967][20381] Updated weights for policy 1, policy_version 938632 (0.0007) [2023-12-27 12:09:22,975][20347] Updated weights for policy 0, policy_version 939499 (0.0008) [2023-12-27 12:09:23,030][20347] Updated weights for policy 0, policy_version 939509 (0.0010) [2023-12-27 12:09:23,030][20381] Updated weights for policy 1, policy_version 938642 (0.0007) [2023-12-27 12:09:23,088][20347] Updated weights for policy 0, policy_version 939519 (0.0010) [2023-12-27 12:09:23,625][20381] Updated weights for policy 1, policy_version 938652 (0.0006) [2023-12-27 12:09:23,684][20381] Updated weights for policy 1, policy_version 938662 (0.0007) [2023-12-27 12:09:23,705][20347] Updated weights for policy 0, policy_version 939529 (0.0006) [2023-12-27 12:09:23,746][20381] Updated weights for policy 1, policy_version 938672 (0.0009) [2023-12-27 12:09:23,752][20347] Updated weights for policy 0, policy_version 939539 (0.0005) [2023-12-27 12:09:23,810][20347] Updated weights for policy 0, policy_version 939549 (0.0005) [2023-12-27 12:09:24,334][20381] Updated weights for policy 1, policy_version 938682 (0.0009) [2023-12-27 12:09:24,393][20381] Updated weights for policy 1, policy_version 938692 (0.0009) [2023-12-27 12:09:24,446][20347] Updated weights for policy 0, policy_version 939559 (0.0008) [2023-12-27 12:09:24,449][20381] Updated weights for policy 1, policy_version 938702 (0.0010) [2023-12-27 12:09:24,504][20381] Updated weights for policy 1, policy_version 938712 (0.0009) [2023-12-27 12:09:24,505][20347] Updated weights for policy 0, policy_version 939569 (0.0006) [2023-12-27 12:09:24,573][20347] Updated weights for policy 0, policy_version 939579 (0.0007) [2023-12-27 12:09:25,141][20381] Updated weights for policy 1, policy_version 938722 (0.0005) [2023-12-27 12:09:25,199][20381] Updated weights for policy 1, policy_version 938732 (0.0005) [2023-12-27 12:09:25,206][20347] Updated weights for policy 0, policy_version 939589 (0.0007) [2023-12-27 12:09:25,252][20347] Updated weights for policy 0, policy_version 939599 (0.0007) [2023-12-27 12:09:25,255][20381] Updated weights for policy 1, policy_version 938742 (0.0010) [2023-12-27 12:09:25,295][20347] Updated weights for policy 0, policy_version 939609 (0.0006) [2023-12-27 12:09:25,891][20381] Updated weights for policy 1, policy_version 938752 (0.0009) [2023-12-27 12:09:25,942][20381] Updated weights for policy 1, policy_version 938762 (0.0009) [2023-12-27 12:09:25,996][20381] Updated weights for policy 1, policy_version 938772 (0.0008) [2023-12-27 12:09:26,023][20347] Updated weights for policy 0, policy_version 939619 (0.0008) [2023-12-27 12:09:26,070][20347] Updated weights for policy 0, policy_version 939629 (0.0008) [2023-12-27 12:09:26,120][20347] Updated weights for policy 0, policy_version 939639 (0.0009) [2023-12-27 12:09:26,624][20381] Updated weights for policy 1, policy_version 938782 (0.0006) [2023-12-27 12:09:26,669][20381] Updated weights for policy 1, policy_version 938792 (0.0006) [2023-12-27 12:09:26,713][20381] Updated weights for policy 1, policy_version 938802 (0.0005) [2023-12-27 12:09:26,780][20347] Updated weights for policy 0, policy_version 939649 (0.0010) [2023-12-27 12:09:26,837][20347] Updated weights for policy 0, policy_version 939659 (0.0007) [2023-12-27 12:09:26,890][20347] Updated weights for policy 0, policy_version 939669 (0.0007) [2023-12-27 12:09:26,947][20347] Updated weights for policy 0, policy_version 939679 (0.0007) [2023-12-27 12:09:27,343][20381] Updated weights for policy 1, policy_version 938812 (0.0010) [2023-12-27 12:09:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 480960512. Throughput: 0: 10269.2, 1: 10274.2. Samples: 480953092. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:27,363][19170] Avg episode reward: [(0, '961.040'), (1, '863.060')] [2023-12-27 12:09:27,394][20381] Updated weights for policy 1, policy_version 938822 (0.0010) [2023-12-27 12:09:27,449][20381] Updated weights for policy 1, policy_version 938832 (0.0007) [2023-12-27 12:09:27,561][20347] Updated weights for policy 0, policy_version 939690 (0.0009) [2023-12-27 12:09:27,613][20347] Updated weights for policy 0, policy_version 939700 (0.0010) [2023-12-27 12:09:27,673][20347] Updated weights for policy 0, policy_version 939710 (0.0009) [2023-12-27 12:09:28,059][20381] Updated weights for policy 1, policy_version 938842 (0.0006) [2023-12-27 12:09:28,114][20381] Updated weights for policy 1, policy_version 938852 (0.0009) [2023-12-27 12:09:28,170][20381] Updated weights for policy 1, policy_version 938862 (0.0009) [2023-12-27 12:09:28,235][20381] Updated weights for policy 1, policy_version 938872 (0.0007) [2023-12-27 12:09:28,336][20347] Updated weights for policy 0, policy_version 939720 (0.0007) [2023-12-27 12:09:28,393][20347] Updated weights for policy 0, policy_version 939730 (0.0009) [2023-12-27 12:09:28,447][20347] Updated weights for policy 0, policy_version 939740 (0.0008) [2023-12-27 12:09:28,879][20381] Updated weights for policy 1, policy_version 938882 (0.0007) [2023-12-27 12:09:28,927][20381] Updated weights for policy 1, policy_version 938892 (0.0009) [2023-12-27 12:09:28,991][20381] Updated weights for policy 1, policy_version 938902 (0.0010) [2023-12-27 12:09:29,111][20347] Updated weights for policy 0, policy_version 939750 (0.0008) [2023-12-27 12:09:29,158][20347] Updated weights for policy 0, policy_version 939760 (0.0010) [2023-12-27 12:09:29,205][20347] Updated weights for policy 0, policy_version 939770 (0.0010) [2023-12-27 12:09:29,665][20381] Updated weights for policy 1, policy_version 938912 (0.0010) [2023-12-27 12:09:29,723][20381] Updated weights for policy 1, policy_version 938922 (0.0010) [2023-12-27 12:09:29,780][20381] Updated weights for policy 1, policy_version 938932 (0.0010) [2023-12-27 12:09:29,923][20347] Updated weights for policy 0, policy_version 939780 (0.0008) [2023-12-27 12:09:29,984][20347] Updated weights for policy 0, policy_version 939790 (0.0008) [2023-12-27 12:09:30,049][20347] Updated weights for policy 0, policy_version 939800 (0.0008) [2023-12-27 12:09:30,435][20381] Updated weights for policy 1, policy_version 938942 (0.0008) [2023-12-27 12:09:30,489][20381] Updated weights for policy 1, policy_version 938952 (0.0006) [2023-12-27 12:09:30,542][20381] Updated weights for policy 1, policy_version 938962 (0.0009) [2023-12-27 12:09:30,681][20347] Updated weights for policy 0, policy_version 939810 (0.0008) [2023-12-27 12:09:30,730][20347] Updated weights for policy 0, policy_version 939820 (0.0010) [2023-12-27 12:09:30,787][20347] Updated weights for policy 0, policy_version 939830 (0.0010) [2023-12-27 12:09:30,848][20347] Updated weights for policy 0, policy_version 939840 (0.0010) [2023-12-27 12:09:31,218][20381] Updated weights for policy 1, policy_version 938972 (0.0009) [2023-12-27 12:09:31,280][20381] Updated weights for policy 1, policy_version 938982 (0.0008) [2023-12-27 12:09:31,332][20381] Updated weights for policy 1, policy_version 938992 (0.0008) [2023-12-27 12:09:31,543][20347] Updated weights for policy 0, policy_version 939850 (0.0009) [2023-12-27 12:09:31,600][20347] Updated weights for policy 0, policy_version 939860 (0.0009) [2023-12-27 12:09:31,665][20347] Updated weights for policy 0, policy_version 939870 (0.0007) [2023-12-27 12:09:32,047][20381] Updated weights for policy 1, policy_version 939002 (0.0007) [2023-12-27 12:09:32,112][20381] Updated weights for policy 1, policy_version 939012 (0.0008) [2023-12-27 12:09:32,177][20381] Updated weights for policy 1, policy_version 939022 (0.0008) [2023-12-27 12:09:32,238][20381] Updated weights for policy 1, policy_version 939032 (0.0008) [2023-12-27 12:09:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 481067008. Throughput: 0: 10238.1, 1: 10365.0. Samples: 481077596. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.310')] [2023-12-27 12:09:32,406][20347] Updated weights for policy 0, policy_version 939880 (0.0008) [2023-12-27 12:09:32,461][20347] Updated weights for policy 0, policy_version 939890 (0.0009) [2023-12-27 12:09:32,514][20347] Updated weights for policy 0, policy_version 939900 (0.0008) [2023-12-27 12:09:32,914][20381] Updated weights for policy 1, policy_version 939042 (0.0008) [2023-12-27 12:09:32,972][20381] Updated weights for policy 1, policy_version 939052 (0.0008) [2023-12-27 12:09:33,026][20381] Updated weights for policy 1, policy_version 939062 (0.0007) [2023-12-27 12:09:33,214][20347] Updated weights for policy 0, policy_version 939910 (0.0007) [2023-12-27 12:09:33,267][20347] Updated weights for policy 0, policy_version 939920 (0.0007) [2023-12-27 12:09:33,315][20347] Updated weights for policy 0, policy_version 939930 (0.0006) [2023-12-27 12:09:33,707][20381] Updated weights for policy 1, policy_version 939072 (0.0008) [2023-12-27 12:09:33,762][20381] Updated weights for policy 1, policy_version 939082 (0.0007) [2023-12-27 12:09:33,806][20381] Updated weights for policy 1, policy_version 939092 (0.0007) [2023-12-27 12:09:34,002][20347] Updated weights for policy 0, policy_version 939940 (0.0008) [2023-12-27 12:09:34,053][20347] Updated weights for policy 0, policy_version 939950 (0.0008) [2023-12-27 12:09:34,098][20347] Updated weights for policy 0, policy_version 939960 (0.0007) [2023-12-27 12:09:34,450][20381] Updated weights for policy 1, policy_version 939102 (0.0008) [2023-12-27 12:09:34,507][20381] Updated weights for policy 1, policy_version 939112 (0.0008) [2023-12-27 12:09:34,566][20381] Updated weights for policy 1, policy_version 939122 (0.0007) [2023-12-27 12:09:34,769][20347] Updated weights for policy 0, policy_version 939970 (0.0007) [2023-12-27 12:09:34,834][20347] Updated weights for policy 0, policy_version 939980 (0.0005) [2023-12-27 12:09:34,891][20347] Updated weights for policy 0, policy_version 939990 (0.0005) [2023-12-27 12:09:34,947][20347] Updated weights for policy 0, policy_version 940000 (0.0005) [2023-12-27 12:09:35,262][20381] Updated weights for policy 1, policy_version 939132 (0.0007) [2023-12-27 12:09:35,307][20381] Updated weights for policy 1, policy_version 939142 (0.0007) [2023-12-27 12:09:35,353][20381] Updated weights for policy 1, policy_version 939152 (0.0009) [2023-12-27 12:09:35,539][20347] Updated weights for policy 0, policy_version 940010 (0.0007) [2023-12-27 12:09:35,595][20347] Updated weights for policy 0, policy_version 940020 (0.0007) [2023-12-27 12:09:35,650][20347] Updated weights for policy 0, policy_version 940030 (0.0008) [2023-12-27 12:09:36,032][20381] Updated weights for policy 1, policy_version 939162 (0.0009) [2023-12-27 12:09:36,080][20381] Updated weights for policy 1, policy_version 939172 (0.0007) [2023-12-27 12:09:36,143][20381] Updated weights for policy 1, policy_version 939182 (0.0008) [2023-12-27 12:09:36,199][20381] Updated weights for policy 1, policy_version 939192 (0.0008) [2023-12-27 12:09:36,353][20347] Updated weights for policy 0, policy_version 940040 (0.0007) [2023-12-27 12:09:36,409][20347] Updated weights for policy 0, policy_version 940050 (0.0008) [2023-12-27 12:09:36,469][20347] Updated weights for policy 0, policy_version 940060 (0.0007) [2023-12-27 12:09:36,904][20381] Updated weights for policy 1, policy_version 939202 (0.0006) [2023-12-27 12:09:36,964][20381] Updated weights for policy 1, policy_version 939212 (0.0007) [2023-12-27 12:09:37,022][20381] Updated weights for policy 1, policy_version 939222 (0.0005) [2023-12-27 12:09:37,207][20347] Updated weights for policy 0, policy_version 940070 (0.0008) [2023-12-27 12:09:37,266][20347] Updated weights for policy 0, policy_version 940080 (0.0007) [2023-12-27 12:09:37,333][20347] Updated weights for policy 0, policy_version 940090 (0.0008) [2023-12-27 12:09:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 481165312. Throughput: 0: 10235.6, 1: 10346.0. Samples: 481139340. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 12:09:37,363][19170] Avg episode reward: [(0, '990.290'), (1, '931.570')] [2023-12-27 12:09:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000939224_240467968.pth... [2023-12-27 12:09:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000940096_240705536.pth... [2023-12-27 12:09:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000938008_240156672.pth [2023-12-27 12:09:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000938880_240394240.pth [2023-12-27 12:09:37,696][20381] Updated weights for policy 1, policy_version 939232 (0.0008) [2023-12-27 12:09:37,758][20381] Updated weights for policy 1, policy_version 939242 (0.0008) [2023-12-27 12:09:37,815][20381] Updated weights for policy 1, policy_version 939252 (0.0008) [2023-12-27 12:09:38,034][20347] Updated weights for policy 0, policy_version 940100 (0.0008) [2023-12-27 12:09:38,082][20347] Updated weights for policy 0, policy_version 940110 (0.0008) [2023-12-27 12:09:38,097][20291] KL-divergence is very high: 141.2269 [2023-12-27 12:09:38,137][20347] Updated weights for policy 0, policy_version 940120 (0.0007) [2023-12-27 12:09:38,487][20381] Updated weights for policy 1, policy_version 939262 (0.0007) [2023-12-27 12:09:38,550][20381] Updated weights for policy 1, policy_version 939272 (0.0008) [2023-12-27 12:09:38,611][20381] Updated weights for policy 1, policy_version 939282 (0.0008) [2023-12-27 12:09:38,875][20347] Updated weights for policy 0, policy_version 940130 (0.0007) [2023-12-27 12:09:38,927][20347] Updated weights for policy 0, policy_version 940140 (0.0008) [2023-12-27 12:09:38,980][20347] Updated weights for policy 0, policy_version 940150 (0.0008) [2023-12-27 12:09:39,029][20347] Updated weights for policy 0, policy_version 940160 (0.0007) [2023-12-27 12:09:39,262][20381] Updated weights for policy 1, policy_version 939292 (0.0009) [2023-12-27 12:09:39,326][20381] Updated weights for policy 1, policy_version 939302 (0.0008) [2023-12-27 12:09:39,386][20381] Updated weights for policy 1, policy_version 939312 (0.0006) [2023-12-27 12:09:39,796][20347] Updated weights for policy 0, policy_version 940170 (0.0008) [2023-12-27 12:09:39,862][20347] Updated weights for policy 0, policy_version 940180 (0.0009) [2023-12-27 12:09:39,929][20347] Updated weights for policy 0, policy_version 940190 (0.0007) [2023-12-27 12:09:40,092][20381] Updated weights for policy 1, policy_version 939322 (0.0007) [2023-12-27 12:09:40,150][20381] Updated weights for policy 1, policy_version 939332 (0.0008) [2023-12-27 12:09:40,211][20381] Updated weights for policy 1, policy_version 939342 (0.0008) [2023-12-27 12:09:40,271][20381] Updated weights for policy 1, policy_version 939352 (0.0008) [2023-12-27 12:09:40,613][20347] Updated weights for policy 0, policy_version 940200 (0.0007) [2023-12-27 12:09:40,668][20347] Updated weights for policy 0, policy_version 940210 (0.0007) [2023-12-27 12:09:40,727][20347] Updated weights for policy 0, policy_version 940220 (0.0005) [2023-12-27 12:09:41,006][20381] Updated weights for policy 1, policy_version 939362 (0.0006) [2023-12-27 12:09:41,071][20381] Updated weights for policy 1, policy_version 939372 (0.0008) [2023-12-27 12:09:41,139][20381] Updated weights for policy 1, policy_version 939382 (0.0008) [2023-12-27 12:09:41,403][20347] Updated weights for policy 0, policy_version 940230 (0.0007) [2023-12-27 12:09:41,462][20347] Updated weights for policy 0, policy_version 940240 (0.0006) [2023-12-27 12:09:41,521][20347] Updated weights for policy 0, policy_version 940250 (0.0005) [2023-12-27 12:09:41,877][20381] Updated weights for policy 1, policy_version 939392 (0.0008) [2023-12-27 12:09:41,933][20381] Updated weights for policy 1, policy_version 939402 (0.0008) [2023-12-27 12:09:41,996][20381] Updated weights for policy 1, policy_version 939412 (0.0008) [2023-12-27 12:09:42,191][20347] Updated weights for policy 0, policy_version 940260 (0.0007) [2023-12-27 12:09:42,254][20347] Updated weights for policy 0, policy_version 940270 (0.0007) [2023-12-27 12:09:42,320][20347] Updated weights for policy 0, policy_version 940280 (0.0008) [2023-12-27 12:09:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 481263616. Throughput: 0: 10203.5, 1: 10346.7. Samples: 481258648. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:09:42,363][19170] Avg episode reward: [(0, '990.290'), (1, '931.550')] [2023-12-27 12:09:42,695][20381] Updated weights for policy 1, policy_version 939422 (0.0008) [2023-12-27 12:09:42,756][20381] Updated weights for policy 1, policy_version 939432 (0.0007) [2023-12-27 12:09:42,817][20381] Updated weights for policy 1, policy_version 939442 (0.0006) [2023-12-27 12:09:43,074][20347] Updated weights for policy 0, policy_version 940290 (0.0007) [2023-12-27 12:09:43,123][20347] Updated weights for policy 0, policy_version 940300 (0.0010) [2023-12-27 12:09:43,168][20347] Updated weights for policy 0, policy_version 940310 (0.0009) [2023-12-27 12:09:43,217][20347] Updated weights for policy 0, policy_version 940320 (0.0008) [2023-12-27 12:09:43,413][20381] Updated weights for policy 1, policy_version 939452 (0.0007) [2023-12-27 12:09:43,474][20381] Updated weights for policy 1, policy_version 939462 (0.0008) [2023-12-27 12:09:43,523][20381] Updated weights for policy 1, policy_version 939472 (0.0008) [2023-12-27 12:09:43,889][20347] Updated weights for policy 0, policy_version 940330 (0.0008) [2023-12-27 12:09:43,940][20347] Updated weights for policy 0, policy_version 940340 (0.0008) [2023-12-27 12:09:44,000][20347] Updated weights for policy 0, policy_version 940350 (0.0008) [2023-12-27 12:09:44,194][20381] Updated weights for policy 1, policy_version 939482 (0.0008) [2023-12-27 12:09:44,259][20381] Updated weights for policy 1, policy_version 939492 (0.0006) [2023-12-27 12:09:44,318][20381] Updated weights for policy 1, policy_version 939502 (0.0008) [2023-12-27 12:09:44,380][20381] Updated weights for policy 1, policy_version 939512 (0.0007) [2023-12-27 12:09:44,680][20347] Updated weights for policy 0, policy_version 940360 (0.0008) [2023-12-27 12:09:44,733][20347] Updated weights for policy 0, policy_version 940370 (0.0008) [2023-12-27 12:09:44,806][20347] Updated weights for policy 0, policy_version 940380 (0.0008) [2023-12-27 12:09:45,034][20381] Updated weights for policy 1, policy_version 939522 (0.0008) [2023-12-27 12:09:45,100][20381] Updated weights for policy 1, policy_version 939532 (0.0009) [2023-12-27 12:09:45,161][20381] Updated weights for policy 1, policy_version 939542 (0.0008) [2023-12-27 12:09:45,470][20347] Updated weights for policy 0, policy_version 940390 (0.0008) [2023-12-27 12:09:45,531][20347] Updated weights for policy 0, policy_version 940400 (0.0007) [2023-12-27 12:09:45,586][20347] Updated weights for policy 0, policy_version 940410 (0.0008) [2023-12-27 12:09:45,819][20381] Updated weights for policy 1, policy_version 939552 (0.0006) [2023-12-27 12:09:45,881][20381] Updated weights for policy 1, policy_version 939562 (0.0005) [2023-12-27 12:09:45,940][20381] Updated weights for policy 1, policy_version 939572 (0.0005) [2023-12-27 12:09:46,175][20347] Updated weights for policy 0, policy_version 940420 (0.0008) [2023-12-27 12:09:46,241][20347] Updated weights for policy 0, policy_version 940430 (0.0007) [2023-12-27 12:09:46,296][20347] Updated weights for policy 0, policy_version 940440 (0.0007) [2023-12-27 12:09:46,479][20381] Updated weights for policy 1, policy_version 939582 (0.0006) [2023-12-27 12:09:46,531][20381] Updated weights for policy 1, policy_version 939592 (0.0005) [2023-12-27 12:09:46,587][20381] Updated weights for policy 1, policy_version 939602 (0.0005) [2023-12-27 12:09:47,027][20347] Updated weights for policy 0, policy_version 940450 (0.0008) [2023-12-27 12:09:47,096][20347] Updated weights for policy 0, policy_version 940460 (0.0005) [2023-12-27 12:09:47,160][20381] Updated weights for policy 1, policy_version 939612 (0.0006) [2023-12-27 12:09:47,161][20347] Updated weights for policy 0, policy_version 940470 (0.0006) [2023-12-27 12:09:47,210][20381] Updated weights for policy 1, policy_version 939622 (0.0006) [2023-12-27 12:09:47,217][20347] Updated weights for policy 0, policy_version 940480 (0.0006) [2023-12-27 12:09:47,256][20381] Updated weights for policy 1, policy_version 939632 (0.0007) [2023-12-27 12:09:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 481378304. Throughput: 0: 10271.0, 1: 10356.5. Samples: 481383316. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:09:47,363][19170] Avg episode reward: [(0, '960.930'), (1, '931.500')] [2023-12-27 12:09:47,854][20381] Updated weights for policy 1, policy_version 939642 (0.0007) [2023-12-27 12:09:47,880][20347] Updated weights for policy 0, policy_version 940490 (0.0007) [2023-12-27 12:09:47,921][20381] Updated weights for policy 1, policy_version 939652 (0.0006) [2023-12-27 12:09:47,934][20347] Updated weights for policy 0, policy_version 940500 (0.0007) [2023-12-27 12:09:47,977][20381] Updated weights for policy 1, policy_version 939662 (0.0005) [2023-12-27 12:09:47,980][20347] Updated weights for policy 0, policy_version 940510 (0.0007) [2023-12-27 12:09:48,034][20381] Updated weights for policy 1, policy_version 939672 (0.0008) [2023-12-27 12:09:48,621][20347] Updated weights for policy 0, policy_version 940520 (0.0007) [2023-12-27 12:09:48,675][20347] Updated weights for policy 0, policy_version 940530 (0.0008) [2023-12-27 12:09:48,729][20347] Updated weights for policy 0, policy_version 940540 (0.0007) [2023-12-27 12:09:48,739][20381] Updated weights for policy 1, policy_version 939682 (0.0006) [2023-12-27 12:09:48,795][20381] Updated weights for policy 1, policy_version 939692 (0.0007) [2023-12-27 12:09:48,859][20381] Updated weights for policy 1, policy_version 939702 (0.0008) [2023-12-27 12:09:49,395][20347] Updated weights for policy 0, policy_version 940550 (0.0006) [2023-12-27 12:09:49,459][20347] Updated weights for policy 0, policy_version 940560 (0.0006) [2023-12-27 12:09:49,518][20347] Updated weights for policy 0, policy_version 940570 (0.0006) [2023-12-27 12:09:49,615][20381] Updated weights for policy 1, policy_version 939712 (0.0008) [2023-12-27 12:09:49,672][20381] Updated weights for policy 1, policy_version 939722 (0.0008) [2023-12-27 12:09:49,729][20381] Updated weights for policy 1, policy_version 939732 (0.0007) [2023-12-27 12:09:50,164][20347] Updated weights for policy 0, policy_version 940580 (0.0006) [2023-12-27 12:09:50,228][20347] Updated weights for policy 0, policy_version 940590 (0.0008) [2023-12-27 12:09:50,283][20347] Updated weights for policy 0, policy_version 940600 (0.0008) [2023-12-27 12:09:50,495][20381] Updated weights for policy 1, policy_version 939742 (0.0008) [2023-12-27 12:09:50,549][20381] Updated weights for policy 1, policy_version 939752 (0.0008) [2023-12-27 12:09:50,608][20381] Updated weights for policy 1, policy_version 939762 (0.0009) [2023-12-27 12:09:50,930][20347] Updated weights for policy 0, policy_version 940610 (0.0007) [2023-12-27 12:09:50,992][20347] Updated weights for policy 0, policy_version 940620 (0.0007) [2023-12-27 12:09:51,056][20347] Updated weights for policy 0, policy_version 940630 (0.0007) [2023-12-27 12:09:51,108][20347] Updated weights for policy 0, policy_version 940640 (0.0009) [2023-12-27 12:09:51,343][20381] Updated weights for policy 1, policy_version 939772 (0.0008) [2023-12-27 12:09:51,409][20381] Updated weights for policy 1, policy_version 939782 (0.0008) [2023-12-27 12:09:51,470][20381] Updated weights for policy 1, policy_version 939792 (0.0006) [2023-12-27 12:09:51,837][20347] Updated weights for policy 0, policy_version 940650 (0.0008) [2023-12-27 12:09:51,901][20347] Updated weights for policy 0, policy_version 940660 (0.0008) [2023-12-27 12:09:51,954][20347] Updated weights for policy 0, policy_version 940670 (0.0009) [2023-12-27 12:09:52,101][20381] Updated weights for policy 1, policy_version 939802 (0.0007) [2023-12-27 12:09:52,151][20381] Updated weights for policy 1, policy_version 939812 (0.0007) [2023-12-27 12:09:52,169][20292] KL-divergence is very high: 157.4769 [2023-12-27 12:09:52,204][20381] Updated weights for policy 1, policy_version 939822 (0.0008) [2023-12-27 12:09:52,266][20381] Updated weights for policy 1, policy_version 939832 (0.0008) [2023-12-27 12:09:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 481476608. Throughput: 0: 10199.5, 1: 10295.5. Samples: 481444072. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:09:52,363][19170] Avg episode reward: [(0, '951.260'), (1, '912.090')] [2023-12-27 12:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000939832_240623616.pth... [2023-12-27 12:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000940672_240852992.pth... [2023-12-27 12:09:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000938616_240312320.pth [2023-12-27 12:09:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000939488_240549888.pth [2023-12-27 12:09:52,657][20347] Updated weights for policy 0, policy_version 940680 (0.0008) [2023-12-27 12:09:52,718][20347] Updated weights for policy 0, policy_version 940690 (0.0007) [2023-12-27 12:09:52,781][20347] Updated weights for policy 0, policy_version 940700 (0.0008) [2023-12-27 12:09:52,943][20381] Updated weights for policy 1, policy_version 939842 (0.0007) [2023-12-27 12:09:52,996][20381] Updated weights for policy 1, policy_version 939852 (0.0008) [2023-12-27 12:09:53,056][20381] Updated weights for policy 1, policy_version 939862 (0.0006) [2023-12-27 12:09:53,421][20347] Updated weights for policy 0, policy_version 940710 (0.0006) [2023-12-27 12:09:53,483][20347] Updated weights for policy 0, policy_version 940720 (0.0005) [2023-12-27 12:09:53,547][20347] Updated weights for policy 0, policy_version 940730 (0.0005) [2023-12-27 12:09:53,640][20381] Updated weights for policy 1, policy_version 939872 (0.0008) [2023-12-27 12:09:53,694][20381] Updated weights for policy 1, policy_version 939882 (0.0006) [2023-12-27 12:09:53,749][20381] Updated weights for policy 1, policy_version 939892 (0.0006) [2023-12-27 12:09:54,153][20347] Updated weights for policy 0, policy_version 940740 (0.0008) [2023-12-27 12:09:54,219][20347] Updated weights for policy 0, policy_version 940750 (0.0005) [2023-12-27 12:09:54,285][20347] Updated weights for policy 0, policy_version 940760 (0.0008) [2023-12-27 12:09:54,405][20381] Updated weights for policy 1, policy_version 939902 (0.0006) [2023-12-27 12:09:54,450][20381] Updated weights for policy 1, policy_version 939912 (0.0009) [2023-12-27 12:09:54,512][20381] Updated weights for policy 1, policy_version 939922 (0.0009) [2023-12-27 12:09:54,846][20347] Updated weights for policy 0, policy_version 940770 (0.0010) [2023-12-27 12:09:54,909][20347] Updated weights for policy 0, policy_version 940780 (0.0006) [2023-12-27 12:09:54,973][20347] Updated weights for policy 0, policy_version 940790 (0.0005) [2023-12-27 12:09:55,037][20347] Updated weights for policy 0, policy_version 940800 (0.0010) [2023-12-27 12:09:55,205][20381] Updated weights for policy 1, policy_version 939932 (0.0009) [2023-12-27 12:09:55,253][20381] Updated weights for policy 1, policy_version 939942 (0.0007) [2023-12-27 12:09:55,301][20381] Updated weights for policy 1, policy_version 939952 (0.0008) [2023-12-27 12:09:55,643][20347] Updated weights for policy 0, policy_version 940810 (0.0010) [2023-12-27 12:09:55,703][20347] Updated weights for policy 0, policy_version 940820 (0.0010) [2023-12-27 12:09:55,763][20347] Updated weights for policy 0, policy_version 940830 (0.0010) [2023-12-27 12:09:56,023][20381] Updated weights for policy 1, policy_version 939962 (0.0007) [2023-12-27 12:09:56,081][20381] Updated weights for policy 1, policy_version 939972 (0.0007) [2023-12-27 12:09:56,139][20381] Updated weights for policy 1, policy_version 939982 (0.0007) [2023-12-27 12:09:56,197][20381] Updated weights for policy 1, policy_version 939992 (0.0007) [2023-12-27 12:09:56,412][20347] Updated weights for policy 0, policy_version 940840 (0.0006) [2023-12-27 12:09:56,473][20347] Updated weights for policy 0, policy_version 940850 (0.0005) [2023-12-27 12:09:56,533][20347] Updated weights for policy 0, policy_version 940860 (0.0005) [2023-12-27 12:09:56,907][20381] Updated weights for policy 1, policy_version 940002 (0.0007) [2023-12-27 12:09:56,959][20381] Updated weights for policy 1, policy_version 940012 (0.0006) [2023-12-27 12:09:57,013][20381] Updated weights for policy 1, policy_version 940022 (0.0007) [2023-12-27 12:09:57,076][20347] Updated weights for policy 0, policy_version 940870 (0.0008) [2023-12-27 12:09:57,130][20347] Updated weights for policy 0, policy_version 940880 (0.0010) [2023-12-27 12:09:57,196][20347] Updated weights for policy 0, policy_version 940890 (0.0010) [2023-12-27 12:09:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 481583104. Throughput: 0: 10259.5, 1: 10309.7. Samples: 481568584. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:09:57,363][19170] Avg episode reward: [(0, '980.550'), (1, '912.110')] [2023-12-27 12:09:57,683][20292] KL-divergence is very high: 354.0546 [2023-12-27 12:09:57,684][20381] Updated weights for policy 1, policy_version 940032 (0.0005) [2023-12-27 12:09:57,720][20292] KL-divergence is very high: 128.4199 [2023-12-27 12:09:57,729][20381] Updated weights for policy 1, policy_version 940042 (0.0005) [2023-12-27 12:09:57,772][20381] Updated weights for policy 1, policy_version 940052 (0.0005) [2023-12-27 12:09:57,862][20347] Updated weights for policy 0, policy_version 940900 (0.0010) [2023-12-27 12:09:57,924][20347] Updated weights for policy 0, policy_version 940910 (0.0010) [2023-12-27 12:09:57,977][20347] Updated weights for policy 0, policy_version 940920 (0.0008) [2023-12-27 12:09:58,437][20381] Updated weights for policy 1, policy_version 940062 (0.0007) [2023-12-27 12:09:58,496][20381] Updated weights for policy 1, policy_version 940072 (0.0007) [2023-12-27 12:09:58,559][20381] Updated weights for policy 1, policy_version 940082 (0.0007) [2023-12-27 12:09:58,681][20347] Updated weights for policy 0, policy_version 940930 (0.0005) [2023-12-27 12:09:58,741][20347] Updated weights for policy 0, policy_version 940940 (0.0006) [2023-12-27 12:09:58,807][20347] Updated weights for policy 0, policy_version 940950 (0.0008) [2023-12-27 12:09:58,878][20347] Updated weights for policy 0, policy_version 940960 (0.0008) [2023-12-27 12:09:59,358][20381] Updated weights for policy 1, policy_version 940092 (0.0008) [2023-12-27 12:09:59,420][20381] Updated weights for policy 1, policy_version 940102 (0.0006) [2023-12-27 12:09:59,488][20381] Updated weights for policy 1, policy_version 940112 (0.0005) [2023-12-27 12:09:59,647][20347] Updated weights for policy 0, policy_version 940970 (0.0007) [2023-12-27 12:09:59,696][20347] Updated weights for policy 0, policy_version 940980 (0.0008) [2023-12-27 12:09:59,753][20347] Updated weights for policy 0, policy_version 940990 (0.0008) [2023-12-27 12:10:00,158][20381] Updated weights for policy 1, policy_version 940122 (0.0008) [2023-12-27 12:10:00,207][20381] Updated weights for policy 1, policy_version 940132 (0.0008) [2023-12-27 12:10:00,260][20381] Updated weights for policy 1, policy_version 940142 (0.0008) [2023-12-27 12:10:00,305][20381] Updated weights for policy 1, policy_version 940152 (0.0007) [2023-12-27 12:10:00,447][20347] Updated weights for policy 0, policy_version 941000 (0.0008) [2023-12-27 12:10:00,506][20347] Updated weights for policy 0, policy_version 941010 (0.0007) [2023-12-27 12:10:00,558][20347] Updated weights for policy 0, policy_version 941020 (0.0008) [2023-12-27 12:10:01,014][20381] Updated weights for policy 1, policy_version 940162 (0.0008) [2023-12-27 12:10:01,067][20381] Updated weights for policy 1, policy_version 940172 (0.0007) [2023-12-27 12:10:01,116][20381] Updated weights for policy 1, policy_version 940182 (0.0007) [2023-12-27 12:10:01,271][20347] Updated weights for policy 0, policy_version 941030 (0.0009) [2023-12-27 12:10:01,330][20347] Updated weights for policy 0, policy_version 941040 (0.0008) [2023-12-27 12:10:01,398][20347] Updated weights for policy 0, policy_version 941050 (0.0008) [2023-12-27 12:10:01,891][20381] Updated weights for policy 1, policy_version 940192 (0.0007) [2023-12-27 12:10:01,941][20381] Updated weights for policy 1, policy_version 940202 (0.0006) [2023-12-27 12:10:02,003][20381] Updated weights for policy 1, policy_version 940212 (0.0006) [2023-12-27 12:10:02,084][20347] Updated weights for policy 0, policy_version 941060 (0.0008) [2023-12-27 12:10:02,146][20347] Updated weights for policy 0, policy_version 941070 (0.0010) [2023-12-27 12:10:02,205][20347] Updated weights for policy 0, policy_version 941080 (0.0011) [2023-12-27 12:10:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 481681408. Throughput: 0: 10191.0, 1: 10235.7. Samples: 481688840. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:02,363][19170] Avg episode reward: [(0, '970.940'), (1, '960.870')] [2023-12-27 12:10:02,643][20381] Updated weights for policy 1, policy_version 940222 (0.0007) [2023-12-27 12:10:02,697][20381] Updated weights for policy 1, policy_version 940232 (0.0008) [2023-12-27 12:10:02,749][20381] Updated weights for policy 1, policy_version 940242 (0.0007) [2023-12-27 12:10:02,867][20347] Updated weights for policy 0, policy_version 941090 (0.0010) [2023-12-27 12:10:02,912][20347] Updated weights for policy 0, policy_version 941100 (0.0008) [2023-12-27 12:10:02,964][20347] Updated weights for policy 0, policy_version 941110 (0.0008) [2023-12-27 12:10:03,028][20347] Updated weights for policy 0, policy_version 941120 (0.0006) [2023-12-27 12:10:03,394][20381] Updated weights for policy 1, policy_version 940252 (0.0009) [2023-12-27 12:10:03,432][20292] KL-divergence is very high: 109.1286 [2023-12-27 12:10:03,449][20381] Updated weights for policy 1, policy_version 940262 (0.0010) [2023-12-27 12:10:03,509][20381] Updated weights for policy 1, policy_version 940272 (0.0006) [2023-12-27 12:10:03,674][20347] Updated weights for policy 0, policy_version 941130 (0.0007) [2023-12-27 12:10:03,727][20347] Updated weights for policy 0, policy_version 941140 (0.0008) [2023-12-27 12:10:03,772][20347] Updated weights for policy 0, policy_version 941150 (0.0009) [2023-12-27 12:10:04,148][20381] Updated weights for policy 1, policy_version 940282 (0.0009) [2023-12-27 12:10:04,212][20381] Updated weights for policy 1, policy_version 940292 (0.0011) [2023-12-27 12:10:04,271][20381] Updated weights for policy 1, policy_version 940302 (0.0010) [2023-12-27 12:10:04,337][20381] Updated weights for policy 1, policy_version 940312 (0.0011) [2023-12-27 12:10:04,418][20347] Updated weights for policy 0, policy_version 941160 (0.0007) [2023-12-27 12:10:04,466][20347] Updated weights for policy 0, policy_version 941170 (0.0007) [2023-12-27 12:10:04,514][20347] Updated weights for policy 0, policy_version 941180 (0.0008) [2023-12-27 12:10:05,020][20381] Updated weights for policy 1, policy_version 940322 (0.0010) [2023-12-27 12:10:05,068][20381] Updated weights for policy 1, policy_version 940332 (0.0007) [2023-12-27 12:10:05,119][20381] Updated weights for policy 1, policy_version 940342 (0.0007) [2023-12-27 12:10:05,167][20347] Updated weights for policy 0, policy_version 941190 (0.0009) [2023-12-27 12:10:05,228][20347] Updated weights for policy 0, policy_version 941200 (0.0010) [2023-12-27 12:10:05,291][20347] Updated weights for policy 0, policy_version 941210 (0.0010) [2023-12-27 12:10:05,761][20381] Updated weights for policy 1, policy_version 940352 (0.0010) [2023-12-27 12:10:05,819][20381] Updated weights for policy 1, policy_version 940362 (0.0010) [2023-12-27 12:10:05,872][20381] Updated weights for policy 1, policy_version 940372 (0.0010) [2023-12-27 12:10:05,895][20347] Updated weights for policy 0, policy_version 941220 (0.0009) [2023-12-27 12:10:05,952][20347] Updated weights for policy 0, policy_version 941230 (0.0008) [2023-12-27 12:10:06,013][20347] Updated weights for policy 0, policy_version 941240 (0.0005) [2023-12-27 12:10:06,528][20381] Updated weights for policy 1, policy_version 940382 (0.0009) [2023-12-27 12:10:06,593][20381] Updated weights for policy 1, policy_version 940392 (0.0008) [2023-12-27 12:10:06,648][20347] Updated weights for policy 0, policy_version 941250 (0.0007) [2023-12-27 12:10:06,652][20381] Updated weights for policy 1, policy_version 940402 (0.0008) [2023-12-27 12:10:06,695][20347] Updated weights for policy 0, policy_version 941260 (0.0007) [2023-12-27 12:10:06,744][20347] Updated weights for policy 0, policy_version 941270 (0.0008) [2023-12-27 12:10:06,800][20347] Updated weights for policy 0, policy_version 941280 (0.0008) [2023-12-27 12:10:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 481779712. Throughput: 0: 10223.2, 1: 10272.4. Samples: 481751852. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:07,363][19170] Avg episode reward: [(0, '961.240'), (1, '941.340')] [2023-12-27 12:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000941280_241008640.pth... [2023-12-27 12:10:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000940096_240705536.pth [2023-12-27 12:10:07,376][20381] Updated weights for policy 1, policy_version 940412 (0.0007) [2023-12-27 12:10:07,436][20381] Updated weights for policy 1, policy_version 940422 (0.0008) [2023-12-27 12:10:07,454][20347] Updated weights for policy 0, policy_version 941290 (0.0006) [2023-12-27 12:10:07,492][20381] Updated weights for policy 1, policy_version 940432 (0.0006) [2023-12-27 12:10:07,514][20347] Updated weights for policy 0, policy_version 941300 (0.0007) [2023-12-27 12:10:07,538][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000940440_240779264.pth... [2023-12-27 12:10:07,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000939224_240467968.pth [2023-12-27 12:10:07,570][20347] Updated weights for policy 0, policy_version 941310 (0.0007) [2023-12-27 12:10:08,125][20381] Updated weights for policy 1, policy_version 940442 (0.0006) [2023-12-27 12:10:08,194][20381] Updated weights for policy 1, policy_version 940452 (0.0006) [2023-12-27 12:10:08,261][20381] Updated weights for policy 1, policy_version 940462 (0.0007) [2023-12-27 12:10:08,283][20347] Updated weights for policy 0, policy_version 941320 (0.0008) [2023-12-27 12:10:08,315][20381] Updated weights for policy 1, policy_version 940472 (0.0008) [2023-12-27 12:10:08,337][20347] Updated weights for policy 0, policy_version 941330 (0.0008) [2023-12-27 12:10:08,400][20347] Updated weights for policy 0, policy_version 941340 (0.0007) [2023-12-27 12:10:09,006][20381] Updated weights for policy 1, policy_version 940482 (0.0007) [2023-12-27 12:10:09,053][20381] Updated weights for policy 1, policy_version 940492 (0.0007) [2023-12-27 12:10:09,067][20347] Updated weights for policy 0, policy_version 941350 (0.0006) [2023-12-27 12:10:09,103][20381] Updated weights for policy 1, policy_version 940502 (0.0005) [2023-12-27 12:10:09,123][20347] Updated weights for policy 0, policy_version 941360 (0.0005) [2023-12-27 12:10:09,179][20347] Updated weights for policy 0, policy_version 941370 (0.0007) [2023-12-27 12:10:09,811][20381] Updated weights for policy 1, policy_version 940512 (0.0006) [2023-12-27 12:10:09,887][20381] Updated weights for policy 1, policy_version 940522 (0.0008) [2023-12-27 12:10:09,927][20347] Updated weights for policy 0, policy_version 941380 (0.0007) [2023-12-27 12:10:09,951][20381] Updated weights for policy 1, policy_version 940532 (0.0008) [2023-12-27 12:10:09,991][20347] Updated weights for policy 0, policy_version 941390 (0.0006) [2023-12-27 12:10:10,055][20347] Updated weights for policy 0, policy_version 941400 (0.0007) [2023-12-27 12:10:10,659][20381] Updated weights for policy 1, policy_version 940542 (0.0008) [2023-12-27 12:10:10,718][20381] Updated weights for policy 1, policy_version 940552 (0.0007) [2023-12-27 12:10:10,719][20347] Updated weights for policy 0, policy_version 941410 (0.0007) [2023-12-27 12:10:10,781][20347] Updated weights for policy 0, policy_version 941420 (0.0005) [2023-12-27 12:10:10,782][20381] Updated weights for policy 1, policy_version 940562 (0.0006) [2023-12-27 12:10:10,847][20347] Updated weights for policy 0, policy_version 941430 (0.0007) [2023-12-27 12:10:10,907][20347] Updated weights for policy 0, policy_version 941440 (0.0007) [2023-12-27 12:10:11,495][20381] Updated weights for policy 1, policy_version 940572 (0.0006) [2023-12-27 12:10:11,559][20381] Updated weights for policy 1, policy_version 940582 (0.0006) [2023-12-27 12:10:11,623][20381] Updated weights for policy 1, policy_version 940592 (0.0006) [2023-12-27 12:10:11,677][20347] Updated weights for policy 0, policy_version 941450 (0.0008) [2023-12-27 12:10:11,735][20347] Updated weights for policy 0, policy_version 941460 (0.0008) [2023-12-27 12:10:11,805][20347] Updated weights for policy 0, policy_version 941470 (0.0008) [2023-12-27 12:10:12,276][20381] Updated weights for policy 1, policy_version 940602 (0.0008) [2023-12-27 12:10:12,346][20381] Updated weights for policy 1, policy_version 940612 (0.0007) [2023-12-27 12:10:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20481.0, 300 sec: 20493.9). Total num frames: 481878016. Throughput: 0: 10244.1, 1: 10221.7. Samples: 481874052. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:12,363][19170] Avg episode reward: [(0, '990.300'), (1, '931.540')] [2023-12-27 12:10:12,413][20381] Updated weights for policy 1, policy_version 940622 (0.0006) [2023-12-27 12:10:12,472][20381] Updated weights for policy 1, policy_version 940632 (0.0008) [2023-12-27 12:10:12,558][20347] Updated weights for policy 0, policy_version 941480 (0.0008) [2023-12-27 12:10:12,619][20347] Updated weights for policy 0, policy_version 941490 (0.0007) [2023-12-27 12:10:12,672][20347] Updated weights for policy 0, policy_version 941500 (0.0006) [2023-12-27 12:10:13,108][20381] Updated weights for policy 1, policy_version 940642 (0.0007) [2023-12-27 12:10:13,163][20381] Updated weights for policy 1, policy_version 940652 (0.0007) [2023-12-27 12:10:13,219][20381] Updated weights for policy 1, policy_version 940662 (0.0009) [2023-12-27 12:10:13,340][20347] Updated weights for policy 0, policy_version 941510 (0.0006) [2023-12-27 12:10:13,395][20347] Updated weights for policy 0, policy_version 941520 (0.0006) [2023-12-27 12:10:13,451][20347] Updated weights for policy 0, policy_version 941530 (0.0008) [2023-12-27 12:10:13,843][20381] Updated weights for policy 1, policy_version 940672 (0.0008) [2023-12-27 12:10:13,900][20381] Updated weights for policy 1, policy_version 940682 (0.0009) [2023-12-27 12:10:13,956][20381] Updated weights for policy 1, policy_version 940692 (0.0011) [2023-12-27 12:10:14,036][20347] Updated weights for policy 0, policy_version 941540 (0.0008) [2023-12-27 12:10:14,093][20347] Updated weights for policy 0, policy_version 941550 (0.0005) [2023-12-27 12:10:14,153][20347] Updated weights for policy 0, policy_version 941560 (0.0008) [2023-12-27 12:10:14,641][20381] Updated weights for policy 1, policy_version 940702 (0.0008) [2023-12-27 12:10:14,667][20292] KL-divergence is very high: 129.4205 [2023-12-27 12:10:14,691][20292] KL-divergence is very high: 113.4064 [2023-12-27 12:10:14,696][20381] Updated weights for policy 1, policy_version 940712 (0.0008) [2023-12-27 12:10:14,705][20292] KL-divergence is very high: 107.3032 [2023-12-27 12:10:14,741][20381] Updated weights for policy 1, policy_version 940722 (0.0006) [2023-12-27 12:10:14,809][20347] Updated weights for policy 0, policy_version 941570 (0.0010) [2023-12-27 12:10:14,866][20347] Updated weights for policy 0, policy_version 941580 (0.0006) [2023-12-27 12:10:14,926][20347] Updated weights for policy 0, policy_version 941590 (0.0007) [2023-12-27 12:10:14,992][20347] Updated weights for policy 0, policy_version 941600 (0.0006) [2023-12-27 12:10:15,500][20381] Updated weights for policy 1, policy_version 940732 (0.0008) [2023-12-27 12:10:15,547][20381] Updated weights for policy 1, policy_version 940742 (0.0008) [2023-12-27 12:10:15,590][20347] Updated weights for policy 0, policy_version 941610 (0.0006) [2023-12-27 12:10:15,606][20381] Updated weights for policy 1, policy_version 940752 (0.0010) [2023-12-27 12:10:15,644][20347] Updated weights for policy 0, policy_version 941620 (0.0005) [2023-12-27 12:10:15,694][20347] Updated weights for policy 0, policy_version 941630 (0.0007) [2023-12-27 12:10:16,216][20381] Updated weights for policy 1, policy_version 940762 (0.0009) [2023-12-27 12:10:16,254][20347] Updated weights for policy 0, policy_version 941640 (0.0008) [2023-12-27 12:10:16,273][20381] Updated weights for policy 1, policy_version 940772 (0.0006) [2023-12-27 12:10:16,306][20347] Updated weights for policy 0, policy_version 941650 (0.0006) [2023-12-27 12:10:16,330][20381] Updated weights for policy 1, policy_version 940782 (0.0007) [2023-12-27 12:10:16,358][20347] Updated weights for policy 0, policy_version 941660 (0.0006) [2023-12-27 12:10:16,383][20381] Updated weights for policy 1, policy_version 940792 (0.0006) [2023-12-27 12:10:17,000][20381] Updated weights for policy 1, policy_version 940802 (0.0005) [2023-12-27 12:10:17,046][20381] Updated weights for policy 1, policy_version 940812 (0.0005) [2023-12-27 12:10:17,051][20347] Updated weights for policy 0, policy_version 941670 (0.0007) [2023-12-27 12:10:17,094][20381] Updated weights for policy 1, policy_version 940822 (0.0010) [2023-12-27 12:10:17,103][20347] Updated weights for policy 0, policy_version 941680 (0.0005) [2023-12-27 12:10:17,159][20347] Updated weights for policy 0, policy_version 941690 (0.0007) [2023-12-27 12:10:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 481992704. Throughput: 0: 10272.1, 1: 10229.4. Samples: 482000164. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:17,363][19170] Avg episode reward: [(0, '990.340'), (1, '911.960')] [2023-12-27 12:10:17,723][20381] Updated weights for policy 1, policy_version 940832 (0.0009) [2023-12-27 12:10:17,784][20381] Updated weights for policy 1, policy_version 940842 (0.0010) [2023-12-27 12:10:17,836][20381] Updated weights for policy 1, policy_version 940852 (0.0010) [2023-12-27 12:10:17,864][20347] Updated weights for policy 0, policy_version 941700 (0.0006) [2023-12-27 12:10:17,925][20347] Updated weights for policy 0, policy_version 941710 (0.0007) [2023-12-27 12:10:17,973][20347] Updated weights for policy 0, policy_version 941720 (0.0008) [2023-12-27 12:10:18,443][20381] Updated weights for policy 1, policy_version 940862 (0.0008) [2023-12-27 12:10:18,503][20381] Updated weights for policy 1, policy_version 940872 (0.0008) [2023-12-27 12:10:18,558][20381] Updated weights for policy 1, policy_version 940882 (0.0006) [2023-12-27 12:10:18,596][20347] Updated weights for policy 0, policy_version 941730 (0.0008) [2023-12-27 12:10:18,657][20347] Updated weights for policy 0, policy_version 941740 (0.0007) [2023-12-27 12:10:18,715][20347] Updated weights for policy 0, policy_version 941750 (0.0008) [2023-12-27 12:10:18,772][20347] Updated weights for policy 0, policy_version 941760 (0.0008) [2023-12-27 12:10:19,198][20381] Updated weights for policy 1, policy_version 940892 (0.0007) [2023-12-27 12:10:19,275][20381] Updated weights for policy 1, policy_version 940902 (0.0008) [2023-12-27 12:10:19,340][20381] Updated weights for policy 1, policy_version 940912 (0.0009) [2023-12-27 12:10:19,550][20347] Updated weights for policy 0, policy_version 941770 (0.0007) [2023-12-27 12:10:19,615][20347] Updated weights for policy 0, policy_version 941780 (0.0006) [2023-12-27 12:10:19,679][20347] Updated weights for policy 0, policy_version 941790 (0.0007) [2023-12-27 12:10:20,003][20381] Updated weights for policy 1, policy_version 940922 (0.0007) [2023-12-27 12:10:20,060][20381] Updated weights for policy 1, policy_version 940932 (0.0008) [2023-12-27 12:10:20,120][20381] Updated weights for policy 1, policy_version 940942 (0.0008) [2023-12-27 12:10:20,181][20381] Updated weights for policy 1, policy_version 940952 (0.0008) [2023-12-27 12:10:20,295][20347] Updated weights for policy 0, policy_version 941800 (0.0009) [2023-12-27 12:10:20,363][20347] Updated weights for policy 0, policy_version 941810 (0.0011) [2023-12-27 12:10:20,428][20347] Updated weights for policy 0, policy_version 941820 (0.0010) [2023-12-27 12:10:20,939][20381] Updated weights for policy 1, policy_version 940962 (0.0008) [2023-12-27 12:10:21,001][20381] Updated weights for policy 1, policy_version 940972 (0.0008) [2023-12-27 12:10:21,072][20381] Updated weights for policy 1, policy_version 940982 (0.0008) [2023-12-27 12:10:21,128][20347] Updated weights for policy 0, policy_version 941830 (0.0007) [2023-12-27 12:10:21,192][20347] Updated weights for policy 0, policy_version 941840 (0.0007) [2023-12-27 12:10:21,254][20347] Updated weights for policy 0, policy_version 941850 (0.0008) [2023-12-27 12:10:21,855][20381] Updated weights for policy 1, policy_version 940992 (0.0006) [2023-12-27 12:10:21,910][20381] Updated weights for policy 1, policy_version 941002 (0.0006) [2023-12-27 12:10:21,975][20381] Updated weights for policy 1, policy_version 941012 (0.0006) [2023-12-27 12:10:22,011][20347] Updated weights for policy 0, policy_version 941860 (0.0011) [2023-12-27 12:10:22,071][20347] Updated weights for policy 0, policy_version 941870 (0.0011) [2023-12-27 12:10:22,124][20347] Updated weights for policy 0, policy_version 941880 (0.0011) [2023-12-27 12:10:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 482091008. Throughput: 0: 10251.4, 1: 10249.1. Samples: 482061864. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:22,363][19170] Avg episode reward: [(0, '980.630'), (1, '951.090')] [2023-12-27 12:10:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000941888_241164288.pth... [2023-12-27 12:10:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000941016_240926720.pth... [2023-12-27 12:10:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000940672_240852992.pth [2023-12-27 12:10:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000939832_240623616.pth [2023-12-27 12:10:22,693][20381] Updated weights for policy 1, policy_version 941022 (0.0008) [2023-12-27 12:10:22,749][20381] Updated weights for policy 1, policy_version 941032 (0.0008) [2023-12-27 12:10:22,803][20381] Updated weights for policy 1, policy_version 941042 (0.0008) [2023-12-27 12:10:22,856][20347] Updated weights for policy 0, policy_version 941890 (0.0010) [2023-12-27 12:10:22,915][20347] Updated weights for policy 0, policy_version 941900 (0.0008) [2023-12-27 12:10:22,972][20347] Updated weights for policy 0, policy_version 941910 (0.0008) [2023-12-27 12:10:23,005][20291] KL-divergence is very high: 122.6424 [2023-12-27 12:10:23,028][20347] Updated weights for policy 0, policy_version 941920 (0.0008) [2023-12-27 12:10:23,532][20381] Updated weights for policy 1, policy_version 941052 (0.0008) [2023-12-27 12:10:23,586][20381] Updated weights for policy 1, policy_version 941062 (0.0008) [2023-12-27 12:10:23,647][20381] Updated weights for policy 1, policy_version 941072 (0.0006) [2023-12-27 12:10:23,667][20347] Updated weights for policy 0, policy_version 941930 (0.0009) [2023-12-27 12:10:23,715][20347] Updated weights for policy 0, policy_version 941940 (0.0005) [2023-12-27 12:10:23,773][20347] Updated weights for policy 0, policy_version 941950 (0.0005) [2023-12-27 12:10:24,328][20381] Updated weights for policy 1, policy_version 941082 (0.0009) [2023-12-27 12:10:24,371][20292] KL-divergence is very high: 219.8305 [2023-12-27 12:10:24,386][20381] Updated weights for policy 1, policy_version 941092 (0.0010) [2023-12-27 12:10:24,413][20292] KL-divergence is very high: 103.0556 [2023-12-27 12:10:24,430][20347] Updated weights for policy 0, policy_version 941960 (0.0009) [2023-12-27 12:10:24,443][20381] Updated weights for policy 1, policy_version 941102 (0.0011) [2023-12-27 12:10:24,486][20347] Updated weights for policy 0, policy_version 941970 (0.0005) [2023-12-27 12:10:24,505][20381] Updated weights for policy 1, policy_version 941112 (0.0010) [2023-12-27 12:10:24,540][20347] Updated weights for policy 0, policy_version 941980 (0.0005) [2023-12-27 12:10:25,081][20347] Updated weights for policy 0, policy_version 941990 (0.0008) [2023-12-27 12:10:25,149][20347] Updated weights for policy 0, policy_version 942000 (0.0008) [2023-12-27 12:10:25,183][20381] Updated weights for policy 1, policy_version 941122 (0.0006) [2023-12-27 12:10:25,203][20347] Updated weights for policy 0, policy_version 942010 (0.0010) [2023-12-27 12:10:25,246][20381] Updated weights for policy 1, policy_version 941132 (0.0006) [2023-12-27 12:10:25,306][20381] Updated weights for policy 1, policy_version 941142 (0.0007) [2023-12-27 12:10:25,827][20347] Updated weights for policy 0, policy_version 942020 (0.0010) [2023-12-27 12:10:25,876][20347] Updated weights for policy 0, policy_version 942030 (0.0010) [2023-12-27 12:10:25,930][20347] Updated weights for policy 0, policy_version 942040 (0.0010) [2023-12-27 12:10:25,953][20381] Updated weights for policy 1, policy_version 941152 (0.0006) [2023-12-27 12:10:26,010][20381] Updated weights for policy 1, policy_version 941162 (0.0005) [2023-12-27 12:10:26,074][20381] Updated weights for policy 1, policy_version 941172 (0.0005) [2023-12-27 12:10:26,569][20347] Updated weights for policy 0, policy_version 942050 (0.0010) [2023-12-27 12:10:26,615][20347] Updated weights for policy 0, policy_version 942060 (0.0005) [2023-12-27 12:10:26,668][20347] Updated weights for policy 0, policy_version 942070 (0.0007) [2023-12-27 12:10:26,700][20381] Updated weights for policy 1, policy_version 941182 (0.0006) [2023-12-27 12:10:26,715][20347] Updated weights for policy 0, policy_version 942080 (0.0010) [2023-12-27 12:10:26,754][20381] Updated weights for policy 1, policy_version 941192 (0.0007) [2023-12-27 12:10:26,801][20381] Updated weights for policy 1, policy_version 941202 (0.0007) [2023-12-27 12:10:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 482189312. Throughput: 0: 10319.8, 1: 10221.4. Samples: 482183004. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '970.660')] [2023-12-27 12:10:27,363][20381] Updated weights for policy 1, policy_version 941212 (0.0008) [2023-12-27 12:10:27,382][20347] Updated weights for policy 0, policy_version 942090 (0.0010) [2023-12-27 12:10:27,415][20381] Updated weights for policy 1, policy_version 941222 (0.0006) [2023-12-27 12:10:27,436][20347] Updated weights for policy 0, policy_version 942100 (0.0010) [2023-12-27 12:10:27,463][20381] Updated weights for policy 1, policy_version 941232 (0.0005) [2023-12-27 12:10:27,496][20347] Updated weights for policy 0, policy_version 942110 (0.0010) [2023-12-27 12:10:28,038][20347] Updated weights for policy 0, policy_version 942120 (0.0006) [2023-12-27 12:10:28,095][20347] Updated weights for policy 0, policy_version 942130 (0.0005) [2023-12-27 12:10:28,113][20381] Updated weights for policy 1, policy_version 941242 (0.0007) [2023-12-27 12:10:28,147][20347] Updated weights for policy 0, policy_version 942140 (0.0006) [2023-12-27 12:10:28,157][20381] Updated weights for policy 1, policy_version 941252 (0.0007) [2023-12-27 12:10:28,213][20381] Updated weights for policy 1, policy_version 941262 (0.0007) [2023-12-27 12:10:28,260][20381] Updated weights for policy 1, policy_version 941272 (0.0007) [2023-12-27 12:10:28,762][20347] Updated weights for policy 0, policy_version 942150 (0.0008) [2023-12-27 12:10:28,817][20347] Updated weights for policy 0, policy_version 942160 (0.0010) [2023-12-27 12:10:28,875][20347] Updated weights for policy 0, policy_version 942170 (0.0008) [2023-12-27 12:10:28,948][20381] Updated weights for policy 1, policy_version 941282 (0.0007) [2023-12-27 12:10:28,997][20381] Updated weights for policy 1, policy_version 941292 (0.0009) [2023-12-27 12:10:29,043][20381] Updated weights for policy 1, policy_version 941302 (0.0008) [2023-12-27 12:10:29,514][20347] Updated weights for policy 0, policy_version 942180 (0.0006) [2023-12-27 12:10:29,573][20347] Updated weights for policy 0, policy_version 942190 (0.0005) [2023-12-27 12:10:29,633][20347] Updated weights for policy 0, policy_version 942200 (0.0005) [2023-12-27 12:10:29,816][20381] Updated weights for policy 1, policy_version 941312 (0.0008) [2023-12-27 12:10:29,878][20381] Updated weights for policy 1, policy_version 941322 (0.0008) [2023-12-27 12:10:29,930][20381] Updated weights for policy 1, policy_version 941332 (0.0008) [2023-12-27 12:10:30,266][20347] Updated weights for policy 0, policy_version 942210 (0.0009) [2023-12-27 12:10:30,318][20347] Updated weights for policy 0, policy_version 942220 (0.0007) [2023-12-27 12:10:30,375][20347] Updated weights for policy 0, policy_version 942230 (0.0005) [2023-12-27 12:10:30,439][20347] Updated weights for policy 0, policy_version 942240 (0.0005) [2023-12-27 12:10:30,654][20381] Updated weights for policy 1, policy_version 941342 (0.0006) [2023-12-27 12:10:30,712][20381] Updated weights for policy 1, policy_version 941352 (0.0005) [2023-12-27 12:10:30,768][20381] Updated weights for policy 1, policy_version 941362 (0.0006) [2023-12-27 12:10:31,086][20347] Updated weights for policy 0, policy_version 942250 (0.0007) [2023-12-27 12:10:31,143][20291] KL-divergence is very high: 153.8505 [2023-12-27 12:10:31,150][20347] Updated weights for policy 0, policy_version 942260 (0.0008) [2023-12-27 12:10:31,158][20291] KL-divergence is very high: 173.8573 [2023-12-27 12:10:31,209][20347] Updated weights for policy 0, policy_version 942270 (0.0008) [2023-12-27 12:10:31,334][20381] Updated weights for policy 1, policy_version 941372 (0.0007) [2023-12-27 12:10:31,397][20381] Updated weights for policy 1, policy_version 941382 (0.0007) [2023-12-27 12:10:31,446][20381] Updated weights for policy 1, policy_version 941392 (0.0007) [2023-12-27 12:10:31,898][20347] Updated weights for policy 0, policy_version 942280 (0.0007) [2023-12-27 12:10:31,950][20347] Updated weights for policy 0, policy_version 942290 (0.0008) [2023-12-27 12:10:32,017][20347] Updated weights for policy 0, policy_version 942300 (0.0010) [2023-12-27 12:10:32,124][20381] Updated weights for policy 1, policy_version 941402 (0.0007) [2023-12-27 12:10:32,181][20381] Updated weights for policy 1, policy_version 941412 (0.0007) [2023-12-27 12:10:32,233][20381] Updated weights for policy 1, policy_version 941422 (0.0008) [2023-12-27 12:10:32,295][20381] Updated weights for policy 1, policy_version 941432 (0.0006) [2023-12-27 12:10:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 482304000. Throughput: 0: 10396.4, 1: 10205.4. Samples: 482310400. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:32,363][19170] Avg episode reward: [(0, '990.440'), (1, '941.420')] [2023-12-27 12:10:32,737][20347] Updated weights for policy 0, policy_version 942310 (0.0010) [2023-12-27 12:10:32,789][20347] Updated weights for policy 0, policy_version 942320 (0.0008) [2023-12-27 12:10:32,849][20347] Updated weights for policy 0, policy_version 942330 (0.0008) [2023-12-27 12:10:33,003][20381] Updated weights for policy 1, policy_version 941442 (0.0007) [2023-12-27 12:10:33,052][20381] Updated weights for policy 1, policy_version 941452 (0.0010) [2023-12-27 12:10:33,111][20381] Updated weights for policy 1, policy_version 941462 (0.0009) [2023-12-27 12:10:33,424][20347] Updated weights for policy 0, policy_version 942340 (0.0007) [2023-12-27 12:10:33,482][20347] Updated weights for policy 0, policy_version 942350 (0.0007) [2023-12-27 12:10:33,537][20347] Updated weights for policy 0, policy_version 942360 (0.0008) [2023-12-27 12:10:33,827][20381] Updated weights for policy 1, policy_version 941472 (0.0010) [2023-12-27 12:10:33,873][20381] Updated weights for policy 1, policy_version 941482 (0.0010) [2023-12-27 12:10:33,915][20381] Updated weights for policy 1, policy_version 941492 (0.0010) [2023-12-27 12:10:34,212][20347] Updated weights for policy 0, policy_version 942370 (0.0007) [2023-12-27 12:10:34,275][20347] Updated weights for policy 0, policy_version 942380 (0.0008) [2023-12-27 12:10:34,338][20347] Updated weights for policy 0, policy_version 942390 (0.0008) [2023-12-27 12:10:34,401][20347] Updated weights for policy 0, policy_version 942400 (0.0008) [2023-12-27 12:10:34,667][20381] Updated weights for policy 1, policy_version 941502 (0.0011) [2023-12-27 12:10:34,726][20381] Updated weights for policy 1, policy_version 941512 (0.0009) [2023-12-27 12:10:34,781][20381] Updated weights for policy 1, policy_version 941522 (0.0011) [2023-12-27 12:10:35,146][20347] Updated weights for policy 0, policy_version 942410 (0.0007) [2023-12-27 12:10:35,204][20347] Updated weights for policy 0, policy_version 942420 (0.0007) [2023-12-27 12:10:35,251][20347] Updated weights for policy 0, policy_version 942430 (0.0007) [2023-12-27 12:10:35,450][20381] Updated weights for policy 1, policy_version 941532 (0.0008) [2023-12-27 12:10:35,510][20381] Updated weights for policy 1, policy_version 941542 (0.0005) [2023-12-27 12:10:35,572][20381] Updated weights for policy 1, policy_version 941552 (0.0007) [2023-12-27 12:10:35,897][20347] Updated weights for policy 0, policy_version 942440 (0.0005) [2023-12-27 12:10:35,947][20347] Updated weights for policy 0, policy_version 942450 (0.0005) [2023-12-27 12:10:35,999][20347] Updated weights for policy 0, policy_version 942460 (0.0006) [2023-12-27 12:10:36,112][20381] Updated weights for policy 1, policy_version 941562 (0.0008) [2023-12-27 12:10:36,177][20381] Updated weights for policy 1, policy_version 941572 (0.0006) [2023-12-27 12:10:36,237][20381] Updated weights for policy 1, policy_version 941582 (0.0006) [2023-12-27 12:10:36,300][20381] Updated weights for policy 1, policy_version 941592 (0.0007) [2023-12-27 12:10:36,656][20347] Updated weights for policy 0, policy_version 942470 (0.0008) [2023-12-27 12:10:36,714][20347] Updated weights for policy 0, policy_version 942480 (0.0010) [2023-12-27 12:10:36,775][20347] Updated weights for policy 0, policy_version 942490 (0.0010) [2023-12-27 12:10:36,974][20381] Updated weights for policy 1, policy_version 941602 (0.0005) [2023-12-27 12:10:37,027][20381] Updated weights for policy 1, policy_version 941612 (0.0005) [2023-12-27 12:10:37,078][20381] Updated weights for policy 1, policy_version 941622 (0.0005) [2023-12-27 12:10:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 482402304. Throughput: 0: 10389.4, 1: 10195.2. Samples: 482370380. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:37,363][19170] Avg episode reward: [(0, '980.670'), (1, '931.730')] [2023-12-27 12:10:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000941624_241082368.pth... [2023-12-27 12:10:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000942496_241319936.pth... [2023-12-27 12:10:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000941280_241008640.pth [2023-12-27 12:10:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000940440_240779264.pth [2023-12-27 12:10:37,469][20347] Updated weights for policy 0, policy_version 942500 (0.0009) [2023-12-27 12:10:37,531][20347] Updated weights for policy 0, policy_version 942510 (0.0007) [2023-12-27 12:10:37,590][20347] Updated weights for policy 0, policy_version 942520 (0.0008) [2023-12-27 12:10:37,696][20381] Updated weights for policy 1, policy_version 941632 (0.0008) [2023-12-27 12:10:37,756][20381] Updated weights for policy 1, policy_version 941642 (0.0009) [2023-12-27 12:10:37,822][20381] Updated weights for policy 1, policy_version 941652 (0.0006) [2023-12-27 12:10:38,293][20347] Updated weights for policy 0, policy_version 942530 (0.0007) [2023-12-27 12:10:38,352][20347] Updated weights for policy 0, policy_version 942540 (0.0007) [2023-12-27 12:10:38,407][20347] Updated weights for policy 0, policy_version 942550 (0.0007) [2023-12-27 12:10:38,470][20347] Updated weights for policy 0, policy_version 942560 (0.0008) [2023-12-27 12:10:38,523][20381] Updated weights for policy 1, policy_version 941662 (0.0008) [2023-12-27 12:10:38,581][20381] Updated weights for policy 1, policy_version 941672 (0.0008) [2023-12-27 12:10:38,638][20381] Updated weights for policy 1, policy_version 941682 (0.0009) [2023-12-27 12:10:39,063][20347] Updated weights for policy 0, policy_version 942570 (0.0005) [2023-12-27 12:10:39,115][20347] Updated weights for policy 0, policy_version 942580 (0.0005) [2023-12-27 12:10:39,169][20347] Updated weights for policy 0, policy_version 942590 (0.0005) [2023-12-27 12:10:39,310][20381] Updated weights for policy 1, policy_version 941692 (0.0007) [2023-12-27 12:10:39,369][20381] Updated weights for policy 1, policy_version 941702 (0.0007) [2023-12-27 12:10:39,435][20381] Updated weights for policy 1, policy_version 941712 (0.0008) [2023-12-27 12:10:39,887][20347] Updated weights for policy 0, policy_version 942600 (0.0007) [2023-12-27 12:10:39,955][20347] Updated weights for policy 0, policy_version 942610 (0.0008) [2023-12-27 12:10:40,019][20347] Updated weights for policy 0, policy_version 942620 (0.0007) [2023-12-27 12:10:40,101][20381] Updated weights for policy 1, policy_version 941722 (0.0006) [2023-12-27 12:10:40,156][20381] Updated weights for policy 1, policy_version 941732 (0.0008) [2023-12-27 12:10:40,224][20381] Updated weights for policy 1, policy_version 941742 (0.0006) [2023-12-27 12:10:40,260][20292] KL-divergence is very high: 152.7992 [2023-12-27 12:10:40,275][20292] KL-divergence is very high: 320.2276 [2023-12-27 12:10:40,282][20381] Updated weights for policy 1, policy_version 941752 (0.0009) [2023-12-27 12:10:40,731][20347] Updated weights for policy 0, policy_version 942630 (0.0007) [2023-12-27 12:10:40,782][20347] Updated weights for policy 0, policy_version 942640 (0.0008) [2023-12-27 12:10:40,844][20347] Updated weights for policy 0, policy_version 942650 (0.0007) [2023-12-27 12:10:40,923][20381] Updated weights for policy 1, policy_version 941762 (0.0011) [2023-12-27 12:10:40,977][20381] Updated weights for policy 1, policy_version 941772 (0.0010) [2023-12-27 12:10:41,027][20381] Updated weights for policy 1, policy_version 941782 (0.0010) [2023-12-27 12:10:41,623][20347] Updated weights for policy 0, policy_version 942660 (0.0009) [2023-12-27 12:10:41,682][20347] Updated weights for policy 0, policy_version 942670 (0.0010) [2023-12-27 12:10:41,752][20347] Updated weights for policy 0, policy_version 942680 (0.0009) [2023-12-27 12:10:41,850][20381] Updated weights for policy 1, policy_version 941792 (0.0007) [2023-12-27 12:10:41,912][20381] Updated weights for policy 1, policy_version 941802 (0.0008) [2023-12-27 12:10:41,971][20381] Updated weights for policy 1, policy_version 941812 (0.0007) [2023-12-27 12:10:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 482500608. Throughput: 0: 10350.7, 1: 10210.2. Samples: 482493828. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:42,363][19170] Avg episode reward: [(0, '980.660'), (1, '931.730')] [2023-12-27 12:10:42,449][20347] Updated weights for policy 0, policy_version 942690 (0.0011) [2023-12-27 12:10:42,501][20347] Updated weights for policy 0, policy_version 942700 (0.0010) [2023-12-27 12:10:42,559][20347] Updated weights for policy 0, policy_version 942710 (0.0010) [2023-12-27 12:10:42,625][20347] Updated weights for policy 0, policy_version 942720 (0.0009) [2023-12-27 12:10:42,691][20381] Updated weights for policy 1, policy_version 941822 (0.0008) [2023-12-27 12:10:42,749][20381] Updated weights for policy 1, policy_version 941832 (0.0008) [2023-12-27 12:10:42,802][20381] Updated weights for policy 1, policy_version 941842 (0.0008) [2023-12-27 12:10:43,300][20347] Updated weights for policy 0, policy_version 942730 (0.0008) [2023-12-27 12:10:43,363][20347] Updated weights for policy 0, policy_version 942740 (0.0007) [2023-12-27 12:10:43,421][20347] Updated weights for policy 0, policy_version 942750 (0.0007) [2023-12-27 12:10:43,462][20381] Updated weights for policy 1, policy_version 941852 (0.0007) [2023-12-27 12:10:43,510][20381] Updated weights for policy 1, policy_version 941862 (0.0008) [2023-12-27 12:10:43,559][20381] Updated weights for policy 1, policy_version 941872 (0.0008) [2023-12-27 12:10:44,099][20347] Updated weights for policy 0, policy_version 942760 (0.0007) [2023-12-27 12:10:44,138][20381] Updated weights for policy 1, policy_version 941882 (0.0007) [2023-12-27 12:10:44,165][20347] Updated weights for policy 0, policy_version 942770 (0.0007) [2023-12-27 12:10:44,203][20381] Updated weights for policy 1, policy_version 941892 (0.0008) [2023-12-27 12:10:44,225][20347] Updated weights for policy 0, policy_version 942780 (0.0006) [2023-12-27 12:10:44,270][20381] Updated weights for policy 1, policy_version 941902 (0.0008) [2023-12-27 12:10:44,331][20381] Updated weights for policy 1, policy_version 941912 (0.0008) [2023-12-27 12:10:44,837][20347] Updated weights for policy 0, policy_version 942790 (0.0007) [2023-12-27 12:10:44,894][20347] Updated weights for policy 0, policy_version 942800 (0.0008) [2023-12-27 12:10:44,960][20347] Updated weights for policy 0, policy_version 942810 (0.0007) [2023-12-27 12:10:44,960][20381] Updated weights for policy 1, policy_version 941922 (0.0008) [2023-12-27 12:10:45,020][20381] Updated weights for policy 1, policy_version 941932 (0.0008) [2023-12-27 12:10:45,085][20381] Updated weights for policy 1, policy_version 941942 (0.0008) [2023-12-27 12:10:45,604][20347] Updated weights for policy 0, policy_version 942820 (0.0007) [2023-12-27 12:10:45,659][20347] Updated weights for policy 0, policy_version 942830 (0.0007) [2023-12-27 12:10:45,720][20347] Updated weights for policy 0, policy_version 942840 (0.0007) [2023-12-27 12:10:45,814][20381] Updated weights for policy 1, policy_version 941952 (0.0008) [2023-12-27 12:10:45,870][20381] Updated weights for policy 1, policy_version 941962 (0.0008) [2023-12-27 12:10:45,926][20381] Updated weights for policy 1, policy_version 941972 (0.0007) [2023-12-27 12:10:46,442][20347] Updated weights for policy 0, policy_version 942850 (0.0007) [2023-12-27 12:10:46,520][20347] Updated weights for policy 0, policy_version 942860 (0.0009) [2023-12-27 12:10:46,529][20381] Updated weights for policy 1, policy_version 941982 (0.0007) [2023-12-27 12:10:46,580][20347] Updated weights for policy 0, policy_version 942870 (0.0008) [2023-12-27 12:10:46,581][20381] Updated weights for policy 1, policy_version 941992 (0.0008) [2023-12-27 12:10:46,586][20292] KL-divergence is very high: 132.7151 [2023-12-27 12:10:46,637][20381] Updated weights for policy 1, policy_version 942002 (0.0007) [2023-12-27 12:10:46,645][20347] Updated weights for policy 0, policy_version 942880 (0.0008) [2023-12-27 12:10:47,254][20347] Updated weights for policy 0, policy_version 942890 (0.0005) [2023-12-27 12:10:47,311][20347] Updated weights for policy 0, policy_version 942900 (0.0006) [2023-12-27 12:10:47,328][20381] Updated weights for policy 1, policy_version 942012 (0.0006) [2023-12-27 12:10:47,360][20347] Updated weights for policy 0, policy_version 942910 (0.0005) [2023-12-27 12:10:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 482598912. Throughput: 0: 10330.9, 1: 10255.8. Samples: 482615244. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:47,363][19170] Avg episode reward: [(0, '990.430'), (1, '941.370')] [2023-12-27 12:10:47,383][20381] Updated weights for policy 1, policy_version 942022 (0.0005) [2023-12-27 12:10:47,442][20381] Updated weights for policy 1, policy_version 942032 (0.0005) [2023-12-27 12:10:48,025][20347] Updated weights for policy 0, policy_version 942920 (0.0006) [2023-12-27 12:10:48,044][20381] Updated weights for policy 1, policy_version 942042 (0.0006) [2023-12-27 12:10:48,084][20347] Updated weights for policy 0, policy_version 942930 (0.0007) [2023-12-27 12:10:48,097][20381] Updated weights for policy 1, policy_version 942052 (0.0007) [2023-12-27 12:10:48,138][20347] Updated weights for policy 0, policy_version 942940 (0.0006) [2023-12-27 12:10:48,151][20381] Updated weights for policy 1, policy_version 942062 (0.0007) [2023-12-27 12:10:48,214][20381] Updated weights for policy 1, policy_version 942072 (0.0008) [2023-12-27 12:10:48,805][20347] Updated weights for policy 0, policy_version 942950 (0.0008) [2023-12-27 12:10:48,861][20347] Updated weights for policy 0, policy_version 942960 (0.0009) [2023-12-27 12:10:48,916][20347] Updated weights for policy 0, policy_version 942970 (0.0009) [2023-12-27 12:10:48,938][20381] Updated weights for policy 1, policy_version 942082 (0.0007) [2023-12-27 12:10:48,993][20381] Updated weights for policy 1, policy_version 942092 (0.0008) [2023-12-27 12:10:49,046][20381] Updated weights for policy 1, policy_version 942102 (0.0008) [2023-12-27 12:10:49,565][20347] Updated weights for policy 0, policy_version 942980 (0.0008) [2023-12-27 12:10:49,630][20347] Updated weights for policy 0, policy_version 942990 (0.0008) [2023-12-27 12:10:49,689][20347] Updated weights for policy 0, policy_version 943000 (0.0008) [2023-12-27 12:10:49,782][20381] Updated weights for policy 1, policy_version 942112 (0.0006) [2023-12-27 12:10:49,846][20381] Updated weights for policy 1, policy_version 942122 (0.0007) [2023-12-27 12:10:49,902][20381] Updated weights for policy 1, policy_version 942132 (0.0006) [2023-12-27 12:10:50,432][20347] Updated weights for policy 0, policy_version 943010 (0.0008) [2023-12-27 12:10:50,482][20347] Updated weights for policy 0, policy_version 943020 (0.0008) [2023-12-27 12:10:50,538][20347] Updated weights for policy 0, policy_version 943030 (0.0007) [2023-12-27 12:10:50,550][20381] Updated weights for policy 1, policy_version 942142 (0.0008) [2023-12-27 12:10:50,604][20347] Updated weights for policy 0, policy_version 943040 (0.0007) [2023-12-27 12:10:50,610][20381] Updated weights for policy 1, policy_version 942152 (0.0008) [2023-12-27 12:10:50,669][20381] Updated weights for policy 1, policy_version 942162 (0.0005) [2023-12-27 12:10:51,367][20347] Updated weights for policy 0, policy_version 943050 (0.0008) [2023-12-27 12:10:51,377][20381] Updated weights for policy 1, policy_version 942172 (0.0007) [2023-12-27 12:10:51,428][20347] Updated weights for policy 0, policy_version 943060 (0.0007) [2023-12-27 12:10:51,439][20381] Updated weights for policy 1, policy_version 942182 (0.0006) [2023-12-27 12:10:51,483][20347] Updated weights for policy 0, policy_version 943070 (0.0006) [2023-12-27 12:10:51,501][20381] Updated weights for policy 1, policy_version 942192 (0.0007) [2023-12-27 12:10:52,224][20347] Updated weights for policy 0, policy_version 943080 (0.0007) [2023-12-27 12:10:52,252][20381] Updated weights for policy 1, policy_version 942202 (0.0008) [2023-12-27 12:10:52,277][20347] Updated weights for policy 0, policy_version 943090 (0.0007) [2023-12-27 12:10:52,315][20381] Updated weights for policy 1, policy_version 942212 (0.0008) [2023-12-27 12:10:52,337][20347] Updated weights for policy 0, policy_version 943100 (0.0007) [2023-12-27 12:10:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 482705408. Throughput: 0: 10305.7, 1: 10251.2. Samples: 482676920. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:52,363][19170] Avg episode reward: [(0, '990.430'), (1, '931.560')] [2023-12-27 12:10:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000943104_241475584.pth... [2023-12-27 12:10:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000941888_241164288.pth [2023-12-27 12:10:52,381][20381] Updated weights for policy 1, policy_version 942222 (0.0007) [2023-12-27 12:10:52,445][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000942232_241238016.pth... [2023-12-27 12:10:52,446][20381] Updated weights for policy 1, policy_version 942232 (0.0006) [2023-12-27 12:10:52,449][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000941016_240926720.pth [2023-12-27 12:10:53,067][20381] Updated weights for policy 1, policy_version 942242 (0.0006) [2023-12-27 12:10:53,068][20347] Updated weights for policy 0, policy_version 943110 (0.0008) [2023-12-27 12:10:53,118][20381] Updated weights for policy 1, policy_version 942252 (0.0006) [2023-12-27 12:10:53,131][20347] Updated weights for policy 0, policy_version 943120 (0.0007) [2023-12-27 12:10:53,173][20381] Updated weights for policy 1, policy_version 942262 (0.0007) [2023-12-27 12:10:53,191][20347] Updated weights for policy 0, policy_version 943130 (0.0007) [2023-12-27 12:10:53,818][20381] Updated weights for policy 1, policy_version 942272 (0.0006) [2023-12-27 12:10:53,874][20381] Updated weights for policy 1, policy_version 942282 (0.0006) [2023-12-27 12:10:53,905][20347] Updated weights for policy 0, policy_version 943140 (0.0007) [2023-12-27 12:10:53,927][20381] Updated weights for policy 1, policy_version 942292 (0.0007) [2023-12-27 12:10:53,965][20347] Updated weights for policy 0, policy_version 943150 (0.0006) [2023-12-27 12:10:54,020][20347] Updated weights for policy 0, policy_version 943160 (0.0007) [2023-12-27 12:10:54,583][20381] Updated weights for policy 1, policy_version 942302 (0.0008) [2023-12-27 12:10:54,637][20381] Updated weights for policy 1, policy_version 942312 (0.0007) [2023-12-27 12:10:54,683][20381] Updated weights for policy 1, policy_version 942322 (0.0007) [2023-12-27 12:10:54,695][20347] Updated weights for policy 0, policy_version 943170 (0.0008) [2023-12-27 12:10:54,749][20347] Updated weights for policy 0, policy_version 943180 (0.0008) [2023-12-27 12:10:54,794][20347] Updated weights for policy 0, policy_version 943190 (0.0008) [2023-12-27 12:10:54,850][20347] Updated weights for policy 0, policy_version 943200 (0.0008) [2023-12-27 12:10:55,360][20381] Updated weights for policy 1, policy_version 942332 (0.0008) [2023-12-27 12:10:55,416][20381] Updated weights for policy 1, policy_version 942342 (0.0008) [2023-12-27 12:10:55,473][20381] Updated weights for policy 1, policy_version 942352 (0.0008) [2023-12-27 12:10:55,516][20347] Updated weights for policy 0, policy_version 943210 (0.0006) [2023-12-27 12:10:55,561][20347] Updated weights for policy 0, policy_version 943220 (0.0008) [2023-12-27 12:10:55,607][20347] Updated weights for policy 0, policy_version 943230 (0.0008) [2023-12-27 12:10:56,151][20381] Updated weights for policy 1, policy_version 942362 (0.0006) [2023-12-27 12:10:56,214][20381] Updated weights for policy 1, policy_version 942372 (0.0005) [2023-12-27 12:10:56,271][20381] Updated weights for policy 1, policy_version 942382 (0.0008) [2023-12-27 12:10:56,330][20381] Updated weights for policy 1, policy_version 942392 (0.0011) [2023-12-27 12:10:56,362][20347] Updated weights for policy 0, policy_version 943240 (0.0007) [2023-12-27 12:10:56,411][20347] Updated weights for policy 0, policy_version 943250 (0.0008) [2023-12-27 12:10:56,465][20347] Updated weights for policy 0, policy_version 943260 (0.0007) [2023-12-27 12:10:56,956][20381] Updated weights for policy 1, policy_version 942402 (0.0005) [2023-12-27 12:10:57,001][20381] Updated weights for policy 1, policy_version 942412 (0.0005) [2023-12-27 12:10:57,052][20381] Updated weights for policy 1, policy_version 942422 (0.0005) [2023-12-27 12:10:57,198][20347] Updated weights for policy 0, policy_version 943270 (0.0008) [2023-12-27 12:10:57,245][20347] Updated weights for policy 0, policy_version 943280 (0.0006) [2023-12-27 12:10:57,309][20347] Updated weights for policy 0, policy_version 943290 (0.0006) [2023-12-27 12:10:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 482811904. Throughput: 0: 10253.1, 1: 10270.4. Samples: 482797608. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:10:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.680')] [2023-12-27 12:10:57,657][20381] Updated weights for policy 1, policy_version 942432 (0.0007) [2023-12-27 12:10:57,709][20381] Updated weights for policy 1, policy_version 942442 (0.0007) [2023-12-27 12:10:57,760][20381] Updated weights for policy 1, policy_version 942452 (0.0008) [2023-12-27 12:10:57,979][20347] Updated weights for policy 0, policy_version 943300 (0.0008) [2023-12-27 12:10:58,057][20347] Updated weights for policy 0, policy_version 943310 (0.0007) [2023-12-27 12:10:58,112][20347] Updated weights for policy 0, policy_version 943320 (0.0007) [2023-12-27 12:10:58,494][20381] Updated weights for policy 1, policy_version 942462 (0.0008) [2023-12-27 12:10:58,556][20381] Updated weights for policy 1, policy_version 942472 (0.0008) [2023-12-27 12:10:58,612][20381] Updated weights for policy 1, policy_version 942482 (0.0008) [2023-12-27 12:10:58,912][20347] Updated weights for policy 0, policy_version 943330 (0.0007) [2023-12-27 12:10:58,969][20347] Updated weights for policy 0, policy_version 943340 (0.0009) [2023-12-27 12:10:59,021][20347] Updated weights for policy 0, policy_version 943350 (0.0009) [2023-12-27 12:10:59,072][20347] Updated weights for policy 0, policy_version 943360 (0.0006) [2023-12-27 12:10:59,429][20381] Updated weights for policy 1, policy_version 942492 (0.0008) [2023-12-27 12:10:59,486][20381] Updated weights for policy 1, policy_version 942502 (0.0008) [2023-12-27 12:10:59,549][20381] Updated weights for policy 1, policy_version 942512 (0.0007) [2023-12-27 12:10:59,769][20347] Updated weights for policy 0, policy_version 943370 (0.0005) [2023-12-27 12:10:59,835][20347] Updated weights for policy 0, policy_version 943380 (0.0007) [2023-12-27 12:10:59,896][20347] Updated weights for policy 0, policy_version 943390 (0.0008) [2023-12-27 12:11:00,276][20381] Updated weights for policy 1, policy_version 942522 (0.0008) [2023-12-27 12:11:00,333][20381] Updated weights for policy 1, policy_version 942532 (0.0009) [2023-12-27 12:11:00,391][20381] Updated weights for policy 1, policy_version 942542 (0.0008) [2023-12-27 12:11:00,446][20381] Updated weights for policy 1, policy_version 942552 (0.0007) [2023-12-27 12:11:00,495][20347] Updated weights for policy 0, policy_version 943400 (0.0008) [2023-12-27 12:11:00,555][20347] Updated weights for policy 0, policy_version 943410 (0.0008) [2023-12-27 12:11:00,606][20347] Updated weights for policy 0, policy_version 943420 (0.0007) [2023-12-27 12:11:01,134][20381] Updated weights for policy 1, policy_version 942562 (0.0007) [2023-12-27 12:11:01,194][20381] Updated weights for policy 1, policy_version 942572 (0.0007) [2023-12-27 12:11:01,223][20347] Updated weights for policy 0, policy_version 943430 (0.0007) [2023-12-27 12:11:01,252][20381] Updated weights for policy 1, policy_version 942582 (0.0008) [2023-12-27 12:11:01,283][20347] Updated weights for policy 0, policy_version 943440 (0.0008) [2023-12-27 12:11:01,342][20291] KL-divergence is very high: 122.9790 [2023-12-27 12:11:01,343][20347] Updated weights for policy 0, policy_version 943450 (0.0010) [2023-12-27 12:11:01,993][20381] Updated weights for policy 1, policy_version 942592 (0.0006) [2023-12-27 12:11:02,054][20381] Updated weights for policy 1, policy_version 942602 (0.0006) [2023-12-27 12:11:02,074][20347] Updated weights for policy 0, policy_version 943460 (0.0009) [2023-12-27 12:11:02,114][20381] Updated weights for policy 1, policy_version 942612 (0.0006) [2023-12-27 12:11:02,133][20347] Updated weights for policy 0, policy_version 943470 (0.0010) [2023-12-27 12:11:02,191][20347] Updated weights for policy 0, policy_version 943480 (0.0010) [2023-12-27 12:11:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 482910208. Throughput: 0: 10179.4, 1: 10188.1. Samples: 482916700. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:02,363][19170] Avg episode reward: [(0, '990.260'), (1, '990.240')] [2023-12-27 12:11:02,713][20381] Updated weights for policy 1, policy_version 942622 (0.0007) [2023-12-27 12:11:02,780][20381] Updated weights for policy 1, policy_version 942632 (0.0007) [2023-12-27 12:11:02,832][20381] Updated weights for policy 1, policy_version 942642 (0.0006) [2023-12-27 12:11:02,873][20347] Updated weights for policy 0, policy_version 943490 (0.0010) [2023-12-27 12:11:02,928][20347] Updated weights for policy 0, policy_version 943500 (0.0010) [2023-12-27 12:11:02,986][20347] Updated weights for policy 0, policy_version 943510 (0.0010) [2023-12-27 12:11:03,037][20347] Updated weights for policy 0, policy_version 943520 (0.0010) [2023-12-27 12:11:03,489][20381] Updated weights for policy 1, policy_version 942652 (0.0006) [2023-12-27 12:11:03,547][20381] Updated weights for policy 1, policy_version 942662 (0.0007) [2023-12-27 12:11:03,596][20381] Updated weights for policy 1, policy_version 942672 (0.0007) [2023-12-27 12:11:03,696][20347] Updated weights for policy 0, policy_version 943530 (0.0010) [2023-12-27 12:11:03,754][20347] Updated weights for policy 0, policy_version 943540 (0.0010) [2023-12-27 12:11:03,802][20347] Updated weights for policy 0, policy_version 943550 (0.0010) [2023-12-27 12:11:04,295][20381] Updated weights for policy 1, policy_version 942682 (0.0006) [2023-12-27 12:11:04,358][20381] Updated weights for policy 1, policy_version 942692 (0.0008) [2023-12-27 12:11:04,415][20381] Updated weights for policy 1, policy_version 942702 (0.0008) [2023-12-27 12:11:04,465][20381] Updated weights for policy 1, policy_version 942712 (0.0008) [2023-12-27 12:11:04,536][20347] Updated weights for policy 0, policy_version 943560 (0.0010) [2023-12-27 12:11:04,591][20347] Updated weights for policy 0, policy_version 943570 (0.0010) [2023-12-27 12:11:04,649][20347] Updated weights for policy 0, policy_version 943580 (0.0010) [2023-12-27 12:11:05,183][20381] Updated weights for policy 1, policy_version 942722 (0.0007) [2023-12-27 12:11:05,239][20381] Updated weights for policy 1, policy_version 942732 (0.0007) [2023-12-27 12:11:05,298][20381] Updated weights for policy 1, policy_version 942742 (0.0007) [2023-12-27 12:11:05,313][20347] Updated weights for policy 0, policy_version 943590 (0.0010) [2023-12-27 12:11:05,346][20291] KL-divergence is very high: 480.3554 [2023-12-27 12:11:05,368][20347] Updated weights for policy 0, policy_version 943600 (0.0010) [2023-12-27 12:11:05,393][20291] KL-divergence is very high: 144.7679 [2023-12-27 12:11:05,432][20347] Updated weights for policy 0, policy_version 943610 (0.0009) [2023-12-27 12:11:05,948][20381] Updated weights for policy 1, policy_version 942752 (0.0008) [2023-12-27 12:11:05,999][20381] Updated weights for policy 1, policy_version 942762 (0.0007) [2023-12-27 12:11:06,047][20381] Updated weights for policy 1, policy_version 942772 (0.0009) [2023-12-27 12:11:06,076][20347] Updated weights for policy 0, policy_version 943620 (0.0007) [2023-12-27 12:11:06,135][20347] Updated weights for policy 0, policy_version 943630 (0.0007) [2023-12-27 12:11:06,194][20347] Updated weights for policy 0, policy_version 943640 (0.0008) [2023-12-27 12:11:06,801][20381] Updated weights for policy 1, policy_version 942782 (0.0008) [2023-12-27 12:11:06,837][20347] Updated weights for policy 0, policy_version 943650 (0.0008) [2023-12-27 12:11:06,862][20381] Updated weights for policy 1, policy_version 942792 (0.0007) [2023-12-27 12:11:06,894][20347] Updated weights for policy 0, policy_version 943660 (0.0006) [2023-12-27 12:11:06,917][20381] Updated weights for policy 1, policy_version 942802 (0.0009) [2023-12-27 12:11:06,947][20347] Updated weights for policy 0, policy_version 943670 (0.0007) [2023-12-27 12:11:06,994][20347] Updated weights for policy 0, policy_version 943680 (0.0009) [2023-12-27 12:11:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 483008512. Throughput: 0: 10184.9, 1: 10168.1. Samples: 482977752. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:07,364][19170] Avg episode reward: [(0, '980.500'), (1, '980.420')] [2023-12-27 12:11:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000942808_241385472.pth... [2023-12-27 12:11:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000943680_241623040.pth... [2023-12-27 12:11:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000941624_241082368.pth [2023-12-27 12:11:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000942496_241319936.pth [2023-12-27 12:11:07,596][20381] Updated weights for policy 1, policy_version 942812 (0.0007) [2023-12-27 12:11:07,613][20347] Updated weights for policy 0, policy_version 943690 (0.0006) [2023-12-27 12:11:07,651][20381] Updated weights for policy 1, policy_version 942822 (0.0007) [2023-12-27 12:11:07,655][20347] Updated weights for policy 0, policy_version 943700 (0.0005) [2023-12-27 12:11:07,701][20347] Updated weights for policy 0, policy_version 943710 (0.0005) [2023-12-27 12:11:07,707][20381] Updated weights for policy 1, policy_version 942832 (0.0008) [2023-12-27 12:11:08,318][20381] Updated weights for policy 1, policy_version 942842 (0.0006) [2023-12-27 12:11:08,381][20381] Updated weights for policy 1, policy_version 942852 (0.0008) [2023-12-27 12:11:08,397][20347] Updated weights for policy 0, policy_version 943720 (0.0010) [2023-12-27 12:11:08,441][20381] Updated weights for policy 1, policy_version 942862 (0.0006) [2023-12-27 12:11:08,454][20347] Updated weights for policy 0, policy_version 943730 (0.0010) [2023-12-27 12:11:08,502][20381] Updated weights for policy 1, policy_version 942872 (0.0007) [2023-12-27 12:11:08,510][20347] Updated weights for policy 0, policy_version 943740 (0.0006) [2023-12-27 12:11:09,151][20347] Updated weights for policy 0, policy_version 943750 (0.0006) [2023-12-27 12:11:09,180][20381] Updated weights for policy 1, policy_version 942882 (0.0006) [2023-12-27 12:11:09,210][20347] Updated weights for policy 0, policy_version 943760 (0.0006) [2023-12-27 12:11:09,246][20381] Updated weights for policy 1, policy_version 942892 (0.0008) [2023-12-27 12:11:09,277][20347] Updated weights for policy 0, policy_version 943770 (0.0010) [2023-12-27 12:11:09,302][20381] Updated weights for policy 1, policy_version 942902 (0.0007) [2023-12-27 12:11:09,976][20347] Updated weights for policy 0, policy_version 943780 (0.0011) [2023-12-27 12:11:10,012][20381] Updated weights for policy 1, policy_version 942912 (0.0007) [2023-12-27 12:11:10,044][20347] Updated weights for policy 0, policy_version 943790 (0.0011) [2023-12-27 12:11:10,072][20381] Updated weights for policy 1, policy_version 942922 (0.0006) [2023-12-27 12:11:10,100][20347] Updated weights for policy 0, policy_version 943800 (0.0011) [2023-12-27 12:11:10,125][20381] Updated weights for policy 1, policy_version 942932 (0.0005) [2023-12-27 12:11:10,764][20347] Updated weights for policy 0, policy_version 943810 (0.0010) [2023-12-27 12:11:10,811][20347] Updated weights for policy 0, policy_version 943820 (0.0007) [2023-12-27 12:11:10,842][20381] Updated weights for policy 1, policy_version 942942 (0.0008) [2023-12-27 12:11:10,866][20347] Updated weights for policy 0, policy_version 943830 (0.0005) [2023-12-27 12:11:10,895][20381] Updated weights for policy 1, policy_version 942952 (0.0006) [2023-12-27 12:11:10,919][20347] Updated weights for policy 0, policy_version 943840 (0.0006) [2023-12-27 12:11:10,953][20381] Updated weights for policy 1, policy_version 942962 (0.0007) [2023-12-27 12:11:11,734][20347] Updated weights for policy 0, policy_version 943850 (0.0006) [2023-12-27 12:11:11,749][20381] Updated weights for policy 1, policy_version 942972 (0.0009) [2023-12-27 12:11:11,795][20347] Updated weights for policy 0, policy_version 943860 (0.0007) [2023-12-27 12:11:11,810][20381] Updated weights for policy 1, policy_version 942982 (0.0008) [2023-12-27 12:11:11,857][20347] Updated weights for policy 0, policy_version 943870 (0.0008) [2023-12-27 12:11:11,878][20381] Updated weights for policy 1, policy_version 942992 (0.0009) [2023-12-27 12:11:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 483106816. Throughput: 0: 10189.9, 1: 10193.8. Samples: 483100272. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:12,363][19170] Avg episode reward: [(0, '980.580'), (1, '970.660')] [2023-12-27 12:11:12,528][20347] Updated weights for policy 0, policy_version 943880 (0.0007) [2023-12-27 12:11:12,575][20347] Updated weights for policy 0, policy_version 943890 (0.0009) [2023-12-27 12:11:12,627][20381] Updated weights for policy 1, policy_version 943002 (0.0008) [2023-12-27 12:11:12,643][20347] Updated weights for policy 0, policy_version 943900 (0.0007) [2023-12-27 12:11:12,686][20381] Updated weights for policy 1, policy_version 943012 (0.0006) [2023-12-27 12:11:12,748][20381] Updated weights for policy 1, policy_version 943022 (0.0008) [2023-12-27 12:11:12,813][20381] Updated weights for policy 1, policy_version 943032 (0.0008) [2023-12-27 12:11:13,299][20347] Updated weights for policy 0, policy_version 943910 (0.0006) [2023-12-27 12:11:13,310][20291] KL-divergence is very high: 108.4833 [2023-12-27 12:11:13,347][20347] Updated weights for policy 0, policy_version 943920 (0.0005) [2023-12-27 12:11:13,403][20347] Updated weights for policy 0, policy_version 943930 (0.0005) [2023-12-27 12:11:13,502][20381] Updated weights for policy 1, policy_version 943042 (0.0008) [2023-12-27 12:11:13,554][20381] Updated weights for policy 1, policy_version 943052 (0.0008) [2023-12-27 12:11:13,610][20381] Updated weights for policy 1, policy_version 943062 (0.0008) [2023-12-27 12:11:14,035][20347] Updated weights for policy 0, policy_version 943940 (0.0006) [2023-12-27 12:11:14,091][20347] Updated weights for policy 0, policy_version 943950 (0.0006) [2023-12-27 12:11:14,148][20347] Updated weights for policy 0, policy_version 943960 (0.0010) [2023-12-27 12:11:14,292][20381] Updated weights for policy 1, policy_version 943072 (0.0008) [2023-12-27 12:11:14,339][20381] Updated weights for policy 1, policy_version 943082 (0.0007) [2023-12-27 12:11:14,387][20381] Updated weights for policy 1, policy_version 943092 (0.0008) [2023-12-27 12:11:14,788][20347] Updated weights for policy 0, policy_version 943970 (0.0010) [2023-12-27 12:11:14,853][20347] Updated weights for policy 0, policy_version 943980 (0.0007) [2023-12-27 12:11:14,911][20347] Updated weights for policy 0, policy_version 943990 (0.0007) [2023-12-27 12:11:14,962][20347] Updated weights for policy 0, policy_version 944000 (0.0007) [2023-12-27 12:11:15,080][20381] Updated weights for policy 1, policy_version 943102 (0.0008) [2023-12-27 12:11:15,148][20381] Updated weights for policy 1, policy_version 943112 (0.0008) [2023-12-27 12:11:15,208][20381] Updated weights for policy 1, policy_version 943122 (0.0007) [2023-12-27 12:11:15,629][20347] Updated weights for policy 0, policy_version 944010 (0.0008) [2023-12-27 12:11:15,686][20347] Updated weights for policy 0, policy_version 944020 (0.0008) [2023-12-27 12:11:15,741][20347] Updated weights for policy 0, policy_version 944030 (0.0007) [2023-12-27 12:11:15,853][20381] Updated weights for policy 1, policy_version 943132 (0.0007) [2023-12-27 12:11:15,899][20381] Updated weights for policy 1, policy_version 943142 (0.0008) [2023-12-27 12:11:15,953][20381] Updated weights for policy 1, policy_version 943152 (0.0007) [2023-12-27 12:11:16,479][20347] Updated weights for policy 0, policy_version 944040 (0.0010) [2023-12-27 12:11:16,514][20381] Updated weights for policy 1, policy_version 943162 (0.0005) [2023-12-27 12:11:16,533][20347] Updated weights for policy 0, policy_version 944050 (0.0009) [2023-12-27 12:11:16,567][20381] Updated weights for policy 1, policy_version 943172 (0.0005) [2023-12-27 12:11:16,582][20347] Updated weights for policy 0, policy_version 944060 (0.0005) [2023-12-27 12:11:16,626][20381] Updated weights for policy 1, policy_version 943182 (0.0005) [2023-12-27 12:11:16,670][20381] Updated weights for policy 1, policy_version 943192 (0.0005) [2023-12-27 12:11:17,195][20381] Updated weights for policy 1, policy_version 943202 (0.0006) [2023-12-27 12:11:17,246][20347] Updated weights for policy 0, policy_version 944070 (0.0009) [2023-12-27 12:11:17,249][20381] Updated weights for policy 1, policy_version 943212 (0.0005) [2023-12-27 12:11:17,298][20381] Updated weights for policy 1, policy_version 943222 (0.0006) [2023-12-27 12:11:17,304][20347] Updated weights for policy 0, policy_version 944080 (0.0010) [2023-12-27 12:11:17,358][20347] Updated weights for policy 0, policy_version 944090 (0.0009) [2023-12-27 12:11:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 483213312. Throughput: 0: 10116.7, 1: 10204.8. Samples: 483224868. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:17,363][19170] Avg episode reward: [(0, '970.870'), (1, '980.460')] [2023-12-27 12:11:17,958][20381] Updated weights for policy 1, policy_version 943232 (0.0007) [2023-12-27 12:11:18,010][20381] Updated weights for policy 1, policy_version 943242 (0.0005) [2023-12-27 12:11:18,054][20347] Updated weights for policy 0, policy_version 944100 (0.0007) [2023-12-27 12:11:18,067][20381] Updated weights for policy 1, policy_version 943252 (0.0006) [2023-12-27 12:11:18,117][20347] Updated weights for policy 0, policy_version 944110 (0.0007) [2023-12-27 12:11:18,184][20347] Updated weights for policy 0, policy_version 944120 (0.0008) [2023-12-27 12:11:18,750][20381] Updated weights for policy 1, policy_version 943262 (0.0008) [2023-12-27 12:11:18,781][20292] KL-divergence is very high: 139.0178 [2023-12-27 12:11:18,799][20292] KL-divergence is very high: 125.9219 [2023-12-27 12:11:18,809][20381] Updated weights for policy 1, policy_version 943272 (0.0007) [2023-12-27 12:11:18,827][20347] Updated weights for policy 0, policy_version 944130 (0.0007) [2023-12-27 12:11:18,859][20381] Updated weights for policy 1, policy_version 943282 (0.0006) [2023-12-27 12:11:18,883][20347] Updated weights for policy 0, policy_version 944140 (0.0008) [2023-12-27 12:11:18,944][20347] Updated weights for policy 0, policy_version 944150 (0.0007) [2023-12-27 12:11:18,991][20347] Updated weights for policy 0, policy_version 944160 (0.0008) [2023-12-27 12:11:19,602][20381] Updated weights for policy 1, policy_version 943292 (0.0008) [2023-12-27 12:11:19,664][20381] Updated weights for policy 1, policy_version 943302 (0.0011) [2023-12-27 12:11:19,719][20347] Updated weights for policy 0, policy_version 944170 (0.0006) [2023-12-27 12:11:19,724][20381] Updated weights for policy 1, policy_version 943312 (0.0009) [2023-12-27 12:11:19,786][20347] Updated weights for policy 0, policy_version 944180 (0.0006) [2023-12-27 12:11:19,855][20347] Updated weights for policy 0, policy_version 944190 (0.0007) [2023-12-27 12:11:20,475][20381] Updated weights for policy 1, policy_version 943322 (0.0009) [2023-12-27 12:11:20,507][20347] Updated weights for policy 0, policy_version 944200 (0.0006) [2023-12-27 12:11:20,528][20381] Updated weights for policy 1, policy_version 943332 (0.0006) [2023-12-27 12:11:20,568][20347] Updated weights for policy 0, policy_version 944210 (0.0007) [2023-12-27 12:11:20,592][20381] Updated weights for policy 1, policy_version 943342 (0.0007) [2023-12-27 12:11:20,633][20347] Updated weights for policy 0, policy_version 944220 (0.0007) [2023-12-27 12:11:20,652][20381] Updated weights for policy 1, policy_version 943352 (0.0006) [2023-12-27 12:11:21,317][20381] Updated weights for policy 1, policy_version 943362 (0.0008) [2023-12-27 12:11:21,340][20347] Updated weights for policy 0, policy_version 944230 (0.0008) [2023-12-27 12:11:21,388][20381] Updated weights for policy 1, policy_version 943372 (0.0007) [2023-12-27 12:11:21,414][20347] Updated weights for policy 0, policy_version 944240 (0.0008) [2023-12-27 12:11:21,445][20381] Updated weights for policy 1, policy_version 943382 (0.0007) [2023-12-27 12:11:21,477][20347] Updated weights for policy 0, policy_version 944250 (0.0006) [2023-12-27 12:11:22,131][20381] Updated weights for policy 1, policy_version 943392 (0.0008) [2023-12-27 12:11:22,172][20347] Updated weights for policy 0, policy_version 944260 (0.0006) [2023-12-27 12:11:22,185][20381] Updated weights for policy 1, policy_version 943402 (0.0007) [2023-12-27 12:11:22,230][20347] Updated weights for policy 0, policy_version 944270 (0.0006) [2023-12-27 12:11:22,239][20381] Updated weights for policy 1, policy_version 943412 (0.0007) [2023-12-27 12:11:22,288][20347] Updated weights for policy 0, policy_version 944280 (0.0007) [2023-12-27 12:11:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 483319808. Throughput: 0: 10100.3, 1: 10210.8. Samples: 483284384. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:22,363][19170] Avg episode reward: [(0, '970.770'), (1, '951.220')] [2023-12-27 12:11:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000943416_241541120.pth... [2023-12-27 12:11:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000944288_241778688.pth... [2023-12-27 12:11:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000942232_241238016.pth [2023-12-27 12:11:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000943104_241475584.pth [2023-12-27 12:11:22,954][20347] Updated weights for policy 0, policy_version 944290 (0.0008) [2023-12-27 12:11:23,007][20381] Updated weights for policy 1, policy_version 943422 (0.0007) [2023-12-27 12:11:23,012][20347] Updated weights for policy 0, policy_version 944300 (0.0006) [2023-12-27 12:11:23,066][20381] Updated weights for policy 1, policy_version 943432 (0.0006) [2023-12-27 12:11:23,075][20347] Updated weights for policy 0, policy_version 944310 (0.0006) [2023-12-27 12:11:23,136][20381] Updated weights for policy 1, policy_version 943442 (0.0008) [2023-12-27 12:11:23,138][20347] Updated weights for policy 0, policy_version 944320 (0.0007) [2023-12-27 12:11:23,842][20347] Updated weights for policy 0, policy_version 944330 (0.0009) [2023-12-27 12:11:23,870][20381] Updated weights for policy 1, policy_version 943452 (0.0008) [2023-12-27 12:11:23,899][20347] Updated weights for policy 0, policy_version 944340 (0.0011) [2023-12-27 12:11:23,927][20381] Updated weights for policy 1, policy_version 943462 (0.0007) [2023-12-27 12:11:23,948][20347] Updated weights for policy 0, policy_version 944350 (0.0010) [2023-12-27 12:11:23,979][20381] Updated weights for policy 1, policy_version 943472 (0.0007) [2023-12-27 12:11:24,554][20347] Updated weights for policy 0, policy_version 944360 (0.0010) [2023-12-27 12:11:24,611][20347] Updated weights for policy 0, policy_version 944370 (0.0007) [2023-12-27 12:11:24,660][20347] Updated weights for policy 0, policy_version 944380 (0.0009) [2023-12-27 12:11:24,670][20381] Updated weights for policy 1, policy_version 943482 (0.0007) [2023-12-27 12:11:24,717][20381] Updated weights for policy 1, policy_version 943492 (0.0007) [2023-12-27 12:11:24,767][20381] Updated weights for policy 1, policy_version 943502 (0.0007) [2023-12-27 12:11:24,824][20381] Updated weights for policy 1, policy_version 943512 (0.0007) [2023-12-27 12:11:25,299][20347] Updated weights for policy 0, policy_version 944390 (0.0007) [2023-12-27 12:11:25,351][20347] Updated weights for policy 0, policy_version 944400 (0.0005) [2023-12-27 12:11:25,405][20347] Updated weights for policy 0, policy_version 944410 (0.0007) [2023-12-27 12:11:25,551][20381] Updated weights for policy 1, policy_version 943522 (0.0007) [2023-12-27 12:11:25,601][20381] Updated weights for policy 1, policy_version 943532 (0.0007) [2023-12-27 12:11:25,655][20381] Updated weights for policy 1, policy_version 943542 (0.0007) [2023-12-27 12:11:25,989][20347] Updated weights for policy 0, policy_version 944420 (0.0008) [2023-12-27 12:11:26,045][20347] Updated weights for policy 0, policy_version 944430 (0.0005) [2023-12-27 12:11:26,101][20347] Updated weights for policy 0, policy_version 944440 (0.0008) [2023-12-27 12:11:26,380][20381] Updated weights for policy 1, policy_version 943552 (0.0008) [2023-12-27 12:11:26,430][20381] Updated weights for policy 1, policy_version 943562 (0.0007) [2023-12-27 12:11:26,477][20381] Updated weights for policy 1, policy_version 943572 (0.0008) [2023-12-27 12:11:26,688][20347] Updated weights for policy 0, policy_version 944450 (0.0009) [2023-12-27 12:11:26,743][20347] Updated weights for policy 0, policy_version 944460 (0.0005) [2023-12-27 12:11:26,790][20347] Updated weights for policy 0, policy_version 944470 (0.0005) [2023-12-27 12:11:26,845][20347] Updated weights for policy 0, policy_version 944480 (0.0008) [2023-12-27 12:11:27,211][20381] Updated weights for policy 1, policy_version 943582 (0.0007) [2023-12-27 12:11:27,265][20381] Updated weights for policy 1, policy_version 943592 (0.0007) [2023-12-27 12:11:27,320][20381] Updated weights for policy 1, policy_version 943602 (0.0008) [2023-12-27 12:11:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 483418112. Throughput: 0: 10127.6, 1: 10123.7. Samples: 483405136. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:27,363][19170] Avg episode reward: [(0, '980.670'), (1, '961.000')] [2023-12-27 12:11:27,452][20347] Updated weights for policy 0, policy_version 944490 (0.0007) [2023-12-27 12:11:27,499][20347] Updated weights for policy 0, policy_version 944500 (0.0007) [2023-12-27 12:11:27,552][20347] Updated weights for policy 0, policy_version 944510 (0.0008) [2023-12-27 12:11:27,953][20381] Updated weights for policy 1, policy_version 943612 (0.0007) [2023-12-27 12:11:28,006][20381] Updated weights for policy 1, policy_version 943622 (0.0007) [2023-12-27 12:11:28,057][20381] Updated weights for policy 1, policy_version 943632 (0.0007) [2023-12-27 12:11:28,236][20347] Updated weights for policy 0, policy_version 944520 (0.0007) [2023-12-27 12:11:28,288][20347] Updated weights for policy 0, policy_version 944530 (0.0008) [2023-12-27 12:11:28,340][20347] Updated weights for policy 0, policy_version 944540 (0.0008) [2023-12-27 12:11:28,800][20381] Updated weights for policy 1, policy_version 943642 (0.0007) [2023-12-27 12:11:28,858][20381] Updated weights for policy 1, policy_version 943652 (0.0007) [2023-12-27 12:11:28,916][20381] Updated weights for policy 1, policy_version 943662 (0.0006) [2023-12-27 12:11:28,974][20381] Updated weights for policy 1, policy_version 943672 (0.0005) [2023-12-27 12:11:29,027][20347] Updated weights for policy 0, policy_version 944550 (0.0008) [2023-12-27 12:11:29,080][20347] Updated weights for policy 0, policy_version 944560 (0.0007) [2023-12-27 12:11:29,142][20347] Updated weights for policy 0, policy_version 944570 (0.0007) [2023-12-27 12:11:29,652][20381] Updated weights for policy 1, policy_version 943682 (0.0008) [2023-12-27 12:11:29,714][20381] Updated weights for policy 1, policy_version 943692 (0.0006) [2023-12-27 12:11:29,778][20381] Updated weights for policy 1, policy_version 943702 (0.0007) [2023-12-27 12:11:29,824][20347] Updated weights for policy 0, policy_version 944580 (0.0007) [2023-12-27 12:11:29,885][20347] Updated weights for policy 0, policy_version 944590 (0.0007) [2023-12-27 12:11:29,947][20347] Updated weights for policy 0, policy_version 944600 (0.0007) [2023-12-27 12:11:30,473][20381] Updated weights for policy 1, policy_version 943712 (0.0005) [2023-12-27 12:11:30,532][20381] Updated weights for policy 1, policy_version 943722 (0.0006) [2023-12-27 12:11:30,587][20381] Updated weights for policy 1, policy_version 943732 (0.0008) [2023-12-27 12:11:30,618][20347] Updated weights for policy 0, policy_version 944610 (0.0007) [2023-12-27 12:11:30,670][20347] Updated weights for policy 0, policy_version 944620 (0.0008) [2023-12-27 12:11:30,720][20347] Updated weights for policy 0, policy_version 944630 (0.0006) [2023-12-27 12:11:30,769][20347] Updated weights for policy 0, policy_version 944640 (0.0005) [2023-12-27 12:11:31,270][20381] Updated weights for policy 1, policy_version 943742 (0.0008) [2023-12-27 12:11:31,326][20381] Updated weights for policy 1, policy_version 943752 (0.0008) [2023-12-27 12:11:31,385][20381] Updated weights for policy 1, policy_version 943762 (0.0008) [2023-12-27 12:11:31,470][20347] Updated weights for policy 0, policy_version 944650 (0.0005) [2023-12-27 12:11:31,528][20347] Updated weights for policy 0, policy_version 944660 (0.0007) [2023-12-27 12:11:31,593][20347] Updated weights for policy 0, policy_version 944670 (0.0010) [2023-12-27 12:11:32,047][20381] Updated weights for policy 1, policy_version 943772 (0.0008) [2023-12-27 12:11:32,097][20381] Updated weights for policy 1, policy_version 943782 (0.0008) [2023-12-27 12:11:32,149][20381] Updated weights for policy 1, policy_version 943792 (0.0006) [2023-12-27 12:11:32,230][20347] Updated weights for policy 0, policy_version 944680 (0.0006) [2023-12-27 12:11:32,286][20347] Updated weights for policy 0, policy_version 944690 (0.0007) [2023-12-27 12:11:32,335][20347] Updated weights for policy 0, policy_version 944700 (0.0006) [2023-12-27 12:11:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 483516416. Throughput: 0: 10201.2, 1: 10106.7. Samples: 483529096. Policy #0 lag: (min: 31.0, avg: 38.1, max: 63.0) [2023-12-27 12:11:32,363][19170] Avg episode reward: [(0, '980.480'), (1, '990.240')] [2023-12-27 12:11:32,849][20381] Updated weights for policy 1, policy_version 943802 (0.0008) [2023-12-27 12:11:32,909][20381] Updated weights for policy 1, policy_version 943812 (0.0007) [2023-12-27 12:11:32,968][20381] Updated weights for policy 1, policy_version 943822 (0.0007) [2023-12-27 12:11:33,023][20381] Updated weights for policy 1, policy_version 943832 (0.0006) [2023-12-27 12:11:33,033][20347] Updated weights for policy 0, policy_version 944710 (0.0008) [2023-12-27 12:11:33,093][20347] Updated weights for policy 0, policy_version 944720 (0.0005) [2023-12-27 12:11:33,149][20347] Updated weights for policy 0, policy_version 944730 (0.0006) [2023-12-27 12:11:33,685][20347] Updated weights for policy 0, policy_version 944740 (0.0006) [2023-12-27 12:11:33,696][20381] Updated weights for policy 1, policy_version 943842 (0.0009) [2023-12-27 12:11:33,741][20347] Updated weights for policy 0, policy_version 944750 (0.0006) [2023-12-27 12:11:33,746][20381] Updated weights for policy 1, policy_version 943852 (0.0010) [2023-12-27 12:11:33,794][20347] Updated weights for policy 0, policy_version 944760 (0.0006) [2023-12-27 12:11:33,799][20381] Updated weights for policy 1, policy_version 943862 (0.0009) [2023-12-27 12:11:34,353][20347] Updated weights for policy 0, policy_version 944770 (0.0006) [2023-12-27 12:11:34,419][20347] Updated weights for policy 0, policy_version 944780 (0.0008) [2023-12-27 12:11:34,466][20381] Updated weights for policy 1, policy_version 943872 (0.0006) [2023-12-27 12:11:34,478][20347] Updated weights for policy 0, policy_version 944790 (0.0007) [2023-12-27 12:11:34,526][20381] Updated weights for policy 1, policy_version 943882 (0.0007) [2023-12-27 12:11:34,543][20347] Updated weights for policy 0, policy_version 944800 (0.0006) [2023-12-27 12:11:34,585][20381] Updated weights for policy 1, policy_version 943892 (0.0007) [2023-12-27 12:11:35,198][20347] Updated weights for policy 0, policy_version 944810 (0.0005) [2023-12-27 12:11:35,248][20381] Updated weights for policy 1, policy_version 943902 (0.0007) [2023-12-27 12:11:35,255][20347] Updated weights for policy 0, policy_version 944820 (0.0006) [2023-12-27 12:11:35,304][20381] Updated weights for policy 1, policy_version 943912 (0.0005) [2023-12-27 12:11:35,306][20347] Updated weights for policy 0, policy_version 944830 (0.0006) [2023-12-27 12:11:35,353][20381] Updated weights for policy 1, policy_version 943922 (0.0005) [2023-12-27 12:11:35,994][20381] Updated weights for policy 1, policy_version 943932 (0.0007) [2023-12-27 12:11:35,996][20347] Updated weights for policy 0, policy_version 944840 (0.0007) [2023-12-27 12:11:36,051][20347] Updated weights for policy 0, policy_version 944850 (0.0006) [2023-12-27 12:11:36,051][20381] Updated weights for policy 1, policy_version 943942 (0.0006) [2023-12-27 12:11:36,094][20381] Updated weights for policy 1, policy_version 943952 (0.0006) [2023-12-27 12:11:36,108][20347] Updated weights for policy 0, policy_version 944860 (0.0007) [2023-12-27 12:11:36,692][20381] Updated weights for policy 1, policy_version 943962 (0.0008) [2023-12-27 12:11:36,753][20381] Updated weights for policy 1, policy_version 943972 (0.0009) [2023-12-27 12:11:36,808][20347] Updated weights for policy 0, policy_version 944870 (0.0007) [2023-12-27 12:11:36,811][20381] Updated weights for policy 1, policy_version 943982 (0.0008) [2023-12-27 12:11:36,859][20347] Updated weights for policy 0, policy_version 944880 (0.0005) [2023-12-27 12:11:36,874][20381] Updated weights for policy 1, policy_version 943992 (0.0007) [2023-12-27 12:11:36,906][20347] Updated weights for policy 0, policy_version 944890 (0.0007) [2023-12-27 12:11:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 483622912. Throughput: 0: 10254.7, 1: 10124.7. Samples: 483593992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:11:37,363][19170] Avg episode reward: [(0, '961.080'), (1, '990.240')] [2023-12-27 12:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000943992_241688576.pth... [2023-12-27 12:11:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000944896_241934336.pth... [2023-12-27 12:11:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000943680_241623040.pth [2023-12-27 12:11:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000942808_241385472.pth [2023-12-27 12:11:37,548][20381] Updated weights for policy 1, policy_version 944002 (0.0008) [2023-12-27 12:11:37,589][20347] Updated weights for policy 0, policy_version 944900 (0.0007) [2023-12-27 12:11:37,606][20381] Updated weights for policy 1, policy_version 944012 (0.0008) [2023-12-27 12:11:37,645][20347] Updated weights for policy 0, policy_version 944910 (0.0006) [2023-12-27 12:11:37,666][20381] Updated weights for policy 1, policy_version 944022 (0.0009) [2023-12-27 12:11:37,701][20347] Updated weights for policy 0, policy_version 944920 (0.0006) [2023-12-27 12:11:38,306][20381] Updated weights for policy 1, policy_version 944032 (0.0007) [2023-12-27 12:11:38,374][20381] Updated weights for policy 1, policy_version 944042 (0.0007) [2023-12-27 12:11:38,442][20381] Updated weights for policy 1, policy_version 944052 (0.0007) [2023-12-27 12:11:38,451][20347] Updated weights for policy 0, policy_version 944930 (0.0008) [2023-12-27 12:11:38,517][20347] Updated weights for policy 0, policy_version 944940 (0.0006) [2023-12-27 12:11:38,581][20347] Updated weights for policy 0, policy_version 944950 (0.0006) [2023-12-27 12:11:38,640][20347] Updated weights for policy 0, policy_version 944960 (0.0007) [2023-12-27 12:11:39,043][20381] Updated weights for policy 1, policy_version 944062 (0.0007) [2023-12-27 12:11:39,099][20381] Updated weights for policy 1, policy_version 944072 (0.0008) [2023-12-27 12:11:39,144][20381] Updated weights for policy 1, policy_version 944082 (0.0008) [2023-12-27 12:11:39,309][20347] Updated weights for policy 0, policy_version 944970 (0.0008) [2023-12-27 12:11:39,372][20347] Updated weights for policy 0, policy_version 944980 (0.0008) [2023-12-27 12:11:39,443][20347] Updated weights for policy 0, policy_version 944990 (0.0008) [2023-12-27 12:11:39,900][20381] Updated weights for policy 1, policy_version 944092 (0.0008) [2023-12-27 12:11:39,967][20381] Updated weights for policy 1, policy_version 944102 (0.0008) [2023-12-27 12:11:40,033][20381] Updated weights for policy 1, policy_version 944112 (0.0008) [2023-12-27 12:11:40,180][20347] Updated weights for policy 0, policy_version 945000 (0.0006) [2023-12-27 12:11:40,246][20347] Updated weights for policy 0, policy_version 945010 (0.0006) [2023-12-27 12:11:40,314][20347] Updated weights for policy 0, policy_version 945020 (0.0008) [2023-12-27 12:11:40,721][20381] Updated weights for policy 1, policy_version 944122 (0.0008) [2023-12-27 12:11:40,783][20381] Updated weights for policy 1, policy_version 944132 (0.0007) [2023-12-27 12:11:40,847][20381] Updated weights for policy 1, policy_version 944142 (0.0009) [2023-12-27 12:11:40,898][20381] Updated weights for policy 1, policy_version 944152 (0.0008) [2023-12-27 12:11:40,922][20347] Updated weights for policy 0, policy_version 945030 (0.0009) [2023-12-27 12:11:40,976][20347] Updated weights for policy 0, policy_version 945040 (0.0010) [2023-12-27 12:11:41,034][20347] Updated weights for policy 0, policy_version 945050 (0.0010) [2023-12-27 12:11:41,687][20381] Updated weights for policy 1, policy_version 944162 (0.0007) [2023-12-27 12:11:41,752][20381] Updated weights for policy 1, policy_version 944172 (0.0008) [2023-12-27 12:11:41,770][20347] Updated weights for policy 0, policy_version 945060 (0.0010) [2023-12-27 12:11:41,808][20381] Updated weights for policy 1, policy_version 944182 (0.0007) [2023-12-27 12:11:41,828][20347] Updated weights for policy 0, policy_version 945070 (0.0007) [2023-12-27 12:11:41,876][20347] Updated weights for policy 0, policy_version 945080 (0.0007) [2023-12-27 12:11:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 483721216. Throughput: 0: 10282.2, 1: 10124.1. Samples: 483715892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:11:42,363][19170] Avg episode reward: [(0, '941.730'), (1, '990.210')] [2023-12-27 12:11:42,499][20381] Updated weights for policy 1, policy_version 944192 (0.0007) [2023-12-27 12:11:42,559][20381] Updated weights for policy 1, policy_version 944202 (0.0011) [2023-12-27 12:11:42,587][20347] Updated weights for policy 0, policy_version 945090 (0.0006) [2023-12-27 12:11:42,616][20381] Updated weights for policy 1, policy_version 944212 (0.0008) [2023-12-27 12:11:42,645][20347] Updated weights for policy 0, policy_version 945100 (0.0008) [2023-12-27 12:11:42,694][20347] Updated weights for policy 0, policy_version 945110 (0.0008) [2023-12-27 12:11:42,746][20347] Updated weights for policy 0, policy_version 945120 (0.0008) [2023-12-27 12:11:43,261][20381] Updated weights for policy 1, policy_version 944222 (0.0007) [2023-12-27 12:11:43,315][20381] Updated weights for policy 1, policy_version 944232 (0.0010) [2023-12-27 12:11:43,366][20381] Updated weights for policy 1, policy_version 944242 (0.0010) [2023-12-27 12:11:43,448][20347] Updated weights for policy 0, policy_version 945130 (0.0007) [2023-12-27 12:11:43,498][20347] Updated weights for policy 0, policy_version 945140 (0.0007) [2023-12-27 12:11:43,557][20347] Updated weights for policy 0, policy_version 945150 (0.0006) [2023-12-27 12:11:44,040][20381] Updated weights for policy 1, policy_version 944252 (0.0008) [2023-12-27 12:11:44,097][20381] Updated weights for policy 1, policy_version 944262 (0.0006) [2023-12-27 12:11:44,108][20347] Updated weights for policy 0, policy_version 945160 (0.0010) [2023-12-27 12:11:44,143][20381] Updated weights for policy 1, policy_version 944272 (0.0010) [2023-12-27 12:11:44,163][20347] Updated weights for policy 0, policy_version 945170 (0.0010) [2023-12-27 12:11:44,218][20347] Updated weights for policy 0, policy_version 945180 (0.0010) [2023-12-27 12:11:44,702][20381] Updated weights for policy 1, policy_version 944282 (0.0009) [2023-12-27 12:11:44,757][20381] Updated weights for policy 1, policy_version 944292 (0.0007) [2023-12-27 12:11:44,817][20381] Updated weights for policy 1, policy_version 944302 (0.0011) [2023-12-27 12:11:44,846][20347] Updated weights for policy 0, policy_version 945190 (0.0009) [2023-12-27 12:11:44,872][20381] Updated weights for policy 1, policy_version 944312 (0.0011) [2023-12-27 12:11:44,900][20347] Updated weights for policy 0, policy_version 945200 (0.0006) [2023-12-27 12:11:44,966][20347] Updated weights for policy 0, policy_version 945210 (0.0007) [2023-12-27 12:11:45,586][20381] Updated weights for policy 1, policy_version 944322 (0.0011) [2023-12-27 12:11:45,599][20347] Updated weights for policy 0, policy_version 945220 (0.0007) [2023-12-27 12:11:45,647][20381] Updated weights for policy 1, policy_version 944332 (0.0011) [2023-12-27 12:11:45,659][20347] Updated weights for policy 0, policy_version 945230 (0.0011) [2023-12-27 12:11:45,713][20381] Updated weights for policy 1, policy_version 944342 (0.0011) [2023-12-27 12:11:45,715][20347] Updated weights for policy 0, policy_version 945240 (0.0011) [2023-12-27 12:11:46,370][20347] Updated weights for policy 0, policy_version 945250 (0.0010) [2023-12-27 12:11:46,388][20381] Updated weights for policy 1, policy_version 944352 (0.0008) [2023-12-27 12:11:46,424][20347] Updated weights for policy 0, policy_version 945260 (0.0010) [2023-12-27 12:11:46,430][20381] Updated weights for policy 1, policy_version 944362 (0.0005) [2023-12-27 12:11:46,476][20381] Updated weights for policy 1, policy_version 944372 (0.0005) [2023-12-27 12:11:46,480][20347] Updated weights for policy 0, policy_version 945270 (0.0009) [2023-12-27 12:11:46,535][20347] Updated weights for policy 0, policy_version 945280 (0.0010) [2023-12-27 12:11:47,088][20381] Updated weights for policy 1, policy_version 944382 (0.0007) [2023-12-27 12:11:47,147][20381] Updated weights for policy 1, policy_version 944392 (0.0009) [2023-12-27 12:11:47,178][20347] Updated weights for policy 0, policy_version 945290 (0.0005) [2023-12-27 12:11:47,206][20381] Updated weights for policy 1, policy_version 944402 (0.0007) [2023-12-27 12:11:47,237][20347] Updated weights for policy 0, policy_version 945300 (0.0005) [2023-12-27 12:11:47,286][20347] Updated weights for policy 0, policy_version 945310 (0.0008) [2023-12-27 12:11:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 483835904. Throughput: 0: 10351.2, 1: 10196.8. Samples: 483841368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:11:47,364][19170] Avg episode reward: [(0, '912.510'), (1, '951.100')] [2023-12-27 12:11:47,885][20347] Updated weights for policy 0, policy_version 945320 (0.0010) [2023-12-27 12:11:47,915][20381] Updated weights for policy 1, policy_version 944412 (0.0007) [2023-12-27 12:11:47,943][20347] Updated weights for policy 0, policy_version 945330 (0.0010) [2023-12-27 12:11:47,946][20292] KL-divergence is very high: 1627.0781 [2023-12-27 12:11:47,970][20381] Updated weights for policy 1, policy_version 944422 (0.0005) [2023-12-27 12:11:47,996][20292] KL-divergence is very high: 101.5590 [2023-12-27 12:11:48,002][20347] Updated weights for policy 0, policy_version 945340 (0.0011) [2023-12-27 12:11:48,030][20381] Updated weights for policy 1, policy_version 944432 (0.0006) [2023-12-27 12:11:48,592][20347] Updated weights for policy 0, policy_version 945350 (0.0010) [2023-12-27 12:11:48,650][20347] Updated weights for policy 0, policy_version 945360 (0.0010) [2023-12-27 12:11:48,710][20347] Updated weights for policy 0, policy_version 945370 (0.0011) [2023-12-27 12:11:48,757][20381] Updated weights for policy 1, policy_version 944442 (0.0008) [2023-12-27 12:11:48,758][20292] KL-divergence is very high: 194.9693 [2023-12-27 12:11:48,764][20292] KL-divergence is very high: 157.1132 [2023-12-27 12:11:48,777][20292] KL-divergence is very high: 2137.9580 [2023-12-27 12:11:48,783][20292] KL-divergence is very high: 302.9228 [2023-12-27 12:11:48,815][20292] KL-divergence is very high: 1928.1469 [2023-12-27 12:11:48,823][20381] Updated weights for policy 1, policy_version 944452 (0.0008) [2023-12-27 12:11:48,829][20292] KL-divergence is very high: 1188.9185 [2023-12-27 12:11:48,836][20292] KL-divergence is very high: 914.1833 [2023-12-27 12:11:48,861][20292] KL-divergence is very high: 120.3607 [2023-12-27 12:11:48,868][20292] KL-divergence is very high: 290.3579 [2023-12-27 12:11:48,882][20292] KL-divergence is very high: 2677.4871 [2023-12-27 12:11:48,888][20292] KL-divergence is very high: 406.5489 [2023-12-27 12:11:48,889][20381] Updated weights for policy 1, policy_version 944462 (0.0008) [2023-12-27 12:11:48,920][20292] KL-divergence is very high: 334.5072 [2023-12-27 12:11:48,933][20292] KL-divergence is very high: 433.6345 [2023-12-27 12:11:48,939][20292] KL-divergence is very high: 437.6755 [2023-12-27 12:11:48,948][20381] Updated weights for policy 1, policy_version 944472 (0.0007) [2023-12-27 12:11:49,343][20347] Updated weights for policy 0, policy_version 945380 (0.0010) [2023-12-27 12:11:49,412][20347] Updated weights for policy 0, policy_version 945390 (0.0009) [2023-12-27 12:11:49,472][20347] Updated weights for policy 0, policy_version 945400 (0.0008) [2023-12-27 12:11:49,597][20292] KL-divergence is very high: 370.1445 [2023-12-27 12:11:49,603][20292] KL-divergence is very high: 377.7758 [2023-12-27 12:11:49,644][20292] KL-divergence is very high: 107.0090 [2023-12-27 12:11:49,650][20292] KL-divergence is very high: 194.3629 [2023-12-27 12:11:49,656][20381] Updated weights for policy 1, policy_version 944482 (0.0008) [2023-12-27 12:11:49,714][20381] Updated weights for policy 1, policy_version 944492 (0.0008) [2023-12-27 12:11:49,777][20381] Updated weights for policy 1, policy_version 944502 (0.0008) [2023-12-27 12:11:50,184][20347] Updated weights for policy 0, policy_version 945410 (0.0009) [2023-12-27 12:11:50,247][20347] Updated weights for policy 0, policy_version 945420 (0.0007) [2023-12-27 12:11:50,303][20347] Updated weights for policy 0, policy_version 945430 (0.0008) [2023-12-27 12:11:50,365][20347] Updated weights for policy 0, policy_version 945440 (0.0007) [2023-12-27 12:11:50,500][20292] KL-divergence is very high: 119.7722 [2023-12-27 12:11:50,506][20381] Updated weights for policy 1, policy_version 944512 (0.0007) [2023-12-27 12:11:50,518][20292] KL-divergence is very high: 147.8014 [2023-12-27 12:11:50,540][20292] KL-divergence is very high: 103.3188 [2023-12-27 12:11:50,559][20381] Updated weights for policy 1, policy_version 944522 (0.0008) [2023-12-27 12:11:50,577][20292] KL-divergence is very high: 105.9445 [2023-12-27 12:11:50,584][20292] KL-divergence is very high: 114.0323 [2023-12-27 12:11:50,613][20381] Updated weights for policy 1, policy_version 944532 (0.0007) [2023-12-27 12:11:51,009][20347] Updated weights for policy 0, policy_version 945450 (0.0007) [2023-12-27 12:11:51,072][20347] Updated weights for policy 0, policy_version 945460 (0.0008) [2023-12-27 12:11:51,139][20347] Updated weights for policy 0, policy_version 945470 (0.0008) [2023-12-27 12:11:51,348][20381] Updated weights for policy 1, policy_version 944542 (0.0008) [2023-12-27 12:11:51,418][20381] Updated weights for policy 1, policy_version 944552 (0.0009) [2023-12-27 12:11:51,481][20381] Updated weights for policy 1, policy_version 944562 (0.0007) [2023-12-27 12:11:51,910][20347] Updated weights for policy 0, policy_version 945480 (0.0008) [2023-12-27 12:11:51,962][20347] Updated weights for policy 0, policy_version 945490 (0.0008) [2023-12-27 12:11:52,028][20347] Updated weights for policy 0, policy_version 945500 (0.0008) [2023-12-27 12:11:52,171][20381] Updated weights for policy 1, policy_version 944572 (0.0007) [2023-12-27 12:11:52,226][20381] Updated weights for policy 1, policy_version 944582 (0.0007) [2023-12-27 12:11:52,271][20292] KL-divergence is very high: 221.7421 [2023-12-27 12:11:52,282][20381] Updated weights for policy 1, policy_version 944592 (0.0008) [2023-12-27 12:11:52,315][20292] KL-divergence is very high: 204.6273 [2023-12-27 12:11:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 483934208. Throughput: 0: 10393.1, 1: 10153.3. Samples: 483902336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:11:52,363][19170] Avg episode reward: [(0, '970.860'), (1, '765.050')] [2023-12-27 12:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000944600_241844224.pth... [2023-12-27 12:11:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000945504_242089984.pth... [2023-12-27 12:11:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000943416_241541120.pth [2023-12-27 12:11:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000944288_241778688.pth [2023-12-27 12:11:52,708][20347] Updated weights for policy 0, policy_version 945510 (0.0008) [2023-12-27 12:11:52,762][20347] Updated weights for policy 0, policy_version 945520 (0.0008) [2023-12-27 12:11:52,810][20347] Updated weights for policy 0, policy_version 945530 (0.0008) [2023-12-27 12:11:52,968][20381] Updated weights for policy 1, policy_version 944602 (0.0009) [2023-12-27 12:11:52,985][20292] KL-divergence is very high: 109.8561 [2023-12-27 12:11:53,023][20381] Updated weights for policy 1, policy_version 944612 (0.0008) [2023-12-27 12:11:53,030][20292] KL-divergence is very high: 114.8148 [2023-12-27 12:11:53,085][20381] Updated weights for policy 1, policy_version 944622 (0.0011) [2023-12-27 12:11:53,140][20381] Updated weights for policy 1, policy_version 944632 (0.0010) [2023-12-27 12:11:53,506][20347] Updated weights for policy 0, policy_version 945540 (0.0009) [2023-12-27 12:11:53,568][20347] Updated weights for policy 0, policy_version 945550 (0.0011) [2023-12-27 12:11:53,630][20347] Updated weights for policy 0, policy_version 945560 (0.0010) [2023-12-27 12:11:53,766][20381] Updated weights for policy 1, policy_version 944642 (0.0010) [2023-12-27 12:11:53,826][20381] Updated weights for policy 1, policy_version 944652 (0.0009) [2023-12-27 12:11:53,851][20292] KL-divergence is very high: 185.2355 [2023-12-27 12:11:53,857][20292] KL-divergence is very high: 164.2738 [2023-12-27 12:11:53,883][20381] Updated weights for policy 1, policy_version 944662 (0.0008) [2023-12-27 12:11:54,324][20347] Updated weights for policy 0, policy_version 945570 (0.0010) [2023-12-27 12:11:54,372][20347] Updated weights for policy 0, policy_version 945580 (0.0010) [2023-12-27 12:11:54,430][20347] Updated weights for policy 0, policy_version 945590 (0.0005) [2023-12-27 12:11:54,490][20347] Updated weights for policy 0, policy_version 945600 (0.0007) [2023-12-27 12:11:54,528][20292] KL-divergence is very high: 118.2104 [2023-12-27 12:11:54,546][20292] KL-divergence is very high: 171.9751 [2023-12-27 12:11:54,565][20381] Updated weights for policy 1, policy_version 944672 (0.0008) [2023-12-27 12:11:54,613][20381] Updated weights for policy 1, policy_version 944682 (0.0008) [2023-12-27 12:11:54,657][20381] Updated weights for policy 1, policy_version 944692 (0.0006) [2023-12-27 12:11:55,155][20347] Updated weights for policy 0, policy_version 945610 (0.0008) [2023-12-27 12:11:55,212][20347] Updated weights for policy 0, policy_version 945620 (0.0008) [2023-12-27 12:11:55,261][20381] Updated weights for policy 1, policy_version 944702 (0.0005) [2023-12-27 12:11:55,276][20347] Updated weights for policy 0, policy_version 945630 (0.0008) [2023-12-27 12:11:55,312][20381] Updated weights for policy 1, policy_version 944712 (0.0008) [2023-12-27 12:11:55,355][20292] KL-divergence is very high: 125.7033 [2023-12-27 12:11:55,360][20381] Updated weights for policy 1, policy_version 944722 (0.0008) [2023-12-27 12:11:55,362][20292] KL-divergence is very high: 153.7808 [2023-12-27 12:11:55,368][20292] KL-divergence is very high: 264.2572 [2023-12-27 12:11:55,374][20292] KL-divergence is very high: 310.8822 [2023-12-27 12:11:55,894][20347] Updated weights for policy 0, policy_version 945640 (0.0006) [2023-12-27 12:11:55,948][20347] Updated weights for policy 0, policy_version 945650 (0.0005) [2023-12-27 12:11:55,952][20291] KL-divergence is very high: 183.3708 [2023-12-27 12:11:55,963][20291] KL-divergence is very high: 114.5954 [2023-12-27 12:11:55,978][20291] KL-divergence is very high: 319.4489 [2023-12-27 12:11:55,993][20291] KL-divergence is very high: 146.3466 [2023-12-27 12:11:55,999][20347] Updated weights for policy 0, policy_version 945660 (0.0005) [2023-12-27 12:11:56,004][20291] KL-divergence is very high: 104.6042 [2023-12-27 12:11:56,044][20292] KL-divergence is very high: 193.9156 [2023-12-27 12:11:56,068][20381] Updated weights for policy 1, policy_version 944732 (0.0008) [2023-12-27 12:11:56,098][20292] KL-divergence is very high: 104.4642 [2023-12-27 12:11:56,126][20381] Updated weights for policy 1, policy_version 944742 (0.0008) [2023-12-27 12:11:56,144][20292] KL-divergence is very high: 102.3860 [2023-12-27 12:11:56,187][20381] Updated weights for policy 1, policy_version 944752 (0.0009) [2023-12-27 12:11:56,501][20291] KL-divergence is very high: 151.2541 [2023-12-27 12:11:56,532][20347] Updated weights for policy 0, policy_version 945670 (0.0007) [2023-12-27 12:11:56,585][20347] Updated weights for policy 0, policy_version 945680 (0.0005) [2023-12-27 12:11:56,650][20347] Updated weights for policy 0, policy_version 945690 (0.0005) [2023-12-27 12:11:56,749][20381] Updated weights for policy 1, policy_version 944762 (0.0008) [2023-12-27 12:11:56,809][20381] Updated weights for policy 1, policy_version 944772 (0.0005) [2023-12-27 12:11:56,868][20381] Updated weights for policy 1, policy_version 944782 (0.0006) [2023-12-27 12:11:56,913][20381] Updated weights for policy 1, policy_version 944792 (0.0007) [2023-12-27 12:11:57,310][20347] Updated weights for policy 0, policy_version 945700 (0.0006) [2023-12-27 12:11:57,356][20347] Updated weights for policy 0, policy_version 945710 (0.0005) [2023-12-27 12:11:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 484032512. Throughput: 0: 10345.7, 1: 10166.4. Samples: 484023316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:11:57,363][19170] Avg episode reward: [(0, '932.280'), (1, '715.840')] [2023-12-27 12:11:57,412][20347] Updated weights for policy 0, policy_version 945720 (0.0005) [2023-12-27 12:11:57,493][20381] Updated weights for policy 1, policy_version 944802 (0.0008) [2023-12-27 12:11:57,545][20381] Updated weights for policy 1, policy_version 944812 (0.0006) [2023-12-27 12:11:57,593][20381] Updated weights for policy 1, policy_version 944822 (0.0007) [2023-12-27 12:11:58,070][20347] Updated weights for policy 0, policy_version 945730 (0.0006) [2023-12-27 12:11:58,119][20347] Updated weights for policy 0, policy_version 945740 (0.0007) [2023-12-27 12:11:58,181][20347] Updated weights for policy 0, policy_version 945750 (0.0008) [2023-12-27 12:11:58,246][20347] Updated weights for policy 0, policy_version 945760 (0.0007) [2023-12-27 12:11:58,264][20381] Updated weights for policy 1, policy_version 944832 (0.0008) [2023-12-27 12:11:58,327][20381] Updated weights for policy 1, policy_version 944842 (0.0008) [2023-12-27 12:11:58,391][20381] Updated weights for policy 1, policy_version 944852 (0.0007) [2023-12-27 12:11:59,099][20347] Updated weights for policy 0, policy_version 945770 (0.0006) [2023-12-27 12:11:59,140][20381] Updated weights for policy 1, policy_version 944862 (0.0010) [2023-12-27 12:11:59,152][20347] Updated weights for policy 0, policy_version 945780 (0.0007) [2023-12-27 12:11:59,198][20381] Updated weights for policy 1, policy_version 944872 (0.0010) [2023-12-27 12:11:59,209][20347] Updated weights for policy 0, policy_version 945790 (0.0007) [2023-12-27 12:11:59,264][20381] Updated weights for policy 1, policy_version 944882 (0.0008) [2023-12-27 12:11:59,953][20347] Updated weights for policy 0, policy_version 945800 (0.0008) [2023-12-27 12:11:59,972][20381] Updated weights for policy 1, policy_version 944892 (0.0007) [2023-12-27 12:12:00,013][20347] Updated weights for policy 0, policy_version 945810 (0.0008) [2023-12-27 12:12:00,014][20292] KL-divergence is very high: 133.1512 [2023-12-27 12:12:00,030][20381] Updated weights for policy 1, policy_version 944902 (0.0006) [2023-12-27 12:12:00,030][20292] KL-divergence is very high: 116.2927 [2023-12-27 12:12:00,065][20347] Updated weights for policy 0, policy_version 945820 (0.0008) [2023-12-27 12:12:00,089][20381] Updated weights for policy 1, policy_version 944912 (0.0005) [2023-12-27 12:12:00,661][20381] Updated weights for policy 1, policy_version 944922 (0.0006) [2023-12-27 12:12:00,668][20347] Updated weights for policy 0, policy_version 945830 (0.0009) [2023-12-27 12:12:00,697][20292] KL-divergence is very high: 139.3840 [2023-12-27 12:12:00,709][20381] Updated weights for policy 1, policy_version 944932 (0.0010) [2023-12-27 12:12:00,723][20347] Updated weights for policy 0, policy_version 945840 (0.0010) [2023-12-27 12:12:00,767][20381] Updated weights for policy 1, policy_version 944942 (0.0010) [2023-12-27 12:12:00,784][20347] Updated weights for policy 0, policy_version 945850 (0.0007) [2023-12-27 12:12:00,819][20381] Updated weights for policy 1, policy_version 944952 (0.0010) [2023-12-27 12:12:01,360][20347] Updated weights for policy 0, policy_version 945860 (0.0006) [2023-12-27 12:12:01,420][20347] Updated weights for policy 0, policy_version 945870 (0.0007) [2023-12-27 12:12:01,480][20347] Updated weights for policy 0, policy_version 945880 (0.0009) [2023-12-27 12:12:01,571][20381] Updated weights for policy 1, policy_version 944962 (0.0005) [2023-12-27 12:12:01,572][20292] KL-divergence is very high: 442.9929 [2023-12-27 12:12:01,584][20292] KL-divergence is very high: 364.3775 [2023-12-27 12:12:01,596][20292] KL-divergence is very high: 401.3401 [2023-12-27 12:12:01,603][20292] KL-divergence is very high: 363.6609 [2023-12-27 12:12:01,623][20292] KL-divergence is very high: 346.9113 [2023-12-27 12:12:01,636][20381] Updated weights for policy 1, policy_version 944972 (0.0007) [2023-12-27 12:12:01,637][20292] KL-divergence is very high: 102.2579 [2023-12-27 12:12:01,665][20292] KL-divergence is very high: 258.6379 [2023-12-27 12:12:01,670][20292] KL-divergence is very high: 102.7250 [2023-12-27 12:12:01,681][20292] KL-divergence is very high: 145.3404 [2023-12-27 12:12:01,690][20381] Updated weights for policy 1, policy_version 944982 (0.0007) [2023-12-27 12:12:02,192][20347] Updated weights for policy 0, policy_version 945890 (0.0010) [2023-12-27 12:12:02,247][20347] Updated weights for policy 0, policy_version 945900 (0.0010) [2023-12-27 12:12:02,313][20347] Updated weights for policy 0, policy_version 945910 (0.0010) [2023-12-27 12:12:02,337][20292] KL-divergence is very high: 225.3850 [2023-12-27 12:12:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 484130816. Throughput: 0: 10380.8, 1: 10154.2. Samples: 484148940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:02,363][19170] Avg episode reward: [(0, '912.910'), (1, '705.900')] [2023-12-27 12:12:02,368][20347] Updated weights for policy 0, policy_version 945920 (0.0009) [2023-12-27 12:12:02,393][20381] Updated weights for policy 1, policy_version 944992 (0.0008) [2023-12-27 12:12:02,395][20292] KL-divergence is very high: 173.5476 [2023-12-27 12:12:02,439][20292] KL-divergence is very high: 257.7472 [2023-12-27 12:12:02,451][20381] Updated weights for policy 1, policy_version 945002 (0.0008) [2023-12-27 12:12:02,477][20292] KL-divergence is very high: 218.9228 [2023-12-27 12:12:02,497][20381] Updated weights for policy 1, policy_version 945012 (0.0007) [2023-12-27 12:12:03,049][20381] Updated weights for policy 1, policy_version 945022 (0.0005) [2023-12-27 12:12:03,059][20347] Updated weights for policy 0, policy_version 945930 (0.0008) [2023-12-27 12:12:03,099][20381] Updated weights for policy 1, policy_version 945032 (0.0006) [2023-12-27 12:12:03,108][20347] Updated weights for policy 0, policy_version 945940 (0.0005) [2023-12-27 12:12:03,144][20381] Updated weights for policy 1, policy_version 945042 (0.0005) [2023-12-27 12:12:03,157][20347] Updated weights for policy 0, policy_version 945950 (0.0005) [2023-12-27 12:12:03,775][20381] Updated weights for policy 1, policy_version 945052 (0.0005) [2023-12-27 12:12:03,836][20381] Updated weights for policy 1, policy_version 945062 (0.0008) [2023-12-27 12:12:03,880][20347] Updated weights for policy 0, policy_version 945960 (0.0008) [2023-12-27 12:12:03,896][20381] Updated weights for policy 1, policy_version 945072 (0.0010) [2023-12-27 12:12:03,931][20347] Updated weights for policy 0, policy_version 945970 (0.0008) [2023-12-27 12:12:03,956][20291] KL-divergence is very high: 479.3899 [2023-12-27 12:12:03,989][20347] Updated weights for policy 0, policy_version 945980 (0.0007) [2023-12-27 12:12:04,001][20291] KL-divergence is very high: 286.7675 [2023-12-27 12:12:04,509][20381] Updated weights for policy 1, policy_version 945082 (0.0011) [2023-12-27 12:12:04,570][20381] Updated weights for policy 1, policy_version 945092 (0.0010) [2023-12-27 12:12:04,623][20381] Updated weights for policy 1, policy_version 945102 (0.0011) [2023-12-27 12:12:04,680][20381] Updated weights for policy 1, policy_version 945112 (0.0011) [2023-12-27 12:12:04,681][20347] Updated weights for policy 0, policy_version 945990 (0.0006) [2023-12-27 12:12:04,739][20347] Updated weights for policy 0, policy_version 946000 (0.0007) [2023-12-27 12:12:04,806][20347] Updated weights for policy 0, policy_version 946010 (0.0007) [2023-12-27 12:12:05,308][20381] Updated weights for policy 1, policy_version 945122 (0.0007) [2023-12-27 12:12:05,356][20381] Updated weights for policy 1, policy_version 945132 (0.0007) [2023-12-27 12:12:05,414][20381] Updated weights for policy 1, policy_version 945142 (0.0005) [2023-12-27 12:12:05,427][20347] Updated weights for policy 0, policy_version 946020 (0.0007) [2023-12-27 12:12:05,471][20347] Updated weights for policy 0, policy_version 946030 (0.0008) [2023-12-27 12:12:05,523][20347] Updated weights for policy 0, policy_version 946040 (0.0008) [2023-12-27 12:12:06,002][20381] Updated weights for policy 1, policy_version 945152 (0.0005) [2023-12-27 12:12:06,054][20381] Updated weights for policy 1, policy_version 945162 (0.0005) [2023-12-27 12:12:06,115][20381] Updated weights for policy 1, policy_version 945172 (0.0007) [2023-12-27 12:12:06,251][20347] Updated weights for policy 0, policy_version 946050 (0.0008) [2023-12-27 12:12:06,317][20347] Updated weights for policy 0, policy_version 946060 (0.0008) [2023-12-27 12:12:06,373][20291] KL-divergence is very high: 224.5227 [2023-12-27 12:12:06,379][20347] Updated weights for policy 0, policy_version 946070 (0.0008) [2023-12-27 12:12:06,380][20291] KL-divergence is very high: 216.8743 [2023-12-27 12:12:06,443][20347] Updated weights for policy 0, policy_version 946080 (0.0008) [2023-12-27 12:12:06,751][20381] Updated weights for policy 1, policy_version 945182 (0.0008) [2023-12-27 12:12:06,807][20381] Updated weights for policy 1, policy_version 945192 (0.0007) [2023-12-27 12:12:06,866][20381] Updated weights for policy 1, policy_version 945202 (0.0008) [2023-12-27 12:12:07,037][20291] KL-divergence is very high: 324.2519 [2023-12-27 12:12:07,053][20291] KL-divergence is very high: 152.9711 [2023-12-27 12:12:07,080][20291] KL-divergence is very high: 339.0742 [2023-12-27 12:12:07,089][20347] Updated weights for policy 0, policy_version 946090 (0.0005) [2023-12-27 12:12:07,120][20291] KL-divergence is very high: 140.4454 [2023-12-27 12:12:07,142][20347] Updated weights for policy 0, policy_version 946100 (0.0008) [2023-12-27 12:12:07,161][20291] KL-divergence is very high: 117.0673 [2023-12-27 12:12:07,199][20347] Updated weights for policy 0, policy_version 946110 (0.0011) [2023-12-27 12:12:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 484245504. Throughput: 0: 10375.2, 1: 10239.5. Samples: 484212044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:07,363][19170] Avg episode reward: [(0, '865.780'), (1, '755.220')] [2023-12-27 12:12:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000945208_241999872.pth... [2023-12-27 12:12:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000946112_242245632.pth... [2023-12-27 12:12:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000944896_241934336.pth [2023-12-27 12:12:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000943992_241688576.pth [2023-12-27 12:12:07,557][20381] Updated weights for policy 1, policy_version 945212 (0.0008) [2023-12-27 12:12:07,606][20381] Updated weights for policy 1, policy_version 945222 (0.0010) [2023-12-27 12:12:07,658][20381] Updated weights for policy 1, policy_version 945232 (0.0010) [2023-12-27 12:12:07,847][20347] Updated weights for policy 0, policy_version 946120 (0.0007) [2023-12-27 12:12:07,904][20347] Updated weights for policy 0, policy_version 946130 (0.0009) [2023-12-27 12:12:07,956][20347] Updated weights for policy 0, policy_version 946140 (0.0009) [2023-12-27 12:12:08,292][20381] Updated weights for policy 1, policy_version 945242 (0.0010) [2023-12-27 12:12:08,348][20381] Updated weights for policy 1, policy_version 945252 (0.0007) [2023-12-27 12:12:08,409][20381] Updated weights for policy 1, policy_version 945262 (0.0007) [2023-12-27 12:12:08,470][20381] Updated weights for policy 1, policy_version 945272 (0.0007) [2023-12-27 12:12:08,667][20347] Updated weights for policy 0, policy_version 946150 (0.0009) [2023-12-27 12:12:08,715][20347] Updated weights for policy 0, policy_version 946160 (0.0007) [2023-12-27 12:12:08,768][20347] Updated weights for policy 0, policy_version 946170 (0.0008) [2023-12-27 12:12:09,141][20381] Updated weights for policy 1, policy_version 945282 (0.0005) [2023-12-27 12:12:09,196][20381] Updated weights for policy 1, policy_version 945292 (0.0006) [2023-12-27 12:12:09,255][20381] Updated weights for policy 1, policy_version 945302 (0.0008) [2023-12-27 12:12:09,557][20347] Updated weights for policy 0, policy_version 946180 (0.0008) [2023-12-27 12:12:09,607][20347] Updated weights for policy 0, policy_version 946190 (0.0008) [2023-12-27 12:12:09,661][20347] Updated weights for policy 0, policy_version 946200 (0.0007) [2023-12-27 12:12:09,983][20381] Updated weights for policy 1, policy_version 945312 (0.0009) [2023-12-27 12:12:10,045][20381] Updated weights for policy 1, policy_version 945322 (0.0008) [2023-12-27 12:12:10,107][20381] Updated weights for policy 1, policy_version 945332 (0.0008) [2023-12-27 12:12:10,341][20347] Updated weights for policy 0, policy_version 946210 (0.0005) [2023-12-27 12:12:10,409][20347] Updated weights for policy 0, policy_version 946220 (0.0006) [2023-12-27 12:12:10,467][20347] Updated weights for policy 0, policy_version 946230 (0.0007) [2023-12-27 12:12:10,528][20347] Updated weights for policy 0, policy_version 946240 (0.0008) [2023-12-27 12:12:10,836][20381] Updated weights for policy 1, policy_version 945342 (0.0007) [2023-12-27 12:12:10,896][20381] Updated weights for policy 1, policy_version 945352 (0.0008) [2023-12-27 12:12:10,955][20381] Updated weights for policy 1, policy_version 945362 (0.0006) [2023-12-27 12:12:11,178][20347] Updated weights for policy 0, policy_version 946250 (0.0008) [2023-12-27 12:12:11,239][20347] Updated weights for policy 0, policy_version 946260 (0.0008) [2023-12-27 12:12:11,302][20347] Updated weights for policy 0, policy_version 946270 (0.0006) [2023-12-27 12:12:11,661][20381] Updated weights for policy 1, policy_version 945372 (0.0007) [2023-12-27 12:12:11,727][20381] Updated weights for policy 1, policy_version 945382 (0.0008) [2023-12-27 12:12:11,793][20381] Updated weights for policy 1, policy_version 945392 (0.0007) [2023-12-27 12:12:12,052][20347] Updated weights for policy 0, policy_version 946280 (0.0006) [2023-12-27 12:12:12,113][20347] Updated weights for policy 0, policy_version 946290 (0.0007) [2023-12-27 12:12:12,172][20347] Updated weights for policy 0, policy_version 946300 (0.0008) [2023-12-27 12:12:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 484343808. Throughput: 0: 10343.0, 1: 10318.0. Samples: 484334880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:12,363][19170] Avg episode reward: [(0, '778.370'), (1, '833.400')] [2023-12-27 12:12:12,456][20381] Updated weights for policy 1, policy_version 945402 (0.0006) [2023-12-27 12:12:12,515][20381] Updated weights for policy 1, policy_version 945412 (0.0007) [2023-12-27 12:12:12,565][20381] Updated weights for policy 1, policy_version 945422 (0.0009) [2023-12-27 12:12:12,622][20381] Updated weights for policy 1, policy_version 945432 (0.0007) [2023-12-27 12:12:12,770][20347] Updated weights for policy 0, policy_version 946310 (0.0007) [2023-12-27 12:12:12,823][20347] Updated weights for policy 0, policy_version 946320 (0.0008) [2023-12-27 12:12:12,886][20347] Updated weights for policy 0, policy_version 946330 (0.0008) [2023-12-27 12:12:13,264][20381] Updated weights for policy 1, policy_version 945442 (0.0006) [2023-12-27 12:12:13,322][20381] Updated weights for policy 1, policy_version 945452 (0.0005) [2023-12-27 12:12:13,380][20381] Updated weights for policy 1, policy_version 945462 (0.0005) [2023-12-27 12:12:13,551][20347] Updated weights for policy 0, policy_version 946340 (0.0007) [2023-12-27 12:12:13,614][20347] Updated weights for policy 0, policy_version 946350 (0.0006) [2023-12-27 12:12:13,671][20347] Updated weights for policy 0, policy_version 946360 (0.0008) [2023-12-27 12:12:13,976][20381] Updated weights for policy 1, policy_version 945472 (0.0010) [2023-12-27 12:12:14,035][20381] Updated weights for policy 1, policy_version 945482 (0.0010) [2023-12-27 12:12:14,095][20381] Updated weights for policy 1, policy_version 945492 (0.0008) [2023-12-27 12:12:14,345][20347] Updated weights for policy 0, policy_version 946370 (0.0008) [2023-12-27 12:12:14,400][20347] Updated weights for policy 0, policy_version 946380 (0.0005) [2023-12-27 12:12:14,452][20347] Updated weights for policy 0, policy_version 946390 (0.0009) [2023-12-27 12:12:14,507][20347] Updated weights for policy 0, policy_version 946400 (0.0010) [2023-12-27 12:12:14,799][20381] Updated weights for policy 1, policy_version 945502 (0.0008) [2023-12-27 12:12:14,852][20381] Updated weights for policy 1, policy_version 945512 (0.0008) [2023-12-27 12:12:14,915][20381] Updated weights for policy 1, policy_version 945522 (0.0008) [2023-12-27 12:12:15,211][20347] Updated weights for policy 0, policy_version 946410 (0.0011) [2023-12-27 12:12:15,271][20347] Updated weights for policy 0, policy_version 946420 (0.0010) [2023-12-27 12:12:15,327][20347] Updated weights for policy 0, policy_version 946430 (0.0010) [2023-12-27 12:12:15,564][20381] Updated weights for policy 1, policy_version 945532 (0.0007) [2023-12-27 12:12:15,624][20381] Updated weights for policy 1, policy_version 945542 (0.0005) [2023-12-27 12:12:15,680][20381] Updated weights for policy 1, policy_version 945552 (0.0006) [2023-12-27 12:12:15,922][20347] Updated weights for policy 0, policy_version 946440 (0.0006) [2023-12-27 12:12:15,982][20347] Updated weights for policy 0, policy_version 946450 (0.0005) [2023-12-27 12:12:16,048][20347] Updated weights for policy 0, policy_version 946460 (0.0007) [2023-12-27 12:12:16,295][20381] Updated weights for policy 1, policy_version 945562 (0.0008) [2023-12-27 12:12:16,362][20381] Updated weights for policy 1, policy_version 945572 (0.0007) [2023-12-27 12:12:16,419][20381] Updated weights for policy 1, policy_version 945582 (0.0007) [2023-12-27 12:12:16,463][20381] Updated weights for policy 1, policy_version 945592 (0.0007) [2023-12-27 12:12:16,696][20347] Updated weights for policy 0, policy_version 946470 (0.0010) [2023-12-27 12:12:16,740][20347] Updated weights for policy 0, policy_version 946480 (0.0010) [2023-12-27 12:12:16,800][20347] Updated weights for policy 0, policy_version 946490 (0.0010) [2023-12-27 12:12:17,148][20381] Updated weights for policy 1, policy_version 945602 (0.0010) [2023-12-27 12:12:17,196][20381] Updated weights for policy 1, policy_version 945612 (0.0010) [2023-12-27 12:12:17,251][20381] Updated weights for policy 1, policy_version 945622 (0.0006) [2023-12-27 12:12:17,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 484450304. Throughput: 0: 10325.1, 1: 10342.7. Samples: 484459152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:17,363][19170] Avg episode reward: [(0, '913.010'), (1, '833.410')] [2023-12-27 12:12:17,467][20347] Updated weights for policy 0, policy_version 946500 (0.0010) [2023-12-27 12:12:17,529][20347] Updated weights for policy 0, policy_version 946510 (0.0006) [2023-12-27 12:12:17,593][20347] Updated weights for policy 0, policy_version 946520 (0.0005) [2023-12-27 12:12:17,853][20381] Updated weights for policy 1, policy_version 945632 (0.0008) [2023-12-27 12:12:17,908][20381] Updated weights for policy 1, policy_version 945642 (0.0010) [2023-12-27 12:12:17,953][20381] Updated weights for policy 1, policy_version 945652 (0.0010) [2023-12-27 12:12:18,198][20347] Updated weights for policy 0, policy_version 946530 (0.0009) [2023-12-27 12:12:18,256][20347] Updated weights for policy 0, policy_version 946540 (0.0009) [2023-12-27 12:12:18,314][20347] Updated weights for policy 0, policy_version 946550 (0.0010) [2023-12-27 12:12:18,374][20347] Updated weights for policy 0, policy_version 946560 (0.0010) [2023-12-27 12:12:18,607][20381] Updated weights for policy 1, policy_version 945662 (0.0007) [2023-12-27 12:12:18,668][20381] Updated weights for policy 1, policy_version 945672 (0.0006) [2023-12-27 12:12:18,732][20381] Updated weights for policy 1, policy_version 945682 (0.0006) [2023-12-27 12:12:19,064][20347] Updated weights for policy 0, policy_version 946570 (0.0011) [2023-12-27 12:12:19,116][20347] Updated weights for policy 0, policy_version 946580 (0.0010) [2023-12-27 12:12:19,172][20347] Updated weights for policy 0, policy_version 946590 (0.0006) [2023-12-27 12:12:19,417][20381] Updated weights for policy 1, policy_version 945692 (0.0008) [2023-12-27 12:12:19,473][20381] Updated weights for policy 1, policy_version 945702 (0.0010) [2023-12-27 12:12:19,537][20381] Updated weights for policy 1, policy_version 945712 (0.0008) [2023-12-27 12:12:19,872][20347] Updated weights for policy 0, policy_version 946600 (0.0009) [2023-12-27 12:12:19,934][20347] Updated weights for policy 0, policy_version 946610 (0.0008) [2023-12-27 12:12:19,999][20347] Updated weights for policy 0, policy_version 946620 (0.0006) [2023-12-27 12:12:20,217][20381] Updated weights for policy 1, policy_version 945722 (0.0008) [2023-12-27 12:12:20,278][20381] Updated weights for policy 1, policy_version 945732 (0.0006) [2023-12-27 12:12:20,342][20381] Updated weights for policy 1, policy_version 945742 (0.0007) [2023-12-27 12:12:20,395][20381] Updated weights for policy 1, policy_version 945752 (0.0008) [2023-12-27 12:12:20,721][20347] Updated weights for policy 0, policy_version 946630 (0.0007) [2023-12-27 12:12:20,783][20347] Updated weights for policy 0, policy_version 946640 (0.0006) [2023-12-27 12:12:20,839][20347] Updated weights for policy 0, policy_version 946650 (0.0005) [2023-12-27 12:12:21,171][20381] Updated weights for policy 1, policy_version 945762 (0.0008) [2023-12-27 12:12:21,237][20381] Updated weights for policy 1, policy_version 945772 (0.0008) [2023-12-27 12:12:21,304][20381] Updated weights for policy 1, policy_version 945782 (0.0007) [2023-12-27 12:12:21,578][20347] Updated weights for policy 0, policy_version 946660 (0.0008) [2023-12-27 12:12:21,649][20347] Updated weights for policy 0, policy_version 946670 (0.0008) [2023-12-27 12:12:21,716][20347] Updated weights for policy 0, policy_version 946680 (0.0007) [2023-12-27 12:12:22,014][20381] Updated weights for policy 1, policy_version 945792 (0.0008) [2023-12-27 12:12:22,074][20381] Updated weights for policy 1, policy_version 945802 (0.0008) [2023-12-27 12:12:22,138][20381] Updated weights for policy 1, policy_version 945812 (0.0009) [2023-12-27 12:12:22,363][20347] Updated weights for policy 0, policy_version 946690 (0.0008) [2023-12-27 12:12:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 484548608. Throughput: 0: 10269.7, 1: 10355.7. Samples: 484522132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:22,363][19170] Avg episode reward: [(0, '912.880'), (1, '557.400')] [2023-12-27 12:12:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000945816_242155520.pth... [2023-12-27 12:12:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000944600_241844224.pth [2023-12-27 12:12:22,419][20347] Updated weights for policy 0, policy_version 946700 (0.0009) [2023-12-27 12:12:22,481][20347] Updated weights for policy 0, policy_version 946710 (0.0010) [2023-12-27 12:12:22,538][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000946720_242401280.pth... [2023-12-27 12:12:22,540][20347] Updated weights for policy 0, policy_version 946720 (0.0010) [2023-12-27 12:12:22,542][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000945504_242089984.pth [2023-12-27 12:12:22,811][20381] Updated weights for policy 1, policy_version 945822 (0.0007) [2023-12-27 12:12:22,860][20381] Updated weights for policy 1, policy_version 945832 (0.0005) [2023-12-27 12:12:22,911][20381] Updated weights for policy 1, policy_version 945842 (0.0006) [2023-12-27 12:12:23,243][20347] Updated weights for policy 0, policy_version 946730 (0.0008) [2023-12-27 12:12:23,294][20347] Updated weights for policy 0, policy_version 946740 (0.0007) [2023-12-27 12:12:23,349][20347] Updated weights for policy 0, policy_version 946750 (0.0008) [2023-12-27 12:12:23,543][20381] Updated weights for policy 1, policy_version 945852 (0.0008) [2023-12-27 12:12:23,607][20381] Updated weights for policy 1, policy_version 945862 (0.0009) [2023-12-27 12:12:23,664][20381] Updated weights for policy 1, policy_version 945872 (0.0008) [2023-12-27 12:12:24,022][20347] Updated weights for policy 0, policy_version 946760 (0.0007) [2023-12-27 12:12:24,079][20347] Updated weights for policy 0, policy_version 946770 (0.0007) [2023-12-27 12:12:24,141][20347] Updated weights for policy 0, policy_version 946780 (0.0007) [2023-12-27 12:12:24,280][20381] Updated weights for policy 1, policy_version 945882 (0.0008) [2023-12-27 12:12:24,336][20381] Updated weights for policy 1, policy_version 945892 (0.0007) [2023-12-27 12:12:24,388][20381] Updated weights for policy 1, policy_version 945902 (0.0005) [2023-12-27 12:12:24,440][20381] Updated weights for policy 1, policy_version 945912 (0.0007) [2023-12-27 12:12:24,857][20347] Updated weights for policy 0, policy_version 946790 (0.0009) [2023-12-27 12:12:24,913][20347] Updated weights for policy 0, policy_version 946800 (0.0010) [2023-12-27 12:12:24,972][20347] Updated weights for policy 0, policy_version 946810 (0.0010) [2023-12-27 12:12:25,045][20381] Updated weights for policy 1, policy_version 945922 (0.0005) [2023-12-27 12:12:25,098][20381] Updated weights for policy 1, policy_version 945932 (0.0005) [2023-12-27 12:12:25,153][20381] Updated weights for policy 1, policy_version 945942 (0.0006) [2023-12-27 12:12:25,644][20347] Updated weights for policy 0, policy_version 946820 (0.0008) [2023-12-27 12:12:25,692][20347] Updated weights for policy 0, policy_version 946830 (0.0005) [2023-12-27 12:12:25,742][20347] Updated weights for policy 0, policy_version 946840 (0.0006) [2023-12-27 12:12:25,805][20381] Updated weights for policy 1, policy_version 945952 (0.0005) [2023-12-27 12:12:25,859][20381] Updated weights for policy 1, policy_version 945962 (0.0007) [2023-12-27 12:12:25,910][20381] Updated weights for policy 1, policy_version 945972 (0.0009) [2023-12-27 12:12:26,339][20347] Updated weights for policy 0, policy_version 946850 (0.0006) [2023-12-27 12:12:26,388][20347] Updated weights for policy 0, policy_version 946860 (0.0007) [2023-12-27 12:12:26,444][20347] Updated weights for policy 0, policy_version 946870 (0.0007) [2023-12-27 12:12:26,492][20347] Updated weights for policy 0, policy_version 946880 (0.0007) [2023-12-27 12:12:26,614][20381] Updated weights for policy 1, policy_version 945982 (0.0009) [2023-12-27 12:12:26,674][20381] Updated weights for policy 1, policy_version 945992 (0.0010) [2023-12-27 12:12:26,732][20381] Updated weights for policy 1, policy_version 946002 (0.0009) [2023-12-27 12:12:27,143][20347] Updated weights for policy 0, policy_version 946890 (0.0005) [2023-12-27 12:12:27,191][20347] Updated weights for policy 0, policy_version 946900 (0.0005) [2023-12-27 12:12:27,239][20347] Updated weights for policy 0, policy_version 946910 (0.0005) [2023-12-27 12:12:27,339][20381] Updated weights for policy 1, policy_version 946012 (0.0007) [2023-12-27 12:12:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 484655104. Throughput: 0: 10265.0, 1: 10351.5. Samples: 484643636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:27,363][19170] Avg episode reward: [(0, '883.160'), (1, '589.430')] [2023-12-27 12:12:27,386][20381] Updated weights for policy 1, policy_version 946022 (0.0007) [2023-12-27 12:12:27,444][20381] Updated weights for policy 1, policy_version 946032 (0.0007) [2023-12-27 12:12:27,846][20347] Updated weights for policy 0, policy_version 946920 (0.0009) [2023-12-27 12:12:27,903][20347] Updated weights for policy 0, policy_version 946930 (0.0010) [2023-12-27 12:12:27,953][20347] Updated weights for policy 0, policy_version 946940 (0.0010) [2023-12-27 12:12:28,148][20381] Updated weights for policy 1, policy_version 946042 (0.0007) [2023-12-27 12:12:28,195][20381] Updated weights for policy 1, policy_version 946052 (0.0007) [2023-12-27 12:12:28,248][20381] Updated weights for policy 1, policy_version 946062 (0.0007) [2023-12-27 12:12:28,299][20381] Updated weights for policy 1, policy_version 946072 (0.0007) [2023-12-27 12:12:28,642][20347] Updated weights for policy 0, policy_version 946950 (0.0010) [2023-12-27 12:12:28,693][20347] Updated weights for policy 0, policy_version 946960 (0.0010) [2023-12-27 12:12:28,750][20347] Updated weights for policy 0, policy_version 946970 (0.0010) [2023-12-27 12:12:28,999][20381] Updated weights for policy 1, policy_version 946082 (0.0007) [2023-12-27 12:12:29,060][20381] Updated weights for policy 1, policy_version 946092 (0.0007) [2023-12-27 12:12:29,114][20381] Updated weights for policy 1, policy_version 946102 (0.0007) [2023-12-27 12:12:29,481][20347] Updated weights for policy 0, policy_version 946980 (0.0010) [2023-12-27 12:12:29,539][20347] Updated weights for policy 0, policy_version 946990 (0.0010) [2023-12-27 12:12:29,598][20347] Updated weights for policy 0, policy_version 947000 (0.0010) [2023-12-27 12:12:29,847][20381] Updated weights for policy 1, policy_version 946112 (0.0008) [2023-12-27 12:12:29,902][20381] Updated weights for policy 1, policy_version 946122 (0.0008) [2023-12-27 12:12:29,959][20381] Updated weights for policy 1, policy_version 946132 (0.0007) [2023-12-27 12:12:30,333][20347] Updated weights for policy 0, policy_version 947010 (0.0010) [2023-12-27 12:12:30,387][20347] Updated weights for policy 0, policy_version 947020 (0.0008) [2023-12-27 12:12:30,437][20347] Updated weights for policy 0, policy_version 947030 (0.0008) [2023-12-27 12:12:30,496][20347] Updated weights for policy 0, policy_version 947040 (0.0006) [2023-12-27 12:12:30,549][20381] Updated weights for policy 1, policy_version 946142 (0.0008) [2023-12-27 12:12:30,597][20381] Updated weights for policy 1, policy_version 946152 (0.0007) [2023-12-27 12:12:30,657][20381] Updated weights for policy 1, policy_version 946162 (0.0008) [2023-12-27 12:12:31,123][20347] Updated weights for policy 0, policy_version 947050 (0.0008) [2023-12-27 12:12:31,181][20347] Updated weights for policy 0, policy_version 947060 (0.0008) [2023-12-27 12:12:31,241][20347] Updated weights for policy 0, policy_version 947070 (0.0007) [2023-12-27 12:12:31,408][20381] Updated weights for policy 1, policy_version 946172 (0.0008) [2023-12-27 12:12:31,465][20381] Updated weights for policy 1, policy_version 946182 (0.0007) [2023-12-27 12:12:31,528][20381] Updated weights for policy 1, policy_version 946192 (0.0009) [2023-12-27 12:12:31,965][20347] Updated weights for policy 0, policy_version 947080 (0.0008) [2023-12-27 12:12:32,024][20347] Updated weights for policy 0, policy_version 947090 (0.0010) [2023-12-27 12:12:32,073][20347] Updated weights for policy 0, policy_version 947100 (0.0010) [2023-12-27 12:12:32,214][20381] Updated weights for policy 1, policy_version 946202 (0.0008) [2023-12-27 12:12:32,275][20381] Updated weights for policy 1, policy_version 946212 (0.0009) [2023-12-27 12:12:32,332][20381] Updated weights for policy 1, policy_version 946222 (0.0009) [2023-12-27 12:12:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 484753408. Throughput: 0: 10231.5, 1: 10319.8. Samples: 484766172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:32,363][19170] Avg episode reward: [(0, '892.970'), (1, '558.550')] [2023-12-27 12:12:32,394][20381] Updated weights for policy 1, policy_version 946232 (0.0011) [2023-12-27 12:12:32,764][20347] Updated weights for policy 0, policy_version 947110 (0.0007) [2023-12-27 12:12:32,818][20347] Updated weights for policy 0, policy_version 947120 (0.0005) [2023-12-27 12:12:32,864][20347] Updated weights for policy 0, policy_version 947130 (0.0005) [2023-12-27 12:12:33,004][20381] Updated weights for policy 1, policy_version 946242 (0.0005) [2023-12-27 12:12:33,063][20381] Updated weights for policy 1, policy_version 946252 (0.0005) [2023-12-27 12:12:33,123][20381] Updated weights for policy 1, policy_version 946262 (0.0008) [2023-12-27 12:12:33,487][20347] Updated weights for policy 0, policy_version 947140 (0.0007) [2023-12-27 12:12:33,545][20347] Updated weights for policy 0, policy_version 947150 (0.0009) [2023-12-27 12:12:33,604][20347] Updated weights for policy 0, policy_version 947160 (0.0007) [2023-12-27 12:12:33,627][20291] KL-divergence is very high: 244.6506 [2023-12-27 12:12:33,699][20381] Updated weights for policy 1, policy_version 946272 (0.0007) [2023-12-27 12:12:33,750][20381] Updated weights for policy 1, policy_version 946282 (0.0007) [2023-12-27 12:12:33,797][20381] Updated weights for policy 1, policy_version 946292 (0.0008) [2023-12-27 12:12:34,195][20347] Updated weights for policy 0, policy_version 947170 (0.0006) [2023-12-27 12:12:34,243][20347] Updated weights for policy 0, policy_version 947180 (0.0006) [2023-12-27 12:12:34,302][20347] Updated weights for policy 0, policy_version 947190 (0.0006) [2023-12-27 12:12:34,365][20347] Updated weights for policy 0, policy_version 947200 (0.0006) [2023-12-27 12:12:34,534][20381] Updated weights for policy 1, policy_version 946302 (0.0007) [2023-12-27 12:12:34,589][20381] Updated weights for policy 1, policy_version 946312 (0.0008) [2023-12-27 12:12:34,646][20381] Updated weights for policy 1, policy_version 946322 (0.0008) [2023-12-27 12:12:34,941][20347] Updated weights for policy 0, policy_version 947210 (0.0005) [2023-12-27 12:12:34,988][20347] Updated weights for policy 0, policy_version 947220 (0.0005) [2023-12-27 12:12:35,038][20347] Updated weights for policy 0, policy_version 947230 (0.0005) [2023-12-27 12:12:35,347][20381] Updated weights for policy 1, policy_version 946332 (0.0007) [2023-12-27 12:12:35,401][20381] Updated weights for policy 1, policy_version 946342 (0.0006) [2023-12-27 12:12:35,461][20381] Updated weights for policy 1, policy_version 946352 (0.0007) [2023-12-27 12:12:35,644][20347] Updated weights for policy 0, policy_version 947240 (0.0007) [2023-12-27 12:12:35,695][20347] Updated weights for policy 0, policy_version 947250 (0.0007) [2023-12-27 12:12:35,753][20347] Updated weights for policy 0, policy_version 947260 (0.0007) [2023-12-27 12:12:36,124][20381] Updated weights for policy 1, policy_version 946362 (0.0010) [2023-12-27 12:12:36,188][20381] Updated weights for policy 1, policy_version 946372 (0.0007) [2023-12-27 12:12:36,251][20381] Updated weights for policy 1, policy_version 946382 (0.0011) [2023-12-27 12:12:36,308][20381] Updated weights for policy 1, policy_version 946392 (0.0011) [2023-12-27 12:12:36,384][20347] Updated weights for policy 0, policy_version 947270 (0.0007) [2023-12-27 12:12:36,444][20347] Updated weights for policy 0, policy_version 947280 (0.0007) [2023-12-27 12:12:36,501][20347] Updated weights for policy 0, policy_version 947290 (0.0008) [2023-12-27 12:12:36,932][20381] Updated weights for policy 1, policy_version 946402 (0.0009) [2023-12-27 12:12:36,990][20381] Updated weights for policy 1, policy_version 946412 (0.0011) [2023-12-27 12:12:37,039][20381] Updated weights for policy 1, policy_version 946422 (0.0010) [2023-12-27 12:12:37,192][20347] Updated weights for policy 0, policy_version 947300 (0.0006) [2023-12-27 12:12:37,253][20347] Updated weights for policy 0, policy_version 947310 (0.0007) [2023-12-27 12:12:37,313][20347] Updated weights for policy 0, policy_version 947320 (0.0006) [2023-12-27 12:12:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 484859904. Throughput: 0: 10254.5, 1: 10376.0. Samples: 484830708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:37,363][19170] Avg episode reward: [(0, '902.640'), (1, '667.060')] [2023-12-27 12:12:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000946424_242311168.pth... [2023-12-27 12:12:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000947328_242556928.pth... [2023-12-27 12:12:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000945208_241999872.pth [2023-12-27 12:12:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000946112_242245632.pth [2023-12-27 12:12:37,669][20381] Updated weights for policy 1, policy_version 946432 (0.0008) [2023-12-27 12:12:37,724][20381] Updated weights for policy 1, policy_version 946442 (0.0007) [2023-12-27 12:12:37,789][20381] Updated weights for policy 1, policy_version 946452 (0.0008) [2023-12-27 12:12:37,960][20347] Updated weights for policy 0, policy_version 947330 (0.0006) [2023-12-27 12:12:38,011][20347] Updated weights for policy 0, policy_version 947340 (0.0007) [2023-12-27 12:12:38,061][20347] Updated weights for policy 0, policy_version 947350 (0.0008) [2023-12-27 12:12:38,112][20347] Updated weights for policy 0, policy_version 947360 (0.0007) [2023-12-27 12:12:38,367][20381] Updated weights for policy 1, policy_version 946462 (0.0007) [2023-12-27 12:12:38,430][20381] Updated weights for policy 1, policy_version 946472 (0.0008) [2023-12-27 12:12:38,490][20381] Updated weights for policy 1, policy_version 946482 (0.0007) [2023-12-27 12:12:38,880][20347] Updated weights for policy 0, policy_version 947370 (0.0008) [2023-12-27 12:12:38,940][20347] Updated weights for policy 0, policy_version 947380 (0.0007) [2023-12-27 12:12:38,996][20347] Updated weights for policy 0, policy_version 947390 (0.0008) [2023-12-27 12:12:39,222][20381] Updated weights for policy 1, policy_version 946492 (0.0008) [2023-12-27 12:12:39,284][20381] Updated weights for policy 1, policy_version 946502 (0.0008) [2023-12-27 12:12:39,343][20381] Updated weights for policy 1, policy_version 946512 (0.0008) [2023-12-27 12:12:39,777][20347] Updated weights for policy 0, policy_version 947400 (0.0006) [2023-12-27 12:12:39,846][20347] Updated weights for policy 0, policy_version 947410 (0.0009) [2023-12-27 12:12:39,906][20347] Updated weights for policy 0, policy_version 947420 (0.0011) [2023-12-27 12:12:40,132][20381] Updated weights for policy 1, policy_version 946522 (0.0008) [2023-12-27 12:12:40,190][20381] Updated weights for policy 1, policy_version 946532 (0.0007) [2023-12-27 12:12:40,248][20381] Updated weights for policy 1, policy_version 946542 (0.0006) [2023-12-27 12:12:40,310][20381] Updated weights for policy 1, policy_version 946552 (0.0008) [2023-12-27 12:12:40,620][20347] Updated weights for policy 0, policy_version 947430 (0.0008) [2023-12-27 12:12:40,667][20347] Updated weights for policy 0, policy_version 947440 (0.0009) [2023-12-27 12:12:40,719][20347] Updated weights for policy 0, policy_version 947450 (0.0008) [2023-12-27 12:12:40,859][20381] Updated weights for policy 1, policy_version 946562 (0.0005) [2023-12-27 12:12:40,910][20381] Updated weights for policy 1, policy_version 946572 (0.0006) [2023-12-27 12:12:40,967][20381] Updated weights for policy 1, policy_version 946582 (0.0008) [2023-12-27 12:12:41,502][20347] Updated weights for policy 0, policy_version 947460 (0.0008) [2023-12-27 12:12:41,559][20347] Updated weights for policy 0, policy_version 947470 (0.0008) [2023-12-27 12:12:41,618][20347] Updated weights for policy 0, policy_version 947480 (0.0008) [2023-12-27 12:12:41,666][20381] Updated weights for policy 1, policy_version 946592 (0.0008) [2023-12-27 12:12:41,728][20381] Updated weights for policy 1, policy_version 946602 (0.0006) [2023-12-27 12:12:41,797][20381] Updated weights for policy 1, policy_version 946612 (0.0008) [2023-12-27 12:12:42,357][20347] Updated weights for policy 0, policy_version 947490 (0.0008) [2023-12-27 12:12:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 484958208. Throughput: 0: 10248.3, 1: 10419.9. Samples: 484953380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:42,363][19170] Avg episode reward: [(0, '941.450'), (1, '669.480')] [2023-12-27 12:12:42,426][20347] Updated weights for policy 0, policy_version 947500 (0.0008) [2023-12-27 12:12:42,480][20381] Updated weights for policy 1, policy_version 946622 (0.0007) [2023-12-27 12:12:42,485][20347] Updated weights for policy 0, policy_version 947510 (0.0007) [2023-12-27 12:12:42,539][20381] Updated weights for policy 1, policy_version 946632 (0.0008) [2023-12-27 12:12:42,545][20347] Updated weights for policy 0, policy_version 947520 (0.0006) [2023-12-27 12:12:42,596][20381] Updated weights for policy 1, policy_version 946642 (0.0008) [2023-12-27 12:12:43,150][20347] Updated weights for policy 0, policy_version 947530 (0.0005) [2023-12-27 12:12:43,214][20347] Updated weights for policy 0, policy_version 947540 (0.0007) [2023-12-27 12:12:43,272][20347] Updated weights for policy 0, policy_version 947550 (0.0008) [2023-12-27 12:12:43,321][20381] Updated weights for policy 1, policy_version 946652 (0.0008) [2023-12-27 12:12:43,375][20381] Updated weights for policy 1, policy_version 946662 (0.0008) [2023-12-27 12:12:43,424][20381] Updated weights for policy 1, policy_version 946672 (0.0005) [2023-12-27 12:12:43,937][20347] Updated weights for policy 0, policy_version 947560 (0.0009) [2023-12-27 12:12:43,994][20347] Updated weights for policy 0, policy_version 947570 (0.0010) [2023-12-27 12:12:44,050][20347] Updated weights for policy 0, policy_version 947580 (0.0010) [2023-12-27 12:12:44,109][20381] Updated weights for policy 1, policy_version 946682 (0.0006) [2023-12-27 12:12:44,155][20381] Updated weights for policy 1, policy_version 946692 (0.0009) [2023-12-27 12:12:44,202][20381] Updated weights for policy 1, policy_version 946702 (0.0007) [2023-12-27 12:12:44,255][20381] Updated weights for policy 1, policy_version 946712 (0.0008) [2023-12-27 12:12:44,649][20347] Updated weights for policy 0, policy_version 947590 (0.0006) [2023-12-27 12:12:44,706][20347] Updated weights for policy 0, policy_version 947600 (0.0007) [2023-12-27 12:12:44,756][20347] Updated weights for policy 0, policy_version 947610 (0.0007) [2023-12-27 12:12:45,044][20381] Updated weights for policy 1, policy_version 946722 (0.0011) [2023-12-27 12:12:45,107][20381] Updated weights for policy 1, policy_version 946732 (0.0011) [2023-12-27 12:12:45,164][20381] Updated weights for policy 1, policy_version 946742 (0.0011) [2023-12-27 12:12:45,491][20347] Updated weights for policy 0, policy_version 947620 (0.0009) [2023-12-27 12:12:45,543][20347] Updated weights for policy 0, policy_version 947630 (0.0010) [2023-12-27 12:12:45,591][20347] Updated weights for policy 0, policy_version 947640 (0.0010) [2023-12-27 12:12:45,821][20381] Updated weights for policy 1, policy_version 946752 (0.0006) [2023-12-27 12:12:45,868][20381] Updated weights for policy 1, policy_version 946762 (0.0006) [2023-12-27 12:12:45,915][20381] Updated weights for policy 1, policy_version 946772 (0.0010) [2023-12-27 12:12:46,306][20347] Updated weights for policy 0, policy_version 947650 (0.0010) [2023-12-27 12:12:46,364][20347] Updated weights for policy 0, policy_version 947660 (0.0010) [2023-12-27 12:12:46,404][20291] KL-divergence is very high: 122.6642 [2023-12-27 12:12:46,429][20347] Updated weights for policy 0, policy_version 947670 (0.0010) [2023-12-27 12:12:46,454][20291] KL-divergence is very high: 149.5779 [2023-12-27 12:12:46,478][20381] Updated weights for policy 1, policy_version 946782 (0.0010) [2023-12-27 12:12:46,487][20347] Updated weights for policy 0, policy_version 947680 (0.0010) [2023-12-27 12:12:46,532][20381] Updated weights for policy 1, policy_version 946792 (0.0010) [2023-12-27 12:12:46,589][20381] Updated weights for policy 1, policy_version 946802 (0.0010) [2023-12-27 12:12:47,088][20347] Updated weights for policy 0, policy_version 947690 (0.0005) [2023-12-27 12:12:47,142][20347] Updated weights for policy 0, policy_version 947700 (0.0005) [2023-12-27 12:12:47,200][20347] Updated weights for policy 0, policy_version 947710 (0.0009) [2023-12-27 12:12:47,217][20381] Updated weights for policy 1, policy_version 946812 (0.0008) [2023-12-27 12:12:47,268][20381] Updated weights for policy 1, policy_version 946822 (0.0008) [2023-12-27 12:12:47,323][20381] Updated weights for policy 1, policy_version 946832 (0.0008) [2023-12-27 12:12:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 485064704. Throughput: 0: 10218.4, 1: 10403.9. Samples: 485076944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:47,363][19170] Avg episode reward: [(0, '951.470'), (1, '621.350')] [2023-12-27 12:12:47,806][20347] Updated weights for policy 0, policy_version 947720 (0.0007) [2023-12-27 12:12:47,867][20347] Updated weights for policy 0, policy_version 947730 (0.0008) [2023-12-27 12:12:47,922][20347] Updated weights for policy 0, policy_version 947740 (0.0006) [2023-12-27 12:12:47,930][20381] Updated weights for policy 1, policy_version 946842 (0.0008) [2023-12-27 12:12:47,981][20381] Updated weights for policy 1, policy_version 946852 (0.0007) [2023-12-27 12:12:48,034][20381] Updated weights for policy 1, policy_version 946862 (0.0007) [2023-12-27 12:12:48,091][20381] Updated weights for policy 1, policy_version 946872 (0.0008) [2023-12-27 12:12:48,498][20347] Updated weights for policy 0, policy_version 947750 (0.0007) [2023-12-27 12:12:48,556][20347] Updated weights for policy 0, policy_version 947760 (0.0008) [2023-12-27 12:12:48,596][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-27 12:12:48,816][20381] Updated weights for policy 1, policy_version 946882 (0.0010) [2023-12-27 12:12:48,872][20381] Updated weights for policy 1, policy_version 946892 (0.0011) [2023-12-27 12:12:48,931][20381] Updated weights for policy 1, policy_version 946902 (0.0010) [2023-12-27 12:12:49,315][20347] Updated weights for policy 0, policy_version 947770 (0.0008) [2023-12-27 12:12:49,379][20347] Updated weights for policy 0, policy_version 947780 (0.0008) [2023-12-27 12:12:49,435][20347] Updated weights for policy 0, policy_version 947790 (0.0007) [2023-12-27 12:12:49,494][20347] Updated weights for policy 0, policy_version 947800 (0.0006) [2023-12-27 12:12:49,643][20381] Updated weights for policy 1, policy_version 946912 (0.0009) [2023-12-27 12:12:49,711][20381] Updated weights for policy 1, policy_version 946922 (0.0011) [2023-12-27 12:12:49,770][20381] Updated weights for policy 1, policy_version 946932 (0.0011) [2023-12-27 12:12:50,145][20347] Updated weights for policy 0, policy_version 947810 (0.0007) [2023-12-27 12:12:50,207][20347] Updated weights for policy 0, policy_version 947820 (0.0007) [2023-12-27 12:12:50,259][20347] Updated weights for policy 0, policy_version 947830 (0.0007) [2023-12-27 12:12:50,424][20381] Updated weights for policy 1, policy_version 946942 (0.0006) [2023-12-27 12:12:50,484][20381] Updated weights for policy 1, policy_version 946952 (0.0009) [2023-12-27 12:12:50,538][20381] Updated weights for policy 1, policy_version 946962 (0.0008) [2023-12-27 12:12:50,960][20347] Updated weights for policy 0, policy_version 947840 (0.0008) [2023-12-27 12:12:51,019][20347] Updated weights for policy 0, policy_version 947850 (0.0008) [2023-12-27 12:12:51,084][20347] Updated weights for policy 0, policy_version 947860 (0.0008) [2023-12-27 12:12:51,245][20381] Updated weights for policy 1, policy_version 946972 (0.0008) [2023-12-27 12:12:51,309][20381] Updated weights for policy 1, policy_version 946982 (0.0007) [2023-12-27 12:12:51,377][20381] Updated weights for policy 1, policy_version 946992 (0.0008) [2023-12-27 12:12:51,888][20347] Updated weights for policy 0, policy_version 947870 (0.0008) [2023-12-27 12:12:51,943][20291] KL-divergence is very high: 121.4124 [2023-12-27 12:12:51,947][20347] Updated weights for policy 0, policy_version 947880 (0.0007) [2023-12-27 12:12:52,013][20347] Updated weights for policy 0, policy_version 947890 (0.0008) [2023-12-27 12:12:52,063][20381] Updated weights for policy 1, policy_version 947002 (0.0007) [2023-12-27 12:12:52,120][20381] Updated weights for policy 1, policy_version 947012 (0.0009) [2023-12-27 12:12:52,179][20381] Updated weights for policy 1, policy_version 947022 (0.0009) [2023-12-27 12:12:52,241][20381] Updated weights for policy 1, policy_version 947032 (0.0008) [2023-12-27 12:12:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 485171200. Throughput: 0: 10278.4, 1: 10330.3. Samples: 485139440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:52,363][19170] Avg episode reward: [(0, '951.410'), (1, '397.360')] [2023-12-27 12:12:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000947032_242466816.pth... [2023-12-27 12:12:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000947896_242704384.pth... [2023-12-27 12:12:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000945816_242155520.pth [2023-12-27 12:12:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000946720_242401280.pth [2023-12-27 12:12:52,727][20347] Updated weights for policy 0, policy_version 947900 (0.0006) [2023-12-27 12:12:52,756][20291] KL-divergence is very high: 291.2796 [2023-12-27 12:12:52,777][20347] Updated weights for policy 0, policy_version 947910 (0.0008) [2023-12-27 12:12:52,830][20347] Updated weights for policy 0, policy_version 947920 (0.0008) [2023-12-27 12:12:52,933][20381] Updated weights for policy 1, policy_version 947042 (0.0007) [2023-12-27 12:12:52,984][20381] Updated weights for policy 1, policy_version 947052 (0.0007) [2023-12-27 12:12:53,031][20381] Updated weights for policy 1, policy_version 947062 (0.0007) [2023-12-27 12:12:53,456][20347] Updated weights for policy 0, policy_version 947930 (0.0007) [2023-12-27 12:12:53,513][20347] Updated weights for policy 0, policy_version 947940 (0.0007) [2023-12-27 12:12:53,561][20347] Updated weights for policy 0, policy_version 947950 (0.0007) [2023-12-27 12:12:53,607][20347] Updated weights for policy 0, policy_version 947960 (0.0006) [2023-12-27 12:12:53,608][20381] Updated weights for policy 1, policy_version 947072 (0.0007) [2023-12-27 12:12:53,668][20381] Updated weights for policy 1, policy_version 947082 (0.0005) [2023-12-27 12:12:53,725][20381] Updated weights for policy 1, policy_version 947092 (0.0005) [2023-12-27 12:12:54,249][20347] Updated weights for policy 0, policy_version 947970 (0.0007) [2023-12-27 12:12:54,309][20347] Updated weights for policy 0, policy_version 947980 (0.0007) [2023-12-27 12:12:54,374][20347] Updated weights for policy 0, policy_version 947990 (0.0007) [2023-12-27 12:12:54,418][20381] Updated weights for policy 1, policy_version 947102 (0.0009) [2023-12-27 12:12:54,474][20381] Updated weights for policy 1, policy_version 947112 (0.0011) [2023-12-27 12:12:54,535][20381] Updated weights for policy 1, policy_version 947122 (0.0009) [2023-12-27 12:12:55,062][20347] Updated weights for policy 0, policy_version 948000 (0.0008) [2023-12-27 12:12:55,127][20347] Updated weights for policy 0, policy_version 948010 (0.0008) [2023-12-27 12:12:55,177][20381] Updated weights for policy 1, policy_version 947132 (0.0008) [2023-12-27 12:12:55,184][20347] Updated weights for policy 0, policy_version 948020 (0.0007) [2023-12-27 12:12:55,228][20381] Updated weights for policy 1, policy_version 947142 (0.0007) [2023-12-27 12:12:55,272][20381] Updated weights for policy 1, policy_version 947152 (0.0007) [2023-12-27 12:12:55,893][20381] Updated weights for policy 1, policy_version 947162 (0.0007) [2023-12-27 12:12:55,904][20347] Updated weights for policy 0, policy_version 948030 (0.0008) [2023-12-27 12:12:55,946][20381] Updated weights for policy 1, policy_version 947172 (0.0006) [2023-12-27 12:12:55,959][20347] Updated weights for policy 0, policy_version 948040 (0.0007) [2023-12-27 12:12:55,994][20381] Updated weights for policy 1, policy_version 947182 (0.0010) [2023-12-27 12:12:56,014][20347] Updated weights for policy 0, policy_version 948050 (0.0005) [2023-12-27 12:12:56,053][20381] Updated weights for policy 1, policy_version 947192 (0.0010) [2023-12-27 12:12:56,688][20347] Updated weights for policy 0, policy_version 948060 (0.0008) [2023-12-27 12:12:56,699][20381] Updated weights for policy 1, policy_version 947202 (0.0005) [2023-12-27 12:12:56,742][20347] Updated weights for policy 0, policy_version 948070 (0.0007) [2023-12-27 12:12:56,751][20381] Updated weights for policy 1, policy_version 947212 (0.0005) [2023-12-27 12:12:56,802][20347] Updated weights for policy 0, policy_version 948080 (0.0007) [2023-12-27 12:12:56,806][20381] Updated weights for policy 1, policy_version 947222 (0.0005) [2023-12-27 12:12:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 485269504. Throughput: 0: 10248.9, 1: 10349.5. Samples: 485261808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:12:57,363][19170] Avg episode reward: [(0, '961.090'), (1, '502.520')] [2023-12-27 12:12:57,387][20381] Updated weights for policy 1, policy_version 947232 (0.0009) [2023-12-27 12:12:57,433][20381] Updated weights for policy 1, policy_version 947242 (0.0010) [2023-12-27 12:12:57,489][20381] Updated weights for policy 1, policy_version 947252 (0.0011) [2023-12-27 12:12:57,500][20347] Updated weights for policy 0, policy_version 948090 (0.0008) [2023-12-27 12:12:57,557][20347] Updated weights for policy 0, policy_version 948100 (0.0007) [2023-12-27 12:12:57,609][20347] Updated weights for policy 0, policy_version 948110 (0.0008) [2023-12-27 12:12:57,665][20347] Updated weights for policy 0, policy_version 948120 (0.0008) [2023-12-27 12:12:58,100][20381] Updated weights for policy 1, policy_version 947262 (0.0007) [2023-12-27 12:12:58,148][20381] Updated weights for policy 1, policy_version 947272 (0.0005) [2023-12-27 12:12:58,209][20381] Updated weights for policy 1, policy_version 947282 (0.0007) [2023-12-27 12:12:58,473][20347] Updated weights for policy 0, policy_version 948130 (0.0008) [2023-12-27 12:12:58,537][20347] Updated weights for policy 0, policy_version 948140 (0.0008) [2023-12-27 12:12:58,600][20347] Updated weights for policy 0, policy_version 948150 (0.0007) [2023-12-27 12:12:58,938][20381] Updated weights for policy 1, policy_version 947292 (0.0009) [2023-12-27 12:12:58,999][20381] Updated weights for policy 1, policy_version 947302 (0.0009) [2023-12-27 12:12:59,059][20381] Updated weights for policy 1, policy_version 947312 (0.0008) [2023-12-27 12:12:59,321][20347] Updated weights for policy 0, policy_version 948160 (0.0008) [2023-12-27 12:12:59,385][20347] Updated weights for policy 0, policy_version 948170 (0.0009) [2023-12-27 12:12:59,440][20347] Updated weights for policy 0, policy_version 948180 (0.0006) [2023-12-27 12:12:59,765][20381] Updated weights for policy 1, policy_version 947322 (0.0007) [2023-12-27 12:12:59,812][20381] Updated weights for policy 1, policy_version 947332 (0.0007) [2023-12-27 12:12:59,879][20381] Updated weights for policy 1, policy_version 947342 (0.0009) [2023-12-27 12:12:59,945][20381] Updated weights for policy 1, policy_version 947352 (0.0009) [2023-12-27 12:13:00,097][20347] Updated weights for policy 0, policy_version 948190 (0.0005) [2023-12-27 12:13:00,148][20347] Updated weights for policy 0, policy_version 948200 (0.0005) [2023-12-27 12:13:00,202][20347] Updated weights for policy 0, policy_version 948210 (0.0007) [2023-12-27 12:13:00,578][20381] Updated weights for policy 1, policy_version 947362 (0.0010) [2023-12-27 12:13:00,630][20381] Updated weights for policy 1, policy_version 947372 (0.0011) [2023-12-27 12:13:00,675][20381] Updated weights for policy 1, policy_version 947382 (0.0008) [2023-12-27 12:13:00,769][20347] Updated weights for policy 0, policy_version 948220 (0.0007) [2023-12-27 12:13:00,819][20347] Updated weights for policy 0, policy_version 948230 (0.0005) [2023-12-27 12:13:00,867][20347] Updated weights for policy 0, policy_version 948240 (0.0005) [2023-12-27 12:13:01,387][20381] Updated weights for policy 1, policy_version 947392 (0.0008) [2023-12-27 12:13:01,439][20381] Updated weights for policy 1, policy_version 947402 (0.0008) [2023-12-27 12:13:01,481][20347] Updated weights for policy 0, policy_version 948250 (0.0006) [2023-12-27 12:13:01,486][20381] Updated weights for policy 1, policy_version 947412 (0.0008) [2023-12-27 12:13:01,532][20347] Updated weights for policy 0, policy_version 948260 (0.0007) [2023-12-27 12:13:01,591][20347] Updated weights for policy 0, policy_version 948270 (0.0006) [2023-12-27 12:13:01,658][20347] Updated weights for policy 0, policy_version 948280 (0.0006) [2023-12-27 12:13:02,176][20381] Updated weights for policy 1, policy_version 947422 (0.0009) [2023-12-27 12:13:02,239][20381] Updated weights for policy 1, policy_version 947432 (0.0008) [2023-12-27 12:13:02,303][20381] Updated weights for policy 1, policy_version 947442 (0.0008) [2023-12-27 12:13:02,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20752.9, 300 sec: 20549.4). Total num frames: 485376000. Throughput: 0: 10224.3, 1: 10352.4. Samples: 485385112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:02,364][19170] Avg episode reward: [(0, '960.990'), (1, '576.460')] [2023-12-27 12:13:02,389][20347] Updated weights for policy 0, policy_version 948290 (0.0008) [2023-12-27 12:13:02,441][20347] Updated weights for policy 0, policy_version 948300 (0.0008) [2023-12-27 12:13:02,487][20347] Updated weights for policy 0, policy_version 948310 (0.0008) [2023-12-27 12:13:02,945][20381] Updated weights for policy 1, policy_version 947452 (0.0008) [2023-12-27 12:13:02,996][20381] Updated weights for policy 1, policy_version 947462 (0.0005) [2023-12-27 12:13:03,044][20381] Updated weights for policy 1, policy_version 947472 (0.0005) [2023-12-27 12:13:03,223][20347] Updated weights for policy 0, policy_version 948320 (0.0008) [2023-12-27 12:13:03,279][20347] Updated weights for policy 0, policy_version 948330 (0.0008) [2023-12-27 12:13:03,331][20347] Updated weights for policy 0, policy_version 948340 (0.0006) [2023-12-27 12:13:03,642][20381] Updated weights for policy 1, policy_version 947482 (0.0005) [2023-12-27 12:13:03,686][20381] Updated weights for policy 1, policy_version 947492 (0.0007) [2023-12-27 12:13:03,735][20381] Updated weights for policy 1, policy_version 947502 (0.0006) [2023-12-27 12:13:03,784][20381] Updated weights for policy 1, policy_version 947512 (0.0010) [2023-12-27 12:13:03,931][20347] Updated weights for policy 0, policy_version 948350 (0.0007) [2023-12-27 12:13:03,994][20347] Updated weights for policy 0, policy_version 948360 (0.0008) [2023-12-27 12:13:04,054][20347] Updated weights for policy 0, policy_version 948370 (0.0008) [2023-12-27 12:13:04,411][20381] Updated weights for policy 1, policy_version 947522 (0.0010) [2023-12-27 12:13:04,478][20381] Updated weights for policy 1, policy_version 947532 (0.0010) [2023-12-27 12:13:04,532][20381] Updated weights for policy 1, policy_version 947542 (0.0008) [2023-12-27 12:13:04,767][20347] Updated weights for policy 0, policy_version 948380 (0.0009) [2023-12-27 12:13:04,813][20347] Updated weights for policy 0, policy_version 948390 (0.0010) [2023-12-27 12:13:04,858][20347] Updated weights for policy 0, policy_version 948400 (0.0010) [2023-12-27 12:13:05,086][20381] Updated weights for policy 1, policy_version 947552 (0.0005) [2023-12-27 12:13:05,139][20381] Updated weights for policy 1, policy_version 947562 (0.0005) [2023-12-27 12:13:05,194][20381] Updated weights for policy 1, policy_version 947572 (0.0005) [2023-12-27 12:13:05,476][20347] Updated weights for policy 0, policy_version 948410 (0.0006) [2023-12-27 12:13:05,532][20347] Updated weights for policy 0, policy_version 948420 (0.0009) [2023-12-27 12:13:05,582][20347] Updated weights for policy 0, policy_version 948430 (0.0007) [2023-12-27 12:13:05,630][20347] Updated weights for policy 0, policy_version 948440 (0.0007) [2023-12-27 12:13:05,737][20381] Updated weights for policy 1, policy_version 947582 (0.0008) [2023-12-27 12:13:05,795][20381] Updated weights for policy 1, policy_version 947592 (0.0011) [2023-12-27 12:13:05,850][20381] Updated weights for policy 1, policy_version 947602 (0.0011) [2023-12-27 12:13:06,236][20347] Updated weights for policy 0, policy_version 948450 (0.0005) [2023-12-27 12:13:06,299][20347] Updated weights for policy 0, policy_version 948460 (0.0005) [2023-12-27 12:13:06,362][20347] Updated weights for policy 0, policy_version 948470 (0.0005) [2023-12-27 12:13:06,552][20381] Updated weights for policy 1, policy_version 947612 (0.0011) [2023-12-27 12:13:06,607][20381] Updated weights for policy 1, policy_version 947622 (0.0009) [2023-12-27 12:13:06,660][20381] Updated weights for policy 1, policy_version 947632 (0.0005) [2023-12-27 12:13:06,958][20347] Updated weights for policy 0, policy_version 948480 (0.0005) [2023-12-27 12:13:07,020][20347] Updated weights for policy 0, policy_version 948490 (0.0005) [2023-12-27 12:13:07,071][20347] Updated weights for policy 0, policy_version 948500 (0.0005) [2023-12-27 12:13:07,293][20381] Updated weights for policy 1, policy_version 947642 (0.0009) [2023-12-27 12:13:07,357][20381] Updated weights for policy 1, policy_version 947652 (0.0005) [2023-12-27 12:13:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 485482496. Throughput: 0: 10222.5, 1: 10395.8. Samples: 485449956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:07,363][19170] Avg episode reward: [(0, '970.830'), (1, '682.290')] [2023-12-27 12:13:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000948504_242860032.pth... [2023-12-27 12:13:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000947328_242556928.pth [2023-12-27 12:13:07,418][20381] Updated weights for policy 1, policy_version 947662 (0.0005) [2023-12-27 12:13:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000947672_242630656.pth... [2023-12-27 12:13:07,482][20381] Updated weights for policy 1, policy_version 947672 (0.0005) [2023-12-27 12:13:07,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000946424_242311168.pth [2023-12-27 12:13:07,604][20347] Updated weights for policy 0, policy_version 948510 (0.0007) [2023-12-27 12:13:07,658][20347] Updated weights for policy 0, policy_version 948520 (0.0007) [2023-12-27 12:13:07,714][20347] Updated weights for policy 0, policy_version 948530 (0.0009) [2023-12-27 12:13:08,016][20381] Updated weights for policy 1, policy_version 947682 (0.0007) [2023-12-27 12:13:08,074][20381] Updated weights for policy 1, policy_version 947692 (0.0009) [2023-12-27 12:13:08,132][20381] Updated weights for policy 1, policy_version 947702 (0.0009) [2023-12-27 12:13:08,356][20347] Updated weights for policy 0, policy_version 948540 (0.0008) [2023-12-27 12:13:08,415][20347] Updated weights for policy 0, policy_version 948550 (0.0008) [2023-12-27 12:13:08,478][20347] Updated weights for policy 0, policy_version 948560 (0.0006) [2023-12-27 12:13:08,835][20381] Updated weights for policy 1, policy_version 947712 (0.0008) [2023-12-27 12:13:08,896][20381] Updated weights for policy 1, policy_version 947722 (0.0007) [2023-12-27 12:13:08,944][20381] Updated weights for policy 1, policy_version 947732 (0.0007) [2023-12-27 12:13:09,101][20347] Updated weights for policy 0, policy_version 948570 (0.0007) [2023-12-27 12:13:09,148][20347] Updated weights for policy 0, policy_version 948580 (0.0005) [2023-12-27 12:13:09,205][20347] Updated weights for policy 0, policy_version 948590 (0.0006) [2023-12-27 12:13:09,267][20347] Updated weights for policy 0, policy_version 948600 (0.0008) [2023-12-27 12:13:09,801][20381] Updated weights for policy 1, policy_version 947742 (0.0008) [2023-12-27 12:13:09,868][20381] Updated weights for policy 1, policy_version 947752 (0.0009) [2023-12-27 12:13:09,923][20381] Updated weights for policy 1, policy_version 947762 (0.0008) [2023-12-27 12:13:10,000][20347] Updated weights for policy 0, policy_version 948610 (0.0007) [2023-12-27 12:13:10,065][20347] Updated weights for policy 0, policy_version 948620 (0.0011) [2023-12-27 12:13:10,122][20347] Updated weights for policy 0, policy_version 948630 (0.0011) [2023-12-27 12:13:10,585][20381] Updated weights for policy 1, policy_version 947772 (0.0008) [2023-12-27 12:13:10,649][20381] Updated weights for policy 1, policy_version 947782 (0.0008) [2023-12-27 12:13:10,702][20381] Updated weights for policy 1, policy_version 947792 (0.0010) [2023-12-27 12:13:10,805][20347] Updated weights for policy 0, policy_version 948640 (0.0011) [2023-12-27 12:13:10,851][20347] Updated weights for policy 0, policy_version 948650 (0.0010) [2023-12-27 12:13:10,908][20347] Updated weights for policy 0, policy_version 948660 (0.0011) [2023-12-27 12:13:11,442][20381] Updated weights for policy 1, policy_version 947802 (0.0010) [2023-12-27 12:13:11,501][20381] Updated weights for policy 1, policy_version 947812 (0.0008) [2023-12-27 12:13:11,563][20381] Updated weights for policy 1, policy_version 947822 (0.0008) [2023-12-27 12:13:11,628][20381] Updated weights for policy 1, policy_version 947832 (0.0008) [2023-12-27 12:13:11,633][20347] Updated weights for policy 0, policy_version 948670 (0.0009) [2023-12-27 12:13:11,693][20347] Updated weights for policy 0, policy_version 948680 (0.0008) [2023-12-27 12:13:11,759][20347] Updated weights for policy 0, policy_version 948690 (0.0007) [2023-12-27 12:13:12,362][19170] Fps is (10 sec: 20481.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 485580800. Throughput: 0: 10320.7, 1: 10404.8. Samples: 485576284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:12,363][19170] Avg episode reward: [(0, '970.770'), (1, '736.680')] [2023-12-27 12:13:12,396][20381] Updated weights for policy 1, policy_version 947842 (0.0011) [2023-12-27 12:13:12,414][20347] Updated weights for policy 0, policy_version 948700 (0.0006) [2023-12-27 12:13:12,452][20381] Updated weights for policy 1, policy_version 947852 (0.0008) [2023-12-27 12:13:12,465][20347] Updated weights for policy 0, policy_version 948710 (0.0007) [2023-12-27 12:13:12,512][20381] Updated weights for policy 1, policy_version 947862 (0.0007) [2023-12-27 12:13:12,521][20347] Updated weights for policy 0, policy_version 948720 (0.0007) [2023-12-27 12:13:13,133][20381] Updated weights for policy 1, policy_version 947872 (0.0010) [2023-12-27 12:13:13,189][20381] Updated weights for policy 1, policy_version 947882 (0.0010) [2023-12-27 12:13:13,213][20347] Updated weights for policy 0, policy_version 948730 (0.0007) [2023-12-27 12:13:13,246][20381] Updated weights for policy 1, policy_version 947892 (0.0005) [2023-12-27 12:13:13,266][20347] Updated weights for policy 0, policy_version 948740 (0.0008) [2023-12-27 12:13:13,321][20347] Updated weights for policy 0, policy_version 948750 (0.0008) [2023-12-27 12:13:13,381][20347] Updated weights for policy 0, policy_version 948760 (0.0008) [2023-12-27 12:13:13,839][20381] Updated weights for policy 1, policy_version 947902 (0.0006) [2023-12-27 12:13:13,900][20381] Updated weights for policy 1, policy_version 947912 (0.0005) [2023-12-27 12:13:13,962][20381] Updated weights for policy 1, policy_version 947922 (0.0008) [2023-12-27 12:13:14,064][20347] Updated weights for policy 0, policy_version 948770 (0.0008) [2023-12-27 12:13:14,112][20347] Updated weights for policy 0, policy_version 948780 (0.0007) [2023-12-27 12:13:14,159][20347] Updated weights for policy 0, policy_version 948790 (0.0007) [2023-12-27 12:13:14,546][20381] Updated weights for policy 1, policy_version 947932 (0.0007) [2023-12-27 12:13:14,602][20381] Updated weights for policy 1, policy_version 947942 (0.0007) [2023-12-27 12:13:14,659][20381] Updated weights for policy 1, policy_version 947952 (0.0005) [2023-12-27 12:13:14,860][20347] Updated weights for policy 0, policy_version 948800 (0.0006) [2023-12-27 12:13:14,919][20347] Updated weights for policy 0, policy_version 948810 (0.0008) [2023-12-27 12:13:14,969][20347] Updated weights for policy 0, policy_version 948820 (0.0007) [2023-12-27 12:13:15,334][20381] Updated weights for policy 1, policy_version 947962 (0.0006) [2023-12-27 12:13:15,393][20381] Updated weights for policy 1, policy_version 947972 (0.0008) [2023-12-27 12:13:15,445][20381] Updated weights for policy 1, policy_version 947982 (0.0008) [2023-12-27 12:13:15,498][20381] Updated weights for policy 1, policy_version 947992 (0.0008) [2023-12-27 12:13:15,670][20347] Updated weights for policy 0, policy_version 948830 (0.0007) [2023-12-27 12:13:15,732][20347] Updated weights for policy 0, policy_version 948840 (0.0007) [2023-12-27 12:13:15,793][20347] Updated weights for policy 0, policy_version 948850 (0.0008) [2023-12-27 12:13:16,195][20381] Updated weights for policy 1, policy_version 948002 (0.0008) [2023-12-27 12:13:16,254][20381] Updated weights for policy 1, policy_version 948012 (0.0008) [2023-12-27 12:13:16,310][20381] Updated weights for policy 1, policy_version 948022 (0.0007) [2023-12-27 12:13:16,462][20347] Updated weights for policy 0, policy_version 948860 (0.0008) [2023-12-27 12:13:16,508][20347] Updated weights for policy 0, policy_version 948870 (0.0008) [2023-12-27 12:13:16,563][20347] Updated weights for policy 0, policy_version 948880 (0.0008) [2023-12-27 12:13:17,021][20381] Updated weights for policy 1, policy_version 948032 (0.0008) [2023-12-27 12:13:17,078][20381] Updated weights for policy 1, policy_version 948042 (0.0008) [2023-12-27 12:13:17,137][20381] Updated weights for policy 1, policy_version 948052 (0.0008) [2023-12-27 12:13:17,231][20347] Updated weights for policy 0, policy_version 948890 (0.0010) [2023-12-27 12:13:17,278][20347] Updated weights for policy 0, policy_version 948900 (0.0007) [2023-12-27 12:13:17,325][20347] Updated weights for policy 0, policy_version 948910 (0.0005) [2023-12-27 12:13:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 485687296. Throughput: 0: 10309.4, 1: 10420.4. Samples: 485699012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:17,363][19170] Avg episode reward: [(0, '951.290'), (1, '755.800')] [2023-12-27 12:13:17,375][20347] Updated weights for policy 0, policy_version 948920 (0.0005) [2023-12-27 12:13:17,821][20381] Updated weights for policy 1, policy_version 948062 (0.0008) [2023-12-27 12:13:17,878][20381] Updated weights for policy 1, policy_version 948072 (0.0008) [2023-12-27 12:13:17,938][20381] Updated weights for policy 1, policy_version 948082 (0.0008) [2023-12-27 12:13:18,020][20347] Updated weights for policy 0, policy_version 948930 (0.0007) [2023-12-27 12:13:18,076][20347] Updated weights for policy 0, policy_version 948940 (0.0008) [2023-12-27 12:13:18,130][20347] Updated weights for policy 0, policy_version 948950 (0.0006) [2023-12-27 12:13:18,578][20381] Updated weights for policy 1, policy_version 948092 (0.0008) [2023-12-27 12:13:18,628][20381] Updated weights for policy 1, policy_version 948102 (0.0008) [2023-12-27 12:13:18,685][20381] Updated weights for policy 1, policy_version 948112 (0.0008) [2023-12-27 12:13:18,838][20347] Updated weights for policy 0, policy_version 948960 (0.0007) [2023-12-27 12:13:18,894][20347] Updated weights for policy 0, policy_version 948970 (0.0008) [2023-12-27 12:13:18,950][20347] Updated weights for policy 0, policy_version 948980 (0.0007) [2023-12-27 12:13:19,446][20381] Updated weights for policy 1, policy_version 948122 (0.0007) [2023-12-27 12:13:19,536][20381] Updated weights for policy 1, policy_version 948132 (0.0008) [2023-12-27 12:13:19,596][20381] Updated weights for policy 1, policy_version 948142 (0.0007) [2023-12-27 12:13:19,661][20381] Updated weights for policy 1, policy_version 948152 (0.0008) [2023-12-27 12:13:19,663][20347] Updated weights for policy 0, policy_version 948990 (0.0008) [2023-12-27 12:13:19,727][20347] Updated weights for policy 0, policy_version 949000 (0.0007) [2023-12-27 12:13:19,790][20347] Updated weights for policy 0, policy_version 949010 (0.0007) [2023-12-27 12:13:20,358][20381] Updated weights for policy 1, policy_version 948162 (0.0008) [2023-12-27 12:13:20,415][20381] Updated weights for policy 1, policy_version 948172 (0.0008) [2023-12-27 12:13:20,469][20381] Updated weights for policy 1, policy_version 948182 (0.0009) [2023-12-27 12:13:20,479][20347] Updated weights for policy 0, policy_version 949020 (0.0007) [2023-12-27 12:13:20,524][20347] Updated weights for policy 0, policy_version 949030 (0.0007) [2023-12-27 12:13:20,582][20347] Updated weights for policy 0, policy_version 949040 (0.0008) [2023-12-27 12:13:21,189][20381] Updated weights for policy 1, policy_version 948192 (0.0008) [2023-12-27 12:13:21,246][20381] Updated weights for policy 1, policy_version 948202 (0.0008) [2023-12-27 12:13:21,297][20381] Updated weights for policy 1, policy_version 948212 (0.0008) [2023-12-27 12:13:21,346][20347] Updated weights for policy 0, policy_version 949050 (0.0008) [2023-12-27 12:13:21,412][20347] Updated weights for policy 0, policy_version 949060 (0.0008) [2023-12-27 12:13:21,476][20347] Updated weights for policy 0, policy_version 949070 (0.0007) [2023-12-27 12:13:21,533][20347] Updated weights for policy 0, policy_version 949080 (0.0007) [2023-12-27 12:13:22,037][20381] Updated weights for policy 1, policy_version 948222 (0.0006) [2023-12-27 12:13:22,099][20381] Updated weights for policy 1, policy_version 948232 (0.0008) [2023-12-27 12:13:22,168][20381] Updated weights for policy 1, policy_version 948242 (0.0008) [2023-12-27 12:13:22,251][20347] Updated weights for policy 0, policy_version 949090 (0.0007) [2023-12-27 12:13:22,314][20347] Updated weights for policy 0, policy_version 949100 (0.0008) [2023-12-27 12:13:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 485785600. Throughput: 0: 10243.7, 1: 10391.6. Samples: 485759296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:22,363][19170] Avg episode reward: [(0, '951.350'), (1, '804.470')] [2023-12-27 12:13:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000948248_242778112.pth... [2023-12-27 12:13:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000947032_242466816.pth [2023-12-27 12:13:22,389][20347] Updated weights for policy 0, policy_version 949110 (0.0007) [2023-12-27 12:13:22,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000949112_243015680.pth... [2023-12-27 12:13:22,405][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000947896_242704384.pth [2023-12-27 12:13:22,957][20381] Updated weights for policy 1, policy_version 948252 (0.0008) [2023-12-27 12:13:23,005][20381] Updated weights for policy 1, policy_version 948262 (0.0008) [2023-12-27 12:13:23,055][20292] KL-divergence is very high: 478.0773 [2023-12-27 12:13:23,058][20347] Updated weights for policy 0, policy_version 949120 (0.0006) [2023-12-27 12:13:23,060][20381] Updated weights for policy 1, policy_version 948272 (0.0006) [2023-12-27 12:13:23,097][20292] KL-divergence is very high: 459.0030 [2023-12-27 12:13:23,112][20347] Updated weights for policy 0, policy_version 949130 (0.0007) [2023-12-27 12:13:23,168][20347] Updated weights for policy 0, policy_version 949140 (0.0007) [2023-12-27 12:13:23,785][20381] Updated weights for policy 1, policy_version 948282 (0.0007) [2023-12-27 12:13:23,810][20347] Updated weights for policy 0, policy_version 949150 (0.0006) [2023-12-27 12:13:23,839][20381] Updated weights for policy 1, policy_version 948292 (0.0005) [2023-12-27 12:13:23,856][20347] Updated weights for policy 0, policy_version 949160 (0.0006) [2023-12-27 12:13:23,885][20381] Updated weights for policy 1, policy_version 948302 (0.0006) [2023-12-27 12:13:23,906][20347] Updated weights for policy 0, policy_version 949170 (0.0005) [2023-12-27 12:13:23,934][20381] Updated weights for policy 1, policy_version 948312 (0.0006) [2023-12-27 12:13:24,634][20347] Updated weights for policy 0, policy_version 949180 (0.0005) [2023-12-27 12:13:24,660][20381] Updated weights for policy 1, policy_version 948322 (0.0008) [2023-12-27 12:13:24,697][20347] Updated weights for policy 0, policy_version 949190 (0.0008) [2023-12-27 12:13:24,718][20381] Updated weights for policy 1, policy_version 948332 (0.0009) [2023-12-27 12:13:24,758][20347] Updated weights for policy 0, policy_version 949200 (0.0008) [2023-12-27 12:13:24,774][20381] Updated weights for policy 1, policy_version 948342 (0.0005) [2023-12-27 12:13:25,369][20347] Updated weights for policy 0, policy_version 949210 (0.0007) [2023-12-27 12:13:25,422][20381] Updated weights for policy 1, policy_version 948352 (0.0009) [2023-12-27 12:13:25,424][20347] Updated weights for policy 0, policy_version 949220 (0.0008) [2023-12-27 12:13:25,474][20381] Updated weights for policy 1, policy_version 948362 (0.0010) [2023-12-27 12:13:25,475][20347] Updated weights for policy 0, policy_version 949230 (0.0010) [2023-12-27 12:13:25,529][20381] Updated weights for policy 1, policy_version 948372 (0.0010) [2023-12-27 12:13:25,532][20347] Updated weights for policy 0, policy_version 949240 (0.0008) [2023-12-27 12:13:26,072][20347] Updated weights for policy 0, policy_version 949250 (0.0010) [2023-12-27 12:13:26,131][20347] Updated weights for policy 0, policy_version 949260 (0.0010) [2023-12-27 12:13:26,146][20381] Updated weights for policy 1, policy_version 948382 (0.0007) [2023-12-27 12:13:26,183][20347] Updated weights for policy 0, policy_version 949270 (0.0010) [2023-12-27 12:13:26,195][20381] Updated weights for policy 1, policy_version 948392 (0.0005) [2023-12-27 12:13:26,242][20381] Updated weights for policy 1, policy_version 948402 (0.0005) [2023-12-27 12:13:26,828][20347] Updated weights for policy 0, policy_version 949280 (0.0010) [2023-12-27 12:13:26,876][20347] Updated weights for policy 0, policy_version 949290 (0.0010) [2023-12-27 12:13:26,890][20381] Updated weights for policy 1, policy_version 948412 (0.0007) [2023-12-27 12:13:26,921][20347] Updated weights for policy 0, policy_version 949300 (0.0010) [2023-12-27 12:13:26,944][20381] Updated weights for policy 1, policy_version 948422 (0.0010) [2023-12-27 12:13:26,998][20381] Updated weights for policy 1, policy_version 948432 (0.0010) [2023-12-27 12:13:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 485892096. Throughput: 0: 10291.5, 1: 10316.1. Samples: 485880724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 35.0) [2023-12-27 12:13:27,363][19170] Avg episode reward: [(0, '980.570'), (1, '814.120')] [2023-12-27 12:13:27,553][20347] Updated weights for policy 0, policy_version 949310 (0.0010) [2023-12-27 12:13:27,601][20347] Updated weights for policy 0, policy_version 949320 (0.0010) [2023-12-27 12:13:27,601][20381] Updated weights for policy 1, policy_version 948442 (0.0010) [2023-12-27 12:13:27,651][20347] Updated weights for policy 0, policy_version 949330 (0.0010) [2023-12-27 12:13:27,655][20381] Updated weights for policy 1, policy_version 948452 (0.0008) [2023-12-27 12:13:27,702][20381] Updated weights for policy 1, policy_version 948462 (0.0008) [2023-12-27 12:13:27,756][20381] Updated weights for policy 1, policy_version 948472 (0.0007) [2023-12-27 12:13:28,267][20347] Updated weights for policy 0, policy_version 949340 (0.0010) [2023-12-27 12:13:28,315][20347] Updated weights for policy 0, policy_version 949350 (0.0009) [2023-12-27 12:13:28,374][20347] Updated weights for policy 0, policy_version 949360 (0.0006) [2023-12-27 12:13:28,443][20381] Updated weights for policy 1, policy_version 948482 (0.0007) [2023-12-27 12:13:28,497][20381] Updated weights for policy 1, policy_version 948492 (0.0008) [2023-12-27 12:13:28,555][20381] Updated weights for policy 1, policy_version 948502 (0.0007) [2023-12-27 12:13:29,042][20347] Updated weights for policy 0, policy_version 949370 (0.0010) [2023-12-27 12:13:29,108][20347] Updated weights for policy 0, policy_version 949380 (0.0009) [2023-12-27 12:13:29,171][20347] Updated weights for policy 0, policy_version 949390 (0.0010) [2023-12-27 12:13:29,212][20381] Updated weights for policy 1, policy_version 948512 (0.0006) [2023-12-27 12:13:29,232][20347] Updated weights for policy 0, policy_version 949400 (0.0011) [2023-12-27 12:13:29,280][20381] Updated weights for policy 1, policy_version 948522 (0.0008) [2023-12-27 12:13:29,344][20381] Updated weights for policy 1, policy_version 948532 (0.0008) [2023-12-27 12:13:29,861][20347] Updated weights for policy 0, policy_version 949410 (0.0009) [2023-12-27 12:13:29,924][20347] Updated weights for policy 0, policy_version 949420 (0.0010) [2023-12-27 12:13:29,987][20347] Updated weights for policy 0, policy_version 949430 (0.0008) [2023-12-27 12:13:30,083][20381] Updated weights for policy 1, policy_version 948542 (0.0006) [2023-12-27 12:13:30,146][20381] Updated weights for policy 1, policy_version 948552 (0.0006) [2023-12-27 12:13:30,204][20381] Updated weights for policy 1, policy_version 948562 (0.0008) [2023-12-27 12:13:30,687][20347] Updated weights for policy 0, policy_version 949440 (0.0006) [2023-12-27 12:13:30,730][20347] Updated weights for policy 0, policy_version 949450 (0.0005) [2023-12-27 12:13:30,780][20347] Updated weights for policy 0, policy_version 949460 (0.0007) [2023-12-27 12:13:30,826][20381] Updated weights for policy 1, policy_version 948572 (0.0009) [2023-12-27 12:13:30,871][20381] Updated weights for policy 1, policy_version 948582 (0.0010) [2023-12-27 12:13:30,918][20381] Updated weights for policy 1, policy_version 948592 (0.0010) [2023-12-27 12:13:31,446][20347] Updated weights for policy 0, policy_version 949470 (0.0007) [2023-12-27 12:13:31,507][20347] Updated weights for policy 0, policy_version 949480 (0.0006) [2023-12-27 12:13:31,571][20347] Updated weights for policy 0, policy_version 949490 (0.0005) [2023-12-27 12:13:31,695][20381] Updated weights for policy 1, policy_version 948602 (0.0009) [2023-12-27 12:13:31,764][20381] Updated weights for policy 1, policy_version 948612 (0.0007) [2023-12-27 12:13:31,826][20381] Updated weights for policy 1, policy_version 948622 (0.0006) [2023-12-27 12:13:31,878][20381] Updated weights for policy 1, policy_version 948632 (0.0010) [2023-12-27 12:13:32,209][20347] Updated weights for policy 0, policy_version 949500 (0.0007) [2023-12-27 12:13:32,263][20347] Updated weights for policy 0, policy_version 949510 (0.0007) [2023-12-27 12:13:32,315][20347] Updated weights for policy 0, policy_version 949520 (0.0006) [2023-12-27 12:13:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 485990400. Throughput: 0: 10330.9, 1: 10310.3. Samples: 486005800. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:32,363][19170] Avg episode reward: [(0, '990.300'), (1, '843.530')] [2023-12-27 12:13:32,556][20381] Updated weights for policy 1, policy_version 948642 (0.0011) [2023-12-27 12:13:32,613][20381] Updated weights for policy 1, policy_version 948652 (0.0011) [2023-12-27 12:13:32,666][20381] Updated weights for policy 1, policy_version 948662 (0.0011) [2023-12-27 12:13:33,023][20347] Updated weights for policy 0, policy_version 949530 (0.0007) [2023-12-27 12:13:33,082][20347] Updated weights for policy 0, policy_version 949540 (0.0007) [2023-12-27 12:13:33,133][20347] Updated weights for policy 0, policy_version 949550 (0.0007) [2023-12-27 12:13:33,176][20347] Updated weights for policy 0, policy_version 949560 (0.0007) [2023-12-27 12:13:33,344][20381] Updated weights for policy 1, policy_version 948672 (0.0008) [2023-12-27 12:13:33,394][20381] Updated weights for policy 1, policy_version 948682 (0.0005) [2023-12-27 12:13:33,446][20381] Updated weights for policy 1, policy_version 948692 (0.0005) [2023-12-27 12:13:33,811][20347] Updated weights for policy 0, policy_version 949570 (0.0010) [2023-12-27 12:13:33,856][20347] Updated weights for policy 0, policy_version 949580 (0.0010) [2023-12-27 12:13:33,901][20347] Updated weights for policy 0, policy_version 949590 (0.0010) [2023-12-27 12:13:34,041][20381] Updated weights for policy 1, policy_version 948702 (0.0008) [2023-12-27 12:13:34,092][20381] Updated weights for policy 1, policy_version 948712 (0.0010) [2023-12-27 12:13:34,159][20381] Updated weights for policy 1, policy_version 948722 (0.0010) [2023-12-27 12:13:34,594][20347] Updated weights for policy 0, policy_version 949600 (0.0008) [2023-12-27 12:13:34,651][20347] Updated weights for policy 0, policy_version 949610 (0.0008) [2023-12-27 12:13:34,713][20347] Updated weights for policy 0, policy_version 949620 (0.0008) [2023-12-27 12:13:34,847][20381] Updated weights for policy 1, policy_version 948732 (0.0009) [2023-12-27 12:13:34,905][20381] Updated weights for policy 1, policy_version 948742 (0.0010) [2023-12-27 12:13:34,960][20381] Updated weights for policy 1, policy_version 948752 (0.0010) [2023-12-27 12:13:35,318][20347] Updated weights for policy 0, policy_version 949630 (0.0006) [2023-12-27 12:13:35,343][20291] KL-divergence is very high: 166.5192 [2023-12-27 12:13:35,356][20291] KL-divergence is very high: 159.3703 [2023-12-27 12:13:35,364][20347] Updated weights for policy 0, policy_version 949640 (0.0005) [2023-12-27 12:13:35,421][20347] Updated weights for policy 0, policy_version 949650 (0.0005) [2023-12-27 12:13:35,447][20291] KL-divergence is very high: 296.7490 [2023-12-27 12:13:35,557][20381] Updated weights for policy 1, policy_version 948762 (0.0010) [2023-12-27 12:13:35,608][20381] Updated weights for policy 1, policy_version 948772 (0.0005) [2023-12-27 12:13:35,661][20381] Updated weights for policy 1, policy_version 948782 (0.0005) [2023-12-27 12:13:35,706][20381] Updated weights for policy 1, policy_version 948792 (0.0005) [2023-12-27 12:13:35,950][20347] Updated weights for policy 0, policy_version 949660 (0.0005) [2023-12-27 12:13:35,996][20347] Updated weights for policy 0, policy_version 949670 (0.0005) [2023-12-27 12:13:36,055][20347] Updated weights for policy 0, policy_version 949680 (0.0007) [2023-12-27 12:13:36,338][20381] Updated weights for policy 1, policy_version 948802 (0.0006) [2023-12-27 12:13:36,400][20381] Updated weights for policy 1, policy_version 948812 (0.0011) [2023-12-27 12:13:36,459][20381] Updated weights for policy 1, policy_version 948822 (0.0011) [2023-12-27 12:13:36,734][20347] Updated weights for policy 0, policy_version 949690 (0.0008) [2023-12-27 12:13:36,795][20347] Updated weights for policy 0, policy_version 949700 (0.0007) [2023-12-27 12:13:36,845][20347] Updated weights for policy 0, policy_version 949710 (0.0006) [2023-12-27 12:13:36,906][20347] Updated weights for policy 0, policy_version 949720 (0.0007) [2023-12-27 12:13:37,122][20381] Updated weights for policy 1, policy_version 948832 (0.0011) [2023-12-27 12:13:37,168][20381] Updated weights for policy 1, policy_version 948842 (0.0010) [2023-12-27 12:13:37,243][20381] Updated weights for policy 1, policy_version 948852 (0.0011) [2023-12-27 12:13:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 486105088. Throughput: 0: 10299.8, 1: 10339.8. Samples: 486068224. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:37,364][19170] Avg episode reward: [(0, '980.660'), (1, '863.250')] [2023-12-27 12:13:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000949720_243171328.pth... [2023-12-27 12:13:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000948856_242933760.pth... [2023-12-27 12:13:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000948504_242860032.pth [2023-12-27 12:13:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000947672_242630656.pth [2023-12-27 12:13:37,547][20347] Updated weights for policy 0, policy_version 949730 (0.0007) [2023-12-27 12:13:37,607][20347] Updated weights for policy 0, policy_version 949740 (0.0007) [2023-12-27 12:13:37,655][20347] Updated weights for policy 0, policy_version 949750 (0.0008) [2023-12-27 12:13:37,917][20381] Updated weights for policy 1, policy_version 948862 (0.0009) [2023-12-27 12:13:37,965][20381] Updated weights for policy 1, policy_version 948872 (0.0006) [2023-12-27 12:13:38,019][20381] Updated weights for policy 1, policy_version 948882 (0.0006) [2023-12-27 12:13:38,319][20347] Updated weights for policy 0, policy_version 949760 (0.0007) [2023-12-27 12:13:38,383][20347] Updated weights for policy 0, policy_version 949770 (0.0010) [2023-12-27 12:13:38,444][20347] Updated weights for policy 0, policy_version 949780 (0.0009) [2023-12-27 12:13:38,655][20381] Updated weights for policy 1, policy_version 948892 (0.0007) [2023-12-27 12:13:38,709][20381] Updated weights for policy 1, policy_version 948902 (0.0007) [2023-12-27 12:13:38,758][20381] Updated weights for policy 1, policy_version 948912 (0.0007) [2023-12-27 12:13:39,101][20347] Updated weights for policy 0, policy_version 949790 (0.0009) [2023-12-27 12:13:39,161][20347] Updated weights for policy 0, policy_version 949800 (0.0006) [2023-12-27 12:13:39,215][20347] Updated weights for policy 0, policy_version 949810 (0.0006) [2023-12-27 12:13:39,537][20381] Updated weights for policy 1, policy_version 948922 (0.0007) [2023-12-27 12:13:39,594][20381] Updated weights for policy 1, policy_version 948932 (0.0008) [2023-12-27 12:13:39,660][20381] Updated weights for policy 1, policy_version 948942 (0.0007) [2023-12-27 12:13:39,722][20381] Updated weights for policy 1, policy_version 948952 (0.0008) [2023-12-27 12:13:39,931][20347] Updated weights for policy 0, policy_version 949820 (0.0009) [2023-12-27 12:13:39,994][20347] Updated weights for policy 0, policy_version 949830 (0.0009) [2023-12-27 12:13:40,057][20347] Updated weights for policy 0, policy_version 949840 (0.0009) [2023-12-27 12:13:40,486][20381] Updated weights for policy 1, policy_version 948962 (0.0007) [2023-12-27 12:13:40,537][20381] Updated weights for policy 1, policy_version 948972 (0.0009) [2023-12-27 12:13:40,591][20381] Updated weights for policy 1, policy_version 948982 (0.0007) [2023-12-27 12:13:40,845][20347] Updated weights for policy 0, policy_version 949850 (0.0009) [2023-12-27 12:13:40,891][20291] KL-divergence is very high: 397.4204 [2023-12-27 12:13:40,903][20347] Updated weights for policy 0, policy_version 949860 (0.0008) [2023-12-27 12:13:40,958][20347] Updated weights for policy 0, policy_version 949870 (0.0007) [2023-12-27 12:13:41,014][20347] Updated weights for policy 0, policy_version 949880 (0.0006) [2023-12-27 12:13:41,307][20381] Updated weights for policy 1, policy_version 948992 (0.0008) [2023-12-27 12:13:41,376][20381] Updated weights for policy 1, policy_version 949002 (0.0009) [2023-12-27 12:13:41,435][20381] Updated weights for policy 1, policy_version 949012 (0.0011) [2023-12-27 12:13:41,804][20347] Updated weights for policy 0, policy_version 949890 (0.0006) [2023-12-27 12:13:41,865][20347] Updated weights for policy 0, policy_version 949900 (0.0008) [2023-12-27 12:13:41,925][20347] Updated weights for policy 0, policy_version 949910 (0.0008) [2023-12-27 12:13:42,172][20381] Updated weights for policy 1, policy_version 949022 (0.0008) [2023-12-27 12:13:42,231][20381] Updated weights for policy 1, policy_version 949032 (0.0008) [2023-12-27 12:13:42,294][20381] Updated weights for policy 1, policy_version 949042 (0.0007) [2023-12-27 12:13:42,374][19170] Fps is (10 sec: 21274.4, 60 sec: 20749.0, 300 sec: 20548.6). Total num frames: 486203392. Throughput: 0: 10369.9, 1: 10311.8. Samples: 486192728. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:42,375][19170] Avg episode reward: [(0, '970.920'), (1, '863.160')] [2023-12-27 12:13:42,612][20347] Updated weights for policy 0, policy_version 949920 (0.0006) [2023-12-27 12:13:42,675][20347] Updated weights for policy 0, policy_version 949930 (0.0007) [2023-12-27 12:13:42,737][20347] Updated weights for policy 0, policy_version 949940 (0.0007) [2023-12-27 12:13:42,941][20381] Updated weights for policy 1, policy_version 949052 (0.0008) [2023-12-27 12:13:42,992][20381] Updated weights for policy 1, policy_version 949062 (0.0006) [2023-12-27 12:13:43,050][20381] Updated weights for policy 1, policy_version 949072 (0.0005) [2023-12-27 12:13:43,384][20347] Updated weights for policy 0, policy_version 949950 (0.0007) [2023-12-27 12:13:43,434][20347] Updated weights for policy 0, policy_version 949960 (0.0005) [2023-12-27 12:13:43,490][20347] Updated weights for policy 0, policy_version 949970 (0.0006) [2023-12-27 12:13:43,727][20381] Updated weights for policy 1, policy_version 949082 (0.0006) [2023-12-27 12:13:43,781][20381] Updated weights for policy 1, policy_version 949092 (0.0007) [2023-12-27 12:13:43,830][20381] Updated weights for policy 1, policy_version 949102 (0.0008) [2023-12-27 12:13:43,887][20381] Updated weights for policy 1, policy_version 949112 (0.0008) [2023-12-27 12:13:44,120][20347] Updated weights for policy 0, policy_version 949980 (0.0006) [2023-12-27 12:13:44,171][20347] Updated weights for policy 0, policy_version 949990 (0.0005) [2023-12-27 12:13:44,219][20347] Updated weights for policy 0, policy_version 950000 (0.0007) [2023-12-27 12:13:44,557][20381] Updated weights for policy 1, policy_version 949122 (0.0006) [2023-12-27 12:13:44,616][20381] Updated weights for policy 1, policy_version 949132 (0.0006) [2023-12-27 12:13:44,679][20381] Updated weights for policy 1, policy_version 949142 (0.0005) [2023-12-27 12:13:44,924][20347] Updated weights for policy 0, policy_version 950010 (0.0007) [2023-12-27 12:13:44,978][20347] Updated weights for policy 0, policy_version 950020 (0.0008) [2023-12-27 12:13:45,043][20347] Updated weights for policy 0, policy_version 950030 (0.0008) [2023-12-27 12:13:45,103][20347] Updated weights for policy 0, policy_version 950040 (0.0008) [2023-12-27 12:13:45,276][20381] Updated weights for policy 1, policy_version 949152 (0.0007) [2023-12-27 12:13:45,330][20381] Updated weights for policy 1, policy_version 949162 (0.0008) [2023-12-27 12:13:45,380][20381] Updated weights for policy 1, policy_version 949172 (0.0008) [2023-12-27 12:13:45,741][20347] Updated weights for policy 0, policy_version 950050 (0.0007) [2023-12-27 12:13:45,788][20347] Updated weights for policy 0, policy_version 950060 (0.0007) [2023-12-27 12:13:45,834][20347] Updated weights for policy 0, policy_version 950070 (0.0007) [2023-12-27 12:13:46,129][20381] Updated weights for policy 1, policy_version 949182 (0.0009) [2023-12-27 12:13:46,185][20381] Updated weights for policy 1, policy_version 949192 (0.0009) [2023-12-27 12:13:46,241][20381] Updated weights for policy 1, policy_version 949202 (0.0008) [2023-12-27 12:13:46,536][20347] Updated weights for policy 0, policy_version 950080 (0.0007) [2023-12-27 12:13:46,587][20347] Updated weights for policy 0, policy_version 950090 (0.0007) [2023-12-27 12:13:46,639][20347] Updated weights for policy 0, policy_version 950100 (0.0008) [2023-12-27 12:13:46,876][20381] Updated weights for policy 1, policy_version 949212 (0.0008) [2023-12-27 12:13:46,922][20381] Updated weights for policy 1, policy_version 949222 (0.0008) [2023-12-27 12:13:46,973][20381] Updated weights for policy 1, policy_version 949232 (0.0008) [2023-12-27 12:13:47,287][20347] Updated weights for policy 0, policy_version 950110 (0.0006) [2023-12-27 12:13:47,337][20347] Updated weights for policy 0, policy_version 950120 (0.0005) [2023-12-27 12:13:47,362][19170] Fps is (10 sec: 19661.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 486301696. Throughput: 0: 10376.5, 1: 10284.0. Samples: 486314824. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:47,363][19170] Avg episode reward: [(0, '961.150'), (1, '882.750')] [2023-12-27 12:13:47,394][20347] Updated weights for policy 0, policy_version 950130 (0.0007) [2023-12-27 12:13:47,573][20381] Updated weights for policy 1, policy_version 949242 (0.0008) [2023-12-27 12:13:47,618][20381] Updated weights for policy 1, policy_version 949252 (0.0009) [2023-12-27 12:13:47,667][20381] Updated weights for policy 1, policy_version 949262 (0.0009) [2023-12-27 12:13:47,989][20347] Updated weights for policy 0, policy_version 950140 (0.0008) [2023-12-27 12:13:48,037][20347] Updated weights for policy 0, policy_version 950150 (0.0005) [2023-12-27 12:13:48,082][20347] Updated weights for policy 0, policy_version 950160 (0.0006) [2023-12-27 12:13:48,271][20381] Updated weights for policy 1, policy_version 949273 (0.0012) [2023-12-27 12:13:48,324][20381] Updated weights for policy 1, policy_version 949283 (0.0011) [2023-12-27 12:13:48,383][20381] Updated weights for policy 1, policy_version 949293 (0.0010) [2023-12-27 12:13:48,441][20381] Updated weights for policy 1, policy_version 949303 (0.0010) [2023-12-27 12:13:48,728][20347] Updated weights for policy 0, policy_version 950170 (0.0006) [2023-12-27 12:13:48,781][20347] Updated weights for policy 0, policy_version 950180 (0.0010) [2023-12-27 12:13:48,842][20347] Updated weights for policy 0, policy_version 950190 (0.0010) [2023-12-27 12:13:48,854][20291] KL-divergence is very high: 114.0343 [2023-12-27 12:13:48,878][20291] KL-divergence is very high: 196.8437 [2023-12-27 12:13:48,904][20347] Updated weights for policy 0, policy_version 950200 (0.0010) [2023-12-27 12:13:49,157][20381] Updated weights for policy 1, policy_version 949313 (0.0008) [2023-12-27 12:13:49,225][20381] Updated weights for policy 1, policy_version 949323 (0.0008) [2023-12-27 12:13:49,288][20381] Updated weights for policy 1, policy_version 949333 (0.0008) [2023-12-27 12:13:49,569][20347] Updated weights for policy 0, policy_version 950210 (0.0005) [2023-12-27 12:13:49,634][20347] Updated weights for policy 0, policy_version 950220 (0.0007) [2023-12-27 12:13:49,701][20347] Updated weights for policy 0, policy_version 950230 (0.0008) [2023-12-27 12:13:50,032][20381] Updated weights for policy 1, policy_version 949343 (0.0008) [2023-12-27 12:13:50,088][20381] Updated weights for policy 1, policy_version 949353 (0.0008) [2023-12-27 12:13:50,143][20381] Updated weights for policy 1, policy_version 949363 (0.0008) [2023-12-27 12:13:50,322][20347] Updated weights for policy 0, policy_version 950240 (0.0006) [2023-12-27 12:13:50,390][20347] Updated weights for policy 0, policy_version 950250 (0.0006) [2023-12-27 12:13:50,455][20347] Updated weights for policy 0, policy_version 950260 (0.0006) [2023-12-27 12:13:50,870][20381] Updated weights for policy 1, policy_version 949373 (0.0009) [2023-12-27 12:13:50,937][20381] Updated weights for policy 1, policy_version 949383 (0.0010) [2023-12-27 12:13:50,994][20381] Updated weights for policy 1, policy_version 949393 (0.0009) [2023-12-27 12:13:51,145][20347] Updated weights for policy 0, policy_version 950270 (0.0006) [2023-12-27 12:13:51,207][20347] Updated weights for policy 0, policy_version 950280 (0.0006) [2023-12-27 12:13:51,269][20347] Updated weights for policy 0, policy_version 950290 (0.0006) [2023-12-27 12:13:51,721][20381] Updated weights for policy 1, policy_version 949403 (0.0010) [2023-12-27 12:13:51,777][20381] Updated weights for policy 1, policy_version 949413 (0.0010) [2023-12-27 12:13:51,829][20381] Updated weights for policy 1, policy_version 949423 (0.0006) [2023-12-27 12:13:52,016][20347] Updated weights for policy 0, policy_version 950300 (0.0008) [2023-12-27 12:13:52,074][20347] Updated weights for policy 0, policy_version 950310 (0.0007) [2023-12-27 12:13:52,124][20347] Updated weights for policy 0, policy_version 950320 (0.0009) [2023-12-27 12:13:52,362][19170] Fps is (10 sec: 20503.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 486408192. Throughput: 0: 10419.8, 1: 10212.5. Samples: 486378412. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:52,363][19170] Avg episode reward: [(0, '922.270'), (1, '941.420')] [2023-12-27 12:13:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000949432_243081216.pth... [2023-12-27 12:13:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000950328_243326976.pth... [2023-12-27 12:13:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000948248_242778112.pth [2023-12-27 12:13:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000949112_243015680.pth [2023-12-27 12:13:52,445][20381] Updated weights for policy 1, policy_version 949433 (0.0006) [2023-12-27 12:13:52,502][20381] Updated weights for policy 1, policy_version 949443 (0.0008) [2023-12-27 12:13:52,550][20381] Updated weights for policy 1, policy_version 949453 (0.0008) [2023-12-27 12:13:52,608][20381] Updated weights for policy 1, policy_version 949463 (0.0008) [2023-12-27 12:13:52,848][20347] Updated weights for policy 0, policy_version 950330 (0.0007) [2023-12-27 12:13:52,910][20347] Updated weights for policy 0, policy_version 950340 (0.0007) [2023-12-27 12:13:52,967][20347] Updated weights for policy 0, policy_version 950350 (0.0005) [2023-12-27 12:13:53,027][20347] Updated weights for policy 0, policy_version 950360 (0.0007) [2023-12-27 12:13:53,385][20381] Updated weights for policy 1, policy_version 949473 (0.0008) [2023-12-27 12:13:53,437][20381] Updated weights for policy 1, policy_version 949483 (0.0006) [2023-12-27 12:13:53,495][20381] Updated weights for policy 1, policy_version 949493 (0.0008) [2023-12-27 12:13:53,608][20347] Updated weights for policy 0, policy_version 950370 (0.0005) [2023-12-27 12:13:53,656][20347] Updated weights for policy 0, policy_version 950380 (0.0007) [2023-12-27 12:13:53,708][20347] Updated weights for policy 0, policy_version 950390 (0.0007) [2023-12-27 12:13:54,178][20381] Updated weights for policy 1, policy_version 949503 (0.0006) [2023-12-27 12:13:54,227][20381] Updated weights for policy 1, policy_version 949513 (0.0009) [2023-12-27 12:13:54,280][20381] Updated weights for policy 1, policy_version 949523 (0.0009) [2023-12-27 12:13:54,322][20347] Updated weights for policy 0, policy_version 950400 (0.0006) [2023-12-27 12:13:54,371][20347] Updated weights for policy 0, policy_version 950410 (0.0006) [2023-12-27 12:13:54,438][20347] Updated weights for policy 0, policy_version 950420 (0.0008) [2023-12-27 12:13:54,972][20381] Updated weights for policy 1, policy_version 949533 (0.0008) [2023-12-27 12:13:55,041][20381] Updated weights for policy 1, policy_version 949543 (0.0008) [2023-12-27 12:13:55,105][20381] Updated weights for policy 1, policy_version 949553 (0.0007) [2023-12-27 12:13:55,158][20347] Updated weights for policy 0, policy_version 950430 (0.0006) [2023-12-27 12:13:55,213][20347] Updated weights for policy 0, policy_version 950440 (0.0007) [2023-12-27 12:13:55,265][20347] Updated weights for policy 0, policy_version 950450 (0.0008) [2023-12-27 12:13:55,760][20381] Updated weights for policy 1, policy_version 949563 (0.0007) [2023-12-27 12:13:55,805][20381] Updated weights for policy 1, policy_version 949573 (0.0007) [2023-12-27 12:13:55,852][20381] Updated weights for policy 1, policy_version 949583 (0.0008) [2023-12-27 12:13:55,978][20347] Updated weights for policy 0, policy_version 950460 (0.0008) [2023-12-27 12:13:56,024][20347] Updated weights for policy 0, policy_version 950470 (0.0007) [2023-12-27 12:13:56,086][20347] Updated weights for policy 0, policy_version 950480 (0.0007) [2023-12-27 12:13:56,510][20381] Updated weights for policy 1, policy_version 949593 (0.0008) [2023-12-27 12:13:56,577][20381] Updated weights for policy 1, policy_version 949603 (0.0010) [2023-12-27 12:13:56,631][20381] Updated weights for policy 1, policy_version 949613 (0.0010) [2023-12-27 12:13:56,682][20381] Updated weights for policy 1, policy_version 949623 (0.0010) [2023-12-27 12:13:56,774][20347] Updated weights for policy 0, policy_version 950490 (0.0008) [2023-12-27 12:13:56,821][20347] Updated weights for policy 0, policy_version 950500 (0.0008) [2023-12-27 12:13:56,868][20347] Updated weights for policy 0, policy_version 950510 (0.0007) [2023-12-27 12:13:56,926][20347] Updated weights for policy 0, policy_version 950520 (0.0008) [2023-12-27 12:13:57,246][20381] Updated weights for policy 1, policy_version 949633 (0.0006) [2023-12-27 12:13:57,293][20381] Updated weights for policy 1, policy_version 949643 (0.0005) [2023-12-27 12:13:57,353][20381] Updated weights for policy 1, policy_version 949653 (0.0007) [2023-12-27 12:13:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 486506496. Throughput: 0: 10351.7, 1: 10190.0. Samples: 486500664. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:13:57,363][19170] Avg episode reward: [(0, '903.420'), (1, '913.170')] [2023-12-27 12:13:57,609][20347] Updated weights for policy 0, policy_version 950530 (0.0005) [2023-12-27 12:13:57,667][20347] Updated weights for policy 0, policy_version 950540 (0.0006) [2023-12-27 12:13:57,719][20347] Updated weights for policy 0, policy_version 950550 (0.0008) [2023-12-27 12:13:57,887][20381] Updated weights for policy 1, policy_version 949663 (0.0006) [2023-12-27 12:13:57,932][20381] Updated weights for policy 1, policy_version 949673 (0.0005) [2023-12-27 12:13:57,979][20381] Updated weights for policy 1, policy_version 949683 (0.0005) [2023-12-27 12:13:58,434][20347] Updated weights for policy 0, policy_version 950560 (0.0006) [2023-12-27 12:13:58,504][20347] Updated weights for policy 0, policy_version 950570 (0.0009) [2023-12-27 12:13:58,572][20347] Updated weights for policy 0, policy_version 950580 (0.0010) [2023-12-27 12:13:58,608][20381] Updated weights for policy 1, policy_version 949693 (0.0007) [2023-12-27 12:13:58,674][20381] Updated weights for policy 1, policy_version 949703 (0.0010) [2023-12-27 12:13:58,732][20381] Updated weights for policy 1, policy_version 949713 (0.0011) [2023-12-27 12:13:59,321][20347] Updated weights for policy 0, policy_version 950590 (0.0009) [2023-12-27 12:13:59,388][20347] Updated weights for policy 0, policy_version 950600 (0.0008) [2023-12-27 12:13:59,442][20347] Updated weights for policy 0, policy_version 950610 (0.0007) [2023-12-27 12:13:59,526][20381] Updated weights for policy 1, policy_version 949723 (0.0009) [2023-12-27 12:13:59,576][20381] Updated weights for policy 1, policy_version 949733 (0.0008) [2023-12-27 12:13:59,631][20381] Updated weights for policy 1, policy_version 949743 (0.0007) [2023-12-27 12:14:00,095][20347] Updated weights for policy 0, policy_version 950620 (0.0008) [2023-12-27 12:14:00,147][20347] Updated weights for policy 0, policy_version 950630 (0.0007) [2023-12-27 12:14:00,198][20347] Updated weights for policy 0, policy_version 950640 (0.0007) [2023-12-27 12:14:00,337][20381] Updated weights for policy 1, policy_version 949753 (0.0007) [2023-12-27 12:14:00,392][20381] Updated weights for policy 1, policy_version 949763 (0.0007) [2023-12-27 12:14:00,448][20381] Updated weights for policy 1, policy_version 949773 (0.0008) [2023-12-27 12:14:00,500][20381] Updated weights for policy 1, policy_version 949783 (0.0007) [2023-12-27 12:14:00,925][20347] Updated weights for policy 0, policy_version 950650 (0.0008) [2023-12-27 12:14:00,983][20347] Updated weights for policy 0, policy_version 950660 (0.0006) [2023-12-27 12:14:01,044][20347] Updated weights for policy 0, policy_version 950670 (0.0008) [2023-12-27 12:14:01,105][20347] Updated weights for policy 0, policy_version 950680 (0.0008) [2023-12-27 12:14:01,173][20381] Updated weights for policy 1, policy_version 949793 (0.0008) [2023-12-27 12:14:01,234][20381] Updated weights for policy 1, policy_version 949803 (0.0008) [2023-12-27 12:14:01,295][20381] Updated weights for policy 1, policy_version 949813 (0.0007) [2023-12-27 12:14:01,805][20347] Updated weights for policy 0, policy_version 950690 (0.0008) [2023-12-27 12:14:01,864][20347] Updated weights for policy 0, policy_version 950700 (0.0008) [2023-12-27 12:14:01,921][20347] Updated weights for policy 0, policy_version 950710 (0.0006) [2023-12-27 12:14:02,030][20381] Updated weights for policy 1, policy_version 949823 (0.0010) [2023-12-27 12:14:02,082][20381] Updated weights for policy 1, policy_version 949833 (0.0010) [2023-12-27 12:14:02,138][20381] Updated weights for policy 1, policy_version 949843 (0.0010) [2023-12-27 12:14:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.7, 300 sec: 20549.4). Total num frames: 486612992. Throughput: 0: 10304.5, 1: 10197.9. Samples: 486621624. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:02,363][19170] Avg episode reward: [(0, '952.340'), (1, '893.700')] [2023-12-27 12:14:02,563][20347] Updated weights for policy 0, policy_version 950720 (0.0006) [2023-12-27 12:14:02,620][20347] Updated weights for policy 0, policy_version 950730 (0.0005) [2023-12-27 12:14:02,668][20347] Updated weights for policy 0, policy_version 950740 (0.0006) [2023-12-27 12:14:02,755][20381] Updated weights for policy 1, policy_version 949853 (0.0010) [2023-12-27 12:14:02,803][20381] Updated weights for policy 1, policy_version 949863 (0.0007) [2023-12-27 12:14:02,856][20381] Updated weights for policy 1, policy_version 949873 (0.0009) [2023-12-27 12:14:03,367][20347] Updated weights for policy 0, policy_version 950750 (0.0007) [2023-12-27 12:14:03,419][20347] Updated weights for policy 0, policy_version 950760 (0.0007) [2023-12-27 12:14:03,468][20347] Updated weights for policy 0, policy_version 950770 (0.0006) [2023-12-27 12:14:03,483][20381] Updated weights for policy 1, policy_version 949883 (0.0008) [2023-12-27 12:14:03,538][20381] Updated weights for policy 1, policy_version 949893 (0.0010) [2023-12-27 12:14:03,585][20381] Updated weights for policy 1, policy_version 949903 (0.0010) [2023-12-27 12:14:04,114][20347] Updated weights for policy 0, policy_version 950780 (0.0006) [2023-12-27 12:14:04,170][20347] Updated weights for policy 0, policy_version 950790 (0.0007) [2023-12-27 12:14:04,232][20347] Updated weights for policy 0, policy_version 950800 (0.0007) [2023-12-27 12:14:04,265][20381] Updated weights for policy 1, policy_version 949913 (0.0010) [2023-12-27 12:14:04,317][20381] Updated weights for policy 1, policy_version 949923 (0.0010) [2023-12-27 12:14:04,370][20381] Updated weights for policy 1, policy_version 949933 (0.0011) [2023-12-27 12:14:04,425][20381] Updated weights for policy 1, policy_version 949943 (0.0010) [2023-12-27 12:14:04,840][20347] Updated weights for policy 0, policy_version 950810 (0.0006) [2023-12-27 12:14:04,895][20347] Updated weights for policy 0, policy_version 950820 (0.0005) [2023-12-27 12:14:04,952][20347] Updated weights for policy 0, policy_version 950830 (0.0007) [2023-12-27 12:14:05,003][20347] Updated weights for policy 0, policy_version 950840 (0.0007) [2023-12-27 12:14:05,102][20381] Updated weights for policy 1, policy_version 949953 (0.0011) [2023-12-27 12:14:05,160][20381] Updated weights for policy 1, policy_version 949963 (0.0010) [2023-12-27 12:14:05,219][20381] Updated weights for policy 1, policy_version 949973 (0.0010) [2023-12-27 12:14:05,712][20347] Updated weights for policy 0, policy_version 950850 (0.0007) [2023-12-27 12:14:05,764][20291] KL-divergence is very high: 124.5271 [2023-12-27 12:14:05,764][20347] Updated weights for policy 0, policy_version 950860 (0.0006) [2023-12-27 12:14:05,821][20347] Updated weights for policy 0, policy_version 950870 (0.0007) [2023-12-27 12:14:05,831][20381] Updated weights for policy 1, policy_version 949983 (0.0008) [2023-12-27 12:14:05,880][20381] Updated weights for policy 1, policy_version 949993 (0.0009) [2023-12-27 12:14:05,938][20381] Updated weights for policy 1, policy_version 950003 (0.0006) [2023-12-27 12:14:06,508][20381] Updated weights for policy 1, policy_version 950013 (0.0011) [2023-12-27 12:14:06,561][20347] Updated weights for policy 0, policy_version 950880 (0.0008) [2023-12-27 12:14:06,567][20381] Updated weights for policy 1, policy_version 950023 (0.0011) [2023-12-27 12:14:06,614][20347] Updated weights for policy 0, policy_version 950890 (0.0006) [2023-12-27 12:14:06,627][20381] Updated weights for policy 1, policy_version 950033 (0.0011) [2023-12-27 12:14:06,669][20347] Updated weights for policy 0, policy_version 950900 (0.0009) [2023-12-27 12:14:07,309][20381] Updated weights for policy 1, policy_version 950043 (0.0011) [2023-12-27 12:14:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.6). Total num frames: 486711296. Throughput: 0: 10334.4, 1: 10243.1. Samples: 486685284. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:07,363][19170] Avg episode reward: [(0, '952.350'), (1, '931.740')] [2023-12-27 12:14:07,371][20381] Updated weights for policy 1, policy_version 950053 (0.0008) [2023-12-27 12:14:07,391][20347] Updated weights for policy 0, policy_version 950910 (0.0007) [2023-12-27 12:14:07,426][20381] Updated weights for policy 1, policy_version 950063 (0.0010) [2023-12-27 12:14:07,443][20347] Updated weights for policy 0, policy_version 950920 (0.0007) [2023-12-27 12:14:07,475][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000950072_243245056.pth... [2023-12-27 12:14:07,479][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000948856_242933760.pth [2023-12-27 12:14:07,489][20347] Updated weights for policy 0, policy_version 950930 (0.0008) [2023-12-27 12:14:07,521][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000950936_243482624.pth... [2023-12-27 12:14:07,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000949720_243171328.pth [2023-12-27 12:14:08,066][20347] Updated weights for policy 0, policy_version 950940 (0.0006) [2023-12-27 12:14:08,077][20381] Updated weights for policy 1, policy_version 950073 (0.0010) [2023-12-27 12:14:08,119][20347] Updated weights for policy 0, policy_version 950950 (0.0007) [2023-12-27 12:14:08,132][20381] Updated weights for policy 1, policy_version 950083 (0.0005) [2023-12-27 12:14:08,178][20347] Updated weights for policy 0, policy_version 950960 (0.0009) [2023-12-27 12:14:08,187][20381] Updated weights for policy 1, policy_version 950093 (0.0008) [2023-12-27 12:14:08,240][20381] Updated weights for policy 1, policy_version 950103 (0.0007) [2023-12-27 12:14:08,905][20381] Updated weights for policy 1, policy_version 950113 (0.0007) [2023-12-27 12:14:08,923][20347] Updated weights for policy 0, policy_version 950970 (0.0007) [2023-12-27 12:14:08,963][20381] Updated weights for policy 1, policy_version 950123 (0.0009) [2023-12-27 12:14:08,980][20347] Updated weights for policy 0, policy_version 950980 (0.0010) [2023-12-27 12:14:09,016][20381] Updated weights for policy 1, policy_version 950133 (0.0005) [2023-12-27 12:14:09,037][20347] Updated weights for policy 0, policy_version 950990 (0.0010) [2023-12-27 12:14:09,099][20347] Updated weights for policy 0, policy_version 951000 (0.0011) [2023-12-27 12:14:09,736][20381] Updated weights for policy 1, policy_version 950143 (0.0007) [2023-12-27 12:14:09,788][20381] Updated weights for policy 1, policy_version 950153 (0.0008) [2023-12-27 12:14:09,818][20347] Updated weights for policy 0, policy_version 951010 (0.0006) [2023-12-27 12:14:09,848][20381] Updated weights for policy 1, policy_version 950163 (0.0008) [2023-12-27 12:14:09,875][20347] Updated weights for policy 0, policy_version 951020 (0.0009) [2023-12-27 12:14:09,930][20347] Updated weights for policy 0, policy_version 951030 (0.0007) [2023-12-27 12:14:10,527][20381] Updated weights for policy 1, policy_version 950173 (0.0010) [2023-12-27 12:14:10,586][20381] Updated weights for policy 1, policy_version 950183 (0.0011) [2023-12-27 12:14:10,631][20381] Updated weights for policy 1, policy_version 950193 (0.0010) [2023-12-27 12:14:10,682][20347] Updated weights for policy 0, policy_version 951040 (0.0007) [2023-12-27 12:14:10,731][20347] Updated weights for policy 0, policy_version 951050 (0.0008) [2023-12-27 12:14:10,793][20347] Updated weights for policy 0, policy_version 951060 (0.0008) [2023-12-27 12:14:11,334][20381] Updated weights for policy 1, policy_version 950203 (0.0010) [2023-12-27 12:14:11,413][20381] Updated weights for policy 1, policy_version 950213 (0.0008) [2023-12-27 12:14:11,469][20381] Updated weights for policy 1, policy_version 950223 (0.0008) [2023-12-27 12:14:11,564][20347] Updated weights for policy 0, policy_version 951070 (0.0007) [2023-12-27 12:14:11,628][20347] Updated weights for policy 0, policy_version 951080 (0.0008) [2023-12-27 12:14:11,690][20347] Updated weights for policy 0, policy_version 951090 (0.0008) [2023-12-27 12:14:12,199][20381] Updated weights for policy 1, policy_version 950233 (0.0008) [2023-12-27 12:14:12,257][20381] Updated weights for policy 1, policy_version 950243 (0.0010) [2023-12-27 12:14:12,318][20381] Updated weights for policy 1, policy_version 950253 (0.0011) [2023-12-27 12:14:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 486809600. Throughput: 0: 10264.5, 1: 10322.9. Samples: 486807156. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:12,363][19170] Avg episode reward: [(0, '962.120'), (1, '941.470')] [2023-12-27 12:14:12,379][20381] Updated weights for policy 1, policy_version 950263 (0.0011) [2023-12-27 12:14:12,463][20347] Updated weights for policy 0, policy_version 951100 (0.0008) [2023-12-27 12:14:12,522][20347] Updated weights for policy 0, policy_version 951110 (0.0008) [2023-12-27 12:14:12,578][20347] Updated weights for policy 0, policy_version 951120 (0.0008) [2023-12-27 12:14:13,052][20381] Updated weights for policy 1, policy_version 950273 (0.0008) [2023-12-27 12:14:13,108][20381] Updated weights for policy 1, policy_version 950283 (0.0008) [2023-12-27 12:14:13,157][20381] Updated weights for policy 1, policy_version 950293 (0.0005) [2023-12-27 12:14:13,301][20347] Updated weights for policy 0, policy_version 951130 (0.0008) [2023-12-27 12:14:13,352][20347] Updated weights for policy 0, policy_version 951140 (0.0008) [2023-12-27 12:14:13,405][20347] Updated weights for policy 0, policy_version 951150 (0.0008) [2023-12-27 12:14:13,456][20347] Updated weights for policy 0, policy_version 951160 (0.0007) [2023-12-27 12:14:13,720][20381] Updated weights for policy 1, policy_version 950303 (0.0008) [2023-12-27 12:14:13,780][20381] Updated weights for policy 1, policy_version 950313 (0.0009) [2023-12-27 12:14:13,833][20381] Updated weights for policy 1, policy_version 950323 (0.0008) [2023-12-27 12:14:14,070][20347] Updated weights for policy 0, policy_version 951170 (0.0005) [2023-12-27 12:14:14,130][20347] Updated weights for policy 0, policy_version 951180 (0.0007) [2023-12-27 12:14:14,192][20347] Updated weights for policy 0, policy_version 951190 (0.0008) [2023-12-27 12:14:14,537][20381] Updated weights for policy 1, policy_version 950333 (0.0008) [2023-12-27 12:14:14,592][20381] Updated weights for policy 1, policy_version 950343 (0.0007) [2023-12-27 12:14:14,647][20381] Updated weights for policy 1, policy_version 950353 (0.0008) [2023-12-27 12:14:14,863][20347] Updated weights for policy 0, policy_version 951200 (0.0007) [2023-12-27 12:14:14,914][20347] Updated weights for policy 0, policy_version 951210 (0.0007) [2023-12-27 12:14:14,969][20347] Updated weights for policy 0, policy_version 951220 (0.0008) [2023-12-27 12:14:15,379][20381] Updated weights for policy 1, policy_version 950363 (0.0006) [2023-12-27 12:14:15,440][20381] Updated weights for policy 1, policy_version 950373 (0.0005) [2023-12-27 12:14:15,484][20381] Updated weights for policy 1, policy_version 950383 (0.0005) [2023-12-27 12:14:15,523][20292] KL-divergence is very high: 102.8456 [2023-12-27 12:14:15,648][20347] Updated weights for policy 0, policy_version 951230 (0.0008) [2023-12-27 12:14:15,710][20347] Updated weights for policy 0, policy_version 951240 (0.0007) [2023-12-27 12:14:15,762][20347] Updated weights for policy 0, policy_version 951250 (0.0007) [2023-12-27 12:14:16,064][20381] Updated weights for policy 1, policy_version 950393 (0.0006) [2023-12-27 12:14:16,118][20381] Updated weights for policy 1, policy_version 950403 (0.0008) [2023-12-27 12:14:16,176][20381] Updated weights for policy 1, policy_version 950413 (0.0007) [2023-12-27 12:14:16,236][20381] Updated weights for policy 1, policy_version 950423 (0.0008) [2023-12-27 12:14:16,460][20347] Updated weights for policy 0, policy_version 951260 (0.0006) [2023-12-27 12:14:16,519][20347] Updated weights for policy 0, policy_version 951270 (0.0005) [2023-12-27 12:14:16,576][20347] Updated weights for policy 0, policy_version 951280 (0.0007) [2023-12-27 12:14:16,927][20381] Updated weights for policy 1, policy_version 950433 (0.0006) [2023-12-27 12:14:16,980][20381] Updated weights for policy 1, policy_version 950443 (0.0008) [2023-12-27 12:14:17,029][20381] Updated weights for policy 1, policy_version 950453 (0.0008) [2023-12-27 12:14:17,247][20347] Updated weights for policy 0, policy_version 951290 (0.0007) [2023-12-27 12:14:17,295][20347] Updated weights for policy 0, policy_version 951300 (0.0008) [2023-12-27 12:14:17,340][20347] Updated weights for policy 0, policy_version 951310 (0.0009) [2023-12-27 12:14:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 486916096. Throughput: 0: 10192.0, 1: 10310.0. Samples: 486928388. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:17,363][19170] Avg episode reward: [(0, '990.290'), (1, '863.360')] [2023-12-27 12:14:17,389][20347] Updated weights for policy 0, policy_version 951320 (0.0007) [2023-12-27 12:14:17,645][20381] Updated weights for policy 1, policy_version 950463 (0.0008) [2023-12-27 12:14:17,712][20381] Updated weights for policy 1, policy_version 950473 (0.0008) [2023-12-27 12:14:17,768][20381] Updated weights for policy 1, policy_version 950483 (0.0008) [2023-12-27 12:14:18,134][20347] Updated weights for policy 0, policy_version 951330 (0.0008) [2023-12-27 12:14:18,186][20347] Updated weights for policy 0, policy_version 951340 (0.0008) [2023-12-27 12:14:18,233][20347] Updated weights for policy 0, policy_version 951350 (0.0007) [2023-12-27 12:14:18,460][20381] Updated weights for policy 1, policy_version 950493 (0.0007) [2023-12-27 12:14:18,518][20381] Updated weights for policy 1, policy_version 950503 (0.0010) [2023-12-27 12:14:18,567][20381] Updated weights for policy 1, policy_version 950513 (0.0010) [2023-12-27 12:14:18,905][20347] Updated weights for policy 0, policy_version 951360 (0.0008) [2023-12-27 12:14:18,951][20347] Updated weights for policy 0, policy_version 951370 (0.0007) [2023-12-27 12:14:19,013][20347] Updated weights for policy 0, policy_version 951380 (0.0005) [2023-12-27 12:14:19,301][20381] Updated weights for policy 1, policy_version 950523 (0.0009) [2023-12-27 12:14:19,368][20381] Updated weights for policy 1, policy_version 950533 (0.0008) [2023-12-27 12:14:19,431][20381] Updated weights for policy 1, policy_version 950543 (0.0007) [2023-12-27 12:14:19,696][20347] Updated weights for policy 0, policy_version 951390 (0.0006) [2023-12-27 12:14:19,763][20347] Updated weights for policy 0, policy_version 951400 (0.0007) [2023-12-27 12:14:19,829][20347] Updated weights for policy 0, policy_version 951410 (0.0008) [2023-12-27 12:14:20,158][20381] Updated weights for policy 1, policy_version 950553 (0.0008) [2023-12-27 12:14:20,216][20381] Updated weights for policy 1, policy_version 950563 (0.0007) [2023-12-27 12:14:20,277][20381] Updated weights for policy 1, policy_version 950573 (0.0008) [2023-12-27 12:14:20,335][20381] Updated weights for policy 1, policy_version 950583 (0.0008) [2023-12-27 12:14:20,536][20347] Updated weights for policy 0, policy_version 951420 (0.0006) [2023-12-27 12:14:20,602][20347] Updated weights for policy 0, policy_version 951430 (0.0008) [2023-12-27 12:14:20,655][20347] Updated weights for policy 0, policy_version 951440 (0.0008) [2023-12-27 12:14:21,061][20381] Updated weights for policy 1, policy_version 950593 (0.0007) [2023-12-27 12:14:21,118][20381] Updated weights for policy 1, policy_version 950603 (0.0007) [2023-12-27 12:14:21,186][20381] Updated weights for policy 1, policy_version 950613 (0.0006) [2023-12-27 12:14:21,385][20347] Updated weights for policy 0, policy_version 951450 (0.0008) [2023-12-27 12:14:21,438][20347] Updated weights for policy 0, policy_version 951460 (0.0008) [2023-12-27 12:14:21,490][20347] Updated weights for policy 0, policy_version 951470 (0.0008) [2023-12-27 12:14:21,545][20347] Updated weights for policy 0, policy_version 951480 (0.0008) [2023-12-27 12:14:21,936][20381] Updated weights for policy 1, policy_version 950623 (0.0008) [2023-12-27 12:14:22,000][20381] Updated weights for policy 1, policy_version 950633 (0.0008) [2023-12-27 12:14:22,063][20381] Updated weights for policy 1, policy_version 950643 (0.0008) [2023-12-27 12:14:22,322][20347] Updated weights for policy 0, policy_version 951490 (0.0008) [2023-12-27 12:14:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 487014400. Throughput: 0: 10166.9, 1: 10270.4. Samples: 486987896. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:22,364][19170] Avg episode reward: [(0, '961.180'), (1, '892.750')] [2023-12-27 12:14:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000950648_243392512.pth... [2023-12-27 12:14:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000949432_243081216.pth [2023-12-27 12:14:22,396][20347] Updated weights for policy 0, policy_version 951500 (0.0008) [2023-12-27 12:14:22,462][20347] Updated weights for policy 0, policy_version 951510 (0.0006) [2023-12-27 12:14:22,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000951512_243630080.pth... [2023-12-27 12:14:22,480][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000950328_243326976.pth [2023-12-27 12:14:22,804][20381] Updated weights for policy 1, policy_version 950653 (0.0008) [2023-12-27 12:14:22,866][20381] Updated weights for policy 1, policy_version 950663 (0.0008) [2023-12-27 12:14:22,930][20381] Updated weights for policy 1, policy_version 950673 (0.0008) [2023-12-27 12:14:23,144][20347] Updated weights for policy 0, policy_version 951520 (0.0006) [2023-12-27 12:14:23,206][20347] Updated weights for policy 0, policy_version 951530 (0.0007) [2023-12-27 12:14:23,260][20347] Updated weights for policy 0, policy_version 951540 (0.0008) [2023-12-27 12:14:23,575][20381] Updated weights for policy 1, policy_version 950683 (0.0008) [2023-12-27 12:14:23,624][20381] Updated weights for policy 1, policy_version 950693 (0.0006) [2023-12-27 12:14:23,679][20381] Updated weights for policy 1, policy_version 950703 (0.0007) [2023-12-27 12:14:23,950][20347] Updated weights for policy 0, policy_version 951550 (0.0009) [2023-12-27 12:14:24,005][20347] Updated weights for policy 0, policy_version 951560 (0.0010) [2023-12-27 12:14:24,063][20347] Updated weights for policy 0, policy_version 951570 (0.0010) [2023-12-27 12:14:24,381][20381] Updated weights for policy 1, policy_version 950713 (0.0011) [2023-12-27 12:14:24,438][20381] Updated weights for policy 1, policy_version 950723 (0.0007) [2023-12-27 12:14:24,486][20381] Updated weights for policy 1, policy_version 950733 (0.0007) [2023-12-27 12:14:24,540][20381] Updated weights for policy 1, policy_version 950743 (0.0008) [2023-12-27 12:14:24,703][20347] Updated weights for policy 0, policy_version 951580 (0.0009) [2023-12-27 12:14:24,762][20347] Updated weights for policy 0, policy_version 951590 (0.0007) [2023-12-27 12:14:24,815][20347] Updated weights for policy 0, policy_version 951600 (0.0008) [2023-12-27 12:14:25,233][20381] Updated weights for policy 1, policy_version 950753 (0.0010) [2023-12-27 12:14:25,292][20381] Updated weights for policy 1, policy_version 950763 (0.0011) [2023-12-27 12:14:25,352][20381] Updated weights for policy 1, policy_version 950773 (0.0011) [2023-12-27 12:14:25,501][20347] Updated weights for policy 0, policy_version 951610 (0.0007) [2023-12-27 12:14:25,547][20347] Updated weights for policy 0, policy_version 951620 (0.0005) [2023-12-27 12:14:25,601][20347] Updated weights for policy 0, policy_version 951630 (0.0007) [2023-12-27 12:14:25,658][20347] Updated weights for policy 0, policy_version 951640 (0.0005) [2023-12-27 12:14:26,027][20381] Updated weights for policy 1, policy_version 950783 (0.0007) [2023-12-27 12:14:26,087][20381] Updated weights for policy 1, policy_version 950793 (0.0007) [2023-12-27 12:14:26,145][20381] Updated weights for policy 1, policy_version 950803 (0.0011) [2023-12-27 12:14:26,278][20347] Updated weights for policy 0, policy_version 951650 (0.0008) [2023-12-27 12:14:26,338][20347] Updated weights for policy 0, policy_version 951660 (0.0008) [2023-12-27 12:14:26,393][20347] Updated weights for policy 0, policy_version 951670 (0.0007) [2023-12-27 12:14:26,748][20381] Updated weights for policy 1, policy_version 950813 (0.0011) [2023-12-27 12:14:26,799][20381] Updated weights for policy 1, policy_version 950823 (0.0010) [2023-12-27 12:14:26,848][20381] Updated weights for policy 1, policy_version 950833 (0.0010) [2023-12-27 12:14:27,061][20347] Updated weights for policy 0, policy_version 951680 (0.0007) [2023-12-27 12:14:27,131][20347] Updated weights for policy 0, policy_version 951690 (0.0008) [2023-12-27 12:14:27,198][20347] Updated weights for policy 0, policy_version 951700 (0.0008) [2023-12-27 12:14:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 487120896. Throughput: 0: 10113.3, 1: 10223.6. Samples: 487107652. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:27,363][19170] Avg episode reward: [(0, '980.610'), (1, '922.070')] [2023-12-27 12:14:27,522][20381] Updated weights for policy 1, policy_version 950843 (0.0009) [2023-12-27 12:14:27,590][20381] Updated weights for policy 1, policy_version 950853 (0.0006) [2023-12-27 12:14:27,650][20381] Updated weights for policy 1, policy_version 950863 (0.0005) [2023-12-27 12:14:27,775][20347] Updated weights for policy 0, policy_version 951710 (0.0007) [2023-12-27 12:14:27,816][20347] Updated weights for policy 0, policy_version 951720 (0.0007) [2023-12-27 12:14:27,864][20347] Updated weights for policy 0, policy_version 951730 (0.0009) [2023-12-27 12:14:28,209][20381] Updated weights for policy 1, policy_version 950873 (0.0005) [2023-12-27 12:14:28,267][20381] Updated weights for policy 1, policy_version 950883 (0.0005) [2023-12-27 12:14:28,313][20381] Updated weights for policy 1, policy_version 950893 (0.0005) [2023-12-27 12:14:28,371][20381] Updated weights for policy 1, policy_version 950903 (0.0008) [2023-12-27 12:14:28,460][20347] Updated weights for policy 0, policy_version 951740 (0.0008) [2023-12-27 12:14:28,516][20347] Updated weights for policy 0, policy_version 951750 (0.0008) [2023-12-27 12:14:28,570][20347] Updated weights for policy 0, policy_version 951760 (0.0007) [2023-12-27 12:14:28,928][20381] Updated weights for policy 1, policy_version 950913 (0.0005) [2023-12-27 12:14:28,976][20381] Updated weights for policy 1, policy_version 950923 (0.0010) [2023-12-27 12:14:29,021][20381] Updated weights for policy 1, policy_version 950933 (0.0010) [2023-12-27 12:14:29,332][20347] Updated weights for policy 0, policy_version 951770 (0.0008) [2023-12-27 12:14:29,394][20347] Updated weights for policy 0, policy_version 951780 (0.0008) [2023-12-27 12:14:29,455][20347] Updated weights for policy 0, policy_version 951790 (0.0007) [2023-12-27 12:14:29,512][20347] Updated weights for policy 0, policy_version 951800 (0.0007) [2023-12-27 12:14:29,745][20381] Updated weights for policy 1, policy_version 950943 (0.0010) [2023-12-27 12:14:29,791][20381] Updated weights for policy 1, policy_version 950953 (0.0010) [2023-12-27 12:14:29,845][20381] Updated weights for policy 1, policy_version 950963 (0.0012) [2023-12-27 12:14:30,192][20347] Updated weights for policy 0, policy_version 951810 (0.0005) [2023-12-27 12:14:30,239][20347] Updated weights for policy 0, policy_version 951820 (0.0007) [2023-12-27 12:14:30,284][20347] Updated weights for policy 0, policy_version 951830 (0.0005) [2023-12-27 12:14:30,568][20381] Updated weights for policy 1, policy_version 950973 (0.0010) [2023-12-27 12:14:30,620][20381] Updated weights for policy 1, policy_version 950983 (0.0010) [2023-12-27 12:14:30,667][20381] Updated weights for policy 1, policy_version 950993 (0.0010) [2023-12-27 12:14:30,849][20347] Updated weights for policy 0, policy_version 951840 (0.0009) [2023-12-27 12:14:30,904][20347] Updated weights for policy 0, policy_version 951850 (0.0010) [2023-12-27 12:14:30,952][20347] Updated weights for policy 0, policy_version 951860 (0.0010) [2023-12-27 12:14:31,274][20381] Updated weights for policy 1, policy_version 951003 (0.0006) [2023-12-27 12:14:31,334][20381] Updated weights for policy 1, policy_version 951013 (0.0005) [2023-12-27 12:14:31,403][20381] Updated weights for policy 1, policy_version 951023 (0.0009) [2023-12-27 12:14:31,679][20347] Updated weights for policy 0, policy_version 951870 (0.0009) [2023-12-27 12:14:31,743][20347] Updated weights for policy 0, policy_version 951880 (0.0008) [2023-12-27 12:14:31,802][20347] Updated weights for policy 0, policy_version 951890 (0.0009) [2023-12-27 12:14:32,097][20381] Updated weights for policy 1, policy_version 951033 (0.0008) [2023-12-27 12:14:32,160][20381] Updated weights for policy 1, policy_version 951043 (0.0009) [2023-12-27 12:14:32,217][20381] Updated weights for policy 1, policy_version 951053 (0.0010) [2023-12-27 12:14:32,275][20381] Updated weights for policy 1, policy_version 951063 (0.0011) [2023-12-27 12:14:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 487227392. Throughput: 0: 10139.6, 1: 10292.5. Samples: 487234272. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:32,363][19170] Avg episode reward: [(0, '951.400'), (1, '931.720')] [2023-12-27 12:14:32,498][20347] Updated weights for policy 0, policy_version 951900 (0.0007) [2023-12-27 12:14:32,563][20347] Updated weights for policy 0, policy_version 951910 (0.0007) [2023-12-27 12:14:32,624][20347] Updated weights for policy 0, policy_version 951920 (0.0008) [2023-12-27 12:14:32,988][20381] Updated weights for policy 1, policy_version 951073 (0.0008) [2023-12-27 12:14:33,051][20381] Updated weights for policy 1, policy_version 951083 (0.0009) [2023-12-27 12:14:33,103][20381] Updated weights for policy 1, policy_version 951093 (0.0008) [2023-12-27 12:14:33,299][20347] Updated weights for policy 0, policy_version 951930 (0.0008) [2023-12-27 12:14:33,347][20347] Updated weights for policy 0, policy_version 951940 (0.0007) [2023-12-27 12:14:33,394][20347] Updated weights for policy 0, policy_version 951950 (0.0010) [2023-12-27 12:14:33,447][20347] Updated weights for policy 0, policy_version 951960 (0.0009) [2023-12-27 12:14:33,672][20381] Updated weights for policy 1, policy_version 951103 (0.0009) [2023-12-27 12:14:33,720][20381] Updated weights for policy 1, policy_version 951113 (0.0010) [2023-12-27 12:14:33,767][20381] Updated weights for policy 1, policy_version 951123 (0.0010) [2023-12-27 12:14:34,090][20347] Updated weights for policy 0, policy_version 951970 (0.0010) [2023-12-27 12:14:34,142][20347] Updated weights for policy 0, policy_version 951980 (0.0009) [2023-12-27 12:14:34,203][20347] Updated weights for policy 0, policy_version 951990 (0.0008) [2023-12-27 12:14:34,404][20381] Updated weights for policy 1, policy_version 951133 (0.0008) [2023-12-27 12:14:34,465][20381] Updated weights for policy 1, policy_version 951143 (0.0006) [2023-12-27 12:14:34,531][20381] Updated weights for policy 1, policy_version 951153 (0.0007) [2023-12-27 12:14:34,839][20347] Updated weights for policy 0, policy_version 952000 (0.0008) [2023-12-27 12:14:34,889][20347] Updated weights for policy 0, policy_version 952010 (0.0008) [2023-12-27 12:14:34,941][20347] Updated weights for policy 0, policy_version 952020 (0.0007) [2023-12-27 12:14:35,184][20381] Updated weights for policy 1, policy_version 951163 (0.0007) [2023-12-27 12:14:35,246][20381] Updated weights for policy 1, policy_version 951173 (0.0006) [2023-12-27 12:14:35,310][20381] Updated weights for policy 1, policy_version 951183 (0.0006) [2023-12-27 12:14:35,696][20347] Updated weights for policy 0, policy_version 952030 (0.0007) [2023-12-27 12:14:35,758][20347] Updated weights for policy 0, policy_version 952040 (0.0008) [2023-12-27 12:14:35,815][20347] Updated weights for policy 0, policy_version 952050 (0.0005) [2023-12-27 12:14:35,886][20381] Updated weights for policy 1, policy_version 951193 (0.0008) [2023-12-27 12:14:35,941][20381] Updated weights for policy 1, policy_version 951203 (0.0005) [2023-12-27 12:14:35,986][20381] Updated weights for policy 1, policy_version 951213 (0.0007) [2023-12-27 12:14:36,036][20381] Updated weights for policy 1, policy_version 951223 (0.0009) [2023-12-27 12:14:36,457][20347] Updated weights for policy 0, policy_version 952060 (0.0006) [2023-12-27 12:14:36,523][20347] Updated weights for policy 0, policy_version 952070 (0.0007) [2023-12-27 12:14:36,586][20347] Updated weights for policy 0, policy_version 952080 (0.0008) [2023-12-27 12:14:36,690][20381] Updated weights for policy 1, policy_version 951233 (0.0008) [2023-12-27 12:14:36,756][20381] Updated weights for policy 1, policy_version 951243 (0.0008) [2023-12-27 12:14:36,818][20381] Updated weights for policy 1, policy_version 951253 (0.0007) [2023-12-27 12:14:37,277][20347] Updated weights for policy 0, policy_version 952090 (0.0008) [2023-12-27 12:14:37,334][20347] Updated weights for policy 0, policy_version 952100 (0.0008) [2023-12-27 12:14:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.6, 300 sec: 20549.4). Total num frames: 487325696. Throughput: 0: 10110.2, 1: 10331.6. Samples: 487298296. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:37,363][19170] Avg episode reward: [(0, '951.500'), (1, '990.240')] [2023-12-27 12:14:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000951256_243548160.pth... [2023-12-27 12:14:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000950072_243245056.pth [2023-12-27 12:14:37,391][20347] Updated weights for policy 0, policy_version 952110 (0.0008) [2023-12-27 12:14:37,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000952120_243785728.pth... [2023-12-27 12:14:37,447][20347] Updated weights for policy 0, policy_version 952120 (0.0007) [2023-12-27 12:14:37,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000950936_243482624.pth [2023-12-27 12:14:37,493][20381] Updated weights for policy 1, policy_version 951263 (0.0007) [2023-12-27 12:14:37,549][20381] Updated weights for policy 1, policy_version 951273 (0.0005) [2023-12-27 12:14:37,605][20381] Updated weights for policy 1, policy_version 951283 (0.0005) [2023-12-27 12:14:38,138][20347] Updated weights for policy 0, policy_version 952130 (0.0008) [2023-12-27 12:14:38,195][20347] Updated weights for policy 0, policy_version 952140 (0.0007) [2023-12-27 12:14:38,248][20347] Updated weights for policy 0, policy_version 952150 (0.0010) [2023-12-27 12:14:38,279][20381] Updated weights for policy 1, policy_version 951293 (0.0007) [2023-12-27 12:14:38,350][20381] Updated weights for policy 1, policy_version 951303 (0.0007) [2023-12-27 12:14:38,405][20381] Updated weights for policy 1, policy_version 951313 (0.0007) [2023-12-27 12:14:38,893][20347] Updated weights for policy 0, policy_version 952160 (0.0009) [2023-12-27 12:14:38,959][20347] Updated weights for policy 0, policy_version 952170 (0.0011) [2023-12-27 12:14:39,014][20347] Updated weights for policy 0, policy_version 952180 (0.0010) [2023-12-27 12:14:39,079][20381] Updated weights for policy 1, policy_version 951323 (0.0008) [2023-12-27 12:14:39,141][20381] Updated weights for policy 1, policy_version 951333 (0.0007) [2023-12-27 12:14:39,193][20381] Updated weights for policy 1, policy_version 951343 (0.0007) [2023-12-27 12:14:39,673][20347] Updated weights for policy 0, policy_version 952190 (0.0009) [2023-12-27 12:14:39,733][20347] Updated weights for policy 0, policy_version 952200 (0.0007) [2023-12-27 12:14:39,795][20347] Updated weights for policy 0, policy_version 952210 (0.0009) [2023-12-27 12:14:39,908][20381] Updated weights for policy 1, policy_version 951353 (0.0008) [2023-12-27 12:14:39,966][20381] Updated weights for policy 1, policy_version 951363 (0.0008) [2023-12-27 12:14:40,031][20381] Updated weights for policy 1, policy_version 951373 (0.0008) [2023-12-27 12:14:40,100][20381] Updated weights for policy 1, policy_version 951383 (0.0010) [2023-12-27 12:14:40,532][20347] Updated weights for policy 0, policy_version 952220 (0.0010) [2023-12-27 12:14:40,596][20347] Updated weights for policy 0, policy_version 952230 (0.0005) [2023-12-27 12:14:40,659][20347] Updated weights for policy 0, policy_version 952240 (0.0006) [2023-12-27 12:14:40,796][20381] Updated weights for policy 1, policy_version 951393 (0.0008) [2023-12-27 12:14:40,852][20381] Updated weights for policy 1, policy_version 951403 (0.0008) [2023-12-27 12:14:40,904][20381] Updated weights for policy 1, policy_version 951413 (0.0008) [2023-12-27 12:14:41,270][20347] Updated weights for policy 0, policy_version 952250 (0.0008) [2023-12-27 12:14:41,328][20347] Updated weights for policy 0, policy_version 952260 (0.0009) [2023-12-27 12:14:41,398][20347] Updated weights for policy 0, policy_version 952270 (0.0009) [2023-12-27 12:14:41,452][20347] Updated weights for policy 0, policy_version 952280 (0.0006) [2023-12-27 12:14:41,740][20381] Updated weights for policy 1, policy_version 951423 (0.0008) [2023-12-27 12:14:41,793][20381] Updated weights for policy 1, policy_version 951433 (0.0008) [2023-12-27 12:14:41,843][20381] Updated weights for policy 1, policy_version 951443 (0.0008) [2023-12-27 12:14:42,123][20347] Updated weights for policy 0, policy_version 952290 (0.0005) [2023-12-27 12:14:42,179][20347] Updated weights for policy 0, policy_version 952300 (0.0006) [2023-12-27 12:14:42,231][20347] Updated weights for policy 0, policy_version 952310 (0.0006) [2023-12-27 12:14:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20484.0, 300 sec: 20521.6). Total num frames: 487432192. Throughput: 0: 10112.1, 1: 10326.1. Samples: 487420384. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:42,363][19170] Avg episode reward: [(0, '961.270'), (1, '1000.000')] [2023-12-27 12:14:42,579][20381] Updated weights for policy 1, policy_version 951453 (0.0009) [2023-12-27 12:14:42,630][20381] Updated weights for policy 1, policy_version 951463 (0.0011) [2023-12-27 12:14:42,678][20381] Updated weights for policy 1, policy_version 951473 (0.0011) [2023-12-27 12:14:42,973][20347] Updated weights for policy 0, policy_version 952320 (0.0008) [2023-12-27 12:14:43,027][20347] Updated weights for policy 0, policy_version 952330 (0.0008) [2023-12-27 12:14:43,091][20347] Updated weights for policy 0, policy_version 952340 (0.0006) [2023-12-27 12:14:43,374][20381] Updated weights for policy 1, policy_version 951483 (0.0009) [2023-12-27 12:14:43,429][20381] Updated weights for policy 1, policy_version 951493 (0.0006) [2023-12-27 12:14:43,484][20381] Updated weights for policy 1, policy_version 951503 (0.0007) [2023-12-27 12:14:43,731][20347] Updated weights for policy 0, policy_version 952350 (0.0007) [2023-12-27 12:14:43,788][20347] Updated weights for policy 0, policy_version 952360 (0.0006) [2023-12-27 12:14:43,839][20347] Updated weights for policy 0, policy_version 952370 (0.0005) [2023-12-27 12:14:44,120][20381] Updated weights for policy 1, policy_version 951513 (0.0008) [2023-12-27 12:14:44,177][20381] Updated weights for policy 1, policy_version 951523 (0.0011) [2023-12-27 12:14:44,233][20381] Updated weights for policy 1, policy_version 951533 (0.0010) [2023-12-27 12:14:44,290][20381] Updated weights for policy 1, policy_version 951543 (0.0009) [2023-12-27 12:14:44,495][20347] Updated weights for policy 0, policy_version 952380 (0.0008) [2023-12-27 12:14:44,548][20347] Updated weights for policy 0, policy_version 952390 (0.0007) [2023-12-27 12:14:44,596][20347] Updated weights for policy 0, policy_version 952400 (0.0010) [2023-12-27 12:14:44,984][20381] Updated weights for policy 1, policy_version 951553 (0.0010) [2023-12-27 12:14:45,041][20381] Updated weights for policy 1, policy_version 951563 (0.0011) [2023-12-27 12:14:45,094][20381] Updated weights for policy 1, policy_version 951573 (0.0011) [2023-12-27 12:14:45,305][20347] Updated weights for policy 0, policy_version 952410 (0.0010) [2023-12-27 12:14:45,370][20347] Updated weights for policy 0, policy_version 952420 (0.0011) [2023-12-27 12:14:45,382][20291] KL-divergence is very high: 582.5754 [2023-12-27 12:14:45,394][20291] KL-divergence is very high: 233.2600 [2023-12-27 12:14:45,428][20347] Updated weights for policy 0, policy_version 952430 (0.0010) [2023-12-27 12:14:45,480][20347] Updated weights for policy 0, policy_version 952440 (0.0010) [2023-12-27 12:14:45,745][20381] Updated weights for policy 1, policy_version 951583 (0.0008) [2023-12-27 12:14:45,797][20381] Updated weights for policy 1, policy_version 951593 (0.0007) [2023-12-27 12:14:45,852][20381] Updated weights for policy 1, policy_version 951603 (0.0008) [2023-12-27 12:14:46,083][20347] Updated weights for policy 0, policy_version 952450 (0.0007) [2023-12-27 12:14:46,136][20347] Updated weights for policy 0, policy_version 952460 (0.0007) [2023-12-27 12:14:46,190][20347] Updated weights for policy 0, policy_version 952470 (0.0007) [2023-12-27 12:14:46,568][20381] Updated weights for policy 1, policy_version 951613 (0.0010) [2023-12-27 12:14:46,630][20381] Updated weights for policy 1, policy_version 951623 (0.0011) [2023-12-27 12:14:46,688][20381] Updated weights for policy 1, policy_version 951633 (0.0006) [2023-12-27 12:14:46,778][20347] Updated weights for policy 0, policy_version 952480 (0.0007) [2023-12-27 12:14:46,846][20291] KL-divergence is very high: 314.5642 [2023-12-27 12:14:46,846][20347] Updated weights for policy 0, policy_version 952490 (0.0008) [2023-12-27 12:14:46,887][20291] KL-divergence is very high: 193.5529 [2023-12-27 12:14:46,898][20347] Updated weights for policy 0, policy_version 952500 (0.0006) [2023-12-27 12:14:47,313][20381] Updated weights for policy 1, policy_version 951643 (0.0008) [2023-12-27 12:14:47,362][20381] Updated weights for policy 1, policy_version 951653 (0.0010) [2023-12-27 12:14:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 487530496. Throughput: 0: 10194.5, 1: 10303.0. Samples: 487544012. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:47,363][19170] Avg episode reward: [(0, '941.800'), (1, '1000.000')] [2023-12-27 12:14:47,413][20381] Updated weights for policy 1, policy_version 951663 (0.0010) [2023-12-27 12:14:47,487][20347] Updated weights for policy 0, policy_version 952510 (0.0006) [2023-12-27 12:14:47,536][20347] Updated weights for policy 0, policy_version 952520 (0.0009) [2023-12-27 12:14:47,579][20347] Updated weights for policy 0, policy_version 952530 (0.0006) [2023-12-27 12:14:48,093][20381] Updated weights for policy 1, policy_version 951673 (0.0006) [2023-12-27 12:14:48,154][20381] Updated weights for policy 1, policy_version 951683 (0.0010) [2023-12-27 12:14:48,192][20347] Updated weights for policy 0, policy_version 952540 (0.0005) [2023-12-27 12:14:48,203][20381] Updated weights for policy 1, policy_version 951693 (0.0008) [2023-12-27 12:14:48,247][20347] Updated weights for policy 0, policy_version 952550 (0.0006) [2023-12-27 12:14:48,260][20381] Updated weights for policy 1, policy_version 951703 (0.0007) [2023-12-27 12:14:48,300][20347] Updated weights for policy 0, policy_version 952560 (0.0006) [2023-12-27 12:14:48,871][20381] Updated weights for policy 1, policy_version 951713 (0.0006) [2023-12-27 12:14:48,935][20381] Updated weights for policy 1, policy_version 951723 (0.0006) [2023-12-27 12:14:48,995][20381] Updated weights for policy 1, policy_version 951733 (0.0007) [2023-12-27 12:14:49,081][20347] Updated weights for policy 0, policy_version 952570 (0.0008) [2023-12-27 12:14:49,140][20347] Updated weights for policy 0, policy_version 952580 (0.0008) [2023-12-27 12:14:49,197][20347] Updated weights for policy 0, policy_version 952590 (0.0008) [2023-12-27 12:14:49,262][20347] Updated weights for policy 0, policy_version 952600 (0.0007) [2023-12-27 12:14:49,702][20381] Updated weights for policy 1, policy_version 951743 (0.0008) [2023-12-27 12:14:49,761][20381] Updated weights for policy 1, policy_version 951753 (0.0008) [2023-12-27 12:14:49,821][20381] Updated weights for policy 1, policy_version 951763 (0.0008) [2023-12-27 12:14:49,983][20347] Updated weights for policy 0, policy_version 952610 (0.0007) [2023-12-27 12:14:50,038][20347] Updated weights for policy 0, policy_version 952620 (0.0008) [2023-12-27 12:14:50,102][20347] Updated weights for policy 0, policy_version 952630 (0.0007) [2023-12-27 12:14:50,582][20381] Updated weights for policy 1, policy_version 951773 (0.0008) [2023-12-27 12:14:50,642][20381] Updated weights for policy 1, policy_version 951783 (0.0009) [2023-12-27 12:14:50,703][20381] Updated weights for policy 1, policy_version 951793 (0.0008) [2023-12-27 12:14:50,789][20347] Updated weights for policy 0, policy_version 952640 (0.0008) [2023-12-27 12:14:50,844][20347] Updated weights for policy 0, policy_version 952650 (0.0005) [2023-12-27 12:14:50,897][20347] Updated weights for policy 0, policy_version 952660 (0.0007) [2023-12-27 12:14:51,478][20381] Updated weights for policy 1, policy_version 951803 (0.0008) [2023-12-27 12:14:51,538][20381] Updated weights for policy 1, policy_version 951813 (0.0008) [2023-12-27 12:14:51,562][20347] Updated weights for policy 0, policy_version 952670 (0.0007) [2023-12-27 12:14:51,595][20381] Updated weights for policy 1, policy_version 951823 (0.0007) [2023-12-27 12:14:51,609][20347] Updated weights for policy 0, policy_version 952680 (0.0007) [2023-12-27 12:14:51,673][20347] Updated weights for policy 0, policy_version 952690 (0.0007) [2023-12-27 12:14:52,355][20381] Updated weights for policy 1, policy_version 951833 (0.0007) [2023-12-27 12:14:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 487628800. Throughput: 0: 10180.7, 1: 10266.7. Samples: 487605416. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:52,363][19170] Avg episode reward: [(0, '912.480'), (1, '1000.000')] [2023-12-27 12:14:52,377][20347] Updated weights for policy 0, policy_version 952700 (0.0008) [2023-12-27 12:14:52,411][20381] Updated weights for policy 1, policy_version 951843 (0.0006) [2023-12-27 12:14:52,436][20347] Updated weights for policy 0, policy_version 952710 (0.0006) [2023-12-27 12:14:52,471][20381] Updated weights for policy 1, policy_version 951853 (0.0008) [2023-12-27 12:14:52,499][20347] Updated weights for policy 0, policy_version 952720 (0.0005) [2023-12-27 12:14:52,533][20381] Updated weights for policy 1, policy_version 951863 (0.0008) [2023-12-27 12:14:52,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000951864_243703808.pth... [2023-12-27 12:14:52,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000950648_243392512.pth [2023-12-27 12:14:52,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000952728_243941376.pth... [2023-12-27 12:14:52,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000951512_243630080.pth [2023-12-27 12:14:53,172][20347] Updated weights for policy 0, policy_version 952730 (0.0005) [2023-12-27 12:14:53,227][20347] Updated weights for policy 0, policy_version 952740 (0.0005) [2023-12-27 12:14:53,253][20381] Updated weights for policy 1, policy_version 951873 (0.0008) [2023-12-27 12:14:53,279][20347] Updated weights for policy 0, policy_version 952750 (0.0006) [2023-12-27 12:14:53,307][20381] Updated weights for policy 1, policy_version 951883 (0.0008) [2023-12-27 12:14:53,331][20347] Updated weights for policy 0, policy_version 952760 (0.0005) [2023-12-27 12:14:53,361][20381] Updated weights for policy 1, policy_version 951893 (0.0006) [2023-12-27 12:14:53,880][20347] Updated weights for policy 0, policy_version 952770 (0.0007) [2023-12-27 12:14:53,941][20347] Updated weights for policy 0, policy_version 952780 (0.0008) [2023-12-27 12:14:53,996][20347] Updated weights for policy 0, policy_version 952790 (0.0006) [2023-12-27 12:14:54,137][20381] Updated weights for policy 1, policy_version 951903 (0.0008) [2023-12-27 12:14:54,199][20381] Updated weights for policy 1, policy_version 951913 (0.0008) [2023-12-27 12:14:54,255][20381] Updated weights for policy 1, policy_version 951923 (0.0008) [2023-12-27 12:14:54,656][20347] Updated weights for policy 0, policy_version 952800 (0.0005) [2023-12-27 12:14:54,724][20347] Updated weights for policy 0, policy_version 952810 (0.0005) [2023-12-27 12:14:54,795][20347] Updated weights for policy 0, policy_version 952820 (0.0005) [2023-12-27 12:14:54,965][20381] Updated weights for policy 1, policy_version 951933 (0.0008) [2023-12-27 12:14:55,029][20381] Updated weights for policy 1, policy_version 951943 (0.0006) [2023-12-27 12:14:55,093][20381] Updated weights for policy 1, policy_version 951953 (0.0006) [2023-12-27 12:14:55,423][20347] Updated weights for policy 0, policy_version 952830 (0.0007) [2023-12-27 12:14:55,481][20347] Updated weights for policy 0, policy_version 952840 (0.0008) [2023-12-27 12:14:55,534][20347] Updated weights for policy 0, policy_version 952850 (0.0009) [2023-12-27 12:14:55,723][20381] Updated weights for policy 1, policy_version 951963 (0.0007) [2023-12-27 12:14:55,779][20381] Updated weights for policy 1, policy_version 951973 (0.0006) [2023-12-27 12:14:55,843][20381] Updated weights for policy 1, policy_version 951983 (0.0005) [2023-12-27 12:14:56,226][20347] Updated weights for policy 0, policy_version 952860 (0.0008) [2023-12-27 12:14:56,272][20347] Updated weights for policy 0, policy_version 952870 (0.0007) [2023-12-27 12:14:56,316][20347] Updated weights for policy 0, policy_version 952880 (0.0008) [2023-12-27 12:14:56,490][20381] Updated weights for policy 1, policy_version 951993 (0.0005) [2023-12-27 12:14:56,537][20381] Updated weights for policy 1, policy_version 952003 (0.0007) [2023-12-27 12:14:56,580][20381] Updated weights for policy 1, policy_version 952013 (0.0006) [2023-12-27 12:14:56,624][20381] Updated weights for policy 1, policy_version 952023 (0.0005) [2023-12-27 12:14:56,997][20347] Updated weights for policy 0, policy_version 952890 (0.0008) [2023-12-27 12:14:57,051][20347] Updated weights for policy 0, policy_version 952900 (0.0010) [2023-12-27 12:14:57,094][20347] Updated weights for policy 0, policy_version 952910 (0.0007) [2023-12-27 12:14:57,147][20347] Updated weights for policy 0, policy_version 952920 (0.0005) [2023-12-27 12:14:57,199][20381] Updated weights for policy 1, policy_version 952033 (0.0007) [2023-12-27 12:14:57,252][20381] Updated weights for policy 1, policy_version 952043 (0.0007) [2023-12-27 12:14:57,301][20381] Updated weights for policy 1, policy_version 952053 (0.0007) [2023-12-27 12:14:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 487743488. Throughput: 0: 10272.1, 1: 10173.7. Samples: 487727220. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:14:57,364][19170] Avg episode reward: [(0, '872.150'), (1, '1000.000')] [2023-12-27 12:14:57,779][20347] Updated weights for policy 0, policy_version 952930 (0.0010) [2023-12-27 12:14:57,827][20347] Updated weights for policy 0, policy_version 952940 (0.0010) [2023-12-27 12:14:57,869][20381] Updated weights for policy 1, policy_version 952063 (0.0005) [2023-12-27 12:14:57,881][20347] Updated weights for policy 0, policy_version 952950 (0.0010) [2023-12-27 12:14:57,925][20381] Updated weights for policy 1, policy_version 952073 (0.0005) [2023-12-27 12:14:57,985][20381] Updated weights for policy 1, policy_version 952083 (0.0005) [2023-12-27 12:14:58,621][20381] Updated weights for policy 1, policy_version 952093 (0.0007) [2023-12-27 12:14:58,622][20347] Updated weights for policy 0, policy_version 952960 (0.0008) [2023-12-27 12:14:58,682][20347] Updated weights for policy 0, policy_version 952970 (0.0007) [2023-12-27 12:14:58,683][20381] Updated weights for policy 1, policy_version 952103 (0.0009) [2023-12-27 12:14:58,748][20347] Updated weights for policy 0, policy_version 952980 (0.0007) [2023-12-27 12:14:58,750][20381] Updated weights for policy 1, policy_version 952113 (0.0009) [2023-12-27 12:14:59,445][20347] Updated weights for policy 0, policy_version 952990 (0.0008) [2023-12-27 12:14:59,503][20347] Updated weights for policy 0, policy_version 953000 (0.0007) [2023-12-27 12:14:59,523][20381] Updated weights for policy 1, policy_version 952123 (0.0008) [2023-12-27 12:14:59,559][20347] Updated weights for policy 0, policy_version 953010 (0.0008) [2023-12-27 12:14:59,577][20381] Updated weights for policy 1, policy_version 952133 (0.0006) [2023-12-27 12:14:59,630][20381] Updated weights for policy 1, policy_version 952143 (0.0008) [2023-12-27 12:15:00,263][20347] Updated weights for policy 0, policy_version 953020 (0.0008) [2023-12-27 12:15:00,302][20381] Updated weights for policy 1, policy_version 952153 (0.0007) [2023-12-27 12:15:00,317][20347] Updated weights for policy 0, policy_version 953030 (0.0007) [2023-12-27 12:15:00,362][20381] Updated weights for policy 1, policy_version 952163 (0.0007) [2023-12-27 12:15:00,375][20347] Updated weights for policy 0, policy_version 953040 (0.0006) [2023-12-27 12:15:00,418][20381] Updated weights for policy 1, policy_version 952173 (0.0005) [2023-12-27 12:15:00,473][20381] Updated weights for policy 1, policy_version 952183 (0.0008) [2023-12-27 12:15:01,016][20347] Updated weights for policy 0, policy_version 953050 (0.0006) [2023-12-27 12:15:01,076][20347] Updated weights for policy 0, policy_version 953060 (0.0007) [2023-12-27 12:15:01,131][20347] Updated weights for policy 0, policy_version 953070 (0.0007) [2023-12-27 12:15:01,189][20381] Updated weights for policy 1, policy_version 952193 (0.0007) [2023-12-27 12:15:01,195][20347] Updated weights for policy 0, policy_version 953080 (0.0008) [2023-12-27 12:15:01,243][20381] Updated weights for policy 1, policy_version 952203 (0.0007) [2023-12-27 12:15:01,296][20381] Updated weights for policy 1, policy_version 952213 (0.0008) [2023-12-27 12:15:01,960][20347] Updated weights for policy 0, policy_version 953090 (0.0007) [2023-12-27 12:15:01,978][20381] Updated weights for policy 1, policy_version 952223 (0.0008) [2023-12-27 12:15:02,014][20347] Updated weights for policy 0, policy_version 953100 (0.0006) [2023-12-27 12:15:02,028][20381] Updated weights for policy 1, policy_version 952233 (0.0005) [2023-12-27 12:15:02,068][20347] Updated weights for policy 0, policy_version 953110 (0.0006) [2023-12-27 12:15:02,081][20381] Updated weights for policy 1, policy_version 952243 (0.0006) [2023-12-27 12:15:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 487841792. Throughput: 0: 10276.5, 1: 10199.8. Samples: 487849824. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:02,363][19170] Avg episode reward: [(0, '901.340'), (1, '1000.000')] [2023-12-27 12:15:02,720][20381] Updated weights for policy 1, policy_version 952253 (0.0007) [2023-12-27 12:15:02,779][20381] Updated weights for policy 1, policy_version 952263 (0.0005) [2023-12-27 12:15:02,787][20347] Updated weights for policy 0, policy_version 953120 (0.0007) [2023-12-27 12:15:02,838][20381] Updated weights for policy 1, policy_version 952273 (0.0005) [2023-12-27 12:15:02,843][20347] Updated weights for policy 0, policy_version 953130 (0.0008) [2023-12-27 12:15:02,894][20347] Updated weights for policy 0, policy_version 953140 (0.0007) [2023-12-27 12:15:03,506][20347] Updated weights for policy 0, policy_version 953150 (0.0008) [2023-12-27 12:15:03,516][20381] Updated weights for policy 1, policy_version 952283 (0.0007) [2023-12-27 12:15:03,557][20347] Updated weights for policy 0, policy_version 953160 (0.0007) [2023-12-27 12:15:03,564][20381] Updated weights for policy 1, policy_version 952293 (0.0007) [2023-12-27 12:15:03,615][20381] Updated weights for policy 1, policy_version 952303 (0.0005) [2023-12-27 12:15:03,616][20347] Updated weights for policy 0, policy_version 953170 (0.0007) [2023-12-27 12:15:04,272][20347] Updated weights for policy 0, policy_version 953180 (0.0007) [2023-12-27 12:15:04,294][20381] Updated weights for policy 1, policy_version 952313 (0.0006) [2023-12-27 12:15:04,332][20347] Updated weights for policy 0, policy_version 953190 (0.0006) [2023-12-27 12:15:04,356][20381] Updated weights for policy 1, policy_version 952323 (0.0008) [2023-12-27 12:15:04,388][20347] Updated weights for policy 0, policy_version 953200 (0.0006) [2023-12-27 12:15:04,414][20381] Updated weights for policy 1, policy_version 952333 (0.0007) [2023-12-27 12:15:04,480][20381] Updated weights for policy 1, policy_version 952343 (0.0007) [2023-12-27 12:15:04,979][20347] Updated weights for policy 0, policy_version 953210 (0.0006) [2023-12-27 12:15:05,024][20347] Updated weights for policy 0, policy_version 953220 (0.0005) [2023-12-27 12:15:05,072][20347] Updated weights for policy 0, policy_version 953230 (0.0005) [2023-12-27 12:15:05,131][20347] Updated weights for policy 0, policy_version 953240 (0.0007) [2023-12-27 12:15:05,146][20381] Updated weights for policy 1, policy_version 952353 (0.0005) [2023-12-27 12:15:05,207][20381] Updated weights for policy 1, policy_version 952363 (0.0008) [2023-12-27 12:15:05,267][20381] Updated weights for policy 1, policy_version 952373 (0.0008) [2023-12-27 12:15:05,758][20347] Updated weights for policy 0, policy_version 953250 (0.0008) [2023-12-27 12:15:05,802][20347] Updated weights for policy 0, policy_version 953260 (0.0010) [2023-12-27 12:15:05,851][20347] Updated weights for policy 0, policy_version 953270 (0.0010) [2023-12-27 12:15:05,925][20381] Updated weights for policy 1, policy_version 952383 (0.0009) [2023-12-27 12:15:05,974][20381] Updated weights for policy 1, policy_version 952393 (0.0010) [2023-12-27 12:15:06,018][20381] Updated weights for policy 1, policy_version 952403 (0.0010) [2023-12-27 12:15:06,628][20347] Updated weights for policy 0, policy_version 953280 (0.0008) [2023-12-27 12:15:06,691][20347] Updated weights for policy 0, policy_version 953290 (0.0008) [2023-12-27 12:15:06,709][20381] Updated weights for policy 1, policy_version 952413 (0.0008) [2023-12-27 12:15:06,748][20347] Updated weights for policy 0, policy_version 953300 (0.0006) [2023-12-27 12:15:06,765][20381] Updated weights for policy 1, policy_version 952423 (0.0006) [2023-12-27 12:15:06,824][20381] Updated weights for policy 1, policy_version 952433 (0.0007) [2023-12-27 12:15:07,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 487940096. Throughput: 0: 10335.2, 1: 10235.5. Samples: 487913576. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:07,363][19170] Avg episode reward: [(0, '989.500'), (1, '1000.000')] [2023-12-27 12:15:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000952440_243851264.pth... [2023-12-27 12:15:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000953304_244088832.pth... [2023-12-27 12:15:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000951256_243548160.pth [2023-12-27 12:15:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000952120_243785728.pth [2023-12-27 12:15:07,433][20347] Updated weights for policy 0, policy_version 953310 (0.0007) [2023-12-27 12:15:07,474][20381] Updated weights for policy 1, policy_version 952443 (0.0007) [2023-12-27 12:15:07,485][20347] Updated weights for policy 0, policy_version 953320 (0.0008) [2023-12-27 12:15:07,535][20381] Updated weights for policy 1, policy_version 952453 (0.0005) [2023-12-27 12:15:07,543][20347] Updated weights for policy 0, policy_version 953330 (0.0008) [2023-12-27 12:15:07,591][20381] Updated weights for policy 1, policy_version 952463 (0.0005) [2023-12-27 12:15:08,171][20381] Updated weights for policy 1, policy_version 952473 (0.0005) [2023-12-27 12:15:08,222][20347] Updated weights for policy 0, policy_version 953340 (0.0007) [2023-12-27 12:15:08,229][20381] Updated weights for policy 1, policy_version 952483 (0.0005) [2023-12-27 12:15:08,274][20347] Updated weights for policy 0, policy_version 953350 (0.0007) [2023-12-27 12:15:08,283][20381] Updated weights for policy 1, policy_version 952493 (0.0005) [2023-12-27 12:15:08,342][20347] Updated weights for policy 0, policy_version 953360 (0.0007) [2023-12-27 12:15:08,347][20381] Updated weights for policy 1, policy_version 952503 (0.0008) [2023-12-27 12:15:08,982][20381] Updated weights for policy 1, policy_version 952513 (0.0008) [2023-12-27 12:15:09,031][20381] Updated weights for policy 1, policy_version 952523 (0.0008) [2023-12-27 12:15:09,031][20347] Updated weights for policy 0, policy_version 953370 (0.0007) [2023-12-27 12:15:09,088][20347] Updated weights for policy 0, policy_version 953380 (0.0005) [2023-12-27 12:15:09,092][20381] Updated weights for policy 1, policy_version 952533 (0.0007) [2023-12-27 12:15:09,136][20347] Updated weights for policy 0, policy_version 953390 (0.0008) [2023-12-27 12:15:09,194][20347] Updated weights for policy 0, policy_version 953400 (0.0009) [2023-12-27 12:15:09,777][20381] Updated weights for policy 1, policy_version 952543 (0.0009) [2023-12-27 12:15:09,842][20381] Updated weights for policy 1, policy_version 952553 (0.0010) [2023-12-27 12:15:09,896][20347] Updated weights for policy 0, policy_version 953410 (0.0008) [2023-12-27 12:15:09,906][20381] Updated weights for policy 1, policy_version 952563 (0.0007) [2023-12-27 12:15:09,957][20347] Updated weights for policy 0, policy_version 953420 (0.0008) [2023-12-27 12:15:10,012][20347] Updated weights for policy 0, policy_version 953430 (0.0009) [2023-12-27 12:15:10,567][20381] Updated weights for policy 1, policy_version 952573 (0.0007) [2023-12-27 12:15:10,614][20381] Updated weights for policy 1, policy_version 952583 (0.0005) [2023-12-27 12:15:10,672][20381] Updated weights for policy 1, policy_version 952593 (0.0005) [2023-12-27 12:15:10,788][20347] Updated weights for policy 0, policy_version 953440 (0.0008) [2023-12-27 12:15:10,840][20347] Updated weights for policy 0, policy_version 953450 (0.0008) [2023-12-27 12:15:10,888][20347] Updated weights for policy 0, policy_version 953460 (0.0007) [2023-12-27 12:15:11,300][20381] Updated weights for policy 1, policy_version 952603 (0.0006) [2023-12-27 12:15:11,356][20381] Updated weights for policy 1, policy_version 952613 (0.0007) [2023-12-27 12:15:11,422][20381] Updated weights for policy 1, policy_version 952623 (0.0008) [2023-12-27 12:15:11,683][20347] Updated weights for policy 0, policy_version 953470 (0.0008) [2023-12-27 12:15:11,755][20347] Updated weights for policy 0, policy_version 953480 (0.0009) [2023-12-27 12:15:11,814][20347] Updated weights for policy 0, policy_version 953490 (0.0008) [2023-12-27 12:15:12,176][20381] Updated weights for policy 1, policy_version 952633 (0.0008) [2023-12-27 12:15:12,236][20381] Updated weights for policy 1, policy_version 952644 (0.0008) [2023-12-27 12:15:12,298][20381] Updated weights for policy 1, policy_version 952654 (0.0009) [2023-12-27 12:15:12,361][20381] Updated weights for policy 1, policy_version 952664 (0.0008) [2023-12-27 12:15:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 488046592. Throughput: 0: 10305.3, 1: 10321.2. Samples: 488035848. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 12:15:12,473][20347] Updated weights for policy 0, policy_version 953500 (0.0008) [2023-12-27 12:15:12,531][20347] Updated weights for policy 0, policy_version 953510 (0.0007) [2023-12-27 12:15:12,587][20347] Updated weights for policy 0, policy_version 953520 (0.0008) [2023-12-27 12:15:13,015][20381] Updated weights for policy 1, policy_version 952674 (0.0009) [2023-12-27 12:15:13,062][20381] Updated weights for policy 1, policy_version 952684 (0.0009) [2023-12-27 12:15:13,111][20381] Updated weights for policy 1, policy_version 952694 (0.0009) [2023-12-27 12:15:13,221][20347] Updated weights for policy 0, policy_version 953530 (0.0007) [2023-12-27 12:15:13,270][20347] Updated weights for policy 0, policy_version 953540 (0.0005) [2023-12-27 12:15:13,320][20347] Updated weights for policy 0, policy_version 953550 (0.0005) [2023-12-27 12:15:13,373][20347] Updated weights for policy 0, policy_version 953560 (0.0005) [2023-12-27 12:15:13,864][20381] Updated weights for policy 1, policy_version 952704 (0.0010) [2023-12-27 12:15:13,920][20381] Updated weights for policy 1, policy_version 952714 (0.0005) [2023-12-27 12:15:13,979][20381] Updated weights for policy 1, policy_version 952724 (0.0008) [2023-12-27 12:15:14,028][20347] Updated weights for policy 0, policy_version 953570 (0.0006) [2023-12-27 12:15:14,087][20347] Updated weights for policy 0, policy_version 953580 (0.0008) [2023-12-27 12:15:14,145][20347] Updated weights for policy 0, policy_version 953590 (0.0007) [2023-12-27 12:15:14,620][20381] Updated weights for policy 1, policy_version 952734 (0.0011) [2023-12-27 12:15:14,680][20381] Updated weights for policy 1, policy_version 952744 (0.0011) [2023-12-27 12:15:14,737][20381] Updated weights for policy 1, policy_version 952754 (0.0009) [2023-12-27 12:15:14,876][20347] Updated weights for policy 0, policy_version 953600 (0.0007) [2023-12-27 12:15:14,934][20347] Updated weights for policy 0, policy_version 953610 (0.0008) [2023-12-27 12:15:14,998][20347] Updated weights for policy 0, policy_version 953620 (0.0008) [2023-12-27 12:15:15,436][20381] Updated weights for policy 1, policy_version 952764 (0.0007) [2023-12-27 12:15:15,491][20381] Updated weights for policy 1, policy_version 952774 (0.0008) [2023-12-27 12:15:15,540][20381] Updated weights for policy 1, policy_version 952784 (0.0008) [2023-12-27 12:15:15,654][20347] Updated weights for policy 0, policy_version 953630 (0.0008) [2023-12-27 12:15:15,714][20347] Updated weights for policy 0, policy_version 953640 (0.0006) [2023-12-27 12:15:15,770][20347] Updated weights for policy 0, policy_version 953650 (0.0005) [2023-12-27 12:15:16,283][20381] Updated weights for policy 1, policy_version 952794 (0.0009) [2023-12-27 12:15:16,335][20381] Updated weights for policy 1, policy_version 952804 (0.0008) [2023-12-27 12:15:16,387][20381] Updated weights for policy 1, policy_version 952814 (0.0008) [2023-12-27 12:15:16,410][20347] Updated weights for policy 0, policy_version 953660 (0.0007) [2023-12-27 12:15:16,445][20381] Updated weights for policy 1, policy_version 952824 (0.0006) [2023-12-27 12:15:16,469][20347] Updated weights for policy 0, policy_version 953670 (0.0006) [2023-12-27 12:15:16,528][20347] Updated weights for policy 0, policy_version 953680 (0.0007) [2023-12-27 12:15:17,138][20381] Updated weights for policy 1, policy_version 952834 (0.0007) [2023-12-27 12:15:17,188][20381] Updated weights for policy 1, policy_version 952844 (0.0008) [2023-12-27 12:15:17,211][20347] Updated weights for policy 0, policy_version 953690 (0.0007) [2023-12-27 12:15:17,242][20381] Updated weights for policy 1, policy_version 952854 (0.0007) [2023-12-27 12:15:17,266][20347] Updated weights for policy 0, policy_version 953700 (0.0006) [2023-12-27 12:15:17,323][20347] Updated weights for policy 0, policy_version 953710 (0.0007) [2023-12-27 12:15:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 488144896. Throughput: 0: 10281.9, 1: 10202.3. Samples: 488156060. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 12:15:17,381][20347] Updated weights for policy 0, policy_version 953720 (0.0005) [2023-12-27 12:15:17,920][20347] Updated weights for policy 0, policy_version 953730 (0.0007) [2023-12-27 12:15:17,979][20347] Updated weights for policy 0, policy_version 953740 (0.0007) [2023-12-27 12:15:17,993][20381] Updated weights for policy 1, policy_version 952864 (0.0006) [2023-12-27 12:15:18,030][20347] Updated weights for policy 0, policy_version 953750 (0.0006) [2023-12-27 12:15:18,054][20381] Updated weights for policy 1, policy_version 952874 (0.0006) [2023-12-27 12:15:18,112][20381] Updated weights for policy 1, policy_version 952884 (0.0007) [2023-12-27 12:15:18,677][20347] Updated weights for policy 0, policy_version 953760 (0.0007) [2023-12-27 12:15:18,724][20381] Updated weights for policy 1, policy_version 952894 (0.0006) [2023-12-27 12:15:18,730][20347] Updated weights for policy 0, policy_version 953770 (0.0007) [2023-12-27 12:15:18,782][20381] Updated weights for policy 1, policy_version 952904 (0.0006) [2023-12-27 12:15:18,787][20347] Updated weights for policy 0, policy_version 953780 (0.0007) [2023-12-27 12:15:18,838][20381] Updated weights for policy 1, policy_version 952914 (0.0007) [2023-12-27 12:15:19,504][20347] Updated weights for policy 0, policy_version 953790 (0.0008) [2023-12-27 12:15:19,559][20381] Updated weights for policy 1, policy_version 952924 (0.0006) [2023-12-27 12:15:19,570][20347] Updated weights for policy 0, policy_version 953800 (0.0007) [2023-12-27 12:15:19,613][20381] Updated weights for policy 1, policy_version 952934 (0.0007) [2023-12-27 12:15:19,637][20347] Updated weights for policy 0, policy_version 953810 (0.0006) [2023-12-27 12:15:19,674][20381] Updated weights for policy 1, policy_version 952944 (0.0008) [2023-12-27 12:15:20,376][20347] Updated weights for policy 0, policy_version 953820 (0.0007) [2023-12-27 12:15:20,438][20347] Updated weights for policy 0, policy_version 953830 (0.0008) [2023-12-27 12:15:20,469][20381] Updated weights for policy 1, policy_version 952954 (0.0008) [2023-12-27 12:15:20,495][20347] Updated weights for policy 0, policy_version 953840 (0.0009) [2023-12-27 12:15:20,525][20381] Updated weights for policy 1, policy_version 952964 (0.0008) [2023-12-27 12:15:20,590][20381] Updated weights for policy 1, policy_version 952974 (0.0007) [2023-12-27 12:15:20,652][20381] Updated weights for policy 1, policy_version 952984 (0.0008) [2023-12-27 12:15:21,258][20347] Updated weights for policy 0, policy_version 953850 (0.0006) [2023-12-27 12:15:21,323][20347] Updated weights for policy 0, policy_version 953860 (0.0008) [2023-12-27 12:15:21,391][20347] Updated weights for policy 0, policy_version 953870 (0.0008) [2023-12-27 12:15:21,438][20381] Updated weights for policy 1, policy_version 952994 (0.0007) [2023-12-27 12:15:21,451][20347] Updated weights for policy 0, policy_version 953880 (0.0007) [2023-12-27 12:15:21,497][20381] Updated weights for policy 1, policy_version 953004 (0.0007) [2023-12-27 12:15:21,555][20381] Updated weights for policy 1, policy_version 953014 (0.0005) [2023-12-27 12:15:22,258][20347] Updated weights for policy 0, policy_version 953890 (0.0011) [2023-12-27 12:15:22,291][20381] Updated weights for policy 1, policy_version 953024 (0.0009) [2023-12-27 12:15:22,318][20347] Updated weights for policy 0, policy_version 953900 (0.0011) [2023-12-27 12:15:22,353][20381] Updated weights for policy 1, policy_version 953034 (0.0007) [2023-12-27 12:15:22,362][19170] Fps is (10 sec: 18841.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 488235008. Throughput: 0: 10275.8, 1: 10138.3. Samples: 488216932. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 12:15:22,381][20347] Updated weights for policy 0, policy_version 953910 (0.0010) [2023-12-27 12:15:22,392][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000953912_244244480.pth... [2023-12-27 12:15:22,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000952728_243941376.pth [2023-12-27 12:15:22,420][20381] Updated weights for policy 1, policy_version 953044 (0.0007) [2023-12-27 12:15:22,444][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000953048_244006912.pth... [2023-12-27 12:15:22,448][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000951864_243703808.pth [2023-12-27 12:15:23,102][20347] Updated weights for policy 0, policy_version 953920 (0.0010) [2023-12-27 12:15:23,118][20381] Updated weights for policy 1, policy_version 953054 (0.0006) [2023-12-27 12:15:23,169][20347] Updated weights for policy 0, policy_version 953930 (0.0010) [2023-12-27 12:15:23,178][20381] Updated weights for policy 1, policy_version 953064 (0.0006) [2023-12-27 12:15:23,229][20347] Updated weights for policy 0, policy_version 953940 (0.0010) [2023-12-27 12:15:23,236][20381] Updated weights for policy 1, policy_version 953074 (0.0008) [2023-12-27 12:15:23,903][20347] Updated weights for policy 0, policy_version 953950 (0.0010) [2023-12-27 12:15:23,944][20381] Updated weights for policy 1, policy_version 953084 (0.0008) [2023-12-27 12:15:23,958][20347] Updated weights for policy 0, policy_version 953960 (0.0010) [2023-12-27 12:15:23,996][20381] Updated weights for policy 1, policy_version 953094 (0.0005) [2023-12-27 12:15:24,004][20347] Updated weights for policy 0, policy_version 953970 (0.0009) [2023-12-27 12:15:24,054][20381] Updated weights for policy 1, policy_version 953104 (0.0006) [2023-12-27 12:15:24,627][20381] Updated weights for policy 1, policy_version 953114 (0.0006) [2023-12-27 12:15:24,692][20381] Updated weights for policy 1, policy_version 953124 (0.0009) [2023-12-27 12:15:24,720][20347] Updated weights for policy 0, policy_version 953980 (0.0009) [2023-12-27 12:15:24,743][20381] Updated weights for policy 1, policy_version 953134 (0.0010) [2023-12-27 12:15:24,781][20347] Updated weights for policy 0, policy_version 953990 (0.0007) [2023-12-27 12:15:24,797][20381] Updated weights for policy 1, policy_version 953144 (0.0011) [2023-12-27 12:15:24,843][20347] Updated weights for policy 0, policy_version 954000 (0.0007) [2023-12-27 12:15:25,532][20381] Updated weights for policy 1, policy_version 953154 (0.0011) [2023-12-27 12:15:25,537][20347] Updated weights for policy 0, policy_version 954010 (0.0008) [2023-12-27 12:15:25,589][20381] Updated weights for policy 1, policy_version 953164 (0.0011) [2023-12-27 12:15:25,598][20347] Updated weights for policy 0, policy_version 954020 (0.0008) [2023-12-27 12:15:25,649][20381] Updated weights for policy 1, policy_version 953174 (0.0011) [2023-12-27 12:15:25,655][20347] Updated weights for policy 0, policy_version 954030 (0.0006) [2023-12-27 12:15:25,720][20347] Updated weights for policy 0, policy_version 954040 (0.0007) [2023-12-27 12:15:26,365][20381] Updated weights for policy 1, policy_version 953184 (0.0008) [2023-12-27 12:15:26,426][20381] Updated weights for policy 1, policy_version 953194 (0.0008) [2023-12-27 12:15:26,435][20347] Updated weights for policy 0, policy_version 954050 (0.0006) [2023-12-27 12:15:26,480][20381] Updated weights for policy 1, policy_version 953204 (0.0008) [2023-12-27 12:15:26,489][20347] Updated weights for policy 0, policy_version 954060 (0.0005) [2023-12-27 12:15:26,547][20347] Updated weights for policy 0, policy_version 954070 (0.0006) [2023-12-27 12:15:27,171][20347] Updated weights for policy 0, policy_version 954080 (0.0007) [2023-12-27 12:15:27,203][20381] Updated weights for policy 1, policy_version 953214 (0.0007) [2023-12-27 12:15:27,231][20347] Updated weights for policy 0, policy_version 954090 (0.0006) [2023-12-27 12:15:27,256][20381] Updated weights for policy 1, policy_version 953224 (0.0005) [2023-12-27 12:15:27,288][20347] Updated weights for policy 0, policy_version 954100 (0.0009) [2023-12-27 12:15:27,310][20381] Updated weights for policy 1, policy_version 953234 (0.0007) [2023-12-27 12:15:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 488349696. Throughput: 0: 10179.1, 1: 10093.4. Samples: 488332648. Policy #0 lag: (min: 25.0, avg: 40.6, max: 57.0) [2023-12-27 12:15:27,363][19170] Avg episode reward: [(0, '961.050'), (1, '1000.000')] [2023-12-27 12:15:27,899][20381] Updated weights for policy 1, policy_version 953244 (0.0008) [2023-12-27 12:15:27,955][20381] Updated weights for policy 1, policy_version 953254 (0.0007) [2023-12-27 12:15:28,011][20381] Updated weights for policy 1, policy_version 953264 (0.0007) [2023-12-27 12:15:28,036][20347] Updated weights for policy 0, policy_version 954110 (0.0006) [2023-12-27 12:15:28,090][20347] Updated weights for policy 0, policy_version 954120 (0.0006) [2023-12-27 12:15:28,138][20347] Updated weights for policy 0, policy_version 954130 (0.0005) [2023-12-27 12:15:28,743][20381] Updated weights for policy 1, policy_version 953274 (0.0006) [2023-12-27 12:15:28,778][20347] Updated weights for policy 0, policy_version 954140 (0.0007) [2023-12-27 12:15:28,798][20381] Updated weights for policy 1, policy_version 953284 (0.0008) [2023-12-27 12:15:28,839][20347] Updated weights for policy 0, policy_version 954150 (0.0007) [2023-12-27 12:15:28,855][20381] Updated weights for policy 1, policy_version 953294 (0.0008) [2023-12-27 12:15:28,892][20347] Updated weights for policy 0, policy_version 954160 (0.0007) [2023-12-27 12:15:28,911][20381] Updated weights for policy 1, policy_version 953304 (0.0008) [2023-12-27 12:15:29,659][20347] Updated weights for policy 0, policy_version 954170 (0.0009) [2023-12-27 12:15:29,688][20381] Updated weights for policy 1, policy_version 953314 (0.0008) [2023-12-27 12:15:29,722][20347] Updated weights for policy 0, policy_version 954180 (0.0010) [2023-12-27 12:15:29,743][20381] Updated weights for policy 1, policy_version 953324 (0.0009) [2023-12-27 12:15:29,775][20347] Updated weights for policy 0, policy_version 954190 (0.0010) [2023-12-27 12:15:29,800][20381] Updated weights for policy 1, policy_version 953334 (0.0011) [2023-12-27 12:15:29,837][20347] Updated weights for policy 0, policy_version 954200 (0.0011) [2023-12-27 12:15:30,533][20381] Updated weights for policy 1, policy_version 953344 (0.0007) [2023-12-27 12:15:30,544][20347] Updated weights for policy 0, policy_version 954210 (0.0011) [2023-12-27 12:15:30,591][20381] Updated weights for policy 1, policy_version 953354 (0.0006) [2023-12-27 12:15:30,607][20347] Updated weights for policy 0, policy_version 954220 (0.0010) [2023-12-27 12:15:30,644][20381] Updated weights for policy 1, policy_version 953364 (0.0006) [2023-12-27 12:15:30,661][20347] Updated weights for policy 0, policy_version 954230 (0.0010) [2023-12-27 12:15:31,368][20381] Updated weights for policy 1, policy_version 953374 (0.0006) [2023-12-27 12:15:31,397][20347] Updated weights for policy 0, policy_version 954240 (0.0009) [2023-12-27 12:15:31,444][20381] Updated weights for policy 1, policy_version 953384 (0.0008) [2023-12-27 12:15:31,458][20347] Updated weights for policy 0, policy_version 954250 (0.0008) [2023-12-27 12:15:31,503][20381] Updated weights for policy 1, policy_version 953394 (0.0007) [2023-12-27 12:15:31,522][20347] Updated weights for policy 0, policy_version 954260 (0.0008) [2023-12-27 12:15:32,165][20347] Updated weights for policy 0, policy_version 954270 (0.0008) [2023-12-27 12:15:32,174][20381] Updated weights for policy 1, policy_version 953404 (0.0006) [2023-12-27 12:15:32,222][20347] Updated weights for policy 0, policy_version 954280 (0.0008) [2023-12-27 12:15:32,228][20381] Updated weights for policy 1, policy_version 953414 (0.0005) [2023-12-27 12:15:32,283][20347] Updated weights for policy 0, policy_version 954290 (0.0008) [2023-12-27 12:15:32,286][20381] Updated weights for policy 1, policy_version 953424 (0.0007) [2023-12-27 12:15:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 488448000. Throughput: 0: 10117.1, 1: 10039.7. Samples: 488451072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:32,363][19170] Avg episode reward: [(0, '961.050'), (1, '1000.000')] [2023-12-27 12:15:32,970][20381] Updated weights for policy 1, policy_version 953434 (0.0008) [2023-12-27 12:15:32,997][20347] Updated weights for policy 0, policy_version 954300 (0.0006) [2023-12-27 12:15:33,020][20381] Updated weights for policy 1, policy_version 953444 (0.0005) [2023-12-27 12:15:33,058][20347] Updated weights for policy 0, policy_version 954310 (0.0006) [2023-12-27 12:15:33,075][20381] Updated weights for policy 1, policy_version 953454 (0.0005) [2023-12-27 12:15:33,109][20347] Updated weights for policy 0, policy_version 954320 (0.0007) [2023-12-27 12:15:33,135][20381] Updated weights for policy 1, policy_version 953464 (0.0006) [2023-12-27 12:15:33,667][20381] Updated weights for policy 1, policy_version 953474 (0.0005) [2023-12-27 12:15:33,688][20347] Updated weights for policy 0, policy_version 954330 (0.0006) [2023-12-27 12:15:33,727][20381] Updated weights for policy 1, policy_version 953484 (0.0005) [2023-12-27 12:15:33,735][20347] Updated weights for policy 0, policy_version 954340 (0.0005) [2023-12-27 12:15:33,779][20381] Updated weights for policy 1, policy_version 953494 (0.0005) [2023-12-27 12:15:33,791][20347] Updated weights for policy 0, policy_version 954350 (0.0006) [2023-12-27 12:15:33,840][20347] Updated weights for policy 0, policy_version 954360 (0.0007) [2023-12-27 12:15:34,347][20381] Updated weights for policy 1, policy_version 953504 (0.0005) [2023-12-27 12:15:34,416][20381] Updated weights for policy 1, policy_version 953514 (0.0006) [2023-12-27 12:15:34,482][20381] Updated weights for policy 1, policy_version 953524 (0.0006) [2023-12-27 12:15:34,531][20347] Updated weights for policy 0, policy_version 954370 (0.0007) [2023-12-27 12:15:34,592][20347] Updated weights for policy 0, policy_version 954380 (0.0007) [2023-12-27 12:15:34,653][20347] Updated weights for policy 0, policy_version 954390 (0.0007) [2023-12-27 12:15:35,075][20381] Updated weights for policy 1, policy_version 953534 (0.0009) [2023-12-27 12:15:35,128][20381] Updated weights for policy 1, policy_version 953544 (0.0010) [2023-12-27 12:15:35,177][20381] Updated weights for policy 1, policy_version 953554 (0.0006) [2023-12-27 12:15:35,326][20347] Updated weights for policy 0, policy_version 954400 (0.0006) [2023-12-27 12:15:35,393][20347] Updated weights for policy 0, policy_version 954410 (0.0008) [2023-12-27 12:15:35,454][20347] Updated weights for policy 0, policy_version 954420 (0.0005) [2023-12-27 12:15:35,857][20381] Updated weights for policy 1, policy_version 953564 (0.0008) [2023-12-27 12:15:35,914][20381] Updated weights for policy 1, policy_version 953574 (0.0010) [2023-12-27 12:15:35,974][20347] Updated weights for policy 0, policy_version 954430 (0.0009) [2023-12-27 12:15:35,977][20381] Updated weights for policy 1, policy_version 953584 (0.0011) [2023-12-27 12:15:36,024][20347] Updated weights for policy 0, policy_version 954440 (0.0010) [2023-12-27 12:15:36,080][20347] Updated weights for policy 0, policy_version 954450 (0.0010) [2023-12-27 12:15:36,634][20381] Updated weights for policy 1, policy_version 953594 (0.0009) [2023-12-27 12:15:36,690][20381] Updated weights for policy 1, policy_version 953604 (0.0008) [2023-12-27 12:15:36,749][20381] Updated weights for policy 1, policy_version 953614 (0.0008) [2023-12-27 12:15:36,807][20381] Updated weights for policy 1, policy_version 953624 (0.0008) [2023-12-27 12:15:36,817][20347] Updated weights for policy 0, policy_version 954460 (0.0008) [2023-12-27 12:15:36,874][20347] Updated weights for policy 0, policy_version 954470 (0.0006) [2023-12-27 12:15:36,930][20347] Updated weights for policy 0, policy_version 954480 (0.0008) [2023-12-27 12:15:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 488546304. Throughput: 0: 10123.3, 1: 10130.0. Samples: 488516816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:37,363][19170] Avg episode reward: [(0, '970.810'), (1, '1000.000')] [2023-12-27 12:15:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000953624_244154368.pth... [2023-12-27 12:15:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000954488_244391936.pth... [2023-12-27 12:15:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000953304_244088832.pth [2023-12-27 12:15:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000952440_243851264.pth [2023-12-27 12:15:37,483][20381] Updated weights for policy 1, policy_version 953634 (0.0011) [2023-12-27 12:15:37,536][20381] Updated weights for policy 1, policy_version 953644 (0.0011) [2023-12-27 12:15:37,591][20347] Updated weights for policy 0, policy_version 954490 (0.0007) [2023-12-27 12:15:37,593][20381] Updated weights for policy 1, policy_version 953654 (0.0011) [2023-12-27 12:15:37,646][20347] Updated weights for policy 0, policy_version 954500 (0.0006) [2023-12-27 12:15:37,701][20347] Updated weights for policy 0, policy_version 954510 (0.0007) [2023-12-27 12:15:37,759][20347] Updated weights for policy 0, policy_version 954520 (0.0007) [2023-12-27 12:15:38,283][20381] Updated weights for policy 1, policy_version 953664 (0.0007) [2023-12-27 12:15:38,343][20381] Updated weights for policy 1, policy_version 953674 (0.0007) [2023-12-27 12:15:38,403][20381] Updated weights for policy 1, policy_version 953684 (0.0006) [2023-12-27 12:15:38,439][20347] Updated weights for policy 0, policy_version 954530 (0.0007) [2023-12-27 12:15:38,503][20347] Updated weights for policy 0, policy_version 954540 (0.0008) [2023-12-27 12:15:38,569][20347] Updated weights for policy 0, policy_version 954550 (0.0008) [2023-12-27 12:15:38,990][20381] Updated weights for policy 1, policy_version 953694 (0.0008) [2023-12-27 12:15:39,045][20381] Updated weights for policy 1, policy_version 953704 (0.0008) [2023-12-27 12:15:39,106][20381] Updated weights for policy 1, policy_version 953714 (0.0007) [2023-12-27 12:15:39,159][20347] Updated weights for policy 0, policy_version 954560 (0.0010) [2023-12-27 12:15:39,207][20347] Updated weights for policy 0, policy_version 954570 (0.0010) [2023-12-27 12:15:39,276][20347] Updated weights for policy 0, policy_version 954580 (0.0007) [2023-12-27 12:15:39,885][20381] Updated weights for policy 1, policy_version 953724 (0.0007) [2023-12-27 12:15:39,953][20381] Updated weights for policy 1, policy_version 953734 (0.0008) [2023-12-27 12:15:39,988][20347] Updated weights for policy 0, policy_version 954590 (0.0007) [2023-12-27 12:15:40,025][20381] Updated weights for policy 1, policy_version 953744 (0.0009) [2023-12-27 12:15:40,054][20347] Updated weights for policy 0, policy_version 954600 (0.0007) [2023-12-27 12:15:40,082][20291] KL-divergence is very high: 307.8981 [2023-12-27 12:15:40,088][20291] KL-divergence is very high: 399.5503 [2023-12-27 12:15:40,118][20347] Updated weights for policy 0, policy_version 954610 (0.0007) [2023-12-27 12:15:40,782][20347] Updated weights for policy 0, policy_version 954620 (0.0007) [2023-12-27 12:15:40,797][20381] Updated weights for policy 1, policy_version 953754 (0.0008) [2023-12-27 12:15:40,836][20347] Updated weights for policy 0, policy_version 954630 (0.0007) [2023-12-27 12:15:40,856][20381] Updated weights for policy 1, policy_version 953764 (0.0007) [2023-12-27 12:15:40,889][20347] Updated weights for policy 0, policy_version 954640 (0.0006) [2023-12-27 12:15:40,910][20381] Updated weights for policy 1, policy_version 953774 (0.0006) [2023-12-27 12:15:40,967][20381] Updated weights for policy 1, policy_version 953784 (0.0006) [2023-12-27 12:15:41,685][20347] Updated weights for policy 0, policy_version 954650 (0.0007) [2023-12-27 12:15:41,752][20347] Updated weights for policy 0, policy_version 954660 (0.0008) [2023-12-27 12:15:41,764][20381] Updated weights for policy 1, policy_version 953794 (0.0008) [2023-12-27 12:15:41,817][20347] Updated weights for policy 0, policy_version 954670 (0.0007) [2023-12-27 12:15:41,822][20381] Updated weights for policy 1, policy_version 953804 (0.0006) [2023-12-27 12:15:41,866][20347] Updated weights for policy 0, policy_version 954680 (0.0007) [2023-12-27 12:15:41,886][20381] Updated weights for policy 1, policy_version 953814 (0.0008) [2023-12-27 12:15:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 488644608. Throughput: 0: 10103.9, 1: 10144.8. Samples: 488638412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:42,363][19170] Avg episode reward: [(0, '912.490'), (1, '1000.000')] [2023-12-27 12:15:42,592][20381] Updated weights for policy 1, policy_version 953824 (0.0007) [2023-12-27 12:15:42,623][20347] Updated weights for policy 0, policy_version 954690 (0.0010) [2023-12-27 12:15:42,650][20381] Updated weights for policy 1, policy_version 953834 (0.0009) [2023-12-27 12:15:42,679][20347] Updated weights for policy 0, policy_version 954700 (0.0007) [2023-12-27 12:15:42,706][20381] Updated weights for policy 1, policy_version 953844 (0.0006) [2023-12-27 12:15:42,743][20347] Updated weights for policy 0, policy_version 954710 (0.0008) [2023-12-27 12:15:43,396][20381] Updated weights for policy 1, policy_version 953854 (0.0009) [2023-12-27 12:15:43,449][20381] Updated weights for policy 1, policy_version 953864 (0.0011) [2023-12-27 12:15:43,449][20291] KL-divergence is very high: 154.0899 [2023-12-27 12:15:43,450][20347] Updated weights for policy 0, policy_version 954720 (0.0006) [2023-12-27 12:15:43,486][20291] KL-divergence is very high: 157.0751 [2023-12-27 12:15:43,495][20381] Updated weights for policy 1, policy_version 953874 (0.0010) [2023-12-27 12:15:43,496][20347] Updated weights for policy 0, policy_version 954730 (0.0005) [2023-12-27 12:15:43,555][20347] Updated weights for policy 0, policy_version 954740 (0.0007) [2023-12-27 12:15:44,211][20381] Updated weights for policy 1, policy_version 953884 (0.0010) [2023-12-27 12:15:44,259][20347] Updated weights for policy 0, policy_version 954750 (0.0006) [2023-12-27 12:15:44,264][20381] Updated weights for policy 1, policy_version 953894 (0.0008) [2023-12-27 12:15:44,317][20381] Updated weights for policy 1, policy_version 953904 (0.0006) [2023-12-27 12:15:44,318][20347] Updated weights for policy 0, policy_version 954760 (0.0010) [2023-12-27 12:15:44,370][20347] Updated weights for policy 0, policy_version 954770 (0.0010) [2023-12-27 12:15:45,011][20347] Updated weights for policy 0, policy_version 954780 (0.0009) [2023-12-27 12:15:45,078][20347] Updated weights for policy 0, policy_version 954790 (0.0007) [2023-12-27 12:15:45,088][20381] Updated weights for policy 1, policy_version 953914 (0.0005) [2023-12-27 12:15:45,145][20347] Updated weights for policy 0, policy_version 954800 (0.0008) [2023-12-27 12:15:45,151][20381] Updated weights for policy 1, policy_version 953924 (0.0008) [2023-12-27 12:15:45,216][20381] Updated weights for policy 1, policy_version 953934 (0.0008) [2023-12-27 12:15:45,278][20381] Updated weights for policy 1, policy_version 953944 (0.0008) [2023-12-27 12:15:45,742][20347] Updated weights for policy 0, policy_version 954810 (0.0008) [2023-12-27 12:15:45,800][20347] Updated weights for policy 0, policy_version 954820 (0.0008) [2023-12-27 12:15:45,856][20347] Updated weights for policy 0, policy_version 954830 (0.0008) [2023-12-27 12:15:45,902][20347] Updated weights for policy 0, policy_version 954840 (0.0008) [2023-12-27 12:15:45,974][20381] Updated weights for policy 1, policy_version 953954 (0.0008) [2023-12-27 12:15:46,023][20381] Updated weights for policy 1, policy_version 953964 (0.0007) [2023-12-27 12:15:46,069][20381] Updated weights for policy 1, policy_version 953974 (0.0005) [2023-12-27 12:15:46,650][20381] Updated weights for policy 1, policy_version 953984 (0.0006) [2023-12-27 12:15:46,674][20347] Updated weights for policy 0, policy_version 954850 (0.0007) [2023-12-27 12:15:46,709][20381] Updated weights for policy 1, policy_version 953994 (0.0011) [2023-12-27 12:15:46,726][20347] Updated weights for policy 0, policy_version 954860 (0.0006) [2023-12-27 12:15:46,762][20381] Updated weights for policy 1, policy_version 954004 (0.0011) [2023-12-27 12:15:46,779][20347] Updated weights for policy 0, policy_version 954870 (0.0005) [2023-12-27 12:15:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 488742912. Throughput: 0: 10062.3, 1: 10092.1. Samples: 488756772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:47,363][19170] Avg episode reward: [(0, '922.180'), (1, '1000.000')] [2023-12-27 12:15:47,410][20381] Updated weights for policy 1, policy_version 954014 (0.0007) [2023-12-27 12:15:47,467][20381] Updated weights for policy 1, policy_version 954024 (0.0005) [2023-12-27 12:15:47,490][20347] Updated weights for policy 0, policy_version 954880 (0.0005) [2023-12-27 12:15:47,505][20291] KL-divergence is very high: 438.3935 [2023-12-27 12:15:47,511][20291] KL-divergence is very high: 394.7404 [2023-12-27 12:15:47,531][20381] Updated weights for policy 1, policy_version 954034 (0.0005) [2023-12-27 12:15:47,551][20347] Updated weights for policy 0, policy_version 954890 (0.0005) [2023-12-27 12:15:47,613][20347] Updated weights for policy 0, policy_version 954900 (0.0007) [2023-12-27 12:15:48,081][20381] Updated weights for policy 1, policy_version 954044 (0.0007) [2023-12-27 12:15:48,145][20381] Updated weights for policy 1, policy_version 954054 (0.0010) [2023-12-27 12:15:48,172][20291] KL-divergence is very high: 646.8627 [2023-12-27 12:15:48,194][20347] Updated weights for policy 0, policy_version 954910 (0.0007) [2023-12-27 12:15:48,203][20381] Updated weights for policy 1, policy_version 954064 (0.0007) [2023-12-27 12:15:48,209][20291] KL-divergence is very high: 546.5090 [2023-12-27 12:15:48,243][20347] Updated weights for policy 0, policy_version 954920 (0.0007) [2023-12-27 12:15:48,249][20291] KL-divergence is very high: 379.3716 [2023-12-27 12:15:48,292][20291] KL-divergence is very high: 387.4271 [2023-12-27 12:15:48,298][20347] Updated weights for policy 0, policy_version 954930 (0.0007) [2023-12-27 12:15:48,794][20381] Updated weights for policy 1, policy_version 954074 (0.0009) [2023-12-27 12:15:48,853][20381] Updated weights for policy 1, policy_version 954084 (0.0008) [2023-12-27 12:15:48,946][20381] Updated weights for policy 1, policy_version 954094 (0.0007) [2023-12-27 12:15:48,997][20381] Updated weights for policy 1, policy_version 954104 (0.0007) [2023-12-27 12:15:49,039][20347] Updated weights for policy 0, policy_version 954940 (0.0008) [2023-12-27 12:15:49,091][20347] Updated weights for policy 0, policy_version 954950 (0.0010) [2023-12-27 12:15:49,151][20347] Updated weights for policy 0, policy_version 954960 (0.0011) [2023-12-27 12:15:49,665][20381] Updated weights for policy 1, policy_version 954114 (0.0007) [2023-12-27 12:15:49,717][20381] Updated weights for policy 1, policy_version 954124 (0.0007) [2023-12-27 12:15:49,772][20381] Updated weights for policy 1, policy_version 954134 (0.0007) [2023-12-27 12:15:49,860][20347] Updated weights for policy 0, policy_version 954970 (0.0010) [2023-12-27 12:15:49,919][20347] Updated weights for policy 0, policy_version 954980 (0.0008) [2023-12-27 12:15:49,992][20347] Updated weights for policy 0, policy_version 954990 (0.0008) [2023-12-27 12:15:50,052][20347] Updated weights for policy 0, policy_version 955000 (0.0008) [2023-12-27 12:15:50,502][20381] Updated weights for policy 1, policy_version 954144 (0.0007) [2023-12-27 12:15:50,556][20381] Updated weights for policy 1, policy_version 954154 (0.0007) [2023-12-27 12:15:50,620][20381] Updated weights for policy 1, policy_version 954164 (0.0007) [2023-12-27 12:15:50,703][20347] Updated weights for policy 0, policy_version 955010 (0.0008) [2023-12-27 12:15:50,766][20347] Updated weights for policy 0, policy_version 955020 (0.0008) [2023-12-27 12:15:50,832][20347] Updated weights for policy 0, policy_version 955030 (0.0008) [2023-12-27 12:15:51,345][20381] Updated weights for policy 1, policy_version 954174 (0.0008) [2023-12-27 12:15:51,410][20381] Updated weights for policy 1, policy_version 954184 (0.0008) [2023-12-27 12:15:51,473][20381] Updated weights for policy 1, policy_version 954194 (0.0010) [2023-12-27 12:15:51,593][20347] Updated weights for policy 0, policy_version 955040 (0.0009) [2023-12-27 12:15:51,662][20347] Updated weights for policy 0, policy_version 955050 (0.0008) [2023-12-27 12:15:51,720][20347] Updated weights for policy 0, policy_version 955060 (0.0007) [2023-12-27 12:15:52,246][20381] Updated weights for policy 1, policy_version 954204 (0.0009) [2023-12-27 12:15:52,307][20381] Updated weights for policy 1, policy_version 954214 (0.0009) [2023-12-27 12:15:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 488841216. Throughput: 0: 10008.7, 1: 10104.7. Samples: 488818680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:52,363][19170] Avg episode reward: [(0, '941.620'), (1, '990.260')] [2023-12-27 12:15:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000955064_244539392.pth... [2023-12-27 12:15:52,369][20381] Updated weights for policy 1, policy_version 954224 (0.0009) [2023-12-27 12:15:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000953912_244244480.pth [2023-12-27 12:15:52,418][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000954232_244310016.pth... [2023-12-27 12:15:52,422][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000953048_244006912.pth [2023-12-27 12:15:52,474][20347] Updated weights for policy 0, policy_version 955070 (0.0007) [2023-12-27 12:15:52,527][20347] Updated weights for policy 0, policy_version 955080 (0.0007) [2023-12-27 12:15:52,582][20347] Updated weights for policy 0, policy_version 955090 (0.0005) [2023-12-27 12:15:53,069][20381] Updated weights for policy 1, policy_version 954234 (0.0008) [2023-12-27 12:15:53,129][20381] Updated weights for policy 1, policy_version 954244 (0.0007) [2023-12-27 12:15:53,182][20381] Updated weights for policy 1, policy_version 954254 (0.0007) [2023-12-27 12:15:53,184][20347] Updated weights for policy 0, policy_version 955100 (0.0005) [2023-12-27 12:15:53,237][20381] Updated weights for policy 1, policy_version 954264 (0.0007) [2023-12-27 12:15:53,243][20347] Updated weights for policy 0, policy_version 955110 (0.0006) [2023-12-27 12:15:53,304][20347] Updated weights for policy 0, policy_version 955120 (0.0008) [2023-12-27 12:15:53,812][20381] Updated weights for policy 1, policy_version 954274 (0.0010) [2023-12-27 12:15:53,870][20381] Updated weights for policy 1, policy_version 954284 (0.0005) [2023-12-27 12:15:53,928][20381] Updated weights for policy 1, policy_version 954294 (0.0011) [2023-12-27 12:15:54,027][20347] Updated weights for policy 0, policy_version 955130 (0.0007) [2023-12-27 12:15:54,090][20347] Updated weights for policy 0, policy_version 955140 (0.0005) [2023-12-27 12:15:54,149][20347] Updated weights for policy 0, policy_version 955150 (0.0005) [2023-12-27 12:15:54,197][20347] Updated weights for policy 0, policy_version 955160 (0.0005) [2023-12-27 12:15:54,522][20381] Updated weights for policy 1, policy_version 954304 (0.0008) [2023-12-27 12:15:54,579][20381] Updated weights for policy 1, policy_version 954314 (0.0007) [2023-12-27 12:15:54,625][20381] Updated weights for policy 1, policy_version 954324 (0.0005) [2023-12-27 12:15:54,796][20347] Updated weights for policy 0, policy_version 955170 (0.0008) [2023-12-27 12:15:54,851][20347] Updated weights for policy 0, policy_version 955180 (0.0009) [2023-12-27 12:15:54,905][20347] Updated weights for policy 0, policy_version 955190 (0.0007) [2023-12-27 12:15:55,288][20381] Updated weights for policy 1, policy_version 954334 (0.0007) [2023-12-27 12:15:55,345][20381] Updated weights for policy 1, policy_version 954344 (0.0007) [2023-12-27 12:15:55,407][20381] Updated weights for policy 1, policy_version 954354 (0.0008) [2023-12-27 12:15:55,593][20347] Updated weights for policy 0, policy_version 955200 (0.0008) [2023-12-27 12:15:55,655][20347] Updated weights for policy 0, policy_version 955210 (0.0008) [2023-12-27 12:15:55,686][20291] KL-divergence is very high: 127.2997 [2023-12-27 12:15:55,709][20291] KL-divergence is very high: 750.9869 [2023-12-27 12:15:55,715][20347] Updated weights for policy 0, policy_version 955220 (0.0008) [2023-12-27 12:15:56,105][20381] Updated weights for policy 1, policy_version 954364 (0.0007) [2023-12-27 12:15:56,163][20381] Updated weights for policy 1, policy_version 954374 (0.0007) [2023-12-27 12:15:56,219][20381] Updated weights for policy 1, policy_version 954384 (0.0007) [2023-12-27 12:15:56,408][20347] Updated weights for policy 0, policy_version 955230 (0.0009) [2023-12-27 12:15:56,484][20347] Updated weights for policy 0, policy_version 955240 (0.0010) [2023-12-27 12:15:56,538][20347] Updated weights for policy 0, policy_version 955250 (0.0010) [2023-12-27 12:15:56,883][20381] Updated weights for policy 1, policy_version 954394 (0.0008) [2023-12-27 12:15:56,941][20381] Updated weights for policy 1, policy_version 954404 (0.0007) [2023-12-27 12:15:56,992][20381] Updated weights for policy 1, policy_version 954414 (0.0008) [2023-12-27 12:15:57,055][20381] Updated weights for policy 1, policy_version 954424 (0.0006) [2023-12-27 12:15:57,173][20347] Updated weights for policy 0, policy_version 955260 (0.0009) [2023-12-27 12:15:57,231][20347] Updated weights for policy 0, policy_version 955270 (0.0008) [2023-12-27 12:15:57,293][20347] Updated weights for policy 0, policy_version 955280 (0.0008) [2023-12-27 12:15:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 488955904. Throughput: 0: 10043.1, 1: 10052.1. Samples: 488940136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:15:57,364][19170] Avg episode reward: [(0, '912.410'), (1, '990.260')] [2023-12-27 12:15:57,615][20381] Updated weights for policy 1, policy_version 954434 (0.0008) [2023-12-27 12:15:57,660][20381] Updated weights for policy 1, policy_version 954444 (0.0008) [2023-12-27 12:15:57,709][20381] Updated weights for policy 1, policy_version 954454 (0.0006) [2023-12-27 12:15:57,862][20347] Updated weights for policy 0, policy_version 955290 (0.0007) [2023-12-27 12:15:57,918][20347] Updated weights for policy 0, policy_version 955300 (0.0007) [2023-12-27 12:15:57,972][20347] Updated weights for policy 0, policy_version 955310 (0.0007) [2023-12-27 12:15:58,037][20347] Updated weights for policy 0, policy_version 955320 (0.0008) [2023-12-27 12:15:58,401][20381] Updated weights for policy 1, policy_version 954464 (0.0006) [2023-12-27 12:15:58,460][20381] Updated weights for policy 1, policy_version 954474 (0.0008) [2023-12-27 12:15:58,523][20381] Updated weights for policy 1, policy_version 954484 (0.0008) [2023-12-27 12:15:58,822][20347] Updated weights for policy 0, policy_version 955330 (0.0009) [2023-12-27 12:15:58,883][20347] Updated weights for policy 0, policy_version 955340 (0.0009) [2023-12-27 12:15:58,943][20347] Updated weights for policy 0, policy_version 955350 (0.0010) [2023-12-27 12:15:59,319][20381] Updated weights for policy 1, policy_version 954494 (0.0008) [2023-12-27 12:15:59,382][20381] Updated weights for policy 1, policy_version 954504 (0.0007) [2023-12-27 12:15:59,435][20381] Updated weights for policy 1, policy_version 954514 (0.0007) [2023-12-27 12:15:59,627][20291] KL-divergence is very high: 157.1621 [2023-12-27 12:15:59,632][20291] KL-divergence is very high: 224.1285 [2023-12-27 12:15:59,637][20291] KL-divergence is very high: 140.9814 [2023-12-27 12:15:59,658][20347] Updated weights for policy 0, policy_version 955360 (0.0010) [2023-12-27 12:15:59,705][20291] KL-divergence is very high: 108.7904 [2023-12-27 12:15:59,707][20347] Updated weights for policy 0, policy_version 955370 (0.0010) [2023-12-27 12:15:59,711][20291] KL-divergence is very high: 129.1093 [2023-12-27 12:15:59,755][20347] Updated weights for policy 0, policy_version 955380 (0.0010) [2023-12-27 12:16:00,116][20381] Updated weights for policy 1, policy_version 954524 (0.0007) [2023-12-27 12:16:00,167][20381] Updated weights for policy 1, policy_version 954534 (0.0006) [2023-12-27 12:16:00,224][20381] Updated weights for policy 1, policy_version 954544 (0.0006) [2023-12-27 12:16:00,435][20291] KL-divergence is very high: 111.3879 [2023-12-27 12:16:00,459][20347] Updated weights for policy 0, policy_version 955390 (0.0010) [2023-12-27 12:16:00,514][20347] Updated weights for policy 0, policy_version 955400 (0.0010) [2023-12-27 12:16:00,573][20347] Updated weights for policy 0, policy_version 955410 (0.0010) [2023-12-27 12:16:00,841][20381] Updated weights for policy 1, policy_version 954554 (0.0006) [2023-12-27 12:16:00,893][20381] Updated weights for policy 1, policy_version 954564 (0.0010) [2023-12-27 12:16:00,938][20381] Updated weights for policy 1, policy_version 954574 (0.0010) [2023-12-27 12:16:00,983][20381] Updated weights for policy 1, policy_version 954584 (0.0010) [2023-12-27 12:16:01,250][20347] Updated weights for policy 0, policy_version 955420 (0.0009) [2023-12-27 12:16:01,314][20347] Updated weights for policy 0, policy_version 955431 (0.0007) [2023-12-27 12:16:01,383][20347] Updated weights for policy 0, policy_version 955441 (0.0008) [2023-12-27 12:16:01,743][20381] Updated weights for policy 1, policy_version 954594 (0.0008) [2023-12-27 12:16:01,797][20381] Updated weights for policy 1, policy_version 954604 (0.0010) [2023-12-27 12:16:01,852][20381] Updated weights for policy 1, policy_version 954614 (0.0008) [2023-12-27 12:16:02,047][20347] Updated weights for policy 0, policy_version 955451 (0.0006) [2023-12-27 12:16:02,103][20347] Updated weights for policy 0, policy_version 955461 (0.0006) [2023-12-27 12:16:02,160][20347] Updated weights for policy 0, policy_version 955471 (0.0006) [2023-12-27 12:16:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 489054208. Throughput: 0: 10032.6, 1: 10106.7. Samples: 489062324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:02,363][19170] Avg episode reward: [(0, '523.050'), (1, '1000.000')] [2023-12-27 12:16:02,625][20381] Updated weights for policy 1, policy_version 954624 (0.0007) [2023-12-27 12:16:02,672][20381] Updated weights for policy 1, policy_version 954634 (0.0005) [2023-12-27 12:16:02,733][20381] Updated weights for policy 1, policy_version 954644 (0.0009) [2023-12-27 12:16:02,765][20347] Updated weights for policy 0, policy_version 955481 (0.0007) [2023-12-27 12:16:02,822][20347] Updated weights for policy 0, policy_version 955491 (0.0005) [2023-12-27 12:16:02,873][20347] Updated weights for policy 0, policy_version 955501 (0.0005) [2023-12-27 12:16:02,923][20347] Updated weights for policy 0, policy_version 955511 (0.0005) [2023-12-27 12:16:03,422][20381] Updated weights for policy 1, policy_version 954654 (0.0010) [2023-12-27 12:16:03,474][20381] Updated weights for policy 1, policy_version 954664 (0.0009) [2023-12-27 12:16:03,521][20381] Updated weights for policy 1, policy_version 954674 (0.0009) [2023-12-27 12:16:03,539][20347] Updated weights for policy 0, policy_version 955521 (0.0007) [2023-12-27 12:16:03,590][20347] Updated weights for policy 0, policy_version 955531 (0.0006) [2023-12-27 12:16:03,643][20347] Updated weights for policy 0, policy_version 955541 (0.0008) [2023-12-27 12:16:04,255][20381] Updated weights for policy 1, policy_version 954684 (0.0006) [2023-12-27 12:16:04,317][20381] Updated weights for policy 1, policy_version 954694 (0.0006) [2023-12-27 12:16:04,322][20347] Updated weights for policy 0, policy_version 955551 (0.0007) [2023-12-27 12:16:04,376][20347] Updated weights for policy 0, policy_version 955561 (0.0006) [2023-12-27 12:16:04,382][20381] Updated weights for policy 1, policy_version 954704 (0.0006) [2023-12-27 12:16:04,434][20347] Updated weights for policy 0, policy_version 955571 (0.0005) [2023-12-27 12:16:04,973][20381] Updated weights for policy 1, policy_version 954714 (0.0006) [2023-12-27 12:16:05,023][20381] Updated weights for policy 1, policy_version 954724 (0.0008) [2023-12-27 12:16:05,080][20381] Updated weights for policy 1, policy_version 954734 (0.0005) [2023-12-27 12:16:05,133][20381] Updated weights for policy 1, policy_version 954744 (0.0007) [2023-12-27 12:16:05,145][20347] Updated weights for policy 0, policy_version 955581 (0.0005) [2023-12-27 12:16:05,206][20347] Updated weights for policy 0, policy_version 955591 (0.0005) [2023-12-27 12:16:05,264][20347] Updated weights for policy 0, policy_version 955601 (0.0005) [2023-12-27 12:16:05,839][20381] Updated weights for policy 1, policy_version 954754 (0.0007) [2023-12-27 12:16:05,876][20347] Updated weights for policy 0, policy_version 955611 (0.0008) [2023-12-27 12:16:05,886][20381] Updated weights for policy 1, policy_version 954764 (0.0006) [2023-12-27 12:16:05,923][20347] Updated weights for policy 0, policy_version 955621 (0.0008) [2023-12-27 12:16:05,937][20381] Updated weights for policy 1, policy_version 954774 (0.0006) [2023-12-27 12:16:05,973][20347] Updated weights for policy 0, policy_version 955631 (0.0005) [2023-12-27 12:16:06,675][20381] Updated weights for policy 1, policy_version 954784 (0.0008) [2023-12-27 12:16:06,677][20347] Updated weights for policy 0, policy_version 955641 (0.0007) [2023-12-27 12:16:06,730][20381] Updated weights for policy 1, policy_version 954794 (0.0006) [2023-12-27 12:16:06,736][20347] Updated weights for policy 0, policy_version 955651 (0.0006) [2023-12-27 12:16:06,784][20381] Updated weights for policy 1, policy_version 954804 (0.0006) [2023-12-27 12:16:06,793][20347] Updated weights for policy 0, policy_version 955661 (0.0006) [2023-12-27 12:16:06,850][20347] Updated weights for policy 0, policy_version 955671 (0.0007) [2023-12-27 12:16:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 489152512. Throughput: 0: 10046.6, 1: 10124.3. Samples: 489124624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:07,363][19170] Avg episode reward: [(0, '683.610'), (1, '1000.000')] [2023-12-27 12:16:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000954808_244457472.pth... [2023-12-27 12:16:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000955672_244695040.pth... [2023-12-27 12:16:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000953624_244154368.pth [2023-12-27 12:16:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000954488_244391936.pth [2023-12-27 12:16:07,508][20381] Updated weights for policy 1, policy_version 954814 (0.0007) [2023-12-27 12:16:07,524][20347] Updated weights for policy 0, policy_version 955681 (0.0007) [2023-12-27 12:16:07,566][20381] Updated weights for policy 1, policy_version 954824 (0.0006) [2023-12-27 12:16:07,571][20347] Updated weights for policy 0, policy_version 955691 (0.0006) [2023-12-27 12:16:07,623][20381] Updated weights for policy 1, policy_version 954834 (0.0006) [2023-12-27 12:16:07,625][20347] Updated weights for policy 0, policy_version 955701 (0.0006) [2023-12-27 12:16:08,290][20381] Updated weights for policy 1, policy_version 954844 (0.0008) [2023-12-27 12:16:08,348][20381] Updated weights for policy 1, policy_version 954854 (0.0010) [2023-12-27 12:16:08,350][20347] Updated weights for policy 0, policy_version 955711 (0.0007) [2023-12-27 12:16:08,407][20347] Updated weights for policy 0, policy_version 955721 (0.0006) [2023-12-27 12:16:08,413][20381] Updated weights for policy 1, policy_version 954864 (0.0010) [2023-12-27 12:16:08,470][20347] Updated weights for policy 0, policy_version 955731 (0.0006) [2023-12-27 12:16:09,024][20381] Updated weights for policy 1, policy_version 954874 (0.0008) [2023-12-27 12:16:09,073][20381] Updated weights for policy 1, policy_version 954884 (0.0007) [2023-12-27 12:16:09,125][20381] Updated weights for policy 1, policy_version 954894 (0.0007) [2023-12-27 12:16:09,127][20347] Updated weights for policy 0, policy_version 955741 (0.0007) [2023-12-27 12:16:09,149][20291] KL-divergence is very high: 298.9345 [2023-12-27 12:16:09,175][20347] Updated weights for policy 0, policy_version 955751 (0.0007) [2023-12-27 12:16:09,180][20381] Updated weights for policy 1, policy_version 954904 (0.0005) [2023-12-27 12:16:09,189][20291] KL-divergence is very high: 298.7349 [2023-12-27 12:16:09,227][20347] Updated weights for policy 0, policy_version 955761 (0.0008) [2023-12-27 12:16:09,235][20291] KL-divergence is very high: 126.8054 [2023-12-27 12:16:09,926][20381] Updated weights for policy 1, policy_version 954914 (0.0010) [2023-12-27 12:16:09,995][20381] Updated weights for policy 1, policy_version 954924 (0.0006) [2023-12-27 12:16:09,998][20347] Updated weights for policy 0, policy_version 955771 (0.0007) [2023-12-27 12:16:10,055][20347] Updated weights for policy 0, policy_version 955781 (0.0007) [2023-12-27 12:16:10,057][20381] Updated weights for policy 1, policy_version 954934 (0.0006) [2023-12-27 12:16:10,111][20347] Updated weights for policy 0, policy_version 955791 (0.0006) [2023-12-27 12:16:10,682][20381] Updated weights for policy 1, policy_version 954944 (0.0010) [2023-12-27 12:16:10,743][20381] Updated weights for policy 1, policy_version 954954 (0.0011) [2023-12-27 12:16:10,801][20381] Updated weights for policy 1, policy_version 954964 (0.0011) [2023-12-27 12:16:10,828][20347] Updated weights for policy 0, policy_version 955801 (0.0007) [2023-12-27 12:16:10,887][20347] Updated weights for policy 0, policy_version 955811 (0.0007) [2023-12-27 12:16:10,940][20347] Updated weights for policy 0, policy_version 955821 (0.0006) [2023-12-27 12:16:11,001][20347] Updated weights for policy 0, policy_version 955831 (0.0007) [2023-12-27 12:16:11,499][20381] Updated weights for policy 1, policy_version 954974 (0.0008) [2023-12-27 12:16:11,559][20381] Updated weights for policy 1, policy_version 954984 (0.0007) [2023-12-27 12:16:11,629][20381] Updated weights for policy 1, policy_version 954994 (0.0008) [2023-12-27 12:16:11,780][20347] Updated weights for policy 0, policy_version 955841 (0.0008) [2023-12-27 12:16:11,837][20347] Updated weights for policy 0, policy_version 955851 (0.0007) [2023-12-27 12:16:11,892][20347] Updated weights for policy 0, policy_version 955861 (0.0007) [2023-12-27 12:16:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 489250816. Throughput: 0: 10118.4, 1: 10169.5. Samples: 489245604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:12,363][19170] Avg episode reward: [(0, '864.270'), (1, '990.250')] [2023-12-27 12:16:12,374][20381] Updated weights for policy 1, policy_version 955004 (0.0009) [2023-12-27 12:16:12,438][20381] Updated weights for policy 1, policy_version 955014 (0.0008) [2023-12-27 12:16:12,495][20381] Updated weights for policy 1, policy_version 955024 (0.0008) [2023-12-27 12:16:12,587][20347] Updated weights for policy 0, policy_version 955871 (0.0007) [2023-12-27 12:16:12,639][20347] Updated weights for policy 0, policy_version 955881 (0.0008) [2023-12-27 12:16:12,697][20347] Updated weights for policy 0, policy_version 955891 (0.0006) [2023-12-27 12:16:13,197][20381] Updated weights for policy 1, policy_version 955034 (0.0008) [2023-12-27 12:16:13,251][20381] Updated weights for policy 1, policy_version 955044 (0.0010) [2023-12-27 12:16:13,302][20381] Updated weights for policy 1, policy_version 955054 (0.0010) [2023-12-27 12:16:13,324][20292] KL-divergence is very high: 199.7378 [2023-12-27 12:16:13,357][20381] Updated weights for policy 1, policy_version 955064 (0.0010) [2023-12-27 12:16:13,377][20347] Updated weights for policy 0, policy_version 955901 (0.0007) [2023-12-27 12:16:13,437][20347] Updated weights for policy 0, policy_version 955911 (0.0005) [2023-12-27 12:16:13,508][20347] Updated weights for policy 0, policy_version 955921 (0.0005) [2023-12-27 12:16:14,009][20381] Updated weights for policy 1, policy_version 955074 (0.0008) [2023-12-27 12:16:14,028][20347] Updated weights for policy 0, policy_version 955931 (0.0005) [2023-12-27 12:16:14,064][20381] Updated weights for policy 1, policy_version 955084 (0.0007) [2023-12-27 12:16:14,082][20347] Updated weights for policy 0, policy_version 955941 (0.0005) [2023-12-27 12:16:14,119][20381] Updated weights for policy 1, policy_version 955094 (0.0007) [2023-12-27 12:16:14,138][20347] Updated weights for policy 0, policy_version 955951 (0.0008) [2023-12-27 12:16:14,745][20347] Updated weights for policy 0, policy_version 955961 (0.0007) [2023-12-27 12:16:14,806][20347] Updated weights for policy 0, policy_version 955971 (0.0010) [2023-12-27 12:16:14,814][20381] Updated weights for policy 1, policy_version 955104 (0.0008) [2023-12-27 12:16:14,865][20347] Updated weights for policy 0, policy_version 955981 (0.0010) [2023-12-27 12:16:14,867][20381] Updated weights for policy 1, policy_version 955114 (0.0006) [2023-12-27 12:16:14,915][20347] Updated weights for policy 0, policy_version 955991 (0.0010) [2023-12-27 12:16:14,925][20381] Updated weights for policy 1, policy_version 955124 (0.0006) [2023-12-27 12:16:15,604][20381] Updated weights for policy 1, policy_version 955134 (0.0006) [2023-12-27 12:16:15,606][20347] Updated weights for policy 0, policy_version 956001 (0.0010) [2023-12-27 12:16:15,649][20381] Updated weights for policy 1, policy_version 955144 (0.0007) [2023-12-27 12:16:15,663][20347] Updated weights for policy 0, policy_version 956011 (0.0010) [2023-12-27 12:16:15,698][20381] Updated weights for policy 1, policy_version 955154 (0.0005) [2023-12-27 12:16:15,714][20347] Updated weights for policy 0, policy_version 956021 (0.0010) [2023-12-27 12:16:16,299][20347] Updated weights for policy 0, policy_version 956031 (0.0007) [2023-12-27 12:16:16,358][20347] Updated weights for policy 0, policy_version 956041 (0.0005) [2023-12-27 12:16:16,422][20347] Updated weights for policy 0, policy_version 956051 (0.0006) [2023-12-27 12:16:16,428][20381] Updated weights for policy 1, policy_version 955164 (0.0006) [2023-12-27 12:16:16,484][20381] Updated weights for policy 1, policy_version 955174 (0.0005) [2023-12-27 12:16:16,535][20381] Updated weights for policy 1, policy_version 955184 (0.0005) [2023-12-27 12:16:16,991][20347] Updated weights for policy 0, policy_version 956061 (0.0006) [2023-12-27 12:16:17,042][20347] Updated weights for policy 0, policy_version 956071 (0.0010) [2023-12-27 12:16:17,093][20347] Updated weights for policy 0, policy_version 956081 (0.0010) [2023-12-27 12:16:17,159][20381] Updated weights for policy 1, policy_version 955194 (0.0006) [2023-12-27 12:16:17,218][20381] Updated weights for policy 1, policy_version 955204 (0.0007) [2023-12-27 12:16:17,271][20381] Updated weights for policy 1, policy_version 955214 (0.0007) [2023-12-27 12:16:17,324][20381] Updated weights for policy 1, policy_version 955224 (0.0008) [2023-12-27 12:16:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 489365504. Throughput: 0: 10223.6, 1: 10210.2. Samples: 489370596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:17,363][19170] Avg episode reward: [(0, '836.220'), (1, '990.250')] [2023-12-27 12:16:17,778][20347] Updated weights for policy 0, policy_version 956091 (0.0009) [2023-12-27 12:16:17,834][20347] Updated weights for policy 0, policy_version 956101 (0.0008) [2023-12-27 12:16:17,893][20347] Updated weights for policy 0, policy_version 956111 (0.0005) [2023-12-27 12:16:18,039][20381] Updated weights for policy 1, policy_version 955234 (0.0008) [2023-12-27 12:16:18,084][20381] Updated weights for policy 1, policy_version 955244 (0.0008) [2023-12-27 12:16:18,139][20381] Updated weights for policy 1, policy_version 955254 (0.0008) [2023-12-27 12:16:18,523][20347] Updated weights for policy 0, policy_version 956121 (0.0006) [2023-12-27 12:16:18,583][20347] Updated weights for policy 0, policy_version 956131 (0.0008) [2023-12-27 12:16:18,647][20347] Updated weights for policy 0, policy_version 956141 (0.0008) [2023-12-27 12:16:18,702][20347] Updated weights for policy 0, policy_version 956151 (0.0007) [2023-12-27 12:16:18,865][20381] Updated weights for policy 1, policy_version 955264 (0.0007) [2023-12-27 12:16:18,912][20381] Updated weights for policy 1, policy_version 955274 (0.0008) [2023-12-27 12:16:18,958][20381] Updated weights for policy 1, policy_version 955284 (0.0007) [2023-12-27 12:16:19,376][20347] Updated weights for policy 0, policy_version 956161 (0.0008) [2023-12-27 12:16:19,434][20347] Updated weights for policy 0, policy_version 956171 (0.0006) [2023-12-27 12:16:19,490][20347] Updated weights for policy 0, policy_version 956181 (0.0008) [2023-12-27 12:16:19,655][20381] Updated weights for policy 1, policy_version 955294 (0.0007) [2023-12-27 12:16:19,718][20381] Updated weights for policy 1, policy_version 955304 (0.0006) [2023-12-27 12:16:19,783][20381] Updated weights for policy 1, policy_version 955314 (0.0008) [2023-12-27 12:16:20,150][20347] Updated weights for policy 0, policy_version 956191 (0.0008) [2023-12-27 12:16:20,210][20347] Updated weights for policy 0, policy_version 956201 (0.0007) [2023-12-27 12:16:20,269][20347] Updated weights for policy 0, policy_version 956211 (0.0007) [2023-12-27 12:16:20,451][20381] Updated weights for policy 1, policy_version 955324 (0.0007) [2023-12-27 12:16:20,506][20381] Updated weights for policy 1, policy_version 955334 (0.0007) [2023-12-27 12:16:20,561][20381] Updated weights for policy 1, policy_version 955344 (0.0006) [2023-12-27 12:16:20,984][20347] Updated weights for policy 0, policy_version 956221 (0.0008) [2023-12-27 12:16:21,047][20347] Updated weights for policy 0, policy_version 956231 (0.0007) [2023-12-27 12:16:21,056][20291] KL-divergence is very high: 455.0373 [2023-12-27 12:16:21,082][20291] KL-divergence is very high: 139.3756 [2023-12-27 12:16:21,107][20291] KL-divergence is very high: 308.2536 [2023-12-27 12:16:21,112][20347] Updated weights for policy 0, policy_version 956241 (0.0008) [2023-12-27 12:16:21,265][20381] Updated weights for policy 1, policy_version 955354 (0.0008) [2023-12-27 12:16:21,327][20381] Updated weights for policy 1, policy_version 955364 (0.0008) [2023-12-27 12:16:21,398][20381] Updated weights for policy 1, policy_version 955374 (0.0008) [2023-12-27 12:16:21,455][20381] Updated weights for policy 1, policy_version 955384 (0.0007) [2023-12-27 12:16:21,937][20347] Updated weights for policy 0, policy_version 956251 (0.0009) [2023-12-27 12:16:22,004][20347] Updated weights for policy 0, policy_version 956261 (0.0010) [2023-12-27 12:16:22,032][20291] KL-divergence is very high: 126.1049 [2023-12-27 12:16:22,070][20347] Updated weights for policy 0, policy_version 956271 (0.0010) [2023-12-27 12:16:22,161][20381] Updated weights for policy 1, policy_version 955394 (0.0006) [2023-12-27 12:16:22,224][20381] Updated weights for policy 1, policy_version 955404 (0.0007) [2023-12-27 12:16:22,282][20381] Updated weights for policy 1, policy_version 955414 (0.0008) [2023-12-27 12:16:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 489463808. Throughput: 0: 10215.1, 1: 10111.5. Samples: 489431516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:22,363][19170] Avg episode reward: [(0, '836.190'), (1, '990.250')] [2023-12-27 12:16:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000955416_244613120.pth... [2023-12-27 12:16:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000956280_244850688.pth... [2023-12-27 12:16:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000954232_244310016.pth [2023-12-27 12:16:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000955064_244539392.pth [2023-12-27 12:16:22,792][20347] Updated weights for policy 0, policy_version 956281 (0.0010) [2023-12-27 12:16:22,850][20347] Updated weights for policy 0, policy_version 956291 (0.0007) [2023-12-27 12:16:22,907][20347] Updated weights for policy 0, policy_version 956301 (0.0008) [2023-12-27 12:16:22,962][20347] Updated weights for policy 0, policy_version 956311 (0.0009) [2023-12-27 12:16:22,993][20381] Updated weights for policy 1, policy_version 955424 (0.0007) [2023-12-27 12:16:23,049][20381] Updated weights for policy 1, policy_version 955434 (0.0007) [2023-12-27 12:16:23,111][20381] Updated weights for policy 1, policy_version 955444 (0.0007) [2023-12-27 12:16:23,655][20347] Updated weights for policy 0, policy_version 956321 (0.0008) [2023-12-27 12:16:23,710][20347] Updated weights for policy 0, policy_version 956331 (0.0007) [2023-12-27 12:16:23,727][20381] Updated weights for policy 1, policy_version 955454 (0.0007) [2023-12-27 12:16:23,766][20347] Updated weights for policy 0, policy_version 956341 (0.0006) [2023-12-27 12:16:23,775][20381] Updated weights for policy 1, policy_version 955464 (0.0005) [2023-12-27 12:16:23,823][20381] Updated weights for policy 1, policy_version 955474 (0.0008) [2023-12-27 12:16:24,348][20347] Updated weights for policy 0, policy_version 956351 (0.0007) [2023-12-27 12:16:24,397][20347] Updated weights for policy 0, policy_version 956361 (0.0007) [2023-12-27 12:16:24,446][20347] Updated weights for policy 0, policy_version 956371 (0.0006) [2023-12-27 12:16:24,497][20381] Updated weights for policy 1, policy_version 955484 (0.0007) [2023-12-27 12:16:24,555][20381] Updated weights for policy 1, policy_version 955494 (0.0007) [2023-12-27 12:16:24,606][20381] Updated weights for policy 1, policy_version 955504 (0.0007) [2023-12-27 12:16:25,149][20347] Updated weights for policy 0, policy_version 956381 (0.0006) [2023-12-27 12:16:25,200][20347] Updated weights for policy 0, policy_version 956391 (0.0007) [2023-12-27 12:16:25,257][20381] Updated weights for policy 1, policy_version 955514 (0.0005) [2023-12-27 12:16:25,258][20347] Updated weights for policy 0, policy_version 956401 (0.0007) [2023-12-27 12:16:25,303][20381] Updated weights for policy 1, policy_version 955524 (0.0006) [2023-12-27 12:16:25,353][20381] Updated weights for policy 1, policy_version 955534 (0.0006) [2023-12-27 12:16:25,401][20381] Updated weights for policy 1, policy_version 955544 (0.0008) [2023-12-27 12:16:25,949][20347] Updated weights for policy 0, policy_version 956411 (0.0007) [2023-12-27 12:16:26,006][20347] Updated weights for policy 0, policy_version 956421 (0.0008) [2023-12-27 12:16:26,054][20347] Updated weights for policy 0, policy_version 956431 (0.0008) [2023-12-27 12:16:26,158][20381] Updated weights for policy 1, policy_version 955554 (0.0008) [2023-12-27 12:16:26,223][20381] Updated weights for policy 1, policy_version 955564 (0.0006) [2023-12-27 12:16:26,270][20381] Updated weights for policy 1, policy_version 955574 (0.0005) [2023-12-27 12:16:26,755][20347] Updated weights for policy 0, policy_version 956441 (0.0007) [2023-12-27 12:16:26,801][20347] Updated weights for policy 0, policy_version 956451 (0.0005) [2023-12-27 12:16:26,856][20347] Updated weights for policy 0, policy_version 956461 (0.0006) [2023-12-27 12:16:26,899][20381] Updated weights for policy 1, policy_version 955584 (0.0007) [2023-12-27 12:16:26,909][20347] Updated weights for policy 0, policy_version 956471 (0.0005) [2023-12-27 12:16:26,953][20381] Updated weights for policy 1, policy_version 955594 (0.0007) [2023-12-27 12:16:27,008][20381] Updated weights for policy 1, policy_version 955604 (0.0005) [2023-12-27 12:16:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 489562112. Throughput: 0: 10157.4, 1: 10135.9. Samples: 489551612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:27,363][19170] Avg episode reward: [(0, '836.760'), (1, '1000.000')] [2023-12-27 12:16:27,508][20347] Updated weights for policy 0, policy_version 956481 (0.0007) [2023-12-27 12:16:27,559][20347] Updated weights for policy 0, policy_version 956491 (0.0006) [2023-12-27 12:16:27,571][20381] Updated weights for policy 1, policy_version 955614 (0.0007) [2023-12-27 12:16:27,610][20347] Updated weights for policy 0, policy_version 956501 (0.0006) [2023-12-27 12:16:27,617][20381] Updated weights for policy 1, policy_version 955624 (0.0006) [2023-12-27 12:16:27,669][20381] Updated weights for policy 1, policy_version 955634 (0.0008) [2023-12-27 12:16:28,249][20381] Updated weights for policy 1, policy_version 955644 (0.0006) [2023-12-27 12:16:28,251][20347] Updated weights for policy 0, policy_version 956511 (0.0008) [2023-12-27 12:16:28,299][20381] Updated weights for policy 1, policy_version 955654 (0.0005) [2023-12-27 12:16:28,300][20347] Updated weights for policy 0, policy_version 956521 (0.0008) [2023-12-27 12:16:28,354][20347] Updated weights for policy 0, policy_version 956531 (0.0007) [2023-12-27 12:16:28,355][20381] Updated weights for policy 1, policy_version 955664 (0.0007) [2023-12-27 12:16:28,976][20381] Updated weights for policy 1, policy_version 955674 (0.0008) [2023-12-27 12:16:29,026][20347] Updated weights for policy 0, policy_version 956541 (0.0007) [2023-12-27 12:16:29,031][20381] Updated weights for policy 1, policy_version 955684 (0.0011) [2023-12-27 12:16:29,076][20381] Updated weights for policy 1, policy_version 955694 (0.0010) [2023-12-27 12:16:29,084][20347] Updated weights for policy 0, policy_version 956551 (0.0006) [2023-12-27 12:16:29,130][20381] Updated weights for policy 1, policy_version 955704 (0.0010) [2023-12-27 12:16:29,138][20347] Updated weights for policy 0, policy_version 956561 (0.0007) [2023-12-27 12:16:29,802][20347] Updated weights for policy 0, policy_version 956571 (0.0007) [2023-12-27 12:16:29,863][20347] Updated weights for policy 0, policy_version 956581 (0.0007) [2023-12-27 12:16:29,867][20381] Updated weights for policy 1, policy_version 955714 (0.0007) [2023-12-27 12:16:29,908][20347] Updated weights for policy 0, policy_version 956591 (0.0007) [2023-12-27 12:16:29,925][20381] Updated weights for policy 1, policy_version 955724 (0.0007) [2023-12-27 12:16:29,986][20381] Updated weights for policy 1, policy_version 955734 (0.0009) [2023-12-27 12:16:30,569][20347] Updated weights for policy 0, policy_version 956601 (0.0008) [2023-12-27 12:16:30,630][20347] Updated weights for policy 0, policy_version 956611 (0.0008) [2023-12-27 12:16:30,634][20381] Updated weights for policy 1, policy_version 955744 (0.0008) [2023-12-27 12:16:30,678][20347] Updated weights for policy 0, policy_version 956621 (0.0009) [2023-12-27 12:16:30,687][20381] Updated weights for policy 1, policy_version 955754 (0.0006) [2023-12-27 12:16:30,724][20347] Updated weights for policy 0, policy_version 956631 (0.0005) [2023-12-27 12:16:30,738][20381] Updated weights for policy 1, policy_version 955764 (0.0006) [2023-12-27 12:16:31,417][20347] Updated weights for policy 0, policy_version 956641 (0.0007) [2023-12-27 12:16:31,474][20381] Updated weights for policy 1, policy_version 955774 (0.0007) [2023-12-27 12:16:31,480][20347] Updated weights for policy 0, policy_version 956651 (0.0006) [2023-12-27 12:16:31,536][20381] Updated weights for policy 1, policy_version 955784 (0.0006) [2023-12-27 12:16:31,543][20347] Updated weights for policy 0, policy_version 956661 (0.0006) [2023-12-27 12:16:31,602][20381] Updated weights for policy 1, policy_version 955794 (0.0006) [2023-12-27 12:16:32,148][20347] Updated weights for policy 0, policy_version 956671 (0.0007) [2023-12-27 12:16:32,205][20347] Updated weights for policy 0, policy_version 956681 (0.0008) [2023-12-27 12:16:32,257][20381] Updated weights for policy 1, policy_version 955804 (0.0008) [2023-12-27 12:16:32,264][20347] Updated weights for policy 0, policy_version 956691 (0.0006) [2023-12-27 12:16:32,312][20381] Updated weights for policy 1, policy_version 955814 (0.0007) [2023-12-27 12:16:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 489668608. Throughput: 0: 10286.2, 1: 10228.4. Samples: 489679928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:32,363][19170] Avg episode reward: [(0, '836.820'), (1, '1000.000')] [2023-12-27 12:16:32,372][20381] Updated weights for policy 1, policy_version 955824 (0.0007) [2023-12-27 12:16:32,890][20347] Updated weights for policy 0, policy_version 956701 (0.0006) [2023-12-27 12:16:32,943][20347] Updated weights for policy 0, policy_version 956711 (0.0007) [2023-12-27 12:16:32,996][20347] Updated weights for policy 0, policy_version 956721 (0.0007) [2023-12-27 12:16:33,077][20381] Updated weights for policy 1, policy_version 955834 (0.0007) [2023-12-27 12:16:33,131][20381] Updated weights for policy 1, policy_version 955844 (0.0005) [2023-12-27 12:16:33,190][20381] Updated weights for policy 1, policy_version 955854 (0.0006) [2023-12-27 12:16:33,246][20381] Updated weights for policy 1, policy_version 955864 (0.0008) [2023-12-27 12:16:33,692][20347] Updated weights for policy 0, policy_version 956731 (0.0008) [2023-12-27 12:16:33,744][20347] Updated weights for policy 0, policy_version 956741 (0.0006) [2023-12-27 12:16:33,795][20347] Updated weights for policy 0, policy_version 956751 (0.0006) [2023-12-27 12:16:33,834][20381] Updated weights for policy 1, policy_version 955874 (0.0010) [2023-12-27 12:16:33,882][20381] Updated weights for policy 1, policy_version 955884 (0.0010) [2023-12-27 12:16:33,937][20381] Updated weights for policy 1, policy_version 955894 (0.0010) [2023-12-27 12:16:34,459][20347] Updated weights for policy 0, policy_version 956761 (0.0006) [2023-12-27 12:16:34,511][20347] Updated weights for policy 0, policy_version 956771 (0.0008) [2023-12-27 12:16:34,560][20347] Updated weights for policy 0, policy_version 956781 (0.0007) [2023-12-27 12:16:34,615][20347] Updated weights for policy 0, policy_version 956791 (0.0007) [2023-12-27 12:16:34,646][20381] Updated weights for policy 1, policy_version 955904 (0.0010) [2023-12-27 12:16:34,710][20381] Updated weights for policy 1, policy_version 955914 (0.0010) [2023-12-27 12:16:34,768][20381] Updated weights for policy 1, policy_version 955924 (0.0009) [2023-12-27 12:16:35,273][20347] Updated weights for policy 0, policy_version 956801 (0.0007) [2023-12-27 12:16:35,317][20347] Updated weights for policy 0, policy_version 956811 (0.0006) [2023-12-27 12:16:35,362][20347] Updated weights for policy 0, policy_version 956821 (0.0007) [2023-12-27 12:16:35,439][20381] Updated weights for policy 1, policy_version 955934 (0.0010) [2023-12-27 12:16:35,492][20381] Updated weights for policy 1, policy_version 955944 (0.0010) [2023-12-27 12:16:35,546][20381] Updated weights for policy 1, policy_version 955954 (0.0010) [2023-12-27 12:16:36,086][20347] Updated weights for policy 0, policy_version 956831 (0.0007) [2023-12-27 12:16:36,110][20381] Updated weights for policy 1, policy_version 955964 (0.0010) [2023-12-27 12:16:36,142][20347] Updated weights for policy 0, policy_version 956841 (0.0007) [2023-12-27 12:16:36,171][20381] Updated weights for policy 1, policy_version 955974 (0.0010) [2023-12-27 12:16:36,207][20347] Updated weights for policy 0, policy_version 956851 (0.0006) [2023-12-27 12:16:36,232][20381] Updated weights for policy 1, policy_version 955984 (0.0011) [2023-12-27 12:16:36,883][20347] Updated weights for policy 0, policy_version 956861 (0.0007) [2023-12-27 12:16:36,942][20347] Updated weights for policy 0, policy_version 956871 (0.0007) [2023-12-27 12:16:36,944][20381] Updated weights for policy 1, policy_version 955994 (0.0010) [2023-12-27 12:16:36,994][20347] Updated weights for policy 0, policy_version 956881 (0.0005) [2023-12-27 12:16:37,007][20381] Updated weights for policy 1, policy_version 956004 (0.0007) [2023-12-27 12:16:37,070][20381] Updated weights for policy 1, policy_version 956014 (0.0008) [2023-12-27 12:16:37,125][20381] Updated weights for policy 1, policy_version 956024 (0.0008) [2023-12-27 12:16:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 489775104. Throughput: 0: 10323.7, 1: 10206.6. Samples: 489742544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:37,363][19170] Avg episode reward: [(0, '845.470'), (1, '1000.000')] [2023-12-27 12:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000956024_244768768.pth... [2023-12-27 12:16:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000956888_245006336.pth... [2023-12-27 12:16:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000955672_244695040.pth [2023-12-27 12:16:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000954808_244457472.pth [2023-12-27 12:16:37,691][20347] Updated weights for policy 0, policy_version 956891 (0.0006) [2023-12-27 12:16:37,753][20347] Updated weights for policy 0, policy_version 956901 (0.0007) [2023-12-27 12:16:37,811][20347] Updated weights for policy 0, policy_version 956911 (0.0007) [2023-12-27 12:16:37,823][20381] Updated weights for policy 1, policy_version 956034 (0.0007) [2023-12-27 12:16:37,888][20381] Updated weights for policy 1, policy_version 956044 (0.0008) [2023-12-27 12:16:37,945][20381] Updated weights for policy 1, policy_version 956054 (0.0007) [2023-12-27 12:16:38,430][20347] Updated weights for policy 0, policy_version 956921 (0.0005) [2023-12-27 12:16:38,484][20347] Updated weights for policy 0, policy_version 956931 (0.0007) [2023-12-27 12:16:38,539][20347] Updated weights for policy 0, policy_version 956941 (0.0005) [2023-12-27 12:16:38,592][20347] Updated weights for policy 0, policy_version 956951 (0.0005) [2023-12-27 12:16:38,662][20381] Updated weights for policy 1, policy_version 956064 (0.0008) [2023-12-27 12:16:38,724][20381] Updated weights for policy 1, policy_version 956074 (0.0008) [2023-12-27 12:16:38,787][20381] Updated weights for policy 1, policy_version 956084 (0.0007) [2023-12-27 12:16:39,264][20347] Updated weights for policy 0, policy_version 956961 (0.0006) [2023-12-27 12:16:39,325][20347] Updated weights for policy 0, policy_version 956971 (0.0009) [2023-12-27 12:16:39,388][20347] Updated weights for policy 0, policy_version 956981 (0.0007) [2023-12-27 12:16:39,549][20381] Updated weights for policy 1, policy_version 956094 (0.0008) [2023-12-27 12:16:39,609][20381] Updated weights for policy 1, policy_version 956104 (0.0008) [2023-12-27 12:16:39,668][20381] Updated weights for policy 1, policy_version 956114 (0.0008) [2023-12-27 12:16:40,130][20347] Updated weights for policy 0, policy_version 956991 (0.0008) [2023-12-27 12:16:40,189][20347] Updated weights for policy 0, policy_version 957001 (0.0007) [2023-12-27 12:16:40,251][20347] Updated weights for policy 0, policy_version 957011 (0.0007) [2023-12-27 12:16:40,371][20381] Updated weights for policy 1, policy_version 956124 (0.0009) [2023-12-27 12:16:40,427][20381] Updated weights for policy 1, policy_version 956134 (0.0010) [2023-12-27 12:16:40,486][20381] Updated weights for policy 1, policy_version 956144 (0.0010) [2023-12-27 12:16:40,922][20347] Updated weights for policy 0, policy_version 957021 (0.0008) [2023-12-27 12:16:40,970][20347] Updated weights for policy 0, policy_version 957031 (0.0008) [2023-12-27 12:16:41,023][20347] Updated weights for policy 0, policy_version 957041 (0.0008) [2023-12-27 12:16:41,171][20381] Updated weights for policy 1, policy_version 956154 (0.0008) [2023-12-27 12:16:41,234][20381] Updated weights for policy 1, policy_version 956164 (0.0006) [2023-12-27 12:16:41,294][20381] Updated weights for policy 1, policy_version 956174 (0.0007) [2023-12-27 12:16:41,353][20381] Updated weights for policy 1, policy_version 956184 (0.0006) [2023-12-27 12:16:41,820][20347] Updated weights for policy 0, policy_version 957051 (0.0008) [2023-12-27 12:16:41,878][20347] Updated weights for policy 0, policy_version 957061 (0.0008) [2023-12-27 12:16:41,943][20347] Updated weights for policy 0, policy_version 957071 (0.0008) [2023-12-27 12:16:42,024][20381] Updated weights for policy 1, policy_version 956194 (0.0007) [2023-12-27 12:16:42,085][20381] Updated weights for policy 1, policy_version 956204 (0.0005) [2023-12-27 12:16:42,145][20381] Updated weights for policy 1, policy_version 956214 (0.0007) [2023-12-27 12:16:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 489873408. Throughput: 0: 10312.7, 1: 10189.5. Samples: 489862732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:42,363][19170] Avg episode reward: [(0, '863.860'), (1, '1000.000')] [2023-12-27 12:16:42,643][20347] Updated weights for policy 0, policy_version 957081 (0.0006) [2023-12-27 12:16:42,703][20347] Updated weights for policy 0, policy_version 957091 (0.0008) [2023-12-27 12:16:42,759][20347] Updated weights for policy 0, policy_version 957101 (0.0007) [2023-12-27 12:16:42,794][20381] Updated weights for policy 1, policy_version 956224 (0.0008) [2023-12-27 12:16:42,820][20347] Updated weights for policy 0, policy_version 957111 (0.0007) [2023-12-27 12:16:42,842][20381] Updated weights for policy 1, policy_version 956234 (0.0006) [2023-12-27 12:16:42,886][20381] Updated weights for policy 1, policy_version 956244 (0.0005) [2023-12-27 12:16:43,499][20347] Updated weights for policy 0, policy_version 957121 (0.0008) [2023-12-27 12:16:43,508][20381] Updated weights for policy 1, policy_version 956254 (0.0005) [2023-12-27 12:16:43,556][20381] Updated weights for policy 1, policy_version 956264 (0.0005) [2023-12-27 12:16:43,558][20347] Updated weights for policy 0, policy_version 957131 (0.0007) [2023-12-27 12:16:43,615][20381] Updated weights for policy 1, policy_version 956274 (0.0006) [2023-12-27 12:16:43,622][20347] Updated weights for policy 0, policy_version 957141 (0.0008) [2023-12-27 12:16:44,167][20381] Updated weights for policy 1, policy_version 956284 (0.0008) [2023-12-27 12:16:44,222][20381] Updated weights for policy 1, policy_version 956294 (0.0006) [2023-12-27 12:16:44,278][20381] Updated weights for policy 1, policy_version 956304 (0.0006) [2023-12-27 12:16:44,278][20347] Updated weights for policy 0, policy_version 957151 (0.0010) [2023-12-27 12:16:44,335][20347] Updated weights for policy 0, policy_version 957161 (0.0009) [2023-12-27 12:16:44,389][20347] Updated weights for policy 0, policy_version 957171 (0.0010) [2023-12-27 12:16:44,930][20381] Updated weights for policy 1, policy_version 956314 (0.0009) [2023-12-27 12:16:44,985][20381] Updated weights for policy 1, policy_version 956324 (0.0009) [2023-12-27 12:16:45,045][20381] Updated weights for policy 1, policy_version 956334 (0.0008) [2023-12-27 12:16:45,055][20347] Updated weights for policy 0, policy_version 957181 (0.0009) [2023-12-27 12:16:45,103][20381] Updated weights for policy 1, policy_version 956344 (0.0009) [2023-12-27 12:16:45,110][20347] Updated weights for policy 0, policy_version 957191 (0.0007) [2023-12-27 12:16:45,166][20347] Updated weights for policy 0, policy_version 957201 (0.0008) [2023-12-27 12:16:45,782][20381] Updated weights for policy 1, policy_version 956354 (0.0006) [2023-12-27 12:16:45,836][20381] Updated weights for policy 1, policy_version 956364 (0.0005) [2023-12-27 12:16:45,857][20347] Updated weights for policy 0, policy_version 957211 (0.0007) [2023-12-27 12:16:45,891][20381] Updated weights for policy 1, policy_version 956374 (0.0005) [2023-12-27 12:16:45,912][20347] Updated weights for policy 0, policy_version 957221 (0.0007) [2023-12-27 12:16:45,973][20347] Updated weights for policy 0, policy_version 957231 (0.0006) [2023-12-27 12:16:46,473][20381] Updated weights for policy 1, policy_version 956384 (0.0005) [2023-12-27 12:16:46,521][20381] Updated weights for policy 1, policy_version 956394 (0.0006) [2023-12-27 12:16:46,545][20347] Updated weights for policy 0, policy_version 957241 (0.0006) [2023-12-27 12:16:46,567][20381] Updated weights for policy 1, policy_version 956404 (0.0010) [2023-12-27 12:16:46,601][20347] Updated weights for policy 0, policy_version 957251 (0.0007) [2023-12-27 12:16:46,655][20347] Updated weights for policy 0, policy_version 957261 (0.0007) [2023-12-27 12:16:46,704][20347] Updated weights for policy 0, policy_version 957271 (0.0008) [2023-12-27 12:16:47,231][20381] Updated weights for policy 1, policy_version 956414 (0.0010) [2023-12-27 12:16:47,276][20381] Updated weights for policy 1, policy_version 956424 (0.0010) [2023-12-27 12:16:47,331][20381] Updated weights for policy 1, policy_version 956434 (0.0010) [2023-12-27 12:16:47,337][20347] Updated weights for policy 0, policy_version 957281 (0.0006) [2023-12-27 12:16:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 489979904. Throughput: 0: 10306.4, 1: 10270.8. Samples: 489988300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:47,364][19170] Avg episode reward: [(0, '921.950'), (1, '1000.000')] [2023-12-27 12:16:47,387][20347] Updated weights for policy 0, policy_version 957291 (0.0006) [2023-12-27 12:16:47,432][20347] Updated weights for policy 0, policy_version 957301 (0.0007) [2023-12-27 12:16:48,010][20347] Updated weights for policy 0, policy_version 957311 (0.0007) [2023-12-27 12:16:48,020][20381] Updated weights for policy 1, policy_version 956444 (0.0011) [2023-12-27 12:16:48,063][20347] Updated weights for policy 0, policy_version 957321 (0.0006) [2023-12-27 12:16:48,076][20381] Updated weights for policy 1, policy_version 956454 (0.0011) [2023-12-27 12:16:48,111][20347] Updated weights for policy 0, policy_version 957331 (0.0005) [2023-12-27 12:16:48,127][20381] Updated weights for policy 1, policy_version 956464 (0.0010) [2023-12-27 12:16:48,715][20347] Updated weights for policy 0, policy_version 957341 (0.0005) [2023-12-27 12:16:48,761][20381] Updated weights for policy 1, policy_version 956474 (0.0010) [2023-12-27 12:16:48,774][20347] Updated weights for policy 0, policy_version 957351 (0.0006) [2023-12-27 12:16:48,821][20381] Updated weights for policy 1, policy_version 956484 (0.0007) [2023-12-27 12:16:48,831][20347] Updated weights for policy 0, policy_version 957361 (0.0006) [2023-12-27 12:16:48,882][20381] Updated weights for policy 1, policy_version 956494 (0.0008) [2023-12-27 12:16:48,938][20381] Updated weights for policy 1, policy_version 956504 (0.0008) [2023-12-27 12:16:49,422][20347] Updated weights for policy 0, policy_version 957371 (0.0006) [2023-12-27 12:16:49,487][20347] Updated weights for policy 0, policy_version 957381 (0.0008) [2023-12-27 12:16:49,530][20381] Updated weights for policy 1, policy_version 956514 (0.0006) [2023-12-27 12:16:49,538][20347] Updated weights for policy 0, policy_version 957391 (0.0007) [2023-12-27 12:16:49,584][20381] Updated weights for policy 1, policy_version 956524 (0.0006) [2023-12-27 12:16:49,638][20381] Updated weights for policy 1, policy_version 956534 (0.0005) [2023-12-27 12:16:50,286][20381] Updated weights for policy 1, policy_version 956544 (0.0008) [2023-12-27 12:16:50,323][20347] Updated weights for policy 0, policy_version 957401 (0.0007) [2023-12-27 12:16:50,345][20381] Updated weights for policy 1, policy_version 956554 (0.0007) [2023-12-27 12:16:50,385][20347] Updated weights for policy 0, policy_version 957411 (0.0006) [2023-12-27 12:16:50,409][20381] Updated weights for policy 1, policy_version 956564 (0.0008) [2023-12-27 12:16:50,446][20347] Updated weights for policy 0, policy_version 957421 (0.0006) [2023-12-27 12:16:50,459][20291] KL-divergence is very high: 153.8804 [2023-12-27 12:16:50,501][20347] Updated weights for policy 0, policy_version 957431 (0.0008) [2023-12-27 12:16:51,072][20381] Updated weights for policy 1, policy_version 956574 (0.0007) [2023-12-27 12:16:51,131][20381] Updated weights for policy 1, policy_version 956584 (0.0008) [2023-12-27 12:16:51,192][20381] Updated weights for policy 1, policy_version 956594 (0.0008) [2023-12-27 12:16:51,204][20347] Updated weights for policy 0, policy_version 957441 (0.0007) [2023-12-27 12:16:51,258][20347] Updated weights for policy 0, policy_version 957451 (0.0007) [2023-12-27 12:16:51,314][20347] Updated weights for policy 0, policy_version 957461 (0.0008) [2023-12-27 12:16:51,906][20381] Updated weights for policy 1, policy_version 956604 (0.0007) [2023-12-27 12:16:51,967][20381] Updated weights for policy 1, policy_version 956614 (0.0007) [2023-12-27 12:16:52,025][20381] Updated weights for policy 1, policy_version 956624 (0.0007) [2023-12-27 12:16:52,080][20347] Updated weights for policy 0, policy_version 957471 (0.0007) [2023-12-27 12:16:52,136][20347] Updated weights for policy 0, policy_version 957481 (0.0008) [2023-12-27 12:16:52,191][20347] Updated weights for policy 0, policy_version 957491 (0.0007) [2023-12-27 12:16:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 490086400. Throughput: 0: 10338.1, 1: 10302.6. Samples: 490053456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:52,363][19170] Avg episode reward: [(0, '886.460'), (1, '1000.000')] [2023-12-27 12:16:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000956632_244924416.pth... [2023-12-27 12:16:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000957496_245161984.pth... [2023-12-27 12:16:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000956280_244850688.pth [2023-12-27 12:16:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000955416_244613120.pth [2023-12-27 12:16:52,695][20381] Updated weights for policy 1, policy_version 956634 (0.0008) [2023-12-27 12:16:52,761][20381] Updated weights for policy 1, policy_version 956644 (0.0008) [2023-12-27 12:16:52,820][20381] Updated weights for policy 1, policy_version 956654 (0.0007) [2023-12-27 12:16:52,879][20381] Updated weights for policy 1, policy_version 956664 (0.0005) [2023-12-27 12:16:52,941][20347] Updated weights for policy 0, policy_version 957501 (0.0008) [2023-12-27 12:16:52,994][20347] Updated weights for policy 0, policy_version 957511 (0.0008) [2023-12-27 12:16:53,058][20347] Updated weights for policy 0, policy_version 957521 (0.0008) [2023-12-27 12:16:53,490][20381] Updated weights for policy 1, policy_version 956674 (0.0008) [2023-12-27 12:16:53,539][20381] Updated weights for policy 1, policy_version 956684 (0.0008) [2023-12-27 12:16:53,585][20381] Updated weights for policy 1, policy_version 956694 (0.0007) [2023-12-27 12:16:53,747][20347] Updated weights for policy 0, policy_version 957531 (0.0007) [2023-12-27 12:16:53,803][20347] Updated weights for policy 0, policy_version 957541 (0.0007) [2023-12-27 12:16:53,859][20347] Updated weights for policy 0, policy_version 957551 (0.0009) [2023-12-27 12:16:54,232][20381] Updated weights for policy 1, policy_version 956704 (0.0006) [2023-12-27 12:16:54,295][20381] Updated weights for policy 1, policy_version 956714 (0.0005) [2023-12-27 12:16:54,347][20381] Updated weights for policy 1, policy_version 956724 (0.0006) [2023-12-27 12:16:54,592][20347] Updated weights for policy 0, policy_version 957561 (0.0007) [2023-12-27 12:16:54,649][20347] Updated weights for policy 0, policy_version 957571 (0.0007) [2023-12-27 12:16:54,697][20347] Updated weights for policy 0, policy_version 957581 (0.0008) [2023-12-27 12:16:54,746][20347] Updated weights for policy 0, policy_version 957591 (0.0009) [2023-12-27 12:16:54,897][20381] Updated weights for policy 1, policy_version 956734 (0.0005) [2023-12-27 12:16:54,950][20381] Updated weights for policy 1, policy_version 956744 (0.0005) [2023-12-27 12:16:55,010][20381] Updated weights for policy 1, policy_version 956754 (0.0007) [2023-12-27 12:16:55,514][20347] Updated weights for policy 0, policy_version 957601 (0.0006) [2023-12-27 12:16:55,575][20347] Updated weights for policy 0, policy_version 957611 (0.0005) [2023-12-27 12:16:55,632][20347] Updated weights for policy 0, policy_version 957621 (0.0005) [2023-12-27 12:16:55,657][20381] Updated weights for policy 1, policy_version 956764 (0.0011) [2023-12-27 12:16:55,717][20381] Updated weights for policy 1, policy_version 956774 (0.0010) [2023-12-27 12:16:55,775][20381] Updated weights for policy 1, policy_version 956784 (0.0010) [2023-12-27 12:16:56,177][20347] Updated weights for policy 0, policy_version 957631 (0.0007) [2023-12-27 12:16:56,236][20347] Updated weights for policy 0, policy_version 957641 (0.0007) [2023-12-27 12:16:56,299][20347] Updated weights for policy 0, policy_version 957651 (0.0006) [2023-12-27 12:16:56,485][20381] Updated weights for policy 1, policy_version 956794 (0.0010) [2023-12-27 12:16:56,547][20381] Updated weights for policy 1, policy_version 956804 (0.0006) [2023-12-27 12:16:56,616][20381] Updated weights for policy 1, policy_version 956814 (0.0006) [2023-12-27 12:16:56,670][20381] Updated weights for policy 1, policy_version 956824 (0.0008) [2023-12-27 12:16:56,957][20347] Updated weights for policy 0, policy_version 957661 (0.0008) [2023-12-27 12:16:57,006][20347] Updated weights for policy 0, policy_version 957671 (0.0007) [2023-12-27 12:16:57,056][20347] Updated weights for policy 0, policy_version 957681 (0.0007) [2023-12-27 12:16:57,302][20381] Updated weights for policy 1, policy_version 956834 (0.0011) [2023-12-27 12:16:57,356][20381] Updated weights for policy 1, policy_version 956844 (0.0008) [2023-12-27 12:16:57,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 490184704. Throughput: 0: 10313.3, 1: 10356.6. Samples: 490175752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:16:57,363][19170] Avg episode reward: [(0, '879.180'), (1, '1000.000')] [2023-12-27 12:16:57,410][20381] Updated weights for policy 1, policy_version 956854 (0.0010) [2023-12-27 12:16:57,662][20347] Updated weights for policy 0, policy_version 957691 (0.0007) [2023-12-27 12:16:57,714][20347] Updated weights for policy 0, policy_version 957701 (0.0006) [2023-12-27 12:16:57,773][20347] Updated weights for policy 0, policy_version 957711 (0.0005) [2023-12-27 12:16:58,016][20381] Updated weights for policy 1, policy_version 956864 (0.0010) [2023-12-27 12:16:58,073][20381] Updated weights for policy 1, policy_version 956874 (0.0010) [2023-12-27 12:16:58,127][20381] Updated weights for policy 1, policy_version 956884 (0.0009) [2023-12-27 12:16:58,370][20347] Updated weights for policy 0, policy_version 957721 (0.0005) [2023-12-27 12:16:58,433][20347] Updated weights for policy 0, policy_version 957731 (0.0008) [2023-12-27 12:16:58,499][20347] Updated weights for policy 0, policy_version 957741 (0.0011) [2023-12-27 12:16:58,559][20347] Updated weights for policy 0, policy_version 957751 (0.0011) [2023-12-27 12:16:58,894][20381] Updated weights for policy 1, policy_version 956894 (0.0006) [2023-12-27 12:16:58,954][20381] Updated weights for policy 1, policy_version 956904 (0.0008) [2023-12-27 12:16:59,019][20381] Updated weights for policy 1, policy_version 956914 (0.0007) [2023-12-27 12:16:59,259][20347] Updated weights for policy 0, policy_version 957761 (0.0007) [2023-12-27 12:16:59,322][20347] Updated weights for policy 0, policy_version 957771 (0.0007) [2023-12-27 12:16:59,390][20347] Updated weights for policy 0, policy_version 957781 (0.0008) [2023-12-27 12:16:59,690][20381] Updated weights for policy 1, policy_version 956924 (0.0007) [2023-12-27 12:16:59,742][20381] Updated weights for policy 1, policy_version 956934 (0.0008) [2023-12-27 12:16:59,798][20381] Updated weights for policy 1, policy_version 956944 (0.0008) [2023-12-27 12:17:00,040][20347] Updated weights for policy 0, policy_version 957791 (0.0009) [2023-12-27 12:17:00,102][20347] Updated weights for policy 0, policy_version 957801 (0.0007) [2023-12-27 12:17:00,158][20347] Updated weights for policy 0, policy_version 957811 (0.0006) [2023-12-27 12:17:00,488][20381] Updated weights for policy 1, policy_version 956954 (0.0008) [2023-12-27 12:17:00,548][20381] Updated weights for policy 1, policy_version 956964 (0.0008) [2023-12-27 12:17:00,609][20381] Updated weights for policy 1, policy_version 956974 (0.0008) [2023-12-27 12:17:00,659][20381] Updated weights for policy 1, policy_version 956984 (0.0007) [2023-12-27 12:17:00,754][20347] Updated weights for policy 0, policy_version 957821 (0.0006) [2023-12-27 12:17:00,801][20347] Updated weights for policy 0, policy_version 957831 (0.0005) [2023-12-27 12:17:00,846][20347] Updated weights for policy 0, policy_version 957841 (0.0005) [2023-12-27 12:17:01,417][20381] Updated weights for policy 1, policy_version 956994 (0.0008) [2023-12-27 12:17:01,424][20292] KL-divergence is very high: 380.9196 [2023-12-27 12:17:01,477][20381] Updated weights for policy 1, policy_version 957004 (0.0007) [2023-12-27 12:17:01,504][20347] Updated weights for policy 0, policy_version 957851 (0.0005) [2023-12-27 12:17:01,515][20292] KL-divergence is very high: 407.5694 [2023-12-27 12:17:01,533][20381] Updated weights for policy 1, policy_version 957014 (0.0008) [2023-12-27 12:17:01,563][20347] Updated weights for policy 0, policy_version 957861 (0.0005) [2023-12-27 12:17:01,630][20347] Updated weights for policy 0, policy_version 957871 (0.0007) [2023-12-27 12:17:02,211][20347] Updated weights for policy 0, policy_version 957881 (0.0007) [2023-12-27 12:17:02,274][20347] Updated weights for policy 0, policy_version 957891 (0.0007) [2023-12-27 12:17:02,317][20381] Updated weights for policy 1, policy_version 957024 (0.0007) [2023-12-27 12:17:02,327][20347] Updated weights for policy 0, policy_version 957901 (0.0006) [2023-12-27 12:17:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 490283008. Throughput: 0: 10305.2, 1: 10343.1. Samples: 490299772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:02,363][19170] Avg episode reward: [(0, '912.860'), (1, '990.250')] [2023-12-27 12:17:02,378][20381] Updated weights for policy 1, policy_version 957034 (0.0008) [2023-12-27 12:17:02,394][20347] Updated weights for policy 0, policy_version 957911 (0.0007) [2023-12-27 12:17:02,436][20381] Updated weights for policy 1, policy_version 957044 (0.0007) [2023-12-27 12:17:03,079][20347] Updated weights for policy 0, policy_version 957921 (0.0007) [2023-12-27 12:17:03,128][20347] Updated weights for policy 0, policy_version 957931 (0.0006) [2023-12-27 12:17:03,130][20381] Updated weights for policy 1, policy_version 957054 (0.0006) [2023-12-27 12:17:03,184][20347] Updated weights for policy 0, policy_version 957941 (0.0006) [2023-12-27 12:17:03,186][20381] Updated weights for policy 1, policy_version 957064 (0.0006) [2023-12-27 12:17:03,241][20381] Updated weights for policy 1, policy_version 957074 (0.0007) [2023-12-27 12:17:03,866][20347] Updated weights for policy 0, policy_version 957951 (0.0007) [2023-12-27 12:17:03,926][20347] Updated weights for policy 0, policy_version 957961 (0.0007) [2023-12-27 12:17:03,944][20381] Updated weights for policy 1, policy_version 957084 (0.0007) [2023-12-27 12:17:03,980][20347] Updated weights for policy 0, policy_version 957971 (0.0008) [2023-12-27 12:17:04,001][20381] Updated weights for policy 1, policy_version 957094 (0.0005) [2023-12-27 12:17:04,060][20381] Updated weights for policy 1, policy_version 957104 (0.0008) [2023-12-27 12:17:04,683][20347] Updated weights for policy 0, policy_version 957981 (0.0009) [2023-12-27 12:17:04,723][20381] Updated weights for policy 1, policy_version 957114 (0.0007) [2023-12-27 12:17:04,733][20347] Updated weights for policy 0, policy_version 957991 (0.0008) [2023-12-27 12:17:04,772][20381] Updated weights for policy 1, policy_version 957124 (0.0007) [2023-12-27 12:17:04,785][20347] Updated weights for policy 0, policy_version 958001 (0.0008) [2023-12-27 12:17:04,823][20381] Updated weights for policy 1, policy_version 957134 (0.0005) [2023-12-27 12:17:04,876][20381] Updated weights for policy 1, policy_version 957144 (0.0007) [2023-12-27 12:17:05,386][20347] Updated weights for policy 0, policy_version 958011 (0.0010) [2023-12-27 12:17:05,443][20347] Updated weights for policy 0, policy_version 958021 (0.0010) [2023-12-27 12:17:05,504][20347] Updated weights for policy 0, policy_version 958031 (0.0010) [2023-12-27 12:17:05,513][20381] Updated weights for policy 1, policy_version 957154 (0.0005) [2023-12-27 12:17:05,568][20381] Updated weights for policy 1, policy_version 957164 (0.0009) [2023-12-27 12:17:05,626][20381] Updated weights for policy 1, policy_version 957174 (0.0005) [2023-12-27 12:17:06,195][20347] Updated weights for policy 0, policy_version 958041 (0.0010) [2023-12-27 12:17:06,240][20381] Updated weights for policy 1, policy_version 957184 (0.0006) [2023-12-27 12:17:06,256][20347] Updated weights for policy 0, policy_version 958051 (0.0008) [2023-12-27 12:17:06,307][20381] Updated weights for policy 1, policy_version 957194 (0.0007) [2023-12-27 12:17:06,310][20347] Updated weights for policy 0, policy_version 958061 (0.0006) [2023-12-27 12:17:06,368][20347] Updated weights for policy 0, policy_version 958071 (0.0007) [2023-12-27 12:17:06,370][20381] Updated weights for policy 1, policy_version 957204 (0.0007) [2023-12-27 12:17:07,025][20381] Updated weights for policy 1, policy_version 957214 (0.0006) [2023-12-27 12:17:07,078][20381] Updated weights for policy 1, policy_version 957224 (0.0006) [2023-12-27 12:17:07,097][20347] Updated weights for policy 0, policy_version 958081 (0.0008) [2023-12-27 12:17:07,135][20381] Updated weights for policy 1, policy_version 957234 (0.0007) [2023-12-27 12:17:07,158][20347] Updated weights for policy 0, policy_version 958091 (0.0007) [2023-12-27 12:17:07,212][20347] Updated weights for policy 0, policy_version 958101 (0.0007) [2023-12-27 12:17:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 490397696. Throughput: 0: 10311.3, 1: 10359.3. Samples: 490361700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:07,364][19170] Avg episode reward: [(0, '900.960'), (1, '990.250')] [2023-12-27 12:17:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000958104_245317632.pth... [2023-12-27 12:17:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000957240_245080064.pth... [2023-12-27 12:17:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000956888_245006336.pth [2023-12-27 12:17:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000956024_244768768.pth [2023-12-27 12:17:07,805][20381] Updated weights for policy 1, policy_version 957244 (0.0006) [2023-12-27 12:17:07,853][20381] Updated weights for policy 1, policy_version 957254 (0.0008) [2023-12-27 12:17:07,900][20347] Updated weights for policy 0, policy_version 958111 (0.0006) [2023-12-27 12:17:07,913][20381] Updated weights for policy 1, policy_version 957264 (0.0007) [2023-12-27 12:17:07,961][20347] Updated weights for policy 0, policy_version 958121 (0.0006) [2023-12-27 12:17:08,022][20347] Updated weights for policy 0, policy_version 958131 (0.0005) [2023-12-27 12:17:08,628][20381] Updated weights for policy 1, policy_version 957274 (0.0006) [2023-12-27 12:17:08,652][20347] Updated weights for policy 0, policy_version 958141 (0.0006) [2023-12-27 12:17:08,689][20381] Updated weights for policy 1, policy_version 957284 (0.0007) [2023-12-27 12:17:08,709][20347] Updated weights for policy 0, policy_version 958151 (0.0008) [2023-12-27 12:17:08,746][20381] Updated weights for policy 1, policy_version 957294 (0.0006) [2023-12-27 12:17:08,768][20347] Updated weights for policy 0, policy_version 958161 (0.0007) [2023-12-27 12:17:08,803][20381] Updated weights for policy 1, policy_version 957304 (0.0010) [2023-12-27 12:17:09,477][20347] Updated weights for policy 0, policy_version 958171 (0.0009) [2023-12-27 12:17:09,497][20381] Updated weights for policy 1, policy_version 957314 (0.0007) [2023-12-27 12:17:09,538][20347] Updated weights for policy 0, policy_version 958181 (0.0007) [2023-12-27 12:17:09,559][20381] Updated weights for policy 1, policy_version 957324 (0.0007) [2023-12-27 12:17:09,598][20347] Updated weights for policy 0, policy_version 958191 (0.0006) [2023-12-27 12:17:09,622][20381] Updated weights for policy 1, policy_version 957334 (0.0008) [2023-12-27 12:17:10,283][20381] Updated weights for policy 1, policy_version 957344 (0.0006) [2023-12-27 12:17:10,347][20381] Updated weights for policy 1, policy_version 957354 (0.0006) [2023-12-27 12:17:10,350][20347] Updated weights for policy 0, policy_version 958201 (0.0007) [2023-12-27 12:17:10,405][20381] Updated weights for policy 1, policy_version 957364 (0.0006) [2023-12-27 12:17:10,414][20347] Updated weights for policy 0, policy_version 958211 (0.0006) [2023-12-27 12:17:10,475][20347] Updated weights for policy 0, policy_version 958221 (0.0007) [2023-12-27 12:17:10,534][20347] Updated weights for policy 0, policy_version 958231 (0.0008) [2023-12-27 12:17:11,067][20381] Updated weights for policy 1, policy_version 957374 (0.0006) [2023-12-27 12:17:11,132][20381] Updated weights for policy 1, policy_version 957384 (0.0006) [2023-12-27 12:17:11,202][20381] Updated weights for policy 1, policy_version 957394 (0.0008) [2023-12-27 12:17:11,262][20347] Updated weights for policy 0, policy_version 958241 (0.0007) [2023-12-27 12:17:11,324][20347] Updated weights for policy 0, policy_version 958251 (0.0008) [2023-12-27 12:17:11,395][20347] Updated weights for policy 0, policy_version 958261 (0.0008) [2023-12-27 12:17:12,010][20381] Updated weights for policy 1, policy_version 957404 (0.0009) [2023-12-27 12:17:12,028][20347] Updated weights for policy 0, policy_version 958271 (0.0008) [2023-12-27 12:17:12,068][20381] Updated weights for policy 1, policy_version 957414 (0.0008) [2023-12-27 12:17:12,085][20347] Updated weights for policy 0, policy_version 958281 (0.0006) [2023-12-27 12:17:12,124][20381] Updated weights for policy 1, policy_version 957424 (0.0006) [2023-12-27 12:17:12,137][20347] Updated weights for policy 0, policy_version 958291 (0.0005) [2023-12-27 12:17:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 490496000. Throughput: 0: 10310.9, 1: 10376.1. Samples: 490482528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:12,363][19170] Avg episode reward: [(0, '835.750'), (1, '1000.000')] [2023-12-27 12:17:12,842][20347] Updated weights for policy 0, policy_version 958301 (0.0007) [2023-12-27 12:17:12,861][20381] Updated weights for policy 1, policy_version 957434 (0.0006) [2023-12-27 12:17:12,893][20347] Updated weights for policy 0, policy_version 958311 (0.0006) [2023-12-27 12:17:12,917][20381] Updated weights for policy 1, policy_version 957444 (0.0007) [2023-12-27 12:17:12,947][20347] Updated weights for policy 0, policy_version 958321 (0.0008) [2023-12-27 12:17:12,968][20381] Updated weights for policy 1, policy_version 957454 (0.0006) [2023-12-27 12:17:13,021][20381] Updated weights for policy 1, policy_version 957464 (0.0007) [2023-12-27 12:17:13,647][20347] Updated weights for policy 0, policy_version 958331 (0.0005) [2023-12-27 12:17:13,683][20381] Updated weights for policy 1, policy_version 957474 (0.0005) [2023-12-27 12:17:13,701][20347] Updated weights for policy 0, policy_version 958341 (0.0005) [2023-12-27 12:17:13,727][20381] Updated weights for policy 1, policy_version 957484 (0.0005) [2023-12-27 12:17:13,757][20347] Updated weights for policy 0, policy_version 958351 (0.0006) [2023-12-27 12:17:13,772][20381] Updated weights for policy 1, policy_version 957494 (0.0008) [2023-12-27 12:17:14,284][20347] Updated weights for policy 0, policy_version 958361 (0.0006) [2023-12-27 12:17:14,337][20347] Updated weights for policy 0, policy_version 958371 (0.0008) [2023-12-27 12:17:14,358][20381] Updated weights for policy 1, policy_version 957504 (0.0009) [2023-12-27 12:17:14,401][20347] Updated weights for policy 0, policy_version 958381 (0.0008) [2023-12-27 12:17:14,412][20381] Updated weights for policy 1, policy_version 957514 (0.0007) [2023-12-27 12:17:14,458][20347] Updated weights for policy 0, policy_version 958391 (0.0007) [2023-12-27 12:17:14,463][20381] Updated weights for policy 1, policy_version 957524 (0.0005) [2023-12-27 12:17:15,070][20347] Updated weights for policy 0, policy_version 958401 (0.0010) [2023-12-27 12:17:15,078][20381] Updated weights for policy 1, policy_version 957534 (0.0007) [2023-12-27 12:17:15,133][20347] Updated weights for policy 0, policy_version 958411 (0.0008) [2023-12-27 12:17:15,148][20381] Updated weights for policy 1, policy_version 957544 (0.0008) [2023-12-27 12:17:15,193][20347] Updated weights for policy 0, policy_version 958421 (0.0011) [2023-12-27 12:17:15,203][20381] Updated weights for policy 1, policy_version 957554 (0.0009) [2023-12-27 12:17:15,812][20347] Updated weights for policy 0, policy_version 958431 (0.0007) [2023-12-27 12:17:15,877][20347] Updated weights for policy 0, policy_version 958441 (0.0006) [2023-12-27 12:17:15,881][20381] Updated weights for policy 1, policy_version 957564 (0.0007) [2023-12-27 12:17:15,936][20381] Updated weights for policy 1, policy_version 957574 (0.0010) [2023-12-27 12:17:15,936][20347] Updated weights for policy 0, policy_version 958451 (0.0010) [2023-12-27 12:17:15,988][20381] Updated weights for policy 1, policy_version 957584 (0.0010) [2023-12-27 12:17:16,551][20347] Updated weights for policy 0, policy_version 958461 (0.0010) [2023-12-27 12:17:16,610][20347] Updated weights for policy 0, policy_version 958471 (0.0010) [2023-12-27 12:17:16,665][20347] Updated weights for policy 0, policy_version 958481 (0.0010) [2023-12-27 12:17:16,678][20381] Updated weights for policy 1, policy_version 957594 (0.0010) [2023-12-27 12:17:16,727][20381] Updated weights for policy 1, policy_version 957604 (0.0010) [2023-12-27 12:17:16,776][20381] Updated weights for policy 1, policy_version 957614 (0.0010) [2023-12-27 12:17:16,824][20381] Updated weights for policy 1, policy_version 957624 (0.0009) [2023-12-27 12:17:17,311][20347] Updated weights for policy 0, policy_version 958491 (0.0010) [2023-12-27 12:17:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 490594304. Throughput: 0: 10315.2, 1: 10341.2. Samples: 490609464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:17,363][19170] Avg episode reward: [(0, '875.140'), (1, '990.270')] [2023-12-27 12:17:17,372][20347] Updated weights for policy 0, policy_version 958501 (0.0008) [2023-12-27 12:17:17,441][20347] Updated weights for policy 0, policy_version 958511 (0.0009) [2023-12-27 12:17:17,445][20381] Updated weights for policy 1, policy_version 957634 (0.0011) [2023-12-27 12:17:17,501][20381] Updated weights for policy 1, policy_version 957644 (0.0010) [2023-12-27 12:17:17,554][20381] Updated weights for policy 1, policy_version 957654 (0.0006) [2023-12-27 12:17:18,077][20347] Updated weights for policy 0, policy_version 958521 (0.0010) [2023-12-27 12:17:18,142][20347] Updated weights for policy 0, policy_version 958531 (0.0011) [2023-12-27 12:17:18,164][20381] Updated weights for policy 1, policy_version 957664 (0.0007) [2023-12-27 12:17:18,208][20347] Updated weights for policy 0, policy_version 958541 (0.0011) [2023-12-27 12:17:18,225][20381] Updated weights for policy 1, policy_version 957674 (0.0010) [2023-12-27 12:17:18,275][20347] Updated weights for policy 0, policy_version 958551 (0.0011) [2023-12-27 12:17:18,288][20381] Updated weights for policy 1, policy_version 957684 (0.0008) [2023-12-27 12:17:18,977][20381] Updated weights for policy 1, policy_version 957695 (0.0010) [2023-12-27 12:17:19,001][20347] Updated weights for policy 0, policy_version 958561 (0.0009) [2023-12-27 12:17:19,033][20381] Updated weights for policy 1, policy_version 957705 (0.0011) [2023-12-27 12:17:19,064][20347] Updated weights for policy 0, policy_version 958571 (0.0010) [2023-12-27 12:17:19,095][20381] Updated weights for policy 1, policy_version 957715 (0.0011) [2023-12-27 12:17:19,120][20347] Updated weights for policy 0, policy_version 958581 (0.0010) [2023-12-27 12:17:19,893][20347] Updated weights for policy 0, policy_version 958591 (0.0009) [2023-12-27 12:17:19,895][20381] Updated weights for policy 1, policy_version 957725 (0.0008) [2023-12-27 12:17:19,965][20347] Updated weights for policy 0, policy_version 958601 (0.0009) [2023-12-27 12:17:19,980][20381] Updated weights for policy 1, policy_version 957735 (0.0007) [2023-12-27 12:17:20,026][20347] Updated weights for policy 0, policy_version 958611 (0.0008) [2023-12-27 12:17:20,045][20381] Updated weights for policy 1, policy_version 957745 (0.0007) [2023-12-27 12:17:20,669][20381] Updated weights for policy 1, policy_version 957755 (0.0006) [2023-12-27 12:17:20,726][20381] Updated weights for policy 1, policy_version 957765 (0.0006) [2023-12-27 12:17:20,781][20381] Updated weights for policy 1, policy_version 957775 (0.0008) [2023-12-27 12:17:20,801][20347] Updated weights for policy 0, policy_version 958621 (0.0007) [2023-12-27 12:17:20,859][20347] Updated weights for policy 0, policy_version 958631 (0.0006) [2023-12-27 12:17:20,923][20347] Updated weights for policy 0, policy_version 958641 (0.0006) [2023-12-27 12:17:21,495][20381] Updated weights for policy 1, policy_version 957785 (0.0006) [2023-12-27 12:17:21,560][20381] Updated weights for policy 1, policy_version 957795 (0.0007) [2023-12-27 12:17:21,626][20381] Updated weights for policy 1, policy_version 957805 (0.0008) [2023-12-27 12:17:21,691][20381] Updated weights for policy 1, policy_version 957815 (0.0008) [2023-12-27 12:17:21,691][20347] Updated weights for policy 0, policy_version 958651 (0.0006) [2023-12-27 12:17:21,765][20347] Updated weights for policy 0, policy_version 958661 (0.0008) [2023-12-27 12:17:21,836][20347] Updated weights for policy 0, policy_version 958671 (0.0008) [2023-12-27 12:17:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 490692608. Throughput: 0: 10247.5, 1: 10321.3. Samples: 490668144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:22,363][19170] Avg episode reward: [(0, '922.060'), (1, '980.540')] [2023-12-27 12:17:22,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000958680_245465088.pth... [2023-12-27 12:17:22,386][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000957496_245161984.pth [2023-12-27 12:17:22,415][20381] Updated weights for policy 1, policy_version 957825 (0.0009) [2023-12-27 12:17:22,474][20381] Updated weights for policy 1, policy_version 957835 (0.0008) [2023-12-27 12:17:22,527][20381] Updated weights for policy 1, policy_version 957845 (0.0008) [2023-12-27 12:17:22,539][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000957848_245235712.pth... [2023-12-27 12:17:22,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000956632_244924416.pth [2023-12-27 12:17:22,597][20347] Updated weights for policy 0, policy_version 958681 (0.0009) [2023-12-27 12:17:22,647][20347] Updated weights for policy 0, policy_version 958691 (0.0008) [2023-12-27 12:17:22,693][20291] KL-divergence is very high: 109.7825 [2023-12-27 12:17:22,699][20347] Updated weights for policy 0, policy_version 958701 (0.0008) [2023-12-27 12:17:22,761][20347] Updated weights for policy 0, policy_version 958711 (0.0008) [2023-12-27 12:17:23,264][20381] Updated weights for policy 1, policy_version 957855 (0.0006) [2023-12-27 12:17:23,319][20381] Updated weights for policy 1, policy_version 957865 (0.0007) [2023-12-27 12:17:23,377][20381] Updated weights for policy 1, policy_version 957875 (0.0008) [2023-12-27 12:17:23,468][20347] Updated weights for policy 0, policy_version 958721 (0.0008) [2023-12-27 12:17:23,519][20347] Updated weights for policy 0, policy_version 958731 (0.0007) [2023-12-27 12:17:23,587][20347] Updated weights for policy 0, policy_version 958741 (0.0008) [2023-12-27 12:17:24,044][20381] Updated weights for policy 1, policy_version 957885 (0.0007) [2023-12-27 12:17:24,097][20381] Updated weights for policy 1, policy_version 957895 (0.0007) [2023-12-27 12:17:24,157][20381] Updated weights for policy 1, policy_version 957905 (0.0008) [2023-12-27 12:17:24,309][20347] Updated weights for policy 0, policy_version 958751 (0.0008) [2023-12-27 12:17:24,378][20347] Updated weights for policy 0, policy_version 958761 (0.0007) [2023-12-27 12:17:24,427][20347] Updated weights for policy 0, policy_version 958771 (0.0008) [2023-12-27 12:17:24,820][20381] Updated weights for policy 1, policy_version 957915 (0.0007) [2023-12-27 12:17:24,870][20381] Updated weights for policy 1, policy_version 957925 (0.0008) [2023-12-27 12:17:24,919][20381] Updated weights for policy 1, policy_version 957935 (0.0007) [2023-12-27 12:17:25,081][20347] Updated weights for policy 0, policy_version 958781 (0.0007) [2023-12-27 12:17:25,128][20347] Updated weights for policy 0, policy_version 958791 (0.0006) [2023-12-27 12:17:25,180][20347] Updated weights for policy 0, policy_version 958801 (0.0008) [2023-12-27 12:17:25,657][20381] Updated weights for policy 1, policy_version 957945 (0.0009) [2023-12-27 12:17:25,713][20381] Updated weights for policy 1, policy_version 957955 (0.0008) [2023-12-27 12:17:25,769][20381] Updated weights for policy 1, policy_version 957965 (0.0008) [2023-12-27 12:17:25,815][20381] Updated weights for policy 1, policy_version 957975 (0.0008) [2023-12-27 12:17:25,849][20347] Updated weights for policy 0, policy_version 958811 (0.0008) [2023-12-27 12:17:25,894][20347] Updated weights for policy 0, policy_version 958821 (0.0007) [2023-12-27 12:17:25,944][20347] Updated weights for policy 0, policy_version 958831 (0.0007) [2023-12-27 12:17:26,575][20381] Updated weights for policy 1, policy_version 957985 (0.0009) [2023-12-27 12:17:26,636][20381] Updated weights for policy 1, policy_version 957995 (0.0008) [2023-12-27 12:17:26,647][20347] Updated weights for policy 0, policy_version 958841 (0.0008) [2023-12-27 12:17:26,696][20381] Updated weights for policy 1, policy_version 958005 (0.0006) [2023-12-27 12:17:26,712][20347] Updated weights for policy 0, policy_version 958851 (0.0011) [2023-12-27 12:17:26,775][20347] Updated weights for policy 0, policy_version 958861 (0.0010) [2023-12-27 12:17:26,841][20347] Updated weights for policy 0, policy_version 958871 (0.0010) [2023-12-27 12:17:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 490790912. Throughput: 0: 10204.1, 1: 10298.1. Samples: 490785332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:27,363][19170] Avg episode reward: [(0, '883.300'), (1, '990.270')] [2023-12-27 12:17:27,395][20381] Updated weights for policy 1, policy_version 958015 (0.0006) [2023-12-27 12:17:27,450][20381] Updated weights for policy 1, policy_version 958025 (0.0007) [2023-12-27 12:17:27,506][20381] Updated weights for policy 1, policy_version 958035 (0.0006) [2023-12-27 12:17:27,533][20347] Updated weights for policy 0, policy_version 958881 (0.0010) [2023-12-27 12:17:27,578][20347] Updated weights for policy 0, policy_version 958891 (0.0010) [2023-12-27 12:17:27,637][20347] Updated weights for policy 0, policy_version 958901 (0.0010) [2023-12-27 12:17:28,218][20381] Updated weights for policy 1, policy_version 958045 (0.0006) [2023-12-27 12:17:28,281][20381] Updated weights for policy 1, policy_version 958055 (0.0007) [2023-12-27 12:17:28,287][20347] Updated weights for policy 0, policy_version 958911 (0.0008) [2023-12-27 12:17:28,341][20347] Updated weights for policy 0, policy_version 958921 (0.0007) [2023-12-27 12:17:28,351][20381] Updated weights for policy 1, policy_version 958065 (0.0007) [2023-12-27 12:17:28,405][20347] Updated weights for policy 0, policy_version 958931 (0.0007) [2023-12-27 12:17:29,050][20381] Updated weights for policy 1, policy_version 958075 (0.0007) [2023-12-27 12:17:29,111][20347] Updated weights for policy 0, policy_version 958941 (0.0006) [2023-12-27 12:17:29,112][20381] Updated weights for policy 1, policy_version 958085 (0.0008) [2023-12-27 12:17:29,167][20347] Updated weights for policy 0, policy_version 958951 (0.0005) [2023-12-27 12:17:29,174][20381] Updated weights for policy 1, policy_version 958095 (0.0008) [2023-12-27 12:17:29,235][20347] Updated weights for policy 0, policy_version 958961 (0.0007) [2023-12-27 12:17:29,813][20347] Updated weights for policy 0, policy_version 958971 (0.0007) [2023-12-27 12:17:29,882][20347] Updated weights for policy 0, policy_version 958981 (0.0008) [2023-12-27 12:17:29,911][20381] Updated weights for policy 1, policy_version 958105 (0.0009) [2023-12-27 12:17:29,947][20347] Updated weights for policy 0, policy_version 958991 (0.0008) [2023-12-27 12:17:29,969][20381] Updated weights for policy 1, policy_version 958115 (0.0006) [2023-12-27 12:17:30,032][20381] Updated weights for policy 1, policy_version 958125 (0.0007) [2023-12-27 12:17:30,091][20381] Updated weights for policy 1, policy_version 958135 (0.0007) [2023-12-27 12:17:30,655][20347] Updated weights for policy 0, policy_version 959001 (0.0008) [2023-12-27 12:17:30,707][20347] Updated weights for policy 0, policy_version 959011 (0.0005) [2023-12-27 12:17:30,764][20347] Updated weights for policy 0, policy_version 959021 (0.0008) [2023-12-27 12:17:30,788][20381] Updated weights for policy 1, policy_version 958145 (0.0006) [2023-12-27 12:17:30,816][20347] Updated weights for policy 0, policy_version 959031 (0.0008) [2023-12-27 12:17:30,864][20381] Updated weights for policy 1, policy_version 958155 (0.0008) [2023-12-27 12:17:30,916][20381] Updated weights for policy 1, policy_version 958165 (0.0009) [2023-12-27 12:17:31,476][20347] Updated weights for policy 0, policy_version 959041 (0.0009) [2023-12-27 12:17:31,538][20347] Updated weights for policy 0, policy_version 959051 (0.0005) [2023-12-27 12:17:31,597][20347] Updated weights for policy 0, policy_version 959061 (0.0005) [2023-12-27 12:17:31,665][20381] Updated weights for policy 1, policy_version 958175 (0.0009) [2023-12-27 12:17:31,727][20381] Updated weights for policy 1, policy_version 958185 (0.0008) [2023-12-27 12:17:31,806][20381] Updated weights for policy 1, policy_version 958195 (0.0008) [2023-12-27 12:17:32,259][20347] Updated weights for policy 0, policy_version 959071 (0.0009) [2023-12-27 12:17:32,322][20347] Updated weights for policy 0, policy_version 959081 (0.0011) [2023-12-27 12:17:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 490889216. Throughput: 0: 10224.4, 1: 10142.0. Samples: 490904784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:32,363][19170] Avg episode reward: [(0, '919.520'), (1, '990.270')] [2023-12-27 12:17:32,387][20347] Updated weights for policy 0, policy_version 959091 (0.0014) [2023-12-27 12:17:32,450][20381] Updated weights for policy 1, policy_version 958205 (0.0009) [2023-12-27 12:17:32,510][20381] Updated weights for policy 1, policy_version 958215 (0.0011) [2023-12-27 12:17:32,566][20381] Updated weights for policy 1, policy_version 958225 (0.0011) [2023-12-27 12:17:33,056][20347] Updated weights for policy 0, policy_version 959101 (0.0009) [2023-12-27 12:17:33,107][20347] Updated weights for policy 0, policy_version 959111 (0.0010) [2023-12-27 12:17:33,166][20347] Updated weights for policy 0, policy_version 959121 (0.0010) [2023-12-27 12:17:33,271][20381] Updated weights for policy 1, policy_version 958235 (0.0010) [2023-12-27 12:17:33,327][20381] Updated weights for policy 1, policy_version 958245 (0.0011) [2023-12-27 12:17:33,386][20381] Updated weights for policy 1, policy_version 958255 (0.0011) [2023-12-27 12:17:33,863][20347] Updated weights for policy 0, policy_version 959131 (0.0006) [2023-12-27 12:17:33,926][20347] Updated weights for policy 0, policy_version 959141 (0.0006) [2023-12-27 12:17:33,983][20347] Updated weights for policy 0, policy_version 959151 (0.0010) [2023-12-27 12:17:34,042][20381] Updated weights for policy 1, policy_version 958265 (0.0010) [2023-12-27 12:17:34,098][20381] Updated weights for policy 1, policy_version 958275 (0.0006) [2023-12-27 12:17:34,158][20381] Updated weights for policy 1, policy_version 958285 (0.0008) [2023-12-27 12:17:34,213][20381] Updated weights for policy 1, policy_version 958295 (0.0010) [2023-12-27 12:17:34,620][20347] Updated weights for policy 0, policy_version 959161 (0.0010) [2023-12-27 12:17:34,684][20347] Updated weights for policy 0, policy_version 959171 (0.0008) [2023-12-27 12:17:34,743][20347] Updated weights for policy 0, policy_version 959181 (0.0008) [2023-12-27 12:17:34,803][20347] Updated weights for policy 0, policy_version 959191 (0.0009) [2023-12-27 12:17:34,903][20381] Updated weights for policy 1, policy_version 958305 (0.0007) [2023-12-27 12:17:34,950][20381] Updated weights for policy 1, policy_version 958315 (0.0007) [2023-12-27 12:17:34,994][20381] Updated weights for policy 1, policy_version 958325 (0.0007) [2023-12-27 12:17:35,509][20347] Updated weights for policy 0, policy_version 959201 (0.0006) [2023-12-27 12:17:35,560][20347] Updated weights for policy 0, policy_version 959211 (0.0005) [2023-12-27 12:17:35,611][20347] Updated weights for policy 0, policy_version 959221 (0.0005) [2023-12-27 12:17:35,736][20381] Updated weights for policy 1, policy_version 958335 (0.0007) [2023-12-27 12:17:35,782][20381] Updated weights for policy 1, policy_version 958345 (0.0008) [2023-12-27 12:17:35,834][20381] Updated weights for policy 1, policy_version 958355 (0.0008) [2023-12-27 12:17:36,153][20347] Updated weights for policy 0, policy_version 959232 (0.0007) [2023-12-27 12:17:36,211][20347] Updated weights for policy 0, policy_version 959242 (0.0007) [2023-12-27 12:17:36,269][20347] Updated weights for policy 0, policy_version 959252 (0.0008) [2023-12-27 12:17:36,614][20381] Updated weights for policy 1, policy_version 958365 (0.0008) [2023-12-27 12:17:36,676][20381] Updated weights for policy 1, policy_version 958375 (0.0007) [2023-12-27 12:17:36,737][20381] Updated weights for policy 1, policy_version 958385 (0.0007) [2023-12-27 12:17:37,005][20347] Updated weights for policy 0, policy_version 959262 (0.0009) [2023-12-27 12:17:37,055][20347] Updated weights for policy 0, policy_version 959272 (0.0010) [2023-12-27 12:17:37,107][20347] Updated weights for policy 0, policy_version 959282 (0.0010) [2023-12-27 12:17:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 490995712. Throughput: 0: 10157.8, 1: 10094.9. Samples: 490964824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:37,363][19170] Avg episode reward: [(0, '899.540'), (1, '1000.000')] [2023-12-27 12:17:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000958392_245374976.pth... [2023-12-27 12:17:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000959288_245620736.pth... [2023-12-27 12:17:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000958104_245317632.pth [2023-12-27 12:17:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000957240_245080064.pth [2023-12-27 12:17:37,432][20381] Updated weights for policy 1, policy_version 958395 (0.0007) [2023-12-27 12:17:37,491][20381] Updated weights for policy 1, policy_version 958405 (0.0007) [2023-12-27 12:17:37,550][20381] Updated weights for policy 1, policy_version 958415 (0.0008) [2023-12-27 12:17:37,825][20347] Updated weights for policy 0, policy_version 959292 (0.0010) [2023-12-27 12:17:37,885][20347] Updated weights for policy 0, policy_version 959302 (0.0010) [2023-12-27 12:17:37,948][20347] Updated weights for policy 0, policy_version 959312 (0.0010) [2023-12-27 12:17:38,257][20381] Updated weights for policy 1, policy_version 958425 (0.0008) [2023-12-27 12:17:38,304][20381] Updated weights for policy 1, policy_version 958435 (0.0008) [2023-12-27 12:17:38,369][20381] Updated weights for policy 1, policy_version 958445 (0.0009) [2023-12-27 12:17:38,431][20381] Updated weights for policy 1, policy_version 958455 (0.0008) [2023-12-27 12:17:38,680][20347] Updated weights for policy 0, policy_version 959322 (0.0009) [2023-12-27 12:17:38,742][20347] Updated weights for policy 0, policy_version 959332 (0.0011) [2023-12-27 12:17:38,804][20347] Updated weights for policy 0, policy_version 959342 (0.0011) [2023-12-27 12:17:38,857][20347] Updated weights for policy 0, policy_version 959352 (0.0009) [2023-12-27 12:17:39,140][20381] Updated weights for policy 1, policy_version 958465 (0.0007) [2023-12-27 12:17:39,200][20381] Updated weights for policy 1, policy_version 958475 (0.0007) [2023-12-27 12:17:39,264][20381] Updated weights for policy 1, policy_version 958485 (0.0008) [2023-12-27 12:17:39,618][20347] Updated weights for policy 0, policy_version 959362 (0.0011) [2023-12-27 12:17:39,681][20347] Updated weights for policy 0, policy_version 959372 (0.0011) [2023-12-27 12:17:39,746][20347] Updated weights for policy 0, policy_version 959382 (0.0010) [2023-12-27 12:17:40,059][20381] Updated weights for policy 1, policy_version 958495 (0.0008) [2023-12-27 12:17:40,119][20381] Updated weights for policy 1, policy_version 958505 (0.0009) [2023-12-27 12:17:40,175][20381] Updated weights for policy 1, policy_version 958515 (0.0008) [2023-12-27 12:17:40,423][20347] Updated weights for policy 0, policy_version 959392 (0.0008) [2023-12-27 12:17:40,470][20347] Updated weights for policy 0, policy_version 959402 (0.0008) [2023-12-27 12:17:40,539][20347] Updated weights for policy 0, policy_version 959412 (0.0008) [2023-12-27 12:17:40,870][20381] Updated weights for policy 1, policy_version 958525 (0.0009) [2023-12-27 12:17:40,935][20381] Updated weights for policy 1, policy_version 958535 (0.0008) [2023-12-27 12:17:40,992][20381] Updated weights for policy 1, policy_version 958545 (0.0008) [2023-12-27 12:17:41,334][20347] Updated weights for policy 0, policy_version 959422 (0.0007) [2023-12-27 12:17:41,421][20347] Updated weights for policy 0, policy_version 959432 (0.0009) [2023-12-27 12:17:41,482][20347] Updated weights for policy 0, policy_version 959442 (0.0007) [2023-12-27 12:17:41,708][20381] Updated weights for policy 1, policy_version 958555 (0.0008) [2023-12-27 12:17:41,774][20381] Updated weights for policy 1, policy_version 958565 (0.0008) [2023-12-27 12:17:41,837][20381] Updated weights for policy 1, policy_version 958575 (0.0007) [2023-12-27 12:17:42,235][20347] Updated weights for policy 0, policy_version 959452 (0.0007) [2023-12-27 12:17:42,301][20347] Updated weights for policy 0, policy_version 959462 (0.0008) [2023-12-27 12:17:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 491085824. Throughput: 0: 10166.0, 1: 9969.7. Samples: 491081856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:42,363][19170] Avg episode reward: [(0, '864.150'), (1, '1000.000')] [2023-12-27 12:17:42,377][20347] Updated weights for policy 0, policy_version 959472 (0.0008) [2023-12-27 12:17:42,555][20381] Updated weights for policy 1, policy_version 958585 (0.0008) [2023-12-27 12:17:42,617][20381] Updated weights for policy 1, policy_version 958595 (0.0008) [2023-12-27 12:17:42,676][20381] Updated weights for policy 1, policy_version 958605 (0.0008) [2023-12-27 12:17:42,734][20381] Updated weights for policy 1, policy_version 958615 (0.0008) [2023-12-27 12:17:43,093][20347] Updated weights for policy 0, policy_version 959482 (0.0008) [2023-12-27 12:17:43,158][20347] Updated weights for policy 0, policy_version 959492 (0.0007) [2023-12-27 12:17:43,215][20347] Updated weights for policy 0, policy_version 959502 (0.0006) [2023-12-27 12:17:43,276][20347] Updated weights for policy 0, policy_version 959512 (0.0008) [2023-12-27 12:17:43,376][20381] Updated weights for policy 1, policy_version 958625 (0.0006) [2023-12-27 12:17:43,419][20381] Updated weights for policy 1, policy_version 958635 (0.0007) [2023-12-27 12:17:43,467][20381] Updated weights for policy 1, policy_version 958645 (0.0006) [2023-12-27 12:17:43,925][20347] Updated weights for policy 0, policy_version 959522 (0.0009) [2023-12-27 12:17:44,005][20347] Updated weights for policy 0, policy_version 959532 (0.0011) [2023-12-27 12:17:44,057][20347] Updated weights for policy 0, policy_version 959542 (0.0010) [2023-12-27 12:17:44,143][20381] Updated weights for policy 1, policy_version 958655 (0.0008) [2023-12-27 12:17:44,199][20381] Updated weights for policy 1, policy_version 958665 (0.0010) [2023-12-27 12:17:44,255][20381] Updated weights for policy 1, policy_version 958675 (0.0010) [2023-12-27 12:17:44,734][20347] Updated weights for policy 0, policy_version 959552 (0.0008) [2023-12-27 12:17:44,804][20347] Updated weights for policy 0, policy_version 959562 (0.0009) [2023-12-27 12:17:44,865][20347] Updated weights for policy 0, policy_version 959572 (0.0006) [2023-12-27 12:17:44,948][20381] Updated weights for policy 1, policy_version 958685 (0.0009) [2023-12-27 12:17:45,004][20381] Updated weights for policy 1, policy_version 958695 (0.0008) [2023-12-27 12:17:45,064][20381] Updated weights for policy 1, policy_version 958705 (0.0008) [2023-12-27 12:17:45,542][20347] Updated weights for policy 0, policy_version 959582 (0.0007) [2023-12-27 12:17:45,612][20347] Updated weights for policy 0, policy_version 959592 (0.0008) [2023-12-27 12:17:45,662][20347] Updated weights for policy 0, policy_version 959602 (0.0007) [2023-12-27 12:17:45,796][20381] Updated weights for policy 1, policy_version 958715 (0.0007) [2023-12-27 12:17:45,864][20381] Updated weights for policy 1, policy_version 958725 (0.0005) [2023-12-27 12:17:45,929][20381] Updated weights for policy 1, policy_version 958735 (0.0007) [2023-12-27 12:17:46,312][20347] Updated weights for policy 0, policy_version 959612 (0.0008) [2023-12-27 12:17:46,366][20347] Updated weights for policy 0, policy_version 959622 (0.0007) [2023-12-27 12:17:46,430][20347] Updated weights for policy 0, policy_version 959632 (0.0006) [2023-12-27 12:17:46,602][20381] Updated weights for policy 1, policy_version 958745 (0.0007) [2023-12-27 12:17:46,658][20381] Updated weights for policy 1, policy_version 958755 (0.0011) [2023-12-27 12:17:46,706][20381] Updated weights for policy 1, policy_version 958765 (0.0010) [2023-12-27 12:17:46,756][20381] Updated weights for policy 1, policy_version 958775 (0.0010) [2023-12-27 12:17:47,103][20347] Updated weights for policy 0, policy_version 959642 (0.0006) [2023-12-27 12:17:47,151][20347] Updated weights for policy 0, policy_version 959652 (0.0010) [2023-12-27 12:17:47,227][20347] Updated weights for policy 0, policy_version 959662 (0.0007) [2023-12-27 12:17:47,272][20347] Updated weights for policy 0, policy_version 959672 (0.0010) [2023-12-27 12:17:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 491192320. Throughput: 0: 10058.5, 1: 9976.7. Samples: 491201356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:47,363][19170] Avg episode reward: [(0, '737.830'), (1, '1000.000')] [2023-12-27 12:17:47,453][20381] Updated weights for policy 1, policy_version 958785 (0.0011) [2023-12-27 12:17:47,500][20381] Updated weights for policy 1, policy_version 958795 (0.0007) [2023-12-27 12:17:47,551][20381] Updated weights for policy 1, policy_version 958805 (0.0005) [2023-12-27 12:17:47,975][20347] Updated weights for policy 0, policy_version 959682 (0.0011) [2023-12-27 12:17:48,041][20347] Updated weights for policy 0, policy_version 959692 (0.0010) [2023-12-27 12:17:48,099][20347] Updated weights for policy 0, policy_version 959702 (0.0010) [2023-12-27 12:17:48,176][20381] Updated weights for policy 1, policy_version 958815 (0.0007) [2023-12-27 12:17:48,232][20381] Updated weights for policy 1, policy_version 958825 (0.0010) [2023-12-27 12:17:48,287][20381] Updated weights for policy 1, policy_version 958835 (0.0009) [2023-12-27 12:17:48,724][20347] Updated weights for policy 0, policy_version 959712 (0.0006) [2023-12-27 12:17:48,789][20347] Updated weights for policy 0, policy_version 959722 (0.0009) [2023-12-27 12:17:48,850][20347] Updated weights for policy 0, policy_version 959732 (0.0010) [2023-12-27 12:17:49,006][20381] Updated weights for policy 1, policy_version 958845 (0.0006) [2023-12-27 12:17:49,055][20381] Updated weights for policy 1, policy_version 958855 (0.0005) [2023-12-27 12:17:49,111][20381] Updated weights for policy 1, policy_version 958865 (0.0006) [2023-12-27 12:17:49,546][20347] Updated weights for policy 0, policy_version 959742 (0.0008) [2023-12-27 12:17:49,607][20347] Updated weights for policy 0, policy_version 959752 (0.0007) [2023-12-27 12:17:49,666][20347] Updated weights for policy 0, policy_version 959762 (0.0007) [2023-12-27 12:17:49,797][20381] Updated weights for policy 1, policy_version 958875 (0.0007) [2023-12-27 12:17:49,865][20381] Updated weights for policy 1, policy_version 958885 (0.0010) [2023-12-27 12:17:49,918][20381] Updated weights for policy 1, policy_version 958895 (0.0011) [2023-12-27 12:17:50,342][20347] Updated weights for policy 0, policy_version 959772 (0.0008) [2023-12-27 12:17:50,413][20347] Updated weights for policy 0, policy_version 959782 (0.0007) [2023-12-27 12:17:50,473][20347] Updated weights for policy 0, policy_version 959792 (0.0008) [2023-12-27 12:17:50,547][20381] Updated weights for policy 1, policy_version 958905 (0.0008) [2023-12-27 12:17:50,609][20381] Updated weights for policy 1, policy_version 958915 (0.0008) [2023-12-27 12:17:50,664][20381] Updated weights for policy 1, policy_version 958925 (0.0007) [2023-12-27 12:17:50,716][20381] Updated weights for policy 1, policy_version 958935 (0.0010) [2023-12-27 12:17:51,255][20347] Updated weights for policy 0, policy_version 959802 (0.0008) [2023-12-27 12:17:51,311][20347] Updated weights for policy 0, policy_version 959812 (0.0008) [2023-12-27 12:17:51,370][20347] Updated weights for policy 0, policy_version 959822 (0.0008) [2023-12-27 12:17:51,404][20381] Updated weights for policy 1, policy_version 958945 (0.0010) [2023-12-27 12:17:51,443][20347] Updated weights for policy 0, policy_version 959832 (0.0008) [2023-12-27 12:17:51,462][20381] Updated weights for policy 1, policy_version 958955 (0.0008) [2023-12-27 12:17:51,511][20381] Updated weights for policy 1, policy_version 958965 (0.0007) [2023-12-27 12:17:52,156][20347] Updated weights for policy 0, policy_version 959842 (0.0007) [2023-12-27 12:17:52,216][20347] Updated weights for policy 0, policy_version 959852 (0.0007) [2023-12-27 12:17:52,234][20381] Updated weights for policy 1, policy_version 958975 (0.0010) [2023-12-27 12:17:52,270][20347] Updated weights for policy 0, policy_version 959862 (0.0007) [2023-12-27 12:17:52,295][20381] Updated weights for policy 1, policy_version 958985 (0.0010) [2023-12-27 12:17:52,356][20381] Updated weights for policy 1, policy_version 958995 (0.0010) [2023-12-27 12:17:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 20410.6). Total num frames: 491290624. Throughput: 0: 10032.6, 1: 9981.4. Samples: 491262324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:52,363][19170] Avg episode reward: [(0, '776.440'), (1, '1000.000')] [2023-12-27 12:17:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000959864_245768192.pth... [2023-12-27 12:17:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000958680_245465088.pth [2023-12-27 12:17:52,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000959000_245530624.pth... [2023-12-27 12:17:52,388][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000957848_245235712.pth [2023-12-27 12:17:52,958][20347] Updated weights for policy 0, policy_version 959872 (0.0007) [2023-12-27 12:17:53,019][20347] Updated weights for policy 0, policy_version 959882 (0.0008) [2023-12-27 12:17:53,030][20291] KL-divergence is very high: 142.8025 [2023-12-27 12:17:53,036][20291] KL-divergence is very high: 119.9295 [2023-12-27 12:17:53,074][20347] Updated weights for policy 0, policy_version 959892 (0.0008) [2023-12-27 12:17:53,111][20381] Updated weights for policy 1, policy_version 959005 (0.0007) [2023-12-27 12:17:53,171][20381] Updated weights for policy 1, policy_version 959015 (0.0005) [2023-12-27 12:17:53,231][20381] Updated weights for policy 1, policy_version 959025 (0.0006) [2023-12-27 12:17:53,782][20347] Updated weights for policy 0, policy_version 959902 (0.0008) [2023-12-27 12:17:53,843][20347] Updated weights for policy 0, policy_version 959912 (0.0007) [2023-12-27 12:17:53,867][20381] Updated weights for policy 1, policy_version 959035 (0.0007) [2023-12-27 12:17:53,900][20347] Updated weights for policy 0, policy_version 959922 (0.0007) [2023-12-27 12:17:53,929][20381] Updated weights for policy 1, policy_version 959045 (0.0005) [2023-12-27 12:17:53,986][20381] Updated weights for policy 1, policy_version 959055 (0.0005) [2023-12-27 12:17:54,608][20381] Updated weights for policy 1, policy_version 959065 (0.0005) [2023-12-27 12:17:54,636][20347] Updated weights for policy 0, policy_version 959934 (0.0007) [2023-12-27 12:17:54,660][20381] Updated weights for policy 1, policy_version 959075 (0.0006) [2023-12-27 12:17:54,690][20347] Updated weights for policy 0, policy_version 959944 (0.0010) [2023-12-27 12:17:54,711][20381] Updated weights for policy 1, policy_version 959085 (0.0005) [2023-12-27 12:17:54,752][20347] Updated weights for policy 0, policy_version 959954 (0.0010) [2023-12-27 12:17:54,761][20381] Updated weights for policy 1, policy_version 959095 (0.0006) [2023-12-27 12:17:55,402][20347] Updated weights for policy 0, policy_version 959964 (0.0008) [2023-12-27 12:17:55,464][20347] Updated weights for policy 0, policy_version 959974 (0.0010) [2023-12-27 12:17:55,497][20381] Updated weights for policy 1, policy_version 959105 (0.0008) [2023-12-27 12:17:55,518][20347] Updated weights for policy 0, policy_version 959984 (0.0010) [2023-12-27 12:17:55,549][20381] Updated weights for policy 1, policy_version 959115 (0.0006) [2023-12-27 12:17:55,608][20381] Updated weights for policy 1, policy_version 959125 (0.0005) [2023-12-27 12:17:56,212][20347] Updated weights for policy 0, policy_version 959994 (0.0010) [2023-12-27 12:17:56,226][20381] Updated weights for policy 1, policy_version 959135 (0.0007) [2023-12-27 12:17:56,268][20347] Updated weights for policy 0, policy_version 960004 (0.0010) [2023-12-27 12:17:56,287][20381] Updated weights for policy 1, policy_version 959145 (0.0010) [2023-12-27 12:17:56,323][20347] Updated weights for policy 0, policy_version 960014 (0.0010) [2023-12-27 12:17:56,339][20381] Updated weights for policy 1, policy_version 959155 (0.0007) [2023-12-27 12:17:56,342][20291] KL-divergence is very high: 225.7211 [2023-12-27 12:17:56,352][20291] KL-divergence is very high: 179.8006 [2023-12-27 12:17:56,357][20291] KL-divergence is very high: 163.5949 [2023-12-27 12:17:56,377][20347] Updated weights for policy 0, policy_version 960024 (0.0010) [2023-12-27 12:17:57,033][20291] KL-divergence is very high: 135.0503 [2023-12-27 12:17:57,049][20381] Updated weights for policy 1, policy_version 959165 (0.0006) [2023-12-27 12:17:57,080][20291] KL-divergence is very high: 225.3732 [2023-12-27 12:17:57,085][20291] KL-divergence is very high: 273.2667 [2023-12-27 12:17:57,090][20347] Updated weights for policy 0, policy_version 960034 (0.0006) [2023-12-27 12:17:57,111][20381] Updated weights for policy 1, policy_version 959175 (0.0007) [2023-12-27 12:17:57,125][20291] KL-divergence is very high: 137.9380 [2023-12-27 12:17:57,147][20347] Updated weights for policy 0, policy_version 960044 (0.0006) [2023-12-27 12:17:57,174][20381] Updated weights for policy 1, policy_version 959185 (0.0008) [2023-12-27 12:17:57,210][20347] Updated weights for policy 0, policy_version 960054 (0.0007) [2023-12-27 12:17:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 491397120. Throughput: 0: 10016.7, 1: 9999.1. Samples: 491383240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:17:57,363][19170] Avg episode reward: [(0, '708.800'), (1, '1000.000')] [2023-12-27 12:17:57,873][20347] Updated weights for policy 0, policy_version 960064 (0.0005) [2023-12-27 12:17:57,916][20381] Updated weights for policy 1, policy_version 959195 (0.0006) [2023-12-27 12:17:57,924][20347] Updated weights for policy 0, policy_version 960074 (0.0005) [2023-12-27 12:17:57,973][20381] Updated weights for policy 1, policy_version 959205 (0.0006) [2023-12-27 12:17:57,979][20347] Updated weights for policy 0, policy_version 960084 (0.0005) [2023-12-27 12:17:58,025][20381] Updated weights for policy 1, policy_version 959215 (0.0007) [2023-12-27 12:17:58,735][20347] Updated weights for policy 0, policy_version 960094 (0.0007) [2023-12-27 12:17:58,742][20381] Updated weights for policy 1, policy_version 959225 (0.0008) [2023-12-27 12:17:58,803][20347] Updated weights for policy 0, policy_version 960104 (0.0008) [2023-12-27 12:17:58,813][20381] Updated weights for policy 1, policy_version 959235 (0.0007) [2023-12-27 12:17:58,877][20347] Updated weights for policy 0, policy_version 960114 (0.0008) [2023-12-27 12:17:58,886][20381] Updated weights for policy 1, policy_version 959245 (0.0007) [2023-12-27 12:17:58,955][20381] Updated weights for policy 1, policy_version 959255 (0.0009) [2023-12-27 12:17:59,605][20347] Updated weights for policy 0, policy_version 960124 (0.0007) [2023-12-27 12:17:59,660][20347] Updated weights for policy 0, policy_version 960134 (0.0008) [2023-12-27 12:17:59,701][20381] Updated weights for policy 1, policy_version 959265 (0.0006) [2023-12-27 12:17:59,707][20347] Updated weights for policy 0, policy_version 960144 (0.0005) [2023-12-27 12:17:59,755][20381] Updated weights for policy 1, policy_version 959275 (0.0007) [2023-12-27 12:17:59,806][20381] Updated weights for policy 1, policy_version 959285 (0.0008) [2023-12-27 12:18:00,461][20347] Updated weights for policy 0, policy_version 960154 (0.0007) [2023-12-27 12:18:00,500][20381] Updated weights for policy 1, policy_version 959295 (0.0007) [2023-12-27 12:18:00,539][20347] Updated weights for policy 0, policy_version 960164 (0.0006) [2023-12-27 12:18:00,548][20381] Updated weights for policy 1, policy_version 959305 (0.0007) [2023-12-27 12:18:00,596][20347] Updated weights for policy 0, policy_version 960174 (0.0007) [2023-12-27 12:18:00,604][20381] Updated weights for policy 1, policy_version 959315 (0.0010) [2023-12-27 12:18:00,647][20347] Updated weights for policy 0, policy_version 960184 (0.0007) [2023-12-27 12:18:01,325][20381] Updated weights for policy 1, policy_version 959325 (0.0008) [2023-12-27 12:18:01,361][20347] Updated weights for policy 0, policy_version 960194 (0.0007) [2023-12-27 12:18:01,404][20381] Updated weights for policy 1, policy_version 959335 (0.0008) [2023-12-27 12:18:01,418][20347] Updated weights for policy 0, policy_version 960204 (0.0008) [2023-12-27 12:18:01,463][20381] Updated weights for policy 1, policy_version 959345 (0.0010) [2023-12-27 12:18:01,469][20347] Updated weights for policy 0, policy_version 960214 (0.0006) [2023-12-27 12:18:02,168][20381] Updated weights for policy 1, policy_version 959355 (0.0008) [2023-12-27 12:18:02,181][20347] Updated weights for policy 0, policy_version 960224 (0.0006) [2023-12-27 12:18:02,226][20381] Updated weights for policy 1, policy_version 959365 (0.0007) [2023-12-27 12:18:02,240][20347] Updated weights for policy 0, policy_version 960234 (0.0007) [2023-12-27 12:18:02,291][20381] Updated weights for policy 1, policy_version 959375 (0.0007) [2023-12-27 12:18:02,306][20347] Updated weights for policy 0, policy_version 960244 (0.0009) [2023-12-27 12:18:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20382.8). Total num frames: 491495424. Throughput: 0: 9883.4, 1: 9894.3. Samples: 491499464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:02,363][19170] Avg episode reward: [(0, '701.010'), (1, '1000.000')] [2023-12-27 12:18:02,906][20381] Updated weights for policy 1, policy_version 959385 (0.0008) [2023-12-27 12:18:02,956][20381] Updated weights for policy 1, policy_version 959395 (0.0005) [2023-12-27 12:18:03,007][20381] Updated weights for policy 1, policy_version 959405 (0.0005) [2023-12-27 12:18:03,063][20381] Updated weights for policy 1, policy_version 959415 (0.0005) [2023-12-27 12:18:03,084][20347] Updated weights for policy 0, policy_version 960254 (0.0008) [2023-12-27 12:18:03,143][20347] Updated weights for policy 0, policy_version 960264 (0.0007) [2023-12-27 12:18:03,198][20347] Updated weights for policy 0, policy_version 960274 (0.0008) [2023-12-27 12:18:03,665][20381] Updated weights for policy 1, policy_version 959425 (0.0005) [2023-12-27 12:18:03,716][20381] Updated weights for policy 1, policy_version 959435 (0.0006) [2023-12-27 12:18:03,768][20381] Updated weights for policy 1, policy_version 959445 (0.0005) [2023-12-27 12:18:03,934][20347] Updated weights for policy 0, policy_version 960284 (0.0008) [2023-12-27 12:18:04,001][20347] Updated weights for policy 0, policy_version 960294 (0.0008) [2023-12-27 12:18:04,062][20347] Updated weights for policy 0, policy_version 960304 (0.0008) [2023-12-27 12:18:04,399][20381] Updated weights for policy 1, policy_version 959455 (0.0007) [2023-12-27 12:18:04,452][20381] Updated weights for policy 1, policy_version 959465 (0.0008) [2023-12-27 12:18:04,508][20381] Updated weights for policy 1, policy_version 959475 (0.0007) [2023-12-27 12:18:04,823][20347] Updated weights for policy 0, policy_version 960314 (0.0009) [2023-12-27 12:18:04,883][20347] Updated weights for policy 0, policy_version 960324 (0.0008) [2023-12-27 12:18:04,959][20347] Updated weights for policy 0, policy_version 960334 (0.0006) [2023-12-27 12:18:05,013][20347] Updated weights for policy 0, policy_version 960344 (0.0006) [2023-12-27 12:18:05,135][20381] Updated weights for policy 1, policy_version 959485 (0.0007) [2023-12-27 12:18:05,192][20381] Updated weights for policy 1, policy_version 959495 (0.0008) [2023-12-27 12:18:05,248][20381] Updated weights for policy 1, policy_version 959505 (0.0007) [2023-12-27 12:18:05,621][20347] Updated weights for policy 0, policy_version 960354 (0.0007) [2023-12-27 12:18:05,670][20347] Updated weights for policy 0, policy_version 960364 (0.0008) [2023-12-27 12:18:05,729][20347] Updated weights for policy 0, policy_version 960374 (0.0008) [2023-12-27 12:18:05,956][20381] Updated weights for policy 1, policy_version 959515 (0.0008) [2023-12-27 12:18:06,012][20381] Updated weights for policy 1, policy_version 959525 (0.0006) [2023-12-27 12:18:06,065][20381] Updated weights for policy 1, policy_version 959535 (0.0006) [2023-12-27 12:18:06,503][20347] Updated weights for policy 0, policy_version 960384 (0.0008) [2023-12-27 12:18:06,563][20347] Updated weights for policy 0, policy_version 960394 (0.0008) [2023-12-27 12:18:06,619][20347] Updated weights for policy 0, policy_version 960404 (0.0007) [2023-12-27 12:18:06,727][20381] Updated weights for policy 1, policy_version 959545 (0.0008) [2023-12-27 12:18:06,777][20381] Updated weights for policy 1, policy_version 959555 (0.0007) [2023-12-27 12:18:06,840][20381] Updated weights for policy 1, policy_version 959565 (0.0005) [2023-12-27 12:18:06,896][20381] Updated weights for policy 1, policy_version 959575 (0.0006) [2023-12-27 12:18:07,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 20355.0). Total num frames: 491585536. Throughput: 0: 9860.2, 1: 9957.6. Samples: 491559940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:07,363][19170] Avg episode reward: [(0, '699.750'), (1, '1000.000')] [2023-12-27 12:18:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000959576_245678080.pth... [2023-12-27 12:18:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000958392_245374976.pth [2023-12-27 12:18:07,391][20347] Updated weights for policy 0, policy_version 960414 (0.0008) [2023-12-27 12:18:07,436][20347] Updated weights for policy 0, policy_version 960424 (0.0007) [2023-12-27 12:18:07,491][20381] Updated weights for policy 1, policy_version 959585 (0.0006) [2023-12-27 12:18:07,494][20347] Updated weights for policy 0, policy_version 960434 (0.0006) [2023-12-27 12:18:07,535][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000960440_245915648.pth... [2023-12-27 12:18:07,539][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000959288_245620736.pth [2023-12-27 12:18:07,544][20381] Updated weights for policy 1, policy_version 959595 (0.0007) [2023-12-27 12:18:07,600][20381] Updated weights for policy 1, policy_version 959605 (0.0007) [2023-12-27 12:18:08,214][20347] Updated weights for policy 0, policy_version 960444 (0.0008) [2023-12-27 12:18:08,245][20381] Updated weights for policy 1, policy_version 959615 (0.0006) [2023-12-27 12:18:08,262][20347] Updated weights for policy 0, policy_version 960454 (0.0006) [2023-12-27 12:18:08,290][20381] Updated weights for policy 1, policy_version 959625 (0.0005) [2023-12-27 12:18:08,318][20347] Updated weights for policy 0, policy_version 960464 (0.0006) [2023-12-27 12:18:08,361][20381] Updated weights for policy 1, policy_version 959635 (0.0008) [2023-12-27 12:18:09,043][20347] Updated weights for policy 0, policy_version 960474 (0.0007) [2023-12-27 12:18:09,095][20381] Updated weights for policy 1, policy_version 959645 (0.0007) [2023-12-27 12:18:09,109][20347] Updated weights for policy 0, policy_version 960484 (0.0008) [2023-12-27 12:18:09,141][20381] Updated weights for policy 1, policy_version 959655 (0.0007) [2023-12-27 12:18:09,171][20347] Updated weights for policy 0, policy_version 960494 (0.0008) [2023-12-27 12:18:09,196][20381] Updated weights for policy 1, policy_version 959665 (0.0007) [2023-12-27 12:18:09,254][20347] Updated weights for policy 0, policy_version 960504 (0.0007) [2023-12-27 12:18:09,922][20381] Updated weights for policy 1, policy_version 959675 (0.0008) [2023-12-27 12:18:09,982][20381] Updated weights for policy 1, policy_version 959685 (0.0007) [2023-12-27 12:18:10,035][20381] Updated weights for policy 1, policy_version 959695 (0.0008) [2023-12-27 12:18:10,082][20347] Updated weights for policy 0, policy_version 960514 (0.0008) [2023-12-27 12:18:10,145][20347] Updated weights for policy 0, policy_version 960524 (0.0006) [2023-12-27 12:18:10,209][20347] Updated weights for policy 0, policy_version 960534 (0.0006) [2023-12-27 12:18:10,760][20381] Updated weights for policy 1, policy_version 959705 (0.0008) [2023-12-27 12:18:10,819][20381] Updated weights for policy 1, policy_version 959715 (0.0011) [2023-12-27 12:18:10,870][20347] Updated weights for policy 0, policy_version 960544 (0.0006) [2023-12-27 12:18:10,872][20381] Updated weights for policy 1, policy_version 959725 (0.0011) [2023-12-27 12:18:10,921][20381] Updated weights for policy 1, policy_version 959735 (0.0011) [2023-12-27 12:18:10,922][20347] Updated weights for policy 0, policy_version 960554 (0.0007) [2023-12-27 12:18:10,974][20347] Updated weights for policy 0, policy_version 960564 (0.0007) [2023-12-27 12:18:11,762][20381] Updated weights for policy 1, policy_version 959745 (0.0008) [2023-12-27 12:18:11,775][20347] Updated weights for policy 0, policy_version 960574 (0.0007) [2023-12-27 12:18:11,824][20381] Updated weights for policy 1, policy_version 959755 (0.0007) [2023-12-27 12:18:11,837][20347] Updated weights for policy 0, policy_version 960584 (0.0006) [2023-12-27 12:18:11,883][20381] Updated weights for policy 1, policy_version 959765 (0.0006) [2023-12-27 12:18:11,899][20347] Updated weights for policy 0, policy_version 960594 (0.0006) [2023-12-27 12:18:12,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19797.3, 300 sec: 20327.3). Total num frames: 491683840. Throughput: 0: 9841.3, 1: 9979.5. Samples: 491677268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:12,363][19170] Avg episode reward: [(0, '678.860'), (1, '1000.000')] [2023-12-27 12:18:12,575][20381] Updated weights for policy 1, policy_version 959775 (0.0006) [2023-12-27 12:18:12,628][20347] Updated weights for policy 0, policy_version 960604 (0.0008) [2023-12-27 12:18:12,634][20381] Updated weights for policy 1, policy_version 959785 (0.0007) [2023-12-27 12:18:12,684][20347] Updated weights for policy 0, policy_version 960614 (0.0007) [2023-12-27 12:18:12,686][20381] Updated weights for policy 1, policy_version 959795 (0.0008) [2023-12-27 12:18:12,752][20347] Updated weights for policy 0, policy_version 960624 (0.0008) [2023-12-27 12:18:13,301][20381] Updated weights for policy 1, policy_version 959805 (0.0008) [2023-12-27 12:18:13,363][20381] Updated weights for policy 1, policy_version 959815 (0.0006) [2023-12-27 12:18:13,420][20381] Updated weights for policy 1, policy_version 959825 (0.0007) [2023-12-27 12:18:13,522][20347] Updated weights for policy 0, policy_version 960634 (0.0007) [2023-12-27 12:18:13,571][20347] Updated weights for policy 0, policy_version 960644 (0.0008) [2023-12-27 12:18:13,623][20347] Updated weights for policy 0, policy_version 960654 (0.0007) [2023-12-27 12:18:13,679][20347] Updated weights for policy 0, policy_version 960664 (0.0009) [2023-12-27 12:18:13,996][20381] Updated weights for policy 1, policy_version 959835 (0.0007) [2023-12-27 12:18:14,059][20381] Updated weights for policy 1, policy_version 959845 (0.0006) [2023-12-27 12:18:14,117][20381] Updated weights for policy 1, policy_version 959855 (0.0005) [2023-12-27 12:18:14,466][20347] Updated weights for policy 0, policy_version 960674 (0.0005) [2023-12-27 12:18:14,524][20347] Updated weights for policy 0, policy_version 960684 (0.0008) [2023-12-27 12:18:14,592][20347] Updated weights for policy 0, policy_version 960694 (0.0010) [2023-12-27 12:18:14,661][20381] Updated weights for policy 1, policy_version 959865 (0.0005) [2023-12-27 12:18:14,719][20381] Updated weights for policy 1, policy_version 959875 (0.0007) [2023-12-27 12:18:14,784][20381] Updated weights for policy 1, policy_version 959885 (0.0007) [2023-12-27 12:18:14,847][20381] Updated weights for policy 1, policy_version 959895 (0.0008) [2023-12-27 12:18:15,338][20347] Updated weights for policy 0, policy_version 960704 (0.0009) [2023-12-27 12:18:15,398][20347] Updated weights for policy 0, policy_version 960714 (0.0008) [2023-12-27 12:18:15,453][20347] Updated weights for policy 0, policy_version 960724 (0.0007) [2023-12-27 12:18:15,497][20381] Updated weights for policy 1, policy_version 959905 (0.0006) [2023-12-27 12:18:15,558][20381] Updated weights for policy 1, policy_version 959915 (0.0006) [2023-12-27 12:18:15,607][20381] Updated weights for policy 1, policy_version 959925 (0.0008) [2023-12-27 12:18:16,184][20347] Updated weights for policy 0, policy_version 960734 (0.0008) [2023-12-27 12:18:16,239][20347] Updated weights for policy 0, policy_version 960744 (0.0007) [2023-12-27 12:18:16,270][20381] Updated weights for policy 1, policy_version 959935 (0.0006) [2023-12-27 12:18:16,295][20347] Updated weights for policy 0, policy_version 960754 (0.0007) [2023-12-27 12:18:16,333][20381] Updated weights for policy 1, policy_version 959945 (0.0008) [2023-12-27 12:18:16,390][20381] Updated weights for policy 1, policy_version 959955 (0.0007) [2023-12-27 12:18:16,988][20347] Updated weights for policy 0, policy_version 960764 (0.0007) [2023-12-27 12:18:17,041][20381] Updated weights for policy 1, policy_version 959965 (0.0006) [2023-12-27 12:18:17,047][20347] Updated weights for policy 0, policy_version 960774 (0.0008) [2023-12-27 12:18:17,090][20381] Updated weights for policy 1, policy_version 959975 (0.0005) [2023-12-27 12:18:17,106][20347] Updated weights for policy 0, policy_version 960784 (0.0007) [2023-12-27 12:18:17,143][20381] Updated weights for policy 1, policy_version 959985 (0.0006) [2023-12-27 12:18:17,365][19170] Fps is (10 sec: 20474.5, 60 sec: 19933.0, 300 sec: 20354.9). Total num frames: 491790336. Throughput: 0: 9712.0, 1: 10109.8. Samples: 491796816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:17,366][19170] Avg episode reward: [(0, '689.450'), (1, '1000.000')] [2023-12-27 12:18:17,707][20381] Updated weights for policy 1, policy_version 959995 (0.0007) [2023-12-27 12:18:17,770][20381] Updated weights for policy 1, policy_version 960005 (0.0007) [2023-12-27 12:18:17,822][20381] Updated weights for policy 1, policy_version 960015 (0.0008) [2023-12-27 12:18:17,859][20347] Updated weights for policy 0, policy_version 960794 (0.0007) [2023-12-27 12:18:17,912][20347] Updated weights for policy 0, policy_version 960804 (0.0007) [2023-12-27 12:18:17,975][20347] Updated weights for policy 0, policy_version 960814 (0.0007) [2023-12-27 12:18:18,039][20347] Updated weights for policy 0, policy_version 960824 (0.0006) [2023-12-27 12:18:18,579][20381] Updated weights for policy 1, policy_version 960025 (0.0007) [2023-12-27 12:18:18,626][20381] Updated weights for policy 1, policy_version 960035 (0.0006) [2023-12-27 12:18:18,686][20381] Updated weights for policy 1, policy_version 960045 (0.0007) [2023-12-27 12:18:18,707][20347] Updated weights for policy 0, policy_version 960834 (0.0008) [2023-12-27 12:18:18,765][20381] Updated weights for policy 1, policy_version 960055 (0.0006) [2023-12-27 12:18:18,793][20347] Updated weights for policy 0, policy_version 960844 (0.0008) [2023-12-27 12:18:18,848][20347] Updated weights for policy 0, policy_version 960854 (0.0008) [2023-12-27 12:18:19,346][20381] Updated weights for policy 1, policy_version 960065 (0.0009) [2023-12-27 12:18:19,413][20381] Updated weights for policy 1, policy_version 960075 (0.0007) [2023-12-27 12:18:19,466][20381] Updated weights for policy 1, policy_version 960085 (0.0006) [2023-12-27 12:18:19,646][20347] Updated weights for policy 0, policy_version 960864 (0.0008) [2023-12-27 12:18:19,708][20347] Updated weights for policy 0, policy_version 960874 (0.0007) [2023-12-27 12:18:19,768][20347] Updated weights for policy 0, policy_version 960884 (0.0009) [2023-12-27 12:18:20,262][20381] Updated weights for policy 1, policy_version 960095 (0.0006) [2023-12-27 12:18:20,325][20381] Updated weights for policy 1, policy_version 960105 (0.0007) [2023-12-27 12:18:20,408][20381] Updated weights for policy 1, policy_version 960115 (0.0008) [2023-12-27 12:18:20,519][20347] Updated weights for policy 0, policy_version 960894 (0.0008) [2023-12-27 12:18:20,589][20347] Updated weights for policy 0, policy_version 960904 (0.0009) [2023-12-27 12:18:20,652][20347] Updated weights for policy 0, policy_version 960914 (0.0008) [2023-12-27 12:18:21,008][20381] Updated weights for policy 1, policy_version 960128 (0.0008) [2023-12-27 12:18:21,077][20381] Updated weights for policy 1, policy_version 960138 (0.0008) [2023-12-27 12:18:21,138][20381] Updated weights for policy 1, policy_version 960148 (0.0009) [2023-12-27 12:18:21,506][20347] Updated weights for policy 0, policy_version 960924 (0.0007) [2023-12-27 12:18:21,557][20347] Updated weights for policy 0, policy_version 960934 (0.0008) [2023-12-27 12:18:21,631][20347] Updated weights for policy 0, policy_version 960944 (0.0008) [2023-12-27 12:18:21,837][20381] Updated weights for policy 1, policy_version 960158 (0.0007) [2023-12-27 12:18:21,897][20381] Updated weights for policy 1, policy_version 960168 (0.0008) [2023-12-27 12:18:21,949][20381] Updated weights for policy 1, policy_version 960178 (0.0008) [2023-12-27 12:18:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 20299.5). Total num frames: 491880448. Throughput: 0: 9661.3, 1: 10133.2. Samples: 491855580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:22,364][19170] Avg episode reward: [(0, '757.810'), (1, '1000.000')] [2023-12-27 12:18:22,382][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000960184_245833728.pth... [2023-12-27 12:18:22,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000959000_245530624.pth [2023-12-27 12:18:22,401][20347] Updated weights for policy 0, policy_version 960954 (0.0007) [2023-12-27 12:18:22,461][20347] Updated weights for policy 0, policy_version 960964 (0.0008) [2023-12-27 12:18:22,524][20347] Updated weights for policy 0, policy_version 960974 (0.0008) [2023-12-27 12:18:22,591][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000960984_246054912.pth... [2023-12-27 12:18:22,593][20347] Updated weights for policy 0, policy_version 960984 (0.0007) [2023-12-27 12:18:22,595][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000959864_245768192.pth [2023-12-27 12:18:22,604][20381] Updated weights for policy 1, policy_version 960188 (0.0008) [2023-12-27 12:18:22,663][20381] Updated weights for policy 1, policy_version 960198 (0.0008) [2023-12-27 12:18:22,719][20381] Updated weights for policy 1, policy_version 960208 (0.0006) [2023-12-27 12:18:23,268][20347] Updated weights for policy 0, policy_version 960994 (0.0007) [2023-12-27 12:18:23,330][20347] Updated weights for policy 0, policy_version 961004 (0.0006) [2023-12-27 12:18:23,367][20381] Updated weights for policy 1, policy_version 960218 (0.0005) [2023-12-27 12:18:23,392][20347] Updated weights for policy 0, policy_version 961014 (0.0008) [2023-12-27 12:18:23,426][20381] Updated weights for policy 1, policy_version 960228 (0.0008) [2023-12-27 12:18:23,485][20381] Updated weights for policy 1, policy_version 960238 (0.0008) [2023-12-27 12:18:23,537][20381] Updated weights for policy 1, policy_version 960248 (0.0006) [2023-12-27 12:18:23,956][20347] Updated weights for policy 0, policy_version 961024 (0.0006) [2023-12-27 12:18:24,017][20347] Updated weights for policy 0, policy_version 961034 (0.0006) [2023-12-27 12:18:24,066][20347] Updated weights for policy 0, policy_version 961044 (0.0005) [2023-12-27 12:18:24,205][20381] Updated weights for policy 1, policy_version 960258 (0.0009) [2023-12-27 12:18:24,265][20381] Updated weights for policy 1, policy_version 960268 (0.0007) [2023-12-27 12:18:24,322][20381] Updated weights for policy 1, policy_version 960278 (0.0008) [2023-12-27 12:18:24,691][20347] Updated weights for policy 0, policy_version 961054 (0.0008) [2023-12-27 12:18:24,739][20347] Updated weights for policy 0, policy_version 961064 (0.0010) [2023-12-27 12:18:24,800][20347] Updated weights for policy 0, policy_version 961074 (0.0009) [2023-12-27 12:18:25,033][20381] Updated weights for policy 1, policy_version 960288 (0.0007) [2023-12-27 12:18:25,088][20381] Updated weights for policy 1, policy_version 960298 (0.0007) [2023-12-27 12:18:25,144][20381] Updated weights for policy 1, policy_version 960308 (0.0007) [2023-12-27 12:18:25,513][20347] Updated weights for policy 0, policy_version 961084 (0.0008) [2023-12-27 12:18:25,571][20347] Updated weights for policy 0, policy_version 961094 (0.0010) [2023-12-27 12:18:25,629][20347] Updated weights for policy 0, policy_version 961104 (0.0010) [2023-12-27 12:18:25,843][20381] Updated weights for policy 1, policy_version 960318 (0.0007) [2023-12-27 12:18:25,897][20381] Updated weights for policy 1, policy_version 960328 (0.0009) [2023-12-27 12:18:25,952][20381] Updated weights for policy 1, policy_version 960338 (0.0007) [2023-12-27 12:18:26,250][20347] Updated weights for policy 0, policy_version 961114 (0.0010) [2023-12-27 12:18:26,299][20347] Updated weights for policy 0, policy_version 961124 (0.0007) [2023-12-27 12:18:26,344][20347] Updated weights for policy 0, policy_version 961134 (0.0008) [2023-12-27 12:18:26,401][20347] Updated weights for policy 0, policy_version 961144 (0.0008) [2023-12-27 12:18:26,645][20381] Updated weights for policy 1, policy_version 960348 (0.0008) [2023-12-27 12:18:26,694][20381] Updated weights for policy 1, policy_version 960358 (0.0008) [2023-12-27 12:18:26,756][20381] Updated weights for policy 1, policy_version 960368 (0.0007) [2023-12-27 12:18:27,101][20347] Updated weights for policy 0, policy_version 961154 (0.0005) [2023-12-27 12:18:27,147][20347] Updated weights for policy 0, policy_version 961164 (0.0005) [2023-12-27 12:18:27,202][20347] Updated weights for policy 0, policy_version 961174 (0.0006) [2023-12-27 12:18:27,362][19170] Fps is (10 sec: 19666.0, 60 sec: 19933.9, 300 sec: 20327.3). Total num frames: 491986944. Throughput: 0: 9652.9, 1: 10203.3. Samples: 491975384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:27,363][19170] Avg episode reward: [(0, '785.970'), (1, '1000.000')] [2023-12-27 12:18:27,430][20381] Updated weights for policy 1, policy_version 960378 (0.0007) [2023-12-27 12:18:27,491][20381] Updated weights for policy 1, policy_version 960388 (0.0009) [2023-12-27 12:18:27,555][20381] Updated weights for policy 1, policy_version 960398 (0.0008) [2023-12-27 12:18:27,615][20381] Updated weights for policy 1, policy_version 960408 (0.0007) [2023-12-27 12:18:27,775][20347] Updated weights for policy 0, policy_version 961184 (0.0008) [2023-12-27 12:18:27,840][20347] Updated weights for policy 0, policy_version 961194 (0.0008) [2023-12-27 12:18:27,922][20347] Updated weights for policy 0, policy_version 961204 (0.0008) [2023-12-27 12:18:28,305][20381] Updated weights for policy 1, policy_version 960418 (0.0005) [2023-12-27 12:18:28,365][20381] Updated weights for policy 1, policy_version 960428 (0.0008) [2023-12-27 12:18:28,417][20381] Updated weights for policy 1, policy_version 960438 (0.0008) [2023-12-27 12:18:28,637][20347] Updated weights for policy 0, policy_version 961214 (0.0008) [2023-12-27 12:18:28,687][20347] Updated weights for policy 0, policy_version 961224 (0.0005) [2023-12-27 12:18:28,744][20347] Updated weights for policy 0, policy_version 961234 (0.0008) [2023-12-27 12:18:29,131][20381] Updated weights for policy 1, policy_version 960448 (0.0006) [2023-12-27 12:18:29,192][20381] Updated weights for policy 1, policy_version 960458 (0.0006) [2023-12-27 12:18:29,263][20381] Updated weights for policy 1, policy_version 960468 (0.0008) [2023-12-27 12:18:29,494][20347] Updated weights for policy 0, policy_version 961244 (0.0008) [2023-12-27 12:18:29,551][20347] Updated weights for policy 0, policy_version 961254 (0.0007) [2023-12-27 12:18:29,604][20347] Updated weights for policy 0, policy_version 961264 (0.0007) [2023-12-27 12:18:29,937][20381] Updated weights for policy 1, policy_version 960478 (0.0008) [2023-12-27 12:18:29,996][20381] Updated weights for policy 1, policy_version 960488 (0.0006) [2023-12-27 12:18:30,061][20381] Updated weights for policy 1, policy_version 960498 (0.0007) [2023-12-27 12:18:30,321][20347] Updated weights for policy 0, policy_version 961274 (0.0008) [2023-12-27 12:18:30,376][20347] Updated weights for policy 0, policy_version 961284 (0.0006) [2023-12-27 12:18:30,430][20347] Updated weights for policy 0, policy_version 961294 (0.0009) [2023-12-27 12:18:30,491][20347] Updated weights for policy 0, policy_version 961304 (0.0010) [2023-12-27 12:18:30,657][20381] Updated weights for policy 1, policy_version 960508 (0.0007) [2023-12-27 12:18:30,705][20381] Updated weights for policy 1, policy_version 960518 (0.0009) [2023-12-27 12:18:30,764][20381] Updated weights for policy 1, policy_version 960528 (0.0007) [2023-12-27 12:18:31,132][20347] Updated weights for policy 0, policy_version 961314 (0.0008) [2023-12-27 12:18:31,195][20347] Updated weights for policy 0, policy_version 961324 (0.0008) [2023-12-27 12:18:31,262][20347] Updated weights for policy 0, policy_version 961334 (0.0008) [2023-12-27 12:18:31,441][20381] Updated weights for policy 1, policy_version 960538 (0.0006) [2023-12-27 12:18:31,498][20381] Updated weights for policy 1, policy_version 960548 (0.0007) [2023-12-27 12:18:31,559][20381] Updated weights for policy 1, policy_version 960558 (0.0007) [2023-12-27 12:18:31,612][20381] Updated weights for policy 1, policy_version 960568 (0.0007) [2023-12-27 12:18:32,040][20347] Updated weights for policy 0, policy_version 961344 (0.0010) [2023-12-27 12:18:32,103][20347] Updated weights for policy 0, policy_version 961354 (0.0010) [2023-12-27 12:18:32,161][20347] Updated weights for policy 0, policy_version 961364 (0.0007) [2023-12-27 12:18:32,335][20381] Updated weights for policy 1, policy_version 960578 (0.0010) [2023-12-27 12:18:32,362][19170] Fps is (10 sec: 20480.5, 60 sec: 19933.9, 300 sec: 20271.8). Total num frames: 492085248. Throughput: 0: 9671.6, 1: 10225.4. Samples: 492096724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:32,363][19170] Avg episode reward: [(0, '833.740'), (1, '1000.000')] [2023-12-27 12:18:32,405][20381] Updated weights for policy 1, policy_version 960588 (0.0011) [2023-12-27 12:18:32,459][20381] Updated weights for policy 1, policy_version 960598 (0.0008) [2023-12-27 12:18:32,858][20347] Updated weights for policy 0, policy_version 961374 (0.0007) [2023-12-27 12:18:32,923][20347] Updated weights for policy 0, policy_version 961384 (0.0009) [2023-12-27 12:18:32,986][20347] Updated weights for policy 0, policy_version 961394 (0.0008) [2023-12-27 12:18:33,096][20381] Updated weights for policy 1, policy_version 960608 (0.0010) [2023-12-27 12:18:33,146][20381] Updated weights for policy 1, policy_version 960618 (0.0010) [2023-12-27 12:18:33,208][20381] Updated weights for policy 1, policy_version 960628 (0.0010) [2023-12-27 12:18:33,625][20347] Updated weights for policy 0, policy_version 961405 (0.0007) [2023-12-27 12:18:33,671][20347] Updated weights for policy 0, policy_version 961415 (0.0005) [2023-12-27 12:18:33,732][20347] Updated weights for policy 0, policy_version 961425 (0.0006) [2023-12-27 12:18:33,839][20381] Updated weights for policy 1, policy_version 960638 (0.0009) [2023-12-27 12:18:33,893][20381] Updated weights for policy 1, policy_version 960648 (0.0010) [2023-12-27 12:18:33,949][20381] Updated weights for policy 1, policy_version 960658 (0.0010) [2023-12-27 12:18:34,378][20347] Updated weights for policy 0, policy_version 961435 (0.0008) [2023-12-27 12:18:34,437][20347] Updated weights for policy 0, policy_version 961445 (0.0006) [2023-12-27 12:18:34,500][20347] Updated weights for policy 0, policy_version 961455 (0.0006) [2023-12-27 12:18:34,596][20381] Updated weights for policy 1, policy_version 960668 (0.0010) [2023-12-27 12:18:34,649][20381] Updated weights for policy 1, policy_version 960678 (0.0010) [2023-12-27 12:18:34,700][20381] Updated weights for policy 1, policy_version 960688 (0.0007) [2023-12-27 12:18:35,144][20347] Updated weights for policy 0, policy_version 961465 (0.0006) [2023-12-27 12:18:35,204][20347] Updated weights for policy 0, policy_version 961475 (0.0007) [2023-12-27 12:18:35,261][20347] Updated weights for policy 0, policy_version 961485 (0.0010) [2023-12-27 12:18:35,312][20347] Updated weights for policy 0, policy_version 961495 (0.0010) [2023-12-27 12:18:35,316][20381] Updated weights for policy 1, policy_version 960698 (0.0006) [2023-12-27 12:18:35,372][20381] Updated weights for policy 1, policy_version 960708 (0.0005) [2023-12-27 12:18:35,430][20381] Updated weights for policy 1, policy_version 960718 (0.0006) [2023-12-27 12:18:35,490][20381] Updated weights for policy 1, policy_version 960728 (0.0006) [2023-12-27 12:18:35,976][20347] Updated weights for policy 0, policy_version 961505 (0.0010) [2023-12-27 12:18:36,025][20347] Updated weights for policy 0, policy_version 961515 (0.0010) [2023-12-27 12:18:36,059][20381] Updated weights for policy 1, policy_version 960738 (0.0005) [2023-12-27 12:18:36,072][20347] Updated weights for policy 0, policy_version 961525 (0.0009) [2023-12-27 12:18:36,118][20381] Updated weights for policy 1, policy_version 960748 (0.0009) [2023-12-27 12:18:36,184][20381] Updated weights for policy 1, policy_version 960758 (0.0008) [2023-12-27 12:18:36,847][20347] Updated weights for policy 0, policy_version 961535 (0.0008) [2023-12-27 12:18:36,854][20291] KL-divergence is very high: 182.2145 [2023-12-27 12:18:36,855][20381] Updated weights for policy 1, policy_version 960768 (0.0005) [2023-12-27 12:18:36,867][20291] KL-divergence is very high: 172.1771 [2023-12-27 12:18:36,873][20291] KL-divergence is very high: 123.0800 [2023-12-27 12:18:36,912][20381] Updated weights for policy 1, policy_version 960778 (0.0009) [2023-12-27 12:18:36,914][20347] Updated weights for policy 0, policy_version 961545 (0.0008) [2023-12-27 12:18:36,958][20347] Updated weights for policy 0, policy_version 961555 (0.0006) [2023-12-27 12:18:36,962][20381] Updated weights for policy 1, policy_version 960788 (0.0009) [2023-12-27 12:18:37,363][19170] Fps is (10 sec: 20478.7, 60 sec: 19933.7, 300 sec: 20300.3). Total num frames: 492191744. Throughput: 0: 9696.2, 1: 10260.7. Samples: 492160400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:37,364][19170] Avg episode reward: [(0, '814.680'), (1, '1000.000')] [2023-12-27 12:18:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000960792_245989376.pth... [2023-12-27 12:18:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000961560_246202368.pth... [2023-12-27 12:18:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000960440_245915648.pth [2023-12-27 12:18:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000959576_245678080.pth [2023-12-27 12:18:37,614][20347] Updated weights for policy 0, policy_version 961565 (0.0007) [2023-12-27 12:18:37,670][20347] Updated weights for policy 0, policy_version 961575 (0.0006) [2023-12-27 12:18:37,673][20381] Updated weights for policy 1, policy_version 960798 (0.0007) [2023-12-27 12:18:37,731][20381] Updated weights for policy 1, policy_version 960808 (0.0007) [2023-12-27 12:18:37,733][20347] Updated weights for policy 0, policy_version 961585 (0.0007) [2023-12-27 12:18:37,786][20381] Updated weights for policy 1, policy_version 960818 (0.0006) [2023-12-27 12:18:38,437][20347] Updated weights for policy 0, policy_version 961595 (0.0006) [2023-12-27 12:18:38,496][20347] Updated weights for policy 0, policy_version 961605 (0.0008) [2023-12-27 12:18:38,534][20381] Updated weights for policy 1, policy_version 960828 (0.0007) [2023-12-27 12:18:38,559][20347] Updated weights for policy 0, policy_version 961615 (0.0009) [2023-12-27 12:18:38,594][20381] Updated weights for policy 1, policy_version 960838 (0.0007) [2023-12-27 12:18:38,656][20381] Updated weights for policy 1, policy_version 960848 (0.0007) [2023-12-27 12:18:39,297][20347] Updated weights for policy 0, policy_version 961625 (0.0007) [2023-12-27 12:18:39,342][20381] Updated weights for policy 1, policy_version 960858 (0.0008) [2023-12-27 12:18:39,366][20347] Updated weights for policy 0, policy_version 961635 (0.0007) [2023-12-27 12:18:39,416][20381] Updated weights for policy 1, policy_version 960868 (0.0008) [2023-12-27 12:18:39,433][20347] Updated weights for policy 0, policy_version 961645 (0.0008) [2023-12-27 12:18:39,484][20381] Updated weights for policy 1, policy_version 960878 (0.0007) [2023-12-27 12:18:39,489][20347] Updated weights for policy 0, policy_version 961655 (0.0006) [2023-12-27 12:18:39,545][20381] Updated weights for policy 1, policy_version 960888 (0.0006) [2023-12-27 12:18:40,181][20381] Updated weights for policy 1, policy_version 960898 (0.0007) [2023-12-27 12:18:40,232][20347] Updated weights for policy 0, policy_version 961665 (0.0008) [2023-12-27 12:18:40,238][20381] Updated weights for policy 1, policy_version 960908 (0.0006) [2023-12-27 12:18:40,296][20381] Updated weights for policy 1, policy_version 960918 (0.0006) [2023-12-27 12:18:40,303][20347] Updated weights for policy 0, policy_version 961675 (0.0009) [2023-12-27 12:18:40,375][20347] Updated weights for policy 0, policy_version 961685 (0.0008) [2023-12-27 12:18:40,956][20347] Updated weights for policy 0, policy_version 961695 (0.0007) [2023-12-27 12:18:41,000][20381] Updated weights for policy 1, policy_version 960928 (0.0010) [2023-12-27 12:18:41,010][20347] Updated weights for policy 0, policy_version 961705 (0.0006) [2023-12-27 12:18:41,067][20381] Updated weights for policy 1, policy_version 960938 (0.0011) [2023-12-27 12:18:41,076][20347] Updated weights for policy 0, policy_version 961715 (0.0008) [2023-12-27 12:18:41,129][20381] Updated weights for policy 1, policy_version 960948 (0.0010) [2023-12-27 12:18:41,891][20347] Updated weights for policy 0, policy_version 961725 (0.0008) [2023-12-27 12:18:41,946][20381] Updated weights for policy 1, policy_version 960958 (0.0008) [2023-12-27 12:18:41,959][20347] Updated weights for policy 0, policy_version 961735 (0.0007) [2023-12-27 12:18:42,002][20381] Updated weights for policy 1, policy_version 960968 (0.0008) [2023-12-27 12:18:42,015][20347] Updated weights for policy 0, policy_version 961745 (0.0006) [2023-12-27 12:18:42,054][20381] Updated weights for policy 1, policy_version 960978 (0.0008) [2023-12-27 12:18:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 20299.5). Total num frames: 492290048. Throughput: 0: 9708.5, 1: 10228.6. Samples: 492280408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:42,363][19170] Avg episode reward: [(0, '863.850'), (1, '1000.000')] [2023-12-27 12:18:42,696][20347] Updated weights for policy 0, policy_version 961755 (0.0005) [2023-12-27 12:18:42,748][20347] Updated weights for policy 0, policy_version 961765 (0.0008) [2023-12-27 12:18:42,813][20347] Updated weights for policy 0, policy_version 961775 (0.0007) [2023-12-27 12:18:42,831][20381] Updated weights for policy 1, policy_version 960988 (0.0007) [2023-12-27 12:18:42,889][20381] Updated weights for policy 1, policy_version 960998 (0.0007) [2023-12-27 12:18:42,942][20381] Updated weights for policy 1, policy_version 961008 (0.0007) [2023-12-27 12:18:43,490][20347] Updated weights for policy 0, policy_version 961785 (0.0007) [2023-12-27 12:18:43,549][20347] Updated weights for policy 0, policy_version 961795 (0.0008) [2023-12-27 12:18:43,605][20347] Updated weights for policy 0, policy_version 961805 (0.0007) [2023-12-27 12:18:43,616][20381] Updated weights for policy 1, policy_version 961018 (0.0007) [2023-12-27 12:18:43,663][20347] Updated weights for policy 0, policy_version 961815 (0.0006) [2023-12-27 12:18:43,673][20381] Updated weights for policy 1, policy_version 961028 (0.0006) [2023-12-27 12:18:43,719][20381] Updated weights for policy 1, policy_version 961038 (0.0007) [2023-12-27 12:18:43,778][20381] Updated weights for policy 1, policy_version 961048 (0.0008) [2023-12-27 12:18:44,319][20347] Updated weights for policy 0, policy_version 961825 (0.0008) [2023-12-27 12:18:44,386][20347] Updated weights for policy 0, policy_version 961835 (0.0008) [2023-12-27 12:18:44,439][20347] Updated weights for policy 0, policy_version 961845 (0.0008) [2023-12-27 12:18:44,504][20381] Updated weights for policy 1, policy_version 961058 (0.0007) [2023-12-27 12:18:44,569][20381] Updated weights for policy 1, policy_version 961068 (0.0008) [2023-12-27 12:18:44,626][20381] Updated weights for policy 1, policy_version 961078 (0.0007) [2023-12-27 12:18:45,189][20347] Updated weights for policy 0, policy_version 961855 (0.0008) [2023-12-27 12:18:45,244][20347] Updated weights for policy 0, policy_version 961865 (0.0008) [2023-12-27 12:18:45,305][20347] Updated weights for policy 0, policy_version 961875 (0.0007) [2023-12-27 12:18:45,318][20381] Updated weights for policy 1, policy_version 961088 (0.0006) [2023-12-27 12:18:45,371][20381] Updated weights for policy 1, policy_version 961098 (0.0007) [2023-12-27 12:18:45,433][20381] Updated weights for policy 1, policy_version 961108 (0.0007) [2023-12-27 12:18:46,038][20381] Updated weights for policy 1, policy_version 961118 (0.0006) [2023-12-27 12:18:46,044][20347] Updated weights for policy 0, policy_version 961885 (0.0008) [2023-12-27 12:18:46,097][20381] Updated weights for policy 1, policy_version 961128 (0.0006) [2023-12-27 12:18:46,102][20347] Updated weights for policy 0, policy_version 961895 (0.0011) [2023-12-27 12:18:46,152][20381] Updated weights for policy 1, policy_version 961138 (0.0005) [2023-12-27 12:18:46,155][20347] Updated weights for policy 0, policy_version 961905 (0.0009) [2023-12-27 12:18:46,753][20347] Updated weights for policy 0, policy_version 961915 (0.0006) [2023-12-27 12:18:46,783][20381] Updated weights for policy 1, policy_version 961148 (0.0007) [2023-12-27 12:18:46,813][20347] Updated weights for policy 0, policy_version 961925 (0.0009) [2023-12-27 12:18:46,833][20381] Updated weights for policy 1, policy_version 961158 (0.0005) [2023-12-27 12:18:46,860][20347] Updated weights for policy 0, policy_version 961935 (0.0010) [2023-12-27 12:18:46,888][20381] Updated weights for policy 1, policy_version 961168 (0.0005) [2023-12-27 12:18:47,363][19170] Fps is (10 sec: 19661.7, 60 sec: 19933.8, 300 sec: 20271.7). Total num frames: 492388352. Throughput: 0: 9722.5, 1: 10259.5. Samples: 492398656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:47,364][19170] Avg episode reward: [(0, '922.240'), (1, '1000.000')] [2023-12-27 12:18:47,470][20347] Updated weights for policy 0, policy_version 961945 (0.0010) [2023-12-27 12:18:47,533][20347] Updated weights for policy 0, policy_version 961955 (0.0005) [2023-12-27 12:18:47,601][20347] Updated weights for policy 0, policy_version 961965 (0.0006) [2023-12-27 12:18:47,637][20381] Updated weights for policy 1, policy_version 961178 (0.0007) [2023-12-27 12:18:47,661][20347] Updated weights for policy 0, policy_version 961975 (0.0005) [2023-12-27 12:18:47,681][20381] Updated weights for policy 1, policy_version 961188 (0.0008) [2023-12-27 12:18:47,728][20381] Updated weights for policy 1, policy_version 961198 (0.0009) [2023-12-27 12:18:47,774][20381] Updated weights for policy 1, policy_version 961208 (0.0009) [2023-12-27 12:18:48,239][20347] Updated weights for policy 0, policy_version 961985 (0.0010) [2023-12-27 12:18:48,299][20347] Updated weights for policy 0, policy_version 961995 (0.0010) [2023-12-27 12:18:48,359][20347] Updated weights for policy 0, policy_version 962005 (0.0008) [2023-12-27 12:18:48,605][20381] Updated weights for policy 1, policy_version 961218 (0.0010) [2023-12-27 12:18:48,666][20381] Updated weights for policy 1, policy_version 961228 (0.0009) [2023-12-27 12:18:48,732][20381] Updated weights for policy 1, policy_version 961238 (0.0011) [2023-12-27 12:18:49,035][20347] Updated weights for policy 0, policy_version 962015 (0.0005) [2023-12-27 12:18:49,094][20347] Updated weights for policy 0, policy_version 962025 (0.0007) [2023-12-27 12:18:49,146][20347] Updated weights for policy 0, policy_version 962035 (0.0010) [2023-12-27 12:18:49,427][20381] Updated weights for policy 1, policy_version 961248 (0.0010) [2023-12-27 12:18:49,486][20381] Updated weights for policy 1, policy_version 961258 (0.0010) [2023-12-27 12:18:49,539][20381] Updated weights for policy 1, policy_version 961268 (0.0010) [2023-12-27 12:18:49,864][20347] Updated weights for policy 0, policy_version 962045 (0.0011) [2023-12-27 12:18:49,928][20347] Updated weights for policy 0, policy_version 962055 (0.0009) [2023-12-27 12:18:49,990][20347] Updated weights for policy 0, policy_version 962065 (0.0008) [2023-12-27 12:18:50,218][20381] Updated weights for policy 1, policy_version 961278 (0.0008) [2023-12-27 12:18:50,281][20381] Updated weights for policy 1, policy_version 961288 (0.0008) [2023-12-27 12:18:50,340][20381] Updated weights for policy 1, policy_version 961298 (0.0007) [2023-12-27 12:18:50,755][20347] Updated weights for policy 0, policy_version 962075 (0.0009) [2023-12-27 12:18:50,817][20347] Updated weights for policy 0, policy_version 962085 (0.0008) [2023-12-27 12:18:50,877][20347] Updated weights for policy 0, policy_version 962095 (0.0008) [2023-12-27 12:18:51,049][20381] Updated weights for policy 1, policy_version 961308 (0.0008) [2023-12-27 12:18:51,113][20381] Updated weights for policy 1, policy_version 961318 (0.0008) [2023-12-27 12:18:51,179][20381] Updated weights for policy 1, policy_version 961328 (0.0009) [2023-12-27 12:18:51,619][20347] Updated weights for policy 0, policy_version 962105 (0.0007) [2023-12-27 12:18:51,686][20347] Updated weights for policy 0, policy_version 962115 (0.0009) [2023-12-27 12:18:51,750][20347] Updated weights for policy 0, policy_version 962125 (0.0008) [2023-12-27 12:18:51,818][20347] Updated weights for policy 0, policy_version 962135 (0.0009) [2023-12-27 12:18:51,919][20381] Updated weights for policy 1, policy_version 961338 (0.0007) [2023-12-27 12:18:51,976][20381] Updated weights for policy 1, policy_version 961348 (0.0008) [2023-12-27 12:18:52,038][20381] Updated weights for policy 1, policy_version 961358 (0.0008) [2023-12-27 12:18:52,100][20381] Updated weights for policy 1, policy_version 961368 (0.0008) [2023-12-27 12:18:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19933.9, 300 sec: 20271.7). Total num frames: 492486656. Throughput: 0: 9808.3, 1: 10175.5. Samples: 492459208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:52,363][19170] Avg episode reward: [(0, '931.860'), (1, '1000.000')] [2023-12-27 12:18:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000961368_246136832.pth... [2023-12-27 12:18:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000962136_246349824.pth... [2023-12-27 12:18:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000960984_246054912.pth [2023-12-27 12:18:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000960184_245833728.pth [2023-12-27 12:18:52,556][20347] Updated weights for policy 0, policy_version 962145 (0.0008) [2023-12-27 12:18:52,617][20347] Updated weights for policy 0, policy_version 962155 (0.0008) [2023-12-27 12:18:52,671][20347] Updated weights for policy 0, policy_version 962165 (0.0008) [2023-12-27 12:18:52,820][20381] Updated weights for policy 1, policy_version 961378 (0.0007) [2023-12-27 12:18:52,878][20381] Updated weights for policy 1, policy_version 961388 (0.0007) [2023-12-27 12:18:52,930][20381] Updated weights for policy 1, policy_version 961398 (0.0005) [2023-12-27 12:18:53,411][20347] Updated weights for policy 0, policy_version 962175 (0.0007) [2023-12-27 12:18:53,470][20347] Updated weights for policy 0, policy_version 962185 (0.0008) [2023-12-27 12:18:53,529][20347] Updated weights for policy 0, policy_version 962195 (0.0008) [2023-12-27 12:18:53,582][20381] Updated weights for policy 1, policy_version 961408 (0.0008) [2023-12-27 12:18:53,635][20381] Updated weights for policy 1, policy_version 961418 (0.0008) [2023-12-27 12:18:53,695][20381] Updated weights for policy 1, policy_version 961428 (0.0008) [2023-12-27 12:18:54,241][20347] Updated weights for policy 0, policy_version 962205 (0.0008) [2023-12-27 12:18:54,304][20347] Updated weights for policy 0, policy_version 962215 (0.0008) [2023-12-27 12:18:54,362][20347] Updated weights for policy 0, policy_version 962225 (0.0007) [2023-12-27 12:18:54,368][20381] Updated weights for policy 1, policy_version 961438 (0.0006) [2023-12-27 12:18:54,414][20381] Updated weights for policy 1, policy_version 961448 (0.0005) [2023-12-27 12:18:54,460][20381] Updated weights for policy 1, policy_version 961458 (0.0007) [2023-12-27 12:18:55,053][20347] Updated weights for policy 0, policy_version 962235 (0.0007) [2023-12-27 12:18:55,112][20347] Updated weights for policy 0, policy_version 962245 (0.0006) [2023-12-27 12:18:55,140][20381] Updated weights for policy 1, policy_version 961468 (0.0010) [2023-12-27 12:18:55,171][20347] Updated weights for policy 0, policy_version 962255 (0.0006) [2023-12-27 12:18:55,208][20381] Updated weights for policy 1, policy_version 961478 (0.0010) [2023-12-27 12:18:55,279][20381] Updated weights for policy 1, policy_version 961488 (0.0006) [2023-12-27 12:18:55,870][20381] Updated weights for policy 1, policy_version 961498 (0.0008) [2023-12-27 12:18:55,881][20347] Updated weights for policy 0, policy_version 962265 (0.0007) [2023-12-27 12:18:55,935][20381] Updated weights for policy 1, policy_version 961508 (0.0006) [2023-12-27 12:18:55,951][20347] Updated weights for policy 0, policy_version 962275 (0.0007) [2023-12-27 12:18:55,995][20381] Updated weights for policy 1, policy_version 961518 (0.0011) [2023-12-27 12:18:56,015][20347] Updated weights for policy 0, policy_version 962285 (0.0007) [2023-12-27 12:18:56,047][20381] Updated weights for policy 1, policy_version 961528 (0.0011) [2023-12-27 12:18:56,081][20347] Updated weights for policy 0, policy_version 962295 (0.0007) [2023-12-27 12:18:56,622][20381] Updated weights for policy 1, policy_version 961538 (0.0005) [2023-12-27 12:18:56,672][20381] Updated weights for policy 1, policy_version 961548 (0.0005) [2023-12-27 12:18:56,718][20381] Updated weights for policy 1, policy_version 961558 (0.0007) [2023-12-27 12:18:56,721][20347] Updated weights for policy 0, policy_version 962305 (0.0006) [2023-12-27 12:18:56,772][20291] KL-divergence is very high: 1319.3208 [2023-12-27 12:18:56,773][20347] Updated weights for policy 0, policy_version 962315 (0.0007) [2023-12-27 12:18:56,829][20291] KL-divergence is very high: 569.0247 [2023-12-27 12:18:56,839][20347] Updated weights for policy 0, policy_version 962325 (0.0006) [2023-12-27 12:18:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19797.3, 300 sec: 20244.0). Total num frames: 492584960. Throughput: 0: 9807.5, 1: 10200.0. Samples: 492577604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:18:57,363][19170] Avg episode reward: [(0, '951.170'), (1, '1000.000')] [2023-12-27 12:18:57,372][20381] Updated weights for policy 1, policy_version 961568 (0.0008) [2023-12-27 12:18:57,437][20381] Updated weights for policy 1, policy_version 961578 (0.0006) [2023-12-27 12:18:57,507][20381] Updated weights for policy 1, policy_version 961588 (0.0006) [2023-12-27 12:18:57,551][20347] Updated weights for policy 0, policy_version 962335 (0.0005) [2023-12-27 12:18:57,601][20347] Updated weights for policy 0, policy_version 962345 (0.0007) [2023-12-27 12:18:57,651][20347] Updated weights for policy 0, policy_version 962355 (0.0007) [2023-12-27 12:18:58,143][20381] Updated weights for policy 1, policy_version 961598 (0.0006) [2023-12-27 12:18:58,213][20381] Updated weights for policy 1, policy_version 961608 (0.0007) [2023-12-27 12:18:58,274][20381] Updated weights for policy 1, policy_version 961618 (0.0007) [2023-12-27 12:18:58,319][20347] Updated weights for policy 0, policy_version 962365 (0.0008) [2023-12-27 12:18:58,390][20347] Updated weights for policy 0, policy_version 962375 (0.0010) [2023-12-27 12:18:58,467][20347] Updated weights for policy 0, policy_version 962385 (0.0011) [2023-12-27 12:18:59,043][20381] Updated weights for policy 1, policy_version 961628 (0.0007) [2023-12-27 12:18:59,095][20381] Updated weights for policy 1, policy_version 961638 (0.0008) [2023-12-27 12:18:59,160][20381] Updated weights for policy 1, policy_version 961648 (0.0008) [2023-12-27 12:18:59,308][20347] Updated weights for policy 0, policy_version 962395 (0.0008) [2023-12-27 12:18:59,373][20347] Updated weights for policy 0, policy_version 962405 (0.0008) [2023-12-27 12:18:59,438][20347] Updated weights for policy 0, policy_version 962415 (0.0008) [2023-12-27 12:18:59,858][20381] Updated weights for policy 1, policy_version 961658 (0.0007) [2023-12-27 12:18:59,920][20381] Updated weights for policy 1, policy_version 961668 (0.0008) [2023-12-27 12:18:59,986][20381] Updated weights for policy 1, policy_version 961678 (0.0006) [2023-12-27 12:19:00,051][20381] Updated weights for policy 1, policy_version 961688 (0.0008) [2023-12-27 12:19:00,218][20347] Updated weights for policy 0, policy_version 962425 (0.0009) [2023-12-27 12:19:00,278][20347] Updated weights for policy 0, policy_version 962435 (0.0007) [2023-12-27 12:19:00,331][20347] Updated weights for policy 0, policy_version 962445 (0.0006) [2023-12-27 12:19:00,388][20347] Updated weights for policy 0, policy_version 962455 (0.0007) [2023-12-27 12:19:00,632][20381] Updated weights for policy 1, policy_version 961698 (0.0007) [2023-12-27 12:19:00,675][20381] Updated weights for policy 1, policy_version 961708 (0.0006) [2023-12-27 12:19:00,729][20381] Updated weights for policy 1, policy_version 961718 (0.0005) [2023-12-27 12:19:01,067][20347] Updated weights for policy 0, policy_version 962465 (0.0011) [2023-12-27 12:19:01,121][20347] Updated weights for policy 0, policy_version 962475 (0.0010) [2023-12-27 12:19:01,184][20347] Updated weights for policy 0, policy_version 962485 (0.0009) [2023-12-27 12:19:01,356][20381] Updated weights for policy 1, policy_version 961728 (0.0007) [2023-12-27 12:19:01,423][20381] Updated weights for policy 1, policy_version 961738 (0.0008) [2023-12-27 12:19:01,478][20381] Updated weights for policy 1, policy_version 961748 (0.0007) [2023-12-27 12:19:01,973][20347] Updated weights for policy 0, policy_version 962495 (0.0008) [2023-12-27 12:19:02,026][20347] Updated weights for policy 0, policy_version 962505 (0.0008) [2023-12-27 12:19:02,092][20347] Updated weights for policy 0, policy_version 962515 (0.0008) [2023-12-27 12:19:02,220][20381] Updated weights for policy 1, policy_version 961758 (0.0008) [2023-12-27 12:19:02,277][20381] Updated weights for policy 1, policy_version 961768 (0.0007) [2023-12-27 12:19:02,344][20381] Updated weights for policy 1, policy_version 961778 (0.0007) [2023-12-27 12:19:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 20244.0). Total num frames: 492683264. Throughput: 0: 9834.9, 1: 10146.1. Samples: 492695908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:19:02,363][19170] Avg episode reward: [(0, '941.690'), (1, '1000.000')] [2023-12-27 12:19:02,821][20347] Updated weights for policy 0, policy_version 962525 (0.0008) [2023-12-27 12:19:02,872][20347] Updated weights for policy 0, policy_version 962535 (0.0007) [2023-12-27 12:19:02,947][20347] Updated weights for policy 0, policy_version 962545 (0.0008) [2023-12-27 12:19:03,024][20381] Updated weights for policy 1, policy_version 961788 (0.0009) [2023-12-27 12:19:03,082][20381] Updated weights for policy 1, policy_version 961798 (0.0010) [2023-12-27 12:19:03,130][20381] Updated weights for policy 1, policy_version 961808 (0.0010) [2023-12-27 12:19:03,577][20347] Updated weights for policy 0, policy_version 962555 (0.0006) [2023-12-27 12:19:03,626][20347] Updated weights for policy 0, policy_version 962565 (0.0008) [2023-12-27 12:19:03,693][20347] Updated weights for policy 0, policy_version 962575 (0.0009) [2023-12-27 12:19:03,746][20381] Updated weights for policy 1, policy_version 961818 (0.0010) [2023-12-27 12:19:03,800][20381] Updated weights for policy 1, policy_version 961828 (0.0010) [2023-12-27 12:19:03,861][20381] Updated weights for policy 1, policy_version 961838 (0.0010) [2023-12-27 12:19:03,922][20381] Updated weights for policy 1, policy_version 961848 (0.0006) [2023-12-27 12:19:04,488][20347] Updated weights for policy 0, policy_version 962585 (0.0007) [2023-12-27 12:19:04,505][20381] Updated weights for policy 1, policy_version 961858 (0.0011) [2023-12-27 12:19:04,558][20347] Updated weights for policy 0, policy_version 962595 (0.0009) [2023-12-27 12:19:04,562][20381] Updated weights for policy 1, policy_version 961868 (0.0010) [2023-12-27 12:19:04,615][20381] Updated weights for policy 1, policy_version 961878 (0.0010) [2023-12-27 12:19:04,620][20347] Updated weights for policy 0, policy_version 962605 (0.0005) [2023-12-27 12:19:04,681][20347] Updated weights for policy 0, policy_version 962615 (0.0005) [2023-12-27 12:19:05,280][20381] Updated weights for policy 1, policy_version 961888 (0.0010) [2023-12-27 12:19:05,329][20381] Updated weights for policy 1, policy_version 961898 (0.0011) [2023-12-27 12:19:05,356][20347] Updated weights for policy 0, policy_version 962625 (0.0007) [2023-12-27 12:19:05,373][20381] Updated weights for policy 1, policy_version 961908 (0.0010) [2023-12-27 12:19:05,414][20347] Updated weights for policy 0, policy_version 962635 (0.0009) [2023-12-27 12:19:05,470][20347] Updated weights for policy 0, policy_version 962645 (0.0006) [2023-12-27 12:19:06,067][20381] Updated weights for policy 1, policy_version 961918 (0.0010) [2023-12-27 12:19:06,138][20381] Updated weights for policy 1, policy_version 961928 (0.0010) [2023-12-27 12:19:06,202][20381] Updated weights for policy 1, policy_version 961938 (0.0011) [2023-12-27 12:19:06,223][20347] Updated weights for policy 0, policy_version 962655 (0.0009) [2023-12-27 12:19:06,283][20347] Updated weights for policy 0, policy_version 962665 (0.0008) [2023-12-27 12:19:06,339][20347] Updated weights for policy 0, policy_version 962675 (0.0007) [2023-12-27 12:19:06,860][20381] Updated weights for policy 1, policy_version 961948 (0.0011) [2023-12-27 12:19:06,918][20381] Updated weights for policy 1, policy_version 961958 (0.0010) [2023-12-27 12:19:06,973][20381] Updated weights for policy 1, policy_version 961968 (0.0011) [2023-12-27 12:19:07,063][20347] Updated weights for policy 0, policy_version 962685 (0.0008) [2023-12-27 12:19:07,114][20347] Updated weights for policy 0, policy_version 962695 (0.0005) [2023-12-27 12:19:07,175][20347] Updated weights for policy 0, policy_version 962705 (0.0007) [2023-12-27 12:19:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20070.4, 300 sec: 20271.7). Total num frames: 492789760. Throughput: 0: 9865.6, 1: 10178.7. Samples: 492757572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:19:07,363][19170] Avg episode reward: [(0, '931.970'), (1, '1000.000')] [2023-12-27 12:19:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000961976_246292480.pth... [2023-12-27 12:19:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000962712_246497280.pth... [2023-12-27 12:19:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000960792_245989376.pth [2023-12-27 12:19:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000961560_246202368.pth [2023-12-27 12:19:07,676][20381] Updated weights for policy 1, policy_version 961978 (0.0010) [2023-12-27 12:19:07,732][20381] Updated weights for policy 1, policy_version 961988 (0.0008) [2023-12-27 12:19:07,785][20347] Updated weights for policy 0, policy_version 962715 (0.0008) [2023-12-27 12:19:07,789][20381] Updated weights for policy 1, policy_version 961998 (0.0006) [2023-12-27 12:19:07,848][20347] Updated weights for policy 0, policy_version 962725 (0.0007) [2023-12-27 12:19:07,854][20381] Updated weights for policy 1, policy_version 962008 (0.0006) [2023-12-27 12:19:07,903][20347] Updated weights for policy 0, policy_version 962735 (0.0008) [2023-12-27 12:19:08,484][20381] Updated weights for policy 1, policy_version 962018 (0.0009) [2023-12-27 12:19:08,543][20381] Updated weights for policy 1, policy_version 962028 (0.0006) [2023-12-27 12:19:08,563][20347] Updated weights for policy 0, policy_version 962745 (0.0007) [2023-12-27 12:19:08,600][20381] Updated weights for policy 1, policy_version 962038 (0.0007) [2023-12-27 12:19:08,628][20347] Updated weights for policy 0, policy_version 962755 (0.0007) [2023-12-27 12:19:08,691][20347] Updated weights for policy 0, policy_version 962765 (0.0008) [2023-12-27 12:19:08,760][20347] Updated weights for policy 0, policy_version 962775 (0.0008) [2023-12-27 12:19:09,306][20381] Updated weights for policy 1, policy_version 962048 (0.0010) [2023-12-27 12:19:09,371][20381] Updated weights for policy 1, policy_version 962058 (0.0008) [2023-12-27 12:19:09,439][20381] Updated weights for policy 1, policy_version 962068 (0.0009) [2023-12-27 12:19:09,467][20347] Updated weights for policy 0, policy_version 962785 (0.0006) [2023-12-27 12:19:09,522][20347] Updated weights for policy 0, policy_version 962795 (0.0007) [2023-12-27 12:19:09,582][20347] Updated weights for policy 0, policy_version 962805 (0.0008) [2023-12-27 12:19:10,239][20381] Updated weights for policy 1, policy_version 962078 (0.0009) [2023-12-27 12:19:10,240][20347] Updated weights for policy 0, policy_version 962815 (0.0007) [2023-12-27 12:19:10,292][20347] Updated weights for policy 0, policy_version 962825 (0.0006) [2023-12-27 12:19:10,296][20381] Updated weights for policy 1, policy_version 962088 (0.0009) [2023-12-27 12:19:10,347][20347] Updated weights for policy 0, policy_version 962835 (0.0008) [2023-12-27 12:19:10,358][20381] Updated weights for policy 1, policy_version 962098 (0.0009) [2023-12-27 12:19:11,035][20347] Updated weights for policy 0, policy_version 962845 (0.0007) [2023-12-27 12:19:11,106][20347] Updated weights for policy 0, policy_version 962855 (0.0008) [2023-12-27 12:19:11,148][20381] Updated weights for policy 1, policy_version 962108 (0.0008) [2023-12-27 12:19:11,198][20347] Updated weights for policy 0, policy_version 962865 (0.0008) [2023-12-27 12:19:11,210][20381] Updated weights for policy 1, policy_version 962118 (0.0007) [2023-12-27 12:19:11,267][20381] Updated weights for policy 1, policy_version 962128 (0.0007) [2023-12-27 12:19:11,908][20347] Updated weights for policy 0, policy_version 962875 (0.0007) [2023-12-27 12:19:11,969][20347] Updated weights for policy 0, policy_version 962885 (0.0006) [2023-12-27 12:19:11,988][20381] Updated weights for policy 1, policy_version 962138 (0.0007) [2023-12-27 12:19:12,035][20347] Updated weights for policy 0, policy_version 962895 (0.0008) [2023-12-27 12:19:12,049][20381] Updated weights for policy 1, policy_version 962148 (0.0007) [2023-12-27 12:19:12,109][20381] Updated weights for policy 1, policy_version 962158 (0.0008) [2023-12-27 12:19:12,176][20381] Updated weights for policy 1, policy_version 962168 (0.0007) [2023-12-27 12:19:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20070.4, 300 sec: 20244.0). Total num frames: 492888064. Throughput: 0: 9897.6, 1: 10127.6. Samples: 492876516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:19:12,363][19170] Avg episode reward: [(0, '922.100'), (1, '1000.000')] [2023-12-27 12:19:12,727][20347] Updated weights for policy 0, policy_version 962905 (0.0007) [2023-12-27 12:19:12,793][20347] Updated weights for policy 0, policy_version 962915 (0.0007) [2023-12-27 12:19:12,859][20347] Updated weights for policy 0, policy_version 962925 (0.0007) [2023-12-27 12:19:12,915][20347] Updated weights for policy 0, policy_version 962935 (0.0007) [2023-12-27 12:19:12,937][20381] Updated weights for policy 1, policy_version 962178 (0.0011) [2023-12-27 12:19:12,985][20381] Updated weights for policy 1, policy_version 962188 (0.0010) [2023-12-27 12:19:13,034][20381] Updated weights for policy 1, policy_version 962198 (0.0010) [2023-12-27 12:19:13,512][20347] Updated weights for policy 0, policy_version 962945 (0.0005) [2023-12-27 12:19:13,571][20347] Updated weights for policy 0, policy_version 962955 (0.0009) [2023-12-27 12:19:13,635][20347] Updated weights for policy 0, policy_version 962965 (0.0009) [2023-12-27 12:19:13,649][20381] Updated weights for policy 1, policy_version 962208 (0.0007) [2023-12-27 12:19:13,706][20381] Updated weights for policy 1, policy_version 962218 (0.0010) [2023-12-27 12:19:13,761][20381] Updated weights for policy 1, policy_version 962228 (0.0009) [2023-12-27 12:19:14,314][20347] Updated weights for policy 0, policy_version 962975 (0.0006) [2023-12-27 12:19:14,320][20381] Updated weights for policy 1, policy_version 962238 (0.0006) [2023-12-27 12:19:14,366][20347] Updated weights for policy 0, policy_version 962985 (0.0006) [2023-12-27 12:19:14,382][20381] Updated weights for policy 1, policy_version 962248 (0.0006) [2023-12-27 12:19:14,427][20347] Updated weights for policy 0, policy_version 962995 (0.0007) [2023-12-27 12:19:14,437][20381] Updated weights for policy 1, policy_version 962258 (0.0007) [2023-12-27 12:19:15,143][20381] Updated weights for policy 1, policy_version 962268 (0.0007) [2023-12-27 12:19:15,153][20347] Updated weights for policy 0, policy_version 963005 (0.0007) [2023-12-27 12:19:15,194][20381] Updated weights for policy 1, policy_version 962278 (0.0011) [2023-12-27 12:19:15,222][20347] Updated weights for policy 0, policy_version 963015 (0.0007) [2023-12-27 12:19:15,258][20381] Updated weights for policy 1, policy_version 962288 (0.0011) [2023-12-27 12:19:15,288][20347] Updated weights for policy 0, policy_version 963025 (0.0008) [2023-12-27 12:19:15,940][20381] Updated weights for policy 1, policy_version 962298 (0.0009) [2023-12-27 12:19:15,993][20381] Updated weights for policy 1, policy_version 962308 (0.0005) [2023-12-27 12:19:15,999][20347] Updated weights for policy 0, policy_version 963035 (0.0006) [2023-12-27 12:19:16,049][20381] Updated weights for policy 1, policy_version 962318 (0.0006) [2023-12-27 12:19:16,064][20347] Updated weights for policy 0, policy_version 963045 (0.0007) [2023-12-27 12:19:16,104][20381] Updated weights for policy 1, policy_version 962328 (0.0005) [2023-12-27 12:19:16,118][20347] Updated weights for policy 0, policy_version 963055 (0.0008) [2023-12-27 12:19:16,710][20381] Updated weights for policy 1, policy_version 962338 (0.0007) [2023-12-27 12:19:16,758][20381] Updated weights for policy 1, policy_version 962348 (0.0008) [2023-12-27 12:19:16,813][20381] Updated weights for policy 1, policy_version 962358 (0.0008) [2023-12-27 12:19:16,834][20347] Updated weights for policy 0, policy_version 963065 (0.0008) [2023-12-27 12:19:16,883][20347] Updated weights for policy 0, policy_version 963075 (0.0009) [2023-12-27 12:19:16,932][20347] Updated weights for policy 0, policy_version 963085 (0.0009) [2023-12-27 12:19:16,977][20347] Updated weights for policy 0, policy_version 963095 (0.0007) [2023-12-27 12:19:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19934.8, 300 sec: 20244.0). Total num frames: 492986368. Throughput: 0: 9849.6, 1: 10159.6. Samples: 492997136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:19:17,363][19170] Avg episode reward: [(0, '921.980'), (1, '1000.000')] [2023-12-27 12:19:17,455][20381] Updated weights for policy 1, policy_version 962368 (0.0007) [2023-12-27 12:19:17,512][20381] Updated weights for policy 1, policy_version 962378 (0.0007) [2023-12-27 12:19:17,571][20381] Updated weights for policy 1, policy_version 962388 (0.0007) [2023-12-27 12:19:17,735][20347] Updated weights for policy 0, policy_version 963105 (0.0008) [2023-12-27 12:19:17,777][20347] Updated weights for policy 0, policy_version 963115 (0.0007) [2023-12-27 12:19:17,837][20347] Updated weights for policy 0, policy_version 963125 (0.0008) [2023-12-27 12:19:18,285][20381] Updated weights for policy 1, policy_version 962398 (0.0006) [2023-12-27 12:19:18,354][20381] Updated weights for policy 1, policy_version 962408 (0.0009) [2023-12-27 12:19:18,404][20381] Updated weights for policy 1, policy_version 962418 (0.0008) [2023-12-27 12:19:18,588][20347] Updated weights for policy 0, policy_version 963135 (0.0007) [2023-12-27 12:19:18,672][20347] Updated weights for policy 0, policy_version 963145 (0.0007) [2023-12-27 12:19:18,723][20347] Updated weights for policy 0, policy_version 963155 (0.0006) [2023-12-27 12:19:19,092][20381] Updated weights for policy 1, policy_version 962428 (0.0007) [2023-12-27 12:19:19,154][20381] Updated weights for policy 1, policy_version 962438 (0.0005) [2023-12-27 12:19:19,211][20381] Updated weights for policy 1, policy_version 962448 (0.0008) [2023-12-27 12:19:19,447][20347] Updated weights for policy 0, policy_version 963165 (0.0009) [2023-12-27 12:19:19,512][20347] Updated weights for policy 0, policy_version 963175 (0.0007) [2023-12-27 12:19:19,570][20347] Updated weights for policy 0, policy_version 963185 (0.0007) [2023-12-27 12:19:19,903][20381] Updated weights for policy 1, policy_version 962458 (0.0007) [2023-12-27 12:19:19,970][20381] Updated weights for policy 1, policy_version 962468 (0.0010) [2023-12-27 12:19:20,036][20381] Updated weights for policy 1, policy_version 962478 (0.0007) [2023-12-27 12:19:20,096][20381] Updated weights for policy 1, policy_version 962488 (0.0011) [2023-12-27 12:19:20,391][20347] Updated weights for policy 0, policy_version 963195 (0.0007) [2023-12-27 12:19:20,470][20347] Updated weights for policy 0, policy_version 963205 (0.0007) [2023-12-27 12:19:20,536][20347] Updated weights for policy 0, policy_version 963215 (0.0007) [2023-12-27 12:19:20,820][20381] Updated weights for policy 1, policy_version 962498 (0.0006) [2023-12-27 12:19:20,879][20381] Updated weights for policy 1, policy_version 962508 (0.0008) [2023-12-27 12:19:20,929][20381] Updated weights for policy 1, policy_version 962518 (0.0007) [2023-12-27 12:19:21,335][20347] Updated weights for policy 0, policy_version 963225 (0.0008) [2023-12-27 12:19:21,404][20347] Updated weights for policy 0, policy_version 963235 (0.0008) [2023-12-27 12:19:21,460][20347] Updated weights for policy 0, policy_version 963245 (0.0007) [2023-12-27 12:19:21,516][20347] Updated weights for policy 0, policy_version 963255 (0.0008) [2023-12-27 12:19:21,729][20381] Updated weights for policy 1, policy_version 962528 (0.0009) [2023-12-27 12:19:21,794][20381] Updated weights for policy 1, policy_version 962538 (0.0009) [2023-12-27 12:19:21,880][20381] Updated weights for policy 1, policy_version 962548 (0.0008) [2023-12-27 12:19:22,320][20347] Updated weights for policy 0, policy_version 963265 (0.0009) [2023-12-27 12:19:22,362][19170] Fps is (10 sec: 18841.6, 60 sec: 19933.9, 300 sec: 20188.4). Total num frames: 493076480. Throughput: 0: 9786.4, 1: 10108.1. Samples: 493055640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:19:22,363][19170] Avg episode reward: [(0, '922.050'), (1, '1000.000')] [2023-12-27 12:19:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000962552_246439936.pth... [2023-12-27 12:19:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000961368_246136832.pth [2023-12-27 12:19:22,391][20347] Updated weights for policy 0, policy_version 963275 (0.0008) [2023-12-27 12:19:22,444][20347] Updated weights for policy 0, policy_version 963285 (0.0008) [2023-12-27 12:19:22,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000963288_246644736.pth... [2023-12-27 12:19:22,464][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000962136_246349824.pth [2023-12-27 12:19:22,566][20381] Updated weights for policy 1, policy_version 962558 (0.0008) [2023-12-27 12:19:22,622][20381] Updated weights for policy 1, policy_version 962568 (0.0009) [2023-12-27 12:19:22,672][20381] Updated weights for policy 1, policy_version 962578 (0.0007) [2023-12-27 12:19:23,124][20347] Updated weights for policy 0, policy_version 963295 (0.0007) [2023-12-27 12:19:23,172][20347] Updated weights for policy 0, policy_version 963305 (0.0008) [2023-12-27 12:19:23,228][20347] Updated weights for policy 0, policy_version 963315 (0.0008) [2023-12-27 12:19:23,376][20381] Updated weights for policy 1, policy_version 962588 (0.0008) [2023-12-27 12:19:23,422][20381] Updated weights for policy 1, policy_version 962598 (0.0009) [2023-12-27 12:19:23,472][20381] Updated weights for policy 1, policy_version 962608 (0.0009) [2023-12-27 12:19:23,830][20347] Updated weights for policy 0, policy_version 963325 (0.0007) [2023-12-27 12:19:23,893][20347] Updated weights for policy 0, policy_version 963335 (0.0007) [2023-12-27 12:19:23,949][20347] Updated weights for policy 0, policy_version 963345 (0.0008) [2023-12-27 12:19:24,135][20381] Updated weights for policy 1, policy_version 962618 (0.0008) [2023-12-27 12:19:24,189][20381] Updated weights for policy 1, policy_version 962628 (0.0009) [2023-12-27 12:19:24,250][20381] Updated weights for policy 1, policy_version 962638 (0.0009) [2023-12-27 12:19:24,300][20381] Updated weights for policy 1, policy_version 962648 (0.0008) [2023-12-27 12:19:24,544][20347] Updated weights for policy 0, policy_version 963355 (0.0008) [2023-12-27 12:19:24,599][20347] Updated weights for policy 0, policy_version 963365 (0.0008) [2023-12-27 12:19:24,659][20347] Updated weights for policy 0, policy_version 963375 (0.0007) [2023-12-27 12:19:24,943][20381] Updated weights for policy 1, policy_version 962658 (0.0008) [2023-12-27 12:19:24,990][20381] Updated weights for policy 1, policy_version 962668 (0.0007) [2023-12-27 12:19:25,040][20381] Updated weights for policy 1, policy_version 962678 (0.0006) [2023-12-27 12:19:25,356][20347] Updated weights for policy 0, policy_version 963385 (0.0007) [2023-12-27 12:19:25,414][20347] Updated weights for policy 0, policy_version 963395 (0.0005) [2023-12-27 12:19:25,475][20347] Updated weights for policy 0, policy_version 963405 (0.0005) [2023-12-27 12:19:25,534][20347] Updated weights for policy 0, policy_version 963415 (0.0005) [2023-12-27 12:19:25,782][20381] Updated weights for policy 1, policy_version 962688 (0.0007) [2023-12-27 12:19:25,842][20381] Updated weights for policy 1, policy_version 962698 (0.0008) [2023-12-27 12:19:25,890][20381] Updated weights for policy 1, policy_version 962708 (0.0009) [2023-12-27 12:19:26,121][20347] Updated weights for policy 0, policy_version 963425 (0.0007) [2023-12-27 12:19:26,175][20347] Updated weights for policy 0, policy_version 963435 (0.0007) [2023-12-27 12:19:26,228][20347] Updated weights for policy 0, policy_version 963445 (0.0005) [2023-12-27 12:19:26,592][20381] Updated weights for policy 1, policy_version 962718 (0.0008) [2023-12-27 12:19:26,644][20381] Updated weights for policy 1, policy_version 962728 (0.0007) [2023-12-27 12:19:26,698][20381] Updated weights for policy 1, policy_version 962738 (0.0008) [2023-12-27 12:19:26,841][20347] Updated weights for policy 0, policy_version 963455 (0.0006) [2023-12-27 12:19:26,888][20347] Updated weights for policy 0, policy_version 963465 (0.0008) [2023-12-27 12:19:26,941][20347] Updated weights for policy 0, policy_version 963475 (0.0007) [2023-12-27 12:19:27,354][20381] Updated weights for policy 1, policy_version 962748 (0.0007) [2023-12-27 12:19:27,362][19170] Fps is (10 sec: 19660.5, 60 sec: 19933.9, 300 sec: 20188.4). Total num frames: 493182976. Throughput: 0: 9793.1, 1: 10080.7. Samples: 493174728. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:27,363][19170] Avg episode reward: [(0, '941.440'), (1, '1000.000')] [2023-12-27 12:19:27,425][20381] Updated weights for policy 1, policy_version 962758 (0.0006) [2023-12-27 12:19:27,481][20381] Updated weights for policy 1, policy_version 962768 (0.0009) [2023-12-27 12:19:27,540][20347] Updated weights for policy 0, policy_version 963485 (0.0006) [2023-12-27 12:19:27,591][20347] Updated weights for policy 0, policy_version 963495 (0.0009) [2023-12-27 12:19:27,647][20347] Updated weights for policy 0, policy_version 963505 (0.0008) [2023-12-27 12:19:28,136][20381] Updated weights for policy 1, policy_version 962778 (0.0008) [2023-12-27 12:19:28,185][20381] Updated weights for policy 1, policy_version 962788 (0.0008) [2023-12-27 12:19:28,240][20381] Updated weights for policy 1, policy_version 962798 (0.0009) [2023-12-27 12:19:28,296][20381] Updated weights for policy 1, policy_version 962808 (0.0005) [2023-12-27 12:19:28,302][20347] Updated weights for policy 0, policy_version 963515 (0.0008) [2023-12-27 12:19:28,370][20347] Updated weights for policy 0, policy_version 963525 (0.0007) [2023-12-27 12:19:28,453][20347] Updated weights for policy 0, policy_version 963535 (0.0009) [2023-12-27 12:19:29,069][20381] Updated weights for policy 1, policy_version 962818 (0.0008) [2023-12-27 12:19:29,128][20381] Updated weights for policy 1, policy_version 962828 (0.0006) [2023-12-27 12:19:29,192][20381] Updated weights for policy 1, policy_version 962838 (0.0006) [2023-12-27 12:19:29,298][20347] Updated weights for policy 0, policy_version 963545 (0.0008) [2023-12-27 12:19:29,371][20347] Updated weights for policy 0, policy_version 963555 (0.0008) [2023-12-27 12:19:29,436][20347] Updated weights for policy 0, policy_version 963565 (0.0008) [2023-12-27 12:19:29,510][20347] Updated weights for policy 0, policy_version 963575 (0.0008) [2023-12-27 12:19:29,922][20381] Updated weights for policy 1, policy_version 962848 (0.0008) [2023-12-27 12:19:29,991][20381] Updated weights for policy 1, policy_version 962858 (0.0008) [2023-12-27 12:19:30,056][20381] Updated weights for policy 1, policy_version 962868 (0.0007) [2023-12-27 12:19:30,207][20347] Updated weights for policy 0, policy_version 963585 (0.0006) [2023-12-27 12:19:30,263][20347] Updated weights for policy 0, policy_version 963595 (0.0007) [2023-12-27 12:19:30,317][20347] Updated weights for policy 0, policy_version 963605 (0.0006) [2023-12-27 12:19:30,766][20381] Updated weights for policy 1, policy_version 962878 (0.0008) [2023-12-27 12:19:30,818][20381] Updated weights for policy 1, policy_version 962888 (0.0008) [2023-12-27 12:19:30,876][20381] Updated weights for policy 1, policy_version 962898 (0.0008) [2023-12-27 12:19:30,929][20347] Updated weights for policy 0, policy_version 963615 (0.0006) [2023-12-27 12:19:30,976][20347] Updated weights for policy 0, policy_version 963625 (0.0005) [2023-12-27 12:19:31,024][20347] Updated weights for policy 0, policy_version 963635 (0.0005) [2023-12-27 12:19:31,623][20381] Updated weights for policy 1, policy_version 962908 (0.0007) [2023-12-27 12:19:31,676][20347] Updated weights for policy 0, policy_version 963645 (0.0007) [2023-12-27 12:19:31,688][20381] Updated weights for policy 1, policy_version 962918 (0.0008) [2023-12-27 12:19:31,741][20347] Updated weights for policy 0, policy_version 963655 (0.0009) [2023-12-27 12:19:31,753][20381] Updated weights for policy 1, policy_version 962928 (0.0008) [2023-12-27 12:19:31,804][20347] Updated weights for policy 0, policy_version 963665 (0.0006) [2023-12-27 12:19:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19933.9, 300 sec: 20188.4). Total num frames: 493281280. Throughput: 0: 9846.7, 1: 10044.0. Samples: 493293732. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:32,363][19170] Avg episode reward: [(0, '883.260'), (1, '1000.000')] [2023-12-27 12:19:32,482][20381] Updated weights for policy 1, policy_version 962938 (0.0008) [2023-12-27 12:19:32,497][20347] Updated weights for policy 0, policy_version 963675 (0.0005) [2023-12-27 12:19:32,545][20381] Updated weights for policy 1, policy_version 962948 (0.0008) [2023-12-27 12:19:32,557][20347] Updated weights for policy 0, policy_version 963685 (0.0008) [2023-12-27 12:19:32,594][20381] Updated weights for policy 1, policy_version 962958 (0.0007) [2023-12-27 12:19:32,614][20347] Updated weights for policy 0, policy_version 963695 (0.0009) [2023-12-27 12:19:32,651][20381] Updated weights for policy 1, policy_version 962968 (0.0006) [2023-12-27 12:19:33,315][20347] Updated weights for policy 0, policy_version 963705 (0.0009) [2023-12-27 12:19:33,342][20381] Updated weights for policy 1, policy_version 962978 (0.0008) [2023-12-27 12:19:33,373][20347] Updated weights for policy 0, policy_version 963715 (0.0005) [2023-12-27 12:19:33,390][20381] Updated weights for policy 1, policy_version 962988 (0.0007) [2023-12-27 12:19:33,429][20347] Updated weights for policy 0, policy_version 963725 (0.0005) [2023-12-27 12:19:33,436][20291] KL-divergence is very high: 102.5464 [2023-12-27 12:19:33,438][20381] Updated weights for policy 1, policy_version 962998 (0.0008) [2023-12-27 12:19:33,484][20347] Updated weights for policy 0, policy_version 963735 (0.0009) [2023-12-27 12:19:34,182][20381] Updated weights for policy 1, policy_version 963008 (0.0008) [2023-12-27 12:19:34,232][20347] Updated weights for policy 0, policy_version 963745 (0.0008) [2023-12-27 12:19:34,245][20381] Updated weights for policy 1, policy_version 963018 (0.0007) [2023-12-27 12:19:34,289][20347] Updated weights for policy 0, policy_version 963755 (0.0010) [2023-12-27 12:19:34,303][20381] Updated weights for policy 1, policy_version 963028 (0.0007) [2023-12-27 12:19:34,357][20347] Updated weights for policy 0, policy_version 963765 (0.0007) [2023-12-27 12:19:35,079][20347] Updated weights for policy 0, policy_version 963775 (0.0008) [2023-12-27 12:19:35,131][20381] Updated weights for policy 1, policy_version 963038 (0.0006) [2023-12-27 12:19:35,143][20347] Updated weights for policy 0, policy_version 963785 (0.0008) [2023-12-27 12:19:35,186][20381] Updated weights for policy 1, policy_version 963048 (0.0007) [2023-12-27 12:19:35,196][20347] Updated weights for policy 0, policy_version 963795 (0.0006) [2023-12-27 12:19:35,233][20381] Updated weights for policy 1, policy_version 963058 (0.0006) [2023-12-27 12:19:35,905][20347] Updated weights for policy 0, policy_version 963805 (0.0007) [2023-12-27 12:19:35,915][20381] Updated weights for policy 1, policy_version 963068 (0.0007) [2023-12-27 12:19:35,959][20347] Updated weights for policy 0, policy_version 963815 (0.0007) [2023-12-27 12:19:35,965][20381] Updated weights for policy 1, policy_version 963078 (0.0005) [2023-12-27 12:19:36,015][20347] Updated weights for policy 0, policy_version 963825 (0.0006) [2023-12-27 12:19:36,021][20381] Updated weights for policy 1, policy_version 963088 (0.0006) [2023-12-27 12:19:36,727][20347] Updated weights for policy 0, policy_version 963835 (0.0006) [2023-12-27 12:19:36,747][20381] Updated weights for policy 1, policy_version 963098 (0.0007) [2023-12-27 12:19:36,788][20347] Updated weights for policy 0, policy_version 963845 (0.0006) [2023-12-27 12:19:36,801][20381] Updated weights for policy 1, policy_version 963108 (0.0006) [2023-12-27 12:19:36,841][20347] Updated weights for policy 0, policy_version 963855 (0.0005) [2023-12-27 12:19:36,855][20381] Updated weights for policy 1, policy_version 963118 (0.0006) [2023-12-27 12:19:36,905][20381] Updated weights for policy 1, policy_version 963128 (0.0007) [2023-12-27 12:19:37,363][19170] Fps is (10 sec: 19660.4, 60 sec: 19797.5, 300 sec: 20160.6). Total num frames: 493379584. Throughput: 0: 9798.0, 1: 10043.1. Samples: 493352064. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:37,364][19170] Avg episode reward: [(0, '873.670'), (1, '1000.000')] [2023-12-27 12:19:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000963128_246587392.pth... [2023-12-27 12:19:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000963864_246792192.pth... [2023-12-27 12:19:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000961976_246292480.pth [2023-12-27 12:19:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000962712_246497280.pth [2023-12-27 12:19:37,478][20347] Updated weights for policy 0, policy_version 963865 (0.0008) [2023-12-27 12:19:37,539][20347] Updated weights for policy 0, policy_version 963875 (0.0006) [2023-12-27 12:19:37,596][20347] Updated weights for policy 0, policy_version 963885 (0.0008) [2023-12-27 12:19:37,661][20347] Updated weights for policy 0, policy_version 963895 (0.0007) [2023-12-27 12:19:37,682][20381] Updated weights for policy 1, policy_version 963138 (0.0008) [2023-12-27 12:19:37,747][20381] Updated weights for policy 1, policy_version 963148 (0.0008) [2023-12-27 12:19:37,803][20381] Updated weights for policy 1, policy_version 963158 (0.0007) [2023-12-27 12:19:38,433][20347] Updated weights for policy 0, policy_version 963905 (0.0008) [2023-12-27 12:19:38,474][20381] Updated weights for policy 1, policy_version 963168 (0.0007) [2023-12-27 12:19:38,490][20347] Updated weights for policy 0, policy_version 963915 (0.0006) [2023-12-27 12:19:38,528][20381] Updated weights for policy 1, policy_version 963178 (0.0007) [2023-12-27 12:19:38,545][20347] Updated weights for policy 0, policy_version 963925 (0.0005) [2023-12-27 12:19:38,587][20381] Updated weights for policy 1, policy_version 963188 (0.0007) [2023-12-27 12:19:39,228][20347] Updated weights for policy 0, policy_version 963935 (0.0007) [2023-12-27 12:19:39,286][20347] Updated weights for policy 0, policy_version 963945 (0.0008) [2023-12-27 12:19:39,338][20381] Updated weights for policy 1, policy_version 963198 (0.0008) [2023-12-27 12:19:39,349][20347] Updated weights for policy 0, policy_version 963955 (0.0008) [2023-12-27 12:19:39,410][20381] Updated weights for policy 1, policy_version 963208 (0.0008) [2023-12-27 12:19:39,471][20381] Updated weights for policy 1, policy_version 963218 (0.0008) [2023-12-27 12:19:40,132][20381] Updated weights for policy 1, policy_version 963228 (0.0007) [2023-12-27 12:19:40,173][20347] Updated weights for policy 0, policy_version 963967 (0.0008) [2023-12-27 12:19:40,195][20381] Updated weights for policy 1, policy_version 963238 (0.0006) [2023-12-27 12:19:40,232][20347] Updated weights for policy 0, policy_version 963977 (0.0007) [2023-12-27 12:19:40,263][20381] Updated weights for policy 1, policy_version 963248 (0.0008) [2023-12-27 12:19:40,293][20347] Updated weights for policy 0, policy_version 963987 (0.0006) [2023-12-27 12:19:40,963][20347] Updated weights for policy 0, policy_version 963997 (0.0007) [2023-12-27 12:19:40,968][20291] KL-divergence is very high: 148.3235 [2023-12-27 12:19:40,974][20381] Updated weights for policy 1, policy_version 963258 (0.0008) [2023-12-27 12:19:41,021][20347] Updated weights for policy 0, policy_version 964007 (0.0008) [2023-12-27 12:19:41,035][20381] Updated weights for policy 1, policy_version 963268 (0.0008) [2023-12-27 12:19:41,094][20347] Updated weights for policy 0, policy_version 964017 (0.0011) [2023-12-27 12:19:41,100][20381] Updated weights for policy 1, policy_version 963278 (0.0008) [2023-12-27 12:19:41,164][20381] Updated weights for policy 1, policy_version 963288 (0.0007) [2023-12-27 12:19:41,858][20347] Updated weights for policy 0, policy_version 964027 (0.0007) [2023-12-27 12:19:41,905][20381] Updated weights for policy 1, policy_version 963298 (0.0008) [2023-12-27 12:19:41,914][20347] Updated weights for policy 0, policy_version 964037 (0.0008) [2023-12-27 12:19:41,962][20381] Updated weights for policy 1, policy_version 963308 (0.0008) [2023-12-27 12:19:41,965][20347] Updated weights for policy 0, policy_version 964047 (0.0007) [2023-12-27 12:19:41,972][20291] KL-divergence is very high: 100.4651 [2023-12-27 12:19:42,021][20381] Updated weights for policy 1, policy_version 963318 (0.0008) [2023-12-27 12:19:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19797.3, 300 sec: 20160.6). Total num frames: 493477888. Throughput: 0: 9819.3, 1: 9976.6. Samples: 493468420. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:42,363][19170] Avg episode reward: [(0, '912.590'), (1, '1000.000')] [2023-12-27 12:19:42,715][20347] Updated weights for policy 0, policy_version 964057 (0.0007) [2023-12-27 12:19:42,774][20347] Updated weights for policy 0, policy_version 964067 (0.0007) [2023-12-27 12:19:42,809][20381] Updated weights for policy 1, policy_version 963328 (0.0010) [2023-12-27 12:19:42,827][20347] Updated weights for policy 0, policy_version 964077 (0.0007) [2023-12-27 12:19:42,870][20381] Updated weights for policy 1, policy_version 963338 (0.0010) [2023-12-27 12:19:42,885][20347] Updated weights for policy 0, policy_version 964087 (0.0005) [2023-12-27 12:19:42,930][20381] Updated weights for policy 1, policy_version 963348 (0.0007) [2023-12-27 12:19:43,472][20381] Updated weights for policy 1, policy_version 963358 (0.0005) [2023-12-27 12:19:43,541][20381] Updated weights for policy 1, policy_version 963368 (0.0007) [2023-12-27 12:19:43,579][20347] Updated weights for policy 0, policy_version 964097 (0.0007) [2023-12-27 12:19:43,608][20381] Updated weights for policy 1, policy_version 963378 (0.0005) [2023-12-27 12:19:43,629][20347] Updated weights for policy 0, policy_version 964107 (0.0008) [2023-12-27 12:19:43,678][20347] Updated weights for policy 0, policy_version 964117 (0.0011) [2023-12-27 12:19:44,220][20381] Updated weights for policy 1, policy_version 963388 (0.0009) [2023-12-27 12:19:44,261][20347] Updated weights for policy 0, policy_version 964127 (0.0006) [2023-12-27 12:19:44,276][20381] Updated weights for policy 1, policy_version 963398 (0.0007) [2023-12-27 12:19:44,309][20347] Updated weights for policy 0, policy_version 964137 (0.0007) [2023-12-27 12:19:44,332][20381] Updated weights for policy 1, policy_version 963408 (0.0005) [2023-12-27 12:19:44,364][20347] Updated weights for policy 0, policy_version 964147 (0.0006) [2023-12-27 12:19:44,961][20381] Updated weights for policy 1, policy_version 963418 (0.0006) [2023-12-27 12:19:45,021][20381] Updated weights for policy 1, policy_version 963428 (0.0007) [2023-12-27 12:19:45,074][20347] Updated weights for policy 0, policy_version 964157 (0.0009) [2023-12-27 12:19:45,076][20381] Updated weights for policy 1, policy_version 963438 (0.0007) [2023-12-27 12:19:45,133][20381] Updated weights for policy 1, policy_version 963448 (0.0007) [2023-12-27 12:19:45,134][20347] Updated weights for policy 0, policy_version 964167 (0.0011) [2023-12-27 12:19:45,187][20347] Updated weights for policy 0, policy_version 964177 (0.0011) [2023-12-27 12:19:45,836][20381] Updated weights for policy 1, policy_version 963458 (0.0007) [2023-12-27 12:19:45,882][20347] Updated weights for policy 0, policy_version 964187 (0.0011) [2023-12-27 12:19:45,899][20381] Updated weights for policy 1, policy_version 963468 (0.0007) [2023-12-27 12:19:45,937][20347] Updated weights for policy 0, policy_version 964197 (0.0011) [2023-12-27 12:19:45,958][20381] Updated weights for policy 1, policy_version 963478 (0.0006) [2023-12-27 12:19:45,995][20347] Updated weights for policy 0, policy_version 964207 (0.0011) [2023-12-27 12:19:46,633][20347] Updated weights for policy 0, policy_version 964217 (0.0010) [2023-12-27 12:19:46,663][20381] Updated weights for policy 1, policy_version 963488 (0.0006) [2023-12-27 12:19:46,692][20347] Updated weights for policy 0, policy_version 964227 (0.0007) [2023-12-27 12:19:46,713][20381] Updated weights for policy 1, policy_version 963498 (0.0009) [2023-12-27 12:19:46,745][20347] Updated weights for policy 0, policy_version 964237 (0.0009) [2023-12-27 12:19:46,762][20381] Updated weights for policy 1, policy_version 963508 (0.0005) [2023-12-27 12:19:46,797][20347] Updated weights for policy 0, policy_version 964247 (0.0010) [2023-12-27 12:19:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 19797.4, 300 sec: 20160.7). Total num frames: 493576192. Throughput: 0: 9891.8, 1: 9973.6. Samples: 493589852. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:47,363][19170] Avg episode reward: [(0, '883.250'), (1, '1000.000')] [2023-12-27 12:19:47,483][20381] Updated weights for policy 1, policy_version 963518 (0.0006) [2023-12-27 12:19:47,496][20347] Updated weights for policy 0, policy_version 964257 (0.0007) [2023-12-27 12:19:47,532][20381] Updated weights for policy 1, policy_version 963528 (0.0005) [2023-12-27 12:19:47,552][20347] Updated weights for policy 0, policy_version 964267 (0.0006) [2023-12-27 12:19:47,595][20381] Updated weights for policy 1, policy_version 963538 (0.0006) [2023-12-27 12:19:47,608][20347] Updated weights for policy 0, policy_version 964277 (0.0006) [2023-12-27 12:19:48,226][20347] Updated weights for policy 0, policy_version 964287 (0.0005) [2023-12-27 12:19:48,277][20347] Updated weights for policy 0, policy_version 964297 (0.0005) [2023-12-27 12:19:48,278][20381] Updated weights for policy 1, policy_version 963548 (0.0006) [2023-12-27 12:19:48,327][20347] Updated weights for policy 0, policy_version 964307 (0.0006) [2023-12-27 12:19:48,336][20381] Updated weights for policy 1, policy_version 963558 (0.0007) [2023-12-27 12:19:48,396][20381] Updated weights for policy 1, policy_version 963568 (0.0006) [2023-12-27 12:19:49,022][20381] Updated weights for policy 1, policy_version 963578 (0.0006) [2023-12-27 12:19:49,023][20347] Updated weights for policy 0, policy_version 964317 (0.0009) [2023-12-27 12:19:49,076][20347] Updated weights for policy 0, policy_version 964327 (0.0009) [2023-12-27 12:19:49,078][20381] Updated weights for policy 1, policy_version 963588 (0.0005) [2023-12-27 12:19:49,132][20381] Updated weights for policy 1, policy_version 963598 (0.0007) [2023-12-27 12:19:49,134][20347] Updated weights for policy 0, policy_version 964337 (0.0005) [2023-12-27 12:19:49,186][20381] Updated weights for policy 1, policy_version 963608 (0.0006) [2023-12-27 12:19:49,808][20347] Updated weights for policy 0, policy_version 964347 (0.0010) [2023-12-27 12:19:49,870][20347] Updated weights for policy 0, policy_version 964357 (0.0011) [2023-12-27 12:19:49,873][20381] Updated weights for policy 1, policy_version 963618 (0.0008) [2023-12-27 12:19:49,928][20347] Updated weights for policy 0, policy_version 964367 (0.0010) [2023-12-27 12:19:49,937][20381] Updated weights for policy 1, policy_version 963628 (0.0007) [2023-12-27 12:19:49,998][20381] Updated weights for policy 1, policy_version 963638 (0.0008) [2023-12-27 12:19:50,679][20381] Updated weights for policy 1, policy_version 963648 (0.0008) [2023-12-27 12:19:50,700][20347] Updated weights for policy 0, policy_version 964377 (0.0008) [2023-12-27 12:19:50,737][20381] Updated weights for policy 1, policy_version 963658 (0.0007) [2023-12-27 12:19:50,761][20347] Updated weights for policy 0, policy_version 964387 (0.0008) [2023-12-27 12:19:50,805][20381] Updated weights for policy 1, policy_version 963668 (0.0006) [2023-12-27 12:19:50,823][20347] Updated weights for policy 0, policy_version 964397 (0.0006) [2023-12-27 12:19:50,881][20347] Updated weights for policy 0, policy_version 964407 (0.0007) [2023-12-27 12:19:51,586][20347] Updated weights for policy 0, policy_version 964417 (0.0006) [2023-12-27 12:19:51,602][20381] Updated weights for policy 1, policy_version 963678 (0.0008) [2023-12-27 12:19:51,656][20347] Updated weights for policy 0, policy_version 964427 (0.0007) [2023-12-27 12:19:51,669][20381] Updated weights for policy 1, policy_version 963688 (0.0008) [2023-12-27 12:19:51,716][20347] Updated weights for policy 0, policy_version 964437 (0.0008) [2023-12-27 12:19:51,728][20381] Updated weights for policy 1, policy_version 963698 (0.0007) [2023-12-27 12:19:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 19797.3, 300 sec: 20105.1). Total num frames: 493674496. Throughput: 0: 9922.1, 1: 9932.6. Samples: 493651036. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:52,363][19170] Avg episode reward: [(0, '921.910'), (1, '1000.000')] [2023-12-27 12:19:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000963704_246734848.pth... [2023-12-27 12:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000962552_246439936.pth [2023-12-27 12:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_000963704_246734848.pth [2023-12-27 12:19:52,374][20347] Updated weights for policy 0, policy_version 964447 (0.0010) [2023-12-27 12:19:52,432][20347] Updated weights for policy 0, policy_version 964457 (0.0007) [2023-12-27 12:19:52,436][20381] Updated weights for policy 1, policy_version 963708 (0.0008) [2023-12-27 12:19:52,487][20347] Updated weights for policy 0, policy_version 964467 (0.0006) [2023-12-27 12:19:52,496][20381] Updated weights for policy 1, policy_version 963718 (0.0008) [2023-12-27 12:19:52,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000964472_246947840.pth... [2023-12-27 12:19:52,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000963288_246644736.pth [2023-12-27 12:19:52,518][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_000964472_246947840.pth [2023-12-27 12:19:52,548][20381] Updated weights for policy 1, policy_version 963728 (0.0007) [2023-12-27 12:19:53,231][20347] Updated weights for policy 0, policy_version 964477 (0.0008) [2023-12-27 12:19:53,248][20381] Updated weights for policy 1, policy_version 963738 (0.0007) [2023-12-27 12:19:53,296][20347] Updated weights for policy 0, policy_version 964487 (0.0010) [2023-12-27 12:19:53,307][20381] Updated weights for policy 1, policy_version 963748 (0.0006) [2023-12-27 12:19:53,355][20347] Updated weights for policy 0, policy_version 964497 (0.0010) [2023-12-27 12:19:53,365][20381] Updated weights for policy 1, policy_version 963758 (0.0006) [2023-12-27 12:19:53,424][20381] Updated weights for policy 1, policy_version 963768 (0.0007) [2023-12-27 12:19:53,970][20347] Updated weights for policy 0, policy_version 964507 (0.0010) [2023-12-27 12:19:54,029][20347] Updated weights for policy 0, policy_version 964517 (0.0007) [2023-12-27 12:19:54,057][20381] Updated weights for policy 1, policy_version 963778 (0.0005) [2023-12-27 12:19:54,088][20347] Updated weights for policy 0, policy_version 964527 (0.0007) [2023-12-27 12:19:54,123][20381] Updated weights for policy 1, policy_version 963788 (0.0006) [2023-12-27 12:19:54,182][20381] Updated weights for policy 1, policy_version 963798 (0.0009) [2023-12-27 12:19:54,726][20347] Updated weights for policy 0, policy_version 964537 (0.0007) [2023-12-27 12:19:54,783][20381] Updated weights for policy 1, policy_version 963808 (0.0010) [2023-12-27 12:19:54,791][20347] Updated weights for policy 0, policy_version 964547 (0.0007) [2023-12-27 12:19:54,845][20381] Updated weights for policy 1, policy_version 963818 (0.0010) [2023-12-27 12:19:54,855][20347] Updated weights for policy 0, policy_version 964557 (0.0005) [2023-12-27 12:19:54,908][20381] Updated weights for policy 1, policy_version 963828 (0.0011) [2023-12-27 12:19:54,913][20347] Updated weights for policy 0, policy_version 964567 (0.0006) [2023-12-27 12:19:55,504][20347] Updated weights for policy 0, policy_version 964577 (0.0010) [2023-12-27 12:19:55,548][20347] Updated weights for policy 0, policy_version 964587 (0.0010) [2023-12-27 12:19:55,590][20381] Updated weights for policy 1, policy_version 963838 (0.0009) [2023-12-27 12:19:55,597][20347] Updated weights for policy 0, policy_version 964597 (0.0010) [2023-12-27 12:19:55,648][20381] Updated weights for policy 1, policy_version 963848 (0.0011) [2023-12-27 12:19:55,696][20381] Updated weights for policy 1, policy_version 963858 (0.0010) [2023-12-27 12:19:56,238][20347] Updated weights for policy 0, policy_version 964607 (0.0007) [2023-12-27 12:19:56,293][20347] Updated weights for policy 0, policy_version 964617 (0.0009) [2023-12-27 12:19:56,339][20381] Updated weights for policy 1, policy_version 963868 (0.0010) [2023-12-27 12:19:56,351][20347] Updated weights for policy 0, policy_version 964627 (0.0010) [2023-12-27 12:19:56,396][20381] Updated weights for policy 1, policy_version 963878 (0.0009) [2023-12-27 12:19:56,447][20381] Updated weights for policy 1, policy_version 963888 (0.0005) [2023-12-27 12:19:56,943][20347] Updated weights for policy 0, policy_version 964637 (0.0006) [2023-12-27 12:19:57,005][20347] Updated weights for policy 0, policy_version 964647 (0.0009) [2023-12-27 12:19:57,067][20347] Updated weights for policy 0, policy_version 964657 (0.0010) [2023-12-27 12:19:57,070][20381] Updated weights for policy 1, policy_version 963898 (0.0006) [2023-12-27 12:19:57,119][20381] Updated weights for policy 1, policy_version 963908 (0.0010) [2023-12-27 12:19:57,163][20381] Updated weights for policy 1, policy_version 963918 (0.0010) [2023-12-27 12:19:57,217][20381] Updated weights for policy 1, policy_version 963928 (0.0010) [2023-12-27 12:19:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20070.4, 300 sec: 20160.7). Total num frames: 493789184. Throughput: 0: 9948.0, 1: 9989.6. Samples: 493773708. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:19:57,363][19170] Avg episode reward: [(0, '941.560'), (1, '1000.000')] [2023-12-27 12:19:57,649][20347] Updated weights for policy 0, policy_version 964667 (0.0005) [2023-12-27 12:19:57,699][20347] Updated weights for policy 0, policy_version 964677 (0.0006) [2023-12-27 12:19:57,758][20347] Updated weights for policy 0, policy_version 964687 (0.0008) [2023-12-27 12:19:57,897][20381] Updated weights for policy 1, policy_version 963938 (0.0010) [2023-12-27 12:19:57,953][20381] Updated weights for policy 1, policy_version 963948 (0.0010) [2023-12-27 12:19:58,008][20381] Updated weights for policy 1, policy_version 963958 (0.0011) [2023-12-27 12:19:58,433][20347] Updated weights for policy 0, policy_version 964697 (0.0008) [2023-12-27 12:19:58,501][20347] Updated weights for policy 0, policy_version 964707 (0.0007) [2023-12-27 12:19:58,567][20347] Updated weights for policy 0, policy_version 964717 (0.0006) [2023-12-27 12:19:58,638][20347] Updated weights for policy 0, policy_version 964727 (0.0007) [2023-12-27 12:19:58,743][20381] Updated weights for policy 1, policy_version 963968 (0.0009) [2023-12-27 12:19:58,811][20381] Updated weights for policy 1, policy_version 963978 (0.0010) [2023-12-27 12:19:58,876][20381] Updated weights for policy 1, policy_version 963988 (0.0007) [2023-12-27 12:19:59,347][20347] Updated weights for policy 0, policy_version 964737 (0.0008) [2023-12-27 12:19:59,411][20347] Updated weights for policy 0, policy_version 964747 (0.0008) [2023-12-27 12:19:59,482][20347] Updated weights for policy 0, policy_version 964757 (0.0009) [2023-12-27 12:19:59,627][20381] Updated weights for policy 1, policy_version 963998 (0.0007) [2023-12-27 12:19:59,683][20381] Updated weights for policy 1, policy_version 964008 (0.0006) [2023-12-27 12:19:59,744][20381] Updated weights for policy 1, policy_version 964018 (0.0006) [2023-12-27 12:20:00,142][20347] Updated weights for policy 0, policy_version 964767 (0.0008) [2023-12-27 12:20:00,204][20347] Updated weights for policy 0, policy_version 964777 (0.0007) [2023-12-27 12:20:00,273][20347] Updated weights for policy 0, policy_version 964787 (0.0006) [2023-12-27 12:20:00,427][20381] Updated weights for policy 1, policy_version 964028 (0.0008) [2023-12-27 12:20:00,479][20381] Updated weights for policy 1, policy_version 964038 (0.0008) [2023-12-27 12:20:00,538][20381] Updated weights for policy 1, policy_version 964048 (0.0008) [2023-12-27 12:20:00,862][20347] Updated weights for policy 0, policy_version 964797 (0.0007) [2023-12-27 12:20:00,921][20347] Updated weights for policy 0, policy_version 964807 (0.0005) [2023-12-27 12:20:00,976][20347] Updated weights for policy 0, policy_version 964817 (0.0005) [2023-12-27 12:20:01,188][20381] Updated weights for policy 1, policy_version 964058 (0.0007) [2023-12-27 12:20:01,242][20381] Updated weights for policy 1, policy_version 964068 (0.0007) [2023-12-27 12:20:01,305][20381] Updated weights for policy 1, policy_version 964078 (0.0008) [2023-12-27 12:20:01,367][20381] Updated weights for policy 1, policy_version 964088 (0.0008) [2023-12-27 12:20:01,632][20347] Updated weights for policy 0, policy_version 964827 (0.0006) [2023-12-27 12:20:01,657][20291] KL-divergence is very high: 142.3527 [2023-12-27 12:20:01,682][20347] Updated weights for policy 0, policy_version 964837 (0.0005) [2023-12-27 12:20:01,696][20291] KL-divergence is very high: 144.4746 [2023-12-27 12:20:01,743][20347] Updated weights for policy 0, policy_version 964847 (0.0007) [2023-12-27 12:20:02,045][20381] Updated weights for policy 1, policy_version 964098 (0.0008) [2023-12-27 12:20:02,103][20381] Updated weights for policy 1, policy_version 964108 (0.0007) [2023-12-27 12:20:02,159][20381] Updated weights for policy 1, policy_version 964118 (0.0007) [2023-12-27 12:20:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20070.4, 300 sec: 20160.6). Total num frames: 493887488. Throughput: 0: 10059.4, 1: 9936.9. Samples: 493896972. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:02,363][19170] Avg episode reward: [(0, '912.360'), (1, '1000.000')] [2023-12-27 12:20:02,433][20347] Updated weights for policy 0, policy_version 964857 (0.0007) [2023-12-27 12:20:02,493][20347] Updated weights for policy 0, policy_version 964867 (0.0006) [2023-12-27 12:20:02,547][20347] Updated weights for policy 0, policy_version 964877 (0.0005) [2023-12-27 12:20:02,606][20347] Updated weights for policy 0, policy_version 964887 (0.0007) [2023-12-27 12:20:02,842][20381] Updated weights for policy 1, policy_version 964128 (0.0008) [2023-12-27 12:20:02,896][20381] Updated weights for policy 1, policy_version 964138 (0.0010) [2023-12-27 12:20:02,948][20381] Updated weights for policy 1, policy_version 964148 (0.0010) [2023-12-27 12:20:03,186][20347] Updated weights for policy 0, policy_version 964897 (0.0005) [2023-12-27 12:20:03,235][20347] Updated weights for policy 0, policy_version 964907 (0.0005) [2023-12-27 12:20:03,286][20347] Updated weights for policy 0, policy_version 964917 (0.0005) [2023-12-27 12:20:03,578][20381] Updated weights for policy 1, policy_version 964158 (0.0010) [2023-12-27 12:20:03,626][20381] Updated weights for policy 1, policy_version 964169 (0.0011) [2023-12-27 12:20:03,675][20381] Updated weights for policy 1, policy_version 964179 (0.0006) [2023-12-27 12:20:03,992][20347] Updated weights for policy 0, policy_version 964927 (0.0007) [2023-12-27 12:20:04,050][20347] Updated weights for policy 0, policy_version 964937 (0.0007) [2023-12-27 12:20:04,109][20347] Updated weights for policy 0, policy_version 964947 (0.0007) [2023-12-27 12:20:04,351][20381] Updated weights for policy 1, policy_version 964189 (0.0006) [2023-12-27 12:20:04,412][20381] Updated weights for policy 1, policy_version 964199 (0.0008) [2023-12-27 12:20:04,471][20381] Updated weights for policy 1, policy_version 964209 (0.0008) [2023-12-27 12:20:04,778][20347] Updated weights for policy 0, policy_version 964957 (0.0007) [2023-12-27 12:20:04,833][20347] Updated weights for policy 0, policy_version 964967 (0.0008) [2023-12-27 12:20:04,891][20347] Updated weights for policy 0, policy_version 964977 (0.0006) [2023-12-27 12:20:05,168][20381] Updated weights for policy 1, policy_version 964219 (0.0006) [2023-12-27 12:20:05,232][20381] Updated weights for policy 1, policy_version 964229 (0.0005) [2023-12-27 12:20:05,294][20381] Updated weights for policy 1, policy_version 964239 (0.0005) [2023-12-27 12:20:05,483][20347] Updated weights for policy 0, policy_version 964987 (0.0007) [2023-12-27 12:20:05,537][20347] Updated weights for policy 0, policy_version 964997 (0.0010) [2023-12-27 12:20:05,602][20347] Updated weights for policy 0, policy_version 965007 (0.0010) [2023-12-27 12:20:05,809][20381] Updated weights for policy 1, policy_version 964249 (0.0006) [2023-12-27 12:20:05,864][20381] Updated weights for policy 1, policy_version 964259 (0.0010) [2023-12-27 12:20:05,912][20381] Updated weights for policy 1, policy_version 964269 (0.0007) [2023-12-27 12:20:05,971][20381] Updated weights for policy 1, policy_version 964279 (0.0006) [2023-12-27 12:20:06,175][20347] Updated weights for policy 0, policy_version 965017 (0.0009) [2023-12-27 12:20:06,239][20347] Updated weights for policy 0, policy_version 965027 (0.0006) [2023-12-27 12:20:06,297][20347] Updated weights for policy 0, policy_version 965037 (0.0006) [2023-12-27 12:20:06,351][20347] Updated weights for policy 0, policy_version 965047 (0.0006) [2023-12-27 12:20:06,625][20381] Updated weights for policy 1, policy_version 964289 (0.0008) [2023-12-27 12:20:06,679][20381] Updated weights for policy 1, policy_version 964299 (0.0008) [2023-12-27 12:20:06,725][20381] Updated weights for policy 1, policy_version 964309 (0.0008) [2023-12-27 12:20:06,962][20347] Updated weights for policy 0, policy_version 965057 (0.0007) [2023-12-27 12:20:07,015][20347] Updated weights for policy 0, policy_version 965067 (0.0007) [2023-12-27 12:20:07,065][20347] Updated weights for policy 0, policy_version 965077 (0.0007) [2023-12-27 12:20:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20070.4, 300 sec: 20160.6). Total num frames: 493993984. Throughput: 0: 10131.7, 1: 9971.4. Samples: 493960280. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:07,363][19170] Avg episode reward: [(0, '912.360'), (1, '1000.000')] [2023-12-27 12:20:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000964312_246890496.pth... [2023-12-27 12:20:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000965080_247103488.pth... [2023-12-27 12:20:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000963128_246587392.pth [2023-12-27 12:20:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000963864_246792192.pth [2023-12-27 12:20:07,453][20381] Updated weights for policy 1, policy_version 964319 (0.0006) [2023-12-27 12:20:07,508][20381] Updated weights for policy 1, policy_version 964329 (0.0005) [2023-12-27 12:20:07,563][20381] Updated weights for policy 1, policy_version 964339 (0.0005) [2023-12-27 12:20:07,746][20347] Updated weights for policy 0, policy_version 965087 (0.0008) [2023-12-27 12:20:07,801][20347] Updated weights for policy 0, policy_version 965097 (0.0007) [2023-12-27 12:20:07,853][20347] Updated weights for policy 0, policy_version 965107 (0.0007) [2023-12-27 12:20:08,167][20381] Updated weights for policy 1, policy_version 964349 (0.0005) [2023-12-27 12:20:08,219][20381] Updated weights for policy 1, policy_version 964359 (0.0005) [2023-12-27 12:20:08,270][20381] Updated weights for policy 1, policy_version 964369 (0.0005) [2023-12-27 12:20:08,540][20347] Updated weights for policy 0, policy_version 965117 (0.0008) [2023-12-27 12:20:08,587][20347] Updated weights for policy 0, policy_version 965127 (0.0007) [2023-12-27 12:20:08,640][20347] Updated weights for policy 0, policy_version 965137 (0.0007) [2023-12-27 12:20:08,947][20381] Updated weights for policy 1, policy_version 964379 (0.0007) [2023-12-27 12:20:09,002][20381] Updated weights for policy 1, policy_version 964389 (0.0010) [2023-12-27 12:20:09,061][20381] Updated weights for policy 1, policy_version 964399 (0.0011) [2023-12-27 12:20:09,332][20347] Updated weights for policy 0, policy_version 965147 (0.0007) [2023-12-27 12:20:09,398][20347] Updated weights for policy 0, policy_version 965157 (0.0008) [2023-12-27 12:20:09,464][20347] Updated weights for policy 0, policy_version 965167 (0.0009) [2023-12-27 12:20:09,807][20381] Updated weights for policy 1, policy_version 964409 (0.0010) [2023-12-27 12:20:09,877][20381] Updated weights for policy 1, policy_version 964419 (0.0008) [2023-12-27 12:20:09,945][20381] Updated weights for policy 1, policy_version 964429 (0.0008) [2023-12-27 12:20:10,006][20381] Updated weights for policy 1, policy_version 964439 (0.0008) [2023-12-27 12:20:10,119][20347] Updated weights for policy 0, policy_version 965177 (0.0008) [2023-12-27 12:20:10,178][20347] Updated weights for policy 0, policy_version 965187 (0.0008) [2023-12-27 12:20:10,190][20291] KL-divergence is very high: 388.4342 [2023-12-27 12:20:10,221][20291] KL-divergence is very high: 112.6762 [2023-12-27 12:20:10,236][20291] KL-divergence is very high: 575.0389 [2023-12-27 12:20:10,237][20347] Updated weights for policy 0, policy_version 965197 (0.0008) [2023-12-27 12:20:10,261][20291] KL-divergence is very high: 104.6985 [2023-12-27 12:20:10,276][20291] KL-divergence is very high: 421.4008 [2023-12-27 12:20:10,287][20347] Updated weights for policy 0, policy_version 965207 (0.0007) [2023-12-27 12:20:10,700][20381] Updated weights for policy 1, policy_version 964449 (0.0006) [2023-12-27 12:20:10,764][20381] Updated weights for policy 1, policy_version 964459 (0.0010) [2023-12-27 12:20:10,820][20381] Updated weights for policy 1, policy_version 964469 (0.0008) [2023-12-27 12:20:10,976][20291] KL-divergence is very high: 238.7535 [2023-12-27 12:20:11,016][20291] KL-divergence is very high: 110.5768 [2023-12-27 12:20:11,017][20347] Updated weights for policy 0, policy_version 965217 (0.0010) [2023-12-27 12:20:11,067][20291] KL-divergence is very high: 114.4801 [2023-12-27 12:20:11,081][20347] Updated weights for policy 0, policy_version 965227 (0.0008) [2023-12-27 12:20:11,118][20291] KL-divergence is very high: 110.6175 [2023-12-27 12:20:11,146][20347] Updated weights for policy 0, policy_version 965237 (0.0008) [2023-12-27 12:20:11,504][20381] Updated weights for policy 1, policy_version 964479 (0.0006) [2023-12-27 12:20:11,564][20381] Updated weights for policy 1, policy_version 964489 (0.0007) [2023-12-27 12:20:11,634][20381] Updated weights for policy 1, policy_version 964499 (0.0008) [2023-12-27 12:20:11,994][20347] Updated weights for policy 0, policy_version 965247 (0.0007) [2023-12-27 12:20:12,049][20347] Updated weights for policy 0, policy_version 965257 (0.0009) [2023-12-27 12:20:12,103][20347] Updated weights for policy 0, policy_version 965267 (0.0007) [2023-12-27 12:20:12,332][20381] Updated weights for policy 1, policy_version 964509 (0.0007) [2023-12-27 12:20:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20070.4, 300 sec: 20160.7). Total num frames: 494092288. Throughput: 0: 10209.6, 1: 10039.8. Samples: 494085952. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:12,363][19170] Avg episode reward: [(0, '912.140'), (1, '1000.000')] [2023-12-27 12:20:12,400][20381] Updated weights for policy 1, policy_version 964519 (0.0007) [2023-12-27 12:20:12,453][20381] Updated weights for policy 1, policy_version 964529 (0.0006) [2023-12-27 12:20:12,836][20347] Updated weights for policy 0, policy_version 965277 (0.0007) [2023-12-27 12:20:12,883][20347] Updated weights for policy 0, policy_version 965287 (0.0008) [2023-12-27 12:20:12,934][20347] Updated weights for policy 0, policy_version 965297 (0.0005) [2023-12-27 12:20:13,081][20381] Updated weights for policy 1, policy_version 964539 (0.0007) [2023-12-27 12:20:13,146][20381] Updated weights for policy 1, policy_version 964549 (0.0006) [2023-12-27 12:20:13,211][20381] Updated weights for policy 1, policy_version 964559 (0.0007) [2023-12-27 12:20:13,568][20347] Updated weights for policy 0, policy_version 965307 (0.0009) [2023-12-27 12:20:13,622][20347] Updated weights for policy 0, policy_version 965317 (0.0010) [2023-12-27 12:20:13,679][20347] Updated weights for policy 0, policy_version 965327 (0.0010) [2023-12-27 12:20:13,847][20381] Updated weights for policy 1, policy_version 964569 (0.0008) [2023-12-27 12:20:13,901][20381] Updated weights for policy 1, policy_version 964579 (0.0010) [2023-12-27 12:20:13,962][20381] Updated weights for policy 1, policy_version 964589 (0.0006) [2023-12-27 12:20:14,023][20381] Updated weights for policy 1, policy_version 964599 (0.0006) [2023-12-27 12:20:14,261][20347] Updated weights for policy 0, policy_version 965337 (0.0009) [2023-12-27 12:20:14,319][20347] Updated weights for policy 0, policy_version 965347 (0.0006) [2023-12-27 12:20:14,373][20347] Updated weights for policy 0, policy_version 965357 (0.0010) [2023-12-27 12:20:14,431][20347] Updated weights for policy 0, policy_version 965367 (0.0010) [2023-12-27 12:20:14,659][20381] Updated weights for policy 1, policy_version 964609 (0.0006) [2023-12-27 12:20:14,723][20381] Updated weights for policy 1, policy_version 964619 (0.0006) [2023-12-27 12:20:14,786][20381] Updated weights for policy 1, policy_version 964629 (0.0008) [2023-12-27 12:20:15,101][20347] Updated weights for policy 0, policy_version 965377 (0.0011) [2023-12-27 12:20:15,159][20347] Updated weights for policy 0, policy_version 965387 (0.0011) [2023-12-27 12:20:15,212][20347] Updated weights for policy 0, policy_version 965397 (0.0008) [2023-12-27 12:20:15,445][20381] Updated weights for policy 1, policy_version 964639 (0.0010) [2023-12-27 12:20:15,495][20381] Updated weights for policy 1, policy_version 964649 (0.0010) [2023-12-27 12:20:15,544][20381] Updated weights for policy 1, policy_version 964659 (0.0010) [2023-12-27 12:20:15,889][20347] Updated weights for policy 0, policy_version 965407 (0.0009) [2023-12-27 12:20:15,944][20347] Updated weights for policy 0, policy_version 965417 (0.0010) [2023-12-27 12:20:15,996][20347] Updated weights for policy 0, policy_version 965427 (0.0010) [2023-12-27 12:20:16,215][20381] Updated weights for policy 1, policy_version 964669 (0.0011) [2023-12-27 12:20:16,270][20381] Updated weights for policy 1, policy_version 964679 (0.0011) [2023-12-27 12:20:16,325][20381] Updated weights for policy 1, policy_version 964689 (0.0011) [2023-12-27 12:20:16,589][20347] Updated weights for policy 0, policy_version 965437 (0.0010) [2023-12-27 12:20:16,637][20347] Updated weights for policy 0, policy_version 965447 (0.0010) [2023-12-27 12:20:16,688][20347] Updated weights for policy 0, policy_version 965457 (0.0010) [2023-12-27 12:20:16,973][20381] Updated weights for policy 1, policy_version 964699 (0.0009) [2023-12-27 12:20:17,031][20381] Updated weights for policy 1, policy_version 964709 (0.0006) [2023-12-27 12:20:17,089][20381] Updated weights for policy 1, policy_version 964719 (0.0006) [2023-12-27 12:20:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20216.2). Total num frames: 494198784. Throughput: 0: 10227.7, 1: 10146.1. Samples: 494210556. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:17,363][19170] Avg episode reward: [(0, '863.260'), (1, '1000.000')] [2023-12-27 12:20:17,371][20347] Updated weights for policy 0, policy_version 965467 (0.0010) [2023-12-27 12:20:17,431][20347] Updated weights for policy 0, policy_version 965477 (0.0007) [2023-12-27 12:20:17,495][20347] Updated weights for policy 0, policy_version 965487 (0.0009) [2023-12-27 12:20:17,706][20381] Updated weights for policy 1, policy_version 964729 (0.0006) [2023-12-27 12:20:17,751][20381] Updated weights for policy 1, policy_version 964739 (0.0007) [2023-12-27 12:20:17,799][20381] Updated weights for policy 1, policy_version 964749 (0.0007) [2023-12-27 12:20:17,841][20381] Updated weights for policy 1, policy_version 964759 (0.0007) [2023-12-27 12:20:18,154][20347] Updated weights for policy 0, policy_version 965497 (0.0010) [2023-12-27 12:20:18,204][20347] Updated weights for policy 0, policy_version 965507 (0.0005) [2023-12-27 12:20:18,266][20347] Updated weights for policy 0, policy_version 965517 (0.0011) [2023-12-27 12:20:18,328][20347] Updated weights for policy 0, policy_version 965527 (0.0007) [2023-12-27 12:20:18,561][20381] Updated weights for policy 1, policy_version 964769 (0.0010) [2023-12-27 12:20:18,612][20381] Updated weights for policy 1, policy_version 964779 (0.0009) [2023-12-27 12:20:18,666][20381] Updated weights for policy 1, policy_version 964789 (0.0006) [2023-12-27 12:20:18,993][20347] Updated weights for policy 0, policy_version 965537 (0.0008) [2023-12-27 12:20:18,994][20291] KL-divergence is very high: 102.4568 [2023-12-27 12:20:19,057][20347] Updated weights for policy 0, policy_version 965547 (0.0007) [2023-12-27 12:20:19,110][20347] Updated weights for policy 0, policy_version 965557 (0.0009) [2023-12-27 12:20:19,308][20381] Updated weights for policy 1, policy_version 964799 (0.0009) [2023-12-27 12:20:19,382][20381] Updated weights for policy 1, policy_version 964809 (0.0008) [2023-12-27 12:20:19,438][20381] Updated weights for policy 1, policy_version 964819 (0.0008) [2023-12-27 12:20:19,868][20347] Updated weights for policy 0, policy_version 965567 (0.0008) [2023-12-27 12:20:19,925][20347] Updated weights for policy 0, policy_version 965577 (0.0007) [2023-12-27 12:20:19,988][20347] Updated weights for policy 0, policy_version 965587 (0.0006) [2023-12-27 12:20:20,168][20381] Updated weights for policy 1, policy_version 964829 (0.0007) [2023-12-27 12:20:20,220][20381] Updated weights for policy 1, policy_version 964839 (0.0008) [2023-12-27 12:20:20,270][20381] Updated weights for policy 1, policy_version 964849 (0.0009) [2023-12-27 12:20:20,701][20347] Updated weights for policy 0, policy_version 965597 (0.0006) [2023-12-27 12:20:20,764][20347] Updated weights for policy 0, policy_version 965607 (0.0008) [2023-12-27 12:20:20,827][20347] Updated weights for policy 0, policy_version 965617 (0.0008) [2023-12-27 12:20:20,967][20381] Updated weights for policy 1, policy_version 964859 (0.0008) [2023-12-27 12:20:21,034][20381] Updated weights for policy 1, policy_version 964869 (0.0008) [2023-12-27 12:20:21,091][20381] Updated weights for policy 1, policy_version 964879 (0.0007) [2023-12-27 12:20:21,560][20347] Updated weights for policy 0, policy_version 965627 (0.0009) [2023-12-27 12:20:21,625][20347] Updated weights for policy 0, policy_version 965637 (0.0008) [2023-12-27 12:20:21,693][20347] Updated weights for policy 0, policy_version 965647 (0.0008) [2023-12-27 12:20:21,765][20381] Updated weights for policy 1, policy_version 964889 (0.0008) [2023-12-27 12:20:21,827][20381] Updated weights for policy 1, policy_version 964899 (0.0008) [2023-12-27 12:20:21,886][20381] Updated weights for policy 1, policy_version 964909 (0.0008) [2023-12-27 12:20:21,945][20381] Updated weights for policy 1, policy_version 964919 (0.0008) [2023-12-27 12:20:22,358][20347] Updated weights for policy 0, policy_version 965657 (0.0008) [2023-12-27 12:20:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20160.6). Total num frames: 494297088. Throughput: 0: 10232.7, 1: 10178.2. Samples: 494270552. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:22,363][19170] Avg episode reward: [(0, '853.530'), (1, '1000.000')] [2023-12-27 12:20:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000964920_247046144.pth... [2023-12-27 12:20:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000963704_246734848.pth [2023-12-27 12:20:22,428][20347] Updated weights for policy 0, policy_version 965667 (0.0008) [2023-12-27 12:20:22,436][20291] KL-divergence is very high: 106.4547 [2023-12-27 12:20:22,489][20347] Updated weights for policy 0, policy_version 965677 (0.0007) [2023-12-27 12:20:22,546][20347] Updated weights for policy 0, policy_version 965687 (0.0006) [2023-12-27 12:20:22,551][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000965688_247259136.pth... [2023-12-27 12:20:22,554][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000964472_246947840.pth [2023-12-27 12:20:22,664][20381] Updated weights for policy 1, policy_version 964929 (0.0008) [2023-12-27 12:20:22,724][20381] Updated weights for policy 1, policy_version 964939 (0.0007) [2023-12-27 12:20:22,780][20381] Updated weights for policy 1, policy_version 964949 (0.0007) [2023-12-27 12:20:23,276][20347] Updated weights for policy 0, policy_version 965697 (0.0007) [2023-12-27 12:20:23,339][20347] Updated weights for policy 0, policy_version 965707 (0.0008) [2023-12-27 12:20:23,401][20347] Updated weights for policy 0, policy_version 965717 (0.0007) [2023-12-27 12:20:23,404][20381] Updated weights for policy 1, policy_version 964959 (0.0006) [2023-12-27 12:20:23,465][20381] Updated weights for policy 1, policy_version 964969 (0.0010) [2023-12-27 12:20:23,520][20381] Updated weights for policy 1, policy_version 964979 (0.0011) [2023-12-27 12:20:24,004][20347] Updated weights for policy 0, policy_version 965727 (0.0006) [2023-12-27 12:20:24,052][20347] Updated weights for policy 0, policy_version 965737 (0.0005) [2023-12-27 12:20:24,104][20347] Updated weights for policy 0, policy_version 965747 (0.0005) [2023-12-27 12:20:24,185][20381] Updated weights for policy 1, policy_version 964989 (0.0009) [2023-12-27 12:20:24,240][20381] Updated weights for policy 1, policy_version 964999 (0.0007) [2023-12-27 12:20:24,301][20381] Updated weights for policy 1, policy_version 965009 (0.0009) [2023-12-27 12:20:24,724][20347] Updated weights for policy 0, policy_version 965757 (0.0006) [2023-12-27 12:20:24,773][20347] Updated weights for policy 0, policy_version 965768 (0.0008) [2023-12-27 12:20:24,831][20347] Updated weights for policy 0, policy_version 965778 (0.0005) [2023-12-27 12:20:24,947][20381] Updated weights for policy 1, policy_version 965019 (0.0009) [2023-12-27 12:20:25,008][20381] Updated weights for policy 1, policy_version 965029 (0.0011) [2023-12-27 12:20:25,059][20381] Updated weights for policy 1, policy_version 965039 (0.0010) [2023-12-27 12:20:25,496][20347] Updated weights for policy 0, policy_version 965788 (0.0007) [2023-12-27 12:20:25,554][20347] Updated weights for policy 0, policy_version 965798 (0.0010) [2023-12-27 12:20:25,612][20347] Updated weights for policy 0, policy_version 965808 (0.0010) [2023-12-27 12:20:25,733][20381] Updated weights for policy 1, policy_version 965049 (0.0006) [2023-12-27 12:20:25,779][20381] Updated weights for policy 1, policy_version 965059 (0.0011) [2023-12-27 12:20:25,832][20381] Updated weights for policy 1, policy_version 965069 (0.0011) [2023-12-27 12:20:25,881][20381] Updated weights for policy 1, policy_version 965079 (0.0011) [2023-12-27 12:20:26,247][20347] Updated weights for policy 0, policy_version 965818 (0.0007) [2023-12-27 12:20:26,305][20347] Updated weights for policy 0, policy_version 965828 (0.0010) [2023-12-27 12:20:26,361][20347] Updated weights for policy 0, policy_version 965838 (0.0010) [2023-12-27 12:20:26,416][20347] Updated weights for policy 0, policy_version 965848 (0.0009) [2023-12-27 12:20:26,581][20381] Updated weights for policy 1, policy_version 965089 (0.0010) [2023-12-27 12:20:26,626][20381] Updated weights for policy 1, policy_version 965099 (0.0010) [2023-12-27 12:20:26,681][20381] Updated weights for policy 1, policy_version 965109 (0.0011) [2023-12-27 12:20:27,156][20347] Updated weights for policy 0, policy_version 965858 (0.0009) [2023-12-27 12:20:27,210][20347] Updated weights for policy 0, policy_version 965868 (0.0007) [2023-12-27 12:20:27,261][20347] Updated weights for policy 0, policy_version 965878 (0.0007) [2023-12-27 12:20:27,307][20381] Updated weights for policy 1, policy_version 965119 (0.0011) [2023-12-27 12:20:27,362][20381] Updated weights for policy 1, policy_version 965129 (0.0010) [2023-12-27 12:20:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20188.4). Total num frames: 494403584. Throughput: 0: 10315.7, 1: 10246.2. Samples: 494393700. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:27,363][19170] Avg episode reward: [(0, '825.040'), (1, '1000.000')] [2023-12-27 12:20:27,421][20381] Updated weights for policy 1, policy_version 965139 (0.0007) [2023-12-27 12:20:27,920][20347] Updated weights for policy 0, policy_version 965888 (0.0007) [2023-12-27 12:20:27,985][20347] Updated weights for policy 0, policy_version 965898 (0.0006) [2023-12-27 12:20:28,049][20347] Updated weights for policy 0, policy_version 965908 (0.0006) [2023-12-27 12:20:28,066][20381] Updated weights for policy 1, policy_version 965149 (0.0008) [2023-12-27 12:20:28,125][20381] Updated weights for policy 1, policy_version 965159 (0.0010) [2023-12-27 12:20:28,185][20381] Updated weights for policy 1, policy_version 965169 (0.0010) [2023-12-27 12:20:28,718][20347] Updated weights for policy 0, policy_version 965918 (0.0008) [2023-12-27 12:20:28,781][20347] Updated weights for policy 0, policy_version 965928 (0.0007) [2023-12-27 12:20:28,842][20347] Updated weights for policy 0, policy_version 965938 (0.0008) [2023-12-27 12:20:28,875][20381] Updated weights for policy 1, policy_version 965179 (0.0009) [2023-12-27 12:20:28,930][20381] Updated weights for policy 1, policy_version 965189 (0.0006) [2023-12-27 12:20:28,985][20381] Updated weights for policy 1, policy_version 965199 (0.0007) [2023-12-27 12:20:29,527][20347] Updated weights for policy 0, policy_version 965948 (0.0007) [2023-12-27 12:20:29,579][20347] Updated weights for policy 0, policy_version 965958 (0.0007) [2023-12-27 12:20:29,643][20347] Updated weights for policy 0, policy_version 965968 (0.0007) [2023-12-27 12:20:29,723][20381] Updated weights for policy 1, policy_version 965209 (0.0007) [2023-12-27 12:20:29,772][20381] Updated weights for policy 1, policy_version 965219 (0.0011) [2023-12-27 12:20:29,819][20381] Updated weights for policy 1, policy_version 965229 (0.0010) [2023-12-27 12:20:29,884][20381] Updated weights for policy 1, policy_version 965239 (0.0011) [2023-12-27 12:20:30,359][20347] Updated weights for policy 0, policy_version 965978 (0.0008) [2023-12-27 12:20:30,376][20291] KL-divergence is very high: 112.1677 [2023-12-27 12:20:30,418][20347] Updated weights for policy 0, policy_version 965988 (0.0009) [2023-12-27 12:20:30,448][20291] KL-divergence is very high: 153.1485 [2023-12-27 12:20:30,458][20291] KL-divergence is very high: 125.2375 [2023-12-27 12:20:30,474][20347] Updated weights for policy 0, policy_version 965998 (0.0009) [2023-12-27 12:20:30,521][20347] Updated weights for policy 0, policy_version 966008 (0.0008) [2023-12-27 12:20:30,626][20381] Updated weights for policy 1, policy_version 965249 (0.0011) [2023-12-27 12:20:30,689][20381] Updated weights for policy 1, policy_version 965259 (0.0011) [2023-12-27 12:20:30,741][20381] Updated weights for policy 1, policy_version 965269 (0.0009) [2023-12-27 12:20:31,251][20347] Updated weights for policy 0, policy_version 966018 (0.0010) [2023-12-27 12:20:31,313][20347] Updated weights for policy 0, policy_version 966028 (0.0010) [2023-12-27 12:20:31,365][20347] Updated weights for policy 0, policy_version 966038 (0.0010) [2023-12-27 12:20:31,452][20381] Updated weights for policy 1, policy_version 965279 (0.0006) [2023-12-27 12:20:31,507][20381] Updated weights for policy 1, policy_version 965289 (0.0008) [2023-12-27 12:20:31,556][20381] Updated weights for policy 1, policy_version 965299 (0.0008) [2023-12-27 12:20:32,078][20347] Updated weights for policy 0, policy_version 966048 (0.0010) [2023-12-27 12:20:32,133][20347] Updated weights for policy 0, policy_version 966058 (0.0010) [2023-12-27 12:20:32,182][20347] Updated weights for policy 0, policy_version 966068 (0.0010) [2023-12-27 12:20:32,293][20381] Updated weights for policy 1, policy_version 965309 (0.0008) [2023-12-27 12:20:32,350][20381] Updated weights for policy 1, policy_version 965319 (0.0008) [2023-12-27 12:20:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20188.4). Total num frames: 494501888. Throughput: 0: 10298.0, 1: 10230.5. Samples: 494513636. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:32,363][19170] Avg episode reward: [(0, '475.480'), (1, '1000.000')] [2023-12-27 12:20:32,419][20381] Updated weights for policy 1, policy_version 965329 (0.0008) [2023-12-27 12:20:32,781][20291] KL-divergence is very high: 163.3579 [2023-12-27 12:20:32,814][20347] Updated weights for policy 0, policy_version 966078 (0.0009) [2023-12-27 12:20:32,877][20347] Updated weights for policy 0, policy_version 966088 (0.0008) [2023-12-27 12:20:32,884][20291] KL-divergence is very high: 143.8418 [2023-12-27 12:20:32,936][20291] KL-divergence is very high: 170.9497 [2023-12-27 12:20:32,940][20347] Updated weights for policy 0, policy_version 966098 (0.0008) [2023-12-27 12:20:33,154][20381] Updated weights for policy 1, policy_version 965339 (0.0007) [2023-12-27 12:20:33,215][20381] Updated weights for policy 1, policy_version 965349 (0.0008) [2023-12-27 12:20:33,270][20381] Updated weights for policy 1, policy_version 965359 (0.0007) [2023-12-27 12:20:33,521][20347] Updated weights for policy 0, policy_version 966108 (0.0006) [2023-12-27 12:20:33,578][20347] Updated weights for policy 0, policy_version 966118 (0.0007) [2023-12-27 12:20:33,625][20347] Updated weights for policy 0, policy_version 966128 (0.0007) [2023-12-27 12:20:33,886][20381] Updated weights for policy 1, policy_version 965369 (0.0007) [2023-12-27 12:20:33,940][20381] Updated weights for policy 1, policy_version 965379 (0.0008) [2023-12-27 12:20:33,982][20381] Updated weights for policy 1, policy_version 965389 (0.0007) [2023-12-27 12:20:34,030][20381] Updated weights for policy 1, policy_version 965399 (0.0007) [2023-12-27 12:20:34,333][20347] Updated weights for policy 0, policy_version 966138 (0.0007) [2023-12-27 12:20:34,398][20347] Updated weights for policy 0, policy_version 966148 (0.0007) [2023-12-27 12:20:34,468][20347] Updated weights for policy 0, policy_version 966158 (0.0007) [2023-12-27 12:20:34,536][20347] Updated weights for policy 0, policy_version 966168 (0.0007) [2023-12-27 12:20:34,740][20381] Updated weights for policy 1, policy_version 965409 (0.0008) [2023-12-27 12:20:34,795][20381] Updated weights for policy 1, policy_version 965419 (0.0009) [2023-12-27 12:20:34,843][20381] Updated weights for policy 1, policy_version 965429 (0.0009) [2023-12-27 12:20:35,139][20347] Updated weights for policy 0, policy_version 966178 (0.0005) [2023-12-27 12:20:35,191][20347] Updated weights for policy 0, policy_version 966188 (0.0005) [2023-12-27 12:20:35,240][20347] Updated weights for policy 0, policy_version 966198 (0.0005) [2023-12-27 12:20:35,515][20381] Updated weights for policy 1, policy_version 965439 (0.0007) [2023-12-27 12:20:35,567][20381] Updated weights for policy 1, policy_version 965449 (0.0007) [2023-12-27 12:20:35,625][20381] Updated weights for policy 1, policy_version 965459 (0.0006) [2023-12-27 12:20:35,805][20347] Updated weights for policy 0, policy_version 966208 (0.0007) [2023-12-27 12:20:35,863][20347] Updated weights for policy 0, policy_version 966218 (0.0007) [2023-12-27 12:20:35,928][20347] Updated weights for policy 0, policy_version 966228 (0.0007) [2023-12-27 12:20:36,300][20381] Updated weights for policy 1, policy_version 965469 (0.0006) [2023-12-27 12:20:36,352][20381] Updated weights for policy 1, policy_version 965479 (0.0008) [2023-12-27 12:20:36,412][20381] Updated weights for policy 1, policy_version 965489 (0.0008) [2023-12-27 12:20:36,648][20347] Updated weights for policy 0, policy_version 966238 (0.0007) [2023-12-27 12:20:36,711][20347] Updated weights for policy 0, policy_version 966248 (0.0006) [2023-12-27 12:20:36,773][20347] Updated weights for policy 0, policy_version 966258 (0.0005) [2023-12-27 12:20:37,124][20381] Updated weights for policy 1, policy_version 965499 (0.0008) [2023-12-27 12:20:37,186][20381] Updated weights for policy 1, policy_version 965509 (0.0007) [2023-12-27 12:20:37,241][20381] Updated weights for policy 1, policy_version 965519 (0.0008) [2023-12-27 12:20:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20216.2). Total num frames: 494608384. Throughput: 0: 10329.6, 1: 10227.6. Samples: 494576104. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:37,363][19170] Avg episode reward: [(0, '617.520'), (1, '1000.000')] [2023-12-27 12:20:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000965528_247201792.pth... [2023-12-27 12:20:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000964312_246890496.pth [2023-12-27 12:20:37,387][20347] Updated weights for policy 0, policy_version 966268 (0.0005) [2023-12-27 12:20:37,437][20347] Updated weights for policy 0, policy_version 966278 (0.0006) [2023-12-27 12:20:37,456][20291] KL-divergence is very high: 103.4382 [2023-12-27 12:20:37,484][20347] Updated weights for policy 0, policy_version 966288 (0.0007) [2023-12-27 12:20:37,523][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000966296_247414784.pth... [2023-12-27 12:20:37,526][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000965080_247103488.pth [2023-12-27 12:20:37,935][20381] Updated weights for policy 1, policy_version 965529 (0.0007) [2023-12-27 12:20:37,990][20381] Updated weights for policy 1, policy_version 965539 (0.0008) [2023-12-27 12:20:38,047][20381] Updated weights for policy 1, policy_version 965549 (0.0007) [2023-12-27 12:20:38,102][20381] Updated weights for policy 1, policy_version 965559 (0.0007) [2023-12-27 12:20:38,152][20347] Updated weights for policy 0, policy_version 966298 (0.0008) [2023-12-27 12:20:38,211][20347] Updated weights for policy 0, policy_version 966308 (0.0008) [2023-12-27 12:20:38,263][20347] Updated weights for policy 0, policy_version 966318 (0.0006) [2023-12-27 12:20:38,317][20347] Updated weights for policy 0, policy_version 966328 (0.0010) [2023-12-27 12:20:38,825][20381] Updated weights for policy 1, policy_version 965569 (0.0008) [2023-12-27 12:20:38,884][20381] Updated weights for policy 1, policy_version 965579 (0.0008) [2023-12-27 12:20:38,942][20381] Updated weights for policy 1, policy_version 965589 (0.0007) [2023-12-27 12:20:38,996][20347] Updated weights for policy 0, policy_version 966338 (0.0009) [2023-12-27 12:20:39,053][20347] Updated weights for policy 0, policy_version 966348 (0.0006) [2023-12-27 12:20:39,121][20347] Updated weights for policy 0, policy_version 966358 (0.0005) [2023-12-27 12:20:39,693][20381] Updated weights for policy 1, policy_version 965599 (0.0009) [2023-12-27 12:20:39,754][20381] Updated weights for policy 1, policy_version 965609 (0.0010) [2023-12-27 12:20:39,818][20381] Updated weights for policy 1, policy_version 965619 (0.0011) [2023-12-27 12:20:39,869][20347] Updated weights for policy 0, policy_version 966368 (0.0009) [2023-12-27 12:20:39,931][20347] Updated weights for policy 0, policy_version 966378 (0.0007) [2023-12-27 12:20:39,992][20347] Updated weights for policy 0, policy_version 966388 (0.0007) [2023-12-27 12:20:40,523][20381] Updated weights for policy 1, policy_version 965629 (0.0009) [2023-12-27 12:20:40,572][20381] Updated weights for policy 1, policy_version 965639 (0.0007) [2023-12-27 12:20:40,624][20381] Updated weights for policy 1, policy_version 965649 (0.0008) [2023-12-27 12:20:40,657][20347] Updated weights for policy 0, policy_version 966398 (0.0007) [2023-12-27 12:20:40,709][20347] Updated weights for policy 0, policy_version 966408 (0.0007) [2023-12-27 12:20:40,773][20347] Updated weights for policy 0, policy_version 966418 (0.0007) [2023-12-27 12:20:41,354][20381] Updated weights for policy 1, policy_version 965659 (0.0008) [2023-12-27 12:20:41,423][20381] Updated weights for policy 1, policy_version 965669 (0.0009) [2023-12-27 12:20:41,493][20381] Updated weights for policy 1, policy_version 965679 (0.0007) [2023-12-27 12:20:41,507][20347] Updated weights for policy 0, policy_version 966428 (0.0008) [2023-12-27 12:20:41,567][20347] Updated weights for policy 0, policy_version 966438 (0.0008) [2023-12-27 12:20:41,629][20347] Updated weights for policy 0, policy_version 966448 (0.0008) [2023-12-27 12:20:42,198][20381] Updated weights for policy 1, policy_version 965689 (0.0009) [2023-12-27 12:20:42,259][20381] Updated weights for policy 1, policy_version 965699 (0.0008) [2023-12-27 12:20:42,317][20347] Updated weights for policy 0, policy_version 966458 (0.0008) [2023-12-27 12:20:42,326][20381] Updated weights for policy 1, policy_version 965709 (0.0007) [2023-12-27 12:20:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20188.4). Total num frames: 494698496. Throughput: 0: 10337.1, 1: 10187.7. Samples: 494697324. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:42,363][19170] Avg episode reward: [(0, '223.290'), (1, '1000.000')] [2023-12-27 12:20:42,385][20381] Updated weights for policy 1, policy_version 965719 (0.0008) [2023-12-27 12:20:42,390][20347] Updated weights for policy 0, policy_version 966468 (0.0008) [2023-12-27 12:20:42,447][20347] Updated weights for policy 0, policy_version 966478 (0.0008) [2023-12-27 12:20:42,508][20347] Updated weights for policy 0, policy_version 966488 (0.0008) [2023-12-27 12:20:43,135][20381] Updated weights for policy 1, policy_version 965729 (0.0008) [2023-12-27 12:20:43,193][20381] Updated weights for policy 1, policy_version 965739 (0.0006) [2023-12-27 12:20:43,213][20347] Updated weights for policy 0, policy_version 966498 (0.0005) [2023-12-27 12:20:43,258][20381] Updated weights for policy 1, policy_version 965749 (0.0005) [2023-12-27 12:20:43,262][20347] Updated weights for policy 0, policy_version 966508 (0.0005) [2023-12-27 12:20:43,321][20347] Updated weights for policy 0, policy_version 966518 (0.0008) [2023-12-27 12:20:43,888][20381] Updated weights for policy 1, policy_version 965759 (0.0006) [2023-12-27 12:20:43,929][20347] Updated weights for policy 0, policy_version 966528 (0.0008) [2023-12-27 12:20:43,944][20381] Updated weights for policy 1, policy_version 965769 (0.0005) [2023-12-27 12:20:43,984][20347] Updated weights for policy 0, policy_version 966538 (0.0007) [2023-12-27 12:20:43,990][20381] Updated weights for policy 1, policy_version 965779 (0.0005) [2023-12-27 12:20:44,035][20347] Updated weights for policy 0, policy_version 966548 (0.0008) [2023-12-27 12:20:44,546][20381] Updated weights for policy 1, policy_version 965789 (0.0005) [2023-12-27 12:20:44,594][20381] Updated weights for policy 1, policy_version 965799 (0.0005) [2023-12-27 12:20:44,651][20381] Updated weights for policy 1, policy_version 965809 (0.0006) [2023-12-27 12:20:44,772][20347] Updated weights for policy 0, policy_version 966559 (0.0008) [2023-12-27 12:20:44,836][20347] Updated weights for policy 0, policy_version 966569 (0.0009) [2023-12-27 12:20:44,901][20347] Updated weights for policy 0, policy_version 966579 (0.0008) [2023-12-27 12:20:45,342][20381] Updated weights for policy 1, policy_version 965819 (0.0008) [2023-12-27 12:20:45,409][20381] Updated weights for policy 1, policy_version 965829 (0.0005) [2023-12-27 12:20:45,476][20381] Updated weights for policy 1, policy_version 965839 (0.0005) [2023-12-27 12:20:45,596][20347] Updated weights for policy 0, policy_version 966589 (0.0007) [2023-12-27 12:20:45,654][20347] Updated weights for policy 0, policy_version 966599 (0.0005) [2023-12-27 12:20:45,707][20347] Updated weights for policy 0, policy_version 966609 (0.0005) [2023-12-27 12:20:46,033][20381] Updated weights for policy 1, policy_version 965849 (0.0006) [2023-12-27 12:20:46,091][20381] Updated weights for policy 1, policy_version 965859 (0.0006) [2023-12-27 12:20:46,155][20381] Updated weights for policy 1, policy_version 965869 (0.0006) [2023-12-27 12:20:46,208][20381] Updated weights for policy 1, policy_version 965879 (0.0005) [2023-12-27 12:20:46,265][20347] Updated weights for policy 0, policy_version 966619 (0.0008) [2023-12-27 12:20:46,322][20347] Updated weights for policy 0, policy_version 966629 (0.0010) [2023-12-27 12:20:46,373][20347] Updated weights for policy 0, policy_version 966639 (0.0010) [2023-12-27 12:20:46,805][20381] Updated weights for policy 1, policy_version 965889 (0.0009) [2023-12-27 12:20:46,857][20381] Updated weights for policy 1, policy_version 965899 (0.0006) [2023-12-27 12:20:46,909][20381] Updated weights for policy 1, policy_version 965909 (0.0007) [2023-12-27 12:20:47,019][20347] Updated weights for policy 0, policy_version 966649 (0.0010) [2023-12-27 12:20:47,075][20347] Updated weights for policy 0, policy_version 966659 (0.0006) [2023-12-27 12:20:47,120][20347] Updated weights for policy 0, policy_version 966669 (0.0005) [2023-12-27 12:20:47,188][20347] Updated weights for policy 0, policy_version 966679 (0.0006) [2023-12-27 12:20:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 494813184. Throughput: 0: 10314.0, 1: 10267.2. Samples: 494823128. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:47,363][19170] Avg episode reward: [(0, '189.080'), (1, '1000.000')] [2023-12-27 12:20:47,545][20381] Updated weights for policy 1, policy_version 965919 (0.0007) [2023-12-27 12:20:47,599][20381] Updated weights for policy 1, policy_version 965929 (0.0008) [2023-12-27 12:20:47,656][20381] Updated weights for policy 1, policy_version 965939 (0.0008) [2023-12-27 12:20:47,846][20347] Updated weights for policy 0, policy_version 966689 (0.0009) [2023-12-27 12:20:47,901][20347] Updated weights for policy 0, policy_version 966699 (0.0010) [2023-12-27 12:20:47,952][20347] Updated weights for policy 0, policy_version 966709 (0.0010) [2023-12-27 12:20:48,387][20381] Updated weights for policy 1, policy_version 965949 (0.0008) [2023-12-27 12:20:48,449][20381] Updated weights for policy 1, policy_version 965959 (0.0007) [2023-12-27 12:20:48,505][20381] Updated weights for policy 1, policy_version 965969 (0.0008) [2023-12-27 12:20:48,621][20347] Updated weights for policy 0, policy_version 966719 (0.0010) [2023-12-27 12:20:48,681][20347] Updated weights for policy 0, policy_version 966729 (0.0010) [2023-12-27 12:20:48,740][20347] Updated weights for policy 0, policy_version 966739 (0.0010) [2023-12-27 12:20:49,250][20381] Updated weights for policy 1, policy_version 965979 (0.0009) [2023-12-27 12:20:49,310][20381] Updated weights for policy 1, policy_version 965989 (0.0008) [2023-12-27 12:20:49,375][20381] Updated weights for policy 1, policy_version 965999 (0.0009) [2023-12-27 12:20:49,417][20347] Updated weights for policy 0, policy_version 966749 (0.0010) [2023-12-27 12:20:49,476][20347] Updated weights for policy 0, policy_version 966759 (0.0010) [2023-12-27 12:20:49,525][20347] Updated weights for policy 0, policy_version 966769 (0.0010) [2023-12-27 12:20:50,027][20381] Updated weights for policy 1, policy_version 966009 (0.0008) [2023-12-27 12:20:50,092][20381] Updated weights for policy 1, policy_version 966019 (0.0007) [2023-12-27 12:20:50,156][20381] Updated weights for policy 1, policy_version 966029 (0.0008) [2023-12-27 12:20:50,216][20381] Updated weights for policy 1, policy_version 966039 (0.0008) [2023-12-27 12:20:50,243][20347] Updated weights for policy 0, policy_version 966779 (0.0010) [2023-12-27 12:20:50,305][20347] Updated weights for policy 0, policy_version 966789 (0.0007) [2023-12-27 12:20:50,366][20347] Updated weights for policy 0, policy_version 966799 (0.0007) [2023-12-27 12:20:50,949][20381] Updated weights for policy 1, policy_version 966049 (0.0008) [2023-12-27 12:20:51,010][20381] Updated weights for policy 1, policy_version 966059 (0.0008) [2023-12-27 12:20:51,040][20347] Updated weights for policy 0, policy_version 966809 (0.0007) [2023-12-27 12:20:51,079][20381] Updated weights for policy 1, policy_version 966069 (0.0007) [2023-12-27 12:20:51,104][20347] Updated weights for policy 0, policy_version 966819 (0.0008) [2023-12-27 12:20:51,172][20347] Updated weights for policy 0, policy_version 966829 (0.0007) [2023-12-27 12:20:51,231][20347] Updated weights for policy 0, policy_version 966839 (0.0007) [2023-12-27 12:20:51,774][20381] Updated weights for policy 1, policy_version 966079 (0.0008) [2023-12-27 12:20:51,831][20381] Updated weights for policy 1, policy_version 966089 (0.0007) [2023-12-27 12:20:51,890][20381] Updated weights for policy 1, policy_version 966099 (0.0007) [2023-12-27 12:20:51,968][20347] Updated weights for policy 0, policy_version 966849 (0.0005) [2023-12-27 12:20:52,026][20347] Updated weights for policy 0, policy_version 966859 (0.0008) [2023-12-27 12:20:52,082][20347] Updated weights for policy 0, policy_version 966869 (0.0008) [2023-12-27 12:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20188.4). Total num frames: 494911488. Throughput: 0: 10287.8, 1: 10213.3. Samples: 494882828. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:52,363][19170] Avg episode reward: [(0, '449.090'), (1, '1000.000')] [2023-12-27 12:20:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000966104_247349248.pth... [2023-12-27 12:20:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000966872_247562240.pth... [2023-12-27 12:20:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000965688_247259136.pth [2023-12-27 12:20:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000964920_247046144.pth [2023-12-27 12:20:52,632][20381] Updated weights for policy 1, policy_version 966109 (0.0008) [2023-12-27 12:20:52,692][20381] Updated weights for policy 1, policy_version 966119 (0.0008) [2023-12-27 12:20:52,748][20381] Updated weights for policy 1, policy_version 966129 (0.0008) [2023-12-27 12:20:52,779][20347] Updated weights for policy 0, policy_version 966879 (0.0007) [2023-12-27 12:20:52,838][20347] Updated weights for policy 0, policy_version 966889 (0.0007) [2023-12-27 12:20:52,889][20347] Updated weights for policy 0, policy_version 966899 (0.0008) [2023-12-27 12:20:53,430][20381] Updated weights for policy 1, policy_version 966139 (0.0006) [2023-12-27 12:20:53,490][20381] Updated weights for policy 1, policy_version 966149 (0.0006) [2023-12-27 12:20:53,552][20381] Updated weights for policy 1, policy_version 966159 (0.0006) [2023-12-27 12:20:53,587][20347] Updated weights for policy 0, policy_version 966909 (0.0007) [2023-12-27 12:20:53,641][20347] Updated weights for policy 0, policy_version 966919 (0.0009) [2023-12-27 12:20:53,692][20347] Updated weights for policy 0, policy_version 966929 (0.0007) [2023-12-27 12:20:54,204][20381] Updated weights for policy 1, policy_version 966169 (0.0006) [2023-12-27 12:20:54,251][20381] Updated weights for policy 1, policy_version 966179 (0.0007) [2023-12-27 12:20:54,305][20381] Updated weights for policy 1, policy_version 966189 (0.0007) [2023-12-27 12:20:54,354][20381] Updated weights for policy 1, policy_version 966199 (0.0008) [2023-12-27 12:20:54,376][20347] Updated weights for policy 0, policy_version 966939 (0.0007) [2023-12-27 12:20:54,441][20347] Updated weights for policy 0, policy_version 966949 (0.0010) [2023-12-27 12:20:54,498][20347] Updated weights for policy 0, policy_version 966959 (0.0010) [2023-12-27 12:20:55,084][20347] Updated weights for policy 0, policy_version 966969 (0.0010) [2023-12-27 12:20:55,112][20381] Updated weights for policy 1, policy_version 966209 (0.0006) [2023-12-27 12:20:55,137][20347] Updated weights for policy 0, policy_version 966979 (0.0006) [2023-12-27 12:20:55,166][20381] Updated weights for policy 1, policy_version 966219 (0.0007) [2023-12-27 12:20:55,187][20347] Updated weights for policy 0, policy_version 966989 (0.0005) [2023-12-27 12:20:55,214][20381] Updated weights for policy 1, policy_version 966229 (0.0005) [2023-12-27 12:20:55,244][20347] Updated weights for policy 0, policy_version 966999 (0.0008) [2023-12-27 12:20:55,830][20381] Updated weights for policy 1, policy_version 966239 (0.0006) [2023-12-27 12:20:55,884][20381] Updated weights for policy 1, policy_version 966249 (0.0007) [2023-12-27 12:20:55,932][20381] Updated weights for policy 1, policy_version 966259 (0.0007) [2023-12-27 12:20:55,973][20347] Updated weights for policy 0, policy_version 967009 (0.0008) [2023-12-27 12:20:56,032][20347] Updated weights for policy 0, policy_version 967019 (0.0008) [2023-12-27 12:20:56,089][20347] Updated weights for policy 0, policy_version 967029 (0.0008) [2023-12-27 12:20:56,618][20381] Updated weights for policy 1, policy_version 966269 (0.0006) [2023-12-27 12:20:56,675][20381] Updated weights for policy 1, policy_version 966279 (0.0007) [2023-12-27 12:20:56,732][20381] Updated weights for policy 1, policy_version 966289 (0.0006) [2023-12-27 12:20:56,738][20347] Updated weights for policy 0, policy_version 967039 (0.0010) [2023-12-27 12:20:56,789][20347] Updated weights for policy 0, policy_version 967049 (0.0010) [2023-12-27 12:20:56,839][20347] Updated weights for policy 0, policy_version 967059 (0.0010) [2023-12-27 12:20:57,321][20381] Updated weights for policy 1, policy_version 966299 (0.0006) [2023-12-27 12:20:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20188.4). Total num frames: 495009792. Throughput: 0: 10231.7, 1: 10155.9. Samples: 495003396. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:20:57,363][20381] Updated weights for policy 1, policy_version 966309 (0.0008) [2023-12-27 12:20:57,363][19170] Avg episode reward: [(0, '585.720'), (1, '1000.000')] [2023-12-27 12:20:57,408][20381] Updated weights for policy 1, policy_version 966319 (0.0008) [2023-12-27 12:20:57,522][20347] Updated weights for policy 0, policy_version 967069 (0.0010) [2023-12-27 12:20:57,577][20347] Updated weights for policy 0, policy_version 967079 (0.0010) [2023-12-27 12:20:57,628][20347] Updated weights for policy 0, policy_version 967089 (0.0010) [2023-12-27 12:20:58,168][20381] Updated weights for policy 1, policy_version 966329 (0.0008) [2023-12-27 12:20:58,236][20381] Updated weights for policy 1, policy_version 966339 (0.0009) [2023-12-27 12:20:58,301][20381] Updated weights for policy 1, policy_version 966349 (0.0008) [2023-12-27 12:20:58,355][20347] Updated weights for policy 0, policy_version 967099 (0.0010) [2023-12-27 12:20:58,373][20381] Updated weights for policy 1, policy_version 966359 (0.0008) [2023-12-27 12:20:58,423][20347] Updated weights for policy 0, policy_version 967109 (0.0007) [2023-12-27 12:20:58,488][20347] Updated weights for policy 0, policy_version 967119 (0.0008) [2023-12-27 12:20:59,103][20381] Updated weights for policy 1, policy_version 966369 (0.0006) [2023-12-27 12:20:59,167][20381] Updated weights for policy 1, policy_version 966379 (0.0007) [2023-12-27 12:20:59,230][20381] Updated weights for policy 1, policy_version 966389 (0.0007) [2023-12-27 12:20:59,294][20347] Updated weights for policy 0, policy_version 967129 (0.0008) [2023-12-27 12:20:59,353][20347] Updated weights for policy 0, policy_version 967139 (0.0008) [2023-12-27 12:20:59,417][20347] Updated weights for policy 0, policy_version 967149 (0.0007) [2023-12-27 12:20:59,480][20347] Updated weights for policy 0, policy_version 967159 (0.0008) [2023-12-27 12:20:59,830][20381] Updated weights for policy 1, policy_version 966399 (0.0008) [2023-12-27 12:20:59,887][20381] Updated weights for policy 1, policy_version 966409 (0.0007) [2023-12-27 12:20:59,953][20381] Updated weights for policy 1, policy_version 966419 (0.0006) [2023-12-27 12:21:00,207][20347] Updated weights for policy 0, policy_version 967169 (0.0010) [2023-12-27 12:21:00,261][20347] Updated weights for policy 0, policy_version 967179 (0.0010) [2023-12-27 12:21:00,320][20347] Updated weights for policy 0, policy_version 967189 (0.0007) [2023-12-27 12:21:00,576][20381] Updated weights for policy 1, policy_version 966429 (0.0007) [2023-12-27 12:21:00,630][20381] Updated weights for policy 1, policy_version 966439 (0.0007) [2023-12-27 12:21:00,679][20381] Updated weights for policy 1, policy_version 966449 (0.0005) [2023-12-27 12:21:00,983][20347] Updated weights for policy 0, policy_version 967199 (0.0009) [2023-12-27 12:21:01,062][20347] Updated weights for policy 0, policy_version 967209 (0.0010) [2023-12-27 12:21:01,120][20347] Updated weights for policy 0, policy_version 967219 (0.0009) [2023-12-27 12:21:01,332][20381] Updated weights for policy 1, policy_version 966459 (0.0005) [2023-12-27 12:21:01,404][20381] Updated weights for policy 1, policy_version 966469 (0.0009) [2023-12-27 12:21:01,471][20381] Updated weights for policy 1, policy_version 966479 (0.0007) [2023-12-27 12:21:01,861][20347] Updated weights for policy 0, policy_version 967229 (0.0007) [2023-12-27 12:21:01,921][20347] Updated weights for policy 0, policy_version 967239 (0.0006) [2023-12-27 12:21:01,986][20347] Updated weights for policy 0, policy_version 967249 (0.0008) [2023-12-27 12:21:02,141][20381] Updated weights for policy 1, policy_version 966489 (0.0007) [2023-12-27 12:21:02,209][20381] Updated weights for policy 1, policy_version 966499 (0.0007) [2023-12-27 12:21:02,260][20381] Updated weights for policy 1, policy_version 966509 (0.0008) [2023-12-27 12:21:02,316][20381] Updated weights for policy 1, policy_version 966519 (0.0007) [2023-12-27 12:21:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 495116288. Throughput: 0: 10149.2, 1: 10138.1. Samples: 495123480. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:21:02,363][19170] Avg episode reward: [(0, '622.700'), (1, '1000.000')] [2023-12-27 12:21:02,582][20347] Updated weights for policy 0, policy_version 967259 (0.0008) [2023-12-27 12:21:02,631][20347] Updated weights for policy 0, policy_version 967269 (0.0007) [2023-12-27 12:21:02,682][20347] Updated weights for policy 0, policy_version 967279 (0.0008) [2023-12-27 12:21:03,068][20381] Updated weights for policy 1, policy_version 966529 (0.0006) [2023-12-27 12:21:03,124][20381] Updated weights for policy 1, policy_version 966539 (0.0005) [2023-12-27 12:21:03,187][20381] Updated weights for policy 1, policy_version 966549 (0.0005) [2023-12-27 12:21:03,269][20347] Updated weights for policy 0, policy_version 967289 (0.0007) [2023-12-27 12:21:03,313][20347] Updated weights for policy 0, policy_version 967299 (0.0005) [2023-12-27 12:21:03,359][20347] Updated weights for policy 0, policy_version 967309 (0.0005) [2023-12-27 12:21:03,407][20347] Updated weights for policy 0, policy_version 967319 (0.0009) [2023-12-27 12:21:03,825][20381] Updated weights for policy 1, policy_version 966559 (0.0007) [2023-12-27 12:21:03,896][20381] Updated weights for policy 1, policy_version 966569 (0.0009) [2023-12-27 12:21:03,958][20381] Updated weights for policy 1, policy_version 966579 (0.0007) [2023-12-27 12:21:04,051][20347] Updated weights for policy 0, policy_version 967329 (0.0007) [2023-12-27 12:21:04,115][20347] Updated weights for policy 0, policy_version 967339 (0.0008) [2023-12-27 12:21:04,176][20347] Updated weights for policy 0, policy_version 967349 (0.0006) [2023-12-27 12:21:04,645][20381] Updated weights for policy 1, policy_version 966589 (0.0007) [2023-12-27 12:21:04,698][20381] Updated weights for policy 1, policy_version 966599 (0.0005) [2023-12-27 12:21:04,755][20381] Updated weights for policy 1, policy_version 966609 (0.0006) [2023-12-27 12:21:04,832][20347] Updated weights for policy 0, policy_version 967359 (0.0007) [2023-12-27 12:21:04,891][20347] Updated weights for policy 0, policy_version 967369 (0.0007) [2023-12-27 12:21:04,949][20347] Updated weights for policy 0, policy_version 967379 (0.0007) [2023-12-27 12:21:05,420][20381] Updated weights for policy 1, policy_version 966619 (0.0008) [2023-12-27 12:21:05,479][20381] Updated weights for policy 1, policy_version 966629 (0.0007) [2023-12-27 12:21:05,527][20381] Updated weights for policy 1, policy_version 966639 (0.0005) [2023-12-27 12:21:05,619][20347] Updated weights for policy 0, policy_version 967389 (0.0007) [2023-12-27 12:21:05,670][20347] Updated weights for policy 0, policy_version 967399 (0.0006) [2023-12-27 12:21:05,719][20347] Updated weights for policy 0, policy_version 967409 (0.0006) [2023-12-27 12:21:06,191][20381] Updated weights for policy 1, policy_version 966649 (0.0006) [2023-12-27 12:21:06,248][20381] Updated weights for policy 1, policy_version 966659 (0.0011) [2023-12-27 12:21:06,307][20381] Updated weights for policy 1, policy_version 966669 (0.0011) [2023-12-27 12:21:06,363][20381] Updated weights for policy 1, policy_version 966679 (0.0008) [2023-12-27 12:21:06,393][20347] Updated weights for policy 0, policy_version 967419 (0.0008) [2023-12-27 12:21:06,449][20347] Updated weights for policy 0, policy_version 967429 (0.0008) [2023-12-27 12:21:06,512][20347] Updated weights for policy 0, policy_version 967439 (0.0007) [2023-12-27 12:21:07,027][20381] Updated weights for policy 1, policy_version 966689 (0.0006) [2023-12-27 12:21:07,079][20381] Updated weights for policy 1, policy_version 966699 (0.0005) [2023-12-27 12:21:07,141][20381] Updated weights for policy 1, policy_version 966709 (0.0005) [2023-12-27 12:21:07,156][20347] Updated weights for policy 0, policy_version 967449 (0.0006) [2023-12-27 12:21:07,215][20347] Updated weights for policy 0, policy_version 967459 (0.0010) [2023-12-27 12:21:07,275][20347] Updated weights for policy 0, policy_version 967469 (0.0010) [2023-12-27 12:21:07,332][20347] Updated weights for policy 0, policy_version 967479 (0.0008) [2023-12-27 12:21:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 495222784. Throughput: 0: 10217.0, 1: 10133.5. Samples: 495186324. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:21:07,363][19170] Avg episode reward: [(0, '644.680'), (1, '1000.000')] [2023-12-27 12:21:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000966712_247504896.pth... [2023-12-27 12:21:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000967480_247717888.pth... [2023-12-27 12:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000965528_247201792.pth [2023-12-27 12:21:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000966296_247414784.pth [2023-12-27 12:21:07,707][20381] Updated weights for policy 1, policy_version 966719 (0.0005) [2023-12-27 12:21:07,756][20381] Updated weights for policy 1, policy_version 966729 (0.0005) [2023-12-27 12:21:07,804][20381] Updated weights for policy 1, policy_version 966739 (0.0005) [2023-12-27 12:21:07,911][20291] KL-divergence is very high: 160.3240 [2023-12-27 12:21:07,955][20347] Updated weights for policy 0, policy_version 967489 (0.0006) [2023-12-27 12:21:08,024][20347] Updated weights for policy 0, policy_version 967499 (0.0005) [2023-12-27 12:21:08,083][20347] Updated weights for policy 0, policy_version 967509 (0.0005) [2023-12-27 12:21:08,344][20381] Updated weights for policy 1, policy_version 966749 (0.0007) [2023-12-27 12:21:08,402][20381] Updated weights for policy 1, policy_version 966759 (0.0006) [2023-12-27 12:21:08,456][20381] Updated weights for policy 1, policy_version 966769 (0.0010) [2023-12-27 12:21:08,699][20347] Updated weights for policy 0, policy_version 967519 (0.0009) [2023-12-27 12:21:08,761][20347] Updated weights for policy 0, policy_version 967529 (0.0010) [2023-12-27 12:21:08,821][20347] Updated weights for policy 0, policy_version 967539 (0.0010) [2023-12-27 12:21:09,115][20381] Updated weights for policy 1, policy_version 966779 (0.0009) [2023-12-27 12:21:09,167][20381] Updated weights for policy 1, policy_version 966789 (0.0010) [2023-12-27 12:21:09,216][20381] Updated weights for policy 1, policy_version 966799 (0.0010) [2023-12-27 12:21:09,515][20347] Updated weights for policy 0, policy_version 967549 (0.0009) [2023-12-27 12:21:09,571][20347] Updated weights for policy 0, policy_version 967559 (0.0006) [2023-12-27 12:21:09,630][20347] Updated weights for policy 0, policy_version 967569 (0.0007) [2023-12-27 12:21:09,914][20381] Updated weights for policy 1, policy_version 966809 (0.0007) [2023-12-27 12:21:09,976][20381] Updated weights for policy 1, policy_version 966819 (0.0006) [2023-12-27 12:21:10,039][20381] Updated weights for policy 1, policy_version 966829 (0.0007) [2023-12-27 12:21:10,099][20381] Updated weights for policy 1, policy_version 966839 (0.0010) [2023-12-27 12:21:10,342][20347] Updated weights for policy 0, policy_version 967579 (0.0008) [2023-12-27 12:21:10,395][20347] Updated weights for policy 0, policy_version 967589 (0.0007) [2023-12-27 12:21:10,458][20347] Updated weights for policy 0, policy_version 967599 (0.0006) [2023-12-27 12:21:10,768][20381] Updated weights for policy 1, policy_version 966849 (0.0010) [2023-12-27 12:21:10,823][20381] Updated weights for policy 1, policy_version 966859 (0.0005) [2023-12-27 12:21:10,890][20381] Updated weights for policy 1, policy_version 966869 (0.0007) [2023-12-27 12:21:11,095][20347] Updated weights for policy 0, policy_version 967609 (0.0007) [2023-12-27 12:21:11,154][20347] Updated weights for policy 0, policy_version 967619 (0.0010) [2023-12-27 12:21:11,216][20347] Updated weights for policy 0, policy_version 967629 (0.0010) [2023-12-27 12:21:11,277][20347] Updated weights for policy 0, policy_version 967639 (0.0010) [2023-12-27 12:21:11,548][20381] Updated weights for policy 1, policy_version 966879 (0.0008) [2023-12-27 12:21:11,610][20381] Updated weights for policy 1, policy_version 966889 (0.0008) [2023-12-27 12:21:11,675][20381] Updated weights for policy 1, policy_version 966899 (0.0007) [2023-12-27 12:21:12,018][20347] Updated weights for policy 0, policy_version 967649 (0.0006) [2023-12-27 12:21:12,077][20347] Updated weights for policy 0, policy_version 967659 (0.0005) [2023-12-27 12:21:12,138][20347] Updated weights for policy 0, policy_version 967669 (0.0008) [2023-12-27 12:21:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 495321088. Throughput: 0: 10236.2, 1: 10196.1. Samples: 495313152. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:21:12,363][19170] Avg episode reward: [(0, '632.750'), (1, '1000.000')] [2023-12-27 12:21:12,407][20381] Updated weights for policy 1, policy_version 966909 (0.0007) [2023-12-27 12:21:12,466][20381] Updated weights for policy 1, policy_version 966919 (0.0008) [2023-12-27 12:21:12,524][20381] Updated weights for policy 1, policy_version 966929 (0.0008) [2023-12-27 12:21:12,811][20347] Updated weights for policy 0, policy_version 967679 (0.0008) [2023-12-27 12:21:12,867][20347] Updated weights for policy 0, policy_version 967689 (0.0008) [2023-12-27 12:21:12,916][20347] Updated weights for policy 0, policy_version 967699 (0.0009) [2023-12-27 12:21:13,228][20381] Updated weights for policy 1, policy_version 966939 (0.0007) [2023-12-27 12:21:13,281][20381] Updated weights for policy 1, policy_version 966949 (0.0008) [2023-12-27 12:21:13,335][20381] Updated weights for policy 1, policy_version 966959 (0.0008) [2023-12-27 12:21:13,617][20347] Updated weights for policy 0, policy_version 967709 (0.0008) [2023-12-27 12:21:13,666][20347] Updated weights for policy 0, policy_version 967719 (0.0008) [2023-12-27 12:21:13,713][20347] Updated weights for policy 0, policy_version 967729 (0.0009) [2023-12-27 12:21:13,929][20381] Updated weights for policy 1, policy_version 966969 (0.0008) [2023-12-27 12:21:13,990][20381] Updated weights for policy 1, policy_version 966979 (0.0007) [2023-12-27 12:21:14,040][20381] Updated weights for policy 1, policy_version 966989 (0.0007) [2023-12-27 12:21:14,089][20381] Updated weights for policy 1, policy_version 966999 (0.0007) [2023-12-27 12:21:14,486][20347] Updated weights for policy 0, policy_version 967739 (0.0008) [2023-12-27 12:21:14,552][20347] Updated weights for policy 0, policy_version 967749 (0.0008) [2023-12-27 12:21:14,610][20347] Updated weights for policy 0, policy_version 967759 (0.0007) [2023-12-27 12:21:14,685][20381] Updated weights for policy 1, policy_version 967009 (0.0006) [2023-12-27 12:21:14,745][20381] Updated weights for policy 1, policy_version 967019 (0.0006) [2023-12-27 12:21:14,812][20381] Updated weights for policy 1, policy_version 967029 (0.0009) [2023-12-27 12:21:15,314][20347] Updated weights for policy 0, policy_version 967769 (0.0007) [2023-12-27 12:21:15,367][20347] Updated weights for policy 0, policy_version 967779 (0.0008) [2023-12-27 12:21:15,424][20347] Updated weights for policy 0, policy_version 967789 (0.0006) [2023-12-27 12:21:15,480][20347] Updated weights for policy 0, policy_version 967799 (0.0007) [2023-12-27 12:21:15,511][20381] Updated weights for policy 1, policy_version 967039 (0.0010) [2023-12-27 12:21:15,572][20381] Updated weights for policy 1, policy_version 967049 (0.0010) [2023-12-27 12:21:15,625][20381] Updated weights for policy 1, policy_version 967059 (0.0005) [2023-12-27 12:21:16,200][20347] Updated weights for policy 0, policy_version 967809 (0.0008) [2023-12-27 12:21:16,230][20381] Updated weights for policy 1, policy_version 967069 (0.0007) [2023-12-27 12:21:16,246][20347] Updated weights for policy 0, policy_version 967819 (0.0008) [2023-12-27 12:21:16,282][20381] Updated weights for policy 1, policy_version 967079 (0.0006) [2023-12-27 12:21:16,293][20347] Updated weights for policy 0, policy_version 967829 (0.0006) [2023-12-27 12:21:16,331][20381] Updated weights for policy 1, policy_version 967089 (0.0006) [2023-12-27 12:21:16,956][20381] Updated weights for policy 1, policy_version 967099 (0.0008) [2023-12-27 12:21:17,009][20381] Updated weights for policy 1, policy_version 967109 (0.0006) [2023-12-27 12:21:17,014][20347] Updated weights for policy 0, policy_version 967839 (0.0007) [2023-12-27 12:21:17,063][20381] Updated weights for policy 1, policy_version 967119 (0.0005) [2023-12-27 12:21:17,064][20347] Updated weights for policy 0, policy_version 967849 (0.0008) [2023-12-27 12:21:17,125][20347] Updated weights for policy 0, policy_version 967859 (0.0007) [2023-12-27 12:21:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 495427584. Throughput: 0: 10209.2, 1: 10260.0. Samples: 495434752. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:21:17,364][19170] Avg episode reward: [(0, '701.950'), (1, '1000.000')] [2023-12-27 12:21:17,625][20381] Updated weights for policy 1, policy_version 967129 (0.0005) [2023-12-27 12:21:17,682][20381] Updated weights for policy 1, policy_version 967139 (0.0006) [2023-12-27 12:21:17,735][20381] Updated weights for policy 1, policy_version 967149 (0.0007) [2023-12-27 12:21:17,783][20381] Updated weights for policy 1, policy_version 967159 (0.0007) [2023-12-27 12:21:17,865][20347] Updated weights for policy 0, policy_version 967869 (0.0008) [2023-12-27 12:21:17,922][20347] Updated weights for policy 0, policy_version 967879 (0.0008) [2023-12-27 12:21:17,985][20347] Updated weights for policy 0, policy_version 967889 (0.0008) [2023-12-27 12:21:18,453][20381] Updated weights for policy 1, policy_version 967169 (0.0007) [2023-12-27 12:21:18,504][20381] Updated weights for policy 1, policy_version 967179 (0.0006) [2023-12-27 12:21:18,552][20381] Updated weights for policy 1, policy_version 967189 (0.0008) [2023-12-27 12:21:18,660][20347] Updated weights for policy 0, policy_version 967899 (0.0008) [2023-12-27 12:21:18,709][20347] Updated weights for policy 0, policy_version 967909 (0.0010) [2023-12-27 12:21:18,769][20347] Updated weights for policy 0, policy_version 967919 (0.0010) [2023-12-27 12:21:19,191][20381] Updated weights for policy 1, policy_version 967199 (0.0007) [2023-12-27 12:21:19,255][20381] Updated weights for policy 1, policy_version 967209 (0.0008) [2023-12-27 12:21:19,319][20381] Updated weights for policy 1, policy_version 967219 (0.0008) [2023-12-27 12:21:19,455][20347] Updated weights for policy 0, policy_version 967929 (0.0010) [2023-12-27 12:21:19,517][20347] Updated weights for policy 0, policy_version 967939 (0.0008) [2023-12-27 12:21:19,576][20347] Updated weights for policy 0, policy_version 967949 (0.0008) [2023-12-27 12:21:19,636][20347] Updated weights for policy 0, policy_version 967959 (0.0008) [2023-12-27 12:21:20,059][20381] Updated weights for policy 1, policy_version 967229 (0.0008) [2023-12-27 12:21:20,117][20381] Updated weights for policy 1, policy_version 967239 (0.0008) [2023-12-27 12:21:20,175][20381] Updated weights for policy 1, policy_version 967249 (0.0007) [2023-12-27 12:21:20,336][20347] Updated weights for policy 0, policy_version 967969 (0.0007) [2023-12-27 12:21:20,397][20347] Updated weights for policy 0, policy_version 967979 (0.0009) [2023-12-27 12:21:20,464][20347] Updated weights for policy 0, policy_version 967989 (0.0007) [2023-12-27 12:21:20,858][20381] Updated weights for policy 1, policy_version 967259 (0.0006) [2023-12-27 12:21:20,912][20381] Updated weights for policy 1, policy_version 967269 (0.0008) [2023-12-27 12:21:20,964][20381] Updated weights for policy 1, policy_version 967279 (0.0007) [2023-12-27 12:21:21,173][20347] Updated weights for policy 0, policy_version 967999 (0.0009) [2023-12-27 12:21:21,225][20347] Updated weights for policy 0, policy_version 968009 (0.0008) [2023-12-27 12:21:21,279][20347] Updated weights for policy 0, policy_version 968019 (0.0008) [2023-12-27 12:21:21,777][20381] Updated weights for policy 1, policy_version 967289 (0.0008) [2023-12-27 12:21:21,828][20381] Updated weights for policy 1, policy_version 967299 (0.0009) [2023-12-27 12:21:21,880][20381] Updated weights for policy 1, policy_version 967309 (0.0007) [2023-12-27 12:21:21,935][20381] Updated weights for policy 1, policy_version 967319 (0.0007) [2023-12-27 12:21:22,090][20347] Updated weights for policy 0, policy_version 968029 (0.0007) [2023-12-27 12:21:22,153][20347] Updated weights for policy 0, policy_version 968039 (0.0006) [2023-12-27 12:21:22,215][20347] Updated weights for policy 0, policy_version 968049 (0.0006) [2023-12-27 12:21:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 495525888. Throughput: 0: 10153.5, 1: 10273.2. Samples: 495495308. Policy #0 lag: (min: 17.0, avg: 40.8, max: 49.0) [2023-12-27 12:21:22,363][19170] Avg episode reward: [(0, '737.910'), (1, '1000.000')] [2023-12-27 12:21:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000967320_247660544.pth... [2023-12-27 12:21:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000968056_247865344.pth... [2023-12-27 12:21:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000966104_247349248.pth [2023-12-27 12:21:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000966872_247562240.pth [2023-12-27 12:21:22,713][20381] Updated weights for policy 1, policy_version 967329 (0.0011) [2023-12-27 12:21:22,775][20381] Updated weights for policy 1, policy_version 967339 (0.0010) [2023-12-27 12:21:22,825][20381] Updated weights for policy 1, policy_version 967349 (0.0010) [2023-12-27 12:21:22,937][20347] Updated weights for policy 0, policy_version 968059 (0.0009) [2023-12-27 12:21:22,983][20347] Updated weights for policy 0, policy_version 968069 (0.0010) [2023-12-27 12:21:23,045][20347] Updated weights for policy 0, policy_version 968079 (0.0010) [2023-12-27 12:21:23,511][20381] Updated weights for policy 1, policy_version 967359 (0.0009) [2023-12-27 12:21:23,555][20381] Updated weights for policy 1, policy_version 967369 (0.0010) [2023-12-27 12:21:23,599][20381] Updated weights for policy 1, policy_version 967379 (0.0010) [2023-12-27 12:21:23,737][20347] Updated weights for policy 0, policy_version 968089 (0.0010) [2023-12-27 12:21:23,787][20347] Updated weights for policy 0, policy_version 968099 (0.0010) [2023-12-27 12:21:23,834][20347] Updated weights for policy 0, policy_version 968109 (0.0009) [2023-12-27 12:21:23,883][20347] Updated weights for policy 0, policy_version 968119 (0.0007) [2023-12-27 12:21:24,291][20381] Updated weights for policy 1, policy_version 967389 (0.0010) [2023-12-27 12:21:24,344][20381] Updated weights for policy 1, policy_version 967399 (0.0010) [2023-12-27 12:21:24,398][20381] Updated weights for policy 1, policy_version 967409 (0.0010) [2023-12-27 12:21:24,465][20347] Updated weights for policy 0, policy_version 968129 (0.0010) [2023-12-27 12:21:24,527][20347] Updated weights for policy 0, policy_version 968139 (0.0010) [2023-12-27 12:21:24,579][20347] Updated weights for policy 0, policy_version 968149 (0.0010) [2023-12-27 12:21:25,059][20381] Updated weights for policy 1, policy_version 967419 (0.0009) [2023-12-27 12:21:25,105][20381] Updated weights for policy 1, policy_version 967429 (0.0005) [2023-12-27 12:21:25,151][20381] Updated weights for policy 1, policy_version 967439 (0.0005) [2023-12-27 12:21:25,170][20347] Updated weights for policy 0, policy_version 968159 (0.0007) [2023-12-27 12:21:25,229][20347] Updated weights for policy 0, policy_version 968169 (0.0006) [2023-12-27 12:21:25,280][20347] Updated weights for policy 0, policy_version 968179 (0.0010) [2023-12-27 12:21:25,714][20381] Updated weights for policy 1, policy_version 967449 (0.0006) [2023-12-27 12:21:25,767][20381] Updated weights for policy 1, policy_version 967459 (0.0005) [2023-12-27 12:21:25,824][20381] Updated weights for policy 1, policy_version 967469 (0.0005) [2023-12-27 12:21:25,875][20381] Updated weights for policy 1, policy_version 967479 (0.0006) [2023-12-27 12:21:25,878][20347] Updated weights for policy 0, policy_version 968189 (0.0008) [2023-12-27 12:21:25,926][20347] Updated weights for policy 0, policy_version 968199 (0.0005) [2023-12-27 12:21:25,974][20347] Updated weights for policy 0, policy_version 968209 (0.0005) [2023-12-27 12:21:26,535][20381] Updated weights for policy 1, policy_version 967489 (0.0005) [2023-12-27 12:21:26,573][20347] Updated weights for policy 0, policy_version 968219 (0.0009) [2023-12-27 12:21:26,586][20381] Updated weights for policy 1, policy_version 967499 (0.0005) [2023-12-27 12:21:26,626][20347] Updated weights for policy 0, policy_version 968229 (0.0009) [2023-12-27 12:21:26,633][20381] Updated weights for policy 1, policy_version 967509 (0.0005) [2023-12-27 12:21:26,674][20347] Updated weights for policy 0, policy_version 968239 (0.0010) [2023-12-27 12:21:27,245][20381] Updated weights for policy 1, policy_version 967519 (0.0006) [2023-12-27 12:21:27,303][20381] Updated weights for policy 1, policy_version 967529 (0.0007) [2023-12-27 12:21:27,347][20347] Updated weights for policy 0, policy_version 968249 (0.0010) [2023-12-27 12:21:27,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.3, 300 sec: 20188.4). Total num frames: 495624192. Throughput: 0: 10146.3, 1: 10319.9. Samples: 495618308. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:27,364][20381] Updated weights for policy 1, policy_version 967539 (0.0007) [2023-12-27 12:21:27,364][19170] Avg episode reward: [(0, '795.910'), (1, '1000.000')] [2023-12-27 12:21:27,400][20347] Updated weights for policy 0, policy_version 968259 (0.0008) [2023-12-27 12:21:27,446][20347] Updated weights for policy 0, policy_version 968269 (0.0007) [2023-12-27 12:21:27,498][20347] Updated weights for policy 0, policy_version 968279 (0.0010) [2023-12-27 12:21:28,048][20381] Updated weights for policy 1, policy_version 967549 (0.0006) [2023-12-27 12:21:28,095][20381] Updated weights for policy 1, policy_version 967559 (0.0007) [2023-12-27 12:21:28,151][20347] Updated weights for policy 0, policy_version 968289 (0.0008) [2023-12-27 12:21:28,151][20381] Updated weights for policy 1, policy_version 967569 (0.0009) [2023-12-27 12:21:28,201][20347] Updated weights for policy 0, policy_version 968299 (0.0007) [2023-12-27 12:21:28,253][20347] Updated weights for policy 0, policy_version 968309 (0.0006) [2023-12-27 12:21:28,896][20347] Updated weights for policy 0, policy_version 968319 (0.0008) [2023-12-27 12:21:28,909][20381] Updated weights for policy 1, policy_version 967579 (0.0006) [2023-12-27 12:21:28,954][20347] Updated weights for policy 0, policy_version 968329 (0.0005) [2023-12-27 12:21:28,963][20381] Updated weights for policy 1, policy_version 967589 (0.0006) [2023-12-27 12:21:29,009][20347] Updated weights for policy 0, policy_version 968339 (0.0005) [2023-12-27 12:21:29,011][20381] Updated weights for policy 1, policy_version 967599 (0.0005) [2023-12-27 12:21:29,686][20347] Updated weights for policy 0, policy_version 968349 (0.0007) [2023-12-27 12:21:29,734][20347] Updated weights for policy 0, policy_version 968359 (0.0007) [2023-12-27 12:21:29,745][20381] Updated weights for policy 1, policy_version 967609 (0.0007) [2023-12-27 12:21:29,792][20347] Updated weights for policy 0, policy_version 968369 (0.0007) [2023-12-27 12:21:29,798][20381] Updated weights for policy 1, policy_version 967619 (0.0007) [2023-12-27 12:21:29,861][20381] Updated weights for policy 1, policy_version 967629 (0.0008) [2023-12-27 12:21:29,924][20381] Updated weights for policy 1, policy_version 967639 (0.0008) [2023-12-27 12:21:30,480][20347] Updated weights for policy 0, policy_version 968379 (0.0006) [2023-12-27 12:21:30,538][20347] Updated weights for policy 0, policy_version 968389 (0.0005) [2023-12-27 12:21:30,594][20347] Updated weights for policy 0, policy_version 968399 (0.0006) [2023-12-27 12:21:30,640][20381] Updated weights for policy 1, policy_version 967649 (0.0007) [2023-12-27 12:21:30,694][20381] Updated weights for policy 1, policy_version 967659 (0.0008) [2023-12-27 12:21:30,744][20381] Updated weights for policy 1, policy_version 967669 (0.0009) [2023-12-27 12:21:31,246][20347] Updated weights for policy 0, policy_version 968409 (0.0006) [2023-12-27 12:21:31,310][20347] Updated weights for policy 0, policy_version 968419 (0.0008) [2023-12-27 12:21:31,366][20347] Updated weights for policy 0, policy_version 968429 (0.0008) [2023-12-27 12:21:31,432][20347] Updated weights for policy 0, policy_version 968439 (0.0008) [2023-12-27 12:21:31,470][20381] Updated weights for policy 1, policy_version 967679 (0.0009) [2023-12-27 12:21:31,531][20381] Updated weights for policy 1, policy_version 967689 (0.0008) [2023-12-27 12:21:31,595][20381] Updated weights for policy 1, policy_version 967699 (0.0008) [2023-12-27 12:21:32,121][20347] Updated weights for policy 0, policy_version 968449 (0.0006) [2023-12-27 12:21:32,180][20347] Updated weights for policy 0, policy_version 968459 (0.0005) [2023-12-27 12:21:32,235][20347] Updated weights for policy 0, policy_version 968469 (0.0007) [2023-12-27 12:21:32,355][20381] Updated weights for policy 1, policy_version 967709 (0.0008) [2023-12-27 12:21:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 495730688. Throughput: 0: 10171.9, 1: 10214.8. Samples: 495740528. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:32,363][19170] Avg episode reward: [(0, '824.210'), (1, '1000.000')] [2023-12-27 12:21:32,419][20381] Updated weights for policy 1, policy_version 967719 (0.0008) [2023-12-27 12:21:32,477][20381] Updated weights for policy 1, policy_version 967729 (0.0007) [2023-12-27 12:21:32,875][20347] Updated weights for policy 0, policy_version 968479 (0.0007) [2023-12-27 12:21:32,933][20347] Updated weights for policy 0, policy_version 968489 (0.0008) [2023-12-27 12:21:32,994][20347] Updated weights for policy 0, policy_version 968499 (0.0007) [2023-12-27 12:21:33,087][20381] Updated weights for policy 1, policy_version 967739 (0.0007) [2023-12-27 12:21:33,144][20381] Updated weights for policy 1, policy_version 967749 (0.0007) [2023-12-27 12:21:33,202][20381] Updated weights for policy 1, policy_version 967759 (0.0007) [2023-12-27 12:21:33,633][20347] Updated weights for policy 0, policy_version 968509 (0.0007) [2023-12-27 12:21:33,695][20347] Updated weights for policy 0, policy_version 968519 (0.0008) [2023-12-27 12:21:33,752][20381] Updated weights for policy 1, policy_version 967769 (0.0007) [2023-12-27 12:21:33,753][20347] Updated weights for policy 0, policy_version 968529 (0.0008) [2023-12-27 12:21:33,809][20381] Updated weights for policy 1, policy_version 967779 (0.0005) [2023-12-27 12:21:33,856][20381] Updated weights for policy 1, policy_version 967789 (0.0007) [2023-12-27 12:21:33,903][20381] Updated weights for policy 1, policy_version 967799 (0.0007) [2023-12-27 12:21:34,477][20347] Updated weights for policy 0, policy_version 968539 (0.0007) [2023-12-27 12:21:34,527][20347] Updated weights for policy 0, policy_version 968549 (0.0007) [2023-12-27 12:21:34,537][20381] Updated weights for policy 1, policy_version 967809 (0.0005) [2023-12-27 12:21:34,587][20381] Updated weights for policy 1, policy_version 967819 (0.0006) [2023-12-27 12:21:34,589][20347] Updated weights for policy 0, policy_version 968559 (0.0008) [2023-12-27 12:21:34,643][20381] Updated weights for policy 1, policy_version 967829 (0.0006) [2023-12-27 12:21:35,267][20381] Updated weights for policy 1, policy_version 967839 (0.0007) [2023-12-27 12:21:35,315][20347] Updated weights for policy 0, policy_version 968569 (0.0007) [2023-12-27 12:21:35,329][20381] Updated weights for policy 1, policy_version 967849 (0.0006) [2023-12-27 12:21:35,373][20347] Updated weights for policy 0, policy_version 968579 (0.0008) [2023-12-27 12:21:35,382][20381] Updated weights for policy 1, policy_version 967859 (0.0007) [2023-12-27 12:21:35,427][20347] Updated weights for policy 0, policy_version 968589 (0.0005) [2023-12-27 12:21:35,478][20347] Updated weights for policy 0, policy_version 968599 (0.0005) [2023-12-27 12:21:35,919][20381] Updated weights for policy 1, policy_version 967869 (0.0006) [2023-12-27 12:21:35,979][20381] Updated weights for policy 1, policy_version 967879 (0.0006) [2023-12-27 12:21:36,028][20381] Updated weights for policy 1, policy_version 967889 (0.0007) [2023-12-27 12:21:36,158][20347] Updated weights for policy 0, policy_version 968609 (0.0007) [2023-12-27 12:21:36,219][20347] Updated weights for policy 0, policy_version 968619 (0.0005) [2023-12-27 12:21:36,275][20347] Updated weights for policy 0, policy_version 968629 (0.0006) [2023-12-27 12:21:36,679][20381] Updated weights for policy 1, policy_version 967899 (0.0006) [2023-12-27 12:21:36,738][20381] Updated weights for policy 1, policy_version 967909 (0.0007) [2023-12-27 12:21:36,796][20381] Updated weights for policy 1, policy_version 967919 (0.0008) [2023-12-27 12:21:36,984][20347] Updated weights for policy 0, policy_version 968639 (0.0006) [2023-12-27 12:21:37,039][20347] Updated weights for policy 0, policy_version 968649 (0.0005) [2023-12-27 12:21:37,083][20347] Updated weights for policy 0, policy_version 968659 (0.0005) [2023-12-27 12:21:37,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 495837184. Throughput: 0: 10170.2, 1: 10313.7. Samples: 495804604. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:37,363][19170] Avg episode reward: [(0, '814.920'), (1, '1000.000')] [2023-12-27 12:21:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000967928_247816192.pth... [2023-12-27 12:21:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000968664_248020992.pth... [2023-12-27 12:21:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000967480_247717888.pth [2023-12-27 12:21:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000966712_247504896.pth [2023-12-27 12:21:37,494][20381] Updated weights for policy 1, policy_version 967929 (0.0008) [2023-12-27 12:21:37,552][20381] Updated weights for policy 1, policy_version 967939 (0.0011) [2023-12-27 12:21:37,607][20381] Updated weights for policy 1, policy_version 967949 (0.0010) [2023-12-27 12:21:37,639][20347] Updated weights for policy 0, policy_version 968669 (0.0005) [2023-12-27 12:21:37,659][20381] Updated weights for policy 1, policy_version 967959 (0.0010) [2023-12-27 12:21:37,694][20347] Updated weights for policy 0, policy_version 968679 (0.0006) [2023-12-27 12:21:37,750][20347] Updated weights for policy 0, policy_version 968689 (0.0007) [2023-12-27 12:21:38,284][20381] Updated weights for policy 1, policy_version 967969 (0.0010) [2023-12-27 12:21:38,338][20381] Updated weights for policy 1, policy_version 967979 (0.0009) [2023-12-27 12:21:38,402][20381] Updated weights for policy 1, policy_version 967989 (0.0008) [2023-12-27 12:21:38,485][20347] Updated weights for policy 0, policy_version 968699 (0.0008) [2023-12-27 12:21:38,541][20347] Updated weights for policy 0, policy_version 968709 (0.0007) [2023-12-27 12:21:38,595][20347] Updated weights for policy 0, policy_version 968719 (0.0007) [2023-12-27 12:21:39,067][20381] Updated weights for policy 1, policy_version 967999 (0.0010) [2023-12-27 12:21:39,124][20381] Updated weights for policy 1, policy_version 968009 (0.0011) [2023-12-27 12:21:39,182][20381] Updated weights for policy 1, policy_version 968019 (0.0011) [2023-12-27 12:21:39,303][20347] Updated weights for policy 0, policy_version 968729 (0.0007) [2023-12-27 12:21:39,325][20291] KL-divergence is very high: 108.9798 [2023-12-27 12:21:39,372][20347] Updated weights for policy 0, policy_version 968739 (0.0008) [2023-12-27 12:21:39,379][20291] KL-divergence is very high: 218.4853 [2023-12-27 12:21:39,431][20291] KL-divergence is very high: 258.8487 [2023-12-27 12:21:39,438][20347] Updated weights for policy 0, policy_version 968749 (0.0008) [2023-12-27 12:21:39,479][20291] KL-divergence is very high: 106.3606 [2023-12-27 12:21:39,497][20347] Updated weights for policy 0, policy_version 968759 (0.0009) [2023-12-27 12:21:39,932][20381] Updated weights for policy 1, policy_version 968029 (0.0008) [2023-12-27 12:21:39,997][20381] Updated weights for policy 1, policy_version 968039 (0.0008) [2023-12-27 12:21:40,052][20381] Updated weights for policy 1, policy_version 968049 (0.0006) [2023-12-27 12:21:40,257][20347] Updated weights for policy 0, policy_version 968769 (0.0008) [2023-12-27 12:21:40,320][20347] Updated weights for policy 0, policy_version 968779 (0.0007) [2023-12-27 12:21:40,384][20347] Updated weights for policy 0, policy_version 968789 (0.0008) [2023-12-27 12:21:40,756][20381] Updated weights for policy 1, policy_version 968059 (0.0006) [2023-12-27 12:21:40,804][20381] Updated weights for policy 1, policy_version 968069 (0.0007) [2023-12-27 12:21:40,865][20381] Updated weights for policy 1, policy_version 968079 (0.0008) [2023-12-27 12:21:40,993][20347] Updated weights for policy 0, policy_version 968799 (0.0008) [2023-12-27 12:21:41,055][20347] Updated weights for policy 0, policy_version 968809 (0.0009) [2023-12-27 12:21:41,114][20347] Updated weights for policy 0, policy_version 968819 (0.0007) [2023-12-27 12:21:41,619][20381] Updated weights for policy 1, policy_version 968089 (0.0007) [2023-12-27 12:21:41,687][20381] Updated weights for policy 1, policy_version 968099 (0.0008) [2023-12-27 12:21:41,758][20381] Updated weights for policy 1, policy_version 968109 (0.0008) [2023-12-27 12:21:41,822][20381] Updated weights for policy 1, policy_version 968119 (0.0008) [2023-12-27 12:21:41,863][20347] Updated weights for policy 0, policy_version 968829 (0.0007) [2023-12-27 12:21:41,926][20347] Updated weights for policy 0, policy_version 968839 (0.0008) [2023-12-27 12:21:41,986][20347] Updated weights for policy 0, policy_version 968849 (0.0008) [2023-12-27 12:21:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20188.4). Total num frames: 495935488. Throughput: 0: 10175.3, 1: 10358.0. Samples: 495927396. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:42,363][19170] Avg episode reward: [(0, '805.420'), (1, '1000.000')] [2023-12-27 12:21:42,521][20381] Updated weights for policy 1, policy_version 968129 (0.0008) [2023-12-27 12:21:42,577][20381] Updated weights for policy 1, policy_version 968139 (0.0008) [2023-12-27 12:21:42,638][20381] Updated weights for policy 1, policy_version 968149 (0.0006) [2023-12-27 12:21:42,755][20347] Updated weights for policy 0, policy_version 968859 (0.0007) [2023-12-27 12:21:42,819][20347] Updated weights for policy 0, policy_version 968869 (0.0006) [2023-12-27 12:21:42,878][20347] Updated weights for policy 0, policy_version 968879 (0.0006) [2023-12-27 12:21:43,312][20381] Updated weights for policy 1, policy_version 968159 (0.0010) [2023-12-27 12:21:43,367][20381] Updated weights for policy 1, policy_version 968169 (0.0011) [2023-12-27 12:21:43,427][20381] Updated weights for policy 1, policy_version 968179 (0.0009) [2023-12-27 12:21:43,554][20347] Updated weights for policy 0, policy_version 968889 (0.0006) [2023-12-27 12:21:43,608][20347] Updated weights for policy 0, policy_version 968899 (0.0007) [2023-12-27 12:21:43,666][20347] Updated weights for policy 0, policy_version 968909 (0.0006) [2023-12-27 12:21:43,723][20347] Updated weights for policy 0, policy_version 968919 (0.0008) [2023-12-27 12:21:44,118][20381] Updated weights for policy 1, policy_version 968189 (0.0009) [2023-12-27 12:21:44,163][20381] Updated weights for policy 1, policy_version 968199 (0.0008) [2023-12-27 12:21:44,219][20381] Updated weights for policy 1, policy_version 968209 (0.0009) [2023-12-27 12:21:44,382][20347] Updated weights for policy 0, policy_version 968929 (0.0007) [2023-12-27 12:21:44,440][20347] Updated weights for policy 0, policy_version 968939 (0.0007) [2023-12-27 12:21:44,492][20347] Updated weights for policy 0, policy_version 968949 (0.0005) [2023-12-27 12:21:44,931][20381] Updated weights for policy 1, policy_version 968219 (0.0011) [2023-12-27 12:21:45,001][20381] Updated weights for policy 1, policy_version 968229 (0.0011) [2023-12-27 12:21:45,062][20381] Updated weights for policy 1, policy_version 968239 (0.0011) [2023-12-27 12:21:45,086][20347] Updated weights for policy 0, policy_version 968959 (0.0005) [2023-12-27 12:21:45,150][20347] Updated weights for policy 0, policy_version 968969 (0.0007) [2023-12-27 12:21:45,212][20347] Updated weights for policy 0, policy_version 968979 (0.0007) [2023-12-27 12:21:45,717][20381] Updated weights for policy 1, policy_version 968249 (0.0011) [2023-12-27 12:21:45,776][20381] Updated weights for policy 1, policy_version 968259 (0.0008) [2023-12-27 12:21:45,833][20381] Updated weights for policy 1, policy_version 968269 (0.0007) [2023-12-27 12:21:45,884][20381] Updated weights for policy 1, policy_version 968279 (0.0007) [2023-12-27 12:21:45,905][20347] Updated weights for policy 0, policy_version 968989 (0.0007) [2023-12-27 12:21:45,963][20347] Updated weights for policy 0, policy_version 968999 (0.0007) [2023-12-27 12:21:46,024][20347] Updated weights for policy 0, policy_version 969009 (0.0007) [2023-12-27 12:21:46,582][20381] Updated weights for policy 1, policy_version 968289 (0.0008) [2023-12-27 12:21:46,625][20347] Updated weights for policy 0, policy_version 969019 (0.0006) [2023-12-27 12:21:46,639][20381] Updated weights for policy 1, policy_version 968299 (0.0009) [2023-12-27 12:21:46,679][20347] Updated weights for policy 0, policy_version 969029 (0.0005) [2023-12-27 12:21:46,697][20381] Updated weights for policy 1, policy_version 968309 (0.0007) [2023-12-27 12:21:46,736][20347] Updated weights for policy 0, policy_version 969039 (0.0005) [2023-12-27 12:21:47,324][20347] Updated weights for policy 0, policy_version 969049 (0.0006) [2023-12-27 12:21:47,338][20381] Updated weights for policy 1, policy_version 968319 (0.0006) [2023-12-27 12:21:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20160.7). Total num frames: 496033792. Throughput: 0: 10253.2, 1: 10330.0. Samples: 496049724. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:47,363][19170] Avg episode reward: [(0, '845.490'), (1, '1000.000')] [2023-12-27 12:21:47,371][20347] Updated weights for policy 0, policy_version 969059 (0.0007) [2023-12-27 12:21:47,387][20381] Updated weights for policy 1, policy_version 968329 (0.0005) [2023-12-27 12:21:47,422][20347] Updated weights for policy 0, policy_version 969069 (0.0007) [2023-12-27 12:21:47,438][20381] Updated weights for policy 1, policy_version 968339 (0.0006) [2023-12-27 12:21:47,478][20347] Updated weights for policy 0, policy_version 969079 (0.0006) [2023-12-27 12:21:48,109][20381] Updated weights for policy 1, policy_version 968349 (0.0007) [2023-12-27 12:21:48,164][20381] Updated weights for policy 1, policy_version 968359 (0.0010) [2023-12-27 12:21:48,171][20347] Updated weights for policy 0, policy_version 969089 (0.0007) [2023-12-27 12:21:48,221][20381] Updated weights for policy 1, policy_version 968369 (0.0011) [2023-12-27 12:21:48,223][20347] Updated weights for policy 0, policy_version 969099 (0.0006) [2023-12-27 12:21:48,275][20347] Updated weights for policy 0, policy_version 969109 (0.0006) [2023-12-27 12:21:48,914][20381] Updated weights for policy 1, policy_version 968379 (0.0010) [2023-12-27 12:21:48,966][20381] Updated weights for policy 1, policy_version 968389 (0.0010) [2023-12-27 12:21:48,979][20347] Updated weights for policy 0, policy_version 969119 (0.0006) [2023-12-27 12:21:49,019][20381] Updated weights for policy 1, policy_version 968399 (0.0010) [2023-12-27 12:21:49,027][20347] Updated weights for policy 0, policy_version 969129 (0.0005) [2023-12-27 12:21:49,078][20347] Updated weights for policy 0, policy_version 969139 (0.0005) [2023-12-27 12:21:49,700][20381] Updated weights for policy 1, policy_version 968409 (0.0011) [2023-12-27 12:21:49,758][20381] Updated weights for policy 1, policy_version 968419 (0.0010) [2023-12-27 12:21:49,767][20347] Updated weights for policy 0, policy_version 969149 (0.0005) [2023-12-27 12:21:49,813][20381] Updated weights for policy 1, policy_version 968429 (0.0010) [2023-12-27 12:21:49,822][20347] Updated weights for policy 0, policy_version 969159 (0.0006) [2023-12-27 12:21:49,884][20347] Updated weights for policy 0, policy_version 969169 (0.0006) [2023-12-27 12:21:49,887][20381] Updated weights for policy 1, policy_version 968439 (0.0008) [2023-12-27 12:21:50,514][20347] Updated weights for policy 0, policy_version 969179 (0.0006) [2023-12-27 12:21:50,562][20381] Updated weights for policy 1, policy_version 968449 (0.0010) [2023-12-27 12:21:50,571][20347] Updated weights for policy 0, policy_version 969189 (0.0006) [2023-12-27 12:21:50,623][20381] Updated weights for policy 1, policy_version 968459 (0.0011) [2023-12-27 12:21:50,628][20347] Updated weights for policy 0, policy_version 969199 (0.0006) [2023-12-27 12:21:50,678][20381] Updated weights for policy 1, policy_version 968469 (0.0011) [2023-12-27 12:21:51,339][20347] Updated weights for policy 0, policy_version 969209 (0.0006) [2023-12-27 12:21:51,404][20347] Updated weights for policy 0, policy_version 969219 (0.0008) [2023-12-27 12:21:51,435][20381] Updated weights for policy 1, policy_version 968479 (0.0009) [2023-12-27 12:21:51,463][20347] Updated weights for policy 0, policy_version 969229 (0.0007) [2023-12-27 12:21:51,492][20381] Updated weights for policy 1, policy_version 968489 (0.0008) [2023-12-27 12:21:51,527][20347] Updated weights for policy 0, policy_version 969239 (0.0007) [2023-12-27 12:21:51,542][20381] Updated weights for policy 1, policy_version 968499 (0.0006) [2023-12-27 12:21:52,176][20347] Updated weights for policy 0, policy_version 969249 (0.0008) [2023-12-27 12:21:52,226][20347] Updated weights for policy 0, policy_version 969259 (0.0007) [2023-12-27 12:21:52,243][20381] Updated weights for policy 1, policy_version 968509 (0.0005) [2023-12-27 12:21:52,289][20347] Updated weights for policy 0, policy_version 969269 (0.0006) [2023-12-27 12:21:52,305][20381] Updated weights for policy 1, policy_version 968519 (0.0008) [2023-12-27 12:21:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 496140288. Throughput: 0: 10201.6, 1: 10327.6. Samples: 496110136. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:52,363][19170] Avg episode reward: [(0, '845.290'), (1, '1000.000')] [2023-12-27 12:21:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000969272_248176640.pth... [2023-12-27 12:21:52,370][20381] Updated weights for policy 1, policy_version 968529 (0.0009) [2023-12-27 12:21:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000968056_247865344.pth [2023-12-27 12:21:52,414][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000968536_247971840.pth... [2023-12-27 12:21:52,419][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000967320_247660544.pth [2023-12-27 12:21:53,019][20347] Updated weights for policy 0, policy_version 969279 (0.0010) [2023-12-27 12:21:53,068][20381] Updated weights for policy 1, policy_version 968539 (0.0008) [2023-12-27 12:21:53,074][20347] Updated weights for policy 0, policy_version 969289 (0.0007) [2023-12-27 12:21:53,118][20381] Updated weights for policy 1, policy_version 968549 (0.0006) [2023-12-27 12:21:53,132][20347] Updated weights for policy 0, policy_version 969299 (0.0006) [2023-12-27 12:21:53,172][20381] Updated weights for policy 1, policy_version 968559 (0.0008) [2023-12-27 12:21:53,777][20347] Updated weights for policy 0, policy_version 969309 (0.0006) [2023-12-27 12:21:53,836][20347] Updated weights for policy 0, policy_version 969319 (0.0005) [2023-12-27 12:21:53,890][20347] Updated weights for policy 0, policy_version 969329 (0.0006) [2023-12-27 12:21:53,929][20381] Updated weights for policy 1, policy_version 968569 (0.0007) [2023-12-27 12:21:53,987][20381] Updated weights for policy 1, policy_version 968579 (0.0008) [2023-12-27 12:21:54,046][20381] Updated weights for policy 1, policy_version 968589 (0.0008) [2023-12-27 12:21:54,104][20381] Updated weights for policy 1, policy_version 968599 (0.0007) [2023-12-27 12:21:54,467][20347] Updated weights for policy 0, policy_version 969339 (0.0010) [2023-12-27 12:21:54,522][20347] Updated weights for policy 0, policy_version 969349 (0.0010) [2023-12-27 12:21:54,568][20347] Updated weights for policy 0, policy_version 969359 (0.0010) [2023-12-27 12:21:54,821][20381] Updated weights for policy 1, policy_version 968609 (0.0007) [2023-12-27 12:21:54,867][20381] Updated weights for policy 1, policy_version 968619 (0.0006) [2023-12-27 12:21:54,920][20381] Updated weights for policy 1, policy_version 968629 (0.0005) [2023-12-27 12:21:55,233][20347] Updated weights for policy 0, policy_version 969369 (0.0010) [2023-12-27 12:21:55,296][20347] Updated weights for policy 0, policy_version 969379 (0.0010) [2023-12-27 12:21:55,357][20347] Updated weights for policy 0, policy_version 969389 (0.0010) [2023-12-27 12:21:55,408][20347] Updated weights for policy 0, policy_version 969399 (0.0010) [2023-12-27 12:21:55,555][20381] Updated weights for policy 1, policy_version 968639 (0.0007) [2023-12-27 12:21:55,605][20381] Updated weights for policy 1, policy_version 968649 (0.0008) [2023-12-27 12:21:55,659][20381] Updated weights for policy 1, policy_version 968659 (0.0007) [2023-12-27 12:21:56,052][20347] Updated weights for policy 0, policy_version 969409 (0.0010) [2023-12-27 12:21:56,109][20347] Updated weights for policy 0, policy_version 969419 (0.0010) [2023-12-27 12:21:56,166][20347] Updated weights for policy 0, policy_version 969429 (0.0010) [2023-12-27 12:21:56,395][20381] Updated weights for policy 1, policy_version 968669 (0.0008) [2023-12-27 12:21:56,450][20381] Updated weights for policy 1, policy_version 968679 (0.0007) [2023-12-27 12:21:56,494][20381] Updated weights for policy 1, policy_version 968689 (0.0007) [2023-12-27 12:21:56,839][20347] Updated weights for policy 0, policy_version 969439 (0.0010) [2023-12-27 12:21:56,890][20347] Updated weights for policy 0, policy_version 969449 (0.0010) [2023-12-27 12:21:56,935][20347] Updated weights for policy 0, policy_version 969459 (0.0010) [2023-12-27 12:21:57,159][20381] Updated weights for policy 1, policy_version 968699 (0.0007) [2023-12-27 12:21:57,211][20381] Updated weights for policy 1, policy_version 968709 (0.0008) [2023-12-27 12:21:57,265][20381] Updated weights for policy 1, policy_version 968719 (0.0007) [2023-12-27 12:21:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20216.2). Total num frames: 496246784. Throughput: 0: 10213.5, 1: 10234.1. Samples: 496233296. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:21:57,363][19170] Avg episode reward: [(0, '838.570'), (1, '1000.000')] [2023-12-27 12:21:57,606][20347] Updated weights for policy 0, policy_version 969469 (0.0009) [2023-12-27 12:21:57,664][20347] Updated weights for policy 0, policy_version 969479 (0.0008) [2023-12-27 12:21:57,722][20347] Updated weights for policy 0, policy_version 969489 (0.0007) [2023-12-27 12:21:57,990][20381] Updated weights for policy 1, policy_version 968729 (0.0007) [2023-12-27 12:21:58,053][20381] Updated weights for policy 1, policy_version 968739 (0.0008) [2023-12-27 12:21:58,115][20381] Updated weights for policy 1, policy_version 968749 (0.0008) [2023-12-27 12:21:58,171][20381] Updated weights for policy 1, policy_version 968759 (0.0008) [2023-12-27 12:21:58,319][20347] Updated weights for policy 0, policy_version 969499 (0.0008) [2023-12-27 12:21:58,385][20347] Updated weights for policy 0, policy_version 969509 (0.0008) [2023-12-27 12:21:58,453][20347] Updated weights for policy 0, policy_version 969519 (0.0006) [2023-12-27 12:21:59,012][20381] Updated weights for policy 1, policy_version 968769 (0.0006) [2023-12-27 12:21:59,065][20381] Updated weights for policy 1, policy_version 968779 (0.0007) [2023-12-27 12:21:59,127][20381] Updated weights for policy 1, policy_version 968789 (0.0007) [2023-12-27 12:21:59,248][20347] Updated weights for policy 0, policy_version 969529 (0.0007) [2023-12-27 12:21:59,314][20347] Updated weights for policy 0, policy_version 969539 (0.0009) [2023-12-27 12:21:59,374][20347] Updated weights for policy 0, policy_version 969549 (0.0008) [2023-12-27 12:21:59,432][20347] Updated weights for policy 0, policy_version 969559 (0.0005) [2023-12-27 12:21:59,831][20381] Updated weights for policy 1, policy_version 968799 (0.0010) [2023-12-27 12:21:59,880][20381] Updated weights for policy 1, policy_version 968809 (0.0010) [2023-12-27 12:21:59,934][20381] Updated weights for policy 1, policy_version 968819 (0.0010) [2023-12-27 12:22:00,083][20347] Updated weights for policy 0, policy_version 969569 (0.0007) [2023-12-27 12:22:00,134][20347] Updated weights for policy 0, policy_version 969579 (0.0007) [2023-12-27 12:22:00,185][20347] Updated weights for policy 0, policy_version 969589 (0.0007) [2023-12-27 12:22:00,607][20381] Updated weights for policy 1, policy_version 968829 (0.0007) [2023-12-27 12:22:00,662][20381] Updated weights for policy 1, policy_version 968839 (0.0008) [2023-12-27 12:22:00,710][20381] Updated weights for policy 1, policy_version 968849 (0.0008) [2023-12-27 12:22:00,861][20347] Updated weights for policy 0, policy_version 969599 (0.0006) [2023-12-27 12:22:00,903][20347] Updated weights for policy 0, policy_version 969609 (0.0006) [2023-12-27 12:22:00,946][20347] Updated weights for policy 0, policy_version 969619 (0.0005) [2023-12-27 12:22:01,342][20381] Updated weights for policy 1, policy_version 968859 (0.0008) [2023-12-27 12:22:01,415][20381] Updated weights for policy 1, policy_version 968869 (0.0008) [2023-12-27 12:22:01,473][20381] Updated weights for policy 1, policy_version 968879 (0.0006) [2023-12-27 12:22:01,704][20347] Updated weights for policy 0, policy_version 969629 (0.0006) [2023-12-27 12:22:01,764][20347] Updated weights for policy 0, policy_version 969639 (0.0006) [2023-12-27 12:22:01,812][20347] Updated weights for policy 0, policy_version 969649 (0.0006) [2023-12-27 12:22:02,158][20381] Updated weights for policy 1, policy_version 968889 (0.0007) [2023-12-27 12:22:02,216][20381] Updated weights for policy 1, policy_version 968899 (0.0008) [2023-12-27 12:22:02,274][20381] Updated weights for policy 1, policy_version 968909 (0.0007) [2023-12-27 12:22:02,327][20381] Updated weights for policy 1, policy_version 968919 (0.0007) [2023-12-27 12:22:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20160.7). Total num frames: 496345088. Throughput: 0: 10262.8, 1: 10143.4. Samples: 496353028. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:02,363][19170] Avg episode reward: [(0, '858.150'), (1, '1000.000')] [2023-12-27 12:22:02,518][20347] Updated weights for policy 0, policy_version 969659 (0.0008) [2023-12-27 12:22:02,579][20347] Updated weights for policy 0, policy_version 969669 (0.0008) [2023-12-27 12:22:02,636][20347] Updated weights for policy 0, policy_version 969679 (0.0008) [2023-12-27 12:22:02,966][20381] Updated weights for policy 1, policy_version 968929 (0.0007) [2023-12-27 12:22:03,017][20381] Updated weights for policy 1, policy_version 968939 (0.0007) [2023-12-27 12:22:03,070][20381] Updated weights for policy 1, policy_version 968949 (0.0009) [2023-12-27 12:22:03,271][20347] Updated weights for policy 0, policy_version 969689 (0.0008) [2023-12-27 12:22:03,324][20347] Updated weights for policy 0, policy_version 969699 (0.0005) [2023-12-27 12:22:03,381][20347] Updated weights for policy 0, policy_version 969709 (0.0005) [2023-12-27 12:22:03,436][20347] Updated weights for policy 0, policy_version 969719 (0.0007) [2023-12-27 12:22:03,757][20381] Updated weights for policy 1, policy_version 968959 (0.0007) [2023-12-27 12:22:03,809][20381] Updated weights for policy 1, policy_version 968969 (0.0008) [2023-12-27 12:22:03,865][20381] Updated weights for policy 1, policy_version 968979 (0.0007) [2023-12-27 12:22:04,045][20347] Updated weights for policy 0, policy_version 969729 (0.0008) [2023-12-27 12:22:04,106][20347] Updated weights for policy 0, policy_version 969739 (0.0007) [2023-12-27 12:22:04,163][20347] Updated weights for policy 0, policy_version 969749 (0.0007) [2023-12-27 12:22:04,501][20381] Updated weights for policy 1, policy_version 968989 (0.0007) [2023-12-27 12:22:04,561][20381] Updated weights for policy 1, policy_version 968999 (0.0007) [2023-12-27 12:22:04,621][20381] Updated weights for policy 1, policy_version 969009 (0.0008) [2023-12-27 12:22:04,833][20347] Updated weights for policy 0, policy_version 969759 (0.0008) [2023-12-27 12:22:04,884][20347] Updated weights for policy 0, policy_version 969769 (0.0008) [2023-12-27 12:22:04,941][20347] Updated weights for policy 0, policy_version 969779 (0.0010) [2023-12-27 12:22:05,269][20381] Updated weights for policy 1, policy_version 969019 (0.0008) [2023-12-27 12:22:05,320][20381] Updated weights for policy 1, policy_version 969029 (0.0008) [2023-12-27 12:22:05,380][20381] Updated weights for policy 1, policy_version 969039 (0.0007) [2023-12-27 12:22:05,611][20347] Updated weights for policy 0, policy_version 969789 (0.0010) [2023-12-27 12:22:05,662][20347] Updated weights for policy 0, policy_version 969799 (0.0010) [2023-12-27 12:22:05,710][20347] Updated weights for policy 0, policy_version 969809 (0.0010) [2023-12-27 12:22:06,079][20381] Updated weights for policy 1, policy_version 969049 (0.0007) [2023-12-27 12:22:06,143][20381] Updated weights for policy 1, policy_version 969059 (0.0007) [2023-12-27 12:22:06,203][20381] Updated weights for policy 1, policy_version 969069 (0.0008) [2023-12-27 12:22:06,253][20381] Updated weights for policy 1, policy_version 969079 (0.0008) [2023-12-27 12:22:06,435][20347] Updated weights for policy 0, policy_version 969819 (0.0010) [2023-12-27 12:22:06,494][20347] Updated weights for policy 0, policy_version 969829 (0.0010) [2023-12-27 12:22:06,549][20347] Updated weights for policy 0, policy_version 969839 (0.0011) [2023-12-27 12:22:06,977][20381] Updated weights for policy 1, policy_version 969089 (0.0005) [2023-12-27 12:22:07,025][20381] Updated weights for policy 1, policy_version 969099 (0.0005) [2023-12-27 12:22:07,074][20381] Updated weights for policy 1, policy_version 969109 (0.0005) [2023-12-27 12:22:07,225][20347] Updated weights for policy 0, policy_version 969849 (0.0010) [2023-12-27 12:22:07,290][20347] Updated weights for policy 0, policy_version 969859 (0.0007) [2023-12-27 12:22:07,353][20347] Updated weights for policy 0, policy_version 969869 (0.0009) [2023-12-27 12:22:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20343.4, 300 sec: 20160.6). Total num frames: 496443392. Throughput: 0: 10304.0, 1: 10172.8. Samples: 496416772. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:07,364][19170] Avg episode reward: [(0, '882.420'), (1, '1000.000')] [2023-12-27 12:22:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000969112_248119296.pth... [2023-12-27 12:22:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000967928_247816192.pth [2023-12-27 12:22:07,416][20347] Updated weights for policy 0, policy_version 969879 (0.0005) [2023-12-27 12:22:07,422][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000969880_248332288.pth... [2023-12-27 12:22:07,426][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000968664_248020992.pth [2023-12-27 12:22:07,691][20381] Updated weights for policy 1, policy_version 969119 (0.0008) [2023-12-27 12:22:07,747][20381] Updated weights for policy 1, policy_version 969129 (0.0007) [2023-12-27 12:22:07,805][20381] Updated weights for policy 1, policy_version 969139 (0.0008) [2023-12-27 12:22:07,999][20347] Updated weights for policy 0, policy_version 969889 (0.0010) [2023-12-27 12:22:08,058][20347] Updated weights for policy 0, policy_version 969899 (0.0010) [2023-12-27 12:22:08,111][20347] Updated weights for policy 0, policy_version 969909 (0.0009) [2023-12-27 12:22:08,477][20381] Updated weights for policy 1, policy_version 969149 (0.0006) [2023-12-27 12:22:08,537][20381] Updated weights for policy 1, policy_version 969159 (0.0005) [2023-12-27 12:22:08,591][20381] Updated weights for policy 1, policy_version 969169 (0.0005) [2023-12-27 12:22:08,765][20347] Updated weights for policy 0, policy_version 969919 (0.0007) [2023-12-27 12:22:08,828][20347] Updated weights for policy 0, policy_version 969929 (0.0005) [2023-12-27 12:22:08,891][20347] Updated weights for policy 0, policy_version 969939 (0.0009) [2023-12-27 12:22:09,224][20381] Updated weights for policy 1, policy_version 969179 (0.0010) [2023-12-27 12:22:09,288][20381] Updated weights for policy 1, policy_version 969189 (0.0008) [2023-12-27 12:22:09,347][20381] Updated weights for policy 1, policy_version 969199 (0.0008) [2023-12-27 12:22:09,542][20347] Updated weights for policy 0, policy_version 969949 (0.0008) [2023-12-27 12:22:09,608][20347] Updated weights for policy 0, policy_version 969959 (0.0007) [2023-12-27 12:22:09,673][20347] Updated weights for policy 0, policy_version 969969 (0.0005) [2023-12-27 12:22:10,074][20381] Updated weights for policy 1, policy_version 969209 (0.0008) [2023-12-27 12:22:10,134][20381] Updated weights for policy 1, policy_version 969219 (0.0008) [2023-12-27 12:22:10,191][20381] Updated weights for policy 1, policy_version 969229 (0.0009) [2023-12-27 12:22:10,245][20381] Updated weights for policy 1, policy_version 969239 (0.0008) [2023-12-27 12:22:10,321][20347] Updated weights for policy 0, policy_version 969979 (0.0007) [2023-12-27 12:22:10,381][20347] Updated weights for policy 0, policy_version 969989 (0.0009) [2023-12-27 12:22:10,436][20347] Updated weights for policy 0, policy_version 969999 (0.0007) [2023-12-27 12:22:10,896][20381] Updated weights for policy 1, policy_version 969249 (0.0008) [2023-12-27 12:22:10,951][20381] Updated weights for policy 1, policy_version 969259 (0.0007) [2023-12-27 12:22:11,004][20381] Updated weights for policy 1, policy_version 969269 (0.0008) [2023-12-27 12:22:11,181][20347] Updated weights for policy 0, policy_version 970009 (0.0007) [2023-12-27 12:22:11,243][20347] Updated weights for policy 0, policy_version 970019 (0.0009) [2023-12-27 12:22:11,304][20347] Updated weights for policy 0, policy_version 970029 (0.0007) [2023-12-27 12:22:11,363][20347] Updated weights for policy 0, policy_version 970039 (0.0007) [2023-12-27 12:22:11,729][20381] Updated weights for policy 1, policy_version 969279 (0.0008) [2023-12-27 12:22:11,790][20381] Updated weights for policy 1, policy_version 969289 (0.0008) [2023-12-27 12:22:11,854][20381] Updated weights for policy 1, policy_version 969299 (0.0008) [2023-12-27 12:22:12,130][20347] Updated weights for policy 0, policy_version 970049 (0.0008) [2023-12-27 12:22:12,187][20347] Updated weights for policy 0, policy_version 970059 (0.0008) [2023-12-27 12:22:12,236][20347] Updated weights for policy 0, policy_version 970069 (0.0007) [2023-12-27 12:22:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 496549888. Throughput: 0: 10315.9, 1: 10168.5. Samples: 496540100. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:12,363][19170] Avg episode reward: [(0, '854.420'), (1, '1000.000')] [2023-12-27 12:22:12,538][20381] Updated weights for policy 1, policy_version 969309 (0.0006) [2023-12-27 12:22:12,592][20381] Updated weights for policy 1, policy_version 969319 (0.0006) [2023-12-27 12:22:12,658][20381] Updated weights for policy 1, policy_version 969329 (0.0006) [2023-12-27 12:22:12,966][20347] Updated weights for policy 0, policy_version 970079 (0.0008) [2023-12-27 12:22:13,024][20347] Updated weights for policy 0, policy_version 970089 (0.0007) [2023-12-27 12:22:13,090][20347] Updated weights for policy 0, policy_version 970099 (0.0006) [2023-12-27 12:22:13,278][20381] Updated weights for policy 1, policy_version 969339 (0.0008) [2023-12-27 12:22:13,334][20381] Updated weights for policy 1, policy_version 969349 (0.0010) [2023-12-27 12:22:13,391][20381] Updated weights for policy 1, policy_version 969359 (0.0010) [2023-12-27 12:22:13,746][20347] Updated weights for policy 0, policy_version 970109 (0.0007) [2023-12-27 12:22:13,803][20347] Updated weights for policy 0, policy_version 970119 (0.0008) [2023-12-27 12:22:13,858][20347] Updated weights for policy 0, policy_version 970129 (0.0007) [2023-12-27 12:22:14,048][20381] Updated weights for policy 1, policy_version 969369 (0.0010) [2023-12-27 12:22:14,103][20381] Updated weights for policy 1, policy_version 969379 (0.0010) [2023-12-27 12:22:14,163][20381] Updated weights for policy 1, policy_version 969389 (0.0011) [2023-12-27 12:22:14,223][20381] Updated weights for policy 1, policy_version 969399 (0.0010) [2023-12-27 12:22:14,556][20347] Updated weights for policy 0, policy_version 970139 (0.0010) [2023-12-27 12:22:14,608][20347] Updated weights for policy 0, policy_version 970149 (0.0006) [2023-12-27 12:22:14,672][20347] Updated weights for policy 0, policy_version 970159 (0.0005) [2023-12-27 12:22:14,870][20381] Updated weights for policy 1, policy_version 969409 (0.0008) [2023-12-27 12:22:14,928][20381] Updated weights for policy 1, policy_version 969419 (0.0008) [2023-12-27 12:22:14,992][20381] Updated weights for policy 1, policy_version 969429 (0.0007) [2023-12-27 12:22:15,321][20347] Updated weights for policy 0, policy_version 970169 (0.0008) [2023-12-27 12:22:15,376][20347] Updated weights for policy 0, policy_version 970179 (0.0007) [2023-12-27 12:22:15,432][20347] Updated weights for policy 0, policy_version 970189 (0.0007) [2023-12-27 12:22:15,490][20347] Updated weights for policy 0, policy_version 970199 (0.0008) [2023-12-27 12:22:15,694][20381] Updated weights for policy 1, policy_version 969439 (0.0007) [2023-12-27 12:22:15,742][20381] Updated weights for policy 1, policy_version 969449 (0.0005) [2023-12-27 12:22:15,795][20381] Updated weights for policy 1, policy_version 969459 (0.0007) [2023-12-27 12:22:16,183][20347] Updated weights for policy 0, policy_version 970209 (0.0010) [2023-12-27 12:22:16,233][20347] Updated weights for policy 0, policy_version 970219 (0.0007) [2023-12-27 12:22:16,281][20347] Updated weights for policy 0, policy_version 970229 (0.0005) [2023-12-27 12:22:16,452][20381] Updated weights for policy 1, policy_version 969469 (0.0006) [2023-12-27 12:22:16,502][20381] Updated weights for policy 1, policy_version 969479 (0.0005) [2023-12-27 12:22:16,552][20381] Updated weights for policy 1, policy_version 969489 (0.0005) [2023-12-27 12:22:16,880][20347] Updated weights for policy 0, policy_version 970239 (0.0005) [2023-12-27 12:22:16,931][20347] Updated weights for policy 0, policy_version 970249 (0.0005) [2023-12-27 12:22:16,984][20347] Updated weights for policy 0, policy_version 970259 (0.0005) [2023-12-27 12:22:17,165][20381] Updated weights for policy 1, policy_version 969499 (0.0006) [2023-12-27 12:22:17,217][20381] Updated weights for policy 1, policy_version 969509 (0.0008) [2023-12-27 12:22:17,273][20381] Updated weights for policy 1, policy_version 969519 (0.0007) [2023-12-27 12:22:17,363][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 496656384. Throughput: 0: 10261.6, 1: 10249.4. Samples: 496663524. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:17,364][19170] Avg episode reward: [(0, '853.840'), (1, '1000.000')] [2023-12-27 12:22:17,629][20347] Updated weights for policy 0, policy_version 970269 (0.0008) [2023-12-27 12:22:17,676][20347] Updated weights for policy 0, policy_version 970279 (0.0010) [2023-12-27 12:22:17,732][20347] Updated weights for policy 0, policy_version 970289 (0.0010) [2023-12-27 12:22:17,975][20381] Updated weights for policy 1, policy_version 969529 (0.0008) [2023-12-27 12:22:18,022][20381] Updated weights for policy 1, policy_version 969539 (0.0007) [2023-12-27 12:22:18,081][20381] Updated weights for policy 1, policy_version 969549 (0.0007) [2023-12-27 12:22:18,132][20381] Updated weights for policy 1, policy_version 969559 (0.0007) [2023-12-27 12:22:18,390][20347] Updated weights for policy 0, policy_version 970299 (0.0009) [2023-12-27 12:22:18,445][20347] Updated weights for policy 0, policy_version 970309 (0.0008) [2023-12-27 12:22:18,499][20347] Updated weights for policy 0, policy_version 970319 (0.0008) [2023-12-27 12:22:18,755][20381] Updated weights for policy 1, policy_version 969569 (0.0009) [2023-12-27 12:22:18,811][20381] Updated weights for policy 1, policy_version 969579 (0.0009) [2023-12-27 12:22:18,864][20381] Updated weights for policy 1, policy_version 969589 (0.0008) [2023-12-27 12:22:19,182][20347] Updated weights for policy 0, policy_version 970329 (0.0008) [2023-12-27 12:22:19,240][20347] Updated weights for policy 0, policy_version 970339 (0.0010) [2023-12-27 12:22:19,289][20347] Updated weights for policy 0, policy_version 970349 (0.0010) [2023-12-27 12:22:19,342][20347] Updated weights for policy 0, policy_version 970359 (0.0010) [2023-12-27 12:22:19,567][20381] Updated weights for policy 1, policy_version 969599 (0.0007) [2023-12-27 12:22:19,614][20381] Updated weights for policy 1, policy_version 969609 (0.0011) [2023-12-27 12:22:19,671][20381] Updated weights for policy 1, policy_version 969619 (0.0011) [2023-12-27 12:22:20,104][20347] Updated weights for policy 0, policy_version 970369 (0.0010) [2023-12-27 12:22:20,161][20347] Updated weights for policy 0, policy_version 970379 (0.0011) [2023-12-27 12:22:20,227][20347] Updated weights for policy 0, policy_version 970389 (0.0011) [2023-12-27 12:22:20,373][20381] Updated weights for policy 1, policy_version 969629 (0.0009) [2023-12-27 12:22:20,433][20381] Updated weights for policy 1, policy_version 969639 (0.0009) [2023-12-27 12:22:20,488][20381] Updated weights for policy 1, policy_version 969649 (0.0007) [2023-12-27 12:22:20,926][20347] Updated weights for policy 0, policy_version 970399 (0.0008) [2023-12-27 12:22:20,983][20347] Updated weights for policy 0, policy_version 970409 (0.0011) [2023-12-27 12:22:21,045][20347] Updated weights for policy 0, policy_version 970419 (0.0011) [2023-12-27 12:22:21,180][20381] Updated weights for policy 1, policy_version 969659 (0.0007) [2023-12-27 12:22:21,244][20381] Updated weights for policy 1, policy_version 969669 (0.0008) [2023-12-27 12:22:21,306][20381] Updated weights for policy 1, policy_version 969679 (0.0008) [2023-12-27 12:22:21,775][20347] Updated weights for policy 0, policy_version 970429 (0.0008) [2023-12-27 12:22:21,838][20347] Updated weights for policy 0, policy_version 970439 (0.0006) [2023-12-27 12:22:21,897][20347] Updated weights for policy 0, policy_version 970449 (0.0008) [2023-12-27 12:22:22,057][20381] Updated weights for policy 1, policy_version 969689 (0.0007) [2023-12-27 12:22:22,114][20381] Updated weights for policy 1, policy_version 969699 (0.0006) [2023-12-27 12:22:22,177][20381] Updated weights for policy 1, policy_version 969709 (0.0008) [2023-12-27 12:22:22,247][20381] Updated weights for policy 1, policy_version 969719 (0.0008) [2023-12-27 12:22:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 496754688. Throughput: 0: 10270.8, 1: 10175.9. Samples: 496724704. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:22,363][19170] Avg episode reward: [(0, '796.060'), (1, '1000.000')] [2023-12-27 12:22:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000970456_248479744.pth... [2023-12-27 12:22:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000969720_248274944.pth... [2023-12-27 12:22:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000969272_248176640.pth [2023-12-27 12:22:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000968536_247971840.pth [2023-12-27 12:22:22,580][20347] Updated weights for policy 0, policy_version 970459 (0.0008) [2023-12-27 12:22:22,634][20347] Updated weights for policy 0, policy_version 970469 (0.0007) [2023-12-27 12:22:22,693][20347] Updated weights for policy 0, policy_version 970479 (0.0005) [2023-12-27 12:22:22,955][20381] Updated weights for policy 1, policy_version 969729 (0.0009) [2023-12-27 12:22:23,012][20381] Updated weights for policy 1, policy_version 969739 (0.0008) [2023-12-27 12:22:23,071][20381] Updated weights for policy 1, policy_version 969749 (0.0008) [2023-12-27 12:22:23,304][20347] Updated weights for policy 0, policy_version 970489 (0.0006) [2023-12-27 12:22:23,362][20347] Updated weights for policy 0, policy_version 970499 (0.0007) [2023-12-27 12:22:23,412][20347] Updated weights for policy 0, policy_version 970509 (0.0007) [2023-12-27 12:22:23,464][20347] Updated weights for policy 0, policy_version 970519 (0.0006) [2023-12-27 12:22:23,684][20381] Updated weights for policy 1, policy_version 969759 (0.0008) [2023-12-27 12:22:23,736][20381] Updated weights for policy 1, policy_version 969769 (0.0008) [2023-12-27 12:22:23,783][20381] Updated weights for policy 1, policy_version 969779 (0.0008) [2023-12-27 12:22:24,077][20347] Updated weights for policy 0, policy_version 970529 (0.0007) [2023-12-27 12:22:24,131][20347] Updated weights for policy 0, policy_version 970539 (0.0007) [2023-12-27 12:22:24,187][20347] Updated weights for policy 0, policy_version 970549 (0.0009) [2023-12-27 12:22:24,455][20381] Updated weights for policy 1, policy_version 969789 (0.0008) [2023-12-27 12:22:24,513][20381] Updated weights for policy 1, policy_version 969799 (0.0007) [2023-12-27 12:22:24,572][20381] Updated weights for policy 1, policy_version 969809 (0.0007) [2023-12-27 12:22:24,918][20347] Updated weights for policy 0, policy_version 970559 (0.0006) [2023-12-27 12:22:24,973][20347] Updated weights for policy 0, policy_version 970569 (0.0005) [2023-12-27 12:22:25,024][20347] Updated weights for policy 0, policy_version 970579 (0.0005) [2023-12-27 12:22:25,199][20381] Updated weights for policy 1, policy_version 969819 (0.0007) [2023-12-27 12:22:25,262][20381] Updated weights for policy 1, policy_version 969829 (0.0011) [2023-12-27 12:22:25,311][20381] Updated weights for policy 1, policy_version 969839 (0.0010) [2023-12-27 12:22:25,611][20347] Updated weights for policy 0, policy_version 970589 (0.0007) [2023-12-27 12:22:25,674][20347] Updated weights for policy 0, policy_version 970599 (0.0008) [2023-12-27 12:22:25,735][20347] Updated weights for policy 0, policy_version 970609 (0.0005) [2023-12-27 12:22:25,966][20381] Updated weights for policy 1, policy_version 969849 (0.0010) [2023-12-27 12:22:26,024][20381] Updated weights for policy 1, policy_version 969859 (0.0007) [2023-12-27 12:22:26,080][20381] Updated weights for policy 1, policy_version 969869 (0.0007) [2023-12-27 12:22:26,129][20381] Updated weights for policy 1, policy_version 969879 (0.0007) [2023-12-27 12:22:26,309][20347] Updated weights for policy 0, policy_version 970619 (0.0007) [2023-12-27 12:22:26,355][20347] Updated weights for policy 0, policy_version 970629 (0.0006) [2023-12-27 12:22:26,406][20347] Updated weights for policy 0, policy_version 970639 (0.0005) [2023-12-27 12:22:26,862][20381] Updated weights for policy 1, policy_version 969889 (0.0006) [2023-12-27 12:22:26,912][20381] Updated weights for policy 1, policy_version 969899 (0.0007) [2023-12-27 12:22:26,963][20381] Updated weights for policy 1, policy_version 969909 (0.0007) [2023-12-27 12:22:26,990][20347] Updated weights for policy 0, policy_version 970649 (0.0006) [2023-12-27 12:22:27,047][20347] Updated weights for policy 0, policy_version 970659 (0.0010) [2023-12-27 12:22:27,103][20347] Updated weights for policy 0, policy_version 970669 (0.0010) [2023-12-27 12:22:27,147][20347] Updated weights for policy 0, policy_version 970679 (0.0010) [2023-12-27 12:22:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.7, 300 sec: 20244.0). Total num frames: 496861184. Throughput: 0: 10300.2, 1: 10159.9. Samples: 496848100. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:27,363][19170] Avg episode reward: [(0, '835.380'), (1, '1000.000')] [2023-12-27 12:22:27,637][20381] Updated weights for policy 1, policy_version 969919 (0.0008) [2023-12-27 12:22:27,679][20381] Updated weights for policy 1, policy_version 969929 (0.0007) [2023-12-27 12:22:27,729][20381] Updated weights for policy 1, policy_version 969939 (0.0008) [2023-12-27 12:22:27,770][20347] Updated weights for policy 0, policy_version 970689 (0.0006) [2023-12-27 12:22:27,829][20347] Updated weights for policy 0, policy_version 970699 (0.0008) [2023-12-27 12:22:27,882][20347] Updated weights for policy 0, policy_version 970709 (0.0007) [2023-12-27 12:22:28,491][20381] Updated weights for policy 1, policy_version 969949 (0.0008) [2023-12-27 12:22:28,543][20381] Updated weights for policy 1, policy_version 969959 (0.0008) [2023-12-27 12:22:28,605][20381] Updated weights for policy 1, policy_version 969969 (0.0008) [2023-12-27 12:22:28,652][20347] Updated weights for policy 0, policy_version 970719 (0.0008) [2023-12-27 12:22:28,699][20347] Updated weights for policy 0, policy_version 970729 (0.0008) [2023-12-27 12:22:28,752][20347] Updated weights for policy 0, policy_version 970739 (0.0008) [2023-12-27 12:22:29,341][20381] Updated weights for policy 1, policy_version 969979 (0.0008) [2023-12-27 12:22:29,394][20381] Updated weights for policy 1, policy_version 969989 (0.0008) [2023-12-27 12:22:29,442][20347] Updated weights for policy 0, policy_version 970749 (0.0007) [2023-12-27 12:22:29,443][20381] Updated weights for policy 1, policy_version 969999 (0.0008) [2023-12-27 12:22:29,504][20347] Updated weights for policy 0, policy_version 970759 (0.0006) [2023-12-27 12:22:29,567][20347] Updated weights for policy 0, policy_version 970769 (0.0005) [2023-12-27 12:22:30,166][20381] Updated weights for policy 1, policy_version 970009 (0.0007) [2023-12-27 12:22:30,203][20347] Updated weights for policy 0, policy_version 970779 (0.0006) [2023-12-27 12:22:30,228][20381] Updated weights for policy 1, policy_version 970019 (0.0010) [2023-12-27 12:22:30,238][20292] KL-divergence is very high: 150.9546 [2023-12-27 12:22:30,262][20347] Updated weights for policy 0, policy_version 970789 (0.0006) [2023-12-27 12:22:30,282][20381] Updated weights for policy 1, policy_version 970029 (0.0010) [2023-12-27 12:22:30,315][20347] Updated weights for policy 0, policy_version 970799 (0.0005) [2023-12-27 12:22:30,336][20381] Updated weights for policy 1, policy_version 970039 (0.0009) [2023-12-27 12:22:30,881][20347] Updated weights for policy 0, policy_version 970809 (0.0005) [2023-12-27 12:22:30,941][20347] Updated weights for policy 0, policy_version 970819 (0.0009) [2023-12-27 12:22:30,994][20347] Updated weights for policy 0, policy_version 970829 (0.0010) [2023-12-27 12:22:31,001][20381] Updated weights for policy 1, policy_version 970049 (0.0010) [2023-12-27 12:22:31,042][20292] KL-divergence is very high: 237.2600 [2023-12-27 12:22:31,059][20347] Updated weights for policy 0, policy_version 970839 (0.0009) [2023-12-27 12:22:31,069][20381] Updated weights for policy 1, policy_version 970059 (0.0009) [2023-12-27 12:22:31,096][20292] KL-divergence is very high: 109.5355 [2023-12-27 12:22:31,129][20381] Updated weights for policy 1, policy_version 970069 (0.0005) [2023-12-27 12:22:31,724][20347] Updated weights for policy 0, policy_version 970849 (0.0008) [2023-12-27 12:22:31,784][20347] Updated weights for policy 0, policy_version 970859 (0.0010) [2023-12-27 12:22:31,839][20381] Updated weights for policy 1, policy_version 970079 (0.0010) [2023-12-27 12:22:31,840][20347] Updated weights for policy 0, policy_version 970869 (0.0011) [2023-12-27 12:22:31,897][20381] Updated weights for policy 1, policy_version 970089 (0.0009) [2023-12-27 12:22:31,941][20292] KL-divergence is very high: 143.9846 [2023-12-27 12:22:31,951][20381] Updated weights for policy 1, policy_version 970099 (0.0005) [2023-12-27 12:22:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 496959488. Throughput: 0: 10346.6, 1: 10149.5. Samples: 496972052. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:32,363][19170] Avg episode reward: [(0, '834.340'), (1, '980.540')] [2023-12-27 12:22:32,520][20347] Updated weights for policy 0, policy_version 970879 (0.0009) [2023-12-27 12:22:32,577][20347] Updated weights for policy 0, policy_version 970889 (0.0007) [2023-12-27 12:22:32,611][20381] Updated weights for policy 1, policy_version 970109 (0.0005) [2023-12-27 12:22:32,635][20347] Updated weights for policy 0, policy_version 970899 (0.0005) [2023-12-27 12:22:32,674][20381] Updated weights for policy 1, policy_version 970119 (0.0005) [2023-12-27 12:22:32,736][20381] Updated weights for policy 1, policy_version 970129 (0.0009) [2023-12-27 12:22:33,244][20347] Updated weights for policy 0, policy_version 970909 (0.0006) [2023-12-27 12:22:33,301][20347] Updated weights for policy 0, policy_version 970919 (0.0007) [2023-12-27 12:22:33,357][20347] Updated weights for policy 0, policy_version 970929 (0.0007) [2023-12-27 12:22:33,363][20381] Updated weights for policy 1, policy_version 970139 (0.0011) [2023-12-27 12:22:33,413][20292] KL-divergence is very high: 934.1070 [2023-12-27 12:22:33,413][20381] Updated weights for policy 1, policy_version 970149 (0.0011) [2023-12-27 12:22:33,475][20381] Updated weights for policy 1, policy_version 970159 (0.0011) [2023-12-27 12:22:34,067][20347] Updated weights for policy 0, policy_version 970939 (0.0006) [2023-12-27 12:22:34,086][20381] Updated weights for policy 1, policy_version 970169 (0.0011) [2023-12-27 12:22:34,121][20292] KL-divergence is very high: 497.9828 [2023-12-27 12:22:34,129][20347] Updated weights for policy 0, policy_version 970949 (0.0006) [2023-12-27 12:22:34,145][20381] Updated weights for policy 1, policy_version 970179 (0.0010) [2023-12-27 12:22:34,159][20292] KL-divergence is very high: 1453.7946 [2023-12-27 12:22:34,174][20292] KL-divergence is very high: 1054.7705 [2023-12-27 12:22:34,192][20347] Updated weights for policy 0, policy_version 970959 (0.0006) [2023-12-27 12:22:34,205][20381] Updated weights for policy 1, policy_version 970189 (0.0007) [2023-12-27 12:22:34,205][20292] KL-divergence is very high: 140.6186 [2023-12-27 12:22:34,257][20381] Updated weights for policy 1, policy_version 970199 (0.0006) [2023-12-27 12:22:34,258][20292] KL-divergence is very high: 193.1399 [2023-12-27 12:22:34,816][20292] KL-divergence is very high: 661.7792 [2023-12-27 12:22:34,853][20381] Updated weights for policy 1, policy_version 970209 (0.0006) [2023-12-27 12:22:34,861][20292] KL-divergence is very high: 277.2719 [2023-12-27 12:22:34,873][20292] KL-divergence is very high: 202.3871 [2023-12-27 12:22:34,901][20292] KL-divergence is very high: 286.7361 [2023-12-27 12:22:34,906][20381] Updated weights for policy 1, policy_version 970219 (0.0005) [2023-12-27 12:22:34,914][20347] Updated weights for policy 0, policy_version 970969 (0.0008) [2023-12-27 12:22:34,944][20292] KL-divergence is very high: 254.6842 [2023-12-27 12:22:34,960][20381] Updated weights for policy 1, policy_version 970229 (0.0007) [2023-12-27 12:22:34,967][20347] Updated weights for policy 0, policy_version 970979 (0.0007) [2023-12-27 12:22:35,031][20347] Updated weights for policy 0, policy_version 970989 (0.0007) [2023-12-27 12:22:35,076][20347] Updated weights for policy 0, policy_version 970999 (0.0007) [2023-12-27 12:22:35,557][20381] Updated weights for policy 1, policy_version 970239 (0.0006) [2023-12-27 12:22:35,588][20292] KL-divergence is very high: 109.2467 [2023-12-27 12:22:35,616][20381] Updated weights for policy 1, policy_version 970249 (0.0006) [2023-12-27 12:22:35,680][20381] Updated weights for policy 1, policy_version 970259 (0.0008) [2023-12-27 12:22:35,685][20292] KL-divergence is very high: 128.6443 [2023-12-27 12:22:35,696][20292] KL-divergence is very high: 137.7245 [2023-12-27 12:22:35,808][20347] Updated weights for policy 0, policy_version 971009 (0.0010) [2023-12-27 12:22:35,856][20347] Updated weights for policy 0, policy_version 971019 (0.0010) [2023-12-27 12:22:35,910][20347] Updated weights for policy 0, policy_version 971029 (0.0010) [2023-12-27 12:22:36,365][20292] KL-divergence is very high: 137.9952 [2023-12-27 12:22:36,382][20381] Updated weights for policy 1, policy_version 970269 (0.0007) [2023-12-27 12:22:36,440][20381] Updated weights for policy 1, policy_version 970279 (0.0007) [2023-12-27 12:22:36,457][20292] KL-divergence is very high: 190.8232 [2023-12-27 12:22:36,493][20381] Updated weights for policy 1, policy_version 970289 (0.0007) [2023-12-27 12:22:36,616][20347] Updated weights for policy 0, policy_version 971039 (0.0009) [2023-12-27 12:22:36,679][20347] Updated weights for policy 0, policy_version 971049 (0.0008) [2023-12-27 12:22:36,737][20347] Updated weights for policy 0, policy_version 971059 (0.0008) [2023-12-27 12:22:37,184][20381] Updated weights for policy 1, policy_version 970299 (0.0007) [2023-12-27 12:22:37,231][20381] Updated weights for policy 1, policy_version 970309 (0.0005) [2023-12-27 12:22:37,248][20292] KL-divergence is very high: 169.1918 [2023-12-27 12:22:37,289][20381] Updated weights for policy 1, policy_version 970319 (0.0006) [2023-12-27 12:22:37,293][20292] KL-divergence is very high: 116.4761 [2023-12-27 12:22:37,313][20347] Updated weights for policy 0, policy_version 971069 (0.0007) [2023-12-27 12:22:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 497065984. Throughput: 0: 10355.9, 1: 10198.9. Samples: 497035104. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:37,363][19170] Avg episode reward: [(0, '814.570'), (1, '775.350')] [2023-12-27 12:22:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000970328_248430592.pth... [2023-12-27 12:22:37,369][20347] Updated weights for policy 0, policy_version 971079 (0.0006) [2023-12-27 12:22:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000969112_248119296.pth [2023-12-27 12:22:37,420][20347] Updated weights for policy 0, policy_version 971089 (0.0007) [2023-12-27 12:22:37,456][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000971096_248643584.pth... [2023-12-27 12:22:37,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000969880_248332288.pth [2023-12-27 12:22:37,898][20381] Updated weights for policy 1, policy_version 970329 (0.0010) [2023-12-27 12:22:37,899][20292] KL-divergence is very high: 109.5325 [2023-12-27 12:22:37,936][20292] KL-divergence is very high: 101.8641 [2023-12-27 12:22:37,946][20381] Updated weights for policy 1, policy_version 970339 (0.0005) [2023-12-27 12:22:37,999][20381] Updated weights for policy 1, policy_version 970349 (0.0005) [2023-12-27 12:22:38,016][20347] Updated weights for policy 0, policy_version 971099 (0.0008) [2023-12-27 12:22:38,055][20381] Updated weights for policy 1, policy_version 970359 (0.0008) [2023-12-27 12:22:38,074][20347] Updated weights for policy 0, policy_version 971109 (0.0010) [2023-12-27 12:22:38,122][20347] Updated weights for policy 0, policy_version 971119 (0.0010) [2023-12-27 12:22:38,763][20381] Updated weights for policy 1, policy_version 970369 (0.0011) [2023-12-27 12:22:38,802][20347] Updated weights for policy 0, policy_version 971129 (0.0010) [2023-12-27 12:22:38,811][20292] KL-divergence is very high: 104.3058 [2023-12-27 12:22:38,823][20381] Updated weights for policy 1, policy_version 970379 (0.0011) [2023-12-27 12:22:38,855][20347] Updated weights for policy 0, policy_version 971139 (0.0005) [2023-12-27 12:22:38,882][20381] Updated weights for policy 1, policy_version 970389 (0.0011) [2023-12-27 12:22:38,908][20347] Updated weights for policy 0, policy_version 971149 (0.0007) [2023-12-27 12:22:38,957][20347] Updated weights for policy 0, policy_version 971159 (0.0006) [2023-12-27 12:22:39,584][20381] Updated weights for policy 1, policy_version 970399 (0.0009) [2023-12-27 12:22:39,612][20347] Updated weights for policy 0, policy_version 971169 (0.0006) [2023-12-27 12:22:39,641][20381] Updated weights for policy 1, policy_version 970409 (0.0008) [2023-12-27 12:22:39,666][20347] Updated weights for policy 0, policy_version 971179 (0.0005) [2023-12-27 12:22:39,700][20381] Updated weights for policy 1, policy_version 970419 (0.0008) [2023-12-27 12:22:39,714][20347] Updated weights for policy 0, policy_version 971189 (0.0006) [2023-12-27 12:22:40,399][20292] KL-divergence is very high: 140.9606 [2023-12-27 12:22:40,413][20381] Updated weights for policy 1, policy_version 970429 (0.0007) [2023-12-27 12:22:40,414][20292] KL-divergence is very high: 112.8209 [2023-12-27 12:22:40,449][20347] Updated weights for policy 0, policy_version 971199 (0.0006) [2023-12-27 12:22:40,455][20292] KL-divergence is very high: 145.9736 [2023-12-27 12:22:40,465][20381] Updated weights for policy 1, policy_version 970439 (0.0008) [2023-12-27 12:22:40,482][20292] KL-divergence is very high: 126.3325 [2023-12-27 12:22:40,498][20292] KL-divergence is very high: 132.1566 [2023-12-27 12:22:40,507][20347] Updated weights for policy 0, policy_version 971209 (0.0007) [2023-12-27 12:22:40,519][20381] Updated weights for policy 1, policy_version 970449 (0.0008) [2023-12-27 12:22:40,538][20292] KL-divergence is very high: 112.9352 [2023-12-27 12:22:40,565][20347] Updated weights for policy 0, policy_version 971219 (0.0006) [2023-12-27 12:22:41,235][20381] Updated weights for policy 1, policy_version 970459 (0.0009) [2023-12-27 12:22:41,271][20347] Updated weights for policy 0, policy_version 971229 (0.0008) [2023-12-27 12:22:41,298][20381] Updated weights for policy 1, policy_version 970469 (0.0008) [2023-12-27 12:22:41,325][20347] Updated weights for policy 0, policy_version 971239 (0.0007) [2023-12-27 12:22:41,367][20381] Updated weights for policy 1, policy_version 970479 (0.0008) [2023-12-27 12:22:41,399][20347] Updated weights for policy 0, policy_version 971249 (0.0007) [2023-12-27 12:22:42,079][20381] Updated weights for policy 1, policy_version 970489 (0.0008) [2023-12-27 12:22:42,099][20347] Updated weights for policy 0, policy_version 971259 (0.0006) [2023-12-27 12:22:42,139][20381] Updated weights for policy 1, policy_version 970499 (0.0006) [2023-12-27 12:22:42,155][20347] Updated weights for policy 0, policy_version 971269 (0.0008) [2023-12-27 12:22:42,197][20381] Updated weights for policy 1, policy_version 970509 (0.0005) [2023-12-27 12:22:42,214][20347] Updated weights for policy 0, policy_version 971279 (0.0007) [2023-12-27 12:22:42,253][20381] Updated weights for policy 1, policy_version 970519 (0.0006) [2023-12-27 12:22:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20271.7). Total num frames: 497172480. Throughput: 0: 10323.0, 1: 10238.4. Samples: 497158560. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:42,363][19170] Avg episode reward: [(0, '824.690'), (1, '574.840')] [2023-12-27 12:22:42,868][20381] Updated weights for policy 1, policy_version 970529 (0.0007) [2023-12-27 12:22:42,923][20381] Updated weights for policy 1, policy_version 970539 (0.0008) [2023-12-27 12:22:42,976][20347] Updated weights for policy 0, policy_version 971289 (0.0007) [2023-12-27 12:22:42,984][20381] Updated weights for policy 1, policy_version 970549 (0.0008) [2023-12-27 12:22:43,031][20347] Updated weights for policy 0, policy_version 971299 (0.0006) [2023-12-27 12:22:43,085][20347] Updated weights for policy 0, policy_version 971309 (0.0007) [2023-12-27 12:22:43,147][20347] Updated weights for policy 0, policy_version 971319 (0.0007) [2023-12-27 12:22:43,656][20381] Updated weights for policy 1, policy_version 970559 (0.0008) [2023-12-27 12:22:43,712][20381] Updated weights for policy 1, policy_version 970569 (0.0008) [2023-12-27 12:22:43,765][20381] Updated weights for policy 1, policy_version 970579 (0.0010) [2023-12-27 12:22:43,856][20347] Updated weights for policy 0, policy_version 971329 (0.0007) [2023-12-27 12:22:43,911][20347] Updated weights for policy 0, policy_version 971339 (0.0008) [2023-12-27 12:22:43,958][20347] Updated weights for policy 0, policy_version 971349 (0.0008) [2023-12-27 12:22:44,502][20381] Updated weights for policy 1, policy_version 970589 (0.0009) [2023-12-27 12:22:44,561][20381] Updated weights for policy 1, policy_version 970599 (0.0008) [2023-12-27 12:22:44,623][20381] Updated weights for policy 1, policy_version 970609 (0.0007) [2023-12-27 12:22:44,647][20347] Updated weights for policy 0, policy_version 971359 (0.0007) [2023-12-27 12:22:44,691][20347] Updated weights for policy 0, policy_version 971369 (0.0007) [2023-12-27 12:22:44,736][20347] Updated weights for policy 0, policy_version 971379 (0.0008) [2023-12-27 12:22:45,328][20381] Updated weights for policy 1, policy_version 970619 (0.0007) [2023-12-27 12:22:45,384][20381] Updated weights for policy 1, policy_version 970629 (0.0007) [2023-12-27 12:22:45,402][20347] Updated weights for policy 0, policy_version 971389 (0.0006) [2023-12-27 12:22:45,443][20381] Updated weights for policy 1, policy_version 970639 (0.0005) [2023-12-27 12:22:45,459][20347] Updated weights for policy 0, policy_version 971399 (0.0007) [2023-12-27 12:22:45,520][20347] Updated weights for policy 0, policy_version 971409 (0.0007) [2023-12-27 12:22:45,982][20381] Updated weights for policy 1, policy_version 970649 (0.0005) [2023-12-27 12:22:46,038][20381] Updated weights for policy 1, policy_version 970659 (0.0008) [2023-12-27 12:22:46,090][20381] Updated weights for policy 1, policy_version 970669 (0.0007) [2023-12-27 12:22:46,125][20347] Updated weights for policy 0, policy_version 971419 (0.0006) [2023-12-27 12:22:46,140][20381] Updated weights for policy 1, policy_version 970679 (0.0008) [2023-12-27 12:22:46,180][20347] Updated weights for policy 0, policy_version 971429 (0.0005) [2023-12-27 12:22:46,231][20347] Updated weights for policy 0, policy_version 971439 (0.0005) [2023-12-27 12:22:46,810][20381] Updated weights for policy 1, policy_version 970689 (0.0007) [2023-12-27 12:22:46,864][20381] Updated weights for policy 1, policy_version 970699 (0.0007) [2023-12-27 12:22:46,885][20347] Updated weights for policy 0, policy_version 971449 (0.0007) [2023-12-27 12:22:46,917][20381] Updated weights for policy 1, policy_version 970709 (0.0007) [2023-12-27 12:22:46,943][20347] Updated weights for policy 0, policy_version 971459 (0.0005) [2023-12-27 12:22:46,993][20347] Updated weights for policy 0, policy_version 971469 (0.0005) [2023-12-27 12:22:47,049][20347] Updated weights for policy 0, policy_version 971479 (0.0005) [2023-12-27 12:22:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 497270784. Throughput: 0: 10341.3, 1: 10288.6. Samples: 497281376. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:47,364][19170] Avg episode reward: [(0, '824.830'), (1, '345.970')] [2023-12-27 12:22:47,626][20347] Updated weights for policy 0, policy_version 971489 (0.0010) [2023-12-27 12:22:47,650][20381] Updated weights for policy 1, policy_version 970719 (0.0007) [2023-12-27 12:22:47,677][20347] Updated weights for policy 0, policy_version 971499 (0.0009) [2023-12-27 12:22:47,705][20381] Updated weights for policy 1, policy_version 970729 (0.0006) [2023-12-27 12:22:47,733][20347] Updated weights for policy 0, policy_version 971509 (0.0006) [2023-12-27 12:22:47,757][20381] Updated weights for policy 1, policy_version 970739 (0.0005) [2023-12-27 12:22:48,377][20347] Updated weights for policy 0, policy_version 971519 (0.0009) [2023-12-27 12:22:48,440][20347] Updated weights for policy 0, policy_version 971529 (0.0010) [2023-12-27 12:22:48,478][20381] Updated weights for policy 1, policy_version 970749 (0.0009) [2023-12-27 12:22:48,496][20347] Updated weights for policy 0, policy_version 971539 (0.0010) [2023-12-27 12:22:48,537][20381] Updated weights for policy 1, policy_version 970759 (0.0011) [2023-12-27 12:22:48,590][20381] Updated weights for policy 1, policy_version 970769 (0.0011) [2023-12-27 12:22:49,158][20347] Updated weights for policy 0, policy_version 971549 (0.0009) [2023-12-27 12:22:49,214][20347] Updated weights for policy 0, policy_version 971559 (0.0007) [2023-12-27 12:22:49,267][20381] Updated weights for policy 1, policy_version 970779 (0.0010) [2023-12-27 12:22:49,275][20347] Updated weights for policy 0, policy_version 971569 (0.0009) [2023-12-27 12:22:49,331][20381] Updated weights for policy 1, policy_version 970789 (0.0006) [2023-12-27 12:22:49,401][20381] Updated weights for policy 1, policy_version 970799 (0.0008) [2023-12-27 12:22:50,016][20347] Updated weights for policy 0, policy_version 971579 (0.0009) [2023-12-27 12:22:50,060][20381] Updated weights for policy 1, policy_version 970809 (0.0008) [2023-12-27 12:22:50,085][20347] Updated weights for policy 0, policy_version 971589 (0.0008) [2023-12-27 12:22:50,118][20381] Updated weights for policy 1, policy_version 970819 (0.0007) [2023-12-27 12:22:50,144][20347] Updated weights for policy 0, policy_version 971599 (0.0007) [2023-12-27 12:22:50,173][20381] Updated weights for policy 1, policy_version 970829 (0.0005) [2023-12-27 12:22:50,230][20381] Updated weights for policy 1, policy_version 970839 (0.0007) [2023-12-27 12:22:50,824][20347] Updated weights for policy 0, policy_version 971609 (0.0007) [2023-12-27 12:22:50,885][20347] Updated weights for policy 0, policy_version 971619 (0.0006) [2023-12-27 12:22:50,940][20347] Updated weights for policy 0, policy_version 971629 (0.0007) [2023-12-27 12:22:50,950][20381] Updated weights for policy 1, policy_version 970849 (0.0006) [2023-12-27 12:22:50,996][20347] Updated weights for policy 0, policy_version 971639 (0.0006) [2023-12-27 12:22:51,007][20381] Updated weights for policy 1, policy_version 970859 (0.0006) [2023-12-27 12:22:51,064][20381] Updated weights for policy 1, policy_version 970869 (0.0008) [2023-12-27 12:22:51,749][20347] Updated weights for policy 0, policy_version 971649 (0.0008) [2023-12-27 12:22:51,807][20347] Updated weights for policy 0, policy_version 971659 (0.0008) [2023-12-27 12:22:51,859][20381] Updated weights for policy 1, policy_version 970879 (0.0007) [2023-12-27 12:22:51,864][20347] Updated weights for policy 0, policy_version 971669 (0.0008) [2023-12-27 12:22:51,947][20381] Updated weights for policy 1, policy_version 970889 (0.0007) [2023-12-27 12:22:52,005][20381] Updated weights for policy 1, policy_version 970899 (0.0008) [2023-12-27 12:22:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 497369088. Throughput: 0: 10335.1, 1: 10228.8. Samples: 497342144. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:52,363][19170] Avg episode reward: [(0, '824.620'), (1, '505.800')] [2023-12-27 12:22:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000970904_248578048.pth... [2023-12-27 12:22:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000971672_248791040.pth... [2023-12-27 12:22:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000969720_248274944.pth [2023-12-27 12:22:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000970456_248479744.pth [2023-12-27 12:22:52,598][20347] Updated weights for policy 0, policy_version 971679 (0.0007) [2023-12-27 12:22:52,652][20381] Updated weights for policy 1, policy_version 970909 (0.0008) [2023-12-27 12:22:52,658][20347] Updated weights for policy 0, policy_version 971689 (0.0006) [2023-12-27 12:22:52,703][20381] Updated weights for policy 1, policy_version 970919 (0.0005) [2023-12-27 12:22:52,715][20347] Updated weights for policy 0, policy_version 971699 (0.0007) [2023-12-27 12:22:52,757][20381] Updated weights for policy 1, policy_version 970929 (0.0006) [2023-12-27 12:22:53,406][20347] Updated weights for policy 0, policy_version 971709 (0.0006) [2023-12-27 12:22:53,408][20381] Updated weights for policy 1, policy_version 970939 (0.0008) [2023-12-27 12:22:53,456][20381] Updated weights for policy 1, policy_version 970949 (0.0005) [2023-12-27 12:22:53,458][20347] Updated weights for policy 0, policy_version 971719 (0.0005) [2023-12-27 12:22:53,507][20347] Updated weights for policy 0, policy_version 971729 (0.0005) [2023-12-27 12:22:53,508][20381] Updated weights for policy 1, policy_version 970959 (0.0006) [2023-12-27 12:22:54,207][20381] Updated weights for policy 1, policy_version 970969 (0.0006) [2023-12-27 12:22:54,217][20347] Updated weights for policy 0, policy_version 971739 (0.0007) [2023-12-27 12:22:54,267][20381] Updated weights for policy 1, policy_version 970979 (0.0008) [2023-12-27 12:22:54,270][20347] Updated weights for policy 0, policy_version 971749 (0.0010) [2023-12-27 12:22:54,330][20381] Updated weights for policy 1, policy_version 970989 (0.0007) [2023-12-27 12:22:54,331][20347] Updated weights for policy 0, policy_version 971759 (0.0011) [2023-12-27 12:22:54,395][20381] Updated weights for policy 1, policy_version 970999 (0.0006) [2023-12-27 12:22:54,966][20347] Updated weights for policy 0, policy_version 971769 (0.0008) [2023-12-27 12:22:55,024][20347] Updated weights for policy 0, policy_version 971779 (0.0006) [2023-12-27 12:22:55,081][20347] Updated weights for policy 0, policy_version 971789 (0.0005) [2023-12-27 12:22:55,108][20381] Updated weights for policy 1, policy_version 971009 (0.0008) [2023-12-27 12:22:55,136][20347] Updated weights for policy 0, policy_version 971799 (0.0005) [2023-12-27 12:22:55,170][20381] Updated weights for policy 1, policy_version 971019 (0.0008) [2023-12-27 12:22:55,231][20381] Updated weights for policy 1, policy_version 971029 (0.0007) [2023-12-27 12:22:55,743][20347] Updated weights for policy 0, policy_version 971809 (0.0008) [2023-12-27 12:22:55,809][20347] Updated weights for policy 0, policy_version 971819 (0.0009) [2023-12-27 12:22:55,879][20347] Updated weights for policy 0, policy_version 971829 (0.0008) [2023-12-27 12:22:55,916][20381] Updated weights for policy 1, policy_version 971039 (0.0007) [2023-12-27 12:22:55,977][20381] Updated weights for policy 1, policy_version 971049 (0.0008) [2023-12-27 12:22:56,042][20381] Updated weights for policy 1, policy_version 971059 (0.0007) [2023-12-27 12:22:56,588][20347] Updated weights for policy 0, policy_version 971839 (0.0006) [2023-12-27 12:22:56,637][20347] Updated weights for policy 0, policy_version 971849 (0.0005) [2023-12-27 12:22:56,659][20381] Updated weights for policy 1, policy_version 971069 (0.0006) [2023-12-27 12:22:56,686][20347] Updated weights for policy 0, policy_version 971859 (0.0005) [2023-12-27 12:22:56,704][20381] Updated weights for policy 1, policy_version 971079 (0.0007) [2023-12-27 12:22:56,754][20381] Updated weights for policy 1, policy_version 971089 (0.0007) [2023-12-27 12:22:57,284][20347] Updated weights for policy 0, policy_version 971869 (0.0008) [2023-12-27 12:22:57,343][20381] Updated weights for policy 1, policy_version 971099 (0.0007) [2023-12-27 12:22:57,350][20347] Updated weights for policy 0, policy_version 971879 (0.0011) [2023-12-27 12:22:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20244.0). Total num frames: 497467392. Throughput: 0: 10276.8, 1: 10193.2. Samples: 497461252. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:22:57,363][19170] Avg episode reward: [(0, '824.280'), (1, '653.540')] [2023-12-27 12:22:57,397][20381] Updated weights for policy 1, policy_version 971109 (0.0008) [2023-12-27 12:22:57,411][20347] Updated weights for policy 0, policy_version 971889 (0.0011) [2023-12-27 12:22:57,455][20381] Updated weights for policy 1, policy_version 971119 (0.0006) [2023-12-27 12:22:58,037][20347] Updated weights for policy 0, policy_version 971899 (0.0011) [2023-12-27 12:22:58,071][20381] Updated weights for policy 1, policy_version 971129 (0.0007) [2023-12-27 12:22:58,092][20347] Updated weights for policy 0, policy_version 971909 (0.0008) [2023-12-27 12:22:58,126][20381] Updated weights for policy 1, policy_version 971139 (0.0007) [2023-12-27 12:22:58,142][20347] Updated weights for policy 0, policy_version 971919 (0.0005) [2023-12-27 12:22:58,171][20381] Updated weights for policy 1, policy_version 971149 (0.0008) [2023-12-27 12:22:58,240][20381] Updated weights for policy 1, policy_version 971159 (0.0008) [2023-12-27 12:22:58,928][20347] Updated weights for policy 0, policy_version 971929 (0.0006) [2023-12-27 12:22:58,992][20347] Updated weights for policy 0, policy_version 971939 (0.0008) [2023-12-27 12:22:59,056][20347] Updated weights for policy 0, policy_version 971949 (0.0008) [2023-12-27 12:22:59,115][20381] Updated weights for policy 1, policy_version 971169 (0.0008) [2023-12-27 12:22:59,116][20347] Updated weights for policy 0, policy_version 971959 (0.0008) [2023-12-27 12:22:59,182][20381] Updated weights for policy 1, policy_version 971179 (0.0008) [2023-12-27 12:22:59,254][20381] Updated weights for policy 1, policy_version 971189 (0.0008) [2023-12-27 12:22:59,875][20347] Updated weights for policy 0, policy_version 971969 (0.0007) [2023-12-27 12:22:59,940][20347] Updated weights for policy 0, policy_version 971979 (0.0007) [2023-12-27 12:22:59,999][20381] Updated weights for policy 1, policy_version 971199 (0.0009) [2023-12-27 12:23:00,000][20347] Updated weights for policy 0, policy_version 971989 (0.0010) [2023-12-27 12:23:00,057][20381] Updated weights for policy 1, policy_version 971209 (0.0007) [2023-12-27 12:23:00,107][20381] Updated weights for policy 1, policy_version 971219 (0.0006) [2023-12-27 12:23:00,678][20347] Updated weights for policy 0, policy_version 971999 (0.0010) [2023-12-27 12:23:00,704][20381] Updated weights for policy 1, policy_version 971229 (0.0009) [2023-12-27 12:23:00,725][20292] KL-divergence is very high: 177.9323 [2023-12-27 12:23:00,732][20347] Updated weights for policy 0, policy_version 972009 (0.0010) [2023-12-27 12:23:00,755][20381] Updated weights for policy 1, policy_version 971239 (0.0007) [2023-12-27 12:23:00,767][20292] KL-divergence is very high: 203.3483 [2023-12-27 12:23:00,789][20347] Updated weights for policy 0, policy_version 972019 (0.0010) [2023-12-27 12:23:00,807][20381] Updated weights for policy 1, policy_version 971249 (0.0008) [2023-12-27 12:23:00,807][20292] KL-divergence is very high: 147.8264 [2023-12-27 12:23:01,423][20381] Updated weights for policy 1, policy_version 971259 (0.0007) [2023-12-27 12:23:01,467][20347] Updated weights for policy 0, policy_version 972029 (0.0007) [2023-12-27 12:23:01,484][20381] Updated weights for policy 1, policy_version 971269 (0.0006) [2023-12-27 12:23:01,528][20347] Updated weights for policy 0, policy_version 972039 (0.0006) [2023-12-27 12:23:01,538][20381] Updated weights for policy 1, policy_version 971279 (0.0006) [2023-12-27 12:23:01,586][20347] Updated weights for policy 0, policy_version 972049 (0.0006) [2023-12-27 12:23:02,254][20381] Updated weights for policy 1, policy_version 971289 (0.0006) [2023-12-27 12:23:02,272][20347] Updated weights for policy 0, policy_version 972059 (0.0008) [2023-12-27 12:23:02,319][20381] Updated weights for policy 1, policy_version 971299 (0.0007) [2023-12-27 12:23:02,329][20347] Updated weights for policy 0, policy_version 972069 (0.0006) [2023-12-27 12:23:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20271.7). Total num frames: 497565696. Throughput: 0: 10276.6, 1: 10171.1. Samples: 497583668. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:23:02,363][19170] Avg episode reward: [(0, '873.460'), (1, '430.220')] [2023-12-27 12:23:02,386][20381] Updated weights for policy 1, policy_version 971309 (0.0007) [2023-12-27 12:23:02,391][20347] Updated weights for policy 0, policy_version 972079 (0.0006) [2023-12-27 12:23:02,440][20381] Updated weights for policy 1, policy_version 971319 (0.0007) [2023-12-27 12:23:03,065][20347] Updated weights for policy 0, policy_version 972089 (0.0006) [2023-12-27 12:23:03,124][20347] Updated weights for policy 0, policy_version 972099 (0.0007) [2023-12-27 12:23:03,159][20381] Updated weights for policy 1, policy_version 971329 (0.0008) [2023-12-27 12:23:03,177][20347] Updated weights for policy 0, policy_version 972109 (0.0006) [2023-12-27 12:23:03,206][20381] Updated weights for policy 1, policy_version 971339 (0.0005) [2023-12-27 12:23:03,230][20347] Updated weights for policy 0, policy_version 972119 (0.0007) [2023-12-27 12:23:03,252][20381] Updated weights for policy 1, policy_version 971349 (0.0007) [2023-12-27 12:23:03,890][20381] Updated weights for policy 1, policy_version 971359 (0.0007) [2023-12-27 12:23:03,947][20381] Updated weights for policy 1, policy_version 971369 (0.0007) [2023-12-27 12:23:03,966][20347] Updated weights for policy 0, policy_version 972129 (0.0008) [2023-12-27 12:23:04,006][20381] Updated weights for policy 1, policy_version 971379 (0.0007) [2023-12-27 12:23:04,026][20347] Updated weights for policy 0, policy_version 972139 (0.0008) [2023-12-27 12:23:04,089][20347] Updated weights for policy 0, policy_version 972149 (0.0008) [2023-12-27 12:23:04,707][20381] Updated weights for policy 1, policy_version 971389 (0.0007) [2023-12-27 12:23:04,766][20381] Updated weights for policy 1, policy_version 971399 (0.0007) [2023-12-27 12:23:04,818][20347] Updated weights for policy 0, policy_version 972159 (0.0006) [2023-12-27 12:23:04,820][20381] Updated weights for policy 1, policy_version 971409 (0.0008) [2023-12-27 12:23:04,881][20347] Updated weights for policy 0, policy_version 972169 (0.0005) [2023-12-27 12:23:04,938][20347] Updated weights for policy 0, policy_version 972179 (0.0005) [2023-12-27 12:23:05,492][20381] Updated weights for policy 1, policy_version 971419 (0.0008) [2023-12-27 12:23:05,540][20347] Updated weights for policy 0, policy_version 972189 (0.0008) [2023-12-27 12:23:05,548][20381] Updated weights for policy 1, policy_version 971429 (0.0010) [2023-12-27 12:23:05,595][20347] Updated weights for policy 0, policy_version 972199 (0.0010) [2023-12-27 12:23:05,607][20381] Updated weights for policy 1, policy_version 971439 (0.0010) [2023-12-27 12:23:05,654][20347] Updated weights for policy 0, policy_version 972209 (0.0010) [2023-12-27 12:23:06,299][20381] Updated weights for policy 1, policy_version 971449 (0.0010) [2023-12-27 12:23:06,309][20347] Updated weights for policy 0, policy_version 972219 (0.0010) [2023-12-27 12:23:06,361][20291] KL-divergence is very high: 109.4947 [2023-12-27 12:23:06,363][20381] Updated weights for policy 1, policy_version 971459 (0.0011) [2023-12-27 12:23:06,369][20347] Updated weights for policy 0, policy_version 972229 (0.0011) [2023-12-27 12:23:06,422][20381] Updated weights for policy 1, policy_version 971469 (0.0010) [2023-12-27 12:23:06,428][20347] Updated weights for policy 0, policy_version 972239 (0.0010) [2023-12-27 12:23:06,481][20381] Updated weights for policy 1, policy_version 971479 (0.0011) [2023-12-27 12:23:07,060][20347] Updated weights for policy 0, policy_version 972249 (0.0009) [2023-12-27 12:23:07,126][20347] Updated weights for policy 0, policy_version 972259 (0.0008) [2023-12-27 12:23:07,163][20381] Updated weights for policy 1, policy_version 971489 (0.0009) [2023-12-27 12:23:07,192][20347] Updated weights for policy 0, policy_version 972269 (0.0007) [2023-12-27 12:23:07,225][20381] Updated weights for policy 1, policy_version 971499 (0.0008) [2023-12-27 12:23:07,255][20347] Updated weights for policy 0, policy_version 972279 (0.0008) [2023-12-27 12:23:07,282][20381] Updated weights for policy 1, policy_version 971509 (0.0010) [2023-12-27 12:23:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20327.3). Total num frames: 497680384. Throughput: 0: 10258.5, 1: 10174.2. Samples: 497644176. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:23:07,363][19170] Avg episode reward: [(0, '892.770'), (1, '193.680')] [2023-12-27 12:23:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000971512_248733696.pth... [2023-12-27 12:23:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000972280_248946688.pth... [2023-12-27 12:23:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000971096_248643584.pth [2023-12-27 12:23:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000970328_248430592.pth [2023-12-27 12:23:07,883][20347] Updated weights for policy 0, policy_version 972289 (0.0005) [2023-12-27 12:23:07,918][20381] Updated weights for policy 1, policy_version 971520 (0.0007) [2023-12-27 12:23:07,945][20347] Updated weights for policy 0, policy_version 972299 (0.0007) [2023-12-27 12:23:07,972][20381] Updated weights for policy 1, policy_version 971530 (0.0005) [2023-12-27 12:23:07,999][20347] Updated weights for policy 0, policy_version 972309 (0.0008) [2023-12-27 12:23:08,025][20381] Updated weights for policy 1, policy_version 971540 (0.0006) [2023-12-27 12:23:08,621][20381] Updated weights for policy 1, policy_version 971550 (0.0008) [2023-12-27 12:23:08,680][20381] Updated weights for policy 1, policy_version 971560 (0.0007) [2023-12-27 12:23:08,703][20347] Updated weights for policy 0, policy_version 972319 (0.0008) [2023-12-27 12:23:08,735][20381] Updated weights for policy 1, policy_version 971570 (0.0010) [2023-12-27 12:23:08,752][20347] Updated weights for policy 0, policy_version 972329 (0.0007) [2023-12-27 12:23:08,809][20347] Updated weights for policy 0, policy_version 972339 (0.0005) [2023-12-27 12:23:09,333][20381] Updated weights for policy 1, policy_version 971580 (0.0010) [2023-12-27 12:23:09,401][20381] Updated weights for policy 1, policy_version 971590 (0.0008) [2023-12-27 12:23:09,434][20347] Updated weights for policy 0, policy_version 972349 (0.0007) [2023-12-27 12:23:09,466][20381] Updated weights for policy 1, policy_version 971600 (0.0008) [2023-12-27 12:23:09,491][20347] Updated weights for policy 0, policy_version 972359 (0.0008) [2023-12-27 12:23:09,547][20347] Updated weights for policy 0, policy_version 972369 (0.0008) [2023-12-27 12:23:10,272][20381] Updated weights for policy 1, policy_version 971610 (0.0010) [2023-12-27 12:23:10,322][20381] Updated weights for policy 1, policy_version 971620 (0.0009) [2023-12-27 12:23:10,372][20381] Updated weights for policy 1, policy_version 971630 (0.0009) [2023-12-27 12:23:10,430][20381] Updated weights for policy 1, policy_version 971640 (0.0009) [2023-12-27 12:23:10,433][20347] Updated weights for policy 0, policy_version 972379 (0.0008) [2023-12-27 12:23:10,482][20347] Updated weights for policy 0, policy_version 972389 (0.0009) [2023-12-27 12:23:10,536][20347] Updated weights for policy 0, policy_version 972399 (0.0009) [2023-12-27 12:23:11,324][20381] Updated weights for policy 1, policy_version 971650 (0.0011) [2023-12-27 12:23:11,389][20381] Updated weights for policy 1, policy_version 971660 (0.0010) [2023-12-27 12:23:11,423][20347] Updated weights for policy 0, policy_version 972409 (0.0010) [2023-12-27 12:23:11,446][20381] Updated weights for policy 1, policy_version 971670 (0.0009) [2023-12-27 12:23:11,483][20347] Updated weights for policy 0, policy_version 972419 (0.0007) [2023-12-27 12:23:11,538][20347] Updated weights for policy 0, policy_version 972429 (0.0007) [2023-12-27 12:23:11,601][20347] Updated weights for policy 0, policy_version 972439 (0.0008) [2023-12-27 12:23:12,206][20381] Updated weights for policy 1, policy_version 971680 (0.0008) [2023-12-27 12:23:12,259][20381] Updated weights for policy 1, policy_version 971690 (0.0009) [2023-12-27 12:23:12,316][20381] Updated weights for policy 1, policy_version 971700 (0.0006) [2023-12-27 12:23:12,320][20347] Updated weights for policy 0, policy_version 972449 (0.0008) [2023-12-27 12:23:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20271.9). Total num frames: 497770496. Throughput: 0: 10184.7, 1: 10139.9. Samples: 497762708. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:23:12,363][19170] Avg episode reward: [(0, '882.970'), (1, '85.040')] [2023-12-27 12:23:12,389][20347] Updated weights for policy 0, policy_version 972459 (0.0008) [2023-12-27 12:23:12,447][20347] Updated weights for policy 0, policy_version 972469 (0.0005) [2023-12-27 12:23:13,052][20381] Updated weights for policy 1, policy_version 971710 (0.0009) [2023-12-27 12:23:13,111][20381] Updated weights for policy 1, policy_version 971720 (0.0009) [2023-12-27 12:23:13,149][20347] Updated weights for policy 0, policy_version 972479 (0.0006) [2023-12-27 12:23:13,166][20381] Updated weights for policy 1, policy_version 971730 (0.0011) [2023-12-27 12:23:13,203][20347] Updated weights for policy 0, policy_version 972489 (0.0007) [2023-12-27 12:23:13,253][20347] Updated weights for policy 0, policy_version 972499 (0.0007) [2023-12-27 12:23:13,845][20381] Updated weights for policy 1, policy_version 971740 (0.0010) [2023-12-27 12:23:13,911][20381] Updated weights for policy 1, policy_version 971750 (0.0008) [2023-12-27 12:23:13,918][20347] Updated weights for policy 0, policy_version 972509 (0.0006) [2023-12-27 12:23:13,964][20381] Updated weights for policy 1, policy_version 971760 (0.0007) [2023-12-27 12:23:13,995][20347] Updated weights for policy 0, policy_version 972519 (0.0006) [2023-12-27 12:23:14,047][20347] Updated weights for policy 0, policy_version 972529 (0.0008) [2023-12-27 12:23:14,614][20381] Updated weights for policy 1, policy_version 971770 (0.0006) [2023-12-27 12:23:14,676][20381] Updated weights for policy 1, policy_version 971780 (0.0008) [2023-12-27 12:23:14,722][20347] Updated weights for policy 0, policy_version 972539 (0.0008) [2023-12-27 12:23:14,739][20381] Updated weights for policy 1, policy_version 971790 (0.0007) [2023-12-27 12:23:14,787][20347] Updated weights for policy 0, policy_version 972549 (0.0008) [2023-12-27 12:23:14,798][20381] Updated weights for policy 1, policy_version 971800 (0.0007) [2023-12-27 12:23:14,847][20347] Updated weights for policy 0, policy_version 972559 (0.0007) [2023-12-27 12:23:15,538][20347] Updated weights for policy 0, policy_version 972569 (0.0007) [2023-12-27 12:23:15,549][20381] Updated weights for policy 1, policy_version 971810 (0.0008) [2023-12-27 12:23:15,594][20347] Updated weights for policy 0, policy_version 972579 (0.0005) [2023-12-27 12:23:15,600][20381] Updated weights for policy 1, policy_version 971820 (0.0007) [2023-12-27 12:23:15,640][20347] Updated weights for policy 0, policy_version 972589 (0.0005) [2023-12-27 12:23:15,652][20381] Updated weights for policy 1, policy_version 971830 (0.0007) [2023-12-27 12:23:15,702][20347] Updated weights for policy 0, policy_version 972599 (0.0006) [2023-12-27 12:23:16,277][20381] Updated weights for policy 1, policy_version 971840 (0.0007) [2023-12-27 12:23:16,325][20381] Updated weights for policy 1, policy_version 971850 (0.0008) [2023-12-27 12:23:16,370][20381] Updated weights for policy 1, policy_version 971860 (0.0007) [2023-12-27 12:23:16,399][20347] Updated weights for policy 0, policy_version 972609 (0.0006) [2023-12-27 12:23:16,445][20347] Updated weights for policy 0, policy_version 972619 (0.0008) [2023-12-27 12:23:16,491][20347] Updated weights for policy 0, policy_version 972629 (0.0007) [2023-12-27 12:23:17,092][20347] Updated weights for policy 0, policy_version 972639 (0.0006) [2023-12-27 12:23:17,143][20381] Updated weights for policy 1, policy_version 971870 (0.0007) [2023-12-27 12:23:17,152][20347] Updated weights for policy 0, policy_version 972649 (0.0006) [2023-12-27 12:23:17,198][20381] Updated weights for policy 1, policy_version 971880 (0.0006) [2023-12-27 12:23:17,210][20347] Updated weights for policy 0, policy_version 972659 (0.0005) [2023-12-27 12:23:17,256][20381] Updated weights for policy 1, policy_version 971890 (0.0006) [2023-12-27 12:23:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.6, 300 sec: 20327.3). Total num frames: 497876992. Throughput: 0: 10101.9, 1: 10107.7. Samples: 497881480. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) [2023-12-27 12:23:17,363][19170] Avg episode reward: [(0, '863.220'), (1, '112.520')] [2023-12-27 12:23:17,833][20347] Updated weights for policy 0, policy_version 972669 (0.0006) [2023-12-27 12:23:17,882][20347] Updated weights for policy 0, policy_version 972679 (0.0005) [2023-12-27 12:23:17,909][20381] Updated weights for policy 1, policy_version 971900 (0.0006) [2023-12-27 12:23:17,934][20347] Updated weights for policy 0, policy_version 972689 (0.0005) [2023-12-27 12:23:17,956][20381] Updated weights for policy 1, policy_version 971910 (0.0007) [2023-12-27 12:23:18,005][20381] Updated weights for policy 1, policy_version 971920 (0.0007) [2023-12-27 12:23:18,551][20347] Updated weights for policy 0, policy_version 972699 (0.0006) [2023-12-27 12:23:18,616][20347] Updated weights for policy 0, policy_version 972709 (0.0008) [2023-12-27 12:23:18,673][20347] Updated weights for policy 0, policy_version 972719 (0.0007) [2023-12-27 12:23:18,702][20381] Updated weights for policy 1, policy_version 971930 (0.0008) [2023-12-27 12:23:18,761][20381] Updated weights for policy 1, policy_version 971940 (0.0008) [2023-12-27 12:23:18,822][20381] Updated weights for policy 1, policy_version 971950 (0.0008) [2023-12-27 12:23:18,881][20381] Updated weights for policy 1, policy_version 971960 (0.0008) [2023-12-27 12:23:19,394][20347] Updated weights for policy 0, policy_version 972729 (0.0007) [2023-12-27 12:23:19,459][20347] Updated weights for policy 0, policy_version 972739 (0.0007) [2023-12-27 12:23:19,522][20347] Updated weights for policy 0, policy_version 972749 (0.0008) [2023-12-27 12:23:19,582][20381] Updated weights for policy 1, policy_version 971970 (0.0007) [2023-12-27 12:23:19,584][20347] Updated weights for policy 0, policy_version 972759 (0.0006) [2023-12-27 12:23:19,632][20381] Updated weights for policy 1, policy_version 971980 (0.0007) [2023-12-27 12:23:19,682][20381] Updated weights for policy 1, policy_version 971990 (0.0008) [2023-12-27 12:23:20,314][20347] Updated weights for policy 0, policy_version 972769 (0.0008) [2023-12-27 12:23:20,374][20347] Updated weights for policy 0, policy_version 972779 (0.0007) [2023-12-27 12:23:20,399][20381] Updated weights for policy 1, policy_version 972000 (0.0008) [2023-12-27 12:23:20,439][20347] Updated weights for policy 0, policy_version 972789 (0.0007) [2023-12-27 12:23:20,457][20381] Updated weights for policy 1, policy_version 972010 (0.0006) [2023-12-27 12:23:20,518][20381] Updated weights for policy 1, policy_version 972020 (0.0006) [2023-12-27 12:23:21,134][20347] Updated weights for policy 0, policy_version 972799 (0.0007) [2023-12-27 12:23:21,196][20347] Updated weights for policy 0, policy_version 972809 (0.0008) [2023-12-27 12:23:21,199][20381] Updated weights for policy 1, policy_version 972030 (0.0007) [2023-12-27 12:23:21,256][20347] Updated weights for policy 0, policy_version 972819 (0.0008) [2023-12-27 12:23:21,264][20381] Updated weights for policy 1, policy_version 972040 (0.0007) [2023-12-27 12:23:21,320][20381] Updated weights for policy 1, policy_version 972050 (0.0006) [2023-12-27 12:23:21,953][20347] Updated weights for policy 0, policy_version 972829 (0.0007) [2023-12-27 12:23:22,012][20347] Updated weights for policy 0, policy_version 972839 (0.0008) [2023-12-27 12:23:22,069][20347] Updated weights for policy 0, policy_version 972849 (0.0008) [2023-12-27 12:23:22,070][20381] Updated weights for policy 1, policy_version 972060 (0.0008) [2023-12-27 12:23:22,126][20381] Updated weights for policy 1, policy_version 972070 (0.0007) [2023-12-27 12:23:22,179][20381] Updated weights for policy 1, policy_version 972080 (0.0005) [2023-12-27 12:23:22,364][19170] Fps is (10 sec: 20477.7, 60 sec: 20343.1, 300 sec: 20299.4). Total num frames: 497975296. Throughput: 0: 10100.2, 1: 10071.4. Samples: 497942848. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:22,364][19170] Avg episode reward: [(0, '882.880'), (1, '104.040')] [2023-12-27 12:23:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000972088_248881152.pth... [2023-12-27 12:23:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000972856_249094144.pth... [2023-12-27 12:23:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000970904_248578048.pth [2023-12-27 12:23:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000971672_248791040.pth [2023-12-27 12:23:22,758][20347] Updated weights for policy 0, policy_version 972859 (0.0007) [2023-12-27 12:23:22,816][20347] Updated weights for policy 0, policy_version 972869 (0.0008) [2023-12-27 12:23:22,871][20347] Updated weights for policy 0, policy_version 972879 (0.0008) [2023-12-27 12:23:22,895][20381] Updated weights for policy 1, policy_version 972090 (0.0006) [2023-12-27 12:23:22,951][20381] Updated weights for policy 1, policy_version 972100 (0.0006) [2023-12-27 12:23:23,006][20381] Updated weights for policy 1, policy_version 972110 (0.0005) [2023-12-27 12:23:23,065][20381] Updated weights for policy 1, policy_version 972120 (0.0005) [2023-12-27 12:23:23,525][20347] Updated weights for policy 0, policy_version 972889 (0.0007) [2023-12-27 12:23:23,583][20347] Updated weights for policy 0, policy_version 972899 (0.0005) [2023-12-27 12:23:23,635][20347] Updated weights for policy 0, policy_version 972909 (0.0006) [2023-12-27 12:23:23,674][20381] Updated weights for policy 1, policy_version 972130 (0.0006) [2023-12-27 12:23:23,695][20347] Updated weights for policy 0, policy_version 972919 (0.0006) [2023-12-27 12:23:23,726][20381] Updated weights for policy 1, policy_version 972140 (0.0007) [2023-12-27 12:23:23,773][20381] Updated weights for policy 1, policy_version 972150 (0.0007) [2023-12-27 12:23:24,287][20347] Updated weights for policy 0, policy_version 972929 (0.0008) [2023-12-27 12:23:24,339][20347] Updated weights for policy 0, policy_version 972939 (0.0006) [2023-12-27 12:23:24,385][20347] Updated weights for policy 0, policy_version 972949 (0.0005) [2023-12-27 12:23:24,462][20381] Updated weights for policy 1, policy_version 972160 (0.0008) [2023-12-27 12:23:24,522][20381] Updated weights for policy 1, policy_version 972170 (0.0007) [2023-12-27 12:23:24,582][20381] Updated weights for policy 1, policy_version 972180 (0.0007) [2023-12-27 12:23:25,005][20347] Updated weights for policy 0, policy_version 972959 (0.0008) [2023-12-27 12:23:25,067][20347] Updated weights for policy 0, policy_version 972969 (0.0008) [2023-12-27 12:23:25,131][20347] Updated weights for policy 0, policy_version 972979 (0.0005) [2023-12-27 12:23:25,225][20381] Updated weights for policy 1, policy_version 972190 (0.0006) [2023-12-27 12:23:25,284][20381] Updated weights for policy 1, policy_version 972200 (0.0006) [2023-12-27 12:23:25,345][20381] Updated weights for policy 1, policy_version 972210 (0.0005) [2023-12-27 12:23:25,722][20347] Updated weights for policy 0, policy_version 972989 (0.0005) [2023-12-27 12:23:25,776][20347] Updated weights for policy 0, policy_version 972999 (0.0006) [2023-12-27 12:23:25,837][20347] Updated weights for policy 0, policy_version 973009 (0.0010) [2023-12-27 12:23:26,012][20381] Updated weights for policy 1, policy_version 972220 (0.0008) [2023-12-27 12:23:26,070][20381] Updated weights for policy 1, policy_version 972230 (0.0011) [2023-12-27 12:23:26,129][20381] Updated weights for policy 1, policy_version 972240 (0.0011) [2023-12-27 12:23:26,453][20347] Updated weights for policy 0, policy_version 973019 (0.0009) [2023-12-27 12:23:26,490][20291] KL-divergence is very high: 275.8693 [2023-12-27 12:23:26,509][20347] Updated weights for policy 0, policy_version 973029 (0.0010) [2023-12-27 12:23:26,556][20347] Updated weights for policy 0, policy_version 973039 (0.0010) [2023-12-27 12:23:26,804][20381] Updated weights for policy 1, policy_version 972250 (0.0010) [2023-12-27 12:23:26,859][20381] Updated weights for policy 1, policy_version 972260 (0.0010) [2023-12-27 12:23:26,929][20381] Updated weights for policy 1, policy_version 972270 (0.0008) [2023-12-27 12:23:26,977][20381] Updated weights for policy 1, policy_version 972280 (0.0010) [2023-12-27 12:23:27,234][20347] Updated weights for policy 0, policy_version 973049 (0.0010) [2023-12-27 12:23:27,291][20347] Updated weights for policy 0, policy_version 973059 (0.0007) [2023-12-27 12:23:27,346][20347] Updated weights for policy 0, policy_version 973069 (0.0006) [2023-12-27 12:23:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20299.5). Total num frames: 498073600. Throughput: 0: 10131.5, 1: 10053.2. Samples: 498066876. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:27,363][19170] Avg episode reward: [(0, '805.300'), (1, '111.470')] [2023-12-27 12:23:27,403][20347] Updated weights for policy 0, policy_version 973079 (0.0006) [2023-12-27 12:23:27,621][20381] Updated weights for policy 1, policy_version 972290 (0.0010) [2023-12-27 12:23:27,679][20381] Updated weights for policy 1, policy_version 972300 (0.0010) [2023-12-27 12:23:27,741][20381] Updated weights for policy 1, policy_version 972310 (0.0011) [2023-12-27 12:23:28,055][20347] Updated weights for policy 0, policy_version 973089 (0.0006) [2023-12-27 12:23:28,110][20347] Updated weights for policy 0, policy_version 973099 (0.0005) [2023-12-27 12:23:28,154][20347] Updated weights for policy 0, policy_version 973109 (0.0005) [2023-12-27 12:23:28,350][20381] Updated weights for policy 1, policy_version 972320 (0.0009) [2023-12-27 12:23:28,407][20381] Updated weights for policy 1, policy_version 972330 (0.0009) [2023-12-27 12:23:28,468][20381] Updated weights for policy 1, policy_version 972340 (0.0010) [2023-12-27 12:23:28,781][20347] Updated weights for policy 0, policy_version 973119 (0.0006) [2023-12-27 12:23:28,839][20347] Updated weights for policy 0, policy_version 973129 (0.0006) [2023-12-27 12:23:28,900][20347] Updated weights for policy 0, policy_version 973139 (0.0007) [2023-12-27 12:23:29,160][20381] Updated weights for policy 1, policy_version 972350 (0.0011) [2023-12-27 12:23:29,206][20381] Updated weights for policy 1, policy_version 972360 (0.0010) [2023-12-27 12:23:29,260][20381] Updated weights for policy 1, policy_version 972370 (0.0008) [2023-12-27 12:23:29,471][20347] Updated weights for policy 0, policy_version 973149 (0.0007) [2023-12-27 12:23:29,525][20347] Updated weights for policy 0, policy_version 973159 (0.0008) [2023-12-27 12:23:29,573][20347] Updated weights for policy 0, policy_version 973169 (0.0006) [2023-12-27 12:23:29,993][20381] Updated weights for policy 1, policy_version 972380 (0.0007) [2023-12-27 12:23:30,043][20381] Updated weights for policy 1, policy_version 972390 (0.0005) [2023-12-27 12:23:30,093][20381] Updated weights for policy 1, policy_version 972400 (0.0006) [2023-12-27 12:23:30,330][20347] Updated weights for policy 0, policy_version 973179 (0.0009) [2023-12-27 12:23:30,383][20347] Updated weights for policy 0, policy_version 973189 (0.0009) [2023-12-27 12:23:30,433][20347] Updated weights for policy 0, policy_version 973199 (0.0007) [2023-12-27 12:23:30,758][20381] Updated weights for policy 1, policy_version 972410 (0.0011) [2023-12-27 12:23:30,811][20381] Updated weights for policy 1, policy_version 972420 (0.0011) [2023-12-27 12:23:30,863][20381] Updated weights for policy 1, policy_version 972430 (0.0010) [2023-12-27 12:23:30,919][20381] Updated weights for policy 1, policy_version 972440 (0.0010) [2023-12-27 12:23:31,110][20347] Updated weights for policy 0, policy_version 973209 (0.0007) [2023-12-27 12:23:31,181][20347] Updated weights for policy 0, policy_version 973219 (0.0008) [2023-12-27 12:23:31,238][20347] Updated weights for policy 0, policy_version 973229 (0.0007) [2023-12-27 12:23:31,297][20347] Updated weights for policy 0, policy_version 973239 (0.0007) [2023-12-27 12:23:31,727][20381] Updated weights for policy 1, policy_version 972450 (0.0009) [2023-12-27 12:23:31,779][20381] Updated weights for policy 1, policy_version 972460 (0.0010) [2023-12-27 12:23:31,827][20381] Updated weights for policy 1, policy_version 972470 (0.0006) [2023-12-27 12:23:31,957][20347] Updated weights for policy 0, policy_version 973249 (0.0009) [2023-12-27 12:23:32,014][20347] Updated weights for policy 0, policy_version 973259 (0.0009) [2023-12-27 12:23:32,067][20347] Updated weights for policy 0, policy_version 973269 (0.0006) [2023-12-27 12:23:32,362][19170] Fps is (10 sec: 20482.5, 60 sec: 20343.5, 300 sec: 20299.5). Total num frames: 498180096. Throughput: 0: 10176.5, 1: 10056.5. Samples: 498191856. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:32,363][19170] Avg episode reward: [(0, '853.790'), (1, '146.930')] [2023-12-27 12:23:32,429][20381] Updated weights for policy 1, policy_version 972480 (0.0008) [2023-12-27 12:23:32,472][20381] Updated weights for policy 1, policy_version 972490 (0.0005) [2023-12-27 12:23:32,524][20381] Updated weights for policy 1, policy_version 972500 (0.0006) [2023-12-27 12:23:32,738][20347] Updated weights for policy 0, policy_version 973279 (0.0008) [2023-12-27 12:23:32,787][20347] Updated weights for policy 0, policy_version 973289 (0.0008) [2023-12-27 12:23:32,843][20347] Updated weights for policy 0, policy_version 973299 (0.0008) [2023-12-27 12:23:33,143][20381] Updated weights for policy 1, policy_version 972510 (0.0007) [2023-12-27 12:23:33,200][20381] Updated weights for policy 1, policy_version 972520 (0.0007) [2023-12-27 12:23:33,258][20381] Updated weights for policy 1, policy_version 972530 (0.0008) [2023-12-27 12:23:33,596][20347] Updated weights for policy 0, policy_version 973309 (0.0010) [2023-12-27 12:23:33,641][20347] Updated weights for policy 0, policy_version 973319 (0.0010) [2023-12-27 12:23:33,687][20347] Updated weights for policy 0, policy_version 973329 (0.0010) [2023-12-27 12:23:33,886][20381] Updated weights for policy 1, policy_version 972540 (0.0008) [2023-12-27 12:23:33,937][20381] Updated weights for policy 1, policy_version 972550 (0.0009) [2023-12-27 12:23:33,989][20381] Updated weights for policy 1, policy_version 972560 (0.0007) [2023-12-27 12:23:34,357][20347] Updated weights for policy 0, policy_version 973339 (0.0008) [2023-12-27 12:23:34,410][20347] Updated weights for policy 0, policy_version 973349 (0.0008) [2023-12-27 12:23:34,475][20347] Updated weights for policy 0, policy_version 973359 (0.0008) [2023-12-27 12:23:34,694][20381] Updated weights for policy 1, policy_version 972570 (0.0007) [2023-12-27 12:23:34,752][20381] Updated weights for policy 1, policy_version 972580 (0.0008) [2023-12-27 12:23:34,808][20381] Updated weights for policy 1, policy_version 972590 (0.0007) [2023-12-27 12:23:34,870][20381] Updated weights for policy 1, policy_version 972600 (0.0008) [2023-12-27 12:23:35,214][20347] Updated weights for policy 0, policy_version 973369 (0.0007) [2023-12-27 12:23:35,268][20347] Updated weights for policy 0, policy_version 973379 (0.0007) [2023-12-27 12:23:35,315][20347] Updated weights for policy 0, policy_version 973389 (0.0007) [2023-12-27 12:23:35,365][20347] Updated weights for policy 0, policy_version 973399 (0.0008) [2023-12-27 12:23:35,574][20381] Updated weights for policy 1, policy_version 972610 (0.0011) [2023-12-27 12:23:35,629][20381] Updated weights for policy 1, policy_version 972620 (0.0010) [2023-12-27 12:23:35,680][20381] Updated weights for policy 1, policy_version 972630 (0.0008) [2023-12-27 12:23:36,033][20347] Updated weights for policy 0, policy_version 973409 (0.0009) [2023-12-27 12:23:36,093][20347] Updated weights for policy 0, policy_version 973419 (0.0010) [2023-12-27 12:23:36,154][20347] Updated weights for policy 0, policy_version 973429 (0.0011) [2023-12-27 12:23:36,345][20381] Updated weights for policy 1, policy_version 972640 (0.0006) [2023-12-27 12:23:36,419][20381] Updated weights for policy 1, policy_version 972650 (0.0006) [2023-12-27 12:23:36,482][20381] Updated weights for policy 1, policy_version 972660 (0.0007) [2023-12-27 12:23:36,860][20347] Updated weights for policy 0, policy_version 973439 (0.0008) [2023-12-27 12:23:36,928][20347] Updated weights for policy 0, policy_version 973449 (0.0008) [2023-12-27 12:23:36,987][20347] Updated weights for policy 0, policy_version 973459 (0.0006) [2023-12-27 12:23:37,149][20381] Updated weights for policy 1, policy_version 972670 (0.0007) [2023-12-27 12:23:37,216][20381] Updated weights for policy 1, policy_version 972680 (0.0006) [2023-12-27 12:23:37,284][20381] Updated weights for policy 1, policy_version 972690 (0.0006) [2023-12-27 12:23:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20327.3). Total num frames: 498286592. Throughput: 0: 10155.6, 1: 10099.8. Samples: 498253632. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:37,363][19170] Avg episode reward: [(0, '824.210'), (1, '192.420')] [2023-12-27 12:23:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000972696_249036800.pth... [2023-12-27 12:23:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000973464_249249792.pth... [2023-12-27 12:23:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000971512_248733696.pth [2023-12-27 12:23:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000972280_248946688.pth [2023-12-27 12:23:37,687][20347] Updated weights for policy 0, policy_version 973469 (0.0007) [2023-12-27 12:23:37,732][20347] Updated weights for policy 0, policy_version 973479 (0.0007) [2023-12-27 12:23:37,777][20347] Updated weights for policy 0, policy_version 973489 (0.0006) [2023-12-27 12:23:37,921][20381] Updated weights for policy 1, policy_version 972700 (0.0007) [2023-12-27 12:23:37,979][20381] Updated weights for policy 1, policy_version 972710 (0.0007) [2023-12-27 12:23:38,037][20381] Updated weights for policy 1, policy_version 972720 (0.0007) [2023-12-27 12:23:38,501][20347] Updated weights for policy 0, policy_version 973499 (0.0007) [2023-12-27 12:23:38,563][20347] Updated weights for policy 0, policy_version 973509 (0.0008) [2023-12-27 12:23:38,622][20347] Updated weights for policy 0, policy_version 973519 (0.0008) [2023-12-27 12:23:38,730][20381] Updated weights for policy 1, policy_version 972730 (0.0006) [2023-12-27 12:23:38,788][20381] Updated weights for policy 1, policy_version 972740 (0.0007) [2023-12-27 12:23:38,846][20381] Updated weights for policy 1, policy_version 972750 (0.0008) [2023-12-27 12:23:38,906][20381] Updated weights for policy 1, policy_version 972760 (0.0007) [2023-12-27 12:23:39,411][20347] Updated weights for policy 0, policy_version 973529 (0.0007) [2023-12-27 12:23:39,475][20347] Updated weights for policy 0, policy_version 973539 (0.0008) [2023-12-27 12:23:39,531][20347] Updated weights for policy 0, policy_version 973549 (0.0006) [2023-12-27 12:23:39,556][20381] Updated weights for policy 1, policy_version 972770 (0.0006) [2023-12-27 12:23:39,582][20347] Updated weights for policy 0, policy_version 973559 (0.0006) [2023-12-27 12:23:39,616][20381] Updated weights for policy 1, policy_version 972780 (0.0008) [2023-12-27 12:23:39,683][20381] Updated weights for policy 1, policy_version 972790 (0.0009) [2023-12-27 12:23:40,327][20347] Updated weights for policy 0, policy_version 973569 (0.0006) [2023-12-27 12:23:40,384][20347] Updated weights for policy 0, policy_version 973579 (0.0006) [2023-12-27 12:23:40,429][20381] Updated weights for policy 1, policy_version 972800 (0.0007) [2023-12-27 12:23:40,452][20347] Updated weights for policy 0, policy_version 973589 (0.0005) [2023-12-27 12:23:40,486][20381] Updated weights for policy 1, policy_version 972810 (0.0006) [2023-12-27 12:23:40,540][20381] Updated weights for policy 1, policy_version 972820 (0.0006) [2023-12-27 12:23:41,113][20347] Updated weights for policy 0, policy_version 973599 (0.0008) [2023-12-27 12:23:41,175][20347] Updated weights for policy 0, policy_version 973609 (0.0008) [2023-12-27 12:23:41,222][20381] Updated weights for policy 1, policy_version 972830 (0.0009) [2023-12-27 12:23:41,240][20347] Updated weights for policy 0, policy_version 973619 (0.0007) [2023-12-27 12:23:41,286][20381] Updated weights for policy 1, policy_version 972840 (0.0006) [2023-12-27 12:23:41,350][20381] Updated weights for policy 1, policy_version 972850 (0.0007) [2023-12-27 12:23:42,045][20381] Updated weights for policy 1, policy_version 972860 (0.0008) [2023-12-27 12:23:42,059][20347] Updated weights for policy 0, policy_version 973629 (0.0008) [2023-12-27 12:23:42,106][20381] Updated weights for policy 1, policy_version 972870 (0.0006) [2023-12-27 12:23:42,115][20347] Updated weights for policy 0, policy_version 973639 (0.0008) [2023-12-27 12:23:42,170][20347] Updated weights for policy 0, policy_version 973649 (0.0008) [2023-12-27 12:23:42,175][20381] Updated weights for policy 1, policy_version 972880 (0.0006) [2023-12-27 12:23:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20327.3). Total num frames: 498384896. Throughput: 0: 10127.9, 1: 10123.3. Samples: 498372556. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:42,363][19170] Avg episode reward: [(0, '863.030'), (1, '295.710')] [2023-12-27 12:23:42,847][20381] Updated weights for policy 1, policy_version 972890 (0.0007) [2023-12-27 12:23:42,887][20347] Updated weights for policy 0, policy_version 973659 (0.0008) [2023-12-27 12:23:42,904][20381] Updated weights for policy 1, policy_version 972900 (0.0006) [2023-12-27 12:23:42,946][20347] Updated weights for policy 0, policy_version 973669 (0.0007) [2023-12-27 12:23:42,957][20381] Updated weights for policy 1, policy_version 972910 (0.0006) [2023-12-27 12:23:42,994][20347] Updated weights for policy 0, policy_version 973679 (0.0005) [2023-12-27 12:23:43,012][20381] Updated weights for policy 1, policy_version 972920 (0.0009) [2023-12-27 12:23:43,640][20381] Updated weights for policy 1, policy_version 972930 (0.0010) [2023-12-27 12:23:43,701][20381] Updated weights for policy 1, policy_version 972940 (0.0010) [2023-12-27 12:23:43,715][20347] Updated weights for policy 0, policy_version 973689 (0.0005) [2023-12-27 12:23:43,776][20347] Updated weights for policy 0, policy_version 973699 (0.0007) [2023-12-27 12:23:43,780][20381] Updated weights for policy 1, policy_version 972950 (0.0010) [2023-12-27 12:23:43,835][20347] Updated weights for policy 0, policy_version 973709 (0.0008) [2023-12-27 12:23:43,881][20347] Updated weights for policy 0, policy_version 973719 (0.0007) [2023-12-27 12:23:44,434][20381] Updated weights for policy 1, policy_version 972960 (0.0010) [2023-12-27 12:23:44,483][20381] Updated weights for policy 1, policy_version 972970 (0.0010) [2023-12-27 12:23:44,541][20381] Updated weights for policy 1, policy_version 972980 (0.0011) [2023-12-27 12:23:44,605][20347] Updated weights for policy 0, policy_version 973729 (0.0007) [2023-12-27 12:23:44,670][20347] Updated weights for policy 0, policy_version 973739 (0.0007) [2023-12-27 12:23:44,721][20347] Updated weights for policy 0, policy_version 973749 (0.0007) [2023-12-27 12:23:45,292][20381] Updated weights for policy 1, policy_version 972990 (0.0007) [2023-12-27 12:23:45,349][20381] Updated weights for policy 1, policy_version 973000 (0.0009) [2023-12-27 12:23:45,406][20381] Updated weights for policy 1, policy_version 973010 (0.0010) [2023-12-27 12:23:45,445][20347] Updated weights for policy 0, policy_version 973759 (0.0007) [2023-12-27 12:23:45,507][20347] Updated weights for policy 0, policy_version 973769 (0.0007) [2023-12-27 12:23:45,569][20347] Updated weights for policy 0, policy_version 973779 (0.0010) [2023-12-27 12:23:46,088][20381] Updated weights for policy 1, policy_version 973020 (0.0011) [2023-12-27 12:23:46,154][20381] Updated weights for policy 1, policy_version 973030 (0.0011) [2023-12-27 12:23:46,203][20347] Updated weights for policy 0, policy_version 973789 (0.0008) [2023-12-27 12:23:46,224][20381] Updated weights for policy 1, policy_version 973040 (0.0011) [2023-12-27 12:23:46,252][20347] Updated weights for policy 0, policy_version 973799 (0.0006) [2023-12-27 12:23:46,302][20347] Updated weights for policy 0, policy_version 973809 (0.0006) [2023-12-27 12:23:46,807][20381] Updated weights for policy 1, policy_version 973050 (0.0010) [2023-12-27 12:23:46,856][20381] Updated weights for policy 1, policy_version 973060 (0.0005) [2023-12-27 12:23:46,903][20347] Updated weights for policy 0, policy_version 973819 (0.0007) [2023-12-27 12:23:46,914][20381] Updated weights for policy 1, policy_version 973070 (0.0010) [2023-12-27 12:23:46,950][20347] Updated weights for policy 0, policy_version 973829 (0.0010) [2023-12-27 12:23:46,964][20381] Updated weights for policy 1, policy_version 973080 (0.0010) [2023-12-27 12:23:47,003][20347] Updated weights for policy 0, policy_version 973839 (0.0010) [2023-12-27 12:23:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 20327.3). Total num frames: 498483200. Throughput: 0: 10094.1, 1: 10110.0. Samples: 498492848. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:47,363][19170] Avg episode reward: [(0, '833.650'), (1, '663.780')] [2023-12-27 12:23:47,646][20381] Updated weights for policy 1, policy_version 973090 (0.0007) [2023-12-27 12:23:47,678][20347] Updated weights for policy 0, policy_version 973849 (0.0010) [2023-12-27 12:23:47,698][20381] Updated weights for policy 1, policy_version 973100 (0.0007) [2023-12-27 12:23:47,730][20347] Updated weights for policy 0, policy_version 973859 (0.0010) [2023-12-27 12:23:47,753][20381] Updated weights for policy 1, policy_version 973110 (0.0007) [2023-12-27 12:23:47,778][20347] Updated weights for policy 0, policy_version 973869 (0.0010) [2023-12-27 12:23:47,829][20347] Updated weights for policy 0, policy_version 973879 (0.0010) [2023-12-27 12:23:48,476][20381] Updated weights for policy 1, policy_version 973122 (0.0007) [2023-12-27 12:23:48,538][20381] Updated weights for policy 1, policy_version 973132 (0.0008) [2023-12-27 12:23:48,559][20347] Updated weights for policy 0, policy_version 973889 (0.0008) [2023-12-27 12:23:48,597][20381] Updated weights for policy 1, policy_version 973142 (0.0008) [2023-12-27 12:23:48,606][20347] Updated weights for policy 0, policy_version 973899 (0.0008) [2023-12-27 12:23:48,659][20347] Updated weights for policy 0, policy_version 973909 (0.0007) [2023-12-27 12:23:49,265][20381] Updated weights for policy 1, policy_version 973152 (0.0007) [2023-12-27 12:23:49,325][20381] Updated weights for policy 1, policy_version 973162 (0.0008) [2023-12-27 12:23:49,391][20381] Updated weights for policy 1, policy_version 973172 (0.0010) [2023-12-27 12:23:49,415][20347] Updated weights for policy 0, policy_version 973919 (0.0007) [2023-12-27 12:23:49,475][20347] Updated weights for policy 0, policy_version 973929 (0.0007) [2023-12-27 12:23:49,538][20347] Updated weights for policy 0, policy_version 973939 (0.0008) [2023-12-27 12:23:50,124][20381] Updated weights for policy 1, policy_version 973182 (0.0009) [2023-12-27 12:23:50,170][20347] Updated weights for policy 0, policy_version 973949 (0.0006) [2023-12-27 12:23:50,183][20381] Updated weights for policy 1, policy_version 973192 (0.0007) [2023-12-27 12:23:50,223][20347] Updated weights for policy 0, policy_version 973959 (0.0006) [2023-12-27 12:23:50,243][20381] Updated weights for policy 1, policy_version 973202 (0.0008) [2023-12-27 12:23:50,278][20347] Updated weights for policy 0, policy_version 973969 (0.0007) [2023-12-27 12:23:50,984][20381] Updated weights for policy 1, policy_version 973212 (0.0007) [2023-12-27 12:23:51,026][20347] Updated weights for policy 0, policy_version 973979 (0.0007) [2023-12-27 12:23:51,049][20381] Updated weights for policy 1, policy_version 973222 (0.0008) [2023-12-27 12:23:51,091][20347] Updated weights for policy 0, policy_version 973989 (0.0007) [2023-12-27 12:23:51,115][20381] Updated weights for policy 1, policy_version 973232 (0.0008) [2023-12-27 12:23:51,153][20347] Updated weights for policy 0, policy_version 973999 (0.0007) [2023-12-27 12:23:51,791][20381] Updated weights for policy 1, policy_version 973242 (0.0007) [2023-12-27 12:23:51,848][20381] Updated weights for policy 1, policy_version 973252 (0.0009) [2023-12-27 12:23:51,858][20347] Updated weights for policy 0, policy_version 974009 (0.0008) [2023-12-27 12:23:51,904][20381] Updated weights for policy 1, policy_version 973262 (0.0008) [2023-12-27 12:23:51,914][20347] Updated weights for policy 0, policy_version 974019 (0.0007) [2023-12-27 12:23:51,958][20381] Updated weights for policy 1, policy_version 973272 (0.0005) [2023-12-27 12:23:51,974][20347] Updated weights for policy 0, policy_version 974029 (0.0006) [2023-12-27 12:23:52,033][20347] Updated weights for policy 0, policy_version 974039 (0.0006) [2023-12-27 12:23:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20327.3). Total num frames: 498581504. Throughput: 0: 10097.9, 1: 10088.0. Samples: 498552540. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:52,363][19170] Avg episode reward: [(0, '795.560'), (1, '742.520')] [2023-12-27 12:23:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000973272_249184256.pth... [2023-12-27 12:23:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000974040_249397248.pth... [2023-12-27 12:23:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000972088_248881152.pth [2023-12-27 12:23:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000972856_249094144.pth [2023-12-27 12:23:52,648][20381] Updated weights for policy 1, policy_version 973282 (0.0008) [2023-12-27 12:23:52,703][20381] Updated weights for policy 1, policy_version 973292 (0.0007) [2023-12-27 12:23:52,714][20347] Updated weights for policy 0, policy_version 974049 (0.0007) [2023-12-27 12:23:52,755][20381] Updated weights for policy 1, policy_version 973302 (0.0006) [2023-12-27 12:23:52,776][20347] Updated weights for policy 0, policy_version 974059 (0.0006) [2023-12-27 12:23:52,835][20347] Updated weights for policy 0, policy_version 974069 (0.0007) [2023-12-27 12:23:53,446][20347] Updated weights for policy 0, policy_version 974079 (0.0008) [2023-12-27 12:23:53,486][20381] Updated weights for policy 1, policy_version 973312 (0.0006) [2023-12-27 12:23:53,495][20347] Updated weights for policy 0, policy_version 974089 (0.0010) [2023-12-27 12:23:53,542][20381] Updated weights for policy 1, policy_version 973322 (0.0006) [2023-12-27 12:23:53,550][20347] Updated weights for policy 0, policy_version 974099 (0.0010) [2023-12-27 12:23:53,592][20381] Updated weights for policy 1, policy_version 973332 (0.0009) [2023-12-27 12:23:54,253][20347] Updated weights for policy 0, policy_version 974109 (0.0009) [2023-12-27 12:23:54,317][20347] Updated weights for policy 0, policy_version 974119 (0.0007) [2023-12-27 12:23:54,355][20381] Updated weights for policy 1, policy_version 973342 (0.0009) [2023-12-27 12:23:54,386][20347] Updated weights for policy 0, policy_version 974129 (0.0009) [2023-12-27 12:23:54,419][20381] Updated weights for policy 1, policy_version 973352 (0.0007) [2023-12-27 12:23:54,481][20381] Updated weights for policy 1, policy_version 973362 (0.0007) [2023-12-27 12:23:55,112][20347] Updated weights for policy 0, policy_version 974139 (0.0011) [2023-12-27 12:23:55,169][20381] Updated weights for policy 1, policy_version 973372 (0.0009) [2023-12-27 12:23:55,177][20347] Updated weights for policy 0, policy_version 974149 (0.0011) [2023-12-27 12:23:55,234][20381] Updated weights for policy 1, policy_version 973382 (0.0011) [2023-12-27 12:23:55,243][20347] Updated weights for policy 0, policy_version 974159 (0.0011) [2023-12-27 12:23:55,300][20381] Updated weights for policy 1, policy_version 973392 (0.0008) [2023-12-27 12:23:55,955][20347] Updated weights for policy 0, policy_version 974169 (0.0011) [2023-12-27 12:23:55,956][20381] Updated weights for policy 1, policy_version 973402 (0.0007) [2023-12-27 12:23:56,006][20347] Updated weights for policy 0, policy_version 974179 (0.0009) [2023-12-27 12:23:56,020][20381] Updated weights for policy 1, policy_version 973412 (0.0007) [2023-12-27 12:23:56,063][20347] Updated weights for policy 0, policy_version 974189 (0.0006) [2023-12-27 12:23:56,072][20381] Updated weights for policy 1, policy_version 973422 (0.0008) [2023-12-27 12:23:56,115][20347] Updated weights for policy 0, policy_version 974199 (0.0006) [2023-12-27 12:23:56,125][20381] Updated weights for policy 1, policy_version 973432 (0.0008) [2023-12-27 12:23:56,722][20381] Updated weights for policy 1, policy_version 973442 (0.0007) [2023-12-27 12:23:56,776][20381] Updated weights for policy 1, policy_version 973452 (0.0008) [2023-12-27 12:23:56,835][20381] Updated weights for policy 1, policy_version 973462 (0.0007) [2023-12-27 12:23:56,883][20347] Updated weights for policy 0, policy_version 974209 (0.0007) [2023-12-27 12:23:56,942][20347] Updated weights for policy 0, policy_version 974219 (0.0008) [2023-12-27 12:23:56,993][20347] Updated weights for policy 0, policy_version 974229 (0.0009) [2023-12-27 12:23:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 20327.3). Total num frames: 498679808. Throughput: 0: 10106.9, 1: 10074.9. Samples: 498670888. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:23:57,363][19170] Avg episode reward: [(0, '873.540'), (1, '692.550')] [2023-12-27 12:23:57,536][20381] Updated weights for policy 1, policy_version 973472 (0.0008) [2023-12-27 12:23:57,598][20381] Updated weights for policy 1, policy_version 973482 (0.0009) [2023-12-27 12:23:57,654][20381] Updated weights for policy 1, policy_version 973492 (0.0007) [2023-12-27 12:23:57,718][20347] Updated weights for policy 0, policy_version 974239 (0.0009) [2023-12-27 12:23:57,770][20347] Updated weights for policy 0, policy_version 974249 (0.0009) [2023-12-27 12:23:57,848][20347] Updated weights for policy 0, policy_version 974259 (0.0009) [2023-12-27 12:23:58,369][20381] Updated weights for policy 1, policy_version 973503 (0.0009) [2023-12-27 12:23:58,441][20381] Updated weights for policy 1, policy_version 973513 (0.0013) [2023-12-27 12:23:58,513][20381] Updated weights for policy 1, policy_version 973523 (0.0010) [2023-12-27 12:23:58,908][20347] Updated weights for policy 0, policy_version 974269 (0.0008) [2023-12-27 12:23:59,018][20347] Updated weights for policy 0, policy_version 974280 (0.0012) [2023-12-27 12:23:59,096][20347] Updated weights for policy 0, policy_version 974290 (0.0010) [2023-12-27 12:23:59,525][20381] Updated weights for policy 1, policy_version 973533 (0.0009) [2023-12-27 12:23:59,581][20381] Updated weights for policy 1, policy_version 973543 (0.0008) [2023-12-27 12:23:59,641][20381] Updated weights for policy 1, policy_version 973553 (0.0008) [2023-12-27 12:24:00,086][20347] Updated weights for policy 0, policy_version 974300 (0.0010) [2023-12-27 12:24:00,171][20347] Updated weights for policy 0, policy_version 974310 (0.0010) [2023-12-27 12:24:00,255][20347] Updated weights for policy 0, policy_version 974320 (0.0008) [2023-12-27 12:24:00,431][20381] Updated weights for policy 1, policy_version 973563 (0.0008) [2023-12-27 12:24:00,494][20381] Updated weights for policy 1, policy_version 973573 (0.0007) [2023-12-27 12:24:00,558][20381] Updated weights for policy 1, policy_version 973583 (0.0008) [2023-12-27 12:24:01,049][20347] Updated weights for policy 0, policy_version 974330 (0.0008) [2023-12-27 12:24:01,130][20347] Updated weights for policy 0, policy_version 974340 (0.0007) [2023-12-27 12:24:01,221][20347] Updated weights for policy 0, policy_version 974350 (0.0010) [2023-12-27 12:24:01,274][20381] Updated weights for policy 1, policy_version 973593 (0.0008) [2023-12-27 12:24:01,305][20347] Updated weights for policy 0, policy_version 974360 (0.0008) [2023-12-27 12:24:01,330][20381] Updated weights for policy 1, policy_version 973603 (0.0008) [2023-12-27 12:24:01,405][20381] Updated weights for policy 1, policy_version 973613 (0.0008) [2023-12-27 12:24:01,466][20381] Updated weights for policy 1, policy_version 973623 (0.0008) [2023-12-27 12:24:02,103][20347] Updated weights for policy 0, policy_version 974370 (0.0008) [2023-12-27 12:24:02,160][20347] Updated weights for policy 0, policy_version 974380 (0.0006) [2023-12-27 12:24:02,164][20381] Updated weights for policy 1, policy_version 973633 (0.0006) [2023-12-27 12:24:02,224][20347] Updated weights for policy 0, policy_version 974390 (0.0008) [2023-12-27 12:24:02,227][20381] Updated weights for policy 1, policy_version 973643 (0.0006) [2023-12-27 12:24:02,284][20381] Updated weights for policy 1, policy_version 973653 (0.0008) [2023-12-27 12:24:02,362][19170] Fps is (10 sec: 18841.8, 60 sec: 20070.4, 300 sec: 20271.7). Total num frames: 498769920. Throughput: 0: 9843.6, 1: 10001.0. Samples: 498774488. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:02,363][19170] Avg episode reward: [(0, '883.320'), (1, '517.340')] [2023-12-27 12:24:02,981][20381] Updated weights for policy 1, policy_version 973663 (0.0007) [2023-12-27 12:24:03,009][20347] Updated weights for policy 0, policy_version 974400 (0.0007) [2023-12-27 12:24:03,038][20381] Updated weights for policy 1, policy_version 973673 (0.0007) [2023-12-27 12:24:03,055][20347] Updated weights for policy 0, policy_version 974410 (0.0007) [2023-12-27 12:24:03,090][20381] Updated weights for policy 1, policy_version 973683 (0.0006) [2023-12-27 12:24:03,111][20347] Updated weights for policy 0, policy_version 974420 (0.0005) [2023-12-27 12:24:03,732][20381] Updated weights for policy 1, policy_version 973693 (0.0008) [2023-12-27 12:24:03,790][20381] Updated weights for policy 1, policy_version 973703 (0.0007) [2023-12-27 12:24:03,826][20347] Updated weights for policy 0, policy_version 974430 (0.0008) [2023-12-27 12:24:03,849][20381] Updated weights for policy 1, policy_version 973713 (0.0007) [2023-12-27 12:24:03,891][20347] Updated weights for policy 0, policy_version 974440 (0.0008) [2023-12-27 12:24:03,950][20347] Updated weights for policy 0, policy_version 974450 (0.0007) [2023-12-27 12:24:04,540][20381] Updated weights for policy 1, policy_version 973723 (0.0006) [2023-12-27 12:24:04,598][20347] Updated weights for policy 0, policy_version 974460 (0.0008) [2023-12-27 12:24:04,603][20381] Updated weights for policy 1, policy_version 973733 (0.0006) [2023-12-27 12:24:04,652][20347] Updated weights for policy 0, policy_version 974470 (0.0008) [2023-12-27 12:24:04,668][20381] Updated weights for policy 1, policy_version 973743 (0.0007) [2023-12-27 12:24:04,704][20347] Updated weights for policy 0, policy_version 974480 (0.0007) [2023-12-27 12:24:05,256][20381] Updated weights for policy 1, policy_version 973753 (0.0007) [2023-12-27 12:24:05,319][20381] Updated weights for policy 1, policy_version 973763 (0.0008) [2023-12-27 12:24:05,363][20347] Updated weights for policy 0, policy_version 974490 (0.0006) [2023-12-27 12:24:05,368][20381] Updated weights for policy 1, policy_version 973773 (0.0009) [2023-12-27 12:24:05,429][20381] Updated weights for policy 1, policy_version 973783 (0.0008) [2023-12-27 12:24:05,439][20347] Updated weights for policy 0, policy_version 974500 (0.0008) [2023-12-27 12:24:05,514][20347] Updated weights for policy 0, policy_version 974510 (0.0007) [2023-12-27 12:24:05,577][20347] Updated weights for policy 0, policy_version 974520 (0.0008) [2023-12-27 12:24:06,134][20381] Updated weights for policy 1, policy_version 973793 (0.0009) [2023-12-27 12:24:06,198][20381] Updated weights for policy 1, policy_version 973803 (0.0008) [2023-12-27 12:24:06,256][20381] Updated weights for policy 1, policy_version 973813 (0.0006) [2023-12-27 12:24:06,263][20347] Updated weights for policy 0, policy_version 974530 (0.0008) [2023-12-27 12:24:06,333][20347] Updated weights for policy 0, policy_version 974540 (0.0009) [2023-12-27 12:24:06,408][20347] Updated weights for policy 0, policy_version 974550 (0.0009) [2023-12-27 12:24:06,982][20381] Updated weights for policy 1, policy_version 973823 (0.0007) [2023-12-27 12:24:07,031][20381] Updated weights for policy 1, policy_version 973833 (0.0008) [2023-12-27 12:24:07,080][20381] Updated weights for policy 1, policy_version 973843 (0.0007) [2023-12-27 12:24:07,224][20347] Updated weights for policy 0, policy_version 974560 (0.0011) [2023-12-27 12:24:07,288][20347] Updated weights for policy 0, policy_version 974570 (0.0010) [2023-12-27 12:24:07,348][20347] Updated weights for policy 0, policy_version 974580 (0.0010) [2023-12-27 12:24:07,362][19170] Fps is (10 sec: 18022.2, 60 sec: 19660.8, 300 sec: 20244.0). Total num frames: 498860032. Throughput: 0: 9813.8, 1: 10012.0. Samples: 498834988. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:07,363][19170] Avg episode reward: [(0, '941.420'), (1, '598.400')] [2023-12-27 12:24:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000973848_249331712.pth... [2023-12-27 12:24:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000974584_249536512.pth... [2023-12-27 12:24:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000972696_249036800.pth [2023-12-27 12:24:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000973464_249249792.pth [2023-12-27 12:24:07,761][20381] Updated weights for policy 1, policy_version 973853 (0.0005) [2023-12-27 12:24:07,824][20381] Updated weights for policy 1, policy_version 973863 (0.0006) [2023-12-27 12:24:07,880][20381] Updated weights for policy 1, policy_version 973873 (0.0006) [2023-12-27 12:24:08,077][20347] Updated weights for policy 0, policy_version 974590 (0.0011) [2023-12-27 12:24:08,136][20347] Updated weights for policy 0, policy_version 974600 (0.0010) [2023-12-27 12:24:08,191][20347] Updated weights for policy 0, policy_version 974610 (0.0010) [2023-12-27 12:24:08,534][20381] Updated weights for policy 1, policy_version 973883 (0.0007) [2023-12-27 12:24:08,580][20381] Updated weights for policy 1, policy_version 973893 (0.0008) [2023-12-27 12:24:08,628][20381] Updated weights for policy 1, policy_version 973903 (0.0007) [2023-12-27 12:24:09,047][20347] Updated weights for policy 0, policy_version 974620 (0.0010) [2023-12-27 12:24:09,181][20347] Updated weights for policy 0, policy_version 974634 (0.0034) [2023-12-27 12:24:09,250][20347] Updated weights for policy 0, policy_version 974644 (0.0008) [2023-12-27 12:24:09,324][20381] Updated weights for policy 1, policy_version 973913 (0.0007) [2023-12-27 12:24:09,392][20381] Updated weights for policy 1, policy_version 973923 (0.0009) [2023-12-27 12:24:09,457][20381] Updated weights for policy 1, policy_version 973933 (0.0008) [2023-12-27 12:24:09,518][20381] Updated weights for policy 1, policy_version 973943 (0.0009) [2023-12-27 12:24:10,031][20347] Updated weights for policy 0, policy_version 974654 (0.0008) [2023-12-27 12:24:10,081][20347] Updated weights for policy 0, policy_version 974664 (0.0007) [2023-12-27 12:24:10,141][20347] Updated weights for policy 0, policy_version 974674 (0.0006) [2023-12-27 12:24:10,308][20381] Updated weights for policy 1, policy_version 973953 (0.0007) [2023-12-27 12:24:10,376][20381] Updated weights for policy 1, policy_version 973963 (0.0006) [2023-12-27 12:24:10,439][20381] Updated weights for policy 1, policy_version 973973 (0.0007) [2023-12-27 12:24:10,963][20347] Updated weights for policy 0, policy_version 974684 (0.0007) [2023-12-27 12:24:11,018][20347] Updated weights for policy 0, policy_version 974694 (0.0006) [2023-12-27 12:24:11,092][20347] Updated weights for policy 0, policy_version 974706 (0.0007) [2023-12-27 12:24:11,185][20381] Updated weights for policy 1, policy_version 973983 (0.0008) [2023-12-27 12:24:11,250][20381] Updated weights for policy 1, policy_version 973993 (0.0008) [2023-12-27 12:24:11,312][20381] Updated weights for policy 1, policy_version 974003 (0.0008) [2023-12-27 12:24:11,801][20347] Updated weights for policy 0, policy_version 974716 (0.0007) [2023-12-27 12:24:11,861][20347] Updated weights for policy 0, policy_version 974726 (0.0008) [2023-12-27 12:24:11,921][20347] Updated weights for policy 0, policy_version 974736 (0.0007) [2023-12-27 12:24:12,105][20381] Updated weights for policy 1, policy_version 974013 (0.0008) [2023-12-27 12:24:12,158][20381] Updated weights for policy 1, policy_version 974023 (0.0008) [2023-12-27 12:24:12,214][20381] Updated weights for policy 1, policy_version 974033 (0.0008) [2023-12-27 12:24:12,362][19170] Fps is (10 sec: 18841.7, 60 sec: 19797.4, 300 sec: 20244.0). Total num frames: 498958336. Throughput: 0: 9611.0, 1: 9964.4. Samples: 498947768. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:12,363][19170] Avg episode reward: [(0, '912.280'), (1, '562.230')] [2023-12-27 12:24:12,535][20347] Updated weights for policy 0, policy_version 974746 (0.0007) [2023-12-27 12:24:12,615][20347] Updated weights for policy 0, policy_version 974756 (0.0009) [2023-12-27 12:24:12,707][20347] Updated weights for policy 0, policy_version 974766 (0.0008) [2023-12-27 12:24:12,784][20347] Updated weights for policy 0, policy_version 974776 (0.0009) [2023-12-27 12:24:12,935][20381] Updated weights for policy 1, policy_version 974043 (0.0009) [2023-12-27 12:24:12,991][20381] Updated weights for policy 1, policy_version 974053 (0.0007) [2023-12-27 12:24:13,053][20381] Updated weights for policy 1, policy_version 974063 (0.0007) [2023-12-27 12:24:13,416][20347] Updated weights for policy 0, policy_version 974786 (0.0008) [2023-12-27 12:24:13,473][20347] Updated weights for policy 0, policy_version 974796 (0.0008) [2023-12-27 12:24:13,531][20347] Updated weights for policy 0, policy_version 974806 (0.0009) [2023-12-27 12:24:13,758][20381] Updated weights for policy 1, policy_version 974073 (0.0007) [2023-12-27 12:24:13,813][20381] Updated weights for policy 1, policy_version 974083 (0.0006) [2023-12-27 12:24:13,872][20381] Updated weights for policy 1, policy_version 974093 (0.0007) [2023-12-27 12:24:13,925][20381] Updated weights for policy 1, policy_version 974103 (0.0007) [2023-12-27 12:24:14,262][20347] Updated weights for policy 0, policy_version 974816 (0.0009) [2023-12-27 12:24:14,312][20347] Updated weights for policy 0, policy_version 974826 (0.0007) [2023-12-27 12:24:14,360][20347] Updated weights for policy 0, policy_version 974836 (0.0005) [2023-12-27 12:24:14,613][20381] Updated weights for policy 1, policy_version 974113 (0.0006) [2023-12-27 12:24:14,687][20381] Updated weights for policy 1, policy_version 974123 (0.0006) [2023-12-27 12:24:14,731][20381] Updated weights for policy 1, policy_version 974133 (0.0007) [2023-12-27 12:24:15,221][20347] Updated weights for policy 0, policy_version 974846 (0.0009) [2023-12-27 12:24:15,288][20347] Updated weights for policy 0, policy_version 974856 (0.0008) [2023-12-27 12:24:15,344][20347] Updated weights for policy 0, policy_version 974866 (0.0008) [2023-12-27 12:24:15,432][20381] Updated weights for policy 1, policy_version 974143 (0.0007) [2023-12-27 12:24:15,485][20381] Updated weights for policy 1, policy_version 974153 (0.0008) [2023-12-27 12:24:15,542][20381] Updated weights for policy 1, policy_version 974163 (0.0008) [2023-12-27 12:24:16,158][20381] Updated weights for policy 1, policy_version 974173 (0.0008) [2023-12-27 12:24:16,199][20347] Updated weights for policy 0, policy_version 974876 (0.0008) [2023-12-27 12:24:16,218][20381] Updated weights for policy 1, policy_version 974183 (0.0007) [2023-12-27 12:24:16,283][20381] Updated weights for policy 1, policy_version 974193 (0.0007) [2023-12-27 12:24:16,321][20347] Updated weights for policy 0, policy_version 974889 (0.0032) [2023-12-27 12:24:16,392][20347] Updated weights for policy 0, policy_version 974899 (0.0010) [2023-12-27 12:24:16,964][20381] Updated weights for policy 1, policy_version 974203 (0.0008) [2023-12-27 12:24:17,024][20381] Updated weights for policy 1, policy_version 974213 (0.0008) [2023-12-27 12:24:17,085][20381] Updated weights for policy 1, policy_version 974223 (0.0007) [2023-12-27 12:24:17,102][20347] Updated weights for policy 0, policy_version 974909 (0.0009) [2023-12-27 12:24:17,158][20347] Updated weights for policy 0, policy_version 974919 (0.0010) [2023-12-27 12:24:17,212][20347] Updated weights for policy 0, policy_version 974929 (0.0008) [2023-12-27 12:24:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 19660.8, 300 sec: 20271.7). Total num frames: 499056640. Throughput: 0: 9437.4, 1: 9929.1. Samples: 499063352. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:17,364][19170] Avg episode reward: [(0, '902.650'), (1, '653.150')] [2023-12-27 12:24:17,664][20381] Updated weights for policy 1, policy_version 974233 (0.0006) [2023-12-27 12:24:17,719][20381] Updated weights for policy 1, policy_version 974243 (0.0010) [2023-12-27 12:24:17,774][20381] Updated weights for policy 1, policy_version 974253 (0.0010) [2023-12-27 12:24:17,832][20381] Updated weights for policy 1, policy_version 974263 (0.0010) [2023-12-27 12:24:17,855][20347] Updated weights for policy 0, policy_version 974939 (0.0009) [2023-12-27 12:24:17,919][20347] Updated weights for policy 0, policy_version 974949 (0.0010) [2023-12-27 12:24:17,984][20347] Updated weights for policy 0, policy_version 974959 (0.0010) [2023-12-27 12:24:18,497][20381] Updated weights for policy 1, policy_version 974273 (0.0007) [2023-12-27 12:24:18,550][20381] Updated weights for policy 1, policy_version 974283 (0.0011) [2023-12-27 12:24:18,601][20381] Updated weights for policy 1, policy_version 974293 (0.0010) [2023-12-27 12:24:18,646][20347] Updated weights for policy 0, policy_version 974969 (0.0010) [2023-12-27 12:24:18,703][20347] Updated weights for policy 0, policy_version 974979 (0.0010) [2023-12-27 12:24:18,763][20347] Updated weights for policy 0, policy_version 974989 (0.0010) [2023-12-27 12:24:18,816][20347] Updated weights for policy 0, policy_version 974999 (0.0010) [2023-12-27 12:24:19,307][20381] Updated weights for policy 1, policy_version 974303 (0.0009) [2023-12-27 12:24:19,359][20381] Updated weights for policy 1, policy_version 974313 (0.0008) [2023-12-27 12:24:19,420][20381] Updated weights for policy 1, policy_version 974323 (0.0008) [2023-12-27 12:24:19,520][20347] Updated weights for policy 0, policy_version 975009 (0.0009) [2023-12-27 12:24:19,582][20347] Updated weights for policy 0, policy_version 975019 (0.0008) [2023-12-27 12:24:19,634][20347] Updated weights for policy 0, policy_version 975029 (0.0008) [2023-12-27 12:24:20,155][20381] Updated weights for policy 1, policy_version 974333 (0.0007) [2023-12-27 12:24:20,212][20381] Updated weights for policy 1, policy_version 974343 (0.0006) [2023-12-27 12:24:20,273][20381] Updated weights for policy 1, policy_version 974353 (0.0008) [2023-12-27 12:24:20,395][20347] Updated weights for policy 0, policy_version 975039 (0.0008) [2023-12-27 12:24:20,453][20347] Updated weights for policy 0, policy_version 975049 (0.0007) [2023-12-27 12:24:20,512][20347] Updated weights for policy 0, policy_version 975059 (0.0005) [2023-12-27 12:24:20,976][20381] Updated weights for policy 1, policy_version 974363 (0.0008) [2023-12-27 12:24:21,035][20381] Updated weights for policy 1, policy_version 974373 (0.0007) [2023-12-27 12:24:21,099][20381] Updated weights for policy 1, policy_version 974383 (0.0008) [2023-12-27 12:24:21,193][20347] Updated weights for policy 0, policy_version 975069 (0.0006) [2023-12-27 12:24:21,260][20347] Updated weights for policy 0, policy_version 975079 (0.0008) [2023-12-27 12:24:21,330][20347] Updated weights for policy 0, policy_version 975089 (0.0008) [2023-12-27 12:24:21,846][20381] Updated weights for policy 1, policy_version 974393 (0.0007) [2023-12-27 12:24:21,915][20381] Updated weights for policy 1, policy_version 974403 (0.0006) [2023-12-27 12:24:21,967][20381] Updated weights for policy 1, policy_version 974413 (0.0007) [2023-12-27 12:24:22,020][20381] Updated weights for policy 1, policy_version 974423 (0.0007) [2023-12-27 12:24:22,097][20347] Updated weights for policy 0, policy_version 975099 (0.0008) [2023-12-27 12:24:22,159][20347] Updated weights for policy 0, policy_version 975109 (0.0007) [2023-12-27 12:24:22,221][20347] Updated weights for policy 0, policy_version 975119 (0.0009) [2023-12-27 12:24:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19661.2, 300 sec: 20244.0). Total num frames: 499154944. Throughput: 0: 9430.4, 1: 9924.3. Samples: 499124596. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:22,363][19170] Avg episode reward: [(0, '902.580'), (1, '836.620')] [2023-12-27 12:24:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000974424_249479168.pth... [2023-12-27 12:24:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000975128_249675776.pth... [2023-12-27 12:24:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000973272_249184256.pth [2023-12-27 12:24:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000974040_249397248.pth [2023-12-27 12:24:22,700][20381] Updated weights for policy 1, policy_version 974433 (0.0007) [2023-12-27 12:24:22,757][20381] Updated weights for policy 1, policy_version 974443 (0.0008) [2023-12-27 12:24:22,815][20381] Updated weights for policy 1, policy_version 974453 (0.0008) [2023-12-27 12:24:22,942][20347] Updated weights for policy 0, policy_version 975129 (0.0010) [2023-12-27 12:24:23,009][20347] Updated weights for policy 0, policy_version 975139 (0.0007) [2023-12-27 12:24:23,072][20347] Updated weights for policy 0, policy_version 975149 (0.0008) [2023-12-27 12:24:23,134][20347] Updated weights for policy 0, policy_version 975159 (0.0008) [2023-12-27 12:24:23,513][20381] Updated weights for policy 1, policy_version 974463 (0.0008) [2023-12-27 12:24:23,566][20381] Updated weights for policy 1, policy_version 974473 (0.0008) [2023-12-27 12:24:23,637][20381] Updated weights for policy 1, policy_version 974483 (0.0008) [2023-12-27 12:24:23,817][20347] Updated weights for policy 0, policy_version 975169 (0.0008) [2023-12-27 12:24:23,876][20347] Updated weights for policy 0, policy_version 975179 (0.0008) [2023-12-27 12:24:23,931][20347] Updated weights for policy 0, policy_version 975189 (0.0007) [2023-12-27 12:24:24,307][20381] Updated weights for policy 1, policy_version 974493 (0.0009) [2023-12-27 12:24:24,359][20381] Updated weights for policy 1, policy_version 974503 (0.0011) [2023-12-27 12:24:24,413][20381] Updated weights for policy 1, policy_version 974513 (0.0011) [2023-12-27 12:24:24,577][20347] Updated weights for policy 0, policy_version 975199 (0.0007) [2023-12-27 12:24:24,631][20347] Updated weights for policy 0, policy_version 975209 (0.0008) [2023-12-27 12:24:24,676][20347] Updated weights for policy 0, policy_version 975219 (0.0007) [2023-12-27 12:24:25,072][20381] Updated weights for policy 1, policy_version 974523 (0.0009) [2023-12-27 12:24:25,125][20381] Updated weights for policy 1, policy_version 974533 (0.0005) [2023-12-27 12:24:25,184][20381] Updated weights for policy 1, policy_version 974543 (0.0010) [2023-12-27 12:24:25,250][20347] Updated weights for policy 0, policy_version 975229 (0.0006) [2023-12-27 12:24:25,310][20347] Updated weights for policy 0, policy_version 975239 (0.0007) [2023-12-27 12:24:25,364][20347] Updated weights for policy 0, policy_version 975249 (0.0007) [2023-12-27 12:24:25,840][20381] Updated weights for policy 1, policy_version 974553 (0.0011) [2023-12-27 12:24:25,894][20381] Updated weights for policy 1, policy_version 974563 (0.0010) [2023-12-27 12:24:25,943][20381] Updated weights for policy 1, policy_version 974573 (0.0010) [2023-12-27 12:24:25,970][20347] Updated weights for policy 0, policy_version 975259 (0.0006) [2023-12-27 12:24:25,994][20381] Updated weights for policy 1, policy_version 974583 (0.0011) [2023-12-27 12:24:26,028][20347] Updated weights for policy 0, policy_version 975269 (0.0005) [2023-12-27 12:24:26,088][20347] Updated weights for policy 0, policy_version 975279 (0.0006) [2023-12-27 12:24:26,678][20347] Updated weights for policy 0, policy_version 975289 (0.0010) [2023-12-27 12:24:26,682][20381] Updated weights for policy 1, policy_version 974593 (0.0010) [2023-12-27 12:24:26,729][20347] Updated weights for policy 0, policy_version 975299 (0.0010) [2023-12-27 12:24:26,730][20381] Updated weights for policy 1, policy_version 974603 (0.0010) [2023-12-27 12:24:26,778][20381] Updated weights for policy 1, policy_version 974613 (0.0010) [2023-12-27 12:24:26,781][20347] Updated weights for policy 0, policy_version 975309 (0.0010) [2023-12-27 12:24:26,832][20347] Updated weights for policy 0, policy_version 975319 (0.0010) [2023-12-27 12:24:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 19660.8, 300 sec: 20244.0). Total num frames: 499253248. Throughput: 0: 9501.2, 1: 9917.6. Samples: 499246404. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:27,363][19170] Avg episode reward: [(0, '941.530'), (1, '818.180')] [2023-12-27 12:24:27,468][20381] Updated weights for policy 1, policy_version 974623 (0.0010) [2023-12-27 12:24:27,495][20347] Updated weights for policy 0, policy_version 975329 (0.0010) [2023-12-27 12:24:27,522][20381] Updated weights for policy 1, policy_version 974633 (0.0010) [2023-12-27 12:24:27,550][20347] Updated weights for policy 0, policy_version 975339 (0.0010) [2023-12-27 12:24:27,577][20381] Updated weights for policy 1, policy_version 974643 (0.0010) [2023-12-27 12:24:27,601][20347] Updated weights for policy 0, policy_version 975349 (0.0010) [2023-12-27 12:24:28,184][20347] Updated weights for policy 0, policy_version 975359 (0.0007) [2023-12-27 12:24:28,194][20381] Updated weights for policy 1, policy_version 974653 (0.0010) [2023-12-27 12:24:28,236][20347] Updated weights for policy 0, policy_version 975369 (0.0005) [2023-12-27 12:24:28,239][20381] Updated weights for policy 1, policy_version 974663 (0.0010) [2023-12-27 12:24:28,282][20347] Updated weights for policy 0, policy_version 975379 (0.0005) [2023-12-27 12:24:28,284][20381] Updated weights for policy 1, policy_version 974673 (0.0010) [2023-12-27 12:24:28,921][20347] Updated weights for policy 0, policy_version 975389 (0.0008) [2023-12-27 12:24:28,980][20347] Updated weights for policy 0, policy_version 975399 (0.0010) [2023-12-27 12:24:28,992][20381] Updated weights for policy 1, policy_version 974683 (0.0010) [2023-12-27 12:24:29,034][20347] Updated weights for policy 0, policy_version 975409 (0.0009) [2023-12-27 12:24:29,045][20381] Updated weights for policy 1, policy_version 974693 (0.0010) [2023-12-27 12:24:29,096][20381] Updated weights for policy 1, policy_version 974703 (0.0010) [2023-12-27 12:24:29,739][20347] Updated weights for policy 0, policy_version 975419 (0.0009) [2023-12-27 12:24:29,759][20381] Updated weights for policy 1, policy_version 974713 (0.0008) [2023-12-27 12:24:29,794][20347] Updated weights for policy 0, policy_version 975429 (0.0008) [2023-12-27 12:24:29,814][20381] Updated weights for policy 1, policy_version 974723 (0.0007) [2023-12-27 12:24:29,856][20347] Updated weights for policy 0, policy_version 975439 (0.0010) [2023-12-27 12:24:29,875][20381] Updated weights for policy 1, policy_version 974733 (0.0009) [2023-12-27 12:24:29,943][20381] Updated weights for policy 1, policy_version 974743 (0.0010) [2023-12-27 12:24:30,516][20347] Updated weights for policy 0, policy_version 975449 (0.0008) [2023-12-27 12:24:30,571][20347] Updated weights for policy 0, policy_version 975459 (0.0010) [2023-12-27 12:24:30,582][20381] Updated weights for policy 1, policy_version 974753 (0.0006) [2023-12-27 12:24:30,622][20347] Updated weights for policy 0, policy_version 975469 (0.0010) [2023-12-27 12:24:30,627][20381] Updated weights for policy 1, policy_version 974763 (0.0008) [2023-12-27 12:24:30,669][20381] Updated weights for policy 1, policy_version 974773 (0.0005) [2023-12-27 12:24:30,670][20347] Updated weights for policy 0, policy_version 975479 (0.0010) [2023-12-27 12:24:31,319][20347] Updated weights for policy 0, policy_version 975489 (0.0008) [2023-12-27 12:24:31,320][20381] Updated weights for policy 1, policy_version 974783 (0.0007) [2023-12-27 12:24:31,382][20347] Updated weights for policy 0, policy_version 975499 (0.0006) [2023-12-27 12:24:31,382][20381] Updated weights for policy 1, policy_version 974793 (0.0008) [2023-12-27 12:24:31,446][20381] Updated weights for policy 1, policy_version 974803 (0.0006) [2023-12-27 12:24:31,451][20347] Updated weights for policy 0, policy_version 975509 (0.0007) [2023-12-27 12:24:32,124][20381] Updated weights for policy 1, policy_version 974813 (0.0006) [2023-12-27 12:24:32,176][20381] Updated weights for policy 1, policy_version 974823 (0.0006) [2023-12-27 12:24:32,195][20347] Updated weights for policy 0, policy_version 975519 (0.0007) [2023-12-27 12:24:32,229][20381] Updated weights for policy 1, policy_version 974833 (0.0007) [2023-12-27 12:24:32,253][20347] Updated weights for policy 0, policy_version 975529 (0.0006) [2023-12-27 12:24:32,316][20347] Updated weights for policy 0, policy_version 975539 (0.0007) [2023-12-27 12:24:32,364][19170] Fps is (10 sec: 21295.6, 60 sec: 19796.8, 300 sec: 20299.4). Total num frames: 499367936. Throughput: 0: 9584.4, 1: 9945.9. Samples: 499371748. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:32,365][19170] Avg episode reward: [(0, '961.250'), (1, '873.670')] [2023-12-27 12:24:32,877][20381] Updated weights for policy 1, policy_version 974843 (0.0007) [2023-12-27 12:24:32,937][20381] Updated weights for policy 1, policy_version 974853 (0.0005) [2023-12-27 12:24:33,003][20381] Updated weights for policy 1, policy_version 974863 (0.0005) [2023-12-27 12:24:33,036][20347] Updated weights for policy 0, policy_version 975549 (0.0006) [2023-12-27 12:24:33,082][20347] Updated weights for policy 0, policy_version 975559 (0.0007) [2023-12-27 12:24:33,126][20347] Updated weights for policy 0, policy_version 975569 (0.0007) [2023-12-27 12:24:33,535][20381] Updated weights for policy 1, policy_version 974873 (0.0006) [2023-12-27 12:24:33,584][20381] Updated weights for policy 1, policy_version 974883 (0.0005) [2023-12-27 12:24:33,631][20381] Updated weights for policy 1, policy_version 974893 (0.0005) [2023-12-27 12:24:33,684][20381] Updated weights for policy 1, policy_version 974903 (0.0005) [2023-12-27 12:24:33,773][20347] Updated weights for policy 0, policy_version 975579 (0.0007) [2023-12-27 12:24:33,830][20347] Updated weights for policy 0, policy_version 975589 (0.0009) [2023-12-27 12:24:33,880][20347] Updated weights for policy 0, policy_version 975599 (0.0008) [2023-12-27 12:24:34,296][20381] Updated weights for policy 1, policy_version 974913 (0.0008) [2023-12-27 12:24:34,348][20381] Updated weights for policy 1, policy_version 974923 (0.0007) [2023-12-27 12:24:34,397][20381] Updated weights for policy 1, policy_version 974933 (0.0007) [2023-12-27 12:24:34,496][20347] Updated weights for policy 0, policy_version 975609 (0.0010) [2023-12-27 12:24:34,560][20347] Updated weights for policy 0, policy_version 975619 (0.0011) [2023-12-27 12:24:34,623][20347] Updated weights for policy 0, policy_version 975629 (0.0007) [2023-12-27 12:24:34,687][20347] Updated weights for policy 0, policy_version 975639 (0.0007) [2023-12-27 12:24:35,099][20381] Updated weights for policy 1, policy_version 974943 (0.0007) [2023-12-27 12:24:35,150][20381] Updated weights for policy 1, policy_version 974953 (0.0007) [2023-12-27 12:24:35,206][20381] Updated weights for policy 1, policy_version 974963 (0.0008) [2023-12-27 12:24:35,309][20347] Updated weights for policy 0, policy_version 975649 (0.0006) [2023-12-27 12:24:35,370][20347] Updated weights for policy 0, policy_version 975659 (0.0005) [2023-12-27 12:24:35,425][20347] Updated weights for policy 0, policy_version 975669 (0.0005) [2023-12-27 12:24:35,922][20347] Updated weights for policy 0, policy_version 975679 (0.0005) [2023-12-27 12:24:35,983][20347] Updated weights for policy 0, policy_version 975689 (0.0006) [2023-12-27 12:24:35,989][20381] Updated weights for policy 1, policy_version 974973 (0.0007) [2023-12-27 12:24:36,043][20381] Updated weights for policy 1, policy_version 974983 (0.0007) [2023-12-27 12:24:36,045][20347] Updated weights for policy 0, policy_version 975699 (0.0007) [2023-12-27 12:24:36,111][20381] Updated weights for policy 1, policy_version 974994 (0.0008) [2023-12-27 12:24:36,716][20347] Updated weights for policy 0, policy_version 975709 (0.0007) [2023-12-27 12:24:36,760][20347] Updated weights for policy 0, policy_version 975719 (0.0007) [2023-12-27 12:24:36,810][20347] Updated weights for policy 0, policy_version 975729 (0.0007) [2023-12-27 12:24:36,869][20381] Updated weights for policy 1, policy_version 975004 (0.0009) [2023-12-27 12:24:36,926][20381] Updated weights for policy 1, policy_version 975014 (0.0011) [2023-12-27 12:24:36,978][20381] Updated weights for policy 1, policy_version 975024 (0.0011) [2023-12-27 12:24:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 19660.8, 300 sec: 20299.5). Total num frames: 499466240. Throughput: 0: 9611.9, 1: 10021.7. Samples: 499436052. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:37,363][19170] Avg episode reward: [(0, '970.790'), (1, '902.990')] [2023-12-27 12:24:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000975032_249634816.pth... [2023-12-27 12:24:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000975736_249831424.pth... [2023-12-27 12:24:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000973848_249331712.pth [2023-12-27 12:24:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000974584_249536512.pth [2023-12-27 12:24:37,477][20347] Updated weights for policy 0, policy_version 975739 (0.0007) [2023-12-27 12:24:37,538][20347] Updated weights for policy 0, policy_version 975749 (0.0005) [2023-12-27 12:24:37,607][20347] Updated weights for policy 0, policy_version 975759 (0.0006) [2023-12-27 12:24:37,666][20381] Updated weights for policy 1, policy_version 975034 (0.0009) [2023-12-27 12:24:37,729][20381] Updated weights for policy 1, policy_version 975044 (0.0008) [2023-12-27 12:24:37,791][20381] Updated weights for policy 1, policy_version 975054 (0.0008) [2023-12-27 12:24:37,853][20381] Updated weights for policy 1, policy_version 975064 (0.0008) [2023-12-27 12:24:38,234][20347] Updated weights for policy 0, policy_version 975769 (0.0007) [2023-12-27 12:24:38,284][20347] Updated weights for policy 0, policy_version 975779 (0.0007) [2023-12-27 12:24:38,341][20347] Updated weights for policy 0, policy_version 975789 (0.0007) [2023-12-27 12:24:38,400][20347] Updated weights for policy 0, policy_version 975799 (0.0007) [2023-12-27 12:24:38,489][20381] Updated weights for policy 1, policy_version 975074 (0.0011) [2023-12-27 12:24:38,552][20381] Updated weights for policy 1, policy_version 975084 (0.0011) [2023-12-27 12:24:38,615][20381] Updated weights for policy 1, policy_version 975094 (0.0011) [2023-12-27 12:24:39,132][20347] Updated weights for policy 0, policy_version 975809 (0.0007) [2023-12-27 12:24:39,192][20347] Updated weights for policy 0, policy_version 975819 (0.0007) [2023-12-27 12:24:39,260][20347] Updated weights for policy 0, policy_version 975829 (0.0007) [2023-12-27 12:24:39,273][20381] Updated weights for policy 1, policy_version 975104 (0.0010) [2023-12-27 12:24:39,330][20381] Updated weights for policy 1, policy_version 975114 (0.0011) [2023-12-27 12:24:39,401][20381] Updated weights for policy 1, policy_version 975124 (0.0011) [2023-12-27 12:24:40,060][20347] Updated weights for policy 0, policy_version 975839 (0.0008) [2023-12-27 12:24:40,109][20347] Updated weights for policy 0, policy_version 975849 (0.0009) [2023-12-27 12:24:40,162][20347] Updated weights for policy 0, policy_version 975859 (0.0009) [2023-12-27 12:24:40,297][20381] Updated weights for policy 1, policy_version 975134 (0.0008) [2023-12-27 12:24:40,362][20381] Updated weights for policy 1, policy_version 975144 (0.0007) [2023-12-27 12:24:40,428][20381] Updated weights for policy 1, policy_version 975154 (0.0008) [2023-12-27 12:24:40,799][20347] Updated weights for policy 0, policy_version 975869 (0.0009) [2023-12-27 12:24:40,848][20347] Updated weights for policy 0, policy_version 975879 (0.0009) [2023-12-27 12:24:40,900][20347] Updated weights for policy 0, policy_version 975889 (0.0007) [2023-12-27 12:24:41,012][20381] Updated weights for policy 1, policy_version 975164 (0.0006) [2023-12-27 12:24:41,077][20381] Updated weights for policy 1, policy_version 975174 (0.0009) [2023-12-27 12:24:41,130][20381] Updated weights for policy 1, policy_version 975184 (0.0007) [2023-12-27 12:24:41,718][20347] Updated weights for policy 0, policy_version 975899 (0.0007) [2023-12-27 12:24:41,781][20347] Updated weights for policy 0, policy_version 975909 (0.0006) [2023-12-27 12:24:41,844][20347] Updated weights for policy 0, policy_version 975919 (0.0006) [2023-12-27 12:24:41,876][20381] Updated weights for policy 1, policy_version 975194 (0.0009) [2023-12-27 12:24:41,935][20381] Updated weights for policy 1, policy_version 975204 (0.0011) [2023-12-27 12:24:41,991][20381] Updated weights for policy 1, policy_version 975214 (0.0008) [2023-12-27 12:24:42,045][20381] Updated weights for policy 1, policy_version 975224 (0.0007) [2023-12-27 12:24:42,362][19170] Fps is (10 sec: 19664.0, 60 sec: 19660.8, 300 sec: 20299.5). Total num frames: 499564544. Throughput: 0: 9667.3, 1: 9991.4. Samples: 499555528. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:42,363][19170] Avg episode reward: [(0, '961.370'), (1, '873.630')] [2023-12-27 12:24:42,482][20347] Updated weights for policy 0, policy_version 975929 (0.0006) [2023-12-27 12:24:42,540][20347] Updated weights for policy 0, policy_version 975939 (0.0005) [2023-12-27 12:24:42,599][20347] Updated weights for policy 0, policy_version 975949 (0.0007) [2023-12-27 12:24:42,655][20347] Updated weights for policy 0, policy_version 975959 (0.0008) [2023-12-27 12:24:42,704][20381] Updated weights for policy 1, policy_version 975234 (0.0005) [2023-12-27 12:24:42,760][20381] Updated weights for policy 1, policy_version 975244 (0.0005) [2023-12-27 12:24:42,822][20381] Updated weights for policy 1, policy_version 975254 (0.0006) [2023-12-27 12:24:43,357][20381] Updated weights for policy 1, policy_version 975264 (0.0005) [2023-12-27 12:24:43,380][20347] Updated weights for policy 0, policy_version 975969 (0.0008) [2023-12-27 12:24:43,419][20381] Updated weights for policy 1, policy_version 975274 (0.0005) [2023-12-27 12:24:43,426][20347] Updated weights for policy 0, policy_version 975979 (0.0007) [2023-12-27 12:24:43,472][20347] Updated weights for policy 0, policy_version 975989 (0.0008) [2023-12-27 12:24:43,481][20381] Updated weights for policy 1, policy_version 975284 (0.0006) [2023-12-27 12:24:44,099][20381] Updated weights for policy 1, policy_version 975294 (0.0011) [2023-12-27 12:24:44,130][20347] Updated weights for policy 0, policy_version 975999 (0.0006) [2023-12-27 12:24:44,154][20381] Updated weights for policy 1, policy_version 975304 (0.0011) [2023-12-27 12:24:44,178][20347] Updated weights for policy 0, policy_version 976009 (0.0005) [2023-12-27 12:24:44,208][20381] Updated weights for policy 1, policy_version 975314 (0.0010) [2023-12-27 12:24:44,233][20347] Updated weights for policy 0, policy_version 976019 (0.0005) [2023-12-27 12:24:44,812][20381] Updated weights for policy 1, policy_version 975324 (0.0009) [2023-12-27 12:24:44,873][20381] Updated weights for policy 1, policy_version 975334 (0.0010) [2023-12-27 12:24:44,932][20381] Updated weights for policy 1, policy_version 975344 (0.0011) [2023-12-27 12:24:44,945][20347] Updated weights for policy 0, policy_version 976029 (0.0006) [2023-12-27 12:24:44,974][20291] KL-divergence is very high: 287.3593 [2023-12-27 12:24:45,006][20347] Updated weights for policy 0, policy_version 976039 (0.0008) [2023-12-27 12:24:45,033][20291] KL-divergence is very high: 299.1745 [2023-12-27 12:24:45,078][20347] Updated weights for policy 0, policy_version 976049 (0.0008) [2023-12-27 12:24:45,087][20291] KL-divergence is very high: 140.1386 [2023-12-27 12:24:45,591][20381] Updated weights for policy 1, policy_version 975354 (0.0007) [2023-12-27 12:24:45,653][20381] Updated weights for policy 1, policy_version 975364 (0.0006) [2023-12-27 12:24:45,711][20381] Updated weights for policy 1, policy_version 975374 (0.0007) [2023-12-27 12:24:45,747][20347] Updated weights for policy 0, policy_version 976059 (0.0006) [2023-12-27 12:24:45,768][20381] Updated weights for policy 1, policy_version 975384 (0.0006) [2023-12-27 12:24:45,805][20347] Updated weights for policy 0, policy_version 976069 (0.0007) [2023-12-27 12:24:45,862][20347] Updated weights for policy 0, policy_version 976079 (0.0006) [2023-12-27 12:24:46,461][20381] Updated weights for policy 1, policy_version 975394 (0.0008) [2023-12-27 12:24:46,466][20347] Updated weights for policy 0, policy_version 976089 (0.0005) [2023-12-27 12:24:46,515][20347] Updated weights for policy 0, policy_version 976099 (0.0006) [2023-12-27 12:24:46,516][20381] Updated weights for policy 1, policy_version 975404 (0.0006) [2023-12-27 12:24:46,566][20347] Updated weights for policy 0, policy_version 976109 (0.0006) [2023-12-27 12:24:46,574][20381] Updated weights for policy 1, policy_version 975414 (0.0006) [2023-12-27 12:24:46,616][20347] Updated weights for policy 0, policy_version 976119 (0.0007) [2023-12-27 12:24:47,229][20347] Updated weights for policy 0, policy_version 976129 (0.0005) [2023-12-27 12:24:47,280][20347] Updated weights for policy 0, policy_version 976139 (0.0006) [2023-12-27 12:24:47,293][20381] Updated weights for policy 1, policy_version 975424 (0.0006) [2023-12-27 12:24:47,337][20381] Updated weights for policy 1, policy_version 975434 (0.0006) [2023-12-27 12:24:47,338][20347] Updated weights for policy 0, policy_version 976149 (0.0008) [2023-12-27 12:24:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19797.3, 300 sec: 20327.3). Total num frames: 499671040. Throughput: 0: 9958.0, 1: 10181.0. Samples: 499680740. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:47,363][19170] Avg episode reward: [(0, '951.760'), (1, '863.690')] [2023-12-27 12:24:47,385][20381] Updated weights for policy 1, policy_version 975444 (0.0006) [2023-12-27 12:24:47,906][20347] Updated weights for policy 0, policy_version 976159 (0.0007) [2023-12-27 12:24:47,951][20347] Updated weights for policy 0, policy_version 976169 (0.0007) [2023-12-27 12:24:48,001][20347] Updated weights for policy 0, policy_version 976179 (0.0008) [2023-12-27 12:24:48,118][20381] Updated weights for policy 1, policy_version 975454 (0.0008) [2023-12-27 12:24:48,164][20381] Updated weights for policy 1, policy_version 975464 (0.0009) [2023-12-27 12:24:48,210][20381] Updated weights for policy 1, policy_version 975474 (0.0008) [2023-12-27 12:24:48,687][20347] Updated weights for policy 0, policy_version 976189 (0.0008) [2023-12-27 12:24:48,748][20347] Updated weights for policy 0, policy_version 976199 (0.0008) [2023-12-27 12:24:48,808][20347] Updated weights for policy 0, policy_version 976209 (0.0007) [2023-12-27 12:24:48,961][20381] Updated weights for policy 1, policy_version 975484 (0.0008) [2023-12-27 12:24:49,016][20381] Updated weights for policy 1, policy_version 975494 (0.0008) [2023-12-27 12:24:49,073][20381] Updated weights for policy 1, policy_version 975504 (0.0008) [2023-12-27 12:24:49,457][20347] Updated weights for policy 0, policy_version 976219 (0.0007) [2023-12-27 12:24:49,510][20347] Updated weights for policy 0, policy_version 976229 (0.0006) [2023-12-27 12:24:49,573][20347] Updated weights for policy 0, policy_version 976239 (0.0007) [2023-12-27 12:24:49,828][20381] Updated weights for policy 1, policy_version 975514 (0.0008) [2023-12-27 12:24:49,892][20381] Updated weights for policy 1, policy_version 975524 (0.0008) [2023-12-27 12:24:49,956][20381] Updated weights for policy 1, policy_version 975534 (0.0008) [2023-12-27 12:24:50,024][20381] Updated weights for policy 1, policy_version 975544 (0.0008) [2023-12-27 12:24:50,254][20347] Updated weights for policy 0, policy_version 976249 (0.0008) [2023-12-27 12:24:50,308][20347] Updated weights for policy 0, policy_version 976259 (0.0007) [2023-12-27 12:24:50,364][20291] KL-divergence is very high: 100.8345 [2023-12-27 12:24:50,370][20347] Updated weights for policy 0, policy_version 976269 (0.0008) [2023-12-27 12:24:50,434][20347] Updated weights for policy 0, policy_version 976279 (0.0008) [2023-12-27 12:24:50,753][20381] Updated weights for policy 1, policy_version 975554 (0.0007) [2023-12-27 12:24:50,801][20381] Updated weights for policy 1, policy_version 975564 (0.0008) [2023-12-27 12:24:50,848][20381] Updated weights for policy 1, policy_version 975574 (0.0008) [2023-12-27 12:24:51,038][20347] Updated weights for policy 0, policy_version 976289 (0.0007) [2023-12-27 12:24:51,102][20347] Updated weights for policy 0, policy_version 976299 (0.0007) [2023-12-27 12:24:51,166][20347] Updated weights for policy 0, policy_version 976309 (0.0007) [2023-12-27 12:24:51,574][20381] Updated weights for policy 1, policy_version 975584 (0.0007) [2023-12-27 12:24:51,644][20381] Updated weights for policy 1, policy_version 975594 (0.0008) [2023-12-27 12:24:51,707][20381] Updated weights for policy 1, policy_version 975604 (0.0011) [2023-12-27 12:24:51,829][20347] Updated weights for policy 0, policy_version 976319 (0.0007) [2023-12-27 12:24:51,881][20347] Updated weights for policy 0, policy_version 976329 (0.0007) [2023-12-27 12:24:51,933][20347] Updated weights for policy 0, policy_version 976339 (0.0006) [2023-12-27 12:24:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19797.4, 300 sec: 20271.7). Total num frames: 499769344. Throughput: 0: 10032.4, 1: 10121.3. Samples: 499741900. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:52,363][19170] Avg episode reward: [(0, '961.310'), (1, '856.630')] [2023-12-27 12:24:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000976344_249987072.pth... [2023-12-27 12:24:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000975128_249675776.pth [2023-12-27 12:24:52,375][20381] Updated weights for policy 1, policy_version 975614 (0.0008) [2023-12-27 12:24:52,436][20381] Updated weights for policy 1, policy_version 975624 (0.0007) [2023-12-27 12:24:52,494][20381] Updated weights for policy 1, policy_version 975634 (0.0008) [2023-12-27 12:24:52,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000975640_249790464.pth... [2023-12-27 12:24:52,536][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000974424_249479168.pth [2023-12-27 12:24:52,657][20347] Updated weights for policy 0, policy_version 976349 (0.0008) [2023-12-27 12:24:52,722][20347] Updated weights for policy 0, policy_version 976359 (0.0010) [2023-12-27 12:24:52,778][20347] Updated weights for policy 0, policy_version 976369 (0.0010) [2023-12-27 12:24:53,175][20381] Updated weights for policy 1, policy_version 975644 (0.0006) [2023-12-27 12:24:53,224][20381] Updated weights for policy 1, policy_version 975654 (0.0006) [2023-12-27 12:24:53,268][20381] Updated weights for policy 1, policy_version 975664 (0.0006) [2023-12-27 12:24:53,450][20347] Updated weights for policy 0, policy_version 976379 (0.0010) [2023-12-27 12:24:53,495][20347] Updated weights for policy 0, policy_version 976389 (0.0010) [2023-12-27 12:24:53,543][20347] Updated weights for policy 0, policy_version 976399 (0.0010) [2023-12-27 12:24:53,941][20381] Updated weights for policy 1, policy_version 975674 (0.0006) [2023-12-27 12:24:53,991][20381] Updated weights for policy 1, policy_version 975684 (0.0010) [2023-12-27 12:24:54,049][20381] Updated weights for policy 1, policy_version 975694 (0.0009) [2023-12-27 12:24:54,101][20381] Updated weights for policy 1, policy_version 975704 (0.0008) [2023-12-27 12:24:54,189][20347] Updated weights for policy 0, policy_version 976409 (0.0009) [2023-12-27 12:24:54,247][20347] Updated weights for policy 0, policy_version 976419 (0.0006) [2023-12-27 12:24:54,297][20347] Updated weights for policy 0, policy_version 976429 (0.0005) [2023-12-27 12:24:54,357][20347] Updated weights for policy 0, policy_version 976439 (0.0005) [2023-12-27 12:24:54,869][20381] Updated weights for policy 1, policy_version 975714 (0.0008) [2023-12-27 12:24:54,897][20347] Updated weights for policy 0, policy_version 976449 (0.0005) [2023-12-27 12:24:54,929][20381] Updated weights for policy 1, policy_version 975724 (0.0007) [2023-12-27 12:24:54,947][20347] Updated weights for policy 0, policy_version 976459 (0.0005) [2023-12-27 12:24:54,996][20381] Updated weights for policy 1, policy_version 975734 (0.0008) [2023-12-27 12:24:55,002][20347] Updated weights for policy 0, policy_version 976469 (0.0006) [2023-12-27 12:24:55,616][20347] Updated weights for policy 0, policy_version 976479 (0.0007) [2023-12-27 12:24:55,663][20381] Updated weights for policy 1, policy_version 975744 (0.0006) [2023-12-27 12:24:55,672][20347] Updated weights for policy 0, policy_version 976489 (0.0006) [2023-12-27 12:24:55,716][20381] Updated weights for policy 1, policy_version 975754 (0.0005) [2023-12-27 12:24:55,725][20347] Updated weights for policy 0, policy_version 976499 (0.0007) [2023-12-27 12:24:55,774][20381] Updated weights for policy 1, policy_version 975764 (0.0006) [2023-12-27 12:24:56,439][20381] Updated weights for policy 1, policy_version 975774 (0.0007) [2023-12-27 12:24:56,440][20347] Updated weights for policy 0, policy_version 976509 (0.0007) [2023-12-27 12:24:56,493][20381] Updated weights for policy 1, policy_version 975784 (0.0006) [2023-12-27 12:24:56,494][20347] Updated weights for policy 0, policy_version 976519 (0.0007) [2023-12-27 12:24:56,540][20347] Updated weights for policy 0, policy_version 976529 (0.0007) [2023-12-27 12:24:56,547][20381] Updated weights for policy 1, policy_version 975794 (0.0006) [2023-12-27 12:24:57,231][20347] Updated weights for policy 0, policy_version 976539 (0.0008) [2023-12-27 12:24:57,237][20381] Updated weights for policy 1, policy_version 975804 (0.0007) [2023-12-27 12:24:57,280][20347] Updated weights for policy 0, policy_version 976549 (0.0007) [2023-12-27 12:24:57,300][20381] Updated weights for policy 1, policy_version 975814 (0.0005) [2023-12-27 12:24:57,332][20347] Updated weights for policy 0, policy_version 976559 (0.0006) [2023-12-27 12:24:57,354][20381] Updated weights for policy 1, policy_version 975824 (0.0007) [2023-12-27 12:24:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19797.3, 300 sec: 20271.7). Total num frames: 499867648. Throughput: 0: 10250.1, 1: 10154.0. Samples: 499865956. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:24:57,363][19170] Avg episode reward: [(0, '990.370'), (1, '858.640')] [2023-12-27 12:24:57,962][20347] Updated weights for policy 0, policy_version 976569 (0.0005) [2023-12-27 12:24:58,013][20347] Updated weights for policy 0, policy_version 976579 (0.0007) [2023-12-27 12:24:58,039][20381] Updated weights for policy 1, policy_version 975834 (0.0007) [2023-12-27 12:24:58,069][20347] Updated weights for policy 0, policy_version 976589 (0.0005) [2023-12-27 12:24:58,090][20381] Updated weights for policy 1, policy_version 975844 (0.0007) [2023-12-27 12:24:58,115][20347] Updated weights for policy 0, policy_version 976599 (0.0005) [2023-12-27 12:24:58,144][20381] Updated weights for policy 1, policy_version 975854 (0.0008) [2023-12-27 12:24:58,206][20381] Updated weights for policy 1, policy_version 975864 (0.0008) [2023-12-27 12:24:58,827][20347] Updated weights for policy 0, policy_version 976609 (0.0009) [2023-12-27 12:24:58,889][20347] Updated weights for policy 0, policy_version 976619 (0.0008) [2023-12-27 12:24:58,955][20347] Updated weights for policy 0, policy_version 976629 (0.0008) [2023-12-27 12:24:58,999][20381] Updated weights for policy 1, policy_version 975874 (0.0006) [2023-12-27 12:24:59,045][20381] Updated weights for policy 1, policy_version 975884 (0.0006) [2023-12-27 12:24:59,099][20381] Updated weights for policy 1, policy_version 975894 (0.0007) [2023-12-27 12:24:59,707][20347] Updated weights for policy 0, policy_version 976639 (0.0007) [2023-12-27 12:24:59,775][20347] Updated weights for policy 0, policy_version 976649 (0.0006) [2023-12-27 12:24:59,777][20381] Updated weights for policy 1, policy_version 975904 (0.0007) [2023-12-27 12:24:59,830][20381] Updated weights for policy 1, policy_version 975914 (0.0008) [2023-12-27 12:24:59,841][20347] Updated weights for policy 0, policy_version 976659 (0.0007) [2023-12-27 12:24:59,890][20381] Updated weights for policy 1, policy_version 975924 (0.0007) [2023-12-27 12:25:00,477][20347] Updated weights for policy 0, policy_version 976669 (0.0007) [2023-12-27 12:25:00,533][20347] Updated weights for policy 0, policy_version 976679 (0.0007) [2023-12-27 12:25:00,550][20381] Updated weights for policy 1, policy_version 975934 (0.0007) [2023-12-27 12:25:00,592][20347] Updated weights for policy 0, policy_version 976689 (0.0007) [2023-12-27 12:25:00,600][20381] Updated weights for policy 1, policy_version 975944 (0.0005) [2023-12-27 12:25:00,646][20381] Updated weights for policy 1, policy_version 975954 (0.0005) [2023-12-27 12:25:01,266][20381] Updated weights for policy 1, policy_version 975964 (0.0006) [2023-12-27 12:25:01,289][20292] KL-divergence is very high: 116.5711 [2023-12-27 12:25:01,321][20381] Updated weights for policy 1, policy_version 975974 (0.0007) [2023-12-27 12:25:01,336][20347] Updated weights for policy 0, policy_version 976699 (0.0007) [2023-12-27 12:25:01,393][20381] Updated weights for policy 1, policy_version 975984 (0.0007) [2023-12-27 12:25:01,399][20347] Updated weights for policy 0, policy_version 976709 (0.0006) [2023-12-27 12:25:01,457][20347] Updated weights for policy 0, policy_version 976719 (0.0008) [2023-12-27 12:25:02,026][20381] Updated weights for policy 1, policy_version 975994 (0.0007) [2023-12-27 12:25:02,086][20381] Updated weights for policy 1, policy_version 976004 (0.0006) [2023-12-27 12:25:02,162][20381] Updated weights for policy 1, policy_version 976014 (0.0008) [2023-12-27 12:25:02,208][20347] Updated weights for policy 0, policy_version 976729 (0.0008) [2023-12-27 12:25:02,226][20381] Updated weights for policy 1, policy_version 976024 (0.0007) [2023-12-27 12:25:02,266][20347] Updated weights for policy 0, policy_version 976739 (0.0007) [2023-12-27 12:25:02,324][20347] Updated weights for policy 0, policy_version 976749 (0.0008) [2023-12-27 12:25:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20070.4, 300 sec: 20271.7). Total num frames: 499974144. Throughput: 0: 10328.8, 1: 10176.0. Samples: 499986068. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:25:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '879.190')] [2023-12-27 12:25:02,392][20347] Updated weights for policy 0, policy_version 976759 (0.0009) [2023-12-27 12:25:02,954][20381] Updated weights for policy 1, policy_version 976034 (0.0009) [2023-12-27 12:25:03,008][20381] Updated weights for policy 1, policy_version 976044 (0.0008) [2023-12-27 12:25:03,063][20381] Updated weights for policy 1, policy_version 976054 (0.0007) [2023-12-27 12:25:03,064][20347] Updated weights for policy 0, policy_version 976769 (0.0005) [2023-12-27 12:25:03,111][20347] Updated weights for policy 0, policy_version 976779 (0.0007) [2023-12-27 12:25:03,172][20347] Updated weights for policy 0, policy_version 976789 (0.0007) [2023-12-27 12:25:03,691][20381] Updated weights for policy 1, policy_version 976064 (0.0005) [2023-12-27 12:25:03,748][20381] Updated weights for policy 1, policy_version 976074 (0.0005) [2023-12-27 12:25:03,799][20381] Updated weights for policy 1, policy_version 976084 (0.0007) [2023-12-27 12:25:03,860][20347] Updated weights for policy 0, policy_version 976799 (0.0007) [2023-12-27 12:25:03,920][20347] Updated weights for policy 0, policy_version 976809 (0.0007) [2023-12-27 12:25:03,978][20347] Updated weights for policy 0, policy_version 976819 (0.0007) [2023-12-27 12:25:04,469][20381] Updated weights for policy 1, policy_version 976094 (0.0009) [2023-12-27 12:25:04,522][20381] Updated weights for policy 1, policy_version 976104 (0.0010) [2023-12-27 12:25:04,580][20381] Updated weights for policy 1, policy_version 976114 (0.0011) [2023-12-27 12:25:04,705][20347] Updated weights for policy 0, policy_version 976829 (0.0007) [2023-12-27 12:25:04,754][20347] Updated weights for policy 0, policy_version 976839 (0.0007) [2023-12-27 12:25:04,803][20347] Updated weights for policy 0, policy_version 976849 (0.0006) [2023-12-27 12:25:05,232][20381] Updated weights for policy 1, policy_version 976124 (0.0009) [2023-12-27 12:25:05,286][20381] Updated weights for policy 1, policy_version 976134 (0.0005) [2023-12-27 12:25:05,332][20381] Updated weights for policy 1, policy_version 976144 (0.0005) [2023-12-27 12:25:05,542][20347] Updated weights for policy 0, policy_version 976859 (0.0008) [2023-12-27 12:25:05,590][20347] Updated weights for policy 0, policy_version 976869 (0.0008) [2023-12-27 12:25:05,638][20347] Updated weights for policy 0, policy_version 976879 (0.0009) [2023-12-27 12:25:05,858][20381] Updated weights for policy 1, policy_version 976154 (0.0006) [2023-12-27 12:25:05,921][20381] Updated weights for policy 1, policy_version 976164 (0.0005) [2023-12-27 12:25:05,979][20381] Updated weights for policy 1, policy_version 976174 (0.0007) [2023-12-27 12:25:06,035][20381] Updated weights for policy 1, policy_version 976184 (0.0006) [2023-12-27 12:25:06,379][20347] Updated weights for policy 0, policy_version 976889 (0.0009) [2023-12-27 12:25:06,440][20347] Updated weights for policy 0, policy_version 976899 (0.0007) [2023-12-27 12:25:06,500][20347] Updated weights for policy 0, policy_version 976909 (0.0007) [2023-12-27 12:25:06,559][20347] Updated weights for policy 0, policy_version 976919 (0.0006) [2023-12-27 12:25:06,693][20381] Updated weights for policy 1, policy_version 976194 (0.0008) [2023-12-27 12:25:06,743][20381] Updated weights for policy 1, policy_version 976204 (0.0007) [2023-12-27 12:25:06,792][20381] Updated weights for policy 1, policy_version 976214 (0.0008) [2023-12-27 12:25:07,223][20347] Updated weights for policy 0, policy_version 976929 (0.0007) [2023-12-27 12:25:07,275][20347] Updated weights for policy 0, policy_version 976939 (0.0007) [2023-12-27 12:25:07,327][20347] Updated weights for policy 0, policy_version 976949 (0.0007) [2023-12-27 12:25:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20299.5). Total num frames: 500080640. Throughput: 0: 10321.8, 1: 10175.6. Samples: 500046976. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:25:07,363][19170] Avg episode reward: [(0, '990.270'), (1, '866.440')] [2023-12-27 12:25:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000976216_249937920.pth... [2023-12-27 12:25:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000976952_250142720.pth... [2023-12-27 12:25:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000975736_249831424.pth [2023-12-27 12:25:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000975032_249634816.pth [2023-12-27 12:25:07,495][20381] Updated weights for policy 1, policy_version 976224 (0.0008) [2023-12-27 12:25:07,549][20381] Updated weights for policy 1, policy_version 976234 (0.0005) [2023-12-27 12:25:07,600][20381] Updated weights for policy 1, policy_version 976244 (0.0005) [2023-12-27 12:25:08,065][20347] Updated weights for policy 0, policy_version 976959 (0.0007) [2023-12-27 12:25:08,120][20347] Updated weights for policy 0, policy_version 976969 (0.0008) [2023-12-27 12:25:08,177][20347] Updated weights for policy 0, policy_version 976979 (0.0009) [2023-12-27 12:25:08,188][20381] Updated weights for policy 1, policy_version 976254 (0.0005) [2023-12-27 12:25:08,244][20381] Updated weights for policy 1, policy_version 976264 (0.0005) [2023-12-27 12:25:08,298][20381] Updated weights for policy 1, policy_version 976274 (0.0006) [2023-12-27 12:25:08,893][20347] Updated weights for policy 0, policy_version 976989 (0.0008) [2023-12-27 12:25:08,951][20347] Updated weights for policy 0, policy_version 976999 (0.0007) [2023-12-27 12:25:08,964][20381] Updated weights for policy 1, policy_version 976284 (0.0007) [2023-12-27 12:25:09,008][20347] Updated weights for policy 0, policy_version 977009 (0.0006) [2023-12-27 12:25:09,014][20381] Updated weights for policy 1, policy_version 976294 (0.0005) [2023-12-27 12:25:09,062][20381] Updated weights for policy 1, policy_version 976304 (0.0009) [2023-12-27 12:25:09,736][20347] Updated weights for policy 0, policy_version 977019 (0.0007) [2023-12-27 12:25:09,800][20347] Updated weights for policy 0, policy_version 977029 (0.0008) [2023-12-27 12:25:09,825][20381] Updated weights for policy 1, policy_version 976314 (0.0008) [2023-12-27 12:25:09,865][20347] Updated weights for policy 0, policy_version 977039 (0.0009) [2023-12-27 12:25:09,887][20381] Updated weights for policy 1, policy_version 976324 (0.0007) [2023-12-27 12:25:09,953][20381] Updated weights for policy 1, policy_version 976334 (0.0007) [2023-12-27 12:25:10,017][20381] Updated weights for policy 1, policy_version 976344 (0.0007) [2023-12-27 12:25:10,636][20347] Updated weights for policy 0, policy_version 977049 (0.0007) [2023-12-27 12:25:10,677][20381] Updated weights for policy 1, policy_version 976354 (0.0007) [2023-12-27 12:25:10,691][20347] Updated weights for policy 0, policy_version 977059 (0.0007) [2023-12-27 12:25:10,742][20381] Updated weights for policy 1, policy_version 976364 (0.0010) [2023-12-27 12:25:10,746][20347] Updated weights for policy 0, policy_version 977069 (0.0006) [2023-12-27 12:25:10,799][20347] Updated weights for policy 0, policy_version 977079 (0.0006) [2023-12-27 12:25:10,802][20381] Updated weights for policy 1, policy_version 976374 (0.0008) [2023-12-27 12:25:11,464][20381] Updated weights for policy 1, policy_version 976384 (0.0008) [2023-12-27 12:25:11,516][20347] Updated weights for policy 0, policy_version 977089 (0.0006) [2023-12-27 12:25:11,519][20381] Updated weights for policy 1, policy_version 976394 (0.0009) [2023-12-27 12:25:11,576][20347] Updated weights for policy 0, policy_version 977099 (0.0008) [2023-12-27 12:25:11,581][20381] Updated weights for policy 1, policy_version 976404 (0.0009) [2023-12-27 12:25:11,637][20347] Updated weights for policy 0, policy_version 977109 (0.0008) [2023-12-27 12:25:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20244.0). Total num frames: 500170752. Throughput: 0: 10245.6, 1: 10250.9. Samples: 500168748. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:25:12,363][19170] Avg episode reward: [(0, '980.590'), (1, '854.010')] [2023-12-27 12:25:12,394][20381] Updated weights for policy 1, policy_version 976414 (0.0008) [2023-12-27 12:25:12,395][20347] Updated weights for policy 0, policy_version 977119 (0.0007) [2023-12-27 12:25:12,454][20347] Updated weights for policy 0, policy_version 977129 (0.0006) [2023-12-27 12:25:12,456][20381] Updated weights for policy 1, policy_version 976424 (0.0006) [2023-12-27 12:25:12,510][20347] Updated weights for policy 0, policy_version 977139 (0.0005) [2023-12-27 12:25:12,510][20381] Updated weights for policy 1, policy_version 976434 (0.0008) [2023-12-27 12:25:13,189][20347] Updated weights for policy 0, policy_version 977149 (0.0008) [2023-12-27 12:25:13,204][20381] Updated weights for policy 1, policy_version 976444 (0.0009) [2023-12-27 12:25:13,237][20347] Updated weights for policy 0, policy_version 977159 (0.0005) [2023-12-27 12:25:13,254][20381] Updated weights for policy 1, policy_version 976454 (0.0011) [2023-12-27 12:25:13,286][20347] Updated weights for policy 0, policy_version 977169 (0.0006) [2023-12-27 12:25:13,300][20381] Updated weights for policy 1, policy_version 976464 (0.0010) [2023-12-27 12:25:13,909][20347] Updated weights for policy 0, policy_version 977179 (0.0010) [2023-12-27 12:25:13,936][20381] Updated weights for policy 1, policy_version 976474 (0.0010) [2023-12-27 12:25:13,965][20347] Updated weights for policy 0, policy_version 977189 (0.0006) [2023-12-27 12:25:13,987][20381] Updated weights for policy 1, policy_version 976484 (0.0007) [2023-12-27 12:25:14,026][20347] Updated weights for policy 0, policy_version 977199 (0.0008) [2023-12-27 12:25:14,043][20381] Updated weights for policy 1, policy_version 976494 (0.0007) [2023-12-27 12:25:14,058][20291] KL-divergence is very high: 138.4484 [2023-12-27 12:25:14,099][20381] Updated weights for policy 1, policy_version 976504 (0.0007) [2023-12-27 12:25:14,649][20347] Updated weights for policy 0, policy_version 977209 (0.0009) [2023-12-27 12:25:14,699][20347] Updated weights for policy 0, policy_version 977219 (0.0007) [2023-12-27 12:25:14,700][20291] KL-divergence is very high: 134.0211 [2023-12-27 12:25:14,705][20291] KL-divergence is very high: 1198.0298 [2023-12-27 12:25:14,730][20381] Updated weights for policy 1, policy_version 976514 (0.0006) [2023-12-27 12:25:14,747][20347] Updated weights for policy 0, policy_version 977229 (0.0008) [2023-12-27 12:25:14,786][20381] Updated weights for policy 1, policy_version 976524 (0.0007) [2023-12-27 12:25:14,810][20347] Updated weights for policy 0, policy_version 977239 (0.0008) [2023-12-27 12:25:14,853][20381] Updated weights for policy 1, policy_version 976534 (0.0007) [2023-12-27 12:25:15,555][20347] Updated weights for policy 0, policy_version 977249 (0.0008) [2023-12-27 12:25:15,570][20381] Updated weights for policy 1, policy_version 976544 (0.0010) [2023-12-27 12:25:15,606][20347] Updated weights for policy 0, policy_version 977259 (0.0008) [2023-12-27 12:25:15,631][20381] Updated weights for policy 1, policy_version 976554 (0.0010) [2023-12-27 12:25:15,660][20347] Updated weights for policy 0, policy_version 977269 (0.0008) [2023-12-27 12:25:15,689][20381] Updated weights for policy 1, policy_version 976564 (0.0011) [2023-12-27 12:25:16,342][20381] Updated weights for policy 1, policy_version 976574 (0.0008) [2023-12-27 12:25:16,391][20347] Updated weights for policy 0, policy_version 977279 (0.0007) [2023-12-27 12:25:16,397][20381] Updated weights for policy 1, policy_version 976584 (0.0011) [2023-12-27 12:25:16,450][20347] Updated weights for policy 0, policy_version 977289 (0.0005) [2023-12-27 12:25:16,452][20381] Updated weights for policy 1, policy_version 976594 (0.0010) [2023-12-27 12:25:16,510][20347] Updated weights for policy 0, policy_version 977299 (0.0006) [2023-12-27 12:25:17,107][20381] Updated weights for policy 1, policy_version 976604 (0.0009) [2023-12-27 12:25:17,116][20347] Updated weights for policy 0, policy_version 977309 (0.0007) [2023-12-27 12:25:17,157][20381] Updated weights for policy 1, policy_version 976614 (0.0006) [2023-12-27 12:25:17,170][20347] Updated weights for policy 0, policy_version 977319 (0.0008) [2023-12-27 12:25:17,216][20381] Updated weights for policy 1, policy_version 976624 (0.0007) [2023-12-27 12:25:17,233][20347] Updated weights for policy 0, policy_version 977329 (0.0007) [2023-12-27 12:25:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20299.5). Total num frames: 500285440. Throughput: 0: 10195.5, 1: 10204.8. Samples: 500289732. Policy #0 lag: (min: 26.0, avg: 40.8, max: 58.0) [2023-12-27 12:25:17,364][19170] Avg episode reward: [(0, '946.690'), (1, '883.090')] [2023-12-27 12:25:17,911][20381] Updated weights for policy 1, policy_version 976634 (0.0010) [2023-12-27 12:25:17,917][20347] Updated weights for policy 0, policy_version 977339 (0.0005) [2023-12-27 12:25:17,962][20381] Updated weights for policy 1, policy_version 976644 (0.0010) [2023-12-27 12:25:17,965][20347] Updated weights for policy 0, policy_version 977349 (0.0005) [2023-12-27 12:25:18,016][20347] Updated weights for policy 0, policy_version 977359 (0.0005) [2023-12-27 12:25:18,018][20381] Updated weights for policy 1, policy_version 976654 (0.0007) [2023-12-27 12:25:18,066][20381] Updated weights for policy 1, policy_version 976664 (0.0007) [2023-12-27 12:25:18,674][20347] Updated weights for policy 0, policy_version 977369 (0.0005) [2023-12-27 12:25:18,732][20347] Updated weights for policy 0, policy_version 977379 (0.0010) [2023-12-27 12:25:18,752][20381] Updated weights for policy 1, policy_version 976674 (0.0011) [2023-12-27 12:25:18,788][20347] Updated weights for policy 0, policy_version 977389 (0.0010) [2023-12-27 12:25:18,808][20381] Updated weights for policy 1, policy_version 976684 (0.0010) [2023-12-27 12:25:18,841][20347] Updated weights for policy 0, policy_version 977399 (0.0010) [2023-12-27 12:25:18,855][20381] Updated weights for policy 1, policy_version 976694 (0.0011) [2023-12-27 12:25:19,572][20347] Updated weights for policy 0, policy_version 977409 (0.0007) [2023-12-27 12:25:19,623][20347] Updated weights for policy 0, policy_version 977419 (0.0006) [2023-12-27 12:25:19,627][20381] Updated weights for policy 1, policy_version 976704 (0.0008) [2023-12-27 12:25:19,679][20347] Updated weights for policy 0, policy_version 977429 (0.0006) [2023-12-27 12:25:19,683][20381] Updated weights for policy 1, policy_version 976714 (0.0008) [2023-12-27 12:25:19,746][20381] Updated weights for policy 1, policy_version 976724 (0.0008) [2023-12-27 12:25:20,376][20347] Updated weights for policy 0, policy_version 977439 (0.0008) [2023-12-27 12:25:20,435][20347] Updated weights for policy 0, policy_version 977449 (0.0008) [2023-12-27 12:25:20,499][20347] Updated weights for policy 0, policy_version 977459 (0.0007) [2023-12-27 12:25:20,514][20381] Updated weights for policy 1, policy_version 976734 (0.0008) [2023-12-27 12:25:20,580][20381] Updated weights for policy 1, policy_version 976744 (0.0008) [2023-12-27 12:25:20,642][20381] Updated weights for policy 1, policy_version 976754 (0.0008) [2023-12-27 12:25:21,168][20347] Updated weights for policy 0, policy_version 977469 (0.0007) [2023-12-27 12:25:21,227][20347] Updated weights for policy 0, policy_version 977479 (0.0006) [2023-12-27 12:25:21,293][20347] Updated weights for policy 0, policy_version 977489 (0.0008) [2023-12-27 12:25:21,381][20381] Updated weights for policy 1, policy_version 976764 (0.0010) [2023-12-27 12:25:21,447][20381] Updated weights for policy 1, policy_version 976774 (0.0008) [2023-12-27 12:25:21,503][20381] Updated weights for policy 1, policy_version 976784 (0.0006) [2023-12-27 12:25:21,964][20347] Updated weights for policy 0, policy_version 977499 (0.0007) [2023-12-27 12:25:22,030][20347] Updated weights for policy 0, policy_version 977509 (0.0008) [2023-12-27 12:25:22,090][20347] Updated weights for policy 0, policy_version 977519 (0.0008) [2023-12-27 12:25:22,222][20381] Updated weights for policy 1, policy_version 976794 (0.0010) [2023-12-27 12:25:22,287][20381] Updated weights for policy 1, policy_version 976804 (0.0008) [2023-12-27 12:25:22,352][20381] Updated weights for policy 1, policy_version 976814 (0.0007) [2023-12-27 12:25:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20243.9). Total num frames: 500375552. Throughput: 0: 10163.7, 1: 10118.4. Samples: 500348748. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:22,363][19170] Avg episode reward: [(0, '946.810'), (1, '941.660')] [2023-12-27 12:25:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000977528_250290176.pth... [2023-12-27 12:25:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000976344_249987072.pth [2023-12-27 12:25:22,418][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000976824_250093568.pth... [2023-12-27 12:25:22,420][20381] Updated weights for policy 1, policy_version 976824 (0.0007) [2023-12-27 12:25:22,422][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000975640_249790464.pth [2023-12-27 12:25:22,810][20347] Updated weights for policy 0, policy_version 977529 (0.0008) [2023-12-27 12:25:22,870][20347] Updated weights for policy 0, policy_version 977539 (0.0008) [2023-12-27 12:25:22,927][20347] Updated weights for policy 0, policy_version 977549 (0.0007) [2023-12-27 12:25:22,982][20347] Updated weights for policy 0, policy_version 977559 (0.0007) [2023-12-27 12:25:23,112][20381] Updated weights for policy 1, policy_version 976834 (0.0007) [2023-12-27 12:25:23,170][20381] Updated weights for policy 1, policy_version 976844 (0.0007) [2023-12-27 12:25:23,259][20381] Updated weights for policy 1, policy_version 976854 (0.0006) [2023-12-27 12:25:23,662][20347] Updated weights for policy 0, policy_version 977569 (0.0008) [2023-12-27 12:25:23,718][20347] Updated weights for policy 0, policy_version 977579 (0.0008) [2023-12-27 12:25:23,777][20347] Updated weights for policy 0, policy_version 977589 (0.0007) [2023-12-27 12:25:23,835][20381] Updated weights for policy 1, policy_version 976864 (0.0008) [2023-12-27 12:25:23,894][20381] Updated weights for policy 1, policy_version 976874 (0.0008) [2023-12-27 12:25:23,941][20381] Updated weights for policy 1, policy_version 976884 (0.0007) [2023-12-27 12:25:24,414][20347] Updated weights for policy 0, policy_version 977599 (0.0008) [2023-12-27 12:25:24,473][20347] Updated weights for policy 0, policy_version 977609 (0.0008) [2023-12-27 12:25:24,536][20347] Updated weights for policy 0, policy_version 977619 (0.0008) [2023-12-27 12:25:24,624][20381] Updated weights for policy 1, policy_version 976894 (0.0008) [2023-12-27 12:25:24,673][20381] Updated weights for policy 1, policy_version 976904 (0.0008) [2023-12-27 12:25:24,725][20381] Updated weights for policy 1, policy_version 976914 (0.0008) [2023-12-27 12:25:25,191][20347] Updated weights for policy 0, policy_version 977629 (0.0008) [2023-12-27 12:25:25,244][20347] Updated weights for policy 0, policy_version 977639 (0.0006) [2023-12-27 12:25:25,302][20347] Updated weights for policy 0, policy_version 977649 (0.0010) [2023-12-27 12:25:25,431][20381] Updated weights for policy 1, policy_version 976924 (0.0007) [2023-12-27 12:25:25,485][20381] Updated weights for policy 1, policy_version 976934 (0.0009) [2023-12-27 12:25:25,538][20381] Updated weights for policy 1, policy_version 976944 (0.0008) [2023-12-27 12:25:25,870][20347] Updated weights for policy 0, policy_version 977659 (0.0010) [2023-12-27 12:25:25,916][20347] Updated weights for policy 0, policy_version 977669 (0.0008) [2023-12-27 12:25:25,966][20347] Updated weights for policy 0, policy_version 977679 (0.0005) [2023-12-27 12:25:26,285][20381] Updated weights for policy 1, policy_version 976954 (0.0009) [2023-12-27 12:25:26,342][20381] Updated weights for policy 1, policy_version 976964 (0.0008) [2023-12-27 12:25:26,386][20381] Updated weights for policy 1, policy_version 976974 (0.0008) [2023-12-27 12:25:26,435][20381] Updated weights for policy 1, policy_version 976984 (0.0008) [2023-12-27 12:25:26,516][20347] Updated weights for policy 0, policy_version 977689 (0.0005) [2023-12-27 12:25:26,569][20347] Updated weights for policy 0, policy_version 977699 (0.0005) [2023-12-27 12:25:26,620][20347] Updated weights for policy 0, policy_version 977709 (0.0006) [2023-12-27 12:25:26,677][20347] Updated weights for policy 0, policy_version 977719 (0.0007) [2023-12-27 12:25:27,182][20381] Updated weights for policy 1, policy_version 976994 (0.0007) [2023-12-27 12:25:27,231][20381] Updated weights for policy 1, policy_version 977004 (0.0007) [2023-12-27 12:25:27,233][20347] Updated weights for policy 0, policy_version 977729 (0.0006) [2023-12-27 12:25:27,278][20381] Updated weights for policy 1, policy_version 977014 (0.0007) [2023-12-27 12:25:27,290][20347] Updated weights for policy 0, policy_version 977739 (0.0008) [2023-12-27 12:25:27,347][20347] Updated weights for policy 0, policy_version 977749 (0.0008) [2023-12-27 12:25:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 500482048. Throughput: 0: 10181.0, 1: 10158.7. Samples: 500470812. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:27,363][19170] Avg episode reward: [(0, '971.010'), (1, '941.640')] [2023-12-27 12:25:27,967][20347] Updated weights for policy 0, policy_version 977759 (0.0007) [2023-12-27 12:25:28,019][20347] Updated weights for policy 0, policy_version 977769 (0.0007) [2023-12-27 12:25:28,043][20381] Updated weights for policy 1, policy_version 977024 (0.0008) [2023-12-27 12:25:28,077][20347] Updated weights for policy 0, policy_version 977779 (0.0005) [2023-12-27 12:25:28,093][20381] Updated weights for policy 1, policy_version 977034 (0.0007) [2023-12-27 12:25:28,141][20381] Updated weights for policy 1, policy_version 977044 (0.0008) [2023-12-27 12:25:28,729][20347] Updated weights for policy 0, policy_version 977789 (0.0007) [2023-12-27 12:25:28,784][20347] Updated weights for policy 0, policy_version 977799 (0.0008) [2023-12-27 12:25:28,823][20381] Updated weights for policy 1, policy_version 977054 (0.0007) [2023-12-27 12:25:28,837][20347] Updated weights for policy 0, policy_version 977809 (0.0008) [2023-12-27 12:25:28,874][20381] Updated weights for policy 1, policy_version 977064 (0.0008) [2023-12-27 12:25:28,928][20381] Updated weights for policy 1, policy_version 977074 (0.0007) [2023-12-27 12:25:29,546][20347] Updated weights for policy 0, policy_version 977819 (0.0009) [2023-12-27 12:25:29,597][20347] Updated weights for policy 0, policy_version 977829 (0.0007) [2023-12-27 12:25:29,605][20381] Updated weights for policy 1, policy_version 977084 (0.0006) [2023-12-27 12:25:29,650][20347] Updated weights for policy 0, policy_version 977839 (0.0005) [2023-12-27 12:25:29,656][20381] Updated weights for policy 1, policy_version 977094 (0.0006) [2023-12-27 12:25:29,711][20381] Updated weights for policy 1, policy_version 977104 (0.0007) [2023-12-27 12:25:30,344][20347] Updated weights for policy 0, policy_version 977849 (0.0008) [2023-12-27 12:25:30,406][20347] Updated weights for policy 0, policy_version 977859 (0.0008) [2023-12-27 12:25:30,416][20381] Updated weights for policy 1, policy_version 977114 (0.0007) [2023-12-27 12:25:30,463][20347] Updated weights for policy 0, policy_version 977869 (0.0008) [2023-12-27 12:25:30,470][20381] Updated weights for policy 1, policy_version 977124 (0.0006) [2023-12-27 12:25:30,522][20347] Updated weights for policy 0, policy_version 977879 (0.0008) [2023-12-27 12:25:30,527][20381] Updated weights for policy 1, policy_version 977134 (0.0008) [2023-12-27 12:25:30,580][20381] Updated weights for policy 1, policy_version 977144 (0.0007) [2023-12-27 12:25:31,219][20347] Updated weights for policy 0, policy_version 977889 (0.0007) [2023-12-27 12:25:31,283][20347] Updated weights for policy 0, policy_version 977899 (0.0007) [2023-12-27 12:25:31,286][20381] Updated weights for policy 1, policy_version 977154 (0.0007) [2023-12-27 12:25:31,334][20347] Updated weights for policy 0, policy_version 977909 (0.0006) [2023-12-27 12:25:31,340][20381] Updated weights for policy 1, policy_version 977164 (0.0006) [2023-12-27 12:25:31,407][20381] Updated weights for policy 1, policy_version 977174 (0.0006) [2023-12-27 12:25:32,049][20347] Updated weights for policy 0, policy_version 977919 (0.0009) [2023-12-27 12:25:32,099][20347] Updated weights for policy 0, policy_version 977929 (0.0006) [2023-12-27 12:25:32,105][20381] Updated weights for policy 1, policy_version 977184 (0.0006) [2023-12-27 12:25:32,149][20347] Updated weights for policy 0, policy_version 977939 (0.0007) [2023-12-27 12:25:32,156][20381] Updated weights for policy 1, policy_version 977194 (0.0006) [2023-12-27 12:25:32,207][20381] Updated weights for policy 1, policy_version 977204 (0.0007) [2023-12-27 12:25:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20344.0, 300 sec: 20271.7). Total num frames: 500588544. Throughput: 0: 10238.9, 1: 10046.7. Samples: 500593592. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:32,363][19170] Avg episode reward: [(0, '970.870'), (1, '941.540')] [2023-12-27 12:25:32,829][20347] Updated weights for policy 0, policy_version 977949 (0.0007) [2023-12-27 12:25:32,891][20347] Updated weights for policy 0, policy_version 977959 (0.0008) [2023-12-27 12:25:32,941][20347] Updated weights for policy 0, policy_version 977969 (0.0005) [2023-12-27 12:25:32,975][20381] Updated weights for policy 1, policy_version 977214 (0.0009) [2023-12-27 12:25:33,021][20381] Updated weights for policy 1, policy_version 977224 (0.0010) [2023-12-27 12:25:33,069][20381] Updated weights for policy 1, policy_version 977234 (0.0010) [2023-12-27 12:25:33,546][20347] Updated weights for policy 0, policy_version 977979 (0.0006) [2023-12-27 12:25:33,613][20347] Updated weights for policy 0, policy_version 977989 (0.0008) [2023-12-27 12:25:33,669][20347] Updated weights for policy 0, policy_version 977999 (0.0008) [2023-12-27 12:25:33,696][20291] KL-divergence is very high: 247.8472 [2023-12-27 12:25:33,755][20381] Updated weights for policy 1, policy_version 977244 (0.0009) [2023-12-27 12:25:33,813][20381] Updated weights for policy 1, policy_version 977254 (0.0006) [2023-12-27 12:25:33,863][20381] Updated weights for policy 1, policy_version 977264 (0.0005) [2023-12-27 12:25:34,395][20347] Updated weights for policy 0, policy_version 978009 (0.0007) [2023-12-27 12:25:34,455][20347] Updated weights for policy 0, policy_version 978019 (0.0008) [2023-12-27 12:25:34,500][20381] Updated weights for policy 1, policy_version 977274 (0.0006) [2023-12-27 12:25:34,517][20347] Updated weights for policy 0, policy_version 978029 (0.0007) [2023-12-27 12:25:34,556][20381] Updated weights for policy 1, policy_version 977284 (0.0007) [2023-12-27 12:25:34,577][20347] Updated weights for policy 0, policy_version 978039 (0.0006) [2023-12-27 12:25:34,619][20381] Updated weights for policy 1, policy_version 977294 (0.0007) [2023-12-27 12:25:34,677][20381] Updated weights for policy 1, policy_version 977304 (0.0008) [2023-12-27 12:25:35,229][20347] Updated weights for policy 0, policy_version 978049 (0.0006) [2023-12-27 12:25:35,287][20347] Updated weights for policy 0, policy_version 978059 (0.0005) [2023-12-27 12:25:35,338][20347] Updated weights for policy 0, policy_version 978069 (0.0008) [2023-12-27 12:25:35,417][20381] Updated weights for policy 1, policy_version 977314 (0.0008) [2023-12-27 12:25:35,474][20381] Updated weights for policy 1, policy_version 977324 (0.0008) [2023-12-27 12:25:35,531][20381] Updated weights for policy 1, policy_version 977334 (0.0008) [2023-12-27 12:25:35,947][20347] Updated weights for policy 0, policy_version 978079 (0.0006) [2023-12-27 12:25:36,001][20347] Updated weights for policy 0, policy_version 978089 (0.0007) [2023-12-27 12:25:36,048][20347] Updated weights for policy 0, policy_version 978099 (0.0009) [2023-12-27 12:25:36,245][20381] Updated weights for policy 1, policy_version 977344 (0.0007) [2023-12-27 12:25:36,306][20381] Updated weights for policy 1, policy_version 977354 (0.0009) [2023-12-27 12:25:36,371][20381] Updated weights for policy 1, policy_version 977364 (0.0007) [2023-12-27 12:25:36,758][20347] Updated weights for policy 0, policy_version 978109 (0.0008) [2023-12-27 12:25:36,817][20347] Updated weights for policy 0, policy_version 978119 (0.0008) [2023-12-27 12:25:36,861][20347] Updated weights for policy 0, policy_version 978129 (0.0007) [2023-12-27 12:25:37,078][20381] Updated weights for policy 1, policy_version 977374 (0.0007) [2023-12-27 12:25:37,134][20381] Updated weights for policy 1, policy_version 977384 (0.0011) [2023-12-27 12:25:37,191][20381] Updated weights for policy 1, policy_version 977394 (0.0011) [2023-12-27 12:25:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20299.5). Total num frames: 500686848. Throughput: 0: 10201.3, 1: 10084.1. Samples: 500654740. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:37,363][19170] Avg episode reward: [(0, '922.610'), (1, '854.380')] [2023-12-27 12:25:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000978136_250445824.pth... [2023-12-27 12:25:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000977400_250241024.pth... [2023-12-27 12:25:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000976952_250142720.pth [2023-12-27 12:25:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000976216_249937920.pth [2023-12-27 12:25:37,540][20347] Updated weights for policy 0, policy_version 978139 (0.0007) [2023-12-27 12:25:37,596][20347] Updated weights for policy 0, policy_version 978149 (0.0007) [2023-12-27 12:25:37,654][20347] Updated weights for policy 0, policy_version 978159 (0.0007) [2023-12-27 12:25:37,655][20291] KL-divergence is very high: 123.9709 [2023-12-27 12:25:37,862][20381] Updated weights for policy 1, policy_version 977404 (0.0011) [2023-12-27 12:25:37,923][20381] Updated weights for policy 1, policy_version 977414 (0.0008) [2023-12-27 12:25:37,981][20381] Updated weights for policy 1, policy_version 977424 (0.0005) [2023-12-27 12:25:38,356][20347] Updated weights for policy 0, policy_version 978169 (0.0008) [2023-12-27 12:25:38,411][20347] Updated weights for policy 0, policy_version 978179 (0.0008) [2023-12-27 12:25:38,430][20291] KL-divergence is very high: 109.7643 [2023-12-27 12:25:38,467][20347] Updated weights for policy 0, policy_version 978189 (0.0008) [2023-12-27 12:25:38,525][20347] Updated weights for policy 0, policy_version 978199 (0.0008) [2023-12-27 12:25:38,634][20381] Updated weights for policy 1, policy_version 977434 (0.0009) [2023-12-27 12:25:38,685][20381] Updated weights for policy 1, policy_version 977444 (0.0007) [2023-12-27 12:25:38,736][20381] Updated weights for policy 1, policy_version 977454 (0.0007) [2023-12-27 12:25:38,790][20381] Updated weights for policy 1, policy_version 977464 (0.0007) [2023-12-27 12:25:39,190][20347] Updated weights for policy 0, policy_version 978209 (0.0006) [2023-12-27 12:25:39,248][20347] Updated weights for policy 0, policy_version 978219 (0.0006) [2023-12-27 12:25:39,307][20347] Updated weights for policy 0, policy_version 978229 (0.0007) [2023-12-27 12:25:39,444][20381] Updated weights for policy 1, policy_version 977474 (0.0007) [2023-12-27 12:25:39,507][20381] Updated weights for policy 1, policy_version 977484 (0.0009) [2023-12-27 12:25:39,557][20381] Updated weights for policy 1, policy_version 977494 (0.0010) [2023-12-27 12:25:40,010][20347] Updated weights for policy 0, policy_version 978239 (0.0006) [2023-12-27 12:25:40,071][20347] Updated weights for policy 0, policy_version 978249 (0.0006) [2023-12-27 12:25:40,136][20347] Updated weights for policy 0, policy_version 978259 (0.0008) [2023-12-27 12:25:40,297][20381] Updated weights for policy 1, policy_version 977504 (0.0008) [2023-12-27 12:25:40,358][20381] Updated weights for policy 1, policy_version 977514 (0.0008) [2023-12-27 12:25:40,421][20381] Updated weights for policy 1, policy_version 977524 (0.0008) [2023-12-27 12:25:40,798][20347] Updated weights for policy 0, policy_version 978269 (0.0006) [2023-12-27 12:25:40,850][20291] KL-divergence is very high: 132.7054 [2023-12-27 12:25:40,861][20347] Updated weights for policy 0, policy_version 978279 (0.0006) [2023-12-27 12:25:40,917][20347] Updated weights for policy 0, policy_version 978289 (0.0007) [2023-12-27 12:25:41,115][20381] Updated weights for policy 1, policy_version 977534 (0.0008) [2023-12-27 12:25:41,184][20381] Updated weights for policy 1, policy_version 977544 (0.0008) [2023-12-27 12:25:41,240][20381] Updated weights for policy 1, policy_version 977554 (0.0008) [2023-12-27 12:25:41,689][20347] Updated weights for policy 0, policy_version 978299 (0.0008) [2023-12-27 12:25:41,723][20291] KL-divergence is very high: 208.1573 [2023-12-27 12:25:41,758][20347] Updated weights for policy 0, policy_version 978309 (0.0010) [2023-12-27 12:25:41,824][20347] Updated weights for policy 0, policy_version 978319 (0.0008) [2023-12-27 12:25:42,013][20381] Updated weights for policy 1, policy_version 977564 (0.0008) [2023-12-27 12:25:42,069][20381] Updated weights for policy 1, policy_version 977574 (0.0006) [2023-12-27 12:25:42,114][20381] Updated weights for policy 1, policy_version 977584 (0.0005) [2023-12-27 12:25:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 500785152. Throughput: 0: 10148.7, 1: 10068.0. Samples: 500775708. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:42,363][19170] Avg episode reward: [(0, '767.740'), (1, '805.690')] [2023-12-27 12:25:42,469][20347] Updated weights for policy 0, policy_version 978329 (0.0008) [2023-12-27 12:25:42,526][20347] Updated weights for policy 0, policy_version 978339 (0.0007) [2023-12-27 12:25:42,585][20347] Updated weights for policy 0, policy_version 978349 (0.0006) [2023-12-27 12:25:42,645][20347] Updated weights for policy 0, policy_version 978359 (0.0007) [2023-12-27 12:25:42,876][20381] Updated weights for policy 1, policy_version 977594 (0.0007) [2023-12-27 12:25:42,944][20381] Updated weights for policy 1, policy_version 977604 (0.0008) [2023-12-27 12:25:43,010][20381] Updated weights for policy 1, policy_version 977614 (0.0009) [2023-12-27 12:25:43,072][20381] Updated weights for policy 1, policy_version 977624 (0.0008) [2023-12-27 12:25:43,298][20347] Updated weights for policy 0, policy_version 978369 (0.0008) [2023-12-27 12:25:43,350][20347] Updated weights for policy 0, policy_version 978379 (0.0010) [2023-12-27 12:25:43,398][20347] Updated weights for policy 0, policy_version 978389 (0.0010) [2023-12-27 12:25:43,696][20381] Updated weights for policy 1, policy_version 977634 (0.0005) [2023-12-27 12:25:43,757][20381] Updated weights for policy 1, policy_version 977644 (0.0005) [2023-12-27 12:25:43,814][20381] Updated weights for policy 1, policy_version 977654 (0.0007) [2023-12-27 12:25:43,999][20347] Updated weights for policy 0, policy_version 978399 (0.0010) [2023-12-27 12:25:44,048][20347] Updated weights for policy 0, policy_version 978409 (0.0010) [2023-12-27 12:25:44,109][20347] Updated weights for policy 0, policy_version 978419 (0.0010) [2023-12-27 12:25:44,430][20381] Updated weights for policy 1, policy_version 977664 (0.0007) [2023-12-27 12:25:44,478][20381] Updated weights for policy 1, policy_version 977674 (0.0007) [2023-12-27 12:25:44,533][20381] Updated weights for policy 1, policy_version 977684 (0.0007) [2023-12-27 12:25:44,804][20347] Updated weights for policy 0, policy_version 978429 (0.0009) [2023-12-27 12:25:44,856][20347] Updated weights for policy 0, policy_version 978439 (0.0005) [2023-12-27 12:25:44,911][20347] Updated weights for policy 0, policy_version 978449 (0.0006) [2023-12-27 12:25:45,295][20381] Updated weights for policy 1, policy_version 977694 (0.0007) [2023-12-27 12:25:45,356][20381] Updated weights for policy 1, policy_version 977704 (0.0007) [2023-12-27 12:25:45,415][20381] Updated weights for policy 1, policy_version 977714 (0.0008) [2023-12-27 12:25:45,577][20347] Updated weights for policy 0, policy_version 978459 (0.0006) [2023-12-27 12:25:45,633][20347] Updated weights for policy 0, policy_version 978469 (0.0008) [2023-12-27 12:25:45,689][20347] Updated weights for policy 0, policy_version 978479 (0.0008) [2023-12-27 12:25:46,100][20381] Updated weights for policy 1, policy_version 977724 (0.0008) [2023-12-27 12:25:46,156][20381] Updated weights for policy 1, policy_version 977734 (0.0008) [2023-12-27 12:25:46,214][20381] Updated weights for policy 1, policy_version 977744 (0.0008) [2023-12-27 12:25:46,402][20347] Updated weights for policy 0, policy_version 978489 (0.0007) [2023-12-27 12:25:46,462][20347] Updated weights for policy 0, policy_version 978499 (0.0007) [2023-12-27 12:25:46,492][20291] KL-divergence is very high: 140.6903 [2023-12-27 12:25:46,522][20347] Updated weights for policy 0, policy_version 978509 (0.0008) [2023-12-27 12:25:46,578][20347] Updated weights for policy 0, policy_version 978519 (0.0007) [2023-12-27 12:25:46,863][20381] Updated weights for policy 1, policy_version 977754 (0.0006) [2023-12-27 12:25:46,921][20381] Updated weights for policy 1, policy_version 977764 (0.0010) [2023-12-27 12:25:46,976][20381] Updated weights for policy 1, policy_version 977774 (0.0010) [2023-12-27 12:25:47,028][20381] Updated weights for policy 1, policy_version 977784 (0.0010) [2023-12-27 12:25:47,181][20347] Updated weights for policy 0, policy_version 978529 (0.0005) [2023-12-27 12:25:47,232][20347] Updated weights for policy 0, policy_version 978539 (0.0005) [2023-12-27 12:25:47,282][20347] Updated weights for policy 0, policy_version 978549 (0.0005) [2023-12-27 12:25:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20271.7). Total num frames: 500891648. Throughput: 0: 10220.0, 1: 10062.1. Samples: 500898764. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:47,363][19170] Avg episode reward: [(0, '664.970'), (1, '863.710')] [2023-12-27 12:25:47,611][20381] Updated weights for policy 1, policy_version 977794 (0.0011) [2023-12-27 12:25:47,672][20381] Updated weights for policy 1, policy_version 977804 (0.0010) [2023-12-27 12:25:47,733][20381] Updated weights for policy 1, policy_version 977814 (0.0010) [2023-12-27 12:25:47,865][20347] Updated weights for policy 0, policy_version 978559 (0.0007) [2023-12-27 12:25:47,928][20347] Updated weights for policy 0, policy_version 978569 (0.0009) [2023-12-27 12:25:47,991][20347] Updated weights for policy 0, policy_version 978579 (0.0007) [2023-12-27 12:25:48,315][20381] Updated weights for policy 1, policy_version 977824 (0.0011) [2023-12-27 12:25:48,382][20381] Updated weights for policy 1, policy_version 977834 (0.0009) [2023-12-27 12:25:48,438][20381] Updated weights for policy 1, policy_version 977844 (0.0009) [2023-12-27 12:25:48,671][20347] Updated weights for policy 0, policy_version 978589 (0.0007) [2023-12-27 12:25:48,732][20347] Updated weights for policy 0, policy_version 978599 (0.0007) [2023-12-27 12:25:48,783][20347] Updated weights for policy 0, policy_version 978609 (0.0007) [2023-12-27 12:25:49,100][20381] Updated weights for policy 1, policy_version 977854 (0.0006) [2023-12-27 12:25:49,157][20381] Updated weights for policy 1, policy_version 977864 (0.0006) [2023-12-27 12:25:49,203][20381] Updated weights for policy 1, policy_version 977874 (0.0005) [2023-12-27 12:25:49,507][20347] Updated weights for policy 0, policy_version 978619 (0.0008) [2023-12-27 12:25:49,567][20347] Updated weights for policy 0, policy_version 978629 (0.0007) [2023-12-27 12:25:49,624][20347] Updated weights for policy 0, policy_version 978639 (0.0008) [2023-12-27 12:25:49,883][20381] Updated weights for policy 1, policy_version 977884 (0.0008) [2023-12-27 12:25:49,946][20381] Updated weights for policy 1, policy_version 977894 (0.0008) [2023-12-27 12:25:50,009][20381] Updated weights for policy 1, policy_version 977904 (0.0009) [2023-12-27 12:25:50,392][20347] Updated weights for policy 0, policy_version 978649 (0.0008) [2023-12-27 12:25:50,442][20347] Updated weights for policy 0, policy_version 978659 (0.0007) [2023-12-27 12:25:50,493][20347] Updated weights for policy 0, policy_version 978669 (0.0007) [2023-12-27 12:25:50,551][20347] Updated weights for policy 0, policy_version 978679 (0.0008) [2023-12-27 12:25:50,717][20381] Updated weights for policy 1, policy_version 977914 (0.0008) [2023-12-27 12:25:50,773][20381] Updated weights for policy 1, policy_version 977924 (0.0008) [2023-12-27 12:25:50,828][20381] Updated weights for policy 1, policy_version 977934 (0.0007) [2023-12-27 12:25:50,895][20381] Updated weights for policy 1, policy_version 977944 (0.0006) [2023-12-27 12:25:51,271][20347] Updated weights for policy 0, policy_version 978689 (0.0006) [2023-12-27 12:25:51,330][20347] Updated weights for policy 0, policy_version 978699 (0.0008) [2023-12-27 12:25:51,397][20347] Updated weights for policy 0, policy_version 978709 (0.0008) [2023-12-27 12:25:51,596][20381] Updated weights for policy 1, policy_version 977954 (0.0007) [2023-12-27 12:25:51,663][20381] Updated weights for policy 1, policy_version 977964 (0.0009) [2023-12-27 12:25:51,716][20381] Updated weights for policy 1, policy_version 977974 (0.0007) [2023-12-27 12:25:52,118][20347] Updated weights for policy 0, policy_version 978719 (0.0007) [2023-12-27 12:25:52,177][20347] Updated weights for policy 0, policy_version 978729 (0.0008) [2023-12-27 12:25:52,236][20347] Updated weights for policy 0, policy_version 978739 (0.0008) [2023-12-27 12:25:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20271.7). Total num frames: 500989952. Throughput: 0: 10245.4, 1: 10074.7. Samples: 500961384. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:52,363][19170] Avg episode reward: [(0, '732.970'), (1, '931.770')] [2023-12-27 12:25:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000978744_250601472.pth... [2023-12-27 12:25:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000977528_250290176.pth [2023-12-27 12:25:52,395][20381] Updated weights for policy 1, policy_version 977984 (0.0007) [2023-12-27 12:25:52,445][20381] Updated weights for policy 1, policy_version 977994 (0.0008) [2023-12-27 12:25:52,487][20381] Updated weights for policy 1, policy_version 978004 (0.0007) [2023-12-27 12:25:52,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000978008_250396672.pth... [2023-12-27 12:25:52,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000976824_250093568.pth [2023-12-27 12:25:52,956][20347] Updated weights for policy 0, policy_version 978749 (0.0007) [2023-12-27 12:25:53,016][20347] Updated weights for policy 0, policy_version 978759 (0.0005) [2023-12-27 12:25:53,073][20347] Updated weights for policy 0, policy_version 978769 (0.0006) [2023-12-27 12:25:53,213][20381] Updated weights for policy 1, policy_version 978014 (0.0006) [2023-12-27 12:25:53,258][20381] Updated weights for policy 1, policy_version 978024 (0.0005) [2023-12-27 12:25:53,310][20381] Updated weights for policy 1, policy_version 978034 (0.0008) [2023-12-27 12:25:53,661][20347] Updated weights for policy 0, policy_version 978779 (0.0008) [2023-12-27 12:25:53,712][20347] Updated weights for policy 0, policy_version 978789 (0.0007) [2023-12-27 12:25:53,770][20347] Updated weights for policy 0, policy_version 978799 (0.0005) [2023-12-27 12:25:53,911][20381] Updated weights for policy 1, policy_version 978044 (0.0007) [2023-12-27 12:25:53,972][20381] Updated weights for policy 1, policy_version 978054 (0.0005) [2023-12-27 12:25:54,033][20381] Updated weights for policy 1, policy_version 978064 (0.0006) [2023-12-27 12:25:54,359][20347] Updated weights for policy 0, policy_version 978809 (0.0005) [2023-12-27 12:25:54,406][20347] Updated weights for policy 0, policy_version 978819 (0.0005) [2023-12-27 12:25:54,453][20347] Updated weights for policy 0, policy_version 978829 (0.0005) [2023-12-27 12:25:54,503][20347] Updated weights for policy 0, policy_version 978839 (0.0008) [2023-12-27 12:25:54,647][20381] Updated weights for policy 1, policy_version 978074 (0.0007) [2023-12-27 12:25:54,699][20381] Updated weights for policy 1, policy_version 978084 (0.0008) [2023-12-27 12:25:54,765][20381] Updated weights for policy 1, policy_version 978094 (0.0008) [2023-12-27 12:25:54,821][20381] Updated weights for policy 1, policy_version 978104 (0.0008) [2023-12-27 12:25:55,145][20347] Updated weights for policy 0, policy_version 978849 (0.0008) [2023-12-27 12:25:55,200][20347] Updated weights for policy 0, policy_version 978859 (0.0008) [2023-12-27 12:25:55,264][20347] Updated weights for policy 0, policy_version 978869 (0.0008) [2023-12-27 12:25:55,486][20381] Updated weights for policy 1, policy_version 978114 (0.0008) [2023-12-27 12:25:55,540][20381] Updated weights for policy 1, policy_version 978124 (0.0009) [2023-12-27 12:25:55,596][20381] Updated weights for policy 1, policy_version 978134 (0.0008) [2023-12-27 12:25:55,883][20347] Updated weights for policy 0, policy_version 978879 (0.0009) [2023-12-27 12:25:55,928][20347] Updated weights for policy 0, policy_version 978889 (0.0010) [2023-12-27 12:25:55,992][20347] Updated weights for policy 0, policy_version 978899 (0.0010) [2023-12-27 12:25:56,329][20381] Updated weights for policy 1, policy_version 978144 (0.0008) [2023-12-27 12:25:56,383][20381] Updated weights for policy 1, policy_version 978154 (0.0006) [2023-12-27 12:25:56,441][20381] Updated weights for policy 1, policy_version 978164 (0.0005) [2023-12-27 12:25:56,680][20347] Updated weights for policy 0, policy_version 978909 (0.0010) [2023-12-27 12:25:56,736][20347] Updated weights for policy 0, policy_version 978919 (0.0008) [2023-12-27 12:25:56,789][20347] Updated weights for policy 0, policy_version 978929 (0.0008) [2023-12-27 12:25:57,070][20381] Updated weights for policy 1, policy_version 978174 (0.0007) [2023-12-27 12:25:57,112][20381] Updated weights for policy 1, policy_version 978184 (0.0007) [2023-12-27 12:25:57,161][20381] Updated weights for policy 1, policy_version 978194 (0.0008) [2023-12-27 12:25:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 501096448. Throughput: 0: 10340.9, 1: 10039.5. Samples: 501085864. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:25:57,363][19170] Avg episode reward: [(0, '749.170'), (1, '951.260')] [2023-12-27 12:25:57,452][20347] Updated weights for policy 0, policy_version 978939 (0.0007) [2023-12-27 12:25:57,503][20347] Updated weights for policy 0, policy_version 978949 (0.0008) [2023-12-27 12:25:57,548][20347] Updated weights for policy 0, policy_version 978959 (0.0007) [2023-12-27 12:25:57,857][20381] Updated weights for policy 1, policy_version 978204 (0.0008) [2023-12-27 12:25:57,904][20381] Updated weights for policy 1, policy_version 978214 (0.0006) [2023-12-27 12:25:57,956][20381] Updated weights for policy 1, policy_version 978224 (0.0007) [2023-12-27 12:25:58,271][20347] Updated weights for policy 0, policy_version 978969 (0.0007) [2023-12-27 12:25:58,326][20347] Updated weights for policy 0, policy_version 978979 (0.0008) [2023-12-27 12:25:58,397][20347] Updated weights for policy 0, policy_version 978989 (0.0009) [2023-12-27 12:25:58,464][20347] Updated weights for policy 0, policy_version 978999 (0.0008) [2023-12-27 12:25:58,615][20381] Updated weights for policy 1, policy_version 978234 (0.0007) [2023-12-27 12:25:58,681][20381] Updated weights for policy 1, policy_version 978244 (0.0009) [2023-12-27 12:25:58,748][20381] Updated weights for policy 1, policy_version 978254 (0.0009) [2023-12-27 12:25:58,815][20381] Updated weights for policy 1, policy_version 978264 (0.0008) [2023-12-27 12:25:59,265][20347] Updated weights for policy 0, policy_version 979009 (0.0008) [2023-12-27 12:25:59,273][20291] KL-divergence is very high: 106.6891 [2023-12-27 12:25:59,316][20347] Updated weights for policy 0, policy_version 979019 (0.0008) [2023-12-27 12:25:59,367][20347] Updated weights for policy 0, policy_version 979029 (0.0008) [2023-12-27 12:25:59,524][20381] Updated weights for policy 1, policy_version 978274 (0.0006) [2023-12-27 12:25:59,579][20381] Updated weights for policy 1, policy_version 978284 (0.0008) [2023-12-27 12:25:59,634][20381] Updated weights for policy 1, policy_version 978294 (0.0006) [2023-12-27 12:26:00,140][20347] Updated weights for policy 0, policy_version 979039 (0.0007) [2023-12-27 12:26:00,207][20347] Updated weights for policy 0, policy_version 979049 (0.0006) [2023-12-27 12:26:00,264][20347] Updated weights for policy 0, policy_version 979059 (0.0007) [2023-12-27 12:26:00,277][20381] Updated weights for policy 1, policy_version 978304 (0.0008) [2023-12-27 12:26:00,334][20381] Updated weights for policy 1, policy_version 978314 (0.0007) [2023-12-27 12:26:00,393][20381] Updated weights for policy 1, policy_version 978324 (0.0007) [2023-12-27 12:26:00,994][20347] Updated weights for policy 0, policy_version 979069 (0.0007) [2023-12-27 12:26:01,011][20381] Updated weights for policy 1, policy_version 978334 (0.0005) [2023-12-27 12:26:01,049][20347] Updated weights for policy 0, policy_version 979079 (0.0006) [2023-12-27 12:26:01,070][20381] Updated weights for policy 1, policy_version 978344 (0.0008) [2023-12-27 12:26:01,107][20347] Updated weights for policy 0, policy_version 979089 (0.0006) [2023-12-27 12:26:01,132][20381] Updated weights for policy 1, policy_version 978354 (0.0008) [2023-12-27 12:26:01,773][20381] Updated weights for policy 1, policy_version 978364 (0.0008) [2023-12-27 12:26:01,817][20347] Updated weights for policy 0, policy_version 979099 (0.0007) [2023-12-27 12:26:01,830][20381] Updated weights for policy 1, policy_version 978374 (0.0007) [2023-12-27 12:26:01,877][20381] Updated weights for policy 1, policy_version 978384 (0.0007) [2023-12-27 12:26:01,879][20347] Updated weights for policy 0, policy_version 979109 (0.0008) [2023-12-27 12:26:01,941][20347] Updated weights for policy 0, policy_version 979120 (0.0007) [2023-12-27 12:26:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 501194752. Throughput: 0: 10286.4, 1: 10080.6. Samples: 501206240. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:02,363][19170] Avg episode reward: [(0, '742.120'), (1, '922.030')] [2023-12-27 12:26:02,566][20347] Updated weights for policy 0, policy_version 979130 (0.0010) [2023-12-27 12:26:02,609][20381] Updated weights for policy 1, policy_version 978394 (0.0008) [2023-12-27 12:26:02,624][20347] Updated weights for policy 0, policy_version 979140 (0.0010) [2023-12-27 12:26:02,663][20381] Updated weights for policy 1, policy_version 978404 (0.0006) [2023-12-27 12:26:02,683][20347] Updated weights for policy 0, policy_version 979150 (0.0011) [2023-12-27 12:26:02,715][20381] Updated weights for policy 1, policy_version 978414 (0.0005) [2023-12-27 12:26:02,732][20347] Updated weights for policy 0, policy_version 979160 (0.0010) [2023-12-27 12:26:02,763][20381] Updated weights for policy 1, policy_version 978424 (0.0007) [2023-12-27 12:26:03,364][20347] Updated weights for policy 0, policy_version 979170 (0.0006) [2023-12-27 12:26:03,418][20347] Updated weights for policy 0, policy_version 979180 (0.0006) [2023-12-27 12:26:03,480][20347] Updated weights for policy 0, policy_version 979190 (0.0006) [2023-12-27 12:26:03,502][20381] Updated weights for policy 1, policy_version 978434 (0.0008) [2023-12-27 12:26:03,551][20381] Updated weights for policy 1, policy_version 978444 (0.0007) [2023-12-27 12:26:03,604][20381] Updated weights for policy 1, policy_version 978454 (0.0009) [2023-12-27 12:26:04,026][20347] Updated weights for policy 0, policy_version 979200 (0.0010) [2023-12-27 12:26:04,081][20347] Updated weights for policy 0, policy_version 979210 (0.0010) [2023-12-27 12:26:04,141][20347] Updated weights for policy 0, policy_version 979220 (0.0011) [2023-12-27 12:26:04,342][20381] Updated weights for policy 1, policy_version 978464 (0.0007) [2023-12-27 12:26:04,403][20381] Updated weights for policy 1, policy_version 978474 (0.0008) [2023-12-27 12:26:04,458][20381] Updated weights for policy 1, policy_version 978484 (0.0007) [2023-12-27 12:26:04,859][20347] Updated weights for policy 0, policy_version 979230 (0.0009) [2023-12-27 12:26:04,914][20347] Updated weights for policy 0, policy_version 979240 (0.0009) [2023-12-27 12:26:04,975][20347] Updated weights for policy 0, policy_version 979250 (0.0010) [2023-12-27 12:26:05,108][20381] Updated weights for policy 1, policy_version 978494 (0.0006) [2023-12-27 12:26:05,163][20381] Updated weights for policy 1, policy_version 978504 (0.0007) [2023-12-27 12:26:05,213][20381] Updated weights for policy 1, policy_version 978514 (0.0007) [2023-12-27 12:26:05,612][20347] Updated weights for policy 0, policy_version 979260 (0.0009) [2023-12-27 12:26:05,673][20347] Updated weights for policy 0, policy_version 979270 (0.0007) [2023-12-27 12:26:05,729][20347] Updated weights for policy 0, policy_version 979280 (0.0010) [2023-12-27 12:26:05,823][20381] Updated weights for policy 1, policy_version 978524 (0.0007) [2023-12-27 12:26:05,881][20381] Updated weights for policy 1, policy_version 978534 (0.0007) [2023-12-27 12:26:05,943][20381] Updated weights for policy 1, policy_version 978544 (0.0006) [2023-12-27 12:26:06,387][20347] Updated weights for policy 0, policy_version 979290 (0.0010) [2023-12-27 12:26:06,446][20347] Updated weights for policy 0, policy_version 979300 (0.0010) [2023-12-27 12:26:06,520][20347] Updated weights for policy 0, policy_version 979310 (0.0009) [2023-12-27 12:26:06,579][20347] Updated weights for policy 0, policy_version 979320 (0.0007) [2023-12-27 12:26:06,686][20381] Updated weights for policy 1, policy_version 978554 (0.0011) [2023-12-27 12:26:06,741][20381] Updated weights for policy 1, policy_version 978564 (0.0011) [2023-12-27 12:26:06,799][20381] Updated weights for policy 1, policy_version 978574 (0.0010) [2023-12-27 12:26:06,848][20381] Updated weights for policy 1, policy_version 978584 (0.0011) [2023-12-27 12:26:07,216][20347] Updated weights for policy 0, policy_version 979330 (0.0010) [2023-12-27 12:26:07,270][20347] Updated weights for policy 0, policy_version 979340 (0.0008) [2023-12-27 12:26:07,327][20347] Updated weights for policy 0, policy_version 979350 (0.0006) [2023-12-27 12:26:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20271.7). Total num frames: 501301248. Throughput: 0: 10333.3, 1: 10086.6. Samples: 501267644. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:07,363][19170] Avg episode reward: [(0, '462.080'), (1, '912.370')] [2023-12-27 12:26:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000978584_250544128.pth... [2023-12-27 12:26:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000979352_250757120.pth... [2023-12-27 12:26:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000978136_250445824.pth [2023-12-27 12:26:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000977400_250241024.pth [2023-12-27 12:26:07,567][20381] Updated weights for policy 1, policy_version 978594 (0.0011) [2023-12-27 12:26:07,630][20381] Updated weights for policy 1, policy_version 978604 (0.0010) [2023-12-27 12:26:07,689][20381] Updated weights for policy 1, policy_version 978614 (0.0011) [2023-12-27 12:26:07,902][20347] Updated weights for policy 0, policy_version 979360 (0.0011) [2023-12-27 12:26:07,964][20347] Updated weights for policy 0, policy_version 979370 (0.0010) [2023-12-27 12:26:08,023][20347] Updated weights for policy 0, policy_version 979380 (0.0010) [2023-12-27 12:26:08,381][20381] Updated weights for policy 1, policy_version 978624 (0.0008) [2023-12-27 12:26:08,436][20381] Updated weights for policy 1, policy_version 978634 (0.0008) [2023-12-27 12:26:08,495][20381] Updated weights for policy 1, policy_version 978644 (0.0008) [2023-12-27 12:26:08,683][20347] Updated weights for policy 0, policy_version 979390 (0.0009) [2023-12-27 12:26:08,741][20347] Updated weights for policy 0, policy_version 979400 (0.0010) [2023-12-27 12:26:08,806][20347] Updated weights for policy 0, policy_version 979410 (0.0010) [2023-12-27 12:26:09,127][20381] Updated weights for policy 1, policy_version 978654 (0.0011) [2023-12-27 12:26:09,177][20381] Updated weights for policy 1, policy_version 978664 (0.0010) [2023-12-27 12:26:09,255][20381] Updated weights for policy 1, policy_version 978674 (0.0011) [2023-12-27 12:26:09,617][20347] Updated weights for policy 0, policy_version 979420 (0.0010) [2023-12-27 12:26:09,686][20347] Updated weights for policy 0, policy_version 979430 (0.0011) [2023-12-27 12:26:09,751][20347] Updated weights for policy 0, policy_version 979440 (0.0009) [2023-12-27 12:26:10,004][20381] Updated weights for policy 1, policy_version 978684 (0.0009) [2023-12-27 12:26:10,056][20381] Updated weights for policy 1, policy_version 978694 (0.0009) [2023-12-27 12:26:10,109][20381] Updated weights for policy 1, policy_version 978704 (0.0009) [2023-12-27 12:26:10,410][20347] Updated weights for policy 0, policy_version 979450 (0.0010) [2023-12-27 12:26:10,478][20347] Updated weights for policy 0, policy_version 979460 (0.0007) [2023-12-27 12:26:10,537][20347] Updated weights for policy 0, policy_version 979470 (0.0006) [2023-12-27 12:26:10,597][20347] Updated weights for policy 0, policy_version 979480 (0.0006) [2023-12-27 12:26:10,882][20381] Updated weights for policy 1, policy_version 978714 (0.0008) [2023-12-27 12:26:10,938][20381] Updated weights for policy 1, policy_version 978724 (0.0007) [2023-12-27 12:26:10,996][20381] Updated weights for policy 1, policy_version 978734 (0.0010) [2023-12-27 12:26:11,062][20381] Updated weights for policy 1, policy_version 978744 (0.0009) [2023-12-27 12:26:11,283][20347] Updated weights for policy 0, policy_version 979490 (0.0009) [2023-12-27 12:26:11,347][20347] Updated weights for policy 0, policy_version 979500 (0.0009) [2023-12-27 12:26:11,407][20347] Updated weights for policy 0, policy_version 979510 (0.0012) [2023-12-27 12:26:11,768][20381] Updated weights for policy 1, policy_version 978754 (0.0008) [2023-12-27 12:26:11,825][20381] Updated weights for policy 1, policy_version 978764 (0.0008) [2023-12-27 12:26:11,892][20381] Updated weights for policy 1, policy_version 978774 (0.0007) [2023-12-27 12:26:12,148][20347] Updated weights for policy 0, policy_version 979520 (0.0008) [2023-12-27 12:26:12,199][20347] Updated weights for policy 0, policy_version 979530 (0.0006) [2023-12-27 12:26:12,261][20347] Updated weights for policy 0, policy_version 979540 (0.0007) [2023-12-27 12:26:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 501399552. Throughput: 0: 10318.7, 1: 10081.5. Samples: 501388824. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:12,363][19170] Avg episode reward: [(0, '579.950'), (1, '931.860')] [2023-12-27 12:26:12,630][20381] Updated weights for policy 1, policy_version 978784 (0.0009) [2023-12-27 12:26:12,687][20381] Updated weights for policy 1, policy_version 978794 (0.0007) [2023-12-27 12:26:12,744][20381] Updated weights for policy 1, policy_version 978804 (0.0010) [2023-12-27 12:26:12,949][20347] Updated weights for policy 0, policy_version 979550 (0.0007) [2023-12-27 12:26:13,005][20347] Updated weights for policy 0, policy_version 979560 (0.0007) [2023-12-27 12:26:13,058][20347] Updated weights for policy 0, policy_version 979570 (0.0007) [2023-12-27 12:26:13,439][20381] Updated weights for policy 1, policy_version 978814 (0.0008) [2023-12-27 12:26:13,493][20381] Updated weights for policy 1, policy_version 978824 (0.0006) [2023-12-27 12:26:13,544][20381] Updated weights for policy 1, policy_version 978834 (0.0005) [2023-12-27 12:26:13,656][20347] Updated weights for policy 0, policy_version 979580 (0.0006) [2023-12-27 12:26:13,707][20347] Updated weights for policy 0, policy_version 979590 (0.0005) [2023-12-27 12:26:13,758][20347] Updated weights for policy 0, policy_version 979600 (0.0005) [2023-12-27 12:26:14,162][20381] Updated weights for policy 1, policy_version 978844 (0.0006) [2023-12-27 12:26:14,218][20381] Updated weights for policy 1, policy_version 978854 (0.0007) [2023-12-27 12:26:14,275][20381] Updated weights for policy 1, policy_version 978864 (0.0007) [2023-12-27 12:26:14,317][20347] Updated weights for policy 0, policy_version 979610 (0.0007) [2023-12-27 12:26:14,372][20347] Updated weights for policy 0, policy_version 979620 (0.0008) [2023-12-27 12:26:14,437][20347] Updated weights for policy 0, policy_version 979630 (0.0005) [2023-12-27 12:26:14,497][20347] Updated weights for policy 0, policy_version 979640 (0.0006) [2023-12-27 12:26:15,031][20381] Updated weights for policy 1, policy_version 978874 (0.0007) [2023-12-27 12:26:15,072][20347] Updated weights for policy 0, policy_version 979650 (0.0006) [2023-12-27 12:26:15,090][20381] Updated weights for policy 1, policy_version 978884 (0.0008) [2023-12-27 12:26:15,132][20347] Updated weights for policy 0, policy_version 979660 (0.0006) [2023-12-27 12:26:15,144][20381] Updated weights for policy 1, policy_version 978894 (0.0007) [2023-12-27 12:26:15,194][20347] Updated weights for policy 0, policy_version 979670 (0.0006) [2023-12-27 12:26:15,208][20381] Updated weights for policy 1, policy_version 978904 (0.0008) [2023-12-27 12:26:15,817][20347] Updated weights for policy 0, policy_version 979680 (0.0006) [2023-12-27 12:26:15,883][20347] Updated weights for policy 0, policy_version 979690 (0.0009) [2023-12-27 12:26:15,945][20347] Updated weights for policy 0, policy_version 979700 (0.0011) [2023-12-27 12:26:15,957][20381] Updated weights for policy 1, policy_version 978914 (0.0008) [2023-12-27 12:26:16,006][20381] Updated weights for policy 1, policy_version 978924 (0.0007) [2023-12-27 12:26:16,048][20292] KL-divergence is very high: 113.9245 [2023-12-27 12:26:16,061][20381] Updated weights for policy 1, policy_version 978934 (0.0007) [2023-12-27 12:26:16,621][20347] Updated weights for policy 0, policy_version 979710 (0.0010) [2023-12-27 12:26:16,670][20347] Updated weights for policy 0, policy_version 979720 (0.0010) [2023-12-27 12:26:16,718][20347] Updated weights for policy 0, policy_version 979730 (0.0010) [2023-12-27 12:26:16,781][20381] Updated weights for policy 1, policy_version 978944 (0.0007) [2023-12-27 12:26:16,838][20381] Updated weights for policy 1, policy_version 978954 (0.0009) [2023-12-27 12:26:16,899][20381] Updated weights for policy 1, policy_version 978964 (0.0005) [2023-12-27 12:26:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20207.0, 300 sec: 20244.0). Total num frames: 501497856. Throughput: 0: 10307.7, 1: 10115.1. Samples: 501512620. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:17,363][19170] Avg episode reward: [(0, '601.610'), (1, '912.330')] [2023-12-27 12:26:17,366][20347] Updated weights for policy 0, policy_version 979740 (0.0008) [2023-12-27 12:26:17,425][20347] Updated weights for policy 0, policy_version 979750 (0.0006) [2023-12-27 12:26:17,454][20381] Updated weights for policy 1, policy_version 978974 (0.0008) [2023-12-27 12:26:17,480][20347] Updated weights for policy 0, policy_version 979760 (0.0006) [2023-12-27 12:26:17,499][20381] Updated weights for policy 1, policy_version 978984 (0.0010) [2023-12-27 12:26:17,547][20381] Updated weights for policy 1, policy_version 978994 (0.0010) [2023-12-27 12:26:18,154][20347] Updated weights for policy 0, policy_version 979770 (0.0007) [2023-12-27 12:26:18,173][20381] Updated weights for policy 1, policy_version 979004 (0.0011) [2023-12-27 12:26:18,208][20347] Updated weights for policy 0, policy_version 979780 (0.0006) [2023-12-27 12:26:18,233][20381] Updated weights for policy 1, policy_version 979014 (0.0011) [2023-12-27 12:26:18,267][20347] Updated weights for policy 0, policy_version 979790 (0.0007) [2023-12-27 12:26:18,295][20381] Updated weights for policy 1, policy_version 979024 (0.0010) [2023-12-27 12:26:18,316][20347] Updated weights for policy 0, policy_version 979800 (0.0005) [2023-12-27 12:26:18,942][20381] Updated weights for policy 1, policy_version 979034 (0.0009) [2023-12-27 12:26:18,996][20381] Updated weights for policy 1, policy_version 979044 (0.0009) [2023-12-27 12:26:19,020][20347] Updated weights for policy 0, policy_version 979810 (0.0006) [2023-12-27 12:26:19,052][20381] Updated weights for policy 1, policy_version 979054 (0.0011) [2023-12-27 12:26:19,078][20347] Updated weights for policy 0, policy_version 979820 (0.0006) [2023-12-27 12:26:19,106][20381] Updated weights for policy 1, policy_version 979064 (0.0011) [2023-12-27 12:26:19,135][20347] Updated weights for policy 0, policy_version 979830 (0.0006) [2023-12-27 12:26:19,820][20347] Updated weights for policy 0, policy_version 979840 (0.0009) [2023-12-27 12:26:19,840][20381] Updated weights for policy 1, policy_version 979074 (0.0008) [2023-12-27 12:26:19,886][20347] Updated weights for policy 0, policy_version 979850 (0.0008) [2023-12-27 12:26:19,906][20381] Updated weights for policy 1, policy_version 979084 (0.0009) [2023-12-27 12:26:19,953][20347] Updated weights for policy 0, policy_version 979860 (0.0007) [2023-12-27 12:26:19,972][20381] Updated weights for policy 1, policy_version 979094 (0.0011) [2023-12-27 12:26:20,657][20347] Updated weights for policy 0, policy_version 979870 (0.0006) [2023-12-27 12:26:20,659][20381] Updated weights for policy 1, policy_version 979104 (0.0007) [2023-12-27 12:26:20,712][20381] Updated weights for policy 1, policy_version 979114 (0.0010) [2023-12-27 12:26:20,714][20347] Updated weights for policy 0, policy_version 979880 (0.0005) [2023-12-27 12:26:20,771][20347] Updated weights for policy 0, policy_version 979890 (0.0007) [2023-12-27 12:26:20,772][20381] Updated weights for policy 1, policy_version 979124 (0.0007) [2023-12-27 12:26:21,496][20381] Updated weights for policy 1, policy_version 979134 (0.0008) [2023-12-27 12:26:21,536][20347] Updated weights for policy 0, policy_version 979900 (0.0008) [2023-12-27 12:26:21,552][20381] Updated weights for policy 1, policy_version 979144 (0.0010) [2023-12-27 12:26:21,592][20347] Updated weights for policy 0, policy_version 979910 (0.0006) [2023-12-27 12:26:21,614][20381] Updated weights for policy 1, policy_version 979154 (0.0011) [2023-12-27 12:26:21,666][20347] Updated weights for policy 0, policy_version 979920 (0.0008) [2023-12-27 12:26:22,345][20381] Updated weights for policy 1, policy_version 979164 (0.0011) [2023-12-27 12:26:22,354][20347] Updated weights for policy 0, policy_version 979930 (0.0008) [2023-12-27 12:26:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 501596160. Throughput: 0: 10290.7, 1: 10135.0. Samples: 501573900. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:22,363][19170] Avg episode reward: [(0, '608.380'), (1, '922.120')] [2023-12-27 12:26:22,411][20381] Updated weights for policy 1, policy_version 979174 (0.0011) [2023-12-27 12:26:22,422][20347] Updated weights for policy 0, policy_version 979940 (0.0007) [2023-12-27 12:26:22,466][20381] Updated weights for policy 1, policy_version 979184 (0.0010) [2023-12-27 12:26:22,481][20347] Updated weights for policy 0, policy_version 979950 (0.0008) [2023-12-27 12:26:22,520][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000979192_250699776.pth... [2023-12-27 12:26:22,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000978008_250396672.pth [2023-12-27 12:26:22,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000979960_250912768.pth... [2023-12-27 12:26:22,545][20347] Updated weights for policy 0, policy_version 979960 (0.0006) [2023-12-27 12:26:22,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000978744_250601472.pth [2023-12-27 12:26:23,129][20381] Updated weights for policy 1, policy_version 979194 (0.0009) [2023-12-27 12:26:23,185][20381] Updated weights for policy 1, policy_version 979204 (0.0009) [2023-12-27 12:26:23,238][20381] Updated weights for policy 1, policy_version 979215 (0.0009) [2023-12-27 12:26:23,247][20347] Updated weights for policy 0, policy_version 979970 (0.0010) [2023-12-27 12:26:23,294][20347] Updated weights for policy 0, policy_version 979980 (0.0010) [2023-12-27 12:26:23,343][20347] Updated weights for policy 0, policy_version 979990 (0.0009) [2023-12-27 12:26:23,885][20381] Updated weights for policy 1, policy_version 979225 (0.0008) [2023-12-27 12:26:23,938][20381] Updated weights for policy 1, policy_version 979235 (0.0005) [2023-12-27 12:26:23,989][20381] Updated weights for policy 1, policy_version 979245 (0.0006) [2023-12-27 12:26:24,038][20381] Updated weights for policy 1, policy_version 979255 (0.0007) [2023-12-27 12:26:24,060][20347] Updated weights for policy 0, policy_version 980000 (0.0006) [2023-12-27 12:26:24,119][20347] Updated weights for policy 0, policy_version 980010 (0.0006) [2023-12-27 12:26:24,175][20347] Updated weights for policy 0, policy_version 980020 (0.0006) [2023-12-27 12:26:24,724][20347] Updated weights for policy 0, policy_version 980030 (0.0006) [2023-12-27 12:26:24,764][20381] Updated weights for policy 1, policy_version 979265 (0.0005) [2023-12-27 12:26:24,776][20347] Updated weights for policy 0, policy_version 980040 (0.0006) [2023-12-27 12:26:24,809][20381] Updated weights for policy 1, policy_version 979275 (0.0005) [2023-12-27 12:26:24,828][20347] Updated weights for policy 0, policy_version 980050 (0.0007) [2023-12-27 12:26:24,852][20381] Updated weights for policy 1, policy_version 979285 (0.0008) [2023-12-27 12:26:25,462][20347] Updated weights for policy 0, policy_version 980060 (0.0008) [2023-12-27 12:26:25,516][20347] Updated weights for policy 0, policy_version 980070 (0.0008) [2023-12-27 12:26:25,574][20347] Updated weights for policy 0, policy_version 980080 (0.0007) [2023-12-27 12:26:25,615][20381] Updated weights for policy 1, policy_version 979295 (0.0006) [2023-12-27 12:26:25,670][20381] Updated weights for policy 1, policy_version 979305 (0.0007) [2023-12-27 12:26:25,723][20381] Updated weights for policy 1, policy_version 979315 (0.0006) [2023-12-27 12:26:26,208][20347] Updated weights for policy 0, policy_version 980090 (0.0008) [2023-12-27 12:26:26,274][20347] Updated weights for policy 0, policy_version 980100 (0.0008) [2023-12-27 12:26:26,334][20347] Updated weights for policy 0, policy_version 980110 (0.0008) [2023-12-27 12:26:26,338][20381] Updated weights for policy 1, policy_version 979325 (0.0010) [2023-12-27 12:26:26,384][20347] Updated weights for policy 0, policy_version 980120 (0.0006) [2023-12-27 12:26:26,393][20381] Updated weights for policy 1, policy_version 979335 (0.0009) [2023-12-27 12:26:26,454][20381] Updated weights for policy 1, policy_version 979345 (0.0007) [2023-12-27 12:26:26,998][20381] Updated weights for policy 1, policy_version 979355 (0.0008) [2023-12-27 12:26:27,012][20347] Updated weights for policy 0, policy_version 980130 (0.0006) [2023-12-27 12:26:27,046][20381] Updated weights for policy 1, policy_version 979365 (0.0005) [2023-12-27 12:26:27,057][20347] Updated weights for policy 0, policy_version 980140 (0.0007) [2023-12-27 12:26:27,101][20381] Updated weights for policy 1, policy_version 979375 (0.0010) [2023-12-27 12:26:27,108][20347] Updated weights for policy 0, policy_version 980150 (0.0005) [2023-12-27 12:26:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20479.9, 300 sec: 20271.7). Total num frames: 501710848. Throughput: 0: 10290.2, 1: 10157.9. Samples: 501695872. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:27,363][19170] Avg episode reward: [(0, '628.960'), (1, '951.840')] [2023-12-27 12:26:27,754][20347] Updated weights for policy 0, policy_version 980160 (0.0009) [2023-12-27 12:26:27,758][20381] Updated weights for policy 1, policy_version 979385 (0.0011) [2023-12-27 12:26:27,806][20347] Updated weights for policy 0, policy_version 980170 (0.0010) [2023-12-27 12:26:27,813][20381] Updated weights for policy 1, policy_version 979395 (0.0010) [2023-12-27 12:26:27,851][20347] Updated weights for policy 0, policy_version 980180 (0.0006) [2023-12-27 12:26:27,867][20381] Updated weights for policy 1, policy_version 979405 (0.0009) [2023-12-27 12:26:27,920][20381] Updated weights for policy 1, policy_version 979415 (0.0007) [2023-12-27 12:26:28,515][20347] Updated weights for policy 0, policy_version 980190 (0.0009) [2023-12-27 12:26:28,568][20347] Updated weights for policy 0, policy_version 980200 (0.0009) [2023-12-27 12:26:28,617][20381] Updated weights for policy 1, policy_version 979425 (0.0005) [2023-12-27 12:26:28,618][20347] Updated weights for policy 0, policy_version 980210 (0.0008) [2023-12-27 12:26:28,673][20381] Updated weights for policy 1, policy_version 979435 (0.0005) [2023-12-27 12:26:28,731][20381] Updated weights for policy 1, policy_version 979445 (0.0005) [2023-12-27 12:26:29,294][20381] Updated weights for policy 1, policy_version 979455 (0.0009) [2023-12-27 12:26:29,359][20381] Updated weights for policy 1, policy_version 979465 (0.0011) [2023-12-27 12:26:29,398][20347] Updated weights for policy 0, policy_version 980220 (0.0007) [2023-12-27 12:26:29,411][20381] Updated weights for policy 1, policy_version 979475 (0.0011) [2023-12-27 12:26:29,455][20347] Updated weights for policy 0, policy_version 980230 (0.0006) [2023-12-27 12:26:29,516][20347] Updated weights for policy 0, policy_version 980240 (0.0008) [2023-12-27 12:26:30,091][20381] Updated weights for policy 1, policy_version 979485 (0.0009) [2023-12-27 12:26:30,137][20381] Updated weights for policy 1, policy_version 979495 (0.0007) [2023-12-27 12:26:30,178][20347] Updated weights for policy 0, policy_version 980250 (0.0007) [2023-12-27 12:26:30,196][20381] Updated weights for policy 1, policy_version 979505 (0.0008) [2023-12-27 12:26:30,232][20347] Updated weights for policy 0, policy_version 980260 (0.0009) [2023-12-27 12:26:30,281][20347] Updated weights for policy 0, policy_version 980270 (0.0010) [2023-12-27 12:26:30,331][20347] Updated weights for policy 0, policy_version 980280 (0.0010) [2023-12-27 12:26:30,895][20381] Updated weights for policy 1, policy_version 979515 (0.0006) [2023-12-27 12:26:30,939][20381] Updated weights for policy 1, policy_version 979525 (0.0007) [2023-12-27 12:26:30,982][20347] Updated weights for policy 0, policy_version 980290 (0.0010) [2023-12-27 12:26:30,995][20381] Updated weights for policy 1, policy_version 979535 (0.0007) [2023-12-27 12:26:31,041][20347] Updated weights for policy 0, policy_version 980300 (0.0010) [2023-12-27 12:26:31,095][20347] Updated weights for policy 0, policy_version 980310 (0.0008) [2023-12-27 12:26:31,739][20381] Updated weights for policy 1, policy_version 979545 (0.0007) [2023-12-27 12:26:31,801][20381] Updated weights for policy 1, policy_version 979555 (0.0008) [2023-12-27 12:26:31,849][20347] Updated weights for policy 0, policy_version 980320 (0.0006) [2023-12-27 12:26:31,862][20381] Updated weights for policy 1, policy_version 979565 (0.0008) [2023-12-27 12:26:31,908][20347] Updated weights for policy 0, policy_version 980330 (0.0006) [2023-12-27 12:26:31,920][20381] Updated weights for policy 1, policy_version 979575 (0.0007) [2023-12-27 12:26:31,966][20347] Updated weights for policy 0, policy_version 980340 (0.0007) [2023-12-27 12:26:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 501809152. Throughput: 0: 10264.0, 1: 10210.4. Samples: 501820112. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:32,363][19170] Avg episode reward: [(0, '693.220'), (1, '961.650')] [2023-12-27 12:26:32,590][20381] Updated weights for policy 1, policy_version 979585 (0.0008) [2023-12-27 12:26:32,639][20347] Updated weights for policy 0, policy_version 980350 (0.0006) [2023-12-27 12:26:32,644][20381] Updated weights for policy 1, policy_version 979595 (0.0007) [2023-12-27 12:26:32,696][20347] Updated weights for policy 0, policy_version 980360 (0.0006) [2023-12-27 12:26:32,702][20381] Updated weights for policy 1, policy_version 979605 (0.0005) [2023-12-27 12:26:32,756][20347] Updated weights for policy 0, policy_version 980370 (0.0005) [2023-12-27 12:26:33,344][20347] Updated weights for policy 0, policy_version 980380 (0.0005) [2023-12-27 12:26:33,392][20347] Updated weights for policy 0, policy_version 980390 (0.0005) [2023-12-27 12:26:33,395][20381] Updated weights for policy 1, policy_version 979615 (0.0009) [2023-12-27 12:26:33,441][20347] Updated weights for policy 0, policy_version 980400 (0.0005) [2023-12-27 12:26:33,449][20381] Updated weights for policy 1, policy_version 979625 (0.0008) [2023-12-27 12:26:33,505][20381] Updated weights for policy 1, policy_version 979635 (0.0007) [2023-12-27 12:26:34,046][20347] Updated weights for policy 0, policy_version 980410 (0.0006) [2023-12-27 12:26:34,096][20347] Updated weights for policy 0, policy_version 980420 (0.0010) [2023-12-27 12:26:34,101][20381] Updated weights for policy 1, policy_version 979645 (0.0009) [2023-12-27 12:26:34,155][20347] Updated weights for policy 0, policy_version 980430 (0.0010) [2023-12-27 12:26:34,162][20381] Updated weights for policy 1, policy_version 979655 (0.0006) [2023-12-27 12:26:34,217][20347] Updated weights for policy 0, policy_version 980440 (0.0011) [2023-12-27 12:26:34,221][20381] Updated weights for policy 1, policy_version 979665 (0.0010) [2023-12-27 12:26:34,936][20381] Updated weights for policy 1, policy_version 979675 (0.0010) [2023-12-27 12:26:34,953][20347] Updated weights for policy 0, policy_version 980450 (0.0006) [2023-12-27 12:26:34,988][20381] Updated weights for policy 1, policy_version 979685 (0.0007) [2023-12-27 12:26:35,013][20347] Updated weights for policy 0, policy_version 980460 (0.0006) [2023-12-27 12:26:35,043][20381] Updated weights for policy 1, policy_version 979695 (0.0008) [2023-12-27 12:26:35,067][20347] Updated weights for policy 0, policy_version 980470 (0.0008) [2023-12-27 12:26:35,624][20381] Updated weights for policy 1, policy_version 979705 (0.0007) [2023-12-27 12:26:35,671][20381] Updated weights for policy 1, policy_version 979715 (0.0008) [2023-12-27 12:26:35,717][20381] Updated weights for policy 1, policy_version 979725 (0.0009) [2023-12-27 12:26:35,769][20381] Updated weights for policy 1, policy_version 979735 (0.0010) [2023-12-27 12:26:35,791][20347] Updated weights for policy 0, policy_version 980480 (0.0006) [2023-12-27 12:26:35,850][20347] Updated weights for policy 0, policy_version 980490 (0.0007) [2023-12-27 12:26:35,912][20347] Updated weights for policy 0, policy_version 980500 (0.0008) [2023-12-27 12:26:36,487][20381] Updated weights for policy 1, policy_version 979745 (0.0011) [2023-12-27 12:26:36,549][20381] Updated weights for policy 1, policy_version 979755 (0.0007) [2023-12-27 12:26:36,614][20381] Updated weights for policy 1, policy_version 979765 (0.0009) [2023-12-27 12:26:36,629][20347] Updated weights for policy 0, policy_version 980510 (0.0009) [2023-12-27 12:26:36,691][20347] Updated weights for policy 0, policy_version 980520 (0.0009) [2023-12-27 12:26:36,747][20347] Updated weights for policy 0, policy_version 980530 (0.0007) [2023-12-27 12:26:37,288][20381] Updated weights for policy 1, policy_version 979775 (0.0010) [2023-12-27 12:26:37,351][20381] Updated weights for policy 1, policy_version 979785 (0.0011) [2023-12-27 12:26:37,363][19170] Fps is (10 sec: 19660.0, 60 sec: 20343.3, 300 sec: 20243.9). Total num frames: 501907456. Throughput: 0: 10296.4, 1: 10177.3. Samples: 501882704. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:37,364][19170] Avg episode reward: [(0, '673.060'), (1, '941.570')] [2023-12-27 12:26:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000980536_251060224.pth... [2023-12-27 12:26:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000979352_250757120.pth [2023-12-27 12:26:37,410][20381] Updated weights for policy 1, policy_version 979795 (0.0011) [2023-12-27 12:26:37,435][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000979800_250855424.pth... [2023-12-27 12:26:37,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000978584_250544128.pth [2023-12-27 12:26:37,449][20347] Updated weights for policy 0, policy_version 980540 (0.0006) [2023-12-27 12:26:37,510][20347] Updated weights for policy 0, policy_version 980550 (0.0009) [2023-12-27 12:26:37,568][20347] Updated weights for policy 0, policy_version 980560 (0.0007) [2023-12-27 12:26:38,101][20381] Updated weights for policy 1, policy_version 979805 (0.0008) [2023-12-27 12:26:38,154][20381] Updated weights for policy 1, policy_version 979815 (0.0005) [2023-12-27 12:26:38,187][20347] Updated weights for policy 0, policy_version 980570 (0.0007) [2023-12-27 12:26:38,215][20381] Updated weights for policy 1, policy_version 979825 (0.0008) [2023-12-27 12:26:38,238][20347] Updated weights for policy 0, policy_version 980580 (0.0010) [2023-12-27 12:26:38,292][20347] Updated weights for policy 0, policy_version 980590 (0.0010) [2023-12-27 12:26:38,347][20347] Updated weights for policy 0, policy_version 980600 (0.0012) [2023-12-27 12:26:38,893][20381] Updated weights for policy 1, policy_version 979835 (0.0011) [2023-12-27 12:26:38,948][20292] KL-divergence is very high: 112.5761 [2023-12-27 12:26:38,949][20381] Updated weights for policy 1, policy_version 979845 (0.0011) [2023-12-27 12:26:39,003][20381] Updated weights for policy 1, policy_version 979855 (0.0011) [2023-12-27 12:26:39,067][20347] Updated weights for policy 0, policy_version 980610 (0.0006) [2023-12-27 12:26:39,134][20347] Updated weights for policy 0, policy_version 980620 (0.0006) [2023-12-27 12:26:39,197][20347] Updated weights for policy 0, policy_version 980630 (0.0006) [2023-12-27 12:26:39,748][20381] Updated weights for policy 1, policy_version 979865 (0.0010) [2023-12-27 12:26:39,810][20381] Updated weights for policy 1, policy_version 979875 (0.0006) [2023-12-27 12:26:39,862][20347] Updated weights for policy 0, policy_version 980640 (0.0008) [2023-12-27 12:26:39,876][20381] Updated weights for policy 1, policy_version 979885 (0.0007) [2023-12-27 12:26:39,926][20347] Updated weights for policy 0, policy_version 980650 (0.0008) [2023-12-27 12:26:39,939][20381] Updated weights for policy 1, policy_version 979895 (0.0008) [2023-12-27 12:26:39,981][20347] Updated weights for policy 0, policy_version 980660 (0.0007) [2023-12-27 12:26:40,620][20381] Updated weights for policy 1, policy_version 979905 (0.0007) [2023-12-27 12:26:40,645][20347] Updated weights for policy 0, policy_version 980670 (0.0006) [2023-12-27 12:26:40,681][20381] Updated weights for policy 1, policy_version 979915 (0.0006) [2023-12-27 12:26:40,702][20347] Updated weights for policy 0, policy_version 980680 (0.0005) [2023-12-27 12:26:40,737][20381] Updated weights for policy 1, policy_version 979925 (0.0006) [2023-12-27 12:26:40,758][20347] Updated weights for policy 0, policy_version 980690 (0.0006) [2023-12-27 12:26:41,504][20381] Updated weights for policy 1, policy_version 979935 (0.0007) [2023-12-27 12:26:41,524][20347] Updated weights for policy 0, policy_version 980700 (0.0007) [2023-12-27 12:26:41,558][20381] Updated weights for policy 1, policy_version 979945 (0.0006) [2023-12-27 12:26:41,583][20347] Updated weights for policy 0, policy_version 980710 (0.0006) [2023-12-27 12:26:41,623][20381] Updated weights for policy 1, policy_version 979955 (0.0006) [2023-12-27 12:26:41,648][20347] Updated weights for policy 0, policy_version 980720 (0.0008) [2023-12-27 12:26:42,357][20347] Updated weights for policy 0, policy_version 980730 (0.0008) [2023-12-27 12:26:42,358][20381] Updated weights for policy 1, policy_version 979965 (0.0008) [2023-12-27 12:26:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 502005760. Throughput: 0: 10240.5, 1: 10150.5. Samples: 502003460. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:42,363][19170] Avg episode reward: [(0, '644.020'), (1, '931.790')] [2023-12-27 12:26:42,418][20347] Updated weights for policy 0, policy_version 980740 (0.0008) [2023-12-27 12:26:42,424][20381] Updated weights for policy 1, policy_version 979975 (0.0008) [2023-12-27 12:26:42,481][20381] Updated weights for policy 1, policy_version 979985 (0.0008) [2023-12-27 12:26:42,483][20347] Updated weights for policy 0, policy_version 980750 (0.0007) [2023-12-27 12:26:42,545][20347] Updated weights for policy 0, policy_version 980760 (0.0008) [2023-12-27 12:26:43,163][20381] Updated weights for policy 1, policy_version 979995 (0.0006) [2023-12-27 12:26:43,224][20291] KL-divergence is very high: 234.9072 [2023-12-27 12:26:43,224][20381] Updated weights for policy 1, policy_version 980005 (0.0007) [2023-12-27 12:26:43,240][20291] KL-divergence is very high: 189.0142 [2023-12-27 12:26:43,248][20347] Updated weights for policy 0, policy_version 980770 (0.0005) [2023-12-27 12:26:43,266][20291] KL-divergence is very high: 154.4512 [2023-12-27 12:26:43,285][20381] Updated weights for policy 1, policy_version 980015 (0.0007) [2023-12-27 12:26:43,305][20347] Updated weights for policy 0, policy_version 980780 (0.0007) [2023-12-27 12:26:43,363][20347] Updated weights for policy 0, policy_version 980790 (0.0008) [2023-12-27 12:26:43,950][20381] Updated weights for policy 1, policy_version 980025 (0.0009) [2023-12-27 12:26:43,992][20291] KL-divergence is very high: 294.9261 [2023-12-27 12:26:44,009][20381] Updated weights for policy 1, policy_version 980035 (0.0006) [2023-12-27 12:26:44,033][20347] Updated weights for policy 0, policy_version 980800 (0.0009) [2023-12-27 12:26:44,041][20291] KL-divergence is very high: 126.7560 [2023-12-27 12:26:44,065][20381] Updated weights for policy 1, policy_version 980045 (0.0006) [2023-12-27 12:26:44,096][20291] KL-divergence is very high: 164.2617 [2023-12-27 12:26:44,101][20347] Updated weights for policy 0, policy_version 980810 (0.0010) [2023-12-27 12:26:44,128][20381] Updated weights for policy 1, policy_version 980055 (0.0005) [2023-12-27 12:26:44,148][20291] KL-divergence is very high: 164.6178 [2023-12-27 12:26:44,165][20347] Updated weights for policy 0, policy_version 980820 (0.0010) [2023-12-27 12:26:44,677][20381] Updated weights for policy 1, policy_version 980065 (0.0006) [2023-12-27 12:26:44,738][20381] Updated weights for policy 1, policy_version 980075 (0.0008) [2023-12-27 12:26:44,827][20381] Updated weights for policy 1, policy_version 980085 (0.0008) [2023-12-27 12:26:44,833][20347] Updated weights for policy 0, policy_version 980830 (0.0008) [2023-12-27 12:26:44,896][20347] Updated weights for policy 0, policy_version 980840 (0.0008) [2023-12-27 12:26:44,950][20347] Updated weights for policy 0, policy_version 980850 (0.0006) [2023-12-27 12:26:45,497][20381] Updated weights for policy 1, policy_version 980095 (0.0006) [2023-12-27 12:26:45,553][20381] Updated weights for policy 1, policy_version 980105 (0.0005) [2023-12-27 12:26:45,611][20381] Updated weights for policy 1, policy_version 980115 (0.0005) [2023-12-27 12:26:45,654][20347] Updated weights for policy 0, policy_version 980860 (0.0007) [2023-12-27 12:26:45,708][20347] Updated weights for policy 0, policy_version 980870 (0.0008) [2023-12-27 12:26:45,762][20347] Updated weights for policy 0, policy_version 980880 (0.0007) [2023-12-27 12:26:46,153][20381] Updated weights for policy 1, policy_version 980125 (0.0007) [2023-12-27 12:26:46,216][20381] Updated weights for policy 1, policy_version 980135 (0.0008) [2023-12-27 12:26:46,274][20381] Updated weights for policy 1, policy_version 980145 (0.0008) [2023-12-27 12:26:46,516][20347] Updated weights for policy 0, policy_version 980890 (0.0007) [2023-12-27 12:26:46,566][20347] Updated weights for policy 0, policy_version 980900 (0.0007) [2023-12-27 12:26:46,620][20347] Updated weights for policy 0, policy_version 980910 (0.0007) [2023-12-27 12:26:46,680][20347] Updated weights for policy 0, policy_version 980920 (0.0007) [2023-12-27 12:26:46,969][20381] Updated weights for policy 1, policy_version 980155 (0.0008) [2023-12-27 12:26:47,019][20381] Updated weights for policy 1, policy_version 980165 (0.0008) [2023-12-27 12:26:47,070][20381] Updated weights for policy 1, policy_version 980175 (0.0007) [2023-12-27 12:26:47,349][20347] Updated weights for policy 0, policy_version 980930 (0.0006) [2023-12-27 12:26:47,363][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20243.9). Total num frames: 502112256. Throughput: 0: 10251.3, 1: 10145.0. Samples: 502124080. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:47,364][19170] Avg episode reward: [(0, '733.120'), (1, '931.780')] [2023-12-27 12:26:47,411][20347] Updated weights for policy 0, policy_version 980940 (0.0006) [2023-12-27 12:26:47,467][20347] Updated weights for policy 0, policy_version 980950 (0.0005) [2023-12-27 12:26:47,675][20381] Updated weights for policy 1, policy_version 980185 (0.0008) [2023-12-27 12:26:47,736][20381] Updated weights for policy 1, policy_version 980195 (0.0005) [2023-12-27 12:26:47,792][20381] Updated weights for policy 1, policy_version 980205 (0.0005) [2023-12-27 12:26:47,847][20381] Updated weights for policy 1, policy_version 980215 (0.0008) [2023-12-27 12:26:48,058][20347] Updated weights for policy 0, policy_version 980960 (0.0007) [2023-12-27 12:26:48,116][20347] Updated weights for policy 0, policy_version 980970 (0.0008) [2023-12-27 12:26:48,184][20347] Updated weights for policy 0, policy_version 980980 (0.0006) [2023-12-27 12:26:48,487][20381] Updated weights for policy 1, policy_version 980225 (0.0007) [2023-12-27 12:26:48,547][20381] Updated weights for policy 1, policy_version 980235 (0.0008) [2023-12-27 12:26:48,603][20381] Updated weights for policy 1, policy_version 980245 (0.0008) [2023-12-27 12:26:48,832][20347] Updated weights for policy 0, policy_version 980990 (0.0007) [2023-12-27 12:26:48,895][20347] Updated weights for policy 0, policy_version 981000 (0.0009) [2023-12-27 12:26:48,962][20347] Updated weights for policy 0, policy_version 981010 (0.0008) [2023-12-27 12:26:49,258][20381] Updated weights for policy 1, policy_version 980255 (0.0008) [2023-12-27 12:26:49,318][20381] Updated weights for policy 1, policy_version 980265 (0.0006) [2023-12-27 12:26:49,383][20381] Updated weights for policy 1, policy_version 980275 (0.0008) [2023-12-27 12:26:49,686][20347] Updated weights for policy 0, policy_version 981020 (0.0008) [2023-12-27 12:26:49,731][20347] Updated weights for policy 0, policy_version 981030 (0.0007) [2023-12-27 12:26:49,778][20347] Updated weights for policy 0, policy_version 981040 (0.0008) [2023-12-27 12:26:50,062][20381] Updated weights for policy 1, policy_version 980285 (0.0007) [2023-12-27 12:26:50,119][20381] Updated weights for policy 1, policy_version 980295 (0.0008) [2023-12-27 12:26:50,169][20381] Updated weights for policy 1, policy_version 980305 (0.0008) [2023-12-27 12:26:50,451][20347] Updated weights for policy 0, policy_version 981050 (0.0007) [2023-12-27 12:26:50,514][20347] Updated weights for policy 0, policy_version 981060 (0.0006) [2023-12-27 12:26:50,577][20347] Updated weights for policy 0, policy_version 981070 (0.0008) [2023-12-27 12:26:50,640][20347] Updated weights for policy 0, policy_version 981080 (0.0009) [2023-12-27 12:26:50,916][20381] Updated weights for policy 1, policy_version 980315 (0.0008) [2023-12-27 12:26:50,977][20381] Updated weights for policy 1, policy_version 980325 (0.0008) [2023-12-27 12:26:51,043][20381] Updated weights for policy 1, policy_version 980335 (0.0007) [2023-12-27 12:26:51,257][20347] Updated weights for policy 0, policy_version 981090 (0.0011) [2023-12-27 12:26:51,316][20347] Updated weights for policy 0, policy_version 981100 (0.0010) [2023-12-27 12:26:51,383][20347] Updated weights for policy 0, policy_version 981110 (0.0009) [2023-12-27 12:26:51,791][20381] Updated weights for policy 1, policy_version 980345 (0.0008) [2023-12-27 12:26:51,858][20381] Updated weights for policy 1, policy_version 980355 (0.0008) [2023-12-27 12:26:51,928][20381] Updated weights for policy 1, policy_version 980365 (0.0008) [2023-12-27 12:26:51,992][20381] Updated weights for policy 1, policy_version 980375 (0.0008) [2023-12-27 12:26:52,084][20347] Updated weights for policy 0, policy_version 981120 (0.0008) [2023-12-27 12:26:52,136][20347] Updated weights for policy 0, policy_version 981130 (0.0007) [2023-12-27 12:26:52,192][20347] Updated weights for policy 0, policy_version 981140 (0.0008) [2023-12-27 12:26:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20243.9). Total num frames: 502218752. Throughput: 0: 10216.2, 1: 10195.8. Samples: 502186184. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:52,363][19170] Avg episode reward: [(0, '598.900'), (1, '961.010')] [2023-12-27 12:26:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000980376_251002880.pth... [2023-12-27 12:26:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000981144_251215872.pth... [2023-12-27 12:26:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000979192_250699776.pth [2023-12-27 12:26:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000979960_250912768.pth [2023-12-27 12:26:52,710][20381] Updated weights for policy 1, policy_version 980385 (0.0009) [2023-12-27 12:26:52,767][20381] Updated weights for policy 1, policy_version 980395 (0.0007) [2023-12-27 12:26:52,821][20347] Updated weights for policy 0, policy_version 981150 (0.0006) [2023-12-27 12:26:52,827][20381] Updated weights for policy 1, policy_version 980405 (0.0006) [2023-12-27 12:26:52,870][20347] Updated weights for policy 0, policy_version 981160 (0.0005) [2023-12-27 12:26:52,920][20347] Updated weights for policy 0, policy_version 981170 (0.0006) [2023-12-27 12:26:53,507][20381] Updated weights for policy 1, policy_version 980415 (0.0007) [2023-12-27 12:26:53,558][20381] Updated weights for policy 1, policy_version 980425 (0.0007) [2023-12-27 12:26:53,582][20347] Updated weights for policy 0, policy_version 981180 (0.0006) [2023-12-27 12:26:53,614][20381] Updated weights for policy 1, policy_version 980435 (0.0005) [2023-12-27 12:26:53,638][20347] Updated weights for policy 0, policy_version 981190 (0.0007) [2023-12-27 12:26:53,690][20347] Updated weights for policy 0, policy_version 981200 (0.0007) [2023-12-27 12:26:54,333][20381] Updated weights for policy 1, policy_version 980445 (0.0007) [2023-12-27 12:26:54,353][20347] Updated weights for policy 0, policy_version 981210 (0.0008) [2023-12-27 12:26:54,378][20381] Updated weights for policy 1, policy_version 980455 (0.0006) [2023-12-27 12:26:54,403][20347] Updated weights for policy 0, policy_version 981220 (0.0006) [2023-12-27 12:26:54,432][20381] Updated weights for policy 1, policy_version 980465 (0.0006) [2023-12-27 12:26:54,450][20347] Updated weights for policy 0, policy_version 981230 (0.0010) [2023-12-27 12:26:54,504][20347] Updated weights for policy 0, policy_version 981240 (0.0008) [2023-12-27 12:26:55,109][20381] Updated weights for policy 1, policy_version 980475 (0.0006) [2023-12-27 12:26:55,152][20347] Updated weights for policy 0, policy_version 981250 (0.0005) [2023-12-27 12:26:55,169][20381] Updated weights for policy 1, policy_version 980485 (0.0006) [2023-12-27 12:26:55,199][20347] Updated weights for policy 0, policy_version 981260 (0.0007) [2023-12-27 12:26:55,220][20381] Updated weights for policy 1, policy_version 980495 (0.0005) [2023-12-27 12:26:55,247][20347] Updated weights for policy 0, policy_version 981270 (0.0008) [2023-12-27 12:26:55,729][20381] Updated weights for policy 1, policy_version 980505 (0.0005) [2023-12-27 12:26:55,787][20381] Updated weights for policy 1, policy_version 980515 (0.0005) [2023-12-27 12:26:55,837][20381] Updated weights for policy 1, policy_version 980525 (0.0005) [2023-12-27 12:26:55,886][20381] Updated weights for policy 1, policy_version 980535 (0.0005) [2023-12-27 12:26:55,958][20347] Updated weights for policy 0, policy_version 981280 (0.0008) [2023-12-27 12:26:56,016][20347] Updated weights for policy 0, policy_version 981290 (0.0008) [2023-12-27 12:26:56,072][20347] Updated weights for policy 0, policy_version 981300 (0.0006) [2023-12-27 12:26:56,491][20381] Updated weights for policy 1, policy_version 980545 (0.0009) [2023-12-27 12:26:56,554][20381] Updated weights for policy 1, policy_version 980555 (0.0008) [2023-12-27 12:26:56,602][20381] Updated weights for policy 1, policy_version 980565 (0.0009) [2023-12-27 12:26:56,674][20347] Updated weights for policy 0, policy_version 981310 (0.0005) [2023-12-27 12:26:56,732][20347] Updated weights for policy 0, policy_version 981320 (0.0005) [2023-12-27 12:26:56,778][20347] Updated weights for policy 0, policy_version 981330 (0.0005) [2023-12-27 12:26:57,313][20347] Updated weights for policy 0, policy_version 981340 (0.0006) [2023-12-27 12:26:57,357][20381] Updated weights for policy 1, policy_version 980575 (0.0007) [2023-12-27 12:26:57,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 502317056. Throughput: 0: 10235.1, 1: 10257.8. Samples: 502311004. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:26:57,363][19170] Avg episode reward: [(0, '666.910'), (1, '931.790')] [2023-12-27 12:26:57,373][20347] Updated weights for policy 0, policy_version 981350 (0.0006) [2023-12-27 12:26:57,408][20381] Updated weights for policy 1, policy_version 980585 (0.0005) [2023-12-27 12:26:57,425][20347] Updated weights for policy 0, policy_version 981360 (0.0006) [2023-12-27 12:26:57,452][20381] Updated weights for policy 1, policy_version 980595 (0.0008) [2023-12-27 12:26:58,082][20347] Updated weights for policy 0, policy_version 981370 (0.0005) [2023-12-27 12:26:58,138][20347] Updated weights for policy 0, policy_version 981380 (0.0008) [2023-12-27 12:26:58,144][20381] Updated weights for policy 1, policy_version 980605 (0.0006) [2023-12-27 12:26:58,202][20347] Updated weights for policy 0, policy_version 981390 (0.0007) [2023-12-27 12:26:58,204][20381] Updated weights for policy 1, policy_version 980615 (0.0006) [2023-12-27 12:26:58,257][20381] Updated weights for policy 1, policy_version 980625 (0.0006) [2023-12-27 12:26:58,259][20347] Updated weights for policy 0, policy_version 981400 (0.0006) [2023-12-27 12:26:59,024][20347] Updated weights for policy 0, policy_version 981410 (0.0008) [2023-12-27 12:26:59,088][20347] Updated weights for policy 0, policy_version 981420 (0.0007) [2023-12-27 12:26:59,115][20381] Updated weights for policy 1, policy_version 980635 (0.0008) [2023-12-27 12:26:59,151][20347] Updated weights for policy 0, policy_version 981430 (0.0007) [2023-12-27 12:26:59,176][20381] Updated weights for policy 1, policy_version 980645 (0.0007) [2023-12-27 12:26:59,234][20381] Updated weights for policy 1, policy_version 980655 (0.0007) [2023-12-27 12:26:59,812][20347] Updated weights for policy 0, policy_version 981440 (0.0010) [2023-12-27 12:26:59,864][20381] Updated weights for policy 1, policy_version 980665 (0.0007) [2023-12-27 12:26:59,890][20347] Updated weights for policy 0, policy_version 981450 (0.0011) [2023-12-27 12:26:59,922][20381] Updated weights for policy 1, policy_version 980675 (0.0010) [2023-12-27 12:26:59,954][20347] Updated weights for policy 0, policy_version 981460 (0.0010) [2023-12-27 12:26:59,984][20381] Updated weights for policy 1, policy_version 980686 (0.0007) [2023-12-27 12:27:00,038][20381] Updated weights for policy 1, policy_version 980696 (0.0008) [2023-12-27 12:27:00,558][20347] Updated weights for policy 0, policy_version 981470 (0.0006) [2023-12-27 12:27:00,612][20347] Updated weights for policy 0, policy_version 981480 (0.0010) [2023-12-27 12:27:00,677][20347] Updated weights for policy 0, policy_version 981490 (0.0010) [2023-12-27 12:27:00,715][20381] Updated weights for policy 1, policy_version 980706 (0.0005) [2023-12-27 12:27:00,774][20381] Updated weights for policy 1, policy_version 980716 (0.0007) [2023-12-27 12:27:00,823][20381] Updated weights for policy 1, policy_version 980726 (0.0010) [2023-12-27 12:27:01,298][20347] Updated weights for policy 0, policy_version 981500 (0.0008) [2023-12-27 12:27:01,353][20347] Updated weights for policy 0, policy_version 981510 (0.0007) [2023-12-27 12:27:01,421][20347] Updated weights for policy 0, policy_version 981520 (0.0006) [2023-12-27 12:27:01,484][20381] Updated weights for policy 1, policy_version 980736 (0.0008) [2023-12-27 12:27:01,534][20381] Updated weights for policy 1, policy_version 980746 (0.0008) [2023-12-27 12:27:01,592][20381] Updated weights for policy 1, policy_version 980756 (0.0008) [2023-12-27 12:27:02,057][20347] Updated weights for policy 0, policy_version 981530 (0.0006) [2023-12-27 12:27:02,114][20347] Updated weights for policy 0, policy_version 981540 (0.0007) [2023-12-27 12:27:02,165][20347] Updated weights for policy 0, policy_version 981550 (0.0007) [2023-12-27 12:27:02,210][20347] Updated weights for policy 0, policy_version 981560 (0.0009) [2023-12-27 12:27:02,276][20381] Updated weights for policy 1, policy_version 980766 (0.0008) [2023-12-27 12:27:02,338][20381] Updated weights for policy 1, policy_version 980776 (0.0006) [2023-12-27 12:27:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 502423552. Throughput: 0: 10258.2, 1: 10261.8. Samples: 502436020. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:27:02,363][19170] Avg episode reward: [(0, '666.550'), (1, '883.140')] [2023-12-27 12:27:02,403][20381] Updated weights for policy 1, policy_version 980786 (0.0008) [2023-12-27 12:27:02,849][20347] Updated weights for policy 0, policy_version 981570 (0.0008) [2023-12-27 12:27:02,903][20347] Updated weights for policy 0, policy_version 981580 (0.0007) [2023-12-27 12:27:02,960][20347] Updated weights for policy 0, policy_version 981590 (0.0010) [2023-12-27 12:27:03,113][20381] Updated weights for policy 1, policy_version 980797 (0.0008) [2023-12-27 12:27:03,165][20381] Updated weights for policy 1, policy_version 980807 (0.0006) [2023-12-27 12:27:03,221][20381] Updated weights for policy 1, policy_version 980817 (0.0005) [2023-12-27 12:27:03,628][20347] Updated weights for policy 0, policy_version 981600 (0.0010) [2023-12-27 12:27:03,682][20347] Updated weights for policy 0, policy_version 981610 (0.0010) [2023-12-27 12:27:03,736][20347] Updated weights for policy 0, policy_version 981620 (0.0010) [2023-12-27 12:27:03,896][20381] Updated weights for policy 1, policy_version 980827 (0.0007) [2023-12-27 12:27:03,952][20381] Updated weights for policy 1, policy_version 980837 (0.0011) [2023-12-27 12:27:04,012][20381] Updated weights for policy 1, policy_version 980847 (0.0011) [2023-12-27 12:27:04,437][20347] Updated weights for policy 0, policy_version 981630 (0.0010) [2023-12-27 12:27:04,495][20347] Updated weights for policy 0, policy_version 981640 (0.0007) [2023-12-27 12:27:04,551][20347] Updated weights for policy 0, policy_version 981650 (0.0006) [2023-12-27 12:27:04,704][20381] Updated weights for policy 1, policy_version 980857 (0.0011) [2023-12-27 12:27:04,753][20381] Updated weights for policy 1, policy_version 980867 (0.0011) [2023-12-27 12:27:04,803][20381] Updated weights for policy 1, policy_version 980877 (0.0011) [2023-12-27 12:27:04,852][20381] Updated weights for policy 1, policy_version 980887 (0.0010) [2023-12-27 12:27:05,124][20347] Updated weights for policy 0, policy_version 981660 (0.0008) [2023-12-27 12:27:05,171][20347] Updated weights for policy 0, policy_version 981670 (0.0009) [2023-12-27 12:27:05,216][20347] Updated weights for policy 0, policy_version 981680 (0.0010) [2023-12-27 12:27:05,466][20381] Updated weights for policy 1, policy_version 980897 (0.0011) [2023-12-27 12:27:05,520][20381] Updated weights for policy 1, policy_version 980907 (0.0010) [2023-12-27 12:27:05,578][20381] Updated weights for policy 1, policy_version 980917 (0.0010) [2023-12-27 12:27:05,941][20347] Updated weights for policy 0, policy_version 981690 (0.0008) [2023-12-27 12:27:06,002][20347] Updated weights for policy 0, policy_version 981700 (0.0008) [2023-12-27 12:27:06,060][20347] Updated weights for policy 0, policy_version 981710 (0.0007) [2023-12-27 12:27:06,130][20347] Updated weights for policy 0, policy_version 981720 (0.0009) [2023-12-27 12:27:06,246][20381] Updated weights for policy 1, policy_version 980927 (0.0009) [2023-12-27 12:27:06,306][20381] Updated weights for policy 1, policy_version 980937 (0.0009) [2023-12-27 12:27:06,362][20381] Updated weights for policy 1, policy_version 980947 (0.0007) [2023-12-27 12:27:06,796][20347] Updated weights for policy 0, policy_version 981730 (0.0007) [2023-12-27 12:27:06,855][20347] Updated weights for policy 0, policy_version 981740 (0.0007) [2023-12-27 12:27:06,913][20347] Updated weights for policy 0, policy_version 981750 (0.0007) [2023-12-27 12:27:07,093][20381] Updated weights for policy 1, policy_version 980957 (0.0009) [2023-12-27 12:27:07,146][20381] Updated weights for policy 1, policy_version 980967 (0.0008) [2023-12-27 12:27:07,206][20381] Updated weights for policy 1, policy_version 980977 (0.0007) [2023-12-27 12:27:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 502530048. Throughput: 0: 10301.8, 1: 10243.0. Samples: 502498416. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:27:07,363][19170] Avg episode reward: [(0, '723.210'), (1, '883.990')] [2023-12-27 12:27:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000980984_251158528.pth... [2023-12-27 12:27:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000981752_251371520.pth... [2023-12-27 12:27:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000979800_250855424.pth [2023-12-27 12:27:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000980536_251060224.pth [2023-12-27 12:27:07,532][20347] Updated weights for policy 0, policy_version 981760 (0.0005) [2023-12-27 12:27:07,595][20347] Updated weights for policy 0, policy_version 981770 (0.0006) [2023-12-27 12:27:07,651][20347] Updated weights for policy 0, policy_version 981780 (0.0010) [2023-12-27 12:27:07,982][20381] Updated weights for policy 1, policy_version 980987 (0.0007) [2023-12-27 12:27:08,046][20381] Updated weights for policy 1, policy_version 980997 (0.0007) [2023-12-27 12:27:08,113][20381] Updated weights for policy 1, policy_version 981007 (0.0006) [2023-12-27 12:27:08,222][20347] Updated weights for policy 0, policy_version 981790 (0.0010) [2023-12-27 12:27:08,284][20347] Updated weights for policy 0, policy_version 981800 (0.0010) [2023-12-27 12:27:08,346][20347] Updated weights for policy 0, policy_version 981810 (0.0010) [2023-12-27 12:27:08,815][20381] Updated weights for policy 1, policy_version 981017 (0.0008) [2023-12-27 12:27:08,871][20381] Updated weights for policy 1, policy_version 981027 (0.0007) [2023-12-27 12:27:08,920][20381] Updated weights for policy 1, policy_version 981037 (0.0008) [2023-12-27 12:27:08,971][20381] Updated weights for policy 1, policy_version 981047 (0.0008) [2023-12-27 12:27:09,059][20347] Updated weights for policy 0, policy_version 981820 (0.0006) [2023-12-27 12:27:09,118][20347] Updated weights for policy 0, policy_version 981830 (0.0007) [2023-12-27 12:27:09,178][20347] Updated weights for policy 0, policy_version 981840 (0.0008) [2023-12-27 12:27:09,790][20381] Updated weights for policy 1, policy_version 981057 (0.0009) [2023-12-27 12:27:09,848][20381] Updated weights for policy 1, policy_version 981067 (0.0009) [2023-12-27 12:27:09,878][20347] Updated weights for policy 0, policy_version 981850 (0.0008) [2023-12-27 12:27:09,908][20381] Updated weights for policy 1, policy_version 981077 (0.0009) [2023-12-27 12:27:09,935][20347] Updated weights for policy 0, policy_version 981860 (0.0006) [2023-12-27 12:27:10,002][20347] Updated weights for policy 0, policy_version 981870 (0.0008) [2023-12-27 12:27:10,057][20347] Updated weights for policy 0, policy_version 981880 (0.0006) [2023-12-27 12:27:10,609][20381] Updated weights for policy 1, policy_version 981087 (0.0010) [2023-12-27 12:27:10,672][20381] Updated weights for policy 1, policy_version 981097 (0.0009) [2023-12-27 12:27:10,730][20381] Updated weights for policy 1, policy_version 981107 (0.0007) [2023-12-27 12:27:10,790][20347] Updated weights for policy 0, policy_version 981890 (0.0010) [2023-12-27 12:27:10,848][20347] Updated weights for policy 0, policy_version 981900 (0.0010) [2023-12-27 12:27:10,901][20347] Updated weights for policy 0, policy_version 981910 (0.0010) [2023-12-27 12:27:11,438][20381] Updated weights for policy 1, policy_version 981117 (0.0007) [2023-12-27 12:27:11,498][20381] Updated weights for policy 1, policy_version 981127 (0.0008) [2023-12-27 12:27:11,560][20381] Updated weights for policy 1, policy_version 981137 (0.0008) [2023-12-27 12:27:11,653][20347] Updated weights for policy 0, policy_version 981920 (0.0008) [2023-12-27 12:27:11,720][20347] Updated weights for policy 0, policy_version 981930 (0.0007) [2023-12-27 12:27:11,779][20347] Updated weights for policy 0, policy_version 981940 (0.0006) [2023-12-27 12:27:12,307][20381] Updated weights for policy 1, policy_version 981147 (0.0007) [2023-12-27 12:27:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20216.2). Total num frames: 502620160. Throughput: 0: 10282.5, 1: 10185.3. Samples: 502616920. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:27:12,363][19170] Avg episode reward: [(0, '786.150'), (1, '902.890')] [2023-12-27 12:27:12,373][20381] Updated weights for policy 1, policy_version 981157 (0.0007) [2023-12-27 12:27:12,431][20381] Updated weights for policy 1, policy_version 981167 (0.0008) [2023-12-27 12:27:12,457][20347] Updated weights for policy 0, policy_version 981950 (0.0007) [2023-12-27 12:27:12,516][20347] Updated weights for policy 0, policy_version 981960 (0.0007) [2023-12-27 12:27:12,578][20347] Updated weights for policy 0, policy_version 981970 (0.0007) [2023-12-27 12:27:13,149][20381] Updated weights for policy 1, policy_version 981177 (0.0008) [2023-12-27 12:27:13,198][20381] Updated weights for policy 1, policy_version 981187 (0.0007) [2023-12-27 12:27:13,236][20347] Updated weights for policy 0, policy_version 981980 (0.0007) [2023-12-27 12:27:13,257][20381] Updated weights for policy 1, policy_version 981197 (0.0006) [2023-12-27 12:27:13,297][20347] Updated weights for policy 0, policy_version 981990 (0.0006) [2023-12-27 12:27:13,314][20381] Updated weights for policy 1, policy_version 981207 (0.0006) [2023-12-27 12:27:13,356][20347] Updated weights for policy 0, policy_version 982000 (0.0005) [2023-12-27 12:27:13,931][20347] Updated weights for policy 0, policy_version 982010 (0.0006) [2023-12-27 12:27:13,940][20292] KL-divergence is very high: 213.4025 [2023-12-27 12:27:13,946][20381] Updated weights for policy 1, policy_version 981217 (0.0007) [2023-12-27 12:27:13,991][20347] Updated weights for policy 0, policy_version 982020 (0.0005) [2023-12-27 12:27:14,005][20381] Updated weights for policy 1, policy_version 981227 (0.0011) [2023-12-27 12:27:14,044][20347] Updated weights for policy 0, policy_version 982030 (0.0005) [2023-12-27 12:27:14,064][20381] Updated weights for policy 1, policy_version 981237 (0.0010) [2023-12-27 12:27:14,098][20347] Updated weights for policy 0, policy_version 982040 (0.0006) [2023-12-27 12:27:14,668][20381] Updated weights for policy 1, policy_version 981247 (0.0010) [2023-12-27 12:27:14,677][20347] Updated weights for policy 0, policy_version 982050 (0.0005) [2023-12-27 12:27:14,720][20381] Updated weights for policy 1, policy_version 981257 (0.0010) [2023-12-27 12:27:14,724][20347] Updated weights for policy 0, policy_version 982060 (0.0005) [2023-12-27 12:27:14,783][20347] Updated weights for policy 0, policy_version 982070 (0.0007) [2023-12-27 12:27:14,785][20381] Updated weights for policy 1, policy_version 981267 (0.0010) [2023-12-27 12:27:15,472][20381] Updated weights for policy 1, policy_version 981277 (0.0009) [2023-12-27 12:27:15,490][20347] Updated weights for policy 0, policy_version 982080 (0.0007) [2023-12-27 12:27:15,527][20381] Updated weights for policy 1, policy_version 981287 (0.0010) [2023-12-27 12:27:15,541][20347] Updated weights for policy 0, policy_version 982090 (0.0005) [2023-12-27 12:27:15,582][20381] Updated weights for policy 1, policy_version 981297 (0.0010) [2023-12-27 12:27:15,593][20347] Updated weights for policy 0, policy_version 982100 (0.0005) [2023-12-27 12:27:16,227][20347] Updated weights for policy 0, policy_version 982110 (0.0007) [2023-12-27 12:27:16,259][20381] Updated weights for policy 1, policy_version 981307 (0.0009) [2023-12-27 12:27:16,276][20347] Updated weights for policy 0, policy_version 982120 (0.0005) [2023-12-27 12:27:16,314][20381] Updated weights for policy 1, policy_version 981317 (0.0007) [2023-12-27 12:27:16,336][20347] Updated weights for policy 0, policy_version 982130 (0.0006) [2023-12-27 12:27:16,366][20381] Updated weights for policy 1, policy_version 981327 (0.0006) [2023-12-27 12:27:16,931][20381] Updated weights for policy 1, policy_version 981338 (0.0012) [2023-12-27 12:27:16,989][20381] Updated weights for policy 1, policy_version 981348 (0.0011) [2023-12-27 12:27:17,032][20347] Updated weights for policy 0, policy_version 982140 (0.0010) [2023-12-27 12:27:17,049][20381] Updated weights for policy 1, policy_version 981358 (0.0011) [2023-12-27 12:27:17,085][20347] Updated weights for policy 0, policy_version 982150 (0.0011) [2023-12-27 12:27:17,106][20381] Updated weights for policy 1, policy_version 981368 (0.0011) [2023-12-27 12:27:17,149][20347] Updated weights for policy 0, policy_version 982160 (0.0010) [2023-12-27 12:27:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 502734848. Throughput: 0: 10323.9, 1: 10157.4. Samples: 502741772. Policy #0 lag: (min: 31.0, avg: 33.4, max: 63.0) [2023-12-27 12:27:17,363][19170] Avg episode reward: [(0, '819.100'), (1, '951.510')] [2023-12-27 12:27:17,739][20347] Updated weights for policy 0, policy_version 982170 (0.0010) [2023-12-27 12:27:17,767][20381] Updated weights for policy 1, policy_version 981378 (0.0006) [2023-12-27 12:27:17,791][20347] Updated weights for policy 0, policy_version 982180 (0.0010) [2023-12-27 12:27:17,818][20381] Updated weights for policy 1, policy_version 981388 (0.0006) [2023-12-27 12:27:17,858][20347] Updated weights for policy 0, policy_version 982190 (0.0007) [2023-12-27 12:27:17,862][20381] Updated weights for policy 1, policy_version 981398 (0.0005) [2023-12-27 12:27:17,911][20347] Updated weights for policy 0, policy_version 982200 (0.0005) [2023-12-27 12:27:18,537][20381] Updated weights for policy 1, policy_version 981408 (0.0007) [2023-12-27 12:27:18,538][20347] Updated weights for policy 0, policy_version 982210 (0.0011) [2023-12-27 12:27:18,594][20381] Updated weights for policy 1, policy_version 981418 (0.0011) [2023-12-27 12:27:18,597][20347] Updated weights for policy 0, policy_version 982220 (0.0011) [2023-12-27 12:27:18,650][20381] Updated weights for policy 1, policy_version 981428 (0.0009) [2023-12-27 12:27:18,652][20347] Updated weights for policy 0, policy_version 982230 (0.0010) [2023-12-27 12:27:19,366][20381] Updated weights for policy 1, policy_version 981438 (0.0008) [2023-12-27 12:27:19,385][20347] Updated weights for policy 0, policy_version 982240 (0.0009) [2023-12-27 12:27:19,428][20381] Updated weights for policy 1, policy_version 981448 (0.0006) [2023-12-27 12:27:19,447][20347] Updated weights for policy 0, policy_version 982250 (0.0008) [2023-12-27 12:27:19,484][20381] Updated weights for policy 1, policy_version 981458 (0.0006) [2023-12-27 12:27:19,515][20347] Updated weights for policy 0, policy_version 982260 (0.0009) [2023-12-27 12:27:20,135][20381] Updated weights for policy 1, policy_version 981468 (0.0008) [2023-12-27 12:27:20,195][20381] Updated weights for policy 1, policy_version 981478 (0.0007) [2023-12-27 12:27:20,245][20347] Updated weights for policy 0, policy_version 982270 (0.0007) [2023-12-27 12:27:20,258][20381] Updated weights for policy 1, policy_version 981488 (0.0008) [2023-12-27 12:27:20,305][20347] Updated weights for policy 0, policy_version 982280 (0.0006) [2023-12-27 12:27:20,368][20347] Updated weights for policy 0, policy_version 982290 (0.0008) [2023-12-27 12:27:20,908][20381] Updated weights for policy 1, policy_version 981498 (0.0008) [2023-12-27 12:27:20,974][20381] Updated weights for policy 1, policy_version 981508 (0.0008) [2023-12-27 12:27:21,042][20381] Updated weights for policy 1, policy_version 981518 (0.0007) [2023-12-27 12:27:21,092][20347] Updated weights for policy 0, policy_version 982300 (0.0008) [2023-12-27 12:27:21,110][20381] Updated weights for policy 1, policy_version 981528 (0.0007) [2023-12-27 12:27:21,148][20347] Updated weights for policy 0, policy_version 982310 (0.0006) [2023-12-27 12:27:21,196][20347] Updated weights for policy 0, policy_version 982320 (0.0007) [2023-12-27 12:27:21,857][20381] Updated weights for policy 1, policy_version 981538 (0.0006) [2023-12-27 12:27:21,892][20347] Updated weights for policy 0, policy_version 982330 (0.0009) [2023-12-27 12:27:21,917][20381] Updated weights for policy 1, policy_version 981548 (0.0006) [2023-12-27 12:27:21,946][20347] Updated weights for policy 0, policy_version 982340 (0.0006) [2023-12-27 12:27:21,971][20381] Updated weights for policy 1, policy_version 981558 (0.0006) [2023-12-27 12:27:22,002][20347] Updated weights for policy 0, policy_version 982350 (0.0007) [2023-12-27 12:27:22,052][20347] Updated weights for policy 0, policy_version 982360 (0.0007) [2023-12-27 12:27:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20243.9). Total num frames: 502833152. Throughput: 0: 10302.8, 1: 10162.3. Samples: 502803624. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:22,363][19170] Avg episode reward: [(0, '827.720'), (1, '923.060')] [2023-12-27 12:27:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000981560_251305984.pth... [2023-12-27 12:27:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000982360_251527168.pth... [2023-12-27 12:27:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000981144_251215872.pth [2023-12-27 12:27:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000980376_251002880.pth [2023-12-27 12:27:22,681][20381] Updated weights for policy 1, policy_version 981568 (0.0011) [2023-12-27 12:27:22,741][20381] Updated weights for policy 1, policy_version 981578 (0.0009) [2023-12-27 12:27:22,771][20347] Updated weights for policy 0, policy_version 982370 (0.0006) [2023-12-27 12:27:22,800][20381] Updated weights for policy 1, policy_version 981588 (0.0010) [2023-12-27 12:27:22,828][20347] Updated weights for policy 0, policy_version 982380 (0.0006) [2023-12-27 12:27:22,876][20347] Updated weights for policy 0, policy_version 982390 (0.0007) [2023-12-27 12:27:23,462][20381] Updated weights for policy 1, policy_version 981598 (0.0011) [2023-12-27 12:27:23,518][20381] Updated weights for policy 1, policy_version 981608 (0.0011) [2023-12-27 12:27:23,577][20381] Updated weights for policy 1, policy_version 981618 (0.0005) [2023-12-27 12:27:23,607][20347] Updated weights for policy 0, policy_version 982400 (0.0008) [2023-12-27 12:27:23,664][20347] Updated weights for policy 0, policy_version 982410 (0.0005) [2023-12-27 12:27:23,721][20347] Updated weights for policy 0, policy_version 982420 (0.0008) [2023-12-27 12:27:24,106][20381] Updated weights for policy 1, policy_version 981628 (0.0005) [2023-12-27 12:27:24,162][20381] Updated weights for policy 1, policy_version 981638 (0.0005) [2023-12-27 12:27:24,227][20381] Updated weights for policy 1, policy_version 981648 (0.0005) [2023-12-27 12:27:24,466][20347] Updated weights for policy 0, policy_version 982430 (0.0007) [2023-12-27 12:27:24,520][20347] Updated weights for policy 0, policy_version 982440 (0.0007) [2023-12-27 12:27:24,574][20347] Updated weights for policy 0, policy_version 982450 (0.0007) [2023-12-27 12:27:24,818][20381] Updated weights for policy 1, policy_version 981658 (0.0006) [2023-12-27 12:27:24,870][20381] Updated weights for policy 1, policy_version 981668 (0.0010) [2023-12-27 12:27:24,928][20381] Updated weights for policy 1, policy_version 981678 (0.0010) [2023-12-27 12:27:24,976][20381] Updated weights for policy 1, policy_version 981688 (0.0008) [2023-12-27 12:27:25,299][20347] Updated weights for policy 0, policy_version 982460 (0.0009) [2023-12-27 12:27:25,354][20347] Updated weights for policy 0, policy_version 982470 (0.0008) [2023-12-27 12:27:25,409][20347] Updated weights for policy 0, policy_version 982480 (0.0010) [2023-12-27 12:27:25,581][20381] Updated weights for policy 1, policy_version 981698 (0.0005) [2023-12-27 12:27:25,628][20381] Updated weights for policy 1, policy_version 981708 (0.0007) [2023-12-27 12:27:25,690][20381] Updated weights for policy 1, policy_version 981718 (0.0007) [2023-12-27 12:27:25,992][20347] Updated weights for policy 0, policy_version 982490 (0.0009) [2023-12-27 12:27:26,044][20347] Updated weights for policy 0, policy_version 982500 (0.0005) [2023-12-27 12:27:26,098][20347] Updated weights for policy 0, policy_version 982510 (0.0008) [2023-12-27 12:27:26,149][20347] Updated weights for policy 0, policy_version 982520 (0.0010) [2023-12-27 12:27:26,334][20381] Updated weights for policy 1, policy_version 981728 (0.0010) [2023-12-27 12:27:26,389][20381] Updated weights for policy 1, policy_version 981738 (0.0010) [2023-12-27 12:27:26,443][20381] Updated weights for policy 1, policy_version 981748 (0.0010) [2023-12-27 12:27:26,775][20347] Updated weights for policy 0, policy_version 982530 (0.0010) [2023-12-27 12:27:26,829][20347] Updated weights for policy 0, policy_version 982540 (0.0006) [2023-12-27 12:27:26,875][20347] Updated weights for policy 0, policy_version 982550 (0.0005) [2023-12-27 12:27:27,097][20381] Updated weights for policy 1, policy_version 981758 (0.0008) [2023-12-27 12:27:27,146][20381] Updated weights for policy 1, policy_version 981768 (0.0007) [2023-12-27 12:27:27,202][20381] Updated weights for policy 1, policy_version 981778 (0.0006) [2023-12-27 12:27:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20271.7). Total num frames: 502939648. Throughput: 0: 10298.9, 1: 10225.0. Samples: 502927032. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:27,363][19170] Avg episode reward: [(0, '777.520'), (1, '903.470')] [2023-12-27 12:27:27,532][20347] Updated weights for policy 0, policy_version 982560 (0.0009) [2023-12-27 12:27:27,588][20347] Updated weights for policy 0, policy_version 982570 (0.0010) [2023-12-27 12:27:27,639][20347] Updated weights for policy 0, policy_version 982580 (0.0009) [2023-12-27 12:27:27,861][20381] Updated weights for policy 1, policy_version 981788 (0.0009) [2023-12-27 12:27:27,918][20381] Updated weights for policy 1, policy_version 981798 (0.0010) [2023-12-27 12:27:27,981][20381] Updated weights for policy 1, policy_version 981808 (0.0010) [2023-12-27 12:27:28,193][20347] Updated weights for policy 0, policy_version 982590 (0.0005) [2023-12-27 12:27:28,241][20347] Updated weights for policy 0, policy_version 982600 (0.0005) [2023-12-27 12:27:28,289][20347] Updated weights for policy 0, policy_version 982610 (0.0005) [2023-12-27 12:27:28,660][20381] Updated weights for policy 1, policy_version 981818 (0.0011) [2023-12-27 12:27:28,720][20381] Updated weights for policy 1, policy_version 981828 (0.0011) [2023-12-27 12:27:28,777][20381] Updated weights for policy 1, policy_version 981838 (0.0011) [2023-12-27 12:27:28,836][20381] Updated weights for policy 1, policy_version 981848 (0.0011) [2023-12-27 12:27:28,963][20347] Updated weights for policy 0, policy_version 982620 (0.0007) [2023-12-27 12:27:29,007][20347] Updated weights for policy 0, policy_version 982630 (0.0010) [2023-12-27 12:27:29,056][20347] Updated weights for policy 0, policy_version 982640 (0.0010) [2023-12-27 12:27:29,511][20381] Updated weights for policy 1, policy_version 981858 (0.0009) [2023-12-27 12:27:29,563][20381] Updated weights for policy 1, policy_version 981868 (0.0006) [2023-12-27 12:27:29,625][20381] Updated weights for policy 1, policy_version 981878 (0.0006) [2023-12-27 12:27:29,802][20347] Updated weights for policy 0, policy_version 982650 (0.0010) [2023-12-27 12:27:29,864][20347] Updated weights for policy 0, policy_version 982660 (0.0009) [2023-12-27 12:27:29,924][20347] Updated weights for policy 0, policy_version 982670 (0.0008) [2023-12-27 12:27:29,976][20347] Updated weights for policy 0, policy_version 982680 (0.0009) [2023-12-27 12:27:30,288][20381] Updated weights for policy 1, policy_version 981888 (0.0010) [2023-12-27 12:27:30,347][20381] Updated weights for policy 1, policy_version 981898 (0.0011) [2023-12-27 12:27:30,395][20381] Updated weights for policy 1, policy_version 981908 (0.0010) [2023-12-27 12:27:30,648][20347] Updated weights for policy 0, policy_version 982690 (0.0010) [2023-12-27 12:27:30,699][20347] Updated weights for policy 0, policy_version 982700 (0.0010) [2023-12-27 12:27:30,753][20347] Updated weights for policy 0, policy_version 982710 (0.0010) [2023-12-27 12:27:31,061][20381] Updated weights for policy 1, policy_version 981918 (0.0011) [2023-12-27 12:27:31,118][20381] Updated weights for policy 1, policy_version 981928 (0.0007) [2023-12-27 12:27:31,179][20381] Updated weights for policy 1, policy_version 981938 (0.0007) [2023-12-27 12:27:31,504][20347] Updated weights for policy 0, policy_version 982720 (0.0010) [2023-12-27 12:27:31,570][20347] Updated weights for policy 0, policy_version 982730 (0.0011) [2023-12-27 12:27:31,628][20347] Updated weights for policy 0, policy_version 982740 (0.0010) [2023-12-27 12:27:31,894][20381] Updated weights for policy 1, policy_version 981948 (0.0008) [2023-12-27 12:27:31,947][20381] Updated weights for policy 1, policy_version 981958 (0.0008) [2023-12-27 12:27:31,996][20381] Updated weights for policy 1, policy_version 981968 (0.0007) [2023-12-27 12:27:32,262][20347] Updated weights for policy 0, policy_version 982750 (0.0008) [2023-12-27 12:27:32,322][20347] Updated weights for policy 0, policy_version 982760 (0.0008) [2023-12-27 12:27:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 503037952. Throughput: 0: 10401.0, 1: 10224.4. Samples: 503052212. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:32,363][19170] Avg episode reward: [(0, '816.380'), (1, '902.870')] [2023-12-27 12:27:32,389][20347] Updated weights for policy 0, policy_version 982770 (0.0008) [2023-12-27 12:27:32,624][20381] Updated weights for policy 1, policy_version 981978 (0.0007) [2023-12-27 12:27:32,687][20381] Updated weights for policy 1, policy_version 981988 (0.0007) [2023-12-27 12:27:32,743][20381] Updated weights for policy 1, policy_version 981998 (0.0007) [2023-12-27 12:27:32,800][20381] Updated weights for policy 1, policy_version 982008 (0.0007) [2023-12-27 12:27:33,024][20347] Updated weights for policy 0, policy_version 982780 (0.0007) [2023-12-27 12:27:33,072][20347] Updated weights for policy 0, policy_version 982790 (0.0005) [2023-12-27 12:27:33,118][20347] Updated weights for policy 0, policy_version 982800 (0.0007) [2023-12-27 12:27:33,473][20381] Updated weights for policy 1, policy_version 982018 (0.0011) [2023-12-27 12:27:33,524][20381] Updated weights for policy 1, policy_version 982028 (0.0010) [2023-12-27 12:27:33,578][20381] Updated weights for policy 1, policy_version 982038 (0.0006) [2023-12-27 12:27:33,710][20347] Updated weights for policy 0, policy_version 982810 (0.0009) [2023-12-27 12:27:33,766][20347] Updated weights for policy 0, policy_version 982820 (0.0007) [2023-12-27 12:27:33,808][20347] Updated weights for policy 0, policy_version 982830 (0.0007) [2023-12-27 12:27:33,860][20347] Updated weights for policy 0, policy_version 982840 (0.0007) [2023-12-27 12:27:34,194][20381] Updated weights for policy 1, policy_version 982048 (0.0007) [2023-12-27 12:27:34,252][20381] Updated weights for policy 1, policy_version 982058 (0.0008) [2023-12-27 12:27:34,305][20381] Updated weights for policy 1, policy_version 982068 (0.0008) [2023-12-27 12:27:34,589][20347] Updated weights for policy 0, policy_version 982850 (0.0008) [2023-12-27 12:27:34,639][20347] Updated weights for policy 0, policy_version 982860 (0.0008) [2023-12-27 12:27:34,706][20347] Updated weights for policy 0, policy_version 982870 (0.0007) [2023-12-27 12:27:35,019][20381] Updated weights for policy 1, policy_version 982078 (0.0008) [2023-12-27 12:27:35,067][20381] Updated weights for policy 1, policy_version 982088 (0.0005) [2023-12-27 12:27:35,124][20381] Updated weights for policy 1, policy_version 982098 (0.0006) [2023-12-27 12:27:35,376][20347] Updated weights for policy 0, policy_version 982880 (0.0008) [2023-12-27 12:27:35,436][20347] Updated weights for policy 0, policy_version 982890 (0.0008) [2023-12-27 12:27:35,501][20347] Updated weights for policy 0, policy_version 982900 (0.0008) [2023-12-27 12:27:35,695][20381] Updated weights for policy 1, policy_version 982108 (0.0007) [2023-12-27 12:27:35,747][20381] Updated weights for policy 1, policy_version 982118 (0.0008) [2023-12-27 12:27:35,794][20381] Updated weights for policy 1, policy_version 982128 (0.0008) [2023-12-27 12:27:36,226][20347] Updated weights for policy 0, policy_version 982910 (0.0008) [2023-12-27 12:27:36,287][20347] Updated weights for policy 0, policy_version 982920 (0.0008) [2023-12-27 12:27:36,340][20347] Updated weights for policy 0, policy_version 982930 (0.0008) [2023-12-27 12:27:36,453][20381] Updated weights for policy 1, policy_version 982138 (0.0007) [2023-12-27 12:27:36,517][20381] Updated weights for policy 1, policy_version 982148 (0.0006) [2023-12-27 12:27:36,574][20381] Updated weights for policy 1, policy_version 982158 (0.0008) [2023-12-27 12:27:36,640][20381] Updated weights for policy 1, policy_version 982168 (0.0008) [2023-12-27 12:27:37,076][20347] Updated weights for policy 0, policy_version 982940 (0.0006) [2023-12-27 12:27:37,127][20347] Updated weights for policy 0, policy_version 982950 (0.0005) [2023-12-27 12:27:37,184][20347] Updated weights for policy 0, policy_version 982960 (0.0007) [2023-12-27 12:27:37,305][20381] Updated weights for policy 1, policy_version 982178 (0.0007) [2023-12-27 12:27:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.7, 300 sec: 20243.9). Total num frames: 503144448. Throughput: 0: 10424.8, 1: 10220.8. Samples: 503115236. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:37,363][19170] Avg episode reward: [(0, '873.730'), (1, '923.450')] [2023-12-27 12:27:37,366][20381] Updated weights for policy 1, policy_version 982188 (0.0006) [2023-12-27 12:27:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000982968_251682816.pth... [2023-12-27 12:27:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000981752_251371520.pth [2023-12-27 12:27:37,431][20381] Updated weights for policy 1, policy_version 982198 (0.0008) [2023-12-27 12:27:37,441][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000982200_251469824.pth... [2023-12-27 12:27:37,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000980984_251158528.pth [2023-12-27 12:27:37,742][20347] Updated weights for policy 0, policy_version 982970 (0.0007) [2023-12-27 12:27:37,797][20347] Updated weights for policy 0, policy_version 982980 (0.0008) [2023-12-27 12:27:37,845][20347] Updated weights for policy 0, policy_version 982990 (0.0007) [2023-12-27 12:27:37,899][20347] Updated weights for policy 0, policy_version 983000 (0.0008) [2023-12-27 12:27:38,085][20381] Updated weights for policy 1, policy_version 982208 (0.0007) [2023-12-27 12:27:38,139][20381] Updated weights for policy 1, policy_version 982218 (0.0007) [2023-12-27 12:27:38,200][20381] Updated weights for policy 1, policy_version 982228 (0.0006) [2023-12-27 12:27:38,595][20347] Updated weights for policy 0, policy_version 983010 (0.0010) [2023-12-27 12:27:38,651][20347] Updated weights for policy 0, policy_version 983020 (0.0010) [2023-12-27 12:27:38,707][20347] Updated weights for policy 0, policy_version 983030 (0.0010) [2023-12-27 12:27:38,855][20381] Updated weights for policy 1, policy_version 982238 (0.0007) [2023-12-27 12:27:38,910][20381] Updated weights for policy 1, policy_version 982248 (0.0007) [2023-12-27 12:27:38,973][20381] Updated weights for policy 1, policy_version 982258 (0.0008) [2023-12-27 12:27:39,394][20347] Updated weights for policy 0, policy_version 983040 (0.0009) [2023-12-27 12:27:39,456][20347] Updated weights for policy 0, policy_version 983050 (0.0008) [2023-12-27 12:27:39,518][20347] Updated weights for policy 0, policy_version 983060 (0.0008) [2023-12-27 12:27:39,714][20381] Updated weights for policy 1, policy_version 982268 (0.0008) [2023-12-27 12:27:39,778][20381] Updated weights for policy 1, policy_version 982278 (0.0007) [2023-12-27 12:27:39,845][20381] Updated weights for policy 1, policy_version 982288 (0.0008) [2023-12-27 12:27:40,290][20347] Updated weights for policy 0, policy_version 983070 (0.0007) [2023-12-27 12:27:40,344][20347] Updated weights for policy 0, policy_version 983080 (0.0007) [2023-12-27 12:27:40,405][20347] Updated weights for policy 0, policy_version 983090 (0.0009) [2023-12-27 12:27:40,609][20381] Updated weights for policy 1, policy_version 982298 (0.0008) [2023-12-27 12:27:40,672][20381] Updated weights for policy 1, policy_version 982308 (0.0006) [2023-12-27 12:27:40,732][20381] Updated weights for policy 1, policy_version 982318 (0.0006) [2023-12-27 12:27:40,786][20381] Updated weights for policy 1, policy_version 982328 (0.0005) [2023-12-27 12:27:41,185][20347] Updated weights for policy 0, policy_version 983100 (0.0007) [2023-12-27 12:27:41,238][20347] Updated weights for policy 0, policy_version 983110 (0.0008) [2023-12-27 12:27:41,299][20347] Updated weights for policy 0, policy_version 983120 (0.0008) [2023-12-27 12:27:41,411][20381] Updated weights for policy 1, policy_version 982338 (0.0009) [2023-12-27 12:27:41,465][20381] Updated weights for policy 1, policy_version 982348 (0.0008) [2023-12-27 12:27:41,518][20381] Updated weights for policy 1, policy_version 982358 (0.0005) [2023-12-27 12:27:42,167][20347] Updated weights for policy 0, policy_version 983130 (0.0008) [2023-12-27 12:27:42,222][20381] Updated weights for policy 1, policy_version 982368 (0.0007) [2023-12-27 12:27:42,234][20347] Updated weights for policy 0, policy_version 983140 (0.0006) [2023-12-27 12:27:42,288][20381] Updated weights for policy 1, policy_version 982378 (0.0009) [2023-12-27 12:27:42,297][20347] Updated weights for policy 0, policy_version 983150 (0.0007) [2023-12-27 12:27:42,346][20381] Updated weights for policy 1, policy_version 982388 (0.0006) [2023-12-27 12:27:42,354][20347] Updated weights for policy 0, policy_version 983160 (0.0007) [2023-12-27 12:27:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 503242752. Throughput: 0: 10347.5, 1: 10223.5. Samples: 503236704. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:42,363][19170] Avg episode reward: [(0, '875.520'), (1, '913.680')] [2023-12-27 12:27:43,004][20381] Updated weights for policy 1, policy_version 982398 (0.0007) [2023-12-27 12:27:43,017][20347] Updated weights for policy 0, policy_version 983170 (0.0007) [2023-12-27 12:27:43,050][20381] Updated weights for policy 1, policy_version 982408 (0.0005) [2023-12-27 12:27:43,069][20347] Updated weights for policy 0, policy_version 983180 (0.0008) [2023-12-27 12:27:43,096][20381] Updated weights for policy 1, policy_version 982418 (0.0005) [2023-12-27 12:27:43,121][20347] Updated weights for policy 0, policy_version 983190 (0.0008) [2023-12-27 12:27:43,623][20381] Updated weights for policy 1, policy_version 982428 (0.0005) [2023-12-27 12:27:43,669][20381] Updated weights for policy 1, policy_version 982438 (0.0005) [2023-12-27 12:27:43,723][20381] Updated weights for policy 1, policy_version 982448 (0.0007) [2023-12-27 12:27:43,886][20347] Updated weights for policy 0, policy_version 983200 (0.0007) [2023-12-27 12:27:43,946][20347] Updated weights for policy 0, policy_version 983210 (0.0007) [2023-12-27 12:27:44,003][20347] Updated weights for policy 0, policy_version 983220 (0.0007) [2023-12-27 12:27:44,402][20381] Updated weights for policy 1, policy_version 982458 (0.0011) [2023-12-27 12:27:44,454][20381] Updated weights for policy 1, policy_version 982468 (0.0011) [2023-12-27 12:27:44,509][20381] Updated weights for policy 1, policy_version 982478 (0.0009) [2023-12-27 12:27:44,561][20381] Updated weights for policy 1, policy_version 982488 (0.0006) [2023-12-27 12:27:44,698][20347] Updated weights for policy 0, policy_version 983230 (0.0008) [2023-12-27 12:27:44,764][20347] Updated weights for policy 0, policy_version 983240 (0.0008) [2023-12-27 12:27:44,823][20347] Updated weights for policy 0, policy_version 983250 (0.0007) [2023-12-27 12:27:45,295][20381] Updated weights for policy 1, policy_version 982498 (0.0007) [2023-12-27 12:27:45,350][20381] Updated weights for policy 1, policy_version 982508 (0.0008) [2023-12-27 12:27:45,409][20381] Updated weights for policy 1, policy_version 982518 (0.0008) [2023-12-27 12:27:45,512][20347] Updated weights for policy 0, policy_version 983260 (0.0007) [2023-12-27 12:27:45,565][20347] Updated weights for policy 0, policy_version 983270 (0.0009) [2023-12-27 12:27:45,626][20347] Updated weights for policy 0, policy_version 983280 (0.0008) [2023-12-27 12:27:46,029][20381] Updated weights for policy 1, policy_version 982528 (0.0006) [2023-12-27 12:27:46,095][20381] Updated weights for policy 1, policy_version 982538 (0.0005) [2023-12-27 12:27:46,158][20381] Updated weights for policy 1, policy_version 982548 (0.0005) [2023-12-27 12:27:46,369][20347] Updated weights for policy 0, policy_version 983290 (0.0007) [2023-12-27 12:27:46,435][20347] Updated weights for policy 0, policy_version 983300 (0.0008) [2023-12-27 12:27:46,498][20347] Updated weights for policy 0, policy_version 983310 (0.0009) [2023-12-27 12:27:46,545][20347] Updated weights for policy 0, policy_version 983320 (0.0009) [2023-12-27 12:27:46,682][20381] Updated weights for policy 1, policy_version 982558 (0.0006) [2023-12-27 12:27:46,741][20381] Updated weights for policy 1, policy_version 982568 (0.0008) [2023-12-27 12:27:46,798][20381] Updated weights for policy 1, policy_version 982578 (0.0007) [2023-12-27 12:27:47,266][20347] Updated weights for policy 0, policy_version 983330 (0.0008) [2023-12-27 12:27:47,314][20347] Updated weights for policy 0, policy_version 983340 (0.0008) [2023-12-27 12:27:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20244.0). Total num frames: 503341056. Throughput: 0: 10171.0, 1: 10309.1. Samples: 503357624. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:47,363][19170] Avg episode reward: [(0, '855.970'), (1, '922.170')] [2023-12-27 12:27:47,366][20347] Updated weights for policy 0, policy_version 983350 (0.0008) [2023-12-27 12:27:47,473][20381] Updated weights for policy 1, policy_version 982588 (0.0007) [2023-12-27 12:27:47,534][20381] Updated weights for policy 1, policy_version 982598 (0.0006) [2023-12-27 12:27:47,583][20381] Updated weights for policy 1, policy_version 982608 (0.0005) [2023-12-27 12:27:48,094][20347] Updated weights for policy 0, policy_version 983360 (0.0007) [2023-12-27 12:27:48,151][20347] Updated weights for policy 0, policy_version 983370 (0.0008) [2023-12-27 12:27:48,208][20347] Updated weights for policy 0, policy_version 983380 (0.0007) [2023-12-27 12:27:48,216][20381] Updated weights for policy 1, policy_version 982618 (0.0006) [2023-12-27 12:27:48,265][20292] KL-divergence is very high: 223.3725 [2023-12-27 12:27:48,273][20381] Updated weights for policy 1, policy_version 982628 (0.0007) [2023-12-27 12:27:48,326][20381] Updated weights for policy 1, policy_version 982638 (0.0008) [2023-12-27 12:27:48,389][20381] Updated weights for policy 1, policy_version 982648 (0.0007) [2023-12-27 12:27:48,943][20347] Updated weights for policy 0, policy_version 983390 (0.0008) [2023-12-27 12:27:49,001][20381] Updated weights for policy 1, policy_version 982658 (0.0006) [2023-12-27 12:27:49,002][20347] Updated weights for policy 0, policy_version 983400 (0.0006) [2023-12-27 12:27:49,050][20381] Updated weights for policy 1, policy_version 982668 (0.0005) [2023-12-27 12:27:49,055][20347] Updated weights for policy 0, policy_version 983410 (0.0006) [2023-12-27 12:27:49,099][20381] Updated weights for policy 1, policy_version 982678 (0.0006) [2023-12-27 12:27:49,716][20347] Updated weights for policy 0, policy_version 983420 (0.0007) [2023-12-27 12:27:49,768][20347] Updated weights for policy 0, policy_version 983430 (0.0007) [2023-12-27 12:27:49,794][20381] Updated weights for policy 1, policy_version 982688 (0.0008) [2023-12-27 12:27:49,824][20347] Updated weights for policy 0, policy_version 983440 (0.0006) [2023-12-27 12:27:49,863][20381] Updated weights for policy 1, policy_version 982698 (0.0009) [2023-12-27 12:27:49,919][20381] Updated weights for policy 1, policy_version 982708 (0.0008) [2023-12-27 12:27:50,564][20347] Updated weights for policy 0, policy_version 983450 (0.0008) [2023-12-27 12:27:50,620][20347] Updated weights for policy 0, policy_version 983460 (0.0007) [2023-12-27 12:27:50,660][20381] Updated weights for policy 1, policy_version 982718 (0.0007) [2023-12-27 12:27:50,680][20347] Updated weights for policy 0, policy_version 983470 (0.0008) [2023-12-27 12:27:50,721][20381] Updated weights for policy 1, policy_version 982728 (0.0006) [2023-12-27 12:27:50,741][20347] Updated weights for policy 0, policy_version 983480 (0.0007) [2023-12-27 12:27:50,783][20381] Updated weights for policy 1, policy_version 982738 (0.0007) [2023-12-27 12:27:51,406][20347] Updated weights for policy 0, policy_version 983490 (0.0008) [2023-12-27 12:27:51,458][20347] Updated weights for policy 0, policy_version 983500 (0.0008) [2023-12-27 12:27:51,509][20347] Updated weights for policy 0, policy_version 983510 (0.0008) [2023-12-27 12:27:51,542][20381] Updated weights for policy 1, policy_version 982748 (0.0007) [2023-12-27 12:27:51,602][20381] Updated weights for policy 1, policy_version 982758 (0.0007) [2023-12-27 12:27:51,663][20381] Updated weights for policy 1, policy_version 982768 (0.0008) [2023-12-27 12:27:52,253][20381] Updated weights for policy 1, policy_version 982778 (0.0008) [2023-12-27 12:27:52,293][20347] Updated weights for policy 0, policy_version 983520 (0.0010) [2023-12-27 12:27:52,314][20381] Updated weights for policy 1, policy_version 982788 (0.0010) [2023-12-27 12:27:52,350][20347] Updated weights for policy 0, policy_version 983530 (0.0010) [2023-12-27 12:27:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 503439360. Throughput: 0: 10103.3, 1: 10332.1. Samples: 503418008. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:52,363][19170] Avg episode reward: [(0, '893.260'), (1, '912.990')] [2023-12-27 12:27:52,375][20381] Updated weights for policy 1, policy_version 982798 (0.0011) [2023-12-27 12:27:52,416][20347] Updated weights for policy 0, policy_version 983540 (0.0009) [2023-12-27 12:27:52,440][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000983544_251830272.pth... [2023-12-27 12:27:52,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000982360_251527168.pth [2023-12-27 12:27:52,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000982808_251625472.pth... [2023-12-27 12:27:52,458][20381] Updated weights for policy 1, policy_version 982808 (0.0011) [2023-12-27 12:27:52,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000981560_251305984.pth [2023-12-27 12:27:53,076][20347] Updated weights for policy 0, policy_version 983550 (0.0006) [2023-12-27 12:27:53,103][20381] Updated weights for policy 1, policy_version 982818 (0.0011) [2023-12-27 12:27:53,124][20347] Updated weights for policy 0, policy_version 983560 (0.0008) [2023-12-27 12:27:53,151][20381] Updated weights for policy 1, policy_version 982828 (0.0010) [2023-12-27 12:27:53,173][20347] Updated weights for policy 0, policy_version 983570 (0.0006) [2023-12-27 12:27:53,199][20381] Updated weights for policy 1, policy_version 982838 (0.0010) [2023-12-27 12:27:53,765][20347] Updated weights for policy 0, policy_version 983580 (0.0005) [2023-12-27 12:27:53,813][20347] Updated weights for policy 0, policy_version 983590 (0.0008) [2023-12-27 12:27:53,868][20347] Updated weights for policy 0, policy_version 983600 (0.0008) [2023-12-27 12:27:53,871][20381] Updated weights for policy 1, policy_version 982848 (0.0010) [2023-12-27 12:27:53,929][20381] Updated weights for policy 1, policy_version 982858 (0.0011) [2023-12-27 12:27:53,991][20381] Updated weights for policy 1, policy_version 982868 (0.0011) [2023-12-27 12:27:54,549][20347] Updated weights for policy 0, policy_version 983610 (0.0007) [2023-12-27 12:27:54,606][20347] Updated weights for policy 0, policy_version 983620 (0.0007) [2023-12-27 12:27:54,652][20347] Updated weights for policy 0, policy_version 983630 (0.0006) [2023-12-27 12:27:54,664][20381] Updated weights for policy 1, policy_version 982878 (0.0011) [2023-12-27 12:27:54,713][20347] Updated weights for policy 0, policy_version 983640 (0.0005) [2023-12-27 12:27:54,725][20381] Updated weights for policy 1, policy_version 982888 (0.0011) [2023-12-27 12:27:54,773][20381] Updated weights for policy 1, policy_version 982898 (0.0010) [2023-12-27 12:27:55,372][20347] Updated weights for policy 0, policy_version 983650 (0.0006) [2023-12-27 12:27:55,412][20381] Updated weights for policy 1, policy_version 982908 (0.0008) [2023-12-27 12:27:55,431][20347] Updated weights for policy 0, policy_version 983660 (0.0007) [2023-12-27 12:27:55,465][20381] Updated weights for policy 1, policy_version 982918 (0.0007) [2023-12-27 12:27:55,483][20347] Updated weights for policy 0, policy_version 983670 (0.0005) [2023-12-27 12:27:55,520][20381] Updated weights for policy 1, policy_version 982928 (0.0010) [2023-12-27 12:27:56,133][20381] Updated weights for policy 1, policy_version 982938 (0.0010) [2023-12-27 12:27:56,135][20347] Updated weights for policy 0, policy_version 983680 (0.0009) [2023-12-27 12:27:56,192][20381] Updated weights for policy 1, policy_version 982948 (0.0010) [2023-12-27 12:27:56,197][20347] Updated weights for policy 0, policy_version 983690 (0.0010) [2023-12-27 12:27:56,249][20347] Updated weights for policy 0, policy_version 983700 (0.0010) [2023-12-27 12:27:56,250][20381] Updated weights for policy 1, policy_version 982958 (0.0010) [2023-12-27 12:27:56,301][20381] Updated weights for policy 1, policy_version 982968 (0.0010) [2023-12-27 12:27:56,827][20347] Updated weights for policy 0, policy_version 983710 (0.0007) [2023-12-27 12:27:56,882][20347] Updated weights for policy 0, policy_version 983720 (0.0005) [2023-12-27 12:27:56,936][20347] Updated weights for policy 0, policy_version 983730 (0.0005) [2023-12-27 12:27:56,985][20381] Updated weights for policy 1, policy_version 982978 (0.0011) [2023-12-27 12:27:57,027][20381] Updated weights for policy 1, policy_version 982988 (0.0010) [2023-12-27 12:27:57,072][20381] Updated weights for policy 1, policy_version 982998 (0.0009) [2023-12-27 12:27:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20299.5). Total num frames: 503554048. Throughput: 0: 10138.5, 1: 10419.3. Samples: 503542020. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:27:57,363][19170] Avg episode reward: [(0, '893.330'), (1, '845.110')] [2023-12-27 12:27:57,496][20347] Updated weights for policy 0, policy_version 983740 (0.0008) [2023-12-27 12:27:57,556][20347] Updated weights for policy 0, policy_version 983750 (0.0010) [2023-12-27 12:27:57,610][20347] Updated weights for policy 0, policy_version 983760 (0.0010) [2023-12-27 12:27:57,724][20381] Updated weights for policy 1, policy_version 983008 (0.0007) [2023-12-27 12:27:57,785][20381] Updated weights for policy 1, policy_version 983018 (0.0005) [2023-12-27 12:27:57,847][20292] KL-divergence is very high: 179.6122 [2023-12-27 12:27:57,853][20381] Updated weights for policy 1, policy_version 983028 (0.0006) [2023-12-27 12:27:58,156][20347] Updated weights for policy 0, policy_version 983770 (0.0009) [2023-12-27 12:27:58,216][20347] Updated weights for policy 0, policy_version 983780 (0.0008) [2023-12-27 12:27:58,269][20347] Updated weights for policy 0, policy_version 983790 (0.0006) [2023-12-27 12:27:58,337][20347] Updated weights for policy 0, policy_version 983800 (0.0006) [2023-12-27 12:27:58,508][20381] Updated weights for policy 1, policy_version 983038 (0.0010) [2023-12-27 12:27:58,565][20381] Updated weights for policy 1, policy_version 983048 (0.0011) [2023-12-27 12:27:58,628][20381] Updated weights for policy 1, policy_version 983058 (0.0008) [2023-12-27 12:27:59,130][20347] Updated weights for policy 0, policy_version 983810 (0.0011) [2023-12-27 12:27:59,199][20347] Updated weights for policy 0, policy_version 983820 (0.0010) [2023-12-27 12:27:59,260][20347] Updated weights for policy 0, policy_version 983830 (0.0010) [2023-12-27 12:27:59,433][20381] Updated weights for policy 1, policy_version 983068 (0.0009) [2023-12-27 12:27:59,490][20381] Updated weights for policy 1, policy_version 983079 (0.0010) [2023-12-27 12:27:59,546][20381] Updated weights for policy 1, policy_version 983089 (0.0006) [2023-12-27 12:27:59,932][20347] Updated weights for policy 0, policy_version 983840 (0.0008) [2023-12-27 12:27:59,997][20347] Updated weights for policy 0, policy_version 983850 (0.0008) [2023-12-27 12:28:00,064][20347] Updated weights for policy 0, policy_version 983860 (0.0007) [2023-12-27 12:28:00,236][20381] Updated weights for policy 1, policy_version 983099 (0.0006) [2023-12-27 12:28:00,291][20381] Updated weights for policy 1, policy_version 983109 (0.0006) [2023-12-27 12:28:00,350][20381] Updated weights for policy 1, policy_version 983119 (0.0008) [2023-12-27 12:28:00,752][20347] Updated weights for policy 0, policy_version 983870 (0.0008) [2023-12-27 12:28:00,796][20347] Updated weights for policy 0, policy_version 983880 (0.0007) [2023-12-27 12:28:00,850][20347] Updated weights for policy 0, policy_version 983890 (0.0007) [2023-12-27 12:28:01,001][20381] Updated weights for policy 1, policy_version 983129 (0.0007) [2023-12-27 12:28:01,067][20381] Updated weights for policy 1, policy_version 983139 (0.0008) [2023-12-27 12:28:01,132][20381] Updated weights for policy 1, policy_version 983149 (0.0006) [2023-12-27 12:28:01,194][20381] Updated weights for policy 1, policy_version 983159 (0.0008) [2023-12-27 12:28:01,540][20347] Updated weights for policy 0, policy_version 983900 (0.0007) [2023-12-27 12:28:01,587][20347] Updated weights for policy 0, policy_version 983910 (0.0007) [2023-12-27 12:28:01,652][20347] Updated weights for policy 0, policy_version 983920 (0.0007) [2023-12-27 12:28:01,921][20381] Updated weights for policy 1, policy_version 983169 (0.0008) [2023-12-27 12:28:01,977][20381] Updated weights for policy 1, policy_version 983179 (0.0008) [2023-12-27 12:28:01,985][20292] KL-divergence is very high: 118.3216 [2023-12-27 12:28:02,039][20381] Updated weights for policy 1, policy_version 983189 (0.0007) [2023-12-27 12:28:02,295][20347] Updated weights for policy 0, policy_version 983930 (0.0006) [2023-12-27 12:28:02,354][20347] Updated weights for policy 0, policy_version 983940 (0.0007) [2023-12-27 12:28:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 503652352. Throughput: 0: 10139.1, 1: 10372.8. Samples: 503664808. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:02,363][19170] Avg episode reward: [(0, '873.600'), (1, '893.070')] [2023-12-27 12:28:02,410][20347] Updated weights for policy 0, policy_version 983950 (0.0007) [2023-12-27 12:28:02,464][20347] Updated weights for policy 0, policy_version 983960 (0.0008) [2023-12-27 12:28:02,735][20381] Updated weights for policy 1, policy_version 983199 (0.0006) [2023-12-27 12:28:02,782][20381] Updated weights for policy 1, policy_version 983209 (0.0005) [2023-12-27 12:28:02,833][20381] Updated weights for policy 1, policy_version 983219 (0.0007) [2023-12-27 12:28:03,167][20347] Updated weights for policy 0, policy_version 983970 (0.0007) [2023-12-27 12:28:03,220][20347] Updated weights for policy 0, policy_version 983980 (0.0006) [2023-12-27 12:28:03,268][20347] Updated weights for policy 0, policy_version 983990 (0.0005) [2023-12-27 12:28:03,470][20381] Updated weights for policy 1, policy_version 983229 (0.0007) [2023-12-27 12:28:03,518][20381] Updated weights for policy 1, policy_version 983239 (0.0008) [2023-12-27 12:28:03,562][20381] Updated weights for policy 1, policy_version 983249 (0.0007) [2023-12-27 12:28:03,838][20347] Updated weights for policy 0, policy_version 984000 (0.0009) [2023-12-27 12:28:03,898][20347] Updated weights for policy 0, policy_version 984010 (0.0008) [2023-12-27 12:28:03,959][20347] Updated weights for policy 0, policy_version 984020 (0.0007) [2023-12-27 12:28:04,325][20381] Updated weights for policy 1, policy_version 983259 (0.0009) [2023-12-27 12:28:04,385][20381] Updated weights for policy 1, policy_version 983269 (0.0008) [2023-12-27 12:28:04,445][20381] Updated weights for policy 1, policy_version 983279 (0.0008) [2023-12-27 12:28:04,591][20347] Updated weights for policy 0, policy_version 984030 (0.0007) [2023-12-27 12:28:04,648][20347] Updated weights for policy 0, policy_version 984040 (0.0008) [2023-12-27 12:28:04,703][20347] Updated weights for policy 0, policy_version 984050 (0.0007) [2023-12-27 12:28:05,065][20381] Updated weights for policy 1, policy_version 983289 (0.0008) [2023-12-27 12:28:05,112][20381] Updated weights for policy 1, policy_version 983299 (0.0008) [2023-12-27 12:28:05,164][20381] Updated weights for policy 1, policy_version 983309 (0.0006) [2023-12-27 12:28:05,213][20381] Updated weights for policy 1, policy_version 983319 (0.0010) [2023-12-27 12:28:05,395][20347] Updated weights for policy 0, policy_version 984060 (0.0009) [2023-12-27 12:28:05,456][20347] Updated weights for policy 0, policy_version 984070 (0.0010) [2023-12-27 12:28:05,505][20347] Updated weights for policy 0, policy_version 984080 (0.0010) [2023-12-27 12:28:05,839][20381] Updated weights for policy 1, policy_version 983329 (0.0010) [2023-12-27 12:28:05,894][20381] Updated weights for policy 1, policy_version 983339 (0.0010) [2023-12-27 12:28:05,948][20381] Updated weights for policy 1, policy_version 983349 (0.0011) [2023-12-27 12:28:06,219][20347] Updated weights for policy 0, policy_version 984090 (0.0010) [2023-12-27 12:28:06,278][20347] Updated weights for policy 0, policy_version 984100 (0.0008) [2023-12-27 12:28:06,339][20347] Updated weights for policy 0, policy_version 984110 (0.0007) [2023-12-27 12:28:06,398][20347] Updated weights for policy 0, policy_version 984120 (0.0007) [2023-12-27 12:28:06,670][20381] Updated weights for policy 1, policy_version 983359 (0.0007) [2023-12-27 12:28:06,728][20381] Updated weights for policy 1, policy_version 983369 (0.0007) [2023-12-27 12:28:06,780][20381] Updated weights for policy 1, policy_version 983379 (0.0007) [2023-12-27 12:28:07,076][20347] Updated weights for policy 0, policy_version 984130 (0.0006) [2023-12-27 12:28:07,127][20347] Updated weights for policy 0, policy_version 984140 (0.0005) [2023-12-27 12:28:07,184][20347] Updated weights for policy 0, policy_version 984150 (0.0006) [2023-12-27 12:28:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 503758848. Throughput: 0: 10152.9, 1: 10381.9. Samples: 503727688. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:07,363][19170] Avg episode reward: [(0, '863.540'), (1, '922.030')] [2023-12-27 12:28:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000983384_251772928.pth... [2023-12-27 12:28:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000984152_251985920.pth... [2023-12-27 12:28:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000982200_251469824.pth [2023-12-27 12:28:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000982968_251682816.pth [2023-12-27 12:28:07,483][20381] Updated weights for policy 1, policy_version 983389 (0.0010) [2023-12-27 12:28:07,531][20381] Updated weights for policy 1, policy_version 983399 (0.0011) [2023-12-27 12:28:07,590][20381] Updated weights for policy 1, policy_version 983409 (0.0010) [2023-12-27 12:28:07,791][20347] Updated weights for policy 0, policy_version 984160 (0.0007) [2023-12-27 12:28:07,857][20347] Updated weights for policy 0, policy_version 984170 (0.0007) [2023-12-27 12:28:07,918][20347] Updated weights for policy 0, policy_version 984180 (0.0007) [2023-12-27 12:28:08,287][20381] Updated weights for policy 1, policy_version 983419 (0.0010) [2023-12-27 12:28:08,345][20381] Updated weights for policy 1, policy_version 983429 (0.0008) [2023-12-27 12:28:08,411][20381] Updated weights for policy 1, policy_version 983439 (0.0008) [2023-12-27 12:28:08,598][20347] Updated weights for policy 0, policy_version 984190 (0.0007) [2023-12-27 12:28:08,660][20347] Updated weights for policy 0, policy_version 984200 (0.0007) [2023-12-27 12:28:08,713][20347] Updated weights for policy 0, policy_version 984210 (0.0007) [2023-12-27 12:28:09,006][20381] Updated weights for policy 1, policy_version 983449 (0.0008) [2023-12-27 12:28:09,065][20381] Updated weights for policy 1, policy_version 983459 (0.0011) [2023-12-27 12:28:09,122][20381] Updated weights for policy 1, policy_version 983469 (0.0011) [2023-12-27 12:28:09,177][20381] Updated weights for policy 1, policy_version 983479 (0.0010) [2023-12-27 12:28:09,468][20347] Updated weights for policy 0, policy_version 984220 (0.0008) [2023-12-27 12:28:09,533][20347] Updated weights for policy 0, policy_version 984230 (0.0010) [2023-12-27 12:28:09,597][20347] Updated weights for policy 0, policy_version 984240 (0.0007) [2023-12-27 12:28:09,927][20381] Updated weights for policy 1, policy_version 983489 (0.0011) [2023-12-27 12:28:09,989][20381] Updated weights for policy 1, policy_version 983499 (0.0011) [2023-12-27 12:28:10,049][20381] Updated weights for policy 1, policy_version 983509 (0.0011) [2023-12-27 12:28:10,302][20347] Updated weights for policy 0, policy_version 984250 (0.0010) [2023-12-27 12:28:10,361][20347] Updated weights for policy 0, policy_version 984260 (0.0011) [2023-12-27 12:28:10,420][20347] Updated weights for policy 0, policy_version 984270 (0.0010) [2023-12-27 12:28:10,479][20347] Updated weights for policy 0, policy_version 984280 (0.0010) [2023-12-27 12:28:10,769][20381] Updated weights for policy 1, policy_version 983519 (0.0009) [2023-12-27 12:28:10,831][20381] Updated weights for policy 1, policy_version 983529 (0.0007) [2023-12-27 12:28:10,894][20381] Updated weights for policy 1, policy_version 983539 (0.0008) [2023-12-27 12:28:11,155][20347] Updated weights for policy 0, policy_version 984290 (0.0010) [2023-12-27 12:28:11,217][20347] Updated weights for policy 0, policy_version 984300 (0.0007) [2023-12-27 12:28:11,288][20347] Updated weights for policy 0, policy_version 984310 (0.0008) [2023-12-27 12:28:11,705][20381] Updated weights for policy 1, policy_version 983549 (0.0009) [2023-12-27 12:28:11,778][20381] Updated weights for policy 1, policy_version 983559 (0.0008) [2023-12-27 12:28:11,840][20381] Updated weights for policy 1, policy_version 983569 (0.0006) [2023-12-27 12:28:11,984][20347] Updated weights for policy 0, policy_version 984320 (0.0007) [2023-12-27 12:28:12,044][20347] Updated weights for policy 0, policy_version 984330 (0.0008) [2023-12-27 12:28:12,107][20347] Updated weights for policy 0, policy_version 984340 (0.0006) [2023-12-27 12:28:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 503857152. Throughput: 0: 10169.7, 1: 10308.0. Samples: 503848528. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:12,363][19170] Avg episode reward: [(0, '747.150'), (1, '903.230')] [2023-12-27 12:28:12,535][20381] Updated weights for policy 1, policy_version 983579 (0.0006) [2023-12-27 12:28:12,582][20381] Updated weights for policy 1, policy_version 983589 (0.0005) [2023-12-27 12:28:12,644][20381] Updated weights for policy 1, policy_version 983599 (0.0005) [2023-12-27 12:28:12,814][20347] Updated weights for policy 0, policy_version 984350 (0.0007) [2023-12-27 12:28:12,858][20291] KL-divergence is very high: 101.2691 [2023-12-27 12:28:12,876][20347] Updated weights for policy 0, policy_version 984360 (0.0007) [2023-12-27 12:28:12,880][20291] KL-divergence is very high: 100.1581 [2023-12-27 12:28:12,928][20347] Updated weights for policy 0, policy_version 984370 (0.0007) [2023-12-27 12:28:13,297][20381] Updated weights for policy 1, policy_version 983609 (0.0006) [2023-12-27 12:28:13,345][20381] Updated weights for policy 1, policy_version 983619 (0.0007) [2023-12-27 12:28:13,392][20381] Updated weights for policy 1, policy_version 983629 (0.0005) [2023-12-27 12:28:13,438][20381] Updated weights for policy 1, policy_version 983639 (0.0005) [2023-12-27 12:28:13,543][20347] Updated weights for policy 0, policy_version 984380 (0.0007) [2023-12-27 12:28:13,588][20347] Updated weights for policy 0, policy_version 984390 (0.0007) [2023-12-27 12:28:13,643][20347] Updated weights for policy 0, policy_version 984400 (0.0006) [2023-12-27 12:28:14,007][20381] Updated weights for policy 1, policy_version 983649 (0.0005) [2023-12-27 12:28:14,065][20381] Updated weights for policy 1, policy_version 983659 (0.0007) [2023-12-27 12:28:14,121][20381] Updated weights for policy 1, policy_version 983669 (0.0008) [2023-12-27 12:28:14,300][20347] Updated weights for policy 0, policy_version 984410 (0.0005) [2023-12-27 12:28:14,354][20347] Updated weights for policy 0, policy_version 984420 (0.0005) [2023-12-27 12:28:14,402][20347] Updated weights for policy 0, policy_version 984430 (0.0005) [2023-12-27 12:28:14,457][20347] Updated weights for policy 0, policy_version 984440 (0.0006) [2023-12-27 12:28:14,702][20381] Updated weights for policy 1, policy_version 983679 (0.0006) [2023-12-27 12:28:14,753][20381] Updated weights for policy 1, policy_version 983689 (0.0007) [2023-12-27 12:28:14,816][20381] Updated weights for policy 1, policy_version 983699 (0.0008) [2023-12-27 12:28:15,146][20347] Updated weights for policy 0, policy_version 984450 (0.0007) [2023-12-27 12:28:15,210][20347] Updated weights for policy 0, policy_version 984460 (0.0008) [2023-12-27 12:28:15,266][20347] Updated weights for policy 0, policy_version 984470 (0.0008) [2023-12-27 12:28:15,551][20381] Updated weights for policy 1, policy_version 983709 (0.0007) [2023-12-27 12:28:15,598][20381] Updated weights for policy 1, policy_version 983719 (0.0005) [2023-12-27 12:28:15,657][20381] Updated weights for policy 1, policy_version 983729 (0.0006) [2023-12-27 12:28:15,823][20347] Updated weights for policy 0, policy_version 984480 (0.0006) [2023-12-27 12:28:15,876][20347] Updated weights for policy 0, policy_version 984490 (0.0006) [2023-12-27 12:28:15,924][20347] Updated weights for policy 0, policy_version 984500 (0.0009) [2023-12-27 12:28:16,355][20381] Updated weights for policy 1, policy_version 983739 (0.0008) [2023-12-27 12:28:16,412][20381] Updated weights for policy 1, policy_version 983749 (0.0005) [2023-12-27 12:28:16,474][20381] Updated weights for policy 1, policy_version 983759 (0.0006) [2023-12-27 12:28:16,555][20347] Updated weights for policy 0, policy_version 984510 (0.0010) [2023-12-27 12:28:16,612][20347] Updated weights for policy 0, policy_version 984520 (0.0010) [2023-12-27 12:28:16,663][20347] Updated weights for policy 0, policy_version 984530 (0.0010) [2023-12-27 12:28:17,126][20381] Updated weights for policy 1, policy_version 983769 (0.0007) [2023-12-27 12:28:17,189][20381] Updated weights for policy 1, policy_version 983779 (0.0007) [2023-12-27 12:28:17,252][20381] Updated weights for policy 1, policy_version 983789 (0.0008) [2023-12-27 12:28:17,314][20381] Updated weights for policy 1, policy_version 983799 (0.0008) [2023-12-27 12:28:17,357][20347] Updated weights for policy 0, policy_version 984540 (0.0010) [2023-12-27 12:28:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20299.6). Total num frames: 503963648. Throughput: 0: 10159.5, 1: 10308.9. Samples: 503973288. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:17,363][19170] Avg episode reward: [(0, '776.170'), (1, '864.230')] [2023-12-27 12:28:17,406][20347] Updated weights for policy 0, policy_version 984550 (0.0010) [2023-12-27 12:28:17,463][20347] Updated weights for policy 0, policy_version 984560 (0.0010) [2023-12-27 12:28:17,977][20381] Updated weights for policy 1, policy_version 983809 (0.0008) [2023-12-27 12:28:18,034][20381] Updated weights for policy 1, policy_version 983819 (0.0006) [2023-12-27 12:28:18,092][20381] Updated weights for policy 1, policy_version 983829 (0.0006) [2023-12-27 12:28:18,096][20347] Updated weights for policy 0, policy_version 984570 (0.0009) [2023-12-27 12:28:18,162][20347] Updated weights for policy 0, policy_version 984580 (0.0007) [2023-12-27 12:28:18,230][20347] Updated weights for policy 0, policy_version 984590 (0.0007) [2023-12-27 12:28:18,294][20347] Updated weights for policy 0, policy_version 984600 (0.0007) [2023-12-27 12:28:18,722][20381] Updated weights for policy 1, policy_version 983839 (0.0007) [2023-12-27 12:28:18,779][20381] Updated weights for policy 1, policy_version 983849 (0.0007) [2023-12-27 12:28:18,835][20381] Updated weights for policy 1, policy_version 983859 (0.0008) [2023-12-27 12:28:18,924][20347] Updated weights for policy 0, policy_version 984610 (0.0010) [2023-12-27 12:28:18,976][20347] Updated weights for policy 0, policy_version 984620 (0.0010) [2023-12-27 12:28:19,031][20347] Updated weights for policy 0, policy_version 984630 (0.0010) [2023-12-27 12:28:19,614][20381] Updated weights for policy 1, policy_version 983869 (0.0008) [2023-12-27 12:28:19,670][20381] Updated weights for policy 1, policy_version 983879 (0.0008) [2023-12-27 12:28:19,699][20347] Updated weights for policy 0, policy_version 984640 (0.0010) [2023-12-27 12:28:19,726][20381] Updated weights for policy 1, policy_version 983889 (0.0008) [2023-12-27 12:28:19,755][20347] Updated weights for policy 0, policy_version 984650 (0.0010) [2023-12-27 12:28:19,815][20347] Updated weights for policy 0, policy_version 984660 (0.0011) [2023-12-27 12:28:20,406][20381] Updated weights for policy 1, policy_version 983899 (0.0008) [2023-12-27 12:28:20,454][20381] Updated weights for policy 1, policy_version 983909 (0.0007) [2023-12-27 12:28:20,507][20381] Updated weights for policy 1, policy_version 983919 (0.0007) [2023-12-27 12:28:20,536][20347] Updated weights for policy 0, policy_version 984670 (0.0009) [2023-12-27 12:28:20,600][20347] Updated weights for policy 0, policy_version 984680 (0.0010) [2023-12-27 12:28:20,603][20291] KL-divergence is very high: 160.1056 [2023-12-27 12:28:20,616][20291] KL-divergence is very high: 204.0945 [2023-12-27 12:28:20,670][20347] Updated weights for policy 0, policy_version 984690 (0.0010) [2023-12-27 12:28:21,244][20381] Updated weights for policy 1, policy_version 983929 (0.0006) [2023-12-27 12:28:21,310][20381] Updated weights for policy 1, policy_version 983939 (0.0008) [2023-12-27 12:28:21,383][20381] Updated weights for policy 1, policy_version 983949 (0.0009) [2023-12-27 12:28:21,390][20347] Updated weights for policy 0, policy_version 984700 (0.0011) [2023-12-27 12:28:21,444][20381] Updated weights for policy 1, policy_version 983959 (0.0007) [2023-12-27 12:28:21,453][20347] Updated weights for policy 0, policy_version 984710 (0.0008) [2023-12-27 12:28:21,510][20347] Updated weights for policy 0, policy_version 984720 (0.0010) [2023-12-27 12:28:22,208][20381] Updated weights for policy 1, policy_version 983969 (0.0007) [2023-12-27 12:28:22,270][20381] Updated weights for policy 1, policy_version 983979 (0.0008) [2023-12-27 12:28:22,300][20347] Updated weights for policy 0, policy_version 984730 (0.0010) [2023-12-27 12:28:22,329][20381] Updated weights for policy 1, policy_version 983989 (0.0009) [2023-12-27 12:28:22,360][20347] Updated weights for policy 0, policy_version 984740 (0.0007) [2023-12-27 12:28:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 504061952. Throughput: 0: 10140.6, 1: 10272.1. Samples: 504033804. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:22,363][19170] Avg episode reward: [(0, '795.570'), (1, '873.860')] [2023-12-27 12:28:22,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000983992_251928576.pth... [2023-12-27 12:28:22,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000982808_251625472.pth [2023-12-27 12:28:22,426][20347] Updated weights for policy 0, policy_version 984750 (0.0007) [2023-12-27 12:28:22,474][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000984760_252141568.pth... [2023-12-27 12:28:22,475][20347] Updated weights for policy 0, policy_version 984760 (0.0008) [2023-12-27 12:28:22,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000983544_251830272.pth [2023-12-27 12:28:23,067][20381] Updated weights for policy 1, policy_version 983999 (0.0007) [2023-12-27 12:28:23,119][20381] Updated weights for policy 1, policy_version 984009 (0.0007) [2023-12-27 12:28:23,177][20347] Updated weights for policy 0, policy_version 984770 (0.0006) [2023-12-27 12:28:23,179][20381] Updated weights for policy 1, policy_version 984019 (0.0008) [2023-12-27 12:28:23,232][20347] Updated weights for policy 0, policy_version 984780 (0.0007) [2023-12-27 12:28:23,291][20347] Updated weights for policy 0, policy_version 984790 (0.0007) [2023-12-27 12:28:23,868][20381] Updated weights for policy 1, policy_version 984029 (0.0008) [2023-12-27 12:28:23,911][20292] KL-divergence is very high: 184.2314 [2023-12-27 12:28:23,924][20381] Updated weights for policy 1, policy_version 984039 (0.0007) [2023-12-27 12:28:23,941][20347] Updated weights for policy 0, policy_version 984800 (0.0006) [2023-12-27 12:28:23,980][20381] Updated weights for policy 1, policy_version 984049 (0.0005) [2023-12-27 12:28:23,998][20347] Updated weights for policy 0, policy_version 984810 (0.0006) [2023-12-27 12:28:24,052][20347] Updated weights for policy 0, policy_version 984820 (0.0007) [2023-12-27 12:28:24,690][20381] Updated weights for policy 1, policy_version 984059 (0.0007) [2023-12-27 12:28:24,744][20381] Updated weights for policy 1, policy_version 984069 (0.0007) [2023-12-27 12:28:24,753][20347] Updated weights for policy 0, policy_version 984830 (0.0007) [2023-12-27 12:28:24,800][20381] Updated weights for policy 1, policy_version 984079 (0.0006) [2023-12-27 12:28:24,809][20347] Updated weights for policy 0, policy_version 984840 (0.0005) [2023-12-27 12:28:24,862][20347] Updated weights for policy 0, policy_version 984850 (0.0006) [2023-12-27 12:28:25,432][20381] Updated weights for policy 1, policy_version 984089 (0.0006) [2023-12-27 12:28:25,499][20381] Updated weights for policy 1, policy_version 984099 (0.0006) [2023-12-27 12:28:25,532][20347] Updated weights for policy 0, policy_version 984860 (0.0006) [2023-12-27 12:28:25,560][20381] Updated weights for policy 1, policy_version 984109 (0.0007) [2023-12-27 12:28:25,581][20347] Updated weights for policy 0, policy_version 984870 (0.0005) [2023-12-27 12:28:25,618][20381] Updated weights for policy 1, policy_version 984119 (0.0005) [2023-12-27 12:28:25,637][20347] Updated weights for policy 0, policy_version 984880 (0.0006) [2023-12-27 12:28:26,189][20381] Updated weights for policy 1, policy_version 984129 (0.0005) [2023-12-27 12:28:26,248][20381] Updated weights for policy 1, policy_version 984139 (0.0005) [2023-12-27 12:28:26,286][20347] Updated weights for policy 0, policy_version 984890 (0.0008) [2023-12-27 12:28:26,298][20381] Updated weights for policy 1, policy_version 984149 (0.0007) [2023-12-27 12:28:26,299][20292] KL-divergence is very high: 134.0660 [2023-12-27 12:28:26,331][20347] Updated weights for policy 0, policy_version 984900 (0.0006) [2023-12-27 12:28:26,385][20347] Updated weights for policy 0, policy_version 984910 (0.0007) [2023-12-27 12:28:26,447][20347] Updated weights for policy 0, policy_version 984920 (0.0007) [2023-12-27 12:28:26,899][20381] Updated weights for policy 1, policy_version 984159 (0.0009) [2023-12-27 12:28:26,957][20381] Updated weights for policy 1, policy_version 984169 (0.0010) [2023-12-27 12:28:27,014][20381] Updated weights for policy 1, policy_version 984179 (0.0010) [2023-12-27 12:28:27,181][20347] Updated weights for policy 0, policy_version 984930 (0.0008) [2023-12-27 12:28:27,235][20347] Updated weights for policy 0, policy_version 984940 (0.0008) [2023-12-27 12:28:27,296][20347] Updated weights for policy 0, policy_version 984950 (0.0008) [2023-12-27 12:28:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 504168448. Throughput: 0: 10138.4, 1: 10224.6. Samples: 504153040. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:27,363][19170] Avg episode reward: [(0, '767.520'), (1, '913.160')] [2023-12-27 12:28:27,622][20381] Updated weights for policy 1, policy_version 984189 (0.0007) [2023-12-27 12:28:27,684][20381] Updated weights for policy 1, policy_version 984199 (0.0007) [2023-12-27 12:28:27,737][20381] Updated weights for policy 1, policy_version 984209 (0.0009) [2023-12-27 12:28:27,934][20347] Updated weights for policy 0, policy_version 984960 (0.0006) [2023-12-27 12:28:27,991][20347] Updated weights for policy 0, policy_version 984970 (0.0005) [2023-12-27 12:28:28,042][20347] Updated weights for policy 0, policy_version 984980 (0.0007) [2023-12-27 12:28:28,397][20381] Updated weights for policy 1, policy_version 984219 (0.0008) [2023-12-27 12:28:28,453][20381] Updated weights for policy 1, policy_version 984229 (0.0008) [2023-12-27 12:28:28,502][20381] Updated weights for policy 1, policy_version 984239 (0.0008) [2023-12-27 12:28:28,694][20347] Updated weights for policy 0, policy_version 984990 (0.0008) [2023-12-27 12:28:28,754][20347] Updated weights for policy 0, policy_version 985000 (0.0007) [2023-12-27 12:28:28,814][20347] Updated weights for policy 0, policy_version 985010 (0.0008) [2023-12-27 12:28:29,197][20381] Updated weights for policy 1, policy_version 984249 (0.0008) [2023-12-27 12:28:29,253][20381] Updated weights for policy 1, policy_version 984259 (0.0008) [2023-12-27 12:28:29,314][20381] Updated weights for policy 1, policy_version 984269 (0.0008) [2023-12-27 12:28:29,376][20381] Updated weights for policy 1, policy_version 984279 (0.0008) [2023-12-27 12:28:29,542][20347] Updated weights for policy 0, policy_version 985020 (0.0007) [2023-12-27 12:28:29,598][20347] Updated weights for policy 0, policy_version 985030 (0.0008) [2023-12-27 12:28:29,649][20347] Updated weights for policy 0, policy_version 985040 (0.0007) [2023-12-27 12:28:30,024][20381] Updated weights for policy 1, policy_version 984289 (0.0008) [2023-12-27 12:28:30,070][20381] Updated weights for policy 1, policy_version 984299 (0.0008) [2023-12-27 12:28:30,118][20381] Updated weights for policy 1, policy_version 984309 (0.0008) [2023-12-27 12:28:30,353][20347] Updated weights for policy 0, policy_version 985050 (0.0007) [2023-12-27 12:28:30,416][20347] Updated weights for policy 0, policy_version 985060 (0.0008) [2023-12-27 12:28:30,478][20347] Updated weights for policy 0, policy_version 985070 (0.0007) [2023-12-27 12:28:30,537][20347] Updated weights for policy 0, policy_version 985080 (0.0008) [2023-12-27 12:28:30,806][20381] Updated weights for policy 1, policy_version 984319 (0.0008) [2023-12-27 12:28:30,859][20381] Updated weights for policy 1, policy_version 984329 (0.0007) [2023-12-27 12:28:30,913][20381] Updated weights for policy 1, policy_version 984339 (0.0008) [2023-12-27 12:28:31,279][20347] Updated weights for policy 0, policy_version 985090 (0.0008) [2023-12-27 12:28:31,336][20347] Updated weights for policy 0, policy_version 985100 (0.0008) [2023-12-27 12:28:31,401][20347] Updated weights for policy 0, policy_version 985110 (0.0007) [2023-12-27 12:28:31,683][20381] Updated weights for policy 1, policy_version 984349 (0.0008) [2023-12-27 12:28:31,746][20381] Updated weights for policy 1, policy_version 984359 (0.0009) [2023-12-27 12:28:31,810][20381] Updated weights for policy 1, policy_version 984369 (0.0007) [2023-12-27 12:28:32,146][20347] Updated weights for policy 0, policy_version 985120 (0.0008) [2023-12-27 12:28:32,204][20347] Updated weights for policy 0, policy_version 985130 (0.0008) [2023-12-27 12:28:32,267][20347] Updated weights for policy 0, policy_version 985140 (0.0007) [2023-12-27 12:28:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 504266752. Throughput: 0: 10205.6, 1: 10175.0. Samples: 504274748. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:32,363][19170] Avg episode reward: [(0, '806.480'), (1, '894.230')] [2023-12-27 12:28:32,536][20381] Updated weights for policy 1, policy_version 984379 (0.0008) [2023-12-27 12:28:32,595][20381] Updated weights for policy 1, policy_version 984389 (0.0007) [2023-12-27 12:28:32,645][20381] Updated weights for policy 1, policy_version 984399 (0.0008) [2023-12-27 12:28:33,006][20347] Updated weights for policy 0, policy_version 985150 (0.0008) [2023-12-27 12:28:33,055][20347] Updated weights for policy 0, policy_version 985160 (0.0009) [2023-12-27 12:28:33,100][20347] Updated weights for policy 0, policy_version 985170 (0.0009) [2023-12-27 12:28:33,205][20381] Updated weights for policy 1, policy_version 984409 (0.0008) [2023-12-27 12:28:33,257][20381] Updated weights for policy 1, policy_version 984419 (0.0005) [2023-12-27 12:28:33,310][20381] Updated weights for policy 1, policy_version 984429 (0.0005) [2023-12-27 12:28:33,357][20381] Updated weights for policy 1, policy_version 984439 (0.0007) [2023-12-27 12:28:33,818][20347] Updated weights for policy 0, policy_version 985180 (0.0008) [2023-12-27 12:28:33,872][20347] Updated weights for policy 0, policy_version 985190 (0.0007) [2023-12-27 12:28:33,927][20347] Updated weights for policy 0, policy_version 985200 (0.0007) [2023-12-27 12:28:34,014][20381] Updated weights for policy 1, policy_version 984449 (0.0007) [2023-12-27 12:28:34,062][20381] Updated weights for policy 1, policy_version 984459 (0.0007) [2023-12-27 12:28:34,107][20381] Updated weights for policy 1, policy_version 984469 (0.0007) [2023-12-27 12:28:34,590][20347] Updated weights for policy 0, policy_version 985210 (0.0008) [2023-12-27 12:28:34,650][20347] Updated weights for policy 0, policy_version 985220 (0.0008) [2023-12-27 12:28:34,709][20347] Updated weights for policy 0, policy_version 985230 (0.0008) [2023-12-27 12:28:34,768][20347] Updated weights for policy 0, policy_version 985240 (0.0008) [2023-12-27 12:28:34,810][20381] Updated weights for policy 1, policy_version 984479 (0.0007) [2023-12-27 12:28:34,865][20381] Updated weights for policy 1, policy_version 984489 (0.0008) [2023-12-27 12:28:34,916][20381] Updated weights for policy 1, policy_version 984499 (0.0006) [2023-12-27 12:28:35,452][20347] Updated weights for policy 0, policy_version 985250 (0.0010) [2023-12-27 12:28:35,509][20347] Updated weights for policy 0, policy_version 985260 (0.0011) [2023-12-27 12:28:35,516][20381] Updated weights for policy 1, policy_version 984509 (0.0007) [2023-12-27 12:28:35,564][20381] Updated weights for policy 1, policy_version 984519 (0.0006) [2023-12-27 12:28:35,565][20347] Updated weights for policy 0, policy_version 985270 (0.0010) [2023-12-27 12:28:35,614][20381] Updated weights for policy 1, policy_version 984529 (0.0007) [2023-12-27 12:28:36,263][20347] Updated weights for policy 0, policy_version 985280 (0.0008) [2023-12-27 12:28:36,289][20381] Updated weights for policy 1, policy_version 984539 (0.0006) [2023-12-27 12:28:36,322][20347] Updated weights for policy 0, policy_version 985290 (0.0007) [2023-12-27 12:28:36,350][20381] Updated weights for policy 1, policy_version 984549 (0.0007) [2023-12-27 12:28:36,383][20347] Updated weights for policy 0, policy_version 985300 (0.0007) [2023-12-27 12:28:36,423][20381] Updated weights for policy 1, policy_version 984559 (0.0009) [2023-12-27 12:28:37,143][20347] Updated weights for policy 0, policy_version 985310 (0.0008) [2023-12-27 12:28:37,206][20347] Updated weights for policy 0, policy_version 985320 (0.0008) [2023-12-27 12:28:37,208][20381] Updated weights for policy 1, policy_version 984569 (0.0009) [2023-12-27 12:28:37,256][20347] Updated weights for policy 0, policy_version 985330 (0.0006) [2023-12-27 12:28:37,257][20381] Updated weights for policy 1, policy_version 984579 (0.0006) [2023-12-27 12:28:37,312][20381] Updated weights for policy 1, policy_version 984589 (0.0005) [2023-12-27 12:28:37,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20343.4, 300 sec: 20271.7). Total num frames: 504365056. Throughput: 0: 10218.3, 1: 10189.1. Samples: 504336344. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:37,364][19170] Avg episode reward: [(0, '883.210'), (1, '913.630')] [2023-12-27 12:28:37,371][20381] Updated weights for policy 1, policy_version 984599 (0.0006) [2023-12-27 12:28:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000985336_252289024.pth... [2023-12-27 12:28:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000984152_251985920.pth [2023-12-27 12:28:37,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000984600_252084224.pth... [2023-12-27 12:28:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000983384_251772928.pth [2023-12-27 12:28:37,975][20381] Updated weights for policy 1, policy_version 984609 (0.0006) [2023-12-27 12:28:37,990][20347] Updated weights for policy 0, policy_version 985340 (0.0008) [2023-12-27 12:28:38,034][20381] Updated weights for policy 1, policy_version 984619 (0.0006) [2023-12-27 12:28:38,045][20347] Updated weights for policy 0, policy_version 985350 (0.0008) [2023-12-27 12:28:38,090][20381] Updated weights for policy 1, policy_version 984629 (0.0006) [2023-12-27 12:28:38,109][20347] Updated weights for policy 0, policy_version 985360 (0.0007) [2023-12-27 12:28:38,733][20381] Updated weights for policy 1, policy_version 984639 (0.0007) [2023-12-27 12:28:38,792][20381] Updated weights for policy 1, policy_version 984649 (0.0008) [2023-12-27 12:28:38,817][20347] Updated weights for policy 0, policy_version 985370 (0.0008) [2023-12-27 12:28:38,849][20381] Updated weights for policy 1, policy_version 984659 (0.0006) [2023-12-27 12:28:38,878][20347] Updated weights for policy 0, policy_version 985380 (0.0008) [2023-12-27 12:28:38,937][20347] Updated weights for policy 0, policy_version 985390 (0.0008) [2023-12-27 12:28:38,992][20347] Updated weights for policy 0, policy_version 985400 (0.0007) [2023-12-27 12:28:39,534][20381] Updated weights for policy 1, policy_version 984669 (0.0008) [2023-12-27 12:28:39,556][20292] KL-divergence is very high: 176.8296 [2023-12-27 12:28:39,592][20381] Updated weights for policy 1, policy_version 984679 (0.0007) [2023-12-27 12:28:39,652][20381] Updated weights for policy 1, policy_version 984689 (0.0009) [2023-12-27 12:28:39,796][20347] Updated weights for policy 0, policy_version 985410 (0.0011) [2023-12-27 12:28:39,857][20347] Updated weights for policy 0, policy_version 985420 (0.0010) [2023-12-27 12:28:39,918][20347] Updated weights for policy 0, policy_version 985430 (0.0011) [2023-12-27 12:28:40,386][20381] Updated weights for policy 1, policy_version 984699 (0.0011) [2023-12-27 12:28:40,409][20292] KL-divergence is very high: 193.2093 [2023-12-27 12:28:40,415][20292] KL-divergence is very high: 627.9168 [2023-12-27 12:28:40,446][20381] Updated weights for policy 1, policy_version 984709 (0.0011) [2023-12-27 12:28:40,458][20292] KL-divergence is very high: 130.5928 [2023-12-27 12:28:40,509][20381] Updated weights for policy 1, policy_version 984719 (0.0011) [2023-12-27 12:28:40,644][20347] Updated weights for policy 0, policy_version 985440 (0.0011) [2023-12-27 12:28:40,706][20347] Updated weights for policy 0, policy_version 985450 (0.0008) [2023-12-27 12:28:40,767][20347] Updated weights for policy 0, policy_version 985460 (0.0008) [2023-12-27 12:28:41,238][20381] Updated weights for policy 1, policy_version 984729 (0.0010) [2023-12-27 12:28:41,304][20381] Updated weights for policy 1, policy_version 984739 (0.0008) [2023-12-27 12:28:41,369][20381] Updated weights for policy 1, policy_version 984749 (0.0012) [2023-12-27 12:28:41,436][20381] Updated weights for policy 1, policy_version 984759 (0.0008) [2023-12-27 12:28:41,472][20347] Updated weights for policy 0, policy_version 985470 (0.0009) [2023-12-27 12:28:41,531][20347] Updated weights for policy 0, policy_version 985480 (0.0008) [2023-12-27 12:28:41,591][20347] Updated weights for policy 0, policy_version 985490 (0.0008) [2023-12-27 12:28:42,189][20381] Updated weights for policy 1, policy_version 984769 (0.0007) [2023-12-27 12:28:42,239][20381] Updated weights for policy 1, policy_version 984779 (0.0007) [2023-12-27 12:28:42,298][20381] Updated weights for policy 1, policy_version 984789 (0.0007) [2023-12-27 12:28:42,344][20347] Updated weights for policy 0, policy_version 985500 (0.0009) [2023-12-27 12:28:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20271.7). Total num frames: 504463360. Throughput: 0: 10123.6, 1: 10172.7. Samples: 504455356. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:42,363][19170] Avg episode reward: [(0, '893.020'), (1, '932.110')] [2023-12-27 12:28:42,412][20347] Updated weights for policy 0, policy_version 985510 (0.0010) [2023-12-27 12:28:42,471][20347] Updated weights for policy 0, policy_version 985520 (0.0009) [2023-12-27 12:28:43,035][20381] Updated weights for policy 1, policy_version 984799 (0.0007) [2023-12-27 12:28:43,092][20381] Updated weights for policy 1, policy_version 984809 (0.0008) [2023-12-27 12:28:43,104][20347] Updated weights for policy 0, policy_version 985530 (0.0011) [2023-12-27 12:28:43,151][20381] Updated weights for policy 1, policy_version 984819 (0.0007) [2023-12-27 12:28:43,159][20347] Updated weights for policy 0, policy_version 985540 (0.0009) [2023-12-27 12:28:43,222][20347] Updated weights for policy 0, policy_version 985550 (0.0006) [2023-12-27 12:28:43,243][20291] KL-divergence is very high: 145.4556 [2023-12-27 12:28:43,283][20347] Updated weights for policy 0, policy_version 985560 (0.0010) [2023-12-27 12:28:43,823][20381] Updated weights for policy 1, policy_version 984829 (0.0006) [2023-12-27 12:28:43,882][20381] Updated weights for policy 1, policy_version 984839 (0.0006) [2023-12-27 12:28:43,924][20347] Updated weights for policy 0, policy_version 985570 (0.0006) [2023-12-27 12:28:43,942][20381] Updated weights for policy 1, policy_version 984849 (0.0007) [2023-12-27 12:28:43,983][20347] Updated weights for policy 0, policy_version 985580 (0.0005) [2023-12-27 12:28:44,048][20347] Updated weights for policy 0, policy_version 985590 (0.0008) [2023-12-27 12:28:44,503][20381] Updated weights for policy 1, policy_version 984859 (0.0006) [2023-12-27 12:28:44,562][20381] Updated weights for policy 1, policy_version 984869 (0.0008) [2023-12-27 12:28:44,618][20381] Updated weights for policy 1, policy_version 984879 (0.0008) [2023-12-27 12:28:44,681][20347] Updated weights for policy 0, policy_version 985600 (0.0006) [2023-12-27 12:28:44,733][20347] Updated weights for policy 0, policy_version 985610 (0.0007) [2023-12-27 12:28:44,795][20347] Updated weights for policy 0, policy_version 985620 (0.0009) [2023-12-27 12:28:45,318][20381] Updated weights for policy 1, policy_version 984889 (0.0007) [2023-12-27 12:28:45,380][20381] Updated weights for policy 1, policy_version 984899 (0.0008) [2023-12-27 12:28:45,433][20381] Updated weights for policy 1, policy_version 984909 (0.0009) [2023-12-27 12:28:45,475][20347] Updated weights for policy 0, policy_version 985630 (0.0007) [2023-12-27 12:28:45,488][20381] Updated weights for policy 1, policy_version 984919 (0.0008) [2023-12-27 12:28:45,525][20347] Updated weights for policy 0, policy_version 985640 (0.0005) [2023-12-27 12:28:45,580][20347] Updated weights for policy 0, policy_version 985650 (0.0005) [2023-12-27 12:28:46,218][20347] Updated weights for policy 0, policy_version 985660 (0.0006) [2023-12-27 12:28:46,220][20381] Updated weights for policy 1, policy_version 984929 (0.0010) [2023-12-27 12:28:46,279][20347] Updated weights for policy 0, policy_version 985670 (0.0005) [2023-12-27 12:28:46,281][20381] Updated weights for policy 1, policy_version 984939 (0.0011) [2023-12-27 12:28:46,339][20347] Updated weights for policy 0, policy_version 985680 (0.0009) [2023-12-27 12:28:46,340][20381] Updated weights for policy 1, policy_version 984949 (0.0008) [2023-12-27 12:28:46,953][20347] Updated weights for policy 0, policy_version 985690 (0.0009) [2023-12-27 12:28:47,014][20347] Updated weights for policy 0, policy_version 985700 (0.0005) [2023-12-27 12:28:47,022][20381] Updated weights for policy 1, policy_version 984959 (0.0011) [2023-12-27 12:28:47,072][20347] Updated weights for policy 0, policy_version 985710 (0.0005) [2023-12-27 12:28:47,079][20381] Updated weights for policy 1, policy_version 984969 (0.0010) [2023-12-27 12:28:47,129][20347] Updated weights for policy 0, policy_version 985720 (0.0005) [2023-12-27 12:28:47,135][20381] Updated weights for policy 1, policy_version 984979 (0.0009) [2023-12-27 12:28:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 504569856. Throughput: 0: 10098.2, 1: 10171.1. Samples: 504576928. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:47,363][19170] Avg episode reward: [(0, '884.120'), (1, '932.480')] [2023-12-27 12:28:47,782][20347] Updated weights for policy 0, policy_version 985730 (0.0010) [2023-12-27 12:28:47,837][20347] Updated weights for policy 0, policy_version 985740 (0.0010) [2023-12-27 12:28:47,849][20381] Updated weights for policy 1, policy_version 984989 (0.0010) [2023-12-27 12:28:47,892][20347] Updated weights for policy 0, policy_version 985750 (0.0010) [2023-12-27 12:28:47,908][20381] Updated weights for policy 1, policy_version 984999 (0.0011) [2023-12-27 12:28:47,968][20381] Updated weights for policy 1, policy_version 985009 (0.0011) [2023-12-27 12:28:48,600][20347] Updated weights for policy 0, policy_version 985760 (0.0008) [2023-12-27 12:28:48,646][20381] Updated weights for policy 1, policy_version 985019 (0.0011) [2023-12-27 12:28:48,657][20347] Updated weights for policy 0, policy_version 985770 (0.0009) [2023-12-27 12:28:48,696][20381] Updated weights for policy 1, policy_version 985029 (0.0011) [2023-12-27 12:28:48,719][20347] Updated weights for policy 0, policy_version 985780 (0.0006) [2023-12-27 12:28:48,753][20381] Updated weights for policy 1, policy_version 985039 (0.0011) [2023-12-27 12:28:49,387][20347] Updated weights for policy 0, policy_version 985790 (0.0008) [2023-12-27 12:28:49,448][20347] Updated weights for policy 0, policy_version 985800 (0.0007) [2023-12-27 12:28:49,470][20381] Updated weights for policy 1, policy_version 985049 (0.0011) [2023-12-27 12:28:49,516][20347] Updated weights for policy 0, policy_version 985810 (0.0007) [2023-12-27 12:28:49,523][20381] Updated weights for policy 1, policy_version 985059 (0.0011) [2023-12-27 12:28:49,576][20381] Updated weights for policy 1, policy_version 985069 (0.0010) [2023-12-27 12:28:49,626][20381] Updated weights for policy 1, policy_version 985079 (0.0011) [2023-12-27 12:28:50,235][20347] Updated weights for policy 0, policy_version 985820 (0.0007) [2023-12-27 12:28:50,305][20347] Updated weights for policy 0, policy_version 985830 (0.0010) [2023-12-27 12:28:50,367][20381] Updated weights for policy 1, policy_version 985089 (0.0011) [2023-12-27 12:28:50,367][20347] Updated weights for policy 0, policy_version 985840 (0.0010) [2023-12-27 12:28:50,421][20381] Updated weights for policy 1, policy_version 985099 (0.0010) [2023-12-27 12:28:50,472][20381] Updated weights for policy 1, policy_version 985109 (0.0010) [2023-12-27 12:28:50,980][20347] Updated weights for policy 0, policy_version 985850 (0.0009) [2023-12-27 12:28:51,043][20347] Updated weights for policy 0, policy_version 985860 (0.0007) [2023-12-27 12:28:51,104][20347] Updated weights for policy 0, policy_version 985870 (0.0008) [2023-12-27 12:28:51,162][20381] Updated weights for policy 1, policy_version 985119 (0.0009) [2023-12-27 12:28:51,170][20347] Updated weights for policy 0, policy_version 985880 (0.0007) [2023-12-27 12:28:51,220][20381] Updated weights for policy 1, policy_version 985129 (0.0007) [2023-12-27 12:28:51,275][20381] Updated weights for policy 1, policy_version 985139 (0.0007) [2023-12-27 12:28:51,837][20347] Updated weights for policy 0, policy_version 985890 (0.0008) [2023-12-27 12:28:51,896][20347] Updated weights for policy 0, policy_version 985900 (0.0006) [2023-12-27 12:28:51,955][20347] Updated weights for policy 0, policy_version 985910 (0.0008) [2023-12-27 12:28:52,035][20381] Updated weights for policy 1, policy_version 985149 (0.0007) [2023-12-27 12:28:52,093][20381] Updated weights for policy 1, policy_version 985159 (0.0008) [2023-12-27 12:28:52,142][20381] Updated weights for policy 1, policy_version 985169 (0.0009) [2023-12-27 12:28:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20299.5). Total num frames: 504668160. Throughput: 0: 10054.8, 1: 10139.1. Samples: 504636412. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:52,363][19170] Avg episode reward: [(0, '864.720'), (1, '932.260')] [2023-12-27 12:28:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000985176_252231680.pth... [2023-12-27 12:28:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000985912_252436480.pth... [2023-12-27 12:28:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000983992_251928576.pth [2023-12-27 12:28:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000984760_252141568.pth [2023-12-27 12:28:52,588][20347] Updated weights for policy 0, policy_version 985920 (0.0008) [2023-12-27 12:28:52,642][20347] Updated weights for policy 0, policy_version 985930 (0.0007) [2023-12-27 12:28:52,710][20347] Updated weights for policy 0, policy_version 985940 (0.0011) [2023-12-27 12:28:52,844][20381] Updated weights for policy 1, policy_version 985179 (0.0007) [2023-12-27 12:28:52,903][20381] Updated weights for policy 1, policy_version 985189 (0.0005) [2023-12-27 12:28:52,962][20381] Updated weights for policy 1, policy_version 985199 (0.0005) [2023-12-27 12:28:53,347][20347] Updated weights for policy 0, policy_version 985950 (0.0010) [2023-12-27 12:28:53,405][20347] Updated weights for policy 0, policy_version 985960 (0.0010) [2023-12-27 12:28:53,462][20347] Updated weights for policy 0, policy_version 985970 (0.0010) [2023-12-27 12:28:53,526][20381] Updated weights for policy 1, policy_version 985209 (0.0006) [2023-12-27 12:28:53,580][20381] Updated weights for policy 1, policy_version 985219 (0.0005) [2023-12-27 12:28:53,642][20381] Updated weights for policy 1, policy_version 985229 (0.0005) [2023-12-27 12:28:53,694][20381] Updated weights for policy 1, policy_version 985239 (0.0006) [2023-12-27 12:28:54,018][20347] Updated weights for policy 0, policy_version 985980 (0.0007) [2023-12-27 12:28:54,066][20347] Updated weights for policy 0, policy_version 985990 (0.0010) [2023-12-27 12:28:54,125][20347] Updated weights for policy 0, policy_version 986000 (0.0010) [2023-12-27 12:28:54,216][20381] Updated weights for policy 1, policy_version 985249 (0.0005) [2023-12-27 12:28:54,271][20381] Updated weights for policy 1, policy_version 985259 (0.0007) [2023-12-27 12:28:54,328][20381] Updated weights for policy 1, policy_version 985269 (0.0005) [2023-12-27 12:28:54,825][20347] Updated weights for policy 0, policy_version 986010 (0.0009) [2023-12-27 12:28:54,859][20381] Updated weights for policy 1, policy_version 985279 (0.0008) [2023-12-27 12:28:54,872][20347] Updated weights for policy 0, policy_version 986020 (0.0005) [2023-12-27 12:28:54,907][20381] Updated weights for policy 1, policy_version 985289 (0.0010) [2023-12-27 12:28:54,916][20347] Updated weights for policy 0, policy_version 986030 (0.0005) [2023-12-27 12:28:54,948][20381] Updated weights for policy 1, policy_version 985299 (0.0010) [2023-12-27 12:28:54,963][20347] Updated weights for policy 0, policy_version 986040 (0.0006) [2023-12-27 12:28:55,600][20381] Updated weights for policy 1, policy_version 985309 (0.0007) [2023-12-27 12:28:55,655][20381] Updated weights for policy 1, policy_version 985319 (0.0008) [2023-12-27 12:28:55,661][20347] Updated weights for policy 0, policy_version 986050 (0.0007) [2023-12-27 12:28:55,703][20381] Updated weights for policy 1, policy_version 985329 (0.0010) [2023-12-27 12:28:55,718][20347] Updated weights for policy 0, policy_version 986060 (0.0007) [2023-12-27 12:28:55,767][20347] Updated weights for policy 0, policy_version 986070 (0.0006) [2023-12-27 12:28:56,329][20381] Updated weights for policy 1, policy_version 985339 (0.0009) [2023-12-27 12:28:56,375][20381] Updated weights for policy 1, policy_version 985349 (0.0005) [2023-12-27 12:28:56,415][20347] Updated weights for policy 0, policy_version 986080 (0.0007) [2023-12-27 12:28:56,421][20381] Updated weights for policy 1, policy_version 985359 (0.0005) [2023-12-27 12:28:56,452][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 12:28:56,934][20381] Updated weights for policy 1, policy_version 985369 (0.0005) [2023-12-27 12:28:56,988][20381] Updated weights for policy 1, policy_version 985379 (0.0005) [2023-12-27 12:28:57,043][20381] Updated weights for policy 1, policy_version 985389 (0.0005) [2023-12-27 12:28:57,097][20381] Updated weights for policy 1, policy_version 985399 (0.0005) [2023-12-27 12:28:57,226][20347] Updated weights for policy 0, policy_version 986090 (0.0007) [2023-12-27 12:28:57,270][20347] Updated weights for policy 0, policy_version 986100 (0.0007) [2023-12-27 12:28:57,321][20347] Updated weights for policy 0, policy_version 986110 (0.0008) [2023-12-27 12:28:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20355.0). Total num frames: 504774656. Throughput: 0: 10107.7, 1: 10254.9. Samples: 504764844. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:28:57,363][19170] Avg episode reward: [(0, '912.410'), (1, '951.770')] [2023-12-27 12:28:57,374][20347] Updated weights for policy 0, policy_version 986120 (0.0008) [2023-12-27 12:28:57,609][20381] Updated weights for policy 1, policy_version 985409 (0.0005) [2023-12-27 12:28:57,655][20381] Updated weights for policy 1, policy_version 985419 (0.0005) [2023-12-27 12:28:57,702][20381] Updated weights for policy 1, policy_version 985429 (0.0006) [2023-12-27 12:28:58,109][20347] Updated weights for policy 0, policy_version 986130 (0.0005) [2023-12-27 12:28:58,163][20347] Updated weights for policy 0, policy_version 986140 (0.0007) [2023-12-27 12:28:58,228][20347] Updated weights for policy 0, policy_version 986150 (0.0008) [2023-12-27 12:28:58,408][20381] Updated weights for policy 1, policy_version 985439 (0.0010) [2023-12-27 12:28:58,469][20381] Updated weights for policy 1, policy_version 985449 (0.0009) [2023-12-27 12:28:58,526][20381] Updated weights for policy 1, policy_version 985459 (0.0008) [2023-12-27 12:28:58,979][20347] Updated weights for policy 0, policy_version 986160 (0.0008) [2023-12-27 12:28:59,039][20347] Updated weights for policy 0, policy_version 986170 (0.0006) [2023-12-27 12:28:59,099][20347] Updated weights for policy 0, policy_version 986180 (0.0005) [2023-12-27 12:28:59,260][20381] Updated weights for policy 1, policy_version 985469 (0.0007) [2023-12-27 12:28:59,318][20381] Updated weights for policy 1, policy_version 985479 (0.0006) [2023-12-27 12:28:59,384][20381] Updated weights for policy 1, policy_version 985489 (0.0007) [2023-12-27 12:28:59,742][20347] Updated weights for policy 0, policy_version 986190 (0.0006) [2023-12-27 12:28:59,790][20347] Updated weights for policy 0, policy_version 986200 (0.0005) [2023-12-27 12:28:59,853][20347] Updated weights for policy 0, policy_version 986210 (0.0008) [2023-12-27 12:29:00,134][20381] Updated weights for policy 1, policy_version 985499 (0.0006) [2023-12-27 12:29:00,181][20381] Updated weights for policy 1, policy_version 985509 (0.0008) [2023-12-27 12:29:00,232][20381] Updated weights for policy 1, policy_version 985519 (0.0008) [2023-12-27 12:29:00,533][20347] Updated weights for policy 0, policy_version 986220 (0.0006) [2023-12-27 12:29:00,585][20347] Updated weights for policy 0, policy_version 986230 (0.0008) [2023-12-27 12:29:00,638][20347] Updated weights for policy 0, policy_version 986240 (0.0007) [2023-12-27 12:29:00,907][20381] Updated weights for policy 1, policy_version 985529 (0.0008) [2023-12-27 12:29:00,957][20381] Updated weights for policy 1, policy_version 985539 (0.0005) [2023-12-27 12:29:01,018][20381] Updated weights for policy 1, policy_version 985549 (0.0008) [2023-12-27 12:29:01,082][20381] Updated weights for policy 1, policy_version 985559 (0.0009) [2023-12-27 12:29:01,387][20347] Updated weights for policy 0, policy_version 986250 (0.0007) [2023-12-27 12:29:01,441][20347] Updated weights for policy 0, policy_version 986260 (0.0008) [2023-12-27 12:29:01,493][20347] Updated weights for policy 0, policy_version 986270 (0.0008) [2023-12-27 12:29:01,541][20347] Updated weights for policy 0, policy_version 986280 (0.0008) [2023-12-27 12:29:01,695][20381] Updated weights for policy 1, policy_version 985569 (0.0009) [2023-12-27 12:29:01,758][20381] Updated weights for policy 1, policy_version 985579 (0.0009) [2023-12-27 12:29:01,815][20381] Updated weights for policy 1, policy_version 985589 (0.0008) [2023-12-27 12:29:02,252][20347] Updated weights for policy 0, policy_version 986290 (0.0006) [2023-12-27 12:29:02,314][20347] Updated weights for policy 0, policy_version 986300 (0.0008) [2023-12-27 12:29:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20382.8). Total num frames: 504872960. Throughput: 0: 10026.0, 1: 10304.4. Samples: 504888160. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:29:02,363][19170] Avg episode reward: [(0, '941.600'), (1, '971.230')] [2023-12-27 12:29:02,381][20347] Updated weights for policy 0, policy_version 986310 (0.0008) [2023-12-27 12:29:02,491][20381] Updated weights for policy 1, policy_version 985599 (0.0008) [2023-12-27 12:29:02,551][20381] Updated weights for policy 1, policy_version 985609 (0.0008) [2023-12-27 12:29:02,607][20381] Updated weights for policy 1, policy_version 985619 (0.0006) [2023-12-27 12:29:03,039][20347] Updated weights for policy 0, policy_version 986320 (0.0007) [2023-12-27 12:29:03,097][20347] Updated weights for policy 0, policy_version 986330 (0.0007) [2023-12-27 12:29:03,151][20347] Updated weights for policy 0, policy_version 986340 (0.0008) [2023-12-27 12:29:03,247][20381] Updated weights for policy 1, policy_version 985629 (0.0005) [2023-12-27 12:29:03,302][20381] Updated weights for policy 1, policy_version 985639 (0.0005) [2023-12-27 12:29:03,356][20381] Updated weights for policy 1, policy_version 985649 (0.0005) [2023-12-27 12:29:03,863][20347] Updated weights for policy 0, policy_version 986350 (0.0007) [2023-12-27 12:29:03,923][20347] Updated weights for policy 0, policy_version 986360 (0.0007) [2023-12-27 12:29:03,981][20347] Updated weights for policy 0, policy_version 986370 (0.0007) [2023-12-27 12:29:04,015][20381] Updated weights for policy 1, policy_version 985659 (0.0007) [2023-12-27 12:29:04,077][20381] Updated weights for policy 1, policy_version 985669 (0.0009) [2023-12-27 12:29:04,136][20381] Updated weights for policy 1, policy_version 985679 (0.0010) [2023-12-27 12:29:04,146][20292] KL-divergence is very high: 293.7399 [2023-12-27 12:29:04,660][20347] Updated weights for policy 0, policy_version 986380 (0.0007) [2023-12-27 12:29:04,723][20347] Updated weights for policy 0, policy_version 986390 (0.0008) [2023-12-27 12:29:04,784][20347] Updated weights for policy 0, policy_version 986400 (0.0010) [2023-12-27 12:29:04,815][20381] Updated weights for policy 1, policy_version 985689 (0.0010) [2023-12-27 12:29:04,870][20381] Updated weights for policy 1, policy_version 985699 (0.0005) [2023-12-27 12:29:04,924][20381] Updated weights for policy 1, policy_version 985709 (0.0008) [2023-12-27 12:29:04,982][20381] Updated weights for policy 1, policy_version 985719 (0.0010) [2023-12-27 12:29:05,379][20347] Updated weights for policy 0, policy_version 986410 (0.0009) [2023-12-27 12:29:05,433][20347] Updated weights for policy 0, policy_version 986420 (0.0005) [2023-12-27 12:29:05,485][20347] Updated weights for policy 0, policy_version 986430 (0.0008) [2023-12-27 12:29:05,540][20347] Updated weights for policy 0, policy_version 986440 (0.0006) [2023-12-27 12:29:05,561][20381] Updated weights for policy 1, policy_version 985729 (0.0008) [2023-12-27 12:29:05,615][20381] Updated weights for policy 1, policy_version 985739 (0.0008) [2023-12-27 12:29:05,669][20381] Updated weights for policy 1, policy_version 985749 (0.0007) [2023-12-27 12:29:06,180][20347] Updated weights for policy 0, policy_version 986450 (0.0008) [2023-12-27 12:29:06,244][20347] Updated weights for policy 0, policy_version 986460 (0.0007) [2023-12-27 12:29:06,303][20347] Updated weights for policy 0, policy_version 986470 (0.0008) [2023-12-27 12:29:06,397][20381] Updated weights for policy 1, policy_version 985759 (0.0006) [2023-12-27 12:29:06,463][20381] Updated weights for policy 1, policy_version 985769 (0.0010) [2023-12-27 12:29:06,524][20381] Updated weights for policy 1, policy_version 985779 (0.0010) [2023-12-27 12:29:06,986][20347] Updated weights for policy 0, policy_version 986480 (0.0010) [2023-12-27 12:29:07,038][20347] Updated weights for policy 0, policy_version 986490 (0.0010) [2023-12-27 12:29:07,091][20347] Updated weights for policy 0, policy_version 986500 (0.0010) [2023-12-27 12:29:07,127][20381] Updated weights for policy 1, policy_version 985789 (0.0009) [2023-12-27 12:29:07,172][20381] Updated weights for policy 1, policy_version 985799 (0.0008) [2023-12-27 12:29:07,228][20381] Updated weights for policy 1, policy_version 985809 (0.0006) [2023-12-27 12:29:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 504987648. Throughput: 0: 10028.2, 1: 10351.7. Samples: 504950904. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:29:07,363][19170] Avg episode reward: [(0, '931.550'), (1, '951.790')] [2023-12-27 12:29:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000985816_252395520.pth... [2023-12-27 12:29:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000986504_252592128.pth... [2023-12-27 12:29:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000985336_252289024.pth [2023-12-27 12:29:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000984600_252084224.pth [2023-12-27 12:29:07,775][20347] Updated weights for policy 0, policy_version 986510 (0.0009) [2023-12-27 12:29:07,784][20381] Updated weights for policy 1, policy_version 985819 (0.0006) [2023-12-27 12:29:07,830][20347] Updated weights for policy 0, policy_version 986520 (0.0010) [2023-12-27 12:29:07,836][20381] Updated weights for policy 1, policy_version 985829 (0.0005) [2023-12-27 12:29:07,882][20381] Updated weights for policy 1, policy_version 985839 (0.0006) [2023-12-27 12:29:07,885][20347] Updated weights for policy 0, policy_version 986530 (0.0007) [2023-12-27 12:29:08,473][20347] Updated weights for policy 0, policy_version 986540 (0.0005) [2023-12-27 12:29:08,536][20347] Updated weights for policy 0, policy_version 986550 (0.0005) [2023-12-27 12:29:08,598][20347] Updated weights for policy 0, policy_version 986560 (0.0008) [2023-12-27 12:29:08,609][20381] Updated weights for policy 1, policy_version 985849 (0.0008) [2023-12-27 12:29:08,670][20381] Updated weights for policy 1, policy_version 985859 (0.0009) [2023-12-27 12:29:08,732][20381] Updated weights for policy 1, policy_version 985869 (0.0008) [2023-12-27 12:29:08,807][20381] Updated weights for policy 1, policy_version 985879 (0.0008) [2023-12-27 12:29:09,280][20347] Updated weights for policy 0, policy_version 986570 (0.0005) [2023-12-27 12:29:09,349][20347] Updated weights for policy 0, policy_version 986580 (0.0008) [2023-12-27 12:29:09,417][20347] Updated weights for policy 0, policy_version 986590 (0.0008) [2023-12-27 12:29:09,476][20347] Updated weights for policy 0, policy_version 986600 (0.0006) [2023-12-27 12:29:09,505][20381] Updated weights for policy 1, policy_version 985889 (0.0005) [2023-12-27 12:29:09,570][20381] Updated weights for policy 1, policy_version 985899 (0.0006) [2023-12-27 12:29:09,622][20381] Updated weights for policy 1, policy_version 985909 (0.0009) [2023-12-27 12:29:10,148][20347] Updated weights for policy 0, policy_version 986610 (0.0007) [2023-12-27 12:29:10,208][20347] Updated weights for policy 0, policy_version 986620 (0.0008) [2023-12-27 12:29:10,270][20347] Updated weights for policy 0, policy_version 986630 (0.0008) [2023-12-27 12:29:10,350][20381] Updated weights for policy 1, policy_version 985919 (0.0008) [2023-12-27 12:29:10,404][20381] Updated weights for policy 1, policy_version 985929 (0.0008) [2023-12-27 12:29:10,456][20381] Updated weights for policy 1, policy_version 985939 (0.0009) [2023-12-27 12:29:10,983][20347] Updated weights for policy 0, policy_version 986640 (0.0006) [2023-12-27 12:29:11,048][20347] Updated weights for policy 0, policy_version 986650 (0.0007) [2023-12-27 12:29:11,107][20347] Updated weights for policy 0, policy_version 986660 (0.0006) [2023-12-27 12:29:11,111][20381] Updated weights for policy 1, policy_version 985949 (0.0009) [2023-12-27 12:29:11,177][20381] Updated weights for policy 1, policy_version 985959 (0.0008) [2023-12-27 12:29:11,248][20381] Updated weights for policy 1, policy_version 985969 (0.0009) [2023-12-27 12:29:11,820][20347] Updated weights for policy 0, policy_version 986670 (0.0007) [2023-12-27 12:29:11,886][20347] Updated weights for policy 0, policy_version 986680 (0.0008) [2023-12-27 12:29:11,955][20347] Updated weights for policy 0, policy_version 986690 (0.0008) [2023-12-27 12:29:12,042][20381] Updated weights for policy 1, policy_version 985979 (0.0010) [2023-12-27 12:29:12,106][20381] Updated weights for policy 1, policy_version 985989 (0.0008) [2023-12-27 12:29:12,168][20381] Updated weights for policy 1, policy_version 985999 (0.0009) [2023-12-27 12:29:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 505085952. Throughput: 0: 10107.4, 1: 10393.4. Samples: 505075580. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:29:12,363][19170] Avg episode reward: [(0, '833.820'), (1, '971.320')] [2023-12-27 12:29:12,624][20347] Updated weights for policy 0, policy_version 986700 (0.0007) [2023-12-27 12:29:12,685][20347] Updated weights for policy 0, policy_version 986710 (0.0006) [2023-12-27 12:29:12,739][20347] Updated weights for policy 0, policy_version 986720 (0.0005) [2023-12-27 12:29:12,865][20381] Updated weights for policy 1, policy_version 986009 (0.0008) [2023-12-27 12:29:12,922][20381] Updated weights for policy 1, policy_version 986019 (0.0006) [2023-12-27 12:29:12,982][20381] Updated weights for policy 1, policy_version 986029 (0.0006) [2023-12-27 12:29:13,035][20381] Updated weights for policy 1, policy_version 986039 (0.0011) [2023-12-27 12:29:13,370][20347] Updated weights for policy 0, policy_version 986730 (0.0006) [2023-12-27 12:29:13,422][20347] Updated weights for policy 0, policy_version 986740 (0.0006) [2023-12-27 12:29:13,481][20347] Updated weights for policy 0, policy_version 986750 (0.0006) [2023-12-27 12:29:13,537][20347] Updated weights for policy 0, policy_version 986760 (0.0005) [2023-12-27 12:29:13,674][20381] Updated weights for policy 1, policy_version 986049 (0.0006) [2023-12-27 12:29:13,729][20381] Updated weights for policy 1, policy_version 986059 (0.0006) [2023-12-27 12:29:13,783][20381] Updated weights for policy 1, policy_version 986069 (0.0006) [2023-12-27 12:29:14,143][20347] Updated weights for policy 0, policy_version 986770 (0.0006) [2023-12-27 12:29:14,206][20347] Updated weights for policy 0, policy_version 986780 (0.0006) [2023-12-27 12:29:14,236][20291] KL-divergence is very high: 232.2028 [2023-12-27 12:29:14,263][20347] Updated weights for policy 0, policy_version 986790 (0.0006) [2023-12-27 12:29:14,343][20381] Updated weights for policy 1, policy_version 986079 (0.0009) [2023-12-27 12:29:14,396][20381] Updated weights for policy 1, policy_version 986089 (0.0011) [2023-12-27 12:29:14,449][20381] Updated weights for policy 1, policy_version 986099 (0.0010) [2023-12-27 12:29:14,824][20347] Updated weights for policy 0, policy_version 986800 (0.0007) [2023-12-27 12:29:14,882][20347] Updated weights for policy 0, policy_version 986810 (0.0006) [2023-12-27 12:29:14,947][20347] Updated weights for policy 0, policy_version 986820 (0.0008) [2023-12-27 12:29:15,174][20381] Updated weights for policy 1, policy_version 986109 (0.0010) [2023-12-27 12:29:15,236][20381] Updated weights for policy 1, policy_version 986119 (0.0008) [2023-12-27 12:29:15,292][20381] Updated weights for policy 1, policy_version 986129 (0.0008) [2023-12-27 12:29:15,626][20347] Updated weights for policy 0, policy_version 986830 (0.0005) [2023-12-27 12:29:15,653][20291] KL-divergence is very high: 167.6156 [2023-12-27 12:29:15,671][20347] Updated weights for policy 0, policy_version 986840 (0.0005) [2023-12-27 12:29:15,719][20347] Updated weights for policy 0, policy_version 986850 (0.0007) [2023-12-27 12:29:15,898][20381] Updated weights for policy 1, policy_version 986139 (0.0006) [2023-12-27 12:29:15,944][20381] Updated weights for policy 1, policy_version 986149 (0.0005) [2023-12-27 12:29:15,989][20381] Updated weights for policy 1, policy_version 986159 (0.0005) [2023-12-27 12:29:16,464][20347] Updated weights for policy 0, policy_version 986860 (0.0008) [2023-12-27 12:29:16,511][20347] Updated weights for policy 0, policy_version 986870 (0.0007) [2023-12-27 12:29:16,540][20381] Updated weights for policy 1, policy_version 986169 (0.0006) [2023-12-27 12:29:16,560][20347] Updated weights for policy 0, policy_version 986880 (0.0006) [2023-12-27 12:29:16,591][20381] Updated weights for policy 1, policy_version 986179 (0.0010) [2023-12-27 12:29:16,640][20381] Updated weights for policy 1, policy_version 986189 (0.0010) [2023-12-27 12:29:16,697][20381] Updated weights for policy 1, policy_version 986199 (0.0010) [2023-12-27 12:29:17,247][20347] Updated weights for policy 0, policy_version 986890 (0.0006) [2023-12-27 12:29:17,297][20347] Updated weights for policy 0, policy_version 986900 (0.0007) [2023-12-27 12:29:17,352][20381] Updated weights for policy 1, policy_version 986209 (0.0010) [2023-12-27 12:29:17,356][20347] Updated weights for policy 0, policy_version 986910 (0.0009) [2023-12-27 12:29:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 505184256. Throughput: 0: 10170.8, 1: 10423.7. Samples: 505201500. Policy #0 lag: (min: 31.0, avg: 42.8, max: 63.0) [2023-12-27 12:29:17,363][19170] Avg episode reward: [(0, '824.120'), (1, '990.760')] [2023-12-27 12:29:17,403][20381] Updated weights for policy 1, policy_version 986219 (0.0010) [2023-12-27 12:29:17,414][20347] Updated weights for policy 0, policy_version 986920 (0.0009) [2023-12-27 12:29:17,459][20381] Updated weights for policy 1, policy_version 986229 (0.0007) [2023-12-27 12:29:18,114][20381] Updated weights for policy 1, policy_version 986239 (0.0007) [2023-12-27 12:29:18,137][20347] Updated weights for policy 0, policy_version 986930 (0.0006) [2023-12-27 12:29:18,165][20381] Updated weights for policy 1, policy_version 986249 (0.0006) [2023-12-27 12:29:18,193][20347] Updated weights for policy 0, policy_version 986940 (0.0007) [2023-12-27 12:29:18,211][20381] Updated weights for policy 1, policy_version 986259 (0.0005) [2023-12-27 12:29:18,250][20347] Updated weights for policy 0, policy_version 986950 (0.0006) [2023-12-27 12:29:18,890][20381] Updated weights for policy 1, policy_version 986269 (0.0006) [2023-12-27 12:29:18,943][20381] Updated weights for policy 1, policy_version 986279 (0.0008) [2023-12-27 12:29:18,968][20347] Updated weights for policy 0, policy_version 986960 (0.0008) [2023-12-27 12:29:18,996][20381] Updated weights for policy 1, policy_version 986289 (0.0008) [2023-12-27 12:29:19,024][20347] Updated weights for policy 0, policy_version 986970 (0.0005) [2023-12-27 12:29:19,081][20347] Updated weights for policy 0, policy_version 986980 (0.0006) [2023-12-27 12:29:19,680][20381] Updated weights for policy 1, policy_version 986299 (0.0006) [2023-12-27 12:29:19,742][20381] Updated weights for policy 1, policy_version 986309 (0.0008) [2023-12-27 12:29:19,797][20381] Updated weights for policy 1, policy_version 986319 (0.0007) [2023-12-27 12:29:19,801][20347] Updated weights for policy 0, policy_version 986990 (0.0006) [2023-12-27 12:29:19,867][20347] Updated weights for policy 0, policy_version 987000 (0.0008) [2023-12-27 12:29:19,940][20347] Updated weights for policy 0, policy_version 987010 (0.0007) [2023-12-27 12:29:20,565][20381] Updated weights for policy 1, policy_version 986329 (0.0007) [2023-12-27 12:29:20,631][20381] Updated weights for policy 1, policy_version 986339 (0.0007) [2023-12-27 12:29:20,633][20347] Updated weights for policy 0, policy_version 987020 (0.0007) [2023-12-27 12:29:20,688][20381] Updated weights for policy 1, policy_version 986349 (0.0006) [2023-12-27 12:29:20,690][20347] Updated weights for policy 0, policy_version 987030 (0.0005) [2023-12-27 12:29:20,751][20347] Updated weights for policy 0, policy_version 987040 (0.0006) [2023-12-27 12:29:20,752][20381] Updated weights for policy 1, policy_version 986359 (0.0007) [2023-12-27 12:29:21,506][20381] Updated weights for policy 1, policy_version 986369 (0.0007) [2023-12-27 12:29:21,531][20347] Updated weights for policy 0, policy_version 987050 (0.0008) [2023-12-27 12:29:21,557][20381] Updated weights for policy 1, policy_version 986379 (0.0007) [2023-12-27 12:29:21,592][20347] Updated weights for policy 0, policy_version 987060 (0.0006) [2023-12-27 12:29:21,616][20381] Updated weights for policy 1, policy_version 986389 (0.0006) [2023-12-27 12:29:21,657][20347] Updated weights for policy 0, policy_version 987070 (0.0007) [2023-12-27 12:29:21,730][20347] Updated weights for policy 0, policy_version 987080 (0.0008) [2023-12-27 12:29:22,339][20381] Updated weights for policy 1, policy_version 986399 (0.0010) [2023-12-27 12:29:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 505282560. Throughput: 0: 10152.6, 1: 10395.9. Samples: 505261020. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:22,363][19170] Avg episode reward: [(0, '843.920'), (1, '990.260')] [2023-12-27 12:29:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000987080_252739584.pth... [2023-12-27 12:29:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000985912_252436480.pth [2023-12-27 12:29:22,407][20381] Updated weights for policy 1, policy_version 986409 (0.0010) [2023-12-27 12:29:22,467][20381] Updated weights for policy 1, policy_version 986419 (0.0010) [2023-12-27 12:29:22,491][20347] Updated weights for policy 0, policy_version 987090 (0.0006) [2023-12-27 12:29:22,492][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000986424_252551168.pth... [2023-12-27 12:29:22,497][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000985176_252231680.pth [2023-12-27 12:29:22,553][20347] Updated weights for policy 0, policy_version 987100 (0.0007) [2023-12-27 12:29:22,617][20347] Updated weights for policy 0, policy_version 987110 (0.0007) [2023-12-27 12:29:23,118][20381] Updated weights for policy 1, policy_version 986429 (0.0009) [2023-12-27 12:29:23,179][20381] Updated weights for policy 1, policy_version 986439 (0.0010) [2023-12-27 12:29:23,238][20381] Updated weights for policy 1, policy_version 986449 (0.0010) [2023-12-27 12:29:23,293][20347] Updated weights for policy 0, policy_version 987120 (0.0007) [2023-12-27 12:29:23,348][20347] Updated weights for policy 0, policy_version 987130 (0.0009) [2023-12-27 12:29:23,401][20347] Updated weights for policy 0, policy_version 987140 (0.0010) [2023-12-27 12:29:23,889][20381] Updated weights for policy 1, policy_version 986459 (0.0011) [2023-12-27 12:29:23,950][20381] Updated weights for policy 1, policy_version 986469 (0.0011) [2023-12-27 12:29:24,008][20381] Updated weights for policy 1, policy_version 986479 (0.0011) [2023-12-27 12:29:24,068][20347] Updated weights for policy 0, policy_version 987150 (0.0007) [2023-12-27 12:29:24,120][20347] Updated weights for policy 0, policy_version 987160 (0.0007) [2023-12-27 12:29:24,178][20347] Updated weights for policy 0, policy_version 987170 (0.0007) [2023-12-27 12:29:24,676][20381] Updated weights for policy 1, policy_version 986489 (0.0010) [2023-12-27 12:29:24,740][20381] Updated weights for policy 1, policy_version 986499 (0.0006) [2023-12-27 12:29:24,782][20347] Updated weights for policy 0, policy_version 987180 (0.0006) [2023-12-27 12:29:24,802][20381] Updated weights for policy 1, policy_version 986509 (0.0006) [2023-12-27 12:29:24,829][20347] Updated weights for policy 0, policy_version 987190 (0.0007) [2023-12-27 12:29:24,863][20381] Updated weights for policy 1, policy_version 986519 (0.0006) [2023-12-27 12:29:24,876][20347] Updated weights for policy 0, policy_version 987200 (0.0006) [2023-12-27 12:29:25,478][20381] Updated weights for policy 1, policy_version 986529 (0.0007) [2023-12-27 12:29:25,536][20381] Updated weights for policy 1, policy_version 986539 (0.0008) [2023-12-27 12:29:25,589][20381] Updated weights for policy 1, policy_version 986549 (0.0008) [2023-12-27 12:29:25,591][20347] Updated weights for policy 0, policy_version 987210 (0.0008) [2023-12-27 12:29:25,647][20347] Updated weights for policy 0, policy_version 987220 (0.0007) [2023-12-27 12:29:25,695][20347] Updated weights for policy 0, policy_version 987230 (0.0007) [2023-12-27 12:29:25,739][20347] Updated weights for policy 0, policy_version 987240 (0.0007) [2023-12-27 12:29:26,276][20381] Updated weights for policy 1, policy_version 986559 (0.0007) [2023-12-27 12:29:26,326][20381] Updated weights for policy 1, policy_version 986569 (0.0007) [2023-12-27 12:29:26,375][20381] Updated weights for policy 1, policy_version 986579 (0.0008) [2023-12-27 12:29:26,422][20347] Updated weights for policy 0, policy_version 987250 (0.0006) [2023-12-27 12:29:26,477][20347] Updated weights for policy 0, policy_version 987260 (0.0008) [2023-12-27 12:29:26,527][20347] Updated weights for policy 0, policy_version 987270 (0.0007) [2023-12-27 12:29:27,013][20381] Updated weights for policy 1, policy_version 986589 (0.0008) [2023-12-27 12:29:27,074][20381] Updated weights for policy 1, policy_version 986599 (0.0011) [2023-12-27 12:29:27,130][20381] Updated weights for policy 1, policy_version 986609 (0.0010) [2023-12-27 12:29:27,209][20347] Updated weights for policy 0, policy_version 987280 (0.0007) [2023-12-27 12:29:27,257][20347] Updated weights for policy 0, policy_version 987290 (0.0007) [2023-12-27 12:29:27,311][20347] Updated weights for policy 0, policy_version 987300 (0.0006) [2023-12-27 12:29:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.5). Total num frames: 505397248. Throughput: 0: 10208.4, 1: 10382.4. Samples: 505381940. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:27,363][19170] Avg episode reward: [(0, '853.870'), (1, '970.860')] [2023-12-27 12:29:27,788][20381] Updated weights for policy 1, policy_version 986619 (0.0011) [2023-12-27 12:29:27,816][20292] KL-divergence is very high: 324.7231 [2023-12-27 12:29:27,833][20381] Updated weights for policy 1, policy_version 986629 (0.0010) [2023-12-27 12:29:27,852][20292] KL-divergence is very high: 169.2518 [2023-12-27 12:29:27,881][20381] Updated weights for policy 1, policy_version 986639 (0.0010) [2023-12-27 12:29:27,949][20347] Updated weights for policy 0, policy_version 987310 (0.0007) [2023-12-27 12:29:27,993][20347] Updated weights for policy 0, policy_version 987320 (0.0008) [2023-12-27 12:29:28,044][20347] Updated weights for policy 0, policy_version 987330 (0.0009) [2023-12-27 12:29:28,460][20381] Updated weights for policy 1, policy_version 986649 (0.0006) [2023-12-27 12:29:28,509][20381] Updated weights for policy 1, policy_version 986659 (0.0010) [2023-12-27 12:29:28,564][20381] Updated weights for policy 1, policy_version 986669 (0.0010) [2023-12-27 12:29:28,619][20381] Updated weights for policy 1, policy_version 986679 (0.0010) [2023-12-27 12:29:28,787][20347] Updated weights for policy 0, policy_version 987340 (0.0007) [2023-12-27 12:29:28,835][20347] Updated weights for policy 0, policy_version 987350 (0.0007) [2023-12-27 12:29:28,885][20347] Updated weights for policy 0, policy_version 987360 (0.0007) [2023-12-27 12:29:29,270][20381] Updated weights for policy 1, policy_version 986689 (0.0008) [2023-12-27 12:29:29,325][20381] Updated weights for policy 1, policy_version 986699 (0.0008) [2023-12-27 12:29:29,390][20381] Updated weights for policy 1, policy_version 986709 (0.0008) [2023-12-27 12:29:29,553][20347] Updated weights for policy 0, policy_version 987370 (0.0007) [2023-12-27 12:29:29,610][20347] Updated weights for policy 0, policy_version 987380 (0.0007) [2023-12-27 12:29:29,672][20347] Updated weights for policy 0, policy_version 987390 (0.0006) [2023-12-27 12:29:29,724][20347] Updated weights for policy 0, policy_version 987400 (0.0007) [2023-12-27 12:29:30,125][20381] Updated weights for policy 1, policy_version 986719 (0.0006) [2023-12-27 12:29:30,189][20381] Updated weights for policy 1, policy_version 986729 (0.0008) [2023-12-27 12:29:30,248][20381] Updated weights for policy 1, policy_version 986739 (0.0008) [2023-12-27 12:29:30,385][20347] Updated weights for policy 0, policy_version 987410 (0.0008) [2023-12-27 12:29:30,447][20347] Updated weights for policy 0, policy_version 987420 (0.0008) [2023-12-27 12:29:30,514][20347] Updated weights for policy 0, policy_version 987430 (0.0007) [2023-12-27 12:29:30,890][20381] Updated weights for policy 1, policy_version 986749 (0.0007) [2023-12-27 12:29:30,954][20381] Updated weights for policy 1, policy_version 986759 (0.0008) [2023-12-27 12:29:31,015][20381] Updated weights for policy 1, policy_version 986769 (0.0008) [2023-12-27 12:29:31,207][20347] Updated weights for policy 0, policy_version 987440 (0.0010) [2023-12-27 12:29:31,273][20347] Updated weights for policy 0, policy_version 987450 (0.0011) [2023-12-27 12:29:31,339][20347] Updated weights for policy 0, policy_version 987460 (0.0010) [2023-12-27 12:29:31,773][20381] Updated weights for policy 1, policy_version 986779 (0.0008) [2023-12-27 12:29:31,828][20381] Updated weights for policy 1, policy_version 986789 (0.0008) [2023-12-27 12:29:31,876][20381] Updated weights for policy 1, policy_version 986799 (0.0008) [2023-12-27 12:29:32,032][20347] Updated weights for policy 0, policy_version 987470 (0.0008) [2023-12-27 12:29:32,091][20347] Updated weights for policy 0, policy_version 987480 (0.0006) [2023-12-27 12:29:32,156][20347] Updated weights for policy 0, policy_version 987490 (0.0006) [2023-12-27 12:29:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 505495552. Throughput: 0: 10191.4, 1: 10433.9. Samples: 505505068. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:32,363][19170] Avg episode reward: [(0, '883.040'), (1, '961.120')] [2023-12-27 12:29:32,553][20381] Updated weights for policy 1, policy_version 986809 (0.0008) [2023-12-27 12:29:32,612][20381] Updated weights for policy 1, policy_version 986819 (0.0007) [2023-12-27 12:29:32,672][20381] Updated weights for policy 1, policy_version 986829 (0.0006) [2023-12-27 12:29:32,728][20381] Updated weights for policy 1, policy_version 986839 (0.0008) [2023-12-27 12:29:32,786][20347] Updated weights for policy 0, policy_version 987500 (0.0006) [2023-12-27 12:29:32,844][20347] Updated weights for policy 0, policy_version 987510 (0.0008) [2023-12-27 12:29:32,902][20347] Updated weights for policy 0, policy_version 987520 (0.0008) [2023-12-27 12:29:33,411][20381] Updated weights for policy 1, policy_version 986849 (0.0010) [2023-12-27 12:29:33,456][20381] Updated weights for policy 1, policy_version 986859 (0.0008) [2023-12-27 12:29:33,505][20381] Updated weights for policy 1, policy_version 986869 (0.0009) [2023-12-27 12:29:33,537][20347] Updated weights for policy 0, policy_version 987530 (0.0008) [2023-12-27 12:29:33,585][20347] Updated weights for policy 0, policy_version 987540 (0.0010) [2023-12-27 12:29:33,631][20347] Updated weights for policy 0, policy_version 987550 (0.0010) [2023-12-27 12:29:33,681][20347] Updated weights for policy 0, policy_version 987560 (0.0010) [2023-12-27 12:29:34,247][20381] Updated weights for policy 1, policy_version 986879 (0.0008) [2023-12-27 12:29:34,301][20381] Updated weights for policy 1, policy_version 986889 (0.0007) [2023-12-27 12:29:34,306][20291] KL-divergence is very high: 129.6977 [2023-12-27 12:29:34,308][20347] Updated weights for policy 0, policy_version 987570 (0.0010) [2023-12-27 12:29:34,357][20381] Updated weights for policy 1, policy_version 986899 (0.0005) [2023-12-27 12:29:34,364][20347] Updated weights for policy 0, policy_version 987580 (0.0009) [2023-12-27 12:29:34,421][20347] Updated weights for policy 0, policy_version 987590 (0.0010) [2023-12-27 12:29:35,027][20381] Updated weights for policy 1, policy_version 986909 (0.0007) [2023-12-27 12:29:35,079][20381] Updated weights for policy 1, policy_version 986919 (0.0008) [2023-12-27 12:29:35,109][20347] Updated weights for policy 0, policy_version 987600 (0.0008) [2023-12-27 12:29:35,133][20381] Updated weights for policy 1, policy_version 986929 (0.0005) [2023-12-27 12:29:35,158][20347] Updated weights for policy 0, policy_version 987610 (0.0008) [2023-12-27 12:29:35,201][20347] Updated weights for policy 0, policy_version 987620 (0.0008) [2023-12-27 12:29:35,746][20381] Updated weights for policy 1, policy_version 986939 (0.0006) [2023-12-27 12:29:35,802][20381] Updated weights for policy 1, policy_version 986949 (0.0006) [2023-12-27 12:29:35,857][20381] Updated weights for policy 1, policy_version 986959 (0.0006) [2023-12-27 12:29:35,929][20347] Updated weights for policy 0, policy_version 987630 (0.0006) [2023-12-27 12:29:35,983][20347] Updated weights for policy 0, policy_version 987640 (0.0010) [2023-12-27 12:29:36,041][20347] Updated weights for policy 0, policy_version 987650 (0.0010) [2023-12-27 12:29:36,486][20381] Updated weights for policy 1, policy_version 986969 (0.0006) [2023-12-27 12:29:36,552][20381] Updated weights for policy 1, policy_version 986979 (0.0006) [2023-12-27 12:29:36,619][20381] Updated weights for policy 1, policy_version 986989 (0.0009) [2023-12-27 12:29:36,683][20381] Updated weights for policy 1, policy_version 986999 (0.0011) [2023-12-27 12:29:36,739][20347] Updated weights for policy 0, policy_version 987660 (0.0010) [2023-12-27 12:29:36,788][20347] Updated weights for policy 0, policy_version 987670 (0.0010) [2023-12-27 12:29:36,847][20347] Updated weights for policy 0, policy_version 987680 (0.0011) [2023-12-27 12:29:37,334][20381] Updated weights for policy 1, policy_version 987009 (0.0010) [2023-12-27 12:29:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.1, 300 sec: 20438.3). Total num frames: 505593856. Throughput: 0: 10238.1, 1: 10448.4. Samples: 505567308. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:37,363][19170] Avg episode reward: [(0, '883.090'), (1, '970.840')] [2023-12-27 12:29:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000987688_252895232.pth... [2023-12-27 12:29:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000986504_252592128.pth [2023-12-27 12:29:37,385][20381] Updated weights for policy 1, policy_version 987019 (0.0010) [2023-12-27 12:29:37,435][20381] Updated weights for policy 1, policy_version 987029 (0.0008) [2023-12-27 12:29:37,449][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000987032_252706816.pth... [2023-12-27 12:29:37,452][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000985816_252395520.pth [2023-12-27 12:29:37,529][20347] Updated weights for policy 0, policy_version 987690 (0.0010) [2023-12-27 12:29:37,582][20347] Updated weights for policy 0, policy_version 987700 (0.0008) [2023-12-27 12:29:37,634][20347] Updated weights for policy 0, policy_version 987710 (0.0008) [2023-12-27 12:29:37,688][20347] Updated weights for policy 0, policy_version 987720 (0.0007) [2023-12-27 12:29:38,025][20381] Updated weights for policy 1, policy_version 987039 (0.0008) [2023-12-27 12:29:38,080][20381] Updated weights for policy 1, policy_version 987049 (0.0005) [2023-12-27 12:29:38,137][20381] Updated weights for policy 1, policy_version 987059 (0.0007) [2023-12-27 12:29:38,448][20347] Updated weights for policy 0, policy_version 987730 (0.0008) [2023-12-27 12:29:38,503][20347] Updated weights for policy 0, policy_version 987740 (0.0007) [2023-12-27 12:29:38,562][20347] Updated weights for policy 0, policy_version 987750 (0.0008) [2023-12-27 12:29:38,805][20381] Updated weights for policy 1, policy_version 987069 (0.0009) [2023-12-27 12:29:38,858][20381] Updated weights for policy 1, policy_version 987079 (0.0008) [2023-12-27 12:29:38,911][20381] Updated weights for policy 1, policy_version 987089 (0.0009) [2023-12-27 12:29:39,288][20347] Updated weights for policy 0, policy_version 987760 (0.0008) [2023-12-27 12:29:39,354][20347] Updated weights for policy 0, policy_version 987770 (0.0009) [2023-12-27 12:29:39,422][20347] Updated weights for policy 0, policy_version 987780 (0.0008) [2023-12-27 12:29:39,715][20381] Updated weights for policy 1, policy_version 987100 (0.0009) [2023-12-27 12:29:39,779][20381] Updated weights for policy 1, policy_version 987110 (0.0010) [2023-12-27 12:29:39,846][20381] Updated weights for policy 1, policy_version 987120 (0.0009) [2023-12-27 12:29:40,159][20347] Updated weights for policy 0, policy_version 987790 (0.0008) [2023-12-27 12:29:40,219][20347] Updated weights for policy 0, policy_version 987800 (0.0007) [2023-12-27 12:29:40,277][20347] Updated weights for policy 0, policy_version 987810 (0.0007) [2023-12-27 12:29:40,539][20381] Updated weights for policy 1, policy_version 987130 (0.0007) [2023-12-27 12:29:40,597][20381] Updated weights for policy 1, policy_version 987140 (0.0005) [2023-12-27 12:29:40,654][20381] Updated weights for policy 1, policy_version 987150 (0.0009) [2023-12-27 12:29:40,713][20381] Updated weights for policy 1, policy_version 987160 (0.0011) [2023-12-27 12:29:40,968][20347] Updated weights for policy 0, policy_version 987820 (0.0007) [2023-12-27 12:29:41,019][20347] Updated weights for policy 0, policy_version 987830 (0.0006) [2023-12-27 12:29:41,080][20347] Updated weights for policy 0, policy_version 987840 (0.0008) [2023-12-27 12:29:41,340][20381] Updated weights for policy 1, policy_version 987170 (0.0010) [2023-12-27 12:29:41,342][20292] KL-divergence is very high: 467.8470 [2023-12-27 12:29:41,348][20292] KL-divergence is very high: 405.3406 [2023-12-27 12:29:41,396][20292] KL-divergence is very high: 119.6538 [2023-12-27 12:29:41,408][20381] Updated weights for policy 1, policy_version 987180 (0.0009) [2023-12-27 12:29:41,473][20381] Updated weights for policy 1, policy_version 987190 (0.0008) [2023-12-27 12:29:41,808][20347] Updated weights for policy 0, policy_version 987850 (0.0007) [2023-12-27 12:29:41,869][20347] Updated weights for policy 0, policy_version 987860 (0.0005) [2023-12-27 12:29:41,926][20347] Updated weights for policy 0, policy_version 987870 (0.0005) [2023-12-27 12:29:41,973][20347] Updated weights for policy 0, policy_version 987880 (0.0005) [2023-12-27 12:29:42,207][20381] Updated weights for policy 1, policy_version 987200 (0.0008) [2023-12-27 12:29:42,269][20381] Updated weights for policy 1, policy_version 987210 (0.0008) [2023-12-27 12:29:42,333][20381] Updated weights for policy 1, policy_version 987220 (0.0007) [2023-12-27 12:29:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 505692160. Throughput: 0: 10135.9, 1: 10387.1. Samples: 505688380. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:42,363][19170] Avg episode reward: [(0, '892.740'), (1, '971.280')] [2023-12-27 12:29:42,650][20347] Updated weights for policy 0, policy_version 987890 (0.0008) [2023-12-27 12:29:42,719][20347] Updated weights for policy 0, policy_version 987900 (0.0010) [2023-12-27 12:29:42,778][20347] Updated weights for policy 0, policy_version 987910 (0.0008) [2023-12-27 12:29:43,086][20381] Updated weights for policy 1, policy_version 987230 (0.0008) [2023-12-27 12:29:43,096][20292] KL-divergence is very high: 129.1720 [2023-12-27 12:29:43,133][20381] Updated weights for policy 1, policy_version 987240 (0.0008) [2023-12-27 12:29:43,181][20292] KL-divergence is very high: 147.7755 [2023-12-27 12:29:43,191][20381] Updated weights for policy 1, policy_version 987250 (0.0007) [2023-12-27 12:29:43,219][20292] KL-divergence is very high: 240.2715 [2023-12-27 12:29:43,372][20347] Updated weights for policy 0, policy_version 987920 (0.0010) [2023-12-27 12:29:43,426][20347] Updated weights for policy 0, policy_version 987930 (0.0010) [2023-12-27 12:29:43,480][20347] Updated weights for policy 0, policy_version 987940 (0.0010) [2023-12-27 12:29:43,892][20381] Updated weights for policy 1, policy_version 987260 (0.0007) [2023-12-27 12:29:43,943][20381] Updated weights for policy 1, policy_version 987270 (0.0007) [2023-12-27 12:29:44,001][20381] Updated weights for policy 1, policy_version 987280 (0.0009) [2023-12-27 12:29:44,127][20347] Updated weights for policy 0, policy_version 987950 (0.0010) [2023-12-27 12:29:44,174][20347] Updated weights for policy 0, policy_version 987960 (0.0010) [2023-12-27 12:29:44,232][20347] Updated weights for policy 0, policy_version 987970 (0.0010) [2023-12-27 12:29:44,726][20381] Updated weights for policy 1, policy_version 987290 (0.0007) [2023-12-27 12:29:44,788][20381] Updated weights for policy 1, policy_version 987300 (0.0006) [2023-12-27 12:29:44,788][20292] KL-divergence is very high: 126.0419 [2023-12-27 12:29:44,794][20292] KL-divergence is very high: 201.5674 [2023-12-27 12:29:44,843][20381] Updated weights for policy 1, policy_version 987310 (0.0007) [2023-12-27 12:29:44,896][20381] Updated weights for policy 1, policy_version 987320 (0.0007) [2023-12-27 12:29:44,925][20347] Updated weights for policy 0, policy_version 987980 (0.0010) [2023-12-27 12:29:44,981][20347] Updated weights for policy 0, policy_version 987990 (0.0010) [2023-12-27 12:29:45,041][20347] Updated weights for policy 0, policy_version 988000 (0.0010) [2023-12-27 12:29:45,581][20381] Updated weights for policy 1, policy_version 987330 (0.0005) [2023-12-27 12:29:45,613][20292] KL-divergence is very high: 626.4429 [2023-12-27 12:29:45,627][20292] KL-divergence is very high: 459.7315 [2023-12-27 12:29:45,641][20381] Updated weights for policy 1, policy_version 987340 (0.0007) [2023-12-27 12:29:45,658][20292] KL-divergence is very high: 147.0522 [2023-12-27 12:29:45,669][20292] KL-divergence is very high: 473.6637 [2023-12-27 12:29:45,694][20381] Updated weights for policy 1, policy_version 987350 (0.0007) [2023-12-27 12:29:45,723][20347] Updated weights for policy 0, policy_version 988010 (0.0010) [2023-12-27 12:29:45,778][20347] Updated weights for policy 0, policy_version 988020 (0.0010) [2023-12-27 12:29:45,836][20347] Updated weights for policy 0, policy_version 988030 (0.0010) [2023-12-27 12:29:45,891][20347] Updated weights for policy 0, policy_version 988040 (0.0010) [2023-12-27 12:29:46,368][20381] Updated weights for policy 1, policy_version 987360 (0.0007) [2023-12-27 12:29:46,386][20292] KL-divergence is very high: 171.9129 [2023-12-27 12:29:46,423][20381] Updated weights for policy 1, policy_version 987370 (0.0005) [2023-12-27 12:29:46,431][20292] KL-divergence is very high: 126.5350 [2023-12-27 12:29:46,480][20381] Updated weights for policy 1, policy_version 987380 (0.0006) [2023-12-27 12:29:46,575][20347] Updated weights for policy 0, policy_version 988050 (0.0005) [2023-12-27 12:29:46,636][20347] Updated weights for policy 0, policy_version 988060 (0.0008) [2023-12-27 12:29:46,693][20347] Updated weights for policy 0, policy_version 988070 (0.0006) [2023-12-27 12:29:47,125][20381] Updated weights for policy 1, policy_version 987390 (0.0011) [2023-12-27 12:29:47,176][20381] Updated weights for policy 1, policy_version 987400 (0.0010) [2023-12-27 12:29:47,238][20381] Updated weights for policy 1, policy_version 987410 (0.0007) [2023-12-27 12:29:47,274][20347] Updated weights for policy 0, policy_version 988080 (0.0006) [2023-12-27 12:29:47,331][20347] Updated weights for policy 0, policy_version 988090 (0.0007) [2023-12-27 12:29:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 505798656. Throughput: 0: 10207.6, 1: 10280.4. Samples: 505810124. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:47,363][19170] Avg episode reward: [(0, '960.860'), (1, '854.490')] [2023-12-27 12:29:47,365][20291] KL-divergence is very high: 103.1898 [2023-12-27 12:29:47,390][20347] Updated weights for policy 0, policy_version 988100 (0.0007) [2023-12-27 12:29:47,925][20381] Updated weights for policy 1, policy_version 987420 (0.0010) [2023-12-27 12:29:47,960][20292] KL-divergence is very high: 120.1087 [2023-12-27 12:29:47,981][20381] Updated weights for policy 1, policy_version 987430 (0.0011) [2023-12-27 12:29:48,040][20381] Updated weights for policy 1, policy_version 987440 (0.0011) [2023-12-27 12:29:48,049][20347] Updated weights for policy 0, policy_version 988110 (0.0007) [2023-12-27 12:29:48,103][20347] Updated weights for policy 0, policy_version 988120 (0.0006) [2023-12-27 12:29:48,161][20347] Updated weights for policy 0, policy_version 988130 (0.0008) [2023-12-27 12:29:48,701][20381] Updated weights for policy 1, policy_version 987450 (0.0010) [2023-12-27 12:29:48,752][20381] Updated weights for policy 1, policy_version 987460 (0.0007) [2023-12-27 12:29:48,811][20347] Updated weights for policy 0, policy_version 988140 (0.0007) [2023-12-27 12:29:48,813][20381] Updated weights for policy 1, policy_version 987470 (0.0010) [2023-12-27 12:29:48,865][20381] Updated weights for policy 1, policy_version 987480 (0.0006) [2023-12-27 12:29:48,870][20347] Updated weights for policy 0, policy_version 988150 (0.0007) [2023-12-27 12:29:48,933][20347] Updated weights for policy 0, policy_version 988160 (0.0007) [2023-12-27 12:29:49,540][20381] Updated weights for policy 1, policy_version 987490 (0.0011) [2023-12-27 12:29:49,583][20347] Updated weights for policy 0, policy_version 988170 (0.0006) [2023-12-27 12:29:49,596][20381] Updated weights for policy 1, policy_version 987500 (0.0009) [2023-12-27 12:29:49,633][20347] Updated weights for policy 0, policy_version 988180 (0.0008) [2023-12-27 12:29:49,651][20381] Updated weights for policy 1, policy_version 987510 (0.0010) [2023-12-27 12:29:49,689][20347] Updated weights for policy 0, policy_version 988190 (0.0010) [2023-12-27 12:29:49,737][20347] Updated weights for policy 0, policy_version 988200 (0.0010) [2023-12-27 12:29:50,305][20381] Updated weights for policy 1, policy_version 987520 (0.0007) [2023-12-27 12:29:50,371][20381] Updated weights for policy 1, policy_version 987530 (0.0006) [2023-12-27 12:29:50,432][20381] Updated weights for policy 1, policy_version 987540 (0.0006) [2023-12-27 12:29:50,466][20347] Updated weights for policy 0, policy_version 988210 (0.0010) [2023-12-27 12:29:50,522][20347] Updated weights for policy 0, policy_version 988220 (0.0010) [2023-12-27 12:29:50,589][20347] Updated weights for policy 0, policy_version 988230 (0.0010) [2023-12-27 12:29:51,041][20381] Updated weights for policy 1, policy_version 987550 (0.0007) [2023-12-27 12:29:51,082][20292] KL-divergence is very high: 109.4680 [2023-12-27 12:29:51,088][20292] KL-divergence is very high: 204.2227 [2023-12-27 12:29:51,100][20381] Updated weights for policy 1, policy_version 987560 (0.0008) [2023-12-27 12:29:51,138][20292] KL-divergence is very high: 135.5654 [2023-12-27 12:29:51,166][20381] Updated weights for policy 1, policy_version 987570 (0.0007) [2023-12-27 12:29:51,350][20347] Updated weights for policy 0, policy_version 988240 (0.0008) [2023-12-27 12:29:51,418][20347] Updated weights for policy 0, policy_version 988250 (0.0009) [2023-12-27 12:29:51,478][20347] Updated weights for policy 0, policy_version 988260 (0.0007) [2023-12-27 12:29:51,890][20381] Updated weights for policy 1, policy_version 987580 (0.0007) [2023-12-27 12:29:51,894][20292] KL-divergence is very high: 141.3337 [2023-12-27 12:29:51,941][20381] Updated weights for policy 1, policy_version 987590 (0.0008) [2023-12-27 12:29:51,997][20381] Updated weights for policy 1, policy_version 987600 (0.0008) [2023-12-27 12:29:52,027][20292] KL-divergence is very high: 136.3493 [2023-12-27 12:29:52,195][20347] Updated weights for policy 0, policy_version 988270 (0.0008) [2023-12-27 12:29:52,259][20347] Updated weights for policy 0, policy_version 988280 (0.0008) [2023-12-27 12:29:52,319][20347] Updated weights for policy 0, policy_version 988290 (0.0008) [2023-12-27 12:29:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 505905152. Throughput: 0: 10221.1, 1: 10256.4. Samples: 505872396. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:52,363][19170] Avg episode reward: [(0, '980.420'), (1, '766.930')] [2023-12-27 12:29:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000987608_252854272.pth... [2023-12-27 12:29:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000988296_253050880.pth... [2023-12-27 12:29:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000986424_252551168.pth [2023-12-27 12:29:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000987080_252739584.pth [2023-12-27 12:29:52,691][20381] Updated weights for policy 1, policy_version 987610 (0.0007) [2023-12-27 12:29:52,727][20292] KL-divergence is very high: 118.7711 [2023-12-27 12:29:52,747][20381] Updated weights for policy 1, policy_version 987620 (0.0008) [2023-12-27 12:29:52,796][20381] Updated weights for policy 1, policy_version 987630 (0.0007) [2023-12-27 12:29:52,845][20381] Updated weights for policy 1, policy_version 987640 (0.0007) [2023-12-27 12:29:53,055][20347] Updated weights for policy 0, policy_version 988300 (0.0008) [2023-12-27 12:29:53,108][20347] Updated weights for policy 0, policy_version 988310 (0.0008) [2023-12-27 12:29:53,154][20347] Updated weights for policy 0, policy_version 988320 (0.0008) [2023-12-27 12:29:53,551][20381] Updated weights for policy 1, policy_version 987650 (0.0007) [2023-12-27 12:29:53,584][20292] KL-divergence is very high: 164.4317 [2023-12-27 12:29:53,589][20292] KL-divergence is very high: 141.7317 [2023-12-27 12:29:53,601][20381] Updated weights for policy 1, policy_version 987660 (0.0007) [2023-12-27 12:29:53,653][20381] Updated weights for policy 1, policy_version 987670 (0.0008) [2023-12-27 12:29:53,841][20347] Updated weights for policy 0, policy_version 988330 (0.0007) [2023-12-27 12:29:53,890][20347] Updated weights for policy 0, policy_version 988340 (0.0005) [2023-12-27 12:29:53,945][20347] Updated weights for policy 0, policy_version 988350 (0.0005) [2023-12-27 12:29:53,998][20347] Updated weights for policy 0, policy_version 988360 (0.0005) [2023-12-27 12:29:54,365][20292] KL-divergence is very high: 109.3946 [2023-12-27 12:29:54,419][20381] Updated weights for policy 1, policy_version 987680 (0.0008) [2023-12-27 12:29:54,456][20292] KL-divergence is very high: 170.8469 [2023-12-27 12:29:54,466][20381] Updated weights for policy 1, policy_version 987690 (0.0007) [2023-12-27 12:29:54,526][20381] Updated weights for policy 1, policy_version 987700 (0.0007) [2023-12-27 12:29:54,570][20347] Updated weights for policy 0, policy_version 988370 (0.0010) [2023-12-27 12:29:54,621][20347] Updated weights for policy 0, policy_version 988380 (0.0010) [2023-12-27 12:29:54,666][20347] Updated weights for policy 0, policy_version 988390 (0.0010) [2023-12-27 12:29:55,249][20381] Updated weights for policy 1, policy_version 987710 (0.0007) [2023-12-27 12:29:55,285][20347] Updated weights for policy 0, policy_version 988400 (0.0010) [2023-12-27 12:29:55,305][20381] Updated weights for policy 1, policy_version 987720 (0.0005) [2023-12-27 12:29:55,330][20347] Updated weights for policy 0, policy_version 988410 (0.0010) [2023-12-27 12:29:55,359][20381] Updated weights for policy 1, policy_version 987730 (0.0006) [2023-12-27 12:29:55,380][20347] Updated weights for policy 0, policy_version 988420 (0.0010) [2023-12-27 12:29:56,029][20347] Updated weights for policy 0, policy_version 988430 (0.0007) [2023-12-27 12:29:56,075][20381] Updated weights for policy 1, policy_version 987740 (0.0007) [2023-12-27 12:29:56,086][20347] Updated weights for policy 0, policy_version 988440 (0.0008) [2023-12-27 12:29:56,131][20381] Updated weights for policy 1, policy_version 987750 (0.0006) [2023-12-27 12:29:56,140][20347] Updated weights for policy 0, policy_version 988450 (0.0010) [2023-12-27 12:29:56,189][20381] Updated weights for policy 1, policy_version 987760 (0.0007) [2023-12-27 12:29:56,217][20292] KL-divergence is very high: 100.4727 [2023-12-27 12:29:56,733][20347] Updated weights for policy 0, policy_version 988460 (0.0009) [2023-12-27 12:29:56,778][20347] Updated weights for policy 0, policy_version 988470 (0.0005) [2023-12-27 12:29:56,830][20347] Updated weights for policy 0, policy_version 988480 (0.0005) [2023-12-27 12:29:56,912][20381] Updated weights for policy 1, policy_version 987770 (0.0007) [2023-12-27 12:29:56,968][20381] Updated weights for policy 1, policy_version 987780 (0.0008) [2023-12-27 12:29:57,020][20381] Updated weights for policy 1, policy_version 987790 (0.0010) [2023-12-27 12:29:57,073][20381] Updated weights for policy 1, policy_version 987800 (0.0008) [2023-12-27 12:29:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 506003456. Throughput: 0: 10193.3, 1: 10190.6. Samples: 505992852. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:29:57,363][19170] Avg episode reward: [(0, '970.710'), (1, '662.200')] [2023-12-27 12:29:57,443][20347] Updated weights for policy 0, policy_version 988490 (0.0006) [2023-12-27 12:29:57,493][20347] Updated weights for policy 0, policy_version 988500 (0.0009) [2023-12-27 12:29:57,549][20347] Updated weights for policy 0, policy_version 988510 (0.0007) [2023-12-27 12:29:57,607][20347] Updated weights for policy 0, policy_version 988520 (0.0007) [2023-12-27 12:29:57,730][20381] Updated weights for policy 1, policy_version 987810 (0.0005) [2023-12-27 12:29:57,735][20292] KL-divergence is very high: 146.2961 [2023-12-27 12:29:57,754][20292] KL-divergence is very high: 433.8987 [2023-12-27 12:29:57,773][20292] KL-divergence is very high: 102.1163 [2023-12-27 12:29:57,778][20381] Updated weights for policy 1, policy_version 987820 (0.0006) [2023-12-27 12:29:57,831][20381] Updated weights for policy 1, policy_version 987830 (0.0005) [2023-12-27 12:29:58,354][20347] Updated weights for policy 0, policy_version 988530 (0.0008) [2023-12-27 12:29:58,415][20347] Updated weights for policy 0, policy_version 988540 (0.0008) [2023-12-27 12:29:58,462][20292] KL-divergence is very high: 433.2336 [2023-12-27 12:29:58,474][20292] KL-divergence is very high: 435.6351 [2023-12-27 12:29:58,477][20347] Updated weights for policy 0, policy_version 988550 (0.0008) [2023-12-27 12:29:58,511][20292] KL-divergence is very high: 344.0717 [2023-12-27 12:29:58,512][20381] Updated weights for policy 1, policy_version 987840 (0.0010) [2023-12-27 12:29:58,524][20292] KL-divergence is very high: 209.4505 [2023-12-27 12:29:58,575][20381] Updated weights for policy 1, policy_version 987850 (0.0011) [2023-12-27 12:29:58,576][20292] KL-divergence is very high: 124.1025 [2023-12-27 12:29:58,599][20292] KL-divergence is very high: 103.2986 [2023-12-27 12:29:58,609][20292] KL-divergence is very high: 134.2094 [2023-12-27 12:29:58,623][20292] KL-divergence is very high: 352.3333 [2023-12-27 12:29:58,637][20381] Updated weights for policy 1, policy_version 987860 (0.0011) [2023-12-27 12:29:58,641][20292] KL-divergence is very high: 114.2681 [2023-12-27 12:29:59,222][20347] Updated weights for policy 0, policy_version 988560 (0.0010) [2023-12-27 12:29:59,243][20291] KL-divergence is very high: 131.8442 [2023-12-27 12:29:59,288][20347] Updated weights for policy 0, policy_version 988570 (0.0007) [2023-12-27 12:29:59,341][20347] Updated weights for policy 0, policy_version 988580 (0.0006) [2023-12-27 12:29:59,408][20292] KL-divergence is very high: 445.2854 [2023-12-27 12:29:59,435][20381] Updated weights for policy 1, policy_version 987870 (0.0009) [2023-12-27 12:29:59,447][20292] KL-divergence is very high: 739.3628 [2023-12-27 12:29:59,479][20381] Updated weights for policy 1, policy_version 987880 (0.0008) [2023-12-27 12:29:59,482][20292] KL-divergence is very high: 258.4465 [2023-12-27 12:29:59,521][20292] KL-divergence is very high: 867.9072 [2023-12-27 12:29:59,527][20381] Updated weights for policy 1, policy_version 987890 (0.0008) [2023-12-27 12:29:59,995][20347] Updated weights for policy 0, policy_version 988590 (0.0010) [2023-12-27 12:30:00,014][20291] KL-divergence is very high: 129.4768 [2023-12-27 12:30:00,053][20347] Updated weights for policy 0, policy_version 988600 (0.0010) [2023-12-27 12:30:00,106][20347] Updated weights for policy 0, policy_version 988610 (0.0010) [2023-12-27 12:30:00,239][20292] KL-divergence is very high: 219.9918 [2023-12-27 12:30:00,249][20292] KL-divergence is very high: 215.0672 [2023-12-27 12:30:00,263][20381] Updated weights for policy 1, policy_version 987900 (0.0009) [2023-12-27 12:30:00,294][20292] KL-divergence is very high: 106.6647 [2023-12-27 12:30:00,318][20292] KL-divergence is very high: 120.2859 [2023-12-27 12:30:00,318][20381] Updated weights for policy 1, policy_version 987910 (0.0010) [2023-12-27 12:30:00,336][20292] KL-divergence is very high: 120.3191 [2023-12-27 12:30:00,342][20292] KL-divergence is very high: 100.3318 [2023-12-27 12:30:00,364][20292] KL-divergence is very high: 157.9302 [2023-12-27 12:30:00,373][20381] Updated weights for policy 1, policy_version 987920 (0.0009) [2023-12-27 12:30:00,375][20292] KL-divergence is very high: 105.0400 [2023-12-27 12:30:00,380][20292] KL-divergence is very high: 396.0319 [2023-12-27 12:30:00,385][20292] KL-divergence is very high: 340.5251 [2023-12-27 12:30:00,776][20347] Updated weights for policy 0, policy_version 988620 (0.0010) [2023-12-27 12:30:00,823][20347] Updated weights for policy 0, policy_version 988630 (0.0010) [2023-12-27 12:30:00,873][20347] Updated weights for policy 0, policy_version 988640 (0.0010) [2023-12-27 12:30:00,997][20381] Updated weights for policy 1, policy_version 987930 (0.0005) [2023-12-27 12:30:01,060][20292] KL-divergence is very high: 100.9665 [2023-12-27 12:30:01,065][20381] Updated weights for policy 1, policy_version 987940 (0.0006) [2023-12-27 12:30:01,124][20381] Updated weights for policy 1, policy_version 987950 (0.0008) [2023-12-27 12:30:01,190][20381] Updated weights for policy 1, policy_version 987960 (0.0008) [2023-12-27 12:30:01,565][20347] Updated weights for policy 0, policy_version 988650 (0.0010) [2023-12-27 12:30:01,625][20347] Updated weights for policy 0, policy_version 988660 (0.0010) [2023-12-27 12:30:01,682][20347] Updated weights for policy 0, policy_version 988670 (0.0010) [2023-12-27 12:30:01,743][20347] Updated weights for policy 0, policy_version 988680 (0.0009) [2023-12-27 12:30:01,902][20381] Updated weights for policy 1, policy_version 987970 (0.0008) [2023-12-27 12:30:01,958][20381] Updated weights for policy 1, policy_version 987980 (0.0008) [2023-12-27 12:30:02,024][20381] Updated weights for policy 1, policy_version 987990 (0.0008) [2023-12-27 12:30:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 506101760. Throughput: 0: 10164.8, 1: 10110.2. Samples: 506113876. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:02,363][19170] Avg episode reward: [(0, '941.410'), (1, '291.050')] [2023-12-27 12:30:02,364][20347] Updated weights for policy 0, policy_version 988690 (0.0011) [2023-12-27 12:30:02,419][20347] Updated weights for policy 0, policy_version 988700 (0.0010) [2023-12-27 12:30:02,463][20347] Updated weights for policy 0, policy_version 988710 (0.0007) [2023-12-27 12:30:02,687][20381] Updated weights for policy 1, policy_version 988000 (0.0010) [2023-12-27 12:30:02,742][20381] Updated weights for policy 1, policy_version 988010 (0.0011) [2023-12-27 12:30:02,805][20381] Updated weights for policy 1, policy_version 988020 (0.0011) [2023-12-27 12:30:03,163][20347] Updated weights for policy 0, policy_version 988720 (0.0008) [2023-12-27 12:30:03,220][20347] Updated weights for policy 0, policy_version 988730 (0.0005) [2023-12-27 12:30:03,270][20347] Updated weights for policy 0, policy_version 988740 (0.0009) [2023-12-27 12:30:03,488][20381] Updated weights for policy 1, policy_version 988030 (0.0008) [2023-12-27 12:30:03,539][20381] Updated weights for policy 1, policy_version 988040 (0.0007) [2023-12-27 12:30:03,593][20381] Updated weights for policy 1, policy_version 988050 (0.0007) [2023-12-27 12:30:03,923][20347] Updated weights for policy 0, policy_version 988750 (0.0010) [2023-12-27 12:30:03,975][20347] Updated weights for policy 0, policy_version 988760 (0.0010) [2023-12-27 12:30:04,030][20347] Updated weights for policy 0, policy_version 988770 (0.0010) [2023-12-27 12:30:04,307][20381] Updated weights for policy 1, policy_version 988060 (0.0008) [2023-12-27 12:30:04,367][20381] Updated weights for policy 1, policy_version 988070 (0.0008) [2023-12-27 12:30:04,427][20381] Updated weights for policy 1, policy_version 988080 (0.0008) [2023-12-27 12:30:04,745][20347] Updated weights for policy 0, policy_version 988780 (0.0009) [2023-12-27 12:30:04,805][20347] Updated weights for policy 0, policy_version 988790 (0.0005) [2023-12-27 12:30:04,861][20347] Updated weights for policy 0, policy_version 988800 (0.0005) [2023-12-27 12:30:05,172][20381] Updated weights for policy 1, policy_version 988090 (0.0008) [2023-12-27 12:30:05,224][20381] Updated weights for policy 1, policy_version 988100 (0.0008) [2023-12-27 12:30:05,268][20381] Updated weights for policy 1, policy_version 988110 (0.0008) [2023-12-27 12:30:05,327][20381] Updated weights for policy 1, policy_version 988120 (0.0008) [2023-12-27 12:30:05,425][20347] Updated weights for policy 0, policy_version 988810 (0.0006) [2023-12-27 12:30:05,477][20347] Updated weights for policy 0, policy_version 988820 (0.0007) [2023-12-27 12:30:05,534][20347] Updated weights for policy 0, policy_version 988830 (0.0007) [2023-12-27 12:30:05,588][20347] Updated weights for policy 0, policy_version 988840 (0.0006) [2023-12-27 12:30:05,945][20381] Updated weights for policy 1, policy_version 988130 (0.0006) [2023-12-27 12:30:05,996][20381] Updated weights for policy 1, policy_version 988140 (0.0005) [2023-12-27 12:30:06,041][20381] Updated weights for policy 1, policy_version 988150 (0.0005) [2023-12-27 12:30:06,232][20347] Updated weights for policy 0, policy_version 988850 (0.0008) [2023-12-27 12:30:06,301][20347] Updated weights for policy 0, policy_version 988860 (0.0006) [2023-12-27 12:30:06,362][20347] Updated weights for policy 0, policy_version 988870 (0.0006) [2023-12-27 12:30:06,726][20381] Updated weights for policy 1, policy_version 988160 (0.0007) [2023-12-27 12:30:06,779][20381] Updated weights for policy 1, policy_version 988170 (0.0007) [2023-12-27 12:30:06,833][20381] Updated weights for policy 1, policy_version 988180 (0.0008) [2023-12-27 12:30:06,978][20347] Updated weights for policy 0, policy_version 988880 (0.0009) [2023-12-27 12:30:07,034][20347] Updated weights for policy 0, policy_version 988890 (0.0007) [2023-12-27 12:30:07,089][20347] Updated weights for policy 0, policy_version 988900 (0.0008) [2023-12-27 12:30:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 506208256. Throughput: 0: 10227.0, 1: 10090.3. Samples: 506175296. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:07,363][19170] Avg episode reward: [(0, '951.180'), (1, '43.810')] [2023-12-27 12:30:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000988904_253206528.pth... [2023-12-27 12:30:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000988184_253001728.pth... [2023-12-27 12:30:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000987688_252895232.pth [2023-12-27 12:30:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000987032_252706816.pth [2023-12-27 12:30:07,433][20381] Updated weights for policy 1, policy_version 988190 (0.0006) [2023-12-27 12:30:07,492][20381] Updated weights for policy 1, policy_version 988200 (0.0005) [2023-12-27 12:30:07,555][20381] Updated weights for policy 1, policy_version 988210 (0.0005) [2023-12-27 12:30:07,683][20347] Updated weights for policy 0, policy_version 988910 (0.0006) [2023-12-27 12:30:07,745][20347] Updated weights for policy 0, policy_version 988920 (0.0005) [2023-12-27 12:30:07,810][20347] Updated weights for policy 0, policy_version 988930 (0.0007) [2023-12-27 12:30:08,126][20381] Updated weights for policy 1, policy_version 988220 (0.0006) [2023-12-27 12:30:08,181][20381] Updated weights for policy 1, policy_version 988230 (0.0008) [2023-12-27 12:30:08,242][20381] Updated weights for policy 1, policy_version 988240 (0.0008) [2023-12-27 12:30:08,363][20347] Updated weights for policy 0, policy_version 988940 (0.0007) [2023-12-27 12:30:08,425][20347] Updated weights for policy 0, policy_version 988950 (0.0008) [2023-12-27 12:30:08,486][20347] Updated weights for policy 0, policy_version 988960 (0.0008) [2023-12-27 12:30:08,945][20381] Updated weights for policy 1, policy_version 988250 (0.0008) [2023-12-27 12:30:08,994][20381] Updated weights for policy 1, policy_version 988260 (0.0008) [2023-12-27 12:30:09,056][20381] Updated weights for policy 1, policy_version 988270 (0.0007) [2023-12-27 12:30:09,110][20381] Updated weights for policy 1, policy_version 988280 (0.0008) [2023-12-27 12:30:09,224][20347] Updated weights for policy 0, policy_version 988970 (0.0008) [2023-12-27 12:30:09,285][20347] Updated weights for policy 0, policy_version 988980 (0.0009) [2023-12-27 12:30:09,337][20347] Updated weights for policy 0, policy_version 988990 (0.0008) [2023-12-27 12:30:09,400][20347] Updated weights for policy 0, policy_version 989000 (0.0010) [2023-12-27 12:30:09,935][20381] Updated weights for policy 1, policy_version 988290 (0.0007) [2023-12-27 12:30:10,000][20381] Updated weights for policy 1, policy_version 988300 (0.0007) [2023-12-27 12:30:10,057][20381] Updated weights for policy 1, policy_version 988310 (0.0007) [2023-12-27 12:30:10,165][20347] Updated weights for policy 0, policy_version 989010 (0.0007) [2023-12-27 12:30:10,222][20347] Updated weights for policy 0, policy_version 989020 (0.0007) [2023-12-27 12:30:10,284][20347] Updated weights for policy 0, policy_version 989030 (0.0007) [2023-12-27 12:30:10,750][20381] Updated weights for policy 1, policy_version 988320 (0.0009) [2023-12-27 12:30:10,811][20381] Updated weights for policy 1, policy_version 988330 (0.0006) [2023-12-27 12:30:10,864][20381] Updated weights for policy 1, policy_version 988340 (0.0006) [2023-12-27 12:30:10,953][20347] Updated weights for policy 0, policy_version 989040 (0.0008) [2023-12-27 12:30:11,010][20347] Updated weights for policy 0, policy_version 989050 (0.0008) [2023-12-27 12:30:11,078][20347] Updated weights for policy 0, policy_version 989060 (0.0007) [2023-12-27 12:30:11,595][20381] Updated weights for policy 1, policy_version 988350 (0.0007) [2023-12-27 12:30:11,657][20381] Updated weights for policy 1, policy_version 988360 (0.0008) [2023-12-27 12:30:11,728][20381] Updated weights for policy 1, policy_version 988370 (0.0008) [2023-12-27 12:30:11,822][20347] Updated weights for policy 0, policy_version 989070 (0.0006) [2023-12-27 12:30:11,884][20347] Updated weights for policy 0, policy_version 989080 (0.0006) [2023-12-27 12:30:11,950][20347] Updated weights for policy 0, policy_version 989090 (0.0006) [2023-12-27 12:30:12,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20343.3, 300 sec: 20410.6). Total num frames: 506306560. Throughput: 0: 10282.1, 1: 10129.9. Samples: 506300488. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:12,363][19170] Avg episode reward: [(0, '951.170'), (1, '41.110')] [2023-12-27 12:30:12,418][20381] Updated weights for policy 1, policy_version 988380 (0.0008) [2023-12-27 12:30:12,483][20381] Updated weights for policy 1, policy_version 988390 (0.0006) [2023-12-27 12:30:12,547][20381] Updated weights for policy 1, policy_version 988400 (0.0007) [2023-12-27 12:30:12,652][20347] Updated weights for policy 0, policy_version 989100 (0.0006) [2023-12-27 12:30:12,712][20347] Updated weights for policy 0, policy_version 989110 (0.0008) [2023-12-27 12:30:12,777][20347] Updated weights for policy 0, policy_version 989120 (0.0007) [2023-12-27 12:30:13,205][20381] Updated weights for policy 1, policy_version 988410 (0.0007) [2023-12-27 12:30:13,270][20381] Updated weights for policy 1, policy_version 988420 (0.0005) [2023-12-27 12:30:13,330][20381] Updated weights for policy 1, policy_version 988430 (0.0005) [2023-12-27 12:30:13,388][20381] Updated weights for policy 1, policy_version 988440 (0.0005) [2023-12-27 12:30:13,462][20347] Updated weights for policy 0, policy_version 989130 (0.0007) [2023-12-27 12:30:13,520][20347] Updated weights for policy 0, policy_version 989140 (0.0007) [2023-12-27 12:30:13,573][20347] Updated weights for policy 0, policy_version 989150 (0.0007) [2023-12-27 12:30:13,626][20347] Updated weights for policy 0, policy_version 989160 (0.0005) [2023-12-27 12:30:14,002][20381] Updated weights for policy 1, policy_version 988450 (0.0010) [2023-12-27 12:30:14,050][20381] Updated weights for policy 1, policy_version 988460 (0.0010) [2023-12-27 12:30:14,097][20381] Updated weights for policy 1, policy_version 988470 (0.0006) [2023-12-27 12:30:14,235][20347] Updated weights for policy 0, policy_version 989170 (0.0010) [2023-12-27 12:30:14,294][20347] Updated weights for policy 0, policy_version 989180 (0.0010) [2023-12-27 12:30:14,347][20347] Updated weights for policy 0, policy_version 989190 (0.0010) [2023-12-27 12:30:14,675][20381] Updated weights for policy 1, policy_version 988480 (0.0005) [2023-12-27 12:30:14,733][20381] Updated weights for policy 1, policy_version 988490 (0.0005) [2023-12-27 12:30:14,799][20381] Updated weights for policy 1, policy_version 988500 (0.0008) [2023-12-27 12:30:15,032][20347] Updated weights for policy 0, policy_version 989200 (0.0010) [2023-12-27 12:30:15,092][20347] Updated weights for policy 0, policy_version 989210 (0.0011) [2023-12-27 12:30:15,153][20347] Updated weights for policy 0, policy_version 989220 (0.0011) [2023-12-27 12:30:15,439][20381] Updated weights for policy 1, policy_version 988510 (0.0006) [2023-12-27 12:30:15,505][20381] Updated weights for policy 1, policy_version 988520 (0.0006) [2023-12-27 12:30:15,559][20381] Updated weights for policy 1, policy_version 988530 (0.0006) [2023-12-27 12:30:15,841][20347] Updated weights for policy 0, policy_version 989230 (0.0010) [2023-12-27 12:30:15,889][20347] Updated weights for policy 0, policy_version 989240 (0.0010) [2023-12-27 12:30:15,949][20347] Updated weights for policy 0, policy_version 989250 (0.0008) [2023-12-27 12:30:16,089][20381] Updated weights for policy 1, policy_version 988540 (0.0005) [2023-12-27 12:30:16,142][20381] Updated weights for policy 1, policy_version 988550 (0.0007) [2023-12-27 12:30:16,185][20381] Updated weights for policy 1, policy_version 988560 (0.0005) [2023-12-27 12:30:16,592][20347] Updated weights for policy 0, policy_version 989260 (0.0007) [2023-12-27 12:30:16,651][20347] Updated weights for policy 0, policy_version 989270 (0.0010) [2023-12-27 12:30:16,702][20347] Updated weights for policy 0, policy_version 989280 (0.0005) [2023-12-27 12:30:16,767][20381] Updated weights for policy 1, policy_version 988570 (0.0006) [2023-12-27 12:30:16,814][20381] Updated weights for policy 1, policy_version 988580 (0.0007) [2023-12-27 12:30:16,868][20381] Updated weights for policy 1, policy_version 988590 (0.0007) [2023-12-27 12:30:16,925][20381] Updated weights for policy 1, policy_version 988600 (0.0008) [2023-12-27 12:30:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 506413056. Throughput: 0: 10277.5, 1: 10200.1. Samples: 506426560. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:17,363][19170] Avg episode reward: [(0, '902.490'), (1, '37.110')] [2023-12-27 12:30:17,380][20347] Updated weights for policy 0, policy_version 989290 (0.0009) [2023-12-27 12:30:17,442][20347] Updated weights for policy 0, policy_version 989300 (0.0010) [2023-12-27 12:30:17,501][20347] Updated weights for policy 0, policy_version 989310 (0.0011) [2023-12-27 12:30:17,548][20381] Updated weights for policy 1, policy_version 988610 (0.0005) [2023-12-27 12:30:17,559][20347] Updated weights for policy 0, policy_version 989320 (0.0010) [2023-12-27 12:30:17,598][20381] Updated weights for policy 1, policy_version 988620 (0.0006) [2023-12-27 12:30:17,648][20381] Updated weights for policy 1, policy_version 988630 (0.0007) [2023-12-27 12:30:18,150][20347] Updated weights for policy 0, policy_version 989330 (0.0008) [2023-12-27 12:30:18,176][20291] KL-divergence is very high: 473.5126 [2023-12-27 12:30:18,186][20291] KL-divergence is very high: 497.1091 [2023-12-27 12:30:18,191][20291] KL-divergence is very high: 598.2441 [2023-12-27 12:30:18,202][20347] Updated weights for policy 0, policy_version 989340 (0.0008) [2023-12-27 12:30:18,217][20291] KL-divergence is very high: 263.5722 [2023-12-27 12:30:18,223][20291] KL-divergence is very high: 118.1091 [2023-12-27 12:30:18,229][20291] KL-divergence is very high: 123.1394 [2023-12-27 12:30:18,234][20291] KL-divergence is very high: 147.2423 [2023-12-27 12:30:18,254][20347] Updated weights for policy 0, policy_version 989350 (0.0008) [2023-12-27 12:30:18,356][20381] Updated weights for policy 1, policy_version 988640 (0.0009) [2023-12-27 12:30:18,421][20381] Updated weights for policy 1, policy_version 988650 (0.0008) [2023-12-27 12:30:18,485][20381] Updated weights for policy 1, policy_version 988660 (0.0008) [2023-12-27 12:30:19,030][20347] Updated weights for policy 0, policy_version 989360 (0.0008) [2023-12-27 12:30:19,067][20381] Updated weights for policy 1, policy_version 988670 (0.0006) [2023-12-27 12:30:19,078][20347] Updated weights for policy 0, policy_version 989370 (0.0007) [2023-12-27 12:30:19,124][20347] Updated weights for policy 0, policy_version 989380 (0.0007) [2023-12-27 12:30:19,126][20381] Updated weights for policy 1, policy_version 988680 (0.0005) [2023-12-27 12:30:19,184][20381] Updated weights for policy 1, policy_version 988690 (0.0005) [2023-12-27 12:30:19,883][20381] Updated weights for policy 1, policy_version 988700 (0.0007) [2023-12-27 12:30:19,920][20347] Updated weights for policy 0, policy_version 989390 (0.0008) [2023-12-27 12:30:19,945][20381] Updated weights for policy 1, policy_version 988710 (0.0007) [2023-12-27 12:30:19,985][20347] Updated weights for policy 0, policy_version 989400 (0.0007) [2023-12-27 12:30:20,010][20381] Updated weights for policy 1, policy_version 988720 (0.0006) [2023-12-27 12:30:20,045][20347] Updated weights for policy 0, policy_version 989410 (0.0007) [2023-12-27 12:30:20,689][20381] Updated weights for policy 1, policy_version 988730 (0.0007) [2023-12-27 12:30:20,720][20347] Updated weights for policy 0, policy_version 989420 (0.0007) [2023-12-27 12:30:20,747][20381] Updated weights for policy 1, policy_version 988740 (0.0010) [2023-12-27 12:30:20,779][20347] Updated weights for policy 0, policy_version 989430 (0.0006) [2023-12-27 12:30:20,807][20381] Updated weights for policy 1, policy_version 988750 (0.0011) [2023-12-27 12:30:20,838][20347] Updated weights for policy 0, policy_version 989440 (0.0006) [2023-12-27 12:30:20,868][20381] Updated weights for policy 1, policy_version 988760 (0.0011) [2023-12-27 12:30:21,550][20347] Updated weights for policy 0, policy_version 989450 (0.0008) [2023-12-27 12:30:21,594][20381] Updated weights for policy 1, policy_version 988770 (0.0006) [2023-12-27 12:30:21,611][20347] Updated weights for policy 0, policy_version 989460 (0.0008) [2023-12-27 12:30:21,661][20381] Updated weights for policy 1, policy_version 988780 (0.0007) [2023-12-27 12:30:21,675][20347] Updated weights for policy 0, policy_version 989470 (0.0007) [2023-12-27 12:30:21,731][20381] Updated weights for policy 1, policy_version 988790 (0.0007) [2023-12-27 12:30:21,739][20347] Updated weights for policy 0, policy_version 989480 (0.0008) [2023-12-27 12:30:22,320][20381] Updated weights for policy 1, policy_version 988800 (0.0008) [2023-12-27 12:30:22,363][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 506511360. Throughput: 0: 10221.2, 1: 10224.3. Samples: 506487356. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:22,363][19170] Avg episode reward: [(0, '902.570'), (1, '62.290')] [2023-12-27 12:30:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000989480_253353984.pth... [2023-12-27 12:30:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000988296_253050880.pth [2023-12-27 12:30:22,387][20381] Updated weights for policy 1, policy_version 988810 (0.0008) [2023-12-27 12:30:22,452][20381] Updated weights for policy 1, policy_version 988820 (0.0008) [2023-12-27 12:30:22,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000988824_253165568.pth... [2023-12-27 12:30:22,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000987608_252854272.pth [2023-12-27 12:30:22,533][20347] Updated weights for policy 0, policy_version 989490 (0.0008) [2023-12-27 12:30:22,593][20347] Updated weights for policy 0, policy_version 989500 (0.0008) [2023-12-27 12:30:22,659][20347] Updated weights for policy 0, policy_version 989510 (0.0007) [2023-12-27 12:30:23,155][20381] Updated weights for policy 1, policy_version 988830 (0.0009) [2023-12-27 12:30:23,208][20381] Updated weights for policy 1, policy_version 988840 (0.0008) [2023-12-27 12:30:23,260][20381] Updated weights for policy 1, policy_version 988850 (0.0008) [2023-12-27 12:30:23,370][20347] Updated weights for policy 0, policy_version 989520 (0.0007) [2023-12-27 12:30:23,432][20347] Updated weights for policy 0, policy_version 989530 (0.0008) [2023-12-27 12:30:23,486][20347] Updated weights for policy 0, policy_version 989540 (0.0010) [2023-12-27 12:30:23,961][20381] Updated weights for policy 1, policy_version 988860 (0.0007) [2023-12-27 12:30:24,007][20381] Updated weights for policy 1, policy_version 988870 (0.0007) [2023-12-27 12:30:24,062][20381] Updated weights for policy 1, policy_version 988880 (0.0008) [2023-12-27 12:30:24,107][20347] Updated weights for policy 0, policy_version 989550 (0.0009) [2023-12-27 12:30:24,176][20347] Updated weights for policy 0, policy_version 989560 (0.0010) [2023-12-27 12:30:24,238][20347] Updated weights for policy 0, policy_version 989570 (0.0010) [2023-12-27 12:30:24,761][20381] Updated weights for policy 1, policy_version 988890 (0.0007) [2023-12-27 12:30:24,821][20381] Updated weights for policy 1, policy_version 988900 (0.0007) [2023-12-27 12:30:24,863][20347] Updated weights for policy 0, policy_version 989580 (0.0010) [2023-12-27 12:30:24,874][20381] Updated weights for policy 1, policy_version 988910 (0.0008) [2023-12-27 12:30:24,917][20347] Updated weights for policy 0, policy_version 989590 (0.0010) [2023-12-27 12:30:24,930][20381] Updated weights for policy 1, policy_version 988920 (0.0007) [2023-12-27 12:30:24,963][20347] Updated weights for policy 0, policy_version 989600 (0.0010) [2023-12-27 12:30:25,595][20381] Updated weights for policy 1, policy_version 988930 (0.0008) [2023-12-27 12:30:25,597][20347] Updated weights for policy 0, policy_version 989610 (0.0009) [2023-12-27 12:30:25,640][20381] Updated weights for policy 1, policy_version 988940 (0.0005) [2023-12-27 12:30:25,642][20347] Updated weights for policy 0, policy_version 989620 (0.0005) [2023-12-27 12:30:25,688][20381] Updated weights for policy 1, policy_version 988950 (0.0005) [2023-12-27 12:30:25,694][20347] Updated weights for policy 0, policy_version 989630 (0.0006) [2023-12-27 12:30:25,746][20347] Updated weights for policy 0, policy_version 989640 (0.0007) [2023-12-27 12:30:26,393][20381] Updated weights for policy 1, policy_version 988960 (0.0007) [2023-12-27 12:30:26,439][20381] Updated weights for policy 1, policy_version 988970 (0.0006) [2023-12-27 12:30:26,446][20347] Updated weights for policy 0, policy_version 989650 (0.0005) [2023-12-27 12:30:26,492][20381] Updated weights for policy 1, policy_version 988980 (0.0007) [2023-12-27 12:30:26,498][20347] Updated weights for policy 0, policy_version 989660 (0.0006) [2023-12-27 12:30:26,556][20347] Updated weights for policy 0, policy_version 989670 (0.0007) [2023-12-27 12:30:27,150][20381] Updated weights for policy 1, policy_version 988990 (0.0010) [2023-12-27 12:30:27,199][20381] Updated weights for policy 1, policy_version 989000 (0.0010) [2023-12-27 12:30:27,258][20347] Updated weights for policy 0, policy_version 989680 (0.0006) [2023-12-27 12:30:27,263][20381] Updated weights for policy 1, policy_version 989010 (0.0011) [2023-12-27 12:30:27,305][20347] Updated weights for policy 0, policy_version 989690 (0.0005) [2023-12-27 12:30:27,356][20347] Updated weights for policy 0, policy_version 989700 (0.0007) [2023-12-27 12:30:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 506617856. Throughput: 0: 10268.7, 1: 10176.6. Samples: 506608420. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:27,363][19170] Avg episode reward: [(0, '902.770'), (1, '73.940')] [2023-12-27 12:30:27,903][20381] Updated weights for policy 1, policy_version 989020 (0.0008) [2023-12-27 12:30:27,962][20381] Updated weights for policy 1, policy_version 989030 (0.0005) [2023-12-27 12:30:28,020][20381] Updated weights for policy 1, policy_version 989040 (0.0011) [2023-12-27 12:30:28,090][20347] Updated weights for policy 0, policy_version 989710 (0.0007) [2023-12-27 12:30:28,147][20347] Updated weights for policy 0, policy_version 989720 (0.0008) [2023-12-27 12:30:28,210][20347] Updated weights for policy 0, policy_version 989730 (0.0007) [2023-12-27 12:30:28,655][20381] Updated weights for policy 1, policy_version 989050 (0.0010) [2023-12-27 12:30:28,713][20381] Updated weights for policy 1, policy_version 989060 (0.0007) [2023-12-27 12:30:28,768][20381] Updated weights for policy 1, policy_version 989070 (0.0008) [2023-12-27 12:30:28,831][20381] Updated weights for policy 1, policy_version 989080 (0.0007) [2023-12-27 12:30:28,832][20347] Updated weights for policy 0, policy_version 989740 (0.0005) [2023-12-27 12:30:28,882][20347] Updated weights for policy 0, policy_version 989750 (0.0006) [2023-12-27 12:30:28,941][20347] Updated weights for policy 0, policy_version 989760 (0.0007) [2023-12-27 12:30:29,518][20381] Updated weights for policy 1, policy_version 989090 (0.0011) [2023-12-27 12:30:29,572][20381] Updated weights for policy 1, policy_version 989100 (0.0007) [2023-12-27 12:30:29,620][20347] Updated weights for policy 0, policy_version 989770 (0.0008) [2023-12-27 12:30:29,628][20381] Updated weights for policy 1, policy_version 989110 (0.0005) [2023-12-27 12:30:29,677][20347] Updated weights for policy 0, policy_version 989780 (0.0008) [2023-12-27 12:30:29,726][20347] Updated weights for policy 0, policy_version 989790 (0.0009) [2023-12-27 12:30:29,789][20347] Updated weights for policy 0, policy_version 989800 (0.0008) [2023-12-27 12:30:30,266][20381] Updated weights for policy 1, policy_version 989120 (0.0010) [2023-12-27 12:30:30,327][20381] Updated weights for policy 1, policy_version 989130 (0.0012) [2023-12-27 12:30:30,388][20381] Updated weights for policy 1, policy_version 989140 (0.0011) [2023-12-27 12:30:30,589][20347] Updated weights for policy 0, policy_version 989810 (0.0007) [2023-12-27 12:30:30,653][20347] Updated weights for policy 0, policy_version 989820 (0.0007) [2023-12-27 12:30:30,706][20347] Updated weights for policy 0, policy_version 989830 (0.0008) [2023-12-27 12:30:31,009][20381] Updated weights for policy 1, policy_version 989150 (0.0011) [2023-12-27 12:30:31,075][20381] Updated weights for policy 1, policy_version 989160 (0.0008) [2023-12-27 12:30:31,137][20381] Updated weights for policy 1, policy_version 989170 (0.0010) [2023-12-27 12:30:31,442][20347] Updated weights for policy 0, policy_version 989840 (0.0007) [2023-12-27 12:30:31,504][20347] Updated weights for policy 0, policy_version 989850 (0.0005) [2023-12-27 12:30:31,558][20347] Updated weights for policy 0, policy_version 989860 (0.0007) [2023-12-27 12:30:31,578][20291] KL-divergence is very high: 183.1656 [2023-12-27 12:30:31,871][20381] Updated weights for policy 1, policy_version 989180 (0.0010) [2023-12-27 12:30:31,935][20381] Updated weights for policy 1, policy_version 989190 (0.0008) [2023-12-27 12:30:31,997][20381] Updated weights for policy 1, policy_version 989200 (0.0008) [2023-12-27 12:30:32,278][20347] Updated weights for policy 0, policy_version 989870 (0.0007) [2023-12-27 12:30:32,332][20347] Updated weights for policy 0, policy_version 989880 (0.0008) [2023-12-27 12:30:32,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 506716160. Throughput: 0: 10190.1, 1: 10243.1. Samples: 506729612. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:32,363][19170] Avg episode reward: [(0, '932.710'), (1, '30.650')] [2023-12-27 12:30:32,397][20347] Updated weights for policy 0, policy_version 989890 (0.0008) [2023-12-27 12:30:32,674][20381] Updated weights for policy 1, policy_version 989210 (0.0008) [2023-12-27 12:30:32,730][20381] Updated weights for policy 1, policy_version 989220 (0.0006) [2023-12-27 12:30:32,789][20381] Updated weights for policy 1, policy_version 989230 (0.0006) [2023-12-27 12:30:32,840][20381] Updated weights for policy 1, policy_version 989240 (0.0005) [2023-12-27 12:30:33,186][20347] Updated weights for policy 0, policy_version 989900 (0.0007) [2023-12-27 12:30:33,249][20347] Updated weights for policy 0, policy_version 989910 (0.0008) [2023-12-27 12:30:33,299][20347] Updated weights for policy 0, policy_version 989920 (0.0008) [2023-12-27 12:30:33,466][20381] Updated weights for policy 1, policy_version 989250 (0.0008) [2023-12-27 12:30:33,527][20381] Updated weights for policy 1, policy_version 989260 (0.0007) [2023-12-27 12:30:33,588][20381] Updated weights for policy 1, policy_version 989270 (0.0008) [2023-12-27 12:30:34,001][20347] Updated weights for policy 0, policy_version 989930 (0.0008) [2023-12-27 12:30:34,052][20347] Updated weights for policy 0, policy_version 989940 (0.0010) [2023-12-27 12:30:34,104][20347] Updated weights for policy 0, policy_version 989950 (0.0010) [2023-12-27 12:30:34,160][20347] Updated weights for policy 0, policy_version 989960 (0.0008) [2023-12-27 12:30:34,278][20381] Updated weights for policy 1, policy_version 989280 (0.0007) [2023-12-27 12:30:34,333][20381] Updated weights for policy 1, policy_version 989290 (0.0007) [2023-12-27 12:30:34,389][20381] Updated weights for policy 1, policy_version 989300 (0.0007) [2023-12-27 12:30:34,860][20347] Updated weights for policy 0, policy_version 989970 (0.0010) [2023-12-27 12:30:34,908][20347] Updated weights for policy 0, policy_version 989980 (0.0010) [2023-12-27 12:30:34,956][20347] Updated weights for policy 0, policy_version 989990 (0.0010) [2023-12-27 12:30:35,025][20381] Updated weights for policy 1, policy_version 989310 (0.0007) [2023-12-27 12:30:35,084][20381] Updated weights for policy 1, policy_version 989320 (0.0007) [2023-12-27 12:30:35,145][20381] Updated weights for policy 1, policy_version 989330 (0.0008) [2023-12-27 12:30:35,651][20347] Updated weights for policy 0, policy_version 990000 (0.0007) [2023-12-27 12:30:35,700][20347] Updated weights for policy 0, policy_version 990010 (0.0007) [2023-12-27 12:30:35,757][20347] Updated weights for policy 0, policy_version 990020 (0.0008) [2023-12-27 12:30:35,811][20381] Updated weights for policy 1, policy_version 989340 (0.0007) [2023-12-27 12:30:35,859][20381] Updated weights for policy 1, policy_version 989350 (0.0005) [2023-12-27 12:30:35,914][20381] Updated weights for policy 1, policy_version 989360 (0.0005) [2023-12-27 12:30:36,445][20347] Updated weights for policy 0, policy_version 990030 (0.0007) [2023-12-27 12:30:36,506][20347] Updated weights for policy 0, policy_version 990040 (0.0008) [2023-12-27 12:30:36,559][20381] Updated weights for policy 1, policy_version 989370 (0.0005) [2023-12-27 12:30:36,567][20347] Updated weights for policy 0, policy_version 990050 (0.0009) [2023-12-27 12:30:36,618][20381] Updated weights for policy 1, policy_version 989380 (0.0006) [2023-12-27 12:30:36,674][20381] Updated weights for policy 1, policy_version 989390 (0.0007) [2023-12-27 12:30:36,727][20381] Updated weights for policy 1, policy_version 989400 (0.0008) [2023-12-27 12:30:37,206][20347] Updated weights for policy 0, policy_version 990060 (0.0008) [2023-12-27 12:30:37,253][20347] Updated weights for policy 0, policy_version 990070 (0.0008) [2023-12-27 12:30:37,306][20347] Updated weights for policy 0, policy_version 990080 (0.0008) [2023-12-27 12:30:37,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 506822656. Throughput: 0: 10135.2, 1: 10251.9. Samples: 506789816. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:37,363][19170] Avg episode reward: [(0, '980.740'), (1, '37.750')] [2023-12-27 12:30:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000989400_253313024.pth... [2023-12-27 12:30:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000990088_253509632.pth... [2023-12-27 12:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000988184_253001728.pth [2023-12-27 12:30:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000988904_253206528.pth [2023-12-27 12:30:37,460][20381] Updated weights for policy 1, policy_version 989410 (0.0008) [2023-12-27 12:30:37,507][20381] Updated weights for policy 1, policy_version 989420 (0.0007) [2023-12-27 12:30:37,560][20381] Updated weights for policy 1, policy_version 989430 (0.0007) [2023-12-27 12:30:37,917][20347] Updated weights for policy 0, policy_version 990090 (0.0007) [2023-12-27 12:30:37,979][20347] Updated weights for policy 0, policy_version 990100 (0.0007) [2023-12-27 12:30:38,039][20347] Updated weights for policy 0, policy_version 990110 (0.0008) [2023-12-27 12:30:38,095][20347] Updated weights for policy 0, policy_version 990120 (0.0007) [2023-12-27 12:30:38,231][20381] Updated weights for policy 1, policy_version 989440 (0.0006) [2023-12-27 12:30:38,279][20381] Updated weights for policy 1, policy_version 989450 (0.0005) [2023-12-27 12:30:38,334][20381] Updated weights for policy 1, policy_version 989460 (0.0006) [2023-12-27 12:30:38,783][20347] Updated weights for policy 0, policy_version 990130 (0.0008) [2023-12-27 12:30:38,843][20347] Updated weights for policy 0, policy_version 990140 (0.0007) [2023-12-27 12:30:38,906][20347] Updated weights for policy 0, policy_version 990150 (0.0007) [2023-12-27 12:30:38,971][20381] Updated weights for policy 1, policy_version 989470 (0.0008) [2023-12-27 12:30:39,034][20381] Updated weights for policy 1, policy_version 989480 (0.0008) [2023-12-27 12:30:39,092][20381] Updated weights for policy 1, policy_version 989490 (0.0007) [2023-12-27 12:30:39,625][20347] Updated weights for policy 0, policy_version 990160 (0.0008) [2023-12-27 12:30:39,685][20347] Updated weights for policy 0, policy_version 990170 (0.0007) [2023-12-27 12:30:39,745][20347] Updated weights for policy 0, policy_version 990180 (0.0008) [2023-12-27 12:30:39,799][20381] Updated weights for policy 1, policy_version 989500 (0.0008) [2023-12-27 12:30:39,867][20381] Updated weights for policy 1, policy_version 989510 (0.0008) [2023-12-27 12:30:39,917][20381] Updated weights for policy 1, policy_version 989520 (0.0008) [2023-12-27 12:30:40,480][20347] Updated weights for policy 0, policy_version 990190 (0.0008) [2023-12-27 12:30:40,536][20347] Updated weights for policy 0, policy_version 990200 (0.0006) [2023-12-27 12:30:40,605][20347] Updated weights for policy 0, policy_version 990210 (0.0006) [2023-12-27 12:30:40,611][20381] Updated weights for policy 1, policy_version 989530 (0.0007) [2023-12-27 12:30:40,673][20381] Updated weights for policy 1, policy_version 989540 (0.0006) [2023-12-27 12:30:40,730][20381] Updated weights for policy 1, policy_version 989550 (0.0008) [2023-12-27 12:30:40,792][20381] Updated weights for policy 1, policy_version 989560 (0.0008) [2023-12-27 12:30:41,265][20347] Updated weights for policy 0, policy_version 990220 (0.0006) [2023-12-27 12:30:41,328][20347] Updated weights for policy 0, policy_version 990230 (0.0007) [2023-12-27 12:30:41,395][20347] Updated weights for policy 0, policy_version 990240 (0.0007) [2023-12-27 12:30:41,554][20381] Updated weights for policy 1, policy_version 989570 (0.0011) [2023-12-27 12:30:41,615][20381] Updated weights for policy 1, policy_version 989580 (0.0011) [2023-12-27 12:30:41,676][20381] Updated weights for policy 1, policy_version 989590 (0.0011) [2023-12-27 12:30:42,080][20347] Updated weights for policy 0, policy_version 990250 (0.0006) [2023-12-27 12:30:42,139][20347] Updated weights for policy 0, policy_version 990260 (0.0007) [2023-12-27 12:30:42,199][20347] Updated weights for policy 0, policy_version 990270 (0.0008) [2023-12-27 12:30:42,247][20347] Updated weights for policy 0, policy_version 990280 (0.0007) [2023-12-27 12:30:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 506920960. Throughput: 0: 10128.0, 1: 10301.9. Samples: 506912196. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:42,363][19170] Avg episode reward: [(0, '980.740'), (1, '28.630')] [2023-12-27 12:30:42,395][20381] Updated weights for policy 1, policy_version 989600 (0.0008) [2023-12-27 12:30:42,452][20381] Updated weights for policy 1, policy_version 989610 (0.0006) [2023-12-27 12:30:42,513][20381] Updated weights for policy 1, policy_version 989620 (0.0005) [2023-12-27 12:30:42,969][20347] Updated weights for policy 0, policy_version 990290 (0.0006) [2023-12-27 12:30:43,029][20291] KL-divergence is very high: 274.8290 [2023-12-27 12:30:43,030][20347] Updated weights for policy 0, policy_version 990300 (0.0005) [2023-12-27 12:30:43,069][20291] KL-divergence is very high: 168.1633 [2023-12-27 12:30:43,079][20347] Updated weights for policy 0, policy_version 990310 (0.0005) [2023-12-27 12:30:43,147][20381] Updated weights for policy 1, policy_version 989630 (0.0007) [2023-12-27 12:30:43,206][20381] Updated weights for policy 1, policy_version 989640 (0.0006) [2023-12-27 12:30:43,268][20381] Updated weights for policy 1, policy_version 989650 (0.0005) [2023-12-27 12:30:43,714][20347] Updated weights for policy 0, policy_version 990320 (0.0008) [2023-12-27 12:30:43,765][20347] Updated weights for policy 0, policy_version 990330 (0.0008) [2023-12-27 12:30:43,813][20347] Updated weights for policy 0, policy_version 990340 (0.0007) [2023-12-27 12:30:43,852][20381] Updated weights for policy 1, policy_version 989660 (0.0008) [2023-12-27 12:30:43,901][20381] Updated weights for policy 1, policy_version 989670 (0.0007) [2023-12-27 12:30:43,961][20381] Updated weights for policy 1, policy_version 989680 (0.0005) [2023-12-27 12:30:44,552][20347] Updated weights for policy 0, policy_version 990350 (0.0007) [2023-12-27 12:30:44,567][20381] Updated weights for policy 1, policy_version 989690 (0.0007) [2023-12-27 12:30:44,608][20347] Updated weights for policy 0, policy_version 990360 (0.0006) [2023-12-27 12:30:44,625][20381] Updated weights for policy 1, policy_version 989700 (0.0007) [2023-12-27 12:30:44,665][20347] Updated weights for policy 0, policy_version 990370 (0.0005) [2023-12-27 12:30:44,678][20381] Updated weights for policy 1, policy_version 989710 (0.0007) [2023-12-27 12:30:44,729][20381] Updated weights for policy 1, policy_version 989720 (0.0007) [2023-12-27 12:30:45,365][20347] Updated weights for policy 0, policy_version 990380 (0.0008) [2023-12-27 12:30:45,422][20381] Updated weights for policy 1, policy_version 989730 (0.0006) [2023-12-27 12:30:45,423][20347] Updated weights for policy 0, policy_version 990390 (0.0010) [2023-12-27 12:30:45,477][20381] Updated weights for policy 1, policy_version 989740 (0.0005) [2023-12-27 12:30:45,483][20347] Updated weights for policy 0, policy_version 990400 (0.0010) [2023-12-27 12:30:45,532][20381] Updated weights for policy 1, policy_version 989750 (0.0005) [2023-12-27 12:30:46,159][20381] Updated weights for policy 1, policy_version 989760 (0.0006) [2023-12-27 12:30:46,173][20347] Updated weights for policy 0, policy_version 990410 (0.0010) [2023-12-27 12:30:46,217][20381] Updated weights for policy 1, policy_version 989770 (0.0006) [2023-12-27 12:30:46,255][20347] Updated weights for policy 0, policy_version 990420 (0.0009) [2023-12-27 12:30:46,269][20381] Updated weights for policy 1, policy_version 989780 (0.0007) [2023-12-27 12:30:46,309][20347] Updated weights for policy 0, policy_version 990430 (0.0008) [2023-12-27 12:30:46,366][20347] Updated weights for policy 0, policy_version 990440 (0.0010) [2023-12-27 12:30:46,862][20381] Updated weights for policy 1, policy_version 989790 (0.0006) [2023-12-27 12:30:46,918][20381] Updated weights for policy 1, policy_version 989800 (0.0005) [2023-12-27 12:30:46,968][20381] Updated weights for policy 1, policy_version 989810 (0.0005) [2023-12-27 12:30:46,991][20347] Updated weights for policy 0, policy_version 990450 (0.0005) [2023-12-27 12:30:47,052][20347] Updated weights for policy 0, policy_version 990460 (0.0005) [2023-12-27 12:30:47,110][20347] Updated weights for policy 0, policy_version 990470 (0.0006) [2023-12-27 12:30:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 507027456. Throughput: 0: 10107.8, 1: 10400.7. Samples: 507036760. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:47,363][19170] Avg episode reward: [(0, '961.110'), (1, '34.590')] [2023-12-27 12:30:47,623][20381] Updated weights for policy 1, policy_version 989820 (0.0008) [2023-12-27 12:30:47,670][20381] Updated weights for policy 1, policy_version 989830 (0.0008) [2023-12-27 12:30:47,695][20347] Updated weights for policy 0, policy_version 990480 (0.0010) [2023-12-27 12:30:47,724][20381] Updated weights for policy 1, policy_version 989840 (0.0006) [2023-12-27 12:30:47,754][20347] Updated weights for policy 0, policy_version 990490 (0.0010) [2023-12-27 12:30:47,812][20347] Updated weights for policy 0, policy_version 990500 (0.0010) [2023-12-27 12:30:48,392][20381] Updated weights for policy 1, policy_version 989850 (0.0011) [2023-12-27 12:30:48,459][20381] Updated weights for policy 1, policy_version 989860 (0.0011) [2023-12-27 12:30:48,473][20347] Updated weights for policy 0, policy_version 990510 (0.0011) [2023-12-27 12:30:48,515][20381] Updated weights for policy 1, policy_version 989870 (0.0011) [2023-12-27 12:30:48,532][20347] Updated weights for policy 0, policy_version 990520 (0.0010) [2023-12-27 12:30:48,574][20381] Updated weights for policy 1, policy_version 989880 (0.0008) [2023-12-27 12:30:48,591][20347] Updated weights for policy 0, policy_version 990530 (0.0010) [2023-12-27 12:30:49,200][20347] Updated weights for policy 0, policy_version 990540 (0.0010) [2023-12-27 12:30:49,265][20347] Updated weights for policy 0, policy_version 990550 (0.0010) [2023-12-27 12:30:49,324][20347] Updated weights for policy 0, policy_version 990560 (0.0008) [2023-12-27 12:30:49,333][20381] Updated weights for policy 1, policy_version 989890 (0.0006) [2023-12-27 12:30:49,401][20381] Updated weights for policy 1, policy_version 989900 (0.0009) [2023-12-27 12:30:49,469][20381] Updated weights for policy 1, policy_version 989910 (0.0008) [2023-12-27 12:30:50,072][20347] Updated weights for policy 0, policy_version 990570 (0.0008) [2023-12-27 12:30:50,138][20347] Updated weights for policy 0, policy_version 990580 (0.0007) [2023-12-27 12:30:50,146][20381] Updated weights for policy 1, policy_version 989920 (0.0006) [2023-12-27 12:30:50,202][20381] Updated weights for policy 1, policy_version 989930 (0.0005) [2023-12-27 12:30:50,204][20347] Updated weights for policy 0, policy_version 990590 (0.0008) [2023-12-27 12:30:50,256][20381] Updated weights for policy 1, policy_version 989940 (0.0005) [2023-12-27 12:30:50,268][20347] Updated weights for policy 0, policy_version 990600 (0.0008) [2023-12-27 12:30:50,908][20381] Updated weights for policy 1, policy_version 989950 (0.0007) [2023-12-27 12:30:50,952][20347] Updated weights for policy 0, policy_version 990610 (0.0005) [2023-12-27 12:30:50,973][20381] Updated weights for policy 1, policy_version 989960 (0.0008) [2023-12-27 12:30:51,011][20347] Updated weights for policy 0, policy_version 990620 (0.0007) [2023-12-27 12:30:51,036][20381] Updated weights for policy 1, policy_version 989970 (0.0008) [2023-12-27 12:30:51,081][20347] Updated weights for policy 0, policy_version 990630 (0.0008) [2023-12-27 12:30:51,790][20381] Updated weights for policy 1, policy_version 989980 (0.0007) [2023-12-27 12:30:51,798][20347] Updated weights for policy 0, policy_version 990640 (0.0009) [2023-12-27 12:30:51,848][20381] Updated weights for policy 1, policy_version 989990 (0.0006) [2023-12-27 12:30:51,859][20347] Updated weights for policy 0, policy_version 990650 (0.0008) [2023-12-27 12:30:51,908][20381] Updated weights for policy 1, policy_version 990000 (0.0010) [2023-12-27 12:30:51,921][20347] Updated weights for policy 0, policy_version 990660 (0.0010) [2023-12-27 12:30:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 507125760. Throughput: 0: 10100.0, 1: 10398.9. Samples: 507097748. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:52,363][19170] Avg episode reward: [(0, '951.330'), (1, '34.390')] [2023-12-27 12:30:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000990664_253657088.pth... [2023-12-27 12:30:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000990008_253468672.pth... [2023-12-27 12:30:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000988824_253165568.pth [2023-12-27 12:30:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000989480_253353984.pth [2023-12-27 12:30:52,522][20381] Updated weights for policy 1, policy_version 990010 (0.0006) [2023-12-27 12:30:52,585][20381] Updated weights for policy 1, policy_version 990020 (0.0006) [2023-12-27 12:30:52,594][20347] Updated weights for policy 0, policy_version 990670 (0.0008) [2023-12-27 12:30:52,641][20381] Updated weights for policy 1, policy_version 990030 (0.0006) [2023-12-27 12:30:52,648][20347] Updated weights for policy 0, policy_version 990680 (0.0009) [2023-12-27 12:30:52,693][20381] Updated weights for policy 1, policy_version 990040 (0.0005) [2023-12-27 12:30:52,697][20347] Updated weights for policy 0, policy_version 990690 (0.0010) [2023-12-27 12:30:53,261][20381] Updated weights for policy 1, policy_version 990050 (0.0009) [2023-12-27 12:30:53,272][20347] Updated weights for policy 0, policy_version 990700 (0.0008) [2023-12-27 12:30:53,310][20381] Updated weights for policy 1, policy_version 990060 (0.0010) [2023-12-27 12:30:53,319][20347] Updated weights for policy 0, policy_version 990710 (0.0005) [2023-12-27 12:30:53,363][20381] Updated weights for policy 1, policy_version 990070 (0.0011) [2023-12-27 12:30:53,372][20347] Updated weights for policy 0, policy_version 990720 (0.0007) [2023-12-27 12:30:53,985][20347] Updated weights for policy 0, policy_version 990730 (0.0007) [2023-12-27 12:30:54,035][20381] Updated weights for policy 1, policy_version 990080 (0.0011) [2023-12-27 12:30:54,043][20347] Updated weights for policy 0, policy_version 990740 (0.0005) [2023-12-27 12:30:54,093][20347] Updated weights for policy 0, policy_version 990750 (0.0007) [2023-12-27 12:30:54,094][20381] Updated weights for policy 1, policy_version 990090 (0.0011) [2023-12-27 12:30:54,151][20347] Updated weights for policy 0, policy_version 990760 (0.0006) [2023-12-27 12:30:54,154][20381] Updated weights for policy 1, policy_version 990100 (0.0011) [2023-12-27 12:30:54,813][20381] Updated weights for policy 1, policy_version 990110 (0.0011) [2023-12-27 12:30:54,841][20347] Updated weights for policy 0, policy_version 990770 (0.0007) [2023-12-27 12:30:54,862][20381] Updated weights for policy 1, policy_version 990120 (0.0010) [2023-12-27 12:30:54,897][20347] Updated weights for policy 0, policy_version 990780 (0.0006) [2023-12-27 12:30:54,911][20381] Updated weights for policy 1, policy_version 990130 (0.0010) [2023-12-27 12:30:54,960][20347] Updated weights for policy 0, policy_version 990790 (0.0008) [2023-12-27 12:30:55,498][20381] Updated weights for policy 1, policy_version 990140 (0.0008) [2023-12-27 12:30:55,549][20381] Updated weights for policy 1, policy_version 990150 (0.0005) [2023-12-27 12:30:55,595][20381] Updated weights for policy 1, policy_version 990160 (0.0005) [2023-12-27 12:30:55,656][20347] Updated weights for policy 0, policy_version 990800 (0.0008) [2023-12-27 12:30:55,707][20347] Updated weights for policy 0, policy_version 990810 (0.0009) [2023-12-27 12:30:55,756][20347] Updated weights for policy 0, policy_version 990820 (0.0008) [2023-12-27 12:30:56,117][20381] Updated weights for policy 1, policy_version 990170 (0.0005) [2023-12-27 12:30:56,181][20381] Updated weights for policy 1, policy_version 990180 (0.0008) [2023-12-27 12:30:56,241][20381] Updated weights for policy 1, policy_version 990190 (0.0005) [2023-12-27 12:30:56,290][20381] Updated weights for policy 1, policy_version 990200 (0.0005) [2023-12-27 12:30:56,485][20347] Updated weights for policy 0, policy_version 990830 (0.0007) [2023-12-27 12:30:56,543][20347] Updated weights for policy 0, policy_version 990840 (0.0009) [2023-12-27 12:30:56,600][20347] Updated weights for policy 0, policy_version 990850 (0.0010) [2023-12-27 12:30:56,857][20381] Updated weights for policy 1, policy_version 990210 (0.0005) [2023-12-27 12:30:56,905][20381] Updated weights for policy 1, policy_version 990220 (0.0005) [2023-12-27 12:30:56,957][20381] Updated weights for policy 1, policy_version 990230 (0.0005) [2023-12-27 12:30:57,182][20347] Updated weights for policy 0, policy_version 990860 (0.0008) [2023-12-27 12:30:57,242][20347] Updated weights for policy 0, policy_version 990870 (0.0006) [2023-12-27 12:30:57,296][20347] Updated weights for policy 0, policy_version 990880 (0.0005) [2023-12-27 12:30:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 507240448. Throughput: 0: 10070.0, 1: 10467.9. Samples: 507224680. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:30:57,363][19170] Avg episode reward: [(0, '873.480'), (1, '57.990')] [2023-12-27 12:30:57,544][20381] Updated weights for policy 1, policy_version 990240 (0.0009) [2023-12-27 12:30:57,598][20381] Updated weights for policy 1, policy_version 990250 (0.0010) [2023-12-27 12:30:57,666][20381] Updated weights for policy 1, policy_version 990260 (0.0011) [2023-12-27 12:30:57,856][20347] Updated weights for policy 0, policy_version 990890 (0.0008) [2023-12-27 12:30:57,907][20347] Updated weights for policy 0, policy_version 990900 (0.0010) [2023-12-27 12:30:57,961][20347] Updated weights for policy 0, policy_version 990910 (0.0010) [2023-12-27 12:30:58,012][20347] Updated weights for policy 0, policy_version 990920 (0.0010) [2023-12-27 12:30:58,244][20381] Updated weights for policy 1, policy_version 990270 (0.0008) [2023-12-27 12:30:58,302][20381] Updated weights for policy 1, policy_version 990280 (0.0008) [2023-12-27 12:30:58,369][20381] Updated weights for policy 1, policy_version 990291 (0.0009) [2023-12-27 12:30:58,746][20347] Updated weights for policy 0, policy_version 990930 (0.0009) [2023-12-27 12:30:58,810][20347] Updated weights for policy 0, policy_version 990940 (0.0009) [2023-12-27 12:30:58,873][20347] Updated weights for policy 0, policy_version 990950 (0.0009) [2023-12-27 12:30:59,199][20381] Updated weights for policy 1, policy_version 990301 (0.0006) [2023-12-27 12:30:59,265][20381] Updated weights for policy 1, policy_version 990311 (0.0008) [2023-12-27 12:30:59,331][20381] Updated weights for policy 1, policy_version 990321 (0.0006) [2023-12-27 12:30:59,677][20347] Updated weights for policy 0, policy_version 990960 (0.0006) [2023-12-27 12:30:59,736][20347] Updated weights for policy 0, policy_version 990970 (0.0011) [2023-12-27 12:30:59,785][20347] Updated weights for policy 0, policy_version 990980 (0.0010) [2023-12-27 12:30:59,939][20381] Updated weights for policy 1, policy_version 990331 (0.0008) [2023-12-27 12:30:59,999][20381] Updated weights for policy 1, policy_version 990341 (0.0006) [2023-12-27 12:31:00,061][20381] Updated weights for policy 1, policy_version 990351 (0.0006) [2023-12-27 12:31:00,485][20347] Updated weights for policy 0, policy_version 990990 (0.0010) [2023-12-27 12:31:00,536][20347] Updated weights for policy 0, policy_version 991000 (0.0010) [2023-12-27 12:31:00,591][20347] Updated weights for policy 0, policy_version 991010 (0.0005) [2023-12-27 12:31:00,746][20381] Updated weights for policy 1, policy_version 990361 (0.0007) [2023-12-27 12:31:00,810][20381] Updated weights for policy 1, policy_version 990371 (0.0006) [2023-12-27 12:31:00,865][20381] Updated weights for policy 1, policy_version 990381 (0.0005) [2023-12-27 12:31:00,920][20381] Updated weights for policy 1, policy_version 990391 (0.0005) [2023-12-27 12:31:01,263][20347] Updated weights for policy 0, policy_version 991020 (0.0006) [2023-12-27 12:31:01,323][20347] Updated weights for policy 0, policy_version 991030 (0.0007) [2023-12-27 12:31:01,390][20347] Updated weights for policy 0, policy_version 991040 (0.0008) [2023-12-27 12:31:01,541][20381] Updated weights for policy 1, policy_version 990401 (0.0010) [2023-12-27 12:31:01,596][20381] Updated weights for policy 1, policy_version 990411 (0.0011) [2023-12-27 12:31:01,663][20381] Updated weights for policy 1, policy_version 990421 (0.0009) [2023-12-27 12:31:02,051][20347] Updated weights for policy 0, policy_version 991050 (0.0007) [2023-12-27 12:31:02,100][20347] Updated weights for policy 0, policy_version 991060 (0.0010) [2023-12-27 12:31:02,145][20347] Updated weights for policy 0, policy_version 991070 (0.0010) [2023-12-27 12:31:02,202][20347] Updated weights for policy 0, policy_version 991080 (0.0010) [2023-12-27 12:31:02,337][20381] Updated weights for policy 1, policy_version 990431 (0.0009) [2023-12-27 12:31:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 507338752. Throughput: 0: 10094.2, 1: 10425.9. Samples: 507349964. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:31:02,363][19170] Avg episode reward: [(0, '873.720'), (1, '90.990')] [2023-12-27 12:31:02,404][20381] Updated weights for policy 1, policy_version 990441 (0.0008) [2023-12-27 12:31:02,464][20381] Updated weights for policy 1, policy_version 990451 (0.0010) [2023-12-27 12:31:02,853][20347] Updated weights for policy 0, policy_version 991090 (0.0010) [2023-12-27 12:31:02,910][20347] Updated weights for policy 0, policy_version 991100 (0.0010) [2023-12-27 12:31:02,945][20291] KL-divergence is very high: 110.5937 [2023-12-27 12:31:02,968][20347] Updated weights for policy 0, policy_version 991110 (0.0010) [2023-12-27 12:31:03,088][20381] Updated weights for policy 1, policy_version 990461 (0.0008) [2023-12-27 12:31:03,147][20381] Updated weights for policy 1, policy_version 990471 (0.0010) [2023-12-27 12:31:03,204][20381] Updated weights for policy 1, policy_version 990481 (0.0011) [2023-12-27 12:31:03,537][20347] Updated weights for policy 0, policy_version 991120 (0.0010) [2023-12-27 12:31:03,593][20347] Updated weights for policy 0, policy_version 991130 (0.0010) [2023-12-27 12:31:03,648][20347] Updated weights for policy 0, policy_version 991140 (0.0010) [2023-12-27 12:31:03,904][20381] Updated weights for policy 1, policy_version 990491 (0.0010) [2023-12-27 12:31:03,970][20381] Updated weights for policy 1, policy_version 990501 (0.0008) [2023-12-27 12:31:04,041][20381] Updated weights for policy 1, policy_version 990511 (0.0009) [2023-12-27 12:31:04,347][20347] Updated weights for policy 0, policy_version 991150 (0.0008) [2023-12-27 12:31:04,410][20347] Updated weights for policy 0, policy_version 991160 (0.0008) [2023-12-27 12:31:04,470][20347] Updated weights for policy 0, policy_version 991170 (0.0007) [2023-12-27 12:31:04,709][20381] Updated weights for policy 1, policy_version 990521 (0.0007) [2023-12-27 12:31:04,763][20381] Updated weights for policy 1, policy_version 990531 (0.0008) [2023-12-27 12:31:04,823][20381] Updated weights for policy 1, policy_version 990541 (0.0007) [2023-12-27 12:31:04,882][20381] Updated weights for policy 1, policy_version 990551 (0.0008) [2023-12-27 12:31:05,149][20347] Updated weights for policy 0, policy_version 991180 (0.0006) [2023-12-27 12:31:05,208][20347] Updated weights for policy 0, policy_version 991190 (0.0005) [2023-12-27 12:31:05,270][20347] Updated weights for policy 0, policy_version 991200 (0.0006) [2023-12-27 12:31:05,601][20381] Updated weights for policy 1, policy_version 990561 (0.0006) [2023-12-27 12:31:05,659][20381] Updated weights for policy 1, policy_version 990571 (0.0007) [2023-12-27 12:31:05,709][20381] Updated weights for policy 1, policy_version 990581 (0.0010) [2023-12-27 12:31:05,815][20347] Updated weights for policy 0, policy_version 991210 (0.0006) [2023-12-27 12:31:05,866][20347] Updated weights for policy 0, policy_version 991220 (0.0009) [2023-12-27 12:31:05,922][20347] Updated weights for policy 0, policy_version 991230 (0.0010) [2023-12-27 12:31:05,974][20347] Updated weights for policy 0, policy_version 991240 (0.0010) [2023-12-27 12:31:06,382][20381] Updated weights for policy 1, policy_version 990591 (0.0009) [2023-12-27 12:31:06,447][20381] Updated weights for policy 1, policy_version 990601 (0.0008) [2023-12-27 12:31:06,509][20381] Updated weights for policy 1, policy_version 990611 (0.0008) [2023-12-27 12:31:06,684][20347] Updated weights for policy 0, policy_version 991250 (0.0011) [2023-12-27 12:31:06,746][20347] Updated weights for policy 0, policy_version 991260 (0.0009) [2023-12-27 12:31:06,808][20347] Updated weights for policy 0, policy_version 991270 (0.0010) [2023-12-27 12:31:07,193][20381] Updated weights for policy 1, policy_version 990621 (0.0008) [2023-12-27 12:31:07,249][20381] Updated weights for policy 1, policy_version 990631 (0.0007) [2023-12-27 12:31:07,300][20381] Updated weights for policy 1, policy_version 990641 (0.0008) [2023-12-27 12:31:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 507445248. Throughput: 0: 10144.2, 1: 10397.4. Samples: 507411728. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:31:07,364][19170] Avg episode reward: [(0, '892.870'), (1, '176.360')] [2023-12-27 12:31:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000990648_253632512.pth... [2023-12-27 12:31:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000991272_253812736.pth... [2023-12-27 12:31:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000989400_253313024.pth [2023-12-27 12:31:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000990088_253509632.pth [2023-12-27 12:31:07,482][20347] Updated weights for policy 0, policy_version 991280 (0.0010) [2023-12-27 12:31:07,537][20347] Updated weights for policy 0, policy_version 991290 (0.0010) [2023-12-27 12:31:07,590][20347] Updated weights for policy 0, policy_version 991300 (0.0010) [2023-12-27 12:31:08,009][20381] Updated weights for policy 1, policy_version 990651 (0.0007) [2023-12-27 12:31:08,073][20381] Updated weights for policy 1, policy_version 990661 (0.0007) [2023-12-27 12:31:08,131][20381] Updated weights for policy 1, policy_version 990671 (0.0007) [2023-12-27 12:31:08,258][20347] Updated weights for policy 0, policy_version 991310 (0.0010) [2023-12-27 12:31:08,322][20347] Updated weights for policy 0, policy_version 991320 (0.0010) [2023-12-27 12:31:08,388][20347] Updated weights for policy 0, policy_version 991330 (0.0010) [2023-12-27 12:31:08,761][20381] Updated weights for policy 1, policy_version 990681 (0.0007) [2023-12-27 12:31:08,814][20381] Updated weights for policy 1, policy_version 990691 (0.0005) [2023-12-27 12:31:08,861][20381] Updated weights for policy 1, policy_version 990701 (0.0005) [2023-12-27 12:31:08,908][20381] Updated weights for policy 1, policy_version 990711 (0.0005) [2023-12-27 12:31:09,058][20347] Updated weights for policy 0, policy_version 991340 (0.0009) [2023-12-27 12:31:09,119][20347] Updated weights for policy 0, policy_version 991350 (0.0006) [2023-12-27 12:31:09,170][20347] Updated weights for policy 0, policy_version 991360 (0.0005) [2023-12-27 12:31:09,646][20381] Updated weights for policy 1, policy_version 990721 (0.0007) [2023-12-27 12:31:09,701][20381] Updated weights for policy 1, policy_version 990731 (0.0008) [2023-12-27 12:31:09,760][20381] Updated weights for policy 1, policy_version 990741 (0.0008) [2023-12-27 12:31:09,813][20347] Updated weights for policy 0, policy_version 991370 (0.0007) [2023-12-27 12:31:09,880][20347] Updated weights for policy 0, policy_version 991380 (0.0009) [2023-12-27 12:31:09,944][20347] Updated weights for policy 0, policy_version 991390 (0.0009) [2023-12-27 12:31:10,008][20347] Updated weights for policy 0, policy_version 991400 (0.0008) [2023-12-27 12:31:10,482][20381] Updated weights for policy 1, policy_version 990751 (0.0008) [2023-12-27 12:31:10,545][20381] Updated weights for policy 1, policy_version 990761 (0.0008) [2023-12-27 12:31:10,604][20381] Updated weights for policy 1, policy_version 990771 (0.0007) [2023-12-27 12:31:10,626][20347] Updated weights for policy 0, policy_version 991410 (0.0006) [2023-12-27 12:31:10,682][20347] Updated weights for policy 0, policy_version 991420 (0.0005) [2023-12-27 12:31:10,738][20347] Updated weights for policy 0, policy_version 991430 (0.0005) [2023-12-27 12:31:11,397][20381] Updated weights for policy 1, policy_version 990781 (0.0008) [2023-12-27 12:31:11,413][20347] Updated weights for policy 0, policy_version 991440 (0.0008) [2023-12-27 12:31:11,458][20381] Updated weights for policy 1, policy_version 990791 (0.0007) [2023-12-27 12:31:11,475][20347] Updated weights for policy 0, policy_version 991450 (0.0007) [2023-12-27 12:31:11,476][20291] KL-divergence is very high: 117.5450 [2023-12-27 12:31:11,515][20381] Updated weights for policy 1, policy_version 990801 (0.0006) [2023-12-27 12:31:11,523][20291] KL-divergence is very high: 108.6002 [2023-12-27 12:31:11,536][20347] Updated weights for policy 0, policy_version 991460 (0.0007) [2023-12-27 12:31:12,214][20347] Updated weights for policy 0, policy_version 991470 (0.0006) [2023-12-27 12:31:12,250][20381] Updated weights for policy 1, policy_version 990811 (0.0007) [2023-12-27 12:31:12,277][20347] Updated weights for policy 0, policy_version 991480 (0.0007) [2023-12-27 12:31:12,310][20381] Updated weights for policy 1, policy_version 990821 (0.0007) [2023-12-27 12:31:12,339][20347] Updated weights for policy 0, policy_version 991490 (0.0006) [2023-12-27 12:31:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 507535360. Throughput: 0: 10204.2, 1: 10375.5. Samples: 507534508. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:31:12,363][19170] Avg episode reward: [(0, '842.270'), (1, '189.880')] [2023-12-27 12:31:12,374][20381] Updated weights for policy 1, policy_version 990831 (0.0007) [2023-12-27 12:31:13,015][20347] Updated weights for policy 0, policy_version 991500 (0.0008) [2023-12-27 12:31:13,060][20347] Updated weights for policy 0, policy_version 991510 (0.0007) [2023-12-27 12:31:13,094][20381] Updated weights for policy 1, policy_version 990841 (0.0007) [2023-12-27 12:31:13,111][20347] Updated weights for policy 0, policy_version 991520 (0.0007) [2023-12-27 12:31:13,146][20381] Updated weights for policy 1, policy_version 990851 (0.0006) [2023-12-27 12:31:13,207][20381] Updated weights for policy 1, policy_version 990861 (0.0007) [2023-12-27 12:31:13,263][20381] Updated weights for policy 1, policy_version 990871 (0.0008) [2023-12-27 12:31:13,752][20347] Updated weights for policy 0, policy_version 991530 (0.0007) [2023-12-27 12:31:13,816][20347] Updated weights for policy 0, policy_version 991540 (0.0008) [2023-12-27 12:31:13,868][20347] Updated weights for policy 0, policy_version 991550 (0.0007) [2023-12-27 12:31:13,925][20347] Updated weights for policy 0, policy_version 991560 (0.0008) [2023-12-27 12:31:13,989][20381] Updated weights for policy 1, policy_version 990881 (0.0008) [2023-12-27 12:31:14,049][20381] Updated weights for policy 1, policy_version 990891 (0.0008) [2023-12-27 12:31:14,113][20381] Updated weights for policy 1, policy_version 990901 (0.0008) [2023-12-27 12:31:14,616][20347] Updated weights for policy 0, policy_version 991570 (0.0009) [2023-12-27 12:31:14,661][20347] Updated weights for policy 0, policy_version 991580 (0.0005) [2023-12-27 12:31:14,707][20347] Updated weights for policy 0, policy_version 991590 (0.0005) [2023-12-27 12:31:14,818][20381] Updated weights for policy 1, policy_version 990911 (0.0008) [2023-12-27 12:31:14,876][20381] Updated weights for policy 1, policy_version 990921 (0.0007) [2023-12-27 12:31:14,932][20381] Updated weights for policy 1, policy_version 990931 (0.0009) [2023-12-27 12:31:15,322][20347] Updated weights for policy 0, policy_version 991600 (0.0005) [2023-12-27 12:31:15,372][20347] Updated weights for policy 0, policy_version 991610 (0.0007) [2023-12-27 12:31:15,427][20347] Updated weights for policy 0, policy_version 991620 (0.0008) [2023-12-27 12:31:15,663][20381] Updated weights for policy 1, policy_version 990941 (0.0008) [2023-12-27 12:31:15,721][20381] Updated weights for policy 1, policy_version 990951 (0.0008) [2023-12-27 12:31:15,776][20381] Updated weights for policy 1, policy_version 990961 (0.0008) [2023-12-27 12:31:16,116][20347] Updated weights for policy 0, policy_version 991630 (0.0006) [2023-12-27 12:31:16,180][20347] Updated weights for policy 0, policy_version 991640 (0.0007) [2023-12-27 12:31:16,235][20347] Updated weights for policy 0, policy_version 991650 (0.0007) [2023-12-27 12:31:16,459][20381] Updated weights for policy 1, policy_version 990971 (0.0008) [2023-12-27 12:31:16,511][20381] Updated weights for policy 1, policy_version 990981 (0.0008) [2023-12-27 12:31:16,564][20381] Updated weights for policy 1, policy_version 990991 (0.0007) [2023-12-27 12:31:16,839][20347] Updated weights for policy 0, policy_version 991660 (0.0009) [2023-12-27 12:31:16,898][20347] Updated weights for policy 0, policy_version 991670 (0.0008) [2023-12-27 12:31:16,952][20347] Updated weights for policy 0, policy_version 991680 (0.0007) [2023-12-27 12:31:17,315][20381] Updated weights for policy 1, policy_version 991001 (0.0008) [2023-12-27 12:31:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 507641856. Throughput: 0: 10280.3, 1: 10272.6. Samples: 507654492. Policy #0 lag: (min: 29.0, avg: 35.7, max: 61.0) [2023-12-27 12:31:17,363][19170] Avg episode reward: [(0, '775.460'), (1, '380.710')] [2023-12-27 12:31:17,370][20381] Updated weights for policy 1, policy_version 991011 (0.0008) [2023-12-27 12:31:17,433][20381] Updated weights for policy 1, policy_version 991021 (0.0008) [2023-12-27 12:31:17,496][20381] Updated weights for policy 1, policy_version 991031 (0.0008) [2023-12-27 12:31:17,528][20347] Updated weights for policy 0, policy_version 991690 (0.0006) [2023-12-27 12:31:17,576][20347] Updated weights for policy 0, policy_version 991700 (0.0007) [2023-12-27 12:31:17,628][20347] Updated weights for policy 0, policy_version 991710 (0.0008) [2023-12-27 12:31:17,679][20347] Updated weights for policy 0, policy_version 991720 (0.0008) [2023-12-27 12:31:18,192][20381] Updated weights for policy 1, policy_version 991041 (0.0008) [2023-12-27 12:31:18,250][20381] Updated weights for policy 1, policy_version 991051 (0.0009) [2023-12-27 12:31:18,296][20381] Updated weights for policy 1, policy_version 991061 (0.0008) [2023-12-27 12:31:18,341][20347] Updated weights for policy 0, policy_version 991730 (0.0007) [2023-12-27 12:31:18,404][20347] Updated weights for policy 0, policy_version 991740 (0.0008) [2023-12-27 12:31:18,459][20347] Updated weights for policy 0, policy_version 991750 (0.0007) [2023-12-27 12:31:18,995][20381] Updated weights for policy 1, policy_version 991071 (0.0010) [2023-12-27 12:31:19,056][20381] Updated weights for policy 1, policy_version 991081 (0.0010) [2023-12-27 12:31:19,120][20381] Updated weights for policy 1, policy_version 991091 (0.0010) [2023-12-27 12:31:19,194][20347] Updated weights for policy 0, policy_version 991760 (0.0006) [2023-12-27 12:31:19,260][20347] Updated weights for policy 0, policy_version 991770 (0.0007) [2023-12-27 12:31:19,315][20347] Updated weights for policy 0, policy_version 991780 (0.0008) [2023-12-27 12:31:19,769][20381] Updated weights for policy 1, policy_version 991101 (0.0011) [2023-12-27 12:31:19,831][20381] Updated weights for policy 1, policy_version 991111 (0.0010) [2023-12-27 12:31:19,898][20381] Updated weights for policy 1, policy_version 991121 (0.0010) [2023-12-27 12:31:19,971][20347] Updated weights for policy 0, policy_version 991790 (0.0007) [2023-12-27 12:31:20,033][20347] Updated weights for policy 0, policy_version 991800 (0.0007) [2023-12-27 12:31:20,097][20347] Updated weights for policy 0, policy_version 991810 (0.0007) [2023-12-27 12:31:20,663][20381] Updated weights for policy 1, policy_version 991131 (0.0008) [2023-12-27 12:31:20,728][20381] Updated weights for policy 1, policy_version 991141 (0.0008) [2023-12-27 12:31:20,790][20381] Updated weights for policy 1, policy_version 991151 (0.0008) [2023-12-27 12:31:20,801][20347] Updated weights for policy 0, policy_version 991820 (0.0007) [2023-12-27 12:31:20,861][20347] Updated weights for policy 0, policy_version 991830 (0.0006) [2023-12-27 12:31:20,926][20347] Updated weights for policy 0, policy_version 991840 (0.0008) [2023-12-27 12:31:21,572][20381] Updated weights for policy 1, policy_version 991161 (0.0007) [2023-12-27 12:31:21,630][20381] Updated weights for policy 1, policy_version 991171 (0.0011) [2023-12-27 12:31:21,687][20381] Updated weights for policy 1, policy_version 991181 (0.0011) [2023-12-27 12:31:21,715][20347] Updated weights for policy 0, policy_version 991850 (0.0008) [2023-12-27 12:31:21,748][20381] Updated weights for policy 1, policy_version 991191 (0.0011) [2023-12-27 12:31:21,785][20347] Updated weights for policy 0, policy_version 991860 (0.0008) [2023-12-27 12:31:21,827][20291] KL-divergence is very high: 184.7016 [2023-12-27 12:31:21,834][20347] Updated weights for policy 0, policy_version 991870 (0.0008) [2023-12-27 12:31:21,879][20347] Updated weights for policy 0, policy_version 991880 (0.0008) [2023-12-27 12:31:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20438.4). Total num frames: 507740160. Throughput: 0: 10335.4, 1: 10237.8. Samples: 507715608. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:22,363][19170] Avg episode reward: [(0, '747.180'), (1, '371.200')] [2023-12-27 12:31:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000991192_253771776.pth... [2023-12-27 12:31:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000991880_253968384.pth... [2023-12-27 12:31:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000990008_253468672.pth [2023-12-27 12:31:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000990664_253657088.pth [2023-12-27 12:31:22,472][20381] Updated weights for policy 1, policy_version 991201 (0.0010) [2023-12-27 12:31:22,524][20381] Updated weights for policy 1, policy_version 991211 (0.0010) [2023-12-27 12:31:22,581][20381] Updated weights for policy 1, policy_version 991221 (0.0011) [2023-12-27 12:31:22,612][20347] Updated weights for policy 0, policy_version 991890 (0.0010) [2023-12-27 12:31:22,671][20347] Updated weights for policy 0, policy_version 991900 (0.0010) [2023-12-27 12:31:22,723][20347] Updated weights for policy 0, policy_version 991910 (0.0008) [2023-12-27 12:31:23,229][20381] Updated weights for policy 1, policy_version 991231 (0.0010) [2023-12-27 12:31:23,292][20381] Updated weights for policy 1, policy_version 991241 (0.0011) [2023-12-27 12:31:23,356][20381] Updated weights for policy 1, policy_version 991251 (0.0011) [2023-12-27 12:31:23,423][20347] Updated weights for policy 0, policy_version 991920 (0.0008) [2023-12-27 12:31:23,469][20347] Updated weights for policy 0, policy_version 991930 (0.0007) [2023-12-27 12:31:23,522][20347] Updated weights for policy 0, policy_version 991940 (0.0007) [2023-12-27 12:31:23,979][20381] Updated weights for policy 1, policy_version 991261 (0.0008) [2023-12-27 12:31:24,041][20381] Updated weights for policy 1, policy_version 991271 (0.0005) [2023-12-27 12:31:24,093][20381] Updated weights for policy 1, policy_version 991281 (0.0010) [2023-12-27 12:31:24,201][20347] Updated weights for policy 0, policy_version 991950 (0.0007) [2023-12-27 12:31:24,245][20347] Updated weights for policy 0, policy_version 991960 (0.0007) [2023-12-27 12:31:24,296][20347] Updated weights for policy 0, policy_version 991970 (0.0007) [2023-12-27 12:31:24,759][20381] Updated weights for policy 1, policy_version 991291 (0.0009) [2023-12-27 12:31:24,821][20381] Updated weights for policy 1, policy_version 991301 (0.0005) [2023-12-27 12:31:24,872][20381] Updated weights for policy 1, policy_version 991311 (0.0005) [2023-12-27 12:31:25,005][20347] Updated weights for policy 0, policy_version 991980 (0.0006) [2023-12-27 12:31:25,034][20291] KL-divergence is very high: 259.6153 [2023-12-27 12:31:25,039][20291] KL-divergence is very high: 202.0497 [2023-12-27 12:31:25,051][20347] Updated weights for policy 0, policy_version 991990 (0.0005) [2023-12-27 12:31:25,070][20291] KL-divergence is very high: 392.7199 [2023-12-27 12:31:25,075][20291] KL-divergence is very high: 371.7933 [2023-12-27 12:31:25,080][20291] KL-divergence is very high: 184.2302 [2023-12-27 12:31:25,099][20347] Updated weights for policy 0, policy_version 992000 (0.0005) [2023-12-27 12:31:25,109][20291] KL-divergence is very high: 104.2327 [2023-12-27 12:31:25,114][20291] KL-divergence is very high: 130.7953 [2023-12-27 12:31:25,119][20291] KL-divergence is very high: 108.6368 [2023-12-27 12:31:25,427][20381] Updated weights for policy 1, policy_version 991321 (0.0005) [2023-12-27 12:31:25,487][20381] Updated weights for policy 1, policy_version 991331 (0.0007) [2023-12-27 12:31:25,545][20381] Updated weights for policy 1, policy_version 991341 (0.0007) [2023-12-27 12:31:25,602][20381] Updated weights for policy 1, policy_version 991351 (0.0007) [2023-12-27 12:31:25,759][20291] KL-divergence is very high: 139.4456 [2023-12-27 12:31:25,766][20347] Updated weights for policy 0, policy_version 992010 (0.0009) [2023-12-27 12:31:25,772][20291] KL-divergence is very high: 487.2627 [2023-12-27 12:31:25,807][20291] KL-divergence is very high: 222.1240 [2023-12-27 12:31:25,818][20291] KL-divergence is very high: 566.3007 [2023-12-27 12:31:25,824][20347] Updated weights for policy 0, policy_version 992020 (0.0008) [2023-12-27 12:31:25,854][20291] KL-divergence is very high: 246.5687 [2023-12-27 12:31:25,867][20347] Updated weights for policy 0, policy_version 992030 (0.0007) [2023-12-27 12:31:25,882][20291] KL-divergence is very high: 235.3942 [2023-12-27 12:31:25,891][20291] KL-divergence is very high: 181.7186 [2023-12-27 12:31:25,915][20347] Updated weights for policy 0, policy_version 992040 (0.0007) [2023-12-27 12:31:26,213][20381] Updated weights for policy 1, policy_version 991361 (0.0006) [2023-12-27 12:31:26,273][20381] Updated weights for policy 1, policy_version 991371 (0.0006) [2023-12-27 12:31:26,329][20381] Updated weights for policy 1, policy_version 991381 (0.0007) [2023-12-27 12:31:26,553][20347] Updated weights for policy 0, policy_version 992050 (0.0005) [2023-12-27 12:31:26,598][20347] Updated weights for policy 0, policy_version 992060 (0.0009) [2023-12-27 12:31:26,643][20347] Updated weights for policy 0, policy_version 992070 (0.0010) [2023-12-27 12:31:26,956][20381] Updated weights for policy 1, policy_version 991391 (0.0006) [2023-12-27 12:31:27,006][20381] Updated weights for policy 1, policy_version 991401 (0.0007) [2023-12-27 12:31:27,053][20381] Updated weights for policy 1, policy_version 991411 (0.0008) [2023-12-27 12:31:27,278][20347] Updated weights for policy 0, policy_version 992080 (0.0007) [2023-12-27 12:31:27,340][20347] Updated weights for policy 0, policy_version 992090 (0.0006) [2023-12-27 12:31:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 507846656. Throughput: 0: 10288.6, 1: 10258.0. Samples: 507836792. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:27,363][19170] Avg episode reward: [(0, '776.020'), (1, '486.350')] [2023-12-27 12:31:27,396][20347] Updated weights for policy 0, policy_version 992100 (0.0005) [2023-12-27 12:31:27,758][20381] Updated weights for policy 1, policy_version 991421 (0.0008) [2023-12-27 12:31:27,818][20381] Updated weights for policy 1, policy_version 991431 (0.0009) [2023-12-27 12:31:27,872][20381] Updated weights for policy 1, policy_version 991441 (0.0007) [2023-12-27 12:31:27,952][20347] Updated weights for policy 0, policy_version 992110 (0.0006) [2023-12-27 12:31:27,995][20347] Updated weights for policy 0, policy_version 992120 (0.0008) [2023-12-27 12:31:28,047][20347] Updated weights for policy 0, policy_version 992130 (0.0008) [2023-12-27 12:31:28,541][20381] Updated weights for policy 1, policy_version 991451 (0.0005) [2023-12-27 12:31:28,600][20381] Updated weights for policy 1, policy_version 991461 (0.0005) [2023-12-27 12:31:28,653][20381] Updated weights for policy 1, policy_version 991471 (0.0005) [2023-12-27 12:31:28,738][20347] Updated weights for policy 0, policy_version 992140 (0.0008) [2023-12-27 12:31:28,785][20347] Updated weights for policy 0, policy_version 992150 (0.0008) [2023-12-27 12:31:28,831][20347] Updated weights for policy 0, policy_version 992160 (0.0008) [2023-12-27 12:31:29,294][20381] Updated weights for policy 1, policy_version 991481 (0.0006) [2023-12-27 12:31:29,361][20381] Updated weights for policy 1, policy_version 991491 (0.0008) [2023-12-27 12:31:29,436][20381] Updated weights for policy 1, policy_version 991501 (0.0007) [2023-12-27 12:31:29,492][20381] Updated weights for policy 1, policy_version 991511 (0.0007) [2023-12-27 12:31:29,575][20347] Updated weights for policy 0, policy_version 992170 (0.0008) [2023-12-27 12:31:29,630][20347] Updated weights for policy 0, policy_version 992180 (0.0010) [2023-12-27 12:31:29,688][20347] Updated weights for policy 0, policy_version 992190 (0.0010) [2023-12-27 12:31:29,750][20347] Updated weights for policy 0, policy_version 992200 (0.0010) [2023-12-27 12:31:30,159][20381] Updated weights for policy 1, policy_version 991521 (0.0007) [2023-12-27 12:31:30,222][20381] Updated weights for policy 1, policy_version 991531 (0.0005) [2023-12-27 12:31:30,283][20381] Updated weights for policy 1, policy_version 991541 (0.0011) [2023-12-27 12:31:30,430][20347] Updated weights for policy 0, policy_version 992210 (0.0005) [2023-12-27 12:31:30,476][20347] Updated weights for policy 0, policy_version 992220 (0.0005) [2023-12-27 12:31:30,536][20347] Updated weights for policy 0, policy_version 992230 (0.0007) [2023-12-27 12:31:30,855][20381] Updated weights for policy 1, policy_version 991551 (0.0008) [2023-12-27 12:31:30,910][20381] Updated weights for policy 1, policy_version 991561 (0.0011) [2023-12-27 12:31:30,966][20381] Updated weights for policy 1, policy_version 991571 (0.0009) [2023-12-27 12:31:31,202][20347] Updated weights for policy 0, policy_version 992240 (0.0009) [2023-12-27 12:31:31,250][20347] Updated weights for policy 0, policy_version 992250 (0.0008) [2023-12-27 12:31:31,304][20347] Updated weights for policy 0, policy_version 992260 (0.0008) [2023-12-27 12:31:31,646][20381] Updated weights for policy 1, policy_version 991581 (0.0008) [2023-12-27 12:31:31,705][20381] Updated weights for policy 1, policy_version 991591 (0.0011) [2023-12-27 12:31:31,773][20381] Updated weights for policy 1, policy_version 991601 (0.0011) [2023-12-27 12:31:32,017][20347] Updated weights for policy 0, policy_version 992270 (0.0007) [2023-12-27 12:31:32,072][20347] Updated weights for policy 0, policy_version 992280 (0.0008) [2023-12-27 12:31:32,133][20347] Updated weights for policy 0, policy_version 992290 (0.0006) [2023-12-27 12:31:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 507953152. Throughput: 0: 10359.4, 1: 10225.0. Samples: 507963060. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:32,363][19170] Avg episode reward: [(0, '795.480'), (1, '75.870')] [2023-12-27 12:31:32,505][20381] Updated weights for policy 1, policy_version 991611 (0.0010) [2023-12-27 12:31:32,556][20381] Updated weights for policy 1, policy_version 991621 (0.0008) [2023-12-27 12:31:32,604][20381] Updated weights for policy 1, policy_version 991631 (0.0008) [2023-12-27 12:31:32,782][20347] Updated weights for policy 0, policy_version 992300 (0.0006) [2023-12-27 12:31:32,829][20347] Updated weights for policy 0, policy_version 992310 (0.0008) [2023-12-27 12:31:32,882][20347] Updated weights for policy 0, policy_version 992320 (0.0006) [2023-12-27 12:31:33,305][20381] Updated weights for policy 1, policy_version 991641 (0.0008) [2023-12-27 12:31:33,353][20381] Updated weights for policy 1, policy_version 991651 (0.0005) [2023-12-27 12:31:33,405][20381] Updated weights for policy 1, policy_version 991661 (0.0005) [2023-12-27 12:31:33,454][20381] Updated weights for policy 1, policy_version 991671 (0.0005) [2023-12-27 12:31:33,607][20347] Updated weights for policy 0, policy_version 992330 (0.0007) [2023-12-27 12:31:33,620][20291] KL-divergence is very high: 108.9807 [2023-12-27 12:31:33,624][20291] KL-divergence is very high: 136.3324 [2023-12-27 12:31:33,650][20291] KL-divergence is very high: 103.9938 [2023-12-27 12:31:33,657][20347] Updated weights for policy 0, policy_version 992340 (0.0008) [2023-12-27 12:31:33,704][20347] Updated weights for policy 0, policy_version 992350 (0.0007) [2023-12-27 12:31:33,767][20347] Updated weights for policy 0, policy_version 992360 (0.0008) [2023-12-27 12:31:34,076][20381] Updated weights for policy 1, policy_version 991681 (0.0007) [2023-12-27 12:31:34,132][20381] Updated weights for policy 1, policy_version 991691 (0.0007) [2023-12-27 12:31:34,193][20381] Updated weights for policy 1, policy_version 991701 (0.0008) [2023-12-27 12:31:34,530][20347] Updated weights for policy 0, policy_version 992370 (0.0007) [2023-12-27 12:31:34,587][20347] Updated weights for policy 0, policy_version 992380 (0.0008) [2023-12-27 12:31:34,638][20347] Updated weights for policy 0, policy_version 992390 (0.0006) [2023-12-27 12:31:34,872][20381] Updated weights for policy 1, policy_version 991711 (0.0006) [2023-12-27 12:31:34,927][20381] Updated weights for policy 1, policy_version 991721 (0.0005) [2023-12-27 12:31:34,989][20381] Updated weights for policy 1, policy_version 991731 (0.0006) [2023-12-27 12:31:35,284][20347] Updated weights for policy 0, policy_version 992400 (0.0009) [2023-12-27 12:31:35,330][20347] Updated weights for policy 0, policy_version 992410 (0.0010) [2023-12-27 12:31:35,375][20347] Updated weights for policy 0, policy_version 992420 (0.0009) [2023-12-27 12:31:35,575][20381] Updated weights for policy 1, policy_version 991741 (0.0007) [2023-12-27 12:31:35,632][20381] Updated weights for policy 1, policy_version 991751 (0.0007) [2023-12-27 12:31:35,683][20381] Updated weights for policy 1, policy_version 991761 (0.0008) [2023-12-27 12:31:36,020][20347] Updated weights for policy 0, policy_version 992430 (0.0007) [2023-12-27 12:31:36,070][20347] Updated weights for policy 0, policy_version 992440 (0.0005) [2023-12-27 12:31:36,137][20347] Updated weights for policy 0, policy_version 992450 (0.0007) [2023-12-27 12:31:36,241][20381] Updated weights for policy 1, policy_version 991771 (0.0007) [2023-12-27 12:31:36,302][20381] Updated weights for policy 1, policy_version 991781 (0.0006) [2023-12-27 12:31:36,364][20381] Updated weights for policy 1, policy_version 991791 (0.0007) [2023-12-27 12:31:36,792][20347] Updated weights for policy 0, policy_version 992460 (0.0008) [2023-12-27 12:31:36,855][20347] Updated weights for policy 0, policy_version 992470 (0.0011) [2023-12-27 12:31:36,911][20347] Updated weights for policy 0, policy_version 992480 (0.0010) [2023-12-27 12:31:37,009][20381] Updated weights for policy 1, policy_version 991801 (0.0008) [2023-12-27 12:31:37,065][20381] Updated weights for policy 1, policy_version 991811 (0.0006) [2023-12-27 12:31:37,120][20381] Updated weights for policy 1, policy_version 991821 (0.0007) [2023-12-27 12:31:37,179][20381] Updated weights for policy 1, policy_version 991831 (0.0008) [2023-12-27 12:31:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 508059648. Throughput: 0: 10335.3, 1: 10255.9. Samples: 508024352. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:37,363][19170] Avg episode reward: [(0, '795.040'), (1, '226.890')] [2023-12-27 12:31:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000992488_254124032.pth... [2023-12-27 12:31:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000991832_253935616.pth... [2023-12-27 12:31:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000990648_253632512.pth [2023-12-27 12:31:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000991272_253812736.pth [2023-12-27 12:31:37,574][20347] Updated weights for policy 0, policy_version 992490 (0.0011) [2023-12-27 12:31:37,630][20347] Updated weights for policy 0, policy_version 992500 (0.0010) [2023-12-27 12:31:37,688][20347] Updated weights for policy 0, policy_version 992510 (0.0011) [2023-12-27 12:31:37,738][20347] Updated weights for policy 0, policy_version 992520 (0.0010) [2023-12-27 12:31:37,839][20381] Updated weights for policy 1, policy_version 991841 (0.0010) [2023-12-27 12:31:37,891][20381] Updated weights for policy 1, policy_version 991851 (0.0010) [2023-12-27 12:31:37,938][20381] Updated weights for policy 1, policy_version 991861 (0.0010) [2023-12-27 12:31:38,431][20347] Updated weights for policy 0, policy_version 992530 (0.0010) [2023-12-27 12:31:38,493][20347] Updated weights for policy 0, policy_version 992540 (0.0011) [2023-12-27 12:31:38,551][20347] Updated weights for policy 0, policy_version 992550 (0.0007) [2023-12-27 12:31:38,649][20381] Updated weights for policy 1, policy_version 991871 (0.0009) [2023-12-27 12:31:38,699][20381] Updated weights for policy 1, policy_version 991881 (0.0008) [2023-12-27 12:31:38,761][20381] Updated weights for policy 1, policy_version 991891 (0.0008) [2023-12-27 12:31:39,139][20347] Updated weights for policy 0, policy_version 992560 (0.0005) [2023-12-27 12:31:39,186][20347] Updated weights for policy 0, policy_version 992570 (0.0005) [2023-12-27 12:31:39,250][20347] Updated weights for policy 0, policy_version 992580 (0.0007) [2023-12-27 12:31:39,468][20381] Updated weights for policy 1, policy_version 991901 (0.0008) [2023-12-27 12:31:39,525][20381] Updated weights for policy 1, policy_version 991911 (0.0008) [2023-12-27 12:31:39,584][20381] Updated weights for policy 1, policy_version 991921 (0.0008) [2023-12-27 12:31:39,874][20347] Updated weights for policy 0, policy_version 992590 (0.0007) [2023-12-27 12:31:39,930][20347] Updated weights for policy 0, policy_version 992600 (0.0007) [2023-12-27 12:31:39,996][20347] Updated weights for policy 0, policy_version 992610 (0.0006) [2023-12-27 12:31:40,325][20381] Updated weights for policy 1, policy_version 991931 (0.0007) [2023-12-27 12:31:40,380][20381] Updated weights for policy 1, policy_version 991941 (0.0008) [2023-12-27 12:31:40,431][20381] Updated weights for policy 1, policy_version 991951 (0.0010) [2023-12-27 12:31:40,697][20347] Updated weights for policy 0, policy_version 992620 (0.0007) [2023-12-27 12:31:40,752][20347] Updated weights for policy 0, policy_version 992630 (0.0006) [2023-12-27 12:31:40,807][20347] Updated weights for policy 0, policy_version 992640 (0.0005) [2023-12-27 12:31:41,100][20381] Updated weights for policy 1, policy_version 991961 (0.0010) [2023-12-27 12:31:41,168][20381] Updated weights for policy 1, policy_version 991971 (0.0007) [2023-12-27 12:31:41,225][20381] Updated weights for policy 1, policy_version 991981 (0.0008) [2023-12-27 12:31:41,290][20381] Updated weights for policy 1, policy_version 991991 (0.0007) [2023-12-27 12:31:41,482][20347] Updated weights for policy 0, policy_version 992650 (0.0005) [2023-12-27 12:31:41,545][20347] Updated weights for policy 0, policy_version 992660 (0.0006) [2023-12-27 12:31:41,608][20347] Updated weights for policy 0, policy_version 992670 (0.0007) [2023-12-27 12:31:41,679][20347] Updated weights for policy 0, policy_version 992680 (0.0008) [2023-12-27 12:31:42,026][20381] Updated weights for policy 1, policy_version 992001 (0.0007) [2023-12-27 12:31:42,092][20381] Updated weights for policy 1, policy_version 992011 (0.0010) [2023-12-27 12:31:42,152][20381] Updated weights for policy 1, policy_version 992021 (0.0011) [2023-12-27 12:31:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 508157952. Throughput: 0: 10371.6, 1: 10195.2. Samples: 508150188. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:42,363][19170] Avg episode reward: [(0, '844.370'), (1, '39.960')] [2023-12-27 12:31:42,382][20347] Updated weights for policy 0, policy_version 992690 (0.0008) [2023-12-27 12:31:42,441][20347] Updated weights for policy 0, policy_version 992700 (0.0006) [2023-12-27 12:31:42,499][20347] Updated weights for policy 0, policy_version 992710 (0.0007) [2023-12-27 12:31:42,878][20381] Updated weights for policy 1, policy_version 992031 (0.0010) [2023-12-27 12:31:42,933][20381] Updated weights for policy 1, policy_version 992041 (0.0010) [2023-12-27 12:31:42,989][20381] Updated weights for policy 1, policy_version 992051 (0.0011) [2023-12-27 12:31:43,184][20347] Updated weights for policy 0, policy_version 992720 (0.0006) [2023-12-27 12:31:43,227][20347] Updated weights for policy 0, policy_version 992730 (0.0006) [2023-12-27 12:31:43,273][20347] Updated weights for policy 0, policy_version 992740 (0.0005) [2023-12-27 12:31:43,682][20381] Updated weights for policy 1, policy_version 992061 (0.0010) [2023-12-27 12:31:43,737][20381] Updated weights for policy 1, policy_version 992071 (0.0010) [2023-12-27 12:31:43,796][20381] Updated weights for policy 1, policy_version 992081 (0.0010) [2023-12-27 12:31:43,909][20347] Updated weights for policy 0, policy_version 992750 (0.0007) [2023-12-27 12:31:43,961][20347] Updated weights for policy 0, policy_version 992760 (0.0007) [2023-12-27 12:31:44,021][20347] Updated weights for policy 0, policy_version 992770 (0.0007) [2023-12-27 12:31:44,488][20381] Updated weights for policy 1, policy_version 992091 (0.0010) [2023-12-27 12:31:44,545][20381] Updated weights for policy 1, policy_version 992101 (0.0011) [2023-12-27 12:31:44,602][20381] Updated weights for policy 1, policy_version 992111 (0.0011) [2023-12-27 12:31:44,727][20347] Updated weights for policy 0, policy_version 992780 (0.0007) [2023-12-27 12:31:44,790][20347] Updated weights for policy 0, policy_version 992790 (0.0008) [2023-12-27 12:31:44,850][20347] Updated weights for policy 0, policy_version 992800 (0.0007) [2023-12-27 12:31:45,326][20381] Updated weights for policy 1, policy_version 992121 (0.0011) [2023-12-27 12:31:45,384][20381] Updated weights for policy 1, policy_version 992131 (0.0011) [2023-12-27 12:31:45,429][20381] Updated weights for policy 1, policy_version 992141 (0.0010) [2023-12-27 12:31:45,481][20381] Updated weights for policy 1, policy_version 992151 (0.0010) [2023-12-27 12:31:45,636][20347] Updated weights for policy 0, policy_version 992810 (0.0007) [2023-12-27 12:31:45,688][20347] Updated weights for policy 0, policy_version 992820 (0.0005) [2023-12-27 12:31:45,749][20347] Updated weights for policy 0, policy_version 992830 (0.0006) [2023-12-27 12:31:45,803][20347] Updated weights for policy 0, policy_version 992840 (0.0006) [2023-12-27 12:31:46,185][20381] Updated weights for policy 1, policy_version 992161 (0.0009) [2023-12-27 12:31:46,230][20381] Updated weights for policy 1, policy_version 992171 (0.0007) [2023-12-27 12:31:46,276][20381] Updated weights for policy 1, policy_version 992181 (0.0008) [2023-12-27 12:31:46,404][20347] Updated weights for policy 0, policy_version 992850 (0.0008) [2023-12-27 12:31:46,445][20291] KL-divergence is very high: 108.7797 [2023-12-27 12:31:46,456][20291] KL-divergence is very high: 112.5626 [2023-12-27 12:31:46,458][20347] Updated weights for policy 0, policy_version 992860 (0.0010) [2023-12-27 12:31:46,468][20291] KL-divergence is very high: 133.5374 [2023-12-27 12:31:46,479][20291] KL-divergence is very high: 127.9608 [2023-12-27 12:31:46,511][20347] Updated weights for policy 0, policy_version 992870 (0.0009) [2023-12-27 12:31:47,006][20381] Updated weights for policy 1, policy_version 992191 (0.0008) [2023-12-27 12:31:47,068][20381] Updated weights for policy 1, policy_version 992201 (0.0008) [2023-12-27 12:31:47,118][20381] Updated weights for policy 1, policy_version 992211 (0.0007) [2023-12-27 12:31:47,172][20347] Updated weights for policy 0, policy_version 992880 (0.0009) [2023-12-27 12:31:47,230][20347] Updated weights for policy 0, policy_version 992890 (0.0010) [2023-12-27 12:31:47,288][20347] Updated weights for policy 0, policy_version 992900 (0.0011) [2023-12-27 12:31:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 508264448. Throughput: 0: 10350.4, 1: 10085.4. Samples: 508269576. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:47,363][19170] Avg episode reward: [(0, '765.760'), (1, '29.630')] [2023-12-27 12:31:47,801][20381] Updated weights for policy 1, policy_version 992221 (0.0008) [2023-12-27 12:31:47,860][20381] Updated weights for policy 1, policy_version 992231 (0.0009) [2023-12-27 12:31:47,911][20347] Updated weights for policy 0, policy_version 992910 (0.0010) [2023-12-27 12:31:47,912][20381] Updated weights for policy 1, policy_version 992241 (0.0007) [2023-12-27 12:31:47,960][20347] Updated weights for policy 0, policy_version 992920 (0.0010) [2023-12-27 12:31:48,008][20347] Updated weights for policy 0, policy_version 992930 (0.0010) [2023-12-27 12:31:48,575][20381] Updated weights for policy 1, policy_version 992251 (0.0008) [2023-12-27 12:31:48,624][20347] Updated weights for policy 0, policy_version 992940 (0.0007) [2023-12-27 12:31:48,628][20381] Updated weights for policy 1, policy_version 992261 (0.0006) [2023-12-27 12:31:48,676][20381] Updated weights for policy 1, policy_version 992271 (0.0005) [2023-12-27 12:31:48,680][20347] Updated weights for policy 0, policy_version 992950 (0.0006) [2023-12-27 12:31:48,739][20347] Updated weights for policy 0, policy_version 992960 (0.0006) [2023-12-27 12:31:49,371][20381] Updated weights for policy 1, policy_version 992281 (0.0006) [2023-12-27 12:31:49,409][20347] Updated weights for policy 0, policy_version 992970 (0.0010) [2023-12-27 12:31:49,424][20381] Updated weights for policy 1, policy_version 992291 (0.0009) [2023-12-27 12:31:49,458][20347] Updated weights for policy 0, policy_version 992980 (0.0009) [2023-12-27 12:31:49,473][20381] Updated weights for policy 1, policy_version 992301 (0.0009) [2023-12-27 12:31:49,515][20347] Updated weights for policy 0, policy_version 992990 (0.0009) [2023-12-27 12:31:49,533][20381] Updated weights for policy 1, policy_version 992311 (0.0009) [2023-12-27 12:31:49,571][20347] Updated weights for policy 0, policy_version 993000 (0.0009) [2023-12-27 12:31:50,292][20381] Updated weights for policy 1, policy_version 992321 (0.0006) [2023-12-27 12:31:50,344][20347] Updated weights for policy 0, policy_version 993010 (0.0007) [2023-12-27 12:31:50,355][20381] Updated weights for policy 1, policy_version 992331 (0.0006) [2023-12-27 12:31:50,402][20347] Updated weights for policy 0, policy_version 993020 (0.0007) [2023-12-27 12:31:50,415][20381] Updated weights for policy 1, policy_version 992341 (0.0006) [2023-12-27 12:31:50,456][20347] Updated weights for policy 0, policy_version 993030 (0.0007) [2023-12-27 12:31:51,042][20381] Updated weights for policy 1, policy_version 992351 (0.0007) [2023-12-27 12:31:51,103][20381] Updated weights for policy 1, policy_version 992361 (0.0008) [2023-12-27 12:31:51,170][20381] Updated weights for policy 1, policy_version 992371 (0.0008) [2023-12-27 12:31:51,219][20347] Updated weights for policy 0, policy_version 993040 (0.0006) [2023-12-27 12:31:51,286][20347] Updated weights for policy 0, policy_version 993050 (0.0007) [2023-12-27 12:31:51,350][20347] Updated weights for policy 0, policy_version 993060 (0.0008) [2023-12-27 12:31:51,880][20381] Updated weights for policy 1, policy_version 992381 (0.0007) [2023-12-27 12:31:51,938][20381] Updated weights for policy 1, policy_version 992391 (0.0007) [2023-12-27 12:31:51,999][20381] Updated weights for policy 1, policy_version 992401 (0.0008) [2023-12-27 12:31:52,063][20347] Updated weights for policy 0, policy_version 993070 (0.0008) [2023-12-27 12:31:52,128][20347] Updated weights for policy 0, policy_version 993080 (0.0008) [2023-12-27 12:31:52,194][20347] Updated weights for policy 0, policy_version 993090 (0.0007) [2023-12-27 12:31:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 508362752. Throughput: 0: 10335.4, 1: 10097.3. Samples: 508331200. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:52,363][19170] Avg episode reward: [(0, '756.870'), (1, '30.370')] [2023-12-27 12:31:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000992408_254083072.pth... [2023-12-27 12:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000993096_254279680.pth... [2023-12-27 12:31:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000991192_253771776.pth [2023-12-27 12:31:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000991880_253968384.pth [2023-12-27 12:31:52,704][20381] Updated weights for policy 1, policy_version 992411 (0.0009) [2023-12-27 12:31:52,764][20381] Updated weights for policy 1, policy_version 992421 (0.0008) [2023-12-27 12:31:52,816][20381] Updated weights for policy 1, policy_version 992431 (0.0007) [2023-12-27 12:31:52,858][20347] Updated weights for policy 0, policy_version 993100 (0.0006) [2023-12-27 12:31:52,916][20347] Updated weights for policy 0, policy_version 993110 (0.0007) [2023-12-27 12:31:52,969][20347] Updated weights for policy 0, policy_version 993120 (0.0008) [2023-12-27 12:31:53,459][20381] Updated weights for policy 1, policy_version 992441 (0.0006) [2023-12-27 12:31:53,510][20381] Updated weights for policy 1, policy_version 992451 (0.0007) [2023-12-27 12:31:53,554][20381] Updated weights for policy 1, policy_version 992461 (0.0008) [2023-12-27 12:31:53,611][20381] Updated weights for policy 1, policy_version 992471 (0.0007) [2023-12-27 12:31:53,619][20347] Updated weights for policy 0, policy_version 993130 (0.0008) [2023-12-27 12:31:53,674][20347] Updated weights for policy 0, policy_version 993140 (0.0010) [2023-12-27 12:31:53,722][20347] Updated weights for policy 0, policy_version 993150 (0.0010) [2023-12-27 12:31:53,775][20347] Updated weights for policy 0, policy_version 993160 (0.0009) [2023-12-27 12:31:54,262][20381] Updated weights for policy 1, policy_version 992481 (0.0007) [2023-12-27 12:31:54,318][20381] Updated weights for policy 1, policy_version 992491 (0.0008) [2023-12-27 12:31:54,367][20381] Updated weights for policy 1, policy_version 992501 (0.0008) [2023-12-27 12:31:54,453][20347] Updated weights for policy 0, policy_version 993170 (0.0007) [2023-12-27 12:31:54,505][20347] Updated weights for policy 0, policy_version 993180 (0.0007) [2023-12-27 12:31:54,555][20347] Updated weights for policy 0, policy_version 993190 (0.0007) [2023-12-27 12:31:55,093][20381] Updated weights for policy 1, policy_version 992511 (0.0010) [2023-12-27 12:31:55,153][20381] Updated weights for policy 1, policy_version 992521 (0.0011) [2023-12-27 12:31:55,191][20347] Updated weights for policy 0, policy_version 993200 (0.0007) [2023-12-27 12:31:55,213][20381] Updated weights for policy 1, policy_version 992531 (0.0011) [2023-12-27 12:31:55,240][20347] Updated weights for policy 0, policy_version 993210 (0.0006) [2023-12-27 12:31:55,295][20347] Updated weights for policy 0, policy_version 993220 (0.0007) [2023-12-27 12:31:55,898][20381] Updated weights for policy 1, policy_version 992541 (0.0011) [2023-12-27 12:31:55,920][20347] Updated weights for policy 0, policy_version 993230 (0.0005) [2023-12-27 12:31:55,951][20381] Updated weights for policy 1, policy_version 992551 (0.0010) [2023-12-27 12:31:55,966][20347] Updated weights for policy 0, policy_version 993240 (0.0005) [2023-12-27 12:31:55,999][20381] Updated weights for policy 1, policy_version 992561 (0.0010) [2023-12-27 12:31:56,020][20347] Updated weights for policy 0, policy_version 993250 (0.0006) [2023-12-27 12:31:56,678][20347] Updated weights for policy 0, policy_version 993260 (0.0008) [2023-12-27 12:31:56,701][20381] Updated weights for policy 1, policy_version 992571 (0.0010) [2023-12-27 12:31:56,728][20347] Updated weights for policy 0, policy_version 993270 (0.0007) [2023-12-27 12:31:56,752][20381] Updated weights for policy 1, policy_version 992581 (0.0010) [2023-12-27 12:31:56,776][20347] Updated weights for policy 0, policy_version 993280 (0.0006) [2023-12-27 12:31:56,803][20381] Updated weights for policy 1, policy_version 992591 (0.0010) [2023-12-27 12:31:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 508461056. Throughput: 0: 10296.8, 1: 10119.5. Samples: 508453244. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:31:57,363][19170] Avg episode reward: [(0, '720.310'), (1, '64.740')] [2023-12-27 12:31:57,480][20381] Updated weights for policy 1, policy_version 992601 (0.0010) [2023-12-27 12:31:57,483][20347] Updated weights for policy 0, policy_version 993290 (0.0006) [2023-12-27 12:31:57,532][20347] Updated weights for policy 0, policy_version 993300 (0.0009) [2023-12-27 12:31:57,535][20381] Updated weights for policy 1, policy_version 992611 (0.0010) [2023-12-27 12:31:57,581][20347] Updated weights for policy 0, policy_version 993310 (0.0006) [2023-12-27 12:31:57,583][20381] Updated weights for policy 1, policy_version 992621 (0.0010) [2023-12-27 12:31:57,632][20347] Updated weights for policy 0, policy_version 993320 (0.0006) [2023-12-27 12:31:57,634][20381] Updated weights for policy 1, policy_version 992631 (0.0010) [2023-12-27 12:31:58,300][20347] Updated weights for policy 0, policy_version 993330 (0.0008) [2023-12-27 12:31:58,315][20381] Updated weights for policy 1, policy_version 992641 (0.0007) [2023-12-27 12:31:58,370][20347] Updated weights for policy 0, policy_version 993340 (0.0009) [2023-12-27 12:31:58,382][20381] Updated weights for policy 1, policy_version 992651 (0.0008) [2023-12-27 12:31:58,434][20347] Updated weights for policy 0, policy_version 993350 (0.0008) [2023-12-27 12:31:58,447][20381] Updated weights for policy 1, policy_version 992661 (0.0008) [2023-12-27 12:31:59,210][20347] Updated weights for policy 0, policy_version 993360 (0.0007) [2023-12-27 12:31:59,219][20381] Updated weights for policy 1, policy_version 992671 (0.0007) [2023-12-27 12:31:59,274][20347] Updated weights for policy 0, policy_version 993370 (0.0008) [2023-12-27 12:31:59,277][20381] Updated weights for policy 1, policy_version 992681 (0.0008) [2023-12-27 12:31:59,333][20347] Updated weights for policy 0, policy_version 993380 (0.0008) [2023-12-27 12:31:59,335][20381] Updated weights for policy 1, policy_version 992691 (0.0006) [2023-12-27 12:32:00,007][20381] Updated weights for policy 1, policy_version 992701 (0.0008) [2023-12-27 12:32:00,033][20347] Updated weights for policy 0, policy_version 993390 (0.0010) [2023-12-27 12:32:00,058][20381] Updated weights for policy 1, policy_version 992711 (0.0006) [2023-12-27 12:32:00,091][20347] Updated weights for policy 0, policy_version 993400 (0.0010) [2023-12-27 12:32:00,113][20381] Updated weights for policy 1, policy_version 992721 (0.0005) [2023-12-27 12:32:00,146][20347] Updated weights for policy 0, policy_version 993410 (0.0010) [2023-12-27 12:32:00,692][20381] Updated weights for policy 1, policy_version 992731 (0.0008) [2023-12-27 12:32:00,746][20381] Updated weights for policy 1, policy_version 992741 (0.0007) [2023-12-27 12:32:00,797][20381] Updated weights for policy 1, policy_version 992751 (0.0006) [2023-12-27 12:32:00,825][20347] Updated weights for policy 0, policy_version 993420 (0.0008) [2023-12-27 12:32:00,883][20347] Updated weights for policy 0, policy_version 993430 (0.0008) [2023-12-27 12:32:00,942][20347] Updated weights for policy 0, policy_version 993440 (0.0008) [2023-12-27 12:32:01,445][20381] Updated weights for policy 1, policy_version 992761 (0.0006) [2023-12-27 12:32:01,504][20381] Updated weights for policy 1, policy_version 992771 (0.0008) [2023-12-27 12:32:01,569][20381] Updated weights for policy 1, policy_version 992781 (0.0008) [2023-12-27 12:32:01,617][20381] Updated weights for policy 1, policy_version 992791 (0.0006) [2023-12-27 12:32:01,681][20347] Updated weights for policy 0, policy_version 993450 (0.0009) [2023-12-27 12:32:01,741][20347] Updated weights for policy 0, policy_version 993460 (0.0008) [2023-12-27 12:32:01,799][20347] Updated weights for policy 0, policy_version 993470 (0.0005) [2023-12-27 12:32:01,855][20347] Updated weights for policy 0, policy_version 993480 (0.0005) [2023-12-27 12:32:02,291][20381] Updated weights for policy 1, policy_version 992801 (0.0006) [2023-12-27 12:32:02,347][20381] Updated weights for policy 1, policy_version 992811 (0.0006) [2023-12-27 12:32:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 508559360. Throughput: 0: 10237.0, 1: 10213.3. Samples: 508574756. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:02,363][19170] Avg episode reward: [(0, '748.700'), (1, '128.080')] [2023-12-27 12:32:02,408][20381] Updated weights for policy 1, policy_version 992821 (0.0008) [2023-12-27 12:32:02,562][20347] Updated weights for policy 0, policy_version 993490 (0.0008) [2023-12-27 12:32:02,608][20347] Updated weights for policy 0, policy_version 993500 (0.0007) [2023-12-27 12:32:02,655][20347] Updated weights for policy 0, policy_version 993510 (0.0007) [2023-12-27 12:32:03,024][20381] Updated weights for policy 1, policy_version 992831 (0.0008) [2023-12-27 12:32:03,076][20381] Updated weights for policy 1, policy_version 992841 (0.0008) [2023-12-27 12:32:03,135][20381] Updated weights for policy 1, policy_version 992851 (0.0007) [2023-12-27 12:32:03,334][20347] Updated weights for policy 0, policy_version 993520 (0.0006) [2023-12-27 12:32:03,388][20347] Updated weights for policy 0, policy_version 993530 (0.0006) [2023-12-27 12:32:03,439][20347] Updated weights for policy 0, policy_version 993540 (0.0005) [2023-12-27 12:32:03,724][20381] Updated weights for policy 1, policy_version 992861 (0.0007) [2023-12-27 12:32:03,781][20381] Updated weights for policy 1, policy_version 992871 (0.0010) [2023-12-27 12:32:03,839][20381] Updated weights for policy 1, policy_version 992881 (0.0009) [2023-12-27 12:32:04,048][20347] Updated weights for policy 0, policy_version 993550 (0.0008) [2023-12-27 12:32:04,103][20347] Updated weights for policy 0, policy_version 993560 (0.0010) [2023-12-27 12:32:04,161][20347] Updated weights for policy 0, policy_version 993570 (0.0010) [2023-12-27 12:32:04,501][20381] Updated weights for policy 1, policy_version 992891 (0.0006) [2023-12-27 12:32:04,559][20381] Updated weights for policy 1, policy_version 992901 (0.0009) [2023-12-27 12:32:04,618][20381] Updated weights for policy 1, policy_version 992911 (0.0010) [2023-12-27 12:32:04,813][20347] Updated weights for policy 0, policy_version 993580 (0.0009) [2023-12-27 12:32:04,862][20347] Updated weights for policy 0, policy_version 993590 (0.0010) [2023-12-27 12:32:04,906][20347] Updated weights for policy 0, policy_version 993600 (0.0010) [2023-12-27 12:32:05,242][20381] Updated weights for policy 1, policy_version 992921 (0.0011) [2023-12-27 12:32:05,300][20381] Updated weights for policy 1, policy_version 992931 (0.0010) [2023-12-27 12:32:05,350][20381] Updated weights for policy 1, policy_version 992941 (0.0010) [2023-12-27 12:32:05,399][20381] Updated weights for policy 1, policy_version 992951 (0.0011) [2023-12-27 12:32:05,510][20347] Updated weights for policy 0, policy_version 993610 (0.0007) [2023-12-27 12:32:05,557][20347] Updated weights for policy 0, policy_version 993620 (0.0007) [2023-12-27 12:32:05,609][20347] Updated weights for policy 0, policy_version 993630 (0.0009) [2023-12-27 12:32:05,655][20347] Updated weights for policy 0, policy_version 993640 (0.0008) [2023-12-27 12:32:05,995][20381] Updated weights for policy 1, policy_version 992961 (0.0006) [2023-12-27 12:32:06,048][20381] Updated weights for policy 1, policy_version 992971 (0.0007) [2023-12-27 12:32:06,096][20381] Updated weights for policy 1, policy_version 992981 (0.0010) [2023-12-27 12:32:06,413][20347] Updated weights for policy 0, policy_version 993650 (0.0011) [2023-12-27 12:32:06,476][20347] Updated weights for policy 0, policy_version 993660 (0.0010) [2023-12-27 12:32:06,539][20347] Updated weights for policy 0, policy_version 993670 (0.0010) [2023-12-27 12:32:06,777][20381] Updated weights for policy 1, policy_version 992991 (0.0010) [2023-12-27 12:32:06,849][20381] Updated weights for policy 1, policy_version 993001 (0.0010) [2023-12-27 12:32:06,903][20381] Updated weights for policy 1, policy_version 993011 (0.0006) [2023-12-27 12:32:07,216][20347] Updated weights for policy 0, policy_version 993680 (0.0007) [2023-12-27 12:32:07,265][20347] Updated weights for policy 0, policy_version 993690 (0.0008) [2023-12-27 12:32:07,312][20347] Updated weights for policy 0, policy_version 993700 (0.0006) [2023-12-27 12:32:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 508674048. Throughput: 0: 10242.0, 1: 10280.7. Samples: 508639128. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:07,363][19170] Avg episode reward: [(0, '789.690'), (1, '79.700')] [2023-12-27 12:32:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000993016_254238720.pth... [2023-12-27 12:32:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000993704_254435328.pth... [2023-12-27 12:32:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000991832_253935616.pth [2023-12-27 12:32:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000992488_254124032.pth [2023-12-27 12:32:07,514][20381] Updated weights for policy 1, policy_version 993021 (0.0006) [2023-12-27 12:32:07,568][20381] Updated weights for policy 1, policy_version 993031 (0.0005) [2023-12-27 12:32:07,625][20381] Updated weights for policy 1, policy_version 993041 (0.0005) [2023-12-27 12:32:07,978][20347] Updated weights for policy 0, policy_version 993710 (0.0007) [2023-12-27 12:32:08,024][20347] Updated weights for policy 0, policy_version 993720 (0.0007) [2023-12-27 12:32:08,066][20291] KL-divergence is very high: 111.3706 [2023-12-27 12:32:08,070][20347] Updated weights for policy 0, policy_version 993730 (0.0005) [2023-12-27 12:32:08,257][20381] Updated weights for policy 1, policy_version 993051 (0.0006) [2023-12-27 12:32:08,305][20381] Updated weights for policy 1, policy_version 993061 (0.0008) [2023-12-27 12:32:08,365][20381] Updated weights for policy 1, policy_version 993071 (0.0008) [2023-12-27 12:32:08,750][20347] Updated weights for policy 0, policy_version 993740 (0.0006) [2023-12-27 12:32:08,808][20347] Updated weights for policy 0, policy_version 993750 (0.0007) [2023-12-27 12:32:08,862][20347] Updated weights for policy 0, policy_version 993760 (0.0007) [2023-12-27 12:32:09,144][20381] Updated weights for policy 1, policy_version 993081 (0.0008) [2023-12-27 12:32:09,193][20381] Updated weights for policy 1, policy_version 993091 (0.0010) [2023-12-27 12:32:09,255][20381] Updated weights for policy 1, policy_version 993101 (0.0010) [2023-12-27 12:32:09,311][20381] Updated weights for policy 1, policy_version 993111 (0.0010) [2023-12-27 12:32:09,581][20347] Updated weights for policy 0, policy_version 993770 (0.0007) [2023-12-27 12:32:09,625][20347] Updated weights for policy 0, policy_version 993780 (0.0008) [2023-12-27 12:32:09,674][20347] Updated weights for policy 0, policy_version 993790 (0.0008) [2023-12-27 12:32:09,719][20347] Updated weights for policy 0, policy_version 993800 (0.0008) [2023-12-27 12:32:10,058][20381] Updated weights for policy 1, policy_version 993121 (0.0007) [2023-12-27 12:32:10,111][20381] Updated weights for policy 1, policy_version 993131 (0.0008) [2023-12-27 12:32:10,168][20381] Updated weights for policy 1, policy_version 993141 (0.0011) [2023-12-27 12:32:10,488][20347] Updated weights for policy 0, policy_version 993810 (0.0010) [2023-12-27 12:32:10,548][20347] Updated weights for policy 0, policy_version 993820 (0.0010) [2023-12-27 12:32:10,604][20347] Updated weights for policy 0, policy_version 993830 (0.0010) [2023-12-27 12:32:10,861][20381] Updated weights for policy 1, policy_version 993151 (0.0009) [2023-12-27 12:32:10,919][20381] Updated weights for policy 1, policy_version 993161 (0.0010) [2023-12-27 12:32:10,981][20381] Updated weights for policy 1, policy_version 993171 (0.0011) [2023-12-27 12:32:11,315][20347] Updated weights for policy 0, policy_version 993840 (0.0008) [2023-12-27 12:32:11,394][20347] Updated weights for policy 0, policy_version 993852 (0.0008) [2023-12-27 12:32:11,446][20347] Updated weights for policy 0, policy_version 993862 (0.0008) [2023-12-27 12:32:11,733][20381] Updated weights for policy 1, policy_version 993181 (0.0009) [2023-12-27 12:32:11,795][20381] Updated weights for policy 1, policy_version 993191 (0.0006) [2023-12-27 12:32:11,859][20381] Updated weights for policy 1, policy_version 993201 (0.0006) [2023-12-27 12:32:12,183][20347] Updated weights for policy 0, policy_version 993872 (0.0006) [2023-12-27 12:32:12,245][20347] Updated weights for policy 0, policy_version 993882 (0.0006) [2023-12-27 12:32:12,309][20347] Updated weights for policy 0, policy_version 993892 (0.0011) [2023-12-27 12:32:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 508772352. Throughput: 0: 10281.9, 1: 10258.4. Samples: 508761108. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:12,363][19170] Avg episode reward: [(0, '641.230'), (1, '69.300')] [2023-12-27 12:32:12,449][20381] Updated weights for policy 1, policy_version 993211 (0.0007) [2023-12-27 12:32:12,503][20381] Updated weights for policy 1, policy_version 993221 (0.0010) [2023-12-27 12:32:12,562][20381] Updated weights for policy 1, policy_version 993231 (0.0009) [2023-12-27 12:32:12,974][20347] Updated weights for policy 0, policy_version 993902 (0.0010) [2023-12-27 12:32:13,024][20347] Updated weights for policy 0, policy_version 993912 (0.0010) [2023-12-27 12:32:13,069][20347] Updated weights for policy 0, policy_version 993922 (0.0010) [2023-12-27 12:32:13,238][20381] Updated weights for policy 1, policy_version 993241 (0.0008) [2023-12-27 12:32:13,290][20381] Updated weights for policy 1, policy_version 993251 (0.0007) [2023-12-27 12:32:13,338][20381] Updated weights for policy 1, policy_version 993261 (0.0005) [2023-12-27 12:32:13,387][20381] Updated weights for policy 1, policy_version 993271 (0.0006) [2023-12-27 12:32:13,753][20347] Updated weights for policy 0, policy_version 993932 (0.0010) [2023-12-27 12:32:13,808][20347] Updated weights for policy 0, policy_version 993942 (0.0010) [2023-12-27 12:32:13,866][20347] Updated weights for policy 0, policy_version 993952 (0.0010) [2023-12-27 12:32:14,042][20381] Updated weights for policy 1, policy_version 993281 (0.0006) [2023-12-27 12:32:14,093][20381] Updated weights for policy 1, policy_version 993291 (0.0008) [2023-12-27 12:32:14,145][20381] Updated weights for policy 1, policy_version 993301 (0.0010) [2023-12-27 12:32:14,501][20347] Updated weights for policy 0, policy_version 993962 (0.0010) [2023-12-27 12:32:14,560][20347] Updated weights for policy 0, policy_version 993972 (0.0010) [2023-12-27 12:32:14,611][20347] Updated weights for policy 0, policy_version 993982 (0.0010) [2023-12-27 12:32:14,658][20347] Updated weights for policy 0, policy_version 993992 (0.0007) [2023-12-27 12:32:14,810][20381] Updated weights for policy 1, policy_version 993311 (0.0009) [2023-12-27 12:32:14,862][20381] Updated weights for policy 1, policy_version 993321 (0.0008) [2023-12-27 12:32:14,922][20381] Updated weights for policy 1, policy_version 993331 (0.0008) [2023-12-27 12:32:15,383][20347] Updated weights for policy 0, policy_version 994002 (0.0008) [2023-12-27 12:32:15,444][20347] Updated weights for policy 0, policy_version 994012 (0.0006) [2023-12-27 12:32:15,497][20347] Updated weights for policy 0, policy_version 994022 (0.0005) [2023-12-27 12:32:15,664][20381] Updated weights for policy 1, policy_version 993341 (0.0007) [2023-12-27 12:32:15,730][20381] Updated weights for policy 1, policy_version 993351 (0.0005) [2023-12-27 12:32:15,797][20381] Updated weights for policy 1, policy_version 993361 (0.0005) [2023-12-27 12:32:16,131][20347] Updated weights for policy 0, policy_version 994032 (0.0005) [2023-12-27 12:32:16,195][20347] Updated weights for policy 0, policy_version 994042 (0.0010) [2023-12-27 12:32:16,254][20347] Updated weights for policy 0, policy_version 994052 (0.0010) [2023-12-27 12:32:16,437][20381] Updated weights for policy 1, policy_version 993371 (0.0010) [2023-12-27 12:32:16,490][20381] Updated weights for policy 1, policy_version 993381 (0.0005) [2023-12-27 12:32:16,543][20381] Updated weights for policy 1, policy_version 993391 (0.0006) [2023-12-27 12:32:16,834][20347] Updated weights for policy 0, policy_version 994062 (0.0010) [2023-12-27 12:32:16,889][20347] Updated weights for policy 0, policy_version 994072 (0.0010) [2023-12-27 12:32:16,941][20347] Updated weights for policy 0, policy_version 994082 (0.0010) [2023-12-27 12:32:17,183][20381] Updated weights for policy 1, policy_version 993401 (0.0006) [2023-12-27 12:32:17,241][20381] Updated weights for policy 1, policy_version 993411 (0.0010) [2023-12-27 12:32:17,298][20381] Updated weights for policy 1, policy_version 993421 (0.0007) [2023-12-27 12:32:17,358][20381] Updated weights for policy 1, policy_version 993431 (0.0007) [2023-12-27 12:32:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 508878848. Throughput: 0: 10238.6, 1: 10256.9. Samples: 508885356. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:17,363][19170] Avg episode reward: [(0, '750.830'), (1, '334.680')] [2023-12-27 12:32:17,622][20347] Updated weights for policy 0, policy_version 994092 (0.0010) [2023-12-27 12:32:17,672][20347] Updated weights for policy 0, policy_version 994102 (0.0010) [2023-12-27 12:32:17,728][20347] Updated weights for policy 0, policy_version 994112 (0.0010) [2023-12-27 12:32:18,017][20381] Updated weights for policy 1, policy_version 993441 (0.0011) [2023-12-27 12:32:18,072][20381] Updated weights for policy 1, policy_version 993451 (0.0010) [2023-12-27 12:32:18,133][20381] Updated weights for policy 1, policy_version 993461 (0.0011) [2023-12-27 12:32:18,418][20347] Updated weights for policy 0, policy_version 994122 (0.0009) [2023-12-27 12:32:18,479][20347] Updated weights for policy 0, policy_version 994132 (0.0006) [2023-12-27 12:32:18,545][20347] Updated weights for policy 0, policy_version 994142 (0.0007) [2023-12-27 12:32:18,613][20347] Updated weights for policy 0, policy_version 994152 (0.0007) [2023-12-27 12:32:18,767][20381] Updated weights for policy 1, policy_version 993471 (0.0007) [2023-12-27 12:32:18,829][20381] Updated weights for policy 1, policy_version 993481 (0.0005) [2023-12-27 12:32:18,888][20381] Updated weights for policy 1, policy_version 993491 (0.0008) [2023-12-27 12:32:19,276][20347] Updated weights for policy 0, policy_version 994162 (0.0008) [2023-12-27 12:32:19,335][20347] Updated weights for policy 0, policy_version 994172 (0.0007) [2023-12-27 12:32:19,403][20347] Updated weights for policy 0, policy_version 994182 (0.0008) [2023-12-27 12:32:19,561][20381] Updated weights for policy 1, policy_version 993501 (0.0007) [2023-12-27 12:32:19,625][20381] Updated weights for policy 1, policy_version 993511 (0.0010) [2023-12-27 12:32:19,684][20381] Updated weights for policy 1, policy_version 993521 (0.0007) [2023-12-27 12:32:20,164][20347] Updated weights for policy 0, policy_version 994192 (0.0008) [2023-12-27 12:32:20,226][20347] Updated weights for policy 0, policy_version 994202 (0.0008) [2023-12-27 12:32:20,278][20347] Updated weights for policy 0, policy_version 994212 (0.0008) [2023-12-27 12:32:20,380][20381] Updated weights for policy 1, policy_version 993531 (0.0011) [2023-12-27 12:32:20,436][20381] Updated weights for policy 1, policy_version 993541 (0.0010) [2023-12-27 12:32:20,492][20381] Updated weights for policy 1, policy_version 993551 (0.0011) [2023-12-27 12:32:21,005][20347] Updated weights for policy 0, policy_version 994222 (0.0006) [2023-12-27 12:32:21,069][20347] Updated weights for policy 0, policy_version 994232 (0.0009) [2023-12-27 12:32:21,129][20347] Updated weights for policy 0, policy_version 994242 (0.0008) [2023-12-27 12:32:21,243][20381] Updated weights for policy 1, policy_version 993561 (0.0011) [2023-12-27 12:32:21,309][20381] Updated weights for policy 1, policy_version 993571 (0.0011) [2023-12-27 12:32:21,376][20381] Updated weights for policy 1, policy_version 993581 (0.0010) [2023-12-27 12:32:21,437][20381] Updated weights for policy 1, policy_version 993591 (0.0011) [2023-12-27 12:32:21,920][20347] Updated weights for policy 0, policy_version 994252 (0.0008) [2023-12-27 12:32:21,976][20347] Updated weights for policy 0, policy_version 994262 (0.0010) [2023-12-27 12:32:22,036][20347] Updated weights for policy 0, policy_version 994272 (0.0007) [2023-12-27 12:32:22,168][20381] Updated weights for policy 1, policy_version 993601 (0.0008) [2023-12-27 12:32:22,220][20381] Updated weights for policy 1, policy_version 993611 (0.0008) [2023-12-27 12:32:22,280][20381] Updated weights for policy 1, policy_version 993621 (0.0008) [2023-12-27 12:32:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 508977152. Throughput: 0: 10231.5, 1: 10255.4. Samples: 508946260. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:22,363][19170] Avg episode reward: [(0, '834.360'), (1, '315.480')] [2023-12-27 12:32:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000993624_254394368.pth... [2023-12-27 12:32:22,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000994280_254582784.pth... [2023-12-27 12:32:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000992408_254083072.pth [2023-12-27 12:32:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000993096_254279680.pth [2023-12-27 12:32:22,747][20347] Updated weights for policy 0, policy_version 994282 (0.0006) [2023-12-27 12:32:22,797][20347] Updated weights for policy 0, policy_version 994292 (0.0010) [2023-12-27 12:32:22,853][20347] Updated weights for policy 0, policy_version 994302 (0.0008) [2023-12-27 12:32:22,907][20347] Updated weights for policy 0, policy_version 994312 (0.0008) [2023-12-27 12:32:22,960][20381] Updated weights for policy 1, policy_version 993631 (0.0007) [2023-12-27 12:32:23,017][20381] Updated weights for policy 1, policy_version 993641 (0.0007) [2023-12-27 12:32:23,074][20381] Updated weights for policy 1, policy_version 993651 (0.0005) [2023-12-27 12:32:23,509][20347] Updated weights for policy 0, policy_version 994322 (0.0005) [2023-12-27 12:32:23,562][20347] Updated weights for policy 0, policy_version 994332 (0.0005) [2023-12-27 12:32:23,601][20381] Updated weights for policy 1, policy_version 993661 (0.0008) [2023-12-27 12:32:23,626][20347] Updated weights for policy 0, policy_version 994342 (0.0006) [2023-12-27 12:32:23,661][20381] Updated weights for policy 1, policy_version 993671 (0.0006) [2023-12-27 12:32:23,712][20381] Updated weights for policy 1, policy_version 993681 (0.0005) [2023-12-27 12:32:24,295][20381] Updated weights for policy 1, policy_version 993691 (0.0007) [2023-12-27 12:32:24,316][20347] Updated weights for policy 0, policy_version 994352 (0.0006) [2023-12-27 12:32:24,353][20381] Updated weights for policy 1, policy_version 993701 (0.0011) [2023-12-27 12:32:24,374][20347] Updated weights for policy 0, policy_version 994362 (0.0007) [2023-12-27 12:32:24,407][20381] Updated weights for policy 1, policy_version 993711 (0.0011) [2023-12-27 12:32:24,431][20347] Updated weights for policy 0, policy_version 994372 (0.0006) [2023-12-27 12:32:25,101][20381] Updated weights for policy 1, policy_version 993721 (0.0011) [2023-12-27 12:32:25,121][20347] Updated weights for policy 0, policy_version 994382 (0.0005) [2023-12-27 12:32:25,155][20381] Updated weights for policy 1, policy_version 993731 (0.0010) [2023-12-27 12:32:25,167][20347] Updated weights for policy 0, policy_version 994392 (0.0007) [2023-12-27 12:32:25,209][20381] Updated weights for policy 1, policy_version 993741 (0.0010) [2023-12-27 12:32:25,218][20347] Updated weights for policy 0, policy_version 994402 (0.0005) [2023-12-27 12:32:25,263][20381] Updated weights for policy 1, policy_version 993751 (0.0010) [2023-12-27 12:32:25,872][20347] Updated weights for policy 0, policy_version 994412 (0.0006) [2023-12-27 12:32:25,926][20347] Updated weights for policy 0, policy_version 994422 (0.0007) [2023-12-27 12:32:25,943][20381] Updated weights for policy 1, policy_version 993761 (0.0010) [2023-12-27 12:32:25,978][20347] Updated weights for policy 0, policy_version 994432 (0.0006) [2023-12-27 12:32:25,992][20381] Updated weights for policy 1, policy_version 993771 (0.0010) [2023-12-27 12:32:26,041][20381] Updated weights for policy 1, policy_version 993781 (0.0009) [2023-12-27 12:32:26,667][20347] Updated weights for policy 0, policy_version 994442 (0.0007) [2023-12-27 12:32:26,711][20347] Updated weights for policy 0, policy_version 994452 (0.0007) [2023-12-27 12:32:26,759][20347] Updated weights for policy 0, policy_version 994462 (0.0006) [2023-12-27 12:32:26,760][20381] Updated weights for policy 1, policy_version 993791 (0.0007) [2023-12-27 12:32:26,807][20347] Updated weights for policy 0, policy_version 994472 (0.0007) [2023-12-27 12:32:26,814][20381] Updated weights for policy 1, policy_version 993801 (0.0008) [2023-12-27 12:32:26,861][20381] Updated weights for policy 1, policy_version 993811 (0.0008) [2023-12-27 12:32:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 509075456. Throughput: 0: 10160.9, 1: 10239.8. Samples: 509068220. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:27,363][19170] Avg episode reward: [(0, '817.010'), (1, '334.440')] [2023-12-27 12:32:27,485][20381] Updated weights for policy 1, policy_version 993821 (0.0009) [2023-12-27 12:32:27,502][20347] Updated weights for policy 0, policy_version 994482 (0.0006) [2023-12-27 12:32:27,534][20381] Updated weights for policy 1, policy_version 993831 (0.0010) [2023-12-27 12:32:27,559][20347] Updated weights for policy 0, policy_version 994492 (0.0006) [2023-12-27 12:32:27,579][20381] Updated weights for policy 1, policy_version 993841 (0.0010) [2023-12-27 12:32:27,614][20347] Updated weights for policy 0, policy_version 994502 (0.0005) [2023-12-27 12:32:28,209][20381] Updated weights for policy 1, policy_version 993851 (0.0008) [2023-12-27 12:32:28,248][20347] Updated weights for policy 0, policy_version 994512 (0.0006) [2023-12-27 12:32:28,260][20381] Updated weights for policy 1, policy_version 993861 (0.0007) [2023-12-27 12:32:28,309][20347] Updated weights for policy 0, policy_version 994522 (0.0005) [2023-12-27 12:32:28,315][20381] Updated weights for policy 1, policy_version 993871 (0.0006) [2023-12-27 12:32:28,369][20347] Updated weights for policy 0, policy_version 994532 (0.0006) [2023-12-27 12:32:28,940][20381] Updated weights for policy 1, policy_version 993881 (0.0008) [2023-12-27 12:32:29,000][20381] Updated weights for policy 1, policy_version 993891 (0.0007) [2023-12-27 12:32:29,043][20347] Updated weights for policy 0, policy_version 994542 (0.0006) [2023-12-27 12:32:29,055][20381] Updated weights for policy 1, policy_version 993901 (0.0007) [2023-12-27 12:32:29,087][20347] Updated weights for policy 0, policy_version 994552 (0.0010) [2023-12-27 12:32:29,100][20381] Updated weights for policy 1, policy_version 993911 (0.0005) [2023-12-27 12:32:29,141][20347] Updated weights for policy 0, policy_version 994562 (0.0010) [2023-12-27 12:32:29,793][20381] Updated weights for policy 1, policy_version 993921 (0.0007) [2023-12-27 12:32:29,802][20347] Updated weights for policy 0, policy_version 994572 (0.0010) [2023-12-27 12:32:29,852][20381] Updated weights for policy 1, policy_version 993931 (0.0007) [2023-12-27 12:32:29,860][20347] Updated weights for policy 0, policy_version 994582 (0.0009) [2023-12-27 12:32:29,915][20381] Updated weights for policy 1, policy_version 993941 (0.0007) [2023-12-27 12:32:29,919][20347] Updated weights for policy 0, policy_version 994592 (0.0008) [2023-12-27 12:32:30,509][20381] Updated weights for policy 1, policy_version 993951 (0.0006) [2023-12-27 12:32:30,562][20381] Updated weights for policy 1, policy_version 993961 (0.0005) [2023-12-27 12:32:30,611][20347] Updated weights for policy 0, policy_version 994602 (0.0006) [2023-12-27 12:32:30,616][20381] Updated weights for policy 1, policy_version 993971 (0.0006) [2023-12-27 12:32:30,670][20347] Updated weights for policy 0, policy_version 994612 (0.0007) [2023-12-27 12:32:30,723][20347] Updated weights for policy 0, policy_version 994622 (0.0007) [2023-12-27 12:32:30,774][20347] Updated weights for policy 0, policy_version 994632 (0.0006) [2023-12-27 12:32:31,260][20381] Updated weights for policy 1, policy_version 993981 (0.0008) [2023-12-27 12:32:31,320][20381] Updated weights for policy 1, policy_version 993991 (0.0007) [2023-12-27 12:32:31,389][20381] Updated weights for policy 1, policy_version 994001 (0.0007) [2023-12-27 12:32:31,408][20347] Updated weights for policy 0, policy_version 994642 (0.0007) [2023-12-27 12:32:31,459][20347] Updated weights for policy 0, policy_version 994652 (0.0007) [2023-12-27 12:32:31,512][20347] Updated weights for policy 0, policy_version 994662 (0.0006) [2023-12-27 12:32:32,122][20381] Updated weights for policy 1, policy_version 994011 (0.0008) [2023-12-27 12:32:32,131][20347] Updated weights for policy 0, policy_version 994672 (0.0006) [2023-12-27 12:32:32,181][20381] Updated weights for policy 1, policy_version 994021 (0.0011) [2023-12-27 12:32:32,183][20347] Updated weights for policy 0, policy_version 994682 (0.0006) [2023-12-27 12:32:32,229][20347] Updated weights for policy 0, policy_version 994692 (0.0005) [2023-12-27 12:32:32,237][20381] Updated weights for policy 1, policy_version 994031 (0.0010) [2023-12-27 12:32:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 509190144. Throughput: 0: 10208.1, 1: 10350.2. Samples: 509194700. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:32,363][19170] Avg episode reward: [(0, '824.970'), (1, '511.290')] [2023-12-27 12:32:32,861][20347] Updated weights for policy 0, policy_version 994702 (0.0005) [2023-12-27 12:32:32,869][20381] Updated weights for policy 1, policy_version 994041 (0.0010) [2023-12-27 12:32:32,911][20347] Updated weights for policy 0, policy_version 994712 (0.0005) [2023-12-27 12:32:32,917][20381] Updated weights for policy 1, policy_version 994051 (0.0010) [2023-12-27 12:32:32,962][20347] Updated weights for policy 0, policy_version 994722 (0.0006) [2023-12-27 12:32:32,976][20381] Updated weights for policy 1, policy_version 994061 (0.0008) [2023-12-27 12:32:33,026][20381] Updated weights for policy 1, policy_version 994071 (0.0006) [2023-12-27 12:32:33,594][20381] Updated weights for policy 1, policy_version 994081 (0.0005) [2023-12-27 12:32:33,656][20381] Updated weights for policy 1, policy_version 994091 (0.0007) [2023-12-27 12:32:33,661][20347] Updated weights for policy 0, policy_version 994732 (0.0006) [2023-12-27 12:32:33,709][20347] Updated weights for policy 0, policy_version 994742 (0.0005) [2023-12-27 12:32:33,710][20381] Updated weights for policy 1, policy_version 994101 (0.0010) [2023-12-27 12:32:33,766][20347] Updated weights for policy 0, policy_version 994752 (0.0006) [2023-12-27 12:32:34,397][20381] Updated weights for policy 1, policy_version 994111 (0.0010) [2023-12-27 12:32:34,462][20381] Updated weights for policy 1, policy_version 994121 (0.0011) [2023-12-27 12:32:34,475][20347] Updated weights for policy 0, policy_version 994762 (0.0007) [2023-12-27 12:32:34,511][20381] Updated weights for policy 1, policy_version 994131 (0.0010) [2023-12-27 12:32:34,531][20347] Updated weights for policy 0, policy_version 994772 (0.0006) [2023-12-27 12:32:34,591][20347] Updated weights for policy 0, policy_version 994782 (0.0008) [2023-12-27 12:32:34,651][20347] Updated weights for policy 0, policy_version 994792 (0.0008) [2023-12-27 12:32:35,179][20381] Updated weights for policy 1, policy_version 994141 (0.0010) [2023-12-27 12:32:35,245][20381] Updated weights for policy 1, policy_version 994151 (0.0010) [2023-12-27 12:32:35,288][20347] Updated weights for policy 0, policy_version 994802 (0.0005) [2023-12-27 12:32:35,305][20381] Updated weights for policy 1, policy_version 994161 (0.0010) [2023-12-27 12:32:35,346][20347] Updated weights for policy 0, policy_version 994812 (0.0007) [2023-12-27 12:32:35,394][20347] Updated weights for policy 0, policy_version 994822 (0.0008) [2023-12-27 12:32:35,935][20381] Updated weights for policy 1, policy_version 994171 (0.0009) [2023-12-27 12:32:35,990][20381] Updated weights for policy 1, policy_version 994181 (0.0010) [2023-12-27 12:32:36,038][20381] Updated weights for policy 1, policy_version 994191 (0.0010) [2023-12-27 12:32:36,094][20347] Updated weights for policy 0, policy_version 994832 (0.0007) [2023-12-27 12:32:36,156][20347] Updated weights for policy 0, policy_version 994842 (0.0008) [2023-12-27 12:32:36,216][20347] Updated weights for policy 0, policy_version 994852 (0.0007) [2023-12-27 12:32:36,722][20381] Updated weights for policy 1, policy_version 994201 (0.0007) [2023-12-27 12:32:36,783][20381] Updated weights for policy 1, policy_version 994211 (0.0009) [2023-12-27 12:32:36,839][20381] Updated weights for policy 1, policy_version 994221 (0.0011) [2023-12-27 12:32:36,895][20381] Updated weights for policy 1, policy_version 994231 (0.0011) [2023-12-27 12:32:36,944][20347] Updated weights for policy 0, policy_version 994862 (0.0008) [2023-12-27 12:32:37,001][20347] Updated weights for policy 0, policy_version 994872 (0.0007) [2023-12-27 12:32:37,053][20347] Updated weights for policy 0, policy_version 994882 (0.0007) [2023-12-27 12:32:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 509288448. Throughput: 0: 10202.1, 1: 10401.5. Samples: 509258364. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:37,364][19170] Avg episode reward: [(0, '863.530'), (1, '657.260')] [2023-12-27 12:32:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000994232_254550016.pth... [2023-12-27 12:32:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000994888_254738432.pth... [2023-12-27 12:32:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000993016_254238720.pth [2023-12-27 12:32:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000993704_254435328.pth [2023-12-27 12:32:37,510][20381] Updated weights for policy 1, policy_version 994241 (0.0006) [2023-12-27 12:32:37,567][20381] Updated weights for policy 1, policy_version 994251 (0.0006) [2023-12-27 12:32:37,627][20381] Updated weights for policy 1, policy_version 994261 (0.0006) [2023-12-27 12:32:37,809][20347] Updated weights for policy 0, policy_version 994892 (0.0007) [2023-12-27 12:32:37,875][20347] Updated weights for policy 0, policy_version 994902 (0.0008) [2023-12-27 12:32:37,932][20347] Updated weights for policy 0, policy_version 994912 (0.0007) [2023-12-27 12:32:38,198][20381] Updated weights for policy 1, policy_version 994271 (0.0007) [2023-12-27 12:32:38,259][20381] Updated weights for policy 1, policy_version 994281 (0.0006) [2023-12-27 12:32:38,322][20381] Updated weights for policy 1, policy_version 994291 (0.0006) [2023-12-27 12:32:38,633][20347] Updated weights for policy 0, policy_version 994922 (0.0007) [2023-12-27 12:32:38,693][20347] Updated weights for policy 0, policy_version 994932 (0.0005) [2023-12-27 12:32:38,752][20347] Updated weights for policy 0, policy_version 994942 (0.0007) [2023-12-27 12:32:38,799][20347] Updated weights for policy 0, policy_version 994952 (0.0008) [2023-12-27 12:32:38,982][20381] Updated weights for policy 1, policy_version 994301 (0.0008) [2023-12-27 12:32:39,036][20381] Updated weights for policy 1, policy_version 994311 (0.0008) [2023-12-27 12:32:39,091][20381] Updated weights for policy 1, policy_version 994321 (0.0008) [2023-12-27 12:32:39,501][20347] Updated weights for policy 0, policy_version 994962 (0.0006) [2023-12-27 12:32:39,561][20347] Updated weights for policy 0, policy_version 994972 (0.0007) [2023-12-27 12:32:39,620][20347] Updated weights for policy 0, policy_version 994982 (0.0006) [2023-12-27 12:32:39,780][20381] Updated weights for policy 1, policy_version 994331 (0.0008) [2023-12-27 12:32:39,845][20381] Updated weights for policy 1, policy_version 994341 (0.0007) [2023-12-27 12:32:39,915][20381] Updated weights for policy 1, policy_version 994351 (0.0007) [2023-12-27 12:32:40,272][20347] Updated weights for policy 0, policy_version 994992 (0.0008) [2023-12-27 12:32:40,321][20347] Updated weights for policy 0, policy_version 995002 (0.0007) [2023-12-27 12:32:40,377][20347] Updated weights for policy 0, policy_version 995012 (0.0008) [2023-12-27 12:32:40,651][20381] Updated weights for policy 1, policy_version 994361 (0.0009) [2023-12-27 12:32:40,711][20381] Updated weights for policy 1, policy_version 994371 (0.0008) [2023-12-27 12:32:40,768][20381] Updated weights for policy 1, policy_version 994381 (0.0007) [2023-12-27 12:32:40,825][20381] Updated weights for policy 1, policy_version 994391 (0.0008) [2023-12-27 12:32:41,056][20347] Updated weights for policy 0, policy_version 995022 (0.0008) [2023-12-27 12:32:41,121][20347] Updated weights for policy 0, policy_version 995032 (0.0008) [2023-12-27 12:32:41,191][20347] Updated weights for policy 0, policy_version 995042 (0.0007) [2023-12-27 12:32:41,590][20381] Updated weights for policy 1, policy_version 994401 (0.0008) [2023-12-27 12:32:41,657][20381] Updated weights for policy 1, policy_version 994411 (0.0008) [2023-12-27 12:32:41,723][20381] Updated weights for policy 1, policy_version 994421 (0.0008) [2023-12-27 12:32:41,921][20347] Updated weights for policy 0, policy_version 995052 (0.0008) [2023-12-27 12:32:41,972][20347] Updated weights for policy 0, policy_version 995062 (0.0009) [2023-12-27 12:32:42,029][20347] Updated weights for policy 0, policy_version 995072 (0.0008) [2023-12-27 12:32:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 509386752. Throughput: 0: 10170.3, 1: 10421.6. Samples: 509379880. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:42,363][19170] Avg episode reward: [(0, '921.800'), (1, '438.910')] [2023-12-27 12:32:42,451][20381] Updated weights for policy 1, policy_version 994431 (0.0008) [2023-12-27 12:32:42,511][20381] Updated weights for policy 1, policy_version 994441 (0.0008) [2023-12-27 12:32:42,577][20381] Updated weights for policy 1, policy_version 994451 (0.0008) [2023-12-27 12:32:42,719][20347] Updated weights for policy 0, policy_version 995082 (0.0008) [2023-12-27 12:32:42,774][20347] Updated weights for policy 0, policy_version 995092 (0.0007) [2023-12-27 12:32:42,820][20347] Updated weights for policy 0, policy_version 995102 (0.0008) [2023-12-27 12:32:42,867][20347] Updated weights for policy 0, policy_version 995112 (0.0008) [2023-12-27 12:32:43,252][20381] Updated weights for policy 1, policy_version 994461 (0.0007) [2023-12-27 12:32:43,304][20381] Updated weights for policy 1, policy_version 994471 (0.0008) [2023-12-27 12:32:43,357][20381] Updated weights for policy 1, policy_version 994481 (0.0008) [2023-12-27 12:32:43,567][20347] Updated weights for policy 0, policy_version 995122 (0.0007) [2023-12-27 12:32:43,619][20347] Updated weights for policy 0, policy_version 995132 (0.0007) [2023-12-27 12:32:43,675][20347] Updated weights for policy 0, policy_version 995142 (0.0006) [2023-12-27 12:32:44,024][20381] Updated weights for policy 1, policy_version 994491 (0.0008) [2023-12-27 12:32:44,073][20381] Updated weights for policy 1, policy_version 994501 (0.0008) [2023-12-27 12:32:44,118][20381] Updated weights for policy 1, policy_version 994511 (0.0007) [2023-12-27 12:32:44,295][20347] Updated weights for policy 0, policy_version 995152 (0.0007) [2023-12-27 12:32:44,347][20347] Updated weights for policy 0, policy_version 995162 (0.0007) [2023-12-27 12:32:44,399][20347] Updated weights for policy 0, policy_version 995172 (0.0008) [2023-12-27 12:32:44,735][20381] Updated weights for policy 1, policy_version 994521 (0.0007) [2023-12-27 12:32:44,794][20381] Updated weights for policy 1, policy_version 994531 (0.0007) [2023-12-27 12:32:44,844][20381] Updated weights for policy 1, policy_version 994541 (0.0008) [2023-12-27 12:32:44,904][20381] Updated weights for policy 1, policy_version 994551 (0.0007) [2023-12-27 12:32:45,132][20347] Updated weights for policy 0, policy_version 995182 (0.0008) [2023-12-27 12:32:45,191][20347] Updated weights for policy 0, policy_version 995192 (0.0008) [2023-12-27 12:32:45,241][20347] Updated weights for policy 0, policy_version 995202 (0.0007) [2023-12-27 12:32:45,652][20381] Updated weights for policy 1, policy_version 994561 (0.0008) [2023-12-27 12:32:45,697][20381] Updated weights for policy 1, policy_version 994571 (0.0008) [2023-12-27 12:32:45,751][20381] Updated weights for policy 1, policy_version 994581 (0.0008) [2023-12-27 12:32:45,849][20347] Updated weights for policy 0, policy_version 995212 (0.0006) [2023-12-27 12:32:45,914][20347] Updated weights for policy 0, policy_version 995222 (0.0005) [2023-12-27 12:32:45,972][20347] Updated weights for policy 0, policy_version 995232 (0.0007) [2023-12-27 12:32:46,473][20381] Updated weights for policy 1, policy_version 994591 (0.0006) [2023-12-27 12:32:46,521][20381] Updated weights for policy 1, policy_version 994601 (0.0005) [2023-12-27 12:32:46,569][20381] Updated weights for policy 1, policy_version 994611 (0.0005) [2023-12-27 12:32:46,639][20347] Updated weights for policy 0, policy_version 995242 (0.0007) [2023-12-27 12:32:46,687][20347] Updated weights for policy 0, policy_version 995252 (0.0008) [2023-12-27 12:32:46,735][20347] Updated weights for policy 0, policy_version 995262 (0.0007) [2023-12-27 12:32:46,790][20347] Updated weights for policy 0, policy_version 995272 (0.0008) [2023-12-27 12:32:47,209][20381] Updated weights for policy 1, policy_version 994621 (0.0005) [2023-12-27 12:32:47,266][20381] Updated weights for policy 1, policy_version 994631 (0.0006) [2023-12-27 12:32:47,317][20381] Updated weights for policy 1, policy_version 994641 (0.0010) [2023-12-27 12:32:47,363][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 509493248. Throughput: 0: 10198.5, 1: 10400.3. Samples: 509501704. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:47,363][19170] Avg episode reward: [(0, '844.590'), (1, '586.540')] [2023-12-27 12:32:47,469][20347] Updated weights for policy 0, policy_version 995282 (0.0005) [2023-12-27 12:32:47,530][20347] Updated weights for policy 0, policy_version 995292 (0.0007) [2023-12-27 12:32:47,578][20347] Updated weights for policy 0, policy_version 995302 (0.0007) [2023-12-27 12:32:47,900][20381] Updated weights for policy 1, policy_version 994651 (0.0005) [2023-12-27 12:32:47,962][20381] Updated weights for policy 1, policy_version 994661 (0.0005) [2023-12-27 12:32:48,010][20381] Updated weights for policy 1, policy_version 994671 (0.0006) [2023-12-27 12:32:48,281][20347] Updated weights for policy 0, policy_version 995312 (0.0007) [2023-12-27 12:32:48,340][20347] Updated weights for policy 0, policy_version 995322 (0.0008) [2023-12-27 12:32:48,399][20347] Updated weights for policy 0, policy_version 995332 (0.0007) [2023-12-27 12:32:48,660][20381] Updated weights for policy 1, policy_version 994681 (0.0007) [2023-12-27 12:32:48,722][20381] Updated weights for policy 1, policy_version 994691 (0.0007) [2023-12-27 12:32:48,779][20381] Updated weights for policy 1, policy_version 994701 (0.0010) [2023-12-27 12:32:48,837][20381] Updated weights for policy 1, policy_version 994711 (0.0012) [2023-12-27 12:32:49,120][20347] Updated weights for policy 0, policy_version 995342 (0.0008) [2023-12-27 12:32:49,164][20347] Updated weights for policy 0, policy_version 995352 (0.0007) [2023-12-27 12:32:49,215][20347] Updated weights for policy 0, policy_version 995362 (0.0008) [2023-12-27 12:32:49,514][20381] Updated weights for policy 1, policy_version 994721 (0.0010) [2023-12-27 12:32:49,572][20381] Updated weights for policy 1, policy_version 994731 (0.0011) [2023-12-27 12:32:49,650][20381] Updated weights for policy 1, policy_version 994741 (0.0011) [2023-12-27 12:32:49,949][20347] Updated weights for policy 0, policy_version 995372 (0.0007) [2023-12-27 12:32:50,014][20347] Updated weights for policy 0, policy_version 995382 (0.0006) [2023-12-27 12:32:50,072][20347] Updated weights for policy 0, policy_version 995392 (0.0006) [2023-12-27 12:32:50,336][20381] Updated weights for policy 1, policy_version 994751 (0.0009) [2023-12-27 12:32:50,400][20381] Updated weights for policy 1, policy_version 994761 (0.0008) [2023-12-27 12:32:50,455][20381] Updated weights for policy 1, policy_version 994771 (0.0010) [2023-12-27 12:32:50,675][20347] Updated weights for policy 0, policy_version 995402 (0.0008) [2023-12-27 12:32:50,736][20347] Updated weights for policy 0, policy_version 995412 (0.0007) [2023-12-27 12:32:50,792][20347] Updated weights for policy 0, policy_version 995422 (0.0007) [2023-12-27 12:32:50,847][20347] Updated weights for policy 0, policy_version 995432 (0.0008) [2023-12-27 12:32:51,140][20381] Updated weights for policy 1, policy_version 994781 (0.0009) [2023-12-27 12:32:51,201][20381] Updated weights for policy 1, policy_version 994791 (0.0008) [2023-12-27 12:32:51,259][20381] Updated weights for policy 1, policy_version 994801 (0.0008) [2023-12-27 12:32:51,579][20347] Updated weights for policy 0, policy_version 995442 (0.0008) [2023-12-27 12:32:51,646][20347] Updated weights for policy 0, policy_version 995452 (0.0008) [2023-12-27 12:32:51,710][20347] Updated weights for policy 0, policy_version 995462 (0.0008) [2023-12-27 12:32:51,972][20381] Updated weights for policy 1, policy_version 994811 (0.0008) [2023-12-27 12:32:52,040][20381] Updated weights for policy 1, policy_version 994821 (0.0008) [2023-12-27 12:32:52,101][20381] Updated weights for policy 1, policy_version 994831 (0.0008) [2023-12-27 12:32:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 509591552. Throughput: 0: 10167.2, 1: 10378.5. Samples: 509563684. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:52,363][19170] Avg episode reward: [(0, '796.490'), (1, '560.450')] [2023-12-27 12:32:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000994840_254705664.pth... [2023-12-27 12:32:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000993624_254394368.pth [2023-12-27 12:32:52,401][20347] Updated weights for policy 0, policy_version 995472 (0.0007) [2023-12-27 12:32:52,458][20347] Updated weights for policy 0, policy_version 995482 (0.0007) [2023-12-27 12:32:52,512][20347] Updated weights for policy 0, policy_version 995492 (0.0007) [2023-12-27 12:32:52,531][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000995496_254894080.pth... [2023-12-27 12:32:52,535][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000994280_254582784.pth [2023-12-27 12:32:52,849][20381] Updated weights for policy 1, policy_version 994841 (0.0008) [2023-12-27 12:32:52,894][20381] Updated weights for policy 1, policy_version 994851 (0.0007) [2023-12-27 12:32:52,945][20381] Updated weights for policy 1, policy_version 994861 (0.0008) [2023-12-27 12:32:52,993][20381] Updated weights for policy 1, policy_version 994871 (0.0008) [2023-12-27 12:32:53,162][20347] Updated weights for policy 0, policy_version 995502 (0.0007) [2023-12-27 12:32:53,208][20347] Updated weights for policy 0, policy_version 995512 (0.0009) [2023-12-27 12:32:53,253][20347] Updated weights for policy 0, policy_version 995522 (0.0007) [2023-12-27 12:32:53,595][20381] Updated weights for policy 1, policy_version 994881 (0.0008) [2023-12-27 12:32:53,651][20381] Updated weights for policy 1, policy_version 994891 (0.0008) [2023-12-27 12:32:53,712][20381] Updated weights for policy 1, policy_version 994901 (0.0008) [2023-12-27 12:32:53,992][20347] Updated weights for policy 0, policy_version 995532 (0.0007) [2023-12-27 12:32:54,049][20347] Updated weights for policy 0, policy_version 995542 (0.0008) [2023-12-27 12:32:54,100][20347] Updated weights for policy 0, policy_version 995552 (0.0007) [2023-12-27 12:32:54,342][20381] Updated weights for policy 1, policy_version 994911 (0.0007) [2023-12-27 12:32:54,398][20381] Updated weights for policy 1, policy_version 994921 (0.0008) [2023-12-27 12:32:54,456][20381] Updated weights for policy 1, policy_version 994931 (0.0008) [2023-12-27 12:32:54,803][20347] Updated weights for policy 0, policy_version 995562 (0.0006) [2023-12-27 12:32:54,860][20347] Updated weights for policy 0, policy_version 995572 (0.0008) [2023-12-27 12:32:54,924][20347] Updated weights for policy 0, policy_version 995582 (0.0009) [2023-12-27 12:32:54,986][20347] Updated weights for policy 0, policy_version 995592 (0.0008) [2023-12-27 12:32:55,008][20381] Updated weights for policy 1, policy_version 994941 (0.0007) [2023-12-27 12:32:55,070][20381] Updated weights for policy 1, policy_version 994951 (0.0005) [2023-12-27 12:32:55,131][20381] Updated weights for policy 1, policy_version 994961 (0.0006) [2023-12-27 12:32:55,698][20347] Updated weights for policy 0, policy_version 995602 (0.0007) [2023-12-27 12:32:55,760][20347] Updated weights for policy 0, policy_version 995612 (0.0007) [2023-12-27 12:32:55,786][20381] Updated weights for policy 1, policy_version 994971 (0.0007) [2023-12-27 12:32:55,813][20347] Updated weights for policy 0, policy_version 995622 (0.0007) [2023-12-27 12:32:55,842][20381] Updated weights for policy 1, policy_version 994981 (0.0006) [2023-12-27 12:32:55,895][20381] Updated weights for policy 1, policy_version 994991 (0.0008) [2023-12-27 12:32:56,507][20347] Updated weights for policy 0, policy_version 995632 (0.0007) [2023-12-27 12:32:56,553][20347] Updated weights for policy 0, policy_version 995642 (0.0008) [2023-12-27 12:32:56,595][20381] Updated weights for policy 1, policy_version 995001 (0.0007) [2023-12-27 12:32:56,600][20347] Updated weights for policy 0, policy_version 995652 (0.0008) [2023-12-27 12:32:56,641][20381] Updated weights for policy 1, policy_version 995011 (0.0007) [2023-12-27 12:32:56,692][20381] Updated weights for policy 1, policy_version 995021 (0.0007) [2023-12-27 12:32:56,739][20381] Updated weights for policy 1, policy_version 995031 (0.0008) [2023-12-27 12:32:57,308][20347] Updated weights for policy 0, policy_version 995662 (0.0007) [2023-12-27 12:32:57,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 509689856. Throughput: 0: 10139.8, 1: 10405.1. Samples: 509685632. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:32:57,364][19170] Avg episode reward: [(0, '884.210'), (1, '647.140')] [2023-12-27 12:32:57,367][20347] Updated weights for policy 0, policy_version 995672 (0.0008) [2023-12-27 12:32:57,423][20347] Updated weights for policy 0, policy_version 995682 (0.0007) [2023-12-27 12:32:57,457][20381] Updated weights for policy 1, policy_version 995041 (0.0008) [2023-12-27 12:32:57,508][20381] Updated weights for policy 1, policy_version 995051 (0.0008) [2023-12-27 12:32:57,562][20381] Updated weights for policy 1, policy_version 995061 (0.0007) [2023-12-27 12:32:58,113][20347] Updated weights for policy 0, policy_version 995692 (0.0007) [2023-12-27 12:32:58,169][20347] Updated weights for policy 0, policy_version 995702 (0.0008) [2023-12-27 12:32:58,223][20381] Updated weights for policy 1, policy_version 995071 (0.0007) [2023-12-27 12:32:58,230][20347] Updated weights for policy 0, policy_version 995712 (0.0007) [2023-12-27 12:32:58,280][20381] Updated weights for policy 1, policy_version 995081 (0.0007) [2023-12-27 12:32:58,339][20381] Updated weights for policy 1, policy_version 995091 (0.0008) [2023-12-27 12:32:59,095][20347] Updated weights for policy 0, policy_version 995722 (0.0007) [2023-12-27 12:32:59,100][20381] Updated weights for policy 1, policy_version 995101 (0.0008) [2023-12-27 12:32:59,157][20347] Updated weights for policy 0, policy_version 995732 (0.0007) [2023-12-27 12:32:59,158][20381] Updated weights for policy 1, policy_version 995111 (0.0007) [2023-12-27 12:32:59,222][20381] Updated weights for policy 1, policy_version 995121 (0.0008) [2023-12-27 12:32:59,222][20347] Updated weights for policy 0, policy_version 995742 (0.0007) [2023-12-27 12:32:59,282][20347] Updated weights for policy 0, policy_version 995752 (0.0008) [2023-12-27 12:32:59,956][20381] Updated weights for policy 1, policy_version 995131 (0.0012) [2023-12-27 12:32:59,978][20347] Updated weights for policy 0, policy_version 995762 (0.0008) [2023-12-27 12:33:00,018][20381] Updated weights for policy 1, policy_version 995141 (0.0007) [2023-12-27 12:33:00,040][20347] Updated weights for policy 0, policy_version 995772 (0.0007) [2023-12-27 12:33:00,071][20381] Updated weights for policy 1, policy_version 995151 (0.0009) [2023-12-27 12:33:00,103][20347] Updated weights for policy 0, policy_version 995782 (0.0006) [2023-12-27 12:33:00,690][20381] Updated weights for policy 1, policy_version 995161 (0.0007) [2023-12-27 12:33:00,745][20381] Updated weights for policy 1, policy_version 995171 (0.0006) [2023-12-27 12:33:00,806][20347] Updated weights for policy 0, policy_version 995792 (0.0006) [2023-12-27 12:33:00,808][20381] Updated weights for policy 1, policy_version 995181 (0.0005) [2023-12-27 12:33:00,852][20347] Updated weights for policy 0, policy_version 995802 (0.0005) [2023-12-27 12:33:00,868][20381] Updated weights for policy 1, policy_version 995191 (0.0006) [2023-12-27 12:33:00,900][20347] Updated weights for policy 0, policy_version 995812 (0.0005) [2023-12-27 12:33:01,525][20381] Updated weights for policy 1, policy_version 995201 (0.0005) [2023-12-27 12:33:01,568][20347] Updated weights for policy 0, policy_version 995822 (0.0006) [2023-12-27 12:33:01,586][20381] Updated weights for policy 1, policy_version 995211 (0.0006) [2023-12-27 12:33:01,627][20347] Updated weights for policy 0, policy_version 995832 (0.0007) [2023-12-27 12:33:01,649][20381] Updated weights for policy 1, policy_version 995221 (0.0008) [2023-12-27 12:33:01,681][20347] Updated weights for policy 0, policy_version 995842 (0.0008) [2023-12-27 12:33:02,281][20381] Updated weights for policy 1, policy_version 995231 (0.0008) [2023-12-27 12:33:02,322][20347] Updated weights for policy 0, policy_version 995852 (0.0006) [2023-12-27 12:33:02,343][20381] Updated weights for policy 1, policy_version 995241 (0.0007) [2023-12-27 12:33:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 509788160. Throughput: 0: 10090.9, 1: 10376.4. Samples: 509806380. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:33:02,363][19170] Avg episode reward: [(0, '777.840'), (1, '570.470')] [2023-12-27 12:33:02,387][20347] Updated weights for policy 0, policy_version 995862 (0.0009) [2023-12-27 12:33:02,399][20381] Updated weights for policy 1, policy_version 995251 (0.0007) [2023-12-27 12:33:02,437][20347] Updated weights for policy 0, policy_version 995872 (0.0006) [2023-12-27 12:33:03,026][20381] Updated weights for policy 1, policy_version 995261 (0.0007) [2023-12-27 12:33:03,082][20381] Updated weights for policy 1, policy_version 995271 (0.0005) [2023-12-27 12:33:03,137][20381] Updated weights for policy 1, policy_version 995281 (0.0007) [2023-12-27 12:33:03,146][20347] Updated weights for policy 0, policy_version 995882 (0.0007) [2023-12-27 12:33:03,199][20347] Updated weights for policy 0, policy_version 995892 (0.0006) [2023-12-27 12:33:03,216][20291] KL-divergence is very high: 115.3316 [2023-12-27 12:33:03,233][20291] KL-divergence is very high: 103.2854 [2023-12-27 12:33:03,257][20347] Updated weights for policy 0, policy_version 995902 (0.0007) [2023-12-27 12:33:03,308][20347] Updated weights for policy 0, policy_version 995912 (0.0007) [2023-12-27 12:33:03,728][20381] Updated weights for policy 1, policy_version 995291 (0.0006) [2023-12-27 12:33:03,773][20381] Updated weights for policy 1, policy_version 995301 (0.0007) [2023-12-27 12:33:03,815][20381] Updated weights for policy 1, policy_version 995311 (0.0006) [2023-12-27 12:33:04,058][20347] Updated weights for policy 0, policy_version 995922 (0.0007) [2023-12-27 12:33:04,121][20347] Updated weights for policy 0, policy_version 995932 (0.0006) [2023-12-27 12:33:04,178][20347] Updated weights for policy 0, policy_version 995942 (0.0008) [2023-12-27 12:33:04,486][20381] Updated weights for policy 1, policy_version 995321 (0.0008) [2023-12-27 12:33:04,558][20381] Updated weights for policy 1, policy_version 995331 (0.0008) [2023-12-27 12:33:04,620][20381] Updated weights for policy 1, policy_version 995341 (0.0007) [2023-12-27 12:33:04,686][20381] Updated weights for policy 1, policy_version 995351 (0.0007) [2023-12-27 12:33:04,894][20347] Updated weights for policy 0, policy_version 995952 (0.0007) [2023-12-27 12:33:04,951][20347] Updated weights for policy 0, policy_version 995962 (0.0007) [2023-12-27 12:33:05,011][20347] Updated weights for policy 0, policy_version 995972 (0.0007) [2023-12-27 12:33:05,345][20381] Updated weights for policy 1, policy_version 995361 (0.0008) [2023-12-27 12:33:05,401][20381] Updated weights for policy 1, policy_version 995371 (0.0008) [2023-12-27 12:33:05,449][20381] Updated weights for policy 1, policy_version 995381 (0.0007) [2023-12-27 12:33:05,625][20347] Updated weights for policy 0, policy_version 995982 (0.0009) [2023-12-27 12:33:05,673][20347] Updated weights for policy 0, policy_version 995992 (0.0010) [2023-12-27 12:33:05,718][20347] Updated weights for policy 0, policy_version 996002 (0.0010) [2023-12-27 12:33:06,089][20381] Updated weights for policy 1, policy_version 995391 (0.0007) [2023-12-27 12:33:06,154][20381] Updated weights for policy 1, policy_version 995401 (0.0008) [2023-12-27 12:33:06,215][20381] Updated weights for policy 1, policy_version 995411 (0.0010) [2023-12-27 12:33:06,360][20347] Updated weights for policy 0, policy_version 996012 (0.0010) [2023-12-27 12:33:06,426][20347] Updated weights for policy 0, policy_version 996022 (0.0008) [2023-12-27 12:33:06,486][20347] Updated weights for policy 0, policy_version 996032 (0.0008) [2023-12-27 12:33:06,906][20381] Updated weights for policy 1, policy_version 995421 (0.0008) [2023-12-27 12:33:06,973][20381] Updated weights for policy 1, policy_version 995431 (0.0006) [2023-12-27 12:33:07,040][20381] Updated weights for policy 1, policy_version 995441 (0.0005) [2023-12-27 12:33:07,095][20347] Updated weights for policy 0, policy_version 996042 (0.0007) [2023-12-27 12:33:07,144][20347] Updated weights for policy 0, policy_version 996052 (0.0007) [2023-12-27 12:33:07,194][20347] Updated weights for policy 0, policy_version 996062 (0.0008) [2023-12-27 12:33:07,252][20347] Updated weights for policy 0, policy_version 996072 (0.0008) [2023-12-27 12:33:07,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 509902848. Throughput: 0: 10069.0, 1: 10390.8. Samples: 509866948. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:33:07,363][19170] Avg episode reward: [(0, '776.500'), (1, '637.850')] [2023-12-27 12:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000995448_254861312.pth... [2023-12-27 12:33:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000996072_255041536.pth... [2023-12-27 12:33:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000994232_254550016.pth [2023-12-27 12:33:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000994888_254738432.pth [2023-12-27 12:33:07,590][20381] Updated weights for policy 1, policy_version 995451 (0.0007) [2023-12-27 12:33:07,651][20381] Updated weights for policy 1, policy_version 995461 (0.0008) [2023-12-27 12:33:07,713][20381] Updated weights for policy 1, policy_version 995471 (0.0008) [2023-12-27 12:33:07,986][20347] Updated weights for policy 0, policy_version 996082 (0.0007) [2023-12-27 12:33:08,048][20347] Updated weights for policy 0, policy_version 996092 (0.0007) [2023-12-27 12:33:08,104][20347] Updated weights for policy 0, policy_version 996102 (0.0007) [2023-12-27 12:33:08,395][20381] Updated weights for policy 1, policy_version 995481 (0.0006) [2023-12-27 12:33:08,457][20381] Updated weights for policy 1, policy_version 995491 (0.0008) [2023-12-27 12:33:08,515][20381] Updated weights for policy 1, policy_version 995501 (0.0007) [2023-12-27 12:33:08,574][20381] Updated weights for policy 1, policy_version 995511 (0.0007) [2023-12-27 12:33:08,767][20347] Updated weights for policy 0, policy_version 996112 (0.0008) [2023-12-27 12:33:08,819][20347] Updated weights for policy 0, policy_version 996122 (0.0008) [2023-12-27 12:33:08,871][20347] Updated weights for policy 0, policy_version 996132 (0.0007) [2023-12-27 12:33:09,250][20381] Updated weights for policy 1, policy_version 995521 (0.0008) [2023-12-27 12:33:09,313][20381] Updated weights for policy 1, policy_version 995531 (0.0008) [2023-12-27 12:33:09,380][20381] Updated weights for policy 1, policy_version 995541 (0.0009) [2023-12-27 12:33:09,646][20347] Updated weights for policy 0, policy_version 996142 (0.0009) [2023-12-27 12:33:09,705][20347] Updated weights for policy 0, policy_version 996152 (0.0010) [2023-12-27 12:33:09,762][20347] Updated weights for policy 0, policy_version 996162 (0.0010) [2023-12-27 12:33:10,036][20381] Updated weights for policy 1, policy_version 995551 (0.0006) [2023-12-27 12:33:10,088][20381] Updated weights for policy 1, policy_version 995561 (0.0007) [2023-12-27 12:33:10,141][20381] Updated weights for policy 1, policy_version 995571 (0.0008) [2023-12-27 12:33:10,473][20347] Updated weights for policy 0, policy_version 996172 (0.0009) [2023-12-27 12:33:10,533][20347] Updated weights for policy 0, policy_version 996182 (0.0008) [2023-12-27 12:33:10,591][20347] Updated weights for policy 0, policy_version 996192 (0.0008) [2023-12-27 12:33:10,811][20381] Updated weights for policy 1, policy_version 995581 (0.0007) [2023-12-27 12:33:10,867][20381] Updated weights for policy 1, policy_version 995591 (0.0008) [2023-12-27 12:33:10,929][20381] Updated weights for policy 1, policy_version 995601 (0.0009) [2023-12-27 12:33:11,312][20347] Updated weights for policy 0, policy_version 996202 (0.0007) [2023-12-27 12:33:11,371][20347] Updated weights for policy 0, policy_version 996212 (0.0009) [2023-12-27 12:33:11,441][20347] Updated weights for policy 0, policy_version 996222 (0.0008) [2023-12-27 12:33:11,495][20347] Updated weights for policy 0, policy_version 996232 (0.0007) [2023-12-27 12:33:11,764][20381] Updated weights for policy 1, policy_version 995611 (0.0008) [2023-12-27 12:33:11,823][20381] Updated weights for policy 1, policy_version 995621 (0.0008) [2023-12-27 12:33:11,889][20381] Updated weights for policy 1, policy_version 995631 (0.0008) [2023-12-27 12:33:12,213][20347] Updated weights for policy 0, policy_version 996242 (0.0008) [2023-12-27 12:33:12,266][20347] Updated weights for policy 0, policy_version 996252 (0.0009) [2023-12-27 12:33:12,326][20347] Updated weights for policy 0, policy_version 996262 (0.0007) [2023-12-27 12:33:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 510001152. Throughput: 0: 10098.0, 1: 10390.8. Samples: 509990216. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:33:12,363][19170] Avg episode reward: [(0, '843.940'), (1, '607.680')] [2023-12-27 12:33:12,560][20381] Updated weights for policy 1, policy_version 995641 (0.0008) [2023-12-27 12:33:12,620][20381] Updated weights for policy 1, policy_version 995651 (0.0008) [2023-12-27 12:33:12,665][20381] Updated weights for policy 1, policy_version 995661 (0.0006) [2023-12-27 12:33:12,719][20381] Updated weights for policy 1, policy_version 995671 (0.0007) [2023-12-27 12:33:13,045][20347] Updated weights for policy 0, policy_version 996272 (0.0006) [2023-12-27 12:33:13,091][20347] Updated weights for policy 0, policy_version 996282 (0.0005) [2023-12-27 12:33:13,147][20347] Updated weights for policy 0, policy_version 996292 (0.0005) [2023-12-27 12:33:13,267][20381] Updated weights for policy 1, policy_version 995681 (0.0008) [2023-12-27 12:33:13,313][20381] Updated weights for policy 1, policy_version 995691 (0.0008) [2023-12-27 12:33:13,352][20381] Updated weights for policy 1, policy_version 995701 (0.0006) [2023-12-27 12:33:13,809][20347] Updated weights for policy 0, policy_version 996302 (0.0006) [2023-12-27 12:33:13,860][20347] Updated weights for policy 0, policy_version 996312 (0.0007) [2023-12-27 12:33:13,915][20347] Updated weights for policy 0, policy_version 996322 (0.0007) [2023-12-27 12:33:14,056][20381] Updated weights for policy 1, policy_version 995711 (0.0007) [2023-12-27 12:33:14,114][20381] Updated weights for policy 1, policy_version 995721 (0.0007) [2023-12-27 12:33:14,174][20381] Updated weights for policy 1, policy_version 995731 (0.0007) [2023-12-27 12:33:14,605][20347] Updated weights for policy 0, policy_version 996332 (0.0007) [2023-12-27 12:33:14,663][20347] Updated weights for policy 0, policy_version 996342 (0.0007) [2023-12-27 12:33:14,709][20347] Updated weights for policy 0, policy_version 996352 (0.0010) [2023-12-27 12:33:14,832][20381] Updated weights for policy 1, policy_version 995741 (0.0008) [2023-12-27 12:33:14,895][20381] Updated weights for policy 1, policy_version 995751 (0.0008) [2023-12-27 12:33:14,957][20381] Updated weights for policy 1, policy_version 995761 (0.0008) [2023-12-27 12:33:15,446][20347] Updated weights for policy 0, policy_version 996362 (0.0009) [2023-12-27 12:33:15,496][20347] Updated weights for policy 0, policy_version 996372 (0.0007) [2023-12-27 12:33:15,569][20347] Updated weights for policy 0, policy_version 996382 (0.0007) [2023-12-27 12:33:15,620][20347] Updated weights for policy 0, policy_version 996392 (0.0009) [2023-12-27 12:33:15,646][20381] Updated weights for policy 1, policy_version 995771 (0.0009) [2023-12-27 12:33:15,701][20381] Updated weights for policy 1, policy_version 995781 (0.0010) [2023-12-27 12:33:15,746][20381] Updated weights for policy 1, policy_version 995791 (0.0007) [2023-12-27 12:33:16,340][20381] Updated weights for policy 1, policy_version 995801 (0.0005) [2023-12-27 12:33:16,374][20347] Updated weights for policy 0, policy_version 996402 (0.0007) [2023-12-27 12:33:16,391][20381] Updated weights for policy 1, policy_version 995811 (0.0009) [2023-12-27 12:33:16,430][20347] Updated weights for policy 0, policy_version 996412 (0.0005) [2023-12-27 12:33:16,448][20381] Updated weights for policy 1, policy_version 995821 (0.0010) [2023-12-27 12:33:16,486][20347] Updated weights for policy 0, policy_version 996422 (0.0005) [2023-12-27 12:33:16,503][20381] Updated weights for policy 1, policy_version 995831 (0.0008) [2023-12-27 12:33:17,143][20381] Updated weights for policy 1, policy_version 995841 (0.0010) [2023-12-27 12:33:17,173][20347] Updated weights for policy 0, policy_version 996432 (0.0007) [2023-12-27 12:33:17,195][20381] Updated weights for policy 1, policy_version 995851 (0.0010) [2023-12-27 12:33:17,233][20347] Updated weights for policy 0, policy_version 996442 (0.0007) [2023-12-27 12:33:17,246][20381] Updated weights for policy 1, policy_version 995861 (0.0010) [2023-12-27 12:33:17,288][20347] Updated weights for policy 0, policy_version 996452 (0.0006) [2023-12-27 12:33:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 510107648. Throughput: 0: 10005.5, 1: 10378.9. Samples: 510112000. Policy #0 lag: (min: 31.0, avg: 44.4, max: 63.0) [2023-12-27 12:33:17,363][19170] Avg episode reward: [(0, '739.480'), (1, '598.200')] [2023-12-27 12:33:17,845][20381] Updated weights for policy 1, policy_version 995871 (0.0011) [2023-12-27 12:33:17,894][20381] Updated weights for policy 1, policy_version 995881 (0.0010) [2023-12-27 12:33:17,909][20347] Updated weights for policy 0, policy_version 996462 (0.0006) [2023-12-27 12:33:17,940][20381] Updated weights for policy 1, policy_version 995891 (0.0011) [2023-12-27 12:33:17,967][20347] Updated weights for policy 0, policy_version 996472 (0.0005) [2023-12-27 12:33:18,027][20347] Updated weights for policy 0, policy_version 996482 (0.0006) [2023-12-27 12:33:18,638][20381] Updated weights for policy 1, policy_version 995901 (0.0009) [2023-12-27 12:33:18,653][20347] Updated weights for policy 0, policy_version 996492 (0.0009) [2023-12-27 12:33:18,695][20381] Updated weights for policy 1, policy_version 995911 (0.0008) [2023-12-27 12:33:18,708][20347] Updated weights for policy 0, policy_version 996502 (0.0011) [2023-12-27 12:33:18,746][20381] Updated weights for policy 1, policy_version 995921 (0.0007) [2023-12-27 12:33:18,767][20347] Updated weights for policy 0, policy_version 996512 (0.0010) [2023-12-27 12:33:19,456][20347] Updated weights for policy 0, policy_version 996522 (0.0009) [2023-12-27 12:33:19,485][20381] Updated weights for policy 1, policy_version 995931 (0.0007) [2023-12-27 12:33:19,523][20347] Updated weights for policy 0, policy_version 996532 (0.0006) [2023-12-27 12:33:19,547][20381] Updated weights for policy 1, policy_version 995941 (0.0007) [2023-12-27 12:33:19,584][20347] Updated weights for policy 0, policy_version 996542 (0.0006) [2023-12-27 12:33:19,607][20381] Updated weights for policy 1, policy_version 995951 (0.0007) [2023-12-27 12:33:19,648][20347] Updated weights for policy 0, policy_version 996552 (0.0008) [2023-12-27 12:33:20,335][20381] Updated weights for policy 1, policy_version 995961 (0.0007) [2023-12-27 12:33:20,393][20347] Updated weights for policy 0, policy_version 996562 (0.0006) [2023-12-27 12:33:20,394][20381] Updated weights for policy 1, policy_version 995971 (0.0007) [2023-12-27 12:33:20,452][20347] Updated weights for policy 0, policy_version 996572 (0.0006) [2023-12-27 12:33:20,454][20381] Updated weights for policy 1, policy_version 995981 (0.0006) [2023-12-27 12:33:20,508][20381] Updated weights for policy 1, policy_version 995991 (0.0007) [2023-12-27 12:33:20,511][20347] Updated weights for policy 0, policy_version 996582 (0.0006) [2023-12-27 12:33:21,221][20381] Updated weights for policy 1, policy_version 996001 (0.0007) [2023-12-27 12:33:21,250][20347] Updated weights for policy 0, policy_version 996592 (0.0007) [2023-12-27 12:33:21,284][20381] Updated weights for policy 1, policy_version 996011 (0.0006) [2023-12-27 12:33:21,309][20347] Updated weights for policy 0, policy_version 996602 (0.0007) [2023-12-27 12:33:21,354][20381] Updated weights for policy 1, policy_version 996021 (0.0008) [2023-12-27 12:33:21,378][20347] Updated weights for policy 0, policy_version 996612 (0.0008) [2023-12-27 12:33:22,077][20381] Updated weights for policy 1, policy_version 996031 (0.0008) [2023-12-27 12:33:22,091][20347] Updated weights for policy 0, policy_version 996622 (0.0006) [2023-12-27 12:33:22,137][20381] Updated weights for policy 1, policy_version 996041 (0.0008) [2023-12-27 12:33:22,154][20347] Updated weights for policy 0, policy_version 996632 (0.0006) [2023-12-27 12:33:22,194][20381] Updated weights for policy 1, policy_version 996051 (0.0009) [2023-12-27 12:33:22,218][20347] Updated weights for policy 0, policy_version 996642 (0.0006) [2023-12-27 12:33:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 510205952. Throughput: 0: 10008.7, 1: 10320.7. Samples: 510173188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:22,363][19170] Avg episode reward: [(0, '776.850'), (1, '706.050')] [2023-12-27 12:33:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000996056_255016960.pth... [2023-12-27 12:33:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000996648_255188992.pth... [2023-12-27 12:33:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000994840_254705664.pth [2023-12-27 12:33:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000995496_254894080.pth [2023-12-27 12:33:22,877][20347] Updated weights for policy 0, policy_version 996652 (0.0007) [2023-12-27 12:33:22,937][20347] Updated weights for policy 0, policy_version 996662 (0.0007) [2023-12-27 12:33:22,939][20381] Updated weights for policy 1, policy_version 996061 (0.0009) [2023-12-27 12:33:22,993][20347] Updated weights for policy 0, policy_version 996672 (0.0008) [2023-12-27 12:33:23,001][20381] Updated weights for policy 1, policy_version 996071 (0.0011) [2023-12-27 12:33:23,060][20381] Updated weights for policy 1, policy_version 996081 (0.0010) [2023-12-27 12:33:23,638][20381] Updated weights for policy 1, policy_version 996091 (0.0009) [2023-12-27 12:33:23,644][20347] Updated weights for policy 0, policy_version 996682 (0.0009) [2023-12-27 12:33:23,690][20381] Updated weights for policy 1, policy_version 996101 (0.0006) [2023-12-27 12:33:23,706][20347] Updated weights for policy 0, policy_version 996692 (0.0011) [2023-12-27 12:33:23,742][20381] Updated weights for policy 1, policy_version 996111 (0.0005) [2023-12-27 12:33:23,758][20347] Updated weights for policy 0, policy_version 996702 (0.0010) [2023-12-27 12:33:23,814][20347] Updated weights for policy 0, policy_version 996712 (0.0010) [2023-12-27 12:33:24,406][20381] Updated weights for policy 1, policy_version 996121 (0.0008) [2023-12-27 12:33:24,453][20381] Updated weights for policy 1, policy_version 996131 (0.0006) [2023-12-27 12:33:24,489][20347] Updated weights for policy 0, policy_version 996722 (0.0011) [2023-12-27 12:33:24,510][20381] Updated weights for policy 1, policy_version 996141 (0.0005) [2023-12-27 12:33:24,550][20347] Updated weights for policy 0, policy_version 996732 (0.0011) [2023-12-27 12:33:24,567][20381] Updated weights for policy 1, policy_version 996151 (0.0006) [2023-12-27 12:33:24,608][20347] Updated weights for policy 0, policy_version 996742 (0.0010) [2023-12-27 12:33:25,162][20381] Updated weights for policy 1, policy_version 996161 (0.0008) [2023-12-27 12:33:25,216][20381] Updated weights for policy 1, policy_version 996171 (0.0007) [2023-12-27 12:33:25,262][20347] Updated weights for policy 0, policy_version 996752 (0.0010) [2023-12-27 12:33:25,268][20381] Updated weights for policy 1, policy_version 996181 (0.0005) [2023-12-27 12:33:25,313][20347] Updated weights for policy 0, policy_version 996762 (0.0010) [2023-12-27 12:33:25,367][20347] Updated weights for policy 0, policy_version 996772 (0.0010) [2023-12-27 12:33:25,841][20381] Updated weights for policy 1, policy_version 996191 (0.0005) [2023-12-27 12:33:25,891][20381] Updated weights for policy 1, policy_version 996201 (0.0005) [2023-12-27 12:33:25,947][20381] Updated weights for policy 1, policy_version 996211 (0.0005) [2023-12-27 12:33:26,052][20347] Updated weights for policy 0, policy_version 996782 (0.0010) [2023-12-27 12:33:26,104][20347] Updated weights for policy 0, policy_version 996792 (0.0010) [2023-12-27 12:33:26,161][20347] Updated weights for policy 0, policy_version 996802 (0.0010) [2023-12-27 12:33:26,524][20381] Updated weights for policy 1, policy_version 996221 (0.0007) [2023-12-27 12:33:26,585][20381] Updated weights for policy 1, policy_version 996231 (0.0007) [2023-12-27 12:33:26,635][20381] Updated weights for policy 1, policy_version 996241 (0.0007) [2023-12-27 12:33:26,854][20347] Updated weights for policy 0, policy_version 996812 (0.0008) [2023-12-27 12:33:26,918][20347] Updated weights for policy 0, policy_version 996822 (0.0011) [2023-12-27 12:33:26,980][20347] Updated weights for policy 0, policy_version 996832 (0.0010) [2023-12-27 12:33:27,303][20381] Updated weights for policy 1, policy_version 996251 (0.0008) [2023-12-27 12:33:27,361][20381] Updated weights for policy 1, policy_version 996261 (0.0006) [2023-12-27 12:33:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 510304256. Throughput: 0: 10017.5, 1: 10357.9. Samples: 510296776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:27,363][19170] Avg episode reward: [(0, '766.820'), (1, '735.190')] [2023-12-27 12:33:27,418][20381] Updated weights for policy 1, policy_version 996271 (0.0007) [2023-12-27 12:33:27,583][20347] Updated weights for policy 0, policy_version 996842 (0.0010) [2023-12-27 12:33:27,636][20347] Updated weights for policy 0, policy_version 996852 (0.0007) [2023-12-27 12:33:27,687][20347] Updated weights for policy 0, policy_version 996862 (0.0005) [2023-12-27 12:33:27,744][20347] Updated weights for policy 0, policy_version 996872 (0.0006) [2023-12-27 12:33:27,981][20381] Updated weights for policy 1, policy_version 996281 (0.0007) [2023-12-27 12:33:28,045][20381] Updated weights for policy 1, policy_version 996291 (0.0005) [2023-12-27 12:33:28,099][20381] Updated weights for policy 1, policy_version 996301 (0.0005) [2023-12-27 12:33:28,149][20381] Updated weights for policy 1, policy_version 996311 (0.0006) [2023-12-27 12:33:28,365][20347] Updated weights for policy 0, policy_version 996882 (0.0011) [2023-12-27 12:33:28,419][20347] Updated weights for policy 0, policy_version 996892 (0.0009) [2023-12-27 12:33:28,480][20347] Updated weights for policy 0, policy_version 996902 (0.0008) [2023-12-27 12:33:28,743][20381] Updated weights for policy 1, policy_version 996321 (0.0010) [2023-12-27 12:33:28,791][20381] Updated weights for policy 1, policy_version 996331 (0.0011) [2023-12-27 12:33:28,841][20381] Updated weights for policy 1, policy_version 996341 (0.0011) [2023-12-27 12:33:29,081][20347] Updated weights for policy 0, policy_version 996912 (0.0007) [2023-12-27 12:33:29,136][20347] Updated weights for policy 0, policy_version 996922 (0.0007) [2023-12-27 12:33:29,199][20347] Updated weights for policy 0, policy_version 996932 (0.0008) [2023-12-27 12:33:29,593][20381] Updated weights for policy 1, policy_version 996351 (0.0010) [2023-12-27 12:33:29,649][20381] Updated weights for policy 1, policy_version 996361 (0.0011) [2023-12-27 12:33:29,703][20381] Updated weights for policy 1, policy_version 996371 (0.0010) [2023-12-27 12:33:29,911][20347] Updated weights for policy 0, policy_version 996942 (0.0007) [2023-12-27 12:33:29,976][20347] Updated weights for policy 0, policy_version 996952 (0.0010) [2023-12-27 12:33:30,037][20347] Updated weights for policy 0, policy_version 996962 (0.0008) [2023-12-27 12:33:30,423][20381] Updated weights for policy 1, policy_version 996381 (0.0008) [2023-12-27 12:33:30,488][20381] Updated weights for policy 1, policy_version 996391 (0.0005) [2023-12-27 12:33:30,553][20381] Updated weights for policy 1, policy_version 996401 (0.0005) [2023-12-27 12:33:30,671][20347] Updated weights for policy 0, policy_version 996972 (0.0008) [2023-12-27 12:33:30,716][20347] Updated weights for policy 0, policy_version 996982 (0.0010) [2023-12-27 12:33:30,770][20347] Updated weights for policy 0, policy_version 996992 (0.0010) [2023-12-27 12:33:31,201][20381] Updated weights for policy 1, policy_version 996411 (0.0007) [2023-12-27 12:33:31,260][20381] Updated weights for policy 1, policy_version 996421 (0.0008) [2023-12-27 12:33:31,312][20381] Updated weights for policy 1, policy_version 996431 (0.0008) [2023-12-27 12:33:31,455][20347] Updated weights for policy 0, policy_version 997002 (0.0010) [2023-12-27 12:33:31,524][20347] Updated weights for policy 0, policy_version 997012 (0.0008) [2023-12-27 12:33:31,584][20347] Updated weights for policy 0, policy_version 997022 (0.0008) [2023-12-27 12:33:31,651][20347] Updated weights for policy 0, policy_version 997032 (0.0007) [2023-12-27 12:33:31,984][20381] Updated weights for policy 1, policy_version 996441 (0.0008) [2023-12-27 12:33:32,034][20381] Updated weights for policy 1, policy_version 996451 (0.0008) [2023-12-27 12:33:32,084][20381] Updated weights for policy 1, policy_version 996461 (0.0007) [2023-12-27 12:33:32,135][20381] Updated weights for policy 1, policy_version 996471 (0.0006) [2023-12-27 12:33:32,281][20347] Updated weights for policy 0, policy_version 997042 (0.0008) [2023-12-27 12:33:32,350][20347] Updated weights for policy 0, policy_version 997052 (0.0008) [2023-12-27 12:33:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 510410752. Throughput: 0: 10074.8, 1: 10402.2. Samples: 510423164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:32,363][19170] Avg episode reward: [(0, '787.030'), (1, '745.140')] [2023-12-27 12:33:32,410][20347] Updated weights for policy 0, policy_version 997062 (0.0007) [2023-12-27 12:33:32,856][20381] Updated weights for policy 1, policy_version 996481 (0.0008) [2023-12-27 12:33:32,915][20381] Updated weights for policy 1, policy_version 996491 (0.0008) [2023-12-27 12:33:32,971][20381] Updated weights for policy 1, policy_version 996501 (0.0008) [2023-12-27 12:33:33,057][20347] Updated weights for policy 0, policy_version 997072 (0.0007) [2023-12-27 12:33:33,112][20347] Updated weights for policy 0, policy_version 997082 (0.0007) [2023-12-27 12:33:33,164][20347] Updated weights for policy 0, policy_version 997092 (0.0006) [2023-12-27 12:33:33,674][20381] Updated weights for policy 1, policy_version 996511 (0.0009) [2023-12-27 12:33:33,728][20381] Updated weights for policy 1, policy_version 996521 (0.0007) [2023-12-27 12:33:33,776][20381] Updated weights for policy 1, policy_version 996531 (0.0007) [2023-12-27 12:33:33,799][20347] Updated weights for policy 0, policy_version 997102 (0.0007) [2023-12-27 12:33:33,850][20347] Updated weights for policy 0, policy_version 997112 (0.0007) [2023-12-27 12:33:33,912][20347] Updated weights for policy 0, policy_version 997122 (0.0008) [2023-12-27 12:33:34,443][20381] Updated weights for policy 1, policy_version 996541 (0.0006) [2023-12-27 12:33:34,506][20381] Updated weights for policy 1, policy_version 996551 (0.0007) [2023-12-27 12:33:34,567][20381] Updated weights for policy 1, policy_version 996561 (0.0008) [2023-12-27 12:33:34,637][20347] Updated weights for policy 0, policy_version 997132 (0.0009) [2023-12-27 12:33:34,686][20347] Updated weights for policy 0, policy_version 997142 (0.0010) [2023-12-27 12:33:34,746][20347] Updated weights for policy 0, policy_version 997152 (0.0011) [2023-12-27 12:33:35,296][20381] Updated weights for policy 1, policy_version 996571 (0.0007) [2023-12-27 12:33:35,347][20381] Updated weights for policy 1, policy_version 996581 (0.0007) [2023-12-27 12:33:35,394][20381] Updated weights for policy 1, policy_version 996591 (0.0007) [2023-12-27 12:33:35,456][20347] Updated weights for policy 0, policy_version 997162 (0.0010) [2023-12-27 12:33:35,510][20347] Updated weights for policy 0, policy_version 997172 (0.0007) [2023-12-27 12:33:35,570][20347] Updated weights for policy 0, policy_version 997182 (0.0005) [2023-12-27 12:33:35,626][20347] Updated weights for policy 0, policy_version 997192 (0.0005) [2023-12-27 12:33:36,046][20381] Updated weights for policy 1, policy_version 996601 (0.0009) [2023-12-27 12:33:36,100][20381] Updated weights for policy 1, policy_version 996611 (0.0007) [2023-12-27 12:33:36,168][20381] Updated weights for policy 1, policy_version 996621 (0.0007) [2023-12-27 12:33:36,223][20381] Updated weights for policy 1, policy_version 996631 (0.0006) [2023-12-27 12:33:36,285][20347] Updated weights for policy 0, policy_version 997202 (0.0006) [2023-12-27 12:33:36,354][20347] Updated weights for policy 0, policy_version 997212 (0.0006) [2023-12-27 12:33:36,420][20347] Updated weights for policy 0, policy_version 997222 (0.0006) [2023-12-27 12:33:36,887][20381] Updated weights for policy 1, policy_version 996641 (0.0007) [2023-12-27 12:33:36,951][20381] Updated weights for policy 1, policy_version 996651 (0.0007) [2023-12-27 12:33:37,017][20381] Updated weights for policy 1, policy_version 996661 (0.0007) [2023-12-27 12:33:37,022][20347] Updated weights for policy 0, policy_version 997232 (0.0009) [2023-12-27 12:33:37,073][20347] Updated weights for policy 0, policy_version 997242 (0.0010) [2023-12-27 12:33:37,128][20347] Updated weights for policy 0, policy_version 997252 (0.0010) [2023-12-27 12:33:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.1, 300 sec: 20521.6). Total num frames: 510517248. Throughput: 0: 10081.9, 1: 10363.0. Samples: 510483704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:37,363][19170] Avg episode reward: [(0, '804.690'), (1, '686.440')] [2023-12-27 12:33:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000996664_255172608.pth... [2023-12-27 12:33:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000997256_255344640.pth... [2023-12-27 12:33:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000996072_255041536.pth [2023-12-27 12:33:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000995448_254861312.pth [2023-12-27 12:33:37,581][20381] Updated weights for policy 1, policy_version 996671 (0.0005) [2023-12-27 12:33:37,642][20381] Updated weights for policy 1, policy_version 996681 (0.0007) [2023-12-27 12:33:37,697][20381] Updated weights for policy 1, policy_version 996691 (0.0007) [2023-12-27 12:33:37,829][20347] Updated weights for policy 0, policy_version 997262 (0.0010) [2023-12-27 12:33:37,884][20347] Updated weights for policy 0, policy_version 997272 (0.0010) [2023-12-27 12:33:37,936][20347] Updated weights for policy 0, policy_version 997282 (0.0010) [2023-12-27 12:33:38,377][20381] Updated weights for policy 1, policy_version 996701 (0.0008) [2023-12-27 12:33:38,429][20381] Updated weights for policy 1, policy_version 996711 (0.0007) [2023-12-27 12:33:38,481][20381] Updated weights for policy 1, policy_version 996721 (0.0007) [2023-12-27 12:33:38,644][20347] Updated weights for policy 0, policy_version 997292 (0.0010) [2023-12-27 12:33:38,695][20347] Updated weights for policy 0, policy_version 997302 (0.0010) [2023-12-27 12:33:38,764][20347] Updated weights for policy 0, policy_version 997312 (0.0010) [2023-12-27 12:33:39,181][20381] Updated weights for policy 1, policy_version 996731 (0.0007) [2023-12-27 12:33:39,244][20381] Updated weights for policy 1, policy_version 996741 (0.0008) [2023-12-27 12:33:39,294][20381] Updated weights for policy 1, policy_version 996751 (0.0007) [2023-12-27 12:33:39,452][20347] Updated weights for policy 0, policy_version 997322 (0.0010) [2023-12-27 12:33:39,502][20347] Updated weights for policy 0, policy_version 997332 (0.0011) [2023-12-27 12:33:39,548][20347] Updated weights for policy 0, policy_version 997342 (0.0010) [2023-12-27 12:33:39,595][20347] Updated weights for policy 0, policy_version 997352 (0.0010) [2023-12-27 12:33:40,100][20381] Updated weights for policy 1, policy_version 996761 (0.0008) [2023-12-27 12:33:40,170][20381] Updated weights for policy 1, policy_version 996771 (0.0006) [2023-12-27 12:33:40,229][20381] Updated weights for policy 1, policy_version 996781 (0.0008) [2023-12-27 12:33:40,290][20381] Updated weights for policy 1, policy_version 996791 (0.0008) [2023-12-27 12:33:40,385][20347] Updated weights for policy 0, policy_version 997362 (0.0008) [2023-12-27 12:33:40,450][20347] Updated weights for policy 0, policy_version 997372 (0.0008) [2023-12-27 12:33:40,497][20347] Updated weights for policy 0, policy_version 997382 (0.0008) [2023-12-27 12:33:40,953][20381] Updated weights for policy 1, policy_version 996801 (0.0008) [2023-12-27 12:33:41,009][20381] Updated weights for policy 1, policy_version 996811 (0.0007) [2023-12-27 12:33:41,077][20381] Updated weights for policy 1, policy_version 996821 (0.0008) [2023-12-27 12:33:41,194][20347] Updated weights for policy 0, policy_version 997392 (0.0008) [2023-12-27 12:33:41,257][20347] Updated weights for policy 0, policy_version 997402 (0.0007) [2023-12-27 12:33:41,320][20347] Updated weights for policy 0, policy_version 997412 (0.0008) [2023-12-27 12:33:41,889][20381] Updated weights for policy 1, policy_version 996831 (0.0007) [2023-12-27 12:33:41,948][20381] Updated weights for policy 1, policy_version 996841 (0.0008) [2023-12-27 12:33:42,012][20381] Updated weights for policy 1, policy_version 996851 (0.0008) [2023-12-27 12:33:42,048][20347] Updated weights for policy 0, policy_version 997422 (0.0007) [2023-12-27 12:33:42,108][20347] Updated weights for policy 0, policy_version 997432 (0.0005) [2023-12-27 12:33:42,170][20347] Updated weights for policy 0, policy_version 997442 (0.0005) [2023-12-27 12:33:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 510615552. Throughput: 0: 10102.7, 1: 10327.9. Samples: 510605004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:42,363][19170] Avg episode reward: [(0, '809.650'), (1, '676.480')] [2023-12-27 12:33:42,758][20381] Updated weights for policy 1, policy_version 996861 (0.0008) [2023-12-27 12:33:42,819][20381] Updated weights for policy 1, policy_version 996871 (0.0007) [2023-12-27 12:33:42,820][20347] Updated weights for policy 0, policy_version 997452 (0.0007) [2023-12-27 12:33:42,878][20347] Updated weights for policy 0, policy_version 997462 (0.0008) [2023-12-27 12:33:42,879][20381] Updated weights for policy 1, policy_version 996881 (0.0005) [2023-12-27 12:33:42,941][20347] Updated weights for policy 0, policy_version 997472 (0.0007) [2023-12-27 12:33:43,546][20381] Updated weights for policy 1, policy_version 996891 (0.0007) [2023-12-27 12:33:43,601][20381] Updated weights for policy 1, policy_version 996901 (0.0009) [2023-12-27 12:33:43,623][20347] Updated weights for policy 0, policy_version 997482 (0.0007) [2023-12-27 12:33:43,662][20381] Updated weights for policy 1, policy_version 996911 (0.0006) [2023-12-27 12:33:43,679][20347] Updated weights for policy 0, policy_version 997492 (0.0006) [2023-12-27 12:33:43,729][20347] Updated weights for policy 0, policy_version 997502 (0.0007) [2023-12-27 12:33:43,784][20347] Updated weights for policy 0, policy_version 997512 (0.0008) [2023-12-27 12:33:44,369][20381] Updated weights for policy 1, policy_version 996921 (0.0007) [2023-12-27 12:33:44,425][20381] Updated weights for policy 1, policy_version 996931 (0.0008) [2023-12-27 12:33:44,438][20347] Updated weights for policy 0, policy_version 997522 (0.0006) [2023-12-27 12:33:44,479][20381] Updated weights for policy 1, policy_version 996941 (0.0006) [2023-12-27 12:33:44,488][20347] Updated weights for policy 0, policy_version 997532 (0.0006) [2023-12-27 12:33:44,535][20381] Updated weights for policy 1, policy_version 996951 (0.0005) [2023-12-27 12:33:44,537][20347] Updated weights for policy 0, policy_version 997542 (0.0007) [2023-12-27 12:33:45,240][20347] Updated weights for policy 0, policy_version 997552 (0.0006) [2023-12-27 12:33:45,254][20381] Updated weights for policy 1, policy_version 996961 (0.0010) [2023-12-27 12:33:45,295][20347] Updated weights for policy 0, policy_version 997562 (0.0007) [2023-12-27 12:33:45,304][20381] Updated weights for policy 1, policy_version 996971 (0.0011) [2023-12-27 12:33:45,347][20347] Updated weights for policy 0, policy_version 997572 (0.0005) [2023-12-27 12:33:45,357][20381] Updated weights for policy 1, policy_version 996981 (0.0011) [2023-12-27 12:33:46,043][20381] Updated weights for policy 1, policy_version 996991 (0.0008) [2023-12-27 12:33:46,054][20347] Updated weights for policy 0, policy_version 997582 (0.0007) [2023-12-27 12:33:46,106][20381] Updated weights for policy 1, policy_version 997001 (0.0006) [2023-12-27 12:33:46,108][20347] Updated weights for policy 0, policy_version 997592 (0.0007) [2023-12-27 12:33:46,163][20381] Updated weights for policy 1, policy_version 997011 (0.0005) [2023-12-27 12:33:46,167][20347] Updated weights for policy 0, policy_version 997602 (0.0007) [2023-12-27 12:33:46,812][20381] Updated weights for policy 1, policy_version 997021 (0.0007) [2023-12-27 12:33:46,857][20347] Updated weights for policy 0, policy_version 997612 (0.0006) [2023-12-27 12:33:46,870][20381] Updated weights for policy 1, policy_version 997031 (0.0007) [2023-12-27 12:33:46,907][20347] Updated weights for policy 0, policy_version 997622 (0.0006) [2023-12-27 12:33:46,920][20381] Updated weights for policy 1, policy_version 997041 (0.0005) [2023-12-27 12:33:46,962][20347] Updated weights for policy 0, policy_version 997632 (0.0007) [2023-12-27 12:33:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 510713856. Throughput: 0: 10119.7, 1: 10288.6. Samples: 510724752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:47,363][19170] Avg episode reward: [(0, '781.570'), (1, '607.570')] [2023-12-27 12:33:47,590][20381] Updated weights for policy 1, policy_version 997051 (0.0008) [2023-12-27 12:33:47,650][20381] Updated weights for policy 1, policy_version 997061 (0.0006) [2023-12-27 12:33:47,651][20347] Updated weights for policy 0, policy_version 997642 (0.0006) [2023-12-27 12:33:47,705][20381] Updated weights for policy 1, policy_version 997071 (0.0005) [2023-12-27 12:33:47,719][20347] Updated weights for policy 0, policy_version 997652 (0.0008) [2023-12-27 12:33:47,778][20347] Updated weights for policy 0, policy_version 997662 (0.0011) [2023-12-27 12:33:47,837][20347] Updated weights for policy 0, policy_version 997672 (0.0010) [2023-12-27 12:33:48,372][20381] Updated weights for policy 1, policy_version 997081 (0.0008) [2023-12-27 12:33:48,425][20381] Updated weights for policy 1, policy_version 997091 (0.0007) [2023-12-27 12:33:48,442][20347] Updated weights for policy 0, policy_version 997682 (0.0010) [2023-12-27 12:33:48,480][20381] Updated weights for policy 1, policy_version 997101 (0.0006) [2023-12-27 12:33:48,505][20347] Updated weights for policy 0, policy_version 997692 (0.0010) [2023-12-27 12:33:48,532][20381] Updated weights for policy 1, policy_version 997111 (0.0006) [2023-12-27 12:33:48,563][20347] Updated weights for policy 0, policy_version 997702 (0.0010) [2023-12-27 12:33:49,245][20347] Updated weights for policy 0, policy_version 997712 (0.0010) [2023-12-27 12:33:49,247][20381] Updated weights for policy 1, policy_version 997121 (0.0007) [2023-12-27 12:33:49,302][20381] Updated weights for policy 1, policy_version 997131 (0.0006) [2023-12-27 12:33:49,308][20347] Updated weights for policy 0, policy_version 997722 (0.0010) [2023-12-27 12:33:49,362][20381] Updated weights for policy 1, policy_version 997141 (0.0007) [2023-12-27 12:33:49,369][20347] Updated weights for policy 0, policy_version 997732 (0.0008) [2023-12-27 12:33:50,040][20381] Updated weights for policy 1, policy_version 997151 (0.0009) [2023-12-27 12:33:50,066][20347] Updated weights for policy 0, policy_version 997742 (0.0008) [2023-12-27 12:33:50,095][20381] Updated weights for policy 1, policy_version 997161 (0.0006) [2023-12-27 12:33:50,126][20347] Updated weights for policy 0, policy_version 997752 (0.0008) [2023-12-27 12:33:50,153][20381] Updated weights for policy 1, policy_version 997171 (0.0006) [2023-12-27 12:33:50,186][20347] Updated weights for policy 0, policy_version 997762 (0.0007) [2023-12-27 12:33:50,844][20347] Updated weights for policy 0, policy_version 997772 (0.0005) [2023-12-27 12:33:50,889][20381] Updated weights for policy 1, policy_version 997181 (0.0006) [2023-12-27 12:33:50,905][20347] Updated weights for policy 0, policy_version 997782 (0.0007) [2023-12-27 12:33:50,957][20381] Updated weights for policy 1, policy_version 997191 (0.0006) [2023-12-27 12:33:50,960][20347] Updated weights for policy 0, policy_version 997792 (0.0007) [2023-12-27 12:33:51,022][20381] Updated weights for policy 1, policy_version 997201 (0.0008) [2023-12-27 12:33:51,639][20347] Updated weights for policy 0, policy_version 997802 (0.0006) [2023-12-27 12:33:51,686][20347] Updated weights for policy 0, policy_version 997812 (0.0007) [2023-12-27 12:33:51,747][20347] Updated weights for policy 0, policy_version 997822 (0.0007) [2023-12-27 12:33:51,757][20381] Updated weights for policy 1, policy_version 997211 (0.0008) [2023-12-27 12:33:51,802][20347] Updated weights for policy 0, policy_version 997832 (0.0009) [2023-12-27 12:33:51,820][20381] Updated weights for policy 1, policy_version 997221 (0.0008) [2023-12-27 12:33:51,882][20381] Updated weights for policy 1, policy_version 997231 (0.0008) [2023-12-27 12:33:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 510812160. Throughput: 0: 10162.0, 1: 10252.0. Samples: 510785580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:52,363][19170] Avg episode reward: [(0, '765.130'), (1, '637.290')] [2023-12-27 12:33:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000997240_255320064.pth... [2023-12-27 12:33:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000997832_255492096.pth... [2023-12-27 12:33:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000996056_255016960.pth [2023-12-27 12:33:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000996648_255188992.pth [2023-12-27 12:33:52,516][20347] Updated weights for policy 0, policy_version 997842 (0.0007) [2023-12-27 12:33:52,562][20347] Updated weights for policy 0, policy_version 997852 (0.0007) [2023-12-27 12:33:52,588][20381] Updated weights for policy 1, policy_version 997241 (0.0008) [2023-12-27 12:33:52,615][20347] Updated weights for policy 0, policy_version 997862 (0.0006) [2023-12-27 12:33:52,647][20381] Updated weights for policy 1, policy_version 997251 (0.0008) [2023-12-27 12:33:52,706][20381] Updated weights for policy 1, policy_version 997261 (0.0008) [2023-12-27 12:33:52,766][20381] Updated weights for policy 1, policy_version 997271 (0.0007) [2023-12-27 12:33:53,369][20347] Updated weights for policy 0, policy_version 997872 (0.0007) [2023-12-27 12:33:53,402][20381] Updated weights for policy 1, policy_version 997281 (0.0007) [2023-12-27 12:33:53,427][20347] Updated weights for policy 0, policy_version 997882 (0.0007) [2023-12-27 12:33:53,449][20381] Updated weights for policy 1, policy_version 997291 (0.0006) [2023-12-27 12:33:53,482][20347] Updated weights for policy 0, policy_version 997892 (0.0007) [2023-12-27 12:33:53,507][20381] Updated weights for policy 1, policy_version 997301 (0.0005) [2023-12-27 12:33:54,159][20381] Updated weights for policy 1, policy_version 997311 (0.0007) [2023-12-27 12:33:54,180][20347] Updated weights for policy 0, policy_version 997902 (0.0007) [2023-12-27 12:33:54,216][20381] Updated weights for policy 1, policy_version 997321 (0.0006) [2023-12-27 12:33:54,234][20347] Updated weights for policy 0, policy_version 997912 (0.0006) [2023-12-27 12:33:54,274][20381] Updated weights for policy 1, policy_version 997331 (0.0006) [2023-12-27 12:33:54,287][20347] Updated weights for policy 0, policy_version 997922 (0.0006) [2023-12-27 12:33:54,960][20381] Updated weights for policy 1, policy_version 997341 (0.0007) [2023-12-27 12:33:54,967][20347] Updated weights for policy 0, policy_version 997932 (0.0007) [2023-12-27 12:33:55,017][20347] Updated weights for policy 0, policy_version 997942 (0.0007) [2023-12-27 12:33:55,021][20381] Updated weights for policy 1, policy_version 997351 (0.0006) [2023-12-27 12:33:55,072][20347] Updated weights for policy 0, policy_version 997952 (0.0009) [2023-12-27 12:33:55,078][20381] Updated weights for policy 1, policy_version 997361 (0.0007) [2023-12-27 12:33:55,761][20381] Updated weights for policy 1, policy_version 997371 (0.0006) [2023-12-27 12:33:55,768][20347] Updated weights for policy 0, policy_version 997962 (0.0006) [2023-12-27 12:33:55,808][20381] Updated weights for policy 1, policy_version 997381 (0.0005) [2023-12-27 12:33:55,817][20347] Updated weights for policy 0, policy_version 997972 (0.0006) [2023-12-27 12:33:55,860][20381] Updated weights for policy 1, policy_version 997391 (0.0006) [2023-12-27 12:33:55,865][20347] Updated weights for policy 0, policy_version 997982 (0.0005) [2023-12-27 12:33:55,912][20347] Updated weights for policy 0, policy_version 997992 (0.0006) [2023-12-27 12:33:56,575][20381] Updated weights for policy 1, policy_version 997401 (0.0006) [2023-12-27 12:33:56,586][20347] Updated weights for policy 0, policy_version 998002 (0.0007) [2023-12-27 12:33:56,621][20381] Updated weights for policy 1, policy_version 997411 (0.0007) [2023-12-27 12:33:56,630][20347] Updated weights for policy 0, policy_version 998012 (0.0005) [2023-12-27 12:33:56,667][20381] Updated weights for policy 1, policy_version 997421 (0.0006) [2023-12-27 12:33:56,687][20347] Updated weights for policy 0, policy_version 998022 (0.0006) [2023-12-27 12:33:56,709][20381] Updated weights for policy 1, policy_version 997431 (0.0006) [2023-12-27 12:33:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.6, 300 sec: 20466.1). Total num frames: 510910464. Throughput: 0: 10142.3, 1: 10205.5. Samples: 510905868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:33:57,363][19170] Avg episode reward: [(0, '801.370'), (1, '686.490')] [2023-12-27 12:33:57,382][20381] Updated weights for policy 1, policy_version 997441 (0.0007) [2023-12-27 12:33:57,384][20347] Updated weights for policy 0, policy_version 998032 (0.0006) [2023-12-27 12:33:57,430][20381] Updated weights for policy 1, policy_version 997451 (0.0006) [2023-12-27 12:33:57,439][20347] Updated weights for policy 0, policy_version 998042 (0.0007) [2023-12-27 12:33:57,475][20381] Updated weights for policy 1, policy_version 997461 (0.0006) [2023-12-27 12:33:57,498][20347] Updated weights for policy 0, policy_version 998052 (0.0007) [2023-12-27 12:33:58,191][20381] Updated weights for policy 1, policy_version 997471 (0.0008) [2023-12-27 12:33:58,203][20347] Updated weights for policy 0, policy_version 998062 (0.0008) [2023-12-27 12:33:58,253][20381] Updated weights for policy 1, policy_version 997481 (0.0008) [2023-12-27 12:33:58,259][20347] Updated weights for policy 0, policy_version 998072 (0.0007) [2023-12-27 12:33:58,317][20347] Updated weights for policy 0, policy_version 998082 (0.0007) [2023-12-27 12:33:58,318][20381] Updated weights for policy 1, policy_version 997491 (0.0007) [2023-12-27 12:33:59,078][20381] Updated weights for policy 1, policy_version 997501 (0.0008) [2023-12-27 12:33:59,140][20381] Updated weights for policy 1, policy_version 997511 (0.0009) [2023-12-27 12:33:59,158][20347] Updated weights for policy 0, policy_version 998092 (0.0008) [2023-12-27 12:33:59,201][20381] Updated weights for policy 1, policy_version 997521 (0.0007) [2023-12-27 12:33:59,218][20347] Updated weights for policy 0, policy_version 998102 (0.0006) [2023-12-27 12:33:59,286][20347] Updated weights for policy 0, policy_version 998112 (0.0007) [2023-12-27 12:33:59,896][20347] Updated weights for policy 0, policy_version 998122 (0.0007) [2023-12-27 12:33:59,948][20381] Updated weights for policy 1, policy_version 997531 (0.0008) [2023-12-27 12:33:59,960][20347] Updated weights for policy 0, policy_version 998132 (0.0007) [2023-12-27 12:34:00,004][20381] Updated weights for policy 1, policy_version 997541 (0.0007) [2023-12-27 12:34:00,021][20347] Updated weights for policy 0, policy_version 998142 (0.0007) [2023-12-27 12:34:00,064][20381] Updated weights for policy 1, policy_version 997551 (0.0008) [2023-12-27 12:34:00,073][20347] Updated weights for policy 0, policy_version 998152 (0.0006) [2023-12-27 12:34:00,695][20347] Updated weights for policy 0, policy_version 998162 (0.0005) [2023-12-27 12:34:00,742][20347] Updated weights for policy 0, policy_version 998172 (0.0005) [2023-12-27 12:34:00,789][20347] Updated weights for policy 0, policy_version 998182 (0.0006) [2023-12-27 12:34:00,806][20381] Updated weights for policy 1, policy_version 997561 (0.0008) [2023-12-27 12:34:00,851][20381] Updated weights for policy 1, policy_version 997571 (0.0005) [2023-12-27 12:34:00,898][20381] Updated weights for policy 1, policy_version 997581 (0.0007) [2023-12-27 12:34:00,953][20381] Updated weights for policy 1, policy_version 997591 (0.0007) [2023-12-27 12:34:01,458][20347] Updated weights for policy 0, policy_version 998192 (0.0007) [2023-12-27 12:34:01,512][20347] Updated weights for policy 0, policy_version 998202 (0.0007) [2023-12-27 12:34:01,568][20347] Updated weights for policy 0, policy_version 998212 (0.0008) [2023-12-27 12:34:01,639][20381] Updated weights for policy 1, policy_version 997601 (0.0008) [2023-12-27 12:34:01,701][20381] Updated weights for policy 1, policy_version 997611 (0.0007) [2023-12-27 12:34:01,760][20381] Updated weights for policy 1, policy_version 997621 (0.0009) [2023-12-27 12:34:02,242][20347] Updated weights for policy 0, policy_version 998222 (0.0007) [2023-12-27 12:34:02,311][20347] Updated weights for policy 0, policy_version 998232 (0.0008) [2023-12-27 12:34:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 511008768. Throughput: 0: 10176.1, 1: 10106.6. Samples: 511024720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:02,363][19170] Avg episode reward: [(0, '861.610'), (1, '686.490')] [2023-12-27 12:34:02,373][20347] Updated weights for policy 0, policy_version 998242 (0.0007) [2023-12-27 12:34:02,513][20381] Updated weights for policy 1, policy_version 997631 (0.0008) [2023-12-27 12:34:02,569][20381] Updated weights for policy 1, policy_version 997641 (0.0008) [2023-12-27 12:34:02,620][20381] Updated weights for policy 1, policy_version 997651 (0.0007) [2023-12-27 12:34:03,051][20347] Updated weights for policy 0, policy_version 998252 (0.0008) [2023-12-27 12:34:03,104][20347] Updated weights for policy 0, policy_version 998262 (0.0010) [2023-12-27 12:34:03,160][20347] Updated weights for policy 0, policy_version 998272 (0.0008) [2023-12-27 12:34:03,326][20381] Updated weights for policy 1, policy_version 997661 (0.0007) [2023-12-27 12:34:03,373][20381] Updated weights for policy 1, policy_version 997671 (0.0007) [2023-12-27 12:34:03,428][20381] Updated weights for policy 1, policy_version 997681 (0.0007) [2023-12-27 12:34:03,724][20347] Updated weights for policy 0, policy_version 998282 (0.0006) [2023-12-27 12:34:03,768][20347] Updated weights for policy 0, policy_version 998292 (0.0007) [2023-12-27 12:34:03,823][20347] Updated weights for policy 0, policy_version 998302 (0.0005) [2023-12-27 12:34:03,887][20347] Updated weights for policy 0, policy_version 998312 (0.0008) [2023-12-27 12:34:04,130][20381] Updated weights for policy 1, policy_version 997691 (0.0006) [2023-12-27 12:34:04,195][20381] Updated weights for policy 1, policy_version 997701 (0.0008) [2023-12-27 12:34:04,260][20381] Updated weights for policy 1, policy_version 997711 (0.0008) [2023-12-27 12:34:04,571][20347] Updated weights for policy 0, policy_version 998322 (0.0010) [2023-12-27 12:34:04,594][20291] KL-divergence is very high: 116.6655 [2023-12-27 12:34:04,627][20347] Updated weights for policy 0, policy_version 998332 (0.0009) [2023-12-27 12:34:04,685][20347] Updated weights for policy 0, policy_version 998342 (0.0010) [2023-12-27 12:34:04,907][20381] Updated weights for policy 1, policy_version 997721 (0.0007) [2023-12-27 12:34:04,971][20381] Updated weights for policy 1, policy_version 997731 (0.0006) [2023-12-27 12:34:05,041][20381] Updated weights for policy 1, policy_version 997741 (0.0007) [2023-12-27 12:34:05,102][20381] Updated weights for policy 1, policy_version 997751 (0.0008) [2023-12-27 12:34:05,321][20347] Updated weights for policy 0, policy_version 998352 (0.0010) [2023-12-27 12:34:05,374][20347] Updated weights for policy 0, policy_version 998362 (0.0010) [2023-12-27 12:34:05,423][20347] Updated weights for policy 0, policy_version 998372 (0.0010) [2023-12-27 12:34:05,696][20381] Updated weights for policy 1, policy_version 997761 (0.0010) [2023-12-27 12:34:05,758][20381] Updated weights for policy 1, policy_version 997771 (0.0007) [2023-12-27 12:34:05,814][20381] Updated weights for policy 1, policy_version 997781 (0.0007) [2023-12-27 12:34:06,083][20347] Updated weights for policy 0, policy_version 998382 (0.0010) [2023-12-27 12:34:06,151][20347] Updated weights for policy 0, policy_version 998392 (0.0010) [2023-12-27 12:34:06,209][20347] Updated weights for policy 0, policy_version 998402 (0.0010) [2023-12-27 12:34:06,553][20381] Updated weights for policy 1, policy_version 997791 (0.0010) [2023-12-27 12:34:06,622][20381] Updated weights for policy 1, policy_version 997801 (0.0011) [2023-12-27 12:34:06,687][20381] Updated weights for policy 1, policy_version 997811 (0.0011) [2023-12-27 12:34:06,936][20347] Updated weights for policy 0, policy_version 998412 (0.0009) [2023-12-27 12:34:06,987][20347] Updated weights for policy 0, policy_version 998422 (0.0008) [2023-12-27 12:34:07,053][20347] Updated weights for policy 0, policy_version 998432 (0.0007) [2023-12-27 12:34:07,287][20381] Updated weights for policy 1, policy_version 997821 (0.0008) [2023-12-27 12:34:07,349][20381] Updated weights for policy 1, policy_version 997831 (0.0011) [2023-12-27 12:34:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 511115264. Throughput: 0: 10189.6, 1: 10110.9. Samples: 511086712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:07,363][19170] Avg episode reward: [(0, '843.950'), (1, '784.420')] [2023-12-27 12:34:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000998440_255647744.pth... [2023-12-27 12:34:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000997256_255344640.pth [2023-12-27 12:34:07,407][20381] Updated weights for policy 1, policy_version 997841 (0.0011) [2023-12-27 12:34:07,440][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000997848_255475712.pth... [2023-12-27 12:34:07,443][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000996664_255172608.pth [2023-12-27 12:34:07,697][20347] Updated weights for policy 0, policy_version 998442 (0.0008) [2023-12-27 12:34:07,749][20347] Updated weights for policy 0, policy_version 998452 (0.0010) [2023-12-27 12:34:07,810][20347] Updated weights for policy 0, policy_version 998462 (0.0010) [2023-12-27 12:34:07,864][20347] Updated weights for policy 0, policy_version 998472 (0.0010) [2023-12-27 12:34:08,064][20381] Updated weights for policy 1, policy_version 997851 (0.0010) [2023-12-27 12:34:08,116][20381] Updated weights for policy 1, policy_version 997861 (0.0010) [2023-12-27 12:34:08,170][20381] Updated weights for policy 1, policy_version 997871 (0.0010) [2023-12-27 12:34:08,549][20347] Updated weights for policy 0, policy_version 998482 (0.0010) [2023-12-27 12:34:08,605][20347] Updated weights for policy 0, policy_version 998492 (0.0010) [2023-12-27 12:34:08,664][20347] Updated weights for policy 0, policy_version 998502 (0.0010) [2023-12-27 12:34:08,856][20381] Updated weights for policy 1, policy_version 997881 (0.0010) [2023-12-27 12:34:08,908][20381] Updated weights for policy 1, policy_version 997891 (0.0010) [2023-12-27 12:34:08,958][20381] Updated weights for policy 1, policy_version 997901 (0.0006) [2023-12-27 12:34:09,012][20381] Updated weights for policy 1, policy_version 997911 (0.0008) [2023-12-27 12:34:09,373][20347] Updated weights for policy 0, policy_version 998512 (0.0010) [2023-12-27 12:34:09,435][20347] Updated weights for policy 0, policy_version 998522 (0.0007) [2023-12-27 12:34:09,497][20347] Updated weights for policy 0, policy_version 998532 (0.0007) [2023-12-27 12:34:09,706][20381] Updated weights for policy 1, policy_version 997921 (0.0008) [2023-12-27 12:34:09,769][20381] Updated weights for policy 1, policy_version 997931 (0.0007) [2023-12-27 12:34:09,847][20381] Updated weights for policy 1, policy_version 997941 (0.0008) [2023-12-27 12:34:10,181][20347] Updated weights for policy 0, policy_version 998542 (0.0009) [2023-12-27 12:34:10,228][20347] Updated weights for policy 0, policy_version 998552 (0.0010) [2023-12-27 12:34:10,289][20347] Updated weights for policy 0, policy_version 998562 (0.0011) [2023-12-27 12:34:10,520][20381] Updated weights for policy 1, policy_version 997951 (0.0010) [2023-12-27 12:34:10,573][20381] Updated weights for policy 1, policy_version 997961 (0.0009) [2023-12-27 12:34:10,623][20381] Updated weights for policy 1, policy_version 997971 (0.0011) [2023-12-27 12:34:10,963][20347] Updated weights for policy 0, policy_version 998572 (0.0008) [2023-12-27 12:34:11,015][20347] Updated weights for policy 0, policy_version 998582 (0.0008) [2023-12-27 12:34:11,078][20347] Updated weights for policy 0, policy_version 998592 (0.0011) [2023-12-27 12:34:11,315][20381] Updated weights for policy 1, policy_version 997981 (0.0010) [2023-12-27 12:34:11,383][20381] Updated weights for policy 1, policy_version 997991 (0.0010) [2023-12-27 12:34:11,443][20381] Updated weights for policy 1, policy_version 998001 (0.0008) [2023-12-27 12:34:11,867][20347] Updated weights for policy 0, policy_version 998602 (0.0009) [2023-12-27 12:34:11,920][20347] Updated weights for policy 0, policy_version 998612 (0.0007) [2023-12-27 12:34:11,982][20347] Updated weights for policy 0, policy_version 998622 (0.0007) [2023-12-27 12:34:12,041][20347] Updated weights for policy 0, policy_version 998632 (0.0007) [2023-12-27 12:34:12,145][20381] Updated weights for policy 1, policy_version 998011 (0.0007) [2023-12-27 12:34:12,205][20381] Updated weights for policy 1, policy_version 998021 (0.0005) [2023-12-27 12:34:12,272][20381] Updated weights for policy 1, policy_version 998031 (0.0010) [2023-12-27 12:34:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 511221760. Throughput: 0: 10199.9, 1: 10080.1. Samples: 511209376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:12,363][19170] Avg episode reward: [(0, '767.480'), (1, '814.030')] [2023-12-27 12:34:12,758][20347] Updated weights for policy 0, policy_version 998642 (0.0008) [2023-12-27 12:34:12,822][20347] Updated weights for policy 0, policy_version 998652 (0.0007) [2023-12-27 12:34:12,877][20347] Updated weights for policy 0, policy_version 998662 (0.0008) [2023-12-27 12:34:12,948][20381] Updated weights for policy 1, policy_version 998041 (0.0008) [2023-12-27 12:34:12,994][20381] Updated weights for policy 1, policy_version 998051 (0.0010) [2023-12-27 12:34:13,044][20381] Updated weights for policy 1, policy_version 998061 (0.0011) [2023-12-27 12:34:13,098][20381] Updated weights for policy 1, policy_version 998071 (0.0011) [2023-12-27 12:34:13,480][20347] Updated weights for policy 0, policy_version 998672 (0.0009) [2023-12-27 12:34:13,535][20347] Updated weights for policy 0, policy_version 998682 (0.0011) [2023-12-27 12:34:13,592][20347] Updated weights for policy 0, policy_version 998692 (0.0010) [2023-12-27 12:34:13,773][20381] Updated weights for policy 1, policy_version 998081 (0.0006) [2023-12-27 12:34:13,828][20381] Updated weights for policy 1, policy_version 998091 (0.0008) [2023-12-27 12:34:13,877][20381] Updated weights for policy 1, policy_version 998101 (0.0006) [2023-12-27 12:34:14,284][20347] Updated weights for policy 0, policy_version 998702 (0.0009) [2023-12-27 12:34:14,341][20347] Updated weights for policy 0, policy_version 998712 (0.0007) [2023-12-27 12:34:14,395][20347] Updated weights for policy 0, policy_version 998722 (0.0007) [2023-12-27 12:34:14,485][20381] Updated weights for policy 1, policy_version 998111 (0.0009) [2023-12-27 12:34:14,540][20381] Updated weights for policy 1, policy_version 998121 (0.0010) [2023-12-27 12:34:14,594][20381] Updated weights for policy 1, policy_version 998131 (0.0010) [2023-12-27 12:34:15,116][20347] Updated weights for policy 0, policy_version 998732 (0.0008) [2023-12-27 12:34:15,185][20347] Updated weights for policy 0, policy_version 998742 (0.0008) [2023-12-27 12:34:15,247][20347] Updated weights for policy 0, policy_version 998752 (0.0008) [2023-12-27 12:34:15,309][20381] Updated weights for policy 1, policy_version 998141 (0.0011) [2023-12-27 12:34:15,362][20381] Updated weights for policy 1, policy_version 998151 (0.0011) [2023-12-27 12:34:15,411][20381] Updated weights for policy 1, policy_version 998161 (0.0010) [2023-12-27 12:34:15,933][20347] Updated weights for policy 0, policy_version 998762 (0.0008) [2023-12-27 12:34:15,998][20347] Updated weights for policy 0, policy_version 998772 (0.0009) [2023-12-27 12:34:16,051][20347] Updated weights for policy 0, policy_version 998782 (0.0010) [2023-12-27 12:34:16,061][20381] Updated weights for policy 1, policy_version 998171 (0.0010) [2023-12-27 12:34:16,097][20347] Updated weights for policy 0, policy_version 998792 (0.0006) [2023-12-27 12:34:16,113][20381] Updated weights for policy 1, policy_version 998181 (0.0010) [2023-12-27 12:34:16,168][20381] Updated weights for policy 1, policy_version 998191 (0.0010) [2023-12-27 12:34:16,646][20347] Updated weights for policy 0, policy_version 998802 (0.0010) [2023-12-27 12:34:16,703][20347] Updated weights for policy 0, policy_version 998812 (0.0010) [2023-12-27 12:34:16,750][20347] Updated weights for policy 0, policy_version 998822 (0.0010) [2023-12-27 12:34:16,751][20291] KL-divergence is very high: 201.7427 [2023-12-27 12:34:16,870][20381] Updated weights for policy 1, policy_version 998201 (0.0010) [2023-12-27 12:34:16,939][20381] Updated weights for policy 1, policy_version 998211 (0.0009) [2023-12-27 12:34:16,985][20381] Updated weights for policy 1, policy_version 998221 (0.0010) [2023-12-27 12:34:17,031][20381] Updated weights for policy 1, policy_version 998231 (0.0007) [2023-12-27 12:34:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 511320064. Throughput: 0: 10161.2, 1: 10054.7. Samples: 511332884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:17,363][19170] Avg episode reward: [(0, '805.110'), (1, '784.600')] [2023-12-27 12:34:17,373][20291] KL-divergence is very high: 286.2325 [2023-12-27 12:34:17,378][20291] KL-divergence is very high: 301.1967 [2023-12-27 12:34:17,397][20291] KL-divergence is very high: 107.6378 [2023-12-27 12:34:17,414][20347] Updated weights for policy 0, policy_version 998832 (0.0008) [2023-12-27 12:34:17,421][20291] KL-divergence is very high: 188.2438 [2023-12-27 12:34:17,459][20291] KL-divergence is very high: 168.5033 [2023-12-27 12:34:17,461][20347] Updated weights for policy 0, policy_version 998842 (0.0010) [2023-12-27 12:34:17,495][20291] KL-divergence is very high: 189.8490 [2023-12-27 12:34:17,506][20347] Updated weights for policy 0, policy_version 998852 (0.0009) [2023-12-27 12:34:17,589][20381] Updated weights for policy 1, policy_version 998241 (0.0007) [2023-12-27 12:34:17,650][20381] Updated weights for policy 1, policy_version 998251 (0.0007) [2023-12-27 12:34:17,703][20381] Updated weights for policy 1, policy_version 998261 (0.0008) [2023-12-27 12:34:18,222][20347] Updated weights for policy 0, policy_version 998862 (0.0007) [2023-12-27 12:34:18,223][20291] KL-divergence is very high: 253.0263 [2023-12-27 12:34:18,240][20291] KL-divergence is very high: 379.9713 [2023-12-27 12:34:18,268][20291] KL-divergence is very high: 303.6992 [2023-12-27 12:34:18,280][20347] Updated weights for policy 0, policy_version 998872 (0.0006) [2023-12-27 12:34:18,286][20291] KL-divergence is very high: 152.6083 [2023-12-27 12:34:18,335][20291] KL-divergence is very high: 140.2783 [2023-12-27 12:34:18,341][20347] Updated weights for policy 0, policy_version 998882 (0.0007) [2023-12-27 12:34:18,369][20381] Updated weights for policy 1, policy_version 998271 (0.0008) [2023-12-27 12:34:18,429][20381] Updated weights for policy 1, policy_version 998281 (0.0007) [2023-12-27 12:34:18,484][20381] Updated weights for policy 1, policy_version 998291 (0.0007) [2023-12-27 12:34:18,951][20347] Updated weights for policy 0, policy_version 998892 (0.0006) [2023-12-27 12:34:19,001][20347] Updated weights for policy 0, policy_version 998902 (0.0005) [2023-12-27 12:34:19,052][20347] Updated weights for policy 0, policy_version 998912 (0.0005) [2023-12-27 12:34:19,128][20381] Updated weights for policy 1, policy_version 998301 (0.0008) [2023-12-27 12:34:19,192][20381] Updated weights for policy 1, policy_version 998311 (0.0008) [2023-12-27 12:34:19,254][20381] Updated weights for policy 1, policy_version 998321 (0.0009) [2023-12-27 12:34:19,724][20347] Updated weights for policy 0, policy_version 998922 (0.0006) [2023-12-27 12:34:19,777][20347] Updated weights for policy 0, policy_version 998932 (0.0009) [2023-12-27 12:34:19,834][20347] Updated weights for policy 0, policy_version 998942 (0.0008) [2023-12-27 12:34:19,897][20347] Updated weights for policy 0, policy_version 998952 (0.0007) [2023-12-27 12:34:19,987][20381] Updated weights for policy 1, policy_version 998331 (0.0007) [2023-12-27 12:34:20,047][20381] Updated weights for policy 1, policy_version 998341 (0.0006) [2023-12-27 12:34:20,113][20381] Updated weights for policy 1, policy_version 998351 (0.0007) [2023-12-27 12:34:20,639][20347] Updated weights for policy 0, policy_version 998962 (0.0006) [2023-12-27 12:34:20,706][20347] Updated weights for policy 0, policy_version 998972 (0.0008) [2023-12-27 12:34:20,770][20381] Updated weights for policy 1, policy_version 998361 (0.0007) [2023-12-27 12:34:20,774][20347] Updated weights for policy 0, policy_version 998982 (0.0008) [2023-12-27 12:34:20,835][20381] Updated weights for policy 1, policy_version 998371 (0.0006) [2023-12-27 12:34:20,888][20381] Updated weights for policy 1, policy_version 998381 (0.0008) [2023-12-27 12:34:20,940][20381] Updated weights for policy 1, policy_version 998391 (0.0008) [2023-12-27 12:34:21,446][20347] Updated weights for policy 0, policy_version 998992 (0.0008) [2023-12-27 12:34:21,499][20347] Updated weights for policy 0, policy_version 999002 (0.0005) [2023-12-27 12:34:21,560][20347] Updated weights for policy 0, policy_version 999012 (0.0006) [2023-12-27 12:34:21,614][20381] Updated weights for policy 1, policy_version 998401 (0.0007) [2023-12-27 12:34:21,674][20381] Updated weights for policy 1, policy_version 998411 (0.0007) [2023-12-27 12:34:21,738][20381] Updated weights for policy 1, policy_version 998421 (0.0008) [2023-12-27 12:34:22,279][20347] Updated weights for policy 0, policy_version 999022 (0.0009) [2023-12-27 12:34:22,293][20291] KL-divergence is very high: 219.4911 [2023-12-27 12:34:22,340][20347] Updated weights for policy 0, policy_version 999032 (0.0006) [2023-12-27 12:34:22,342][20291] KL-divergence is very high: 279.9148 [2023-12-27 12:34:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 511418368. Throughput: 0: 10161.2, 1: 10082.2. Samples: 511394660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:22,363][19170] Avg episode reward: [(0, '825.350'), (1, '764.710')] [2023-12-27 12:34:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000998424_255623168.pth... [2023-12-27 12:34:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000997240_255320064.pth [2023-12-27 12:34:22,397][20291] KL-divergence is very high: 192.0517 [2023-12-27 12:34:22,409][20347] Updated weights for policy 0, policy_version 999042 (0.0008) [2023-12-27 12:34:22,444][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000999048_255803392.pth... [2023-12-27 12:34:22,448][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000997832_255492096.pth [2023-12-27 12:34:22,449][20381] Updated weights for policy 1, policy_version 998431 (0.0007) [2023-12-27 12:34:22,509][20381] Updated weights for policy 1, policy_version 998441 (0.0008) [2023-12-27 12:34:22,517][20292] KL-divergence is very high: 107.7556 [2023-12-27 12:34:22,562][20381] Updated weights for policy 1, policy_version 998451 (0.0008) [2023-12-27 12:34:23,134][20347] Updated weights for policy 0, policy_version 999052 (0.0008) [2023-12-27 12:34:23,172][20291] KL-divergence is very high: 102.6946 [2023-12-27 12:34:23,185][20347] Updated weights for policy 0, policy_version 999062 (0.0007) [2023-12-27 12:34:23,254][20347] Updated weights for policy 0, policy_version 999072 (0.0007) [2023-12-27 12:34:23,256][20381] Updated weights for policy 1, policy_version 998461 (0.0009) [2023-12-27 12:34:23,315][20381] Updated weights for policy 1, policy_version 998471 (0.0005) [2023-12-27 12:34:23,351][20292] KL-divergence is very high: 148.8616 [2023-12-27 12:34:23,372][20381] Updated weights for policy 1, policy_version 998481 (0.0005) [2023-12-27 12:34:23,927][20291] KL-divergence is very high: 113.3183 [2023-12-27 12:34:23,940][20347] Updated weights for policy 0, policy_version 999082 (0.0007) [2023-12-27 12:34:23,943][20291] KL-divergence is very high: 143.9288 [2023-12-27 12:34:23,950][20381] Updated weights for policy 1, policy_version 998491 (0.0007) [2023-12-27 12:34:23,966][20291] KL-divergence is very high: 149.9241 [2023-12-27 12:34:23,987][20347] Updated weights for policy 0, policy_version 999092 (0.0008) [2023-12-27 12:34:24,007][20381] Updated weights for policy 1, policy_version 998501 (0.0010) [2023-12-27 12:34:24,034][20347] Updated weights for policy 0, policy_version 999102 (0.0006) [2023-12-27 12:34:24,047][20291] KL-divergence is very high: 118.7767 [2023-12-27 12:34:24,065][20381] Updated weights for policy 1, policy_version 998511 (0.0011) [2023-12-27 12:34:24,090][20347] Updated weights for policy 0, policy_version 999112 (0.0005) [2023-12-27 12:34:24,675][20347] Updated weights for policy 0, policy_version 999122 (0.0007) [2023-12-27 12:34:24,718][20381] Updated weights for policy 1, policy_version 998521 (0.0010) [2023-12-27 12:34:24,732][20347] Updated weights for policy 0, policy_version 999132 (0.0009) [2023-12-27 12:34:24,773][20381] Updated weights for policy 1, policy_version 998531 (0.0011) [2023-12-27 12:34:24,780][20347] Updated weights for policy 0, policy_version 999142 (0.0010) [2023-12-27 12:34:24,825][20381] Updated weights for policy 1, policy_version 998541 (0.0010) [2023-12-27 12:34:24,871][20381] Updated weights for policy 1, policy_version 998551 (0.0007) [2023-12-27 12:34:25,412][20347] Updated weights for policy 0, policy_version 999152 (0.0010) [2023-12-27 12:34:25,473][20347] Updated weights for policy 0, policy_version 999162 (0.0010) [2023-12-27 12:34:25,525][20347] Updated weights for policy 0, policy_version 999172 (0.0010) [2023-12-27 12:34:25,526][20381] Updated weights for policy 1, policy_version 998561 (0.0005) [2023-12-27 12:34:25,572][20381] Updated weights for policy 1, policy_version 998571 (0.0006) [2023-12-27 12:34:25,619][20381] Updated weights for policy 1, policy_version 998581 (0.0007) [2023-12-27 12:34:26,207][20347] Updated weights for policy 0, policy_version 999182 (0.0010) [2023-12-27 12:34:26,262][20347] Updated weights for policy 0, policy_version 999192 (0.0010) [2023-12-27 12:34:26,315][20347] Updated weights for policy 0, policy_version 999202 (0.0010) [2023-12-27 12:34:26,339][20381] Updated weights for policy 1, policy_version 998591 (0.0006) [2023-12-27 12:34:26,395][20381] Updated weights for policy 1, policy_version 998601 (0.0007) [2023-12-27 12:34:26,441][20381] Updated weights for policy 1, policy_version 998611 (0.0007) [2023-12-27 12:34:26,987][20347] Updated weights for policy 0, policy_version 999212 (0.0009) [2023-12-27 12:34:27,031][20347] Updated weights for policy 0, policy_version 999222 (0.0009) [2023-12-27 12:34:27,078][20347] Updated weights for policy 0, policy_version 999232 (0.0008) [2023-12-27 12:34:27,078][20381] Updated weights for policy 1, policy_version 998621 (0.0007) [2023-12-27 12:34:27,132][20381] Updated weights for policy 1, policy_version 998631 (0.0008) [2023-12-27 12:34:27,189][20381] Updated weights for policy 1, policy_version 998641 (0.0007) [2023-12-27 12:34:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 511533056. Throughput: 0: 10177.1, 1: 10116.1. Samples: 511518196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:27,363][19170] Avg episode reward: [(0, '816.270'), (1, '579.310')] [2023-12-27 12:34:27,735][20347] Updated weights for policy 0, policy_version 999242 (0.0007) [2023-12-27 12:34:27,791][20347] Updated weights for policy 0, policy_version 999252 (0.0006) [2023-12-27 12:34:27,846][20347] Updated weights for policy 0, policy_version 999262 (0.0008) [2023-12-27 12:34:27,884][20381] Updated weights for policy 1, policy_version 998651 (0.0008) [2023-12-27 12:34:27,900][20347] Updated weights for policy 0, policy_version 999272 (0.0006) [2023-12-27 12:34:27,933][20381] Updated weights for policy 1, policy_version 998661 (0.0007) [2023-12-27 12:34:27,979][20381] Updated weights for policy 1, policy_version 998671 (0.0010) [2023-12-27 12:34:28,511][20347] Updated weights for policy 0, policy_version 999282 (0.0006) [2023-12-27 12:34:28,571][20347] Updated weights for policy 0, policy_version 999292 (0.0006) [2023-12-27 12:34:28,625][20347] Updated weights for policy 0, policy_version 999302 (0.0008) [2023-12-27 12:34:28,732][20381] Updated weights for policy 1, policy_version 998681 (0.0010) [2023-12-27 12:34:28,782][20381] Updated weights for policy 1, policy_version 998691 (0.0005) [2023-12-27 12:34:28,834][20381] Updated weights for policy 1, policy_version 998701 (0.0007) [2023-12-27 12:34:28,885][20381] Updated weights for policy 1, policy_version 998711 (0.0010) [2023-12-27 12:34:29,299][20347] Updated weights for policy 0, policy_version 999312 (0.0010) [2023-12-27 12:34:29,352][20347] Updated weights for policy 0, policy_version 999322 (0.0010) [2023-12-27 12:34:29,422][20347] Updated weights for policy 0, policy_version 999332 (0.0010) [2023-12-27 12:34:29,580][20381] Updated weights for policy 1, policy_version 998721 (0.0008) [2023-12-27 12:34:29,640][20381] Updated weights for policy 1, policy_version 998731 (0.0006) [2023-12-27 12:34:29,695][20381] Updated weights for policy 1, policy_version 998741 (0.0005) [2023-12-27 12:34:30,121][20347] Updated weights for policy 0, policy_version 999342 (0.0010) [2023-12-27 12:34:30,176][20347] Updated weights for policy 0, policy_version 999352 (0.0010) [2023-12-27 12:34:30,235][20347] Updated weights for policy 0, policy_version 999362 (0.0010) [2023-12-27 12:34:30,317][20381] Updated weights for policy 1, policy_version 998751 (0.0006) [2023-12-27 12:34:30,374][20381] Updated weights for policy 1, policy_version 998761 (0.0007) [2023-12-27 12:34:30,429][20381] Updated weights for policy 1, policy_version 998771 (0.0008) [2023-12-27 12:34:30,841][20347] Updated weights for policy 0, policy_version 999372 (0.0008) [2023-12-27 12:34:30,897][20347] Updated weights for policy 0, policy_version 999382 (0.0005) [2023-12-27 12:34:30,955][20347] Updated weights for policy 0, policy_version 999392 (0.0005) [2023-12-27 12:34:31,127][20381] Updated weights for policy 1, policy_version 998781 (0.0007) [2023-12-27 12:34:31,192][20381] Updated weights for policy 1, policy_version 998791 (0.0007) [2023-12-27 12:34:31,261][20381] Updated weights for policy 1, policy_version 998801 (0.0008) [2023-12-27 12:34:31,614][20347] Updated weights for policy 0, policy_version 999402 (0.0006) [2023-12-27 12:34:31,679][20347] Updated weights for policy 0, policy_version 999412 (0.0008) [2023-12-27 12:34:31,743][20347] Updated weights for policy 0, policy_version 999422 (0.0008) [2023-12-27 12:34:31,807][20347] Updated weights for policy 0, policy_version 999432 (0.0008) [2023-12-27 12:34:31,935][20381] Updated weights for policy 1, policy_version 998811 (0.0008) [2023-12-27 12:34:31,953][20292] KL-divergence is very high: 104.6180 [2023-12-27 12:34:31,984][20381] Updated weights for policy 1, policy_version 998821 (0.0008) [2023-12-27 12:34:31,994][20292] KL-divergence is very high: 139.3304 [2023-12-27 12:34:32,012][20292] KL-divergence is very high: 103.0344 [2023-12-27 12:34:32,018][20292] KL-divergence is very high: 143.1144 [2023-12-27 12:34:32,024][20292] KL-divergence is very high: 134.5650 [2023-12-27 12:34:32,039][20381] Updated weights for policy 1, policy_version 998831 (0.0010) [2023-12-27 12:34:32,059][20292] KL-divergence is very high: 119.1096 [2023-12-27 12:34:32,064][20292] KL-divergence is very high: 127.2220 [2023-12-27 12:34:32,364][19170] Fps is (10 sec: 21296.5, 60 sec: 20343.0, 300 sec: 20466.0). Total num frames: 511631360. Throughput: 0: 10255.0, 1: 10148.9. Samples: 511642956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:32,365][19170] Avg episode reward: [(0, '708.050'), (1, '45.010')] [2023-12-27 12:34:32,402][20347] Updated weights for policy 0, policy_version 999442 (0.0009) [2023-12-27 12:34:32,468][20347] Updated weights for policy 0, policy_version 999452 (0.0009) [2023-12-27 12:34:32,525][20347] Updated weights for policy 0, policy_version 999462 (0.0007) [2023-12-27 12:34:32,699][20381] Updated weights for policy 1, policy_version 998841 (0.0010) [2023-12-27 12:34:32,755][20381] Updated weights for policy 1, policy_version 998851 (0.0005) [2023-12-27 12:34:32,813][20381] Updated weights for policy 1, policy_version 998861 (0.0007) [2023-12-27 12:34:32,868][20381] Updated weights for policy 1, policy_version 998871 (0.0006) [2023-12-27 12:34:33,220][20347] Updated weights for policy 0, policy_version 999472 (0.0007) [2023-12-27 12:34:33,281][20347] Updated weights for policy 0, policy_version 999482 (0.0007) [2023-12-27 12:34:33,326][20347] Updated weights for policy 0, policy_version 999492 (0.0009) [2023-12-27 12:34:33,418][20381] Updated weights for policy 1, policy_version 998881 (0.0005) [2023-12-27 12:34:33,470][20381] Updated weights for policy 1, policy_version 998891 (0.0005) [2023-12-27 12:34:33,521][20381] Updated weights for policy 1, policy_version 998901 (0.0006) [2023-12-27 12:34:33,978][20347] Updated weights for policy 0, policy_version 999502 (0.0006) [2023-12-27 12:34:34,026][20347] Updated weights for policy 0, policy_version 999512 (0.0005) [2023-12-27 12:34:34,075][20347] Updated weights for policy 0, policy_version 999522 (0.0005) [2023-12-27 12:34:34,141][20381] Updated weights for policy 1, policy_version 998911 (0.0007) [2023-12-27 12:34:34,204][20381] Updated weights for policy 1, policy_version 998921 (0.0007) [2023-12-27 12:34:34,254][20381] Updated weights for policy 1, policy_version 998931 (0.0007) [2023-12-27 12:34:34,722][20347] Updated weights for policy 0, policy_version 999532 (0.0007) [2023-12-27 12:34:34,781][20347] Updated weights for policy 0, policy_version 999542 (0.0007) [2023-12-27 12:34:34,835][20347] Updated weights for policy 0, policy_version 999552 (0.0005) [2023-12-27 12:34:34,957][20381] Updated weights for policy 1, policy_version 998941 (0.0007) [2023-12-27 12:34:35,009][20381] Updated weights for policy 1, policy_version 998951 (0.0008) [2023-12-27 12:34:35,060][20381] Updated weights for policy 1, policy_version 998961 (0.0007) [2023-12-27 12:34:35,531][20347] Updated weights for policy 0, policy_version 999562 (0.0006) [2023-12-27 12:34:35,589][20347] Updated weights for policy 0, policy_version 999572 (0.0007) [2023-12-27 12:34:35,646][20347] Updated weights for policy 0, policy_version 999582 (0.0007) [2023-12-27 12:34:35,700][20347] Updated weights for policy 0, policy_version 999592 (0.0007) [2023-12-27 12:34:35,752][20381] Updated weights for policy 1, policy_version 998971 (0.0007) [2023-12-27 12:34:35,807][20381] Updated weights for policy 1, policy_version 998981 (0.0010) [2023-12-27 12:34:35,863][20381] Updated weights for policy 1, policy_version 998991 (0.0010) [2023-12-27 12:34:36,342][20347] Updated weights for policy 0, policy_version 999602 (0.0006) [2023-12-27 12:34:36,395][20347] Updated weights for policy 0, policy_version 999612 (0.0006) [2023-12-27 12:34:36,461][20347] Updated weights for policy 0, policy_version 999622 (0.0009) [2023-12-27 12:34:36,556][20381] Updated weights for policy 1, policy_version 999001 (0.0010) [2023-12-27 12:34:36,614][20381] Updated weights for policy 1, policy_version 999011 (0.0010) [2023-12-27 12:34:36,675][20381] Updated weights for policy 1, policy_version 999021 (0.0011) [2023-12-27 12:34:36,734][20381] Updated weights for policy 1, policy_version 999031 (0.0010) [2023-12-27 12:34:37,121][20347] Updated weights for policy 0, policy_version 999632 (0.0011) [2023-12-27 12:34:37,179][20347] Updated weights for policy 0, policy_version 999642 (0.0010) [2023-12-27 12:34:37,235][20347] Updated weights for policy 0, policy_version 999652 (0.0010) [2023-12-27 12:34:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 511737856. Throughput: 0: 10275.2, 1: 10214.1. Samples: 511707596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:37,363][19170] Avg episode reward: [(0, '786.940'), (1, '122.670')] [2023-12-27 12:34:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000999656_255959040.pth... [2023-12-27 12:34:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000998440_255647744.pth [2023-12-27 12:34:37,399][20381] Updated weights for policy 1, policy_version 999041 (0.0010) [2023-12-27 12:34:37,457][20381] Updated weights for policy 1, policy_version 999051 (0.0006) [2023-12-27 12:34:37,517][20381] Updated weights for policy 1, policy_version 999061 (0.0011) [2023-12-27 12:34:37,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000999064_255787008.pth... [2023-12-27 12:34:37,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000997848_255475712.pth [2023-12-27 12:34:37,891][20347] Updated weights for policy 0, policy_version 999662 (0.0005) [2023-12-27 12:34:37,936][20347] Updated weights for policy 0, policy_version 999672 (0.0005) [2023-12-27 12:34:37,982][20347] Updated weights for policy 0, policy_version 999682 (0.0005) [2023-12-27 12:34:38,208][20381] Updated weights for policy 1, policy_version 999071 (0.0011) [2023-12-27 12:34:38,265][20381] Updated weights for policy 1, policy_version 999081 (0.0008) [2023-12-27 12:34:38,314][20381] Updated weights for policy 1, policy_version 999091 (0.0010) [2023-12-27 12:34:38,660][20347] Updated weights for policy 0, policy_version 999692 (0.0008) [2023-12-27 12:34:38,723][20347] Updated weights for policy 0, policy_version 999702 (0.0010) [2023-12-27 12:34:38,784][20347] Updated weights for policy 0, policy_version 999712 (0.0010) [2023-12-27 12:34:39,010][20381] Updated weights for policy 1, policy_version 999101 (0.0009) [2023-12-27 12:34:39,068][20381] Updated weights for policy 1, policy_version 999111 (0.0007) [2023-12-27 12:34:39,127][20381] Updated weights for policy 1, policy_version 999121 (0.0007) [2023-12-27 12:34:39,452][20347] Updated weights for policy 0, policy_version 999722 (0.0010) [2023-12-27 12:34:39,505][20347] Updated weights for policy 0, policy_version 999732 (0.0008) [2023-12-27 12:34:39,558][20347] Updated weights for policy 0, policy_version 999742 (0.0009) [2023-12-27 12:34:39,613][20347] Updated weights for policy 0, policy_version 999752 (0.0008) [2023-12-27 12:34:39,859][20381] Updated weights for policy 1, policy_version 999131 (0.0008) [2023-12-27 12:34:39,924][20381] Updated weights for policy 1, policy_version 999141 (0.0008) [2023-12-27 12:34:39,989][20381] Updated weights for policy 1, policy_version 999151 (0.0008) [2023-12-27 12:34:40,346][20347] Updated weights for policy 0, policy_version 999762 (0.0007) [2023-12-27 12:34:40,406][20347] Updated weights for policy 0, policy_version 999772 (0.0008) [2023-12-27 12:34:40,470][20347] Updated weights for policy 0, policy_version 999782 (0.0008) [2023-12-27 12:34:40,608][20381] Updated weights for policy 1, policy_version 999161 (0.0006) [2023-12-27 12:34:40,662][20381] Updated weights for policy 1, policy_version 999171 (0.0006) [2023-12-27 12:34:40,715][20381] Updated weights for policy 1, policy_version 999181 (0.0009) [2023-12-27 12:34:40,769][20381] Updated weights for policy 1, policy_version 999191 (0.0010) [2023-12-27 12:34:41,111][20347] Updated weights for policy 0, policy_version 999792 (0.0007) [2023-12-27 12:34:41,181][20347] Updated weights for policy 0, policy_version 999802 (0.0009) [2023-12-27 12:34:41,238][20347] Updated weights for policy 0, policy_version 999812 (0.0010) [2023-12-27 12:34:41,480][20381] Updated weights for policy 1, policy_version 999201 (0.0008) [2023-12-27 12:34:41,534][20381] Updated weights for policy 1, policy_version 999211 (0.0006) [2023-12-27 12:34:41,587][20381] Updated weights for policy 1, policy_version 999221 (0.0006) [2023-12-27 12:34:42,005][20347] Updated weights for policy 0, policy_version 999822 (0.0009) [2023-12-27 12:34:42,064][20347] Updated weights for policy 0, policy_version 999832 (0.0007) [2023-12-27 12:34:42,116][20347] Updated weights for policy 0, policy_version 999842 (0.0008) [2023-12-27 12:34:42,267][20381] Updated weights for policy 1, policy_version 999231 (0.0008) [2023-12-27 12:34:42,330][20381] Updated weights for policy 1, policy_version 999241 (0.0010) [2023-12-27 12:34:42,362][19170] Fps is (10 sec: 20482.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 511836160. Throughput: 0: 10301.3, 1: 10231.5. Samples: 511829844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:42,363][19170] Avg episode reward: [(0, '711.390'), (1, '135.430')] [2023-12-27 12:34:42,389][20381] Updated weights for policy 1, policy_version 999251 (0.0008) [2023-12-27 12:34:42,791][20347] Updated weights for policy 0, policy_version 999852 (0.0008) [2023-12-27 12:34:42,841][20347] Updated weights for policy 0, policy_version 999862 (0.0007) [2023-12-27 12:34:42,889][20347] Updated weights for policy 0, policy_version 999872 (0.0008) [2023-12-27 12:34:43,070][20381] Updated weights for policy 1, policy_version 999261 (0.0008) [2023-12-27 12:34:43,122][20381] Updated weights for policy 1, policy_version 999271 (0.0008) [2023-12-27 12:34:43,167][20381] Updated weights for policy 1, policy_version 999281 (0.0007) [2023-12-27 12:34:43,639][20347] Updated weights for policy 0, policy_version 999882 (0.0009) [2023-12-27 12:34:43,689][20347] Updated weights for policy 0, policy_version 999892 (0.0009) [2023-12-27 12:34:43,749][20347] Updated weights for policy 0, policy_version 999902 (0.0009) [2023-12-27 12:34:43,772][20381] Updated weights for policy 1, policy_version 999291 (0.0005) [2023-12-27 12:34:43,803][20347] Updated weights for policy 0, policy_version 999912 (0.0008) [2023-12-27 12:34:43,835][20381] Updated weights for policy 1, policy_version 999301 (0.0005) [2023-12-27 12:34:43,889][20381] Updated weights for policy 1, policy_version 999311 (0.0005) [2023-12-27 12:34:44,466][20381] Updated weights for policy 1, policy_version 999321 (0.0005) [2023-12-27 12:34:44,523][20381] Updated weights for policy 1, policy_version 999331 (0.0008) [2023-12-27 12:34:44,549][20347] Updated weights for policy 0, policy_version 999922 (0.0010) [2023-12-27 12:34:44,578][20381] Updated weights for policy 1, policy_version 999341 (0.0007) [2023-12-27 12:34:44,602][20347] Updated weights for policy 0, policy_version 999932 (0.0011) [2023-12-27 12:34:44,638][20381] Updated weights for policy 1, policy_version 999351 (0.0005) [2023-12-27 12:34:44,655][20347] Updated weights for policy 0, policy_version 999942 (0.0011) [2023-12-27 12:34:45,363][20347] Updated weights for policy 0, policy_version 999952 (0.0008) [2023-12-27 12:34:45,373][20381] Updated weights for policy 1, policy_version 999361 (0.0006) [2023-12-27 12:34:45,413][20347] Updated weights for policy 0, policy_version 999962 (0.0006) [2023-12-27 12:34:45,422][20381] Updated weights for policy 1, policy_version 999371 (0.0006) [2023-12-27 12:34:45,466][20347] Updated weights for policy 0, policy_version 999972 (0.0005) [2023-12-27 12:34:45,471][20381] Updated weights for policy 1, policy_version 999381 (0.0007) [2023-12-27 12:34:46,126][20347] Updated weights for policy 0, policy_version 999982 (0.0005) [2023-12-27 12:34:46,143][20381] Updated weights for policy 1, policy_version 999391 (0.0006) [2023-12-27 12:34:46,180][20347] Updated weights for policy 0, policy_version 999992 (0.0006) [2023-12-27 12:34:46,201][20381] Updated weights for policy 1, policy_version 999401 (0.0006) [2023-12-27 12:34:46,233][20347] Updated weights for policy 0, policy_version 1000002 (0.0006) [2023-12-27 12:34:46,257][20381] Updated weights for policy 1, policy_version 999411 (0.0005) [2023-12-27 12:34:46,798][20347] Updated weights for policy 0, policy_version 1000012 (0.0006) [2023-12-27 12:34:46,853][20347] Updated weights for policy 0, policy_version 1000022 (0.0009) [2023-12-27 12:34:46,907][20347] Updated weights for policy 0, policy_version 1000032 (0.0010) [2023-12-27 12:34:46,968][20381] Updated weights for policy 1, policy_version 999421 (0.0008) [2023-12-27 12:34:47,020][20381] Updated weights for policy 1, policy_version 999431 (0.0009) [2023-12-27 12:34:47,077][20381] Updated weights for policy 1, policy_version 999441 (0.0008) [2023-12-27 12:34:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 511942656. Throughput: 0: 10295.6, 1: 10301.7. Samples: 511951604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:47,364][19170] Avg episode reward: [(0, '740.420'), (1, '168.700')] [2023-12-27 12:34:47,562][20347] Updated weights for policy 0, policy_version 1000042 (0.0007) [2023-12-27 12:34:47,614][20347] Updated weights for policy 0, policy_version 1000052 (0.0008) [2023-12-27 12:34:47,660][20347] Updated weights for policy 0, policy_version 1000062 (0.0007) [2023-12-27 12:34:47,675][20381] Updated weights for policy 1, policy_version 999451 (0.0007) [2023-12-27 12:34:47,706][20347] Updated weights for policy 0, policy_version 1000072 (0.0007) [2023-12-27 12:34:47,726][20381] Updated weights for policy 1, policy_version 999461 (0.0007) [2023-12-27 12:34:47,790][20381] Updated weights for policy 1, policy_version 999471 (0.0007) [2023-12-27 12:34:48,454][20347] Updated weights for policy 0, policy_version 1000082 (0.0006) [2023-12-27 12:34:48,468][20381] Updated weights for policy 1, policy_version 999481 (0.0007) [2023-12-27 12:34:48,512][20347] Updated weights for policy 0, policy_version 1000092 (0.0005) [2023-12-27 12:34:48,529][20381] Updated weights for policy 1, policy_version 999491 (0.0006) [2023-12-27 12:34:48,569][20347] Updated weights for policy 0, policy_version 1000102 (0.0006) [2023-12-27 12:34:48,589][20381] Updated weights for policy 1, policy_version 999501 (0.0006) [2023-12-27 12:34:48,647][20381] Updated weights for policy 1, policy_version 999511 (0.0008) [2023-12-27 12:34:49,159][20347] Updated weights for policy 0, policy_version 1000112 (0.0007) [2023-12-27 12:34:49,213][20347] Updated weights for policy 0, policy_version 1000122 (0.0008) [2023-12-27 12:34:49,282][20347] Updated weights for policy 0, policy_version 1000132 (0.0008) [2023-12-27 12:34:49,379][20381] Updated weights for policy 1, policy_version 999521 (0.0008) [2023-12-27 12:34:49,441][20381] Updated weights for policy 1, policy_version 999531 (0.0008) [2023-12-27 12:34:49,494][20381] Updated weights for policy 1, policy_version 999541 (0.0008) [2023-12-27 12:34:49,872][20347] Updated weights for policy 0, policy_version 1000142 (0.0007) [2023-12-27 12:34:49,931][20347] Updated weights for policy 0, policy_version 1000152 (0.0007) [2023-12-27 12:34:49,999][20347] Updated weights for policy 0, policy_version 1000162 (0.0007) [2023-12-27 12:34:50,237][20381] Updated weights for policy 1, policy_version 999551 (0.0008) [2023-12-27 12:34:50,293][20381] Updated weights for policy 1, policy_version 999561 (0.0007) [2023-12-27 12:34:50,350][20381] Updated weights for policy 1, policy_version 999571 (0.0008) [2023-12-27 12:34:50,754][20347] Updated weights for policy 0, policy_version 1000172 (0.0008) [2023-12-27 12:34:50,811][20347] Updated weights for policy 0, policy_version 1000182 (0.0008) [2023-12-27 12:34:50,873][20347] Updated weights for policy 0, policy_version 1000192 (0.0008) [2023-12-27 12:34:51,033][20381] Updated weights for policy 1, policy_version 999581 (0.0006) [2023-12-27 12:34:51,092][20381] Updated weights for policy 1, policy_version 999591 (0.0007) [2023-12-27 12:34:51,154][20381] Updated weights for policy 1, policy_version 999601 (0.0009) [2023-12-27 12:34:51,594][20347] Updated weights for policy 0, policy_version 1000202 (0.0008) [2023-12-27 12:34:51,661][20347] Updated weights for policy 0, policy_version 1000212 (0.0008) [2023-12-27 12:34:51,722][20347] Updated weights for policy 0, policy_version 1000222 (0.0008) [2023-12-27 12:34:51,787][20347] Updated weights for policy 0, policy_version 1000232 (0.0007) [2023-12-27 12:34:51,907][20381] Updated weights for policy 1, policy_version 999611 (0.0008) [2023-12-27 12:34:51,971][20381] Updated weights for policy 1, policy_version 999621 (0.0008) [2023-12-27 12:34:52,035][20381] Updated weights for policy 1, policy_version 999631 (0.0008) [2023-12-27 12:34:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 512040960. Throughput: 0: 10294.7, 1: 10303.0. Samples: 512013612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:52,363][19170] Avg episode reward: [(0, '703.910'), (1, '424.120')] [2023-12-27 12:34:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001000232_256106496.pth... [2023-12-27 12:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000999640_255934464.pth... [2023-12-27 12:34:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000999048_255803392.pth [2023-12-27 12:34:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000998424_255623168.pth [2023-12-27 12:34:52,481][20347] Updated weights for policy 0, policy_version 1000242 (0.0006) [2023-12-27 12:34:52,540][20347] Updated weights for policy 0, policy_version 1000252 (0.0006) [2023-12-27 12:34:52,598][20347] Updated weights for policy 0, policy_version 1000262 (0.0005) [2023-12-27 12:34:52,782][20381] Updated weights for policy 1, policy_version 999641 (0.0008) [2023-12-27 12:34:52,834][20381] Updated weights for policy 1, policy_version 999651 (0.0008) [2023-12-27 12:34:52,886][20381] Updated weights for policy 1, policy_version 999661 (0.0008) [2023-12-27 12:34:52,937][20381] Updated weights for policy 1, policy_version 999671 (0.0009) [2023-12-27 12:34:53,185][20347] Updated weights for policy 0, policy_version 1000272 (0.0007) [2023-12-27 12:34:53,245][20347] Updated weights for policy 0, policy_version 1000282 (0.0008) [2023-12-27 12:34:53,298][20347] Updated weights for policy 0, policy_version 1000292 (0.0008) [2023-12-27 12:34:53,633][20381] Updated weights for policy 1, policy_version 999681 (0.0006) [2023-12-27 12:34:53,705][20381] Updated weights for policy 1, policy_version 999691 (0.0007) [2023-12-27 12:34:53,770][20381] Updated weights for policy 1, policy_version 999701 (0.0008) [2023-12-27 12:34:53,961][20347] Updated weights for policy 0, policy_version 1000302 (0.0006) [2023-12-27 12:34:54,018][20347] Updated weights for policy 0, policy_version 1000312 (0.0005) [2023-12-27 12:34:54,074][20347] Updated weights for policy 0, policy_version 1000322 (0.0006) [2023-12-27 12:34:54,400][20381] Updated weights for policy 1, policy_version 999711 (0.0007) [2023-12-27 12:34:54,455][20381] Updated weights for policy 1, policy_version 999721 (0.0007) [2023-12-27 12:34:54,512][20381] Updated weights for policy 1, policy_version 999731 (0.0007) [2023-12-27 12:34:54,696][20347] Updated weights for policy 0, policy_version 1000332 (0.0007) [2023-12-27 12:34:54,746][20347] Updated weights for policy 0, policy_version 1000342 (0.0007) [2023-12-27 12:34:54,799][20347] Updated weights for policy 0, policy_version 1000352 (0.0006) [2023-12-27 12:34:55,179][20381] Updated weights for policy 1, policy_version 999741 (0.0006) [2023-12-27 12:34:55,242][20381] Updated weights for policy 1, policy_version 999751 (0.0008) [2023-12-27 12:34:55,296][20381] Updated weights for policy 1, policy_version 999761 (0.0007) [2023-12-27 12:34:55,472][20347] Updated weights for policy 0, policy_version 1000362 (0.0006) [2023-12-27 12:34:55,532][20347] Updated weights for policy 0, policy_version 1000372 (0.0007) [2023-12-27 12:34:55,581][20347] Updated weights for policy 0, policy_version 1000382 (0.0008) [2023-12-27 12:34:55,627][20347] Updated weights for policy 0, policy_version 1000392 (0.0007) [2023-12-27 12:34:55,941][20381] Updated weights for policy 1, policy_version 999771 (0.0008) [2023-12-27 12:34:55,993][20381] Updated weights for policy 1, policy_version 999781 (0.0007) [2023-12-27 12:34:56,049][20381] Updated weights for policy 1, policy_version 999791 (0.0008) [2023-12-27 12:34:56,314][20347] Updated weights for policy 0, policy_version 1000402 (0.0008) [2023-12-27 12:34:56,367][20347] Updated weights for policy 0, policy_version 1000412 (0.0007) [2023-12-27 12:34:56,416][20347] Updated weights for policy 0, policy_version 1000422 (0.0007) [2023-12-27 12:34:56,721][20381] Updated weights for policy 1, policy_version 999801 (0.0008) [2023-12-27 12:34:56,765][20381] Updated weights for policy 1, policy_version 999811 (0.0008) [2023-12-27 12:34:56,819][20381] Updated weights for policy 1, policy_version 999821 (0.0007) [2023-12-27 12:34:56,877][20381] Updated weights for policy 1, policy_version 999831 (0.0006) [2023-12-27 12:34:57,067][20347] Updated weights for policy 0, policy_version 1000432 (0.0006) [2023-12-27 12:34:57,115][20347] Updated weights for policy 0, policy_version 1000442 (0.0007) [2023-12-27 12:34:57,159][20347] Updated weights for policy 0, policy_version 1000452 (0.0007) [2023-12-27 12:34:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 512147456. Throughput: 0: 10319.8, 1: 10262.8. Samples: 512135592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:34:57,363][19170] Avg episode reward: [(0, '722.080'), (1, '367.250')] [2023-12-27 12:34:57,495][20381] Updated weights for policy 1, policy_version 999841 (0.0007) [2023-12-27 12:34:57,540][20381] Updated weights for policy 1, policy_version 999851 (0.0009) [2023-12-27 12:34:57,591][20381] Updated weights for policy 1, policy_version 999861 (0.0008) [2023-12-27 12:34:57,786][20347] Updated weights for policy 0, policy_version 1000462 (0.0008) [2023-12-27 12:34:57,835][20347] Updated weights for policy 0, policy_version 1000472 (0.0007) [2023-12-27 12:34:57,881][20347] Updated weights for policy 0, policy_version 1000482 (0.0007) [2023-12-27 12:34:58,314][20381] Updated weights for policy 1, policy_version 999871 (0.0006) [2023-12-27 12:34:58,389][20381] Updated weights for policy 1, policy_version 999883 (0.0008) [2023-12-27 12:34:58,459][20381] Updated weights for policy 1, policy_version 999893 (0.0009) [2023-12-27 12:34:58,659][20347] Updated weights for policy 0, policy_version 1000492 (0.0008) [2023-12-27 12:34:58,726][20347] Updated weights for policy 0, policy_version 1000502 (0.0009) [2023-12-27 12:34:58,793][20347] Updated weights for policy 0, policy_version 1000512 (0.0013) [2023-12-27 12:34:59,273][20381] Updated weights for policy 1, policy_version 999903 (0.0008) [2023-12-27 12:34:59,335][20381] Updated weights for policy 1, policy_version 999913 (0.0006) [2023-12-27 12:34:59,397][20381] Updated weights for policy 1, policy_version 999923 (0.0008) [2023-12-27 12:34:59,555][20347] Updated weights for policy 0, policy_version 1000522 (0.0008) [2023-12-27 12:34:59,620][20347] Updated weights for policy 0, policy_version 1000532 (0.0006) [2023-12-27 12:34:59,677][20347] Updated weights for policy 0, policy_version 1000542 (0.0007) [2023-12-27 12:34:59,733][20347] Updated weights for policy 0, policy_version 1000552 (0.0006) [2023-12-27 12:35:00,041][20381] Updated weights for policy 1, policy_version 999933 (0.0007) [2023-12-27 12:35:00,099][20381] Updated weights for policy 1, policy_version 999943 (0.0007) [2023-12-27 12:35:00,157][20381] Updated weights for policy 1, policy_version 999953 (0.0007) [2023-12-27 12:35:00,401][20347] Updated weights for policy 0, policy_version 1000562 (0.0011) [2023-12-27 12:35:00,461][20347] Updated weights for policy 0, policy_version 1000572 (0.0008) [2023-12-27 12:35:00,520][20347] Updated weights for policy 0, policy_version 1000582 (0.0005) [2023-12-27 12:35:00,858][20381] Updated weights for policy 1, policy_version 999963 (0.0008) [2023-12-27 12:35:00,917][20381] Updated weights for policy 1, policy_version 999973 (0.0009) [2023-12-27 12:35:00,979][20381] Updated weights for policy 1, policy_version 999983 (0.0007) [2023-12-27 12:35:01,173][20347] Updated weights for policy 0, policy_version 1000592 (0.0007) [2023-12-27 12:35:01,229][20347] Updated weights for policy 0, policy_version 1000602 (0.0006) [2023-12-27 12:35:01,293][20347] Updated weights for policy 0, policy_version 1000612 (0.0006) [2023-12-27 12:35:01,793][20381] Updated weights for policy 1, policy_version 999993 (0.0009) [2023-12-27 12:35:01,854][20381] Updated weights for policy 1, policy_version 1000003 (0.0009) [2023-12-27 12:35:01,911][20381] Updated weights for policy 1, policy_version 1000013 (0.0009) [2023-12-27 12:35:01,972][20381] Updated weights for policy 1, policy_version 1000023 (0.0009) [2023-12-27 12:35:02,091][20347] Updated weights for policy 0, policy_version 1000622 (0.0007) [2023-12-27 12:35:02,150][20347] Updated weights for policy 0, policy_version 1000632 (0.0008) [2023-12-27 12:35:02,208][20347] Updated weights for policy 0, policy_version 1000642 (0.0008) [2023-12-27 12:35:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 512245760. Throughput: 0: 10270.5, 1: 10183.7. Samples: 512253320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:35:02,363][19170] Avg episode reward: [(0, '844.870'), (1, '46.390')] [2023-12-27 12:35:02,679][20381] Updated weights for policy 1, policy_version 1000033 (0.0008) [2023-12-27 12:35:02,735][20381] Updated weights for policy 1, policy_version 1000043 (0.0008) [2023-12-27 12:35:02,793][20381] Updated weights for policy 1, policy_version 1000053 (0.0008) [2023-12-27 12:35:02,915][20347] Updated weights for policy 0, policy_version 1000652 (0.0008) [2023-12-27 12:35:02,976][20347] Updated weights for policy 0, policy_version 1000662 (0.0009) [2023-12-27 12:35:03,030][20347] Updated weights for policy 0, policy_version 1000672 (0.0009) [2023-12-27 12:35:03,487][20381] Updated weights for policy 1, policy_version 1000063 (0.0007) [2023-12-27 12:35:03,551][20381] Updated weights for policy 1, policy_version 1000073 (0.0008) [2023-12-27 12:35:03,602][20381] Updated weights for policy 1, policy_version 1000083 (0.0007) [2023-12-27 12:35:03,683][20347] Updated weights for policy 0, policy_version 1000682 (0.0006) [2023-12-27 12:35:03,728][20347] Updated weights for policy 0, policy_version 1000692 (0.0007) [2023-12-27 12:35:03,777][20347] Updated weights for policy 0, policy_version 1000702 (0.0008) [2023-12-27 12:35:03,818][20347] Updated weights for policy 0, policy_version 1000712 (0.0007) [2023-12-27 12:35:04,284][20381] Updated weights for policy 1, policy_version 1000093 (0.0008) [2023-12-27 12:35:04,347][20381] Updated weights for policy 1, policy_version 1000103 (0.0008) [2023-12-27 12:35:04,407][20381] Updated weights for policy 1, policy_version 1000113 (0.0008) [2023-12-27 12:35:04,543][20347] Updated weights for policy 0, policy_version 1000722 (0.0008) [2023-12-27 12:35:04,609][20347] Updated weights for policy 0, policy_version 1000732 (0.0008) [2023-12-27 12:35:04,673][20347] Updated weights for policy 0, policy_version 1000742 (0.0008) [2023-12-27 12:35:05,133][20381] Updated weights for policy 1, policy_version 1000123 (0.0007) [2023-12-27 12:35:05,192][20381] Updated weights for policy 1, policy_version 1000133 (0.0009) [2023-12-27 12:35:05,225][20347] Updated weights for policy 0, policy_version 1000752 (0.0006) [2023-12-27 12:35:05,238][20381] Updated weights for policy 1, policy_version 1000143 (0.0008) [2023-12-27 12:35:05,281][20347] Updated weights for policy 0, policy_version 1000762 (0.0005) [2023-12-27 12:35:05,335][20347] Updated weights for policy 0, policy_version 1000772 (0.0005) [2023-12-27 12:35:05,894][20347] Updated weights for policy 0, policy_version 1000782 (0.0006) [2023-12-27 12:35:05,945][20347] Updated weights for policy 0, policy_version 1000792 (0.0008) [2023-12-27 12:35:06,000][20347] Updated weights for policy 0, policy_version 1000802 (0.0008) [2023-12-27 12:35:06,014][20381] Updated weights for policy 1, policy_version 1000153 (0.0007) [2023-12-27 12:35:06,073][20381] Updated weights for policy 1, policy_version 1000163 (0.0007) [2023-12-27 12:35:06,135][20381] Updated weights for policy 1, policy_version 1000173 (0.0008) [2023-12-27 12:35:06,195][20381] Updated weights for policy 1, policy_version 1000183 (0.0008) [2023-12-27 12:35:06,608][20347] Updated weights for policy 0, policy_version 1000812 (0.0006) [2023-12-27 12:35:06,672][20347] Updated weights for policy 0, policy_version 1000822 (0.0005) [2023-12-27 12:35:06,735][20347] Updated weights for policy 0, policy_version 1000832 (0.0005) [2023-12-27 12:35:06,939][20381] Updated weights for policy 1, policy_version 1000193 (0.0009) [2023-12-27 12:35:06,999][20381] Updated weights for policy 1, policy_version 1000203 (0.0008) [2023-12-27 12:35:07,048][20381] Updated weights for policy 1, policy_version 1000213 (0.0008) [2023-12-27 12:35:07,286][20347] Updated weights for policy 0, policy_version 1000842 (0.0006) [2023-12-27 12:35:07,341][20347] Updated weights for policy 0, policy_version 1000852 (0.0008) [2023-12-27 12:35:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 512344064. Throughput: 0: 10283.4, 1: 10150.5. Samples: 512314184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:35:07,363][19170] Avg episode reward: [(0, '765.840'), (1, '41.650')] [2023-12-27 12:35:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001000216_256081920.pth... [2023-12-27 12:35:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000999064_255787008.pth [2023-12-27 12:35:07,404][20347] Updated weights for policy 0, policy_version 1000862 (0.0008) [2023-12-27 12:35:07,459][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001000872_256270336.pth... [2023-12-27 12:35:07,460][20347] Updated weights for policy 0, policy_version 1000872 (0.0007) [2023-12-27 12:35:07,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_000999656_255959040.pth [2023-12-27 12:35:07,736][20381] Updated weights for policy 1, policy_version 1000223 (0.0008) [2023-12-27 12:35:07,787][20381] Updated weights for policy 1, policy_version 1000233 (0.0008) [2023-12-27 12:35:07,839][20381] Updated weights for policy 1, policy_version 1000243 (0.0009) [2023-12-27 12:35:08,115][20347] Updated weights for policy 0, policy_version 1000882 (0.0006) [2023-12-27 12:35:08,171][20347] Updated weights for policy 0, policy_version 1000892 (0.0008) [2023-12-27 12:35:08,234][20347] Updated weights for policy 0, policy_version 1000902 (0.0005) [2023-12-27 12:35:08,555][20381] Updated weights for policy 1, policy_version 1000253 (0.0007) [2023-12-27 12:35:08,614][20381] Updated weights for policy 1, policy_version 1000263 (0.0008) [2023-12-27 12:35:08,671][20381] Updated weights for policy 1, policy_version 1000273 (0.0007) [2023-12-27 12:35:08,939][20347] Updated weights for policy 0, policy_version 1000912 (0.0008) [2023-12-27 12:35:08,994][20347] Updated weights for policy 0, policy_version 1000922 (0.0008) [2023-12-27 12:35:09,050][20347] Updated weights for policy 0, policy_version 1000932 (0.0007) [2023-12-27 12:35:09,261][20381] Updated weights for policy 1, policy_version 1000283 (0.0008) [2023-12-27 12:35:09,312][20381] Updated weights for policy 1, policy_version 1000293 (0.0008) [2023-12-27 12:35:09,379][20381] Updated weights for policy 1, policy_version 1000303 (0.0008) [2023-12-27 12:35:09,858][20347] Updated weights for policy 0, policy_version 1000942 (0.0008) [2023-12-27 12:35:09,921][20347] Updated weights for policy 0, policy_version 1000952 (0.0007) [2023-12-27 12:35:09,985][20347] Updated weights for policy 0, policy_version 1000962 (0.0006) [2023-12-27 12:35:10,158][20381] Updated weights for policy 1, policy_version 1000313 (0.0006) [2023-12-27 12:35:10,221][20381] Updated weights for policy 1, policy_version 1000323 (0.0007) [2023-12-27 12:35:10,285][20381] Updated weights for policy 1, policy_version 1000333 (0.0006) [2023-12-27 12:35:10,351][20381] Updated weights for policy 1, policy_version 1000343 (0.0006) [2023-12-27 12:35:10,612][20347] Updated weights for policy 0, policy_version 1000972 (0.0008) [2023-12-27 12:35:10,668][20347] Updated weights for policy 0, policy_version 1000982 (0.0010) [2023-12-27 12:35:10,720][20347] Updated weights for policy 0, policy_version 1000992 (0.0010) [2023-12-27 12:35:10,974][20381] Updated weights for policy 1, policy_version 1000353 (0.0008) [2023-12-27 12:35:11,035][20381] Updated weights for policy 1, policy_version 1000363 (0.0007) [2023-12-27 12:35:11,098][20381] Updated weights for policy 1, policy_version 1000373 (0.0008) [2023-12-27 12:35:11,427][20347] Updated weights for policy 0, policy_version 1001002 (0.0008) [2023-12-27 12:35:11,486][20347] Updated weights for policy 0, policy_version 1001012 (0.0008) [2023-12-27 12:35:11,535][20347] Updated weights for policy 0, policy_version 1001022 (0.0008) [2023-12-27 12:35:11,586][20347] Updated weights for policy 0, policy_version 1001032 (0.0008) [2023-12-27 12:35:11,870][20381] Updated weights for policy 1, policy_version 1000383 (0.0007) [2023-12-27 12:35:11,930][20381] Updated weights for policy 1, policy_version 1000393 (0.0007) [2023-12-27 12:35:11,985][20381] Updated weights for policy 1, policy_version 1000403 (0.0006) [2023-12-27 12:35:12,357][20347] Updated weights for policy 0, policy_version 1001042 (0.0007) [2023-12-27 12:35:12,363][19170] Fps is (10 sec: 19659.1, 60 sec: 20343.2, 300 sec: 20438.3). Total num frames: 512442368. Throughput: 0: 10339.4, 1: 10080.5. Samples: 512437108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:35:12,364][19170] Avg episode reward: [(0, '785.190'), (1, '165.960')] [2023-12-27 12:35:12,412][20347] Updated weights for policy 0, policy_version 1001052 (0.0007) [2023-12-27 12:35:12,464][20347] Updated weights for policy 0, policy_version 1001062 (0.0007) [2023-12-27 12:35:12,630][20381] Updated weights for policy 1, policy_version 1000413 (0.0007) [2023-12-27 12:35:12,685][20381] Updated weights for policy 1, policy_version 1000423 (0.0007) [2023-12-27 12:35:12,754][20381] Updated weights for policy 1, policy_version 1000433 (0.0008) [2023-12-27 12:35:13,143][20347] Updated weights for policy 0, policy_version 1001072 (0.0006) [2023-12-27 12:35:13,194][20347] Updated weights for policy 0, policy_version 1001082 (0.0005) [2023-12-27 12:35:13,250][20347] Updated weights for policy 0, policy_version 1001092 (0.0005) [2023-12-27 12:35:13,415][20381] Updated weights for policy 1, policy_version 1000443 (0.0008) [2023-12-27 12:35:13,471][20381] Updated weights for policy 1, policy_version 1000453 (0.0011) [2023-12-27 12:35:13,534][20381] Updated weights for policy 1, policy_version 1000463 (0.0011) [2023-12-27 12:35:13,894][20347] Updated weights for policy 0, policy_version 1001102 (0.0006) [2023-12-27 12:35:13,945][20347] Updated weights for policy 0, policy_version 1001112 (0.0007) [2023-12-27 12:35:13,993][20347] Updated weights for policy 0, policy_version 1001122 (0.0008) [2023-12-27 12:35:14,155][20381] Updated weights for policy 1, policy_version 1000473 (0.0011) [2023-12-27 12:35:14,205][20381] Updated weights for policy 1, policy_version 1000483 (0.0011) [2023-12-27 12:35:14,254][20381] Updated weights for policy 1, policy_version 1000493 (0.0010) [2023-12-27 12:35:14,309][20381] Updated weights for policy 1, policy_version 1000503 (0.0010) [2023-12-27 12:35:14,660][20347] Updated weights for policy 0, policy_version 1001132 (0.0009) [2023-12-27 12:35:14,719][20347] Updated weights for policy 0, policy_version 1001142 (0.0008) [2023-12-27 12:35:14,785][20347] Updated weights for policy 0, policy_version 1001154 (0.0008) [2023-12-27 12:35:14,983][20381] Updated weights for policy 1, policy_version 1000513 (0.0010) [2023-12-27 12:35:15,042][20381] Updated weights for policy 1, policy_version 1000523 (0.0011) [2023-12-27 12:35:15,104][20381] Updated weights for policy 1, policy_version 1000533 (0.0011) [2023-12-27 12:35:15,510][20347] Updated weights for policy 0, policy_version 1001164 (0.0007) [2023-12-27 12:35:15,555][20347] Updated weights for policy 0, policy_version 1001174 (0.0007) [2023-12-27 12:35:15,600][20347] Updated weights for policy 0, policy_version 1001184 (0.0007) [2023-12-27 12:35:15,794][20381] Updated weights for policy 1, policy_version 1000543 (0.0010) [2023-12-27 12:35:15,854][20381] Updated weights for policy 1, policy_version 1000553 (0.0010) [2023-12-27 12:35:15,907][20381] Updated weights for policy 1, policy_version 1000563 (0.0010) [2023-12-27 12:35:16,324][20347] Updated weights for policy 0, policy_version 1001194 (0.0007) [2023-12-27 12:35:16,377][20347] Updated weights for policy 0, policy_version 1001204 (0.0007) [2023-12-27 12:35:16,433][20347] Updated weights for policy 0, policy_version 1001214 (0.0008) [2023-12-27 12:35:16,498][20347] Updated weights for policy 0, policy_version 1001224 (0.0007) [2023-12-27 12:35:16,584][20381] Updated weights for policy 1, policy_version 1000573 (0.0010) [2023-12-27 12:35:16,635][20381] Updated weights for policy 1, policy_version 1000583 (0.0010) [2023-12-27 12:35:16,682][20381] Updated weights for policy 1, policy_version 1000593 (0.0010) [2023-12-27 12:35:17,196][20347] Updated weights for policy 0, policy_version 1001234 (0.0007) [2023-12-27 12:35:17,250][20347] Updated weights for policy 0, policy_version 1001244 (0.0007) [2023-12-27 12:35:17,305][20347] Updated weights for policy 0, policy_version 1001254 (0.0007) [2023-12-27 12:35:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 512548864. Throughput: 0: 10253.1, 1: 10099.2. Samples: 512558780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:35:17,363][19170] Avg episode reward: [(0, '748.140'), (1, '295.700')] [2023-12-27 12:35:17,367][20381] Updated weights for policy 1, policy_version 1000603 (0.0010) [2023-12-27 12:35:17,427][20381] Updated weights for policy 1, policy_version 1000613 (0.0009) [2023-12-27 12:35:17,494][20381] Updated weights for policy 1, policy_version 1000623 (0.0008) [2023-12-27 12:35:17,914][20347] Updated weights for policy 0, policy_version 1001264 (0.0006) [2023-12-27 12:35:17,925][20291] KL-divergence is very high: 213.9702 [2023-12-27 12:35:17,969][20347] Updated weights for policy 0, policy_version 1001274 (0.0005) [2023-12-27 12:35:17,971][20291] KL-divergence is very high: 186.9789 [2023-12-27 12:35:18,021][20347] Updated weights for policy 0, policy_version 1001284 (0.0005) [2023-12-27 12:35:18,202][20381] Updated weights for policy 1, policy_version 1000633 (0.0009) [2023-12-27 12:35:18,252][20381] Updated weights for policy 1, policy_version 1000643 (0.0006) [2023-12-27 12:35:18,311][20381] Updated weights for policy 1, policy_version 1000653 (0.0008) [2023-12-27 12:35:18,375][20381] Updated weights for policy 1, policy_version 1000663 (0.0010) [2023-12-27 12:35:18,645][20347] Updated weights for policy 0, policy_version 1001294 (0.0007) [2023-12-27 12:35:18,714][20347] Updated weights for policy 0, policy_version 1001304 (0.0010) [2023-12-27 12:35:18,779][20347] Updated weights for policy 0, policy_version 1001314 (0.0009) [2023-12-27 12:35:19,069][20381] Updated weights for policy 1, policy_version 1000673 (0.0010) [2023-12-27 12:35:19,127][20381] Updated weights for policy 1, policy_version 1000683 (0.0010) [2023-12-27 12:35:19,182][20381] Updated weights for policy 1, policy_version 1000693 (0.0010) [2023-12-27 12:35:19,475][20347] Updated weights for policy 0, policy_version 1001324 (0.0008) [2023-12-27 12:35:19,540][20347] Updated weights for policy 0, policy_version 1001334 (0.0008) [2023-12-27 12:35:19,593][20347] Updated weights for policy 0, policy_version 1001344 (0.0008) [2023-12-27 12:35:19,923][20381] Updated weights for policy 1, policy_version 1000703 (0.0009) [2023-12-27 12:35:19,980][20381] Updated weights for policy 1, policy_version 1000713 (0.0010) [2023-12-27 12:35:20,041][20381] Updated weights for policy 1, policy_version 1000723 (0.0011) [2023-12-27 12:35:20,311][20347] Updated weights for policy 0, policy_version 1001354 (0.0008) [2023-12-27 12:35:20,366][20347] Updated weights for policy 0, policy_version 1001364 (0.0009) [2023-12-27 12:35:20,412][20347] Updated weights for policy 0, policy_version 1001374 (0.0008) [2023-12-27 12:35:20,461][20347] Updated weights for policy 0, policy_version 1001384 (0.0007) [2023-12-27 12:35:20,704][20381] Updated weights for policy 1, policy_version 1000733 (0.0010) [2023-12-27 12:35:20,750][20381] Updated weights for policy 1, policy_version 1000743 (0.0005) [2023-12-27 12:35:20,805][20381] Updated weights for policy 1, policy_version 1000753 (0.0010) [2023-12-27 12:35:21,246][20347] Updated weights for policy 0, policy_version 1001394 (0.0007) [2023-12-27 12:35:21,311][20347] Updated weights for policy 0, policy_version 1001404 (0.0006) [2023-12-27 12:35:21,379][20347] Updated weights for policy 0, policy_version 1001414 (0.0008) [2023-12-27 12:35:21,546][20381] Updated weights for policy 1, policy_version 1000763 (0.0008) [2023-12-27 12:35:21,606][20381] Updated weights for policy 1, policy_version 1000773 (0.0008) [2023-12-27 12:35:21,667][20381] Updated weights for policy 1, policy_version 1000783 (0.0010) [2023-12-27 12:35:22,095][20347] Updated weights for policy 0, policy_version 1001424 (0.0009) [2023-12-27 12:35:22,152][20347] Updated weights for policy 0, policy_version 1001434 (0.0008) [2023-12-27 12:35:22,212][20347] Updated weights for policy 0, policy_version 1001444 (0.0009) [2023-12-27 12:35:22,362][19170] Fps is (10 sec: 20481.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 512647168. Throughput: 0: 10233.5, 1: 10010.7. Samples: 512618588. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:22,363][19170] Avg episode reward: [(0, '688.840'), (1, '373.760')] [2023-12-27 12:35:22,368][20381] Updated weights for policy 1, policy_version 1000793 (0.0011) [2023-12-27 12:35:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001001448_256417792.pth... [2023-12-27 12:35:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001000232_256106496.pth [2023-12-27 12:35:22,429][20381] Updated weights for policy 1, policy_version 1000803 (0.0011) [2023-12-27 12:35:22,492][20381] Updated weights for policy 1, policy_version 1000813 (0.0011) [2023-12-27 12:35:22,556][20381] Updated weights for policy 1, policy_version 1000823 (0.0011) [2023-12-27 12:35:22,559][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001000824_256237568.pth... [2023-12-27 12:35:22,563][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_000999640_255934464.pth [2023-12-27 12:35:22,903][20347] Updated weights for policy 0, policy_version 1001454 (0.0008) [2023-12-27 12:35:22,968][20347] Updated weights for policy 0, policy_version 1001464 (0.0007) [2023-12-27 12:35:23,018][20347] Updated weights for policy 0, policy_version 1001474 (0.0010) [2023-12-27 12:35:23,190][20381] Updated weights for policy 1, policy_version 1000833 (0.0006) [2023-12-27 12:35:23,241][20381] Updated weights for policy 1, policy_version 1000843 (0.0005) [2023-12-27 12:35:23,299][20381] Updated weights for policy 1, policy_version 1000853 (0.0005) [2023-12-27 12:35:23,596][20347] Updated weights for policy 0, policy_version 1001484 (0.0008) [2023-12-27 12:35:23,646][20347] Updated weights for policy 0, policy_version 1001494 (0.0006) [2023-12-27 12:35:23,695][20347] Updated weights for policy 0, policy_version 1001504 (0.0005) [2023-12-27 12:35:23,946][20381] Updated weights for policy 1, policy_version 1000863 (0.0006) [2023-12-27 12:35:24,008][20381] Updated weights for policy 1, policy_version 1000873 (0.0007) [2023-12-27 12:35:24,064][20381] Updated weights for policy 1, policy_version 1000883 (0.0008) [2023-12-27 12:35:24,299][20347] Updated weights for policy 0, policy_version 1001514 (0.0006) [2023-12-27 12:35:24,354][20347] Updated weights for policy 0, policy_version 1001524 (0.0007) [2023-12-27 12:35:24,407][20347] Updated weights for policy 0, policy_version 1001534 (0.0008) [2023-12-27 12:35:24,457][20347] Updated weights for policy 0, policy_version 1001544 (0.0008) [2023-12-27 12:35:24,764][20381] Updated weights for policy 1, policy_version 1000893 (0.0008) [2023-12-27 12:35:24,814][20381] Updated weights for policy 1, policy_version 1000903 (0.0010) [2023-12-27 12:35:24,869][20381] Updated weights for policy 1, policy_version 1000913 (0.0009) [2023-12-27 12:35:25,033][20347] Updated weights for policy 0, policy_version 1001554 (0.0007) [2023-12-27 12:35:25,088][20347] Updated weights for policy 0, policy_version 1001564 (0.0008) [2023-12-27 12:35:25,143][20347] Updated weights for policy 0, policy_version 1001574 (0.0008) [2023-12-27 12:35:25,471][20381] Updated weights for policy 1, policy_version 1000923 (0.0008) [2023-12-27 12:35:25,532][20381] Updated weights for policy 1, policy_version 1000933 (0.0005) [2023-12-27 12:35:25,579][20381] Updated weights for policy 1, policy_version 1000943 (0.0005) [2023-12-27 12:35:25,889][20347] Updated weights for policy 0, policy_version 1001584 (0.0009) [2023-12-27 12:35:25,938][20347] Updated weights for policy 0, policy_version 1001594 (0.0010) [2023-12-27 12:35:25,982][20347] Updated weights for policy 0, policy_version 1001604 (0.0008) [2023-12-27 12:35:26,099][20381] Updated weights for policy 1, policy_version 1000953 (0.0008) [2023-12-27 12:35:26,156][20381] Updated weights for policy 1, policy_version 1000963 (0.0005) [2023-12-27 12:35:26,213][20381] Updated weights for policy 1, policy_version 1000973 (0.0009) [2023-12-27 12:35:26,266][20381] Updated weights for policy 1, policy_version 1000983 (0.0011) [2023-12-27 12:35:26,646][20347] Updated weights for policy 0, policy_version 1001614 (0.0008) [2023-12-27 12:35:26,672][20291] KL-divergence is very high: 101.6153 [2023-12-27 12:35:26,698][20347] Updated weights for policy 0, policy_version 1001624 (0.0007) [2023-12-27 12:35:26,757][20347] Updated weights for policy 0, policy_version 1001634 (0.0005) [2023-12-27 12:35:26,829][20381] Updated weights for policy 1, policy_version 1000993 (0.0006) [2023-12-27 12:35:26,881][20381] Updated weights for policy 1, policy_version 1001003 (0.0007) [2023-12-27 12:35:26,934][20381] Updated weights for policy 1, policy_version 1001013 (0.0007) [2023-12-27 12:35:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 512753664. Throughput: 0: 10226.6, 1: 10079.0. Samples: 512743592. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:27,363][19170] Avg episode reward: [(0, '727.880'), (1, '431.470')] [2023-12-27 12:35:27,405][20347] Updated weights for policy 0, policy_version 1001644 (0.0006) [2023-12-27 12:35:27,458][20347] Updated weights for policy 0, policy_version 1001654 (0.0007) [2023-12-27 12:35:27,486][20381] Updated weights for policy 1, policy_version 1001023 (0.0009) [2023-12-27 12:35:27,510][20347] Updated weights for policy 0, policy_version 1001664 (0.0006) [2023-12-27 12:35:27,540][20381] Updated weights for policy 1, policy_version 1001033 (0.0006) [2023-12-27 12:35:27,597][20381] Updated weights for policy 1, policy_version 1001043 (0.0005) [2023-12-27 12:35:28,128][20347] Updated weights for policy 0, policy_version 1001674 (0.0006) [2023-12-27 12:35:28,183][20347] Updated weights for policy 0, policy_version 1001684 (0.0005) [2023-12-27 12:35:28,194][20381] Updated weights for policy 1, policy_version 1001053 (0.0005) [2023-12-27 12:35:28,239][20347] Updated weights for policy 0, policy_version 1001694 (0.0005) [2023-12-27 12:35:28,249][20381] Updated weights for policy 1, policy_version 1001063 (0.0005) [2023-12-27 12:35:28,297][20347] Updated weights for policy 0, policy_version 1001704 (0.0005) [2023-12-27 12:35:28,311][20381] Updated weights for policy 1, policy_version 1001073 (0.0009) [2023-12-27 12:35:28,873][20347] Updated weights for policy 0, policy_version 1001714 (0.0005) [2023-12-27 12:35:28,931][20347] Updated weights for policy 0, policy_version 1001724 (0.0007) [2023-12-27 12:35:28,976][20381] Updated weights for policy 1, policy_version 1001083 (0.0008) [2023-12-27 12:35:28,984][20347] Updated weights for policy 0, policy_version 1001734 (0.0008) [2023-12-27 12:35:29,027][20381] Updated weights for policy 1, policy_version 1001093 (0.0005) [2023-12-27 12:35:29,080][20381] Updated weights for policy 1, policy_version 1001103 (0.0005) [2023-12-27 12:35:29,683][20347] Updated weights for policy 0, policy_version 1001744 (0.0010) [2023-12-27 12:35:29,731][20347] Updated weights for policy 0, policy_version 1001754 (0.0010) [2023-12-27 12:35:29,764][20381] Updated weights for policy 1, policy_version 1001113 (0.0006) [2023-12-27 12:35:29,783][20347] Updated weights for policy 0, policy_version 1001764 (0.0011) [2023-12-27 12:35:29,828][20381] Updated weights for policy 1, policy_version 1001123 (0.0006) [2023-12-27 12:35:29,888][20381] Updated weights for policy 1, policy_version 1001133 (0.0007) [2023-12-27 12:35:29,955][20381] Updated weights for policy 1, policy_version 1001143 (0.0007) [2023-12-27 12:35:30,532][20347] Updated weights for policy 0, policy_version 1001774 (0.0009) [2023-12-27 12:35:30,588][20347] Updated weights for policy 0, policy_version 1001784 (0.0007) [2023-12-27 12:35:30,608][20381] Updated weights for policy 1, policy_version 1001153 (0.0007) [2023-12-27 12:35:30,650][20347] Updated weights for policy 0, policy_version 1001794 (0.0006) [2023-12-27 12:35:30,659][20381] Updated weights for policy 1, policy_version 1001163 (0.0005) [2023-12-27 12:35:30,712][20381] Updated weights for policy 1, policy_version 1001173 (0.0007) [2023-12-27 12:35:31,272][20347] Updated weights for policy 0, policy_version 1001804 (0.0007) [2023-12-27 12:35:31,333][20347] Updated weights for policy 0, policy_version 1001814 (0.0007) [2023-12-27 12:35:31,400][20347] Updated weights for policy 0, policy_version 1001824 (0.0007) [2023-12-27 12:35:31,474][20381] Updated weights for policy 1, policy_version 1001183 (0.0006) [2023-12-27 12:35:31,534][20381] Updated weights for policy 1, policy_version 1001193 (0.0007) [2023-12-27 12:35:31,586][20381] Updated weights for policy 1, policy_version 1001203 (0.0006) [2023-12-27 12:35:32,075][20347] Updated weights for policy 0, policy_version 1001834 (0.0006) [2023-12-27 12:35:32,135][20347] Updated weights for policy 0, policy_version 1001844 (0.0007) [2023-12-27 12:35:32,199][20347] Updated weights for policy 0, policy_version 1001854 (0.0007) [2023-12-27 12:35:32,260][20347] Updated weights for policy 0, policy_version 1001864 (0.0008) [2023-12-27 12:35:32,265][20381] Updated weights for policy 1, policy_version 1001213 (0.0007) [2023-12-27 12:35:32,324][20381] Updated weights for policy 1, policy_version 1001223 (0.0008) [2023-12-27 12:35:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.5, 300 sec: 20466.1). Total num frames: 512860160. Throughput: 0: 10307.4, 1: 10138.3. Samples: 512871660. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:32,363][19170] Avg episode reward: [(0, '758.710'), (1, '441.110')] [2023-12-27 12:35:32,387][20381] Updated weights for policy 1, policy_version 1001233 (0.0007) [2023-12-27 12:35:32,902][20347] Updated weights for policy 0, policy_version 1001874 (0.0005) [2023-12-27 12:35:32,949][20347] Updated weights for policy 0, policy_version 1001884 (0.0005) [2023-12-27 12:35:33,004][20347] Updated weights for policy 0, policy_version 1001894 (0.0006) [2023-12-27 12:35:33,120][20381] Updated weights for policy 1, policy_version 1001243 (0.0007) [2023-12-27 12:35:33,170][20381] Updated weights for policy 1, policy_version 1001253 (0.0008) [2023-12-27 12:35:33,216][20381] Updated weights for policy 1, policy_version 1001263 (0.0008) [2023-12-27 12:35:33,587][20347] Updated weights for policy 0, policy_version 1001904 (0.0007) [2023-12-27 12:35:33,631][20347] Updated weights for policy 0, policy_version 1001914 (0.0008) [2023-12-27 12:35:33,684][20347] Updated weights for policy 0, policy_version 1001924 (0.0007) [2023-12-27 12:35:33,912][20381] Updated weights for policy 1, policy_version 1001273 (0.0008) [2023-12-27 12:35:33,962][20381] Updated weights for policy 1, policy_version 1001283 (0.0005) [2023-12-27 12:35:34,013][20381] Updated weights for policy 1, policy_version 1001293 (0.0006) [2023-12-27 12:35:34,064][20381] Updated weights for policy 1, policy_version 1001303 (0.0007) [2023-12-27 12:35:34,418][20347] Updated weights for policy 0, policy_version 1001934 (0.0007) [2023-12-27 12:35:34,476][20347] Updated weights for policy 0, policy_version 1001944 (0.0006) [2023-12-27 12:35:34,536][20347] Updated weights for policy 0, policy_version 1001954 (0.0008) [2023-12-27 12:35:34,751][20381] Updated weights for policy 1, policy_version 1001313 (0.0007) [2023-12-27 12:35:34,806][20381] Updated weights for policy 1, policy_version 1001323 (0.0007) [2023-12-27 12:35:34,865][20381] Updated weights for policy 1, policy_version 1001333 (0.0008) [2023-12-27 12:35:35,209][20347] Updated weights for policy 0, policy_version 1001964 (0.0008) [2023-12-27 12:35:35,262][20347] Updated weights for policy 0, policy_version 1001974 (0.0007) [2023-12-27 12:35:35,318][20347] Updated weights for policy 0, policy_version 1001984 (0.0005) [2023-12-27 12:35:35,546][20381] Updated weights for policy 1, policy_version 1001343 (0.0008) [2023-12-27 12:35:35,597][20381] Updated weights for policy 1, policy_version 1001353 (0.0008) [2023-12-27 12:35:35,643][20381] Updated weights for policy 1, policy_version 1001363 (0.0007) [2023-12-27 12:35:35,945][20347] Updated weights for policy 0, policy_version 1001994 (0.0005) [2023-12-27 12:35:36,002][20347] Updated weights for policy 0, policy_version 1002004 (0.0009) [2023-12-27 12:35:36,059][20347] Updated weights for policy 0, policy_version 1002014 (0.0010) [2023-12-27 12:35:36,120][20347] Updated weights for policy 0, policy_version 1002024 (0.0010) [2023-12-27 12:35:36,357][20381] Updated weights for policy 1, policy_version 1001373 (0.0006) [2023-12-27 12:35:36,416][20381] Updated weights for policy 1, policy_version 1001383 (0.0008) [2023-12-27 12:35:36,476][20381] Updated weights for policy 1, policy_version 1001393 (0.0008) [2023-12-27 12:35:36,764][20347] Updated weights for policy 0, policy_version 1002034 (0.0006) [2023-12-27 12:35:36,821][20347] Updated weights for policy 0, policy_version 1002044 (0.0008) [2023-12-27 12:35:36,886][20347] Updated weights for policy 0, policy_version 1002054 (0.0010) [2023-12-27 12:35:37,138][20381] Updated weights for policy 1, policy_version 1001403 (0.0008) [2023-12-27 12:35:37,200][20381] Updated weights for policy 1, policy_version 1001413 (0.0011) [2023-12-27 12:35:37,245][20381] Updated weights for policy 1, policy_version 1001423 (0.0008) [2023-12-27 12:35:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 512966656. Throughput: 0: 10304.0, 1: 10114.3. Samples: 512932436. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:37,363][19170] Avg episode reward: [(0, '853.380'), (1, '460.990')] [2023-12-27 12:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001001432_256393216.pth... [2023-12-27 12:35:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001002056_256573440.pth... [2023-12-27 12:35:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001000216_256081920.pth [2023-12-27 12:35:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001000872_256270336.pth [2023-12-27 12:35:37,531][20347] Updated weights for policy 0, policy_version 1002064 (0.0008) [2023-12-27 12:35:37,581][20347] Updated weights for policy 0, policy_version 1002074 (0.0008) [2023-12-27 12:35:37,639][20347] Updated weights for policy 0, policy_version 1002084 (0.0008) [2023-12-27 12:35:37,931][20381] Updated weights for policy 1, policy_version 1001433 (0.0007) [2023-12-27 12:35:37,980][20381] Updated weights for policy 1, policy_version 1001443 (0.0008) [2023-12-27 12:35:38,037][20381] Updated weights for policy 1, policy_version 1001453 (0.0009) [2023-12-27 12:35:38,085][20381] Updated weights for policy 1, policy_version 1001463 (0.0007) [2023-12-27 12:35:38,265][20347] Updated weights for policy 0, policy_version 1002094 (0.0007) [2023-12-27 12:35:38,311][20347] Updated weights for policy 0, policy_version 1002104 (0.0005) [2023-12-27 12:35:38,373][20347] Updated weights for policy 0, policy_version 1002114 (0.0010) [2023-12-27 12:35:38,853][20381] Updated weights for policy 1, policy_version 1001473 (0.0008) [2023-12-27 12:35:38,914][20381] Updated weights for policy 1, policy_version 1001483 (0.0008) [2023-12-27 12:35:38,971][20381] Updated weights for policy 1, policy_version 1001493 (0.0008) [2023-12-27 12:35:38,991][20347] Updated weights for policy 0, policy_version 1002124 (0.0007) [2023-12-27 12:35:39,050][20347] Updated weights for policy 0, policy_version 1002134 (0.0010) [2023-12-27 12:35:39,108][20347] Updated weights for policy 0, policy_version 1002144 (0.0010) [2023-12-27 12:35:39,655][20381] Updated weights for policy 1, policy_version 1001503 (0.0007) [2023-12-27 12:35:39,717][20381] Updated weights for policy 1, policy_version 1001513 (0.0008) [2023-12-27 12:35:39,778][20381] Updated weights for policy 1, policy_version 1001523 (0.0007) [2023-12-27 12:35:39,837][20347] Updated weights for policy 0, policy_version 1002154 (0.0010) [2023-12-27 12:35:39,915][20347] Updated weights for policy 0, policy_version 1002164 (0.0010) [2023-12-27 12:35:39,976][20347] Updated weights for policy 0, policy_version 1002174 (0.0008) [2023-12-27 12:35:40,038][20347] Updated weights for policy 0, policy_version 1002184 (0.0008) [2023-12-27 12:35:40,495][20381] Updated weights for policy 1, policy_version 1001533 (0.0008) [2023-12-27 12:35:40,550][20381] Updated weights for policy 1, policy_version 1001543 (0.0007) [2023-12-27 12:35:40,613][20381] Updated weights for policy 1, policy_version 1001553 (0.0007) [2023-12-27 12:35:40,695][20347] Updated weights for policy 0, policy_version 1002194 (0.0005) [2023-12-27 12:35:40,752][20347] Updated weights for policy 0, policy_version 1002204 (0.0005) [2023-12-27 12:35:40,813][20347] Updated weights for policy 0, policy_version 1002214 (0.0005) [2023-12-27 12:35:41,310][20381] Updated weights for policy 1, policy_version 1001563 (0.0007) [2023-12-27 12:35:41,376][20381] Updated weights for policy 1, policy_version 1001573 (0.0008) [2023-12-27 12:35:41,430][20381] Updated weights for policy 1, policy_version 1001583 (0.0007) [2023-12-27 12:35:41,499][20347] Updated weights for policy 0, policy_version 1002224 (0.0007) [2023-12-27 12:35:41,559][20347] Updated weights for policy 0, policy_version 1002234 (0.0008) [2023-12-27 12:35:41,616][20347] Updated weights for policy 0, policy_version 1002244 (0.0008) [2023-12-27 12:35:42,135][20381] Updated weights for policy 1, policy_version 1001593 (0.0007) [2023-12-27 12:35:42,194][20381] Updated weights for policy 1, policy_version 1001603 (0.0008) [2023-12-27 12:35:42,252][20381] Updated weights for policy 1, policy_version 1001613 (0.0008) [2023-12-27 12:35:42,310][20381] Updated weights for policy 1, policy_version 1001623 (0.0007) [2023-12-27 12:35:42,314][20347] Updated weights for policy 0, policy_version 1002254 (0.0008) [2023-12-27 12:35:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 513064960. Throughput: 0: 10323.9, 1: 10126.5. Samples: 513055860. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:42,363][19170] Avg episode reward: [(0, '717.200'), (1, '597.860')] [2023-12-27 12:35:42,376][20347] Updated weights for policy 0, policy_version 1002264 (0.0008) [2023-12-27 12:35:42,423][20347] Updated weights for policy 0, policy_version 1002274 (0.0008) [2023-12-27 12:35:43,001][20381] Updated weights for policy 1, policy_version 1001633 (0.0007) [2023-12-27 12:35:43,056][20381] Updated weights for policy 1, policy_version 1001643 (0.0008) [2023-12-27 12:35:43,104][20381] Updated weights for policy 1, policy_version 1001653 (0.0007) [2023-12-27 12:35:43,143][20347] Updated weights for policy 0, policy_version 1002284 (0.0009) [2023-12-27 12:35:43,194][20347] Updated weights for policy 0, policy_version 1002294 (0.0010) [2023-12-27 12:35:43,249][20347] Updated weights for policy 0, policy_version 1002304 (0.0010) [2023-12-27 12:35:43,805][20381] Updated weights for policy 1, policy_version 1001663 (0.0007) [2023-12-27 12:35:43,865][20381] Updated weights for policy 1, policy_version 1001673 (0.0007) [2023-12-27 12:35:43,920][20381] Updated weights for policy 1, policy_version 1001683 (0.0005) [2023-12-27 12:35:43,935][20347] Updated weights for policy 0, policy_version 1002314 (0.0009) [2023-12-27 12:35:43,983][20347] Updated weights for policy 0, policy_version 1002324 (0.0005) [2023-12-27 12:35:44,034][20347] Updated weights for policy 0, policy_version 1002334 (0.0009) [2023-12-27 12:35:44,115][20347] Updated weights for policy 0, policy_version 1002344 (0.0009) [2023-12-27 12:35:44,603][20381] Updated weights for policy 1, policy_version 1001693 (0.0007) [2023-12-27 12:35:44,657][20381] Updated weights for policy 1, policy_version 1001703 (0.0008) [2023-12-27 12:35:44,717][20381] Updated weights for policy 1, policy_version 1001713 (0.0007) [2023-12-27 12:35:44,787][20347] Updated weights for policy 0, policy_version 1002355 (0.0007) [2023-12-27 12:35:44,839][20347] Updated weights for policy 0, policy_version 1002365 (0.0008) [2023-12-27 12:35:44,891][20347] Updated weights for policy 0, policy_version 1002375 (0.0008) [2023-12-27 12:35:45,420][20381] Updated weights for policy 1, policy_version 1001723 (0.0009) [2023-12-27 12:35:45,479][20381] Updated weights for policy 1, policy_version 1001733 (0.0011) [2023-12-27 12:35:45,538][20381] Updated weights for policy 1, policy_version 1001743 (0.0010) [2023-12-27 12:35:45,639][20347] Updated weights for policy 0, policy_version 1002385 (0.0010) [2023-12-27 12:35:45,703][20347] Updated weights for policy 0, policy_version 1002395 (0.0008) [2023-12-27 12:35:45,768][20347] Updated weights for policy 0, policy_version 1002405 (0.0006) [2023-12-27 12:35:46,190][20381] Updated weights for policy 1, policy_version 1001753 (0.0010) [2023-12-27 12:35:46,246][20381] Updated weights for policy 1, policy_version 1001763 (0.0005) [2023-12-27 12:35:46,297][20381] Updated weights for policy 1, policy_version 1001773 (0.0009) [2023-12-27 12:35:46,354][20381] Updated weights for policy 1, policy_version 1001783 (0.0010) [2023-12-27 12:35:46,376][20347] Updated weights for policy 0, policy_version 1002415 (0.0010) [2023-12-27 12:35:46,421][20347] Updated weights for policy 0, policy_version 1002425 (0.0010) [2023-12-27 12:35:46,466][20347] Updated weights for policy 0, policy_version 1002435 (0.0010) [2023-12-27 12:35:46,967][20381] Updated weights for policy 1, policy_version 1001793 (0.0008) [2023-12-27 12:35:47,035][20381] Updated weights for policy 1, policy_version 1001803 (0.0006) [2023-12-27 12:35:47,054][20347] Updated weights for policy 0, policy_version 1002445 (0.0008) [2023-12-27 12:35:47,090][20381] Updated weights for policy 1, policy_version 1001813 (0.0006) [2023-12-27 12:35:47,108][20347] Updated weights for policy 0, policy_version 1002455 (0.0005) [2023-12-27 12:35:47,158][20347] Updated weights for policy 0, policy_version 1002465 (0.0005) [2023-12-27 12:35:47,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 513171456. Throughput: 0: 10373.2, 1: 10182.8. Samples: 513178344. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:47,364][19170] Avg episode reward: [(0, '756.480'), (1, '627.580')] [2023-12-27 12:35:47,731][20381] Updated weights for policy 1, policy_version 1001823 (0.0006) [2023-12-27 12:35:47,786][20347] Updated weights for policy 0, policy_version 1002475 (0.0007) [2023-12-27 12:35:47,792][20381] Updated weights for policy 1, policy_version 1001833 (0.0006) [2023-12-27 12:35:47,841][20381] Updated weights for policy 1, policy_version 1001843 (0.0007) [2023-12-27 12:35:47,848][20347] Updated weights for policy 0, policy_version 1002485 (0.0007) [2023-12-27 12:35:47,901][20347] Updated weights for policy 0, policy_version 1002495 (0.0007) [2023-12-27 12:35:48,493][20381] Updated weights for policy 1, policy_version 1001853 (0.0007) [2023-12-27 12:35:48,548][20381] Updated weights for policy 1, policy_version 1001863 (0.0008) [2023-12-27 12:35:48,578][20347] Updated weights for policy 0, policy_version 1002505 (0.0008) [2023-12-27 12:35:48,604][20381] Updated weights for policy 1, policy_version 1001873 (0.0009) [2023-12-27 12:35:48,636][20347] Updated weights for policy 0, policy_version 1002515 (0.0010) [2023-12-27 12:35:48,695][20347] Updated weights for policy 0, policy_version 1002525 (0.0011) [2023-12-27 12:35:48,754][20347] Updated weights for policy 0, policy_version 1002535 (0.0010) [2023-12-27 12:35:49,301][20381] Updated weights for policy 1, policy_version 1001883 (0.0008) [2023-12-27 12:35:49,367][20381] Updated weights for policy 1, policy_version 1001893 (0.0008) [2023-12-27 12:35:49,388][20347] Updated weights for policy 0, policy_version 1002545 (0.0009) [2023-12-27 12:35:49,434][20381] Updated weights for policy 1, policy_version 1001903 (0.0010) [2023-12-27 12:35:49,446][20347] Updated weights for policy 0, policy_version 1002555 (0.0008) [2023-12-27 12:35:49,507][20347] Updated weights for policy 0, policy_version 1002565 (0.0008) [2023-12-27 12:35:50,207][20381] Updated weights for policy 1, policy_version 1001913 (0.0007) [2023-12-27 12:35:50,214][20347] Updated weights for policy 0, policy_version 1002575 (0.0007) [2023-12-27 12:35:50,259][20381] Updated weights for policy 1, policy_version 1001923 (0.0008) [2023-12-27 12:35:50,267][20347] Updated weights for policy 0, policy_version 1002585 (0.0006) [2023-12-27 12:35:50,304][20381] Updated weights for policy 1, policy_version 1001933 (0.0006) [2023-12-27 12:35:50,320][20347] Updated weights for policy 0, policy_version 1002595 (0.0008) [2023-12-27 12:35:50,359][20381] Updated weights for policy 1, policy_version 1001943 (0.0010) [2023-12-27 12:35:51,045][20347] Updated weights for policy 0, policy_version 1002605 (0.0006) [2023-12-27 12:35:51,048][20381] Updated weights for policy 1, policy_version 1001953 (0.0007) [2023-12-27 12:35:51,103][20347] Updated weights for policy 0, policy_version 1002615 (0.0006) [2023-12-27 12:35:51,109][20381] Updated weights for policy 1, policy_version 1001963 (0.0006) [2023-12-27 12:35:51,165][20347] Updated weights for policy 0, policy_version 1002625 (0.0006) [2023-12-27 12:35:51,174][20381] Updated weights for policy 1, policy_version 1001973 (0.0008) [2023-12-27 12:35:51,871][20347] Updated weights for policy 0, policy_version 1002635 (0.0007) [2023-12-27 12:35:51,913][20381] Updated weights for policy 1, policy_version 1001983 (0.0008) [2023-12-27 12:35:51,928][20347] Updated weights for policy 0, policy_version 1002645 (0.0006) [2023-12-27 12:35:51,967][20381] Updated weights for policy 1, policy_version 1001993 (0.0008) [2023-12-27 12:35:51,981][20347] Updated weights for policy 0, policy_version 1002655 (0.0007) [2023-12-27 12:35:52,026][20381] Updated weights for policy 1, policy_version 1002003 (0.0006) [2023-12-27 12:35:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 513269760. Throughput: 0: 10367.3, 1: 10191.2. Samples: 513239324. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:52,363][19170] Avg episode reward: [(0, '794.990'), (1, '617.720')] [2023-12-27 12:35:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001002008_256540672.pth... [2023-12-27 12:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001002664_256729088.pth... [2023-12-27 12:35:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001000824_256237568.pth [2023-12-27 12:35:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001001448_256417792.pth [2023-12-27 12:35:52,671][20347] Updated weights for policy 0, policy_version 1002665 (0.0007) [2023-12-27 12:35:52,734][20347] Updated weights for policy 0, policy_version 1002675 (0.0007) [2023-12-27 12:35:52,747][20381] Updated weights for policy 1, policy_version 1002013 (0.0008) [2023-12-27 12:35:52,792][20347] Updated weights for policy 0, policy_version 1002685 (0.0007) [2023-12-27 12:35:52,799][20381] Updated weights for policy 1, policy_version 1002023 (0.0008) [2023-12-27 12:35:52,847][20347] Updated weights for policy 0, policy_version 1002695 (0.0007) [2023-12-27 12:35:52,848][20381] Updated weights for policy 1, policy_version 1002033 (0.0006) [2023-12-27 12:35:53,442][20347] Updated weights for policy 0, policy_version 1002705 (0.0007) [2023-12-27 12:35:53,505][20347] Updated weights for policy 0, policy_version 1002715 (0.0007) [2023-12-27 12:35:53,566][20347] Updated weights for policy 0, policy_version 1002725 (0.0007) [2023-12-27 12:35:53,569][20381] Updated weights for policy 1, policy_version 1002043 (0.0007) [2023-12-27 12:35:53,619][20381] Updated weights for policy 1, policy_version 1002053 (0.0007) [2023-12-27 12:35:53,671][20381] Updated weights for policy 1, policy_version 1002063 (0.0007) [2023-12-27 12:35:54,210][20347] Updated weights for policy 0, policy_version 1002735 (0.0007) [2023-12-27 12:35:54,269][20347] Updated weights for policy 0, policy_version 1002745 (0.0008) [2023-12-27 12:35:54,324][20347] Updated weights for policy 0, policy_version 1002755 (0.0007) [2023-12-27 12:35:54,364][20381] Updated weights for policy 1, policy_version 1002073 (0.0007) [2023-12-27 12:35:54,426][20381] Updated weights for policy 1, policy_version 1002083 (0.0005) [2023-12-27 12:35:54,488][20381] Updated weights for policy 1, policy_version 1002093 (0.0005) [2023-12-27 12:35:54,549][20381] Updated weights for policy 1, policy_version 1002103 (0.0007) [2023-12-27 12:35:54,948][20347] Updated weights for policy 0, policy_version 1002765 (0.0008) [2023-12-27 12:35:55,010][20347] Updated weights for policy 0, policy_version 1002775 (0.0008) [2023-12-27 12:35:55,074][20347] Updated weights for policy 0, policy_version 1002785 (0.0006) [2023-12-27 12:35:55,130][20381] Updated weights for policy 1, policy_version 1002113 (0.0011) [2023-12-27 12:35:55,186][20381] Updated weights for policy 1, policy_version 1002123 (0.0011) [2023-12-27 12:35:55,241][20381] Updated weights for policy 1, policy_version 1002133 (0.0010) [2023-12-27 12:35:55,684][20347] Updated weights for policy 0, policy_version 1002795 (0.0006) [2023-12-27 12:35:55,741][20347] Updated weights for policy 0, policy_version 1002805 (0.0005) [2023-12-27 12:35:55,802][20347] Updated weights for policy 0, policy_version 1002815 (0.0006) [2023-12-27 12:35:55,868][20381] Updated weights for policy 1, policy_version 1002143 (0.0011) [2023-12-27 12:35:55,918][20381] Updated weights for policy 1, policy_version 1002153 (0.0011) [2023-12-27 12:35:55,963][20381] Updated weights for policy 1, policy_version 1002163 (0.0010) [2023-12-27 12:35:56,409][20347] Updated weights for policy 0, policy_version 1002825 (0.0007) [2023-12-27 12:35:56,459][20347] Updated weights for policy 0, policy_version 1002835 (0.0007) [2023-12-27 12:35:56,507][20347] Updated weights for policy 0, policy_version 1002845 (0.0008) [2023-12-27 12:35:56,557][20347] Updated weights for policy 0, policy_version 1002855 (0.0007) [2023-12-27 12:35:56,631][20381] Updated weights for policy 1, policy_version 1002173 (0.0010) [2023-12-27 12:35:56,685][20381] Updated weights for policy 1, policy_version 1002183 (0.0010) [2023-12-27 12:35:56,733][20381] Updated weights for policy 1, policy_version 1002193 (0.0010) [2023-12-27 12:35:57,189][20347] Updated weights for policy 0, policy_version 1002865 (0.0007) [2023-12-27 12:35:57,236][20347] Updated weights for policy 0, policy_version 1002875 (0.0007) [2023-12-27 12:35:57,280][20347] Updated weights for policy 0, policy_version 1002885 (0.0007) [2023-12-27 12:35:57,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 513376256. Throughput: 0: 10353.2, 1: 10231.8. Samples: 513363420. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:35:57,364][19170] Avg episode reward: [(0, '863.480'), (1, '627.520')] [2023-12-27 12:35:57,391][20381] Updated weights for policy 1, policy_version 1002203 (0.0010) [2023-12-27 12:35:57,445][20381] Updated weights for policy 1, policy_version 1002213 (0.0010) [2023-12-27 12:35:57,490][20381] Updated weights for policy 1, policy_version 1002223 (0.0010) [2023-12-27 12:35:57,972][20347] Updated weights for policy 0, policy_version 1002895 (0.0007) [2023-12-27 12:35:58,025][20347] Updated weights for policy 0, policy_version 1002905 (0.0007) [2023-12-27 12:35:58,083][20347] Updated weights for policy 0, policy_version 1002915 (0.0007) [2023-12-27 12:35:58,183][20381] Updated weights for policy 1, policy_version 1002233 (0.0010) [2023-12-27 12:35:58,246][20381] Updated weights for policy 1, policy_version 1002243 (0.0008) [2023-12-27 12:35:58,308][20381] Updated weights for policy 1, policy_version 1002253 (0.0008) [2023-12-27 12:35:58,369][20381] Updated weights for policy 1, policy_version 1002263 (0.0008) [2023-12-27 12:35:58,881][20347] Updated weights for policy 0, policy_version 1002925 (0.0008) [2023-12-27 12:35:58,954][20347] Updated weights for policy 0, policy_version 1002935 (0.0008) [2023-12-27 12:35:59,013][20347] Updated weights for policy 0, policy_version 1002945 (0.0009) [2023-12-27 12:35:59,147][20381] Updated weights for policy 1, policy_version 1002273 (0.0008) [2023-12-27 12:35:59,205][20381] Updated weights for policy 1, policy_version 1002283 (0.0008) [2023-12-27 12:35:59,274][20381] Updated weights for policy 1, policy_version 1002293 (0.0009) [2023-12-27 12:35:59,723][20347] Updated weights for policy 0, policy_version 1002955 (0.0008) [2023-12-27 12:35:59,784][20347] Updated weights for policy 0, policy_version 1002965 (0.0007) [2023-12-27 12:35:59,858][20347] Updated weights for policy 0, policy_version 1002975 (0.0006) [2023-12-27 12:36:00,048][20381] Updated weights for policy 1, policy_version 1002303 (0.0007) [2023-12-27 12:36:00,103][20381] Updated weights for policy 1, policy_version 1002313 (0.0009) [2023-12-27 12:36:00,159][20381] Updated weights for policy 1, policy_version 1002323 (0.0008) [2023-12-27 12:36:00,543][20347] Updated weights for policy 0, policy_version 1002985 (0.0007) [2023-12-27 12:36:00,600][20347] Updated weights for policy 0, policy_version 1002995 (0.0005) [2023-12-27 12:36:00,651][20347] Updated weights for policy 0, policy_version 1003005 (0.0005) [2023-12-27 12:36:00,711][20347] Updated weights for policy 0, policy_version 1003015 (0.0005) [2023-12-27 12:36:00,807][20381] Updated weights for policy 1, policy_version 1002333 (0.0006) [2023-12-27 12:36:00,871][20381] Updated weights for policy 1, policy_version 1002343 (0.0007) [2023-12-27 12:36:00,925][20381] Updated weights for policy 1, policy_version 1002353 (0.0006) [2023-12-27 12:36:01,323][20291] KL-divergence is very high: 148.1997 [2023-12-27 12:36:01,324][20347] Updated weights for policy 0, policy_version 1003025 (0.0007) [2023-12-27 12:36:01,337][20291] KL-divergence is very high: 173.7345 [2023-12-27 12:36:01,376][20291] KL-divergence is very high: 106.5807 [2023-12-27 12:36:01,388][20347] Updated weights for policy 0, policy_version 1003035 (0.0008) [2023-12-27 12:36:01,451][20347] Updated weights for policy 0, policy_version 1003045 (0.0007) [2023-12-27 12:36:01,642][20381] Updated weights for policy 1, policy_version 1002363 (0.0006) [2023-12-27 12:36:01,702][20381] Updated weights for policy 1, policy_version 1002373 (0.0007) [2023-12-27 12:36:01,763][20381] Updated weights for policy 1, policy_version 1002383 (0.0007) [2023-12-27 12:36:02,117][20347] Updated weights for policy 0, policy_version 1003055 (0.0006) [2023-12-27 12:36:02,174][20347] Updated weights for policy 0, policy_version 1003065 (0.0005) [2023-12-27 12:36:02,234][20347] Updated weights for policy 0, policy_version 1003075 (0.0005) [2023-12-27 12:36:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 513474560. Throughput: 0: 10370.0, 1: 10186.7. Samples: 513483832. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:02,363][19170] Avg episode reward: [(0, '912.490'), (1, '646.590')] [2023-12-27 12:36:02,441][20381] Updated weights for policy 1, policy_version 1002393 (0.0006) [2023-12-27 12:36:02,498][20381] Updated weights for policy 1, policy_version 1002403 (0.0009) [2023-12-27 12:36:02,547][20381] Updated weights for policy 1, policy_version 1002413 (0.0008) [2023-12-27 12:36:02,598][20381] Updated weights for policy 1, policy_version 1002423 (0.0008) [2023-12-27 12:36:02,776][20347] Updated weights for policy 0, policy_version 1003085 (0.0006) [2023-12-27 12:36:02,829][20347] Updated weights for policy 0, policy_version 1003095 (0.0008) [2023-12-27 12:36:02,881][20347] Updated weights for policy 0, policy_version 1003105 (0.0008) [2023-12-27 12:36:03,353][20381] Updated weights for policy 1, policy_version 1002433 (0.0008) [2023-12-27 12:36:03,413][20381] Updated weights for policy 1, policy_version 1002443 (0.0008) [2023-12-27 12:36:03,467][20381] Updated weights for policy 1, policy_version 1002453 (0.0007) [2023-12-27 12:36:03,472][20347] Updated weights for policy 0, policy_version 1003115 (0.0007) [2023-12-27 12:36:03,530][20347] Updated weights for policy 0, policy_version 1003125 (0.0007) [2023-12-27 12:36:03,586][20347] Updated weights for policy 0, policy_version 1003135 (0.0005) [2023-12-27 12:36:04,207][20381] Updated weights for policy 1, policy_version 1002463 (0.0007) [2023-12-27 12:36:04,267][20381] Updated weights for policy 1, policy_version 1002473 (0.0008) [2023-12-27 12:36:04,280][20347] Updated weights for policy 0, policy_version 1003145 (0.0006) [2023-12-27 12:36:04,328][20381] Updated weights for policy 1, policy_version 1002483 (0.0007) [2023-12-27 12:36:04,342][20347] Updated weights for policy 0, policy_version 1003155 (0.0007) [2023-12-27 12:36:04,406][20347] Updated weights for policy 0, policy_version 1003165 (0.0007) [2023-12-27 12:36:04,462][20347] Updated weights for policy 0, policy_version 1003175 (0.0007) [2023-12-27 12:36:05,013][20381] Updated weights for policy 1, policy_version 1002493 (0.0007) [2023-12-27 12:36:05,072][20381] Updated weights for policy 1, policy_version 1002503 (0.0007) [2023-12-27 12:36:05,124][20347] Updated weights for policy 0, policy_version 1003185 (0.0009) [2023-12-27 12:36:05,125][20381] Updated weights for policy 1, policy_version 1002513 (0.0007) [2023-12-27 12:36:05,170][20347] Updated weights for policy 0, policy_version 1003195 (0.0007) [2023-12-27 12:36:05,222][20347] Updated weights for policy 0, policy_version 1003205 (0.0010) [2023-12-27 12:36:05,728][20381] Updated weights for policy 1, policy_version 1002523 (0.0007) [2023-12-27 12:36:05,778][20381] Updated weights for policy 1, policy_version 1002533 (0.0005) [2023-12-27 12:36:05,826][20381] Updated weights for policy 1, policy_version 1002543 (0.0005) [2023-12-27 12:36:05,840][20347] Updated weights for policy 0, policy_version 1003215 (0.0009) [2023-12-27 12:36:05,895][20347] Updated weights for policy 0, policy_version 1003225 (0.0010) [2023-12-27 12:36:05,952][20347] Updated weights for policy 0, policy_version 1003235 (0.0010) [2023-12-27 12:36:06,534][20381] Updated weights for policy 1, policy_version 1002553 (0.0006) [2023-12-27 12:36:06,594][20381] Updated weights for policy 1, policy_version 1002563 (0.0009) [2023-12-27 12:36:06,660][20381] Updated weights for policy 1, policy_version 1002573 (0.0007) [2023-12-27 12:36:06,671][20347] Updated weights for policy 0, policy_version 1003245 (0.0005) [2023-12-27 12:36:06,720][20381] Updated weights for policy 1, policy_version 1002583 (0.0008) [2023-12-27 12:36:06,725][20347] Updated weights for policy 0, policy_version 1003255 (0.0006) [2023-12-27 12:36:06,785][20347] Updated weights for policy 0, policy_version 1003265 (0.0009) [2023-12-27 12:36:07,363][19170] Fps is (10 sec: 19660.9, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 513572864. Throughput: 0: 10421.2, 1: 10184.6. Samples: 513545852. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:07,364][19170] Avg episode reward: [(0, '922.360'), (1, '587.970')] [2023-12-27 12:36:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001003272_256884736.pth... [2023-12-27 12:36:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001002056_256573440.pth [2023-12-27 12:36:07,385][20381] Updated weights for policy 1, policy_version 1002593 (0.0005) [2023-12-27 12:36:07,436][20347] Updated weights for policy 0, policy_version 1003275 (0.0007) [2023-12-27 12:36:07,443][20381] Updated weights for policy 1, policy_version 1002603 (0.0005) [2023-12-27 12:36:07,488][20347] Updated weights for policy 0, policy_version 1003285 (0.0007) [2023-12-27 12:36:07,489][20291] KL-divergence is very high: 144.5461 [2023-12-27 12:36:07,494][20291] KL-divergence is very high: 134.3835 [2023-12-27 12:36:07,506][20381] Updated weights for policy 1, policy_version 1002613 (0.0005) [2023-12-27 12:36:07,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001002616_256696320.pth... [2023-12-27 12:36:07,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001001432_256393216.pth [2023-12-27 12:36:07,541][20347] Updated weights for policy 0, policy_version 1003295 (0.0008) [2023-12-27 12:36:08,084][20381] Updated weights for policy 1, policy_version 1002623 (0.0009) [2023-12-27 12:36:08,133][20381] Updated weights for policy 1, policy_version 1002633 (0.0010) [2023-12-27 12:36:08,183][20381] Updated weights for policy 1, policy_version 1002643 (0.0010) [2023-12-27 12:36:08,188][20347] Updated weights for policy 0, policy_version 1003305 (0.0007) [2023-12-27 12:36:08,245][20347] Updated weights for policy 0, policy_version 1003315 (0.0007) [2023-12-27 12:36:08,290][20347] Updated weights for policy 0, policy_version 1003325 (0.0007) [2023-12-27 12:36:08,341][20347] Updated weights for policy 0, policy_version 1003335 (0.0007) [2023-12-27 12:36:08,897][20381] Updated weights for policy 1, policy_version 1002653 (0.0011) [2023-12-27 12:36:08,936][20292] KL-divergence is very high: 109.1432 [2023-12-27 12:36:08,962][20381] Updated weights for policy 1, policy_version 1002663 (0.0009) [2023-12-27 12:36:08,969][20292] KL-divergence is very high: 117.7706 [2023-12-27 12:36:08,985][20292] KL-divergence is very high: 191.7780 [2023-12-27 12:36:09,015][20381] Updated weights for policy 1, policy_version 1002673 (0.0006) [2023-12-27 12:36:09,022][20347] Updated weights for policy 0, policy_version 1003345 (0.0007) [2023-12-27 12:36:09,024][20292] KL-divergence is very high: 130.5166 [2023-12-27 12:36:09,069][20347] Updated weights for policy 0, policy_version 1003355 (0.0006) [2023-12-27 12:36:09,124][20347] Updated weights for policy 0, policy_version 1003365 (0.0008) [2023-12-27 12:36:09,721][20381] Updated weights for policy 1, policy_version 1002683 (0.0008) [2023-12-27 12:36:09,782][20381] Updated weights for policy 1, policy_version 1002693 (0.0007) [2023-12-27 12:36:09,850][20381] Updated weights for policy 1, policy_version 1002703 (0.0009) [2023-12-27 12:36:09,880][20347] Updated weights for policy 0, policy_version 1003375 (0.0009) [2023-12-27 12:36:09,940][20347] Updated weights for policy 0, policy_version 1003385 (0.0007) [2023-12-27 12:36:09,998][20347] Updated weights for policy 0, policy_version 1003395 (0.0008) [2023-12-27 12:36:10,521][20381] Updated weights for policy 1, policy_version 1002713 (0.0008) [2023-12-27 12:36:10,581][20381] Updated weights for policy 1, policy_version 1002723 (0.0011) [2023-12-27 12:36:10,642][20347] Updated weights for policy 0, policy_version 1003405 (0.0006) [2023-12-27 12:36:10,645][20381] Updated weights for policy 1, policy_version 1002733 (0.0011) [2023-12-27 12:36:10,699][20347] Updated weights for policy 0, policy_version 1003415 (0.0006) [2023-12-27 12:36:10,705][20381] Updated weights for policy 1, policy_version 1002743 (0.0011) [2023-12-27 12:36:10,756][20347] Updated weights for policy 0, policy_version 1003425 (0.0006) [2023-12-27 12:36:11,435][20347] Updated weights for policy 0, policy_version 1003435 (0.0006) [2023-12-27 12:36:11,450][20381] Updated weights for policy 1, policy_version 1002753 (0.0008) [2023-12-27 12:36:11,493][20347] Updated weights for policy 0, policy_version 1003445 (0.0008) [2023-12-27 12:36:11,507][20381] Updated weights for policy 1, policy_version 1002763 (0.0007) [2023-12-27 12:36:11,560][20347] Updated weights for policy 0, policy_version 1003455 (0.0006) [2023-12-27 12:36:11,566][20381] Updated weights for policy 1, policy_version 1002773 (0.0006) [2023-12-27 12:36:12,241][20347] Updated weights for policy 0, policy_version 1003465 (0.0007) [2023-12-27 12:36:12,306][20381] Updated weights for policy 1, policy_version 1002783 (0.0007) [2023-12-27 12:36:12,308][20347] Updated weights for policy 0, policy_version 1003475 (0.0008) [2023-12-27 12:36:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.3, 300 sec: 20438.3). Total num frames: 513671168. Throughput: 0: 10443.4, 1: 10141.5. Samples: 513669916. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:12,363][19170] Avg episode reward: [(0, '863.640'), (1, '637.030')] [2023-12-27 12:36:12,373][20347] Updated weights for policy 0, policy_version 1003485 (0.0007) [2023-12-27 12:36:12,374][20381] Updated weights for policy 1, policy_version 1002793 (0.0007) [2023-12-27 12:36:12,428][20381] Updated weights for policy 1, policy_version 1002803 (0.0006) [2023-12-27 12:36:12,429][20347] Updated weights for policy 0, policy_version 1003495 (0.0006) [2023-12-27 12:36:12,996][20381] Updated weights for policy 1, policy_version 1002813 (0.0006) [2023-12-27 12:36:13,054][20381] Updated weights for policy 1, policy_version 1002823 (0.0007) [2023-12-27 12:36:13,115][20381] Updated weights for policy 1, policy_version 1002833 (0.0008) [2023-12-27 12:36:13,174][20347] Updated weights for policy 0, policy_version 1003505 (0.0008) [2023-12-27 12:36:13,224][20347] Updated weights for policy 0, policy_version 1003515 (0.0006) [2023-12-27 12:36:13,274][20347] Updated weights for policy 0, policy_version 1003525 (0.0005) [2023-12-27 12:36:13,793][20381] Updated weights for policy 1, policy_version 1002843 (0.0006) [2023-12-27 12:36:13,842][20381] Updated weights for policy 1, policy_version 1002853 (0.0007) [2023-12-27 12:36:13,895][20381] Updated weights for policy 1, policy_version 1002863 (0.0007) [2023-12-27 12:36:13,948][20347] Updated weights for policy 0, policy_version 1003535 (0.0007) [2023-12-27 12:36:13,999][20347] Updated weights for policy 0, policy_version 1003545 (0.0007) [2023-12-27 12:36:14,057][20347] Updated weights for policy 0, policy_version 1003555 (0.0008) [2023-12-27 12:36:14,509][20381] Updated weights for policy 1, policy_version 1002873 (0.0006) [2023-12-27 12:36:14,577][20381] Updated weights for policy 1, policy_version 1002883 (0.0005) [2023-12-27 12:36:14,629][20381] Updated weights for policy 1, policy_version 1002893 (0.0006) [2023-12-27 12:36:14,678][20381] Updated weights for policy 1, policy_version 1002903 (0.0008) [2023-12-27 12:36:14,800][20347] Updated weights for policy 0, policy_version 1003565 (0.0008) [2023-12-27 12:36:14,850][20347] Updated weights for policy 0, policy_version 1003575 (0.0007) [2023-12-27 12:36:14,900][20347] Updated weights for policy 0, policy_version 1003585 (0.0009) [2023-12-27 12:36:15,315][20381] Updated weights for policy 1, policy_version 1002913 (0.0008) [2023-12-27 12:36:15,376][20381] Updated weights for policy 1, policy_version 1002923 (0.0008) [2023-12-27 12:36:15,439][20381] Updated weights for policy 1, policy_version 1002933 (0.0008) [2023-12-27 12:36:15,576][20347] Updated weights for policy 0, policy_version 1003595 (0.0007) [2023-12-27 12:36:15,635][20347] Updated weights for policy 0, policy_version 1003605 (0.0006) [2023-12-27 12:36:15,691][20347] Updated weights for policy 0, policy_version 1003615 (0.0006) [2023-12-27 12:36:16,067][20381] Updated weights for policy 1, policy_version 1002943 (0.0006) [2023-12-27 12:36:16,131][20381] Updated weights for policy 1, policy_version 1002953 (0.0006) [2023-12-27 12:36:16,186][20381] Updated weights for policy 1, policy_version 1002963 (0.0005) [2023-12-27 12:36:16,354][20347] Updated weights for policy 0, policy_version 1003625 (0.0007) [2023-12-27 12:36:16,416][20347] Updated weights for policy 0, policy_version 1003635 (0.0010) [2023-12-27 12:36:16,468][20347] Updated weights for policy 0, policy_version 1003645 (0.0010) [2023-12-27 12:36:16,519][20347] Updated weights for policy 0, policy_version 1003655 (0.0010) [2023-12-27 12:36:16,801][20381] Updated weights for policy 1, policy_version 1002973 (0.0007) [2023-12-27 12:36:16,853][20381] Updated weights for policy 1, policy_version 1002983 (0.0007) [2023-12-27 12:36:16,908][20381] Updated weights for policy 1, policy_version 1002993 (0.0007) [2023-12-27 12:36:17,197][20347] Updated weights for policy 0, policy_version 1003665 (0.0008) [2023-12-27 12:36:17,256][20347] Updated weights for policy 0, policy_version 1003675 (0.0008) [2023-12-27 12:36:17,315][20347] Updated weights for policy 0, policy_version 1003685 (0.0008) [2023-12-27 12:36:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 513785856. Throughput: 0: 10354.1, 1: 10124.4. Samples: 513793196. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:17,363][19170] Avg episode reward: [(0, '824.090'), (1, '607.960')] [2023-12-27 12:36:17,477][20381] Updated weights for policy 1, policy_version 1003003 (0.0006) [2023-12-27 12:36:17,531][20381] Updated weights for policy 1, policy_version 1003013 (0.0005) [2023-12-27 12:36:17,587][20381] Updated weights for policy 1, policy_version 1003023 (0.0006) [2023-12-27 12:36:17,987][20347] Updated weights for policy 0, policy_version 1003695 (0.0008) [2023-12-27 12:36:18,036][20347] Updated weights for policy 0, policy_version 1003705 (0.0008) [2023-12-27 12:36:18,086][20347] Updated weights for policy 0, policy_version 1003715 (0.0007) [2023-12-27 12:36:18,238][20381] Updated weights for policy 1, policy_version 1003033 (0.0007) [2023-12-27 12:36:18,306][20381] Updated weights for policy 1, policy_version 1003043 (0.0006) [2023-12-27 12:36:18,374][20381] Updated weights for policy 1, policy_version 1003053 (0.0007) [2023-12-27 12:36:18,430][20381] Updated weights for policy 1, policy_version 1003063 (0.0007) [2023-12-27 12:36:18,825][20347] Updated weights for policy 0, policy_version 1003725 (0.0006) [2023-12-27 12:36:18,882][20347] Updated weights for policy 0, policy_version 1003735 (0.0008) [2023-12-27 12:36:18,939][20347] Updated weights for policy 0, policy_version 1003745 (0.0007) [2023-12-27 12:36:19,092][20381] Updated weights for policy 1, policy_version 1003073 (0.0007) [2023-12-27 12:36:19,140][20381] Updated weights for policy 1, policy_version 1003083 (0.0008) [2023-12-27 12:36:19,203][20381] Updated weights for policy 1, policy_version 1003093 (0.0008) [2023-12-27 12:36:19,677][20347] Updated weights for policy 0, policy_version 1003755 (0.0008) [2023-12-27 12:36:19,744][20347] Updated weights for policy 0, policy_version 1003765 (0.0009) [2023-12-27 12:36:19,806][20347] Updated weights for policy 0, policy_version 1003775 (0.0008) [2023-12-27 12:36:19,981][20381] Updated weights for policy 1, policy_version 1003103 (0.0010) [2023-12-27 12:36:20,030][20381] Updated weights for policy 1, policy_version 1003113 (0.0008) [2023-12-27 12:36:20,086][20381] Updated weights for policy 1, policy_version 1003123 (0.0008) [2023-12-27 12:36:20,580][20347] Updated weights for policy 0, policy_version 1003785 (0.0008) [2023-12-27 12:36:20,639][20347] Updated weights for policy 0, policy_version 1003795 (0.0011) [2023-12-27 12:36:20,698][20347] Updated weights for policy 0, policy_version 1003805 (0.0010) [2023-12-27 12:36:20,756][20347] Updated weights for policy 0, policy_version 1003815 (0.0009) [2023-12-27 12:36:20,809][20381] Updated weights for policy 1, policy_version 1003133 (0.0009) [2023-12-27 12:36:20,862][20381] Updated weights for policy 1, policy_version 1003143 (0.0007) [2023-12-27 12:36:20,928][20381] Updated weights for policy 1, policy_version 1003153 (0.0007) [2023-12-27 12:36:21,459][20347] Updated weights for policy 0, policy_version 1003825 (0.0006) [2023-12-27 12:36:21,520][20347] Updated weights for policy 0, policy_version 1003835 (0.0006) [2023-12-27 12:36:21,581][20347] Updated weights for policy 0, policy_version 1003845 (0.0007) [2023-12-27 12:36:21,715][20381] Updated weights for policy 1, policy_version 1003163 (0.0008) [2023-12-27 12:36:21,790][20381] Updated weights for policy 1, policy_version 1003173 (0.0007) [2023-12-27 12:36:21,853][20381] Updated weights for policy 1, policy_version 1003183 (0.0010) [2023-12-27 12:36:22,244][20347] Updated weights for policy 0, policy_version 1003855 (0.0006) [2023-12-27 12:36:22,312][20347] Updated weights for policy 0, policy_version 1003865 (0.0008) [2023-12-27 12:36:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 513875968. Throughput: 0: 10290.1, 1: 10136.5. Samples: 513851636. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:22,363][19170] Avg episode reward: [(0, '804.370'), (1, '539.000')] [2023-12-27 12:36:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001003192_256843776.pth... [2023-12-27 12:36:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001002008_256540672.pth [2023-12-27 12:36:22,378][20347] Updated weights for policy 0, policy_version 1003875 (0.0006) [2023-12-27 12:36:22,409][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001003880_257040384.pth... [2023-12-27 12:36:22,414][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001002664_256729088.pth [2023-12-27 12:36:22,550][20381] Updated weights for policy 1, policy_version 1003193 (0.0007) [2023-12-27 12:36:22,611][20381] Updated weights for policy 1, policy_version 1003203 (0.0008) [2023-12-27 12:36:22,675][20381] Updated weights for policy 1, policy_version 1003213 (0.0007) [2023-12-27 12:36:22,731][20381] Updated weights for policy 1, policy_version 1003223 (0.0009) [2023-12-27 12:36:22,938][20347] Updated weights for policy 0, policy_version 1003885 (0.0008) [2023-12-27 12:36:22,984][20347] Updated weights for policy 0, policy_version 1003895 (0.0010) [2023-12-27 12:36:23,038][20347] Updated weights for policy 0, policy_version 1003905 (0.0009) [2023-12-27 12:36:23,367][20381] Updated weights for policy 1, policy_version 1003233 (0.0006) [2023-12-27 12:36:23,420][20381] Updated weights for policy 1, policy_version 1003243 (0.0006) [2023-12-27 12:36:23,470][20381] Updated weights for policy 1, policy_version 1003253 (0.0010) [2023-12-27 12:36:23,663][20347] Updated weights for policy 0, policy_version 1003915 (0.0009) [2023-12-27 12:36:23,716][20347] Updated weights for policy 0, policy_version 1003925 (0.0009) [2023-12-27 12:36:23,767][20347] Updated weights for policy 0, policy_version 1003935 (0.0010) [2023-12-27 12:36:24,153][20381] Updated weights for policy 1, policy_version 1003263 (0.0011) [2023-12-27 12:36:24,198][20381] Updated weights for policy 1, policy_version 1003273 (0.0010) [2023-12-27 12:36:24,251][20381] Updated weights for policy 1, policy_version 1003283 (0.0011) [2023-12-27 12:36:24,389][20347] Updated weights for policy 0, policy_version 1003945 (0.0010) [2023-12-27 12:36:24,432][20347] Updated weights for policy 0, policy_version 1003955 (0.0006) [2023-12-27 12:36:24,478][20347] Updated weights for policy 0, policy_version 1003965 (0.0005) [2023-12-27 12:36:24,528][20347] Updated weights for policy 0, policy_version 1003975 (0.0006) [2023-12-27 12:36:24,931][20381] Updated weights for policy 1, policy_version 1003293 (0.0009) [2023-12-27 12:36:24,979][20381] Updated weights for policy 1, policy_version 1003303 (0.0006) [2023-12-27 12:36:25,034][20381] Updated weights for policy 1, policy_version 1003313 (0.0005) [2023-12-27 12:36:25,179][20347] Updated weights for policy 0, policy_version 1003985 (0.0007) [2023-12-27 12:36:25,229][20347] Updated weights for policy 0, policy_version 1003995 (0.0007) [2023-12-27 12:36:25,290][20347] Updated weights for policy 0, policy_version 1004005 (0.0007) [2023-12-27 12:36:25,580][20381] Updated weights for policy 1, policy_version 1003323 (0.0007) [2023-12-27 12:36:25,628][20381] Updated weights for policy 1, policy_version 1003333 (0.0010) [2023-12-27 12:36:25,677][20381] Updated weights for policy 1, policy_version 1003343 (0.0010) [2023-12-27 12:36:25,904][20347] Updated weights for policy 0, policy_version 1004015 (0.0010) [2023-12-27 12:36:25,956][20347] Updated weights for policy 0, policy_version 1004025 (0.0010) [2023-12-27 12:36:26,014][20347] Updated weights for policy 0, policy_version 1004035 (0.0009) [2023-12-27 12:36:26,365][20381] Updated weights for policy 1, policy_version 1003353 (0.0010) [2023-12-27 12:36:26,417][20381] Updated weights for policy 1, policy_version 1003363 (0.0010) [2023-12-27 12:36:26,471][20381] Updated weights for policy 1, policy_version 1003373 (0.0010) [2023-12-27 12:36:26,525][20381] Updated weights for policy 1, policy_version 1003383 (0.0010) [2023-12-27 12:36:26,687][20347] Updated weights for policy 0, policy_version 1004045 (0.0008) [2023-12-27 12:36:26,744][20347] Updated weights for policy 0, policy_version 1004055 (0.0010) [2023-12-27 12:36:26,805][20347] Updated weights for policy 0, policy_version 1004065 (0.0007) [2023-12-27 12:36:27,134][20381] Updated weights for policy 1, policy_version 1003393 (0.0009) [2023-12-27 12:36:27,187][20381] Updated weights for policy 1, policy_version 1003403 (0.0005) [2023-12-27 12:36:27,246][20381] Updated weights for policy 1, policy_version 1003413 (0.0005) [2023-12-27 12:36:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 513990656. Throughput: 0: 10318.0, 1: 10168.5. Samples: 513977752. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:27,363][19170] Avg episode reward: [(0, '814.540'), (1, '627.180')] [2023-12-27 12:36:27,465][20347] Updated weights for policy 0, policy_version 1004075 (0.0010) [2023-12-27 12:36:27,520][20347] Updated weights for policy 0, policy_version 1004085 (0.0006) [2023-12-27 12:36:27,577][20347] Updated weights for policy 0, policy_version 1004095 (0.0007) [2023-12-27 12:36:27,847][20381] Updated weights for policy 1, policy_version 1003423 (0.0007) [2023-12-27 12:36:27,910][20381] Updated weights for policy 1, policy_version 1003433 (0.0005) [2023-12-27 12:36:27,973][20381] Updated weights for policy 1, policy_version 1003443 (0.0005) [2023-12-27 12:36:28,174][20347] Updated weights for policy 0, policy_version 1004105 (0.0010) [2023-12-27 12:36:28,228][20347] Updated weights for policy 0, policy_version 1004115 (0.0010) [2023-12-27 12:36:28,285][20347] Updated weights for policy 0, policy_version 1004125 (0.0010) [2023-12-27 12:36:28,348][20347] Updated weights for policy 0, policy_version 1004135 (0.0010) [2023-12-27 12:36:28,570][20381] Updated weights for policy 1, policy_version 1003453 (0.0006) [2023-12-27 12:36:28,622][20381] Updated weights for policy 1, policy_version 1003463 (0.0008) [2023-12-27 12:36:28,679][20381] Updated weights for policy 1, policy_version 1003473 (0.0010) [2023-12-27 12:36:29,076][20347] Updated weights for policy 0, policy_version 1004145 (0.0007) [2023-12-27 12:36:29,133][20347] Updated weights for policy 0, policy_version 1004155 (0.0006) [2023-12-27 12:36:29,187][20347] Updated weights for policy 0, policy_version 1004165 (0.0006) [2023-12-27 12:36:29,308][20381] Updated weights for policy 1, policy_version 1003483 (0.0008) [2023-12-27 12:36:29,375][20381] Updated weights for policy 1, policy_version 1003493 (0.0008) [2023-12-27 12:36:29,433][20381] Updated weights for policy 1, policy_version 1003503 (0.0011) [2023-12-27 12:36:29,844][20347] Updated weights for policy 0, policy_version 1004175 (0.0008) [2023-12-27 12:36:29,898][20347] Updated weights for policy 0, policy_version 1004185 (0.0007) [2023-12-27 12:36:29,961][20347] Updated weights for policy 0, policy_version 1004195 (0.0006) [2023-12-27 12:36:30,107][20381] Updated weights for policy 1, policy_version 1003513 (0.0010) [2023-12-27 12:36:30,166][20381] Updated weights for policy 1, policy_version 1003523 (0.0011) [2023-12-27 12:36:30,172][20292] KL-divergence is very high: 148.1530 [2023-12-27 12:36:30,179][20292] KL-divergence is very high: 153.9630 [2023-12-27 12:36:30,225][20381] Updated weights for policy 1, policy_version 1003533 (0.0011) [2023-12-27 12:36:30,275][20381] Updated weights for policy 1, policy_version 1003543 (0.0010) [2023-12-27 12:36:30,615][20347] Updated weights for policy 0, policy_version 1004205 (0.0008) [2023-12-27 12:36:30,669][20347] Updated weights for policy 0, policy_version 1004215 (0.0007) [2023-12-27 12:36:30,729][20347] Updated weights for policy 0, policy_version 1004225 (0.0005) [2023-12-27 12:36:30,948][20381] Updated weights for policy 1, policy_version 1003553 (0.0006) [2023-12-27 12:36:30,995][20381] Updated weights for policy 1, policy_version 1003563 (0.0005) [2023-12-27 12:36:31,058][20381] Updated weights for policy 1, policy_version 1003573 (0.0010) [2023-12-27 12:36:31,323][20347] Updated weights for policy 0, policy_version 1004235 (0.0007) [2023-12-27 12:36:31,390][20347] Updated weights for policy 0, policy_version 1004245 (0.0012) [2023-12-27 12:36:31,455][20347] Updated weights for policy 0, policy_version 1004255 (0.0010) [2023-12-27 12:36:31,778][20381] Updated weights for policy 1, policy_version 1003583 (0.0007) [2023-12-27 12:36:31,824][20381] Updated weights for policy 1, policy_version 1003593 (0.0007) [2023-12-27 12:36:31,879][20381] Updated weights for policy 1, policy_version 1003603 (0.0005) [2023-12-27 12:36:32,134][20347] Updated weights for policy 0, policy_version 1004265 (0.0010) [2023-12-27 12:36:32,196][20347] Updated weights for policy 0, policy_version 1004275 (0.0007) [2023-12-27 12:36:32,256][20347] Updated weights for policy 0, policy_version 1004285 (0.0008) [2023-12-27 12:36:32,314][20347] Updated weights for policy 0, policy_version 1004295 (0.0008) [2023-12-27 12:36:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 514097152. Throughput: 0: 10347.9, 1: 10232.3. Samples: 514104448. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:32,363][19170] Avg episode reward: [(0, '849.540'), (1, '666.370')] [2023-12-27 12:36:32,547][20381] Updated weights for policy 1, policy_version 1003613 (0.0006) [2023-12-27 12:36:32,605][20381] Updated weights for policy 1, policy_version 1003623 (0.0005) [2023-12-27 12:36:32,665][20381] Updated weights for policy 1, policy_version 1003633 (0.0005) [2023-12-27 12:36:32,980][20347] Updated weights for policy 0, policy_version 1004305 (0.0006) [2023-12-27 12:36:33,042][20347] Updated weights for policy 0, policy_version 1004315 (0.0006) [2023-12-27 12:36:33,096][20347] Updated weights for policy 0, policy_version 1004325 (0.0007) [2023-12-27 12:36:33,284][20381] Updated weights for policy 1, policy_version 1003643 (0.0006) [2023-12-27 12:36:33,329][20381] Updated weights for policy 1, policy_version 1003653 (0.0005) [2023-12-27 12:36:33,382][20381] Updated weights for policy 1, policy_version 1003663 (0.0007) [2023-12-27 12:36:33,693][20347] Updated weights for policy 0, policy_version 1004335 (0.0007) [2023-12-27 12:36:33,740][20347] Updated weights for policy 0, policy_version 1004345 (0.0007) [2023-12-27 12:36:33,797][20347] Updated weights for policy 0, policy_version 1004355 (0.0008) [2023-12-27 12:36:34,022][20381] Updated weights for policy 1, policy_version 1003673 (0.0005) [2023-12-27 12:36:34,073][20381] Updated weights for policy 1, policy_version 1003683 (0.0007) [2023-12-27 12:36:34,126][20381] Updated weights for policy 1, policy_version 1003693 (0.0005) [2023-12-27 12:36:34,192][20381] Updated weights for policy 1, policy_version 1003703 (0.0008) [2023-12-27 12:36:34,530][20347] Updated weights for policy 0, policy_version 1004365 (0.0008) [2023-12-27 12:36:34,587][20347] Updated weights for policy 0, policy_version 1004375 (0.0010) [2023-12-27 12:36:34,649][20347] Updated weights for policy 0, policy_version 1004385 (0.0010) [2023-12-27 12:36:34,809][20381] Updated weights for policy 1, policy_version 1003713 (0.0005) [2023-12-27 12:36:34,867][20381] Updated weights for policy 1, policy_version 1003723 (0.0006) [2023-12-27 12:36:34,922][20381] Updated weights for policy 1, policy_version 1003733 (0.0005) [2023-12-27 12:36:35,252][20347] Updated weights for policy 0, policy_version 1004395 (0.0007) [2023-12-27 12:36:35,307][20347] Updated weights for policy 0, policy_version 1004405 (0.0010) [2023-12-27 12:36:35,361][20347] Updated weights for policy 0, policy_version 1004415 (0.0010) [2023-12-27 12:36:35,517][20381] Updated weights for policy 1, policy_version 1003743 (0.0006) [2023-12-27 12:36:35,569][20381] Updated weights for policy 1, policy_version 1003753 (0.0006) [2023-12-27 12:36:35,628][20381] Updated weights for policy 1, policy_version 1003763 (0.0008) [2023-12-27 12:36:35,984][20347] Updated weights for policy 0, policy_version 1004425 (0.0007) [2023-12-27 12:36:36,033][20347] Updated weights for policy 0, policy_version 1004435 (0.0010) [2023-12-27 12:36:36,088][20347] Updated weights for policy 0, policy_version 1004445 (0.0010) [2023-12-27 12:36:36,151][20347] Updated weights for policy 0, policy_version 1004455 (0.0009) [2023-12-27 12:36:36,381][20381] Updated weights for policy 1, policy_version 1003773 (0.0008) [2023-12-27 12:36:36,444][20381] Updated weights for policy 1, policy_version 1003783 (0.0007) [2023-12-27 12:36:36,507][20381] Updated weights for policy 1, policy_version 1003793 (0.0006) [2023-12-27 12:36:36,840][20347] Updated weights for policy 0, policy_version 1004465 (0.0006) [2023-12-27 12:36:36,897][20347] Updated weights for policy 0, policy_version 1004475 (0.0005) [2023-12-27 12:36:36,959][20347] Updated weights for policy 0, policy_version 1004485 (0.0008) [2023-12-27 12:36:37,159][20381] Updated weights for policy 1, policy_version 1003803 (0.0006) [2023-12-27 12:36:37,212][20381] Updated weights for policy 1, policy_version 1003813 (0.0006) [2023-12-27 12:36:37,257][20381] Updated weights for policy 1, policy_version 1003823 (0.0005) [2023-12-27 12:36:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 514203648. Throughput: 0: 10365.5, 1: 10300.3. Samples: 514169284. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:37,363][19170] Avg episode reward: [(0, '773.890'), (1, '529.010')] [2023-12-27 12:36:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001003832_257007616.pth... [2023-12-27 12:36:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001004488_257196032.pth... [2023-12-27 12:36:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001002616_256696320.pth [2023-12-27 12:36:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001003272_256884736.pth [2023-12-27 12:36:37,599][20347] Updated weights for policy 0, policy_version 1004495 (0.0010) [2023-12-27 12:36:37,660][20347] Updated weights for policy 0, policy_version 1004505 (0.0010) [2023-12-27 12:36:37,722][20347] Updated weights for policy 0, policy_version 1004515 (0.0010) [2023-12-27 12:36:37,824][20381] Updated weights for policy 1, policy_version 1003833 (0.0006) [2023-12-27 12:36:37,878][20381] Updated weights for policy 1, policy_version 1003843 (0.0005) [2023-12-27 12:36:37,938][20381] Updated weights for policy 1, policy_version 1003853 (0.0005) [2023-12-27 12:36:37,991][20381] Updated weights for policy 1, policy_version 1003863 (0.0005) [2023-12-27 12:36:38,356][20347] Updated weights for policy 0, policy_version 1004525 (0.0009) [2023-12-27 12:36:38,419][20347] Updated weights for policy 0, policy_version 1004535 (0.0008) [2023-12-27 12:36:38,475][20347] Updated weights for policy 0, policy_version 1004545 (0.0008) [2023-12-27 12:36:38,619][20381] Updated weights for policy 1, policy_version 1003873 (0.0007) [2023-12-27 12:36:38,681][20381] Updated weights for policy 1, policy_version 1003883 (0.0007) [2023-12-27 12:36:38,742][20381] Updated weights for policy 1, policy_version 1003893 (0.0007) [2023-12-27 12:36:39,072][20347] Updated weights for policy 0, policy_version 1004555 (0.0008) [2023-12-27 12:36:39,128][20347] Updated weights for policy 0, policy_version 1004565 (0.0007) [2023-12-27 12:36:39,184][20347] Updated weights for policy 0, policy_version 1004575 (0.0006) [2023-12-27 12:36:39,325][20381] Updated weights for policy 1, policy_version 1003903 (0.0008) [2023-12-27 12:36:39,388][20381] Updated weights for policy 1, policy_version 1003913 (0.0008) [2023-12-27 12:36:39,464][20381] Updated weights for policy 1, policy_version 1003923 (0.0008) [2023-12-27 12:36:39,881][20347] Updated weights for policy 0, policy_version 1004585 (0.0007) [2023-12-27 12:36:39,953][20347] Updated weights for policy 0, policy_version 1004595 (0.0008) [2023-12-27 12:36:40,012][20347] Updated weights for policy 0, policy_version 1004605 (0.0008) [2023-12-27 12:36:40,071][20347] Updated weights for policy 0, policy_version 1004615 (0.0006) [2023-12-27 12:36:40,129][20381] Updated weights for policy 1, policy_version 1003933 (0.0008) [2023-12-27 12:36:40,190][20381] Updated weights for policy 1, policy_version 1003943 (0.0008) [2023-12-27 12:36:40,255][20381] Updated weights for policy 1, policy_version 1003953 (0.0007) [2023-12-27 12:36:40,731][20347] Updated weights for policy 0, policy_version 1004625 (0.0008) [2023-12-27 12:36:40,783][20347] Updated weights for policy 0, policy_version 1004635 (0.0007) [2023-12-27 12:36:40,838][20347] Updated weights for policy 0, policy_version 1004645 (0.0007) [2023-12-27 12:36:40,895][20381] Updated weights for policy 1, policy_version 1003963 (0.0007) [2023-12-27 12:36:40,938][20381] Updated weights for policy 1, policy_version 1003973 (0.0008) [2023-12-27 12:36:40,984][20381] Updated weights for policy 1, policy_version 1003983 (0.0009) [2023-12-27 12:36:41,524][20347] Updated weights for policy 0, policy_version 1004655 (0.0007) [2023-12-27 12:36:41,578][20347] Updated weights for policy 0, policy_version 1004665 (0.0007) [2023-12-27 12:36:41,641][20347] Updated weights for policy 0, policy_version 1004675 (0.0008) [2023-12-27 12:36:41,793][20381] Updated weights for policy 1, policy_version 1003993 (0.0009) [2023-12-27 12:36:41,859][20381] Updated weights for policy 1, policy_version 1004003 (0.0008) [2023-12-27 12:36:41,916][20381] Updated weights for policy 1, policy_version 1004013 (0.0007) [2023-12-27 12:36:41,976][20381] Updated weights for policy 1, policy_version 1004023 (0.0008) [2023-12-27 12:36:42,297][20347] Updated weights for policy 0, policy_version 1004685 (0.0007) [2023-12-27 12:36:42,354][20347] Updated weights for policy 0, policy_version 1004695 (0.0011) [2023-12-27 12:36:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 514301952. Throughput: 0: 10366.2, 1: 10361.1. Samples: 514296140. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:42,363][19170] Avg episode reward: [(0, '814.790'), (1, '607.850')] [2023-12-27 12:36:42,416][20347] Updated weights for policy 0, policy_version 1004705 (0.0010) [2023-12-27 12:36:42,741][20381] Updated weights for policy 1, policy_version 1004033 (0.0007) [2023-12-27 12:36:42,801][20381] Updated weights for policy 1, policy_version 1004043 (0.0007) [2023-12-27 12:36:42,854][20381] Updated weights for policy 1, policy_version 1004053 (0.0008) [2023-12-27 12:36:43,058][20347] Updated weights for policy 0, policy_version 1004715 (0.0009) [2023-12-27 12:36:43,114][20347] Updated weights for policy 0, policy_version 1004725 (0.0005) [2023-12-27 12:36:43,178][20347] Updated weights for policy 0, policy_version 1004735 (0.0005) [2023-12-27 12:36:43,559][20381] Updated weights for policy 1, policy_version 1004063 (0.0008) [2023-12-27 12:36:43,614][20381] Updated weights for policy 1, policy_version 1004073 (0.0008) [2023-12-27 12:36:43,672][20381] Updated weights for policy 1, policy_version 1004083 (0.0008) [2023-12-27 12:36:43,794][20347] Updated weights for policy 0, policy_version 1004745 (0.0006) [2023-12-27 12:36:43,849][20347] Updated weights for policy 0, policy_version 1004755 (0.0008) [2023-12-27 12:36:43,901][20347] Updated weights for policy 0, policy_version 1004765 (0.0008) [2023-12-27 12:36:43,957][20347] Updated weights for policy 0, policy_version 1004775 (0.0008) [2023-12-27 12:36:44,365][20381] Updated weights for policy 1, policy_version 1004093 (0.0007) [2023-12-27 12:36:44,410][20381] Updated weights for policy 1, policy_version 1004103 (0.0007) [2023-12-27 12:36:44,460][20381] Updated weights for policy 1, policy_version 1004113 (0.0007) [2023-12-27 12:36:44,676][20347] Updated weights for policy 0, policy_version 1004785 (0.0007) [2023-12-27 12:36:44,733][20347] Updated weights for policy 0, policy_version 1004795 (0.0008) [2023-12-27 12:36:44,797][20347] Updated weights for policy 0, policy_version 1004805 (0.0012) [2023-12-27 12:36:45,149][20381] Updated weights for policy 1, policy_version 1004123 (0.0007) [2023-12-27 12:36:45,212][20381] Updated weights for policy 1, policy_version 1004133 (0.0008) [2023-12-27 12:36:45,269][20381] Updated weights for policy 1, policy_version 1004143 (0.0007) [2023-12-27 12:36:45,536][20347] Updated weights for policy 0, policy_version 1004815 (0.0008) [2023-12-27 12:36:45,587][20347] Updated weights for policy 0, policy_version 1004825 (0.0006) [2023-12-27 12:36:45,636][20347] Updated weights for policy 0, policy_version 1004835 (0.0007) [2023-12-27 12:36:45,946][20381] Updated weights for policy 1, policy_version 1004153 (0.0008) [2023-12-27 12:36:46,005][20381] Updated weights for policy 1, policy_version 1004163 (0.0008) [2023-12-27 12:36:46,074][20381] Updated weights for policy 1, policy_version 1004173 (0.0008) [2023-12-27 12:36:46,132][20381] Updated weights for policy 1, policy_version 1004183 (0.0008) [2023-12-27 12:36:46,263][20347] Updated weights for policy 0, policy_version 1004845 (0.0006) [2023-12-27 12:36:46,319][20347] Updated weights for policy 0, policy_version 1004855 (0.0007) [2023-12-27 12:36:46,377][20347] Updated weights for policy 0, policy_version 1004865 (0.0010) [2023-12-27 12:36:46,839][20381] Updated weights for policy 1, policy_version 1004193 (0.0007) [2023-12-27 12:36:46,891][20381] Updated weights for policy 1, policy_version 1004203 (0.0006) [2023-12-27 12:36:46,947][20381] Updated weights for policy 1, policy_version 1004213 (0.0005) [2023-12-27 12:36:47,051][20347] Updated weights for policy 0, policy_version 1004875 (0.0009) [2023-12-27 12:36:47,113][20347] Updated weights for policy 0, policy_version 1004885 (0.0008) [2023-12-27 12:36:47,169][20347] Updated weights for policy 0, policy_version 1004895 (0.0006) [2023-12-27 12:36:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 514408448. Throughput: 0: 10396.9, 1: 10347.3. Samples: 514417320. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:47,363][19170] Avg episode reward: [(0, '803.530'), (1, '637.020')] [2023-12-27 12:36:47,623][20381] Updated weights for policy 1, policy_version 1004223 (0.0007) [2023-12-27 12:36:47,688][20381] Updated weights for policy 1, policy_version 1004233 (0.0008) [2023-12-27 12:36:47,744][20381] Updated weights for policy 1, policy_version 1004243 (0.0008) [2023-12-27 12:36:47,765][20347] Updated weights for policy 0, policy_version 1004905 (0.0006) [2023-12-27 12:36:47,821][20347] Updated weights for policy 0, policy_version 1004915 (0.0008) [2023-12-27 12:36:47,878][20347] Updated weights for policy 0, policy_version 1004925 (0.0007) [2023-12-27 12:36:47,933][20347] Updated weights for policy 0, policy_version 1004935 (0.0008) [2023-12-27 12:36:48,435][20381] Updated weights for policy 1, policy_version 1004253 (0.0007) [2023-12-27 12:36:48,499][20381] Updated weights for policy 1, policy_version 1004263 (0.0007) [2023-12-27 12:36:48,560][20381] Updated weights for policy 1, policy_version 1004273 (0.0006) [2023-12-27 12:36:48,615][20347] Updated weights for policy 0, policy_version 1004945 (0.0008) [2023-12-27 12:36:48,663][20347] Updated weights for policy 0, policy_version 1004955 (0.0009) [2023-12-27 12:36:48,722][20347] Updated weights for policy 0, policy_version 1004965 (0.0008) [2023-12-27 12:36:49,187][20381] Updated weights for policy 1, policy_version 1004283 (0.0007) [2023-12-27 12:36:49,250][20381] Updated weights for policy 1, policy_version 1004293 (0.0008) [2023-12-27 12:36:49,309][20381] Updated weights for policy 1, policy_version 1004303 (0.0008) [2023-12-27 12:36:49,400][20347] Updated weights for policy 0, policy_version 1004975 (0.0007) [2023-12-27 12:36:49,457][20347] Updated weights for policy 0, policy_version 1004985 (0.0007) [2023-12-27 12:36:49,513][20347] Updated weights for policy 0, policy_version 1004995 (0.0007) [2023-12-27 12:36:50,037][20381] Updated weights for policy 1, policy_version 1004313 (0.0008) [2023-12-27 12:36:50,095][20381] Updated weights for policy 1, policy_version 1004323 (0.0008) [2023-12-27 12:36:50,157][20381] Updated weights for policy 1, policy_version 1004333 (0.0008) [2023-12-27 12:36:50,185][20347] Updated weights for policy 0, policy_version 1005005 (0.0008) [2023-12-27 12:36:50,210][20381] Updated weights for policy 1, policy_version 1004343 (0.0007) [2023-12-27 12:36:50,245][20347] Updated weights for policy 0, policy_version 1005015 (0.0006) [2023-12-27 12:36:50,305][20347] Updated weights for policy 0, policy_version 1005025 (0.0008) [2023-12-27 12:36:50,964][20381] Updated weights for policy 1, policy_version 1004353 (0.0008) [2023-12-27 12:36:51,002][20347] Updated weights for policy 0, policy_version 1005035 (0.0008) [2023-12-27 12:36:51,022][20381] Updated weights for policy 1, policy_version 1004363 (0.0009) [2023-12-27 12:36:51,057][20291] KL-divergence is very high: 138.6695 [2023-12-27 12:36:51,065][20291] KL-divergence is very high: 122.2565 [2023-12-27 12:36:51,073][20347] Updated weights for policy 0, policy_version 1005045 (0.0008) [2023-12-27 12:36:51,083][20381] Updated weights for policy 1, policy_version 1004373 (0.0006) [2023-12-27 12:36:51,112][20291] KL-divergence is very high: 125.3488 [2023-12-27 12:36:51,139][20347] Updated weights for policy 0, policy_version 1005055 (0.0009) [2023-12-27 12:36:51,905][20381] Updated weights for policy 1, policy_version 1004383 (0.0007) [2023-12-27 12:36:51,914][20347] Updated weights for policy 0, policy_version 1005065 (0.0007) [2023-12-27 12:36:51,965][20381] Updated weights for policy 1, policy_version 1004393 (0.0007) [2023-12-27 12:36:51,973][20347] Updated weights for policy 0, policy_version 1005075 (0.0008) [2023-12-27 12:36:51,976][20291] KL-divergence is very high: 165.2414 [2023-12-27 12:36:51,990][20291] KL-divergence is very high: 106.5273 [2023-12-27 12:36:52,024][20381] Updated weights for policy 1, policy_version 1004403 (0.0006) [2023-12-27 12:36:52,037][20347] Updated weights for policy 0, policy_version 1005085 (0.0006) [2023-12-27 12:36:52,091][20347] Updated weights for policy 0, policy_version 1005095 (0.0007) [2023-12-27 12:36:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 514506752. Throughput: 0: 10358.2, 1: 10361.2. Samples: 514478220. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:52,363][19170] Avg episode reward: [(0, '795.130'), (1, '715.150')] [2023-12-27 12:36:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001004408_257155072.pth... [2023-12-27 12:36:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001005096_257351680.pth... [2023-12-27 12:36:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001003192_256843776.pth [2023-12-27 12:36:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001003880_257040384.pth [2023-12-27 12:36:52,731][20381] Updated weights for policy 1, policy_version 1004413 (0.0007) [2023-12-27 12:36:52,791][20381] Updated weights for policy 1, policy_version 1004423 (0.0007) [2023-12-27 12:36:52,809][20347] Updated weights for policy 0, policy_version 1005105 (0.0007) [2023-12-27 12:36:52,851][20381] Updated weights for policy 1, policy_version 1004433 (0.0008) [2023-12-27 12:36:52,869][20347] Updated weights for policy 0, policy_version 1005115 (0.0009) [2023-12-27 12:36:52,924][20347] Updated weights for policy 0, policy_version 1005125 (0.0010) [2023-12-27 12:36:53,496][20381] Updated weights for policy 1, policy_version 1004443 (0.0006) [2023-12-27 12:36:53,554][20381] Updated weights for policy 1, policy_version 1004453 (0.0006) [2023-12-27 12:36:53,574][20347] Updated weights for policy 0, policy_version 1005135 (0.0009) [2023-12-27 12:36:53,602][20381] Updated weights for policy 1, policy_version 1004463 (0.0006) [2023-12-27 12:36:53,632][20347] Updated weights for policy 0, policy_version 1005145 (0.0010) [2023-12-27 12:36:53,689][20347] Updated weights for policy 0, policy_version 1005155 (0.0010) [2023-12-27 12:36:54,236][20381] Updated weights for policy 1, policy_version 1004473 (0.0006) [2023-12-27 12:36:54,280][20381] Updated weights for policy 1, policy_version 1004483 (0.0005) [2023-12-27 12:36:54,329][20381] Updated weights for policy 1, policy_version 1004493 (0.0007) [2023-12-27 12:36:54,337][20347] Updated weights for policy 0, policy_version 1005165 (0.0010) [2023-12-27 12:36:54,384][20381] Updated weights for policy 1, policy_version 1004503 (0.0007) [2023-12-27 12:36:54,399][20347] Updated weights for policy 0, policy_version 1005175 (0.0009) [2023-12-27 12:36:54,455][20347] Updated weights for policy 0, policy_version 1005185 (0.0007) [2023-12-27 12:36:55,029][20347] Updated weights for policy 0, policy_version 1005195 (0.0006) [2023-12-27 12:36:55,064][20381] Updated weights for policy 1, policy_version 1004513 (0.0006) [2023-12-27 12:36:55,090][20347] Updated weights for policy 0, policy_version 1005205 (0.0006) [2023-12-27 12:36:55,115][20381] Updated weights for policy 1, policy_version 1004523 (0.0007) [2023-12-27 12:36:55,151][20347] Updated weights for policy 0, policy_version 1005215 (0.0008) [2023-12-27 12:36:55,167][20381] Updated weights for policy 1, policy_version 1004533 (0.0007) [2023-12-27 12:36:55,780][20347] Updated weights for policy 0, policy_version 1005225 (0.0006) [2023-12-27 12:36:55,812][20381] Updated weights for policy 1, policy_version 1004543 (0.0006) [2023-12-27 12:36:55,844][20347] Updated weights for policy 0, policy_version 1005235 (0.0007) [2023-12-27 12:36:55,861][20381] Updated weights for policy 1, policy_version 1004553 (0.0006) [2023-12-27 12:36:55,908][20347] Updated weights for policy 0, policy_version 1005245 (0.0006) [2023-12-27 12:36:55,909][20381] Updated weights for policy 1, policy_version 1004563 (0.0006) [2023-12-27 12:36:55,972][20347] Updated weights for policy 0, policy_version 1005255 (0.0008) [2023-12-27 12:36:56,530][20381] Updated weights for policy 1, policy_version 1004573 (0.0009) [2023-12-27 12:36:56,583][20381] Updated weights for policy 1, policy_version 1004583 (0.0010) [2023-12-27 12:36:56,585][20347] Updated weights for policy 0, policy_version 1005265 (0.0005) [2023-12-27 12:36:56,628][20381] Updated weights for policy 1, policy_version 1004593 (0.0010) [2023-12-27 12:36:56,629][20347] Updated weights for policy 0, policy_version 1005275 (0.0005) [2023-12-27 12:36:56,682][20347] Updated weights for policy 0, policy_version 1005285 (0.0007) [2023-12-27 12:36:57,321][20381] Updated weights for policy 1, policy_version 1004603 (0.0010) [2023-12-27 12:36:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 514605056. Throughput: 0: 10321.3, 1: 10350.3. Samples: 514600136. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:36:57,363][19170] Avg episode reward: [(0, '824.510'), (1, '784.210')] [2023-12-27 12:36:57,380][20381] Updated weights for policy 1, policy_version 1004613 (0.0008) [2023-12-27 12:36:57,393][20347] Updated weights for policy 0, policy_version 1005295 (0.0006) [2023-12-27 12:36:57,435][20381] Updated weights for policy 1, policy_version 1004623 (0.0007) [2023-12-27 12:36:57,455][20347] Updated weights for policy 0, policy_version 1005305 (0.0006) [2023-12-27 12:36:57,510][20347] Updated weights for policy 0, policy_version 1005315 (0.0006) [2023-12-27 12:36:58,062][20381] Updated weights for policy 1, policy_version 1004633 (0.0010) [2023-12-27 12:36:58,122][20381] Updated weights for policy 1, policy_version 1004643 (0.0010) [2023-12-27 12:36:58,164][20381] Updated weights for policy 1, policy_version 1004653 (0.0010) [2023-12-27 12:36:58,206][20347] Updated weights for policy 0, policy_version 1005325 (0.0008) [2023-12-27 12:36:58,224][20381] Updated weights for policy 1, policy_version 1004663 (0.0009) [2023-12-27 12:36:58,266][20347] Updated weights for policy 0, policy_version 1005335 (0.0008) [2023-12-27 12:36:58,321][20347] Updated weights for policy 0, policy_version 1005345 (0.0006) [2023-12-27 12:36:58,976][20381] Updated weights for policy 1, policy_version 1004673 (0.0008) [2023-12-27 12:36:59,037][20381] Updated weights for policy 1, policy_version 1004683 (0.0007) [2023-12-27 12:36:59,094][20381] Updated weights for policy 1, policy_version 1004693 (0.0008) [2023-12-27 12:36:59,094][20347] Updated weights for policy 0, policy_version 1005355 (0.0009) [2023-12-27 12:36:59,152][20347] Updated weights for policy 0, policy_version 1005365 (0.0007) [2023-12-27 12:36:59,212][20347] Updated weights for policy 0, policy_version 1005375 (0.0006) [2023-12-27 12:36:59,900][20347] Updated weights for policy 0, policy_version 1005385 (0.0008) [2023-12-27 12:36:59,905][20381] Updated weights for policy 1, policy_version 1004703 (0.0008) [2023-12-27 12:36:59,965][20347] Updated weights for policy 0, policy_version 1005395 (0.0007) [2023-12-27 12:36:59,970][20381] Updated weights for policy 1, policy_version 1004713 (0.0008) [2023-12-27 12:37:00,018][20347] Updated weights for policy 0, policy_version 1005405 (0.0006) [2023-12-27 12:37:00,028][20381] Updated weights for policy 1, policy_version 1004723 (0.0008) [2023-12-27 12:37:00,071][20347] Updated weights for policy 0, policy_version 1005415 (0.0007) [2023-12-27 12:37:00,688][20381] Updated weights for policy 1, policy_version 1004733 (0.0009) [2023-12-27 12:37:00,736][20381] Updated weights for policy 1, policy_version 1004743 (0.0011) [2023-12-27 12:37:00,763][20347] Updated weights for policy 0, policy_version 1005425 (0.0010) [2023-12-27 12:37:00,790][20381] Updated weights for policy 1, policy_version 1004753 (0.0010) [2023-12-27 12:37:00,815][20347] Updated weights for policy 0, policy_version 1005435 (0.0010) [2023-12-27 12:37:00,867][20347] Updated weights for policy 0, policy_version 1005445 (0.0010) [2023-12-27 12:37:01,449][20381] Updated weights for policy 1, policy_version 1004763 (0.0006) [2023-12-27 12:37:01,504][20381] Updated weights for policy 1, policy_version 1004773 (0.0008) [2023-12-27 12:37:01,561][20381] Updated weights for policy 1, policy_version 1004783 (0.0008) [2023-12-27 12:37:01,592][20347] Updated weights for policy 0, policy_version 1005455 (0.0007) [2023-12-27 12:37:01,652][20347] Updated weights for policy 0, policy_version 1005465 (0.0007) [2023-12-27 12:37:01,711][20347] Updated weights for policy 0, policy_version 1005475 (0.0007) [2023-12-27 12:37:02,278][20381] Updated weights for policy 1, policy_version 1004793 (0.0008) [2023-12-27 12:37:02,342][20381] Updated weights for policy 1, policy_version 1004803 (0.0006) [2023-12-27 12:37:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 514703360. Throughput: 0: 10322.2, 1: 10269.6. Samples: 514719824. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:37:02,363][19170] Avg episode reward: [(0, '814.570'), (1, '754.760')] [2023-12-27 12:37:02,398][20381] Updated weights for policy 1, policy_version 1004813 (0.0010) [2023-12-27 12:37:02,407][20347] Updated weights for policy 0, policy_version 1005485 (0.0009) [2023-12-27 12:37:02,453][20381] Updated weights for policy 1, policy_version 1004823 (0.0010) [2023-12-27 12:37:02,456][20347] Updated weights for policy 0, policy_version 1005495 (0.0009) [2023-12-27 12:37:02,509][20347] Updated weights for policy 0, policy_version 1005505 (0.0007) [2023-12-27 12:37:03,095][20381] Updated weights for policy 1, policy_version 1004833 (0.0006) [2023-12-27 12:37:03,145][20381] Updated weights for policy 1, policy_version 1004843 (0.0007) [2023-12-27 12:37:03,156][20347] Updated weights for policy 0, policy_version 1005515 (0.0008) [2023-12-27 12:37:03,200][20381] Updated weights for policy 1, policy_version 1004853 (0.0010) [2023-12-27 12:37:03,205][20347] Updated weights for policy 0, policy_version 1005525 (0.0010) [2023-12-27 12:37:03,257][20347] Updated weights for policy 0, policy_version 1005535 (0.0010) [2023-12-27 12:37:03,798][20381] Updated weights for policy 1, policy_version 1004863 (0.0007) [2023-12-27 12:37:03,857][20381] Updated weights for policy 1, policy_version 1004873 (0.0006) [2023-12-27 12:37:03,915][20381] Updated weights for policy 1, policy_version 1004883 (0.0008) [2023-12-27 12:37:03,980][20347] Updated weights for policy 0, policy_version 1005545 (0.0010) [2023-12-27 12:37:04,035][20347] Updated weights for policy 0, policy_version 1005555 (0.0008) [2023-12-27 12:37:04,093][20347] Updated weights for policy 0, policy_version 1005565 (0.0008) [2023-12-27 12:37:04,148][20347] Updated weights for policy 0, policy_version 1005575 (0.0007) [2023-12-27 12:37:04,677][20381] Updated weights for policy 1, policy_version 1004893 (0.0009) [2023-12-27 12:37:04,729][20381] Updated weights for policy 1, policy_version 1004903 (0.0007) [2023-12-27 12:37:04,783][20381] Updated weights for policy 1, policy_version 1004913 (0.0007) [2023-12-27 12:37:04,831][20347] Updated weights for policy 0, policy_version 1005585 (0.0010) [2023-12-27 12:37:04,884][20347] Updated weights for policy 0, policy_version 1005595 (0.0010) [2023-12-27 12:37:04,939][20347] Updated weights for policy 0, policy_version 1005605 (0.0010) [2023-12-27 12:37:05,472][20381] Updated weights for policy 1, policy_version 1004923 (0.0007) [2023-12-27 12:37:05,520][20381] Updated weights for policy 1, policy_version 1004933 (0.0010) [2023-12-27 12:37:05,584][20381] Updated weights for policy 1, policy_version 1004943 (0.0010) [2023-12-27 12:37:05,601][20347] Updated weights for policy 0, policy_version 1005615 (0.0007) [2023-12-27 12:37:05,654][20347] Updated weights for policy 0, policy_version 1005625 (0.0005) [2023-12-27 12:37:05,701][20347] Updated weights for policy 0, policy_version 1005635 (0.0005) [2023-12-27 12:37:06,229][20381] Updated weights for policy 1, policy_version 1004953 (0.0010) [2023-12-27 12:37:06,292][20381] Updated weights for policy 1, policy_version 1004963 (0.0010) [2023-12-27 12:37:06,320][20347] Updated weights for policy 0, policy_version 1005645 (0.0006) [2023-12-27 12:37:06,352][20381] Updated weights for policy 1, policy_version 1004973 (0.0011) [2023-12-27 12:37:06,381][20347] Updated weights for policy 0, policy_version 1005655 (0.0006) [2023-12-27 12:37:06,408][20381] Updated weights for policy 1, policy_version 1004983 (0.0011) [2023-12-27 12:37:06,436][20347] Updated weights for policy 0, policy_version 1005665 (0.0009) [2023-12-27 12:37:07,116][20347] Updated weights for policy 0, policy_version 1005675 (0.0007) [2023-12-27 12:37:07,137][20381] Updated weights for policy 1, policy_version 1004993 (0.0006) [2023-12-27 12:37:07,182][20381] Updated weights for policy 1, policy_version 1005003 (0.0005) [2023-12-27 12:37:07,183][20347] Updated weights for policy 0, policy_version 1005685 (0.0010) [2023-12-27 12:37:07,238][20381] Updated weights for policy 1, policy_version 1005013 (0.0006) [2023-12-27 12:37:07,243][20347] Updated weights for policy 0, policy_version 1005695 (0.0010) [2023-12-27 12:37:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 514818048. Throughput: 0: 10364.5, 1: 10296.4. Samples: 514781380. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:37:07,363][19170] Avg episode reward: [(0, '873.140'), (1, '725.290')] [2023-12-27 12:37:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001005016_257310720.pth... [2023-12-27 12:37:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001005704_257507328.pth... [2023-12-27 12:37:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001003832_257007616.pth [2023-12-27 12:37:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001004488_257196032.pth [2023-12-27 12:37:07,805][20381] Updated weights for policy 1, policy_version 1005023 (0.0007) [2023-12-27 12:37:07,863][20381] Updated weights for policy 1, policy_version 1005033 (0.0007) [2023-12-27 12:37:07,910][20347] Updated weights for policy 0, policy_version 1005705 (0.0009) [2023-12-27 12:37:07,920][20381] Updated weights for policy 1, policy_version 1005043 (0.0008) [2023-12-27 12:37:07,969][20347] Updated weights for policy 0, policy_version 1005715 (0.0010) [2023-12-27 12:37:08,021][20347] Updated weights for policy 0, policy_version 1005725 (0.0010) [2023-12-27 12:37:08,070][20347] Updated weights for policy 0, policy_version 1005735 (0.0010) [2023-12-27 12:37:08,642][20381] Updated weights for policy 1, policy_version 1005053 (0.0008) [2023-12-27 12:37:08,703][20381] Updated weights for policy 1, policy_version 1005063 (0.0008) [2023-12-27 12:37:08,761][20381] Updated weights for policy 1, policy_version 1005073 (0.0007) [2023-12-27 12:37:08,800][20347] Updated weights for policy 0, policy_version 1005745 (0.0011) [2023-12-27 12:37:08,852][20347] Updated weights for policy 0, policy_version 1005755 (0.0010) [2023-12-27 12:37:08,913][20347] Updated weights for policy 0, policy_version 1005765 (0.0010) [2023-12-27 12:37:09,455][20381] Updated weights for policy 1, policy_version 1005083 (0.0006) [2023-12-27 12:37:09,515][20381] Updated weights for policy 1, policy_version 1005093 (0.0008) [2023-12-27 12:37:09,571][20381] Updated weights for policy 1, policy_version 1005103 (0.0007) [2023-12-27 12:37:09,652][20347] Updated weights for policy 0, policy_version 1005775 (0.0010) [2023-12-27 12:37:09,717][20347] Updated weights for policy 0, policy_version 1005785 (0.0011) [2023-12-27 12:37:09,779][20347] Updated weights for policy 0, policy_version 1005795 (0.0010) [2023-12-27 12:37:10,300][20381] Updated weights for policy 1, policy_version 1005113 (0.0007) [2023-12-27 12:37:10,360][20381] Updated weights for policy 1, policy_version 1005123 (0.0008) [2023-12-27 12:37:10,413][20381] Updated weights for policy 1, policy_version 1005133 (0.0007) [2023-12-27 12:37:10,467][20381] Updated weights for policy 1, policy_version 1005143 (0.0007) [2023-12-27 12:37:10,508][20347] Updated weights for policy 0, policy_version 1005805 (0.0010) [2023-12-27 12:37:10,560][20347] Updated weights for policy 0, policy_version 1005815 (0.0010) [2023-12-27 12:37:10,608][20347] Updated weights for policy 0, policy_version 1005825 (0.0010) [2023-12-27 12:37:11,145][20381] Updated weights for policy 1, policy_version 1005153 (0.0007) [2023-12-27 12:37:11,206][20381] Updated weights for policy 1, policy_version 1005163 (0.0009) [2023-12-27 12:37:11,272][20347] Updated weights for policy 0, policy_version 1005835 (0.0010) [2023-12-27 12:37:11,272][20381] Updated weights for policy 1, policy_version 1005173 (0.0009) [2023-12-27 12:37:11,321][20347] Updated weights for policy 0, policy_version 1005845 (0.0008) [2023-12-27 12:37:11,378][20347] Updated weights for policy 0, policy_version 1005855 (0.0008) [2023-12-27 12:37:12,014][20381] Updated weights for policy 1, policy_version 1005183 (0.0009) [2023-12-27 12:37:12,066][20381] Updated weights for policy 1, policy_version 1005193 (0.0008) [2023-12-27 12:37:12,107][20347] Updated weights for policy 0, policy_version 1005865 (0.0008) [2023-12-27 12:37:12,120][20381] Updated weights for policy 1, policy_version 1005203 (0.0007) [2023-12-27 12:37:12,167][20347] Updated weights for policy 0, policy_version 1005875 (0.0010) [2023-12-27 12:37:12,230][20347] Updated weights for policy 0, policy_version 1005885 (0.0011) [2023-12-27 12:37:12,289][20347] Updated weights for policy 0, policy_version 1005895 (0.0010) [2023-12-27 12:37:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 514916352. Throughput: 0: 10289.3, 1: 10271.2. Samples: 514902976. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:37:12,363][19170] Avg episode reward: [(0, '893.490'), (1, '549.100')] [2023-12-27 12:37:12,843][20381] Updated weights for policy 1, policy_version 1005213 (0.0007) [2023-12-27 12:37:12,899][20381] Updated weights for policy 1, policy_version 1005223 (0.0008) [2023-12-27 12:37:12,950][20381] Updated weights for policy 1, policy_version 1005233 (0.0007) [2023-12-27 12:37:13,009][20347] Updated weights for policy 0, policy_version 1005905 (0.0006) [2023-12-27 12:37:13,065][20347] Updated weights for policy 0, policy_version 1005915 (0.0008) [2023-12-27 12:37:13,124][20347] Updated weights for policy 0, policy_version 1005925 (0.0008) [2023-12-27 12:37:13,609][20381] Updated weights for policy 1, policy_version 1005243 (0.0006) [2023-12-27 12:37:13,661][20381] Updated weights for policy 1, policy_version 1005253 (0.0008) [2023-12-27 12:37:13,707][20347] Updated weights for policy 0, policy_version 1005935 (0.0006) [2023-12-27 12:37:13,713][20381] Updated weights for policy 1, policy_version 1005263 (0.0008) [2023-12-27 12:37:13,761][20347] Updated weights for policy 0, policy_version 1005945 (0.0006) [2023-12-27 12:37:13,817][20347] Updated weights for policy 0, policy_version 1005955 (0.0009) [2023-12-27 12:37:14,425][20381] Updated weights for policy 1, policy_version 1005273 (0.0006) [2023-12-27 12:37:14,456][20347] Updated weights for policy 0, policy_version 1005965 (0.0008) [2023-12-27 12:37:14,477][20381] Updated weights for policy 1, policy_version 1005283 (0.0008) [2023-12-27 12:37:14,507][20347] Updated weights for policy 0, policy_version 1005975 (0.0005) [2023-12-27 12:37:14,522][20381] Updated weights for policy 1, policy_version 1005293 (0.0008) [2023-12-27 12:37:14,564][20347] Updated weights for policy 0, policy_version 1005985 (0.0005) [2023-12-27 12:37:14,576][20381] Updated weights for policy 1, policy_version 1005303 (0.0008) [2023-12-27 12:37:15,199][20347] Updated weights for policy 0, policy_version 1005995 (0.0006) [2023-12-27 12:37:15,253][20347] Updated weights for policy 0, policy_version 1006005 (0.0007) [2023-12-27 12:37:15,305][20347] Updated weights for policy 0, policy_version 1006015 (0.0007) [2023-12-27 12:37:15,326][20381] Updated weights for policy 1, policy_version 1005313 (0.0006) [2023-12-27 12:37:15,380][20381] Updated weights for policy 1, policy_version 1005323 (0.0006) [2023-12-27 12:37:15,435][20381] Updated weights for policy 1, policy_version 1005333 (0.0007) [2023-12-27 12:37:16,005][20347] Updated weights for policy 0, policy_version 1006025 (0.0005) [2023-12-27 12:37:16,060][20347] Updated weights for policy 0, policy_version 1006035 (0.0007) [2023-12-27 12:37:16,111][20347] Updated weights for policy 0, policy_version 1006045 (0.0007) [2023-12-27 12:37:16,144][20381] Updated weights for policy 1, policy_version 1005343 (0.0007) [2023-12-27 12:37:16,171][20347] Updated weights for policy 0, policy_version 1006055 (0.0006) [2023-12-27 12:37:16,200][20381] Updated weights for policy 1, policy_version 1005353 (0.0008) [2023-12-27 12:37:16,252][20381] Updated weights for policy 1, policy_version 1005363 (0.0008) [2023-12-27 12:37:16,814][20347] Updated weights for policy 0, policy_version 1006065 (0.0005) [2023-12-27 12:37:16,874][20347] Updated weights for policy 0, policy_version 1006075 (0.0005) [2023-12-27 12:37:16,929][20347] Updated weights for policy 0, policy_version 1006085 (0.0006) [2023-12-27 12:37:16,966][20381] Updated weights for policy 1, policy_version 1005373 (0.0008) [2023-12-27 12:37:17,015][20381] Updated weights for policy 1, policy_version 1005383 (0.0007) [2023-12-27 12:37:17,078][20381] Updated weights for policy 1, policy_version 1005393 (0.0009) [2023-12-27 12:37:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 515014656. Throughput: 0: 10284.8, 1: 10161.0. Samples: 515024508. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 12:37:17,363][19170] Avg episode reward: [(0, '893.720'), (1, '656.960')] [2023-12-27 12:37:17,483][20347] Updated weights for policy 0, policy_version 1006095 (0.0009) [2023-12-27 12:37:17,539][20347] Updated weights for policy 0, policy_version 1006105 (0.0010) [2023-12-27 12:37:17,594][20347] Updated weights for policy 0, policy_version 1006115 (0.0010) [2023-12-27 12:37:17,607][20291] KL-divergence is very high: 100.8151 [2023-12-27 12:37:17,831][20381] Updated weights for policy 1, policy_version 1005403 (0.0008) [2023-12-27 12:37:17,884][20381] Updated weights for policy 1, policy_version 1005413 (0.0009) [2023-12-27 12:37:17,934][20381] Updated weights for policy 1, policy_version 1005423 (0.0008) [2023-12-27 12:37:18,206][20347] Updated weights for policy 0, policy_version 1006125 (0.0008) [2023-12-27 12:37:18,268][20347] Updated weights for policy 0, policy_version 1006135 (0.0005) [2023-12-27 12:37:18,333][20347] Updated weights for policy 0, policy_version 1006145 (0.0006) [2023-12-27 12:37:18,631][20381] Updated weights for policy 1, policy_version 1005433 (0.0007) [2023-12-27 12:37:18,695][20381] Updated weights for policy 1, policy_version 1005443 (0.0008) [2023-12-27 12:37:18,760][20381] Updated weights for policy 1, policy_version 1005453 (0.0007) [2023-12-27 12:37:18,823][20381] Updated weights for policy 1, policy_version 1005463 (0.0008) [2023-12-27 12:37:18,978][20347] Updated weights for policy 0, policy_version 1006155 (0.0008) [2023-12-27 12:37:19,037][20347] Updated weights for policy 0, policy_version 1006165 (0.0008) [2023-12-27 12:37:19,086][20347] Updated weights for policy 0, policy_version 1006175 (0.0007) [2023-12-27 12:37:19,504][20381] Updated weights for policy 1, policy_version 1005473 (0.0007) [2023-12-27 12:37:19,562][20381] Updated weights for policy 1, policy_version 1005483 (0.0006) [2023-12-27 12:37:19,611][20381] Updated weights for policy 1, policy_version 1005493 (0.0010) [2023-12-27 12:37:19,758][20347] Updated weights for policy 0, policy_version 1006185 (0.0008) [2023-12-27 12:37:19,826][20347] Updated weights for policy 0, policy_version 1006195 (0.0009) [2023-12-27 12:37:19,884][20347] Updated weights for policy 0, policy_version 1006205 (0.0010) [2023-12-27 12:37:19,942][20347] Updated weights for policy 0, policy_version 1006215 (0.0009) [2023-12-27 12:37:20,349][20381] Updated weights for policy 1, policy_version 1005503 (0.0008) [2023-12-27 12:37:20,407][20381] Updated weights for policy 1, policy_version 1005513 (0.0008) [2023-12-27 12:37:20,458][20381] Updated weights for policy 1, policy_version 1005523 (0.0008) [2023-12-27 12:37:20,746][20347] Updated weights for policy 0, policy_version 1006225 (0.0008) [2023-12-27 12:37:20,809][20347] Updated weights for policy 0, policy_version 1006235 (0.0008) [2023-12-27 12:37:20,873][20347] Updated weights for policy 0, policy_version 1006245 (0.0009) [2023-12-27 12:37:21,183][20381] Updated weights for policy 1, policy_version 1005533 (0.0007) [2023-12-27 12:37:21,253][20381] Updated weights for policy 1, policy_version 1005543 (0.0007) [2023-12-27 12:37:21,318][20381] Updated weights for policy 1, policy_version 1005553 (0.0008) [2023-12-27 12:37:21,596][20347] Updated weights for policy 0, policy_version 1006255 (0.0008) [2023-12-27 12:37:21,663][20347] Updated weights for policy 0, policy_version 1006265 (0.0008) [2023-12-27 12:37:21,733][20347] Updated weights for policy 0, policy_version 1006275 (0.0007) [2023-12-27 12:37:22,055][20381] Updated weights for policy 1, policy_version 1005563 (0.0008) [2023-12-27 12:37:22,117][20381] Updated weights for policy 1, policy_version 1005573 (0.0006) [2023-12-27 12:37:22,178][20381] Updated weights for policy 1, policy_version 1005583 (0.0008) [2023-12-27 12:37:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 515112960. Throughput: 0: 10287.8, 1: 10074.0. Samples: 515085560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:22,363][19170] Avg episode reward: [(0, '873.440'), (1, '666.210')] [2023-12-27 12:37:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001005592_257458176.pth... [2023-12-27 12:37:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001006280_257654784.pth... [2023-12-27 12:37:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001004408_257155072.pth [2023-12-27 12:37:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001005096_257351680.pth [2023-12-27 12:37:22,430][20347] Updated weights for policy 0, policy_version 1006285 (0.0008) [2023-12-27 12:37:22,487][20347] Updated weights for policy 0, policy_version 1006295 (0.0007) [2023-12-27 12:37:22,548][20347] Updated weights for policy 0, policy_version 1006305 (0.0008) [2023-12-27 12:37:22,875][20381] Updated weights for policy 1, policy_version 1005593 (0.0010) [2023-12-27 12:37:22,936][20381] Updated weights for policy 1, policy_version 1005603 (0.0006) [2023-12-27 12:37:22,991][20381] Updated weights for policy 1, policy_version 1005613 (0.0006) [2023-12-27 12:37:23,047][20381] Updated weights for policy 1, policy_version 1005623 (0.0007) [2023-12-27 12:37:23,239][20347] Updated weights for policy 0, policy_version 1006315 (0.0007) [2023-12-27 12:37:23,289][20347] Updated weights for policy 0, policy_version 1006325 (0.0007) [2023-12-27 12:37:23,337][20347] Updated weights for policy 0, policy_version 1006335 (0.0008) [2023-12-27 12:37:23,659][20381] Updated weights for policy 1, policy_version 1005633 (0.0010) [2023-12-27 12:37:23,710][20381] Updated weights for policy 1, policy_version 1005643 (0.0010) [2023-12-27 12:37:23,741][20292] KL-divergence is very high: 250.4571 [2023-12-27 12:37:23,746][20292] KL-divergence is very high: 207.6808 [2023-12-27 12:37:23,763][20381] Updated weights for policy 1, policy_version 1005653 (0.0010) [2023-12-27 12:37:24,042][20347] Updated weights for policy 0, policy_version 1006345 (0.0007) [2023-12-27 12:37:24,098][20347] Updated weights for policy 0, policy_version 1006355 (0.0008) [2023-12-27 12:37:24,154][20347] Updated weights for policy 0, policy_version 1006365 (0.0009) [2023-12-27 12:37:24,199][20347] Updated weights for policy 0, policy_version 1006375 (0.0009) [2023-12-27 12:37:24,414][20381] Updated weights for policy 1, policy_version 1005663 (0.0009) [2023-12-27 12:37:24,465][20381] Updated weights for policy 1, policy_version 1005673 (0.0006) [2023-12-27 12:37:24,514][20381] Updated weights for policy 1, policy_version 1005683 (0.0005) [2023-12-27 12:37:24,945][20347] Updated weights for policy 0, policy_version 1006385 (0.0007) [2023-12-27 12:37:24,988][20347] Updated weights for policy 0, policy_version 1006395 (0.0006) [2023-12-27 12:37:25,038][20347] Updated weights for policy 0, policy_version 1006405 (0.0007) [2023-12-27 12:37:25,097][20381] Updated weights for policy 1, policy_version 1005693 (0.0007) [2023-12-27 12:37:25,156][20381] Updated weights for policy 1, policy_version 1005703 (0.0005) [2023-12-27 12:37:25,214][20381] Updated weights for policy 1, policy_version 1005713 (0.0005) [2023-12-27 12:37:25,748][20347] Updated weights for policy 0, policy_version 1006415 (0.0007) [2023-12-27 12:37:25,801][20347] Updated weights for policy 0, policy_version 1006425 (0.0007) [2023-12-27 12:37:25,811][20381] Updated weights for policy 1, policy_version 1005723 (0.0005) [2023-12-27 12:37:25,857][20347] Updated weights for policy 0, policy_version 1006435 (0.0006) [2023-12-27 12:37:25,859][20381] Updated weights for policy 1, policy_version 1005733 (0.0005) [2023-12-27 12:37:25,906][20381] Updated weights for policy 1, policy_version 1005743 (0.0006) [2023-12-27 12:37:26,492][20347] Updated weights for policy 0, policy_version 1006445 (0.0007) [2023-12-27 12:37:26,545][20347] Updated weights for policy 0, policy_version 1006455 (0.0008) [2023-12-27 12:37:26,592][20381] Updated weights for policy 1, policy_version 1005753 (0.0008) [2023-12-27 12:37:26,595][20347] Updated weights for policy 0, policy_version 1006465 (0.0007) [2023-12-27 12:37:26,651][20381] Updated weights for policy 1, policy_version 1005763 (0.0010) [2023-12-27 12:37:26,702][20381] Updated weights for policy 1, policy_version 1005773 (0.0010) [2023-12-27 12:37:26,749][20381] Updated weights for policy 1, policy_version 1005783 (0.0010) [2023-12-27 12:37:27,301][20347] Updated weights for policy 0, policy_version 1006475 (0.0007) [2023-12-27 12:37:27,348][20347] Updated weights for policy 0, policy_version 1006485 (0.0007) [2023-12-27 12:37:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 515211264. Throughput: 0: 10179.8, 1: 10030.6. Samples: 515205608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:27,363][19170] Avg episode reward: [(0, '931.850'), (1, '715.670')] [2023-12-27 12:37:27,395][20347] Updated weights for policy 0, policy_version 1006495 (0.0007) [2023-12-27 12:37:27,430][20381] Updated weights for policy 1, policy_version 1005793 (0.0010) [2023-12-27 12:37:27,486][20381] Updated weights for policy 1, policy_version 1005803 (0.0010) [2023-12-27 12:37:27,536][20381] Updated weights for policy 1, policy_version 1005813 (0.0010) [2023-12-27 12:37:28,097][20347] Updated weights for policy 0, policy_version 1006505 (0.0005) [2023-12-27 12:37:28,151][20347] Updated weights for policy 0, policy_version 1006515 (0.0007) [2023-12-27 12:37:28,203][20347] Updated weights for policy 0, policy_version 1006525 (0.0006) [2023-12-27 12:37:28,216][20381] Updated weights for policy 1, policy_version 1005823 (0.0010) [2023-12-27 12:37:28,260][20292] KL-divergence is very high: 127.9270 [2023-12-27 12:37:28,264][20347] Updated weights for policy 0, policy_version 1006535 (0.0005) [2023-12-27 12:37:28,266][20292] KL-divergence is very high: 119.7493 [2023-12-27 12:37:28,277][20381] Updated weights for policy 1, policy_version 1005833 (0.0010) [2023-12-27 12:37:28,339][20381] Updated weights for policy 1, policy_version 1005843 (0.0010) [2023-12-27 12:37:28,990][20347] Updated weights for policy 0, policy_version 1006545 (0.0008) [2023-12-27 12:37:29,014][20381] Updated weights for policy 1, policy_version 1005853 (0.0009) [2023-12-27 12:37:29,042][20347] Updated weights for policy 0, policy_version 1006555 (0.0007) [2023-12-27 12:37:29,069][20381] Updated weights for policy 1, policy_version 1005863 (0.0010) [2023-12-27 12:37:29,093][20347] Updated weights for policy 0, policy_version 1006565 (0.0005) [2023-12-27 12:37:29,117][20381] Updated weights for policy 1, policy_version 1005873 (0.0010) [2023-12-27 12:37:29,795][20347] Updated weights for policy 0, policy_version 1006575 (0.0006) [2023-12-27 12:37:29,856][20381] Updated weights for policy 1, policy_version 1005883 (0.0011) [2023-12-27 12:37:29,861][20347] Updated weights for policy 0, policy_version 1006585 (0.0007) [2023-12-27 12:37:29,908][20381] Updated weights for policy 1, policy_version 1005893 (0.0010) [2023-12-27 12:37:29,923][20347] Updated weights for policy 0, policy_version 1006595 (0.0006) [2023-12-27 12:37:29,974][20381] Updated weights for policy 1, policy_version 1005903 (0.0011) [2023-12-27 12:37:30,592][20347] Updated weights for policy 0, policy_version 1006605 (0.0009) [2023-12-27 12:37:30,651][20381] Updated weights for policy 1, policy_version 1005913 (0.0010) [2023-12-27 12:37:30,653][20347] Updated weights for policy 0, policy_version 1006615 (0.0006) [2023-12-27 12:37:30,699][20381] Updated weights for policy 1, policy_version 1005923 (0.0010) [2023-12-27 12:37:30,714][20347] Updated weights for policy 0, policy_version 1006625 (0.0011) [2023-12-27 12:37:30,748][20381] Updated weights for policy 1, policy_version 1005933 (0.0010) [2023-12-27 12:37:30,799][20381] Updated weights for policy 1, policy_version 1005943 (0.0010) [2023-12-27 12:37:31,280][20347] Updated weights for policy 0, policy_version 1006635 (0.0006) [2023-12-27 12:37:31,343][20347] Updated weights for policy 0, policy_version 1006645 (0.0007) [2023-12-27 12:37:31,403][20347] Updated weights for policy 0, policy_version 1006655 (0.0006) [2023-12-27 12:37:31,480][20381] Updated weights for policy 1, policy_version 1005953 (0.0011) [2023-12-27 12:37:31,541][20381] Updated weights for policy 1, policy_version 1005963 (0.0008) [2023-12-27 12:37:31,600][20381] Updated weights for policy 1, policy_version 1005973 (0.0008) [2023-12-27 12:37:32,016][20347] Updated weights for policy 0, policy_version 1006665 (0.0005) [2023-12-27 12:37:32,070][20347] Updated weights for policy 0, policy_version 1006675 (0.0005) [2023-12-27 12:37:32,123][20347] Updated weights for policy 0, policy_version 1006685 (0.0005) [2023-12-27 12:37:32,176][20347] Updated weights for policy 0, policy_version 1006695 (0.0006) [2023-12-27 12:37:32,271][20381] Updated weights for policy 1, policy_version 1005983 (0.0008) [2023-12-27 12:37:32,330][20381] Updated weights for policy 1, policy_version 1005993 (0.0007) [2023-12-27 12:37:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 515317760. Throughput: 0: 10189.7, 1: 10075.3. Samples: 515329244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:32,363][19170] Avg episode reward: [(0, '961.230'), (1, '666.560')] [2023-12-27 12:37:32,394][20381] Updated weights for policy 1, policy_version 1006003 (0.0009) [2023-12-27 12:37:32,810][20347] Updated weights for policy 0, policy_version 1006705 (0.0007) [2023-12-27 12:37:32,866][20347] Updated weights for policy 0, policy_version 1006715 (0.0008) [2023-12-27 12:37:32,919][20347] Updated weights for policy 0, policy_version 1006726 (0.0008) [2023-12-27 12:37:33,050][20381] Updated weights for policy 1, policy_version 1006013 (0.0008) [2023-12-27 12:37:33,103][20381] Updated weights for policy 1, policy_version 1006023 (0.0008) [2023-12-27 12:37:33,160][20381] Updated weights for policy 1, policy_version 1006033 (0.0006) [2023-12-27 12:37:33,646][20347] Updated weights for policy 0, policy_version 1006736 (0.0007) [2023-12-27 12:37:33,693][20347] Updated weights for policy 0, policy_version 1006746 (0.0007) [2023-12-27 12:37:33,703][20381] Updated weights for policy 1, policy_version 1006043 (0.0005) [2023-12-27 12:37:33,748][20347] Updated weights for policy 0, policy_version 1006756 (0.0006) [2023-12-27 12:37:33,754][20381] Updated weights for policy 1, policy_version 1006053 (0.0005) [2023-12-27 12:37:33,809][20381] Updated weights for policy 1, policy_version 1006063 (0.0007) [2023-12-27 12:37:34,394][20347] Updated weights for policy 0, policy_version 1006766 (0.0009) [2023-12-27 12:37:34,458][20347] Updated weights for policy 0, policy_version 1006776 (0.0010) [2023-12-27 12:37:34,491][20381] Updated weights for policy 1, policy_version 1006073 (0.0007) [2023-12-27 12:37:34,520][20347] Updated weights for policy 0, policy_version 1006786 (0.0011) [2023-12-27 12:37:34,553][20381] Updated weights for policy 1, policy_version 1006083 (0.0008) [2023-12-27 12:37:34,611][20381] Updated weights for policy 1, policy_version 1006093 (0.0006) [2023-12-27 12:37:34,668][20381] Updated weights for policy 1, policy_version 1006103 (0.0005) [2023-12-27 12:37:35,215][20347] Updated weights for policy 0, policy_version 1006796 (0.0010) [2023-12-27 12:37:35,276][20347] Updated weights for policy 0, policy_version 1006806 (0.0008) [2023-12-27 12:37:35,306][20381] Updated weights for policy 1, policy_version 1006113 (0.0005) [2023-12-27 12:37:35,333][20347] Updated weights for policy 0, policy_version 1006816 (0.0007) [2023-12-27 12:37:35,358][20381] Updated weights for policy 1, policy_version 1006123 (0.0005) [2023-12-27 12:37:35,406][20381] Updated weights for policy 1, policy_version 1006133 (0.0005) [2023-12-27 12:37:35,994][20347] Updated weights for policy 0, policy_version 1006826 (0.0008) [2023-12-27 12:37:36,001][20381] Updated weights for policy 1, policy_version 1006143 (0.0007) [2023-12-27 12:37:36,046][20347] Updated weights for policy 0, policy_version 1006836 (0.0010) [2023-12-27 12:37:36,048][20381] Updated weights for policy 1, policy_version 1006153 (0.0007) [2023-12-27 12:37:36,097][20347] Updated weights for policy 0, policy_version 1006846 (0.0010) [2023-12-27 12:37:36,099][20381] Updated weights for policy 1, policy_version 1006163 (0.0006) [2023-12-27 12:37:36,157][20347] Updated weights for policy 0, policy_version 1006856 (0.0010) [2023-12-27 12:37:36,747][20381] Updated weights for policy 1, policy_version 1006173 (0.0008) [2023-12-27 12:37:36,813][20381] Updated weights for policy 1, policy_version 1006183 (0.0008) [2023-12-27 12:37:36,870][20381] Updated weights for policy 1, policy_version 1006193 (0.0007) [2023-12-27 12:37:36,891][20347] Updated weights for policy 0, policy_version 1006866 (0.0010) [2023-12-27 12:37:36,952][20347] Updated weights for policy 0, policy_version 1006876 (0.0008) [2023-12-27 12:37:37,013][20347] Updated weights for policy 0, policy_version 1006886 (0.0006) [2023-12-27 12:37:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 515424256. Throughput: 0: 10178.2, 1: 10134.2. Samples: 515392280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:37,363][19170] Avg episode reward: [(0, '932.210'), (1, '676.650')] [2023-12-27 12:37:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001006888_257810432.pth... [2023-12-27 12:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001006200_257613824.pth... [2023-12-27 12:37:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001005704_257507328.pth [2023-12-27 12:37:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001005016_257310720.pth [2023-12-27 12:37:37,538][20381] Updated weights for policy 1, policy_version 1006203 (0.0006) [2023-12-27 12:37:37,589][20381] Updated weights for policy 1, policy_version 1006213 (0.0007) [2023-12-27 12:37:37,634][20381] Updated weights for policy 1, policy_version 1006223 (0.0008) [2023-12-27 12:37:37,689][20347] Updated weights for policy 0, policy_version 1006896 (0.0010) [2023-12-27 12:37:37,746][20347] Updated weights for policy 0, policy_version 1006906 (0.0009) [2023-12-27 12:37:37,802][20347] Updated weights for policy 0, policy_version 1006916 (0.0007) [2023-12-27 12:37:38,373][20381] Updated weights for policy 1, policy_version 1006233 (0.0008) [2023-12-27 12:37:38,417][20347] Updated weights for policy 0, policy_version 1006926 (0.0008) [2023-12-27 12:37:38,434][20381] Updated weights for policy 1, policy_version 1006243 (0.0008) [2023-12-27 12:37:38,470][20347] Updated weights for policy 0, policy_version 1006936 (0.0005) [2023-12-27 12:37:38,495][20381] Updated weights for policy 1, policy_version 1006253 (0.0006) [2023-12-27 12:37:38,527][20347] Updated weights for policy 0, policy_version 1006946 (0.0006) [2023-12-27 12:37:38,557][20381] Updated weights for policy 1, policy_version 1006263 (0.0008) [2023-12-27 12:37:39,197][20347] Updated weights for policy 0, policy_version 1006956 (0.0007) [2023-12-27 12:37:39,263][20347] Updated weights for policy 0, policy_version 1006966 (0.0008) [2023-12-27 12:37:39,295][20381] Updated weights for policy 1, policy_version 1006273 (0.0009) [2023-12-27 12:37:39,316][20347] Updated weights for policy 0, policy_version 1006976 (0.0007) [2023-12-27 12:37:39,356][20381] Updated weights for policy 1, policy_version 1006283 (0.0008) [2023-12-27 12:37:39,429][20381] Updated weights for policy 1, policy_version 1006293 (0.0009) [2023-12-27 12:37:40,083][20347] Updated weights for policy 0, policy_version 1006986 (0.0008) [2023-12-27 12:37:40,143][20347] Updated weights for policy 0, policy_version 1006996 (0.0006) [2023-12-27 12:37:40,153][20381] Updated weights for policy 1, policy_version 1006303 (0.0008) [2023-12-27 12:37:40,204][20347] Updated weights for policy 0, policy_version 1007006 (0.0007) [2023-12-27 12:37:40,211][20381] Updated weights for policy 1, policy_version 1006313 (0.0006) [2023-12-27 12:37:40,268][20347] Updated weights for policy 0, policy_version 1007016 (0.0007) [2023-12-27 12:37:40,271][20381] Updated weights for policy 1, policy_version 1006323 (0.0006) [2023-12-27 12:37:40,935][20381] Updated weights for policy 1, policy_version 1006333 (0.0010) [2023-12-27 12:37:40,982][20347] Updated weights for policy 0, policy_version 1007026 (0.0005) [2023-12-27 12:37:40,984][20381] Updated weights for policy 1, policy_version 1006343 (0.0010) [2023-12-27 12:37:41,038][20347] Updated weights for policy 0, policy_version 1007036 (0.0006) [2023-12-27 12:37:41,042][20381] Updated weights for policy 1, policy_version 1006353 (0.0010) [2023-12-27 12:37:41,095][20347] Updated weights for policy 0, policy_version 1007046 (0.0007) [2023-12-27 12:37:41,845][20381] Updated weights for policy 1, policy_version 1006363 (0.0008) [2023-12-27 12:37:41,902][20381] Updated weights for policy 1, policy_version 1006373 (0.0007) [2023-12-27 12:37:41,912][20347] Updated weights for policy 0, policy_version 1007056 (0.0006) [2023-12-27 12:37:41,956][20381] Updated weights for policy 1, policy_version 1006383 (0.0006) [2023-12-27 12:37:41,962][20347] Updated weights for policy 0, policy_version 1007066 (0.0005) [2023-12-27 12:37:42,018][20347] Updated weights for policy 0, policy_version 1007076 (0.0006) [2023-12-27 12:37:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20438.4). Total num frames: 515522560. Throughput: 0: 10161.2, 1: 10120.9. Samples: 515512832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:42,363][19170] Avg episode reward: [(0, '922.160'), (1, '745.120')] [2023-12-27 12:37:42,615][20381] Updated weights for policy 1, policy_version 1006393 (0.0006) [2023-12-27 12:37:42,672][20381] Updated weights for policy 1, policy_version 1006403 (0.0008) [2023-12-27 12:37:42,725][20381] Updated weights for policy 1, policy_version 1006413 (0.0009) [2023-12-27 12:37:42,788][20381] Updated weights for policy 1, policy_version 1006423 (0.0006) [2023-12-27 12:37:42,809][20347] Updated weights for policy 0, policy_version 1007086 (0.0007) [2023-12-27 12:37:42,860][20347] Updated weights for policy 0, policy_version 1007096 (0.0005) [2023-12-27 12:37:42,912][20347] Updated weights for policy 0, policy_version 1007106 (0.0005) [2023-12-27 12:37:43,442][20381] Updated weights for policy 1, policy_version 1006433 (0.0008) [2023-12-27 12:37:43,504][20381] Updated weights for policy 1, policy_version 1006443 (0.0009) [2023-12-27 12:37:43,541][20347] Updated weights for policy 0, policy_version 1007116 (0.0006) [2023-12-27 12:37:43,555][20291] KL-divergence is very high: 116.7926 [2023-12-27 12:37:43,556][20381] Updated weights for policy 1, policy_version 1006453 (0.0005) [2023-12-27 12:37:43,591][20347] Updated weights for policy 0, policy_version 1007126 (0.0007) [2023-12-27 12:37:43,643][20347] Updated weights for policy 0, policy_version 1007136 (0.0008) [2023-12-27 12:37:44,214][20381] Updated weights for policy 1, policy_version 1006463 (0.0009) [2023-12-27 12:37:44,272][20381] Updated weights for policy 1, policy_version 1006473 (0.0010) [2023-12-27 12:37:44,326][20347] Updated weights for policy 0, policy_version 1007146 (0.0007) [2023-12-27 12:37:44,329][20381] Updated weights for policy 1, policy_version 1006483 (0.0011) [2023-12-27 12:37:44,375][20347] Updated weights for policy 0, policy_version 1007156 (0.0007) [2023-12-27 12:37:44,428][20347] Updated weights for policy 0, policy_version 1007166 (0.0006) [2023-12-27 12:37:44,476][20347] Updated weights for policy 0, policy_version 1007176 (0.0007) [2023-12-27 12:37:44,997][20381] Updated weights for policy 1, policy_version 1006493 (0.0011) [2023-12-27 12:37:45,052][20381] Updated weights for policy 1, policy_version 1006503 (0.0006) [2023-12-27 12:37:45,118][20381] Updated weights for policy 1, policy_version 1006513 (0.0006) [2023-12-27 12:37:45,199][20347] Updated weights for policy 0, policy_version 1007186 (0.0007) [2023-12-27 12:37:45,260][20347] Updated weights for policy 0, policy_version 1007196 (0.0007) [2023-12-27 12:37:45,312][20347] Updated weights for policy 0, policy_version 1007206 (0.0007) [2023-12-27 12:37:45,784][20381] Updated weights for policy 1, policy_version 1006523 (0.0009) [2023-12-27 12:37:45,847][20381] Updated weights for policy 1, policy_version 1006534 (0.0006) [2023-12-27 12:37:45,901][20381] Updated weights for policy 1, policy_version 1006544 (0.0005) [2023-12-27 12:37:45,961][20347] Updated weights for policy 0, policy_version 1007216 (0.0007) [2023-12-27 12:37:46,025][20347] Updated weights for policy 0, policy_version 1007226 (0.0006) [2023-12-27 12:37:46,091][20347] Updated weights for policy 0, policy_version 1007236 (0.0009) [2023-12-27 12:37:46,565][20381] Updated weights for policy 1, policy_version 1006554 (0.0007) [2023-12-27 12:37:46,627][20381] Updated weights for policy 1, policy_version 1006564 (0.0008) [2023-12-27 12:37:46,648][20347] Updated weights for policy 0, policy_version 1007246 (0.0010) [2023-12-27 12:37:46,680][20381] Updated weights for policy 1, policy_version 1006574 (0.0005) [2023-12-27 12:37:46,700][20347] Updated weights for policy 0, policy_version 1007256 (0.0010) [2023-12-27 12:37:46,739][20381] Updated weights for policy 1, policy_version 1006584 (0.0005) [2023-12-27 12:37:46,752][20347] Updated weights for policy 0, policy_version 1007266 (0.0010) [2023-12-27 12:37:47,336][20381] Updated weights for policy 1, policy_version 1006594 (0.0006) [2023-12-27 12:37:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 515620864. Throughput: 0: 10185.1, 1: 10176.3. Samples: 515636092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:47,363][19170] Avg episode reward: [(0, '922.580'), (1, '686.310')] [2023-12-27 12:37:47,382][20292] KL-divergence is very high: 120.5802 [2023-12-27 12:37:47,393][20381] Updated weights for policy 1, policy_version 1006604 (0.0007) [2023-12-27 12:37:47,398][20292] KL-divergence is very high: 120.4097 [2023-12-27 12:37:47,413][20347] Updated weights for policy 0, policy_version 1007276 (0.0009) [2023-12-27 12:37:47,447][20381] Updated weights for policy 1, policy_version 1006614 (0.0006) [2023-12-27 12:37:47,466][20347] Updated weights for policy 0, policy_version 1007286 (0.0006) [2023-12-27 12:37:47,515][20347] Updated weights for policy 0, policy_version 1007296 (0.0005) [2023-12-27 12:37:48,105][20381] Updated weights for policy 1, policy_version 1006624 (0.0007) [2023-12-27 12:37:48,119][20347] Updated weights for policy 0, policy_version 1007306 (0.0005) [2023-12-27 12:37:48,158][20381] Updated weights for policy 1, policy_version 1006634 (0.0006) [2023-12-27 12:37:48,164][20347] Updated weights for policy 0, policy_version 1007316 (0.0005) [2023-12-27 12:37:48,213][20381] Updated weights for policy 1, policy_version 1006644 (0.0006) [2023-12-27 12:37:48,219][20347] Updated weights for policy 0, policy_version 1007326 (0.0006) [2023-12-27 12:37:48,274][20347] Updated weights for policy 0, policy_version 1007336 (0.0006) [2023-12-27 12:37:48,878][20381] Updated weights for policy 1, policy_version 1006654 (0.0006) [2023-12-27 12:37:48,930][20347] Updated weights for policy 0, policy_version 1007346 (0.0007) [2023-12-27 12:37:48,975][20381] Updated weights for policy 1, policy_version 1006664 (0.0007) [2023-12-27 12:37:48,981][20347] Updated weights for policy 0, policy_version 1007356 (0.0007) [2023-12-27 12:37:49,025][20347] Updated weights for policy 0, policy_version 1007366 (0.0007) [2023-12-27 12:37:49,038][20381] Updated weights for policy 1, policy_version 1006674 (0.0006) [2023-12-27 12:37:49,658][20381] Updated weights for policy 1, policy_version 1006684 (0.0007) [2023-12-27 12:37:49,711][20381] Updated weights for policy 1, policy_version 1006694 (0.0008) [2023-12-27 12:37:49,766][20381] Updated weights for policy 1, policy_version 1006704 (0.0008) [2023-12-27 12:37:49,852][20347] Updated weights for policy 0, policy_version 1007376 (0.0007) [2023-12-27 12:37:49,913][20347] Updated weights for policy 0, policy_version 1007386 (0.0007) [2023-12-27 12:37:49,981][20347] Updated weights for policy 0, policy_version 1007396 (0.0006) [2023-12-27 12:37:50,521][20381] Updated weights for policy 1, policy_version 1006714 (0.0007) [2023-12-27 12:37:50,582][20381] Updated weights for policy 1, policy_version 1006724 (0.0008) [2023-12-27 12:37:50,598][20347] Updated weights for policy 0, policy_version 1007406 (0.0007) [2023-12-27 12:37:50,644][20381] Updated weights for policy 1, policy_version 1006734 (0.0007) [2023-12-27 12:37:50,656][20347] Updated weights for policy 0, policy_version 1007416 (0.0006) [2023-12-27 12:37:50,698][20381] Updated weights for policy 1, policy_version 1006744 (0.0009) [2023-12-27 12:37:50,714][20347] Updated weights for policy 0, policy_version 1007426 (0.0005) [2023-12-27 12:37:51,430][20347] Updated weights for policy 0, policy_version 1007436 (0.0007) [2023-12-27 12:37:51,439][20381] Updated weights for policy 1, policy_version 1006754 (0.0008) [2023-12-27 12:37:51,492][20347] Updated weights for policy 0, policy_version 1007446 (0.0007) [2023-12-27 12:37:51,503][20381] Updated weights for policy 1, policy_version 1006764 (0.0008) [2023-12-27 12:37:51,551][20347] Updated weights for policy 0, policy_version 1007456 (0.0007) [2023-12-27 12:37:51,559][20381] Updated weights for policy 1, policy_version 1006774 (0.0008) [2023-12-27 12:37:52,302][20347] Updated weights for policy 0, policy_version 1007466 (0.0007) [2023-12-27 12:37:52,312][20381] Updated weights for policy 1, policy_version 1006784 (0.0008) [2023-12-27 12:37:52,358][20347] Updated weights for policy 0, policy_version 1007476 (0.0009) [2023-12-27 12:37:52,366][19170] Fps is (10 sec: 19653.3, 60 sec: 20205.7, 300 sec: 20438.1). Total num frames: 515719168. Throughput: 0: 10199.3, 1: 10156.7. Samples: 515697472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:52,367][19170] Avg episode reward: [(0, '942.250'), (1, '725.650')] [2023-12-27 12:37:52,374][20381] Updated weights for policy 1, policy_version 1006794 (0.0009) [2023-12-27 12:37:52,419][20347] Updated weights for policy 0, policy_version 1007486 (0.0007) [2023-12-27 12:37:52,428][20381] Updated weights for policy 1, policy_version 1006804 (0.0007) [2023-12-27 12:37:52,450][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001006808_257769472.pth... [2023-12-27 12:37:52,455][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001005592_257458176.pth [2023-12-27 12:37:52,471][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001007496_257966080.pth... [2023-12-27 12:37:52,473][20347] Updated weights for policy 0, policy_version 1007496 (0.0008) [2023-12-27 12:37:52,476][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001006280_257654784.pth [2023-12-27 12:37:53,055][20381] Updated weights for policy 1, policy_version 1006814 (0.0006) [2023-12-27 12:37:53,086][20347] Updated weights for policy 0, policy_version 1007506 (0.0006) [2023-12-27 12:37:53,116][20381] Updated weights for policy 1, policy_version 1006824 (0.0008) [2023-12-27 12:37:53,143][20347] Updated weights for policy 0, policy_version 1007516 (0.0007) [2023-12-27 12:37:53,174][20381] Updated weights for policy 1, policy_version 1006834 (0.0008) [2023-12-27 12:37:53,192][20347] Updated weights for policy 0, policy_version 1007526 (0.0005) [2023-12-27 12:37:53,843][20347] Updated weights for policy 0, policy_version 1007536 (0.0007) [2023-12-27 12:37:53,860][20381] Updated weights for policy 1, policy_version 1006844 (0.0006) [2023-12-27 12:37:53,891][20347] Updated weights for policy 0, policy_version 1007546 (0.0006) [2023-12-27 12:37:53,912][20381] Updated weights for policy 1, policy_version 1006854 (0.0006) [2023-12-27 12:37:53,936][20347] Updated weights for policy 0, policy_version 1007556 (0.0005) [2023-12-27 12:37:53,969][20381] Updated weights for policy 1, policy_version 1006864 (0.0007) [2023-12-27 12:37:54,631][20347] Updated weights for policy 0, policy_version 1007566 (0.0005) [2023-12-27 12:37:54,673][20381] Updated weights for policy 1, policy_version 1006874 (0.0007) [2023-12-27 12:37:54,682][20347] Updated weights for policy 0, policy_version 1007576 (0.0006) [2023-12-27 12:37:54,728][20381] Updated weights for policy 1, policy_version 1006884 (0.0005) [2023-12-27 12:37:54,737][20347] Updated weights for policy 0, policy_version 1007586 (0.0007) [2023-12-27 12:37:54,784][20381] Updated weights for policy 1, policy_version 1006894 (0.0006) [2023-12-27 12:37:54,840][20381] Updated weights for policy 1, policy_version 1006904 (0.0007) [2023-12-27 12:37:55,327][20347] Updated weights for policy 0, policy_version 1007596 (0.0006) [2023-12-27 12:37:55,367][20291] KL-divergence is very high: 120.8045 [2023-12-27 12:37:55,377][20347] Updated weights for policy 0, policy_version 1007606 (0.0007) [2023-12-27 12:37:55,431][20347] Updated weights for policy 0, policy_version 1007616 (0.0007) [2023-12-27 12:37:55,566][20381] Updated weights for policy 1, policy_version 1006914 (0.0008) [2023-12-27 12:37:55,624][20381] Updated weights for policy 1, policy_version 1006924 (0.0007) [2023-12-27 12:37:55,680][20381] Updated weights for policy 1, policy_version 1006934 (0.0005) [2023-12-27 12:37:56,106][20347] Updated weights for policy 0, policy_version 1007626 (0.0006) [2023-12-27 12:37:56,163][20347] Updated weights for policy 0, policy_version 1007636 (0.0010) [2023-12-27 12:37:56,223][20347] Updated weights for policy 0, policy_version 1007646 (0.0010) [2023-12-27 12:37:56,281][20347] Updated weights for policy 0, policy_version 1007656 (0.0010) [2023-12-27 12:37:56,307][20381] Updated weights for policy 1, policy_version 1006944 (0.0008) [2023-12-27 12:37:56,359][20381] Updated weights for policy 1, policy_version 1006954 (0.0007) [2023-12-27 12:37:56,415][20381] Updated weights for policy 1, policy_version 1006964 (0.0007) [2023-12-27 12:37:56,861][20347] Updated weights for policy 0, policy_version 1007666 (0.0010) [2023-12-27 12:37:56,916][20347] Updated weights for policy 0, policy_version 1007676 (0.0010) [2023-12-27 12:37:56,967][20347] Updated weights for policy 0, policy_version 1007686 (0.0010) [2023-12-27 12:37:57,120][20381] Updated weights for policy 1, policy_version 1006974 (0.0007) [2023-12-27 12:37:57,175][20381] Updated weights for policy 1, policy_version 1006984 (0.0007) [2023-12-27 12:37:57,223][20381] Updated weights for policy 1, policy_version 1006994 (0.0008) [2023-12-27 12:37:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 515833856. Throughput: 0: 10240.4, 1: 10133.0. Samples: 515819776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:37:57,363][19170] Avg episode reward: [(0, '970.720'), (1, '735.360')] [2023-12-27 12:37:57,669][20347] Updated weights for policy 0, policy_version 1007696 (0.0010) [2023-12-27 12:37:57,721][20347] Updated weights for policy 0, policy_version 1007706 (0.0010) [2023-12-27 12:37:57,775][20347] Updated weights for policy 0, policy_version 1007716 (0.0010) [2023-12-27 12:37:57,795][20381] Updated weights for policy 1, policy_version 1007004 (0.0006) [2023-12-27 12:37:57,842][20381] Updated weights for policy 1, policy_version 1007014 (0.0007) [2023-12-27 12:37:57,889][20381] Updated weights for policy 1, policy_version 1007024 (0.0007) [2023-12-27 12:37:58,519][20347] Updated weights for policy 0, policy_version 1007726 (0.0008) [2023-12-27 12:37:58,584][20347] Updated weights for policy 0, policy_version 1007736 (0.0007) [2023-12-27 12:37:58,644][20347] Updated weights for policy 0, policy_version 1007746 (0.0008) [2023-12-27 12:37:58,690][20381] Updated weights for policy 1, policy_version 1007034 (0.0007) [2023-12-27 12:37:58,754][20381] Updated weights for policy 1, policy_version 1007044 (0.0008) [2023-12-27 12:37:58,821][20381] Updated weights for policy 1, policy_version 1007054 (0.0008) [2023-12-27 12:37:58,885][20381] Updated weights for policy 1, policy_version 1007064 (0.0008) [2023-12-27 12:37:59,403][20347] Updated weights for policy 0, policy_version 1007756 (0.0007) [2023-12-27 12:37:59,462][20347] Updated weights for policy 0, policy_version 1007766 (0.0007) [2023-12-27 12:37:59,521][20347] Updated weights for policy 0, policy_version 1007776 (0.0007) [2023-12-27 12:37:59,617][20381] Updated weights for policy 1, policy_version 1007074 (0.0008) [2023-12-27 12:37:59,677][20381] Updated weights for policy 1, policy_version 1007084 (0.0008) [2023-12-27 12:37:59,735][20381] Updated weights for policy 1, policy_version 1007094 (0.0007) [2023-12-27 12:38:00,266][20347] Updated weights for policy 0, policy_version 1007786 (0.0006) [2023-12-27 12:38:00,327][20347] Updated weights for policy 0, policy_version 1007796 (0.0007) [2023-12-27 12:38:00,375][20381] Updated weights for policy 1, policy_version 1007104 (0.0006) [2023-12-27 12:38:00,378][20347] Updated weights for policy 0, policy_version 1007806 (0.0006) [2023-12-27 12:38:00,420][20381] Updated weights for policy 1, policy_version 1007114 (0.0006) [2023-12-27 12:38:00,433][20347] Updated weights for policy 0, policy_version 1007816 (0.0006) [2023-12-27 12:38:00,471][20381] Updated weights for policy 1, policy_version 1007124 (0.0007) [2023-12-27 12:38:00,481][20292] KL-divergence is very high: 239.0603 [2023-12-27 12:38:00,487][20292] KL-divergence is very high: 219.7843 [2023-12-27 12:38:01,101][20291] KL-divergence is very high: 131.9155 [2023-12-27 12:38:01,115][20347] Updated weights for policy 0, policy_version 1007826 (0.0006) [2023-12-27 12:38:01,127][20291] KL-divergence is very high: 227.4787 [2023-12-27 12:38:01,135][20291] KL-divergence is very high: 407.4471 [2023-12-27 12:38:01,181][20292] KL-divergence is very high: 223.5982 [2023-12-27 12:38:01,182][20347] Updated weights for policy 0, policy_version 1007836 (0.0006) [2023-12-27 12:38:01,194][20292] KL-divergence is very high: 141.7472 [2023-12-27 12:38:01,216][20381] Updated weights for policy 1, policy_version 1007134 (0.0007) [2023-12-27 12:38:01,216][20292] KL-divergence is very high: 123.7451 [2023-12-27 12:38:01,242][20347] Updated weights for policy 0, policy_version 1007846 (0.0006) [2023-12-27 12:38:01,273][20292] KL-divergence is very high: 148.9257 [2023-12-27 12:38:01,274][20381] Updated weights for policy 1, policy_version 1007144 (0.0008) [2023-12-27 12:38:01,279][20292] KL-divergence is very high: 116.8163 [2023-12-27 12:38:01,284][20292] KL-divergence is very high: 162.2273 [2023-12-27 12:38:01,326][20381] Updated weights for policy 1, policy_version 1007154 (0.0007) [2023-12-27 12:38:01,341][20292] KL-divergence is very high: 118.0652 [2023-12-27 12:38:01,865][20347] Updated weights for policy 0, policy_version 1007856 (0.0008) [2023-12-27 12:38:01,924][20347] Updated weights for policy 0, policy_version 1007866 (0.0007) [2023-12-27 12:38:01,988][20347] Updated weights for policy 0, policy_version 1007876 (0.0008) [2023-12-27 12:38:02,077][20381] Updated weights for policy 1, policy_version 1007164 (0.0007) [2023-12-27 12:38:02,094][20292] KL-divergence is very high: 107.2115 [2023-12-27 12:38:02,136][20381] Updated weights for policy 1, policy_version 1007174 (0.0008) [2023-12-27 12:38:02,191][20381] Updated weights for policy 1, policy_version 1007184 (0.0008) [2023-12-27 12:38:02,362][19170] Fps is (10 sec: 21307.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 515932160. Throughput: 0: 10176.7, 1: 10152.2. Samples: 515939308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:02,363][19170] Avg episode reward: [(0, '970.690'), (1, '676.600')] [2023-12-27 12:38:02,700][20347] Updated weights for policy 0, policy_version 1007886 (0.0008) [2023-12-27 12:38:02,758][20347] Updated weights for policy 0, policy_version 1007896 (0.0007) [2023-12-27 12:38:02,815][20347] Updated weights for policy 0, policy_version 1007906 (0.0007) [2023-12-27 12:38:02,906][20381] Updated weights for policy 1, policy_version 1007194 (0.0008) [2023-12-27 12:38:02,962][20381] Updated weights for policy 1, policy_version 1007204 (0.0008) [2023-12-27 12:38:03,006][20381] Updated weights for policy 1, policy_version 1007214 (0.0006) [2023-12-27 12:38:03,052][20381] Updated weights for policy 1, policy_version 1007224 (0.0005) [2023-12-27 12:38:03,459][20347] Updated weights for policy 0, policy_version 1007916 (0.0007) [2023-12-27 12:38:03,516][20347] Updated weights for policy 0, policy_version 1007926 (0.0008) [2023-12-27 12:38:03,563][20347] Updated weights for policy 0, policy_version 1007936 (0.0008) [2023-12-27 12:38:03,691][20381] Updated weights for policy 1, policy_version 1007234 (0.0008) [2023-12-27 12:38:03,743][20381] Updated weights for policy 1, policy_version 1007244 (0.0008) [2023-12-27 12:38:03,787][20381] Updated weights for policy 1, policy_version 1007254 (0.0007) [2023-12-27 12:38:04,250][20347] Updated weights for policy 0, policy_version 1007946 (0.0007) [2023-12-27 12:38:04,301][20347] Updated weights for policy 0, policy_version 1007956 (0.0008) [2023-12-27 12:38:04,354][20347] Updated weights for policy 0, policy_version 1007966 (0.0007) [2023-12-27 12:38:04,405][20347] Updated weights for policy 0, policy_version 1007976 (0.0007) [2023-12-27 12:38:04,493][20381] Updated weights for policy 1, policy_version 1007264 (0.0009) [2023-12-27 12:38:04,536][20381] Updated weights for policy 1, policy_version 1007274 (0.0008) [2023-12-27 12:38:04,585][20381] Updated weights for policy 1, policy_version 1007284 (0.0008) [2023-12-27 12:38:05,025][20347] Updated weights for policy 0, policy_version 1007986 (0.0008) [2023-12-27 12:38:05,070][20347] Updated weights for policy 0, policy_version 1007996 (0.0005) [2023-12-27 12:38:05,116][20347] Updated weights for policy 0, policy_version 1008006 (0.0006) [2023-12-27 12:38:05,190][20381] Updated weights for policy 1, policy_version 1007294 (0.0005) [2023-12-27 12:38:05,245][20381] Updated weights for policy 1, policy_version 1007304 (0.0007) [2023-12-27 12:38:05,300][20381] Updated weights for policy 1, policy_version 1007314 (0.0007) [2023-12-27 12:38:05,728][20347] Updated weights for policy 0, policy_version 1008016 (0.0009) [2023-12-27 12:38:05,775][20347] Updated weights for policy 0, policy_version 1008026 (0.0010) [2023-12-27 12:38:05,836][20347] Updated weights for policy 0, policy_version 1008036 (0.0010) [2023-12-27 12:38:05,853][20381] Updated weights for policy 1, policy_version 1007324 (0.0006) [2023-12-27 12:38:05,903][20381] Updated weights for policy 1, policy_version 1007334 (0.0007) [2023-12-27 12:38:05,949][20381] Updated weights for policy 1, policy_version 1007344 (0.0006) [2023-12-27 12:38:06,483][20347] Updated weights for policy 0, policy_version 1008046 (0.0007) [2023-12-27 12:38:06,538][20347] Updated weights for policy 0, policy_version 1008056 (0.0011) [2023-12-27 12:38:06,574][20381] Updated weights for policy 1, policy_version 1007354 (0.0007) [2023-12-27 12:38:06,601][20347] Updated weights for policy 0, policy_version 1008066 (0.0010) [2023-12-27 12:38:06,636][20381] Updated weights for policy 1, policy_version 1007364 (0.0005) [2023-12-27 12:38:06,707][20381] Updated weights for policy 1, policy_version 1007374 (0.0007) [2023-12-27 12:38:06,765][20381] Updated weights for policy 1, policy_version 1007384 (0.0007) [2023-12-27 12:38:07,291][20347] Updated weights for policy 0, policy_version 1008076 (0.0009) [2023-12-27 12:38:07,357][20347] Updated weights for policy 0, policy_version 1008086 (0.0008) [2023-12-27 12:38:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 20438.3). Total num frames: 516030464. Throughput: 0: 10176.4, 1: 10213.7. Samples: 516003112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:07,363][19170] Avg episode reward: [(0, '980.460'), (1, '745.250')] [2023-12-27 12:38:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001007384_257916928.pth... [2023-12-27 12:38:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001006200_257613824.pth [2023-12-27 12:38:07,410][20347] Updated weights for policy 0, policy_version 1008096 (0.0008) [2023-12-27 12:38:07,457][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001008104_258121728.pth... [2023-12-27 12:38:07,461][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001006888_257810432.pth [2023-12-27 12:38:07,481][20381] Updated weights for policy 1, policy_version 1007394 (0.0008) [2023-12-27 12:38:07,540][20381] Updated weights for policy 1, policy_version 1007404 (0.0008) [2023-12-27 12:38:07,600][20381] Updated weights for policy 1, policy_version 1007414 (0.0007) [2023-12-27 12:38:08,066][20347] Updated weights for policy 0, policy_version 1008106 (0.0006) [2023-12-27 12:38:08,130][20347] Updated weights for policy 0, policy_version 1008116 (0.0006) [2023-12-27 12:38:08,187][20347] Updated weights for policy 0, policy_version 1008126 (0.0005) [2023-12-27 12:38:08,238][20347] Updated weights for policy 0, policy_version 1008136 (0.0007) [2023-12-27 12:38:08,321][20381] Updated weights for policy 1, policy_version 1007424 (0.0007) [2023-12-27 12:38:08,336][20292] KL-divergence is very high: 832.3716 [2023-12-27 12:38:08,382][20381] Updated weights for policy 1, policy_version 1007434 (0.0006) [2023-12-27 12:38:08,383][20292] KL-divergence is very high: 388.5072 [2023-12-27 12:38:08,439][20381] Updated weights for policy 1, policy_version 1007444 (0.0005) [2023-12-27 12:38:08,925][20347] Updated weights for policy 0, policy_version 1008146 (0.0007) [2023-12-27 12:38:08,977][20347] Updated weights for policy 0, policy_version 1008156 (0.0007) [2023-12-27 12:38:09,034][20347] Updated weights for policy 0, policy_version 1008166 (0.0008) [2023-12-27 12:38:09,061][20381] Updated weights for policy 1, policy_version 1007454 (0.0006) [2023-12-27 12:38:09,117][20381] Updated weights for policy 1, policy_version 1007464 (0.0007) [2023-12-27 12:38:09,180][20381] Updated weights for policy 1, policy_version 1007474 (0.0009) [2023-12-27 12:38:09,794][20347] Updated weights for policy 0, policy_version 1008176 (0.0007) [2023-12-27 12:38:09,858][20347] Updated weights for policy 0, policy_version 1008186 (0.0008) [2023-12-27 12:38:09,920][20347] Updated weights for policy 0, policy_version 1008196 (0.0006) [2023-12-27 12:38:09,921][20381] Updated weights for policy 1, policy_version 1007484 (0.0008) [2023-12-27 12:38:09,993][20381] Updated weights for policy 1, policy_version 1007494 (0.0007) [2023-12-27 12:38:10,061][20381] Updated weights for policy 1, policy_version 1007504 (0.0006) [2023-12-27 12:38:10,626][20347] Updated weights for policy 0, policy_version 1008206 (0.0007) [2023-12-27 12:38:10,666][20381] Updated weights for policy 1, policy_version 1007514 (0.0006) [2023-12-27 12:38:10,677][20347] Updated weights for policy 0, policy_version 1008216 (0.0006) [2023-12-27 12:38:10,728][20381] Updated weights for policy 1, policy_version 1007524 (0.0006) [2023-12-27 12:38:10,741][20347] Updated weights for policy 0, policy_version 1008226 (0.0006) [2023-12-27 12:38:10,785][20381] Updated weights for policy 1, policy_version 1007534 (0.0006) [2023-12-27 12:38:10,834][20381] Updated weights for policy 1, policy_version 1007544 (0.0008) [2023-12-27 12:38:11,444][20347] Updated weights for policy 0, policy_version 1008236 (0.0006) [2023-12-27 12:38:11,502][20347] Updated weights for policy 0, policy_version 1008246 (0.0007) [2023-12-27 12:38:11,561][20347] Updated weights for policy 0, policy_version 1008256 (0.0007) [2023-12-27 12:38:11,591][20381] Updated weights for policy 1, policy_version 1007554 (0.0011) [2023-12-27 12:38:11,662][20381] Updated weights for policy 1, policy_version 1007564 (0.0009) [2023-12-27 12:38:11,726][20381] Updated weights for policy 1, policy_version 1007574 (0.0010) [2023-12-27 12:38:12,287][20347] Updated weights for policy 0, policy_version 1008266 (0.0009) [2023-12-27 12:38:12,343][20347] Updated weights for policy 0, policy_version 1008276 (0.0006) [2023-12-27 12:38:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 516128768. Throughput: 0: 10232.2, 1: 10234.2. Samples: 516126596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '784.370')] [2023-12-27 12:38:12,381][20381] Updated weights for policy 1, policy_version 1007584 (0.0007) [2023-12-27 12:38:12,407][20347] Updated weights for policy 0, policy_version 1008286 (0.0007) [2023-12-27 12:38:12,444][20381] Updated weights for policy 1, policy_version 1007594 (0.0007) [2023-12-27 12:38:12,462][20347] Updated weights for policy 0, policy_version 1008296 (0.0006) [2023-12-27 12:38:12,500][20381] Updated weights for policy 1, policy_version 1007604 (0.0006) [2023-12-27 12:38:13,120][20381] Updated weights for policy 1, policy_version 1007614 (0.0009) [2023-12-27 12:38:13,178][20381] Updated weights for policy 1, policy_version 1007624 (0.0011) [2023-12-27 12:38:13,237][20381] Updated weights for policy 1, policy_version 1007634 (0.0011) [2023-12-27 12:38:13,256][20347] Updated weights for policy 0, policy_version 1008306 (0.0006) [2023-12-27 12:38:13,306][20347] Updated weights for policy 0, policy_version 1008316 (0.0007) [2023-12-27 12:38:13,357][20347] Updated weights for policy 0, policy_version 1008326 (0.0007) [2023-12-27 12:38:13,920][20381] Updated weights for policy 1, policy_version 1007644 (0.0010) [2023-12-27 12:38:13,971][20381] Updated weights for policy 1, policy_version 1007654 (0.0010) [2023-12-27 12:38:14,026][20381] Updated weights for policy 1, policy_version 1007664 (0.0011) [2023-12-27 12:38:14,033][20347] Updated weights for policy 0, policy_version 1008336 (0.0007) [2023-12-27 12:38:14,081][20347] Updated weights for policy 0, policy_version 1008346 (0.0007) [2023-12-27 12:38:14,135][20347] Updated weights for policy 0, policy_version 1008356 (0.0006) [2023-12-27 12:38:14,741][20381] Updated weights for policy 1, policy_version 1007674 (0.0010) [2023-12-27 12:38:14,816][20381] Updated weights for policy 1, policy_version 1007684 (0.0010) [2023-12-27 12:38:14,821][20347] Updated weights for policy 0, policy_version 1008366 (0.0007) [2023-12-27 12:38:14,877][20381] Updated weights for policy 1, policy_version 1007694 (0.0011) [2023-12-27 12:38:14,887][20347] Updated weights for policy 0, policy_version 1008376 (0.0007) [2023-12-27 12:38:14,934][20381] Updated weights for policy 1, policy_version 1007704 (0.0011) [2023-12-27 12:38:14,948][20347] Updated weights for policy 0, policy_version 1008386 (0.0006) [2023-12-27 12:38:15,601][20381] Updated weights for policy 1, policy_version 1007714 (0.0010) [2023-12-27 12:38:15,655][20347] Updated weights for policy 0, policy_version 1008396 (0.0006) [2023-12-27 12:38:15,656][20381] Updated weights for policy 1, policy_version 1007724 (0.0010) [2023-12-27 12:38:15,703][20347] Updated weights for policy 0, policy_version 1008406 (0.0006) [2023-12-27 12:38:15,709][20381] Updated weights for policy 1, policy_version 1007734 (0.0011) [2023-12-27 12:38:15,752][20347] Updated weights for policy 0, policy_version 1008416 (0.0006) [2023-12-27 12:38:16,285][20381] Updated weights for policy 1, policy_version 1007744 (0.0006) [2023-12-27 12:38:16,335][20381] Updated weights for policy 1, policy_version 1007754 (0.0005) [2023-12-27 12:38:16,381][20381] Updated weights for policy 1, policy_version 1007764 (0.0005) [2023-12-27 12:38:16,450][20347] Updated weights for policy 0, policy_version 1008426 (0.0009) [2023-12-27 12:38:16,497][20347] Updated weights for policy 0, policy_version 1008436 (0.0008) [2023-12-27 12:38:16,542][20347] Updated weights for policy 0, policy_version 1008446 (0.0009) [2023-12-27 12:38:16,586][20347] Updated weights for policy 0, policy_version 1008456 (0.0008) [2023-12-27 12:38:16,945][20381] Updated weights for policy 1, policy_version 1007774 (0.0005) [2023-12-27 12:38:17,006][20381] Updated weights for policy 1, policy_version 1007784 (0.0005) [2023-12-27 12:38:17,065][20381] Updated weights for policy 1, policy_version 1007794 (0.0006) [2023-12-27 12:38:17,336][20347] Updated weights for policy 0, policy_version 1008466 (0.0008) [2023-12-27 12:38:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 516235264. Throughput: 0: 10140.2, 1: 10279.6. Samples: 516248140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:17,363][19170] Avg episode reward: [(0, '970.730'), (1, '794.240')] [2023-12-27 12:38:17,381][20347] Updated weights for policy 0, policy_version 1008476 (0.0008) [2023-12-27 12:38:17,434][20347] Updated weights for policy 0, policy_version 1008486 (0.0009) [2023-12-27 12:38:17,625][20381] Updated weights for policy 1, policy_version 1007804 (0.0006) [2023-12-27 12:38:17,678][20381] Updated weights for policy 1, policy_version 1007814 (0.0007) [2023-12-27 12:38:17,727][20381] Updated weights for policy 1, policy_version 1007824 (0.0008) [2023-12-27 12:38:18,150][20347] Updated weights for policy 0, policy_version 1008496 (0.0005) [2023-12-27 12:38:18,205][20347] Updated weights for policy 0, policy_version 1008506 (0.0007) [2023-12-27 12:38:18,266][20347] Updated weights for policy 0, policy_version 1008516 (0.0008) [2023-12-27 12:38:18,386][20381] Updated weights for policy 1, policy_version 1007834 (0.0008) [2023-12-27 12:38:18,443][20381] Updated weights for policy 1, policy_version 1007844 (0.0008) [2023-12-27 12:38:18,501][20381] Updated weights for policy 1, policy_version 1007854 (0.0008) [2023-12-27 12:38:18,552][20381] Updated weights for policy 1, policy_version 1007864 (0.0008) [2023-12-27 12:38:19,003][20347] Updated weights for policy 0, policy_version 1008526 (0.0008) [2023-12-27 12:38:19,061][20347] Updated weights for policy 0, policy_version 1008536 (0.0008) [2023-12-27 12:38:19,124][20347] Updated weights for policy 0, policy_version 1008546 (0.0005) [2023-12-27 12:38:19,191][20381] Updated weights for policy 1, policy_version 1007874 (0.0005) [2023-12-27 12:38:19,260][20381] Updated weights for policy 1, policy_version 1007884 (0.0008) [2023-12-27 12:38:19,318][20381] Updated weights for policy 1, policy_version 1007894 (0.0008) [2023-12-27 12:38:19,790][20347] Updated weights for policy 0, policy_version 1008556 (0.0007) [2023-12-27 12:38:19,856][20347] Updated weights for policy 0, policy_version 1008566 (0.0006) [2023-12-27 12:38:19,916][20347] Updated weights for policy 0, policy_version 1008576 (0.0006) [2023-12-27 12:38:20,133][20381] Updated weights for policy 1, policy_version 1007904 (0.0008) [2023-12-27 12:38:20,197][20381] Updated weights for policy 1, policy_version 1007914 (0.0008) [2023-12-27 12:38:20,263][20381] Updated weights for policy 1, policy_version 1007924 (0.0008) [2023-12-27 12:38:20,588][20347] Updated weights for policy 0, policy_version 1008586 (0.0009) [2023-12-27 12:38:20,648][20347] Updated weights for policy 0, policy_version 1008596 (0.0008) [2023-12-27 12:38:20,706][20347] Updated weights for policy 0, policy_version 1008606 (0.0008) [2023-12-27 12:38:20,762][20347] Updated weights for policy 0, policy_version 1008616 (0.0008) [2023-12-27 12:38:20,862][20381] Updated weights for policy 1, policy_version 1007934 (0.0006) [2023-12-27 12:38:20,925][20381] Updated weights for policy 1, policy_version 1007944 (0.0007) [2023-12-27 12:38:20,984][20381] Updated weights for policy 1, policy_version 1007954 (0.0007) [2023-12-27 12:38:21,499][20347] Updated weights for policy 0, policy_version 1008626 (0.0005) [2023-12-27 12:38:21,558][20347] Updated weights for policy 0, policy_version 1008636 (0.0005) [2023-12-27 12:38:21,623][20347] Updated weights for policy 0, policy_version 1008646 (0.0006) [2023-12-27 12:38:21,775][20381] Updated weights for policy 1, policy_version 1007964 (0.0008) [2023-12-27 12:38:21,841][20381] Updated weights for policy 1, policy_version 1007974 (0.0008) [2023-12-27 12:38:21,903][20381] Updated weights for policy 1, policy_version 1007984 (0.0007) [2023-12-27 12:38:22,352][20347] Updated weights for policy 0, policy_version 1008656 (0.0008) [2023-12-27 12:38:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 516333568. Throughput: 0: 10112.4, 1: 10245.2. Samples: 516308372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:22,363][19170] Avg episode reward: [(0, '970.640'), (1, '735.290')] [2023-12-27 12:38:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001007992_258072576.pth... [2023-12-27 12:38:22,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001006808_257769472.pth [2023-12-27 12:38:22,413][20347] Updated weights for policy 0, policy_version 1008666 (0.0008) [2023-12-27 12:38:22,476][20347] Updated weights for policy 0, policy_version 1008676 (0.0008) [2023-12-27 12:38:22,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001008680_258269184.pth... [2023-12-27 12:38:22,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001007496_257966080.pth [2023-12-27 12:38:22,589][20381] Updated weights for policy 1, policy_version 1007994 (0.0007) [2023-12-27 12:38:22,644][20381] Updated weights for policy 1, policy_version 1008004 (0.0008) [2023-12-27 12:38:22,702][20381] Updated weights for policy 1, policy_version 1008014 (0.0008) [2023-12-27 12:38:22,763][20381] Updated weights for policy 1, policy_version 1008024 (0.0006) [2023-12-27 12:38:23,230][20347] Updated weights for policy 0, policy_version 1008686 (0.0007) [2023-12-27 12:38:23,285][20347] Updated weights for policy 0, policy_version 1008696 (0.0005) [2023-12-27 12:38:23,336][20347] Updated weights for policy 0, policy_version 1008706 (0.0005) [2023-12-27 12:38:23,425][20381] Updated weights for policy 1, policy_version 1008034 (0.0009) [2023-12-27 12:38:23,483][20381] Updated weights for policy 1, policy_version 1008044 (0.0005) [2023-12-27 12:38:23,537][20381] Updated weights for policy 1, policy_version 1008054 (0.0006) [2023-12-27 12:38:24,064][20347] Updated weights for policy 0, policy_version 1008716 (0.0006) [2023-12-27 12:38:24,088][20381] Updated weights for policy 1, policy_version 1008064 (0.0006) [2023-12-27 12:38:24,119][20347] Updated weights for policy 0, policy_version 1008726 (0.0007) [2023-12-27 12:38:24,147][20381] Updated weights for policy 1, policy_version 1008074 (0.0010) [2023-12-27 12:38:24,171][20347] Updated weights for policy 0, policy_version 1008736 (0.0005) [2023-12-27 12:38:24,203][20381] Updated weights for policy 1, policy_version 1008084 (0.0009) [2023-12-27 12:38:24,854][20381] Updated weights for policy 1, policy_version 1008094 (0.0007) [2023-12-27 12:38:24,911][20347] Updated weights for policy 0, policy_version 1008746 (0.0006) [2023-12-27 12:38:24,912][20381] Updated weights for policy 1, policy_version 1008104 (0.0009) [2023-12-27 12:38:24,962][20347] Updated weights for policy 0, policy_version 1008756 (0.0006) [2023-12-27 12:38:24,967][20381] Updated weights for policy 1, policy_version 1008114 (0.0011) [2023-12-27 12:38:25,018][20347] Updated weights for policy 0, policy_version 1008766 (0.0010) [2023-12-27 12:38:25,072][20347] Updated weights for policy 0, policy_version 1008776 (0.0006) [2023-12-27 12:38:25,604][20381] Updated weights for policy 1, policy_version 1008124 (0.0010) [2023-12-27 12:38:25,624][20347] Updated weights for policy 0, policy_version 1008786 (0.0010) [2023-12-27 12:38:25,658][20381] Updated weights for policy 1, policy_version 1008134 (0.0010) [2023-12-27 12:38:25,676][20347] Updated weights for policy 0, policy_version 1008796 (0.0008) [2023-12-27 12:38:25,706][20381] Updated weights for policy 1, policy_version 1008144 (0.0010) [2023-12-27 12:38:25,722][20347] Updated weights for policy 0, policy_version 1008806 (0.0005) [2023-12-27 12:38:26,303][20347] Updated weights for policy 0, policy_version 1008816 (0.0006) [2023-12-27 12:38:26,354][20347] Updated weights for policy 0, policy_version 1008826 (0.0008) [2023-12-27 12:38:26,390][20381] Updated weights for policy 1, policy_version 1008154 (0.0010) [2023-12-27 12:38:26,408][20347] Updated weights for policy 0, policy_version 1008836 (0.0008) [2023-12-27 12:38:26,444][20381] Updated weights for policy 1, policy_version 1008164 (0.0009) [2023-12-27 12:38:26,488][20381] Updated weights for policy 1, policy_version 1008174 (0.0010) [2023-12-27 12:38:26,539][20381] Updated weights for policy 1, policy_version 1008184 (0.0010) [2023-12-27 12:38:27,087][20347] Updated weights for policy 0, policy_version 1008846 (0.0007) [2023-12-27 12:38:27,143][20347] Updated weights for policy 0, policy_version 1008856 (0.0007) [2023-12-27 12:38:27,196][20347] Updated weights for policy 0, policy_version 1008866 (0.0007) [2023-12-27 12:38:27,232][20381] Updated weights for policy 1, policy_version 1008194 (0.0011) [2023-12-27 12:38:27,280][20381] Updated weights for policy 1, policy_version 1008204 (0.0010) [2023-12-27 12:38:27,342][20381] Updated weights for policy 1, policy_version 1008214 (0.0008) [2023-12-27 12:38:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 516448256. Throughput: 0: 10123.1, 1: 10299.0. Samples: 516431828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:27,363][19170] Avg episode reward: [(0, '970.750'), (1, '705.900')] [2023-12-27 12:38:27,790][20347] Updated weights for policy 0, policy_version 1008876 (0.0007) [2023-12-27 12:38:27,835][20347] Updated weights for policy 0, policy_version 1008886 (0.0007) [2023-12-27 12:38:27,881][20347] Updated weights for policy 0, policy_version 1008896 (0.0007) [2023-12-27 12:38:27,929][20381] Updated weights for policy 1, policy_version 1008224 (0.0005) [2023-12-27 12:38:27,976][20381] Updated weights for policy 1, policy_version 1008234 (0.0005) [2023-12-27 12:38:28,024][20381] Updated weights for policy 1, policy_version 1008244 (0.0005) [2023-12-27 12:38:28,576][20381] Updated weights for policy 1, policy_version 1008254 (0.0008) [2023-12-27 12:38:28,614][20347] Updated weights for policy 0, policy_version 1008906 (0.0007) [2023-12-27 12:38:28,638][20381] Updated weights for policy 1, policy_version 1008264 (0.0011) [2023-12-27 12:38:28,672][20347] Updated weights for policy 0, policy_version 1008916 (0.0005) [2023-12-27 12:38:28,697][20381] Updated weights for policy 1, policy_version 1008274 (0.0011) [2023-12-27 12:38:28,731][20347] Updated weights for policy 0, policy_version 1008926 (0.0005) [2023-12-27 12:38:28,788][20347] Updated weights for policy 0, policy_version 1008936 (0.0008) [2023-12-27 12:38:29,359][20381] Updated weights for policy 1, policy_version 1008284 (0.0010) [2023-12-27 12:38:29,426][20381] Updated weights for policy 1, policy_version 1008294 (0.0010) [2023-12-27 12:38:29,485][20381] Updated weights for policy 1, policy_version 1008304 (0.0011) [2023-12-27 12:38:29,501][20347] Updated weights for policy 0, policy_version 1008946 (0.0005) [2023-12-27 12:38:29,553][20347] Updated weights for policy 0, policy_version 1008956 (0.0006) [2023-12-27 12:38:29,612][20347] Updated weights for policy 0, policy_version 1008966 (0.0005) [2023-12-27 12:38:30,138][20381] Updated weights for policy 1, policy_version 1008314 (0.0010) [2023-12-27 12:38:30,197][20381] Updated weights for policy 1, policy_version 1008324 (0.0009) [2023-12-27 12:38:30,256][20381] Updated weights for policy 1, policy_version 1008334 (0.0009) [2023-12-27 12:38:30,267][20347] Updated weights for policy 0, policy_version 1008976 (0.0005) [2023-12-27 12:38:30,313][20381] Updated weights for policy 1, policy_version 1008344 (0.0009) [2023-12-27 12:38:30,322][20347] Updated weights for policy 0, policy_version 1008986 (0.0008) [2023-12-27 12:38:30,383][20347] Updated weights for policy 0, policy_version 1008996 (0.0008) [2023-12-27 12:38:30,973][20381] Updated weights for policy 1, policy_version 1008354 (0.0008) [2023-12-27 12:38:31,031][20381] Updated weights for policy 1, policy_version 1008364 (0.0008) [2023-12-27 12:38:31,063][20347] Updated weights for policy 0, policy_version 1009006 (0.0008) [2023-12-27 12:38:31,085][20381] Updated weights for policy 1, policy_version 1008374 (0.0006) [2023-12-27 12:38:31,119][20347] Updated weights for policy 0, policy_version 1009016 (0.0007) [2023-12-27 12:38:31,181][20347] Updated weights for policy 0, policy_version 1009026 (0.0009) [2023-12-27 12:38:31,794][20381] Updated weights for policy 1, policy_version 1008384 (0.0007) [2023-12-27 12:38:31,845][20381] Updated weights for policy 1, policy_version 1008394 (0.0008) [2023-12-27 12:38:31,897][20347] Updated weights for policy 0, policy_version 1009036 (0.0007) [2023-12-27 12:38:31,899][20381] Updated weights for policy 1, policy_version 1008404 (0.0008) [2023-12-27 12:38:31,930][20291] KL-divergence is very high: 178.1135 [2023-12-27 12:38:31,953][20347] Updated weights for policy 0, policy_version 1009046 (0.0008) [2023-12-27 12:38:32,005][20347] Updated weights for policy 0, policy_version 1009056 (0.0008) [2023-12-27 12:38:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 516546560. Throughput: 0: 10126.3, 1: 10319.0. Samples: 516556128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:32,363][19170] Avg episode reward: [(0, '961.020'), (1, '764.680')] [2023-12-27 12:38:32,655][20381] Updated weights for policy 1, policy_version 1008414 (0.0007) [2023-12-27 12:38:32,706][20381] Updated weights for policy 1, policy_version 1008424 (0.0008) [2023-12-27 12:38:32,718][20347] Updated weights for policy 0, policy_version 1009066 (0.0010) [2023-12-27 12:38:32,764][20381] Updated weights for policy 1, policy_version 1008434 (0.0008) [2023-12-27 12:38:32,772][20347] Updated weights for policy 0, policy_version 1009076 (0.0007) [2023-12-27 12:38:32,823][20347] Updated weights for policy 0, policy_version 1009086 (0.0007) [2023-12-27 12:38:32,873][20347] Updated weights for policy 0, policy_version 1009096 (0.0007) [2023-12-27 12:38:33,457][20381] Updated weights for policy 1, policy_version 1008444 (0.0008) [2023-12-27 12:38:33,500][20381] Updated weights for policy 1, policy_version 1008454 (0.0007) [2023-12-27 12:38:33,545][20381] Updated weights for policy 1, policy_version 1008464 (0.0007) [2023-12-27 12:38:33,555][20347] Updated weights for policy 0, policy_version 1009106 (0.0010) [2023-12-27 12:38:33,569][20291] KL-divergence is very high: 103.1224 [2023-12-27 12:38:33,608][20347] Updated weights for policy 0, policy_version 1009116 (0.0010) [2023-12-27 12:38:33,662][20347] Updated weights for policy 0, policy_version 1009126 (0.0010) [2023-12-27 12:38:34,251][20381] Updated weights for policy 1, policy_version 1008474 (0.0008) [2023-12-27 12:38:34,315][20381] Updated weights for policy 1, policy_version 1008484 (0.0007) [2023-12-27 12:38:34,346][20347] Updated weights for policy 0, policy_version 1009136 (0.0009) [2023-12-27 12:38:34,373][20381] Updated weights for policy 1, policy_version 1008494 (0.0007) [2023-12-27 12:38:34,405][20347] Updated weights for policy 0, policy_version 1009146 (0.0009) [2023-12-27 12:38:34,435][20381] Updated weights for policy 1, policy_version 1008504 (0.0008) [2023-12-27 12:38:34,461][20347] Updated weights for policy 0, policy_version 1009156 (0.0010) [2023-12-27 12:38:35,034][20347] Updated weights for policy 0, policy_version 1009166 (0.0007) [2023-12-27 12:38:35,085][20347] Updated weights for policy 0, policy_version 1009176 (0.0010) [2023-12-27 12:38:35,140][20347] Updated weights for policy 0, policy_version 1009186 (0.0010) [2023-12-27 12:38:35,163][20381] Updated weights for policy 1, policy_version 1008514 (0.0009) [2023-12-27 12:38:35,206][20381] Updated weights for policy 1, policy_version 1008524 (0.0008) [2023-12-27 12:38:35,260][20381] Updated weights for policy 1, policy_version 1008534 (0.0009) [2023-12-27 12:38:35,791][20347] Updated weights for policy 0, policy_version 1009196 (0.0008) [2023-12-27 12:38:35,845][20347] Updated weights for policy 0, policy_version 1009206 (0.0008) [2023-12-27 12:38:35,899][20347] Updated weights for policy 0, policy_version 1009216 (0.0007) [2023-12-27 12:38:35,949][20381] Updated weights for policy 1, policy_version 1008544 (0.0007) [2023-12-27 12:38:36,010][20381] Updated weights for policy 1, policy_version 1008554 (0.0008) [2023-12-27 12:38:36,068][20381] Updated weights for policy 1, policy_version 1008564 (0.0008) [2023-12-27 12:38:36,603][20347] Updated weights for policy 0, policy_version 1009226 (0.0006) [2023-12-27 12:38:36,667][20347] Updated weights for policy 0, policy_version 1009236 (0.0006) [2023-12-27 12:38:36,732][20347] Updated weights for policy 0, policy_version 1009246 (0.0006) [2023-12-27 12:38:36,791][20381] Updated weights for policy 1, policy_version 1008574 (0.0008) [2023-12-27 12:38:36,792][20347] Updated weights for policy 0, policy_version 1009256 (0.0006) [2023-12-27 12:38:36,857][20381] Updated weights for policy 1, policy_version 1008584 (0.0008) [2023-12-27 12:38:36,921][20381] Updated weights for policy 1, policy_version 1008594 (0.0008) [2023-12-27 12:38:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 516644864. Throughput: 0: 10141.0, 1: 10295.2. Samples: 516617024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:37,363][19170] Avg episode reward: [(0, '941.420'), (1, '715.800')] [2023-12-27 12:38:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001008600_258228224.pth... [2023-12-27 12:38:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001007384_257916928.pth [2023-12-27 12:38:37,409][20347] Updated weights for policy 0, policy_version 1009266 (0.0010) [2023-12-27 12:38:37,470][20347] Updated weights for policy 0, policy_version 1009276 (0.0010) [2023-12-27 12:38:37,517][20381] Updated weights for policy 1, policy_version 1008604 (0.0008) [2023-12-27 12:38:37,534][20347] Updated weights for policy 0, policy_version 1009286 (0.0009) [2023-12-27 12:38:37,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001009288_258424832.pth... [2023-12-27 12:38:37,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001008104_258121728.pth [2023-12-27 12:38:37,576][20381] Updated weights for policy 1, policy_version 1008614 (0.0007) [2023-12-27 12:38:37,634][20381] Updated weights for policy 1, policy_version 1008624 (0.0008) [2023-12-27 12:38:38,159][20347] Updated weights for policy 0, policy_version 1009296 (0.0010) [2023-12-27 12:38:38,217][20347] Updated weights for policy 0, policy_version 1009306 (0.0008) [2023-12-27 12:38:38,273][20347] Updated weights for policy 0, policy_version 1009316 (0.0007) [2023-12-27 12:38:38,373][20381] Updated weights for policy 1, policy_version 1008634 (0.0008) [2023-12-27 12:38:38,428][20381] Updated weights for policy 1, policy_version 1008644 (0.0006) [2023-12-27 12:38:38,478][20381] Updated weights for policy 1, policy_version 1008654 (0.0008) [2023-12-27 12:38:38,541][20381] Updated weights for policy 1, policy_version 1008664 (0.0008) [2023-12-27 12:38:38,962][20347] Updated weights for policy 0, policy_version 1009326 (0.0006) [2023-12-27 12:38:38,991][20291] KL-divergence is very high: 184.5765 [2023-12-27 12:38:39,028][20347] Updated weights for policy 0, policy_version 1009336 (0.0005) [2023-12-27 12:38:39,042][20291] KL-divergence is very high: 139.5996 [2023-12-27 12:38:39,085][20347] Updated weights for policy 0, policy_version 1009346 (0.0005) [2023-12-27 12:38:39,190][20381] Updated weights for policy 1, policy_version 1008674 (0.0008) [2023-12-27 12:38:39,250][20381] Updated weights for policy 1, policy_version 1008684 (0.0008) [2023-12-27 12:38:39,306][20381] Updated weights for policy 1, policy_version 1008694 (0.0006) [2023-12-27 12:38:39,739][20347] Updated weights for policy 0, policy_version 1009356 (0.0005) [2023-12-27 12:38:39,808][20347] Updated weights for policy 0, policy_version 1009366 (0.0006) [2023-12-27 12:38:39,877][20347] Updated weights for policy 0, policy_version 1009376 (0.0008) [2023-12-27 12:38:40,053][20381] Updated weights for policy 1, policy_version 1008704 (0.0008) [2023-12-27 12:38:40,118][20381] Updated weights for policy 1, policy_version 1008714 (0.0009) [2023-12-27 12:38:40,185][20381] Updated weights for policy 1, policy_version 1008724 (0.0008) [2023-12-27 12:38:40,475][20347] Updated weights for policy 0, policy_version 1009386 (0.0008) [2023-12-27 12:38:40,533][20347] Updated weights for policy 0, policy_version 1009396 (0.0008) [2023-12-27 12:38:40,605][20347] Updated weights for policy 0, policy_version 1009406 (0.0006) [2023-12-27 12:38:40,658][20347] Updated weights for policy 0, policy_version 1009416 (0.0008) [2023-12-27 12:38:40,848][20381] Updated weights for policy 1, policy_version 1008734 (0.0009) [2023-12-27 12:38:40,901][20381] Updated weights for policy 1, policy_version 1008744 (0.0009) [2023-12-27 12:38:40,952][20381] Updated weights for policy 1, policy_version 1008754 (0.0007) [2023-12-27 12:38:41,286][20347] Updated weights for policy 0, policy_version 1009426 (0.0008) [2023-12-27 12:38:41,355][20347] Updated weights for policy 0, policy_version 1009436 (0.0009) [2023-12-27 12:38:41,428][20347] Updated weights for policy 0, policy_version 1009446 (0.0009) [2023-12-27 12:38:41,714][20381] Updated weights for policy 1, policy_version 1008764 (0.0008) [2023-12-27 12:38:41,782][20381] Updated weights for policy 1, policy_version 1008774 (0.0007) [2023-12-27 12:38:41,840][20381] Updated weights for policy 1, policy_version 1008784 (0.0006) [2023-12-27 12:38:42,185][20347] Updated weights for policy 0, policy_version 1009456 (0.0008) [2023-12-27 12:38:42,241][20347] Updated weights for policy 0, policy_version 1009466 (0.0007) [2023-12-27 12:38:42,272][20291] KL-divergence is very high: 133.8922 [2023-12-27 12:38:42,303][20347] Updated weights for policy 0, policy_version 1009476 (0.0007) [2023-12-27 12:38:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 516751360. Throughput: 0: 10157.4, 1: 10292.3. Samples: 516740016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:42,363][19170] Avg episode reward: [(0, '931.680'), (1, '696.160')] [2023-12-27 12:38:42,530][20381] Updated weights for policy 1, policy_version 1008794 (0.0006) [2023-12-27 12:38:42,596][20381] Updated weights for policy 1, policy_version 1008804 (0.0008) [2023-12-27 12:38:42,656][20381] Updated weights for policy 1, policy_version 1008814 (0.0008) [2023-12-27 12:38:42,718][20381] Updated weights for policy 1, policy_version 1008824 (0.0008) [2023-12-27 12:38:43,041][20347] Updated weights for policy 0, policy_version 1009486 (0.0010) [2023-12-27 12:38:43,096][20347] Updated weights for policy 0, policy_version 1009496 (0.0010) [2023-12-27 12:38:43,147][20347] Updated weights for policy 0, policy_version 1009506 (0.0010) [2023-12-27 12:38:43,380][20381] Updated weights for policy 1, policy_version 1008834 (0.0008) [2023-12-27 12:38:43,427][20381] Updated weights for policy 1, policy_version 1008844 (0.0007) [2023-12-27 12:38:43,470][20381] Updated weights for policy 1, policy_version 1008854 (0.0007) [2023-12-27 12:38:43,810][20347] Updated weights for policy 0, policy_version 1009516 (0.0008) [2023-12-27 12:38:43,877][20347] Updated weights for policy 0, policy_version 1009526 (0.0005) [2023-12-27 12:38:43,940][20347] Updated weights for policy 0, policy_version 1009536 (0.0006) [2023-12-27 12:38:44,080][20381] Updated weights for policy 1, policy_version 1008864 (0.0007) [2023-12-27 12:38:44,138][20381] Updated weights for policy 1, policy_version 1008874 (0.0008) [2023-12-27 12:38:44,189][20381] Updated weights for policy 1, policy_version 1008884 (0.0009) [2023-12-27 12:38:44,510][20347] Updated weights for policy 0, policy_version 1009546 (0.0007) [2023-12-27 12:38:44,544][20291] KL-divergence is very high: 277.0078 [2023-12-27 12:38:44,560][20291] KL-divergence is very high: 302.9719 [2023-12-27 12:38:44,566][20347] Updated weights for policy 0, policy_version 1009556 (0.0008) [2023-12-27 12:38:44,591][20291] KL-divergence is very high: 161.1165 [2023-12-27 12:38:44,608][20291] KL-divergence is very high: 135.1467 [2023-12-27 12:38:44,626][20347] Updated weights for policy 0, policy_version 1009566 (0.0008) [2023-12-27 12:38:44,681][20347] Updated weights for policy 0, policy_version 1009576 (0.0007) [2023-12-27 12:38:44,792][20381] Updated weights for policy 1, policy_version 1008894 (0.0008) [2023-12-27 12:38:44,843][20292] KL-divergence is very high: 103.3917 [2023-12-27 12:38:44,847][20381] Updated weights for policy 1, policy_version 1008904 (0.0008) [2023-12-27 12:38:44,906][20381] Updated weights for policy 1, policy_version 1008914 (0.0008) [2023-12-27 12:38:45,331][20347] Updated weights for policy 0, policy_version 1009586 (0.0006) [2023-12-27 12:38:45,381][20347] Updated weights for policy 0, policy_version 1009596 (0.0006) [2023-12-27 12:38:45,429][20347] Updated weights for policy 0, policy_version 1009606 (0.0006) [2023-12-27 12:38:45,652][20381] Updated weights for policy 1, policy_version 1008924 (0.0009) [2023-12-27 12:38:45,711][20381] Updated weights for policy 1, policy_version 1008934 (0.0011) [2023-12-27 12:38:45,766][20381] Updated weights for policy 1, policy_version 1008944 (0.0010) [2023-12-27 12:38:46,080][20347] Updated weights for policy 0, policy_version 1009616 (0.0007) [2023-12-27 12:38:46,129][20347] Updated weights for policy 0, policy_version 1009626 (0.0006) [2023-12-27 12:38:46,176][20347] Updated weights for policy 0, policy_version 1009636 (0.0007) [2023-12-27 12:38:46,455][20381] Updated weights for policy 1, policy_version 1008954 (0.0010) [2023-12-27 12:38:46,510][20381] Updated weights for policy 1, policy_version 1008964 (0.0011) [2023-12-27 12:38:46,558][20381] Updated weights for policy 1, policy_version 1008974 (0.0010) [2023-12-27 12:38:46,609][20381] Updated weights for policy 1, policy_version 1008984 (0.0010) [2023-12-27 12:38:46,819][20347] Updated weights for policy 0, policy_version 1009646 (0.0006) [2023-12-27 12:38:46,845][20291] KL-divergence is very high: 140.0545 [2023-12-27 12:38:46,870][20347] Updated weights for policy 0, policy_version 1009656 (0.0005) [2023-12-27 12:38:46,922][20347] Updated weights for policy 0, policy_version 1009666 (0.0005) [2023-12-27 12:38:47,206][20381] Updated weights for policy 1, policy_version 1008994 (0.0011) [2023-12-27 12:38:47,252][20381] Updated weights for policy 1, policy_version 1009004 (0.0010) [2023-12-27 12:38:47,300][20381] Updated weights for policy 1, policy_version 1009014 (0.0010) [2023-12-27 12:38:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 516857856. Throughput: 0: 10197.6, 1: 10362.1. Samples: 516864492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:47,363][19170] Avg episode reward: [(0, '844.410'), (1, '725.730')] [2023-12-27 12:38:47,580][20347] Updated weights for policy 0, policy_version 1009676 (0.0007) [2023-12-27 12:38:47,641][20347] Updated weights for policy 0, policy_version 1009686 (0.0007) [2023-12-27 12:38:47,700][20347] Updated weights for policy 0, policy_version 1009696 (0.0007) [2023-12-27 12:38:47,974][20381] Updated weights for policy 1, policy_version 1009024 (0.0010) [2023-12-27 12:38:48,029][20381] Updated weights for policy 1, policy_version 1009034 (0.0010) [2023-12-27 12:38:48,084][20381] Updated weights for policy 1, policy_version 1009044 (0.0010) [2023-12-27 12:38:48,351][20347] Updated weights for policy 0, policy_version 1009706 (0.0008) [2023-12-27 12:38:48,411][20347] Updated weights for policy 0, policy_version 1009716 (0.0007) [2023-12-27 12:38:48,475][20347] Updated weights for policy 0, policy_version 1009726 (0.0008) [2023-12-27 12:38:48,535][20347] Updated weights for policy 0, policy_version 1009736 (0.0007) [2023-12-27 12:38:48,786][20381] Updated weights for policy 1, policy_version 1009054 (0.0008) [2023-12-27 12:38:48,845][20381] Updated weights for policy 1, policy_version 1009064 (0.0007) [2023-12-27 12:38:48,905][20381] Updated weights for policy 1, policy_version 1009074 (0.0008) [2023-12-27 12:38:49,192][20347] Updated weights for policy 0, policy_version 1009746 (0.0007) [2023-12-27 12:38:49,258][20347] Updated weights for policy 0, policy_version 1009756 (0.0007) [2023-12-27 12:38:49,314][20347] Updated weights for policy 0, policy_version 1009766 (0.0006) [2023-12-27 12:38:49,601][20381] Updated weights for policy 1, policy_version 1009084 (0.0008) [2023-12-27 12:38:49,665][20381] Updated weights for policy 1, policy_version 1009094 (0.0007) [2023-12-27 12:38:49,728][20381] Updated weights for policy 1, policy_version 1009104 (0.0005) [2023-12-27 12:38:49,944][20347] Updated weights for policy 0, policy_version 1009776 (0.0008) [2023-12-27 12:38:50,011][20347] Updated weights for policy 0, policy_version 1009786 (0.0008) [2023-12-27 12:38:50,075][20347] Updated weights for policy 0, policy_version 1009796 (0.0008) [2023-12-27 12:38:50,408][20381] Updated weights for policy 1, policy_version 1009114 (0.0007) [2023-12-27 12:38:50,463][20381] Updated weights for policy 1, policy_version 1009124 (0.0008) [2023-12-27 12:38:50,510][20381] Updated weights for policy 1, policy_version 1009134 (0.0007) [2023-12-27 12:38:50,556][20381] Updated weights for policy 1, policy_version 1009144 (0.0008) [2023-12-27 12:38:50,780][20347] Updated weights for policy 0, policy_version 1009806 (0.0010) [2023-12-27 12:38:50,842][20347] Updated weights for policy 0, policy_version 1009816 (0.0010) [2023-12-27 12:38:50,900][20347] Updated weights for policy 0, policy_version 1009826 (0.0010) [2023-12-27 12:38:51,274][20381] Updated weights for policy 1, policy_version 1009154 (0.0007) [2023-12-27 12:38:51,336][20381] Updated weights for policy 1, policy_version 1009164 (0.0008) [2023-12-27 12:38:51,406][20381] Updated weights for policy 1, policy_version 1009174 (0.0008) [2023-12-27 12:38:51,603][20347] Updated weights for policy 0, policy_version 1009836 (0.0010) [2023-12-27 12:38:51,670][20347] Updated weights for policy 0, policy_version 1009846 (0.0009) [2023-12-27 12:38:51,751][20347] Updated weights for policy 0, policy_version 1009856 (0.0007) [2023-12-27 12:38:52,150][20381] Updated weights for policy 1, policy_version 1009184 (0.0008) [2023-12-27 12:38:52,203][20381] Updated weights for policy 1, policy_version 1009194 (0.0007) [2023-12-27 12:38:52,225][20292] KL-divergence is very high: 311.6382 [2023-12-27 12:38:52,264][20381] Updated weights for policy 1, policy_version 1009204 (0.0008) [2023-12-27 12:38:52,276][20292] KL-divergence is very high: 198.6912 [2023-12-27 12:38:52,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20617.8, 300 sec: 20493.9). Total num frames: 516956160. Throughput: 0: 10187.4, 1: 10324.7. Samples: 516926160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:52,363][19170] Avg episode reward: [(0, '863.710'), (1, '804.040')] [2023-12-27 12:38:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001009208_258383872.pth... [2023-12-27 12:38:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001009864_258572288.pth... [2023-12-27 12:38:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001007992_258072576.pth [2023-12-27 12:38:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001008680_258269184.pth [2023-12-27 12:38:52,407][20347] Updated weights for policy 0, policy_version 1009866 (0.0006) [2023-12-27 12:38:52,474][20347] Updated weights for policy 0, policy_version 1009876 (0.0009) [2023-12-27 12:38:52,537][20347] Updated weights for policy 0, policy_version 1009886 (0.0008) [2023-12-27 12:38:52,594][20347] Updated weights for policy 0, policy_version 1009896 (0.0008) [2023-12-27 12:38:53,023][20381] Updated weights for policy 1, policy_version 1009214 (0.0008) [2023-12-27 12:38:53,082][20381] Updated weights for policy 1, policy_version 1009224 (0.0008) [2023-12-27 12:38:53,143][20381] Updated weights for policy 1, policy_version 1009234 (0.0008) [2023-12-27 12:38:53,305][20347] Updated weights for policy 0, policy_version 1009906 (0.0008) [2023-12-27 12:38:53,364][20347] Updated weights for policy 0, policy_version 1009916 (0.0009) [2023-12-27 12:38:53,414][20347] Updated weights for policy 0, policy_version 1009926 (0.0007) [2023-12-27 12:38:53,723][20381] Updated weights for policy 1, policy_version 1009244 (0.0007) [2023-12-27 12:38:53,772][20381] Updated weights for policy 1, policy_version 1009254 (0.0005) [2023-12-27 12:38:53,830][20381] Updated weights for policy 1, policy_version 1009264 (0.0006) [2023-12-27 12:38:54,159][20347] Updated weights for policy 0, policy_version 1009936 (0.0007) [2023-12-27 12:38:54,223][20347] Updated weights for policy 0, policy_version 1009946 (0.0008) [2023-12-27 12:38:54,284][20347] Updated weights for policy 0, policy_version 1009956 (0.0008) [2023-12-27 12:38:54,446][20381] Updated weights for policy 1, policy_version 1009274 (0.0006) [2023-12-27 12:38:54,498][20381] Updated weights for policy 1, policy_version 1009284 (0.0008) [2023-12-27 12:38:54,559][20381] Updated weights for policy 1, policy_version 1009294 (0.0007) [2023-12-27 12:38:54,616][20381] Updated weights for policy 1, policy_version 1009304 (0.0007) [2023-12-27 12:38:54,899][20347] Updated weights for policy 0, policy_version 1009966 (0.0008) [2023-12-27 12:38:54,954][20347] Updated weights for policy 0, policy_version 1009976 (0.0007) [2023-12-27 12:38:55,007][20347] Updated weights for policy 0, policy_version 1009986 (0.0007) [2023-12-27 12:38:55,331][20381] Updated weights for policy 1, policy_version 1009314 (0.0008) [2023-12-27 12:38:55,378][20381] Updated weights for policy 1, policy_version 1009324 (0.0007) [2023-12-27 12:38:55,435][20381] Updated weights for policy 1, policy_version 1009334 (0.0009) [2023-12-27 12:38:55,591][20347] Updated weights for policy 0, policy_version 1009996 (0.0007) [2023-12-27 12:38:55,649][20347] Updated weights for policy 0, policy_version 1010006 (0.0005) [2023-12-27 12:38:55,704][20347] Updated weights for policy 0, policy_version 1010016 (0.0005) [2023-12-27 12:38:56,199][20381] Updated weights for policy 1, policy_version 1009344 (0.0008) [2023-12-27 12:38:56,223][20347] Updated weights for policy 0, policy_version 1010026 (0.0005) [2023-12-27 12:38:56,253][20381] Updated weights for policy 1, policy_version 1009354 (0.0006) [2023-12-27 12:38:56,275][20347] Updated weights for policy 0, policy_version 1010036 (0.0006) [2023-12-27 12:38:56,309][20381] Updated weights for policy 1, policy_version 1009364 (0.0008) [2023-12-27 12:38:56,313][20292] KL-divergence is very high: 120.3974 [2023-12-27 12:38:56,332][20347] Updated weights for policy 0, policy_version 1010046 (0.0008) [2023-12-27 12:38:56,387][20347] Updated weights for policy 0, policy_version 1010056 (0.0007) [2023-12-27 12:38:56,886][20381] Updated weights for policy 1, policy_version 1009374 (0.0007) [2023-12-27 12:38:56,938][20381] Updated weights for policy 1, policy_version 1009384 (0.0008) [2023-12-27 12:38:56,984][20381] Updated weights for policy 1, policy_version 1009394 (0.0006) [2023-12-27 12:38:57,072][20347] Updated weights for policy 0, policy_version 1010066 (0.0009) [2023-12-27 12:38:57,120][20347] Updated weights for policy 0, policy_version 1010076 (0.0007) [2023-12-27 12:38:57,168][20347] Updated weights for policy 0, policy_version 1010086 (0.0007) [2023-12-27 12:38:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 517062656. Throughput: 0: 10214.5, 1: 10264.9. Samples: 517048168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:38:57,363][19170] Avg episode reward: [(0, '893.090'), (1, '794.080')] [2023-12-27 12:38:57,577][20381] Updated weights for policy 1, policy_version 1009404 (0.0006) [2023-12-27 12:38:57,626][20381] Updated weights for policy 1, policy_version 1009414 (0.0005) [2023-12-27 12:38:57,694][20381] Updated weights for policy 1, policy_version 1009424 (0.0005) [2023-12-27 12:38:57,910][20347] Updated weights for policy 0, policy_version 1010096 (0.0006) [2023-12-27 12:38:57,959][20347] Updated weights for policy 0, policy_version 1010106 (0.0010) [2023-12-27 12:38:58,016][20347] Updated weights for policy 0, policy_version 1010116 (0.0010) [2023-12-27 12:38:58,278][20381] Updated weights for policy 1, policy_version 1009434 (0.0006) [2023-12-27 12:38:58,335][20381] Updated weights for policy 1, policy_version 1009444 (0.0007) [2023-12-27 12:38:58,410][20381] Updated weights for policy 1, policy_version 1009454 (0.0008) [2023-12-27 12:38:58,476][20381] Updated weights for policy 1, policy_version 1009464 (0.0007) [2023-12-27 12:38:58,843][20347] Updated weights for policy 0, policy_version 1010126 (0.0009) [2023-12-27 12:38:58,900][20347] Updated weights for policy 0, policy_version 1010136 (0.0006) [2023-12-27 12:38:58,913][20291] KL-divergence is very high: 260.3764 [2023-12-27 12:38:58,925][20291] KL-divergence is very high: 168.3295 [2023-12-27 12:38:58,964][20347] Updated weights for policy 0, policy_version 1010146 (0.0007) [2023-12-27 12:38:59,225][20381] Updated weights for policy 1, policy_version 1009474 (0.0008) [2023-12-27 12:38:59,290][20381] Updated weights for policy 1, policy_version 1009484 (0.0008) [2023-12-27 12:38:59,349][20381] Updated weights for policy 1, policy_version 1009494 (0.0008) [2023-12-27 12:38:59,627][20347] Updated weights for policy 0, policy_version 1010156 (0.0007) [2023-12-27 12:38:59,686][20291] KL-divergence is very high: 159.2020 [2023-12-27 12:38:59,690][20347] Updated weights for policy 0, policy_version 1010166 (0.0006) [2023-12-27 12:38:59,753][20347] Updated weights for policy 0, policy_version 1010176 (0.0008) [2023-12-27 12:39:00,025][20381] Updated weights for policy 1, policy_version 1009504 (0.0008) [2023-12-27 12:39:00,070][20381] Updated weights for policy 1, policy_version 1009514 (0.0006) [2023-12-27 12:39:00,117][20381] Updated weights for policy 1, policy_version 1009524 (0.0007) [2023-12-27 12:39:00,353][20347] Updated weights for policy 0, policy_version 1010186 (0.0006) [2023-12-27 12:39:00,410][20347] Updated weights for policy 0, policy_version 1010196 (0.0007) [2023-12-27 12:39:00,472][20347] Updated weights for policy 0, policy_version 1010206 (0.0008) [2023-12-27 12:39:00,527][20347] Updated weights for policy 0, policy_version 1010216 (0.0007) [2023-12-27 12:39:00,826][20381] Updated weights for policy 1, policy_version 1009534 (0.0007) [2023-12-27 12:39:00,858][20292] KL-divergence is very high: 269.1447 [2023-12-27 12:39:00,870][20292] KL-divergence is very high: 254.1552 [2023-12-27 12:39:00,887][20381] Updated weights for policy 1, policy_version 1009544 (0.0006) [2023-12-27 12:39:00,951][20381] Updated weights for policy 1, policy_version 1009554 (0.0005) [2023-12-27 12:39:01,177][20347] Updated weights for policy 0, policy_version 1010226 (0.0009) [2023-12-27 12:39:01,237][20347] Updated weights for policy 0, policy_version 1010236 (0.0007) [2023-12-27 12:39:01,297][20347] Updated weights for policy 0, policy_version 1010246 (0.0010) [2023-12-27 12:39:01,594][20381] Updated weights for policy 1, policy_version 1009564 (0.0006) [2023-12-27 12:39:01,663][20381] Updated weights for policy 1, policy_version 1009574 (0.0009) [2023-12-27 12:39:01,708][20292] KL-divergence is very high: 105.5631 [2023-12-27 12:39:01,725][20381] Updated weights for policy 1, policy_version 1009584 (0.0008) [2023-12-27 12:39:02,002][20347] Updated weights for policy 0, policy_version 1010256 (0.0006) [2023-12-27 12:39:02,052][20347] Updated weights for policy 0, policy_version 1010266 (0.0007) [2023-12-27 12:39:02,106][20347] Updated weights for policy 0, policy_version 1010276 (0.0006) [2023-12-27 12:39:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 517160960. Throughput: 0: 10259.0, 1: 10255.3. Samples: 517171280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:39:02,363][19170] Avg episode reward: [(0, '873.560'), (1, '617.550')] [2023-12-27 12:39:02,416][20381] Updated weights for policy 1, policy_version 1009594 (0.0008) [2023-12-27 12:39:02,466][20381] Updated weights for policy 1, policy_version 1009604 (0.0008) [2023-12-27 12:39:02,514][20381] Updated weights for policy 1, policy_version 1009614 (0.0008) [2023-12-27 12:39:02,573][20381] Updated weights for policy 1, policy_version 1009624 (0.0008) [2023-12-27 12:39:02,763][20347] Updated weights for policy 0, policy_version 1010286 (0.0006) [2023-12-27 12:39:02,811][20347] Updated weights for policy 0, policy_version 1010296 (0.0005) [2023-12-27 12:39:02,863][20347] Updated weights for policy 0, policy_version 1010306 (0.0005) [2023-12-27 12:39:02,866][20291] KL-divergence is very high: 119.3898 [2023-12-27 12:39:03,221][20381] Updated weights for policy 1, policy_version 1009634 (0.0005) [2023-12-27 12:39:03,282][20381] Updated weights for policy 1, policy_version 1009644 (0.0006) [2023-12-27 12:39:03,332][20381] Updated weights for policy 1, policy_version 1009654 (0.0005) [2023-12-27 12:39:03,531][20347] Updated weights for policy 0, policy_version 1010316 (0.0007) [2023-12-27 12:39:03,574][20347] Updated weights for policy 0, policy_version 1010326 (0.0006) [2023-12-27 12:39:03,624][20347] Updated weights for policy 0, policy_version 1010336 (0.0008) [2023-12-27 12:39:03,952][20381] Updated weights for policy 1, policy_version 1009664 (0.0008) [2023-12-27 12:39:04,002][20381] Updated weights for policy 1, policy_version 1009674 (0.0008) [2023-12-27 12:39:04,054][20381] Updated weights for policy 1, policy_version 1009684 (0.0007) [2023-12-27 12:39:04,325][20347] Updated weights for policy 0, policy_version 1010346 (0.0008) [2023-12-27 12:39:04,391][20347] Updated weights for policy 0, policy_version 1010356 (0.0011) [2023-12-27 12:39:04,450][20347] Updated weights for policy 0, policy_version 1010366 (0.0010) [2023-12-27 12:39:04,515][20347] Updated weights for policy 0, policy_version 1010376 (0.0011) [2023-12-27 12:39:04,788][20381] Updated weights for policy 1, policy_version 1009694 (0.0007) [2023-12-27 12:39:04,838][20381] Updated weights for policy 1, policy_version 1009704 (0.0007) [2023-12-27 12:39:04,899][20381] Updated weights for policy 1, policy_version 1009714 (0.0008) [2023-12-27 12:39:05,206][20347] Updated weights for policy 0, policy_version 1010387 (0.0010) [2023-12-27 12:39:05,257][20347] Updated weights for policy 0, policy_version 1010397 (0.0007) [2023-12-27 12:39:05,318][20347] Updated weights for policy 0, policy_version 1010407 (0.0007) [2023-12-27 12:39:05,584][20381] Updated weights for policy 1, policy_version 1009724 (0.0007) [2023-12-27 12:39:05,646][20381] Updated weights for policy 1, policy_version 1009734 (0.0007) [2023-12-27 12:39:05,697][20381] Updated weights for policy 1, policy_version 1009744 (0.0008) [2023-12-27 12:39:06,000][20347] Updated weights for policy 0, policy_version 1010417 (0.0010) [2023-12-27 12:39:06,061][20347] Updated weights for policy 0, policy_version 1010427 (0.0010) [2023-12-27 12:39:06,123][20347] Updated weights for policy 0, policy_version 1010437 (0.0010) [2023-12-27 12:39:06,395][20381] Updated weights for policy 1, policy_version 1009754 (0.0007) [2023-12-27 12:39:06,447][20381] Updated weights for policy 1, policy_version 1009764 (0.0008) [2023-12-27 12:39:06,499][20381] Updated weights for policy 1, policy_version 1009774 (0.0007) [2023-12-27 12:39:06,554][20381] Updated weights for policy 1, policy_version 1009784 (0.0006) [2023-12-27 12:39:06,804][20347] Updated weights for policy 0, policy_version 1010447 (0.0009) [2023-12-27 12:39:06,852][20347] Updated weights for policy 0, policy_version 1010457 (0.0007) [2023-12-27 12:39:06,905][20347] Updated weights for policy 0, policy_version 1010467 (0.0007) [2023-12-27 12:39:07,273][20381] Updated weights for policy 1, policy_version 1009794 (0.0008) [2023-12-27 12:39:07,322][20381] Updated weights for policy 1, policy_version 1009804 (0.0008) [2023-12-27 12:39:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 517259264. Throughput: 0: 10290.0, 1: 10261.2. Samples: 517233180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:39:07,364][19170] Avg episode reward: [(0, '882.920'), (1, '686.600')] [2023-12-27 12:39:07,368][20381] Updated weights for policy 1, policy_version 1009814 (0.0008) [2023-12-27 12:39:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001010472_258727936.pth... [2023-12-27 12:39:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001009288_258424832.pth [2023-12-27 12:39:07,378][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001009816_258539520.pth... [2023-12-27 12:39:07,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001008600_258228224.pth [2023-12-27 12:39:07,617][20347] Updated weights for policy 0, policy_version 1010477 (0.0007) [2023-12-27 12:39:07,676][20347] Updated weights for policy 0, policy_version 1010487 (0.0009) [2023-12-27 12:39:07,733][20347] Updated weights for policy 0, policy_version 1010497 (0.0008) [2023-12-27 12:39:08,091][20381] Updated weights for policy 1, policy_version 1009824 (0.0008) [2023-12-27 12:39:08,148][20381] Updated weights for policy 1, policy_version 1009834 (0.0008) [2023-12-27 12:39:08,200][20381] Updated weights for policy 1, policy_version 1009844 (0.0008) [2023-12-27 12:39:08,453][20347] Updated weights for policy 0, policy_version 1010507 (0.0007) [2023-12-27 12:39:08,512][20347] Updated weights for policy 0, policy_version 1010517 (0.0007) [2023-12-27 12:39:08,565][20347] Updated weights for policy 0, policy_version 1010527 (0.0007) [2023-12-27 12:39:08,904][20381] Updated weights for policy 1, policy_version 1009854 (0.0006) [2023-12-27 12:39:08,953][20381] Updated weights for policy 1, policy_version 1009864 (0.0005) [2023-12-27 12:39:09,002][20381] Updated weights for policy 1, policy_version 1009874 (0.0006) [2023-12-27 12:39:09,274][20347] Updated weights for policy 0, policy_version 1010537 (0.0007) [2023-12-27 12:39:09,330][20347] Updated weights for policy 0, policy_version 1010547 (0.0008) [2023-12-27 12:39:09,356][20291] KL-divergence is very high: 257.1161 [2023-12-27 12:39:09,378][20291] KL-divergence is very high: 252.9382 [2023-12-27 12:39:09,398][20347] Updated weights for policy 0, policy_version 1010557 (0.0008) [2023-12-27 12:39:09,411][20291] KL-divergence is very high: 165.2449 [2023-12-27 12:39:09,460][20347] Updated weights for policy 0, policy_version 1010567 (0.0007) [2023-12-27 12:39:09,666][20381] Updated weights for policy 1, policy_version 1009884 (0.0008) [2023-12-27 12:39:09,733][20381] Updated weights for policy 1, policy_version 1009894 (0.0007) [2023-12-27 12:39:09,798][20381] Updated weights for policy 1, policy_version 1009904 (0.0008) [2023-12-27 12:39:10,224][20347] Updated weights for policy 0, policy_version 1010577 (0.0007) [2023-12-27 12:39:10,286][20347] Updated weights for policy 0, policy_version 1010587 (0.0008) [2023-12-27 12:39:10,349][20347] Updated weights for policy 0, policy_version 1010597 (0.0007) [2023-12-27 12:39:10,437][20381] Updated weights for policy 1, policy_version 1009914 (0.0009) [2023-12-27 12:39:10,490][20381] Updated weights for policy 1, policy_version 1009924 (0.0008) [2023-12-27 12:39:10,541][20381] Updated weights for policy 1, policy_version 1009934 (0.0009) [2023-12-27 12:39:10,586][20381] Updated weights for policy 1, policy_version 1009944 (0.0008) [2023-12-27 12:39:10,913][20347] Updated weights for policy 0, policy_version 1010607 (0.0006) [2023-12-27 12:39:10,966][20347] Updated weights for policy 0, policy_version 1010617 (0.0008) [2023-12-27 12:39:11,030][20347] Updated weights for policy 0, policy_version 1010627 (0.0008) [2023-12-27 12:39:11,401][20381] Updated weights for policy 1, policy_version 1009954 (0.0008) [2023-12-27 12:39:11,459][20381] Updated weights for policy 1, policy_version 1009964 (0.0006) [2023-12-27 12:39:11,517][20381] Updated weights for policy 1, policy_version 1009974 (0.0006) [2023-12-27 12:39:11,775][20347] Updated weights for policy 0, policy_version 1010637 (0.0008) [2023-12-27 12:39:11,837][20347] Updated weights for policy 0, policy_version 1010647 (0.0007) [2023-12-27 12:39:11,884][20347] Updated weights for policy 0, policy_version 1010657 (0.0008) [2023-12-27 12:39:12,208][20381] Updated weights for policy 1, policy_version 1009984 (0.0008) [2023-12-27 12:39:12,269][20381] Updated weights for policy 1, policy_version 1009994 (0.0008) [2023-12-27 12:39:12,336][20381] Updated weights for policy 1, policy_version 1010004 (0.0008) [2023-12-27 12:39:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 517357568. Throughput: 0: 10281.9, 1: 10189.4. Samples: 517353036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:39:12,363][19170] Avg episode reward: [(0, '815.270'), (1, '627.330')] [2023-12-27 12:39:12,597][20347] Updated weights for policy 0, policy_version 1010667 (0.0008) [2023-12-27 12:39:12,649][20347] Updated weights for policy 0, policy_version 1010677 (0.0007) [2023-12-27 12:39:12,707][20347] Updated weights for policy 0, policy_version 1010687 (0.0007) [2023-12-27 12:39:13,058][20381] Updated weights for policy 1, policy_version 1010014 (0.0010) [2023-12-27 12:39:13,102][20292] KL-divergence is very high: 145.4649 [2023-12-27 12:39:13,108][20381] Updated weights for policy 1, policy_version 1010024 (0.0011) [2023-12-27 12:39:13,144][20292] KL-divergence is very high: 107.3502 [2023-12-27 12:39:13,160][20381] Updated weights for policy 1, policy_version 1010034 (0.0010) [2023-12-27 12:39:13,188][20292] KL-divergence is very high: 167.3326 [2023-12-27 12:39:13,411][20347] Updated weights for policy 0, policy_version 1010697 (0.0008) [2023-12-27 12:39:13,457][20347] Updated weights for policy 0, policy_version 1010707 (0.0007) [2023-12-27 12:39:13,507][20347] Updated weights for policy 0, policy_version 1010717 (0.0007) [2023-12-27 12:39:13,556][20347] Updated weights for policy 0, policy_version 1010727 (0.0007) [2023-12-27 12:39:13,866][20381] Updated weights for policy 1, policy_version 1010044 (0.0011) [2023-12-27 12:39:13,915][20381] Updated weights for policy 1, policy_version 1010054 (0.0011) [2023-12-27 12:39:13,960][20381] Updated weights for policy 1, policy_version 1010064 (0.0010) [2023-12-27 12:39:14,265][20347] Updated weights for policy 0, policy_version 1010737 (0.0007) [2023-12-27 12:39:14,321][20347] Updated weights for policy 0, policy_version 1010747 (0.0007) [2023-12-27 12:39:14,373][20347] Updated weights for policy 0, policy_version 1010757 (0.0007) [2023-12-27 12:39:14,659][20381] Updated weights for policy 1, policy_version 1010074 (0.0010) [2023-12-27 12:39:14,714][20381] Updated weights for policy 1, policy_version 1010084 (0.0010) [2023-12-27 12:39:14,765][20381] Updated weights for policy 1, policy_version 1010094 (0.0011) [2023-12-27 12:39:14,822][20381] Updated weights for policy 1, policy_version 1010104 (0.0011) [2023-12-27 12:39:15,091][20347] Updated weights for policy 0, policy_version 1010767 (0.0006) [2023-12-27 12:39:15,150][20347] Updated weights for policy 0, policy_version 1010777 (0.0007) [2023-12-27 12:39:15,208][20347] Updated weights for policy 0, policy_version 1010787 (0.0007) [2023-12-27 12:39:15,567][20381] Updated weights for policy 1, policy_version 1010114 (0.0011) [2023-12-27 12:39:15,626][20381] Updated weights for policy 1, policy_version 1010124 (0.0011) [2023-12-27 12:39:15,684][20381] Updated weights for policy 1, policy_version 1010134 (0.0010) [2023-12-27 12:39:15,806][20347] Updated weights for policy 0, policy_version 1010797 (0.0006) [2023-12-27 12:39:15,862][20347] Updated weights for policy 0, policy_version 1010807 (0.0005) [2023-12-27 12:39:15,912][20347] Updated weights for policy 0, policy_version 1010817 (0.0008) [2023-12-27 12:39:16,326][20381] Updated weights for policy 1, policy_version 1010144 (0.0009) [2023-12-27 12:39:16,375][20381] Updated weights for policy 1, policy_version 1010154 (0.0007) [2023-12-27 12:39:16,434][20381] Updated weights for policy 1, policy_version 1010164 (0.0006) [2023-12-27 12:39:16,446][20347] Updated weights for policy 0, policy_version 1010827 (0.0009) [2023-12-27 12:39:16,499][20347] Updated weights for policy 0, policy_version 1010837 (0.0009) [2023-12-27 12:39:16,560][20347] Updated weights for policy 0, policy_version 1010847 (0.0010) [2023-12-27 12:39:17,090][20381] Updated weights for policy 1, policy_version 1010174 (0.0008) [2023-12-27 12:39:17,144][20381] Updated weights for policy 1, policy_version 1010184 (0.0009) [2023-12-27 12:39:17,191][20347] Updated weights for policy 0, policy_version 1010857 (0.0010) [2023-12-27 12:39:17,199][20381] Updated weights for policy 1, policy_version 1010194 (0.0009) [2023-12-27 12:39:17,252][20347] Updated weights for policy 0, policy_version 1010867 (0.0005) [2023-12-27 12:39:17,303][20347] Updated weights for policy 0, policy_version 1010877 (0.0007) [2023-12-27 12:39:17,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 517464064. Throughput: 0: 10297.7, 1: 10106.3. Samples: 517474308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:39:17,363][20347] Updated weights for policy 0, policy_version 1010887 (0.0008) [2023-12-27 12:39:17,363][19170] Avg episode reward: [(0, '786.490'), (1, '607.420')] [2023-12-27 12:39:17,819][20381] Updated weights for policy 1, policy_version 1010204 (0.0007) [2023-12-27 12:39:17,879][20381] Updated weights for policy 1, policy_version 1010214 (0.0007) [2023-12-27 12:39:17,934][20381] Updated weights for policy 1, policy_version 1010224 (0.0005) [2023-12-27 12:39:17,964][20347] Updated weights for policy 0, policy_version 1010897 (0.0007) [2023-12-27 12:39:18,010][20347] Updated weights for policy 0, policy_version 1010907 (0.0008) [2023-12-27 12:39:18,067][20347] Updated weights for policy 0, policy_version 1010917 (0.0008) [2023-12-27 12:39:18,602][20381] Updated weights for policy 1, policy_version 1010234 (0.0006) [2023-12-27 12:39:18,662][20381] Updated weights for policy 1, policy_version 1010244 (0.0007) [2023-12-27 12:39:18,713][20381] Updated weights for policy 1, policy_version 1010254 (0.0007) [2023-12-27 12:39:18,763][20347] Updated weights for policy 0, policy_version 1010927 (0.0006) [2023-12-27 12:39:18,771][20381] Updated weights for policy 1, policy_version 1010264 (0.0006) [2023-12-27 12:39:18,824][20347] Updated weights for policy 0, policy_version 1010937 (0.0008) [2023-12-27 12:39:18,882][20347] Updated weights for policy 0, policy_version 1010947 (0.0008) [2023-12-27 12:39:19,504][20381] Updated weights for policy 1, policy_version 1010274 (0.0008) [2023-12-27 12:39:19,545][20347] Updated weights for policy 0, policy_version 1010957 (0.0007) [2023-12-27 12:39:19,570][20381] Updated weights for policy 1, policy_version 1010284 (0.0006) [2023-12-27 12:39:19,592][20347] Updated weights for policy 0, policy_version 1010967 (0.0006) [2023-12-27 12:39:19,625][20381] Updated weights for policy 1, policy_version 1010294 (0.0006) [2023-12-27 12:39:19,648][20347] Updated weights for policy 0, policy_version 1010977 (0.0006) [2023-12-27 12:39:20,324][20347] Updated weights for policy 0, policy_version 1010987 (0.0006) [2023-12-27 12:39:20,380][20347] Updated weights for policy 0, policy_version 1010997 (0.0006) [2023-12-27 12:39:20,408][20381] Updated weights for policy 1, policy_version 1010304 (0.0007) [2023-12-27 12:39:20,441][20347] Updated weights for policy 0, policy_version 1011007 (0.0006) [2023-12-27 12:39:20,456][20381] Updated weights for policy 1, policy_version 1010314 (0.0008) [2023-12-27 12:39:20,505][20381] Updated weights for policy 1, policy_version 1010324 (0.0005) [2023-12-27 12:39:21,136][20347] Updated weights for policy 0, policy_version 1011017 (0.0008) [2023-12-27 12:39:21,196][20347] Updated weights for policy 0, policy_version 1011027 (0.0010) [2023-12-27 12:39:21,236][20381] Updated weights for policy 1, policy_version 1010334 (0.0007) [2023-12-27 12:39:21,259][20347] Updated weights for policy 0, policy_version 1011037 (0.0010) [2023-12-27 12:39:21,301][20381] Updated weights for policy 1, policy_version 1010344 (0.0008) [2023-12-27 12:39:21,316][20347] Updated weights for policy 0, policy_version 1011047 (0.0010) [2023-12-27 12:39:21,368][20381] Updated weights for policy 1, policy_version 1010354 (0.0009) [2023-12-27 12:39:22,067][20347] Updated weights for policy 0, policy_version 1011057 (0.0007) [2023-12-27 12:39:22,124][20347] Updated weights for policy 0, policy_version 1011067 (0.0008) [2023-12-27 12:39:22,148][20381] Updated weights for policy 1, policy_version 1010364 (0.0007) [2023-12-27 12:39:22,186][20347] Updated weights for policy 0, policy_version 1011077 (0.0007) [2023-12-27 12:39:22,206][20381] Updated weights for policy 1, policy_version 1010374 (0.0010) [2023-12-27 12:39:22,267][20381] Updated weights for policy 1, policy_version 1010384 (0.0010) [2023-12-27 12:39:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 517570560. Throughput: 0: 10291.2, 1: 10131.5. Samples: 517536048. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:22,363][19170] Avg episode reward: [(0, '805.920'), (1, '637.050')] [2023-12-27 12:39:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001010392_258686976.pth... [2023-12-27 12:39:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001011080_258883584.pth... [2023-12-27 12:39:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001009864_258572288.pth [2023-12-27 12:39:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001009208_258383872.pth [2023-12-27 12:39:22,972][20347] Updated weights for policy 0, policy_version 1011087 (0.0009) [2023-12-27 12:39:23,007][20381] Updated weights for policy 1, policy_version 1010394 (0.0010) [2023-12-27 12:39:23,037][20347] Updated weights for policy 0, policy_version 1011097 (0.0006) [2023-12-27 12:39:23,062][20381] Updated weights for policy 1, policy_version 1010404 (0.0011) [2023-12-27 12:39:23,102][20347] Updated weights for policy 0, policy_version 1011107 (0.0006) [2023-12-27 12:39:23,122][20381] Updated weights for policy 1, policy_version 1010414 (0.0012) [2023-12-27 12:39:23,184][20381] Updated weights for policy 1, policy_version 1010424 (0.0011) [2023-12-27 12:39:23,722][20347] Updated weights for policy 0, policy_version 1011117 (0.0005) [2023-12-27 12:39:23,782][20347] Updated weights for policy 0, policy_version 1011127 (0.0007) [2023-12-27 12:39:23,802][20291] KL-divergence is very high: 139.9042 [2023-12-27 12:39:23,836][20381] Updated weights for policy 1, policy_version 1010434 (0.0007) [2023-12-27 12:39:23,846][20347] Updated weights for policy 0, policy_version 1011137 (0.0007) [2023-12-27 12:39:23,854][20291] KL-divergence is very high: 115.3931 [2023-12-27 12:39:23,890][20381] Updated weights for policy 1, policy_version 1010444 (0.0011) [2023-12-27 12:39:23,935][20381] Updated weights for policy 1, policy_version 1010454 (0.0010) [2023-12-27 12:39:24,463][20347] Updated weights for policy 0, policy_version 1011147 (0.0007) [2023-12-27 12:39:24,511][20347] Updated weights for policy 0, policy_version 1011157 (0.0010) [2023-12-27 12:39:24,568][20347] Updated weights for policy 0, policy_version 1011167 (0.0010) [2023-12-27 12:39:24,601][20381] Updated weights for policy 1, policy_version 1010464 (0.0008) [2023-12-27 12:39:24,652][20381] Updated weights for policy 1, policy_version 1010474 (0.0006) [2023-12-27 12:39:24,705][20381] Updated weights for policy 1, policy_version 1010484 (0.0005) [2023-12-27 12:39:25,248][20347] Updated weights for policy 0, policy_version 1011177 (0.0010) [2023-12-27 12:39:25,256][20381] Updated weights for policy 1, policy_version 1010494 (0.0008) [2023-12-27 12:39:25,305][20347] Updated weights for policy 0, policy_version 1011187 (0.0005) [2023-12-27 12:39:25,309][20381] Updated weights for policy 1, policy_version 1010504 (0.0008) [2023-12-27 12:39:25,359][20347] Updated weights for policy 0, policy_version 1011197 (0.0005) [2023-12-27 12:39:25,372][20381] Updated weights for policy 1, policy_version 1010514 (0.0008) [2023-12-27 12:39:25,415][20347] Updated weights for policy 0, policy_version 1011207 (0.0005) [2023-12-27 12:39:26,027][20347] Updated weights for policy 0, policy_version 1011217 (0.0010) [2023-12-27 12:39:26,070][20381] Updated weights for policy 1, policy_version 1010525 (0.0011) [2023-12-27 12:39:26,079][20347] Updated weights for policy 0, policy_version 1011227 (0.0010) [2023-12-27 12:39:26,125][20381] Updated weights for policy 1, policy_version 1010535 (0.0010) [2023-12-27 12:39:26,132][20347] Updated weights for policy 0, policy_version 1011237 (0.0010) [2023-12-27 12:39:26,179][20381] Updated weights for policy 1, policy_version 1010545 (0.0010) [2023-12-27 12:39:26,742][20347] Updated weights for policy 0, policy_version 1011247 (0.0009) [2023-12-27 12:39:26,791][20347] Updated weights for policy 0, policy_version 1011257 (0.0005) [2023-12-27 12:39:26,807][20381] Updated weights for policy 1, policy_version 1010555 (0.0009) [2023-12-27 12:39:26,843][20347] Updated weights for policy 0, policy_version 1011267 (0.0005) [2023-12-27 12:39:26,859][20381] Updated weights for policy 1, policy_version 1010565 (0.0005) [2023-12-27 12:39:26,911][20381] Updated weights for policy 1, policy_version 1010575 (0.0005) [2023-12-27 12:39:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20466.2). Total num frames: 517668864. Throughput: 0: 10249.1, 1: 10144.7. Samples: 517657736. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:27,363][19170] Avg episode reward: [(0, '864.280'), (1, '647.100')] [2023-12-27 12:39:27,502][20381] Updated weights for policy 1, policy_version 1010585 (0.0005) [2023-12-27 12:39:27,513][20347] Updated weights for policy 0, policy_version 1011277 (0.0006) [2023-12-27 12:39:27,560][20347] Updated weights for policy 0, policy_version 1011287 (0.0007) [2023-12-27 12:39:27,561][20381] Updated weights for policy 1, policy_version 1010595 (0.0005) [2023-12-27 12:39:27,622][20347] Updated weights for policy 0, policy_version 1011298 (0.0007) [2023-12-27 12:39:27,627][20381] Updated weights for policy 1, policy_version 1010605 (0.0006) [2023-12-27 12:39:27,680][20381] Updated weights for policy 1, policy_version 1010615 (0.0005) [2023-12-27 12:39:28,185][20381] Updated weights for policy 1, policy_version 1010625 (0.0010) [2023-12-27 12:39:28,243][20381] Updated weights for policy 1, policy_version 1010635 (0.0011) [2023-12-27 12:39:28,272][20347] Updated weights for policy 0, policy_version 1011308 (0.0009) [2023-12-27 12:39:28,296][20381] Updated weights for policy 1, policy_version 1010645 (0.0010) [2023-12-27 12:39:28,318][20347] Updated weights for policy 0, policy_version 1011318 (0.0010) [2023-12-27 12:39:28,378][20347] Updated weights for policy 0, policy_version 1011328 (0.0010) [2023-12-27 12:39:28,996][20381] Updated weights for policy 1, policy_version 1010655 (0.0008) [2023-12-27 12:39:29,047][20347] Updated weights for policy 0, policy_version 1011338 (0.0006) [2023-12-27 12:39:29,057][20381] Updated weights for policy 1, policy_version 1010665 (0.0007) [2023-12-27 12:39:29,105][20347] Updated weights for policy 0, policy_version 1011348 (0.0006) [2023-12-27 12:39:29,107][20381] Updated weights for policy 1, policy_version 1010675 (0.0005) [2023-12-27 12:39:29,154][20347] Updated weights for policy 0, policy_version 1011358 (0.0008) [2023-12-27 12:39:29,205][20347] Updated weights for policy 0, policy_version 1011368 (0.0005) [2023-12-27 12:39:29,814][20381] Updated weights for policy 1, policy_version 1010685 (0.0008) [2023-12-27 12:39:29,877][20381] Updated weights for policy 1, policy_version 1010695 (0.0008) [2023-12-27 12:39:29,885][20347] Updated weights for policy 0, policy_version 1011378 (0.0011) [2023-12-27 12:39:29,941][20381] Updated weights for policy 1, policy_version 1010705 (0.0007) [2023-12-27 12:39:29,948][20347] Updated weights for policy 0, policy_version 1011388 (0.0011) [2023-12-27 12:39:30,006][20347] Updated weights for policy 0, policy_version 1011398 (0.0010) [2023-12-27 12:39:30,577][20381] Updated weights for policy 1, policy_version 1010715 (0.0006) [2023-12-27 12:39:30,627][20381] Updated weights for policy 1, policy_version 1010725 (0.0005) [2023-12-27 12:39:30,666][20347] Updated weights for policy 0, policy_version 1011408 (0.0006) [2023-12-27 12:39:30,679][20381] Updated weights for policy 1, policy_version 1010735 (0.0010) [2023-12-27 12:39:30,724][20347] Updated weights for policy 0, policy_version 1011418 (0.0005) [2023-12-27 12:39:30,775][20347] Updated weights for policy 0, policy_version 1011428 (0.0005) [2023-12-27 12:39:31,351][20381] Updated weights for policy 1, policy_version 1010745 (0.0010) [2023-12-27 12:39:31,367][20347] Updated weights for policy 0, policy_version 1011438 (0.0008) [2023-12-27 12:39:31,416][20381] Updated weights for policy 1, policy_version 1010755 (0.0008) [2023-12-27 12:39:31,423][20347] Updated weights for policy 0, policy_version 1011448 (0.0008) [2023-12-27 12:39:31,463][20381] Updated weights for policy 1, policy_version 1010765 (0.0008) [2023-12-27 12:39:31,473][20347] Updated weights for policy 0, policy_version 1011458 (0.0007) [2023-12-27 12:39:31,512][20381] Updated weights for policy 1, policy_version 1010775 (0.0006) [2023-12-27 12:39:32,126][20347] Updated weights for policy 0, policy_version 1011468 (0.0007) [2023-12-27 12:39:32,184][20347] Updated weights for policy 0, policy_version 1011478 (0.0006) [2023-12-27 12:39:32,184][20381] Updated weights for policy 1, policy_version 1010785 (0.0006) [2023-12-27 12:39:32,240][20381] Updated weights for policy 1, policy_version 1010795 (0.0006) [2023-12-27 12:39:32,253][20347] Updated weights for policy 0, policy_version 1011488 (0.0006) [2023-12-27 12:39:32,301][20381] Updated weights for policy 1, policy_version 1010805 (0.0008) [2023-12-27 12:39:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 517783552. Throughput: 0: 10300.9, 1: 10197.5. Samples: 517786920. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:32,363][19170] Avg episode reward: [(0, '922.170'), (1, '519.170')] [2023-12-27 12:39:32,927][20347] Updated weights for policy 0, policy_version 1011498 (0.0007) [2023-12-27 12:39:32,981][20381] Updated weights for policy 1, policy_version 1010815 (0.0008) [2023-12-27 12:39:32,983][20347] Updated weights for policy 0, policy_version 1011508 (0.0006) [2023-12-27 12:39:33,034][20347] Updated weights for policy 0, policy_version 1011518 (0.0008) [2023-12-27 12:39:33,037][20381] Updated weights for policy 1, policy_version 1010825 (0.0006) [2023-12-27 12:39:33,083][20347] Updated weights for policy 0, policy_version 1011528 (0.0006) [2023-12-27 12:39:33,085][20381] Updated weights for policy 1, policy_version 1010835 (0.0005) [2023-12-27 12:39:33,692][20381] Updated weights for policy 1, policy_version 1010845 (0.0008) [2023-12-27 12:39:33,756][20381] Updated weights for policy 1, policy_version 1010855 (0.0006) [2023-12-27 12:39:33,800][20347] Updated weights for policy 0, policy_version 1011538 (0.0007) [2023-12-27 12:39:33,816][20381] Updated weights for policy 1, policy_version 1010865 (0.0005) [2023-12-27 12:39:33,848][20347] Updated weights for policy 0, policy_version 1011548 (0.0007) [2023-12-27 12:39:33,897][20347] Updated weights for policy 0, policy_version 1011558 (0.0007) [2023-12-27 12:39:34,393][20381] Updated weights for policy 1, policy_version 1010875 (0.0005) [2023-12-27 12:39:34,448][20381] Updated weights for policy 1, policy_version 1010885 (0.0006) [2023-12-27 12:39:34,503][20381] Updated weights for policy 1, policy_version 1010895 (0.0005) [2023-12-27 12:39:34,652][20347] Updated weights for policy 0, policy_version 1011568 (0.0009) [2023-12-27 12:39:34,709][20347] Updated weights for policy 0, policy_version 1011578 (0.0008) [2023-12-27 12:39:34,768][20347] Updated weights for policy 0, policy_version 1011588 (0.0006) [2023-12-27 12:39:35,181][20381] Updated weights for policy 1, policy_version 1010905 (0.0010) [2023-12-27 12:39:35,238][20381] Updated weights for policy 1, policy_version 1010915 (0.0009) [2023-12-27 12:39:35,301][20381] Updated weights for policy 1, policy_version 1010925 (0.0008) [2023-12-27 12:39:35,328][20347] Updated weights for policy 0, policy_version 1011598 (0.0007) [2023-12-27 12:39:35,356][20381] Updated weights for policy 1, policy_version 1010935 (0.0006) [2023-12-27 12:39:35,386][20347] Updated weights for policy 0, policy_version 1011608 (0.0008) [2023-12-27 12:39:35,438][20347] Updated weights for policy 0, policy_version 1011618 (0.0008) [2023-12-27 12:39:36,035][20381] Updated weights for policy 1, policy_version 1010945 (0.0008) [2023-12-27 12:39:36,084][20381] Updated weights for policy 1, policy_version 1010955 (0.0010) [2023-12-27 12:39:36,113][20347] Updated weights for policy 0, policy_version 1011628 (0.0009) [2023-12-27 12:39:36,146][20381] Updated weights for policy 1, policy_version 1010965 (0.0007) [2023-12-27 12:39:36,171][20347] Updated weights for policy 0, policy_version 1011638 (0.0007) [2023-12-27 12:39:36,231][20347] Updated weights for policy 0, policy_version 1011648 (0.0008) [2023-12-27 12:39:36,778][20381] Updated weights for policy 1, policy_version 1010975 (0.0008) [2023-12-27 12:39:36,834][20381] Updated weights for policy 1, policy_version 1010985 (0.0011) [2023-12-27 12:39:36,892][20381] Updated weights for policy 1, policy_version 1010995 (0.0011) [2023-12-27 12:39:36,919][20347] Updated weights for policy 0, policy_version 1011658 (0.0007) [2023-12-27 12:39:36,970][20347] Updated weights for policy 0, policy_version 1011668 (0.0007) [2023-12-27 12:39:37,018][20347] Updated weights for policy 0, policy_version 1011678 (0.0008) [2023-12-27 12:39:37,073][20347] Updated weights for policy 0, policy_version 1011688 (0.0007) [2023-12-27 12:39:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 517881856. Throughput: 0: 10285.9, 1: 10242.5. Samples: 517849944. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:37,364][19170] Avg episode reward: [(0, '931.880'), (1, '548.900')] [2023-12-27 12:39:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001011000_258842624.pth... [2023-12-27 12:39:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001011688_259039232.pth... [2023-12-27 12:39:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001009816_258539520.pth [2023-12-27 12:39:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001010472_258727936.pth [2023-12-27 12:39:37,522][20381] Updated weights for policy 1, policy_version 1011005 (0.0011) [2023-12-27 12:39:37,590][20381] Updated weights for policy 1, policy_version 1011015 (0.0011) [2023-12-27 12:39:37,653][20381] Updated weights for policy 1, policy_version 1011025 (0.0011) [2023-12-27 12:39:37,813][20347] Updated weights for policy 0, policy_version 1011698 (0.0007) [2023-12-27 12:39:37,878][20347] Updated weights for policy 0, policy_version 1011708 (0.0009) [2023-12-27 12:39:37,930][20347] Updated weights for policy 0, policy_version 1011718 (0.0008) [2023-12-27 12:39:38,278][20381] Updated weights for policy 1, policy_version 1011035 (0.0008) [2023-12-27 12:39:38,343][20381] Updated weights for policy 1, policy_version 1011045 (0.0010) [2023-12-27 12:39:38,366][20292] KL-divergence is very high: 108.1066 [2023-12-27 12:39:38,406][20381] Updated weights for policy 1, policy_version 1011055 (0.0010) [2023-12-27 12:39:38,739][20347] Updated weights for policy 0, policy_version 1011728 (0.0010) [2023-12-27 12:39:38,800][20347] Updated weights for policy 0, policy_version 1011738 (0.0010) [2023-12-27 12:39:38,855][20347] Updated weights for policy 0, policy_version 1011748 (0.0010) [2023-12-27 12:39:38,986][20381] Updated weights for policy 1, policy_version 1011065 (0.0008) [2023-12-27 12:39:39,035][20381] Updated weights for policy 1, policy_version 1011075 (0.0005) [2023-12-27 12:39:39,082][20381] Updated weights for policy 1, policy_version 1011085 (0.0006) [2023-12-27 12:39:39,136][20381] Updated weights for policy 1, policy_version 1011095 (0.0005) [2023-12-27 12:39:39,477][20347] Updated weights for policy 0, policy_version 1011758 (0.0009) [2023-12-27 12:39:39,536][20347] Updated weights for policy 0, policy_version 1011768 (0.0006) [2023-12-27 12:39:39,589][20347] Updated weights for policy 0, policy_version 1011778 (0.0007) [2023-12-27 12:39:39,761][20381] Updated weights for policy 1, policy_version 1011105 (0.0010) [2023-12-27 12:39:39,819][20381] Updated weights for policy 1, policy_version 1011115 (0.0011) [2023-12-27 12:39:39,880][20381] Updated weights for policy 1, policy_version 1011125 (0.0008) [2023-12-27 12:39:40,225][20347] Updated weights for policy 0, policy_version 1011788 (0.0006) [2023-12-27 12:39:40,278][20347] Updated weights for policy 0, policy_version 1011798 (0.0006) [2023-12-27 12:39:40,325][20347] Updated weights for policy 0, policy_version 1011808 (0.0007) [2023-12-27 12:39:40,545][20381] Updated weights for policy 1, policy_version 1011135 (0.0006) [2023-12-27 12:39:40,592][20381] Updated weights for policy 1, policy_version 1011145 (0.0005) [2023-12-27 12:39:40,646][20381] Updated weights for policy 1, policy_version 1011155 (0.0005) [2023-12-27 12:39:40,966][20347] Updated weights for policy 0, policy_version 1011818 (0.0007) [2023-12-27 12:39:41,019][20347] Updated weights for policy 0, policy_version 1011828 (0.0005) [2023-12-27 12:39:41,089][20347] Updated weights for policy 0, policy_version 1011838 (0.0008) [2023-12-27 12:39:41,169][20347] Updated weights for policy 0, policy_version 1011848 (0.0008) [2023-12-27 12:39:41,354][20381] Updated weights for policy 1, policy_version 1011165 (0.0007) [2023-12-27 12:39:41,430][20381] Updated weights for policy 1, policy_version 1011175 (0.0008) [2023-12-27 12:39:41,497][20381] Updated weights for policy 1, policy_version 1011185 (0.0006) [2023-12-27 12:39:41,889][20347] Updated weights for policy 0, policy_version 1011858 (0.0009) [2023-12-27 12:39:41,946][20347] Updated weights for policy 0, policy_version 1011868 (0.0008) [2023-12-27 12:39:41,998][20347] Updated weights for policy 0, policy_version 1011878 (0.0007) [2023-12-27 12:39:42,144][20381] Updated weights for policy 1, policy_version 1011195 (0.0007) [2023-12-27 12:39:42,194][20381] Updated weights for policy 1, policy_version 1011205 (0.0008) [2023-12-27 12:39:42,247][20381] Updated weights for policy 1, policy_version 1011215 (0.0008) [2023-12-27 12:39:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 517988352. Throughput: 0: 10273.0, 1: 10341.3. Samples: 517975812. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:42,363][19170] Avg episode reward: [(0, '937.910'), (1, '597.690')] [2023-12-27 12:39:42,656][20347] Updated weights for policy 0, policy_version 1011888 (0.0006) [2023-12-27 12:39:42,702][20347] Updated weights for policy 0, policy_version 1011898 (0.0007) [2023-12-27 12:39:42,750][20347] Updated weights for policy 0, policy_version 1011908 (0.0007) [2023-12-27 12:39:43,000][20381] Updated weights for policy 1, policy_version 1011225 (0.0007) [2023-12-27 12:39:43,064][20381] Updated weights for policy 1, policy_version 1011235 (0.0011) [2023-12-27 12:39:43,125][20381] Updated weights for policy 1, policy_version 1011245 (0.0010) [2023-12-27 12:39:43,191][20381] Updated weights for policy 1, policy_version 1011255 (0.0011) [2023-12-27 12:39:43,392][20347] Updated weights for policy 0, policy_version 1011918 (0.0007) [2023-12-27 12:39:43,446][20347] Updated weights for policy 0, policy_version 1011928 (0.0007) [2023-12-27 12:39:43,502][20347] Updated weights for policy 0, policy_version 1011938 (0.0007) [2023-12-27 12:39:43,829][20381] Updated weights for policy 1, policy_version 1011265 (0.0007) [2023-12-27 12:39:43,850][20292] KL-divergence is very high: 104.2355 [2023-12-27 12:39:43,878][20381] Updated weights for policy 1, policy_version 1011275 (0.0010) [2023-12-27 12:39:43,925][20381] Updated weights for policy 1, policy_version 1011285 (0.0010) [2023-12-27 12:39:44,189][20347] Updated weights for policy 0, policy_version 1011948 (0.0007) [2023-12-27 12:39:44,236][20347] Updated weights for policy 0, policy_version 1011958 (0.0007) [2023-12-27 12:39:44,290][20347] Updated weights for policy 0, policy_version 1011968 (0.0007) [2023-12-27 12:39:44,577][20381] Updated weights for policy 1, policy_version 1011295 (0.0007) [2023-12-27 12:39:44,633][20381] Updated weights for policy 1, policy_version 1011305 (0.0009) [2023-12-27 12:39:44,691][20381] Updated weights for policy 1, policy_version 1011315 (0.0010) [2023-12-27 12:39:44,976][20347] Updated weights for policy 0, policy_version 1011978 (0.0008) [2023-12-27 12:39:45,039][20347] Updated weights for policy 0, policy_version 1011988 (0.0007) [2023-12-27 12:39:45,105][20347] Updated weights for policy 0, policy_version 1011998 (0.0008) [2023-12-27 12:39:45,165][20347] Updated weights for policy 0, policy_version 1012008 (0.0007) [2023-12-27 12:39:45,398][20381] Updated weights for policy 1, policy_version 1011325 (0.0011) [2023-12-27 12:39:45,459][20381] Updated weights for policy 1, policy_version 1011335 (0.0010) [2023-12-27 12:39:45,514][20381] Updated weights for policy 1, policy_version 1011345 (0.0010) [2023-12-27 12:39:45,865][20347] Updated weights for policy 0, policy_version 1012018 (0.0008) [2023-12-27 12:39:45,919][20347] Updated weights for policy 0, policy_version 1012028 (0.0007) [2023-12-27 12:39:45,974][20347] Updated weights for policy 0, policy_version 1012038 (0.0007) [2023-12-27 12:39:46,157][20381] Updated weights for policy 1, policy_version 1011355 (0.0007) [2023-12-27 12:39:46,202][20381] Updated weights for policy 1, policy_version 1011365 (0.0007) [2023-12-27 12:39:46,248][20381] Updated weights for policy 1, policy_version 1011375 (0.0007) [2023-12-27 12:39:46,654][20347] Updated weights for policy 0, policy_version 1012048 (0.0007) [2023-12-27 12:39:46,700][20347] Updated weights for policy 0, policy_version 1012058 (0.0007) [2023-12-27 12:39:46,746][20347] Updated weights for policy 0, policy_version 1012068 (0.0008) [2023-12-27 12:39:46,885][20381] Updated weights for policy 1, policy_version 1011385 (0.0008) [2023-12-27 12:39:46,943][20381] Updated weights for policy 1, policy_version 1011395 (0.0008) [2023-12-27 12:39:47,004][20381] Updated weights for policy 1, policy_version 1011405 (0.0007) [2023-12-27 12:39:47,062][20381] Updated weights for policy 1, policy_version 1011415 (0.0007) [2023-12-27 12:39:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 518086656. Throughput: 0: 10280.4, 1: 10302.6. Samples: 518097512. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:47,363][19170] Avg episode reward: [(0, '889.110'), (1, '317.260')] [2023-12-27 12:39:47,419][20347] Updated weights for policy 0, policy_version 1012078 (0.0007) [2023-12-27 12:39:47,473][20347] Updated weights for policy 0, policy_version 1012088 (0.0007) [2023-12-27 12:39:47,528][20347] Updated weights for policy 0, policy_version 1012098 (0.0008) [2023-12-27 12:39:47,697][20381] Updated weights for policy 1, policy_version 1011425 (0.0006) [2023-12-27 12:39:47,764][20381] Updated weights for policy 1, policy_version 1011435 (0.0006) [2023-12-27 12:39:47,821][20381] Updated weights for policy 1, policy_version 1011445 (0.0006) [2023-12-27 12:39:48,232][20347] Updated weights for policy 0, policy_version 1012108 (0.0008) [2023-12-27 12:39:48,294][20347] Updated weights for policy 0, policy_version 1012118 (0.0008) [2023-12-27 12:39:48,356][20347] Updated weights for policy 0, policy_version 1012128 (0.0008) [2023-12-27 12:39:48,527][20381] Updated weights for policy 1, policy_version 1011455 (0.0006) [2023-12-27 12:39:48,578][20381] Updated weights for policy 1, policy_version 1011465 (0.0006) [2023-12-27 12:39:48,628][20381] Updated weights for policy 1, policy_version 1011475 (0.0007) [2023-12-27 12:39:49,054][20347] Updated weights for policy 0, policy_version 1012138 (0.0007) [2023-12-27 12:39:49,104][20347] Updated weights for policy 0, policy_version 1012148 (0.0008) [2023-12-27 12:39:49,160][20347] Updated weights for policy 0, policy_version 1012158 (0.0008) [2023-12-27 12:39:49,220][20347] Updated weights for policy 0, policy_version 1012168 (0.0008) [2023-12-27 12:39:49,236][20381] Updated weights for policy 1, policy_version 1011485 (0.0009) [2023-12-27 12:39:49,293][20381] Updated weights for policy 1, policy_version 1011495 (0.0008) [2023-12-27 12:39:49,363][20381] Updated weights for policy 1, policy_version 1011505 (0.0007) [2023-12-27 12:39:49,982][20347] Updated weights for policy 0, policy_version 1012178 (0.0008) [2023-12-27 12:39:50,017][20381] Updated weights for policy 1, policy_version 1011515 (0.0007) [2023-12-27 12:39:50,047][20347] Updated weights for policy 0, policy_version 1012188 (0.0007) [2023-12-27 12:39:50,080][20381] Updated weights for policy 1, policy_version 1011525 (0.0009) [2023-12-27 12:39:50,106][20347] Updated weights for policy 0, policy_version 1012198 (0.0006) [2023-12-27 12:39:50,137][20381] Updated weights for policy 1, policy_version 1011535 (0.0007) [2023-12-27 12:39:50,816][20381] Updated weights for policy 1, policy_version 1011545 (0.0007) [2023-12-27 12:39:50,829][20347] Updated weights for policy 0, policy_version 1012208 (0.0010) [2023-12-27 12:39:50,881][20381] Updated weights for policy 1, policy_version 1011555 (0.0005) [2023-12-27 12:39:50,894][20347] Updated weights for policy 0, policy_version 1012218 (0.0010) [2023-12-27 12:39:50,946][20381] Updated weights for policy 1, policy_version 1011565 (0.0007) [2023-12-27 12:39:50,952][20347] Updated weights for policy 0, policy_version 1012228 (0.0006) [2023-12-27 12:39:51,014][20381] Updated weights for policy 1, policy_version 1011575 (0.0008) [2023-12-27 12:39:51,594][20347] Updated weights for policy 0, policy_version 1012238 (0.0005) [2023-12-27 12:39:51,658][20347] Updated weights for policy 0, policy_version 1012248 (0.0007) [2023-12-27 12:39:51,717][20347] Updated weights for policy 0, policy_version 1012258 (0.0007) [2023-12-27 12:39:51,725][20381] Updated weights for policy 1, policy_version 1011585 (0.0007) [2023-12-27 12:39:51,789][20381] Updated weights for policy 1, policy_version 1011595 (0.0007) [2023-12-27 12:39:51,850][20381] Updated weights for policy 1, policy_version 1011605 (0.0008) [2023-12-27 12:39:52,354][20347] Updated weights for policy 0, policy_version 1012268 (0.0008) [2023-12-27 12:39:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 518184960. Throughput: 0: 10243.7, 1: 10318.0. Samples: 518158452. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:52,363][19170] Avg episode reward: [(0, '892.860'), (1, '352.890')] [2023-12-27 12:39:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001011608_258998272.pth... [2023-12-27 12:39:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001010392_258686976.pth [2023-12-27 12:39:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001011608_258998272.pth [2023-12-27 12:39:52,421][20347] Updated weights for policy 0, policy_version 1012278 (0.0009) [2023-12-27 12:39:52,485][20347] Updated weights for policy 0, policy_version 1012288 (0.0006) [2023-12-27 12:39:52,533][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001012296_259194880.pth... [2023-12-27 12:39:52,538][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001011080_258883584.pth [2023-12-27 12:39:52,539][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001012296_259194880.pth [2023-12-27 12:39:52,571][20381] Updated weights for policy 1, policy_version 1011615 (0.0008) [2023-12-27 12:39:52,637][20381] Updated weights for policy 1, policy_version 1011625 (0.0007) [2023-12-27 12:39:52,699][20381] Updated weights for policy 1, policy_version 1011635 (0.0007) [2023-12-27 12:39:53,148][20347] Updated weights for policy 0, policy_version 1012298 (0.0007) [2023-12-27 12:39:53,209][20347] Updated weights for policy 0, policy_version 1012308 (0.0008) [2023-12-27 12:39:53,274][20347] Updated weights for policy 0, policy_version 1012318 (0.0011) [2023-12-27 12:39:53,336][20347] Updated weights for policy 0, policy_version 1012328 (0.0010) [2023-12-27 12:39:53,374][20381] Updated weights for policy 1, policy_version 1011645 (0.0007) [2023-12-27 12:39:53,436][20381] Updated weights for policy 1, policy_version 1011655 (0.0007) [2023-12-27 12:39:53,496][20381] Updated weights for policy 1, policy_version 1011665 (0.0008) [2023-12-27 12:39:54,009][20347] Updated weights for policy 0, policy_version 1012338 (0.0010) [2023-12-27 12:39:54,067][20347] Updated weights for policy 0, policy_version 1012348 (0.0010) [2023-12-27 12:39:54,123][20347] Updated weights for policy 0, policy_version 1012358 (0.0010) [2023-12-27 12:39:54,190][20381] Updated weights for policy 1, policy_version 1011675 (0.0008) [2023-12-27 12:39:54,251][20381] Updated weights for policy 1, policy_version 1011685 (0.0007) [2023-12-27 12:39:54,307][20381] Updated weights for policy 1, policy_version 1011695 (0.0008) [2023-12-27 12:39:54,817][20347] Updated weights for policy 0, policy_version 1012368 (0.0008) [2023-12-27 12:39:54,874][20347] Updated weights for policy 0, policy_version 1012378 (0.0007) [2023-12-27 12:39:54,929][20347] Updated weights for policy 0, policy_version 1012388 (0.0007) [2023-12-27 12:39:54,951][20381] Updated weights for policy 1, policy_version 1011705 (0.0007) [2023-12-27 12:39:55,013][20381] Updated weights for policy 1, policy_version 1011715 (0.0007) [2023-12-27 12:39:55,072][20381] Updated weights for policy 1, policy_version 1011725 (0.0005) [2023-12-27 12:39:55,125][20381] Updated weights for policy 1, policy_version 1011735 (0.0005) [2023-12-27 12:39:55,520][20347] Updated weights for policy 0, policy_version 1012398 (0.0006) [2023-12-27 12:39:55,567][20347] Updated weights for policy 0, policy_version 1012408 (0.0005) [2023-12-27 12:39:55,617][20347] Updated weights for policy 0, policy_version 1012418 (0.0005) [2023-12-27 12:39:55,739][20381] Updated weights for policy 1, policy_version 1011745 (0.0007) [2023-12-27 12:39:55,797][20381] Updated weights for policy 1, policy_version 1011755 (0.0006) [2023-12-27 12:39:55,852][20381] Updated weights for policy 1, policy_version 1011765 (0.0007) [2023-12-27 12:39:56,273][20347] Updated weights for policy 0, policy_version 1012428 (0.0006) [2023-12-27 12:39:56,332][20347] Updated weights for policy 0, policy_version 1012438 (0.0007) [2023-12-27 12:39:56,386][20347] Updated weights for policy 0, policy_version 1012448 (0.0006) [2023-12-27 12:39:56,461][20381] Updated weights for policy 1, policy_version 1011775 (0.0009) [2023-12-27 12:39:56,524][20381] Updated weights for policy 1, policy_version 1011785 (0.0010) [2023-12-27 12:39:56,581][20381] Updated weights for policy 1, policy_version 1011795 (0.0010) [2023-12-27 12:39:57,051][20347] Updated weights for policy 0, policy_version 1012458 (0.0007) [2023-12-27 12:39:57,098][20347] Updated weights for policy 0, policy_version 1012468 (0.0007) [2023-12-27 12:39:57,154][20347] Updated weights for policy 0, policy_version 1012478 (0.0007) [2023-12-27 12:39:57,200][20347] Updated weights for policy 0, policy_version 1012488 (0.0009) [2023-12-27 12:39:57,244][20381] Updated weights for policy 1, policy_version 1011805 (0.0010) [2023-12-27 12:39:57,295][20381] Updated weights for policy 1, policy_version 1011815 (0.0010) [2023-12-27 12:39:57,347][20381] Updated weights for policy 1, policy_version 1011825 (0.0010) [2023-12-27 12:39:57,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 518291456. Throughput: 0: 10302.3, 1: 10359.2. Samples: 518282804. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:39:57,363][19170] Avg episode reward: [(0, '863.850'), (1, '202.680')] [2023-12-27 12:39:57,852][20347] Updated weights for policy 0, policy_version 1012498 (0.0010) [2023-12-27 12:39:57,906][20347] Updated weights for policy 0, policy_version 1012508 (0.0008) [2023-12-27 12:39:57,964][20347] Updated weights for policy 0, policy_version 1012518 (0.0005) [2023-12-27 12:39:57,978][20381] Updated weights for policy 1, policy_version 1011835 (0.0010) [2023-12-27 12:39:58,032][20381] Updated weights for policy 1, policy_version 1011845 (0.0010) [2023-12-27 12:39:58,086][20381] Updated weights for policy 1, policy_version 1011855 (0.0010) [2023-12-27 12:39:58,601][20347] Updated weights for policy 0, policy_version 1012528 (0.0007) [2023-12-27 12:39:58,666][20347] Updated weights for policy 0, policy_version 1012538 (0.0007) [2023-12-27 12:39:58,735][20347] Updated weights for policy 0, policy_version 1012548 (0.0007) [2023-12-27 12:39:58,938][20381] Updated weights for policy 1, policy_version 1011865 (0.0010) [2023-12-27 12:39:58,999][20381] Updated weights for policy 1, policy_version 1011875 (0.0010) [2023-12-27 12:39:59,054][20381] Updated weights for policy 1, policy_version 1011885 (0.0006) [2023-12-27 12:39:59,112][20381] Updated weights for policy 1, policy_version 1011895 (0.0006) [2023-12-27 12:39:59,512][20347] Updated weights for policy 0, policy_version 1012558 (0.0009) [2023-12-27 12:39:59,560][20347] Updated weights for policy 0, policy_version 1012568 (0.0010) [2023-12-27 12:39:59,613][20347] Updated weights for policy 0, policy_version 1012578 (0.0010) [2023-12-27 12:39:59,719][20381] Updated weights for policy 1, policy_version 1011905 (0.0005) [2023-12-27 12:39:59,766][20381] Updated weights for policy 1, policy_version 1011915 (0.0007) [2023-12-27 12:39:59,821][20381] Updated weights for policy 1, policy_version 1011925 (0.0007) [2023-12-27 12:40:00,329][20347] Updated weights for policy 0, policy_version 1012588 (0.0008) [2023-12-27 12:40:00,390][20347] Updated weights for policy 0, policy_version 1012598 (0.0007) [2023-12-27 12:40:00,446][20347] Updated weights for policy 0, policy_version 1012608 (0.0007) [2023-12-27 12:40:00,503][20381] Updated weights for policy 1, policy_version 1011935 (0.0009) [2023-12-27 12:40:00,555][20381] Updated weights for policy 1, policy_version 1011945 (0.0008) [2023-12-27 12:40:00,617][20381] Updated weights for policy 1, policy_version 1011955 (0.0006) [2023-12-27 12:40:01,037][20347] Updated weights for policy 0, policy_version 1012618 (0.0007) [2023-12-27 12:40:01,098][20347] Updated weights for policy 0, policy_version 1012628 (0.0006) [2023-12-27 12:40:01,163][20347] Updated weights for policy 0, policy_version 1012638 (0.0007) [2023-12-27 12:40:01,223][20347] Updated weights for policy 0, policy_version 1012648 (0.0006) [2023-12-27 12:40:01,284][20381] Updated weights for policy 1, policy_version 1011965 (0.0007) [2023-12-27 12:40:01,339][20381] Updated weights for policy 1, policy_version 1011975 (0.0008) [2023-12-27 12:40:01,408][20381] Updated weights for policy 1, policy_version 1011985 (0.0008) [2023-12-27 12:40:01,841][20347] Updated weights for policy 0, policy_version 1012658 (0.0008) [2023-12-27 12:40:01,905][20347] Updated weights for policy 0, policy_version 1012668 (0.0008) [2023-12-27 12:40:01,967][20347] Updated weights for policy 0, policy_version 1012678 (0.0007) [2023-12-27 12:40:02,105][20381] Updated weights for policy 1, policy_version 1011995 (0.0007) [2023-12-27 12:40:02,155][20381] Updated weights for policy 1, policy_version 1012005 (0.0005) [2023-12-27 12:40:02,203][20381] Updated weights for policy 1, policy_version 1012015 (0.0005) [2023-12-27 12:40:02,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 518397952. Throughput: 0: 10296.3, 1: 10393.4. Samples: 518405344. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:02,363][19170] Avg episode reward: [(0, '834.560'), (1, '55.520')] [2023-12-27 12:40:02,659][20347] Updated weights for policy 0, policy_version 1012688 (0.0007) [2023-12-27 12:40:02,716][20347] Updated weights for policy 0, policy_version 1012698 (0.0007) [2023-12-27 12:40:02,770][20347] Updated weights for policy 0, policy_version 1012708 (0.0007) [2023-12-27 12:40:02,888][20381] Updated weights for policy 1, policy_version 1012025 (0.0006) [2023-12-27 12:40:02,939][20381] Updated weights for policy 1, policy_version 1012035 (0.0007) [2023-12-27 12:40:02,984][20381] Updated weights for policy 1, policy_version 1012045 (0.0007) [2023-12-27 12:40:03,028][20381] Updated weights for policy 1, policy_version 1012055 (0.0007) [2023-12-27 12:40:03,469][20347] Updated weights for policy 0, policy_version 1012718 (0.0007) [2023-12-27 12:40:03,520][20347] Updated weights for policy 0, policy_version 1012728 (0.0008) [2023-12-27 12:40:03,579][20347] Updated weights for policy 0, policy_version 1012738 (0.0006) [2023-12-27 12:40:03,732][20381] Updated weights for policy 1, policy_version 1012065 (0.0005) [2023-12-27 12:40:03,783][20381] Updated weights for policy 1, policy_version 1012075 (0.0005) [2023-12-27 12:40:03,844][20381] Updated weights for policy 1, policy_version 1012085 (0.0006) [2023-12-27 12:40:04,250][20347] Updated weights for policy 0, policy_version 1012748 (0.0008) [2023-12-27 12:40:04,299][20347] Updated weights for policy 0, policy_version 1012758 (0.0007) [2023-12-27 12:40:04,356][20347] Updated weights for policy 0, policy_version 1012768 (0.0007) [2023-12-27 12:40:04,517][20381] Updated weights for policy 1, policy_version 1012095 (0.0010) [2023-12-27 12:40:04,573][20381] Updated weights for policy 1, policy_version 1012105 (0.0010) [2023-12-27 12:40:04,625][20381] Updated weights for policy 1, policy_version 1012115 (0.0010) [2023-12-27 12:40:05,020][20347] Updated weights for policy 0, policy_version 1012778 (0.0007) [2023-12-27 12:40:05,075][20347] Updated weights for policy 0, policy_version 1012788 (0.0007) [2023-12-27 12:40:05,128][20347] Updated weights for policy 0, policy_version 1012798 (0.0007) [2023-12-27 12:40:05,186][20347] Updated weights for policy 0, policy_version 1012808 (0.0007) [2023-12-27 12:40:05,329][20381] Updated weights for policy 1, policy_version 1012125 (0.0008) [2023-12-27 12:40:05,383][20381] Updated weights for policy 1, policy_version 1012135 (0.0005) [2023-12-27 12:40:05,440][20381] Updated weights for policy 1, policy_version 1012145 (0.0006) [2023-12-27 12:40:05,866][20347] Updated weights for policy 0, policy_version 1012818 (0.0006) [2023-12-27 12:40:05,921][20347] Updated weights for policy 0, policy_version 1012828 (0.0007) [2023-12-27 12:40:05,972][20347] Updated weights for policy 0, policy_version 1012838 (0.0007) [2023-12-27 12:40:06,068][20381] Updated weights for policy 1, policy_version 1012155 (0.0006) [2023-12-27 12:40:06,126][20381] Updated weights for policy 1, policy_version 1012165 (0.0007) [2023-12-27 12:40:06,181][20381] Updated weights for policy 1, policy_version 1012175 (0.0007) [2023-12-27 12:40:06,649][20347] Updated weights for policy 0, policy_version 1012848 (0.0008) [2023-12-27 12:40:06,701][20347] Updated weights for policy 0, policy_version 1012858 (0.0007) [2023-12-27 12:40:06,753][20347] Updated weights for policy 0, policy_version 1012868 (0.0008) [2023-12-27 12:40:06,912][20381] Updated weights for policy 1, policy_version 1012185 (0.0007) [2023-12-27 12:40:06,973][20381] Updated weights for policy 1, policy_version 1012195 (0.0006) [2023-12-27 12:40:07,033][20381] Updated weights for policy 1, policy_version 1012205 (0.0005) [2023-12-27 12:40:07,093][20381] Updated weights for policy 1, policy_version 1012215 (0.0007) [2023-12-27 12:40:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 518496256. Throughput: 0: 10287.4, 1: 10398.5. Samples: 518466916. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:07,364][19170] Avg episode reward: [(0, '854.160'), (1, '162.250')] [2023-12-27 12:40:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001012216_259153920.pth... [2023-12-27 12:40:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001012872_259342336.pth... [2023-12-27 12:40:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001011688_259039232.pth [2023-12-27 12:40:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001011000_258842624.pth [2023-12-27 12:40:07,449][20347] Updated weights for policy 0, policy_version 1012878 (0.0008) [2023-12-27 12:40:07,513][20347] Updated weights for policy 0, policy_version 1012888 (0.0008) [2023-12-27 12:40:07,577][20347] Updated weights for policy 0, policy_version 1012898 (0.0008) [2023-12-27 12:40:07,652][20381] Updated weights for policy 1, policy_version 1012225 (0.0007) [2023-12-27 12:40:07,707][20381] Updated weights for policy 1, policy_version 1012235 (0.0008) [2023-12-27 12:40:07,765][20381] Updated weights for policy 1, policy_version 1012245 (0.0006) [2023-12-27 12:40:08,259][20347] Updated weights for policy 0, policy_version 1012908 (0.0007) [2023-12-27 12:40:08,317][20347] Updated weights for policy 0, policy_version 1012918 (0.0008) [2023-12-27 12:40:08,388][20347] Updated weights for policy 0, policy_version 1012928 (0.0006) [2023-12-27 12:40:08,453][20381] Updated weights for policy 1, policy_version 1012255 (0.0007) [2023-12-27 12:40:08,512][20381] Updated weights for policy 1, policy_version 1012265 (0.0005) [2023-12-27 12:40:08,577][20381] Updated weights for policy 1, policy_version 1012275 (0.0005) [2023-12-27 12:40:09,118][20347] Updated weights for policy 0, policy_version 1012938 (0.0008) [2023-12-27 12:40:09,162][20381] Updated weights for policy 1, policy_version 1012285 (0.0006) [2023-12-27 12:40:09,175][20347] Updated weights for policy 0, policy_version 1012948 (0.0007) [2023-12-27 12:40:09,218][20381] Updated weights for policy 1, policy_version 1012295 (0.0006) [2023-12-27 12:40:09,240][20347] Updated weights for policy 0, policy_version 1012958 (0.0007) [2023-12-27 12:40:09,281][20381] Updated weights for policy 1, policy_version 1012305 (0.0008) [2023-12-27 12:40:09,300][20347] Updated weights for policy 0, policy_version 1012968 (0.0008) [2023-12-27 12:40:10,009][20381] Updated weights for policy 1, policy_version 1012315 (0.0008) [2023-12-27 12:40:10,064][20347] Updated weights for policy 0, policy_version 1012978 (0.0007) [2023-12-27 12:40:10,075][20381] Updated weights for policy 1, policy_version 1012325 (0.0008) [2023-12-27 12:40:10,119][20347] Updated weights for policy 0, policy_version 1012988 (0.0006) [2023-12-27 12:40:10,143][20381] Updated weights for policy 1, policy_version 1012335 (0.0008) [2023-12-27 12:40:10,182][20347] Updated weights for policy 0, policy_version 1012998 (0.0005) [2023-12-27 12:40:10,759][20381] Updated weights for policy 1, policy_version 1012345 (0.0008) [2023-12-27 12:40:10,798][20347] Updated weights for policy 0, policy_version 1013008 (0.0006) [2023-12-27 12:40:10,815][20381] Updated weights for policy 1, policy_version 1012355 (0.0006) [2023-12-27 12:40:10,849][20347] Updated weights for policy 0, policy_version 1013018 (0.0006) [2023-12-27 12:40:10,870][20381] Updated weights for policy 1, policy_version 1012365 (0.0006) [2023-12-27 12:40:10,893][20291] KL-divergence is very high: 111.5788 [2023-12-27 12:40:10,905][20347] Updated weights for policy 0, policy_version 1013028 (0.0005) [2023-12-27 12:40:10,923][20381] Updated weights for policy 1, policy_version 1012375 (0.0006) [2023-12-27 12:40:11,650][20347] Updated weights for policy 0, policy_version 1013038 (0.0008) [2023-12-27 12:40:11,681][20381] Updated weights for policy 1, policy_version 1012385 (0.0007) [2023-12-27 12:40:11,709][20347] Updated weights for policy 0, policy_version 1013048 (0.0007) [2023-12-27 12:40:11,745][20381] Updated weights for policy 1, policy_version 1012395 (0.0006) [2023-12-27 12:40:11,779][20347] Updated weights for policy 0, policy_version 1013058 (0.0007) [2023-12-27 12:40:11,809][20381] Updated weights for policy 1, policy_version 1012405 (0.0006) [2023-12-27 12:40:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 518594560. Throughput: 0: 10242.7, 1: 10448.9. Samples: 518588856. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:12,363][19170] Avg episode reward: [(0, '815.060'), (1, '172.680')] [2023-12-27 12:40:12,465][20347] Updated weights for policy 0, policy_version 1013068 (0.0008) [2023-12-27 12:40:12,511][20347] Updated weights for policy 0, policy_version 1013078 (0.0008) [2023-12-27 12:40:12,551][20381] Updated weights for policy 1, policy_version 1012415 (0.0006) [2023-12-27 12:40:12,567][20347] Updated weights for policy 0, policy_version 1013088 (0.0007) [2023-12-27 12:40:12,606][20381] Updated weights for policy 1, policy_version 1012425 (0.0006) [2023-12-27 12:40:12,665][20381] Updated weights for policy 1, policy_version 1012435 (0.0008) [2023-12-27 12:40:13,288][20347] Updated weights for policy 0, policy_version 1013098 (0.0006) [2023-12-27 12:40:13,301][20381] Updated weights for policy 1, policy_version 1012445 (0.0007) [2023-12-27 12:40:13,336][20347] Updated weights for policy 0, policy_version 1013108 (0.0006) [2023-12-27 12:40:13,353][20381] Updated weights for policy 1, policy_version 1012455 (0.0005) [2023-12-27 12:40:13,388][20347] Updated weights for policy 0, policy_version 1013118 (0.0005) [2023-12-27 12:40:13,408][20381] Updated weights for policy 1, policy_version 1012465 (0.0006) [2023-12-27 12:40:13,443][20347] Updated weights for policy 0, policy_version 1013128 (0.0005) [2023-12-27 12:40:14,014][20381] Updated weights for policy 1, policy_version 1012475 (0.0006) [2023-12-27 12:40:14,061][20347] Updated weights for policy 0, policy_version 1013138 (0.0005) [2023-12-27 12:40:14,070][20381] Updated weights for policy 1, policy_version 1012485 (0.0005) [2023-12-27 12:40:14,117][20347] Updated weights for policy 0, policy_version 1013148 (0.0007) [2023-12-27 12:40:14,126][20381] Updated weights for policy 1, policy_version 1012495 (0.0005) [2023-12-27 12:40:14,173][20347] Updated weights for policy 0, policy_version 1013158 (0.0008) [2023-12-27 12:40:14,770][20347] Updated weights for policy 0, policy_version 1013168 (0.0007) [2023-12-27 12:40:14,818][20381] Updated weights for policy 1, policy_version 1012505 (0.0009) [2023-12-27 12:40:14,829][20347] Updated weights for policy 0, policy_version 1013178 (0.0008) [2023-12-27 12:40:14,869][20381] Updated weights for policy 1, policy_version 1012515 (0.0007) [2023-12-27 12:40:14,885][20347] Updated weights for policy 0, policy_version 1013188 (0.0008) [2023-12-27 12:40:14,927][20381] Updated weights for policy 1, policy_version 1012525 (0.0006) [2023-12-27 12:40:14,981][20381] Updated weights for policy 1, policy_version 1012535 (0.0007) [2023-12-27 12:40:15,532][20347] Updated weights for policy 0, policy_version 1013198 (0.0008) [2023-12-27 12:40:15,586][20347] Updated weights for policy 0, policy_version 1013208 (0.0007) [2023-12-27 12:40:15,647][20347] Updated weights for policy 0, policy_version 1013218 (0.0005) [2023-12-27 12:40:15,711][20381] Updated weights for policy 1, policy_version 1012545 (0.0008) [2023-12-27 12:40:15,770][20381] Updated weights for policy 1, policy_version 1012555 (0.0006) [2023-12-27 12:40:15,825][20381] Updated weights for policy 1, policy_version 1012565 (0.0007) [2023-12-27 12:40:16,280][20347] Updated weights for policy 0, policy_version 1013228 (0.0007) [2023-12-27 12:40:16,330][20347] Updated weights for policy 0, policy_version 1013238 (0.0008) [2023-12-27 12:40:16,382][20347] Updated weights for policy 0, policy_version 1013248 (0.0010) [2023-12-27 12:40:16,529][20381] Updated weights for policy 1, policy_version 1012575 (0.0008) [2023-12-27 12:40:16,589][20381] Updated weights for policy 1, policy_version 1012585 (0.0008) [2023-12-27 12:40:16,634][20381] Updated weights for policy 1, policy_version 1012595 (0.0005) [2023-12-27 12:40:17,051][20347] Updated weights for policy 0, policy_version 1013258 (0.0009) [2023-12-27 12:40:17,103][20347] Updated weights for policy 0, policy_version 1013268 (0.0005) [2023-12-27 12:40:17,156][20347] Updated weights for policy 0, policy_version 1013278 (0.0005) [2023-12-27 12:40:17,208][20347] Updated weights for policy 0, policy_version 1013288 (0.0007) [2023-12-27 12:40:17,265][20381] Updated weights for policy 1, policy_version 1012605 (0.0006) [2023-12-27 12:40:17,310][20381] Updated weights for policy 1, policy_version 1012615 (0.0005) [2023-12-27 12:40:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 518701056. Throughput: 0: 10218.0, 1: 10356.6. Samples: 518712772. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:17,363][19170] Avg episode reward: [(0, '776.210'), (1, '332.020')] [2023-12-27 12:40:17,398][20381] Updated weights for policy 1, policy_version 1012625 (0.0005) [2023-12-27 12:40:17,786][20347] Updated weights for policy 0, policy_version 1013298 (0.0006) [2023-12-27 12:40:17,850][20347] Updated weights for policy 0, policy_version 1013308 (0.0006) [2023-12-27 12:40:17,896][20381] Updated weights for policy 1, policy_version 1012635 (0.0005) [2023-12-27 12:40:17,911][20347] Updated weights for policy 0, policy_version 1013318 (0.0007) [2023-12-27 12:40:17,949][20381] Updated weights for policy 1, policy_version 1012645 (0.0008) [2023-12-27 12:40:18,006][20381] Updated weights for policy 1, policy_version 1012655 (0.0008) [2023-12-27 12:40:18,468][20291] KL-divergence is very high: 108.2428 [2023-12-27 12:40:18,483][20291] KL-divergence is very high: 214.4554 [2023-12-27 12:40:18,493][20347] Updated weights for policy 0, policy_version 1013328 (0.0006) [2023-12-27 12:40:18,548][20347] Updated weights for policy 0, policy_version 1013338 (0.0006) [2023-12-27 12:40:18,601][20347] Updated weights for policy 0, policy_version 1013348 (0.0008) [2023-12-27 12:40:18,696][20381] Updated weights for policy 1, policy_version 1012665 (0.0008) [2023-12-27 12:40:18,748][20381] Updated weights for policy 1, policy_version 1012675 (0.0007) [2023-12-27 12:40:18,802][20381] Updated weights for policy 1, policy_version 1012685 (0.0007) [2023-12-27 12:40:18,866][20381] Updated weights for policy 1, policy_version 1012695 (0.0009) [2023-12-27 12:40:19,293][20347] Updated weights for policy 0, policy_version 1013358 (0.0008) [2023-12-27 12:40:19,353][20347] Updated weights for policy 0, policy_version 1013368 (0.0008) [2023-12-27 12:40:19,416][20347] Updated weights for policy 0, policy_version 1013378 (0.0007) [2023-12-27 12:40:19,584][20381] Updated weights for policy 1, policy_version 1012705 (0.0008) [2023-12-27 12:40:19,646][20381] Updated weights for policy 1, policy_version 1012715 (0.0007) [2023-12-27 12:40:19,705][20381] Updated weights for policy 1, policy_version 1012725 (0.0008) [2023-12-27 12:40:20,142][20347] Updated weights for policy 0, policy_version 1013388 (0.0008) [2023-12-27 12:40:20,195][20347] Updated weights for policy 0, policy_version 1013398 (0.0008) [2023-12-27 12:40:20,249][20347] Updated weights for policy 0, policy_version 1013408 (0.0007) [2023-12-27 12:40:20,424][20381] Updated weights for policy 1, policy_version 1012735 (0.0008) [2023-12-27 12:40:20,485][20381] Updated weights for policy 1, policy_version 1012745 (0.0008) [2023-12-27 12:40:20,532][20381] Updated weights for policy 1, policy_version 1012755 (0.0008) [2023-12-27 12:40:20,944][20347] Updated weights for policy 0, policy_version 1013418 (0.0007) [2023-12-27 12:40:20,998][20347] Updated weights for policy 0, policy_version 1013428 (0.0008) [2023-12-27 12:40:21,066][20347] Updated weights for policy 0, policy_version 1013438 (0.0008) [2023-12-27 12:40:21,134][20347] Updated weights for policy 0, policy_version 1013448 (0.0008) [2023-12-27 12:40:21,302][20381] Updated weights for policy 1, policy_version 1012765 (0.0008) [2023-12-27 12:40:21,369][20381] Updated weights for policy 1, policy_version 1012775 (0.0007) [2023-12-27 12:40:21,432][20381] Updated weights for policy 1, policy_version 1012785 (0.0008) [2023-12-27 12:40:21,826][20347] Updated weights for policy 0, policy_version 1013458 (0.0008) [2023-12-27 12:40:21,876][20347] Updated weights for policy 0, policy_version 1013468 (0.0008) [2023-12-27 12:40:21,927][20347] Updated weights for policy 0, policy_version 1013478 (0.0008) [2023-12-27 12:40:22,185][20381] Updated weights for policy 1, policy_version 1012795 (0.0009) [2023-12-27 12:40:22,244][20381] Updated weights for policy 1, policy_version 1012805 (0.0007) [2023-12-27 12:40:22,307][20381] Updated weights for policy 1, policy_version 1012815 (0.0007) [2023-12-27 12:40:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 518799360. Throughput: 0: 10244.0, 1: 10329.3. Samples: 518775740. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:22,363][19170] Avg episode reward: [(0, '815.290'), (1, '419.230')] [2023-12-27 12:40:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001012824_259309568.pth... [2023-12-27 12:40:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001013480_259497984.pth... [2023-12-27 12:40:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001012296_259194880.pth [2023-12-27 12:40:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001011608_258998272.pth [2023-12-27 12:40:22,691][20347] Updated weights for policy 0, policy_version 1013488 (0.0008) [2023-12-27 12:40:22,757][20347] Updated weights for policy 0, policy_version 1013498 (0.0008) [2023-12-27 12:40:22,812][20347] Updated weights for policy 0, policy_version 1013508 (0.0008) [2023-12-27 12:40:22,933][20381] Updated weights for policy 1, policy_version 1012825 (0.0008) [2023-12-27 12:40:22,992][20381] Updated weights for policy 1, policy_version 1012835 (0.0008) [2023-12-27 12:40:23,048][20381] Updated weights for policy 1, policy_version 1012845 (0.0008) [2023-12-27 12:40:23,107][20381] Updated weights for policy 1, policy_version 1012855 (0.0008) [2023-12-27 12:40:23,529][20347] Updated weights for policy 0, policy_version 1013518 (0.0008) [2023-12-27 12:40:23,592][20347] Updated weights for policy 0, policy_version 1013528 (0.0008) [2023-12-27 12:40:23,656][20347] Updated weights for policy 0, policy_version 1013538 (0.0008) [2023-12-27 12:40:23,748][20381] Updated weights for policy 1, policy_version 1012865 (0.0006) [2023-12-27 12:40:23,806][20381] Updated weights for policy 1, policy_version 1012875 (0.0008) [2023-12-27 12:40:23,869][20381] Updated weights for policy 1, policy_version 1012885 (0.0009) [2023-12-27 12:40:24,252][20347] Updated weights for policy 0, policy_version 1013548 (0.0008) [2023-12-27 12:40:24,318][20347] Updated weights for policy 0, policy_version 1013558 (0.0009) [2023-12-27 12:40:24,368][20347] Updated weights for policy 0, policy_version 1013568 (0.0009) [2023-12-27 12:40:24,480][20381] Updated weights for policy 1, policy_version 1012895 (0.0009) [2023-12-27 12:40:24,543][20381] Updated weights for policy 1, policy_version 1012905 (0.0007) [2023-12-27 12:40:24,610][20381] Updated weights for policy 1, policy_version 1012915 (0.0010) [2023-12-27 12:40:24,990][20347] Updated weights for policy 0, policy_version 1013578 (0.0008) [2023-12-27 12:40:25,048][20347] Updated weights for policy 0, policy_version 1013588 (0.0007) [2023-12-27 12:40:25,110][20347] Updated weights for policy 0, policy_version 1013598 (0.0008) [2023-12-27 12:40:25,164][20347] Updated weights for policy 0, policy_version 1013608 (0.0005) [2023-12-27 12:40:25,258][20381] Updated weights for policy 1, policy_version 1012925 (0.0011) [2023-12-27 12:40:25,314][20381] Updated weights for policy 1, policy_version 1012935 (0.0011) [2023-12-27 12:40:25,371][20381] Updated weights for policy 1, policy_version 1012945 (0.0011) [2023-12-27 12:40:25,750][20347] Updated weights for policy 0, policy_version 1013618 (0.0006) [2023-12-27 12:40:25,789][20291] KL-divergence is very high: 117.2073 [2023-12-27 12:40:25,804][20291] KL-divergence is very high: 128.7759 [2023-12-27 12:40:25,805][20347] Updated weights for policy 0, policy_version 1013628 (0.0008) [2023-12-27 12:40:25,860][20347] Updated weights for policy 0, policy_version 1013638 (0.0008) [2023-12-27 12:40:25,973][20381] Updated weights for policy 1, policy_version 1012955 (0.0009) [2023-12-27 12:40:26,026][20381] Updated weights for policy 1, policy_version 1012965 (0.0007) [2023-12-27 12:40:26,077][20381] Updated weights for policy 1, policy_version 1012975 (0.0008) [2023-12-27 12:40:26,478][20347] Updated weights for policy 0, policy_version 1013648 (0.0010) [2023-12-27 12:40:26,527][20347] Updated weights for policy 0, policy_version 1013658 (0.0006) [2023-12-27 12:40:26,580][20347] Updated weights for policy 0, policy_version 1013668 (0.0005) [2023-12-27 12:40:26,752][20381] Updated weights for policy 1, policy_version 1012985 (0.0009) [2023-12-27 12:40:26,802][20381] Updated weights for policy 1, policy_version 1012995 (0.0005) [2023-12-27 12:40:26,859][20381] Updated weights for policy 1, policy_version 1013005 (0.0007) [2023-12-27 12:40:26,914][20381] Updated weights for policy 1, policy_version 1013015 (0.0006) [2023-12-27 12:40:27,138][20347] Updated weights for policy 0, policy_version 1013678 (0.0005) [2023-12-27 12:40:27,183][20347] Updated weights for policy 0, policy_version 1013688 (0.0005) [2023-12-27 12:40:27,229][20347] Updated weights for policy 0, policy_version 1013698 (0.0005) [2023-12-27 12:40:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.2, 300 sec: 20521.7). Total num frames: 518914048. Throughput: 0: 10238.2, 1: 10270.1. Samples: 518898680. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:27,363][19170] Avg episode reward: [(0, '864.750'), (1, '257.080')] [2023-12-27 12:40:27,630][20381] Updated weights for policy 1, policy_version 1013025 (0.0009) [2023-12-27 12:40:27,687][20381] Updated weights for policy 1, policy_version 1013035 (0.0009) [2023-12-27 12:40:27,745][20381] Updated weights for policy 1, policy_version 1013045 (0.0008) [2023-12-27 12:40:27,780][20347] Updated weights for policy 0, policy_version 1013708 (0.0005) [2023-12-27 12:40:27,837][20347] Updated weights for policy 0, policy_version 1013718 (0.0005) [2023-12-27 12:40:27,886][20347] Updated weights for policy 0, policy_version 1013728 (0.0005) [2023-12-27 12:40:28,383][20381] Updated weights for policy 1, policy_version 1013055 (0.0009) [2023-12-27 12:40:28,438][20381] Updated weights for policy 1, policy_version 1013065 (0.0008) [2023-12-27 12:40:28,489][20381] Updated weights for policy 1, policy_version 1013075 (0.0008) [2023-12-27 12:40:28,559][20347] Updated weights for policy 0, policy_version 1013738 (0.0005) [2023-12-27 12:40:28,610][20347] Updated weights for policy 0, policy_version 1013748 (0.0006) [2023-12-27 12:40:28,665][20347] Updated weights for policy 0, policy_version 1013758 (0.0008) [2023-12-27 12:40:28,722][20347] Updated weights for policy 0, policy_version 1013768 (0.0007) [2023-12-27 12:40:29,199][20381] Updated weights for policy 1, policy_version 1013085 (0.0008) [2023-12-27 12:40:29,276][20381] Updated weights for policy 1, policy_version 1013095 (0.0007) [2023-12-27 12:40:29,340][20381] Updated weights for policy 1, policy_version 1013105 (0.0007) [2023-12-27 12:40:29,418][20347] Updated weights for policy 0, policy_version 1013778 (0.0008) [2023-12-27 12:40:29,468][20347] Updated weights for policy 0, policy_version 1013788 (0.0008) [2023-12-27 12:40:29,513][20347] Updated weights for policy 0, policy_version 1013798 (0.0007) [2023-12-27 12:40:30,092][20381] Updated weights for policy 1, policy_version 1013115 (0.0007) [2023-12-27 12:40:30,149][20347] Updated weights for policy 0, policy_version 1013808 (0.0007) [2023-12-27 12:40:30,154][20381] Updated weights for policy 1, policy_version 1013125 (0.0008) [2023-12-27 12:40:30,205][20347] Updated weights for policy 0, policy_version 1013818 (0.0006) [2023-12-27 12:40:30,217][20381] Updated weights for policy 1, policy_version 1013135 (0.0006) [2023-12-27 12:40:30,263][20347] Updated weights for policy 0, policy_version 1013828 (0.0007) [2023-12-27 12:40:30,891][20347] Updated weights for policy 0, policy_version 1013838 (0.0008) [2023-12-27 12:40:30,893][20381] Updated weights for policy 1, policy_version 1013145 (0.0005) [2023-12-27 12:40:30,940][20381] Updated weights for policy 1, policy_version 1013155 (0.0006) [2023-12-27 12:40:30,946][20347] Updated weights for policy 0, policy_version 1013848 (0.0006) [2023-12-27 12:40:30,994][20381] Updated weights for policy 1, policy_version 1013165 (0.0006) [2023-12-27 12:40:31,002][20347] Updated weights for policy 0, policy_version 1013858 (0.0006) [2023-12-27 12:40:31,049][20381] Updated weights for policy 1, policy_version 1013175 (0.0006) [2023-12-27 12:40:31,740][20381] Updated weights for policy 1, policy_version 1013185 (0.0007) [2023-12-27 12:40:31,754][20347] Updated weights for policy 0, policy_version 1013868 (0.0007) [2023-12-27 12:40:31,808][20381] Updated weights for policy 1, policy_version 1013195 (0.0007) [2023-12-27 12:40:31,812][20347] Updated weights for policy 0, policy_version 1013878 (0.0005) [2023-12-27 12:40:31,864][20381] Updated weights for policy 1, policy_version 1013205 (0.0008) [2023-12-27 12:40:31,872][20347] Updated weights for policy 0, policy_version 1013888 (0.0005) [2023-12-27 12:40:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 519012352. Throughput: 0: 10348.2, 1: 10250.2. Samples: 519024444. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:32,363][19170] Avg episode reward: [(0, '853.580'), (1, '55.500')] [2023-12-27 12:40:32,541][20347] Updated weights for policy 0, policy_version 1013898 (0.0006) [2023-12-27 12:40:32,569][20381] Updated weights for policy 1, policy_version 1013215 (0.0005) [2023-12-27 12:40:32,597][20347] Updated weights for policy 0, policy_version 1013908 (0.0005) [2023-12-27 12:40:32,632][20381] Updated weights for policy 1, policy_version 1013225 (0.0006) [2023-12-27 12:40:32,652][20347] Updated weights for policy 0, policy_version 1013918 (0.0006) [2023-12-27 12:40:32,690][20381] Updated weights for policy 1, policy_version 1013235 (0.0008) [2023-12-27 12:40:32,706][20347] Updated weights for policy 0, policy_version 1013928 (0.0006) [2023-12-27 12:40:33,295][20347] Updated weights for policy 0, policy_version 1013938 (0.0007) [2023-12-27 12:40:33,354][20347] Updated weights for policy 0, policy_version 1013948 (0.0007) [2023-12-27 12:40:33,382][20381] Updated weights for policy 1, policy_version 1013245 (0.0007) [2023-12-27 12:40:33,402][20347] Updated weights for policy 0, policy_version 1013958 (0.0006) [2023-12-27 12:40:33,431][20381] Updated weights for policy 1, policy_version 1013255 (0.0006) [2023-12-27 12:40:33,493][20381] Updated weights for policy 1, policy_version 1013265 (0.0009) [2023-12-27 12:40:34,076][20347] Updated weights for policy 0, policy_version 1013968 (0.0010) [2023-12-27 12:40:34,122][20381] Updated weights for policy 1, policy_version 1013275 (0.0007) [2023-12-27 12:40:34,132][20347] Updated weights for policy 0, policy_version 1013978 (0.0010) [2023-12-27 12:40:34,185][20381] Updated weights for policy 1, policy_version 1013285 (0.0006) [2023-12-27 12:40:34,193][20347] Updated weights for policy 0, policy_version 1013988 (0.0011) [2023-12-27 12:40:34,249][20381] Updated weights for policy 1, policy_version 1013295 (0.0007) [2023-12-27 12:40:34,907][20347] Updated weights for policy 0, policy_version 1013998 (0.0008) [2023-12-27 12:40:34,907][20381] Updated weights for policy 1, policy_version 1013305 (0.0008) [2023-12-27 12:40:34,960][20347] Updated weights for policy 0, policy_version 1014008 (0.0007) [2023-12-27 12:40:34,963][20381] Updated weights for policy 1, policy_version 1013315 (0.0006) [2023-12-27 12:40:35,009][20381] Updated weights for policy 1, policy_version 1013325 (0.0010) [2023-12-27 12:40:35,011][20347] Updated weights for policy 0, policy_version 1014018 (0.0005) [2023-12-27 12:40:35,057][20381] Updated weights for policy 1, policy_version 1013335 (0.0010) [2023-12-27 12:40:35,663][20347] Updated weights for policy 0, policy_version 1014028 (0.0006) [2023-12-27 12:40:35,702][20381] Updated weights for policy 1, policy_version 1013345 (0.0007) [2023-12-27 12:40:35,716][20347] Updated weights for policy 0, policy_version 1014038 (0.0006) [2023-12-27 12:40:35,761][20381] Updated weights for policy 1, policy_version 1013355 (0.0009) [2023-12-27 12:40:35,775][20347] Updated weights for policy 0, policy_version 1014048 (0.0008) [2023-12-27 12:40:35,814][20381] Updated weights for policy 1, policy_version 1013365 (0.0008) [2023-12-27 12:40:36,411][20347] Updated weights for policy 0, policy_version 1014058 (0.0007) [2023-12-27 12:40:36,476][20347] Updated weights for policy 0, policy_version 1014068 (0.0006) [2023-12-27 12:40:36,493][20381] Updated weights for policy 1, policy_version 1013375 (0.0007) [2023-12-27 12:40:36,536][20347] Updated weights for policy 0, policy_version 1014078 (0.0006) [2023-12-27 12:40:36,553][20381] Updated weights for policy 1, policy_version 1013385 (0.0006) [2023-12-27 12:40:36,600][20347] Updated weights for policy 0, policy_version 1014088 (0.0006) [2023-12-27 12:40:36,610][20381] Updated weights for policy 1, policy_version 1013395 (0.0006) [2023-12-27 12:40:37,188][20347] Updated weights for policy 0, policy_version 1014098 (0.0008) [2023-12-27 12:40:37,232][20347] Updated weights for policy 0, policy_version 1014108 (0.0007) [2023-12-27 12:40:37,274][20381] Updated weights for policy 1, policy_version 1013405 (0.0007) [2023-12-27 12:40:37,283][20347] Updated weights for policy 0, policy_version 1014118 (0.0008) [2023-12-27 12:40:37,325][20381] Updated weights for policy 1, policy_version 1013415 (0.0006) [2023-12-27 12:40:37,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 519118848. Throughput: 0: 10398.5, 1: 10232.0. Samples: 519086824. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:37,364][19170] Avg episode reward: [(0, '873.350'), (1, '302.210')] [2023-12-27 12:40:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001014120_259661824.pth... [2023-12-27 12:40:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001012872_259342336.pth [2023-12-27 12:40:37,381][20381] Updated weights for policy 1, policy_version 1013425 (0.0008) [2023-12-27 12:40:37,412][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001013432_259465216.pth... [2023-12-27 12:40:37,415][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001012216_259153920.pth [2023-12-27 12:40:37,998][20381] Updated weights for policy 1, policy_version 1013435 (0.0007) [2023-12-27 12:40:38,034][20347] Updated weights for policy 0, policy_version 1014128 (0.0006) [2023-12-27 12:40:38,057][20381] Updated weights for policy 1, policy_version 1013445 (0.0011) [2023-12-27 12:40:38,095][20347] Updated weights for policy 0, policy_version 1014138 (0.0007) [2023-12-27 12:40:38,113][20381] Updated weights for policy 1, policy_version 1013455 (0.0010) [2023-12-27 12:40:38,155][20347] Updated weights for policy 0, policy_version 1014148 (0.0007) [2023-12-27 12:40:38,757][20381] Updated weights for policy 1, policy_version 1013465 (0.0010) [2023-12-27 12:40:38,820][20381] Updated weights for policy 1, policy_version 1013475 (0.0006) [2023-12-27 12:40:38,862][20347] Updated weights for policy 0, policy_version 1014158 (0.0007) [2023-12-27 12:40:38,879][20381] Updated weights for policy 1, policy_version 1013485 (0.0006) [2023-12-27 12:40:38,927][20347] Updated weights for policy 0, policy_version 1014168 (0.0005) [2023-12-27 12:40:38,935][20381] Updated weights for policy 1, policy_version 1013495 (0.0007) [2023-12-27 12:40:38,991][20347] Updated weights for policy 0, policy_version 1014178 (0.0006) [2023-12-27 12:40:39,578][20381] Updated weights for policy 1, policy_version 1013505 (0.0008) [2023-12-27 12:40:39,631][20347] Updated weights for policy 0, policy_version 1014188 (0.0007) [2023-12-27 12:40:39,633][20381] Updated weights for policy 1, policy_version 1013515 (0.0008) [2023-12-27 12:40:39,692][20381] Updated weights for policy 1, policy_version 1013525 (0.0008) [2023-12-27 12:40:39,695][20347] Updated weights for policy 0, policy_version 1014198 (0.0008) [2023-12-27 12:40:39,752][20347] Updated weights for policy 0, policy_version 1014208 (0.0007) [2023-12-27 12:40:40,435][20381] Updated weights for policy 1, policy_version 1013535 (0.0008) [2023-12-27 12:40:40,476][20347] Updated weights for policy 0, policy_version 1014218 (0.0007) [2023-12-27 12:40:40,494][20381] Updated weights for policy 1, policy_version 1013545 (0.0006) [2023-12-27 12:40:40,533][20347] Updated weights for policy 0, policy_version 1014228 (0.0007) [2023-12-27 12:40:40,554][20381] Updated weights for policy 1, policy_version 1013555 (0.0006) [2023-12-27 12:40:40,595][20347] Updated weights for policy 0, policy_version 1014238 (0.0006) [2023-12-27 12:40:40,644][20347] Updated weights for policy 0, policy_version 1014248 (0.0008) [2023-12-27 12:40:41,262][20381] Updated weights for policy 1, policy_version 1013565 (0.0007) [2023-12-27 12:40:41,336][20381] Updated weights for policy 1, policy_version 1013575 (0.0008) [2023-12-27 12:40:41,382][20347] Updated weights for policy 0, policy_version 1014258 (0.0008) [2023-12-27 12:40:41,400][20381] Updated weights for policy 1, policy_version 1013585 (0.0007) [2023-12-27 12:40:41,470][20347] Updated weights for policy 0, policy_version 1014268 (0.0008) [2023-12-27 12:40:41,532][20347] Updated weights for policy 0, policy_version 1014278 (0.0008) [2023-12-27 12:40:42,093][20381] Updated weights for policy 1, policy_version 1013595 (0.0008) [2023-12-27 12:40:42,144][20381] Updated weights for policy 1, policy_version 1013605 (0.0007) [2023-12-27 12:40:42,189][20347] Updated weights for policy 0, policy_version 1014288 (0.0007) [2023-12-27 12:40:42,199][20381] Updated weights for policy 1, policy_version 1013615 (0.0009) [2023-12-27 12:40:42,247][20347] Updated weights for policy 0, policy_version 1014298 (0.0006) [2023-12-27 12:40:42,309][20347] Updated weights for policy 0, policy_version 1014308 (0.0008) [2023-12-27 12:40:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 519225344. Throughput: 0: 10368.2, 1: 10244.8. Samples: 519210388. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:42,363][19170] Avg episode reward: [(0, '893.030'), (1, '248.170')] [2023-12-27 12:40:42,880][20381] Updated weights for policy 1, policy_version 1013625 (0.0010) [2023-12-27 12:40:42,943][20381] Updated weights for policy 1, policy_version 1013635 (0.0007) [2023-12-27 12:40:42,992][20347] Updated weights for policy 0, policy_version 1014318 (0.0008) [2023-12-27 12:40:43,005][20381] Updated weights for policy 1, policy_version 1013645 (0.0006) [2023-12-27 12:40:43,040][20347] Updated weights for policy 0, policy_version 1014328 (0.0005) [2023-12-27 12:40:43,061][20381] Updated weights for policy 1, policy_version 1013655 (0.0006) [2023-12-27 12:40:43,096][20347] Updated weights for policy 0, policy_version 1014338 (0.0005) [2023-12-27 12:40:43,595][20381] Updated weights for policy 1, policy_version 1013665 (0.0006) [2023-12-27 12:40:43,639][20381] Updated weights for policy 1, policy_version 1013675 (0.0005) [2023-12-27 12:40:43,684][20381] Updated weights for policy 1, policy_version 1013685 (0.0005) [2023-12-27 12:40:43,780][20347] Updated weights for policy 0, policy_version 1014348 (0.0007) [2023-12-27 12:40:43,832][20347] Updated weights for policy 0, policy_version 1014358 (0.0010) [2023-12-27 12:40:43,880][20347] Updated weights for policy 0, policy_version 1014368 (0.0010) [2023-12-27 12:40:44,340][20381] Updated weights for policy 1, policy_version 1013695 (0.0007) [2023-12-27 12:40:44,396][20381] Updated weights for policy 1, policy_version 1013705 (0.0008) [2023-12-27 12:40:44,446][20381] Updated weights for policy 1, policy_version 1013715 (0.0007) [2023-12-27 12:40:44,582][20347] Updated weights for policy 0, policy_version 1014378 (0.0009) [2023-12-27 12:40:44,640][20347] Updated weights for policy 0, policy_version 1014388 (0.0006) [2023-12-27 12:40:44,696][20347] Updated weights for policy 0, policy_version 1014398 (0.0006) [2023-12-27 12:40:44,754][20347] Updated weights for policy 0, policy_version 1014408 (0.0008) [2023-12-27 12:40:45,065][20381] Updated weights for policy 1, policy_version 1013725 (0.0006) [2023-12-27 12:40:45,127][20381] Updated weights for policy 1, policy_version 1013735 (0.0008) [2023-12-27 12:40:45,190][20381] Updated weights for policy 1, policy_version 1013745 (0.0008) [2023-12-27 12:40:45,450][20347] Updated weights for policy 0, policy_version 1014418 (0.0008) [2023-12-27 12:40:45,505][20347] Updated weights for policy 0, policy_version 1014428 (0.0007) [2023-12-27 12:40:45,558][20347] Updated weights for policy 0, policy_version 1014438 (0.0008) [2023-12-27 12:40:45,807][20381] Updated weights for policy 1, policy_version 1013755 (0.0007) [2023-12-27 12:40:45,865][20381] Updated weights for policy 1, policy_version 1013765 (0.0005) [2023-12-27 12:40:45,921][20381] Updated weights for policy 1, policy_version 1013775 (0.0005) [2023-12-27 12:40:46,234][20347] Updated weights for policy 0, policy_version 1014448 (0.0007) [2023-12-27 12:40:46,288][20347] Updated weights for policy 0, policy_version 1014458 (0.0008) [2023-12-27 12:40:46,347][20347] Updated weights for policy 0, policy_version 1014468 (0.0008) [2023-12-27 12:40:46,482][20381] Updated weights for policy 1, policy_version 1013785 (0.0005) [2023-12-27 12:40:46,543][20381] Updated weights for policy 1, policy_version 1013795 (0.0007) [2023-12-27 12:40:46,603][20381] Updated weights for policy 1, policy_version 1013805 (0.0007) [2023-12-27 12:40:46,665][20381] Updated weights for policy 1, policy_version 1013815 (0.0008) [2023-12-27 12:40:47,042][20347] Updated weights for policy 0, policy_version 1014478 (0.0007) [2023-12-27 12:40:47,096][20347] Updated weights for policy 0, policy_version 1014488 (0.0008) [2023-12-27 12:40:47,139][20347] Updated weights for policy 0, policy_version 1014498 (0.0008) [2023-12-27 12:40:47,304][20381] Updated weights for policy 1, policy_version 1013825 (0.0007) [2023-12-27 12:40:47,355][20381] Updated weights for policy 1, policy_version 1013835 (0.0007) [2023-12-27 12:40:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 519323648. Throughput: 0: 10341.4, 1: 10333.5. Samples: 519335716. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:47,363][19170] Avg episode reward: [(0, '863.860'), (1, '290.180')] [2023-12-27 12:40:47,404][20381] Updated weights for policy 1, policy_version 1013845 (0.0008) [2023-12-27 12:40:47,889][20347] Updated weights for policy 0, policy_version 1014508 (0.0007) [2023-12-27 12:40:47,946][20347] Updated weights for policy 0, policy_version 1014518 (0.0009) [2023-12-27 12:40:48,002][20347] Updated weights for policy 0, policy_version 1014528 (0.0009) [2023-12-27 12:40:48,016][20381] Updated weights for policy 1, policy_version 1013855 (0.0006) [2023-12-27 12:40:48,072][20381] Updated weights for policy 1, policy_version 1013865 (0.0010) [2023-12-27 12:40:48,127][20381] Updated weights for policy 1, policy_version 1013875 (0.0011) [2023-12-27 12:40:48,737][20381] Updated weights for policy 1, policy_version 1013885 (0.0011) [2023-12-27 12:40:48,769][20347] Updated weights for policy 0, policy_version 1014538 (0.0007) [2023-12-27 12:40:48,799][20381] Updated weights for policy 1, policy_version 1013895 (0.0011) [2023-12-27 12:40:48,822][20347] Updated weights for policy 0, policy_version 1014548 (0.0010) [2023-12-27 12:40:48,851][20381] Updated weights for policy 1, policy_version 1013905 (0.0011) [2023-12-27 12:40:48,880][20347] Updated weights for policy 0, policy_version 1014558 (0.0007) [2023-12-27 12:40:48,939][20347] Updated weights for policy 0, policy_version 1014568 (0.0007) [2023-12-27 12:40:49,549][20381] Updated weights for policy 1, policy_version 1013915 (0.0011) [2023-12-27 12:40:49,559][20347] Updated weights for policy 0, policy_version 1014578 (0.0008) [2023-12-27 12:40:49,604][20381] Updated weights for policy 1, policy_version 1013925 (0.0011) [2023-12-27 12:40:49,617][20347] Updated weights for policy 0, policy_version 1014588 (0.0007) [2023-12-27 12:40:49,663][20381] Updated weights for policy 1, policy_version 1013935 (0.0011) [2023-12-27 12:40:49,669][20347] Updated weights for policy 0, policy_version 1014598 (0.0006) [2023-12-27 12:40:50,362][20381] Updated weights for policy 1, policy_version 1013945 (0.0011) [2023-12-27 12:40:50,396][20347] Updated weights for policy 0, policy_version 1014608 (0.0007) [2023-12-27 12:40:50,414][20381] Updated weights for policy 1, policy_version 1013955 (0.0010) [2023-12-27 12:40:50,455][20347] Updated weights for policy 0, policy_version 1014618 (0.0006) [2023-12-27 12:40:50,472][20381] Updated weights for policy 1, policy_version 1013965 (0.0010) [2023-12-27 12:40:50,513][20347] Updated weights for policy 0, policy_version 1014628 (0.0005) [2023-12-27 12:40:50,526][20381] Updated weights for policy 1, policy_version 1013975 (0.0010) [2023-12-27 12:40:51,127][20347] Updated weights for policy 0, policy_version 1014638 (0.0007) [2023-12-27 12:40:51,183][20347] Updated weights for policy 0, policy_version 1014648 (0.0008) [2023-12-27 12:40:51,214][20381] Updated weights for policy 1, policy_version 1013985 (0.0008) [2023-12-27 12:40:51,235][20347] Updated weights for policy 0, policy_version 1014658 (0.0008) [2023-12-27 12:40:51,273][20381] Updated weights for policy 1, policy_version 1013995 (0.0008) [2023-12-27 12:40:51,334][20381] Updated weights for policy 1, policy_version 1014005 (0.0007) [2023-12-27 12:40:52,015][20347] Updated weights for policy 0, policy_version 1014668 (0.0008) [2023-12-27 12:40:52,074][20347] Updated weights for policy 0, policy_version 1014678 (0.0007) [2023-12-27 12:40:52,091][20381] Updated weights for policy 1, policy_version 1014015 (0.0007) [2023-12-27 12:40:52,131][20347] Updated weights for policy 0, policy_version 1014688 (0.0006) [2023-12-27 12:40:52,148][20381] Updated weights for policy 1, policy_version 1014025 (0.0006) [2023-12-27 12:40:52,208][20381] Updated weights for policy 1, policy_version 1014035 (0.0007) [2023-12-27 12:40:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 519430144. Throughput: 0: 10302.9, 1: 10352.6. Samples: 519396408. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:52,363][19170] Avg episode reward: [(0, '853.930'), (1, '265.800')] [2023-12-27 12:40:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001014040_259620864.pth... [2023-12-27 12:40:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001014696_259809280.pth... [2023-12-27 12:40:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001012824_259309568.pth [2023-12-27 12:40:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001013480_259497984.pth [2023-12-27 12:40:52,836][20381] Updated weights for policy 1, policy_version 1014045 (0.0008) [2023-12-27 12:40:52,860][20347] Updated weights for policy 0, policy_version 1014698 (0.0005) [2023-12-27 12:40:52,895][20381] Updated weights for policy 1, policy_version 1014055 (0.0007) [2023-12-27 12:40:52,914][20347] Updated weights for policy 0, policy_version 1014708 (0.0006) [2023-12-27 12:40:52,938][20381] Updated weights for policy 1, policy_version 1014065 (0.0008) [2023-12-27 12:40:52,960][20347] Updated weights for policy 0, policy_version 1014718 (0.0006) [2023-12-27 12:40:53,004][20347] Updated weights for policy 0, policy_version 1014728 (0.0007) [2023-12-27 12:40:53,674][20347] Updated weights for policy 0, policy_version 1014738 (0.0005) [2023-12-27 12:40:53,678][20381] Updated weights for policy 1, policy_version 1014075 (0.0006) [2023-12-27 12:40:53,729][20347] Updated weights for policy 0, policy_version 1014748 (0.0005) [2023-12-27 12:40:53,738][20381] Updated weights for policy 1, policy_version 1014085 (0.0008) [2023-12-27 12:40:53,793][20347] Updated weights for policy 0, policy_version 1014758 (0.0009) [2023-12-27 12:40:53,795][20381] Updated weights for policy 1, policy_version 1014095 (0.0007) [2023-12-27 12:40:54,382][20347] Updated weights for policy 0, policy_version 1014768 (0.0009) [2023-12-27 12:40:54,438][20347] Updated weights for policy 0, policy_version 1014778 (0.0007) [2023-12-27 12:40:54,452][20381] Updated weights for policy 1, policy_version 1014105 (0.0006) [2023-12-27 12:40:54,501][20347] Updated weights for policy 0, policy_version 1014788 (0.0008) [2023-12-27 12:40:54,503][20381] Updated weights for policy 1, policy_version 1014115 (0.0005) [2023-12-27 12:40:54,550][20381] Updated weights for policy 1, policy_version 1014125 (0.0006) [2023-12-27 12:40:54,595][20381] Updated weights for policy 1, policy_version 1014135 (0.0007) [2023-12-27 12:40:55,135][20347] Updated weights for policy 0, policy_version 1014798 (0.0010) [2023-12-27 12:40:55,194][20347] Updated weights for policy 0, policy_version 1014808 (0.0011) [2023-12-27 12:40:55,244][20347] Updated weights for policy 0, policy_version 1014818 (0.0010) [2023-12-27 12:40:55,282][20381] Updated weights for policy 1, policy_version 1014145 (0.0009) [2023-12-27 12:40:55,341][20381] Updated weights for policy 1, policy_version 1014155 (0.0008) [2023-12-27 12:40:55,396][20381] Updated weights for policy 1, policy_version 1014165 (0.0007) [2023-12-27 12:40:55,895][20347] Updated weights for policy 0, policy_version 1014828 (0.0008) [2023-12-27 12:40:55,951][20347] Updated weights for policy 0, policy_version 1014838 (0.0007) [2023-12-27 12:40:56,005][20347] Updated weights for policy 0, policy_version 1014848 (0.0009) [2023-12-27 12:40:56,012][20381] Updated weights for policy 1, policy_version 1014175 (0.0007) [2023-12-27 12:40:56,066][20381] Updated weights for policy 1, policy_version 1014185 (0.0005) [2023-12-27 12:40:56,121][20381] Updated weights for policy 1, policy_version 1014195 (0.0005) [2023-12-27 12:40:56,648][20347] Updated weights for policy 0, policy_version 1014858 (0.0009) [2023-12-27 12:40:56,703][20347] Updated weights for policy 0, policy_version 1014868 (0.0008) [2023-12-27 12:40:56,726][20381] Updated weights for policy 1, policy_version 1014205 (0.0005) [2023-12-27 12:40:56,751][20347] Updated weights for policy 0, policy_version 1014878 (0.0010) [2023-12-27 12:40:56,783][20381] Updated weights for policy 1, policy_version 1014215 (0.0010) [2023-12-27 12:40:56,807][20347] Updated weights for policy 0, policy_version 1014888 (0.0010) [2023-12-27 12:40:56,836][20381] Updated weights for policy 1, policy_version 1014225 (0.0011) [2023-12-27 12:40:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 519528448. Throughput: 0: 10353.8, 1: 10329.1. Samples: 519519584. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:40:57,363][19170] Avg episode reward: [(0, '902.530'), (1, '308.610')] [2023-12-27 12:40:57,464][20347] Updated weights for policy 0, policy_version 1014898 (0.0007) [2023-12-27 12:40:57,501][20381] Updated weights for policy 1, policy_version 1014235 (0.0010) [2023-12-27 12:40:57,521][20347] Updated weights for policy 0, policy_version 1014908 (0.0007) [2023-12-27 12:40:57,555][20381] Updated weights for policy 1, policy_version 1014245 (0.0010) [2023-12-27 12:40:57,573][20347] Updated weights for policy 0, policy_version 1014918 (0.0010) [2023-12-27 12:40:57,605][20381] Updated weights for policy 1, policy_version 1014255 (0.0007) [2023-12-27 12:40:58,117][20347] Updated weights for policy 0, policy_version 1014928 (0.0006) [2023-12-27 12:40:58,180][20347] Updated weights for policy 0, policy_version 1014938 (0.0005) [2023-12-27 12:40:58,241][20347] Updated weights for policy 0, policy_version 1014948 (0.0006) [2023-12-27 12:40:58,307][20381] Updated weights for policy 1, policy_version 1014265 (0.0010) [2023-12-27 12:40:58,374][20381] Updated weights for policy 1, policy_version 1014275 (0.0009) [2023-12-27 12:40:58,437][20381] Updated weights for policy 1, policy_version 1014285 (0.0011) [2023-12-27 12:40:58,498][20381] Updated weights for policy 1, policy_version 1014295 (0.0011) [2023-12-27 12:40:58,879][20347] Updated weights for policy 0, policy_version 1014958 (0.0008) [2023-12-27 12:40:58,938][20347] Updated weights for policy 0, policy_version 1014968 (0.0008) [2023-12-27 12:40:59,004][20347] Updated weights for policy 0, policy_version 1014978 (0.0008) [2023-12-27 12:40:59,326][20381] Updated weights for policy 1, policy_version 1014305 (0.0007) [2023-12-27 12:40:59,384][20381] Updated weights for policy 1, policy_version 1014315 (0.0007) [2023-12-27 12:40:59,443][20381] Updated weights for policy 1, policy_version 1014325 (0.0006) [2023-12-27 12:40:59,709][20347] Updated weights for policy 0, policy_version 1014988 (0.0007) [2023-12-27 12:40:59,764][20347] Updated weights for policy 0, policy_version 1014998 (0.0005) [2023-12-27 12:40:59,827][20347] Updated weights for policy 0, policy_version 1015008 (0.0006) [2023-12-27 12:41:00,094][20381] Updated weights for policy 1, policy_version 1014335 (0.0007) [2023-12-27 12:41:00,147][20381] Updated weights for policy 1, policy_version 1014345 (0.0008) [2023-12-27 12:41:00,199][20381] Updated weights for policy 1, policy_version 1014355 (0.0008) [2023-12-27 12:41:00,463][20347] Updated weights for policy 0, policy_version 1015018 (0.0008) [2023-12-27 12:41:00,511][20347] Updated weights for policy 0, policy_version 1015028 (0.0010) [2023-12-27 12:41:00,558][20347] Updated weights for policy 0, policy_version 1015038 (0.0006) [2023-12-27 12:41:00,610][20347] Updated weights for policy 0, policy_version 1015048 (0.0007) [2023-12-27 12:41:00,867][20381] Updated weights for policy 1, policy_version 1014365 (0.0010) [2023-12-27 12:41:00,928][20381] Updated weights for policy 1, policy_version 1014375 (0.0010) [2023-12-27 12:41:00,981][20381] Updated weights for policy 1, policy_version 1014385 (0.0010) [2023-12-27 12:41:01,320][20347] Updated weights for policy 0, policy_version 1015058 (0.0006) [2023-12-27 12:41:01,381][20347] Updated weights for policy 0, policy_version 1015068 (0.0008) [2023-12-27 12:41:01,443][20347] Updated weights for policy 0, policy_version 1015078 (0.0008) [2023-12-27 12:41:01,645][20381] Updated weights for policy 1, policy_version 1014395 (0.0011) [2023-12-27 12:41:01,709][20381] Updated weights for policy 1, policy_version 1014405 (0.0007) [2023-12-27 12:41:01,777][20381] Updated weights for policy 1, policy_version 1014415 (0.0008) [2023-12-27 12:41:02,093][20347] Updated weights for policy 0, policy_version 1015088 (0.0008) [2023-12-27 12:41:02,152][20347] Updated weights for policy 0, policy_version 1015098 (0.0007) [2023-12-27 12:41:02,202][20347] Updated weights for policy 0, policy_version 1015108 (0.0007) [2023-12-27 12:41:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 519634944. Throughput: 0: 10364.0, 1: 10319.5. Samples: 519643532. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:41:02,363][19170] Avg episode reward: [(0, '873.130'), (1, '217.310')] [2023-12-27 12:41:02,467][20381] Updated weights for policy 1, policy_version 1014425 (0.0011) [2023-12-27 12:41:02,516][20381] Updated weights for policy 1, policy_version 1014435 (0.0010) [2023-12-27 12:41:02,565][20381] Updated weights for policy 1, policy_version 1014445 (0.0010) [2023-12-27 12:41:02,613][20381] Updated weights for policy 1, policy_version 1014455 (0.0008) [2023-12-27 12:41:02,929][20347] Updated weights for policy 0, policy_version 1015118 (0.0007) [2023-12-27 12:41:02,977][20347] Updated weights for policy 0, policy_version 1015128 (0.0008) [2023-12-27 12:41:03,031][20347] Updated weights for policy 0, policy_version 1015138 (0.0007) [2023-12-27 12:41:03,319][20381] Updated weights for policy 1, policy_version 1014465 (0.0007) [2023-12-27 12:41:03,370][20381] Updated weights for policy 1, policy_version 1014475 (0.0007) [2023-12-27 12:41:03,419][20381] Updated weights for policy 1, policy_version 1014485 (0.0008) [2023-12-27 12:41:03,696][20347] Updated weights for policy 0, policy_version 1015148 (0.0008) [2023-12-27 12:41:03,753][20347] Updated weights for policy 0, policy_version 1015158 (0.0010) [2023-12-27 12:41:03,807][20347] Updated weights for policy 0, policy_version 1015168 (0.0010) [2023-12-27 12:41:04,088][20381] Updated weights for policy 1, policy_version 1014495 (0.0010) [2023-12-27 12:41:04,145][20381] Updated weights for policy 1, policy_version 1014505 (0.0010) [2023-12-27 12:41:04,198][20381] Updated weights for policy 1, policy_version 1014515 (0.0011) [2023-12-27 12:41:04,446][20347] Updated weights for policy 0, policy_version 1015178 (0.0013) [2023-12-27 12:41:04,507][20347] Updated weights for policy 0, policy_version 1015188 (0.0007) [2023-12-27 12:41:04,564][20347] Updated weights for policy 0, policy_version 1015198 (0.0008) [2023-12-27 12:41:04,622][20347] Updated weights for policy 0, policy_version 1015208 (0.0007) [2023-12-27 12:41:04,865][20381] Updated weights for policy 1, policy_version 1014525 (0.0009) [2023-12-27 12:41:04,921][20381] Updated weights for policy 1, policy_version 1014535 (0.0006) [2023-12-27 12:41:04,979][20381] Updated weights for policy 1, policy_version 1014545 (0.0005) [2023-12-27 12:41:05,189][20347] Updated weights for policy 0, policy_version 1015218 (0.0005) [2023-12-27 12:41:05,236][20347] Updated weights for policy 0, policy_version 1015228 (0.0005) [2023-12-27 12:41:05,292][20347] Updated weights for policy 0, policy_version 1015238 (0.0006) [2023-12-27 12:41:05,560][20381] Updated weights for policy 1, policy_version 1014555 (0.0006) [2023-12-27 12:41:05,618][20381] Updated weights for policy 1, policy_version 1014565 (0.0005) [2023-12-27 12:41:05,673][20381] Updated weights for policy 1, policy_version 1014575 (0.0005) [2023-12-27 12:41:05,915][20347] Updated weights for policy 0, policy_version 1015248 (0.0007) [2023-12-27 12:41:05,979][20347] Updated weights for policy 0, policy_version 1015258 (0.0009) [2023-12-27 12:41:06,040][20347] Updated weights for policy 0, policy_version 1015268 (0.0010) [2023-12-27 12:41:06,255][20381] Updated weights for policy 1, policy_version 1014585 (0.0005) [2023-12-27 12:41:06,318][20381] Updated weights for policy 1, policy_version 1014595 (0.0006) [2023-12-27 12:41:06,381][20381] Updated weights for policy 1, policy_version 1014605 (0.0006) [2023-12-27 12:41:06,442][20381] Updated weights for policy 1, policy_version 1014615 (0.0006) [2023-12-27 12:41:06,716][20347] Updated weights for policy 0, policy_version 1015278 (0.0010) [2023-12-27 12:41:06,779][20347] Updated weights for policy 0, policy_version 1015288 (0.0010) [2023-12-27 12:41:06,837][20347] Updated weights for policy 0, policy_version 1015298 (0.0010) [2023-12-27 12:41:07,063][20381] Updated weights for policy 1, policy_version 1014625 (0.0008) [2023-12-27 12:41:07,114][20381] Updated weights for policy 1, policy_version 1014635 (0.0010) [2023-12-27 12:41:07,164][20381] Updated weights for policy 1, policy_version 1014645 (0.0010) [2023-12-27 12:41:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20577.2). Total num frames: 519741440. Throughput: 0: 10358.6, 1: 10321.5. Samples: 519706344. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:41:07,363][19170] Avg episode reward: [(0, '882.840'), (1, '265.860')] [2023-12-27 12:41:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001014648_259776512.pth... [2023-12-27 12:41:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001015304_259964928.pth... [2023-12-27 12:41:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001014120_259661824.pth [2023-12-27 12:41:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001013432_259465216.pth [2023-12-27 12:41:07,421][20347] Updated weights for policy 0, policy_version 1015308 (0.0008) [2023-12-27 12:41:07,486][20347] Updated weights for policy 0, policy_version 1015318 (0.0007) [2023-12-27 12:41:07,552][20347] Updated weights for policy 0, policy_version 1015328 (0.0009) [2023-12-27 12:41:07,859][20381] Updated weights for policy 1, policy_version 1014655 (0.0007) [2023-12-27 12:41:07,908][20381] Updated weights for policy 1, policy_version 1014665 (0.0008) [2023-12-27 12:41:07,961][20381] Updated weights for policy 1, policy_version 1014675 (0.0008) [2023-12-27 12:41:08,138][20347] Updated weights for policy 0, policy_version 1015338 (0.0008) [2023-12-27 12:41:08,195][20347] Updated weights for policy 0, policy_version 1015348 (0.0008) [2023-12-27 12:41:08,248][20347] Updated weights for policy 0, policy_version 1015358 (0.0005) [2023-12-27 12:41:08,304][20347] Updated weights for policy 0, policy_version 1015368 (0.0005) [2023-12-27 12:41:08,634][20381] Updated weights for policy 1, policy_version 1014685 (0.0008) [2023-12-27 12:41:08,697][20381] Updated weights for policy 1, policy_version 1014695 (0.0006) [2023-12-27 12:41:08,758][20381] Updated weights for policy 1, policy_version 1014705 (0.0006) [2023-12-27 12:41:09,044][20347] Updated weights for policy 0, policy_version 1015378 (0.0005) [2023-12-27 12:41:09,108][20347] Updated weights for policy 0, policy_version 1015388 (0.0005) [2023-12-27 12:41:09,166][20347] Updated weights for policy 0, policy_version 1015398 (0.0005) [2023-12-27 12:41:09,402][20381] Updated weights for policy 1, policy_version 1014715 (0.0008) [2023-12-27 12:41:09,464][20381] Updated weights for policy 1, policy_version 1014725 (0.0007) [2023-12-27 12:41:09,529][20381] Updated weights for policy 1, policy_version 1014735 (0.0005) [2023-12-27 12:41:09,820][20347] Updated weights for policy 0, policy_version 1015408 (0.0007) [2023-12-27 12:41:09,876][20347] Updated weights for policy 0, policy_version 1015418 (0.0008) [2023-12-27 12:41:09,929][20347] Updated weights for policy 0, policy_version 1015428 (0.0008) [2023-12-27 12:41:10,217][20381] Updated weights for policy 1, policy_version 1014745 (0.0006) [2023-12-27 12:41:10,280][20381] Updated weights for policy 1, policy_version 1014755 (0.0006) [2023-12-27 12:41:10,346][20381] Updated weights for policy 1, policy_version 1014765 (0.0006) [2023-12-27 12:41:10,418][20381] Updated weights for policy 1, policy_version 1014775 (0.0007) [2023-12-27 12:41:10,619][20347] Updated weights for policy 0, policy_version 1015438 (0.0007) [2023-12-27 12:41:10,683][20347] Updated weights for policy 0, policy_version 1015448 (0.0005) [2023-12-27 12:41:10,747][20347] Updated weights for policy 0, policy_version 1015458 (0.0008) [2023-12-27 12:41:11,008][20381] Updated weights for policy 1, policy_version 1014785 (0.0007) [2023-12-27 12:41:11,076][20381] Updated weights for policy 1, policy_version 1014795 (0.0008) [2023-12-27 12:41:11,137][20381] Updated weights for policy 1, policy_version 1014805 (0.0007) [2023-12-27 12:41:11,413][20347] Updated weights for policy 0, policy_version 1015468 (0.0011) [2023-12-27 12:41:11,463][20347] Updated weights for policy 0, policy_version 1015478 (0.0010) [2023-12-27 12:41:11,522][20347] Updated weights for policy 0, policy_version 1015488 (0.0008) [2023-12-27 12:41:11,886][20381] Updated weights for policy 1, policy_version 1014815 (0.0008) [2023-12-27 12:41:11,946][20381] Updated weights for policy 1, policy_version 1014825 (0.0007) [2023-12-27 12:41:12,002][20381] Updated weights for policy 1, policy_version 1014835 (0.0008) [2023-12-27 12:41:12,240][20347] Updated weights for policy 0, policy_version 1015498 (0.0008) [2023-12-27 12:41:12,309][20347] Updated weights for policy 0, policy_version 1015508 (0.0006) [2023-12-27 12:41:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 519839744. Throughput: 0: 10417.6, 1: 10369.5. Samples: 519834100. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:41:12,363][19170] Avg episode reward: [(0, '814.220'), (1, '202.980')] [2023-12-27 12:41:12,379][20347] Updated weights for policy 0, policy_version 1015518 (0.0008) [2023-12-27 12:41:12,441][20347] Updated weights for policy 0, policy_version 1015528 (0.0008) [2023-12-27 12:41:12,723][20381] Updated weights for policy 1, policy_version 1014845 (0.0007) [2023-12-27 12:41:12,777][20381] Updated weights for policy 1, policy_version 1014855 (0.0007) [2023-12-27 12:41:12,836][20381] Updated weights for policy 1, policy_version 1014865 (0.0007) [2023-12-27 12:41:13,114][20347] Updated weights for policy 0, policy_version 1015538 (0.0008) [2023-12-27 12:41:13,170][20347] Updated weights for policy 0, policy_version 1015548 (0.0008) [2023-12-27 12:41:13,227][20347] Updated weights for policy 0, policy_version 1015558 (0.0007) [2023-12-27 12:41:13,532][20381] Updated weights for policy 1, policy_version 1014875 (0.0007) [2023-12-27 12:41:13,582][20381] Updated weights for policy 1, policy_version 1014885 (0.0005) [2023-12-27 12:41:13,637][20381] Updated weights for policy 1, policy_version 1014895 (0.0005) [2023-12-27 12:41:13,787][20347] Updated weights for policy 0, policy_version 1015568 (0.0007) [2023-12-27 12:41:13,842][20347] Updated weights for policy 0, policy_version 1015578 (0.0007) [2023-12-27 12:41:13,888][20347] Updated weights for policy 0, policy_version 1015588 (0.0010) [2023-12-27 12:41:14,153][20381] Updated weights for policy 1, policy_version 1014905 (0.0006) [2023-12-27 12:41:14,198][20381] Updated weights for policy 1, policy_version 1014915 (0.0008) [2023-12-27 12:41:14,254][20381] Updated weights for policy 1, policy_version 1014925 (0.0008) [2023-12-27 12:41:14,305][20381] Updated weights for policy 1, policy_version 1014935 (0.0007) [2023-12-27 12:41:14,553][20347] Updated weights for policy 0, policy_version 1015598 (0.0006) [2023-12-27 12:41:14,607][20347] Updated weights for policy 0, policy_version 1015608 (0.0005) [2023-12-27 12:41:14,663][20347] Updated weights for policy 0, policy_version 1015618 (0.0005) [2023-12-27 12:41:15,039][20381] Updated weights for policy 1, policy_version 1014945 (0.0008) [2023-12-27 12:41:15,100][20381] Updated weights for policy 1, policy_version 1014955 (0.0008) [2023-12-27 12:41:15,160][20381] Updated weights for policy 1, policy_version 1014965 (0.0008) [2023-12-27 12:41:15,280][20347] Updated weights for policy 0, policy_version 1015628 (0.0005) [2023-12-27 12:41:15,336][20347] Updated weights for policy 0, policy_version 1015638 (0.0006) [2023-12-27 12:41:15,387][20347] Updated weights for policy 0, policy_version 1015648 (0.0005) [2023-12-27 12:41:15,823][20381] Updated weights for policy 1, policy_version 1014975 (0.0008) [2023-12-27 12:41:15,879][20381] Updated weights for policy 1, policy_version 1014985 (0.0008) [2023-12-27 12:41:15,927][20381] Updated weights for policy 1, policy_version 1014995 (0.0009) [2023-12-27 12:41:15,959][20347] Updated weights for policy 0, policy_version 1015658 (0.0005) [2023-12-27 12:41:16,004][20347] Updated weights for policy 0, policy_version 1015668 (0.0007) [2023-12-27 12:41:16,051][20347] Updated weights for policy 0, policy_version 1015678 (0.0007) [2023-12-27 12:41:16,107][20347] Updated weights for policy 0, policy_version 1015688 (0.0008) [2023-12-27 12:41:16,519][20381] Updated weights for policy 1, policy_version 1015005 (0.0007) [2023-12-27 12:41:16,569][20381] Updated weights for policy 1, policy_version 1015015 (0.0006) [2023-12-27 12:41:16,629][20381] Updated weights for policy 1, policy_version 1015025 (0.0005) [2023-12-27 12:41:16,868][20347] Updated weights for policy 0, policy_version 1015698 (0.0008) [2023-12-27 12:41:16,921][20347] Updated weights for policy 0, policy_version 1015708 (0.0008) [2023-12-27 12:41:16,979][20347] Updated weights for policy 0, policy_version 1015718 (0.0007) [2023-12-27 12:41:17,230][20381] Updated weights for policy 1, policy_version 1015035 (0.0006) [2023-12-27 12:41:17,275][20381] Updated weights for policy 1, policy_version 1015045 (0.0008) [2023-12-27 12:41:17,325][20381] Updated weights for policy 1, policy_version 1015055 (0.0008) [2023-12-27 12:41:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 519946240. Throughput: 0: 10342.8, 1: 10438.1. Samples: 519959588. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:41:17,363][19170] Avg episode reward: [(0, '805.060'), (1, '201.560')] [2023-12-27 12:41:17,684][20347] Updated weights for policy 0, policy_version 1015728 (0.0007) [2023-12-27 12:41:17,736][20347] Updated weights for policy 0, policy_version 1015738 (0.0007) [2023-12-27 12:41:17,793][20347] Updated weights for policy 0, policy_version 1015748 (0.0007) [2023-12-27 12:41:18,045][20381] Updated weights for policy 1, policy_version 1015065 (0.0008) [2023-12-27 12:41:18,099][20381] Updated weights for policy 1, policy_version 1015075 (0.0008) [2023-12-27 12:41:18,146][20381] Updated weights for policy 1, policy_version 1015085 (0.0008) [2023-12-27 12:41:18,195][20381] Updated weights for policy 1, policy_version 1015095 (0.0006) [2023-12-27 12:41:18,533][20347] Updated weights for policy 0, policy_version 1015758 (0.0008) [2023-12-27 12:41:18,593][20347] Updated weights for policy 0, policy_version 1015768 (0.0009) [2023-12-27 12:41:18,656][20347] Updated weights for policy 0, policy_version 1015778 (0.0008) [2023-12-27 12:41:18,777][20381] Updated weights for policy 1, policy_version 1015105 (0.0006) [2023-12-27 12:41:18,837][20381] Updated weights for policy 1, policy_version 1015115 (0.0006) [2023-12-27 12:41:18,891][20381] Updated weights for policy 1, policy_version 1015125 (0.0006) [2023-12-27 12:41:19,399][20347] Updated weights for policy 0, policy_version 1015788 (0.0008) [2023-12-27 12:41:19,459][20347] Updated weights for policy 0, policy_version 1015798 (0.0008) [2023-12-27 12:41:19,519][20381] Updated weights for policy 1, policy_version 1015135 (0.0009) [2023-12-27 12:41:19,525][20347] Updated weights for policy 0, policy_version 1015808 (0.0008) [2023-12-27 12:41:19,577][20381] Updated weights for policy 1, policy_version 1015145 (0.0011) [2023-12-27 12:41:19,630][20381] Updated weights for policy 1, policy_version 1015155 (0.0011) [2023-12-27 12:41:20,252][20347] Updated weights for policy 0, policy_version 1015818 (0.0007) [2023-12-27 12:41:20,314][20347] Updated weights for policy 0, policy_version 1015828 (0.0007) [2023-12-27 12:41:20,325][20291] KL-divergence is very high: 725.4254 [2023-12-27 12:41:20,370][20381] Updated weights for policy 1, policy_version 1015165 (0.0010) [2023-12-27 12:41:20,375][20347] Updated weights for policy 0, policy_version 1015838 (0.0007) [2023-12-27 12:41:20,375][20291] KL-divergence is very high: 354.2116 [2023-12-27 12:41:20,427][20381] Updated weights for policy 1, policy_version 1015175 (0.0007) [2023-12-27 12:41:20,432][20347] Updated weights for policy 0, policy_version 1015848 (0.0007) [2023-12-27 12:41:20,485][20381] Updated weights for policy 1, policy_version 1015185 (0.0007) [2023-12-27 12:41:21,007][20291] KL-divergence is very high: 271.1475 [2023-12-27 12:41:21,014][20291] KL-divergence is very high: 580.6046 [2023-12-27 12:41:21,069][20347] Updated weights for policy 0, policy_version 1015858 (0.0007) [2023-12-27 12:41:21,071][20291] KL-divergence is very high: 252.3008 [2023-12-27 12:41:21,084][20291] KL-divergence is very high: 142.6924 [2023-12-27 12:41:21,091][20291] KL-divergence is very high: 139.0572 [2023-12-27 12:41:21,118][20291] KL-divergence is very high: 141.7923 [2023-12-27 12:41:21,124][20291] KL-divergence is very high: 395.2467 [2023-12-27 12:41:21,136][20347] Updated weights for policy 0, policy_version 1015868 (0.0007) [2023-12-27 12:41:21,172][20291] KL-divergence is very high: 178.0859 [2023-12-27 12:41:21,179][20291] KL-divergence is very high: 347.7514 [2023-12-27 12:41:21,192][20291] KL-divergence is very high: 134.4991 [2023-12-27 12:41:21,198][20291] KL-divergence is very high: 173.7467 [2023-12-27 12:41:21,205][20347] Updated weights for policy 0, policy_version 1015878 (0.0008) [2023-12-27 12:41:21,284][20381] Updated weights for policy 1, policy_version 1015195 (0.0007) [2023-12-27 12:41:21,346][20381] Updated weights for policy 1, policy_version 1015205 (0.0007) [2023-12-27 12:41:21,413][20381] Updated weights for policy 1, policy_version 1015215 (0.0009) [2023-12-27 12:41:21,924][20291] KL-divergence is very high: 146.0345 [2023-12-27 12:41:21,967][20347] Updated weights for policy 0, policy_version 1015888 (0.0008) [2023-12-27 12:41:22,021][20347] Updated weights for policy 0, policy_version 1015898 (0.0008) [2023-12-27 12:41:22,087][20347] Updated weights for policy 0, policy_version 1015908 (0.0008) [2023-12-27 12:41:22,128][20381] Updated weights for policy 1, policy_version 1015225 (0.0007) [2023-12-27 12:41:22,183][20381] Updated weights for policy 1, policy_version 1015235 (0.0008) [2023-12-27 12:41:22,238][20381] Updated weights for policy 1, policy_version 1015245 (0.0008) [2023-12-27 12:41:22,303][20381] Updated weights for policy 1, policy_version 1015255 (0.0008) [2023-12-27 12:41:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 520052736. Throughput: 0: 10277.5, 1: 10438.3. Samples: 520019032. Policy #0 lag: (min: 23.0, avg: 25.2, max: 55.0) [2023-12-27 12:41:22,363][19170] Avg episode reward: [(0, '873.010'), (1, '201.640')] [2023-12-27 12:41:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001015256_259932160.pth... [2023-12-27 12:41:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001015912_260120576.pth... [2023-12-27 12:41:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001014696_259809280.pth [2023-12-27 12:41:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001014040_259620864.pth [2023-12-27 12:41:22,805][20347] Updated weights for policy 0, policy_version 1015918 (0.0008) [2023-12-27 12:41:22,856][20347] Updated weights for policy 0, policy_version 1015928 (0.0009) [2023-12-27 12:41:22,906][20347] Updated weights for policy 0, policy_version 1015938 (0.0007) [2023-12-27 12:41:23,029][20381] Updated weights for policy 1, policy_version 1015265 (0.0010) [2023-12-27 12:41:23,088][20381] Updated weights for policy 1, policy_version 1015275 (0.0010) [2023-12-27 12:41:23,137][20381] Updated weights for policy 1, policy_version 1015285 (0.0010) [2023-12-27 12:41:23,623][20347] Updated weights for policy 0, policy_version 1015948 (0.0007) [2023-12-27 12:41:23,684][20347] Updated weights for policy 0, policy_version 1015958 (0.0008) [2023-12-27 12:41:23,739][20347] Updated weights for policy 0, policy_version 1015968 (0.0007) [2023-12-27 12:41:23,804][20381] Updated weights for policy 1, policy_version 1015295 (0.0011) [2023-12-27 12:41:23,858][20381] Updated weights for policy 1, policy_version 1015305 (0.0008) [2023-12-27 12:41:23,915][20381] Updated weights for policy 1, policy_version 1015315 (0.0006) [2023-12-27 12:41:24,457][20347] Updated weights for policy 0, policy_version 1015978 (0.0007) [2023-12-27 12:41:24,520][20347] Updated weights for policy 0, policy_version 1015988 (0.0010) [2023-12-27 12:41:24,578][20381] Updated weights for policy 1, policy_version 1015325 (0.0008) [2023-12-27 12:41:24,586][20347] Updated weights for policy 0, policy_version 1015998 (0.0008) [2023-12-27 12:41:24,632][20381] Updated weights for policy 1, policy_version 1015335 (0.0011) [2023-12-27 12:41:24,648][20347] Updated weights for policy 0, policy_version 1016008 (0.0007) [2023-12-27 12:41:24,686][20381] Updated weights for policy 1, policy_version 1015345 (0.0010) [2023-12-27 12:41:25,274][20347] Updated weights for policy 0, policy_version 1016018 (0.0010) [2023-12-27 12:41:25,311][20381] Updated weights for policy 1, policy_version 1015355 (0.0011) [2023-12-27 12:41:25,335][20347] Updated weights for policy 0, policy_version 1016028 (0.0010) [2023-12-27 12:41:25,367][20381] Updated weights for policy 1, policy_version 1015365 (0.0010) [2023-12-27 12:41:25,390][20347] Updated weights for policy 0, policy_version 1016038 (0.0010) [2023-12-27 12:41:25,427][20381] Updated weights for policy 1, policy_version 1015375 (0.0011) [2023-12-27 12:41:26,063][20381] Updated weights for policy 1, policy_version 1015385 (0.0010) [2023-12-27 12:41:26,066][20347] Updated weights for policy 0, policy_version 1016048 (0.0010) [2023-12-27 12:41:26,120][20381] Updated weights for policy 1, policy_version 1015395 (0.0006) [2023-12-27 12:41:26,125][20347] Updated weights for policy 0, policy_version 1016058 (0.0010) [2023-12-27 12:41:26,173][20381] Updated weights for policy 1, policy_version 1015405 (0.0005) [2023-12-27 12:41:26,184][20347] Updated weights for policy 0, policy_version 1016068 (0.0011) [2023-12-27 12:41:26,229][20381] Updated weights for policy 1, policy_version 1015415 (0.0005) [2023-12-27 12:41:26,826][20381] Updated weights for policy 1, policy_version 1015425 (0.0005) [2023-12-27 12:41:26,857][20347] Updated weights for policy 0, policy_version 1016078 (0.0007) [2023-12-27 12:41:26,876][20381] Updated weights for policy 1, policy_version 1015435 (0.0005) [2023-12-27 12:41:26,905][20291] KL-divergence is very high: 103.2279 [2023-12-27 12:41:26,910][20291] KL-divergence is very high: 184.3393 [2023-12-27 12:41:26,915][20347] Updated weights for policy 0, policy_version 1016088 (0.0005) [2023-12-27 12:41:26,925][20381] Updated weights for policy 1, policy_version 1015445 (0.0005) [2023-12-27 12:41:26,947][20291] KL-divergence is very high: 172.9941 [2023-12-27 12:41:26,952][20291] KL-divergence is very high: 110.0979 [2023-12-27 12:41:26,958][20291] KL-divergence is very high: 123.7250 [2023-12-27 12:41:26,974][20347] Updated weights for policy 0, policy_version 1016098 (0.0005) [2023-12-27 12:41:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 520151040. Throughput: 0: 10243.3, 1: 10414.9. Samples: 520140008. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:27,363][19170] Avg episode reward: [(0, '872.740'), (1, '252.410')] [2023-12-27 12:41:27,470][20381] Updated weights for policy 1, policy_version 1015455 (0.0005) [2023-12-27 12:41:27,521][20381] Updated weights for policy 1, policy_version 1015465 (0.0005) [2023-12-27 12:41:27,555][20347] Updated weights for policy 0, policy_version 1016108 (0.0007) [2023-12-27 12:41:27,566][20381] Updated weights for policy 1, policy_version 1015475 (0.0005) [2023-12-27 12:41:27,603][20347] Updated weights for policy 0, policy_version 1016118 (0.0010) [2023-12-27 12:41:27,657][20347] Updated weights for policy 0, policy_version 1016128 (0.0010) [2023-12-27 12:41:28,114][20381] Updated weights for policy 1, policy_version 1015485 (0.0007) [2023-12-27 12:41:28,162][20381] Updated weights for policy 1, policy_version 1015495 (0.0007) [2023-12-27 12:41:28,218][20381] Updated weights for policy 1, policy_version 1015505 (0.0005) [2023-12-27 12:41:28,334][20347] Updated weights for policy 0, policy_version 1016138 (0.0010) [2023-12-27 12:41:28,400][20347] Updated weights for policy 0, policy_version 1016148 (0.0011) [2023-12-27 12:41:28,460][20347] Updated weights for policy 0, policy_version 1016158 (0.0010) [2023-12-27 12:41:28,515][20347] Updated weights for policy 0, policy_version 1016168 (0.0005) [2023-12-27 12:41:28,827][20381] Updated weights for policy 1, policy_version 1015515 (0.0006) [2023-12-27 12:41:28,886][20381] Updated weights for policy 1, policy_version 1015525 (0.0006) [2023-12-27 12:41:28,941][20381] Updated weights for policy 1, policy_version 1015535 (0.0007) [2023-12-27 12:41:29,114][20347] Updated weights for policy 0, policy_version 1016178 (0.0005) [2023-12-27 12:41:29,170][20347] Updated weights for policy 0, policy_version 1016188 (0.0007) [2023-12-27 12:41:29,228][20347] Updated weights for policy 0, policy_version 1016198 (0.0009) [2023-12-27 12:41:29,546][20381] Updated weights for policy 1, policy_version 1015545 (0.0007) [2023-12-27 12:41:29,597][20381] Updated weights for policy 1, policy_version 1015555 (0.0005) [2023-12-27 12:41:29,656][20381] Updated weights for policy 1, policy_version 1015565 (0.0006) [2023-12-27 12:41:29,711][20381] Updated weights for policy 1, policy_version 1015575 (0.0006) [2023-12-27 12:41:29,869][20347] Updated weights for policy 0, policy_version 1016208 (0.0008) [2023-12-27 12:41:29,935][20347] Updated weights for policy 0, policy_version 1016218 (0.0006) [2023-12-27 12:41:29,999][20347] Updated weights for policy 0, policy_version 1016228 (0.0006) [2023-12-27 12:41:30,361][20381] Updated weights for policy 1, policy_version 1015585 (0.0008) [2023-12-27 12:41:30,412][20381] Updated weights for policy 1, policy_version 1015595 (0.0008) [2023-12-27 12:41:30,460][20381] Updated weights for policy 1, policy_version 1015605 (0.0009) [2023-12-27 12:41:30,567][20347] Updated weights for policy 0, policy_version 1016238 (0.0006) [2023-12-27 12:41:30,628][20347] Updated weights for policy 0, policy_version 1016248 (0.0005) [2023-12-27 12:41:30,683][20347] Updated weights for policy 0, policy_version 1016258 (0.0005) [2023-12-27 12:41:31,215][20381] Updated weights for policy 1, policy_version 1015615 (0.0007) [2023-12-27 12:41:31,277][20381] Updated weights for policy 1, policy_version 1015625 (0.0008) [2023-12-27 12:41:31,298][20347] Updated weights for policy 0, policy_version 1016268 (0.0006) [2023-12-27 12:41:31,338][20381] Updated weights for policy 1, policy_version 1015635 (0.0006) [2023-12-27 12:41:31,363][20347] Updated weights for policy 0, policy_version 1016278 (0.0008) [2023-12-27 12:41:31,426][20347] Updated weights for policy 0, policy_version 1016288 (0.0008) [2023-12-27 12:41:32,066][20381] Updated weights for policy 1, policy_version 1015645 (0.0007) [2023-12-27 12:41:32,122][20381] Updated weights for policy 1, policy_version 1015655 (0.0008) [2023-12-27 12:41:32,147][20347] Updated weights for policy 0, policy_version 1016298 (0.0007) [2023-12-27 12:41:32,184][20381] Updated weights for policy 1, policy_version 1015665 (0.0007) [2023-12-27 12:41:32,197][20347] Updated weights for policy 0, policy_version 1016308 (0.0007) [2023-12-27 12:41:32,244][20347] Updated weights for policy 0, policy_version 1016318 (0.0007) [2023-12-27 12:41:32,307][20347] Updated weights for policy 0, policy_version 1016328 (0.0007) [2023-12-27 12:41:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 520265728. Throughput: 0: 10334.3, 1: 10417.4. Samples: 520269544. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:32,363][19170] Avg episode reward: [(0, '911.800'), (1, '239.410')] [2023-12-27 12:41:32,896][20381] Updated weights for policy 1, policy_version 1015675 (0.0005) [2023-12-27 12:41:32,933][20347] Updated weights for policy 0, policy_version 1016338 (0.0007) [2023-12-27 12:41:32,954][20381] Updated weights for policy 1, policy_version 1015685 (0.0006) [2023-12-27 12:41:32,982][20347] Updated weights for policy 0, policy_version 1016348 (0.0006) [2023-12-27 12:41:33,010][20381] Updated weights for policy 1, policy_version 1015695 (0.0008) [2023-12-27 12:41:33,031][20347] Updated weights for policy 0, policy_version 1016358 (0.0007) [2023-12-27 12:41:33,671][20381] Updated weights for policy 1, policy_version 1015705 (0.0008) [2023-12-27 12:41:33,712][20347] Updated weights for policy 0, policy_version 1016368 (0.0009) [2023-12-27 12:41:33,727][20381] Updated weights for policy 1, policy_version 1015715 (0.0011) [2023-12-27 12:41:33,761][20347] Updated weights for policy 0, policy_version 1016378 (0.0010) [2023-12-27 12:41:33,783][20381] Updated weights for policy 1, policy_version 1015725 (0.0011) [2023-12-27 12:41:33,820][20347] Updated weights for policy 0, policy_version 1016388 (0.0009) [2023-12-27 12:41:33,838][20381] Updated weights for policy 1, policy_version 1015735 (0.0011) [2023-12-27 12:41:34,511][20347] Updated weights for policy 0, policy_version 1016398 (0.0007) [2023-12-27 12:41:34,517][20381] Updated weights for policy 1, policy_version 1015745 (0.0010) [2023-12-27 12:41:34,558][20347] Updated weights for policy 0, policy_version 1016408 (0.0007) [2023-12-27 12:41:34,573][20381] Updated weights for policy 1, policy_version 1015755 (0.0010) [2023-12-27 12:41:34,612][20347] Updated weights for policy 0, policy_version 1016418 (0.0006) [2023-12-27 12:41:34,626][20381] Updated weights for policy 1, policy_version 1015765 (0.0010) [2023-12-27 12:41:35,251][20381] Updated weights for policy 1, policy_version 1015775 (0.0010) [2023-12-27 12:41:35,305][20381] Updated weights for policy 1, policy_version 1015785 (0.0010) [2023-12-27 12:41:35,353][20347] Updated weights for policy 0, policy_version 1016428 (0.0006) [2023-12-27 12:41:35,360][20381] Updated weights for policy 1, policy_version 1015795 (0.0010) [2023-12-27 12:41:35,403][20347] Updated weights for policy 0, policy_version 1016438 (0.0006) [2023-12-27 12:41:35,455][20347] Updated weights for policy 0, policy_version 1016448 (0.0007) [2023-12-27 12:41:36,050][20381] Updated weights for policy 1, policy_version 1015805 (0.0011) [2023-12-27 12:41:36,099][20381] Updated weights for policy 1, policy_version 1015815 (0.0010) [2023-12-27 12:41:36,119][20347] Updated weights for policy 0, policy_version 1016458 (0.0007) [2023-12-27 12:41:36,153][20381] Updated weights for policy 1, policy_version 1015825 (0.0007) [2023-12-27 12:41:36,181][20347] Updated weights for policy 0, policy_version 1016469 (0.0007) [2023-12-27 12:41:36,235][20347] Updated weights for policy 0, policy_version 1016479 (0.0006) [2023-12-27 12:41:36,845][20381] Updated weights for policy 1, policy_version 1015835 (0.0010) [2023-12-27 12:41:36,869][20347] Updated weights for policy 0, policy_version 1016489 (0.0006) [2023-12-27 12:41:36,898][20381] Updated weights for policy 1, policy_version 1015845 (0.0010) [2023-12-27 12:41:36,921][20347] Updated weights for policy 0, policy_version 1016499 (0.0006) [2023-12-27 12:41:36,956][20381] Updated weights for policy 1, policy_version 1015855 (0.0006) [2023-12-27 12:41:36,977][20347] Updated weights for policy 0, policy_version 1016509 (0.0007) [2023-12-27 12:41:37,035][20347] Updated weights for policy 0, policy_version 1016519 (0.0006) [2023-12-27 12:41:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.2, 300 sec: 20549.4). Total num frames: 520364032. Throughput: 0: 10371.2, 1: 10407.8. Samples: 520331464. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:37,363][19170] Avg episode reward: [(0, '941.300'), (1, '197.150')] [2023-12-27 12:41:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001016520_260276224.pth... [2023-12-27 12:41:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001015864_260087808.pth... [2023-12-27 12:41:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001015304_259964928.pth [2023-12-27 12:41:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001014648_259776512.pth [2023-12-27 12:41:37,644][20381] Updated weights for policy 1, policy_version 1015865 (0.0006) [2023-12-27 12:41:37,694][20347] Updated weights for policy 0, policy_version 1016529 (0.0008) [2023-12-27 12:41:37,701][20381] Updated weights for policy 1, policy_version 1015875 (0.0010) [2023-12-27 12:41:37,741][20347] Updated weights for policy 0, policy_version 1016539 (0.0007) [2023-12-27 12:41:37,751][20381] Updated weights for policy 1, policy_version 1015885 (0.0007) [2023-12-27 12:41:37,796][20347] Updated weights for policy 0, policy_version 1016549 (0.0008) [2023-12-27 12:41:37,804][20381] Updated weights for policy 1, policy_version 1015895 (0.0007) [2023-12-27 12:41:38,383][20381] Updated weights for policy 1, policy_version 1015905 (0.0008) [2023-12-27 12:41:38,439][20381] Updated weights for policy 1, policy_version 1015915 (0.0007) [2023-12-27 12:41:38,484][20347] Updated weights for policy 0, policy_version 1016559 (0.0009) [2023-12-27 12:41:38,496][20381] Updated weights for policy 1, policy_version 1015925 (0.0009) [2023-12-27 12:41:38,545][20347] Updated weights for policy 0, policy_version 1016569 (0.0011) [2023-12-27 12:41:38,609][20347] Updated weights for policy 0, policy_version 1016579 (0.0010) [2023-12-27 12:41:39,243][20381] Updated weights for policy 1, policy_version 1015935 (0.0007) [2023-12-27 12:41:39,296][20381] Updated weights for policy 1, policy_version 1015945 (0.0007) [2023-12-27 12:41:39,305][20347] Updated weights for policy 0, policy_version 1016589 (0.0010) [2023-12-27 12:41:39,355][20381] Updated weights for policy 1, policy_version 1015955 (0.0007) [2023-12-27 12:41:39,369][20347] Updated weights for policy 0, policy_version 1016599 (0.0008) [2023-12-27 12:41:39,435][20347] Updated weights for policy 0, policy_version 1016609 (0.0008) [2023-12-27 12:41:40,079][20347] Updated weights for policy 0, policy_version 1016619 (0.0006) [2023-12-27 12:41:40,139][20347] Updated weights for policy 0, policy_version 1016629 (0.0007) [2023-12-27 12:41:40,152][20381] Updated weights for policy 1, policy_version 1015965 (0.0009) [2023-12-27 12:41:40,188][20347] Updated weights for policy 0, policy_version 1016639 (0.0009) [2023-12-27 12:41:40,202][20381] Updated weights for policy 1, policy_version 1015975 (0.0007) [2023-12-27 12:41:40,254][20381] Updated weights for policy 1, policy_version 1015985 (0.0007) [2023-12-27 12:41:40,870][20347] Updated weights for policy 0, policy_version 1016649 (0.0006) [2023-12-27 12:41:40,931][20347] Updated weights for policy 0, policy_version 1016659 (0.0006) [2023-12-27 12:41:40,969][20381] Updated weights for policy 1, policy_version 1015995 (0.0007) [2023-12-27 12:41:40,993][20347] Updated weights for policy 0, policy_version 1016669 (0.0005) [2023-12-27 12:41:41,027][20381] Updated weights for policy 1, policy_version 1016005 (0.0008) [2023-12-27 12:41:41,074][20347] Updated weights for policy 0, policy_version 1016679 (0.0007) [2023-12-27 12:41:41,085][20381] Updated weights for policy 1, policy_version 1016015 (0.0008) [2023-12-27 12:41:41,777][20347] Updated weights for policy 0, policy_version 1016689 (0.0008) [2023-12-27 12:41:41,830][20347] Updated weights for policy 0, policy_version 1016699 (0.0007) [2023-12-27 12:41:41,856][20291] KL-divergence is very high: 122.0882 [2023-12-27 12:41:41,884][20347] Updated weights for policy 0, policy_version 1016709 (0.0008) [2023-12-27 12:41:41,884][20381] Updated weights for policy 1, policy_version 1016025 (0.0009) [2023-12-27 12:41:41,936][20381] Updated weights for policy 1, policy_version 1016035 (0.0007) [2023-12-27 12:41:41,992][20381] Updated weights for policy 1, policy_version 1016045 (0.0008) [2023-12-27 12:41:42,052][20381] Updated weights for policy 1, policy_version 1016055 (0.0007) [2023-12-27 12:41:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 520462336. Throughput: 0: 10367.7, 1: 10363.6. Samples: 520452496. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:42,363][19170] Avg episode reward: [(0, '872.900'), (1, '272.320')] [2023-12-27 12:41:42,579][20347] Updated weights for policy 0, policy_version 1016719 (0.0008) [2023-12-27 12:41:42,641][20347] Updated weights for policy 0, policy_version 1016729 (0.0007) [2023-12-27 12:41:42,699][20347] Updated weights for policy 0, policy_version 1016739 (0.0007) [2023-12-27 12:41:42,750][20381] Updated weights for policy 1, policy_version 1016065 (0.0007) [2023-12-27 12:41:42,808][20381] Updated weights for policy 1, policy_version 1016075 (0.0008) [2023-12-27 12:41:42,861][20381] Updated weights for policy 1, policy_version 1016085 (0.0008) [2023-12-27 12:41:43,353][20347] Updated weights for policy 0, policy_version 1016749 (0.0009) [2023-12-27 12:41:43,408][20347] Updated weights for policy 0, policy_version 1016759 (0.0007) [2023-12-27 12:41:43,471][20347] Updated weights for policy 0, policy_version 1016769 (0.0010) [2023-12-27 12:41:43,578][20381] Updated weights for policy 1, policy_version 1016095 (0.0007) [2023-12-27 12:41:43,634][20381] Updated weights for policy 1, policy_version 1016105 (0.0005) [2023-12-27 12:41:43,685][20381] Updated weights for policy 1, policy_version 1016115 (0.0005) [2023-12-27 12:41:44,039][20347] Updated weights for policy 0, policy_version 1016779 (0.0010) [2023-12-27 12:41:44,093][20347] Updated weights for policy 0, policy_version 1016789 (0.0010) [2023-12-27 12:41:44,147][20347] Updated weights for policy 0, policy_version 1016799 (0.0010) [2023-12-27 12:41:44,339][20381] Updated weights for policy 1, policy_version 1016125 (0.0011) [2023-12-27 12:41:44,392][20381] Updated weights for policy 1, policy_version 1016135 (0.0011) [2023-12-27 12:41:44,450][20381] Updated weights for policy 1, policy_version 1016145 (0.0011) [2023-12-27 12:41:44,868][20347] Updated weights for policy 0, policy_version 1016809 (0.0010) [2023-12-27 12:41:44,928][20347] Updated weights for policy 0, policy_version 1016819 (0.0010) [2023-12-27 12:41:44,975][20347] Updated weights for policy 0, policy_version 1016829 (0.0010) [2023-12-27 12:41:45,030][20347] Updated weights for policy 0, policy_version 1016839 (0.0010) [2023-12-27 12:41:45,127][20381] Updated weights for policy 1, policy_version 1016155 (0.0010) [2023-12-27 12:41:45,185][20381] Updated weights for policy 1, policy_version 1016165 (0.0011) [2023-12-27 12:41:45,234][20381] Updated weights for policy 1, policy_version 1016175 (0.0010) [2023-12-27 12:41:45,740][20347] Updated weights for policy 0, policy_version 1016849 (0.0010) [2023-12-27 12:41:45,795][20347] Updated weights for policy 0, policy_version 1016859 (0.0010) [2023-12-27 12:41:45,819][20381] Updated weights for policy 1, policy_version 1016185 (0.0006) [2023-12-27 12:41:45,848][20347] Updated weights for policy 0, policy_version 1016869 (0.0010) [2023-12-27 12:41:45,875][20381] Updated weights for policy 1, policy_version 1016195 (0.0010) [2023-12-27 12:41:45,929][20381] Updated weights for policy 1, policy_version 1016205 (0.0007) [2023-12-27 12:41:45,983][20381] Updated weights for policy 1, policy_version 1016215 (0.0005) [2023-12-27 12:41:46,498][20347] Updated weights for policy 0, policy_version 1016879 (0.0007) [2023-12-27 12:41:46,554][20347] Updated weights for policy 0, policy_version 1016889 (0.0005) [2023-12-27 12:41:46,562][20381] Updated weights for policy 1, policy_version 1016225 (0.0010) [2023-12-27 12:41:46,598][20347] Updated weights for policy 0, policy_version 1016899 (0.0005) [2023-12-27 12:41:46,626][20381] Updated weights for policy 1, policy_version 1016235 (0.0010) [2023-12-27 12:41:46,686][20381] Updated weights for policy 1, policy_version 1016245 (0.0010) [2023-12-27 12:41:47,223][20347] Updated weights for policy 0, policy_version 1016909 (0.0007) [2023-12-27 12:41:47,277][20347] Updated weights for policy 0, policy_version 1016919 (0.0007) [2023-12-27 12:41:47,321][20381] Updated weights for policy 1, policy_version 1016255 (0.0010) [2023-12-27 12:41:47,332][20347] Updated weights for policy 0, policy_version 1016929 (0.0010) [2023-12-27 12:41:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 520560640. Throughput: 0: 10344.5, 1: 10418.9. Samples: 520577884. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:47,363][19170] Avg episode reward: [(0, '911.860'), (1, '133.460')] [2023-12-27 12:41:47,382][20381] Updated weights for policy 1, policy_version 1016265 (0.0010) [2023-12-27 12:41:47,430][20381] Updated weights for policy 1, policy_version 1016275 (0.0010) [2023-12-27 12:41:47,906][20347] Updated weights for policy 0, policy_version 1016939 (0.0010) [2023-12-27 12:41:47,976][20347] Updated weights for policy 0, policy_version 1016949 (0.0010) [2023-12-27 12:41:48,037][20347] Updated weights for policy 0, policy_version 1016959 (0.0010) [2023-12-27 12:41:48,101][20381] Updated weights for policy 1, policy_version 1016285 (0.0010) [2023-12-27 12:41:48,155][20381] Updated weights for policy 1, policy_version 1016295 (0.0009) [2023-12-27 12:41:48,206][20381] Updated weights for policy 1, policy_version 1016305 (0.0009) [2023-12-27 12:41:48,733][20347] Updated weights for policy 0, policy_version 1016969 (0.0010) [2023-12-27 12:41:48,794][20347] Updated weights for policy 0, policy_version 1016979 (0.0010) [2023-12-27 12:41:48,827][20381] Updated weights for policy 1, policy_version 1016315 (0.0006) [2023-12-27 12:41:48,852][20347] Updated weights for policy 0, policy_version 1016989 (0.0010) [2023-12-27 12:41:48,879][20381] Updated weights for policy 1, policy_version 1016325 (0.0007) [2023-12-27 12:41:48,908][20347] Updated weights for policy 0, policy_version 1016999 (0.0010) [2023-12-27 12:41:48,937][20381] Updated weights for policy 1, policy_version 1016335 (0.0007) [2023-12-27 12:41:49,518][20347] Updated weights for policy 0, policy_version 1017009 (0.0010) [2023-12-27 12:41:49,571][20347] Updated weights for policy 0, policy_version 1017019 (0.0010) [2023-12-27 12:41:49,630][20347] Updated weights for policy 0, policy_version 1017029 (0.0010) [2023-12-27 12:41:49,661][20381] Updated weights for policy 1, policy_version 1016345 (0.0007) [2023-12-27 12:41:49,717][20381] Updated weights for policy 1, policy_version 1016355 (0.0007) [2023-12-27 12:41:49,772][20381] Updated weights for policy 1, policy_version 1016365 (0.0007) [2023-12-27 12:41:49,833][20381] Updated weights for policy 1, policy_version 1016375 (0.0009) [2023-12-27 12:41:50,310][20347] Updated weights for policy 0, policy_version 1017039 (0.0011) [2023-12-27 12:41:50,370][20347] Updated weights for policy 0, policy_version 1017049 (0.0009) [2023-12-27 12:41:50,425][20347] Updated weights for policy 0, policy_version 1017059 (0.0007) [2023-12-27 12:41:50,539][20381] Updated weights for policy 1, policy_version 1016385 (0.0010) [2023-12-27 12:41:50,597][20381] Updated weights for policy 1, policy_version 1016395 (0.0007) [2023-12-27 12:41:50,656][20381] Updated weights for policy 1, policy_version 1016405 (0.0010) [2023-12-27 12:41:51,146][20347] Updated weights for policy 0, policy_version 1017069 (0.0008) [2023-12-27 12:41:51,209][20347] Updated weights for policy 0, policy_version 1017079 (0.0008) [2023-12-27 12:41:51,264][20347] Updated weights for policy 0, policy_version 1017089 (0.0008) [2023-12-27 12:41:51,345][20381] Updated weights for policy 1, policy_version 1016415 (0.0008) [2023-12-27 12:41:51,415][20381] Updated weights for policy 1, policy_version 1016425 (0.0011) [2023-12-27 12:41:51,476][20381] Updated weights for policy 1, policy_version 1016435 (0.0011) [2023-12-27 12:41:51,943][20347] Updated weights for policy 0, policy_version 1017099 (0.0009) [2023-12-27 12:41:52,008][20347] Updated weights for policy 0, policy_version 1017109 (0.0010) [2023-12-27 12:41:52,070][20347] Updated weights for policy 0, policy_version 1017119 (0.0010) [2023-12-27 12:41:52,209][20381] Updated weights for policy 1, policy_version 1016445 (0.0010) [2023-12-27 12:41:52,264][20381] Updated weights for policy 1, policy_version 1016455 (0.0007) [2023-12-27 12:41:52,321][20381] Updated weights for policy 1, policy_version 1016465 (0.0005) [2023-12-27 12:41:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 520675328. Throughput: 0: 10341.6, 1: 10396.0. Samples: 520639536. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:52,363][19170] Avg episode reward: [(0, '960.860'), (1, '442.120')] [2023-12-27 12:41:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001016472_260243456.pth... [2023-12-27 12:41:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001017128_260431872.pth... [2023-12-27 12:41:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001015256_259932160.pth [2023-12-27 12:41:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001015912_260120576.pth [2023-12-27 12:41:52,737][20347] Updated weights for policy 0, policy_version 1017129 (0.0010) [2023-12-27 12:41:52,796][20347] Updated weights for policy 0, policy_version 1017139 (0.0011) [2023-12-27 12:41:52,855][20347] Updated weights for policy 0, policy_version 1017149 (0.0010) [2023-12-27 12:41:52,916][20347] Updated weights for policy 0, policy_version 1017159 (0.0009) [2023-12-27 12:41:53,001][20381] Updated weights for policy 1, policy_version 1016475 (0.0008) [2023-12-27 12:41:53,055][20381] Updated weights for policy 1, policy_version 1016485 (0.0008) [2023-12-27 12:41:53,104][20381] Updated weights for policy 1, policy_version 1016495 (0.0007) [2023-12-27 12:41:53,559][20347] Updated weights for policy 0, policy_version 1017169 (0.0005) [2023-12-27 12:41:53,618][20347] Updated weights for policy 0, policy_version 1017179 (0.0005) [2023-12-27 12:41:53,672][20347] Updated weights for policy 0, policy_version 1017189 (0.0005) [2023-12-27 12:41:53,801][20381] Updated weights for policy 1, policy_version 1016505 (0.0007) [2023-12-27 12:41:53,855][20381] Updated weights for policy 1, policy_version 1016515 (0.0005) [2023-12-27 12:41:53,904][20381] Updated weights for policy 1, policy_version 1016525 (0.0006) [2023-12-27 12:41:53,962][20381] Updated weights for policy 1, policy_version 1016535 (0.0010) [2023-12-27 12:41:54,243][20347] Updated weights for policy 0, policy_version 1017199 (0.0007) [2023-12-27 12:41:54,295][20347] Updated weights for policy 0, policy_version 1017209 (0.0008) [2023-12-27 12:41:54,355][20347] Updated weights for policy 0, policy_version 1017219 (0.0008) [2023-12-27 12:41:54,649][20381] Updated weights for policy 1, policy_version 1016545 (0.0010) [2023-12-27 12:41:54,707][20381] Updated weights for policy 1, policy_version 1016555 (0.0010) [2023-12-27 12:41:54,764][20381] Updated weights for policy 1, policy_version 1016565 (0.0010) [2023-12-27 12:41:55,046][20347] Updated weights for policy 0, policy_version 1017229 (0.0009) [2023-12-27 12:41:55,109][20347] Updated weights for policy 0, policy_version 1017239 (0.0009) [2023-12-27 12:41:55,164][20347] Updated weights for policy 0, policy_version 1017249 (0.0008) [2023-12-27 12:41:55,446][20381] Updated weights for policy 1, policy_version 1016575 (0.0010) [2023-12-27 12:41:55,492][20381] Updated weights for policy 1, policy_version 1016585 (0.0010) [2023-12-27 12:41:55,544][20381] Updated weights for policy 1, policy_version 1016595 (0.0010) [2023-12-27 12:41:55,733][20347] Updated weights for policy 0, policy_version 1017259 (0.0006) [2023-12-27 12:41:55,784][20347] Updated weights for policy 0, policy_version 1017269 (0.0007) [2023-12-27 12:41:55,847][20347] Updated weights for policy 0, policy_version 1017279 (0.0005) [2023-12-27 12:41:56,257][20381] Updated weights for policy 1, policy_version 1016605 (0.0010) [2023-12-27 12:41:56,312][20381] Updated weights for policy 1, policy_version 1016615 (0.0010) [2023-12-27 12:41:56,367][20381] Updated weights for policy 1, policy_version 1016625 (0.0010) [2023-12-27 12:41:56,428][20347] Updated weights for policy 0, policy_version 1017289 (0.0006) [2023-12-27 12:41:56,483][20347] Updated weights for policy 0, policy_version 1017299 (0.0007) [2023-12-27 12:41:56,534][20347] Updated weights for policy 0, policy_version 1017309 (0.0005) [2023-12-27 12:41:56,583][20347] Updated weights for policy 0, policy_version 1017319 (0.0005) [2023-12-27 12:41:56,972][20381] Updated weights for policy 1, policy_version 1016635 (0.0011) [2023-12-27 12:41:57,018][20381] Updated weights for policy 1, policy_version 1016645 (0.0009) [2023-12-27 12:41:57,078][20381] Updated weights for policy 1, policy_version 1016655 (0.0005) [2023-12-27 12:41:57,219][20347] Updated weights for policy 0, policy_version 1017329 (0.0010) [2023-12-27 12:41:57,276][20347] Updated weights for policy 0, policy_version 1017339 (0.0010) [2023-12-27 12:41:57,329][20347] Updated weights for policy 0, policy_version 1017349 (0.0009) [2023-12-27 12:41:57,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 520781824. Throughput: 0: 10339.1, 1: 10314.7. Samples: 520763524. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:41:57,363][19170] Avg episode reward: [(0, '931.560'), (1, '230.770')] [2023-12-27 12:41:57,731][20381] Updated weights for policy 1, policy_version 1016665 (0.0006) [2023-12-27 12:41:57,778][20381] Updated weights for policy 1, policy_version 1016675 (0.0010) [2023-12-27 12:41:57,825][20381] Updated weights for policy 1, policy_version 1016685 (0.0010) [2023-12-27 12:41:57,882][20381] Updated weights for policy 1, policy_version 1016695 (0.0010) [2023-12-27 12:41:57,936][20347] Updated weights for policy 0, policy_version 1017359 (0.0010) [2023-12-27 12:41:57,992][20347] Updated weights for policy 0, policy_version 1017369 (0.0008) [2023-12-27 12:41:58,056][20347] Updated weights for policy 0, policy_version 1017379 (0.0005) [2023-12-27 12:41:58,612][20381] Updated weights for policy 1, policy_version 1016705 (0.0008) [2023-12-27 12:41:58,673][20381] Updated weights for policy 1, policy_version 1016715 (0.0008) [2023-12-27 12:41:58,746][20381] Updated weights for policy 1, policy_version 1016725 (0.0009) [2023-12-27 12:41:58,750][20347] Updated weights for policy 0, policy_version 1017389 (0.0006) [2023-12-27 12:41:58,826][20347] Updated weights for policy 0, policy_version 1017399 (0.0008) [2023-12-27 12:41:58,896][20347] Updated weights for policy 0, policy_version 1017409 (0.0006) [2023-12-27 12:41:59,503][20381] Updated weights for policy 1, policy_version 1016735 (0.0009) [2023-12-27 12:41:59,548][20347] Updated weights for policy 0, policy_version 1017419 (0.0006) [2023-12-27 12:41:59,565][20381] Updated weights for policy 1, policy_version 1016745 (0.0009) [2023-12-27 12:41:59,604][20347] Updated weights for policy 0, policy_version 1017429 (0.0005) [2023-12-27 12:41:59,617][20381] Updated weights for policy 1, policy_version 1016755 (0.0010) [2023-12-27 12:41:59,666][20347] Updated weights for policy 0, policy_version 1017439 (0.0006) [2023-12-27 12:42:00,330][20381] Updated weights for policy 1, policy_version 1016765 (0.0011) [2023-12-27 12:42:00,378][20347] Updated weights for policy 0, policy_version 1017449 (0.0008) [2023-12-27 12:42:00,390][20381] Updated weights for policy 1, policy_version 1016775 (0.0011) [2023-12-27 12:42:00,440][20347] Updated weights for policy 0, policy_version 1017459 (0.0009) [2023-12-27 12:42:00,449][20381] Updated weights for policy 1, policy_version 1016785 (0.0011) [2023-12-27 12:42:00,494][20347] Updated weights for policy 0, policy_version 1017469 (0.0010) [2023-12-27 12:42:00,542][20347] Updated weights for policy 0, policy_version 1017479 (0.0009) [2023-12-27 12:42:01,139][20381] Updated weights for policy 1, policy_version 1016795 (0.0010) [2023-12-27 12:42:01,206][20381] Updated weights for policy 1, policy_version 1016805 (0.0007) [2023-12-27 12:42:01,227][20347] Updated weights for policy 0, policy_version 1017489 (0.0006) [2023-12-27 12:42:01,271][20381] Updated weights for policy 1, policy_version 1016815 (0.0008) [2023-12-27 12:42:01,293][20347] Updated weights for policy 0, policy_version 1017499 (0.0007) [2023-12-27 12:42:01,351][20347] Updated weights for policy 0, policy_version 1017509 (0.0007) [2023-12-27 12:42:01,964][20381] Updated weights for policy 1, policy_version 1016825 (0.0008) [2023-12-27 12:42:02,010][20347] Updated weights for policy 0, policy_version 1017519 (0.0006) [2023-12-27 12:42:02,029][20381] Updated weights for policy 1, policy_version 1016835 (0.0006) [2023-12-27 12:42:02,071][20347] Updated weights for policy 0, policy_version 1017529 (0.0005) [2023-12-27 12:42:02,086][20381] Updated weights for policy 1, policy_version 1016845 (0.0007) [2023-12-27 12:42:02,128][20347] Updated weights for policy 0, policy_version 1017539 (0.0005) [2023-12-27 12:42:02,145][20381] Updated weights for policy 1, policy_version 1016855 (0.0011) [2023-12-27 12:42:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 520880128. Throughput: 0: 10339.6, 1: 10242.2. Samples: 520885764. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:02,363][19170] Avg episode reward: [(0, '873.060'), (1, '243.380')] [2023-12-27 12:42:02,783][20381] Updated weights for policy 1, policy_version 1016865 (0.0006) [2023-12-27 12:42:02,788][20347] Updated weights for policy 0, policy_version 1017549 (0.0007) [2023-12-27 12:42:02,840][20381] Updated weights for policy 1, policy_version 1016875 (0.0009) [2023-12-27 12:42:02,842][20347] Updated weights for policy 0, policy_version 1017559 (0.0006) [2023-12-27 12:42:02,894][20347] Updated weights for policy 0, policy_version 1017569 (0.0005) [2023-12-27 12:42:02,895][20381] Updated weights for policy 1, policy_version 1016885 (0.0010) [2023-12-27 12:42:03,510][20381] Updated weights for policy 1, policy_version 1016895 (0.0007) [2023-12-27 12:42:03,543][20347] Updated weights for policy 0, policy_version 1017579 (0.0006) [2023-12-27 12:42:03,556][20381] Updated weights for policy 1, policy_version 1016905 (0.0006) [2023-12-27 12:42:03,595][20347] Updated weights for policy 0, policy_version 1017589 (0.0006) [2023-12-27 12:42:03,603][20381] Updated weights for policy 1, policy_version 1016915 (0.0006) [2023-12-27 12:42:03,655][20347] Updated weights for policy 0, policy_version 1017599 (0.0005) [2023-12-27 12:42:04,255][20347] Updated weights for policy 0, policy_version 1017609 (0.0006) [2023-12-27 12:42:04,310][20347] Updated weights for policy 0, policy_version 1017619 (0.0007) [2023-12-27 12:42:04,315][20381] Updated weights for policy 1, policy_version 1016925 (0.0007) [2023-12-27 12:42:04,363][20347] Updated weights for policy 0, policy_version 1017629 (0.0008) [2023-12-27 12:42:04,381][20381] Updated weights for policy 1, policy_version 1016935 (0.0009) [2023-12-27 12:42:04,419][20347] Updated weights for policy 0, policy_version 1017639 (0.0007) [2023-12-27 12:42:04,442][20381] Updated weights for policy 1, policy_version 1016945 (0.0007) [2023-12-27 12:42:05,060][20347] Updated weights for policy 0, policy_version 1017649 (0.0008) [2023-12-27 12:42:05,111][20347] Updated weights for policy 0, policy_version 1017659 (0.0007) [2023-12-27 12:42:05,113][20381] Updated weights for policy 1, policy_version 1016955 (0.0007) [2023-12-27 12:42:05,160][20347] Updated weights for policy 0, policy_version 1017669 (0.0006) [2023-12-27 12:42:05,165][20381] Updated weights for policy 1, policy_version 1016965 (0.0006) [2023-12-27 12:42:05,215][20381] Updated weights for policy 1, policy_version 1016975 (0.0007) [2023-12-27 12:42:05,774][20347] Updated weights for policy 0, policy_version 1017679 (0.0006) [2023-12-27 12:42:05,826][20347] Updated weights for policy 0, policy_version 1017689 (0.0005) [2023-12-27 12:42:05,879][20347] Updated weights for policy 0, policy_version 1017699 (0.0005) [2023-12-27 12:42:05,892][20381] Updated weights for policy 1, policy_version 1016985 (0.0009) [2023-12-27 12:42:05,941][20381] Updated weights for policy 1, policy_version 1016995 (0.0006) [2023-12-27 12:42:05,995][20381] Updated weights for policy 1, policy_version 1017005 (0.0007) [2023-12-27 12:42:06,048][20381] Updated weights for policy 1, policy_version 1017015 (0.0007) [2023-12-27 12:42:06,537][20347] Updated weights for policy 0, policy_version 1017709 (0.0007) [2023-12-27 12:42:06,594][20347] Updated weights for policy 0, policy_version 1017719 (0.0008) [2023-12-27 12:42:06,653][20347] Updated weights for policy 0, policy_version 1017729 (0.0007) [2023-12-27 12:42:06,752][20381] Updated weights for policy 1, policy_version 1017025 (0.0008) [2023-12-27 12:42:06,811][20381] Updated weights for policy 1, policy_version 1017035 (0.0008) [2023-12-27 12:42:06,860][20381] Updated weights for policy 1, policy_version 1017045 (0.0008) [2023-12-27 12:42:07,294][20347] Updated weights for policy 0, policy_version 1017739 (0.0007) [2023-12-27 12:42:07,353][20347] Updated weights for policy 0, policy_version 1017749 (0.0005) [2023-12-27 12:42:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 520978432. Throughput: 0: 10424.9, 1: 10254.6. Samples: 520949608. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:07,363][19170] Avg episode reward: [(0, '912.330'), (1, '275.900')] [2023-12-27 12:42:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001017048_260390912.pth... [2023-12-27 12:42:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001015864_260087808.pth [2023-12-27 12:42:07,414][20347] Updated weights for policy 0, policy_version 1017759 (0.0005) [2023-12-27 12:42:07,458][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001017768_260595712.pth... [2023-12-27 12:42:07,461][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001016520_260276224.pth [2023-12-27 12:42:07,607][20381] Updated weights for policy 1, policy_version 1017055 (0.0008) [2023-12-27 12:42:07,676][20381] Updated weights for policy 1, policy_version 1017065 (0.0008) [2023-12-27 12:42:07,733][20381] Updated weights for policy 1, policy_version 1017075 (0.0008) [2023-12-27 12:42:08,048][20347] Updated weights for policy 0, policy_version 1017769 (0.0007) [2023-12-27 12:42:08,102][20347] Updated weights for policy 0, policy_version 1017779 (0.0008) [2023-12-27 12:42:08,157][20347] Updated weights for policy 0, policy_version 1017789 (0.0008) [2023-12-27 12:42:08,210][20347] Updated weights for policy 0, policy_version 1017799 (0.0007) [2023-12-27 12:42:08,398][20381] Updated weights for policy 1, policy_version 1017085 (0.0008) [2023-12-27 12:42:08,450][20381] Updated weights for policy 1, policy_version 1017095 (0.0006) [2023-12-27 12:42:08,493][20381] Updated weights for policy 1, policy_version 1017105 (0.0007) [2023-12-27 12:42:08,945][20347] Updated weights for policy 0, policy_version 1017809 (0.0008) [2023-12-27 12:42:09,004][20347] Updated weights for policy 0, policy_version 1017819 (0.0007) [2023-12-27 12:42:09,051][20347] Updated weights for policy 0, policy_version 1017829 (0.0008) [2023-12-27 12:42:09,206][20381] Updated weights for policy 1, policy_version 1017115 (0.0007) [2023-12-27 12:42:09,275][20381] Updated weights for policy 1, policy_version 1017125 (0.0008) [2023-12-27 12:42:09,337][20381] Updated weights for policy 1, policy_version 1017135 (0.0007) [2023-12-27 12:42:09,772][20347] Updated weights for policy 0, policy_version 1017839 (0.0007) [2023-12-27 12:42:09,837][20347] Updated weights for policy 0, policy_version 1017849 (0.0007) [2023-12-27 12:42:09,903][20347] Updated weights for policy 0, policy_version 1017859 (0.0007) [2023-12-27 12:42:10,084][20381] Updated weights for policy 1, policy_version 1017145 (0.0007) [2023-12-27 12:42:10,149][20381] Updated weights for policy 1, policy_version 1017155 (0.0011) [2023-12-27 12:42:10,215][20381] Updated weights for policy 1, policy_version 1017165 (0.0008) [2023-12-27 12:42:10,278][20381] Updated weights for policy 1, policy_version 1017175 (0.0008) [2023-12-27 12:42:10,567][20347] Updated weights for policy 0, policy_version 1017869 (0.0009) [2023-12-27 12:42:10,619][20347] Updated weights for policy 0, policy_version 1017879 (0.0008) [2023-12-27 12:42:10,683][20347] Updated weights for policy 0, policy_version 1017889 (0.0007) [2023-12-27 12:42:10,890][20381] Updated weights for policy 1, policy_version 1017185 (0.0006) [2023-12-27 12:42:10,942][20381] Updated weights for policy 1, policy_version 1017195 (0.0008) [2023-12-27 12:42:10,996][20381] Updated weights for policy 1, policy_version 1017205 (0.0005) [2023-12-27 12:42:11,431][20347] Updated weights for policy 0, policy_version 1017899 (0.0008) [2023-12-27 12:42:11,491][20347] Updated weights for policy 0, policy_version 1017909 (0.0008) [2023-12-27 12:42:11,549][20347] Updated weights for policy 0, policy_version 1017919 (0.0008) [2023-12-27 12:42:11,673][20381] Updated weights for policy 1, policy_version 1017215 (0.0008) [2023-12-27 12:42:11,738][20381] Updated weights for policy 1, policy_version 1017225 (0.0008) [2023-12-27 12:42:11,801][20381] Updated weights for policy 1, policy_version 1017235 (0.0006) [2023-12-27 12:42:12,341][20347] Updated weights for policy 0, policy_version 1017929 (0.0008) [2023-12-27 12:42:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 521076736. Throughput: 0: 10472.5, 1: 10227.9. Samples: 521071524. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:12,363][19170] Avg episode reward: [(0, '941.310'), (1, '306.020')] [2023-12-27 12:42:12,402][20347] Updated weights for policy 0, policy_version 1017939 (0.0008) [2023-12-27 12:42:12,451][20347] Updated weights for policy 0, policy_version 1017949 (0.0008) [2023-12-27 12:42:12,497][20381] Updated weights for policy 1, policy_version 1017245 (0.0007) [2023-12-27 12:42:12,513][20347] Updated weights for policy 0, policy_version 1017959 (0.0008) [2023-12-27 12:42:12,560][20381] Updated weights for policy 1, policy_version 1017255 (0.0008) [2023-12-27 12:42:12,625][20381] Updated weights for policy 1, policy_version 1017265 (0.0007) [2023-12-27 12:42:13,163][20347] Updated weights for policy 0, policy_version 1017969 (0.0008) [2023-12-27 12:42:13,221][20347] Updated weights for policy 0, policy_version 1017979 (0.0007) [2023-12-27 12:42:13,269][20347] Updated weights for policy 0, policy_version 1017989 (0.0005) [2023-12-27 12:42:13,317][20381] Updated weights for policy 1, policy_version 1017275 (0.0006) [2023-12-27 12:42:13,367][20381] Updated weights for policy 1, policy_version 1017285 (0.0007) [2023-12-27 12:42:13,422][20381] Updated weights for policy 1, policy_version 1017295 (0.0010) [2023-12-27 12:42:13,900][20347] Updated weights for policy 0, policy_version 1017999 (0.0006) [2023-12-27 12:42:13,949][20347] Updated weights for policy 0, policy_version 1018009 (0.0007) [2023-12-27 12:42:13,993][20347] Updated weights for policy 0, policy_version 1018019 (0.0007) [2023-12-27 12:42:14,099][20381] Updated weights for policy 1, policy_version 1017305 (0.0010) [2023-12-27 12:42:14,151][20381] Updated weights for policy 1, policy_version 1017315 (0.0005) [2023-12-27 12:42:14,213][20381] Updated weights for policy 1, policy_version 1017325 (0.0005) [2023-12-27 12:42:14,273][20381] Updated weights for policy 1, policy_version 1017335 (0.0005) [2023-12-27 12:42:14,608][20347] Updated weights for policy 0, policy_version 1018029 (0.0007) [2023-12-27 12:42:14,668][20347] Updated weights for policy 0, policy_version 1018039 (0.0008) [2023-12-27 12:42:14,725][20347] Updated weights for policy 0, policy_version 1018049 (0.0006) [2023-12-27 12:42:14,872][20381] Updated weights for policy 1, policy_version 1017345 (0.0006) [2023-12-27 12:42:14,931][20381] Updated weights for policy 1, policy_version 1017355 (0.0007) [2023-12-27 12:42:14,991][20381] Updated weights for policy 1, policy_version 1017365 (0.0006) [2023-12-27 12:42:15,394][20347] Updated weights for policy 0, policy_version 1018059 (0.0006) [2023-12-27 12:42:15,458][20347] Updated weights for policy 0, policy_version 1018069 (0.0005) [2023-12-27 12:42:15,511][20347] Updated weights for policy 0, policy_version 1018079 (0.0009) [2023-12-27 12:42:15,587][20381] Updated weights for policy 1, policy_version 1017375 (0.0005) [2023-12-27 12:42:15,642][20381] Updated weights for policy 1, policy_version 1017385 (0.0005) [2023-12-27 12:42:15,701][20381] Updated weights for policy 1, policy_version 1017395 (0.0005) [2023-12-27 12:42:16,102][20347] Updated weights for policy 0, policy_version 1018089 (0.0010) [2023-12-27 12:42:16,158][20347] Updated weights for policy 0, policy_version 1018099 (0.0007) [2023-12-27 12:42:16,215][20347] Updated weights for policy 0, policy_version 1018109 (0.0005) [2023-12-27 12:42:16,271][20347] Updated weights for policy 0, policy_version 1018119 (0.0005) [2023-12-27 12:42:16,297][20381] Updated weights for policy 1, policy_version 1017405 (0.0007) [2023-12-27 12:42:16,356][20381] Updated weights for policy 1, policy_version 1017415 (0.0008) [2023-12-27 12:42:16,412][20381] Updated weights for policy 1, policy_version 1017425 (0.0008) [2023-12-27 12:42:16,876][20347] Updated weights for policy 0, policy_version 1018129 (0.0005) [2023-12-27 12:42:16,925][20347] Updated weights for policy 0, policy_version 1018139 (0.0006) [2023-12-27 12:42:16,973][20347] Updated weights for policy 0, policy_version 1018149 (0.0008) [2023-12-27 12:42:17,079][20381] Updated weights for policy 1, policy_version 1017435 (0.0008) [2023-12-27 12:42:17,140][20381] Updated weights for policy 1, policy_version 1017445 (0.0007) [2023-12-27 12:42:17,195][20381] Updated weights for policy 1, policy_version 1017455 (0.0008) [2023-12-27 12:42:17,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 521191424. Throughput: 0: 10446.9, 1: 10180.1. Samples: 521197764. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:17,364][19170] Avg episode reward: [(0, '912.020'), (1, '225.910')] [2023-12-27 12:42:17,612][20347] Updated weights for policy 0, policy_version 1018159 (0.0006) [2023-12-27 12:42:17,667][20347] Updated weights for policy 0, policy_version 1018169 (0.0005) [2023-12-27 12:42:17,723][20347] Updated weights for policy 0, policy_version 1018179 (0.0005) [2023-12-27 12:42:17,759][20381] Updated weights for policy 1, policy_version 1017465 (0.0007) [2023-12-27 12:42:17,809][20381] Updated weights for policy 1, policy_version 1017475 (0.0010) [2023-12-27 12:42:17,854][20381] Updated weights for policy 1, policy_version 1017485 (0.0010) [2023-12-27 12:42:17,907][20381] Updated weights for policy 1, policy_version 1017495 (0.0009) [2023-12-27 12:42:18,344][20347] Updated weights for policy 0, policy_version 1018189 (0.0006) [2023-12-27 12:42:18,401][20347] Updated weights for policy 0, policy_version 1018199 (0.0010) [2023-12-27 12:42:18,460][20347] Updated weights for policy 0, policy_version 1018209 (0.0010) [2023-12-27 12:42:18,518][20381] Updated weights for policy 1, policy_version 1017505 (0.0007) [2023-12-27 12:42:18,582][20381] Updated weights for policy 1, policy_version 1017515 (0.0008) [2023-12-27 12:42:18,648][20381] Updated weights for policy 1, policy_version 1017525 (0.0007) [2023-12-27 12:42:19,001][20347] Updated weights for policy 0, policy_version 1018219 (0.0009) [2023-12-27 12:42:19,057][20347] Updated weights for policy 0, policy_version 1018229 (0.0007) [2023-12-27 12:42:19,115][20347] Updated weights for policy 0, policy_version 1018239 (0.0010) [2023-12-27 12:42:19,252][20381] Updated weights for policy 1, policy_version 1017535 (0.0007) [2023-12-27 12:42:19,319][20381] Updated weights for policy 1, policy_version 1017545 (0.0007) [2023-12-27 12:42:19,390][20381] Updated weights for policy 1, policy_version 1017555 (0.0008) [2023-12-27 12:42:19,762][20347] Updated weights for policy 0, policy_version 1018249 (0.0009) [2023-12-27 12:42:19,813][20347] Updated weights for policy 0, policy_version 1018259 (0.0008) [2023-12-27 12:42:19,882][20347] Updated weights for policy 0, policy_version 1018269 (0.0008) [2023-12-27 12:42:19,950][20347] Updated weights for policy 0, policy_version 1018279 (0.0007) [2023-12-27 12:42:20,119][20381] Updated weights for policy 1, policy_version 1017565 (0.0008) [2023-12-27 12:42:20,172][20381] Updated weights for policy 1, policy_version 1017575 (0.0008) [2023-12-27 12:42:20,233][20381] Updated weights for policy 1, policy_version 1017585 (0.0007) [2023-12-27 12:42:20,645][20347] Updated weights for policy 0, policy_version 1018289 (0.0006) [2023-12-27 12:42:20,701][20347] Updated weights for policy 0, policy_version 1018299 (0.0008) [2023-12-27 12:42:20,767][20347] Updated weights for policy 0, policy_version 1018309 (0.0008) [2023-12-27 12:42:20,939][20381] Updated weights for policy 1, policy_version 1017595 (0.0008) [2023-12-27 12:42:20,992][20381] Updated weights for policy 1, policy_version 1017605 (0.0008) [2023-12-27 12:42:21,059][20381] Updated weights for policy 1, policy_version 1017615 (0.0008) [2023-12-27 12:42:21,514][20347] Updated weights for policy 0, policy_version 1018319 (0.0006) [2023-12-27 12:42:21,577][20347] Updated weights for policy 0, policy_version 1018329 (0.0007) [2023-12-27 12:42:21,641][20347] Updated weights for policy 0, policy_version 1018339 (0.0007) [2023-12-27 12:42:21,893][20381] Updated weights for policy 1, policy_version 1017625 (0.0008) [2023-12-27 12:42:21,957][20381] Updated weights for policy 1, policy_version 1017635 (0.0007) [2023-12-27 12:42:22,019][20381] Updated weights for policy 1, policy_version 1017645 (0.0007) [2023-12-27 12:42:22,083][20381] Updated weights for policy 1, policy_version 1017655 (0.0008) [2023-12-27 12:42:22,334][20347] Updated weights for policy 0, policy_version 1018349 (0.0008) [2023-12-27 12:42:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 521289728. Throughput: 0: 10495.2, 1: 10211.4. Samples: 521263264. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:22,363][19170] Avg episode reward: [(0, '941.420'), (1, '208.390')] [2023-12-27 12:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001017656_260546560.pth... [2023-12-27 12:42:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001016472_260243456.pth [2023-12-27 12:42:22,415][20347] Updated weights for policy 0, policy_version 1018359 (0.0008) [2023-12-27 12:42:22,480][20347] Updated weights for policy 0, policy_version 1018369 (0.0008) [2023-12-27 12:42:22,520][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001018376_260751360.pth... [2023-12-27 12:42:22,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001017128_260431872.pth [2023-12-27 12:42:22,778][20381] Updated weights for policy 1, policy_version 1017665 (0.0008) [2023-12-27 12:42:22,835][20381] Updated weights for policy 1, policy_version 1017675 (0.0008) [2023-12-27 12:42:22,894][20381] Updated weights for policy 1, policy_version 1017685 (0.0006) [2023-12-27 12:42:23,215][20347] Updated weights for policy 0, policy_version 1018379 (0.0007) [2023-12-27 12:42:23,270][20347] Updated weights for policy 0, policy_version 1018389 (0.0007) [2023-12-27 12:42:23,294][20291] KL-divergence is very high: 361.3397 [2023-12-27 12:42:23,299][20291] KL-divergence is very high: 359.2926 [2023-12-27 12:42:23,320][20347] Updated weights for policy 0, policy_version 1018399 (0.0009) [2023-12-27 12:42:23,476][20381] Updated weights for policy 1, policy_version 1017695 (0.0006) [2023-12-27 12:42:23,527][20381] Updated weights for policy 1, policy_version 1017705 (0.0005) [2023-12-27 12:42:23,579][20381] Updated weights for policy 1, policy_version 1017715 (0.0005) [2023-12-27 12:42:23,982][20347] Updated weights for policy 0, policy_version 1018409 (0.0008) [2023-12-27 12:42:24,029][20347] Updated weights for policy 0, policy_version 1018419 (0.0005) [2023-12-27 12:42:24,078][20347] Updated weights for policy 0, policy_version 1018429 (0.0005) [2023-12-27 12:42:24,132][20347] Updated weights for policy 0, policy_version 1018439 (0.0005) [2023-12-27 12:42:24,190][20381] Updated weights for policy 1, policy_version 1017725 (0.0009) [2023-12-27 12:42:24,252][20381] Updated weights for policy 1, policy_version 1017735 (0.0011) [2023-12-27 12:42:24,300][20381] Updated weights for policy 1, policy_version 1017745 (0.0008) [2023-12-27 12:42:24,798][20347] Updated weights for policy 0, policy_version 1018449 (0.0008) [2023-12-27 12:42:24,849][20347] Updated weights for policy 0, policy_version 1018459 (0.0008) [2023-12-27 12:42:24,897][20347] Updated weights for policy 0, policy_version 1018469 (0.0008) [2023-12-27 12:42:24,975][20381] Updated weights for policy 1, policy_version 1017755 (0.0009) [2023-12-27 12:42:25,035][20381] Updated weights for policy 1, policy_version 1017765 (0.0011) [2023-12-27 12:42:25,095][20381] Updated weights for policy 1, policy_version 1017775 (0.0008) [2023-12-27 12:42:25,584][20347] Updated weights for policy 0, policy_version 1018479 (0.0010) [2023-12-27 12:42:25,630][20381] Updated weights for policy 1, policy_version 1017785 (0.0006) [2023-12-27 12:42:25,635][20347] Updated weights for policy 0, policy_version 1018489 (0.0010) [2023-12-27 12:42:25,689][20381] Updated weights for policy 1, policy_version 1017795 (0.0011) [2023-12-27 12:42:25,693][20347] Updated weights for policy 0, policy_version 1018499 (0.0010) [2023-12-27 12:42:25,732][20381] Updated weights for policy 1, policy_version 1017805 (0.0010) [2023-12-27 12:42:25,784][20381] Updated weights for policy 1, policy_version 1017815 (0.0011) [2023-12-27 12:42:26,285][20347] Updated weights for policy 0, policy_version 1018509 (0.0007) [2023-12-27 12:42:26,342][20347] Updated weights for policy 0, policy_version 1018519 (0.0005) [2023-12-27 12:42:26,392][20347] Updated weights for policy 0, policy_version 1018529 (0.0005) [2023-12-27 12:42:26,486][20381] Updated weights for policy 1, policy_version 1017825 (0.0011) [2023-12-27 12:42:26,540][20381] Updated weights for policy 1, policy_version 1017835 (0.0011) [2023-12-27 12:42:26,597][20381] Updated weights for policy 1, policy_version 1017845 (0.0011) [2023-12-27 12:42:26,981][20347] Updated weights for policy 0, policy_version 1018539 (0.0006) [2023-12-27 12:42:27,032][20347] Updated weights for policy 0, policy_version 1018549 (0.0007) [2023-12-27 12:42:27,092][20347] Updated weights for policy 0, policy_version 1018559 (0.0007) [2023-12-27 12:42:27,301][20381] Updated weights for policy 1, policy_version 1017855 (0.0011) [2023-12-27 12:42:27,355][20381] Updated weights for policy 1, policy_version 1017865 (0.0007) [2023-12-27 12:42:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 521396224. Throughput: 0: 10462.8, 1: 10289.8. Samples: 521386360. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:27,363][19170] Avg episode reward: [(0, '941.420'), (1, '279.200')] [2023-12-27 12:42:27,409][20381] Updated weights for policy 1, policy_version 1017875 (0.0009) [2023-12-27 12:42:27,751][20347] Updated weights for policy 0, policy_version 1018569 (0.0006) [2023-12-27 12:42:27,797][20347] Updated weights for policy 0, policy_version 1018579 (0.0007) [2023-12-27 12:42:27,843][20347] Updated weights for policy 0, policy_version 1018589 (0.0007) [2023-12-27 12:42:27,890][20347] Updated weights for policy 0, policy_version 1018599 (0.0007) [2023-12-27 12:42:28,089][20381] Updated weights for policy 1, policy_version 1017885 (0.0010) [2023-12-27 12:42:28,147][20381] Updated weights for policy 1, policy_version 1017895 (0.0010) [2023-12-27 12:42:28,193][20381] Updated weights for policy 1, policy_version 1017905 (0.0010) [2023-12-27 12:42:28,505][20347] Updated weights for policy 0, policy_version 1018609 (0.0007) [2023-12-27 12:42:28,561][20347] Updated weights for policy 0, policy_version 1018619 (0.0008) [2023-12-27 12:42:28,615][20347] Updated weights for policy 0, policy_version 1018629 (0.0010) [2023-12-27 12:42:28,893][20381] Updated weights for policy 1, policy_version 1017915 (0.0008) [2023-12-27 12:42:28,948][20381] Updated weights for policy 1, policy_version 1017925 (0.0008) [2023-12-27 12:42:29,004][20381] Updated weights for policy 1, policy_version 1017935 (0.0005) [2023-12-27 12:42:29,286][20347] Updated weights for policy 0, policy_version 1018639 (0.0008) [2023-12-27 12:42:29,346][20347] Updated weights for policy 0, policy_version 1018649 (0.0008) [2023-12-27 12:42:29,394][20291] KL-divergence is very high: 105.0788 [2023-12-27 12:42:29,409][20347] Updated weights for policy 0, policy_version 1018660 (0.0009) [2023-12-27 12:42:29,679][20381] Updated weights for policy 1, policy_version 1017945 (0.0006) [2023-12-27 12:42:29,734][20381] Updated weights for policy 1, policy_version 1017955 (0.0008) [2023-12-27 12:42:29,793][20381] Updated weights for policy 1, policy_version 1017965 (0.0008) [2023-12-27 12:42:29,861][20381] Updated weights for policy 1, policy_version 1017975 (0.0009) [2023-12-27 12:42:30,113][20347] Updated weights for policy 0, policy_version 1018670 (0.0008) [2023-12-27 12:42:30,171][20347] Updated weights for policy 0, policy_version 1018680 (0.0007) [2023-12-27 12:42:30,227][20347] Updated weights for policy 0, policy_version 1018690 (0.0007) [2023-12-27 12:42:30,531][20381] Updated weights for policy 1, policy_version 1017985 (0.0011) [2023-12-27 12:42:30,583][20381] Updated weights for policy 1, policy_version 1017995 (0.0010) [2023-12-27 12:42:30,630][20381] Updated weights for policy 1, policy_version 1018005 (0.0005) [2023-12-27 12:42:30,903][20347] Updated weights for policy 0, policy_version 1018700 (0.0007) [2023-12-27 12:42:30,952][20347] Updated weights for policy 0, policy_version 1018710 (0.0007) [2023-12-27 12:42:31,008][20347] Updated weights for policy 0, policy_version 1018720 (0.0008) [2023-12-27 12:42:31,261][20381] Updated weights for policy 1, policy_version 1018015 (0.0005) [2023-12-27 12:42:31,324][20381] Updated weights for policy 1, policy_version 1018025 (0.0006) [2023-12-27 12:42:31,394][20381] Updated weights for policy 1, policy_version 1018035 (0.0007) [2023-12-27 12:42:31,691][20347] Updated weights for policy 0, policy_version 1018730 (0.0007) [2023-12-27 12:42:31,757][20347] Updated weights for policy 0, policy_version 1018740 (0.0007) [2023-12-27 12:42:31,818][20347] Updated weights for policy 0, policy_version 1018750 (0.0008) [2023-12-27 12:42:31,885][20347] Updated weights for policy 0, policy_version 1018760 (0.0008) [2023-12-27 12:42:31,997][20381] Updated weights for policy 1, policy_version 1018045 (0.0007) [2023-12-27 12:42:32,046][20381] Updated weights for policy 1, policy_version 1018055 (0.0007) [2023-12-27 12:42:32,096][20381] Updated weights for policy 1, policy_version 1018065 (0.0008) [2023-12-27 12:42:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 521502720. Throughput: 0: 10485.6, 1: 10261.4. Samples: 521511500. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:32,363][19170] Avg episode reward: [(0, '882.710'), (1, '301.100')] [2023-12-27 12:42:32,575][20347] Updated weights for policy 0, policy_version 1018770 (0.0006) [2023-12-27 12:42:32,646][20347] Updated weights for policy 0, policy_version 1018780 (0.0008) [2023-12-27 12:42:32,706][20347] Updated weights for policy 0, policy_version 1018790 (0.0008) [2023-12-27 12:42:32,754][20381] Updated weights for policy 1, policy_version 1018075 (0.0008) [2023-12-27 12:42:32,811][20381] Updated weights for policy 1, policy_version 1018085 (0.0006) [2023-12-27 12:42:32,871][20381] Updated weights for policy 1, policy_version 1018095 (0.0005) [2023-12-27 12:42:33,323][20347] Updated weights for policy 0, policy_version 1018800 (0.0010) [2023-12-27 12:42:33,378][20347] Updated weights for policy 0, policy_version 1018810 (0.0010) [2023-12-27 12:42:33,396][20381] Updated weights for policy 1, policy_version 1018105 (0.0005) [2023-12-27 12:42:33,426][20347] Updated weights for policy 0, policy_version 1018820 (0.0010) [2023-12-27 12:42:33,447][20381] Updated weights for policy 1, policy_version 1018115 (0.0005) [2023-12-27 12:42:33,501][20381] Updated weights for policy 1, policy_version 1018125 (0.0005) [2023-12-27 12:42:33,558][20381] Updated weights for policy 1, policy_version 1018135 (0.0007) [2023-12-27 12:42:34,035][20347] Updated weights for policy 0, policy_version 1018830 (0.0007) [2023-12-27 12:42:34,085][20347] Updated weights for policy 0, policy_version 1018840 (0.0005) [2023-12-27 12:42:34,134][20347] Updated weights for policy 0, policy_version 1018850 (0.0005) [2023-12-27 12:42:34,230][20381] Updated weights for policy 1, policy_version 1018145 (0.0008) [2023-12-27 12:42:34,284][20381] Updated weights for policy 1, policy_version 1018155 (0.0007) [2023-12-27 12:42:34,341][20381] Updated weights for policy 1, policy_version 1018165 (0.0008) [2023-12-27 12:42:34,830][20347] Updated weights for policy 0, policy_version 1018860 (0.0008) [2023-12-27 12:42:34,885][20347] Updated weights for policy 0, policy_version 1018870 (0.0007) [2023-12-27 12:42:34,939][20347] Updated weights for policy 0, policy_version 1018880 (0.0010) [2023-12-27 12:42:35,003][20381] Updated weights for policy 1, policy_version 1018175 (0.0007) [2023-12-27 12:42:35,067][20381] Updated weights for policy 1, policy_version 1018185 (0.0007) [2023-12-27 12:42:35,126][20381] Updated weights for policy 1, policy_version 1018195 (0.0006) [2023-12-27 12:42:35,549][20347] Updated weights for policy 0, policy_version 1018890 (0.0007) [2023-12-27 12:42:35,607][20347] Updated weights for policy 0, policy_version 1018900 (0.0006) [2023-12-27 12:42:35,661][20347] Updated weights for policy 0, policy_version 1018910 (0.0008) [2023-12-27 12:42:35,701][20381] Updated weights for policy 1, policy_version 1018205 (0.0005) [2023-12-27 12:42:35,714][20347] Updated weights for policy 0, policy_version 1018920 (0.0006) [2023-12-27 12:42:35,754][20381] Updated weights for policy 1, policy_version 1018215 (0.0007) [2023-12-27 12:42:35,812][20381] Updated weights for policy 1, policy_version 1018225 (0.0008) [2023-12-27 12:42:36,382][20347] Updated weights for policy 0, policy_version 1018930 (0.0007) [2023-12-27 12:42:36,444][20347] Updated weights for policy 0, policy_version 1018940 (0.0007) [2023-12-27 12:42:36,512][20347] Updated weights for policy 0, policy_version 1018950 (0.0007) [2023-12-27 12:42:36,533][20381] Updated weights for policy 1, policy_version 1018235 (0.0008) [2023-12-27 12:42:36,586][20381] Updated weights for policy 1, policy_version 1018245 (0.0007) [2023-12-27 12:42:36,637][20381] Updated weights for policy 1, policy_version 1018255 (0.0006) [2023-12-27 12:42:37,223][20347] Updated weights for policy 0, policy_version 1018960 (0.0007) [2023-12-27 12:42:37,281][20381] Updated weights for policy 1, policy_version 1018265 (0.0007) [2023-12-27 12:42:37,284][20347] Updated weights for policy 0, policy_version 1018970 (0.0008) [2023-12-27 12:42:37,333][20347] Updated weights for policy 0, policy_version 1018980 (0.0008) [2023-12-27 12:42:37,336][20381] Updated weights for policy 1, policy_version 1018275 (0.0007) [2023-12-27 12:42:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 521609216. Throughput: 0: 10505.8, 1: 10324.3. Samples: 521576888. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:37,363][19170] Avg episode reward: [(0, '911.900'), (1, '279.420')] [2023-12-27 12:42:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001018984_260907008.pth... [2023-12-27 12:42:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001017768_260595712.pth [2023-12-27 12:42:37,391][20381] Updated weights for policy 1, policy_version 1018285 (0.0010) [2023-12-27 12:42:37,450][20381] Updated weights for policy 1, policy_version 1018295 (0.0009) [2023-12-27 12:42:37,454][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001018296_260710400.pth... [2023-12-27 12:42:37,457][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001017048_260390912.pth [2023-12-27 12:42:38,040][20347] Updated weights for policy 0, policy_version 1018990 (0.0007) [2023-12-27 12:42:38,049][20381] Updated weights for policy 1, policy_version 1018305 (0.0006) [2023-12-27 12:42:38,085][20347] Updated weights for policy 0, policy_version 1019000 (0.0007) [2023-12-27 12:42:38,106][20381] Updated weights for policy 1, policy_version 1018315 (0.0005) [2023-12-27 12:42:38,132][20347] Updated weights for policy 0, policy_version 1019010 (0.0008) [2023-12-27 12:42:38,153][20381] Updated weights for policy 1, policy_version 1018325 (0.0005) [2023-12-27 12:42:38,830][20381] Updated weights for policy 1, policy_version 1018335 (0.0006) [2023-12-27 12:42:38,854][20347] Updated weights for policy 0, policy_version 1019020 (0.0008) [2023-12-27 12:42:38,899][20381] Updated weights for policy 1, policy_version 1018345 (0.0007) [2023-12-27 12:42:38,906][20347] Updated weights for policy 0, policy_version 1019030 (0.0007) [2023-12-27 12:42:38,951][20347] Updated weights for policy 0, policy_version 1019040 (0.0007) [2023-12-27 12:42:38,962][20381] Updated weights for policy 1, policy_version 1018355 (0.0008) [2023-12-27 12:42:39,699][20381] Updated weights for policy 1, policy_version 1018365 (0.0007) [2023-12-27 12:42:39,717][20347] Updated weights for policy 0, policy_version 1019050 (0.0008) [2023-12-27 12:42:39,754][20381] Updated weights for policy 1, policy_version 1018375 (0.0006) [2023-12-27 12:42:39,775][20347] Updated weights for policy 0, policy_version 1019060 (0.0006) [2023-12-27 12:42:39,815][20381] Updated weights for policy 1, policy_version 1018385 (0.0007) [2023-12-27 12:42:39,835][20347] Updated weights for policy 0, policy_version 1019070 (0.0007) [2023-12-27 12:42:39,893][20347] Updated weights for policy 0, policy_version 1019080 (0.0008) [2023-12-27 12:42:40,542][20381] Updated weights for policy 1, policy_version 1018395 (0.0007) [2023-12-27 12:42:40,596][20381] Updated weights for policy 1, policy_version 1018405 (0.0007) [2023-12-27 12:42:40,600][20347] Updated weights for policy 0, policy_version 1019090 (0.0005) [2023-12-27 12:42:40,647][20347] Updated weights for policy 0, policy_version 1019100 (0.0005) [2023-12-27 12:42:40,652][20381] Updated weights for policy 1, policy_version 1018415 (0.0008) [2023-12-27 12:42:40,701][20347] Updated weights for policy 0, policy_version 1019110 (0.0005) [2023-12-27 12:42:41,375][20347] Updated weights for policy 0, policy_version 1019120 (0.0007) [2023-12-27 12:42:41,428][20381] Updated weights for policy 1, policy_version 1018425 (0.0008) [2023-12-27 12:42:41,446][20347] Updated weights for policy 0, policy_version 1019130 (0.0007) [2023-12-27 12:42:41,483][20381] Updated weights for policy 1, policy_version 1018435 (0.0006) [2023-12-27 12:42:41,505][20347] Updated weights for policy 0, policy_version 1019140 (0.0006) [2023-12-27 12:42:41,537][20381] Updated weights for policy 1, policy_version 1018445 (0.0007) [2023-12-27 12:42:41,598][20381] Updated weights for policy 1, policy_version 1018455 (0.0010) [2023-12-27 12:42:42,243][20347] Updated weights for policy 0, policy_version 1019150 (0.0008) [2023-12-27 12:42:42,297][20381] Updated weights for policy 1, policy_version 1018465 (0.0008) [2023-12-27 12:42:42,307][20347] Updated weights for policy 0, policy_version 1019160 (0.0007) [2023-12-27 12:42:42,359][20381] Updated weights for policy 1, policy_version 1018475 (0.0008) [2023-12-27 12:42:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 521699328. Throughput: 0: 10436.5, 1: 10348.6. Samples: 521698856. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:42,363][19170] Avg episode reward: [(0, '951.200'), (1, '221.540')] [2023-12-27 12:42:42,374][20347] Updated weights for policy 0, policy_version 1019170 (0.0008) [2023-12-27 12:42:42,423][20381] Updated weights for policy 1, policy_version 1018485 (0.0007) [2023-12-27 12:42:43,057][20381] Updated weights for policy 1, policy_version 1018495 (0.0009) [2023-12-27 12:42:43,098][20347] Updated weights for policy 0, policy_version 1019180 (0.0007) [2023-12-27 12:42:43,112][20381] Updated weights for policy 1, policy_version 1018505 (0.0010) [2023-12-27 12:42:43,154][20347] Updated weights for policy 0, policy_version 1019190 (0.0005) [2023-12-27 12:42:43,163][20381] Updated weights for policy 1, policy_version 1018515 (0.0010) [2023-12-27 12:42:43,207][20347] Updated weights for policy 0, policy_version 1019200 (0.0006) [2023-12-27 12:42:43,814][20381] Updated weights for policy 1, policy_version 1018525 (0.0006) [2023-12-27 12:42:43,864][20381] Updated weights for policy 1, policy_version 1018535 (0.0005) [2023-12-27 12:42:43,897][20347] Updated weights for policy 0, policy_version 1019210 (0.0008) [2023-12-27 12:42:43,923][20381] Updated weights for policy 1, policy_version 1018545 (0.0005) [2023-12-27 12:42:43,955][20347] Updated weights for policy 0, policy_version 1019220 (0.0008) [2023-12-27 12:42:44,010][20347] Updated weights for policy 0, policy_version 1019230 (0.0008) [2023-12-27 12:42:44,076][20347] Updated weights for policy 0, policy_version 1019240 (0.0005) [2023-12-27 12:42:44,568][20381] Updated weights for policy 1, policy_version 1018555 (0.0006) [2023-12-27 12:42:44,617][20381] Updated weights for policy 1, policy_version 1018565 (0.0007) [2023-12-27 12:42:44,669][20381] Updated weights for policy 1, policy_version 1018575 (0.0008) [2023-12-27 12:42:44,735][20347] Updated weights for policy 0, policy_version 1019250 (0.0007) [2023-12-27 12:42:44,795][20347] Updated weights for policy 0, policy_version 1019260 (0.0008) [2023-12-27 12:42:44,852][20347] Updated weights for policy 0, policy_version 1019270 (0.0007) [2023-12-27 12:42:45,359][20381] Updated weights for policy 1, policy_version 1018585 (0.0006) [2023-12-27 12:42:45,415][20381] Updated weights for policy 1, policy_version 1018595 (0.0008) [2023-12-27 12:42:45,470][20381] Updated weights for policy 1, policy_version 1018605 (0.0008) [2023-12-27 12:42:45,523][20381] Updated weights for policy 1, policy_version 1018615 (0.0006) [2023-12-27 12:42:45,574][20347] Updated weights for policy 0, policy_version 1019280 (0.0008) [2023-12-27 12:42:45,630][20347] Updated weights for policy 0, policy_version 1019290 (0.0008) [2023-12-27 12:42:45,678][20347] Updated weights for policy 0, policy_version 1019300 (0.0009) [2023-12-27 12:42:46,098][20381] Updated weights for policy 1, policy_version 1018625 (0.0008) [2023-12-27 12:42:46,153][20381] Updated weights for policy 1, policy_version 1018635 (0.0009) [2023-12-27 12:42:46,197][20381] Updated weights for policy 1, policy_version 1018645 (0.0008) [2023-12-27 12:42:46,349][20347] Updated weights for policy 0, policy_version 1019310 (0.0008) [2023-12-27 12:42:46,407][20347] Updated weights for policy 0, policy_version 1019320 (0.0008) [2023-12-27 12:42:46,464][20347] Updated weights for policy 0, policy_version 1019330 (0.0007) [2023-12-27 12:42:46,849][20381] Updated weights for policy 1, policy_version 1018655 (0.0008) [2023-12-27 12:42:46,897][20381] Updated weights for policy 1, policy_version 1018665 (0.0007) [2023-12-27 12:42:46,948][20381] Updated weights for policy 1, policy_version 1018675 (0.0007) [2023-12-27 12:42:47,187][20347] Updated weights for policy 0, policy_version 1019340 (0.0007) [2023-12-27 12:42:47,240][20347] Updated weights for policy 0, policy_version 1019350 (0.0007) [2023-12-27 12:42:47,298][20347] Updated weights for policy 0, policy_version 1019360 (0.0008) [2023-12-27 12:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20660.8). Total num frames: 521814016. Throughput: 0: 10363.9, 1: 10416.0. Samples: 521820860. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:47,363][19170] Avg episode reward: [(0, '941.520'), (1, '244.670')] [2023-12-27 12:42:47,622][20381] Updated weights for policy 1, policy_version 1018685 (0.0008) [2023-12-27 12:42:47,679][20381] Updated weights for policy 1, policy_version 1018695 (0.0008) [2023-12-27 12:42:47,737][20381] Updated weights for policy 1, policy_version 1018705 (0.0006) [2023-12-27 12:42:47,965][20347] Updated weights for policy 0, policy_version 1019370 (0.0007) [2023-12-27 12:42:48,016][20347] Updated weights for policy 0, policy_version 1019380 (0.0007) [2023-12-27 12:42:48,060][20347] Updated weights for policy 0, policy_version 1019390 (0.0006) [2023-12-27 12:42:48,112][20347] Updated weights for policy 0, policy_version 1019400 (0.0008) [2023-12-27 12:42:48,387][20381] Updated weights for policy 1, policy_version 1018715 (0.0006) [2023-12-27 12:42:48,437][20381] Updated weights for policy 1, policy_version 1018725 (0.0008) [2023-12-27 12:42:48,490][20381] Updated weights for policy 1, policy_version 1018735 (0.0008) [2023-12-27 12:42:48,855][20347] Updated weights for policy 0, policy_version 1019410 (0.0005) [2023-12-27 12:42:48,914][20347] Updated weights for policy 0, policy_version 1019420 (0.0006) [2023-12-27 12:42:48,973][20347] Updated weights for policy 0, policy_version 1019430 (0.0008) [2023-12-27 12:42:49,223][20381] Updated weights for policy 1, policy_version 1018745 (0.0008) [2023-12-27 12:42:49,289][20381] Updated weights for policy 1, policy_version 1018755 (0.0007) [2023-12-27 12:42:49,347][20381] Updated weights for policy 1, policy_version 1018765 (0.0007) [2023-12-27 12:42:49,413][20381] Updated weights for policy 1, policy_version 1018775 (0.0008) [2023-12-27 12:42:49,665][20347] Updated weights for policy 0, policy_version 1019440 (0.0008) [2023-12-27 12:42:49,724][20347] Updated weights for policy 0, policy_version 1019450 (0.0007) [2023-12-27 12:42:49,776][20347] Updated weights for policy 0, policy_version 1019460 (0.0007) [2023-12-27 12:42:50,141][20381] Updated weights for policy 1, policy_version 1018785 (0.0007) [2023-12-27 12:42:50,188][20381] Updated weights for policy 1, policy_version 1018795 (0.0007) [2023-12-27 12:42:50,233][20381] Updated weights for policy 1, policy_version 1018805 (0.0007) [2023-12-27 12:42:50,545][20347] Updated weights for policy 0, policy_version 1019470 (0.0008) [2023-12-27 12:42:50,609][20347] Updated weights for policy 0, policy_version 1019480 (0.0008) [2023-12-27 12:42:50,663][20347] Updated weights for policy 0, policy_version 1019490 (0.0007) [2023-12-27 12:42:50,945][20381] Updated weights for policy 1, policy_version 1018815 (0.0007) [2023-12-27 12:42:51,000][20381] Updated weights for policy 1, policy_version 1018825 (0.0007) [2023-12-27 12:42:51,065][20381] Updated weights for policy 1, policy_version 1018835 (0.0008) [2023-12-27 12:42:51,420][20347] Updated weights for policy 0, policy_version 1019500 (0.0007) [2023-12-27 12:42:51,480][20347] Updated weights for policy 0, policy_version 1019510 (0.0005) [2023-12-27 12:42:51,542][20347] Updated weights for policy 0, policy_version 1019520 (0.0005) [2023-12-27 12:42:51,790][20381] Updated weights for policy 1, policy_version 1018845 (0.0007) [2023-12-27 12:42:51,844][20381] Updated weights for policy 1, policy_version 1018855 (0.0006) [2023-12-27 12:42:51,900][20381] Updated weights for policy 1, policy_version 1018865 (0.0008) [2023-12-27 12:42:52,160][20347] Updated weights for policy 0, policy_version 1019530 (0.0007) [2023-12-27 12:42:52,216][20347] Updated weights for policy 0, policy_version 1019540 (0.0008) [2023-12-27 12:42:52,277][20347] Updated weights for policy 0, policy_version 1019550 (0.0008) [2023-12-27 12:42:52,329][20347] Updated weights for policy 0, policy_version 1019560 (0.0006) [2023-12-27 12:42:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 521912320. Throughput: 0: 10293.9, 1: 10387.3. Samples: 521880264. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:52,363][19170] Avg episode reward: [(0, '931.880'), (1, '185.130')] [2023-12-27 12:42:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001018872_260857856.pth... [2023-12-27 12:42:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001019560_261054464.pth... [2023-12-27 12:42:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001017656_260546560.pth [2023-12-27 12:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001018376_260751360.pth [2023-12-27 12:42:52,613][20381] Updated weights for policy 1, policy_version 1018875 (0.0008) [2023-12-27 12:42:52,672][20381] Updated weights for policy 1, policy_version 1018885 (0.0008) [2023-12-27 12:42:52,727][20381] Updated weights for policy 1, policy_version 1018895 (0.0008) [2023-12-27 12:42:52,997][20347] Updated weights for policy 0, policy_version 1019570 (0.0008) [2023-12-27 12:42:53,048][20347] Updated weights for policy 0, policy_version 1019580 (0.0008) [2023-12-27 12:42:53,098][20347] Updated weights for policy 0, policy_version 1019590 (0.0007) [2023-12-27 12:42:53,369][20381] Updated weights for policy 1, policy_version 1018905 (0.0008) [2023-12-27 12:42:53,422][20381] Updated weights for policy 1, policy_version 1018915 (0.0008) [2023-12-27 12:42:53,469][20381] Updated weights for policy 1, policy_version 1018925 (0.0007) [2023-12-27 12:42:53,517][20381] Updated weights for policy 1, policy_version 1018935 (0.0009) [2023-12-27 12:42:53,793][20347] Updated weights for policy 0, policy_version 1019600 (0.0010) [2023-12-27 12:42:53,857][20347] Updated weights for policy 0, policy_version 1019610 (0.0010) [2023-12-27 12:42:53,920][20347] Updated weights for policy 0, policy_version 1019620 (0.0010) [2023-12-27 12:42:54,265][20381] Updated weights for policy 1, policy_version 1018945 (0.0008) [2023-12-27 12:42:54,320][20381] Updated weights for policy 1, policy_version 1018955 (0.0008) [2023-12-27 12:42:54,385][20381] Updated weights for policy 1, policy_version 1018965 (0.0009) [2023-12-27 12:42:54,561][20347] Updated weights for policy 0, policy_version 1019630 (0.0011) [2023-12-27 12:42:54,615][20347] Updated weights for policy 0, policy_version 1019640 (0.0011) [2023-12-27 12:42:54,677][20347] Updated weights for policy 0, policy_version 1019650 (0.0008) [2023-12-27 12:42:55,096][20381] Updated weights for policy 1, policy_version 1018975 (0.0006) [2023-12-27 12:42:55,161][20381] Updated weights for policy 1, policy_version 1018985 (0.0006) [2023-12-27 12:42:55,222][20381] Updated weights for policy 1, policy_version 1018995 (0.0006) [2023-12-27 12:42:55,331][20347] Updated weights for policy 0, policy_version 1019660 (0.0006) [2023-12-27 12:42:55,395][20347] Updated weights for policy 0, policy_version 1019670 (0.0007) [2023-12-27 12:42:55,455][20347] Updated weights for policy 0, policy_version 1019680 (0.0007) [2023-12-27 12:42:55,869][20381] Updated weights for policy 1, policy_version 1019005 (0.0007) [2023-12-27 12:42:55,917][20381] Updated weights for policy 1, policy_version 1019015 (0.0009) [2023-12-27 12:42:55,967][20381] Updated weights for policy 1, policy_version 1019025 (0.0009) [2023-12-27 12:42:55,982][20347] Updated weights for policy 0, policy_version 1019690 (0.0005) [2023-12-27 12:42:56,038][20347] Updated weights for policy 0, policy_version 1019700 (0.0005) [2023-12-27 12:42:56,095][20347] Updated weights for policy 0, policy_version 1019710 (0.0006) [2023-12-27 12:42:56,151][20347] Updated weights for policy 0, policy_version 1019720 (0.0008) [2023-12-27 12:42:56,594][20381] Updated weights for policy 1, policy_version 1019035 (0.0007) [2023-12-27 12:42:56,657][20381] Updated weights for policy 1, policy_version 1019045 (0.0005) [2023-12-27 12:42:56,711][20381] Updated weights for policy 1, policy_version 1019055 (0.0007) [2023-12-27 12:42:56,723][20347] Updated weights for policy 0, policy_version 1019730 (0.0006) [2023-12-27 12:42:56,770][20347] Updated weights for policy 0, policy_version 1019740 (0.0008) [2023-12-27 12:42:56,827][20347] Updated weights for policy 0, policy_version 1019750 (0.0008) [2023-12-27 12:42:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 522010624. Throughput: 0: 10310.1, 1: 10376.2. Samples: 522002404. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:42:57,363][19170] Avg episode reward: [(0, '951.450'), (1, '241.630')] [2023-12-27 12:42:57,381][20381] Updated weights for policy 1, policy_version 1019065 (0.0007) [2023-12-27 12:42:57,436][20381] Updated weights for policy 1, policy_version 1019075 (0.0007) [2023-12-27 12:42:57,471][20347] Updated weights for policy 0, policy_version 1019760 (0.0006) [2023-12-27 12:42:57,492][20381] Updated weights for policy 1, policy_version 1019085 (0.0006) [2023-12-27 12:42:57,524][20347] Updated weights for policy 0, policy_version 1019770 (0.0006) [2023-12-27 12:42:57,549][20381] Updated weights for policy 1, policy_version 1019095 (0.0008) [2023-12-27 12:42:57,575][20347] Updated weights for policy 0, policy_version 1019780 (0.0007) [2023-12-27 12:42:58,219][20347] Updated weights for policy 0, policy_version 1019790 (0.0007) [2023-12-27 12:42:58,274][20347] Updated weights for policy 0, policy_version 1019800 (0.0007) [2023-12-27 12:42:58,296][20381] Updated weights for policy 1, policy_version 1019105 (0.0006) [2023-12-27 12:42:58,353][20347] Updated weights for policy 0, policy_version 1019810 (0.0007) [2023-12-27 12:42:58,359][20381] Updated weights for policy 1, policy_version 1019115 (0.0008) [2023-12-27 12:42:58,422][20381] Updated weights for policy 1, policy_version 1019125 (0.0008) [2023-12-27 12:42:59,106][20347] Updated weights for policy 0, policy_version 1019820 (0.0008) [2023-12-27 12:42:59,169][20347] Updated weights for policy 0, policy_version 1019830 (0.0007) [2023-12-27 12:42:59,189][20381] Updated weights for policy 1, policy_version 1019135 (0.0008) [2023-12-27 12:42:59,226][20347] Updated weights for policy 0, policy_version 1019840 (0.0006) [2023-12-27 12:42:59,259][20381] Updated weights for policy 1, policy_version 1019146 (0.0007) [2023-12-27 12:42:59,320][20381] Updated weights for policy 1, policy_version 1019156 (0.0007) [2023-12-27 12:42:59,918][20347] Updated weights for policy 0, policy_version 1019850 (0.0008) [2023-12-27 12:42:59,939][20381] Updated weights for policy 1, policy_version 1019166 (0.0008) [2023-12-27 12:42:59,974][20347] Updated weights for policy 0, policy_version 1019860 (0.0009) [2023-12-27 12:42:59,991][20381] Updated weights for policy 1, policy_version 1019176 (0.0006) [2023-12-27 12:43:00,030][20347] Updated weights for policy 0, policy_version 1019870 (0.0007) [2023-12-27 12:43:00,045][20381] Updated weights for policy 1, policy_version 1019186 (0.0006) [2023-12-27 12:43:00,086][20347] Updated weights for policy 0, policy_version 1019880 (0.0009) [2023-12-27 12:43:00,707][20381] Updated weights for policy 1, policy_version 1019196 (0.0007) [2023-12-27 12:43:00,764][20381] Updated weights for policy 1, policy_version 1019206 (0.0007) [2023-12-27 12:43:00,793][20347] Updated weights for policy 0, policy_version 1019890 (0.0009) [2023-12-27 12:43:00,822][20381] Updated weights for policy 1, policy_version 1019216 (0.0006) [2023-12-27 12:43:00,846][20347] Updated weights for policy 0, policy_version 1019900 (0.0005) [2023-12-27 12:43:00,880][20291] KL-divergence is very high: 176.2426 [2023-12-27 12:43:00,894][20347] Updated weights for policy 0, policy_version 1019910 (0.0005) [2023-12-27 12:43:01,474][20381] Updated weights for policy 1, policy_version 1019226 (0.0005) [2023-12-27 12:43:01,532][20381] Updated weights for policy 1, policy_version 1019236 (0.0006) [2023-12-27 12:43:01,546][20347] Updated weights for policy 0, policy_version 1019920 (0.0007) [2023-12-27 12:43:01,588][20381] Updated weights for policy 1, policy_version 1019246 (0.0007) [2023-12-27 12:43:01,607][20347] Updated weights for policy 0, policy_version 1019930 (0.0008) [2023-12-27 12:43:01,650][20381] Updated weights for policy 1, policy_version 1019256 (0.0008) [2023-12-27 12:43:01,665][20347] Updated weights for policy 0, policy_version 1019940 (0.0008) [2023-12-27 12:43:02,269][20381] Updated weights for policy 1, policy_version 1019266 (0.0009) [2023-12-27 12:43:02,317][20381] Updated weights for policy 1, policy_version 1019276 (0.0007) [2023-12-27 12:43:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 522108928. Throughput: 0: 10272.9, 1: 10358.5. Samples: 522126176. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:43:02,363][19170] Avg episode reward: [(0, '932.770'), (1, '275.920')] [2023-12-27 12:43:02,371][20347] Updated weights for policy 0, policy_version 1019950 (0.0007) [2023-12-27 12:43:02,377][20381] Updated weights for policy 1, policy_version 1019286 (0.0008) [2023-12-27 12:43:02,429][20347] Updated weights for policy 0, policy_version 1019960 (0.0008) [2023-12-27 12:43:02,482][20347] Updated weights for policy 0, policy_version 1019970 (0.0007) [2023-12-27 12:43:02,981][20381] Updated weights for policy 1, policy_version 1019296 (0.0006) [2023-12-27 12:43:03,041][20381] Updated weights for policy 1, policy_version 1019306 (0.0005) [2023-12-27 12:43:03,093][20381] Updated weights for policy 1, policy_version 1019316 (0.0005) [2023-12-27 12:43:03,115][20347] Updated weights for policy 0, policy_version 1019980 (0.0006) [2023-12-27 12:43:03,171][20347] Updated weights for policy 0, policy_version 1019990 (0.0005) [2023-12-27 12:43:03,217][20347] Updated weights for policy 0, policy_version 1020000 (0.0005) [2023-12-27 12:43:03,702][20381] Updated weights for policy 1, policy_version 1019326 (0.0008) [2023-12-27 12:43:03,749][20381] Updated weights for policy 1, policy_version 1019336 (0.0010) [2023-12-27 12:43:03,793][20381] Updated weights for policy 1, policy_version 1019346 (0.0010) [2023-12-27 12:43:03,819][20347] Updated weights for policy 0, policy_version 1020010 (0.0006) [2023-12-27 12:43:03,886][20347] Updated weights for policy 0, policy_version 1020020 (0.0010) [2023-12-27 12:43:03,945][20347] Updated weights for policy 0, policy_version 1020030 (0.0009) [2023-12-27 12:43:04,007][20347] Updated weights for policy 0, policy_version 1020040 (0.0010) [2023-12-27 12:43:04,482][20381] Updated weights for policy 1, policy_version 1019356 (0.0008) [2023-12-27 12:43:04,535][20381] Updated weights for policy 1, policy_version 1019366 (0.0005) [2023-12-27 12:43:04,604][20381] Updated weights for policy 1, policy_version 1019376 (0.0007) [2023-12-27 12:43:04,675][20347] Updated weights for policy 0, policy_version 1020050 (0.0005) [2023-12-27 12:43:04,732][20347] Updated weights for policy 0, policy_version 1020060 (0.0005) [2023-12-27 12:43:04,789][20347] Updated weights for policy 0, policy_version 1020070 (0.0005) [2023-12-27 12:43:05,180][20381] Updated weights for policy 1, policy_version 1019386 (0.0008) [2023-12-27 12:43:05,241][20381] Updated weights for policy 1, policy_version 1019396 (0.0005) [2023-12-27 12:43:05,298][20381] Updated weights for policy 1, policy_version 1019406 (0.0005) [2023-12-27 12:43:05,359][20381] Updated weights for policy 1, policy_version 1019416 (0.0006) [2023-12-27 12:43:05,431][20347] Updated weights for policy 0, policy_version 1020080 (0.0006) [2023-12-27 12:43:05,484][20347] Updated weights for policy 0, policy_version 1020090 (0.0007) [2023-12-27 12:43:05,541][20347] Updated weights for policy 0, policy_version 1020100 (0.0007) [2023-12-27 12:43:05,890][20381] Updated weights for policy 1, policy_version 1019426 (0.0005) [2023-12-27 12:43:05,947][20381] Updated weights for policy 1, policy_version 1019436 (0.0005) [2023-12-27 12:43:06,010][20381] Updated weights for policy 1, policy_version 1019446 (0.0005) [2023-12-27 12:43:06,174][20347] Updated weights for policy 0, policy_version 1020110 (0.0008) [2023-12-27 12:43:06,236][20347] Updated weights for policy 0, policy_version 1020120 (0.0008) [2023-12-27 12:43:06,297][20347] Updated weights for policy 0, policy_version 1020130 (0.0008) [2023-12-27 12:43:06,652][20381] Updated weights for policy 1, policy_version 1019456 (0.0010) [2023-12-27 12:43:06,715][20381] Updated weights for policy 1, policy_version 1019466 (0.0011) [2023-12-27 12:43:06,771][20381] Updated weights for policy 1, policy_version 1019476 (0.0011) [2023-12-27 12:43:06,986][20347] Updated weights for policy 0, policy_version 1020140 (0.0007) [2023-12-27 12:43:07,048][20347] Updated weights for policy 0, policy_version 1020150 (0.0006) [2023-12-27 12:43:07,079][20291] KL-divergence is very high: 126.8457 [2023-12-27 12:43:07,120][20347] Updated weights for policy 0, policy_version 1020160 (0.0008) [2023-12-27 12:43:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 522223616. Throughput: 0: 10257.8, 1: 10382.5. Samples: 522192076. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:43:07,363][19170] Avg episode reward: [(0, '903.450'), (1, '295.350')] [2023-12-27 12:43:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001019480_261013504.pth... [2023-12-27 12:43:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001020168_261210112.pth... [2023-12-27 12:43:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001018296_260710400.pth [2023-12-27 12:43:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001018984_260907008.pth [2023-12-27 12:43:07,440][20381] Updated weights for policy 1, policy_version 1019486 (0.0011) [2023-12-27 12:43:07,501][20381] Updated weights for policy 1, policy_version 1019496 (0.0011) [2023-12-27 12:43:07,561][20381] Updated weights for policy 1, policy_version 1019506 (0.0010) [2023-12-27 12:43:07,730][20347] Updated weights for policy 0, policy_version 1020170 (0.0006) [2023-12-27 12:43:07,778][20347] Updated weights for policy 0, policy_version 1020180 (0.0010) [2023-12-27 12:43:07,823][20347] Updated weights for policy 0, policy_version 1020190 (0.0010) [2023-12-27 12:43:07,868][20347] Updated weights for policy 0, policy_version 1020200 (0.0010) [2023-12-27 12:43:08,208][20381] Updated weights for policy 1, policy_version 1019516 (0.0009) [2023-12-27 12:43:08,254][20381] Updated weights for policy 1, policy_version 1019526 (0.0005) [2023-12-27 12:43:08,309][20381] Updated weights for policy 1, policy_version 1019536 (0.0008) [2023-12-27 12:43:08,553][20347] Updated weights for policy 0, policy_version 1020210 (0.0011) [2023-12-27 12:43:08,618][20347] Updated weights for policy 0, policy_version 1020220 (0.0011) [2023-12-27 12:43:08,706][20347] Updated weights for policy 0, policy_version 1020230 (0.0010) [2023-12-27 12:43:08,992][20381] Updated weights for policy 1, policy_version 1019547 (0.0008) [2023-12-27 12:43:09,046][20381] Updated weights for policy 1, policy_version 1019557 (0.0007) [2023-12-27 12:43:09,098][20381] Updated weights for policy 1, policy_version 1019567 (0.0007) [2023-12-27 12:43:09,381][20347] Updated weights for policy 0, policy_version 1020240 (0.0008) [2023-12-27 12:43:09,454][20347] Updated weights for policy 0, policy_version 1020250 (0.0007) [2023-12-27 12:43:09,522][20347] Updated weights for policy 0, policy_version 1020260 (0.0008) [2023-12-27 12:43:09,819][20381] Updated weights for policy 1, policy_version 1019577 (0.0006) [2023-12-27 12:43:09,885][20381] Updated weights for policy 1, policy_version 1019587 (0.0008) [2023-12-27 12:43:09,935][20381] Updated weights for policy 1, policy_version 1019597 (0.0007) [2023-12-27 12:43:09,988][20381] Updated weights for policy 1, policy_version 1019607 (0.0008) [2023-12-27 12:43:10,251][20347] Updated weights for policy 0, policy_version 1020270 (0.0006) [2023-12-27 12:43:10,309][20347] Updated weights for policy 0, policy_version 1020280 (0.0005) [2023-12-27 12:43:10,372][20347] Updated weights for policy 0, policy_version 1020290 (0.0007) [2023-12-27 12:43:10,681][20381] Updated weights for policy 1, policy_version 1019617 (0.0008) [2023-12-27 12:43:10,737][20381] Updated weights for policy 1, policy_version 1019627 (0.0007) [2023-12-27 12:43:10,783][20381] Updated weights for policy 1, policy_version 1019637 (0.0008) [2023-12-27 12:43:11,029][20347] Updated weights for policy 0, policy_version 1020300 (0.0007) [2023-12-27 12:43:11,091][20347] Updated weights for policy 0, policy_version 1020310 (0.0009) [2023-12-27 12:43:11,155][20347] Updated weights for policy 0, policy_version 1020320 (0.0007) [2023-12-27 12:43:11,521][20381] Updated weights for policy 1, policy_version 1019647 (0.0008) [2023-12-27 12:43:11,581][20381] Updated weights for policy 1, policy_version 1019657 (0.0008) [2023-12-27 12:43:11,646][20381] Updated weights for policy 1, policy_version 1019667 (0.0008) [2023-12-27 12:43:11,938][20347] Updated weights for policy 0, policy_version 1020330 (0.0007) [2023-12-27 12:43:11,992][20347] Updated weights for policy 0, policy_version 1020340 (0.0008) [2023-12-27 12:43:12,047][20347] Updated weights for policy 0, policy_version 1020350 (0.0007) [2023-12-27 12:43:12,102][20347] Updated weights for policy 0, policy_version 1020360 (0.0007) [2023-12-27 12:43:12,348][20381] Updated weights for policy 1, policy_version 1019677 (0.0009) [2023-12-27 12:43:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 522321920. Throughput: 0: 10275.4, 1: 10365.1. Samples: 522315188. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:43:12,363][19170] Avg episode reward: [(0, '922.130'), (1, '234.460')] [2023-12-27 12:43:12,410][20381] Updated weights for policy 1, policy_version 1019687 (0.0008) [2023-12-27 12:43:12,473][20381] Updated weights for policy 1, policy_version 1019697 (0.0005) [2023-12-27 12:43:12,847][20347] Updated weights for policy 0, policy_version 1020370 (0.0007) [2023-12-27 12:43:12,905][20347] Updated weights for policy 0, policy_version 1020380 (0.0006) [2023-12-27 12:43:12,962][20347] Updated weights for policy 0, policy_version 1020390 (0.0006) [2023-12-27 12:43:13,166][20381] Updated weights for policy 1, policy_version 1019707 (0.0007) [2023-12-27 12:43:13,228][20381] Updated weights for policy 1, policy_version 1019717 (0.0011) [2023-12-27 12:43:13,289][20381] Updated weights for policy 1, policy_version 1019727 (0.0011) [2023-12-27 12:43:13,648][20347] Updated weights for policy 0, policy_version 1020400 (0.0007) [2023-12-27 12:43:13,710][20347] Updated weights for policy 0, policy_version 1020410 (0.0007) [2023-12-27 12:43:13,774][20347] Updated weights for policy 0, policy_version 1020420 (0.0005) [2023-12-27 12:43:13,821][20381] Updated weights for policy 1, policy_version 1019737 (0.0007) [2023-12-27 12:43:13,863][20381] Updated weights for policy 1, policy_version 1019747 (0.0010) [2023-12-27 12:43:13,913][20381] Updated weights for policy 1, policy_version 1019757 (0.0007) [2023-12-27 12:43:13,976][20381] Updated weights for policy 1, policy_version 1019767 (0.0010) [2023-12-27 12:43:14,347][20347] Updated weights for policy 0, policy_version 1020430 (0.0008) [2023-12-27 12:43:14,413][20347] Updated weights for policy 0, policy_version 1020440 (0.0008) [2023-12-27 12:43:14,468][20347] Updated weights for policy 0, policy_version 1020450 (0.0006) [2023-12-27 12:43:14,594][20381] Updated weights for policy 1, policy_version 1019777 (0.0011) [2023-12-27 12:43:14,658][20381] Updated weights for policy 1, policy_version 1019787 (0.0010) [2023-12-27 12:43:14,719][20381] Updated weights for policy 1, policy_version 1019797 (0.0011) [2023-12-27 12:43:15,093][20347] Updated weights for policy 0, policy_version 1020460 (0.0008) [2023-12-27 12:43:15,151][20347] Updated weights for policy 0, policy_version 1020470 (0.0007) [2023-12-27 12:43:15,214][20347] Updated weights for policy 0, policy_version 1020480 (0.0007) [2023-12-27 12:43:15,398][20381] Updated weights for policy 1, policy_version 1019807 (0.0011) [2023-12-27 12:43:15,457][20381] Updated weights for policy 1, policy_version 1019817 (0.0007) [2023-12-27 12:43:15,520][20381] Updated weights for policy 1, policy_version 1019827 (0.0008) [2023-12-27 12:43:15,850][20347] Updated weights for policy 0, policy_version 1020490 (0.0010) [2023-12-27 12:43:15,902][20347] Updated weights for policy 0, policy_version 1020500 (0.0010) [2023-12-27 12:43:15,952][20347] Updated weights for policy 0, policy_version 1020510 (0.0010) [2023-12-27 12:43:16,013][20347] Updated weights for policy 0, policy_version 1020520 (0.0010) [2023-12-27 12:43:16,170][20381] Updated weights for policy 1, policy_version 1019837 (0.0009) [2023-12-27 12:43:16,231][20381] Updated weights for policy 1, policy_version 1019847 (0.0008) [2023-12-27 12:43:16,282][20381] Updated weights for policy 1, policy_version 1019857 (0.0007) [2023-12-27 12:43:16,702][20347] Updated weights for policy 0, policy_version 1020530 (0.0010) [2023-12-27 12:43:16,750][20347] Updated weights for policy 0, policy_version 1020540 (0.0010) [2023-12-27 12:43:16,796][20347] Updated weights for policy 0, policy_version 1020550 (0.0010) [2023-12-27 12:43:16,968][20381] Updated weights for policy 1, policy_version 1019867 (0.0007) [2023-12-27 12:43:17,015][20381] Updated weights for policy 1, policy_version 1019877 (0.0007) [2023-12-27 12:43:17,076][20381] Updated weights for policy 1, policy_version 1019887 (0.0007) [2023-12-27 12:43:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 522428416. Throughput: 0: 10222.0, 1: 10380.2. Samples: 522438604. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:43:17,364][19170] Avg episode reward: [(0, '941.630'), (1, '176.010')] [2023-12-27 12:43:17,498][20347] Updated weights for policy 0, policy_version 1020560 (0.0008) [2023-12-27 12:43:17,557][20347] Updated weights for policy 0, policy_version 1020570 (0.0007) [2023-12-27 12:43:17,616][20347] Updated weights for policy 0, policy_version 1020580 (0.0007) [2023-12-27 12:43:17,750][20381] Updated weights for policy 1, policy_version 1019897 (0.0007) [2023-12-27 12:43:17,802][20381] Updated weights for policy 1, policy_version 1019907 (0.0011) [2023-12-27 12:43:17,860][20381] Updated weights for policy 1, policy_version 1019917 (0.0010) [2023-12-27 12:43:17,909][20381] Updated weights for policy 1, policy_version 1019927 (0.0011) [2023-12-27 12:43:18,279][20347] Updated weights for policy 0, policy_version 1020590 (0.0006) [2023-12-27 12:43:18,340][20347] Updated weights for policy 0, policy_version 1020600 (0.0006) [2023-12-27 12:43:18,391][20347] Updated weights for policy 0, policy_version 1020610 (0.0007) [2023-12-27 12:43:18,611][20381] Updated weights for policy 1, policy_version 1019937 (0.0006) [2023-12-27 12:43:18,661][20381] Updated weights for policy 1, policy_version 1019947 (0.0005) [2023-12-27 12:43:18,717][20381] Updated weights for policy 1, policy_version 1019957 (0.0007) [2023-12-27 12:43:19,052][20347] Updated weights for policy 0, policy_version 1020620 (0.0006) [2023-12-27 12:43:19,115][20347] Updated weights for policy 0, policy_version 1020630 (0.0007) [2023-12-27 12:43:19,173][20347] Updated weights for policy 0, policy_version 1020640 (0.0005) [2023-12-27 12:43:19,318][20381] Updated weights for policy 1, policy_version 1019967 (0.0007) [2023-12-27 12:43:19,388][20381] Updated weights for policy 1, policy_version 1019977 (0.0007) [2023-12-27 12:43:19,443][20381] Updated weights for policy 1, policy_version 1019987 (0.0007) [2023-12-27 12:43:19,960][20347] Updated weights for policy 0, policy_version 1020650 (0.0007) [2023-12-27 12:43:20,018][20347] Updated weights for policy 0, policy_version 1020660 (0.0008) [2023-12-27 12:43:20,056][20381] Updated weights for policy 1, policy_version 1019997 (0.0007) [2023-12-27 12:43:20,077][20347] Updated weights for policy 0, policy_version 1020670 (0.0006) [2023-12-27 12:43:20,112][20381] Updated weights for policy 1, policy_version 1020007 (0.0006) [2023-12-27 12:43:20,133][20347] Updated weights for policy 0, policy_version 1020680 (0.0007) [2023-12-27 12:43:20,175][20381] Updated weights for policy 1, policy_version 1020017 (0.0006) [2023-12-27 12:43:20,836][20347] Updated weights for policy 0, policy_version 1020690 (0.0007) [2023-12-27 12:43:20,845][20381] Updated weights for policy 1, policy_version 1020027 (0.0006) [2023-12-27 12:43:20,895][20347] Updated weights for policy 0, policy_version 1020700 (0.0008) [2023-12-27 12:43:20,910][20381] Updated weights for policy 1, policy_version 1020037 (0.0006) [2023-12-27 12:43:20,944][20291] KL-divergence is very high: 167.0512 [2023-12-27 12:43:20,951][20291] KL-divergence is very high: 148.0886 [2023-12-27 12:43:20,962][20347] Updated weights for policy 0, policy_version 1020710 (0.0007) [2023-12-27 12:43:20,975][20381] Updated weights for policy 1, policy_version 1020047 (0.0006) [2023-12-27 12:43:21,653][20347] Updated weights for policy 0, policy_version 1020720 (0.0008) [2023-12-27 12:43:21,671][20381] Updated weights for policy 1, policy_version 1020057 (0.0008) [2023-12-27 12:43:21,672][20291] KL-divergence is very high: 120.1889 [2023-12-27 12:43:21,715][20347] Updated weights for policy 0, policy_version 1020730 (0.0006) [2023-12-27 12:43:21,737][20381] Updated weights for policy 1, policy_version 1020067 (0.0008) [2023-12-27 12:43:21,778][20291] KL-divergence is very high: 303.9738 [2023-12-27 12:43:21,783][20347] Updated weights for policy 0, policy_version 1020740 (0.0006) [2023-12-27 12:43:21,801][20381] Updated weights for policy 1, policy_version 1020077 (0.0006) [2023-12-27 12:43:21,866][20381] Updated weights for policy 1, policy_version 1020087 (0.0008) [2023-12-27 12:43:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 522526720. Throughput: 0: 10166.6, 1: 10357.8. Samples: 522500484. Policy #0 lag: (min: 27.0, avg: 33.8, max: 59.0) [2023-12-27 12:43:22,363][19170] Avg episode reward: [(0, '932.080'), (1, '171.030')] [2023-12-27 12:43:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001020088_261169152.pth... [2023-12-27 12:43:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001020744_261357568.pth... [2023-12-27 12:43:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001018872_260857856.pth [2023-12-27 12:43:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001019560_261054464.pth [2023-12-27 12:43:22,487][20347] Updated weights for policy 0, policy_version 1020750 (0.0008) [2023-12-27 12:43:22,543][20347] Updated weights for policy 0, policy_version 1020760 (0.0008) [2023-12-27 12:43:22,571][20381] Updated weights for policy 1, policy_version 1020097 (0.0008) [2023-12-27 12:43:22,599][20347] Updated weights for policy 0, policy_version 1020770 (0.0008) [2023-12-27 12:43:22,638][20381] Updated weights for policy 1, policy_version 1020107 (0.0008) [2023-12-27 12:43:22,700][20381] Updated weights for policy 1, policy_version 1020117 (0.0008) [2023-12-27 12:43:23,258][20347] Updated weights for policy 0, policy_version 1020780 (0.0008) [2023-12-27 12:43:23,301][20347] Updated weights for policy 0, policy_version 1020790 (0.0007) [2023-12-27 12:43:23,348][20347] Updated weights for policy 0, policy_version 1020800 (0.0007) [2023-12-27 12:43:23,380][20381] Updated weights for policy 1, policy_version 1020127 (0.0007) [2023-12-27 12:43:23,430][20381] Updated weights for policy 1, policy_version 1020137 (0.0007) [2023-12-27 12:43:23,487][20381] Updated weights for policy 1, policy_version 1020147 (0.0007) [2023-12-27 12:43:24,057][20347] Updated weights for policy 0, policy_version 1020810 (0.0006) [2023-12-27 12:43:24,120][20347] Updated weights for policy 0, policy_version 1020820 (0.0006) [2023-12-27 12:43:24,178][20347] Updated weights for policy 0, policy_version 1020830 (0.0008) [2023-12-27 12:43:24,183][20381] Updated weights for policy 1, policy_version 1020157 (0.0007) [2023-12-27 12:43:24,238][20381] Updated weights for policy 1, policy_version 1020167 (0.0007) [2023-12-27 12:43:24,239][20347] Updated weights for policy 0, policy_version 1020840 (0.0007) [2023-12-27 12:43:24,292][20381] Updated weights for policy 1, policy_version 1020177 (0.0007) [2023-12-27 12:43:24,858][20347] Updated weights for policy 0, policy_version 1020850 (0.0006) [2023-12-27 12:43:24,925][20347] Updated weights for policy 0, policy_version 1020860 (0.0007) [2023-12-27 12:43:24,984][20347] Updated weights for policy 0, policy_version 1020870 (0.0007) [2023-12-27 12:43:25,060][20381] Updated weights for policy 1, policy_version 1020187 (0.0008) [2023-12-27 12:43:25,119][20381] Updated weights for policy 1, policy_version 1020197 (0.0008) [2023-12-27 12:43:25,179][20381] Updated weights for policy 1, policy_version 1020207 (0.0008) [2023-12-27 12:43:25,671][20347] Updated weights for policy 0, policy_version 1020880 (0.0007) [2023-12-27 12:43:25,730][20347] Updated weights for policy 0, policy_version 1020890 (0.0008) [2023-12-27 12:43:25,789][20347] Updated weights for policy 0, policy_version 1020900 (0.0008) [2023-12-27 12:43:25,885][20381] Updated weights for policy 1, policy_version 1020217 (0.0008) [2023-12-27 12:43:25,939][20381] Updated weights for policy 1, policy_version 1020227 (0.0007) [2023-12-27 12:43:25,995][20381] Updated weights for policy 1, policy_version 1020237 (0.0006) [2023-12-27 12:43:26,049][20381] Updated weights for policy 1, policy_version 1020247 (0.0006) [2023-12-27 12:43:26,476][20347] Updated weights for policy 0, policy_version 1020910 (0.0008) [2023-12-27 12:43:26,526][20347] Updated weights for policy 0, policy_version 1020920 (0.0008) [2023-12-27 12:43:26,575][20347] Updated weights for policy 0, policy_version 1020930 (0.0007) [2023-12-27 12:43:26,681][20381] Updated weights for policy 1, policy_version 1020257 (0.0005) [2023-12-27 12:43:26,741][20381] Updated weights for policy 1, policy_version 1020267 (0.0005) [2023-12-27 12:43:26,796][20381] Updated weights for policy 1, policy_version 1020277 (0.0005) [2023-12-27 12:43:27,324][20347] Updated weights for policy 0, policy_version 1020940 (0.0009) [2023-12-27 12:43:27,336][20381] Updated weights for policy 1, policy_version 1020287 (0.0006) [2023-12-27 12:43:27,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 522625024. Throughput: 0: 10160.3, 1: 10298.4. Samples: 522619496. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:27,363][19170] Avg episode reward: [(0, '922.350'), (1, '191.370')] [2023-12-27 12:43:27,376][20347] Updated weights for policy 0, policy_version 1020950 (0.0005) [2023-12-27 12:43:27,389][20381] Updated weights for policy 1, policy_version 1020297 (0.0006) [2023-12-27 12:43:27,427][20347] Updated weights for policy 0, policy_version 1020960 (0.0007) [2023-12-27 12:43:27,443][20381] Updated weights for policy 1, policy_version 1020307 (0.0006) [2023-12-27 12:43:28,049][20347] Updated weights for policy 0, policy_version 1020970 (0.0006) [2023-12-27 12:43:28,098][20347] Updated weights for policy 0, policy_version 1020980 (0.0007) [2023-12-27 12:43:28,153][20381] Updated weights for policy 1, policy_version 1020317 (0.0007) [2023-12-27 12:43:28,155][20347] Updated weights for policy 0, policy_version 1020990 (0.0006) [2023-12-27 12:43:28,206][20381] Updated weights for policy 1, policy_version 1020327 (0.0007) [2023-12-27 12:43:28,208][20347] Updated weights for policy 0, policy_version 1021000 (0.0006) [2023-12-27 12:43:28,262][20381] Updated weights for policy 1, policy_version 1020337 (0.0008) [2023-12-27 12:43:28,885][20347] Updated weights for policy 0, policy_version 1021010 (0.0007) [2023-12-27 12:43:28,935][20347] Updated weights for policy 0, policy_version 1021020 (0.0007) [2023-12-27 12:43:28,979][20347] Updated weights for policy 0, policy_version 1021030 (0.0007) [2023-12-27 12:43:28,980][20381] Updated weights for policy 1, policy_version 1020347 (0.0009) [2023-12-27 12:43:29,034][20381] Updated weights for policy 1, policy_version 1020357 (0.0010) [2023-12-27 12:43:29,084][20381] Updated weights for policy 1, policy_version 1020367 (0.0010) [2023-12-27 12:43:29,681][20347] Updated weights for policy 0, policy_version 1021040 (0.0005) [2023-12-27 12:43:29,739][20347] Updated weights for policy 0, policy_version 1021050 (0.0007) [2023-12-27 12:43:29,786][20381] Updated weights for policy 1, policy_version 1020377 (0.0007) [2023-12-27 12:43:29,797][20347] Updated weights for policy 0, policy_version 1021060 (0.0007) [2023-12-27 12:43:29,853][20381] Updated weights for policy 1, policy_version 1020387 (0.0007) [2023-12-27 12:43:29,909][20381] Updated weights for policy 1, policy_version 1020397 (0.0008) [2023-12-27 12:43:29,971][20381] Updated weights for policy 1, policy_version 1020407 (0.0007) [2023-12-27 12:43:30,439][20347] Updated weights for policy 0, policy_version 1021070 (0.0009) [2023-12-27 12:43:30,487][20347] Updated weights for policy 0, policy_version 1021080 (0.0010) [2023-12-27 12:43:30,544][20347] Updated weights for policy 0, policy_version 1021090 (0.0010) [2023-12-27 12:43:30,692][20381] Updated weights for policy 1, policy_version 1020417 (0.0007) [2023-12-27 12:43:30,747][20381] Updated weights for policy 1, policy_version 1020427 (0.0009) [2023-12-27 12:43:30,798][20381] Updated weights for policy 1, policy_version 1020437 (0.0009) [2023-12-27 12:43:31,163][20347] Updated weights for policy 0, policy_version 1021100 (0.0009) [2023-12-27 12:43:31,218][20347] Updated weights for policy 0, policy_version 1021110 (0.0008) [2023-12-27 12:43:31,278][20347] Updated weights for policy 0, policy_version 1021120 (0.0008) [2023-12-27 12:43:31,523][20381] Updated weights for policy 1, policy_version 1020447 (0.0007) [2023-12-27 12:43:31,573][20381] Updated weights for policy 1, policy_version 1020457 (0.0006) [2023-12-27 12:43:31,635][20381] Updated weights for policy 1, policy_version 1020467 (0.0007) [2023-12-27 12:43:32,017][20347] Updated weights for policy 0, policy_version 1021130 (0.0008) [2023-12-27 12:43:32,069][20347] Updated weights for policy 0, policy_version 1021140 (0.0008) [2023-12-27 12:43:32,118][20347] Updated weights for policy 0, policy_version 1021150 (0.0008) [2023-12-27 12:43:32,167][20347] Updated weights for policy 0, policy_version 1021160 (0.0008) [2023-12-27 12:43:32,274][20381] Updated weights for policy 1, policy_version 1020477 (0.0007) [2023-12-27 12:43:32,336][20381] Updated weights for policy 1, policy_version 1020487 (0.0008) [2023-12-27 12:43:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 522731520. Throughput: 0: 10223.3, 1: 10283.6. Samples: 522743672. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:32,363][19170] Avg episode reward: [(0, '970.810'), (1, '189.490')] [2023-12-27 12:43:32,408][20381] Updated weights for policy 1, policy_version 1020497 (0.0007) [2023-12-27 12:43:32,846][20347] Updated weights for policy 0, policy_version 1021170 (0.0007) [2023-12-27 12:43:32,908][20347] Updated weights for policy 0, policy_version 1021180 (0.0008) [2023-12-27 12:43:32,956][20347] Updated weights for policy 0, policy_version 1021190 (0.0007) [2023-12-27 12:43:33,045][20381] Updated weights for policy 1, policy_version 1020507 (0.0007) [2023-12-27 12:43:33,093][20381] Updated weights for policy 1, policy_version 1020517 (0.0007) [2023-12-27 12:43:33,138][20381] Updated weights for policy 1, policy_version 1020527 (0.0008) [2023-12-27 12:43:33,564][20347] Updated weights for policy 0, policy_version 1021200 (0.0009) [2023-12-27 12:43:33,624][20347] Updated weights for policy 0, policy_version 1021210 (0.0010) [2023-12-27 12:43:33,672][20347] Updated weights for policy 0, policy_version 1021220 (0.0010) [2023-12-27 12:43:33,882][20381] Updated weights for policy 1, policy_version 1020537 (0.0008) [2023-12-27 12:43:33,930][20381] Updated weights for policy 1, policy_version 1020547 (0.0009) [2023-12-27 12:43:33,983][20381] Updated weights for policy 1, policy_version 1020557 (0.0007) [2023-12-27 12:43:34,028][20381] Updated weights for policy 1, policy_version 1020567 (0.0009) [2023-12-27 12:43:34,294][20347] Updated weights for policy 0, policy_version 1021230 (0.0009) [2023-12-27 12:43:34,346][20347] Updated weights for policy 0, policy_version 1021240 (0.0008) [2023-12-27 12:43:34,404][20347] Updated weights for policy 0, policy_version 1021250 (0.0008) [2023-12-27 12:43:34,763][20381] Updated weights for policy 1, policy_version 1020577 (0.0008) [2023-12-27 12:43:34,822][20381] Updated weights for policy 1, policy_version 1020587 (0.0007) [2023-12-27 12:43:34,881][20381] Updated weights for policy 1, policy_version 1020597 (0.0007) [2023-12-27 12:43:35,130][20347] Updated weights for policy 0, policy_version 1021260 (0.0009) [2023-12-27 12:43:35,187][20347] Updated weights for policy 0, policy_version 1021270 (0.0008) [2023-12-27 12:43:35,238][20347] Updated weights for policy 0, policy_version 1021280 (0.0006) [2023-12-27 12:43:35,582][20381] Updated weights for policy 1, policy_version 1020607 (0.0007) [2023-12-27 12:43:35,636][20381] Updated weights for policy 1, policy_version 1020617 (0.0008) [2023-12-27 12:43:35,694][20381] Updated weights for policy 1, policy_version 1020627 (0.0007) [2023-12-27 12:43:35,893][20347] Updated weights for policy 0, policy_version 1021290 (0.0007) [2023-12-27 12:43:35,944][20347] Updated weights for policy 0, policy_version 1021300 (0.0010) [2023-12-27 12:43:35,989][20347] Updated weights for policy 0, policy_version 1021310 (0.0010) [2023-12-27 12:43:36,039][20347] Updated weights for policy 0, policy_version 1021320 (0.0010) [2023-12-27 12:43:36,448][20381] Updated weights for policy 1, policy_version 1020637 (0.0008) [2023-12-27 12:43:36,509][20381] Updated weights for policy 1, policy_version 1020647 (0.0008) [2023-12-27 12:43:36,569][20381] Updated weights for policy 1, policy_version 1020657 (0.0008) [2023-12-27 12:43:36,730][20347] Updated weights for policy 0, policy_version 1021330 (0.0010) [2023-12-27 12:43:36,784][20347] Updated weights for policy 0, policy_version 1021340 (0.0007) [2023-12-27 12:43:36,840][20347] Updated weights for policy 0, policy_version 1021350 (0.0009) [2023-12-27 12:43:37,307][20381] Updated weights for policy 1, policy_version 1020667 (0.0007) [2023-12-27 12:43:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 522829824. Throughput: 0: 10275.8, 1: 10271.7. Samples: 522804896. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:37,363][19170] Avg episode reward: [(0, '951.290'), (1, '197.660')] [2023-12-27 12:43:37,364][20381] Updated weights for policy 1, policy_version 1020677 (0.0007) [2023-12-27 12:43:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001021352_261513216.pth... [2023-12-27 12:43:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001020168_261210112.pth [2023-12-27 12:43:37,424][20381] Updated weights for policy 1, policy_version 1020687 (0.0008) [2023-12-27 12:43:37,439][20347] Updated weights for policy 0, policy_version 1021360 (0.0006) [2023-12-27 12:43:37,476][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001020696_261324800.pth... [2023-12-27 12:43:37,480][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001019480_261013504.pth [2023-12-27 12:43:37,505][20347] Updated weights for policy 0, policy_version 1021370 (0.0007) [2023-12-27 12:43:37,565][20347] Updated weights for policy 0, policy_version 1021380 (0.0006) [2023-12-27 12:43:38,161][20381] Updated weights for policy 1, policy_version 1020697 (0.0007) [2023-12-27 12:43:38,168][20347] Updated weights for policy 0, policy_version 1021390 (0.0007) [2023-12-27 12:43:38,216][20381] Updated weights for policy 1, policy_version 1020707 (0.0008) [2023-12-27 12:43:38,222][20347] Updated weights for policy 0, policy_version 1021400 (0.0008) [2023-12-27 12:43:38,268][20381] Updated weights for policy 1, policy_version 1020717 (0.0005) [2023-12-27 12:43:38,280][20347] Updated weights for policy 0, policy_version 1021410 (0.0008) [2023-12-27 12:43:38,335][20381] Updated weights for policy 1, policy_version 1020727 (0.0007) [2023-12-27 12:43:38,963][20347] Updated weights for policy 0, policy_version 1021420 (0.0007) [2023-12-27 12:43:39,024][20347] Updated weights for policy 0, policy_version 1021430 (0.0006) [2023-12-27 12:43:39,030][20291] KL-divergence is very high: 213.1533 [2023-12-27 12:43:39,050][20381] Updated weights for policy 1, policy_version 1020737 (0.0008) [2023-12-27 12:43:39,076][20291] KL-divergence is very high: 182.9149 [2023-12-27 12:43:39,080][20347] Updated weights for policy 0, policy_version 1021440 (0.0007) [2023-12-27 12:43:39,115][20381] Updated weights for policy 1, policy_version 1020747 (0.0008) [2023-12-27 12:43:39,178][20381] Updated weights for policy 1, policy_version 1020757 (0.0008) [2023-12-27 12:43:39,720][20347] Updated weights for policy 0, policy_version 1021450 (0.0005) [2023-12-27 12:43:39,784][20347] Updated weights for policy 0, policy_version 1021460 (0.0006) [2023-12-27 12:43:39,851][20347] Updated weights for policy 0, policy_version 1021470 (0.0008) [2023-12-27 12:43:39,909][20347] Updated weights for policy 0, policy_version 1021480 (0.0009) [2023-12-27 12:43:39,914][20381] Updated weights for policy 1, policy_version 1020767 (0.0009) [2023-12-27 12:43:39,973][20381] Updated weights for policy 1, policy_version 1020777 (0.0009) [2023-12-27 12:43:40,034][20381] Updated weights for policy 1, policy_version 1020787 (0.0009) [2023-12-27 12:43:40,643][20347] Updated weights for policy 0, policy_version 1021490 (0.0011) [2023-12-27 12:43:40,704][20347] Updated weights for policy 0, policy_version 1021500 (0.0011) [2023-12-27 12:43:40,735][20381] Updated weights for policy 1, policy_version 1020797 (0.0009) [2023-12-27 12:43:40,768][20347] Updated weights for policy 0, policy_version 1021510 (0.0009) [2023-12-27 12:43:40,796][20381] Updated weights for policy 1, policy_version 1020807 (0.0008) [2023-12-27 12:43:40,859][20381] Updated weights for policy 1, policy_version 1020817 (0.0007) [2023-12-27 12:43:41,508][20347] Updated weights for policy 0, policy_version 1021520 (0.0008) [2023-12-27 12:43:41,572][20347] Updated weights for policy 0, policy_version 1021530 (0.0008) [2023-12-27 12:43:41,618][20381] Updated weights for policy 1, policy_version 1020827 (0.0008) [2023-12-27 12:43:41,639][20347] Updated weights for policy 0, policy_version 1021540 (0.0008) [2023-12-27 12:43:41,691][20381] Updated weights for policy 1, policy_version 1020837 (0.0008) [2023-12-27 12:43:41,755][20381] Updated weights for policy 1, policy_version 1020847 (0.0009) [2023-12-27 12:43:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 522928128. Throughput: 0: 10254.7, 1: 10222.6. Samples: 522923884. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:42,363][19170] Avg episode reward: [(0, '941.460'), (1, '213.810')] [2023-12-27 12:43:42,370][20347] Updated weights for policy 0, policy_version 1021550 (0.0007) [2023-12-27 12:43:42,428][20347] Updated weights for policy 0, policy_version 1021560 (0.0008) [2023-12-27 12:43:42,455][20381] Updated weights for policy 1, policy_version 1020857 (0.0008) [2023-12-27 12:43:42,483][20347] Updated weights for policy 0, policy_version 1021570 (0.0008) [2023-12-27 12:43:42,512][20381] Updated weights for policy 1, policy_version 1020867 (0.0007) [2023-12-27 12:43:42,575][20381] Updated weights for policy 1, policy_version 1020877 (0.0008) [2023-12-27 12:43:42,625][20381] Updated weights for policy 1, policy_version 1020887 (0.0007) [2023-12-27 12:43:43,197][20347] Updated weights for policy 0, policy_version 1021580 (0.0008) [2023-12-27 12:43:43,255][20347] Updated weights for policy 0, policy_version 1021590 (0.0008) [2023-12-27 12:43:43,308][20347] Updated weights for policy 0, policy_version 1021600 (0.0006) [2023-12-27 12:43:43,329][20381] Updated weights for policy 1, policy_version 1020897 (0.0006) [2023-12-27 12:43:43,387][20381] Updated weights for policy 1, policy_version 1020907 (0.0006) [2023-12-27 12:43:43,436][20381] Updated weights for policy 1, policy_version 1020917 (0.0007) [2023-12-27 12:43:43,978][20347] Updated weights for policy 0, policy_version 1021610 (0.0007) [2023-12-27 12:43:44,031][20347] Updated weights for policy 0, policy_version 1021620 (0.0007) [2023-12-27 12:43:44,068][20381] Updated weights for policy 1, policy_version 1020927 (0.0006) [2023-12-27 12:43:44,080][20347] Updated weights for policy 0, policy_version 1021630 (0.0006) [2023-12-27 12:43:44,128][20347] Updated weights for policy 0, policy_version 1021640 (0.0005) [2023-12-27 12:43:44,133][20381] Updated weights for policy 1, policy_version 1020937 (0.0009) [2023-12-27 12:43:44,192][20381] Updated weights for policy 1, policy_version 1020947 (0.0011) [2023-12-27 12:43:44,816][20347] Updated weights for policy 0, policy_version 1021650 (0.0008) [2023-12-27 12:43:44,872][20347] Updated weights for policy 0, policy_version 1021660 (0.0007) [2023-12-27 12:43:44,891][20381] Updated weights for policy 1, policy_version 1020957 (0.0010) [2023-12-27 12:43:44,932][20347] Updated weights for policy 0, policy_version 1021670 (0.0007) [2023-12-27 12:43:44,946][20381] Updated weights for policy 1, policy_version 1020967 (0.0011) [2023-12-27 12:43:44,997][20381] Updated weights for policy 1, policy_version 1020977 (0.0011) [2023-12-27 12:43:45,637][20347] Updated weights for policy 0, policy_version 1021680 (0.0007) [2023-12-27 12:43:45,699][20347] Updated weights for policy 0, policy_version 1021690 (0.0007) [2023-12-27 12:43:45,712][20381] Updated weights for policy 1, policy_version 1020987 (0.0010) [2023-12-27 12:43:45,754][20347] Updated weights for policy 0, policy_version 1021700 (0.0005) [2023-12-27 12:43:45,767][20381] Updated weights for policy 1, policy_version 1020997 (0.0007) [2023-12-27 12:43:45,825][20381] Updated weights for policy 1, policy_version 1021007 (0.0007) [2023-12-27 12:43:46,440][20347] Updated weights for policy 0, policy_version 1021710 (0.0007) [2023-12-27 12:43:46,493][20347] Updated weights for policy 0, policy_version 1021720 (0.0007) [2023-12-27 12:43:46,523][20381] Updated weights for policy 1, policy_version 1021017 (0.0008) [2023-12-27 12:43:46,546][20347] Updated weights for policy 0, policy_version 1021730 (0.0006) [2023-12-27 12:43:46,575][20381] Updated weights for policy 1, policy_version 1021027 (0.0007) [2023-12-27 12:43:46,631][20381] Updated weights for policy 1, policy_version 1021037 (0.0008) [2023-12-27 12:43:46,692][20381] Updated weights for policy 1, policy_version 1021047 (0.0011) [2023-12-27 12:43:47,269][20381] Updated weights for policy 1, policy_version 1021057 (0.0008) [2023-12-27 12:43:47,286][20347] Updated weights for policy 0, policy_version 1021740 (0.0006) [2023-12-27 12:43:47,320][20381] Updated weights for policy 1, policy_version 1021067 (0.0006) [2023-12-27 12:43:47,342][20347] Updated weights for policy 0, policy_version 1021750 (0.0005) [2023-12-27 12:43:47,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 523026432. Throughput: 0: 10203.3, 1: 10197.3. Samples: 523044204. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:47,364][19170] Avg episode reward: [(0, '951.210'), (1, '192.770')] [2023-12-27 12:43:47,379][20381] Updated weights for policy 1, policy_version 1021077 (0.0005) [2023-12-27 12:43:47,401][20347] Updated weights for policy 0, policy_version 1021760 (0.0005) [2023-12-27 12:43:47,945][20381] Updated weights for policy 1, policy_version 1021087 (0.0006) [2023-12-27 12:43:47,993][20381] Updated weights for policy 1, policy_version 1021097 (0.0009) [2023-12-27 12:43:48,045][20381] Updated weights for policy 1, policy_version 1021107 (0.0010) [2023-12-27 12:43:48,091][20347] Updated weights for policy 0, policy_version 1021770 (0.0007) [2023-12-27 12:43:48,145][20347] Updated weights for policy 0, policy_version 1021780 (0.0010) [2023-12-27 12:43:48,194][20347] Updated weights for policy 0, policy_version 1021790 (0.0010) [2023-12-27 12:43:48,243][20347] Updated weights for policy 0, policy_version 1021800 (0.0010) [2023-12-27 12:43:48,644][20381] Updated weights for policy 1, policy_version 1021117 (0.0008) [2023-12-27 12:43:48,705][20381] Updated weights for policy 1, policy_version 1021127 (0.0007) [2023-12-27 12:43:48,767][20381] Updated weights for policy 1, policy_version 1021137 (0.0010) [2023-12-27 12:43:48,969][20347] Updated weights for policy 0, policy_version 1021810 (0.0010) [2023-12-27 12:43:49,028][20347] Updated weights for policy 0, policy_version 1021820 (0.0010) [2023-12-27 12:43:49,086][20347] Updated weights for policy 0, policy_version 1021830 (0.0010) [2023-12-27 12:43:49,406][20381] Updated weights for policy 1, policy_version 1021147 (0.0007) [2023-12-27 12:43:49,461][20381] Updated weights for policy 1, policy_version 1021157 (0.0010) [2023-12-27 12:43:49,520][20381] Updated weights for policy 1, policy_version 1021167 (0.0010) [2023-12-27 12:43:49,745][20347] Updated weights for policy 0, policy_version 1021840 (0.0010) [2023-12-27 12:43:49,804][20347] Updated weights for policy 0, policy_version 1021850 (0.0010) [2023-12-27 12:43:49,869][20347] Updated weights for policy 0, policy_version 1021860 (0.0009) [2023-12-27 12:43:50,186][20381] Updated weights for policy 1, policy_version 1021177 (0.0010) [2023-12-27 12:43:50,239][20381] Updated weights for policy 1, policy_version 1021187 (0.0006) [2023-12-27 12:43:50,294][20381] Updated weights for policy 1, policy_version 1021197 (0.0006) [2023-12-27 12:43:50,352][20381] Updated weights for policy 1, policy_version 1021207 (0.0006) [2023-12-27 12:43:50,551][20347] Updated weights for policy 0, policy_version 1021870 (0.0007) [2023-12-27 12:43:50,610][20347] Updated weights for policy 0, policy_version 1021880 (0.0008) [2023-12-27 12:43:50,670][20347] Updated weights for policy 0, policy_version 1021890 (0.0008) [2023-12-27 12:43:50,996][20381] Updated weights for policy 1, policy_version 1021217 (0.0007) [2023-12-27 12:43:51,057][20381] Updated weights for policy 1, policy_version 1021227 (0.0007) [2023-12-27 12:43:51,122][20381] Updated weights for policy 1, policy_version 1021237 (0.0008) [2023-12-27 12:43:51,376][20347] Updated weights for policy 0, policy_version 1021900 (0.0008) [2023-12-27 12:43:51,442][20347] Updated weights for policy 0, policy_version 1021910 (0.0007) [2023-12-27 12:43:51,504][20347] Updated weights for policy 0, policy_version 1021920 (0.0006) [2023-12-27 12:43:51,820][20381] Updated weights for policy 1, policy_version 1021247 (0.0008) [2023-12-27 12:43:51,879][20381] Updated weights for policy 1, policy_version 1021257 (0.0008) [2023-12-27 12:43:51,942][20381] Updated weights for policy 1, policy_version 1021267 (0.0008) [2023-12-27 12:43:52,163][20347] Updated weights for policy 0, policy_version 1021930 (0.0006) [2023-12-27 12:43:52,219][20347] Updated weights for policy 0, policy_version 1021940 (0.0005) [2023-12-27 12:43:52,287][20347] Updated weights for policy 0, policy_version 1021950 (0.0007) [2023-12-27 12:43:52,339][20347] Updated weights for policy 0, policy_version 1021960 (0.0008) [2023-12-27 12:43:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 523141120. Throughput: 0: 10145.4, 1: 10181.9. Samples: 523106804. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:52,363][19170] Avg episode reward: [(0, '922.220'), (1, '246.350')] [2023-12-27 12:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001021272_261472256.pth... [2023-12-27 12:43:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001021960_261668864.pth... [2023-12-27 12:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001020088_261169152.pth [2023-12-27 12:43:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001020744_261357568.pth [2023-12-27 12:43:52,659][20381] Updated weights for policy 1, policy_version 1021277 (0.0007) [2023-12-27 12:43:52,720][20381] Updated weights for policy 1, policy_version 1021287 (0.0008) [2023-12-27 12:43:52,772][20381] Updated weights for policy 1, policy_version 1021297 (0.0009) [2023-12-27 12:43:53,012][20347] Updated weights for policy 0, policy_version 1021970 (0.0007) [2023-12-27 12:43:53,066][20347] Updated weights for policy 0, policy_version 1021980 (0.0007) [2023-12-27 12:43:53,128][20347] Updated weights for policy 0, policy_version 1021990 (0.0005) [2023-12-27 12:43:53,463][20381] Updated weights for policy 1, policy_version 1021307 (0.0009) [2023-12-27 12:43:53,510][20381] Updated weights for policy 1, policy_version 1021317 (0.0010) [2023-12-27 12:43:53,565][20381] Updated weights for policy 1, policy_version 1021327 (0.0010) [2023-12-27 12:43:53,678][20347] Updated weights for policy 0, policy_version 1022000 (0.0009) [2023-12-27 12:43:53,735][20347] Updated weights for policy 0, policy_version 1022010 (0.0010) [2023-12-27 12:43:53,786][20347] Updated weights for policy 0, policy_version 1022020 (0.0006) [2023-12-27 12:43:54,192][20381] Updated weights for policy 1, policy_version 1021337 (0.0008) [2023-12-27 12:43:54,246][20381] Updated weights for policy 1, policy_version 1021347 (0.0010) [2023-12-27 12:43:54,295][20381] Updated weights for policy 1, policy_version 1021357 (0.0010) [2023-12-27 12:43:54,349][20381] Updated weights for policy 1, policy_version 1021367 (0.0010) [2023-12-27 12:43:54,465][20347] Updated weights for policy 0, policy_version 1022030 (0.0009) [2023-12-27 12:43:54,525][20347] Updated weights for policy 0, policy_version 1022040 (0.0010) [2023-12-27 12:43:54,573][20347] Updated weights for policy 0, policy_version 1022050 (0.0010) [2023-12-27 12:43:54,988][20381] Updated weights for policy 1, policy_version 1021377 (0.0010) [2023-12-27 12:43:55,044][20381] Updated weights for policy 1, policy_version 1021387 (0.0011) [2023-12-27 12:43:55,102][20381] Updated weights for policy 1, policy_version 1021397 (0.0010) [2023-12-27 12:43:55,274][20347] Updated weights for policy 0, policy_version 1022060 (0.0010) [2023-12-27 12:43:55,338][20347] Updated weights for policy 0, policy_version 1022070 (0.0010) [2023-12-27 12:43:55,402][20347] Updated weights for policy 0, policy_version 1022080 (0.0010) [2023-12-27 12:43:55,792][20381] Updated weights for policy 1, policy_version 1021407 (0.0009) [2023-12-27 12:43:55,845][20381] Updated weights for policy 1, policy_version 1021417 (0.0005) [2023-12-27 12:43:55,905][20381] Updated weights for policy 1, policy_version 1021427 (0.0008) [2023-12-27 12:43:56,046][20347] Updated weights for policy 0, policy_version 1022090 (0.0010) [2023-12-27 12:43:56,102][20347] Updated weights for policy 0, policy_version 1022100 (0.0010) [2023-12-27 12:43:56,161][20347] Updated weights for policy 0, policy_version 1022110 (0.0010) [2023-12-27 12:43:56,224][20347] Updated weights for policy 0, policy_version 1022120 (0.0010) [2023-12-27 12:43:56,579][20381] Updated weights for policy 1, policy_version 1021437 (0.0011) [2023-12-27 12:43:56,637][20381] Updated weights for policy 1, policy_version 1021447 (0.0011) [2023-12-27 12:43:56,696][20381] Updated weights for policy 1, policy_version 1021457 (0.0007) [2023-12-27 12:43:56,866][20347] Updated weights for policy 0, policy_version 1022130 (0.0005) [2023-12-27 12:43:56,912][20347] Updated weights for policy 0, policy_version 1022140 (0.0005) [2023-12-27 12:43:56,958][20347] Updated weights for policy 0, policy_version 1022150 (0.0006) [2023-12-27 12:43:57,356][20381] Updated weights for policy 1, policy_version 1021467 (0.0011) [2023-12-27 12:43:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 523239424. Throughput: 0: 10181.1, 1: 10173.8. Samples: 523231156. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:43:57,363][19170] Avg episode reward: [(0, '873.160'), (1, '183.240')] [2023-12-27 12:43:57,401][20381] Updated weights for policy 1, policy_version 1021477 (0.0010) [2023-12-27 12:43:57,445][20381] Updated weights for policy 1, policy_version 1021487 (0.0010) [2023-12-27 12:43:57,576][20347] Updated weights for policy 0, policy_version 1022160 (0.0005) [2023-12-27 12:43:57,620][20347] Updated weights for policy 0, policy_version 1022170 (0.0005) [2023-12-27 12:43:57,663][20347] Updated weights for policy 0, policy_version 1022180 (0.0007) [2023-12-27 12:43:58,137][20381] Updated weights for policy 1, policy_version 1021497 (0.0010) [2023-12-27 12:43:58,201][20381] Updated weights for policy 1, policy_version 1021507 (0.0010) [2023-12-27 12:43:58,255][20381] Updated weights for policy 1, policy_version 1021517 (0.0008) [2023-12-27 12:43:58,321][20381] Updated weights for policy 1, policy_version 1021527 (0.0010) [2023-12-27 12:43:58,327][20347] Updated weights for policy 0, policy_version 1022190 (0.0006) [2023-12-27 12:43:58,389][20347] Updated weights for policy 0, policy_version 1022200 (0.0008) [2023-12-27 12:43:58,444][20291] KL-divergence is very high: 150.3097 [2023-12-27 12:43:58,456][20347] Updated weights for policy 0, policy_version 1022210 (0.0008) [2023-12-27 12:43:59,136][20381] Updated weights for policy 1, policy_version 1021537 (0.0008) [2023-12-27 12:43:59,195][20381] Updated weights for policy 1, policy_version 1021547 (0.0007) [2023-12-27 12:43:59,273][20381] Updated weights for policy 1, policy_version 1021557 (0.0009) [2023-12-27 12:43:59,288][20347] Updated weights for policy 0, policy_version 1022220 (0.0008) [2023-12-27 12:43:59,351][20347] Updated weights for policy 0, policy_version 1022230 (0.0009) [2023-12-27 12:43:59,418][20347] Updated weights for policy 0, policy_version 1022240 (0.0007) [2023-12-27 12:43:59,950][20381] Updated weights for policy 1, policy_version 1021567 (0.0010) [2023-12-27 12:44:00,009][20381] Updated weights for policy 1, policy_version 1021577 (0.0011) [2023-12-27 12:44:00,069][20381] Updated weights for policy 1, policy_version 1021587 (0.0008) [2023-12-27 12:44:00,143][20347] Updated weights for policy 0, policy_version 1022250 (0.0007) [2023-12-27 12:44:00,194][20347] Updated weights for policy 0, policy_version 1022260 (0.0007) [2023-12-27 12:44:00,258][20347] Updated weights for policy 0, policy_version 1022270 (0.0006) [2023-12-27 12:44:00,311][20347] Updated weights for policy 0, policy_version 1022280 (0.0007) [2023-12-27 12:44:00,739][20381] Updated weights for policy 1, policy_version 1021597 (0.0006) [2023-12-27 12:44:00,790][20381] Updated weights for policy 1, policy_version 1021607 (0.0006) [2023-12-27 12:44:00,841][20381] Updated weights for policy 1, policy_version 1021617 (0.0005) [2023-12-27 12:44:00,914][20347] Updated weights for policy 0, policy_version 1022290 (0.0008) [2023-12-27 12:44:00,973][20347] Updated weights for policy 0, policy_version 1022300 (0.0007) [2023-12-27 12:44:01,038][20347] Updated weights for policy 0, policy_version 1022310 (0.0008) [2023-12-27 12:44:01,469][20381] Updated weights for policy 1, policy_version 1021627 (0.0007) [2023-12-27 12:44:01,529][20381] Updated weights for policy 1, policy_version 1021637 (0.0011) [2023-12-27 12:44:01,585][20381] Updated weights for policy 1, policy_version 1021647 (0.0010) [2023-12-27 12:44:01,710][20347] Updated weights for policy 0, policy_version 1022320 (0.0008) [2023-12-27 12:44:01,771][20347] Updated weights for policy 0, policy_version 1022330 (0.0007) [2023-12-27 12:44:01,821][20347] Updated weights for policy 0, policy_version 1022340 (0.0007) [2023-12-27 12:44:02,205][20381] Updated weights for policy 1, policy_version 1021657 (0.0010) [2023-12-27 12:44:02,266][20381] Updated weights for policy 1, policy_version 1021667 (0.0010) [2023-12-27 12:44:02,322][20381] Updated weights for policy 1, policy_version 1021677 (0.0011) [2023-12-27 12:44:02,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 523337728. Throughput: 0: 10175.0, 1: 10139.3. Samples: 523352744. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:02,363][19170] Avg episode reward: [(0, '882.780'), (1, '268.000')] [2023-12-27 12:44:02,412][20381] Updated weights for policy 1, policy_version 1021687 (0.0011) [2023-12-27 12:44:02,558][20347] Updated weights for policy 0, policy_version 1022350 (0.0008) [2023-12-27 12:44:02,619][20347] Updated weights for policy 0, policy_version 1022360 (0.0006) [2023-12-27 12:44:02,682][20347] Updated weights for policy 0, policy_version 1022370 (0.0005) [2023-12-27 12:44:02,999][20381] Updated weights for policy 1, policy_version 1021697 (0.0006) [2023-12-27 12:44:03,055][20381] Updated weights for policy 1, policy_version 1021707 (0.0005) [2023-12-27 12:44:03,113][20381] Updated weights for policy 1, policy_version 1021717 (0.0005) [2023-12-27 12:44:03,330][20347] Updated weights for policy 0, policy_version 1022380 (0.0007) [2023-12-27 12:44:03,388][20347] Updated weights for policy 0, policy_version 1022390 (0.0007) [2023-12-27 12:44:03,442][20347] Updated weights for policy 0, policy_version 1022400 (0.0007) [2023-12-27 12:44:03,700][20381] Updated weights for policy 1, policy_version 1021727 (0.0009) [2023-12-27 12:44:03,760][20381] Updated weights for policy 1, policy_version 1021737 (0.0010) [2023-12-27 12:44:03,817][20381] Updated weights for policy 1, policy_version 1021747 (0.0010) [2023-12-27 12:44:04,084][20347] Updated weights for policy 0, policy_version 1022410 (0.0006) [2023-12-27 12:44:04,142][20347] Updated weights for policy 0, policy_version 1022420 (0.0010) [2023-12-27 12:44:04,206][20347] Updated weights for policy 0, policy_version 1022430 (0.0011) [2023-12-27 12:44:04,262][20347] Updated weights for policy 0, policy_version 1022440 (0.0010) [2023-12-27 12:44:04,504][20381] Updated weights for policy 1, policy_version 1021757 (0.0010) [2023-12-27 12:44:04,553][20381] Updated weights for policy 1, policy_version 1021767 (0.0010) [2023-12-27 12:44:04,605][20381] Updated weights for policy 1, policy_version 1021777 (0.0010) [2023-12-27 12:44:04,801][20347] Updated weights for policy 0, policy_version 1022450 (0.0005) [2023-12-27 12:44:04,849][20347] Updated weights for policy 0, policy_version 1022460 (0.0006) [2023-12-27 12:44:04,898][20347] Updated weights for policy 0, policy_version 1022470 (0.0010) [2023-12-27 12:44:05,234][20381] Updated weights for policy 1, policy_version 1021787 (0.0009) [2023-12-27 12:44:05,297][20381] Updated weights for policy 1, policy_version 1021797 (0.0005) [2023-12-27 12:44:05,352][20381] Updated weights for policy 1, policy_version 1021807 (0.0005) [2023-12-27 12:44:05,564][20347] Updated weights for policy 0, policy_version 1022480 (0.0010) [2023-12-27 12:44:05,616][20347] Updated weights for policy 0, policy_version 1022490 (0.0010) [2023-12-27 12:44:05,661][20347] Updated weights for policy 0, policy_version 1022500 (0.0008) [2023-12-27 12:44:05,934][20381] Updated weights for policy 1, policy_version 1021817 (0.0007) [2023-12-27 12:44:05,993][20381] Updated weights for policy 1, policy_version 1021827 (0.0011) [2023-12-27 12:44:06,050][20381] Updated weights for policy 1, policy_version 1021837 (0.0006) [2023-12-27 12:44:06,113][20381] Updated weights for policy 1, policy_version 1021847 (0.0010) [2023-12-27 12:44:06,371][20347] Updated weights for policy 0, policy_version 1022510 (0.0010) [2023-12-27 12:44:06,433][20347] Updated weights for policy 0, policy_version 1022520 (0.0006) [2023-12-27 12:44:06,496][20347] Updated weights for policy 0, policy_version 1022530 (0.0010) [2023-12-27 12:44:06,804][20381] Updated weights for policy 1, policy_version 1021857 (0.0009) [2023-12-27 12:44:06,867][20381] Updated weights for policy 1, policy_version 1021867 (0.0006) [2023-12-27 12:44:06,939][20381] Updated weights for policy 1, policy_version 1021877 (0.0007) [2023-12-27 12:44:07,116][20347] Updated weights for policy 0, policy_version 1022540 (0.0008) [2023-12-27 12:44:07,174][20347] Updated weights for policy 0, policy_version 1022550 (0.0006) [2023-12-27 12:44:07,234][20347] Updated weights for policy 0, policy_version 1022560 (0.0005) [2023-12-27 12:44:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 523452416. Throughput: 0: 10230.9, 1: 10147.0. Samples: 523417484. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:07,363][19170] Avg episode reward: [(0, '873.300'), (1, '236.470')] [2023-12-27 12:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001021880_261627904.pth... [2023-12-27 12:44:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001022568_261824512.pth... [2023-12-27 12:44:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001020696_261324800.pth [2023-12-27 12:44:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001021352_261513216.pth [2023-12-27 12:44:07,539][20381] Updated weights for policy 1, policy_version 1021887 (0.0009) [2023-12-27 12:44:07,598][20381] Updated weights for policy 1, policy_version 1021897 (0.0008) [2023-12-27 12:44:07,662][20381] Updated weights for policy 1, policy_version 1021907 (0.0005) [2023-12-27 12:44:07,773][20347] Updated weights for policy 0, policy_version 1022570 (0.0006) [2023-12-27 12:44:07,837][20347] Updated weights for policy 0, policy_version 1022580 (0.0008) [2023-12-27 12:44:07,900][20347] Updated weights for policy 0, policy_version 1022590 (0.0008) [2023-12-27 12:44:07,968][20347] Updated weights for policy 0, policy_version 1022600 (0.0007) [2023-12-27 12:44:08,220][20381] Updated weights for policy 1, policy_version 1021917 (0.0006) [2023-12-27 12:44:08,278][20381] Updated weights for policy 1, policy_version 1021927 (0.0005) [2023-12-27 12:44:08,339][20381] Updated weights for policy 1, policy_version 1021937 (0.0007) [2023-12-27 12:44:08,571][20347] Updated weights for policy 0, policy_version 1022610 (0.0006) [2023-12-27 12:44:08,623][20347] Updated weights for policy 0, policy_version 1022620 (0.0007) [2023-12-27 12:44:08,681][20347] Updated weights for policy 0, policy_version 1022630 (0.0010) [2023-12-27 12:44:09,011][20381] Updated weights for policy 1, policy_version 1021947 (0.0008) [2023-12-27 12:44:09,061][20381] Updated weights for policy 1, policy_version 1021957 (0.0007) [2023-12-27 12:44:09,116][20381] Updated weights for policy 1, policy_version 1021967 (0.0005) [2023-12-27 12:44:09,302][20347] Updated weights for policy 0, policy_version 1022640 (0.0008) [2023-12-27 12:44:09,367][20347] Updated weights for policy 0, policy_version 1022650 (0.0009) [2023-12-27 12:44:09,433][20347] Updated weights for policy 0, policy_version 1022660 (0.0010) [2023-12-27 12:44:09,867][20381] Updated weights for policy 1, policy_version 1021977 (0.0009) [2023-12-27 12:44:09,925][20381] Updated weights for policy 1, policy_version 1021987 (0.0009) [2023-12-27 12:44:09,984][20381] Updated weights for policy 1, policy_version 1021997 (0.0008) [2023-12-27 12:44:10,041][20381] Updated weights for policy 1, policy_version 1022007 (0.0008) [2023-12-27 12:44:10,141][20347] Updated weights for policy 0, policy_version 1022670 (0.0008) [2023-12-27 12:44:10,204][20347] Updated weights for policy 0, policy_version 1022680 (0.0007) [2023-12-27 12:44:10,269][20347] Updated weights for policy 0, policy_version 1022690 (0.0007) [2023-12-27 12:44:10,841][20381] Updated weights for policy 1, policy_version 1022017 (0.0007) [2023-12-27 12:44:10,887][20381] Updated weights for policy 1, policy_version 1022027 (0.0008) [2023-12-27 12:44:10,903][20347] Updated weights for policy 0, policy_version 1022700 (0.0007) [2023-12-27 12:44:10,940][20381] Updated weights for policy 1, policy_version 1022037 (0.0008) [2023-12-27 12:44:10,953][20347] Updated weights for policy 0, policy_version 1022710 (0.0005) [2023-12-27 12:44:11,000][20347] Updated weights for policy 0, policy_version 1022720 (0.0005) [2023-12-27 12:44:11,762][20347] Updated weights for policy 0, policy_version 1022730 (0.0007) [2023-12-27 12:44:11,775][20381] Updated weights for policy 1, policy_version 1022047 (0.0008) [2023-12-27 12:44:11,817][20347] Updated weights for policy 0, policy_version 1022740 (0.0006) [2023-12-27 12:44:11,831][20381] Updated weights for policy 1, policy_version 1022057 (0.0007) [2023-12-27 12:44:11,876][20347] Updated weights for policy 0, policy_version 1022750 (0.0007) [2023-12-27 12:44:11,895][20381] Updated weights for policy 1, policy_version 1022067 (0.0007) [2023-12-27 12:44:11,937][20347] Updated weights for policy 0, policy_version 1022760 (0.0006) [2023-12-27 12:44:12,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 523550720. Throughput: 0: 10309.3, 1: 10211.8. Samples: 523542944. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:12,363][19170] Avg episode reward: [(0, '922.250'), (1, '309.610')] [2023-12-27 12:44:12,616][20347] Updated weights for policy 0, policy_version 1022770 (0.0009) [2023-12-27 12:44:12,638][20381] Updated weights for policy 1, policy_version 1022077 (0.0007) [2023-12-27 12:44:12,671][20347] Updated weights for policy 0, policy_version 1022780 (0.0005) [2023-12-27 12:44:12,700][20381] Updated weights for policy 1, policy_version 1022087 (0.0008) [2023-12-27 12:44:12,723][20347] Updated weights for policy 0, policy_version 1022790 (0.0006) [2023-12-27 12:44:12,766][20381] Updated weights for policy 1, policy_version 1022097 (0.0008) [2023-12-27 12:44:13,392][20347] Updated weights for policy 0, policy_version 1022800 (0.0008) [2023-12-27 12:44:13,400][20381] Updated weights for policy 1, policy_version 1022107 (0.0007) [2023-12-27 12:44:13,450][20347] Updated weights for policy 0, policy_version 1022810 (0.0007) [2023-12-27 12:44:13,450][20381] Updated weights for policy 1, policy_version 1022117 (0.0005) [2023-12-27 12:44:13,501][20381] Updated weights for policy 1, policy_version 1022127 (0.0005) [2023-12-27 12:44:13,506][20347] Updated weights for policy 0, policy_version 1022820 (0.0008) [2023-12-27 12:44:14,049][20381] Updated weights for policy 1, policy_version 1022137 (0.0005) [2023-12-27 12:44:14,100][20381] Updated weights for policy 1, policy_version 1022147 (0.0007) [2023-12-27 12:44:14,158][20381] Updated weights for policy 1, policy_version 1022157 (0.0008) [2023-12-27 12:44:14,214][20381] Updated weights for policy 1, policy_version 1022167 (0.0007) [2023-12-27 12:44:14,238][20347] Updated weights for policy 0, policy_version 1022830 (0.0007) [2023-12-27 12:44:14,307][20347] Updated weights for policy 0, policy_version 1022840 (0.0007) [2023-12-27 12:44:14,362][20347] Updated weights for policy 0, policy_version 1022850 (0.0006) [2023-12-27 12:44:14,815][20381] Updated weights for policy 1, policy_version 1022177 (0.0008) [2023-12-27 12:44:14,874][20381] Updated weights for policy 1, policy_version 1022187 (0.0008) [2023-12-27 12:44:14,937][20381] Updated weights for policy 1, policy_version 1022197 (0.0008) [2023-12-27 12:44:15,069][20347] Updated weights for policy 0, policy_version 1022860 (0.0008) [2023-12-27 12:44:15,132][20347] Updated weights for policy 0, policy_version 1022870 (0.0007) [2023-12-27 12:44:15,182][20347] Updated weights for policy 0, policy_version 1022880 (0.0007) [2023-12-27 12:44:15,628][20381] Updated weights for policy 1, policy_version 1022207 (0.0007) [2023-12-27 12:44:15,683][20381] Updated weights for policy 1, policy_version 1022217 (0.0007) [2023-12-27 12:44:15,736][20381] Updated weights for policy 1, policy_version 1022227 (0.0007) [2023-12-27 12:44:15,860][20347] Updated weights for policy 0, policy_version 1022890 (0.0007) [2023-12-27 12:44:15,923][20347] Updated weights for policy 0, policy_version 1022900 (0.0006) [2023-12-27 12:44:15,974][20347] Updated weights for policy 0, policy_version 1022910 (0.0007) [2023-12-27 12:44:16,021][20347] Updated weights for policy 0, policy_version 1022920 (0.0007) [2023-12-27 12:44:16,429][20381] Updated weights for policy 1, policy_version 1022237 (0.0010) [2023-12-27 12:44:16,488][20381] Updated weights for policy 1, policy_version 1022247 (0.0010) [2023-12-27 12:44:16,549][20381] Updated weights for policy 1, policy_version 1022257 (0.0011) [2023-12-27 12:44:16,612][20347] Updated weights for policy 0, policy_version 1022930 (0.0010) [2023-12-27 12:44:16,670][20347] Updated weights for policy 0, policy_version 1022940 (0.0008) [2023-12-27 12:44:16,722][20347] Updated weights for policy 0, policy_version 1022950 (0.0007) [2023-12-27 12:44:17,220][20381] Updated weights for policy 1, policy_version 1022267 (0.0010) [2023-12-27 12:44:17,262][20347] Updated weights for policy 0, policy_version 1022960 (0.0010) [2023-12-27 12:44:17,273][20381] Updated weights for policy 1, policy_version 1022277 (0.0010) [2023-12-27 12:44:17,318][20347] Updated weights for policy 0, policy_version 1022970 (0.0009) [2023-12-27 12:44:17,322][20381] Updated weights for policy 1, policy_version 1022287 (0.0010) [2023-12-27 12:44:17,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 523649024. Throughput: 0: 10302.0, 1: 10209.0. Samples: 523666672. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:17,363][19170] Avg episode reward: [(0, '912.310'), (1, '322.550')] [2023-12-27 12:44:17,371][20347] Updated weights for policy 0, policy_version 1022980 (0.0007) [2023-12-27 12:44:17,909][20381] Updated weights for policy 1, policy_version 1022297 (0.0010) [2023-12-27 12:44:17,969][20381] Updated weights for policy 1, policy_version 1022307 (0.0005) [2023-12-27 12:44:17,985][20347] Updated weights for policy 0, policy_version 1022990 (0.0005) [2023-12-27 12:44:18,026][20381] Updated weights for policy 1, policy_version 1022317 (0.0006) [2023-12-27 12:44:18,046][20347] Updated weights for policy 0, policy_version 1023000 (0.0006) [2023-12-27 12:44:18,082][20381] Updated weights for policy 1, policy_version 1022327 (0.0006) [2023-12-27 12:44:18,111][20347] Updated weights for policy 0, policy_version 1023010 (0.0006) [2023-12-27 12:44:18,732][20347] Updated weights for policy 0, policy_version 1023020 (0.0008) [2023-12-27 12:44:18,758][20381] Updated weights for policy 1, policy_version 1022337 (0.0006) [2023-12-27 12:44:18,789][20347] Updated weights for policy 0, policy_version 1023030 (0.0009) [2023-12-27 12:44:18,812][20381] Updated weights for policy 1, policy_version 1022347 (0.0005) [2023-12-27 12:44:18,846][20347] Updated weights for policy 0, policy_version 1023040 (0.0009) [2023-12-27 12:44:18,858][20381] Updated weights for policy 1, policy_version 1022357 (0.0007) [2023-12-27 12:44:19,591][20381] Updated weights for policy 1, policy_version 1022367 (0.0008) [2023-12-27 12:44:19,612][20347] Updated weights for policy 0, policy_version 1023050 (0.0008) [2023-12-27 12:44:19,651][20381] Updated weights for policy 1, policy_version 1022377 (0.0008) [2023-12-27 12:44:19,670][20347] Updated weights for policy 0, policy_version 1023060 (0.0006) [2023-12-27 12:44:19,714][20381] Updated weights for policy 1, policy_version 1022387 (0.0008) [2023-12-27 12:44:19,729][20347] Updated weights for policy 0, policy_version 1023070 (0.0006) [2023-12-27 12:44:19,784][20347] Updated weights for policy 0, policy_version 1023080 (0.0008) [2023-12-27 12:44:20,452][20381] Updated weights for policy 1, policy_version 1022397 (0.0007) [2023-12-27 12:44:20,498][20347] Updated weights for policy 0, policy_version 1023090 (0.0006) [2023-12-27 12:44:20,508][20381] Updated weights for policy 1, policy_version 1022407 (0.0005) [2023-12-27 12:44:20,562][20347] Updated weights for policy 0, policy_version 1023100 (0.0008) [2023-12-27 12:44:20,567][20381] Updated weights for policy 1, policy_version 1022417 (0.0007) [2023-12-27 12:44:20,626][20347] Updated weights for policy 0, policy_version 1023110 (0.0008) [2023-12-27 12:44:21,285][20381] Updated weights for policy 1, policy_version 1022427 (0.0008) [2023-12-27 12:44:21,342][20381] Updated weights for policy 1, policy_version 1022437 (0.0008) [2023-12-27 12:44:21,388][20347] Updated weights for policy 0, policy_version 1023120 (0.0007) [2023-12-27 12:44:21,412][20381] Updated weights for policy 1, policy_version 1022447 (0.0008) [2023-12-27 12:44:21,446][20347] Updated weights for policy 0, policy_version 1023130 (0.0006) [2023-12-27 12:44:21,494][20347] Updated weights for policy 0, policy_version 1023140 (0.0007) [2023-12-27 12:44:22,142][20381] Updated weights for policy 1, policy_version 1022457 (0.0007) [2023-12-27 12:44:22,197][20381] Updated weights for policy 1, policy_version 1022467 (0.0007) [2023-12-27 12:44:22,211][20347] Updated weights for policy 0, policy_version 1023150 (0.0006) [2023-12-27 12:44:22,255][20381] Updated weights for policy 1, policy_version 1022477 (0.0006) [2023-12-27 12:44:22,271][20347] Updated weights for policy 0, policy_version 1023160 (0.0007) [2023-12-27 12:44:22,318][20381] Updated weights for policy 1, policy_version 1022487 (0.0008) [2023-12-27 12:44:22,329][20347] Updated weights for policy 0, policy_version 1023170 (0.0007) [2023-12-27 12:44:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 523755520. Throughput: 0: 10291.8, 1: 10233.8. Samples: 523728548. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:22,363][19170] Avg episode reward: [(0, '902.750'), (1, '205.670')] [2023-12-27 12:44:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001023176_261980160.pth... [2023-12-27 12:44:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001022488_261783552.pth... [2023-12-27 12:44:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001021272_261472256.pth [2023-12-27 12:44:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001021960_261668864.pth [2023-12-27 12:44:22,985][20347] Updated weights for policy 0, policy_version 1023180 (0.0006) [2023-12-27 12:44:23,040][20347] Updated weights for policy 0, policy_version 1023190 (0.0006) [2023-12-27 12:44:23,068][20381] Updated weights for policy 1, policy_version 1022497 (0.0006) [2023-12-27 12:44:23,090][20347] Updated weights for policy 0, policy_version 1023200 (0.0007) [2023-12-27 12:44:23,128][20381] Updated weights for policy 1, policy_version 1022507 (0.0006) [2023-12-27 12:44:23,185][20381] Updated weights for policy 1, policy_version 1022517 (0.0005) [2023-12-27 12:44:23,762][20347] Updated weights for policy 0, policy_version 1023210 (0.0008) [2023-12-27 12:44:23,788][20381] Updated weights for policy 1, policy_version 1022527 (0.0008) [2023-12-27 12:44:23,810][20291] KL-divergence is very high: 390.0774 [2023-12-27 12:44:23,821][20347] Updated weights for policy 0, policy_version 1023220 (0.0010) [2023-12-27 12:44:23,845][20381] Updated weights for policy 1, policy_version 1022537 (0.0010) [2023-12-27 12:44:23,856][20291] KL-divergence is very high: 266.7101 [2023-12-27 12:44:23,876][20347] Updated weights for policy 0, policy_version 1023230 (0.0010) [2023-12-27 12:44:23,900][20381] Updated weights for policy 1, policy_version 1022547 (0.0010) [2023-12-27 12:44:23,924][20347] Updated weights for policy 0, policy_version 1023240 (0.0010) [2023-12-27 12:44:24,530][20347] Updated weights for policy 0, policy_version 1023250 (0.0006) [2023-12-27 12:44:24,532][20381] Updated weights for policy 1, policy_version 1022557 (0.0010) [2023-12-27 12:44:24,587][20347] Updated weights for policy 0, policy_version 1023260 (0.0009) [2023-12-27 12:44:24,593][20381] Updated weights for policy 1, policy_version 1022567 (0.0010) [2023-12-27 12:44:24,639][20347] Updated weights for policy 0, policy_version 1023270 (0.0010) [2023-12-27 12:44:24,647][20381] Updated weights for policy 1, policy_version 1022577 (0.0010) [2023-12-27 12:44:25,231][20381] Updated weights for policy 1, policy_version 1022587 (0.0010) [2023-12-27 12:44:25,281][20381] Updated weights for policy 1, policy_version 1022597 (0.0010) [2023-12-27 12:44:25,288][20347] Updated weights for policy 0, policy_version 1023280 (0.0010) [2023-12-27 12:44:25,332][20381] Updated weights for policy 1, policy_version 1022607 (0.0010) [2023-12-27 12:44:25,345][20347] Updated weights for policy 0, policy_version 1023290 (0.0007) [2023-12-27 12:44:25,405][20347] Updated weights for policy 0, policy_version 1023300 (0.0006) [2023-12-27 12:44:26,061][20381] Updated weights for policy 1, policy_version 1022617 (0.0010) [2023-12-27 12:44:26,072][20347] Updated weights for policy 0, policy_version 1023310 (0.0007) [2023-12-27 12:44:26,122][20381] Updated weights for policy 1, policy_version 1022627 (0.0007) [2023-12-27 12:44:26,131][20347] Updated weights for policy 0, policy_version 1023320 (0.0008) [2023-12-27 12:44:26,174][20381] Updated weights for policy 1, policy_version 1022637 (0.0006) [2023-12-27 12:44:26,187][20347] Updated weights for policy 0, policy_version 1023330 (0.0005) [2023-12-27 12:44:26,228][20381] Updated weights for policy 1, policy_version 1022647 (0.0006) [2023-12-27 12:44:26,799][20381] Updated weights for policy 1, policy_version 1022657 (0.0007) [2023-12-27 12:44:26,843][20347] Updated weights for policy 0, policy_version 1023340 (0.0006) [2023-12-27 12:44:26,857][20381] Updated weights for policy 1, policy_version 1022667 (0.0007) [2023-12-27 12:44:26,892][20347] Updated weights for policy 0, policy_version 1023350 (0.0007) [2023-12-27 12:44:26,917][20381] Updated weights for policy 1, policy_version 1022677 (0.0005) [2023-12-27 12:44:26,949][20347] Updated weights for policy 0, policy_version 1023360 (0.0008) [2023-12-27 12:44:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 523862016. Throughput: 0: 10293.7, 1: 10307.8. Samples: 523850948. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:27,363][19170] Avg episode reward: [(0, '932.000'), (1, '230.490')] [2023-12-27 12:44:27,561][20381] Updated weights for policy 1, policy_version 1022687 (0.0005) [2023-12-27 12:44:27,614][20381] Updated weights for policy 1, policy_version 1022697 (0.0005) [2023-12-27 12:44:27,649][20347] Updated weights for policy 0, policy_version 1023370 (0.0008) [2023-12-27 12:44:27,669][20381] Updated weights for policy 1, policy_version 1022707 (0.0005) [2023-12-27 12:44:27,694][20347] Updated weights for policy 0, policy_version 1023380 (0.0007) [2023-12-27 12:44:27,746][20347] Updated weights for policy 0, policy_version 1023390 (0.0007) [2023-12-27 12:44:27,799][20347] Updated weights for policy 0, policy_version 1023400 (0.0008) [2023-12-27 12:44:28,245][20381] Updated weights for policy 1, policy_version 1022717 (0.0006) [2023-12-27 12:44:28,302][20381] Updated weights for policy 1, policy_version 1022727 (0.0007) [2023-12-27 12:44:28,366][20381] Updated weights for policy 1, policy_version 1022737 (0.0007) [2023-12-27 12:44:28,522][20347] Updated weights for policy 0, policy_version 1023410 (0.0010) [2023-12-27 12:44:28,587][20347] Updated weights for policy 0, policy_version 1023420 (0.0010) [2023-12-27 12:44:28,649][20347] Updated weights for policy 0, policy_version 1023430 (0.0010) [2023-12-27 12:44:29,052][20381] Updated weights for policy 1, policy_version 1022747 (0.0008) [2023-12-27 12:44:29,102][20381] Updated weights for policy 1, policy_version 1022757 (0.0008) [2023-12-27 12:44:29,154][20381] Updated weights for policy 1, policy_version 1022767 (0.0011) [2023-12-27 12:44:29,304][20347] Updated weights for policy 0, policy_version 1023440 (0.0008) [2023-12-27 12:44:29,371][20347] Updated weights for policy 0, policy_version 1023450 (0.0010) [2023-12-27 12:44:29,432][20347] Updated weights for policy 0, policy_version 1023460 (0.0010) [2023-12-27 12:44:29,862][20381] Updated weights for policy 1, policy_version 1022777 (0.0011) [2023-12-27 12:44:29,917][20381] Updated weights for policy 1, policy_version 1022787 (0.0011) [2023-12-27 12:44:29,978][20381] Updated weights for policy 1, policy_version 1022797 (0.0010) [2023-12-27 12:44:30,029][20381] Updated weights for policy 1, policy_version 1022807 (0.0010) [2023-12-27 12:44:30,159][20347] Updated weights for policy 0, policy_version 1023470 (0.0008) [2023-12-27 12:44:30,210][20347] Updated weights for policy 0, policy_version 1023480 (0.0007) [2023-12-27 12:44:30,269][20347] Updated weights for policy 0, policy_version 1023490 (0.0006) [2023-12-27 12:44:30,714][20381] Updated weights for policy 1, policy_version 1022817 (0.0008) [2023-12-27 12:44:30,765][20381] Updated weights for policy 1, policy_version 1022827 (0.0007) [2023-12-27 12:44:30,822][20381] Updated weights for policy 1, policy_version 1022837 (0.0007) [2023-12-27 12:44:30,876][20347] Updated weights for policy 0, policy_version 1023500 (0.0006) [2023-12-27 12:44:30,928][20347] Updated weights for policy 0, policy_version 1023510 (0.0007) [2023-12-27 12:44:30,976][20347] Updated weights for policy 0, policy_version 1023520 (0.0005) [2023-12-27 12:44:31,528][20381] Updated weights for policy 1, policy_version 1022847 (0.0006) [2023-12-27 12:44:31,585][20381] Updated weights for policy 1, policy_version 1022857 (0.0007) [2023-12-27 12:44:31,625][20347] Updated weights for policy 0, policy_version 1023530 (0.0007) [2023-12-27 12:44:31,646][20381] Updated weights for policy 1, policy_version 1022867 (0.0008) [2023-12-27 12:44:31,685][20347] Updated weights for policy 0, policy_version 1023540 (0.0007) [2023-12-27 12:44:31,751][20347] Updated weights for policy 0, policy_version 1023550 (0.0008) [2023-12-27 12:44:31,797][20347] Updated weights for policy 0, policy_version 1023560 (0.0005) [2023-12-27 12:44:32,292][20381] Updated weights for policy 1, policy_version 1022877 (0.0007) [2023-12-27 12:44:32,362][20381] Updated weights for policy 1, policy_version 1022887 (0.0008) [2023-12-27 12:44:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 523960320. Throughput: 0: 10347.8, 1: 10355.1. Samples: 523975828. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:32,363][19170] Avg episode reward: [(0, '941.630'), (1, '295.110')] [2023-12-27 12:44:32,421][20381] Updated weights for policy 1, policy_version 1022897 (0.0007) [2023-12-27 12:44:32,480][20347] Updated weights for policy 0, policy_version 1023570 (0.0008) [2023-12-27 12:44:32,534][20347] Updated weights for policy 0, policy_version 1023580 (0.0007) [2023-12-27 12:44:32,592][20347] Updated weights for policy 0, policy_version 1023590 (0.0006) [2023-12-27 12:44:33,082][20381] Updated weights for policy 1, policy_version 1022907 (0.0006) [2023-12-27 12:44:33,133][20381] Updated weights for policy 1, policy_version 1022917 (0.0005) [2023-12-27 12:44:33,198][20381] Updated weights for policy 1, policy_version 1022927 (0.0006) [2023-12-27 12:44:33,242][20347] Updated weights for policy 0, policy_version 1023600 (0.0006) [2023-12-27 12:44:33,297][20347] Updated weights for policy 0, policy_version 1023610 (0.0007) [2023-12-27 12:44:33,350][20347] Updated weights for policy 0, policy_version 1023620 (0.0006) [2023-12-27 12:44:33,844][20381] Updated weights for policy 1, policy_version 1022937 (0.0006) [2023-12-27 12:44:33,889][20381] Updated weights for policy 1, policy_version 1022947 (0.0008) [2023-12-27 12:44:33,936][20381] Updated weights for policy 1, policy_version 1022957 (0.0007) [2023-12-27 12:44:33,984][20381] Updated weights for policy 1, policy_version 1022967 (0.0006) [2023-12-27 12:44:33,993][20347] Updated weights for policy 0, policy_version 1023630 (0.0006) [2023-12-27 12:44:34,041][20347] Updated weights for policy 0, policy_version 1023640 (0.0009) [2023-12-27 12:44:34,088][20347] Updated weights for policy 0, policy_version 1023650 (0.0008) [2023-12-27 12:44:34,720][20381] Updated weights for policy 1, policy_version 1022977 (0.0008) [2023-12-27 12:44:34,773][20381] Updated weights for policy 1, policy_version 1022987 (0.0005) [2023-12-27 12:44:34,821][20347] Updated weights for policy 0, policy_version 1023660 (0.0007) [2023-12-27 12:44:34,831][20381] Updated weights for policy 1, policy_version 1022997 (0.0006) [2023-12-27 12:44:34,871][20347] Updated weights for policy 0, policy_version 1023670 (0.0006) [2023-12-27 12:44:34,925][20347] Updated weights for policy 0, policy_version 1023680 (0.0007) [2023-12-27 12:44:34,953][20291] KL-divergence is very high: 195.7707 [2023-12-27 12:44:34,959][20291] KL-divergence is very high: 196.3701 [2023-12-27 12:44:35,497][20381] Updated weights for policy 1, policy_version 1023007 (0.0008) [2023-12-27 12:44:35,547][20381] Updated weights for policy 1, policy_version 1023017 (0.0009) [2023-12-27 12:44:35,581][20347] Updated weights for policy 0, policy_version 1023690 (0.0007) [2023-12-27 12:44:35,582][20291] KL-divergence is very high: 132.5444 [2023-12-27 12:44:35,602][20381] Updated weights for policy 1, policy_version 1023027 (0.0007) [2023-12-27 12:44:35,626][20291] KL-divergence is very high: 124.4208 [2023-12-27 12:44:35,638][20347] Updated weights for policy 0, policy_version 1023700 (0.0006) [2023-12-27 12:44:35,693][20347] Updated weights for policy 0, policy_version 1023710 (0.0007) [2023-12-27 12:44:35,763][20347] Updated weights for policy 0, policy_version 1023720 (0.0008) [2023-12-27 12:44:36,223][20381] Updated weights for policy 1, policy_version 1023037 (0.0007) [2023-12-27 12:44:36,282][20381] Updated weights for policy 1, policy_version 1023047 (0.0005) [2023-12-27 12:44:36,342][20381] Updated weights for policy 1, policy_version 1023057 (0.0007) [2023-12-27 12:44:36,481][20347] Updated weights for policy 0, policy_version 1023730 (0.0008) [2023-12-27 12:44:36,537][20347] Updated weights for policy 0, policy_version 1023740 (0.0007) [2023-12-27 12:44:36,599][20347] Updated weights for policy 0, policy_version 1023750 (0.0007) [2023-12-27 12:44:37,029][20381] Updated weights for policy 1, policy_version 1023067 (0.0007) [2023-12-27 12:44:37,085][20381] Updated weights for policy 1, policy_version 1023077 (0.0007) [2023-12-27 12:44:37,136][20381] Updated weights for policy 1, policy_version 1023087 (0.0008) [2023-12-27 12:44:37,320][20347] Updated weights for policy 0, policy_version 1023760 (0.0006) [2023-12-27 12:44:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 524066816. Throughput: 0: 10370.8, 1: 10312.3. Samples: 524037544. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:37,363][19170] Avg episode reward: [(0, '922.000'), (1, '425.620')] [2023-12-27 12:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001023096_261939200.pth... [2023-12-27 12:44:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001021880_261627904.pth [2023-12-27 12:44:37,376][20347] Updated weights for policy 0, policy_version 1023770 (0.0008) [2023-12-27 12:44:37,433][20347] Updated weights for policy 0, policy_version 1023780 (0.0008) [2023-12-27 12:44:37,454][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001023784_262135808.pth... [2023-12-27 12:44:37,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001022568_261824512.pth [2023-12-27 12:44:37,779][20381] Updated weights for policy 1, policy_version 1023097 (0.0007) [2023-12-27 12:44:37,835][20381] Updated weights for policy 1, policy_version 1023107 (0.0007) [2023-12-27 12:44:37,880][20381] Updated weights for policy 1, policy_version 1023117 (0.0007) [2023-12-27 12:44:37,923][20381] Updated weights for policy 1, policy_version 1023127 (0.0007) [2023-12-27 12:44:38,140][20347] Updated weights for policy 0, policy_version 1023790 (0.0007) [2023-12-27 12:44:38,191][20347] Updated weights for policy 0, policy_version 1023800 (0.0005) [2023-12-27 12:44:38,237][20347] Updated weights for policy 0, policy_version 1023810 (0.0005) [2023-12-27 12:44:38,691][20381] Updated weights for policy 1, policy_version 1023137 (0.0008) [2023-12-27 12:44:38,741][20381] Updated weights for policy 1, policy_version 1023147 (0.0008) [2023-12-27 12:44:38,788][20381] Updated weights for policy 1, policy_version 1023157 (0.0008) [2023-12-27 12:44:38,908][20347] Updated weights for policy 0, policy_version 1023820 (0.0006) [2023-12-27 12:44:38,971][20347] Updated weights for policy 0, policy_version 1023830 (0.0008) [2023-12-27 12:44:39,018][20347] Updated weights for policy 0, policy_version 1023840 (0.0007) [2023-12-27 12:44:39,441][20381] Updated weights for policy 1, policy_version 1023167 (0.0007) [2023-12-27 12:44:39,499][20381] Updated weights for policy 1, policy_version 1023177 (0.0007) [2023-12-27 12:44:39,560][20381] Updated weights for policy 1, policy_version 1023187 (0.0008) [2023-12-27 12:44:39,781][20347] Updated weights for policy 0, policy_version 1023850 (0.0008) [2023-12-27 12:44:39,836][20347] Updated weights for policy 0, policy_version 1023860 (0.0008) [2023-12-27 12:44:39,896][20347] Updated weights for policy 0, policy_version 1023870 (0.0008) [2023-12-27 12:44:39,959][20347] Updated weights for policy 0, policy_version 1023880 (0.0008) [2023-12-27 12:44:40,346][20381] Updated weights for policy 1, policy_version 1023197 (0.0008) [2023-12-27 12:44:40,408][20381] Updated weights for policy 1, policy_version 1023207 (0.0009) [2023-12-27 12:44:40,471][20381] Updated weights for policy 1, policy_version 1023217 (0.0010) [2023-12-27 12:44:40,640][20347] Updated weights for policy 0, policy_version 1023890 (0.0008) [2023-12-27 12:44:40,687][20347] Updated weights for policy 0, policy_version 1023900 (0.0007) [2023-12-27 12:44:40,737][20347] Updated weights for policy 0, policy_version 1023910 (0.0007) [2023-12-27 12:44:41,133][20381] Updated weights for policy 1, policy_version 1023227 (0.0009) [2023-12-27 12:44:41,189][20381] Updated weights for policy 1, policy_version 1023237 (0.0006) [2023-12-27 12:44:41,244][20381] Updated weights for policy 1, policy_version 1023247 (0.0008) [2023-12-27 12:44:41,534][20347] Updated weights for policy 0, policy_version 1023920 (0.0008) [2023-12-27 12:44:41,597][20347] Updated weights for policy 0, policy_version 1023930 (0.0008) [2023-12-27 12:44:41,663][20347] Updated weights for policy 0, policy_version 1023940 (0.0008) [2023-12-27 12:44:42,004][20381] Updated weights for policy 1, policy_version 1023257 (0.0009) [2023-12-27 12:44:42,060][20381] Updated weights for policy 1, policy_version 1023267 (0.0011) [2023-12-27 12:44:42,125][20381] Updated weights for policy 1, policy_version 1023277 (0.0011) [2023-12-27 12:44:42,184][20381] Updated weights for policy 1, policy_version 1023287 (0.0011) [2023-12-27 12:44:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 524165120. Throughput: 0: 10296.7, 1: 10297.0. Samples: 524157876. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:42,363][19170] Avg episode reward: [(0, '844.210'), (1, '187.770')] [2023-12-27 12:44:42,385][20347] Updated weights for policy 0, policy_version 1023950 (0.0007) [2023-12-27 12:44:42,449][20347] Updated weights for policy 0, policy_version 1023960 (0.0006) [2023-12-27 12:44:42,506][20347] Updated weights for policy 0, policy_version 1023970 (0.0008) [2023-12-27 12:44:42,881][20381] Updated weights for policy 1, policy_version 1023297 (0.0008) [2023-12-27 12:44:42,937][20381] Updated weights for policy 1, policy_version 1023307 (0.0006) [2023-12-27 12:44:42,991][20381] Updated weights for policy 1, policy_version 1023317 (0.0009) [2023-12-27 12:44:43,122][20347] Updated weights for policy 0, policy_version 1023980 (0.0007) [2023-12-27 12:44:43,170][20347] Updated weights for policy 0, policy_version 1023990 (0.0008) [2023-12-27 12:44:43,230][20347] Updated weights for policy 0, policy_version 1024000 (0.0009) [2023-12-27 12:44:43,598][20381] Updated weights for policy 1, policy_version 1023327 (0.0007) [2023-12-27 12:44:43,658][20381] Updated weights for policy 1, policy_version 1023337 (0.0008) [2023-12-27 12:44:43,717][20381] Updated weights for policy 1, policy_version 1023347 (0.0008) [2023-12-27 12:44:43,923][20347] Updated weights for policy 0, policy_version 1024011 (0.0008) [2023-12-27 12:44:43,972][20347] Updated weights for policy 0, policy_version 1024021 (0.0005) [2023-12-27 12:44:44,020][20347] Updated weights for policy 0, policy_version 1024031 (0.0005) [2023-12-27 12:44:44,435][20381] Updated weights for policy 1, policy_version 1023357 (0.0006) [2023-12-27 12:44:44,488][20381] Updated weights for policy 1, policy_version 1023367 (0.0005) [2023-12-27 12:44:44,549][20381] Updated weights for policy 1, policy_version 1023377 (0.0006) [2023-12-27 12:44:44,583][20347] Updated weights for policy 0, policy_version 1024041 (0.0005) [2023-12-27 12:44:44,638][20347] Updated weights for policy 0, policy_version 1024051 (0.0005) [2023-12-27 12:44:44,702][20347] Updated weights for policy 0, policy_version 1024061 (0.0008) [2023-12-27 12:44:44,764][20347] Updated weights for policy 0, policy_version 1024071 (0.0007) [2023-12-27 12:44:45,251][20381] Updated weights for policy 1, policy_version 1023387 (0.0006) [2023-12-27 12:44:45,310][20381] Updated weights for policy 1, policy_version 1023397 (0.0007) [2023-12-27 12:44:45,367][20381] Updated weights for policy 1, policy_version 1023407 (0.0007) [2023-12-27 12:44:45,369][20347] Updated weights for policy 0, policy_version 1024081 (0.0007) [2023-12-27 12:44:45,415][20347] Updated weights for policy 0, policy_version 1024091 (0.0005) [2023-12-27 12:44:45,469][20347] Updated weights for policy 0, policy_version 1024101 (0.0008) [2023-12-27 12:44:46,054][20381] Updated weights for policy 1, policy_version 1023417 (0.0006) [2023-12-27 12:44:46,105][20381] Updated weights for policy 1, policy_version 1023427 (0.0007) [2023-12-27 12:44:46,136][20347] Updated weights for policy 0, policy_version 1024111 (0.0008) [2023-12-27 12:44:46,149][20381] Updated weights for policy 1, policy_version 1023437 (0.0008) [2023-12-27 12:44:46,190][20347] Updated weights for policy 0, policy_version 1024121 (0.0005) [2023-12-27 12:44:46,196][20381] Updated weights for policy 1, policy_version 1023447 (0.0006) [2023-12-27 12:44:46,243][20347] Updated weights for policy 0, policy_version 1024131 (0.0007) [2023-12-27 12:44:46,823][20347] Updated weights for policy 0, policy_version 1024141 (0.0006) [2023-12-27 12:44:46,874][20347] Updated weights for policy 0, policy_version 1024151 (0.0005) [2023-12-27 12:44:46,933][20347] Updated weights for policy 0, policy_version 1024161 (0.0006) [2023-12-27 12:44:46,968][20381] Updated weights for policy 1, policy_version 1023457 (0.0006) [2023-12-27 12:44:47,027][20381] Updated weights for policy 1, policy_version 1023467 (0.0008) [2023-12-27 12:44:47,079][20381] Updated weights for policy 1, policy_version 1023477 (0.0008) [2023-12-27 12:44:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 524271616. Throughput: 0: 10374.1, 1: 10262.0. Samples: 524281364. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:47,363][19170] Avg episode reward: [(0, '873.530'), (1, '171.880')] [2023-12-27 12:44:47,483][20347] Updated weights for policy 0, policy_version 1024171 (0.0005) [2023-12-27 12:44:47,547][20347] Updated weights for policy 0, policy_version 1024181 (0.0005) [2023-12-27 12:44:47,607][20347] Updated weights for policy 0, policy_version 1024191 (0.0006) [2023-12-27 12:44:47,775][20381] Updated weights for policy 1, policy_version 1023487 (0.0006) [2023-12-27 12:44:47,828][20381] Updated weights for policy 1, policy_version 1023497 (0.0005) [2023-12-27 12:44:47,882][20381] Updated weights for policy 1, policy_version 1023507 (0.0005) [2023-12-27 12:44:48,135][20347] Updated weights for policy 0, policy_version 1024201 (0.0005) [2023-12-27 12:44:48,194][20347] Updated weights for policy 0, policy_version 1024211 (0.0007) [2023-12-27 12:44:48,242][20347] Updated weights for policy 0, policy_version 1024221 (0.0010) [2023-12-27 12:44:48,285][20347] Updated weights for policy 0, policy_version 1024231 (0.0009) [2023-12-27 12:44:48,473][20381] Updated weights for policy 1, policy_version 1023517 (0.0006) [2023-12-27 12:44:48,533][20381] Updated weights for policy 1, policy_version 1023527 (0.0008) [2023-12-27 12:44:48,584][20381] Updated weights for policy 1, policy_version 1023537 (0.0007) [2023-12-27 12:44:48,981][20347] Updated weights for policy 0, policy_version 1024241 (0.0010) [2023-12-27 12:44:49,038][20347] Updated weights for policy 0, policy_version 1024251 (0.0011) [2023-12-27 12:44:49,098][20347] Updated weights for policy 0, policy_version 1024261 (0.0009) [2023-12-27 12:44:49,307][20381] Updated weights for policy 1, policy_version 1023547 (0.0007) [2023-12-27 12:44:49,375][20381] Updated weights for policy 1, policy_version 1023557 (0.0008) [2023-12-27 12:44:49,439][20381] Updated weights for policy 1, policy_version 1023567 (0.0007) [2023-12-27 12:44:49,817][20347] Updated weights for policy 0, policy_version 1024271 (0.0009) [2023-12-27 12:44:49,884][20347] Updated weights for policy 0, policy_version 1024281 (0.0009) [2023-12-27 12:44:49,947][20347] Updated weights for policy 0, policy_version 1024291 (0.0009) [2023-12-27 12:44:50,175][20381] Updated weights for policy 1, policy_version 1023577 (0.0008) [2023-12-27 12:44:50,237][20381] Updated weights for policy 1, policy_version 1023587 (0.0009) [2023-12-27 12:44:50,298][20381] Updated weights for policy 1, policy_version 1023597 (0.0008) [2023-12-27 12:44:50,362][20381] Updated weights for policy 1, policy_version 1023607 (0.0008) [2023-12-27 12:44:50,543][20347] Updated weights for policy 0, policy_version 1024301 (0.0007) [2023-12-27 12:44:50,602][20347] Updated weights for policy 0, policy_version 1024311 (0.0007) [2023-12-27 12:44:50,657][20347] Updated weights for policy 0, policy_version 1024321 (0.0008) [2023-12-27 12:44:51,115][20381] Updated weights for policy 1, policy_version 1023617 (0.0008) [2023-12-27 12:44:51,177][20381] Updated weights for policy 1, policy_version 1023627 (0.0008) [2023-12-27 12:44:51,235][20381] Updated weights for policy 1, policy_version 1023637 (0.0007) [2023-12-27 12:44:51,286][20347] Updated weights for policy 0, policy_version 1024331 (0.0007) [2023-12-27 12:44:51,342][20347] Updated weights for policy 0, policy_version 1024341 (0.0007) [2023-12-27 12:44:51,409][20347] Updated weights for policy 0, policy_version 1024351 (0.0008) [2023-12-27 12:44:52,022][20381] Updated weights for policy 1, policy_version 1023647 (0.0008) [2023-12-27 12:44:52,074][20381] Updated weights for policy 1, policy_version 1023657 (0.0008) [2023-12-27 12:44:52,083][20347] Updated weights for policy 0, policy_version 1024361 (0.0007) [2023-12-27 12:44:52,127][20381] Updated weights for policy 1, policy_version 1023667 (0.0005) [2023-12-27 12:44:52,135][20347] Updated weights for policy 0, policy_version 1024371 (0.0006) [2023-12-27 12:44:52,186][20347] Updated weights for policy 0, policy_version 1024381 (0.0008) [2023-12-27 12:44:52,236][20347] Updated weights for policy 0, policy_version 1024391 (0.0009) [2023-12-27 12:44:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 524378112. Throughput: 0: 10376.1, 1: 10218.6. Samples: 524344244. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:52,363][19170] Avg episode reward: [(0, '863.460'), (1, '246.190')] [2023-12-27 12:44:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001023672_262086656.pth... [2023-12-27 12:44:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001024392_262291456.pth... [2023-12-27 12:44:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001022488_261783552.pth [2023-12-27 12:44:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001023176_261980160.pth [2023-12-27 12:44:52,819][20381] Updated weights for policy 1, policy_version 1023677 (0.0005) [2023-12-27 12:44:52,880][20381] Updated weights for policy 1, policy_version 1023687 (0.0007) [2023-12-27 12:44:52,940][20381] Updated weights for policy 1, policy_version 1023697 (0.0006) [2023-12-27 12:44:52,958][20347] Updated weights for policy 0, policy_version 1024401 (0.0007) [2023-12-27 12:44:53,009][20347] Updated weights for policy 0, policy_version 1024411 (0.0008) [2023-12-27 12:44:53,063][20347] Updated weights for policy 0, policy_version 1024421 (0.0008) [2023-12-27 12:44:53,580][20381] Updated weights for policy 1, policy_version 1023707 (0.0006) [2023-12-27 12:44:53,627][20381] Updated weights for policy 1, policy_version 1023717 (0.0008) [2023-12-27 12:44:53,632][20347] Updated weights for policy 0, policy_version 1024431 (0.0006) [2023-12-27 12:44:53,682][20347] Updated weights for policy 0, policy_version 1024441 (0.0006) [2023-12-27 12:44:53,684][20381] Updated weights for policy 1, policy_version 1023727 (0.0008) [2023-12-27 12:44:53,731][20347] Updated weights for policy 0, policy_version 1024451 (0.0006) [2023-12-27 12:44:54,297][20381] Updated weights for policy 1, policy_version 1023737 (0.0007) [2023-12-27 12:44:54,348][20381] Updated weights for policy 1, policy_version 1023747 (0.0005) [2023-12-27 12:44:54,377][20347] Updated weights for policy 0, policy_version 1024461 (0.0009) [2023-12-27 12:44:54,399][20381] Updated weights for policy 1, policy_version 1023757 (0.0006) [2023-12-27 12:44:54,435][20347] Updated weights for policy 0, policy_version 1024471 (0.0010) [2023-12-27 12:44:54,452][20381] Updated weights for policy 1, policy_version 1023767 (0.0005) [2023-12-27 12:44:54,494][20347] Updated weights for policy 0, policy_version 1024481 (0.0009) [2023-12-27 12:44:55,046][20381] Updated weights for policy 1, policy_version 1023777 (0.0008) [2023-12-27 12:44:55,101][20381] Updated weights for policy 1, policy_version 1023787 (0.0007) [2023-12-27 12:44:55,111][20347] Updated weights for policy 0, policy_version 1024491 (0.0007) [2023-12-27 12:44:55,151][20381] Updated weights for policy 1, policy_version 1023797 (0.0008) [2023-12-27 12:44:55,165][20347] Updated weights for policy 0, policy_version 1024501 (0.0005) [2023-12-27 12:44:55,222][20347] Updated weights for policy 0, policy_version 1024511 (0.0007) [2023-12-27 12:44:55,842][20347] Updated weights for policy 0, policy_version 1024521 (0.0010) [2023-12-27 12:44:55,848][20381] Updated weights for policy 1, policy_version 1023807 (0.0006) [2023-12-27 12:44:55,900][20381] Updated weights for policy 1, policy_version 1023817 (0.0007) [2023-12-27 12:44:55,903][20347] Updated weights for policy 0, policy_version 1024531 (0.0008) [2023-12-27 12:44:55,959][20347] Updated weights for policy 0, policy_version 1024541 (0.0007) [2023-12-27 12:44:55,961][20381] Updated weights for policy 1, policy_version 1023827 (0.0007) [2023-12-27 12:44:56,016][20347] Updated weights for policy 0, policy_version 1024551 (0.0006) [2023-12-27 12:44:56,597][20381] Updated weights for policy 1, policy_version 1023837 (0.0007) [2023-12-27 12:44:56,643][20381] Updated weights for policy 1, policy_version 1023847 (0.0007) [2023-12-27 12:44:56,667][20347] Updated weights for policy 0, policy_version 1024561 (0.0006) [2023-12-27 12:44:56,698][20381] Updated weights for policy 1, policy_version 1023857 (0.0006) [2023-12-27 12:44:56,720][20347] Updated weights for policy 0, policy_version 1024571 (0.0005) [2023-12-27 12:44:56,771][20347] Updated weights for policy 0, policy_version 1024581 (0.0007) [2023-12-27 12:44:57,341][20381] Updated weights for policy 1, policy_version 1023867 (0.0008) [2023-12-27 12:44:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 524476416. Throughput: 0: 10391.8, 1: 10211.4. Samples: 524470088. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:44:57,363][19170] Avg episode reward: [(0, '873.210'), (1, '310.450')] [2023-12-27 12:44:57,389][20347] Updated weights for policy 0, policy_version 1024591 (0.0006) [2023-12-27 12:44:57,391][20381] Updated weights for policy 1, policy_version 1023877 (0.0006) [2023-12-27 12:44:57,438][20347] Updated weights for policy 0, policy_version 1024601 (0.0005) [2023-12-27 12:44:57,440][20381] Updated weights for policy 1, policy_version 1023887 (0.0006) [2023-12-27 12:44:57,490][20347] Updated weights for policy 0, policy_version 1024611 (0.0006) [2023-12-27 12:44:58,067][20381] Updated weights for policy 1, policy_version 1023897 (0.0005) [2023-12-27 12:44:58,121][20381] Updated weights for policy 1, policy_version 1023907 (0.0005) [2023-12-27 12:44:58,172][20381] Updated weights for policy 1, policy_version 1023917 (0.0007) [2023-12-27 12:44:58,233][20347] Updated weights for policy 0, policy_version 1024621 (0.0007) [2023-12-27 12:44:58,236][20381] Updated weights for policy 1, policy_version 1023927 (0.0008) [2023-12-27 12:44:58,292][20347] Updated weights for policy 0, policy_version 1024631 (0.0008) [2023-12-27 12:44:58,361][20347] Updated weights for policy 0, policy_version 1024641 (0.0008) [2023-12-27 12:44:58,370][20291] KL-divergence is very high: 128.0432 [2023-12-27 12:44:58,377][20291] KL-divergence is very high: 153.0067 [2023-12-27 12:44:58,997][20381] Updated weights for policy 1, policy_version 1023937 (0.0006) [2023-12-27 12:44:59,054][20381] Updated weights for policy 1, policy_version 1023947 (0.0007) [2023-12-27 12:44:59,110][20381] Updated weights for policy 1, policy_version 1023957 (0.0008) [2023-12-27 12:44:59,165][20347] Updated weights for policy 0, policy_version 1024651 (0.0007) [2023-12-27 12:44:59,233][20347] Updated weights for policy 0, policy_version 1024661 (0.0011) [2023-12-27 12:44:59,292][20347] Updated weights for policy 0, policy_version 1024671 (0.0007) [2023-12-27 12:44:59,976][20381] Updated weights for policy 1, policy_version 1023967 (0.0008) [2023-12-27 12:45:00,031][20347] Updated weights for policy 0, policy_version 1024681 (0.0007) [2023-12-27 12:45:00,037][20381] Updated weights for policy 1, policy_version 1023977 (0.0008) [2023-12-27 12:45:00,084][20347] Updated weights for policy 0, policy_version 1024691 (0.0006) [2023-12-27 12:45:00,096][20381] Updated weights for policy 1, policy_version 1023987 (0.0008) [2023-12-27 12:45:00,140][20347] Updated weights for policy 0, policy_version 1024701 (0.0007) [2023-12-27 12:45:00,200][20347] Updated weights for policy 0, policy_version 1024711 (0.0007) [2023-12-27 12:45:00,780][20381] Updated weights for policy 1, policy_version 1023997 (0.0008) [2023-12-27 12:45:00,840][20381] Updated weights for policy 1, policy_version 1024007 (0.0008) [2023-12-27 12:45:00,902][20381] Updated weights for policy 1, policy_version 1024017 (0.0007) [2023-12-27 12:45:00,915][20347] Updated weights for policy 0, policy_version 1024721 (0.0007) [2023-12-27 12:45:00,981][20347] Updated weights for policy 0, policy_version 1024731 (0.0007) [2023-12-27 12:45:01,043][20347] Updated weights for policy 0, policy_version 1024741 (0.0008) [2023-12-27 12:45:01,568][20381] Updated weights for policy 1, policy_version 1024027 (0.0006) [2023-12-27 12:45:01,634][20381] Updated weights for policy 1, policy_version 1024037 (0.0007) [2023-12-27 12:45:01,696][20381] Updated weights for policy 1, policy_version 1024047 (0.0006) [2023-12-27 12:45:01,775][20347] Updated weights for policy 0, policy_version 1024751 (0.0008) [2023-12-27 12:45:01,823][20347] Updated weights for policy 0, policy_version 1024761 (0.0008) [2023-12-27 12:45:01,881][20347] Updated weights for policy 0, policy_version 1024771 (0.0007) [2023-12-27 12:45:02,325][20381] Updated weights for policy 1, policy_version 1024057 (0.0007) [2023-12-27 12:45:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 524574720. Throughput: 0: 10332.0, 1: 10188.5. Samples: 524590092. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:45:02,363][19170] Avg episode reward: [(0, '883.020'), (1, '216.930')] [2023-12-27 12:45:02,392][20381] Updated weights for policy 1, policy_version 1024067 (0.0008) [2023-12-27 12:45:02,451][20381] Updated weights for policy 1, policy_version 1024077 (0.0007) [2023-12-27 12:45:02,510][20381] Updated weights for policy 1, policy_version 1024087 (0.0008) [2023-12-27 12:45:02,571][20347] Updated weights for policy 0, policy_version 1024781 (0.0006) [2023-12-27 12:45:02,623][20347] Updated weights for policy 0, policy_version 1024791 (0.0007) [2023-12-27 12:45:02,679][20347] Updated weights for policy 0, policy_version 1024801 (0.0007) [2023-12-27 12:45:03,225][20381] Updated weights for policy 1, policy_version 1024097 (0.0007) [2023-12-27 12:45:03,250][20347] Updated weights for policy 0, policy_version 1024811 (0.0006) [2023-12-27 12:45:03,277][20381] Updated weights for policy 1, policy_version 1024107 (0.0005) [2023-12-27 12:45:03,302][20347] Updated weights for policy 0, policy_version 1024821 (0.0007) [2023-12-27 12:45:03,324][20381] Updated weights for policy 1, policy_version 1024117 (0.0005) [2023-12-27 12:45:03,344][20347] Updated weights for policy 0, policy_version 1024831 (0.0007) [2023-12-27 12:45:03,917][20381] Updated weights for policy 1, policy_version 1024127 (0.0007) [2023-12-27 12:45:03,969][20381] Updated weights for policy 1, policy_version 1024137 (0.0005) [2023-12-27 12:45:04,021][20381] Updated weights for policy 1, policy_version 1024147 (0.0005) [2023-12-27 12:45:04,029][20347] Updated weights for policy 0, policy_version 1024841 (0.0007) [2023-12-27 12:45:04,090][20347] Updated weights for policy 0, policy_version 1024851 (0.0007) [2023-12-27 12:45:04,153][20347] Updated weights for policy 0, policy_version 1024861 (0.0008) [2023-12-27 12:45:04,216][20347] Updated weights for policy 0, policy_version 1024871 (0.0009) [2023-12-27 12:45:04,687][20381] Updated weights for policy 1, policy_version 1024157 (0.0006) [2023-12-27 12:45:04,746][20381] Updated weights for policy 1, policy_version 1024167 (0.0007) [2023-12-27 12:45:04,812][20381] Updated weights for policy 1, policy_version 1024177 (0.0008) [2023-12-27 12:45:04,899][20347] Updated weights for policy 0, policy_version 1024881 (0.0006) [2023-12-27 12:45:04,953][20347] Updated weights for policy 0, policy_version 1024891 (0.0006) [2023-12-27 12:45:05,007][20347] Updated weights for policy 0, policy_version 1024901 (0.0005) [2023-12-27 12:45:05,521][20381] Updated weights for policy 1, policy_version 1024187 (0.0008) [2023-12-27 12:45:05,576][20381] Updated weights for policy 1, policy_version 1024197 (0.0007) [2023-12-27 12:45:05,581][20347] Updated weights for policy 0, policy_version 1024911 (0.0006) [2023-12-27 12:45:05,628][20347] Updated weights for policy 0, policy_version 1024921 (0.0006) [2023-12-27 12:45:05,630][20381] Updated weights for policy 1, policy_version 1024207 (0.0007) [2023-12-27 12:45:05,674][20347] Updated weights for policy 0, policy_version 1024931 (0.0006) [2023-12-27 12:45:06,339][20381] Updated weights for policy 1, policy_version 1024217 (0.0006) [2023-12-27 12:45:06,372][20347] Updated weights for policy 0, policy_version 1024941 (0.0009) [2023-12-27 12:45:06,408][20381] Updated weights for policy 1, policy_version 1024227 (0.0007) [2023-12-27 12:45:06,428][20347] Updated weights for policy 0, policy_version 1024951 (0.0010) [2023-12-27 12:45:06,475][20381] Updated weights for policy 1, policy_version 1024237 (0.0006) [2023-12-27 12:45:06,488][20347] Updated weights for policy 0, policy_version 1024961 (0.0010) [2023-12-27 12:45:06,534][20381] Updated weights for policy 1, policy_version 1024247 (0.0005) [2023-12-27 12:45:07,175][20347] Updated weights for policy 0, policy_version 1024971 (0.0009) [2023-12-27 12:45:07,225][20381] Updated weights for policy 1, policy_version 1024257 (0.0008) [2023-12-27 12:45:07,233][20347] Updated weights for policy 0, policy_version 1024981 (0.0006) [2023-12-27 12:45:07,281][20381] Updated weights for policy 1, policy_version 1024267 (0.0006) [2023-12-27 12:45:07,297][20347] Updated weights for policy 0, policy_version 1024991 (0.0008) [2023-12-27 12:45:07,338][20381] Updated weights for policy 1, policy_version 1024277 (0.0007) [2023-12-27 12:45:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 524689408. Throughput: 0: 10348.3, 1: 10198.0. Samples: 524653128. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:45:07,363][19170] Avg episode reward: [(0, '912.400'), (1, '319.480')] [2023-12-27 12:45:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001024280_262242304.pth... [2023-12-27 12:45:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001025000_262447104.pth... [2023-12-27 12:45:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001023096_261939200.pth [2023-12-27 12:45:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001023784_262135808.pth [2023-12-27 12:45:07,854][20347] Updated weights for policy 0, policy_version 1025001 (0.0007) [2023-12-27 12:45:07,917][20347] Updated weights for policy 0, policy_version 1025011 (0.0010) [2023-12-27 12:45:07,975][20347] Updated weights for policy 0, policy_version 1025021 (0.0006) [2023-12-27 12:45:07,982][20381] Updated weights for policy 1, policy_version 1024287 (0.0008) [2023-12-27 12:45:08,026][20347] Updated weights for policy 0, policy_version 1025031 (0.0005) [2023-12-27 12:45:08,039][20381] Updated weights for policy 1, policy_version 1024297 (0.0007) [2023-12-27 12:45:08,096][20381] Updated weights for policy 1, policy_version 1024307 (0.0009) [2023-12-27 12:45:08,593][20347] Updated weights for policy 0, policy_version 1025041 (0.0008) [2023-12-27 12:45:08,652][20347] Updated weights for policy 0, policy_version 1025051 (0.0007) [2023-12-27 12:45:08,706][20347] Updated weights for policy 0, policy_version 1025061 (0.0008) [2023-12-27 12:45:08,829][20381] Updated weights for policy 1, policy_version 1024317 (0.0008) [2023-12-27 12:45:08,892][20381] Updated weights for policy 1, policy_version 1024327 (0.0008) [2023-12-27 12:45:08,945][20381] Updated weights for policy 1, policy_version 1024337 (0.0006) [2023-12-27 12:45:09,406][20347] Updated weights for policy 0, policy_version 1025071 (0.0007) [2023-12-27 12:45:09,474][20347] Updated weights for policy 0, policy_version 1025081 (0.0007) [2023-12-27 12:45:09,540][20347] Updated weights for policy 0, policy_version 1025091 (0.0006) [2023-12-27 12:45:09,596][20381] Updated weights for policy 1, policy_version 1024347 (0.0006) [2023-12-27 12:45:09,658][20381] Updated weights for policy 1, policy_version 1024357 (0.0008) [2023-12-27 12:45:09,723][20381] Updated weights for policy 1, policy_version 1024367 (0.0005) [2023-12-27 12:45:10,220][20347] Updated weights for policy 0, policy_version 1025101 (0.0007) [2023-12-27 12:45:10,280][20347] Updated weights for policy 0, policy_version 1025111 (0.0007) [2023-12-27 12:45:10,340][20347] Updated weights for policy 0, policy_version 1025121 (0.0008) [2023-12-27 12:45:10,418][20381] Updated weights for policy 1, policy_version 1024377 (0.0008) [2023-12-27 12:45:10,486][20381] Updated weights for policy 1, policy_version 1024387 (0.0007) [2023-12-27 12:45:10,540][20381] Updated weights for policy 1, policy_version 1024397 (0.0008) [2023-12-27 12:45:10,601][20381] Updated weights for policy 1, policy_version 1024407 (0.0007) [2023-12-27 12:45:11,006][20347] Updated weights for policy 0, policy_version 1025131 (0.0009) [2023-12-27 12:45:11,071][20347] Updated weights for policy 0, policy_version 1025141 (0.0010) [2023-12-27 12:45:11,136][20347] Updated weights for policy 0, policy_version 1025151 (0.0010) [2023-12-27 12:45:11,350][20381] Updated weights for policy 1, policy_version 1024417 (0.0009) [2023-12-27 12:45:11,428][20381] Updated weights for policy 1, policy_version 1024427 (0.0009) [2023-12-27 12:45:11,490][20381] Updated weights for policy 1, policy_version 1024437 (0.0006) [2023-12-27 12:45:11,911][20347] Updated weights for policy 0, policy_version 1025161 (0.0010) [2023-12-27 12:45:11,972][20347] Updated weights for policy 0, policy_version 1025171 (0.0007) [2023-12-27 12:45:12,038][20347] Updated weights for policy 0, policy_version 1025181 (0.0006) [2023-12-27 12:45:12,100][20347] Updated weights for policy 0, policy_version 1025191 (0.0005) [2023-12-27 12:45:12,202][20381] Updated weights for policy 1, policy_version 1024447 (0.0008) [2023-12-27 12:45:12,254][20381] Updated weights for policy 1, policy_version 1024457 (0.0008) [2023-12-27 12:45:12,315][20381] Updated weights for policy 1, policy_version 1024467 (0.0007) [2023-12-27 12:45:12,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 524787712. Throughput: 0: 10388.3, 1: 10148.8. Samples: 524775116. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:45:12,363][19170] Avg episode reward: [(0, '912.460'), (1, '334.190')] [2023-12-27 12:45:12,723][20347] Updated weights for policy 0, policy_version 1025201 (0.0006) [2023-12-27 12:45:12,782][20347] Updated weights for policy 0, policy_version 1025211 (0.0006) [2023-12-27 12:45:12,843][20347] Updated weights for policy 0, policy_version 1025221 (0.0006) [2023-12-27 12:45:12,991][20381] Updated weights for policy 1, policy_version 1024477 (0.0007) [2023-12-27 12:45:13,045][20381] Updated weights for policy 1, policy_version 1024487 (0.0007) [2023-12-27 12:45:13,095][20381] Updated weights for policy 1, policy_version 1024497 (0.0006) [2023-12-27 12:45:13,468][20347] Updated weights for policy 0, policy_version 1025231 (0.0009) [2023-12-27 12:45:13,528][20347] Updated weights for policy 0, policy_version 1025241 (0.0010) [2023-12-27 12:45:13,581][20347] Updated weights for policy 0, policy_version 1025251 (0.0010) [2023-12-27 12:45:13,756][20381] Updated weights for policy 1, policy_version 1024507 (0.0006) [2023-12-27 12:45:13,803][20381] Updated weights for policy 1, policy_version 1024517 (0.0006) [2023-12-27 12:45:13,862][20381] Updated weights for policy 1, policy_version 1024527 (0.0005) [2023-12-27 12:45:14,206][20347] Updated weights for policy 0, policy_version 1025261 (0.0010) [2023-12-27 12:45:14,254][20347] Updated weights for policy 0, policy_version 1025271 (0.0010) [2023-12-27 12:45:14,300][20347] Updated weights for policy 0, policy_version 1025281 (0.0010) [2023-12-27 12:45:14,570][20381] Updated weights for policy 1, policy_version 1024537 (0.0007) [2023-12-27 12:45:14,626][20381] Updated weights for policy 1, policy_version 1024547 (0.0007) [2023-12-27 12:45:14,673][20381] Updated weights for policy 1, policy_version 1024557 (0.0007) [2023-12-27 12:45:14,723][20381] Updated weights for policy 1, policy_version 1024567 (0.0007) [2023-12-27 12:45:14,987][20347] Updated weights for policy 0, policy_version 1025291 (0.0009) [2023-12-27 12:45:15,044][20347] Updated weights for policy 0, policy_version 1025301 (0.0011) [2023-12-27 12:45:15,094][20347] Updated weights for policy 0, policy_version 1025311 (0.0009) [2023-12-27 12:45:15,368][20381] Updated weights for policy 1, policy_version 1024577 (0.0007) [2023-12-27 12:45:15,428][20381] Updated weights for policy 1, policy_version 1024587 (0.0007) [2023-12-27 12:45:15,484][20381] Updated weights for policy 1, policy_version 1024597 (0.0005) [2023-12-27 12:45:15,778][20347] Updated weights for policy 0, policy_version 1025321 (0.0008) [2023-12-27 12:45:15,836][20347] Updated weights for policy 0, policy_version 1025331 (0.0010) [2023-12-27 12:45:15,887][20347] Updated weights for policy 0, policy_version 1025341 (0.0006) [2023-12-27 12:45:15,936][20347] Updated weights for policy 0, policy_version 1025351 (0.0005) [2023-12-27 12:45:16,072][20381] Updated weights for policy 1, policy_version 1024607 (0.0007) [2023-12-27 12:45:16,119][20381] Updated weights for policy 1, policy_version 1024617 (0.0008) [2023-12-27 12:45:16,168][20381] Updated weights for policy 1, policy_version 1024627 (0.0005) [2023-12-27 12:45:16,579][20347] Updated weights for policy 0, policy_version 1025361 (0.0007) [2023-12-27 12:45:16,638][20347] Updated weights for policy 0, policy_version 1025371 (0.0005) [2023-12-27 12:45:16,691][20347] Updated weights for policy 0, policy_version 1025381 (0.0005) [2023-12-27 12:45:16,888][20381] Updated weights for policy 1, policy_version 1024637 (0.0007) [2023-12-27 12:45:16,936][20381] Updated weights for policy 1, policy_version 1024647 (0.0007) [2023-12-27 12:45:16,983][20381] Updated weights for policy 1, policy_version 1024657 (0.0007) [2023-12-27 12:45:17,304][20347] Updated weights for policy 0, policy_version 1025391 (0.0006) [2023-12-27 12:45:17,336][20291] KL-divergence is very high: 159.4721 [2023-12-27 12:45:17,361][20347] Updated weights for policy 0, policy_version 1025401 (0.0005) [2023-12-27 12:45:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 524886016. Throughput: 0: 10406.4, 1: 10104.7. Samples: 524898824. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:45:17,363][19170] Avg episode reward: [(0, '922.300'), (1, '433.030')] [2023-12-27 12:45:17,410][20347] Updated weights for policy 0, policy_version 1025411 (0.0005) [2023-12-27 12:45:17,708][20381] Updated weights for policy 1, policy_version 1024667 (0.0007) [2023-12-27 12:45:17,780][20381] Updated weights for policy 1, policy_version 1024677 (0.0006) [2023-12-27 12:45:17,838][20381] Updated weights for policy 1, policy_version 1024687 (0.0007) [2023-12-27 12:45:18,060][20347] Updated weights for policy 0, policy_version 1025421 (0.0008) [2023-12-27 12:45:18,124][20347] Updated weights for policy 0, policy_version 1025431 (0.0010) [2023-12-27 12:45:18,184][20347] Updated weights for policy 0, policy_version 1025441 (0.0010) [2023-12-27 12:45:18,487][20381] Updated weights for policy 1, policy_version 1024697 (0.0007) [2023-12-27 12:45:18,547][20381] Updated weights for policy 1, policy_version 1024707 (0.0008) [2023-12-27 12:45:18,612][20381] Updated weights for policy 1, policy_version 1024717 (0.0008) [2023-12-27 12:45:18,670][20381] Updated weights for policy 1, policy_version 1024727 (0.0007) [2023-12-27 12:45:18,846][20347] Updated weights for policy 0, policy_version 1025451 (0.0009) [2023-12-27 12:45:18,907][20347] Updated weights for policy 0, policy_version 1025461 (0.0008) [2023-12-27 12:45:18,967][20347] Updated weights for policy 0, policy_version 1025471 (0.0008) [2023-12-27 12:45:19,310][20381] Updated weights for policy 1, policy_version 1024737 (0.0010) [2023-12-27 12:45:19,377][20381] Updated weights for policy 1, policy_version 1024747 (0.0011) [2023-12-27 12:45:19,438][20381] Updated weights for policy 1, policy_version 1024757 (0.0010) [2023-12-27 12:45:19,643][20347] Updated weights for policy 0, policy_version 1025481 (0.0008) [2023-12-27 12:45:19,706][20347] Updated weights for policy 0, policy_version 1025491 (0.0008) [2023-12-27 12:45:19,770][20347] Updated weights for policy 0, policy_version 1025501 (0.0008) [2023-12-27 12:45:19,839][20347] Updated weights for policy 0, policy_version 1025511 (0.0008) [2023-12-27 12:45:20,147][20381] Updated weights for policy 1, policy_version 1024767 (0.0008) [2023-12-27 12:45:20,210][20381] Updated weights for policy 1, policy_version 1024777 (0.0006) [2023-12-27 12:45:20,277][20381] Updated weights for policy 1, policy_version 1024787 (0.0007) [2023-12-27 12:45:20,573][20347] Updated weights for policy 0, policy_version 1025521 (0.0010) [2023-12-27 12:45:20,631][20347] Updated weights for policy 0, policy_version 1025531 (0.0008) [2023-12-27 12:45:20,685][20347] Updated weights for policy 0, policy_version 1025541 (0.0008) [2023-12-27 12:45:20,920][20381] Updated weights for policy 1, policy_version 1024797 (0.0008) [2023-12-27 12:45:20,979][20381] Updated weights for policy 1, policy_version 1024807 (0.0007) [2023-12-27 12:45:21,047][20381] Updated weights for policy 1, policy_version 1024817 (0.0008) [2023-12-27 12:45:21,392][20347] Updated weights for policy 0, policy_version 1025551 (0.0007) [2023-12-27 12:45:21,448][20347] Updated weights for policy 0, policy_version 1025561 (0.0010) [2023-12-27 12:45:21,503][20347] Updated weights for policy 0, policy_version 1025571 (0.0010) [2023-12-27 12:45:21,830][20381] Updated weights for policy 1, policy_version 1024827 (0.0007) [2023-12-27 12:45:21,880][20381] Updated weights for policy 1, policy_version 1024837 (0.0008) [2023-12-27 12:45:21,942][20381] Updated weights for policy 1, policy_version 1024847 (0.0006) [2023-12-27 12:45:22,262][20347] Updated weights for policy 0, policy_version 1025581 (0.0010) [2023-12-27 12:45:22,327][20347] Updated weights for policy 0, policy_version 1025591 (0.0011) [2023-12-27 12:45:22,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 524984320. Throughput: 0: 10404.9, 1: 10094.6. Samples: 524960024. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-12-27 12:45:22,363][19170] Avg episode reward: [(0, '873.720'), (1, '394.740')] [2023-12-27 12:45:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001024856_262389760.pth... [2023-12-27 12:45:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001023672_262086656.pth [2023-12-27 12:45:22,399][20347] Updated weights for policy 0, policy_version 1025601 (0.0008) [2023-12-27 12:45:22,442][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001025608_262602752.pth... [2023-12-27 12:45:22,447][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001024392_262291456.pth [2023-12-27 12:45:22,633][20381] Updated weights for policy 1, policy_version 1024857 (0.0007) [2023-12-27 12:45:22,682][20381] Updated weights for policy 1, policy_version 1024867 (0.0008) [2023-12-27 12:45:22,732][20381] Updated weights for policy 1, policy_version 1024877 (0.0007) [2023-12-27 12:45:22,785][20381] Updated weights for policy 1, policy_version 1024887 (0.0007) [2023-12-27 12:45:23,068][20347] Updated weights for policy 0, policy_version 1025611 (0.0008) [2023-12-27 12:45:23,129][20347] Updated weights for policy 0, policy_version 1025621 (0.0006) [2023-12-27 12:45:23,182][20347] Updated weights for policy 0, policy_version 1025631 (0.0007) [2023-12-27 12:45:23,571][20381] Updated weights for policy 1, policy_version 1024897 (0.0006) [2023-12-27 12:45:23,622][20381] Updated weights for policy 1, policy_version 1024907 (0.0007) [2023-12-27 12:45:23,676][20381] Updated weights for policy 1, policy_version 1024917 (0.0007) [2023-12-27 12:45:23,818][20347] Updated weights for policy 0, policy_version 1025641 (0.0007) [2023-12-27 12:45:23,879][20347] Updated weights for policy 0, policy_version 1025651 (0.0006) [2023-12-27 12:45:23,942][20347] Updated weights for policy 0, policy_version 1025661 (0.0008) [2023-12-27 12:45:23,995][20347] Updated weights for policy 0, policy_version 1025671 (0.0008) [2023-12-27 12:45:24,383][20381] Updated weights for policy 1, policy_version 1024927 (0.0007) [2023-12-27 12:45:24,444][20381] Updated weights for policy 1, policy_version 1024937 (0.0008) [2023-12-27 12:45:24,506][20381] Updated weights for policy 1, policy_version 1024947 (0.0007) [2023-12-27 12:45:24,679][20347] Updated weights for policy 0, policy_version 1025681 (0.0008) [2023-12-27 12:45:24,735][20347] Updated weights for policy 0, policy_version 1025691 (0.0008) [2023-12-27 12:45:24,785][20347] Updated weights for policy 0, policy_version 1025701 (0.0008) [2023-12-27 12:45:25,125][20381] Updated weights for policy 1, policy_version 1024957 (0.0006) [2023-12-27 12:45:25,182][20381] Updated weights for policy 1, policy_version 1024967 (0.0006) [2023-12-27 12:45:25,235][20381] Updated weights for policy 1, policy_version 1024977 (0.0008) [2023-12-27 12:45:25,460][20347] Updated weights for policy 0, policy_version 1025711 (0.0007) [2023-12-27 12:45:25,508][20347] Updated weights for policy 0, policy_version 1025721 (0.0007) [2023-12-27 12:45:25,561][20347] Updated weights for policy 0, policy_version 1025731 (0.0007) [2023-12-27 12:45:25,890][20381] Updated weights for policy 1, policy_version 1024987 (0.0008) [2023-12-27 12:45:25,955][20381] Updated weights for policy 1, policy_version 1024997 (0.0007) [2023-12-27 12:45:26,015][20381] Updated weights for policy 1, policy_version 1025007 (0.0007) [2023-12-27 12:45:26,336][20347] Updated weights for policy 0, policy_version 1025741 (0.0007) [2023-12-27 12:45:26,392][20347] Updated weights for policy 0, policy_version 1025751 (0.0005) [2023-12-27 12:45:26,450][20347] Updated weights for policy 0, policy_version 1025761 (0.0005) [2023-12-27 12:45:26,664][20381] Updated weights for policy 1, policy_version 1025017 (0.0006) [2023-12-27 12:45:26,721][20381] Updated weights for policy 1, policy_version 1025027 (0.0006) [2023-12-27 12:45:26,780][20381] Updated weights for policy 1, policy_version 1025037 (0.0010) [2023-12-27 12:45:26,839][20381] Updated weights for policy 1, policy_version 1025047 (0.0010) [2023-12-27 12:45:27,039][20347] Updated weights for policy 0, policy_version 1025771 (0.0007) [2023-12-27 12:45:27,101][20347] Updated weights for policy 0, policy_version 1025781 (0.0011) [2023-12-27 12:45:27,159][20347] Updated weights for policy 0, policy_version 1025791 (0.0011) [2023-12-27 12:45:27,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 525090816. Throughput: 0: 10411.4, 1: 10067.8. Samples: 525079440. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:27,363][19170] Avg episode reward: [(0, '824.900'), (1, '458.960')] [2023-12-27 12:45:27,439][20381] Updated weights for policy 1, policy_version 1025057 (0.0010) [2023-12-27 12:45:27,500][20381] Updated weights for policy 1, policy_version 1025067 (0.0010) [2023-12-27 12:45:27,556][20381] Updated weights for policy 1, policy_version 1025077 (0.0010) [2023-12-27 12:45:27,790][20347] Updated weights for policy 0, policy_version 1025801 (0.0010) [2023-12-27 12:45:27,839][20347] Updated weights for policy 0, policy_version 1025811 (0.0007) [2023-12-27 12:45:27,881][20347] Updated weights for policy 0, policy_version 1025821 (0.0006) [2023-12-27 12:45:27,934][20347] Updated weights for policy 0, policy_version 1025831 (0.0007) [2023-12-27 12:45:28,232][20381] Updated weights for policy 1, policy_version 1025087 (0.0009) [2023-12-27 12:45:28,284][20381] Updated weights for policy 1, policy_version 1025097 (0.0010) [2023-12-27 12:45:28,348][20381] Updated weights for policy 1, policy_version 1025107 (0.0011) [2023-12-27 12:45:28,609][20347] Updated weights for policy 0, policy_version 1025841 (0.0007) [2023-12-27 12:45:28,666][20347] Updated weights for policy 0, policy_version 1025851 (0.0008) [2023-12-27 12:45:28,724][20347] Updated weights for policy 0, policy_version 1025861 (0.0007) [2023-12-27 12:45:29,053][20381] Updated weights for policy 1, policy_version 1025117 (0.0008) [2023-12-27 12:45:29,109][20381] Updated weights for policy 1, policy_version 1025127 (0.0007) [2023-12-27 12:45:29,154][20381] Updated weights for policy 1, policy_version 1025137 (0.0010) [2023-12-27 12:45:29,426][20347] Updated weights for policy 0, policy_version 1025871 (0.0008) [2023-12-27 12:45:29,475][20347] Updated weights for policy 0, policy_version 1025881 (0.0007) [2023-12-27 12:45:29,524][20347] Updated weights for policy 0, policy_version 1025891 (0.0008) [2023-12-27 12:45:29,842][20381] Updated weights for policy 1, policy_version 1025147 (0.0009) [2023-12-27 12:45:29,908][20381] Updated weights for policy 1, policy_version 1025157 (0.0007) [2023-12-27 12:45:29,971][20381] Updated weights for policy 1, policy_version 1025167 (0.0008) [2023-12-27 12:45:30,232][20347] Updated weights for policy 0, policy_version 1025901 (0.0009) [2023-12-27 12:45:30,289][20347] Updated weights for policy 0, policy_version 1025911 (0.0010) [2023-12-27 12:45:30,346][20347] Updated weights for policy 0, policy_version 1025921 (0.0011) [2023-12-27 12:45:30,616][20381] Updated weights for policy 1, policy_version 1025177 (0.0008) [2023-12-27 12:45:30,682][20381] Updated weights for policy 1, policy_version 1025187 (0.0008) [2023-12-27 12:45:30,745][20381] Updated weights for policy 1, policy_version 1025197 (0.0008) [2023-12-27 12:45:30,804][20381] Updated weights for policy 1, policy_version 1025207 (0.0009) [2023-12-27 12:45:31,028][20347] Updated weights for policy 0, policy_version 1025931 (0.0010) [2023-12-27 12:45:31,096][20347] Updated weights for policy 0, policy_version 1025941 (0.0009) [2023-12-27 12:45:31,168][20347] Updated weights for policy 0, policy_version 1025951 (0.0010) [2023-12-27 12:45:31,633][20381] Updated weights for policy 1, policy_version 1025217 (0.0008) [2023-12-27 12:45:31,695][20381] Updated weights for policy 1, policy_version 1025227 (0.0009) [2023-12-27 12:45:31,763][20381] Updated weights for policy 1, policy_version 1025237 (0.0007) [2023-12-27 12:45:31,904][20347] Updated weights for policy 0, policy_version 1025961 (0.0010) [2023-12-27 12:45:31,958][20347] Updated weights for policy 0, policy_version 1025971 (0.0010) [2023-12-27 12:45:32,020][20347] Updated weights for policy 0, policy_version 1025981 (0.0011) [2023-12-27 12:45:32,081][20347] Updated weights for policy 0, policy_version 1025991 (0.0009) [2023-12-27 12:45:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 525189120. Throughput: 0: 10332.2, 1: 10081.7. Samples: 525199996. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:32,363][19170] Avg episode reward: [(0, '853.920'), (1, '450.150')] [2023-12-27 12:45:32,484][20381] Updated weights for policy 1, policy_version 1025247 (0.0008) [2023-12-27 12:45:32,536][20381] Updated weights for policy 1, policy_version 1025257 (0.0008) [2023-12-27 12:45:32,592][20381] Updated weights for policy 1, policy_version 1025267 (0.0010) [2023-12-27 12:45:32,818][20347] Updated weights for policy 0, policy_version 1026001 (0.0012) [2023-12-27 12:45:32,871][20347] Updated weights for policy 0, policy_version 1026011 (0.0011) [2023-12-27 12:45:32,925][20347] Updated weights for policy 0, policy_version 1026021 (0.0010) [2023-12-27 12:45:33,262][20381] Updated weights for policy 1, policy_version 1025277 (0.0010) [2023-12-27 12:45:33,310][20381] Updated weights for policy 1, policy_version 1025287 (0.0010) [2023-12-27 12:45:33,361][20381] Updated weights for policy 1, policy_version 1025297 (0.0010) [2023-12-27 12:45:33,578][20347] Updated weights for policy 0, policy_version 1026031 (0.0010) [2023-12-27 12:45:33,623][20347] Updated weights for policy 0, policy_version 1026041 (0.0010) [2023-12-27 12:45:33,675][20347] Updated weights for policy 0, policy_version 1026051 (0.0010) [2023-12-27 12:45:33,933][20381] Updated weights for policy 1, policy_version 1025307 (0.0009) [2023-12-27 12:45:33,991][20381] Updated weights for policy 1, policy_version 1025317 (0.0005) [2023-12-27 12:45:34,056][20381] Updated weights for policy 1, policy_version 1025327 (0.0005) [2023-12-27 12:45:34,299][20347] Updated weights for policy 0, policy_version 1026061 (0.0007) [2023-12-27 12:45:34,358][20347] Updated weights for policy 0, policy_version 1026071 (0.0008) [2023-12-27 12:45:34,414][20347] Updated weights for policy 0, policy_version 1026081 (0.0008) [2023-12-27 12:45:34,720][20381] Updated weights for policy 1, policy_version 1025337 (0.0009) [2023-12-27 12:45:34,771][20381] Updated weights for policy 1, policy_version 1025347 (0.0008) [2023-12-27 12:45:34,829][20381] Updated weights for policy 1, policy_version 1025357 (0.0007) [2023-12-27 12:45:34,883][20381] Updated weights for policy 1, policy_version 1025367 (0.0008) [2023-12-27 12:45:35,159][20347] Updated weights for policy 0, policy_version 1026091 (0.0009) [2023-12-27 12:45:35,209][20347] Updated weights for policy 0, policy_version 1026101 (0.0010) [2023-12-27 12:45:35,264][20347] Updated weights for policy 0, policy_version 1026111 (0.0010) [2023-12-27 12:45:35,469][20381] Updated weights for policy 1, policy_version 1025377 (0.0010) [2023-12-27 12:45:35,527][20381] Updated weights for policy 1, policy_version 1025387 (0.0010) [2023-12-27 12:45:35,584][20381] Updated weights for policy 1, policy_version 1025397 (0.0010) [2023-12-27 12:45:35,951][20347] Updated weights for policy 0, policy_version 1026121 (0.0010) [2023-12-27 12:45:36,008][20347] Updated weights for policy 0, policy_version 1026131 (0.0010) [2023-12-27 12:45:36,066][20347] Updated weights for policy 0, policy_version 1026141 (0.0010) [2023-12-27 12:45:36,128][20347] Updated weights for policy 0, policy_version 1026151 (0.0010) [2023-12-27 12:45:36,239][20381] Updated weights for policy 1, policy_version 1025407 (0.0011) [2023-12-27 12:45:36,299][20381] Updated weights for policy 1, policy_version 1025417 (0.0007) [2023-12-27 12:45:36,362][20381] Updated weights for policy 1, policy_version 1025427 (0.0006) [2023-12-27 12:45:36,829][20347] Updated weights for policy 0, policy_version 1026161 (0.0007) [2023-12-27 12:45:36,879][20347] Updated weights for policy 0, policy_version 1026171 (0.0007) [2023-12-27 12:45:36,933][20347] Updated weights for policy 0, policy_version 1026181 (0.0007) [2023-12-27 12:45:37,003][20381] Updated weights for policy 1, policy_version 1025437 (0.0008) [2023-12-27 12:45:37,053][20381] Updated weights for policy 1, policy_version 1025447 (0.0011) [2023-12-27 12:45:37,102][20381] Updated weights for policy 1, policy_version 1025457 (0.0011) [2023-12-27 12:45:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 525295616. Throughput: 0: 10290.6, 1: 10135.7. Samples: 525263428. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:37,363][19170] Avg episode reward: [(0, '786.310'), (1, '528.510')] [2023-12-27 12:45:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001026184_262750208.pth... [2023-12-27 12:45:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001025464_262545408.pth... [2023-12-27 12:45:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001025000_262447104.pth [2023-12-27 12:45:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001024280_262242304.pth [2023-12-27 12:45:37,582][20347] Updated weights for policy 0, policy_version 1026191 (0.0008) [2023-12-27 12:45:37,641][20347] Updated weights for policy 0, policy_version 1026201 (0.0010) [2023-12-27 12:45:37,696][20347] Updated weights for policy 0, policy_version 1026211 (0.0010) [2023-12-27 12:45:37,765][20381] Updated weights for policy 1, policy_version 1025467 (0.0011) [2023-12-27 12:45:37,821][20381] Updated weights for policy 1, policy_version 1025477 (0.0011) [2023-12-27 12:45:37,883][20381] Updated weights for policy 1, policy_version 1025487 (0.0011) [2023-12-27 12:45:38,314][20347] Updated weights for policy 0, policy_version 1026221 (0.0010) [2023-12-27 12:45:38,380][20347] Updated weights for policy 0, policy_version 1026231 (0.0010) [2023-12-27 12:45:38,438][20347] Updated weights for policy 0, policy_version 1026241 (0.0007) [2023-12-27 12:45:38,567][20381] Updated weights for policy 1, policy_version 1025497 (0.0011) [2023-12-27 12:45:38,627][20381] Updated weights for policy 1, policy_version 1025507 (0.0011) [2023-12-27 12:45:38,680][20381] Updated weights for policy 1, policy_version 1025517 (0.0011) [2023-12-27 12:45:38,739][20381] Updated weights for policy 1, policy_version 1025527 (0.0011) [2023-12-27 12:45:39,074][20347] Updated weights for policy 0, policy_version 1026251 (0.0008) [2023-12-27 12:45:39,124][20347] Updated weights for policy 0, policy_version 1026261 (0.0009) [2023-12-27 12:45:39,185][20347] Updated weights for policy 0, policy_version 1026271 (0.0006) [2023-12-27 12:45:39,476][20381] Updated weights for policy 1, policy_version 1025537 (0.0008) [2023-12-27 12:45:39,541][20381] Updated weights for policy 1, policy_version 1025547 (0.0006) [2023-12-27 12:45:39,604][20381] Updated weights for policy 1, policy_version 1025557 (0.0007) [2023-12-27 12:45:39,941][20347] Updated weights for policy 0, policy_version 1026281 (0.0010) [2023-12-27 12:45:40,001][20347] Updated weights for policy 0, policy_version 1026291 (0.0009) [2023-12-27 12:45:40,059][20347] Updated weights for policy 0, policy_version 1026301 (0.0009) [2023-12-27 12:45:40,113][20347] Updated weights for policy 0, policy_version 1026311 (0.0006) [2023-12-27 12:45:40,392][20381] Updated weights for policy 1, policy_version 1025567 (0.0008) [2023-12-27 12:45:40,451][20381] Updated weights for policy 1, policy_version 1025577 (0.0008) [2023-12-27 12:45:40,511][20381] Updated weights for policy 1, policy_version 1025587 (0.0008) [2023-12-27 12:45:40,736][20347] Updated weights for policy 0, policy_version 1026321 (0.0006) [2023-12-27 12:45:40,793][20347] Updated weights for policy 0, policy_version 1026331 (0.0005) [2023-12-27 12:45:40,849][20347] Updated weights for policy 0, policy_version 1026341 (0.0005) [2023-12-27 12:45:41,230][20381] Updated weights for policy 1, policy_version 1025597 (0.0009) [2023-12-27 12:45:41,289][20381] Updated weights for policy 1, policy_version 1025607 (0.0009) [2023-12-27 12:45:41,353][20381] Updated weights for policy 1, policy_version 1025617 (0.0011) [2023-12-27 12:45:41,533][20347] Updated weights for policy 0, policy_version 1026351 (0.0007) [2023-12-27 12:45:41,587][20347] Updated weights for policy 0, policy_version 1026361 (0.0010) [2023-12-27 12:45:41,598][20291] KL-divergence is very high: 124.4882 [2023-12-27 12:45:41,616][20291] KL-divergence is very high: 105.4658 [2023-12-27 12:45:41,650][20347] Updated weights for policy 0, policy_version 1026371 (0.0008) [2023-12-27 12:45:42,141][20381] Updated weights for policy 1, policy_version 1025627 (0.0009) [2023-12-27 12:45:42,193][20381] Updated weights for policy 1, policy_version 1025637 (0.0007) [2023-12-27 12:45:42,252][20381] Updated weights for policy 1, policy_version 1025647 (0.0007) [2023-12-27 12:45:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 525393920. Throughput: 0: 10240.5, 1: 10109.9. Samples: 525385860. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:42,363][19170] Avg episode reward: [(0, '775.830'), (1, '489.310')] [2023-12-27 12:45:42,373][20347] Updated weights for policy 0, policy_version 1026381 (0.0008) [2023-12-27 12:45:42,378][20291] KL-divergence is very high: 129.7572 [2023-12-27 12:45:42,424][20291] KL-divergence is very high: 127.1579 [2023-12-27 12:45:42,429][20347] Updated weights for policy 0, policy_version 1026391 (0.0008) [2023-12-27 12:45:42,480][20347] Updated weights for policy 0, policy_version 1026401 (0.0007) [2023-12-27 12:45:42,510][20291] KL-divergence is very high: 120.9652 [2023-12-27 12:45:42,985][20381] Updated weights for policy 1, policy_version 1025657 (0.0008) [2023-12-27 12:45:43,046][20381] Updated weights for policy 1, policy_version 1025667 (0.0010) [2023-12-27 12:45:43,094][20381] Updated weights for policy 1, policy_version 1025677 (0.0010) [2023-12-27 12:45:43,139][20347] Updated weights for policy 0, policy_version 1026411 (0.0006) [2023-12-27 12:45:43,145][20381] Updated weights for policy 1, policy_version 1025687 (0.0010) [2023-12-27 12:45:43,192][20347] Updated weights for policy 0, policy_version 1026421 (0.0006) [2023-12-27 12:45:43,244][20347] Updated weights for policy 0, policy_version 1026431 (0.0007) [2023-12-27 12:45:43,840][20381] Updated weights for policy 1, policy_version 1025697 (0.0010) [2023-12-27 12:45:43,888][20381] Updated weights for policy 1, policy_version 1025707 (0.0010) [2023-12-27 12:45:43,916][20347] Updated weights for policy 0, policy_version 1026441 (0.0007) [2023-12-27 12:45:43,936][20381] Updated weights for policy 1, policy_version 1025717 (0.0010) [2023-12-27 12:45:43,971][20347] Updated weights for policy 0, policy_version 1026451 (0.0006) [2023-12-27 12:45:44,024][20347] Updated weights for policy 0, policy_version 1026461 (0.0007) [2023-12-27 12:45:44,079][20347] Updated weights for policy 0, policy_version 1026471 (0.0007) [2023-12-27 12:45:44,640][20381] Updated weights for policy 1, policy_version 1025727 (0.0008) [2023-12-27 12:45:44,695][20381] Updated weights for policy 1, policy_version 1025737 (0.0006) [2023-12-27 12:45:44,751][20381] Updated weights for policy 1, policy_version 1025747 (0.0006) [2023-12-27 12:45:44,783][20347] Updated weights for policy 0, policy_version 1026481 (0.0008) [2023-12-27 12:45:44,848][20347] Updated weights for policy 0, policy_version 1026491 (0.0011) [2023-12-27 12:45:44,902][20347] Updated weights for policy 0, policy_version 1026501 (0.0010) [2023-12-27 12:45:45,441][20381] Updated weights for policy 1, policy_version 1025757 (0.0008) [2023-12-27 12:45:45,467][20347] Updated weights for policy 0, policy_version 1026511 (0.0007) [2023-12-27 12:45:45,493][20381] Updated weights for policy 1, policy_version 1025767 (0.0006) [2023-12-27 12:45:45,525][20347] Updated weights for policy 0, policy_version 1026521 (0.0005) [2023-12-27 12:45:45,542][20381] Updated weights for policy 1, policy_version 1025777 (0.0005) [2023-12-27 12:45:45,577][20347] Updated weights for policy 0, policy_version 1026531 (0.0005) [2023-12-27 12:45:46,117][20347] Updated weights for policy 0, policy_version 1026541 (0.0005) [2023-12-27 12:45:46,162][20347] Updated weights for policy 0, policy_version 1026551 (0.0005) [2023-12-27 12:45:46,211][20347] Updated weights for policy 0, policy_version 1026561 (0.0005) [2023-12-27 12:45:46,215][20381] Updated weights for policy 1, policy_version 1025787 (0.0007) [2023-12-27 12:45:46,281][20381] Updated weights for policy 1, policy_version 1025797 (0.0007) [2023-12-27 12:45:46,340][20381] Updated weights for policy 1, policy_version 1025807 (0.0006) [2023-12-27 12:45:46,786][20347] Updated weights for policy 0, policy_version 1026571 (0.0005) [2023-12-27 12:45:46,839][20347] Updated weights for policy 0, policy_version 1026581 (0.0005) [2023-12-27 12:45:46,888][20347] Updated weights for policy 0, policy_version 1026591 (0.0005) [2023-12-27 12:45:47,030][20381] Updated weights for policy 1, policy_version 1025817 (0.0007) [2023-12-27 12:45:47,085][20381] Updated weights for policy 1, policy_version 1025827 (0.0007) [2023-12-27 12:45:47,145][20381] Updated weights for policy 1, policy_version 1025837 (0.0008) [2023-12-27 12:45:47,200][20381] Updated weights for policy 1, policy_version 1025847 (0.0008) [2023-12-27 12:45:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 525500416. Throughput: 0: 10367.7, 1: 10061.7. Samples: 525509412. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:47,363][19170] Avg episode reward: [(0, '834.070'), (1, '499.170')] [2023-12-27 12:45:47,507][20347] Updated weights for policy 0, policy_version 1026601 (0.0006) [2023-12-27 12:45:47,562][20347] Updated weights for policy 0, policy_version 1026611 (0.0005) [2023-12-27 12:45:47,615][20347] Updated weights for policy 0, policy_version 1026621 (0.0005) [2023-12-27 12:45:47,665][20347] Updated weights for policy 0, policy_version 1026631 (0.0005) [2023-12-27 12:45:47,913][20381] Updated weights for policy 1, policy_version 1025857 (0.0008) [2023-12-27 12:45:47,959][20381] Updated weights for policy 1, policy_version 1025867 (0.0008) [2023-12-27 12:45:48,003][20381] Updated weights for policy 1, policy_version 1025877 (0.0009) [2023-12-27 12:45:48,240][20347] Updated weights for policy 0, policy_version 1026641 (0.0006) [2023-12-27 12:45:48,292][20347] Updated weights for policy 0, policy_version 1026651 (0.0005) [2023-12-27 12:45:48,350][20347] Updated weights for policy 0, policy_version 1026661 (0.0007) [2023-12-27 12:45:48,735][20381] Updated weights for policy 1, policy_version 1025887 (0.0007) [2023-12-27 12:45:48,795][20381] Updated weights for policy 1, policy_version 1025897 (0.0007) [2023-12-27 12:45:48,851][20381] Updated weights for policy 1, policy_version 1025907 (0.0008) [2023-12-27 12:45:49,004][20347] Updated weights for policy 0, policy_version 1026671 (0.0008) [2023-12-27 12:45:49,063][20347] Updated weights for policy 0, policy_version 1026681 (0.0008) [2023-12-27 12:45:49,118][20347] Updated weights for policy 0, policy_version 1026691 (0.0008) [2023-12-27 12:45:49,631][20381] Updated weights for policy 1, policy_version 1025917 (0.0008) [2023-12-27 12:45:49,692][20381] Updated weights for policy 1, policy_version 1025927 (0.0008) [2023-12-27 12:45:49,748][20381] Updated weights for policy 1, policy_version 1025937 (0.0007) [2023-12-27 12:45:49,855][20347] Updated weights for policy 0, policy_version 1026701 (0.0007) [2023-12-27 12:45:49,905][20347] Updated weights for policy 0, policy_version 1026711 (0.0007) [2023-12-27 12:45:49,967][20347] Updated weights for policy 0, policy_version 1026721 (0.0009) [2023-12-27 12:45:50,458][20381] Updated weights for policy 1, policy_version 1025947 (0.0008) [2023-12-27 12:45:50,511][20381] Updated weights for policy 1, policy_version 1025957 (0.0007) [2023-12-27 12:45:50,561][20381] Updated weights for policy 1, policy_version 1025967 (0.0007) [2023-12-27 12:45:50,671][20347] Updated weights for policy 0, policy_version 1026731 (0.0009) [2023-12-27 12:45:50,725][20347] Updated weights for policy 0, policy_version 1026741 (0.0011) [2023-12-27 12:45:50,782][20347] Updated weights for policy 0, policy_version 1026751 (0.0009) [2023-12-27 12:45:51,270][20381] Updated weights for policy 1, policy_version 1025977 (0.0007) [2023-12-27 12:45:51,329][20381] Updated weights for policy 1, policy_version 1025987 (0.0010) [2023-12-27 12:45:51,394][20381] Updated weights for policy 1, policy_version 1025997 (0.0008) [2023-12-27 12:45:51,453][20381] Updated weights for policy 1, policy_version 1026007 (0.0009) [2023-12-27 12:45:51,524][20347] Updated weights for policy 0, policy_version 1026761 (0.0010) [2023-12-27 12:45:51,584][20347] Updated weights for policy 0, policy_version 1026771 (0.0007) [2023-12-27 12:45:51,651][20347] Updated weights for policy 0, policy_version 1026781 (0.0009) [2023-12-27 12:45:51,707][20347] Updated weights for policy 0, policy_version 1026791 (0.0010) [2023-12-27 12:45:52,140][20381] Updated weights for policy 1, policy_version 1026017 (0.0010) [2023-12-27 12:45:52,185][20381] Updated weights for policy 1, policy_version 1026027 (0.0010) [2023-12-27 12:45:52,235][20381] Updated weights for policy 1, policy_version 1026037 (0.0008) [2023-12-27 12:45:52,348][20347] Updated weights for policy 0, policy_version 1026801 (0.0008) [2023-12-27 12:45:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 525598720. Throughput: 0: 10367.5, 1: 10020.5. Samples: 525570588. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:52,363][19170] Avg episode reward: [(0, '853.490'), (1, '538.610')] [2023-12-27 12:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001026040_262692864.pth... [2023-12-27 12:45:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001024856_262389760.pth [2023-12-27 12:45:52,410][20347] Updated weights for policy 0, policy_version 1026811 (0.0008) [2023-12-27 12:45:52,467][20347] Updated weights for policy 0, policy_version 1026821 (0.0005) [2023-12-27 12:45:52,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001026824_262914048.pth... [2023-12-27 12:45:52,486][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001025608_262602752.pth [2023-12-27 12:45:52,967][20381] Updated weights for policy 1, policy_version 1026047 (0.0011) [2023-12-27 12:45:53,012][20381] Updated weights for policy 1, policy_version 1026057 (0.0009) [2023-12-27 12:45:53,057][20347] Updated weights for policy 0, policy_version 1026831 (0.0008) [2023-12-27 12:45:53,063][20381] Updated weights for policy 1, policy_version 1026067 (0.0007) [2023-12-27 12:45:53,105][20347] Updated weights for policy 0, policy_version 1026841 (0.0010) [2023-12-27 12:45:53,159][20347] Updated weights for policy 0, policy_version 1026851 (0.0010) [2023-12-27 12:45:53,772][20381] Updated weights for policy 1, policy_version 1026077 (0.0009) [2023-12-27 12:45:53,821][20347] Updated weights for policy 0, policy_version 1026861 (0.0008) [2023-12-27 12:45:53,824][20381] Updated weights for policy 1, policy_version 1026087 (0.0010) [2023-12-27 12:45:53,879][20347] Updated weights for policy 0, policy_version 1026871 (0.0006) [2023-12-27 12:45:53,881][20381] Updated weights for policy 1, policy_version 1026097 (0.0010) [2023-12-27 12:45:53,933][20347] Updated weights for policy 0, policy_version 1026881 (0.0007) [2023-12-27 12:45:54,511][20381] Updated weights for policy 1, policy_version 1026107 (0.0008) [2023-12-27 12:45:54,532][20347] Updated weights for policy 0, policy_version 1026891 (0.0009) [2023-12-27 12:45:54,573][20381] Updated weights for policy 1, policy_version 1026117 (0.0011) [2023-12-27 12:45:54,580][20347] Updated weights for policy 0, policy_version 1026901 (0.0005) [2023-12-27 12:45:54,631][20381] Updated weights for policy 1, policy_version 1026127 (0.0010) [2023-12-27 12:45:54,643][20347] Updated weights for policy 0, policy_version 1026911 (0.0010) [2023-12-27 12:45:55,240][20347] Updated weights for policy 0, policy_version 1026921 (0.0010) [2023-12-27 12:45:55,296][20347] Updated weights for policy 0, policy_version 1026931 (0.0007) [2023-12-27 12:45:55,297][20381] Updated weights for policy 1, policy_version 1026137 (0.0010) [2023-12-27 12:45:55,354][20347] Updated weights for policy 0, policy_version 1026941 (0.0005) [2023-12-27 12:45:55,355][20381] Updated weights for policy 1, policy_version 1026147 (0.0010) [2023-12-27 12:45:55,410][20347] Updated weights for policy 0, policy_version 1026951 (0.0005) [2023-12-27 12:45:55,413][20381] Updated weights for policy 1, policy_version 1026157 (0.0011) [2023-12-27 12:45:55,467][20381] Updated weights for policy 1, policy_version 1026167 (0.0010) [2023-12-27 12:45:56,030][20347] Updated weights for policy 0, policy_version 1026961 (0.0006) [2023-12-27 12:45:56,090][20347] Updated weights for policy 0, policy_version 1026971 (0.0009) [2023-12-27 12:45:56,104][20381] Updated weights for policy 1, policy_version 1026177 (0.0010) [2023-12-27 12:45:56,145][20347] Updated weights for policy 0, policy_version 1026981 (0.0010) [2023-12-27 12:45:56,156][20381] Updated weights for policy 1, policy_version 1026187 (0.0010) [2023-12-27 12:45:56,220][20381] Updated weights for policy 1, policy_version 1026197 (0.0010) [2023-12-27 12:45:56,749][20347] Updated weights for policy 0, policy_version 1026991 (0.0007) [2023-12-27 12:45:56,803][20347] Updated weights for policy 0, policy_version 1027001 (0.0009) [2023-12-27 12:45:56,855][20347] Updated weights for policy 0, policy_version 1027011 (0.0006) [2023-12-27 12:45:56,901][20381] Updated weights for policy 1, policy_version 1026207 (0.0010) [2023-12-27 12:45:56,961][20381] Updated weights for policy 1, policy_version 1026217 (0.0010) [2023-12-27 12:45:57,008][20381] Updated weights for policy 1, policy_version 1026227 (0.0010) [2023-12-27 12:45:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 525705216. Throughput: 0: 10386.5, 1: 10077.7. Samples: 525696008. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:45:57,363][19170] Avg episode reward: [(0, '902.380'), (1, '508.840')] [2023-12-27 12:45:57,424][20347] Updated weights for policy 0, policy_version 1027021 (0.0008) [2023-12-27 12:45:57,481][20347] Updated weights for policy 0, policy_version 1027031 (0.0010) [2023-12-27 12:45:57,539][20347] Updated weights for policy 0, policy_version 1027041 (0.0010) [2023-12-27 12:45:57,687][20381] Updated weights for policy 1, policy_version 1026237 (0.0010) [2023-12-27 12:45:57,740][20381] Updated weights for policy 1, policy_version 1026247 (0.0011) [2023-12-27 12:45:57,789][20381] Updated weights for policy 1, policy_version 1026257 (0.0011) [2023-12-27 12:45:58,216][20347] Updated weights for policy 0, policy_version 1027051 (0.0010) [2023-12-27 12:45:58,272][20347] Updated weights for policy 0, policy_version 1027061 (0.0010) [2023-12-27 12:45:58,334][20347] Updated weights for policy 0, policy_version 1027071 (0.0010) [2023-12-27 12:45:58,508][20381] Updated weights for policy 1, policy_version 1026267 (0.0010) [2023-12-27 12:45:58,573][20381] Updated weights for policy 1, policy_version 1026277 (0.0008) [2023-12-27 12:45:58,631][20381] Updated weights for policy 1, policy_version 1026287 (0.0007) [2023-12-27 12:45:59,118][20347] Updated weights for policy 0, policy_version 1027081 (0.0010) [2023-12-27 12:45:59,175][20347] Updated weights for policy 0, policy_version 1027091 (0.0009) [2023-12-27 12:45:59,241][20347] Updated weights for policy 0, policy_version 1027101 (0.0009) [2023-12-27 12:45:59,294][20347] Updated weights for policy 0, policy_version 1027111 (0.0007) [2023-12-27 12:45:59,451][20381] Updated weights for policy 1, policy_version 1026297 (0.0008) [2023-12-27 12:45:59,511][20381] Updated weights for policy 1, policy_version 1026307 (0.0010) [2023-12-27 12:45:59,569][20381] Updated weights for policy 1, policy_version 1026317 (0.0010) [2023-12-27 12:45:59,628][20381] Updated weights for policy 1, policy_version 1026327 (0.0011) [2023-12-27 12:46:00,056][20347] Updated weights for policy 0, policy_version 1027121 (0.0008) [2023-12-27 12:46:00,108][20347] Updated weights for policy 0, policy_version 1027131 (0.0010) [2023-12-27 12:46:00,169][20347] Updated weights for policy 0, policy_version 1027141 (0.0009) [2023-12-27 12:46:00,300][20381] Updated weights for policy 1, policy_version 1026337 (0.0007) [2023-12-27 12:46:00,362][20381] Updated weights for policy 1, policy_version 1026347 (0.0011) [2023-12-27 12:46:00,417][20381] Updated weights for policy 1, policy_version 1026357 (0.0011) [2023-12-27 12:46:00,808][20347] Updated weights for policy 0, policy_version 1027151 (0.0009) [2023-12-27 12:46:00,861][20347] Updated weights for policy 0, policy_version 1027161 (0.0007) [2023-12-27 12:46:00,920][20347] Updated weights for policy 0, policy_version 1027171 (0.0005) [2023-12-27 12:46:01,085][20381] Updated weights for policy 1, policy_version 1026367 (0.0011) [2023-12-27 12:46:01,147][20381] Updated weights for policy 1, policy_version 1026377 (0.0009) [2023-12-27 12:46:01,211][20381] Updated weights for policy 1, policy_version 1026387 (0.0012) [2023-12-27 12:46:01,606][20347] Updated weights for policy 0, policy_version 1027181 (0.0006) [2023-12-27 12:46:01,668][20347] Updated weights for policy 0, policy_version 1027191 (0.0008) [2023-12-27 12:46:01,726][20347] Updated weights for policy 0, policy_version 1027201 (0.0008) [2023-12-27 12:46:01,940][20381] Updated weights for policy 1, policy_version 1026397 (0.0009) [2023-12-27 12:46:01,994][20381] Updated weights for policy 1, policy_version 1026407 (0.0007) [2023-12-27 12:46:02,048][20381] Updated weights for policy 1, policy_version 1026417 (0.0009) [2023-12-27 12:46:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 525803520. Throughput: 0: 10340.6, 1: 10013.2. Samples: 525814748. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:02,363][19170] Avg episode reward: [(0, '941.470'), (1, '538.490')] [2023-12-27 12:46:02,463][20347] Updated weights for policy 0, policy_version 1027211 (0.0008) [2023-12-27 12:46:02,529][20347] Updated weights for policy 0, policy_version 1027221 (0.0008) [2023-12-27 12:46:02,592][20347] Updated weights for policy 0, policy_version 1027231 (0.0007) [2023-12-27 12:46:02,779][20381] Updated weights for policy 1, policy_version 1026427 (0.0008) [2023-12-27 12:46:02,841][20381] Updated weights for policy 1, policy_version 1026437 (0.0009) [2023-12-27 12:46:02,892][20381] Updated weights for policy 1, policy_version 1026447 (0.0008) [2023-12-27 12:46:03,270][20347] Updated weights for policy 0, policy_version 1027241 (0.0008) [2023-12-27 12:46:03,328][20347] Updated weights for policy 0, policy_version 1027251 (0.0007) [2023-12-27 12:46:03,381][20347] Updated weights for policy 0, policy_version 1027261 (0.0008) [2023-12-27 12:46:03,428][20347] Updated weights for policy 0, policy_version 1027271 (0.0008) [2023-12-27 12:46:03,510][20381] Updated weights for policy 1, policy_version 1026457 (0.0008) [2023-12-27 12:46:03,558][20381] Updated weights for policy 1, policy_version 1026467 (0.0007) [2023-12-27 12:46:03,613][20381] Updated weights for policy 1, policy_version 1026477 (0.0005) [2023-12-27 12:46:03,666][20381] Updated weights for policy 1, policy_version 1026487 (0.0006) [2023-12-27 12:46:04,126][20347] Updated weights for policy 0, policy_version 1027281 (0.0006) [2023-12-27 12:46:04,193][20347] Updated weights for policy 0, policy_version 1027291 (0.0006) [2023-12-27 12:46:04,257][20347] Updated weights for policy 0, policy_version 1027301 (0.0008) [2023-12-27 12:46:04,357][20381] Updated weights for policy 1, policy_version 1026497 (0.0008) [2023-12-27 12:46:04,417][20381] Updated weights for policy 1, policy_version 1026507 (0.0008) [2023-12-27 12:46:04,477][20381] Updated weights for policy 1, policy_version 1026517 (0.0008) [2023-12-27 12:46:04,866][20347] Updated weights for policy 0, policy_version 1027311 (0.0007) [2023-12-27 12:46:04,913][20347] Updated weights for policy 0, policy_version 1027321 (0.0007) [2023-12-27 12:46:04,962][20347] Updated weights for policy 0, policy_version 1027331 (0.0008) [2023-12-27 12:46:05,182][20381] Updated weights for policy 1, policy_version 1026527 (0.0007) [2023-12-27 12:46:05,232][20381] Updated weights for policy 1, policy_version 1026537 (0.0008) [2023-12-27 12:46:05,296][20381] Updated weights for policy 1, policy_version 1026547 (0.0005) [2023-12-27 12:46:05,670][20347] Updated weights for policy 0, policy_version 1027341 (0.0006) [2023-12-27 12:46:05,728][20347] Updated weights for policy 0, policy_version 1027351 (0.0008) [2023-12-27 12:46:05,781][20347] Updated weights for policy 0, policy_version 1027361 (0.0008) [2023-12-27 12:46:05,946][20381] Updated weights for policy 1, policy_version 1026557 (0.0006) [2023-12-27 12:46:05,992][20381] Updated weights for policy 1, policy_version 1026567 (0.0007) [2023-12-27 12:46:06,043][20381] Updated weights for policy 1, policy_version 1026577 (0.0008) [2023-12-27 12:46:06,456][20347] Updated weights for policy 0, policy_version 1027371 (0.0008) [2023-12-27 12:46:06,518][20347] Updated weights for policy 0, policy_version 1027381 (0.0008) [2023-12-27 12:46:06,572][20347] Updated weights for policy 0, policy_version 1027391 (0.0008) [2023-12-27 12:46:06,786][20381] Updated weights for policy 1, policy_version 1026587 (0.0007) [2023-12-27 12:46:06,835][20381] Updated weights for policy 1, policy_version 1026597 (0.0008) [2023-12-27 12:46:06,882][20381] Updated weights for policy 1, policy_version 1026607 (0.0008) [2023-12-27 12:46:07,283][20347] Updated weights for policy 0, policy_version 1027401 (0.0008) [2023-12-27 12:46:07,335][20347] Updated weights for policy 0, policy_version 1027411 (0.0008) [2023-12-27 12:46:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 525901824. Throughput: 0: 10333.4, 1: 10023.7. Samples: 525876096. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:07,363][19170] Avg episode reward: [(0, '951.110'), (1, '607.230')] [2023-12-27 12:46:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001026616_262840320.pth... [2023-12-27 12:46:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001025464_262545408.pth [2023-12-27 12:46:07,383][20347] Updated weights for policy 0, policy_version 1027421 (0.0007) [2023-12-27 12:46:07,441][20347] Updated weights for policy 0, policy_version 1027431 (0.0007) [2023-12-27 12:46:07,445][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001027432_263069696.pth... [2023-12-27 12:46:07,449][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001026184_262750208.pth [2023-12-27 12:46:07,525][20381] Updated weights for policy 1, policy_version 1026617 (0.0008) [2023-12-27 12:46:07,584][20381] Updated weights for policy 1, policy_version 1026627 (0.0007) [2023-12-27 12:46:07,632][20381] Updated weights for policy 1, policy_version 1026637 (0.0009) [2023-12-27 12:46:07,689][20381] Updated weights for policy 1, policy_version 1026647 (0.0008) [2023-12-27 12:46:08,142][20347] Updated weights for policy 0, policy_version 1027441 (0.0008) [2023-12-27 12:46:08,199][20347] Updated weights for policy 0, policy_version 1027451 (0.0008) [2023-12-27 12:46:08,261][20347] Updated weights for policy 0, policy_version 1027461 (0.0008) [2023-12-27 12:46:08,356][20381] Updated weights for policy 1, policy_version 1026657 (0.0007) [2023-12-27 12:46:08,418][20381] Updated weights for policy 1, policy_version 1026667 (0.0006) [2023-12-27 12:46:08,465][20381] Updated weights for policy 1, policy_version 1026677 (0.0007) [2023-12-27 12:46:08,950][20347] Updated weights for policy 0, policy_version 1027471 (0.0007) [2023-12-27 12:46:08,999][20347] Updated weights for policy 0, policy_version 1027481 (0.0009) [2023-12-27 12:46:09,056][20347] Updated weights for policy 0, policy_version 1027491 (0.0009) [2023-12-27 12:46:09,163][20381] Updated weights for policy 1, policy_version 1026687 (0.0006) [2023-12-27 12:46:09,225][20381] Updated weights for policy 1, policy_version 1026697 (0.0010) [2023-12-27 12:46:09,291][20381] Updated weights for policy 1, policy_version 1026707 (0.0007) [2023-12-27 12:46:09,806][20347] Updated weights for policy 0, policy_version 1027501 (0.0009) [2023-12-27 12:46:09,868][20347] Updated weights for policy 0, policy_version 1027511 (0.0008) [2023-12-27 12:46:09,928][20347] Updated weights for policy 0, policy_version 1027521 (0.0008) [2023-12-27 12:46:09,992][20381] Updated weights for policy 1, policy_version 1026717 (0.0008) [2023-12-27 12:46:10,055][20381] Updated weights for policy 1, policy_version 1026727 (0.0009) [2023-12-27 12:46:10,116][20381] Updated weights for policy 1, policy_version 1026737 (0.0008) [2023-12-27 12:46:10,554][20347] Updated weights for policy 0, policy_version 1027531 (0.0007) [2023-12-27 12:46:10,613][20347] Updated weights for policy 0, policy_version 1027541 (0.0008) [2023-12-27 12:46:10,667][20347] Updated weights for policy 0, policy_version 1027551 (0.0007) [2023-12-27 12:46:10,847][20381] Updated weights for policy 1, policy_version 1026747 (0.0009) [2023-12-27 12:46:10,908][20381] Updated weights for policy 1, policy_version 1026757 (0.0009) [2023-12-27 12:46:10,964][20381] Updated weights for policy 1, policy_version 1026767 (0.0008) [2023-12-27 12:46:11,401][20347] Updated weights for policy 0, policy_version 1027561 (0.0008) [2023-12-27 12:46:11,462][20347] Updated weights for policy 0, policy_version 1027571 (0.0006) [2023-12-27 12:46:11,518][20347] Updated weights for policy 0, policy_version 1027581 (0.0007) [2023-12-27 12:46:11,580][20347] Updated weights for policy 0, policy_version 1027591 (0.0008) [2023-12-27 12:46:11,753][20381] Updated weights for policy 1, policy_version 1026777 (0.0008) [2023-12-27 12:46:11,818][20381] Updated weights for policy 1, policy_version 1026787 (0.0008) [2023-12-27 12:46:11,884][20381] Updated weights for policy 1, policy_version 1026797 (0.0008) [2023-12-27 12:46:11,948][20381] Updated weights for policy 1, policy_version 1026807 (0.0007) [2023-12-27 12:46:12,226][20347] Updated weights for policy 0, policy_version 1027601 (0.0006) [2023-12-27 12:46:12,293][20347] Updated weights for policy 0, policy_version 1027611 (0.0008) [2023-12-27 12:46:12,350][20347] Updated weights for policy 0, policy_version 1027621 (0.0007) [2023-12-27 12:46:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 526000128. Throughput: 0: 10360.2, 1: 10024.4. Samples: 525996744. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:12,363][19170] Avg episode reward: [(0, '970.720'), (1, '645.410')] [2023-12-27 12:46:12,633][20381] Updated weights for policy 1, policy_version 1026817 (0.0008) [2023-12-27 12:46:12,691][20381] Updated weights for policy 1, policy_version 1026827 (0.0008) [2023-12-27 12:46:12,750][20381] Updated weights for policy 1, policy_version 1026837 (0.0008) [2023-12-27 12:46:13,046][20347] Updated weights for policy 0, policy_version 1027631 (0.0008) [2023-12-27 12:46:13,090][20347] Updated weights for policy 0, policy_version 1027641 (0.0007) [2023-12-27 12:46:13,135][20347] Updated weights for policy 0, policy_version 1027651 (0.0007) [2023-12-27 12:46:13,433][20381] Updated weights for policy 1, policy_version 1026847 (0.0010) [2023-12-27 12:46:13,482][20381] Updated weights for policy 1, policy_version 1026857 (0.0010) [2023-12-27 12:46:13,537][20381] Updated weights for policy 1, policy_version 1026867 (0.0011) [2023-12-27 12:46:13,807][20347] Updated weights for policy 0, policy_version 1027661 (0.0007) [2023-12-27 12:46:13,864][20347] Updated weights for policy 0, policy_version 1027671 (0.0007) [2023-12-27 12:46:13,925][20347] Updated weights for policy 0, policy_version 1027681 (0.0007) [2023-12-27 12:46:14,208][20381] Updated weights for policy 1, policy_version 1026877 (0.0008) [2023-12-27 12:46:14,266][20381] Updated weights for policy 1, policy_version 1026887 (0.0005) [2023-12-27 12:46:14,327][20381] Updated weights for policy 1, policy_version 1026897 (0.0005) [2023-12-27 12:46:14,595][20347] Updated weights for policy 0, policy_version 1027691 (0.0007) [2023-12-27 12:46:14,649][20347] Updated weights for policy 0, policy_version 1027701 (0.0006) [2023-12-27 12:46:14,704][20347] Updated weights for policy 0, policy_version 1027711 (0.0007) [2023-12-27 12:46:14,976][20381] Updated weights for policy 1, policy_version 1026907 (0.0010) [2023-12-27 12:46:15,036][20381] Updated weights for policy 1, policy_version 1026917 (0.0010) [2023-12-27 12:46:15,092][20381] Updated weights for policy 1, policy_version 1026927 (0.0007) [2023-12-27 12:46:15,411][20347] Updated weights for policy 0, policy_version 1027721 (0.0007) [2023-12-27 12:46:15,458][20347] Updated weights for policy 0, policy_version 1027731 (0.0008) [2023-12-27 12:46:15,504][20347] Updated weights for policy 0, policy_version 1027741 (0.0008) [2023-12-27 12:46:15,557][20347] Updated weights for policy 0, policy_version 1027751 (0.0008) [2023-12-27 12:46:15,760][20381] Updated weights for policy 1, policy_version 1026937 (0.0007) [2023-12-27 12:46:15,812][20381] Updated weights for policy 1, policy_version 1026947 (0.0008) [2023-12-27 12:46:15,861][20381] Updated weights for policy 1, policy_version 1026957 (0.0008) [2023-12-27 12:46:15,906][20381] Updated weights for policy 1, policy_version 1026967 (0.0007) [2023-12-27 12:46:16,291][20347] Updated weights for policy 0, policy_version 1027761 (0.0007) [2023-12-27 12:46:16,340][20347] Updated weights for policy 0, policy_version 1027771 (0.0007) [2023-12-27 12:46:16,393][20347] Updated weights for policy 0, policy_version 1027781 (0.0007) [2023-12-27 12:46:16,580][20381] Updated weights for policy 1, policy_version 1026977 (0.0006) [2023-12-27 12:46:16,629][20381] Updated weights for policy 1, policy_version 1026987 (0.0005) [2023-12-27 12:46:16,677][20381] Updated weights for policy 1, policy_version 1026997 (0.0005) [2023-12-27 12:46:17,078][20347] Updated weights for policy 0, policy_version 1027791 (0.0006) [2023-12-27 12:46:17,143][20347] Updated weights for policy 0, policy_version 1027801 (0.0005) [2023-12-27 12:46:17,199][20347] Updated weights for policy 0, policy_version 1027811 (0.0010) [2023-12-27 12:46:17,244][20381] Updated weights for policy 1, policy_version 1027007 (0.0006) [2023-12-27 12:46:17,294][20381] Updated weights for policy 1, policy_version 1027017 (0.0007) [2023-12-27 12:46:17,340][20381] Updated weights for policy 1, policy_version 1027027 (0.0007) [2023-12-27 12:46:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 526114816. Throughput: 0: 10347.1, 1: 10068.7. Samples: 526118708. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:17,363][19170] Avg episode reward: [(0, '960.970'), (1, '505.530')] [2023-12-27 12:46:17,762][20347] Updated weights for policy 0, policy_version 1027821 (0.0008) [2023-12-27 12:46:17,822][20347] Updated weights for policy 0, policy_version 1027831 (0.0005) [2023-12-27 12:46:17,879][20347] Updated weights for policy 0, policy_version 1027841 (0.0006) [2023-12-27 12:46:18,094][20381] Updated weights for policy 1, policy_version 1027037 (0.0007) [2023-12-27 12:46:18,154][20381] Updated weights for policy 1, policy_version 1027047 (0.0008) [2023-12-27 12:46:18,202][20381] Updated weights for policy 1, policy_version 1027057 (0.0006) [2023-12-27 12:46:18,475][20347] Updated weights for policy 0, policy_version 1027851 (0.0010) [2023-12-27 12:46:18,539][20347] Updated weights for policy 0, policy_version 1027861 (0.0008) [2023-12-27 12:46:18,602][20347] Updated weights for policy 0, policy_version 1027871 (0.0007) [2023-12-27 12:46:18,894][20381] Updated weights for policy 1, policy_version 1027067 (0.0008) [2023-12-27 12:46:18,948][20381] Updated weights for policy 1, policy_version 1027077 (0.0006) [2023-12-27 12:46:19,002][20381] Updated weights for policy 1, policy_version 1027087 (0.0007) [2023-12-27 12:46:19,291][20347] Updated weights for policy 0, policy_version 1027881 (0.0010) [2023-12-27 12:46:19,355][20347] Updated weights for policy 0, policy_version 1027891 (0.0008) [2023-12-27 12:46:19,415][20347] Updated weights for policy 0, policy_version 1027901 (0.0008) [2023-12-27 12:46:19,474][20347] Updated weights for policy 0, policy_version 1027911 (0.0007) [2023-12-27 12:46:19,668][20381] Updated weights for policy 1, policy_version 1027097 (0.0008) [2023-12-27 12:46:19,728][20381] Updated weights for policy 1, policy_version 1027107 (0.0007) [2023-12-27 12:46:19,794][20381] Updated weights for policy 1, policy_version 1027117 (0.0006) [2023-12-27 12:46:19,860][20381] Updated weights for policy 1, policy_version 1027127 (0.0007) [2023-12-27 12:46:20,240][20347] Updated weights for policy 0, policy_version 1027921 (0.0008) [2023-12-27 12:46:20,296][20347] Updated weights for policy 0, policy_version 1027931 (0.0007) [2023-12-27 12:46:20,359][20347] Updated weights for policy 0, policy_version 1027941 (0.0009) [2023-12-27 12:46:20,525][20381] Updated weights for policy 1, policy_version 1027137 (0.0008) [2023-12-27 12:46:20,592][20381] Updated weights for policy 1, policy_version 1027147 (0.0007) [2023-12-27 12:46:20,648][20381] Updated weights for policy 1, policy_version 1027157 (0.0006) [2023-12-27 12:46:21,086][20347] Updated weights for policy 0, policy_version 1027951 (0.0008) [2023-12-27 12:46:21,148][20347] Updated weights for policy 0, policy_version 1027961 (0.0008) [2023-12-27 12:46:21,209][20347] Updated weights for policy 0, policy_version 1027971 (0.0008) [2023-12-27 12:46:21,341][20381] Updated weights for policy 1, policy_version 1027167 (0.0008) [2023-12-27 12:46:21,412][20381] Updated weights for policy 1, policy_version 1027177 (0.0008) [2023-12-27 12:46:21,471][20381] Updated weights for policy 1, policy_version 1027187 (0.0008) [2023-12-27 12:46:22,020][20347] Updated weights for policy 0, policy_version 1027981 (0.0007) [2023-12-27 12:46:22,077][20347] Updated weights for policy 0, policy_version 1027991 (0.0008) [2023-12-27 12:46:22,143][20347] Updated weights for policy 0, policy_version 1028001 (0.0008) [2023-12-27 12:46:22,237][20381] Updated weights for policy 1, policy_version 1027197 (0.0009) [2023-12-27 12:46:22,303][20381] Updated weights for policy 1, policy_version 1027207 (0.0011) [2023-12-27 12:46:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 526204928. Throughput: 0: 10365.3, 1: 10001.9. Samples: 526179952. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:22,364][19170] Avg episode reward: [(0, '990.270'), (1, '525.700')] [2023-12-27 12:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001028008_263217152.pth... [2023-12-27 12:46:22,371][20381] Updated weights for policy 1, policy_version 1027217 (0.0011) [2023-12-27 12:46:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001026824_262914048.pth [2023-12-27 12:46:22,413][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001027224_262995968.pth... [2023-12-27 12:46:22,417][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001026040_262692864.pth [2023-12-27 12:46:22,867][20347] Updated weights for policy 0, policy_version 1028011 (0.0006) [2023-12-27 12:46:22,921][20347] Updated weights for policy 0, policy_version 1028021 (0.0007) [2023-12-27 12:46:22,971][20347] Updated weights for policy 0, policy_version 1028031 (0.0007) [2023-12-27 12:46:23,006][20381] Updated weights for policy 1, policy_version 1027227 (0.0010) [2023-12-27 12:46:23,063][20381] Updated weights for policy 1, policy_version 1027237 (0.0007) [2023-12-27 12:46:23,124][20381] Updated weights for policy 1, policy_version 1027247 (0.0006) [2023-12-27 12:46:23,662][20347] Updated weights for policy 0, policy_version 1028041 (0.0009) [2023-12-27 12:46:23,719][20347] Updated weights for policy 0, policy_version 1028051 (0.0009) [2023-12-27 12:46:23,761][20381] Updated weights for policy 1, policy_version 1027257 (0.0010) [2023-12-27 12:46:23,778][20347] Updated weights for policy 0, policy_version 1028061 (0.0007) [2023-12-27 12:46:23,807][20381] Updated weights for policy 1, policy_version 1027267 (0.0006) [2023-12-27 12:46:23,829][20347] Updated weights for policy 0, policy_version 1028071 (0.0007) [2023-12-27 12:46:23,854][20381] Updated weights for policy 1, policy_version 1027277 (0.0006) [2023-12-27 12:46:23,907][20381] Updated weights for policy 1, policy_version 1027287 (0.0005) [2023-12-27 12:46:24,545][20347] Updated weights for policy 0, policy_version 1028081 (0.0007) [2023-12-27 12:46:24,607][20347] Updated weights for policy 0, policy_version 1028091 (0.0005) [2023-12-27 12:46:24,623][20381] Updated weights for policy 1, policy_version 1027297 (0.0006) [2023-12-27 12:46:24,668][20347] Updated weights for policy 0, policy_version 1028101 (0.0005) [2023-12-27 12:46:24,680][20381] Updated weights for policy 1, policy_version 1027307 (0.0007) [2023-12-27 12:46:24,731][20381] Updated weights for policy 1, policy_version 1027317 (0.0007) [2023-12-27 12:46:25,226][20347] Updated weights for policy 0, policy_version 1028111 (0.0009) [2023-12-27 12:46:25,285][20347] Updated weights for policy 0, policy_version 1028121 (0.0007) [2023-12-27 12:46:25,346][20347] Updated weights for policy 0, policy_version 1028131 (0.0007) [2023-12-27 12:46:25,404][20381] Updated weights for policy 1, policy_version 1027327 (0.0008) [2023-12-27 12:46:25,458][20381] Updated weights for policy 1, policy_version 1027337 (0.0008) [2023-12-27 12:46:25,505][20381] Updated weights for policy 1, policy_version 1027347 (0.0008) [2023-12-27 12:46:25,958][20347] Updated weights for policy 0, policy_version 1028141 (0.0006) [2023-12-27 12:46:26,016][20347] Updated weights for policy 0, policy_version 1028151 (0.0005) [2023-12-27 12:46:26,063][20347] Updated weights for policy 0, policy_version 1028161 (0.0005) [2023-12-27 12:46:26,238][20381] Updated weights for policy 1, policy_version 1027357 (0.0009) [2023-12-27 12:46:26,287][20381] Updated weights for policy 1, policy_version 1027367 (0.0007) [2023-12-27 12:46:26,337][20381] Updated weights for policy 1, policy_version 1027377 (0.0009) [2023-12-27 12:46:26,610][20347] Updated weights for policy 0, policy_version 1028171 (0.0005) [2023-12-27 12:46:26,670][20347] Updated weights for policy 0, policy_version 1028181 (0.0005) [2023-12-27 12:46:26,733][20347] Updated weights for policy 0, policy_version 1028191 (0.0006) [2023-12-27 12:46:27,058][20381] Updated weights for policy 1, policy_version 1027387 (0.0008) [2023-12-27 12:46:27,102][20381] Updated weights for policy 1, policy_version 1027397 (0.0005) [2023-12-27 12:46:27,147][20381] Updated weights for policy 1, policy_version 1027407 (0.0006) [2023-12-27 12:46:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 526311424. Throughput: 0: 10322.2, 1: 10004.9. Samples: 526300576. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:27,363][19170] Avg episode reward: [(0, '980.580'), (1, '371.130')] [2023-12-27 12:46:27,368][20347] Updated weights for policy 0, policy_version 1028201 (0.0007) [2023-12-27 12:46:27,418][20347] Updated weights for policy 0, policy_version 1028211 (0.0007) [2023-12-27 12:46:27,475][20347] Updated weights for policy 0, policy_version 1028221 (0.0007) [2023-12-27 12:46:27,535][20347] Updated weights for policy 0, policy_version 1028231 (0.0005) [2023-12-27 12:46:27,725][20381] Updated weights for policy 1, policy_version 1027417 (0.0005) [2023-12-27 12:46:27,785][20381] Updated weights for policy 1, policy_version 1027427 (0.0005) [2023-12-27 12:46:27,838][20381] Updated weights for policy 1, policy_version 1027437 (0.0005) [2023-12-27 12:46:27,896][20381] Updated weights for policy 1, policy_version 1027447 (0.0006) [2023-12-27 12:46:28,087][20347] Updated weights for policy 0, policy_version 1028241 (0.0009) [2023-12-27 12:46:28,139][20347] Updated weights for policy 0, policy_version 1028251 (0.0008) [2023-12-27 12:46:28,182][20347] Updated weights for policy 0, policy_version 1028261 (0.0007) [2023-12-27 12:46:28,481][20381] Updated weights for policy 1, policy_version 1027457 (0.0010) [2023-12-27 12:46:28,537][20381] Updated weights for policy 1, policy_version 1027467 (0.0010) [2023-12-27 12:46:28,591][20381] Updated weights for policy 1, policy_version 1027477 (0.0010) [2023-12-27 12:46:28,946][20347] Updated weights for policy 0, policy_version 1028271 (0.0007) [2023-12-27 12:46:28,991][20347] Updated weights for policy 0, policy_version 1028281 (0.0007) [2023-12-27 12:46:29,036][20347] Updated weights for policy 0, policy_version 1028291 (0.0007) [2023-12-27 12:46:29,236][20381] Updated weights for policy 1, policy_version 1027487 (0.0009) [2023-12-27 12:46:29,306][20381] Updated weights for policy 1, policy_version 1027497 (0.0008) [2023-12-27 12:46:29,376][20381] Updated weights for policy 1, policy_version 1027507 (0.0008) [2023-12-27 12:46:29,814][20347] Updated weights for policy 0, policy_version 1028301 (0.0009) [2023-12-27 12:46:29,877][20347] Updated weights for policy 0, policy_version 1028311 (0.0008) [2023-12-27 12:46:29,938][20347] Updated weights for policy 0, policy_version 1028321 (0.0007) [2023-12-27 12:46:30,087][20381] Updated weights for policy 1, policy_version 1027517 (0.0008) [2023-12-27 12:46:30,147][20381] Updated weights for policy 1, policy_version 1027527 (0.0008) [2023-12-27 12:46:30,203][20381] Updated weights for policy 1, policy_version 1027537 (0.0006) [2023-12-27 12:46:30,614][20347] Updated weights for policy 0, policy_version 1028331 (0.0007) [2023-12-27 12:46:30,679][20347] Updated weights for policy 0, policy_version 1028341 (0.0007) [2023-12-27 12:46:30,739][20347] Updated weights for policy 0, policy_version 1028351 (0.0007) [2023-12-27 12:46:30,848][20381] Updated weights for policy 1, policy_version 1027547 (0.0006) [2023-12-27 12:46:30,903][20381] Updated weights for policy 1, policy_version 1027557 (0.0008) [2023-12-27 12:46:30,960][20381] Updated weights for policy 1, policy_version 1027567 (0.0007) [2023-12-27 12:46:31,455][20347] Updated weights for policy 0, policy_version 1028361 (0.0007) [2023-12-27 12:46:31,510][20347] Updated weights for policy 0, policy_version 1028371 (0.0007) [2023-12-27 12:46:31,558][20347] Updated weights for policy 0, policy_version 1028381 (0.0005) [2023-12-27 12:46:31,609][20347] Updated weights for policy 0, policy_version 1028391 (0.0006) [2023-12-27 12:46:31,616][20381] Updated weights for policy 1, policy_version 1027577 (0.0006) [2023-12-27 12:46:31,685][20381] Updated weights for policy 1, policy_version 1027587 (0.0007) [2023-12-27 12:46:31,749][20381] Updated weights for policy 1, policy_version 1027597 (0.0008) [2023-12-27 12:46:31,806][20381] Updated weights for policy 1, policy_version 1027607 (0.0006) [2023-12-27 12:46:32,359][20347] Updated weights for policy 0, policy_version 1028401 (0.0008) [2023-12-27 12:46:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 526409728. Throughput: 0: 10256.6, 1: 10104.4. Samples: 526425660. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:32,363][19170] Avg episode reward: [(0, '961.180'), (1, '337.290')] [2023-12-27 12:46:32,421][20347] Updated weights for policy 0, policy_version 1028411 (0.0006) [2023-12-27 12:46:32,461][20381] Updated weights for policy 1, policy_version 1027617 (0.0008) [2023-12-27 12:46:32,476][20347] Updated weights for policy 0, policy_version 1028421 (0.0006) [2023-12-27 12:46:32,514][20381] Updated weights for policy 1, policy_version 1027627 (0.0008) [2023-12-27 12:46:32,575][20381] Updated weights for policy 1, policy_version 1027637 (0.0008) [2023-12-27 12:46:33,181][20347] Updated weights for policy 0, policy_version 1028431 (0.0007) [2023-12-27 12:46:33,234][20381] Updated weights for policy 1, policy_version 1027647 (0.0008) [2023-12-27 12:46:33,237][20347] Updated weights for policy 0, policy_version 1028441 (0.0006) [2023-12-27 12:46:33,287][20381] Updated weights for policy 1, policy_version 1027657 (0.0007) [2023-12-27 12:46:33,290][20347] Updated weights for policy 0, policy_version 1028451 (0.0005) [2023-12-27 12:46:33,336][20381] Updated weights for policy 1, policy_version 1027667 (0.0008) [2023-12-27 12:46:33,947][20381] Updated weights for policy 1, policy_version 1027677 (0.0007) [2023-12-27 12:46:33,952][20347] Updated weights for policy 0, policy_version 1028461 (0.0006) [2023-12-27 12:46:33,998][20347] Updated weights for policy 0, policy_version 1028471 (0.0008) [2023-12-27 12:46:34,003][20381] Updated weights for policy 1, policy_version 1027687 (0.0005) [2023-12-27 12:46:34,044][20347] Updated weights for policy 0, policy_version 1028481 (0.0007) [2023-12-27 12:46:34,052][20381] Updated weights for policy 1, policy_version 1027697 (0.0005) [2023-12-27 12:46:34,731][20381] Updated weights for policy 1, policy_version 1027707 (0.0007) [2023-12-27 12:46:34,787][20381] Updated weights for policy 1, policy_version 1027717 (0.0008) [2023-12-27 12:46:34,830][20347] Updated weights for policy 0, policy_version 1028491 (0.0007) [2023-12-27 12:46:34,837][20381] Updated weights for policy 1, policy_version 1027727 (0.0008) [2023-12-27 12:46:34,885][20347] Updated weights for policy 0, policy_version 1028501 (0.0006) [2023-12-27 12:46:34,939][20347] Updated weights for policy 0, policy_version 1028511 (0.0007) [2023-12-27 12:46:35,477][20381] Updated weights for policy 1, policy_version 1027737 (0.0005) [2023-12-27 12:46:35,528][20381] Updated weights for policy 1, policy_version 1027747 (0.0005) [2023-12-27 12:46:35,573][20381] Updated weights for policy 1, policy_version 1027757 (0.0005) [2023-12-27 12:46:35,621][20381] Updated weights for policy 1, policy_version 1027767 (0.0005) [2023-12-27 12:46:35,663][20347] Updated weights for policy 0, policy_version 1028521 (0.0007) [2023-12-27 12:46:35,727][20347] Updated weights for policy 0, policy_version 1028531 (0.0008) [2023-12-27 12:46:35,783][20347] Updated weights for policy 0, policy_version 1028541 (0.0008) [2023-12-27 12:46:35,827][20347] Updated weights for policy 0, policy_version 1028551 (0.0007) [2023-12-27 12:46:36,187][20381] Updated weights for policy 1, policy_version 1027777 (0.0008) [2023-12-27 12:46:36,243][20381] Updated weights for policy 1, policy_version 1027787 (0.0007) [2023-12-27 12:46:36,303][20381] Updated weights for policy 1, policy_version 1027797 (0.0008) [2023-12-27 12:46:36,546][20347] Updated weights for policy 0, policy_version 1028561 (0.0007) [2023-12-27 12:46:36,606][20347] Updated weights for policy 0, policy_version 1028571 (0.0006) [2023-12-27 12:46:36,666][20347] Updated weights for policy 0, policy_version 1028581 (0.0007) [2023-12-27 12:46:37,006][20381] Updated weights for policy 1, policy_version 1027807 (0.0008) [2023-12-27 12:46:37,065][20381] Updated weights for policy 1, policy_version 1027817 (0.0007) [2023-12-27 12:46:37,119][20381] Updated weights for policy 1, policy_version 1027827 (0.0007) [2023-12-27 12:46:37,344][20347] Updated weights for policy 0, policy_version 1028591 (0.0010) [2023-12-27 12:46:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 526516224. Throughput: 0: 10198.2, 1: 10159.1. Samples: 526486668. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:37,363][19170] Avg episode reward: [(0, '922.440'), (1, '459.680')] [2023-12-27 12:46:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001027832_263151616.pth... [2023-12-27 12:46:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001026616_262840320.pth [2023-12-27 12:46:37,402][20347] Updated weights for policy 0, policy_version 1028601 (0.0010) [2023-12-27 12:46:37,455][20347] Updated weights for policy 0, policy_version 1028611 (0.0007) [2023-12-27 12:46:37,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001028616_263372800.pth... [2023-12-27 12:46:37,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001027432_263069696.pth [2023-12-27 12:46:37,853][20381] Updated weights for policy 1, policy_version 1027837 (0.0007) [2023-12-27 12:46:37,910][20381] Updated weights for policy 1, policy_version 1027847 (0.0005) [2023-12-27 12:46:37,974][20381] Updated weights for policy 1, policy_version 1027857 (0.0008) [2023-12-27 12:46:38,085][20347] Updated weights for policy 0, policy_version 1028621 (0.0006) [2023-12-27 12:46:38,148][20347] Updated weights for policy 0, policy_version 1028631 (0.0008) [2023-12-27 12:46:38,201][20347] Updated weights for policy 0, policy_version 1028641 (0.0009) [2023-12-27 12:46:38,590][20381] Updated weights for policy 1, policy_version 1027867 (0.0007) [2023-12-27 12:46:38,652][20381] Updated weights for policy 1, policy_version 1027877 (0.0007) [2023-12-27 12:46:38,712][20381] Updated weights for policy 1, policy_version 1027887 (0.0007) [2023-12-27 12:46:38,880][20347] Updated weights for policy 0, policy_version 1028651 (0.0007) [2023-12-27 12:46:38,934][20347] Updated weights for policy 0, policy_version 1028661 (0.0008) [2023-12-27 12:46:38,986][20347] Updated weights for policy 0, policy_version 1028671 (0.0007) [2023-12-27 12:46:39,027][20291] KL-divergence is very high: 100.3228 [2023-12-27 12:46:39,427][20381] Updated weights for policy 1, policy_version 1027897 (0.0008) [2023-12-27 12:46:39,480][20381] Updated weights for policy 1, policy_version 1027907 (0.0005) [2023-12-27 12:46:39,539][20381] Updated weights for policy 1, policy_version 1027917 (0.0007) [2023-12-27 12:46:39,598][20381] Updated weights for policy 1, policy_version 1027927 (0.0008) [2023-12-27 12:46:39,712][20347] Updated weights for policy 0, policy_version 1028681 (0.0008) [2023-12-27 12:46:39,775][20347] Updated weights for policy 0, policy_version 1028691 (0.0007) [2023-12-27 12:46:39,841][20347] Updated weights for policy 0, policy_version 1028701 (0.0007) [2023-12-27 12:46:39,904][20347] Updated weights for policy 0, policy_version 1028711 (0.0008) [2023-12-27 12:46:40,346][20381] Updated weights for policy 1, policy_version 1027937 (0.0008) [2023-12-27 12:46:40,412][20381] Updated weights for policy 1, policy_version 1027947 (0.0007) [2023-12-27 12:46:40,468][20381] Updated weights for policy 1, policy_version 1027957 (0.0008) [2023-12-27 12:46:40,599][20347] Updated weights for policy 0, policy_version 1028721 (0.0007) [2023-12-27 12:46:40,665][20347] Updated weights for policy 0, policy_version 1028731 (0.0010) [2023-12-27 12:46:40,733][20347] Updated weights for policy 0, policy_version 1028741 (0.0010) [2023-12-27 12:46:41,184][20381] Updated weights for policy 1, policy_version 1027967 (0.0008) [2023-12-27 12:46:41,249][20381] Updated weights for policy 1, policy_version 1027977 (0.0007) [2023-12-27 12:46:41,311][20381] Updated weights for policy 1, policy_version 1027987 (0.0007) [2023-12-27 12:46:41,427][20347] Updated weights for policy 0, policy_version 1028751 (0.0008) [2023-12-27 12:46:41,480][20347] Updated weights for policy 0, policy_version 1028761 (0.0007) [2023-12-27 12:46:41,537][20347] Updated weights for policy 0, policy_version 1028771 (0.0008) [2023-12-27 12:46:42,042][20381] Updated weights for policy 1, policy_version 1027997 (0.0009) [2023-12-27 12:46:42,105][20381] Updated weights for policy 1, policy_version 1028007 (0.0010) [2023-12-27 12:46:42,162][20381] Updated weights for policy 1, policy_version 1028017 (0.0011) [2023-12-27 12:46:42,181][20347] Updated weights for policy 0, policy_version 1028781 (0.0007) [2023-12-27 12:46:42,239][20347] Updated weights for policy 0, policy_version 1028791 (0.0008) [2023-12-27 12:46:42,301][20347] Updated weights for policy 0, policy_version 1028801 (0.0008) [2023-12-27 12:46:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 526622720. Throughput: 0: 10102.7, 1: 10157.7. Samples: 526607724. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:42,363][19170] Avg episode reward: [(0, '922.280'), (1, '478.510')] [2023-12-27 12:46:42,875][20381] Updated weights for policy 1, policy_version 1028027 (0.0009) [2023-12-27 12:46:42,932][20381] Updated weights for policy 1, policy_version 1028037 (0.0011) [2023-12-27 12:46:42,997][20381] Updated weights for policy 1, policy_version 1028047 (0.0011) [2023-12-27 12:46:43,022][20347] Updated weights for policy 0, policy_version 1028811 (0.0007) [2023-12-27 12:46:43,071][20347] Updated weights for policy 0, policy_version 1028821 (0.0008) [2023-12-27 12:46:43,118][20347] Updated weights for policy 0, policy_version 1028831 (0.0007) [2023-12-27 12:46:43,689][20381] Updated weights for policy 1, policy_version 1028057 (0.0011) [2023-12-27 12:46:43,734][20381] Updated weights for policy 1, policy_version 1028067 (0.0010) [2023-12-27 12:46:43,789][20381] Updated weights for policy 1, policy_version 1028077 (0.0010) [2023-12-27 12:46:43,805][20347] Updated weights for policy 0, policy_version 1028841 (0.0007) [2023-12-27 12:46:43,844][20381] Updated weights for policy 1, policy_version 1028087 (0.0010) [2023-12-27 12:46:43,854][20347] Updated weights for policy 0, policy_version 1028851 (0.0007) [2023-12-27 12:46:43,903][20347] Updated weights for policy 0, policy_version 1028861 (0.0010) [2023-12-27 12:46:43,954][20347] Updated weights for policy 0, policy_version 1028871 (0.0010) [2023-12-27 12:46:44,422][20381] Updated weights for policy 1, policy_version 1028097 (0.0006) [2023-12-27 12:46:44,465][20381] Updated weights for policy 1, policy_version 1028107 (0.0005) [2023-12-27 12:46:44,525][20381] Updated weights for policy 1, policy_version 1028117 (0.0006) [2023-12-27 12:46:44,593][20347] Updated weights for policy 0, policy_version 1028881 (0.0010) [2023-12-27 12:46:44,653][20347] Updated weights for policy 0, policy_version 1028891 (0.0009) [2023-12-27 12:46:44,705][20347] Updated weights for policy 0, policy_version 1028901 (0.0010) [2023-12-27 12:46:45,162][20381] Updated weights for policy 1, policy_version 1028127 (0.0009) [2023-12-27 12:46:45,219][20381] Updated weights for policy 1, policy_version 1028137 (0.0011) [2023-12-27 12:46:45,276][20381] Updated weights for policy 1, policy_version 1028147 (0.0011) [2023-12-27 12:46:45,399][20347] Updated weights for policy 0, policy_version 1028911 (0.0008) [2023-12-27 12:46:45,445][20347] Updated weights for policy 0, policy_version 1028921 (0.0006) [2023-12-27 12:46:45,491][20347] Updated weights for policy 0, policy_version 1028931 (0.0005) [2023-12-27 12:46:45,975][20381] Updated weights for policy 1, policy_version 1028157 (0.0010) [2023-12-27 12:46:46,031][20381] Updated weights for policy 1, policy_version 1028167 (0.0011) [2023-12-27 12:46:46,036][20347] Updated weights for policy 0, policy_version 1028941 (0.0007) [2023-12-27 12:46:46,087][20381] Updated weights for policy 1, policy_version 1028177 (0.0011) [2023-12-27 12:46:46,094][20347] Updated weights for policy 0, policy_version 1028951 (0.0010) [2023-12-27 12:46:46,149][20347] Updated weights for policy 0, policy_version 1028961 (0.0010) [2023-12-27 12:46:46,786][20381] Updated weights for policy 1, policy_version 1028187 (0.0010) [2023-12-27 12:46:46,826][20347] Updated weights for policy 0, policy_version 1028971 (0.0010) [2023-12-27 12:46:46,831][20381] Updated weights for policy 1, policy_version 1028197 (0.0010) [2023-12-27 12:46:46,881][20347] Updated weights for policy 0, policy_version 1028981 (0.0010) [2023-12-27 12:46:46,893][20381] Updated weights for policy 1, policy_version 1028207 (0.0011) [2023-12-27 12:46:46,936][20347] Updated weights for policy 0, policy_version 1028991 (0.0010) [2023-12-27 12:46:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 526721024. Throughput: 0: 10140.7, 1: 10217.2. Samples: 526730856. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:47,363][19170] Avg episode reward: [(0, '970.810'), (1, '341.570')] [2023-12-27 12:46:47,580][20347] Updated weights for policy 0, policy_version 1029001 (0.0010) [2023-12-27 12:46:47,584][20381] Updated weights for policy 1, policy_version 1028217 (0.0011) [2023-12-27 12:46:47,633][20381] Updated weights for policy 1, policy_version 1028227 (0.0010) [2023-12-27 12:46:47,643][20347] Updated weights for policy 0, policy_version 1029011 (0.0006) [2023-12-27 12:46:47,681][20381] Updated weights for policy 1, policy_version 1028237 (0.0010) [2023-12-27 12:46:47,707][20347] Updated weights for policy 0, policy_version 1029021 (0.0005) [2023-12-27 12:46:47,736][20381] Updated weights for policy 1, policy_version 1028247 (0.0008) [2023-12-27 12:46:47,772][20347] Updated weights for policy 0, policy_version 1029031 (0.0005) [2023-12-27 12:46:48,277][20347] Updated weights for policy 0, policy_version 1029041 (0.0005) [2023-12-27 12:46:48,308][20381] Updated weights for policy 1, policy_version 1028257 (0.0010) [2023-12-27 12:46:48,337][20347] Updated weights for policy 0, policy_version 1029051 (0.0008) [2023-12-27 12:46:48,367][20381] Updated weights for policy 1, policy_version 1028267 (0.0011) [2023-12-27 12:46:48,396][20347] Updated weights for policy 0, policy_version 1029061 (0.0008) [2023-12-27 12:46:48,424][20381] Updated weights for policy 1, policy_version 1028277 (0.0011) [2023-12-27 12:46:49,067][20347] Updated weights for policy 0, policy_version 1029071 (0.0009) [2023-12-27 12:46:49,116][20347] Updated weights for policy 0, policy_version 1029081 (0.0008) [2023-12-27 12:46:49,132][20381] Updated weights for policy 1, policy_version 1028287 (0.0011) [2023-12-27 12:46:49,167][20347] Updated weights for policy 0, policy_version 1029091 (0.0006) [2023-12-27 12:46:49,180][20381] Updated weights for policy 1, policy_version 1028297 (0.0010) [2023-12-27 12:46:49,237][20381] Updated weights for policy 1, policy_version 1028307 (0.0009) [2023-12-27 12:46:49,854][20381] Updated weights for policy 1, policy_version 1028317 (0.0009) [2023-12-27 12:46:49,907][20347] Updated weights for policy 0, policy_version 1029101 (0.0010) [2023-12-27 12:46:49,916][20381] Updated weights for policy 1, policy_version 1028327 (0.0006) [2023-12-27 12:46:49,968][20347] Updated weights for policy 0, policy_version 1029111 (0.0009) [2023-12-27 12:46:49,977][20381] Updated weights for policy 1, policy_version 1028337 (0.0008) [2023-12-27 12:46:50,030][20347] Updated weights for policy 0, policy_version 1029121 (0.0006) [2023-12-27 12:46:50,617][20347] Updated weights for policy 0, policy_version 1029131 (0.0007) [2023-12-27 12:46:50,677][20347] Updated weights for policy 0, policy_version 1029141 (0.0006) [2023-12-27 12:46:50,712][20381] Updated weights for policy 1, policy_version 1028347 (0.0008) [2023-12-27 12:46:50,745][20347] Updated weights for policy 0, policy_version 1029151 (0.0005) [2023-12-27 12:46:50,772][20381] Updated weights for policy 1, policy_version 1028357 (0.0007) [2023-12-27 12:46:50,829][20381] Updated weights for policy 1, policy_version 1028367 (0.0007) [2023-12-27 12:46:51,390][20347] Updated weights for policy 0, policy_version 1029161 (0.0006) [2023-12-27 12:46:51,459][20347] Updated weights for policy 0, policy_version 1029171 (0.0008) [2023-12-27 12:46:51,520][20347] Updated weights for policy 0, policy_version 1029181 (0.0008) [2023-12-27 12:46:51,575][20347] Updated weights for policy 0, policy_version 1029191 (0.0007) [2023-12-27 12:46:51,579][20381] Updated weights for policy 1, policy_version 1028377 (0.0007) [2023-12-27 12:46:51,641][20381] Updated weights for policy 1, policy_version 1028387 (0.0008) [2023-12-27 12:46:51,696][20381] Updated weights for policy 1, policy_version 1028397 (0.0008) [2023-12-27 12:46:51,759][20381] Updated weights for policy 1, policy_version 1028407 (0.0008) [2023-12-27 12:46:52,256][20347] Updated weights for policy 0, policy_version 1029201 (0.0007) [2023-12-27 12:46:52,314][20347] Updated weights for policy 0, policy_version 1029211 (0.0006) [2023-12-27 12:46:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 526819328. Throughput: 0: 10175.0, 1: 10240.9. Samples: 526794812. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:52,363][19170] Avg episode reward: [(0, '990.310'), (1, '415.200')] [2023-12-27 12:46:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001028408_263299072.pth... [2023-12-27 12:46:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001027224_262995968.pth [2023-12-27 12:46:52,382][20347] Updated weights for policy 0, policy_version 1029221 (0.0007) [2023-12-27 12:46:52,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001029224_263528448.pth... [2023-12-27 12:46:52,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001028008_263217152.pth [2023-12-27 12:46:52,508][20381] Updated weights for policy 1, policy_version 1028417 (0.0006) [2023-12-27 12:46:52,567][20381] Updated weights for policy 1, policy_version 1028427 (0.0011) [2023-12-27 12:46:52,630][20381] Updated weights for policy 1, policy_version 1028437 (0.0011) [2023-12-27 12:46:53,041][20347] Updated weights for policy 0, policy_version 1029231 (0.0008) [2023-12-27 12:46:53,104][20347] Updated weights for policy 0, policy_version 1029241 (0.0005) [2023-12-27 12:46:53,158][20347] Updated weights for policy 0, policy_version 1029251 (0.0005) [2023-12-27 12:46:53,240][20381] Updated weights for policy 1, policy_version 1028447 (0.0007) [2023-12-27 12:46:53,305][20381] Updated weights for policy 1, policy_version 1028457 (0.0006) [2023-12-27 12:46:53,368][20381] Updated weights for policy 1, policy_version 1028467 (0.0006) [2023-12-27 12:46:53,696][20347] Updated weights for policy 0, policy_version 1029261 (0.0005) [2023-12-27 12:46:53,758][20347] Updated weights for policy 0, policy_version 1029271 (0.0005) [2023-12-27 12:46:53,811][20347] Updated weights for policy 0, policy_version 1029281 (0.0005) [2023-12-27 12:46:53,912][20381] Updated weights for policy 1, policy_version 1028477 (0.0009) [2023-12-27 12:46:53,973][20381] Updated weights for policy 1, policy_version 1028487 (0.0010) [2023-12-27 12:46:54,031][20381] Updated weights for policy 1, policy_version 1028497 (0.0011) [2023-12-27 12:46:54,321][20347] Updated weights for policy 0, policy_version 1029291 (0.0007) [2023-12-27 12:46:54,369][20347] Updated weights for policy 0, policy_version 1029301 (0.0010) [2023-12-27 12:46:54,428][20347] Updated weights for policy 0, policy_version 1029311 (0.0010) [2023-12-27 12:46:54,659][20381] Updated weights for policy 1, policy_version 1028507 (0.0009) [2023-12-27 12:46:54,709][20381] Updated weights for policy 1, policy_version 1028517 (0.0005) [2023-12-27 12:46:54,757][20381] Updated weights for policy 1, policy_version 1028527 (0.0005) [2023-12-27 12:46:55,107][20347] Updated weights for policy 0, policy_version 1029321 (0.0010) [2023-12-27 12:46:55,164][20347] Updated weights for policy 0, policy_version 1029331 (0.0010) [2023-12-27 12:46:55,216][20347] Updated weights for policy 0, policy_version 1029341 (0.0010) [2023-12-27 12:46:55,271][20347] Updated weights for policy 0, policy_version 1029351 (0.0010) [2023-12-27 12:46:55,363][20381] Updated weights for policy 1, policy_version 1028537 (0.0006) [2023-12-27 12:46:55,410][20381] Updated weights for policy 1, policy_version 1028547 (0.0010) [2023-12-27 12:46:55,459][20381] Updated weights for policy 1, policy_version 1028557 (0.0010) [2023-12-27 12:46:55,520][20381] Updated weights for policy 1, policy_version 1028567 (0.0011) [2023-12-27 12:46:55,908][20347] Updated weights for policy 0, policy_version 1029361 (0.0010) [2023-12-27 12:46:55,961][20347] Updated weights for policy 0, policy_version 1029371 (0.0010) [2023-12-27 12:46:56,021][20347] Updated weights for policy 0, policy_version 1029381 (0.0010) [2023-12-27 12:46:56,223][20381] Updated weights for policy 1, policy_version 1028577 (0.0010) [2023-12-27 12:46:56,274][20381] Updated weights for policy 1, policy_version 1028587 (0.0010) [2023-12-27 12:46:56,325][20381] Updated weights for policy 1, policy_version 1028597 (0.0010) [2023-12-27 12:46:56,596][20347] Updated weights for policy 0, policy_version 1029391 (0.0010) [2023-12-27 12:46:56,652][20347] Updated weights for policy 0, policy_version 1029401 (0.0006) [2023-12-27 12:46:56,707][20347] Updated weights for policy 0, policy_version 1029411 (0.0006) [2023-12-27 12:46:57,025][20381] Updated weights for policy 1, policy_version 1028607 (0.0008) [2023-12-27 12:46:57,083][20381] Updated weights for policy 1, policy_version 1028617 (0.0007) [2023-12-27 12:46:57,131][20381] Updated weights for policy 1, policy_version 1028627 (0.0008) [2023-12-27 12:46:57,334][20347] Updated weights for policy 0, policy_version 1029421 (0.0007) [2023-12-27 12:46:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 526934016. Throughput: 0: 10281.8, 1: 10301.1. Samples: 526922976. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:46:57,363][19170] Avg episode reward: [(0, '990.220'), (1, '367.260')] [2023-12-27 12:46:57,380][20347] Updated weights for policy 0, policy_version 1029431 (0.0007) [2023-12-27 12:46:57,428][20347] Updated weights for policy 0, policy_version 1029441 (0.0010) [2023-12-27 12:46:57,810][20381] Updated weights for policy 1, policy_version 1028637 (0.0009) [2023-12-27 12:46:57,851][20381] Updated weights for policy 1, policy_version 1028647 (0.0010) [2023-12-27 12:46:57,898][20381] Updated weights for policy 1, policy_version 1028657 (0.0010) [2023-12-27 12:46:58,117][20347] Updated weights for policy 0, policy_version 1029451 (0.0010) [2023-12-27 12:46:58,168][20347] Updated weights for policy 0, policy_version 1029461 (0.0010) [2023-12-27 12:46:58,230][20347] Updated weights for policy 0, policy_version 1029471 (0.0008) [2023-12-27 12:46:58,633][20381] Updated weights for policy 1, policy_version 1028667 (0.0009) [2023-12-27 12:46:58,695][20381] Updated weights for policy 1, policy_version 1028677 (0.0007) [2023-12-27 12:46:58,764][20381] Updated weights for policy 1, policy_version 1028687 (0.0009) [2023-12-27 12:46:59,023][20347] Updated weights for policy 0, policy_version 1029481 (0.0007) [2023-12-27 12:46:59,086][20347] Updated weights for policy 0, policy_version 1029491 (0.0006) [2023-12-27 12:46:59,145][20347] Updated weights for policy 0, policy_version 1029501 (0.0006) [2023-12-27 12:46:59,203][20347] Updated weights for policy 0, policy_version 1029511 (0.0005) [2023-12-27 12:46:59,592][20381] Updated weights for policy 1, policy_version 1028697 (0.0008) [2023-12-27 12:46:59,641][20381] Updated weights for policy 1, policy_version 1028707 (0.0008) [2023-12-27 12:46:59,696][20381] Updated weights for policy 1, policy_version 1028717 (0.0008) [2023-12-27 12:46:59,759][20381] Updated weights for policy 1, policy_version 1028727 (0.0008) [2023-12-27 12:46:59,875][20347] Updated weights for policy 0, policy_version 1029521 (0.0009) [2023-12-27 12:46:59,933][20347] Updated weights for policy 0, policy_version 1029531 (0.0010) [2023-12-27 12:46:59,994][20347] Updated weights for policy 0, policy_version 1029541 (0.0009) [2023-12-27 12:47:00,534][20381] Updated weights for policy 1, policy_version 1028737 (0.0009) [2023-12-27 12:47:00,578][20381] Updated weights for policy 1, policy_version 1028747 (0.0007) [2023-12-27 12:47:00,598][20347] Updated weights for policy 0, policy_version 1029551 (0.0009) [2023-12-27 12:47:00,634][20381] Updated weights for policy 1, policy_version 1028757 (0.0006) [2023-12-27 12:47:00,644][20347] Updated weights for policy 0, policy_version 1029561 (0.0006) [2023-12-27 12:47:00,690][20347] Updated weights for policy 0, policy_version 1029571 (0.0007) [2023-12-27 12:47:01,317][20381] Updated weights for policy 1, policy_version 1028767 (0.0006) [2023-12-27 12:47:01,384][20381] Updated weights for policy 1, policy_version 1028777 (0.0007) [2023-12-27 12:47:01,403][20347] Updated weights for policy 0, policy_version 1029581 (0.0007) [2023-12-27 12:47:01,447][20381] Updated weights for policy 1, policy_version 1028787 (0.0007) [2023-12-27 12:47:01,459][20347] Updated weights for policy 0, policy_version 1029591 (0.0005) [2023-12-27 12:47:01,518][20347] Updated weights for policy 0, policy_version 1029601 (0.0006) [2023-12-27 12:47:02,077][20381] Updated weights for policy 1, policy_version 1028797 (0.0006) [2023-12-27 12:47:02,133][20381] Updated weights for policy 1, policy_version 1028807 (0.0006) [2023-12-27 12:47:02,189][20347] Updated weights for policy 0, policy_version 1029611 (0.0007) [2023-12-27 12:47:02,193][20381] Updated weights for policy 1, policy_version 1028817 (0.0008) [2023-12-27 12:47:02,247][20347] Updated weights for policy 0, policy_version 1029621 (0.0006) [2023-12-27 12:47:02,306][20347] Updated weights for policy 0, policy_version 1029631 (0.0008) [2023-12-27 12:47:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 527032320. Throughput: 0: 10348.2, 1: 10220.3. Samples: 527044288. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:47:02,363][19170] Avg episode reward: [(0, '951.200'), (1, '389.820')] [2023-12-27 12:47:02,884][20381] Updated weights for policy 1, policy_version 1028827 (0.0007) [2023-12-27 12:47:02,937][20381] Updated weights for policy 1, policy_version 1028837 (0.0007) [2023-12-27 12:47:02,990][20347] Updated weights for policy 0, policy_version 1029641 (0.0007) [2023-12-27 12:47:02,992][20381] Updated weights for policy 1, policy_version 1028847 (0.0007) [2023-12-27 12:47:03,050][20347] Updated weights for policy 0, policy_version 1029651 (0.0006) [2023-12-27 12:47:03,115][20347] Updated weights for policy 0, policy_version 1029661 (0.0007) [2023-12-27 12:47:03,178][20347] Updated weights for policy 0, policy_version 1029671 (0.0008) [2023-12-27 12:47:03,616][20381] Updated weights for policy 1, policy_version 1028857 (0.0006) [2023-12-27 12:47:03,674][20381] Updated weights for policy 1, policy_version 1028867 (0.0005) [2023-12-27 12:47:03,727][20381] Updated weights for policy 1, policy_version 1028877 (0.0005) [2023-12-27 12:47:03,774][20381] Updated weights for policy 1, policy_version 1028887 (0.0005) [2023-12-27 12:47:03,903][20347] Updated weights for policy 0, policy_version 1029681 (0.0008) [2023-12-27 12:47:03,958][20347] Updated weights for policy 0, policy_version 1029691 (0.0007) [2023-12-27 12:47:04,013][20347] Updated weights for policy 0, policy_version 1029701 (0.0008) [2023-12-27 12:47:04,406][20381] Updated weights for policy 1, policy_version 1028897 (0.0008) [2023-12-27 12:47:04,458][20381] Updated weights for policy 1, policy_version 1028907 (0.0008) [2023-12-27 12:47:04,514][20381] Updated weights for policy 1, policy_version 1028917 (0.0008) [2023-12-27 12:47:04,683][20347] Updated weights for policy 0, policy_version 1029711 (0.0007) [2023-12-27 12:47:04,752][20347] Updated weights for policy 0, policy_version 1029721 (0.0008) [2023-12-27 12:47:04,815][20347] Updated weights for policy 0, policy_version 1029731 (0.0008) [2023-12-27 12:47:05,228][20381] Updated weights for policy 1, policy_version 1028927 (0.0009) [2023-12-27 12:47:05,286][20381] Updated weights for policy 1, policy_version 1028937 (0.0011) [2023-12-27 12:47:05,344][20381] Updated weights for policy 1, policy_version 1028947 (0.0008) [2023-12-27 12:47:05,364][20347] Updated weights for policy 0, policy_version 1029741 (0.0005) [2023-12-27 12:47:05,423][20347] Updated weights for policy 0, policy_version 1029751 (0.0005) [2023-12-27 12:47:05,472][20347] Updated weights for policy 0, policy_version 1029761 (0.0005) [2023-12-27 12:47:05,935][20381] Updated weights for policy 1, policy_version 1028957 (0.0008) [2023-12-27 12:47:05,984][20381] Updated weights for policy 1, policy_version 1028967 (0.0010) [2023-12-27 12:47:06,037][20381] Updated weights for policy 1, policy_version 1028977 (0.0010) [2023-12-27 12:47:06,054][20347] Updated weights for policy 0, policy_version 1029771 (0.0006) [2023-12-27 12:47:06,108][20347] Updated weights for policy 0, policy_version 1029781 (0.0006) [2023-12-27 12:47:06,173][20347] Updated weights for policy 0, policy_version 1029791 (0.0007) [2023-12-27 12:47:06,717][20381] Updated weights for policy 1, policy_version 1028987 (0.0006) [2023-12-27 12:47:06,785][20381] Updated weights for policy 1, policy_version 1028997 (0.0008) [2023-12-27 12:47:06,843][20381] Updated weights for policy 1, policy_version 1029007 (0.0008) [2023-12-27 12:47:06,906][20347] Updated weights for policy 0, policy_version 1029801 (0.0009) [2023-12-27 12:47:06,960][20347] Updated weights for policy 0, policy_version 1029811 (0.0008) [2023-12-27 12:47:07,011][20347] Updated weights for policy 0, policy_version 1029821 (0.0007) [2023-12-27 12:47:07,064][20347] Updated weights for policy 0, policy_version 1029831 (0.0007) [2023-12-27 12:47:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 527138816. Throughput: 0: 10312.0, 1: 10253.9. Samples: 527105412. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:47:07,363][19170] Avg episode reward: [(0, '941.530'), (1, '449.440')] [2023-12-27 12:47:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001029016_263454720.pth... [2023-12-27 12:47:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001029832_263684096.pth... [2023-12-27 12:47:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001027832_263151616.pth [2023-12-27 12:47:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001028616_263372800.pth [2023-12-27 12:47:07,479][20381] Updated weights for policy 1, policy_version 1029017 (0.0006) [2023-12-27 12:47:07,530][20381] Updated weights for policy 1, policy_version 1029027 (0.0010) [2023-12-27 12:47:07,578][20381] Updated weights for policy 1, policy_version 1029037 (0.0005) [2023-12-27 12:47:07,629][20381] Updated weights for policy 1, policy_version 1029047 (0.0005) [2023-12-27 12:47:07,804][20347] Updated weights for policy 0, policy_version 1029841 (0.0008) [2023-12-27 12:47:07,863][20347] Updated weights for policy 0, policy_version 1029851 (0.0008) [2023-12-27 12:47:07,926][20347] Updated weights for policy 0, policy_version 1029861 (0.0008) [2023-12-27 12:47:08,170][20381] Updated weights for policy 1, policy_version 1029057 (0.0007) [2023-12-27 12:47:08,220][20381] Updated weights for policy 1, policy_version 1029067 (0.0008) [2023-12-27 12:47:08,275][20381] Updated weights for policy 1, policy_version 1029077 (0.0008) [2023-12-27 12:47:08,675][20347] Updated weights for policy 0, policy_version 1029871 (0.0007) [2023-12-27 12:47:08,737][20347] Updated weights for policy 0, policy_version 1029881 (0.0008) [2023-12-27 12:47:08,802][20347] Updated weights for policy 0, policy_version 1029891 (0.0008) [2023-12-27 12:47:08,930][20381] Updated weights for policy 1, policy_version 1029087 (0.0007) [2023-12-27 12:47:08,988][20381] Updated weights for policy 1, policy_version 1029097 (0.0008) [2023-12-27 12:47:09,043][20381] Updated weights for policy 1, policy_version 1029107 (0.0008) [2023-12-27 12:47:09,527][20347] Updated weights for policy 0, policy_version 1029901 (0.0008) [2023-12-27 12:47:09,577][20347] Updated weights for policy 0, policy_version 1029911 (0.0008) [2023-12-27 12:47:09,634][20347] Updated weights for policy 0, policy_version 1029921 (0.0008) [2023-12-27 12:47:09,801][20381] Updated weights for policy 1, policy_version 1029117 (0.0007) [2023-12-27 12:47:09,866][20381] Updated weights for policy 1, policy_version 1029127 (0.0007) [2023-12-27 12:47:09,933][20381] Updated weights for policy 1, policy_version 1029137 (0.0007) [2023-12-27 12:47:10,333][20347] Updated weights for policy 0, policy_version 1029931 (0.0008) [2023-12-27 12:47:10,393][20347] Updated weights for policy 0, policy_version 1029941 (0.0007) [2023-12-27 12:47:10,458][20347] Updated weights for policy 0, policy_version 1029951 (0.0008) [2023-12-27 12:47:10,663][20381] Updated weights for policy 1, policy_version 1029147 (0.0008) [2023-12-27 12:47:10,721][20381] Updated weights for policy 1, policy_version 1029157 (0.0006) [2023-12-27 12:47:10,780][20381] Updated weights for policy 1, policy_version 1029167 (0.0006) [2023-12-27 12:47:11,146][20347] Updated weights for policy 0, policy_version 1029961 (0.0007) [2023-12-27 12:47:11,211][20347] Updated weights for policy 0, policy_version 1029971 (0.0007) [2023-12-27 12:47:11,276][20347] Updated weights for policy 0, policy_version 1029981 (0.0008) [2023-12-27 12:47:11,344][20347] Updated weights for policy 0, policy_version 1029991 (0.0008) [2023-12-27 12:47:11,472][20381] Updated weights for policy 1, policy_version 1029177 (0.0007) [2023-12-27 12:47:11,539][20381] Updated weights for policy 1, policy_version 1029187 (0.0007) [2023-12-27 12:47:11,605][20381] Updated weights for policy 1, policy_version 1029197 (0.0006) [2023-12-27 12:47:11,672][20381] Updated weights for policy 1, policy_version 1029207 (0.0008) [2023-12-27 12:47:12,165][20347] Updated weights for policy 0, policy_version 1030001 (0.0006) [2023-12-27 12:47:12,219][20347] Updated weights for policy 0, policy_version 1030011 (0.0005) [2023-12-27 12:47:12,282][20347] Updated weights for policy 0, policy_version 1030021 (0.0009) [2023-12-27 12:47:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 527237120. Throughput: 0: 10313.2, 1: 10323.5. Samples: 527229232. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:47:12,363][19170] Avg episode reward: [(0, '980.550'), (1, '616.800')] [2023-12-27 12:47:12,443][20381] Updated weights for policy 1, policy_version 1029217 (0.0005) [2023-12-27 12:47:12,498][20381] Updated weights for policy 1, policy_version 1029227 (0.0009) [2023-12-27 12:47:12,551][20381] Updated weights for policy 1, policy_version 1029237 (0.0008) [2023-12-27 12:47:12,951][20347] Updated weights for policy 0, policy_version 1030031 (0.0007) [2023-12-27 12:47:13,015][20347] Updated weights for policy 0, policy_version 1030041 (0.0006) [2023-12-27 12:47:13,070][20347] Updated weights for policy 0, policy_version 1030051 (0.0005) [2023-12-27 12:47:13,227][20381] Updated weights for policy 1, policy_version 1029247 (0.0009) [2023-12-27 12:47:13,273][20381] Updated weights for policy 1, policy_version 1029257 (0.0005) [2023-12-27 12:47:13,324][20381] Updated weights for policy 1, policy_version 1029267 (0.0006) [2023-12-27 12:47:13,705][20347] Updated weights for policy 0, policy_version 1030061 (0.0005) [2023-12-27 12:47:13,752][20347] Updated weights for policy 0, policy_version 1030071 (0.0006) [2023-12-27 12:47:13,800][20347] Updated weights for policy 0, policy_version 1030081 (0.0005) [2023-12-27 12:47:13,913][20381] Updated weights for policy 1, policy_version 1029277 (0.0005) [2023-12-27 12:47:13,972][20381] Updated weights for policy 1, policy_version 1029287 (0.0007) [2023-12-27 12:47:14,030][20381] Updated weights for policy 1, policy_version 1029297 (0.0010) [2023-12-27 12:47:14,431][20347] Updated weights for policy 0, policy_version 1030091 (0.0007) [2023-12-27 12:47:14,491][20347] Updated weights for policy 0, policy_version 1030101 (0.0007) [2023-12-27 12:47:14,549][20347] Updated weights for policy 0, policy_version 1030111 (0.0007) [2023-12-27 12:47:14,621][20381] Updated weights for policy 1, policy_version 1029307 (0.0010) [2023-12-27 12:47:14,670][20381] Updated weights for policy 1, policy_version 1029317 (0.0010) [2023-12-27 12:47:14,718][20381] Updated weights for policy 1, policy_version 1029327 (0.0011) [2023-12-27 12:47:15,224][20347] Updated weights for policy 0, policy_version 1030121 (0.0007) [2023-12-27 12:47:15,289][20347] Updated weights for policy 0, policy_version 1030131 (0.0008) [2023-12-27 12:47:15,358][20347] Updated weights for policy 0, policy_version 1030141 (0.0008) [2023-12-27 12:47:15,426][20347] Updated weights for policy 0, policy_version 1030151 (0.0008) [2023-12-27 12:47:15,457][20381] Updated weights for policy 1, policy_version 1029337 (0.0006) [2023-12-27 12:47:15,521][20381] Updated weights for policy 1, policy_version 1029347 (0.0005) [2023-12-27 12:47:15,582][20381] Updated weights for policy 1, policy_version 1029357 (0.0005) [2023-12-27 12:47:15,637][20381] Updated weights for policy 1, policy_version 1029367 (0.0006) [2023-12-27 12:47:16,085][20347] Updated weights for policy 0, policy_version 1030161 (0.0006) [2023-12-27 12:47:16,135][20347] Updated weights for policy 0, policy_version 1030171 (0.0005) [2023-12-27 12:47:16,197][20347] Updated weights for policy 0, policy_version 1030181 (0.0005) [2023-12-27 12:47:16,249][20381] Updated weights for policy 1, policy_version 1029377 (0.0006) [2023-12-27 12:47:16,311][20381] Updated weights for policy 1, policy_version 1029387 (0.0008) [2023-12-27 12:47:16,363][20381] Updated weights for policy 1, policy_version 1029397 (0.0008) [2023-12-27 12:47:16,803][20347] Updated weights for policy 0, policy_version 1030191 (0.0007) [2023-12-27 12:47:16,865][20347] Updated weights for policy 0, policy_version 1030201 (0.0008) [2023-12-27 12:47:16,918][20347] Updated weights for policy 0, policy_version 1030211 (0.0008) [2023-12-27 12:47:17,021][20381] Updated weights for policy 1, policy_version 1029407 (0.0008) [2023-12-27 12:47:17,079][20381] Updated weights for policy 1, policy_version 1029417 (0.0007) [2023-12-27 12:47:17,128][20381] Updated weights for policy 1, policy_version 1029427 (0.0007) [2023-12-27 12:47:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 527343616. Throughput: 0: 10297.2, 1: 10293.7. Samples: 527352248. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:47:17,363][19170] Avg episode reward: [(0, '980.530'), (1, '636.690')] [2023-12-27 12:47:17,575][20347] Updated weights for policy 0, policy_version 1030221 (0.0008) [2023-12-27 12:47:17,630][20347] Updated weights for policy 0, policy_version 1030231 (0.0007) [2023-12-27 12:47:17,695][20347] Updated weights for policy 0, policy_version 1030241 (0.0010) [2023-12-27 12:47:17,797][20381] Updated weights for policy 1, policy_version 1029437 (0.0008) [2023-12-27 12:47:17,847][20381] Updated weights for policy 1, policy_version 1029447 (0.0008) [2023-12-27 12:47:17,899][20381] Updated weights for policy 1, policy_version 1029457 (0.0008) [2023-12-27 12:47:18,283][20347] Updated weights for policy 0, policy_version 1030251 (0.0009) [2023-12-27 12:47:18,352][20347] Updated weights for policy 0, policy_version 1030261 (0.0008) [2023-12-27 12:47:18,409][20347] Updated weights for policy 0, policy_version 1030271 (0.0008) [2023-12-27 12:47:18,659][20381] Updated weights for policy 1, policy_version 1029467 (0.0009) [2023-12-27 12:47:18,716][20381] Updated weights for policy 1, policy_version 1029477 (0.0008) [2023-12-27 12:47:18,775][20381] Updated weights for policy 1, policy_version 1029487 (0.0008) [2023-12-27 12:47:19,108][20347] Updated weights for policy 0, policy_version 1030281 (0.0007) [2023-12-27 12:47:19,165][20347] Updated weights for policy 0, policy_version 1030291 (0.0007) [2023-12-27 12:47:19,219][20347] Updated weights for policy 0, policy_version 1030301 (0.0009) [2023-12-27 12:47:19,284][20347] Updated weights for policy 0, policy_version 1030311 (0.0007) [2023-12-27 12:47:19,468][20381] Updated weights for policy 1, policy_version 1029497 (0.0008) [2023-12-27 12:47:19,529][20381] Updated weights for policy 1, policy_version 1029507 (0.0007) [2023-12-27 12:47:19,591][20381] Updated weights for policy 1, policy_version 1029517 (0.0006) [2023-12-27 12:47:19,650][20381] Updated weights for policy 1, policy_version 1029527 (0.0007) [2023-12-27 12:47:20,004][20347] Updated weights for policy 0, policy_version 1030321 (0.0006) [2023-12-27 12:47:20,068][20347] Updated weights for policy 0, policy_version 1030331 (0.0008) [2023-12-27 12:47:20,125][20347] Updated weights for policy 0, policy_version 1030341 (0.0008) [2023-12-27 12:47:20,276][20381] Updated weights for policy 1, policy_version 1029537 (0.0009) [2023-12-27 12:47:20,327][20381] Updated weights for policy 1, policy_version 1029547 (0.0008) [2023-12-27 12:47:20,385][20381] Updated weights for policy 1, policy_version 1029557 (0.0008) [2023-12-27 12:47:20,816][20347] Updated weights for policy 0, policy_version 1030351 (0.0008) [2023-12-27 12:47:20,875][20347] Updated weights for policy 0, policy_version 1030361 (0.0007) [2023-12-27 12:47:20,937][20347] Updated weights for policy 0, policy_version 1030371 (0.0007) [2023-12-27 12:47:21,081][20381] Updated weights for policy 1, policy_version 1029567 (0.0008) [2023-12-27 12:47:21,144][20381] Updated weights for policy 1, policy_version 1029577 (0.0008) [2023-12-27 12:47:21,205][20381] Updated weights for policy 1, policy_version 1029587 (0.0008) [2023-12-27 12:47:21,663][20347] Updated weights for policy 0, policy_version 1030381 (0.0008) [2023-12-27 12:47:21,730][20347] Updated weights for policy 0, policy_version 1030391 (0.0007) [2023-12-27 12:47:21,794][20347] Updated weights for policy 0, policy_version 1030401 (0.0008) [2023-12-27 12:47:21,996][20381] Updated weights for policy 1, policy_version 1029597 (0.0009) [2023-12-27 12:47:22,056][20381] Updated weights for policy 1, policy_version 1029607 (0.0010) [2023-12-27 12:47:22,116][20381] Updated weights for policy 1, policy_version 1029617 (0.0011) [2023-12-27 12:47:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 527441920. Throughput: 0: 10335.2, 1: 10265.2. Samples: 527413688. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 12:47:22,363][19170] Avg episode reward: [(0, '951.380'), (1, '577.960')] [2023-12-27 12:47:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001029624_263610368.pth... [2023-12-27 12:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001030408_263831552.pth... [2023-12-27 12:47:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001028408_263299072.pth [2023-12-27 12:47:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001029224_263528448.pth [2023-12-27 12:47:22,480][20347] Updated weights for policy 0, policy_version 1030411 (0.0007) [2023-12-27 12:47:22,543][20347] Updated weights for policy 0, policy_version 1030421 (0.0007) [2023-12-27 12:47:22,603][20347] Updated weights for policy 0, policy_version 1030431 (0.0006) [2023-12-27 12:47:22,801][20381] Updated weights for policy 1, policy_version 1029627 (0.0010) [2023-12-27 12:47:22,862][20381] Updated weights for policy 1, policy_version 1029637 (0.0007) [2023-12-27 12:47:22,923][20381] Updated weights for policy 1, policy_version 1029647 (0.0005) [2023-12-27 12:47:23,288][20347] Updated weights for policy 0, policy_version 1030441 (0.0007) [2023-12-27 12:47:23,349][20347] Updated weights for policy 0, policy_version 1030451 (0.0006) [2023-12-27 12:47:23,411][20347] Updated weights for policy 0, policy_version 1030461 (0.0008) [2023-12-27 12:47:23,472][20347] Updated weights for policy 0, policy_version 1030471 (0.0007) [2023-12-27 12:47:23,500][20381] Updated weights for policy 1, policy_version 1029657 (0.0010) [2023-12-27 12:47:23,564][20381] Updated weights for policy 1, policy_version 1029667 (0.0010) [2023-12-27 12:47:23,615][20381] Updated weights for policy 1, policy_version 1029677 (0.0010) [2023-12-27 12:47:23,673][20381] Updated weights for policy 1, policy_version 1029687 (0.0010) [2023-12-27 12:47:24,167][20347] Updated weights for policy 0, policy_version 1030481 (0.0007) [2023-12-27 12:47:24,218][20381] Updated weights for policy 1, policy_version 1029697 (0.0006) [2023-12-27 12:47:24,227][20347] Updated weights for policy 0, policy_version 1030491 (0.0008) [2023-12-27 12:47:24,272][20381] Updated weights for policy 1, policy_version 1029707 (0.0006) [2023-12-27 12:47:24,274][20347] Updated weights for policy 0, policy_version 1030501 (0.0007) [2023-12-27 12:47:24,323][20381] Updated weights for policy 1, policy_version 1029717 (0.0005) [2023-12-27 12:47:24,924][20381] Updated weights for policy 1, policy_version 1029727 (0.0009) [2023-12-27 12:47:24,929][20347] Updated weights for policy 0, policy_version 1030511 (0.0006) [2023-12-27 12:47:24,983][20381] Updated weights for policy 1, policy_version 1029737 (0.0010) [2023-12-27 12:47:24,993][20347] Updated weights for policy 0, policy_version 1030521 (0.0005) [2023-12-27 12:47:25,038][20381] Updated weights for policy 1, policy_version 1029747 (0.0011) [2023-12-27 12:47:25,049][20347] Updated weights for policy 0, policy_version 1030531 (0.0005) [2023-12-27 12:47:25,558][20347] Updated weights for policy 0, policy_version 1030541 (0.0007) [2023-12-27 12:47:25,620][20347] Updated weights for policy 0, policy_version 1030551 (0.0007) [2023-12-27 12:47:25,683][20347] Updated weights for policy 0, policy_version 1030561 (0.0007) [2023-12-27 12:47:25,718][20381] Updated weights for policy 1, policy_version 1029757 (0.0010) [2023-12-27 12:47:25,766][20381] Updated weights for policy 1, policy_version 1029767 (0.0010) [2023-12-27 12:47:25,827][20381] Updated weights for policy 1, policy_version 1029777 (0.0010) [2023-12-27 12:47:26,283][20347] Updated weights for policy 0, policy_version 1030571 (0.0007) [2023-12-27 12:47:26,347][20347] Updated weights for policy 0, policy_version 1030581 (0.0007) [2023-12-27 12:47:26,402][20347] Updated weights for policy 0, policy_version 1030591 (0.0007) [2023-12-27 12:47:26,511][20381] Updated weights for policy 1, policy_version 1029787 (0.0011) [2023-12-27 12:47:26,565][20381] Updated weights for policy 1, policy_version 1029797 (0.0010) [2023-12-27 12:47:26,619][20381] Updated weights for policy 1, policy_version 1029807 (0.0010) [2023-12-27 12:47:27,057][20347] Updated weights for policy 0, policy_version 1030601 (0.0007) [2023-12-27 12:47:27,104][20347] Updated weights for policy 0, policy_version 1030611 (0.0007) [2023-12-27 12:47:27,151][20347] Updated weights for policy 0, policy_version 1030621 (0.0007) [2023-12-27 12:47:27,205][20347] Updated weights for policy 0, policy_version 1030631 (0.0008) [2023-12-27 12:47:27,320][20381] Updated weights for policy 1, policy_version 1029817 (0.0010) [2023-12-27 12:47:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 527548416. Throughput: 0: 10384.7, 1: 10304.9. Samples: 527538756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:27,363][19170] Avg episode reward: [(0, '961.160'), (1, '666.510')] [2023-12-27 12:47:27,376][20381] Updated weights for policy 1, policy_version 1029827 (0.0011) [2023-12-27 12:47:27,431][20381] Updated weights for policy 1, policy_version 1029837 (0.0011) [2023-12-27 12:47:27,477][20381] Updated weights for policy 1, policy_version 1029847 (0.0010) [2023-12-27 12:47:27,922][20347] Updated weights for policy 0, policy_version 1030641 (0.0007) [2023-12-27 12:47:27,976][20347] Updated weights for policy 0, policy_version 1030651 (0.0007) [2023-12-27 12:47:28,027][20347] Updated weights for policy 0, policy_version 1030661 (0.0009) [2023-12-27 12:47:28,090][20381] Updated weights for policy 1, policy_version 1029857 (0.0006) [2023-12-27 12:47:28,137][20381] Updated weights for policy 1, policy_version 1029867 (0.0005) [2023-12-27 12:47:28,183][20381] Updated weights for policy 1, policy_version 1029877 (0.0007) [2023-12-27 12:47:28,744][20347] Updated weights for policy 0, policy_version 1030671 (0.0006) [2023-12-27 12:47:28,807][20347] Updated weights for policy 0, policy_version 1030681 (0.0006) [2023-12-27 12:47:28,835][20381] Updated weights for policy 1, policy_version 1029887 (0.0008) [2023-12-27 12:47:28,862][20347] Updated weights for policy 0, policy_version 1030691 (0.0005) [2023-12-27 12:47:28,902][20381] Updated weights for policy 1, policy_version 1029897 (0.0008) [2023-12-27 12:47:28,948][20381] Updated weights for policy 1, policy_version 1029907 (0.0006) [2023-12-27 12:47:29,519][20347] Updated weights for policy 0, policy_version 1030701 (0.0006) [2023-12-27 12:47:29,575][20347] Updated weights for policy 0, policy_version 1030711 (0.0008) [2023-12-27 12:47:29,619][20381] Updated weights for policy 1, policy_version 1029917 (0.0007) [2023-12-27 12:47:29,632][20347] Updated weights for policy 0, policy_version 1030721 (0.0006) [2023-12-27 12:47:29,681][20381] Updated weights for policy 1, policy_version 1029927 (0.0005) [2023-12-27 12:47:29,738][20381] Updated weights for policy 1, policy_version 1029937 (0.0006) [2023-12-27 12:47:30,354][20347] Updated weights for policy 0, policy_version 1030731 (0.0008) [2023-12-27 12:47:30,406][20347] Updated weights for policy 0, policy_version 1030741 (0.0007) [2023-12-27 12:47:30,416][20381] Updated weights for policy 1, policy_version 1029947 (0.0007) [2023-12-27 12:47:30,454][20347] Updated weights for policy 0, policy_version 1030751 (0.0009) [2023-12-27 12:47:30,465][20381] Updated weights for policy 1, policy_version 1029957 (0.0009) [2023-12-27 12:47:30,511][20381] Updated weights for policy 1, policy_version 1029967 (0.0006) [2023-12-27 12:47:31,121][20347] Updated weights for policy 0, policy_version 1030761 (0.0007) [2023-12-27 12:47:31,188][20347] Updated weights for policy 0, policy_version 1030771 (0.0006) [2023-12-27 12:47:31,238][20347] Updated weights for policy 0, policy_version 1030781 (0.0007) [2023-12-27 12:47:31,257][20381] Updated weights for policy 1, policy_version 1029977 (0.0008) [2023-12-27 12:47:31,298][20347] Updated weights for policy 0, policy_version 1030791 (0.0007) [2023-12-27 12:47:31,312][20381] Updated weights for policy 1, policy_version 1029987 (0.0007) [2023-12-27 12:47:31,371][20381] Updated weights for policy 1, policy_version 1029997 (0.0008) [2023-12-27 12:47:31,431][20381] Updated weights for policy 1, policy_version 1030007 (0.0007) [2023-12-27 12:47:31,995][20347] Updated weights for policy 0, policy_version 1030801 (0.0007) [2023-12-27 12:47:32,048][20347] Updated weights for policy 0, policy_version 1030811 (0.0008) [2023-12-27 12:47:32,101][20347] Updated weights for policy 0, policy_version 1030821 (0.0007) [2023-12-27 12:47:32,129][20381] Updated weights for policy 1, policy_version 1030017 (0.0007) [2023-12-27 12:47:32,187][20381] Updated weights for policy 1, policy_version 1030027 (0.0007) [2023-12-27 12:47:32,239][20381] Updated weights for policy 1, policy_version 1030037 (0.0007) [2023-12-27 12:47:32,363][19170] Fps is (10 sec: 21297.2, 60 sec: 20752.7, 300 sec: 20493.8). Total num frames: 527654912. Throughput: 0: 10350.4, 1: 10317.2. Samples: 527660920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:32,364][19170] Avg episode reward: [(0, '980.700'), (1, '598.170')] [2023-12-27 12:47:32,766][20347] Updated weights for policy 0, policy_version 1030831 (0.0007) [2023-12-27 12:47:32,823][20347] Updated weights for policy 0, policy_version 1030841 (0.0007) [2023-12-27 12:47:32,880][20347] Updated weights for policy 0, policy_version 1030851 (0.0008) [2023-12-27 12:47:32,910][20381] Updated weights for policy 1, policy_version 1030047 (0.0006) [2023-12-27 12:47:32,964][20381] Updated weights for policy 1, policy_version 1030057 (0.0005) [2023-12-27 12:47:33,021][20381] Updated weights for policy 1, policy_version 1030067 (0.0007) [2023-12-27 12:47:33,548][20347] Updated weights for policy 0, policy_version 1030861 (0.0008) [2023-12-27 12:47:33,593][20347] Updated weights for policy 0, policy_version 1030871 (0.0007) [2023-12-27 12:47:33,648][20347] Updated weights for policy 0, policy_version 1030881 (0.0007) [2023-12-27 12:47:33,650][20381] Updated weights for policy 1, policy_version 1030077 (0.0008) [2023-12-27 12:47:33,700][20381] Updated weights for policy 1, policy_version 1030087 (0.0007) [2023-12-27 12:47:33,754][20381] Updated weights for policy 1, policy_version 1030097 (0.0007) [2023-12-27 12:47:34,359][20347] Updated weights for policy 0, policy_version 1030891 (0.0007) [2023-12-27 12:47:34,427][20347] Updated weights for policy 0, policy_version 1030901 (0.0008) [2023-12-27 12:47:34,453][20381] Updated weights for policy 1, policy_version 1030107 (0.0007) [2023-12-27 12:47:34,485][20347] Updated weights for policy 0, policy_version 1030911 (0.0008) [2023-12-27 12:47:34,507][20381] Updated weights for policy 1, policy_version 1030117 (0.0006) [2023-12-27 12:47:34,565][20381] Updated weights for policy 1, policy_version 1030127 (0.0008) [2023-12-27 12:47:35,088][20347] Updated weights for policy 0, policy_version 1030921 (0.0008) [2023-12-27 12:47:35,151][20347] Updated weights for policy 0, policy_version 1030931 (0.0005) [2023-12-27 12:47:35,206][20347] Updated weights for policy 0, policy_version 1030941 (0.0005) [2023-12-27 12:47:35,260][20347] Updated weights for policy 0, policy_version 1030951 (0.0007) [2023-12-27 12:47:35,274][20381] Updated weights for policy 1, policy_version 1030137 (0.0008) [2023-12-27 12:47:35,320][20381] Updated weights for policy 1, policy_version 1030147 (0.0010) [2023-12-27 12:47:35,366][20381] Updated weights for policy 1, policy_version 1030157 (0.0009) [2023-12-27 12:47:35,415][20381] Updated weights for policy 1, policy_version 1030167 (0.0007) [2023-12-27 12:47:35,825][20347] Updated weights for policy 0, policy_version 1030961 (0.0008) [2023-12-27 12:47:35,875][20347] Updated weights for policy 0, policy_version 1030971 (0.0009) [2023-12-27 12:47:35,924][20347] Updated weights for policy 0, policy_version 1030981 (0.0008) [2023-12-27 12:47:36,107][20381] Updated weights for policy 1, policy_version 1030177 (0.0006) [2023-12-27 12:47:36,167][20381] Updated weights for policy 1, policy_version 1030187 (0.0008) [2023-12-27 12:47:36,228][20381] Updated weights for policy 1, policy_version 1030197 (0.0008) [2023-12-27 12:47:36,690][20347] Updated weights for policy 0, policy_version 1030991 (0.0008) [2023-12-27 12:47:36,748][20347] Updated weights for policy 0, policy_version 1031001 (0.0007) [2023-12-27 12:47:36,812][20347] Updated weights for policy 0, policy_version 1031011 (0.0007) [2023-12-27 12:47:36,866][20381] Updated weights for policy 1, policy_version 1030207 (0.0007) [2023-12-27 12:47:36,929][20381] Updated weights for policy 1, policy_version 1030217 (0.0006) [2023-12-27 12:47:36,988][20381] Updated weights for policy 1, policy_version 1030227 (0.0008) [2023-12-27 12:47:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 527753216. Throughput: 0: 10333.7, 1: 10294.5. Samples: 527723080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:37,363][19170] Avg episode reward: [(0, '980.580'), (1, '705.640')] [2023-12-27 12:47:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001030232_263766016.pth... [2023-12-27 12:47:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001031016_263987200.pth... [2023-12-27 12:47:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001029016_263454720.pth [2023-12-27 12:47:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001029832_263684096.pth [2023-12-27 12:47:37,462][20347] Updated weights for policy 0, policy_version 1031021 (0.0007) [2023-12-27 12:47:37,521][20347] Updated weights for policy 0, policy_version 1031031 (0.0007) [2023-12-27 12:47:37,577][20347] Updated weights for policy 0, policy_version 1031041 (0.0008) [2023-12-27 12:47:37,686][20381] Updated weights for policy 1, policy_version 1030237 (0.0009) [2023-12-27 12:47:37,740][20381] Updated weights for policy 1, policy_version 1030247 (0.0008) [2023-12-27 12:47:37,790][20381] Updated weights for policy 1, policy_version 1030257 (0.0008) [2023-12-27 12:47:38,253][20347] Updated weights for policy 0, policy_version 1031051 (0.0007) [2023-12-27 12:47:38,309][20347] Updated weights for policy 0, policy_version 1031061 (0.0007) [2023-12-27 12:47:38,372][20347] Updated weights for policy 0, policy_version 1031071 (0.0007) [2023-12-27 12:47:38,509][20381] Updated weights for policy 1, policy_version 1030267 (0.0008) [2023-12-27 12:47:38,565][20381] Updated weights for policy 1, policy_version 1030277 (0.0008) [2023-12-27 12:47:38,611][20381] Updated weights for policy 1, policy_version 1030287 (0.0007) [2023-12-27 12:47:39,032][20347] Updated weights for policy 0, policy_version 1031081 (0.0007) [2023-12-27 12:47:39,082][20347] Updated weights for policy 0, policy_version 1031091 (0.0005) [2023-12-27 12:47:39,127][20347] Updated weights for policy 0, policy_version 1031101 (0.0005) [2023-12-27 12:47:39,174][20347] Updated weights for policy 0, policy_version 1031111 (0.0005) [2023-12-27 12:47:39,364][20381] Updated weights for policy 1, policy_version 1030297 (0.0007) [2023-12-27 12:47:39,430][20381] Updated weights for policy 1, policy_version 1030307 (0.0009) [2023-12-27 12:47:39,493][20381] Updated weights for policy 1, policy_version 1030317 (0.0007) [2023-12-27 12:47:39,556][20381] Updated weights for policy 1, policy_version 1030327 (0.0008) [2023-12-27 12:47:39,864][20347] Updated weights for policy 0, policy_version 1031121 (0.0008) [2023-12-27 12:47:39,930][20347] Updated weights for policy 0, policy_version 1031131 (0.0007) [2023-12-27 12:47:39,991][20347] Updated weights for policy 0, policy_version 1031141 (0.0007) [2023-12-27 12:47:40,322][20381] Updated weights for policy 1, policy_version 1030337 (0.0007) [2023-12-27 12:47:40,392][20381] Updated weights for policy 1, policy_version 1030347 (0.0009) [2023-12-27 12:47:40,455][20381] Updated weights for policy 1, policy_version 1030357 (0.0008) [2023-12-27 12:47:40,651][20347] Updated weights for policy 0, policy_version 1031151 (0.0010) [2023-12-27 12:47:40,703][20347] Updated weights for policy 0, policy_version 1031161 (0.0010) [2023-12-27 12:47:40,753][20347] Updated weights for policy 0, policy_version 1031171 (0.0010) [2023-12-27 12:47:41,047][20381] Updated weights for policy 1, policy_version 1030367 (0.0008) [2023-12-27 12:47:41,111][20381] Updated weights for policy 1, policy_version 1030377 (0.0008) [2023-12-27 12:47:41,177][20381] Updated weights for policy 1, policy_version 1030387 (0.0006) [2023-12-27 12:47:41,553][20347] Updated weights for policy 0, policy_version 1031181 (0.0009) [2023-12-27 12:47:41,613][20347] Updated weights for policy 0, policy_version 1031191 (0.0008) [2023-12-27 12:47:41,677][20347] Updated weights for policy 0, policy_version 1031201 (0.0008) [2023-12-27 12:47:41,922][20381] Updated weights for policy 1, policy_version 1030397 (0.0008) [2023-12-27 12:47:41,976][20381] Updated weights for policy 1, policy_version 1030407 (0.0008) [2023-12-27 12:47:42,031][20381] Updated weights for policy 1, policy_version 1030417 (0.0008) [2023-12-27 12:47:42,362][19170] Fps is (10 sec: 19662.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 527851520. Throughput: 0: 10252.2, 1: 10215.6. Samples: 527844028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:42,363][19170] Avg episode reward: [(0, '980.570'), (1, '607.730')] [2023-12-27 12:47:42,386][20347] Updated weights for policy 0, policy_version 1031211 (0.0007) [2023-12-27 12:47:42,446][20347] Updated weights for policy 0, policy_version 1031221 (0.0008) [2023-12-27 12:47:42,505][20347] Updated weights for policy 0, policy_version 1031231 (0.0007) [2023-12-27 12:47:42,755][20381] Updated weights for policy 1, policy_version 1030427 (0.0008) [2023-12-27 12:47:42,810][20381] Updated weights for policy 1, policy_version 1030437 (0.0007) [2023-12-27 12:47:42,873][20381] Updated weights for policy 1, policy_version 1030447 (0.0007) [2023-12-27 12:47:43,152][20347] Updated weights for policy 0, policy_version 1031241 (0.0007) [2023-12-27 12:47:43,208][20347] Updated weights for policy 0, policy_version 1031251 (0.0006) [2023-12-27 12:47:43,268][20347] Updated weights for policy 0, policy_version 1031261 (0.0008) [2023-12-27 12:47:43,325][20347] Updated weights for policy 0, policy_version 1031271 (0.0007) [2023-12-27 12:47:43,521][20381] Updated weights for policy 1, policy_version 1030457 (0.0009) [2023-12-27 12:47:43,583][20381] Updated weights for policy 1, policy_version 1030467 (0.0008) [2023-12-27 12:47:43,644][20381] Updated weights for policy 1, policy_version 1030477 (0.0008) [2023-12-27 12:47:43,697][20381] Updated weights for policy 1, policy_version 1030487 (0.0005) [2023-12-27 12:47:43,921][20347] Updated weights for policy 0, policy_version 1031281 (0.0010) [2023-12-27 12:47:43,979][20347] Updated weights for policy 0, policy_version 1031291 (0.0010) [2023-12-27 12:47:44,027][20347] Updated weights for policy 0, policy_version 1031301 (0.0010) [2023-12-27 12:47:44,345][20381] Updated weights for policy 1, policy_version 1030497 (0.0010) [2023-12-27 12:47:44,392][20381] Updated weights for policy 1, policy_version 1030507 (0.0010) [2023-12-27 12:47:44,441][20381] Updated weights for policy 1, policy_version 1030517 (0.0010) [2023-12-27 12:47:44,677][20347] Updated weights for policy 0, policy_version 1031311 (0.0006) [2023-12-27 12:47:44,732][20347] Updated weights for policy 0, policy_version 1031321 (0.0008) [2023-12-27 12:47:44,800][20347] Updated weights for policy 0, policy_version 1031332 (0.0008) [2023-12-27 12:47:45,102][20381] Updated weights for policy 1, policy_version 1030527 (0.0010) [2023-12-27 12:47:45,159][20381] Updated weights for policy 1, policy_version 1030537 (0.0011) [2023-12-27 12:47:45,213][20381] Updated weights for policy 1, policy_version 1030547 (0.0011) [2023-12-27 12:47:45,456][20347] Updated weights for policy 0, policy_version 1031342 (0.0006) [2023-12-27 12:47:45,506][20347] Updated weights for policy 0, policy_version 1031352 (0.0006) [2023-12-27 12:47:45,561][20347] Updated weights for policy 0, policy_version 1031362 (0.0007) [2023-12-27 12:47:45,865][20381] Updated weights for policy 1, policy_version 1030557 (0.0008) [2023-12-27 12:47:45,931][20381] Updated weights for policy 1, policy_version 1030567 (0.0005) [2023-12-27 12:47:45,995][20381] Updated weights for policy 1, policy_version 1030577 (0.0008) [2023-12-27 12:47:46,185][20347] Updated weights for policy 0, policy_version 1031372 (0.0007) [2023-12-27 12:47:46,246][20347] Updated weights for policy 0, policy_version 1031382 (0.0007) [2023-12-27 12:47:46,306][20347] Updated weights for policy 0, policy_version 1031392 (0.0007) [2023-12-27 12:47:46,589][20381] Updated weights for policy 1, policy_version 1030587 (0.0011) [2023-12-27 12:47:46,638][20381] Updated weights for policy 1, policy_version 1030597 (0.0010) [2023-12-27 12:47:46,686][20381] Updated weights for policy 1, policy_version 1030607 (0.0010) [2023-12-27 12:47:46,981][20347] Updated weights for policy 0, policy_version 1031402 (0.0007) [2023-12-27 12:47:47,038][20347] Updated weights for policy 0, policy_version 1031412 (0.0007) [2023-12-27 12:47:47,095][20347] Updated weights for policy 0, policy_version 1031422 (0.0007) [2023-12-27 12:47:47,155][20347] Updated weights for policy 0, policy_version 1031432 (0.0007) [2023-12-27 12:47:47,351][20381] Updated weights for policy 1, policy_version 1030617 (0.0010) [2023-12-27 12:47:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 527958016. Throughput: 0: 10235.2, 1: 10314.7. Samples: 527969032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:47,363][19170] Avg episode reward: [(0, '980.510'), (1, '627.410')] [2023-12-27 12:47:47,405][20381] Updated weights for policy 1, policy_version 1030627 (0.0005) [2023-12-27 12:47:47,448][20381] Updated weights for policy 1, policy_version 1030637 (0.0006) [2023-12-27 12:47:47,508][20381] Updated weights for policy 1, policy_version 1030647 (0.0006) [2023-12-27 12:47:47,767][20347] Updated weights for policy 0, policy_version 1031442 (0.0006) [2023-12-27 12:47:47,816][20347] Updated weights for policy 0, policy_version 1031452 (0.0007) [2023-12-27 12:47:47,866][20347] Updated weights for policy 0, policy_version 1031462 (0.0007) [2023-12-27 12:47:48,089][20381] Updated weights for policy 1, policy_version 1030657 (0.0010) [2023-12-27 12:47:48,146][20381] Updated weights for policy 1, policy_version 1030667 (0.0005) [2023-12-27 12:47:48,209][20381] Updated weights for policy 1, policy_version 1030677 (0.0005) [2023-12-27 12:47:48,611][20347] Updated weights for policy 0, policy_version 1031472 (0.0009) [2023-12-27 12:47:48,667][20347] Updated weights for policy 0, policy_version 1031482 (0.0009) [2023-12-27 12:47:48,681][20291] KL-divergence is very high: 306.3377 [2023-12-27 12:47:48,720][20347] Updated weights for policy 0, policy_version 1031492 (0.0009) [2023-12-27 12:47:48,727][20291] KL-divergence is very high: 110.7854 [2023-12-27 12:47:48,855][20381] Updated weights for policy 1, policy_version 1030687 (0.0010) [2023-12-27 12:47:48,915][20381] Updated weights for policy 1, policy_version 1030697 (0.0011) [2023-12-27 12:47:48,972][20381] Updated weights for policy 1, policy_version 1030707 (0.0010) [2023-12-27 12:47:49,454][20347] Updated weights for policy 0, policy_version 1031502 (0.0008) [2023-12-27 12:47:49,519][20347] Updated weights for policy 0, policy_version 1031512 (0.0007) [2023-12-27 12:47:49,587][20347] Updated weights for policy 0, policy_version 1031522 (0.0007) [2023-12-27 12:47:49,641][20381] Updated weights for policy 1, policy_version 1030717 (0.0008) [2023-12-27 12:47:49,700][20381] Updated weights for policy 1, policy_version 1030727 (0.0008) [2023-12-27 12:47:49,759][20381] Updated weights for policy 1, policy_version 1030737 (0.0006) [2023-12-27 12:47:50,276][20347] Updated weights for policy 0, policy_version 1031532 (0.0007) [2023-12-27 12:47:50,324][20347] Updated weights for policy 0, policy_version 1031542 (0.0007) [2023-12-27 12:47:50,373][20347] Updated weights for policy 0, policy_version 1031552 (0.0007) [2023-12-27 12:47:50,443][20381] Updated weights for policy 1, policy_version 1030747 (0.0005) [2023-12-27 12:47:50,496][20381] Updated weights for policy 1, policy_version 1030757 (0.0005) [2023-12-27 12:47:50,552][20381] Updated weights for policy 1, policy_version 1030767 (0.0005) [2023-12-27 12:47:51,151][20347] Updated weights for policy 0, policy_version 1031562 (0.0008) [2023-12-27 12:47:51,158][20381] Updated weights for policy 1, policy_version 1030777 (0.0008) [2023-12-27 12:47:51,212][20381] Updated weights for policy 1, policy_version 1030787 (0.0011) [2023-12-27 12:47:51,213][20347] Updated weights for policy 0, policy_version 1031572 (0.0007) [2023-12-27 12:47:51,275][20381] Updated weights for policy 1, policy_version 1030797 (0.0010) [2023-12-27 12:47:51,279][20347] Updated weights for policy 0, policy_version 1031582 (0.0007) [2023-12-27 12:47:51,337][20381] Updated weights for policy 1, policy_version 1030807 (0.0006) [2023-12-27 12:47:51,342][20347] Updated weights for policy 0, policy_version 1031592 (0.0007) [2023-12-27 12:47:52,027][20381] Updated weights for policy 1, policy_version 1030817 (0.0006) [2023-12-27 12:47:52,046][20347] Updated weights for policy 0, policy_version 1031602 (0.0007) [2023-12-27 12:47:52,087][20381] Updated weights for policy 1, policy_version 1030827 (0.0007) [2023-12-27 12:47:52,104][20347] Updated weights for policy 0, policy_version 1031612 (0.0006) [2023-12-27 12:47:52,146][20381] Updated weights for policy 1, policy_version 1030837 (0.0006) [2023-12-27 12:47:52,156][20347] Updated weights for policy 0, policy_version 1031622 (0.0006) [2023-12-27 12:47:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 528064512. Throughput: 0: 10240.9, 1: 10332.6. Samples: 528031220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:52,363][19170] Avg episode reward: [(0, '980.480'), (1, '686.060')] [2023-12-27 12:47:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001030840_263921664.pth... [2023-12-27 12:47:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001031624_264142848.pth... [2023-12-27 12:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001029624_263610368.pth [2023-12-27 12:47:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001030408_263831552.pth [2023-12-27 12:47:52,841][20381] Updated weights for policy 1, policy_version 1030847 (0.0006) [2023-12-27 12:47:52,886][20347] Updated weights for policy 0, policy_version 1031632 (0.0007) [2023-12-27 12:47:52,900][20381] Updated weights for policy 1, policy_version 1030857 (0.0006) [2023-12-27 12:47:52,940][20347] Updated weights for policy 0, policy_version 1031642 (0.0006) [2023-12-27 12:47:52,954][20381] Updated weights for policy 1, policy_version 1030867 (0.0005) [2023-12-27 12:47:52,994][20347] Updated weights for policy 0, policy_version 1031652 (0.0007) [2023-12-27 12:47:53,556][20381] Updated weights for policy 1, policy_version 1030877 (0.0008) [2023-12-27 12:47:53,604][20381] Updated weights for policy 1, policy_version 1030887 (0.0010) [2023-12-27 12:47:53,618][20347] Updated weights for policy 0, policy_version 1031662 (0.0007) [2023-12-27 12:47:53,650][20381] Updated weights for policy 1, policy_version 1030897 (0.0010) [2023-12-27 12:47:53,666][20347] Updated weights for policy 0, policy_version 1031672 (0.0006) [2023-12-27 12:47:53,721][20347] Updated weights for policy 0, policy_version 1031682 (0.0006) [2023-12-27 12:47:54,297][20347] Updated weights for policy 0, policy_version 1031692 (0.0007) [2023-12-27 12:47:54,351][20347] Updated weights for policy 0, policy_version 1031702 (0.0007) [2023-12-27 12:47:54,368][20381] Updated weights for policy 1, policy_version 1030907 (0.0010) [2023-12-27 12:47:54,407][20347] Updated weights for policy 0, policy_version 1031712 (0.0005) [2023-12-27 12:47:54,429][20381] Updated weights for policy 1, policy_version 1030917 (0.0010) [2023-12-27 12:47:54,478][20381] Updated weights for policy 1, policy_version 1030927 (0.0010) [2023-12-27 12:47:54,979][20347] Updated weights for policy 0, policy_version 1031722 (0.0006) [2023-12-27 12:47:55,037][20347] Updated weights for policy 0, policy_version 1031732 (0.0005) [2023-12-27 12:47:55,092][20347] Updated weights for policy 0, policy_version 1031742 (0.0009) [2023-12-27 12:47:55,148][20347] Updated weights for policy 0, policy_version 1031752 (0.0010) [2023-12-27 12:47:55,192][20381] Updated weights for policy 1, policy_version 1030937 (0.0011) [2023-12-27 12:47:55,251][20381] Updated weights for policy 1, policy_version 1030947 (0.0011) [2023-12-27 12:47:55,309][20381] Updated weights for policy 1, policy_version 1030957 (0.0011) [2023-12-27 12:47:55,369][20381] Updated weights for policy 1, policy_version 1030967 (0.0011) [2023-12-27 12:47:55,800][20347] Updated weights for policy 0, policy_version 1031762 (0.0011) [2023-12-27 12:47:55,848][20347] Updated weights for policy 0, policy_version 1031772 (0.0010) [2023-12-27 12:47:55,896][20347] Updated weights for policy 0, policy_version 1031782 (0.0010) [2023-12-27 12:47:56,057][20381] Updated weights for policy 1, policy_version 1030977 (0.0011) [2023-12-27 12:47:56,112][20381] Updated weights for policy 1, policy_version 1030987 (0.0011) [2023-12-27 12:47:56,162][20381] Updated weights for policy 1, policy_version 1030997 (0.0009) [2023-12-27 12:47:56,569][20347] Updated weights for policy 0, policy_version 1031792 (0.0010) [2023-12-27 12:47:56,623][20347] Updated weights for policy 0, policy_version 1031802 (0.0009) [2023-12-27 12:47:56,676][20347] Updated weights for policy 0, policy_version 1031812 (0.0008) [2023-12-27 12:47:56,839][20381] Updated weights for policy 1, policy_version 1031007 (0.0010) [2023-12-27 12:47:56,890][20381] Updated weights for policy 1, policy_version 1031017 (0.0010) [2023-12-27 12:47:56,934][20381] Updated weights for policy 1, policy_version 1031027 (0.0010) [2023-12-27 12:47:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 528162816. Throughput: 0: 10276.1, 1: 10298.7. Samples: 528155096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:47:57,363][19170] Avg episode reward: [(0, '970.740'), (1, '676.370')] [2023-12-27 12:47:57,365][20347] Updated weights for policy 0, policy_version 1031822 (0.0009) [2023-12-27 12:47:57,413][20347] Updated weights for policy 0, policy_version 1031832 (0.0008) [2023-12-27 12:47:57,467][20347] Updated weights for policy 0, policy_version 1031842 (0.0008) [2023-12-27 12:47:57,491][20291] KL-divergence is very high: 171.9273 [2023-12-27 12:47:57,656][20381] Updated weights for policy 1, policy_version 1031037 (0.0010) [2023-12-27 12:47:57,718][20381] Updated weights for policy 1, policy_version 1031047 (0.0011) [2023-12-27 12:47:57,766][20381] Updated weights for policy 1, policy_version 1031057 (0.0010) [2023-12-27 12:47:58,102][20291] KL-divergence is very high: 159.1000 [2023-12-27 12:47:58,122][20347] Updated weights for policy 0, policy_version 1031852 (0.0008) [2023-12-27 12:47:58,172][20347] Updated weights for policy 0, policy_version 1031862 (0.0010) [2023-12-27 12:47:58,235][20347] Updated weights for policy 0, policy_version 1031872 (0.0009) [2023-12-27 12:47:58,277][20291] KL-divergence is very high: 100.1172 [2023-12-27 12:47:58,478][20381] Updated weights for policy 1, policy_version 1031067 (0.0010) [2023-12-27 12:47:58,541][20381] Updated weights for policy 1, policy_version 1031077 (0.0011) [2023-12-27 12:47:58,615][20381] Updated weights for policy 1, policy_version 1031087 (0.0012) [2023-12-27 12:47:59,050][20347] Updated weights for policy 0, policy_version 1031882 (0.0008) [2023-12-27 12:47:59,116][20347] Updated weights for policy 0, policy_version 1031892 (0.0007) [2023-12-27 12:47:59,172][20347] Updated weights for policy 0, policy_version 1031902 (0.0007) [2023-12-27 12:47:59,231][20347] Updated weights for policy 0, policy_version 1031912 (0.0009) [2023-12-27 12:47:59,378][20381] Updated weights for policy 1, policy_version 1031097 (0.0010) [2023-12-27 12:47:59,442][20381] Updated weights for policy 1, policy_version 1031107 (0.0007) [2023-12-27 12:47:59,500][20381] Updated weights for policy 1, policy_version 1031117 (0.0008) [2023-12-27 12:47:59,560][20381] Updated weights for policy 1, policy_version 1031127 (0.0008) [2023-12-27 12:47:59,954][20347] Updated weights for policy 0, policy_version 1031922 (0.0008) [2023-12-27 12:48:00,015][20347] Updated weights for policy 0, policy_version 1031932 (0.0007) [2023-12-27 12:48:00,076][20347] Updated weights for policy 0, policy_version 1031942 (0.0007) [2023-12-27 12:48:00,268][20381] Updated weights for policy 1, policy_version 1031137 (0.0010) [2023-12-27 12:48:00,324][20381] Updated weights for policy 1, policy_version 1031147 (0.0010) [2023-12-27 12:48:00,380][20381] Updated weights for policy 1, policy_version 1031157 (0.0011) [2023-12-27 12:48:00,754][20347] Updated weights for policy 0, policy_version 1031952 (0.0009) [2023-12-27 12:48:00,799][20347] Updated weights for policy 0, policy_version 1031962 (0.0006) [2023-12-27 12:48:00,842][20347] Updated weights for policy 0, policy_version 1031972 (0.0007) [2023-12-27 12:48:00,996][20381] Updated weights for policy 1, policy_version 1031167 (0.0009) [2023-12-27 12:48:01,056][20381] Updated weights for policy 1, policy_version 1031177 (0.0009) [2023-12-27 12:48:01,126][20381] Updated weights for policy 1, policy_version 1031187 (0.0007) [2023-12-27 12:48:01,578][20347] Updated weights for policy 0, policy_version 1031982 (0.0008) [2023-12-27 12:48:01,637][20347] Updated weights for policy 0, policy_version 1031992 (0.0009) [2023-12-27 12:48:01,696][20347] Updated weights for policy 0, policy_version 1032002 (0.0009) [2023-12-27 12:48:01,762][20381] Updated weights for policy 1, policy_version 1031197 (0.0008) [2023-12-27 12:48:01,814][20381] Updated weights for policy 1, policy_version 1031207 (0.0007) [2023-12-27 12:48:01,869][20381] Updated weights for policy 1, policy_version 1031217 (0.0007) [2023-12-27 12:48:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 528261120. Throughput: 0: 10239.4, 1: 10249.3. Samples: 528274240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:02,363][19170] Avg episode reward: [(0, '941.640'), (1, '725.570')] [2023-12-27 12:48:02,380][20347] Updated weights for policy 0, policy_version 1032012 (0.0008) [2023-12-27 12:48:02,433][20347] Updated weights for policy 0, policy_version 1032022 (0.0007) [2023-12-27 12:48:02,493][20347] Updated weights for policy 0, policy_version 1032032 (0.0007) [2023-12-27 12:48:02,549][20381] Updated weights for policy 1, policy_version 1031227 (0.0007) [2023-12-27 12:48:02,598][20381] Updated weights for policy 1, policy_version 1031237 (0.0008) [2023-12-27 12:48:02,651][20381] Updated weights for policy 1, policy_version 1031247 (0.0007) [2023-12-27 12:48:03,135][20347] Updated weights for policy 0, policy_version 1032042 (0.0007) [2023-12-27 12:48:03,182][20347] Updated weights for policy 0, policy_version 1032052 (0.0006) [2023-12-27 12:48:03,233][20347] Updated weights for policy 0, policy_version 1032062 (0.0007) [2023-12-27 12:48:03,285][20347] Updated weights for policy 0, policy_version 1032072 (0.0007) [2023-12-27 12:48:03,361][20381] Updated weights for policy 1, policy_version 1031257 (0.0010) [2023-12-27 12:48:03,418][20381] Updated weights for policy 1, policy_version 1031267 (0.0009) [2023-12-27 12:48:03,479][20381] Updated weights for policy 1, policy_version 1031277 (0.0005) [2023-12-27 12:48:03,528][20381] Updated weights for policy 1, policy_version 1031287 (0.0005) [2023-12-27 12:48:04,029][20347] Updated weights for policy 0, policy_version 1032082 (0.0007) [2023-12-27 12:48:04,100][20347] Updated weights for policy 0, policy_version 1032092 (0.0006) [2023-12-27 12:48:04,112][20381] Updated weights for policy 1, policy_version 1031297 (0.0007) [2023-12-27 12:48:04,165][20347] Updated weights for policy 0, policy_version 1032102 (0.0006) [2023-12-27 12:48:04,172][20381] Updated weights for policy 1, policy_version 1031307 (0.0009) [2023-12-27 12:48:04,232][20381] Updated weights for policy 1, policy_version 1031317 (0.0009) [2023-12-27 12:48:04,831][20347] Updated weights for policy 0, policy_version 1032112 (0.0007) [2023-12-27 12:48:04,886][20347] Updated weights for policy 0, policy_version 1032123 (0.0009) [2023-12-27 12:48:04,900][20381] Updated weights for policy 1, policy_version 1031327 (0.0006) [2023-12-27 12:48:04,935][20347] Updated weights for policy 0, policy_version 1032133 (0.0008) [2023-12-27 12:48:04,946][20381] Updated weights for policy 1, policy_version 1031337 (0.0005) [2023-12-27 12:48:04,992][20381] Updated weights for policy 1, policy_version 1031347 (0.0007) [2023-12-27 12:48:05,554][20381] Updated weights for policy 1, policy_version 1031357 (0.0006) [2023-12-27 12:48:05,614][20381] Updated weights for policy 1, policy_version 1031367 (0.0005) [2023-12-27 12:48:05,667][20381] Updated weights for policy 1, policy_version 1031377 (0.0005) [2023-12-27 12:48:05,690][20347] Updated weights for policy 0, policy_version 1032143 (0.0006) [2023-12-27 12:48:05,761][20347] Updated weights for policy 0, policy_version 1032153 (0.0006) [2023-12-27 12:48:05,813][20347] Updated weights for policy 0, policy_version 1032163 (0.0006) [2023-12-27 12:48:06,342][20381] Updated weights for policy 1, policy_version 1031387 (0.0005) [2023-12-27 12:48:06,373][20347] Updated weights for policy 0, policy_version 1032173 (0.0005) [2023-12-27 12:48:06,400][20381] Updated weights for policy 1, policy_version 1031397 (0.0008) [2023-12-27 12:48:06,431][20347] Updated weights for policy 0, policy_version 1032183 (0.0006) [2023-12-27 12:48:06,460][20381] Updated weights for policy 1, policy_version 1031407 (0.0008) [2023-12-27 12:48:06,492][20347] Updated weights for policy 0, policy_version 1032193 (0.0006) [2023-12-27 12:48:07,139][20347] Updated weights for policy 0, policy_version 1032203 (0.0007) [2023-12-27 12:48:07,139][20381] Updated weights for policy 1, policy_version 1031417 (0.0008) [2023-12-27 12:48:07,189][20347] Updated weights for policy 0, policy_version 1032213 (0.0011) [2023-12-27 12:48:07,195][20381] Updated weights for policy 1, policy_version 1031427 (0.0006) [2023-12-27 12:48:07,245][20347] Updated weights for policy 0, policy_version 1032223 (0.0010) [2023-12-27 12:48:07,251][20381] Updated weights for policy 1, policy_version 1031437 (0.0005) [2023-12-27 12:48:07,306][20381] Updated weights for policy 1, policy_version 1031447 (0.0007) [2023-12-27 12:48:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 528375808. Throughput: 0: 10221.4, 1: 10276.7. Samples: 528336100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:07,363][19170] Avg episode reward: [(0, '951.390'), (1, '774.420')] [2023-12-27 12:48:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001031448_264077312.pth... [2023-12-27 12:48:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001032232_264298496.pth... [2023-12-27 12:48:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001030232_263766016.pth [2023-12-27 12:48:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001031016_263987200.pth [2023-12-27 12:48:07,929][20347] Updated weights for policy 0, policy_version 1032233 (0.0010) [2023-12-27 12:48:07,967][20381] Updated weights for policy 1, policy_version 1031457 (0.0007) [2023-12-27 12:48:07,987][20347] Updated weights for policy 0, policy_version 1032243 (0.0010) [2023-12-27 12:48:08,024][20381] Updated weights for policy 1, policy_version 1031467 (0.0006) [2023-12-27 12:48:08,040][20347] Updated weights for policy 0, policy_version 1032253 (0.0010) [2023-12-27 12:48:08,079][20381] Updated weights for policy 1, policy_version 1031477 (0.0006) [2023-12-27 12:48:08,099][20347] Updated weights for policy 0, policy_version 1032263 (0.0009) [2023-12-27 12:48:08,810][20347] Updated weights for policy 0, policy_version 1032273 (0.0011) [2023-12-27 12:48:08,815][20381] Updated weights for policy 1, policy_version 1031487 (0.0009) [2023-12-27 12:48:08,866][20347] Updated weights for policy 0, policy_version 1032283 (0.0010) [2023-12-27 12:48:08,876][20381] Updated weights for policy 1, policy_version 1031497 (0.0006) [2023-12-27 12:48:08,921][20347] Updated weights for policy 0, policy_version 1032293 (0.0010) [2023-12-27 12:48:08,932][20381] Updated weights for policy 1, policy_version 1031507 (0.0009) [2023-12-27 12:48:09,607][20347] Updated weights for policy 0, policy_version 1032303 (0.0009) [2023-12-27 12:48:09,663][20381] Updated weights for policy 1, policy_version 1031517 (0.0009) [2023-12-27 12:48:09,671][20347] Updated weights for policy 0, policy_version 1032313 (0.0008) [2023-12-27 12:48:09,723][20381] Updated weights for policy 1, policy_version 1031527 (0.0008) [2023-12-27 12:48:09,738][20347] Updated weights for policy 0, policy_version 1032323 (0.0009) [2023-12-27 12:48:09,787][20381] Updated weights for policy 1, policy_version 1031537 (0.0009) [2023-12-27 12:48:10,468][20347] Updated weights for policy 0, policy_version 1032333 (0.0008) [2023-12-27 12:48:10,478][20381] Updated weights for policy 1, policy_version 1031547 (0.0008) [2023-12-27 12:48:10,526][20347] Updated weights for policy 0, policy_version 1032343 (0.0006) [2023-12-27 12:48:10,543][20381] Updated weights for policy 1, policy_version 1031557 (0.0007) [2023-12-27 12:48:10,586][20347] Updated weights for policy 0, policy_version 1032353 (0.0006) [2023-12-27 12:48:10,603][20381] Updated weights for policy 1, policy_version 1031567 (0.0008) [2023-12-27 12:48:11,221][20347] Updated weights for policy 0, policy_version 1032363 (0.0006) [2023-12-27 12:48:11,290][20347] Updated weights for policy 0, policy_version 1032373 (0.0008) [2023-12-27 12:48:11,315][20381] Updated weights for policy 1, policy_version 1031577 (0.0007) [2023-12-27 12:48:11,359][20347] Updated weights for policy 0, policy_version 1032383 (0.0008) [2023-12-27 12:48:11,391][20381] Updated weights for policy 1, policy_version 1031587 (0.0010) [2023-12-27 12:48:11,455][20381] Updated weights for policy 1, policy_version 1031597 (0.0007) [2023-12-27 12:48:11,515][20381] Updated weights for policy 1, policy_version 1031607 (0.0008) [2023-12-27 12:48:12,073][20347] Updated weights for policy 0, policy_version 1032393 (0.0008) [2023-12-27 12:48:12,130][20347] Updated weights for policy 0, policy_version 1032403 (0.0007) [2023-12-27 12:48:12,186][20347] Updated weights for policy 0, policy_version 1032413 (0.0007) [2023-12-27 12:48:12,244][20347] Updated weights for policy 0, policy_version 1032423 (0.0009) [2023-12-27 12:48:12,259][20381] Updated weights for policy 1, policy_version 1031617 (0.0010) [2023-12-27 12:48:12,320][20381] Updated weights for policy 1, policy_version 1031627 (0.0009) [2023-12-27 12:48:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 528465920. Throughput: 0: 10210.7, 1: 10233.3. Samples: 528458736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:12,363][19170] Avg episode reward: [(0, '960.970'), (1, '745.070')] [2023-12-27 12:48:12,390][20381] Updated weights for policy 1, policy_version 1031637 (0.0010) [2023-12-27 12:48:12,868][20347] Updated weights for policy 0, policy_version 1032433 (0.0007) [2023-12-27 12:48:12,920][20347] Updated weights for policy 0, policy_version 1032443 (0.0007) [2023-12-27 12:48:12,979][20347] Updated weights for policy 0, policy_version 1032453 (0.0005) [2023-12-27 12:48:13,050][20381] Updated weights for policy 1, policy_version 1031647 (0.0010) [2023-12-27 12:48:13,111][20381] Updated weights for policy 1, policy_version 1031657 (0.0010) [2023-12-27 12:48:13,171][20381] Updated weights for policy 1, policy_version 1031667 (0.0007) [2023-12-27 12:48:13,505][20347] Updated weights for policy 0, policy_version 1032463 (0.0005) [2023-12-27 12:48:13,555][20347] Updated weights for policy 0, policy_version 1032473 (0.0005) [2023-12-27 12:48:13,606][20347] Updated weights for policy 0, policy_version 1032483 (0.0005) [2023-12-27 12:48:13,807][20381] Updated weights for policy 1, policy_version 1031677 (0.0008) [2023-12-27 12:48:13,871][20381] Updated weights for policy 1, policy_version 1031687 (0.0010) [2023-12-27 12:48:13,924][20381] Updated weights for policy 1, policy_version 1031697 (0.0010) [2023-12-27 12:48:14,222][20347] Updated weights for policy 0, policy_version 1032493 (0.0007) [2023-12-27 12:48:14,278][20347] Updated weights for policy 0, policy_version 1032503 (0.0006) [2023-12-27 12:48:14,324][20347] Updated weights for policy 0, policy_version 1032513 (0.0007) [2023-12-27 12:48:14,514][20381] Updated weights for policy 1, policy_version 1031707 (0.0010) [2023-12-27 12:48:14,573][20381] Updated weights for policy 1, policy_version 1031717 (0.0007) [2023-12-27 12:48:14,634][20381] Updated weights for policy 1, policy_version 1031727 (0.0005) [2023-12-27 12:48:14,984][20347] Updated weights for policy 0, policy_version 1032523 (0.0008) [2023-12-27 12:48:15,043][20347] Updated weights for policy 0, policy_version 1032533 (0.0006) [2023-12-27 12:48:15,097][20347] Updated weights for policy 0, policy_version 1032543 (0.0006) [2023-12-27 12:48:15,308][20381] Updated weights for policy 1, policy_version 1031737 (0.0005) [2023-12-27 12:48:15,368][20381] Updated weights for policy 1, policy_version 1031747 (0.0006) [2023-12-27 12:48:15,427][20381] Updated weights for policy 1, policy_version 1031757 (0.0008) [2023-12-27 12:48:15,490][20381] Updated weights for policy 1, policy_version 1031767 (0.0008) [2023-12-27 12:48:15,755][20347] Updated weights for policy 0, policy_version 1032553 (0.0006) [2023-12-27 12:48:15,810][20347] Updated weights for policy 0, policy_version 1032563 (0.0007) [2023-12-27 12:48:15,871][20347] Updated weights for policy 0, policy_version 1032573 (0.0007) [2023-12-27 12:48:15,925][20347] Updated weights for policy 0, policy_version 1032583 (0.0005) [2023-12-27 12:48:16,142][20381] Updated weights for policy 1, policy_version 1031777 (0.0007) [2023-12-27 12:48:16,200][20381] Updated weights for policy 1, policy_version 1031787 (0.0007) [2023-12-27 12:48:16,257][20381] Updated weights for policy 1, policy_version 1031797 (0.0008) [2023-12-27 12:48:16,534][20347] Updated weights for policy 0, policy_version 1032593 (0.0007) [2023-12-27 12:48:16,589][20347] Updated weights for policy 0, policy_version 1032603 (0.0008) [2023-12-27 12:48:16,635][20347] Updated weights for policy 0, policy_version 1032613 (0.0008) [2023-12-27 12:48:16,961][20381] Updated weights for policy 1, policy_version 1031807 (0.0008) [2023-12-27 12:48:17,013][20381] Updated weights for policy 1, policy_version 1031817 (0.0007) [2023-12-27 12:48:17,068][20381] Updated weights for policy 1, policy_version 1031827 (0.0007) [2023-12-27 12:48:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 528572416. Throughput: 0: 10290.0, 1: 10220.8. Samples: 528583888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:17,364][19170] Avg episode reward: [(0, '970.830'), (1, '823.490')] [2023-12-27 12:48:17,376][20347] Updated weights for policy 0, policy_version 1032623 (0.0008) [2023-12-27 12:48:17,425][20347] Updated weights for policy 0, policy_version 1032633 (0.0008) [2023-12-27 12:48:17,475][20347] Updated weights for policy 0, policy_version 1032643 (0.0008) [2023-12-27 12:48:17,651][20381] Updated weights for policy 1, policy_version 1031837 (0.0006) [2023-12-27 12:48:17,700][20381] Updated weights for policy 1, policy_version 1031847 (0.0008) [2023-12-27 12:48:17,753][20381] Updated weights for policy 1, policy_version 1031857 (0.0009) [2023-12-27 12:48:18,170][20347] Updated weights for policy 0, policy_version 1032653 (0.0010) [2023-12-27 12:48:18,219][20347] Updated weights for policy 0, policy_version 1032663 (0.0007) [2023-12-27 12:48:18,267][20347] Updated weights for policy 0, policy_version 1032673 (0.0008) [2023-12-27 12:48:18,392][20381] Updated weights for policy 1, policy_version 1031867 (0.0010) [2023-12-27 12:48:18,449][20381] Updated weights for policy 1, policy_version 1031877 (0.0009) [2023-12-27 12:48:18,512][20381] Updated weights for policy 1, policy_version 1031887 (0.0008) [2023-12-27 12:48:19,011][20347] Updated weights for policy 0, policy_version 1032683 (0.0008) [2023-12-27 12:48:19,072][20347] Updated weights for policy 0, policy_version 1032693 (0.0007) [2023-12-27 12:48:19,122][20347] Updated weights for policy 0, policy_version 1032703 (0.0006) [2023-12-27 12:48:19,206][20381] Updated weights for policy 1, policy_version 1031897 (0.0008) [2023-12-27 12:48:19,271][20381] Updated weights for policy 1, policy_version 1031907 (0.0008) [2023-12-27 12:48:19,327][20381] Updated weights for policy 1, policy_version 1031917 (0.0008) [2023-12-27 12:48:19,389][20381] Updated weights for policy 1, policy_version 1031927 (0.0008) [2023-12-27 12:48:19,804][20347] Updated weights for policy 0, policy_version 1032713 (0.0007) [2023-12-27 12:48:19,872][20347] Updated weights for policy 0, policy_version 1032723 (0.0009) [2023-12-27 12:48:19,935][20347] Updated weights for policy 0, policy_version 1032733 (0.0009) [2023-12-27 12:48:19,992][20347] Updated weights for policy 0, policy_version 1032743 (0.0009) [2023-12-27 12:48:20,207][20381] Updated weights for policy 1, policy_version 1031937 (0.0009) [2023-12-27 12:48:20,264][20381] Updated weights for policy 1, policy_version 1031947 (0.0009) [2023-12-27 12:48:20,326][20381] Updated weights for policy 1, policy_version 1031957 (0.0008) [2023-12-27 12:48:20,760][20347] Updated weights for policy 0, policy_version 1032753 (0.0008) [2023-12-27 12:48:20,822][20347] Updated weights for policy 0, policy_version 1032763 (0.0007) [2023-12-27 12:48:20,880][20347] Updated weights for policy 0, policy_version 1032773 (0.0007) [2023-12-27 12:48:21,074][20381] Updated weights for policy 1, policy_version 1031967 (0.0008) [2023-12-27 12:48:21,137][20381] Updated weights for policy 1, policy_version 1031977 (0.0008) [2023-12-27 12:48:21,202][20381] Updated weights for policy 1, policy_version 1031987 (0.0008) [2023-12-27 12:48:21,609][20347] Updated weights for policy 0, policy_version 1032783 (0.0009) [2023-12-27 12:48:21,673][20347] Updated weights for policy 0, policy_version 1032793 (0.0009) [2023-12-27 12:48:21,742][20347] Updated weights for policy 0, policy_version 1032803 (0.0008) [2023-12-27 12:48:21,971][20381] Updated weights for policy 1, policy_version 1031997 (0.0008) [2023-12-27 12:48:22,024][20381] Updated weights for policy 1, policy_version 1032007 (0.0009) [2023-12-27 12:48:22,078][20381] Updated weights for policy 1, policy_version 1032017 (0.0009) [2023-12-27 12:48:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 528670720. Throughput: 0: 10238.3, 1: 10210.0. Samples: 528643256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:22,363][19170] Avg episode reward: [(0, '980.610'), (1, '833.360')] [2023-12-27 12:48:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001032024_264224768.pth... [2023-12-27 12:48:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001032808_264445952.pth... [2023-12-27 12:48:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001030840_263921664.pth [2023-12-27 12:48:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001031624_264142848.pth [2023-12-27 12:48:22,467][20347] Updated weights for policy 0, policy_version 1032813 (0.0007) [2023-12-27 12:48:22,528][20347] Updated weights for policy 0, policy_version 1032823 (0.0008) [2023-12-27 12:48:22,596][20347] Updated weights for policy 0, policy_version 1032833 (0.0009) [2023-12-27 12:48:22,827][20381] Updated weights for policy 1, policy_version 1032027 (0.0009) [2023-12-27 12:48:22,892][20381] Updated weights for policy 1, policy_version 1032037 (0.0008) [2023-12-27 12:48:22,950][20381] Updated weights for policy 1, policy_version 1032047 (0.0007) [2023-12-27 12:48:23,275][20347] Updated weights for policy 0, policy_version 1032843 (0.0009) [2023-12-27 12:48:23,322][20347] Updated weights for policy 0, policy_version 1032853 (0.0010) [2023-12-27 12:48:23,371][20347] Updated weights for policy 0, policy_version 1032863 (0.0010) [2023-12-27 12:48:23,565][20381] Updated weights for policy 1, policy_version 1032057 (0.0007) [2023-12-27 12:48:23,627][20381] Updated weights for policy 1, policy_version 1032067 (0.0007) [2023-12-27 12:48:23,689][20381] Updated weights for policy 1, policy_version 1032077 (0.0007) [2023-12-27 12:48:23,747][20381] Updated weights for policy 1, policy_version 1032087 (0.0008) [2023-12-27 12:48:24,046][20347] Updated weights for policy 0, policy_version 1032873 (0.0010) [2023-12-27 12:48:24,094][20347] Updated weights for policy 0, policy_version 1032883 (0.0010) [2023-12-27 12:48:24,145][20347] Updated weights for policy 0, policy_version 1032893 (0.0006) [2023-12-27 12:48:24,201][20347] Updated weights for policy 0, policy_version 1032903 (0.0005) [2023-12-27 12:48:24,438][20381] Updated weights for policy 1, policy_version 1032097 (0.0007) [2023-12-27 12:48:24,495][20381] Updated weights for policy 1, policy_version 1032107 (0.0007) [2023-12-27 12:48:24,559][20381] Updated weights for policy 1, policy_version 1032117 (0.0006) [2023-12-27 12:48:24,814][20347] Updated weights for policy 0, policy_version 1032913 (0.0009) [2023-12-27 12:48:24,874][20347] Updated weights for policy 0, policy_version 1032923 (0.0005) [2023-12-27 12:48:24,928][20347] Updated weights for policy 0, policy_version 1032933 (0.0010) [2023-12-27 12:48:25,106][20381] Updated weights for policy 1, policy_version 1032127 (0.0007) [2023-12-27 12:48:25,163][20381] Updated weights for policy 1, policy_version 1032137 (0.0009) [2023-12-27 12:48:25,223][20381] Updated weights for policy 1, policy_version 1032147 (0.0007) [2023-12-27 12:48:25,511][20347] Updated weights for policy 0, policy_version 1032943 (0.0010) [2023-12-27 12:48:25,566][20347] Updated weights for policy 0, policy_version 1032953 (0.0010) [2023-12-27 12:48:25,620][20347] Updated weights for policy 0, policy_version 1032963 (0.0009) [2023-12-27 12:48:25,922][20381] Updated weights for policy 1, policy_version 1032157 (0.0008) [2023-12-27 12:48:25,975][20381] Updated weights for policy 1, policy_version 1032167 (0.0007) [2023-12-27 12:48:26,027][20381] Updated weights for policy 1, policy_version 1032177 (0.0008) [2023-12-27 12:48:26,241][20347] Updated weights for policy 0, policy_version 1032973 (0.0008) [2023-12-27 12:48:26,297][20347] Updated weights for policy 0, policy_version 1032983 (0.0010) [2023-12-27 12:48:26,357][20347] Updated weights for policy 0, policy_version 1032993 (0.0009) [2023-12-27 12:48:26,711][20381] Updated weights for policy 1, policy_version 1032187 (0.0008) [2023-12-27 12:48:26,765][20381] Updated weights for policy 1, policy_version 1032197 (0.0008) [2023-12-27 12:48:26,828][20381] Updated weights for policy 1, policy_version 1032207 (0.0008) [2023-12-27 12:48:26,991][20347] Updated weights for policy 0, policy_version 1033003 (0.0009) [2023-12-27 12:48:27,044][20347] Updated weights for policy 0, policy_version 1033013 (0.0005) [2023-12-27 12:48:27,095][20347] Updated weights for policy 0, policy_version 1033023 (0.0005) [2023-12-27 12:48:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 528777216. Throughput: 0: 10232.2, 1: 10232.0. Samples: 528764916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '725.560')] [2023-12-27 12:48:27,504][20381] Updated weights for policy 1, policy_version 1032217 (0.0006) [2023-12-27 12:48:27,561][20381] Updated weights for policy 1, policy_version 1032227 (0.0007) [2023-12-27 12:48:27,611][20381] Updated weights for policy 1, policy_version 1032237 (0.0007) [2023-12-27 12:48:27,668][20381] Updated weights for policy 1, policy_version 1032247 (0.0008) [2023-12-27 12:48:27,697][20347] Updated weights for policy 0, policy_version 1033033 (0.0006) [2023-12-27 12:48:27,758][20347] Updated weights for policy 0, policy_version 1033043 (0.0010) [2023-12-27 12:48:27,815][20347] Updated weights for policy 0, policy_version 1033053 (0.0009) [2023-12-27 12:48:27,866][20347] Updated weights for policy 0, policy_version 1033063 (0.0010) [2023-12-27 12:48:28,301][20381] Updated weights for policy 1, policy_version 1032257 (0.0006) [2023-12-27 12:48:28,365][20381] Updated weights for policy 1, policy_version 1032267 (0.0007) [2023-12-27 12:48:28,424][20381] Updated weights for policy 1, policy_version 1032277 (0.0008) [2023-12-27 12:48:28,480][20347] Updated weights for policy 0, policy_version 1033073 (0.0010) [2023-12-27 12:48:28,538][20347] Updated weights for policy 0, policy_version 1033083 (0.0006) [2023-12-27 12:48:28,597][20347] Updated weights for policy 0, policy_version 1033093 (0.0005) [2023-12-27 12:48:29,105][20381] Updated weights for policy 1, policy_version 1032287 (0.0008) [2023-12-27 12:48:29,150][20381] Updated weights for policy 1, policy_version 1032297 (0.0007) [2023-12-27 12:48:29,195][20381] Updated weights for policy 1, policy_version 1032307 (0.0008) [2023-12-27 12:48:29,253][20347] Updated weights for policy 0, policy_version 1033103 (0.0008) [2023-12-27 12:48:29,316][20347] Updated weights for policy 0, policy_version 1033113 (0.0006) [2023-12-27 12:48:29,384][20347] Updated weights for policy 0, policy_version 1033123 (0.0008) [2023-12-27 12:48:29,834][20381] Updated weights for policy 1, policy_version 1032317 (0.0008) [2023-12-27 12:48:29,893][20381] Updated weights for policy 1, policy_version 1032327 (0.0008) [2023-12-27 12:48:29,956][20381] Updated weights for policy 1, policy_version 1032337 (0.0008) [2023-12-27 12:48:29,980][20347] Updated weights for policy 0, policy_version 1033133 (0.0006) [2023-12-27 12:48:30,038][20347] Updated weights for policy 0, policy_version 1033143 (0.0008) [2023-12-27 12:48:30,095][20347] Updated weights for policy 0, policy_version 1033153 (0.0007) [2023-12-27 12:48:30,610][20381] Updated weights for policy 1, policy_version 1032347 (0.0008) [2023-12-27 12:48:30,672][20381] Updated weights for policy 1, policy_version 1032357 (0.0008) [2023-12-27 12:48:30,733][20381] Updated weights for policy 1, policy_version 1032367 (0.0008) [2023-12-27 12:48:30,741][20347] Updated weights for policy 0, policy_version 1033163 (0.0007) [2023-12-27 12:48:30,809][20347] Updated weights for policy 0, policy_version 1033173 (0.0008) [2023-12-27 12:48:30,868][20347] Updated weights for policy 0, policy_version 1033183 (0.0008) [2023-12-27 12:48:31,424][20381] Updated weights for policy 1, policy_version 1032377 (0.0006) [2023-12-27 12:48:31,484][20381] Updated weights for policy 1, policy_version 1032387 (0.0006) [2023-12-27 12:48:31,546][20381] Updated weights for policy 1, policy_version 1032397 (0.0010) [2023-12-27 12:48:31,555][20347] Updated weights for policy 0, policy_version 1033193 (0.0007) [2023-12-27 12:48:31,603][20381] Updated weights for policy 1, policy_version 1032407 (0.0011) [2023-12-27 12:48:31,610][20347] Updated weights for policy 0, policy_version 1033203 (0.0006) [2023-12-27 12:48:31,673][20347] Updated weights for policy 0, policy_version 1033213 (0.0007) [2023-12-27 12:48:31,738][20347] Updated weights for policy 0, policy_version 1033223 (0.0006) [2023-12-27 12:48:32,292][20381] Updated weights for policy 1, policy_version 1032417 (0.0011) [2023-12-27 12:48:32,337][20347] Updated weights for policy 0, policy_version 1033233 (0.0006) [2023-12-27 12:48:32,344][20381] Updated weights for policy 1, policy_version 1032427 (0.0010) [2023-12-27 12:48:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.8, 300 sec: 20493.9). Total num frames: 528875520. Throughput: 0: 10283.5, 1: 10222.8. Samples: 528891820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:32,363][19170] Avg episode reward: [(0, '970.810'), (1, '686.530')] [2023-12-27 12:48:32,400][20347] Updated weights for policy 0, policy_version 1033243 (0.0008) [2023-12-27 12:48:32,407][20381] Updated weights for policy 1, policy_version 1032437 (0.0009) [2023-12-27 12:48:32,458][20347] Updated weights for policy 0, policy_version 1033253 (0.0008) [2023-12-27 12:48:33,034][20347] Updated weights for policy 0, policy_version 1033263 (0.0007) [2023-12-27 12:48:33,079][20347] Updated weights for policy 0, policy_version 1033273 (0.0006) [2023-12-27 12:48:33,096][20381] Updated weights for policy 1, policy_version 1032447 (0.0007) [2023-12-27 12:48:33,127][20347] Updated weights for policy 0, policy_version 1033283 (0.0006) [2023-12-27 12:48:33,147][20381] Updated weights for policy 1, policy_version 1032457 (0.0005) [2023-12-27 12:48:33,202][20381] Updated weights for policy 1, policy_version 1032467 (0.0007) [2023-12-27 12:48:33,842][20347] Updated weights for policy 0, policy_version 1033293 (0.0007) [2023-12-27 12:48:33,845][20381] Updated weights for policy 1, policy_version 1032477 (0.0006) [2023-12-27 12:48:33,891][20347] Updated weights for policy 0, policy_version 1033303 (0.0007) [2023-12-27 12:48:33,898][20381] Updated weights for policy 1, policy_version 1032487 (0.0005) [2023-12-27 12:48:33,945][20347] Updated weights for policy 0, policy_version 1033313 (0.0007) [2023-12-27 12:48:33,951][20381] Updated weights for policy 1, policy_version 1032497 (0.0005) [2023-12-27 12:48:34,572][20381] Updated weights for policy 1, policy_version 1032507 (0.0006) [2023-12-27 12:48:34,636][20381] Updated weights for policy 1, policy_version 1032517 (0.0010) [2023-12-27 12:48:34,687][20347] Updated weights for policy 0, policy_version 1033323 (0.0007) [2023-12-27 12:48:34,697][20381] Updated weights for policy 1, policy_version 1032527 (0.0011) [2023-12-27 12:48:34,744][20347] Updated weights for policy 0, policy_version 1033333 (0.0009) [2023-12-27 12:48:34,802][20347] Updated weights for policy 0, policy_version 1033343 (0.0007) [2023-12-27 12:48:35,385][20381] Updated weights for policy 1, policy_version 1032537 (0.0010) [2023-12-27 12:48:35,442][20381] Updated weights for policy 1, policy_version 1032547 (0.0008) [2023-12-27 12:48:35,488][20347] Updated weights for policy 0, policy_version 1033353 (0.0007) [2023-12-27 12:48:35,497][20381] Updated weights for policy 1, policy_version 1032557 (0.0008) [2023-12-27 12:48:35,537][20347] Updated weights for policy 0, policy_version 1033363 (0.0005) [2023-12-27 12:48:35,547][20381] Updated weights for policy 1, policy_version 1032567 (0.0005) [2023-12-27 12:48:35,590][20347] Updated weights for policy 0, policy_version 1033373 (0.0006) [2023-12-27 12:48:35,635][20347] Updated weights for policy 0, policy_version 1033383 (0.0005) [2023-12-27 12:48:36,225][20347] Updated weights for policy 0, policy_version 1033393 (0.0008) [2023-12-27 12:48:36,278][20381] Updated weights for policy 1, policy_version 1032577 (0.0005) [2023-12-27 12:48:36,286][20347] Updated weights for policy 0, policy_version 1033403 (0.0009) [2023-12-27 12:48:36,341][20381] Updated weights for policy 1, policy_version 1032587 (0.0006) [2023-12-27 12:48:36,344][20347] Updated weights for policy 0, policy_version 1033413 (0.0008) [2023-12-27 12:48:36,403][20381] Updated weights for policy 1, policy_version 1032597 (0.0006) [2023-12-27 12:48:37,005][20347] Updated weights for policy 0, policy_version 1033423 (0.0008) [2023-12-27 12:48:37,031][20381] Updated weights for policy 1, policy_version 1032607 (0.0006) [2023-12-27 12:48:37,052][20347] Updated weights for policy 0, policy_version 1033433 (0.0006) [2023-12-27 12:48:37,085][20381] Updated weights for policy 1, policy_version 1032617 (0.0006) [2023-12-27 12:48:37,100][20347] Updated weights for policy 0, policy_version 1033443 (0.0007) [2023-12-27 12:48:37,133][20381] Updated weights for policy 1, policy_version 1032627 (0.0005) [2023-12-27 12:48:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 528990208. Throughput: 0: 10312.0, 1: 10199.8. Samples: 528954248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:37,363][19170] Avg episode reward: [(0, '970.830'), (1, '735.470')] [2023-12-27 12:48:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001032632_264380416.pth... [2023-12-27 12:48:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001033448_264609792.pth... [2023-12-27 12:48:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001032232_264298496.pth [2023-12-27 12:48:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001031448_264077312.pth [2023-12-27 12:48:37,755][20381] Updated weights for policy 1, policy_version 1032637 (0.0007) [2023-12-27 12:48:37,776][20347] Updated weights for policy 0, policy_version 1033453 (0.0007) [2023-12-27 12:48:37,811][20381] Updated weights for policy 1, policy_version 1032647 (0.0007) [2023-12-27 12:48:37,825][20347] Updated weights for policy 0, policy_version 1033463 (0.0008) [2023-12-27 12:48:37,862][20381] Updated weights for policy 1, policy_version 1032657 (0.0005) [2023-12-27 12:48:37,879][20347] Updated weights for policy 0, policy_version 1033473 (0.0008) [2023-12-27 12:48:38,511][20381] Updated weights for policy 1, policy_version 1032667 (0.0007) [2023-12-27 12:48:38,536][20347] Updated weights for policy 0, policy_version 1033483 (0.0007) [2023-12-27 12:48:38,557][20381] Updated weights for policy 1, policy_version 1032677 (0.0010) [2023-12-27 12:48:38,605][20347] Updated weights for policy 0, policy_version 1033493 (0.0008) [2023-12-27 12:48:38,609][20381] Updated weights for policy 1, policy_version 1032687 (0.0009) [2023-12-27 12:48:38,672][20347] Updated weights for policy 0, policy_version 1033503 (0.0008) [2023-12-27 12:48:39,218][20381] Updated weights for policy 1, policy_version 1032697 (0.0006) [2023-12-27 12:48:39,285][20381] Updated weights for policy 1, policy_version 1032707 (0.0009) [2023-12-27 12:48:39,324][20347] Updated weights for policy 0, policy_version 1033513 (0.0007) [2023-12-27 12:48:39,344][20381] Updated weights for policy 1, policy_version 1032717 (0.0010) [2023-12-27 12:48:39,394][20347] Updated weights for policy 0, policy_version 1033523 (0.0008) [2023-12-27 12:48:39,417][20381] Updated weights for policy 1, policy_version 1032727 (0.0008) [2023-12-27 12:48:39,460][20347] Updated weights for policy 0, policy_version 1033533 (0.0007) [2023-12-27 12:48:39,512][20347] Updated weights for policy 0, policy_version 1033543 (0.0009) [2023-12-27 12:48:40,143][20381] Updated weights for policy 1, policy_version 1032737 (0.0008) [2023-12-27 12:48:40,204][20381] Updated weights for policy 1, policy_version 1032747 (0.0008) [2023-12-27 12:48:40,268][20381] Updated weights for policy 1, policy_version 1032757 (0.0008) [2023-12-27 12:48:40,280][20347] Updated weights for policy 0, policy_version 1033553 (0.0006) [2023-12-27 12:48:40,341][20347] Updated weights for policy 0, policy_version 1033563 (0.0007) [2023-12-27 12:48:40,405][20347] Updated weights for policy 0, policy_version 1033573 (0.0008) [2023-12-27 12:48:40,987][20381] Updated weights for policy 1, policy_version 1032767 (0.0008) [2023-12-27 12:48:41,058][20381] Updated weights for policy 1, policy_version 1032777 (0.0008) [2023-12-27 12:48:41,121][20381] Updated weights for policy 1, policy_version 1032787 (0.0006) [2023-12-27 12:48:41,139][20347] Updated weights for policy 0, policy_version 1033583 (0.0008) [2023-12-27 12:48:41,208][20347] Updated weights for policy 0, policy_version 1033593 (0.0008) [2023-12-27 12:48:41,268][20347] Updated weights for policy 0, policy_version 1033603 (0.0007) [2023-12-27 12:48:41,903][20381] Updated weights for policy 1, policy_version 1032797 (0.0008) [2023-12-27 12:48:41,965][20381] Updated weights for policy 1, policy_version 1032807 (0.0008) [2023-12-27 12:48:42,026][20381] Updated weights for policy 1, policy_version 1032817 (0.0008) [2023-12-27 12:48:42,040][20347] Updated weights for policy 0, policy_version 1033613 (0.0007) [2023-12-27 12:48:42,095][20347] Updated weights for policy 0, policy_version 1033623 (0.0006) [2023-12-27 12:48:42,149][20347] Updated weights for policy 0, policy_version 1033633 (0.0009) [2023-12-27 12:48:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 529088512. Throughput: 0: 10304.1, 1: 10190.2. Samples: 529077340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:42,363][19170] Avg episode reward: [(0, '980.570'), (1, '764.860')] [2023-12-27 12:48:42,697][20381] Updated weights for policy 1, policy_version 1032827 (0.0007) [2023-12-27 12:48:42,748][20381] Updated weights for policy 1, policy_version 1032837 (0.0008) [2023-12-27 12:48:42,808][20381] Updated weights for policy 1, policy_version 1032847 (0.0009) [2023-12-27 12:48:42,854][20347] Updated weights for policy 0, policy_version 1033643 (0.0009) [2023-12-27 12:48:42,913][20347] Updated weights for policy 0, policy_version 1033653 (0.0007) [2023-12-27 12:48:42,975][20347] Updated weights for policy 0, policy_version 1033663 (0.0008) [2023-12-27 12:48:43,391][20381] Updated weights for policy 1, policy_version 1032857 (0.0008) [2023-12-27 12:48:43,443][20381] Updated weights for policy 1, policy_version 1032867 (0.0006) [2023-12-27 12:48:43,490][20381] Updated weights for policy 1, policy_version 1032877 (0.0007) [2023-12-27 12:48:43,544][20381] Updated weights for policy 1, policy_version 1032887 (0.0007) [2023-12-27 12:48:43,648][20347] Updated weights for policy 0, policy_version 1033673 (0.0008) [2023-12-27 12:48:43,704][20347] Updated weights for policy 0, policy_version 1033683 (0.0007) [2023-12-27 12:48:43,752][20347] Updated weights for policy 0, policy_version 1033693 (0.0007) [2023-12-27 12:48:43,799][20347] Updated weights for policy 0, policy_version 1033703 (0.0007) [2023-12-27 12:48:44,252][20381] Updated weights for policy 1, policy_version 1032897 (0.0009) [2023-12-27 12:48:44,296][20381] Updated weights for policy 1, policy_version 1032907 (0.0005) [2023-12-27 12:48:44,343][20381] Updated weights for policy 1, policy_version 1032917 (0.0005) [2023-12-27 12:48:44,512][20347] Updated weights for policy 0, policy_version 1033713 (0.0007) [2023-12-27 12:48:44,569][20347] Updated weights for policy 0, policy_version 1033723 (0.0007) [2023-12-27 12:48:44,627][20347] Updated weights for policy 0, policy_version 1033733 (0.0007) [2023-12-27 12:48:45,018][20381] Updated weights for policy 1, policy_version 1032927 (0.0008) [2023-12-27 12:48:45,078][20381] Updated weights for policy 1, policy_version 1032937 (0.0008) [2023-12-27 12:48:45,138][20381] Updated weights for policy 1, policy_version 1032947 (0.0008) [2023-12-27 12:48:45,283][20347] Updated weights for policy 0, policy_version 1033743 (0.0006) [2023-12-27 12:48:45,297][20291] KL-divergence is very high: 117.3743 [2023-12-27 12:48:45,346][20347] Updated weights for policy 0, policy_version 1033753 (0.0007) [2023-12-27 12:48:45,396][20291] KL-divergence is very high: 110.1771 [2023-12-27 12:48:45,410][20347] Updated weights for policy 0, policy_version 1033763 (0.0006) [2023-12-27 12:48:45,776][20381] Updated weights for policy 1, policy_version 1032957 (0.0007) [2023-12-27 12:48:45,825][20381] Updated weights for policy 1, policy_version 1032967 (0.0005) [2023-12-27 12:48:45,876][20381] Updated weights for policy 1, policy_version 1032977 (0.0006) [2023-12-27 12:48:45,976][20347] Updated weights for policy 0, policy_version 1033773 (0.0008) [2023-12-27 12:48:46,026][20347] Updated weights for policy 0, policy_version 1033783 (0.0008) [2023-12-27 12:48:46,072][20347] Updated weights for policy 0, policy_version 1033793 (0.0005) [2023-12-27 12:48:46,521][20381] Updated weights for policy 1, policy_version 1032987 (0.0008) [2023-12-27 12:48:46,568][20381] Updated weights for policy 1, policy_version 1032997 (0.0008) [2023-12-27 12:48:46,618][20381] Updated weights for policy 1, policy_version 1033007 (0.0008) [2023-12-27 12:48:46,717][20347] Updated weights for policy 0, policy_version 1033803 (0.0005) [2023-12-27 12:48:46,774][20347] Updated weights for policy 0, policy_version 1033813 (0.0006) [2023-12-27 12:48:46,830][20347] Updated weights for policy 0, policy_version 1033823 (0.0007) [2023-12-27 12:48:47,237][20381] Updated weights for policy 1, policy_version 1033017 (0.0007) [2023-12-27 12:48:47,291][20381] Updated weights for policy 1, policy_version 1033027 (0.0005) [2023-12-27 12:48:47,340][20381] Updated weights for policy 1, policy_version 1033037 (0.0005) [2023-12-27 12:48:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 529186816. Throughput: 0: 10365.6, 1: 10258.9. Samples: 529202344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:47,363][19170] Avg episode reward: [(0, '980.500'), (1, '647.250')] [2023-12-27 12:48:47,389][20381] Updated weights for policy 1, policy_version 1033047 (0.0007) [2023-12-27 12:48:47,498][20347] Updated weights for policy 0, policy_version 1033833 (0.0008) [2023-12-27 12:48:47,546][20347] Updated weights for policy 0, policy_version 1033843 (0.0006) [2023-12-27 12:48:47,600][20347] Updated weights for policy 0, policy_version 1033853 (0.0006) [2023-12-27 12:48:47,666][20347] Updated weights for policy 0, policy_version 1033863 (0.0005) [2023-12-27 12:48:48,046][20381] Updated weights for policy 1, policy_version 1033057 (0.0009) [2023-12-27 12:48:48,105][20381] Updated weights for policy 1, policy_version 1033067 (0.0011) [2023-12-27 12:48:48,151][20381] Updated weights for policy 1, policy_version 1033077 (0.0008) [2023-12-27 12:48:48,324][20347] Updated weights for policy 0, policy_version 1033873 (0.0008) [2023-12-27 12:48:48,381][20347] Updated weights for policy 0, policy_version 1033883 (0.0007) [2023-12-27 12:48:48,434][20347] Updated weights for policy 0, policy_version 1033893 (0.0006) [2023-12-27 12:48:48,831][20381] Updated weights for policy 1, policy_version 1033087 (0.0007) [2023-12-27 12:48:48,881][20381] Updated weights for policy 1, policy_version 1033097 (0.0008) [2023-12-27 12:48:48,931][20381] Updated weights for policy 1, policy_version 1033107 (0.0008) [2023-12-27 12:48:49,125][20347] Updated weights for policy 0, policy_version 1033903 (0.0008) [2023-12-27 12:48:49,173][20347] Updated weights for policy 0, policy_version 1033913 (0.0007) [2023-12-27 12:48:49,233][20347] Updated weights for policy 0, policy_version 1033923 (0.0008) [2023-12-27 12:48:49,653][20381] Updated weights for policy 1, policy_version 1033117 (0.0008) [2023-12-27 12:48:49,713][20381] Updated weights for policy 1, policy_version 1033127 (0.0008) [2023-12-27 12:48:49,779][20381] Updated weights for policy 1, policy_version 1033137 (0.0008) [2023-12-27 12:48:49,879][20347] Updated weights for policy 0, policy_version 1033933 (0.0006) [2023-12-27 12:48:49,943][20347] Updated weights for policy 0, policy_version 1033943 (0.0007) [2023-12-27 12:48:49,993][20347] Updated weights for policy 0, policy_version 1033953 (0.0005) [2023-12-27 12:48:50,551][20381] Updated weights for policy 1, policy_version 1033147 (0.0007) [2023-12-27 12:48:50,618][20381] Updated weights for policy 1, policy_version 1033157 (0.0008) [2023-12-27 12:48:50,683][20347] Updated weights for policy 0, policy_version 1033963 (0.0007) [2023-12-27 12:48:50,687][20381] Updated weights for policy 1, policy_version 1033167 (0.0008) [2023-12-27 12:48:50,739][20347] Updated weights for policy 0, policy_version 1033973 (0.0008) [2023-12-27 12:48:50,796][20347] Updated weights for policy 0, policy_version 1033983 (0.0007) [2023-12-27 12:48:51,476][20381] Updated weights for policy 1, policy_version 1033177 (0.0008) [2023-12-27 12:48:51,535][20381] Updated weights for policy 1, policy_version 1033187 (0.0008) [2023-12-27 12:48:51,550][20347] Updated weights for policy 0, policy_version 1033993 (0.0007) [2023-12-27 12:48:51,589][20381] Updated weights for policy 1, policy_version 1033197 (0.0007) [2023-12-27 12:48:51,602][20347] Updated weights for policy 0, policy_version 1034003 (0.0006) [2023-12-27 12:48:51,647][20381] Updated weights for policy 1, policy_version 1033207 (0.0007) [2023-12-27 12:48:51,666][20347] Updated weights for policy 0, policy_version 1034013 (0.0009) [2023-12-27 12:48:51,713][20347] Updated weights for policy 0, policy_version 1034023 (0.0008) [2023-12-27 12:48:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 529285120. Throughput: 0: 10374.6, 1: 10219.0. Samples: 529262812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:52,363][19170] Avg episode reward: [(0, '931.800'), (1, '696.390')] [2023-12-27 12:48:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001034024_264757248.pth... [2023-12-27 12:48:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001032808_264445952.pth [2023-12-27 12:48:52,397][20381] Updated weights for policy 1, policy_version 1033217 (0.0010) [2023-12-27 12:48:52,455][20381] Updated weights for policy 1, policy_version 1033227 (0.0010) [2023-12-27 12:48:52,472][20347] Updated weights for policy 0, policy_version 1034033 (0.0006) [2023-12-27 12:48:52,514][20381] Updated weights for policy 1, policy_version 1033237 (0.0011) [2023-12-27 12:48:52,528][20347] Updated weights for policy 0, policy_version 1034043 (0.0005) [2023-12-27 12:48:52,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001033240_264536064.pth... [2023-12-27 12:48:52,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001032024_264224768.pth [2023-12-27 12:48:52,586][20347] Updated weights for policy 0, policy_version 1034053 (0.0008) [2023-12-27 12:48:53,191][20381] Updated weights for policy 1, policy_version 1033247 (0.0007) [2023-12-27 12:48:53,248][20381] Updated weights for policy 1, policy_version 1033257 (0.0005) [2023-12-27 12:48:53,296][20381] Updated weights for policy 1, policy_version 1033267 (0.0005) [2023-12-27 12:48:53,298][20347] Updated weights for policy 0, policy_version 1034063 (0.0007) [2023-12-27 12:48:53,345][20347] Updated weights for policy 0, policy_version 1034073 (0.0007) [2023-12-27 12:48:53,401][20347] Updated weights for policy 0, policy_version 1034083 (0.0007) [2023-12-27 12:48:53,915][20381] Updated weights for policy 1, policy_version 1033277 (0.0005) [2023-12-27 12:48:53,971][20381] Updated weights for policy 1, policy_version 1033287 (0.0006) [2023-12-27 12:48:54,026][20381] Updated weights for policy 1, policy_version 1033297 (0.0007) [2023-12-27 12:48:54,123][20347] Updated weights for policy 0, policy_version 1034093 (0.0008) [2023-12-27 12:48:54,186][20347] Updated weights for policy 0, policy_version 1034103 (0.0008) [2023-12-27 12:48:54,245][20347] Updated weights for policy 0, policy_version 1034113 (0.0007) [2023-12-27 12:48:54,621][20381] Updated weights for policy 1, policy_version 1033307 (0.0008) [2023-12-27 12:48:54,674][20381] Updated weights for policy 1, policy_version 1033317 (0.0008) [2023-12-27 12:48:54,735][20381] Updated weights for policy 1, policy_version 1033327 (0.0008) [2023-12-27 12:48:54,953][20347] Updated weights for policy 0, policy_version 1034123 (0.0007) [2023-12-27 12:48:54,980][20291] KL-divergence is very high: 343.0135 [2023-12-27 12:48:54,987][20291] KL-divergence is very high: 297.1996 [2023-12-27 12:48:55,012][20347] Updated weights for policy 0, policy_version 1034133 (0.0008) [2023-12-27 12:48:55,068][20347] Updated weights for policy 0, policy_version 1034143 (0.0008) [2023-12-27 12:48:55,073][20291] KL-divergence is very high: 120.4326 [2023-12-27 12:48:55,079][20291] KL-divergence is very high: 103.1992 [2023-12-27 12:48:55,354][20381] Updated weights for policy 1, policy_version 1033337 (0.0007) [2023-12-27 12:48:55,407][20381] Updated weights for policy 1, policy_version 1033347 (0.0005) [2023-12-27 12:48:55,463][20381] Updated weights for policy 1, policy_version 1033357 (0.0005) [2023-12-27 12:48:55,522][20381] Updated weights for policy 1, policy_version 1033367 (0.0005) [2023-12-27 12:48:55,830][20347] Updated weights for policy 0, policy_version 1034153 (0.0007) [2023-12-27 12:48:55,880][20347] Updated weights for policy 0, policy_version 1034163 (0.0010) [2023-12-27 12:48:55,931][20347] Updated weights for policy 0, policy_version 1034173 (0.0009) [2023-12-27 12:48:55,980][20347] Updated weights for policy 0, policy_version 1034183 (0.0007) [2023-12-27 12:48:56,044][20381] Updated weights for policy 1, policy_version 1033377 (0.0005) [2023-12-27 12:48:56,102][20381] Updated weights for policy 1, policy_version 1033387 (0.0005) [2023-12-27 12:48:56,156][20381] Updated weights for policy 1, policy_version 1033397 (0.0007) [2023-12-27 12:48:56,704][20381] Updated weights for policy 1, policy_version 1033407 (0.0007) [2023-12-27 12:48:56,749][20347] Updated weights for policy 0, policy_version 1034193 (0.0008) [2023-12-27 12:48:56,757][20381] Updated weights for policy 1, policy_version 1033417 (0.0005) [2023-12-27 12:48:56,795][20347] Updated weights for policy 0, policy_version 1034203 (0.0007) [2023-12-27 12:48:56,806][20381] Updated weights for policy 1, policy_version 1033427 (0.0005) [2023-12-27 12:48:56,852][20347] Updated weights for policy 0, policy_version 1034213 (0.0007) [2023-12-27 12:48:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 529391616. Throughput: 0: 10270.3, 1: 10274.6. Samples: 529383256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:48:57,363][19170] Avg episode reward: [(0, '941.620'), (1, '803.890')] [2023-12-27 12:48:57,376][20381] Updated weights for policy 1, policy_version 1033437 (0.0008) [2023-12-27 12:48:57,437][20381] Updated weights for policy 1, policy_version 1033447 (0.0010) [2023-12-27 12:48:57,494][20381] Updated weights for policy 1, policy_version 1033457 (0.0010) [2023-12-27 12:48:57,593][20347] Updated weights for policy 0, policy_version 1034223 (0.0006) [2023-12-27 12:48:57,646][20347] Updated weights for policy 0, policy_version 1034233 (0.0007) [2023-12-27 12:48:57,700][20347] Updated weights for policy 0, policy_version 1034243 (0.0007) [2023-12-27 12:48:58,192][20381] Updated weights for policy 1, policy_version 1033467 (0.0010) [2023-12-27 12:48:58,256][20381] Updated weights for policy 1, policy_version 1033477 (0.0007) [2023-12-27 12:48:58,316][20381] Updated weights for policy 1, policy_version 1033487 (0.0008) [2023-12-27 12:48:58,483][20347] Updated weights for policy 0, policy_version 1034253 (0.0008) [2023-12-27 12:48:58,547][20347] Updated weights for policy 0, policy_version 1034263 (0.0008) [2023-12-27 12:48:58,606][20347] Updated weights for policy 0, policy_version 1034273 (0.0007) [2023-12-27 12:48:59,076][20381] Updated weights for policy 1, policy_version 1033497 (0.0008) [2023-12-27 12:48:59,125][20381] Updated weights for policy 1, policy_version 1033507 (0.0010) [2023-12-27 12:48:59,187][20381] Updated weights for policy 1, policy_version 1033517 (0.0010) [2023-12-27 12:48:59,251][20381] Updated weights for policy 1, policy_version 1033527 (0.0013) [2023-12-27 12:48:59,410][20347] Updated weights for policy 0, policy_version 1034283 (0.0007) [2023-12-27 12:48:59,467][20347] Updated weights for policy 0, policy_version 1034293 (0.0008) [2023-12-27 12:48:59,526][20347] Updated weights for policy 0, policy_version 1034303 (0.0005) [2023-12-27 12:48:59,903][20381] Updated weights for policy 1, policy_version 1033537 (0.0007) [2023-12-27 12:48:59,965][20381] Updated weights for policy 1, policy_version 1033547 (0.0008) [2023-12-27 12:49:00,022][20381] Updated weights for policy 1, policy_version 1033557 (0.0009) [2023-12-27 12:49:00,118][20347] Updated weights for policy 0, policy_version 1034313 (0.0006) [2023-12-27 12:49:00,167][20347] Updated weights for policy 0, policy_version 1034323 (0.0007) [2023-12-27 12:49:00,214][20347] Updated weights for policy 0, policy_version 1034333 (0.0009) [2023-12-27 12:49:00,279][20347] Updated weights for policy 0, policy_version 1034343 (0.0010) [2023-12-27 12:49:00,704][20381] Updated weights for policy 1, policy_version 1033567 (0.0007) [2023-12-27 12:49:00,759][20381] Updated weights for policy 1, policy_version 1033577 (0.0005) [2023-12-27 12:49:00,819][20381] Updated weights for policy 1, policy_version 1033587 (0.0005) [2023-12-27 12:49:00,916][20347] Updated weights for policy 0, policy_version 1034353 (0.0008) [2023-12-27 12:49:00,977][20347] Updated weights for policy 0, policy_version 1034363 (0.0009) [2023-12-27 12:49:01,037][20347] Updated weights for policy 0, policy_version 1034373 (0.0009) [2023-12-27 12:49:01,455][20381] Updated weights for policy 1, policy_version 1033597 (0.0006) [2023-12-27 12:49:01,511][20381] Updated weights for policy 1, policy_version 1033607 (0.0007) [2023-12-27 12:49:01,571][20381] Updated weights for policy 1, policy_version 1033617 (0.0006) [2023-12-27 12:49:01,739][20347] Updated weights for policy 0, policy_version 1034383 (0.0008) [2023-12-27 12:49:01,793][20347] Updated weights for policy 0, policy_version 1034393 (0.0008) [2023-12-27 12:49:01,844][20347] Updated weights for policy 0, policy_version 1034403 (0.0007) [2023-12-27 12:49:02,257][20381] Updated weights for policy 1, policy_version 1033627 (0.0007) [2023-12-27 12:49:02,318][20381] Updated weights for policy 1, policy_version 1033637 (0.0006) [2023-12-27 12:49:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 529489920. Throughput: 0: 10161.4, 1: 10310.3. Samples: 529505112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:49:02,363][19170] Avg episode reward: [(0, '970.860'), (1, '715.700')] [2023-12-27 12:49:02,390][20381] Updated weights for policy 1, policy_version 1033647 (0.0007) [2023-12-27 12:49:02,553][20347] Updated weights for policy 0, policy_version 1034413 (0.0007) [2023-12-27 12:49:02,610][20347] Updated weights for policy 0, policy_version 1034423 (0.0008) [2023-12-27 12:49:02,674][20347] Updated weights for policy 0, policy_version 1034433 (0.0008) [2023-12-27 12:49:02,942][20381] Updated weights for policy 1, policy_version 1033657 (0.0008) [2023-12-27 12:49:02,990][20381] Updated weights for policy 1, policy_version 1033667 (0.0005) [2023-12-27 12:49:03,037][20381] Updated weights for policy 1, policy_version 1033677 (0.0005) [2023-12-27 12:49:03,089][20381] Updated weights for policy 1, policy_version 1033687 (0.0005) [2023-12-27 12:49:03,388][20347] Updated weights for policy 0, policy_version 1034443 (0.0008) [2023-12-27 12:49:03,442][20347] Updated weights for policy 0, policy_version 1034453 (0.0010) [2023-12-27 12:49:03,490][20347] Updated weights for policy 0, policy_version 1034463 (0.0010) [2023-12-27 12:49:03,708][20381] Updated weights for policy 1, policy_version 1033697 (0.0010) [2023-12-27 12:49:03,770][20381] Updated weights for policy 1, policy_version 1033707 (0.0011) [2023-12-27 12:49:03,821][20381] Updated weights for policy 1, policy_version 1033717 (0.0010) [2023-12-27 12:49:04,067][20347] Updated weights for policy 0, policy_version 1034473 (0.0010) [2023-12-27 12:49:04,130][20347] Updated weights for policy 0, policy_version 1034483 (0.0007) [2023-12-27 12:49:04,190][20347] Updated weights for policy 0, policy_version 1034493 (0.0008) [2023-12-27 12:49:04,249][20347] Updated weights for policy 0, policy_version 1034503 (0.0008) [2023-12-27 12:49:04,467][20381] Updated weights for policy 1, policy_version 1033727 (0.0007) [2023-12-27 12:49:04,522][20381] Updated weights for policy 1, policy_version 1033737 (0.0006) [2023-12-27 12:49:04,585][20381] Updated weights for policy 1, policy_version 1033747 (0.0007) [2023-12-27 12:49:04,915][20347] Updated weights for policy 0, policy_version 1034513 (0.0008) [2023-12-27 12:49:04,979][20347] Updated weights for policy 0, policy_version 1034523 (0.0008) [2023-12-27 12:49:05,041][20347] Updated weights for policy 0, policy_version 1034533 (0.0010) [2023-12-27 12:49:05,147][20381] Updated weights for policy 1, policy_version 1033757 (0.0008) [2023-12-27 12:49:05,202][20381] Updated weights for policy 1, policy_version 1033767 (0.0006) [2023-12-27 12:49:05,260][20381] Updated weights for policy 1, policy_version 1033777 (0.0008) [2023-12-27 12:49:05,708][20347] Updated weights for policy 0, policy_version 1034543 (0.0007) [2023-12-27 12:49:05,762][20347] Updated weights for policy 0, policy_version 1034553 (0.0005) [2023-12-27 12:49:05,821][20347] Updated weights for policy 0, policy_version 1034563 (0.0008) [2023-12-27 12:49:05,881][20381] Updated weights for policy 1, policy_version 1033787 (0.0006) [2023-12-27 12:49:05,929][20381] Updated weights for policy 1, policy_version 1033797 (0.0008) [2023-12-27 12:49:05,980][20381] Updated weights for policy 1, policy_version 1033807 (0.0006) [2023-12-27 12:49:06,465][20347] Updated weights for policy 0, policy_version 1034573 (0.0010) [2023-12-27 12:49:06,522][20347] Updated weights for policy 0, policy_version 1034583 (0.0011) [2023-12-27 12:49:06,580][20347] Updated weights for policy 0, policy_version 1034593 (0.0010) [2023-12-27 12:49:06,729][20381] Updated weights for policy 1, policy_version 1033817 (0.0005) [2023-12-27 12:49:06,790][20381] Updated weights for policy 1, policy_version 1033827 (0.0008) [2023-12-27 12:49:06,852][20381] Updated weights for policy 1, policy_version 1033837 (0.0008) [2023-12-27 12:49:06,914][20381] Updated weights for policy 1, policy_version 1033847 (0.0008) [2023-12-27 12:49:07,267][20347] Updated weights for policy 0, policy_version 1034603 (0.0010) [2023-12-27 12:49:07,322][20347] Updated weights for policy 0, policy_version 1034613 (0.0010) [2023-12-27 12:49:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 529596416. Throughput: 0: 10200.5, 1: 10390.5. Samples: 529569848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:49:07,363][19170] Avg episode reward: [(0, '980.520'), (1, '666.730')] [2023-12-27 12:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001033848_264691712.pth... [2023-12-27 12:49:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001032632_264380416.pth [2023-12-27 12:49:07,381][20347] Updated weights for policy 0, policy_version 1034623 (0.0008) [2023-12-27 12:49:07,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001034632_264912896.pth... [2023-12-27 12:49:07,439][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001033448_264609792.pth [2023-12-27 12:49:07,585][20381] Updated weights for policy 1, policy_version 1033857 (0.0007) [2023-12-27 12:49:07,641][20381] Updated weights for policy 1, policy_version 1033867 (0.0008) [2023-12-27 12:49:07,693][20381] Updated weights for policy 1, policy_version 1033877 (0.0006) [2023-12-27 12:49:07,989][20347] Updated weights for policy 0, policy_version 1034633 (0.0005) [2023-12-27 12:49:08,041][20347] Updated weights for policy 0, policy_version 1034643 (0.0005) [2023-12-27 12:49:08,098][20347] Updated weights for policy 0, policy_version 1034653 (0.0005) [2023-12-27 12:49:08,151][20347] Updated weights for policy 0, policy_version 1034663 (0.0005) [2023-12-27 12:49:08,418][20381] Updated weights for policy 1, policy_version 1033887 (0.0007) [2023-12-27 12:49:08,471][20381] Updated weights for policy 1, policy_version 1033897 (0.0008) [2023-12-27 12:49:08,529][20381] Updated weights for policy 1, policy_version 1033907 (0.0007) [2023-12-27 12:49:08,793][20347] Updated weights for policy 0, policy_version 1034673 (0.0010) [2023-12-27 12:49:08,852][20347] Updated weights for policy 0, policy_version 1034683 (0.0010) [2023-12-27 12:49:08,903][20347] Updated weights for policy 0, policy_version 1034693 (0.0007) [2023-12-27 12:49:09,284][20381] Updated weights for policy 1, policy_version 1033917 (0.0008) [2023-12-27 12:49:09,353][20381] Updated weights for policy 1, policy_version 1033927 (0.0008) [2023-12-27 12:49:09,421][20381] Updated weights for policy 1, policy_version 1033937 (0.0008) [2023-12-27 12:49:09,508][20347] Updated weights for policy 0, policy_version 1034703 (0.0006) [2023-12-27 12:49:09,564][20347] Updated weights for policy 0, policy_version 1034713 (0.0007) [2023-12-27 12:49:09,628][20347] Updated weights for policy 0, policy_version 1034723 (0.0008) [2023-12-27 12:49:10,121][20381] Updated weights for policy 1, policy_version 1033947 (0.0009) [2023-12-27 12:49:10,180][20381] Updated weights for policy 1, policy_version 1033957 (0.0007) [2023-12-27 12:49:10,243][20381] Updated weights for policy 1, policy_version 1033967 (0.0006) [2023-12-27 12:49:10,355][20347] Updated weights for policy 0, policy_version 1034733 (0.0008) [2023-12-27 12:49:10,415][20347] Updated weights for policy 0, policy_version 1034743 (0.0009) [2023-12-27 12:49:10,474][20347] Updated weights for policy 0, policy_version 1034753 (0.0010) [2023-12-27 12:49:10,904][20381] Updated weights for policy 1, policy_version 1033977 (0.0006) [2023-12-27 12:49:10,972][20381] Updated weights for policy 1, policy_version 1033987 (0.0007) [2023-12-27 12:49:11,033][20381] Updated weights for policy 1, policy_version 1033997 (0.0009) [2023-12-27 12:49:11,092][20381] Updated weights for policy 1, policy_version 1034007 (0.0006) [2023-12-27 12:49:11,171][20347] Updated weights for policy 0, policy_version 1034763 (0.0010) [2023-12-27 12:49:11,225][20347] Updated weights for policy 0, policy_version 1034773 (0.0011) [2023-12-27 12:49:11,288][20347] Updated weights for policy 0, policy_version 1034783 (0.0011) [2023-12-27 12:49:11,852][20381] Updated weights for policy 1, policy_version 1034017 (0.0008) [2023-12-27 12:49:11,909][20381] Updated weights for policy 1, policy_version 1034027 (0.0008) [2023-12-27 12:49:11,962][20381] Updated weights for policy 1, policy_version 1034037 (0.0008) [2023-12-27 12:49:12,033][20347] Updated weights for policy 0, policy_version 1034793 (0.0010) [2023-12-27 12:49:12,092][20347] Updated weights for policy 0, policy_version 1034803 (0.0011) [2023-12-27 12:49:12,149][20347] Updated weights for policy 0, policy_version 1034813 (0.0009) [2023-12-27 12:49:12,216][20347] Updated weights for policy 0, policy_version 1034823 (0.0011) [2023-12-27 12:49:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 529702912. Throughput: 0: 10232.6, 1: 10381.6. Samples: 529692556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:49:12,363][19170] Avg episode reward: [(0, '961.100'), (1, '745.090')] [2023-12-27 12:49:12,660][20381] Updated weights for policy 1, policy_version 1034047 (0.0007) [2023-12-27 12:49:12,728][20381] Updated weights for policy 1, policy_version 1034057 (0.0008) [2023-12-27 12:49:12,785][20381] Updated weights for policy 1, policy_version 1034067 (0.0008) [2023-12-27 12:49:12,928][20347] Updated weights for policy 0, policy_version 1034833 (0.0008) [2023-12-27 12:49:12,983][20347] Updated weights for policy 0, policy_version 1034843 (0.0008) [2023-12-27 12:49:13,044][20347] Updated weights for policy 0, policy_version 1034853 (0.0007) [2023-12-27 12:49:13,481][20381] Updated weights for policy 1, policy_version 1034077 (0.0006) [2023-12-27 12:49:13,539][20381] Updated weights for policy 1, policy_version 1034087 (0.0008) [2023-12-27 12:49:13,590][20381] Updated weights for policy 1, policy_version 1034097 (0.0007) [2023-12-27 12:49:13,718][20347] Updated weights for policy 0, policy_version 1034863 (0.0007) [2023-12-27 12:49:13,774][20347] Updated weights for policy 0, policy_version 1034873 (0.0005) [2023-12-27 12:49:13,830][20347] Updated weights for policy 0, policy_version 1034883 (0.0005) [2023-12-27 12:49:14,159][20381] Updated weights for policy 1, policy_version 1034107 (0.0005) [2023-12-27 12:49:14,210][20381] Updated weights for policy 1, policy_version 1034117 (0.0005) [2023-12-27 12:49:14,265][20381] Updated weights for policy 1, policy_version 1034127 (0.0006) [2023-12-27 12:49:14,459][20347] Updated weights for policy 0, policy_version 1034893 (0.0008) [2023-12-27 12:49:14,510][20347] Updated weights for policy 0, policy_version 1034903 (0.0010) [2023-12-27 12:49:14,561][20347] Updated weights for policy 0, policy_version 1034913 (0.0010) [2023-12-27 12:49:14,824][20381] Updated weights for policy 1, policy_version 1034137 (0.0006) [2023-12-27 12:49:14,884][20381] Updated weights for policy 1, policy_version 1034147 (0.0008) [2023-12-27 12:49:14,945][20381] Updated weights for policy 1, policy_version 1034157 (0.0006) [2023-12-27 12:49:14,999][20381] Updated weights for policy 1, policy_version 1034167 (0.0006) [2023-12-27 12:49:15,227][20347] Updated weights for policy 0, policy_version 1034923 (0.0010) [2023-12-27 12:49:15,290][20347] Updated weights for policy 0, policy_version 1034933 (0.0010) [2023-12-27 12:49:15,352][20347] Updated weights for policy 0, policy_version 1034943 (0.0010) [2023-12-27 12:49:15,600][20381] Updated weights for policy 1, policy_version 1034177 (0.0006) [2023-12-27 12:49:15,661][20381] Updated weights for policy 1, policy_version 1034187 (0.0005) [2023-12-27 12:49:15,718][20381] Updated weights for policy 1, policy_version 1034197 (0.0006) [2023-12-27 12:49:15,972][20347] Updated weights for policy 0, policy_version 1034953 (0.0010) [2023-12-27 12:49:16,019][20347] Updated weights for policy 0, policy_version 1034963 (0.0010) [2023-12-27 12:49:16,064][20347] Updated weights for policy 0, policy_version 1034973 (0.0010) [2023-12-27 12:49:16,109][20347] Updated weights for policy 0, policy_version 1034983 (0.0010) [2023-12-27 12:49:16,285][20381] Updated weights for policy 1, policy_version 1034207 (0.0007) [2023-12-27 12:49:16,331][20381] Updated weights for policy 1, policy_version 1034217 (0.0008) [2023-12-27 12:49:16,377][20381] Updated weights for policy 1, policy_version 1034227 (0.0007) [2023-12-27 12:49:16,817][20347] Updated weights for policy 0, policy_version 1034993 (0.0007) [2023-12-27 12:49:16,862][20347] Updated weights for policy 0, policy_version 1035003 (0.0007) [2023-12-27 12:49:16,909][20347] Updated weights for policy 0, policy_version 1035013 (0.0007) [2023-12-27 12:49:17,040][20381] Updated weights for policy 1, policy_version 1034237 (0.0008) [2023-12-27 12:49:17,101][20381] Updated weights for policy 1, policy_version 1034247 (0.0010) [2023-12-27 12:49:17,146][20381] Updated weights for policy 1, policy_version 1034257 (0.0010) [2023-12-27 12:49:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 529809408. Throughput: 0: 10144.0, 1: 10427.9. Samples: 529817556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:49:17,364][19170] Avg episode reward: [(0, '961.050'), (1, '833.330')] [2023-12-27 12:49:17,628][20347] Updated weights for policy 0, policy_version 1035023 (0.0007) [2023-12-27 12:49:17,676][20347] Updated weights for policy 0, policy_version 1035033 (0.0005) [2023-12-27 12:49:17,730][20347] Updated weights for policy 0, policy_version 1035043 (0.0008) [2023-12-27 12:49:17,816][20381] Updated weights for policy 1, policy_version 1034267 (0.0010) [2023-12-27 12:49:17,870][20381] Updated weights for policy 1, policy_version 1034277 (0.0010) [2023-12-27 12:49:17,925][20381] Updated weights for policy 1, policy_version 1034287 (0.0010) [2023-12-27 12:49:18,371][20347] Updated weights for policy 0, policy_version 1035053 (0.0010) [2023-12-27 12:49:18,430][20347] Updated weights for policy 0, policy_version 1035063 (0.0011) [2023-12-27 12:49:18,493][20347] Updated weights for policy 0, policy_version 1035073 (0.0009) [2023-12-27 12:49:18,592][20381] Updated weights for policy 1, policy_version 1034297 (0.0010) [2023-12-27 12:49:18,652][20381] Updated weights for policy 1, policy_version 1034307 (0.0008) [2023-12-27 12:49:18,715][20381] Updated weights for policy 1, policy_version 1034317 (0.0007) [2023-12-27 12:49:18,773][20381] Updated weights for policy 1, policy_version 1034327 (0.0008) [2023-12-27 12:49:19,168][20347] Updated weights for policy 0, policy_version 1035083 (0.0007) [2023-12-27 12:49:19,227][20347] Updated weights for policy 0, policy_version 1035093 (0.0010) [2023-12-27 12:49:19,296][20347] Updated weights for policy 0, policy_version 1035103 (0.0008) [2023-12-27 12:49:19,500][20381] Updated weights for policy 1, policy_version 1034337 (0.0008) [2023-12-27 12:49:19,562][20381] Updated weights for policy 1, policy_version 1034347 (0.0007) [2023-12-27 12:49:19,629][20381] Updated weights for policy 1, policy_version 1034357 (0.0006) [2023-12-27 12:49:20,001][20347] Updated weights for policy 0, policy_version 1035113 (0.0007) [2023-12-27 12:49:20,058][20347] Updated weights for policy 0, policy_version 1035123 (0.0006) [2023-12-27 12:49:20,118][20347] Updated weights for policy 0, policy_version 1035133 (0.0006) [2023-12-27 12:49:20,182][20347] Updated weights for policy 0, policy_version 1035143 (0.0006) [2023-12-27 12:49:20,282][20381] Updated weights for policy 1, policy_version 1034367 (0.0006) [2023-12-27 12:49:20,339][20381] Updated weights for policy 1, policy_version 1034377 (0.0006) [2023-12-27 12:49:20,407][20381] Updated weights for policy 1, policy_version 1034387 (0.0009) [2023-12-27 12:49:20,794][20347] Updated weights for policy 0, policy_version 1035153 (0.0010) [2023-12-27 12:49:20,847][20347] Updated weights for policy 0, policy_version 1035163 (0.0010) [2023-12-27 12:49:20,906][20347] Updated weights for policy 0, policy_version 1035173 (0.0010) [2023-12-27 12:49:21,086][20381] Updated weights for policy 1, policy_version 1034397 (0.0008) [2023-12-27 12:49:21,152][20381] Updated weights for policy 1, policy_version 1034407 (0.0006) [2023-12-27 12:49:21,208][20381] Updated weights for policy 1, policy_version 1034417 (0.0008) [2023-12-27 12:49:21,617][20347] Updated weights for policy 0, policy_version 1035183 (0.0010) [2023-12-27 12:49:21,691][20347] Updated weights for policy 0, policy_version 1035193 (0.0008) [2023-12-27 12:49:21,764][20347] Updated weights for policy 0, policy_version 1035203 (0.0009) [2023-12-27 12:49:21,940][20381] Updated weights for policy 1, policy_version 1034427 (0.0008) [2023-12-27 12:49:22,008][20381] Updated weights for policy 1, policy_version 1034437 (0.0009) [2023-12-27 12:49:22,076][20381] Updated weights for policy 1, policy_version 1034447 (0.0009) [2023-12-27 12:49:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 529907712. Throughput: 0: 10129.5, 1: 10407.1. Samples: 529878396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:49:22,363][19170] Avg episode reward: [(0, '864.450'), (1, '695.870')] [2023-12-27 12:49:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001034456_264847360.pth... [2023-12-27 12:49:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001035208_265060352.pth... [2023-12-27 12:49:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001033240_264536064.pth [2023-12-27 12:49:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001034024_264757248.pth [2023-12-27 12:49:22,474][20347] Updated weights for policy 0, policy_version 1035213 (0.0007) [2023-12-27 12:49:22,521][20347] Updated weights for policy 0, policy_version 1035223 (0.0006) [2023-12-27 12:49:22,578][20347] Updated weights for policy 0, policy_version 1035233 (0.0012) [2023-12-27 12:49:22,858][20381] Updated weights for policy 1, policy_version 1034457 (0.0009) [2023-12-27 12:49:22,914][20381] Updated weights for policy 1, policy_version 1034467 (0.0007) [2023-12-27 12:49:22,975][20381] Updated weights for policy 1, policy_version 1034477 (0.0008) [2023-12-27 12:49:23,021][20381] Updated weights for policy 1, policy_version 1034487 (0.0010) [2023-12-27 12:49:23,179][20347] Updated weights for policy 0, policy_version 1035243 (0.0009) [2023-12-27 12:49:23,234][20347] Updated weights for policy 0, policy_version 1035253 (0.0008) [2023-12-27 12:49:23,291][20347] Updated weights for policy 0, policy_version 1035263 (0.0009) [2023-12-27 12:49:23,746][20381] Updated weights for policy 1, policy_version 1034497 (0.0007) [2023-12-27 12:49:23,798][20381] Updated weights for policy 1, policy_version 1034507 (0.0008) [2023-12-27 12:49:23,855][20381] Updated weights for policy 1, policy_version 1034517 (0.0008) [2023-12-27 12:49:24,005][20347] Updated weights for policy 0, policy_version 1035273 (0.0008) [2023-12-27 12:49:24,060][20347] Updated weights for policy 0, policy_version 1035283 (0.0007) [2023-12-27 12:49:24,118][20347] Updated weights for policy 0, policy_version 1035293 (0.0005) [2023-12-27 12:49:24,173][20347] Updated weights for policy 0, policy_version 1035303 (0.0007) [2023-12-27 12:49:24,578][20381] Updated weights for policy 1, policy_version 1034527 (0.0008) [2023-12-27 12:49:24,623][20381] Updated weights for policy 1, policy_version 1034537 (0.0009) [2023-12-27 12:49:24,668][20381] Updated weights for policy 1, policy_version 1034547 (0.0010) [2023-12-27 12:49:24,782][20347] Updated weights for policy 0, policy_version 1035313 (0.0008) [2023-12-27 12:49:24,834][20347] Updated weights for policy 0, policy_version 1035323 (0.0008) [2023-12-27 12:49:24,885][20347] Updated weights for policy 0, policy_version 1035333 (0.0007) [2023-12-27 12:49:25,407][20381] Updated weights for policy 1, policy_version 1034557 (0.0009) [2023-12-27 12:49:25,452][20381] Updated weights for policy 1, policy_version 1034567 (0.0008) [2023-12-27 12:49:25,500][20381] Updated weights for policy 1, policy_version 1034577 (0.0006) [2023-12-27 12:49:25,528][20347] Updated weights for policy 0, policy_version 1035343 (0.0007) [2023-12-27 12:49:25,580][20347] Updated weights for policy 0, policy_version 1035353 (0.0006) [2023-12-27 12:49:25,633][20347] Updated weights for policy 0, policy_version 1035363 (0.0006) [2023-12-27 12:49:26,193][20381] Updated weights for policy 1, policy_version 1034587 (0.0010) [2023-12-27 12:49:26,212][20347] Updated weights for policy 0, policy_version 1035373 (0.0008) [2023-12-27 12:49:26,240][20381] Updated weights for policy 1, policy_version 1034597 (0.0010) [2023-12-27 12:49:26,269][20347] Updated weights for policy 0, policy_version 1035383 (0.0008) [2023-12-27 12:49:26,290][20381] Updated weights for policy 1, policy_version 1034607 (0.0011) [2023-12-27 12:49:26,325][20347] Updated weights for policy 0, policy_version 1035393 (0.0007) [2023-12-27 12:49:26,939][20347] Updated weights for policy 0, policy_version 1035403 (0.0010) [2023-12-27 12:49:26,971][20381] Updated weights for policy 1, policy_version 1034617 (0.0010) [2023-12-27 12:49:26,993][20347] Updated weights for policy 0, policy_version 1035413 (0.0010) [2023-12-27 12:49:27,021][20381] Updated weights for policy 1, policy_version 1034627 (0.0007) [2023-12-27 12:49:27,039][20347] Updated weights for policy 0, policy_version 1035423 (0.0007) [2023-12-27 12:49:27,070][20381] Updated weights for policy 1, policy_version 1034637 (0.0005) [2023-12-27 12:49:27,119][20381] Updated weights for policy 1, policy_version 1034647 (0.0007) [2023-12-27 12:49:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 530014208. Throughput: 0: 10158.7, 1: 10324.8. Samples: 529999096. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:27,363][19170] Avg episode reward: [(0, '893.720'), (1, '686.010')] [2023-12-27 12:49:27,664][20347] Updated weights for policy 0, policy_version 1035433 (0.0006) [2023-12-27 12:49:27,721][20347] Updated weights for policy 0, policy_version 1035443 (0.0010) [2023-12-27 12:49:27,746][20381] Updated weights for policy 1, policy_version 1034657 (0.0006) [2023-12-27 12:49:27,772][20347] Updated weights for policy 0, policy_version 1035453 (0.0010) [2023-12-27 12:49:27,780][20291] KL-divergence is very high: 299.5765 [2023-12-27 12:49:27,785][20291] KL-divergence is very high: 369.6001 [2023-12-27 12:49:27,793][20381] Updated weights for policy 1, policy_version 1034667 (0.0005) [2023-12-27 12:49:27,820][20347] Updated weights for policy 0, policy_version 1035463 (0.0010) [2023-12-27 12:49:27,840][20381] Updated weights for policy 1, policy_version 1034677 (0.0005) [2023-12-27 12:49:28,475][20381] Updated weights for policy 1, policy_version 1034687 (0.0008) [2023-12-27 12:49:28,530][20381] Updated weights for policy 1, policy_version 1034697 (0.0006) [2023-12-27 12:49:28,536][20347] Updated weights for policy 0, policy_version 1035473 (0.0010) [2023-12-27 12:49:28,586][20381] Updated weights for policy 1, policy_version 1034707 (0.0005) [2023-12-27 12:49:28,588][20347] Updated weights for policy 0, policy_version 1035483 (0.0010) [2023-12-27 12:49:28,649][20347] Updated weights for policy 0, policy_version 1035493 (0.0010) [2023-12-27 12:49:29,232][20381] Updated weights for policy 1, policy_version 1034717 (0.0008) [2023-12-27 12:49:29,289][20381] Updated weights for policy 1, policy_version 1034727 (0.0007) [2023-12-27 12:49:29,346][20381] Updated weights for policy 1, policy_version 1034737 (0.0008) [2023-12-27 12:49:29,349][20347] Updated weights for policy 0, policy_version 1035503 (0.0011) [2023-12-27 12:49:29,419][20347] Updated weights for policy 0, policy_version 1035513 (0.0010) [2023-12-27 12:49:29,480][20347] Updated weights for policy 0, policy_version 1035523 (0.0010) [2023-12-27 12:49:29,998][20381] Updated weights for policy 1, policy_version 1034747 (0.0006) [2023-12-27 12:49:30,059][20381] Updated weights for policy 1, policy_version 1034757 (0.0007) [2023-12-27 12:49:30,117][20381] Updated weights for policy 1, policy_version 1034767 (0.0006) [2023-12-27 12:49:30,175][20347] Updated weights for policy 0, policy_version 1035533 (0.0010) [2023-12-27 12:49:30,228][20347] Updated weights for policy 0, policy_version 1035543 (0.0010) [2023-12-27 12:49:30,279][20347] Updated weights for policy 0, policy_version 1035553 (0.0010) [2023-12-27 12:49:30,799][20381] Updated weights for policy 1, policy_version 1034777 (0.0007) [2023-12-27 12:49:30,856][20381] Updated weights for policy 1, policy_version 1034787 (0.0007) [2023-12-27 12:49:30,902][20381] Updated weights for policy 1, policy_version 1034797 (0.0007) [2023-12-27 12:49:30,903][20347] Updated weights for policy 0, policy_version 1035563 (0.0010) [2023-12-27 12:49:30,947][20381] Updated weights for policy 1, policy_version 1034807 (0.0007) [2023-12-27 12:49:30,957][20347] Updated weights for policy 0, policy_version 1035573 (0.0010) [2023-12-27 12:49:31,008][20347] Updated weights for policy 0, policy_version 1035583 (0.0010) [2023-12-27 12:49:31,681][20381] Updated weights for policy 1, policy_version 1034817 (0.0008) [2023-12-27 12:49:31,744][20381] Updated weights for policy 1, policy_version 1034827 (0.0008) [2023-12-27 12:49:31,747][20347] Updated weights for policy 0, policy_version 1035593 (0.0009) [2023-12-27 12:49:31,797][20347] Updated weights for policy 0, policy_version 1035603 (0.0007) [2023-12-27 12:49:31,799][20381] Updated weights for policy 1, policy_version 1034837 (0.0006) [2023-12-27 12:49:31,846][20347] Updated weights for policy 0, policy_version 1035613 (0.0006) [2023-12-27 12:49:31,891][20347] Updated weights for policy 0, policy_version 1035623 (0.0008) [2023-12-27 12:49:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 530112512. Throughput: 0: 10157.0, 1: 10333.7. Samples: 530124428. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:32,363][19170] Avg episode reward: [(0, '941.560'), (1, '725.450')] [2023-12-27 12:49:32,429][20381] Updated weights for policy 1, policy_version 1034847 (0.0007) [2023-12-27 12:49:32,481][20381] Updated weights for policy 1, policy_version 1034857 (0.0007) [2023-12-27 12:49:32,536][20381] Updated weights for policy 1, policy_version 1034867 (0.0007) [2023-12-27 12:49:32,652][20347] Updated weights for policy 0, policy_version 1035633 (0.0008) [2023-12-27 12:49:32,710][20347] Updated weights for policy 0, policy_version 1035643 (0.0007) [2023-12-27 12:49:32,761][20347] Updated weights for policy 0, policy_version 1035653 (0.0008) [2023-12-27 12:49:33,255][20381] Updated weights for policy 1, policy_version 1034877 (0.0008) [2023-12-27 12:49:33,317][20381] Updated weights for policy 1, policy_version 1034887 (0.0008) [2023-12-27 12:49:33,370][20347] Updated weights for policy 0, policy_version 1035663 (0.0007) [2023-12-27 12:49:33,374][20381] Updated weights for policy 1, policy_version 1034897 (0.0008) [2023-12-27 12:49:33,414][20347] Updated weights for policy 0, policy_version 1035673 (0.0005) [2023-12-27 12:49:33,463][20347] Updated weights for policy 0, policy_version 1035683 (0.0005) [2023-12-27 12:49:34,023][20347] Updated weights for policy 0, policy_version 1035693 (0.0006) [2023-12-27 12:49:34,076][20347] Updated weights for policy 0, policy_version 1035703 (0.0007) [2023-12-27 12:49:34,107][20381] Updated weights for policy 1, policy_version 1034907 (0.0007) [2023-12-27 12:49:34,134][20347] Updated weights for policy 0, policy_version 1035713 (0.0006) [2023-12-27 12:49:34,174][20381] Updated weights for policy 1, policy_version 1034918 (0.0009) [2023-12-27 12:49:34,233][20381] Updated weights for policy 1, policy_version 1034928 (0.0007) [2023-12-27 12:49:34,732][20347] Updated weights for policy 0, policy_version 1035723 (0.0007) [2023-12-27 12:49:34,793][20347] Updated weights for policy 0, policy_version 1035733 (0.0008) [2023-12-27 12:49:34,852][20347] Updated weights for policy 0, policy_version 1035743 (0.0006) [2023-12-27 12:49:34,902][20381] Updated weights for policy 1, policy_version 1034938 (0.0009) [2023-12-27 12:49:34,960][20381] Updated weights for policy 1, policy_version 1034948 (0.0010) [2023-12-27 12:49:35,020][20381] Updated weights for policy 1, policy_version 1034958 (0.0009) [2023-12-27 12:49:35,081][20381] Updated weights for policy 1, policy_version 1034968 (0.0010) [2023-12-27 12:49:35,508][20347] Updated weights for policy 0, policy_version 1035753 (0.0008) [2023-12-27 12:49:35,569][20347] Updated weights for policy 0, policy_version 1035763 (0.0006) [2023-12-27 12:49:35,623][20347] Updated weights for policy 0, policy_version 1035773 (0.0005) [2023-12-27 12:49:35,673][20347] Updated weights for policy 0, policy_version 1035783 (0.0005) [2023-12-27 12:49:35,687][20381] Updated weights for policy 1, policy_version 1034978 (0.0006) [2023-12-27 12:49:35,748][20381] Updated weights for policy 1, policy_version 1034988 (0.0005) [2023-12-27 12:49:35,799][20381] Updated weights for policy 1, policy_version 1034998 (0.0009) [2023-12-27 12:49:36,313][20347] Updated weights for policy 0, policy_version 1035793 (0.0010) [2023-12-27 12:49:36,379][20347] Updated weights for policy 0, policy_version 1035803 (0.0011) [2023-12-27 12:49:36,426][20381] Updated weights for policy 1, policy_version 1035008 (0.0008) [2023-12-27 12:49:36,439][20347] Updated weights for policy 0, policy_version 1035813 (0.0010) [2023-12-27 12:49:36,488][20381] Updated weights for policy 1, policy_version 1035018 (0.0011) [2023-12-27 12:49:36,551][20381] Updated weights for policy 1, policy_version 1035028 (0.0011) [2023-12-27 12:49:37,138][20347] Updated weights for policy 0, policy_version 1035823 (0.0010) [2023-12-27 12:49:37,190][20347] Updated weights for policy 0, policy_version 1035833 (0.0010) [2023-12-27 12:49:37,209][20381] Updated weights for policy 1, policy_version 1035038 (0.0006) [2023-12-27 12:49:37,246][20347] Updated weights for policy 0, policy_version 1035843 (0.0011) [2023-12-27 12:49:37,267][20381] Updated weights for policy 1, policy_version 1035048 (0.0006) [2023-12-27 12:49:37,327][20381] Updated weights for policy 1, policy_version 1035058 (0.0005) [2023-12-27 12:49:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 530227200. Throughput: 0: 10217.9, 1: 10333.2. Samples: 530187608. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:37,363][19170] Avg episode reward: [(0, '990.220'), (1, '686.520')] [2023-12-27 12:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001035064_265003008.pth... [2023-12-27 12:49:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001035848_265224192.pth... [2023-12-27 12:49:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001033848_264691712.pth [2023-12-27 12:49:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001034632_264912896.pth [2023-12-27 12:49:37,897][20381] Updated weights for policy 1, policy_version 1035068 (0.0007) [2023-12-27 12:49:37,954][20381] Updated weights for policy 1, policy_version 1035078 (0.0005) [2023-12-27 12:49:37,956][20347] Updated weights for policy 0, policy_version 1035853 (0.0010) [2023-12-27 12:49:38,003][20381] Updated weights for policy 1, policy_version 1035088 (0.0005) [2023-12-27 12:49:38,006][20347] Updated weights for policy 0, policy_version 1035863 (0.0008) [2023-12-27 12:49:38,065][20347] Updated weights for policy 0, policy_version 1035873 (0.0010) [2023-12-27 12:49:38,595][20381] Updated weights for policy 1, policy_version 1035098 (0.0006) [2023-12-27 12:49:38,648][20381] Updated weights for policy 1, policy_version 1035108 (0.0008) [2023-12-27 12:49:38,710][20381] Updated weights for policy 1, policy_version 1035118 (0.0007) [2023-12-27 12:49:38,747][20347] Updated weights for policy 0, policy_version 1035883 (0.0010) [2023-12-27 12:49:38,768][20381] Updated weights for policy 1, policy_version 1035128 (0.0006) [2023-12-27 12:49:38,799][20347] Updated weights for policy 0, policy_version 1035893 (0.0010) [2023-12-27 12:49:38,845][20347] Updated weights for policy 0, policy_version 1035903 (0.0010) [2023-12-27 12:49:39,481][20381] Updated weights for policy 1, policy_version 1035138 (0.0008) [2023-12-27 12:49:39,545][20381] Updated weights for policy 1, policy_version 1035148 (0.0008) [2023-12-27 12:49:39,569][20347] Updated weights for policy 0, policy_version 1035913 (0.0010) [2023-12-27 12:49:39,609][20381] Updated weights for policy 1, policy_version 1035158 (0.0006) [2023-12-27 12:49:39,626][20347] Updated weights for policy 0, policy_version 1035923 (0.0007) [2023-12-27 12:49:39,680][20347] Updated weights for policy 0, policy_version 1035933 (0.0007) [2023-12-27 12:49:39,733][20347] Updated weights for policy 0, policy_version 1035943 (0.0008) [2023-12-27 12:49:40,304][20381] Updated weights for policy 1, policy_version 1035168 (0.0008) [2023-12-27 12:49:40,361][20381] Updated weights for policy 1, policy_version 1035178 (0.0009) [2023-12-27 12:49:40,418][20381] Updated weights for policy 1, policy_version 1035188 (0.0009) [2023-12-27 12:49:40,448][20347] Updated weights for policy 0, policy_version 1035953 (0.0008) [2023-12-27 12:49:40,507][20347] Updated weights for policy 0, policy_version 1035963 (0.0007) [2023-12-27 12:49:40,573][20347] Updated weights for policy 0, policy_version 1035973 (0.0008) [2023-12-27 12:49:41,129][20381] Updated weights for policy 1, policy_version 1035198 (0.0008) [2023-12-27 12:49:41,191][20381] Updated weights for policy 1, policy_version 1035208 (0.0008) [2023-12-27 12:49:41,258][20381] Updated weights for policy 1, policy_version 1035218 (0.0008) [2023-12-27 12:49:41,309][20347] Updated weights for policy 0, policy_version 1035983 (0.0008) [2023-12-27 12:49:41,378][20347] Updated weights for policy 0, policy_version 1035993 (0.0008) [2023-12-27 12:49:41,440][20347] Updated weights for policy 0, policy_version 1036003 (0.0010) [2023-12-27 12:49:42,011][20381] Updated weights for policy 1, policy_version 1035228 (0.0006) [2023-12-27 12:49:42,070][20381] Updated weights for policy 1, policy_version 1035238 (0.0008) [2023-12-27 12:49:42,128][20381] Updated weights for policy 1, policy_version 1035248 (0.0007) [2023-12-27 12:49:42,140][20347] Updated weights for policy 0, policy_version 1036013 (0.0010) [2023-12-27 12:49:42,196][20347] Updated weights for policy 0, policy_version 1036023 (0.0010) [2023-12-27 12:49:42,252][20347] Updated weights for policy 0, policy_version 1036033 (0.0010) [2023-12-27 12:49:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 530325504. Throughput: 0: 10299.2, 1: 10344.8. Samples: 530312236. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:42,363][19170] Avg episode reward: [(0, '990.220'), (1, '676.420')] [2023-12-27 12:49:42,749][20381] Updated weights for policy 1, policy_version 1035258 (0.0008) [2023-12-27 12:49:42,812][20381] Updated weights for policy 1, policy_version 1035268 (0.0009) [2023-12-27 12:49:42,878][20381] Updated weights for policy 1, policy_version 1035278 (0.0009) [2023-12-27 12:49:42,920][20347] Updated weights for policy 0, policy_version 1036043 (0.0009) [2023-12-27 12:49:42,944][20381] Updated weights for policy 1, policy_version 1035288 (0.0008) [2023-12-27 12:49:42,975][20347] Updated weights for policy 0, policy_version 1036053 (0.0005) [2023-12-27 12:49:43,023][20347] Updated weights for policy 0, policy_version 1036063 (0.0005) [2023-12-27 12:49:43,565][20381] Updated weights for policy 1, policy_version 1035298 (0.0008) [2023-12-27 12:49:43,623][20381] Updated weights for policy 1, policy_version 1035308 (0.0007) [2023-12-27 12:49:43,686][20347] Updated weights for policy 0, policy_version 1036073 (0.0006) [2023-12-27 12:49:43,686][20381] Updated weights for policy 1, policy_version 1035318 (0.0008) [2023-12-27 12:49:43,747][20347] Updated weights for policy 0, policy_version 1036083 (0.0008) [2023-12-27 12:49:43,808][20347] Updated weights for policy 0, policy_version 1036093 (0.0008) [2023-12-27 12:49:43,872][20347] Updated weights for policy 0, policy_version 1036103 (0.0008) [2023-12-27 12:49:44,260][20381] Updated weights for policy 1, policy_version 1035328 (0.0006) [2023-12-27 12:49:44,311][20381] Updated weights for policy 1, policy_version 1035338 (0.0009) [2023-12-27 12:49:44,360][20381] Updated weights for policy 1, policy_version 1035348 (0.0007) [2023-12-27 12:49:44,567][20347] Updated weights for policy 0, policy_version 1036113 (0.0006) [2023-12-27 12:49:44,621][20347] Updated weights for policy 0, policy_version 1036123 (0.0006) [2023-12-27 12:49:44,673][20347] Updated weights for policy 0, policy_version 1036133 (0.0005) [2023-12-27 12:49:45,070][20381] Updated weights for policy 1, policy_version 1035358 (0.0006) [2023-12-27 12:49:45,134][20381] Updated weights for policy 1, policy_version 1035368 (0.0006) [2023-12-27 12:49:45,198][20381] Updated weights for policy 1, policy_version 1035378 (0.0006) [2023-12-27 12:49:45,342][20347] Updated weights for policy 0, policy_version 1036143 (0.0011) [2023-12-27 12:49:45,407][20347] Updated weights for policy 0, policy_version 1036153 (0.0011) [2023-12-27 12:49:45,468][20347] Updated weights for policy 0, policy_version 1036163 (0.0010) [2023-12-27 12:49:45,747][20381] Updated weights for policy 1, policy_version 1035388 (0.0006) [2023-12-27 12:49:45,798][20381] Updated weights for policy 1, policy_version 1035398 (0.0005) [2023-12-27 12:49:45,850][20381] Updated weights for policy 1, policy_version 1035408 (0.0007) [2023-12-27 12:49:46,103][20347] Updated weights for policy 0, policy_version 1036173 (0.0009) [2023-12-27 12:49:46,167][20347] Updated weights for policy 0, policy_version 1036183 (0.0008) [2023-12-27 12:49:46,227][20347] Updated weights for policy 0, policy_version 1036193 (0.0008) [2023-12-27 12:49:46,419][20381] Updated weights for policy 1, policy_version 1035418 (0.0007) [2023-12-27 12:49:46,477][20381] Updated weights for policy 1, policy_version 1035428 (0.0008) [2023-12-27 12:49:46,530][20381] Updated weights for policy 1, policy_version 1035438 (0.0007) [2023-12-27 12:49:46,589][20381] Updated weights for policy 1, policy_version 1035448 (0.0008) [2023-12-27 12:49:46,917][20347] Updated weights for policy 0, policy_version 1036203 (0.0008) [2023-12-27 12:49:46,965][20347] Updated weights for policy 0, policy_version 1036213 (0.0007) [2023-12-27 12:49:47,013][20347] Updated weights for policy 0, policy_version 1036223 (0.0008) [2023-12-27 12:49:47,237][20381] Updated weights for policy 1, policy_version 1035458 (0.0008) [2023-12-27 12:49:47,285][20381] Updated weights for policy 1, policy_version 1035468 (0.0010) [2023-12-27 12:49:47,342][20381] Updated weights for policy 1, policy_version 1035478 (0.0006) [2023-12-27 12:49:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 530432000. Throughput: 0: 10322.7, 1: 10391.3. Samples: 530437244. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:47,363][19170] Avg episode reward: [(0, '933.010'), (1, '725.510')] [2023-12-27 12:49:47,703][20347] Updated weights for policy 0, policy_version 1036233 (0.0010) [2023-12-27 12:49:47,748][20347] Updated weights for policy 0, policy_version 1036243 (0.0010) [2023-12-27 12:49:47,805][20347] Updated weights for policy 0, policy_version 1036253 (0.0010) [2023-12-27 12:49:47,858][20347] Updated weights for policy 0, policy_version 1036263 (0.0010) [2023-12-27 12:49:47,882][20381] Updated weights for policy 1, policy_version 1035488 (0.0005) [2023-12-27 12:49:47,936][20381] Updated weights for policy 1, policy_version 1035498 (0.0005) [2023-12-27 12:49:47,987][20381] Updated weights for policy 1, policy_version 1035508 (0.0007) [2023-12-27 12:49:48,431][20347] Updated weights for policy 0, policy_version 1036273 (0.0006) [2023-12-27 12:49:48,491][20347] Updated weights for policy 0, policy_version 1036283 (0.0005) [2023-12-27 12:49:48,545][20347] Updated weights for policy 0, policy_version 1036293 (0.0006) [2023-12-27 12:49:48,644][20381] Updated weights for policy 1, policy_version 1035518 (0.0008) [2023-12-27 12:49:48,705][20381] Updated weights for policy 1, policy_version 1035528 (0.0008) [2023-12-27 12:49:48,766][20381] Updated weights for policy 1, policy_version 1035538 (0.0007) [2023-12-27 12:49:49,203][20347] Updated weights for policy 0, policy_version 1036303 (0.0009) [2023-12-27 12:49:49,274][20347] Updated weights for policy 0, policy_version 1036313 (0.0008) [2023-12-27 12:49:49,326][20347] Updated weights for policy 0, policy_version 1036323 (0.0008) [2023-12-27 12:49:49,439][20381] Updated weights for policy 1, policy_version 1035548 (0.0008) [2023-12-27 12:49:49,501][20381] Updated weights for policy 1, policy_version 1035558 (0.0008) [2023-12-27 12:49:49,557][20381] Updated weights for policy 1, policy_version 1035568 (0.0007) [2023-12-27 12:49:50,036][20347] Updated weights for policy 0, policy_version 1036333 (0.0008) [2023-12-27 12:49:50,101][20347] Updated weights for policy 0, policy_version 1036343 (0.0007) [2023-12-27 12:49:50,169][20347] Updated weights for policy 0, policy_version 1036354 (0.0010) [2023-12-27 12:49:50,300][20381] Updated weights for policy 1, policy_version 1035578 (0.0008) [2023-12-27 12:49:50,359][20381] Updated weights for policy 1, policy_version 1035588 (0.0007) [2023-12-27 12:49:50,420][20381] Updated weights for policy 1, policy_version 1035598 (0.0006) [2023-12-27 12:49:50,475][20381] Updated weights for policy 1, policy_version 1035608 (0.0008) [2023-12-27 12:49:50,778][20347] Updated weights for policy 0, policy_version 1036364 (0.0006) [2023-12-27 12:49:50,823][20347] Updated weights for policy 0, policy_version 1036374 (0.0008) [2023-12-27 12:49:50,871][20347] Updated weights for policy 0, policy_version 1036384 (0.0007) [2023-12-27 12:49:51,125][20381] Updated weights for policy 1, policy_version 1035618 (0.0008) [2023-12-27 12:49:51,184][20381] Updated weights for policy 1, policy_version 1035628 (0.0009) [2023-12-27 12:49:51,242][20381] Updated weights for policy 1, policy_version 1035638 (0.0007) [2023-12-27 12:49:51,574][20347] Updated weights for policy 0, policy_version 1036394 (0.0008) [2023-12-27 12:49:51,643][20347] Updated weights for policy 0, policy_version 1036404 (0.0007) [2023-12-27 12:49:51,702][20347] Updated weights for policy 0, policy_version 1036414 (0.0008) [2023-12-27 12:49:51,764][20347] Updated weights for policy 0, policy_version 1036424 (0.0008) [2023-12-27 12:49:51,979][20381] Updated weights for policy 1, policy_version 1035648 (0.0006) [2023-12-27 12:49:52,037][20381] Updated weights for policy 1, policy_version 1035658 (0.0008) [2023-12-27 12:49:52,089][20381] Updated weights for policy 1, policy_version 1035668 (0.0007) [2023-12-27 12:49:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 530530304. Throughput: 0: 10353.2, 1: 10334.2. Samples: 530500784. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:52,363][19170] Avg episode reward: [(0, '932.940'), (1, '745.000')] [2023-12-27 12:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001035672_265158656.pth... [2023-12-27 12:49:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001036424_265371648.pth... [2023-12-27 12:49:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001035208_265060352.pth [2023-12-27 12:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001034456_264847360.pth [2023-12-27 12:49:52,451][20347] Updated weights for policy 0, policy_version 1036434 (0.0005) [2023-12-27 12:49:52,512][20347] Updated weights for policy 0, policy_version 1036444 (0.0006) [2023-12-27 12:49:52,568][20347] Updated weights for policy 0, policy_version 1036454 (0.0006) [2023-12-27 12:49:52,753][20381] Updated weights for policy 1, policy_version 1035678 (0.0007) [2023-12-27 12:49:52,810][20381] Updated weights for policy 1, policy_version 1035688 (0.0008) [2023-12-27 12:49:52,863][20381] Updated weights for policy 1, policy_version 1035698 (0.0007) [2023-12-27 12:49:53,175][20347] Updated weights for policy 0, policy_version 1036464 (0.0006) [2023-12-27 12:49:53,232][20347] Updated weights for policy 0, policy_version 1036474 (0.0008) [2023-12-27 12:49:53,284][20347] Updated weights for policy 0, policy_version 1036484 (0.0009) [2023-12-27 12:49:53,441][20381] Updated weights for policy 1, policy_version 1035708 (0.0007) [2023-12-27 12:49:53,481][20292] KL-divergence is very high: 123.8654 [2023-12-27 12:49:53,508][20381] Updated weights for policy 1, policy_version 1035718 (0.0008) [2023-12-27 12:49:53,573][20381] Updated weights for policy 1, policy_version 1035728 (0.0008) [2023-12-27 12:49:53,585][20292] KL-divergence is very high: 160.6862 [2023-12-27 12:49:53,904][20347] Updated weights for policy 0, policy_version 1036494 (0.0009) [2023-12-27 12:49:53,947][20347] Updated weights for policy 0, policy_version 1036504 (0.0008) [2023-12-27 12:49:53,993][20347] Updated weights for policy 0, policy_version 1036514 (0.0008) [2023-12-27 12:49:54,234][20381] Updated weights for policy 1, policy_version 1035738 (0.0008) [2023-12-27 12:49:54,299][20381] Updated weights for policy 1, policy_version 1035748 (0.0008) [2023-12-27 12:49:54,363][20381] Updated weights for policy 1, policy_version 1035758 (0.0009) [2023-12-27 12:49:54,416][20381] Updated weights for policy 1, policy_version 1035768 (0.0008) [2023-12-27 12:49:54,647][20347] Updated weights for policy 0, policy_version 1036524 (0.0007) [2023-12-27 12:49:54,691][20347] Updated weights for policy 0, policy_version 1036534 (0.0007) [2023-12-27 12:49:54,742][20347] Updated weights for policy 0, policy_version 1036544 (0.0007) [2023-12-27 12:49:55,061][20381] Updated weights for policy 1, policy_version 1035778 (0.0008) [2023-12-27 12:49:55,108][20381] Updated weights for policy 1, policy_version 1035788 (0.0008) [2023-12-27 12:49:55,165][20381] Updated weights for policy 1, policy_version 1035798 (0.0008) [2023-12-27 12:49:55,462][20347] Updated weights for policy 0, policy_version 1036554 (0.0008) [2023-12-27 12:49:55,516][20347] Updated weights for policy 0, policy_version 1036564 (0.0008) [2023-12-27 12:49:55,570][20347] Updated weights for policy 0, policy_version 1036574 (0.0007) [2023-12-27 12:49:55,625][20347] Updated weights for policy 0, policy_version 1036584 (0.0008) [2023-12-27 12:49:55,872][20381] Updated weights for policy 1, policy_version 1035808 (0.0008) [2023-12-27 12:49:55,933][20381] Updated weights for policy 1, policy_version 1035818 (0.0007) [2023-12-27 12:49:55,991][20381] Updated weights for policy 1, policy_version 1035828 (0.0008) [2023-12-27 12:49:56,320][20347] Updated weights for policy 0, policy_version 1036594 (0.0008) [2023-12-27 12:49:56,382][20347] Updated weights for policy 0, policy_version 1036604 (0.0008) [2023-12-27 12:49:56,440][20347] Updated weights for policy 0, policy_version 1036614 (0.0007) [2023-12-27 12:49:56,601][20381] Updated weights for policy 1, policy_version 1035838 (0.0007) [2023-12-27 12:49:56,653][20381] Updated weights for policy 1, policy_version 1035848 (0.0007) [2023-12-27 12:49:56,706][20381] Updated weights for policy 1, policy_version 1035858 (0.0008) [2023-12-27 12:49:57,043][20347] Updated weights for policy 0, policy_version 1036624 (0.0009) [2023-12-27 12:49:57,093][20347] Updated weights for policy 0, policy_version 1036634 (0.0005) [2023-12-27 12:49:57,149][20347] Updated weights for policy 0, policy_version 1036644 (0.0005) [2023-12-27 12:49:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 530636800. Throughput: 0: 10343.6, 1: 10363.2. Samples: 530624360. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:49:57,363][19170] Avg episode reward: [(0, '961.030'), (1, '735.280')] [2023-12-27 12:49:57,377][20381] Updated weights for policy 1, policy_version 1035868 (0.0007) [2023-12-27 12:49:57,388][20292] KL-divergence is very high: 104.8038 [2023-12-27 12:49:57,434][20381] Updated weights for policy 1, policy_version 1035878 (0.0007) [2023-12-27 12:49:57,492][20381] Updated weights for policy 1, policy_version 1035888 (0.0007) [2023-12-27 12:49:57,702][20347] Updated weights for policy 0, policy_version 1036654 (0.0005) [2023-12-27 12:49:57,755][20347] Updated weights for policy 0, policy_version 1036664 (0.0005) [2023-12-27 12:49:57,810][20347] Updated weights for policy 0, policy_version 1036674 (0.0007) [2023-12-27 12:49:58,194][20381] Updated weights for policy 1, policy_version 1035898 (0.0008) [2023-12-27 12:49:58,256][20381] Updated weights for policy 1, policy_version 1035908 (0.0008) [2023-12-27 12:49:58,322][20381] Updated weights for policy 1, policy_version 1035918 (0.0008) [2023-12-27 12:49:58,389][20381] Updated weights for policy 1, policy_version 1035928 (0.0008) [2023-12-27 12:49:58,510][20347] Updated weights for policy 0, policy_version 1036684 (0.0008) [2023-12-27 12:49:58,578][20347] Updated weights for policy 0, policy_version 1036694 (0.0009) [2023-12-27 12:49:58,645][20347] Updated weights for policy 0, policy_version 1036704 (0.0008) [2023-12-27 12:49:59,092][20381] Updated weights for policy 1, policy_version 1035938 (0.0008) [2023-12-27 12:49:59,142][20381] Updated weights for policy 1, policy_version 1035948 (0.0006) [2023-12-27 12:49:59,203][20381] Updated weights for policy 1, policy_version 1035958 (0.0007) [2023-12-27 12:49:59,408][20347] Updated weights for policy 0, policy_version 1036714 (0.0008) [2023-12-27 12:49:59,467][20347] Updated weights for policy 0, policy_version 1036724 (0.0007) [2023-12-27 12:49:59,525][20347] Updated weights for policy 0, policy_version 1036734 (0.0008) [2023-12-27 12:49:59,588][20347] Updated weights for policy 0, policy_version 1036744 (0.0007) [2023-12-27 12:49:59,898][20381] Updated weights for policy 1, policy_version 1035968 (0.0008) [2023-12-27 12:49:59,958][20381] Updated weights for policy 1, policy_version 1035978 (0.0007) [2023-12-27 12:50:00,021][20381] Updated weights for policy 1, policy_version 1035988 (0.0006) [2023-12-27 12:50:00,245][20347] Updated weights for policy 0, policy_version 1036754 (0.0010) [2023-12-27 12:50:00,300][20347] Updated weights for policy 0, policy_version 1036764 (0.0008) [2023-12-27 12:50:00,368][20347] Updated weights for policy 0, policy_version 1036774 (0.0006) [2023-12-27 12:50:00,675][20381] Updated weights for policy 1, policy_version 1035998 (0.0009) [2023-12-27 12:50:00,730][20381] Updated weights for policy 1, policy_version 1036008 (0.0011) [2023-12-27 12:50:00,791][20381] Updated weights for policy 1, policy_version 1036018 (0.0010) [2023-12-27 12:50:00,973][20347] Updated weights for policy 0, policy_version 1036784 (0.0007) [2023-12-27 12:50:01,040][20347] Updated weights for policy 0, policy_version 1036794 (0.0010) [2023-12-27 12:50:01,099][20347] Updated weights for policy 0, policy_version 1036804 (0.0009) [2023-12-27 12:50:01,469][20381] Updated weights for policy 1, policy_version 1036028 (0.0010) [2023-12-27 12:50:01,521][20381] Updated weights for policy 1, policy_version 1036038 (0.0010) [2023-12-27 12:50:01,579][20381] Updated weights for policy 1, policy_version 1036048 (0.0010) [2023-12-27 12:50:01,765][20347] Updated weights for policy 0, policy_version 1036814 (0.0009) [2023-12-27 12:50:01,816][20347] Updated weights for policy 0, policy_version 1036824 (0.0006) [2023-12-27 12:50:01,864][20347] Updated weights for policy 0, policy_version 1036834 (0.0007) [2023-12-27 12:50:02,259][20381] Updated weights for policy 1, policy_version 1036058 (0.0011) [2023-12-27 12:50:02,321][20381] Updated weights for policy 1, policy_version 1036068 (0.0010) [2023-12-27 12:50:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 530735104. Throughput: 0: 10395.6, 1: 10304.5. Samples: 530749056. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:02,363][19170] Avg episode reward: [(0, '941.370'), (1, '843.060')] [2023-12-27 12:50:02,386][20381] Updated weights for policy 1, policy_version 1036078 (0.0008) [2023-12-27 12:50:02,447][20381] Updated weights for policy 1, policy_version 1036088 (0.0005) [2023-12-27 12:50:02,571][20347] Updated weights for policy 0, policy_version 1036844 (0.0008) [2023-12-27 12:50:02,630][20347] Updated weights for policy 0, policy_version 1036854 (0.0010) [2023-12-27 12:50:02,681][20347] Updated weights for policy 0, policy_version 1036864 (0.0010) [2023-12-27 12:50:03,003][20381] Updated weights for policy 1, policy_version 1036098 (0.0010) [2023-12-27 12:50:03,061][20381] Updated weights for policy 1, policy_version 1036108 (0.0010) [2023-12-27 12:50:03,116][20381] Updated weights for policy 1, policy_version 1036118 (0.0010) [2023-12-27 12:50:03,321][20347] Updated weights for policy 0, policy_version 1036874 (0.0010) [2023-12-27 12:50:03,376][20347] Updated weights for policy 0, policy_version 1036884 (0.0007) [2023-12-27 12:50:03,421][20347] Updated weights for policy 0, policy_version 1036894 (0.0005) [2023-12-27 12:50:03,474][20347] Updated weights for policy 0, policy_version 1036904 (0.0005) [2023-12-27 12:50:03,788][20381] Updated weights for policy 1, policy_version 1036128 (0.0009) [2023-12-27 12:50:03,844][20381] Updated weights for policy 1, policy_version 1036138 (0.0009) [2023-12-27 12:50:03,906][20381] Updated weights for policy 1, policy_version 1036148 (0.0010) [2023-12-27 12:50:04,149][20347] Updated weights for policy 0, policy_version 1036914 (0.0007) [2023-12-27 12:50:04,209][20347] Updated weights for policy 0, policy_version 1036924 (0.0011) [2023-12-27 12:50:04,267][20347] Updated weights for policy 0, policy_version 1036934 (0.0010) [2023-12-27 12:50:04,599][20381] Updated weights for policy 1, policy_version 1036158 (0.0011) [2023-12-27 12:50:04,657][20381] Updated weights for policy 1, policy_version 1036168 (0.0010) [2023-12-27 12:50:04,715][20381] Updated weights for policy 1, policy_version 1036178 (0.0010) [2023-12-27 12:50:04,908][20347] Updated weights for policy 0, policy_version 1036944 (0.0009) [2023-12-27 12:50:04,963][20347] Updated weights for policy 0, policy_version 1036954 (0.0009) [2023-12-27 12:50:05,020][20347] Updated weights for policy 0, policy_version 1036964 (0.0006) [2023-12-27 12:50:05,332][20381] Updated weights for policy 1, policy_version 1036188 (0.0011) [2023-12-27 12:50:05,380][20381] Updated weights for policy 1, policy_version 1036198 (0.0010) [2023-12-27 12:50:05,429][20381] Updated weights for policy 1, policy_version 1036208 (0.0010) [2023-12-27 12:50:05,640][20347] Updated weights for policy 0, policy_version 1036974 (0.0005) [2023-12-27 12:50:05,684][20347] Updated weights for policy 0, policy_version 1036984 (0.0010) [2023-12-27 12:50:05,737][20347] Updated weights for policy 0, policy_version 1036994 (0.0010) [2023-12-27 12:50:06,095][20381] Updated weights for policy 1, policy_version 1036218 (0.0009) [2023-12-27 12:50:06,158][20381] Updated weights for policy 1, policy_version 1036228 (0.0008) [2023-12-27 12:50:06,221][20381] Updated weights for policy 1, policy_version 1036238 (0.0008) [2023-12-27 12:50:06,283][20381] Updated weights for policy 1, policy_version 1036248 (0.0007) [2023-12-27 12:50:06,427][20347] Updated weights for policy 0, policy_version 1037004 (0.0010) [2023-12-27 12:50:06,490][20347] Updated weights for policy 0, policy_version 1037014 (0.0009) [2023-12-27 12:50:06,554][20347] Updated weights for policy 0, policy_version 1037024 (0.0009) [2023-12-27 12:50:07,002][20381] Updated weights for policy 1, policy_version 1036258 (0.0010) [2023-12-27 12:50:07,062][20381] Updated weights for policy 1, policy_version 1036268 (0.0011) [2023-12-27 12:50:07,114][20381] Updated weights for policy 1, policy_version 1036278 (0.0010) [2023-12-27 12:50:07,304][20347] Updated weights for policy 0, policy_version 1037034 (0.0009) [2023-12-27 12:50:07,360][20347] Updated weights for policy 0, policy_version 1037044 (0.0010) [2023-12-27 12:50:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 530841600. Throughput: 0: 10421.6, 1: 10340.2. Samples: 530812676. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:07,363][19170] Avg episode reward: [(0, '960.880'), (1, '784.460')] [2023-12-27 12:50:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001036280_265314304.pth... [2023-12-27 12:50:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001035064_265003008.pth [2023-12-27 12:50:07,419][20347] Updated weights for policy 0, policy_version 1037054 (0.0010) [2023-12-27 12:50:07,476][20347] Updated weights for policy 0, policy_version 1037064 (0.0009) [2023-12-27 12:50:07,476][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001037064_265535488.pth... [2023-12-27 12:50:07,480][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001035848_265224192.pth [2023-12-27 12:50:07,795][20381] Updated weights for policy 1, policy_version 1036288 (0.0008) [2023-12-27 12:50:07,843][20381] Updated weights for policy 1, policy_version 1036298 (0.0010) [2023-12-27 12:50:07,897][20381] Updated weights for policy 1, policy_version 1036308 (0.0010) [2023-12-27 12:50:08,118][20347] Updated weights for policy 0, policy_version 1037074 (0.0007) [2023-12-27 12:50:08,168][20347] Updated weights for policy 0, policy_version 1037084 (0.0007) [2023-12-27 12:50:08,219][20347] Updated weights for policy 0, policy_version 1037094 (0.0007) [2023-12-27 12:50:08,602][20381] Updated weights for policy 1, policy_version 1036318 (0.0009) [2023-12-27 12:50:08,665][20381] Updated weights for policy 1, policy_version 1036328 (0.0007) [2023-12-27 12:50:08,721][20381] Updated weights for policy 1, policy_version 1036338 (0.0008) [2023-12-27 12:50:08,924][20347] Updated weights for policy 0, policy_version 1037104 (0.0007) [2023-12-27 12:50:08,984][20347] Updated weights for policy 0, policy_version 1037114 (0.0007) [2023-12-27 12:50:09,042][20347] Updated weights for policy 0, policy_version 1037124 (0.0008) [2023-12-27 12:50:09,498][20381] Updated weights for policy 1, policy_version 1036348 (0.0007) [2023-12-27 12:50:09,563][20381] Updated weights for policy 1, policy_version 1036358 (0.0008) [2023-12-27 12:50:09,624][20381] Updated weights for policy 1, policy_version 1036368 (0.0008) [2023-12-27 12:50:09,741][20347] Updated weights for policy 0, policy_version 1037134 (0.0007) [2023-12-27 12:50:09,802][20347] Updated weights for policy 0, policy_version 1037144 (0.0008) [2023-12-27 12:50:09,862][20347] Updated weights for policy 0, policy_version 1037154 (0.0008) [2023-12-27 12:50:10,283][20381] Updated weights for policy 1, policy_version 1036378 (0.0007) [2023-12-27 12:50:10,343][20381] Updated weights for policy 1, policy_version 1036388 (0.0008) [2023-12-27 12:50:10,403][20381] Updated weights for policy 1, policy_version 1036398 (0.0008) [2023-12-27 12:50:10,408][20292] KL-divergence is very high: 100.2374 [2023-12-27 12:50:10,413][20292] KL-divergence is very high: 108.7882 [2023-12-27 12:50:10,457][20381] Updated weights for policy 1, policy_version 1036408 (0.0008) [2023-12-27 12:50:10,596][20347] Updated weights for policy 0, policy_version 1037164 (0.0007) [2023-12-27 12:50:10,655][20347] Updated weights for policy 0, policy_version 1037174 (0.0007) [2023-12-27 12:50:10,716][20347] Updated weights for policy 0, policy_version 1037184 (0.0006) [2023-12-27 12:50:11,198][20381] Updated weights for policy 1, policy_version 1036418 (0.0008) [2023-12-27 12:50:11,259][20381] Updated weights for policy 1, policy_version 1036428 (0.0008) [2023-12-27 12:50:11,315][20381] Updated weights for policy 1, policy_version 1036438 (0.0006) [2023-12-27 12:50:11,425][20347] Updated weights for policy 0, policy_version 1037194 (0.0007) [2023-12-27 12:50:11,482][20347] Updated weights for policy 0, policy_version 1037204 (0.0007) [2023-12-27 12:50:11,532][20347] Updated weights for policy 0, policy_version 1037214 (0.0007) [2023-12-27 12:50:11,582][20347] Updated weights for policy 0, policy_version 1037224 (0.0007) [2023-12-27 12:50:12,026][20381] Updated weights for policy 1, policy_version 1036448 (0.0006) [2023-12-27 12:50:12,088][20381] Updated weights for policy 1, policy_version 1036458 (0.0006) [2023-12-27 12:50:12,148][20381] Updated weights for policy 1, policy_version 1036468 (0.0006) [2023-12-27 12:50:12,284][20347] Updated weights for policy 0, policy_version 1037234 (0.0008) [2023-12-27 12:50:12,345][20347] Updated weights for policy 0, policy_version 1037244 (0.0006) [2023-12-27 12:50:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 530939904. Throughput: 0: 10357.7, 1: 10382.3. Samples: 530932400. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:12,363][19170] Avg episode reward: [(0, '980.500'), (1, '715.750')] [2023-12-27 12:50:12,407][20347] Updated weights for policy 0, policy_version 1037254 (0.0008) [2023-12-27 12:50:12,798][20381] Updated weights for policy 1, policy_version 1036478 (0.0006) [2023-12-27 12:50:12,847][20381] Updated weights for policy 1, policy_version 1036488 (0.0005) [2023-12-27 12:50:12,899][20381] Updated weights for policy 1, policy_version 1036498 (0.0008) [2023-12-27 12:50:13,061][20347] Updated weights for policy 0, policy_version 1037264 (0.0008) [2023-12-27 12:50:13,115][20347] Updated weights for policy 0, policy_version 1037274 (0.0008) [2023-12-27 12:50:13,181][20347] Updated weights for policy 0, policy_version 1037284 (0.0008) [2023-12-27 12:50:13,508][20381] Updated weights for policy 1, policy_version 1036508 (0.0007) [2023-12-27 12:50:13,564][20381] Updated weights for policy 1, policy_version 1036518 (0.0007) [2023-12-27 12:50:13,621][20381] Updated weights for policy 1, policy_version 1036528 (0.0008) [2023-12-27 12:50:13,893][20347] Updated weights for policy 0, policy_version 1037294 (0.0008) [2023-12-27 12:50:13,951][20347] Updated weights for policy 0, policy_version 1037304 (0.0007) [2023-12-27 12:50:14,011][20347] Updated weights for policy 0, policy_version 1037314 (0.0008) [2023-12-27 12:50:14,299][20381] Updated weights for policy 1, policy_version 1036538 (0.0008) [2023-12-27 12:50:14,360][20381] Updated weights for policy 1, policy_version 1036548 (0.0009) [2023-12-27 12:50:14,419][20381] Updated weights for policy 1, policy_version 1036558 (0.0008) [2023-12-27 12:50:14,479][20381] Updated weights for policy 1, policy_version 1036568 (0.0008) [2023-12-27 12:50:14,679][20347] Updated weights for policy 0, policy_version 1037324 (0.0008) [2023-12-27 12:50:14,738][20347] Updated weights for policy 0, policy_version 1037334 (0.0008) [2023-12-27 12:50:14,802][20347] Updated weights for policy 0, policy_version 1037344 (0.0007) [2023-12-27 12:50:15,154][20381] Updated weights for policy 1, policy_version 1036578 (0.0007) [2023-12-27 12:50:15,213][20381] Updated weights for policy 1, policy_version 1036588 (0.0007) [2023-12-27 12:50:15,270][20381] Updated weights for policy 1, policy_version 1036598 (0.0008) [2023-12-27 12:50:15,505][20347] Updated weights for policy 0, policy_version 1037354 (0.0008) [2023-12-27 12:50:15,566][20347] Updated weights for policy 0, policy_version 1037364 (0.0007) [2023-12-27 12:50:15,626][20347] Updated weights for policy 0, policy_version 1037374 (0.0008) [2023-12-27 12:50:15,684][20347] Updated weights for policy 0, policy_version 1037384 (0.0007) [2023-12-27 12:50:15,923][20381] Updated weights for policy 1, policy_version 1036608 (0.0007) [2023-12-27 12:50:15,978][20381] Updated weights for policy 1, policy_version 1036618 (0.0007) [2023-12-27 12:50:16,036][20381] Updated weights for policy 1, policy_version 1036628 (0.0008) [2023-12-27 12:50:16,339][20347] Updated weights for policy 0, policy_version 1037394 (0.0007) [2023-12-27 12:50:16,388][20347] Updated weights for policy 0, policy_version 1037404 (0.0007) [2023-12-27 12:50:16,444][20347] Updated weights for policy 0, policy_version 1037414 (0.0007) [2023-12-27 12:50:16,735][20381] Updated weights for policy 1, policy_version 1036638 (0.0009) [2023-12-27 12:50:16,795][20381] Updated weights for policy 1, policy_version 1036648 (0.0010) [2023-12-27 12:50:16,843][20381] Updated weights for policy 1, policy_version 1036658 (0.0010) [2023-12-27 12:50:17,156][20347] Updated weights for policy 0, policy_version 1037424 (0.0007) [2023-12-27 12:50:17,209][20347] Updated weights for policy 0, policy_version 1037434 (0.0007) [2023-12-27 12:50:17,258][20347] Updated weights for policy 0, policy_version 1037444 (0.0007) [2023-12-27 12:50:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 531046400. Throughput: 0: 10309.6, 1: 10344.8. Samples: 531053876. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:17,363][19170] Avg episode reward: [(0, '942.080'), (1, '696.050')] [2023-12-27 12:50:17,524][20381] Updated weights for policy 1, policy_version 1036668 (0.0010) [2023-12-27 12:50:17,576][20381] Updated weights for policy 1, policy_version 1036678 (0.0010) [2023-12-27 12:50:17,630][20381] Updated weights for policy 1, policy_version 1036688 (0.0009) [2023-12-27 12:50:17,973][20347] Updated weights for policy 0, policy_version 1037454 (0.0008) [2023-12-27 12:50:18,021][20347] Updated weights for policy 0, policy_version 1037464 (0.0007) [2023-12-27 12:50:18,068][20347] Updated weights for policy 0, policy_version 1037474 (0.0007) [2023-12-27 12:50:18,305][20381] Updated weights for policy 1, policy_version 1036698 (0.0010) [2023-12-27 12:50:18,369][20381] Updated weights for policy 1, policy_version 1036708 (0.0011) [2023-12-27 12:50:18,423][20381] Updated weights for policy 1, policy_version 1036718 (0.0010) [2023-12-27 12:50:18,480][20381] Updated weights for policy 1, policy_version 1036728 (0.0007) [2023-12-27 12:50:18,778][20347] Updated weights for policy 0, policy_version 1037484 (0.0007) [2023-12-27 12:50:18,833][20347] Updated weights for policy 0, policy_version 1037494 (0.0008) [2023-12-27 12:50:18,883][20347] Updated weights for policy 0, policy_version 1037504 (0.0008) [2023-12-27 12:50:19,171][20381] Updated weights for policy 1, policy_version 1036738 (0.0008) [2023-12-27 12:50:19,224][20381] Updated weights for policy 1, policy_version 1036748 (0.0008) [2023-12-27 12:50:19,285][20381] Updated weights for policy 1, policy_version 1036758 (0.0008) [2023-12-27 12:50:19,615][20347] Updated weights for policy 0, policy_version 1037514 (0.0008) [2023-12-27 12:50:19,676][20347] Updated weights for policy 0, policy_version 1037524 (0.0007) [2023-12-27 12:50:19,738][20347] Updated weights for policy 0, policy_version 1037534 (0.0007) [2023-12-27 12:50:19,807][20347] Updated weights for policy 0, policy_version 1037544 (0.0008) [2023-12-27 12:50:20,088][20381] Updated weights for policy 1, policy_version 1036768 (0.0010) [2023-12-27 12:50:20,143][20381] Updated weights for policy 1, policy_version 1036778 (0.0011) [2023-12-27 12:50:20,195][20381] Updated weights for policy 1, policy_version 1036788 (0.0011) [2023-12-27 12:50:20,521][20347] Updated weights for policy 0, policy_version 1037554 (0.0007) [2023-12-27 12:50:20,581][20347] Updated weights for policy 0, policy_version 1037564 (0.0008) [2023-12-27 12:50:20,642][20347] Updated weights for policy 0, policy_version 1037574 (0.0007) [2023-12-27 12:50:20,944][20381] Updated weights for policy 1, policy_version 1036798 (0.0009) [2023-12-27 12:50:21,001][20381] Updated weights for policy 1, policy_version 1036808 (0.0007) [2023-12-27 12:50:21,063][20381] Updated weights for policy 1, policy_version 1036818 (0.0009) [2023-12-27 12:50:21,372][20347] Updated weights for policy 0, policy_version 1037584 (0.0008) [2023-12-27 12:50:21,434][20347] Updated weights for policy 0, policy_version 1037594 (0.0006) [2023-12-27 12:50:21,495][20347] Updated weights for policy 0, policy_version 1037604 (0.0007) [2023-12-27 12:50:21,830][20381] Updated weights for policy 1, policy_version 1036828 (0.0008) [2023-12-27 12:50:21,891][20381] Updated weights for policy 1, policy_version 1036838 (0.0011) [2023-12-27 12:50:21,951][20381] Updated weights for policy 1, policy_version 1036848 (0.0011) [2023-12-27 12:50:22,176][20347] Updated weights for policy 0, policy_version 1037614 (0.0007) [2023-12-27 12:50:22,239][20347] Updated weights for policy 0, policy_version 1037624 (0.0008) [2023-12-27 12:50:22,304][20347] Updated weights for policy 0, policy_version 1037634 (0.0008) [2023-12-27 12:50:22,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 531144704. Throughput: 0: 10220.8, 1: 10324.9. Samples: 531112164. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:22,363][19170] Avg episode reward: [(0, '932.450'), (1, '725.550')] [2023-12-27 12:50:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001036856_265461760.pth... [2023-12-27 12:50:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001037640_265682944.pth... [2023-12-27 12:50:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001035672_265158656.pth [2023-12-27 12:50:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001036424_265371648.pth [2023-12-27 12:50:22,663][20381] Updated weights for policy 1, policy_version 1036858 (0.0010) [2023-12-27 12:50:22,723][20381] Updated weights for policy 1, policy_version 1036868 (0.0010) [2023-12-27 12:50:22,783][20381] Updated weights for policy 1, policy_version 1036878 (0.0011) [2023-12-27 12:50:22,843][20381] Updated weights for policy 1, policy_version 1036888 (0.0011) [2023-12-27 12:50:22,965][20347] Updated weights for policy 0, policy_version 1037644 (0.0007) [2023-12-27 12:50:23,032][20347] Updated weights for policy 0, policy_version 1037654 (0.0009) [2023-12-27 12:50:23,089][20347] Updated weights for policy 0, policy_version 1037664 (0.0010) [2023-12-27 12:50:23,486][20381] Updated weights for policy 1, policy_version 1036898 (0.0005) [2023-12-27 12:50:23,546][20381] Updated weights for policy 1, policy_version 1036908 (0.0005) [2023-12-27 12:50:23,605][20381] Updated weights for policy 1, policy_version 1036918 (0.0007) [2023-12-27 12:50:23,687][20347] Updated weights for policy 0, policy_version 1037674 (0.0009) [2023-12-27 12:50:23,737][20347] Updated weights for policy 0, policy_version 1037684 (0.0007) [2023-12-27 12:50:23,794][20347] Updated weights for policy 0, policy_version 1037694 (0.0008) [2023-12-27 12:50:23,857][20347] Updated weights for policy 0, policy_version 1037704 (0.0007) [2023-12-27 12:50:24,227][20381] Updated weights for policy 1, policy_version 1036928 (0.0008) [2023-12-27 12:50:24,284][20381] Updated weights for policy 1, policy_version 1036938 (0.0008) [2023-12-27 12:50:24,339][20381] Updated weights for policy 1, policy_version 1036948 (0.0007) [2023-12-27 12:50:24,590][20347] Updated weights for policy 0, policy_version 1037714 (0.0007) [2023-12-27 12:50:24,640][20347] Updated weights for policy 0, policy_version 1037724 (0.0007) [2023-12-27 12:50:24,688][20347] Updated weights for policy 0, policy_version 1037734 (0.0007) [2023-12-27 12:50:24,988][20381] Updated weights for policy 1, policy_version 1036958 (0.0007) [2023-12-27 12:50:25,049][20381] Updated weights for policy 1, policy_version 1036968 (0.0007) [2023-12-27 12:50:25,100][20381] Updated weights for policy 1, policy_version 1036978 (0.0007) [2023-12-27 12:50:25,379][20347] Updated weights for policy 0, policy_version 1037744 (0.0007) [2023-12-27 12:50:25,428][20347] Updated weights for policy 0, policy_version 1037754 (0.0007) [2023-12-27 12:50:25,493][20347] Updated weights for policy 0, policy_version 1037764 (0.0008) [2023-12-27 12:50:25,763][20381] Updated weights for policy 1, policy_version 1036988 (0.0007) [2023-12-27 12:50:25,826][20381] Updated weights for policy 1, policy_version 1036998 (0.0005) [2023-12-27 12:50:25,886][20381] Updated weights for policy 1, policy_version 1037008 (0.0006) [2023-12-27 12:50:26,215][20347] Updated weights for policy 0, policy_version 1037774 (0.0007) [2023-12-27 12:50:26,274][20347] Updated weights for policy 0, policy_version 1037784 (0.0007) [2023-12-27 12:50:26,342][20347] Updated weights for policy 0, policy_version 1037794 (0.0007) [2023-12-27 12:50:26,500][20381] Updated weights for policy 1, policy_version 1037018 (0.0007) [2023-12-27 12:50:26,546][20381] Updated weights for policy 1, policy_version 1037028 (0.0007) [2023-12-27 12:50:26,602][20381] Updated weights for policy 1, policy_version 1037038 (0.0005) [2023-12-27 12:50:26,651][20381] Updated weights for policy 1, policy_version 1037048 (0.0007) [2023-12-27 12:50:27,044][20347] Updated weights for policy 0, policy_version 1037804 (0.0007) [2023-12-27 12:50:27,106][20347] Updated weights for policy 0, policy_version 1037814 (0.0008) [2023-12-27 12:50:27,160][20347] Updated weights for policy 0, policy_version 1037824 (0.0007) [2023-12-27 12:50:27,264][20381] Updated weights for policy 1, policy_version 1037058 (0.0009) [2023-12-27 12:50:27,310][20381] Updated weights for policy 1, policy_version 1037068 (0.0008) [2023-12-27 12:50:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 531243008. Throughput: 0: 10204.0, 1: 10275.7. Samples: 531233824. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:27,363][19170] Avg episode reward: [(0, '942.110'), (1, '715.570')] [2023-12-27 12:50:27,364][20381] Updated weights for policy 1, policy_version 1037078 (0.0006) [2023-12-27 12:50:27,797][20347] Updated weights for policy 0, policy_version 1037834 (0.0008) [2023-12-27 12:50:27,854][20347] Updated weights for policy 0, policy_version 1037844 (0.0010) [2023-12-27 12:50:27,914][20347] Updated weights for policy 0, policy_version 1037854 (0.0010) [2023-12-27 12:50:27,971][20347] Updated weights for policy 0, policy_version 1037864 (0.0010) [2023-12-27 12:50:28,006][20381] Updated weights for policy 1, policy_version 1037088 (0.0007) [2023-12-27 12:50:28,057][20381] Updated weights for policy 1, policy_version 1037098 (0.0005) [2023-12-27 12:50:28,109][20381] Updated weights for policy 1, policy_version 1037108 (0.0007) [2023-12-27 12:50:28,627][20347] Updated weights for policy 0, policy_version 1037874 (0.0008) [2023-12-27 12:50:28,689][20347] Updated weights for policy 0, policy_version 1037884 (0.0008) [2023-12-27 12:50:28,743][20347] Updated weights for policy 0, policy_version 1037894 (0.0008) [2023-12-27 12:50:28,804][20381] Updated weights for policy 1, policy_version 1037118 (0.0007) [2023-12-27 12:50:28,865][20381] Updated weights for policy 1, policy_version 1037128 (0.0009) [2023-12-27 12:50:28,921][20381] Updated weights for policy 1, policy_version 1037138 (0.0009) [2023-12-27 12:50:29,425][20347] Updated weights for policy 0, policy_version 1037904 (0.0005) [2023-12-27 12:50:29,475][20347] Updated weights for policy 0, policy_version 1037914 (0.0008) [2023-12-27 12:50:29,535][20347] Updated weights for policy 0, policy_version 1037924 (0.0007) [2023-12-27 12:50:29,566][20381] Updated weights for policy 1, policy_version 1037148 (0.0010) [2023-12-27 12:50:29,633][20381] Updated weights for policy 1, policy_version 1037158 (0.0011) [2023-12-27 12:50:29,688][20381] Updated weights for policy 1, policy_version 1037168 (0.0010) [2023-12-27 12:50:30,176][20347] Updated weights for policy 0, policy_version 1037934 (0.0007) [2023-12-27 12:50:30,234][20347] Updated weights for policy 0, policy_version 1037944 (0.0007) [2023-12-27 12:50:30,286][20347] Updated weights for policy 0, policy_version 1037954 (0.0007) [2023-12-27 12:50:30,356][20381] Updated weights for policy 1, policy_version 1037178 (0.0010) [2023-12-27 12:50:30,408][20381] Updated weights for policy 1, policy_version 1037188 (0.0011) [2023-12-27 12:50:30,467][20381] Updated weights for policy 1, policy_version 1037198 (0.0009) [2023-12-27 12:50:30,523][20381] Updated weights for policy 1, policy_version 1037208 (0.0008) [2023-12-27 12:50:30,952][20347] Updated weights for policy 0, policy_version 1037964 (0.0008) [2023-12-27 12:50:31,002][20347] Updated weights for policy 0, policy_version 1037974 (0.0007) [2023-12-27 12:50:31,065][20347] Updated weights for policy 0, policy_version 1037984 (0.0007) [2023-12-27 12:50:31,227][20381] Updated weights for policy 1, policy_version 1037218 (0.0005) [2023-12-27 12:50:31,287][20381] Updated weights for policy 1, policy_version 1037228 (0.0006) [2023-12-27 12:50:31,351][20381] Updated weights for policy 1, policy_version 1037238 (0.0010) [2023-12-27 12:50:31,849][20347] Updated weights for policy 0, policy_version 1037994 (0.0007) [2023-12-27 12:50:31,903][20347] Updated weights for policy 0, policy_version 1038004 (0.0007) [2023-12-27 12:50:31,957][20347] Updated weights for policy 0, policy_version 1038014 (0.0007) [2023-12-27 12:50:32,012][20381] Updated weights for policy 1, policy_version 1037248 (0.0011) [2023-12-27 12:50:32,016][20347] Updated weights for policy 0, policy_version 1038024 (0.0007) [2023-12-27 12:50:32,062][20381] Updated weights for policy 1, policy_version 1037258 (0.0010) [2023-12-27 12:50:32,118][20381] Updated weights for policy 1, policy_version 1037268 (0.0011) [2023-12-27 12:50:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 531349504. Throughput: 0: 10214.9, 1: 10227.7. Samples: 531357156. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:32,363][19170] Avg episode reward: [(0, '932.280'), (1, '696.120')] [2023-12-27 12:50:32,695][20347] Updated weights for policy 0, policy_version 1038034 (0.0005) [2023-12-27 12:50:32,759][20347] Updated weights for policy 0, policy_version 1038044 (0.0005) [2023-12-27 12:50:32,814][20347] Updated weights for policy 0, policy_version 1038054 (0.0007) [2023-12-27 12:50:32,861][20381] Updated weights for policy 1, policy_version 1037278 (0.0009) [2023-12-27 12:50:32,915][20381] Updated weights for policy 1, policy_version 1037288 (0.0007) [2023-12-27 12:50:32,970][20381] Updated weights for policy 1, policy_version 1037298 (0.0006) [2023-12-27 12:50:33,485][20347] Updated weights for policy 0, policy_version 1038064 (0.0008) [2023-12-27 12:50:33,532][20347] Updated weights for policy 0, policy_version 1038074 (0.0007) [2023-12-27 12:50:33,553][20381] Updated weights for policy 1, policy_version 1037308 (0.0005) [2023-12-27 12:50:33,582][20347] Updated weights for policy 0, policy_version 1038084 (0.0007) [2023-12-27 12:50:33,613][20381] Updated weights for policy 1, policy_version 1037318 (0.0010) [2023-12-27 12:50:33,661][20381] Updated weights for policy 1, policy_version 1037328 (0.0010) [2023-12-27 12:50:34,275][20381] Updated weights for policy 1, policy_version 1037338 (0.0011) [2023-12-27 12:50:34,328][20347] Updated weights for policy 0, policy_version 1038094 (0.0005) [2023-12-27 12:50:34,338][20381] Updated weights for policy 1, policy_version 1037348 (0.0011) [2023-12-27 12:50:34,382][20291] KL-divergence is very high: 127.8648 [2023-12-27 12:50:34,392][20347] Updated weights for policy 0, policy_version 1038104 (0.0008) [2023-12-27 12:50:34,399][20381] Updated weights for policy 1, policy_version 1037358 (0.0010) [2023-12-27 12:50:34,428][20291] KL-divergence is very high: 106.1343 [2023-12-27 12:50:34,449][20347] Updated weights for policy 0, policy_version 1038114 (0.0007) [2023-12-27 12:50:34,459][20381] Updated weights for policy 1, policy_version 1037368 (0.0011) [2023-12-27 12:50:35,076][20347] Updated weights for policy 0, policy_version 1038124 (0.0006) [2023-12-27 12:50:35,139][20347] Updated weights for policy 0, policy_version 1038134 (0.0005) [2023-12-27 12:50:35,139][20381] Updated weights for policy 1, policy_version 1037378 (0.0010) [2023-12-27 12:50:35,197][20381] Updated weights for policy 1, policy_version 1037388 (0.0010) [2023-12-27 12:50:35,199][20347] Updated weights for policy 0, policy_version 1038144 (0.0005) [2023-12-27 12:50:35,243][20381] Updated weights for policy 1, policy_version 1037398 (0.0010) [2023-12-27 12:50:35,795][20347] Updated weights for policy 0, policy_version 1038154 (0.0006) [2023-12-27 12:50:35,858][20347] Updated weights for policy 0, policy_version 1038164 (0.0005) [2023-12-27 12:50:35,910][20381] Updated weights for policy 1, policy_version 1037408 (0.0009) [2023-12-27 12:50:35,914][20347] Updated weights for policy 0, policy_version 1038174 (0.0005) [2023-12-27 12:50:35,955][20381] Updated weights for policy 1, policy_version 1037418 (0.0010) [2023-12-27 12:50:35,965][20347] Updated weights for policy 0, policy_version 1038184 (0.0005) [2023-12-27 12:50:36,003][20381] Updated weights for policy 1, policy_version 1037428 (0.0010) [2023-12-27 12:50:36,609][20347] Updated weights for policy 0, policy_version 1038194 (0.0007) [2023-12-27 12:50:36,673][20347] Updated weights for policy 0, policy_version 1038204 (0.0007) [2023-12-27 12:50:36,713][20381] Updated weights for policy 1, policy_version 1037438 (0.0010) [2023-12-27 12:50:36,733][20347] Updated weights for policy 0, policy_version 1038214 (0.0006) [2023-12-27 12:50:36,765][20381] Updated weights for policy 1, policy_version 1037448 (0.0010) [2023-12-27 12:50:36,829][20381] Updated weights for policy 1, policy_version 1037458 (0.0011) [2023-12-27 12:50:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.3, 300 sec: 20521.6). Total num frames: 531447808. Throughput: 0: 10186.6, 1: 10234.7. Samples: 531419744. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:37,364][19170] Avg episode reward: [(0, '903.380'), (1, '735.250')] [2023-12-27 12:50:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001038216_265830400.pth... [2023-12-27 12:50:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001037464_265617408.pth... [2023-12-27 12:50:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001037064_265535488.pth [2023-12-27 12:50:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001036280_265314304.pth [2023-12-27 12:50:37,442][20347] Updated weights for policy 0, policy_version 1038224 (0.0006) [2023-12-27 12:50:37,505][20347] Updated weights for policy 0, policy_version 1038234 (0.0005) [2023-12-27 12:50:37,547][20381] Updated weights for policy 1, policy_version 1037468 (0.0011) [2023-12-27 12:50:37,566][20347] Updated weights for policy 0, policy_version 1038244 (0.0007) [2023-12-27 12:50:37,599][20292] KL-divergence is very high: 125.1962 [2023-12-27 12:50:37,604][20381] Updated weights for policy 1, policy_version 1037478 (0.0011) [2023-12-27 12:50:37,670][20381] Updated weights for policy 1, policy_version 1037488 (0.0011) [2023-12-27 12:50:38,212][20347] Updated weights for policy 0, policy_version 1038254 (0.0006) [2023-12-27 12:50:38,266][20347] Updated weights for policy 0, policy_version 1038264 (0.0007) [2023-12-27 12:50:38,321][20347] Updated weights for policy 0, policy_version 1038274 (0.0006) [2023-12-27 12:50:38,328][20381] Updated weights for policy 1, policy_version 1037498 (0.0010) [2023-12-27 12:50:38,394][20381] Updated weights for policy 1, policy_version 1037508 (0.0011) [2023-12-27 12:50:38,456][20381] Updated weights for policy 1, policy_version 1037518 (0.0010) [2023-12-27 12:50:38,512][20381] Updated weights for policy 1, policy_version 1037528 (0.0011) [2023-12-27 12:50:39,027][20347] Updated weights for policy 0, policy_version 1038284 (0.0007) [2023-12-27 12:50:39,078][20347] Updated weights for policy 0, policy_version 1038294 (0.0009) [2023-12-27 12:50:39,133][20347] Updated weights for policy 0, policy_version 1038304 (0.0007) [2023-12-27 12:50:39,168][20381] Updated weights for policy 1, policy_version 1037538 (0.0010) [2023-12-27 12:50:39,232][20381] Updated weights for policy 1, policy_version 1037548 (0.0009) [2023-12-27 12:50:39,296][20381] Updated weights for policy 1, policy_version 1037558 (0.0008) [2023-12-27 12:50:39,886][20347] Updated weights for policy 0, policy_version 1038314 (0.0006) [2023-12-27 12:50:39,945][20347] Updated weights for policy 0, policy_version 1038324 (0.0006) [2023-12-27 12:50:39,994][20347] Updated weights for policy 0, policy_version 1038334 (0.0008) [2023-12-27 12:50:40,047][20347] Updated weights for policy 0, policy_version 1038344 (0.0008) [2023-12-27 12:50:40,066][20381] Updated weights for policy 1, policy_version 1037568 (0.0007) [2023-12-27 12:50:40,079][20292] KL-divergence is very high: 109.0400 [2023-12-27 12:50:40,130][20381] Updated weights for policy 1, policy_version 1037578 (0.0007) [2023-12-27 12:50:40,191][20381] Updated weights for policy 1, policy_version 1037588 (0.0008) [2023-12-27 12:50:40,783][20347] Updated weights for policy 0, policy_version 1038354 (0.0007) [2023-12-27 12:50:40,819][20381] Updated weights for policy 1, policy_version 1037598 (0.0007) [2023-12-27 12:50:40,833][20347] Updated weights for policy 0, policy_version 1038364 (0.0007) [2023-12-27 12:50:40,875][20381] Updated weights for policy 1, policy_version 1037608 (0.0006) [2023-12-27 12:50:40,884][20347] Updated weights for policy 0, policy_version 1038374 (0.0005) [2023-12-27 12:50:40,929][20381] Updated weights for policy 1, policy_version 1037618 (0.0007) [2023-12-27 12:50:41,636][20347] Updated weights for policy 0, policy_version 1038384 (0.0007) [2023-12-27 12:50:41,649][20381] Updated weights for policy 1, policy_version 1037628 (0.0008) [2023-12-27 12:50:41,707][20347] Updated weights for policy 0, policy_version 1038394 (0.0007) [2023-12-27 12:50:41,712][20381] Updated weights for policy 1, policy_version 1037638 (0.0007) [2023-12-27 12:50:41,774][20347] Updated weights for policy 0, policy_version 1038404 (0.0007) [2023-12-27 12:50:41,775][20381] Updated weights for policy 1, policy_version 1037648 (0.0007) [2023-12-27 12:50:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 531546112. Throughput: 0: 10141.2, 1: 10214.9. Samples: 531540384. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:42,363][19170] Avg episode reward: [(0, '884.310'), (1, '725.340')] [2023-12-27 12:50:42,484][20347] Updated weights for policy 0, policy_version 1038414 (0.0006) [2023-12-27 12:50:42,486][20381] Updated weights for policy 1, policy_version 1037658 (0.0007) [2023-12-27 12:50:42,537][20381] Updated weights for policy 1, policy_version 1037668 (0.0006) [2023-12-27 12:50:42,545][20347] Updated weights for policy 0, policy_version 1038424 (0.0007) [2023-12-27 12:50:42,599][20381] Updated weights for policy 1, policy_version 1037678 (0.0007) [2023-12-27 12:50:42,608][20347] Updated weights for policy 0, policy_version 1038434 (0.0006) [2023-12-27 12:50:42,653][20381] Updated weights for policy 1, policy_version 1037688 (0.0006) [2023-12-27 12:50:43,230][20347] Updated weights for policy 0, policy_version 1038444 (0.0005) [2023-12-27 12:50:43,291][20347] Updated weights for policy 0, policy_version 1038454 (0.0005) [2023-12-27 12:50:43,339][20381] Updated weights for policy 1, policy_version 1037698 (0.0007) [2023-12-27 12:50:43,349][20347] Updated weights for policy 0, policy_version 1038464 (0.0005) [2023-12-27 12:50:43,392][20381] Updated weights for policy 1, policy_version 1037708 (0.0006) [2023-12-27 12:50:43,450][20381] Updated weights for policy 1, policy_version 1037718 (0.0008) [2023-12-27 12:50:43,963][20347] Updated weights for policy 0, policy_version 1038474 (0.0006) [2023-12-27 12:50:44,014][20347] Updated weights for policy 0, policy_version 1038484 (0.0010) [2023-12-27 12:50:44,070][20347] Updated weights for policy 0, policy_version 1038494 (0.0010) [2023-12-27 12:50:44,132][20381] Updated weights for policy 1, policy_version 1037728 (0.0007) [2023-12-27 12:50:44,132][20347] Updated weights for policy 0, policy_version 1038504 (0.0010) [2023-12-27 12:50:44,196][20381] Updated weights for policy 1, policy_version 1037738 (0.0006) [2023-12-27 12:50:44,246][20381] Updated weights for policy 1, policy_version 1037748 (0.0008) [2023-12-27 12:50:44,755][20347] Updated weights for policy 0, policy_version 1038514 (0.0011) [2023-12-27 12:50:44,820][20347] Updated weights for policy 0, policy_version 1038524 (0.0010) [2023-12-27 12:50:44,881][20291] KL-divergence is very high: 138.4131 [2023-12-27 12:50:44,882][20347] Updated weights for policy 0, policy_version 1038534 (0.0011) [2023-12-27 12:50:44,926][20381] Updated weights for policy 1, policy_version 1037758 (0.0007) [2023-12-27 12:50:44,979][20381] Updated weights for policy 1, policy_version 1037768 (0.0007) [2023-12-27 12:50:45,035][20381] Updated weights for policy 1, policy_version 1037778 (0.0007) [2023-12-27 12:50:45,569][20347] Updated weights for policy 0, policy_version 1038544 (0.0007) [2023-12-27 12:50:45,633][20347] Updated weights for policy 0, policy_version 1038554 (0.0006) [2023-12-27 12:50:45,676][20381] Updated weights for policy 1, policy_version 1037788 (0.0006) [2023-12-27 12:50:45,682][20347] Updated weights for policy 0, policy_version 1038564 (0.0008) [2023-12-27 12:50:45,733][20381] Updated weights for policy 1, policy_version 1037798 (0.0006) [2023-12-27 12:50:45,795][20381] Updated weights for policy 1, policy_version 1037808 (0.0008) [2023-12-27 12:50:46,316][20347] Updated weights for policy 0, policy_version 1038574 (0.0006) [2023-12-27 12:50:46,364][20347] Updated weights for policy 0, policy_version 1038584 (0.0007) [2023-12-27 12:50:46,417][20347] Updated weights for policy 0, policy_version 1038594 (0.0010) [2023-12-27 12:50:46,473][20381] Updated weights for policy 1, policy_version 1037818 (0.0008) [2023-12-27 12:50:46,532][20381] Updated weights for policy 1, policy_version 1037828 (0.0010) [2023-12-27 12:50:46,588][20381] Updated weights for policy 1, policy_version 1037838 (0.0010) [2023-12-27 12:50:46,646][20381] Updated weights for policy 1, policy_version 1037848 (0.0010) [2023-12-27 12:50:47,105][20347] Updated weights for policy 0, policy_version 1038604 (0.0011) [2023-12-27 12:50:47,169][20347] Updated weights for policy 0, policy_version 1038614 (0.0010) [2023-12-27 12:50:47,226][20347] Updated weights for policy 0, policy_version 1038624 (0.0010) [2023-12-27 12:50:47,303][20381] Updated weights for policy 1, policy_version 1037858 (0.0006) [2023-12-27 12:50:47,362][20381] Updated weights for policy 1, policy_version 1037868 (0.0007) [2023-12-27 12:50:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 531652608. Throughput: 0: 10121.2, 1: 10196.5. Samples: 531663356. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:47,364][19170] Avg episode reward: [(0, '748.450'), (1, '754.720')] [2023-12-27 12:50:47,422][20381] Updated weights for policy 1, policy_version 1037878 (0.0005) [2023-12-27 12:50:47,891][20347] Updated weights for policy 0, policy_version 1038634 (0.0010) [2023-12-27 12:50:47,942][20347] Updated weights for policy 0, policy_version 1038644 (0.0010) [2023-12-27 12:50:48,000][20347] Updated weights for policy 0, policy_version 1038654 (0.0010) [2023-12-27 12:50:48,015][20381] Updated weights for policy 1, policy_version 1037888 (0.0006) [2023-12-27 12:50:48,053][20347] Updated weights for policy 0, policy_version 1038664 (0.0010) [2023-12-27 12:50:48,070][20381] Updated weights for policy 1, policy_version 1037898 (0.0009) [2023-12-27 12:50:48,120][20381] Updated weights for policy 1, policy_version 1037908 (0.0010) [2023-12-27 12:50:48,748][20347] Updated weights for policy 0, policy_version 1038674 (0.0009) [2023-12-27 12:50:48,797][20381] Updated weights for policy 1, policy_version 1037918 (0.0009) [2023-12-27 12:50:48,799][20347] Updated weights for policy 0, policy_version 1038684 (0.0006) [2023-12-27 12:50:48,859][20347] Updated weights for policy 0, policy_version 1038694 (0.0008) [2023-12-27 12:50:48,859][20381] Updated weights for policy 1, policy_version 1037928 (0.0006) [2023-12-27 12:50:48,918][20381] Updated weights for policy 1, policy_version 1037938 (0.0006) [2023-12-27 12:50:49,573][20347] Updated weights for policy 0, policy_version 1038704 (0.0006) [2023-12-27 12:50:49,629][20347] Updated weights for policy 0, policy_version 1038714 (0.0008) [2023-12-27 12:50:49,653][20381] Updated weights for policy 1, policy_version 1037948 (0.0009) [2023-12-27 12:50:49,685][20347] Updated weights for policy 0, policy_version 1038724 (0.0007) [2023-12-27 12:50:49,715][20381] Updated weights for policy 1, policy_version 1037958 (0.0011) [2023-12-27 12:50:49,775][20381] Updated weights for policy 1, policy_version 1037968 (0.0010) [2023-12-27 12:50:50,401][20347] Updated weights for policy 0, policy_version 1038734 (0.0008) [2023-12-27 12:50:50,459][20347] Updated weights for policy 0, policy_version 1038744 (0.0007) [2023-12-27 12:50:50,491][20381] Updated weights for policy 1, policy_version 1037978 (0.0010) [2023-12-27 12:50:50,512][20347] Updated weights for policy 0, policy_version 1038754 (0.0010) [2023-12-27 12:50:50,534][20381] Updated weights for policy 1, policy_version 1037988 (0.0005) [2023-12-27 12:50:50,592][20381] Updated weights for policy 1, policy_version 1037998 (0.0007) [2023-12-27 12:50:50,651][20381] Updated weights for policy 1, policy_version 1038008 (0.0008) [2023-12-27 12:50:51,199][20347] Updated weights for policy 0, policy_version 1038764 (0.0008) [2023-12-27 12:50:51,258][20347] Updated weights for policy 0, policy_version 1038774 (0.0007) [2023-12-27 12:50:51,316][20347] Updated weights for policy 0, policy_version 1038784 (0.0010) [2023-12-27 12:50:51,424][20381] Updated weights for policy 1, policy_version 1038018 (0.0007) [2023-12-27 12:50:51,488][20381] Updated weights for policy 1, policy_version 1038028 (0.0007) [2023-12-27 12:50:51,551][20381] Updated weights for policy 1, policy_version 1038038 (0.0007) [2023-12-27 12:50:52,073][20347] Updated weights for policy 0, policy_version 1038794 (0.0009) [2023-12-27 12:50:52,133][20347] Updated weights for policy 0, policy_version 1038804 (0.0010) [2023-12-27 12:50:52,192][20347] Updated weights for policy 0, policy_version 1038814 (0.0010) [2023-12-27 12:50:52,252][20347] Updated weights for policy 0, policy_version 1038824 (0.0010) [2023-12-27 12:50:52,264][20381] Updated weights for policy 1, policy_version 1038048 (0.0007) [2023-12-27 12:50:52,324][20381] Updated weights for policy 1, policy_version 1038058 (0.0007) [2023-12-27 12:50:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 531750912. Throughput: 0: 10075.1, 1: 10175.1. Samples: 531723944. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:52,363][19170] Avg episode reward: [(0, '806.230'), (1, '735.350')] [2023-12-27 12:50:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001038824_265986048.pth... [2023-12-27 12:50:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001037640_265682944.pth [2023-12-27 12:50:52,380][20381] Updated weights for policy 1, policy_version 1038068 (0.0008) [2023-12-27 12:50:52,403][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001038072_265773056.pth... [2023-12-27 12:50:52,406][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001036856_265461760.pth [2023-12-27 12:50:52,982][20347] Updated weights for policy 0, policy_version 1038834 (0.0008) [2023-12-27 12:50:53,034][20347] Updated weights for policy 0, policy_version 1038844 (0.0010) [2023-12-27 12:50:53,081][20347] Updated weights for policy 0, policy_version 1038854 (0.0008) [2023-12-27 12:50:53,088][20381] Updated weights for policy 1, policy_version 1038078 (0.0007) [2023-12-27 12:50:53,144][20381] Updated weights for policy 1, policy_version 1038088 (0.0007) [2023-12-27 12:50:53,195][20381] Updated weights for policy 1, policy_version 1038098 (0.0007) [2023-12-27 12:50:53,748][20347] Updated weights for policy 0, policy_version 1038864 (0.0006) [2023-12-27 12:50:53,805][20347] Updated weights for policy 0, policy_version 1038874 (0.0005) [2023-12-27 12:50:53,873][20347] Updated weights for policy 0, policy_version 1038884 (0.0006) [2023-12-27 12:50:53,934][20381] Updated weights for policy 1, policy_version 1038108 (0.0008) [2023-12-27 12:50:53,991][20381] Updated weights for policy 1, policy_version 1038118 (0.0009) [2023-12-27 12:50:54,050][20381] Updated weights for policy 1, policy_version 1038128 (0.0008) [2023-12-27 12:50:54,496][20347] Updated weights for policy 0, policy_version 1038894 (0.0007) [2023-12-27 12:50:54,508][20291] KL-divergence is very high: 110.4358 [2023-12-27 12:50:54,525][20291] KL-divergence is very high: 339.9920 [2023-12-27 12:50:54,545][20347] Updated weights for policy 0, policy_version 1038904 (0.0007) [2023-12-27 12:50:54,548][20291] KL-divergence is very high: 207.9633 [2023-12-27 12:50:54,562][20291] KL-divergence is very high: 315.0833 [2023-12-27 12:50:54,591][20347] Updated weights for policy 0, policy_version 1038914 (0.0009) [2023-12-27 12:50:54,603][20291] KL-divergence is very high: 101.4666 [2023-12-27 12:50:54,684][20381] Updated weights for policy 1, policy_version 1038138 (0.0008) [2023-12-27 12:50:54,736][20381] Updated weights for policy 1, policy_version 1038148 (0.0010) [2023-12-27 12:50:54,782][20381] Updated weights for policy 1, policy_version 1038158 (0.0010) [2023-12-27 12:50:54,828][20381] Updated weights for policy 1, policy_version 1038168 (0.0010) [2023-12-27 12:50:55,308][20347] Updated weights for policy 0, policy_version 1038924 (0.0008) [2023-12-27 12:50:55,362][20347] Updated weights for policy 0, policy_version 1038934 (0.0007) [2023-12-27 12:50:55,415][20347] Updated weights for policy 0, policy_version 1038944 (0.0007) [2023-12-27 12:50:55,538][20381] Updated weights for policy 1, policy_version 1038178 (0.0010) [2023-12-27 12:50:55,592][20381] Updated weights for policy 1, policy_version 1038188 (0.0010) [2023-12-27 12:50:55,640][20381] Updated weights for policy 1, policy_version 1038198 (0.0010) [2023-12-27 12:50:56,005][20347] Updated weights for policy 0, policy_version 1038954 (0.0007) [2023-12-27 12:50:56,065][20347] Updated weights for policy 0, policy_version 1038964 (0.0005) [2023-12-27 12:50:56,097][20291] KL-divergence is very high: 102.6332 [2023-12-27 12:50:56,127][20347] Updated weights for policy 0, policy_version 1038974 (0.0005) [2023-12-27 12:50:56,185][20347] Updated weights for policy 0, policy_version 1038984 (0.0005) [2023-12-27 12:50:56,334][20381] Updated weights for policy 1, policy_version 1038208 (0.0010) [2023-12-27 12:50:56,385][20381] Updated weights for policy 1, policy_version 1038218 (0.0010) [2023-12-27 12:50:56,432][20381] Updated weights for policy 1, policy_version 1038228 (0.0010) [2023-12-27 12:50:56,756][20347] Updated weights for policy 0, policy_version 1038994 (0.0007) [2023-12-27 12:50:56,802][20347] Updated weights for policy 0, policy_version 1039004 (0.0007) [2023-12-27 12:50:56,859][20347] Updated weights for policy 0, policy_version 1039014 (0.0007) [2023-12-27 12:50:57,127][20381] Updated weights for policy 1, policy_version 1038238 (0.0010) [2023-12-27 12:50:57,189][20381] Updated weights for policy 1, policy_version 1038248 (0.0011) [2023-12-27 12:50:57,242][20381] Updated weights for policy 1, policy_version 1038258 (0.0010) [2023-12-27 12:50:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 531857408. Throughput: 0: 10107.1, 1: 10176.1. Samples: 531845144. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:50:57,363][19170] Avg episode reward: [(0, '902.910'), (1, '627.110')] [2023-12-27 12:50:57,548][20347] Updated weights for policy 0, policy_version 1039024 (0.0006) [2023-12-27 12:50:57,602][20347] Updated weights for policy 0, policy_version 1039034 (0.0005) [2023-12-27 12:50:57,648][20347] Updated weights for policy 0, policy_version 1039044 (0.0005) [2023-12-27 12:50:57,903][20381] Updated weights for policy 1, policy_version 1038268 (0.0010) [2023-12-27 12:50:57,951][20381] Updated weights for policy 1, policy_version 1038278 (0.0010) [2023-12-27 12:50:58,003][20381] Updated weights for policy 1, policy_version 1038288 (0.0010) [2023-12-27 12:50:58,218][20347] Updated weights for policy 0, policy_version 1039054 (0.0007) [2023-12-27 12:50:58,275][20347] Updated weights for policy 0, policy_version 1039064 (0.0007) [2023-12-27 12:50:58,339][20347] Updated weights for policy 0, policy_version 1039074 (0.0008) [2023-12-27 12:50:58,796][20381] Updated weights for policy 1, policy_version 1038298 (0.0010) [2023-12-27 12:50:58,859][20381] Updated weights for policy 1, policy_version 1038308 (0.0007) [2023-12-27 12:50:58,925][20381] Updated weights for policy 1, policy_version 1038318 (0.0008) [2023-12-27 12:50:58,986][20381] Updated weights for policy 1, policy_version 1038328 (0.0010) [2023-12-27 12:50:59,115][20347] Updated weights for policy 0, policy_version 1039084 (0.0009) [2023-12-27 12:50:59,177][20347] Updated weights for policy 0, policy_version 1039094 (0.0008) [2023-12-27 12:50:59,237][20347] Updated weights for policy 0, policy_version 1039104 (0.0008) [2023-12-27 12:50:59,683][20381] Updated weights for policy 1, policy_version 1038338 (0.0007) [2023-12-27 12:50:59,743][20381] Updated weights for policy 1, policy_version 1038348 (0.0008) [2023-12-27 12:50:59,799][20381] Updated weights for policy 1, policy_version 1038358 (0.0008) [2023-12-27 12:50:59,906][20347] Updated weights for policy 0, policy_version 1039114 (0.0007) [2023-12-27 12:50:59,969][20347] Updated weights for policy 0, policy_version 1039124 (0.0007) [2023-12-27 12:51:00,027][20347] Updated weights for policy 0, policy_version 1039134 (0.0008) [2023-12-27 12:51:00,080][20347] Updated weights for policy 0, policy_version 1039144 (0.0007) [2023-12-27 12:51:00,548][20381] Updated weights for policy 1, policy_version 1038368 (0.0006) [2023-12-27 12:51:00,604][20381] Updated weights for policy 1, policy_version 1038378 (0.0005) [2023-12-27 12:51:00,662][20381] Updated weights for policy 1, policy_version 1038388 (0.0009) [2023-12-27 12:51:00,774][20347] Updated weights for policy 0, policy_version 1039154 (0.0007) [2023-12-27 12:51:00,833][20347] Updated weights for policy 0, policy_version 1039164 (0.0005) [2023-12-27 12:51:00,888][20347] Updated weights for policy 0, policy_version 1039174 (0.0005) [2023-12-27 12:51:01,260][20381] Updated weights for policy 1, policy_version 1038398 (0.0008) [2023-12-27 12:51:01,326][20381] Updated weights for policy 1, policy_version 1038408 (0.0008) [2023-12-27 12:51:01,391][20381] Updated weights for policy 1, policy_version 1038418 (0.0008) [2023-12-27 12:51:01,537][20347] Updated weights for policy 0, policy_version 1039184 (0.0005) [2023-12-27 12:51:01,605][20347] Updated weights for policy 0, policy_version 1039194 (0.0008) [2023-12-27 12:51:01,674][20347] Updated weights for policy 0, policy_version 1039204 (0.0008) [2023-12-27 12:51:02,064][20381] Updated weights for policy 1, policy_version 1038428 (0.0008) [2023-12-27 12:51:02,113][20381] Updated weights for policy 1, policy_version 1038438 (0.0007) [2023-12-27 12:51:02,163][20381] Updated weights for policy 1, policy_version 1038448 (0.0008) [2023-12-27 12:51:02,306][20347] Updated weights for policy 0, policy_version 1039214 (0.0008) [2023-12-27 12:51:02,362][20291] KL-divergence is very high: 118.8527 [2023-12-27 12:51:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 531955712. Throughput: 0: 10150.0, 1: 10126.8. Samples: 531966328. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:51:02,363][19170] Avg episode reward: [(0, '893.030'), (1, '705.630')] [2023-12-27 12:51:02,367][20347] Updated weights for policy 0, policy_version 1039224 (0.0008) [2023-12-27 12:51:02,398][20291] KL-divergence is very high: 113.4019 [2023-12-27 12:51:02,425][20347] Updated weights for policy 0, policy_version 1039234 (0.0008) [2023-12-27 12:51:02,916][20381] Updated weights for policy 1, policy_version 1038458 (0.0007) [2023-12-27 12:51:02,971][20381] Updated weights for policy 1, policy_version 1038468 (0.0008) [2023-12-27 12:51:03,027][20381] Updated weights for policy 1, policy_version 1038478 (0.0007) [2023-12-27 12:51:03,078][20381] Updated weights for policy 1, policy_version 1038488 (0.0008) [2023-12-27 12:51:03,080][20347] Updated weights for policy 0, policy_version 1039244 (0.0007) [2023-12-27 12:51:03,137][20347] Updated weights for policy 0, policy_version 1039254 (0.0007) [2023-12-27 12:51:03,187][20347] Updated weights for policy 0, policy_version 1039264 (0.0007) [2023-12-27 12:51:03,746][20381] Updated weights for policy 1, policy_version 1038498 (0.0010) [2023-12-27 12:51:03,794][20381] Updated weights for policy 1, policy_version 1038508 (0.0010) [2023-12-27 12:51:03,845][20381] Updated weights for policy 1, policy_version 1038518 (0.0008) [2023-12-27 12:51:03,922][20347] Updated weights for policy 0, policy_version 1039274 (0.0007) [2023-12-27 12:51:03,978][20347] Updated weights for policy 0, policy_version 1039284 (0.0007) [2023-12-27 12:51:04,036][20347] Updated weights for policy 0, policy_version 1039294 (0.0007) [2023-12-27 12:51:04,095][20347] Updated weights for policy 0, policy_version 1039304 (0.0007) [2023-12-27 12:51:04,588][20381] Updated weights for policy 1, policy_version 1038528 (0.0008) [2023-12-27 12:51:04,648][20381] Updated weights for policy 1, policy_version 1038538 (0.0008) [2023-12-27 12:51:04,709][20381] Updated weights for policy 1, policy_version 1038548 (0.0007) [2023-12-27 12:51:04,776][20347] Updated weights for policy 0, policy_version 1039314 (0.0007) [2023-12-27 12:51:04,832][20347] Updated weights for policy 0, policy_version 1039324 (0.0007) [2023-12-27 12:51:04,890][20347] Updated weights for policy 0, policy_version 1039334 (0.0007) [2023-12-27 12:51:05,373][20381] Updated weights for policy 1, policy_version 1038558 (0.0007) [2023-12-27 12:51:05,431][20381] Updated weights for policy 1, policy_version 1038568 (0.0008) [2023-12-27 12:51:05,476][20381] Updated weights for policy 1, policy_version 1038578 (0.0008) [2023-12-27 12:51:05,517][20347] Updated weights for policy 0, policy_version 1039344 (0.0007) [2023-12-27 12:51:05,575][20347] Updated weights for policy 0, policy_version 1039354 (0.0008) [2023-12-27 12:51:05,629][20347] Updated weights for policy 0, policy_version 1039364 (0.0007) [2023-12-27 12:51:06,168][20381] Updated weights for policy 1, policy_version 1038588 (0.0007) [2023-12-27 12:51:06,225][20381] Updated weights for policy 1, policy_version 1038598 (0.0008) [2023-12-27 12:51:06,276][20381] Updated weights for policy 1, policy_version 1038608 (0.0011) [2023-12-27 12:51:06,327][20347] Updated weights for policy 0, policy_version 1039374 (0.0009) [2023-12-27 12:51:06,389][20347] Updated weights for policy 0, policy_version 1039384 (0.0011) [2023-12-27 12:51:06,451][20347] Updated weights for policy 0, policy_version 1039394 (0.0011) [2023-12-27 12:51:06,999][20381] Updated weights for policy 1, policy_version 1038618 (0.0011) [2023-12-27 12:51:07,057][20381] Updated weights for policy 1, policy_version 1038628 (0.0010) [2023-12-27 12:51:07,121][20381] Updated weights for policy 1, policy_version 1038638 (0.0007) [2023-12-27 12:51:07,133][20347] Updated weights for policy 0, policy_version 1039404 (0.0011) [2023-12-27 12:51:07,177][20381] Updated weights for policy 1, policy_version 1038648 (0.0008) [2023-12-27 12:51:07,186][20347] Updated weights for policy 0, policy_version 1039414 (0.0010) [2023-12-27 12:51:07,243][20347] Updated weights for policy 0, policy_version 1039424 (0.0007) [2023-12-27 12:51:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 532062208. Throughput: 0: 10173.4, 1: 10155.8. Samples: 532026988. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:51:07,364][19170] Avg episode reward: [(0, '892.530'), (1, '754.670')] [2023-12-27 12:51:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001038648_265920512.pth... [2023-12-27 12:51:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001039432_266141696.pth... [2023-12-27 12:51:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001038216_265830400.pth [2023-12-27 12:51:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001037464_265617408.pth [2023-12-27 12:51:07,863][20347] Updated weights for policy 0, policy_version 1039434 (0.0006) [2023-12-27 12:51:07,865][20381] Updated weights for policy 1, policy_version 1038658 (0.0011) [2023-12-27 12:51:07,914][20347] Updated weights for policy 0, policy_version 1039444 (0.0010) [2023-12-27 12:51:07,925][20381] Updated weights for policy 1, policy_version 1038668 (0.0009) [2023-12-27 12:51:07,968][20347] Updated weights for policy 0, policy_version 1039454 (0.0010) [2023-12-27 12:51:07,986][20381] Updated weights for policy 1, policy_version 1038678 (0.0008) [2023-12-27 12:51:08,027][20347] Updated weights for policy 0, policy_version 1039464 (0.0011) [2023-12-27 12:51:08,678][20347] Updated weights for policy 0, policy_version 1039474 (0.0010) [2023-12-27 12:51:08,712][20381] Updated weights for policy 1, policy_version 1038688 (0.0010) [2023-12-27 12:51:08,734][20347] Updated weights for policy 0, policy_version 1039484 (0.0010) [2023-12-27 12:51:08,764][20381] Updated weights for policy 1, policy_version 1038698 (0.0010) [2023-12-27 12:51:08,797][20347] Updated weights for policy 0, policy_version 1039494 (0.0010) [2023-12-27 12:51:08,823][20381] Updated weights for policy 1, policy_version 1038708 (0.0011) [2023-12-27 12:51:09,492][20347] Updated weights for policy 0, policy_version 1039504 (0.0008) [2023-12-27 12:51:09,521][20381] Updated weights for policy 1, policy_version 1038718 (0.0009) [2023-12-27 12:51:09,557][20347] Updated weights for policy 0, policy_version 1039514 (0.0007) [2023-12-27 12:51:09,581][20381] Updated weights for policy 1, policy_version 1038728 (0.0009) [2023-12-27 12:51:09,614][20347] Updated weights for policy 0, policy_version 1039524 (0.0008) [2023-12-27 12:51:09,640][20381] Updated weights for policy 1, policy_version 1038738 (0.0010) [2023-12-27 12:51:10,282][20347] Updated weights for policy 0, policy_version 1039534 (0.0007) [2023-12-27 12:51:10,348][20347] Updated weights for policy 0, policy_version 1039544 (0.0007) [2023-12-27 12:51:10,367][20381] Updated weights for policy 1, policy_version 1038748 (0.0009) [2023-12-27 12:51:10,411][20347] Updated weights for policy 0, policy_version 1039554 (0.0006) [2023-12-27 12:51:10,421][20381] Updated weights for policy 1, policy_version 1038758 (0.0008) [2023-12-27 12:51:10,480][20381] Updated weights for policy 1, policy_version 1038768 (0.0007) [2023-12-27 12:51:11,033][20347] Updated weights for policy 0, policy_version 1039564 (0.0007) [2023-12-27 12:51:11,096][20347] Updated weights for policy 0, policy_version 1039574 (0.0007) [2023-12-27 12:51:11,129][20291] KL-divergence is very high: 108.0728 [2023-12-27 12:51:11,165][20347] Updated weights for policy 0, policy_version 1039584 (0.0008) [2023-12-27 12:51:11,182][20291] KL-divergence is very high: 129.9109 [2023-12-27 12:51:11,221][20381] Updated weights for policy 1, policy_version 1038778 (0.0008) [2023-12-27 12:51:11,289][20381] Updated weights for policy 1, policy_version 1038788 (0.0008) [2023-12-27 12:51:11,363][20381] Updated weights for policy 1, policy_version 1038798 (0.0009) [2023-12-27 12:51:11,431][20381] Updated weights for policy 1, policy_version 1038808 (0.0008) [2023-12-27 12:51:11,805][20347] Updated weights for policy 0, policy_version 1039594 (0.0007) [2023-12-27 12:51:11,870][20347] Updated weights for policy 0, policy_version 1039604 (0.0008) [2023-12-27 12:51:11,928][20347] Updated weights for policy 0, policy_version 1039614 (0.0008) [2023-12-27 12:51:11,988][20347] Updated weights for policy 0, policy_version 1039624 (0.0008) [2023-12-27 12:51:12,135][20381] Updated weights for policy 1, policy_version 1038818 (0.0006) [2023-12-27 12:51:12,198][20381] Updated weights for policy 1, policy_version 1038828 (0.0006) [2023-12-27 12:51:12,265][20381] Updated weights for policy 1, policy_version 1038838 (0.0006) [2023-12-27 12:51:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 532160512. Throughput: 0: 10245.4, 1: 10084.7. Samples: 532148680. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:51:12,364][19170] Avg episode reward: [(0, '814.360'), (1, '627.360')] [2023-12-27 12:51:12,740][20347] Updated weights for policy 0, policy_version 1039634 (0.0008) [2023-12-27 12:51:12,794][20347] Updated weights for policy 0, policy_version 1039644 (0.0007) [2023-12-27 12:51:12,843][20347] Updated weights for policy 0, policy_version 1039654 (0.0008) [2023-12-27 12:51:12,903][20381] Updated weights for policy 1, policy_version 1038848 (0.0008) [2023-12-27 12:51:12,963][20381] Updated weights for policy 1, policy_version 1038858 (0.0008) [2023-12-27 12:51:13,027][20381] Updated weights for policy 1, policy_version 1038868 (0.0008) [2023-12-27 12:51:13,487][20347] Updated weights for policy 0, policy_version 1039664 (0.0006) [2023-12-27 12:51:13,546][20347] Updated weights for policy 0, policy_version 1039674 (0.0006) [2023-12-27 12:51:13,597][20347] Updated weights for policy 0, policy_version 1039684 (0.0008) [2023-12-27 12:51:13,687][20381] Updated weights for policy 1, policy_version 1038878 (0.0007) [2023-12-27 12:51:13,739][20381] Updated weights for policy 1, policy_version 1038888 (0.0005) [2023-12-27 12:51:13,789][20381] Updated weights for policy 1, policy_version 1038898 (0.0007) [2023-12-27 12:51:14,293][20347] Updated weights for policy 0, policy_version 1039694 (0.0007) [2023-12-27 12:51:14,343][20347] Updated weights for policy 0, policy_version 1039704 (0.0008) [2023-12-27 12:51:14,392][20381] Updated weights for policy 1, policy_version 1038908 (0.0007) [2023-12-27 12:51:14,403][20347] Updated weights for policy 0, policy_version 1039714 (0.0007) [2023-12-27 12:51:14,450][20381] Updated weights for policy 1, policy_version 1038918 (0.0006) [2023-12-27 12:51:14,515][20381] Updated weights for policy 1, policy_version 1038928 (0.0006) [2023-12-27 12:51:15,156][20381] Updated weights for policy 1, policy_version 1038938 (0.0009) [2023-12-27 12:51:15,177][20347] Updated weights for policy 0, policy_version 1039724 (0.0006) [2023-12-27 12:51:15,213][20381] Updated weights for policy 1, policy_version 1038948 (0.0011) [2023-12-27 12:51:15,234][20347] Updated weights for policy 0, policy_version 1039734 (0.0005) [2023-12-27 12:51:15,273][20381] Updated weights for policy 1, policy_version 1038958 (0.0011) [2023-12-27 12:51:15,286][20347] Updated weights for policy 0, policy_version 1039744 (0.0006) [2023-12-27 12:51:15,325][20381] Updated weights for policy 1, policy_version 1038968 (0.0010) [2023-12-27 12:51:15,971][20381] Updated weights for policy 1, policy_version 1038978 (0.0005) [2023-12-27 12:51:16,025][20347] Updated weights for policy 0, policy_version 1039754 (0.0005) [2023-12-27 12:51:16,028][20381] Updated weights for policy 1, policy_version 1038988 (0.0007) [2023-12-27 12:51:16,074][20347] Updated weights for policy 0, policy_version 1039764 (0.0005) [2023-12-27 12:51:16,084][20381] Updated weights for policy 1, policy_version 1038998 (0.0011) [2023-12-27 12:51:16,124][20347] Updated weights for policy 0, policy_version 1039774 (0.0007) [2023-12-27 12:51:16,178][20347] Updated weights for policy 0, policy_version 1039784 (0.0007) [2023-12-27 12:51:16,714][20381] Updated weights for policy 1, policy_version 1039008 (0.0011) [2023-12-27 12:51:16,760][20381] Updated weights for policy 1, policy_version 1039018 (0.0010) [2023-12-27 12:51:16,809][20381] Updated weights for policy 1, policy_version 1039028 (0.0010) [2023-12-27 12:51:16,881][20347] Updated weights for policy 0, policy_version 1039794 (0.0008) [2023-12-27 12:51:16,942][20347] Updated weights for policy 0, policy_version 1039804 (0.0007) [2023-12-27 12:51:17,000][20347] Updated weights for policy 0, policy_version 1039814 (0.0007) [2023-12-27 12:51:17,363][19170] Fps is (10 sec: 19661.1, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 532258816. Throughput: 0: 10202.1, 1: 10096.3. Samples: 532270588. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:51:17,364][19170] Avg episode reward: [(0, '834.120'), (1, '676.480')] [2023-12-27 12:51:17,514][20381] Updated weights for policy 1, policy_version 1039038 (0.0011) [2023-12-27 12:51:17,575][20381] Updated weights for policy 1, policy_version 1039048 (0.0011) [2023-12-27 12:51:17,632][20381] Updated weights for policy 1, policy_version 1039058 (0.0011) [2023-12-27 12:51:17,645][20347] Updated weights for policy 0, policy_version 1039824 (0.0006) [2023-12-27 12:51:17,699][20347] Updated weights for policy 0, policy_version 1039834 (0.0006) [2023-12-27 12:51:17,762][20347] Updated weights for policy 0, policy_version 1039844 (0.0006) [2023-12-27 12:51:18,291][20381] Updated weights for policy 1, policy_version 1039068 (0.0010) [2023-12-27 12:51:18,344][20381] Updated weights for policy 1, policy_version 1039078 (0.0009) [2023-12-27 12:51:18,403][20381] Updated weights for policy 1, policy_version 1039088 (0.0011) [2023-12-27 12:51:18,422][20347] Updated weights for policy 0, policy_version 1039854 (0.0008) [2023-12-27 12:51:18,483][20347] Updated weights for policy 0, policy_version 1039864 (0.0006) [2023-12-27 12:51:18,541][20347] Updated weights for policy 0, policy_version 1039874 (0.0007) [2023-12-27 12:51:19,077][20381] Updated weights for policy 1, policy_version 1039098 (0.0010) [2023-12-27 12:51:19,125][20381] Updated weights for policy 1, policy_version 1039108 (0.0010) [2023-12-27 12:51:19,187][20381] Updated weights for policy 1, policy_version 1039118 (0.0011) [2023-12-27 12:51:19,252][20347] Updated weights for policy 0, policy_version 1039884 (0.0007) [2023-12-27 12:51:19,256][20381] Updated weights for policy 1, policy_version 1039128 (0.0009) [2023-12-27 12:51:19,315][20347] Updated weights for policy 0, policy_version 1039894 (0.0006) [2023-12-27 12:51:19,389][20347] Updated weights for policy 0, policy_version 1039904 (0.0008) [2023-12-27 12:51:19,981][20381] Updated weights for policy 1, policy_version 1039138 (0.0008) [2023-12-27 12:51:20,045][20381] Updated weights for policy 1, policy_version 1039148 (0.0007) [2023-12-27 12:51:20,107][20381] Updated weights for policy 1, policy_version 1039158 (0.0008) [2023-12-27 12:51:20,112][20347] Updated weights for policy 0, policy_version 1039914 (0.0007) [2023-12-27 12:51:20,166][20347] Updated weights for policy 0, policy_version 1039924 (0.0008) [2023-12-27 12:51:20,235][20347] Updated weights for policy 0, policy_version 1039934 (0.0008) [2023-12-27 12:51:20,305][20347] Updated weights for policy 0, policy_version 1039944 (0.0008) [2023-12-27 12:51:20,762][20381] Updated weights for policy 1, policy_version 1039168 (0.0007) [2023-12-27 12:51:20,820][20381] Updated weights for policy 1, policy_version 1039178 (0.0008) [2023-12-27 12:51:20,871][20381] Updated weights for policy 1, policy_version 1039188 (0.0007) [2023-12-27 12:51:21,005][20347] Updated weights for policy 0, policy_version 1039954 (0.0008) [2023-12-27 12:51:21,073][20347] Updated weights for policy 0, policy_version 1039964 (0.0009) [2023-12-27 12:51:21,136][20347] Updated weights for policy 0, policy_version 1039974 (0.0007) [2023-12-27 12:51:21,660][20381] Updated weights for policy 1, policy_version 1039198 (0.0009) [2023-12-27 12:51:21,725][20381] Updated weights for policy 1, policy_version 1039208 (0.0009) [2023-12-27 12:51:21,785][20381] Updated weights for policy 1, policy_version 1039218 (0.0008) [2023-12-27 12:51:21,829][20347] Updated weights for policy 0, policy_version 1039984 (0.0007) [2023-12-27 12:51:21,890][20347] Updated weights for policy 0, policy_version 1039994 (0.0008) [2023-12-27 12:51:21,939][20347] Updated weights for policy 0, policy_version 1040004 (0.0009) [2023-12-27 12:51:22,366][19170] Fps is (10 sec: 19654.2, 60 sec: 20205.8, 300 sec: 20493.6). Total num frames: 532357120. Throughput: 0: 10192.3, 1: 10047.1. Samples: 532330584. Policy #0 lag: (min: 2.0, avg: 16.6, max: 34.0) [2023-12-27 12:51:22,366][19170] Avg episode reward: [(0, '921.960'), (1, '656.770')] [2023-12-27 12:51:22,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001039224_266067968.pth... [2023-12-27 12:51:22,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001040008_266289152.pth... [2023-12-27 12:51:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001038072_265773056.pth [2023-12-27 12:51:22,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001038824_265986048.pth [2023-12-27 12:51:22,432][20292] KL-divergence is very high: 229.5992 [2023-12-27 12:51:22,438][20381] Updated weights for policy 1, policy_version 1039228 (0.0009) [2023-12-27 12:51:22,480][20292] KL-divergence is very high: 213.7157 [2023-12-27 12:51:22,497][20381] Updated weights for policy 1, policy_version 1039238 (0.0005) [2023-12-27 12:51:22,529][20292] KL-divergence is very high: 156.3177 [2023-12-27 12:51:22,560][20381] Updated weights for policy 1, policy_version 1039248 (0.0010) [2023-12-27 12:51:22,664][20347] Updated weights for policy 0, policy_version 1040014 (0.0007) [2023-12-27 12:51:22,716][20347] Updated weights for policy 0, policy_version 1040024 (0.0007) [2023-12-27 12:51:22,764][20347] Updated weights for policy 0, policy_version 1040034 (0.0006) [2023-12-27 12:51:23,195][20381] Updated weights for policy 1, policy_version 1039258 (0.0010) [2023-12-27 12:51:23,249][20381] Updated weights for policy 1, policy_version 1039268 (0.0010) [2023-12-27 12:51:23,303][20381] Updated weights for policy 1, policy_version 1039278 (0.0010) [2023-12-27 12:51:23,357][20381] Updated weights for policy 1, policy_version 1039288 (0.0010) [2023-12-27 12:51:23,399][20347] Updated weights for policy 0, policy_version 1040044 (0.0007) [2023-12-27 12:51:23,446][20347] Updated weights for policy 0, policy_version 1040054 (0.0005) [2023-12-27 12:51:23,490][20347] Updated weights for policy 0, policy_version 1040064 (0.0008) [2023-12-27 12:51:23,975][20381] Updated weights for policy 1, policy_version 1039298 (0.0006) [2023-12-27 12:51:24,040][20381] Updated weights for policy 1, policy_version 1039308 (0.0005) [2023-12-27 12:51:24,095][20381] Updated weights for policy 1, policy_version 1039318 (0.0006) [2023-12-27 12:51:24,228][20347] Updated weights for policy 0, policy_version 1040074 (0.0007) [2023-12-27 12:51:24,279][20347] Updated weights for policy 0, policy_version 1040084 (0.0007) [2023-12-27 12:51:24,324][20347] Updated weights for policy 0, policy_version 1040094 (0.0007) [2023-12-27 12:51:24,373][20347] Updated weights for policy 0, policy_version 1040104 (0.0005) [2023-12-27 12:51:24,720][20381] Updated weights for policy 1, policy_version 1039328 (0.0005) [2023-12-27 12:51:24,772][20381] Updated weights for policy 1, policy_version 1039338 (0.0011) [2023-12-27 12:51:24,824][20381] Updated weights for policy 1, policy_version 1039348 (0.0010) [2023-12-27 12:51:24,966][20347] Updated weights for policy 0, policy_version 1040114 (0.0005) [2023-12-27 12:51:25,017][20347] Updated weights for policy 0, policy_version 1040124 (0.0005) [2023-12-27 12:51:25,068][20347] Updated weights for policy 0, policy_version 1040134 (0.0005) [2023-12-27 12:51:25,481][20381] Updated weights for policy 1, policy_version 1039358 (0.0011) [2023-12-27 12:51:25,539][20381] Updated weights for policy 1, policy_version 1039368 (0.0011) [2023-12-27 12:51:25,590][20381] Updated weights for policy 1, policy_version 1039378 (0.0010) [2023-12-27 12:51:25,666][20347] Updated weights for policy 0, policy_version 1040144 (0.0007) [2023-12-27 12:51:25,723][20347] Updated weights for policy 0, policy_version 1040154 (0.0007) [2023-12-27 12:51:25,776][20347] Updated weights for policy 0, policy_version 1040164 (0.0007) [2023-12-27 12:51:26,248][20381] Updated weights for policy 1, policy_version 1039388 (0.0011) [2023-12-27 12:51:26,297][20381] Updated weights for policy 1, policy_version 1039398 (0.0011) [2023-12-27 12:51:26,351][20381] Updated weights for policy 1, policy_version 1039408 (0.0010) [2023-12-27 12:51:26,483][20347] Updated weights for policy 0, policy_version 1040174 (0.0008) [2023-12-27 12:51:26,528][20347] Updated weights for policy 0, policy_version 1040184 (0.0009) [2023-12-27 12:51:26,577][20347] Updated weights for policy 0, policy_version 1040194 (0.0009) [2023-12-27 12:51:26,971][20381] Updated weights for policy 1, policy_version 1039418 (0.0010) [2023-12-27 12:51:27,031][20381] Updated weights for policy 1, policy_version 1039428 (0.0010) [2023-12-27 12:51:27,088][20381] Updated weights for policy 1, policy_version 1039438 (0.0010) [2023-12-27 12:51:27,145][20381] Updated weights for policy 1, policy_version 1039448 (0.0010) [2023-12-27 12:51:27,307][20347] Updated weights for policy 0, policy_version 1040204 (0.0009) [2023-12-27 12:51:27,358][20347] Updated weights for policy 0, policy_version 1040214 (0.0006) [2023-12-27 12:51:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 532463616. Throughput: 0: 10213.8, 1: 10111.2. Samples: 532455008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:27,363][19170] Avg episode reward: [(0, '951.480'), (1, '715.470')] [2023-12-27 12:51:27,407][20347] Updated weights for policy 0, policy_version 1040224 (0.0007) [2023-12-27 12:51:27,746][20381] Updated weights for policy 1, policy_version 1039458 (0.0011) [2023-12-27 12:51:27,798][20381] Updated weights for policy 1, policy_version 1039468 (0.0010) [2023-12-27 12:51:27,846][20381] Updated weights for policy 1, policy_version 1039478 (0.0010) [2023-12-27 12:51:28,114][20347] Updated weights for policy 0, policy_version 1040234 (0.0005) [2023-12-27 12:51:28,158][20347] Updated weights for policy 0, policy_version 1040244 (0.0008) [2023-12-27 12:51:28,205][20347] Updated weights for policy 0, policy_version 1040254 (0.0007) [2023-12-27 12:51:28,256][20347] Updated weights for policy 0, policy_version 1040264 (0.0005) [2023-12-27 12:51:28,486][20381] Updated weights for policy 1, policy_version 1039488 (0.0011) [2023-12-27 12:51:28,538][20381] Updated weights for policy 1, policy_version 1039498 (0.0010) [2023-12-27 12:51:28,587][20381] Updated weights for policy 1, policy_version 1039508 (0.0010) [2023-12-27 12:51:28,938][20347] Updated weights for policy 0, policy_version 1040274 (0.0007) [2023-12-27 12:51:28,984][20347] Updated weights for policy 0, policy_version 1040284 (0.0007) [2023-12-27 12:51:29,031][20347] Updated weights for policy 0, policy_version 1040294 (0.0007) [2023-12-27 12:51:29,308][20381] Updated weights for policy 1, policy_version 1039518 (0.0009) [2023-12-27 12:51:29,402][20381] Updated weights for policy 1, policy_version 1039528 (0.0008) [2023-12-27 12:51:29,456][20381] Updated weights for policy 1, policy_version 1039538 (0.0008) [2023-12-27 12:51:29,724][20347] Updated weights for policy 0, policy_version 1040304 (0.0006) [2023-12-27 12:51:29,783][20347] Updated weights for policy 0, policy_version 1040314 (0.0006) [2023-12-27 12:51:29,839][20347] Updated weights for policy 0, policy_version 1040324 (0.0008) [2023-12-27 12:51:30,180][20381] Updated weights for policy 1, policy_version 1039548 (0.0008) [2023-12-27 12:51:30,242][20381] Updated weights for policy 1, policy_version 1039558 (0.0007) [2023-12-27 12:51:30,298][20381] Updated weights for policy 1, policy_version 1039568 (0.0007) [2023-12-27 12:51:30,520][20347] Updated weights for policy 0, policy_version 1040334 (0.0007) [2023-12-27 12:51:30,571][20347] Updated weights for policy 0, policy_version 1040344 (0.0006) [2023-12-27 12:51:30,629][20347] Updated weights for policy 0, policy_version 1040354 (0.0006) [2023-12-27 12:51:30,930][20381] Updated weights for policy 1, policy_version 1039578 (0.0007) [2023-12-27 12:51:30,976][20381] Updated weights for policy 1, policy_version 1039588 (0.0007) [2023-12-27 12:51:31,028][20381] Updated weights for policy 1, policy_version 1039598 (0.0006) [2023-12-27 12:51:31,093][20381] Updated weights for policy 1, policy_version 1039608 (0.0007) [2023-12-27 12:51:31,334][20347] Updated weights for policy 0, policy_version 1040364 (0.0007) [2023-12-27 12:51:31,408][20347] Updated weights for policy 0, policy_version 1040374 (0.0008) [2023-12-27 12:51:31,470][20347] Updated weights for policy 0, policy_version 1040384 (0.0006) [2023-12-27 12:51:31,816][20381] Updated weights for policy 1, policy_version 1039618 (0.0008) [2023-12-27 12:51:31,876][20381] Updated weights for policy 1, policy_version 1039628 (0.0006) [2023-12-27 12:51:31,933][20381] Updated weights for policy 1, policy_version 1039638 (0.0005) [2023-12-27 12:51:32,079][20347] Updated weights for policy 0, policy_version 1040394 (0.0005) [2023-12-27 12:51:32,141][20347] Updated weights for policy 0, policy_version 1040404 (0.0005) [2023-12-27 12:51:32,198][20347] Updated weights for policy 0, policy_version 1040414 (0.0007) [2023-12-27 12:51:32,248][20347] Updated weights for policy 0, policy_version 1040424 (0.0009) [2023-12-27 12:51:32,362][19170] Fps is (10 sec: 21306.6, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 532570112. Throughput: 0: 10209.2, 1: 10151.3. Samples: 532579576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:32,363][19170] Avg episode reward: [(0, '941.830'), (1, '695.970')] [2023-12-27 12:51:32,539][20381] Updated weights for policy 1, policy_version 1039648 (0.0007) [2023-12-27 12:51:32,600][20381] Updated weights for policy 1, policy_version 1039658 (0.0008) [2023-12-27 12:51:32,663][20381] Updated weights for policy 1, policy_version 1039668 (0.0008) [2023-12-27 12:51:32,883][20347] Updated weights for policy 0, policy_version 1040434 (0.0010) [2023-12-27 12:51:32,933][20347] Updated weights for policy 0, policy_version 1040444 (0.0010) [2023-12-27 12:51:32,989][20347] Updated weights for policy 0, policy_version 1040454 (0.0010) [2023-12-27 12:51:33,323][20381] Updated weights for policy 1, policy_version 1039678 (0.0007) [2023-12-27 12:51:33,377][20381] Updated weights for policy 1, policy_version 1039688 (0.0008) [2023-12-27 12:51:33,423][20381] Updated weights for policy 1, policy_version 1039698 (0.0007) [2023-12-27 12:51:33,623][20347] Updated weights for policy 0, policy_version 1040464 (0.0010) [2023-12-27 12:51:33,677][20347] Updated weights for policy 0, policy_version 1040474 (0.0010) [2023-12-27 12:51:33,735][20347] Updated weights for policy 0, policy_version 1040484 (0.0008) [2023-12-27 12:51:34,015][20381] Updated weights for policy 1, policy_version 1039708 (0.0007) [2023-12-27 12:51:34,069][20381] Updated weights for policy 1, policy_version 1039718 (0.0007) [2023-12-27 12:51:34,131][20381] Updated weights for policy 1, policy_version 1039728 (0.0008) [2023-12-27 12:51:34,409][20347] Updated weights for policy 0, policy_version 1040494 (0.0009) [2023-12-27 12:51:34,469][20347] Updated weights for policy 0, policy_version 1040504 (0.0007) [2023-12-27 12:51:34,531][20347] Updated weights for policy 0, policy_version 1040514 (0.0005) [2023-12-27 12:51:34,858][20381] Updated weights for policy 1, policy_version 1039738 (0.0008) [2023-12-27 12:51:34,904][20381] Updated weights for policy 1, policy_version 1039748 (0.0008) [2023-12-27 12:51:34,954][20381] Updated weights for policy 1, policy_version 1039758 (0.0008) [2023-12-27 12:51:35,017][20381] Updated weights for policy 1, policy_version 1039768 (0.0008) [2023-12-27 12:51:35,198][20347] Updated weights for policy 0, policy_version 1040524 (0.0007) [2023-12-27 12:51:35,248][20347] Updated weights for policy 0, policy_version 1040534 (0.0006) [2023-12-27 12:51:35,303][20347] Updated weights for policy 0, policy_version 1040544 (0.0008) [2023-12-27 12:51:35,669][20381] Updated weights for policy 1, policy_version 1039778 (0.0005) [2023-12-27 12:51:35,721][20381] Updated weights for policy 1, policy_version 1039788 (0.0006) [2023-12-27 12:51:35,772][20381] Updated weights for policy 1, policy_version 1039798 (0.0007) [2023-12-27 12:51:35,953][20347] Updated weights for policy 0, policy_version 1040554 (0.0007) [2023-12-27 12:51:36,004][20347] Updated weights for policy 0, policy_version 1040564 (0.0006) [2023-12-27 12:51:36,049][20347] Updated weights for policy 0, policy_version 1040574 (0.0007) [2023-12-27 12:51:36,094][20347] Updated weights for policy 0, policy_version 1040584 (0.0006) [2023-12-27 12:51:36,378][20381] Updated weights for policy 1, policy_version 1039808 (0.0007) [2023-12-27 12:51:36,428][20381] Updated weights for policy 1, policy_version 1039818 (0.0007) [2023-12-27 12:51:36,474][20381] Updated weights for policy 1, policy_version 1039828 (0.0007) [2023-12-27 12:51:36,771][20347] Updated weights for policy 0, policy_version 1040594 (0.0010) [2023-12-27 12:51:36,820][20347] Updated weights for policy 0, policy_version 1040604 (0.0010) [2023-12-27 12:51:36,884][20347] Updated weights for policy 0, policy_version 1040614 (0.0010) [2023-12-27 12:51:37,201][20381] Updated weights for policy 1, policy_version 1039838 (0.0009) [2023-12-27 12:51:37,262][20381] Updated weights for policy 1, policy_version 1039848 (0.0011) [2023-12-27 12:51:37,321][20381] Updated weights for policy 1, policy_version 1039858 (0.0011) [2023-12-27 12:51:37,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 532676608. Throughput: 0: 10246.7, 1: 10150.0. Samples: 532641792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:37,363][19170] Avg episode reward: [(0, '922.350'), (1, '764.820')] [2023-12-27 12:51:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001039864_266231808.pth... [2023-12-27 12:51:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001040616_266444800.pth... [2023-12-27 12:51:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001038648_265920512.pth [2023-12-27 12:51:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001039432_266141696.pth [2023-12-27 12:51:37,606][20347] Updated weights for policy 0, policy_version 1040624 (0.0009) [2023-12-27 12:51:37,672][20347] Updated weights for policy 0, policy_version 1040634 (0.0011) [2023-12-27 12:51:37,729][20347] Updated weights for policy 0, policy_version 1040644 (0.0010) [2023-12-27 12:51:38,000][20381] Updated weights for policy 1, policy_version 1039868 (0.0010) [2023-12-27 12:51:38,062][20381] Updated weights for policy 1, policy_version 1039878 (0.0011) [2023-12-27 12:51:38,115][20381] Updated weights for policy 1, policy_version 1039888 (0.0011) [2023-12-27 12:51:38,385][20347] Updated weights for policy 0, policy_version 1040654 (0.0009) [2023-12-27 12:51:38,447][20347] Updated weights for policy 0, policy_version 1040664 (0.0006) [2023-12-27 12:51:38,518][20347] Updated weights for policy 0, policy_version 1040674 (0.0008) [2023-12-27 12:51:38,820][20381] Updated weights for policy 1, policy_version 1039898 (0.0010) [2023-12-27 12:51:38,873][20381] Updated weights for policy 1, policy_version 1039908 (0.0010) [2023-12-27 12:51:38,932][20381] Updated weights for policy 1, policy_version 1039918 (0.0010) [2023-12-27 12:51:38,986][20381] Updated weights for policy 1, policy_version 1039928 (0.0010) [2023-12-27 12:51:39,167][20347] Updated weights for policy 0, policy_version 1040684 (0.0009) [2023-12-27 12:51:39,220][20347] Updated weights for policy 0, policy_version 1040694 (0.0010) [2023-12-27 12:51:39,282][20347] Updated weights for policy 0, policy_version 1040704 (0.0011) [2023-12-27 12:51:39,724][20381] Updated weights for policy 1, policy_version 1039938 (0.0009) [2023-12-27 12:51:39,785][20381] Updated weights for policy 1, policy_version 1039948 (0.0009) [2023-12-27 12:51:39,853][20381] Updated weights for policy 1, policy_version 1039958 (0.0009) [2023-12-27 12:51:40,071][20347] Updated weights for policy 0, policy_version 1040714 (0.0010) [2023-12-27 12:51:40,130][20347] Updated weights for policy 0, policy_version 1040724 (0.0007) [2023-12-27 12:51:40,190][20347] Updated weights for policy 0, policy_version 1040734 (0.0007) [2023-12-27 12:51:40,250][20347] Updated weights for policy 0, policy_version 1040744 (0.0006) [2023-12-27 12:51:40,574][20381] Updated weights for policy 1, policy_version 1039968 (0.0006) [2023-12-27 12:51:40,621][20381] Updated weights for policy 1, policy_version 1039978 (0.0008) [2023-12-27 12:51:40,676][20381] Updated weights for policy 1, policy_version 1039988 (0.0010) [2023-12-27 12:51:40,950][20347] Updated weights for policy 0, policy_version 1040754 (0.0007) [2023-12-27 12:51:41,006][20347] Updated weights for policy 0, policy_version 1040764 (0.0008) [2023-12-27 12:51:41,073][20347] Updated weights for policy 0, policy_version 1040774 (0.0009) [2023-12-27 12:51:41,380][20381] Updated weights for policy 1, policy_version 1039998 (0.0009) [2023-12-27 12:51:41,449][20381] Updated weights for policy 1, policy_version 1040008 (0.0008) [2023-12-27 12:51:41,513][20381] Updated weights for policy 1, policy_version 1040018 (0.0007) [2023-12-27 12:51:41,845][20347] Updated weights for policy 0, policy_version 1040784 (0.0006) [2023-12-27 12:51:41,902][20347] Updated weights for policy 0, policy_version 1040794 (0.0007) [2023-12-27 12:51:41,960][20347] Updated weights for policy 0, policy_version 1040804 (0.0006) [2023-12-27 12:51:42,269][20381] Updated weights for policy 1, policy_version 1040028 (0.0008) [2023-12-27 12:51:42,330][20381] Updated weights for policy 1, policy_version 1040038 (0.0010) [2023-12-27 12:51:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 532766720. Throughput: 0: 10202.7, 1: 10173.4. Samples: 532762072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:42,363][19170] Avg episode reward: [(0, '892.920'), (1, '725.490')] [2023-12-27 12:51:42,397][20381] Updated weights for policy 1, policy_version 1040048 (0.0009) [2023-12-27 12:51:42,620][20347] Updated weights for policy 0, policy_version 1040814 (0.0008) [2023-12-27 12:51:42,693][20347] Updated weights for policy 0, policy_version 1040824 (0.0007) [2023-12-27 12:51:42,760][20347] Updated weights for policy 0, policy_version 1040834 (0.0008) [2023-12-27 12:51:43,011][20381] Updated weights for policy 1, policy_version 1040058 (0.0008) [2023-12-27 12:51:43,062][20381] Updated weights for policy 1, policy_version 1040068 (0.0010) [2023-12-27 12:51:43,107][20381] Updated weights for policy 1, policy_version 1040078 (0.0010) [2023-12-27 12:51:43,161][20381] Updated weights for policy 1, policy_version 1040088 (0.0010) [2023-12-27 12:51:43,332][20347] Updated weights for policy 0, policy_version 1040844 (0.0007) [2023-12-27 12:51:43,381][20347] Updated weights for policy 0, policy_version 1040854 (0.0005) [2023-12-27 12:51:43,431][20347] Updated weights for policy 0, policy_version 1040864 (0.0010) [2023-12-27 12:51:43,851][20381] Updated weights for policy 1, policy_version 1040098 (0.0008) [2023-12-27 12:51:43,906][20381] Updated weights for policy 1, policy_version 1040108 (0.0007) [2023-12-27 12:51:43,958][20381] Updated weights for policy 1, policy_version 1040118 (0.0008) [2023-12-27 12:51:43,986][20347] Updated weights for policy 0, policy_version 1040874 (0.0010) [2023-12-27 12:51:44,039][20347] Updated weights for policy 0, policy_version 1040884 (0.0008) [2023-12-27 12:51:44,090][20347] Updated weights for policy 0, policy_version 1040894 (0.0010) [2023-12-27 12:51:44,145][20347] Updated weights for policy 0, policy_version 1040904 (0.0006) [2023-12-27 12:51:44,694][20381] Updated weights for policy 1, policy_version 1040128 (0.0007) [2023-12-27 12:51:44,746][20381] Updated weights for policy 1, policy_version 1040138 (0.0008) [2023-12-27 12:51:44,780][20291] KL-divergence is very high: 127.2694 [2023-12-27 12:51:44,794][20291] KL-divergence is very high: 113.8475 [2023-12-27 12:51:44,801][20291] KL-divergence is very high: 109.6807 [2023-12-27 12:51:44,802][20347] Updated weights for policy 0, policy_version 1040914 (0.0011) [2023-12-27 12:51:44,811][20381] Updated weights for policy 1, policy_version 1040148 (0.0007) [2023-12-27 12:51:44,855][20347] Updated weights for policy 0, policy_version 1040924 (0.0010) [2023-12-27 12:51:44,913][20347] Updated weights for policy 0, policy_version 1040934 (0.0010) [2023-12-27 12:51:45,474][20381] Updated weights for policy 1, policy_version 1040158 (0.0006) [2023-12-27 12:51:45,520][20381] Updated weights for policy 1, policy_version 1040168 (0.0005) [2023-12-27 12:51:45,553][20291] KL-divergence is very high: 405.1411 [2023-12-27 12:51:45,568][20381] Updated weights for policy 1, policy_version 1040178 (0.0007) [2023-12-27 12:51:45,596][20347] Updated weights for policy 0, policy_version 1040944 (0.0006) [2023-12-27 12:51:45,603][20291] KL-divergence is very high: 271.4387 [2023-12-27 12:51:45,648][20291] KL-divergence is very high: 182.1608 [2023-12-27 12:51:45,652][20347] Updated weights for policy 0, policy_version 1040954 (0.0005) [2023-12-27 12:51:45,686][20291] KL-divergence is very high: 149.3092 [2023-12-27 12:51:45,711][20347] Updated weights for policy 0, policy_version 1040964 (0.0006) [2023-12-27 12:51:46,268][20347] Updated weights for policy 0, policy_version 1040974 (0.0008) [2023-12-27 12:51:46,268][20381] Updated weights for policy 1, policy_version 1040188 (0.0009) [2023-12-27 12:51:46,318][20381] Updated weights for policy 1, policy_version 1040198 (0.0010) [2023-12-27 12:51:46,320][20347] Updated weights for policy 0, policy_version 1040984 (0.0010) [2023-12-27 12:51:46,368][20347] Updated weights for policy 0, policy_version 1040994 (0.0010) [2023-12-27 12:51:46,375][20381] Updated weights for policy 1, policy_version 1040208 (0.0009) [2023-12-27 12:51:46,933][20381] Updated weights for policy 1, policy_version 1040218 (0.0008) [2023-12-27 12:51:46,976][20381] Updated weights for policy 1, policy_version 1040228 (0.0005) [2023-12-27 12:51:47,036][20381] Updated weights for policy 1, policy_version 1040238 (0.0005) [2023-12-27 12:51:47,080][20347] Updated weights for policy 0, policy_version 1041004 (0.0010) [2023-12-27 12:51:47,096][20381] Updated weights for policy 1, policy_version 1040248 (0.0006) [2023-12-27 12:51:47,146][20347] Updated weights for policy 0, policy_version 1041014 (0.0010) [2023-12-27 12:51:47,209][20347] Updated weights for policy 0, policy_version 1041024 (0.0010) [2023-12-27 12:51:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 532881408. Throughput: 0: 10255.7, 1: 10221.9. Samples: 532887820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:47,363][19170] Avg episode reward: [(0, '863.240'), (1, '705.780')] [2023-12-27 12:51:47,680][20381] Updated weights for policy 1, policy_version 1040258 (0.0007) [2023-12-27 12:51:47,728][20381] Updated weights for policy 1, policy_version 1040268 (0.0008) [2023-12-27 12:51:47,785][20381] Updated weights for policy 1, policy_version 1040278 (0.0007) [2023-12-27 12:51:47,800][20347] Updated weights for policy 0, policy_version 1041034 (0.0010) [2023-12-27 12:51:47,851][20347] Updated weights for policy 0, policy_version 1041044 (0.0010) [2023-12-27 12:51:47,919][20347] Updated weights for policy 0, policy_version 1041054 (0.0007) [2023-12-27 12:51:47,968][20347] Updated weights for policy 0, policy_version 1041064 (0.0006) [2023-12-27 12:51:48,458][20381] Updated weights for policy 1, policy_version 1040288 (0.0008) [2023-12-27 12:51:48,522][20381] Updated weights for policy 1, policy_version 1040298 (0.0007) [2023-12-27 12:51:48,564][20347] Updated weights for policy 0, policy_version 1041074 (0.0010) [2023-12-27 12:51:48,579][20381] Updated weights for policy 1, policy_version 1040308 (0.0009) [2023-12-27 12:51:48,627][20347] Updated weights for policy 0, policy_version 1041084 (0.0010) [2023-12-27 12:51:48,694][20347] Updated weights for policy 0, policy_version 1041094 (0.0008) [2023-12-27 12:51:49,247][20347] Updated weights for policy 0, policy_version 1041104 (0.0009) [2023-12-27 12:51:49,310][20347] Updated weights for policy 0, policy_version 1041114 (0.0008) [2023-12-27 12:51:49,372][20347] Updated weights for policy 0, policy_version 1041124 (0.0008) [2023-12-27 12:51:49,380][20381] Updated weights for policy 1, policy_version 1040318 (0.0008) [2023-12-27 12:51:49,438][20381] Updated weights for policy 1, policy_version 1040328 (0.0007) [2023-12-27 12:51:49,491][20381] Updated weights for policy 1, policy_version 1040338 (0.0008) [2023-12-27 12:51:50,070][20347] Updated weights for policy 0, policy_version 1041134 (0.0009) [2023-12-27 12:51:50,130][20347] Updated weights for policy 0, policy_version 1041144 (0.0008) [2023-12-27 12:51:50,186][20347] Updated weights for policy 0, policy_version 1041154 (0.0008) [2023-12-27 12:51:50,244][20381] Updated weights for policy 1, policy_version 1040348 (0.0007) [2023-12-27 12:51:50,299][20381] Updated weights for policy 1, policy_version 1040358 (0.0008) [2023-12-27 12:51:50,362][20381] Updated weights for policy 1, policy_version 1040368 (0.0005) [2023-12-27 12:51:50,879][20347] Updated weights for policy 0, policy_version 1041164 (0.0008) [2023-12-27 12:51:50,932][20347] Updated weights for policy 0, policy_version 1041174 (0.0007) [2023-12-27 12:51:50,986][20381] Updated weights for policy 1, policy_version 1040378 (0.0005) [2023-12-27 12:51:50,992][20347] Updated weights for policy 0, policy_version 1041184 (0.0007) [2023-12-27 12:51:51,049][20381] Updated weights for policy 1, policy_version 1040388 (0.0007) [2023-12-27 12:51:51,117][20381] Updated weights for policy 1, policy_version 1040398 (0.0009) [2023-12-27 12:51:51,184][20381] Updated weights for policy 1, policy_version 1040408 (0.0008) [2023-12-27 12:51:51,732][20347] Updated weights for policy 0, policy_version 1041194 (0.0007) [2023-12-27 12:51:51,800][20347] Updated weights for policy 0, policy_version 1041204 (0.0008) [2023-12-27 12:51:51,872][20347] Updated weights for policy 0, policy_version 1041214 (0.0008) [2023-12-27 12:51:51,933][20347] Updated weights for policy 0, policy_version 1041224 (0.0009) [2023-12-27 12:51:51,944][20381] Updated weights for policy 1, policy_version 1040418 (0.0009) [2023-12-27 12:51:52,009][20381] Updated weights for policy 1, policy_version 1040428 (0.0008) [2023-12-27 12:51:52,068][20381] Updated weights for policy 1, policy_version 1040438 (0.0009) [2023-12-27 12:51:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 532979712. Throughput: 0: 10306.3, 1: 10211.1. Samples: 532950272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:52,363][19170] Avg episode reward: [(0, '892.690'), (1, '705.680')] [2023-12-27 12:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001040440_266379264.pth... [2023-12-27 12:51:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001041224_266600448.pth... [2023-12-27 12:51:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001039224_266067968.pth [2023-12-27 12:51:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001040008_266289152.pth [2023-12-27 12:51:52,555][20347] Updated weights for policy 0, policy_version 1041234 (0.0007) [2023-12-27 12:51:52,602][20347] Updated weights for policy 0, policy_version 1041244 (0.0008) [2023-12-27 12:51:52,661][20347] Updated weights for policy 0, policy_version 1041254 (0.0008) [2023-12-27 12:51:52,819][20381] Updated weights for policy 1, policy_version 1040448 (0.0007) [2023-12-27 12:51:52,868][20381] Updated weights for policy 1, policy_version 1040458 (0.0005) [2023-12-27 12:51:52,923][20381] Updated weights for policy 1, policy_version 1040468 (0.0008) [2023-12-27 12:51:53,352][20347] Updated weights for policy 0, policy_version 1041264 (0.0007) [2023-12-27 12:51:53,423][20347] Updated weights for policy 0, policy_version 1041274 (0.0008) [2023-12-27 12:51:53,478][20347] Updated weights for policy 0, policy_version 1041284 (0.0007) [2023-12-27 12:51:53,555][20381] Updated weights for policy 1, policy_version 1040478 (0.0007) [2023-12-27 12:51:53,607][20381] Updated weights for policy 1, policy_version 1040488 (0.0007) [2023-12-27 12:51:53,665][20381] Updated weights for policy 1, policy_version 1040498 (0.0006) [2023-12-27 12:51:54,182][20347] Updated weights for policy 0, policy_version 1041294 (0.0008) [2023-12-27 12:51:54,233][20347] Updated weights for policy 0, policy_version 1041304 (0.0007) [2023-12-27 12:51:54,255][20381] Updated weights for policy 1, policy_version 1040508 (0.0005) [2023-12-27 12:51:54,287][20347] Updated weights for policy 0, policy_version 1041314 (0.0007) [2023-12-27 12:51:54,308][20381] Updated weights for policy 1, policy_version 1040518 (0.0006) [2023-12-27 12:51:54,363][20381] Updated weights for policy 1, policy_version 1040528 (0.0007) [2023-12-27 12:51:54,943][20381] Updated weights for policy 1, policy_version 1040538 (0.0008) [2023-12-27 12:51:55,003][20381] Updated weights for policy 1, policy_version 1040548 (0.0007) [2023-12-27 12:51:55,035][20347] Updated weights for policy 0, policy_version 1041324 (0.0006) [2023-12-27 12:51:55,058][20381] Updated weights for policy 1, policy_version 1040558 (0.0008) [2023-12-27 12:51:55,082][20347] Updated weights for policy 0, policy_version 1041334 (0.0006) [2023-12-27 12:51:55,104][20381] Updated weights for policy 1, policy_version 1040568 (0.0006) [2023-12-27 12:51:55,130][20347] Updated weights for policy 0, policy_version 1041344 (0.0006) [2023-12-27 12:51:55,807][20347] Updated weights for policy 0, policy_version 1041354 (0.0008) [2023-12-27 12:51:55,809][20381] Updated weights for policy 1, policy_version 1040578 (0.0007) [2023-12-27 12:51:55,857][20347] Updated weights for policy 0, policy_version 1041364 (0.0007) [2023-12-27 12:51:55,859][20381] Updated weights for policy 1, policy_version 1040588 (0.0007) [2023-12-27 12:51:55,902][20347] Updated weights for policy 0, policy_version 1041374 (0.0005) [2023-12-27 12:51:55,907][20381] Updated weights for policy 1, policy_version 1040598 (0.0006) [2023-12-27 12:51:55,950][20347] Updated weights for policy 0, policy_version 1041384 (0.0007) [2023-12-27 12:51:56,629][20347] Updated weights for policy 0, policy_version 1041394 (0.0005) [2023-12-27 12:51:56,634][20381] Updated weights for policy 1, policy_version 1040608 (0.0007) [2023-12-27 12:51:56,677][20347] Updated weights for policy 0, policy_version 1041404 (0.0005) [2023-12-27 12:51:56,693][20381] Updated weights for policy 1, policy_version 1040618 (0.0007) [2023-12-27 12:51:56,729][20347] Updated weights for policy 0, policy_version 1041414 (0.0005) [2023-12-27 12:51:56,742][20381] Updated weights for policy 1, policy_version 1040628 (0.0008) [2023-12-27 12:51:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 533078016. Throughput: 0: 10221.8, 1: 10297.1. Samples: 533072028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:51:57,363][19170] Avg episode reward: [(0, '912.290'), (1, '686.070')] [2023-12-27 12:51:57,368][20347] Updated weights for policy 0, policy_version 1041424 (0.0007) [2023-12-27 12:51:57,392][20381] Updated weights for policy 1, policy_version 1040638 (0.0007) [2023-12-27 12:51:57,420][20347] Updated weights for policy 0, policy_version 1041434 (0.0006) [2023-12-27 12:51:57,441][20381] Updated weights for policy 1, policy_version 1040648 (0.0006) [2023-12-27 12:51:57,468][20347] Updated weights for policy 0, policy_version 1041444 (0.0007) [2023-12-27 12:51:57,492][20381] Updated weights for policy 1, policy_version 1040658 (0.0006) [2023-12-27 12:51:58,101][20347] Updated weights for policy 0, policy_version 1041454 (0.0005) [2023-12-27 12:51:58,144][20347] Updated weights for policy 0, policy_version 1041464 (0.0005) [2023-12-27 12:51:58,168][20381] Updated weights for policy 1, policy_version 1040668 (0.0009) [2023-12-27 12:51:58,196][20347] Updated weights for policy 0, policy_version 1041474 (0.0006) [2023-12-27 12:51:58,227][20381] Updated weights for policy 1, policy_version 1040678 (0.0011) [2023-12-27 12:51:58,285][20381] Updated weights for policy 1, policy_version 1040688 (0.0011) [2023-12-27 12:51:59,000][20347] Updated weights for policy 0, policy_version 1041484 (0.0008) [2023-12-27 12:51:59,025][20291] KL-divergence is very high: 352.1785 [2023-12-27 12:51:59,031][20381] Updated weights for policy 1, policy_version 1040698 (0.0013) [2023-12-27 12:51:59,038][20291] KL-divergence is very high: 241.9907 [2023-12-27 12:51:59,065][20347] Updated weights for policy 0, policy_version 1041494 (0.0008) [2023-12-27 12:51:59,092][20381] Updated weights for policy 1, policy_version 1040708 (0.0008) [2023-12-27 12:51:59,134][20347] Updated weights for policy 0, policy_version 1041504 (0.0008) [2023-12-27 12:51:59,158][20381] Updated weights for policy 1, policy_version 1040718 (0.0008) [2023-12-27 12:51:59,220][20381] Updated weights for policy 1, policy_version 1040728 (0.0008) [2023-12-27 12:51:59,914][20381] Updated weights for policy 1, policy_version 1040738 (0.0007) [2023-12-27 12:51:59,919][20347] Updated weights for policy 0, policy_version 1041514 (0.0008) [2023-12-27 12:51:59,975][20381] Updated weights for policy 1, policy_version 1040748 (0.0007) [2023-12-27 12:51:59,976][20347] Updated weights for policy 0, policy_version 1041524 (0.0011) [2023-12-27 12:52:00,030][20347] Updated weights for policy 0, policy_version 1041534 (0.0010) [2023-12-27 12:52:00,035][20381] Updated weights for policy 1, policy_version 1040758 (0.0006) [2023-12-27 12:52:00,079][20347] Updated weights for policy 0, policy_version 1041544 (0.0010) [2023-12-27 12:52:00,687][20381] Updated weights for policy 1, policy_version 1040768 (0.0010) [2023-12-27 12:52:00,692][20347] Updated weights for policy 0, policy_version 1041554 (0.0005) [2023-12-27 12:52:00,741][20381] Updated weights for policy 1, policy_version 1040778 (0.0010) [2023-12-27 12:52:00,743][20347] Updated weights for policy 0, policy_version 1041564 (0.0005) [2023-12-27 12:52:00,791][20347] Updated weights for policy 0, policy_version 1041574 (0.0006) [2023-12-27 12:52:00,802][20381] Updated weights for policy 1, policy_version 1040788 (0.0010) [2023-12-27 12:52:01,533][20381] Updated weights for policy 1, policy_version 1040798 (0.0010) [2023-12-27 12:52:01,561][20347] Updated weights for policy 0, policy_version 1041584 (0.0009) [2023-12-27 12:52:01,591][20381] Updated weights for policy 1, policy_version 1040808 (0.0010) [2023-12-27 12:52:01,620][20347] Updated weights for policy 0, policy_version 1041594 (0.0006) [2023-12-27 12:52:01,654][20381] Updated weights for policy 1, policy_version 1040818 (0.0011) [2023-12-27 12:52:01,688][20347] Updated weights for policy 0, policy_version 1041604 (0.0007) [2023-12-27 12:52:02,231][20381] Updated weights for policy 1, policy_version 1040828 (0.0007) [2023-12-27 12:52:02,283][20381] Updated weights for policy 1, policy_version 1040838 (0.0007) [2023-12-27 12:52:02,337][20381] Updated weights for policy 1, policy_version 1040848 (0.0007) [2023-12-27 12:52:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 533176320. Throughput: 0: 10266.3, 1: 10262.7. Samples: 533194392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:02,363][19170] Avg episode reward: [(0, '892.780'), (1, '627.300')] [2023-12-27 12:52:02,418][20347] Updated weights for policy 0, policy_version 1041614 (0.0008) [2023-12-27 12:52:02,466][20347] Updated weights for policy 0, policy_version 1041624 (0.0005) [2023-12-27 12:52:02,521][20347] Updated weights for policy 0, policy_version 1041634 (0.0007) [2023-12-27 12:52:02,979][20381] Updated weights for policy 1, policy_version 1040858 (0.0008) [2023-12-27 12:52:03,022][20381] Updated weights for policy 1, policy_version 1040868 (0.0005) [2023-12-27 12:52:03,069][20381] Updated weights for policy 1, policy_version 1040878 (0.0009) [2023-12-27 12:52:03,111][20347] Updated weights for policy 0, policy_version 1041644 (0.0007) [2023-12-27 12:52:03,116][20381] Updated weights for policy 1, policy_version 1040888 (0.0010) [2023-12-27 12:52:03,149][20291] KL-divergence is very high: 179.5284 [2023-12-27 12:52:03,166][20347] Updated weights for policy 0, policy_version 1041654 (0.0007) [2023-12-27 12:52:03,229][20347] Updated weights for policy 0, policy_version 1041664 (0.0007) [2023-12-27 12:52:03,765][20381] Updated weights for policy 1, policy_version 1040898 (0.0011) [2023-12-27 12:52:03,821][20381] Updated weights for policy 1, policy_version 1040908 (0.0011) [2023-12-27 12:52:03,835][20347] Updated weights for policy 0, policy_version 1041674 (0.0008) [2023-12-27 12:52:03,875][20291] KL-divergence is very high: 342.5233 [2023-12-27 12:52:03,884][20381] Updated weights for policy 1, policy_version 1040918 (0.0011) [2023-12-27 12:52:03,899][20347] Updated weights for policy 0, policy_version 1041684 (0.0008) [2023-12-27 12:52:03,959][20347] Updated weights for policy 0, policy_version 1041694 (0.0009) [2023-12-27 12:52:04,024][20347] Updated weights for policy 0, policy_version 1041704 (0.0008) [2023-12-27 12:52:04,585][20381] Updated weights for policy 1, policy_version 1040928 (0.0006) [2023-12-27 12:52:04,586][20292] KL-divergence is very high: 227.2605 [2023-12-27 12:52:04,626][20292] KL-divergence is very high: 173.4350 [2023-12-27 12:52:04,637][20381] Updated weights for policy 1, policy_version 1040938 (0.0005) [2023-12-27 12:52:04,648][20347] Updated weights for policy 0, policy_version 1041714 (0.0010) [2023-12-27 12:52:04,691][20381] Updated weights for policy 1, policy_version 1040948 (0.0006) [2023-12-27 12:52:04,697][20347] Updated weights for policy 0, policy_version 1041724 (0.0010) [2023-12-27 12:52:04,751][20347] Updated weights for policy 0, policy_version 1041734 (0.0010) [2023-12-27 12:52:05,354][20292] KL-divergence is very high: 113.9140 [2023-12-27 12:52:05,356][20381] Updated weights for policy 1, policy_version 1040958 (0.0008) [2023-12-27 12:52:05,376][20347] Updated weights for policy 0, policy_version 1041744 (0.0009) [2023-12-27 12:52:05,404][20381] Updated weights for policy 1, policy_version 1040968 (0.0010) [2023-12-27 12:52:05,426][20347] Updated weights for policy 0, policy_version 1041754 (0.0009) [2023-12-27 12:52:05,455][20381] Updated weights for policy 1, policy_version 1040978 (0.0010) [2023-12-27 12:52:05,482][20347] Updated weights for policy 0, policy_version 1041764 (0.0007) [2023-12-27 12:52:06,090][20381] Updated weights for policy 1, policy_version 1040988 (0.0009) [2023-12-27 12:52:06,105][20347] Updated weights for policy 0, policy_version 1041774 (0.0008) [2023-12-27 12:52:06,159][20381] Updated weights for policy 1, policy_version 1040998 (0.0007) [2023-12-27 12:52:06,169][20347] Updated weights for policy 0, policy_version 1041784 (0.0010) [2023-12-27 12:52:06,214][20381] Updated weights for policy 1, policy_version 1041008 (0.0008) [2023-12-27 12:52:06,230][20347] Updated weights for policy 0, policy_version 1041794 (0.0011) [2023-12-27 12:52:06,254][20292] KL-divergence is very high: 313.2902 [2023-12-27 12:52:06,897][20347] Updated weights for policy 0, policy_version 1041804 (0.0008) [2023-12-27 12:52:06,949][20381] Updated weights for policy 1, policy_version 1041018 (0.0008) [2023-12-27 12:52:06,955][20347] Updated weights for policy 0, policy_version 1041814 (0.0007) [2023-12-27 12:52:07,011][20381] Updated weights for policy 1, policy_version 1041028 (0.0007) [2023-12-27 12:52:07,013][20347] Updated weights for policy 0, policy_version 1041824 (0.0008) [2023-12-27 12:52:07,077][20381] Updated weights for policy 1, policy_version 1041038 (0.0008) [2023-12-27 12:52:07,137][20381] Updated weights for policy 1, policy_version 1041048 (0.0008) [2023-12-27 12:52:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 533291008. Throughput: 0: 10316.1, 1: 10303.9. Samples: 533258412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:07,363][19170] Avg episode reward: [(0, '921.950'), (1, '725.500')] [2023-12-27 12:52:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001041048_266534912.pth... [2023-12-27 12:52:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001041832_266756096.pth... [2023-12-27 12:52:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001040616_266444800.pth [2023-12-27 12:52:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001039864_266231808.pth [2023-12-27 12:52:07,653][20347] Updated weights for policy 0, policy_version 1041834 (0.0010) [2023-12-27 12:52:07,710][20347] Updated weights for policy 0, policy_version 1041844 (0.0005) [2023-12-27 12:52:07,773][20347] Updated weights for policy 0, policy_version 1041854 (0.0006) [2023-12-27 12:52:07,815][20381] Updated weights for policy 1, policy_version 1041058 (0.0006) [2023-12-27 12:52:07,829][20347] Updated weights for policy 0, policy_version 1041864 (0.0005) [2023-12-27 12:52:07,870][20381] Updated weights for policy 1, policy_version 1041068 (0.0007) [2023-12-27 12:52:07,920][20381] Updated weights for policy 1, policy_version 1041078 (0.0008) [2023-12-27 12:52:08,443][20347] Updated weights for policy 0, policy_version 1041874 (0.0008) [2023-12-27 12:52:08,505][20347] Updated weights for policy 0, policy_version 1041884 (0.0007) [2023-12-27 12:52:08,555][20347] Updated weights for policy 0, policy_version 1041894 (0.0007) [2023-12-27 12:52:08,570][20381] Updated weights for policy 1, policy_version 1041088 (0.0009) [2023-12-27 12:52:08,632][20381] Updated weights for policy 1, policy_version 1041098 (0.0007) [2023-12-27 12:52:08,698][20381] Updated weights for policy 1, policy_version 1041108 (0.0008) [2023-12-27 12:52:09,267][20347] Updated weights for policy 0, policy_version 1041904 (0.0008) [2023-12-27 12:52:09,328][20347] Updated weights for policy 0, policy_version 1041914 (0.0007) [2023-12-27 12:52:09,401][20381] Updated weights for policy 1, policy_version 1041118 (0.0008) [2023-12-27 12:52:09,403][20347] Updated weights for policy 0, policy_version 1041924 (0.0009) [2023-12-27 12:52:09,468][20381] Updated weights for policy 1, policy_version 1041128 (0.0008) [2023-12-27 12:52:09,533][20381] Updated weights for policy 1, policy_version 1041138 (0.0007) [2023-12-27 12:52:10,201][20347] Updated weights for policy 0, policy_version 1041934 (0.0008) [2023-12-27 12:52:10,230][20381] Updated weights for policy 1, policy_version 1041148 (0.0008) [2023-12-27 12:52:10,263][20347] Updated weights for policy 0, policy_version 1041944 (0.0007) [2023-12-27 12:52:10,286][20381] Updated weights for policy 1, policy_version 1041158 (0.0007) [2023-12-27 12:52:10,317][20347] Updated weights for policy 0, policy_version 1041954 (0.0007) [2023-12-27 12:52:10,343][20381] Updated weights for policy 1, policy_version 1041168 (0.0010) [2023-12-27 12:52:10,963][20381] Updated weights for policy 1, policy_version 1041178 (0.0006) [2023-12-27 12:52:11,019][20381] Updated weights for policy 1, policy_version 1041188 (0.0010) [2023-12-27 12:52:11,056][20347] Updated weights for policy 0, policy_version 1041964 (0.0008) [2023-12-27 12:52:11,090][20381] Updated weights for policy 1, policy_version 1041199 (0.0009) [2023-12-27 12:52:11,117][20347] Updated weights for policy 0, policy_version 1041974 (0.0010) [2023-12-27 12:52:11,182][20347] Updated weights for policy 0, policy_version 1041984 (0.0011) [2023-12-27 12:52:11,803][20381] Updated weights for policy 1, policy_version 1041209 (0.0008) [2023-12-27 12:52:11,852][20381] Updated weights for policy 1, policy_version 1041219 (0.0010) [2023-12-27 12:52:11,906][20381] Updated weights for policy 1, policy_version 1041229 (0.0011) [2023-12-27 12:52:11,943][20347] Updated weights for policy 0, policy_version 1041994 (0.0010) [2023-12-27 12:52:11,967][20381] Updated weights for policy 1, policy_version 1041239 (0.0006) [2023-12-27 12:52:12,005][20347] Updated weights for policy 0, policy_version 1042004 (0.0006) [2023-12-27 12:52:12,067][20347] Updated weights for policy 0, policy_version 1042014 (0.0006) [2023-12-27 12:52:12,123][20347] Updated weights for policy 0, policy_version 1042024 (0.0007) [2023-12-27 12:52:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 533389312. Throughput: 0: 10302.7, 1: 10247.7. Samples: 533379772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:12,363][19170] Avg episode reward: [(0, '863.260'), (1, '823.570')] [2023-12-27 12:52:12,691][20381] Updated weights for policy 1, policy_version 1041249 (0.0010) [2023-12-27 12:52:12,706][20347] Updated weights for policy 0, policy_version 1042034 (0.0006) [2023-12-27 12:52:12,745][20381] Updated weights for policy 1, policy_version 1041259 (0.0011) [2023-12-27 12:52:12,770][20347] Updated weights for policy 0, policy_version 1042044 (0.0006) [2023-12-27 12:52:12,804][20381] Updated weights for policy 1, policy_version 1041269 (0.0011) [2023-12-27 12:52:12,820][20347] Updated weights for policy 0, policy_version 1042054 (0.0007) [2023-12-27 12:52:13,491][20381] Updated weights for policy 1, policy_version 1041279 (0.0010) [2023-12-27 12:52:13,514][20347] Updated weights for policy 0, policy_version 1042064 (0.0008) [2023-12-27 12:52:13,546][20381] Updated weights for policy 1, policy_version 1041289 (0.0010) [2023-12-27 12:52:13,559][20292] KL-divergence is very high: 118.0297 [2023-12-27 12:52:13,564][20292] KL-divergence is very high: 112.1954 [2023-12-27 12:52:13,566][20347] Updated weights for policy 0, policy_version 1042074 (0.0005) [2023-12-27 12:52:13,594][20381] Updated weights for policy 1, policy_version 1041299 (0.0010) [2023-12-27 12:52:13,625][20347] Updated weights for policy 0, policy_version 1042084 (0.0006) [2023-12-27 12:52:14,205][20381] Updated weights for policy 1, policy_version 1041309 (0.0010) [2023-12-27 12:52:14,239][20347] Updated weights for policy 0, policy_version 1042094 (0.0009) [2023-12-27 12:52:14,258][20381] Updated weights for policy 1, policy_version 1041319 (0.0007) [2023-12-27 12:52:14,299][20347] Updated weights for policy 0, policy_version 1042104 (0.0010) [2023-12-27 12:52:14,316][20381] Updated weights for policy 1, policy_version 1041329 (0.0006) [2023-12-27 12:52:14,357][20347] Updated weights for policy 0, policy_version 1042114 (0.0009) [2023-12-27 12:52:14,982][20347] Updated weights for policy 0, policy_version 1042124 (0.0007) [2023-12-27 12:52:14,989][20381] Updated weights for policy 1, policy_version 1041339 (0.0007) [2023-12-27 12:52:15,041][20347] Updated weights for policy 0, policy_version 1042134 (0.0009) [2023-12-27 12:52:15,044][20381] Updated weights for policy 1, policy_version 1041349 (0.0007) [2023-12-27 12:52:15,091][20347] Updated weights for policy 0, policy_version 1042144 (0.0010) [2023-12-27 12:52:15,104][20381] Updated weights for policy 1, policy_version 1041359 (0.0006) [2023-12-27 12:52:15,745][20347] Updated weights for policy 0, policy_version 1042154 (0.0011) [2023-12-27 12:52:15,809][20347] Updated weights for policy 0, policy_version 1042164 (0.0010) [2023-12-27 12:52:15,813][20381] Updated weights for policy 1, policy_version 1041369 (0.0007) [2023-12-27 12:52:15,858][20347] Updated weights for policy 0, policy_version 1042174 (0.0010) [2023-12-27 12:52:15,871][20381] Updated weights for policy 1, policy_version 1041379 (0.0008) [2023-12-27 12:52:15,916][20347] Updated weights for policy 0, policy_version 1042184 (0.0010) [2023-12-27 12:52:15,923][20381] Updated weights for policy 1, policy_version 1041389 (0.0007) [2023-12-27 12:52:15,970][20381] Updated weights for policy 1, policy_version 1041399 (0.0007) [2023-12-27 12:52:16,534][20381] Updated weights for policy 1, policy_version 1041409 (0.0007) [2023-12-27 12:52:16,556][20347] Updated weights for policy 0, policy_version 1042194 (0.0010) [2023-12-27 12:52:16,579][20381] Updated weights for policy 1, policy_version 1041419 (0.0006) [2023-12-27 12:52:16,617][20347] Updated weights for policy 0, policy_version 1042204 (0.0010) [2023-12-27 12:52:16,624][20381] Updated weights for policy 1, policy_version 1041429 (0.0006) [2023-12-27 12:52:16,668][20347] Updated weights for policy 0, policy_version 1042214 (0.0010) [2023-12-27 12:52:17,234][20381] Updated weights for policy 1, policy_version 1041439 (0.0006) [2023-12-27 12:52:17,281][20381] Updated weights for policy 1, policy_version 1041449 (0.0007) [2023-12-27 12:52:17,309][20347] Updated weights for policy 0, policy_version 1042224 (0.0010) [2023-12-27 12:52:17,330][20381] Updated weights for policy 1, policy_version 1041459 (0.0006) [2023-12-27 12:52:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 533495808. Throughput: 0: 10331.6, 1: 10264.2. Samples: 533506392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:17,363][20347] Updated weights for policy 0, policy_version 1042234 (0.0010) [2023-12-27 12:52:17,363][19170] Avg episode reward: [(0, '854.170'), (1, '833.260')] [2023-12-27 12:52:17,423][20347] Updated weights for policy 0, policy_version 1042244 (0.0010) [2023-12-27 12:52:18,023][20381] Updated weights for policy 1, policy_version 1041469 (0.0007) [2023-12-27 12:52:18,078][20381] Updated weights for policy 1, policy_version 1041479 (0.0008) [2023-12-27 12:52:18,093][20347] Updated weights for policy 0, policy_version 1042254 (0.0009) [2023-12-27 12:52:18,126][20381] Updated weights for policy 1, policy_version 1041489 (0.0007) [2023-12-27 12:52:18,155][20347] Updated weights for policy 0, policy_version 1042264 (0.0007) [2023-12-27 12:52:18,204][20347] Updated weights for policy 0, policy_version 1042274 (0.0010) [2023-12-27 12:52:18,823][20347] Updated weights for policy 0, policy_version 1042284 (0.0008) [2023-12-27 12:52:18,876][20347] Updated weights for policy 0, policy_version 1042294 (0.0008) [2023-12-27 12:52:18,885][20381] Updated weights for policy 1, policy_version 1041499 (0.0007) [2023-12-27 12:52:18,922][20347] Updated weights for policy 0, policy_version 1042304 (0.0010) [2023-12-27 12:52:18,938][20381] Updated weights for policy 1, policy_version 1041509 (0.0006) [2023-12-27 12:52:18,995][20381] Updated weights for policy 1, policy_version 1041519 (0.0007) [2023-12-27 12:52:19,616][20347] Updated weights for policy 0, policy_version 1042314 (0.0010) [2023-12-27 12:52:19,678][20347] Updated weights for policy 0, policy_version 1042324 (0.0011) [2023-12-27 12:52:19,715][20291] KL-divergence is very high: 146.2289 [2023-12-27 12:52:19,727][20381] Updated weights for policy 1, policy_version 1041529 (0.0007) [2023-12-27 12:52:19,740][20347] Updated weights for policy 0, policy_version 1042334 (0.0011) [2023-12-27 12:52:19,768][20291] KL-divergence is very high: 199.2724 [2023-12-27 12:52:19,786][20381] Updated weights for policy 1, policy_version 1041539 (0.0005) [2023-12-27 12:52:19,805][20347] Updated weights for policy 0, policy_version 1042344 (0.0011) [2023-12-27 12:52:19,852][20381] Updated weights for policy 1, policy_version 1041549 (0.0008) [2023-12-27 12:52:19,909][20381] Updated weights for policy 1, policy_version 1041559 (0.0009) [2023-12-27 12:52:20,515][20347] Updated weights for policy 0, policy_version 1042354 (0.0011) [2023-12-27 12:52:20,579][20347] Updated weights for policy 0, policy_version 1042364 (0.0011) [2023-12-27 12:52:20,628][20381] Updated weights for policy 1, policy_version 1041569 (0.0006) [2023-12-27 12:52:20,643][20347] Updated weights for policy 0, policy_version 1042374 (0.0008) [2023-12-27 12:52:20,689][20381] Updated weights for policy 1, policy_version 1041579 (0.0008) [2023-12-27 12:52:20,750][20381] Updated weights for policy 1, policy_version 1041589 (0.0008) [2023-12-27 12:52:21,365][20347] Updated weights for policy 0, policy_version 1042384 (0.0009) [2023-12-27 12:52:21,432][20347] Updated weights for policy 0, policy_version 1042394 (0.0008) [2023-12-27 12:52:21,482][20347] Updated weights for policy 0, policy_version 1042404 (0.0008) [2023-12-27 12:52:21,533][20381] Updated weights for policy 1, policy_version 1041599 (0.0008) [2023-12-27 12:52:21,591][20381] Updated weights for policy 1, policy_version 1041609 (0.0008) [2023-12-27 12:52:21,656][20381] Updated weights for policy 1, policy_version 1041619 (0.0008) [2023-12-27 12:52:22,205][20347] Updated weights for policy 0, policy_version 1042414 (0.0009) [2023-12-27 12:52:22,267][20347] Updated weights for policy 0, policy_version 1042424 (0.0010) [2023-12-27 12:52:22,327][20347] Updated weights for policy 0, policy_version 1042434 (0.0007) [2023-12-27 12:52:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20481.2, 300 sec: 20466.1). Total num frames: 533585920. Throughput: 0: 10319.7, 1: 10225.7. Samples: 533566332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:22,363][19170] Avg episode reward: [(0, '912.430'), (1, '852.900')] [2023-12-27 12:52:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001042440_266911744.pth... [2023-12-27 12:52:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001041224_266600448.pth [2023-12-27 12:52:22,401][20381] Updated weights for policy 1, policy_version 1041629 (0.0008) [2023-12-27 12:52:22,456][20381] Updated weights for policy 1, policy_version 1041639 (0.0008) [2023-12-27 12:52:22,514][20381] Updated weights for policy 1, policy_version 1041649 (0.0008) [2023-12-27 12:52:22,557][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001041656_266690560.pth... [2023-12-27 12:52:22,562][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001040440_266379264.pth [2023-12-27 12:52:23,004][20347] Updated weights for policy 0, policy_version 1042445 (0.0009) [2023-12-27 12:52:23,060][20347] Updated weights for policy 0, policy_version 1042455 (0.0010) [2023-12-27 12:52:23,113][20347] Updated weights for policy 0, policy_version 1042465 (0.0010) [2023-12-27 12:52:23,190][20381] Updated weights for policy 1, policy_version 1041659 (0.0008) [2023-12-27 12:52:23,247][20381] Updated weights for policy 1, policy_version 1041669 (0.0006) [2023-12-27 12:52:23,305][20381] Updated weights for policy 1, policy_version 1041679 (0.0009) [2023-12-27 12:52:23,688][20347] Updated weights for policy 0, policy_version 1042475 (0.0009) [2023-12-27 12:52:23,739][20347] Updated weights for policy 0, policy_version 1042485 (0.0005) [2023-12-27 12:52:23,790][20347] Updated weights for policy 0, policy_version 1042495 (0.0005) [2023-12-27 12:52:23,981][20381] Updated weights for policy 1, policy_version 1041689 (0.0007) [2023-12-27 12:52:24,034][20381] Updated weights for policy 1, policy_version 1041699 (0.0007) [2023-12-27 12:52:24,082][20381] Updated weights for policy 1, policy_version 1041709 (0.0006) [2023-12-27 12:52:24,144][20381] Updated weights for policy 1, policy_version 1041719 (0.0006) [2023-12-27 12:52:24,424][20347] Updated weights for policy 0, policy_version 1042505 (0.0005) [2023-12-27 12:52:24,478][20347] Updated weights for policy 0, policy_version 1042515 (0.0008) [2023-12-27 12:52:24,533][20347] Updated weights for policy 0, policy_version 1042525 (0.0006) [2023-12-27 12:52:24,585][20347] Updated weights for policy 0, policy_version 1042535 (0.0008) [2023-12-27 12:52:24,830][20381] Updated weights for policy 1, policy_version 1041729 (0.0008) [2023-12-27 12:52:24,886][20381] Updated weights for policy 1, policy_version 1041739 (0.0007) [2023-12-27 12:52:24,942][20381] Updated weights for policy 1, policy_version 1041749 (0.0007) [2023-12-27 12:52:25,198][20347] Updated weights for policy 0, policy_version 1042545 (0.0010) [2023-12-27 12:52:25,254][20347] Updated weights for policy 0, policy_version 1042555 (0.0005) [2023-12-27 12:52:25,307][20347] Updated weights for policy 0, policy_version 1042565 (0.0005) [2023-12-27 12:52:25,583][20381] Updated weights for policy 1, policy_version 1041759 (0.0009) [2023-12-27 12:52:25,632][20381] Updated weights for policy 1, policy_version 1041769 (0.0007) [2023-12-27 12:52:25,679][20381] Updated weights for policy 1, policy_version 1041779 (0.0007) [2023-12-27 12:52:25,861][20347] Updated weights for policy 0, policy_version 1042575 (0.0005) [2023-12-27 12:52:25,925][20347] Updated weights for policy 0, policy_version 1042585 (0.0005) [2023-12-27 12:52:25,979][20347] Updated weights for policy 0, policy_version 1042595 (0.0008) [2023-12-27 12:52:26,311][20381] Updated weights for policy 1, policy_version 1041789 (0.0008) [2023-12-27 12:52:26,357][20381] Updated weights for policy 1, policy_version 1041799 (0.0005) [2023-12-27 12:52:26,407][20381] Updated weights for policy 1, policy_version 1041809 (0.0005) [2023-12-27 12:52:26,613][20347] Updated weights for policy 0, policy_version 1042605 (0.0010) [2023-12-27 12:52:26,670][20347] Updated weights for policy 0, policy_version 1042615 (0.0006) [2023-12-27 12:52:26,735][20347] Updated weights for policy 0, policy_version 1042625 (0.0005) [2023-12-27 12:52:26,929][20381] Updated weights for policy 1, policy_version 1041819 (0.0005) [2023-12-27 12:52:26,974][20381] Updated weights for policy 1, policy_version 1041829 (0.0005) [2023-12-27 12:52:27,024][20381] Updated weights for policy 1, policy_version 1041839 (0.0005) [2023-12-27 12:52:27,296][20347] Updated weights for policy 0, policy_version 1042635 (0.0005) [2023-12-27 12:52:27,344][20347] Updated weights for policy 0, policy_version 1042645 (0.0009) [2023-12-27 12:52:27,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20494.0). Total num frames: 533700608. Throughput: 0: 10407.8, 1: 10211.4. Samples: 533689936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:27,363][19170] Avg episode reward: [(0, '931.730'), (1, '921.610')] [2023-12-27 12:52:27,398][20347] Updated weights for policy 0, policy_version 1042655 (0.0010) [2023-12-27 12:52:27,606][20381] Updated weights for policy 1, policy_version 1041849 (0.0006) [2023-12-27 12:52:27,661][20381] Updated weights for policy 1, policy_version 1041859 (0.0010) [2023-12-27 12:52:27,712][20381] Updated weights for policy 1, policy_version 1041869 (0.0010) [2023-12-27 12:52:27,763][20381] Updated weights for policy 1, policy_version 1041879 (0.0010) [2023-12-27 12:52:28,012][20347] Updated weights for policy 0, policy_version 1042665 (0.0009) [2023-12-27 12:52:28,072][20347] Updated weights for policy 0, policy_version 1042675 (0.0005) [2023-12-27 12:52:28,134][20347] Updated weights for policy 0, policy_version 1042685 (0.0005) [2023-12-27 12:52:28,190][20347] Updated weights for policy 0, policy_version 1042695 (0.0005) [2023-12-27 12:52:28,458][20381] Updated weights for policy 1, policy_version 1041889 (0.0011) [2023-12-27 12:52:28,513][20381] Updated weights for policy 1, policy_version 1041899 (0.0010) [2023-12-27 12:52:28,575][20381] Updated weights for policy 1, policy_version 1041909 (0.0011) [2023-12-27 12:52:28,781][20347] Updated weights for policy 0, policy_version 1042705 (0.0010) [2023-12-27 12:52:28,844][20347] Updated weights for policy 0, policy_version 1042715 (0.0011) [2023-12-27 12:52:28,904][20347] Updated weights for policy 0, policy_version 1042725 (0.0010) [2023-12-27 12:52:29,252][20381] Updated weights for policy 1, policy_version 1041919 (0.0008) [2023-12-27 12:52:29,311][20381] Updated weights for policy 1, policy_version 1041929 (0.0007) [2023-12-27 12:52:29,377][20381] Updated weights for policy 1, policy_version 1041939 (0.0009) [2023-12-27 12:52:29,543][20347] Updated weights for policy 0, policy_version 1042735 (0.0008) [2023-12-27 12:52:29,603][20347] Updated weights for policy 0, policy_version 1042745 (0.0008) [2023-12-27 12:52:29,660][20347] Updated weights for policy 0, policy_version 1042755 (0.0007) [2023-12-27 12:52:30,013][20381] Updated weights for policy 1, policy_version 1041949 (0.0006) [2023-12-27 12:52:30,072][20381] Updated weights for policy 1, policy_version 1041959 (0.0005) [2023-12-27 12:52:30,133][20381] Updated weights for policy 1, policy_version 1041969 (0.0008) [2023-12-27 12:52:30,331][20347] Updated weights for policy 0, policy_version 1042765 (0.0007) [2023-12-27 12:52:30,388][20347] Updated weights for policy 0, policy_version 1042775 (0.0008) [2023-12-27 12:52:30,445][20347] Updated weights for policy 0, policy_version 1042785 (0.0007) [2023-12-27 12:52:30,774][20381] Updated weights for policy 1, policy_version 1041979 (0.0009) [2023-12-27 12:52:30,823][20381] Updated weights for policy 1, policy_version 1041989 (0.0005) [2023-12-27 12:52:30,874][20381] Updated weights for policy 1, policy_version 1041999 (0.0005) [2023-12-27 12:52:31,034][20347] Updated weights for policy 0, policy_version 1042795 (0.0007) [2023-12-27 12:52:31,089][20347] Updated weights for policy 0, policy_version 1042805 (0.0010) [2023-12-27 12:52:31,144][20347] Updated weights for policy 0, policy_version 1042815 (0.0008) [2023-12-27 12:52:31,549][20381] Updated weights for policy 1, policy_version 1042009 (0.0009) [2023-12-27 12:52:31,613][20381] Updated weights for policy 1, policy_version 1042019 (0.0006) [2023-12-27 12:52:31,680][20381] Updated weights for policy 1, policy_version 1042029 (0.0008) [2023-12-27 12:52:31,746][20381] Updated weights for policy 1, policy_version 1042039 (0.0009) [2023-12-27 12:52:31,855][20347] Updated weights for policy 0, policy_version 1042825 (0.0010) [2023-12-27 12:52:31,912][20347] Updated weights for policy 0, policy_version 1042835 (0.0010) [2023-12-27 12:52:31,964][20347] Updated weights for policy 0, policy_version 1042845 (0.0010) [2023-12-27 12:52:32,015][20347] Updated weights for policy 0, policy_version 1042855 (0.0007) [2023-12-27 12:52:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 533807104. Throughput: 0: 10430.7, 1: 10286.8. Samples: 533820108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:32,363][19170] Avg episode reward: [(0, '951.260'), (1, '862.770')] [2023-12-27 12:52:32,403][20381] Updated weights for policy 1, policy_version 1042049 (0.0008) [2023-12-27 12:52:32,464][20381] Updated weights for policy 1, policy_version 1042059 (0.0007) [2023-12-27 12:52:32,524][20381] Updated weights for policy 1, policy_version 1042069 (0.0008) [2023-12-27 12:52:32,659][20347] Updated weights for policy 0, policy_version 1042865 (0.0010) [2023-12-27 12:52:32,710][20347] Updated weights for policy 0, policy_version 1042875 (0.0010) [2023-12-27 12:52:32,769][20347] Updated weights for policy 0, policy_version 1042885 (0.0010) [2023-12-27 12:52:33,226][20381] Updated weights for policy 1, policy_version 1042079 (0.0007) [2023-12-27 12:52:33,272][20381] Updated weights for policy 1, policy_version 1042089 (0.0007) [2023-12-27 12:52:33,318][20381] Updated weights for policy 1, policy_version 1042099 (0.0007) [2023-12-27 12:52:33,425][20347] Updated weights for policy 0, policy_version 1042895 (0.0010) [2023-12-27 12:52:33,479][20347] Updated weights for policy 0, policy_version 1042905 (0.0010) [2023-12-27 12:52:33,529][20347] Updated weights for policy 0, policy_version 1042915 (0.0005) [2023-12-27 12:52:34,053][20381] Updated weights for policy 1, policy_version 1042109 (0.0008) [2023-12-27 12:52:34,078][20347] Updated weights for policy 0, policy_version 1042925 (0.0005) [2023-12-27 12:52:34,103][20381] Updated weights for policy 1, policy_version 1042119 (0.0007) [2023-12-27 12:52:34,130][20347] Updated weights for policy 0, policy_version 1042935 (0.0007) [2023-12-27 12:52:34,157][20381] Updated weights for policy 1, policy_version 1042129 (0.0006) [2023-12-27 12:52:34,189][20347] Updated weights for policy 0, policy_version 1042945 (0.0008) [2023-12-27 12:52:34,849][20347] Updated weights for policy 0, policy_version 1042955 (0.0008) [2023-12-27 12:52:34,892][20381] Updated weights for policy 1, policy_version 1042139 (0.0007) [2023-12-27 12:52:34,897][20347] Updated weights for policy 0, policy_version 1042965 (0.0007) [2023-12-27 12:52:34,941][20381] Updated weights for policy 1, policy_version 1042149 (0.0006) [2023-12-27 12:52:34,947][20347] Updated weights for policy 0, policy_version 1042975 (0.0006) [2023-12-27 12:52:34,997][20381] Updated weights for policy 1, policy_version 1042159 (0.0009) [2023-12-27 12:52:35,625][20347] Updated weights for policy 0, policy_version 1042985 (0.0007) [2023-12-27 12:52:35,679][20347] Updated weights for policy 0, policy_version 1042995 (0.0007) [2023-12-27 12:52:35,704][20381] Updated weights for policy 1, policy_version 1042169 (0.0008) [2023-12-27 12:52:35,731][20347] Updated weights for policy 0, policy_version 1043005 (0.0006) [2023-12-27 12:52:35,749][20381] Updated weights for policy 1, policy_version 1042179 (0.0010) [2023-12-27 12:52:35,781][20347] Updated weights for policy 0, policy_version 1043015 (0.0005) [2023-12-27 12:52:35,798][20381] Updated weights for policy 1, policy_version 1042189 (0.0010) [2023-12-27 12:52:35,847][20381] Updated weights for policy 1, policy_version 1042199 (0.0010) [2023-12-27 12:52:36,512][20347] Updated weights for policy 0, policy_version 1043025 (0.0007) [2023-12-27 12:52:36,576][20347] Updated weights for policy 0, policy_version 1043035 (0.0006) [2023-12-27 12:52:36,600][20381] Updated weights for policy 1, policy_version 1042209 (0.0008) [2023-12-27 12:52:36,641][20347] Updated weights for policy 0, policy_version 1043045 (0.0007) [2023-12-27 12:52:36,658][20381] Updated weights for policy 1, policy_version 1042219 (0.0007) [2023-12-27 12:52:36,708][20381] Updated weights for policy 1, policy_version 1042229 (0.0007) [2023-12-27 12:52:37,246][20347] Updated weights for policy 0, policy_version 1043055 (0.0009) [2023-12-27 12:52:37,299][20347] Updated weights for policy 0, policy_version 1043065 (0.0010) [2023-12-27 12:52:37,351][20347] Updated weights for policy 0, policy_version 1043075 (0.0010) [2023-12-27 12:52:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 533905408. Throughput: 0: 10430.4, 1: 10275.3. Samples: 533882028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:37,363][19170] Avg episode reward: [(0, '960.900'), (1, '853.020')] [2023-12-27 12:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001042232_266838016.pth... [2023-12-27 12:52:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001041048_266534912.pth [2023-12-27 12:52:37,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001043080_267075584.pth... [2023-12-27 12:52:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001041832_266756096.pth [2023-12-27 12:52:37,426][20381] Updated weights for policy 1, policy_version 1042239 (0.0009) [2023-12-27 12:52:37,486][20381] Updated weights for policy 1, policy_version 1042249 (0.0011) [2023-12-27 12:52:37,548][20381] Updated weights for policy 1, policy_version 1042259 (0.0011) [2023-12-27 12:52:37,984][20347] Updated weights for policy 0, policy_version 1043085 (0.0008) [2023-12-27 12:52:38,041][20347] Updated weights for policy 0, policy_version 1043095 (0.0006) [2023-12-27 12:52:38,100][20347] Updated weights for policy 0, policy_version 1043105 (0.0007) [2023-12-27 12:52:38,227][20381] Updated weights for policy 1, policy_version 1042269 (0.0010) [2023-12-27 12:52:38,275][20381] Updated weights for policy 1, policy_version 1042279 (0.0010) [2023-12-27 12:52:38,332][20381] Updated weights for policy 1, policy_version 1042289 (0.0006) [2023-12-27 12:52:38,792][20347] Updated weights for policy 0, policy_version 1043115 (0.0008) [2023-12-27 12:52:38,845][20347] Updated weights for policy 0, policy_version 1043125 (0.0008) [2023-12-27 12:52:38,907][20347] Updated weights for policy 0, policy_version 1043135 (0.0008) [2023-12-27 12:52:38,947][20381] Updated weights for policy 1, policy_version 1042299 (0.0007) [2023-12-27 12:52:39,005][20381] Updated weights for policy 1, policy_version 1042309 (0.0008) [2023-12-27 12:52:39,064][20381] Updated weights for policy 1, policy_version 1042319 (0.0007) [2023-12-27 12:52:39,652][20347] Updated weights for policy 0, policy_version 1043145 (0.0007) [2023-12-27 12:52:39,715][20347] Updated weights for policy 0, policy_version 1043155 (0.0009) [2023-12-27 12:52:39,776][20381] Updated weights for policy 1, policy_version 1042329 (0.0008) [2023-12-27 12:52:39,784][20347] Updated weights for policy 0, policy_version 1043165 (0.0009) [2023-12-27 12:52:39,835][20381] Updated weights for policy 1, policy_version 1042339 (0.0008) [2023-12-27 12:52:39,848][20347] Updated weights for policy 0, policy_version 1043175 (0.0009) [2023-12-27 12:52:39,904][20381] Updated weights for policy 1, policy_version 1042349 (0.0009) [2023-12-27 12:52:39,976][20381] Updated weights for policy 1, policy_version 1042359 (0.0007) [2023-12-27 12:52:40,494][20347] Updated weights for policy 0, policy_version 1043185 (0.0009) [2023-12-27 12:52:40,548][20347] Updated weights for policy 0, policy_version 1043195 (0.0008) [2023-12-27 12:52:40,605][20347] Updated weights for policy 0, policy_version 1043205 (0.0008) [2023-12-27 12:52:40,746][20381] Updated weights for policy 1, policy_version 1042369 (0.0007) [2023-12-27 12:52:40,804][20381] Updated weights for policy 1, policy_version 1042379 (0.0006) [2023-12-27 12:52:40,867][20381] Updated weights for policy 1, policy_version 1042389 (0.0005) [2023-12-27 12:52:41,386][20347] Updated weights for policy 0, policy_version 1043215 (0.0007) [2023-12-27 12:52:41,451][20347] Updated weights for policy 0, policy_version 1043225 (0.0009) [2023-12-27 12:52:41,509][20381] Updated weights for policy 1, policy_version 1042399 (0.0006) [2023-12-27 12:52:41,514][20347] Updated weights for policy 0, policy_version 1043235 (0.0008) [2023-12-27 12:52:41,568][20381] Updated weights for policy 1, policy_version 1042409 (0.0006) [2023-12-27 12:52:41,635][20381] Updated weights for policy 1, policy_version 1042419 (0.0007) [2023-12-27 12:52:42,228][20347] Updated weights for policy 0, policy_version 1043245 (0.0008) [2023-12-27 12:52:42,288][20347] Updated weights for policy 0, policy_version 1043255 (0.0007) [2023-12-27 12:52:42,309][20381] Updated weights for policy 1, policy_version 1042429 (0.0007) [2023-12-27 12:52:42,348][20347] Updated weights for policy 0, policy_version 1043265 (0.0006) [2023-12-27 12:52:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 534003712. Throughput: 0: 10466.8, 1: 10223.9. Samples: 534003108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:42,363][19170] Avg episode reward: [(0, '941.570'), (1, '843.200')] [2023-12-27 12:52:42,376][20381] Updated weights for policy 1, policy_version 1042439 (0.0007) [2023-12-27 12:52:42,437][20381] Updated weights for policy 1, policy_version 1042449 (0.0008) [2023-12-27 12:52:43,013][20347] Updated weights for policy 0, policy_version 1043275 (0.0007) [2023-12-27 12:52:43,067][20381] Updated weights for policy 1, policy_version 1042459 (0.0006) [2023-12-27 12:52:43,069][20347] Updated weights for policy 0, policy_version 1043285 (0.0008) [2023-12-27 12:52:43,126][20381] Updated weights for policy 1, policy_version 1042469 (0.0006) [2023-12-27 12:52:43,127][20347] Updated weights for policy 0, policy_version 1043295 (0.0009) [2023-12-27 12:52:43,173][20381] Updated weights for policy 1, policy_version 1042479 (0.0008) [2023-12-27 12:52:43,753][20347] Updated weights for policy 0, policy_version 1043305 (0.0008) [2023-12-27 12:52:43,804][20347] Updated weights for policy 0, policy_version 1043315 (0.0005) [2023-12-27 12:52:43,853][20347] Updated weights for policy 0, policy_version 1043325 (0.0005) [2023-12-27 12:52:43,896][20381] Updated weights for policy 1, policy_version 1042489 (0.0007) [2023-12-27 12:52:43,904][20347] Updated weights for policy 0, policy_version 1043335 (0.0005) [2023-12-27 12:52:43,951][20381] Updated weights for policy 1, policy_version 1042499 (0.0008) [2023-12-27 12:52:44,000][20381] Updated weights for policy 1, policy_version 1042509 (0.0007) [2023-12-27 12:52:44,055][20381] Updated weights for policy 1, policy_version 1042519 (0.0005) [2023-12-27 12:52:44,434][20347] Updated weights for policy 0, policy_version 1043345 (0.0010) [2023-12-27 12:52:44,486][20347] Updated weights for policy 0, policy_version 1043355 (0.0010) [2023-12-27 12:52:44,538][20347] Updated weights for policy 0, policy_version 1043365 (0.0010) [2023-12-27 12:52:44,684][20381] Updated weights for policy 1, policy_version 1042529 (0.0010) [2023-12-27 12:52:44,745][20381] Updated weights for policy 1, policy_version 1042539 (0.0008) [2023-12-27 12:52:44,810][20381] Updated weights for policy 1, policy_version 1042549 (0.0011) [2023-12-27 12:52:45,257][20347] Updated weights for policy 0, policy_version 1043375 (0.0008) [2023-12-27 12:52:45,308][20347] Updated weights for policy 0, policy_version 1043385 (0.0007) [2023-12-27 12:52:45,363][20347] Updated weights for policy 0, policy_version 1043395 (0.0008) [2023-12-27 12:52:45,518][20381] Updated weights for policy 1, policy_version 1042559 (0.0011) [2023-12-27 12:52:45,569][20381] Updated weights for policy 1, policy_version 1042569 (0.0010) [2023-12-27 12:52:45,621][20381] Updated weights for policy 1, policy_version 1042579 (0.0010) [2023-12-27 12:52:46,012][20347] Updated weights for policy 0, policy_version 1043405 (0.0007) [2023-12-27 12:52:46,073][20347] Updated weights for policy 0, policy_version 1043415 (0.0007) [2023-12-27 12:52:46,136][20347] Updated weights for policy 0, policy_version 1043425 (0.0007) [2023-12-27 12:52:46,265][20381] Updated weights for policy 1, policy_version 1042589 (0.0008) [2023-12-27 12:52:46,316][20381] Updated weights for policy 1, policy_version 1042599 (0.0005) [2023-12-27 12:52:46,369][20381] Updated weights for policy 1, policy_version 1042609 (0.0005) [2023-12-27 12:52:46,783][20347] Updated weights for policy 0, policy_version 1043435 (0.0008) [2023-12-27 12:52:46,834][20347] Updated weights for policy 0, policy_version 1043445 (0.0008) [2023-12-27 12:52:46,880][20347] Updated weights for policy 0, policy_version 1043455 (0.0007) [2023-12-27 12:52:46,906][20381] Updated weights for policy 1, policy_version 1042619 (0.0007) [2023-12-27 12:52:46,964][20381] Updated weights for policy 1, policy_version 1042629 (0.0011) [2023-12-27 12:52:47,019][20381] Updated weights for policy 1, policy_version 1042639 (0.0011) [2023-12-27 12:52:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 534118400. Throughput: 0: 10507.7, 1: 10254.9. Samples: 534128712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:47,363][19170] Avg episode reward: [(0, '951.520'), (1, '843.200')] [2023-12-27 12:52:47,581][20347] Updated weights for policy 0, policy_version 1043465 (0.0007) [2023-12-27 12:52:47,619][20381] Updated weights for policy 1, policy_version 1042649 (0.0010) [2023-12-27 12:52:47,633][20347] Updated weights for policy 0, policy_version 1043475 (0.0007) [2023-12-27 12:52:47,674][20381] Updated weights for policy 1, policy_version 1042659 (0.0010) [2023-12-27 12:52:47,680][20347] Updated weights for policy 0, policy_version 1043485 (0.0006) [2023-12-27 12:52:47,726][20381] Updated weights for policy 1, policy_version 1042669 (0.0010) [2023-12-27 12:52:47,732][20347] Updated weights for policy 0, policy_version 1043495 (0.0005) [2023-12-27 12:52:47,778][20381] Updated weights for policy 1, policy_version 1042679 (0.0010) [2023-12-27 12:52:48,348][20381] Updated weights for policy 1, policy_version 1042689 (0.0009) [2023-12-27 12:52:48,406][20381] Updated weights for policy 1, policy_version 1042699 (0.0008) [2023-12-27 12:52:48,462][20381] Updated weights for policy 1, policy_version 1042709 (0.0011) [2023-12-27 12:52:48,475][20347] Updated weights for policy 0, policy_version 1043505 (0.0005) [2023-12-27 12:52:48,538][20347] Updated weights for policy 0, policy_version 1043515 (0.0008) [2023-12-27 12:52:48,599][20347] Updated weights for policy 0, policy_version 1043525 (0.0007) [2023-12-27 12:52:49,137][20381] Updated weights for policy 1, policy_version 1042719 (0.0011) [2023-12-27 12:52:49,193][20381] Updated weights for policy 1, policy_version 1042729 (0.0011) [2023-12-27 12:52:49,264][20381] Updated weights for policy 1, policy_version 1042739 (0.0010) [2023-12-27 12:52:49,303][20347] Updated weights for policy 0, policy_version 1043535 (0.0008) [2023-12-27 12:52:49,366][20347] Updated weights for policy 0, policy_version 1043545 (0.0008) [2023-12-27 12:52:49,379][20291] KL-divergence is very high: 269.7126 [2023-12-27 12:52:49,427][20291] KL-divergence is very high: 135.8373 [2023-12-27 12:52:49,427][20347] Updated weights for policy 0, policy_version 1043555 (0.0007) [2023-12-27 12:52:49,966][20381] Updated weights for policy 1, policy_version 1042749 (0.0010) [2023-12-27 12:52:50,028][20381] Updated weights for policy 1, policy_version 1042759 (0.0008) [2023-12-27 12:52:50,092][20381] Updated weights for policy 1, policy_version 1042769 (0.0009) [2023-12-27 12:52:50,147][20347] Updated weights for policy 0, policy_version 1043565 (0.0008) [2023-12-27 12:52:50,209][20347] Updated weights for policy 0, policy_version 1043575 (0.0007) [2023-12-27 12:52:50,266][20347] Updated weights for policy 0, policy_version 1043585 (0.0007) [2023-12-27 12:52:50,798][20381] Updated weights for policy 1, policy_version 1042779 (0.0011) [2023-12-27 12:52:50,843][20381] Updated weights for policy 1, policy_version 1042789 (0.0010) [2023-12-27 12:52:50,895][20381] Updated weights for policy 1, policy_version 1042799 (0.0011) [2023-12-27 12:52:50,981][20347] Updated weights for policy 0, policy_version 1043595 (0.0008) [2023-12-27 12:52:51,042][20347] Updated weights for policy 0, policy_version 1043605 (0.0007) [2023-12-27 12:52:51,098][20347] Updated weights for policy 0, policy_version 1043615 (0.0008) [2023-12-27 12:52:51,623][20381] Updated weights for policy 1, policy_version 1042809 (0.0010) [2023-12-27 12:52:51,696][20381] Updated weights for policy 1, policy_version 1042819 (0.0011) [2023-12-27 12:52:51,772][20381] Updated weights for policy 1, policy_version 1042829 (0.0010) [2023-12-27 12:52:51,832][20381] Updated weights for policy 1, policy_version 1042839 (0.0006) [2023-12-27 12:52:51,883][20347] Updated weights for policy 0, policy_version 1043625 (0.0008) [2023-12-27 12:52:51,946][20347] Updated weights for policy 0, policy_version 1043635 (0.0008) [2023-12-27 12:52:52,009][20347] Updated weights for policy 0, policy_version 1043645 (0.0008) [2023-12-27 12:52:52,062][20347] Updated weights for policy 0, policy_version 1043655 (0.0007) [2023-12-27 12:52:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 534216704. Throughput: 0: 10427.4, 1: 10263.7. Samples: 534189516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:52,363][19170] Avg episode reward: [(0, '912.400'), (1, '882.530')] [2023-12-27 12:52:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001042840_266993664.pth... [2023-12-27 12:52:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001043656_267223040.pth... [2023-12-27 12:52:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001042440_266911744.pth [2023-12-27 12:52:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001041656_266690560.pth [2023-12-27 12:52:52,463][20381] Updated weights for policy 1, policy_version 1042849 (0.0006) [2023-12-27 12:52:52,514][20381] Updated weights for policy 1, policy_version 1042859 (0.0006) [2023-12-27 12:52:52,583][20381] Updated weights for policy 1, policy_version 1042869 (0.0007) [2023-12-27 12:52:52,814][20347] Updated weights for policy 0, policy_version 1043665 (0.0010) [2023-12-27 12:52:52,876][20347] Updated weights for policy 0, policy_version 1043675 (0.0007) [2023-12-27 12:52:52,932][20347] Updated weights for policy 0, policy_version 1043685 (0.0006) [2023-12-27 12:52:53,243][20381] Updated weights for policy 1, policy_version 1042879 (0.0007) [2023-12-27 12:52:53,301][20381] Updated weights for policy 1, policy_version 1042889 (0.0006) [2023-12-27 12:52:53,354][20381] Updated weights for policy 1, policy_version 1042899 (0.0009) [2023-12-27 12:52:53,531][20347] Updated weights for policy 0, policy_version 1043695 (0.0005) [2023-12-27 12:52:53,589][20347] Updated weights for policy 0, policy_version 1043705 (0.0005) [2023-12-27 12:52:53,645][20347] Updated weights for policy 0, policy_version 1043715 (0.0008) [2023-12-27 12:52:54,085][20381] Updated weights for policy 1, policy_version 1042909 (0.0008) [2023-12-27 12:52:54,143][20381] Updated weights for policy 1, policy_version 1042919 (0.0007) [2023-12-27 12:52:54,202][20381] Updated weights for policy 1, policy_version 1042929 (0.0008) [2023-12-27 12:52:54,232][20347] Updated weights for policy 0, policy_version 1043725 (0.0008) [2023-12-27 12:52:54,286][20347] Updated weights for policy 0, policy_version 1043735 (0.0010) [2023-12-27 12:52:54,344][20347] Updated weights for policy 0, policy_version 1043745 (0.0010) [2023-12-27 12:52:54,839][20381] Updated weights for policy 1, policy_version 1042939 (0.0008) [2023-12-27 12:52:54,890][20381] Updated weights for policy 1, policy_version 1042949 (0.0007) [2023-12-27 12:52:54,941][20381] Updated weights for policy 1, policy_version 1042959 (0.0007) [2023-12-27 12:52:54,989][20347] Updated weights for policy 0, policy_version 1043755 (0.0010) [2023-12-27 12:52:55,053][20347] Updated weights for policy 0, policy_version 1043765 (0.0008) [2023-12-27 12:52:55,113][20347] Updated weights for policy 0, policy_version 1043775 (0.0007) [2023-12-27 12:52:55,634][20381] Updated weights for policy 1, policy_version 1042969 (0.0006) [2023-12-27 12:52:55,695][20381] Updated weights for policy 1, policy_version 1042979 (0.0008) [2023-12-27 12:52:55,753][20381] Updated weights for policy 1, policy_version 1042989 (0.0008) [2023-12-27 12:52:55,757][20347] Updated weights for policy 0, policy_version 1043785 (0.0007) [2023-12-27 12:52:55,811][20347] Updated weights for policy 0, policy_version 1043795 (0.0006) [2023-12-27 12:52:55,811][20381] Updated weights for policy 1, policy_version 1042999 (0.0007) [2023-12-27 12:52:55,875][20347] Updated weights for policy 0, policy_version 1043805 (0.0005) [2023-12-27 12:52:55,928][20347] Updated weights for policy 0, policy_version 1043815 (0.0006) [2023-12-27 12:52:56,467][20347] Updated weights for policy 0, policy_version 1043825 (0.0009) [2023-12-27 12:52:56,517][20381] Updated weights for policy 1, policy_version 1043009 (0.0007) [2023-12-27 12:52:56,524][20347] Updated weights for policy 0, policy_version 1043835 (0.0009) [2023-12-27 12:52:56,569][20381] Updated weights for policy 1, policy_version 1043019 (0.0006) [2023-12-27 12:52:56,579][20347] Updated weights for policy 0, policy_version 1043845 (0.0010) [2023-12-27 12:52:56,611][20381] Updated weights for policy 1, policy_version 1043029 (0.0006) [2023-12-27 12:52:57,131][20347] Updated weights for policy 0, policy_version 1043855 (0.0010) [2023-12-27 12:52:57,191][20347] Updated weights for policy 0, policy_version 1043865 (0.0010) [2023-12-27 12:52:57,242][20347] Updated weights for policy 0, policy_version 1043875 (0.0010) [2023-12-27 12:52:57,323][20381] Updated weights for policy 1, policy_version 1043039 (0.0006) [2023-12-27 12:52:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 534323200. Throughput: 0: 10437.6, 1: 10266.9. Samples: 534311476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:52:57,363][19170] Avg episode reward: [(0, '892.620'), (1, '882.680')] [2023-12-27 12:52:57,382][20381] Updated weights for policy 1, policy_version 1043049 (0.0007) [2023-12-27 12:52:57,430][20381] Updated weights for policy 1, policy_version 1043059 (0.0008) [2023-12-27 12:52:57,929][20347] Updated weights for policy 0, policy_version 1043885 (0.0008) [2023-12-27 12:52:57,985][20347] Updated weights for policy 0, policy_version 1043895 (0.0006) [2023-12-27 12:52:58,040][20347] Updated weights for policy 0, policy_version 1043905 (0.0007) [2023-12-27 12:52:58,082][20381] Updated weights for policy 1, policy_version 1043069 (0.0007) [2023-12-27 12:52:58,130][20381] Updated weights for policy 1, policy_version 1043079 (0.0007) [2023-12-27 12:52:58,179][20381] Updated weights for policy 1, policy_version 1043089 (0.0008) [2023-12-27 12:52:58,745][20347] Updated weights for policy 0, policy_version 1043915 (0.0007) [2023-12-27 12:52:58,811][20347] Updated weights for policy 0, policy_version 1043925 (0.0006) [2023-12-27 12:52:58,873][20347] Updated weights for policy 0, policy_version 1043935 (0.0007) [2023-12-27 12:52:58,992][20381] Updated weights for policy 1, policy_version 1043099 (0.0007) [2023-12-27 12:52:59,054][20381] Updated weights for policy 1, policy_version 1043109 (0.0008) [2023-12-27 12:52:59,119][20381] Updated weights for policy 1, policy_version 1043119 (0.0007) [2023-12-27 12:52:59,630][20347] Updated weights for policy 0, policy_version 1043945 (0.0009) [2023-12-27 12:52:59,686][20347] Updated weights for policy 0, policy_version 1043955 (0.0007) [2023-12-27 12:52:59,746][20347] Updated weights for policy 0, policy_version 1043965 (0.0008) [2023-12-27 12:52:59,809][20347] Updated weights for policy 0, policy_version 1043975 (0.0008) [2023-12-27 12:52:59,880][20381] Updated weights for policy 1, policy_version 1043129 (0.0005) [2023-12-27 12:52:59,951][20381] Updated weights for policy 1, policy_version 1043139 (0.0008) [2023-12-27 12:53:00,010][20381] Updated weights for policy 1, policy_version 1043149 (0.0006) [2023-12-27 12:53:00,057][20381] Updated weights for policy 1, policy_version 1043159 (0.0005) [2023-12-27 12:53:00,512][20347] Updated weights for policy 0, policy_version 1043985 (0.0006) [2023-12-27 12:53:00,576][20347] Updated weights for policy 0, policy_version 1043995 (0.0006) [2023-12-27 12:53:00,624][20347] Updated weights for policy 0, policy_version 1044005 (0.0006) [2023-12-27 12:53:00,722][20381] Updated weights for policy 1, policy_version 1043169 (0.0008) [2023-12-27 12:53:00,768][20381] Updated weights for policy 1, policy_version 1043179 (0.0007) [2023-12-27 12:53:00,811][20381] Updated weights for policy 1, policy_version 1043189 (0.0007) [2023-12-27 12:53:01,253][20347] Updated weights for policy 0, policy_version 1044015 (0.0007) [2023-12-27 12:53:01,304][20347] Updated weights for policy 0, policy_version 1044025 (0.0008) [2023-12-27 12:53:01,360][20347] Updated weights for policy 0, policy_version 1044035 (0.0008) [2023-12-27 12:53:01,501][20381] Updated weights for policy 1, policy_version 1043199 (0.0006) [2023-12-27 12:53:01,562][20381] Updated weights for policy 1, policy_version 1043209 (0.0008) [2023-12-27 12:53:01,631][20381] Updated weights for policy 1, policy_version 1043219 (0.0009) [2023-12-27 12:53:02,081][20347] Updated weights for policy 0, policy_version 1044045 (0.0008) [2023-12-27 12:53:02,135][20347] Updated weights for policy 0, policy_version 1044055 (0.0007) [2023-12-27 12:53:02,195][20347] Updated weights for policy 0, policy_version 1044065 (0.0006) [2023-12-27 12:53:02,312][20381] Updated weights for policy 1, policy_version 1043229 (0.0009) [2023-12-27 12:53:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 534421504. Throughput: 0: 10411.0, 1: 10172.2. Samples: 534432636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:53:02,363][19170] Avg episode reward: [(0, '931.530'), (1, '863.050')] [2023-12-27 12:53:02,375][20381] Updated weights for policy 1, policy_version 1043239 (0.0008) [2023-12-27 12:53:02,434][20381] Updated weights for policy 1, policy_version 1043249 (0.0007) [2023-12-27 12:53:02,878][20347] Updated weights for policy 0, policy_version 1044075 (0.0007) [2023-12-27 12:53:02,926][20347] Updated weights for policy 0, policy_version 1044085 (0.0005) [2023-12-27 12:53:02,976][20347] Updated weights for policy 0, policy_version 1044095 (0.0005) [2023-12-27 12:53:03,136][20381] Updated weights for policy 1, policy_version 1043259 (0.0007) [2023-12-27 12:53:03,183][20381] Updated weights for policy 1, policy_version 1043269 (0.0007) [2023-12-27 12:53:03,238][20381] Updated weights for policy 1, policy_version 1043279 (0.0007) [2023-12-27 12:53:03,631][20347] Updated weights for policy 0, policy_version 1044105 (0.0005) [2023-12-27 12:53:03,680][20347] Updated weights for policy 0, policy_version 1044115 (0.0008) [2023-12-27 12:53:03,723][20347] Updated weights for policy 0, policy_version 1044125 (0.0007) [2023-12-27 12:53:03,774][20347] Updated weights for policy 0, policy_version 1044135 (0.0005) [2023-12-27 12:53:03,873][20381] Updated weights for policy 1, policy_version 1043289 (0.0007) [2023-12-27 12:53:03,928][20381] Updated weights for policy 1, policy_version 1043299 (0.0008) [2023-12-27 12:53:03,991][20381] Updated weights for policy 1, policy_version 1043309 (0.0008) [2023-12-27 12:53:04,042][20381] Updated weights for policy 1, policy_version 1043319 (0.0008) [2023-12-27 12:53:04,463][20291] KL-divergence is very high: 387.0686 [2023-12-27 12:53:04,475][20291] KL-divergence is very high: 335.0259 [2023-12-27 12:53:04,476][20347] Updated weights for policy 0, policy_version 1044145 (0.0007) [2023-12-27 12:53:04,511][20291] KL-divergence is very high: 679.8607 [2023-12-27 12:53:04,523][20291] KL-divergence is very high: 377.3716 [2023-12-27 12:53:04,537][20347] Updated weights for policy 0, policy_version 1044155 (0.0007) [2023-12-27 12:53:04,593][20347] Updated weights for policy 0, policy_version 1044165 (0.0007) [2023-12-27 12:53:04,607][20291] KL-divergence is very high: 109.5847 [2023-12-27 12:53:04,751][20381] Updated weights for policy 1, policy_version 1043329 (0.0010) [2023-12-27 12:53:04,809][20381] Updated weights for policy 1, policy_version 1043339 (0.0010) [2023-12-27 12:53:04,867][20381] Updated weights for policy 1, policy_version 1043349 (0.0010) [2023-12-27 12:53:05,206][20347] Updated weights for policy 0, policy_version 1044175 (0.0006) [2023-12-27 12:53:05,260][20347] Updated weights for policy 0, policy_version 1044185 (0.0007) [2023-12-27 12:53:05,306][20347] Updated weights for policy 0, policy_version 1044195 (0.0008) [2023-12-27 12:53:05,553][20381] Updated weights for policy 1, policy_version 1043359 (0.0010) [2023-12-27 12:53:05,614][20381] Updated weights for policy 1, policy_version 1043369 (0.0010) [2023-12-27 12:53:05,673][20381] Updated weights for policy 1, policy_version 1043379 (0.0010) [2023-12-27 12:53:05,947][20347] Updated weights for policy 0, policy_version 1044205 (0.0008) [2023-12-27 12:53:05,962][20291] KL-divergence is very high: 123.3152 [2023-12-27 12:53:06,006][20347] Updated weights for policy 0, policy_version 1044215 (0.0010) [2023-12-27 12:53:06,056][20347] Updated weights for policy 0, policy_version 1044225 (0.0010) [2023-12-27 12:53:06,376][20381] Updated weights for policy 1, policy_version 1043389 (0.0010) [2023-12-27 12:53:06,440][20381] Updated weights for policy 1, policy_version 1043399 (0.0007) [2023-12-27 12:53:06,503][20381] Updated weights for policy 1, policy_version 1043409 (0.0005) [2023-12-27 12:53:06,764][20347] Updated weights for policy 0, policy_version 1044235 (0.0010) [2023-12-27 12:53:06,812][20347] Updated weights for policy 0, policy_version 1044245 (0.0009) [2023-12-27 12:53:06,860][20347] Updated weights for policy 0, policy_version 1044255 (0.0007) [2023-12-27 12:53:07,106][20381] Updated weights for policy 1, policy_version 1043419 (0.0005) [2023-12-27 12:53:07,170][20381] Updated weights for policy 1, policy_version 1043429 (0.0005) [2023-12-27 12:53:07,225][20381] Updated weights for policy 1, policy_version 1043439 (0.0005) [2023-12-27 12:53:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 534528000. Throughput: 0: 10425.2, 1: 10205.0. Samples: 534494692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:53:07,363][19170] Avg episode reward: [(0, '902.500'), (1, '862.920')] [2023-12-27 12:53:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001043448_267149312.pth... [2023-12-27 12:53:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001044264_267378688.pth... [2023-12-27 12:53:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001042232_266838016.pth [2023-12-27 12:53:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001043080_267075584.pth [2023-12-27 12:53:07,502][20347] Updated weights for policy 0, policy_version 1044265 (0.0006) [2023-12-27 12:53:07,570][20347] Updated weights for policy 0, policy_version 1044275 (0.0010) [2023-12-27 12:53:07,621][20347] Updated weights for policy 0, policy_version 1044285 (0.0010) [2023-12-27 12:53:07,679][20347] Updated weights for policy 0, policy_version 1044295 (0.0010) [2023-12-27 12:53:07,815][20381] Updated weights for policy 1, policy_version 1043449 (0.0006) [2023-12-27 12:53:07,880][20381] Updated weights for policy 1, policy_version 1043459 (0.0008) [2023-12-27 12:53:07,936][20381] Updated weights for policy 1, policy_version 1043469 (0.0005) [2023-12-27 12:53:07,995][20381] Updated weights for policy 1, policy_version 1043479 (0.0008) [2023-12-27 12:53:08,292][20347] Updated weights for policy 0, policy_version 1044305 (0.0009) [2023-12-27 12:53:08,322][20291] KL-divergence is very high: 413.0581 [2023-12-27 12:53:08,346][20347] Updated weights for policy 0, policy_version 1044315 (0.0009) [2023-12-27 12:53:08,371][20291] KL-divergence is very high: 370.3333 [2023-12-27 12:53:08,401][20347] Updated weights for policy 0, policy_version 1044325 (0.0008) [2023-12-27 12:53:08,411][20291] KL-divergence is very high: 142.7181 [2023-12-27 12:53:08,649][20381] Updated weights for policy 1, policy_version 1043489 (0.0010) [2023-12-27 12:53:08,710][20381] Updated weights for policy 1, policy_version 1043499 (0.0011) [2023-12-27 12:53:08,770][20381] Updated weights for policy 1, policy_version 1043509 (0.0011) [2023-12-27 12:53:09,075][20347] Updated weights for policy 0, policy_version 1044335 (0.0010) [2023-12-27 12:53:09,139][20347] Updated weights for policy 0, policy_version 1044345 (0.0010) [2023-12-27 12:53:09,205][20347] Updated weights for policy 0, policy_version 1044355 (0.0011) [2023-12-27 12:53:09,368][20381] Updated weights for policy 1, policy_version 1043519 (0.0009) [2023-12-27 12:53:09,438][20381] Updated weights for policy 1, policy_version 1043529 (0.0008) [2023-12-27 12:53:09,501][20381] Updated weights for policy 1, policy_version 1043539 (0.0010) [2023-12-27 12:53:09,954][20347] Updated weights for policy 0, policy_version 1044365 (0.0009) [2023-12-27 12:53:10,007][20347] Updated weights for policy 0, policy_version 1044375 (0.0007) [2023-12-27 12:53:10,069][20347] Updated weights for policy 0, policy_version 1044385 (0.0007) [2023-12-27 12:53:10,190][20381] Updated weights for policy 1, policy_version 1043549 (0.0011) [2023-12-27 12:53:10,246][20381] Updated weights for policy 1, policy_version 1043559 (0.0011) [2023-12-27 12:53:10,302][20381] Updated weights for policy 1, policy_version 1043569 (0.0011) [2023-12-27 12:53:10,773][20347] Updated weights for policy 0, policy_version 1044395 (0.0007) [2023-12-27 12:53:10,810][20291] KL-divergence is very high: 111.4032 [2023-12-27 12:53:10,832][20347] Updated weights for policy 0, policy_version 1044405 (0.0008) [2023-12-27 12:53:10,885][20347] Updated weights for policy 0, policy_version 1044415 (0.0006) [2023-12-27 12:53:10,989][20381] Updated weights for policy 1, policy_version 1043579 (0.0010) [2023-12-27 12:53:11,054][20381] Updated weights for policy 1, policy_version 1043589 (0.0007) [2023-12-27 12:53:11,125][20381] Updated weights for policy 1, policy_version 1043599 (0.0008) [2023-12-27 12:53:11,624][20347] Updated weights for policy 0, policy_version 1044425 (0.0007) [2023-12-27 12:53:11,692][20347] Updated weights for policy 0, policy_version 1044435 (0.0010) [2023-12-27 12:53:11,728][20291] KL-divergence is very high: 110.7955 [2023-12-27 12:53:11,741][20291] KL-divergence is very high: 170.5162 [2023-12-27 12:53:11,759][20347] Updated weights for policy 0, policy_version 1044445 (0.0009) [2023-12-27 12:53:11,815][20347] Updated weights for policy 0, policy_version 1044455 (0.0007) [2023-12-27 12:53:11,847][20381] Updated weights for policy 1, policy_version 1043609 (0.0008) [2023-12-27 12:53:11,908][20381] Updated weights for policy 1, policy_version 1043619 (0.0006) [2023-12-27 12:53:11,974][20381] Updated weights for policy 1, policy_version 1043629 (0.0007) [2023-12-27 12:53:12,039][20381] Updated weights for policy 1, policy_version 1043639 (0.0006) [2023-12-27 12:53:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 534626304. Throughput: 0: 10373.9, 1: 10252.6. Samples: 534618128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:53:12,363][19170] Avg episode reward: [(0, '814.450'), (1, '843.290')] [2023-12-27 12:53:12,556][20347] Updated weights for policy 0, policy_version 1044465 (0.0008) [2023-12-27 12:53:12,609][20347] Updated weights for policy 0, policy_version 1044475 (0.0008) [2023-12-27 12:53:12,666][20347] Updated weights for policy 0, policy_version 1044485 (0.0007) [2023-12-27 12:53:12,684][20381] Updated weights for policy 1, policy_version 1043649 (0.0006) [2023-12-27 12:53:12,737][20381] Updated weights for policy 1, policy_version 1043659 (0.0008) [2023-12-27 12:53:12,787][20381] Updated weights for policy 1, policy_version 1043669 (0.0008) [2023-12-27 12:53:13,286][20347] Updated weights for policy 0, policy_version 1044495 (0.0006) [2023-12-27 12:53:13,334][20347] Updated weights for policy 0, policy_version 1044505 (0.0006) [2023-12-27 12:53:13,379][20347] Updated weights for policy 0, policy_version 1044515 (0.0008) [2023-12-27 12:53:13,437][20381] Updated weights for policy 1, policy_version 1043679 (0.0008) [2023-12-27 12:53:13,493][20381] Updated weights for policy 1, policy_version 1043689 (0.0008) [2023-12-27 12:53:13,546][20381] Updated weights for policy 1, policy_version 1043699 (0.0008) [2023-12-27 12:53:14,005][20347] Updated weights for policy 0, policy_version 1044525 (0.0006) [2023-12-27 12:53:14,062][20347] Updated weights for policy 0, policy_version 1044535 (0.0007) [2023-12-27 12:53:14,121][20347] Updated weights for policy 0, policy_version 1044545 (0.0006) [2023-12-27 12:53:14,231][20381] Updated weights for policy 1, policy_version 1043709 (0.0008) [2023-12-27 12:53:14,286][20381] Updated weights for policy 1, policy_version 1043719 (0.0010) [2023-12-27 12:53:14,338][20381] Updated weights for policy 1, policy_version 1043729 (0.0010) [2023-12-27 12:53:14,738][20347] Updated weights for policy 0, policy_version 1044555 (0.0005) [2023-12-27 12:53:14,811][20347] Updated weights for policy 0, policy_version 1044565 (0.0007) [2023-12-27 12:53:14,863][20347] Updated weights for policy 0, policy_version 1044575 (0.0008) [2023-12-27 12:53:15,020][20381] Updated weights for policy 1, policy_version 1043739 (0.0010) [2023-12-27 12:53:15,082][20381] Updated weights for policy 1, policy_version 1043749 (0.0008) [2023-12-27 12:53:15,141][20381] Updated weights for policy 1, policy_version 1043759 (0.0007) [2023-12-27 12:53:15,556][20347] Updated weights for policy 0, policy_version 1044585 (0.0008) [2023-12-27 12:53:15,608][20347] Updated weights for policy 0, policy_version 1044595 (0.0005) [2023-12-27 12:53:15,655][20347] Updated weights for policy 0, policy_version 1044605 (0.0005) [2023-12-27 12:53:15,709][20347] Updated weights for policy 0, policy_version 1044615 (0.0006) [2023-12-27 12:53:15,828][20381] Updated weights for policy 1, policy_version 1043769 (0.0009) [2023-12-27 12:53:15,876][20381] Updated weights for policy 1, policy_version 1043779 (0.0010) [2023-12-27 12:53:15,928][20381] Updated weights for policy 1, policy_version 1043789 (0.0010) [2023-12-27 12:53:15,986][20381] Updated weights for policy 1, policy_version 1043799 (0.0010) [2023-12-27 12:53:16,352][20347] Updated weights for policy 0, policy_version 1044625 (0.0010) [2023-12-27 12:53:16,411][20347] Updated weights for policy 0, policy_version 1044635 (0.0010) [2023-12-27 12:53:16,469][20347] Updated weights for policy 0, policy_version 1044645 (0.0010) [2023-12-27 12:53:16,607][20381] Updated weights for policy 1, policy_version 1043809 (0.0010) [2023-12-27 12:53:16,658][20381] Updated weights for policy 1, policy_version 1043819 (0.0010) [2023-12-27 12:53:16,703][20381] Updated weights for policy 1, policy_version 1043829 (0.0010) [2023-12-27 12:53:17,143][20347] Updated weights for policy 0, policy_version 1044655 (0.0011) [2023-12-27 12:53:17,203][20347] Updated weights for policy 0, policy_version 1044665 (0.0010) [2023-12-27 12:53:17,264][20347] Updated weights for policy 0, policy_version 1044675 (0.0010) [2023-12-27 12:53:17,298][20381] Updated weights for policy 1, policy_version 1043839 (0.0007) [2023-12-27 12:53:17,360][20381] Updated weights for policy 1, policy_version 1043849 (0.0005) [2023-12-27 12:53:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 534732800. Throughput: 0: 10310.8, 1: 10201.7. Samples: 534743168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:53:17,363][19170] Avg episode reward: [(0, '853.270'), (1, '863.040')] [2023-12-27 12:53:17,406][20381] Updated weights for policy 1, policy_version 1043859 (0.0008) [2023-12-27 12:53:17,931][20347] Updated weights for policy 0, policy_version 1044685 (0.0010) [2023-12-27 12:53:17,979][20347] Updated weights for policy 0, policy_version 1044695 (0.0010) [2023-12-27 12:53:18,025][20381] Updated weights for policy 1, policy_version 1043869 (0.0009) [2023-12-27 12:53:18,034][20347] Updated weights for policy 0, policy_version 1044705 (0.0010) [2023-12-27 12:53:18,090][20381] Updated weights for policy 1, policy_version 1043879 (0.0009) [2023-12-27 12:53:18,148][20381] Updated weights for policy 1, policy_version 1043889 (0.0006) [2023-12-27 12:53:18,730][20347] Updated weights for policy 0, policy_version 1044715 (0.0010) [2023-12-27 12:53:18,774][20381] Updated weights for policy 1, policy_version 1043899 (0.0006) [2023-12-27 12:53:18,780][20347] Updated weights for policy 0, policy_version 1044725 (0.0007) [2023-12-27 12:53:18,828][20381] Updated weights for policy 1, policy_version 1043909 (0.0006) [2023-12-27 12:53:18,840][20347] Updated weights for policy 0, policy_version 1044735 (0.0007) [2023-12-27 12:53:18,882][20381] Updated weights for policy 1, policy_version 1043919 (0.0006) [2023-12-27 12:53:19,524][20347] Updated weights for policy 0, policy_version 1044745 (0.0006) [2023-12-27 12:53:19,581][20347] Updated weights for policy 0, policy_version 1044755 (0.0007) [2023-12-27 12:53:19,600][20381] Updated weights for policy 1, policy_version 1043929 (0.0007) [2023-12-27 12:53:19,636][20347] Updated weights for policy 0, policy_version 1044765 (0.0009) [2023-12-27 12:53:19,651][20381] Updated weights for policy 1, policy_version 1043939 (0.0006) [2023-12-27 12:53:19,669][20292] KL-divergence is very high: 142.9886 [2023-12-27 12:53:19,687][20292] KL-divergence is very high: 100.2998 [2023-12-27 12:53:19,694][20347] Updated weights for policy 0, policy_version 1044775 (0.0009) [2023-12-27 12:53:19,711][20381] Updated weights for policy 1, policy_version 1043949 (0.0007) [2023-12-27 12:53:19,765][20381] Updated weights for policy 1, policy_version 1043959 (0.0007) [2023-12-27 12:53:20,446][20381] Updated weights for policy 1, policy_version 1043969 (0.0007) [2023-12-27 12:53:20,459][20292] KL-divergence is very high: 127.0760 [2023-12-27 12:53:20,474][20347] Updated weights for policy 0, policy_version 1044785 (0.0008) [2023-12-27 12:53:20,502][20381] Updated weights for policy 1, policy_version 1043979 (0.0006) [2023-12-27 12:53:20,534][20347] Updated weights for policy 0, policy_version 1044795 (0.0008) [2023-12-27 12:53:20,560][20381] Updated weights for policy 1, policy_version 1043989 (0.0007) [2023-12-27 12:53:20,596][20347] Updated weights for policy 0, policy_version 1044805 (0.0007) [2023-12-27 12:53:21,324][20347] Updated weights for policy 0, policy_version 1044815 (0.0009) [2023-12-27 12:53:21,329][20381] Updated weights for policy 1, policy_version 1043999 (0.0006) [2023-12-27 12:53:21,395][20347] Updated weights for policy 0, policy_version 1044825 (0.0009) [2023-12-27 12:53:21,403][20381] Updated weights for policy 1, policy_version 1044009 (0.0008) [2023-12-27 12:53:21,454][20347] Updated weights for policy 0, policy_version 1044835 (0.0007) [2023-12-27 12:53:21,467][20381] Updated weights for policy 1, policy_version 1044019 (0.0007) [2023-12-27 12:53:22,126][20347] Updated weights for policy 0, policy_version 1044845 (0.0008) [2023-12-27 12:53:22,187][20347] Updated weights for policy 0, policy_version 1044855 (0.0010) [2023-12-27 12:53:22,218][20381] Updated weights for policy 1, policy_version 1044029 (0.0007) [2023-12-27 12:53:22,251][20347] Updated weights for policy 0, policy_version 1044865 (0.0010) [2023-12-27 12:53:22,275][20381] Updated weights for policy 1, policy_version 1044039 (0.0007) [2023-12-27 12:53:22,330][20381] Updated weights for policy 1, policy_version 1044049 (0.0008) [2023-12-27 12:53:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 534831104. Throughput: 0: 10245.7, 1: 10253.0. Samples: 534804468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 12:53:22,363][19170] Avg episode reward: [(0, '816.570'), (1, '872.870')] [2023-12-27 12:53:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001044872_267534336.pth... [2023-12-27 12:53:22,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001044056_267304960.pth... [2023-12-27 12:53:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001043656_267223040.pth [2023-12-27 12:53:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001042840_266993664.pth [2023-12-27 12:53:22,904][20347] Updated weights for policy 0, policy_version 1044875 (0.0007) [2023-12-27 12:53:22,955][20291] KL-divergence is very high: 144.7333 [2023-12-27 12:53:22,959][20347] Updated weights for policy 0, policy_version 1044885 (0.0005) [2023-12-27 12:53:23,023][20347] Updated weights for policy 0, policy_version 1044895 (0.0005) [2023-12-27 12:53:23,090][20381] Updated weights for policy 1, policy_version 1044059 (0.0008) [2023-12-27 12:53:23,158][20381] Updated weights for policy 1, policy_version 1044069 (0.0009) [2023-12-27 12:53:23,221][20381] Updated weights for policy 1, policy_version 1044079 (0.0008) [2023-12-27 12:53:23,599][20347] Updated weights for policy 0, policy_version 1044905 (0.0007) [2023-12-27 12:53:23,610][20291] KL-divergence is very high: 236.0535 [2023-12-27 12:53:23,655][20347] Updated weights for policy 0, policy_version 1044915 (0.0009) [2023-12-27 12:53:23,657][20291] KL-divergence is very high: 146.2059 [2023-12-27 12:53:23,708][20347] Updated weights for policy 0, policy_version 1044925 (0.0007) [2023-12-27 12:53:23,759][20347] Updated weights for policy 0, policy_version 1044935 (0.0008) [2023-12-27 12:53:23,887][20381] Updated weights for policy 1, policy_version 1044089 (0.0007) [2023-12-27 12:53:23,945][20381] Updated weights for policy 1, policy_version 1044099 (0.0005) [2023-12-27 12:53:24,004][20381] Updated weights for policy 1, policy_version 1044109 (0.0007) [2023-12-27 12:53:24,070][20381] Updated weights for policy 1, policy_version 1044119 (0.0008) [2023-12-27 12:53:24,466][20347] Updated weights for policy 0, policy_version 1044945 (0.0007) [2023-12-27 12:53:24,516][20347] Updated weights for policy 0, policy_version 1044955 (0.0007) [2023-12-27 12:53:24,568][20347] Updated weights for policy 0, policy_version 1044965 (0.0005) [2023-12-27 12:53:24,719][20381] Updated weights for policy 1, policy_version 1044129 (0.0008) [2023-12-27 12:53:24,775][20381] Updated weights for policy 1, policy_version 1044139 (0.0007) [2023-12-27 12:53:24,823][20381] Updated weights for policy 1, policy_version 1044149 (0.0007) [2023-12-27 12:53:25,163][20347] Updated weights for policy 0, policy_version 1044975 (0.0006) [2023-12-27 12:53:25,216][20347] Updated weights for policy 0, policy_version 1044985 (0.0009) [2023-12-27 12:53:25,262][20347] Updated weights for policy 0, policy_version 1044995 (0.0010) [2023-12-27 12:53:25,497][20381] Updated weights for policy 1, policy_version 1044159 (0.0008) [2023-12-27 12:53:25,548][20381] Updated weights for policy 1, policy_version 1044169 (0.0007) [2023-12-27 12:53:25,605][20381] Updated weights for policy 1, policy_version 1044179 (0.0008) [2023-12-27 12:53:25,884][20347] Updated weights for policy 0, policy_version 1045005 (0.0010) [2023-12-27 12:53:25,934][20291] KL-divergence is very high: 105.8421 [2023-12-27 12:53:25,941][20291] KL-divergence is very high: 125.9811 [2023-12-27 12:53:25,945][20347] Updated weights for policy 0, policy_version 1045015 (0.0007) [2023-12-27 12:53:25,946][20291] KL-divergence is very high: 144.0065 [2023-12-27 12:53:25,961][20291] KL-divergence is very high: 214.7384 [2023-12-27 12:53:25,978][20291] KL-divergence is very high: 194.5316 [2023-12-27 12:53:25,985][20291] KL-divergence is very high: 143.9920 [2023-12-27 12:53:25,990][20291] KL-divergence is very high: 111.6766 [2023-12-27 12:53:25,999][20347] Updated weights for policy 0, policy_version 1045025 (0.0007) [2023-12-27 12:53:26,243][20381] Updated weights for policy 1, policy_version 1044189 (0.0007) [2023-12-27 12:53:26,299][20381] Updated weights for policy 1, policy_version 1044199 (0.0005) [2023-12-27 12:53:26,351][20381] Updated weights for policy 1, policy_version 1044209 (0.0009) [2023-12-27 12:53:26,589][20347] Updated weights for policy 0, policy_version 1045035 (0.0008) [2023-12-27 12:53:26,639][20347] Updated weights for policy 0, policy_version 1045045 (0.0005) [2023-12-27 12:53:26,687][20347] Updated weights for policy 0, policy_version 1045055 (0.0005) [2023-12-27 12:53:27,005][20381] Updated weights for policy 1, policy_version 1044219 (0.0009) [2023-12-27 12:53:27,055][20381] Updated weights for policy 1, policy_version 1044229 (0.0010) [2023-12-27 12:53:27,100][20381] Updated weights for policy 1, policy_version 1044239 (0.0010) [2023-12-27 12:53:27,222][20347] Updated weights for policy 0, policy_version 1045065 (0.0005) [2023-12-27 12:53:27,281][20347] Updated weights for policy 0, policy_version 1045075 (0.0010) [2023-12-27 12:53:27,340][20347] Updated weights for policy 0, policy_version 1045085 (0.0010) [2023-12-27 12:53:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 534937600. Throughput: 0: 10279.8, 1: 10241.2. Samples: 534926552. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:27,363][19170] Avg episode reward: [(0, '676.300'), (1, '863.030')] [2023-12-27 12:53:27,388][20347] Updated weights for policy 0, policy_version 1045095 (0.0010) [2023-12-27 12:53:27,750][20381] Updated weights for policy 1, policy_version 1044249 (0.0010) [2023-12-27 12:53:27,811][20381] Updated weights for policy 1, policy_version 1044259 (0.0005) [2023-12-27 12:53:27,854][20381] Updated weights for policy 1, policy_version 1044269 (0.0006) [2023-12-27 12:53:27,905][20381] Updated weights for policy 1, policy_version 1044279 (0.0007) [2023-12-27 12:53:28,079][20347] Updated weights for policy 0, policy_version 1045105 (0.0008) [2023-12-27 12:53:28,132][20347] Updated weights for policy 0, policy_version 1045115 (0.0006) [2023-12-27 12:53:28,189][20347] Updated weights for policy 0, policy_version 1045125 (0.0008) [2023-12-27 12:53:28,555][20381] Updated weights for policy 1, policy_version 1044289 (0.0008) [2023-12-27 12:53:28,617][20381] Updated weights for policy 1, policy_version 1044299 (0.0007) [2023-12-27 12:53:28,672][20381] Updated weights for policy 1, policy_version 1044309 (0.0007) [2023-12-27 12:53:28,795][20347] Updated weights for policy 0, policy_version 1045135 (0.0006) [2023-12-27 12:53:28,851][20347] Updated weights for policy 0, policy_version 1045145 (0.0005) [2023-12-27 12:53:28,905][20347] Updated weights for policy 0, policy_version 1045155 (0.0005) [2023-12-27 12:53:29,376][20381] Updated weights for policy 1, policy_version 1044319 (0.0008) [2023-12-27 12:53:29,434][20381] Updated weights for policy 1, policy_version 1044329 (0.0010) [2023-12-27 12:53:29,486][20381] Updated weights for policy 1, policy_version 1044339 (0.0011) [2023-12-27 12:53:29,557][20347] Updated weights for policy 0, policy_version 1045165 (0.0007) [2023-12-27 12:53:29,607][20347] Updated weights for policy 0, policy_version 1045175 (0.0007) [2023-12-27 12:53:29,659][20347] Updated weights for policy 0, policy_version 1045185 (0.0007) [2023-12-27 12:53:30,206][20381] Updated weights for policy 1, policy_version 1044349 (0.0010) [2023-12-27 12:53:30,268][20381] Updated weights for policy 1, policy_version 1044359 (0.0011) [2023-12-27 12:53:30,328][20381] Updated weights for policy 1, policy_version 1044369 (0.0006) [2023-12-27 12:53:30,340][20347] Updated weights for policy 0, policy_version 1045195 (0.0007) [2023-12-27 12:53:30,407][20347] Updated weights for policy 0, policy_version 1045205 (0.0009) [2023-12-27 12:53:30,465][20347] Updated weights for policy 0, policy_version 1045215 (0.0009) [2023-12-27 12:53:30,989][20381] Updated weights for policy 1, policy_version 1044379 (0.0010) [2023-12-27 12:53:31,052][20347] Updated weights for policy 0, policy_version 1045225 (0.0009) [2023-12-27 12:53:31,055][20381] Updated weights for policy 1, policy_version 1044389 (0.0008) [2023-12-27 12:53:31,114][20347] Updated weights for policy 0, policy_version 1045235 (0.0006) [2023-12-27 12:53:31,115][20381] Updated weights for policy 1, policy_version 1044399 (0.0007) [2023-12-27 12:53:31,176][20347] Updated weights for policy 0, policy_version 1045245 (0.0009) [2023-12-27 12:53:31,235][20347] Updated weights for policy 0, policy_version 1045255 (0.0008) [2023-12-27 12:53:31,873][20381] Updated weights for policy 1, policy_version 1044410 (0.0009) [2023-12-27 12:53:31,881][20347] Updated weights for policy 0, policy_version 1045265 (0.0010) [2023-12-27 12:53:31,935][20381] Updated weights for policy 1, policy_version 1044420 (0.0005) [2023-12-27 12:53:31,942][20347] Updated weights for policy 0, policy_version 1045275 (0.0010) [2023-12-27 12:53:31,992][20381] Updated weights for policy 1, policy_version 1044430 (0.0005) [2023-12-27 12:53:31,997][20347] Updated weights for policy 0, policy_version 1045285 (0.0010) [2023-12-27 12:53:32,048][20381] Updated weights for policy 1, policy_version 1044440 (0.0007) [2023-12-27 12:53:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 535044096. Throughput: 0: 10352.8, 1: 10194.7. Samples: 535053352. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:32,363][19170] Avg episode reward: [(0, '640.580'), (1, '872.640')] [2023-12-27 12:53:32,661][20347] Updated weights for policy 0, policy_version 1045295 (0.0009) [2023-12-27 12:53:32,690][20381] Updated weights for policy 1, policy_version 1044450 (0.0006) [2023-12-27 12:53:32,718][20347] Updated weights for policy 0, policy_version 1045305 (0.0009) [2023-12-27 12:53:32,746][20381] Updated weights for policy 1, policy_version 1044460 (0.0006) [2023-12-27 12:53:32,771][20347] Updated weights for policy 0, policy_version 1045315 (0.0006) [2023-12-27 12:53:32,803][20381] Updated weights for policy 1, policy_version 1044470 (0.0006) [2023-12-27 12:53:33,411][20347] Updated weights for policy 0, policy_version 1045325 (0.0008) [2023-12-27 12:53:33,443][20381] Updated weights for policy 1, policy_version 1044480 (0.0007) [2023-12-27 12:53:33,465][20347] Updated weights for policy 0, policy_version 1045335 (0.0010) [2023-12-27 12:53:33,489][20381] Updated weights for policy 1, policy_version 1044490 (0.0006) [2023-12-27 12:53:33,520][20347] Updated weights for policy 0, policy_version 1045345 (0.0010) [2023-12-27 12:53:33,537][20381] Updated weights for policy 1, policy_version 1044500 (0.0007) [2023-12-27 12:53:34,158][20347] Updated weights for policy 0, policy_version 1045355 (0.0007) [2023-12-27 12:53:34,218][20347] Updated weights for policy 0, policy_version 1045365 (0.0007) [2023-12-27 12:53:34,259][20381] Updated weights for policy 1, policy_version 1044510 (0.0006) [2023-12-27 12:53:34,280][20347] Updated weights for policy 0, policy_version 1045375 (0.0007) [2023-12-27 12:53:34,317][20381] Updated weights for policy 1, policy_version 1044520 (0.0006) [2023-12-27 12:53:34,369][20381] Updated weights for policy 1, policy_version 1044530 (0.0007) [2023-12-27 12:53:34,959][20347] Updated weights for policy 0, policy_version 1045385 (0.0007) [2023-12-27 12:53:35,014][20347] Updated weights for policy 0, policy_version 1045395 (0.0010) [2023-12-27 12:53:35,064][20347] Updated weights for policy 0, policy_version 1045405 (0.0008) [2023-12-27 12:53:35,103][20381] Updated weights for policy 1, policy_version 1044540 (0.0007) [2023-12-27 12:53:35,109][20347] Updated weights for policy 0, policy_version 1045415 (0.0010) [2023-12-27 12:53:35,160][20381] Updated weights for policy 1, policy_version 1044550 (0.0007) [2023-12-27 12:53:35,217][20381] Updated weights for policy 1, policy_version 1044560 (0.0009) [2023-12-27 12:53:35,715][20347] Updated weights for policy 0, policy_version 1045425 (0.0008) [2023-12-27 12:53:35,767][20347] Updated weights for policy 0, policy_version 1045435 (0.0007) [2023-12-27 12:53:35,817][20347] Updated weights for policy 0, policy_version 1045445 (0.0006) [2023-12-27 12:53:35,950][20381] Updated weights for policy 1, policy_version 1044570 (0.0008) [2023-12-27 12:53:35,998][20381] Updated weights for policy 1, policy_version 1044580 (0.0007) [2023-12-27 12:53:36,043][20381] Updated weights for policy 1, policy_version 1044590 (0.0008) [2023-12-27 12:53:36,101][20381] Updated weights for policy 1, policy_version 1044600 (0.0006) [2023-12-27 12:53:36,539][20347] Updated weights for policy 0, policy_version 1045455 (0.0007) [2023-12-27 12:53:36,602][20347] Updated weights for policy 0, policy_version 1045465 (0.0008) [2023-12-27 12:53:36,667][20347] Updated weights for policy 0, policy_version 1045475 (0.0007) [2023-12-27 12:53:36,700][20381] Updated weights for policy 1, policy_version 1044610 (0.0008) [2023-12-27 12:53:36,757][20381] Updated weights for policy 1, policy_version 1044620 (0.0008) [2023-12-27 12:53:36,819][20381] Updated weights for policy 1, policy_version 1044630 (0.0008) [2023-12-27 12:53:37,257][20347] Updated weights for policy 0, policy_version 1045485 (0.0008) [2023-12-27 12:53:37,314][20347] Updated weights for policy 0, policy_version 1045495 (0.0006) [2023-12-27 12:53:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 535142400. Throughput: 0: 10409.3, 1: 10161.0. Samples: 535115176. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:37,363][19170] Avg episode reward: [(0, '661.810'), (1, '774.660')] [2023-12-27 12:53:37,377][20347] Updated weights for policy 0, policy_version 1045505 (0.0006) [2023-12-27 12:53:37,411][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001045512_267698176.pth... [2023-12-27 12:53:37,414][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001044264_267378688.pth [2023-12-27 12:53:37,417][20381] Updated weights for policy 1, policy_version 1044640 (0.0006) [2023-12-27 12:53:37,480][20381] Updated weights for policy 1, policy_version 1044650 (0.0006) [2023-12-27 12:53:37,548][20381] Updated weights for policy 1, policy_version 1044660 (0.0005) [2023-12-27 12:53:37,574][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001044664_267460608.pth... [2023-12-27 12:53:37,578][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001043448_267149312.pth [2023-12-27 12:53:37,992][20347] Updated weights for policy 0, policy_version 1045515 (0.0007) [2023-12-27 12:53:38,051][20347] Updated weights for policy 0, policy_version 1045525 (0.0007) [2023-12-27 12:53:38,056][20381] Updated weights for policy 1, policy_version 1044670 (0.0006) [2023-12-27 12:53:38,100][20347] Updated weights for policy 0, policy_version 1045535 (0.0006) [2023-12-27 12:53:38,114][20381] Updated weights for policy 1, policy_version 1044680 (0.0007) [2023-12-27 12:53:38,167][20381] Updated weights for policy 1, policy_version 1044690 (0.0007) [2023-12-27 12:53:38,759][20347] Updated weights for policy 0, policy_version 1045545 (0.0010) [2023-12-27 12:53:38,815][20347] Updated weights for policy 0, policy_version 1045555 (0.0010) [2023-12-27 12:53:38,838][20381] Updated weights for policy 1, policy_version 1044700 (0.0007) [2023-12-27 12:53:38,883][20347] Updated weights for policy 0, policy_version 1045565 (0.0007) [2023-12-27 12:53:38,895][20381] Updated weights for policy 1, policy_version 1044710 (0.0007) [2023-12-27 12:53:38,945][20347] Updated weights for policy 0, policy_version 1045575 (0.0009) [2023-12-27 12:53:38,948][20381] Updated weights for policy 1, policy_version 1044720 (0.0005) [2023-12-27 12:53:39,641][20347] Updated weights for policy 0, policy_version 1045585 (0.0008) [2023-12-27 12:53:39,667][20381] Updated weights for policy 1, policy_version 1044730 (0.0007) [2023-12-27 12:53:39,697][20347] Updated weights for policy 0, policy_version 1045595 (0.0008) [2023-12-27 12:53:39,732][20381] Updated weights for policy 1, policy_version 1044740 (0.0009) [2023-12-27 12:53:39,753][20347] Updated weights for policy 0, policy_version 1045605 (0.0008) [2023-12-27 12:53:39,788][20381] Updated weights for policy 1, policy_version 1044750 (0.0007) [2023-12-27 12:53:39,853][20381] Updated weights for policy 1, policy_version 1044760 (0.0008) [2023-12-27 12:53:40,486][20381] Updated weights for policy 1, policy_version 1044770 (0.0008) [2023-12-27 12:53:40,538][20381] Updated weights for policy 1, policy_version 1044780 (0.0007) [2023-12-27 12:53:40,548][20347] Updated weights for policy 0, policy_version 1045615 (0.0006) [2023-12-27 12:53:40,593][20381] Updated weights for policy 1, policy_version 1044790 (0.0008) [2023-12-27 12:53:40,603][20347] Updated weights for policy 0, policy_version 1045625 (0.0006) [2023-12-27 12:53:40,655][20347] Updated weights for policy 0, policy_version 1045635 (0.0008) [2023-12-27 12:53:41,381][20381] Updated weights for policy 1, policy_version 1044800 (0.0008) [2023-12-27 12:53:41,403][20347] Updated weights for policy 0, policy_version 1045645 (0.0008) [2023-12-27 12:53:41,448][20381] Updated weights for policy 1, policy_version 1044810 (0.0009) [2023-12-27 12:53:41,459][20347] Updated weights for policy 0, policy_version 1045655 (0.0007) [2023-12-27 12:53:41,509][20381] Updated weights for policy 1, policy_version 1044820 (0.0009) [2023-12-27 12:53:41,513][20347] Updated weights for policy 0, policy_version 1045665 (0.0008) [2023-12-27 12:53:42,217][20347] Updated weights for policy 0, policy_version 1045675 (0.0010) [2023-12-27 12:53:42,280][20347] Updated weights for policy 0, policy_version 1045685 (0.0008) [2023-12-27 12:53:42,329][20381] Updated weights for policy 1, policy_version 1044830 (0.0008) [2023-12-27 12:53:42,344][20347] Updated weights for policy 0, policy_version 1045695 (0.0007) [2023-12-27 12:53:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 535240704. Throughput: 0: 10422.8, 1: 10225.2. Samples: 535240640. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:42,363][19170] Avg episode reward: [(0, '616.120'), (1, '794.200')] [2023-12-27 12:53:42,400][20381] Updated weights for policy 1, policy_version 1044840 (0.0007) [2023-12-27 12:53:42,456][20381] Updated weights for policy 1, policy_version 1044850 (0.0007) [2023-12-27 12:53:43,065][20347] Updated weights for policy 0, policy_version 1045705 (0.0008) [2023-12-27 12:53:43,116][20381] Updated weights for policy 1, policy_version 1044860 (0.0007) [2023-12-27 12:53:43,117][20347] Updated weights for policy 0, policy_version 1045715 (0.0010) [2023-12-27 12:53:43,171][20381] Updated weights for policy 1, policy_version 1044870 (0.0008) [2023-12-27 12:53:43,175][20347] Updated weights for policy 0, policy_version 1045725 (0.0010) [2023-12-27 12:53:43,224][20381] Updated weights for policy 1, policy_version 1044880 (0.0007) [2023-12-27 12:53:43,229][20347] Updated weights for policy 0, policy_version 1045735 (0.0010) [2023-12-27 12:53:43,870][20381] Updated weights for policy 1, policy_version 1044890 (0.0007) [2023-12-27 12:53:43,909][20347] Updated weights for policy 0, policy_version 1045745 (0.0006) [2023-12-27 12:53:43,923][20381] Updated weights for policy 1, policy_version 1044900 (0.0007) [2023-12-27 12:53:43,963][20347] Updated weights for policy 0, policy_version 1045755 (0.0005) [2023-12-27 12:53:43,981][20381] Updated weights for policy 1, policy_version 1044910 (0.0006) [2023-12-27 12:53:44,010][20347] Updated weights for policy 0, policy_version 1045765 (0.0005) [2023-12-27 12:53:44,032][20381] Updated weights for policy 1, policy_version 1044920 (0.0007) [2023-12-27 12:53:44,541][20347] Updated weights for policy 0, policy_version 1045775 (0.0009) [2023-12-27 12:53:44,593][20347] Updated weights for policy 0, policy_version 1045785 (0.0010) [2023-12-27 12:53:44,612][20381] Updated weights for policy 1, policy_version 1044930 (0.0005) [2023-12-27 12:53:44,644][20347] Updated weights for policy 0, policy_version 1045795 (0.0010) [2023-12-27 12:53:44,666][20381] Updated weights for policy 1, policy_version 1044940 (0.0005) [2023-12-27 12:53:44,724][20381] Updated weights for policy 1, policy_version 1044950 (0.0006) [2023-12-27 12:53:45,363][20347] Updated weights for policy 0, policy_version 1045805 (0.0009) [2023-12-27 12:53:45,396][20381] Updated weights for policy 1, policy_version 1044960 (0.0008) [2023-12-27 12:53:45,419][20347] Updated weights for policy 0, policy_version 1045815 (0.0005) [2023-12-27 12:53:45,441][20381] Updated weights for policy 1, policy_version 1044970 (0.0007) [2023-12-27 12:53:45,478][20347] Updated weights for policy 0, policy_version 1045825 (0.0009) [2023-12-27 12:53:45,491][20381] Updated weights for policy 1, policy_version 1044980 (0.0005) [2023-12-27 12:53:46,101][20347] Updated weights for policy 0, policy_version 1045835 (0.0010) [2023-12-27 12:53:46,126][20381] Updated weights for policy 1, policy_version 1044990 (0.0006) [2023-12-27 12:53:46,160][20347] Updated weights for policy 0, policy_version 1045845 (0.0010) [2023-12-27 12:53:46,180][20381] Updated weights for policy 1, policy_version 1045000 (0.0006) [2023-12-27 12:53:46,217][20347] Updated weights for policy 0, policy_version 1045855 (0.0010) [2023-12-27 12:53:46,238][20381] Updated weights for policy 1, policy_version 1045010 (0.0006) [2023-12-27 12:53:46,784][20381] Updated weights for policy 1, policy_version 1045020 (0.0005) [2023-12-27 12:53:46,837][20381] Updated weights for policy 1, policy_version 1045030 (0.0005) [2023-12-27 12:53:46,870][20347] Updated weights for policy 0, policy_version 1045865 (0.0010) [2023-12-27 12:53:46,894][20381] Updated weights for policy 1, policy_version 1045040 (0.0005) [2023-12-27 12:53:46,921][20347] Updated weights for policy 0, policy_version 1045875 (0.0010) [2023-12-27 12:53:46,976][20347] Updated weights for policy 0, policy_version 1045885 (0.0010) [2023-12-27 12:53:47,031][20347] Updated weights for policy 0, policy_version 1045895 (0.0010) [2023-12-27 12:53:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 535355392. Throughput: 0: 10423.1, 1: 10315.1. Samples: 535365856. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:47,364][19170] Avg episode reward: [(0, '713.810'), (1, '794.210')] [2023-12-27 12:53:47,541][20381] Updated weights for policy 1, policy_version 1045050 (0.0008) [2023-12-27 12:53:47,595][20381] Updated weights for policy 1, policy_version 1045060 (0.0010) [2023-12-27 12:53:47,654][20381] Updated weights for policy 1, policy_version 1045070 (0.0010) [2023-12-27 12:53:47,708][20381] Updated weights for policy 1, policy_version 1045080 (0.0010) [2023-12-27 12:53:47,719][20347] Updated weights for policy 0, policy_version 1045905 (0.0010) [2023-12-27 12:53:47,771][20347] Updated weights for policy 0, policy_version 1045915 (0.0008) [2023-12-27 12:53:47,826][20347] Updated weights for policy 0, policy_version 1045925 (0.0005) [2023-12-27 12:53:48,386][20347] Updated weights for policy 0, policy_version 1045935 (0.0006) [2023-12-27 12:53:48,418][20381] Updated weights for policy 1, policy_version 1045090 (0.0006) [2023-12-27 12:53:48,445][20347] Updated weights for policy 0, policy_version 1045945 (0.0009) [2023-12-27 12:53:48,474][20381] Updated weights for policy 1, policy_version 1045100 (0.0007) [2023-12-27 12:53:48,512][20347] Updated weights for policy 0, policy_version 1045955 (0.0010) [2023-12-27 12:53:48,537][20381] Updated weights for policy 1, policy_version 1045110 (0.0007) [2023-12-27 12:53:49,132][20347] Updated weights for policy 0, policy_version 1045965 (0.0010) [2023-12-27 12:53:49,170][20381] Updated weights for policy 1, policy_version 1045120 (0.0007) [2023-12-27 12:53:49,187][20347] Updated weights for policy 0, policy_version 1045975 (0.0010) [2023-12-27 12:53:49,226][20381] Updated weights for policy 1, policy_version 1045130 (0.0006) [2023-12-27 12:53:49,250][20347] Updated weights for policy 0, policy_version 1045985 (0.0010) [2023-12-27 12:53:49,293][20381] Updated weights for policy 1, policy_version 1045140 (0.0007) [2023-12-27 12:53:49,965][20347] Updated weights for policy 0, policy_version 1045995 (0.0010) [2023-12-27 12:53:50,018][20347] Updated weights for policy 0, policy_version 1046005 (0.0010) [2023-12-27 12:53:50,025][20381] Updated weights for policy 1, policy_version 1045150 (0.0008) [2023-12-27 12:53:50,075][20347] Updated weights for policy 0, policy_version 1046015 (0.0010) [2023-12-27 12:53:50,078][20381] Updated weights for policy 1, policy_version 1045160 (0.0007) [2023-12-27 12:53:50,133][20381] Updated weights for policy 1, policy_version 1045170 (0.0006) [2023-12-27 12:53:50,735][20347] Updated weights for policy 0, policy_version 1046025 (0.0010) [2023-12-27 12:53:50,794][20347] Updated weights for policy 0, policy_version 1046035 (0.0006) [2023-12-27 12:53:50,838][20381] Updated weights for policy 1, policy_version 1045180 (0.0008) [2023-12-27 12:53:50,856][20347] Updated weights for policy 0, policy_version 1046045 (0.0005) [2023-12-27 12:53:50,899][20381] Updated weights for policy 1, policy_version 1045190 (0.0006) [2023-12-27 12:53:50,917][20347] Updated weights for policy 0, policy_version 1046055 (0.0008) [2023-12-27 12:53:50,959][20381] Updated weights for policy 1, policy_version 1045200 (0.0005) [2023-12-27 12:53:51,587][20347] Updated weights for policy 0, policy_version 1046065 (0.0010) [2023-12-27 12:53:51,652][20381] Updated weights for policy 1, policy_version 1045210 (0.0006) [2023-12-27 12:53:51,659][20347] Updated weights for policy 0, policy_version 1046075 (0.0014) [2023-12-27 12:53:51,708][20381] Updated weights for policy 1, policy_version 1045220 (0.0011) [2023-12-27 12:53:51,716][20347] Updated weights for policy 0, policy_version 1046085 (0.0007) [2023-12-27 12:53:51,776][20381] Updated weights for policy 1, policy_version 1045230 (0.0011) [2023-12-27 12:53:51,839][20381] Updated weights for policy 1, policy_version 1045240 (0.0011) [2023-12-27 12:53:52,306][20347] Updated weights for policy 0, policy_version 1046095 (0.0007) [2023-12-27 12:53:52,341][20291] KL-divergence is very high: 106.0252 [2023-12-27 12:53:52,358][20291] KL-divergence is very high: 106.1577 [2023-12-27 12:53:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 535453696. Throughput: 0: 10431.0, 1: 10305.4. Samples: 535427832. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:52,363][19170] Avg episode reward: [(0, '679.780'), (1, '794.330')] [2023-12-27 12:53:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001045240_267608064.pth... [2023-12-27 12:53:52,372][20347] Updated weights for policy 0, policy_version 1046105 (0.0006) [2023-12-27 12:53:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001044056_267304960.pth [2023-12-27 12:53:52,436][20347] Updated weights for policy 0, policy_version 1046115 (0.0006) [2023-12-27 12:53:52,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001046120_267853824.pth... [2023-12-27 12:53:52,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001044872_267534336.pth [2023-12-27 12:53:52,526][20381] Updated weights for policy 1, policy_version 1045250 (0.0011) [2023-12-27 12:53:52,585][20381] Updated weights for policy 1, policy_version 1045260 (0.0011) [2023-12-27 12:53:52,641][20381] Updated weights for policy 1, policy_version 1045270 (0.0011) [2023-12-27 12:53:53,125][20347] Updated weights for policy 0, policy_version 1046125 (0.0008) [2023-12-27 12:53:53,188][20347] Updated weights for policy 0, policy_version 1046135 (0.0007) [2023-12-27 12:53:53,235][20347] Updated weights for policy 0, policy_version 1046145 (0.0005) [2023-12-27 12:53:53,260][20381] Updated weights for policy 1, policy_version 1045280 (0.0011) [2023-12-27 12:53:53,322][20381] Updated weights for policy 1, policy_version 1045290 (0.0011) [2023-12-27 12:53:53,371][20381] Updated weights for policy 1, policy_version 1045300 (0.0010) [2023-12-27 12:53:53,860][20347] Updated weights for policy 0, policy_version 1046155 (0.0006) [2023-12-27 12:53:53,915][20347] Updated weights for policy 0, policy_version 1046165 (0.0008) [2023-12-27 12:53:53,961][20347] Updated weights for policy 0, policy_version 1046175 (0.0008) [2023-12-27 12:53:54,071][20381] Updated weights for policy 1, policy_version 1045310 (0.0008) [2023-12-27 12:53:54,124][20381] Updated weights for policy 1, policy_version 1045320 (0.0007) [2023-12-27 12:53:54,178][20381] Updated weights for policy 1, policy_version 1045330 (0.0008) [2023-12-27 12:53:54,637][20347] Updated weights for policy 0, policy_version 1046185 (0.0008) [2023-12-27 12:53:54,697][20347] Updated weights for policy 0, policy_version 1046195 (0.0007) [2023-12-27 12:53:54,754][20347] Updated weights for policy 0, policy_version 1046205 (0.0007) [2023-12-27 12:53:54,780][20381] Updated weights for policy 1, policy_version 1045340 (0.0006) [2023-12-27 12:53:54,813][20347] Updated weights for policy 0, policy_version 1046215 (0.0007) [2023-12-27 12:53:54,823][20381] Updated weights for policy 1, policy_version 1045350 (0.0007) [2023-12-27 12:53:54,880][20381] Updated weights for policy 1, policy_version 1045360 (0.0007) [2023-12-27 12:53:55,492][20347] Updated weights for policy 0, policy_version 1046225 (0.0010) [2023-12-27 12:53:55,539][20381] Updated weights for policy 1, policy_version 1045370 (0.0007) [2023-12-27 12:53:55,548][20347] Updated weights for policy 0, policy_version 1046235 (0.0010) [2023-12-27 12:53:55,597][20291] KL-divergence is very high: 135.1672 [2023-12-27 12:53:55,597][20381] Updated weights for policy 1, policy_version 1045380 (0.0005) [2023-12-27 12:53:55,602][20291] KL-divergence is very high: 191.9047 [2023-12-27 12:53:55,603][20347] Updated weights for policy 0, policy_version 1046245 (0.0010) [2023-12-27 12:53:55,612][20291] KL-divergence is very high: 109.8790 [2023-12-27 12:53:55,651][20381] Updated weights for policy 1, policy_version 1045390 (0.0005) [2023-12-27 12:53:55,695][20381] Updated weights for policy 1, policy_version 1045400 (0.0005) [2023-12-27 12:53:56,236][20381] Updated weights for policy 1, policy_version 1045410 (0.0008) [2023-12-27 12:53:56,269][20291] KL-divergence is very high: 107.0480 [2023-12-27 12:53:56,288][20347] Updated weights for policy 0, policy_version 1046255 (0.0009) [2023-12-27 12:53:56,298][20381] Updated weights for policy 1, policy_version 1045420 (0.0011) [2023-12-27 12:53:56,334][20347] Updated weights for policy 0, policy_version 1046265 (0.0005) [2023-12-27 12:53:56,353][20381] Updated weights for policy 1, policy_version 1045430 (0.0008) [2023-12-27 12:53:56,383][20347] Updated weights for policy 0, policy_version 1046275 (0.0009) [2023-12-27 12:53:56,882][20381] Updated weights for policy 1, policy_version 1045440 (0.0008) [2023-12-27 12:53:56,943][20381] Updated weights for policy 1, policy_version 1045450 (0.0011) [2023-12-27 12:53:57,002][20381] Updated weights for policy 1, policy_version 1045460 (0.0007) [2023-12-27 12:53:57,016][20347] Updated weights for policy 0, policy_version 1046285 (0.0010) [2023-12-27 12:53:57,061][20347] Updated weights for policy 0, policy_version 1046295 (0.0005) [2023-12-27 12:53:57,117][20347] Updated weights for policy 0, policy_version 1046305 (0.0005) [2023-12-27 12:53:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 535568384. Throughput: 0: 10459.1, 1: 10348.9. Samples: 535554488. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:53:57,363][19170] Avg episode reward: [(0, '732.400'), (1, '862.890')] [2023-12-27 12:53:57,572][20381] Updated weights for policy 1, policy_version 1045470 (0.0009) [2023-12-27 12:53:57,618][20381] Updated weights for policy 1, policy_version 1045480 (0.0010) [2023-12-27 12:53:57,698][20381] Updated weights for policy 1, policy_version 1045490 (0.0009) [2023-12-27 12:53:57,729][20347] Updated weights for policy 0, policy_version 1046315 (0.0007) [2023-12-27 12:53:57,788][20347] Updated weights for policy 0, policy_version 1046325 (0.0010) [2023-12-27 12:53:57,839][20347] Updated weights for policy 0, policy_version 1046335 (0.0010) [2023-12-27 12:53:58,263][20381] Updated weights for policy 1, policy_version 1045500 (0.0007) [2023-12-27 12:53:58,323][20381] Updated weights for policy 1, policy_version 1045510 (0.0008) [2023-12-27 12:53:58,392][20381] Updated weights for policy 1, policy_version 1045520 (0.0008) [2023-12-27 12:53:58,537][20347] Updated weights for policy 0, policy_version 1046345 (0.0010) [2023-12-27 12:53:58,605][20347] Updated weights for policy 0, policy_version 1046355 (0.0007) [2023-12-27 12:53:58,668][20347] Updated weights for policy 0, policy_version 1046365 (0.0008) [2023-12-27 12:53:58,739][20347] Updated weights for policy 0, policy_version 1046375 (0.0009) [2023-12-27 12:53:59,147][20381] Updated weights for policy 1, policy_version 1045530 (0.0008) [2023-12-27 12:53:59,204][20381] Updated weights for policy 1, policy_version 1045540 (0.0007) [2023-12-27 12:53:59,271][20381] Updated weights for policy 1, policy_version 1045550 (0.0008) [2023-12-27 12:53:59,339][20381] Updated weights for policy 1, policy_version 1045560 (0.0006) [2023-12-27 12:53:59,559][20347] Updated weights for policy 0, policy_version 1046385 (0.0007) [2023-12-27 12:53:59,619][20347] Updated weights for policy 0, policy_version 1046395 (0.0007) [2023-12-27 12:53:59,676][20347] Updated weights for policy 0, policy_version 1046405 (0.0007) [2023-12-27 12:54:00,028][20381] Updated weights for policy 1, policy_version 1045570 (0.0006) [2023-12-27 12:54:00,085][20381] Updated weights for policy 1, policy_version 1045580 (0.0007) [2023-12-27 12:54:00,136][20381] Updated weights for policy 1, policy_version 1045590 (0.0007) [2023-12-27 12:54:00,371][20347] Updated weights for policy 0, policy_version 1046415 (0.0007) [2023-12-27 12:54:00,417][20347] Updated weights for policy 0, policy_version 1046425 (0.0007) [2023-12-27 12:54:00,463][20347] Updated weights for policy 0, policy_version 1046435 (0.0008) [2023-12-27 12:54:00,859][20381] Updated weights for policy 1, policy_version 1045600 (0.0007) [2023-12-27 12:54:00,909][20381] Updated weights for policy 1, policy_version 1045610 (0.0006) [2023-12-27 12:54:00,968][20381] Updated weights for policy 1, policy_version 1045620 (0.0007) [2023-12-27 12:54:01,246][20347] Updated weights for policy 0, policy_version 1046445 (0.0008) [2023-12-27 12:54:01,312][20347] Updated weights for policy 0, policy_version 1046455 (0.0009) [2023-12-27 12:54:01,380][20347] Updated weights for policy 0, policy_version 1046465 (0.0009) [2023-12-27 12:54:01,632][20381] Updated weights for policy 1, policy_version 1045630 (0.0007) [2023-12-27 12:54:01,695][20381] Updated weights for policy 1, policy_version 1045640 (0.0008) [2023-12-27 12:54:01,764][20381] Updated weights for policy 1, policy_version 1045650 (0.0008) [2023-12-27 12:54:02,118][20347] Updated weights for policy 0, policy_version 1046475 (0.0007) [2023-12-27 12:54:02,174][20347] Updated weights for policy 0, policy_version 1046485 (0.0005) [2023-12-27 12:54:02,226][20347] Updated weights for policy 0, policy_version 1046495 (0.0007) [2023-12-27 12:54:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 535666688. Throughput: 0: 10391.3, 1: 10365.0. Samples: 535677200. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:02,363][19170] Avg episode reward: [(0, '766.920'), (1, '755.220')] [2023-12-27 12:54:02,409][20381] Updated weights for policy 1, policy_version 1045660 (0.0009) [2023-12-27 12:54:02,465][20381] Updated weights for policy 1, policy_version 1045670 (0.0008) [2023-12-27 12:54:02,520][20381] Updated weights for policy 1, policy_version 1045680 (0.0007) [2023-12-27 12:54:02,884][20347] Updated weights for policy 0, policy_version 1046505 (0.0008) [2023-12-27 12:54:02,935][20347] Updated weights for policy 0, policy_version 1046515 (0.0010) [2023-12-27 12:54:02,992][20347] Updated weights for policy 0, policy_version 1046525 (0.0010) [2023-12-27 12:54:03,037][20347] Updated weights for policy 0, policy_version 1046535 (0.0010) [2023-12-27 12:54:03,157][20381] Updated weights for policy 1, policy_version 1045690 (0.0007) [2023-12-27 12:54:03,208][20381] Updated weights for policy 1, policy_version 1045700 (0.0005) [2023-12-27 12:54:03,255][20381] Updated weights for policy 1, policy_version 1045710 (0.0005) [2023-12-27 12:54:03,301][20381] Updated weights for policy 1, policy_version 1045720 (0.0005) [2023-12-27 12:54:03,630][20347] Updated weights for policy 0, policy_version 1046545 (0.0010) [2023-12-27 12:54:03,685][20347] Updated weights for policy 0, policy_version 1046555 (0.0010) [2023-12-27 12:54:03,738][20347] Updated weights for policy 0, policy_version 1046565 (0.0010) [2023-12-27 12:54:03,839][20381] Updated weights for policy 1, policy_version 1045730 (0.0010) [2023-12-27 12:54:03,897][20381] Updated weights for policy 1, policy_version 1045740 (0.0010) [2023-12-27 12:54:03,943][20381] Updated weights for policy 1, policy_version 1045750 (0.0010) [2023-12-27 12:54:04,415][20347] Updated weights for policy 0, policy_version 1046575 (0.0008) [2023-12-27 12:54:04,478][20347] Updated weights for policy 0, policy_version 1046585 (0.0005) [2023-12-27 12:54:04,529][20347] Updated weights for policy 0, policy_version 1046595 (0.0005) [2023-12-27 12:54:04,657][20381] Updated weights for policy 1, policy_version 1045760 (0.0006) [2023-12-27 12:54:04,708][20381] Updated weights for policy 1, policy_version 1045770 (0.0005) [2023-12-27 12:54:04,758][20381] Updated weights for policy 1, policy_version 1045780 (0.0005) [2023-12-27 12:54:05,127][20347] Updated weights for policy 0, policy_version 1046605 (0.0006) [2023-12-27 12:54:05,183][20347] Updated weights for policy 0, policy_version 1046615 (0.0005) [2023-12-27 12:54:05,240][20347] Updated weights for policy 0, policy_version 1046625 (0.0005) [2023-12-27 12:54:05,393][20381] Updated weights for policy 1, policy_version 1045790 (0.0009) [2023-12-27 12:54:05,451][20381] Updated weights for policy 1, policy_version 1045800 (0.0010) [2023-12-27 12:54:05,503][20381] Updated weights for policy 1, policy_version 1045810 (0.0010) [2023-12-27 12:54:05,807][20347] Updated weights for policy 0, policy_version 1046635 (0.0005) [2023-12-27 12:54:05,860][20347] Updated weights for policy 0, policy_version 1046645 (0.0005) [2023-12-27 12:54:05,909][20347] Updated weights for policy 0, policy_version 1046655 (0.0006) [2023-12-27 12:54:06,194][20381] Updated weights for policy 1, policy_version 1045820 (0.0009) [2023-12-27 12:54:06,253][20381] Updated weights for policy 1, policy_version 1045830 (0.0008) [2023-12-27 12:54:06,305][20381] Updated weights for policy 1, policy_version 1045840 (0.0007) [2023-12-27 12:54:06,560][20347] Updated weights for policy 0, policy_version 1046665 (0.0010) [2023-12-27 12:54:06,623][20347] Updated weights for policy 0, policy_version 1046675 (0.0008) [2023-12-27 12:54:06,678][20347] Updated weights for policy 0, policy_version 1046685 (0.0007) [2023-12-27 12:54:06,737][20347] Updated weights for policy 0, policy_version 1046695 (0.0007) [2023-12-27 12:54:07,030][20381] Updated weights for policy 1, policy_version 1045850 (0.0007) [2023-12-27 12:54:07,082][20381] Updated weights for policy 1, policy_version 1045860 (0.0007) [2023-12-27 12:54:07,137][20381] Updated weights for policy 1, policy_version 1045870 (0.0008) [2023-12-27 12:54:07,195][20381] Updated weights for policy 1, policy_version 1045880 (0.0008) [2023-12-27 12:54:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 535773184. Throughput: 0: 10458.8, 1: 10400.5. Samples: 535743132. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:07,363][19170] Avg episode reward: [(0, '794.330'), (1, '833.640')] [2023-12-27 12:54:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001045880_267771904.pth... [2023-12-27 12:54:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001044664_267460608.pth [2023-12-27 12:54:07,404][20347] Updated weights for policy 0, policy_version 1046705 (0.0007) [2023-12-27 12:54:07,452][20347] Updated weights for policy 0, policy_version 1046715 (0.0005) [2023-12-27 12:54:07,512][20347] Updated weights for policy 0, policy_version 1046725 (0.0006) [2023-12-27 12:54:07,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001046728_268009472.pth... [2023-12-27 12:54:07,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001045512_267698176.pth [2023-12-27 12:54:07,930][20381] Updated weights for policy 1, policy_version 1045890 (0.0008) [2023-12-27 12:54:07,989][20381] Updated weights for policy 1, policy_version 1045900 (0.0007) [2023-12-27 12:54:08,050][20381] Updated weights for policy 1, policy_version 1045910 (0.0007) [2023-12-27 12:54:08,151][20347] Updated weights for policy 0, policy_version 1046735 (0.0008) [2023-12-27 12:54:08,215][20347] Updated weights for policy 0, policy_version 1046745 (0.0006) [2023-12-27 12:54:08,281][20347] Updated weights for policy 0, policy_version 1046755 (0.0006) [2023-12-27 12:54:08,706][20381] Updated weights for policy 1, policy_version 1045920 (0.0006) [2023-12-27 12:54:08,776][20381] Updated weights for policy 1, policy_version 1045930 (0.0006) [2023-12-27 12:54:08,843][20381] Updated weights for policy 1, policy_version 1045940 (0.0006) [2023-12-27 12:54:08,925][20347] Updated weights for policy 0, policy_version 1046765 (0.0006) [2023-12-27 12:54:08,977][20347] Updated weights for policy 0, policy_version 1046775 (0.0006) [2023-12-27 12:54:09,024][20347] Updated weights for policy 0, policy_version 1046785 (0.0007) [2023-12-27 12:54:09,448][20381] Updated weights for policy 1, policy_version 1045950 (0.0008) [2023-12-27 12:54:09,510][20381] Updated weights for policy 1, policy_version 1045960 (0.0009) [2023-12-27 12:54:09,576][20381] Updated weights for policy 1, policy_version 1045970 (0.0009) [2023-12-27 12:54:09,827][20347] Updated weights for policy 0, policy_version 1046795 (0.0008) [2023-12-27 12:54:09,888][20347] Updated weights for policy 0, policy_version 1046805 (0.0009) [2023-12-27 12:54:09,952][20347] Updated weights for policy 0, policy_version 1046815 (0.0008) [2023-12-27 12:54:10,369][20381] Updated weights for policy 1, policy_version 1045980 (0.0008) [2023-12-27 12:54:10,424][20381] Updated weights for policy 1, policy_version 1045990 (0.0007) [2023-12-27 12:54:10,474][20381] Updated weights for policy 1, policy_version 1046000 (0.0007) [2023-12-27 12:54:10,657][20347] Updated weights for policy 0, policy_version 1046825 (0.0009) [2023-12-27 12:54:10,708][20347] Updated weights for policy 0, policy_version 1046835 (0.0010) [2023-12-27 12:54:10,767][20347] Updated weights for policy 0, policy_version 1046845 (0.0010) [2023-12-27 12:54:10,826][20347] Updated weights for policy 0, policy_version 1046855 (0.0010) [2023-12-27 12:54:11,154][20381] Updated weights for policy 1, policy_version 1046010 (0.0008) [2023-12-27 12:54:11,219][20381] Updated weights for policy 1, policy_version 1046020 (0.0007) [2023-12-27 12:54:11,285][20381] Updated weights for policy 1, policy_version 1046030 (0.0008) [2023-12-27 12:54:11,343][20381] Updated weights for policy 1, policy_version 1046040 (0.0007) [2023-12-27 12:54:11,535][20347] Updated weights for policy 0, policy_version 1046865 (0.0007) [2023-12-27 12:54:11,606][20347] Updated weights for policy 0, policy_version 1046875 (0.0006) [2023-12-27 12:54:11,683][20347] Updated weights for policy 0, policy_version 1046885 (0.0008) [2023-12-27 12:54:12,075][20381] Updated weights for policy 1, policy_version 1046050 (0.0006) [2023-12-27 12:54:12,134][20381] Updated weights for policy 1, policy_version 1046060 (0.0008) [2023-12-27 12:54:12,200][20381] Updated weights for policy 1, policy_version 1046070 (0.0008) [2023-12-27 12:54:12,264][20347] Updated weights for policy 0, policy_version 1046895 (0.0007) [2023-12-27 12:54:12,333][20347] Updated weights for policy 0, policy_version 1046905 (0.0007) [2023-12-27 12:54:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 535871488. Throughput: 0: 10448.0, 1: 10394.7. Samples: 535864472. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:12,363][19170] Avg episode reward: [(0, '872.580'), (1, '863.080')] [2023-12-27 12:54:12,399][20347] Updated weights for policy 0, policy_version 1046915 (0.0007) [2023-12-27 12:54:12,820][20381] Updated weights for policy 1, policy_version 1046080 (0.0009) [2023-12-27 12:54:12,874][20381] Updated weights for policy 1, policy_version 1046090 (0.0007) [2023-12-27 12:54:12,930][20381] Updated weights for policy 1, policy_version 1046100 (0.0006) [2023-12-27 12:54:13,079][20347] Updated weights for policy 0, policy_version 1046925 (0.0007) [2023-12-27 12:54:13,134][20347] Updated weights for policy 0, policy_version 1046935 (0.0008) [2023-12-27 12:54:13,189][20347] Updated weights for policy 0, policy_version 1046945 (0.0007) [2023-12-27 12:54:13,570][20381] Updated weights for policy 1, policy_version 1046110 (0.0008) [2023-12-27 12:54:13,621][20381] Updated weights for policy 1, policy_version 1046120 (0.0011) [2023-12-27 12:54:13,676][20381] Updated weights for policy 1, policy_version 1046130 (0.0011) [2023-12-27 12:54:13,769][20347] Updated weights for policy 0, policy_version 1046955 (0.0007) [2023-12-27 12:54:13,820][20347] Updated weights for policy 0, policy_version 1046965 (0.0010) [2023-12-27 12:54:13,868][20347] Updated weights for policy 0, policy_version 1046975 (0.0010) [2023-12-27 12:54:14,352][20381] Updated weights for policy 1, policy_version 1046140 (0.0010) [2023-12-27 12:54:14,397][20381] Updated weights for policy 1, policy_version 1046150 (0.0008) [2023-12-27 12:54:14,446][20381] Updated weights for policy 1, policy_version 1046160 (0.0009) [2023-12-27 12:54:14,516][20347] Updated weights for policy 0, policy_version 1046985 (0.0010) [2023-12-27 12:54:14,565][20347] Updated weights for policy 0, policy_version 1046995 (0.0007) [2023-12-27 12:54:14,619][20347] Updated weights for policy 0, policy_version 1047005 (0.0006) [2023-12-27 12:54:14,680][20347] Updated weights for policy 0, policy_version 1047015 (0.0008) [2023-12-27 12:54:15,175][20381] Updated weights for policy 1, policy_version 1046170 (0.0008) [2023-12-27 12:54:15,241][20381] Updated weights for policy 1, policy_version 1046180 (0.0008) [2023-12-27 12:54:15,296][20381] Updated weights for policy 1, policy_version 1046190 (0.0008) [2023-12-27 12:54:15,328][20347] Updated weights for policy 0, policy_version 1047025 (0.0007) [2023-12-27 12:54:15,353][20381] Updated weights for policy 1, policy_version 1046200 (0.0006) [2023-12-27 12:54:15,385][20347] Updated weights for policy 0, policy_version 1047035 (0.0008) [2023-12-27 12:54:15,443][20347] Updated weights for policy 0, policy_version 1047045 (0.0007) [2023-12-27 12:54:15,998][20381] Updated weights for policy 1, policy_version 1046210 (0.0006) [2023-12-27 12:54:16,053][20381] Updated weights for policy 1, policy_version 1046220 (0.0006) [2023-12-27 12:54:16,069][20347] Updated weights for policy 0, policy_version 1047055 (0.0009) [2023-12-27 12:54:16,105][20381] Updated weights for policy 1, policy_version 1046230 (0.0007) [2023-12-27 12:54:16,132][20347] Updated weights for policy 0, policy_version 1047065 (0.0007) [2023-12-27 12:54:16,193][20347] Updated weights for policy 0, policy_version 1047075 (0.0007) [2023-12-27 12:54:16,741][20381] Updated weights for policy 1, policy_version 1046240 (0.0007) [2023-12-27 12:54:16,790][20381] Updated weights for policy 1, policy_version 1046250 (0.0008) [2023-12-27 12:54:16,843][20381] Updated weights for policy 1, policy_version 1046260 (0.0008) [2023-12-27 12:54:16,885][20347] Updated weights for policy 0, policy_version 1047085 (0.0006) [2023-12-27 12:54:16,932][20347] Updated weights for policy 0, policy_version 1047095 (0.0006) [2023-12-27 12:54:16,978][20347] Updated weights for policy 0, policy_version 1047105 (0.0005) [2023-12-27 12:54:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 535977984. Throughput: 0: 10376.7, 1: 10422.4. Samples: 535989312. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:17,363][19170] Avg episode reward: [(0, '921.740'), (1, '882.630')] [2023-12-27 12:54:17,515][20381] Updated weights for policy 1, policy_version 1046270 (0.0008) [2023-12-27 12:54:17,571][20381] Updated weights for policy 1, policy_version 1046280 (0.0007) [2023-12-27 12:54:17,624][20381] Updated weights for policy 1, policy_version 1046290 (0.0006) [2023-12-27 12:54:17,651][20347] Updated weights for policy 0, policy_version 1047115 (0.0007) [2023-12-27 12:54:17,706][20347] Updated weights for policy 0, policy_version 1047125 (0.0008) [2023-12-27 12:54:17,763][20347] Updated weights for policy 0, policy_version 1047135 (0.0006) [2023-12-27 12:54:18,266][20381] Updated weights for policy 1, policy_version 1046300 (0.0007) [2023-12-27 12:54:18,319][20381] Updated weights for policy 1, policy_version 1046310 (0.0008) [2023-12-27 12:54:18,381][20381] Updated weights for policy 1, policy_version 1046320 (0.0006) [2023-12-27 12:54:18,411][20347] Updated weights for policy 0, policy_version 1047145 (0.0005) [2023-12-27 12:54:18,468][20347] Updated weights for policy 0, policy_version 1047155 (0.0009) [2023-12-27 12:54:18,518][20347] Updated weights for policy 0, policy_version 1047165 (0.0008) [2023-12-27 12:54:18,574][20347] Updated weights for policy 0, policy_version 1047175 (0.0008) [2023-12-27 12:54:19,042][20381] Updated weights for policy 1, policy_version 1046330 (0.0006) [2023-12-27 12:54:19,102][20381] Updated weights for policy 1, policy_version 1046340 (0.0005) [2023-12-27 12:54:19,160][20381] Updated weights for policy 1, policy_version 1046350 (0.0007) [2023-12-27 12:54:19,221][20381] Updated weights for policy 1, policy_version 1046360 (0.0008) [2023-12-27 12:54:19,309][20347] Updated weights for policy 0, policy_version 1047185 (0.0008) [2023-12-27 12:54:19,373][20347] Updated weights for policy 0, policy_version 1047195 (0.0008) [2023-12-27 12:54:19,432][20347] Updated weights for policy 0, policy_version 1047205 (0.0007) [2023-12-27 12:54:19,965][20381] Updated weights for policy 1, policy_version 1046370 (0.0008) [2023-12-27 12:54:20,030][20381] Updated weights for policy 1, policy_version 1046380 (0.0008) [2023-12-27 12:54:20,090][20347] Updated weights for policy 0, policy_version 1047215 (0.0008) [2023-12-27 12:54:20,091][20381] Updated weights for policy 1, policy_version 1046390 (0.0006) [2023-12-27 12:54:20,152][20347] Updated weights for policy 0, policy_version 1047225 (0.0007) [2023-12-27 12:54:20,245][20347] Updated weights for policy 0, policy_version 1047235 (0.0007) [2023-12-27 12:54:20,771][20381] Updated weights for policy 1, policy_version 1046400 (0.0008) [2023-12-27 12:54:20,836][20381] Updated weights for policy 1, policy_version 1046410 (0.0009) [2023-12-27 12:54:20,867][20347] Updated weights for policy 0, policy_version 1047245 (0.0006) [2023-12-27 12:54:20,902][20381] Updated weights for policy 1, policy_version 1046420 (0.0006) [2023-12-27 12:54:20,932][20347] Updated weights for policy 0, policy_version 1047255 (0.0006) [2023-12-27 12:54:20,998][20347] Updated weights for policy 0, policy_version 1047265 (0.0006) [2023-12-27 12:54:21,606][20381] Updated weights for policy 1, policy_version 1046430 (0.0008) [2023-12-27 12:54:21,676][20381] Updated weights for policy 1, policy_version 1046440 (0.0013) [2023-12-27 12:54:21,730][20381] Updated weights for policy 1, policy_version 1046450 (0.0007) [2023-12-27 12:54:21,742][20347] Updated weights for policy 0, policy_version 1047275 (0.0009) [2023-12-27 12:54:21,800][20347] Updated weights for policy 0, policy_version 1047285 (0.0008) [2023-12-27 12:54:21,855][20347] Updated weights for policy 0, policy_version 1047295 (0.0008) [2023-12-27 12:54:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 536076288. Throughput: 0: 10358.2, 1: 10419.4. Samples: 536050168. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:22,363][19170] Avg episode reward: [(0, '902.190'), (1, '853.090')] [2023-12-27 12:54:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001047304_268156928.pth... [2023-12-27 12:54:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001046456_267919360.pth... [2023-12-27 12:54:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001046120_267853824.pth [2023-12-27 12:54:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001045240_267608064.pth [2023-12-27 12:54:22,466][20381] Updated weights for policy 1, policy_version 1046460 (0.0012) [2023-12-27 12:54:22,529][20381] Updated weights for policy 1, policy_version 1046470 (0.0008) [2023-12-27 12:54:22,542][20347] Updated weights for policy 0, policy_version 1047305 (0.0007) [2023-12-27 12:54:22,590][20381] Updated weights for policy 1, policy_version 1046480 (0.0006) [2023-12-27 12:54:22,597][20347] Updated weights for policy 0, policy_version 1047315 (0.0007) [2023-12-27 12:54:22,663][20347] Updated weights for policy 0, policy_version 1047325 (0.0008) [2023-12-27 12:54:22,727][20347] Updated weights for policy 0, policy_version 1047335 (0.0007) [2023-12-27 12:54:23,261][20381] Updated weights for policy 1, policy_version 1046490 (0.0008) [2023-12-27 12:54:23,325][20381] Updated weights for policy 1, policy_version 1046500 (0.0008) [2023-12-27 12:54:23,375][20347] Updated weights for policy 0, policy_version 1047345 (0.0005) [2023-12-27 12:54:23,379][20381] Updated weights for policy 1, policy_version 1046510 (0.0007) [2023-12-27 12:54:23,424][20347] Updated weights for policy 0, policy_version 1047355 (0.0005) [2023-12-27 12:54:23,433][20381] Updated weights for policy 1, policy_version 1046520 (0.0008) [2023-12-27 12:54:23,478][20347] Updated weights for policy 0, policy_version 1047365 (0.0006) [2023-12-27 12:54:24,071][20381] Updated weights for policy 1, policy_version 1046530 (0.0008) [2023-12-27 12:54:24,120][20381] Updated weights for policy 1, policy_version 1046540 (0.0009) [2023-12-27 12:54:24,149][20347] Updated weights for policy 0, policy_version 1047375 (0.0006) [2023-12-27 12:54:24,175][20381] Updated weights for policy 1, policy_version 1046550 (0.0010) [2023-12-27 12:54:24,206][20347] Updated weights for policy 0, policy_version 1047385 (0.0006) [2023-12-27 12:54:24,265][20347] Updated weights for policy 0, policy_version 1047395 (0.0007) [2023-12-27 12:54:24,858][20381] Updated weights for policy 1, policy_version 1046560 (0.0008) [2023-12-27 12:54:24,907][20381] Updated weights for policy 1, policy_version 1046570 (0.0010) [2023-12-27 12:54:24,911][20347] Updated weights for policy 0, policy_version 1047405 (0.0006) [2023-12-27 12:54:24,960][20381] Updated weights for policy 1, policy_version 1046580 (0.0011) [2023-12-27 12:54:24,960][20347] Updated weights for policy 0, policy_version 1047415 (0.0005) [2023-12-27 12:54:25,011][20347] Updated weights for policy 0, policy_version 1047425 (0.0006) [2023-12-27 12:54:25,658][20381] Updated weights for policy 1, policy_version 1046590 (0.0010) [2023-12-27 12:54:25,684][20347] Updated weights for policy 0, policy_version 1047435 (0.0008) [2023-12-27 12:54:25,715][20381] Updated weights for policy 1, policy_version 1046600 (0.0008) [2023-12-27 12:54:25,736][20347] Updated weights for policy 0, policy_version 1047445 (0.0010) [2023-12-27 12:54:25,770][20381] Updated weights for policy 1, policy_version 1046610 (0.0010) [2023-12-27 12:54:25,784][20347] Updated weights for policy 0, policy_version 1047455 (0.0010) [2023-12-27 12:54:26,419][20381] Updated weights for policy 1, policy_version 1046620 (0.0011) [2023-12-27 12:54:26,464][20381] Updated weights for policy 1, policy_version 1046630 (0.0010) [2023-12-27 12:54:26,471][20347] Updated weights for policy 0, policy_version 1047465 (0.0006) [2023-12-27 12:54:26,515][20381] Updated weights for policy 1, policy_version 1046640 (0.0010) [2023-12-27 12:54:26,522][20347] Updated weights for policy 0, policy_version 1047475 (0.0006) [2023-12-27 12:54:26,567][20347] Updated weights for policy 0, policy_version 1047485 (0.0008) [2023-12-27 12:54:26,614][20347] Updated weights for policy 0, policy_version 1047495 (0.0007) [2023-12-27 12:54:27,168][20381] Updated weights for policy 1, policy_version 1046650 (0.0009) [2023-12-27 12:54:27,227][20381] Updated weights for policy 1, policy_version 1046660 (0.0005) [2023-12-27 12:54:27,282][20381] Updated weights for policy 1, policy_version 1046670 (0.0005) [2023-12-27 12:54:27,319][20347] Updated weights for policy 0, policy_version 1047505 (0.0007) [2023-12-27 12:54:27,346][20381] Updated weights for policy 1, policy_version 1046680 (0.0006) [2023-12-27 12:54:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 536182784. Throughput: 0: 10336.8, 1: 10368.3. Samples: 536172368. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:27,363][19170] Avg episode reward: [(0, '843.420'), (1, '784.540')] [2023-12-27 12:54:27,380][20347] Updated weights for policy 0, policy_version 1047515 (0.0007) [2023-12-27 12:54:27,431][20347] Updated weights for policy 0, policy_version 1047525 (0.0009) [2023-12-27 12:54:27,859][20381] Updated weights for policy 1, policy_version 1046690 (0.0005) [2023-12-27 12:54:27,906][20381] Updated weights for policy 1, policy_version 1046700 (0.0009) [2023-12-27 12:54:27,966][20381] Updated weights for policy 1, policy_version 1046710 (0.0009) [2023-12-27 12:54:28,136][20347] Updated weights for policy 0, policy_version 1047535 (0.0008) [2023-12-27 12:54:28,185][20347] Updated weights for policy 0, policy_version 1047545 (0.0007) [2023-12-27 12:54:28,242][20347] Updated weights for policy 0, policy_version 1047555 (0.0006) [2023-12-27 12:54:28,608][20381] Updated weights for policy 1, policy_version 1046720 (0.0007) [2023-12-27 12:54:28,663][20381] Updated weights for policy 1, policy_version 1046730 (0.0005) [2023-12-27 12:54:28,715][20381] Updated weights for policy 1, policy_version 1046740 (0.0005) [2023-12-27 12:54:28,922][20347] Updated weights for policy 0, policy_version 1047565 (0.0007) [2023-12-27 12:54:28,969][20347] Updated weights for policy 0, policy_version 1047575 (0.0007) [2023-12-27 12:54:29,019][20347] Updated weights for policy 0, policy_version 1047585 (0.0008) [2023-12-27 12:54:29,308][20381] Updated weights for policy 1, policy_version 1046750 (0.0007) [2023-12-27 12:54:29,372][20381] Updated weights for policy 1, policy_version 1046760 (0.0006) [2023-12-27 12:54:29,434][20381] Updated weights for policy 1, policy_version 1046770 (0.0007) [2023-12-27 12:54:29,721][20347] Updated weights for policy 0, policy_version 1047595 (0.0007) [2023-12-27 12:54:29,775][20347] Updated weights for policy 0, policy_version 1047605 (0.0007) [2023-12-27 12:54:29,837][20347] Updated weights for policy 0, policy_version 1047615 (0.0008) [2023-12-27 12:54:30,114][20381] Updated weights for policy 1, policy_version 1046780 (0.0007) [2023-12-27 12:54:30,159][20381] Updated weights for policy 1, policy_version 1046790 (0.0008) [2023-12-27 12:54:30,215][20381] Updated weights for policy 1, policy_version 1046800 (0.0008) [2023-12-27 12:54:30,535][20347] Updated weights for policy 0, policy_version 1047625 (0.0007) [2023-12-27 12:54:30,593][20347] Updated weights for policy 0, policy_version 1047635 (0.0009) [2023-12-27 12:54:30,646][20347] Updated weights for policy 0, policy_version 1047645 (0.0007) [2023-12-27 12:54:30,702][20347] Updated weights for policy 0, policy_version 1047655 (0.0008) [2023-12-27 12:54:30,863][20381] Updated weights for policy 1, policy_version 1046810 (0.0007) [2023-12-27 12:54:30,914][20381] Updated weights for policy 1, policy_version 1046820 (0.0005) [2023-12-27 12:54:30,969][20381] Updated weights for policy 1, policy_version 1046830 (0.0006) [2023-12-27 12:54:31,029][20381] Updated weights for policy 1, policy_version 1046840 (0.0009) [2023-12-27 12:54:31,351][20347] Updated weights for policy 0, policy_version 1047665 (0.0007) [2023-12-27 12:54:31,406][20347] Updated weights for policy 0, policy_version 1047675 (0.0008) [2023-12-27 12:54:31,465][20347] Updated weights for policy 0, policy_version 1047685 (0.0011) [2023-12-27 12:54:31,779][20381] Updated weights for policy 1, policy_version 1046850 (0.0007) [2023-12-27 12:54:31,838][20381] Updated weights for policy 1, policy_version 1046860 (0.0008) [2023-12-27 12:54:31,901][20381] Updated weights for policy 1, policy_version 1046870 (0.0009) [2023-12-27 12:54:32,107][20347] Updated weights for policy 0, policy_version 1047695 (0.0010) [2023-12-27 12:54:32,164][20347] Updated weights for policy 0, policy_version 1047705 (0.0010) [2023-12-27 12:54:32,220][20347] Updated weights for policy 0, policy_version 1047715 (0.0010) [2023-12-27 12:54:32,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 536289280. Throughput: 0: 10325.0, 1: 10373.8. Samples: 536297300. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:32,363][19170] Avg episode reward: [(0, '862.980'), (1, '784.370')] [2023-12-27 12:54:32,602][20381] Updated weights for policy 1, policy_version 1046880 (0.0008) [2023-12-27 12:54:32,651][20381] Updated weights for policy 1, policy_version 1046890 (0.0008) [2023-12-27 12:54:32,702][20381] Updated weights for policy 1, policy_version 1046900 (0.0007) [2023-12-27 12:54:32,908][20347] Updated weights for policy 0, policy_version 1047725 (0.0007) [2023-12-27 12:54:32,959][20347] Updated weights for policy 0, policy_version 1047735 (0.0005) [2023-12-27 12:54:33,009][20347] Updated weights for policy 0, policy_version 1047745 (0.0008) [2023-12-27 12:54:33,340][20381] Updated weights for policy 1, policy_version 1046910 (0.0006) [2023-12-27 12:54:33,390][20381] Updated weights for policy 1, policy_version 1046920 (0.0005) [2023-12-27 12:54:33,438][20381] Updated weights for policy 1, policy_version 1046930 (0.0005) [2023-12-27 12:54:33,676][20347] Updated weights for policy 0, policy_version 1047755 (0.0009) [2023-12-27 12:54:33,734][20347] Updated weights for policy 0, policy_version 1047765 (0.0007) [2023-12-27 12:54:33,785][20347] Updated weights for policy 0, policy_version 1047775 (0.0007) [2023-12-27 12:54:34,022][20381] Updated weights for policy 1, policy_version 1046940 (0.0007) [2023-12-27 12:54:34,070][20381] Updated weights for policy 1, policy_version 1046950 (0.0010) [2023-12-27 12:54:34,122][20381] Updated weights for policy 1, policy_version 1046960 (0.0010) [2023-12-27 12:54:34,389][20347] Updated weights for policy 0, policy_version 1047785 (0.0007) [2023-12-27 12:54:34,450][20347] Updated weights for policy 0, policy_version 1047795 (0.0007) [2023-12-27 12:54:34,512][20347] Updated weights for policy 0, policy_version 1047805 (0.0006) [2023-12-27 12:54:34,576][20347] Updated weights for policy 0, policy_version 1047815 (0.0010) [2023-12-27 12:54:34,760][20381] Updated weights for policy 1, policy_version 1046970 (0.0007) [2023-12-27 12:54:34,813][20381] Updated weights for policy 1, policy_version 1046980 (0.0009) [2023-12-27 12:54:34,866][20381] Updated weights for policy 1, policy_version 1046990 (0.0009) [2023-12-27 12:54:34,922][20381] Updated weights for policy 1, policy_version 1047000 (0.0005) [2023-12-27 12:54:35,188][20347] Updated weights for policy 0, policy_version 1047825 (0.0007) [2023-12-27 12:54:35,237][20347] Updated weights for policy 0, policy_version 1047835 (0.0010) [2023-12-27 12:54:35,291][20347] Updated weights for policy 0, policy_version 1047845 (0.0010) [2023-12-27 12:54:35,453][20381] Updated weights for policy 1, policy_version 1047010 (0.0006) [2023-12-27 12:54:35,512][20381] Updated weights for policy 1, policy_version 1047020 (0.0005) [2023-12-27 12:54:35,572][20381] Updated weights for policy 1, policy_version 1047030 (0.0005) [2023-12-27 12:54:35,992][20347] Updated weights for policy 0, policy_version 1047855 (0.0007) [2023-12-27 12:54:36,051][20347] Updated weights for policy 0, policy_version 1047865 (0.0006) [2023-12-27 12:54:36,098][20381] Updated weights for policy 1, policy_version 1047040 (0.0007) [2023-12-27 12:54:36,107][20347] Updated weights for policy 0, policy_version 1047875 (0.0007) [2023-12-27 12:54:36,163][20381] Updated weights for policy 1, policy_version 1047050 (0.0007) [2023-12-27 12:54:36,216][20381] Updated weights for policy 1, policy_version 1047060 (0.0006) [2023-12-27 12:54:36,748][20347] Updated weights for policy 0, policy_version 1047885 (0.0006) [2023-12-27 12:54:36,811][20347] Updated weights for policy 0, policy_version 1047895 (0.0006) [2023-12-27 12:54:36,870][20347] Updated weights for policy 0, policy_version 1047905 (0.0011) [2023-12-27 12:54:36,886][20381] Updated weights for policy 1, policy_version 1047070 (0.0006) [2023-12-27 12:54:36,940][20381] Updated weights for policy 1, policy_version 1047080 (0.0007) [2023-12-27 12:54:36,995][20381] Updated weights for policy 1, policy_version 1047090 (0.0007) [2023-12-27 12:54:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 536395776. Throughput: 0: 10327.3, 1: 10464.2. Samples: 536363448. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:37,363][19170] Avg episode reward: [(0, '853.270'), (1, '774.570')] [2023-12-27 12:54:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001047096_268083200.pth... [2023-12-27 12:54:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001047912_268312576.pth... [2023-12-27 12:54:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001046728_268009472.pth [2023-12-27 12:54:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001045880_267771904.pth [2023-12-27 12:54:37,473][20347] Updated weights for policy 0, policy_version 1047915 (0.0009) [2023-12-27 12:54:37,536][20347] Updated weights for policy 0, policy_version 1047925 (0.0006) [2023-12-27 12:54:37,596][20347] Updated weights for policy 0, policy_version 1047935 (0.0008) [2023-12-27 12:54:37,649][20381] Updated weights for policy 1, policy_version 1047100 (0.0006) [2023-12-27 12:54:37,697][20381] Updated weights for policy 1, policy_version 1047110 (0.0008) [2023-12-27 12:54:37,745][20381] Updated weights for policy 1, policy_version 1047120 (0.0007) [2023-12-27 12:54:38,291][20347] Updated weights for policy 0, policy_version 1047945 (0.0009) [2023-12-27 12:54:38,349][20347] Updated weights for policy 0, policy_version 1047955 (0.0007) [2023-12-27 12:54:38,400][20381] Updated weights for policy 1, policy_version 1047130 (0.0006) [2023-12-27 12:54:38,413][20347] Updated weights for policy 0, policy_version 1047965 (0.0007) [2023-12-27 12:54:38,454][20381] Updated weights for policy 1, policy_version 1047140 (0.0006) [2023-12-27 12:54:38,464][20347] Updated weights for policy 0, policy_version 1047975 (0.0007) [2023-12-27 12:54:38,514][20381] Updated weights for policy 1, policy_version 1047150 (0.0006) [2023-12-27 12:54:38,576][20381] Updated weights for policy 1, policy_version 1047160 (0.0007) [2023-12-27 12:54:39,149][20347] Updated weights for policy 0, policy_version 1047985 (0.0007) [2023-12-27 12:54:39,209][20347] Updated weights for policy 0, policy_version 1047995 (0.0007) [2023-12-27 12:54:39,242][20381] Updated weights for policy 1, policy_version 1047170 (0.0007) [2023-12-27 12:54:39,272][20347] Updated weights for policy 0, policy_version 1048005 (0.0007) [2023-12-27 12:54:39,305][20381] Updated weights for policy 1, policy_version 1047180 (0.0006) [2023-12-27 12:54:39,370][20381] Updated weights for policy 1, policy_version 1047190 (0.0007) [2023-12-27 12:54:39,953][20347] Updated weights for policy 0, policy_version 1048015 (0.0008) [2023-12-27 12:54:40,015][20347] Updated weights for policy 0, policy_version 1048025 (0.0008) [2023-12-27 12:54:40,074][20347] Updated weights for policy 0, policy_version 1048035 (0.0007) [2023-12-27 12:54:40,096][20381] Updated weights for policy 1, policy_version 1047200 (0.0008) [2023-12-27 12:54:40,161][20381] Updated weights for policy 1, policy_version 1047210 (0.0007) [2023-12-27 12:54:40,232][20381] Updated weights for policy 1, policy_version 1047220 (0.0008) [2023-12-27 12:54:40,773][20347] Updated weights for policy 0, policy_version 1048046 (0.0008) [2023-12-27 12:54:40,820][20347] Updated weights for policy 0, policy_version 1048056 (0.0007) [2023-12-27 12:54:40,864][20381] Updated weights for policy 1, policy_version 1047230 (0.0008) [2023-12-27 12:54:40,877][20347] Updated weights for policy 0, policy_version 1048066 (0.0006) [2023-12-27 12:54:40,914][20381] Updated weights for policy 1, policy_version 1047240 (0.0006) [2023-12-27 12:54:40,960][20381] Updated weights for policy 1, policy_version 1047250 (0.0009) [2023-12-27 12:54:41,662][20347] Updated weights for policy 0, policy_version 1048076 (0.0006) [2023-12-27 12:54:41,729][20347] Updated weights for policy 0, policy_version 1048086 (0.0009) [2023-12-27 12:54:41,742][20381] Updated weights for policy 1, policy_version 1047260 (0.0009) [2023-12-27 12:54:41,792][20347] Updated weights for policy 0, policy_version 1048096 (0.0008) [2023-12-27 12:54:41,804][20381] Updated weights for policy 1, policy_version 1047270 (0.0007) [2023-12-27 12:54:41,863][20381] Updated weights for policy 1, policy_version 1047280 (0.0006) [2023-12-27 12:54:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 536494080. Throughput: 0: 10294.1, 1: 10443.7. Samples: 536487684. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:42,363][19170] Avg episode reward: [(0, '892.620'), (1, '813.770')] [2023-12-27 12:54:42,530][20381] Updated weights for policy 1, policy_version 1047290 (0.0006) [2023-12-27 12:54:42,589][20381] Updated weights for policy 1, policy_version 1047300 (0.0005) [2023-12-27 12:54:42,602][20347] Updated weights for policy 0, policy_version 1048106 (0.0008) [2023-12-27 12:54:42,648][20381] Updated weights for policy 1, policy_version 1047310 (0.0006) [2023-12-27 12:54:42,658][20347] Updated weights for policy 0, policy_version 1048116 (0.0007) [2023-12-27 12:54:42,713][20381] Updated weights for policy 1, policy_version 1047320 (0.0006) [2023-12-27 12:54:42,726][20347] Updated weights for policy 0, policy_version 1048126 (0.0006) [2023-12-27 12:54:42,784][20347] Updated weights for policy 0, policy_version 1048136 (0.0007) [2023-12-27 12:54:43,407][20347] Updated weights for policy 0, policy_version 1048146 (0.0006) [2023-12-27 12:54:43,424][20381] Updated weights for policy 1, policy_version 1047330 (0.0006) [2023-12-27 12:54:43,463][20347] Updated weights for policy 0, policy_version 1048156 (0.0010) [2023-12-27 12:54:43,478][20381] Updated weights for policy 1, policy_version 1047340 (0.0006) [2023-12-27 12:54:43,508][20347] Updated weights for policy 0, policy_version 1048166 (0.0010) [2023-12-27 12:54:43,532][20381] Updated weights for policy 1, policy_version 1047350 (0.0009) [2023-12-27 12:54:44,098][20347] Updated weights for policy 0, policy_version 1048176 (0.0007) [2023-12-27 12:54:44,126][20381] Updated weights for policy 1, policy_version 1047360 (0.0011) [2023-12-27 12:54:44,154][20347] Updated weights for policy 0, policy_version 1048186 (0.0006) [2023-12-27 12:54:44,179][20381] Updated weights for policy 1, policy_version 1047370 (0.0010) [2023-12-27 12:54:44,209][20347] Updated weights for policy 0, policy_version 1048196 (0.0007) [2023-12-27 12:54:44,235][20381] Updated weights for policy 1, policy_version 1047380 (0.0010) [2023-12-27 12:54:44,880][20347] Updated weights for policy 0, policy_version 1048206 (0.0008) [2023-12-27 12:54:44,922][20381] Updated weights for policy 1, policy_version 1047390 (0.0009) [2023-12-27 12:54:44,942][20347] Updated weights for policy 0, policy_version 1048216 (0.0008) [2023-12-27 12:54:44,972][20381] Updated weights for policy 1, policy_version 1047400 (0.0008) [2023-12-27 12:54:45,000][20347] Updated weights for policy 0, policy_version 1048226 (0.0007) [2023-12-27 12:54:45,028][20381] Updated weights for policy 1, policy_version 1047410 (0.0007) [2023-12-27 12:54:45,691][20347] Updated weights for policy 0, policy_version 1048236 (0.0007) [2023-12-27 12:54:45,729][20381] Updated weights for policy 1, policy_version 1047420 (0.0011) [2023-12-27 12:54:45,745][20347] Updated weights for policy 0, policy_version 1048246 (0.0006) [2023-12-27 12:54:45,787][20381] Updated weights for policy 1, policy_version 1047430 (0.0011) [2023-12-27 12:54:45,804][20347] Updated weights for policy 0, policy_version 1048256 (0.0005) [2023-12-27 12:54:45,849][20381] Updated weights for policy 1, policy_version 1047440 (0.0010) [2023-12-27 12:54:46,489][20381] Updated weights for policy 1, policy_version 1047450 (0.0010) [2023-12-27 12:54:46,514][20347] Updated weights for policy 0, policy_version 1048266 (0.0007) [2023-12-27 12:54:46,550][20381] Updated weights for policy 1, policy_version 1047460 (0.0009) [2023-12-27 12:54:46,575][20347] Updated weights for policy 0, policy_version 1048276 (0.0007) [2023-12-27 12:54:46,609][20381] Updated weights for policy 1, policy_version 1047470 (0.0006) [2023-12-27 12:54:46,634][20347] Updated weights for policy 0, policy_version 1048286 (0.0006) [2023-12-27 12:54:46,667][20381] Updated weights for policy 1, policy_version 1047480 (0.0006) [2023-12-27 12:54:46,692][20347] Updated weights for policy 0, policy_version 1048296 (0.0007) [2023-12-27 12:54:47,315][20381] Updated weights for policy 1, policy_version 1047490 (0.0007) [2023-12-27 12:54:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 536592384. Throughput: 0: 10310.5, 1: 10406.7. Samples: 536609472. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:47,363][19170] Avg episode reward: [(0, '912.110'), (1, '794.110')] [2023-12-27 12:54:47,368][20381] Updated weights for policy 1, policy_version 1047500 (0.0007) [2023-12-27 12:54:47,370][20347] Updated weights for policy 0, policy_version 1048306 (0.0006) [2023-12-27 12:54:47,418][20381] Updated weights for policy 1, policy_version 1047510 (0.0005) [2023-12-27 12:54:47,426][20347] Updated weights for policy 0, policy_version 1048316 (0.0007) [2023-12-27 12:54:47,475][20347] Updated weights for policy 0, policy_version 1048326 (0.0009) [2023-12-27 12:54:47,980][20381] Updated weights for policy 1, policy_version 1047520 (0.0007) [2023-12-27 12:54:48,033][20381] Updated weights for policy 1, policy_version 1047530 (0.0009) [2023-12-27 12:54:48,085][20381] Updated weights for policy 1, policy_version 1047540 (0.0008) [2023-12-27 12:54:48,199][20347] Updated weights for policy 0, policy_version 1048336 (0.0008) [2023-12-27 12:54:48,256][20347] Updated weights for policy 0, policy_version 1048346 (0.0007) [2023-12-27 12:54:48,309][20347] Updated weights for policy 0, policy_version 1048356 (0.0006) [2023-12-27 12:54:48,793][20381] Updated weights for policy 1, policy_version 1047550 (0.0008) [2023-12-27 12:54:48,849][20381] Updated weights for policy 1, policy_version 1047560 (0.0008) [2023-12-27 12:54:48,901][20381] Updated weights for policy 1, policy_version 1047570 (0.0008) [2023-12-27 12:54:48,945][20347] Updated weights for policy 0, policy_version 1048366 (0.0007) [2023-12-27 12:54:49,000][20347] Updated weights for policy 0, policy_version 1048376 (0.0007) [2023-12-27 12:54:49,055][20347] Updated weights for policy 0, policy_version 1048386 (0.0007) [2023-12-27 12:54:49,556][20381] Updated weights for policy 1, policy_version 1047580 (0.0007) [2023-12-27 12:54:49,617][20381] Updated weights for policy 1, policy_version 1047590 (0.0006) [2023-12-27 12:54:49,679][20381] Updated weights for policy 1, policy_version 1047600 (0.0005) [2023-12-27 12:54:49,767][20347] Updated weights for policy 0, policy_version 1048396 (0.0009) [2023-12-27 12:54:49,832][20347] Updated weights for policy 0, policy_version 1048406 (0.0010) [2023-12-27 12:54:49,896][20347] Updated weights for policy 0, policy_version 1048416 (0.0008) [2023-12-27 12:54:50,337][20381] Updated weights for policy 1, policy_version 1047610 (0.0006) [2023-12-27 12:54:50,394][20381] Updated weights for policy 1, policy_version 1047620 (0.0008) [2023-12-27 12:54:50,449][20381] Updated weights for policy 1, policy_version 1047630 (0.0007) [2023-12-27 12:54:50,502][20381] Updated weights for policy 1, policy_version 1047640 (0.0007) [2023-12-27 12:54:50,582][20347] Updated weights for policy 0, policy_version 1048426 (0.0008) [2023-12-27 12:54:50,638][20347] Updated weights for policy 0, policy_version 1048436 (0.0006) [2023-12-27 12:54:50,688][20347] Updated weights for policy 0, policy_version 1048446 (0.0006) [2023-12-27 12:54:50,739][20347] Updated weights for policy 0, policy_version 1048456 (0.0006) [2023-12-27 12:54:51,270][20381] Updated weights for policy 1, policy_version 1047650 (0.0007) [2023-12-27 12:54:51,330][20381] Updated weights for policy 1, policy_version 1047660 (0.0006) [2023-12-27 12:54:51,396][20381] Updated weights for policy 1, policy_version 1047670 (0.0007) [2023-12-27 12:54:51,490][20347] Updated weights for policy 0, policy_version 1048466 (0.0008) [2023-12-27 12:54:51,545][20347] Updated weights for policy 0, policy_version 1048476 (0.0008) [2023-12-27 12:54:51,615][20347] Updated weights for policy 0, policy_version 1048486 (0.0008) [2023-12-27 12:54:52,063][20381] Updated weights for policy 1, policy_version 1047680 (0.0009) [2023-12-27 12:54:52,119][20381] Updated weights for policy 1, policy_version 1047690 (0.0010) [2023-12-27 12:54:52,171][20381] Updated weights for policy 1, policy_version 1047700 (0.0010) [2023-12-27 12:54:52,361][20347] Updated weights for policy 0, policy_version 1048496 (0.0007) [2023-12-27 12:54:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 536698880. Throughput: 0: 10242.8, 1: 10380.9. Samples: 536671204. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:52,363][19170] Avg episode reward: [(0, '912.000'), (1, '774.680')] [2023-12-27 12:54:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001047704_268238848.pth... [2023-12-27 12:54:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001046456_267919360.pth [2023-12-27 12:54:52,426][20347] Updated weights for policy 0, policy_version 1048506 (0.0008) [2023-12-27 12:54:52,478][20347] Updated weights for policy 0, policy_version 1048516 (0.0007) [2023-12-27 12:54:52,501][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001048520_268468224.pth... [2023-12-27 12:54:52,505][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001047304_268156928.pth [2023-12-27 12:54:52,889][20381] Updated weights for policy 1, policy_version 1047710 (0.0009) [2023-12-27 12:54:52,937][20381] Updated weights for policy 1, policy_version 1047720 (0.0008) [2023-12-27 12:54:52,989][20381] Updated weights for policy 1, policy_version 1047730 (0.0008) [2023-12-27 12:54:53,133][20347] Updated weights for policy 0, policy_version 1048526 (0.0006) [2023-12-27 12:54:53,186][20347] Updated weights for policy 0, policy_version 1048536 (0.0005) [2023-12-27 12:54:53,242][20347] Updated weights for policy 0, policy_version 1048546 (0.0006) [2023-12-27 12:54:53,634][20381] Updated weights for policy 1, policy_version 1047740 (0.0007) [2023-12-27 12:54:53,692][20381] Updated weights for policy 1, policy_version 1047750 (0.0009) [2023-12-27 12:54:53,737][20381] Updated weights for policy 1, policy_version 1047760 (0.0010) [2023-12-27 12:54:53,818][20347] Updated weights for policy 0, policy_version 1048556 (0.0008) [2023-12-27 12:54:53,874][20347] Updated weights for policy 0, policy_version 1048566 (0.0005) [2023-12-27 12:54:53,931][20347] Updated weights for policy 0, policy_version 1048576 (0.0010) [2023-12-27 12:54:54,416][20381] Updated weights for policy 1, policy_version 1047770 (0.0010) [2023-12-27 12:54:54,472][20381] Updated weights for policy 1, policy_version 1047780 (0.0010) [2023-12-27 12:54:54,527][20381] Updated weights for policy 1, policy_version 1047790 (0.0010) [2023-12-27 12:54:54,586][20381] Updated weights for policy 1, policy_version 1047800 (0.0009) [2023-12-27 12:54:54,587][20347] Updated weights for policy 0, policy_version 1048586 (0.0009) [2023-12-27 12:54:54,635][20347] Updated weights for policy 0, policy_version 1048596 (0.0005) [2023-12-27 12:54:54,685][20347] Updated weights for policy 0, policy_version 1048606 (0.0005) [2023-12-27 12:54:54,741][20347] Updated weights for policy 0, policy_version 1048616 (0.0009) [2023-12-27 12:54:55,285][20381] Updated weights for policy 1, policy_version 1047810 (0.0011) [2023-12-27 12:54:55,303][20292] KL-divergence is very high: 162.3342 [2023-12-27 12:54:55,328][20291] KL-divergence is very high: 457.6247 [2023-12-27 12:54:55,334][20291] KL-divergence is very high: 476.2154 [2023-12-27 12:54:55,335][20381] Updated weights for policy 1, policy_version 1047820 (0.0007) [2023-12-27 12:54:55,341][20347] Updated weights for policy 0, policy_version 1048626 (0.0010) [2023-12-27 12:54:55,369][20291] KL-divergence is very high: 195.5358 [2023-12-27 12:54:55,374][20291] KL-divergence is very high: 134.8321 [2023-12-27 12:54:55,382][20381] Updated weights for policy 1, policy_version 1047830 (0.0009) [2023-12-27 12:54:55,391][20347] Updated weights for policy 0, policy_version 1048636 (0.0010) [2023-12-27 12:54:55,451][20347] Updated weights for policy 0, policy_version 1048646 (0.0010) [2023-12-27 12:54:56,057][20347] Updated weights for policy 0, policy_version 1048656 (0.0007) [2023-12-27 12:54:56,107][20347] Updated weights for policy 0, policy_version 1048666 (0.0007) [2023-12-27 12:54:56,110][20381] Updated weights for policy 1, policy_version 1047840 (0.0007) [2023-12-27 12:54:56,160][20381] Updated weights for policy 1, policy_version 1047850 (0.0007) [2023-12-27 12:54:56,161][20347] Updated weights for policy 0, policy_version 1048676 (0.0005) [2023-12-27 12:54:56,190][20292] KL-divergence is very high: 120.1144 [2023-12-27 12:54:56,203][20381] Updated weights for policy 1, policy_version 1047860 (0.0008) [2023-12-27 12:54:56,751][20347] Updated weights for policy 0, policy_version 1048686 (0.0008) [2023-12-27 12:54:56,801][20347] Updated weights for policy 0, policy_version 1048696 (0.0010) [2023-12-27 12:54:56,845][20347] Updated weights for policy 0, policy_version 1048706 (0.0010) [2023-12-27 12:54:56,937][20381] Updated weights for policy 1, policy_version 1047870 (0.0007) [2023-12-27 12:54:56,992][20381] Updated weights for policy 1, policy_version 1047880 (0.0005) [2023-12-27 12:54:57,043][20381] Updated weights for policy 1, policy_version 1047890 (0.0005) [2023-12-27 12:54:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 536805376. Throughput: 0: 10265.8, 1: 10390.1. Samples: 536793988. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:54:57,363][19170] Avg episode reward: [(0, '892.610'), (1, '784.500')] [2023-12-27 12:54:57,514][20347] Updated weights for policy 0, policy_version 1048716 (0.0008) [2023-12-27 12:54:57,573][20347] Updated weights for policy 0, policy_version 1048726 (0.0005) [2023-12-27 12:54:57,576][20381] Updated weights for policy 1, policy_version 1047900 (0.0006) [2023-12-27 12:54:57,581][20291] KL-divergence is very high: 200.0042 [2023-12-27 12:54:57,634][20381] Updated weights for policy 1, policy_version 1047910 (0.0007) [2023-12-27 12:54:57,636][20347] Updated weights for policy 0, policy_version 1048736 (0.0005) [2023-12-27 12:54:57,688][20381] Updated weights for policy 1, policy_version 1047920 (0.0006) [2023-12-27 12:54:58,269][20347] Updated weights for policy 0, policy_version 1048746 (0.0008) [2023-12-27 12:54:58,325][20347] Updated weights for policy 0, policy_version 1048756 (0.0008) [2023-12-27 12:54:58,339][20381] Updated weights for policy 1, policy_version 1047930 (0.0010) [2023-12-27 12:54:58,391][20347] Updated weights for policy 0, policy_version 1048766 (0.0007) [2023-12-27 12:54:58,399][20381] Updated weights for policy 1, policy_version 1047940 (0.0008) [2023-12-27 12:54:58,448][20347] Updated weights for policy 0, policy_version 1048776 (0.0006) [2023-12-27 12:54:58,460][20381] Updated weights for policy 1, policy_version 1047950 (0.0006) [2023-12-27 12:54:58,519][20381] Updated weights for policy 1, policy_version 1047960 (0.0008) [2023-12-27 12:54:59,276][20381] Updated weights for policy 1, policy_version 1047970 (0.0009) [2023-12-27 12:54:59,310][20347] Updated weights for policy 0, policy_version 1048786 (0.0007) [2023-12-27 12:54:59,340][20381] Updated weights for policy 1, policy_version 1047980 (0.0011) [2023-12-27 12:54:59,376][20347] Updated weights for policy 0, policy_version 1048796 (0.0008) [2023-12-27 12:54:59,405][20381] Updated weights for policy 1, policy_version 1047990 (0.0008) [2023-12-27 12:54:59,437][20347] Updated weights for policy 0, policy_version 1048806 (0.0007) [2023-12-27 12:55:00,060][20381] Updated weights for policy 1, policy_version 1048000 (0.0006) [2023-12-27 12:55:00,090][20347] Updated weights for policy 0, policy_version 1048816 (0.0009) [2023-12-27 12:55:00,116][20381] Updated weights for policy 1, policy_version 1048010 (0.0011) [2023-12-27 12:55:00,146][20347] Updated weights for policy 0, policy_version 1048826 (0.0010) [2023-12-27 12:55:00,172][20381] Updated weights for policy 1, policy_version 1048020 (0.0010) [2023-12-27 12:55:00,202][20347] Updated weights for policy 0, policy_version 1048836 (0.0009) [2023-12-27 12:55:00,817][20381] Updated weights for policy 1, policy_version 1048030 (0.0005) [2023-12-27 12:55:00,858][20347] Updated weights for policy 0, policy_version 1048846 (0.0011) [2023-12-27 12:55:00,877][20381] Updated weights for policy 1, policy_version 1048040 (0.0006) [2023-12-27 12:55:00,919][20347] Updated weights for policy 0, policy_version 1048856 (0.0005) [2023-12-27 12:55:00,921][20381] Updated weights for policy 1, policy_version 1048050 (0.0007) [2023-12-27 12:55:00,982][20347] Updated weights for policy 0, policy_version 1048866 (0.0005) [2023-12-27 12:55:01,586][20381] Updated weights for policy 1, policy_version 1048060 (0.0007) [2023-12-27 12:55:01,619][20347] Updated weights for policy 0, policy_version 1048876 (0.0008) [2023-12-27 12:55:01,649][20381] Updated weights for policy 1, policy_version 1048070 (0.0007) [2023-12-27 12:55:01,687][20347] Updated weights for policy 0, policy_version 1048886 (0.0008) [2023-12-27 12:55:01,707][20381] Updated weights for policy 1, policy_version 1048080 (0.0009) [2023-12-27 12:55:01,752][20347] Updated weights for policy 0, policy_version 1048896 (0.0007) [2023-12-27 12:55:02,323][20381] Updated weights for policy 1, policy_version 1048090 (0.0010) [2023-12-27 12:55:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 536903680. Throughput: 0: 10234.8, 1: 10409.4. Samples: 536918304. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:55:02,363][19170] Avg episode reward: [(0, '892.550'), (1, '774.570')] [2023-12-27 12:55:02,397][20381] Updated weights for policy 1, policy_version 1048100 (0.0012) [2023-12-27 12:55:02,461][20381] Updated weights for policy 1, policy_version 1048110 (0.0006) [2023-12-27 12:55:02,474][20347] Updated weights for policy 0, policy_version 1048906 (0.0009) [2023-12-27 12:55:02,529][20381] Updated weights for policy 1, policy_version 1048120 (0.0008) [2023-12-27 12:55:02,529][20347] Updated weights for policy 0, policy_version 1048916 (0.0011) [2023-12-27 12:55:02,590][20347] Updated weights for policy 0, policy_version 1048926 (0.0009) [2023-12-27 12:55:02,648][20347] Updated weights for policy 0, policy_version 1048936 (0.0010) [2023-12-27 12:55:03,189][20381] Updated weights for policy 1, policy_version 1048130 (0.0007) [2023-12-27 12:55:03,238][20381] Updated weights for policy 1, policy_version 1048140 (0.0007) [2023-12-27 12:55:03,298][20381] Updated weights for policy 1, policy_version 1048150 (0.0008) [2023-12-27 12:55:03,327][20347] Updated weights for policy 0, policy_version 1048946 (0.0009) [2023-12-27 12:55:03,376][20347] Updated weights for policy 0, policy_version 1048956 (0.0010) [2023-12-27 12:55:03,426][20347] Updated weights for policy 0, policy_version 1048966 (0.0008) [2023-12-27 12:55:04,020][20381] Updated weights for policy 1, policy_version 1048160 (0.0009) [2023-12-27 12:55:04,029][20347] Updated weights for policy 0, policy_version 1048976 (0.0009) [2023-12-27 12:55:04,029][20291] KL-divergence is very high: 300.4324 [2023-12-27 12:55:04,076][20381] Updated weights for policy 1, policy_version 1048170 (0.0011) [2023-12-27 12:55:04,079][20291] KL-divergence is very high: 176.6111 [2023-12-27 12:55:04,090][20347] Updated weights for policy 0, policy_version 1048986 (0.0010) [2023-12-27 12:55:04,135][20381] Updated weights for policy 1, policy_version 1048180 (0.0011) [2023-12-27 12:55:04,149][20347] Updated weights for policy 0, policy_version 1048996 (0.0009) [2023-12-27 12:55:04,798][20291] KL-divergence is very high: 175.3609 [2023-12-27 12:55:04,799][20381] Updated weights for policy 1, policy_version 1048190 (0.0007) [2023-12-27 12:55:04,822][20347] Updated weights for policy 0, policy_version 1049006 (0.0010) [2023-12-27 12:55:04,848][20291] KL-divergence is very high: 125.5351 [2023-12-27 12:55:04,861][20381] Updated weights for policy 1, policy_version 1048200 (0.0005) [2023-12-27 12:55:04,879][20347] Updated weights for policy 0, policy_version 1049016 (0.0010) [2023-12-27 12:55:04,909][20381] Updated weights for policy 1, policy_version 1048210 (0.0005) [2023-12-27 12:55:04,934][20347] Updated weights for policy 0, policy_version 1049026 (0.0010) [2023-12-27 12:55:05,401][20381] Updated weights for policy 1, policy_version 1048220 (0.0006) [2023-12-27 12:55:05,445][20381] Updated weights for policy 1, policy_version 1048230 (0.0007) [2023-12-27 12:55:05,502][20381] Updated weights for policy 1, policy_version 1048240 (0.0007) [2023-12-27 12:55:05,602][20347] Updated weights for policy 0, policy_version 1049036 (0.0010) [2023-12-27 12:55:05,659][20347] Updated weights for policy 0, policy_version 1049046 (0.0010) [2023-12-27 12:55:05,719][20347] Updated weights for policy 0, policy_version 1049056 (0.0010) [2023-12-27 12:55:06,155][20381] Updated weights for policy 1, policy_version 1048250 (0.0007) [2023-12-27 12:55:06,220][20381] Updated weights for policy 1, policy_version 1048260 (0.0008) [2023-12-27 12:55:06,283][20381] Updated weights for policy 1, policy_version 1048270 (0.0007) [2023-12-27 12:55:06,334][20381] Updated weights for policy 1, policy_version 1048280 (0.0007) [2023-12-27 12:55:06,452][20347] Updated weights for policy 0, policy_version 1049066 (0.0010) [2023-12-27 12:55:06,512][20347] Updated weights for policy 0, policy_version 1049076 (0.0008) [2023-12-27 12:55:06,576][20347] Updated weights for policy 0, policy_version 1049086 (0.0008) [2023-12-27 12:55:06,644][20347] Updated weights for policy 0, policy_version 1049096 (0.0008) [2023-12-27 12:55:07,056][20381] Updated weights for policy 1, policy_version 1048290 (0.0008) [2023-12-27 12:55:07,106][20381] Updated weights for policy 1, policy_version 1048300 (0.0007) [2023-12-27 12:55:07,158][20381] Updated weights for policy 1, policy_version 1048310 (0.0008) [2023-12-27 12:55:07,323][20347] Updated weights for policy 0, policy_version 1049106 (0.0010) [2023-12-27 12:55:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 537010176. Throughput: 0: 10244.8, 1: 10435.3. Samples: 536980772. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:55:07,363][19170] Avg episode reward: [(0, '882.770'), (1, '843.240')] [2023-12-27 12:55:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001048312_268394496.pth... [2023-12-27 12:55:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001047096_268083200.pth [2023-12-27 12:55:07,378][20347] Updated weights for policy 0, policy_version 1049116 (0.0010) [2023-12-27 12:55:07,436][20347] Updated weights for policy 0, policy_version 1049126 (0.0010) [2023-12-27 12:55:07,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001049128_268623872.pth... [2023-12-27 12:55:07,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001047912_268312576.pth [2023-12-27 12:55:07,872][20381] Updated weights for policy 1, policy_version 1048320 (0.0008) [2023-12-27 12:55:07,930][20381] Updated weights for policy 1, policy_version 1048330 (0.0007) [2023-12-27 12:55:07,990][20381] Updated weights for policy 1, policy_version 1048340 (0.0009) [2023-12-27 12:55:08,093][20347] Updated weights for policy 0, policy_version 1049136 (0.0006) [2023-12-27 12:55:08,141][20347] Updated weights for policy 0, policy_version 1049146 (0.0006) [2023-12-27 12:55:08,193][20347] Updated weights for policy 0, policy_version 1049156 (0.0010) [2023-12-27 12:55:08,709][20381] Updated weights for policy 1, policy_version 1048350 (0.0009) [2023-12-27 12:55:08,757][20381] Updated weights for policy 1, policy_version 1048360 (0.0007) [2023-12-27 12:55:08,810][20381] Updated weights for policy 1, policy_version 1048370 (0.0008) [2023-12-27 12:55:08,909][20347] Updated weights for policy 0, policy_version 1049167 (0.0008) [2023-12-27 12:55:08,958][20347] Updated weights for policy 0, policy_version 1049177 (0.0007) [2023-12-27 12:55:09,012][20347] Updated weights for policy 0, policy_version 1049187 (0.0008) [2023-12-27 12:55:09,568][20381] Updated weights for policy 1, policy_version 1048380 (0.0008) [2023-12-27 12:55:09,617][20381] Updated weights for policy 1, policy_version 1048390 (0.0008) [2023-12-27 12:55:09,680][20381] Updated weights for policy 1, policy_version 1048400 (0.0008) [2023-12-27 12:55:09,705][20347] Updated weights for policy 0, policy_version 1049197 (0.0008) [2023-12-27 12:55:09,766][20347] Updated weights for policy 0, policy_version 1049207 (0.0008) [2023-12-27 12:55:09,821][20347] Updated weights for policy 0, policy_version 1049217 (0.0008) [2023-12-27 12:55:10,411][20381] Updated weights for policy 1, policy_version 1048410 (0.0007) [2023-12-27 12:55:10,471][20381] Updated weights for policy 1, policy_version 1048420 (0.0008) [2023-12-27 12:55:10,529][20381] Updated weights for policy 1, policy_version 1048430 (0.0006) [2023-12-27 12:55:10,584][20381] Updated weights for policy 1, policy_version 1048440 (0.0007) [2023-12-27 12:55:10,593][20347] Updated weights for policy 0, policy_version 1049227 (0.0010) [2023-12-27 12:55:10,650][20347] Updated weights for policy 0, policy_version 1049237 (0.0011) [2023-12-27 12:55:10,707][20347] Updated weights for policy 0, policy_version 1049247 (0.0010) [2023-12-27 12:55:11,308][20381] Updated weights for policy 1, policy_version 1048450 (0.0007) [2023-12-27 12:55:11,368][20381] Updated weights for policy 1, policy_version 1048460 (0.0008) [2023-12-27 12:55:11,428][20347] Updated weights for policy 0, policy_version 1049257 (0.0010) [2023-12-27 12:55:11,433][20381] Updated weights for policy 1, policy_version 1048470 (0.0009) [2023-12-27 12:55:11,488][20347] Updated weights for policy 0, policy_version 1049267 (0.0011) [2023-12-27 12:55:11,544][20347] Updated weights for policy 0, policy_version 1049277 (0.0010) [2023-12-27 12:55:11,604][20347] Updated weights for policy 0, policy_version 1049287 (0.0011) [2023-12-27 12:55:12,162][20381] Updated weights for policy 1, policy_version 1048480 (0.0007) [2023-12-27 12:55:12,217][20381] Updated weights for policy 1, policy_version 1048490 (0.0007) [2023-12-27 12:55:12,275][20381] Updated weights for policy 1, policy_version 1048500 (0.0007) [2023-12-27 12:55:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 537108480. Throughput: 0: 10212.0, 1: 10399.3. Samples: 537099876. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:55:12,363][19170] Avg episode reward: [(0, '882.660'), (1, '852.980')] [2023-12-27 12:55:12,404][20347] Updated weights for policy 0, policy_version 1049297 (0.0010) [2023-12-27 12:55:12,463][20347] Updated weights for policy 0, policy_version 1049307 (0.0010) [2023-12-27 12:55:12,516][20347] Updated weights for policy 0, policy_version 1049317 (0.0010) [2023-12-27 12:55:12,992][20381] Updated weights for policy 1, policy_version 1048510 (0.0007) [2023-12-27 12:55:13,054][20381] Updated weights for policy 1, policy_version 1048520 (0.0007) [2023-12-27 12:55:13,117][20381] Updated weights for policy 1, policy_version 1048530 (0.0009) [2023-12-27 12:55:13,236][20347] Updated weights for policy 0, policy_version 1049327 (0.0010) [2023-12-27 12:55:13,297][20347] Updated weights for policy 0, policy_version 1049337 (0.0010) [2023-12-27 12:55:13,359][20347] Updated weights for policy 0, policy_version 1049347 (0.0011) [2023-12-27 12:55:13,779][20381] Updated weights for policy 1, policy_version 1048540 (0.0006) [2023-12-27 12:55:13,839][20381] Updated weights for policy 1, policy_version 1048550 (0.0005) [2023-12-27 12:55:13,864][20292] KL-divergence is very high: 113.3395 [2023-12-27 12:55:13,893][20381] Updated weights for policy 1, policy_version 1048560 (0.0005) [2023-12-27 12:55:13,904][20292] KL-divergence is very high: 120.4603 [2023-12-27 12:55:14,041][20347] Updated weights for policy 0, policy_version 1049357 (0.0008) [2023-12-27 12:55:14,097][20347] Updated weights for policy 0, policy_version 1049367 (0.0005) [2023-12-27 12:55:14,156][20347] Updated weights for policy 0, policy_version 1049377 (0.0005) [2023-12-27 12:55:14,460][20381] Updated weights for policy 1, policy_version 1048570 (0.0005) [2023-12-27 12:55:14,515][20381] Updated weights for policy 1, policy_version 1048580 (0.0005) [2023-12-27 12:55:14,569][20381] Updated weights for policy 1, policy_version 1048590 (0.0005) [2023-12-27 12:55:14,620][20381] Updated weights for policy 1, policy_version 1048600 (0.0006) [2023-12-27 12:55:14,770][20347] Updated weights for policy 0, policy_version 1049387 (0.0007) [2023-12-27 12:55:14,828][20347] Updated weights for policy 0, policy_version 1049397 (0.0009) [2023-12-27 12:55:14,888][20347] Updated weights for policy 0, policy_version 1049407 (0.0008) [2023-12-27 12:55:15,296][20381] Updated weights for policy 1, policy_version 1048610 (0.0007) [2023-12-27 12:55:15,347][20381] Updated weights for policy 1, policy_version 1048620 (0.0006) [2023-12-27 12:55:15,394][20381] Updated weights for policy 1, policy_version 1048630 (0.0010) [2023-12-27 12:55:15,559][20347] Updated weights for policy 0, policy_version 1049417 (0.0008) [2023-12-27 12:55:15,606][20347] Updated weights for policy 0, policy_version 1049427 (0.0008) [2023-12-27 12:55:15,650][20347] Updated weights for policy 0, policy_version 1049437 (0.0009) [2023-12-27 12:55:15,710][20347] Updated weights for policy 0, policy_version 1049447 (0.0007) [2023-12-27 12:55:15,951][20381] Updated weights for policy 1, policy_version 1048640 (0.0011) [2023-12-27 12:55:15,954][20292] KL-divergence is very high: 107.9410 [2023-12-27 12:55:16,000][20381] Updated weights for policy 1, policy_version 1048650 (0.0010) [2023-12-27 12:55:16,045][20381] Updated weights for policy 1, policy_version 1048660 (0.0006) [2023-12-27 12:55:16,471][20347] Updated weights for policy 0, policy_version 1049457 (0.0008) [2023-12-27 12:55:16,530][20347] Updated weights for policy 0, policy_version 1049467 (0.0009) [2023-12-27 12:55:16,595][20347] Updated weights for policy 0, policy_version 1049477 (0.0008) [2023-12-27 12:55:16,622][20381] Updated weights for policy 1, policy_version 1048670 (0.0005) [2023-12-27 12:55:16,684][20381] Updated weights for policy 1, policy_version 1048680 (0.0009) [2023-12-27 12:55:16,744][20381] Updated weights for policy 1, policy_version 1048690 (0.0011) [2023-12-27 12:55:17,335][20381] Updated weights for policy 1, policy_version 1048700 (0.0006) [2023-12-27 12:55:17,340][20347] Updated weights for policy 0, policy_version 1049487 (0.0006) [2023-12-27 12:55:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 537206784. Throughput: 0: 10157.9, 1: 10427.0. Samples: 537223616. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:55:17,363][19170] Avg episode reward: [(0, '853.200'), (1, '745.310')] [2023-12-27 12:55:17,383][20381] Updated weights for policy 1, policy_version 1048710 (0.0006) [2023-12-27 12:55:17,394][20347] Updated weights for policy 0, policy_version 1049497 (0.0007) [2023-12-27 12:55:17,433][20381] Updated weights for policy 1, policy_version 1048720 (0.0007) [2023-12-27 12:55:17,442][20347] Updated weights for policy 0, policy_version 1049507 (0.0006) [2023-12-27 12:55:18,050][20381] Updated weights for policy 1, policy_version 1048730 (0.0006) [2023-12-27 12:55:18,102][20381] Updated weights for policy 1, policy_version 1048740 (0.0008) [2023-12-27 12:55:18,160][20347] Updated weights for policy 0, policy_version 1049517 (0.0005) [2023-12-27 12:55:18,162][20381] Updated weights for policy 1, policy_version 1048750 (0.0007) [2023-12-27 12:55:18,211][20347] Updated weights for policy 0, policy_version 1049527 (0.0005) [2023-12-27 12:55:18,220][20381] Updated weights for policy 1, policy_version 1048760 (0.0006) [2023-12-27 12:55:18,266][20347] Updated weights for policy 0, policy_version 1049537 (0.0007) [2023-12-27 12:55:18,870][20381] Updated weights for policy 1, policy_version 1048770 (0.0007) [2023-12-27 12:55:18,928][20381] Updated weights for policy 1, policy_version 1048780 (0.0007) [2023-12-27 12:55:18,989][20381] Updated weights for policy 1, policy_version 1048790 (0.0007) [2023-12-27 12:55:19,005][20347] Updated weights for policy 0, policy_version 1049547 (0.0007) [2023-12-27 12:55:19,060][20347] Updated weights for policy 0, policy_version 1049557 (0.0005) [2023-12-27 12:55:19,117][20347] Updated weights for policy 0, policy_version 1049567 (0.0005) [2023-12-27 12:55:19,699][20381] Updated weights for policy 1, policy_version 1048800 (0.0010) [2023-12-27 12:55:19,753][20381] Updated weights for policy 1, policy_version 1048810 (0.0011) [2023-12-27 12:55:19,820][20381] Updated weights for policy 1, policy_version 1048820 (0.0011) [2023-12-27 12:55:19,828][20347] Updated weights for policy 0, policy_version 1049577 (0.0007) [2023-12-27 12:55:19,881][20347] Updated weights for policy 0, policy_version 1049587 (0.0008) [2023-12-27 12:55:19,945][20347] Updated weights for policy 0, policy_version 1049597 (0.0008) [2023-12-27 12:55:20,004][20347] Updated weights for policy 0, policy_version 1049607 (0.0008) [2023-12-27 12:55:20,500][20381] Updated weights for policy 1, policy_version 1048830 (0.0009) [2023-12-27 12:55:20,557][20381] Updated weights for policy 1, policy_version 1048840 (0.0007) [2023-12-27 12:55:20,624][20381] Updated weights for policy 1, policy_version 1048850 (0.0008) [2023-12-27 12:55:20,734][20347] Updated weights for policy 0, policy_version 1049617 (0.0010) [2023-12-27 12:55:20,798][20347] Updated weights for policy 0, policy_version 1049627 (0.0011) [2023-12-27 12:55:20,860][20347] Updated weights for policy 0, policy_version 1049637 (0.0010) [2023-12-27 12:55:21,367][20381] Updated weights for policy 1, policy_version 1048860 (0.0007) [2023-12-27 12:55:21,433][20381] Updated weights for policy 1, policy_version 1048870 (0.0007) [2023-12-27 12:55:21,499][20381] Updated weights for policy 1, policy_version 1048880 (0.0006) [2023-12-27 12:55:21,532][20347] Updated weights for policy 0, policy_version 1049647 (0.0009) [2023-12-27 12:55:21,594][20347] Updated weights for policy 0, policy_version 1049657 (0.0007) [2023-12-27 12:55:21,661][20347] Updated weights for policy 0, policy_version 1049667 (0.0010) [2023-12-27 12:55:22,164][20381] Updated weights for policy 1, policy_version 1048890 (0.0007) [2023-12-27 12:55:22,220][20381] Updated weights for policy 1, policy_version 1048900 (0.0007) [2023-12-27 12:55:22,284][20381] Updated weights for policy 1, policy_version 1048910 (0.0007) [2023-12-27 12:55:22,349][20381] Updated weights for policy 1, policy_version 1048920 (0.0006) [2023-12-27 12:55:22,356][20347] Updated weights for policy 0, policy_version 1049677 (0.0011) [2023-12-27 12:55:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 537313280. Throughput: 0: 10092.3, 1: 10355.2. Samples: 537283584. Policy #0 lag: (min: 31.0, avg: 32.1, max: 59.0) [2023-12-27 12:55:22,363][19170] Avg episode reward: [(0, '775.030'), (1, '774.660')] [2023-12-27 12:55:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001048920_268550144.pth... [2023-12-27 12:55:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001047704_268238848.pth [2023-12-27 12:55:22,422][20347] Updated weights for policy 0, policy_version 1049687 (0.0008) [2023-12-27 12:55:22,483][20347] Updated weights for policy 0, policy_version 1049697 (0.0007) [2023-12-27 12:55:22,524][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001049704_268771328.pth... [2023-12-27 12:55:22,528][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001048520_268468224.pth [2023-12-27 12:55:23,064][20381] Updated weights for policy 1, policy_version 1048930 (0.0006) [2023-12-27 12:55:23,119][20381] Updated weights for policy 1, policy_version 1048940 (0.0007) [2023-12-27 12:55:23,154][20347] Updated weights for policy 0, policy_version 1049707 (0.0009) [2023-12-27 12:55:23,181][20381] Updated weights for policy 1, policy_version 1048950 (0.0006) [2023-12-27 12:55:23,210][20347] Updated weights for policy 0, policy_version 1049717 (0.0009) [2023-12-27 12:55:23,217][20291] KL-divergence is very high: 167.7773 [2023-12-27 12:55:23,225][20291] KL-divergence is very high: 169.8800 [2023-12-27 12:55:23,264][20291] KL-divergence is very high: 161.0988 [2023-12-27 12:55:23,270][20291] KL-divergence is very high: 136.8446 [2023-12-27 12:55:23,270][20347] Updated weights for policy 0, policy_version 1049727 (0.0006) [2023-12-27 12:55:23,850][20347] Updated weights for policy 0, policy_version 1049737 (0.0005) [2023-12-27 12:55:23,871][20381] Updated weights for policy 1, policy_version 1048960 (0.0006) [2023-12-27 12:55:23,916][20347] Updated weights for policy 0, policy_version 1049747 (0.0007) [2023-12-27 12:55:23,920][20381] Updated weights for policy 1, policy_version 1048970 (0.0006) [2023-12-27 12:55:23,966][20347] Updated weights for policy 0, policy_version 1049757 (0.0006) [2023-12-27 12:55:23,976][20381] Updated weights for policy 1, policy_version 1048980 (0.0005) [2023-12-27 12:55:24,023][20347] Updated weights for policy 0, policy_version 1049767 (0.0005) [2023-12-27 12:55:24,610][20381] Updated weights for policy 1, policy_version 1048990 (0.0006) [2023-12-27 12:55:24,620][20347] Updated weights for policy 0, policy_version 1049777 (0.0007) [2023-12-27 12:55:24,659][20381] Updated weights for policy 1, policy_version 1049000 (0.0005) [2023-12-27 12:55:24,672][20347] Updated weights for policy 0, policy_version 1049787 (0.0005) [2023-12-27 12:55:24,708][20381] Updated weights for policy 1, policy_version 1049010 (0.0006) [2023-12-27 12:55:24,725][20347] Updated weights for policy 0, policy_version 1049797 (0.0006) [2023-12-27 12:55:25,416][20381] Updated weights for policy 1, policy_version 1049020 (0.0006) [2023-12-27 12:55:25,417][20347] Updated weights for policy 0, policy_version 1049807 (0.0006) [2023-12-27 12:55:25,468][20381] Updated weights for policy 1, policy_version 1049030 (0.0006) [2023-12-27 12:55:25,474][20347] Updated weights for policy 0, policy_version 1049817 (0.0007) [2023-12-27 12:55:25,526][20381] Updated weights for policy 1, policy_version 1049040 (0.0006) [2023-12-27 12:55:25,531][20347] Updated weights for policy 0, policy_version 1049827 (0.0006) [2023-12-27 12:55:26,211][20347] Updated weights for policy 0, policy_version 1049837 (0.0006) [2023-12-27 12:55:26,246][20381] Updated weights for policy 1, policy_version 1049050 (0.0007) [2023-12-27 12:55:26,255][20347] Updated weights for policy 0, policy_version 1049847 (0.0008) [2023-12-27 12:55:26,297][20381] Updated weights for policy 1, policy_version 1049060 (0.0005) [2023-12-27 12:55:26,309][20347] Updated weights for policy 0, policy_version 1049857 (0.0006) [2023-12-27 12:55:26,354][20381] Updated weights for policy 1, policy_version 1049070 (0.0007) [2023-12-27 12:55:26,408][20381] Updated weights for policy 1, policy_version 1049080 (0.0007) [2023-12-27 12:55:26,941][20347] Updated weights for policy 0, policy_version 1049867 (0.0006) [2023-12-27 12:55:26,992][20347] Updated weights for policy 0, policy_version 1049877 (0.0007) [2023-12-27 12:55:27,048][20347] Updated weights for policy 0, policy_version 1049887 (0.0007) [2023-12-27 12:55:27,082][20381] Updated weights for policy 1, policy_version 1049090 (0.0006) [2023-12-27 12:55:27,134][20381] Updated weights for policy 1, policy_version 1049100 (0.0006) [2023-12-27 12:55:27,191][20381] Updated weights for policy 1, policy_version 1049110 (0.0008) [2023-12-27 12:55:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 537419776. Throughput: 0: 10107.2, 1: 10288.1. Samples: 537405472. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:27,363][19170] Avg episode reward: [(0, '814.670'), (1, '823.610')] [2023-12-27 12:55:27,637][20347] Updated weights for policy 0, policy_version 1049897 (0.0005) [2023-12-27 12:55:27,680][20347] Updated weights for policy 0, policy_version 1049907 (0.0005) [2023-12-27 12:55:27,729][20347] Updated weights for policy 0, policy_version 1049917 (0.0007) [2023-12-27 12:55:27,776][20347] Updated weights for policy 0, policy_version 1049927 (0.0007) [2023-12-27 12:55:27,931][20381] Updated weights for policy 1, policy_version 1049120 (0.0007) [2023-12-27 12:55:27,986][20381] Updated weights for policy 1, policy_version 1049130 (0.0008) [2023-12-27 12:55:28,034][20381] Updated weights for policy 1, policy_version 1049140 (0.0007) [2023-12-27 12:55:28,456][20347] Updated weights for policy 0, policy_version 1049937 (0.0007) [2023-12-27 12:55:28,508][20347] Updated weights for policy 0, policy_version 1049947 (0.0008) [2023-12-27 12:55:28,563][20347] Updated weights for policy 0, policy_version 1049957 (0.0008) [2023-12-27 12:55:28,729][20381] Updated weights for policy 1, policy_version 1049150 (0.0007) [2023-12-27 12:55:28,790][20381] Updated weights for policy 1, policy_version 1049160 (0.0008) [2023-12-27 12:55:28,843][20381] Updated weights for policy 1, policy_version 1049170 (0.0008) [2023-12-27 12:55:29,216][20347] Updated weights for policy 0, policy_version 1049967 (0.0006) [2023-12-27 12:55:29,280][20347] Updated weights for policy 0, policy_version 1049977 (0.0010) [2023-12-27 12:55:29,339][20347] Updated weights for policy 0, policy_version 1049987 (0.0011) [2023-12-27 12:55:29,598][20381] Updated weights for policy 1, policy_version 1049180 (0.0007) [2023-12-27 12:55:29,654][20381] Updated weights for policy 1, policy_version 1049190 (0.0008) [2023-12-27 12:55:29,708][20381] Updated weights for policy 1, policy_version 1049200 (0.0008) [2023-12-27 12:55:29,927][20347] Updated weights for policy 0, policy_version 1049997 (0.0011) [2023-12-27 12:55:29,984][20347] Updated weights for policy 0, policy_version 1050007 (0.0009) [2023-12-27 12:55:30,041][20347] Updated weights for policy 0, policy_version 1050017 (0.0010) [2023-12-27 12:55:30,459][20381] Updated weights for policy 1, policy_version 1049210 (0.0009) [2023-12-27 12:55:30,517][20381] Updated weights for policy 1, policy_version 1049220 (0.0007) [2023-12-27 12:55:30,568][20381] Updated weights for policy 1, policy_version 1049230 (0.0007) [2023-12-27 12:55:30,629][20381] Updated weights for policy 1, policy_version 1049240 (0.0009) [2023-12-27 12:55:30,687][20347] Updated weights for policy 0, policy_version 1050027 (0.0011) [2023-12-27 12:55:30,749][20347] Updated weights for policy 0, policy_version 1050037 (0.0010) [2023-12-27 12:55:30,803][20347] Updated weights for policy 0, policy_version 1050047 (0.0010) [2023-12-27 12:55:31,231][20381] Updated weights for policy 1, policy_version 1049250 (0.0008) [2023-12-27 12:55:31,290][20381] Updated weights for policy 1, policy_version 1049260 (0.0008) [2023-12-27 12:55:31,349][20381] Updated weights for policy 1, policy_version 1049270 (0.0009) [2023-12-27 12:55:31,458][20347] Updated weights for policy 0, policy_version 1050057 (0.0010) [2023-12-27 12:55:31,507][20347] Updated weights for policy 0, policy_version 1050067 (0.0006) [2023-12-27 12:55:31,570][20347] Updated weights for policy 0, policy_version 1050077 (0.0006) [2023-12-27 12:55:31,641][20347] Updated weights for policy 0, policy_version 1050087 (0.0007) [2023-12-27 12:55:32,129][20381] Updated weights for policy 1, policy_version 1049280 (0.0008) [2023-12-27 12:55:32,181][20381] Updated weights for policy 1, policy_version 1049290 (0.0008) [2023-12-27 12:55:32,231][20381] Updated weights for policy 1, policy_version 1049300 (0.0007) [2023-12-27 12:55:32,288][20347] Updated weights for policy 0, policy_version 1050097 (0.0006) [2023-12-27 12:55:32,354][20347] Updated weights for policy 0, policy_version 1050107 (0.0006) [2023-12-27 12:55:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 537518080. Throughput: 0: 10202.4, 1: 10218.1. Samples: 537528396. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:32,363][19170] Avg episode reward: [(0, '843.740'), (1, '843.140')] [2023-12-27 12:55:32,405][20347] Updated weights for policy 0, policy_version 1050117 (0.0007) [2023-12-27 12:55:32,968][20381] Updated weights for policy 1, policy_version 1049310 (0.0008) [2023-12-27 12:55:33,016][20381] Updated weights for policy 1, policy_version 1049320 (0.0008) [2023-12-27 12:55:33,065][20381] Updated weights for policy 1, policy_version 1049330 (0.0009) [2023-12-27 12:55:33,077][20347] Updated weights for policy 0, policy_version 1050127 (0.0006) [2023-12-27 12:55:33,127][20347] Updated weights for policy 0, policy_version 1050137 (0.0006) [2023-12-27 12:55:33,172][20347] Updated weights for policy 0, policy_version 1050147 (0.0007) [2023-12-27 12:55:33,699][20381] Updated weights for policy 1, policy_version 1049340 (0.0008) [2023-12-27 12:55:33,754][20381] Updated weights for policy 1, policy_version 1049350 (0.0005) [2023-12-27 12:55:33,810][20381] Updated weights for policy 1, policy_version 1049360 (0.0005) [2023-12-27 12:55:33,864][20347] Updated weights for policy 0, policy_version 1050157 (0.0009) [2023-12-27 12:55:33,927][20347] Updated weights for policy 0, policy_version 1050167 (0.0010) [2023-12-27 12:55:33,991][20347] Updated weights for policy 0, policy_version 1050177 (0.0008) [2023-12-27 12:55:34,476][20381] Updated weights for policy 1, policy_version 1049370 (0.0008) [2023-12-27 12:55:34,537][20381] Updated weights for policy 1, policy_version 1049380 (0.0011) [2023-12-27 12:55:34,597][20381] Updated weights for policy 1, policy_version 1049390 (0.0011) [2023-12-27 12:55:34,650][20381] Updated weights for policy 1, policy_version 1049400 (0.0011) [2023-12-27 12:55:34,666][20347] Updated weights for policy 0, policy_version 1050187 (0.0007) [2023-12-27 12:55:34,719][20347] Updated weights for policy 0, policy_version 1050197 (0.0007) [2023-12-27 12:55:34,773][20347] Updated weights for policy 0, policy_version 1050207 (0.0007) [2023-12-27 12:55:35,355][20381] Updated weights for policy 1, policy_version 1049410 (0.0008) [2023-12-27 12:55:35,411][20381] Updated weights for policy 1, policy_version 1049420 (0.0008) [2023-12-27 12:55:35,463][20381] Updated weights for policy 1, policy_version 1049430 (0.0008) [2023-12-27 12:55:35,471][20347] Updated weights for policy 0, policy_version 1050217 (0.0007) [2023-12-27 12:55:35,521][20347] Updated weights for policy 0, policy_version 1050227 (0.0007) [2023-12-27 12:55:35,572][20347] Updated weights for policy 0, policy_version 1050237 (0.0007) [2023-12-27 12:55:35,624][20347] Updated weights for policy 0, policy_version 1050247 (0.0007) [2023-12-27 12:55:36,133][20381] Updated weights for policy 1, policy_version 1049440 (0.0007) [2023-12-27 12:55:36,195][20381] Updated weights for policy 1, policy_version 1049450 (0.0009) [2023-12-27 12:55:36,253][20381] Updated weights for policy 1, policy_version 1049460 (0.0006) [2023-12-27 12:55:36,260][20347] Updated weights for policy 0, policy_version 1050257 (0.0006) [2023-12-27 12:55:36,318][20347] Updated weights for policy 0, policy_version 1050267 (0.0006) [2023-12-27 12:55:36,379][20347] Updated weights for policy 0, policy_version 1050277 (0.0009) [2023-12-27 12:55:36,871][20381] Updated weights for policy 1, policy_version 1049470 (0.0008) [2023-12-27 12:55:36,934][20381] Updated weights for policy 1, policy_version 1049480 (0.0009) [2023-12-27 12:55:36,993][20381] Updated weights for policy 1, policy_version 1049490 (0.0008) [2023-12-27 12:55:37,010][20347] Updated weights for policy 0, policy_version 1050287 (0.0007) [2023-12-27 12:55:37,067][20347] Updated weights for policy 0, policy_version 1050297 (0.0006) [2023-12-27 12:55:37,123][20347] Updated weights for policy 0, policy_version 1050307 (0.0006) [2023-12-27 12:55:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 537624576. Throughput: 0: 10209.6, 1: 10200.4. Samples: 537589652. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:37,363][19170] Avg episode reward: [(0, '804.840'), (1, '852.890')] [2023-12-27 12:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001050312_268926976.pth... [2023-12-27 12:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001049496_268697600.pth... [2023-12-27 12:55:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001049128_268623872.pth [2023-12-27 12:55:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001048312_268394496.pth [2023-12-27 12:55:37,685][20381] Updated weights for policy 1, policy_version 1049500 (0.0006) [2023-12-27 12:55:37,732][20381] Updated weights for policy 1, policy_version 1049510 (0.0007) [2023-12-27 12:55:37,778][20347] Updated weights for policy 0, policy_version 1050317 (0.0006) [2023-12-27 12:55:37,787][20381] Updated weights for policy 1, policy_version 1049520 (0.0006) [2023-12-27 12:55:37,831][20347] Updated weights for policy 0, policy_version 1050327 (0.0006) [2023-12-27 12:55:37,880][20347] Updated weights for policy 0, policy_version 1050337 (0.0008) [2023-12-27 12:55:38,436][20381] Updated weights for policy 1, policy_version 1049530 (0.0006) [2023-12-27 12:55:38,499][20381] Updated weights for policy 1, policy_version 1049540 (0.0010) [2023-12-27 12:55:38,558][20381] Updated weights for policy 1, policy_version 1049550 (0.0010) [2023-12-27 12:55:38,616][20381] Updated weights for policy 1, policy_version 1049560 (0.0011) [2023-12-27 12:55:38,658][20347] Updated weights for policy 0, policy_version 1050347 (0.0007) [2023-12-27 12:55:38,722][20347] Updated weights for policy 0, policy_version 1050357 (0.0007) [2023-12-27 12:55:38,782][20347] Updated weights for policy 0, policy_version 1050367 (0.0007) [2023-12-27 12:55:39,339][20381] Updated weights for policy 1, policy_version 1049570 (0.0008) [2023-12-27 12:55:39,411][20381] Updated weights for policy 1, policy_version 1049580 (0.0010) [2023-12-27 12:55:39,450][20347] Updated weights for policy 0, policy_version 1050377 (0.0007) [2023-12-27 12:55:39,477][20381] Updated weights for policy 1, policy_version 1049590 (0.0011) [2023-12-27 12:55:39,512][20347] Updated weights for policy 0, policy_version 1050387 (0.0006) [2023-12-27 12:55:39,571][20347] Updated weights for policy 0, policy_version 1050397 (0.0005) [2023-12-27 12:55:39,638][20347] Updated weights for policy 0, policy_version 1050407 (0.0006) [2023-12-27 12:55:40,190][20381] Updated weights for policy 1, policy_version 1049600 (0.0009) [2023-12-27 12:55:40,204][20347] Updated weights for policy 0, policy_version 1050417 (0.0007) [2023-12-27 12:55:40,249][20381] Updated weights for policy 1, policy_version 1049610 (0.0008) [2023-12-27 12:55:40,260][20347] Updated weights for policy 0, policy_version 1050427 (0.0007) [2023-12-27 12:55:40,305][20381] Updated weights for policy 1, policy_version 1049620 (0.0009) [2023-12-27 12:55:40,313][20347] Updated weights for policy 0, policy_version 1050437 (0.0006) [2023-12-27 12:55:40,963][20347] Updated weights for policy 0, policy_version 1050447 (0.0007) [2023-12-27 12:55:41,017][20347] Updated weights for policy 0, policy_version 1050457 (0.0008) [2023-12-27 12:55:41,066][20381] Updated weights for policy 1, policy_version 1049630 (0.0008) [2023-12-27 12:55:41,084][20347] Updated weights for policy 0, policy_version 1050467 (0.0008) [2023-12-27 12:55:41,126][20381] Updated weights for policy 1, policy_version 1049640 (0.0007) [2023-12-27 12:55:41,201][20381] Updated weights for policy 1, policy_version 1049650 (0.0007) [2023-12-27 12:55:41,820][20347] Updated weights for policy 0, policy_version 1050477 (0.0009) [2023-12-27 12:55:41,876][20347] Updated weights for policy 0, policy_version 1050487 (0.0010) [2023-12-27 12:55:41,936][20347] Updated weights for policy 0, policy_version 1050497 (0.0011) [2023-12-27 12:55:41,964][20381] Updated weights for policy 1, policy_version 1049660 (0.0006) [2023-12-27 12:55:42,030][20381] Updated weights for policy 1, policy_version 1049670 (0.0008) [2023-12-27 12:55:42,090][20381] Updated weights for policy 1, policy_version 1049680 (0.0010) [2023-12-27 12:55:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 537722880. Throughput: 0: 10197.5, 1: 10195.1. Samples: 537711656. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:42,363][19170] Avg episode reward: [(0, '833.600'), (1, '656.980')] [2023-12-27 12:55:42,640][20347] Updated weights for policy 0, policy_version 1050507 (0.0010) [2023-12-27 12:55:42,702][20347] Updated weights for policy 0, policy_version 1050517 (0.0008) [2023-12-27 12:55:42,761][20347] Updated weights for policy 0, policy_version 1050527 (0.0006) [2023-12-27 12:55:42,762][20381] Updated weights for policy 1, policy_version 1049690 (0.0010) [2023-12-27 12:55:42,818][20381] Updated weights for policy 1, policy_version 1049700 (0.0007) [2023-12-27 12:55:42,875][20381] Updated weights for policy 1, policy_version 1049710 (0.0009) [2023-12-27 12:55:42,934][20381] Updated weights for policy 1, policy_version 1049720 (0.0008) [2023-12-27 12:55:43,370][20347] Updated weights for policy 0, policy_version 1050537 (0.0005) [2023-12-27 12:55:43,418][20347] Updated weights for policy 0, policy_version 1050547 (0.0007) [2023-12-27 12:55:43,475][20347] Updated weights for policy 0, policy_version 1050557 (0.0008) [2023-12-27 12:55:43,531][20347] Updated weights for policy 0, policy_version 1050567 (0.0007) [2023-12-27 12:55:43,569][20381] Updated weights for policy 1, policy_version 1049730 (0.0007) [2023-12-27 12:55:43,625][20381] Updated weights for policy 1, policy_version 1049740 (0.0010) [2023-12-27 12:55:43,687][20381] Updated weights for policy 1, policy_version 1049750 (0.0010) [2023-12-27 12:55:44,259][20347] Updated weights for policy 0, policy_version 1050577 (0.0008) [2023-12-27 12:55:44,321][20347] Updated weights for policy 0, policy_version 1050587 (0.0007) [2023-12-27 12:55:44,377][20347] Updated weights for policy 0, policy_version 1050597 (0.0006) [2023-12-27 12:55:44,393][20381] Updated weights for policy 1, policy_version 1049760 (0.0008) [2023-12-27 12:55:44,457][20381] Updated weights for policy 1, policy_version 1049770 (0.0008) [2023-12-27 12:55:44,519][20381] Updated weights for policy 1, policy_version 1049780 (0.0006) [2023-12-27 12:55:45,067][20381] Updated weights for policy 1, policy_version 1049790 (0.0005) [2023-12-27 12:55:45,130][20381] Updated weights for policy 1, policy_version 1049800 (0.0006) [2023-12-27 12:55:45,149][20347] Updated weights for policy 0, policy_version 1050607 (0.0007) [2023-12-27 12:55:45,183][20381] Updated weights for policy 1, policy_version 1049810 (0.0005) [2023-12-27 12:55:45,207][20347] Updated weights for policy 0, policy_version 1050617 (0.0008) [2023-12-27 12:55:45,268][20347] Updated weights for policy 0, policy_version 1050627 (0.0008) [2023-12-27 12:55:45,772][20381] Updated weights for policy 1, policy_version 1049820 (0.0006) [2023-12-27 12:55:45,828][20381] Updated weights for policy 1, policy_version 1049830 (0.0008) [2023-12-27 12:55:45,888][20381] Updated weights for policy 1, policy_version 1049840 (0.0008) [2023-12-27 12:55:45,965][20347] Updated weights for policy 0, policy_version 1050637 (0.0008) [2023-12-27 12:55:46,018][20347] Updated weights for policy 0, policy_version 1050647 (0.0008) [2023-12-27 12:55:46,067][20347] Updated weights for policy 0, policy_version 1050657 (0.0008) [2023-12-27 12:55:46,581][20381] Updated weights for policy 1, policy_version 1049850 (0.0007) [2023-12-27 12:55:46,629][20381] Updated weights for policy 1, policy_version 1049860 (0.0009) [2023-12-27 12:55:46,691][20381] Updated weights for policy 1, policy_version 1049870 (0.0008) [2023-12-27 12:55:46,718][20347] Updated weights for policy 0, policy_version 1050667 (0.0008) [2023-12-27 12:55:46,750][20381] Updated weights for policy 1, policy_version 1049880 (0.0006) [2023-12-27 12:55:46,771][20347] Updated weights for policy 0, policy_version 1050677 (0.0007) [2023-12-27 12:55:46,833][20347] Updated weights for policy 0, policy_version 1050687 (0.0006) [2023-12-27 12:55:47,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 537821184. Throughput: 0: 10153.0, 1: 10178.4. Samples: 537833216. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:47,364][19170] Avg episode reward: [(0, '872.920'), (1, '715.770')] [2023-12-27 12:55:47,465][20381] Updated weights for policy 1, policy_version 1049890 (0.0008) [2023-12-27 12:55:47,523][20381] Updated weights for policy 1, policy_version 1049900 (0.0008) [2023-12-27 12:55:47,535][20347] Updated weights for policy 0, policy_version 1050697 (0.0008) [2023-12-27 12:55:47,585][20381] Updated weights for policy 1, policy_version 1049910 (0.0006) [2023-12-27 12:55:47,597][20347] Updated weights for policy 0, policy_version 1050707 (0.0006) [2023-12-27 12:55:47,652][20347] Updated weights for policy 0, policy_version 1050717 (0.0008) [2023-12-27 12:55:47,715][20347] Updated weights for policy 0, policy_version 1050727 (0.0007) [2023-12-27 12:55:48,226][20381] Updated weights for policy 1, policy_version 1049920 (0.0005) [2023-12-27 12:55:48,272][20381] Updated weights for policy 1, policy_version 1049930 (0.0005) [2023-12-27 12:55:48,320][20381] Updated weights for policy 1, policy_version 1049940 (0.0008) [2023-12-27 12:55:48,378][20347] Updated weights for policy 0, policy_version 1050737 (0.0008) [2023-12-27 12:55:48,434][20347] Updated weights for policy 0, policy_version 1050747 (0.0008) [2023-12-27 12:55:48,485][20347] Updated weights for policy 0, policy_version 1050757 (0.0008) [2023-12-27 12:55:48,993][20381] Updated weights for policy 1, policy_version 1049950 (0.0007) [2023-12-27 12:55:49,049][20381] Updated weights for policy 1, policy_version 1049960 (0.0008) [2023-12-27 12:55:49,100][20381] Updated weights for policy 1, policy_version 1049970 (0.0006) [2023-12-27 12:55:49,194][20347] Updated weights for policy 0, policy_version 1050767 (0.0008) [2023-12-27 12:55:49,258][20347] Updated weights for policy 0, policy_version 1050777 (0.0007) [2023-12-27 12:55:49,322][20347] Updated weights for policy 0, policy_version 1050787 (0.0007) [2023-12-27 12:55:49,796][20381] Updated weights for policy 1, policy_version 1049980 (0.0006) [2023-12-27 12:55:49,854][20381] Updated weights for policy 1, policy_version 1049990 (0.0009) [2023-12-27 12:55:49,921][20381] Updated weights for policy 1, policy_version 1050000 (0.0008) [2023-12-27 12:55:50,051][20347] Updated weights for policy 0, policy_version 1050797 (0.0007) [2023-12-27 12:55:50,078][20291] KL-divergence is very high: 100.9472 [2023-12-27 12:55:50,111][20347] Updated weights for policy 0, policy_version 1050807 (0.0006) [2023-12-27 12:55:50,124][20291] KL-divergence is very high: 137.7466 [2023-12-27 12:55:50,175][20347] Updated weights for policy 0, policy_version 1050817 (0.0007) [2023-12-27 12:55:50,628][20381] Updated weights for policy 1, policy_version 1050010 (0.0008) [2023-12-27 12:55:50,687][20381] Updated weights for policy 1, policy_version 1050020 (0.0007) [2023-12-27 12:55:50,748][20381] Updated weights for policy 1, policy_version 1050030 (0.0007) [2023-12-27 12:55:50,772][20292] KL-divergence is very high: 169.6639 [2023-12-27 12:55:50,805][20381] Updated weights for policy 1, policy_version 1050040 (0.0005) [2023-12-27 12:55:50,821][20347] Updated weights for policy 0, policy_version 1050827 (0.0007) [2023-12-27 12:55:50,877][20347] Updated weights for policy 0, policy_version 1050837 (0.0008) [2023-12-27 12:55:50,924][20347] Updated weights for policy 0, policy_version 1050847 (0.0008) [2023-12-27 12:55:51,536][20381] Updated weights for policy 1, policy_version 1050050 (0.0008) [2023-12-27 12:55:51,604][20381] Updated weights for policy 1, policy_version 1050060 (0.0009) [2023-12-27 12:55:51,633][20347] Updated weights for policy 0, policy_version 1050857 (0.0007) [2023-12-27 12:55:51,668][20381] Updated weights for policy 1, policy_version 1050070 (0.0008) [2023-12-27 12:55:51,694][20347] Updated weights for policy 0, policy_version 1050867 (0.0006) [2023-12-27 12:55:51,761][20347] Updated weights for policy 0, policy_version 1050877 (0.0008) [2023-12-27 12:55:51,821][20347] Updated weights for policy 0, policy_version 1050887 (0.0010) [2023-12-27 12:55:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 537919488. Throughput: 0: 10122.6, 1: 10158.6. Samples: 537893428. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:52,363][19170] Avg episode reward: [(0, '931.560'), (1, '774.570')] [2023-12-27 12:55:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001050888_269074432.pth... [2023-12-27 12:55:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001049704_268771328.pth [2023-12-27 12:55:52,396][20381] Updated weights for policy 1, policy_version 1050080 (0.0008) [2023-12-27 12:55:52,451][20381] Updated weights for policy 1, policy_version 1050090 (0.0007) [2023-12-27 12:55:52,469][20347] Updated weights for policy 0, policy_version 1050897 (0.0007) [2023-12-27 12:55:52,502][20381] Updated weights for policy 1, policy_version 1050100 (0.0007) [2023-12-27 12:55:52,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001050104_268853248.pth... [2023-12-27 12:55:52,523][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001048920_268550144.pth [2023-12-27 12:55:52,527][20347] Updated weights for policy 0, policy_version 1050907 (0.0007) [2023-12-27 12:55:52,581][20347] Updated weights for policy 0, policy_version 1050917 (0.0008) [2023-12-27 12:55:53,158][20381] Updated weights for policy 1, policy_version 1050110 (0.0006) [2023-12-27 12:55:53,216][20381] Updated weights for policy 1, policy_version 1050120 (0.0005) [2023-12-27 12:55:53,262][20381] Updated weights for policy 1, policy_version 1050130 (0.0005) [2023-12-27 12:55:53,269][20347] Updated weights for policy 0, policy_version 1050927 (0.0009) [2023-12-27 12:55:53,328][20347] Updated weights for policy 0, policy_version 1050937 (0.0005) [2023-12-27 12:55:53,386][20347] Updated weights for policy 0, policy_version 1050947 (0.0005) [2023-12-27 12:55:53,887][20381] Updated weights for policy 1, policy_version 1050140 (0.0005) [2023-12-27 12:55:53,944][20381] Updated weights for policy 1, policy_version 1050150 (0.0006) [2023-12-27 12:55:54,000][20381] Updated weights for policy 1, policy_version 1050160 (0.0009) [2023-12-27 12:55:54,015][20347] Updated weights for policy 0, policy_version 1050957 (0.0005) [2023-12-27 12:55:54,063][20347] Updated weights for policy 0, policy_version 1050967 (0.0010) [2023-12-27 12:55:54,115][20347] Updated weights for policy 0, policy_version 1050977 (0.0010) [2023-12-27 12:55:54,592][20381] Updated weights for policy 1, policy_version 1050170 (0.0006) [2023-12-27 12:55:54,653][20381] Updated weights for policy 1, policy_version 1050180 (0.0005) [2023-12-27 12:55:54,709][20381] Updated weights for policy 1, policy_version 1050190 (0.0005) [2023-12-27 12:55:54,756][20381] Updated weights for policy 1, policy_version 1050200 (0.0007) [2023-12-27 12:55:54,805][20347] Updated weights for policy 0, policy_version 1050987 (0.0009) [2023-12-27 12:55:54,852][20347] Updated weights for policy 0, policy_version 1050997 (0.0005) [2023-12-27 12:55:54,915][20347] Updated weights for policy 0, policy_version 1051007 (0.0008) [2023-12-27 12:55:55,343][20381] Updated weights for policy 1, policy_version 1050210 (0.0008) [2023-12-27 12:55:55,406][20381] Updated weights for policy 1, policy_version 1050220 (0.0008) [2023-12-27 12:55:55,465][20381] Updated weights for policy 1, policy_version 1050230 (0.0010) [2023-12-27 12:55:55,541][20347] Updated weights for policy 0, policy_version 1051017 (0.0010) [2023-12-27 12:55:55,598][20347] Updated weights for policy 0, policy_version 1051027 (0.0007) [2023-12-27 12:55:55,652][20347] Updated weights for policy 0, policy_version 1051037 (0.0007) [2023-12-27 12:55:55,703][20347] Updated weights for policy 0, policy_version 1051047 (0.0007) [2023-12-27 12:55:56,141][20381] Updated weights for policy 1, policy_version 1050240 (0.0010) [2023-12-27 12:55:56,199][20381] Updated weights for policy 1, policy_version 1050250 (0.0010) [2023-12-27 12:55:56,228][20292] KL-divergence is very high: 108.6003 [2023-12-27 12:55:56,257][20381] Updated weights for policy 1, policy_version 1050260 (0.0010) [2023-12-27 12:55:56,383][20347] Updated weights for policy 0, policy_version 1051057 (0.0008) [2023-12-27 12:55:56,427][20347] Updated weights for policy 0, policy_version 1051067 (0.0007) [2023-12-27 12:55:56,479][20347] Updated weights for policy 0, policy_version 1051077 (0.0007) [2023-12-27 12:55:56,887][20381] Updated weights for policy 1, policy_version 1050270 (0.0009) [2023-12-27 12:55:56,948][20381] Updated weights for policy 1, policy_version 1050280 (0.0008) [2023-12-27 12:55:57,003][20381] Updated weights for policy 1, policy_version 1050290 (0.0008) [2023-12-27 12:55:57,123][20347] Updated weights for policy 0, policy_version 1051087 (0.0007) [2023-12-27 12:55:57,177][20347] Updated weights for policy 0, policy_version 1051097 (0.0006) [2023-12-27 12:55:57,233][20347] Updated weights for policy 0, policy_version 1051107 (0.0006) [2023-12-27 12:55:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 538034176. Throughput: 0: 10178.2, 1: 10226.4. Samples: 538018084. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:55:57,363][19170] Avg episode reward: [(0, '941.330'), (1, '794.190')] [2023-12-27 12:55:57,593][20381] Updated weights for policy 1, policy_version 1050300 (0.0007) [2023-12-27 12:55:57,651][20381] Updated weights for policy 1, policy_version 1050310 (0.0005) [2023-12-27 12:55:57,709][20381] Updated weights for policy 1, policy_version 1050320 (0.0005) [2023-12-27 12:55:57,783][20347] Updated weights for policy 0, policy_version 1051117 (0.0005) [2023-12-27 12:55:57,829][20347] Updated weights for policy 0, policy_version 1051127 (0.0005) [2023-12-27 12:55:57,872][20347] Updated weights for policy 0, policy_version 1051137 (0.0005) [2023-12-27 12:55:58,324][20381] Updated weights for policy 1, policy_version 1050330 (0.0006) [2023-12-27 12:55:58,394][20381] Updated weights for policy 1, policy_version 1050340 (0.0008) [2023-12-27 12:55:58,455][20381] Updated weights for policy 1, policy_version 1050350 (0.0009) [2023-12-27 12:55:58,496][20347] Updated weights for policy 0, policy_version 1051147 (0.0006) [2023-12-27 12:55:58,514][20381] Updated weights for policy 1, policy_version 1050360 (0.0006) [2023-12-27 12:55:58,555][20347] Updated weights for policy 0, policy_version 1051157 (0.0007) [2023-12-27 12:55:58,612][20347] Updated weights for policy 0, policy_version 1051167 (0.0007) [2023-12-27 12:55:59,324][20381] Updated weights for policy 1, policy_version 1050370 (0.0009) [2023-12-27 12:55:59,394][20381] Updated weights for policy 1, policy_version 1050380 (0.0009) [2023-12-27 12:55:59,434][20347] Updated weights for policy 0, policy_version 1051177 (0.0009) [2023-12-27 12:55:59,451][20381] Updated weights for policy 1, policy_version 1050390 (0.0007) [2023-12-27 12:55:59,488][20347] Updated weights for policy 0, policy_version 1051187 (0.0007) [2023-12-27 12:55:59,543][20347] Updated weights for policy 0, policy_version 1051197 (0.0008) [2023-12-27 12:55:59,601][20347] Updated weights for policy 0, policy_version 1051207 (0.0007) [2023-12-27 12:56:00,173][20381] Updated weights for policy 1, policy_version 1050400 (0.0007) [2023-12-27 12:56:00,232][20381] Updated weights for policy 1, policy_version 1050410 (0.0008) [2023-12-27 12:56:00,259][20347] Updated weights for policy 0, policy_version 1051217 (0.0007) [2023-12-27 12:56:00,291][20381] Updated weights for policy 1, policy_version 1050420 (0.0006) [2023-12-27 12:56:00,305][20347] Updated weights for policy 0, policy_version 1051227 (0.0008) [2023-12-27 12:56:00,360][20347] Updated weights for policy 0, policy_version 1051237 (0.0008) [2023-12-27 12:56:00,960][20381] Updated weights for policy 1, policy_version 1050430 (0.0007) [2023-12-27 12:56:01,008][20381] Updated weights for policy 1, policy_version 1050440 (0.0007) [2023-12-27 12:56:01,039][20347] Updated weights for policy 0, policy_version 1051247 (0.0007) [2023-12-27 12:56:01,072][20381] Updated weights for policy 1, policy_version 1050450 (0.0008) [2023-12-27 12:56:01,079][20292] KL-divergence is very high: 223.1475 [2023-12-27 12:56:01,097][20347] Updated weights for policy 0, policy_version 1051257 (0.0006) [2023-12-27 12:56:01,161][20347] Updated weights for policy 0, policy_version 1051267 (0.0007) [2023-12-27 12:56:01,817][20347] Updated weights for policy 0, policy_version 1051277 (0.0008) [2023-12-27 12:56:01,870][20381] Updated weights for policy 1, policy_version 1050460 (0.0006) [2023-12-27 12:56:01,876][20347] Updated weights for policy 0, policy_version 1051287 (0.0007) [2023-12-27 12:56:01,916][20381] Updated weights for policy 1, policy_version 1050470 (0.0005) [2023-12-27 12:56:01,935][20347] Updated weights for policy 0, policy_version 1051297 (0.0008) [2023-12-27 12:56:01,965][20381] Updated weights for policy 1, policy_version 1050480 (0.0008) [2023-12-27 12:56:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 538132480. Throughput: 0: 10261.9, 1: 10128.5. Samples: 538141184. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:02,363][19170] Avg episode reward: [(0, '931.560'), (1, '725.490')] [2023-12-27 12:56:02,646][20381] Updated weights for policy 1, policy_version 1050490 (0.0006) [2023-12-27 12:56:02,661][20347] Updated weights for policy 0, policy_version 1051307 (0.0006) [2023-12-27 12:56:02,703][20381] Updated weights for policy 1, policy_version 1050500 (0.0006) [2023-12-27 12:56:02,715][20347] Updated weights for policy 0, policy_version 1051317 (0.0007) [2023-12-27 12:56:02,765][20381] Updated weights for policy 1, policy_version 1050510 (0.0006) [2023-12-27 12:56:02,776][20347] Updated weights for policy 0, policy_version 1051327 (0.0008) [2023-12-27 12:56:02,823][20381] Updated weights for policy 1, policy_version 1050520 (0.0005) [2023-12-27 12:56:03,360][20381] Updated weights for policy 1, policy_version 1050530 (0.0006) [2023-12-27 12:56:03,416][20381] Updated weights for policy 1, policy_version 1050540 (0.0009) [2023-12-27 12:56:03,465][20347] Updated weights for policy 0, policy_version 1051337 (0.0005) [2023-12-27 12:56:03,471][20381] Updated weights for policy 1, policy_version 1050550 (0.0009) [2023-12-27 12:56:03,522][20347] Updated weights for policy 0, policy_version 1051347 (0.0007) [2023-12-27 12:56:03,578][20347] Updated weights for policy 0, policy_version 1051357 (0.0006) [2023-12-27 12:56:03,634][20347] Updated weights for policy 0, policy_version 1051367 (0.0006) [2023-12-27 12:56:04,219][20381] Updated weights for policy 1, policy_version 1050560 (0.0008) [2023-12-27 12:56:04,274][20381] Updated weights for policy 1, policy_version 1050570 (0.0008) [2023-12-27 12:56:04,297][20347] Updated weights for policy 0, policy_version 1051377 (0.0006) [2023-12-27 12:56:04,336][20381] Updated weights for policy 1, policy_version 1050580 (0.0006) [2023-12-27 12:56:04,358][20347] Updated weights for policy 0, policy_version 1051387 (0.0007) [2023-12-27 12:56:04,410][20347] Updated weights for policy 0, policy_version 1051397 (0.0008) [2023-12-27 12:56:05,022][20381] Updated weights for policy 1, policy_version 1050590 (0.0007) [2023-12-27 12:56:05,074][20381] Updated weights for policy 1, policy_version 1050600 (0.0007) [2023-12-27 12:56:05,094][20347] Updated weights for policy 0, policy_version 1051407 (0.0009) [2023-12-27 12:56:05,125][20381] Updated weights for policy 1, policy_version 1050610 (0.0006) [2023-12-27 12:56:05,152][20347] Updated weights for policy 0, policy_version 1051417 (0.0010) [2023-12-27 12:56:05,210][20347] Updated weights for policy 0, policy_version 1051427 (0.0009) [2023-12-27 12:56:05,841][20381] Updated weights for policy 1, policy_version 1050620 (0.0007) [2023-12-27 12:56:05,849][20347] Updated weights for policy 0, policy_version 1051437 (0.0008) [2023-12-27 12:56:05,894][20381] Updated weights for policy 1, policy_version 1050630 (0.0007) [2023-12-27 12:56:05,916][20347] Updated weights for policy 0, policy_version 1051447 (0.0011) [2023-12-27 12:56:05,948][20381] Updated weights for policy 1, policy_version 1050640 (0.0007) [2023-12-27 12:56:05,973][20347] Updated weights for policy 0, policy_version 1051457 (0.0010) [2023-12-27 12:56:06,606][20347] Updated weights for policy 0, policy_version 1051467 (0.0009) [2023-12-27 12:56:06,670][20347] Updated weights for policy 0, policy_version 1051477 (0.0006) [2023-12-27 12:56:06,711][20381] Updated weights for policy 1, policy_version 1050650 (0.0007) [2023-12-27 12:56:06,735][20347] Updated weights for policy 0, policy_version 1051487 (0.0006) [2023-12-27 12:56:06,768][20381] Updated weights for policy 1, policy_version 1050660 (0.0008) [2023-12-27 12:56:06,830][20381] Updated weights for policy 1, policy_version 1050670 (0.0007) [2023-12-27 12:56:06,886][20381] Updated weights for policy 1, policy_version 1050680 (0.0008) [2023-12-27 12:56:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 538230784. Throughput: 0: 10297.0, 1: 10145.4. Samples: 538203496. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:07,364][19170] Avg episode reward: [(0, '912.170'), (1, '725.520')] [2023-12-27 12:56:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001050680_269000704.pth... [2023-12-27 12:56:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001051496_269230080.pth... [2023-12-27 12:56:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001049496_268697600.pth [2023-12-27 12:56:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001050312_268926976.pth [2023-12-27 12:56:07,394][20347] Updated weights for policy 0, policy_version 1051497 (0.0007) [2023-12-27 12:56:07,450][20347] Updated weights for policy 0, policy_version 1051507 (0.0008) [2023-12-27 12:56:07,507][20347] Updated weights for policy 0, policy_version 1051517 (0.0007) [2023-12-27 12:56:07,564][20347] Updated weights for policy 0, policy_version 1051527 (0.0006) [2023-12-27 12:56:07,566][20381] Updated weights for policy 1, policy_version 1050690 (0.0006) [2023-12-27 12:56:07,617][20381] Updated weights for policy 1, policy_version 1050700 (0.0007) [2023-12-27 12:56:07,677][20381] Updated weights for policy 1, policy_version 1050710 (0.0008) [2023-12-27 12:56:08,233][20347] Updated weights for policy 0, policy_version 1051537 (0.0007) [2023-12-27 12:56:08,285][20347] Updated weights for policy 0, policy_version 1051547 (0.0007) [2023-12-27 12:56:08,335][20347] Updated weights for policy 0, policy_version 1051557 (0.0007) [2023-12-27 12:56:08,376][20381] Updated weights for policy 1, policy_version 1050720 (0.0008) [2023-12-27 12:56:08,431][20381] Updated weights for policy 1, policy_version 1050730 (0.0007) [2023-12-27 12:56:08,493][20381] Updated weights for policy 1, policy_version 1050740 (0.0008) [2023-12-27 12:56:09,017][20347] Updated weights for policy 0, policy_version 1051567 (0.0007) [2023-12-27 12:56:09,065][20347] Updated weights for policy 0, policy_version 1051577 (0.0005) [2023-12-27 12:56:09,118][20347] Updated weights for policy 0, policy_version 1051587 (0.0005) [2023-12-27 12:56:09,193][20381] Updated weights for policy 1, policy_version 1050750 (0.0007) [2023-12-27 12:56:09,250][20381] Updated weights for policy 1, policy_version 1050760 (0.0009) [2023-12-27 12:56:09,307][20381] Updated weights for policy 1, policy_version 1050770 (0.0006) [2023-12-27 12:56:09,826][20347] Updated weights for policy 0, policy_version 1051597 (0.0007) [2023-12-27 12:56:09,889][20347] Updated weights for policy 0, policy_version 1051607 (0.0007) [2023-12-27 12:56:09,953][20347] Updated weights for policy 0, policy_version 1051617 (0.0008) [2023-12-27 12:56:10,063][20381] Updated weights for policy 1, policy_version 1050780 (0.0008) [2023-12-27 12:56:10,116][20381] Updated weights for policy 1, policy_version 1050790 (0.0006) [2023-12-27 12:56:10,164][20381] Updated weights for policy 1, policy_version 1050800 (0.0005) [2023-12-27 12:56:10,693][20347] Updated weights for policy 0, policy_version 1051627 (0.0007) [2023-12-27 12:56:10,751][20347] Updated weights for policy 0, policy_version 1051637 (0.0007) [2023-12-27 12:56:10,786][20381] Updated weights for policy 1, policy_version 1050810 (0.0006) [2023-12-27 12:56:10,807][20347] Updated weights for policy 0, policy_version 1051647 (0.0007) [2023-12-27 12:56:10,843][20381] Updated weights for policy 1, policy_version 1050820 (0.0006) [2023-12-27 12:56:10,899][20381] Updated weights for policy 1, policy_version 1050830 (0.0007) [2023-12-27 12:56:10,964][20381] Updated weights for policy 1, policy_version 1050840 (0.0006) [2023-12-27 12:56:11,567][20347] Updated weights for policy 0, policy_version 1051657 (0.0006) [2023-12-27 12:56:11,635][20347] Updated weights for policy 0, policy_version 1051667 (0.0007) [2023-12-27 12:56:11,702][20347] Updated weights for policy 0, policy_version 1051677 (0.0007) [2023-12-27 12:56:11,766][20347] Updated weights for policy 0, policy_version 1051687 (0.0008) [2023-12-27 12:56:11,787][20381] Updated weights for policy 1, policy_version 1050850 (0.0008) [2023-12-27 12:56:11,849][20381] Updated weights for policy 1, policy_version 1050860 (0.0006) [2023-12-27 12:56:11,908][20381] Updated weights for policy 1, policy_version 1050870 (0.0007) [2023-12-27 12:56:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 538329088. Throughput: 0: 10286.3, 1: 10124.8. Samples: 538323972. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:12,363][19170] Avg episode reward: [(0, '912.590'), (1, '666.840')] [2023-12-27 12:56:12,453][20347] Updated weights for policy 0, policy_version 1051697 (0.0009) [2023-12-27 12:56:12,509][20347] Updated weights for policy 0, policy_version 1051707 (0.0008) [2023-12-27 12:56:12,568][20347] Updated weights for policy 0, policy_version 1051717 (0.0007) [2023-12-27 12:56:12,592][20381] Updated weights for policy 1, policy_version 1050880 (0.0007) [2023-12-27 12:56:12,645][20381] Updated weights for policy 1, policy_version 1050890 (0.0008) [2023-12-27 12:56:12,703][20381] Updated weights for policy 1, policy_version 1050900 (0.0008) [2023-12-27 12:56:13,258][20347] Updated weights for policy 0, policy_version 1051727 (0.0008) [2023-12-27 12:56:13,314][20347] Updated weights for policy 0, policy_version 1051737 (0.0010) [2023-12-27 12:56:13,359][20347] Updated weights for policy 0, policy_version 1051747 (0.0010) [2023-12-27 12:56:13,375][20381] Updated weights for policy 1, policy_version 1050910 (0.0006) [2023-12-27 12:56:13,430][20381] Updated weights for policy 1, policy_version 1050920 (0.0007) [2023-12-27 12:56:13,477][20381] Updated weights for policy 1, policy_version 1050930 (0.0008) [2023-12-27 12:56:13,962][20347] Updated weights for policy 0, policy_version 1051757 (0.0009) [2023-12-27 12:56:14,021][20347] Updated weights for policy 0, policy_version 1051767 (0.0010) [2023-12-27 12:56:14,086][20347] Updated weights for policy 0, policy_version 1051777 (0.0010) [2023-12-27 12:56:14,198][20381] Updated weights for policy 1, policy_version 1050940 (0.0007) [2023-12-27 12:56:14,263][20381] Updated weights for policy 1, policy_version 1050950 (0.0007) [2023-12-27 12:56:14,325][20381] Updated weights for policy 1, policy_version 1050960 (0.0007) [2023-12-27 12:56:14,712][20347] Updated weights for policy 0, policy_version 1051787 (0.0009) [2023-12-27 12:56:14,766][20347] Updated weights for policy 0, policy_version 1051797 (0.0009) [2023-12-27 12:56:14,829][20347] Updated weights for policy 0, policy_version 1051807 (0.0007) [2023-12-27 12:56:14,997][20381] Updated weights for policy 1, policy_version 1050970 (0.0007) [2023-12-27 12:56:15,051][20381] Updated weights for policy 1, policy_version 1050980 (0.0008) [2023-12-27 12:56:15,109][20381] Updated weights for policy 1, policy_version 1050990 (0.0007) [2023-12-27 12:56:15,171][20381] Updated weights for policy 1, policy_version 1051000 (0.0008) [2023-12-27 12:56:15,550][20347] Updated weights for policy 0, policy_version 1051817 (0.0008) [2023-12-27 12:56:15,595][20347] Updated weights for policy 0, policy_version 1051827 (0.0007) [2023-12-27 12:56:15,646][20347] Updated weights for policy 0, policy_version 1051837 (0.0008) [2023-12-27 12:56:15,697][20347] Updated weights for policy 0, policy_version 1051847 (0.0008) [2023-12-27 12:56:15,859][20381] Updated weights for policy 1, policy_version 1051010 (0.0010) [2023-12-27 12:56:15,906][20381] Updated weights for policy 1, policy_version 1051020 (0.0010) [2023-12-27 12:56:15,951][20381] Updated weights for policy 1, policy_version 1051030 (0.0008) [2023-12-27 12:56:16,271][20347] Updated weights for policy 0, policy_version 1051857 (0.0006) [2023-12-27 12:56:16,319][20347] Updated weights for policy 0, policy_version 1051867 (0.0007) [2023-12-27 12:56:16,370][20347] Updated weights for policy 0, policy_version 1051877 (0.0006) [2023-12-27 12:56:16,574][20381] Updated weights for policy 1, policy_version 1051040 (0.0006) [2023-12-27 12:56:16,633][20381] Updated weights for policy 1, policy_version 1051050 (0.0006) [2023-12-27 12:56:16,689][20381] Updated weights for policy 1, policy_version 1051060 (0.0007) [2023-12-27 12:56:17,135][20347] Updated weights for policy 0, policy_version 1051887 (0.0007) [2023-12-27 12:56:17,181][20347] Updated weights for policy 0, policy_version 1051897 (0.0008) [2023-12-27 12:56:17,230][20347] Updated weights for policy 0, policy_version 1051907 (0.0008) [2023-12-27 12:56:17,315][20381] Updated weights for policy 1, policy_version 1051070 (0.0008) [2023-12-27 12:56:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20605.2). Total num frames: 538435584. Throughput: 0: 10245.7, 1: 10179.1. Samples: 538447512. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:17,363][19170] Avg episode reward: [(0, '903.660'), (1, '696.030')] [2023-12-27 12:56:17,372][20381] Updated weights for policy 1, policy_version 1051080 (0.0009) [2023-12-27 12:56:17,424][20381] Updated weights for policy 1, policy_version 1051090 (0.0009) [2023-12-27 12:56:17,903][20347] Updated weights for policy 0, policy_version 1051917 (0.0008) [2023-12-27 12:56:17,967][20347] Updated weights for policy 0, policy_version 1051927 (0.0007) [2023-12-27 12:56:18,030][20347] Updated weights for policy 0, policy_version 1051937 (0.0006) [2023-12-27 12:56:18,108][20381] Updated weights for policy 1, policy_version 1051100 (0.0009) [2023-12-27 12:56:18,156][20381] Updated weights for policy 1, policy_version 1051110 (0.0010) [2023-12-27 12:56:18,208][20381] Updated weights for policy 1, policy_version 1051120 (0.0010) [2023-12-27 12:56:18,671][20347] Updated weights for policy 0, policy_version 1051947 (0.0008) [2023-12-27 12:56:18,732][20347] Updated weights for policy 0, policy_version 1051957 (0.0010) [2023-12-27 12:56:18,787][20347] Updated weights for policy 0, policy_version 1051967 (0.0007) [2023-12-27 12:56:18,865][20381] Updated weights for policy 1, policy_version 1051130 (0.0009) [2023-12-27 12:56:18,930][20381] Updated weights for policy 1, policy_version 1051140 (0.0005) [2023-12-27 12:56:18,992][20381] Updated weights for policy 1, policy_version 1051150 (0.0005) [2023-12-27 12:56:19,048][20381] Updated weights for policy 1, policy_version 1051160 (0.0006) [2023-12-27 12:56:19,394][20347] Updated weights for policy 0, policy_version 1051977 (0.0006) [2023-12-27 12:56:19,449][20347] Updated weights for policy 0, policy_version 1051987 (0.0008) [2023-12-27 12:56:19,512][20347] Updated weights for policy 0, policy_version 1051997 (0.0008) [2023-12-27 12:56:19,572][20347] Updated weights for policy 0, policy_version 1052007 (0.0008) [2023-12-27 12:56:19,679][20381] Updated weights for policy 1, policy_version 1051170 (0.0007) [2023-12-27 12:56:19,751][20381] Updated weights for policy 1, policy_version 1051180 (0.0008) [2023-12-27 12:56:19,814][20381] Updated weights for policy 1, policy_version 1051190 (0.0007) [2023-12-27 12:56:20,291][20347] Updated weights for policy 0, policy_version 1052017 (0.0010) [2023-12-27 12:56:20,351][20347] Updated weights for policy 0, policy_version 1052027 (0.0009) [2023-12-27 12:56:20,411][20347] Updated weights for policy 0, policy_version 1052037 (0.0007) [2023-12-27 12:56:20,479][20381] Updated weights for policy 1, policy_version 1051200 (0.0007) [2023-12-27 12:56:20,531][20381] Updated weights for policy 1, policy_version 1051210 (0.0008) [2023-12-27 12:56:20,590][20381] Updated weights for policy 1, policy_version 1051220 (0.0007) [2023-12-27 12:56:21,099][20347] Updated weights for policy 0, policy_version 1052047 (0.0010) [2023-12-27 12:56:21,163][20347] Updated weights for policy 0, policy_version 1052057 (0.0009) [2023-12-27 12:56:21,217][20347] Updated weights for policy 0, policy_version 1052067 (0.0007) [2023-12-27 12:56:21,295][20381] Updated weights for policy 1, policy_version 1051230 (0.0008) [2023-12-27 12:56:21,354][20381] Updated weights for policy 1, policy_version 1051240 (0.0008) [2023-12-27 12:56:21,425][20381] Updated weights for policy 1, policy_version 1051250 (0.0008) [2023-12-27 12:56:21,942][20347] Updated weights for policy 0, policy_version 1052077 (0.0006) [2023-12-27 12:56:22,003][20347] Updated weights for policy 0, policy_version 1052087 (0.0008) [2023-12-27 12:56:22,064][20347] Updated weights for policy 0, policy_version 1052097 (0.0008) [2023-12-27 12:56:22,162][20381] Updated weights for policy 1, policy_version 1051260 (0.0008) [2023-12-27 12:56:22,224][20381] Updated weights for policy 1, policy_version 1051270 (0.0008) [2023-12-27 12:56:22,289][20381] Updated weights for policy 1, policy_version 1051280 (0.0007) [2023-12-27 12:56:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 538542080. Throughput: 0: 10271.2, 1: 10184.7. Samples: 538510168. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:22,363][19170] Avg episode reward: [(0, '913.620'), (1, '715.870')] [2023-12-27 12:56:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001051288_269156352.pth... [2023-12-27 12:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001050104_268853248.pth [2023-12-27 12:56:22,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001052104_269385728.pth... [2023-12-27 12:56:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001050888_269074432.pth [2023-12-27 12:56:22,802][20347] Updated weights for policy 0, policy_version 1052107 (0.0008) [2023-12-27 12:56:22,857][20347] Updated weights for policy 0, policy_version 1052117 (0.0008) [2023-12-27 12:56:22,911][20381] Updated weights for policy 1, policy_version 1051290 (0.0006) [2023-12-27 12:56:22,916][20347] Updated weights for policy 0, policy_version 1052127 (0.0008) [2023-12-27 12:56:22,966][20381] Updated weights for policy 1, policy_version 1051300 (0.0006) [2023-12-27 12:56:23,018][20381] Updated weights for policy 1, policy_version 1051310 (0.0007) [2023-12-27 12:56:23,074][20381] Updated weights for policy 1, policy_version 1051320 (0.0007) [2023-12-27 12:56:23,587][20347] Updated weights for policy 0, policy_version 1052137 (0.0006) [2023-12-27 12:56:23,633][20347] Updated weights for policy 0, policy_version 1052147 (0.0010) [2023-12-27 12:56:23,675][20381] Updated weights for policy 1, policy_version 1051330 (0.0010) [2023-12-27 12:56:23,682][20347] Updated weights for policy 0, policy_version 1052157 (0.0010) [2023-12-27 12:56:23,728][20381] Updated weights for policy 1, policy_version 1051340 (0.0007) [2023-12-27 12:56:23,737][20347] Updated weights for policy 0, policy_version 1052167 (0.0010) [2023-12-27 12:56:23,785][20381] Updated weights for policy 1, policy_version 1051350 (0.0005) [2023-12-27 12:56:24,331][20381] Updated weights for policy 1, policy_version 1051360 (0.0005) [2023-12-27 12:56:24,390][20381] Updated weights for policy 1, policy_version 1051370 (0.0009) [2023-12-27 12:56:24,413][20347] Updated weights for policy 0, policy_version 1052177 (0.0010) [2023-12-27 12:56:24,448][20381] Updated weights for policy 1, policy_version 1051380 (0.0010) [2023-12-27 12:56:24,462][20347] Updated weights for policy 0, policy_version 1052187 (0.0010) [2023-12-27 12:56:24,519][20347] Updated weights for policy 0, policy_version 1052197 (0.0010) [2023-12-27 12:56:25,109][20381] Updated weights for policy 1, policy_version 1051390 (0.0007) [2023-12-27 12:56:25,146][20347] Updated weights for policy 0, policy_version 1052207 (0.0007) [2023-12-27 12:56:25,159][20381] Updated weights for policy 1, policy_version 1051400 (0.0007) [2023-12-27 12:56:25,198][20347] Updated weights for policy 0, policy_version 1052217 (0.0006) [2023-12-27 12:56:25,208][20381] Updated weights for policy 1, policy_version 1051410 (0.0005) [2023-12-27 12:56:25,250][20347] Updated weights for policy 0, policy_version 1052227 (0.0006) [2023-12-27 12:56:25,841][20381] Updated weights for policy 1, policy_version 1051420 (0.0006) [2023-12-27 12:56:25,848][20347] Updated weights for policy 0, policy_version 1052237 (0.0009) [2023-12-27 12:56:25,893][20381] Updated weights for policy 1, policy_version 1051430 (0.0006) [2023-12-27 12:56:25,895][20347] Updated weights for policy 0, policy_version 1052247 (0.0006) [2023-12-27 12:56:25,940][20347] Updated weights for policy 0, policy_version 1052257 (0.0006) [2023-12-27 12:56:25,948][20381] Updated weights for policy 1, policy_version 1051440 (0.0008) [2023-12-27 12:56:26,524][20347] Updated weights for policy 0, policy_version 1052267 (0.0007) [2023-12-27 12:56:26,574][20347] Updated weights for policy 0, policy_version 1052277 (0.0007) [2023-12-27 12:56:26,627][20347] Updated weights for policy 0, policy_version 1052287 (0.0007) [2023-12-27 12:56:26,666][20381] Updated weights for policy 1, policy_version 1051450 (0.0007) [2023-12-27 12:56:26,726][20381] Updated weights for policy 1, policy_version 1051460 (0.0005) [2023-12-27 12:56:26,787][20381] Updated weights for policy 1, policy_version 1051470 (0.0005) [2023-12-27 12:56:26,841][20381] Updated weights for policy 1, policy_version 1051480 (0.0005) [2023-12-27 12:56:27,258][20347] Updated weights for policy 0, policy_version 1052297 (0.0007) [2023-12-27 12:56:27,312][20347] Updated weights for policy 0, policy_version 1052307 (0.0009) [2023-12-27 12:56:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 538640384. Throughput: 0: 10235.0, 1: 10255.5. Samples: 538633728. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:27,363][19170] Avg episode reward: [(0, '912.640'), (1, '744.800')] [2023-12-27 12:56:27,371][20347] Updated weights for policy 0, policy_version 1052317 (0.0009) [2023-12-27 12:56:27,397][20381] Updated weights for policy 1, policy_version 1051490 (0.0007) [2023-12-27 12:56:27,427][20347] Updated weights for policy 0, policy_version 1052327 (0.0005) [2023-12-27 12:56:27,450][20381] Updated weights for policy 1, policy_version 1051500 (0.0007) [2023-12-27 12:56:27,496][20381] Updated weights for policy 1, policy_version 1051510 (0.0006) [2023-12-27 12:56:28,074][20347] Updated weights for policy 0, policy_version 1052337 (0.0008) [2023-12-27 12:56:28,119][20381] Updated weights for policy 1, policy_version 1051520 (0.0006) [2023-12-27 12:56:28,125][20347] Updated weights for policy 0, policy_version 1052347 (0.0008) [2023-12-27 12:56:28,183][20381] Updated weights for policy 1, policy_version 1051530 (0.0008) [2023-12-27 12:56:28,186][20347] Updated weights for policy 0, policy_version 1052357 (0.0008) [2023-12-27 12:56:28,234][20381] Updated weights for policy 1, policy_version 1051540 (0.0010) [2023-12-27 12:56:28,837][20381] Updated weights for policy 1, policy_version 1051550 (0.0009) [2023-12-27 12:56:28,884][20381] Updated weights for policy 1, policy_version 1051560 (0.0005) [2023-12-27 12:56:28,893][20347] Updated weights for policy 0, policy_version 1052367 (0.0008) [2023-12-27 12:56:28,937][20381] Updated weights for policy 1, policy_version 1051570 (0.0006) [2023-12-27 12:56:28,945][20347] Updated weights for policy 0, policy_version 1052377 (0.0007) [2023-12-27 12:56:29,000][20347] Updated weights for policy 0, policy_version 1052387 (0.0008) [2023-12-27 12:56:29,632][20381] Updated weights for policy 1, policy_version 1051580 (0.0006) [2023-12-27 12:56:29,695][20381] Updated weights for policy 1, policy_version 1051590 (0.0005) [2023-12-27 12:56:29,732][20347] Updated weights for policy 0, policy_version 1052397 (0.0007) [2023-12-27 12:56:29,761][20381] Updated weights for policy 1, policy_version 1051600 (0.0005) [2023-12-27 12:56:29,792][20347] Updated weights for policy 0, policy_version 1052407 (0.0006) [2023-12-27 12:56:29,857][20347] Updated weights for policy 0, policy_version 1052417 (0.0008) [2023-12-27 12:56:30,410][20381] Updated weights for policy 1, policy_version 1051610 (0.0007) [2023-12-27 12:56:30,457][20381] Updated weights for policy 1, policy_version 1051620 (0.0008) [2023-12-27 12:56:30,481][20347] Updated weights for policy 0, policy_version 1052427 (0.0006) [2023-12-27 12:56:30,512][20381] Updated weights for policy 1, policy_version 1051630 (0.0009) [2023-12-27 12:56:30,537][20347] Updated weights for policy 0, policy_version 1052437 (0.0005) [2023-12-27 12:56:30,562][20381] Updated weights for policy 1, policy_version 1051640 (0.0010) [2023-12-27 12:56:30,594][20347] Updated weights for policy 0, policy_version 1052447 (0.0005) [2023-12-27 12:56:31,177][20347] Updated weights for policy 0, policy_version 1052457 (0.0005) [2023-12-27 12:56:31,235][20381] Updated weights for policy 1, policy_version 1051650 (0.0010) [2023-12-27 12:56:31,243][20347] Updated weights for policy 0, policy_version 1052467 (0.0005) [2023-12-27 12:56:31,298][20381] Updated weights for policy 1, policy_version 1051660 (0.0011) [2023-12-27 12:56:31,304][20347] Updated weights for policy 0, policy_version 1052477 (0.0006) [2023-12-27 12:56:31,356][20381] Updated weights for policy 1, policy_version 1051670 (0.0012) [2023-12-27 12:56:31,368][20347] Updated weights for policy 0, policy_version 1052487 (0.0007) [2023-12-27 12:56:32,008][20347] Updated weights for policy 0, policy_version 1052497 (0.0010) [2023-12-27 12:56:32,020][20381] Updated weights for policy 1, policy_version 1051680 (0.0010) [2023-12-27 12:56:32,060][20347] Updated weights for policy 0, policy_version 1052507 (0.0010) [2023-12-27 12:56:32,072][20381] Updated weights for policy 1, policy_version 1051690 (0.0011) [2023-12-27 12:56:32,123][20347] Updated weights for policy 0, policy_version 1052517 (0.0011) [2023-12-27 12:56:32,128][20381] Updated weights for policy 1, policy_version 1051700 (0.0010) [2023-12-27 12:56:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 538755072. Throughput: 0: 10343.9, 1: 10293.0. Samples: 538761876. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:32,363][19170] Avg episode reward: [(0, '912.540'), (1, '725.710')] [2023-12-27 12:56:32,817][20347] Updated weights for policy 0, policy_version 1052527 (0.0007) [2023-12-27 12:56:32,851][20381] Updated weights for policy 1, policy_version 1051710 (0.0007) [2023-12-27 12:56:32,878][20347] Updated weights for policy 0, policy_version 1052537 (0.0005) [2023-12-27 12:56:32,908][20381] Updated weights for policy 1, policy_version 1051720 (0.0005) [2023-12-27 12:56:32,936][20347] Updated weights for policy 0, policy_version 1052547 (0.0005) [2023-12-27 12:56:32,965][20381] Updated weights for policy 1, policy_version 1051730 (0.0005) [2023-12-27 12:56:33,522][20381] Updated weights for policy 1, policy_version 1051740 (0.0005) [2023-12-27 12:56:33,538][20347] Updated weights for policy 0, policy_version 1052557 (0.0006) [2023-12-27 12:56:33,574][20381] Updated weights for policy 1, policy_version 1051750 (0.0005) [2023-12-27 12:56:33,595][20347] Updated weights for policy 0, policy_version 1052567 (0.0006) [2023-12-27 12:56:33,636][20381] Updated weights for policy 1, policy_version 1051760 (0.0006) [2023-12-27 12:56:33,642][20347] Updated weights for policy 0, policy_version 1052577 (0.0007) [2023-12-27 12:56:34,256][20381] Updated weights for policy 1, policy_version 1051770 (0.0006) [2023-12-27 12:56:34,310][20381] Updated weights for policy 1, policy_version 1051780 (0.0007) [2023-12-27 12:56:34,373][20381] Updated weights for policy 1, policy_version 1051790 (0.0011) [2023-12-27 12:56:34,391][20347] Updated weights for policy 0, policy_version 1052587 (0.0007) [2023-12-27 12:56:34,433][20381] Updated weights for policy 1, policy_version 1051800 (0.0011) [2023-12-27 12:56:34,458][20347] Updated weights for policy 0, policy_version 1052597 (0.0008) [2023-12-27 12:56:34,510][20347] Updated weights for policy 0, policy_version 1052607 (0.0008) [2023-12-27 12:56:35,119][20381] Updated weights for policy 1, policy_version 1051810 (0.0007) [2023-12-27 12:56:35,145][20347] Updated weights for policy 0, policy_version 1052617 (0.0008) [2023-12-27 12:56:35,168][20381] Updated weights for policy 1, policy_version 1051820 (0.0010) [2023-12-27 12:56:35,203][20347] Updated weights for policy 0, policy_version 1052627 (0.0006) [2023-12-27 12:56:35,214][20381] Updated weights for policy 1, policy_version 1051830 (0.0009) [2023-12-27 12:56:35,255][20347] Updated weights for policy 0, policy_version 1052637 (0.0008) [2023-12-27 12:56:35,306][20347] Updated weights for policy 0, policy_version 1052647 (0.0007) [2023-12-27 12:56:35,831][20381] Updated weights for policy 1, policy_version 1051840 (0.0007) [2023-12-27 12:56:35,889][20381] Updated weights for policy 1, policy_version 1051850 (0.0007) [2023-12-27 12:56:35,948][20381] Updated weights for policy 1, policy_version 1051860 (0.0008) [2023-12-27 12:56:36,013][20347] Updated weights for policy 0, policy_version 1052657 (0.0007) [2023-12-27 12:56:36,059][20347] Updated weights for policy 0, policy_version 1052667 (0.0008) [2023-12-27 12:56:36,109][20347] Updated weights for policy 0, policy_version 1052677 (0.0008) [2023-12-27 12:56:36,686][20381] Updated weights for policy 1, policy_version 1051870 (0.0008) [2023-12-27 12:56:36,739][20381] Updated weights for policy 1, policy_version 1051880 (0.0006) [2023-12-27 12:56:36,796][20381] Updated weights for policy 1, policy_version 1051890 (0.0007) [2023-12-27 12:56:36,841][20347] Updated weights for policy 0, policy_version 1052687 (0.0008) [2023-12-27 12:56:36,893][20347] Updated weights for policy 0, policy_version 1052697 (0.0008) [2023-12-27 12:56:36,941][20347] Updated weights for policy 0, policy_version 1052707 (0.0007) [2023-12-27 12:56:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20632.7). Total num frames: 538853376. Throughput: 0: 10376.3, 1: 10329.5. Samples: 538825188. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:37,363][19170] Avg episode reward: [(0, '951.230'), (1, '765.030')] [2023-12-27 12:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001051896_269312000.pth... [2023-12-27 12:56:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001052712_269541376.pth... [2023-12-27 12:56:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001050680_269000704.pth [2023-12-27 12:56:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001051496_269230080.pth [2023-12-27 12:56:37,410][20381] Updated weights for policy 1, policy_version 1051900 (0.0007) [2023-12-27 12:56:37,462][20381] Updated weights for policy 1, policy_version 1051910 (0.0007) [2023-12-27 12:56:37,514][20381] Updated weights for policy 1, policy_version 1051920 (0.0010) [2023-12-27 12:56:37,620][20347] Updated weights for policy 0, policy_version 1052717 (0.0007) [2023-12-27 12:56:37,666][20347] Updated weights for policy 0, policy_version 1052727 (0.0005) [2023-12-27 12:56:37,718][20347] Updated weights for policy 0, policy_version 1052737 (0.0006) [2023-12-27 12:56:38,128][20381] Updated weights for policy 1, policy_version 1051930 (0.0009) [2023-12-27 12:56:38,187][20381] Updated weights for policy 1, policy_version 1051940 (0.0005) [2023-12-27 12:56:38,243][20381] Updated weights for policy 1, policy_version 1051950 (0.0006) [2023-12-27 12:56:38,299][20381] Updated weights for policy 1, policy_version 1051960 (0.0007) [2023-12-27 12:56:38,331][20347] Updated weights for policy 0, policy_version 1052747 (0.0006) [2023-12-27 12:56:38,389][20347] Updated weights for policy 0, policy_version 1052757 (0.0008) [2023-12-27 12:56:38,448][20347] Updated weights for policy 0, policy_version 1052767 (0.0008) [2023-12-27 12:56:38,885][20381] Updated weights for policy 1, policy_version 1051970 (0.0011) [2023-12-27 12:56:38,943][20381] Updated weights for policy 1, policy_version 1051980 (0.0011) [2023-12-27 12:56:38,998][20381] Updated weights for policy 1, policy_version 1051990 (0.0010) [2023-12-27 12:56:39,183][20347] Updated weights for policy 0, policy_version 1052777 (0.0008) [2023-12-27 12:56:39,245][20347] Updated weights for policy 0, policy_version 1052787 (0.0008) [2023-12-27 12:56:39,305][20347] Updated weights for policy 0, policy_version 1052797 (0.0007) [2023-12-27 12:56:39,373][20347] Updated weights for policy 0, policy_version 1052807 (0.0008) [2023-12-27 12:56:39,741][20381] Updated weights for policy 1, policy_version 1052000 (0.0011) [2023-12-27 12:56:39,798][20381] Updated weights for policy 1, policy_version 1052010 (0.0011) [2023-12-27 12:56:39,862][20381] Updated weights for policy 1, policy_version 1052020 (0.0008) [2023-12-27 12:56:40,101][20347] Updated weights for policy 0, policy_version 1052817 (0.0007) [2023-12-27 12:56:40,164][20347] Updated weights for policy 0, policy_version 1052827 (0.0007) [2023-12-27 12:56:40,223][20347] Updated weights for policy 0, policy_version 1052837 (0.0008) [2023-12-27 12:56:40,613][20381] Updated weights for policy 1, policy_version 1052030 (0.0009) [2023-12-27 12:56:40,665][20381] Updated weights for policy 1, policy_version 1052040 (0.0011) [2023-12-27 12:56:40,713][20381] Updated weights for policy 1, policy_version 1052050 (0.0009) [2023-12-27 12:56:40,912][20347] Updated weights for policy 0, policy_version 1052847 (0.0007) [2023-12-27 12:56:40,978][20347] Updated weights for policy 0, policy_version 1052857 (0.0008) [2023-12-27 12:56:41,043][20347] Updated weights for policy 0, policy_version 1052867 (0.0007) [2023-12-27 12:56:41,418][20381] Updated weights for policy 1, policy_version 1052060 (0.0006) [2023-12-27 12:56:41,478][20381] Updated weights for policy 1, policy_version 1052070 (0.0008) [2023-12-27 12:56:41,539][20381] Updated weights for policy 1, policy_version 1052080 (0.0008) [2023-12-27 12:56:41,815][20347] Updated weights for policy 0, policy_version 1052877 (0.0007) [2023-12-27 12:56:41,878][20347] Updated weights for policy 0, policy_version 1052887 (0.0007) [2023-12-27 12:56:41,940][20347] Updated weights for policy 0, policy_version 1052897 (0.0006) [2023-12-27 12:56:42,209][20381] Updated weights for policy 1, policy_version 1052090 (0.0008) [2023-12-27 12:56:42,268][20381] Updated weights for policy 1, policy_version 1052100 (0.0006) [2023-12-27 12:56:42,327][20381] Updated weights for policy 1, policy_version 1052110 (0.0007) [2023-12-27 12:56:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 538951680. Throughput: 0: 10324.8, 1: 10328.3. Samples: 538947476. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:42,363][19170] Avg episode reward: [(0, '931.590'), (1, '716.230')] [2023-12-27 12:56:42,392][20381] Updated weights for policy 1, policy_version 1052120 (0.0008) [2023-12-27 12:56:42,588][20347] Updated weights for policy 0, policy_version 1052907 (0.0006) [2023-12-27 12:56:42,645][20347] Updated weights for policy 0, policy_version 1052917 (0.0005) [2023-12-27 12:56:42,704][20347] Updated weights for policy 0, policy_version 1052927 (0.0005) [2023-12-27 12:56:43,110][20381] Updated weights for policy 1, policy_version 1052130 (0.0006) [2023-12-27 12:56:43,159][20381] Updated weights for policy 1, policy_version 1052140 (0.0008) [2023-12-27 12:56:43,219][20381] Updated weights for policy 1, policy_version 1052150 (0.0008) [2023-12-27 12:56:43,283][20347] Updated weights for policy 0, policy_version 1052937 (0.0006) [2023-12-27 12:56:43,339][20347] Updated weights for policy 0, policy_version 1052947 (0.0007) [2023-12-27 12:56:43,392][20347] Updated weights for policy 0, policy_version 1052957 (0.0007) [2023-12-27 12:56:43,442][20347] Updated weights for policy 0, policy_version 1052967 (0.0007) [2023-12-27 12:56:43,889][20381] Updated weights for policy 1, policy_version 1052160 (0.0008) [2023-12-27 12:56:43,950][20381] Updated weights for policy 1, policy_version 1052170 (0.0005) [2023-12-27 12:56:44,001][20381] Updated weights for policy 1, policy_version 1052180 (0.0005) [2023-12-27 12:56:44,047][20347] Updated weights for policy 0, policy_version 1052977 (0.0007) [2023-12-27 12:56:44,098][20347] Updated weights for policy 0, policy_version 1052987 (0.0006) [2023-12-27 12:56:44,157][20347] Updated weights for policy 0, policy_version 1052997 (0.0008) [2023-12-27 12:56:44,589][20381] Updated weights for policy 1, policy_version 1052190 (0.0006) [2023-12-27 12:56:44,640][20381] Updated weights for policy 1, policy_version 1052200 (0.0005) [2023-12-27 12:56:44,695][20381] Updated weights for policy 1, policy_version 1052210 (0.0007) [2023-12-27 12:56:44,850][20347] Updated weights for policy 0, policy_version 1053007 (0.0008) [2023-12-27 12:56:44,913][20347] Updated weights for policy 0, policy_version 1053017 (0.0008) [2023-12-27 12:56:44,968][20347] Updated weights for policy 0, policy_version 1053027 (0.0007) [2023-12-27 12:56:45,366][20381] Updated weights for policy 1, policy_version 1052220 (0.0007) [2023-12-27 12:56:45,428][20381] Updated weights for policy 1, policy_version 1052230 (0.0007) [2023-12-27 12:56:45,492][20381] Updated weights for policy 1, policy_version 1052240 (0.0009) [2023-12-27 12:56:45,683][20347] Updated weights for policy 0, policy_version 1053037 (0.0007) [2023-12-27 12:56:45,742][20347] Updated weights for policy 0, policy_version 1053047 (0.0006) [2023-12-27 12:56:45,795][20347] Updated weights for policy 0, policy_version 1053057 (0.0009) [2023-12-27 12:56:46,173][20381] Updated weights for policy 1, policy_version 1052250 (0.0008) [2023-12-27 12:56:46,230][20381] Updated weights for policy 1, policy_version 1052260 (0.0010) [2023-12-27 12:56:46,284][20381] Updated weights for policy 1, policy_version 1052270 (0.0010) [2023-12-27 12:56:46,342][20381] Updated weights for policy 1, policy_version 1052280 (0.0010) [2023-12-27 12:56:46,434][20347] Updated weights for policy 0, policy_version 1053067 (0.0010) [2023-12-27 12:56:46,488][20347] Updated weights for policy 0, policy_version 1053077 (0.0010) [2023-12-27 12:56:46,542][20347] Updated weights for policy 0, policy_version 1053087 (0.0010) [2023-12-27 12:56:46,991][20381] Updated weights for policy 1, policy_version 1052290 (0.0007) [2023-12-27 12:56:47,041][20381] Updated weights for policy 1, policy_version 1052300 (0.0008) [2023-12-27 12:56:47,090][20381] Updated weights for policy 1, policy_version 1052310 (0.0007) [2023-12-27 12:56:47,227][20347] Updated weights for policy 0, policy_version 1053097 (0.0010) [2023-12-27 12:56:47,282][20347] Updated weights for policy 0, policy_version 1053107 (0.0005) [2023-12-27 12:56:47,340][20347] Updated weights for policy 0, policy_version 1053117 (0.0006) [2023-12-27 12:56:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 539058176. Throughput: 0: 10307.7, 1: 10350.5. Samples: 539070800. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:47,363][19170] Avg episode reward: [(0, '941.390'), (1, '716.150')] [2023-12-27 12:56:47,399][20347] Updated weights for policy 0, policy_version 1053127 (0.0006) [2023-12-27 12:56:47,811][20381] Updated weights for policy 1, policy_version 1052320 (0.0006) [2023-12-27 12:56:47,857][20381] Updated weights for policy 1, policy_version 1052330 (0.0005) [2023-12-27 12:56:47,908][20381] Updated weights for policy 1, policy_version 1052340 (0.0005) [2023-12-27 12:56:48,026][20347] Updated weights for policy 0, policy_version 1053137 (0.0007) [2023-12-27 12:56:48,080][20347] Updated weights for policy 0, policy_version 1053147 (0.0007) [2023-12-27 12:56:48,138][20347] Updated weights for policy 0, policy_version 1053157 (0.0007) [2023-12-27 12:56:48,609][20381] Updated weights for policy 1, policy_version 1052350 (0.0010) [2023-12-27 12:56:48,671][20381] Updated weights for policy 1, policy_version 1052360 (0.0011) [2023-12-27 12:56:48,721][20381] Updated weights for policy 1, policy_version 1052370 (0.0007) [2023-12-27 12:56:48,813][20347] Updated weights for policy 0, policy_version 1053167 (0.0007) [2023-12-27 12:56:48,870][20347] Updated weights for policy 0, policy_version 1053177 (0.0007) [2023-12-27 12:56:48,929][20347] Updated weights for policy 0, policy_version 1053187 (0.0007) [2023-12-27 12:56:49,458][20381] Updated weights for policy 1, policy_version 1052380 (0.0008) [2023-12-27 12:56:49,520][20381] Updated weights for policy 1, policy_version 1052390 (0.0007) [2023-12-27 12:56:49,573][20381] Updated weights for policy 1, policy_version 1052400 (0.0008) [2023-12-27 12:56:49,612][20347] Updated weights for policy 0, policy_version 1053197 (0.0007) [2023-12-27 12:56:49,667][20347] Updated weights for policy 0, policy_version 1053207 (0.0007) [2023-12-27 12:56:49,724][20347] Updated weights for policy 0, policy_version 1053217 (0.0007) [2023-12-27 12:56:50,207][20381] Updated weights for policy 1, policy_version 1052410 (0.0007) [2023-12-27 12:56:50,267][20381] Updated weights for policy 1, policy_version 1052420 (0.0007) [2023-12-27 12:56:50,326][20381] Updated weights for policy 1, policy_version 1052430 (0.0008) [2023-12-27 12:56:50,388][20381] Updated weights for policy 1, policy_version 1052440 (0.0007) [2023-12-27 12:56:50,481][20347] Updated weights for policy 0, policy_version 1053227 (0.0007) [2023-12-27 12:56:50,533][20347] Updated weights for policy 0, policy_version 1053237 (0.0008) [2023-12-27 12:56:50,600][20347] Updated weights for policy 0, policy_version 1053247 (0.0007) [2023-12-27 12:56:51,102][20381] Updated weights for policy 1, policy_version 1052450 (0.0008) [2023-12-27 12:56:51,160][20381] Updated weights for policy 1, policy_version 1052460 (0.0008) [2023-12-27 12:56:51,213][20381] Updated weights for policy 1, policy_version 1052470 (0.0008) [2023-12-27 12:56:51,257][20347] Updated weights for policy 0, policy_version 1053257 (0.0008) [2023-12-27 12:56:51,313][20347] Updated weights for policy 0, policy_version 1053267 (0.0008) [2023-12-27 12:56:51,379][20347] Updated weights for policy 0, policy_version 1053277 (0.0008) [2023-12-27 12:56:51,438][20347] Updated weights for policy 0, policy_version 1053287 (0.0006) [2023-12-27 12:56:51,982][20381] Updated weights for policy 1, policy_version 1052480 (0.0008) [2023-12-27 12:56:52,003][20292] KL-divergence is very high: 110.0055 [2023-12-27 12:56:52,045][20381] Updated weights for policy 1, policy_version 1052490 (0.0008) [2023-12-27 12:56:52,054][20292] KL-divergence is very high: 135.0231 [2023-12-27 12:56:52,106][20381] Updated weights for policy 1, policy_version 1052500 (0.0006) [2023-12-27 12:56:52,107][20347] Updated weights for policy 0, policy_version 1053297 (0.0007) [2023-12-27 12:56:52,165][20347] Updated weights for policy 0, policy_version 1053307 (0.0007) [2023-12-27 12:56:52,224][20347] Updated weights for policy 0, policy_version 1053317 (0.0007) [2023-12-27 12:56:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 539164672. Throughput: 0: 10303.8, 1: 10328.9. Samples: 539131964. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:52,363][19170] Avg episode reward: [(0, '951.160'), (1, '715.900')] [2023-12-27 12:56:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001052504_269467648.pth... [2023-12-27 12:56:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001053320_269697024.pth... [2023-12-27 12:56:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001051288_269156352.pth [2023-12-27 12:56:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001052104_269385728.pth [2023-12-27 12:56:52,809][20381] Updated weights for policy 1, policy_version 1052510 (0.0007) [2023-12-27 12:56:52,871][20381] Updated weights for policy 1, policy_version 1052520 (0.0008) [2023-12-27 12:56:52,924][20381] Updated weights for policy 1, policy_version 1052530 (0.0008) [2023-12-27 12:56:52,930][20347] Updated weights for policy 0, policy_version 1053327 (0.0006) [2023-12-27 12:56:52,986][20347] Updated weights for policy 0, policy_version 1053337 (0.0006) [2023-12-27 12:56:53,041][20347] Updated weights for policy 0, policy_version 1053347 (0.0009) [2023-12-27 12:56:53,614][20381] Updated weights for policy 1, policy_version 1052540 (0.0006) [2023-12-27 12:56:53,664][20381] Updated weights for policy 1, policy_version 1052550 (0.0008) [2023-12-27 12:56:53,715][20381] Updated weights for policy 1, policy_version 1052560 (0.0007) [2023-12-27 12:56:53,750][20347] Updated weights for policy 0, policy_version 1053357 (0.0009) [2023-12-27 12:56:53,799][20347] Updated weights for policy 0, policy_version 1053367 (0.0006) [2023-12-27 12:56:53,846][20347] Updated weights for policy 0, policy_version 1053377 (0.0007) [2023-12-27 12:56:54,428][20381] Updated weights for policy 1, policy_version 1052570 (0.0009) [2023-12-27 12:56:54,477][20381] Updated weights for policy 1, policy_version 1052580 (0.0010) [2023-12-27 12:56:54,525][20381] Updated weights for policy 1, policy_version 1052590 (0.0010) [2023-12-27 12:56:54,550][20347] Updated weights for policy 0, policy_version 1053387 (0.0008) [2023-12-27 12:56:54,584][20381] Updated weights for policy 1, policy_version 1052600 (0.0011) [2023-12-27 12:56:54,608][20347] Updated weights for policy 0, policy_version 1053397 (0.0006) [2023-12-27 12:56:54,674][20347] Updated weights for policy 0, policy_version 1053407 (0.0007) [2023-12-27 12:56:55,258][20381] Updated weights for policy 1, policy_version 1052610 (0.0008) [2023-12-27 12:56:55,306][20347] Updated weights for policy 0, policy_version 1053417 (0.0007) [2023-12-27 12:56:55,309][20381] Updated weights for policy 1, policy_version 1052620 (0.0005) [2023-12-27 12:56:55,352][20381] Updated weights for policy 1, policy_version 1052630 (0.0007) [2023-12-27 12:56:55,352][20347] Updated weights for policy 0, policy_version 1053427 (0.0005) [2023-12-27 12:56:55,404][20347] Updated weights for policy 0, policy_version 1053437 (0.0005) [2023-12-27 12:56:55,450][20347] Updated weights for policy 0, policy_version 1053447 (0.0005) [2023-12-27 12:56:55,955][20381] Updated weights for policy 1, policy_version 1052640 (0.0007) [2023-12-27 12:56:55,980][20347] Updated weights for policy 0, policy_version 1053457 (0.0007) [2023-12-27 12:56:56,006][20381] Updated weights for policy 1, policy_version 1052650 (0.0008) [2023-12-27 12:56:56,032][20347] Updated weights for policy 0, policy_version 1053467 (0.0009) [2023-12-27 12:56:56,056][20381] Updated weights for policy 1, policy_version 1052660 (0.0010) [2023-12-27 12:56:56,081][20347] Updated weights for policy 0, policy_version 1053477 (0.0010) [2023-12-27 12:56:56,708][20381] Updated weights for policy 1, policy_version 1052670 (0.0010) [2023-12-27 12:56:56,770][20381] Updated weights for policy 1, policy_version 1052680 (0.0008) [2023-12-27 12:56:56,780][20347] Updated weights for policy 0, policy_version 1053487 (0.0008) [2023-12-27 12:56:56,831][20347] Updated weights for policy 0, policy_version 1053497 (0.0006) [2023-12-27 12:56:56,832][20381] Updated weights for policy 1, policy_version 1052690 (0.0007) [2023-12-27 12:56:56,876][20347] Updated weights for policy 0, policy_version 1053507 (0.0005) [2023-12-27 12:56:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20632.7). Total num frames: 539262976. Throughput: 0: 10324.8, 1: 10355.5. Samples: 539254588. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:56:57,364][19170] Avg episode reward: [(0, '931.820'), (1, '745.320')] [2023-12-27 12:56:57,466][20381] Updated weights for policy 1, policy_version 1052700 (0.0009) [2023-12-27 12:56:57,476][20347] Updated weights for policy 0, policy_version 1053517 (0.0005) [2023-12-27 12:56:57,507][20381] Updated weights for policy 1, policy_version 1052710 (0.0007) [2023-12-27 12:56:57,528][20291] KL-divergence is very high: 103.9850 [2023-12-27 12:56:57,532][20347] Updated weights for policy 0, policy_version 1053527 (0.0007) [2023-12-27 12:56:57,551][20381] Updated weights for policy 1, policy_version 1052720 (0.0006) [2023-12-27 12:56:57,576][20347] Updated weights for policy 0, policy_version 1053537 (0.0007) [2023-12-27 12:56:57,600][20291] KL-divergence is very high: 126.0267 [2023-12-27 12:56:58,217][20381] Updated weights for policy 1, policy_version 1052730 (0.0006) [2023-12-27 12:56:58,226][20347] Updated weights for policy 0, policy_version 1053547 (0.0008) [2023-12-27 12:56:58,274][20381] Updated weights for policy 1, policy_version 1052740 (0.0006) [2023-12-27 12:56:58,284][20347] Updated weights for policy 0, policy_version 1053557 (0.0006) [2023-12-27 12:56:58,339][20381] Updated weights for policy 1, policy_version 1052750 (0.0007) [2023-12-27 12:56:58,366][20347] Updated weights for policy 0, policy_version 1053567 (0.0011) [2023-12-27 12:56:58,401][20381] Updated weights for policy 1, policy_version 1052760 (0.0006) [2023-12-27 12:56:59,118][20347] Updated weights for policy 0, policy_version 1053577 (0.0007) [2023-12-27 12:56:59,169][20381] Updated weights for policy 1, policy_version 1052770 (0.0006) [2023-12-27 12:56:59,175][20347] Updated weights for policy 0, policy_version 1053587 (0.0007) [2023-12-27 12:56:59,230][20381] Updated weights for policy 1, policy_version 1052780 (0.0008) [2023-12-27 12:56:59,238][20347] Updated weights for policy 0, policy_version 1053597 (0.0007) [2023-12-27 12:56:59,294][20381] Updated weights for policy 1, policy_version 1052790 (0.0007) [2023-12-27 12:56:59,298][20291] KL-divergence is very high: 114.5440 [2023-12-27 12:56:59,300][20347] Updated weights for policy 0, policy_version 1053607 (0.0007) [2023-12-27 12:56:59,986][20347] Updated weights for policy 0, policy_version 1053617 (0.0008) [2023-12-27 12:57:00,017][20381] Updated weights for policy 1, policy_version 1052800 (0.0008) [2023-12-27 12:57:00,049][20347] Updated weights for policy 0, policy_version 1053627 (0.0007) [2023-12-27 12:57:00,076][20381] Updated weights for policy 1, policy_version 1052810 (0.0007) [2023-12-27 12:57:00,110][20347] Updated weights for policy 0, policy_version 1053637 (0.0007) [2023-12-27 12:57:00,127][20381] Updated weights for policy 1, policy_version 1052820 (0.0006) [2023-12-27 12:57:00,786][20381] Updated weights for policy 1, policy_version 1052830 (0.0006) [2023-12-27 12:57:00,822][20347] Updated weights for policy 0, policy_version 1053647 (0.0008) [2023-12-27 12:57:00,837][20381] Updated weights for policy 1, policy_version 1052840 (0.0005) [2023-12-27 12:57:00,870][20347] Updated weights for policy 0, policy_version 1053657 (0.0007) [2023-12-27 12:57:00,887][20381] Updated weights for policy 1, policy_version 1052850 (0.0005) [2023-12-27 12:57:00,928][20347] Updated weights for policy 0, policy_version 1053667 (0.0006) [2023-12-27 12:57:01,550][20381] Updated weights for policy 1, policy_version 1052860 (0.0006) [2023-12-27 12:57:01,582][20347] Updated weights for policy 0, policy_version 1053677 (0.0006) [2023-12-27 12:57:01,609][20381] Updated weights for policy 1, policy_version 1052870 (0.0007) [2023-12-27 12:57:01,645][20347] Updated weights for policy 0, policy_version 1053687 (0.0007) [2023-12-27 12:57:01,672][20381] Updated weights for policy 1, policy_version 1052880 (0.0007) [2023-12-27 12:57:01,705][20347] Updated weights for policy 0, policy_version 1053697 (0.0007) [2023-12-27 12:57:02,356][20347] Updated weights for policy 0, policy_version 1053707 (0.0009) [2023-12-27 12:57:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 539361280. Throughput: 0: 10301.2, 1: 10339.8. Samples: 539376356. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:57:02,363][19170] Avg episode reward: [(0, '912.510'), (1, '794.250')] [2023-12-27 12:57:02,388][20381] Updated weights for policy 1, policy_version 1052890 (0.0007) [2023-12-27 12:57:02,414][20347] Updated weights for policy 0, policy_version 1053717 (0.0008) [2023-12-27 12:57:02,441][20381] Updated weights for policy 1, policy_version 1052900 (0.0005) [2023-12-27 12:57:02,466][20347] Updated weights for policy 0, policy_version 1053727 (0.0010) [2023-12-27 12:57:02,492][20381] Updated weights for policy 1, policy_version 1052910 (0.0005) [2023-12-27 12:57:02,540][20381] Updated weights for policy 1, policy_version 1052920 (0.0006) [2023-12-27 12:57:03,066][20347] Updated weights for policy 0, policy_version 1053737 (0.0010) [2023-12-27 12:57:03,117][20347] Updated weights for policy 0, policy_version 1053747 (0.0010) [2023-12-27 12:57:03,168][20347] Updated weights for policy 0, policy_version 1053757 (0.0010) [2023-12-27 12:57:03,217][20347] Updated weights for policy 0, policy_version 1053767 (0.0010) [2023-12-27 12:57:03,229][20381] Updated weights for policy 1, policy_version 1052930 (0.0006) [2023-12-27 12:57:03,281][20381] Updated weights for policy 1, policy_version 1052940 (0.0008) [2023-12-27 12:57:03,338][20381] Updated weights for policy 1, policy_version 1052950 (0.0007) [2023-12-27 12:57:03,905][20347] Updated weights for policy 0, policy_version 1053777 (0.0010) [2023-12-27 12:57:03,968][20347] Updated weights for policy 0, policy_version 1053787 (0.0010) [2023-12-27 12:57:04,028][20347] Updated weights for policy 0, policy_version 1053797 (0.0010) [2023-12-27 12:57:04,053][20381] Updated weights for policy 1, policy_version 1052960 (0.0006) [2023-12-27 12:57:04,122][20381] Updated weights for policy 1, policy_version 1052970 (0.0009) [2023-12-27 12:57:04,184][20381] Updated weights for policy 1, policy_version 1052980 (0.0009) [2023-12-27 12:57:04,765][20347] Updated weights for policy 0, policy_version 1053807 (0.0010) [2023-12-27 12:57:04,826][20347] Updated weights for policy 0, policy_version 1053817 (0.0010) [2023-12-27 12:57:04,850][20381] Updated weights for policy 1, policy_version 1052990 (0.0007) [2023-12-27 12:57:04,888][20347] Updated weights for policy 0, policy_version 1053827 (0.0010) [2023-12-27 12:57:04,900][20381] Updated weights for policy 1, policy_version 1053000 (0.0008) [2023-12-27 12:57:04,951][20381] Updated weights for policy 1, policy_version 1053010 (0.0007) [2023-12-27 12:57:05,500][20347] Updated weights for policy 0, policy_version 1053837 (0.0008) [2023-12-27 12:57:05,543][20347] Updated weights for policy 0, policy_version 1053847 (0.0005) [2023-12-27 12:57:05,589][20347] Updated weights for policy 0, policy_version 1053857 (0.0005) [2023-12-27 12:57:05,697][20381] Updated weights for policy 1, policy_version 1053020 (0.0008) [2023-12-27 12:57:05,751][20381] Updated weights for policy 1, policy_version 1053030 (0.0007) [2023-12-27 12:57:05,806][20381] Updated weights for policy 1, policy_version 1053040 (0.0007) [2023-12-27 12:57:06,171][20347] Updated weights for policy 0, policy_version 1053867 (0.0005) [2023-12-27 12:57:06,225][20347] Updated weights for policy 0, policy_version 1053877 (0.0009) [2023-12-27 12:57:06,282][20347] Updated weights for policy 0, policy_version 1053887 (0.0008) [2023-12-27 12:57:06,555][20381] Updated weights for policy 1, policy_version 1053050 (0.0008) [2023-12-27 12:57:06,610][20381] Updated weights for policy 1, policy_version 1053060 (0.0008) [2023-12-27 12:57:06,665][20381] Updated weights for policy 1, policy_version 1053070 (0.0007) [2023-12-27 12:57:06,717][20381] Updated weights for policy 1, policy_version 1053080 (0.0007) [2023-12-27 12:57:06,948][20347] Updated weights for policy 0, policy_version 1053897 (0.0006) [2023-12-27 12:57:07,007][20347] Updated weights for policy 0, policy_version 1053907 (0.0011) [2023-12-27 12:57:07,056][20347] Updated weights for policy 0, policy_version 1053917 (0.0010) [2023-12-27 12:57:07,111][20347] Updated weights for policy 0, policy_version 1053927 (0.0009) [2023-12-27 12:57:07,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 539467776. Throughput: 0: 10307.5, 1: 10309.1. Samples: 539437912. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:57:07,363][19170] Avg episode reward: [(0, '960.990'), (1, '754.570')] [2023-12-27 12:57:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001053080_269615104.pth... [2023-12-27 12:57:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001053928_269852672.pth... [2023-12-27 12:57:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001052712_269541376.pth [2023-12-27 12:57:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001051896_269312000.pth [2023-12-27 12:57:07,451][20381] Updated weights for policy 1, policy_version 1053090 (0.0007) [2023-12-27 12:57:07,505][20381] Updated weights for policy 1, policy_version 1053100 (0.0007) [2023-12-27 12:57:07,565][20381] Updated weights for policy 1, policy_version 1053110 (0.0008) [2023-12-27 12:57:07,767][20347] Updated weights for policy 0, policy_version 1053937 (0.0006) [2023-12-27 12:57:07,833][20347] Updated weights for policy 0, policy_version 1053947 (0.0005) [2023-12-27 12:57:07,884][20347] Updated weights for policy 0, policy_version 1053957 (0.0005) [2023-12-27 12:57:08,293][20381] Updated weights for policy 1, policy_version 1053120 (0.0008) [2023-12-27 12:57:08,356][20381] Updated weights for policy 1, policy_version 1053130 (0.0009) [2023-12-27 12:57:08,414][20381] Updated weights for policy 1, policy_version 1053140 (0.0009) [2023-12-27 12:57:08,483][20347] Updated weights for policy 0, policy_version 1053967 (0.0006) [2023-12-27 12:57:08,540][20347] Updated weights for policy 0, policy_version 1053977 (0.0007) [2023-12-27 12:57:08,591][20347] Updated weights for policy 0, policy_version 1053987 (0.0006) [2023-12-27 12:57:09,192][20381] Updated weights for policy 1, policy_version 1053150 (0.0007) [2023-12-27 12:57:09,193][20347] Updated weights for policy 0, policy_version 1053997 (0.0005) [2023-12-27 12:57:09,258][20381] Updated weights for policy 1, policy_version 1053160 (0.0009) [2023-12-27 12:57:09,263][20347] Updated weights for policy 0, policy_version 1054007 (0.0008) [2023-12-27 12:57:09,326][20347] Updated weights for policy 0, policy_version 1054017 (0.0010) [2023-12-27 12:57:09,335][20381] Updated weights for policy 1, policy_version 1053170 (0.0007) [2023-12-27 12:57:10,024][20347] Updated weights for policy 0, policy_version 1054027 (0.0009) [2023-12-27 12:57:10,059][20381] Updated weights for policy 1, policy_version 1053180 (0.0007) [2023-12-27 12:57:10,079][20347] Updated weights for policy 0, policy_version 1054037 (0.0007) [2023-12-27 12:57:10,110][20381] Updated weights for policy 1, policy_version 1053190 (0.0008) [2023-12-27 12:57:10,141][20347] Updated weights for policy 0, policy_version 1054047 (0.0008) [2023-12-27 12:57:10,169][20381] Updated weights for policy 1, policy_version 1053200 (0.0007) [2023-12-27 12:57:10,817][20381] Updated weights for policy 1, policy_version 1053210 (0.0006) [2023-12-27 12:57:10,831][20347] Updated weights for policy 0, policy_version 1054057 (0.0006) [2023-12-27 12:57:10,880][20381] Updated weights for policy 1, policy_version 1053220 (0.0005) [2023-12-27 12:57:10,889][20347] Updated weights for policy 0, policy_version 1054067 (0.0005) [2023-12-27 12:57:10,937][20381] Updated weights for policy 1, policy_version 1053230 (0.0005) [2023-12-27 12:57:10,942][20347] Updated weights for policy 0, policy_version 1054077 (0.0005) [2023-12-27 12:57:10,994][20381] Updated weights for policy 1, policy_version 1053240 (0.0005) [2023-12-27 12:57:11,002][20347] Updated weights for policy 0, policy_version 1054087 (0.0005) [2023-12-27 12:57:11,642][20381] Updated weights for policy 1, policy_version 1053250 (0.0008) [2023-12-27 12:57:11,703][20381] Updated weights for policy 1, policy_version 1053260 (0.0008) [2023-12-27 12:57:11,730][20347] Updated weights for policy 0, policy_version 1054097 (0.0007) [2023-12-27 12:57:11,771][20381] Updated weights for policy 1, policy_version 1053270 (0.0009) [2023-12-27 12:57:11,802][20347] Updated weights for policy 0, policy_version 1054107 (0.0006) [2023-12-27 12:57:11,866][20347] Updated weights for policy 0, policy_version 1054117 (0.0005) [2023-12-27 12:57:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 539566080. Throughput: 0: 10401.0, 1: 10212.0. Samples: 539561312. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:57:12,363][19170] Avg episode reward: [(0, '960.930'), (1, '692.610')] [2023-12-27 12:57:12,460][20347] Updated weights for policy 0, policy_version 1054127 (0.0006) [2023-12-27 12:57:12,519][20347] Updated weights for policy 0, policy_version 1054137 (0.0008) [2023-12-27 12:57:12,540][20381] Updated weights for policy 1, policy_version 1053280 (0.0006) [2023-12-27 12:57:12,578][20347] Updated weights for policy 0, policy_version 1054147 (0.0008) [2023-12-27 12:57:12,599][20381] Updated weights for policy 1, policy_version 1053290 (0.0006) [2023-12-27 12:57:12,656][20381] Updated weights for policy 1, policy_version 1053300 (0.0007) [2023-12-27 12:57:13,257][20347] Updated weights for policy 0, policy_version 1054157 (0.0008) [2023-12-27 12:57:13,303][20381] Updated weights for policy 1, policy_version 1053310 (0.0006) [2023-12-27 12:57:13,315][20347] Updated weights for policy 0, policy_version 1054167 (0.0010) [2023-12-27 12:57:13,357][20381] Updated weights for policy 1, policy_version 1053320 (0.0005) [2023-12-27 12:57:13,366][20347] Updated weights for policy 0, policy_version 1054177 (0.0010) [2023-12-27 12:57:13,409][20381] Updated weights for policy 1, policy_version 1053330 (0.0006) [2023-12-27 12:57:13,913][20347] Updated weights for policy 0, policy_version 1054187 (0.0009) [2023-12-27 12:57:13,970][20347] Updated weights for policy 0, policy_version 1054197 (0.0006) [2023-12-27 12:57:14,021][20347] Updated weights for policy 0, policy_version 1054207 (0.0005) [2023-12-27 12:57:14,098][20381] Updated weights for policy 1, policy_version 1053340 (0.0008) [2023-12-27 12:57:14,144][20381] Updated weights for policy 1, policy_version 1053350 (0.0010) [2023-12-27 12:57:14,193][20381] Updated weights for policy 1, policy_version 1053360 (0.0009) [2023-12-27 12:57:14,598][20347] Updated weights for policy 0, policy_version 1054217 (0.0005) [2023-12-27 12:57:14,647][20347] Updated weights for policy 0, policy_version 1054227 (0.0007) [2023-12-27 12:57:14,706][20347] Updated weights for policy 0, policy_version 1054237 (0.0008) [2023-12-27 12:57:14,766][20347] Updated weights for policy 0, policy_version 1054247 (0.0007) [2023-12-27 12:57:14,966][20381] Updated weights for policy 1, policy_version 1053370 (0.0008) [2023-12-27 12:57:15,027][20381] Updated weights for policy 1, policy_version 1053380 (0.0011) [2023-12-27 12:57:15,088][20381] Updated weights for policy 1, policy_version 1053390 (0.0009) [2023-12-27 12:57:15,150][20381] Updated weights for policy 1, policy_version 1053400 (0.0008) [2023-12-27 12:57:15,481][20347] Updated weights for policy 0, policy_version 1054257 (0.0008) [2023-12-27 12:57:15,531][20347] Updated weights for policy 0, policy_version 1054267 (0.0008) [2023-12-27 12:57:15,579][20347] Updated weights for policy 0, policy_version 1054277 (0.0008) [2023-12-27 12:57:15,842][20381] Updated weights for policy 1, policy_version 1053410 (0.0007) [2023-12-27 12:57:15,900][20381] Updated weights for policy 1, policy_version 1053420 (0.0008) [2023-12-27 12:57:15,963][20381] Updated weights for policy 1, policy_version 1053430 (0.0007) [2023-12-27 12:57:16,267][20347] Updated weights for policy 0, policy_version 1054287 (0.0008) [2023-12-27 12:57:16,320][20347] Updated weights for policy 0, policy_version 1054297 (0.0007) [2023-12-27 12:57:16,383][20347] Updated weights for policy 0, policy_version 1054307 (0.0008) [2023-12-27 12:57:16,526][20381] Updated weights for policy 1, policy_version 1053440 (0.0009) [2023-12-27 12:57:16,570][20381] Updated weights for policy 1, policy_version 1053450 (0.0006) [2023-12-27 12:57:16,622][20381] Updated weights for policy 1, policy_version 1053460 (0.0005) [2023-12-27 12:57:17,023][20347] Updated weights for policy 0, policy_version 1054317 (0.0007) [2023-12-27 12:57:17,086][20347] Updated weights for policy 0, policy_version 1054327 (0.0007) [2023-12-27 12:57:17,135][20347] Updated weights for policy 0, policy_version 1054337 (0.0005) [2023-12-27 12:57:17,297][20381] Updated weights for policy 1, policy_version 1053470 (0.0007) [2023-12-27 12:57:17,345][20381] Updated weights for policy 1, policy_version 1053480 (0.0009) [2023-12-27 12:57:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 539672576. Throughput: 0: 10389.4, 1: 10145.5. Samples: 539685944. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:57:17,363][19170] Avg episode reward: [(0, '970.720'), (1, '711.970')] [2023-12-27 12:57:17,402][20381] Updated weights for policy 1, policy_version 1053490 (0.0009) [2023-12-27 12:57:17,766][20347] Updated weights for policy 0, policy_version 1054347 (0.0006) [2023-12-27 12:57:17,821][20347] Updated weights for policy 0, policy_version 1054357 (0.0008) [2023-12-27 12:57:17,880][20347] Updated weights for policy 0, policy_version 1054367 (0.0008) [2023-12-27 12:57:18,168][20381] Updated weights for policy 1, policy_version 1053500 (0.0009) [2023-12-27 12:57:18,227][20381] Updated weights for policy 1, policy_version 1053510 (0.0009) [2023-12-27 12:57:18,281][20381] Updated weights for policy 1, policy_version 1053520 (0.0009) [2023-12-27 12:57:18,469][20347] Updated weights for policy 0, policy_version 1054377 (0.0005) [2023-12-27 12:57:18,528][20347] Updated weights for policy 0, policy_version 1054387 (0.0005) [2023-12-27 12:57:18,590][20347] Updated weights for policy 0, policy_version 1054397 (0.0005) [2023-12-27 12:57:18,651][20347] Updated weights for policy 0, policy_version 1054407 (0.0006) [2023-12-27 12:57:18,907][20381] Updated weights for policy 1, policy_version 1053530 (0.0008) [2023-12-27 12:57:18,956][20381] Updated weights for policy 1, policy_version 1053540 (0.0007) [2023-12-27 12:57:19,006][20381] Updated weights for policy 1, policy_version 1053550 (0.0007) [2023-12-27 12:57:19,057][20381] Updated weights for policy 1, policy_version 1053560 (0.0010) [2023-12-27 12:57:19,208][20347] Updated weights for policy 0, policy_version 1054417 (0.0006) [2023-12-27 12:57:19,273][20347] Updated weights for policy 0, policy_version 1054427 (0.0008) [2023-12-27 12:57:19,332][20347] Updated weights for policy 0, policy_version 1054437 (0.0008) [2023-12-27 12:57:19,752][20381] Updated weights for policy 1, policy_version 1053570 (0.0007) [2023-12-27 12:57:19,817][20381] Updated weights for policy 1, policy_version 1053580 (0.0007) [2023-12-27 12:57:19,879][20381] Updated weights for policy 1, policy_version 1053590 (0.0007) [2023-12-27 12:57:20,044][20347] Updated weights for policy 0, policy_version 1054447 (0.0010) [2023-12-27 12:57:20,102][20347] Updated weights for policy 0, policy_version 1054457 (0.0009) [2023-12-27 12:57:20,162][20347] Updated weights for policy 0, policy_version 1054467 (0.0008) [2023-12-27 12:57:20,556][20381] Updated weights for policy 1, policy_version 1053600 (0.0007) [2023-12-27 12:57:20,624][20381] Updated weights for policy 1, policy_version 1053610 (0.0008) [2023-12-27 12:57:20,682][20381] Updated weights for policy 1, policy_version 1053620 (0.0007) [2023-12-27 12:57:20,878][20347] Updated weights for policy 0, policy_version 1054477 (0.0008) [2023-12-27 12:57:20,944][20347] Updated weights for policy 0, policy_version 1054487 (0.0007) [2023-12-27 12:57:21,007][20347] Updated weights for policy 0, policy_version 1054497 (0.0008) [2023-12-27 12:57:21,378][20381] Updated weights for policy 1, policy_version 1053630 (0.0008) [2023-12-27 12:57:21,441][20381] Updated weights for policy 1, policy_version 1053640 (0.0008) [2023-12-27 12:57:21,489][20381] Updated weights for policy 1, policy_version 1053650 (0.0006) [2023-12-27 12:57:21,757][20347] Updated weights for policy 0, policy_version 1054507 (0.0009) [2023-12-27 12:57:21,817][20347] Updated weights for policy 0, policy_version 1054517 (0.0006) [2023-12-27 12:57:21,877][20347] Updated weights for policy 0, policy_version 1054527 (0.0007) [2023-12-27 12:57:22,192][20381] Updated weights for policy 1, policy_version 1053660 (0.0008) [2023-12-27 12:57:22,256][20381] Updated weights for policy 1, policy_version 1053670 (0.0011) [2023-12-27 12:57:22,324][20381] Updated weights for policy 1, policy_version 1053680 (0.0009) [2023-12-27 12:57:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 539770880. Throughput: 0: 10423.9, 1: 10100.3. Samples: 539748776. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 12:57:22,363][19170] Avg episode reward: [(0, '931.670'), (1, '694.680')] [2023-12-27 12:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001054536_270008320.pth... [2023-12-27 12:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001053320_269697024.pth [2023-12-27 12:57:22,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001053688_269770752.pth... [2023-12-27 12:57:22,384][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001052504_269467648.pth [2023-12-27 12:57:22,584][20347] Updated weights for policy 0, policy_version 1054537 (0.0007) [2023-12-27 12:57:22,644][20347] Updated weights for policy 0, policy_version 1054547 (0.0006) [2023-12-27 12:57:22,712][20347] Updated weights for policy 0, policy_version 1054557 (0.0005) [2023-12-27 12:57:22,778][20347] Updated weights for policy 0, policy_version 1054567 (0.0006) [2023-12-27 12:57:23,025][20381] Updated weights for policy 1, policy_version 1053690 (0.0011) [2023-12-27 12:57:23,076][20381] Updated weights for policy 1, policy_version 1053700 (0.0010) [2023-12-27 12:57:23,133][20381] Updated weights for policy 1, policy_version 1053710 (0.0010) [2023-12-27 12:57:23,192][20381] Updated weights for policy 1, policy_version 1053720 (0.0008) [2023-12-27 12:57:23,335][20347] Updated weights for policy 0, policy_version 1054577 (0.0005) [2023-12-27 12:57:23,392][20347] Updated weights for policy 0, policy_version 1054587 (0.0005) [2023-12-27 12:57:23,438][20347] Updated weights for policy 0, policy_version 1054597 (0.0005) [2023-12-27 12:57:23,829][20381] Updated weights for policy 1, policy_version 1053730 (0.0008) [2023-12-27 12:57:23,882][20381] Updated weights for policy 1, policy_version 1053740 (0.0009) [2023-12-27 12:57:23,930][20381] Updated weights for policy 1, policy_version 1053750 (0.0007) [2023-12-27 12:57:23,986][20347] Updated weights for policy 0, policy_version 1054607 (0.0005) [2023-12-27 12:57:24,050][20347] Updated weights for policy 0, policy_version 1054617 (0.0005) [2023-12-27 12:57:24,106][20347] Updated weights for policy 0, policy_version 1054627 (0.0006) [2023-12-27 12:57:24,496][20381] Updated weights for policy 1, policy_version 1053760 (0.0007) [2023-12-27 12:57:24,545][20381] Updated weights for policy 1, policy_version 1053770 (0.0008) [2023-12-27 12:57:24,598][20381] Updated weights for policy 1, policy_version 1053780 (0.0008) [2023-12-27 12:57:24,787][20347] Updated weights for policy 0, policy_version 1054637 (0.0008) [2023-12-27 12:57:24,850][20347] Updated weights for policy 0, policy_version 1054647 (0.0007) [2023-12-27 12:57:24,910][20347] Updated weights for policy 0, policy_version 1054657 (0.0008) [2023-12-27 12:57:25,203][20381] Updated weights for policy 1, policy_version 1053790 (0.0008) [2023-12-27 12:57:25,256][20381] Updated weights for policy 1, policy_version 1053800 (0.0008) [2023-12-27 12:57:25,306][20381] Updated weights for policy 1, policy_version 1053810 (0.0008) [2023-12-27 12:57:25,529][20347] Updated weights for policy 0, policy_version 1054667 (0.0007) [2023-12-27 12:57:25,585][20347] Updated weights for policy 0, policy_version 1054677 (0.0006) [2023-12-27 12:57:25,637][20347] Updated weights for policy 0, policy_version 1054687 (0.0007) [2023-12-27 12:57:25,993][20381] Updated weights for policy 1, policy_version 1053820 (0.0007) [2023-12-27 12:57:26,049][20381] Updated weights for policy 1, policy_version 1053830 (0.0008) [2023-12-27 12:57:26,103][20381] Updated weights for policy 1, policy_version 1053840 (0.0008) [2023-12-27 12:57:26,324][20347] Updated weights for policy 0, policy_version 1054697 (0.0008) [2023-12-27 12:57:26,375][20347] Updated weights for policy 0, policy_version 1054707 (0.0010) [2023-12-27 12:57:26,422][20347] Updated weights for policy 0, policy_version 1054717 (0.0010) [2023-12-27 12:57:26,470][20347] Updated weights for policy 0, policy_version 1054727 (0.0010) [2023-12-27 12:57:26,756][20381] Updated weights for policy 1, policy_version 1053850 (0.0008) [2023-12-27 12:57:26,797][20381] Updated weights for policy 1, policy_version 1053860 (0.0006) [2023-12-27 12:57:26,845][20381] Updated weights for policy 1, policy_version 1053870 (0.0007) [2023-12-27 12:57:26,890][20381] Updated weights for policy 1, policy_version 1053880 (0.0008) [2023-12-27 12:57:27,193][20347] Updated weights for policy 0, policy_version 1054737 (0.0007) [2023-12-27 12:57:27,245][20347] Updated weights for policy 0, policy_version 1054747 (0.0007) [2023-12-27 12:57:27,298][20347] Updated weights for policy 0, policy_version 1054757 (0.0007) [2023-12-27 12:57:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 539885568. Throughput: 0: 10475.0, 1: 10100.5. Samples: 539873372. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:27,363][19170] Avg episode reward: [(0, '941.390'), (1, '657.230')] [2023-12-27 12:57:27,527][20381] Updated weights for policy 1, policy_version 1053890 (0.0010) [2023-12-27 12:57:27,578][20381] Updated weights for policy 1, policy_version 1053900 (0.0010) [2023-12-27 12:57:27,627][20381] Updated weights for policy 1, policy_version 1053910 (0.0010) [2023-12-27 12:57:27,999][20347] Updated weights for policy 0, policy_version 1054767 (0.0009) [2023-12-27 12:57:28,047][20347] Updated weights for policy 0, policy_version 1054777 (0.0010) [2023-12-27 12:57:28,095][20347] Updated weights for policy 0, policy_version 1054787 (0.0010) [2023-12-27 12:57:28,237][20381] Updated weights for policy 1, policy_version 1053920 (0.0008) [2023-12-27 12:57:28,286][20381] Updated weights for policy 1, policy_version 1053930 (0.0007) [2023-12-27 12:57:28,353][20381] Updated weights for policy 1, policy_version 1053940 (0.0008) [2023-12-27 12:57:28,755][20347] Updated weights for policy 0, policy_version 1054797 (0.0008) [2023-12-27 12:57:28,817][20347] Updated weights for policy 0, policy_version 1054807 (0.0007) [2023-12-27 12:57:28,876][20347] Updated weights for policy 0, policy_version 1054817 (0.0008) [2023-12-27 12:57:29,037][20381] Updated weights for policy 1, policy_version 1053950 (0.0008) [2023-12-27 12:57:29,092][20381] Updated weights for policy 1, policy_version 1053960 (0.0008) [2023-12-27 12:57:29,147][20381] Updated weights for policy 1, policy_version 1053970 (0.0008) [2023-12-27 12:57:29,522][20347] Updated weights for policy 0, policy_version 1054827 (0.0006) [2023-12-27 12:57:29,568][20347] Updated weights for policy 0, policy_version 1054837 (0.0006) [2023-12-27 12:57:29,627][20347] Updated weights for policy 0, policy_version 1054847 (0.0007) [2023-12-27 12:57:29,909][20381] Updated weights for policy 1, policy_version 1053980 (0.0006) [2023-12-27 12:57:29,967][20381] Updated weights for policy 1, policy_version 1053990 (0.0007) [2023-12-27 12:57:30,016][20381] Updated weights for policy 1, policy_version 1054000 (0.0008) [2023-12-27 12:57:30,251][20347] Updated weights for policy 0, policy_version 1054857 (0.0007) [2023-12-27 12:57:30,309][20347] Updated weights for policy 0, policy_version 1054867 (0.0005) [2023-12-27 12:57:30,368][20347] Updated weights for policy 0, policy_version 1054877 (0.0006) [2023-12-27 12:57:30,433][20347] Updated weights for policy 0, policy_version 1054887 (0.0010) [2023-12-27 12:57:30,778][20381] Updated weights for policy 1, policy_version 1054010 (0.0006) [2023-12-27 12:57:30,836][20381] Updated weights for policy 1, policy_version 1054020 (0.0005) [2023-12-27 12:57:30,899][20381] Updated weights for policy 1, policy_version 1054030 (0.0005) [2023-12-27 12:57:30,948][20381] Updated weights for policy 1, policy_version 1054040 (0.0007) [2023-12-27 12:57:31,080][20347] Updated weights for policy 0, policy_version 1054897 (0.0010) [2023-12-27 12:57:31,138][20347] Updated weights for policy 0, policy_version 1054907 (0.0010) [2023-12-27 12:57:31,201][20347] Updated weights for policy 0, policy_version 1054917 (0.0009) [2023-12-27 12:57:31,640][20381] Updated weights for policy 1, policy_version 1054050 (0.0007) [2023-12-27 12:57:31,705][20381] Updated weights for policy 1, policy_version 1054060 (0.0007) [2023-12-27 12:57:31,770][20381] Updated weights for policy 1, policy_version 1054070 (0.0009) [2023-12-27 12:57:31,871][20347] Updated weights for policy 0, policy_version 1054927 (0.0006) [2023-12-27 12:57:31,923][20347] Updated weights for policy 0, policy_version 1054937 (0.0009) [2023-12-27 12:57:31,975][20347] Updated weights for policy 0, policy_version 1054947 (0.0007) [2023-12-27 12:57:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 539983872. Throughput: 0: 10472.9, 1: 10120.8. Samples: 539997516. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:32,363][19170] Avg episode reward: [(0, '951.180'), (1, '706.340')] [2023-12-27 12:57:32,369][20381] Updated weights for policy 1, policy_version 1054080 (0.0008) [2023-12-27 12:57:32,431][20381] Updated weights for policy 1, policy_version 1054090 (0.0007) [2023-12-27 12:57:32,484][20381] Updated weights for policy 1, policy_version 1054100 (0.0008) [2023-12-27 12:57:32,602][20347] Updated weights for policy 0, policy_version 1054957 (0.0006) [2023-12-27 12:57:32,663][20347] Updated weights for policy 0, policy_version 1054967 (0.0005) [2023-12-27 12:57:32,721][20347] Updated weights for policy 0, policy_version 1054977 (0.0005) [2023-12-27 12:57:33,111][20381] Updated weights for policy 1, policy_version 1054110 (0.0007) [2023-12-27 12:57:33,163][20381] Updated weights for policy 1, policy_version 1054120 (0.0008) [2023-12-27 12:57:33,224][20381] Updated weights for policy 1, policy_version 1054130 (0.0007) [2023-12-27 12:57:33,269][20347] Updated weights for policy 0, policy_version 1054987 (0.0006) [2023-12-27 12:57:33,328][20347] Updated weights for policy 0, policy_version 1054997 (0.0008) [2023-12-27 12:57:33,392][20347] Updated weights for policy 0, policy_version 1055007 (0.0009) [2023-12-27 12:57:33,947][20381] Updated weights for policy 1, policy_version 1054140 (0.0007) [2023-12-27 12:57:33,962][20347] Updated weights for policy 0, policy_version 1055017 (0.0005) [2023-12-27 12:57:34,002][20381] Updated weights for policy 1, policy_version 1054150 (0.0007) [2023-12-27 12:57:34,020][20347] Updated weights for policy 0, policy_version 1055027 (0.0008) [2023-12-27 12:57:34,055][20381] Updated weights for policy 1, policy_version 1054160 (0.0006) [2023-12-27 12:57:34,074][20347] Updated weights for policy 0, policy_version 1055037 (0.0008) [2023-12-27 12:57:34,130][20347] Updated weights for policy 0, policy_version 1055047 (0.0007) [2023-12-27 12:57:34,773][20347] Updated weights for policy 0, policy_version 1055057 (0.0011) [2023-12-27 12:57:34,781][20381] Updated weights for policy 1, policy_version 1054170 (0.0007) [2023-12-27 12:57:34,826][20381] Updated weights for policy 1, policy_version 1054180 (0.0005) [2023-12-27 12:57:34,828][20347] Updated weights for policy 0, policy_version 1055067 (0.0010) [2023-12-27 12:57:34,875][20381] Updated weights for policy 1, policy_version 1054190 (0.0005) [2023-12-27 12:57:34,877][20347] Updated weights for policy 0, policy_version 1055077 (0.0010) [2023-12-27 12:57:34,936][20381] Updated weights for policy 1, policy_version 1054200 (0.0007) [2023-12-27 12:57:35,512][20347] Updated weights for policy 0, policy_version 1055087 (0.0010) [2023-12-27 12:57:35,566][20347] Updated weights for policy 0, policy_version 1055097 (0.0010) [2023-12-27 12:57:35,601][20381] Updated weights for policy 1, policy_version 1054210 (0.0006) [2023-12-27 12:57:35,622][20347] Updated weights for policy 0, policy_version 1055107 (0.0011) [2023-12-27 12:57:35,661][20381] Updated weights for policy 1, policy_version 1054220 (0.0006) [2023-12-27 12:57:35,717][20381] Updated weights for policy 1, policy_version 1054230 (0.0008) [2023-12-27 12:57:36,342][20347] Updated weights for policy 0, policy_version 1055117 (0.0011) [2023-12-27 12:57:36,408][20347] Updated weights for policy 0, policy_version 1055127 (0.0011) [2023-12-27 12:57:36,464][20381] Updated weights for policy 1, policy_version 1054240 (0.0006) [2023-12-27 12:57:36,465][20347] Updated weights for policy 0, policy_version 1055137 (0.0010) [2023-12-27 12:57:36,522][20381] Updated weights for policy 1, policy_version 1054250 (0.0007) [2023-12-27 12:57:36,575][20381] Updated weights for policy 1, policy_version 1054260 (0.0007) [2023-12-27 12:57:37,103][20347] Updated weights for policy 0, policy_version 1055147 (0.0010) [2023-12-27 12:57:37,147][20291] KL-divergence is very high: 110.9998 [2023-12-27 12:57:37,153][20347] Updated weights for policy 0, policy_version 1055157 (0.0008) [2023-12-27 12:57:37,203][20347] Updated weights for policy 0, policy_version 1055167 (0.0006) [2023-12-27 12:57:37,257][20381] Updated weights for policy 1, policy_version 1054270 (0.0007) [2023-12-27 12:57:37,312][20381] Updated weights for policy 1, policy_version 1054280 (0.0008) [2023-12-27 12:57:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 540090368. Throughput: 0: 10549.1, 1: 10108.4. Samples: 540061548. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:37,363][19170] Avg episode reward: [(0, '931.820'), (1, '735.520')] [2023-12-27 12:57:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001055176_270172160.pth... [2023-12-27 12:57:37,374][20381] Updated weights for policy 1, policy_version 1054290 (0.0008) [2023-12-27 12:57:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001053928_269852672.pth [2023-12-27 12:57:37,408][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001054296_269926400.pth... [2023-12-27 12:57:37,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001053080_269615104.pth [2023-12-27 12:57:37,829][20347] Updated weights for policy 0, policy_version 1055177 (0.0008) [2023-12-27 12:57:37,880][20347] Updated weights for policy 0, policy_version 1055187 (0.0010) [2023-12-27 12:57:37,928][20347] Updated weights for policy 0, policy_version 1055197 (0.0010) [2023-12-27 12:57:37,985][20347] Updated weights for policy 0, policy_version 1055207 (0.0010) [2023-12-27 12:57:38,028][20381] Updated weights for policy 1, policy_version 1054300 (0.0007) [2023-12-27 12:57:38,085][20381] Updated weights for policy 1, policy_version 1054310 (0.0005) [2023-12-27 12:57:38,133][20381] Updated weights for policy 1, policy_version 1054320 (0.0010) [2023-12-27 12:57:38,677][20347] Updated weights for policy 0, policy_version 1055217 (0.0010) [2023-12-27 12:57:38,732][20347] Updated weights for policy 0, policy_version 1055227 (0.0010) [2023-12-27 12:57:38,792][20347] Updated weights for policy 0, policy_version 1055237 (0.0011) [2023-12-27 12:57:38,804][20381] Updated weights for policy 1, policy_version 1054330 (0.0010) [2023-12-27 12:57:38,859][20381] Updated weights for policy 1, policy_version 1054340 (0.0008) [2023-12-27 12:57:38,913][20381] Updated weights for policy 1, policy_version 1054350 (0.0008) [2023-12-27 12:57:38,965][20381] Updated weights for policy 1, policy_version 1054360 (0.0006) [2023-12-27 12:57:39,452][20347] Updated weights for policy 0, policy_version 1055247 (0.0009) [2023-12-27 12:57:39,504][20347] Updated weights for policy 0, policy_version 1055257 (0.0011) [2023-12-27 12:57:39,551][20347] Updated weights for policy 0, policy_version 1055267 (0.0010) [2023-12-27 12:57:39,658][20381] Updated weights for policy 1, policy_version 1054370 (0.0007) [2023-12-27 12:57:39,714][20381] Updated weights for policy 1, policy_version 1054380 (0.0008) [2023-12-27 12:57:39,767][20381] Updated weights for policy 1, policy_version 1054390 (0.0008) [2023-12-27 12:57:40,280][20347] Updated weights for policy 0, policy_version 1055277 (0.0010) [2023-12-27 12:57:40,341][20347] Updated weights for policy 0, policy_version 1055287 (0.0006) [2023-12-27 12:57:40,404][20347] Updated weights for policy 0, policy_version 1055297 (0.0007) [2023-12-27 12:57:40,472][20381] Updated weights for policy 1, policy_version 1054400 (0.0007) [2023-12-27 12:57:40,517][20381] Updated weights for policy 1, policy_version 1054410 (0.0008) [2023-12-27 12:57:40,574][20381] Updated weights for policy 1, policy_version 1054420 (0.0008) [2023-12-27 12:57:41,016][20347] Updated weights for policy 0, policy_version 1055307 (0.0007) [2023-12-27 12:57:41,083][20347] Updated weights for policy 0, policy_version 1055317 (0.0008) [2023-12-27 12:57:41,150][20347] Updated weights for policy 0, policy_version 1055327 (0.0008) [2023-12-27 12:57:41,361][20381] Updated weights for policy 1, policy_version 1054430 (0.0008) [2023-12-27 12:57:41,419][20381] Updated weights for policy 1, policy_version 1054440 (0.0009) [2023-12-27 12:57:41,475][20381] Updated weights for policy 1, policy_version 1054450 (0.0008) [2023-12-27 12:57:41,897][20347] Updated weights for policy 0, policy_version 1055337 (0.0007) [2023-12-27 12:57:41,962][20347] Updated weights for policy 0, policy_version 1055347 (0.0008) [2023-12-27 12:57:42,018][20347] Updated weights for policy 0, policy_version 1055357 (0.0007) [2023-12-27 12:57:42,075][20347] Updated weights for policy 0, policy_version 1055367 (0.0008) [2023-12-27 12:57:42,194][20381] Updated weights for policy 1, policy_version 1054460 (0.0007) [2023-12-27 12:57:42,256][20381] Updated weights for policy 1, policy_version 1054470 (0.0006) [2023-12-27 12:57:42,319][20381] Updated weights for policy 1, policy_version 1054480 (0.0007) [2023-12-27 12:57:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 540188672. Throughput: 0: 10556.3, 1: 10097.5. Samples: 540184004. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:42,363][19170] Avg episode reward: [(0, '922.160'), (1, '735.530')] [2023-12-27 12:57:42,773][20347] Updated weights for policy 0, policy_version 1055377 (0.0008) [2023-12-27 12:57:42,833][20347] Updated weights for policy 0, policy_version 1055387 (0.0007) [2023-12-27 12:57:42,895][20347] Updated weights for policy 0, policy_version 1055397 (0.0005) [2023-12-27 12:57:42,992][20381] Updated weights for policy 1, policy_version 1054490 (0.0007) [2023-12-27 12:57:43,040][20381] Updated weights for policy 1, policy_version 1054500 (0.0005) [2023-12-27 12:57:43,087][20381] Updated weights for policy 1, policy_version 1054510 (0.0007) [2023-12-27 12:57:43,133][20381] Updated weights for policy 1, policy_version 1054520 (0.0007) [2023-12-27 12:57:43,517][20347] Updated weights for policy 0, policy_version 1055407 (0.0008) [2023-12-27 12:57:43,571][20347] Updated weights for policy 0, policy_version 1055417 (0.0007) [2023-12-27 12:57:43,617][20347] Updated weights for policy 0, policy_version 1055427 (0.0007) [2023-12-27 12:57:43,790][20381] Updated weights for policy 1, policy_version 1054530 (0.0008) [2023-12-27 12:57:43,837][20381] Updated weights for policy 1, policy_version 1054540 (0.0008) [2023-12-27 12:57:43,886][20381] Updated weights for policy 1, policy_version 1054550 (0.0008) [2023-12-27 12:57:44,317][20347] Updated weights for policy 0, policy_version 1055437 (0.0007) [2023-12-27 12:57:44,368][20347] Updated weights for policy 0, policy_version 1055447 (0.0007) [2023-12-27 12:57:44,420][20347] Updated weights for policy 0, policy_version 1055457 (0.0007) [2023-12-27 12:57:44,600][20381] Updated weights for policy 1, policy_version 1054560 (0.0008) [2023-12-27 12:57:44,657][20381] Updated weights for policy 1, policy_version 1054570 (0.0007) [2023-12-27 12:57:44,713][20381] Updated weights for policy 1, policy_version 1054580 (0.0008) [2023-12-27 12:57:45,132][20347] Updated weights for policy 0, policy_version 1055467 (0.0008) [2023-12-27 12:57:45,194][20347] Updated weights for policy 0, policy_version 1055477 (0.0010) [2023-12-27 12:57:45,253][20347] Updated weights for policy 0, policy_version 1055487 (0.0010) [2023-12-27 12:57:45,415][20381] Updated weights for policy 1, policy_version 1054590 (0.0008) [2023-12-27 12:57:45,477][20381] Updated weights for policy 1, policy_version 1054600 (0.0007) [2023-12-27 12:57:45,538][20381] Updated weights for policy 1, policy_version 1054610 (0.0007) [2023-12-27 12:57:45,938][20347] Updated weights for policy 0, policy_version 1055497 (0.0010) [2023-12-27 12:57:45,992][20347] Updated weights for policy 0, policy_version 1055507 (0.0010) [2023-12-27 12:57:46,047][20347] Updated weights for policy 0, policy_version 1055517 (0.0010) [2023-12-27 12:57:46,102][20347] Updated weights for policy 0, policy_version 1055527 (0.0010) [2023-12-27 12:57:46,145][20381] Updated weights for policy 1, policy_version 1054620 (0.0007) [2023-12-27 12:57:46,199][20381] Updated weights for policy 1, policy_version 1054630 (0.0007) [2023-12-27 12:57:46,250][20381] Updated weights for policy 1, policy_version 1054640 (0.0007) [2023-12-27 12:57:46,804][20347] Updated weights for policy 0, policy_version 1055537 (0.0008) [2023-12-27 12:57:46,867][20347] Updated weights for policy 0, policy_version 1055547 (0.0006) [2023-12-27 12:57:46,922][20347] Updated weights for policy 0, policy_version 1055557 (0.0011) [2023-12-27 12:57:46,941][20381] Updated weights for policy 1, policy_version 1054650 (0.0006) [2023-12-27 12:57:46,988][20381] Updated weights for policy 1, policy_version 1054660 (0.0006) [2023-12-27 12:57:47,046][20381] Updated weights for policy 1, policy_version 1054670 (0.0005) [2023-12-27 12:57:47,104][20381] Updated weights for policy 1, policy_version 1054680 (0.0005) [2023-12-27 12:57:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 540295168. Throughput: 0: 10524.6, 1: 10118.1. Samples: 540305280. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:47,363][19170] Avg episode reward: [(0, '912.380'), (1, '716.240')] [2023-12-27 12:57:47,504][20347] Updated weights for policy 0, policy_version 1055567 (0.0007) [2023-12-27 12:57:47,558][20347] Updated weights for policy 0, policy_version 1055577 (0.0005) [2023-12-27 12:57:47,608][20347] Updated weights for policy 0, policy_version 1055587 (0.0005) [2023-12-27 12:57:47,632][20381] Updated weights for policy 1, policy_version 1054690 (0.0006) [2023-12-27 12:57:47,691][20381] Updated weights for policy 1, policy_version 1054700 (0.0008) [2023-12-27 12:57:47,753][20381] Updated weights for policy 1, policy_version 1054710 (0.0008) [2023-12-27 12:57:48,186][20347] Updated weights for policy 0, policy_version 1055597 (0.0006) [2023-12-27 12:57:48,243][20347] Updated weights for policy 0, policy_version 1055607 (0.0006) [2023-12-27 12:57:48,304][20347] Updated weights for policy 0, policy_version 1055617 (0.0007) [2023-12-27 12:57:48,495][20381] Updated weights for policy 1, policy_version 1054720 (0.0008) [2023-12-27 12:57:48,563][20381] Updated weights for policy 1, policy_version 1054730 (0.0007) [2023-12-27 12:57:48,621][20381] Updated weights for policy 1, policy_version 1054740 (0.0008) [2023-12-27 12:57:48,913][20347] Updated weights for policy 0, policy_version 1055627 (0.0007) [2023-12-27 12:57:48,966][20347] Updated weights for policy 0, policy_version 1055637 (0.0007) [2023-12-27 12:57:49,029][20347] Updated weights for policy 0, policy_version 1055647 (0.0008) [2023-12-27 12:57:49,219][20381] Updated weights for policy 1, policy_version 1054750 (0.0007) [2023-12-27 12:57:49,283][20381] Updated weights for policy 1, policy_version 1054760 (0.0008) [2023-12-27 12:57:49,341][20381] Updated weights for policy 1, policy_version 1054770 (0.0007) [2023-12-27 12:57:49,673][20347] Updated weights for policy 0, policy_version 1055657 (0.0007) [2023-12-27 12:57:49,731][20347] Updated weights for policy 0, policy_version 1055667 (0.0008) [2023-12-27 12:57:49,763][20291] KL-divergence is very high: 141.4731 [2023-12-27 12:57:49,799][20347] Updated weights for policy 0, policy_version 1055677 (0.0008) [2023-12-27 12:57:49,816][20291] KL-divergence is very high: 108.9437 [2023-12-27 12:57:49,863][20347] Updated weights for policy 0, policy_version 1055687 (0.0008) [2023-12-27 12:57:50,051][20381] Updated weights for policy 1, policy_version 1054780 (0.0008) [2023-12-27 12:57:50,113][20381] Updated weights for policy 1, policy_version 1054790 (0.0006) [2023-12-27 12:57:50,170][20381] Updated weights for policy 1, policy_version 1054800 (0.0006) [2023-12-27 12:57:50,613][20347] Updated weights for policy 0, policy_version 1055697 (0.0007) [2023-12-27 12:57:50,672][20347] Updated weights for policy 0, policy_version 1055707 (0.0007) [2023-12-27 12:57:50,738][20347] Updated weights for policy 0, policy_version 1055717 (0.0008) [2023-12-27 12:57:50,788][20381] Updated weights for policy 1, policy_version 1054810 (0.0007) [2023-12-27 12:57:50,841][20381] Updated weights for policy 1, policy_version 1054820 (0.0007) [2023-12-27 12:57:50,908][20381] Updated weights for policy 1, policy_version 1054830 (0.0008) [2023-12-27 12:57:50,972][20381] Updated weights for policy 1, policy_version 1054840 (0.0007) [2023-12-27 12:57:51,444][20347] Updated weights for policy 0, policy_version 1055727 (0.0007) [2023-12-27 12:57:51,496][20347] Updated weights for policy 0, policy_version 1055737 (0.0007) [2023-12-27 12:57:51,545][20347] Updated weights for policy 0, policy_version 1055747 (0.0008) [2023-12-27 12:57:51,723][20381] Updated weights for policy 1, policy_version 1054850 (0.0008) [2023-12-27 12:57:51,788][20381] Updated weights for policy 1, policy_version 1054860 (0.0008) [2023-12-27 12:57:51,847][20381] Updated weights for policy 1, policy_version 1054870 (0.0008) [2023-12-27 12:57:52,202][20347] Updated weights for policy 0, policy_version 1055757 (0.0009) [2023-12-27 12:57:52,268][20347] Updated weights for policy 0, policy_version 1055767 (0.0007) [2023-12-27 12:57:52,336][20347] Updated weights for policy 0, policy_version 1055777 (0.0008) [2023-12-27 12:57:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 540393472. Throughput: 0: 10551.8, 1: 10149.6. Samples: 540369476. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:52,363][19170] Avg episode reward: [(0, '922.120'), (1, '735.450')] [2023-12-27 12:57:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001054872_270073856.pth... [2023-12-27 12:57:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001053688_269770752.pth [2023-12-27 12:57:52,379][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001055784_270327808.pth... [2023-12-27 12:57:52,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001054536_270008320.pth [2023-12-27 12:57:52,578][20381] Updated weights for policy 1, policy_version 1054880 (0.0006) [2023-12-27 12:57:52,633][20381] Updated weights for policy 1, policy_version 1054890 (0.0006) [2023-12-27 12:57:52,693][20381] Updated weights for policy 1, policy_version 1054900 (0.0005) [2023-12-27 12:57:53,012][20347] Updated weights for policy 0, policy_version 1055787 (0.0007) [2023-12-27 12:57:53,062][20347] Updated weights for policy 0, policy_version 1055797 (0.0006) [2023-12-27 12:57:53,116][20347] Updated weights for policy 0, policy_version 1055807 (0.0007) [2023-12-27 12:57:53,357][20381] Updated weights for policy 1, policy_version 1054910 (0.0005) [2023-12-27 12:57:53,415][20381] Updated weights for policy 1, policy_version 1054920 (0.0005) [2023-12-27 12:57:53,470][20381] Updated weights for policy 1, policy_version 1054930 (0.0005) [2023-12-27 12:57:53,856][20347] Updated weights for policy 0, policy_version 1055817 (0.0007) [2023-12-27 12:57:53,919][20347] Updated weights for policy 0, policy_version 1055827 (0.0006) [2023-12-27 12:57:53,980][20347] Updated weights for policy 0, policy_version 1055837 (0.0006) [2023-12-27 12:57:54,010][20381] Updated weights for policy 1, policy_version 1054940 (0.0006) [2023-12-27 12:57:54,040][20347] Updated weights for policy 0, policy_version 1055847 (0.0005) [2023-12-27 12:57:54,067][20381] Updated weights for policy 1, policy_version 1054950 (0.0007) [2023-12-27 12:57:54,125][20381] Updated weights for policy 1, policy_version 1054960 (0.0007) [2023-12-27 12:57:54,613][20347] Updated weights for policy 0, policy_version 1055857 (0.0008) [2023-12-27 12:57:54,665][20347] Updated weights for policy 0, policy_version 1055867 (0.0008) [2023-12-27 12:57:54,726][20347] Updated weights for policy 0, policy_version 1055877 (0.0007) [2023-12-27 12:57:54,811][20381] Updated weights for policy 1, policy_version 1054970 (0.0008) [2023-12-27 12:57:54,866][20381] Updated weights for policy 1, policy_version 1054980 (0.0007) [2023-12-27 12:57:54,925][20381] Updated weights for policy 1, policy_version 1054990 (0.0008) [2023-12-27 12:57:54,983][20381] Updated weights for policy 1, policy_version 1055000 (0.0007) [2023-12-27 12:57:55,440][20347] Updated weights for policy 0, policy_version 1055887 (0.0008) [2023-12-27 12:57:55,488][20347] Updated weights for policy 0, policy_version 1055897 (0.0007) [2023-12-27 12:57:55,536][20347] Updated weights for policy 0, policy_version 1055907 (0.0007) [2023-12-27 12:57:55,634][20381] Updated weights for policy 1, policy_version 1055010 (0.0007) [2023-12-27 12:57:55,680][20381] Updated weights for policy 1, policy_version 1055020 (0.0007) [2023-12-27 12:57:55,727][20381] Updated weights for policy 1, policy_version 1055030 (0.0008) [2023-12-27 12:57:56,216][20347] Updated weights for policy 0, policy_version 1055917 (0.0008) [2023-12-27 12:57:56,273][20347] Updated weights for policy 0, policy_version 1055927 (0.0007) [2023-12-27 12:57:56,329][20347] Updated weights for policy 0, policy_version 1055937 (0.0007) [2023-12-27 12:57:56,395][20381] Updated weights for policy 1, policy_version 1055040 (0.0008) [2023-12-27 12:57:56,441][20381] Updated weights for policy 1, policy_version 1055050 (0.0008) [2023-12-27 12:57:56,486][20381] Updated weights for policy 1, policy_version 1055060 (0.0007) [2023-12-27 12:57:57,011][20347] Updated weights for policy 0, policy_version 1055947 (0.0008) [2023-12-27 12:57:57,067][20347] Updated weights for policy 0, policy_version 1055957 (0.0008) [2023-12-27 12:57:57,121][20347] Updated weights for policy 0, policy_version 1055967 (0.0007) [2023-12-27 12:57:57,123][20291] KL-divergence is very high: 114.6413 [2023-12-27 12:57:57,165][20291] KL-divergence is very high: 145.8385 [2023-12-27 12:57:57,181][20381] Updated weights for policy 1, policy_version 1055070 (0.0006) [2023-12-27 12:57:57,235][20381] Updated weights for policy 1, policy_version 1055080 (0.0007) [2023-12-27 12:57:57,291][20381] Updated weights for policy 1, policy_version 1055090 (0.0007) [2023-12-27 12:57:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 540508160. Throughput: 0: 10463.0, 1: 10234.5. Samples: 540492700. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:57:57,363][19170] Avg episode reward: [(0, '931.780'), (1, '794.370')] [2023-12-27 12:57:57,714][20347] Updated weights for policy 0, policy_version 1055977 (0.0008) [2023-12-27 12:57:57,767][20347] Updated weights for policy 0, policy_version 1055987 (0.0009) [2023-12-27 12:57:57,818][20347] Updated weights for policy 0, policy_version 1055997 (0.0009) [2023-12-27 12:57:57,873][20347] Updated weights for policy 0, policy_version 1056007 (0.0009) [2023-12-27 12:57:57,904][20381] Updated weights for policy 1, policy_version 1055100 (0.0006) [2023-12-27 12:57:57,954][20381] Updated weights for policy 1, policy_version 1055110 (0.0006) [2023-12-27 12:57:58,011][20381] Updated weights for policy 1, policy_version 1055120 (0.0009) [2023-12-27 12:57:58,573][20347] Updated weights for policy 0, policy_version 1056017 (0.0011) [2023-12-27 12:57:58,637][20347] Updated weights for policy 0, policy_version 1056027 (0.0008) [2023-12-27 12:57:58,696][20347] Updated weights for policy 0, policy_version 1056037 (0.0007) [2023-12-27 12:57:58,737][20381] Updated weights for policy 1, policy_version 1055130 (0.0010) [2023-12-27 12:57:58,799][20381] Updated weights for policy 1, policy_version 1055140 (0.0009) [2023-12-27 12:57:58,861][20381] Updated weights for policy 1, policy_version 1055150 (0.0008) [2023-12-27 12:57:58,926][20381] Updated weights for policy 1, policy_version 1055160 (0.0008) [2023-12-27 12:57:59,513][20347] Updated weights for policy 0, policy_version 1056047 (0.0008) [2023-12-27 12:57:59,560][20347] Updated weights for policy 0, policy_version 1056057 (0.0008) [2023-12-27 12:57:59,606][20347] Updated weights for policy 0, policy_version 1056067 (0.0009) [2023-12-27 12:57:59,695][20381] Updated weights for policy 1, policy_version 1055170 (0.0010) [2023-12-27 12:57:59,751][20381] Updated weights for policy 1, policy_version 1055180 (0.0009) [2023-12-27 12:57:59,804][20381] Updated weights for policy 1, policy_version 1055190 (0.0011) [2023-12-27 12:58:00,275][20347] Updated weights for policy 0, policy_version 1056077 (0.0009) [2023-12-27 12:58:00,326][20347] Updated weights for policy 0, policy_version 1056087 (0.0007) [2023-12-27 12:58:00,388][20347] Updated weights for policy 0, policy_version 1056097 (0.0006) [2023-12-27 12:58:00,477][20381] Updated weights for policy 1, policy_version 1055200 (0.0007) [2023-12-27 12:58:00,522][20381] Updated weights for policy 1, policy_version 1055210 (0.0005) [2023-12-27 12:58:00,567][20381] Updated weights for policy 1, policy_version 1055220 (0.0005) [2023-12-27 12:58:01,127][20347] Updated weights for policy 0, policy_version 1056107 (0.0007) [2023-12-27 12:58:01,192][20347] Updated weights for policy 0, policy_version 1056117 (0.0008) [2023-12-27 12:58:01,193][20381] Updated weights for policy 1, policy_version 1055230 (0.0005) [2023-12-27 12:58:01,254][20347] Updated weights for policy 0, policy_version 1056127 (0.0008) [2023-12-27 12:58:01,257][20381] Updated weights for policy 1, policy_version 1055240 (0.0007) [2023-12-27 12:58:01,316][20381] Updated weights for policy 1, policy_version 1055250 (0.0008) [2023-12-27 12:58:01,886][20347] Updated weights for policy 0, policy_version 1056137 (0.0005) [2023-12-27 12:58:01,944][20347] Updated weights for policy 0, policy_version 1056147 (0.0008) [2023-12-27 12:58:02,000][20347] Updated weights for policy 0, policy_version 1056157 (0.0008) [2023-12-27 12:58:02,010][20381] Updated weights for policy 1, policy_version 1055260 (0.0007) [2023-12-27 12:58:02,057][20347] Updated weights for policy 0, policy_version 1056167 (0.0007) [2023-12-27 12:58:02,067][20381] Updated weights for policy 1, policy_version 1055270 (0.0008) [2023-12-27 12:58:02,123][20381] Updated weights for policy 1, policy_version 1055280 (0.0008) [2023-12-27 12:58:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 540606464. Throughput: 0: 10387.7, 1: 10235.6. Samples: 540613996. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:02,363][19170] Avg episode reward: [(0, '970.810'), (1, '794.550')] [2023-12-27 12:58:02,697][20347] Updated weights for policy 0, policy_version 1056177 (0.0007) [2023-12-27 12:58:02,756][20347] Updated weights for policy 0, policy_version 1056187 (0.0008) [2023-12-27 12:58:02,812][20347] Updated weights for policy 0, policy_version 1056197 (0.0007) [2023-12-27 12:58:02,854][20381] Updated weights for policy 1, policy_version 1055290 (0.0008) [2023-12-27 12:58:02,912][20381] Updated weights for policy 1, policy_version 1055300 (0.0007) [2023-12-27 12:58:02,971][20381] Updated weights for policy 1, policy_version 1055310 (0.0005) [2023-12-27 12:58:03,030][20381] Updated weights for policy 1, policy_version 1055320 (0.0005) [2023-12-27 12:58:03,473][20347] Updated weights for policy 0, policy_version 1056207 (0.0008) [2023-12-27 12:58:03,533][20347] Updated weights for policy 0, policy_version 1056217 (0.0007) [2023-12-27 12:58:03,601][20347] Updated weights for policy 0, policy_version 1056227 (0.0006) [2023-12-27 12:58:03,690][20381] Updated weights for policy 1, policy_version 1055330 (0.0005) [2023-12-27 12:58:03,742][20381] Updated weights for policy 1, policy_version 1055340 (0.0005) [2023-12-27 12:58:03,792][20381] Updated weights for policy 1, policy_version 1055350 (0.0005) [2023-12-27 12:58:04,264][20347] Updated weights for policy 0, policy_version 1056237 (0.0007) [2023-12-27 12:58:04,327][20347] Updated weights for policy 0, policy_version 1056247 (0.0008) [2023-12-27 12:58:04,385][20347] Updated weights for policy 0, policy_version 1056257 (0.0008) [2023-12-27 12:58:04,477][20381] Updated weights for policy 1, policy_version 1055360 (0.0008) [2023-12-27 12:58:04,526][20381] Updated weights for policy 1, policy_version 1055370 (0.0008) [2023-12-27 12:58:04,575][20381] Updated weights for policy 1, policy_version 1055380 (0.0008) [2023-12-27 12:58:05,001][20347] Updated weights for policy 0, policy_version 1056267 (0.0005) [2023-12-27 12:58:05,062][20347] Updated weights for policy 0, policy_version 1056277 (0.0005) [2023-12-27 12:58:05,116][20347] Updated weights for policy 0, policy_version 1056287 (0.0007) [2023-12-27 12:58:05,294][20381] Updated weights for policy 1, policy_version 1055390 (0.0008) [2023-12-27 12:58:05,342][20381] Updated weights for policy 1, policy_version 1055400 (0.0007) [2023-12-27 12:58:05,394][20381] Updated weights for policy 1, policy_version 1055410 (0.0008) [2023-12-27 12:58:05,771][20347] Updated weights for policy 0, policy_version 1056297 (0.0008) [2023-12-27 12:58:05,820][20347] Updated weights for policy 0, policy_version 1056307 (0.0008) [2023-12-27 12:58:05,880][20347] Updated weights for policy 0, policy_version 1056317 (0.0007) [2023-12-27 12:58:05,930][20347] Updated weights for policy 0, policy_version 1056327 (0.0007) [2023-12-27 12:58:06,048][20381] Updated weights for policy 1, policy_version 1055420 (0.0007) [2023-12-27 12:58:06,107][20381] Updated weights for policy 1, policy_version 1055430 (0.0008) [2023-12-27 12:58:06,167][20381] Updated weights for policy 1, policy_version 1055440 (0.0007) [2023-12-27 12:58:06,598][20347] Updated weights for policy 0, policy_version 1056337 (0.0006) [2023-12-27 12:58:06,647][20347] Updated weights for policy 0, policy_version 1056347 (0.0007) [2023-12-27 12:58:06,700][20347] Updated weights for policy 0, policy_version 1056357 (0.0008) [2023-12-27 12:58:06,917][20381] Updated weights for policy 1, policy_version 1055450 (0.0008) [2023-12-27 12:58:06,974][20381] Updated weights for policy 1, policy_version 1055460 (0.0008) [2023-12-27 12:58:07,030][20381] Updated weights for policy 1, policy_version 1055470 (0.0008) [2023-12-27 12:58:07,086][20381] Updated weights for policy 1, policy_version 1055480 (0.0008) [2023-12-27 12:58:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 540704768. Throughput: 0: 10377.8, 1: 10241.4. Samples: 540676636. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:07,363][19170] Avg episode reward: [(0, '970.640'), (1, '765.240')] [2023-12-27 12:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001055480_270229504.pth... [2023-12-27 12:58:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001054296_269926400.pth [2023-12-27 12:58:07,416][20347] Updated weights for policy 0, policy_version 1056367 (0.0007) [2023-12-27 12:58:07,478][20347] Updated weights for policy 0, policy_version 1056377 (0.0007) [2023-12-27 12:58:07,534][20347] Updated weights for policy 0, policy_version 1056387 (0.0007) [2023-12-27 12:58:07,563][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001056392_270483456.pth... [2023-12-27 12:58:07,567][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001055176_270172160.pth [2023-12-27 12:58:07,814][20381] Updated weights for policy 1, policy_version 1055490 (0.0006) [2023-12-27 12:58:07,873][20381] Updated weights for policy 1, policy_version 1055500 (0.0005) [2023-12-27 12:58:07,928][20381] Updated weights for policy 1, policy_version 1055510 (0.0006) [2023-12-27 12:58:08,143][20347] Updated weights for policy 0, policy_version 1056397 (0.0006) [2023-12-27 12:58:08,197][20347] Updated weights for policy 0, policy_version 1056407 (0.0008) [2023-12-27 12:58:08,246][20347] Updated weights for policy 0, policy_version 1056417 (0.0007) [2023-12-27 12:58:08,553][20381] Updated weights for policy 1, policy_version 1055520 (0.0006) [2023-12-27 12:58:08,609][20381] Updated weights for policy 1, policy_version 1055530 (0.0007) [2023-12-27 12:58:08,662][20381] Updated weights for policy 1, policy_version 1055540 (0.0010) [2023-12-27 12:58:08,924][20347] Updated weights for policy 0, policy_version 1056427 (0.0008) [2023-12-27 12:58:08,982][20347] Updated weights for policy 0, policy_version 1056437 (0.0007) [2023-12-27 12:58:09,037][20347] Updated weights for policy 0, policy_version 1056447 (0.0007) [2023-12-27 12:58:09,274][20381] Updated weights for policy 1, policy_version 1055550 (0.0007) [2023-12-27 12:58:09,332][20381] Updated weights for policy 1, policy_version 1055560 (0.0007) [2023-12-27 12:58:09,403][20381] Updated weights for policy 1, policy_version 1055570 (0.0009) [2023-12-27 12:58:09,802][20347] Updated weights for policy 0, policy_version 1056457 (0.0008) [2023-12-27 12:58:09,871][20347] Updated weights for policy 0, policy_version 1056467 (0.0008) [2023-12-27 12:58:09,932][20347] Updated weights for policy 0, policy_version 1056477 (0.0008) [2023-12-27 12:58:09,993][20347] Updated weights for policy 0, policy_version 1056487 (0.0008) [2023-12-27 12:58:10,129][20381] Updated weights for policy 1, policy_version 1055580 (0.0006) [2023-12-27 12:58:10,187][20381] Updated weights for policy 1, policy_version 1055590 (0.0006) [2023-12-27 12:58:10,243][20381] Updated weights for policy 1, policy_version 1055600 (0.0006) [2023-12-27 12:58:10,733][20347] Updated weights for policy 0, policy_version 1056497 (0.0010) [2023-12-27 12:58:10,789][20347] Updated weights for policy 0, policy_version 1056507 (0.0007) [2023-12-27 12:58:10,849][20347] Updated weights for policy 0, policy_version 1056517 (0.0007) [2023-12-27 12:58:10,885][20381] Updated weights for policy 1, policy_version 1055610 (0.0007) [2023-12-27 12:58:10,936][20381] Updated weights for policy 1, policy_version 1055620 (0.0008) [2023-12-27 12:58:10,984][20381] Updated weights for policy 1, policy_version 1055630 (0.0007) [2023-12-27 12:58:11,041][20381] Updated weights for policy 1, policy_version 1055640 (0.0008) [2023-12-27 12:58:11,572][20347] Updated weights for policy 0, policy_version 1056527 (0.0009) [2023-12-27 12:58:11,639][20347] Updated weights for policy 0, policy_version 1056537 (0.0009) [2023-12-27 12:58:11,706][20347] Updated weights for policy 0, policy_version 1056547 (0.0012) [2023-12-27 12:58:11,817][20381] Updated weights for policy 1, policy_version 1055650 (0.0008) [2023-12-27 12:58:11,866][20381] Updated weights for policy 1, policy_version 1055660 (0.0008) [2023-12-27 12:58:11,922][20381] Updated weights for policy 1, policy_version 1055670 (0.0010) [2023-12-27 12:58:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 540803072. Throughput: 0: 10344.6, 1: 10208.3. Samples: 540798256. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:12,363][19170] Avg episode reward: [(0, '931.670'), (1, '696.610')] [2023-12-27 12:58:12,405][20347] Updated weights for policy 0, policy_version 1056557 (0.0009) [2023-12-27 12:58:12,456][20347] Updated weights for policy 0, policy_version 1056567 (0.0008) [2023-12-27 12:58:12,509][20347] Updated weights for policy 0, policy_version 1056577 (0.0008) [2023-12-27 12:58:12,639][20381] Updated weights for policy 1, policy_version 1055680 (0.0010) [2023-12-27 12:58:12,698][20381] Updated weights for policy 1, policy_version 1055690 (0.0010) [2023-12-27 12:58:12,762][20381] Updated weights for policy 1, policy_version 1055700 (0.0011) [2023-12-27 12:58:13,214][20347] Updated weights for policy 0, policy_version 1056587 (0.0009) [2023-12-27 12:58:13,258][20347] Updated weights for policy 0, policy_version 1056597 (0.0007) [2023-12-27 12:58:13,306][20347] Updated weights for policy 0, policy_version 1056607 (0.0007) [2023-12-27 12:58:13,416][20381] Updated weights for policy 1, policy_version 1055710 (0.0010) [2023-12-27 12:58:13,461][20381] Updated weights for policy 1, policy_version 1055720 (0.0007) [2023-12-27 12:58:13,507][20381] Updated weights for policy 1, policy_version 1055730 (0.0005) [2023-12-27 12:58:13,944][20347] Updated weights for policy 0, policy_version 1056617 (0.0007) [2023-12-27 12:58:14,004][20347] Updated weights for policy 0, policy_version 1056627 (0.0005) [2023-12-27 12:58:14,054][20347] Updated weights for policy 0, policy_version 1056637 (0.0005) [2023-12-27 12:58:14,111][20347] Updated weights for policy 0, policy_version 1056647 (0.0006) [2023-12-27 12:58:14,206][20381] Updated weights for policy 1, policy_version 1055740 (0.0006) [2023-12-27 12:58:14,259][20381] Updated weights for policy 1, policy_version 1055750 (0.0007) [2023-12-27 12:58:14,306][20381] Updated weights for policy 1, policy_version 1055760 (0.0008) [2023-12-27 12:58:14,694][20347] Updated weights for policy 0, policy_version 1056657 (0.0005) [2023-12-27 12:58:14,738][20347] Updated weights for policy 0, policy_version 1056667 (0.0005) [2023-12-27 12:58:14,795][20347] Updated weights for policy 0, policy_version 1056677 (0.0007) [2023-12-27 12:58:14,960][20381] Updated weights for policy 1, policy_version 1055770 (0.0009) [2023-12-27 12:58:15,010][20381] Updated weights for policy 1, policy_version 1055780 (0.0008) [2023-12-27 12:58:15,061][20381] Updated weights for policy 1, policy_version 1055790 (0.0008) [2023-12-27 12:58:15,114][20381] Updated weights for policy 1, policy_version 1055800 (0.0008) [2023-12-27 12:58:15,495][20347] Updated weights for policy 0, policy_version 1056687 (0.0008) [2023-12-27 12:58:15,559][20347] Updated weights for policy 0, policy_version 1056697 (0.0005) [2023-12-27 12:58:15,614][20347] Updated weights for policy 0, policy_version 1056707 (0.0005) [2023-12-27 12:58:15,774][20292] KL-divergence is very high: 105.8947 [2023-12-27 12:58:15,784][20381] Updated weights for policy 1, policy_version 1055810 (0.0006) [2023-12-27 12:58:15,837][20381] Updated weights for policy 1, policy_version 1055820 (0.0005) [2023-12-27 12:58:15,893][20381] Updated weights for policy 1, policy_version 1055830 (0.0005) [2023-12-27 12:58:16,198][20347] Updated weights for policy 0, policy_version 1056717 (0.0006) [2023-12-27 12:58:16,254][20347] Updated weights for policy 0, policy_version 1056727 (0.0007) [2023-12-27 12:58:16,311][20347] Updated weights for policy 0, policy_version 1056737 (0.0008) [2023-12-27 12:58:16,584][20381] Updated weights for policy 1, policy_version 1055840 (0.0007) [2023-12-27 12:58:16,644][20381] Updated weights for policy 1, policy_version 1055850 (0.0007) [2023-12-27 12:58:16,700][20381] Updated weights for policy 1, policy_version 1055860 (0.0005) [2023-12-27 12:58:16,958][20347] Updated weights for policy 0, policy_version 1056747 (0.0007) [2023-12-27 12:58:17,012][20347] Updated weights for policy 0, policy_version 1056757 (0.0005) [2023-12-27 12:58:17,068][20347] Updated weights for policy 0, policy_version 1056767 (0.0006) [2023-12-27 12:58:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 540909568. Throughput: 0: 10365.7, 1: 10179.5. Samples: 540922048. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:17,363][19170] Avg episode reward: [(0, '843.870'), (1, '657.770')] [2023-12-27 12:58:17,369][20381] Updated weights for policy 1, policy_version 1055870 (0.0005) [2023-12-27 12:58:17,417][20381] Updated weights for policy 1, policy_version 1055880 (0.0005) [2023-12-27 12:58:17,465][20381] Updated weights for policy 1, policy_version 1055890 (0.0005) [2023-12-27 12:58:17,748][20347] Updated weights for policy 0, policy_version 1056777 (0.0008) [2023-12-27 12:58:17,809][20347] Updated weights for policy 0, policy_version 1056787 (0.0008) [2023-12-27 12:58:17,860][20347] Updated weights for policy 0, policy_version 1056797 (0.0008) [2023-12-27 12:58:17,916][20347] Updated weights for policy 0, policy_version 1056807 (0.0007) [2023-12-27 12:58:18,084][20381] Updated weights for policy 1, policy_version 1055900 (0.0007) [2023-12-27 12:58:18,139][20381] Updated weights for policy 1, policy_version 1055910 (0.0009) [2023-12-27 12:58:18,192][20381] Updated weights for policy 1, policy_version 1055920 (0.0007) [2023-12-27 12:58:18,618][20347] Updated weights for policy 0, policy_version 1056817 (0.0007) [2023-12-27 12:58:18,672][20347] Updated weights for policy 0, policy_version 1056827 (0.0007) [2023-12-27 12:58:18,722][20347] Updated weights for policy 0, policy_version 1056837 (0.0008) [2023-12-27 12:58:18,931][20381] Updated weights for policy 1, policy_version 1055930 (0.0008) [2023-12-27 12:58:18,985][20381] Updated weights for policy 1, policy_version 1055940 (0.0010) [2023-12-27 12:58:19,031][20381] Updated weights for policy 1, policy_version 1055950 (0.0009) [2023-12-27 12:58:19,081][20381] Updated weights for policy 1, policy_version 1055960 (0.0009) [2023-12-27 12:58:19,380][20347] Updated weights for policy 0, policy_version 1056847 (0.0008) [2023-12-27 12:58:19,435][20347] Updated weights for policy 0, policy_version 1056857 (0.0005) [2023-12-27 12:58:19,492][20347] Updated weights for policy 0, policy_version 1056867 (0.0007) [2023-12-27 12:58:19,811][20381] Updated weights for policy 1, policy_version 1055970 (0.0007) [2023-12-27 12:58:19,875][20381] Updated weights for policy 1, policy_version 1055980 (0.0007) [2023-12-27 12:58:19,943][20381] Updated weights for policy 1, policy_version 1055990 (0.0008) [2023-12-27 12:58:20,212][20347] Updated weights for policy 0, policy_version 1056877 (0.0009) [2023-12-27 12:58:20,275][20347] Updated weights for policy 0, policy_version 1056887 (0.0011) [2023-12-27 12:58:20,342][20347] Updated weights for policy 0, policy_version 1056897 (0.0011) [2023-12-27 12:58:20,590][20381] Updated weights for policy 1, policy_version 1056000 (0.0007) [2023-12-27 12:58:20,654][20381] Updated weights for policy 1, policy_version 1056010 (0.0007) [2023-12-27 12:58:20,723][20381] Updated weights for policy 1, policy_version 1056020 (0.0007) [2023-12-27 12:58:21,057][20347] Updated weights for policy 0, policy_version 1056907 (0.0010) [2023-12-27 12:58:21,122][20347] Updated weights for policy 0, policy_version 1056917 (0.0010) [2023-12-27 12:58:21,183][20347] Updated weights for policy 0, policy_version 1056927 (0.0010) [2023-12-27 12:58:21,377][20381] Updated weights for policy 1, policy_version 1056030 (0.0012) [2023-12-27 12:58:21,439][20381] Updated weights for policy 1, policy_version 1056040 (0.0009) [2023-12-27 12:58:21,497][20381] Updated weights for policy 1, policy_version 1056050 (0.0007) [2023-12-27 12:58:21,945][20347] Updated weights for policy 0, policy_version 1056937 (0.0010) [2023-12-27 12:58:22,006][20347] Updated weights for policy 0, policy_version 1056947 (0.0006) [2023-12-27 12:58:22,057][20347] Updated weights for policy 0, policy_version 1056957 (0.0008) [2023-12-27 12:58:22,116][20347] Updated weights for policy 0, policy_version 1056967 (0.0007) [2023-12-27 12:58:22,145][20381] Updated weights for policy 1, policy_version 1056060 (0.0008) [2023-12-27 12:58:22,197][20381] Updated weights for policy 1, policy_version 1056070 (0.0005) [2023-12-27 12:58:22,258][20381] Updated weights for policy 1, policy_version 1056080 (0.0007) [2023-12-27 12:58:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20604.9). Total num frames: 541016064. Throughput: 0: 10280.3, 1: 10195.5. Samples: 540982960. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:22,363][19170] Avg episode reward: [(0, '912.070'), (1, '667.430')] [2023-12-27 12:58:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001056088_270385152.pth... [2023-12-27 12:58:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001056968_270630912.pth... [2023-12-27 12:58:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001055784_270327808.pth [2023-12-27 12:58:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001054872_270073856.pth [2023-12-27 12:58:22,831][20347] Updated weights for policy 0, policy_version 1056977 (0.0008) [2023-12-27 12:58:22,891][20347] Updated weights for policy 0, policy_version 1056987 (0.0007) [2023-12-27 12:58:22,943][20381] Updated weights for policy 1, policy_version 1056090 (0.0008) [2023-12-27 12:58:22,954][20347] Updated weights for policy 0, policy_version 1056997 (0.0009) [2023-12-27 12:58:23,001][20381] Updated weights for policy 1, policy_version 1056100 (0.0008) [2023-12-27 12:58:23,060][20381] Updated weights for policy 1, policy_version 1056110 (0.0007) [2023-12-27 12:58:23,108][20381] Updated weights for policy 1, policy_version 1056120 (0.0008) [2023-12-27 12:58:23,621][20347] Updated weights for policy 0, policy_version 1057007 (0.0007) [2023-12-27 12:58:23,675][20291] KL-divergence is very high: 152.5188 [2023-12-27 12:58:23,679][20347] Updated weights for policy 0, policy_version 1057017 (0.0005) [2023-12-27 12:58:23,738][20347] Updated weights for policy 0, policy_version 1057027 (0.0005) [2023-12-27 12:58:23,777][20381] Updated weights for policy 1, policy_version 1056130 (0.0007) [2023-12-27 12:58:23,827][20381] Updated weights for policy 1, policy_version 1056140 (0.0005) [2023-12-27 12:58:23,887][20381] Updated weights for policy 1, policy_version 1056150 (0.0005) [2023-12-27 12:58:24,336][20347] Updated weights for policy 0, policy_version 1057037 (0.0006) [2023-12-27 12:58:24,391][20347] Updated weights for policy 0, policy_version 1057047 (0.0007) [2023-12-27 12:58:24,447][20347] Updated weights for policy 0, policy_version 1057057 (0.0007) [2023-12-27 12:58:24,491][20381] Updated weights for policy 1, policy_version 1056160 (0.0006) [2023-12-27 12:58:24,550][20381] Updated weights for policy 1, policy_version 1056170 (0.0008) [2023-12-27 12:58:24,610][20381] Updated weights for policy 1, policy_version 1056180 (0.0007) [2023-12-27 12:58:25,078][20347] Updated weights for policy 0, policy_version 1057067 (0.0006) [2023-12-27 12:58:25,137][20347] Updated weights for policy 0, policy_version 1057077 (0.0005) [2023-12-27 12:58:25,196][20347] Updated weights for policy 0, policy_version 1057087 (0.0005) [2023-12-27 12:58:25,321][20381] Updated weights for policy 1, policy_version 1056190 (0.0008) [2023-12-27 12:58:25,370][20381] Updated weights for policy 1, policy_version 1056200 (0.0005) [2023-12-27 12:58:25,424][20381] Updated weights for policy 1, policy_version 1056210 (0.0006) [2023-12-27 12:58:25,814][20347] Updated weights for policy 0, policy_version 1057097 (0.0006) [2023-12-27 12:58:25,865][20347] Updated weights for policy 0, policy_version 1057107 (0.0008) [2023-12-27 12:58:25,906][20347] Updated weights for policy 0, policy_version 1057117 (0.0007) [2023-12-27 12:58:25,954][20347] Updated weights for policy 0, policy_version 1057127 (0.0007) [2023-12-27 12:58:26,051][20381] Updated weights for policy 1, policy_version 1056220 (0.0006) [2023-12-27 12:58:26,097][20381] Updated weights for policy 1, policy_version 1056230 (0.0008) [2023-12-27 12:58:26,145][20381] Updated weights for policy 1, policy_version 1056240 (0.0008) [2023-12-27 12:58:26,590][20347] Updated weights for policy 0, policy_version 1057137 (0.0005) [2023-12-27 12:58:26,642][20347] Updated weights for policy 0, policy_version 1057147 (0.0005) [2023-12-27 12:58:26,700][20347] Updated weights for policy 0, policy_version 1057157 (0.0005) [2023-12-27 12:58:26,896][20381] Updated weights for policy 1, policy_version 1056250 (0.0007) [2023-12-27 12:58:26,951][20381] Updated weights for policy 1, policy_version 1056260 (0.0008) [2023-12-27 12:58:26,999][20381] Updated weights for policy 1, policy_version 1056270 (0.0007) [2023-12-27 12:58:27,045][20381] Updated weights for policy 1, policy_version 1056280 (0.0008) [2023-12-27 12:58:27,303][20347] Updated weights for policy 0, policy_version 1057167 (0.0006) [2023-12-27 12:58:27,362][20347] Updated weights for policy 0, policy_version 1057177 (0.0007) [2023-12-27 12:58:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 541114368. Throughput: 0: 10244.0, 1: 10241.7. Samples: 541105860. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:27,363][19170] Avg episode reward: [(0, '941.570'), (1, '784.910')] [2023-12-27 12:58:27,411][20347] Updated weights for policy 0, policy_version 1057187 (0.0008) [2023-12-27 12:58:27,778][20381] Updated weights for policy 1, policy_version 1056290 (0.0006) [2023-12-27 12:58:27,823][20381] Updated weights for policy 1, policy_version 1056300 (0.0005) [2023-12-27 12:58:27,877][20381] Updated weights for policy 1, policy_version 1056310 (0.0005) [2023-12-27 12:58:27,958][20347] Updated weights for policy 0, policy_version 1057197 (0.0007) [2023-12-27 12:58:28,008][20347] Updated weights for policy 0, policy_version 1057207 (0.0005) [2023-12-27 12:58:28,062][20347] Updated weights for policy 0, policy_version 1057217 (0.0005) [2023-12-27 12:58:28,509][20381] Updated weights for policy 1, policy_version 1056320 (0.0009) [2023-12-27 12:58:28,572][20381] Updated weights for policy 1, policy_version 1056330 (0.0010) [2023-12-27 12:58:28,632][20381] Updated weights for policy 1, policy_version 1056340 (0.0010) [2023-12-27 12:58:28,720][20347] Updated weights for policy 0, policy_version 1057227 (0.0006) [2023-12-27 12:58:28,775][20347] Updated weights for policy 0, policy_version 1057237 (0.0008) [2023-12-27 12:58:28,839][20347] Updated weights for policy 0, policy_version 1057247 (0.0009) [2023-12-27 12:58:29,269][20381] Updated weights for policy 1, policy_version 1056350 (0.0009) [2023-12-27 12:58:29,332][20381] Updated weights for policy 1, policy_version 1056360 (0.0009) [2023-12-27 12:58:29,396][20381] Updated weights for policy 1, policy_version 1056370 (0.0009) [2023-12-27 12:58:29,543][20347] Updated weights for policy 0, policy_version 1057257 (0.0008) [2023-12-27 12:58:29,590][20347] Updated weights for policy 0, policy_version 1057267 (0.0008) [2023-12-27 12:58:29,647][20347] Updated weights for policy 0, policy_version 1057277 (0.0007) [2023-12-27 12:58:29,701][20347] Updated weights for policy 0, policy_version 1057287 (0.0007) [2023-12-27 12:58:30,071][20381] Updated weights for policy 1, policy_version 1056380 (0.0008) [2023-12-27 12:58:30,127][20381] Updated weights for policy 1, policy_version 1056390 (0.0006) [2023-12-27 12:58:30,184][20381] Updated weights for policy 1, policy_version 1056400 (0.0005) [2023-12-27 12:58:30,469][20347] Updated weights for policy 0, policy_version 1057297 (0.0008) [2023-12-27 12:58:30,521][20347] Updated weights for policy 0, policy_version 1057307 (0.0008) [2023-12-27 12:58:30,567][20347] Updated weights for policy 0, policy_version 1057317 (0.0007) [2023-12-27 12:58:30,784][20381] Updated weights for policy 1, policy_version 1056410 (0.0007) [2023-12-27 12:58:30,832][20381] Updated weights for policy 1, policy_version 1056420 (0.0010) [2023-12-27 12:58:30,876][20381] Updated weights for policy 1, policy_version 1056430 (0.0010) [2023-12-27 12:58:30,924][20381] Updated weights for policy 1, policy_version 1056440 (0.0006) [2023-12-27 12:58:31,266][20347] Updated weights for policy 0, policy_version 1057327 (0.0009) [2023-12-27 12:58:31,327][20347] Updated weights for policy 0, policy_version 1057337 (0.0010) [2023-12-27 12:58:31,395][20347] Updated weights for policy 0, policy_version 1057347 (0.0009) [2023-12-27 12:58:31,642][20381] Updated weights for policy 1, policy_version 1056450 (0.0007) [2023-12-27 12:58:31,706][20381] Updated weights for policy 1, policy_version 1056460 (0.0008) [2023-12-27 12:58:31,762][20381] Updated weights for policy 1, policy_version 1056470 (0.0008) [2023-12-27 12:58:32,033][20347] Updated weights for policy 0, policy_version 1057357 (0.0007) [2023-12-27 12:58:32,096][20347] Updated weights for policy 0, policy_version 1057367 (0.0006) [2023-12-27 12:58:32,147][20347] Updated weights for policy 0, policy_version 1057377 (0.0010) [2023-12-27 12:58:32,357][20381] Updated weights for policy 1, policy_version 1056480 (0.0008) [2023-12-27 12:58:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 541220864. Throughput: 0: 10328.7, 1: 10268.1. Samples: 541232136. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:32,363][19170] Avg episode reward: [(0, '922.580'), (1, '755.200')] [2023-12-27 12:58:32,408][20381] Updated weights for policy 1, policy_version 1056490 (0.0008) [2023-12-27 12:58:32,451][20381] Updated weights for policy 1, policy_version 1056500 (0.0008) [2023-12-27 12:58:32,782][20347] Updated weights for policy 0, policy_version 1057387 (0.0007) [2023-12-27 12:58:32,833][20347] Updated weights for policy 0, policy_version 1057397 (0.0007) [2023-12-27 12:58:32,890][20347] Updated weights for policy 0, policy_version 1057407 (0.0008) [2023-12-27 12:58:33,187][20381] Updated weights for policy 1, policy_version 1056510 (0.0009) [2023-12-27 12:58:33,246][20381] Updated weights for policy 1, policy_version 1056520 (0.0007) [2023-12-27 12:58:33,301][20381] Updated weights for policy 1, policy_version 1056530 (0.0010) [2023-12-27 12:58:33,514][20347] Updated weights for policy 0, policy_version 1057417 (0.0008) [2023-12-27 12:58:33,573][20347] Updated weights for policy 0, policy_version 1057427 (0.0007) [2023-12-27 12:58:33,630][20347] Updated weights for policy 0, policy_version 1057437 (0.0007) [2023-12-27 12:58:33,686][20347] Updated weights for policy 0, policy_version 1057447 (0.0007) [2023-12-27 12:58:33,955][20381] Updated weights for policy 1, policy_version 1056540 (0.0007) [2023-12-27 12:58:34,009][20381] Updated weights for policy 1, policy_version 1056550 (0.0005) [2023-12-27 12:58:34,062][20381] Updated weights for policy 1, policy_version 1056560 (0.0005) [2023-12-27 12:58:34,400][20347] Updated weights for policy 0, policy_version 1057457 (0.0006) [2023-12-27 12:58:34,458][20347] Updated weights for policy 0, policy_version 1057467 (0.0006) [2023-12-27 12:58:34,520][20347] Updated weights for policy 0, policy_version 1057477 (0.0006) [2023-12-27 12:58:34,696][20381] Updated weights for policy 1, policy_version 1056570 (0.0007) [2023-12-27 12:58:34,755][20381] Updated weights for policy 1, policy_version 1056580 (0.0007) [2023-12-27 12:58:34,801][20381] Updated weights for policy 1, policy_version 1056590 (0.0007) [2023-12-27 12:58:34,849][20381] Updated weights for policy 1, policy_version 1056600 (0.0007) [2023-12-27 12:58:35,163][20347] Updated weights for policy 0, policy_version 1057487 (0.0007) [2023-12-27 12:58:35,222][20347] Updated weights for policy 0, policy_version 1057497 (0.0008) [2023-12-27 12:58:35,280][20347] Updated weights for policy 0, policy_version 1057507 (0.0008) [2023-12-27 12:58:35,468][20381] Updated weights for policy 1, policy_version 1056610 (0.0005) [2023-12-27 12:58:35,513][20381] Updated weights for policy 1, policy_version 1056620 (0.0005) [2023-12-27 12:58:35,558][20381] Updated weights for policy 1, policy_version 1056630 (0.0005) [2023-12-27 12:58:35,948][20347] Updated weights for policy 0, policy_version 1057517 (0.0006) [2023-12-27 12:58:35,997][20347] Updated weights for policy 0, policy_version 1057527 (0.0005) [2023-12-27 12:58:36,047][20347] Updated weights for policy 0, policy_version 1057537 (0.0007) [2023-12-27 12:58:36,191][20381] Updated weights for policy 1, policy_version 1056640 (0.0009) [2023-12-27 12:58:36,241][20381] Updated weights for policy 1, policy_version 1056650 (0.0009) [2023-12-27 12:58:36,296][20381] Updated weights for policy 1, policy_version 1056660 (0.0008) [2023-12-27 12:58:36,725][20347] Updated weights for policy 0, policy_version 1057547 (0.0007) [2023-12-27 12:58:36,789][20347] Updated weights for policy 0, policy_version 1057557 (0.0008) [2023-12-27 12:58:36,853][20347] Updated weights for policy 0, policy_version 1057567 (0.0008) [2023-12-27 12:58:36,998][20381] Updated weights for policy 1, policy_version 1056670 (0.0008) [2023-12-27 12:58:37,050][20381] Updated weights for policy 1, policy_version 1056680 (0.0008) [2023-12-27 12:58:37,100][20381] Updated weights for policy 1, policy_version 1056690 (0.0008) [2023-12-27 12:58:37,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 541327360. Throughput: 0: 10305.8, 1: 10264.2. Samples: 541295124. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:37,363][19170] Avg episode reward: [(0, '912.970'), (1, '774.920')] [2023-12-27 12:58:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001056696_270540800.pth... [2023-12-27 12:58:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001057576_270786560.pth... [2023-12-27 12:58:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001056392_270483456.pth [2023-12-27 12:58:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001055480_270229504.pth [2023-12-27 12:58:37,521][20347] Updated weights for policy 0, policy_version 1057577 (0.0008) [2023-12-27 12:58:37,585][20347] Updated weights for policy 0, policy_version 1057587 (0.0007) [2023-12-27 12:58:37,634][20347] Updated weights for policy 0, policy_version 1057597 (0.0007) [2023-12-27 12:58:37,683][20347] Updated weights for policy 0, policy_version 1057607 (0.0007) [2023-12-27 12:58:37,819][20381] Updated weights for policy 1, policy_version 1056700 (0.0007) [2023-12-27 12:58:37,864][20381] Updated weights for policy 1, policy_version 1056710 (0.0007) [2023-12-27 12:58:37,909][20381] Updated weights for policy 1, policy_version 1056720 (0.0007) [2023-12-27 12:58:38,405][20347] Updated weights for policy 0, policy_version 1057617 (0.0009) [2023-12-27 12:58:38,462][20347] Updated weights for policy 0, policy_version 1057627 (0.0008) [2023-12-27 12:58:38,521][20347] Updated weights for policy 0, policy_version 1057637 (0.0007) [2023-12-27 12:58:38,577][20381] Updated weights for policy 1, policy_version 1056730 (0.0008) [2023-12-27 12:58:38,642][20381] Updated weights for policy 1, policy_version 1056740 (0.0008) [2023-12-27 12:58:38,699][20381] Updated weights for policy 1, policy_version 1056750 (0.0008) [2023-12-27 12:58:38,763][20381] Updated weights for policy 1, policy_version 1056760 (0.0009) [2023-12-27 12:58:39,168][20347] Updated weights for policy 0, policy_version 1057647 (0.0008) [2023-12-27 12:58:39,225][20347] Updated weights for policy 0, policy_version 1057657 (0.0008) [2023-12-27 12:58:39,296][20347] Updated weights for policy 0, policy_version 1057667 (0.0007) [2023-12-27 12:58:39,515][20381] Updated weights for policy 1, policy_version 1056770 (0.0006) [2023-12-27 12:58:39,576][20381] Updated weights for policy 1, policy_version 1056780 (0.0007) [2023-12-27 12:58:39,636][20381] Updated weights for policy 1, policy_version 1056790 (0.0006) [2023-12-27 12:58:40,093][20347] Updated weights for policy 0, policy_version 1057677 (0.0008) [2023-12-27 12:58:40,146][20347] Updated weights for policy 0, policy_version 1057687 (0.0008) [2023-12-27 12:58:40,202][20347] Updated weights for policy 0, policy_version 1057697 (0.0008) [2023-12-27 12:58:40,298][20381] Updated weights for policy 1, policy_version 1056800 (0.0008) [2023-12-27 12:58:40,361][20381] Updated weights for policy 1, policy_version 1056810 (0.0007) [2023-12-27 12:58:40,418][20381] Updated weights for policy 1, policy_version 1056820 (0.0007) [2023-12-27 12:58:40,984][20347] Updated weights for policy 0, policy_version 1057707 (0.0009) [2023-12-27 12:58:41,048][20347] Updated weights for policy 0, policy_version 1057717 (0.0008) [2023-12-27 12:58:41,082][20381] Updated weights for policy 1, policy_version 1056830 (0.0007) [2023-12-27 12:58:41,108][20347] Updated weights for policy 0, policy_version 1057727 (0.0008) [2023-12-27 12:58:41,148][20381] Updated weights for policy 1, policy_version 1056840 (0.0007) [2023-12-27 12:58:41,211][20381] Updated weights for policy 1, policy_version 1056850 (0.0008) [2023-12-27 12:58:41,910][20347] Updated weights for policy 0, policy_version 1057737 (0.0007) [2023-12-27 12:58:41,938][20381] Updated weights for policy 1, policy_version 1056860 (0.0008) [2023-12-27 12:58:41,977][20347] Updated weights for policy 0, policy_version 1057747 (0.0006) [2023-12-27 12:58:41,998][20381] Updated weights for policy 1, policy_version 1056870 (0.0006) [2023-12-27 12:58:42,035][20347] Updated weights for policy 0, policy_version 1057757 (0.0007) [2023-12-27 12:58:42,055][20381] Updated weights for policy 1, policy_version 1056880 (0.0007) [2023-12-27 12:58:42,085][20347] Updated weights for policy 0, policy_version 1057767 (0.0006) [2023-12-27 12:58:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 541425664. Throughput: 0: 10247.0, 1: 10250.3. Samples: 541415080. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:42,363][19170] Avg episode reward: [(0, '902.560'), (1, '716.160')] [2023-12-27 12:58:42,756][20381] Updated weights for policy 1, policy_version 1056890 (0.0007) [2023-12-27 12:58:42,793][20347] Updated weights for policy 0, policy_version 1057777 (0.0006) [2023-12-27 12:58:42,806][20381] Updated weights for policy 1, policy_version 1056900 (0.0005) [2023-12-27 12:58:42,843][20347] Updated weights for policy 0, policy_version 1057787 (0.0006) [2023-12-27 12:58:42,861][20381] Updated weights for policy 1, policy_version 1056910 (0.0007) [2023-12-27 12:58:42,895][20347] Updated weights for policy 0, policy_version 1057797 (0.0007) [2023-12-27 12:58:42,916][20381] Updated weights for policy 1, policy_version 1056920 (0.0006) [2023-12-27 12:58:43,603][20381] Updated weights for policy 1, policy_version 1056930 (0.0006) [2023-12-27 12:58:43,605][20347] Updated weights for policy 0, policy_version 1057807 (0.0007) [2023-12-27 12:58:43,655][20347] Updated weights for policy 0, policy_version 1057817 (0.0006) [2023-12-27 12:58:43,661][20381] Updated weights for policy 1, policy_version 1056940 (0.0006) [2023-12-27 12:58:43,709][20347] Updated weights for policy 0, policy_version 1057827 (0.0006) [2023-12-27 12:58:43,719][20381] Updated weights for policy 1, policy_version 1056950 (0.0007) [2023-12-27 12:58:44,351][20347] Updated weights for policy 0, policy_version 1057837 (0.0006) [2023-12-27 12:58:44,411][20347] Updated weights for policy 0, policy_version 1057847 (0.0006) [2023-12-27 12:58:44,422][20381] Updated weights for policy 1, policy_version 1056960 (0.0008) [2023-12-27 12:58:44,459][20347] Updated weights for policy 0, policy_version 1057857 (0.0007) [2023-12-27 12:58:44,482][20381] Updated weights for policy 1, policy_version 1056970 (0.0007) [2023-12-27 12:58:44,544][20381] Updated weights for policy 1, policy_version 1056980 (0.0009) [2023-12-27 12:58:45,143][20381] Updated weights for policy 1, policy_version 1056990 (0.0008) [2023-12-27 12:58:45,193][20381] Updated weights for policy 1, policy_version 1057000 (0.0011) [2023-12-27 12:58:45,203][20347] Updated weights for policy 0, policy_version 1057867 (0.0008) [2023-12-27 12:58:45,244][20381] Updated weights for policy 1, policy_version 1057010 (0.0011) [2023-12-27 12:58:45,265][20347] Updated weights for policy 0, policy_version 1057877 (0.0006) [2023-12-27 12:58:45,325][20347] Updated weights for policy 0, policy_version 1057887 (0.0007) [2023-12-27 12:58:45,915][20381] Updated weights for policy 1, policy_version 1057020 (0.0009) [2023-12-27 12:58:45,968][20381] Updated weights for policy 1, policy_version 1057030 (0.0006) [2023-12-27 12:58:46,015][20381] Updated weights for policy 1, policy_version 1057040 (0.0005) [2023-12-27 12:58:46,035][20347] Updated weights for policy 0, policy_version 1057897 (0.0007) [2023-12-27 12:58:46,091][20347] Updated weights for policy 0, policy_version 1057907 (0.0006) [2023-12-27 12:58:46,147][20347] Updated weights for policy 0, policy_version 1057917 (0.0008) [2023-12-27 12:58:46,195][20347] Updated weights for policy 0, policy_version 1057927 (0.0008) [2023-12-27 12:58:46,601][20381] Updated weights for policy 1, policy_version 1057050 (0.0008) [2023-12-27 12:58:46,653][20381] Updated weights for policy 1, policy_version 1057060 (0.0007) [2023-12-27 12:58:46,709][20381] Updated weights for policy 1, policy_version 1057070 (0.0008) [2023-12-27 12:58:46,769][20381] Updated weights for policy 1, policy_version 1057080 (0.0008) [2023-12-27 12:58:46,925][20347] Updated weights for policy 0, policy_version 1057937 (0.0007) [2023-12-27 12:58:46,987][20347] Updated weights for policy 0, policy_version 1057947 (0.0007) [2023-12-27 12:58:47,045][20347] Updated weights for policy 0, policy_version 1057957 (0.0008) [2023-12-27 12:58:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 541523968. Throughput: 0: 10199.8, 1: 10263.7. Samples: 541534856. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:47,363][19170] Avg episode reward: [(0, '902.260'), (1, '696.380')] [2023-12-27 12:58:47,452][20381] Updated weights for policy 1, policy_version 1057090 (0.0007) [2023-12-27 12:58:47,498][20381] Updated weights for policy 1, policy_version 1057100 (0.0007) [2023-12-27 12:58:47,556][20381] Updated weights for policy 1, policy_version 1057110 (0.0008) [2023-12-27 12:58:47,749][20347] Updated weights for policy 0, policy_version 1057967 (0.0007) [2023-12-27 12:58:47,805][20347] Updated weights for policy 0, policy_version 1057977 (0.0006) [2023-12-27 12:58:47,865][20347] Updated weights for policy 0, policy_version 1057987 (0.0007) [2023-12-27 12:58:48,271][20381] Updated weights for policy 1, policy_version 1057120 (0.0009) [2023-12-27 12:58:48,329][20381] Updated weights for policy 1, policy_version 1057130 (0.0007) [2023-12-27 12:58:48,391][20381] Updated weights for policy 1, policy_version 1057140 (0.0008) [2023-12-27 12:58:48,588][20347] Updated weights for policy 0, policy_version 1057997 (0.0007) [2023-12-27 12:58:48,652][20347] Updated weights for policy 0, policy_version 1058007 (0.0008) [2023-12-27 12:58:48,712][20347] Updated weights for policy 0, policy_version 1058017 (0.0008) [2023-12-27 12:58:49,114][20381] Updated weights for policy 1, policy_version 1057150 (0.0008) [2023-12-27 12:58:49,172][20381] Updated weights for policy 1, policy_version 1057160 (0.0008) [2023-12-27 12:58:49,235][20381] Updated weights for policy 1, policy_version 1057170 (0.0008) [2023-12-27 12:58:49,410][20347] Updated weights for policy 0, policy_version 1058027 (0.0007) [2023-12-27 12:58:49,471][20347] Updated weights for policy 0, policy_version 1058037 (0.0008) [2023-12-27 12:58:49,530][20347] Updated weights for policy 0, policy_version 1058047 (0.0006) [2023-12-27 12:58:49,929][20381] Updated weights for policy 1, policy_version 1057180 (0.0008) [2023-12-27 12:58:49,993][20381] Updated weights for policy 1, policy_version 1057190 (0.0007) [2023-12-27 12:58:50,052][20381] Updated weights for policy 1, policy_version 1057200 (0.0008) [2023-12-27 12:58:50,185][20347] Updated weights for policy 0, policy_version 1058057 (0.0008) [2023-12-27 12:58:50,243][20347] Updated weights for policy 0, policy_version 1058067 (0.0007) [2023-12-27 12:58:50,294][20347] Updated weights for policy 0, policy_version 1058077 (0.0008) [2023-12-27 12:58:50,353][20347] Updated weights for policy 0, policy_version 1058087 (0.0008) [2023-12-27 12:58:50,811][20381] Updated weights for policy 1, policy_version 1057210 (0.0008) [2023-12-27 12:58:50,868][20381] Updated weights for policy 1, policy_version 1057220 (0.0008) [2023-12-27 12:58:50,924][20381] Updated weights for policy 1, policy_version 1057230 (0.0007) [2023-12-27 12:58:50,989][20381] Updated weights for policy 1, policy_version 1057240 (0.0009) [2023-12-27 12:58:51,046][20347] Updated weights for policy 0, policy_version 1058097 (0.0008) [2023-12-27 12:58:51,110][20347] Updated weights for policy 0, policy_version 1058107 (0.0008) [2023-12-27 12:58:51,178][20347] Updated weights for policy 0, policy_version 1058117 (0.0006) [2023-12-27 12:58:51,714][20381] Updated weights for policy 1, policy_version 1057250 (0.0008) [2023-12-27 12:58:51,784][20381] Updated weights for policy 1, policy_version 1057260 (0.0008) [2023-12-27 12:58:51,848][20381] Updated weights for policy 1, policy_version 1057270 (0.0006) [2023-12-27 12:58:51,881][20347] Updated weights for policy 0, policy_version 1058127 (0.0007) [2023-12-27 12:58:51,944][20347] Updated weights for policy 0, policy_version 1058137 (0.0007) [2023-12-27 12:58:51,997][20347] Updated weights for policy 0, policy_version 1058147 (0.0008) [2023-12-27 12:58:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 541622272. Throughput: 0: 10144.8, 1: 10243.2. Samples: 541594096. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:52,363][19170] Avg episode reward: [(0, '912.030'), (1, '745.490')] [2023-12-27 12:58:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001057272_270688256.pth... [2023-12-27 12:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001058152_270934016.pth... [2023-12-27 12:58:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001056088_270385152.pth [2023-12-27 12:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001056968_270630912.pth [2023-12-27 12:58:52,543][20381] Updated weights for policy 1, policy_version 1057280 (0.0007) [2023-12-27 12:58:52,602][20381] Updated weights for policy 1, policy_version 1057290 (0.0007) [2023-12-27 12:58:52,661][20381] Updated weights for policy 1, policy_version 1057300 (0.0008) [2023-12-27 12:58:52,675][20347] Updated weights for policy 0, policy_version 1058157 (0.0007) [2023-12-27 12:58:52,720][20347] Updated weights for policy 0, policy_version 1058167 (0.0005) [2023-12-27 12:58:52,770][20347] Updated weights for policy 0, policy_version 1058177 (0.0006) [2023-12-27 12:58:53,282][20381] Updated weights for policy 1, policy_version 1057310 (0.0006) [2023-12-27 12:58:53,337][20381] Updated weights for policy 1, policy_version 1057320 (0.0005) [2023-12-27 12:58:53,402][20381] Updated weights for policy 1, policy_version 1057330 (0.0005) [2023-12-27 12:58:53,453][20347] Updated weights for policy 0, policy_version 1058187 (0.0006) [2023-12-27 12:58:53,503][20347] Updated weights for policy 0, policy_version 1058197 (0.0005) [2023-12-27 12:58:53,552][20347] Updated weights for policy 0, policy_version 1058207 (0.0005) [2023-12-27 12:58:53,907][20381] Updated weights for policy 1, policy_version 1057340 (0.0006) [2023-12-27 12:58:53,956][20381] Updated weights for policy 1, policy_version 1057350 (0.0005) [2023-12-27 12:58:54,006][20381] Updated weights for policy 1, policy_version 1057360 (0.0006) [2023-12-27 12:58:54,205][20347] Updated weights for policy 0, policy_version 1058217 (0.0005) [2023-12-27 12:58:54,256][20347] Updated weights for policy 0, policy_version 1058227 (0.0008) [2023-12-27 12:58:54,308][20347] Updated weights for policy 0, policy_version 1058237 (0.0008) [2023-12-27 12:58:54,366][20347] Updated weights for policy 0, policy_version 1058247 (0.0006) [2023-12-27 12:58:54,684][20381] Updated weights for policy 1, policy_version 1057370 (0.0008) [2023-12-27 12:58:54,737][20381] Updated weights for policy 1, policy_version 1057380 (0.0009) [2023-12-27 12:58:54,790][20381] Updated weights for policy 1, policy_version 1057390 (0.0009) [2023-12-27 12:58:54,834][20381] Updated weights for policy 1, policy_version 1057400 (0.0007) [2023-12-27 12:58:55,006][20347] Updated weights for policy 0, policy_version 1058257 (0.0007) [2023-12-27 12:58:55,060][20347] Updated weights for policy 0, policy_version 1058267 (0.0008) [2023-12-27 12:58:55,109][20347] Updated weights for policy 0, policy_version 1058277 (0.0007) [2023-12-27 12:58:55,590][20381] Updated weights for policy 1, policy_version 1057410 (0.0008) [2023-12-27 12:58:55,634][20381] Updated weights for policy 1, policy_version 1057420 (0.0008) [2023-12-27 12:58:55,679][20381] Updated weights for policy 1, policy_version 1057430 (0.0008) [2023-12-27 12:58:55,741][20347] Updated weights for policy 0, policy_version 1058287 (0.0005) [2023-12-27 12:58:55,791][20347] Updated weights for policy 0, policy_version 1058297 (0.0005) [2023-12-27 12:58:55,824][20291] KL-divergence is very high: 446.4030 [2023-12-27 12:58:55,830][20291] KL-divergence is very high: 464.9156 [2023-12-27 12:58:55,835][20291] KL-divergence is very high: 437.6052 [2023-12-27 12:58:55,840][20347] Updated weights for policy 0, policy_version 1058307 (0.0005) [2023-12-27 12:58:55,841][20291] KL-divergence is very high: 238.7005 [2023-12-27 12:58:55,846][20291] KL-divergence is very high: 242.8498 [2023-12-27 12:58:56,394][20381] Updated weights for policy 1, policy_version 1057440 (0.0008) [2023-12-27 12:58:56,440][20381] Updated weights for policy 1, policy_version 1057450 (0.0008) [2023-12-27 12:58:56,483][20291] KL-divergence is very high: 100.8429 [2023-12-27 12:58:56,491][20381] Updated weights for policy 1, policy_version 1057460 (0.0007) [2023-12-27 12:58:56,502][20347] Updated weights for policy 0, policy_version 1058317 (0.0007) [2023-12-27 12:58:56,523][20291] KL-divergence is very high: 166.6339 [2023-12-27 12:58:56,554][20347] Updated weights for policy 0, policy_version 1058327 (0.0007) [2023-12-27 12:58:56,605][20347] Updated weights for policy 0, policy_version 1058337 (0.0007) [2023-12-27 12:58:57,169][20381] Updated weights for policy 1, policy_version 1057470 (0.0005) [2023-12-27 12:58:57,226][20381] Updated weights for policy 1, policy_version 1057480 (0.0005) [2023-12-27 12:58:57,276][20347] Updated weights for policy 0, policy_version 1058347 (0.0007) [2023-12-27 12:58:57,280][20381] Updated weights for policy 1, policy_version 1057490 (0.0005) [2023-12-27 12:58:57,328][20347] Updated weights for policy 0, policy_version 1058357 (0.0006) [2023-12-27 12:58:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 541728768. Throughput: 0: 10185.2, 1: 10270.2. Samples: 541718748. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:58:57,363][19170] Avg episode reward: [(0, '882.950'), (1, '843.350')] [2023-12-27 12:58:57,387][20347] Updated weights for policy 0, policy_version 1058367 (0.0007) [2023-12-27 12:58:57,831][20381] Updated weights for policy 1, policy_version 1057500 (0.0006) [2023-12-27 12:58:57,882][20381] Updated weights for policy 1, policy_version 1057510 (0.0007) [2023-12-27 12:58:57,929][20381] Updated weights for policy 1, policy_version 1057520 (0.0008) [2023-12-27 12:58:58,055][20347] Updated weights for policy 0, policy_version 1058377 (0.0009) [2023-12-27 12:58:58,114][20347] Updated weights for policy 0, policy_version 1058387 (0.0008) [2023-12-27 12:58:58,170][20347] Updated weights for policy 0, policy_version 1058397 (0.0007) [2023-12-27 12:58:58,229][20347] Updated weights for policy 0, policy_version 1058407 (0.0007) [2023-12-27 12:58:58,684][20381] Updated weights for policy 1, policy_version 1057530 (0.0007) [2023-12-27 12:58:58,744][20381] Updated weights for policy 1, policy_version 1057540 (0.0008) [2023-12-27 12:58:58,811][20381] Updated weights for policy 1, policy_version 1057550 (0.0008) [2023-12-27 12:58:58,875][20381] Updated weights for policy 1, policy_version 1057560 (0.0009) [2023-12-27 12:58:59,033][20347] Updated weights for policy 0, policy_version 1058417 (0.0010) [2023-12-27 12:58:59,035][20291] KL-divergence is very high: 115.8157 [2023-12-27 12:58:59,091][20347] Updated weights for policy 0, policy_version 1058427 (0.0010) [2023-12-27 12:58:59,148][20347] Updated weights for policy 0, policy_version 1058437 (0.0010) [2023-12-27 12:58:59,621][20381] Updated weights for policy 1, policy_version 1057570 (0.0010) [2023-12-27 12:58:59,676][20381] Updated weights for policy 1, policy_version 1057580 (0.0010) [2023-12-27 12:58:59,727][20381] Updated weights for policy 1, policy_version 1057590 (0.0010) [2023-12-27 12:58:59,884][20347] Updated weights for policy 0, policy_version 1058447 (0.0008) [2023-12-27 12:58:59,946][20347] Updated weights for policy 0, policy_version 1058457 (0.0008) [2023-12-27 12:58:59,998][20347] Updated weights for policy 0, policy_version 1058467 (0.0007) [2023-12-27 12:59:00,472][20381] Updated weights for policy 1, policy_version 1057600 (0.0010) [2023-12-27 12:59:00,518][20381] Updated weights for policy 1, policy_version 1057610 (0.0007) [2023-12-27 12:59:00,565][20381] Updated weights for policy 1, policy_version 1057620 (0.0007) [2023-12-27 12:59:00,635][20347] Updated weights for policy 0, policy_version 1058477 (0.0006) [2023-12-27 12:59:00,683][20347] Updated weights for policy 0, policy_version 1058487 (0.0010) [2023-12-27 12:59:00,740][20347] Updated weights for policy 0, policy_version 1058497 (0.0010) [2023-12-27 12:59:01,203][20381] Updated weights for policy 1, policy_version 1057630 (0.0009) [2023-12-27 12:59:01,267][20381] Updated weights for policy 1, policy_version 1057640 (0.0008) [2023-12-27 12:59:01,322][20381] Updated weights for policy 1, policy_version 1057650 (0.0008) [2023-12-27 12:59:01,443][20347] Updated weights for policy 0, policy_version 1058507 (0.0010) [2023-12-27 12:59:01,498][20347] Updated weights for policy 0, policy_version 1058517 (0.0011) [2023-12-27 12:59:01,556][20347] Updated weights for policy 0, policy_version 1058527 (0.0010) [2023-12-27 12:59:01,991][20381] Updated weights for policy 1, policy_version 1057660 (0.0008) [2023-12-27 12:59:02,045][20381] Updated weights for policy 1, policy_version 1057670 (0.0007) [2023-12-27 12:59:02,106][20381] Updated weights for policy 1, policy_version 1057680 (0.0008) [2023-12-27 12:59:02,276][20347] Updated weights for policy 0, policy_version 1058537 (0.0010) [2023-12-27 12:59:02,338][20347] Updated weights for policy 0, policy_version 1058547 (0.0008) [2023-12-27 12:59:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 541827072. Throughput: 0: 10112.3, 1: 10268.4. Samples: 541839184. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:59:02,363][19170] Avg episode reward: [(0, '892.540'), (1, '784.640')] [2023-12-27 12:59:02,398][20347] Updated weights for policy 0, policy_version 1058557 (0.0007) [2023-12-27 12:59:02,456][20347] Updated weights for policy 0, policy_version 1058567 (0.0007) [2023-12-27 12:59:02,812][20381] Updated weights for policy 1, policy_version 1057690 (0.0007) [2023-12-27 12:59:02,864][20381] Updated weights for policy 1, policy_version 1057700 (0.0007) [2023-12-27 12:59:02,870][20292] KL-divergence is very high: 122.6769 [2023-12-27 12:59:02,923][20381] Updated weights for policy 1, policy_version 1057710 (0.0008) [2023-12-27 12:59:02,981][20381] Updated weights for policy 1, policy_version 1057720 (0.0008) [2023-12-27 12:59:03,067][20347] Updated weights for policy 0, policy_version 1058577 (0.0010) [2023-12-27 12:59:03,124][20347] Updated weights for policy 0, policy_version 1058587 (0.0008) [2023-12-27 12:59:03,185][20347] Updated weights for policy 0, policy_version 1058597 (0.0005) [2023-12-27 12:59:03,672][20381] Updated weights for policy 1, policy_version 1057730 (0.0007) [2023-12-27 12:59:03,720][20381] Updated weights for policy 1, policy_version 1057740 (0.0008) [2023-12-27 12:59:03,767][20347] Updated weights for policy 0, policy_version 1058607 (0.0009) [2023-12-27 12:59:03,772][20381] Updated weights for policy 1, policy_version 1057750 (0.0006) [2023-12-27 12:59:03,829][20347] Updated weights for policy 0, policy_version 1058617 (0.0010) [2023-12-27 12:59:03,895][20347] Updated weights for policy 0, policy_version 1058627 (0.0007) [2023-12-27 12:59:04,462][20381] Updated weights for policy 1, policy_version 1057760 (0.0010) [2023-12-27 12:59:04,518][20381] Updated weights for policy 1, policy_version 1057770 (0.0011) [2023-12-27 12:59:04,565][20347] Updated weights for policy 0, policy_version 1058637 (0.0007) [2023-12-27 12:59:04,569][20381] Updated weights for policy 1, policy_version 1057780 (0.0008) [2023-12-27 12:59:04,617][20347] Updated weights for policy 0, policy_version 1058647 (0.0007) [2023-12-27 12:59:04,673][20347] Updated weights for policy 0, policy_version 1058657 (0.0005) [2023-12-27 12:59:04,688][20291] KL-divergence is very high: 313.5185 [2023-12-27 12:59:05,282][20381] Updated weights for policy 1, policy_version 1057790 (0.0007) [2023-12-27 12:59:05,328][20347] Updated weights for policy 0, policy_version 1058667 (0.0008) [2023-12-27 12:59:05,342][20381] Updated weights for policy 1, policy_version 1057800 (0.0008) [2023-12-27 12:59:05,387][20347] Updated weights for policy 0, policy_version 1058677 (0.0008) [2023-12-27 12:59:05,392][20381] Updated weights for policy 1, policy_version 1057810 (0.0007) [2023-12-27 12:59:05,435][20347] Updated weights for policy 0, policy_version 1058687 (0.0008) [2023-12-27 12:59:05,984][20381] Updated weights for policy 1, policy_version 1057820 (0.0008) [2023-12-27 12:59:06,043][20347] Updated weights for policy 0, policy_version 1058697 (0.0007) [2023-12-27 12:59:06,046][20381] Updated weights for policy 1, policy_version 1057830 (0.0009) [2023-12-27 12:59:06,100][20381] Updated weights for policy 1, policy_version 1057840 (0.0008) [2023-12-27 12:59:06,100][20347] Updated weights for policy 0, policy_version 1058707 (0.0006) [2023-12-27 12:59:06,162][20347] Updated weights for policy 0, policy_version 1058717 (0.0008) [2023-12-27 12:59:06,224][20347] Updated weights for policy 0, policy_version 1058727 (0.0007) [2023-12-27 12:59:06,826][20381] Updated weights for policy 1, policy_version 1057850 (0.0009) [2023-12-27 12:59:06,859][20347] Updated weights for policy 0, policy_version 1058737 (0.0006) [2023-12-27 12:59:06,876][20381] Updated weights for policy 1, policy_version 1057860 (0.0007) [2023-12-27 12:59:06,919][20381] Updated weights for policy 1, policy_version 1057870 (0.0008) [2023-12-27 12:59:06,924][20347] Updated weights for policy 0, policy_version 1058747 (0.0006) [2023-12-27 12:59:06,967][20381] Updated weights for policy 1, policy_version 1057880 (0.0008) [2023-12-27 12:59:06,984][20347] Updated weights for policy 0, policy_version 1058757 (0.0006) [2023-12-27 12:59:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 541933568. Throughput: 0: 10158.2, 1: 10254.0. Samples: 541901508. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:59:07,363][19170] Avg episode reward: [(0, '912.500'), (1, '735.740')] [2023-12-27 12:59:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001057880_270843904.pth... [2023-12-27 12:59:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001058760_271089664.pth... [2023-12-27 12:59:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001056696_270540800.pth [2023-12-27 12:59:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001057576_270786560.pth [2023-12-27 12:59:07,540][20347] Updated weights for policy 0, policy_version 1058767 (0.0007) [2023-12-27 12:59:07,589][20347] Updated weights for policy 0, policy_version 1058777 (0.0007) [2023-12-27 12:59:07,645][20347] Updated weights for policy 0, policy_version 1058787 (0.0008) [2023-12-27 12:59:07,739][20381] Updated weights for policy 1, policy_version 1057890 (0.0007) [2023-12-27 12:59:07,795][20381] Updated weights for policy 1, policy_version 1057900 (0.0008) [2023-12-27 12:59:07,857][20381] Updated weights for policy 1, policy_version 1057910 (0.0008) [2023-12-27 12:59:08,277][20347] Updated weights for policy 0, policy_version 1058797 (0.0007) [2023-12-27 12:59:08,338][20347] Updated weights for policy 0, policy_version 1058807 (0.0008) [2023-12-27 12:59:08,398][20347] Updated weights for policy 0, policy_version 1058817 (0.0008) [2023-12-27 12:59:08,599][20381] Updated weights for policy 1, policy_version 1057920 (0.0008) [2023-12-27 12:59:08,652][20381] Updated weights for policy 1, policy_version 1057930 (0.0008) [2023-12-27 12:59:08,701][20381] Updated weights for policy 1, policy_version 1057940 (0.0008) [2023-12-27 12:59:09,066][20347] Updated weights for policy 0, policy_version 1058827 (0.0007) [2023-12-27 12:59:09,093][20291] KL-divergence is very high: 209.1045 [2023-12-27 12:59:09,121][20347] Updated weights for policy 0, policy_version 1058837 (0.0005) [2023-12-27 12:59:09,137][20291] KL-divergence is very high: 210.6190 [2023-12-27 12:59:09,172][20347] Updated weights for policy 0, policy_version 1058847 (0.0005) [2023-12-27 12:59:09,470][20381] Updated weights for policy 1, policy_version 1057950 (0.0007) [2023-12-27 12:59:09,532][20381] Updated weights for policy 1, policy_version 1057960 (0.0008) [2023-12-27 12:59:09,586][20381] Updated weights for policy 1, policy_version 1057970 (0.0009) [2023-12-27 12:59:09,771][20347] Updated weights for policy 0, policy_version 1058857 (0.0006) [2023-12-27 12:59:09,835][20347] Updated weights for policy 0, policy_version 1058867 (0.0009) [2023-12-27 12:59:09,897][20347] Updated weights for policy 0, policy_version 1058877 (0.0008) [2023-12-27 12:59:09,956][20347] Updated weights for policy 0, policy_version 1058887 (0.0008) [2023-12-27 12:59:10,304][20381] Updated weights for policy 1, policy_version 1057980 (0.0007) [2023-12-27 12:59:10,374][20381] Updated weights for policy 1, policy_version 1057990 (0.0007) [2023-12-27 12:59:10,434][20381] Updated weights for policy 1, policy_version 1058000 (0.0007) [2023-12-27 12:59:10,685][20347] Updated weights for policy 0, policy_version 1058897 (0.0009) [2023-12-27 12:59:10,749][20347] Updated weights for policy 0, policy_version 1058907 (0.0008) [2023-12-27 12:59:10,805][20347] Updated weights for policy 0, policy_version 1058917 (0.0009) [2023-12-27 12:59:11,058][20381] Updated weights for policy 1, policy_version 1058010 (0.0008) [2023-12-27 12:59:11,126][20381] Updated weights for policy 1, policy_version 1058020 (0.0008) [2023-12-27 12:59:11,203][20381] Updated weights for policy 1, policy_version 1058030 (0.0007) [2023-12-27 12:59:11,269][20381] Updated weights for policy 1, policy_version 1058040 (0.0008) [2023-12-27 12:59:11,573][20347] Updated weights for policy 0, policy_version 1058927 (0.0008) [2023-12-27 12:59:11,636][20347] Updated weights for policy 0, policy_version 1058937 (0.0008) [2023-12-27 12:59:11,695][20347] Updated weights for policy 0, policy_version 1058947 (0.0008) [2023-12-27 12:59:12,004][20381] Updated weights for policy 1, policy_version 1058050 (0.0008) [2023-12-27 12:59:12,069][20381] Updated weights for policy 1, policy_version 1058060 (0.0007) [2023-12-27 12:59:12,123][20381] Updated weights for policy 1, policy_version 1058070 (0.0007) [2023-12-27 12:59:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 542031872. Throughput: 0: 10231.5, 1: 10184.2. Samples: 542024564. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:59:12,363][19170] Avg episode reward: [(0, '961.130'), (1, '765.460')] [2023-12-27 12:59:12,434][20347] Updated weights for policy 0, policy_version 1058957 (0.0007) [2023-12-27 12:59:12,482][20347] Updated weights for policy 0, policy_version 1058967 (0.0007) [2023-12-27 12:59:12,533][20347] Updated weights for policy 0, policy_version 1058977 (0.0007) [2023-12-27 12:59:12,818][20381] Updated weights for policy 1, policy_version 1058080 (0.0008) [2023-12-27 12:59:12,877][20381] Updated weights for policy 1, policy_version 1058090 (0.0006) [2023-12-27 12:59:12,948][20381] Updated weights for policy 1, policy_version 1058100 (0.0011) [2023-12-27 12:59:13,224][20347] Updated weights for policy 0, policy_version 1058987 (0.0006) [2023-12-27 12:59:13,274][20347] Updated weights for policy 0, policy_version 1058997 (0.0008) [2023-12-27 12:59:13,323][20347] Updated weights for policy 0, policy_version 1059007 (0.0009) [2023-12-27 12:59:13,494][20381] Updated weights for policy 1, policy_version 1058110 (0.0008) [2023-12-27 12:59:13,547][20381] Updated weights for policy 1, policy_version 1058120 (0.0005) [2023-12-27 12:59:13,594][20381] Updated weights for policy 1, policy_version 1058130 (0.0005) [2023-12-27 12:59:14,063][20347] Updated weights for policy 0, policy_version 1059017 (0.0007) [2023-12-27 12:59:14,110][20347] Updated weights for policy 0, policy_version 1059027 (0.0007) [2023-12-27 12:59:14,163][20347] Updated weights for policy 0, policy_version 1059037 (0.0008) [2023-12-27 12:59:14,173][20291] KL-divergence is very high: 245.4929 [2023-12-27 12:59:14,213][20347] Updated weights for policy 0, policy_version 1059047 (0.0007) [2023-12-27 12:59:14,213][20291] KL-divergence is very high: 143.2326 [2023-12-27 12:59:14,229][20381] Updated weights for policy 1, policy_version 1058140 (0.0007) [2023-12-27 12:59:14,255][20292] KL-divergence is very high: 104.2272 [2023-12-27 12:59:14,287][20381] Updated weights for policy 1, policy_version 1058150 (0.0010) [2023-12-27 12:59:14,348][20381] Updated weights for policy 1, policy_version 1058160 (0.0011) [2023-12-27 12:59:14,921][20347] Updated weights for policy 0, policy_version 1059057 (0.0007) [2023-12-27 12:59:14,980][20381] Updated weights for policy 1, policy_version 1058170 (0.0009) [2023-12-27 12:59:14,983][20347] Updated weights for policy 0, policy_version 1059067 (0.0007) [2023-12-27 12:59:15,040][20347] Updated weights for policy 0, policy_version 1059077 (0.0006) [2023-12-27 12:59:15,049][20381] Updated weights for policy 1, policy_version 1058180 (0.0009) [2023-12-27 12:59:15,112][20381] Updated weights for policy 1, policy_version 1058190 (0.0008) [2023-12-27 12:59:15,175][20381] Updated weights for policy 1, policy_version 1058200 (0.0008) [2023-12-27 12:59:15,745][20347] Updated weights for policy 0, policy_version 1059087 (0.0007) [2023-12-27 12:59:15,803][20347] Updated weights for policy 0, policy_version 1059097 (0.0006) [2023-12-27 12:59:15,812][20381] Updated weights for policy 1, policy_version 1058210 (0.0010) [2023-12-27 12:59:15,863][20347] Updated weights for policy 0, policy_version 1059107 (0.0008) [2023-12-27 12:59:15,869][20381] Updated weights for policy 1, policy_version 1058220 (0.0010) [2023-12-27 12:59:15,917][20381] Updated weights for policy 1, policy_version 1058230 (0.0010) [2023-12-27 12:59:16,552][20347] Updated weights for policy 0, policy_version 1059117 (0.0007) [2023-12-27 12:59:16,601][20381] Updated weights for policy 1, policy_version 1058240 (0.0006) [2023-12-27 12:59:16,612][20347] Updated weights for policy 0, policy_version 1059127 (0.0008) [2023-12-27 12:59:16,656][20381] Updated weights for policy 1, policy_version 1058250 (0.0005) [2023-12-27 12:59:16,663][20347] Updated weights for policy 0, policy_version 1059137 (0.0008) [2023-12-27 12:59:16,707][20381] Updated weights for policy 1, policy_version 1058260 (0.0006) [2023-12-27 12:59:17,237][20381] Updated weights for policy 1, policy_version 1058270 (0.0005) [2023-12-27 12:59:17,288][20381] Updated weights for policy 1, policy_version 1058280 (0.0005) [2023-12-27 12:59:17,341][20381] Updated weights for policy 1, policy_version 1058290 (0.0010) [2023-12-27 12:59:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 542130176. Throughput: 0: 10119.6, 1: 10228.7. Samples: 542147808. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:59:17,363][19170] Avg episode reward: [(0, '951.390'), (1, '765.420')] [2023-12-27 12:59:17,381][20347] Updated weights for policy 0, policy_version 1059147 (0.0006) [2023-12-27 12:59:17,464][20347] Updated weights for policy 0, policy_version 1059157 (0.0007) [2023-12-27 12:59:17,525][20347] Updated weights for policy 0, policy_version 1059167 (0.0007) [2023-12-27 12:59:17,992][20381] Updated weights for policy 1, policy_version 1058300 (0.0011) [2023-12-27 12:59:18,043][20381] Updated weights for policy 1, policy_version 1058310 (0.0006) [2023-12-27 12:59:18,100][20381] Updated weights for policy 1, policy_version 1058320 (0.0006) [2023-12-27 12:59:18,218][20347] Updated weights for policy 0, policy_version 1059177 (0.0007) [2023-12-27 12:59:18,278][20347] Updated weights for policy 0, policy_version 1059187 (0.0008) [2023-12-27 12:59:18,347][20347] Updated weights for policy 0, policy_version 1059197 (0.0007) [2023-12-27 12:59:18,406][20347] Updated weights for policy 0, policy_version 1059207 (0.0006) [2023-12-27 12:59:18,740][20381] Updated weights for policy 1, policy_version 1058330 (0.0006) [2023-12-27 12:59:18,806][20381] Updated weights for policy 1, policy_version 1058340 (0.0006) [2023-12-27 12:59:18,866][20381] Updated weights for policy 1, policy_version 1058350 (0.0008) [2023-12-27 12:59:18,923][20381] Updated weights for policy 1, policy_version 1058360 (0.0008) [2023-12-27 12:59:19,130][20347] Updated weights for policy 0, policy_version 1059217 (0.0007) [2023-12-27 12:59:19,181][20347] Updated weights for policy 0, policy_version 1059227 (0.0008) [2023-12-27 12:59:19,235][20347] Updated weights for policy 0, policy_version 1059237 (0.0008) [2023-12-27 12:59:19,548][20381] Updated weights for policy 1, policy_version 1058370 (0.0006) [2023-12-27 12:59:19,608][20381] Updated weights for policy 1, policy_version 1058380 (0.0006) [2023-12-27 12:59:19,664][20381] Updated weights for policy 1, policy_version 1058390 (0.0006) [2023-12-27 12:59:19,961][20347] Updated weights for policy 0, policy_version 1059247 (0.0009) [2023-12-27 12:59:20,032][20347] Updated weights for policy 0, policy_version 1059257 (0.0010) [2023-12-27 12:59:20,103][20347] Updated weights for policy 0, policy_version 1059267 (0.0011) [2023-12-27 12:59:20,347][20381] Updated weights for policy 1, policy_version 1058400 (0.0007) [2023-12-27 12:59:20,407][20381] Updated weights for policy 1, policy_version 1058410 (0.0007) [2023-12-27 12:59:20,464][20381] Updated weights for policy 1, policy_version 1058420 (0.0008) [2023-12-27 12:59:20,766][20347] Updated weights for policy 0, policy_version 1059277 (0.0010) [2023-12-27 12:59:20,829][20347] Updated weights for policy 0, policy_version 1059287 (0.0010) [2023-12-27 12:59:20,892][20347] Updated weights for policy 0, policy_version 1059297 (0.0011) [2023-12-27 12:59:21,164][20381] Updated weights for policy 1, policy_version 1058430 (0.0008) [2023-12-27 12:59:21,224][20381] Updated weights for policy 1, policy_version 1058440 (0.0007) [2023-12-27 12:59:21,291][20381] Updated weights for policy 1, policy_version 1058450 (0.0007) [2023-12-27 12:59:21,635][20347] Updated weights for policy 0, policy_version 1059307 (0.0010) [2023-12-27 12:59:21,704][20347] Updated weights for policy 0, policy_version 1059317 (0.0007) [2023-12-27 12:59:21,771][20347] Updated weights for policy 0, policy_version 1059327 (0.0008) [2023-12-27 12:59:22,020][20381] Updated weights for policy 1, policy_version 1058460 (0.0009) [2023-12-27 12:59:22,069][20381] Updated weights for policy 1, policy_version 1058470 (0.0010) [2023-12-27 12:59:22,119][20381] Updated weights for policy 1, policy_version 1058480 (0.0011) [2023-12-27 12:59:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 542236672. Throughput: 0: 10054.2, 1: 10238.9. Samples: 542208312. Policy #0 lag: (min: 31.0, avg: 32.1, max: 58.0) [2023-12-27 12:59:22,363][19170] Avg episode reward: [(0, '912.430'), (1, '726.080')] [2023-12-27 12:59:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001059336_271237120.pth... [2023-12-27 12:59:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001058488_270999552.pth... [2023-12-27 12:59:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001057272_270688256.pth [2023-12-27 12:59:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001058152_270934016.pth [2023-12-27 12:59:22,516][20347] Updated weights for policy 0, policy_version 1059337 (0.0008) [2023-12-27 12:59:22,573][20347] Updated weights for policy 0, policy_version 1059347 (0.0008) [2023-12-27 12:59:22,620][20347] Updated weights for policy 0, policy_version 1059357 (0.0008) [2023-12-27 12:59:22,674][20347] Updated weights for policy 0, policy_version 1059367 (0.0007) [2023-12-27 12:59:22,817][20381] Updated weights for policy 1, policy_version 1058490 (0.0009) [2023-12-27 12:59:22,883][20381] Updated weights for policy 1, policy_version 1058500 (0.0006) [2023-12-27 12:59:22,946][20381] Updated weights for policy 1, policy_version 1058510 (0.0008) [2023-12-27 12:59:23,006][20381] Updated weights for policy 1, policy_version 1058520 (0.0011) [2023-12-27 12:59:23,314][20347] Updated weights for policy 0, policy_version 1059377 (0.0007) [2023-12-27 12:59:23,363][20347] Updated weights for policy 0, policy_version 1059387 (0.0005) [2023-12-27 12:59:23,407][20347] Updated weights for policy 0, policy_version 1059397 (0.0005) [2023-12-27 12:59:23,675][20381] Updated weights for policy 1, policy_version 1058530 (0.0011) [2023-12-27 12:59:23,736][20381] Updated weights for policy 1, policy_version 1058540 (0.0011) [2023-12-27 12:59:23,799][20381] Updated weights for policy 1, policy_version 1058550 (0.0011) [2023-12-27 12:59:24,001][20347] Updated weights for policy 0, policy_version 1059407 (0.0005) [2023-12-27 12:59:24,047][20347] Updated weights for policy 0, policy_version 1059417 (0.0005) [2023-12-27 12:59:24,096][20347] Updated weights for policy 0, policy_version 1059427 (0.0006) [2023-12-27 12:59:24,374][20381] Updated weights for policy 1, policy_version 1058560 (0.0011) [2023-12-27 12:59:24,432][20381] Updated weights for policy 1, policy_version 1058570 (0.0011) [2023-12-27 12:59:24,477][20381] Updated weights for policy 1, policy_version 1058580 (0.0010) [2023-12-27 12:59:24,794][20347] Updated weights for policy 0, policy_version 1059437 (0.0007) [2023-12-27 12:59:24,841][20347] Updated weights for policy 0, policy_version 1059447 (0.0007) [2023-12-27 12:59:24,889][20347] Updated weights for policy 0, policy_version 1059457 (0.0009) [2023-12-27 12:59:25,081][20381] Updated weights for policy 1, policy_version 1058590 (0.0008) [2023-12-27 12:59:25,130][20381] Updated weights for policy 1, policy_version 1058600 (0.0010) [2023-12-27 12:59:25,175][20381] Updated weights for policy 1, policy_version 1058610 (0.0010) [2023-12-27 12:59:25,564][20347] Updated weights for policy 0, policy_version 1059467 (0.0007) [2023-12-27 12:59:25,617][20347] Updated weights for policy 0, policy_version 1059477 (0.0006) [2023-12-27 12:59:25,679][20347] Updated weights for policy 0, policy_version 1059487 (0.0005) [2023-12-27 12:59:25,877][20381] Updated weights for policy 1, policy_version 1058620 (0.0011) [2023-12-27 12:59:25,928][20381] Updated weights for policy 1, policy_version 1058630 (0.0010) [2023-12-27 12:59:25,976][20381] Updated weights for policy 1, policy_version 1058640 (0.0010) [2023-12-27 12:59:26,239][20347] Updated weights for policy 0, policy_version 1059497 (0.0006) [2023-12-27 12:59:26,298][20347] Updated weights for policy 0, policy_version 1059507 (0.0011) [2023-12-27 12:59:26,350][20347] Updated weights for policy 0, policy_version 1059517 (0.0010) [2023-12-27 12:59:26,398][20347] Updated weights for policy 0, policy_version 1059527 (0.0010) [2023-12-27 12:59:26,678][20381] Updated weights for policy 1, policy_version 1058650 (0.0010) [2023-12-27 12:59:26,723][20381] Updated weights for policy 1, policy_version 1058660 (0.0010) [2023-12-27 12:59:26,782][20381] Updated weights for policy 1, policy_version 1058670 (0.0011) [2023-12-27 12:59:26,840][20381] Updated weights for policy 1, policy_version 1058680 (0.0010) [2023-12-27 12:59:26,979][20347] Updated weights for policy 0, policy_version 1059537 (0.0007) [2023-12-27 12:59:27,028][20347] Updated weights for policy 0, policy_version 1059547 (0.0005) [2023-12-27 12:59:27,076][20347] Updated weights for policy 0, policy_version 1059557 (0.0005) [2023-12-27 12:59:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 542343168. Throughput: 0: 10123.3, 1: 10243.6. Samples: 542331592. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:27,363][19170] Avg episode reward: [(0, '912.380'), (1, '745.840')] [2023-12-27 12:59:27,511][20381] Updated weights for policy 1, policy_version 1058690 (0.0010) [2023-12-27 12:59:27,556][20381] Updated weights for policy 1, policy_version 1058700 (0.0010) [2023-12-27 12:59:27,603][20381] Updated weights for policy 1, policy_version 1058710 (0.0010) [2023-12-27 12:59:27,699][20347] Updated weights for policy 0, policy_version 1059567 (0.0005) [2023-12-27 12:59:27,752][20347] Updated weights for policy 0, policy_version 1059577 (0.0007) [2023-12-27 12:59:27,797][20347] Updated weights for policy 0, policy_version 1059587 (0.0008) [2023-12-27 12:59:28,298][20381] Updated weights for policy 1, policy_version 1058720 (0.0010) [2023-12-27 12:59:28,356][20381] Updated weights for policy 1, policy_version 1058730 (0.0010) [2023-12-27 12:59:28,412][20381] Updated weights for policy 1, policy_version 1058740 (0.0010) [2023-12-27 12:59:28,435][20347] Updated weights for policy 0, policy_version 1059598 (0.0008) [2023-12-27 12:59:28,486][20347] Updated weights for policy 0, policy_version 1059608 (0.0007) [2023-12-27 12:59:28,536][20347] Updated weights for policy 0, policy_version 1059618 (0.0007) [2023-12-27 12:59:29,141][20381] Updated weights for policy 1, policy_version 1058750 (0.0010) [2023-12-27 12:59:29,201][20381] Updated weights for policy 1, policy_version 1058760 (0.0010) [2023-12-27 12:59:29,257][20347] Updated weights for policy 0, policy_version 1059628 (0.0007) [2023-12-27 12:59:29,262][20381] Updated weights for policy 1, policy_version 1058770 (0.0009) [2023-12-27 12:59:29,318][20347] Updated weights for policy 0, policy_version 1059638 (0.0008) [2023-12-27 12:59:29,383][20347] Updated weights for policy 0, policy_version 1059648 (0.0008) [2023-12-27 12:59:29,897][20381] Updated weights for policy 1, policy_version 1058780 (0.0008) [2023-12-27 12:59:29,951][20381] Updated weights for policy 1, policy_version 1058790 (0.0007) [2023-12-27 12:59:30,008][20381] Updated weights for policy 1, policy_version 1058800 (0.0008) [2023-12-27 12:59:30,056][20347] Updated weights for policy 0, policy_version 1059658 (0.0007) [2023-12-27 12:59:30,108][20347] Updated weights for policy 0, policy_version 1059668 (0.0007) [2023-12-27 12:59:30,159][20347] Updated weights for policy 0, policy_version 1059678 (0.0008) [2023-12-27 12:59:30,219][20347] Updated weights for policy 0, policy_version 1059688 (0.0005) [2023-12-27 12:59:30,603][20381] Updated weights for policy 1, policy_version 1058810 (0.0007) [2023-12-27 12:59:30,657][20381] Updated weights for policy 1, policy_version 1058820 (0.0005) [2023-12-27 12:59:30,703][20381] Updated weights for policy 1, policy_version 1058830 (0.0005) [2023-12-27 12:59:30,749][20381] Updated weights for policy 1, policy_version 1058840 (0.0005) [2023-12-27 12:59:30,869][20347] Updated weights for policy 0, policy_version 1059698 (0.0005) [2023-12-27 12:59:30,914][20347] Updated weights for policy 0, policy_version 1059708 (0.0005) [2023-12-27 12:59:30,960][20347] Updated weights for policy 0, policy_version 1059718 (0.0005) [2023-12-27 12:59:31,335][20381] Updated weights for policy 1, policy_version 1058850 (0.0007) [2023-12-27 12:59:31,401][20381] Updated weights for policy 1, policy_version 1058860 (0.0011) [2023-12-27 12:59:31,459][20381] Updated weights for policy 1, policy_version 1058870 (0.0010) [2023-12-27 12:59:31,664][20347] Updated weights for policy 0, policy_version 1059728 (0.0007) [2023-12-27 12:59:31,726][20347] Updated weights for policy 0, policy_version 1059738 (0.0008) [2023-12-27 12:59:31,792][20347] Updated weights for policy 0, policy_version 1059748 (0.0008) [2023-12-27 12:59:32,162][20381] Updated weights for policy 1, policy_version 1058880 (0.0007) [2023-12-27 12:59:32,201][20292] KL-divergence is very high: 127.2591 [2023-12-27 12:59:32,224][20381] Updated weights for policy 1, policy_version 1058890 (0.0007) [2023-12-27 12:59:32,283][20381] Updated weights for policy 1, policy_version 1058900 (0.0008) [2023-12-27 12:59:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 542449664. Throughput: 0: 10241.7, 1: 10281.6. Samples: 542458404. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:32,363][19170] Avg episode reward: [(0, '825.660'), (1, '745.750')] [2023-12-27 12:59:32,472][20347] Updated weights for policy 0, policy_version 1059758 (0.0007) [2023-12-27 12:59:32,531][20347] Updated weights for policy 0, policy_version 1059768 (0.0008) [2023-12-27 12:59:32,579][20347] Updated weights for policy 0, policy_version 1059778 (0.0006) [2023-12-27 12:59:32,929][20381] Updated weights for policy 1, policy_version 1058910 (0.0008) [2023-12-27 12:59:32,988][20381] Updated weights for policy 1, policy_version 1058920 (0.0007) [2023-12-27 12:59:33,043][20381] Updated weights for policy 1, policy_version 1058930 (0.0005) [2023-12-27 12:59:33,231][20291] KL-divergence is very high: 103.3958 [2023-12-27 12:59:33,237][20347] Updated weights for policy 0, policy_version 1059788 (0.0008) [2023-12-27 12:59:33,274][20291] KL-divergence is very high: 377.3247 [2023-12-27 12:59:33,280][20291] KL-divergence is very high: 105.9748 [2023-12-27 12:59:33,291][20347] Updated weights for policy 0, policy_version 1059798 (0.0010) [2023-12-27 12:59:33,322][20291] KL-divergence is very high: 100.3842 [2023-12-27 12:59:33,333][20291] KL-divergence is very high: 104.6183 [2023-12-27 12:59:33,343][20347] Updated weights for policy 0, policy_version 1059808 (0.0008) [2023-12-27 12:59:33,667][20381] Updated weights for policy 1, policy_version 1058940 (0.0006) [2023-12-27 12:59:33,727][20381] Updated weights for policy 1, policy_version 1058950 (0.0008) [2023-12-27 12:59:33,787][20381] Updated weights for policy 1, policy_version 1058960 (0.0008) [2023-12-27 12:59:34,003][20347] Updated weights for policy 0, policy_version 1059818 (0.0007) [2023-12-27 12:59:34,018][20291] KL-divergence is very high: 149.2788 [2023-12-27 12:59:34,036][20291] KL-divergence is very high: 201.6040 [2023-12-27 12:59:34,057][20347] Updated weights for policy 0, policy_version 1059828 (0.0006) [2023-12-27 12:59:34,062][20291] KL-divergence is very high: 224.8416 [2023-12-27 12:59:34,120][20347] Updated weights for policy 0, policy_version 1059838 (0.0005) [2023-12-27 12:59:34,185][20347] Updated weights for policy 0, policy_version 1059848 (0.0008) [2023-12-27 12:59:34,508][20381] Updated weights for policy 1, policy_version 1058970 (0.0008) [2023-12-27 12:59:34,565][20381] Updated weights for policy 1, policy_version 1058980 (0.0005) [2023-12-27 12:59:34,630][20381] Updated weights for policy 1, policy_version 1058990 (0.0005) [2023-12-27 12:59:34,689][20381] Updated weights for policy 1, policy_version 1059000 (0.0005) [2023-12-27 12:59:34,804][20291] KL-divergence is very high: 103.1725 [2023-12-27 12:59:34,808][20347] Updated weights for policy 0, policy_version 1059858 (0.0008) [2023-12-27 12:59:34,867][20347] Updated weights for policy 0, policy_version 1059868 (0.0008) [2023-12-27 12:59:34,930][20347] Updated weights for policy 0, policy_version 1059878 (0.0008) [2023-12-27 12:59:35,240][20381] Updated weights for policy 1, policy_version 1059010 (0.0007) [2023-12-27 12:59:35,296][20381] Updated weights for policy 1, policy_version 1059020 (0.0005) [2023-12-27 12:59:35,352][20381] Updated weights for policy 1, policy_version 1059030 (0.0005) [2023-12-27 12:59:35,680][20347] Updated weights for policy 0, policy_version 1059888 (0.0008) [2023-12-27 12:59:35,733][20347] Updated weights for policy 0, policy_version 1059898 (0.0009) [2023-12-27 12:59:35,787][20347] Updated weights for policy 0, policy_version 1059908 (0.0007) [2023-12-27 12:59:35,869][20381] Updated weights for policy 1, policy_version 1059040 (0.0005) [2023-12-27 12:59:35,920][20381] Updated weights for policy 1, policy_version 1059050 (0.0005) [2023-12-27 12:59:35,977][20381] Updated weights for policy 1, policy_version 1059060 (0.0010) [2023-12-27 12:59:36,483][20347] Updated weights for policy 0, policy_version 1059918 (0.0009) [2023-12-27 12:59:36,545][20347] Updated weights for policy 0, policy_version 1059928 (0.0010) [2023-12-27 12:59:36,606][20347] Updated weights for policy 0, policy_version 1059938 (0.0010) [2023-12-27 12:59:36,660][20381] Updated weights for policy 1, policy_version 1059070 (0.0011) [2023-12-27 12:59:36,719][20381] Updated weights for policy 1, policy_version 1059080 (0.0011) [2023-12-27 12:59:36,774][20381] Updated weights for policy 1, policy_version 1059090 (0.0010) [2023-12-27 12:59:37,231][20347] Updated weights for policy 0, policy_version 1059948 (0.0008) [2023-12-27 12:59:37,287][20347] Updated weights for policy 0, policy_version 1059958 (0.0009) [2023-12-27 12:59:37,355][20347] Updated weights for policy 0, policy_version 1059968 (0.0010) [2023-12-27 12:59:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 542547968. Throughput: 0: 10276.3, 1: 10334.8. Samples: 542521592. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:37,363][19170] Avg episode reward: [(0, '796.050'), (1, '784.850')] [2023-12-27 12:59:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001059096_271155200.pth... [2023-12-27 12:59:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001057880_270843904.pth [2023-12-27 12:59:37,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001059976_271400960.pth... [2023-12-27 12:59:37,405][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001058760_271089664.pth [2023-12-27 12:59:37,406][20381] Updated weights for policy 1, policy_version 1059100 (0.0009) [2023-12-27 12:59:37,468][20381] Updated weights for policy 1, policy_version 1059110 (0.0008) [2023-12-27 12:59:37,521][20381] Updated weights for policy 1, policy_version 1059120 (0.0008) [2023-12-27 12:59:38,030][20347] Updated weights for policy 0, policy_version 1059978 (0.0010) [2023-12-27 12:59:38,087][20347] Updated weights for policy 0, policy_version 1059988 (0.0008) [2023-12-27 12:59:38,149][20347] Updated weights for policy 0, policy_version 1059998 (0.0010) [2023-12-27 12:59:38,150][20381] Updated weights for policy 1, policy_version 1059130 (0.0006) [2023-12-27 12:59:38,198][20381] Updated weights for policy 1, policy_version 1059140 (0.0005) [2023-12-27 12:59:38,207][20347] Updated weights for policy 0, policy_version 1060008 (0.0010) [2023-12-27 12:59:38,256][20381] Updated weights for policy 1, policy_version 1059150 (0.0007) [2023-12-27 12:59:38,324][20381] Updated weights for policy 1, policy_version 1059160 (0.0006) [2023-12-27 12:59:38,917][20347] Updated weights for policy 0, policy_version 1060018 (0.0007) [2023-12-27 12:59:38,940][20381] Updated weights for policy 1, policy_version 1059170 (0.0008) [2023-12-27 12:59:38,975][20347] Updated weights for policy 0, policy_version 1060028 (0.0007) [2023-12-27 12:59:38,990][20381] Updated weights for policy 1, policy_version 1059180 (0.0009) [2023-12-27 12:59:39,034][20347] Updated weights for policy 0, policy_version 1060038 (0.0007) [2023-12-27 12:59:39,046][20381] Updated weights for policy 1, policy_version 1059190 (0.0011) [2023-12-27 12:59:39,727][20347] Updated weights for policy 0, policy_version 1060048 (0.0007) [2023-12-27 12:59:39,779][20347] Updated weights for policy 0, policy_version 1060058 (0.0007) [2023-12-27 12:59:39,791][20381] Updated weights for policy 1, policy_version 1059200 (0.0007) [2023-12-27 12:59:39,843][20347] Updated weights for policy 0, policy_version 1060068 (0.0007) [2023-12-27 12:59:39,855][20381] Updated weights for policy 1, policy_version 1059210 (0.0006) [2023-12-27 12:59:39,915][20381] Updated weights for policy 1, policy_version 1059220 (0.0008) [2023-12-27 12:59:40,533][20347] Updated weights for policy 0, policy_version 1060078 (0.0008) [2023-12-27 12:59:40,589][20347] Updated weights for policy 0, policy_version 1060088 (0.0008) [2023-12-27 12:59:40,635][20381] Updated weights for policy 1, policy_version 1059230 (0.0008) [2023-12-27 12:59:40,637][20347] Updated weights for policy 0, policy_version 1060098 (0.0006) [2023-12-27 12:59:40,682][20381] Updated weights for policy 1, policy_version 1059240 (0.0006) [2023-12-27 12:59:40,729][20381] Updated weights for policy 1, policy_version 1059250 (0.0007) [2023-12-27 12:59:41,411][20381] Updated weights for policy 1, policy_version 1059260 (0.0007) [2023-12-27 12:59:41,430][20347] Updated weights for policy 0, policy_version 1060108 (0.0007) [2023-12-27 12:59:41,475][20381] Updated weights for policy 1, policy_version 1059270 (0.0007) [2023-12-27 12:59:41,489][20347] Updated weights for policy 0, policy_version 1060118 (0.0007) [2023-12-27 12:59:41,537][20381] Updated weights for policy 1, policy_version 1059280 (0.0008) [2023-12-27 12:59:41,555][20347] Updated weights for policy 0, policy_version 1060128 (0.0006) [2023-12-27 12:59:42,248][20381] Updated weights for policy 1, policy_version 1059290 (0.0007) [2023-12-27 12:59:42,260][20347] Updated weights for policy 0, policy_version 1060138 (0.0007) [2023-12-27 12:59:42,311][20381] Updated weights for policy 1, policy_version 1059300 (0.0008) [2023-12-27 12:59:42,317][20347] Updated weights for policy 0, policy_version 1060148 (0.0006) [2023-12-27 12:59:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 542646272. Throughput: 0: 10214.1, 1: 10368.1. Samples: 542644948. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:42,363][19170] Avg episode reward: [(0, '834.510'), (1, '784.780')] [2023-12-27 12:59:42,379][20381] Updated weights for policy 1, policy_version 1059310 (0.0008) [2023-12-27 12:59:42,385][20347] Updated weights for policy 0, policy_version 1060158 (0.0007) [2023-12-27 12:59:42,439][20381] Updated weights for policy 1, policy_version 1059320 (0.0008) [2023-12-27 12:59:42,444][20347] Updated weights for policy 0, policy_version 1060168 (0.0007) [2023-12-27 12:59:43,137][20347] Updated weights for policy 0, policy_version 1060178 (0.0007) [2023-12-27 12:59:43,159][20381] Updated weights for policy 1, policy_version 1059330 (0.0005) [2023-12-27 12:59:43,194][20347] Updated weights for policy 0, policy_version 1060188 (0.0006) [2023-12-27 12:59:43,211][20381] Updated weights for policy 1, policy_version 1059340 (0.0006) [2023-12-27 12:59:43,246][20347] Updated weights for policy 0, policy_version 1060198 (0.0005) [2023-12-27 12:59:43,267][20381] Updated weights for policy 1, policy_version 1059350 (0.0006) [2023-12-27 12:59:43,893][20347] Updated weights for policy 0, policy_version 1060208 (0.0007) [2023-12-27 12:59:43,941][20347] Updated weights for policy 0, policy_version 1060218 (0.0006) [2023-12-27 12:59:43,954][20381] Updated weights for policy 1, policy_version 1059360 (0.0006) [2023-12-27 12:59:43,986][20347] Updated weights for policy 0, policy_version 1060228 (0.0005) [2023-12-27 12:59:43,999][20381] Updated weights for policy 1, policy_version 1059370 (0.0005) [2023-12-27 12:59:44,049][20381] Updated weights for policy 1, policy_version 1059380 (0.0007) [2023-12-27 12:59:44,673][20347] Updated weights for policy 0, policy_version 1060238 (0.0007) [2023-12-27 12:59:44,718][20347] Updated weights for policy 0, policy_version 1060248 (0.0007) [2023-12-27 12:59:44,752][20381] Updated weights for policy 1, policy_version 1059390 (0.0007) [2023-12-27 12:59:44,777][20347] Updated weights for policy 0, policy_version 1060258 (0.0007) [2023-12-27 12:59:44,812][20381] Updated weights for policy 1, policy_version 1059400 (0.0009) [2023-12-27 12:59:44,871][20381] Updated weights for policy 1, policy_version 1059410 (0.0009) [2023-12-27 12:59:45,454][20347] Updated weights for policy 0, policy_version 1060268 (0.0006) [2023-12-27 12:59:45,506][20347] Updated weights for policy 0, policy_version 1060278 (0.0006) [2023-12-27 12:59:45,560][20347] Updated weights for policy 0, policy_version 1060288 (0.0005) [2023-12-27 12:59:45,635][20381] Updated weights for policy 1, policy_version 1059420 (0.0007) [2023-12-27 12:59:45,687][20381] Updated weights for policy 1, policy_version 1059430 (0.0008) [2023-12-27 12:59:45,743][20381] Updated weights for policy 1, policy_version 1059440 (0.0007) [2023-12-27 12:59:46,156][20347] Updated weights for policy 0, policy_version 1060298 (0.0007) [2023-12-27 12:59:46,210][20347] Updated weights for policy 0, policy_version 1060308 (0.0007) [2023-12-27 12:59:46,237][20291] KL-divergence is very high: 108.0074 [2023-12-27 12:59:46,259][20291] KL-divergence is very high: 108.2535 [2023-12-27 12:59:46,270][20347] Updated weights for policy 0, policy_version 1060318 (0.0007) [2023-12-27 12:59:46,329][20347] Updated weights for policy 0, policy_version 1060328 (0.0005) [2023-12-27 12:59:46,481][20381] Updated weights for policy 1, policy_version 1059450 (0.0008) [2023-12-27 12:59:46,533][20381] Updated weights for policy 1, policy_version 1059460 (0.0007) [2023-12-27 12:59:46,592][20381] Updated weights for policy 1, policy_version 1059470 (0.0007) [2023-12-27 12:59:46,644][20381] Updated weights for policy 1, policy_version 1059480 (0.0010) [2023-12-27 12:59:46,888][20347] Updated weights for policy 0, policy_version 1060338 (0.0008) [2023-12-27 12:59:46,933][20347] Updated weights for policy 0, policy_version 1060348 (0.0008) [2023-12-27 12:59:46,981][20347] Updated weights for policy 0, policy_version 1060358 (0.0009) [2023-12-27 12:59:47,299][20381] Updated weights for policy 1, policy_version 1059490 (0.0005) [2023-12-27 12:59:47,360][20381] Updated weights for policy 1, policy_version 1059500 (0.0005) [2023-12-27 12:59:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 542752768. Throughput: 0: 10263.6, 1: 10326.1. Samples: 542765716. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:47,363][19170] Avg episode reward: [(0, '843.860'), (1, '755.450')] [2023-12-27 12:59:47,425][20381] Updated weights for policy 1, policy_version 1059510 (0.0005) [2023-12-27 12:59:47,756][20347] Updated weights for policy 0, policy_version 1060368 (0.0008) [2023-12-27 12:59:47,815][20347] Updated weights for policy 0, policy_version 1060378 (0.0007) [2023-12-27 12:59:47,871][20347] Updated weights for policy 0, policy_version 1060388 (0.0007) [2023-12-27 12:59:47,968][20381] Updated weights for policy 1, policy_version 1059520 (0.0007) [2023-12-27 12:59:48,024][20381] Updated weights for policy 1, policy_version 1059530 (0.0007) [2023-12-27 12:59:48,077][20381] Updated weights for policy 1, policy_version 1059540 (0.0007) [2023-12-27 12:59:48,584][20347] Updated weights for policy 0, policy_version 1060398 (0.0007) [2023-12-27 12:59:48,641][20347] Updated weights for policy 0, policy_version 1060408 (0.0007) [2023-12-27 12:59:48,706][20347] Updated weights for policy 0, policy_version 1060418 (0.0006) [2023-12-27 12:59:48,778][20381] Updated weights for policy 1, policy_version 1059550 (0.0010) [2023-12-27 12:59:48,857][20381] Updated weights for policy 1, policy_version 1059560 (0.0011) [2023-12-27 12:59:48,918][20381] Updated weights for policy 1, policy_version 1059570 (0.0008) [2023-12-27 12:59:49,447][20347] Updated weights for policy 0, policy_version 1060428 (0.0006) [2023-12-27 12:59:49,511][20347] Updated weights for policy 0, policy_version 1060438 (0.0007) [2023-12-27 12:59:49,569][20347] Updated weights for policy 0, policy_version 1060448 (0.0007) [2023-12-27 12:59:49,635][20381] Updated weights for policy 1, policy_version 1059580 (0.0007) [2023-12-27 12:59:49,698][20381] Updated weights for policy 1, policy_version 1059590 (0.0009) [2023-12-27 12:59:49,755][20381] Updated weights for policy 1, policy_version 1059600 (0.0007) [2023-12-27 12:59:50,269][20347] Updated weights for policy 0, policy_version 1060458 (0.0008) [2023-12-27 12:59:50,322][20347] Updated weights for policy 0, policy_version 1060468 (0.0009) [2023-12-27 12:59:50,378][20347] Updated weights for policy 0, policy_version 1060478 (0.0008) [2023-12-27 12:59:50,425][20347] Updated weights for policy 0, policy_version 1060488 (0.0008) [2023-12-27 12:59:50,441][20381] Updated weights for policy 1, policy_version 1059610 (0.0007) [2023-12-27 12:59:50,495][20381] Updated weights for policy 1, policy_version 1059620 (0.0008) [2023-12-27 12:59:50,553][20381] Updated weights for policy 1, policy_version 1059630 (0.0008) [2023-12-27 12:59:50,618][20381] Updated weights for policy 1, policy_version 1059640 (0.0008) [2023-12-27 12:59:51,173][20347] Updated weights for policy 0, policy_version 1060498 (0.0007) [2023-12-27 12:59:51,231][20347] Updated weights for policy 0, policy_version 1060508 (0.0007) [2023-12-27 12:59:51,292][20347] Updated weights for policy 0, policy_version 1060518 (0.0007) [2023-12-27 12:59:51,317][20381] Updated weights for policy 1, policy_version 1059650 (0.0009) [2023-12-27 12:59:51,385][20381] Updated weights for policy 1, policy_version 1059660 (0.0008) [2023-12-27 12:59:51,450][20381] Updated weights for policy 1, policy_version 1059670 (0.0005) [2023-12-27 12:59:52,050][20381] Updated weights for policy 1, policy_version 1059680 (0.0006) [2023-12-27 12:59:52,062][20347] Updated weights for policy 0, policy_version 1060528 (0.0009) [2023-12-27 12:59:52,112][20347] Updated weights for policy 0, policy_version 1060538 (0.0010) [2023-12-27 12:59:52,113][20381] Updated weights for policy 1, policy_version 1059690 (0.0006) [2023-12-27 12:59:52,160][20347] Updated weights for policy 0, policy_version 1060548 (0.0010) [2023-12-27 12:59:52,179][20381] Updated weights for policy 1, policy_version 1059700 (0.0006) [2023-12-27 12:59:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 542859264. Throughput: 0: 10200.9, 1: 10326.5. Samples: 542825240. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:52,363][19170] Avg episode reward: [(0, '912.160'), (1, '716.380')] [2023-12-27 12:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001059704_271310848.pth... [2023-12-27 12:59:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001060552_271548416.pth... [2023-12-27 12:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001058488_270999552.pth [2023-12-27 12:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001059704_271310848.pth [2023-12-27 12:59:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001059336_271237120.pth [2023-12-27 12:59:52,377][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001060552_271548416.pth [2023-12-27 12:59:52,813][20381] Updated weights for policy 1, policy_version 1059710 (0.0009) [2023-12-27 12:59:52,863][20381] Updated weights for policy 1, policy_version 1059720 (0.0010) [2023-12-27 12:59:52,868][20347] Updated weights for policy 0, policy_version 1060558 (0.0010) [2023-12-27 12:59:52,913][20381] Updated weights for policy 1, policy_version 1059730 (0.0010) [2023-12-27 12:59:52,921][20347] Updated weights for policy 0, policy_version 1060568 (0.0011) [2023-12-27 12:59:52,980][20347] Updated weights for policy 0, policy_version 1060578 (0.0010) [2023-12-27 12:59:53,590][20381] Updated weights for policy 1, policy_version 1059740 (0.0008) [2023-12-27 12:59:53,615][20347] Updated weights for policy 0, policy_version 1060588 (0.0011) [2023-12-27 12:59:53,646][20381] Updated weights for policy 1, policy_version 1059750 (0.0011) [2023-12-27 12:59:53,664][20347] Updated weights for policy 0, policy_version 1060598 (0.0010) [2023-12-27 12:59:53,699][20381] Updated weights for policy 1, policy_version 1059760 (0.0010) [2023-12-27 12:59:53,720][20347] Updated weights for policy 0, policy_version 1060608 (0.0010) [2023-12-27 12:59:54,341][20381] Updated weights for policy 1, policy_version 1059770 (0.0010) [2023-12-27 12:59:54,398][20381] Updated weights for policy 1, policy_version 1059780 (0.0008) [2023-12-27 12:59:54,416][20347] Updated weights for policy 0, policy_version 1060618 (0.0009) [2023-12-27 12:59:54,459][20381] Updated weights for policy 1, policy_version 1059790 (0.0007) [2023-12-27 12:59:54,478][20347] Updated weights for policy 0, policy_version 1060628 (0.0007) [2023-12-27 12:59:54,521][20381] Updated weights for policy 1, policy_version 1059800 (0.0009) [2023-12-27 12:59:54,537][20347] Updated weights for policy 0, policy_version 1060638 (0.0006) [2023-12-27 12:59:54,588][20347] Updated weights for policy 0, policy_version 1060648 (0.0007) [2023-12-27 12:59:55,201][20381] Updated weights for policy 1, policy_version 1059810 (0.0008) [2023-12-27 12:59:55,252][20381] Updated weights for policy 1, policy_version 1059820 (0.0007) [2023-12-27 12:59:55,281][20347] Updated weights for policy 0, policy_version 1060658 (0.0006) [2023-12-27 12:59:55,294][20381] Updated weights for policy 1, policy_version 1059830 (0.0008) [2023-12-27 12:59:55,333][20347] Updated weights for policy 0, policy_version 1060668 (0.0007) [2023-12-27 12:59:55,394][20347] Updated weights for policy 0, policy_version 1060678 (0.0009) [2023-12-27 12:59:55,933][20381] Updated weights for policy 1, policy_version 1059840 (0.0009) [2023-12-27 12:59:55,955][20347] Updated weights for policy 0, policy_version 1060688 (0.0006) [2023-12-27 12:59:55,999][20381] Updated weights for policy 1, policy_version 1059850 (0.0011) [2023-12-27 12:59:56,004][20347] Updated weights for policy 0, policy_version 1060698 (0.0005) [2023-12-27 12:59:56,055][20381] Updated weights for policy 1, policy_version 1059860 (0.0011) [2023-12-27 12:59:56,063][20347] Updated weights for policy 0, policy_version 1060708 (0.0010) [2023-12-27 12:59:56,680][20347] Updated weights for policy 0, policy_version 1060718 (0.0009) [2023-12-27 12:59:56,708][20381] Updated weights for policy 1, policy_version 1059870 (0.0011) [2023-12-27 12:59:56,737][20347] Updated weights for policy 0, policy_version 1060728 (0.0007) [2023-12-27 12:59:56,757][20381] Updated weights for policy 1, policy_version 1059880 (0.0010) [2023-12-27 12:59:56,786][20347] Updated weights for policy 0, policy_version 1060738 (0.0005) [2023-12-27 12:59:56,802][20381] Updated weights for policy 1, policy_version 1059890 (0.0010) [2023-12-27 12:59:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 542957568. Throughput: 0: 10130.6, 1: 10426.5. Samples: 542949632. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 12:59:57,363][19170] Avg episode reward: [(0, '931.870'), (1, '804.280')] [2023-12-27 12:59:57,396][20381] Updated weights for policy 1, policy_version 1059900 (0.0010) [2023-12-27 12:59:57,444][20347] Updated weights for policy 0, policy_version 1060748 (0.0006) [2023-12-27 12:59:57,444][20381] Updated weights for policy 1, policy_version 1059910 (0.0010) [2023-12-27 12:59:57,492][20381] Updated weights for policy 1, policy_version 1059920 (0.0010) [2023-12-27 12:59:57,494][20347] Updated weights for policy 0, policy_version 1060758 (0.0005) [2023-12-27 12:59:57,539][20347] Updated weights for policy 0, policy_version 1060768 (0.0005) [2023-12-27 12:59:58,141][20347] Updated weights for policy 0, policy_version 1060778 (0.0006) [2023-12-27 12:59:58,189][20381] Updated weights for policy 1, policy_version 1059930 (0.0010) [2023-12-27 12:59:58,201][20347] Updated weights for policy 0, policy_version 1060788 (0.0010) [2023-12-27 12:59:58,254][20381] Updated weights for policy 1, policy_version 1059940 (0.0008) [2023-12-27 12:59:58,265][20347] Updated weights for policy 0, policy_version 1060798 (0.0009) [2023-12-27 12:59:58,317][20381] Updated weights for policy 1, policy_version 1059950 (0.0011) [2023-12-27 12:59:58,325][20347] Updated weights for policy 0, policy_version 1060808 (0.0010) [2023-12-27 12:59:58,387][20381] Updated weights for policy 1, policy_version 1059960 (0.0008) [2023-12-27 12:59:59,118][20347] Updated weights for policy 0, policy_version 1060818 (0.0007) [2023-12-27 12:59:59,175][20347] Updated weights for policy 0, policy_version 1060828 (0.0006) [2023-12-27 12:59:59,219][20381] Updated weights for policy 1, policy_version 1059970 (0.0008) [2023-12-27 12:59:59,232][20347] Updated weights for policy 0, policy_version 1060838 (0.0007) [2023-12-27 12:59:59,292][20381] Updated weights for policy 1, policy_version 1059980 (0.0008) [2023-12-27 12:59:59,351][20381] Updated weights for policy 1, policy_version 1059990 (0.0008) [2023-12-27 12:59:59,920][20347] Updated weights for policy 0, policy_version 1060848 (0.0008) [2023-12-27 12:59:59,988][20347] Updated weights for policy 0, policy_version 1060858 (0.0009) [2023-12-27 13:00:00,050][20347] Updated weights for policy 0, policy_version 1060868 (0.0008) [2023-12-27 13:00:00,072][20381] Updated weights for policy 1, policy_version 1060000 (0.0010) [2023-12-27 13:00:00,133][20381] Updated weights for policy 1, policy_version 1060010 (0.0008) [2023-12-27 13:00:00,192][20381] Updated weights for policy 1, policy_version 1060020 (0.0008) [2023-12-27 13:00:00,713][20347] Updated weights for policy 0, policy_version 1060878 (0.0008) [2023-12-27 13:00:00,766][20347] Updated weights for policy 0, policy_version 1060888 (0.0007) [2023-12-27 13:00:00,803][20381] Updated weights for policy 1, policy_version 1060030 (0.0009) [2023-12-27 13:00:00,822][20347] Updated weights for policy 0, policy_version 1060898 (0.0006) [2023-12-27 13:00:00,851][20381] Updated weights for policy 1, policy_version 1060040 (0.0010) [2023-12-27 13:00:00,905][20381] Updated weights for policy 1, policy_version 1060050 (0.0007) [2023-12-27 13:00:01,529][20347] Updated weights for policy 0, policy_version 1060908 (0.0007) [2023-12-27 13:00:01,582][20347] Updated weights for policy 0, policy_version 1060918 (0.0006) [2023-12-27 13:00:01,593][20381] Updated weights for policy 1, policy_version 1060060 (0.0007) [2023-12-27 13:00:01,635][20347] Updated weights for policy 0, policy_version 1060928 (0.0006) [2023-12-27 13:00:01,656][20381] Updated weights for policy 1, policy_version 1060070 (0.0008) [2023-12-27 13:00:01,716][20381] Updated weights for policy 1, policy_version 1060080 (0.0006) [2023-12-27 13:00:02,305][20347] Updated weights for policy 0, policy_version 1060938 (0.0007) [2023-12-27 13:00:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 543055872. Throughput: 0: 10205.0, 1: 10330.8. Samples: 543071920. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:02,363][19170] Avg episode reward: [(0, '835.630'), (1, '794.490')] [2023-12-27 13:00:02,363][20347] Updated weights for policy 0, policy_version 1060948 (0.0008) [2023-12-27 13:00:02,414][20347] Updated weights for policy 0, policy_version 1060958 (0.0010) [2023-12-27 13:00:02,430][20381] Updated weights for policy 1, policy_version 1060090 (0.0007) [2023-12-27 13:00:02,468][20347] Updated weights for policy 0, policy_version 1060968 (0.0006) [2023-12-27 13:00:02,481][20381] Updated weights for policy 1, policy_version 1060100 (0.0010) [2023-12-27 13:00:02,540][20381] Updated weights for policy 1, policy_version 1060110 (0.0010) [2023-12-27 13:00:02,608][20381] Updated weights for policy 1, policy_version 1060120 (0.0007) [2023-12-27 13:00:03,169][20347] Updated weights for policy 0, policy_version 1060978 (0.0008) [2023-12-27 13:00:03,224][20347] Updated weights for policy 0, policy_version 1060988 (0.0007) [2023-12-27 13:00:03,228][20381] Updated weights for policy 1, policy_version 1060130 (0.0006) [2023-12-27 13:00:03,272][20347] Updated weights for policy 0, policy_version 1060998 (0.0005) [2023-12-27 13:00:03,277][20381] Updated weights for policy 1, policy_version 1060140 (0.0006) [2023-12-27 13:00:03,332][20381] Updated weights for policy 1, policy_version 1060150 (0.0007) [2023-12-27 13:00:03,904][20347] Updated weights for policy 0, policy_version 1061008 (0.0010) [2023-12-27 13:00:03,953][20381] Updated weights for policy 1, policy_version 1060160 (0.0006) [2023-12-27 13:00:03,969][20347] Updated weights for policy 0, policy_version 1061018 (0.0012) [2023-12-27 13:00:04,018][20381] Updated weights for policy 1, policy_version 1060170 (0.0006) [2023-12-27 13:00:04,033][20347] Updated weights for policy 0, policy_version 1061028 (0.0006) [2023-12-27 13:00:04,080][20381] Updated weights for policy 1, policy_version 1060180 (0.0006) [2023-12-27 13:00:04,674][20381] Updated weights for policy 1, policy_version 1060190 (0.0006) [2023-12-27 13:00:04,692][20347] Updated weights for policy 0, policy_version 1061038 (0.0008) [2023-12-27 13:00:04,734][20381] Updated weights for policy 1, policy_version 1060200 (0.0006) [2023-12-27 13:00:04,752][20292] KL-divergence is very high: 101.8214 [2023-12-27 13:00:04,754][20347] Updated weights for policy 0, policy_version 1061048 (0.0009) [2023-12-27 13:00:04,793][20381] Updated weights for policy 1, policy_version 1060210 (0.0008) [2023-12-27 13:00:04,812][20347] Updated weights for policy 0, policy_version 1061058 (0.0010) [2023-12-27 13:00:05,417][20347] Updated weights for policy 0, policy_version 1061068 (0.0009) [2023-12-27 13:00:05,465][20347] Updated weights for policy 0, policy_version 1061078 (0.0005) [2023-12-27 13:00:05,471][20381] Updated weights for policy 1, policy_version 1060220 (0.0007) [2023-12-27 13:00:05,518][20347] Updated weights for policy 0, policy_version 1061088 (0.0005) [2023-12-27 13:00:05,522][20381] Updated weights for policy 1, policy_version 1060230 (0.0010) [2023-12-27 13:00:05,570][20381] Updated weights for policy 1, policy_version 1060240 (0.0010) [2023-12-27 13:00:06,145][20347] Updated weights for policy 0, policy_version 1061098 (0.0006) [2023-12-27 13:00:06,193][20347] Updated weights for policy 0, policy_version 1061108 (0.0008) [2023-12-27 13:00:06,226][20381] Updated weights for policy 1, policy_version 1060250 (0.0005) [2023-12-27 13:00:06,245][20347] Updated weights for policy 0, policy_version 1061118 (0.0008) [2023-12-27 13:00:06,274][20381] Updated weights for policy 1, policy_version 1060260 (0.0007) [2023-12-27 13:00:06,293][20347] Updated weights for policy 0, policy_version 1061128 (0.0008) [2023-12-27 13:00:06,327][20381] Updated weights for policy 1, policy_version 1060270 (0.0007) [2023-12-27 13:00:06,381][20381] Updated weights for policy 1, policy_version 1060280 (0.0007) [2023-12-27 13:00:06,921][20291] KL-divergence is very high: 116.4927 [2023-12-27 13:00:06,941][20347] Updated weights for policy 0, policy_version 1061138 (0.0008) [2023-12-27 13:00:06,963][20291] KL-divergence is very high: 140.1549 [2023-12-27 13:00:06,996][20347] Updated weights for policy 0, policy_version 1061148 (0.0008) [2023-12-27 13:00:07,048][20347] Updated weights for policy 0, policy_version 1061158 (0.0007) [2023-12-27 13:00:07,123][20381] Updated weights for policy 1, policy_version 1060290 (0.0008) [2023-12-27 13:00:07,185][20381] Updated weights for policy 1, policy_version 1060300 (0.0008) [2023-12-27 13:00:07,238][20381] Updated weights for policy 1, policy_version 1060310 (0.0008) [2023-12-27 13:00:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 543170560. Throughput: 0: 10257.7, 1: 10346.5. Samples: 543135500. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:07,363][19170] Avg episode reward: [(0, '786.020'), (1, '765.260')] [2023-12-27 13:00:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001060312_271466496.pth... [2023-12-27 13:00:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001061160_271704064.pth... [2023-12-27 13:00:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001059096_271155200.pth [2023-12-27 13:00:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001059976_271400960.pth [2023-12-27 13:00:07,645][20347] Updated weights for policy 0, policy_version 1061168 (0.0005) [2023-12-27 13:00:07,703][20347] Updated weights for policy 0, policy_version 1061178 (0.0006) [2023-12-27 13:00:07,751][20347] Updated weights for policy 0, policy_version 1061188 (0.0008) [2023-12-27 13:00:07,924][20381] Updated weights for policy 1, policy_version 1060320 (0.0007) [2023-12-27 13:00:07,979][20381] Updated weights for policy 1, policy_version 1060330 (0.0008) [2023-12-27 13:00:08,037][20381] Updated weights for policy 1, policy_version 1060340 (0.0008) [2023-12-27 13:00:08,342][20347] Updated weights for policy 0, policy_version 1061198 (0.0010) [2023-12-27 13:00:08,410][20347] Updated weights for policy 0, policy_version 1061208 (0.0008) [2023-12-27 13:00:08,470][20347] Updated weights for policy 0, policy_version 1061218 (0.0007) [2023-12-27 13:00:08,763][20381] Updated weights for policy 1, policy_version 1060350 (0.0007) [2023-12-27 13:00:08,823][20381] Updated weights for policy 1, policy_version 1060360 (0.0008) [2023-12-27 13:00:08,881][20381] Updated weights for policy 1, policy_version 1060370 (0.0007) [2023-12-27 13:00:09,149][20347] Updated weights for policy 0, policy_version 1061228 (0.0008) [2023-12-27 13:00:09,206][20347] Updated weights for policy 0, policy_version 1061238 (0.0007) [2023-12-27 13:00:09,272][20347] Updated weights for policy 0, policy_version 1061248 (0.0007) [2023-12-27 13:00:09,645][20381] Updated weights for policy 1, policy_version 1060380 (0.0007) [2023-12-27 13:00:09,708][20381] Updated weights for policy 1, policy_version 1060390 (0.0007) [2023-12-27 13:00:09,764][20381] Updated weights for policy 1, policy_version 1060400 (0.0007) [2023-12-27 13:00:09,940][20347] Updated weights for policy 0, policy_version 1061258 (0.0007) [2023-12-27 13:00:10,002][20347] Updated weights for policy 0, policy_version 1061268 (0.0007) [2023-12-27 13:00:10,014][20291] KL-divergence is very high: 210.8936 [2023-12-27 13:00:10,021][20291] KL-divergence is very high: 146.8639 [2023-12-27 13:00:10,040][20291] KL-divergence is very high: 180.9271 [2023-12-27 13:00:10,063][20347] Updated weights for policy 0, policy_version 1061278 (0.0007) [2023-12-27 13:00:10,063][20291] KL-divergence is very high: 117.8425 [2023-12-27 13:00:10,121][20347] Updated weights for policy 0, policy_version 1061288 (0.0007) [2023-12-27 13:00:10,479][20381] Updated weights for policy 1, policy_version 1060410 (0.0007) [2023-12-27 13:00:10,535][20381] Updated weights for policy 1, policy_version 1060420 (0.0005) [2023-12-27 13:00:10,594][20381] Updated weights for policy 1, policy_version 1060430 (0.0008) [2023-12-27 13:00:10,650][20381] Updated weights for policy 1, policy_version 1060440 (0.0009) [2023-12-27 13:00:10,769][20347] Updated weights for policy 0, policy_version 1061298 (0.0008) [2023-12-27 13:00:10,816][20347] Updated weights for policy 0, policy_version 1061308 (0.0007) [2023-12-27 13:00:10,873][20347] Updated weights for policy 0, policy_version 1061318 (0.0007) [2023-12-27 13:00:11,328][20381] Updated weights for policy 1, policy_version 1060450 (0.0008) [2023-12-27 13:00:11,400][20381] Updated weights for policy 1, policy_version 1060460 (0.0008) [2023-12-27 13:00:11,459][20381] Updated weights for policy 1, policy_version 1060470 (0.0008) [2023-12-27 13:00:11,611][20347] Updated weights for policy 0, policy_version 1061328 (0.0007) [2023-12-27 13:00:11,676][20347] Updated weights for policy 0, policy_version 1061338 (0.0006) [2023-12-27 13:00:11,734][20347] Updated weights for policy 0, policy_version 1061348 (0.0007) [2023-12-27 13:00:12,163][20381] Updated weights for policy 1, policy_version 1060480 (0.0009) [2023-12-27 13:00:12,223][20381] Updated weights for policy 1, policy_version 1060490 (0.0008) [2023-12-27 13:00:12,292][20381] Updated weights for policy 1, policy_version 1060500 (0.0009) [2023-12-27 13:00:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 543268864. Throughput: 0: 10313.0, 1: 10295.5. Samples: 543258972. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:12,363][19170] Avg episode reward: [(0, '805.340'), (1, '755.600')] [2023-12-27 13:00:12,458][20347] Updated weights for policy 0, policy_version 1061358 (0.0008) [2023-12-27 13:00:12,514][20347] Updated weights for policy 0, policy_version 1061368 (0.0007) [2023-12-27 13:00:12,574][20347] Updated weights for policy 0, policy_version 1061378 (0.0005) [2023-12-27 13:00:12,881][20381] Updated weights for policy 1, policy_version 1060510 (0.0007) [2023-12-27 13:00:12,945][20381] Updated weights for policy 1, policy_version 1060520 (0.0009) [2023-12-27 13:00:13,007][20381] Updated weights for policy 1, policy_version 1060530 (0.0008) [2023-12-27 13:00:13,249][20347] Updated weights for policy 0, policy_version 1061388 (0.0009) [2023-12-27 13:00:13,305][20347] Updated weights for policy 0, policy_version 1061398 (0.0011) [2023-12-27 13:00:13,362][20347] Updated weights for policy 0, policy_version 1061408 (0.0008) [2023-12-27 13:00:13,579][20381] Updated weights for policy 1, policy_version 1060540 (0.0005) [2023-12-27 13:00:13,643][20381] Updated weights for policy 1, policy_version 1060550 (0.0005) [2023-12-27 13:00:13,705][20381] Updated weights for policy 1, policy_version 1060560 (0.0006) [2023-12-27 13:00:13,939][20347] Updated weights for policy 0, policy_version 1061418 (0.0005) [2023-12-27 13:00:13,999][20347] Updated weights for policy 0, policy_version 1061428 (0.0005) [2023-12-27 13:00:14,056][20347] Updated weights for policy 0, policy_version 1061438 (0.0005) [2023-12-27 13:00:14,108][20347] Updated weights for policy 0, policy_version 1061448 (0.0008) [2023-12-27 13:00:14,291][20381] Updated weights for policy 1, policy_version 1060570 (0.0007) [2023-12-27 13:00:14,349][20381] Updated weights for policy 1, policy_version 1060580 (0.0007) [2023-12-27 13:00:14,411][20381] Updated weights for policy 1, policy_version 1060590 (0.0007) [2023-12-27 13:00:14,469][20381] Updated weights for policy 1, policy_version 1060600 (0.0006) [2023-12-27 13:00:14,746][20347] Updated weights for policy 0, policy_version 1061458 (0.0005) [2023-12-27 13:00:14,816][20347] Updated weights for policy 0, policy_version 1061468 (0.0007) [2023-12-27 13:00:14,878][20347] Updated weights for policy 0, policy_version 1061478 (0.0010) [2023-12-27 13:00:15,121][20381] Updated weights for policy 1, policy_version 1060610 (0.0006) [2023-12-27 13:00:15,177][20381] Updated weights for policy 1, policy_version 1060620 (0.0008) [2023-12-27 13:00:15,228][20381] Updated weights for policy 1, policy_version 1060630 (0.0006) [2023-12-27 13:00:15,466][20347] Updated weights for policy 0, policy_version 1061488 (0.0006) [2023-12-27 13:00:15,520][20347] Updated weights for policy 0, policy_version 1061498 (0.0005) [2023-12-27 13:00:15,574][20347] Updated weights for policy 0, policy_version 1061508 (0.0005) [2023-12-27 13:00:15,851][20381] Updated weights for policy 1, policy_version 1060640 (0.0005) [2023-12-27 13:00:15,902][20381] Updated weights for policy 1, policy_version 1060650 (0.0005) [2023-12-27 13:00:15,958][20381] Updated weights for policy 1, policy_version 1060660 (0.0005) [2023-12-27 13:00:16,133][20347] Updated weights for policy 0, policy_version 1061518 (0.0006) [2023-12-27 13:00:16,179][20347] Updated weights for policy 0, policy_version 1061528 (0.0006) [2023-12-27 13:00:16,231][20347] Updated weights for policy 0, policy_version 1061538 (0.0008) [2023-12-27 13:00:16,534][20381] Updated weights for policy 1, policy_version 1060670 (0.0005) [2023-12-27 13:00:16,591][20381] Updated weights for policy 1, policy_version 1060680 (0.0007) [2023-12-27 13:00:16,640][20381] Updated weights for policy 1, policy_version 1060690 (0.0007) [2023-12-27 13:00:16,922][20347] Updated weights for policy 0, policy_version 1061548 (0.0008) [2023-12-27 13:00:16,968][20347] Updated weights for policy 0, policy_version 1061558 (0.0007) [2023-12-27 13:00:17,014][20347] Updated weights for policy 0, policy_version 1061568 (0.0008) [2023-12-27 13:00:17,316][20381] Updated weights for policy 1, policy_version 1060700 (0.0007) [2023-12-27 13:00:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 543375360. Throughput: 0: 10318.8, 1: 10340.1. Samples: 543388052. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:17,363][19170] Avg episode reward: [(0, '844.110'), (1, '735.840')] [2023-12-27 13:00:17,374][20381] Updated weights for policy 1, policy_version 1060710 (0.0008) [2023-12-27 13:00:17,434][20381] Updated weights for policy 1, policy_version 1060720 (0.0008) [2023-12-27 13:00:17,709][20347] Updated weights for policy 0, policy_version 1061578 (0.0009) [2023-12-27 13:00:17,773][20347] Updated weights for policy 0, policy_version 1061588 (0.0010) [2023-12-27 13:00:17,824][20347] Updated weights for policy 0, policy_version 1061598 (0.0010) [2023-12-27 13:00:17,878][20347] Updated weights for policy 0, policy_version 1061608 (0.0009) [2023-12-27 13:00:18,133][20381] Updated weights for policy 1, policy_version 1060730 (0.0007) [2023-12-27 13:00:18,182][20381] Updated weights for policy 1, policy_version 1060740 (0.0008) [2023-12-27 13:00:18,232][20381] Updated weights for policy 1, policy_version 1060750 (0.0008) [2023-12-27 13:00:18,283][20381] Updated weights for policy 1, policy_version 1060760 (0.0007) [2023-12-27 13:00:18,510][20347] Updated weights for policy 0, policy_version 1061618 (0.0010) [2023-12-27 13:00:18,566][20347] Updated weights for policy 0, policy_version 1061628 (0.0010) [2023-12-27 13:00:18,627][20347] Updated weights for policy 0, policy_version 1061638 (0.0010) [2023-12-27 13:00:19,001][20381] Updated weights for policy 1, policy_version 1060770 (0.0009) [2023-12-27 13:00:19,065][20381] Updated weights for policy 1, policy_version 1060780 (0.0008) [2023-12-27 13:00:19,126][20381] Updated weights for policy 1, policy_version 1060790 (0.0008) [2023-12-27 13:00:19,362][20347] Updated weights for policy 0, policy_version 1061648 (0.0009) [2023-12-27 13:00:19,430][20347] Updated weights for policy 0, policy_version 1061658 (0.0010) [2023-12-27 13:00:19,495][20347] Updated weights for policy 0, policy_version 1061668 (0.0010) [2023-12-27 13:00:19,919][20381] Updated weights for policy 1, policy_version 1060800 (0.0008) [2023-12-27 13:00:19,986][20381] Updated weights for policy 1, policy_version 1060810 (0.0007) [2023-12-27 13:00:20,045][20381] Updated weights for policy 1, policy_version 1060820 (0.0008) [2023-12-27 13:00:20,197][20347] Updated weights for policy 0, policy_version 1061678 (0.0011) [2023-12-27 13:00:20,256][20347] Updated weights for policy 0, policy_version 1061688 (0.0011) [2023-12-27 13:00:20,313][20347] Updated weights for policy 0, policy_version 1061698 (0.0010) [2023-12-27 13:00:20,776][20381] Updated weights for policy 1, policy_version 1060830 (0.0007) [2023-12-27 13:00:20,838][20381] Updated weights for policy 1, policy_version 1060840 (0.0007) [2023-12-27 13:00:20,892][20381] Updated weights for policy 1, policy_version 1060850 (0.0007) [2023-12-27 13:00:21,041][20347] Updated weights for policy 0, policy_version 1061708 (0.0009) [2023-12-27 13:00:21,105][20347] Updated weights for policy 0, policy_version 1061718 (0.0008) [2023-12-27 13:00:21,172][20347] Updated weights for policy 0, policy_version 1061728 (0.0008) [2023-12-27 13:00:21,568][20381] Updated weights for policy 1, policy_version 1060860 (0.0006) [2023-12-27 13:00:21,632][20381] Updated weights for policy 1, policy_version 1060870 (0.0007) [2023-12-27 13:00:21,699][20381] Updated weights for policy 1, policy_version 1060880 (0.0008) [2023-12-27 13:00:21,862][20347] Updated weights for policy 0, policy_version 1061738 (0.0008) [2023-12-27 13:00:21,922][20347] Updated weights for policy 0, policy_version 1061748 (0.0008) [2023-12-27 13:00:21,983][20347] Updated weights for policy 0, policy_version 1061758 (0.0008) [2023-12-27 13:00:22,040][20347] Updated weights for policy 0, policy_version 1061768 (0.0007) [2023-12-27 13:00:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 543473664. Throughput: 0: 10292.0, 1: 10256.2. Samples: 543446260. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:22,363][19170] Avg episode reward: [(0, '853.890'), (1, '687.090')] [2023-12-27 13:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001060888_271613952.pth... [2023-12-27 13:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001061768_271859712.pth... [2023-12-27 13:00:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001059704_271310848.pth [2023-12-27 13:00:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001060552_271548416.pth [2023-12-27 13:00:22,410][20381] Updated weights for policy 1, policy_version 1060890 (0.0008) [2023-12-27 13:00:22,468][20381] Updated weights for policy 1, policy_version 1060900 (0.0009) [2023-12-27 13:00:22,525][20381] Updated weights for policy 1, policy_version 1060910 (0.0009) [2023-12-27 13:00:22,584][20381] Updated weights for policy 1, policy_version 1060920 (0.0008) [2023-12-27 13:00:22,782][20347] Updated weights for policy 0, policy_version 1061778 (0.0008) [2023-12-27 13:00:22,842][20347] Updated weights for policy 0, policy_version 1061788 (0.0008) [2023-12-27 13:00:22,901][20347] Updated weights for policy 0, policy_version 1061798 (0.0007) [2023-12-27 13:00:23,263][20381] Updated weights for policy 1, policy_version 1060930 (0.0008) [2023-12-27 13:00:23,320][20381] Updated weights for policy 1, policy_version 1060940 (0.0008) [2023-12-27 13:00:23,379][20381] Updated weights for policy 1, policy_version 1060950 (0.0008) [2023-12-27 13:00:23,581][20347] Updated weights for policy 0, policy_version 1061808 (0.0005) [2023-12-27 13:00:23,638][20347] Updated weights for policy 0, policy_version 1061818 (0.0008) [2023-12-27 13:00:23,696][20347] Updated weights for policy 0, policy_version 1061828 (0.0008) [2023-12-27 13:00:24,092][20381] Updated weights for policy 1, policy_version 1060960 (0.0009) [2023-12-27 13:00:24,151][20381] Updated weights for policy 1, policy_version 1060970 (0.0008) [2023-12-27 13:00:24,209][20381] Updated weights for policy 1, policy_version 1060980 (0.0008) [2023-12-27 13:00:24,325][20347] Updated weights for policy 0, policy_version 1061838 (0.0008) [2023-12-27 13:00:24,375][20347] Updated weights for policy 0, policy_version 1061848 (0.0007) [2023-12-27 13:00:24,430][20347] Updated weights for policy 0, policy_version 1061858 (0.0005) [2023-12-27 13:00:24,900][20381] Updated weights for policy 1, policy_version 1060990 (0.0008) [2023-12-27 13:00:24,958][20381] Updated weights for policy 1, policy_version 1061000 (0.0008) [2023-12-27 13:00:25,015][20347] Updated weights for policy 0, policy_version 1061868 (0.0006) [2023-12-27 13:00:25,018][20381] Updated weights for policy 1, policy_version 1061010 (0.0006) [2023-12-27 13:00:25,066][20347] Updated weights for policy 0, policy_version 1061878 (0.0006) [2023-12-27 13:00:25,115][20347] Updated weights for policy 0, policy_version 1061888 (0.0008) [2023-12-27 13:00:25,694][20381] Updated weights for policy 1, policy_version 1061020 (0.0007) [2023-12-27 13:00:25,755][20381] Updated weights for policy 1, policy_version 1061030 (0.0008) [2023-12-27 13:00:25,813][20381] Updated weights for policy 1, policy_version 1061040 (0.0008) [2023-12-27 13:00:25,821][20347] Updated weights for policy 0, policy_version 1061898 (0.0008) [2023-12-27 13:00:25,872][20347] Updated weights for policy 0, policy_version 1061908 (0.0006) [2023-12-27 13:00:25,935][20347] Updated weights for policy 0, policy_version 1061918 (0.0010) [2023-12-27 13:00:25,986][20347] Updated weights for policy 0, policy_version 1061928 (0.0010) [2023-12-27 13:00:26,456][20381] Updated weights for policy 1, policy_version 1061050 (0.0007) [2023-12-27 13:00:26,507][20381] Updated weights for policy 1, policy_version 1061060 (0.0005) [2023-12-27 13:00:26,563][20381] Updated weights for policy 1, policy_version 1061070 (0.0009) [2023-12-27 13:00:26,626][20381] Updated weights for policy 1, policy_version 1061080 (0.0008) [2023-12-27 13:00:26,683][20347] Updated weights for policy 0, policy_version 1061938 (0.0008) [2023-12-27 13:00:26,750][20347] Updated weights for policy 0, policy_version 1061948 (0.0007) [2023-12-27 13:00:26,809][20347] Updated weights for policy 0, policy_version 1061958 (0.0007) [2023-12-27 13:00:27,348][20347] Updated weights for policy 0, policy_version 1061968 (0.0009) [2023-12-27 13:00:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 543571968. Throughput: 0: 10302.4, 1: 10173.6. Samples: 543566368. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:27,363][19170] Avg episode reward: [(0, '824.780'), (1, '696.750')] [2023-12-27 13:00:27,368][20381] Updated weights for policy 1, policy_version 1061090 (0.0006) [2023-12-27 13:00:27,399][20347] Updated weights for policy 0, policy_version 1061978 (0.0010) [2023-12-27 13:00:27,423][20381] Updated weights for policy 1, policy_version 1061100 (0.0005) [2023-12-27 13:00:27,447][20347] Updated weights for policy 0, policy_version 1061988 (0.0010) [2023-12-27 13:00:27,478][20381] Updated weights for policy 1, policy_version 1061110 (0.0006) [2023-12-27 13:00:28,024][20347] Updated weights for policy 0, policy_version 1061998 (0.0010) [2023-12-27 13:00:28,072][20347] Updated weights for policy 0, policy_version 1062008 (0.0010) [2023-12-27 13:00:28,125][20347] Updated weights for policy 0, policy_version 1062018 (0.0011) [2023-12-27 13:00:28,177][20381] Updated weights for policy 1, policy_version 1061120 (0.0007) [2023-12-27 13:00:28,235][20381] Updated weights for policy 1, policy_version 1061130 (0.0008) [2023-12-27 13:00:28,296][20381] Updated weights for policy 1, policy_version 1061140 (0.0007) [2023-12-27 13:00:28,839][20347] Updated weights for policy 0, policy_version 1062028 (0.0010) [2023-12-27 13:00:28,903][20347] Updated weights for policy 0, policy_version 1062038 (0.0011) [2023-12-27 13:00:28,957][20347] Updated weights for policy 0, policy_version 1062048 (0.0011) [2023-12-27 13:00:28,988][20381] Updated weights for policy 1, policy_version 1061150 (0.0006) [2023-12-27 13:00:29,044][20381] Updated weights for policy 1, policy_version 1061160 (0.0005) [2023-12-27 13:00:29,101][20381] Updated weights for policy 1, policy_version 1061170 (0.0006) [2023-12-27 13:00:29,765][20347] Updated weights for policy 0, policy_version 1062058 (0.0010) [2023-12-27 13:00:29,808][20381] Updated weights for policy 1, policy_version 1061180 (0.0007) [2023-12-27 13:00:29,829][20347] Updated weights for policy 0, policy_version 1062068 (0.0008) [2023-12-27 13:00:29,874][20381] Updated weights for policy 1, policy_version 1061190 (0.0008) [2023-12-27 13:00:29,894][20347] Updated weights for policy 0, policy_version 1062078 (0.0006) [2023-12-27 13:00:29,939][20381] Updated weights for policy 1, policy_version 1061200 (0.0008) [2023-12-27 13:00:29,962][20347] Updated weights for policy 0, policy_version 1062088 (0.0008) [2023-12-27 13:00:30,601][20381] Updated weights for policy 1, policy_version 1061210 (0.0008) [2023-12-27 13:00:30,645][20381] Updated weights for policy 1, policy_version 1061220 (0.0007) [2023-12-27 13:00:30,695][20381] Updated weights for policy 1, policy_version 1061230 (0.0005) [2023-12-27 13:00:30,737][20347] Updated weights for policy 0, policy_version 1062098 (0.0008) [2023-12-27 13:00:30,755][20381] Updated weights for policy 1, policy_version 1061240 (0.0006) [2023-12-27 13:00:30,794][20347] Updated weights for policy 0, policy_version 1062108 (0.0008) [2023-12-27 13:00:30,849][20347] Updated weights for policy 0, policy_version 1062118 (0.0007) [2023-12-27 13:00:31,454][20381] Updated weights for policy 1, policy_version 1061250 (0.0007) [2023-12-27 13:00:31,505][20381] Updated weights for policy 1, policy_version 1061260 (0.0008) [2023-12-27 13:00:31,556][20347] Updated weights for policy 0, policy_version 1062128 (0.0008) [2023-12-27 13:00:31,563][20381] Updated weights for policy 1, policy_version 1061270 (0.0007) [2023-12-27 13:00:31,616][20347] Updated weights for policy 0, policy_version 1062138 (0.0008) [2023-12-27 13:00:31,687][20347] Updated weights for policy 0, policy_version 1062148 (0.0007) [2023-12-27 13:00:32,304][20381] Updated weights for policy 1, policy_version 1061280 (0.0008) [2023-12-27 13:00:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 543670272. Throughput: 0: 10272.4, 1: 10190.8. Samples: 543686564. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:32,363][19170] Avg episode reward: [(0, '825.650'), (1, '686.970')] [2023-12-27 13:00:32,370][20381] Updated weights for policy 1, policy_version 1061290 (0.0008) [2023-12-27 13:00:32,409][20347] Updated weights for policy 0, policy_version 1062158 (0.0008) [2023-12-27 13:00:32,436][20381] Updated weights for policy 1, policy_version 1061300 (0.0007) [2023-12-27 13:00:32,456][20347] Updated weights for policy 0, policy_version 1062168 (0.0007) [2023-12-27 13:00:32,512][20347] Updated weights for policy 0, policy_version 1062178 (0.0006) [2023-12-27 13:00:33,143][20347] Updated weights for policy 0, policy_version 1062188 (0.0007) [2023-12-27 13:00:33,181][20381] Updated weights for policy 1, policy_version 1061310 (0.0008) [2023-12-27 13:00:33,200][20347] Updated weights for policy 0, policy_version 1062198 (0.0006) [2023-12-27 13:00:33,242][20381] Updated weights for policy 1, policy_version 1061320 (0.0006) [2023-12-27 13:00:33,259][20347] Updated weights for policy 0, policy_version 1062208 (0.0008) [2023-12-27 13:00:33,295][20381] Updated weights for policy 1, policy_version 1061330 (0.0006) [2023-12-27 13:00:33,879][20347] Updated weights for policy 0, policy_version 1062218 (0.0008) [2023-12-27 13:00:33,898][20381] Updated weights for policy 1, policy_version 1061340 (0.0007) [2023-12-27 13:00:33,930][20347] Updated weights for policy 0, policy_version 1062228 (0.0010) [2023-12-27 13:00:33,953][20381] Updated weights for policy 1, policy_version 1061350 (0.0011) [2023-12-27 13:00:33,979][20347] Updated weights for policy 0, policy_version 1062238 (0.0010) [2023-12-27 13:00:34,005][20381] Updated weights for policy 1, policy_version 1061360 (0.0011) [2023-12-27 13:00:34,033][20347] Updated weights for policy 0, policy_version 1062248 (0.0010) [2023-12-27 13:00:34,678][20381] Updated weights for policy 1, policy_version 1061370 (0.0008) [2023-12-27 13:00:34,727][20347] Updated weights for policy 0, policy_version 1062258 (0.0006) [2023-12-27 13:00:34,740][20381] Updated weights for policy 1, policy_version 1061380 (0.0009) [2023-12-27 13:00:34,782][20347] Updated weights for policy 0, policy_version 1062268 (0.0005) [2023-12-27 13:00:34,790][20381] Updated weights for policy 1, policy_version 1061390 (0.0011) [2023-12-27 13:00:34,835][20347] Updated weights for policy 0, policy_version 1062278 (0.0005) [2023-12-27 13:00:34,839][20381] Updated weights for policy 1, policy_version 1061400 (0.0010) [2023-12-27 13:00:35,446][20347] Updated weights for policy 0, policy_version 1062288 (0.0007) [2023-12-27 13:00:35,489][20292] KL-divergence is very high: 115.4321 [2023-12-27 13:00:35,499][20381] Updated weights for policy 1, policy_version 1061410 (0.0010) [2023-12-27 13:00:35,500][20347] Updated weights for policy 0, policy_version 1062298 (0.0010) [2023-12-27 13:00:35,549][20381] Updated weights for policy 1, policy_version 1061420 (0.0010) [2023-12-27 13:00:35,552][20347] Updated weights for policy 0, policy_version 1062308 (0.0010) [2023-12-27 13:00:35,594][20381] Updated weights for policy 1, policy_version 1061430 (0.0009) [2023-12-27 13:00:36,160][20347] Updated weights for policy 0, policy_version 1062318 (0.0009) [2023-12-27 13:00:36,208][20347] Updated weights for policy 0, policy_version 1062328 (0.0008) [2023-12-27 13:00:36,264][20347] Updated weights for policy 0, policy_version 1062338 (0.0008) [2023-12-27 13:00:36,339][20381] Updated weights for policy 1, policy_version 1061440 (0.0011) [2023-12-27 13:00:36,389][20381] Updated weights for policy 1, policy_version 1061450 (0.0011) [2023-12-27 13:00:36,439][20381] Updated weights for policy 1, policy_version 1061460 (0.0010) [2023-12-27 13:00:37,004][20347] Updated weights for policy 0, policy_version 1062348 (0.0007) [2023-12-27 13:00:37,063][20347] Updated weights for policy 0, policy_version 1062358 (0.0005) [2023-12-27 13:00:37,123][20347] Updated weights for policy 0, policy_version 1062368 (0.0007) [2023-12-27 13:00:37,159][20381] Updated weights for policy 1, policy_version 1061470 (0.0007) [2023-12-27 13:00:37,211][20381] Updated weights for policy 1, policy_version 1061480 (0.0005) [2023-12-27 13:00:37,273][20381] Updated weights for policy 1, policy_version 1061490 (0.0005) [2023-12-27 13:00:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 543784960. Throughput: 0: 10339.0, 1: 10194.3. Samples: 543749232. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:37,363][19170] Avg episode reward: [(0, '766.380'), (1, '696.450')] [2023-12-27 13:00:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001061496_271769600.pth... [2023-12-27 13:00:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001062376_272015360.pth... [2023-12-27 13:00:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001061160_271704064.pth [2023-12-27 13:00:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001060312_271466496.pth [2023-12-27 13:00:37,776][20347] Updated weights for policy 0, policy_version 1062378 (0.0006) [2023-12-27 13:00:37,831][20347] Updated weights for policy 0, policy_version 1062388 (0.0008) [2023-12-27 13:00:37,884][20347] Updated weights for policy 0, policy_version 1062398 (0.0007) [2023-12-27 13:00:37,932][20347] Updated weights for policy 0, policy_version 1062408 (0.0007) [2023-12-27 13:00:37,937][20381] Updated weights for policy 1, policy_version 1061500 (0.0007) [2023-12-27 13:00:37,993][20381] Updated weights for policy 1, policy_version 1061510 (0.0007) [2023-12-27 13:00:38,042][20381] Updated weights for policy 1, policy_version 1061520 (0.0008) [2023-12-27 13:00:38,651][20347] Updated weights for policy 0, policy_version 1062418 (0.0007) [2023-12-27 13:00:38,690][20291] KL-divergence is very high: 128.2834 [2023-12-27 13:00:38,713][20347] Updated weights for policy 0, policy_version 1062428 (0.0008) [2023-12-27 13:00:38,739][20381] Updated weights for policy 1, policy_version 1061530 (0.0008) [2023-12-27 13:00:38,772][20347] Updated weights for policy 0, policy_version 1062438 (0.0007) [2023-12-27 13:00:38,797][20381] Updated weights for policy 1, policy_version 1061540 (0.0006) [2023-12-27 13:00:38,857][20381] Updated weights for policy 1, policy_version 1061550 (0.0007) [2023-12-27 13:00:38,923][20381] Updated weights for policy 1, policy_version 1061560 (0.0005) [2023-12-27 13:00:39,508][20347] Updated weights for policy 0, policy_version 1062448 (0.0007) [2023-12-27 13:00:39,563][20347] Updated weights for policy 0, policy_version 1062458 (0.0008) [2023-12-27 13:00:39,592][20381] Updated weights for policy 1, policy_version 1061570 (0.0011) [2023-12-27 13:00:39,620][20347] Updated weights for policy 0, policy_version 1062468 (0.0006) [2023-12-27 13:00:39,653][20381] Updated weights for policy 1, policy_version 1061580 (0.0010) [2023-12-27 13:00:39,710][20381] Updated weights for policy 1, policy_version 1061590 (0.0011) [2023-12-27 13:00:40,363][20381] Updated weights for policy 1, policy_version 1061600 (0.0008) [2023-12-27 13:00:40,386][20347] Updated weights for policy 0, policy_version 1062478 (0.0007) [2023-12-27 13:00:40,435][20381] Updated weights for policy 1, policy_version 1061610 (0.0007) [2023-12-27 13:00:40,443][20347] Updated weights for policy 0, policy_version 1062488 (0.0008) [2023-12-27 13:00:40,500][20347] Updated weights for policy 0, policy_version 1062498 (0.0006) [2023-12-27 13:00:40,505][20291] KL-divergence is very high: 266.4255 [2023-12-27 13:00:40,510][20381] Updated weights for policy 1, policy_version 1061620 (0.0007) [2023-12-27 13:00:41,207][20381] Updated weights for policy 1, policy_version 1061630 (0.0009) [2023-12-27 13:00:41,241][20347] Updated weights for policy 0, policy_version 1062508 (0.0006) [2023-12-27 13:00:41,273][20381] Updated weights for policy 1, policy_version 1061640 (0.0010) [2023-12-27 13:00:41,309][20347] Updated weights for policy 0, policy_version 1062518 (0.0007) [2023-12-27 13:00:41,350][20381] Updated weights for policy 1, policy_version 1061650 (0.0007) [2023-12-27 13:00:41,380][20347] Updated weights for policy 0, policy_version 1062528 (0.0007) [2023-12-27 13:00:42,021][20381] Updated weights for policy 1, policy_version 1061660 (0.0008) [2023-12-27 13:00:42,087][20381] Updated weights for policy 1, policy_version 1061670 (0.0008) [2023-12-27 13:00:42,103][20347] Updated weights for policy 0, policy_version 1062538 (0.0008) [2023-12-27 13:00:42,144][20381] Updated weights for policy 1, policy_version 1061680 (0.0006) [2023-12-27 13:00:42,161][20347] Updated weights for policy 0, policy_version 1062548 (0.0008) [2023-12-27 13:00:42,210][20291] KL-divergence is very high: 113.1296 [2023-12-27 13:00:42,223][20347] Updated weights for policy 0, policy_version 1062558 (0.0007) [2023-12-27 13:00:42,283][20347] Updated weights for policy 0, policy_version 1062568 (0.0009) [2023-12-27 13:00:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 543883264. Throughput: 0: 10310.9, 1: 10139.2. Samples: 543869888. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:42,363][19170] Avg episode reward: [(0, '748.150'), (1, '696.560')] [2023-12-27 13:00:42,796][20381] Updated weights for policy 1, policy_version 1061690 (0.0007) [2023-12-27 13:00:42,857][20381] Updated weights for policy 1, policy_version 1061700 (0.0008) [2023-12-27 13:00:42,910][20381] Updated weights for policy 1, policy_version 1061710 (0.0009) [2023-12-27 13:00:42,949][20347] Updated weights for policy 0, policy_version 1062578 (0.0005) [2023-12-27 13:00:42,966][20381] Updated weights for policy 1, policy_version 1061720 (0.0008) [2023-12-27 13:00:43,004][20347] Updated weights for policy 0, policy_version 1062588 (0.0005) [2023-12-27 13:00:43,050][20347] Updated weights for policy 0, policy_version 1062598 (0.0005) [2023-12-27 13:00:43,663][20381] Updated weights for policy 1, policy_version 1061730 (0.0007) [2023-12-27 13:00:43,690][20347] Updated weights for policy 0, policy_version 1062608 (0.0006) [2023-12-27 13:00:43,721][20381] Updated weights for policy 1, policy_version 1061740 (0.0006) [2023-12-27 13:00:43,742][20347] Updated weights for policy 0, policy_version 1062618 (0.0005) [2023-12-27 13:00:43,770][20381] Updated weights for policy 1, policy_version 1061750 (0.0006) [2023-12-27 13:00:43,794][20347] Updated weights for policy 0, policy_version 1062628 (0.0006) [2023-12-27 13:00:44,389][20381] Updated weights for policy 1, policy_version 1061760 (0.0007) [2023-12-27 13:00:44,439][20381] Updated weights for policy 1, policy_version 1061770 (0.0008) [2023-12-27 13:00:44,487][20381] Updated weights for policy 1, policy_version 1061780 (0.0008) [2023-12-27 13:00:44,497][20347] Updated weights for policy 0, policy_version 1062638 (0.0007) [2023-12-27 13:00:44,548][20347] Updated weights for policy 0, policy_version 1062648 (0.0007) [2023-12-27 13:00:44,608][20347] Updated weights for policy 0, policy_version 1062658 (0.0006) [2023-12-27 13:00:45,240][20381] Updated weights for policy 1, policy_version 1061790 (0.0005) [2023-12-27 13:00:45,276][20347] Updated weights for policy 0, policy_version 1062668 (0.0007) [2023-12-27 13:00:45,297][20381] Updated weights for policy 1, policy_version 1061800 (0.0007) [2023-12-27 13:00:45,327][20347] Updated weights for policy 0, policy_version 1062678 (0.0006) [2023-12-27 13:00:45,348][20381] Updated weights for policy 1, policy_version 1061810 (0.0006) [2023-12-27 13:00:45,373][20347] Updated weights for policy 0, policy_version 1062688 (0.0006) [2023-12-27 13:00:46,037][20381] Updated weights for policy 1, policy_version 1061820 (0.0006) [2023-12-27 13:00:46,090][20381] Updated weights for policy 1, policy_version 1061830 (0.0008) [2023-12-27 13:00:46,105][20347] Updated weights for policy 0, policy_version 1062698 (0.0008) [2023-12-27 13:00:46,149][20381] Updated weights for policy 1, policy_version 1061840 (0.0006) [2023-12-27 13:00:46,163][20347] Updated weights for policy 0, policy_version 1062708 (0.0006) [2023-12-27 13:00:46,220][20347] Updated weights for policy 0, policy_version 1062718 (0.0006) [2023-12-27 13:00:46,283][20347] Updated weights for policy 0, policy_version 1062728 (0.0007) [2023-12-27 13:00:46,767][20381] Updated weights for policy 1, policy_version 1061850 (0.0006) [2023-12-27 13:00:46,826][20381] Updated weights for policy 1, policy_version 1061860 (0.0008) [2023-12-27 13:00:46,881][20381] Updated weights for policy 1, policy_version 1061870 (0.0008) [2023-12-27 13:00:46,937][20381] Updated weights for policy 1, policy_version 1061880 (0.0008) [2023-12-27 13:00:46,995][20347] Updated weights for policy 0, policy_version 1062738 (0.0007) [2023-12-27 13:00:47,042][20347] Updated weights for policy 0, policy_version 1062748 (0.0007) [2023-12-27 13:00:47,086][20347] Updated weights for policy 0, policy_version 1062758 (0.0007) [2023-12-27 13:00:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 543981568. Throughput: 0: 10262.1, 1: 10162.8. Samples: 543991040. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:47,363][19170] Avg episode reward: [(0, '825.360'), (1, '726.020')] [2023-12-27 13:00:47,621][20381] Updated weights for policy 1, policy_version 1061890 (0.0008) [2023-12-27 13:00:47,680][20381] Updated weights for policy 1, policy_version 1061900 (0.0008) [2023-12-27 13:00:47,736][20381] Updated weights for policy 1, policy_version 1061910 (0.0007) [2023-12-27 13:00:47,798][20347] Updated weights for policy 0, policy_version 1062768 (0.0008) [2023-12-27 13:00:47,857][20347] Updated weights for policy 0, policy_version 1062778 (0.0007) [2023-12-27 13:00:47,902][20347] Updated weights for policy 0, policy_version 1062788 (0.0008) [2023-12-27 13:00:48,443][20381] Updated weights for policy 1, policy_version 1061920 (0.0007) [2023-12-27 13:00:48,501][20381] Updated weights for policy 1, policy_version 1061930 (0.0006) [2023-12-27 13:00:48,556][20381] Updated weights for policy 1, policy_version 1061940 (0.0006) [2023-12-27 13:00:48,617][20347] Updated weights for policy 0, policy_version 1062798 (0.0007) [2023-12-27 13:00:48,667][20347] Updated weights for policy 0, policy_version 1062808 (0.0008) [2023-12-27 13:00:48,720][20347] Updated weights for policy 0, policy_version 1062818 (0.0008) [2023-12-27 13:00:49,147][20381] Updated weights for policy 1, policy_version 1061950 (0.0007) [2023-12-27 13:00:49,207][20381] Updated weights for policy 1, policy_version 1061960 (0.0007) [2023-12-27 13:00:49,274][20381] Updated weights for policy 1, policy_version 1061970 (0.0009) [2023-12-27 13:00:49,462][20347] Updated weights for policy 0, policy_version 1062828 (0.0008) [2023-12-27 13:00:49,521][20347] Updated weights for policy 0, policy_version 1062838 (0.0009) [2023-12-27 13:00:49,580][20347] Updated weights for policy 0, policy_version 1062848 (0.0007) [2023-12-27 13:00:50,027][20381] Updated weights for policy 1, policy_version 1061980 (0.0007) [2023-12-27 13:00:50,083][20381] Updated weights for policy 1, policy_version 1061990 (0.0006) [2023-12-27 13:00:50,136][20381] Updated weights for policy 1, policy_version 1062000 (0.0007) [2023-12-27 13:00:50,338][20347] Updated weights for policy 0, policy_version 1062858 (0.0007) [2023-12-27 13:00:50,394][20347] Updated weights for policy 0, policy_version 1062868 (0.0008) [2023-12-27 13:00:50,453][20347] Updated weights for policy 0, policy_version 1062878 (0.0008) [2023-12-27 13:00:50,504][20347] Updated weights for policy 0, policy_version 1062888 (0.0008) [2023-12-27 13:00:50,762][20381] Updated weights for policy 1, policy_version 1062010 (0.0006) [2023-12-27 13:00:50,818][20381] Updated weights for policy 1, policy_version 1062020 (0.0007) [2023-12-27 13:00:50,868][20381] Updated weights for policy 1, policy_version 1062030 (0.0008) [2023-12-27 13:00:50,922][20381] Updated weights for policy 1, policy_version 1062040 (0.0009) [2023-12-27 13:00:51,232][20347] Updated weights for policy 0, policy_version 1062898 (0.0006) [2023-12-27 13:00:51,291][20347] Updated weights for policy 0, policy_version 1062908 (0.0008) [2023-12-27 13:00:51,358][20347] Updated weights for policy 0, policy_version 1062918 (0.0008) [2023-12-27 13:00:51,723][20381] Updated weights for policy 1, policy_version 1062050 (0.0009) [2023-12-27 13:00:51,789][20381] Updated weights for policy 1, policy_version 1062060 (0.0008) [2023-12-27 13:00:51,846][20381] Updated weights for policy 1, policy_version 1062070 (0.0008) [2023-12-27 13:00:52,106][20347] Updated weights for policy 0, policy_version 1062928 (0.0007) [2023-12-27 13:00:52,159][20347] Updated weights for policy 0, policy_version 1062938 (0.0005) [2023-12-27 13:00:52,213][20347] Updated weights for policy 0, policy_version 1062948 (0.0005) [2023-12-27 13:00:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 544079872. Throughput: 0: 10210.3, 1: 10119.5. Samples: 544050344. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:52,363][19170] Avg episode reward: [(0, '824.520'), (1, '716.530')] [2023-12-27 13:00:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001062072_271917056.pth... [2023-12-27 13:00:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001062952_272162816.pth... [2023-12-27 13:00:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001060888_271613952.pth [2023-12-27 13:00:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001061768_271859712.pth [2023-12-27 13:00:52,530][20381] Updated weights for policy 1, policy_version 1062080 (0.0007) [2023-12-27 13:00:52,594][20381] Updated weights for policy 1, policy_version 1062090 (0.0008) [2023-12-27 13:00:52,654][20381] Updated weights for policy 1, policy_version 1062100 (0.0007) [2023-12-27 13:00:52,863][20347] Updated weights for policy 0, policy_version 1062958 (0.0007) [2023-12-27 13:00:52,914][20347] Updated weights for policy 0, policy_version 1062968 (0.0007) [2023-12-27 13:00:52,968][20347] Updated weights for policy 0, policy_version 1062978 (0.0008) [2023-12-27 13:00:53,305][20381] Updated weights for policy 1, policy_version 1062110 (0.0009) [2023-12-27 13:00:53,362][20381] Updated weights for policy 1, policy_version 1062120 (0.0009) [2023-12-27 13:00:53,426][20381] Updated weights for policy 1, policy_version 1062130 (0.0008) [2023-12-27 13:00:53,641][20347] Updated weights for policy 0, policy_version 1062988 (0.0006) [2023-12-27 13:00:53,691][20347] Updated weights for policy 0, policy_version 1062998 (0.0006) [2023-12-27 13:00:53,748][20347] Updated weights for policy 0, policy_version 1063008 (0.0008) [2023-12-27 13:00:54,024][20381] Updated weights for policy 1, policy_version 1062140 (0.0009) [2023-12-27 13:00:54,075][20381] Updated weights for policy 1, policy_version 1062150 (0.0007) [2023-12-27 13:00:54,133][20381] Updated weights for policy 1, policy_version 1062160 (0.0007) [2023-12-27 13:00:54,408][20347] Updated weights for policy 0, policy_version 1063018 (0.0008) [2023-12-27 13:00:54,456][20347] Updated weights for policy 0, policy_version 1063028 (0.0010) [2023-12-27 13:00:54,500][20347] Updated weights for policy 0, policy_version 1063038 (0.0010) [2023-12-27 13:00:54,506][20291] KL-divergence is very high: 104.5097 [2023-12-27 13:00:54,555][20347] Updated weights for policy 0, policy_version 1063048 (0.0010) [2023-12-27 13:00:54,840][20381] Updated weights for policy 1, policy_version 1062170 (0.0007) [2023-12-27 13:00:54,901][20381] Updated weights for policy 1, policy_version 1062180 (0.0008) [2023-12-27 13:00:54,956][20381] Updated weights for policy 1, policy_version 1062190 (0.0007) [2023-12-27 13:00:55,020][20381] Updated weights for policy 1, policy_version 1062200 (0.0007) [2023-12-27 13:00:55,266][20347] Updated weights for policy 0, policy_version 1063058 (0.0008) [2023-12-27 13:00:55,326][20347] Updated weights for policy 0, policy_version 1063068 (0.0006) [2023-12-27 13:00:55,373][20347] Updated weights for policy 0, policy_version 1063078 (0.0008) [2023-12-27 13:00:55,597][20381] Updated weights for policy 1, policy_version 1062210 (0.0005) [2023-12-27 13:00:55,650][20381] Updated weights for policy 1, policy_version 1062220 (0.0007) [2023-12-27 13:00:55,700][20381] Updated weights for policy 1, policy_version 1062230 (0.0005) [2023-12-27 13:00:55,944][20347] Updated weights for policy 0, policy_version 1063088 (0.0006) [2023-12-27 13:00:55,999][20347] Updated weights for policy 0, policy_version 1063098 (0.0008) [2023-12-27 13:00:56,053][20347] Updated weights for policy 0, policy_version 1063108 (0.0008) [2023-12-27 13:00:56,259][20381] Updated weights for policy 1, policy_version 1062240 (0.0008) [2023-12-27 13:00:56,319][20381] Updated weights for policy 1, policy_version 1062250 (0.0009) [2023-12-27 13:00:56,378][20381] Updated weights for policy 1, policy_version 1062260 (0.0006) [2023-12-27 13:00:56,671][20347] Updated weights for policy 0, policy_version 1063118 (0.0007) [2023-12-27 13:00:56,723][20347] Updated weights for policy 0, policy_version 1063128 (0.0008) [2023-12-27 13:00:56,778][20347] Updated weights for policy 0, policy_version 1063138 (0.0010) [2023-12-27 13:00:57,034][20381] Updated weights for policy 1, policy_version 1062270 (0.0007) [2023-12-27 13:00:57,088][20381] Updated weights for policy 1, policy_version 1062280 (0.0009) [2023-12-27 13:00:57,140][20381] Updated weights for policy 1, policy_version 1062290 (0.0008) [2023-12-27 13:00:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 544186368. Throughput: 0: 10155.0, 1: 10215.0. Samples: 544175620. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:00:57,363][19170] Avg episode reward: [(0, '814.840'), (1, '726.330')] [2023-12-27 13:00:57,404][20347] Updated weights for policy 0, policy_version 1063148 (0.0010) [2023-12-27 13:00:57,451][20347] Updated weights for policy 0, policy_version 1063158 (0.0010) [2023-12-27 13:00:57,504][20347] Updated weights for policy 0, policy_version 1063168 (0.0010) [2023-12-27 13:00:57,858][20381] Updated weights for policy 1, policy_version 1062300 (0.0006) [2023-12-27 13:00:57,913][20381] Updated weights for policy 1, policy_version 1062310 (0.0005) [2023-12-27 13:00:57,925][20292] KL-divergence is very high: 107.7440 [2023-12-27 13:00:57,968][20381] Updated weights for policy 1, policy_version 1062320 (0.0005) [2023-12-27 13:00:58,100][20347] Updated weights for policy 0, policy_version 1063178 (0.0009) [2023-12-27 13:00:58,162][20347] Updated weights for policy 0, policy_version 1063188 (0.0010) [2023-12-27 13:00:58,225][20347] Updated weights for policy 0, policy_version 1063198 (0.0010) [2023-12-27 13:00:58,287][20347] Updated weights for policy 0, policy_version 1063208 (0.0010) [2023-12-27 13:00:58,630][20381] Updated weights for policy 1, policy_version 1062330 (0.0008) [2023-12-27 13:00:58,695][20381] Updated weights for policy 1, policy_version 1062340 (0.0007) [2023-12-27 13:00:58,768][20381] Updated weights for policy 1, policy_version 1062350 (0.0008) [2023-12-27 13:00:58,834][20381] Updated weights for policy 1, policy_version 1062360 (0.0008) [2023-12-27 13:00:59,066][20347] Updated weights for policy 0, policy_version 1063218 (0.0008) [2023-12-27 13:00:59,127][20347] Updated weights for policy 0, policy_version 1063228 (0.0007) [2023-12-27 13:00:59,188][20347] Updated weights for policy 0, policy_version 1063238 (0.0008) [2023-12-27 13:00:59,583][20381] Updated weights for policy 1, policy_version 1062370 (0.0009) [2023-12-27 13:00:59,636][20381] Updated weights for policy 1, policy_version 1062380 (0.0011) [2023-12-27 13:00:59,690][20381] Updated weights for policy 1, policy_version 1062390 (0.0008) [2023-12-27 13:00:59,923][20347] Updated weights for policy 0, policy_version 1063248 (0.0008) [2023-12-27 13:00:59,984][20347] Updated weights for policy 0, policy_version 1063258 (0.0006) [2023-12-27 13:01:00,041][20347] Updated weights for policy 0, policy_version 1063268 (0.0008) [2023-12-27 13:01:00,391][20381] Updated weights for policy 1, policy_version 1062400 (0.0005) [2023-12-27 13:01:00,451][20381] Updated weights for policy 1, policy_version 1062410 (0.0008) [2023-12-27 13:01:00,506][20381] Updated weights for policy 1, policy_version 1062420 (0.0008) [2023-12-27 13:01:00,716][20347] Updated weights for policy 0, policy_version 1063278 (0.0007) [2023-12-27 13:01:00,765][20347] Updated weights for policy 0, policy_version 1063288 (0.0005) [2023-12-27 13:01:00,813][20347] Updated weights for policy 0, policy_version 1063298 (0.0005) [2023-12-27 13:01:01,117][20381] Updated weights for policy 1, policy_version 1062430 (0.0008) [2023-12-27 13:01:01,177][20381] Updated weights for policy 1, policy_version 1062440 (0.0007) [2023-12-27 13:01:01,231][20381] Updated weights for policy 1, policy_version 1062450 (0.0007) [2023-12-27 13:01:01,521][20347] Updated weights for policy 0, policy_version 1063308 (0.0008) [2023-12-27 13:01:01,572][20347] Updated weights for policy 0, policy_version 1063318 (0.0007) [2023-12-27 13:01:01,634][20347] Updated weights for policy 0, policy_version 1063328 (0.0008) [2023-12-27 13:01:01,908][20381] Updated weights for policy 1, policy_version 1062460 (0.0006) [2023-12-27 13:01:01,958][20381] Updated weights for policy 1, policy_version 1062470 (0.0007) [2023-12-27 13:01:02,008][20381] Updated weights for policy 1, policy_version 1062480 (0.0009) [2023-12-27 13:01:02,357][20347] Updated weights for policy 0, policy_version 1063338 (0.0008) [2023-12-27 13:01:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 544284672. Throughput: 0: 10101.4, 1: 10116.1. Samples: 544297840. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:02,363][19170] Avg episode reward: [(0, '814.920'), (1, '716.550')] [2023-12-27 13:01:02,415][20347] Updated weights for policy 0, policy_version 1063348 (0.0010) [2023-12-27 13:01:02,474][20347] Updated weights for policy 0, policy_version 1063358 (0.0010) [2023-12-27 13:01:02,529][20347] Updated weights for policy 0, policy_version 1063368 (0.0010) [2023-12-27 13:01:02,644][20381] Updated weights for policy 1, policy_version 1062490 (0.0006) [2023-12-27 13:01:02,696][20381] Updated weights for policy 1, policy_version 1062500 (0.0007) [2023-12-27 13:01:02,751][20381] Updated weights for policy 1, policy_version 1062510 (0.0008) [2023-12-27 13:01:02,810][20381] Updated weights for policy 1, policy_version 1062520 (0.0008) [2023-12-27 13:01:03,180][20347] Updated weights for policy 0, policy_version 1063378 (0.0007) [2023-12-27 13:01:03,234][20347] Updated weights for policy 0, policy_version 1063388 (0.0010) [2023-12-27 13:01:03,293][20347] Updated weights for policy 0, policy_version 1063398 (0.0010) [2023-12-27 13:01:03,529][20381] Updated weights for policy 1, policy_version 1062530 (0.0007) [2023-12-27 13:01:03,588][20381] Updated weights for policy 1, policy_version 1062540 (0.0007) [2023-12-27 13:01:03,646][20381] Updated weights for policy 1, policy_version 1062550 (0.0008) [2023-12-27 13:01:03,936][20347] Updated weights for policy 0, policy_version 1063408 (0.0009) [2023-12-27 13:01:03,997][20347] Updated weights for policy 0, policy_version 1063418 (0.0010) [2023-12-27 13:01:04,056][20347] Updated weights for policy 0, policy_version 1063428 (0.0010) [2023-12-27 13:01:04,283][20381] Updated weights for policy 1, policy_version 1062560 (0.0008) [2023-12-27 13:01:04,337][20381] Updated weights for policy 1, policy_version 1062570 (0.0008) [2023-12-27 13:01:04,401][20381] Updated weights for policy 1, policy_version 1062580 (0.0010) [2023-12-27 13:01:04,735][20347] Updated weights for policy 0, policy_version 1063438 (0.0010) [2023-12-27 13:01:04,787][20347] Updated weights for policy 0, policy_version 1063448 (0.0010) [2023-12-27 13:01:04,832][20347] Updated weights for policy 0, policy_version 1063458 (0.0010) [2023-12-27 13:01:05,148][20381] Updated weights for policy 1, policy_version 1062590 (0.0008) [2023-12-27 13:01:05,194][20381] Updated weights for policy 1, policy_version 1062600 (0.0005) [2023-12-27 13:01:05,241][20381] Updated weights for policy 1, policy_version 1062610 (0.0005) [2023-12-27 13:01:05,527][20347] Updated weights for policy 0, policy_version 1063468 (0.0010) [2023-12-27 13:01:05,587][20347] Updated weights for policy 0, policy_version 1063478 (0.0010) [2023-12-27 13:01:05,603][20291] KL-divergence is very high: 184.1010 [2023-12-27 13:01:05,610][20291] KL-divergence is very high: 188.7072 [2023-12-27 13:01:05,655][20347] Updated weights for policy 0, policy_version 1063488 (0.0010) [2023-12-27 13:01:05,655][20291] KL-divergence is very high: 133.9761 [2023-12-27 13:01:05,661][20291] KL-divergence is very high: 112.9851 [2023-12-27 13:01:05,809][20381] Updated weights for policy 1, policy_version 1062620 (0.0007) [2023-12-27 13:01:05,862][20381] Updated weights for policy 1, policy_version 1062630 (0.0005) [2023-12-27 13:01:05,916][20381] Updated weights for policy 1, policy_version 1062640 (0.0005) [2023-12-27 13:01:06,310][20347] Updated weights for policy 0, policy_version 1063498 (0.0011) [2023-12-27 13:01:06,324][20291] KL-divergence is very high: 115.2845 [2023-12-27 13:01:06,375][20347] Updated weights for policy 0, policy_version 1063508 (0.0010) [2023-12-27 13:01:06,375][20291] KL-divergence is very high: 120.8037 [2023-12-27 13:01:06,437][20347] Updated weights for policy 0, policy_version 1063518 (0.0006) [2023-12-27 13:01:06,501][20347] Updated weights for policy 0, policy_version 1063528 (0.0008) [2023-12-27 13:01:06,501][20381] Updated weights for policy 1, policy_version 1062650 (0.0005) [2023-12-27 13:01:06,561][20381] Updated weights for policy 1, policy_version 1062660 (0.0007) [2023-12-27 13:01:06,622][20381] Updated weights for policy 1, policy_version 1062670 (0.0007) [2023-12-27 13:01:06,682][20381] Updated weights for policy 1, policy_version 1062680 (0.0008) [2023-12-27 13:01:07,189][20347] Updated weights for policy 0, policy_version 1063538 (0.0008) [2023-12-27 13:01:07,241][20347] Updated weights for policy 0, policy_version 1063548 (0.0007) [2023-12-27 13:01:07,294][20347] Updated weights for policy 0, policy_version 1063558 (0.0009) [2023-12-27 13:01:07,350][20381] Updated weights for policy 1, policy_version 1062690 (0.0006) [2023-12-27 13:01:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 544391168. Throughput: 0: 10116.1, 1: 10161.9. Samples: 544358776. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:07,364][19170] Avg episode reward: [(0, '824.370'), (1, '628.590')] [2023-12-27 13:01:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001063560_272318464.pth... [2023-12-27 13:01:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001062376_272015360.pth [2023-12-27 13:01:07,409][20381] Updated weights for policy 1, policy_version 1062700 (0.0008) [2023-12-27 13:01:07,467][20381] Updated weights for policy 1, policy_version 1062710 (0.0008) [2023-12-27 13:01:07,477][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001062712_272080896.pth... [2023-12-27 13:01:07,481][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001061496_271769600.pth [2023-12-27 13:01:07,998][20347] Updated weights for policy 0, policy_version 1063568 (0.0008) [2023-12-27 13:01:08,050][20347] Updated weights for policy 0, policy_version 1063578 (0.0008) [2023-12-27 13:01:08,085][20381] Updated weights for policy 1, policy_version 1062720 (0.0006) [2023-12-27 13:01:08,105][20347] Updated weights for policy 0, policy_version 1063588 (0.0006) [2023-12-27 13:01:08,130][20381] Updated weights for policy 1, policy_version 1062730 (0.0006) [2023-12-27 13:01:08,179][20381] Updated weights for policy 1, policy_version 1062740 (0.0005) [2023-12-27 13:01:08,773][20347] Updated weights for policy 0, policy_version 1063598 (0.0008) [2023-12-27 13:01:08,835][20347] Updated weights for policy 0, policy_version 1063608 (0.0008) [2023-12-27 13:01:08,872][20381] Updated weights for policy 1, policy_version 1062750 (0.0006) [2023-12-27 13:01:08,885][20347] Updated weights for policy 0, policy_version 1063618 (0.0007) [2023-12-27 13:01:08,928][20381] Updated weights for policy 1, policy_version 1062760 (0.0006) [2023-12-27 13:01:08,985][20381] Updated weights for policy 1, policy_version 1062770 (0.0008) [2023-12-27 13:01:09,570][20347] Updated weights for policy 0, policy_version 1063628 (0.0005) [2023-12-27 13:01:09,629][20347] Updated weights for policy 0, policy_version 1063638 (0.0009) [2023-12-27 13:01:09,687][20381] Updated weights for policy 1, policy_version 1062780 (0.0008) [2023-12-27 13:01:09,688][20347] Updated weights for policy 0, policy_version 1063648 (0.0008) [2023-12-27 13:01:09,745][20381] Updated weights for policy 1, policy_version 1062790 (0.0007) [2023-12-27 13:01:09,806][20381] Updated weights for policy 1, policy_version 1062800 (0.0008) [2023-12-27 13:01:10,425][20347] Updated weights for policy 0, policy_version 1063658 (0.0007) [2023-12-27 13:01:10,490][20347] Updated weights for policy 0, policy_version 1063668 (0.0007) [2023-12-27 13:01:10,494][20381] Updated weights for policy 1, policy_version 1062810 (0.0006) [2023-12-27 13:01:10,553][20347] Updated weights for policy 0, policy_version 1063678 (0.0006) [2023-12-27 13:01:10,560][20381] Updated weights for policy 1, policy_version 1062820 (0.0006) [2023-12-27 13:01:10,610][20347] Updated weights for policy 0, policy_version 1063688 (0.0008) [2023-12-27 13:01:10,628][20381] Updated weights for policy 1, policy_version 1062830 (0.0006) [2023-12-27 13:01:10,690][20381] Updated weights for policy 1, policy_version 1062840 (0.0005) [2023-12-27 13:01:11,327][20381] Updated weights for policy 1, policy_version 1062850 (0.0005) [2023-12-27 13:01:11,341][20347] Updated weights for policy 0, policy_version 1063698 (0.0008) [2023-12-27 13:01:11,403][20381] Updated weights for policy 1, policy_version 1062861 (0.0009) [2023-12-27 13:01:11,405][20347] Updated weights for policy 0, policy_version 1063708 (0.0008) [2023-12-27 13:01:11,459][20381] Updated weights for policy 1, policy_version 1062871 (0.0006) [2023-12-27 13:01:11,473][20347] Updated weights for policy 0, policy_version 1063718 (0.0008) [2023-12-27 13:01:12,185][20381] Updated weights for policy 1, policy_version 1062881 (0.0007) [2023-12-27 13:01:12,220][20347] Updated weights for policy 0, policy_version 1063728 (0.0007) [2023-12-27 13:01:12,241][20381] Updated weights for policy 1, policy_version 1062891 (0.0006) [2023-12-27 13:01:12,276][20347] Updated weights for policy 0, policy_version 1063738 (0.0006) [2023-12-27 13:01:12,306][20381] Updated weights for policy 1, policy_version 1062901 (0.0008) [2023-12-27 13:01:12,341][20347] Updated weights for policy 0, policy_version 1063748 (0.0007) [2023-12-27 13:01:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 544489472. Throughput: 0: 10111.7, 1: 10248.8. Samples: 544482592. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:12,363][19170] Avg episode reward: [(0, '814.430'), (1, '716.590')] [2023-12-27 13:01:12,998][20347] Updated weights for policy 0, policy_version 1063758 (0.0006) [2023-12-27 13:01:13,017][20381] Updated weights for policy 1, policy_version 1062911 (0.0010) [2023-12-27 13:01:13,055][20347] Updated weights for policy 0, policy_version 1063768 (0.0005) [2023-12-27 13:01:13,075][20381] Updated weights for policy 1, policy_version 1062921 (0.0011) [2023-12-27 13:01:13,112][20347] Updated weights for policy 0, policy_version 1063778 (0.0006) [2023-12-27 13:01:13,129][20381] Updated weights for policy 1, policy_version 1062931 (0.0010) [2023-12-27 13:01:13,690][20381] Updated weights for policy 1, policy_version 1062941 (0.0008) [2023-12-27 13:01:13,750][20381] Updated weights for policy 1, policy_version 1062951 (0.0006) [2023-12-27 13:01:13,796][20347] Updated weights for policy 0, policy_version 1063788 (0.0008) [2023-12-27 13:01:13,809][20381] Updated weights for policy 1, policy_version 1062961 (0.0005) [2023-12-27 13:01:13,851][20347] Updated weights for policy 0, policy_version 1063798 (0.0010) [2023-12-27 13:01:13,906][20347] Updated weights for policy 0, policy_version 1063808 (0.0010) [2023-12-27 13:01:14,464][20381] Updated weights for policy 1, policy_version 1062971 (0.0006) [2023-12-27 13:01:14,517][20381] Updated weights for policy 1, policy_version 1062981 (0.0007) [2023-12-27 13:01:14,547][20347] Updated weights for policy 0, policy_version 1063818 (0.0010) [2023-12-27 13:01:14,564][20381] Updated weights for policy 1, policy_version 1062991 (0.0006) [2023-12-27 13:01:14,595][20347] Updated weights for policy 0, policy_version 1063828 (0.0010) [2023-12-27 13:01:14,643][20347] Updated weights for policy 0, policy_version 1063838 (0.0010) [2023-12-27 13:01:14,694][20347] Updated weights for policy 0, policy_version 1063848 (0.0010) [2023-12-27 13:01:15,266][20381] Updated weights for policy 1, policy_version 1063001 (0.0005) [2023-12-27 13:01:15,318][20381] Updated weights for policy 1, policy_version 1063011 (0.0005) [2023-12-27 13:01:15,373][20381] Updated weights for policy 1, policy_version 1063021 (0.0007) [2023-12-27 13:01:15,382][20347] Updated weights for policy 0, policy_version 1063858 (0.0007) [2023-12-27 13:01:15,422][20381] Updated weights for policy 1, policy_version 1063031 (0.0005) [2023-12-27 13:01:15,450][20347] Updated weights for policy 0, policy_version 1063868 (0.0008) [2023-12-27 13:01:15,507][20347] Updated weights for policy 0, policy_version 1063878 (0.0008) [2023-12-27 13:01:16,070][20381] Updated weights for policy 1, policy_version 1063041 (0.0005) [2023-12-27 13:01:16,133][20381] Updated weights for policy 1, policy_version 1063051 (0.0008) [2023-12-27 13:01:16,145][20347] Updated weights for policy 0, policy_version 1063888 (0.0006) [2023-12-27 13:01:16,192][20347] Updated weights for policy 0, policy_version 1063898 (0.0007) [2023-12-27 13:01:16,192][20381] Updated weights for policy 1, policy_version 1063061 (0.0011) [2023-12-27 13:01:16,245][20347] Updated weights for policy 0, policy_version 1063908 (0.0007) [2023-12-27 13:01:16,775][20381] Updated weights for policy 1, policy_version 1063071 (0.0010) [2023-12-27 13:01:16,830][20381] Updated weights for policy 1, policy_version 1063081 (0.0010) [2023-12-27 13:01:16,886][20381] Updated weights for policy 1, policy_version 1063091 (0.0011) [2023-12-27 13:01:16,904][20347] Updated weights for policy 0, policy_version 1063918 (0.0007) [2023-12-27 13:01:16,963][20347] Updated weights for policy 0, policy_version 1063928 (0.0011) [2023-12-27 13:01:17,030][20347] Updated weights for policy 0, policy_version 1063938 (0.0010) [2023-12-27 13:01:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 544595968. Throughput: 0: 10132.3, 1: 10320.3. Samples: 544606928. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:17,363][19170] Avg episode reward: [(0, '912.120'), (1, '716.310')] [2023-12-27 13:01:17,552][20381] Updated weights for policy 1, policy_version 1063101 (0.0009) [2023-12-27 13:01:17,612][20381] Updated weights for policy 1, policy_version 1063111 (0.0005) [2023-12-27 13:01:17,663][20381] Updated weights for policy 1, policy_version 1063121 (0.0005) [2023-12-27 13:01:17,716][20347] Updated weights for policy 0, policy_version 1063948 (0.0010) [2023-12-27 13:01:17,770][20347] Updated weights for policy 0, policy_version 1063958 (0.0007) [2023-12-27 13:01:17,831][20347] Updated weights for policy 0, policy_version 1063968 (0.0005) [2023-12-27 13:01:18,309][20381] Updated weights for policy 1, policy_version 1063131 (0.0010) [2023-12-27 13:01:18,371][20381] Updated weights for policy 1, policy_version 1063141 (0.0010) [2023-12-27 13:01:18,420][20381] Updated weights for policy 1, policy_version 1063151 (0.0010) [2023-12-27 13:01:18,459][20347] Updated weights for policy 0, policy_version 1063978 (0.0006) [2023-12-27 13:01:18,521][20347] Updated weights for policy 0, policy_version 1063988 (0.0006) [2023-12-27 13:01:18,579][20347] Updated weights for policy 0, policy_version 1063998 (0.0005) [2023-12-27 13:01:18,629][20347] Updated weights for policy 0, policy_version 1064008 (0.0006) [2023-12-27 13:01:19,120][20381] Updated weights for policy 1, policy_version 1063161 (0.0011) [2023-12-27 13:01:19,167][20381] Updated weights for policy 1, policy_version 1063171 (0.0010) [2023-12-27 13:01:19,226][20381] Updated weights for policy 1, policy_version 1063181 (0.0010) [2023-12-27 13:01:19,283][20347] Updated weights for policy 0, policy_version 1064018 (0.0006) [2023-12-27 13:01:19,293][20381] Updated weights for policy 1, policy_version 1063191 (0.0009) [2023-12-27 13:01:19,348][20347] Updated weights for policy 0, policy_version 1064028 (0.0007) [2023-12-27 13:01:19,413][20347] Updated weights for policy 0, policy_version 1064038 (0.0008) [2023-12-27 13:01:20,044][20381] Updated weights for policy 1, policy_version 1063201 (0.0007) [2023-12-27 13:01:20,086][20347] Updated weights for policy 0, policy_version 1064048 (0.0006) [2023-12-27 13:01:20,102][20381] Updated weights for policy 1, policy_version 1063211 (0.0010) [2023-12-27 13:01:20,146][20347] Updated weights for policy 0, policy_version 1064058 (0.0007) [2023-12-27 13:01:20,162][20381] Updated weights for policy 1, policy_version 1063221 (0.0011) [2023-12-27 13:01:20,205][20347] Updated weights for policy 0, policy_version 1064068 (0.0009) [2023-12-27 13:01:20,887][20381] Updated weights for policy 1, policy_version 1063231 (0.0011) [2023-12-27 13:01:20,947][20381] Updated weights for policy 1, policy_version 1063241 (0.0011) [2023-12-27 13:01:20,961][20347] Updated weights for policy 0, policy_version 1064078 (0.0007) [2023-12-27 13:01:21,011][20381] Updated weights for policy 1, policy_version 1063251 (0.0009) [2023-12-27 13:01:21,026][20347] Updated weights for policy 0, policy_version 1064088 (0.0008) [2023-12-27 13:01:21,091][20347] Updated weights for policy 0, policy_version 1064098 (0.0008) [2023-12-27 13:01:21,727][20381] Updated weights for policy 1, policy_version 1063261 (0.0008) [2023-12-27 13:01:21,792][20381] Updated weights for policy 1, policy_version 1063271 (0.0007) [2023-12-27 13:01:21,851][20381] Updated weights for policy 1, policy_version 1063281 (0.0008) [2023-12-27 13:01:21,862][20347] Updated weights for policy 0, policy_version 1064108 (0.0006) [2023-12-27 13:01:21,918][20347] Updated weights for policy 0, policy_version 1064118 (0.0006) [2023-12-27 13:01:21,974][20347] Updated weights for policy 0, policy_version 1064128 (0.0008) [2023-12-27 13:01:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 544694272. Throughput: 0: 10115.2, 1: 10302.5. Samples: 544668036. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:22,363][19170] Avg episode reward: [(0, '941.550'), (1, '706.720')] [2023-12-27 13:01:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001063288_272228352.pth... [2023-12-27 13:01:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001064136_272465920.pth... [2023-12-27 13:01:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001062952_272162816.pth [2023-12-27 13:01:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001062072_271917056.pth [2023-12-27 13:01:22,599][20381] Updated weights for policy 1, policy_version 1063291 (0.0006) [2023-12-27 13:01:22,660][20381] Updated weights for policy 1, policy_version 1063301 (0.0006) [2023-12-27 13:01:22,660][20347] Updated weights for policy 0, policy_version 1064138 (0.0009) [2023-12-27 13:01:22,718][20381] Updated weights for policy 1, policy_version 1063311 (0.0007) [2023-12-27 13:01:22,723][20347] Updated weights for policy 0, policy_version 1064148 (0.0009) [2023-12-27 13:01:22,785][20347] Updated weights for policy 0, policy_version 1064158 (0.0006) [2023-12-27 13:01:22,850][20347] Updated weights for policy 0, policy_version 1064168 (0.0007) [2023-12-27 13:01:23,390][20381] Updated weights for policy 1, policy_version 1063321 (0.0007) [2023-12-27 13:01:23,446][20381] Updated weights for policy 1, policy_version 1063331 (0.0007) [2023-12-27 13:01:23,493][20381] Updated weights for policy 1, policy_version 1063341 (0.0006) [2023-12-27 13:01:23,520][20347] Updated weights for policy 0, policy_version 1064178 (0.0006) [2023-12-27 13:01:23,545][20381] Updated weights for policy 1, policy_version 1063351 (0.0006) [2023-12-27 13:01:23,577][20347] Updated weights for policy 0, policy_version 1064188 (0.0007) [2023-12-27 13:01:23,632][20347] Updated weights for policy 0, policy_version 1064198 (0.0008) [2023-12-27 13:01:24,189][20381] Updated weights for policy 1, policy_version 1063361 (0.0007) [2023-12-27 13:01:24,270][20381] Updated weights for policy 1, policy_version 1063371 (0.0008) [2023-12-27 13:01:24,328][20381] Updated weights for policy 1, policy_version 1063381 (0.0009) [2023-12-27 13:01:24,338][20347] Updated weights for policy 0, policy_version 1064208 (0.0006) [2023-12-27 13:01:24,386][20347] Updated weights for policy 0, policy_version 1064218 (0.0008) [2023-12-27 13:01:24,440][20347] Updated weights for policy 0, policy_version 1064228 (0.0007) [2023-12-27 13:01:24,932][20381] Updated weights for policy 1, policy_version 1063391 (0.0008) [2023-12-27 13:01:24,993][20381] Updated weights for policy 1, policy_version 1063401 (0.0009) [2023-12-27 13:01:25,048][20381] Updated weights for policy 1, policy_version 1063411 (0.0008) [2023-12-27 13:01:25,142][20347] Updated weights for policy 0, policy_version 1064238 (0.0007) [2023-12-27 13:01:25,174][20291] KL-divergence is very high: 256.8987 [2023-12-27 13:01:25,192][20347] Updated weights for policy 0, policy_version 1064248 (0.0007) [2023-12-27 13:01:25,239][20347] Updated weights for policy 0, policy_version 1064258 (0.0007) [2023-12-27 13:01:25,725][20381] Updated weights for policy 1, policy_version 1063421 (0.0007) [2023-12-27 13:01:25,779][20381] Updated weights for policy 1, policy_version 1063431 (0.0006) [2023-12-27 13:01:25,836][20381] Updated weights for policy 1, policy_version 1063441 (0.0006) [2023-12-27 13:01:25,957][20347] Updated weights for policy 0, policy_version 1064268 (0.0006) [2023-12-27 13:01:26,013][20347] Updated weights for policy 0, policy_version 1064278 (0.0007) [2023-12-27 13:01:26,066][20347] Updated weights for policy 0, policy_version 1064288 (0.0007) [2023-12-27 13:01:26,386][20381] Updated weights for policy 1, policy_version 1063451 (0.0006) [2023-12-27 13:01:26,453][20381] Updated weights for policy 1, policy_version 1063461 (0.0005) [2023-12-27 13:01:26,509][20381] Updated weights for policy 1, policy_version 1063471 (0.0005) [2023-12-27 13:01:26,750][20347] Updated weights for policy 0, policy_version 1064298 (0.0008) [2023-12-27 13:01:26,817][20347] Updated weights for policy 0, policy_version 1064308 (0.0010) [2023-12-27 13:01:26,879][20347] Updated weights for policy 0, policy_version 1064318 (0.0008) [2023-12-27 13:01:26,940][20347] Updated weights for policy 0, policy_version 1064328 (0.0008) [2023-12-27 13:01:27,001][20381] Updated weights for policy 1, policy_version 1063481 (0.0005) [2023-12-27 13:01:27,057][20381] Updated weights for policy 1, policy_version 1063491 (0.0005) [2023-12-27 13:01:27,115][20381] Updated weights for policy 1, policy_version 1063501 (0.0005) [2023-12-27 13:01:27,175][20381] Updated weights for policy 1, policy_version 1063511 (0.0005) [2023-12-27 13:01:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 544800768. Throughput: 0: 10083.1, 1: 10322.8. Samples: 544788156. Policy #0 lag: (min: 20.0, avg: 31.3, max: 32.0) [2023-12-27 13:01:27,363][19170] Avg episode reward: [(0, '912.700'), (1, '775.320')] [2023-12-27 13:01:27,533][20347] Updated weights for policy 0, policy_version 1064338 (0.0010) [2023-12-27 13:01:27,581][20347] Updated weights for policy 0, policy_version 1064348 (0.0010) [2023-12-27 13:01:27,633][20347] Updated weights for policy 0, policy_version 1064358 (0.0010) [2023-12-27 13:01:27,743][20381] Updated weights for policy 1, policy_version 1063521 (0.0006) [2023-12-27 13:01:27,790][20381] Updated weights for policy 1, policy_version 1063531 (0.0010) [2023-12-27 13:01:27,835][20381] Updated weights for policy 1, policy_version 1063541 (0.0010) [2023-12-27 13:01:28,297][20347] Updated weights for policy 0, policy_version 1064368 (0.0010) [2023-12-27 13:01:28,356][20347] Updated weights for policy 0, policy_version 1064379 (0.0010) [2023-12-27 13:01:28,407][20347] Updated weights for policy 0, policy_version 1064390 (0.0007) [2023-12-27 13:01:28,471][20381] Updated weights for policy 1, policy_version 1063551 (0.0010) [2023-12-27 13:01:28,532][20381] Updated weights for policy 1, policy_version 1063561 (0.0010) [2023-12-27 13:01:28,590][20381] Updated weights for policy 1, policy_version 1063571 (0.0010) [2023-12-27 13:01:29,081][20347] Updated weights for policy 0, policy_version 1064400 (0.0009) [2023-12-27 13:01:29,136][20347] Updated weights for policy 0, policy_version 1064410 (0.0009) [2023-12-27 13:01:29,193][20347] Updated weights for policy 0, policy_version 1064420 (0.0006) [2023-12-27 13:01:29,246][20381] Updated weights for policy 1, policy_version 1063581 (0.0009) [2023-12-27 13:01:29,299][20381] Updated weights for policy 1, policy_version 1063591 (0.0007) [2023-12-27 13:01:29,352][20381] Updated weights for policy 1, policy_version 1063601 (0.0008) [2023-12-27 13:01:29,916][20347] Updated weights for policy 0, policy_version 1064430 (0.0009) [2023-12-27 13:01:29,976][20347] Updated weights for policy 0, policy_version 1064440 (0.0008) [2023-12-27 13:01:30,040][20347] Updated weights for policy 0, policy_version 1064450 (0.0009) [2023-12-27 13:01:30,061][20381] Updated weights for policy 1, policy_version 1063611 (0.0007) [2023-12-27 13:01:30,119][20381] Updated weights for policy 1, policy_version 1063621 (0.0006) [2023-12-27 13:01:30,181][20381] Updated weights for policy 1, policy_version 1063631 (0.0005) [2023-12-27 13:01:30,698][20347] Updated weights for policy 0, policy_version 1064460 (0.0008) [2023-12-27 13:01:30,743][20347] Updated weights for policy 0, policy_version 1064470 (0.0010) [2023-12-27 13:01:30,789][20381] Updated weights for policy 1, policy_version 1063641 (0.0006) [2023-12-27 13:01:30,793][20347] Updated weights for policy 0, policy_version 1064480 (0.0007) [2023-12-27 13:01:30,842][20381] Updated weights for policy 1, policy_version 1063651 (0.0008) [2023-12-27 13:01:30,901][20381] Updated weights for policy 1, policy_version 1063661 (0.0008) [2023-12-27 13:01:30,959][20381] Updated weights for policy 1, policy_version 1063671 (0.0009) [2023-12-27 13:01:31,477][20347] Updated weights for policy 0, policy_version 1064490 (0.0006) [2023-12-27 13:01:31,535][20347] Updated weights for policy 0, policy_version 1064500 (0.0010) [2023-12-27 13:01:31,602][20347] Updated weights for policy 0, policy_version 1064510 (0.0010) [2023-12-27 13:01:31,669][20347] Updated weights for policy 0, policy_version 1064520 (0.0008) [2023-12-27 13:01:31,723][20381] Updated weights for policy 1, policy_version 1063681 (0.0010) [2023-12-27 13:01:31,785][20381] Updated weights for policy 1, policy_version 1063691 (0.0009) [2023-12-27 13:01:31,836][20381] Updated weights for policy 1, policy_version 1063701 (0.0005) [2023-12-27 13:01:32,357][20347] Updated weights for policy 0, policy_version 1064530 (0.0009) [2023-12-27 13:01:32,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 544899072. Throughput: 0: 10128.1, 1: 10414.0. Samples: 544915440. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:32,363][19170] Avg episode reward: [(0, '893.170'), (1, '765.350')] [2023-12-27 13:01:32,417][20347] Updated weights for policy 0, policy_version 1064540 (0.0010) [2023-12-27 13:01:32,439][20381] Updated weights for policy 1, policy_version 1063711 (0.0005) [2023-12-27 13:01:32,470][20347] Updated weights for policy 0, policy_version 1064550 (0.0010) [2023-12-27 13:01:32,499][20381] Updated weights for policy 1, policy_version 1063721 (0.0007) [2023-12-27 13:01:32,554][20381] Updated weights for policy 1, policy_version 1063731 (0.0010) [2023-12-27 13:01:33,163][20347] Updated weights for policy 0, policy_version 1064560 (0.0009) [2023-12-27 13:01:33,164][20381] Updated weights for policy 1, policy_version 1063741 (0.0008) [2023-12-27 13:01:33,214][20347] Updated weights for policy 0, policy_version 1064570 (0.0007) [2023-12-27 13:01:33,215][20381] Updated weights for policy 1, policy_version 1063751 (0.0006) [2023-12-27 13:01:33,265][20381] Updated weights for policy 1, policy_version 1063761 (0.0005) [2023-12-27 13:01:33,268][20347] Updated weights for policy 0, policy_version 1064580 (0.0007) [2023-12-27 13:01:33,831][20347] Updated weights for policy 0, policy_version 1064590 (0.0005) [2023-12-27 13:01:33,879][20347] Updated weights for policy 0, policy_version 1064600 (0.0005) [2023-12-27 13:01:33,922][20347] Updated weights for policy 0, policy_version 1064610 (0.0005) [2023-12-27 13:01:33,950][20381] Updated weights for policy 1, policy_version 1063771 (0.0006) [2023-12-27 13:01:34,003][20381] Updated weights for policy 1, policy_version 1063781 (0.0007) [2023-12-27 13:01:34,049][20381] Updated weights for policy 1, policy_version 1063791 (0.0008) [2023-12-27 13:01:34,501][20347] Updated weights for policy 0, policy_version 1064620 (0.0006) [2023-12-27 13:01:34,566][20347] Updated weights for policy 0, policy_version 1064630 (0.0007) [2023-12-27 13:01:34,627][20347] Updated weights for policy 0, policy_version 1064640 (0.0008) [2023-12-27 13:01:34,804][20381] Updated weights for policy 1, policy_version 1063801 (0.0008) [2023-12-27 13:01:34,862][20381] Updated weights for policy 1, policy_version 1063811 (0.0007) [2023-12-27 13:01:34,921][20381] Updated weights for policy 1, policy_version 1063821 (0.0007) [2023-12-27 13:01:34,983][20381] Updated weights for policy 1, policy_version 1063831 (0.0007) [2023-12-27 13:01:35,334][20347] Updated weights for policy 0, policy_version 1064650 (0.0007) [2023-12-27 13:01:35,387][20347] Updated weights for policy 0, policy_version 1064660 (0.0008) [2023-12-27 13:01:35,441][20347] Updated weights for policy 0, policy_version 1064670 (0.0008) [2023-12-27 13:01:35,489][20347] Updated weights for policy 0, policy_version 1064680 (0.0008) [2023-12-27 13:01:35,594][20381] Updated weights for policy 1, policy_version 1063841 (0.0007) [2023-12-27 13:01:35,656][20381] Updated weights for policy 1, policy_version 1063851 (0.0010) [2023-12-27 13:01:35,710][20381] Updated weights for policy 1, policy_version 1063861 (0.0010) [2023-12-27 13:01:36,156][20347] Updated weights for policy 0, policy_version 1064690 (0.0007) [2023-12-27 13:01:36,218][20347] Updated weights for policy 0, policy_version 1064700 (0.0006) [2023-12-27 13:01:36,276][20347] Updated weights for policy 0, policy_version 1064710 (0.0006) [2023-12-27 13:01:36,359][20381] Updated weights for policy 1, policy_version 1063871 (0.0009) [2023-12-27 13:01:36,421][20381] Updated weights for policy 1, policy_version 1063881 (0.0008) [2023-12-27 13:01:36,481][20381] Updated weights for policy 1, policy_version 1063891 (0.0008) [2023-12-27 13:01:36,946][20347] Updated weights for policy 0, policy_version 1064720 (0.0008) [2023-12-27 13:01:36,998][20347] Updated weights for policy 0, policy_version 1064730 (0.0008) [2023-12-27 13:01:37,056][20347] Updated weights for policy 0, policy_version 1064740 (0.0008) [2023-12-27 13:01:37,180][20381] Updated weights for policy 1, policy_version 1063901 (0.0007) [2023-12-27 13:01:37,230][20381] Updated weights for policy 1, policy_version 1063911 (0.0006) [2023-12-27 13:01:37,281][20381] Updated weights for policy 1, policy_version 1063921 (0.0009) [2023-12-27 13:01:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 545013760. Throughput: 0: 10215.1, 1: 10421.6. Samples: 544978996. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:37,363][19170] Avg episode reward: [(0, '951.230'), (1, '785.200')] [2023-12-27 13:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001063928_272392192.pth... [2023-12-27 13:01:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001064744_272621568.pth... [2023-12-27 13:01:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001062712_272080896.pth [2023-12-27 13:01:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001063560_272318464.pth [2023-12-27 13:01:37,655][20347] Updated weights for policy 0, policy_version 1064750 (0.0008) [2023-12-27 13:01:37,703][20347] Updated weights for policy 0, policy_version 1064760 (0.0008) [2023-12-27 13:01:37,759][20347] Updated weights for policy 0, policy_version 1064770 (0.0008) [2023-12-27 13:01:38,007][20381] Updated weights for policy 1, policy_version 1063931 (0.0008) [2023-12-27 13:01:38,053][20381] Updated weights for policy 1, policy_version 1063941 (0.0008) [2023-12-27 13:01:38,111][20381] Updated weights for policy 1, policy_version 1063951 (0.0007) [2023-12-27 13:01:38,442][20347] Updated weights for policy 0, policy_version 1064780 (0.0007) [2023-12-27 13:01:38,505][20347] Updated weights for policy 0, policy_version 1064790 (0.0006) [2023-12-27 13:01:38,564][20347] Updated weights for policy 0, policy_version 1064800 (0.0006) [2023-12-27 13:01:38,771][20381] Updated weights for policy 1, policy_version 1063961 (0.0008) [2023-12-27 13:01:38,829][20381] Updated weights for policy 1, policy_version 1063971 (0.0008) [2023-12-27 13:01:38,885][20381] Updated weights for policy 1, policy_version 1063981 (0.0007) [2023-12-27 13:01:38,940][20381] Updated weights for policy 1, policy_version 1063991 (0.0008) [2023-12-27 13:01:39,184][20347] Updated weights for policy 0, policy_version 1064810 (0.0006) [2023-12-27 13:01:39,246][20347] Updated weights for policy 0, policy_version 1064820 (0.0008) [2023-12-27 13:01:39,304][20347] Updated weights for policy 0, policy_version 1064830 (0.0007) [2023-12-27 13:01:39,367][20347] Updated weights for policy 0, policy_version 1064840 (0.0007) [2023-12-27 13:01:39,684][20381] Updated weights for policy 1, policy_version 1064001 (0.0006) [2023-12-27 13:01:39,746][20381] Updated weights for policy 1, policy_version 1064011 (0.0007) [2023-12-27 13:01:39,814][20381] Updated weights for policy 1, policy_version 1064021 (0.0007) [2023-12-27 13:01:40,148][20347] Updated weights for policy 0, policy_version 1064850 (0.0008) [2023-12-27 13:01:40,203][20347] Updated weights for policy 0, policy_version 1064860 (0.0008) [2023-12-27 13:01:40,268][20347] Updated weights for policy 0, policy_version 1064870 (0.0008) [2023-12-27 13:01:40,492][20381] Updated weights for policy 1, policy_version 1064031 (0.0010) [2023-12-27 13:01:40,548][20381] Updated weights for policy 1, policy_version 1064041 (0.0010) [2023-12-27 13:01:40,597][20381] Updated weights for policy 1, policy_version 1064051 (0.0011) [2023-12-27 13:01:40,871][20347] Updated weights for policy 0, policy_version 1064880 (0.0010) [2023-12-27 13:01:40,930][20347] Updated weights for policy 0, policy_version 1064890 (0.0009) [2023-12-27 13:01:40,985][20347] Updated weights for policy 0, policy_version 1064900 (0.0005) [2023-12-27 13:01:41,239][20381] Updated weights for policy 1, policy_version 1064061 (0.0010) [2023-12-27 13:01:41,303][20381] Updated weights for policy 1, policy_version 1064071 (0.0009) [2023-12-27 13:01:41,363][20381] Updated weights for policy 1, policy_version 1064081 (0.0008) [2023-12-27 13:01:41,652][20347] Updated weights for policy 0, policy_version 1064910 (0.0007) [2023-12-27 13:01:41,716][20347] Updated weights for policy 0, policy_version 1064920 (0.0006) [2023-12-27 13:01:41,783][20347] Updated weights for policy 0, policy_version 1064930 (0.0008) [2023-12-27 13:01:42,081][20381] Updated weights for policy 1, policy_version 1064091 (0.0008) [2023-12-27 13:01:42,140][20381] Updated weights for policy 1, policy_version 1064101 (0.0007) [2023-12-27 13:01:42,200][20381] Updated weights for policy 1, policy_version 1064111 (0.0008) [2023-12-27 13:01:42,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 545112064. Throughput: 0: 10232.0, 1: 10373.7. Samples: 545102876. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:42,363][19170] Avg episode reward: [(0, '970.680'), (1, '755.900')] [2023-12-27 13:01:42,424][20347] Updated weights for policy 0, policy_version 1064940 (0.0007) [2023-12-27 13:01:42,481][20347] Updated weights for policy 0, policy_version 1064950 (0.0006) [2023-12-27 13:01:42,547][20347] Updated weights for policy 0, policy_version 1064960 (0.0006) [2023-12-27 13:01:42,929][20381] Updated weights for policy 1, policy_version 1064121 (0.0008) [2023-12-27 13:01:42,974][20381] Updated weights for policy 1, policy_version 1064131 (0.0007) [2023-12-27 13:01:43,019][20381] Updated weights for policy 1, policy_version 1064141 (0.0007) [2023-12-27 13:01:43,064][20381] Updated weights for policy 1, policy_version 1064151 (0.0007) [2023-12-27 13:01:43,182][20347] Updated weights for policy 0, policy_version 1064970 (0.0007) [2023-12-27 13:01:43,233][20347] Updated weights for policy 0, policy_version 1064980 (0.0010) [2023-12-27 13:01:43,281][20347] Updated weights for policy 0, policy_version 1064990 (0.0010) [2023-12-27 13:01:43,339][20347] Updated weights for policy 0, policy_version 1065000 (0.0010) [2023-12-27 13:01:43,789][20381] Updated weights for policy 1, policy_version 1064161 (0.0007) [2023-12-27 13:01:43,836][20381] Updated weights for policy 1, policy_version 1064171 (0.0007) [2023-12-27 13:01:43,884][20381] Updated weights for policy 1, policy_version 1064181 (0.0007) [2023-12-27 13:01:44,027][20347] Updated weights for policy 0, policy_version 1065010 (0.0010) [2023-12-27 13:01:44,073][20347] Updated weights for policy 0, policy_version 1065020 (0.0010) [2023-12-27 13:01:44,125][20347] Updated weights for policy 0, policy_version 1065030 (0.0010) [2023-12-27 13:01:44,585][20381] Updated weights for policy 1, policy_version 1064191 (0.0010) [2023-12-27 13:01:44,638][20381] Updated weights for policy 1, policy_version 1064201 (0.0010) [2023-12-27 13:01:44,683][20381] Updated weights for policy 1, policy_version 1064211 (0.0010) [2023-12-27 13:01:44,821][20347] Updated weights for policy 0, policy_version 1065040 (0.0008) [2023-12-27 13:01:44,876][20347] Updated weights for policy 0, policy_version 1065050 (0.0008) [2023-12-27 13:01:44,933][20347] Updated weights for policy 0, policy_version 1065060 (0.0007) [2023-12-27 13:01:45,413][20381] Updated weights for policy 1, policy_version 1064221 (0.0009) [2023-12-27 13:01:45,476][20381] Updated weights for policy 1, policy_version 1064231 (0.0008) [2023-12-27 13:01:45,531][20381] Updated weights for policy 1, policy_version 1064241 (0.0007) [2023-12-27 13:01:45,629][20347] Updated weights for policy 0, policy_version 1065070 (0.0009) [2023-12-27 13:01:45,695][20347] Updated weights for policy 0, policy_version 1065080 (0.0010) [2023-12-27 13:01:45,751][20347] Updated weights for policy 0, policy_version 1065090 (0.0007) [2023-12-27 13:01:46,176][20381] Updated weights for policy 1, policy_version 1064251 (0.0009) [2023-12-27 13:01:46,224][20381] Updated weights for policy 1, policy_version 1064261 (0.0010) [2023-12-27 13:01:46,278][20381] Updated weights for policy 1, policy_version 1064271 (0.0010) [2023-12-27 13:01:46,381][20347] Updated weights for policy 0, policy_version 1065100 (0.0006) [2023-12-27 13:01:46,437][20347] Updated weights for policy 0, policy_version 1065110 (0.0005) [2023-12-27 13:01:46,513][20347] Updated weights for policy 0, policy_version 1065120 (0.0010) [2023-12-27 13:01:46,881][20381] Updated weights for policy 1, policy_version 1064281 (0.0009) [2023-12-27 13:01:46,932][20381] Updated weights for policy 1, policy_version 1064291 (0.0007) [2023-12-27 13:01:46,984][20381] Updated weights for policy 1, policy_version 1064301 (0.0008) [2023-12-27 13:01:47,029][20381] Updated weights for policy 1, policy_version 1064311 (0.0008) [2023-12-27 13:01:47,089][20347] Updated weights for policy 0, policy_version 1065130 (0.0009) [2023-12-27 13:01:47,144][20347] Updated weights for policy 0, policy_version 1065140 (0.0009) [2023-12-27 13:01:47,191][20347] Updated weights for policy 0, policy_version 1065150 (0.0010) [2023-12-27 13:01:47,239][20347] Updated weights for policy 0, policy_version 1065160 (0.0007) [2023-12-27 13:01:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 545218560. Throughput: 0: 10252.7, 1: 10380.6. Samples: 545226340. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:47,363][19170] Avg episode reward: [(0, '892.680'), (1, '697.340')] [2023-12-27 13:01:47,656][20381] Updated weights for policy 1, policy_version 1064321 (0.0010) [2023-12-27 13:01:47,714][20381] Updated weights for policy 1, policy_version 1064331 (0.0007) [2023-12-27 13:01:47,769][20381] Updated weights for policy 1, policy_version 1064341 (0.0005) [2023-12-27 13:01:47,848][20347] Updated weights for policy 0, policy_version 1065170 (0.0010) [2023-12-27 13:01:47,912][20347] Updated weights for policy 0, policy_version 1065180 (0.0010) [2023-12-27 13:01:47,961][20347] Updated weights for policy 0, policy_version 1065190 (0.0010) [2023-12-27 13:01:48,400][20381] Updated weights for policy 1, policy_version 1064351 (0.0006) [2023-12-27 13:01:48,460][20381] Updated weights for policy 1, policy_version 1064361 (0.0010) [2023-12-27 13:01:48,526][20381] Updated weights for policy 1, policy_version 1064371 (0.0010) [2023-12-27 13:01:48,625][20347] Updated weights for policy 0, policy_version 1065200 (0.0007) [2023-12-27 13:01:48,683][20347] Updated weights for policy 0, policy_version 1065210 (0.0007) [2023-12-27 13:01:48,744][20347] Updated weights for policy 0, policy_version 1065220 (0.0008) [2023-12-27 13:01:49,130][20381] Updated weights for policy 1, policy_version 1064381 (0.0009) [2023-12-27 13:01:49,172][20381] Updated weights for policy 1, policy_version 1064391 (0.0007) [2023-12-27 13:01:49,229][20381] Updated weights for policy 1, policy_version 1064401 (0.0007) [2023-12-27 13:01:49,405][20347] Updated weights for policy 0, policy_version 1065230 (0.0009) [2023-12-27 13:01:49,461][20347] Updated weights for policy 0, policy_version 1065240 (0.0010) [2023-12-27 13:01:49,514][20347] Updated weights for policy 0, policy_version 1065250 (0.0010) [2023-12-27 13:01:50,022][20381] Updated weights for policy 1, policy_version 1064411 (0.0008) [2023-12-27 13:01:50,081][20381] Updated weights for policy 1, policy_version 1064421 (0.0008) [2023-12-27 13:01:50,133][20381] Updated weights for policy 1, policy_version 1064431 (0.0007) [2023-12-27 13:01:50,191][20291] KL-divergence is very high: 101.1136 [2023-12-27 13:01:50,221][20347] Updated weights for policy 0, policy_version 1065260 (0.0010) [2023-12-27 13:01:50,239][20291] KL-divergence is very high: 173.7358 [2023-12-27 13:01:50,273][20347] Updated weights for policy 0, policy_version 1065270 (0.0010) [2023-12-27 13:01:50,324][20291] KL-divergence is very high: 1515.5792 [2023-12-27 13:01:50,326][20347] Updated weights for policy 0, policy_version 1065280 (0.0010) [2023-12-27 13:01:50,330][20291] KL-divergence is very high: 962.0737 [2023-12-27 13:01:50,807][20381] Updated weights for policy 1, policy_version 1064441 (0.0006) [2023-12-27 13:01:50,865][20381] Updated weights for policy 1, policy_version 1064451 (0.0006) [2023-12-27 13:01:50,885][20292] KL-divergence is very high: 130.6543 [2023-12-27 13:01:50,898][20292] KL-divergence is very high: 150.1012 [2023-12-27 13:01:50,929][20381] Updated weights for policy 1, policy_version 1064461 (0.0007) [2023-12-27 13:01:50,990][20381] Updated weights for policy 1, policy_version 1064471 (0.0008) [2023-12-27 13:01:51,037][20347] Updated weights for policy 0, policy_version 1065290 (0.0010) [2023-12-27 13:01:51,099][20347] Updated weights for policy 0, policy_version 1065300 (0.0010) [2023-12-27 13:01:51,163][20347] Updated weights for policy 0, policy_version 1065310 (0.0010) [2023-12-27 13:01:51,219][20347] Updated weights for policy 0, policy_version 1065320 (0.0010) [2023-12-27 13:01:51,754][20381] Updated weights for policy 1, policy_version 1064481 (0.0009) [2023-12-27 13:01:51,820][20381] Updated weights for policy 1, policy_version 1064491 (0.0008) [2023-12-27 13:01:51,875][20381] Updated weights for policy 1, policy_version 1064501 (0.0006) [2023-12-27 13:01:51,980][20347] Updated weights for policy 0, policy_version 1065330 (0.0007) [2023-12-27 13:01:52,034][20347] Updated weights for policy 0, policy_version 1065340 (0.0008) [2023-12-27 13:01:52,086][20291] KL-divergence is very high: 332.4273 [2023-12-27 13:01:52,090][20347] Updated weights for policy 0, policy_version 1065350 (0.0005) [2023-12-27 13:01:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 545316864. Throughput: 0: 10263.1, 1: 10395.1. Samples: 545288392. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:52,365][19170] Avg episode reward: [(0, '892.770'), (1, '775.550')] [2023-12-27 13:01:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001064504_272539648.pth... [2023-12-27 13:01:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001065352_272777216.pth... [2023-12-27 13:01:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001063288_272228352.pth [2023-12-27 13:01:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001064136_272465920.pth [2023-12-27 13:01:52,516][20381] Updated weights for policy 1, policy_version 1064511 (0.0006) [2023-12-27 13:01:52,539][20292] KL-divergence is very high: 100.5845 [2023-12-27 13:01:52,571][20381] Updated weights for policy 1, policy_version 1064521 (0.0008) [2023-12-27 13:01:52,626][20381] Updated weights for policy 1, policy_version 1064531 (0.0008) [2023-12-27 13:01:52,720][20347] Updated weights for policy 0, policy_version 1065360 (0.0007) [2023-12-27 13:01:52,779][20347] Updated weights for policy 0, policy_version 1065370 (0.0007) [2023-12-27 13:01:52,837][20347] Updated weights for policy 0, policy_version 1065380 (0.0008) [2023-12-27 13:01:53,214][20381] Updated weights for policy 1, policy_version 1064541 (0.0007) [2023-12-27 13:01:53,269][20381] Updated weights for policy 1, policy_version 1064551 (0.0006) [2023-12-27 13:01:53,322][20381] Updated weights for policy 1, policy_version 1064561 (0.0005) [2023-12-27 13:01:53,489][20347] Updated weights for policy 0, policy_version 1065390 (0.0009) [2023-12-27 13:01:53,548][20347] Updated weights for policy 0, policy_version 1065400 (0.0010) [2023-12-27 13:01:53,600][20347] Updated weights for policy 0, policy_version 1065410 (0.0010) [2023-12-27 13:01:53,920][20381] Updated weights for policy 1, policy_version 1064571 (0.0007) [2023-12-27 13:01:53,969][20381] Updated weights for policy 1, policy_version 1064581 (0.0010) [2023-12-27 13:01:54,020][20381] Updated weights for policy 1, policy_version 1064591 (0.0010) [2023-12-27 13:01:54,275][20347] Updated weights for policy 0, policy_version 1065420 (0.0008) [2023-12-27 13:01:54,336][20347] Updated weights for policy 0, policy_version 1065430 (0.0005) [2023-12-27 13:01:54,400][20347] Updated weights for policy 0, policy_version 1065440 (0.0010) [2023-12-27 13:01:54,661][20381] Updated weights for policy 1, policy_version 1064601 (0.0010) [2023-12-27 13:01:54,722][20381] Updated weights for policy 1, policy_version 1064611 (0.0010) [2023-12-27 13:01:54,782][20381] Updated weights for policy 1, policy_version 1064621 (0.0010) [2023-12-27 13:01:54,837][20381] Updated weights for policy 1, policy_version 1064631 (0.0009) [2023-12-27 13:01:54,992][20347] Updated weights for policy 0, policy_version 1065450 (0.0009) [2023-12-27 13:01:55,046][20347] Updated weights for policy 0, policy_version 1065460 (0.0005) [2023-12-27 13:01:55,103][20347] Updated weights for policy 0, policy_version 1065470 (0.0009) [2023-12-27 13:01:55,161][20347] Updated weights for policy 0, policy_version 1065480 (0.0010) [2023-12-27 13:01:55,473][20381] Updated weights for policy 1, policy_version 1064641 (0.0007) [2023-12-27 13:01:55,526][20381] Updated weights for policy 1, policy_version 1064651 (0.0006) [2023-12-27 13:01:55,573][20381] Updated weights for policy 1, policy_version 1064661 (0.0005) [2023-12-27 13:01:55,778][20347] Updated weights for policy 0, policy_version 1065490 (0.0007) [2023-12-27 13:01:55,836][20347] Updated weights for policy 0, policy_version 1065500 (0.0007) [2023-12-27 13:01:55,895][20347] Updated weights for policy 0, policy_version 1065510 (0.0008) [2023-12-27 13:01:56,264][20381] Updated weights for policy 1, policy_version 1064671 (0.0007) [2023-12-27 13:01:56,322][20381] Updated weights for policy 1, policy_version 1064681 (0.0008) [2023-12-27 13:01:56,373][20381] Updated weights for policy 1, policy_version 1064691 (0.0008) [2023-12-27 13:01:56,478][20347] Updated weights for policy 0, policy_version 1065520 (0.0006) [2023-12-27 13:01:56,537][20347] Updated weights for policy 0, policy_version 1065530 (0.0005) [2023-12-27 13:01:56,588][20347] Updated weights for policy 0, policy_version 1065540 (0.0005) [2023-12-27 13:01:57,070][20381] Updated weights for policy 1, policy_version 1064701 (0.0009) [2023-12-27 13:01:57,124][20381] Updated weights for policy 1, policy_version 1064711 (0.0010) [2023-12-27 13:01:57,157][20347] Updated weights for policy 0, policy_version 1065550 (0.0006) [2023-12-27 13:01:57,175][20381] Updated weights for policy 1, policy_version 1064721 (0.0010) [2023-12-27 13:01:57,212][20347] Updated weights for policy 0, policy_version 1065560 (0.0007) [2023-12-27 13:01:57,259][20347] Updated weights for policy 0, policy_version 1065570 (0.0007) [2023-12-27 13:01:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 545431552. Throughput: 0: 10304.6, 1: 10394.0. Samples: 545414028. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:01:57,363][19170] Avg episode reward: [(0, '941.450'), (1, '794.760')] [2023-12-27 13:01:57,776][20381] Updated weights for policy 1, policy_version 1064731 (0.0010) [2023-12-27 13:01:57,829][20381] Updated weights for policy 1, policy_version 1064741 (0.0010) [2023-12-27 13:01:57,873][20347] Updated weights for policy 0, policy_version 1065580 (0.0007) [2023-12-27 13:01:57,883][20381] Updated weights for policy 1, policy_version 1064751 (0.0010) [2023-12-27 13:01:57,924][20347] Updated weights for policy 0, policy_version 1065590 (0.0005) [2023-12-27 13:01:57,982][20347] Updated weights for policy 0, policy_version 1065600 (0.0005) [2023-12-27 13:01:58,542][20381] Updated weights for policy 1, policy_version 1064761 (0.0007) [2023-12-27 13:01:58,600][20381] Updated weights for policy 1, policy_version 1064771 (0.0009) [2023-12-27 13:01:58,662][20381] Updated weights for policy 1, policy_version 1064781 (0.0008) [2023-12-27 13:01:58,669][20347] Updated weights for policy 0, policy_version 1065610 (0.0006) [2023-12-27 13:01:58,724][20381] Updated weights for policy 1, policy_version 1064791 (0.0009) [2023-12-27 13:01:58,730][20347] Updated weights for policy 0, policy_version 1065620 (0.0008) [2023-12-27 13:01:58,808][20347] Updated weights for policy 0, policy_version 1065630 (0.0008) [2023-12-27 13:01:58,866][20347] Updated weights for policy 0, policy_version 1065640 (0.0007) [2023-12-27 13:01:59,452][20381] Updated weights for policy 1, policy_version 1064801 (0.0007) [2023-12-27 13:01:59,497][20381] Updated weights for policy 1, policy_version 1064811 (0.0007) [2023-12-27 13:01:59,551][20381] Updated weights for policy 1, policy_version 1064821 (0.0010) [2023-12-27 13:01:59,566][20347] Updated weights for policy 0, policy_version 1065650 (0.0010) [2023-12-27 13:01:59,624][20347] Updated weights for policy 0, policy_version 1065660 (0.0007) [2023-12-27 13:01:59,680][20347] Updated weights for policy 0, policy_version 1065670 (0.0007) [2023-12-27 13:02:00,285][20381] Updated weights for policy 1, policy_version 1064831 (0.0008) [2023-12-27 13:02:00,333][20381] Updated weights for policy 1, policy_version 1064841 (0.0008) [2023-12-27 13:02:00,391][20381] Updated weights for policy 1, policy_version 1064851 (0.0006) [2023-12-27 13:02:00,408][20347] Updated weights for policy 0, policy_version 1065680 (0.0007) [2023-12-27 13:02:00,463][20347] Updated weights for policy 0, policy_version 1065690 (0.0007) [2023-12-27 13:02:00,508][20347] Updated weights for policy 0, policy_version 1065700 (0.0007) [2023-12-27 13:02:01,065][20381] Updated weights for policy 1, policy_version 1064861 (0.0006) [2023-12-27 13:02:01,131][20381] Updated weights for policy 1, policy_version 1064871 (0.0008) [2023-12-27 13:02:01,187][20347] Updated weights for policy 0, policy_version 1065710 (0.0009) [2023-12-27 13:02:01,190][20381] Updated weights for policy 1, policy_version 1064881 (0.0008) [2023-12-27 13:02:01,252][20347] Updated weights for policy 0, policy_version 1065720 (0.0010) [2023-12-27 13:02:01,310][20347] Updated weights for policy 0, policy_version 1065730 (0.0010) [2023-12-27 13:02:01,926][20381] Updated weights for policy 1, policy_version 1064891 (0.0007) [2023-12-27 13:02:01,978][20381] Updated weights for policy 1, policy_version 1064901 (0.0005) [2023-12-27 13:02:02,039][20381] Updated weights for policy 1, policy_version 1064911 (0.0006) [2023-12-27 13:02:02,047][20347] Updated weights for policy 0, policy_version 1065740 (0.0009) [2023-12-27 13:02:02,081][20291] KL-divergence is very high: 136.1932 [2023-12-27 13:02:02,096][20347] Updated weights for policy 0, policy_version 1065750 (0.0008) [2023-12-27 13:02:02,105][20291] KL-divergence is very high: 159.9767 [2023-12-27 13:02:02,121][20291] KL-divergence is very high: 115.1308 [2023-12-27 13:02:02,149][20347] Updated weights for policy 0, policy_version 1065760 (0.0007) [2023-12-27 13:02:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 545529856. Throughput: 0: 10332.3, 1: 10331.0. Samples: 545536776. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:02,363][19170] Avg episode reward: [(0, '931.840'), (1, '736.140')] [2023-12-27 13:02:02,718][20381] Updated weights for policy 1, policy_version 1064921 (0.0008) [2023-12-27 13:02:02,776][20381] Updated weights for policy 1, policy_version 1064931 (0.0010) [2023-12-27 13:02:02,790][20347] Updated weights for policy 0, policy_version 1065770 (0.0006) [2023-12-27 13:02:02,821][20381] Updated weights for policy 1, policy_version 1064941 (0.0009) [2023-12-27 13:02:02,833][20292] KL-divergence is very high: 181.6649 [2023-12-27 13:02:02,838][20347] Updated weights for policy 0, policy_version 1065780 (0.0010) [2023-12-27 13:02:02,843][20292] KL-divergence is very high: 137.4665 [2023-12-27 13:02:02,867][20381] Updated weights for policy 1, policy_version 1064951 (0.0010) [2023-12-27 13:02:02,893][20347] Updated weights for policy 0, policy_version 1065790 (0.0010) [2023-12-27 13:02:02,946][20347] Updated weights for policy 0, policy_version 1065800 (0.0010) [2023-12-27 13:02:03,531][20381] Updated weights for policy 1, policy_version 1064961 (0.0006) [2023-12-27 13:02:03,544][20347] Updated weights for policy 0, policy_version 1065810 (0.0010) [2023-12-27 13:02:03,582][20381] Updated weights for policy 1, policy_version 1064971 (0.0005) [2023-12-27 13:02:03,595][20347] Updated weights for policy 0, policy_version 1065820 (0.0007) [2023-12-27 13:02:03,640][20381] Updated weights for policy 1, policy_version 1064981 (0.0006) [2023-12-27 13:02:03,642][20347] Updated weights for policy 0, policy_version 1065830 (0.0005) [2023-12-27 13:02:04,244][20347] Updated weights for policy 0, policy_version 1065840 (0.0008) [2023-12-27 13:02:04,307][20347] Updated weights for policy 0, policy_version 1065850 (0.0007) [2023-12-27 13:02:04,309][20291] KL-divergence is very high: 177.0212 [2023-12-27 13:02:04,328][20381] Updated weights for policy 1, policy_version 1064991 (0.0008) [2023-12-27 13:02:04,360][20291] KL-divergence is very high: 170.7323 [2023-12-27 13:02:04,371][20347] Updated weights for policy 0, policy_version 1065860 (0.0006) [2023-12-27 13:02:04,397][20381] Updated weights for policy 1, policy_version 1065001 (0.0008) [2023-12-27 13:02:04,459][20381] Updated weights for policy 1, policy_version 1065011 (0.0009) [2023-12-27 13:02:05,070][20381] Updated weights for policy 1, policy_version 1065021 (0.0007) [2023-12-27 13:02:05,076][20347] Updated weights for policy 0, policy_version 1065870 (0.0006) [2023-12-27 13:02:05,126][20381] Updated weights for policy 1, policy_version 1065031 (0.0008) [2023-12-27 13:02:05,133][20347] Updated weights for policy 0, policy_version 1065880 (0.0007) [2023-12-27 13:02:05,186][20381] Updated weights for policy 1, policy_version 1065041 (0.0008) [2023-12-27 13:02:05,192][20347] Updated weights for policy 0, policy_version 1065890 (0.0006) [2023-12-27 13:02:05,865][20381] Updated weights for policy 1, policy_version 1065051 (0.0008) [2023-12-27 13:02:05,876][20347] Updated weights for policy 0, policy_version 1065900 (0.0006) [2023-12-27 13:02:05,913][20381] Updated weights for policy 1, policy_version 1065061 (0.0007) [2023-12-27 13:02:05,942][20347] Updated weights for policy 0, policy_version 1065910 (0.0006) [2023-12-27 13:02:05,963][20381] Updated weights for policy 1, policy_version 1065071 (0.0005) [2023-12-27 13:02:06,000][20347] Updated weights for policy 0, policy_version 1065920 (0.0005) [2023-12-27 13:02:06,594][20381] Updated weights for policy 1, policy_version 1065081 (0.0007) [2023-12-27 13:02:06,651][20381] Updated weights for policy 1, policy_version 1065091 (0.0006) [2023-12-27 13:02:06,712][20381] Updated weights for policy 1, policy_version 1065101 (0.0006) [2023-12-27 13:02:06,743][20347] Updated weights for policy 0, policy_version 1065930 (0.0006) [2023-12-27 13:02:06,764][20381] Updated weights for policy 1, policy_version 1065111 (0.0007) [2023-12-27 13:02:06,795][20347] Updated weights for policy 0, policy_version 1065940 (0.0006) [2023-12-27 13:02:06,853][20347] Updated weights for policy 0, policy_version 1065950 (0.0008) [2023-12-27 13:02:06,912][20347] Updated weights for policy 0, policy_version 1065960 (0.0007) [2023-12-27 13:02:07,341][20381] Updated weights for policy 1, policy_version 1065121 (0.0008) [2023-12-27 13:02:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 545628160. Throughput: 0: 10366.0, 1: 10357.9. Samples: 545600608. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:07,363][19170] Avg episode reward: [(0, '892.780'), (1, '785.000')] [2023-12-27 13:02:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001065960_272932864.pth... [2023-12-27 13:02:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001064744_272621568.pth [2023-12-27 13:02:07,404][20381] Updated weights for policy 1, policy_version 1065131 (0.0008) [2023-12-27 13:02:07,465][20381] Updated weights for policy 1, policy_version 1065141 (0.0008) [2023-12-27 13:02:07,483][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001065144_272703488.pth... [2023-12-27 13:02:07,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001063928_272392192.pth [2023-12-27 13:02:07,625][20347] Updated weights for policy 0, policy_version 1065970 (0.0010) [2023-12-27 13:02:07,680][20347] Updated weights for policy 0, policy_version 1065980 (0.0010) [2023-12-27 13:02:07,739][20347] Updated weights for policy 0, policy_version 1065990 (0.0008) [2023-12-27 13:02:08,116][20381] Updated weights for policy 1, policy_version 1065151 (0.0007) [2023-12-27 13:02:08,175][20381] Updated weights for policy 1, policy_version 1065161 (0.0007) [2023-12-27 13:02:08,233][20381] Updated weights for policy 1, policy_version 1065171 (0.0008) [2023-12-27 13:02:08,393][20347] Updated weights for policy 0, policy_version 1066000 (0.0010) [2023-12-27 13:02:08,456][20347] Updated weights for policy 0, policy_version 1066010 (0.0010) [2023-12-27 13:02:08,515][20347] Updated weights for policy 0, policy_version 1066020 (0.0010) [2023-12-27 13:02:08,966][20381] Updated weights for policy 1, policy_version 1065181 (0.0007) [2023-12-27 13:02:09,020][20381] Updated weights for policy 1, policy_version 1065191 (0.0007) [2023-12-27 13:02:09,074][20381] Updated weights for policy 1, policy_version 1065201 (0.0005) [2023-12-27 13:02:09,161][20347] Updated weights for policy 0, policy_version 1066030 (0.0008) [2023-12-27 13:02:09,223][20347] Updated weights for policy 0, policy_version 1066040 (0.0008) [2023-12-27 13:02:09,287][20347] Updated weights for policy 0, policy_version 1066050 (0.0008) [2023-12-27 13:02:09,751][20381] Updated weights for policy 1, policy_version 1065211 (0.0011) [2023-12-27 13:02:09,807][20381] Updated weights for policy 1, policy_version 1065221 (0.0011) [2023-12-27 13:02:09,880][20381] Updated weights for policy 1, policy_version 1065231 (0.0009) [2023-12-27 13:02:10,063][20347] Updated weights for policy 0, policy_version 1066060 (0.0008) [2023-12-27 13:02:10,117][20347] Updated weights for policy 0, policy_version 1066070 (0.0008) [2023-12-27 13:02:10,170][20347] Updated weights for policy 0, policy_version 1066080 (0.0007) [2023-12-27 13:02:10,589][20381] Updated weights for policy 1, policy_version 1065241 (0.0011) [2023-12-27 13:02:10,644][20381] Updated weights for policy 1, policy_version 1065251 (0.0007) [2023-12-27 13:02:10,698][20381] Updated weights for policy 1, policy_version 1065261 (0.0006) [2023-12-27 13:02:10,761][20381] Updated weights for policy 1, policy_version 1065271 (0.0006) [2023-12-27 13:02:10,894][20347] Updated weights for policy 0, policy_version 1066090 (0.0007) [2023-12-27 13:02:10,957][20347] Updated weights for policy 0, policy_version 1066100 (0.0008) [2023-12-27 13:02:11,007][20347] Updated weights for policy 0, policy_version 1066110 (0.0009) [2023-12-27 13:02:11,065][20347] Updated weights for policy 0, policy_version 1066120 (0.0007) [2023-12-27 13:02:11,387][20381] Updated weights for policy 1, policy_version 1065281 (0.0008) [2023-12-27 13:02:11,453][20381] Updated weights for policy 1, policy_version 1065291 (0.0009) [2023-12-27 13:02:11,505][20381] Updated weights for policy 1, policy_version 1065301 (0.0008) [2023-12-27 13:02:11,896][20347] Updated weights for policy 0, policy_version 1066130 (0.0007) [2023-12-27 13:02:11,963][20347] Updated weights for policy 0, policy_version 1066140 (0.0007) [2023-12-27 13:02:12,025][20347] Updated weights for policy 0, policy_version 1066150 (0.0008) [2023-12-27 13:02:12,292][20381] Updated weights for policy 1, policy_version 1065311 (0.0008) [2023-12-27 13:02:12,349][20381] Updated weights for policy 1, policy_version 1065321 (0.0008) [2023-12-27 13:02:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 545726464. Throughput: 0: 10373.8, 1: 10389.1. Samples: 545722484. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:12,363][19170] Avg episode reward: [(0, '834.440'), (1, '814.130')] [2023-12-27 13:02:12,408][20381] Updated weights for policy 1, policy_version 1065331 (0.0008) [2023-12-27 13:02:12,671][20347] Updated weights for policy 0, policy_version 1066160 (0.0008) [2023-12-27 13:02:12,733][20347] Updated weights for policy 0, policy_version 1066170 (0.0008) [2023-12-27 13:02:12,794][20347] Updated weights for policy 0, policy_version 1066180 (0.0008) [2023-12-27 13:02:13,075][20381] Updated weights for policy 1, policy_version 1065341 (0.0007) [2023-12-27 13:02:13,121][20381] Updated weights for policy 1, policy_version 1065351 (0.0005) [2023-12-27 13:02:13,171][20381] Updated weights for policy 1, policy_version 1065361 (0.0005) [2023-12-27 13:02:13,519][20347] Updated weights for policy 0, policy_version 1066190 (0.0009) [2023-12-27 13:02:13,581][20347] Updated weights for policy 0, policy_version 1066200 (0.0008) [2023-12-27 13:02:13,642][20347] Updated weights for policy 0, policy_version 1066210 (0.0005) [2023-12-27 13:02:13,769][20381] Updated weights for policy 1, policy_version 1065371 (0.0006) [2023-12-27 13:02:13,825][20381] Updated weights for policy 1, policy_version 1065381 (0.0005) [2023-12-27 13:02:13,878][20381] Updated weights for policy 1, policy_version 1065391 (0.0005) [2023-12-27 13:02:14,258][20347] Updated weights for policy 0, policy_version 1066220 (0.0007) [2023-12-27 13:02:14,304][20347] Updated weights for policy 0, policy_version 1066230 (0.0010) [2023-12-27 13:02:14,359][20347] Updated weights for policy 0, policy_version 1066240 (0.0010) [2023-12-27 13:02:14,540][20381] Updated weights for policy 1, policy_version 1065401 (0.0007) [2023-12-27 13:02:14,595][20381] Updated weights for policy 1, policy_version 1065411 (0.0008) [2023-12-27 13:02:14,650][20381] Updated weights for policy 1, policy_version 1065421 (0.0006) [2023-12-27 13:02:14,706][20381] Updated weights for policy 1, policy_version 1065431 (0.0006) [2023-12-27 13:02:15,068][20347] Updated weights for policy 0, policy_version 1066250 (0.0010) [2023-12-27 13:02:15,124][20347] Updated weights for policy 0, policy_version 1066260 (0.0006) [2023-12-27 13:02:15,190][20347] Updated weights for policy 0, policy_version 1066270 (0.0009) [2023-12-27 13:02:15,245][20347] Updated weights for policy 0, policy_version 1066280 (0.0010) [2023-12-27 13:02:15,431][20381] Updated weights for policy 1, policy_version 1065441 (0.0007) [2023-12-27 13:02:15,483][20381] Updated weights for policy 1, policy_version 1065451 (0.0008) [2023-12-27 13:02:15,546][20381] Updated weights for policy 1, policy_version 1065461 (0.0007) [2023-12-27 13:02:15,818][20347] Updated weights for policy 0, policy_version 1066290 (0.0008) [2023-12-27 13:02:15,866][20347] Updated weights for policy 0, policy_version 1066300 (0.0008) [2023-12-27 13:02:15,917][20347] Updated weights for policy 0, policy_version 1066310 (0.0007) [2023-12-27 13:02:16,268][20381] Updated weights for policy 1, policy_version 1065471 (0.0007) [2023-12-27 13:02:16,323][20381] Updated weights for policy 1, policy_version 1065481 (0.0007) [2023-12-27 13:02:16,375][20381] Updated weights for policy 1, policy_version 1065491 (0.0008) [2023-12-27 13:02:16,640][20347] Updated weights for policy 0, policy_version 1066320 (0.0007) [2023-12-27 13:02:16,693][20347] Updated weights for policy 0, policy_version 1066330 (0.0008) [2023-12-27 13:02:16,744][20347] Updated weights for policy 0, policy_version 1066340 (0.0008) [2023-12-27 13:02:17,034][20381] Updated weights for policy 1, policy_version 1065501 (0.0008) [2023-12-27 13:02:17,087][20381] Updated weights for policy 1, policy_version 1065511 (0.0009) [2023-12-27 13:02:17,141][20381] Updated weights for policy 1, policy_version 1065521 (0.0006) [2023-12-27 13:02:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 545832960. Throughput: 0: 10347.4, 1: 10283.4. Samples: 545843824. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:17,363][19170] Avg episode reward: [(0, '883.230'), (1, '833.700')] [2023-12-27 13:02:17,376][20347] Updated weights for policy 0, policy_version 1066350 (0.0008) [2023-12-27 13:02:17,427][20347] Updated weights for policy 0, policy_version 1066360 (0.0007) [2023-12-27 13:02:17,482][20347] Updated weights for policy 0, policy_version 1066370 (0.0008) [2023-12-27 13:02:17,826][20381] Updated weights for policy 1, policy_version 1065531 (0.0006) [2023-12-27 13:02:17,873][20381] Updated weights for policy 1, policy_version 1065541 (0.0007) [2023-12-27 13:02:17,919][20381] Updated weights for policy 1, policy_version 1065551 (0.0005) [2023-12-27 13:02:18,153][20347] Updated weights for policy 0, policy_version 1066380 (0.0007) [2023-12-27 13:02:18,221][20347] Updated weights for policy 0, policy_version 1066390 (0.0005) [2023-12-27 13:02:18,278][20347] Updated weights for policy 0, policy_version 1066400 (0.0005) [2023-12-27 13:02:18,588][20381] Updated weights for policy 1, policy_version 1065561 (0.0007) [2023-12-27 13:02:18,641][20381] Updated weights for policy 1, policy_version 1065571 (0.0011) [2023-12-27 13:02:18,697][20381] Updated weights for policy 1, policy_version 1065581 (0.0011) [2023-12-27 13:02:18,750][20381] Updated weights for policy 1, policy_version 1065591 (0.0011) [2023-12-27 13:02:18,881][20347] Updated weights for policy 0, policy_version 1066410 (0.0006) [2023-12-27 13:02:18,943][20347] Updated weights for policy 0, policy_version 1066420 (0.0007) [2023-12-27 13:02:18,994][20347] Updated weights for policy 0, policy_version 1066430 (0.0009) [2023-12-27 13:02:19,043][20347] Updated weights for policy 0, policy_version 1066440 (0.0007) [2023-12-27 13:02:19,473][20381] Updated weights for policy 1, policy_version 1065601 (0.0007) [2023-12-27 13:02:19,538][20381] Updated weights for policy 1, policy_version 1065611 (0.0007) [2023-12-27 13:02:19,596][20381] Updated weights for policy 1, policy_version 1065621 (0.0007) [2023-12-27 13:02:19,766][20347] Updated weights for policy 0, policy_version 1066450 (0.0006) [2023-12-27 13:02:19,834][20347] Updated weights for policy 0, policy_version 1066460 (0.0007) [2023-12-27 13:02:19,898][20347] Updated weights for policy 0, policy_version 1066470 (0.0007) [2023-12-27 13:02:20,291][20381] Updated weights for policy 1, policy_version 1065631 (0.0008) [2023-12-27 13:02:20,354][20381] Updated weights for policy 1, policy_version 1065641 (0.0007) [2023-12-27 13:02:20,415][20381] Updated weights for policy 1, policy_version 1065651 (0.0008) [2023-12-27 13:02:20,589][20347] Updated weights for policy 0, policy_version 1066480 (0.0010) [2023-12-27 13:02:20,652][20347] Updated weights for policy 0, policy_version 1066490 (0.0010) [2023-12-27 13:02:20,715][20347] Updated weights for policy 0, policy_version 1066500 (0.0010) [2023-12-27 13:02:21,169][20381] Updated weights for policy 1, policy_version 1065661 (0.0008) [2023-12-27 13:02:21,228][20381] Updated weights for policy 1, policy_version 1065671 (0.0008) [2023-12-27 13:02:21,290][20381] Updated weights for policy 1, policy_version 1065681 (0.0008) [2023-12-27 13:02:21,417][20347] Updated weights for policy 0, policy_version 1066510 (0.0010) [2023-12-27 13:02:21,473][20347] Updated weights for policy 0, policy_version 1066520 (0.0010) [2023-12-27 13:02:21,537][20347] Updated weights for policy 0, policy_version 1066530 (0.0010) [2023-12-27 13:02:22,041][20381] Updated weights for policy 1, policy_version 1065691 (0.0007) [2023-12-27 13:02:22,106][20381] Updated weights for policy 1, policy_version 1065701 (0.0008) [2023-12-27 13:02:22,177][20381] Updated weights for policy 1, policy_version 1065711 (0.0008) [2023-12-27 13:02:22,271][20347] Updated weights for policy 0, policy_version 1066540 (0.0009) [2023-12-27 13:02:22,331][20347] Updated weights for policy 0, policy_version 1066550 (0.0006) [2023-12-27 13:02:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 545931264. Throughput: 0: 10321.8, 1: 10269.1. Samples: 545905588. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:22,363][19170] Avg episode reward: [(0, '922.050'), (1, '814.140')] [2023-12-27 13:02:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001065720_272850944.pth... [2023-12-27 13:02:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001064504_272539648.pth [2023-12-27 13:02:22,409][20347] Updated weights for policy 0, policy_version 1066560 (0.0008) [2023-12-27 13:02:22,453][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001066568_273088512.pth... [2023-12-27 13:02:22,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001065352_272777216.pth [2023-12-27 13:02:22,870][20381] Updated weights for policy 1, policy_version 1065721 (0.0007) [2023-12-27 13:02:22,920][20381] Updated weights for policy 1, policy_version 1065731 (0.0006) [2023-12-27 13:02:22,971][20381] Updated weights for policy 1, policy_version 1065741 (0.0005) [2023-12-27 13:02:23,021][20381] Updated weights for policy 1, policy_version 1065751 (0.0007) [2023-12-27 13:02:23,088][20347] Updated weights for policy 0, policy_version 1066570 (0.0008) [2023-12-27 13:02:23,143][20347] Updated weights for policy 0, policy_version 1066580 (0.0010) [2023-12-27 13:02:23,192][20347] Updated weights for policy 0, policy_version 1066590 (0.0010) [2023-12-27 13:02:23,240][20347] Updated weights for policy 0, policy_version 1066600 (0.0010) [2023-12-27 13:02:23,722][20381] Updated weights for policy 1, policy_version 1065761 (0.0007) [2023-12-27 13:02:23,770][20381] Updated weights for policy 1, policy_version 1065771 (0.0008) [2023-12-27 13:02:23,819][20381] Updated weights for policy 1, policy_version 1065781 (0.0009) [2023-12-27 13:02:23,876][20347] Updated weights for policy 0, policy_version 1066610 (0.0005) [2023-12-27 13:02:23,932][20347] Updated weights for policy 0, policy_version 1066620 (0.0006) [2023-12-27 13:02:23,988][20347] Updated weights for policy 0, policy_version 1066630 (0.0010) [2023-12-27 13:02:24,547][20381] Updated weights for policy 1, policy_version 1065791 (0.0007) [2023-12-27 13:02:24,597][20381] Updated weights for policy 1, policy_version 1065801 (0.0008) [2023-12-27 13:02:24,623][20347] Updated weights for policy 0, policy_version 1066640 (0.0008) [2023-12-27 13:02:24,654][20381] Updated weights for policy 1, policy_version 1065811 (0.0007) [2023-12-27 13:02:24,676][20347] Updated weights for policy 0, policy_version 1066650 (0.0008) [2023-12-27 13:02:24,735][20347] Updated weights for policy 0, policy_version 1066660 (0.0008) [2023-12-27 13:02:25,341][20381] Updated weights for policy 1, policy_version 1065821 (0.0007) [2023-12-27 13:02:25,393][20381] Updated weights for policy 1, policy_version 1065831 (0.0006) [2023-12-27 13:02:25,440][20347] Updated weights for policy 0, policy_version 1066670 (0.0007) [2023-12-27 13:02:25,449][20381] Updated weights for policy 1, policy_version 1065841 (0.0008) [2023-12-27 13:02:25,499][20347] Updated weights for policy 0, policy_version 1066680 (0.0005) [2023-12-27 13:02:25,554][20347] Updated weights for policy 0, policy_version 1066690 (0.0005) [2023-12-27 13:02:26,077][20381] Updated weights for policy 1, policy_version 1065851 (0.0007) [2023-12-27 13:02:26,139][20381] Updated weights for policy 1, policy_version 1065861 (0.0007) [2023-12-27 13:02:26,195][20347] Updated weights for policy 0, policy_version 1066700 (0.0005) [2023-12-27 13:02:26,199][20381] Updated weights for policy 1, policy_version 1065871 (0.0007) [2023-12-27 13:02:26,223][20291] KL-divergence is very high: 355.0521 [2023-12-27 13:02:26,241][20291] KL-divergence is very high: 112.6298 [2023-12-27 13:02:26,252][20347] Updated weights for policy 0, policy_version 1066710 (0.0005) [2023-12-27 13:02:26,268][20291] KL-divergence is very high: 177.2899 [2023-12-27 13:02:26,313][20347] Updated weights for policy 0, policy_version 1066720 (0.0007) [2023-12-27 13:02:26,906][20381] Updated weights for policy 1, policy_version 1065881 (0.0007) [2023-12-27 13:02:26,915][20347] Updated weights for policy 0, policy_version 1066730 (0.0010) [2023-12-27 13:02:26,961][20381] Updated weights for policy 1, policy_version 1065891 (0.0006) [2023-12-27 13:02:26,967][20347] Updated weights for policy 0, policy_version 1066740 (0.0010) [2023-12-27 13:02:27,015][20381] Updated weights for policy 1, policy_version 1065901 (0.0005) [2023-12-27 13:02:27,021][20347] Updated weights for policy 0, policy_version 1066750 (0.0010) [2023-12-27 13:02:27,066][20381] Updated weights for policy 1, policy_version 1065911 (0.0005) [2023-12-27 13:02:27,068][20347] Updated weights for policy 0, policy_version 1066760 (0.0010) [2023-12-27 13:02:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 546037760. Throughput: 0: 10276.1, 1: 10217.8. Samples: 546025104. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:27,363][19170] Avg episode reward: [(0, '873.770'), (1, '833.810')] [2023-12-27 13:02:27,680][20347] Updated weights for policy 0, policy_version 1066770 (0.0005) [2023-12-27 13:02:27,729][20347] Updated weights for policy 0, policy_version 1066780 (0.0005) [2023-12-27 13:02:27,773][20347] Updated weights for policy 0, policy_version 1066790 (0.0006) [2023-12-27 13:02:27,785][20381] Updated weights for policy 1, policy_version 1065921 (0.0007) [2023-12-27 13:02:27,841][20381] Updated weights for policy 1, policy_version 1065931 (0.0008) [2023-12-27 13:02:27,896][20381] Updated weights for policy 1, policy_version 1065941 (0.0009) [2023-12-27 13:02:28,412][20347] Updated weights for policy 0, policy_version 1066800 (0.0008) [2023-12-27 13:02:28,471][20347] Updated weights for policy 0, policy_version 1066810 (0.0007) [2023-12-27 13:02:28,529][20347] Updated weights for policy 0, policy_version 1066820 (0.0006) [2023-12-27 13:02:28,597][20381] Updated weights for policy 1, policy_version 1065951 (0.0008) [2023-12-27 13:02:28,653][20381] Updated weights for policy 1, policy_version 1065961 (0.0008) [2023-12-27 13:02:28,712][20381] Updated weights for policy 1, policy_version 1065971 (0.0008) [2023-12-27 13:02:29,104][20347] Updated weights for policy 0, policy_version 1066830 (0.0006) [2023-12-27 13:02:29,161][20347] Updated weights for policy 0, policy_version 1066840 (0.0007) [2023-12-27 13:02:29,218][20347] Updated weights for policy 0, policy_version 1066850 (0.0008) [2023-12-27 13:02:29,434][20381] Updated weights for policy 1, policy_version 1065981 (0.0008) [2023-12-27 13:02:29,487][20381] Updated weights for policy 1, policy_version 1065991 (0.0007) [2023-12-27 13:02:29,536][20381] Updated weights for policy 1, policy_version 1066001 (0.0008) [2023-12-27 13:02:29,960][20347] Updated weights for policy 0, policy_version 1066860 (0.0009) [2023-12-27 13:02:30,017][20347] Updated weights for policy 0, policy_version 1066870 (0.0007) [2023-12-27 13:02:30,078][20347] Updated weights for policy 0, policy_version 1066880 (0.0009) [2023-12-27 13:02:30,153][20381] Updated weights for policy 1, policy_version 1066011 (0.0007) [2023-12-27 13:02:30,199][20381] Updated weights for policy 1, policy_version 1066021 (0.0006) [2023-12-27 13:02:30,246][20381] Updated weights for policy 1, policy_version 1066031 (0.0005) [2023-12-27 13:02:30,753][20347] Updated weights for policy 0, policy_version 1066890 (0.0011) [2023-12-27 13:02:30,813][20347] Updated weights for policy 0, policy_version 1066900 (0.0010) [2023-12-27 13:02:30,866][20347] Updated weights for policy 0, policy_version 1066910 (0.0010) [2023-12-27 13:02:30,927][20347] Updated weights for policy 0, policy_version 1066920 (0.0010) [2023-12-27 13:02:30,935][20381] Updated weights for policy 1, policy_version 1066041 (0.0005) [2023-12-27 13:02:30,983][20381] Updated weights for policy 1, policy_version 1066051 (0.0005) [2023-12-27 13:02:31,035][20381] Updated weights for policy 1, policy_version 1066061 (0.0006) [2023-12-27 13:02:31,101][20381] Updated weights for policy 1, policy_version 1066071 (0.0006) [2023-12-27 13:02:31,637][20347] Updated weights for policy 0, policy_version 1066930 (0.0007) [2023-12-27 13:02:31,698][20347] Updated weights for policy 0, policy_version 1066940 (0.0009) [2023-12-27 13:02:31,759][20347] Updated weights for policy 0, policy_version 1066950 (0.0007) [2023-12-27 13:02:31,774][20381] Updated weights for policy 1, policy_version 1066081 (0.0006) [2023-12-27 13:02:31,837][20381] Updated weights for policy 1, policy_version 1066091 (0.0005) [2023-12-27 13:02:31,901][20381] Updated weights for policy 1, policy_version 1066101 (0.0005) [2023-12-27 13:02:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 546136064. Throughput: 0: 10286.6, 1: 10219.2. Samples: 546149100. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:32,363][19170] Avg episode reward: [(0, '883.410'), (1, '785.100')] [2023-12-27 13:02:32,463][20347] Updated weights for policy 0, policy_version 1066960 (0.0005) [2023-12-27 13:02:32,519][20347] Updated weights for policy 0, policy_version 1066970 (0.0005) [2023-12-27 13:02:32,523][20381] Updated weights for policy 1, policy_version 1066111 (0.0006) [2023-12-27 13:02:32,574][20381] Updated weights for policy 1, policy_version 1066121 (0.0007) [2023-12-27 13:02:32,584][20347] Updated weights for policy 0, policy_version 1066980 (0.0006) [2023-12-27 13:02:32,628][20381] Updated weights for policy 1, policy_version 1066131 (0.0007) [2023-12-27 13:02:33,177][20347] Updated weights for policy 0, policy_version 1066990 (0.0006) [2023-12-27 13:02:33,234][20347] Updated weights for policy 0, policy_version 1067000 (0.0005) [2023-12-27 13:02:33,286][20347] Updated weights for policy 0, policy_version 1067010 (0.0005) [2023-12-27 13:02:33,326][20381] Updated weights for policy 1, policy_version 1066141 (0.0007) [2023-12-27 13:02:33,372][20381] Updated weights for policy 1, policy_version 1066151 (0.0007) [2023-12-27 13:02:33,430][20381] Updated weights for policy 1, policy_version 1066161 (0.0005) [2023-12-27 13:02:33,873][20347] Updated weights for policy 0, policy_version 1067020 (0.0007) [2023-12-27 13:02:33,927][20347] Updated weights for policy 0, policy_version 1067030 (0.0010) [2023-12-27 13:02:33,975][20347] Updated weights for policy 0, policy_version 1067040 (0.0010) [2023-12-27 13:02:33,997][20381] Updated weights for policy 1, policy_version 1066171 (0.0005) [2023-12-27 13:02:34,044][20381] Updated weights for policy 1, policy_version 1066181 (0.0006) [2023-12-27 13:02:34,099][20381] Updated weights for policy 1, policy_version 1066191 (0.0010) [2023-12-27 13:02:34,662][20347] Updated weights for policy 0, policy_version 1067050 (0.0009) [2023-12-27 13:02:34,724][20347] Updated weights for policy 0, policy_version 1067060 (0.0008) [2023-12-27 13:02:34,735][20381] Updated weights for policy 1, policy_version 1066201 (0.0010) [2023-12-27 13:02:34,784][20347] Updated weights for policy 0, policy_version 1067070 (0.0008) [2023-12-27 13:02:34,796][20381] Updated weights for policy 1, policy_version 1066211 (0.0009) [2023-12-27 13:02:34,841][20347] Updated weights for policy 0, policy_version 1067080 (0.0005) [2023-12-27 13:02:34,858][20381] Updated weights for policy 1, policy_version 1066221 (0.0008) [2023-12-27 13:02:34,916][20381] Updated weights for policy 1, policy_version 1066231 (0.0007) [2023-12-27 13:02:35,469][20347] Updated weights for policy 0, policy_version 1067090 (0.0011) [2023-12-27 13:02:35,528][20347] Updated weights for policy 0, policy_version 1067100 (0.0010) [2023-12-27 13:02:35,590][20347] Updated weights for policy 0, policy_version 1067110 (0.0008) [2023-12-27 13:02:35,601][20381] Updated weights for policy 1, policy_version 1066241 (0.0011) [2023-12-27 13:02:35,655][20381] Updated weights for policy 1, policy_version 1066251 (0.0011) [2023-12-27 13:02:35,708][20381] Updated weights for policy 1, policy_version 1066261 (0.0010) [2023-12-27 13:02:36,298][20347] Updated weights for policy 0, policy_version 1067120 (0.0010) [2023-12-27 13:02:36,351][20347] Updated weights for policy 0, policy_version 1067130 (0.0010) [2023-12-27 13:02:36,368][20381] Updated weights for policy 1, policy_version 1066271 (0.0007) [2023-12-27 13:02:36,410][20347] Updated weights for policy 0, policy_version 1067140 (0.0010) [2023-12-27 13:02:36,432][20381] Updated weights for policy 1, policy_version 1066281 (0.0006) [2023-12-27 13:02:36,500][20381] Updated weights for policy 1, policy_version 1066291 (0.0006) [2023-12-27 13:02:37,089][20347] Updated weights for policy 0, policy_version 1067150 (0.0011) [2023-12-27 13:02:37,138][20347] Updated weights for policy 0, policy_version 1067160 (0.0010) [2023-12-27 13:02:37,156][20381] Updated weights for policy 1, policy_version 1066301 (0.0009) [2023-12-27 13:02:37,187][20347] Updated weights for policy 0, policy_version 1067170 (0.0010) [2023-12-27 13:02:37,205][20381] Updated weights for policy 1, policy_version 1066311 (0.0006) [2023-12-27 13:02:37,255][20381] Updated weights for policy 1, policy_version 1066321 (0.0007) [2023-12-27 13:02:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 546250752. Throughput: 0: 10317.8, 1: 10249.0. Samples: 546213896. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:37,363][19170] Avg episode reward: [(0, '931.650'), (1, '794.820')] [2023-12-27 13:02:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001067176_273244160.pth... [2023-12-27 13:02:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001066328_273006592.pth... [2023-12-27 13:02:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001065960_272932864.pth [2023-12-27 13:02:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001065144_272703488.pth [2023-12-27 13:02:37,888][20347] Updated weights for policy 0, policy_version 1067180 (0.0010) [2023-12-27 13:02:37,934][20381] Updated weights for policy 1, policy_version 1066331 (0.0006) [2023-12-27 13:02:37,951][20347] Updated weights for policy 0, policy_version 1067190 (0.0008) [2023-12-27 13:02:37,993][20381] Updated weights for policy 1, policy_version 1066341 (0.0006) [2023-12-27 13:02:38,012][20347] Updated weights for policy 0, policy_version 1067200 (0.0006) [2023-12-27 13:02:38,054][20381] Updated weights for policy 1, policy_version 1066351 (0.0006) [2023-12-27 13:02:38,629][20347] Updated weights for policy 0, policy_version 1067210 (0.0006) [2023-12-27 13:02:38,678][20381] Updated weights for policy 1, policy_version 1066361 (0.0005) [2023-12-27 13:02:38,688][20347] Updated weights for policy 0, policy_version 1067220 (0.0007) [2023-12-27 13:02:38,738][20381] Updated weights for policy 1, policy_version 1066371 (0.0005) [2023-12-27 13:02:38,743][20347] Updated weights for policy 0, policy_version 1067230 (0.0007) [2023-12-27 13:02:38,790][20381] Updated weights for policy 1, policy_version 1066381 (0.0005) [2023-12-27 13:02:38,806][20347] Updated weights for policy 0, policy_version 1067240 (0.0007) [2023-12-27 13:02:38,840][20381] Updated weights for policy 1, policy_version 1066391 (0.0007) [2023-12-27 13:02:39,504][20347] Updated weights for policy 0, policy_version 1067250 (0.0008) [2023-12-27 13:02:39,512][20381] Updated weights for policy 1, policy_version 1066401 (0.0007) [2023-12-27 13:02:39,565][20347] Updated weights for policy 0, policy_version 1067260 (0.0009) [2023-12-27 13:02:39,572][20381] Updated weights for policy 1, policy_version 1066411 (0.0010) [2023-12-27 13:02:39,614][20347] Updated weights for policy 0, policy_version 1067270 (0.0009) [2023-12-27 13:02:39,625][20381] Updated weights for policy 1, policy_version 1066421 (0.0009) [2023-12-27 13:02:40,401][20381] Updated weights for policy 1, policy_version 1066431 (0.0010) [2023-12-27 13:02:40,414][20347] Updated weights for policy 0, policy_version 1067280 (0.0006) [2023-12-27 13:02:40,454][20381] Updated weights for policy 1, policy_version 1066441 (0.0010) [2023-12-27 13:02:40,470][20347] Updated weights for policy 0, policy_version 1067290 (0.0006) [2023-12-27 13:02:40,507][20381] Updated weights for policy 1, policy_version 1066451 (0.0010) [2023-12-27 13:02:40,524][20347] Updated weights for policy 0, policy_version 1067300 (0.0005) [2023-12-27 13:02:41,234][20381] Updated weights for policy 1, policy_version 1066461 (0.0010) [2023-12-27 13:02:41,275][20347] Updated weights for policy 0, policy_version 1067310 (0.0007) [2023-12-27 13:02:41,292][20381] Updated weights for policy 1, policy_version 1066471 (0.0009) [2023-12-27 13:02:41,329][20347] Updated weights for policy 0, policy_version 1067320 (0.0009) [2023-12-27 13:02:41,347][20381] Updated weights for policy 1, policy_version 1066481 (0.0009) [2023-12-27 13:02:41,399][20347] Updated weights for policy 0, policy_version 1067330 (0.0010) [2023-12-27 13:02:42,210][20381] Updated weights for policy 1, policy_version 1066491 (0.0008) [2023-12-27 13:02:42,265][20347] Updated weights for policy 0, policy_version 1067340 (0.0010) [2023-12-27 13:02:42,267][20381] Updated weights for policy 1, policy_version 1066501 (0.0007) [2023-12-27 13:02:42,321][20381] Updated weights for policy 1, policy_version 1066511 (0.0007) [2023-12-27 13:02:42,326][20347] Updated weights for policy 0, policy_version 1067350 (0.0011) [2023-12-27 13:02:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 546332672. Throughput: 0: 10279.2, 1: 10177.4. Samples: 546334576. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:42,363][19170] Avg episode reward: [(0, '834.260'), (1, '745.770')] [2023-12-27 13:02:42,393][20347] Updated weights for policy 0, policy_version 1067360 (0.0010) [2023-12-27 13:02:42,933][20381] Updated weights for policy 1, policy_version 1066521 (0.0008) [2023-12-27 13:02:42,977][20347] Updated weights for policy 0, policy_version 1067370 (0.0007) [2023-12-27 13:02:42,991][20381] Updated weights for policy 1, policy_version 1066531 (0.0009) [2023-12-27 13:02:43,031][20347] Updated weights for policy 0, policy_version 1067380 (0.0005) [2023-12-27 13:02:43,047][20381] Updated weights for policy 1, policy_version 1066541 (0.0011) [2023-12-27 13:02:43,080][20291] KL-divergence is very high: 162.4006 [2023-12-27 13:02:43,092][20347] Updated weights for policy 0, policy_version 1067390 (0.0005) [2023-12-27 13:02:43,099][20291] KL-divergence is very high: 359.9521 [2023-12-27 13:02:43,102][20381] Updated weights for policy 1, policy_version 1066551 (0.0011) [2023-12-27 13:02:43,129][20291] KL-divergence is very high: 170.6861 [2023-12-27 13:02:43,146][20291] KL-divergence is very high: 274.7570 [2023-12-27 13:02:43,151][20347] Updated weights for policy 0, policy_version 1067400 (0.0006) [2023-12-27 13:02:43,630][20291] KL-divergence is very high: 235.6945 [2023-12-27 13:02:43,684][20347] Updated weights for policy 0, policy_version 1067410 (0.0005) [2023-12-27 13:02:43,717][20381] Updated weights for policy 1, policy_version 1066561 (0.0006) [2023-12-27 13:02:43,741][20347] Updated weights for policy 0, policy_version 1067420 (0.0005) [2023-12-27 13:02:43,770][20381] Updated weights for policy 1, policy_version 1066571 (0.0005) [2023-12-27 13:02:43,788][20291] KL-divergence is very high: 363.1959 [2023-12-27 13:02:43,794][20291] KL-divergence is very high: 100.8508 [2023-12-27 13:02:43,800][20347] Updated weights for policy 0, policy_version 1067430 (0.0007) [2023-12-27 13:02:43,819][20381] Updated weights for policy 1, policy_version 1066581 (0.0005) [2023-12-27 13:02:44,393][20381] Updated weights for policy 1, policy_version 1066591 (0.0009) [2023-12-27 13:02:44,439][20347] Updated weights for policy 0, policy_version 1067440 (0.0006) [2023-12-27 13:02:44,447][20381] Updated weights for policy 1, policy_version 1066601 (0.0010) [2023-12-27 13:02:44,490][20347] Updated weights for policy 0, policy_version 1067450 (0.0006) [2023-12-27 13:02:44,493][20381] Updated weights for policy 1, policy_version 1066611 (0.0010) [2023-12-27 13:02:44,544][20347] Updated weights for policy 0, policy_version 1067460 (0.0007) [2023-12-27 13:02:45,161][20381] Updated weights for policy 1, policy_version 1066621 (0.0009) [2023-12-27 13:02:45,227][20381] Updated weights for policy 1, policy_version 1066631 (0.0008) [2023-12-27 13:02:45,233][20291] KL-divergence is very high: 141.8450 [2023-12-27 13:02:45,256][20347] Updated weights for policy 0, policy_version 1067470 (0.0007) [2023-12-27 13:02:45,295][20381] Updated weights for policy 1, policy_version 1066641 (0.0008) [2023-12-27 13:02:45,315][20347] Updated weights for policy 0, policy_version 1067480 (0.0006) [2023-12-27 13:02:45,375][20347] Updated weights for policy 0, policy_version 1067490 (0.0005) [2023-12-27 13:02:45,985][20347] Updated weights for policy 0, policy_version 1067500 (0.0006) [2023-12-27 13:02:46,001][20381] Updated weights for policy 1, policy_version 1066651 (0.0007) [2023-12-27 13:02:46,048][20347] Updated weights for policy 0, policy_version 1067510 (0.0007) [2023-12-27 13:02:46,063][20381] Updated weights for policy 1, policy_version 1066661 (0.0007) [2023-12-27 13:02:46,112][20347] Updated weights for policy 0, policy_version 1067520 (0.0008) [2023-12-27 13:02:46,128][20381] Updated weights for policy 1, policy_version 1066671 (0.0005) [2023-12-27 13:02:46,642][20347] Updated weights for policy 0, policy_version 1067530 (0.0006) [2023-12-27 13:02:46,699][20347] Updated weights for policy 0, policy_version 1067540 (0.0005) [2023-12-27 13:02:46,757][20347] Updated weights for policy 0, policy_version 1067550 (0.0005) [2023-12-27 13:02:46,790][20381] Updated weights for policy 1, policy_version 1066681 (0.0005) [2023-12-27 13:02:46,816][20347] Updated weights for policy 0, policy_version 1067560 (0.0007) [2023-12-27 13:02:46,840][20381] Updated weights for policy 1, policy_version 1066691 (0.0006) [2023-12-27 13:02:46,889][20381] Updated weights for policy 1, policy_version 1066701 (0.0005) [2023-12-27 13:02:46,948][20381] Updated weights for policy 1, policy_version 1066711 (0.0006) [2023-12-27 13:02:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 546447360. Throughput: 0: 10303.7, 1: 10220.0. Samples: 546460344. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:47,363][19170] Avg episode reward: [(0, '892.810'), (1, '657.630')] [2023-12-27 13:02:47,401][20347] Updated weights for policy 0, policy_version 1067570 (0.0005) [2023-12-27 13:02:47,462][20347] Updated weights for policy 0, policy_version 1067580 (0.0005) [2023-12-27 13:02:47,507][20347] Updated weights for policy 0, policy_version 1067590 (0.0006) [2023-12-27 13:02:47,639][20381] Updated weights for policy 1, policy_version 1066721 (0.0007) [2023-12-27 13:02:47,690][20381] Updated weights for policy 1, policy_version 1066731 (0.0005) [2023-12-27 13:02:47,755][20381] Updated weights for policy 1, policy_version 1066741 (0.0008) [2023-12-27 13:02:48,081][20347] Updated weights for policy 0, policy_version 1067600 (0.0006) [2023-12-27 13:02:48,142][20347] Updated weights for policy 0, policy_version 1067610 (0.0005) [2023-12-27 13:02:48,193][20347] Updated weights for policy 0, policy_version 1067620 (0.0006) [2023-12-27 13:02:48,315][20381] Updated weights for policy 1, policy_version 1066751 (0.0010) [2023-12-27 13:02:48,374][20381] Updated weights for policy 1, policy_version 1066761 (0.0008) [2023-12-27 13:02:48,423][20381] Updated weights for policy 1, policy_version 1066771 (0.0010) [2023-12-27 13:02:48,888][20347] Updated weights for policy 0, policy_version 1067630 (0.0007) [2023-12-27 13:02:48,948][20347] Updated weights for policy 0, policy_version 1067640 (0.0007) [2023-12-27 13:02:49,008][20347] Updated weights for policy 0, policy_version 1067650 (0.0008) [2023-12-27 13:02:49,051][20381] Updated weights for policy 1, policy_version 1066781 (0.0007) [2023-12-27 13:02:49,116][20381] Updated weights for policy 1, policy_version 1066791 (0.0009) [2023-12-27 13:02:49,178][20381] Updated weights for policy 1, policy_version 1066801 (0.0007) [2023-12-27 13:02:49,766][20347] Updated weights for policy 0, policy_version 1067660 (0.0006) [2023-12-27 13:02:49,806][20381] Updated weights for policy 1, policy_version 1066811 (0.0007) [2023-12-27 13:02:49,824][20347] Updated weights for policy 0, policy_version 1067670 (0.0008) [2023-12-27 13:02:49,873][20381] Updated weights for policy 1, policy_version 1066821 (0.0008) [2023-12-27 13:02:49,883][20347] Updated weights for policy 0, policy_version 1067680 (0.0007) [2023-12-27 13:02:49,934][20381] Updated weights for policy 1, policy_version 1066831 (0.0007) [2023-12-27 13:02:50,572][20381] Updated weights for policy 1, policy_version 1066841 (0.0006) [2023-12-27 13:02:50,630][20347] Updated weights for policy 0, policy_version 1067690 (0.0007) [2023-12-27 13:02:50,633][20381] Updated weights for policy 1, policy_version 1066851 (0.0008) [2023-12-27 13:02:50,691][20347] Updated weights for policy 0, policy_version 1067700 (0.0008) [2023-12-27 13:02:50,693][20381] Updated weights for policy 1, policy_version 1066861 (0.0006) [2023-12-27 13:02:50,751][20347] Updated weights for policy 0, policy_version 1067710 (0.0008) [2023-12-27 13:02:50,754][20381] Updated weights for policy 1, policy_version 1066871 (0.0006) [2023-12-27 13:02:50,804][20347] Updated weights for policy 0, policy_version 1067720 (0.0008) [2023-12-27 13:02:51,449][20381] Updated weights for policy 1, policy_version 1066881 (0.0006) [2023-12-27 13:02:51,508][20381] Updated weights for policy 1, policy_version 1066891 (0.0006) [2023-12-27 13:02:51,562][20381] Updated weights for policy 1, policy_version 1066901 (0.0007) [2023-12-27 13:02:51,579][20347] Updated weights for policy 0, policy_version 1067730 (0.0006) [2023-12-27 13:02:51,641][20347] Updated weights for policy 0, policy_version 1067740 (0.0008) [2023-12-27 13:02:51,703][20347] Updated weights for policy 0, policy_version 1067750 (0.0008) [2023-12-27 13:02:52,288][20381] Updated weights for policy 1, policy_version 1066911 (0.0007) [2023-12-27 13:02:52,344][20381] Updated weights for policy 1, policy_version 1066921 (0.0007) [2023-12-27 13:02:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 546545664. Throughput: 0: 10263.3, 1: 10259.7. Samples: 546524148. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:52,363][19170] Avg episode reward: [(0, '941.290'), (1, '647.740')] [2023-12-27 13:02:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001067752_273391616.pth... [2023-12-27 13:02:52,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001066568_273088512.pth [2023-12-27 13:02:52,411][20381] Updated weights for policy 1, policy_version 1066931 (0.0008) [2023-12-27 13:02:52,441][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001066936_273162240.pth... [2023-12-27 13:02:52,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001065720_272850944.pth [2023-12-27 13:02:52,464][20347] Updated weights for policy 0, policy_version 1067760 (0.0008) [2023-12-27 13:02:52,528][20347] Updated weights for policy 0, policy_version 1067770 (0.0007) [2023-12-27 13:02:52,597][20347] Updated weights for policy 0, policy_version 1067780 (0.0007) [2023-12-27 13:02:53,171][20381] Updated weights for policy 1, policy_version 1066941 (0.0007) [2023-12-27 13:02:53,228][20381] Updated weights for policy 1, policy_version 1066951 (0.0007) [2023-12-27 13:02:53,256][20347] Updated weights for policy 0, policy_version 1067790 (0.0006) [2023-12-27 13:02:53,277][20381] Updated weights for policy 1, policy_version 1066961 (0.0008) [2023-12-27 13:02:53,310][20347] Updated weights for policy 0, policy_version 1067800 (0.0005) [2023-12-27 13:02:53,366][20347] Updated weights for policy 0, policy_version 1067810 (0.0010) [2023-12-27 13:02:53,896][20381] Updated weights for policy 1, policy_version 1066971 (0.0008) [2023-12-27 13:02:53,927][20347] Updated weights for policy 0, policy_version 1067820 (0.0009) [2023-12-27 13:02:53,944][20381] Updated weights for policy 1, policy_version 1066981 (0.0007) [2023-12-27 13:02:53,964][20292] KL-divergence is very high: 136.0037 [2023-12-27 13:02:53,979][20347] Updated weights for policy 0, policy_version 1067830 (0.0008) [2023-12-27 13:02:53,994][20381] Updated weights for policy 1, policy_version 1066991 (0.0008) [2023-12-27 13:02:54,031][20347] Updated weights for policy 0, policy_version 1067840 (0.0010) [2023-12-27 13:02:54,625][20381] Updated weights for policy 1, policy_version 1067001 (0.0007) [2023-12-27 13:02:54,640][20347] Updated weights for policy 0, policy_version 1067850 (0.0010) [2023-12-27 13:02:54,672][20381] Updated weights for policy 1, policy_version 1067011 (0.0005) [2023-12-27 13:02:54,696][20347] Updated weights for policy 0, policy_version 1067860 (0.0011) [2023-12-27 13:02:54,730][20381] Updated weights for policy 1, policy_version 1067021 (0.0005) [2023-12-27 13:02:54,742][20347] Updated weights for policy 0, policy_version 1067870 (0.0010) [2023-12-27 13:02:54,788][20347] Updated weights for policy 0, policy_version 1067880 (0.0006) [2023-12-27 13:02:54,792][20381] Updated weights for policy 1, policy_version 1067031 (0.0005) [2023-12-27 13:02:55,323][20381] Updated weights for policy 1, policy_version 1067041 (0.0007) [2023-12-27 13:02:55,379][20381] Updated weights for policy 1, policy_version 1067051 (0.0007) [2023-12-27 13:02:55,428][20381] Updated weights for policy 1, policy_version 1067061 (0.0008) [2023-12-27 13:02:55,486][20347] Updated weights for policy 0, policy_version 1067890 (0.0008) [2023-12-27 13:02:55,538][20291] KL-divergence is very high: 123.3864 [2023-12-27 13:02:55,542][20347] Updated weights for policy 0, policy_version 1067900 (0.0006) [2023-12-27 13:02:55,561][20291] KL-divergence is very high: 184.4129 [2023-12-27 13:02:55,585][20291] KL-divergence is very high: 241.9797 [2023-12-27 13:02:55,602][20347] Updated weights for policy 0, policy_version 1067910 (0.0005) [2023-12-27 13:02:55,610][20291] KL-divergence is very high: 119.3550 [2023-12-27 13:02:56,124][20347] Updated weights for policy 0, policy_version 1067920 (0.0005) [2023-12-27 13:02:56,183][20381] Updated weights for policy 1, policy_version 1067071 (0.0006) [2023-12-27 13:02:56,184][20347] Updated weights for policy 0, policy_version 1067930 (0.0005) [2023-12-27 13:02:56,232][20347] Updated weights for policy 0, policy_version 1067940 (0.0005) [2023-12-27 13:02:56,237][20381] Updated weights for policy 1, policy_version 1067081 (0.0006) [2023-12-27 13:02:56,285][20381] Updated weights for policy 1, policy_version 1067091 (0.0007) [2023-12-27 13:02:56,762][20347] Updated weights for policy 0, policy_version 1067950 (0.0007) [2023-12-27 13:02:56,820][20347] Updated weights for policy 0, policy_version 1067960 (0.0007) [2023-12-27 13:02:56,866][20347] Updated weights for policy 0, policy_version 1067970 (0.0008) [2023-12-27 13:02:57,022][20381] Updated weights for policy 1, policy_version 1067101 (0.0007) [2023-12-27 13:02:57,077][20381] Updated weights for policy 1, policy_version 1067111 (0.0008) [2023-12-27 13:02:57,126][20381] Updated weights for policy 1, policy_version 1067121 (0.0007) [2023-12-27 13:02:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 546660352. Throughput: 0: 10340.7, 1: 10244.5. Samples: 546648816. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:02:57,363][19170] Avg episode reward: [(0, '863.000'), (1, '667.480')] [2023-12-27 13:02:57,536][20347] Updated weights for policy 0, policy_version 1067980 (0.0009) [2023-12-27 13:02:57,587][20347] Updated weights for policy 0, policy_version 1067990 (0.0010) [2023-12-27 13:02:57,640][20347] Updated weights for policy 0, policy_version 1068000 (0.0010) [2023-12-27 13:02:57,836][20381] Updated weights for policy 1, policy_version 1067131 (0.0007) [2023-12-27 13:02:57,897][20381] Updated weights for policy 1, policy_version 1067141 (0.0007) [2023-12-27 13:02:57,955][20381] Updated weights for policy 1, policy_version 1067151 (0.0008) [2023-12-27 13:02:58,305][20347] Updated weights for policy 0, policy_version 1068010 (0.0010) [2023-12-27 13:02:58,369][20347] Updated weights for policy 0, policy_version 1068020 (0.0009) [2023-12-27 13:02:58,440][20347] Updated weights for policy 0, policy_version 1068030 (0.0010) [2023-12-27 13:02:58,503][20347] Updated weights for policy 0, policy_version 1068040 (0.0010) [2023-12-27 13:02:58,682][20381] Updated weights for policy 1, policy_version 1067161 (0.0009) [2023-12-27 13:02:58,747][20381] Updated weights for policy 1, policy_version 1067171 (0.0008) [2023-12-27 13:02:58,807][20381] Updated weights for policy 1, policy_version 1067181 (0.0008) [2023-12-27 13:02:58,878][20381] Updated weights for policy 1, policy_version 1067191 (0.0008) [2023-12-27 13:02:59,268][20347] Updated weights for policy 0, policy_version 1068051 (0.0008) [2023-12-27 13:02:59,333][20347] Updated weights for policy 0, policy_version 1068061 (0.0008) [2023-12-27 13:02:59,396][20347] Updated weights for policy 0, policy_version 1068071 (0.0007) [2023-12-27 13:02:59,634][20381] Updated weights for policy 1, policy_version 1067201 (0.0006) [2023-12-27 13:02:59,688][20381] Updated weights for policy 1, policy_version 1067211 (0.0005) [2023-12-27 13:02:59,737][20381] Updated weights for policy 1, policy_version 1067221 (0.0005) [2023-12-27 13:03:00,033][20347] Updated weights for policy 0, policy_version 1068081 (0.0009) [2023-12-27 13:03:00,082][20347] Updated weights for policy 0, policy_version 1068091 (0.0008) [2023-12-27 13:03:00,132][20347] Updated weights for policy 0, policy_version 1068101 (0.0006) [2023-12-27 13:03:00,429][20381] Updated weights for policy 1, policy_version 1067231 (0.0009) [2023-12-27 13:03:00,482][20381] Updated weights for policy 1, policy_version 1067241 (0.0009) [2023-12-27 13:03:00,540][20381] Updated weights for policy 1, policy_version 1067251 (0.0005) [2023-12-27 13:03:00,791][20347] Updated weights for policy 0, policy_version 1068111 (0.0007) [2023-12-27 13:03:00,850][20347] Updated weights for policy 0, policy_version 1068121 (0.0008) [2023-12-27 13:03:00,906][20347] Updated weights for policy 0, policy_version 1068131 (0.0008) [2023-12-27 13:03:01,147][20381] Updated weights for policy 1, policy_version 1067261 (0.0007) [2023-12-27 13:03:01,202][20381] Updated weights for policy 1, policy_version 1067271 (0.0008) [2023-12-27 13:03:01,255][20381] Updated weights for policy 1, policy_version 1067281 (0.0007) [2023-12-27 13:03:01,653][20347] Updated weights for policy 0, policy_version 1068141 (0.0008) [2023-12-27 13:03:01,719][20347] Updated weights for policy 0, policy_version 1068151 (0.0008) [2023-12-27 13:03:01,787][20347] Updated weights for policy 0, policy_version 1068161 (0.0007) [2023-12-27 13:03:01,936][20381] Updated weights for policy 1, policy_version 1067291 (0.0008) [2023-12-27 13:03:02,000][20381] Updated weights for policy 1, policy_version 1067301 (0.0006) [2023-12-27 13:03:02,061][20381] Updated weights for policy 1, policy_version 1067311 (0.0006) [2023-12-27 13:03:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 546758656. Throughput: 0: 10370.2, 1: 10218.4. Samples: 546770312. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:02,363][19170] Avg episode reward: [(0, '863.090'), (1, '736.090')] [2023-12-27 13:03:02,441][20347] Updated weights for policy 0, policy_version 1068171 (0.0007) [2023-12-27 13:03:02,504][20347] Updated weights for policy 0, policy_version 1068181 (0.0006) [2023-12-27 13:03:02,571][20347] Updated weights for policy 0, policy_version 1068191 (0.0006) [2023-12-27 13:03:02,705][20381] Updated weights for policy 1, policy_version 1067321 (0.0008) [2023-12-27 13:03:02,767][20381] Updated weights for policy 1, policy_version 1067331 (0.0008) [2023-12-27 13:03:02,819][20381] Updated weights for policy 1, policy_version 1067341 (0.0007) [2023-12-27 13:03:02,866][20381] Updated weights for policy 1, policy_version 1067351 (0.0009) [2023-12-27 13:03:03,164][20347] Updated weights for policy 0, policy_version 1068201 (0.0007) [2023-12-27 13:03:03,219][20347] Updated weights for policy 0, policy_version 1068211 (0.0006) [2023-12-27 13:03:03,269][20347] Updated weights for policy 0, policy_version 1068221 (0.0007) [2023-12-27 13:03:03,322][20347] Updated weights for policy 0, policy_version 1068231 (0.0007) [2023-12-27 13:03:03,496][20381] Updated weights for policy 1, policy_version 1067361 (0.0008) [2023-12-27 13:03:03,556][20381] Updated weights for policy 1, policy_version 1067371 (0.0006) [2023-12-27 13:03:03,611][20381] Updated weights for policy 1, policy_version 1067381 (0.0005) [2023-12-27 13:03:04,046][20347] Updated weights for policy 0, policy_version 1068241 (0.0008) [2023-12-27 13:03:04,098][20347] Updated weights for policy 0, policy_version 1068251 (0.0007) [2023-12-27 13:03:04,157][20347] Updated weights for policy 0, policy_version 1068261 (0.0007) [2023-12-27 13:03:04,312][20381] Updated weights for policy 1, policy_version 1067391 (0.0009) [2023-12-27 13:03:04,371][20381] Updated weights for policy 1, policy_version 1067401 (0.0011) [2023-12-27 13:03:04,433][20381] Updated weights for policy 1, policy_version 1067411 (0.0011) [2023-12-27 13:03:04,865][20347] Updated weights for policy 0, policy_version 1068271 (0.0006) [2023-12-27 13:03:04,920][20347] Updated weights for policy 0, policy_version 1068281 (0.0006) [2023-12-27 13:03:04,975][20347] Updated weights for policy 0, policy_version 1068291 (0.0007) [2023-12-27 13:03:05,009][20381] Updated weights for policy 1, policy_version 1067421 (0.0008) [2023-12-27 13:03:05,068][20381] Updated weights for policy 1, policy_version 1067431 (0.0007) [2023-12-27 13:03:05,126][20381] Updated weights for policy 1, policy_version 1067441 (0.0010) [2023-12-27 13:03:05,591][20347] Updated weights for policy 0, policy_version 1068301 (0.0007) [2023-12-27 13:03:05,646][20347] Updated weights for policy 0, policy_version 1068311 (0.0006) [2023-12-27 13:03:05,706][20347] Updated weights for policy 0, policy_version 1068321 (0.0006) [2023-12-27 13:03:05,713][20381] Updated weights for policy 1, policy_version 1067451 (0.0009) [2023-12-27 13:03:05,772][20381] Updated weights for policy 1, policy_version 1067461 (0.0008) [2023-12-27 13:03:05,825][20381] Updated weights for policy 1, policy_version 1067471 (0.0007) [2023-12-27 13:03:06,361][20347] Updated weights for policy 0, policy_version 1068331 (0.0006) [2023-12-27 13:03:06,421][20347] Updated weights for policy 0, policy_version 1068341 (0.0007) [2023-12-27 13:03:06,484][20347] Updated weights for policy 0, policy_version 1068351 (0.0008) [2023-12-27 13:03:06,540][20381] Updated weights for policy 1, policy_version 1067481 (0.0008) [2023-12-27 13:03:06,603][20381] Updated weights for policy 1, policy_version 1067491 (0.0007) [2023-12-27 13:03:06,661][20381] Updated weights for policy 1, policy_version 1067501 (0.0008) [2023-12-27 13:03:06,713][20381] Updated weights for policy 1, policy_version 1067511 (0.0008) [2023-12-27 13:03:07,192][20347] Updated weights for policy 0, policy_version 1068361 (0.0008) [2023-12-27 13:03:07,248][20347] Updated weights for policy 0, policy_version 1068371 (0.0007) [2023-12-27 13:03:07,307][20347] Updated weights for policy 0, policy_version 1068381 (0.0007) [2023-12-27 13:03:07,358][20347] Updated weights for policy 0, policy_version 1068391 (0.0007) [2023-12-27 13:03:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 546865152. Throughput: 0: 10352.5, 1: 10263.4. Samples: 546833304. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:07,363][19170] Avg episode reward: [(0, '931.730'), (1, '823.910')] [2023-12-27 13:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001068392_273555456.pth... [2023-12-27 13:03:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001067176_273244160.pth [2023-12-27 13:03:07,384][20381] Updated weights for policy 1, policy_version 1067521 (0.0007) [2023-12-27 13:03:07,435][20381] Updated weights for policy 1, policy_version 1067531 (0.0006) [2023-12-27 13:03:07,483][20381] Updated weights for policy 1, policy_version 1067541 (0.0005) [2023-12-27 13:03:07,496][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001067544_273317888.pth... [2023-12-27 13:03:07,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001066328_273006592.pth [2023-12-27 13:03:07,997][20347] Updated weights for policy 0, policy_version 1068401 (0.0010) [2023-12-27 13:03:08,050][20347] Updated weights for policy 0, policy_version 1068411 (0.0010) [2023-12-27 13:03:08,096][20381] Updated weights for policy 1, policy_version 1067551 (0.0009) [2023-12-27 13:03:08,107][20347] Updated weights for policy 0, policy_version 1068421 (0.0010) [2023-12-27 13:03:08,146][20381] Updated weights for policy 1, policy_version 1067561 (0.0010) [2023-12-27 13:03:08,209][20381] Updated weights for policy 1, policy_version 1067571 (0.0010) [2023-12-27 13:03:08,779][20347] Updated weights for policy 0, policy_version 1068431 (0.0010) [2023-12-27 13:03:08,829][20347] Updated weights for policy 0, policy_version 1068441 (0.0005) [2023-12-27 13:03:08,876][20347] Updated weights for policy 0, policy_version 1068451 (0.0005) [2023-12-27 13:03:08,898][20381] Updated weights for policy 1, policy_version 1067581 (0.0010) [2023-12-27 13:03:08,959][20381] Updated weights for policy 1, policy_version 1067591 (0.0008) [2023-12-27 13:03:09,027][20381] Updated weights for policy 1, policy_version 1067601 (0.0008) [2023-12-27 13:03:09,546][20347] Updated weights for policy 0, policy_version 1068461 (0.0005) [2023-12-27 13:03:09,604][20347] Updated weights for policy 0, policy_version 1068471 (0.0008) [2023-12-27 13:03:09,650][20347] Updated weights for policy 0, policy_version 1068481 (0.0007) [2023-12-27 13:03:09,714][20381] Updated weights for policy 1, policy_version 1067611 (0.0010) [2023-12-27 13:03:09,770][20381] Updated weights for policy 1, policy_version 1067621 (0.0011) [2023-12-27 13:03:09,834][20381] Updated weights for policy 1, policy_version 1067631 (0.0011) [2023-12-27 13:03:10,386][20347] Updated weights for policy 0, policy_version 1068491 (0.0006) [2023-12-27 13:03:10,447][20347] Updated weights for policy 0, policy_version 1068501 (0.0008) [2023-12-27 13:03:10,504][20347] Updated weights for policy 0, policy_version 1068511 (0.0007) [2023-12-27 13:03:10,558][20381] Updated weights for policy 1, policy_version 1067641 (0.0011) [2023-12-27 13:03:10,616][20381] Updated weights for policy 1, policy_version 1067651 (0.0011) [2023-12-27 13:03:10,674][20381] Updated weights for policy 1, policy_version 1067661 (0.0011) [2023-12-27 13:03:10,728][20381] Updated weights for policy 1, policy_version 1067671 (0.0011) [2023-12-27 13:03:11,173][20347] Updated weights for policy 0, policy_version 1068521 (0.0007) [2023-12-27 13:03:11,232][20347] Updated weights for policy 0, policy_version 1068531 (0.0007) [2023-12-27 13:03:11,292][20347] Updated weights for policy 0, policy_version 1068541 (0.0007) [2023-12-27 13:03:11,355][20347] Updated weights for policy 0, policy_version 1068551 (0.0008) [2023-12-27 13:03:11,443][20381] Updated weights for policy 1, policy_version 1067681 (0.0009) [2023-12-27 13:03:11,508][20381] Updated weights for policy 1, policy_version 1067691 (0.0006) [2023-12-27 13:03:11,573][20381] Updated weights for policy 1, policy_version 1067701 (0.0006) [2023-12-27 13:03:12,061][20347] Updated weights for policy 0, policy_version 1068561 (0.0006) [2023-12-27 13:03:12,108][20347] Updated weights for policy 0, policy_version 1068571 (0.0005) [2023-12-27 13:03:12,157][20347] Updated weights for policy 0, policy_version 1068581 (0.0010) [2023-12-27 13:03:12,242][20381] Updated weights for policy 1, policy_version 1067711 (0.0007) [2023-12-27 13:03:12,295][20381] Updated weights for policy 1, policy_version 1067721 (0.0008) [2023-12-27 13:03:12,355][20381] Updated weights for policy 1, policy_version 1067731 (0.0007) [2023-12-27 13:03:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 546963456. Throughput: 0: 10390.2, 1: 10309.4. Samples: 546956584. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:12,363][19170] Avg episode reward: [(0, '951.190'), (1, '775.060')] [2023-12-27 13:03:12,378][20292] KL-divergence is very high: 153.6768 [2023-12-27 13:03:12,823][20347] Updated weights for policy 0, policy_version 1068591 (0.0008) [2023-12-27 13:03:12,885][20347] Updated weights for policy 0, policy_version 1068601 (0.0007) [2023-12-27 13:03:12,944][20347] Updated weights for policy 0, policy_version 1068611 (0.0008) [2023-12-27 13:03:13,043][20292] KL-divergence is very high: 145.8404 [2023-12-27 13:03:13,058][20292] KL-divergence is very high: 104.1969 [2023-12-27 13:03:13,073][20381] Updated weights for policy 1, policy_version 1067741 (0.0007) [2023-12-27 13:03:13,129][20381] Updated weights for policy 1, policy_version 1067751 (0.0005) [2023-12-27 13:03:13,181][20381] Updated weights for policy 1, policy_version 1067761 (0.0005) [2023-12-27 13:03:13,533][20347] Updated weights for policy 0, policy_version 1068621 (0.0010) [2023-12-27 13:03:13,598][20347] Updated weights for policy 0, policy_version 1068631 (0.0011) [2023-12-27 13:03:13,657][20347] Updated weights for policy 0, policy_version 1068641 (0.0010) [2023-12-27 13:03:13,765][20381] Updated weights for policy 1, policy_version 1067771 (0.0007) [2023-12-27 13:03:13,822][20381] Updated weights for policy 1, policy_version 1067781 (0.0007) [2023-12-27 13:03:13,875][20381] Updated weights for policy 1, policy_version 1067791 (0.0007) [2023-12-27 13:03:14,271][20347] Updated weights for policy 0, policy_version 1068651 (0.0009) [2023-12-27 13:03:14,330][20347] Updated weights for policy 0, policy_version 1068661 (0.0007) [2023-12-27 13:03:14,387][20347] Updated weights for policy 0, policy_version 1068671 (0.0007) [2023-12-27 13:03:14,593][20381] Updated weights for policy 1, policy_version 1067801 (0.0010) [2023-12-27 13:03:14,646][20381] Updated weights for policy 1, policy_version 1067811 (0.0010) [2023-12-27 13:03:14,705][20381] Updated weights for policy 1, policy_version 1067821 (0.0010) [2023-12-27 13:03:14,767][20381] Updated weights for policy 1, policy_version 1067831 (0.0008) [2023-12-27 13:03:15,078][20347] Updated weights for policy 0, policy_version 1068681 (0.0007) [2023-12-27 13:03:15,137][20347] Updated weights for policy 0, policy_version 1068691 (0.0007) [2023-12-27 13:03:15,190][20347] Updated weights for policy 0, policy_version 1068701 (0.0008) [2023-12-27 13:03:15,248][20347] Updated weights for policy 0, policy_version 1068711 (0.0008) [2023-12-27 13:03:15,412][20381] Updated weights for policy 1, policy_version 1067841 (0.0006) [2023-12-27 13:03:15,463][20381] Updated weights for policy 1, policy_version 1067851 (0.0006) [2023-12-27 13:03:15,516][20381] Updated weights for policy 1, policy_version 1067861 (0.0005) [2023-12-27 13:03:15,931][20347] Updated weights for policy 0, policy_version 1068721 (0.0008) [2023-12-27 13:03:15,975][20347] Updated weights for policy 0, policy_version 1068731 (0.0008) [2023-12-27 13:03:16,000][20291] KL-divergence is very high: 264.6385 [2023-12-27 13:03:16,026][20347] Updated weights for policy 0, policy_version 1068741 (0.0007) [2023-12-27 13:03:16,119][20381] Updated weights for policy 1, policy_version 1067871 (0.0005) [2023-12-27 13:03:16,178][20381] Updated weights for policy 1, policy_version 1067881 (0.0006) [2023-12-27 13:03:16,239][20381] Updated weights for policy 1, policy_version 1067891 (0.0005) [2023-12-27 13:03:16,676][20347] Updated weights for policy 0, policy_version 1068751 (0.0006) [2023-12-27 13:03:16,735][20347] Updated weights for policy 0, policy_version 1068761 (0.0008) [2023-12-27 13:03:16,786][20347] Updated weights for policy 0, policy_version 1068771 (0.0008) [2023-12-27 13:03:16,842][20381] Updated weights for policy 1, policy_version 1067901 (0.0006) [2023-12-27 13:03:16,884][20381] Updated weights for policy 1, policy_version 1067911 (0.0007) [2023-12-27 13:03:16,927][20381] Updated weights for policy 1, policy_version 1067921 (0.0007) [2023-12-27 13:03:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 547069952. Throughput: 0: 10382.3, 1: 10344.5. Samples: 547081808. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:17,363][19170] Avg episode reward: [(0, '960.890'), (1, '804.410')] [2023-12-27 13:03:17,447][20347] Updated weights for policy 0, policy_version 1068781 (0.0008) [2023-12-27 13:03:17,493][20347] Updated weights for policy 0, policy_version 1068791 (0.0005) [2023-12-27 13:03:17,541][20347] Updated weights for policy 0, policy_version 1068801 (0.0005) [2023-12-27 13:03:17,636][20381] Updated weights for policy 1, policy_version 1067931 (0.0007) [2023-12-27 13:03:17,690][20381] Updated weights for policy 1, policy_version 1067941 (0.0009) [2023-12-27 13:03:17,748][20381] Updated weights for policy 1, policy_version 1067951 (0.0008) [2023-12-27 13:03:18,130][20347] Updated weights for policy 0, policy_version 1068811 (0.0005) [2023-12-27 13:03:18,198][20347] Updated weights for policy 0, policy_version 1068821 (0.0005) [2023-12-27 13:03:18,264][20347] Updated weights for policy 0, policy_version 1068831 (0.0008) [2023-12-27 13:03:18,457][20381] Updated weights for policy 1, policy_version 1067961 (0.0007) [2023-12-27 13:03:18,510][20381] Updated weights for policy 1, policy_version 1067971 (0.0008) [2023-12-27 13:03:18,566][20381] Updated weights for policy 1, policy_version 1067981 (0.0008) [2023-12-27 13:03:18,622][20381] Updated weights for policy 1, policy_version 1067991 (0.0008) [2023-12-27 13:03:18,934][20347] Updated weights for policy 0, policy_version 1068841 (0.0008) [2023-12-27 13:03:18,985][20347] Updated weights for policy 0, policy_version 1068851 (0.0008) [2023-12-27 13:03:19,034][20347] Updated weights for policy 0, policy_version 1068861 (0.0008) [2023-12-27 13:03:19,088][20347] Updated weights for policy 0, policy_version 1068871 (0.0007) [2023-12-27 13:03:19,344][20381] Updated weights for policy 1, policy_version 1068001 (0.0008) [2023-12-27 13:03:19,412][20381] Updated weights for policy 1, policy_version 1068011 (0.0009) [2023-12-27 13:03:19,469][20381] Updated weights for policy 1, policy_version 1068021 (0.0006) [2023-12-27 13:03:19,867][20347] Updated weights for policy 0, policy_version 1068881 (0.0008) [2023-12-27 13:03:19,932][20347] Updated weights for policy 0, policy_version 1068891 (0.0007) [2023-12-27 13:03:19,994][20347] Updated weights for policy 0, policy_version 1068901 (0.0009) [2023-12-27 13:03:20,210][20381] Updated weights for policy 1, policy_version 1068031 (0.0007) [2023-12-27 13:03:20,271][20381] Updated weights for policy 1, policy_version 1068041 (0.0006) [2023-12-27 13:03:20,330][20381] Updated weights for policy 1, policy_version 1068051 (0.0006) [2023-12-27 13:03:20,724][20347] Updated weights for policy 0, policy_version 1068911 (0.0008) [2023-12-27 13:03:20,784][20347] Updated weights for policy 0, policy_version 1068921 (0.0008) [2023-12-27 13:03:20,843][20347] Updated weights for policy 0, policy_version 1068931 (0.0007) [2023-12-27 13:03:21,020][20381] Updated weights for policy 1, policy_version 1068061 (0.0009) [2023-12-27 13:03:21,083][20381] Updated weights for policy 1, policy_version 1068071 (0.0011) [2023-12-27 13:03:21,147][20381] Updated weights for policy 1, policy_version 1068081 (0.0011) [2023-12-27 13:03:21,546][20347] Updated weights for policy 0, policy_version 1068941 (0.0006) [2023-12-27 13:03:21,601][20347] Updated weights for policy 0, policy_version 1068951 (0.0005) [2023-12-27 13:03:21,663][20347] Updated weights for policy 0, policy_version 1068961 (0.0008) [2023-12-27 13:03:21,989][20381] Updated weights for policy 1, policy_version 1068091 (0.0011) [2023-12-27 13:03:22,049][20381] Updated weights for policy 1, policy_version 1068101 (0.0011) [2023-12-27 13:03:22,111][20381] Updated weights for policy 1, policy_version 1068111 (0.0008) [2023-12-27 13:03:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 547168256. Throughput: 0: 10335.7, 1: 10281.6. Samples: 547141676. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:22,363][19170] Avg episode reward: [(0, '902.190'), (1, '833.650')] [2023-12-27 13:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001068120_273465344.pth... [2023-12-27 13:03:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001066936_273162240.pth [2023-12-27 13:03:22,387][20347] Updated weights for policy 0, policy_version 1068971 (0.0007) [2023-12-27 13:03:22,436][20347] Updated weights for policy 0, policy_version 1068981 (0.0007) [2023-12-27 13:03:22,489][20347] Updated weights for policy 0, policy_version 1068991 (0.0006) [2023-12-27 13:03:22,542][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001069000_273711104.pth... [2023-12-27 13:03:22,546][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001067752_273391616.pth [2023-12-27 13:03:22,789][20381] Updated weights for policy 1, policy_version 1068121 (0.0007) [2023-12-27 13:03:22,843][20381] Updated weights for policy 1, policy_version 1068131 (0.0011) [2023-12-27 13:03:22,893][20381] Updated weights for policy 1, policy_version 1068141 (0.0011) [2023-12-27 13:03:22,953][20381] Updated weights for policy 1, policy_version 1068151 (0.0011) [2023-12-27 13:03:23,125][20347] Updated weights for policy 0, policy_version 1069001 (0.0006) [2023-12-27 13:03:23,192][20347] Updated weights for policy 0, policy_version 1069011 (0.0011) [2023-12-27 13:03:23,261][20347] Updated weights for policy 0, policy_version 1069021 (0.0011) [2023-12-27 13:03:23,323][20347] Updated weights for policy 0, policy_version 1069031 (0.0010) [2023-12-27 13:03:23,671][20381] Updated weights for policy 1, policy_version 1068161 (0.0011) [2023-12-27 13:03:23,726][20381] Updated weights for policy 1, policy_version 1068171 (0.0011) [2023-12-27 13:03:23,784][20381] Updated weights for policy 1, policy_version 1068181 (0.0011) [2023-12-27 13:03:23,989][20347] Updated weights for policy 0, policy_version 1069041 (0.0008) [2023-12-27 13:03:24,016][20291] KL-divergence is very high: 414.9871 [2023-12-27 13:03:24,042][20347] Updated weights for policy 0, policy_version 1069051 (0.0008) [2023-12-27 13:03:24,061][20291] KL-divergence is very high: 280.7126 [2023-12-27 13:03:24,102][20347] Updated weights for policy 0, policy_version 1069061 (0.0005) [2023-12-27 13:03:24,472][20381] Updated weights for policy 1, policy_version 1068191 (0.0011) [2023-12-27 13:03:24,533][20381] Updated weights for policy 1, policy_version 1068201 (0.0011) [2023-12-27 13:03:24,591][20381] Updated weights for policy 1, policy_version 1068211 (0.0007) [2023-12-27 13:03:24,685][20291] KL-divergence is very high: 220.4418 [2023-12-27 13:03:24,690][20291] KL-divergence is very high: 119.1456 [2023-12-27 13:03:24,699][20291] KL-divergence is very high: 206.5964 [2023-12-27 13:03:24,708][20347] Updated weights for policy 0, policy_version 1069071 (0.0006) [2023-12-27 13:03:24,737][20291] KL-divergence is very high: 104.6890 [2023-12-27 13:03:24,755][20347] Updated weights for policy 0, policy_version 1069081 (0.0006) [2023-12-27 13:03:24,778][20291] KL-divergence is very high: 130.8828 [2023-12-27 13:03:24,813][20347] Updated weights for policy 0, policy_version 1069091 (0.0005) [2023-12-27 13:03:24,824][20291] KL-divergence is very high: 111.8078 [2023-12-27 13:03:25,181][20381] Updated weights for policy 1, policy_version 1068221 (0.0007) [2023-12-27 13:03:25,236][20381] Updated weights for policy 1, policy_version 1068231 (0.0008) [2023-12-27 13:03:25,293][20381] Updated weights for policy 1, policy_version 1068241 (0.0007) [2023-12-27 13:03:25,499][20347] Updated weights for policy 0, policy_version 1069101 (0.0007) [2023-12-27 13:03:25,549][20347] Updated weights for policy 0, policy_version 1069111 (0.0005) [2023-12-27 13:03:25,606][20347] Updated weights for policy 0, policy_version 1069121 (0.0005) [2023-12-27 13:03:26,001][20381] Updated weights for policy 1, policy_version 1068251 (0.0007) [2023-12-27 13:03:26,070][20381] Updated weights for policy 1, policy_version 1068261 (0.0007) [2023-12-27 13:03:26,126][20381] Updated weights for policy 1, policy_version 1068271 (0.0008) [2023-12-27 13:03:26,136][20347] Updated weights for policy 0, policy_version 1069131 (0.0006) [2023-12-27 13:03:26,181][20347] Updated weights for policy 0, policy_version 1069141 (0.0006) [2023-12-27 13:03:26,235][20347] Updated weights for policy 0, policy_version 1069151 (0.0007) [2023-12-27 13:03:26,728][20381] Updated weights for policy 1, policy_version 1068281 (0.0007) [2023-12-27 13:03:26,783][20381] Updated weights for policy 1, policy_version 1068291 (0.0005) [2023-12-27 13:03:26,845][20381] Updated weights for policy 1, policy_version 1068301 (0.0010) [2023-12-27 13:03:26,900][20381] Updated weights for policy 1, policy_version 1068311 (0.0010) [2023-12-27 13:03:26,907][20347] Updated weights for policy 0, policy_version 1069161 (0.0008) [2023-12-27 13:03:26,957][20347] Updated weights for policy 0, policy_version 1069171 (0.0007) [2023-12-27 13:03:27,013][20347] Updated weights for policy 0, policy_version 1069181 (0.0007) [2023-12-27 13:03:27,065][20347] Updated weights for policy 0, policy_version 1069191 (0.0006) [2023-12-27 13:03:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 547274752. Throughput: 0: 10388.6, 1: 10254.6. Samples: 547263520. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-12-27 13:03:27,363][19170] Avg episode reward: [(0, '872.670'), (1, '765.410')] [2023-12-27 13:03:27,521][20381] Updated weights for policy 1, policy_version 1068321 (0.0010) [2023-12-27 13:03:27,569][20381] Updated weights for policy 1, policy_version 1068331 (0.0010) [2023-12-27 13:03:27,617][20381] Updated weights for policy 1, policy_version 1068341 (0.0010) [2023-12-27 13:03:27,699][20347] Updated weights for policy 0, policy_version 1069201 (0.0005) [2023-12-27 13:03:27,747][20347] Updated weights for policy 0, policy_version 1069211 (0.0005) [2023-12-27 13:03:27,792][20347] Updated weights for policy 0, policy_version 1069221 (0.0005) [2023-12-27 13:03:28,280][20381] Updated weights for policy 1, policy_version 1068351 (0.0010) [2023-12-27 13:03:28,329][20381] Updated weights for policy 1, policy_version 1068361 (0.0010) [2023-12-27 13:03:28,357][20347] Updated weights for policy 0, policy_version 1069231 (0.0007) [2023-12-27 13:03:28,385][20381] Updated weights for policy 1, policy_version 1068371 (0.0010) [2023-12-27 13:03:28,416][20347] Updated weights for policy 0, policy_version 1069241 (0.0006) [2023-12-27 13:03:28,470][20347] Updated weights for policy 0, policy_version 1069251 (0.0005) [2023-12-27 13:03:29,015][20381] Updated weights for policy 1, policy_version 1068381 (0.0010) [2023-12-27 13:03:29,064][20381] Updated weights for policy 1, policy_version 1068391 (0.0005) [2023-12-27 13:03:29,116][20381] Updated weights for policy 1, policy_version 1068401 (0.0010) [2023-12-27 13:03:29,130][20347] Updated weights for policy 0, policy_version 1069261 (0.0005) [2023-12-27 13:03:29,178][20347] Updated weights for policy 0, policy_version 1069271 (0.0005) [2023-12-27 13:03:29,238][20347] Updated weights for policy 0, policy_version 1069281 (0.0006) [2023-12-27 13:03:29,767][20381] Updated weights for policy 1, policy_version 1068411 (0.0010) [2023-12-27 13:03:29,815][20381] Updated weights for policy 1, policy_version 1068421 (0.0008) [2023-12-27 13:03:29,875][20381] Updated weights for policy 1, policy_version 1068431 (0.0008) [2023-12-27 13:03:29,948][20347] Updated weights for policy 0, policy_version 1069291 (0.0007) [2023-12-27 13:03:30,004][20347] Updated weights for policy 0, policy_version 1069301 (0.0007) [2023-12-27 13:03:30,056][20347] Updated weights for policy 0, policy_version 1069311 (0.0008) [2023-12-27 13:03:30,597][20381] Updated weights for policy 1, policy_version 1068441 (0.0008) [2023-12-27 13:03:30,660][20381] Updated weights for policy 1, policy_version 1068451 (0.0008) [2023-12-27 13:03:30,723][20381] Updated weights for policy 1, policy_version 1068461 (0.0008) [2023-12-27 13:03:30,725][20347] Updated weights for policy 0, policy_version 1069321 (0.0007) [2023-12-27 13:03:30,770][20347] Updated weights for policy 0, policy_version 1069331 (0.0007) [2023-12-27 13:03:30,785][20381] Updated weights for policy 1, policy_version 1068471 (0.0006) [2023-12-27 13:03:30,815][20347] Updated weights for policy 0, policy_version 1069341 (0.0008) [2023-12-27 13:03:30,860][20347] Updated weights for policy 0, policy_version 1069351 (0.0008) [2023-12-27 13:03:31,452][20381] Updated weights for policy 1, policy_version 1068481 (0.0008) [2023-12-27 13:03:31,507][20381] Updated weights for policy 1, policy_version 1068491 (0.0007) [2023-12-27 13:03:31,556][20347] Updated weights for policy 0, policy_version 1069361 (0.0006) [2023-12-27 13:03:31,559][20381] Updated weights for policy 1, policy_version 1068501 (0.0008) [2023-12-27 13:03:31,615][20347] Updated weights for policy 0, policy_version 1069371 (0.0007) [2023-12-27 13:03:31,673][20347] Updated weights for policy 0, policy_version 1069381 (0.0005) [2023-12-27 13:03:32,230][20381] Updated weights for policy 1, policy_version 1068511 (0.0006) [2023-12-27 13:03:32,293][20381] Updated weights for policy 1, policy_version 1068521 (0.0007) [2023-12-27 13:03:32,360][20381] Updated weights for policy 1, policy_version 1068531 (0.0007) [2023-12-27 13:03:32,361][20347] Updated weights for policy 0, policy_version 1069391 (0.0007) [2023-12-27 13:03:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 547373056. Throughput: 0: 10392.8, 1: 10294.2. Samples: 547391260. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:32,363][19170] Avg episode reward: [(0, '853.140'), (1, '765.420')] [2023-12-27 13:03:32,417][20347] Updated weights for policy 0, policy_version 1069401 (0.0008) [2023-12-27 13:03:32,471][20347] Updated weights for policy 0, policy_version 1069411 (0.0008) [2023-12-27 13:03:32,895][20381] Updated weights for policy 1, policy_version 1068541 (0.0008) [2023-12-27 13:03:32,951][20381] Updated weights for policy 1, policy_version 1068551 (0.0008) [2023-12-27 13:03:33,006][20381] Updated weights for policy 1, policy_version 1068561 (0.0008) [2023-12-27 13:03:33,208][20347] Updated weights for policy 0, policy_version 1069421 (0.0008) [2023-12-27 13:03:33,254][20347] Updated weights for policy 0, policy_version 1069431 (0.0007) [2023-12-27 13:03:33,306][20347] Updated weights for policy 0, policy_version 1069441 (0.0007) [2023-12-27 13:03:33,669][20381] Updated weights for policy 1, policy_version 1068571 (0.0006) [2023-12-27 13:03:33,717][20381] Updated weights for policy 1, policy_version 1068581 (0.0005) [2023-12-27 13:03:33,775][20381] Updated weights for policy 1, policy_version 1068591 (0.0008) [2023-12-27 13:03:33,904][20347] Updated weights for policy 0, policy_version 1069451 (0.0006) [2023-12-27 13:03:33,960][20347] Updated weights for policy 0, policy_version 1069461 (0.0005) [2023-12-27 13:03:34,020][20347] Updated weights for policy 0, policy_version 1069471 (0.0005) [2023-12-27 13:03:34,400][20381] Updated weights for policy 1, policy_version 1068601 (0.0007) [2023-12-27 13:03:34,459][20381] Updated weights for policy 1, policy_version 1068611 (0.0006) [2023-12-27 13:03:34,524][20381] Updated weights for policy 1, policy_version 1068621 (0.0008) [2023-12-27 13:03:34,588][20381] Updated weights for policy 1, policy_version 1068631 (0.0011) [2023-12-27 13:03:34,668][20347] Updated weights for policy 0, policy_version 1069481 (0.0009) [2023-12-27 13:03:34,731][20347] Updated weights for policy 0, policy_version 1069491 (0.0010) [2023-12-27 13:03:34,779][20347] Updated weights for policy 0, policy_version 1069501 (0.0010) [2023-12-27 13:03:34,828][20347] Updated weights for policy 0, policy_version 1069511 (0.0008) [2023-12-27 13:03:35,246][20381] Updated weights for policy 1, policy_version 1068641 (0.0010) [2023-12-27 13:03:35,304][20381] Updated weights for policy 1, policy_version 1068651 (0.0010) [2023-12-27 13:03:35,350][20381] Updated weights for policy 1, policy_version 1068661 (0.0010) [2023-12-27 13:03:35,436][20347] Updated weights for policy 0, policy_version 1069521 (0.0006) [2023-12-27 13:03:35,492][20347] Updated weights for policy 0, policy_version 1069531 (0.0005) [2023-12-27 13:03:35,548][20347] Updated weights for policy 0, policy_version 1069541 (0.0005) [2023-12-27 13:03:36,106][20381] Updated weights for policy 1, policy_version 1068671 (0.0011) [2023-12-27 13:03:36,139][20347] Updated weights for policy 0, policy_version 1069551 (0.0007) [2023-12-27 13:03:36,168][20381] Updated weights for policy 1, policy_version 1068681 (0.0011) [2023-12-27 13:03:36,193][20347] Updated weights for policy 0, policy_version 1069561 (0.0005) [2023-12-27 13:03:36,225][20381] Updated weights for policy 1, policy_version 1068691 (0.0011) [2023-12-27 13:03:36,253][20347] Updated weights for policy 0, policy_version 1069571 (0.0006) [2023-12-27 13:03:36,921][20381] Updated weights for policy 1, policy_version 1068701 (0.0011) [2023-12-27 13:03:36,923][20347] Updated weights for policy 0, policy_version 1069581 (0.0006) [2023-12-27 13:03:36,976][20381] Updated weights for policy 1, policy_version 1068711 (0.0009) [2023-12-27 13:03:36,981][20347] Updated weights for policy 0, policy_version 1069591 (0.0010) [2023-12-27 13:03:37,033][20381] Updated weights for policy 1, policy_version 1068721 (0.0010) [2023-12-27 13:03:37,038][20347] Updated weights for policy 0, policy_version 1069601 (0.0010) [2023-12-27 13:03:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 547487744. Throughput: 0: 10394.3, 1: 10288.6. Samples: 547454872. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:37,363][19170] Avg episode reward: [(0, '902.130'), (1, '784.670')] [2023-12-27 13:03:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001069608_273866752.pth... [2023-12-27 13:03:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001068728_273620992.pth... [2023-12-27 13:03:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001068392_273555456.pth [2023-12-27 13:03:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001067544_273317888.pth [2023-12-27 13:03:37,672][20347] Updated weights for policy 0, policy_version 1069611 (0.0010) [2023-12-27 13:03:37,674][20381] Updated weights for policy 1, policy_version 1068731 (0.0010) [2023-12-27 13:03:37,726][20347] Updated weights for policy 0, policy_version 1069621 (0.0006) [2023-12-27 13:03:37,735][20381] Updated weights for policy 1, policy_version 1068741 (0.0007) [2023-12-27 13:03:37,787][20347] Updated weights for policy 0, policy_version 1069631 (0.0007) [2023-12-27 13:03:37,800][20381] Updated weights for policy 1, policy_version 1068751 (0.0006) [2023-12-27 13:03:38,394][20347] Updated weights for policy 0, policy_version 1069641 (0.0006) [2023-12-27 13:03:38,452][20381] Updated weights for policy 1, policy_version 1068761 (0.0006) [2023-12-27 13:03:38,455][20347] Updated weights for policy 0, policy_version 1069651 (0.0010) [2023-12-27 13:03:38,505][20381] Updated weights for policy 1, policy_version 1068771 (0.0006) [2023-12-27 13:03:38,513][20347] Updated weights for policy 0, policy_version 1069661 (0.0011) [2023-12-27 13:03:38,557][20381] Updated weights for policy 1, policy_version 1068781 (0.0005) [2023-12-27 13:03:38,571][20347] Updated weights for policy 0, policy_version 1069671 (0.0010) [2023-12-27 13:03:38,614][20381] Updated weights for policy 1, policy_version 1068791 (0.0005) [2023-12-27 13:03:39,185][20347] Updated weights for policy 0, policy_version 1069681 (0.0006) [2023-12-27 13:03:39,257][20347] Updated weights for policy 0, policy_version 1069691 (0.0008) [2023-12-27 13:03:39,301][20381] Updated weights for policy 1, policy_version 1068801 (0.0006) [2023-12-27 13:03:39,311][20347] Updated weights for policy 0, policy_version 1069701 (0.0006) [2023-12-27 13:03:39,374][20381] Updated weights for policy 1, policy_version 1068811 (0.0007) [2023-12-27 13:03:39,449][20381] Updated weights for policy 1, policy_version 1068821 (0.0008) [2023-12-27 13:03:39,959][20347] Updated weights for policy 0, policy_version 1069711 (0.0007) [2023-12-27 13:03:40,024][20347] Updated weights for policy 0, policy_version 1069721 (0.0011) [2023-12-27 13:03:40,085][20347] Updated weights for policy 0, policy_version 1069731 (0.0011) [2023-12-27 13:03:40,163][20381] Updated weights for policy 1, policy_version 1068831 (0.0007) [2023-12-27 13:03:40,227][20381] Updated weights for policy 1, policy_version 1068841 (0.0007) [2023-12-27 13:03:40,285][20381] Updated weights for policy 1, policy_version 1068851 (0.0008) [2023-12-27 13:03:40,708][20347] Updated weights for policy 0, policy_version 1069741 (0.0010) [2023-12-27 13:03:40,764][20347] Updated weights for policy 0, policy_version 1069751 (0.0009) [2023-12-27 13:03:40,809][20347] Updated weights for policy 0, policy_version 1069761 (0.0010) [2023-12-27 13:03:40,952][20381] Updated weights for policy 1, policy_version 1068861 (0.0007) [2023-12-27 13:03:41,012][20381] Updated weights for policy 1, policy_version 1068871 (0.0007) [2023-12-27 13:03:41,081][20381] Updated weights for policy 1, policy_version 1068881 (0.0009) [2023-12-27 13:03:41,574][20347] Updated weights for policy 0, policy_version 1069771 (0.0009) [2023-12-27 13:03:41,642][20347] Updated weights for policy 0, policy_version 1069781 (0.0008) [2023-12-27 13:03:41,669][20291] KL-divergence is very high: 119.7323 [2023-12-27 13:03:41,709][20347] Updated weights for policy 0, policy_version 1069791 (0.0008) [2023-12-27 13:03:41,726][20291] KL-divergence is very high: 199.6082 [2023-12-27 13:03:41,790][20381] Updated weights for policy 1, policy_version 1068891 (0.0008) [2023-12-27 13:03:41,859][20381] Updated weights for policy 1, policy_version 1068901 (0.0006) [2023-12-27 13:03:41,928][20381] Updated weights for policy 1, policy_version 1068911 (0.0009) [2023-12-27 13:03:42,350][20347] Updated weights for policy 0, policy_version 1069801 (0.0007) [2023-12-27 13:03:42,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 547586048. Throughput: 0: 10461.5, 1: 10236.1. Samples: 547580212. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:42,363][19170] Avg episode reward: [(0, '921.920'), (1, '755.160')] [2023-12-27 13:03:42,372][20291] KL-divergence is very high: 160.6387 [2023-12-27 13:03:42,410][20347] Updated weights for policy 0, policy_version 1069811 (0.0008) [2023-12-27 13:03:42,459][20347] Updated weights for policy 0, policy_version 1069821 (0.0008) [2023-12-27 13:03:42,515][20347] Updated weights for policy 0, policy_version 1069831 (0.0008) [2023-12-27 13:03:42,629][20381] Updated weights for policy 1, policy_version 1068921 (0.0008) [2023-12-27 13:03:42,694][20381] Updated weights for policy 1, policy_version 1068931 (0.0006) [2023-12-27 13:03:42,756][20381] Updated weights for policy 1, policy_version 1068941 (0.0005) [2023-12-27 13:03:42,814][20381] Updated weights for policy 1, policy_version 1068951 (0.0008) [2023-12-27 13:03:43,084][20291] KL-divergence is very high: 106.5206 [2023-12-27 13:03:43,142][20347] Updated weights for policy 0, policy_version 1069841 (0.0006) [2023-12-27 13:03:43,194][20347] Updated weights for policy 0, policy_version 1069851 (0.0007) [2023-12-27 13:03:43,243][20347] Updated weights for policy 0, policy_version 1069861 (0.0005) [2023-12-27 13:03:43,417][20381] Updated weights for policy 1, policy_version 1068961 (0.0007) [2023-12-27 13:03:43,461][20381] Updated weights for policy 1, policy_version 1068971 (0.0007) [2023-12-27 13:03:43,519][20381] Updated weights for policy 1, policy_version 1068981 (0.0008) [2023-12-27 13:03:43,839][20347] Updated weights for policy 0, policy_version 1069871 (0.0008) [2023-12-27 13:03:43,893][20347] Updated weights for policy 0, policy_version 1069881 (0.0010) [2023-12-27 13:03:43,950][20347] Updated weights for policy 0, policy_version 1069891 (0.0010) [2023-12-27 13:03:44,176][20381] Updated weights for policy 1, policy_version 1068991 (0.0008) [2023-12-27 13:03:44,228][20381] Updated weights for policy 1, policy_version 1069001 (0.0008) [2023-12-27 13:03:44,279][20381] Updated weights for policy 1, policy_version 1069011 (0.0007) [2023-12-27 13:03:44,579][20347] Updated weights for policy 0, policy_version 1069901 (0.0008) [2023-12-27 13:03:44,640][20347] Updated weights for policy 0, policy_version 1069911 (0.0005) [2023-12-27 13:03:44,697][20347] Updated weights for policy 0, policy_version 1069921 (0.0005) [2023-12-27 13:03:45,059][20381] Updated weights for policy 1, policy_version 1069021 (0.0008) [2023-12-27 13:03:45,122][20381] Updated weights for policy 1, policy_version 1069031 (0.0008) [2023-12-27 13:03:45,184][20381] Updated weights for policy 1, policy_version 1069041 (0.0008) [2023-12-27 13:03:45,297][20347] Updated weights for policy 0, policy_version 1069931 (0.0006) [2023-12-27 13:03:45,354][20347] Updated weights for policy 0, policy_version 1069941 (0.0007) [2023-12-27 13:03:45,415][20347] Updated weights for policy 0, policy_version 1069951 (0.0007) [2023-12-27 13:03:45,906][20381] Updated weights for policy 1, policy_version 1069051 (0.0008) [2023-12-27 13:03:45,960][20381] Updated weights for policy 1, policy_version 1069061 (0.0008) [2023-12-27 13:03:46,016][20347] Updated weights for policy 0, policy_version 1069961 (0.0007) [2023-12-27 13:03:46,019][20381] Updated weights for policy 1, policy_version 1069071 (0.0007) [2023-12-27 13:03:46,064][20347] Updated weights for policy 0, policy_version 1069971 (0.0005) [2023-12-27 13:03:46,119][20347] Updated weights for policy 0, policy_version 1069981 (0.0006) [2023-12-27 13:03:46,169][20347] Updated weights for policy 0, policy_version 1069991 (0.0007) [2023-12-27 13:03:46,631][20381] Updated weights for policy 1, policy_version 1069081 (0.0006) [2023-12-27 13:03:46,701][20381] Updated weights for policy 1, policy_version 1069091 (0.0008) [2023-12-27 13:03:46,763][20381] Updated weights for policy 1, policy_version 1069101 (0.0008) [2023-12-27 13:03:46,780][20347] Updated weights for policy 0, policy_version 1070001 (0.0006) [2023-12-27 13:03:46,816][20381] Updated weights for policy 1, policy_version 1069111 (0.0008) [2023-12-27 13:03:46,831][20347] Updated weights for policy 0, policy_version 1070011 (0.0005) [2023-12-27 13:03:46,880][20347] Updated weights for policy 0, policy_version 1070021 (0.0005) [2023-12-27 13:03:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 547692544. Throughput: 0: 10544.4, 1: 10259.8. Samples: 547706500. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:47,363][19170] Avg episode reward: [(0, '922.230'), (1, '696.680')] [2023-12-27 13:03:47,521][20381] Updated weights for policy 1, policy_version 1069121 (0.0007) [2023-12-27 13:03:47,537][20347] Updated weights for policy 0, policy_version 1070031 (0.0007) [2023-12-27 13:03:47,581][20381] Updated weights for policy 1, policy_version 1069131 (0.0006) [2023-12-27 13:03:47,592][20347] Updated weights for policy 0, policy_version 1070041 (0.0005) [2023-12-27 13:03:47,641][20347] Updated weights for policy 0, policy_version 1070051 (0.0005) [2023-12-27 13:03:47,641][20381] Updated weights for policy 1, policy_version 1069141 (0.0007) [2023-12-27 13:03:48,211][20347] Updated weights for policy 0, policy_version 1070061 (0.0007) [2023-12-27 13:03:48,275][20347] Updated weights for policy 0, policy_version 1070071 (0.0008) [2023-12-27 13:03:48,340][20347] Updated weights for policy 0, policy_version 1070081 (0.0008) [2023-12-27 13:03:48,367][20381] Updated weights for policy 1, policy_version 1069151 (0.0007) [2023-12-27 13:03:48,416][20381] Updated weights for policy 1, policy_version 1069161 (0.0007) [2023-12-27 13:03:48,466][20381] Updated weights for policy 1, policy_version 1069171 (0.0009) [2023-12-27 13:03:48,928][20347] Updated weights for policy 0, policy_version 1070091 (0.0007) [2023-12-27 13:03:48,980][20347] Updated weights for policy 0, policy_version 1070101 (0.0008) [2023-12-27 13:03:49,034][20347] Updated weights for policy 0, policy_version 1070111 (0.0007) [2023-12-27 13:03:49,202][20381] Updated weights for policy 1, policy_version 1069181 (0.0009) [2023-12-27 13:03:49,263][20381] Updated weights for policy 1, policy_version 1069191 (0.0009) [2023-12-27 13:03:49,314][20381] Updated weights for policy 1, policy_version 1069201 (0.0008) [2023-12-27 13:03:49,722][20347] Updated weights for policy 0, policy_version 1070121 (0.0008) [2023-12-27 13:03:49,790][20347] Updated weights for policy 0, policy_version 1070131 (0.0007) [2023-12-27 13:03:49,854][20347] Updated weights for policy 0, policy_version 1070141 (0.0008) [2023-12-27 13:03:49,906][20347] Updated weights for policy 0, policy_version 1070151 (0.0008) [2023-12-27 13:03:50,033][20381] Updated weights for policy 1, policy_version 1069211 (0.0008) [2023-12-27 13:03:50,097][20381] Updated weights for policy 1, policy_version 1069221 (0.0009) [2023-12-27 13:03:50,165][20381] Updated weights for policy 1, policy_version 1069231 (0.0006) [2023-12-27 13:03:50,634][20347] Updated weights for policy 0, policy_version 1070161 (0.0008) [2023-12-27 13:03:50,694][20347] Updated weights for policy 0, policy_version 1070171 (0.0008) [2023-12-27 13:03:50,762][20347] Updated weights for policy 0, policy_version 1070181 (0.0007) [2023-12-27 13:03:50,862][20381] Updated weights for policy 1, policy_version 1069241 (0.0008) [2023-12-27 13:03:50,917][20381] Updated weights for policy 1, policy_version 1069251 (0.0007) [2023-12-27 13:03:50,974][20381] Updated weights for policy 1, policy_version 1069261 (0.0008) [2023-12-27 13:03:51,025][20381] Updated weights for policy 1, policy_version 1069271 (0.0009) [2023-12-27 13:03:51,449][20347] Updated weights for policy 0, policy_version 1070191 (0.0007) [2023-12-27 13:03:51,508][20347] Updated weights for policy 0, policy_version 1070201 (0.0007) [2023-12-27 13:03:51,560][20347] Updated weights for policy 0, policy_version 1070211 (0.0008) [2023-12-27 13:03:51,814][20381] Updated weights for policy 1, policy_version 1069281 (0.0008) [2023-12-27 13:03:51,874][20381] Updated weights for policy 1, policy_version 1069291 (0.0008) [2023-12-27 13:03:51,921][20381] Updated weights for policy 1, policy_version 1069301 (0.0008) [2023-12-27 13:03:52,274][20347] Updated weights for policy 0, policy_version 1070221 (0.0008) [2023-12-27 13:03:52,322][20291] KL-divergence is very high: 104.8757 [2023-12-27 13:03:52,335][20347] Updated weights for policy 0, policy_version 1070231 (0.0007) [2023-12-27 13:03:52,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 547790848. Throughput: 0: 10566.6, 1: 10179.1. Samples: 547766864. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:52,363][19170] Avg episode reward: [(0, '951.430'), (1, '667.140')] [2023-12-27 13:03:52,367][20291] KL-divergence is very high: 100.9643 [2023-12-27 13:03:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001069304_273768448.pth... [2023-12-27 13:03:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001068120_273465344.pth [2023-12-27 13:03:52,392][20347] Updated weights for policy 0, policy_version 1070241 (0.0008) [2023-12-27 13:03:52,437][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001070248_274030592.pth... [2023-12-27 13:03:52,440][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001069000_273711104.pth [2023-12-27 13:03:52,685][20381] Updated weights for policy 1, policy_version 1069311 (0.0008) [2023-12-27 13:03:52,743][20381] Updated weights for policy 1, policy_version 1069321 (0.0008) [2023-12-27 13:03:52,824][20381] Updated weights for policy 1, policy_version 1069331 (0.0008) [2023-12-27 13:03:53,023][20347] Updated weights for policy 0, policy_version 1070251 (0.0007) [2023-12-27 13:03:53,073][20347] Updated weights for policy 0, policy_version 1070261 (0.0005) [2023-12-27 13:03:53,129][20347] Updated weights for policy 0, policy_version 1070271 (0.0005) [2023-12-27 13:03:53,546][20381] Updated weights for policy 1, policy_version 1069341 (0.0009) [2023-12-27 13:03:53,603][20381] Updated weights for policy 1, policy_version 1069351 (0.0008) [2023-12-27 13:03:53,661][20381] Updated weights for policy 1, policy_version 1069361 (0.0008) [2023-12-27 13:03:53,698][20347] Updated weights for policy 0, policy_version 1070281 (0.0005) [2023-12-27 13:03:53,764][20347] Updated weights for policy 0, policy_version 1070291 (0.0007) [2023-12-27 13:03:53,823][20347] Updated weights for policy 0, policy_version 1070301 (0.0007) [2023-12-27 13:03:53,882][20347] Updated weights for policy 0, policy_version 1070311 (0.0008) [2023-12-27 13:03:54,357][20381] Updated weights for policy 1, policy_version 1069371 (0.0008) [2023-12-27 13:03:54,414][20381] Updated weights for policy 1, policy_version 1069381 (0.0007) [2023-12-27 13:03:54,473][20381] Updated weights for policy 1, policy_version 1069391 (0.0008) [2023-12-27 13:03:54,589][20347] Updated weights for policy 0, policy_version 1070321 (0.0006) [2023-12-27 13:03:54,650][20347] Updated weights for policy 0, policy_version 1070331 (0.0008) [2023-12-27 13:03:54,706][20347] Updated weights for policy 0, policy_version 1070341 (0.0008) [2023-12-27 13:03:55,161][20381] Updated weights for policy 1, policy_version 1069401 (0.0008) [2023-12-27 13:03:55,217][20381] Updated weights for policy 1, policy_version 1069411 (0.0008) [2023-12-27 13:03:55,277][20381] Updated weights for policy 1, policy_version 1069421 (0.0008) [2023-12-27 13:03:55,334][20381] Updated weights for policy 1, policy_version 1069431 (0.0006) [2023-12-27 13:03:55,343][20347] Updated weights for policy 0, policy_version 1070351 (0.0007) [2023-12-27 13:03:55,401][20347] Updated weights for policy 0, policy_version 1070361 (0.0007) [2023-12-27 13:03:55,452][20347] Updated weights for policy 0, policy_version 1070371 (0.0008) [2023-12-27 13:03:56,041][20381] Updated weights for policy 1, policy_version 1069441 (0.0007) [2023-12-27 13:03:56,092][20381] Updated weights for policy 1, policy_version 1069451 (0.0007) [2023-12-27 13:03:56,098][20347] Updated weights for policy 0, policy_version 1070381 (0.0007) [2023-12-27 13:03:56,149][20381] Updated weights for policy 1, policy_version 1069461 (0.0007) [2023-12-27 13:03:56,151][20347] Updated weights for policy 0, policy_version 1070391 (0.0006) [2023-12-27 13:03:56,208][20347] Updated weights for policy 0, policy_version 1070401 (0.0005) [2023-12-27 13:03:56,747][20347] Updated weights for policy 0, policy_version 1070411 (0.0006) [2023-12-27 13:03:56,808][20347] Updated weights for policy 0, policy_version 1070421 (0.0005) [2023-12-27 13:03:56,863][20347] Updated weights for policy 0, policy_version 1070431 (0.0005) [2023-12-27 13:03:56,900][20381] Updated weights for policy 1, policy_version 1069471 (0.0007) [2023-12-27 13:03:56,947][20381] Updated weights for policy 1, policy_version 1069481 (0.0007) [2023-12-27 13:03:56,996][20381] Updated weights for policy 1, policy_version 1069491 (0.0009) [2023-12-27 13:03:57,359][20347] Updated weights for policy 0, policy_version 1070441 (0.0005) [2023-12-27 13:03:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 547897344. Throughput: 0: 10570.1, 1: 10094.8. Samples: 547886500. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:03:57,363][19170] Avg episode reward: [(0, '931.720'), (1, '755.230')] [2023-12-27 13:03:57,405][20347] Updated weights for policy 0, policy_version 1070451 (0.0005) [2023-12-27 13:03:57,451][20347] Updated weights for policy 0, policy_version 1070461 (0.0005) [2023-12-27 13:03:57,494][20347] Updated weights for policy 0, policy_version 1070471 (0.0005) [2023-12-27 13:03:57,693][20381] Updated weights for policy 1, policy_version 1069501 (0.0008) [2023-12-27 13:03:57,742][20381] Updated weights for policy 1, policy_version 1069511 (0.0009) [2023-12-27 13:03:57,802][20381] Updated weights for policy 1, policy_version 1069521 (0.0008) [2023-12-27 13:03:58,085][20347] Updated weights for policy 0, policy_version 1070481 (0.0007) [2023-12-27 13:03:58,133][20347] Updated weights for policy 0, policy_version 1070491 (0.0007) [2023-12-27 13:03:58,187][20347] Updated weights for policy 0, policy_version 1070501 (0.0008) [2023-12-27 13:03:58,583][20381] Updated weights for policy 1, policy_version 1069531 (0.0008) [2023-12-27 13:03:58,641][20381] Updated weights for policy 1, policy_version 1069541 (0.0008) [2023-12-27 13:03:58,706][20381] Updated weights for policy 1, policy_version 1069551 (0.0008) [2023-12-27 13:03:58,988][20347] Updated weights for policy 0, policy_version 1070511 (0.0008) [2023-12-27 13:03:59,045][20347] Updated weights for policy 0, policy_version 1070521 (0.0010) [2023-12-27 13:03:59,104][20347] Updated weights for policy 0, policy_version 1070531 (0.0011) [2023-12-27 13:03:59,474][20381] Updated weights for policy 1, policy_version 1069561 (0.0008) [2023-12-27 13:03:59,529][20381] Updated weights for policy 1, policy_version 1069571 (0.0007) [2023-12-27 13:03:59,592][20381] Updated weights for policy 1, policy_version 1069581 (0.0009) [2023-12-27 13:03:59,652][20381] Updated weights for policy 1, policy_version 1069591 (0.0005) [2023-12-27 13:03:59,764][20347] Updated weights for policy 0, policy_version 1070541 (0.0008) [2023-12-27 13:03:59,824][20347] Updated weights for policy 0, policy_version 1070551 (0.0006) [2023-12-27 13:03:59,876][20347] Updated weights for policy 0, policy_version 1070561 (0.0008) [2023-12-27 13:04:00,301][20381] Updated weights for policy 1, policy_version 1069601 (0.0010) [2023-12-27 13:04:00,357][20381] Updated weights for policy 1, policy_version 1069611 (0.0011) [2023-12-27 13:04:00,409][20381] Updated weights for policy 1, policy_version 1069621 (0.0010) [2023-12-27 13:04:00,593][20347] Updated weights for policy 0, policy_version 1070571 (0.0008) [2023-12-27 13:04:00,639][20347] Updated weights for policy 0, policy_version 1070581 (0.0007) [2023-12-27 13:04:00,686][20347] Updated weights for policy 0, policy_version 1070591 (0.0007) [2023-12-27 13:04:01,057][20381] Updated weights for policy 1, policy_version 1069631 (0.0009) [2023-12-27 13:04:01,118][20381] Updated weights for policy 1, policy_version 1069641 (0.0008) [2023-12-27 13:04:01,181][20381] Updated weights for policy 1, policy_version 1069651 (0.0008) [2023-12-27 13:04:01,373][20347] Updated weights for policy 0, policy_version 1070601 (0.0007) [2023-12-27 13:04:01,434][20347] Updated weights for policy 0, policy_version 1070611 (0.0007) [2023-12-27 13:04:01,493][20347] Updated weights for policy 0, policy_version 1070621 (0.0008) [2023-12-27 13:04:01,552][20347] Updated weights for policy 0, policy_version 1070631 (0.0006) [2023-12-27 13:04:01,954][20381] Updated weights for policy 1, policy_version 1069661 (0.0008) [2023-12-27 13:04:02,018][20381] Updated weights for policy 1, policy_version 1069671 (0.0008) [2023-12-27 13:04:02,084][20381] Updated weights for policy 1, policy_version 1069681 (0.0008) [2023-12-27 13:04:02,220][20347] Updated weights for policy 0, policy_version 1070641 (0.0007) [2023-12-27 13:04:02,273][20347] Updated weights for policy 0, policy_version 1070651 (0.0008) [2023-12-27 13:04:02,327][20347] Updated weights for policy 0, policy_version 1070661 (0.0006) [2023-12-27 13:04:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 548003840. Throughput: 0: 10607.0, 1: 9995.0. Samples: 548008896. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:02,363][19170] Avg episode reward: [(0, '921.950'), (1, '794.470')] [2023-12-27 13:04:02,804][20381] Updated weights for policy 1, policy_version 1069691 (0.0008) [2023-12-27 13:04:02,862][20381] Updated weights for policy 1, policy_version 1069701 (0.0008) [2023-12-27 13:04:02,923][20381] Updated weights for policy 1, policy_version 1069711 (0.0008) [2023-12-27 13:04:02,965][20347] Updated weights for policy 0, policy_version 1070671 (0.0007) [2023-12-27 13:04:03,016][20347] Updated weights for policy 0, policy_version 1070681 (0.0007) [2023-12-27 13:04:03,061][20347] Updated weights for policy 0, policy_version 1070691 (0.0005) [2023-12-27 13:04:03,636][20381] Updated weights for policy 1, policy_version 1069721 (0.0006) [2023-12-27 13:04:03,654][20347] Updated weights for policy 0, policy_version 1070701 (0.0005) [2023-12-27 13:04:03,694][20381] Updated weights for policy 1, policy_version 1069731 (0.0008) [2023-12-27 13:04:03,710][20347] Updated weights for policy 0, policy_version 1070711 (0.0007) [2023-12-27 13:04:03,742][20381] Updated weights for policy 1, policy_version 1069741 (0.0006) [2023-12-27 13:04:03,755][20347] Updated weights for policy 0, policy_version 1070721 (0.0006) [2023-12-27 13:04:03,790][20381] Updated weights for policy 1, policy_version 1069751 (0.0005) [2023-12-27 13:04:04,421][20347] Updated weights for policy 0, policy_version 1070731 (0.0006) [2023-12-27 13:04:04,483][20347] Updated weights for policy 0, policy_version 1070741 (0.0006) [2023-12-27 13:04:04,524][20381] Updated weights for policy 1, policy_version 1069761 (0.0006) [2023-12-27 13:04:04,539][20347] Updated weights for policy 0, policy_version 1070751 (0.0008) [2023-12-27 13:04:04,590][20381] Updated weights for policy 1, policy_version 1069771 (0.0005) [2023-12-27 13:04:04,651][20381] Updated weights for policy 1, policy_version 1069781 (0.0007) [2023-12-27 13:04:05,155][20347] Updated weights for policy 0, policy_version 1070761 (0.0008) [2023-12-27 13:04:05,215][20347] Updated weights for policy 0, policy_version 1070771 (0.0008) [2023-12-27 13:04:05,272][20347] Updated weights for policy 0, policy_version 1070781 (0.0008) [2023-12-27 13:04:05,308][20381] Updated weights for policy 1, policy_version 1069791 (0.0006) [2023-12-27 13:04:05,329][20347] Updated weights for policy 0, policy_version 1070791 (0.0007) [2023-12-27 13:04:05,355][20381] Updated weights for policy 1, policy_version 1069801 (0.0006) [2023-12-27 13:04:05,410][20381] Updated weights for policy 1, policy_version 1069811 (0.0007) [2023-12-27 13:04:06,034][20381] Updated weights for policy 1, policy_version 1069821 (0.0006) [2023-12-27 13:04:06,049][20347] Updated weights for policy 0, policy_version 1070801 (0.0007) [2023-12-27 13:04:06,098][20381] Updated weights for policy 1, policy_version 1069831 (0.0007) [2023-12-27 13:04:06,116][20347] Updated weights for policy 0, policy_version 1070811 (0.0006) [2023-12-27 13:04:06,163][20381] Updated weights for policy 1, policy_version 1069841 (0.0008) [2023-12-27 13:04:06,183][20347] Updated weights for policy 0, policy_version 1070821 (0.0009) [2023-12-27 13:04:06,861][20347] Updated weights for policy 0, policy_version 1070831 (0.0006) [2023-12-27 13:04:06,878][20381] Updated weights for policy 1, policy_version 1069851 (0.0007) [2023-12-27 13:04:06,917][20347] Updated weights for policy 0, policy_version 1070841 (0.0006) [2023-12-27 13:04:06,933][20381] Updated weights for policy 1, policy_version 1069861 (0.0006) [2023-12-27 13:04:06,981][20347] Updated weights for policy 0, policy_version 1070851 (0.0006) [2023-12-27 13:04:06,988][20381] Updated weights for policy 1, policy_version 1069871 (0.0008) [2023-12-27 13:04:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 548102144. Throughput: 0: 10661.9, 1: 9999.6. Samples: 548071444. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:07,363][19170] Avg episode reward: [(0, '941.350'), (1, '686.740')] [2023-12-27 13:04:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001070856_274186240.pth... [2023-12-27 13:04:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001069880_273915904.pth... [2023-12-27 13:04:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001069608_273866752.pth [2023-12-27 13:04:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001068728_273620992.pth [2023-12-27 13:04:07,567][20347] Updated weights for policy 0, policy_version 1070861 (0.0007) [2023-12-27 13:04:07,614][20347] Updated weights for policy 0, policy_version 1070871 (0.0008) [2023-12-27 13:04:07,673][20347] Updated weights for policy 0, policy_version 1070881 (0.0007) [2023-12-27 13:04:07,711][20381] Updated weights for policy 1, policy_version 1069881 (0.0008) [2023-12-27 13:04:07,762][20381] Updated weights for policy 1, policy_version 1069891 (0.0005) [2023-12-27 13:04:07,823][20381] Updated weights for policy 1, policy_version 1069901 (0.0005) [2023-12-27 13:04:07,871][20381] Updated weights for policy 1, policy_version 1069911 (0.0005) [2023-12-27 13:04:08,429][20347] Updated weights for policy 0, policy_version 1070891 (0.0008) [2023-12-27 13:04:08,494][20381] Updated weights for policy 1, policy_version 1069921 (0.0005) [2023-12-27 13:04:08,496][20347] Updated weights for policy 0, policy_version 1070901 (0.0008) [2023-12-27 13:04:08,555][20381] Updated weights for policy 1, policy_version 1069931 (0.0006) [2023-12-27 13:04:08,557][20347] Updated weights for policy 0, policy_version 1070911 (0.0008) [2023-12-27 13:04:08,620][20381] Updated weights for policy 1, policy_version 1069941 (0.0006) [2023-12-27 13:04:09,280][20347] Updated weights for policy 0, policy_version 1070921 (0.0007) [2023-12-27 13:04:09,282][20381] Updated weights for policy 1, policy_version 1069951 (0.0007) [2023-12-27 13:04:09,339][20347] Updated weights for policy 0, policy_version 1070931 (0.0008) [2023-12-27 13:04:09,341][20381] Updated weights for policy 1, policy_version 1069961 (0.0006) [2023-12-27 13:04:09,409][20347] Updated weights for policy 0, policy_version 1070941 (0.0009) [2023-12-27 13:04:09,412][20381] Updated weights for policy 1, policy_version 1069971 (0.0008) [2023-12-27 13:04:09,466][20347] Updated weights for policy 0, policy_version 1070951 (0.0010) [2023-12-27 13:04:10,104][20381] Updated weights for policy 1, policy_version 1069981 (0.0007) [2023-12-27 13:04:10,161][20381] Updated weights for policy 1, policy_version 1069991 (0.0005) [2023-12-27 13:04:10,220][20381] Updated weights for policy 1, policy_version 1070001 (0.0005) [2023-12-27 13:04:10,225][20347] Updated weights for policy 0, policy_version 1070961 (0.0010) [2023-12-27 13:04:10,245][20291] KL-divergence is very high: 709.0229 [2023-12-27 13:04:10,290][20347] Updated weights for policy 0, policy_version 1070971 (0.0011) [2023-12-27 13:04:10,297][20291] KL-divergence is very high: 406.7621 [2023-12-27 13:04:10,353][20347] Updated weights for policy 0, policy_version 1070981 (0.0011) [2023-12-27 13:04:10,884][20381] Updated weights for policy 1, policy_version 1070011 (0.0007) [2023-12-27 13:04:10,913][20347] Updated weights for policy 0, policy_version 1070991 (0.0010) [2023-12-27 13:04:10,934][20381] Updated weights for policy 1, policy_version 1070021 (0.0006) [2023-12-27 13:04:10,963][20347] Updated weights for policy 0, policy_version 1071001 (0.0010) [2023-12-27 13:04:10,990][20381] Updated weights for policy 1, policy_version 1070031 (0.0007) [2023-12-27 13:04:11,016][20347] Updated weights for policy 0, policy_version 1071011 (0.0010) [2023-12-27 13:04:11,795][20381] Updated weights for policy 1, policy_version 1070041 (0.0006) [2023-12-27 13:04:11,825][20347] Updated weights for policy 0, policy_version 1071021 (0.0008) [2023-12-27 13:04:11,853][20381] Updated weights for policy 1, policy_version 1070051 (0.0008) [2023-12-27 13:04:11,884][20347] Updated weights for policy 0, policy_version 1071031 (0.0008) [2023-12-27 13:04:11,912][20381] Updated weights for policy 1, policy_version 1070061 (0.0007) [2023-12-27 13:04:11,939][20347] Updated weights for policy 0, policy_version 1071041 (0.0008) [2023-12-27 13:04:11,966][20381] Updated weights for policy 1, policy_version 1070071 (0.0008) [2023-12-27 13:04:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 548200448. Throughput: 0: 10610.6, 1: 10027.7. Samples: 548192244. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:12,363][19170] Avg episode reward: [(0, '951.080'), (1, '784.650')] [2023-12-27 13:04:12,574][20381] Updated weights for policy 1, policy_version 1070081 (0.0006) [2023-12-27 13:04:12,631][20381] Updated weights for policy 1, policy_version 1070091 (0.0008) [2023-12-27 13:04:12,685][20381] Updated weights for policy 1, policy_version 1070101 (0.0011) [2023-12-27 13:04:12,696][20347] Updated weights for policy 0, policy_version 1071051 (0.0007) [2023-12-27 13:04:12,756][20347] Updated weights for policy 0, policy_version 1071061 (0.0007) [2023-12-27 13:04:12,815][20347] Updated weights for policy 0, policy_version 1071071 (0.0007) [2023-12-27 13:04:13,295][20381] Updated weights for policy 1, policy_version 1070111 (0.0007) [2023-12-27 13:04:13,360][20381] Updated weights for policy 1, policy_version 1070121 (0.0008) [2023-12-27 13:04:13,416][20381] Updated weights for policy 1, policy_version 1070131 (0.0008) [2023-12-27 13:04:13,486][20347] Updated weights for policy 0, policy_version 1071081 (0.0008) [2023-12-27 13:04:13,548][20347] Updated weights for policy 0, policy_version 1071091 (0.0010) [2023-12-27 13:04:13,613][20347] Updated weights for policy 0, policy_version 1071101 (0.0010) [2023-12-27 13:04:13,675][20347] Updated weights for policy 0, policy_version 1071111 (0.0010) [2023-12-27 13:04:13,992][20381] Updated weights for policy 1, policy_version 1070141 (0.0007) [2023-12-27 13:04:14,050][20381] Updated weights for policy 1, policy_version 1070151 (0.0007) [2023-12-27 13:04:14,110][20381] Updated weights for policy 1, policy_version 1070161 (0.0008) [2023-12-27 13:04:14,309][20347] Updated weights for policy 0, policy_version 1071121 (0.0010) [2023-12-27 13:04:14,358][20347] Updated weights for policy 0, policy_version 1071131 (0.0010) [2023-12-27 13:04:14,403][20347] Updated weights for policy 0, policy_version 1071141 (0.0010) [2023-12-27 13:04:14,757][20381] Updated weights for policy 1, policy_version 1070171 (0.0008) [2023-12-27 13:04:14,820][20381] Updated weights for policy 1, policy_version 1070181 (0.0008) [2023-12-27 13:04:14,880][20381] Updated weights for policy 1, policy_version 1070191 (0.0008) [2023-12-27 13:04:15,163][20347] Updated weights for policy 0, policy_version 1071151 (0.0011) [2023-12-27 13:04:15,222][20347] Updated weights for policy 0, policy_version 1071161 (0.0010) [2023-12-27 13:04:15,274][20347] Updated weights for policy 0, policy_version 1071171 (0.0010) [2023-12-27 13:04:15,620][20381] Updated weights for policy 1, policy_version 1070201 (0.0009) [2023-12-27 13:04:15,685][20381] Updated weights for policy 1, policy_version 1070211 (0.0011) [2023-12-27 13:04:15,736][20381] Updated weights for policy 1, policy_version 1070221 (0.0010) [2023-12-27 13:04:15,791][20381] Updated weights for policy 1, policy_version 1070231 (0.0011) [2023-12-27 13:04:15,961][20347] Updated weights for policy 0, policy_version 1071181 (0.0008) [2023-12-27 13:04:16,022][20347] Updated weights for policy 0, policy_version 1071191 (0.0005) [2023-12-27 13:04:16,048][20291] KL-divergence is very high: 130.8522 [2023-12-27 13:04:16,083][20347] Updated weights for policy 0, policy_version 1071201 (0.0005) [2023-12-27 13:04:16,408][20381] Updated weights for policy 1, policy_version 1070241 (0.0010) [2023-12-27 13:04:16,450][20381] Updated weights for policy 1, policy_version 1070251 (0.0010) [2023-12-27 13:04:16,498][20381] Updated weights for policy 1, policy_version 1070261 (0.0010) [2023-12-27 13:04:16,700][20347] Updated weights for policy 0, policy_version 1071211 (0.0006) [2023-12-27 13:04:16,753][20347] Updated weights for policy 0, policy_version 1071221 (0.0007) [2023-12-27 13:04:16,812][20347] Updated weights for policy 0, policy_version 1071231 (0.0008) [2023-12-27 13:04:17,176][20381] Updated weights for policy 1, policy_version 1070271 (0.0010) [2023-12-27 13:04:17,233][20381] Updated weights for policy 1, policy_version 1070281 (0.0010) [2023-12-27 13:04:17,286][20381] Updated weights for policy 1, policy_version 1070291 (0.0008) [2023-12-27 13:04:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 548306944. Throughput: 0: 10514.6, 1: 10019.7. Samples: 548315304. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:17,364][19170] Avg episode reward: [(0, '951.060'), (1, '794.380')] [2023-12-27 13:04:17,513][20347] Updated weights for policy 0, policy_version 1071241 (0.0007) [2023-12-27 13:04:17,576][20347] Updated weights for policy 0, policy_version 1071251 (0.0007) [2023-12-27 13:04:17,631][20347] Updated weights for policy 0, policy_version 1071261 (0.0008) [2023-12-27 13:04:17,694][20347] Updated weights for policy 0, policy_version 1071271 (0.0006) [2023-12-27 13:04:17,890][20381] Updated weights for policy 1, policy_version 1070301 (0.0006) [2023-12-27 13:04:17,936][20381] Updated weights for policy 1, policy_version 1070311 (0.0005) [2023-12-27 13:04:17,992][20381] Updated weights for policy 1, policy_version 1070321 (0.0005) [2023-12-27 13:04:18,375][20347] Updated weights for policy 0, policy_version 1071281 (0.0010) [2023-12-27 13:04:18,429][20347] Updated weights for policy 0, policy_version 1071291 (0.0010) [2023-12-27 13:04:18,480][20347] Updated weights for policy 0, policy_version 1071301 (0.0010) [2023-12-27 13:04:18,613][20381] Updated weights for policy 1, policy_version 1070331 (0.0006) [2023-12-27 13:04:18,672][20381] Updated weights for policy 1, policy_version 1070341 (0.0007) [2023-12-27 13:04:18,731][20381] Updated weights for policy 1, policy_version 1070351 (0.0009) [2023-12-27 13:04:19,165][20347] Updated weights for policy 0, policy_version 1071311 (0.0010) [2023-12-27 13:04:19,214][20347] Updated weights for policy 0, policy_version 1071321 (0.0010) [2023-12-27 13:04:19,274][20347] Updated weights for policy 0, policy_version 1071331 (0.0011) [2023-12-27 13:04:19,365][20381] Updated weights for policy 1, policy_version 1070361 (0.0008) [2023-12-27 13:04:19,427][20381] Updated weights for policy 1, policy_version 1070371 (0.0007) [2023-12-27 13:04:19,476][20381] Updated weights for policy 1, policy_version 1070381 (0.0007) [2023-12-27 13:04:19,538][20381] Updated weights for policy 1, policy_version 1070391 (0.0008) [2023-12-27 13:04:19,994][20347] Updated weights for policy 0, policy_version 1071341 (0.0010) [2023-12-27 13:04:20,052][20347] Updated weights for policy 0, policy_version 1071351 (0.0010) [2023-12-27 13:04:20,110][20347] Updated weights for policy 0, policy_version 1071361 (0.0011) [2023-12-27 13:04:20,295][20381] Updated weights for policy 1, policy_version 1070401 (0.0008) [2023-12-27 13:04:20,356][20381] Updated weights for policy 1, policy_version 1070411 (0.0009) [2023-12-27 13:04:20,418][20381] Updated weights for policy 1, policy_version 1070421 (0.0008) [2023-12-27 13:04:20,770][20347] Updated weights for policy 0, policy_version 1071371 (0.0011) [2023-12-27 13:04:20,827][20347] Updated weights for policy 0, policy_version 1071381 (0.0011) [2023-12-27 13:04:20,891][20347] Updated weights for policy 0, policy_version 1071391 (0.0011) [2023-12-27 13:04:21,159][20381] Updated weights for policy 1, policy_version 1070431 (0.0007) [2023-12-27 13:04:21,213][20381] Updated weights for policy 1, policy_version 1070441 (0.0007) [2023-12-27 13:04:21,274][20381] Updated weights for policy 1, policy_version 1070451 (0.0007) [2023-12-27 13:04:21,618][20347] Updated weights for policy 0, policy_version 1071401 (0.0010) [2023-12-27 13:04:21,685][20347] Updated weights for policy 0, policy_version 1071411 (0.0008) [2023-12-27 13:04:21,753][20347] Updated weights for policy 0, policy_version 1071421 (0.0009) [2023-12-27 13:04:21,822][20347] Updated weights for policy 0, policy_version 1071431 (0.0009) [2023-12-27 13:04:22,023][20381] Updated weights for policy 1, policy_version 1070461 (0.0007) [2023-12-27 13:04:22,060][20292] KL-divergence is very high: 172.7418 [2023-12-27 13:04:22,067][20292] KL-divergence is very high: 286.1897 [2023-12-27 13:04:22,074][20292] KL-divergence is very high: 195.6500 [2023-12-27 13:04:22,080][20292] KL-divergence is very high: 136.8856 [2023-12-27 13:04:22,084][20381] Updated weights for policy 1, policy_version 1070471 (0.0006) [2023-12-27 13:04:22,140][20381] Updated weights for policy 1, policy_version 1070481 (0.0007) [2023-12-27 13:04:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 548405248. Throughput: 0: 10486.4, 1: 10005.7. Samples: 548377020. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:22,363][19170] Avg episode reward: [(0, '960.870'), (1, '724.550')] [2023-12-27 13:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001070488_274071552.pth... [2023-12-27 13:04:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001071432_274333696.pth... [2023-12-27 13:04:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001069304_273768448.pth [2023-12-27 13:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001070248_274030592.pth [2023-12-27 13:04:22,572][20347] Updated weights for policy 0, policy_version 1071441 (0.0008) [2023-12-27 13:04:22,629][20347] Updated weights for policy 0, policy_version 1071451 (0.0009) [2023-12-27 13:04:22,681][20347] Updated weights for policy 0, policy_version 1071461 (0.0008) [2023-12-27 13:04:22,774][20292] KL-divergence is very high: 232.8723 [2023-12-27 13:04:22,787][20381] Updated weights for policy 1, policy_version 1070491 (0.0006) [2023-12-27 13:04:22,824][20292] KL-divergence is very high: 234.0119 [2023-12-27 13:04:22,846][20381] Updated weights for policy 1, policy_version 1070501 (0.0008) [2023-12-27 13:04:22,866][20292] KL-divergence is very high: 124.7131 [2023-12-27 13:04:22,905][20381] Updated weights for policy 1, policy_version 1070511 (0.0007) [2023-12-27 13:04:22,911][20292] KL-divergence is very high: 225.4610 [2023-12-27 13:04:22,918][20292] KL-divergence is very high: 136.1561 [2023-12-27 13:04:23,340][20347] Updated weights for policy 0, policy_version 1071471 (0.0006) [2023-12-27 13:04:23,400][20347] Updated weights for policy 0, policy_version 1071481 (0.0006) [2023-12-27 13:04:23,456][20347] Updated weights for policy 0, policy_version 1071491 (0.0005) [2023-12-27 13:04:23,538][20381] Updated weights for policy 1, policy_version 1070521 (0.0007) [2023-12-27 13:04:23,586][20381] Updated weights for policy 1, policy_version 1070531 (0.0005) [2023-12-27 13:04:23,629][20381] Updated weights for policy 1, policy_version 1070541 (0.0005) [2023-12-27 13:04:23,676][20381] Updated weights for policy 1, policy_version 1070551 (0.0007) [2023-12-27 13:04:24,087][20347] Updated weights for policy 0, policy_version 1071501 (0.0008) [2023-12-27 13:04:24,137][20347] Updated weights for policy 0, policy_version 1071511 (0.0011) [2023-12-27 13:04:24,185][20347] Updated weights for policy 0, policy_version 1071521 (0.0009) [2023-12-27 13:04:24,384][20381] Updated weights for policy 1, policy_version 1070561 (0.0008) [2023-12-27 13:04:24,443][20381] Updated weights for policy 1, policy_version 1070571 (0.0008) [2023-12-27 13:04:24,499][20381] Updated weights for policy 1, policy_version 1070581 (0.0008) [2023-12-27 13:04:24,890][20347] Updated weights for policy 0, policy_version 1071531 (0.0008) [2023-12-27 13:04:24,947][20347] Updated weights for policy 0, policy_version 1071541 (0.0007) [2023-12-27 13:04:24,999][20347] Updated weights for policy 0, policy_version 1071551 (0.0008) [2023-12-27 13:04:25,204][20381] Updated weights for policy 1, policy_version 1070591 (0.0007) [2023-12-27 13:04:25,265][20381] Updated weights for policy 1, policy_version 1070601 (0.0008) [2023-12-27 13:04:25,316][20381] Updated weights for policy 1, policy_version 1070611 (0.0008) [2023-12-27 13:04:25,698][20347] Updated weights for policy 0, policy_version 1071561 (0.0008) [2023-12-27 13:04:25,746][20347] Updated weights for policy 0, policy_version 1071571 (0.0010) [2023-12-27 13:04:25,799][20347] Updated weights for policy 0, policy_version 1071581 (0.0008) [2023-12-27 13:04:25,855][20347] Updated weights for policy 0, policy_version 1071591 (0.0007) [2023-12-27 13:04:26,003][20381] Updated weights for policy 1, policy_version 1070621 (0.0007) [2023-12-27 13:04:26,059][20381] Updated weights for policy 1, policy_version 1070631 (0.0007) [2023-12-27 13:04:26,112][20381] Updated weights for policy 1, policy_version 1070641 (0.0008) [2023-12-27 13:04:26,487][20347] Updated weights for policy 0, policy_version 1071601 (0.0008) [2023-12-27 13:04:26,539][20347] Updated weights for policy 0, policy_version 1071611 (0.0008) [2023-12-27 13:04:26,594][20347] Updated weights for policy 0, policy_version 1071621 (0.0007) [2023-12-27 13:04:26,823][20381] Updated weights for policy 1, policy_version 1070651 (0.0007) [2023-12-27 13:04:26,881][20381] Updated weights for policy 1, policy_version 1070661 (0.0007) [2023-12-27 13:04:26,939][20381] Updated weights for policy 1, policy_version 1070671 (0.0007) [2023-12-27 13:04:27,275][20347] Updated weights for policy 0, policy_version 1071631 (0.0006) [2023-12-27 13:04:27,322][20347] Updated weights for policy 0, policy_version 1071641 (0.0006) [2023-12-27 13:04:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 548503552. Throughput: 0: 10375.6, 1: 9991.2. Samples: 548496716. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:27,363][19170] Avg episode reward: [(0, '941.390'), (1, '667.880')] [2023-12-27 13:04:27,374][20347] Updated weights for policy 0, policy_version 1071651 (0.0006) [2023-12-27 13:04:27,651][20381] Updated weights for policy 1, policy_version 1070681 (0.0007) [2023-12-27 13:04:27,705][20381] Updated weights for policy 1, policy_version 1070691 (0.0005) [2023-12-27 13:04:27,756][20381] Updated weights for policy 1, policy_version 1070701 (0.0005) [2023-12-27 13:04:27,808][20381] Updated weights for policy 1, policy_version 1070711 (0.0007) [2023-12-27 13:04:27,978][20347] Updated weights for policy 0, policy_version 1071661 (0.0007) [2023-12-27 13:04:28,029][20347] Updated weights for policy 0, policy_version 1071671 (0.0005) [2023-12-27 13:04:28,075][20347] Updated weights for policy 0, policy_version 1071681 (0.0005) [2023-12-27 13:04:28,412][20381] Updated weights for policy 1, policy_version 1070721 (0.0007) [2023-12-27 13:04:28,480][20381] Updated weights for policy 1, policy_version 1070731 (0.0008) [2023-12-27 13:04:28,541][20381] Updated weights for policy 1, policy_version 1070741 (0.0008) [2023-12-27 13:04:28,678][20347] Updated weights for policy 0, policy_version 1071691 (0.0005) [2023-12-27 13:04:28,736][20347] Updated weights for policy 0, policy_version 1071701 (0.0010) [2023-12-27 13:04:28,790][20347] Updated weights for policy 0, policy_version 1071711 (0.0011) [2023-12-27 13:04:29,201][20381] Updated weights for policy 1, policy_version 1070751 (0.0007) [2023-12-27 13:04:29,270][20381] Updated weights for policy 1, policy_version 1070761 (0.0008) [2023-12-27 13:04:29,327][20381] Updated weights for policy 1, policy_version 1070771 (0.0008) [2023-12-27 13:04:29,494][20347] Updated weights for policy 0, policy_version 1071721 (0.0011) [2023-12-27 13:04:29,550][20347] Updated weights for policy 0, policy_version 1071731 (0.0011) [2023-12-27 13:04:29,606][20347] Updated weights for policy 0, policy_version 1071741 (0.0010) [2023-12-27 13:04:29,660][20347] Updated weights for policy 0, policy_version 1071751 (0.0010) [2023-12-27 13:04:30,080][20381] Updated weights for policy 1, policy_version 1070781 (0.0008) [2023-12-27 13:04:30,136][20381] Updated weights for policy 1, policy_version 1070791 (0.0009) [2023-12-27 13:04:30,190][20381] Updated weights for policy 1, policy_version 1070801 (0.0008) [2023-12-27 13:04:30,419][20347] Updated weights for policy 0, policy_version 1071761 (0.0011) [2023-12-27 13:04:30,470][20347] Updated weights for policy 0, policy_version 1071771 (0.0010) [2023-12-27 13:04:30,523][20347] Updated weights for policy 0, policy_version 1071781 (0.0007) [2023-12-27 13:04:30,890][20381] Updated weights for policy 1, policy_version 1070811 (0.0006) [2023-12-27 13:04:30,935][20381] Updated weights for policy 1, policy_version 1070821 (0.0008) [2023-12-27 13:04:30,990][20381] Updated weights for policy 1, policy_version 1070831 (0.0007) [2023-12-27 13:04:31,118][20347] Updated weights for policy 0, policy_version 1071791 (0.0009) [2023-12-27 13:04:31,174][20347] Updated weights for policy 0, policy_version 1071801 (0.0008) [2023-12-27 13:04:31,231][20347] Updated weights for policy 0, policy_version 1071811 (0.0007) [2023-12-27 13:04:31,678][20381] Updated weights for policy 1, policy_version 1070841 (0.0007) [2023-12-27 13:04:31,751][20381] Updated weights for policy 1, policy_version 1070851 (0.0009) [2023-12-27 13:04:31,820][20381] Updated weights for policy 1, policy_version 1070861 (0.0007) [2023-12-27 13:04:31,883][20381] Updated weights for policy 1, policy_version 1070871 (0.0008) [2023-12-27 13:04:31,954][20347] Updated weights for policy 0, policy_version 1071821 (0.0006) [2023-12-27 13:04:32,010][20347] Updated weights for policy 0, policy_version 1071831 (0.0005) [2023-12-27 13:04:32,074][20347] Updated weights for policy 0, policy_version 1071841 (0.0006) [2023-12-27 13:04:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 548610048. Throughput: 0: 10322.9, 1: 9987.2. Samples: 548620456. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:32,363][19170] Avg episode reward: [(0, '951.410'), (1, '611.510')] [2023-12-27 13:04:32,568][20381] Updated weights for policy 1, policy_version 1070881 (0.0010) [2023-12-27 13:04:32,617][20381] Updated weights for policy 1, policy_version 1070891 (0.0011) [2023-12-27 13:04:32,663][20347] Updated weights for policy 0, policy_version 1071851 (0.0007) [2023-12-27 13:04:32,686][20381] Updated weights for policy 1, policy_version 1070901 (0.0011) [2023-12-27 13:04:32,726][20347] Updated weights for policy 0, policy_version 1071861 (0.0006) [2023-12-27 13:04:32,788][20347] Updated weights for policy 0, policy_version 1071871 (0.0007) [2023-12-27 13:04:33,276][20381] Updated weights for policy 1, policy_version 1070911 (0.0008) [2023-12-27 13:04:33,330][20381] Updated weights for policy 1, policy_version 1070921 (0.0006) [2023-12-27 13:04:33,382][20381] Updated weights for policy 1, policy_version 1070931 (0.0006) [2023-12-27 13:04:33,396][20347] Updated weights for policy 0, policy_version 1071881 (0.0006) [2023-12-27 13:04:33,459][20347] Updated weights for policy 0, policy_version 1071891 (0.0005) [2023-12-27 13:04:33,511][20347] Updated weights for policy 0, policy_version 1071901 (0.0008) [2023-12-27 13:04:33,555][20347] Updated weights for policy 0, policy_version 1071912 (0.0007) [2023-12-27 13:04:33,946][20381] Updated weights for policy 1, policy_version 1070941 (0.0006) [2023-12-27 13:04:34,001][20381] Updated weights for policy 1, policy_version 1070951 (0.0005) [2023-12-27 13:04:34,059][20381] Updated weights for policy 1, policy_version 1070961 (0.0007) [2023-12-27 13:04:34,132][20347] Updated weights for policy 0, policy_version 1071922 (0.0006) [2023-12-27 13:04:34,187][20347] Updated weights for policy 0, policy_version 1071932 (0.0007) [2023-12-27 13:04:34,246][20347] Updated weights for policy 0, policy_version 1071942 (0.0008) [2023-12-27 13:04:34,764][20381] Updated weights for policy 1, policy_version 1070971 (0.0007) [2023-12-27 13:04:34,813][20381] Updated weights for policy 1, policy_version 1070981 (0.0007) [2023-12-27 13:04:34,859][20381] Updated weights for policy 1, policy_version 1070991 (0.0008) [2023-12-27 13:04:34,927][20347] Updated weights for policy 0, policy_version 1071952 (0.0006) [2023-12-27 13:04:34,975][20347] Updated weights for policy 0, policy_version 1071962 (0.0006) [2023-12-27 13:04:35,030][20347] Updated weights for policy 0, policy_version 1071972 (0.0008) [2023-12-27 13:04:35,547][20381] Updated weights for policy 1, policy_version 1071001 (0.0007) [2023-12-27 13:04:35,600][20381] Updated weights for policy 1, policy_version 1071011 (0.0006) [2023-12-27 13:04:35,606][20347] Updated weights for policy 0, policy_version 1071982 (0.0005) [2023-12-27 13:04:35,656][20381] Updated weights for policy 1, policy_version 1071021 (0.0006) [2023-12-27 13:04:35,657][20347] Updated weights for policy 0, policy_version 1071992 (0.0005) [2023-12-27 13:04:35,709][20347] Updated weights for policy 0, policy_version 1072002 (0.0005) [2023-12-27 13:04:35,719][20381] Updated weights for policy 1, policy_version 1071031 (0.0006) [2023-12-27 13:04:36,365][20347] Updated weights for policy 0, policy_version 1072012 (0.0008) [2023-12-27 13:04:36,429][20347] Updated weights for policy 0, policy_version 1072022 (0.0008) [2023-12-27 13:04:36,483][20381] Updated weights for policy 1, policy_version 1071041 (0.0007) [2023-12-27 13:04:36,495][20347] Updated weights for policy 0, policy_version 1072032 (0.0007) [2023-12-27 13:04:36,543][20381] Updated weights for policy 1, policy_version 1071051 (0.0007) [2023-12-27 13:04:36,600][20381] Updated weights for policy 1, policy_version 1071061 (0.0008) [2023-12-27 13:04:37,146][20347] Updated weights for policy 0, policy_version 1072042 (0.0007) [2023-12-27 13:04:37,211][20347] Updated weights for policy 0, policy_version 1072052 (0.0010) [2023-12-27 13:04:37,280][20347] Updated weights for policy 0, policy_version 1072062 (0.0006) [2023-12-27 13:04:37,301][20381] Updated weights for policy 1, policy_version 1071071 (0.0007) [2023-12-27 13:04:37,351][20347] Updated weights for policy 0, policy_version 1072072 (0.0006) [2023-12-27 13:04:37,361][20381] Updated weights for policy 1, policy_version 1071081 (0.0006) [2023-12-27 13:04:37,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 548716544. Throughput: 0: 10351.7, 1: 10073.3. Samples: 548685988. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:37,363][19170] Avg episode reward: [(0, '980.410'), (1, '588.730')] [2023-12-27 13:04:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001072072_274497536.pth... [2023-12-27 13:04:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001070856_274186240.pth [2023-12-27 13:04:37,414][20381] Updated weights for policy 1, policy_version 1071091 (0.0007) [2023-12-27 13:04:37,439][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001071096_274227200.pth... [2023-12-27 13:04:37,442][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001069880_273915904.pth [2023-12-27 13:04:37,935][20347] Updated weights for policy 0, policy_version 1072082 (0.0010) [2023-12-27 13:04:37,997][20347] Updated weights for policy 0, policy_version 1072092 (0.0010) [2023-12-27 13:04:38,055][20347] Updated weights for policy 0, policy_version 1072102 (0.0010) [2023-12-27 13:04:38,076][20381] Updated weights for policy 1, policy_version 1071101 (0.0007) [2023-12-27 13:04:38,126][20381] Updated weights for policy 1, policy_version 1071111 (0.0007) [2023-12-27 13:04:38,174][20381] Updated weights for policy 1, policy_version 1071121 (0.0008) [2023-12-27 13:04:38,765][20347] Updated weights for policy 0, policy_version 1072112 (0.0008) [2023-12-27 13:04:38,828][20347] Updated weights for policy 0, policy_version 1072122 (0.0007) [2023-12-27 13:04:38,889][20347] Updated weights for policy 0, policy_version 1072132 (0.0007) [2023-12-27 13:04:38,923][20381] Updated weights for policy 1, policy_version 1071131 (0.0008) [2023-12-27 13:04:38,978][20381] Updated weights for policy 1, policy_version 1071141 (0.0008) [2023-12-27 13:04:39,037][20381] Updated weights for policy 1, policy_version 1071151 (0.0008) [2023-12-27 13:04:39,588][20347] Updated weights for policy 0, policy_version 1072142 (0.0008) [2023-12-27 13:04:39,640][20347] Updated weights for policy 0, policy_version 1072152 (0.0009) [2023-12-27 13:04:39,695][20347] Updated weights for policy 0, policy_version 1072162 (0.0008) [2023-12-27 13:04:39,737][20381] Updated weights for policy 1, policy_version 1071161 (0.0007) [2023-12-27 13:04:39,795][20381] Updated weights for policy 1, policy_version 1071171 (0.0008) [2023-12-27 13:04:39,852][20381] Updated weights for policy 1, policy_version 1071181 (0.0009) [2023-12-27 13:04:39,908][20381] Updated weights for policy 1, policy_version 1071191 (0.0009) [2023-12-27 13:04:40,378][20347] Updated weights for policy 0, policy_version 1072172 (0.0006) [2023-12-27 13:04:40,436][20347] Updated weights for policy 0, policy_version 1072182 (0.0008) [2023-12-27 13:04:40,495][20347] Updated weights for policy 0, policy_version 1072192 (0.0007) [2023-12-27 13:04:40,623][20381] Updated weights for policy 1, policy_version 1071201 (0.0007) [2023-12-27 13:04:40,680][20381] Updated weights for policy 1, policy_version 1071211 (0.0008) [2023-12-27 13:04:40,740][20381] Updated weights for policy 1, policy_version 1071221 (0.0008) [2023-12-27 13:04:41,152][20347] Updated weights for policy 0, policy_version 1072202 (0.0007) [2023-12-27 13:04:41,210][20347] Updated weights for policy 0, policy_version 1072212 (0.0007) [2023-12-27 13:04:41,270][20347] Updated weights for policy 0, policy_version 1072222 (0.0007) [2023-12-27 13:04:41,336][20347] Updated weights for policy 0, policy_version 1072232 (0.0008) [2023-12-27 13:04:41,514][20381] Updated weights for policy 1, policy_version 1071231 (0.0008) [2023-12-27 13:04:41,569][20381] Updated weights for policy 1, policy_version 1071241 (0.0007) [2023-12-27 13:04:41,630][20381] Updated weights for policy 1, policy_version 1071251 (0.0008) [2023-12-27 13:04:42,087][20347] Updated weights for policy 0, policy_version 1072242 (0.0008) [2023-12-27 13:04:42,137][20347] Updated weights for policy 0, policy_version 1072252 (0.0010) [2023-12-27 13:04:42,187][20347] Updated weights for policy 0, policy_version 1072262 (0.0008) [2023-12-27 13:04:42,267][20381] Updated weights for policy 1, policy_version 1071261 (0.0008) [2023-12-27 13:04:42,335][20381] Updated weights for policy 1, policy_version 1071271 (0.0008) [2023-12-27 13:04:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 548814848. Throughput: 0: 10367.3, 1: 10084.4. Samples: 548806828. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:42,363][19170] Avg episode reward: [(0, '951.010'), (1, '686.930')] [2023-12-27 13:04:42,400][20381] Updated weights for policy 1, policy_version 1071281 (0.0008) [2023-12-27 13:04:42,955][20347] Updated weights for policy 0, policy_version 1072272 (0.0007) [2023-12-27 13:04:43,005][20347] Updated weights for policy 0, policy_version 1072282 (0.0008) [2023-12-27 13:04:43,056][20347] Updated weights for policy 0, policy_version 1072292 (0.0007) [2023-12-27 13:04:43,084][20381] Updated weights for policy 1, policy_version 1071291 (0.0008) [2023-12-27 13:04:43,130][20381] Updated weights for policy 1, policy_version 1071301 (0.0008) [2023-12-27 13:04:43,186][20381] Updated weights for policy 1, policy_version 1071311 (0.0008) [2023-12-27 13:04:43,779][20347] Updated weights for policy 0, policy_version 1072302 (0.0007) [2023-12-27 13:04:43,822][20381] Updated weights for policy 1, policy_version 1071321 (0.0008) [2023-12-27 13:04:43,835][20347] Updated weights for policy 0, policy_version 1072312 (0.0007) [2023-12-27 13:04:43,867][20381] Updated weights for policy 1, policy_version 1071331 (0.0005) [2023-12-27 13:04:43,889][20347] Updated weights for policy 0, policy_version 1072322 (0.0006) [2023-12-27 13:04:43,913][20381] Updated weights for policy 1, policy_version 1071341 (0.0008) [2023-12-27 13:04:43,970][20381] Updated weights for policy 1, policy_version 1071351 (0.0009) [2023-12-27 13:04:44,589][20347] Updated weights for policy 0, policy_version 1072332 (0.0006) [2023-12-27 13:04:44,637][20381] Updated weights for policy 1, policy_version 1071361 (0.0009) [2023-12-27 13:04:44,645][20347] Updated weights for policy 0, policy_version 1072342 (0.0006) [2023-12-27 13:04:44,687][20381] Updated weights for policy 1, policy_version 1071371 (0.0006) [2023-12-27 13:04:44,693][20347] Updated weights for policy 0, policy_version 1072352 (0.0007) [2023-12-27 13:04:44,744][20381] Updated weights for policy 1, policy_version 1071381 (0.0006) [2023-12-27 13:04:45,415][20381] Updated weights for policy 1, policy_version 1071391 (0.0008) [2023-12-27 13:04:45,453][20347] Updated weights for policy 0, policy_version 1072362 (0.0007) [2023-12-27 13:04:45,473][20381] Updated weights for policy 1, policy_version 1071401 (0.0007) [2023-12-27 13:04:45,508][20347] Updated weights for policy 0, policy_version 1072372 (0.0006) [2023-12-27 13:04:45,532][20381] Updated weights for policy 1, policy_version 1071411 (0.0006) [2023-12-27 13:04:45,564][20347] Updated weights for policy 0, policy_version 1072382 (0.0006) [2023-12-27 13:04:45,612][20347] Updated weights for policy 0, policy_version 1072392 (0.0007) [2023-12-27 13:04:46,209][20381] Updated weights for policy 1, policy_version 1071421 (0.0007) [2023-12-27 13:04:46,265][20381] Updated weights for policy 1, policy_version 1071431 (0.0007) [2023-12-27 13:04:46,309][20347] Updated weights for policy 0, policy_version 1072402 (0.0007) [2023-12-27 13:04:46,318][20381] Updated weights for policy 1, policy_version 1071441 (0.0007) [2023-12-27 13:04:46,358][20347] Updated weights for policy 0, policy_version 1072412 (0.0005) [2023-12-27 13:04:46,409][20347] Updated weights for policy 0, policy_version 1072422 (0.0008) [2023-12-27 13:04:46,978][20381] Updated weights for policy 1, policy_version 1071451 (0.0008) [2023-12-27 13:04:47,025][20381] Updated weights for policy 1, policy_version 1071461 (0.0008) [2023-12-27 13:04:47,081][20381] Updated weights for policy 1, policy_version 1071471 (0.0008) [2023-12-27 13:04:47,135][20347] Updated weights for policy 0, policy_version 1072432 (0.0007) [2023-12-27 13:04:47,179][20347] Updated weights for policy 0, policy_version 1072442 (0.0007) [2023-12-27 13:04:47,227][20347] Updated weights for policy 0, policy_version 1072452 (0.0007) [2023-12-27 13:04:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 548921344. Throughput: 0: 10232.7, 1: 10175.4. Samples: 548927260. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:47,363][19170] Avg episode reward: [(0, '941.370'), (1, '569.450')] [2023-12-27 13:04:47,788][20381] Updated weights for policy 1, policy_version 1071481 (0.0007) [2023-12-27 13:04:47,843][20381] Updated weights for policy 1, policy_version 1071491 (0.0007) [2023-12-27 13:04:47,867][20347] Updated weights for policy 0, policy_version 1072462 (0.0006) [2023-12-27 13:04:47,888][20381] Updated weights for policy 1, policy_version 1071501 (0.0006) [2023-12-27 13:04:47,918][20347] Updated weights for policy 0, policy_version 1072472 (0.0007) [2023-12-27 13:04:47,946][20381] Updated weights for policy 1, policy_version 1071511 (0.0007) [2023-12-27 13:04:47,964][20347] Updated weights for policy 0, policy_version 1072482 (0.0006) [2023-12-27 13:04:48,647][20381] Updated weights for policy 1, policy_version 1071521 (0.0007) [2023-12-27 13:04:48,686][20347] Updated weights for policy 0, policy_version 1072492 (0.0007) [2023-12-27 13:04:48,703][20381] Updated weights for policy 1, policy_version 1071531 (0.0006) [2023-12-27 13:04:48,743][20347] Updated weights for policy 0, policy_version 1072502 (0.0007) [2023-12-27 13:04:48,753][20381] Updated weights for policy 1, policy_version 1071541 (0.0005) [2023-12-27 13:04:48,794][20347] Updated weights for policy 0, policy_version 1072512 (0.0007) [2023-12-27 13:04:49,488][20381] Updated weights for policy 1, policy_version 1071551 (0.0006) [2023-12-27 13:04:49,544][20381] Updated weights for policy 1, policy_version 1071561 (0.0009) [2023-12-27 13:04:49,555][20347] Updated weights for policy 0, policy_version 1072522 (0.0008) [2023-12-27 13:04:49,600][20381] Updated weights for policy 1, policy_version 1071571 (0.0010) [2023-12-27 13:04:49,615][20347] Updated weights for policy 0, policy_version 1072532 (0.0008) [2023-12-27 13:04:49,673][20347] Updated weights for policy 0, policy_version 1072542 (0.0008) [2023-12-27 13:04:49,734][20347] Updated weights for policy 0, policy_version 1072552 (0.0008) [2023-12-27 13:04:50,254][20381] Updated weights for policy 1, policy_version 1071581 (0.0009) [2023-12-27 13:04:50,314][20381] Updated weights for policy 1, policy_version 1071591 (0.0011) [2023-12-27 13:04:50,370][20381] Updated weights for policy 1, policy_version 1071601 (0.0011) [2023-12-27 13:04:50,476][20347] Updated weights for policy 0, policy_version 1072562 (0.0007) [2023-12-27 13:04:50,520][20347] Updated weights for policy 0, policy_version 1072572 (0.0007) [2023-12-27 13:04:50,569][20347] Updated weights for policy 0, policy_version 1072582 (0.0007) [2023-12-27 13:04:51,093][20381] Updated weights for policy 1, policy_version 1071611 (0.0011) [2023-12-27 13:04:51,158][20381] Updated weights for policy 1, policy_version 1071621 (0.0011) [2023-12-27 13:04:51,217][20381] Updated weights for policy 1, policy_version 1071631 (0.0011) [2023-12-27 13:04:51,310][20347] Updated weights for policy 0, policy_version 1072592 (0.0008) [2023-12-27 13:04:51,378][20347] Updated weights for policy 0, policy_version 1072602 (0.0007) [2023-12-27 13:04:51,446][20347] Updated weights for policy 0, policy_version 1072612 (0.0007) [2023-12-27 13:04:51,994][20381] Updated weights for policy 1, policy_version 1071641 (0.0009) [2023-12-27 13:04:52,057][20381] Updated weights for policy 1, policy_version 1071651 (0.0008) [2023-12-27 13:04:52,117][20381] Updated weights for policy 1, policy_version 1071661 (0.0007) [2023-12-27 13:04:52,124][20347] Updated weights for policy 0, policy_version 1072622 (0.0008) [2023-12-27 13:04:52,174][20347] Updated weights for policy 0, policy_version 1072632 (0.0007) [2023-12-27 13:04:52,185][20381] Updated weights for policy 1, policy_version 1071671 (0.0007) [2023-12-27 13:04:52,234][20347] Updated weights for policy 0, policy_version 1072642 (0.0007) [2023-12-27 13:04:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 549019648. Throughput: 0: 10150.7, 1: 10202.4. Samples: 548987336. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:52,363][19170] Avg episode reward: [(0, '980.570'), (1, '579.100')] [2023-12-27 13:04:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001071672_274374656.pth... [2023-12-27 13:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001072648_274644992.pth... [2023-12-27 13:04:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001070488_274071552.pth [2023-12-27 13:04:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001071432_274333696.pth [2023-12-27 13:04:52,860][20381] Updated weights for policy 1, policy_version 1071681 (0.0007) [2023-12-27 13:04:52,920][20381] Updated weights for policy 1, policy_version 1071691 (0.0008) [2023-12-27 13:04:52,929][20347] Updated weights for policy 0, policy_version 1072652 (0.0008) [2023-12-27 13:04:52,967][20381] Updated weights for policy 1, policy_version 1071701 (0.0006) [2023-12-27 13:04:52,975][20347] Updated weights for policy 0, policy_version 1072662 (0.0008) [2023-12-27 13:04:53,023][20347] Updated weights for policy 0, policy_version 1072672 (0.0007) [2023-12-27 13:04:53,639][20381] Updated weights for policy 1, policy_version 1071711 (0.0008) [2023-12-27 13:04:53,686][20381] Updated weights for policy 1, policy_version 1071721 (0.0008) [2023-12-27 13:04:53,699][20347] Updated weights for policy 0, policy_version 1072682 (0.0008) [2023-12-27 13:04:53,749][20381] Updated weights for policy 1, policy_version 1071731 (0.0008) [2023-12-27 13:04:53,754][20347] Updated weights for policy 0, policy_version 1072692 (0.0005) [2023-12-27 13:04:53,803][20347] Updated weights for policy 0, policy_version 1072702 (0.0005) [2023-12-27 13:04:53,855][20347] Updated weights for policy 0, policy_version 1072712 (0.0005) [2023-12-27 13:04:54,456][20381] Updated weights for policy 1, policy_version 1071741 (0.0007) [2023-12-27 13:04:54,506][20381] Updated weights for policy 1, policy_version 1071751 (0.0006) [2023-12-27 13:04:54,523][20347] Updated weights for policy 0, policy_version 1072722 (0.0007) [2023-12-27 13:04:54,557][20381] Updated weights for policy 1, policy_version 1071761 (0.0006) [2023-12-27 13:04:54,579][20347] Updated weights for policy 0, policy_version 1072732 (0.0007) [2023-12-27 13:04:54,626][20347] Updated weights for policy 0, policy_version 1072742 (0.0007) [2023-12-27 13:04:55,278][20381] Updated weights for policy 1, policy_version 1071771 (0.0009) [2023-12-27 13:04:55,319][20347] Updated weights for policy 0, policy_version 1072752 (0.0007) [2023-12-27 13:04:55,328][20381] Updated weights for policy 1, policy_version 1071781 (0.0007) [2023-12-27 13:04:55,376][20381] Updated weights for policy 1, policy_version 1071791 (0.0006) [2023-12-27 13:04:55,377][20347] Updated weights for policy 0, policy_version 1072762 (0.0007) [2023-12-27 13:04:55,428][20347] Updated weights for policy 0, policy_version 1072772 (0.0006) [2023-12-27 13:04:56,057][20381] Updated weights for policy 1, policy_version 1071801 (0.0007) [2023-12-27 13:04:56,090][20347] Updated weights for policy 0, policy_version 1072782 (0.0009) [2023-12-27 13:04:56,114][20381] Updated weights for policy 1, policy_version 1071811 (0.0007) [2023-12-27 13:04:56,148][20347] Updated weights for policy 0, policy_version 1072792 (0.0007) [2023-12-27 13:04:56,172][20381] Updated weights for policy 1, policy_version 1071821 (0.0007) [2023-12-27 13:04:56,206][20347] Updated weights for policy 0, policy_version 1072802 (0.0008) [2023-12-27 13:04:56,220][20381] Updated weights for policy 1, policy_version 1071831 (0.0008) [2023-12-27 13:04:56,790][20381] Updated weights for policy 1, policy_version 1071841 (0.0007) [2023-12-27 13:04:56,839][20381] Updated weights for policy 1, policy_version 1071851 (0.0008) [2023-12-27 13:04:56,895][20381] Updated weights for policy 1, policy_version 1071861 (0.0007) [2023-12-27 13:04:56,942][20347] Updated weights for policy 0, policy_version 1072812 (0.0008) [2023-12-27 13:04:57,000][20347] Updated weights for policy 0, policy_version 1072822 (0.0008) [2023-12-27 13:04:57,046][20347] Updated weights for policy 0, policy_version 1072832 (0.0007) [2023-12-27 13:04:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 549117952. Throughput: 0: 10162.4, 1: 10170.6. Samples: 549107228. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:04:57,363][19170] Avg episode reward: [(0, '980.530'), (1, '667.440')] [2023-12-27 13:04:57,527][20381] Updated weights for policy 1, policy_version 1071871 (0.0005) [2023-12-27 13:04:57,578][20381] Updated weights for policy 1, policy_version 1071881 (0.0005) [2023-12-27 13:04:57,623][20381] Updated weights for policy 1, policy_version 1071891 (0.0006) [2023-12-27 13:04:57,719][20347] Updated weights for policy 0, policy_version 1072842 (0.0007) [2023-12-27 13:04:57,768][20347] Updated weights for policy 0, policy_version 1072852 (0.0008) [2023-12-27 13:04:57,817][20347] Updated weights for policy 0, policy_version 1072862 (0.0008) [2023-12-27 13:04:57,867][20347] Updated weights for policy 0, policy_version 1072872 (0.0007) [2023-12-27 13:04:58,289][20381] Updated weights for policy 1, policy_version 1071901 (0.0008) [2023-12-27 13:04:58,376][20381] Updated weights for policy 1, policy_version 1071911 (0.0008) [2023-12-27 13:04:58,444][20381] Updated weights for policy 1, policy_version 1071921 (0.0008) [2023-12-27 13:04:58,619][20347] Updated weights for policy 0, policy_version 1072882 (0.0008) [2023-12-27 13:04:58,680][20347] Updated weights for policy 0, policy_version 1072892 (0.0007) [2023-12-27 13:04:58,748][20347] Updated weights for policy 0, policy_version 1072902 (0.0011) [2023-12-27 13:04:59,197][20381] Updated weights for policy 1, policy_version 1071931 (0.0009) [2023-12-27 13:04:59,280][20381] Updated weights for policy 1, policy_version 1071942 (0.0008) [2023-12-27 13:04:59,344][20381] Updated weights for policy 1, policy_version 1071952 (0.0008) [2023-12-27 13:04:59,515][20347] Updated weights for policy 0, policy_version 1072912 (0.0006) [2023-12-27 13:04:59,570][20347] Updated weights for policy 0, policy_version 1072922 (0.0007) [2023-12-27 13:04:59,622][20347] Updated weights for policy 0, policy_version 1072932 (0.0007) [2023-12-27 13:05:00,086][20381] Updated weights for policy 1, policy_version 1071962 (0.0010) [2023-12-27 13:05:00,147][20381] Updated weights for policy 1, policy_version 1071972 (0.0010) [2023-12-27 13:05:00,206][20381] Updated weights for policy 1, policy_version 1071982 (0.0010) [2023-12-27 13:05:00,268][20381] Updated weights for policy 1, policy_version 1071992 (0.0011) [2023-12-27 13:05:00,335][20347] Updated weights for policy 0, policy_version 1072942 (0.0007) [2023-12-27 13:05:00,387][20347] Updated weights for policy 0, policy_version 1072952 (0.0008) [2023-12-27 13:05:00,445][20347] Updated weights for policy 0, policy_version 1072962 (0.0008) [2023-12-27 13:05:00,935][20381] Updated weights for policy 1, policy_version 1072002 (0.0005) [2023-12-27 13:05:00,988][20381] Updated weights for policy 1, policy_version 1072012 (0.0005) [2023-12-27 13:05:01,046][20381] Updated weights for policy 1, policy_version 1072022 (0.0007) [2023-12-27 13:05:01,116][20347] Updated weights for policy 0, policy_version 1072972 (0.0006) [2023-12-27 13:05:01,186][20347] Updated weights for policy 0, policy_version 1072982 (0.0008) [2023-12-27 13:05:01,245][20347] Updated weights for policy 0, policy_version 1072992 (0.0008) [2023-12-27 13:05:01,699][20381] Updated weights for policy 1, policy_version 1072032 (0.0007) [2023-12-27 13:05:01,766][20381] Updated weights for policy 1, policy_version 1072042 (0.0008) [2023-12-27 13:05:01,818][20381] Updated weights for policy 1, policy_version 1072052 (0.0006) [2023-12-27 13:05:01,987][20347] Updated weights for policy 0, policy_version 1073002 (0.0008) [2023-12-27 13:05:02,044][20347] Updated weights for policy 0, policy_version 1073012 (0.0009) [2023-12-27 13:05:02,098][20347] Updated weights for policy 0, policy_version 1073022 (0.0008) [2023-12-27 13:05:02,156][20347] Updated weights for policy 0, policy_version 1073032 (0.0008) [2023-12-27 13:05:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 549216256. Throughput: 0: 10135.9, 1: 10133.5. Samples: 549227424. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '745.430')] [2023-12-27 13:05:02,437][20381] Updated weights for policy 1, policy_version 1072062 (0.0005) [2023-12-27 13:05:02,493][20381] Updated weights for policy 1, policy_version 1072072 (0.0007) [2023-12-27 13:05:02,545][20381] Updated weights for policy 1, policy_version 1072082 (0.0006) [2023-12-27 13:05:02,871][20347] Updated weights for policy 0, policy_version 1073042 (0.0008) [2023-12-27 13:05:02,918][20347] Updated weights for policy 0, policy_version 1073052 (0.0007) [2023-12-27 13:05:02,974][20347] Updated weights for policy 0, policy_version 1073062 (0.0007) [2023-12-27 13:05:03,119][20381] Updated weights for policy 1, policy_version 1072092 (0.0006) [2023-12-27 13:05:03,179][20381] Updated weights for policy 1, policy_version 1072102 (0.0005) [2023-12-27 13:05:03,240][20381] Updated weights for policy 1, policy_version 1072112 (0.0007) [2023-12-27 13:05:03,635][20347] Updated weights for policy 0, policy_version 1073072 (0.0006) [2023-12-27 13:05:03,686][20347] Updated weights for policy 0, policy_version 1073082 (0.0005) [2023-12-27 13:05:03,732][20347] Updated weights for policy 0, policy_version 1073092 (0.0007) [2023-12-27 13:05:03,898][20381] Updated weights for policy 1, policy_version 1072122 (0.0008) [2023-12-27 13:05:03,959][20381] Updated weights for policy 1, policy_version 1072132 (0.0008) [2023-12-27 13:05:04,014][20381] Updated weights for policy 1, policy_version 1072142 (0.0008) [2023-12-27 13:05:04,066][20381] Updated weights for policy 1, policy_version 1072152 (0.0008) [2023-12-27 13:05:04,400][20347] Updated weights for policy 0, policy_version 1073102 (0.0007) [2023-12-27 13:05:04,456][20347] Updated weights for policy 0, policy_version 1073112 (0.0008) [2023-12-27 13:05:04,510][20347] Updated weights for policy 0, policy_version 1073122 (0.0009) [2023-12-27 13:05:04,708][20381] Updated weights for policy 1, policy_version 1072162 (0.0007) [2023-12-27 13:05:04,766][20381] Updated weights for policy 1, policy_version 1072172 (0.0008) [2023-12-27 13:05:04,821][20381] Updated weights for policy 1, policy_version 1072182 (0.0008) [2023-12-27 13:05:05,130][20347] Updated weights for policy 0, policy_version 1073132 (0.0008) [2023-12-27 13:05:05,184][20347] Updated weights for policy 0, policy_version 1073142 (0.0007) [2023-12-27 13:05:05,239][20347] Updated weights for policy 0, policy_version 1073152 (0.0007) [2023-12-27 13:05:05,532][20381] Updated weights for policy 1, policy_version 1072192 (0.0008) [2023-12-27 13:05:05,581][20381] Updated weights for policy 1, policy_version 1072202 (0.0007) [2023-12-27 13:05:05,637][20381] Updated weights for policy 1, policy_version 1072212 (0.0007) [2023-12-27 13:05:05,896][20347] Updated weights for policy 0, policy_version 1073162 (0.0008) [2023-12-27 13:05:05,951][20347] Updated weights for policy 0, policy_version 1073172 (0.0008) [2023-12-27 13:05:06,003][20347] Updated weights for policy 0, policy_version 1073182 (0.0007) [2023-12-27 13:05:06,058][20347] Updated weights for policy 0, policy_version 1073192 (0.0005) [2023-12-27 13:05:06,345][20381] Updated weights for policy 1, policy_version 1072222 (0.0008) [2023-12-27 13:05:06,406][20381] Updated weights for policy 1, policy_version 1072232 (0.0007) [2023-12-27 13:05:06,468][20381] Updated weights for policy 1, policy_version 1072242 (0.0007) [2023-12-27 13:05:06,723][20347] Updated weights for policy 0, policy_version 1073202 (0.0006) [2023-12-27 13:05:06,777][20347] Updated weights for policy 0, policy_version 1073212 (0.0007) [2023-12-27 13:05:06,832][20347] Updated weights for policy 0, policy_version 1073222 (0.0007) [2023-12-27 13:05:07,162][20381] Updated weights for policy 1, policy_version 1072252 (0.0008) [2023-12-27 13:05:07,220][20381] Updated weights for policy 1, policy_version 1072262 (0.0007) [2023-12-27 13:05:07,282][20381] Updated weights for policy 1, policy_version 1072272 (0.0007) [2023-12-27 13:05:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 549322752. Throughput: 0: 10164.2, 1: 10144.8. Samples: 549290928. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:07,364][19170] Avg episode reward: [(0, '980.420'), (1, '735.670')] [2023-12-27 13:05:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001072280_274530304.pth... [2023-12-27 13:05:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001073224_274792448.pth... [2023-12-27 13:05:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001071096_274227200.pth [2023-12-27 13:05:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001072072_274497536.pth [2023-12-27 13:05:07,450][20347] Updated weights for policy 0, policy_version 1073232 (0.0006) [2023-12-27 13:05:07,457][20291] KL-divergence is very high: 123.3160 [2023-12-27 13:05:07,509][20347] Updated weights for policy 0, policy_version 1073242 (0.0007) [2023-12-27 13:05:07,566][20347] Updated weights for policy 0, policy_version 1073252 (0.0006) [2023-12-27 13:05:07,933][20381] Updated weights for policy 1, policy_version 1072282 (0.0007) [2023-12-27 13:05:07,977][20381] Updated weights for policy 1, policy_version 1072292 (0.0008) [2023-12-27 13:05:08,026][20381] Updated weights for policy 1, policy_version 1072302 (0.0007) [2023-12-27 13:05:08,072][20381] Updated weights for policy 1, policy_version 1072312 (0.0006) [2023-12-27 13:05:08,211][20347] Updated weights for policy 0, policy_version 1073262 (0.0008) [2023-12-27 13:05:08,277][20347] Updated weights for policy 0, policy_version 1073272 (0.0008) [2023-12-27 13:05:08,331][20347] Updated weights for policy 0, policy_version 1073282 (0.0008) [2023-12-27 13:05:08,796][20381] Updated weights for policy 1, policy_version 1072322 (0.0008) [2023-12-27 13:05:08,845][20381] Updated weights for policy 1, policy_version 1072332 (0.0007) [2023-12-27 13:05:08,903][20381] Updated weights for policy 1, policy_version 1072342 (0.0009) [2023-12-27 13:05:09,063][20347] Updated weights for policy 0, policy_version 1073292 (0.0008) [2023-12-27 13:05:09,108][20347] Updated weights for policy 0, policy_version 1073302 (0.0007) [2023-12-27 13:05:09,145][20291] KL-divergence is very high: 116.2846 [2023-12-27 13:05:09,156][20347] Updated weights for policy 0, policy_version 1073312 (0.0007) [2023-12-27 13:05:09,669][20381] Updated weights for policy 1, policy_version 1072352 (0.0008) [2023-12-27 13:05:09,738][20381] Updated weights for policy 1, policy_version 1072362 (0.0007) [2023-12-27 13:05:09,807][20381] Updated weights for policy 1, policy_version 1072372 (0.0008) [2023-12-27 13:05:09,894][20347] Updated weights for policy 0, policy_version 1073322 (0.0008) [2023-12-27 13:05:09,959][20347] Updated weights for policy 0, policy_version 1073332 (0.0008) [2023-12-27 13:05:10,020][20347] Updated weights for policy 0, policy_version 1073342 (0.0008) [2023-12-27 13:05:10,080][20347] Updated weights for policy 0, policy_version 1073352 (0.0008) [2023-12-27 13:05:10,501][20381] Updated weights for policy 1, policy_version 1072382 (0.0008) [2023-12-27 13:05:10,559][20381] Updated weights for policy 1, policy_version 1072392 (0.0007) [2023-12-27 13:05:10,614][20381] Updated weights for policy 1, policy_version 1072402 (0.0007) [2023-12-27 13:05:10,779][20347] Updated weights for policy 0, policy_version 1073362 (0.0008) [2023-12-27 13:05:10,832][20347] Updated weights for policy 0, policy_version 1073372 (0.0008) [2023-12-27 13:05:10,881][20347] Updated weights for policy 0, policy_version 1073382 (0.0007) [2023-12-27 13:05:11,298][20381] Updated weights for policy 1, policy_version 1072412 (0.0007) [2023-12-27 13:05:11,361][20381] Updated weights for policy 1, policy_version 1072422 (0.0009) [2023-12-27 13:05:11,426][20381] Updated weights for policy 1, policy_version 1072432 (0.0008) [2023-12-27 13:05:11,600][20347] Updated weights for policy 0, policy_version 1073392 (0.0008) [2023-12-27 13:05:11,667][20347] Updated weights for policy 0, policy_version 1073402 (0.0008) [2023-12-27 13:05:11,733][20347] Updated weights for policy 0, policy_version 1073412 (0.0008) [2023-12-27 13:05:12,152][20381] Updated weights for policy 1, policy_version 1072442 (0.0007) [2023-12-27 13:05:12,211][20381] Updated weights for policy 1, policy_version 1072452 (0.0008) [2023-12-27 13:05:12,278][20381] Updated weights for policy 1, policy_version 1072462 (0.0008) [2023-12-27 13:05:12,336][20381] Updated weights for policy 1, policy_version 1072472 (0.0008) [2023-12-27 13:05:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 549421056. Throughput: 0: 10177.5, 1: 10143.7. Samples: 549411172. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:12,363][19170] Avg episode reward: [(0, '960.890'), (1, '745.580')] [2023-12-27 13:05:12,397][20347] Updated weights for policy 0, policy_version 1073422 (0.0008) [2023-12-27 13:05:12,457][20347] Updated weights for policy 0, policy_version 1073432 (0.0008) [2023-12-27 13:05:12,515][20347] Updated weights for policy 0, policy_version 1073442 (0.0007) [2023-12-27 13:05:13,042][20381] Updated weights for policy 1, policy_version 1072482 (0.0008) [2023-12-27 13:05:13,095][20381] Updated weights for policy 1, policy_version 1072492 (0.0005) [2023-12-27 13:05:13,144][20381] Updated weights for policy 1, policy_version 1072502 (0.0006) [2023-12-27 13:05:13,217][20347] Updated weights for policy 0, policy_version 1073452 (0.0007) [2023-12-27 13:05:13,270][20347] Updated weights for policy 0, policy_version 1073462 (0.0006) [2023-12-27 13:05:13,314][20347] Updated weights for policy 0, policy_version 1073472 (0.0007) [2023-12-27 13:05:13,740][20381] Updated weights for policy 1, policy_version 1072512 (0.0007) [2023-12-27 13:05:13,797][20381] Updated weights for policy 1, policy_version 1072522 (0.0007) [2023-12-27 13:05:13,854][20381] Updated weights for policy 1, policy_version 1072532 (0.0007) [2023-12-27 13:05:14,016][20347] Updated weights for policy 0, policy_version 1073482 (0.0009) [2023-12-27 13:05:14,072][20347] Updated weights for policy 0, policy_version 1073492 (0.0010) [2023-12-27 13:05:14,136][20347] Updated weights for policy 0, policy_version 1073502 (0.0010) [2023-12-27 13:05:14,190][20347] Updated weights for policy 0, policy_version 1073512 (0.0010) [2023-12-27 13:05:14,482][20381] Updated weights for policy 1, policy_version 1072542 (0.0007) [2023-12-27 13:05:14,539][20381] Updated weights for policy 1, policy_version 1072552 (0.0008) [2023-12-27 13:05:14,596][20381] Updated weights for policy 1, policy_version 1072562 (0.0007) [2023-12-27 13:05:14,876][20347] Updated weights for policy 0, policy_version 1073522 (0.0008) [2023-12-27 13:05:14,939][20347] Updated weights for policy 0, policy_version 1073532 (0.0007) [2023-12-27 13:05:15,000][20347] Updated weights for policy 0, policy_version 1073542 (0.0008) [2023-12-27 13:05:15,321][20381] Updated weights for policy 1, policy_version 1072572 (0.0007) [2023-12-27 13:05:15,377][20381] Updated weights for policy 1, policy_version 1072582 (0.0008) [2023-12-27 13:05:15,438][20381] Updated weights for policy 1, policy_version 1072592 (0.0008) [2023-12-27 13:05:15,605][20347] Updated weights for policy 0, policy_version 1073552 (0.0006) [2023-12-27 13:05:15,651][20347] Updated weights for policy 0, policy_version 1073562 (0.0007) [2023-12-27 13:05:15,699][20347] Updated weights for policy 0, policy_version 1073572 (0.0006) [2023-12-27 13:05:16,071][20381] Updated weights for policy 1, policy_version 1072602 (0.0007) [2023-12-27 13:05:16,121][20381] Updated weights for policy 1, policy_version 1072612 (0.0005) [2023-12-27 13:05:16,177][20381] Updated weights for policy 1, policy_version 1072622 (0.0005) [2023-12-27 13:05:16,225][20381] Updated weights for policy 1, policy_version 1072632 (0.0005) [2023-12-27 13:05:16,261][20347] Updated weights for policy 0, policy_version 1073582 (0.0008) [2023-12-27 13:05:16,315][20347] Updated weights for policy 0, policy_version 1073592 (0.0007) [2023-12-27 13:05:16,365][20347] Updated weights for policy 0, policy_version 1073602 (0.0007) [2023-12-27 13:05:16,735][20381] Updated weights for policy 1, policy_version 1072642 (0.0007) [2023-12-27 13:05:16,795][20381] Updated weights for policy 1, policy_version 1072652 (0.0007) [2023-12-27 13:05:16,853][20381] Updated weights for policy 1, policy_version 1072662 (0.0007) [2023-12-27 13:05:17,104][20347] Updated weights for policy 0, policy_version 1073612 (0.0007) [2023-12-27 13:05:17,156][20347] Updated weights for policy 0, policy_version 1073622 (0.0010) [2023-12-27 13:05:17,203][20347] Updated weights for policy 0, policy_version 1073632 (0.0010) [2023-12-27 13:05:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 549527552. Throughput: 0: 10148.1, 1: 10194.1. Samples: 549535856. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:17,363][19170] Avg episode reward: [(0, '970.720'), (1, '745.610')] [2023-12-27 13:05:17,526][20381] Updated weights for policy 1, policy_version 1072672 (0.0007) [2023-12-27 13:05:17,580][20381] Updated weights for policy 1, policy_version 1072682 (0.0008) [2023-12-27 13:05:17,640][20381] Updated weights for policy 1, policy_version 1072692 (0.0007) [2023-12-27 13:05:17,896][20347] Updated weights for policy 0, policy_version 1073642 (0.0010) [2023-12-27 13:05:17,956][20347] Updated weights for policy 0, policy_version 1073652 (0.0010) [2023-12-27 13:05:18,007][20347] Updated weights for policy 0, policy_version 1073662 (0.0010) [2023-12-27 13:05:18,062][20347] Updated weights for policy 0, policy_version 1073672 (0.0010) [2023-12-27 13:05:18,308][20381] Updated weights for policy 1, policy_version 1072702 (0.0008) [2023-12-27 13:05:18,373][20381] Updated weights for policy 1, policy_version 1072712 (0.0008) [2023-12-27 13:05:18,428][20381] Updated weights for policy 1, policy_version 1072722 (0.0008) [2023-12-27 13:05:18,702][20347] Updated weights for policy 0, policy_version 1073682 (0.0005) [2023-12-27 13:05:18,763][20347] Updated weights for policy 0, policy_version 1073692 (0.0007) [2023-12-27 13:05:18,823][20347] Updated weights for policy 0, policy_version 1073702 (0.0009) [2023-12-27 13:05:19,122][20381] Updated weights for policy 1, policy_version 1072732 (0.0008) [2023-12-27 13:05:19,169][20381] Updated weights for policy 1, policy_version 1072742 (0.0008) [2023-12-27 13:05:19,221][20381] Updated weights for policy 1, policy_version 1072752 (0.0008) [2023-12-27 13:05:19,491][20347] Updated weights for policy 0, policy_version 1073712 (0.0008) [2023-12-27 13:05:19,549][20347] Updated weights for policy 0, policy_version 1073722 (0.0008) [2023-12-27 13:05:19,606][20347] Updated weights for policy 0, policy_version 1073732 (0.0008) [2023-12-27 13:05:19,951][20381] Updated weights for policy 1, policy_version 1072762 (0.0009) [2023-12-27 13:05:20,008][20381] Updated weights for policy 1, policy_version 1072772 (0.0007) [2023-12-27 13:05:20,070][20381] Updated weights for policy 1, policy_version 1072782 (0.0007) [2023-12-27 13:05:20,138][20381] Updated weights for policy 1, policy_version 1072792 (0.0007) [2023-12-27 13:05:20,363][20347] Updated weights for policy 0, policy_version 1073742 (0.0008) [2023-12-27 13:05:20,427][20347] Updated weights for policy 0, policy_version 1073752 (0.0008) [2023-12-27 13:05:20,485][20347] Updated weights for policy 0, policy_version 1073762 (0.0008) [2023-12-27 13:05:20,783][20381] Updated weights for policy 1, policy_version 1072802 (0.0006) [2023-12-27 13:05:20,845][20381] Updated weights for policy 1, policy_version 1072812 (0.0005) [2023-12-27 13:05:20,895][20381] Updated weights for policy 1, policy_version 1072822 (0.0005) [2023-12-27 13:05:21,232][20347] Updated weights for policy 0, policy_version 1073772 (0.0007) [2023-12-27 13:05:21,296][20347] Updated weights for policy 0, policy_version 1073782 (0.0008) [2023-12-27 13:05:21,356][20347] Updated weights for policy 0, policy_version 1073792 (0.0007) [2023-12-27 13:05:21,702][20381] Updated weights for policy 1, policy_version 1072832 (0.0007) [2023-12-27 13:05:21,770][20381] Updated weights for policy 1, policy_version 1072842 (0.0009) [2023-12-27 13:05:21,839][20381] Updated weights for policy 1, policy_version 1072852 (0.0008) [2023-12-27 13:05:22,043][20347] Updated weights for policy 0, policy_version 1073802 (0.0007) [2023-12-27 13:05:22,103][20347] Updated weights for policy 0, policy_version 1073812 (0.0006) [2023-12-27 13:05:22,164][20347] Updated weights for policy 0, policy_version 1073822 (0.0006) [2023-12-27 13:05:22,219][20347] Updated weights for policy 0, policy_version 1073832 (0.0006) [2023-12-27 13:05:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 549625856. Throughput: 0: 10094.0, 1: 10144.3. Samples: 549596708. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:22,363][19170] Avg episode reward: [(0, '960.980'), (1, '774.650')] [2023-12-27 13:05:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001072856_274677760.pth... [2023-12-27 13:05:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001073832_274948096.pth... [2023-12-27 13:05:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001071672_274374656.pth [2023-12-27 13:05:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001072648_274644992.pth [2023-12-27 13:05:22,573][20381] Updated weights for policy 1, policy_version 1072862 (0.0007) [2023-12-27 13:05:22,633][20381] Updated weights for policy 1, policy_version 1072872 (0.0008) [2023-12-27 13:05:22,696][20381] Updated weights for policy 1, policy_version 1072882 (0.0006) [2023-12-27 13:05:22,892][20347] Updated weights for policy 0, policy_version 1073842 (0.0007) [2023-12-27 13:05:22,944][20347] Updated weights for policy 0, policy_version 1073852 (0.0008) [2023-12-27 13:05:22,989][20347] Updated weights for policy 0, policy_version 1073862 (0.0008) [2023-12-27 13:05:23,272][20381] Updated weights for policy 1, policy_version 1072892 (0.0005) [2023-12-27 13:05:23,331][20381] Updated weights for policy 1, policy_version 1072902 (0.0005) [2023-12-27 13:05:23,391][20381] Updated weights for policy 1, policy_version 1072912 (0.0005) [2023-12-27 13:05:23,737][20347] Updated weights for policy 0, policy_version 1073872 (0.0008) [2023-12-27 13:05:23,790][20347] Updated weights for policy 0, policy_version 1073882 (0.0006) [2023-12-27 13:05:23,839][20347] Updated weights for policy 0, policy_version 1073892 (0.0005) [2023-12-27 13:05:23,995][20381] Updated weights for policy 1, policy_version 1072922 (0.0010) [2023-12-27 13:05:24,049][20381] Updated weights for policy 1, policy_version 1072932 (0.0008) [2023-12-27 13:05:24,106][20381] Updated weights for policy 1, policy_version 1072942 (0.0008) [2023-12-27 13:05:24,160][20381] Updated weights for policy 1, policy_version 1072952 (0.0008) [2023-12-27 13:05:24,504][20347] Updated weights for policy 0, policy_version 1073902 (0.0007) [2023-12-27 13:05:24,562][20347] Updated weights for policy 0, policy_version 1073912 (0.0008) [2023-12-27 13:05:24,619][20347] Updated weights for policy 0, policy_version 1073922 (0.0007) [2023-12-27 13:05:24,847][20381] Updated weights for policy 1, policy_version 1072962 (0.0007) [2023-12-27 13:05:24,895][20381] Updated weights for policy 1, policy_version 1072972 (0.0005) [2023-12-27 13:05:24,951][20381] Updated weights for policy 1, policy_version 1072982 (0.0005) [2023-12-27 13:05:25,292][20347] Updated weights for policy 0, policy_version 1073932 (0.0007) [2023-12-27 13:05:25,357][20347] Updated weights for policy 0, policy_version 1073942 (0.0008) [2023-12-27 13:05:25,421][20347] Updated weights for policy 0, policy_version 1073952 (0.0008) [2023-12-27 13:05:25,613][20381] Updated weights for policy 1, policy_version 1072992 (0.0007) [2023-12-27 13:05:25,668][20381] Updated weights for policy 1, policy_version 1073002 (0.0007) [2023-12-27 13:05:25,723][20381] Updated weights for policy 1, policy_version 1073012 (0.0009) [2023-12-27 13:05:25,965][20347] Updated weights for policy 0, policy_version 1073962 (0.0006) [2023-12-27 13:05:26,032][20347] Updated weights for policy 0, policy_version 1073972 (0.0008) [2023-12-27 13:05:26,097][20347] Updated weights for policy 0, policy_version 1073982 (0.0007) [2023-12-27 13:05:26,152][20347] Updated weights for policy 0, policy_version 1073992 (0.0006) [2023-12-27 13:05:26,414][20381] Updated weights for policy 1, policy_version 1073022 (0.0007) [2023-12-27 13:05:26,470][20381] Updated weights for policy 1, policy_version 1073032 (0.0010) [2023-12-27 13:05:26,525][20381] Updated weights for policy 1, policy_version 1073042 (0.0010) [2023-12-27 13:05:26,783][20347] Updated weights for policy 0, policy_version 1074002 (0.0005) [2023-12-27 13:05:26,835][20347] Updated weights for policy 0, policy_version 1074012 (0.0007) [2023-12-27 13:05:26,892][20347] Updated weights for policy 0, policy_version 1074022 (0.0010) [2023-12-27 13:05:27,107][20381] Updated weights for policy 1, policy_version 1073052 (0.0008) [2023-12-27 13:05:27,150][20381] Updated weights for policy 1, policy_version 1073062 (0.0005) [2023-12-27 13:05:27,196][20381] Updated weights for policy 1, policy_version 1073072 (0.0005) [2023-12-27 13:05:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 549732352. Throughput: 0: 10039.2, 1: 10230.5. Samples: 549718964. Policy #0 lag: (min: 17.0, avg: 32.5, max: 49.0) [2023-12-27 13:05:27,363][19170] Avg episode reward: [(0, '970.680'), (1, '804.220')] [2023-12-27 13:05:27,482][20347] Updated weights for policy 0, policy_version 1074032 (0.0010) [2023-12-27 13:05:27,538][20347] Updated weights for policy 0, policy_version 1074042 (0.0010) [2023-12-27 13:05:27,583][20347] Updated weights for policy 0, policy_version 1074052 (0.0010) [2023-12-27 13:05:27,810][20381] Updated weights for policy 1, policy_version 1073082 (0.0005) [2023-12-27 13:05:27,861][20381] Updated weights for policy 1, policy_version 1073092 (0.0005) [2023-12-27 13:05:27,912][20381] Updated weights for policy 1, policy_version 1073102 (0.0005) [2023-12-27 13:05:27,968][20381] Updated weights for policy 1, policy_version 1073112 (0.0005) [2023-12-27 13:05:28,266][20347] Updated weights for policy 0, policy_version 1074062 (0.0010) [2023-12-27 13:05:28,325][20347] Updated weights for policy 0, policy_version 1074072 (0.0010) [2023-12-27 13:05:28,388][20347] Updated weights for policy 0, policy_version 1074082 (0.0010) [2023-12-27 13:05:28,613][20381] Updated weights for policy 1, policy_version 1073122 (0.0011) [2023-12-27 13:05:28,663][20381] Updated weights for policy 1, policy_version 1073132 (0.0010) [2023-12-27 13:05:28,713][20381] Updated weights for policy 1, policy_version 1073142 (0.0011) [2023-12-27 13:05:29,092][20347] Updated weights for policy 0, policy_version 1074092 (0.0010) [2023-12-27 13:05:29,146][20347] Updated weights for policy 0, policy_version 1074102 (0.0010) [2023-12-27 13:05:29,191][20347] Updated weights for policy 0, policy_version 1074112 (0.0010) [2023-12-27 13:05:29,387][20381] Updated weights for policy 1, policy_version 1073152 (0.0008) [2023-12-27 13:05:29,443][20381] Updated weights for policy 1, policy_version 1073162 (0.0005) [2023-12-27 13:05:29,502][20381] Updated weights for policy 1, policy_version 1073172 (0.0006) [2023-12-27 13:05:29,867][20347] Updated weights for policy 0, policy_version 1074122 (0.0009) [2023-12-27 13:05:29,929][20347] Updated weights for policy 0, policy_version 1074132 (0.0008) [2023-12-27 13:05:29,994][20347] Updated weights for policy 0, policy_version 1074142 (0.0007) [2023-12-27 13:05:30,047][20347] Updated weights for policy 0, policy_version 1074152 (0.0008) [2023-12-27 13:05:30,142][20381] Updated weights for policy 1, policy_version 1073182 (0.0009) [2023-12-27 13:05:30,198][20381] Updated weights for policy 1, policy_version 1073192 (0.0011) [2023-12-27 13:05:30,255][20381] Updated weights for policy 1, policy_version 1073202 (0.0011) [2023-12-27 13:05:30,769][20347] Updated weights for policy 0, policy_version 1074162 (0.0007) [2023-12-27 13:05:30,818][20347] Updated weights for policy 0, policy_version 1074172 (0.0008) [2023-12-27 13:05:30,863][20347] Updated weights for policy 0, policy_version 1074182 (0.0007) [2023-12-27 13:05:30,922][20381] Updated weights for policy 1, policy_version 1073212 (0.0008) [2023-12-27 13:05:30,983][20381] Updated weights for policy 1, policy_version 1073222 (0.0006) [2023-12-27 13:05:31,050][20381] Updated weights for policy 1, policy_version 1073232 (0.0007) [2023-12-27 13:05:31,599][20347] Updated weights for policy 0, policy_version 1074192 (0.0008) [2023-12-27 13:05:31,666][20347] Updated weights for policy 0, policy_version 1074202 (0.0008) [2023-12-27 13:05:31,721][20381] Updated weights for policy 1, policy_version 1073242 (0.0008) [2023-12-27 13:05:31,731][20347] Updated weights for policy 0, policy_version 1074212 (0.0007) [2023-12-27 13:05:31,784][20381] Updated weights for policy 1, policy_version 1073252 (0.0008) [2023-12-27 13:05:31,842][20381] Updated weights for policy 1, policy_version 1073262 (0.0008) [2023-12-27 13:05:31,899][20381] Updated weights for policy 1, policy_version 1073272 (0.0008) [2023-12-27 13:05:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 549830656. Throughput: 0: 10111.7, 1: 10258.0. Samples: 549843896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:32,363][19170] Avg episode reward: [(0, '990.220'), (1, '735.770')] [2023-12-27 13:05:32,437][20347] Updated weights for policy 0, policy_version 1074222 (0.0007) [2023-12-27 13:05:32,493][20347] Updated weights for policy 0, policy_version 1074232 (0.0007) [2023-12-27 13:05:32,549][20347] Updated weights for policy 0, policy_version 1074242 (0.0006) [2023-12-27 13:05:32,589][20381] Updated weights for policy 1, policy_version 1073282 (0.0008) [2023-12-27 13:05:32,635][20381] Updated weights for policy 1, policy_version 1073292 (0.0008) [2023-12-27 13:05:32,686][20381] Updated weights for policy 1, policy_version 1073302 (0.0008) [2023-12-27 13:05:33,228][20347] Updated weights for policy 0, policy_version 1074252 (0.0007) [2023-12-27 13:05:33,269][20347] Updated weights for policy 0, policy_version 1074262 (0.0007) [2023-12-27 13:05:33,317][20347] Updated weights for policy 0, policy_version 1074272 (0.0007) [2023-12-27 13:05:33,413][20381] Updated weights for policy 1, policy_version 1073312 (0.0008) [2023-12-27 13:05:33,462][20381] Updated weights for policy 1, policy_version 1073322 (0.0007) [2023-12-27 13:05:33,518][20381] Updated weights for policy 1, policy_version 1073332 (0.0007) [2023-12-27 13:05:33,966][20347] Updated weights for policy 0, policy_version 1074282 (0.0008) [2023-12-27 13:05:34,014][20347] Updated weights for policy 0, policy_version 1074292 (0.0006) [2023-12-27 13:05:34,059][20347] Updated weights for policy 0, policy_version 1074302 (0.0008) [2023-12-27 13:05:34,114][20347] Updated weights for policy 0, policy_version 1074312 (0.0008) [2023-12-27 13:05:34,210][20381] Updated weights for policy 1, policy_version 1073342 (0.0008) [2023-12-27 13:05:34,265][20381] Updated weights for policy 1, policy_version 1073352 (0.0008) [2023-12-27 13:05:34,333][20381] Updated weights for policy 1, policy_version 1073362 (0.0008) [2023-12-27 13:05:34,788][20347] Updated weights for policy 0, policy_version 1074322 (0.0010) [2023-12-27 13:05:34,849][20347] Updated weights for policy 0, policy_version 1074332 (0.0010) [2023-12-27 13:05:34,900][20347] Updated weights for policy 0, policy_version 1074342 (0.0010) [2023-12-27 13:05:35,059][20381] Updated weights for policy 1, policy_version 1073372 (0.0009) [2023-12-27 13:05:35,107][20381] Updated weights for policy 1, policy_version 1073382 (0.0010) [2023-12-27 13:05:35,155][20381] Updated weights for policy 1, policy_version 1073392 (0.0010) [2023-12-27 13:05:35,580][20347] Updated weights for policy 0, policy_version 1074352 (0.0010) [2023-12-27 13:05:35,636][20347] Updated weights for policy 0, policy_version 1074362 (0.0010) [2023-12-27 13:05:35,692][20347] Updated weights for policy 0, policy_version 1074372 (0.0010) [2023-12-27 13:05:35,854][20381] Updated weights for policy 1, policy_version 1073402 (0.0010) [2023-12-27 13:05:35,903][20381] Updated weights for policy 1, policy_version 1073412 (0.0010) [2023-12-27 13:05:35,952][20381] Updated weights for policy 1, policy_version 1073422 (0.0010) [2023-12-27 13:05:36,002][20381] Updated weights for policy 1, policy_version 1073432 (0.0010) [2023-12-27 13:05:36,395][20347] Updated weights for policy 0, policy_version 1074382 (0.0010) [2023-12-27 13:05:36,454][20347] Updated weights for policy 0, policy_version 1074392 (0.0011) [2023-12-27 13:05:36,519][20347] Updated weights for policy 0, policy_version 1074402 (0.0010) [2023-12-27 13:05:36,729][20381] Updated weights for policy 1, policy_version 1073442 (0.0007) [2023-12-27 13:05:36,793][20381] Updated weights for policy 1, policy_version 1073452 (0.0007) [2023-12-27 13:05:36,845][20381] Updated weights for policy 1, policy_version 1073462 (0.0009) [2023-12-27 13:05:37,194][20347] Updated weights for policy 0, policy_version 1074412 (0.0009) [2023-12-27 13:05:37,246][20347] Updated weights for policy 0, policy_version 1074422 (0.0008) [2023-12-27 13:05:37,302][20347] Updated weights for policy 0, policy_version 1074432 (0.0007) [2023-12-27 13:05:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 549937152. Throughput: 0: 10151.2, 1: 10237.8. Samples: 549904840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:37,363][19170] Avg episode reward: [(0, '980.500'), (1, '774.720')] [2023-12-27 13:05:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001073464_274833408.pth... [2023-12-27 13:05:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001074440_275103744.pth... [2023-12-27 13:05:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001072280_274530304.pth [2023-12-27 13:05:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001073224_274792448.pth [2023-12-27 13:05:37,516][20381] Updated weights for policy 1, policy_version 1073472 (0.0006) [2023-12-27 13:05:37,564][20381] Updated weights for policy 1, policy_version 1073482 (0.0005) [2023-12-27 13:05:37,623][20381] Updated weights for policy 1, policy_version 1073492 (0.0007) [2023-12-27 13:05:38,000][20347] Updated weights for policy 0, policy_version 1074442 (0.0008) [2023-12-27 13:05:38,055][20347] Updated weights for policy 0, policy_version 1074452 (0.0008) [2023-12-27 13:05:38,107][20347] Updated weights for policy 0, policy_version 1074462 (0.0007) [2023-12-27 13:05:38,154][20347] Updated weights for policy 0, policy_version 1074472 (0.0006) [2023-12-27 13:05:38,157][20381] Updated weights for policy 1, policy_version 1073502 (0.0009) [2023-12-27 13:05:38,218][20381] Updated weights for policy 1, policy_version 1073512 (0.0009) [2023-12-27 13:05:38,276][20381] Updated weights for policy 1, policy_version 1073522 (0.0006) [2023-12-27 13:05:38,766][20347] Updated weights for policy 0, policy_version 1074482 (0.0005) [2023-12-27 13:05:38,822][20347] Updated weights for policy 0, policy_version 1074492 (0.0010) [2023-12-27 13:05:38,878][20347] Updated weights for policy 0, policy_version 1074502 (0.0010) [2023-12-27 13:05:38,885][20381] Updated weights for policy 1, policy_version 1073532 (0.0009) [2023-12-27 13:05:38,944][20381] Updated weights for policy 1, policy_version 1073542 (0.0010) [2023-12-27 13:05:38,997][20381] Updated weights for policy 1, policy_version 1073552 (0.0010) [2023-12-27 13:05:39,569][20347] Updated weights for policy 0, policy_version 1074512 (0.0011) [2023-12-27 13:05:39,632][20347] Updated weights for policy 0, policy_version 1074522 (0.0011) [2023-12-27 13:05:39,692][20347] Updated weights for policy 0, policy_version 1074532 (0.0010) [2023-12-27 13:05:39,738][20381] Updated weights for policy 1, policy_version 1073562 (0.0008) [2023-12-27 13:05:39,799][20381] Updated weights for policy 1, policy_version 1073572 (0.0007) [2023-12-27 13:05:39,860][20381] Updated weights for policy 1, policy_version 1073582 (0.0008) [2023-12-27 13:05:39,916][20381] Updated weights for policy 1, policy_version 1073592 (0.0008) [2023-12-27 13:05:40,389][20347] Updated weights for policy 0, policy_version 1074542 (0.0009) [2023-12-27 13:05:40,450][20347] Updated weights for policy 0, policy_version 1074552 (0.0008) [2023-12-27 13:05:40,512][20347] Updated weights for policy 0, policy_version 1074562 (0.0007) [2023-12-27 13:05:40,670][20381] Updated weights for policy 1, policy_version 1073602 (0.0011) [2023-12-27 13:05:40,729][20381] Updated weights for policy 1, policy_version 1073612 (0.0011) [2023-12-27 13:05:40,779][20381] Updated weights for policy 1, policy_version 1073622 (0.0011) [2023-12-27 13:05:41,284][20347] Updated weights for policy 0, policy_version 1074572 (0.0008) [2023-12-27 13:05:41,361][20347] Updated weights for policy 0, policy_version 1074582 (0.0008) [2023-12-27 13:05:41,425][20347] Updated weights for policy 0, policy_version 1074592 (0.0007) [2023-12-27 13:05:41,514][20381] Updated weights for policy 1, policy_version 1073632 (0.0011) [2023-12-27 13:05:41,575][20381] Updated weights for policy 1, policy_version 1073642 (0.0011) [2023-12-27 13:05:41,643][20381] Updated weights for policy 1, policy_version 1073652 (0.0011) [2023-12-27 13:05:42,121][20347] Updated weights for policy 0, policy_version 1074602 (0.0007) [2023-12-27 13:05:42,175][20347] Updated weights for policy 0, policy_version 1074612 (0.0007) [2023-12-27 13:05:42,223][20347] Updated weights for policy 0, policy_version 1074622 (0.0007) [2023-12-27 13:05:42,284][20347] Updated weights for policy 0, policy_version 1074632 (0.0007) [2023-12-27 13:05:42,358][20381] Updated weights for policy 1, policy_version 1073662 (0.0011) [2023-12-27 13:05:42,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20343.3, 300 sec: 20521.6). Total num frames: 550035456. Throughput: 0: 10151.4, 1: 10293.7. Samples: 550027264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:42,363][19170] Avg episode reward: [(0, '980.500'), (1, '706.270')] [2023-12-27 13:05:42,417][20381] Updated weights for policy 1, policy_version 1073672 (0.0012) [2023-12-27 13:05:42,470][20381] Updated weights for policy 1, policy_version 1073682 (0.0008) [2023-12-27 13:05:42,978][20347] Updated weights for policy 0, policy_version 1074642 (0.0006) [2023-12-27 13:05:43,035][20347] Updated weights for policy 0, policy_version 1074652 (0.0005) [2023-12-27 13:05:43,091][20347] Updated weights for policy 0, policy_version 1074662 (0.0006) [2023-12-27 13:05:43,191][20381] Updated weights for policy 1, policy_version 1073692 (0.0008) [2023-12-27 13:05:43,246][20381] Updated weights for policy 1, policy_version 1073702 (0.0007) [2023-12-27 13:05:43,302][20381] Updated weights for policy 1, policy_version 1073712 (0.0007) [2023-12-27 13:05:43,685][20347] Updated weights for policy 0, policy_version 1074672 (0.0006) [2023-12-27 13:05:43,735][20347] Updated weights for policy 0, policy_version 1074682 (0.0005) [2023-12-27 13:05:43,789][20347] Updated weights for policy 0, policy_version 1074692 (0.0005) [2023-12-27 13:05:43,950][20381] Updated weights for policy 1, policy_version 1073722 (0.0007) [2023-12-27 13:05:43,998][20381] Updated weights for policy 1, policy_version 1073732 (0.0005) [2023-12-27 13:05:44,049][20381] Updated weights for policy 1, policy_version 1073742 (0.0005) [2023-12-27 13:05:44,103][20381] Updated weights for policy 1, policy_version 1073752 (0.0005) [2023-12-27 13:05:44,395][20347] Updated weights for policy 0, policy_version 1074702 (0.0005) [2023-12-27 13:05:44,454][20347] Updated weights for policy 0, policy_version 1074712 (0.0006) [2023-12-27 13:05:44,502][20347] Updated weights for policy 0, policy_version 1074722 (0.0010) [2023-12-27 13:05:44,684][20381] Updated weights for policy 1, policy_version 1073762 (0.0007) [2023-12-27 13:05:44,738][20381] Updated weights for policy 1, policy_version 1073772 (0.0007) [2023-12-27 13:05:44,802][20381] Updated weights for policy 1, policy_version 1073782 (0.0008) [2023-12-27 13:05:45,118][20347] Updated weights for policy 0, policy_version 1074732 (0.0010) [2023-12-27 13:05:45,169][20347] Updated weights for policy 0, policy_version 1074742 (0.0005) [2023-12-27 13:05:45,221][20347] Updated weights for policy 0, policy_version 1074752 (0.0005) [2023-12-27 13:05:45,514][20381] Updated weights for policy 1, policy_version 1073792 (0.0006) [2023-12-27 13:05:45,571][20381] Updated weights for policy 1, policy_version 1073802 (0.0007) [2023-12-27 13:05:45,627][20381] Updated weights for policy 1, policy_version 1073812 (0.0006) [2023-12-27 13:05:45,857][20347] Updated weights for policy 0, policy_version 1074762 (0.0005) [2023-12-27 13:05:45,908][20347] Updated weights for policy 0, policy_version 1074772 (0.0005) [2023-12-27 13:05:45,957][20347] Updated weights for policy 0, policy_version 1074782 (0.0005) [2023-12-27 13:05:46,010][20347] Updated weights for policy 0, policy_version 1074792 (0.0009) [2023-12-27 13:05:46,295][20381] Updated weights for policy 1, policy_version 1073822 (0.0006) [2023-12-27 13:05:46,350][20381] Updated weights for policy 1, policy_version 1073832 (0.0005) [2023-12-27 13:05:46,399][20381] Updated weights for policy 1, policy_version 1073842 (0.0006) [2023-12-27 13:05:46,636][20347] Updated weights for policy 0, policy_version 1074802 (0.0011) [2023-12-27 13:05:46,697][20347] Updated weights for policy 0, policy_version 1074812 (0.0010) [2023-12-27 13:05:46,752][20347] Updated weights for policy 0, policy_version 1074822 (0.0010) [2023-12-27 13:05:46,996][20381] Updated weights for policy 1, policy_version 1073852 (0.0006) [2023-12-27 13:05:47,052][20381] Updated weights for policy 1, policy_version 1073862 (0.0009) [2023-12-27 13:05:47,111][20381] Updated weights for policy 1, policy_version 1073872 (0.0007) [2023-12-27 13:05:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 550141952. Throughput: 0: 10273.1, 1: 10288.1. Samples: 550152676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:47,363][19170] Avg episode reward: [(0, '990.220'), (1, '725.990')] [2023-12-27 13:05:47,379][20347] Updated weights for policy 0, policy_version 1074832 (0.0010) [2023-12-27 13:05:47,439][20347] Updated weights for policy 0, policy_version 1074842 (0.0010) [2023-12-27 13:05:47,497][20347] Updated weights for policy 0, policy_version 1074852 (0.0011) [2023-12-27 13:05:47,780][20381] Updated weights for policy 1, policy_version 1073882 (0.0008) [2023-12-27 13:05:47,830][20381] Updated weights for policy 1, policy_version 1073892 (0.0006) [2023-12-27 13:05:47,888][20381] Updated weights for policy 1, policy_version 1073902 (0.0005) [2023-12-27 13:05:47,944][20381] Updated weights for policy 1, policy_version 1073912 (0.0007) [2023-12-27 13:05:48,167][20347] Updated weights for policy 0, policy_version 1074862 (0.0010) [2023-12-27 13:05:48,213][20347] Updated weights for policy 0, policy_version 1074872 (0.0008) [2023-12-27 13:05:48,269][20347] Updated weights for policy 0, policy_version 1074882 (0.0009) [2023-12-27 13:05:48,608][20381] Updated weights for policy 1, policy_version 1073922 (0.0008) [2023-12-27 13:05:48,668][20381] Updated weights for policy 1, policy_version 1073932 (0.0007) [2023-12-27 13:05:48,725][20381] Updated weights for policy 1, policy_version 1073942 (0.0008) [2023-12-27 13:05:48,957][20347] Updated weights for policy 0, policy_version 1074892 (0.0010) [2023-12-27 13:05:49,002][20347] Updated weights for policy 0, policy_version 1074902 (0.0010) [2023-12-27 13:05:49,056][20347] Updated weights for policy 0, policy_version 1074912 (0.0010) [2023-12-27 13:05:49,414][20381] Updated weights for policy 1, policy_version 1073952 (0.0007) [2023-12-27 13:05:49,470][20381] Updated weights for policy 1, policy_version 1073962 (0.0008) [2023-12-27 13:05:49,530][20381] Updated weights for policy 1, policy_version 1073972 (0.0006) [2023-12-27 13:05:49,731][20347] Updated weights for policy 0, policy_version 1074922 (0.0010) [2023-12-27 13:05:49,802][20347] Updated weights for policy 0, policy_version 1074932 (0.0005) [2023-12-27 13:05:49,827][20291] KL-divergence is very high: 116.2421 [2023-12-27 13:05:49,870][20347] Updated weights for policy 0, policy_version 1074942 (0.0007) [2023-12-27 13:05:49,920][20347] Updated weights for policy 0, policy_version 1074952 (0.0009) [2023-12-27 13:05:50,176][20381] Updated weights for policy 1, policy_version 1073982 (0.0007) [2023-12-27 13:05:50,230][20381] Updated weights for policy 1, policy_version 1073992 (0.0007) [2023-12-27 13:05:50,289][20381] Updated weights for policy 1, policy_version 1074002 (0.0007) [2023-12-27 13:05:50,553][20347] Updated weights for policy 0, policy_version 1074962 (0.0010) [2023-12-27 13:05:50,615][20347] Updated weights for policy 0, policy_version 1074972 (0.0011) [2023-12-27 13:05:50,675][20347] Updated weights for policy 0, policy_version 1074982 (0.0009) [2023-12-27 13:05:50,982][20381] Updated weights for policy 1, policy_version 1074012 (0.0007) [2023-12-27 13:05:51,045][20381] Updated weights for policy 1, policy_version 1074022 (0.0007) [2023-12-27 13:05:51,116][20381] Updated weights for policy 1, policy_version 1074032 (0.0006) [2023-12-27 13:05:51,366][20347] Updated weights for policy 0, policy_version 1074992 (0.0008) [2023-12-27 13:05:51,429][20347] Updated weights for policy 0, policy_version 1075002 (0.0008) [2023-12-27 13:05:51,475][20347] Updated weights for policy 0, policy_version 1075012 (0.0007) [2023-12-27 13:05:51,767][20381] Updated weights for policy 1, policy_version 1074042 (0.0007) [2023-12-27 13:05:51,833][20381] Updated weights for policy 1, policy_version 1074052 (0.0006) [2023-12-27 13:05:51,899][20381] Updated weights for policy 1, policy_version 1074062 (0.0007) [2023-12-27 13:05:51,960][20381] Updated weights for policy 1, policy_version 1074072 (0.0007) [2023-12-27 13:05:52,211][20347] Updated weights for policy 0, policy_version 1075022 (0.0007) [2023-12-27 13:05:52,277][20347] Updated weights for policy 0, policy_version 1075032 (0.0007) [2023-12-27 13:05:52,336][20347] Updated weights for policy 0, policy_version 1075042 (0.0007) [2023-12-27 13:05:52,363][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 550240256. Throughput: 0: 10260.4, 1: 10266.5. Samples: 550214636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:52,363][19170] Avg episode reward: [(0, '960.900'), (1, '755.470')] [2023-12-27 13:05:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001074072_274989056.pth... [2023-12-27 13:05:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001072856_274677760.pth [2023-12-27 13:05:52,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001075048_275259392.pth... [2023-12-27 13:05:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001073832_274948096.pth [2023-12-27 13:05:52,671][20381] Updated weights for policy 1, policy_version 1074082 (0.0008) [2023-12-27 13:05:52,735][20381] Updated weights for policy 1, policy_version 1074092 (0.0007) [2023-12-27 13:05:52,788][20381] Updated weights for policy 1, policy_version 1074102 (0.0008) [2023-12-27 13:05:52,977][20347] Updated weights for policy 0, policy_version 1075052 (0.0007) [2023-12-27 13:05:53,042][20347] Updated weights for policy 0, policy_version 1075062 (0.0006) [2023-12-27 13:05:53,104][20347] Updated weights for policy 0, policy_version 1075072 (0.0007) [2023-12-27 13:05:53,484][20381] Updated weights for policy 1, policy_version 1074112 (0.0008) [2023-12-27 13:05:53,544][20381] Updated weights for policy 1, policy_version 1074122 (0.0008) [2023-12-27 13:05:53,597][20381] Updated weights for policy 1, policy_version 1074132 (0.0008) [2023-12-27 13:05:53,692][20347] Updated weights for policy 0, policy_version 1075082 (0.0006) [2023-12-27 13:05:53,743][20347] Updated weights for policy 0, policy_version 1075092 (0.0005) [2023-12-27 13:05:53,792][20347] Updated weights for policy 0, policy_version 1075102 (0.0005) [2023-12-27 13:05:53,847][20347] Updated weights for policy 0, policy_version 1075112 (0.0007) [2023-12-27 13:05:54,219][20381] Updated weights for policy 1, policy_version 1074142 (0.0007) [2023-12-27 13:05:54,269][20381] Updated weights for policy 1, policy_version 1074152 (0.0008) [2023-12-27 13:05:54,319][20381] Updated weights for policy 1, policy_version 1074162 (0.0007) [2023-12-27 13:05:54,543][20347] Updated weights for policy 0, policy_version 1075122 (0.0006) [2023-12-27 13:05:54,596][20347] Updated weights for policy 0, policy_version 1075132 (0.0008) [2023-12-27 13:05:54,657][20347] Updated weights for policy 0, policy_version 1075142 (0.0008) [2023-12-27 13:05:55,012][20381] Updated weights for policy 1, policy_version 1074172 (0.0009) [2023-12-27 13:05:55,063][20381] Updated weights for policy 1, policy_version 1074182 (0.0010) [2023-12-27 13:05:55,109][20381] Updated weights for policy 1, policy_version 1074192 (0.0010) [2023-12-27 13:05:55,350][20347] Updated weights for policy 0, policy_version 1075152 (0.0006) [2023-12-27 13:05:55,399][20347] Updated weights for policy 0, policy_version 1075162 (0.0005) [2023-12-27 13:05:55,448][20347] Updated weights for policy 0, policy_version 1075172 (0.0005) [2023-12-27 13:05:55,775][20381] Updated weights for policy 1, policy_version 1074202 (0.0010) [2023-12-27 13:05:55,828][20381] Updated weights for policy 1, policy_version 1074212 (0.0011) [2023-12-27 13:05:55,880][20381] Updated weights for policy 1, policy_version 1074222 (0.0010) [2023-12-27 13:05:55,935][20381] Updated weights for policy 1, policy_version 1074232 (0.0010) [2023-12-27 13:05:56,013][20347] Updated weights for policy 0, policy_version 1075182 (0.0007) [2023-12-27 13:05:56,070][20347] Updated weights for policy 0, policy_version 1075192 (0.0006) [2023-12-27 13:05:56,130][20347] Updated weights for policy 0, policy_version 1075202 (0.0007) [2023-12-27 13:05:56,584][20381] Updated weights for policy 1, policy_version 1074242 (0.0010) [2023-12-27 13:05:56,639][20381] Updated weights for policy 1, policy_version 1074252 (0.0010) [2023-12-27 13:05:56,690][20381] Updated weights for policy 1, policy_version 1074262 (0.0010) [2023-12-27 13:05:56,789][20347] Updated weights for policy 0, policy_version 1075212 (0.0007) [2023-12-27 13:05:56,845][20347] Updated weights for policy 0, policy_version 1075222 (0.0005) [2023-12-27 13:05:56,902][20347] Updated weights for policy 0, policy_version 1075232 (0.0005) [2023-12-27 13:05:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 550346752. Throughput: 0: 10308.4, 1: 10298.6. Samples: 550338488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:05:57,363][19170] Avg episode reward: [(0, '922.020'), (1, '706.540')] [2023-12-27 13:05:57,394][20381] Updated weights for policy 1, policy_version 1074272 (0.0010) [2023-12-27 13:05:57,443][20381] Updated weights for policy 1, policy_version 1074282 (0.0009) [2023-12-27 13:05:57,480][20347] Updated weights for policy 0, policy_version 1075242 (0.0006) [2023-12-27 13:05:57,488][20381] Updated weights for policy 1, policy_version 1074292 (0.0005) [2023-12-27 13:05:57,538][20347] Updated weights for policy 0, policy_version 1075252 (0.0006) [2023-12-27 13:05:57,599][20347] Updated weights for policy 0, policy_version 1075262 (0.0007) [2023-12-27 13:05:57,654][20347] Updated weights for policy 0, policy_version 1075272 (0.0007) [2023-12-27 13:05:58,066][20381] Updated weights for policy 1, policy_version 1074302 (0.0005) [2023-12-27 13:05:58,113][20381] Updated weights for policy 1, policy_version 1074312 (0.0006) [2023-12-27 13:05:58,158][20381] Updated weights for policy 1, policy_version 1074322 (0.0005) [2023-12-27 13:05:58,331][20347] Updated weights for policy 0, policy_version 1075282 (0.0008) [2023-12-27 13:05:58,394][20347] Updated weights for policy 0, policy_version 1075292 (0.0008) [2023-12-27 13:05:58,454][20347] Updated weights for policy 0, policy_version 1075302 (0.0008) [2023-12-27 13:05:58,889][20381] Updated weights for policy 1, policy_version 1074332 (0.0006) [2023-12-27 13:05:58,942][20381] Updated weights for policy 1, policy_version 1074342 (0.0008) [2023-12-27 13:05:59,003][20381] Updated weights for policy 1, policy_version 1074352 (0.0008) [2023-12-27 13:05:59,292][20347] Updated weights for policy 0, policy_version 1075312 (0.0007) [2023-12-27 13:05:59,349][20347] Updated weights for policy 0, policy_version 1075322 (0.0007) [2023-12-27 13:05:59,407][20347] Updated weights for policy 0, policy_version 1075332 (0.0008) [2023-12-27 13:05:59,682][20381] Updated weights for policy 1, policy_version 1074362 (0.0008) [2023-12-27 13:05:59,738][20381] Updated weights for policy 1, policy_version 1074372 (0.0010) [2023-12-27 13:05:59,797][20381] Updated weights for policy 1, policy_version 1074382 (0.0008) [2023-12-27 13:05:59,866][20381] Updated weights for policy 1, policy_version 1074392 (0.0008) [2023-12-27 13:06:00,050][20347] Updated weights for policy 0, policy_version 1075342 (0.0007) [2023-12-27 13:06:00,113][20347] Updated weights for policy 0, policy_version 1075352 (0.0007) [2023-12-27 13:06:00,169][20347] Updated weights for policy 0, policy_version 1075362 (0.0007) [2023-12-27 13:06:00,541][20381] Updated weights for policy 1, policy_version 1074402 (0.0005) [2023-12-27 13:06:00,589][20381] Updated weights for policy 1, policy_version 1074412 (0.0006) [2023-12-27 13:06:00,643][20381] Updated weights for policy 1, policy_version 1074422 (0.0006) [2023-12-27 13:06:00,893][20347] Updated weights for policy 0, policy_version 1075372 (0.0009) [2023-12-27 13:06:00,941][20347] Updated weights for policy 0, policy_version 1075382 (0.0007) [2023-12-27 13:06:00,984][20347] Updated weights for policy 0, policy_version 1075392 (0.0008) [2023-12-27 13:06:01,311][20381] Updated weights for policy 1, policy_version 1074432 (0.0008) [2023-12-27 13:06:01,376][20381] Updated weights for policy 1, policy_version 1074442 (0.0008) [2023-12-27 13:06:01,428][20381] Updated weights for policy 1, policy_version 1074452 (0.0007) [2023-12-27 13:06:01,773][20347] Updated weights for policy 0, policy_version 1075402 (0.0009) [2023-12-27 13:06:01,821][20347] Updated weights for policy 0, policy_version 1075412 (0.0010) [2023-12-27 13:06:01,871][20347] Updated weights for policy 0, policy_version 1075422 (0.0010) [2023-12-27 13:06:01,933][20347] Updated weights for policy 0, policy_version 1075432 (0.0010) [2023-12-27 13:06:02,140][20381] Updated weights for policy 1, policy_version 1074462 (0.0007) [2023-12-27 13:06:02,190][20381] Updated weights for policy 1, policy_version 1074472 (0.0007) [2023-12-27 13:06:02,251][20381] Updated weights for policy 1, policy_version 1074482 (0.0007) [2023-12-27 13:06:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 550453248. Throughput: 0: 10258.5, 1: 10291.1. Samples: 550460588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:02,363][19170] Avg episode reward: [(0, '951.320'), (1, '726.000')] [2023-12-27 13:06:02,666][20347] Updated weights for policy 0, policy_version 1075442 (0.0010) [2023-12-27 13:06:02,725][20347] Updated weights for policy 0, policy_version 1075452 (0.0010) [2023-12-27 13:06:02,784][20347] Updated weights for policy 0, policy_version 1075462 (0.0010) [2023-12-27 13:06:02,968][20381] Updated weights for policy 1, policy_version 1074492 (0.0008) [2023-12-27 13:06:03,029][20381] Updated weights for policy 1, policy_version 1074502 (0.0007) [2023-12-27 13:06:03,073][20381] Updated weights for policy 1, policy_version 1074512 (0.0007) [2023-12-27 13:06:03,440][20347] Updated weights for policy 0, policy_version 1075472 (0.0010) [2023-12-27 13:06:03,444][20291] KL-divergence is very high: 255.8963 [2023-12-27 13:06:03,469][20291] KL-divergence is very high: 351.8645 [2023-12-27 13:06:03,486][20291] KL-divergence is very high: 185.7585 [2023-12-27 13:06:03,491][20347] Updated weights for policy 0, policy_version 1075482 (0.0010) [2023-12-27 13:06:03,549][20347] Updated weights for policy 0, policy_version 1075492 (0.0010) [2023-12-27 13:06:03,559][20291] KL-divergence is very high: 101.6371 [2023-12-27 13:06:03,765][20381] Updated weights for policy 1, policy_version 1074522 (0.0007) [2023-12-27 13:06:03,808][20381] Updated weights for policy 1, policy_version 1074532 (0.0007) [2023-12-27 13:06:03,864][20381] Updated weights for policy 1, policy_version 1074542 (0.0007) [2023-12-27 13:06:03,917][20381] Updated weights for policy 1, policy_version 1074552 (0.0009) [2023-12-27 13:06:04,242][20347] Updated weights for policy 0, policy_version 1075502 (0.0010) [2023-12-27 13:06:04,292][20347] Updated weights for policy 0, policy_version 1075512 (0.0010) [2023-12-27 13:06:04,345][20347] Updated weights for policy 0, policy_version 1075522 (0.0010) [2023-12-27 13:06:04,619][20381] Updated weights for policy 1, policy_version 1074562 (0.0008) [2023-12-27 13:06:04,678][20381] Updated weights for policy 1, policy_version 1074572 (0.0009) [2023-12-27 13:06:04,738][20381] Updated weights for policy 1, policy_version 1074582 (0.0008) [2023-12-27 13:06:05,030][20347] Updated weights for policy 0, policy_version 1075532 (0.0009) [2023-12-27 13:06:05,084][20347] Updated weights for policy 0, policy_version 1075542 (0.0007) [2023-12-27 13:06:05,146][20347] Updated weights for policy 0, policy_version 1075552 (0.0006) [2023-12-27 13:06:05,345][20381] Updated weights for policy 1, policy_version 1074592 (0.0005) [2023-12-27 13:06:05,397][20381] Updated weights for policy 1, policy_version 1074602 (0.0005) [2023-12-27 13:06:05,451][20381] Updated weights for policy 1, policy_version 1074612 (0.0005) [2023-12-27 13:06:05,831][20347] Updated weights for policy 0, policy_version 1075562 (0.0008) [2023-12-27 13:06:05,889][20347] Updated weights for policy 0, policy_version 1075572 (0.0008) [2023-12-27 13:06:05,952][20347] Updated weights for policy 0, policy_version 1075582 (0.0005) [2023-12-27 13:06:06,001][20347] Updated weights for policy 0, policy_version 1075592 (0.0005) [2023-12-27 13:06:06,121][20381] Updated weights for policy 1, policy_version 1074622 (0.0008) [2023-12-27 13:06:06,191][20381] Updated weights for policy 1, policy_version 1074632 (0.0008) [2023-12-27 13:06:06,254][20381] Updated weights for policy 1, policy_version 1074642 (0.0008) [2023-12-27 13:06:06,567][20291] KL-divergence is very high: 133.3811 [2023-12-27 13:06:06,578][20347] Updated weights for policy 0, policy_version 1075602 (0.0006) [2023-12-27 13:06:06,636][20347] Updated weights for policy 0, policy_version 1075612 (0.0006) [2023-12-27 13:06:06,695][20347] Updated weights for policy 0, policy_version 1075622 (0.0006) [2023-12-27 13:06:07,032][20381] Updated weights for policy 1, policy_version 1074652 (0.0008) [2023-12-27 13:06:07,095][20381] Updated weights for policy 1, policy_version 1074662 (0.0008) [2023-12-27 13:06:07,158][20381] Updated weights for policy 1, policy_version 1074672 (0.0007) [2023-12-27 13:06:07,320][20347] Updated weights for policy 0, policy_version 1075632 (0.0007) [2023-12-27 13:06:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 550551552. Throughput: 0: 10258.9, 1: 10297.0. Samples: 550521720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:07,363][19170] Avg episode reward: [(0, '951.170'), (1, '745.570')] [2023-12-27 13:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001074680_275144704.pth... [2023-12-27 13:06:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001073464_274833408.pth [2023-12-27 13:06:07,382][20347] Updated weights for policy 0, policy_version 1075642 (0.0006) [2023-12-27 13:06:07,438][20347] Updated weights for policy 0, policy_version 1075652 (0.0008) [2023-12-27 13:06:07,458][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001075656_275415040.pth... [2023-12-27 13:06:07,462][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001074440_275103744.pth [2023-12-27 13:06:07,842][20381] Updated weights for policy 1, policy_version 1074682 (0.0008) [2023-12-27 13:06:07,899][20381] Updated weights for policy 1, policy_version 1074692 (0.0007) [2023-12-27 13:06:07,948][20381] Updated weights for policy 1, policy_version 1074702 (0.0008) [2023-12-27 13:06:07,996][20381] Updated weights for policy 1, policy_version 1074712 (0.0008) [2023-12-27 13:06:08,033][20347] Updated weights for policy 0, policy_version 1075662 (0.0006) [2023-12-27 13:06:08,091][20347] Updated weights for policy 0, policy_version 1075672 (0.0005) [2023-12-27 13:06:08,155][20347] Updated weights for policy 0, policy_version 1075682 (0.0007) [2023-12-27 13:06:08,633][20381] Updated weights for policy 1, policy_version 1074722 (0.0005) [2023-12-27 13:06:08,683][20381] Updated weights for policy 1, policy_version 1074732 (0.0008) [2023-12-27 13:06:08,735][20381] Updated weights for policy 1, policy_version 1074742 (0.0007) [2023-12-27 13:06:08,822][20347] Updated weights for policy 0, policy_version 1075692 (0.0009) [2023-12-27 13:06:08,882][20347] Updated weights for policy 0, policy_version 1075702 (0.0007) [2023-12-27 13:06:08,931][20347] Updated weights for policy 0, policy_version 1075712 (0.0006) [2023-12-27 13:06:09,451][20381] Updated weights for policy 1, policy_version 1074752 (0.0010) [2023-12-27 13:06:09,504][20381] Updated weights for policy 1, policy_version 1074762 (0.0011) [2023-12-27 13:06:09,561][20381] Updated weights for policy 1, policy_version 1074772 (0.0011) [2023-12-27 13:06:09,660][20347] Updated weights for policy 0, policy_version 1075722 (0.0006) [2023-12-27 13:06:09,720][20347] Updated weights for policy 0, policy_version 1075732 (0.0007) [2023-12-27 13:06:09,776][20347] Updated weights for policy 0, policy_version 1075742 (0.0007) [2023-12-27 13:06:09,841][20347] Updated weights for policy 0, policy_version 1075752 (0.0009) [2023-12-27 13:06:10,340][20381] Updated weights for policy 1, policy_version 1074782 (0.0009) [2023-12-27 13:06:10,395][20381] Updated weights for policy 1, policy_version 1074792 (0.0008) [2023-12-27 13:06:10,455][20381] Updated weights for policy 1, policy_version 1074802 (0.0008) [2023-12-27 13:06:10,479][20347] Updated weights for policy 0, policy_version 1075762 (0.0008) [2023-12-27 13:06:10,541][20347] Updated weights for policy 0, policy_version 1075772 (0.0007) [2023-12-27 13:06:10,604][20347] Updated weights for policy 0, policy_version 1075782 (0.0006) [2023-12-27 13:06:11,219][20381] Updated weights for policy 1, policy_version 1074812 (0.0008) [2023-12-27 13:06:11,236][20347] Updated weights for policy 0, policy_version 1075792 (0.0007) [2023-12-27 13:06:11,282][20381] Updated weights for policy 1, policy_version 1074822 (0.0008) [2023-12-27 13:06:11,300][20347] Updated weights for policy 0, policy_version 1075802 (0.0007) [2023-12-27 13:06:11,343][20381] Updated weights for policy 1, policy_version 1074832 (0.0006) [2023-12-27 13:06:11,367][20347] Updated weights for policy 0, policy_version 1075812 (0.0008) [2023-12-27 13:06:12,039][20381] Updated weights for policy 1, policy_version 1074842 (0.0008) [2023-12-27 13:06:12,066][20347] Updated weights for policy 0, policy_version 1075822 (0.0006) [2023-12-27 13:06:12,098][20381] Updated weights for policy 1, policy_version 1074852 (0.0011) [2023-12-27 13:06:12,125][20347] Updated weights for policy 0, policy_version 1075832 (0.0005) [2023-12-27 13:06:12,160][20381] Updated weights for policy 1, policy_version 1074862 (0.0011) [2023-12-27 13:06:12,191][20347] Updated weights for policy 0, policy_version 1075842 (0.0006) [2023-12-27 13:06:12,216][20381] Updated weights for policy 1, policy_version 1074872 (0.0008) [2023-12-27 13:06:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 550658048. Throughput: 0: 10326.2, 1: 10238.1. Samples: 550644360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:12,363][19170] Avg episode reward: [(0, '960.940'), (1, '745.720')] [2023-12-27 13:06:12,853][20347] Updated weights for policy 0, policy_version 1075852 (0.0007) [2023-12-27 13:06:12,913][20381] Updated weights for policy 1, policy_version 1074882 (0.0007) [2023-12-27 13:06:12,914][20347] Updated weights for policy 0, policy_version 1075862 (0.0008) [2023-12-27 13:06:12,965][20381] Updated weights for policy 1, policy_version 1074892 (0.0005) [2023-12-27 13:06:12,966][20347] Updated weights for policy 0, policy_version 1075872 (0.0008) [2023-12-27 13:06:13,024][20381] Updated weights for policy 1, policy_version 1074902 (0.0005) [2023-12-27 13:06:13,628][20381] Updated weights for policy 1, policy_version 1074912 (0.0010) [2023-12-27 13:06:13,660][20347] Updated weights for policy 0, policy_version 1075882 (0.0007) [2023-12-27 13:06:13,673][20381] Updated weights for policy 1, policy_version 1074922 (0.0010) [2023-12-27 13:06:13,711][20347] Updated weights for policy 0, policy_version 1075892 (0.0008) [2023-12-27 13:06:13,721][20381] Updated weights for policy 1, policy_version 1074932 (0.0010) [2023-12-27 13:06:13,766][20347] Updated weights for policy 0, policy_version 1075902 (0.0009) [2023-12-27 13:06:13,815][20347] Updated weights for policy 0, policy_version 1075912 (0.0010) [2023-12-27 13:06:14,430][20381] Updated weights for policy 1, policy_version 1074942 (0.0010) [2023-12-27 13:06:14,488][20347] Updated weights for policy 0, policy_version 1075922 (0.0005) [2023-12-27 13:06:14,492][20381] Updated weights for policy 1, policy_version 1074952 (0.0010) [2023-12-27 13:06:14,540][20347] Updated weights for policy 0, policy_version 1075932 (0.0005) [2023-12-27 13:06:14,550][20381] Updated weights for policy 1, policy_version 1074962 (0.0010) [2023-12-27 13:06:14,605][20347] Updated weights for policy 0, policy_version 1075942 (0.0006) [2023-12-27 13:06:15,230][20347] Updated weights for policy 0, policy_version 1075952 (0.0008) [2023-12-27 13:06:15,243][20381] Updated weights for policy 1, policy_version 1074972 (0.0010) [2023-12-27 13:06:15,283][20347] Updated weights for policy 0, policy_version 1075962 (0.0005) [2023-12-27 13:06:15,293][20381] Updated weights for policy 1, policy_version 1074982 (0.0011) [2023-12-27 13:06:15,344][20347] Updated weights for policy 0, policy_version 1075972 (0.0005) [2023-12-27 13:06:15,346][20381] Updated weights for policy 1, policy_version 1074992 (0.0011) [2023-12-27 13:06:15,962][20381] Updated weights for policy 1, policy_version 1075002 (0.0007) [2023-12-27 13:06:16,017][20381] Updated weights for policy 1, policy_version 1075012 (0.0010) [2023-12-27 13:06:16,023][20347] Updated weights for policy 0, policy_version 1075982 (0.0007) [2023-12-27 13:06:16,063][20381] Updated weights for policy 1, policy_version 1075022 (0.0010) [2023-12-27 13:06:16,079][20347] Updated weights for policy 0, policy_version 1075992 (0.0006) [2023-12-27 13:06:16,116][20381] Updated weights for policy 1, policy_version 1075032 (0.0011) [2023-12-27 13:06:16,140][20347] Updated weights for policy 0, policy_version 1076002 (0.0006) [2023-12-27 13:06:16,744][20347] Updated weights for policy 0, policy_version 1076012 (0.0005) [2023-12-27 13:06:16,781][20381] Updated weights for policy 1, policy_version 1075042 (0.0008) [2023-12-27 13:06:16,798][20347] Updated weights for policy 0, policy_version 1076022 (0.0006) [2023-12-27 13:06:16,847][20381] Updated weights for policy 1, policy_version 1075052 (0.0008) [2023-12-27 13:06:16,856][20347] Updated weights for policy 0, policy_version 1076032 (0.0006) [2023-12-27 13:06:16,895][20381] Updated weights for policy 1, policy_version 1075062 (0.0007) [2023-12-27 13:06:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 550756352. Throughput: 0: 10340.4, 1: 10203.3. Samples: 550768364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:17,363][19170] Avg episode reward: [(0, '960.900'), (1, '696.830')] [2023-12-27 13:06:17,427][20347] Updated weights for policy 0, policy_version 1076042 (0.0006) [2023-12-27 13:06:17,488][20347] Updated weights for policy 0, policy_version 1076052 (0.0007) [2023-12-27 13:06:17,512][20381] Updated weights for policy 1, policy_version 1075072 (0.0009) [2023-12-27 13:06:17,534][20347] Updated weights for policy 0, policy_version 1076062 (0.0006) [2023-12-27 13:06:17,558][20381] Updated weights for policy 1, policy_version 1075082 (0.0005) [2023-12-27 13:06:17,583][20347] Updated weights for policy 0, policy_version 1076072 (0.0005) [2023-12-27 13:06:17,607][20381] Updated weights for policy 1, policy_version 1075092 (0.0007) [2023-12-27 13:06:18,277][20347] Updated weights for policy 0, policy_version 1076082 (0.0005) [2023-12-27 13:06:18,289][20291] KL-divergence is very high: 335.1958 [2023-12-27 13:06:18,312][20381] Updated weights for policy 1, policy_version 1075102 (0.0007) [2023-12-27 13:06:18,341][20347] Updated weights for policy 0, policy_version 1076092 (0.0006) [2023-12-27 13:06:18,342][20291] KL-divergence is very high: 225.5073 [2023-12-27 13:06:18,377][20381] Updated weights for policy 1, policy_version 1075112 (0.0008) [2023-12-27 13:06:18,397][20347] Updated weights for policy 0, policy_version 1076102 (0.0006) [2023-12-27 13:06:18,425][20381] Updated weights for policy 1, policy_version 1075122 (0.0008) [2023-12-27 13:06:18,944][20347] Updated weights for policy 0, policy_version 1076112 (0.0007) [2023-12-27 13:06:19,003][20347] Updated weights for policy 0, policy_version 1076122 (0.0008) [2023-12-27 13:06:19,027][20381] Updated weights for policy 1, policy_version 1075132 (0.0007) [2023-12-27 13:06:19,062][20347] Updated weights for policy 0, policy_version 1076132 (0.0006) [2023-12-27 13:06:19,080][20381] Updated weights for policy 1, policy_version 1075142 (0.0006) [2023-12-27 13:06:19,138][20381] Updated weights for policy 1, policy_version 1075152 (0.0007) [2023-12-27 13:06:19,766][20347] Updated weights for policy 0, policy_version 1076142 (0.0007) [2023-12-27 13:06:19,828][20347] Updated weights for policy 0, policy_version 1076152 (0.0007) [2023-12-27 13:06:19,871][20381] Updated weights for policy 1, policy_version 1075162 (0.0007) [2023-12-27 13:06:19,891][20347] Updated weights for policy 0, policy_version 1076162 (0.0008) [2023-12-27 13:06:19,931][20381] Updated weights for policy 1, policy_version 1075172 (0.0007) [2023-12-27 13:06:19,991][20381] Updated weights for policy 1, policy_version 1075182 (0.0007) [2023-12-27 13:06:20,046][20381] Updated weights for policy 1, policy_version 1075192 (0.0008) [2023-12-27 13:06:20,673][20347] Updated weights for policy 0, policy_version 1076172 (0.0008) [2023-12-27 13:06:20,710][20381] Updated weights for policy 1, policy_version 1075202 (0.0006) [2023-12-27 13:06:20,735][20347] Updated weights for policy 0, policy_version 1076182 (0.0006) [2023-12-27 13:06:20,768][20381] Updated weights for policy 1, policy_version 1075212 (0.0007) [2023-12-27 13:06:20,789][20347] Updated weights for policy 0, policy_version 1076192 (0.0006) [2023-12-27 13:06:20,829][20381] Updated weights for policy 1, policy_version 1075222 (0.0006) [2023-12-27 13:06:21,536][20347] Updated weights for policy 0, policy_version 1076202 (0.0006) [2023-12-27 13:06:21,560][20381] Updated weights for policy 1, policy_version 1075232 (0.0006) [2023-12-27 13:06:21,599][20347] Updated weights for policy 0, policy_version 1076212 (0.0008) [2023-12-27 13:06:21,625][20381] Updated weights for policy 1, policy_version 1075242 (0.0006) [2023-12-27 13:06:21,662][20347] Updated weights for policy 0, policy_version 1076222 (0.0007) [2023-12-27 13:06:21,690][20381] Updated weights for policy 1, policy_version 1075252 (0.0007) [2023-12-27 13:06:21,722][20347] Updated weights for policy 0, policy_version 1076232 (0.0006) [2023-12-27 13:06:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 550854656. Throughput: 0: 10349.4, 1: 10226.2. Samples: 550830744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:22,363][19170] Avg episode reward: [(0, '960.840'), (1, '814.060')] [2023-12-27 13:06:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001076232_275562496.pth... [2023-12-27 13:06:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001075048_275259392.pth [2023-12-27 13:06:22,406][20381] Updated weights for policy 1, policy_version 1075262 (0.0007) [2023-12-27 13:06:22,458][20347] Updated weights for policy 0, policy_version 1076242 (0.0011) [2023-12-27 13:06:22,463][20381] Updated weights for policy 1, policy_version 1075272 (0.0006) [2023-12-27 13:06:22,512][20381] Updated weights for policy 1, policy_version 1075282 (0.0006) [2023-12-27 13:06:22,515][20347] Updated weights for policy 0, policy_version 1076252 (0.0010) [2023-12-27 13:06:22,563][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001075288_275300352.pth... [2023-12-27 13:06:22,568][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001074072_274989056.pth [2023-12-27 13:06:22,575][20347] Updated weights for policy 0, policy_version 1076262 (0.0010) [2023-12-27 13:06:23,229][20381] Updated weights for policy 1, policy_version 1075292 (0.0007) [2023-12-27 13:06:23,277][20347] Updated weights for policy 0, policy_version 1076272 (0.0006) [2023-12-27 13:06:23,296][20381] Updated weights for policy 1, policy_version 1075302 (0.0006) [2023-12-27 13:06:23,338][20347] Updated weights for policy 0, policy_version 1076282 (0.0005) [2023-12-27 13:06:23,359][20381] Updated weights for policy 1, policy_version 1075312 (0.0008) [2023-12-27 13:06:23,394][20347] Updated weights for policy 0, policy_version 1076292 (0.0005) [2023-12-27 13:06:24,017][20347] Updated weights for policy 0, policy_version 1076302 (0.0008) [2023-12-27 13:06:24,027][20381] Updated weights for policy 1, policy_version 1075322 (0.0007) [2023-12-27 13:06:24,062][20347] Updated weights for policy 0, policy_version 1076312 (0.0010) [2023-12-27 13:06:24,079][20381] Updated weights for policy 1, policy_version 1075332 (0.0005) [2023-12-27 13:06:24,108][20347] Updated weights for policy 0, policy_version 1076322 (0.0010) [2023-12-27 13:06:24,127][20381] Updated weights for policy 1, policy_version 1075342 (0.0005) [2023-12-27 13:06:24,178][20381] Updated weights for policy 1, policy_version 1075352 (0.0006) [2023-12-27 13:06:24,797][20347] Updated weights for policy 0, policy_version 1076332 (0.0010) [2023-12-27 13:06:24,831][20381] Updated weights for policy 1, policy_version 1075362 (0.0005) [2023-12-27 13:06:24,851][20347] Updated weights for policy 0, policy_version 1076342 (0.0010) [2023-12-27 13:06:24,883][20381] Updated weights for policy 1, policy_version 1075372 (0.0005) [2023-12-27 13:06:24,903][20347] Updated weights for policy 0, policy_version 1076352 (0.0010) [2023-12-27 13:06:24,934][20381] Updated weights for policy 1, policy_version 1075382 (0.0005) [2023-12-27 13:06:25,519][20381] Updated weights for policy 1, policy_version 1075392 (0.0007) [2023-12-27 13:06:25,561][20347] Updated weights for policy 0, policy_version 1076362 (0.0009) [2023-12-27 13:06:25,573][20381] Updated weights for policy 1, policy_version 1075402 (0.0008) [2023-12-27 13:06:25,615][20347] Updated weights for policy 0, policy_version 1076372 (0.0006) [2023-12-27 13:06:25,622][20381] Updated weights for policy 1, policy_version 1075412 (0.0005) [2023-12-27 13:06:25,666][20347] Updated weights for policy 0, policy_version 1076382 (0.0006) [2023-12-27 13:06:25,731][20347] Updated weights for policy 0, policy_version 1076392 (0.0005) [2023-12-27 13:06:26,253][20381] Updated weights for policy 1, policy_version 1075422 (0.0005) [2023-12-27 13:06:26,314][20381] Updated weights for policy 1, policy_version 1075432 (0.0005) [2023-12-27 13:06:26,364][20347] Updated weights for policy 0, policy_version 1076402 (0.0007) [2023-12-27 13:06:26,368][20381] Updated weights for policy 1, policy_version 1075442 (0.0005) [2023-12-27 13:06:26,414][20347] Updated weights for policy 0, policy_version 1076412 (0.0007) [2023-12-27 13:06:26,466][20347] Updated weights for policy 0, policy_version 1076422 (0.0008) [2023-12-27 13:06:26,872][20381] Updated weights for policy 1, policy_version 1075452 (0.0007) [2023-12-27 13:06:26,928][20381] Updated weights for policy 1, policy_version 1075462 (0.0011) [2023-12-27 13:06:26,983][20381] Updated weights for policy 1, policy_version 1075472 (0.0010) [2023-12-27 13:06:27,240][20347] Updated weights for policy 0, policy_version 1076432 (0.0008) [2023-12-27 13:06:27,294][20347] Updated weights for policy 0, policy_version 1076442 (0.0007) [2023-12-27 13:06:27,343][20347] Updated weights for policy 0, policy_version 1076452 (0.0007) [2023-12-27 13:06:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 550969344. Throughput: 0: 10350.9, 1: 10233.4. Samples: 550953552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:27,363][19170] Avg episode reward: [(0, '941.260'), (1, '823.900')] [2023-12-27 13:06:27,608][20381] Updated weights for policy 1, policy_version 1075482 (0.0010) [2023-12-27 13:06:27,654][20381] Updated weights for policy 1, policy_version 1075492 (0.0005) [2023-12-27 13:06:27,699][20381] Updated weights for policy 1, policy_version 1075502 (0.0005) [2023-12-27 13:06:27,750][20381] Updated weights for policy 1, policy_version 1075512 (0.0005) [2023-12-27 13:06:28,085][20347] Updated weights for policy 0, policy_version 1076462 (0.0008) [2023-12-27 13:06:28,135][20347] Updated weights for policy 0, policy_version 1076472 (0.0007) [2023-12-27 13:06:28,189][20347] Updated weights for policy 0, policy_version 1076482 (0.0007) [2023-12-27 13:06:28,368][20381] Updated weights for policy 1, policy_version 1075522 (0.0008) [2023-12-27 13:06:28,421][20381] Updated weights for policy 1, policy_version 1075532 (0.0008) [2023-12-27 13:06:28,479][20381] Updated weights for policy 1, policy_version 1075542 (0.0008) [2023-12-27 13:06:28,933][20347] Updated weights for policy 0, policy_version 1076492 (0.0008) [2023-12-27 13:06:28,990][20347] Updated weights for policy 0, policy_version 1076502 (0.0009) [2023-12-27 13:06:29,044][20347] Updated weights for policy 0, policy_version 1076512 (0.0006) [2023-12-27 13:06:29,056][20381] Updated weights for policy 1, policy_version 1075552 (0.0006) [2023-12-27 13:06:29,112][20381] Updated weights for policy 1, policy_version 1075562 (0.0008) [2023-12-27 13:06:29,168][20381] Updated weights for policy 1, policy_version 1075572 (0.0007) [2023-12-27 13:06:29,790][20347] Updated weights for policy 0, policy_version 1076522 (0.0009) [2023-12-27 13:06:29,836][20381] Updated weights for policy 1, policy_version 1075582 (0.0006) [2023-12-27 13:06:29,849][20347] Updated weights for policy 0, policy_version 1076532 (0.0008) [2023-12-27 13:06:29,890][20381] Updated weights for policy 1, policy_version 1075592 (0.0007) [2023-12-27 13:06:29,907][20347] Updated weights for policy 0, policy_version 1076542 (0.0008) [2023-12-27 13:06:29,952][20381] Updated weights for policy 1, policy_version 1075602 (0.0007) [2023-12-27 13:06:29,972][20347] Updated weights for policy 0, policy_version 1076552 (0.0008) [2023-12-27 13:06:30,620][20347] Updated weights for policy 0, policy_version 1076562 (0.0007) [2023-12-27 13:06:30,640][20381] Updated weights for policy 1, policy_version 1075612 (0.0006) [2023-12-27 13:06:30,678][20347] Updated weights for policy 0, policy_version 1076572 (0.0007) [2023-12-27 13:06:30,689][20381] Updated weights for policy 1, policy_version 1075622 (0.0007) [2023-12-27 13:06:30,733][20347] Updated weights for policy 0, policy_version 1076582 (0.0007) [2023-12-27 13:06:30,743][20381] Updated weights for policy 1, policy_version 1075632 (0.0007) [2023-12-27 13:06:31,374][20381] Updated weights for policy 1, policy_version 1075642 (0.0006) [2023-12-27 13:06:31,428][20381] Updated weights for policy 1, policy_version 1075652 (0.0008) [2023-12-27 13:06:31,474][20381] Updated weights for policy 1, policy_version 1075662 (0.0008) [2023-12-27 13:06:31,492][20347] Updated weights for policy 0, policy_version 1076592 (0.0008) [2023-12-27 13:06:31,528][20381] Updated weights for policy 1, policy_version 1075672 (0.0007) [2023-12-27 13:06:31,554][20347] Updated weights for policy 0, policy_version 1076602 (0.0007) [2023-12-27 13:06:31,613][20347] Updated weights for policy 0, policy_version 1076612 (0.0009) [2023-12-27 13:06:32,274][20381] Updated weights for policy 1, policy_version 1075682 (0.0007) [2023-12-27 13:06:32,298][20347] Updated weights for policy 0, policy_version 1076622 (0.0008) [2023-12-27 13:06:32,332][20381] Updated weights for policy 1, policy_version 1075692 (0.0009) [2023-12-27 13:06:32,347][20347] Updated weights for policy 0, policy_version 1076632 (0.0010) [2023-12-27 13:06:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 551059456. Throughput: 0: 10217.9, 1: 10320.4. Samples: 551076904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:32,363][19170] Avg episode reward: [(0, '902.210'), (1, '774.970')] [2023-12-27 13:06:32,395][20381] Updated weights for policy 1, policy_version 1075702 (0.0007) [2023-12-27 13:06:32,405][20347] Updated weights for policy 0, policy_version 1076642 (0.0008) [2023-12-27 13:06:33,028][20381] Updated weights for policy 1, policy_version 1075712 (0.0007) [2023-12-27 13:06:33,081][20381] Updated weights for policy 1, policy_version 1075722 (0.0007) [2023-12-27 13:06:33,109][20347] Updated weights for policy 0, policy_version 1076652 (0.0009) [2023-12-27 13:06:33,129][20381] Updated weights for policy 1, policy_version 1075732 (0.0006) [2023-12-27 13:06:33,165][20347] Updated weights for policy 0, policy_version 1076662 (0.0006) [2023-12-27 13:06:33,222][20347] Updated weights for policy 0, policy_version 1076672 (0.0007) [2023-12-27 13:06:33,768][20381] Updated weights for policy 1, policy_version 1075742 (0.0008) [2023-12-27 13:06:33,805][20347] Updated weights for policy 0, policy_version 1076682 (0.0008) [2023-12-27 13:06:33,829][20381] Updated weights for policy 1, policy_version 1075752 (0.0007) [2023-12-27 13:06:33,854][20347] Updated weights for policy 0, policy_version 1076692 (0.0005) [2023-12-27 13:06:33,884][20381] Updated weights for policy 1, policy_version 1075762 (0.0007) [2023-12-27 13:06:33,897][20347] Updated weights for policy 0, policy_version 1076702 (0.0005) [2023-12-27 13:06:33,953][20347] Updated weights for policy 0, policy_version 1076712 (0.0007) [2023-12-27 13:06:34,562][20381] Updated weights for policy 1, policy_version 1075772 (0.0007) [2023-12-27 13:06:34,619][20381] Updated weights for policy 1, policy_version 1075782 (0.0008) [2023-12-27 13:06:34,630][20347] Updated weights for policy 0, policy_version 1076722 (0.0005) [2023-12-27 13:06:34,675][20381] Updated weights for policy 1, policy_version 1075792 (0.0009) [2023-12-27 13:06:34,690][20347] Updated weights for policy 0, policy_version 1076732 (0.0009) [2023-12-27 13:06:34,725][20291] KL-divergence is very high: 115.1172 [2023-12-27 13:06:34,748][20347] Updated weights for policy 0, policy_version 1076742 (0.0010) [2023-12-27 13:06:35,337][20381] Updated weights for policy 1, policy_version 1075802 (0.0009) [2023-12-27 13:06:35,383][20381] Updated weights for policy 1, policy_version 1075812 (0.0005) [2023-12-27 13:06:35,405][20347] Updated weights for policy 0, policy_version 1076752 (0.0008) [2023-12-27 13:06:35,432][20381] Updated weights for policy 1, policy_version 1075822 (0.0005) [2023-12-27 13:06:35,449][20347] Updated weights for policy 0, policy_version 1076762 (0.0007) [2023-12-27 13:06:35,481][20381] Updated weights for policy 1, policy_version 1075832 (0.0006) [2023-12-27 13:06:35,493][20347] Updated weights for policy 0, policy_version 1076772 (0.0005) [2023-12-27 13:06:36,016][20381] Updated weights for policy 1, policy_version 1075842 (0.0005) [2023-12-27 13:06:36,071][20381] Updated weights for policy 1, policy_version 1075852 (0.0007) [2023-12-27 13:06:36,130][20347] Updated weights for policy 0, policy_version 1076782 (0.0008) [2023-12-27 13:06:36,136][20381] Updated weights for policy 1, policy_version 1075862 (0.0008) [2023-12-27 13:06:36,181][20347] Updated weights for policy 0, policy_version 1076792 (0.0006) [2023-12-27 13:06:36,239][20347] Updated weights for policy 0, policy_version 1076802 (0.0008) [2023-12-27 13:06:36,838][20347] Updated weights for policy 0, policy_version 1076812 (0.0007) [2023-12-27 13:06:36,867][20381] Updated weights for policy 1, policy_version 1075872 (0.0006) [2023-12-27 13:06:36,901][20347] Updated weights for policy 0, policy_version 1076822 (0.0008) [2023-12-27 13:06:36,923][20381] Updated weights for policy 1, policy_version 1075882 (0.0006) [2023-12-27 13:06:36,957][20347] Updated weights for policy 0, policy_version 1076832 (0.0005) [2023-12-27 13:06:36,975][20381] Updated weights for policy 1, policy_version 1075892 (0.0007) [2023-12-27 13:06:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 551174144. Throughput: 0: 10238.4, 1: 10317.6. Samples: 551139648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:37,363][19170] Avg episode reward: [(0, '873.730'), (1, '843.520')] [2023-12-27 13:06:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001076840_275718144.pth... [2023-12-27 13:06:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001075896_275456000.pth... [2023-12-27 13:06:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001075656_275415040.pth [2023-12-27 13:06:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001074680_275144704.pth [2023-12-27 13:06:37,664][20381] Updated weights for policy 1, policy_version 1075902 (0.0007) [2023-12-27 13:06:37,695][20347] Updated weights for policy 0, policy_version 1076842 (0.0007) [2023-12-27 13:06:37,721][20381] Updated weights for policy 1, policy_version 1075912 (0.0007) [2023-12-27 13:06:37,750][20347] Updated weights for policy 0, policy_version 1076852 (0.0007) [2023-12-27 13:06:37,774][20381] Updated weights for policy 1, policy_version 1075922 (0.0006) [2023-12-27 13:06:37,809][20347] Updated weights for policy 0, policy_version 1076862 (0.0007) [2023-12-27 13:06:37,867][20347] Updated weights for policy 0, policy_version 1076872 (0.0005) [2023-12-27 13:06:38,417][20347] Updated weights for policy 0, policy_version 1076882 (0.0008) [2023-12-27 13:06:38,439][20381] Updated weights for policy 1, policy_version 1075932 (0.0008) [2023-12-27 13:06:38,441][20291] KL-divergence is very high: 433.7307 [2023-12-27 13:06:38,479][20347] Updated weights for policy 0, policy_version 1076892 (0.0007) [2023-12-27 13:06:38,492][20291] KL-divergence is very high: 321.9163 [2023-12-27 13:06:38,495][20381] Updated weights for policy 1, policy_version 1075942 (0.0011) [2023-12-27 13:06:38,532][20347] Updated weights for policy 0, policy_version 1076902 (0.0006) [2023-12-27 13:06:38,534][20291] KL-divergence is very high: 103.1837 [2023-12-27 13:06:38,552][20381] Updated weights for policy 1, policy_version 1075952 (0.0009) [2023-12-27 13:06:39,155][20381] Updated weights for policy 1, policy_version 1075962 (0.0006) [2023-12-27 13:06:39,199][20381] Updated weights for policy 1, policy_version 1075972 (0.0008) [2023-12-27 13:06:39,233][20347] Updated weights for policy 0, policy_version 1076912 (0.0007) [2023-12-27 13:06:39,267][20381] Updated weights for policy 1, policy_version 1075982 (0.0007) [2023-12-27 13:06:39,295][20347] Updated weights for policy 0, policy_version 1076922 (0.0007) [2023-12-27 13:06:39,327][20381] Updated weights for policy 1, policy_version 1075992 (0.0006) [2023-12-27 13:06:39,354][20347] Updated weights for policy 0, policy_version 1076932 (0.0007) [2023-12-27 13:06:40,070][20381] Updated weights for policy 1, policy_version 1076002 (0.0008) [2023-12-27 13:06:40,129][20381] Updated weights for policy 1, policy_version 1076012 (0.0006) [2023-12-27 13:06:40,131][20347] Updated weights for policy 0, policy_version 1076942 (0.0008) [2023-12-27 13:06:40,184][20381] Updated weights for policy 1, policy_version 1076022 (0.0007) [2023-12-27 13:06:40,194][20347] Updated weights for policy 0, policy_version 1076952 (0.0008) [2023-12-27 13:06:40,254][20347] Updated weights for policy 0, policy_version 1076962 (0.0008) [2023-12-27 13:06:40,861][20381] Updated weights for policy 1, policy_version 1076032 (0.0007) [2023-12-27 13:06:40,907][20381] Updated weights for policy 1, policy_version 1076042 (0.0005) [2023-12-27 13:06:40,908][20347] Updated weights for policy 0, policy_version 1076972 (0.0008) [2023-12-27 13:06:40,951][20347] Updated weights for policy 0, policy_version 1076982 (0.0005) [2023-12-27 13:06:40,954][20381] Updated weights for policy 1, policy_version 1076052 (0.0005) [2023-12-27 13:06:41,006][20347] Updated weights for policy 0, policy_version 1076992 (0.0006) [2023-12-27 13:06:41,668][20381] Updated weights for policy 1, policy_version 1076062 (0.0006) [2023-12-27 13:06:41,729][20381] Updated weights for policy 1, policy_version 1076072 (0.0008) [2023-12-27 13:06:41,746][20347] Updated weights for policy 0, policy_version 1077002 (0.0008) [2023-12-27 13:06:41,796][20381] Updated weights for policy 1, policy_version 1076082 (0.0007) [2023-12-27 13:06:41,811][20347] Updated weights for policy 0, policy_version 1077012 (0.0007) [2023-12-27 13:06:41,871][20347] Updated weights for policy 0, policy_version 1077022 (0.0006) [2023-12-27 13:06:41,937][20347] Updated weights for policy 0, policy_version 1077032 (0.0007) [2023-12-27 13:06:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 551272448. Throughput: 0: 10226.1, 1: 10353.0. Samples: 551264544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:42,364][19170] Avg episode reward: [(0, '922.500'), (1, '853.410')] [2023-12-27 13:06:42,472][20381] Updated weights for policy 1, policy_version 1076092 (0.0006) [2023-12-27 13:06:42,533][20381] Updated weights for policy 1, policy_version 1076102 (0.0006) [2023-12-27 13:06:42,595][20381] Updated weights for policy 1, policy_version 1076112 (0.0007) [2023-12-27 13:06:42,632][20347] Updated weights for policy 0, policy_version 1077042 (0.0006) [2023-12-27 13:06:42,686][20347] Updated weights for policy 0, policy_version 1077052 (0.0007) [2023-12-27 13:06:42,750][20347] Updated weights for policy 0, policy_version 1077062 (0.0008) [2023-12-27 13:06:43,240][20381] Updated weights for policy 1, policy_version 1076122 (0.0007) [2023-12-27 13:06:43,302][20381] Updated weights for policy 1, policy_version 1076132 (0.0005) [2023-12-27 13:06:43,360][20381] Updated weights for policy 1, policy_version 1076142 (0.0005) [2023-12-27 13:06:43,414][20381] Updated weights for policy 1, policy_version 1076152 (0.0010) [2023-12-27 13:06:43,422][20347] Updated weights for policy 0, policy_version 1077072 (0.0007) [2023-12-27 13:06:43,482][20347] Updated weights for policy 0, policy_version 1077082 (0.0007) [2023-12-27 13:06:43,528][20347] Updated weights for policy 0, policy_version 1077092 (0.0007) [2023-12-27 13:06:44,024][20381] Updated weights for policy 1, policy_version 1076162 (0.0006) [2023-12-27 13:06:44,074][20381] Updated weights for policy 1, policy_version 1076172 (0.0005) [2023-12-27 13:06:44,089][20347] Updated weights for policy 0, policy_version 1077102 (0.0007) [2023-12-27 13:06:44,134][20381] Updated weights for policy 1, policy_version 1076182 (0.0007) [2023-12-27 13:06:44,136][20347] Updated weights for policy 0, policy_version 1077112 (0.0005) [2023-12-27 13:06:44,188][20347] Updated weights for policy 0, policy_version 1077122 (0.0005) [2023-12-27 13:06:44,676][20381] Updated weights for policy 1, policy_version 1076192 (0.0005) [2023-12-27 13:06:44,729][20381] Updated weights for policy 1, policy_version 1076202 (0.0006) [2023-12-27 13:06:44,792][20381] Updated weights for policy 1, policy_version 1076212 (0.0007) [2023-12-27 13:06:44,822][20347] Updated weights for policy 0, policy_version 1077132 (0.0008) [2023-12-27 13:06:44,881][20347] Updated weights for policy 0, policy_version 1077142 (0.0010) [2023-12-27 13:06:44,935][20347] Updated weights for policy 0, policy_version 1077152 (0.0011) [2023-12-27 13:06:45,503][20381] Updated weights for policy 1, policy_version 1076222 (0.0008) [2023-12-27 13:06:45,556][20381] Updated weights for policy 1, policy_version 1076232 (0.0007) [2023-12-27 13:06:45,616][20381] Updated weights for policy 1, policy_version 1076242 (0.0007) [2023-12-27 13:06:45,634][20347] Updated weights for policy 0, policy_version 1077162 (0.0009) [2023-12-27 13:06:45,685][20347] Updated weights for policy 0, policy_version 1077172 (0.0010) [2023-12-27 13:06:45,737][20347] Updated weights for policy 0, policy_version 1077182 (0.0010) [2023-12-27 13:06:45,789][20347] Updated weights for policy 0, policy_version 1077192 (0.0010) [2023-12-27 13:06:46,281][20381] Updated weights for policy 1, policy_version 1076252 (0.0006) [2023-12-27 13:06:46,306][20292] KL-divergence is very high: 240.9691 [2023-12-27 13:06:46,341][20381] Updated weights for policy 1, policy_version 1076262 (0.0007) [2023-12-27 13:06:46,352][20292] KL-divergence is very high: 103.1442 [2023-12-27 13:06:46,396][20381] Updated weights for policy 1, policy_version 1076272 (0.0007) [2023-12-27 13:06:46,452][20347] Updated weights for policy 0, policy_version 1077202 (0.0009) [2023-12-27 13:06:46,501][20347] Updated weights for policy 0, policy_version 1077212 (0.0010) [2023-12-27 13:06:46,546][20347] Updated weights for policy 0, policy_version 1077222 (0.0010) [2023-12-27 13:06:47,048][20381] Updated weights for policy 1, policy_version 1076282 (0.0009) [2023-12-27 13:06:47,098][20381] Updated weights for policy 1, policy_version 1076292 (0.0007) [2023-12-27 13:06:47,146][20381] Updated weights for policy 1, policy_version 1076302 (0.0005) [2023-12-27 13:06:47,199][20381] Updated weights for policy 1, policy_version 1076312 (0.0005) [2023-12-27 13:06:47,244][20347] Updated weights for policy 0, policy_version 1077232 (0.0010) [2023-12-27 13:06:47,292][20347] Updated weights for policy 0, policy_version 1077242 (0.0010) [2023-12-27 13:06:47,340][20347] Updated weights for policy 0, policy_version 1077252 (0.0010) [2023-12-27 13:06:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 551387136. Throughput: 0: 10281.3, 1: 10376.3. Samples: 551390180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:47,363][19170] Avg episode reward: [(0, '921.890'), (1, '843.340')] [2023-12-27 13:06:47,798][20381] Updated weights for policy 1, policy_version 1076322 (0.0008) [2023-12-27 13:06:47,856][20381] Updated weights for policy 1, policy_version 1076332 (0.0007) [2023-12-27 13:06:47,910][20381] Updated weights for policy 1, policy_version 1076342 (0.0007) [2023-12-27 13:06:48,038][20347] Updated weights for policy 0, policy_version 1077262 (0.0009) [2023-12-27 13:06:48,091][20347] Updated weights for policy 0, policy_version 1077272 (0.0008) [2023-12-27 13:06:48,148][20347] Updated weights for policy 0, policy_version 1077282 (0.0008) [2023-12-27 13:06:48,583][20381] Updated weights for policy 1, policy_version 1076352 (0.0008) [2023-12-27 13:06:48,642][20381] Updated weights for policy 1, policy_version 1076362 (0.0008) [2023-12-27 13:06:48,704][20381] Updated weights for policy 1, policy_version 1076372 (0.0007) [2023-12-27 13:06:48,856][20347] Updated weights for policy 0, policy_version 1077292 (0.0007) [2023-12-27 13:06:48,911][20347] Updated weights for policy 0, policy_version 1077302 (0.0006) [2023-12-27 13:06:48,964][20347] Updated weights for policy 0, policy_version 1077312 (0.0005) [2023-12-27 13:06:49,325][20381] Updated weights for policy 1, policy_version 1076382 (0.0006) [2023-12-27 13:06:49,388][20381] Updated weights for policy 1, policy_version 1076392 (0.0010) [2023-12-27 13:06:49,443][20381] Updated weights for policy 1, policy_version 1076402 (0.0007) [2023-12-27 13:06:49,598][20347] Updated weights for policy 0, policy_version 1077322 (0.0005) [2023-12-27 13:06:49,656][20347] Updated weights for policy 0, policy_version 1077332 (0.0005) [2023-12-27 13:06:49,717][20347] Updated weights for policy 0, policy_version 1077342 (0.0005) [2023-12-27 13:06:49,777][20347] Updated weights for policy 0, policy_version 1077352 (0.0005) [2023-12-27 13:06:50,151][20381] Updated weights for policy 1, policy_version 1076412 (0.0009) [2023-12-27 13:06:50,207][20381] Updated weights for policy 1, policy_version 1076422 (0.0011) [2023-12-27 13:06:50,267][20381] Updated weights for policy 1, policy_version 1076432 (0.0010) [2023-12-27 13:06:50,380][20347] Updated weights for policy 0, policy_version 1077362 (0.0008) [2023-12-27 13:06:50,425][20347] Updated weights for policy 0, policy_version 1077372 (0.0010) [2023-12-27 13:06:50,477][20347] Updated weights for policy 0, policy_version 1077382 (0.0010) [2023-12-27 13:06:50,973][20381] Updated weights for policy 1, policy_version 1076442 (0.0010) [2023-12-27 13:06:51,033][20381] Updated weights for policy 1, policy_version 1076452 (0.0008) [2023-12-27 13:06:51,087][20381] Updated weights for policy 1, policy_version 1076462 (0.0008) [2023-12-27 13:06:51,106][20292] KL-divergence is very high: 345.4631 [2023-12-27 13:06:51,111][20347] Updated weights for policy 0, policy_version 1077392 (0.0006) [2023-12-27 13:06:51,136][20381] Updated weights for policy 1, policy_version 1076472 (0.0008) [2023-12-27 13:06:51,172][20347] Updated weights for policy 0, policy_version 1077402 (0.0006) [2023-12-27 13:06:51,227][20347] Updated weights for policy 0, policy_version 1077412 (0.0006) [2023-12-27 13:06:51,933][20381] Updated weights for policy 1, policy_version 1076482 (0.0006) [2023-12-27 13:06:51,938][20347] Updated weights for policy 0, policy_version 1077422 (0.0006) [2023-12-27 13:06:51,989][20347] Updated weights for policy 0, policy_version 1077432 (0.0006) [2023-12-27 13:06:51,990][20381] Updated weights for policy 1, policy_version 1076492 (0.0008) [2023-12-27 13:06:52,040][20347] Updated weights for policy 0, policy_version 1077442 (0.0006) [2023-12-27 13:06:52,051][20381] Updated weights for policy 1, policy_version 1076502 (0.0007) [2023-12-27 13:06:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 551485440. Throughput: 0: 10293.2, 1: 10398.5. Samples: 551452852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:52,363][19170] Avg episode reward: [(0, '941.490'), (1, '872.750')] [2023-12-27 13:06:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001076504_275611648.pth... [2023-12-27 13:06:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001077448_275873792.pth... [2023-12-27 13:06:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001075288_275300352.pth [2023-12-27 13:06:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001076232_275562496.pth [2023-12-27 13:06:52,722][20381] Updated weights for policy 1, policy_version 1076512 (0.0008) [2023-12-27 13:06:52,723][20347] Updated weights for policy 0, policy_version 1077452 (0.0006) [2023-12-27 13:06:52,778][20381] Updated weights for policy 1, policy_version 1076522 (0.0006) [2023-12-27 13:06:52,779][20347] Updated weights for policy 0, policy_version 1077462 (0.0008) [2023-12-27 13:06:52,838][20381] Updated weights for policy 1, policy_version 1076532 (0.0005) [2023-12-27 13:06:52,843][20347] Updated weights for policy 0, policy_version 1077472 (0.0007) [2023-12-27 13:06:53,418][20381] Updated weights for policy 1, policy_version 1076542 (0.0007) [2023-12-27 13:06:53,471][20381] Updated weights for policy 1, policy_version 1076552 (0.0009) [2023-12-27 13:06:53,486][20347] Updated weights for policy 0, policy_version 1077482 (0.0007) [2023-12-27 13:06:53,526][20381] Updated weights for policy 1, policy_version 1076562 (0.0008) [2023-12-27 13:06:53,535][20347] Updated weights for policy 0, policy_version 1077492 (0.0005) [2023-12-27 13:06:53,587][20347] Updated weights for policy 0, policy_version 1077502 (0.0007) [2023-12-27 13:06:53,638][20347] Updated weights for policy 0, policy_version 1077512 (0.0007) [2023-12-27 13:06:54,285][20381] Updated weights for policy 1, policy_version 1076572 (0.0007) [2023-12-27 13:06:54,297][20347] Updated weights for policy 0, policy_version 1077522 (0.0007) [2023-12-27 13:06:54,337][20381] Updated weights for policy 1, policy_version 1076582 (0.0005) [2023-12-27 13:06:54,349][20347] Updated weights for policy 0, policy_version 1077532 (0.0005) [2023-12-27 13:06:54,392][20381] Updated weights for policy 1, policy_version 1076592 (0.0005) [2023-12-27 13:06:54,405][20347] Updated weights for policy 0, policy_version 1077542 (0.0006) [2023-12-27 13:06:55,034][20347] Updated weights for policy 0, policy_version 1077552 (0.0007) [2023-12-27 13:06:55,091][20347] Updated weights for policy 0, policy_version 1077562 (0.0008) [2023-12-27 13:06:55,113][20381] Updated weights for policy 1, policy_version 1076602 (0.0006) [2023-12-27 13:06:55,145][20347] Updated weights for policy 0, policy_version 1077572 (0.0006) [2023-12-27 13:06:55,176][20381] Updated weights for policy 1, policy_version 1076612 (0.0006) [2023-12-27 13:06:55,234][20381] Updated weights for policy 1, policy_version 1076622 (0.0008) [2023-12-27 13:06:55,287][20381] Updated weights for policy 1, policy_version 1076632 (0.0007) [2023-12-27 13:06:55,796][20347] Updated weights for policy 0, policy_version 1077582 (0.0007) [2023-12-27 13:06:55,852][20347] Updated weights for policy 0, policy_version 1077592 (0.0008) [2023-12-27 13:06:55,911][20347] Updated weights for policy 0, policy_version 1077602 (0.0007) [2023-12-27 13:06:55,988][20381] Updated weights for policy 1, policy_version 1076642 (0.0007) [2023-12-27 13:06:56,036][20381] Updated weights for policy 1, policy_version 1076652 (0.0007) [2023-12-27 13:06:56,096][20381] Updated weights for policy 1, policy_version 1076662 (0.0007) [2023-12-27 13:06:56,574][20347] Updated weights for policy 0, policy_version 1077612 (0.0007) [2023-12-27 13:06:56,629][20347] Updated weights for policy 0, policy_version 1077622 (0.0007) [2023-12-27 13:06:56,682][20347] Updated weights for policy 0, policy_version 1077632 (0.0010) [2023-12-27 13:06:56,794][20381] Updated weights for policy 1, policy_version 1076672 (0.0007) [2023-12-27 13:06:56,857][20381] Updated weights for policy 1, policy_version 1076682 (0.0007) [2023-12-27 13:06:56,910][20381] Updated weights for policy 1, policy_version 1076692 (0.0008) [2023-12-27 13:06:57,328][20347] Updated weights for policy 0, policy_version 1077642 (0.0009) [2023-12-27 13:06:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 551583744. Throughput: 0: 10283.3, 1: 10423.3. Samples: 551576156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:06:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.270')] [2023-12-27 13:06:57,393][20347] Updated weights for policy 0, policy_version 1077652 (0.0006) [2023-12-27 13:06:57,458][20347] Updated weights for policy 0, policy_version 1077662 (0.0005) [2023-12-27 13:06:57,523][20347] Updated weights for policy 0, policy_version 1077672 (0.0007) [2023-12-27 13:06:57,541][20381] Updated weights for policy 1, policy_version 1076702 (0.0007) [2023-12-27 13:06:57,590][20381] Updated weights for policy 1, policy_version 1076712 (0.0005) [2023-12-27 13:06:57,644][20381] Updated weights for policy 1, policy_version 1076722 (0.0006) [2023-12-27 13:06:58,044][20347] Updated weights for policy 0, policy_version 1077682 (0.0008) [2023-12-27 13:06:58,099][20347] Updated weights for policy 0, policy_version 1077692 (0.0008) [2023-12-27 13:06:58,157][20347] Updated weights for policy 0, policy_version 1077702 (0.0008) [2023-12-27 13:06:58,279][20381] Updated weights for policy 1, policy_version 1076732 (0.0007) [2023-12-27 13:06:58,338][20381] Updated weights for policy 1, policy_version 1076742 (0.0008) [2023-12-27 13:06:58,405][20381] Updated weights for policy 1, policy_version 1076752 (0.0008) [2023-12-27 13:06:58,944][20347] Updated weights for policy 0, policy_version 1077712 (0.0007) [2023-12-27 13:06:59,001][20347] Updated weights for policy 0, policy_version 1077722 (0.0008) [2023-12-27 13:06:59,060][20347] Updated weights for policy 0, policy_version 1077732 (0.0007) [2023-12-27 13:06:59,120][20381] Updated weights for policy 1, policy_version 1076762 (0.0007) [2023-12-27 13:06:59,171][20381] Updated weights for policy 1, policy_version 1076772 (0.0007) [2023-12-27 13:06:59,230][20381] Updated weights for policy 1, policy_version 1076782 (0.0008) [2023-12-27 13:06:59,296][20381] Updated weights for policy 1, policy_version 1076792 (0.0006) [2023-12-27 13:06:59,768][20347] Updated weights for policy 0, policy_version 1077742 (0.0009) [2023-12-27 13:06:59,820][20347] Updated weights for policy 0, policy_version 1077752 (0.0010) [2023-12-27 13:06:59,883][20347] Updated weights for policy 0, policy_version 1077762 (0.0010) [2023-12-27 13:07:00,018][20381] Updated weights for policy 1, policy_version 1076802 (0.0008) [2023-12-27 13:07:00,071][20381] Updated weights for policy 1, policy_version 1076812 (0.0009) [2023-12-27 13:07:00,123][20381] Updated weights for policy 1, policy_version 1076822 (0.0007) [2023-12-27 13:07:00,524][20347] Updated weights for policy 0, policy_version 1077772 (0.0006) [2023-12-27 13:07:00,580][20347] Updated weights for policy 0, policy_version 1077782 (0.0007) [2023-12-27 13:07:00,633][20347] Updated weights for policy 0, policy_version 1077792 (0.0008) [2023-12-27 13:07:00,744][20381] Updated weights for policy 1, policy_version 1076832 (0.0007) [2023-12-27 13:07:00,796][20381] Updated weights for policy 1, policy_version 1076842 (0.0006) [2023-12-27 13:07:00,853][20381] Updated weights for policy 1, policy_version 1076852 (0.0005) [2023-12-27 13:07:01,271][20347] Updated weights for policy 0, policy_version 1077802 (0.0008) [2023-12-27 13:07:01,326][20347] Updated weights for policy 0, policy_version 1077812 (0.0010) [2023-12-27 13:07:01,395][20347] Updated weights for policy 0, policy_version 1077822 (0.0009) [2023-12-27 13:07:01,454][20347] Updated weights for policy 0, policy_version 1077832 (0.0006) [2023-12-27 13:07:01,507][20381] Updated weights for policy 1, policy_version 1076862 (0.0007) [2023-12-27 13:07:01,571][20381] Updated weights for policy 1, policy_version 1076872 (0.0007) [2023-12-27 13:07:01,634][20381] Updated weights for policy 1, policy_version 1076882 (0.0007) [2023-12-27 13:07:02,061][20347] Updated weights for policy 0, policy_version 1077842 (0.0008) [2023-12-27 13:07:02,107][20347] Updated weights for policy 0, policy_version 1077852 (0.0009) [2023-12-27 13:07:02,156][20347] Updated weights for policy 0, policy_version 1077862 (0.0007) [2023-12-27 13:07:02,291][20381] Updated weights for policy 1, policy_version 1076892 (0.0009) [2023-12-27 13:07:02,344][20381] Updated weights for policy 1, policy_version 1076902 (0.0007) [2023-12-27 13:07:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 551690240. Throughput: 0: 10307.3, 1: 10417.7. Samples: 551700988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:02,363][19170] Avg episode reward: [(0, '980.760'), (1, '862.880')] [2023-12-27 13:07:02,407][20381] Updated weights for policy 1, policy_version 1076912 (0.0008) [2023-12-27 13:07:02,965][20381] Updated weights for policy 1, policy_version 1076922 (0.0005) [2023-12-27 13:07:02,966][20347] Updated weights for policy 0, policy_version 1077872 (0.0008) [2023-12-27 13:07:03,023][20381] Updated weights for policy 1, policy_version 1076932 (0.0006) [2023-12-27 13:07:03,026][20347] Updated weights for policy 0, policy_version 1077882 (0.0008) [2023-12-27 13:07:03,076][20347] Updated weights for policy 0, policy_version 1077892 (0.0007) [2023-12-27 13:07:03,081][20381] Updated weights for policy 1, policy_version 1076942 (0.0005) [2023-12-27 13:07:03,140][20381] Updated weights for policy 1, policy_version 1076952 (0.0005) [2023-12-27 13:07:03,730][20381] Updated weights for policy 1, policy_version 1076962 (0.0008) [2023-12-27 13:07:03,780][20381] Updated weights for policy 1, policy_version 1076972 (0.0006) [2023-12-27 13:07:03,794][20347] Updated weights for policy 0, policy_version 1077902 (0.0008) [2023-12-27 13:07:03,843][20381] Updated weights for policy 1, policy_version 1076982 (0.0006) [2023-12-27 13:07:03,857][20347] Updated weights for policy 0, policy_version 1077912 (0.0007) [2023-12-27 13:07:03,916][20347] Updated weights for policy 0, policy_version 1077922 (0.0007) [2023-12-27 13:07:04,438][20381] Updated weights for policy 1, policy_version 1076992 (0.0008) [2023-12-27 13:07:04,482][20292] KL-divergence is very high: 108.5077 [2023-12-27 13:07:04,491][20381] Updated weights for policy 1, policy_version 1077002 (0.0007) [2023-12-27 13:07:04,544][20381] Updated weights for policy 1, policy_version 1077012 (0.0008) [2023-12-27 13:07:04,661][20347] Updated weights for policy 0, policy_version 1077932 (0.0008) [2023-12-27 13:07:04,720][20347] Updated weights for policy 0, policy_version 1077942 (0.0008) [2023-12-27 13:07:04,768][20347] Updated weights for policy 0, policy_version 1077952 (0.0007) [2023-12-27 13:07:05,196][20381] Updated weights for policy 1, policy_version 1077022 (0.0007) [2023-12-27 13:07:05,251][20381] Updated weights for policy 1, policy_version 1077032 (0.0008) [2023-12-27 13:07:05,307][20381] Updated weights for policy 1, policy_version 1077042 (0.0005) [2023-12-27 13:07:05,446][20347] Updated weights for policy 0, policy_version 1077962 (0.0007) [2023-12-27 13:07:05,494][20347] Updated weights for policy 0, policy_version 1077972 (0.0005) [2023-12-27 13:07:05,539][20347] Updated weights for policy 0, policy_version 1077982 (0.0005) [2023-12-27 13:07:05,598][20347] Updated weights for policy 0, policy_version 1077992 (0.0005) [2023-12-27 13:07:05,908][20381] Updated weights for policy 1, policy_version 1077052 (0.0006) [2023-12-27 13:07:05,961][20381] Updated weights for policy 1, policy_version 1077062 (0.0007) [2023-12-27 13:07:06,020][20381] Updated weights for policy 1, policy_version 1077072 (0.0007) [2023-12-27 13:07:06,232][20347] Updated weights for policy 0, policy_version 1078002 (0.0008) [2023-12-27 13:07:06,296][20347] Updated weights for policy 0, policy_version 1078012 (0.0008) [2023-12-27 13:07:06,356][20347] Updated weights for policy 0, policy_version 1078022 (0.0007) [2023-12-27 13:07:06,700][20381] Updated weights for policy 1, policy_version 1077082 (0.0007) [2023-12-27 13:07:06,746][20381] Updated weights for policy 1, policy_version 1077092 (0.0008) [2023-12-27 13:07:06,792][20381] Updated weights for policy 1, policy_version 1077102 (0.0007) [2023-12-27 13:07:06,847][20381] Updated weights for policy 1, policy_version 1077112 (0.0008) [2023-12-27 13:07:07,049][20347] Updated weights for policy 0, policy_version 1078032 (0.0007) [2023-12-27 13:07:07,098][20347] Updated weights for policy 0, policy_version 1078042 (0.0009) [2023-12-27 13:07:07,152][20347] Updated weights for policy 0, policy_version 1078052 (0.0009) [2023-12-27 13:07:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 551796736. Throughput: 0: 10230.9, 1: 10485.2. Samples: 551762972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:07,364][19170] Avg episode reward: [(0, '951.390'), (1, '843.280')] [2023-12-27 13:07:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001077112_275767296.pth... [2023-12-27 13:07:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001078056_276029440.pth... [2023-12-27 13:07:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001075896_275456000.pth [2023-12-27 13:07:07,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001076840_275718144.pth [2023-12-27 13:07:07,464][20381] Updated weights for policy 1, policy_version 1077122 (0.0008) [2023-12-27 13:07:07,521][20381] Updated weights for policy 1, policy_version 1077132 (0.0008) [2023-12-27 13:07:07,577][20381] Updated weights for policy 1, policy_version 1077142 (0.0006) [2023-12-27 13:07:07,933][20347] Updated weights for policy 0, policy_version 1078062 (0.0008) [2023-12-27 13:07:07,991][20347] Updated weights for policy 0, policy_version 1078072 (0.0007) [2023-12-27 13:07:08,015][20291] KL-divergence is very high: 113.7297 [2023-12-27 13:07:08,044][20347] Updated weights for policy 0, policy_version 1078082 (0.0007) [2023-12-27 13:07:08,230][20381] Updated weights for policy 1, policy_version 1077152 (0.0006) [2023-12-27 13:07:08,287][20381] Updated weights for policy 1, policy_version 1077162 (0.0005) [2023-12-27 13:07:08,354][20381] Updated weights for policy 1, policy_version 1077172 (0.0007) [2023-12-27 13:07:08,716][20347] Updated weights for policy 0, policy_version 1078092 (0.0006) [2023-12-27 13:07:08,771][20347] Updated weights for policy 0, policy_version 1078102 (0.0006) [2023-12-27 13:07:08,827][20347] Updated weights for policy 0, policy_version 1078112 (0.0007) [2023-12-27 13:07:08,996][20381] Updated weights for policy 1, policy_version 1077182 (0.0009) [2023-12-27 13:07:09,051][20381] Updated weights for policy 1, policy_version 1077192 (0.0007) [2023-12-27 13:07:09,111][20381] Updated weights for policy 1, policy_version 1077202 (0.0007) [2023-12-27 13:07:09,508][20347] Updated weights for policy 0, policy_version 1078122 (0.0007) [2023-12-27 13:07:09,557][20347] Updated weights for policy 0, policy_version 1078132 (0.0007) [2023-12-27 13:07:09,602][20347] Updated weights for policy 0, policy_version 1078142 (0.0007) [2023-12-27 13:07:09,653][20347] Updated weights for policy 0, policy_version 1078152 (0.0007) [2023-12-27 13:07:09,860][20381] Updated weights for policy 1, policy_version 1077212 (0.0008) [2023-12-27 13:07:09,925][20381] Updated weights for policy 1, policy_version 1077222 (0.0009) [2023-12-27 13:07:09,985][20381] Updated weights for policy 1, policy_version 1077232 (0.0011) [2023-12-27 13:07:10,411][20347] Updated weights for policy 0, policy_version 1078162 (0.0008) [2023-12-27 13:07:10,470][20347] Updated weights for policy 0, policy_version 1078172 (0.0007) [2023-12-27 13:07:10,522][20347] Updated weights for policy 0, policy_version 1078182 (0.0007) [2023-12-27 13:07:10,702][20381] Updated weights for policy 1, policy_version 1077242 (0.0010) [2023-12-27 13:07:10,758][20381] Updated weights for policy 1, policy_version 1077252 (0.0008) [2023-12-27 13:07:10,817][20381] Updated weights for policy 1, policy_version 1077262 (0.0008) [2023-12-27 13:07:10,881][20381] Updated weights for policy 1, policy_version 1077272 (0.0007) [2023-12-27 13:07:11,175][20347] Updated weights for policy 0, policy_version 1078192 (0.0009) [2023-12-27 13:07:11,245][20347] Updated weights for policy 0, policy_version 1078202 (0.0008) [2023-12-27 13:07:11,309][20347] Updated weights for policy 0, policy_version 1078212 (0.0007) [2023-12-27 13:07:11,661][20381] Updated weights for policy 1, policy_version 1077282 (0.0008) [2023-12-27 13:07:11,728][20381] Updated weights for policy 1, policy_version 1077292 (0.0008) [2023-12-27 13:07:11,781][20381] Updated weights for policy 1, policy_version 1077302 (0.0007) [2023-12-27 13:07:12,095][20347] Updated weights for policy 0, policy_version 1078222 (0.0007) [2023-12-27 13:07:12,152][20347] Updated weights for policy 0, policy_version 1078232 (0.0006) [2023-12-27 13:07:12,207][20347] Updated weights for policy 0, policy_version 1078242 (0.0006) [2023-12-27 13:07:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 551895040. Throughput: 0: 10246.5, 1: 10461.8. Samples: 551885428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:12,363][19170] Avg episode reward: [(0, '960.840'), (1, '902.190')] [2023-12-27 13:07:12,500][20381] Updated weights for policy 1, policy_version 1077312 (0.0008) [2023-12-27 13:07:12,560][20381] Updated weights for policy 1, policy_version 1077322 (0.0006) [2023-12-27 13:07:12,624][20381] Updated weights for policy 1, policy_version 1077332 (0.0006) [2023-12-27 13:07:12,917][20347] Updated weights for policy 0, policy_version 1078252 (0.0009) [2023-12-27 13:07:12,969][20347] Updated weights for policy 0, policy_version 1078262 (0.0007) [2023-12-27 13:07:13,020][20347] Updated weights for policy 0, policy_version 1078272 (0.0008) [2023-12-27 13:07:13,160][20381] Updated weights for policy 1, policy_version 1077342 (0.0007) [2023-12-27 13:07:13,206][20381] Updated weights for policy 1, policy_version 1077352 (0.0008) [2023-12-27 13:07:13,252][20381] Updated weights for policy 1, policy_version 1077362 (0.0008) [2023-12-27 13:07:13,754][20347] Updated weights for policy 0, policy_version 1078282 (0.0008) [2023-12-27 13:07:13,802][20347] Updated weights for policy 0, policy_version 1078292 (0.0007) [2023-12-27 13:07:13,844][20347] Updated weights for policy 0, policy_version 1078302 (0.0007) [2023-12-27 13:07:13,892][20347] Updated weights for policy 0, policy_version 1078312 (0.0007) [2023-12-27 13:07:13,903][20381] Updated weights for policy 1, policy_version 1077372 (0.0008) [2023-12-27 13:07:13,948][20381] Updated weights for policy 1, policy_version 1077382 (0.0007) [2023-12-27 13:07:13,996][20381] Updated weights for policy 1, policy_version 1077392 (0.0008) [2023-12-27 13:07:14,637][20347] Updated weights for policy 0, policy_version 1078322 (0.0008) [2023-12-27 13:07:14,664][20381] Updated weights for policy 1, policy_version 1077402 (0.0007) [2023-12-27 13:07:14,681][20347] Updated weights for policy 0, policy_version 1078332 (0.0007) [2023-12-27 13:07:14,716][20381] Updated weights for policy 1, policy_version 1077412 (0.0005) [2023-12-27 13:07:14,733][20347] Updated weights for policy 0, policy_version 1078342 (0.0005) [2023-12-27 13:07:14,780][20381] Updated weights for policy 1, policy_version 1077422 (0.0008) [2023-12-27 13:07:14,842][20381] Updated weights for policy 1, policy_version 1077432 (0.0008) [2023-12-27 13:07:15,432][20347] Updated weights for policy 0, policy_version 1078352 (0.0009) [2023-12-27 13:07:15,484][20347] Updated weights for policy 0, policy_version 1078362 (0.0010) [2023-12-27 13:07:15,534][20381] Updated weights for policy 1, policy_version 1077442 (0.0005) [2023-12-27 13:07:15,539][20347] Updated weights for policy 0, policy_version 1078372 (0.0010) [2023-12-27 13:07:15,591][20381] Updated weights for policy 1, policy_version 1077452 (0.0010) [2023-12-27 13:07:15,646][20381] Updated weights for policy 1, policy_version 1077462 (0.0010) [2023-12-27 13:07:16,216][20347] Updated weights for policy 0, policy_version 1078382 (0.0010) [2023-12-27 13:07:16,270][20347] Updated weights for policy 0, policy_version 1078392 (0.0010) [2023-12-27 13:07:16,290][20381] Updated weights for policy 1, policy_version 1077472 (0.0011) [2023-12-27 13:07:16,327][20347] Updated weights for policy 0, policy_version 1078402 (0.0010) [2023-12-27 13:07:16,341][20381] Updated weights for policy 1, policy_version 1077482 (0.0010) [2023-12-27 13:07:16,387][20381] Updated weights for policy 1, policy_version 1077492 (0.0010) [2023-12-27 13:07:16,953][20347] Updated weights for policy 0, policy_version 1078412 (0.0010) [2023-12-27 13:07:16,984][20381] Updated weights for policy 1, policy_version 1077502 (0.0008) [2023-12-27 13:07:17,003][20347] Updated weights for policy 0, policy_version 1078422 (0.0010) [2023-12-27 13:07:17,033][20381] Updated weights for policy 1, policy_version 1077512 (0.0008) [2023-12-27 13:07:17,055][20347] Updated weights for policy 0, policy_version 1078432 (0.0009) [2023-12-27 13:07:17,087][20381] Updated weights for policy 1, policy_version 1077522 (0.0007) [2023-12-27 13:07:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 552001536. Throughput: 0: 10267.8, 1: 10426.5. Samples: 552008144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:17,363][19170] Avg episode reward: [(0, '970.740'), (1, '863.120')] [2023-12-27 13:07:17,691][20347] Updated weights for policy 0, policy_version 1078442 (0.0007) [2023-12-27 13:07:17,745][20347] Updated weights for policy 0, policy_version 1078452 (0.0008) [2023-12-27 13:07:17,745][20381] Updated weights for policy 1, policy_version 1077532 (0.0009) [2023-12-27 13:07:17,796][20381] Updated weights for policy 1, policy_version 1077542 (0.0005) [2023-12-27 13:07:17,797][20347] Updated weights for policy 0, policy_version 1078462 (0.0010) [2023-12-27 13:07:17,847][20381] Updated weights for policy 1, policy_version 1077552 (0.0005) [2023-12-27 13:07:17,847][20347] Updated weights for policy 0, policy_version 1078472 (0.0010) [2023-12-27 13:07:18,416][20347] Updated weights for policy 0, policy_version 1078482 (0.0011) [2023-12-27 13:07:18,474][20347] Updated weights for policy 0, policy_version 1078492 (0.0010) [2023-12-27 13:07:18,480][20381] Updated weights for policy 1, policy_version 1077562 (0.0006) [2023-12-27 13:07:18,532][20347] Updated weights for policy 0, policy_version 1078502 (0.0010) [2023-12-27 13:07:18,535][20292] KL-divergence is very high: 148.2107 [2023-12-27 13:07:18,537][20381] Updated weights for policy 1, policy_version 1077572 (0.0011) [2023-12-27 13:07:18,596][20381] Updated weights for policy 1, policy_version 1077582 (0.0011) [2023-12-27 13:07:18,657][20381] Updated weights for policy 1, policy_version 1077592 (0.0011) [2023-12-27 13:07:19,219][20347] Updated weights for policy 0, policy_version 1078512 (0.0010) [2023-12-27 13:07:19,285][20347] Updated weights for policy 0, policy_version 1078522 (0.0010) [2023-12-27 13:07:19,349][20347] Updated weights for policy 0, policy_version 1078532 (0.0012) [2023-12-27 13:07:19,358][20381] Updated weights for policy 1, policy_version 1077602 (0.0008) [2023-12-27 13:07:19,417][20381] Updated weights for policy 1, policy_version 1077612 (0.0008) [2023-12-27 13:07:19,478][20381] Updated weights for policy 1, policy_version 1077622 (0.0007) [2023-12-27 13:07:20,081][20347] Updated weights for policy 0, policy_version 1078542 (0.0008) [2023-12-27 13:07:20,139][20347] Updated weights for policy 0, policy_version 1078552 (0.0008) [2023-12-27 13:07:20,197][20347] Updated weights for policy 0, policy_version 1078562 (0.0007) [2023-12-27 13:07:20,206][20381] Updated weights for policy 1, policy_version 1077632 (0.0009) [2023-12-27 13:07:20,264][20381] Updated weights for policy 1, policy_version 1077642 (0.0010) [2023-12-27 13:07:20,329][20381] Updated weights for policy 1, policy_version 1077652 (0.0008) [2023-12-27 13:07:20,897][20347] Updated weights for policy 0, policy_version 1078572 (0.0006) [2023-12-27 13:07:20,961][20347] Updated weights for policy 0, policy_version 1078582 (0.0008) [2023-12-27 13:07:20,962][20381] Updated weights for policy 1, policy_version 1077662 (0.0008) [2023-12-27 13:07:21,018][20381] Updated weights for policy 1, policy_version 1077672 (0.0006) [2023-12-27 13:07:21,021][20347] Updated weights for policy 0, policy_version 1078592 (0.0007) [2023-12-27 13:07:21,083][20381] Updated weights for policy 1, policy_version 1077682 (0.0008) [2023-12-27 13:07:21,811][20347] Updated weights for policy 0, policy_version 1078602 (0.0007) [2023-12-27 13:07:21,850][20381] Updated weights for policy 1, policy_version 1077692 (0.0008) [2023-12-27 13:07:21,878][20347] Updated weights for policy 0, policy_version 1078612 (0.0007) [2023-12-27 13:07:21,908][20381] Updated weights for policy 1, policy_version 1077702 (0.0009) [2023-12-27 13:07:21,938][20347] Updated weights for policy 0, policy_version 1078622 (0.0006) [2023-12-27 13:07:21,965][20381] Updated weights for policy 1, policy_version 1077712 (0.0009) [2023-12-27 13:07:21,991][20347] Updated weights for policy 0, policy_version 1078632 (0.0006) [2023-12-27 13:07:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 552099840. Throughput: 0: 10271.6, 1: 10413.0. Samples: 552070460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:22,363][19170] Avg episode reward: [(0, '941.540'), (1, '863.050')] [2023-12-27 13:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001078632_276176896.pth... [2023-12-27 13:07:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001077720_275922944.pth... [2023-12-27 13:07:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001077448_275873792.pth [2023-12-27 13:07:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001076504_275611648.pth [2023-12-27 13:07:22,645][20347] Updated weights for policy 0, policy_version 1078642 (0.0008) [2023-12-27 13:07:22,702][20347] Updated weights for policy 0, policy_version 1078652 (0.0008) [2023-12-27 13:07:22,724][20381] Updated weights for policy 1, policy_version 1077722 (0.0008) [2023-12-27 13:07:22,753][20347] Updated weights for policy 0, policy_version 1078662 (0.0006) [2023-12-27 13:07:22,786][20381] Updated weights for policy 1, policy_version 1077732 (0.0006) [2023-12-27 13:07:22,851][20381] Updated weights for policy 1, policy_version 1077742 (0.0008) [2023-12-27 13:07:22,914][20381] Updated weights for policy 1, policy_version 1077752 (0.0008) [2023-12-27 13:07:23,408][20347] Updated weights for policy 0, policy_version 1078672 (0.0009) [2023-12-27 13:07:23,467][20347] Updated weights for policy 0, policy_version 1078682 (0.0008) [2023-12-27 13:07:23,523][20347] Updated weights for policy 0, policy_version 1078692 (0.0008) [2023-12-27 13:07:23,622][20381] Updated weights for policy 1, policy_version 1077762 (0.0007) [2023-12-27 13:07:23,675][20381] Updated weights for policy 1, policy_version 1077772 (0.0007) [2023-12-27 13:07:23,736][20381] Updated weights for policy 1, policy_version 1077782 (0.0007) [2023-12-27 13:07:24,175][20347] Updated weights for policy 0, policy_version 1078702 (0.0010) [2023-12-27 13:07:24,244][20347] Updated weights for policy 0, policy_version 1078712 (0.0008) [2023-12-27 13:07:24,295][20347] Updated weights for policy 0, policy_version 1078722 (0.0008) [2023-12-27 13:07:24,318][20381] Updated weights for policy 1, policy_version 1077792 (0.0006) [2023-12-27 13:07:24,369][20381] Updated weights for policy 1, policy_version 1077802 (0.0005) [2023-12-27 13:07:24,425][20381] Updated weights for policy 1, policy_version 1077812 (0.0005) [2023-12-27 13:07:24,951][20347] Updated weights for policy 0, policy_version 1078732 (0.0007) [2023-12-27 13:07:24,979][20381] Updated weights for policy 1, policy_version 1077822 (0.0005) [2023-12-27 13:07:25,010][20347] Updated weights for policy 0, policy_version 1078742 (0.0010) [2023-12-27 13:07:25,028][20381] Updated weights for policy 1, policy_version 1077832 (0.0005) [2023-12-27 13:07:25,059][20347] Updated weights for policy 0, policy_version 1078752 (0.0010) [2023-12-27 13:07:25,077][20381] Updated weights for policy 1, policy_version 1077842 (0.0005) [2023-12-27 13:07:25,614][20381] Updated weights for policy 1, policy_version 1077852 (0.0006) [2023-12-27 13:07:25,653][20347] Updated weights for policy 0, policy_version 1078762 (0.0010) [2023-12-27 13:07:25,660][20381] Updated weights for policy 1, policy_version 1077862 (0.0005) [2023-12-27 13:07:25,701][20347] Updated weights for policy 0, policy_version 1078772 (0.0010) [2023-12-27 13:07:25,715][20381] Updated weights for policy 1, policy_version 1077872 (0.0005) [2023-12-27 13:07:25,768][20347] Updated weights for policy 0, policy_version 1078782 (0.0010) [2023-12-27 13:07:25,816][20347] Updated weights for policy 0, policy_version 1078792 (0.0007) [2023-12-27 13:07:26,309][20381] Updated weights for policy 1, policy_version 1077882 (0.0005) [2023-12-27 13:07:26,362][20381] Updated weights for policy 1, policy_version 1077892 (0.0005) [2023-12-27 13:07:26,426][20381] Updated weights for policy 1, policy_version 1077902 (0.0009) [2023-12-27 13:07:26,477][20347] Updated weights for policy 0, policy_version 1078802 (0.0005) [2023-12-27 13:07:26,477][20381] Updated weights for policy 1, policy_version 1077912 (0.0010) [2023-12-27 13:07:26,534][20347] Updated weights for policy 0, policy_version 1078812 (0.0005) [2023-12-27 13:07:26,584][20347] Updated weights for policy 0, policy_version 1078822 (0.0005) [2023-12-27 13:07:27,071][20381] Updated weights for policy 1, policy_version 1077922 (0.0005) [2023-12-27 13:07:27,115][20292] KL-divergence is very high: 104.6862 [2023-12-27 13:07:27,120][20381] Updated weights for policy 1, policy_version 1077932 (0.0007) [2023-12-27 13:07:27,132][20292] KL-divergence is very high: 157.6978 [2023-12-27 13:07:27,141][20347] Updated weights for policy 0, policy_version 1078832 (0.0009) [2023-12-27 13:07:27,148][20292] KL-divergence is very high: 164.2837 [2023-12-27 13:07:27,158][20292] KL-divergence is very high: 127.1938 [2023-12-27 13:07:27,173][20381] Updated weights for policy 1, policy_version 1077942 (0.0009) [2023-12-27 13:07:27,196][20347] Updated weights for policy 0, policy_version 1078842 (0.0010) [2023-12-27 13:07:27,240][20347] Updated weights for policy 0, policy_version 1078852 (0.0010) [2023-12-27 13:07:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 552214528. Throughput: 0: 10248.1, 1: 10451.8. Samples: 552196040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:07:27,363][19170] Avg episode reward: [(0, '970.760'), (1, '794.290')] [2023-12-27 13:07:27,798][20381] Updated weights for policy 1, policy_version 1077952 (0.0011) [2023-12-27 13:07:27,861][20381] Updated weights for policy 1, policy_version 1077962 (0.0011) [2023-12-27 13:07:27,895][20347] Updated weights for policy 0, policy_version 1078862 (0.0009) [2023-12-27 13:07:27,918][20381] Updated weights for policy 1, policy_version 1077972 (0.0010) [2023-12-27 13:07:27,959][20347] Updated weights for policy 0, policy_version 1078872 (0.0007) [2023-12-27 13:07:28,021][20347] Updated weights for policy 0, policy_version 1078882 (0.0006) [2023-12-27 13:07:28,571][20381] Updated weights for policy 1, policy_version 1077982 (0.0007) [2023-12-27 13:07:28,604][20347] Updated weights for policy 0, policy_version 1078892 (0.0006) [2023-12-27 13:07:28,628][20381] Updated weights for policy 1, policy_version 1077992 (0.0006) [2023-12-27 13:07:28,663][20347] Updated weights for policy 0, policy_version 1078902 (0.0007) [2023-12-27 13:07:28,684][20381] Updated weights for policy 1, policy_version 1078002 (0.0006) [2023-12-27 13:07:28,719][20347] Updated weights for policy 0, policy_version 1078912 (0.0006) [2023-12-27 13:07:29,324][20347] Updated weights for policy 0, policy_version 1078922 (0.0006) [2023-12-27 13:07:29,340][20381] Updated weights for policy 1, policy_version 1078012 (0.0007) [2023-12-27 13:07:29,391][20347] Updated weights for policy 0, policy_version 1078932 (0.0008) [2023-12-27 13:07:29,408][20381] Updated weights for policy 1, policy_version 1078022 (0.0008) [2023-12-27 13:07:29,446][20347] Updated weights for policy 0, policy_version 1078942 (0.0008) [2023-12-27 13:07:29,471][20381] Updated weights for policy 1, policy_version 1078032 (0.0006) [2023-12-27 13:07:29,501][20347] Updated weights for policy 0, policy_version 1078952 (0.0007) [2023-12-27 13:07:30,151][20381] Updated weights for policy 1, policy_version 1078042 (0.0008) [2023-12-27 13:07:30,172][20347] Updated weights for policy 0, policy_version 1078962 (0.0006) [2023-12-27 13:07:30,207][20381] Updated weights for policy 1, policy_version 1078052 (0.0008) [2023-12-27 13:07:30,224][20347] Updated weights for policy 0, policy_version 1078972 (0.0006) [2023-12-27 13:07:30,262][20381] Updated weights for policy 1, policy_version 1078062 (0.0007) [2023-12-27 13:07:30,286][20347] Updated weights for policy 0, policy_version 1078982 (0.0006) [2023-12-27 13:07:30,318][20381] Updated weights for policy 1, policy_version 1078072 (0.0007) [2023-12-27 13:07:30,944][20347] Updated weights for policy 0, policy_version 1078992 (0.0006) [2023-12-27 13:07:30,968][20381] Updated weights for policy 1, policy_version 1078082 (0.0011) [2023-12-27 13:07:31,001][20347] Updated weights for policy 0, policy_version 1079002 (0.0006) [2023-12-27 13:07:31,026][20381] Updated weights for policy 1, policy_version 1078092 (0.0010) [2023-12-27 13:07:31,062][20347] Updated weights for policy 0, policy_version 1079012 (0.0007) [2023-12-27 13:07:31,085][20381] Updated weights for policy 1, policy_version 1078102 (0.0010) [2023-12-27 13:07:31,094][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-27 13:07:31,770][20347] Updated weights for policy 0, policy_version 1079022 (0.0008) [2023-12-27 13:07:31,826][20347] Updated weights for policy 0, policy_version 1079032 (0.0005) [2023-12-27 13:07:31,856][20381] Updated weights for policy 1, policy_version 1078112 (0.0009) [2023-12-27 13:07:31,883][20347] Updated weights for policy 0, policy_version 1079042 (0.0006) [2023-12-27 13:07:31,909][20381] Updated weights for policy 1, policy_version 1078122 (0.0007) [2023-12-27 13:07:31,968][20381] Updated weights for policy 1, policy_version 1078132 (0.0008) [2023-12-27 13:07:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 552312832. Throughput: 0: 10308.3, 1: 10428.0. Samples: 552323316. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.220')] [2023-12-27 13:07:32,536][20347] Updated weights for policy 0, policy_version 1079052 (0.0007) [2023-12-27 13:07:32,593][20347] Updated weights for policy 0, policy_version 1079062 (0.0007) [2023-12-27 13:07:32,654][20347] Updated weights for policy 0, policy_version 1079072 (0.0007) [2023-12-27 13:07:32,678][20381] Updated weights for policy 1, policy_version 1078142 (0.0007) [2023-12-27 13:07:32,733][20381] Updated weights for policy 1, policy_version 1078152 (0.0006) [2023-12-27 13:07:32,757][20292] KL-divergence is very high: 147.1481 [2023-12-27 13:07:32,762][20292] KL-divergence is very high: 186.1797 [2023-12-27 13:07:32,778][20292] KL-divergence is very high: 240.3389 [2023-12-27 13:07:32,785][20292] KL-divergence is very high: 210.8966 [2023-12-27 13:07:32,791][20381] Updated weights for policy 1, policy_version 1078162 (0.0008) [2023-12-27 13:07:32,804][20292] KL-divergence is very high: 166.9495 [2023-12-27 13:07:32,811][20292] KL-divergence is very high: 142.0244 [2023-12-27 13:07:33,275][20347] Updated weights for policy 0, policy_version 1079082 (0.0007) [2023-12-27 13:07:33,325][20347] Updated weights for policy 0, policy_version 1079092 (0.0010) [2023-12-27 13:07:33,382][20347] Updated weights for policy 0, policy_version 1079102 (0.0010) [2023-12-27 13:07:33,426][20381] Updated weights for policy 1, policy_version 1078172 (0.0006) [2023-12-27 13:07:33,439][20347] Updated weights for policy 0, policy_version 1079112 (0.0007) [2023-12-27 13:07:33,476][20381] Updated weights for policy 1, policy_version 1078182 (0.0007) [2023-12-27 13:07:33,524][20381] Updated weights for policy 1, policy_version 1078192 (0.0007) [2023-12-27 13:07:34,007][20347] Updated weights for policy 0, policy_version 1079122 (0.0005) [2023-12-27 13:07:34,065][20347] Updated weights for policy 0, policy_version 1079132 (0.0005) [2023-12-27 13:07:34,118][20347] Updated weights for policy 0, policy_version 1079142 (0.0005) [2023-12-27 13:07:34,303][20381] Updated weights for policy 1, policy_version 1078202 (0.0007) [2023-12-27 13:07:34,361][20381] Updated weights for policy 1, policy_version 1078212 (0.0008) [2023-12-27 13:07:34,417][20381] Updated weights for policy 1, policy_version 1078222 (0.0008) [2023-12-27 13:07:34,478][20381] Updated weights for policy 1, policy_version 1078232 (0.0008) [2023-12-27 13:07:34,731][20347] Updated weights for policy 0, policy_version 1079152 (0.0007) [2023-12-27 13:07:34,794][20347] Updated weights for policy 0, policy_version 1079162 (0.0008) [2023-12-27 13:07:34,853][20347] Updated weights for policy 0, policy_version 1079172 (0.0008) [2023-12-27 13:07:35,178][20381] Updated weights for policy 1, policy_version 1078242 (0.0008) [2023-12-27 13:07:35,231][20381] Updated weights for policy 1, policy_version 1078252 (0.0009) [2023-12-27 13:07:35,289][20381] Updated weights for policy 1, policy_version 1078262 (0.0006) [2023-12-27 13:07:35,516][20347] Updated weights for policy 0, policy_version 1079182 (0.0008) [2023-12-27 13:07:35,561][20347] Updated weights for policy 0, policy_version 1079192 (0.0009) [2023-12-27 13:07:35,608][20347] Updated weights for policy 0, policy_version 1079202 (0.0007) [2023-12-27 13:07:35,888][20381] Updated weights for policy 1, policy_version 1078272 (0.0007) [2023-12-27 13:07:35,941][20381] Updated weights for policy 1, policy_version 1078282 (0.0008) [2023-12-27 13:07:35,986][20381] Updated weights for policy 1, policy_version 1078292 (0.0008) [2023-12-27 13:07:36,353][20347] Updated weights for policy 0, policy_version 1079212 (0.0007) [2023-12-27 13:07:36,413][20347] Updated weights for policy 0, policy_version 1079222 (0.0008) [2023-12-27 13:07:36,479][20347] Updated weights for policy 0, policy_version 1079232 (0.0008) [2023-12-27 13:07:36,672][20381] Updated weights for policy 1, policy_version 1078302 (0.0008) [2023-12-27 13:07:36,727][20381] Updated weights for policy 1, policy_version 1078312 (0.0007) [2023-12-27 13:07:36,774][20381] Updated weights for policy 1, policy_version 1078322 (0.0007) [2023-12-27 13:07:37,090][20347] Updated weights for policy 0, policy_version 1079242 (0.0007) [2023-12-27 13:07:37,157][20347] Updated weights for policy 0, policy_version 1079252 (0.0005) [2023-12-27 13:07:37,223][20347] Updated weights for policy 0, policy_version 1079262 (0.0006) [2023-12-27 13:07:37,276][20347] Updated weights for policy 0, policy_version 1079272 (0.0005) [2023-12-27 13:07:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 552419328. Throughput: 0: 10328.4, 1: 10391.9. Samples: 552385268. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '251.020')] [2023-12-27 13:07:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001078328_276078592.pth... [2023-12-27 13:07:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001079272_276340736.pth... [2023-12-27 13:07:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001077112_275767296.pth [2023-12-27 13:07:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001078056_276029440.pth [2023-12-27 13:07:37,524][20381] Updated weights for policy 1, policy_version 1078332 (0.0008) [2023-12-27 13:07:37,578][20381] Updated weights for policy 1, policy_version 1078342 (0.0011) [2023-12-27 13:07:37,634][20381] Updated weights for policy 1, policy_version 1078352 (0.0011) [2023-12-27 13:07:37,907][20347] Updated weights for policy 0, policy_version 1079282 (0.0007) [2023-12-27 13:07:37,968][20347] Updated weights for policy 0, policy_version 1079292 (0.0007) [2023-12-27 13:07:38,026][20347] Updated weights for policy 0, policy_version 1079302 (0.0007) [2023-12-27 13:07:38,316][20381] Updated weights for policy 1, policy_version 1078362 (0.0010) [2023-12-27 13:07:38,374][20381] Updated weights for policy 1, policy_version 1078372 (0.0009) [2023-12-27 13:07:38,434][20381] Updated weights for policy 1, policy_version 1078382 (0.0008) [2023-12-27 13:07:38,488][20381] Updated weights for policy 1, policy_version 1078392 (0.0008) [2023-12-27 13:07:38,711][20347] Updated weights for policy 0, policy_version 1079312 (0.0008) [2023-12-27 13:07:38,770][20347] Updated weights for policy 0, policy_version 1079322 (0.0007) [2023-12-27 13:07:38,830][20347] Updated weights for policy 0, policy_version 1079332 (0.0007) [2023-12-27 13:07:39,177][20381] Updated weights for policy 1, policy_version 1078402 (0.0007) [2023-12-27 13:07:39,237][20381] Updated weights for policy 1, policy_version 1078412 (0.0008) [2023-12-27 13:07:39,293][20381] Updated weights for policy 1, policy_version 1078422 (0.0009) [2023-12-27 13:07:39,492][20347] Updated weights for policy 0, policy_version 1079342 (0.0008) [2023-12-27 13:07:39,550][20347] Updated weights for policy 0, policy_version 1079352 (0.0011) [2023-12-27 13:07:39,612][20347] Updated weights for policy 0, policy_version 1079362 (0.0007) [2023-12-27 13:07:40,046][20381] Updated weights for policy 1, policy_version 1078432 (0.0007) [2023-12-27 13:07:40,108][20381] Updated weights for policy 1, policy_version 1078442 (0.0008) [2023-12-27 13:07:40,173][20381] Updated weights for policy 1, policy_version 1078452 (0.0006) [2023-12-27 13:07:40,249][20347] Updated weights for policy 0, policy_version 1079372 (0.0009) [2023-12-27 13:07:40,315][20347] Updated weights for policy 0, policy_version 1079382 (0.0011) [2023-12-27 13:07:40,375][20347] Updated weights for policy 0, policy_version 1079392 (0.0011) [2023-12-27 13:07:40,716][20381] Updated weights for policy 1, policy_version 1078462 (0.0005) [2023-12-27 13:07:40,771][20381] Updated weights for policy 1, policy_version 1078472 (0.0007) [2023-12-27 13:07:40,821][20381] Updated weights for policy 1, policy_version 1078482 (0.0007) [2023-12-27 13:07:41,049][20347] Updated weights for policy 0, policy_version 1079402 (0.0010) [2023-12-27 13:07:41,116][20347] Updated weights for policy 0, policy_version 1079412 (0.0010) [2023-12-27 13:07:41,180][20347] Updated weights for policy 0, policy_version 1079422 (0.0010) [2023-12-27 13:07:41,233][20347] Updated weights for policy 0, policy_version 1079432 (0.0010) [2023-12-27 13:07:41,602][20381] Updated weights for policy 1, policy_version 1078492 (0.0007) [2023-12-27 13:07:41,668][20381] Updated weights for policy 1, policy_version 1078502 (0.0011) [2023-12-27 13:07:41,731][20381] Updated weights for policy 1, policy_version 1078512 (0.0011) [2023-12-27 13:07:42,000][20347] Updated weights for policy 0, policy_version 1079442 (0.0011) [2023-12-27 13:07:42,056][20347] Updated weights for policy 0, policy_version 1079452 (0.0010) [2023-12-27 13:07:42,114][20347] Updated weights for policy 0, policy_version 1079462 (0.0010) [2023-12-27 13:07:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 552517632. Throughput: 0: 10316.5, 1: 10406.8. Samples: 552508708. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '426.200')] [2023-12-27 13:07:42,419][20381] Updated weights for policy 1, policy_version 1078522 (0.0011) [2023-12-27 13:07:42,465][20381] Updated weights for policy 1, policy_version 1078532 (0.0010) [2023-12-27 13:07:42,516][20381] Updated weights for policy 1, policy_version 1078542 (0.0010) [2023-12-27 13:07:42,571][20381] Updated weights for policy 1, policy_version 1078552 (0.0006) [2023-12-27 13:07:42,857][20347] Updated weights for policy 0, policy_version 1079472 (0.0007) [2023-12-27 13:07:42,912][20347] Updated weights for policy 0, policy_version 1079482 (0.0006) [2023-12-27 13:07:42,964][20347] Updated weights for policy 0, policy_version 1079492 (0.0008) [2023-12-27 13:07:43,205][20381] Updated weights for policy 1, policy_version 1078562 (0.0007) [2023-12-27 13:07:43,250][20381] Updated weights for policy 1, policy_version 1078572 (0.0008) [2023-12-27 13:07:43,302][20381] Updated weights for policy 1, policy_version 1078582 (0.0009) [2023-12-27 13:07:43,600][20347] Updated weights for policy 0, policy_version 1079502 (0.0007) [2023-12-27 13:07:43,650][20347] Updated weights for policy 0, policy_version 1079512 (0.0006) [2023-12-27 13:07:43,701][20347] Updated weights for policy 0, policy_version 1079522 (0.0006) [2023-12-27 13:07:43,728][20291] KL-divergence is very high: 184.9347 [2023-12-27 13:07:44,029][20381] Updated weights for policy 1, policy_version 1078592 (0.0008) [2023-12-27 13:07:44,079][20381] Updated weights for policy 1, policy_version 1078602 (0.0007) [2023-12-27 13:07:44,127][20381] Updated weights for policy 1, policy_version 1078612 (0.0008) [2023-12-27 13:07:44,339][20347] Updated weights for policy 0, policy_version 1079532 (0.0007) [2023-12-27 13:07:44,401][20347] Updated weights for policy 0, policy_version 1079542 (0.0009) [2023-12-27 13:07:44,449][20347] Updated weights for policy 0, policy_version 1079552 (0.0010) [2023-12-27 13:07:44,897][20381] Updated weights for policy 1, policy_version 1078622 (0.0008) [2023-12-27 13:07:44,953][20381] Updated weights for policy 1, policy_version 1078632 (0.0007) [2023-12-27 13:07:45,005][20381] Updated weights for policy 1, policy_version 1078642 (0.0008) [2023-12-27 13:07:45,105][20347] Updated weights for policy 0, policy_version 1079562 (0.0008) [2023-12-27 13:07:45,160][20347] Updated weights for policy 0, policy_version 1079572 (0.0007) [2023-12-27 13:07:45,221][20347] Updated weights for policy 0, policy_version 1079582 (0.0009) [2023-12-27 13:07:45,270][20347] Updated weights for policy 0, policy_version 1079592 (0.0010) [2023-12-27 13:07:45,736][20381] Updated weights for policy 1, policy_version 1078652 (0.0007) [2023-12-27 13:07:45,795][20381] Updated weights for policy 1, policy_version 1078662 (0.0007) [2023-12-27 13:07:45,841][20381] Updated weights for policy 1, policy_version 1078672 (0.0006) [2023-12-27 13:07:45,869][20347] Updated weights for policy 0, policy_version 1079602 (0.0010) [2023-12-27 13:07:45,921][20347] Updated weights for policy 0, policy_version 1079612 (0.0010) [2023-12-27 13:07:45,969][20347] Updated weights for policy 0, policy_version 1079622 (0.0010) [2023-12-27 13:07:46,460][20381] Updated weights for policy 1, policy_version 1078682 (0.0006) [2023-12-27 13:07:46,523][20381] Updated weights for policy 1, policy_version 1078692 (0.0006) [2023-12-27 13:07:46,570][20347] Updated weights for policy 0, policy_version 1079632 (0.0009) [2023-12-27 13:07:46,581][20381] Updated weights for policy 1, policy_version 1078702 (0.0005) [2023-12-27 13:07:46,634][20347] Updated weights for policy 0, policy_version 1079642 (0.0006) [2023-12-27 13:07:46,643][20381] Updated weights for policy 1, policy_version 1078712 (0.0005) [2023-12-27 13:07:46,690][20347] Updated weights for policy 0, policy_version 1079652 (0.0008) [2023-12-27 13:07:47,183][20381] Updated weights for policy 1, policy_version 1078722 (0.0008) [2023-12-27 13:07:47,239][20381] Updated weights for policy 1, policy_version 1078732 (0.0007) [2023-12-27 13:07:47,303][20381] Updated weights for policy 1, policy_version 1078742 (0.0006) [2023-12-27 13:07:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 552624128. Throughput: 0: 10305.9, 1: 10410.1. Samples: 552633208. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:47,363][19170] Avg episode reward: [(0, '990.230'), (1, '84.380')] [2023-12-27 13:07:47,412][20347] Updated weights for policy 0, policy_version 1079662 (0.0007) [2023-12-27 13:07:47,472][20347] Updated weights for policy 0, policy_version 1079672 (0.0007) [2023-12-27 13:07:47,528][20347] Updated weights for policy 0, policy_version 1079682 (0.0007) [2023-12-27 13:07:47,975][20381] Updated weights for policy 1, policy_version 1078752 (0.0008) [2023-12-27 13:07:48,028][20381] Updated weights for policy 1, policy_version 1078762 (0.0008) [2023-12-27 13:07:48,078][20381] Updated weights for policy 1, policy_version 1078772 (0.0008) [2023-12-27 13:07:48,103][20347] Updated weights for policy 0, policy_version 1079692 (0.0007) [2023-12-27 13:07:48,154][20347] Updated weights for policy 0, policy_version 1079702 (0.0008) [2023-12-27 13:07:48,201][20347] Updated weights for policy 0, policy_version 1079712 (0.0007) [2023-12-27 13:07:48,811][20381] Updated weights for policy 1, policy_version 1078782 (0.0007) [2023-12-27 13:07:48,863][20381] Updated weights for policy 1, policy_version 1078792 (0.0008) [2023-12-27 13:07:48,900][20347] Updated weights for policy 0, policy_version 1079722 (0.0007) [2023-12-27 13:07:48,913][20381] Updated weights for policy 1, policy_version 1078802 (0.0007) [2023-12-27 13:07:48,946][20347] Updated weights for policy 0, policy_version 1079732 (0.0007) [2023-12-27 13:07:49,002][20347] Updated weights for policy 0, policy_version 1079742 (0.0008) [2023-12-27 13:07:49,067][20347] Updated weights for policy 0, policy_version 1079752 (0.0008) [2023-12-27 13:07:49,581][20381] Updated weights for policy 1, policy_version 1078812 (0.0007) [2023-12-27 13:07:49,648][20381] Updated weights for policy 1, policy_version 1078822 (0.0008) [2023-12-27 13:07:49,713][20381] Updated weights for policy 1, policy_version 1078832 (0.0008) [2023-12-27 13:07:49,786][20347] Updated weights for policy 0, policy_version 1079762 (0.0005) [2023-12-27 13:07:49,847][20347] Updated weights for policy 0, policy_version 1079772 (0.0007) [2023-12-27 13:07:49,913][20347] Updated weights for policy 0, policy_version 1079782 (0.0007) [2023-12-27 13:07:50,453][20381] Updated weights for policy 1, policy_version 1078842 (0.0007) [2023-12-27 13:07:50,500][20381] Updated weights for policy 1, policy_version 1078852 (0.0007) [2023-12-27 13:07:50,546][20381] Updated weights for policy 1, policy_version 1078862 (0.0007) [2023-12-27 13:07:50,559][20347] Updated weights for policy 0, policy_version 1079792 (0.0010) [2023-12-27 13:07:50,615][20381] Updated weights for policy 1, policy_version 1078872 (0.0007) [2023-12-27 13:07:50,619][20347] Updated weights for policy 0, policy_version 1079802 (0.0010) [2023-12-27 13:07:50,674][20347] Updated weights for policy 0, policy_version 1079812 (0.0010) [2023-12-27 13:07:51,296][20381] Updated weights for policy 1, policy_version 1078882 (0.0008) [2023-12-27 13:07:51,361][20381] Updated weights for policy 1, policy_version 1078892 (0.0007) [2023-12-27 13:07:51,383][20347] Updated weights for policy 0, policy_version 1079822 (0.0008) [2023-12-27 13:07:51,427][20381] Updated weights for policy 1, policy_version 1078902 (0.0007) [2023-12-27 13:07:51,433][20347] Updated weights for policy 0, policy_version 1079832 (0.0005) [2023-12-27 13:07:51,494][20347] Updated weights for policy 0, policy_version 1079842 (0.0007) [2023-12-27 13:07:52,111][20381] Updated weights for policy 1, policy_version 1078912 (0.0006) [2023-12-27 13:07:52,168][20381] Updated weights for policy 1, policy_version 1078922 (0.0007) [2023-12-27 13:07:52,202][20347] Updated weights for policy 0, policy_version 1079852 (0.0007) [2023-12-27 13:07:52,221][20381] Updated weights for policy 1, policy_version 1078932 (0.0006) [2023-12-27 13:07:52,265][20347] Updated weights for policy 0, policy_version 1079862 (0.0008) [2023-12-27 13:07:52,325][20347] Updated weights for policy 0, policy_version 1079872 (0.0008) [2023-12-27 13:07:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 552722432. Throughput: 0: 10372.1, 1: 10317.7. Samples: 552694012. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:52,363][19170] Avg episode reward: [(0, '960.890'), (1, '26.400')] [2023-12-27 13:07:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001078936_276234240.pth... [2023-12-27 13:07:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001077720_275922944.pth [2023-12-27 13:07:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001079880_276496384.pth... [2023-12-27 13:07:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001078632_276176896.pth [2023-12-27 13:07:52,939][20381] Updated weights for policy 1, policy_version 1078942 (0.0007) [2023-12-27 13:07:52,989][20381] Updated weights for policy 1, policy_version 1078952 (0.0007) [2023-12-27 13:07:53,017][20347] Updated weights for policy 0, policy_version 1079882 (0.0008) [2023-12-27 13:07:53,044][20381] Updated weights for policy 1, policy_version 1078962 (0.0007) [2023-12-27 13:07:53,080][20347] Updated weights for policy 0, policy_version 1079892 (0.0006) [2023-12-27 13:07:53,144][20347] Updated weights for policy 0, policy_version 1079902 (0.0006) [2023-12-27 13:07:53,201][20347] Updated weights for policy 0, policy_version 1079912 (0.0005) [2023-12-27 13:07:53,751][20381] Updated weights for policy 1, policy_version 1078972 (0.0007) [2023-12-27 13:07:53,808][20381] Updated weights for policy 1, policy_version 1078982 (0.0006) [2023-12-27 13:07:53,816][20347] Updated weights for policy 0, policy_version 1079922 (0.0007) [2023-12-27 13:07:53,857][20381] Updated weights for policy 1, policy_version 1078992 (0.0005) [2023-12-27 13:07:53,871][20347] Updated weights for policy 0, policy_version 1079932 (0.0007) [2023-12-27 13:07:53,932][20347] Updated weights for policy 0, policy_version 1079942 (0.0005) [2023-12-27 13:07:54,449][20381] Updated weights for policy 1, policy_version 1079002 (0.0006) [2023-12-27 13:07:54,506][20381] Updated weights for policy 1, policy_version 1079012 (0.0008) [2023-12-27 13:07:54,560][20381] Updated weights for policy 1, policy_version 1079022 (0.0007) [2023-12-27 13:07:54,586][20347] Updated weights for policy 0, policy_version 1079952 (0.0006) [2023-12-27 13:07:54,614][20381] Updated weights for policy 1, policy_version 1079032 (0.0008) [2023-12-27 13:07:54,638][20347] Updated weights for policy 0, policy_version 1079962 (0.0006) [2023-12-27 13:07:54,685][20347] Updated weights for policy 0, policy_version 1079972 (0.0005) [2023-12-27 13:07:55,280][20347] Updated weights for policy 0, policy_version 1079982 (0.0008) [2023-12-27 13:07:55,338][20347] Updated weights for policy 0, policy_version 1079992 (0.0007) [2023-12-27 13:07:55,350][20381] Updated weights for policy 1, policy_version 1079042 (0.0005) [2023-12-27 13:07:55,389][20347] Updated weights for policy 0, policy_version 1080002 (0.0006) [2023-12-27 13:07:55,407][20381] Updated weights for policy 1, policy_version 1079052 (0.0008) [2023-12-27 13:07:55,458][20381] Updated weights for policy 1, policy_version 1079062 (0.0007) [2023-12-27 13:07:56,016][20347] Updated weights for policy 0, policy_version 1080012 (0.0006) [2023-12-27 13:07:56,062][20347] Updated weights for policy 0, policy_version 1080022 (0.0007) [2023-12-27 13:07:56,110][20347] Updated weights for policy 0, policy_version 1080032 (0.0007) [2023-12-27 13:07:56,145][20381] Updated weights for policy 1, policy_version 1079072 (0.0007) [2023-12-27 13:07:56,204][20381] Updated weights for policy 1, policy_version 1079082 (0.0007) [2023-12-27 13:07:56,254][20381] Updated weights for policy 1, policy_version 1079092 (0.0008) [2023-12-27 13:07:56,683][20347] Updated weights for policy 0, policy_version 1080042 (0.0006) [2023-12-27 13:07:56,728][20347] Updated weights for policy 0, policy_version 1080052 (0.0005) [2023-12-27 13:07:56,779][20347] Updated weights for policy 0, policy_version 1080062 (0.0005) [2023-12-27 13:07:56,832][20347] Updated weights for policy 0, policy_version 1080072 (0.0006) [2023-12-27 13:07:57,022][20381] Updated weights for policy 1, policy_version 1079102 (0.0008) [2023-12-27 13:07:57,070][20381] Updated weights for policy 1, policy_version 1079112 (0.0006) [2023-12-27 13:07:57,123][20381] Updated weights for policy 1, policy_version 1079122 (0.0005) [2023-12-27 13:07:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 552828928. Throughput: 0: 10402.3, 1: 10306.9. Samples: 552817340. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:07:57,363][20347] Updated weights for policy 0, policy_version 1080082 (0.0008) [2023-12-27 13:07:57,363][19170] Avg episode reward: [(0, '970.600'), (1, '27.590')] [2023-12-27 13:07:57,411][20347] Updated weights for policy 0, policy_version 1080092 (0.0007) [2023-12-27 13:07:57,465][20347] Updated weights for policy 0, policy_version 1080102 (0.0006) [2023-12-27 13:07:57,725][20381] Updated weights for policy 1, policy_version 1079132 (0.0005) [2023-12-27 13:07:57,783][20381] Updated weights for policy 1, policy_version 1079142 (0.0007) [2023-12-27 13:07:57,842][20381] Updated weights for policy 1, policy_version 1079152 (0.0008) [2023-12-27 13:07:58,137][20347] Updated weights for policy 0, policy_version 1080112 (0.0005) [2023-12-27 13:07:58,198][20347] Updated weights for policy 0, policy_version 1080122 (0.0008) [2023-12-27 13:07:58,257][20347] Updated weights for policy 0, policy_version 1080132 (0.0009) [2023-12-27 13:07:58,512][20381] Updated weights for policy 1, policy_version 1079162 (0.0008) [2023-12-27 13:07:58,577][20381] Updated weights for policy 1, policy_version 1079172 (0.0009) [2023-12-27 13:07:58,641][20381] Updated weights for policy 1, policy_version 1079182 (0.0008) [2023-12-27 13:07:58,705][20381] Updated weights for policy 1, policy_version 1079192 (0.0007) [2023-12-27 13:07:59,099][20347] Updated weights for policy 0, policy_version 1080142 (0.0007) [2023-12-27 13:07:59,162][20347] Updated weights for policy 0, policy_version 1080152 (0.0008) [2023-12-27 13:07:59,223][20347] Updated weights for policy 0, policy_version 1080162 (0.0008) [2023-12-27 13:07:59,510][20381] Updated weights for policy 1, policy_version 1079202 (0.0007) [2023-12-27 13:07:59,566][20381] Updated weights for policy 1, policy_version 1079212 (0.0008) [2023-12-27 13:07:59,619][20381] Updated weights for policy 1, policy_version 1079222 (0.0009) [2023-12-27 13:07:59,954][20347] Updated weights for policy 0, policy_version 1080172 (0.0008) [2023-12-27 13:08:00,006][20347] Updated weights for policy 0, policy_version 1080182 (0.0008) [2023-12-27 13:08:00,069][20347] Updated weights for policy 0, policy_version 1080192 (0.0007) [2023-12-27 13:08:00,268][20381] Updated weights for policy 1, policy_version 1079232 (0.0008) [2023-12-27 13:08:00,320][20381] Updated weights for policy 1, policy_version 1079242 (0.0009) [2023-12-27 13:08:00,382][20381] Updated weights for policy 1, policy_version 1079252 (0.0007) [2023-12-27 13:08:00,690][20347] Updated weights for policy 0, policy_version 1080202 (0.0009) [2023-12-27 13:08:00,733][20347] Updated weights for policy 0, policy_version 1080212 (0.0005) [2023-12-27 13:08:00,780][20347] Updated weights for policy 0, policy_version 1080222 (0.0007) [2023-12-27 13:08:00,821][20347] Updated weights for policy 0, policy_version 1080232 (0.0007) [2023-12-27 13:08:01,129][20381] Updated weights for policy 1, policy_version 1079262 (0.0008) [2023-12-27 13:08:01,197][20381] Updated weights for policy 1, policy_version 1079272 (0.0008) [2023-12-27 13:08:01,256][20381] Updated weights for policy 1, policy_version 1079282 (0.0008) [2023-12-27 13:08:01,515][20347] Updated weights for policy 0, policy_version 1080242 (0.0007) [2023-12-27 13:08:01,575][20347] Updated weights for policy 0, policy_version 1080252 (0.0008) [2023-12-27 13:08:01,639][20347] Updated weights for policy 0, policy_version 1080262 (0.0008) [2023-12-27 13:08:01,944][20381] Updated weights for policy 1, policy_version 1079292 (0.0008) [2023-12-27 13:08:01,994][20381] Updated weights for policy 1, policy_version 1079302 (0.0007) [2023-12-27 13:08:02,049][20381] Updated weights for policy 1, policy_version 1079312 (0.0007) [2023-12-27 13:08:02,333][20347] Updated weights for policy 0, policy_version 1080272 (0.0008) [2023-12-27 13:08:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 552927232. Throughput: 0: 10473.7, 1: 10187.1. Samples: 552937884. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:02,363][19170] Avg episode reward: [(0, '980.480'), (1, '29.390')] [2023-12-27 13:08:02,391][20347] Updated weights for policy 0, policy_version 1080282 (0.0007) [2023-12-27 13:08:02,451][20347] Updated weights for policy 0, policy_version 1080292 (0.0006) [2023-12-27 13:08:02,809][20381] Updated weights for policy 1, policy_version 1079322 (0.0009) [2023-12-27 13:08:02,861][20381] Updated weights for policy 1, policy_version 1079332 (0.0008) [2023-12-27 13:08:02,912][20381] Updated weights for policy 1, policy_version 1079342 (0.0007) [2023-12-27 13:08:02,966][20381] Updated weights for policy 1, policy_version 1079352 (0.0006) [2023-12-27 13:08:03,093][20347] Updated weights for policy 0, policy_version 1080302 (0.0008) [2023-12-27 13:08:03,143][20347] Updated weights for policy 0, policy_version 1080312 (0.0007) [2023-12-27 13:08:03,188][20347] Updated weights for policy 0, policy_version 1080322 (0.0008) [2023-12-27 13:08:03,590][20381] Updated weights for policy 1, policy_version 1079362 (0.0007) [2023-12-27 13:08:03,644][20381] Updated weights for policy 1, policy_version 1079372 (0.0007) [2023-12-27 13:08:03,691][20381] Updated weights for policy 1, policy_version 1079382 (0.0007) [2023-12-27 13:08:03,920][20347] Updated weights for policy 0, policy_version 1080332 (0.0007) [2023-12-27 13:08:03,979][20347] Updated weights for policy 0, policy_version 1080342 (0.0006) [2023-12-27 13:08:04,041][20347] Updated weights for policy 0, policy_version 1080352 (0.0008) [2023-12-27 13:08:04,363][20381] Updated weights for policy 1, policy_version 1079392 (0.0007) [2023-12-27 13:08:04,425][20381] Updated weights for policy 1, policy_version 1079402 (0.0007) [2023-12-27 13:08:04,478][20381] Updated weights for policy 1, policy_version 1079412 (0.0006) [2023-12-27 13:08:04,683][20347] Updated weights for policy 0, policy_version 1080362 (0.0008) [2023-12-27 13:08:04,747][20347] Updated weights for policy 0, policy_version 1080372 (0.0008) [2023-12-27 13:08:04,821][20347] Updated weights for policy 0, policy_version 1080382 (0.0008) [2023-12-27 13:08:04,881][20347] Updated weights for policy 0, policy_version 1080392 (0.0008) [2023-12-27 13:08:05,125][20381] Updated weights for policy 1, policy_version 1079422 (0.0009) [2023-12-27 13:08:05,183][20381] Updated weights for policy 1, policy_version 1079432 (0.0010) [2023-12-27 13:08:05,239][20381] Updated weights for policy 1, policy_version 1079442 (0.0010) [2023-12-27 13:08:05,517][20347] Updated weights for policy 0, policy_version 1080402 (0.0007) [2023-12-27 13:08:05,566][20347] Updated weights for policy 0, policy_version 1080412 (0.0008) [2023-12-27 13:08:05,584][20291] KL-divergence is very high: 503.3920 [2023-12-27 13:08:05,613][20347] Updated weights for policy 0, policy_version 1080422 (0.0007) [2023-12-27 13:08:05,825][20381] Updated weights for policy 1, policy_version 1079452 (0.0010) [2023-12-27 13:08:05,881][20381] Updated weights for policy 1, policy_version 1079462 (0.0011) [2023-12-27 13:08:05,941][20381] Updated weights for policy 1, policy_version 1079472 (0.0011) [2023-12-27 13:08:06,329][20347] Updated weights for policy 0, policy_version 1080432 (0.0008) [2023-12-27 13:08:06,397][20347] Updated weights for policy 0, policy_version 1080442 (0.0008) [2023-12-27 13:08:06,460][20347] Updated weights for policy 0, policy_version 1080452 (0.0007) [2023-12-27 13:08:06,557][20381] Updated weights for policy 1, policy_version 1079482 (0.0011) [2023-12-27 13:08:06,616][20381] Updated weights for policy 1, policy_version 1079492 (0.0009) [2023-12-27 13:08:06,684][20381] Updated weights for policy 1, policy_version 1079502 (0.0008) [2023-12-27 13:08:06,745][20381] Updated weights for policy 1, policy_version 1079512 (0.0011) [2023-12-27 13:08:07,098][20347] Updated weights for policy 0, policy_version 1080462 (0.0008) [2023-12-27 13:08:07,156][20347] Updated weights for policy 0, policy_version 1080472 (0.0008) [2023-12-27 13:08:07,210][20347] Updated weights for policy 0, policy_version 1080482 (0.0007) [2023-12-27 13:08:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 553033728. Throughput: 0: 10450.6, 1: 10214.3. Samples: 553000380. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:07,363][19170] Avg episode reward: [(0, '980.640'), (1, '28.020')] [2023-12-27 13:08:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001080488_276652032.pth... [2023-12-27 13:08:07,368][20381] Updated weights for policy 1, policy_version 1079522 (0.0010) [2023-12-27 13:08:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001079272_276340736.pth [2023-12-27 13:08:07,420][20381] Updated weights for policy 1, policy_version 1079532 (0.0010) [2023-12-27 13:08:07,473][20381] Updated weights for policy 1, policy_version 1079542 (0.0010) [2023-12-27 13:08:07,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001079544_276389888.pth... [2023-12-27 13:08:07,486][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001078328_276078592.pth [2023-12-27 13:08:07,910][20347] Updated weights for policy 0, policy_version 1080492 (0.0008) [2023-12-27 13:08:07,974][20347] Updated weights for policy 0, policy_version 1080502 (0.0008) [2023-12-27 13:08:08,028][20347] Updated weights for policy 0, policy_version 1080512 (0.0008) [2023-12-27 13:08:08,129][20381] Updated weights for policy 1, policy_version 1079552 (0.0011) [2023-12-27 13:08:08,186][20381] Updated weights for policy 1, policy_version 1079562 (0.0011) [2023-12-27 13:08:08,240][20381] Updated weights for policy 1, policy_version 1079572 (0.0010) [2023-12-27 13:08:08,847][20347] Updated weights for policy 0, policy_version 1080522 (0.0007) [2023-12-27 13:08:08,886][20381] Updated weights for policy 1, policy_version 1079582 (0.0008) [2023-12-27 13:08:08,900][20347] Updated weights for policy 0, policy_version 1080532 (0.0008) [2023-12-27 13:08:08,935][20381] Updated weights for policy 1, policy_version 1079592 (0.0009) [2023-12-27 13:08:08,956][20347] Updated weights for policy 0, policy_version 1080542 (0.0005) [2023-12-27 13:08:08,988][20381] Updated weights for policy 1, policy_version 1079602 (0.0011) [2023-12-27 13:08:09,013][20347] Updated weights for policy 0, policy_version 1080552 (0.0005) [2023-12-27 13:08:09,732][20347] Updated weights for policy 0, policy_version 1080562 (0.0007) [2023-12-27 13:08:09,789][20347] Updated weights for policy 0, policy_version 1080572 (0.0008) [2023-12-27 13:08:09,798][20381] Updated weights for policy 1, policy_version 1079612 (0.0009) [2023-12-27 13:08:09,856][20347] Updated weights for policy 0, policy_version 1080582 (0.0007) [2023-12-27 13:08:09,862][20381] Updated weights for policy 1, policy_version 1079622 (0.0008) [2023-12-27 13:08:09,919][20381] Updated weights for policy 1, policy_version 1079632 (0.0008) [2023-12-27 13:08:10,539][20381] Updated weights for policy 1, policy_version 1079642 (0.0007) [2023-12-27 13:08:10,598][20381] Updated weights for policy 1, policy_version 1079652 (0.0007) [2023-12-27 13:08:10,618][20347] Updated weights for policy 0, policy_version 1080592 (0.0008) [2023-12-27 13:08:10,654][20381] Updated weights for policy 1, policy_version 1079662 (0.0006) [2023-12-27 13:08:10,671][20347] Updated weights for policy 0, policy_version 1080602 (0.0006) [2023-12-27 13:08:10,710][20381] Updated weights for policy 1, policy_version 1079672 (0.0006) [2023-12-27 13:08:10,727][20347] Updated weights for policy 0, policy_version 1080612 (0.0006) [2023-12-27 13:08:11,394][20381] Updated weights for policy 1, policy_version 1079682 (0.0008) [2023-12-27 13:08:11,450][20381] Updated weights for policy 1, policy_version 1079692 (0.0007) [2023-12-27 13:08:11,471][20347] Updated weights for policy 0, policy_version 1080622 (0.0007) [2023-12-27 13:08:11,507][20381] Updated weights for policy 1, policy_version 1079702 (0.0007) [2023-12-27 13:08:11,529][20347] Updated weights for policy 0, policy_version 1080632 (0.0006) [2023-12-27 13:08:11,580][20347] Updated weights for policy 0, policy_version 1080642 (0.0008) [2023-12-27 13:08:12,252][20381] Updated weights for policy 1, policy_version 1079712 (0.0007) [2023-12-27 13:08:12,320][20381] Updated weights for policy 1, policy_version 1079722 (0.0008) [2023-12-27 13:08:12,347][20347] Updated weights for policy 0, policy_version 1080652 (0.0007) [2023-12-27 13:08:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 553123840. Throughput: 0: 10393.9, 1: 10202.1. Samples: 553122856. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:12,363][19170] Avg episode reward: [(0, '931.710'), (1, '27.940')] [2023-12-27 13:08:12,385][20381] Updated weights for policy 1, policy_version 1079732 (0.0008) [2023-12-27 13:08:12,416][20347] Updated weights for policy 0, policy_version 1080662 (0.0008) [2023-12-27 13:08:12,473][20347] Updated weights for policy 0, policy_version 1080672 (0.0007) [2023-12-27 13:08:13,060][20381] Updated weights for policy 1, policy_version 1079742 (0.0008) [2023-12-27 13:08:13,117][20381] Updated weights for policy 1, policy_version 1079752 (0.0007) [2023-12-27 13:08:13,177][20347] Updated weights for policy 0, policy_version 1080682 (0.0007) [2023-12-27 13:08:13,178][20381] Updated weights for policy 1, policy_version 1079762 (0.0007) [2023-12-27 13:08:13,241][20347] Updated weights for policy 0, policy_version 1080692 (0.0006) [2023-12-27 13:08:13,306][20347] Updated weights for policy 0, policy_version 1080702 (0.0006) [2023-12-27 13:08:13,370][20347] Updated weights for policy 0, policy_version 1080712 (0.0008) [2023-12-27 13:08:13,786][20381] Updated weights for policy 1, policy_version 1079772 (0.0006) [2023-12-27 13:08:13,839][20381] Updated weights for policy 1, policy_version 1079782 (0.0007) [2023-12-27 13:08:13,894][20381] Updated weights for policy 1, policy_version 1079792 (0.0007) [2023-12-27 13:08:13,977][20347] Updated weights for policy 0, policy_version 1080722 (0.0008) [2023-12-27 13:08:14,040][20347] Updated weights for policy 0, policy_version 1080732 (0.0007) [2023-12-27 13:08:14,088][20347] Updated weights for policy 0, policy_version 1080742 (0.0005) [2023-12-27 13:08:14,524][20381] Updated weights for policy 1, policy_version 1079802 (0.0006) [2023-12-27 13:08:14,583][20381] Updated weights for policy 1, policy_version 1079812 (0.0006) [2023-12-27 13:08:14,643][20381] Updated weights for policy 1, policy_version 1079822 (0.0009) [2023-12-27 13:08:14,647][20347] Updated weights for policy 0, policy_version 1080752 (0.0009) [2023-12-27 13:08:14,695][20381] Updated weights for policy 1, policy_version 1079832 (0.0008) [2023-12-27 13:08:14,706][20347] Updated weights for policy 0, policy_version 1080762 (0.0011) [2023-12-27 13:08:14,768][20347] Updated weights for policy 0, policy_version 1080772 (0.0010) [2023-12-27 13:08:15,365][20381] Updated weights for policy 1, policy_version 1079842 (0.0011) [2023-12-27 13:08:15,420][20381] Updated weights for policy 1, policy_version 1079852 (0.0010) [2023-12-27 13:08:15,438][20347] Updated weights for policy 0, policy_version 1080782 (0.0010) [2023-12-27 13:08:15,476][20381] Updated weights for policy 1, policy_version 1079862 (0.0010) [2023-12-27 13:08:15,489][20347] Updated weights for policy 0, policy_version 1080792 (0.0005) [2023-12-27 13:08:15,545][20347] Updated weights for policy 0, policy_version 1080802 (0.0005) [2023-12-27 13:08:16,112][20347] Updated weights for policy 0, policy_version 1080812 (0.0005) [2023-12-27 13:08:16,157][20381] Updated weights for policy 1, policy_version 1079872 (0.0010) [2023-12-27 13:08:16,170][20347] Updated weights for policy 0, policy_version 1080822 (0.0005) [2023-12-27 13:08:16,210][20381] Updated weights for policy 1, policy_version 1079882 (0.0010) [2023-12-27 13:08:16,227][20347] Updated weights for policy 0, policy_version 1080832 (0.0006) [2023-12-27 13:08:16,265][20381] Updated weights for policy 1, policy_version 1079892 (0.0010) [2023-12-27 13:08:16,865][20347] Updated weights for policy 0, policy_version 1080842 (0.0006) [2023-12-27 13:08:16,912][20347] Updated weights for policy 0, policy_version 1080852 (0.0009) [2023-12-27 13:08:16,951][20381] Updated weights for policy 1, policy_version 1079902 (0.0007) [2023-12-27 13:08:16,967][20347] Updated weights for policy 0, policy_version 1080862 (0.0010) [2023-12-27 13:08:17,007][20381] Updated weights for policy 1, policy_version 1079912 (0.0005) [2023-12-27 13:08:17,025][20347] Updated weights for policy 0, policy_version 1080872 (0.0010) [2023-12-27 13:08:17,069][20381] Updated weights for policy 1, policy_version 1079922 (0.0006) [2023-12-27 13:08:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 553238528. Throughput: 0: 10336.8, 1: 10189.6. Samples: 553247008. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:17,364][19170] Avg episode reward: [(0, '902.250'), (1, '29.330')] [2023-12-27 13:08:17,624][20347] Updated weights for policy 0, policy_version 1080882 (0.0010) [2023-12-27 13:08:17,669][20347] Updated weights for policy 0, policy_version 1080892 (0.0010) [2023-12-27 13:08:17,706][20381] Updated weights for policy 1, policy_version 1079932 (0.0008) [2023-12-27 13:08:17,717][20347] Updated weights for policy 0, policy_version 1080902 (0.0010) [2023-12-27 13:08:17,759][20381] Updated weights for policy 1, policy_version 1079942 (0.0010) [2023-12-27 13:08:17,813][20381] Updated weights for policy 1, policy_version 1079952 (0.0010) [2023-12-27 13:08:18,389][20347] Updated weights for policy 0, policy_version 1080912 (0.0008) [2023-12-27 13:08:18,453][20347] Updated weights for policy 0, policy_version 1080922 (0.0008) [2023-12-27 13:08:18,513][20347] Updated weights for policy 0, policy_version 1080932 (0.0008) [2023-12-27 13:08:18,516][20381] Updated weights for policy 1, policy_version 1079962 (0.0010) [2023-12-27 13:08:18,569][20381] Updated weights for policy 1, policy_version 1079972 (0.0007) [2023-12-27 13:08:18,623][20381] Updated weights for policy 1, policy_version 1079982 (0.0007) [2023-12-27 13:08:18,673][20381] Updated weights for policy 1, policy_version 1079992 (0.0007) [2023-12-27 13:08:19,223][20347] Updated weights for policy 0, policy_version 1080942 (0.0009) [2023-12-27 13:08:19,282][20347] Updated weights for policy 0, policy_version 1080952 (0.0008) [2023-12-27 13:08:19,334][20381] Updated weights for policy 1, policy_version 1080002 (0.0007) [2023-12-27 13:08:19,347][20347] Updated weights for policy 0, policy_version 1080962 (0.0010) [2023-12-27 13:08:19,397][20381] Updated weights for policy 1, policy_version 1080012 (0.0008) [2023-12-27 13:08:19,462][20381] Updated weights for policy 1, policy_version 1080022 (0.0007) [2023-12-27 13:08:20,047][20347] Updated weights for policy 0, policy_version 1080972 (0.0009) [2023-12-27 13:08:20,102][20347] Updated weights for policy 0, policy_version 1080982 (0.0006) [2023-12-27 13:08:20,116][20291] KL-divergence is very high: 146.5987 [2023-12-27 13:08:20,122][20291] KL-divergence is very high: 136.1548 [2023-12-27 13:08:20,125][20381] Updated weights for policy 1, policy_version 1080032 (0.0007) [2023-12-27 13:08:20,140][20291] KL-divergence is very high: 157.0116 [2023-12-27 13:08:20,164][20347] Updated weights for policy 0, policy_version 1080992 (0.0008) [2023-12-27 13:08:20,187][20381] Updated weights for policy 1, policy_version 1080042 (0.0008) [2023-12-27 13:08:20,238][20381] Updated weights for policy 1, policy_version 1080052 (0.0008) [2023-12-27 13:08:20,885][20347] Updated weights for policy 0, policy_version 1081002 (0.0007) [2023-12-27 13:08:20,903][20381] Updated weights for policy 1, policy_version 1080062 (0.0008) [2023-12-27 13:08:20,938][20347] Updated weights for policy 0, policy_version 1081012 (0.0010) [2023-12-27 13:08:20,963][20381] Updated weights for policy 1, policy_version 1080072 (0.0006) [2023-12-27 13:08:20,995][20347] Updated weights for policy 0, policy_version 1081022 (0.0009) [2023-12-27 13:08:21,023][20381] Updated weights for policy 1, policy_version 1080082 (0.0008) [2023-12-27 13:08:21,055][20347] Updated weights for policy 0, policy_version 1081032 (0.0007) [2023-12-27 13:08:21,794][20381] Updated weights for policy 1, policy_version 1080092 (0.0006) [2023-12-27 13:08:21,854][20347] Updated weights for policy 0, policy_version 1081042 (0.0008) [2023-12-27 13:08:21,859][20381] Updated weights for policy 1, policy_version 1080102 (0.0006) [2023-12-27 13:08:21,866][20291] KL-divergence is very high: 208.4767 [2023-12-27 13:08:21,913][20381] Updated weights for policy 1, policy_version 1080112 (0.0008) [2023-12-27 13:08:21,914][20347] Updated weights for policy 0, policy_version 1081052 (0.0008) [2023-12-27 13:08:21,916][20291] KL-divergence is very high: 147.5068 [2023-12-27 13:08:21,981][20347] Updated weights for policy 0, policy_version 1081062 (0.0008) [2023-12-27 13:08:22,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 553336832. Throughput: 0: 10316.2, 1: 10201.4. Samples: 553308564. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:22,364][19170] Avg episode reward: [(0, '902.320'), (1, '43.470')] [2023-12-27 13:08:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001080120_276537344.pth... [2023-12-27 13:08:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001081064_276799488.pth... [2023-12-27 13:08:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001078936_276234240.pth [2023-12-27 13:08:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001079880_276496384.pth [2023-12-27 13:08:22,638][20381] Updated weights for policy 1, policy_version 1080122 (0.0007) [2023-12-27 13:08:22,703][20381] Updated weights for policy 1, policy_version 1080132 (0.0008) [2023-12-27 13:08:22,727][20347] Updated weights for policy 0, policy_version 1081072 (0.0007) [2023-12-27 13:08:22,765][20381] Updated weights for policy 1, policy_version 1080142 (0.0007) [2023-12-27 13:08:22,786][20347] Updated weights for policy 0, policy_version 1081082 (0.0009) [2023-12-27 13:08:22,822][20381] Updated weights for policy 1, policy_version 1080152 (0.0006) [2023-12-27 13:08:22,845][20347] Updated weights for policy 0, policy_version 1081092 (0.0010) [2023-12-27 13:08:23,487][20381] Updated weights for policy 1, policy_version 1080162 (0.0007) [2023-12-27 13:08:23,535][20347] Updated weights for policy 0, policy_version 1081102 (0.0008) [2023-12-27 13:08:23,543][20381] Updated weights for policy 1, policy_version 1080172 (0.0007) [2023-12-27 13:08:23,553][20291] KL-divergence is very high: 1546.0785 [2023-12-27 13:08:23,559][20291] KL-divergence is very high: 1211.0304 [2023-12-27 13:08:23,594][20347] Updated weights for policy 0, policy_version 1081112 (0.0009) [2023-12-27 13:08:23,603][20381] Updated weights for policy 1, policy_version 1080182 (0.0006) [2023-12-27 13:08:23,646][20347] Updated weights for policy 0, policy_version 1081122 (0.0010) [2023-12-27 13:08:24,264][20381] Updated weights for policy 1, policy_version 1080192 (0.0009) [2023-12-27 13:08:24,320][20381] Updated weights for policy 1, policy_version 1080202 (0.0011) [2023-12-27 13:08:24,325][20347] Updated weights for policy 0, policy_version 1081132 (0.0010) [2023-12-27 13:08:24,361][20291] KL-divergence is very high: 100.4613 [2023-12-27 13:08:24,366][20381] Updated weights for policy 1, policy_version 1080212 (0.0010) [2023-12-27 13:08:24,376][20347] Updated weights for policy 0, policy_version 1081142 (0.0007) [2023-12-27 13:08:24,423][20347] Updated weights for policy 0, policy_version 1081152 (0.0007) [2023-12-27 13:08:25,016][20347] Updated weights for policy 0, policy_version 1081162 (0.0008) [2023-12-27 13:08:25,059][20381] Updated weights for policy 1, policy_version 1080222 (0.0010) [2023-12-27 13:08:25,061][20347] Updated weights for policy 0, policy_version 1081172 (0.0010) [2023-12-27 13:08:25,107][20347] Updated weights for policy 0, policy_version 1081182 (0.0010) [2023-12-27 13:08:25,108][20381] Updated weights for policy 1, policy_version 1080232 (0.0010) [2023-12-27 13:08:25,121][20291] KL-divergence is very high: 203.5214 [2023-12-27 13:08:25,157][20381] Updated weights for policy 1, policy_version 1080242 (0.0010) [2023-12-27 13:08:25,161][20347] Updated weights for policy 0, policy_version 1081192 (0.0010) [2023-12-27 13:08:25,745][20347] Updated weights for policy 0, policy_version 1081202 (0.0006) [2023-12-27 13:08:25,777][20381] Updated weights for policy 1, policy_version 1080252 (0.0010) [2023-12-27 13:08:25,790][20291] KL-divergence is very high: 103.5206 [2023-12-27 13:08:25,804][20347] Updated weights for policy 0, policy_version 1081212 (0.0006) [2023-12-27 13:08:25,828][20381] Updated weights for policy 1, policy_version 1080262 (0.0005) [2023-12-27 13:08:25,855][20347] Updated weights for policy 0, policy_version 1081222 (0.0008) [2023-12-27 13:08:25,871][20381] Updated weights for policy 1, policy_version 1080272 (0.0005) [2023-12-27 13:08:26,467][20347] Updated weights for policy 0, policy_version 1081232 (0.0006) [2023-12-27 13:08:26,518][20347] Updated weights for policy 0, policy_version 1081242 (0.0005) [2023-12-27 13:08:26,567][20347] Updated weights for policy 0, policy_version 1081252 (0.0005) [2023-12-27 13:08:26,576][20381] Updated weights for policy 1, policy_version 1080282 (0.0005) [2023-12-27 13:08:26,633][20381] Updated weights for policy 1, policy_version 1080292 (0.0006) [2023-12-27 13:08:26,677][20381] Updated weights for policy 1, policy_version 1080302 (0.0008) [2023-12-27 13:08:26,728][20381] Updated weights for policy 1, policy_version 1080312 (0.0009) [2023-12-27 13:08:27,130][20347] Updated weights for policy 0, policy_version 1081262 (0.0006) [2023-12-27 13:08:27,185][20347] Updated weights for policy 0, policy_version 1081272 (0.0008) [2023-12-27 13:08:27,237][20347] Updated weights for policy 0, policy_version 1081282 (0.0008) [2023-12-27 13:08:27,334][20381] Updated weights for policy 1, policy_version 1080322 (0.0010) [2023-12-27 13:08:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 553443328. Throughput: 0: 10271.2, 1: 10195.7. Samples: 553429720. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:27,363][19170] Avg episode reward: [(0, '863.700'), (1, '60.420')] [2023-12-27 13:08:27,392][20381] Updated weights for policy 1, policy_version 1080332 (0.0008) [2023-12-27 13:08:27,451][20381] Updated weights for policy 1, policy_version 1080342 (0.0005) [2023-12-27 13:08:27,875][20347] Updated weights for policy 0, policy_version 1081292 (0.0007) [2023-12-27 13:08:27,934][20347] Updated weights for policy 0, policy_version 1081302 (0.0006) [2023-12-27 13:08:27,988][20347] Updated weights for policy 0, policy_version 1081312 (0.0007) [2023-12-27 13:08:28,104][20381] Updated weights for policy 1, policy_version 1080352 (0.0005) [2023-12-27 13:08:28,153][20381] Updated weights for policy 1, policy_version 1080362 (0.0005) [2023-12-27 13:08:28,204][20381] Updated weights for policy 1, policy_version 1080372 (0.0007) [2023-12-27 13:08:28,620][20347] Updated weights for policy 0, policy_version 1081322 (0.0007) [2023-12-27 13:08:28,682][20347] Updated weights for policy 0, policy_version 1081332 (0.0007) [2023-12-27 13:08:28,742][20347] Updated weights for policy 0, policy_version 1081342 (0.0008) [2023-12-27 13:08:28,800][20347] Updated weights for policy 0, policy_version 1081352 (0.0007) [2023-12-27 13:08:28,868][20381] Updated weights for policy 1, policy_version 1080382 (0.0010) [2023-12-27 13:08:28,924][20381] Updated weights for policy 1, policy_version 1080392 (0.0010) [2023-12-27 13:08:28,975][20381] Updated weights for policy 1, policy_version 1080402 (0.0005) [2023-12-27 13:08:29,517][20347] Updated weights for policy 0, policy_version 1081362 (0.0010) [2023-12-27 13:08:29,569][20347] Updated weights for policy 0, policy_version 1081372 (0.0010) [2023-12-27 13:08:29,627][20347] Updated weights for policy 0, policy_version 1081382 (0.0011) [2023-12-27 13:08:29,709][20381] Updated weights for policy 1, policy_version 1080412 (0.0007) [2023-12-27 13:08:29,769][20381] Updated weights for policy 1, policy_version 1080422 (0.0007) [2023-12-27 13:08:29,833][20381] Updated weights for policy 1, policy_version 1080432 (0.0006) [2023-12-27 13:08:30,349][20347] Updated weights for policy 0, policy_version 1081392 (0.0011) [2023-12-27 13:08:30,408][20347] Updated weights for policy 0, policy_version 1081402 (0.0010) [2023-12-27 13:08:30,463][20347] Updated weights for policy 0, policy_version 1081412 (0.0010) [2023-12-27 13:08:30,546][20381] Updated weights for policy 1, policy_version 1080442 (0.0009) [2023-12-27 13:08:30,597][20381] Updated weights for policy 1, policy_version 1080452 (0.0008) [2023-12-27 13:08:30,651][20381] Updated weights for policy 1, policy_version 1080462 (0.0007) [2023-12-27 13:08:30,695][20381] Updated weights for policy 1, policy_version 1080472 (0.0008) [2023-12-27 13:08:31,026][20347] Updated weights for policy 0, policy_version 1081422 (0.0008) [2023-12-27 13:08:31,090][20347] Updated weights for policy 0, policy_version 1081432 (0.0008) [2023-12-27 13:08:31,158][20347] Updated weights for policy 0, policy_version 1081442 (0.0009) [2023-12-27 13:08:31,414][20381] Updated weights for policy 1, policy_version 1080482 (0.0008) [2023-12-27 13:08:31,475][20381] Updated weights for policy 1, policy_version 1080492 (0.0008) [2023-12-27 13:08:31,531][20381] Updated weights for policy 1, policy_version 1080502 (0.0009) [2023-12-27 13:08:31,804][20347] Updated weights for policy 0, policy_version 1081452 (0.0006) [2023-12-27 13:08:31,872][20347] Updated weights for policy 0, policy_version 1081462 (0.0008) [2023-12-27 13:08:31,931][20347] Updated weights for policy 0, policy_version 1081472 (0.0007) [2023-12-27 13:08:32,274][20381] Updated weights for policy 1, policy_version 1080512 (0.0007) [2023-12-27 13:08:32,336][20381] Updated weights for policy 1, policy_version 1080522 (0.0008) [2023-12-27 13:08:32,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 553541632. Throughput: 0: 10324.7, 1: 10177.1. Samples: 553555792. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:32,363][19170] Avg episode reward: [(0, '863.700'), (1, '102.320')] [2023-12-27 13:08:32,398][20381] Updated weights for policy 1, policy_version 1080532 (0.0007) [2023-12-27 13:08:32,623][20347] Updated weights for policy 0, policy_version 1081482 (0.0007) [2023-12-27 13:08:32,679][20347] Updated weights for policy 0, policy_version 1081492 (0.0008) [2023-12-27 13:08:32,728][20347] Updated weights for policy 0, policy_version 1081502 (0.0006) [2023-12-27 13:08:32,779][20347] Updated weights for policy 0, policy_version 1081512 (0.0005) [2023-12-27 13:08:33,153][20381] Updated weights for policy 1, policy_version 1080542 (0.0006) [2023-12-27 13:08:33,207][20381] Updated weights for policy 1, policy_version 1080552 (0.0008) [2023-12-27 13:08:33,263][20381] Updated weights for policy 1, policy_version 1080562 (0.0008) [2023-12-27 13:08:33,359][20347] Updated weights for policy 0, policy_version 1081522 (0.0008) [2023-12-27 13:08:33,417][20347] Updated weights for policy 0, policy_version 1081532 (0.0008) [2023-12-27 13:08:33,469][20347] Updated weights for policy 0, policy_version 1081542 (0.0007) [2023-12-27 13:08:33,856][20381] Updated weights for policy 1, policy_version 1080572 (0.0009) [2023-12-27 13:08:33,918][20381] Updated weights for policy 1, policy_version 1080582 (0.0006) [2023-12-27 13:08:33,981][20381] Updated weights for policy 1, policy_version 1080592 (0.0006) [2023-12-27 13:08:34,063][20347] Updated weights for policy 0, policy_version 1081552 (0.0006) [2023-12-27 13:08:34,110][20347] Updated weights for policy 0, policy_version 1081562 (0.0007) [2023-12-27 13:08:34,169][20347] Updated weights for policy 0, policy_version 1081572 (0.0008) [2023-12-27 13:08:34,632][20381] Updated weights for policy 1, policy_version 1080602 (0.0010) [2023-12-27 13:08:34,685][20381] Updated weights for policy 1, policy_version 1080612 (0.0006) [2023-12-27 13:08:34,733][20381] Updated weights for policy 1, policy_version 1080622 (0.0006) [2023-12-27 13:08:34,780][20381] Updated weights for policy 1, policy_version 1080632 (0.0008) [2023-12-27 13:08:34,808][20347] Updated weights for policy 0, policy_version 1081582 (0.0007) [2023-12-27 13:08:34,866][20347] Updated weights for policy 0, policy_version 1081592 (0.0007) [2023-12-27 13:08:34,921][20347] Updated weights for policy 0, policy_version 1081602 (0.0007) [2023-12-27 13:08:35,462][20381] Updated weights for policy 1, policy_version 1080642 (0.0006) [2023-12-27 13:08:35,521][20347] Updated weights for policy 0, policy_version 1081612 (0.0007) [2023-12-27 13:08:35,525][20381] Updated weights for policy 1, policy_version 1080652 (0.0007) [2023-12-27 13:08:35,570][20347] Updated weights for policy 0, policy_version 1081622 (0.0006) [2023-12-27 13:08:35,581][20381] Updated weights for policy 1, policy_version 1080662 (0.0008) [2023-12-27 13:08:35,626][20347] Updated weights for policy 0, policy_version 1081632 (0.0007) [2023-12-27 13:08:36,155][20381] Updated weights for policy 1, policy_version 1080672 (0.0009) [2023-12-27 13:08:36,211][20381] Updated weights for policy 1, policy_version 1080682 (0.0007) [2023-12-27 13:08:36,266][20381] Updated weights for policy 1, policy_version 1080692 (0.0008) [2023-12-27 13:08:36,277][20347] Updated weights for policy 0, policy_version 1081642 (0.0007) [2023-12-27 13:08:36,327][20347] Updated weights for policy 0, policy_version 1081652 (0.0007) [2023-12-27 13:08:36,385][20347] Updated weights for policy 0, policy_version 1081662 (0.0008) [2023-12-27 13:08:36,439][20347] Updated weights for policy 0, policy_version 1081672 (0.0008) [2023-12-27 13:08:36,975][20381] Updated weights for policy 1, policy_version 1080702 (0.0008) [2023-12-27 13:08:37,025][20381] Updated weights for policy 1, policy_version 1080712 (0.0008) [2023-12-27 13:08:37,079][20381] Updated weights for policy 1, policy_version 1080722 (0.0008) [2023-12-27 13:08:37,187][20347] Updated weights for policy 0, policy_version 1081682 (0.0008) [2023-12-27 13:08:37,238][20347] Updated weights for policy 0, policy_version 1081692 (0.0008) [2023-12-27 13:08:37,297][20347] Updated weights for policy 0, policy_version 1081702 (0.0006) [2023-12-27 13:08:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 553656320. Throughput: 0: 10362.1, 1: 10194.1. Samples: 553619040. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:37,363][19170] Avg episode reward: [(0, '892.580'), (1, '30.660')] [2023-12-27 13:08:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001080728_276692992.pth... [2023-12-27 13:08:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001081704_276963328.pth... [2023-12-27 13:08:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001079544_276389888.pth [2023-12-27 13:08:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001080488_276652032.pth [2023-12-27 13:08:37,811][20381] Updated weights for policy 1, policy_version 1080732 (0.0008) [2023-12-27 13:08:37,871][20381] Updated weights for policy 1, policy_version 1080742 (0.0006) [2023-12-27 13:08:37,932][20381] Updated weights for policy 1, policy_version 1080752 (0.0007) [2023-12-27 13:08:37,957][20291] KL-divergence is very high: 104.1471 [2023-12-27 13:08:37,963][20347] Updated weights for policy 0, policy_version 1081712 (0.0009) [2023-12-27 13:08:38,018][20347] Updated weights for policy 0, policy_version 1081722 (0.0010) [2023-12-27 13:08:38,076][20347] Updated weights for policy 0, policy_version 1081732 (0.0010) [2023-12-27 13:08:38,593][20381] Updated weights for policy 1, policy_version 1080762 (0.0006) [2023-12-27 13:08:38,647][20381] Updated weights for policy 1, policy_version 1080772 (0.0007) [2023-12-27 13:08:38,708][20381] Updated weights for policy 1, policy_version 1080782 (0.0007) [2023-12-27 13:08:38,766][20381] Updated weights for policy 1, policy_version 1080792 (0.0007) [2023-12-27 13:08:38,821][20347] Updated weights for policy 0, policy_version 1081742 (0.0009) [2023-12-27 13:08:38,870][20347] Updated weights for policy 0, policy_version 1081752 (0.0007) [2023-12-27 13:08:38,922][20347] Updated weights for policy 0, policy_version 1081762 (0.0007) [2023-12-27 13:08:39,403][20381] Updated weights for policy 1, policy_version 1080802 (0.0009) [2023-12-27 13:08:39,459][20381] Updated weights for policy 1, policy_version 1080812 (0.0008) [2023-12-27 13:08:39,507][20381] Updated weights for policy 1, policy_version 1080822 (0.0008) [2023-12-27 13:08:39,654][20347] Updated weights for policy 0, policy_version 1081772 (0.0006) [2023-12-27 13:08:39,710][20347] Updated weights for policy 0, policy_version 1081782 (0.0010) [2023-12-27 13:08:39,767][20347] Updated weights for policy 0, policy_version 1081792 (0.0011) [2023-12-27 13:08:40,302][20381] Updated weights for policy 1, policy_version 1080832 (0.0009) [2023-12-27 13:08:40,356][20381] Updated weights for policy 1, policy_version 1080842 (0.0009) [2023-12-27 13:08:40,409][20381] Updated weights for policy 1, policy_version 1080852 (0.0009) [2023-12-27 13:08:40,623][20347] Updated weights for policy 0, policy_version 1081802 (0.0010) [2023-12-27 13:08:40,672][20347] Updated weights for policy 0, policy_version 1081812 (0.0009) [2023-12-27 13:08:40,720][20347] Updated weights for policy 0, policy_version 1081822 (0.0008) [2023-12-27 13:08:40,778][20347] Updated weights for policy 0, policy_version 1081832 (0.0006) [2023-12-27 13:08:41,190][20381] Updated weights for policy 1, policy_version 1080862 (0.0008) [2023-12-27 13:08:41,257][20381] Updated weights for policy 1, policy_version 1080872 (0.0008) [2023-12-27 13:08:41,316][20381] Updated weights for policy 1, policy_version 1080882 (0.0007) [2023-12-27 13:08:41,569][20347] Updated weights for policy 0, policy_version 1081842 (0.0006) [2023-12-27 13:08:41,636][20347] Updated weights for policy 0, policy_version 1081852 (0.0011) [2023-12-27 13:08:41,699][20347] Updated weights for policy 0, policy_version 1081862 (0.0009) [2023-12-27 13:08:42,085][20381] Updated weights for policy 1, policy_version 1080892 (0.0008) [2023-12-27 13:08:42,139][20381] Updated weights for policy 1, policy_version 1080902 (0.0008) [2023-12-27 13:08:42,188][20381] Updated weights for policy 1, policy_version 1080912 (0.0008) [2023-12-27 13:08:42,303][20347] Updated weights for policy 0, policy_version 1081872 (0.0007) [2023-12-27 13:08:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 553746432. Throughput: 0: 10280.7, 1: 10181.2. Samples: 553738128. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:42,363][19170] Avg episode reward: [(0, '931.700'), (1, '30.490')] [2023-12-27 13:08:42,367][20347] Updated weights for policy 0, policy_version 1081882 (0.0008) [2023-12-27 13:08:42,430][20347] Updated weights for policy 0, policy_version 1081892 (0.0011) [2023-12-27 13:08:42,960][20381] Updated weights for policy 1, policy_version 1080922 (0.0009) [2023-12-27 13:08:43,010][20381] Updated weights for policy 1, policy_version 1080932 (0.0010) [2023-12-27 13:08:43,055][20381] Updated weights for policy 1, policy_version 1080942 (0.0010) [2023-12-27 13:08:43,104][20381] Updated weights for policy 1, policy_version 1080952 (0.0010) [2023-12-27 13:08:43,127][20347] Updated weights for policy 0, policy_version 1081902 (0.0010) [2023-12-27 13:08:43,186][20347] Updated weights for policy 0, policy_version 1081912 (0.0010) [2023-12-27 13:08:43,247][20347] Updated weights for policy 0, policy_version 1081922 (0.0008) [2023-12-27 13:08:43,816][20381] Updated weights for policy 1, policy_version 1080962 (0.0008) [2023-12-27 13:08:43,870][20381] Updated weights for policy 1, policy_version 1080972 (0.0008) [2023-12-27 13:08:43,878][20347] Updated weights for policy 0, policy_version 1081932 (0.0010) [2023-12-27 13:08:43,919][20347] Updated weights for policy 0, policy_version 1081942 (0.0006) [2023-12-27 13:08:43,924][20381] Updated weights for policy 1, policy_version 1080982 (0.0006) [2023-12-27 13:08:43,964][20347] Updated weights for policy 0, policy_version 1081952 (0.0007) [2023-12-27 13:08:44,596][20381] Updated weights for policy 1, policy_version 1080992 (0.0007) [2023-12-27 13:08:44,653][20381] Updated weights for policy 1, policy_version 1081002 (0.0008) [2023-12-27 13:08:44,705][20347] Updated weights for policy 0, policy_version 1081962 (0.0007) [2023-12-27 13:08:44,714][20381] Updated weights for policy 1, policy_version 1081012 (0.0007) [2023-12-27 13:08:44,762][20347] Updated weights for policy 0, policy_version 1081972 (0.0006) [2023-12-27 13:08:44,823][20347] Updated weights for policy 0, policy_version 1081982 (0.0008) [2023-12-27 13:08:44,887][20347] Updated weights for policy 0, policy_version 1081992 (0.0008) [2023-12-27 13:08:45,392][20381] Updated weights for policy 1, policy_version 1081022 (0.0006) [2023-12-27 13:08:45,438][20381] Updated weights for policy 1, policy_version 1081032 (0.0008) [2023-12-27 13:08:45,488][20381] Updated weights for policy 1, policy_version 1081042 (0.0008) [2023-12-27 13:08:45,565][20347] Updated weights for policy 0, policy_version 1082002 (0.0006) [2023-12-27 13:08:45,627][20347] Updated weights for policy 0, policy_version 1082012 (0.0005) [2023-12-27 13:08:45,688][20347] Updated weights for policy 0, policy_version 1082022 (0.0005) [2023-12-27 13:08:46,192][20381] Updated weights for policy 1, policy_version 1081052 (0.0007) [2023-12-27 13:08:46,246][20381] Updated weights for policy 1, policy_version 1081062 (0.0007) [2023-12-27 13:08:46,293][20381] Updated weights for policy 1, policy_version 1081072 (0.0008) [2023-12-27 13:08:46,333][20347] Updated weights for policy 0, policy_version 1082032 (0.0005) [2023-12-27 13:08:46,379][20347] Updated weights for policy 0, policy_version 1082042 (0.0007) [2023-12-27 13:08:46,427][20347] Updated weights for policy 0, policy_version 1082052 (0.0008) [2023-12-27 13:08:46,948][20381] Updated weights for policy 1, policy_version 1081082 (0.0010) [2023-12-27 13:08:46,993][20381] Updated weights for policy 1, policy_version 1081092 (0.0010) [2023-12-27 13:08:47,046][20381] Updated weights for policy 1, policy_version 1081102 (0.0011) [2023-12-27 13:08:47,053][20347] Updated weights for policy 0, policy_version 1082062 (0.0007) [2023-12-27 13:08:47,105][20381] Updated weights for policy 1, policy_version 1081112 (0.0011) [2023-12-27 13:08:47,112][20347] Updated weights for policy 0, policy_version 1082072 (0.0010) [2023-12-27 13:08:47,167][20347] Updated weights for policy 0, policy_version 1082082 (0.0010) [2023-12-27 13:08:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 553852928. Throughput: 0: 10281.0, 1: 10203.0. Samples: 553859660. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:47,363][19170] Avg episode reward: [(0, '912.080'), (1, '51.470')] [2023-12-27 13:08:47,811][20381] Updated weights for policy 1, policy_version 1081122 (0.0011) [2023-12-27 13:08:47,828][20347] Updated weights for policy 0, policy_version 1082092 (0.0008) [2023-12-27 13:08:47,875][20381] Updated weights for policy 1, policy_version 1081132 (0.0010) [2023-12-27 13:08:47,888][20347] Updated weights for policy 0, policy_version 1082102 (0.0005) [2023-12-27 13:08:47,933][20381] Updated weights for policy 1, policy_version 1081142 (0.0011) [2023-12-27 13:08:47,942][20347] Updated weights for policy 0, policy_version 1082112 (0.0006) [2023-12-27 13:08:48,563][20347] Updated weights for policy 0, policy_version 1082122 (0.0010) [2023-12-27 13:08:48,598][20381] Updated weights for policy 1, policy_version 1081152 (0.0006) [2023-12-27 13:08:48,619][20347] Updated weights for policy 0, policy_version 1082132 (0.0010) [2023-12-27 13:08:48,657][20381] Updated weights for policy 1, policy_version 1081162 (0.0006) [2023-12-27 13:08:48,677][20347] Updated weights for policy 0, policy_version 1082142 (0.0010) [2023-12-27 13:08:48,710][20381] Updated weights for policy 1, policy_version 1081172 (0.0007) [2023-12-27 13:08:48,735][20347] Updated weights for policy 0, policy_version 1082152 (0.0009) [2023-12-27 13:08:49,402][20381] Updated weights for policy 1, policy_version 1081182 (0.0008) [2023-12-27 13:08:49,458][20381] Updated weights for policy 1, policy_version 1081192 (0.0008) [2023-12-27 13:08:49,474][20347] Updated weights for policy 0, policy_version 1082162 (0.0011) [2023-12-27 13:08:49,512][20381] Updated weights for policy 1, policy_version 1081202 (0.0010) [2023-12-27 13:08:49,534][20347] Updated weights for policy 0, policy_version 1082172 (0.0011) [2023-12-27 13:08:49,591][20347] Updated weights for policy 0, policy_version 1082182 (0.0009) [2023-12-27 13:08:50,223][20381] Updated weights for policy 1, policy_version 1081212 (0.0007) [2023-12-27 13:08:50,247][20347] Updated weights for policy 0, policy_version 1082192 (0.0006) [2023-12-27 13:08:50,283][20381] Updated weights for policy 1, policy_version 1081222 (0.0010) [2023-12-27 13:08:50,305][20347] Updated weights for policy 0, policy_version 1082202 (0.0006) [2023-12-27 13:08:50,343][20381] Updated weights for policy 1, policy_version 1081232 (0.0011) [2023-12-27 13:08:50,365][20347] Updated weights for policy 0, policy_version 1082212 (0.0006) [2023-12-27 13:08:51,014][20347] Updated weights for policy 0, policy_version 1082222 (0.0008) [2023-12-27 13:08:51,027][20381] Updated weights for policy 1, policy_version 1081242 (0.0007) [2023-12-27 13:08:51,082][20347] Updated weights for policy 0, policy_version 1082232 (0.0008) [2023-12-27 13:08:51,119][20381] Updated weights for policy 1, policy_version 1081252 (0.0011) [2023-12-27 13:08:51,147][20347] Updated weights for policy 0, policy_version 1082242 (0.0006) [2023-12-27 13:08:51,186][20381] Updated weights for policy 1, policy_version 1081262 (0.0007) [2023-12-27 13:08:51,245][20381] Updated weights for policy 1, policy_version 1081272 (0.0011) [2023-12-27 13:08:51,875][20347] Updated weights for policy 0, policy_version 1082252 (0.0007) [2023-12-27 13:08:51,931][20347] Updated weights for policy 0, policy_version 1082262 (0.0006) [2023-12-27 13:08:51,976][20381] Updated weights for policy 1, policy_version 1081282 (0.0008) [2023-12-27 13:08:51,991][20347] Updated weights for policy 0, policy_version 1082272 (0.0006) [2023-12-27 13:08:52,040][20381] Updated weights for policy 1, policy_version 1081292 (0.0008) [2023-12-27 13:08:52,104][20381] Updated weights for policy 1, policy_version 1081302 (0.0008) [2023-12-27 13:08:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 553951232. Throughput: 0: 10277.2, 1: 10176.8. Samples: 553920812. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:52,363][19170] Avg episode reward: [(0, '921.890'), (1, '142.280')] [2023-12-27 13:08:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001081304_276840448.pth... [2023-12-27 13:08:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001082280_277110784.pth... [2023-12-27 13:08:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001080120_276537344.pth [2023-12-27 13:08:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001081064_276799488.pth [2023-12-27 13:08:52,732][20347] Updated weights for policy 0, policy_version 1082282 (0.0006) [2023-12-27 13:08:52,786][20347] Updated weights for policy 0, policy_version 1082292 (0.0007) [2023-12-27 13:08:52,787][20381] Updated weights for policy 1, policy_version 1081312 (0.0006) [2023-12-27 13:08:52,837][20381] Updated weights for policy 1, policy_version 1081322 (0.0005) [2023-12-27 13:08:52,841][20347] Updated weights for policy 0, policy_version 1082302 (0.0007) [2023-12-27 13:08:52,892][20347] Updated weights for policy 0, policy_version 1082312 (0.0007) [2023-12-27 13:08:52,896][20381] Updated weights for policy 1, policy_version 1081332 (0.0006) [2023-12-27 13:08:53,500][20347] Updated weights for policy 0, policy_version 1082322 (0.0007) [2023-12-27 13:08:53,543][20291] KL-divergence is very high: 430.9124 [2023-12-27 13:08:53,566][20347] Updated weights for policy 0, policy_version 1082332 (0.0005) [2023-12-27 13:08:53,592][20291] KL-divergence is very high: 344.8399 [2023-12-27 13:08:53,622][20347] Updated weights for policy 0, policy_version 1082342 (0.0006) [2023-12-27 13:08:53,624][20381] Updated weights for policy 1, policy_version 1081342 (0.0008) [2023-12-27 13:08:53,683][20381] Updated weights for policy 1, policy_version 1081352 (0.0009) [2023-12-27 13:08:53,744][20381] Updated weights for policy 1, policy_version 1081362 (0.0008) [2023-12-27 13:08:54,173][20291] KL-divergence is very high: 189.4960 [2023-12-27 13:08:54,210][20347] Updated weights for policy 0, policy_version 1082352 (0.0005) [2023-12-27 13:08:54,215][20291] KL-divergence is very high: 121.5971 [2023-12-27 13:08:54,264][20347] Updated weights for policy 0, policy_version 1082362 (0.0006) [2023-12-27 13:08:54,320][20347] Updated weights for policy 0, policy_version 1082372 (0.0006) [2023-12-27 13:08:54,428][20381] Updated weights for policy 1, policy_version 1081372 (0.0008) [2023-12-27 13:08:54,480][20381] Updated weights for policy 1, policy_version 1081382 (0.0008) [2023-12-27 13:08:54,528][20381] Updated weights for policy 1, policy_version 1081392 (0.0007) [2023-12-27 13:08:54,891][20347] Updated weights for policy 0, policy_version 1082382 (0.0009) [2023-12-27 13:08:54,948][20347] Updated weights for policy 0, policy_version 1082392 (0.0010) [2023-12-27 13:08:55,009][20347] Updated weights for policy 0, policy_version 1082402 (0.0008) [2023-12-27 13:08:55,197][20381] Updated weights for policy 1, policy_version 1081402 (0.0010) [2023-12-27 13:08:55,242][20381] Updated weights for policy 1, policy_version 1081412 (0.0010) [2023-12-27 13:08:55,304][20381] Updated weights for policy 1, policy_version 1081422 (0.0008) [2023-12-27 13:08:55,364][20381] Updated weights for policy 1, policy_version 1081432 (0.0009) [2023-12-27 13:08:55,655][20347] Updated weights for policy 0, policy_version 1082412 (0.0009) [2023-12-27 13:08:55,709][20347] Updated weights for policy 0, policy_version 1082422 (0.0005) [2023-12-27 13:08:55,758][20347] Updated weights for policy 0, policy_version 1082432 (0.0005) [2023-12-27 13:08:56,047][20381] Updated weights for policy 1, policy_version 1081442 (0.0005) [2023-12-27 13:08:56,107][20381] Updated weights for policy 1, policy_version 1081452 (0.0006) [2023-12-27 13:08:56,156][20381] Updated weights for policy 1, policy_version 1081462 (0.0005) [2023-12-27 13:08:56,338][20347] Updated weights for policy 0, policy_version 1082442 (0.0006) [2023-12-27 13:08:56,393][20347] Updated weights for policy 0, policy_version 1082452 (0.0007) [2023-12-27 13:08:56,456][20347] Updated weights for policy 0, policy_version 1082462 (0.0008) [2023-12-27 13:08:56,509][20347] Updated weights for policy 0, policy_version 1082472 (0.0007) [2023-12-27 13:08:56,703][20381] Updated weights for policy 1, policy_version 1081472 (0.0007) [2023-12-27 13:08:56,755][20381] Updated weights for policy 1, policy_version 1081482 (0.0007) [2023-12-27 13:08:56,815][20381] Updated weights for policy 1, policy_version 1081492 (0.0008) [2023-12-27 13:08:57,220][20347] Updated weights for policy 0, policy_version 1082482 (0.0008) [2023-12-27 13:08:57,283][20347] Updated weights for policy 0, policy_version 1082493 (0.0007) [2023-12-27 13:08:57,335][20347] Updated weights for policy 0, policy_version 1082503 (0.0009) [2023-12-27 13:08:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 554057728. Throughput: 0: 10396.0, 1: 10088.8. Samples: 554044672. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:08:57,363][19170] Avg episode reward: [(0, '921.780'), (1, '79.550')] [2023-12-27 13:08:57,471][20381] Updated weights for policy 1, policy_version 1081502 (0.0006) [2023-12-27 13:08:57,521][20381] Updated weights for policy 1, policy_version 1081512 (0.0005) [2023-12-27 13:08:57,569][20381] Updated weights for policy 1, policy_version 1081522 (0.0005) [2023-12-27 13:08:58,017][20347] Updated weights for policy 0, policy_version 1082513 (0.0010) [2023-12-27 13:08:58,071][20347] Updated weights for policy 0, policy_version 1082523 (0.0010) [2023-12-27 13:08:58,109][20381] Updated weights for policy 1, policy_version 1081532 (0.0008) [2023-12-27 13:08:58,135][20347] Updated weights for policy 0, policy_version 1082533 (0.0010) [2023-12-27 13:08:58,167][20381] Updated weights for policy 1, policy_version 1081542 (0.0005) [2023-12-27 13:08:58,231][20381] Updated weights for policy 1, policy_version 1081552 (0.0008) [2023-12-27 13:08:58,833][20347] Updated weights for policy 0, policy_version 1082543 (0.0009) [2023-12-27 13:08:58,896][20347] Updated weights for policy 0, policy_version 1082553 (0.0008) [2023-12-27 13:08:58,960][20347] Updated weights for policy 0, policy_version 1082563 (0.0008) [2023-12-27 13:08:58,981][20381] Updated weights for policy 1, policy_version 1081562 (0.0008) [2023-12-27 13:08:59,050][20381] Updated weights for policy 1, policy_version 1081572 (0.0008) [2023-12-27 13:08:59,106][20381] Updated weights for policy 1, policy_version 1081582 (0.0008) [2023-12-27 13:08:59,169][20381] Updated weights for policy 1, policy_version 1081592 (0.0007) [2023-12-27 13:08:59,736][20347] Updated weights for policy 0, policy_version 1082573 (0.0009) [2023-12-27 13:08:59,805][20347] Updated weights for policy 0, policy_version 1082583 (0.0009) [2023-12-27 13:08:59,865][20381] Updated weights for policy 1, policy_version 1081602 (0.0008) [2023-12-27 13:08:59,869][20347] Updated weights for policy 0, policy_version 1082593 (0.0008) [2023-12-27 13:08:59,921][20381] Updated weights for policy 1, policy_version 1081612 (0.0006) [2023-12-27 13:08:59,990][20381] Updated weights for policy 1, policy_version 1081622 (0.0008) [2023-12-27 13:09:00,551][20347] Updated weights for policy 0, policy_version 1082603 (0.0008) [2023-12-27 13:09:00,601][20347] Updated weights for policy 0, policy_version 1082613 (0.0007) [2023-12-27 13:09:00,645][20381] Updated weights for policy 1, policy_version 1081632 (0.0005) [2023-12-27 13:09:00,647][20347] Updated weights for policy 0, policy_version 1082623 (0.0006) [2023-12-27 13:09:00,694][20381] Updated weights for policy 1, policy_version 1081642 (0.0006) [2023-12-27 13:09:00,750][20381] Updated weights for policy 1, policy_version 1081652 (0.0008) [2023-12-27 13:09:01,256][20347] Updated weights for policy 0, policy_version 1082633 (0.0006) [2023-12-27 13:09:01,310][20347] Updated weights for policy 0, policy_version 1082643 (0.0007) [2023-12-27 13:09:01,378][20347] Updated weights for policy 0, policy_version 1082653 (0.0008) [2023-12-27 13:09:01,427][20381] Updated weights for policy 1, policy_version 1081662 (0.0008) [2023-12-27 13:09:01,442][20347] Updated weights for policy 0, policy_version 1082663 (0.0007) [2023-12-27 13:09:01,482][20381] Updated weights for policy 1, policy_version 1081672 (0.0008) [2023-12-27 13:09:01,537][20381] Updated weights for policy 1, policy_version 1081682 (0.0008) [2023-12-27 13:09:02,119][20347] Updated weights for policy 0, policy_version 1082673 (0.0008) [2023-12-27 13:09:02,172][20347] Updated weights for policy 0, policy_version 1082683 (0.0007) [2023-12-27 13:09:02,187][20381] Updated weights for policy 1, policy_version 1081692 (0.0008) [2023-12-27 13:09:02,228][20347] Updated weights for policy 0, policy_version 1082693 (0.0007) [2023-12-27 13:09:02,251][20381] Updated weights for policy 1, policy_version 1081702 (0.0007) [2023-12-27 13:09:02,315][20381] Updated weights for policy 1, policy_version 1081712 (0.0008) [2023-12-27 13:09:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 554156032. Throughput: 0: 10337.8, 1: 10129.9. Samples: 554168052. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:02,363][19170] Avg episode reward: [(0, '931.710'), (1, '114.600')] [2023-12-27 13:09:02,847][20347] Updated weights for policy 0, policy_version 1082703 (0.0010) [2023-12-27 13:09:02,911][20347] Updated weights for policy 0, policy_version 1082713 (0.0010) [2023-12-27 13:09:02,974][20347] Updated weights for policy 0, policy_version 1082723 (0.0007) [2023-12-27 13:09:02,980][20381] Updated weights for policy 1, policy_version 1081722 (0.0007) [2023-12-27 13:09:03,029][20381] Updated weights for policy 1, policy_version 1081732 (0.0006) [2023-12-27 13:09:03,079][20381] Updated weights for policy 1, policy_version 1081742 (0.0005) [2023-12-27 13:09:03,126][20381] Updated weights for policy 1, policy_version 1081752 (0.0006) [2023-12-27 13:09:03,639][20347] Updated weights for policy 0, policy_version 1082733 (0.0010) [2023-12-27 13:09:03,694][20347] Updated weights for policy 0, policy_version 1082743 (0.0010) [2023-12-27 13:09:03,711][20381] Updated weights for policy 1, policy_version 1081762 (0.0010) [2023-12-27 13:09:03,745][20347] Updated weights for policy 0, policy_version 1082753 (0.0010) [2023-12-27 13:09:03,763][20381] Updated weights for policy 1, policy_version 1081772 (0.0010) [2023-12-27 13:09:03,818][20381] Updated weights for policy 1, policy_version 1081782 (0.0011) [2023-12-27 13:09:04,402][20347] Updated weights for policy 0, policy_version 1082763 (0.0010) [2023-12-27 13:09:04,456][20347] Updated weights for policy 0, policy_version 1082773 (0.0011) [2023-12-27 13:09:04,477][20381] Updated weights for policy 1, policy_version 1081792 (0.0011) [2023-12-27 13:09:04,503][20347] Updated weights for policy 0, policy_version 1082783 (0.0010) [2023-12-27 13:09:04,532][20381] Updated weights for policy 1, policy_version 1081802 (0.0011) [2023-12-27 13:09:04,588][20381] Updated weights for policy 1, policy_version 1081812 (0.0011) [2023-12-27 13:09:05,196][20347] Updated weights for policy 0, policy_version 1082793 (0.0011) [2023-12-27 13:09:05,250][20347] Updated weights for policy 0, policy_version 1082803 (0.0010) [2023-12-27 13:09:05,292][20381] Updated weights for policy 1, policy_version 1081822 (0.0008) [2023-12-27 13:09:05,307][20347] Updated weights for policy 0, policy_version 1082813 (0.0010) [2023-12-27 13:09:05,343][20381] Updated weights for policy 1, policy_version 1081832 (0.0010) [2023-12-27 13:09:05,364][20347] Updated weights for policy 0, policy_version 1082823 (0.0010) [2023-12-27 13:09:05,410][20381] Updated weights for policy 1, policy_version 1081842 (0.0011) [2023-12-27 13:09:06,016][20347] Updated weights for policy 0, policy_version 1082833 (0.0010) [2023-12-27 13:09:06,054][20381] Updated weights for policy 1, policy_version 1081852 (0.0008) [2023-12-27 13:09:06,074][20347] Updated weights for policy 0, policy_version 1082843 (0.0010) [2023-12-27 13:09:06,116][20381] Updated weights for policy 1, policy_version 1081862 (0.0010) [2023-12-27 13:09:06,133][20347] Updated weights for policy 0, policy_version 1082853 (0.0009) [2023-12-27 13:09:06,176][20381] Updated weights for policy 1, policy_version 1081872 (0.0009) [2023-12-27 13:09:06,814][20347] Updated weights for policy 0, policy_version 1082863 (0.0010) [2023-12-27 13:09:06,866][20347] Updated weights for policy 0, policy_version 1082873 (0.0010) [2023-12-27 13:09:06,885][20381] Updated weights for policy 1, policy_version 1081882 (0.0008) [2023-12-27 13:09:06,921][20347] Updated weights for policy 0, policy_version 1082883 (0.0010) [2023-12-27 13:09:06,942][20381] Updated weights for policy 1, policy_version 1081892 (0.0006) [2023-12-27 13:09:07,006][20381] Updated weights for policy 1, policy_version 1081902 (0.0010) [2023-12-27 13:09:07,062][20381] Updated weights for policy 1, policy_version 1081912 (0.0011) [2023-12-27 13:09:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 554262528. Throughput: 0: 10341.3, 1: 10160.1. Samples: 554231120. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:07,363][19170] Avg episode reward: [(0, '951.280'), (1, '109.420')] [2023-12-27 13:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001081912_276996096.pth... [2023-12-27 13:09:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001082888_277266432.pth... [2023-12-27 13:09:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001081704_276963328.pth [2023-12-27 13:09:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001080728_276692992.pth [2023-12-27 13:09:07,565][20347] Updated weights for policy 0, policy_version 1082893 (0.0008) [2023-12-27 13:09:07,627][20347] Updated weights for policy 0, policy_version 1082903 (0.0010) [2023-12-27 13:09:07,675][20381] Updated weights for policy 1, policy_version 1081922 (0.0006) [2023-12-27 13:09:07,686][20347] Updated weights for policy 0, policy_version 1082913 (0.0010) [2023-12-27 13:09:07,735][20381] Updated weights for policy 1, policy_version 1081932 (0.0011) [2023-12-27 13:09:07,798][20381] Updated weights for policy 1, policy_version 1081942 (0.0011) [2023-12-27 13:09:08,270][20347] Updated weights for policy 0, policy_version 1082923 (0.0006) [2023-12-27 13:09:08,330][20347] Updated weights for policy 0, policy_version 1082933 (0.0007) [2023-12-27 13:09:08,394][20347] Updated weights for policy 0, policy_version 1082943 (0.0009) [2023-12-27 13:09:08,513][20381] Updated weights for policy 1, policy_version 1081952 (0.0008) [2023-12-27 13:09:08,583][20381] Updated weights for policy 1, policy_version 1081962 (0.0008) [2023-12-27 13:09:08,638][20381] Updated weights for policy 1, policy_version 1081972 (0.0006) [2023-12-27 13:09:09,114][20347] Updated weights for policy 0, policy_version 1082953 (0.0009) [2023-12-27 13:09:09,172][20347] Updated weights for policy 0, policy_version 1082963 (0.0010) [2023-12-27 13:09:09,233][20347] Updated weights for policy 0, policy_version 1082973 (0.0011) [2023-12-27 13:09:09,291][20347] Updated weights for policy 0, policy_version 1082983 (0.0008) [2023-12-27 13:09:09,382][20381] Updated weights for policy 1, policy_version 1081982 (0.0008) [2023-12-27 13:09:09,448][20381] Updated weights for policy 1, policy_version 1081992 (0.0008) [2023-12-27 13:09:09,505][20381] Updated weights for policy 1, policy_version 1082002 (0.0005) [2023-12-27 13:09:09,999][20347] Updated weights for policy 0, policy_version 1082993 (0.0010) [2023-12-27 13:09:10,060][20347] Updated weights for policy 0, policy_version 1083003 (0.0009) [2023-12-27 13:09:10,114][20347] Updated weights for policy 0, policy_version 1083013 (0.0011) [2023-12-27 13:09:10,141][20381] Updated weights for policy 1, policy_version 1082012 (0.0007) [2023-12-27 13:09:10,205][20381] Updated weights for policy 1, policy_version 1082022 (0.0005) [2023-12-27 13:09:10,267][20381] Updated weights for policy 1, policy_version 1082032 (0.0007) [2023-12-27 13:09:10,821][20347] Updated weights for policy 0, policy_version 1083023 (0.0010) [2023-12-27 13:09:10,883][20347] Updated weights for policy 0, policy_version 1083033 (0.0008) [2023-12-27 13:09:10,927][20381] Updated weights for policy 1, policy_version 1082042 (0.0008) [2023-12-27 13:09:10,947][20347] Updated weights for policy 0, policy_version 1083043 (0.0010) [2023-12-27 13:09:10,982][20381] Updated weights for policy 1, policy_version 1082052 (0.0006) [2023-12-27 13:09:11,054][20381] Updated weights for policy 1, policy_version 1082064 (0.0007) [2023-12-27 13:09:11,691][20347] Updated weights for policy 0, policy_version 1083053 (0.0009) [2023-12-27 13:09:11,761][20347] Updated weights for policy 0, policy_version 1083063 (0.0008) [2023-12-27 13:09:11,768][20381] Updated weights for policy 1, policy_version 1082074 (0.0006) [2023-12-27 13:09:11,823][20347] Updated weights for policy 0, policy_version 1083073 (0.0006) [2023-12-27 13:09:11,825][20381] Updated weights for policy 1, policy_version 1082084 (0.0006) [2023-12-27 13:09:11,888][20381] Updated weights for policy 1, policy_version 1082094 (0.0007) [2023-12-27 13:09:11,950][20381] Updated weights for policy 1, policy_version 1082104 (0.0008) [2023-12-27 13:09:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 554360832. Throughput: 0: 10368.6, 1: 10161.9. Samples: 554353592. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:12,363][19170] Avg episode reward: [(0, '941.270'), (1, '204.830')] [2023-12-27 13:09:12,487][20347] Updated weights for policy 0, policy_version 1083083 (0.0007) [2023-12-27 13:09:12,547][20347] Updated weights for policy 0, policy_version 1083093 (0.0008) [2023-12-27 13:09:12,609][20347] Updated weights for policy 0, policy_version 1083103 (0.0009) [2023-12-27 13:09:12,702][20381] Updated weights for policy 1, policy_version 1082114 (0.0007) [2023-12-27 13:09:12,757][20381] Updated weights for policy 1, policy_version 1082124 (0.0008) [2023-12-27 13:09:12,817][20381] Updated weights for policy 1, policy_version 1082134 (0.0007) [2023-12-27 13:09:13,236][20347] Updated weights for policy 0, policy_version 1083113 (0.0006) [2023-12-27 13:09:13,296][20347] Updated weights for policy 0, policy_version 1083123 (0.0010) [2023-12-27 13:09:13,353][20347] Updated weights for policy 0, policy_version 1083133 (0.0005) [2023-12-27 13:09:13,409][20347] Updated weights for policy 0, policy_version 1083143 (0.0005) [2023-12-27 13:09:13,505][20381] Updated weights for policy 1, policy_version 1082144 (0.0008) [2023-12-27 13:09:13,558][20381] Updated weights for policy 1, policy_version 1082154 (0.0007) [2023-12-27 13:09:13,606][20381] Updated weights for policy 1, policy_version 1082164 (0.0008) [2023-12-27 13:09:13,979][20347] Updated weights for policy 0, policy_version 1083153 (0.0007) [2023-12-27 13:09:14,009][20291] KL-divergence is very high: 379.0306 [2023-12-27 13:09:14,032][20347] Updated weights for policy 0, policy_version 1083163 (0.0007) [2023-12-27 13:09:14,052][20291] KL-divergence is very high: 413.6184 [2023-12-27 13:09:14,084][20347] Updated weights for policy 0, policy_version 1083173 (0.0007) [2023-12-27 13:09:14,097][20291] KL-divergence is very high: 202.1290 [2023-12-27 13:09:14,296][20381] Updated weights for policy 1, policy_version 1082174 (0.0007) [2023-12-27 13:09:14,351][20381] Updated weights for policy 1, policy_version 1082184 (0.0007) [2023-12-27 13:09:14,404][20381] Updated weights for policy 1, policy_version 1082194 (0.0007) [2023-12-27 13:09:14,746][20347] Updated weights for policy 0, policy_version 1083183 (0.0006) [2023-12-27 13:09:14,812][20347] Updated weights for policy 0, policy_version 1083193 (0.0008) [2023-12-27 13:09:14,879][20347] Updated weights for policy 0, policy_version 1083203 (0.0008) [2023-12-27 13:09:15,072][20381] Updated weights for policy 1, policy_version 1082204 (0.0009) [2023-12-27 13:09:15,122][20381] Updated weights for policy 1, policy_version 1082214 (0.0011) [2023-12-27 13:09:15,179][20381] Updated weights for policy 1, policy_version 1082224 (0.0011) [2023-12-27 13:09:15,496][20347] Updated weights for policy 0, policy_version 1083213 (0.0007) [2023-12-27 13:09:15,541][20347] Updated weights for policy 0, policy_version 1083223 (0.0005) [2023-12-27 13:09:15,583][20347] Updated weights for policy 0, policy_version 1083233 (0.0005) [2023-12-27 13:09:15,877][20381] Updated weights for policy 1, policy_version 1082234 (0.0011) [2023-12-27 13:09:15,935][20381] Updated weights for policy 1, policy_version 1082244 (0.0010) [2023-12-27 13:09:15,987][20381] Updated weights for policy 1, policy_version 1082254 (0.0011) [2023-12-27 13:09:16,039][20381] Updated weights for policy 1, policy_version 1082264 (0.0007) [2023-12-27 13:09:16,287][20347] Updated weights for policy 0, policy_version 1083243 (0.0006) [2023-12-27 13:09:16,335][20347] Updated weights for policy 0, policy_version 1083253 (0.0008) [2023-12-27 13:09:16,384][20347] Updated weights for policy 0, policy_version 1083263 (0.0009) [2023-12-27 13:09:16,588][20381] Updated weights for policy 1, policy_version 1082274 (0.0010) [2023-12-27 13:09:16,643][20381] Updated weights for policy 1, policy_version 1082284 (0.0010) [2023-12-27 13:09:16,696][20381] Updated weights for policy 1, policy_version 1082294 (0.0011) [2023-12-27 13:09:17,068][20347] Updated weights for policy 0, policy_version 1083273 (0.0009) [2023-12-27 13:09:17,125][20347] Updated weights for policy 0, policy_version 1083283 (0.0007) [2023-12-27 13:09:17,179][20347] Updated weights for policy 0, policy_version 1083293 (0.0007) [2023-12-27 13:09:17,239][20347] Updated weights for policy 0, policy_version 1083303 (0.0008) [2023-12-27 13:09:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 554467328. Throughput: 0: 10316.5, 1: 10186.6. Samples: 554478428. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:17,363][19170] Avg episode reward: [(0, '970.750'), (1, '247.900')] [2023-12-27 13:09:17,386][20381] Updated weights for policy 1, policy_version 1082304 (0.0011) [2023-12-27 13:09:17,449][20381] Updated weights for policy 1, policy_version 1082314 (0.0007) [2023-12-27 13:09:17,514][20381] Updated weights for policy 1, policy_version 1082324 (0.0010) [2023-12-27 13:09:17,860][20347] Updated weights for policy 0, policy_version 1083313 (0.0007) [2023-12-27 13:09:17,924][20347] Updated weights for policy 0, policy_version 1083323 (0.0007) [2023-12-27 13:09:17,985][20347] Updated weights for policy 0, policy_version 1083333 (0.0008) [2023-12-27 13:09:18,132][20381] Updated weights for policy 1, policy_version 1082334 (0.0009) [2023-12-27 13:09:18,193][20381] Updated weights for policy 1, policy_version 1082344 (0.0005) [2023-12-27 13:09:18,258][20381] Updated weights for policy 1, policy_version 1082354 (0.0005) [2023-12-27 13:09:18,621][20347] Updated weights for policy 0, policy_version 1083343 (0.0010) [2023-12-27 13:09:18,686][20347] Updated weights for policy 0, policy_version 1083353 (0.0010) [2023-12-27 13:09:18,748][20347] Updated weights for policy 0, policy_version 1083363 (0.0010) [2023-12-27 13:09:18,864][20381] Updated weights for policy 1, policy_version 1082364 (0.0008) [2023-12-27 13:09:18,923][20381] Updated weights for policy 1, policy_version 1082374 (0.0007) [2023-12-27 13:09:18,980][20381] Updated weights for policy 1, policy_version 1082384 (0.0005) [2023-12-27 13:09:19,352][20347] Updated weights for policy 0, policy_version 1083373 (0.0008) [2023-12-27 13:09:19,414][20347] Updated weights for policy 0, policy_version 1083383 (0.0010) [2023-12-27 13:09:19,476][20347] Updated weights for policy 0, policy_version 1083393 (0.0007) [2023-12-27 13:09:19,653][20381] Updated weights for policy 1, policy_version 1082394 (0.0006) [2023-12-27 13:09:19,717][20381] Updated weights for policy 1, policy_version 1082404 (0.0009) [2023-12-27 13:09:19,778][20381] Updated weights for policy 1, policy_version 1082414 (0.0007) [2023-12-27 13:09:19,845][20381] Updated weights for policy 1, policy_version 1082424 (0.0007) [2023-12-27 13:09:20,149][20347] Updated weights for policy 0, policy_version 1083403 (0.0007) [2023-12-27 13:09:20,204][20347] Updated weights for policy 0, policy_version 1083413 (0.0008) [2023-12-27 13:09:20,267][20347] Updated weights for policy 0, policy_version 1083423 (0.0007) [2023-12-27 13:09:20,540][20381] Updated weights for policy 1, policy_version 1082434 (0.0011) [2023-12-27 13:09:20,603][20381] Updated weights for policy 1, policy_version 1082444 (0.0009) [2023-12-27 13:09:20,664][20381] Updated weights for policy 1, policy_version 1082454 (0.0006) [2023-12-27 13:09:20,987][20347] Updated weights for policy 0, policy_version 1083433 (0.0007) [2023-12-27 13:09:21,069][20347] Updated weights for policy 0, policy_version 1083443 (0.0008) [2023-12-27 13:09:21,134][20347] Updated weights for policy 0, policy_version 1083453 (0.0008) [2023-12-27 13:09:21,198][20347] Updated weights for policy 0, policy_version 1083463 (0.0007) [2023-12-27 13:09:21,256][20381] Updated weights for policy 1, policy_version 1082464 (0.0006) [2023-12-27 13:09:21,318][20381] Updated weights for policy 1, policy_version 1082474 (0.0011) [2023-12-27 13:09:21,388][20381] Updated weights for policy 1, policy_version 1082484 (0.0012) [2023-12-27 13:09:21,892][20347] Updated weights for policy 0, policy_version 1083473 (0.0007) [2023-12-27 13:09:21,951][20347] Updated weights for policy 0, policy_version 1083483 (0.0006) [2023-12-27 13:09:22,013][20347] Updated weights for policy 0, policy_version 1083493 (0.0006) [2023-12-27 13:09:22,084][20381] Updated weights for policy 1, policy_version 1082494 (0.0010) [2023-12-27 13:09:22,139][20381] Updated weights for policy 1, policy_version 1082504 (0.0011) [2023-12-27 13:09:22,197][20381] Updated weights for policy 1, policy_version 1082514 (0.0011) [2023-12-27 13:09:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 554573824. Throughput: 0: 10293.0, 1: 10199.3. Samples: 554541200. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:22,363][19170] Avg episode reward: [(0, '951.290'), (1, '309.760')] [2023-12-27 13:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001082520_277151744.pth... [2023-12-27 13:09:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001083496_277422080.pth... [2023-12-27 13:09:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001081304_276840448.pth [2023-12-27 13:09:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001082280_277110784.pth [2023-12-27 13:09:22,705][20347] Updated weights for policy 0, policy_version 1083503 (0.0007) [2023-12-27 13:09:22,724][20291] KL-divergence is very high: 572.6832 [2023-12-27 13:09:22,755][20347] Updated weights for policy 0, policy_version 1083513 (0.0005) [2023-12-27 13:09:22,769][20291] KL-divergence is very high: 355.3238 [2023-12-27 13:09:22,818][20347] Updated weights for policy 0, policy_version 1083523 (0.0006) [2023-12-27 13:09:22,819][20291] KL-divergence is very high: 220.7610 [2023-12-27 13:09:22,915][20381] Updated weights for policy 1, policy_version 1082524 (0.0008) [2023-12-27 13:09:22,981][20381] Updated weights for policy 1, policy_version 1082534 (0.0011) [2023-12-27 13:09:23,044][20381] Updated weights for policy 1, policy_version 1082544 (0.0011) [2023-12-27 13:09:23,418][20347] Updated weights for policy 0, policy_version 1083533 (0.0007) [2023-12-27 13:09:23,473][20347] Updated weights for policy 0, policy_version 1083543 (0.0008) [2023-12-27 13:09:23,521][20347] Updated weights for policy 0, policy_version 1083553 (0.0007) [2023-12-27 13:09:23,720][20381] Updated weights for policy 1, policy_version 1082554 (0.0010) [2023-12-27 13:09:23,776][20381] Updated weights for policy 1, policy_version 1082564 (0.0007) [2023-12-27 13:09:23,835][20381] Updated weights for policy 1, policy_version 1082574 (0.0008) [2023-12-27 13:09:23,888][20381] Updated weights for policy 1, policy_version 1082584 (0.0007) [2023-12-27 13:09:24,192][20347] Updated weights for policy 0, policy_version 1083563 (0.0007) [2023-12-27 13:09:24,244][20347] Updated weights for policy 0, policy_version 1083573 (0.0007) [2023-12-27 13:09:24,301][20347] Updated weights for policy 0, policy_version 1083583 (0.0008) [2023-12-27 13:09:24,606][20381] Updated weights for policy 1, policy_version 1082594 (0.0007) [2023-12-27 13:09:24,665][20381] Updated weights for policy 1, policy_version 1082604 (0.0008) [2023-12-27 13:09:24,723][20381] Updated weights for policy 1, policy_version 1082614 (0.0007) [2023-12-27 13:09:24,952][20347] Updated weights for policy 0, policy_version 1083593 (0.0007) [2023-12-27 13:09:25,013][20347] Updated weights for policy 0, policy_version 1083603 (0.0005) [2023-12-27 13:09:25,069][20347] Updated weights for policy 0, policy_version 1083613 (0.0006) [2023-12-27 13:09:25,124][20347] Updated weights for policy 0, policy_version 1083623 (0.0005) [2023-12-27 13:09:25,381][20381] Updated weights for policy 1, policy_version 1082624 (0.0008) [2023-12-27 13:09:25,429][20381] Updated weights for policy 1, policy_version 1082634 (0.0008) [2023-12-27 13:09:25,473][20381] Updated weights for policy 1, policy_version 1082644 (0.0007) [2023-12-27 13:09:25,639][20347] Updated weights for policy 0, policy_version 1083633 (0.0008) [2023-12-27 13:09:25,684][20347] Updated weights for policy 0, policy_version 1083643 (0.0007) [2023-12-27 13:09:25,738][20347] Updated weights for policy 0, policy_version 1083653 (0.0006) [2023-12-27 13:09:26,180][20381] Updated weights for policy 1, policy_version 1082654 (0.0007) [2023-12-27 13:09:26,233][20381] Updated weights for policy 1, policy_version 1082664 (0.0007) [2023-12-27 13:09:26,293][20381] Updated weights for policy 1, policy_version 1082674 (0.0008) [2023-12-27 13:09:26,403][20347] Updated weights for policy 0, policy_version 1083663 (0.0007) [2023-12-27 13:09:26,455][20347] Updated weights for policy 0, policy_version 1083673 (0.0007) [2023-12-27 13:09:26,499][20347] Updated weights for policy 0, policy_version 1083683 (0.0007) [2023-12-27 13:09:26,912][20381] Updated weights for policy 1, policy_version 1082684 (0.0006) [2023-12-27 13:09:26,967][20381] Updated weights for policy 1, policy_version 1082694 (0.0005) [2023-12-27 13:09:27,026][20381] Updated weights for policy 1, policy_version 1082704 (0.0007) [2023-12-27 13:09:27,159][20347] Updated weights for policy 0, policy_version 1083693 (0.0007) [2023-12-27 13:09:27,189][20291] KL-divergence is very high: 101.2661 [2023-12-27 13:09:27,211][20347] Updated weights for policy 0, policy_version 1083703 (0.0005) [2023-12-27 13:09:27,263][20347] Updated weights for policy 0, policy_version 1083713 (0.0006) [2023-12-27 13:09:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 554680320. Throughput: 0: 10367.5, 1: 10215.2. Samples: 554664348. Policy #0 lag: (min: 25.0, avg: 39.0, max: 57.0) [2023-12-27 13:09:27,363][19170] Avg episode reward: [(0, '931.930'), (1, '165.990')] [2023-12-27 13:09:27,660][20381] Updated weights for policy 1, policy_version 1082714 (0.0009) [2023-12-27 13:09:27,710][20381] Updated weights for policy 1, policy_version 1082724 (0.0009) [2023-12-27 13:09:27,767][20381] Updated weights for policy 1, policy_version 1082734 (0.0005) [2023-12-27 13:09:27,819][20381] Updated weights for policy 1, policy_version 1082744 (0.0005) [2023-12-27 13:09:27,937][20347] Updated weights for policy 0, policy_version 1083723 (0.0008) [2023-12-27 13:09:27,991][20347] Updated weights for policy 0, policy_version 1083733 (0.0008) [2023-12-27 13:09:28,047][20347] Updated weights for policy 0, policy_version 1083743 (0.0007) [2023-12-27 13:09:28,346][20381] Updated weights for policy 1, policy_version 1082754 (0.0010) [2023-12-27 13:09:28,403][20381] Updated weights for policy 1, policy_version 1082764 (0.0008) [2023-12-27 13:09:28,450][20381] Updated weights for policy 1, policy_version 1082774 (0.0007) [2023-12-27 13:09:28,688][20347] Updated weights for policy 0, policy_version 1083753 (0.0008) [2023-12-27 13:09:28,737][20347] Updated weights for policy 0, policy_version 1083763 (0.0008) [2023-12-27 13:09:28,798][20347] Updated weights for policy 0, policy_version 1083773 (0.0006) [2023-12-27 13:09:28,860][20347] Updated weights for policy 0, policy_version 1083783 (0.0007) [2023-12-27 13:09:29,115][20381] Updated weights for policy 1, policy_version 1082784 (0.0008) [2023-12-27 13:09:29,172][20381] Updated weights for policy 1, policy_version 1082794 (0.0010) [2023-12-27 13:09:29,237][20381] Updated weights for policy 1, policy_version 1082804 (0.0009) [2023-12-27 13:09:29,515][20347] Updated weights for policy 0, policy_version 1083793 (0.0007) [2023-12-27 13:09:29,563][20347] Updated weights for policy 0, policy_version 1083803 (0.0007) [2023-12-27 13:09:29,615][20347] Updated weights for policy 0, policy_version 1083813 (0.0007) [2023-12-27 13:09:29,942][20381] Updated weights for policy 1, policy_version 1082814 (0.0010) [2023-12-27 13:09:30,006][20381] Updated weights for policy 1, policy_version 1082824 (0.0007) [2023-12-27 13:09:30,066][20381] Updated weights for policy 1, policy_version 1082834 (0.0008) [2023-12-27 13:09:30,267][20347] Updated weights for policy 0, policy_version 1083823 (0.0006) [2023-12-27 13:09:30,318][20347] Updated weights for policy 0, policy_version 1083833 (0.0007) [2023-12-27 13:09:30,373][20347] Updated weights for policy 0, policy_version 1083843 (0.0007) [2023-12-27 13:09:30,743][20381] Updated weights for policy 1, policy_version 1082844 (0.0007) [2023-12-27 13:09:30,802][20381] Updated weights for policy 1, policy_version 1082854 (0.0008) [2023-12-27 13:09:30,856][20381] Updated weights for policy 1, policy_version 1082864 (0.0008) [2023-12-27 13:09:30,943][20347] Updated weights for policy 0, policy_version 1083853 (0.0006) [2023-12-27 13:09:30,994][20347] Updated weights for policy 0, policy_version 1083863 (0.0007) [2023-12-27 13:09:31,056][20347] Updated weights for policy 0, policy_version 1083873 (0.0007) [2023-12-27 13:09:31,566][20381] Updated weights for policy 1, policy_version 1082874 (0.0009) [2023-12-27 13:09:31,632][20381] Updated weights for policy 1, policy_version 1082884 (0.0007) [2023-12-27 13:09:31,695][20381] Updated weights for policy 1, policy_version 1082894 (0.0008) [2023-12-27 13:09:31,749][20347] Updated weights for policy 0, policy_version 1083883 (0.0009) [2023-12-27 13:09:31,759][20381] Updated weights for policy 1, policy_version 1082904 (0.0009) [2023-12-27 13:09:31,810][20347] Updated weights for policy 0, policy_version 1083893 (0.0009) [2023-12-27 13:09:31,873][20347] Updated weights for policy 0, policy_version 1083903 (0.0008) [2023-12-27 13:09:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 554778624. Throughput: 0: 10398.0, 1: 10299.7. Samples: 554791060. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:32,363][19170] Avg episode reward: [(0, '951.050'), (1, '390.070')] [2023-12-27 13:09:32,471][20381] Updated weights for policy 1, policy_version 1082914 (0.0006) [2023-12-27 13:09:32,531][20381] Updated weights for policy 1, policy_version 1082924 (0.0008) [2023-12-27 13:09:32,548][20347] Updated weights for policy 0, policy_version 1083913 (0.0007) [2023-12-27 13:09:32,589][20381] Updated weights for policy 1, policy_version 1082934 (0.0006) [2023-12-27 13:09:32,602][20347] Updated weights for policy 0, policy_version 1083923 (0.0007) [2023-12-27 13:09:32,663][20347] Updated weights for policy 0, policy_version 1083933 (0.0007) [2023-12-27 13:09:32,722][20347] Updated weights for policy 0, policy_version 1083943 (0.0008) [2023-12-27 13:09:33,251][20381] Updated weights for policy 1, policy_version 1082944 (0.0007) [2023-12-27 13:09:33,310][20381] Updated weights for policy 1, policy_version 1082954 (0.0007) [2023-12-27 13:09:33,332][20347] Updated weights for policy 0, policy_version 1083953 (0.0008) [2023-12-27 13:09:33,368][20381] Updated weights for policy 1, policy_version 1082964 (0.0007) [2023-12-27 13:09:33,389][20347] Updated weights for policy 0, policy_version 1083963 (0.0005) [2023-12-27 13:09:33,436][20347] Updated weights for policy 0, policy_version 1083973 (0.0009) [2023-12-27 13:09:34,017][20347] Updated weights for policy 0, policy_version 1083983 (0.0007) [2023-12-27 13:09:34,075][20347] Updated weights for policy 0, policy_version 1083993 (0.0008) [2023-12-27 13:09:34,084][20381] Updated weights for policy 1, policy_version 1082974 (0.0007) [2023-12-27 13:09:34,129][20347] Updated weights for policy 0, policy_version 1084003 (0.0007) [2023-12-27 13:09:34,130][20381] Updated weights for policy 1, policy_version 1082984 (0.0006) [2023-12-27 13:09:34,194][20381] Updated weights for policy 1, policy_version 1082994 (0.0007) [2023-12-27 13:09:34,807][20347] Updated weights for policy 0, policy_version 1084013 (0.0009) [2023-12-27 13:09:34,852][20347] Updated weights for policy 0, policy_version 1084023 (0.0010) [2023-12-27 13:09:34,899][20347] Updated weights for policy 0, policy_version 1084033 (0.0010) [2023-12-27 13:09:34,913][20381] Updated weights for policy 1, policy_version 1083004 (0.0009) [2023-12-27 13:09:34,965][20381] Updated weights for policy 1, policy_version 1083014 (0.0007) [2023-12-27 13:09:35,025][20381] Updated weights for policy 1, policy_version 1083024 (0.0008) [2023-12-27 13:09:35,578][20347] Updated weights for policy 0, policy_version 1084043 (0.0009) [2023-12-27 13:09:35,624][20347] Updated weights for policy 0, policy_version 1084053 (0.0007) [2023-12-27 13:09:35,684][20347] Updated weights for policy 0, policy_version 1084063 (0.0008) [2023-12-27 13:09:35,708][20381] Updated weights for policy 1, policy_version 1083034 (0.0008) [2023-12-27 13:09:35,765][20381] Updated weights for policy 1, policy_version 1083044 (0.0007) [2023-12-27 13:09:35,817][20381] Updated weights for policy 1, policy_version 1083054 (0.0005) [2023-12-27 13:09:35,881][20381] Updated weights for policy 1, policy_version 1083064 (0.0005) [2023-12-27 13:09:36,440][20347] Updated weights for policy 0, policy_version 1084073 (0.0007) [2023-12-27 13:09:36,493][20381] Updated weights for policy 1, policy_version 1083074 (0.0008) [2023-12-27 13:09:36,495][20347] Updated weights for policy 0, policy_version 1084083 (0.0008) [2023-12-27 13:09:36,547][20381] Updated weights for policy 1, policy_version 1083084 (0.0006) [2023-12-27 13:09:36,549][20347] Updated weights for policy 0, policy_version 1084093 (0.0006) [2023-12-27 13:09:36,604][20381] Updated weights for policy 1, policy_version 1083094 (0.0006) [2023-12-27 13:09:36,606][20347] Updated weights for policy 0, policy_version 1084103 (0.0006) [2023-12-27 13:09:37,289][20347] Updated weights for policy 0, policy_version 1084113 (0.0007) [2023-12-27 13:09:37,317][20381] Updated weights for policy 1, policy_version 1083104 (0.0007) [2023-12-27 13:09:37,346][20347] Updated weights for policy 0, policy_version 1084123 (0.0006) [2023-12-27 13:09:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 554876928. Throughput: 0: 10447.1, 1: 10279.0. Samples: 554853488. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:37,363][19170] Avg episode reward: [(0, '980.390'), (1, '410.320')] [2023-12-27 13:09:37,381][20381] Updated weights for policy 1, policy_version 1083114 (0.0007) [2023-12-27 13:09:37,402][20347] Updated weights for policy 0, policy_version 1084133 (0.0005) [2023-12-27 13:09:37,415][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001084136_277585920.pth... [2023-12-27 13:09:37,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001082888_277266432.pth [2023-12-27 13:09:37,442][20381] Updated weights for policy 1, policy_version 1083124 (0.0007) [2023-12-27 13:09:37,463][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001083128_277307392.pth... [2023-12-27 13:09:37,468][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001081912_276996096.pth [2023-12-27 13:09:38,075][20381] Updated weights for policy 1, policy_version 1083134 (0.0008) [2023-12-27 13:09:38,106][20347] Updated weights for policy 0, policy_version 1084143 (0.0007) [2023-12-27 13:09:38,127][20381] Updated weights for policy 1, policy_version 1083144 (0.0007) [2023-12-27 13:09:38,162][20347] Updated weights for policy 0, policy_version 1084153 (0.0005) [2023-12-27 13:09:38,179][20381] Updated weights for policy 1, policy_version 1083154 (0.0008) [2023-12-27 13:09:38,224][20347] Updated weights for policy 0, policy_version 1084163 (0.0005) [2023-12-27 13:09:38,824][20347] Updated weights for policy 0, policy_version 1084173 (0.0006) [2023-12-27 13:09:38,888][20347] Updated weights for policy 0, policy_version 1084183 (0.0007) [2023-12-27 13:09:38,892][20381] Updated weights for policy 1, policy_version 1083164 (0.0009) [2023-12-27 13:09:38,950][20347] Updated weights for policy 0, policy_version 1084193 (0.0009) [2023-12-27 13:09:38,952][20381] Updated weights for policy 1, policy_version 1083174 (0.0008) [2023-12-27 13:09:39,010][20381] Updated weights for policy 1, policy_version 1083184 (0.0008) [2023-12-27 13:09:39,669][20347] Updated weights for policy 0, policy_version 1084203 (0.0010) [2023-12-27 13:09:39,737][20347] Updated weights for policy 0, policy_version 1084213 (0.0011) [2023-12-27 13:09:39,790][20381] Updated weights for policy 1, policy_version 1083194 (0.0008) [2023-12-27 13:09:39,804][20347] Updated weights for policy 0, policy_version 1084223 (0.0011) [2023-12-27 13:09:39,852][20381] Updated weights for policy 1, policy_version 1083204 (0.0008) [2023-12-27 13:09:39,911][20381] Updated weights for policy 1, policy_version 1083214 (0.0008) [2023-12-27 13:09:39,975][20381] Updated weights for policy 1, policy_version 1083224 (0.0008) [2023-12-27 13:09:40,456][20347] Updated weights for policy 0, policy_version 1084233 (0.0009) [2023-12-27 13:09:40,514][20347] Updated weights for policy 0, policy_version 1084243 (0.0005) [2023-12-27 13:09:40,564][20347] Updated weights for policy 0, policy_version 1084253 (0.0006) [2023-12-27 13:09:40,618][20347] Updated weights for policy 0, policy_version 1084263 (0.0006) [2023-12-27 13:09:40,698][20381] Updated weights for policy 1, policy_version 1083234 (0.0007) [2023-12-27 13:09:40,764][20381] Updated weights for policy 1, policy_version 1083244 (0.0008) [2023-12-27 13:09:40,821][20381] Updated weights for policy 1, policy_version 1083254 (0.0007) [2023-12-27 13:09:41,274][20347] Updated weights for policy 0, policy_version 1084273 (0.0008) [2023-12-27 13:09:41,332][20347] Updated weights for policy 0, policy_version 1084283 (0.0007) [2023-12-27 13:09:41,402][20347] Updated weights for policy 0, policy_version 1084293 (0.0006) [2023-12-27 13:09:41,534][20381] Updated weights for policy 1, policy_version 1083264 (0.0006) [2023-12-27 13:09:41,594][20381] Updated weights for policy 1, policy_version 1083274 (0.0006) [2023-12-27 13:09:41,663][20381] Updated weights for policy 1, policy_version 1083284 (0.0008) [2023-12-27 13:09:42,101][20347] Updated weights for policy 0, policy_version 1084303 (0.0007) [2023-12-27 13:09:42,154][20347] Updated weights for policy 0, policy_version 1084313 (0.0007) [2023-12-27 13:09:42,210][20347] Updated weights for policy 0, policy_version 1084323 (0.0007) [2023-12-27 13:09:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 554983424. Throughput: 0: 10388.0, 1: 10271.0. Samples: 554974328. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '360.730')] [2023-12-27 13:09:42,410][20381] Updated weights for policy 1, policy_version 1083294 (0.0008) [2023-12-27 13:09:42,460][20381] Updated weights for policy 1, policy_version 1083304 (0.0008) [2023-12-27 13:09:42,521][20381] Updated weights for policy 1, policy_version 1083314 (0.0009) [2023-12-27 13:09:42,849][20347] Updated weights for policy 0, policy_version 1084333 (0.0006) [2023-12-27 13:09:42,912][20347] Updated weights for policy 0, policy_version 1084343 (0.0005) [2023-12-27 13:09:42,963][20347] Updated weights for policy 0, policy_version 1084353 (0.0006) [2023-12-27 13:09:43,206][20381] Updated weights for policy 1, policy_version 1083324 (0.0009) [2023-12-27 13:09:43,265][20381] Updated weights for policy 1, policy_version 1083334 (0.0010) [2023-12-27 13:09:43,322][20381] Updated weights for policy 1, policy_version 1083344 (0.0008) [2023-12-27 13:09:43,567][20347] Updated weights for policy 0, policy_version 1084363 (0.0008) [2023-12-27 13:09:43,612][20347] Updated weights for policy 0, policy_version 1084373 (0.0008) [2023-12-27 13:09:43,667][20347] Updated weights for policy 0, policy_version 1084383 (0.0008) [2023-12-27 13:09:43,903][20381] Updated weights for policy 1, policy_version 1083354 (0.0009) [2023-12-27 13:09:43,950][20381] Updated weights for policy 1, policy_version 1083364 (0.0005) [2023-12-27 13:09:44,001][20381] Updated weights for policy 1, policy_version 1083374 (0.0005) [2023-12-27 13:09:44,061][20381] Updated weights for policy 1, policy_version 1083384 (0.0006) [2023-12-27 13:09:44,398][20347] Updated weights for policy 0, policy_version 1084393 (0.0007) [2023-12-27 13:09:44,455][20347] Updated weights for policy 0, policy_version 1084403 (0.0009) [2023-12-27 13:09:44,517][20347] Updated weights for policy 0, policy_version 1084413 (0.0006) [2023-12-27 13:09:44,575][20347] Updated weights for policy 0, policy_version 1084423 (0.0007) [2023-12-27 13:09:44,647][20381] Updated weights for policy 1, policy_version 1083394 (0.0008) [2023-12-27 13:09:44,711][20381] Updated weights for policy 1, policy_version 1083404 (0.0008) [2023-12-27 13:09:44,770][20381] Updated weights for policy 1, policy_version 1083414 (0.0008) [2023-12-27 13:09:45,250][20347] Updated weights for policy 0, policy_version 1084433 (0.0007) [2023-12-27 13:09:45,314][20347] Updated weights for policy 0, policy_version 1084443 (0.0009) [2023-12-27 13:09:45,380][20347] Updated weights for policy 0, policy_version 1084453 (0.0007) [2023-12-27 13:09:45,454][20381] Updated weights for policy 1, policy_version 1083424 (0.0008) [2023-12-27 13:09:45,507][20381] Updated weights for policy 1, policy_version 1083434 (0.0008) [2023-12-27 13:09:45,556][20381] Updated weights for policy 1, policy_version 1083444 (0.0009) [2023-12-27 13:09:46,045][20347] Updated weights for policy 0, policy_version 1084463 (0.0008) [2023-12-27 13:09:46,103][20347] Updated weights for policy 0, policy_version 1084473 (0.0008) [2023-12-27 13:09:46,162][20347] Updated weights for policy 0, policy_version 1084483 (0.0008) [2023-12-27 13:09:46,239][20381] Updated weights for policy 1, policy_version 1083454 (0.0007) [2023-12-27 13:09:46,285][20381] Updated weights for policy 1, policy_version 1083464 (0.0005) [2023-12-27 13:09:46,330][20381] Updated weights for policy 1, policy_version 1083474 (0.0006) [2023-12-27 13:09:46,863][20347] Updated weights for policy 0, policy_version 1084493 (0.0007) [2023-12-27 13:09:46,910][20347] Updated weights for policy 0, policy_version 1084503 (0.0006) [2023-12-27 13:09:46,964][20347] Updated weights for policy 0, policy_version 1084513 (0.0005) [2023-12-27 13:09:47,020][20381] Updated weights for policy 1, policy_version 1083484 (0.0008) [2023-12-27 13:09:47,083][20381] Updated weights for policy 1, policy_version 1083494 (0.0006) [2023-12-27 13:09:47,137][20381] Updated weights for policy 1, policy_version 1083504 (0.0005) [2023-12-27 13:09:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 555089920. Throughput: 0: 10417.2, 1: 10241.2. Samples: 555097676. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:47,363][19170] Avg episode reward: [(0, '990.300'), (1, '408.030')] [2023-12-27 13:09:47,567][20347] Updated weights for policy 0, policy_version 1084523 (0.0005) [2023-12-27 13:09:47,629][20347] Updated weights for policy 0, policy_version 1084533 (0.0010) [2023-12-27 13:09:47,692][20347] Updated weights for policy 0, policy_version 1084543 (0.0006) [2023-12-27 13:09:47,820][20381] Updated weights for policy 1, policy_version 1083514 (0.0006) [2023-12-27 13:09:47,867][20381] Updated weights for policy 1, policy_version 1083524 (0.0007) [2023-12-27 13:09:47,923][20381] Updated weights for policy 1, policy_version 1083534 (0.0009) [2023-12-27 13:09:47,976][20381] Updated weights for policy 1, policy_version 1083544 (0.0009) [2023-12-27 13:09:48,247][20347] Updated weights for policy 0, policy_version 1084553 (0.0008) [2023-12-27 13:09:48,311][20347] Updated weights for policy 0, policy_version 1084563 (0.0009) [2023-12-27 13:09:48,376][20347] Updated weights for policy 0, policy_version 1084573 (0.0009) [2023-12-27 13:09:48,431][20347] Updated weights for policy 0, policy_version 1084583 (0.0006) [2023-12-27 13:09:48,724][20381] Updated weights for policy 1, policy_version 1083554 (0.0008) [2023-12-27 13:09:48,789][20381] Updated weights for policy 1, policy_version 1083564 (0.0007) [2023-12-27 13:09:48,853][20381] Updated weights for policy 1, policy_version 1083574 (0.0006) [2023-12-27 13:09:49,120][20347] Updated weights for policy 0, policy_version 1084593 (0.0011) [2023-12-27 13:09:49,179][20347] Updated weights for policy 0, policy_version 1084603 (0.0010) [2023-12-27 13:09:49,239][20347] Updated weights for policy 0, policy_version 1084613 (0.0010) [2023-12-27 13:09:49,498][20381] Updated weights for policy 1, policy_version 1083584 (0.0008) [2023-12-27 13:09:49,556][20381] Updated weights for policy 1, policy_version 1083594 (0.0008) [2023-12-27 13:09:49,616][20381] Updated weights for policy 1, policy_version 1083604 (0.0008) [2023-12-27 13:09:49,921][20347] Updated weights for policy 0, policy_version 1084623 (0.0010) [2023-12-27 13:09:49,983][20347] Updated weights for policy 0, policy_version 1084633 (0.0007) [2023-12-27 13:09:50,050][20347] Updated weights for policy 0, policy_version 1084643 (0.0008) [2023-12-27 13:09:50,356][20381] Updated weights for policy 1, policy_version 1083614 (0.0009) [2023-12-27 13:09:50,415][20381] Updated weights for policy 1, policy_version 1083624 (0.0009) [2023-12-27 13:09:50,463][20381] Updated weights for policy 1, policy_version 1083634 (0.0008) [2023-12-27 13:09:50,685][20347] Updated weights for policy 0, policy_version 1084653 (0.0009) [2023-12-27 13:09:50,746][20347] Updated weights for policy 0, policy_version 1084663 (0.0006) [2023-12-27 13:09:50,804][20347] Updated weights for policy 0, policy_version 1084673 (0.0005) [2023-12-27 13:09:51,236][20381] Updated weights for policy 1, policy_version 1083644 (0.0008) [2023-12-27 13:09:51,296][20381] Updated weights for policy 1, policy_version 1083654 (0.0007) [2023-12-27 13:09:51,356][20381] Updated weights for policy 1, policy_version 1083664 (0.0006) [2023-12-27 13:09:51,460][20347] Updated weights for policy 0, policy_version 1084683 (0.0007) [2023-12-27 13:09:51,518][20347] Updated weights for policy 0, policy_version 1084693 (0.0008) [2023-12-27 13:09:51,571][20347] Updated weights for policy 0, policy_version 1084703 (0.0007) [2023-12-27 13:09:52,012][20381] Updated weights for policy 1, policy_version 1083674 (0.0006) [2023-12-27 13:09:52,068][20381] Updated weights for policy 1, policy_version 1083684 (0.0005) [2023-12-27 13:09:52,125][20381] Updated weights for policy 1, policy_version 1083694 (0.0005) [2023-12-27 13:09:52,177][20381] Updated weights for policy 1, policy_version 1083704 (0.0007) [2023-12-27 13:09:52,302][20347] Updated weights for policy 0, policy_version 1084713 (0.0006) [2023-12-27 13:09:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 555188224. Throughput: 0: 10430.1, 1: 10182.6. Samples: 555158692. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:52,363][19170] Avg episode reward: [(0, '970.840'), (1, '379.790')] [2023-12-27 13:09:52,367][20347] Updated weights for policy 0, policy_version 1084723 (0.0007) [2023-12-27 13:09:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001083704_277454848.pth... [2023-12-27 13:09:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001082520_277151744.pth [2023-12-27 13:09:52,460][20347] Updated weights for policy 0, policy_version 1084733 (0.0007) [2023-12-27 13:09:52,519][20347] Updated weights for policy 0, policy_version 1084743 (0.0007) [2023-12-27 13:09:52,521][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001084744_277741568.pth... [2023-12-27 13:09:52,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001083496_277422080.pth [2023-12-27 13:09:52,872][20381] Updated weights for policy 1, policy_version 1083714 (0.0008) [2023-12-27 13:09:52,920][20381] Updated weights for policy 1, policy_version 1083724 (0.0009) [2023-12-27 13:09:52,966][20381] Updated weights for policy 1, policy_version 1083734 (0.0008) [2023-12-27 13:09:53,087][20347] Updated weights for policy 0, policy_version 1084753 (0.0005) [2023-12-27 13:09:53,138][20347] Updated weights for policy 0, policy_version 1084763 (0.0007) [2023-12-27 13:09:53,183][20347] Updated weights for policy 0, policy_version 1084773 (0.0007) [2023-12-27 13:09:53,666][20381] Updated weights for policy 1, policy_version 1083744 (0.0007) [2023-12-27 13:09:53,723][20381] Updated weights for policy 1, policy_version 1083754 (0.0008) [2023-12-27 13:09:53,782][20381] Updated weights for policy 1, policy_version 1083764 (0.0007) [2023-12-27 13:09:53,799][20347] Updated weights for policy 0, policy_version 1084783 (0.0007) [2023-12-27 13:09:53,850][20347] Updated weights for policy 0, policy_version 1084793 (0.0005) [2023-12-27 13:09:53,898][20347] Updated weights for policy 0, policy_version 1084803 (0.0005) [2023-12-27 13:09:54,503][20381] Updated weights for policy 1, policy_version 1083774 (0.0006) [2023-12-27 13:09:54,559][20381] Updated weights for policy 1, policy_version 1083784 (0.0006) [2023-12-27 13:09:54,582][20347] Updated weights for policy 0, policy_version 1084813 (0.0007) [2023-12-27 13:09:54,618][20381] Updated weights for policy 1, policy_version 1083794 (0.0007) [2023-12-27 13:09:54,643][20347] Updated weights for policy 0, policy_version 1084823 (0.0009) [2023-12-27 13:09:54,706][20347] Updated weights for policy 0, policy_version 1084833 (0.0011) [2023-12-27 13:09:55,286][20347] Updated weights for policy 0, policy_version 1084843 (0.0008) [2023-12-27 13:09:55,297][20381] Updated weights for policy 1, policy_version 1083804 (0.0006) [2023-12-27 13:09:55,339][20347] Updated weights for policy 0, policy_version 1084853 (0.0005) [2023-12-27 13:09:55,351][20381] Updated weights for policy 1, policy_version 1083814 (0.0006) [2023-12-27 13:09:55,385][20347] Updated weights for policy 0, policy_version 1084863 (0.0006) [2023-12-27 13:09:55,412][20381] Updated weights for policy 1, policy_version 1083824 (0.0005) [2023-12-27 13:09:55,995][20347] Updated weights for policy 0, policy_version 1084873 (0.0006) [2023-12-27 13:09:56,041][20381] Updated weights for policy 1, policy_version 1083834 (0.0006) [2023-12-27 13:09:56,053][20347] Updated weights for policy 0, policy_version 1084883 (0.0008) [2023-12-27 13:09:56,091][20381] Updated weights for policy 1, policy_version 1083844 (0.0010) [2023-12-27 13:09:56,104][20347] Updated weights for policy 0, policy_version 1084893 (0.0006) [2023-12-27 13:09:56,149][20381] Updated weights for policy 1, policy_version 1083854 (0.0010) [2023-12-27 13:09:56,159][20347] Updated weights for policy 0, policy_version 1084903 (0.0005) [2023-12-27 13:09:56,196][20381] Updated weights for policy 1, policy_version 1083864 (0.0009) [2023-12-27 13:09:56,818][20347] Updated weights for policy 0, policy_version 1084913 (0.0008) [2023-12-27 13:09:56,869][20347] Updated weights for policy 0, policy_version 1084923 (0.0010) [2023-12-27 13:09:56,890][20381] Updated weights for policy 1, policy_version 1083874 (0.0010) [2023-12-27 13:09:56,918][20347] Updated weights for policy 0, policy_version 1084933 (0.0010) [2023-12-27 13:09:56,944][20381] Updated weights for policy 1, policy_version 1083884 (0.0010) [2023-12-27 13:09:56,995][20381] Updated weights for policy 1, policy_version 1083894 (0.0010) [2023-12-27 13:09:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 555294720. Throughput: 0: 10500.9, 1: 10177.2. Samples: 555284108. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:09:57,364][19170] Avg episode reward: [(0, '980.540'), (1, '435.040')] [2023-12-27 13:09:57,564][20347] Updated weights for policy 0, policy_version 1084943 (0.0010) [2023-12-27 13:09:57,617][20347] Updated weights for policy 0, policy_version 1084953 (0.0010) [2023-12-27 13:09:57,629][20381] Updated weights for policy 1, policy_version 1083904 (0.0006) [2023-12-27 13:09:57,681][20347] Updated weights for policy 0, policy_version 1084963 (0.0010) [2023-12-27 13:09:57,688][20381] Updated weights for policy 1, policy_version 1083914 (0.0007) [2023-12-27 13:09:57,739][20381] Updated weights for policy 1, policy_version 1083924 (0.0008) [2023-12-27 13:09:58,271][20347] Updated weights for policy 0, policy_version 1084973 (0.0010) [2023-12-27 13:09:58,346][20381] Updated weights for policy 1, policy_version 1083934 (0.0013) [2023-12-27 13:09:58,346][20347] Updated weights for policy 0, policy_version 1084983 (0.0010) [2023-12-27 13:09:58,410][20347] Updated weights for policy 0, policy_version 1084993 (0.0010) [2023-12-27 13:09:58,430][20381] Updated weights for policy 1, policy_version 1083944 (0.0009) [2023-12-27 13:09:58,499][20381] Updated weights for policy 1, policy_version 1083954 (0.0008) [2023-12-27 13:09:59,158][20347] Updated weights for policy 0, policy_version 1085003 (0.0008) [2023-12-27 13:09:59,219][20347] Updated weights for policy 0, policy_version 1085013 (0.0008) [2023-12-27 13:09:59,257][20381] Updated weights for policy 1, policy_version 1083964 (0.0008) [2023-12-27 13:09:59,283][20347] Updated weights for policy 0, policy_version 1085023 (0.0008) [2023-12-27 13:09:59,312][20381] Updated weights for policy 1, policy_version 1083974 (0.0007) [2023-12-27 13:09:59,375][20381] Updated weights for policy 1, policy_version 1083984 (0.0006) [2023-12-27 13:09:59,993][20347] Updated weights for policy 0, policy_version 1085033 (0.0010) [2023-12-27 13:10:00,055][20347] Updated weights for policy 0, policy_version 1085043 (0.0008) [2023-12-27 13:10:00,106][20381] Updated weights for policy 1, policy_version 1083994 (0.0006) [2023-12-27 13:10:00,110][20347] Updated weights for policy 0, policy_version 1085053 (0.0008) [2023-12-27 13:10:00,150][20381] Updated weights for policy 1, policy_version 1084004 (0.0006) [2023-12-27 13:10:00,162][20347] Updated weights for policy 0, policy_version 1085063 (0.0008) [2023-12-27 13:10:00,194][20381] Updated weights for policy 1, policy_version 1084014 (0.0007) [2023-12-27 13:10:00,250][20381] Updated weights for policy 1, policy_version 1084024 (0.0008) [2023-12-27 13:10:00,778][20347] Updated weights for policy 0, policy_version 1085073 (0.0005) [2023-12-27 13:10:00,826][20347] Updated weights for policy 0, policy_version 1085083 (0.0006) [2023-12-27 13:10:00,877][20347] Updated weights for policy 0, policy_version 1085093 (0.0006) [2023-12-27 13:10:01,002][20381] Updated weights for policy 1, policy_version 1084034 (0.0008) [2023-12-27 13:10:01,065][20381] Updated weights for policy 1, policy_version 1084044 (0.0008) [2023-12-27 13:10:01,124][20381] Updated weights for policy 1, policy_version 1084054 (0.0009) [2023-12-27 13:10:01,540][20347] Updated weights for policy 0, policy_version 1085103 (0.0006) [2023-12-27 13:10:01,601][20347] Updated weights for policy 0, policy_version 1085113 (0.0006) [2023-12-27 13:10:01,666][20347] Updated weights for policy 0, policy_version 1085123 (0.0008) [2023-12-27 13:10:01,882][20381] Updated weights for policy 1, policy_version 1084064 (0.0008) [2023-12-27 13:10:01,940][20381] Updated weights for policy 1, policy_version 1084074 (0.0008) [2023-12-27 13:10:02,002][20381] Updated weights for policy 1, policy_version 1084084 (0.0009) [2023-12-27 13:10:02,323][20347] Updated weights for policy 0, policy_version 1085133 (0.0007) [2023-12-27 13:10:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 555393024. Throughput: 0: 10483.8, 1: 10097.6. Samples: 555404592. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '353.690')] [2023-12-27 13:10:02,390][20347] Updated weights for policy 0, policy_version 1085143 (0.0009) [2023-12-27 13:10:02,447][20347] Updated weights for policy 0, policy_version 1085153 (0.0008) [2023-12-27 13:10:02,708][20381] Updated weights for policy 1, policy_version 1084094 (0.0008) [2023-12-27 13:10:02,772][20381] Updated weights for policy 1, policy_version 1084104 (0.0008) [2023-12-27 13:10:02,831][20381] Updated weights for policy 1, policy_version 1084114 (0.0008) [2023-12-27 13:10:03,160][20347] Updated weights for policy 0, policy_version 1085163 (0.0008) [2023-12-27 13:10:03,220][20347] Updated weights for policy 0, policy_version 1085173 (0.0008) [2023-12-27 13:10:03,281][20347] Updated weights for policy 0, policy_version 1085183 (0.0009) [2023-12-27 13:10:03,478][20381] Updated weights for policy 1, policy_version 1084124 (0.0008) [2023-12-27 13:10:03,525][20381] Updated weights for policy 1, policy_version 1084134 (0.0010) [2023-12-27 13:10:03,579][20381] Updated weights for policy 1, policy_version 1084144 (0.0007) [2023-12-27 13:10:03,960][20347] Updated weights for policy 0, policy_version 1085193 (0.0008) [2023-12-27 13:10:04,027][20347] Updated weights for policy 0, policy_version 1085203 (0.0007) [2023-12-27 13:10:04,083][20347] Updated weights for policy 0, policy_version 1085213 (0.0007) [2023-12-27 13:10:04,143][20347] Updated weights for policy 0, policy_version 1085223 (0.0008) [2023-12-27 13:10:04,223][20381] Updated weights for policy 1, policy_version 1084154 (0.0008) [2023-12-27 13:10:04,282][20381] Updated weights for policy 1, policy_version 1084164 (0.0010) [2023-12-27 13:10:04,341][20381] Updated weights for policy 1, policy_version 1084174 (0.0010) [2023-12-27 13:10:04,400][20381] Updated weights for policy 1, policy_version 1084184 (0.0010) [2023-12-27 13:10:04,858][20347] Updated weights for policy 0, policy_version 1085233 (0.0007) [2023-12-27 13:10:04,916][20347] Updated weights for policy 0, policy_version 1085243 (0.0007) [2023-12-27 13:10:04,969][20347] Updated weights for policy 0, policy_version 1085253 (0.0007) [2023-12-27 13:10:05,008][20381] Updated weights for policy 1, policy_version 1084194 (0.0010) [2023-12-27 13:10:05,057][20381] Updated weights for policy 1, policy_version 1084204 (0.0010) [2023-12-27 13:10:05,108][20381] Updated weights for policy 1, policy_version 1084214 (0.0010) [2023-12-27 13:10:05,645][20347] Updated weights for policy 0, policy_version 1085263 (0.0007) [2023-12-27 13:10:05,694][20347] Updated weights for policy 0, policy_version 1085273 (0.0008) [2023-12-27 13:10:05,741][20381] Updated weights for policy 1, policy_version 1084224 (0.0006) [2023-12-27 13:10:05,750][20347] Updated weights for policy 0, policy_version 1085283 (0.0008) [2023-12-27 13:10:05,793][20381] Updated weights for policy 1, policy_version 1084234 (0.0010) [2023-12-27 13:10:05,854][20381] Updated weights for policy 1, policy_version 1084244 (0.0010) [2023-12-27 13:10:06,522][20347] Updated weights for policy 0, policy_version 1085293 (0.0007) [2023-12-27 13:10:06,538][20381] Updated weights for policy 1, policy_version 1084254 (0.0008) [2023-12-27 13:10:06,578][20347] Updated weights for policy 0, policy_version 1085303 (0.0005) [2023-12-27 13:10:06,595][20381] Updated weights for policy 1, policy_version 1084264 (0.0006) [2023-12-27 13:10:06,638][20347] Updated weights for policy 0, policy_version 1085313 (0.0006) [2023-12-27 13:10:06,656][20381] Updated weights for policy 1, policy_version 1084274 (0.0006) [2023-12-27 13:10:07,294][20381] Updated weights for policy 1, policy_version 1084284 (0.0006) [2023-12-27 13:10:07,337][20347] Updated weights for policy 0, policy_version 1085323 (0.0006) [2023-12-27 13:10:07,348][20381] Updated weights for policy 1, policy_version 1084294 (0.0008) [2023-12-27 13:10:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 555491328. Throughput: 0: 10435.7, 1: 10109.5. Samples: 555465732. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:07,363][19170] Avg episode reward: [(0, '990.280'), (1, '292.710')] [2023-12-27 13:10:07,386][20347] Updated weights for policy 0, policy_version 1085333 (0.0008) [2023-12-27 13:10:07,395][20381] Updated weights for policy 1, policy_version 1084304 (0.0006) [2023-12-27 13:10:07,434][20347] Updated weights for policy 0, policy_version 1085343 (0.0005) [2023-12-27 13:10:07,436][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001084312_277610496.pth... [2023-12-27 13:10:07,442][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001083128_277307392.pth [2023-12-27 13:10:07,478][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001085352_277897216.pth... [2023-12-27 13:10:07,482][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001084136_277585920.pth [2023-12-27 13:10:08,017][20381] Updated weights for policy 1, policy_version 1084314 (0.0006) [2023-12-27 13:10:08,066][20381] Updated weights for policy 1, policy_version 1084324 (0.0005) [2023-12-27 13:10:08,126][20381] Updated weights for policy 1, policy_version 1084334 (0.0005) [2023-12-27 13:10:08,136][20347] Updated weights for policy 0, policy_version 1085353 (0.0008) [2023-12-27 13:10:08,173][20381] Updated weights for policy 1, policy_version 1084344 (0.0006) [2023-12-27 13:10:08,192][20347] Updated weights for policy 0, policy_version 1085363 (0.0008) [2023-12-27 13:10:08,258][20347] Updated weights for policy 0, policy_version 1085373 (0.0009) [2023-12-27 13:10:08,320][20347] Updated weights for policy 0, policy_version 1085383 (0.0008) [2023-12-27 13:10:08,848][20381] Updated weights for policy 1, policy_version 1084354 (0.0008) [2023-12-27 13:10:08,901][20381] Updated weights for policy 1, policy_version 1084364 (0.0008) [2023-12-27 13:10:08,954][20381] Updated weights for policy 1, policy_version 1084374 (0.0007) [2023-12-27 13:10:08,954][20347] Updated weights for policy 0, policy_version 1085393 (0.0005) [2023-12-27 13:10:09,011][20347] Updated weights for policy 0, policy_version 1085403 (0.0008) [2023-12-27 13:10:09,070][20347] Updated weights for policy 0, policy_version 1085413 (0.0007) [2023-12-27 13:10:09,730][20381] Updated weights for policy 1, policy_version 1084384 (0.0010) [2023-12-27 13:10:09,732][20347] Updated weights for policy 0, policy_version 1085423 (0.0007) [2023-12-27 13:10:09,788][20381] Updated weights for policy 1, policy_version 1084394 (0.0010) [2023-12-27 13:10:09,792][20347] Updated weights for policy 0, policy_version 1085433 (0.0007) [2023-12-27 13:10:09,852][20381] Updated weights for policy 1, policy_version 1084404 (0.0008) [2023-12-27 13:10:09,859][20347] Updated weights for policy 0, policy_version 1085443 (0.0008) [2023-12-27 13:10:10,520][20381] Updated weights for policy 1, policy_version 1084414 (0.0006) [2023-12-27 13:10:10,582][20381] Updated weights for policy 1, policy_version 1084424 (0.0007) [2023-12-27 13:10:10,606][20347] Updated weights for policy 0, policy_version 1085453 (0.0007) [2023-12-27 13:10:10,638][20381] Updated weights for policy 1, policy_version 1084434 (0.0007) [2023-12-27 13:10:10,671][20347] Updated weights for policy 0, policy_version 1085463 (0.0005) [2023-12-27 13:10:10,735][20347] Updated weights for policy 0, policy_version 1085473 (0.0006) [2023-12-27 13:10:11,415][20347] Updated weights for policy 0, policy_version 1085483 (0.0007) [2023-12-27 13:10:11,433][20381] Updated weights for policy 1, policy_version 1084444 (0.0008) [2023-12-27 13:10:11,473][20347] Updated weights for policy 0, policy_version 1085493 (0.0006) [2023-12-27 13:10:11,493][20381] Updated weights for policy 1, policy_version 1084454 (0.0008) [2023-12-27 13:10:11,536][20347] Updated weights for policy 0, policy_version 1085503 (0.0007) [2023-12-27 13:10:11,558][20381] Updated weights for policy 1, policy_version 1084464 (0.0008) [2023-12-27 13:10:12,219][20347] Updated weights for policy 0, policy_version 1085513 (0.0007) [2023-12-27 13:10:12,284][20347] Updated weights for policy 0, policy_version 1085523 (0.0008) [2023-12-27 13:10:12,305][20381] Updated weights for policy 1, policy_version 1084474 (0.0008) [2023-12-27 13:10:12,345][20347] Updated weights for policy 0, policy_version 1085533 (0.0006) [2023-12-27 13:10:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 555589632. Throughput: 0: 10383.5, 1: 10143.1. Samples: 555588048. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '371.570')] [2023-12-27 13:10:12,367][20381] Updated weights for policy 1, policy_version 1084484 (0.0008) [2023-12-27 13:10:12,414][20347] Updated weights for policy 0, policy_version 1085543 (0.0007) [2023-12-27 13:10:12,428][20381] Updated weights for policy 1, policy_version 1084494 (0.0008) [2023-12-27 13:10:12,478][20381] Updated weights for policy 1, policy_version 1084504 (0.0007) [2023-12-27 13:10:13,050][20347] Updated weights for policy 0, policy_version 1085553 (0.0008) [2023-12-27 13:10:13,109][20347] Updated weights for policy 0, policy_version 1085563 (0.0008) [2023-12-27 13:10:13,163][20347] Updated weights for policy 0, policy_version 1085573 (0.0007) [2023-12-27 13:10:13,203][20381] Updated weights for policy 1, policy_version 1084514 (0.0007) [2023-12-27 13:10:13,255][20381] Updated weights for policy 1, policy_version 1084524 (0.0008) [2023-12-27 13:10:13,309][20381] Updated weights for policy 1, policy_version 1084534 (0.0008) [2023-12-27 13:10:13,873][20347] Updated weights for policy 0, policy_version 1085583 (0.0005) [2023-12-27 13:10:13,875][20381] Updated weights for policy 1, policy_version 1084544 (0.0006) [2023-12-27 13:10:13,921][20347] Updated weights for policy 0, policy_version 1085593 (0.0005) [2023-12-27 13:10:13,934][20381] Updated weights for policy 1, policy_version 1084554 (0.0006) [2023-12-27 13:10:13,979][20347] Updated weights for policy 0, policy_version 1085603 (0.0007) [2023-12-27 13:10:13,988][20381] Updated weights for policy 1, policy_version 1084564 (0.0007) [2023-12-27 13:10:14,624][20347] Updated weights for policy 0, policy_version 1085613 (0.0010) [2023-12-27 13:10:14,670][20381] Updated weights for policy 1, policy_version 1084574 (0.0006) [2023-12-27 13:10:14,676][20347] Updated weights for policy 0, policy_version 1085623 (0.0010) [2023-12-27 13:10:14,716][20381] Updated weights for policy 1, policy_version 1084584 (0.0007) [2023-12-27 13:10:14,725][20347] Updated weights for policy 0, policy_version 1085633 (0.0010) [2023-12-27 13:10:14,772][20381] Updated weights for policy 1, policy_version 1084594 (0.0010) [2023-12-27 13:10:15,461][20347] Updated weights for policy 0, policy_version 1085643 (0.0010) [2023-12-27 13:10:15,488][20381] Updated weights for policy 1, policy_version 1084604 (0.0008) [2023-12-27 13:10:15,528][20347] Updated weights for policy 0, policy_version 1085653 (0.0010) [2023-12-27 13:10:15,541][20381] Updated weights for policy 1, policy_version 1084614 (0.0009) [2023-12-27 13:10:15,580][20347] Updated weights for policy 0, policy_version 1085663 (0.0010) [2023-12-27 13:10:15,596][20381] Updated weights for policy 1, policy_version 1084624 (0.0010) [2023-12-27 13:10:16,233][20347] Updated weights for policy 0, policy_version 1085673 (0.0010) [2023-12-27 13:10:16,248][20381] Updated weights for policy 1, policy_version 1084634 (0.0010) [2023-12-27 13:10:16,288][20347] Updated weights for policy 0, policy_version 1085683 (0.0010) [2023-12-27 13:10:16,296][20381] Updated weights for policy 1, policy_version 1084644 (0.0010) [2023-12-27 13:10:16,339][20347] Updated weights for policy 0, policy_version 1085693 (0.0010) [2023-12-27 13:10:16,344][20381] Updated weights for policy 1, policy_version 1084654 (0.0010) [2023-12-27 13:10:16,388][20347] Updated weights for policy 0, policy_version 1085703 (0.0010) [2023-12-27 13:10:16,392][20381] Updated weights for policy 1, policy_version 1084664 (0.0010) [2023-12-27 13:10:17,066][20381] Updated weights for policy 1, policy_version 1084674 (0.0005) [2023-12-27 13:10:17,075][20347] Updated weights for policy 0, policy_version 1085713 (0.0010) [2023-12-27 13:10:17,122][20381] Updated weights for policy 1, policy_version 1084684 (0.0007) [2023-12-27 13:10:17,131][20347] Updated weights for policy 0, policy_version 1085723 (0.0010) [2023-12-27 13:10:17,170][20381] Updated weights for policy 1, policy_version 1084694 (0.0005) [2023-12-27 13:10:17,180][20347] Updated weights for policy 0, policy_version 1085733 (0.0010) [2023-12-27 13:10:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 555704320. Throughput: 0: 10334.7, 1: 10082.3. Samples: 555709824. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:17,363][19170] Avg episode reward: [(0, '980.590'), (1, '351.420')] [2023-12-27 13:10:17,844][20381] Updated weights for policy 1, policy_version 1084704 (0.0007) [2023-12-27 13:10:17,865][20347] Updated weights for policy 0, policy_version 1085743 (0.0010) [2023-12-27 13:10:17,892][20381] Updated weights for policy 1, policy_version 1084714 (0.0005) [2023-12-27 13:10:17,913][20347] Updated weights for policy 0, policy_version 1085753 (0.0010) [2023-12-27 13:10:17,941][20381] Updated weights for policy 1, policy_version 1084724 (0.0006) [2023-12-27 13:10:17,966][20347] Updated weights for policy 0, policy_version 1085763 (0.0010) [2023-12-27 13:10:18,661][20381] Updated weights for policy 1, policy_version 1084734 (0.0007) [2023-12-27 13:10:18,667][20347] Updated weights for policy 0, policy_version 1085773 (0.0009) [2023-12-27 13:10:18,721][20381] Updated weights for policy 1, policy_version 1084744 (0.0005) [2023-12-27 13:10:18,729][20347] Updated weights for policy 0, policy_version 1085783 (0.0008) [2023-12-27 13:10:18,780][20381] Updated weights for policy 1, policy_version 1084754 (0.0006) [2023-12-27 13:10:18,788][20347] Updated weights for policy 0, policy_version 1085793 (0.0007) [2023-12-27 13:10:19,474][20347] Updated weights for policy 0, policy_version 1085803 (0.0008) [2023-12-27 13:10:19,487][20381] Updated weights for policy 1, policy_version 1084764 (0.0007) [2023-12-27 13:10:19,540][20347] Updated weights for policy 0, policy_version 1085813 (0.0010) [2023-12-27 13:10:19,550][20381] Updated weights for policy 1, policy_version 1084774 (0.0007) [2023-12-27 13:10:19,601][20347] Updated weights for policy 0, policy_version 1085823 (0.0011) [2023-12-27 13:10:19,610][20381] Updated weights for policy 1, policy_version 1084784 (0.0006) [2023-12-27 13:10:20,295][20347] Updated weights for policy 0, policy_version 1085833 (0.0007) [2023-12-27 13:10:20,311][20381] Updated weights for policy 1, policy_version 1084794 (0.0008) [2023-12-27 13:10:20,358][20347] Updated weights for policy 0, policy_version 1085843 (0.0011) [2023-12-27 13:10:20,375][20381] Updated weights for policy 1, policy_version 1084804 (0.0008) [2023-12-27 13:10:20,415][20347] Updated weights for policy 0, policy_version 1085853 (0.0010) [2023-12-27 13:10:20,433][20381] Updated weights for policy 1, policy_version 1084814 (0.0006) [2023-12-27 13:10:20,466][20347] Updated weights for policy 0, policy_version 1085863 (0.0010) [2023-12-27 13:10:20,483][20381] Updated weights for policy 1, policy_version 1084824 (0.0006) [2023-12-27 13:10:21,157][20347] Updated weights for policy 0, policy_version 1085873 (0.0008) [2023-12-27 13:10:21,217][20347] Updated weights for policy 0, policy_version 1085883 (0.0008) [2023-12-27 13:10:21,254][20381] Updated weights for policy 1, policy_version 1084834 (0.0009) [2023-12-27 13:10:21,285][20347] Updated weights for policy 0, policy_version 1085893 (0.0008) [2023-12-27 13:10:21,314][20381] Updated weights for policy 1, policy_version 1084844 (0.0006) [2023-12-27 13:10:21,375][20381] Updated weights for policy 1, policy_version 1084854 (0.0008) [2023-12-27 13:10:22,004][20347] Updated weights for policy 0, policy_version 1085903 (0.0007) [2023-12-27 13:10:22,060][20347] Updated weights for policy 0, policy_version 1085913 (0.0007) [2023-12-27 13:10:22,110][20347] Updated weights for policy 0, policy_version 1085923 (0.0008) [2023-12-27 13:10:22,150][20381] Updated weights for policy 1, policy_version 1084864 (0.0006) [2023-12-27 13:10:22,200][20381] Updated weights for policy 1, policy_version 1084874 (0.0007) [2023-12-27 13:10:22,259][20381] Updated weights for policy 1, policy_version 1084884 (0.0008) [2023-12-27 13:10:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 555802624. Throughput: 0: 10277.1, 1: 10088.5. Samples: 555769944. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:22,363][19170] Avg episode reward: [(0, '961.080'), (1, '154.580')] [2023-12-27 13:10:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001085928_278044672.pth... [2023-12-27 13:10:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001084888_277757952.pth... [2023-12-27 13:10:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001084744_277741568.pth [2023-12-27 13:10:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001083704_277454848.pth [2023-12-27 13:10:22,870][20347] Updated weights for policy 0, policy_version 1085933 (0.0006) [2023-12-27 13:10:22,914][20381] Updated weights for policy 1, policy_version 1084894 (0.0006) [2023-12-27 13:10:22,935][20347] Updated weights for policy 0, policy_version 1085943 (0.0006) [2023-12-27 13:10:22,963][20381] Updated weights for policy 1, policy_version 1084904 (0.0006) [2023-12-27 13:10:23,007][20347] Updated weights for policy 0, policy_version 1085953 (0.0005) [2023-12-27 13:10:23,025][20381] Updated weights for policy 1, policy_version 1084914 (0.0007) [2023-12-27 13:10:23,575][20347] Updated weights for policy 0, policy_version 1085963 (0.0006) [2023-12-27 13:10:23,631][20347] Updated weights for policy 0, policy_version 1085973 (0.0006) [2023-12-27 13:10:23,637][20381] Updated weights for policy 1, policy_version 1084924 (0.0005) [2023-12-27 13:10:23,658][20291] KL-divergence is very high: 1249.2410 [2023-12-27 13:10:23,680][20347] Updated weights for policy 0, policy_version 1085983 (0.0005) [2023-12-27 13:10:23,695][20381] Updated weights for policy 1, policy_version 1084934 (0.0005) [2023-12-27 13:10:23,699][20291] KL-divergence is very high: 615.4953 [2023-12-27 13:10:23,749][20381] Updated weights for policy 1, policy_version 1084944 (0.0006) [2023-12-27 13:10:24,288][20347] Updated weights for policy 0, policy_version 1085993 (0.0008) [2023-12-27 13:10:24,343][20291] KL-divergence is very high: 1523.1539 [2023-12-27 13:10:24,344][20347] Updated weights for policy 0, policy_version 1086003 (0.0008) [2023-12-27 13:10:24,346][20381] Updated weights for policy 1, policy_version 1084954 (0.0005) [2023-12-27 13:10:24,362][20291] KL-divergence is very high: 1028.6261 [2023-12-27 13:10:24,407][20347] Updated weights for policy 0, policy_version 1086013 (0.0008) [2023-12-27 13:10:24,410][20381] Updated weights for policy 1, policy_version 1084964 (0.0007) [2023-12-27 13:10:24,462][20347] Updated weights for policy 0, policy_version 1086023 (0.0008) [2023-12-27 13:10:24,467][20381] Updated weights for policy 1, policy_version 1084974 (0.0007) [2023-12-27 13:10:24,534][20381] Updated weights for policy 1, policy_version 1084984 (0.0008) [2023-12-27 13:10:25,157][20381] Updated weights for policy 1, policy_version 1084994 (0.0008) [2023-12-27 13:10:25,195][20347] Updated weights for policy 0, policy_version 1086033 (0.0009) [2023-12-27 13:10:25,210][20381] Updated weights for policy 1, policy_version 1085004 (0.0009) [2023-12-27 13:10:25,246][20347] Updated weights for policy 0, policy_version 1086043 (0.0009) [2023-12-27 13:10:25,259][20381] Updated weights for policy 1, policy_version 1085014 (0.0007) [2023-12-27 13:10:25,296][20347] Updated weights for policy 0, policy_version 1086053 (0.0010) [2023-12-27 13:10:25,934][20381] Updated weights for policy 1, policy_version 1085024 (0.0009) [2023-12-27 13:10:25,992][20381] Updated weights for policy 1, policy_version 1085034 (0.0010) [2023-12-27 13:10:26,043][20347] Updated weights for policy 0, policy_version 1086063 (0.0008) [2023-12-27 13:10:26,045][20381] Updated weights for policy 1, policy_version 1085044 (0.0011) [2023-12-27 13:10:26,104][20347] Updated weights for policy 0, policy_version 1086073 (0.0011) [2023-12-27 13:10:26,174][20347] Updated weights for policy 0, policy_version 1086083 (0.0010) [2023-12-27 13:10:26,619][20381] Updated weights for policy 1, policy_version 1085054 (0.0007) [2023-12-27 13:10:26,666][20381] Updated weights for policy 1, policy_version 1085064 (0.0005) [2023-12-27 13:10:26,719][20381] Updated weights for policy 1, policy_version 1085074 (0.0007) [2023-12-27 13:10:26,835][20347] Updated weights for policy 0, policy_version 1086093 (0.0009) [2023-12-27 13:10:26,881][20347] Updated weights for policy 0, policy_version 1086103 (0.0005) [2023-12-27 13:10:26,929][20347] Updated weights for policy 0, policy_version 1086113 (0.0005) [2023-12-27 13:10:27,323][20381] Updated weights for policy 1, policy_version 1085084 (0.0011) [2023-12-27 13:10:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 555900928. Throughput: 0: 10254.2, 1: 10138.6. Samples: 555892004. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:27,363][19170] Avg episode reward: [(0, '931.510'), (1, '118.250')] [2023-12-27 13:10:27,381][20381] Updated weights for policy 1, policy_version 1085094 (0.0010) [2023-12-27 13:10:27,428][20381] Updated weights for policy 1, policy_version 1085104 (0.0010) [2023-12-27 13:10:27,524][20347] Updated weights for policy 0, policy_version 1086123 (0.0005) [2023-12-27 13:10:27,574][20347] Updated weights for policy 0, policy_version 1086133 (0.0005) [2023-12-27 13:10:27,627][20347] Updated weights for policy 0, policy_version 1086143 (0.0005) [2023-12-27 13:10:28,113][20381] Updated weights for policy 1, policy_version 1085114 (0.0010) [2023-12-27 13:10:28,167][20381] Updated weights for policy 1, policy_version 1085124 (0.0010) [2023-12-27 13:10:28,198][20347] Updated weights for policy 0, policy_version 1086153 (0.0005) [2023-12-27 13:10:28,228][20381] Updated weights for policy 1, policy_version 1085134 (0.0010) [2023-12-27 13:10:28,257][20347] Updated weights for policy 0, policy_version 1086163 (0.0008) [2023-12-27 13:10:28,272][20381] Updated weights for policy 1, policy_version 1085144 (0.0010) [2023-12-27 13:10:28,302][20291] KL-divergence is very high: 175.2534 [2023-12-27 13:10:28,314][20347] Updated weights for policy 0, policy_version 1086173 (0.0010) [2023-12-27 13:10:28,374][20347] Updated weights for policy 0, policy_version 1086183 (0.0010) [2023-12-27 13:10:28,959][20381] Updated weights for policy 1, policy_version 1085154 (0.0011) [2023-12-27 13:10:29,002][20381] Updated weights for policy 1, policy_version 1085164 (0.0010) [2023-12-27 13:10:29,025][20347] Updated weights for policy 0, policy_version 1086193 (0.0010) [2023-12-27 13:10:29,050][20381] Updated weights for policy 1, policy_version 1085174 (0.0010) [2023-12-27 13:10:29,080][20347] Updated weights for policy 0, policy_version 1086203 (0.0010) [2023-12-27 13:10:29,145][20347] Updated weights for policy 0, policy_version 1086213 (0.0009) [2023-12-27 13:10:29,774][20381] Updated weights for policy 1, policy_version 1085184 (0.0008) [2023-12-27 13:10:29,830][20381] Updated weights for policy 1, policy_version 1085194 (0.0010) [2023-12-27 13:10:29,835][20347] Updated weights for policy 0, policy_version 1086223 (0.0008) [2023-12-27 13:10:29,893][20347] Updated weights for policy 0, policy_version 1086233 (0.0006) [2023-12-27 13:10:29,894][20381] Updated weights for policy 1, policy_version 1085204 (0.0010) [2023-12-27 13:10:29,957][20347] Updated weights for policy 0, policy_version 1086243 (0.0008) [2023-12-27 13:10:30,579][20381] Updated weights for policy 1, policy_version 1085214 (0.0007) [2023-12-27 13:10:30,595][20347] Updated weights for policy 0, policy_version 1086253 (0.0009) [2023-12-27 13:10:30,635][20381] Updated weights for policy 1, policy_version 1085224 (0.0005) [2023-12-27 13:10:30,655][20347] Updated weights for policy 0, policy_version 1086263 (0.0010) [2023-12-27 13:10:30,686][20381] Updated weights for policy 1, policy_version 1085234 (0.0005) [2023-12-27 13:10:30,706][20347] Updated weights for policy 0, policy_version 1086273 (0.0010) [2023-12-27 13:10:31,326][20381] Updated weights for policy 1, policy_version 1085244 (0.0007) [2023-12-27 13:10:31,366][20347] Updated weights for policy 0, policy_version 1086283 (0.0010) [2023-12-27 13:10:31,393][20381] Updated weights for policy 1, policy_version 1085254 (0.0008) [2023-12-27 13:10:31,435][20347] Updated weights for policy 0, policy_version 1086293 (0.0008) [2023-12-27 13:10:31,449][20381] Updated weights for policy 1, policy_version 1085264 (0.0008) [2023-12-27 13:10:31,496][20347] Updated weights for policy 0, policy_version 1086303 (0.0010) [2023-12-27 13:10:32,070][20381] Updated weights for policy 1, policy_version 1085274 (0.0007) [2023-12-27 13:10:32,130][20381] Updated weights for policy 1, policy_version 1085284 (0.0011) [2023-12-27 13:10:32,190][20381] Updated weights for policy 1, policy_version 1085294 (0.0011) [2023-12-27 13:10:32,209][20291] KL-divergence is very high: 128.5498 [2023-12-27 13:10:32,215][20291] KL-divergence is very high: 113.7481 [2023-12-27 13:10:32,220][20347] Updated weights for policy 0, policy_version 1086314 (0.0006) [2023-12-27 13:10:32,253][20381] Updated weights for policy 1, policy_version 1085304 (0.0010) [2023-12-27 13:10:32,277][20347] Updated weights for policy 0, policy_version 1086324 (0.0006) [2023-12-27 13:10:32,338][20347] Updated weights for policy 0, policy_version 1086334 (0.0008) [2023-12-27 13:10:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 556007424. Throughput: 0: 10301.9, 1: 10156.9. Samples: 556018324. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:32,363][19170] Avg episode reward: [(0, '853.080'), (1, '214.660')] [2023-12-27 13:10:32,409][20347] Updated weights for policy 0, policy_version 1086344 (0.0007) [2023-12-27 13:10:32,935][20381] Updated weights for policy 1, policy_version 1085314 (0.0005) [2023-12-27 13:10:32,985][20381] Updated weights for policy 1, policy_version 1085324 (0.0005) [2023-12-27 13:10:33,036][20381] Updated weights for policy 1, policy_version 1085334 (0.0005) [2023-12-27 13:10:33,114][20347] Updated weights for policy 0, policy_version 1086354 (0.0008) [2023-12-27 13:10:33,172][20347] Updated weights for policy 0, policy_version 1086364 (0.0007) [2023-12-27 13:10:33,223][20347] Updated weights for policy 0, policy_version 1086374 (0.0007) [2023-12-27 13:10:33,586][20381] Updated weights for policy 1, policy_version 1085344 (0.0006) [2023-12-27 13:10:33,637][20381] Updated weights for policy 1, policy_version 1085354 (0.0005) [2023-12-27 13:10:33,693][20381] Updated weights for policy 1, policy_version 1085364 (0.0007) [2023-12-27 13:10:33,953][20347] Updated weights for policy 0, policy_version 1086384 (0.0007) [2023-12-27 13:10:34,000][20347] Updated weights for policy 0, policy_version 1086394 (0.0007) [2023-12-27 13:10:34,058][20347] Updated weights for policy 0, policy_version 1086404 (0.0007) [2023-12-27 13:10:34,322][20381] Updated weights for policy 1, policy_version 1085374 (0.0007) [2023-12-27 13:10:34,380][20381] Updated weights for policy 1, policy_version 1085384 (0.0008) [2023-12-27 13:10:34,440][20381] Updated weights for policy 1, policy_version 1085394 (0.0008) [2023-12-27 13:10:34,804][20347] Updated weights for policy 0, policy_version 1086414 (0.0007) [2023-12-27 13:10:34,853][20347] Updated weights for policy 0, policy_version 1086424 (0.0008) [2023-12-27 13:10:34,904][20347] Updated weights for policy 0, policy_version 1086434 (0.0007) [2023-12-27 13:10:35,077][20381] Updated weights for policy 1, policy_version 1085404 (0.0008) [2023-12-27 13:10:35,126][20381] Updated weights for policy 1, policy_version 1085414 (0.0008) [2023-12-27 13:10:35,173][20381] Updated weights for policy 1, policy_version 1085424 (0.0008) [2023-12-27 13:10:35,628][20347] Updated weights for policy 0, policy_version 1086444 (0.0008) [2023-12-27 13:10:35,684][20347] Updated weights for policy 0, policy_version 1086454 (0.0005) [2023-12-27 13:10:35,730][20347] Updated weights for policy 0, policy_version 1086464 (0.0005) [2023-12-27 13:10:35,881][20381] Updated weights for policy 1, policy_version 1085434 (0.0007) [2023-12-27 13:10:35,935][20381] Updated weights for policy 1, policy_version 1085444 (0.0007) [2023-12-27 13:10:35,989][20381] Updated weights for policy 1, policy_version 1085454 (0.0008) [2023-12-27 13:10:36,047][20381] Updated weights for policy 1, policy_version 1085464 (0.0007) [2023-12-27 13:10:36,368][20347] Updated weights for policy 0, policy_version 1086474 (0.0006) [2023-12-27 13:10:36,431][20347] Updated weights for policy 0, policy_version 1086484 (0.0008) [2023-12-27 13:10:36,491][20347] Updated weights for policy 0, policy_version 1086494 (0.0007) [2023-12-27 13:10:36,556][20347] Updated weights for policy 0, policy_version 1086504 (0.0008) [2023-12-27 13:10:36,767][20381] Updated weights for policy 1, policy_version 1085474 (0.0008) [2023-12-27 13:10:36,820][20381] Updated weights for policy 1, policy_version 1085484 (0.0008) [2023-12-27 13:10:36,873][20381] Updated weights for policy 1, policy_version 1085494 (0.0007) [2023-12-27 13:10:37,249][20347] Updated weights for policy 0, policy_version 1086514 (0.0008) [2023-12-27 13:10:37,306][20347] Updated weights for policy 0, policy_version 1086524 (0.0008) [2023-12-27 13:10:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 556105728. Throughput: 0: 10227.3, 1: 10230.9. Samples: 556079312. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:37,363][19170] Avg episode reward: [(0, '883.330'), (1, '474.010')] [2023-12-27 13:10:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001085496_277913600.pth... [2023-12-27 13:10:37,370][20347] Updated weights for policy 0, policy_version 1086534 (0.0009) [2023-12-27 13:10:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001084312_277610496.pth [2023-12-27 13:10:37,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001086536_278200320.pth... [2023-12-27 13:10:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001085352_277897216.pth [2023-12-27 13:10:37,492][20381] Updated weights for policy 1, policy_version 1085504 (0.0006) [2023-12-27 13:10:37,556][20381] Updated weights for policy 1, policy_version 1085514 (0.0006) [2023-12-27 13:10:37,609][20381] Updated weights for policy 1, policy_version 1085524 (0.0006) [2023-12-27 13:10:38,135][20347] Updated weights for policy 0, policy_version 1086544 (0.0006) [2023-12-27 13:10:38,138][20381] Updated weights for policy 1, policy_version 1085534 (0.0006) [2023-12-27 13:10:38,189][20347] Updated weights for policy 0, policy_version 1086554 (0.0006) [2023-12-27 13:10:38,195][20381] Updated weights for policy 1, policy_version 1085544 (0.0006) [2023-12-27 13:10:38,239][20347] Updated weights for policy 0, policy_version 1086564 (0.0006) [2023-12-27 13:10:38,252][20381] Updated weights for policy 1, policy_version 1085554 (0.0006) [2023-12-27 13:10:38,962][20347] Updated weights for policy 0, policy_version 1086574 (0.0007) [2023-12-27 13:10:38,972][20381] Updated weights for policy 1, policy_version 1085564 (0.0007) [2023-12-27 13:10:39,018][20347] Updated weights for policy 0, policy_version 1086584 (0.0007) [2023-12-27 13:10:39,021][20381] Updated weights for policy 1, policy_version 1085574 (0.0007) [2023-12-27 13:10:39,076][20347] Updated weights for policy 0, policy_version 1086594 (0.0007) [2023-12-27 13:10:39,078][20381] Updated weights for policy 1, policy_version 1085584 (0.0005) [2023-12-27 13:10:39,814][20381] Updated weights for policy 1, policy_version 1085594 (0.0006) [2023-12-27 13:10:39,851][20347] Updated weights for policy 0, policy_version 1086604 (0.0008) [2023-12-27 13:10:39,886][20381] Updated weights for policy 1, policy_version 1085604 (0.0008) [2023-12-27 13:10:39,908][20347] Updated weights for policy 0, policy_version 1086614 (0.0006) [2023-12-27 13:10:39,950][20381] Updated weights for policy 1, policy_version 1085614 (0.0009) [2023-12-27 13:10:39,974][20347] Updated weights for policy 0, policy_version 1086624 (0.0006) [2023-12-27 13:10:40,013][20381] Updated weights for policy 1, policy_version 1085624 (0.0008) [2023-12-27 13:10:40,609][20347] Updated weights for policy 0, policy_version 1086634 (0.0007) [2023-12-27 13:10:40,674][20347] Updated weights for policy 0, policy_version 1086644 (0.0007) [2023-12-27 13:10:40,688][20381] Updated weights for policy 1, policy_version 1085634 (0.0010) [2023-12-27 13:10:40,733][20347] Updated weights for policy 0, policy_version 1086654 (0.0006) [2023-12-27 13:10:40,744][20381] Updated weights for policy 1, policy_version 1085644 (0.0011) [2023-12-27 13:10:40,791][20347] Updated weights for policy 0, policy_version 1086664 (0.0006) [2023-12-27 13:10:40,800][20381] Updated weights for policy 1, policy_version 1085654 (0.0011) [2023-12-27 13:10:41,472][20347] Updated weights for policy 0, policy_version 1086674 (0.0008) [2023-12-27 13:10:41,538][20347] Updated weights for policy 0, policy_version 1086684 (0.0008) [2023-12-27 13:10:41,576][20381] Updated weights for policy 1, policy_version 1085664 (0.0011) [2023-12-27 13:10:41,603][20347] Updated weights for policy 0, policy_version 1086694 (0.0009) [2023-12-27 13:10:41,637][20381] Updated weights for policy 1, policy_version 1085674 (0.0010) [2023-12-27 13:10:41,703][20381] Updated weights for policy 1, policy_version 1085684 (0.0009) [2023-12-27 13:10:42,270][20347] Updated weights for policy 0, policy_version 1086704 (0.0006) [2023-12-27 13:10:42,335][20347] Updated weights for policy 0, policy_version 1086714 (0.0011) [2023-12-27 13:10:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 556204032. Throughput: 0: 10120.1, 1: 10267.9. Samples: 556201568. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:42,363][19170] Avg episode reward: [(0, '971.390'), (1, '413.870')] [2023-12-27 13:10:42,397][20347] Updated weights for policy 0, policy_version 1086724 (0.0007) [2023-12-27 13:10:42,452][20381] Updated weights for policy 1, policy_version 1085694 (0.0007) [2023-12-27 13:10:42,513][20381] Updated weights for policy 1, policy_version 1085704 (0.0008) [2023-12-27 13:10:42,578][20381] Updated weights for policy 1, policy_version 1085714 (0.0008) [2023-12-27 13:10:43,064][20347] Updated weights for policy 0, policy_version 1086734 (0.0007) [2023-12-27 13:10:43,133][20347] Updated weights for policy 0, policy_version 1086744 (0.0009) [2023-12-27 13:10:43,203][20347] Updated weights for policy 0, policy_version 1086754 (0.0011) [2023-12-27 13:10:43,248][20381] Updated weights for policy 1, policy_version 1085724 (0.0007) [2023-12-27 13:10:43,307][20381] Updated weights for policy 1, policy_version 1085734 (0.0007) [2023-12-27 13:10:43,362][20381] Updated weights for policy 1, policy_version 1085744 (0.0007) [2023-12-27 13:10:43,789][20347] Updated weights for policy 0, policy_version 1086764 (0.0010) [2023-12-27 13:10:43,857][20347] Updated weights for policy 0, policy_version 1086774 (0.0010) [2023-12-27 13:10:43,926][20347] Updated weights for policy 0, policy_version 1086784 (0.0010) [2023-12-27 13:10:43,984][20381] Updated weights for policy 1, policy_version 1085754 (0.0007) [2023-12-27 13:10:44,044][20381] Updated weights for policy 1, policy_version 1085764 (0.0006) [2023-12-27 13:10:44,105][20381] Updated weights for policy 1, policy_version 1085774 (0.0007) [2023-12-27 13:10:44,155][20381] Updated weights for policy 1, policy_version 1085784 (0.0005) [2023-12-27 13:10:44,544][20347] Updated weights for policy 0, policy_version 1086794 (0.0010) [2023-12-27 13:10:44,604][20347] Updated weights for policy 0, policy_version 1086804 (0.0010) [2023-12-27 13:10:44,667][20347] Updated weights for policy 0, policy_version 1086814 (0.0009) [2023-12-27 13:10:44,713][20347] Updated weights for policy 0, policy_version 1086824 (0.0005) [2023-12-27 13:10:44,797][20381] Updated weights for policy 1, policy_version 1085794 (0.0007) [2023-12-27 13:10:44,854][20381] Updated weights for policy 1, policy_version 1085804 (0.0008) [2023-12-27 13:10:44,907][20381] Updated weights for policy 1, policy_version 1085814 (0.0006) [2023-12-27 13:10:45,419][20347] Updated weights for policy 0, policy_version 1086834 (0.0007) [2023-12-27 13:10:45,475][20347] Updated weights for policy 0, policy_version 1086844 (0.0007) [2023-12-27 13:10:45,525][20381] Updated weights for policy 1, policy_version 1085824 (0.0005) [2023-12-27 13:10:45,531][20347] Updated weights for policy 0, policy_version 1086854 (0.0006) [2023-12-27 13:10:45,590][20381] Updated weights for policy 1, policy_version 1085834 (0.0007) [2023-12-27 13:10:45,636][20381] Updated weights for policy 1, policy_version 1085844 (0.0007) [2023-12-27 13:10:46,190][20347] Updated weights for policy 0, policy_version 1086864 (0.0007) [2023-12-27 13:10:46,226][20381] Updated weights for policy 1, policy_version 1085854 (0.0007) [2023-12-27 13:10:46,253][20347] Updated weights for policy 0, policy_version 1086874 (0.0010) [2023-12-27 13:10:46,282][20381] Updated weights for policy 1, policy_version 1085864 (0.0010) [2023-12-27 13:10:46,306][20347] Updated weights for policy 0, policy_version 1086884 (0.0010) [2023-12-27 13:10:46,340][20381] Updated weights for policy 1, policy_version 1085874 (0.0008) [2023-12-27 13:10:46,919][20347] Updated weights for policy 0, policy_version 1086894 (0.0007) [2023-12-27 13:10:46,951][20381] Updated weights for policy 1, policy_version 1085884 (0.0007) [2023-12-27 13:10:46,964][20347] Updated weights for policy 0, policy_version 1086904 (0.0005) [2023-12-27 13:10:47,002][20381] Updated weights for policy 1, policy_version 1085894 (0.0005) [2023-12-27 13:10:47,014][20347] Updated weights for policy 0, policy_version 1086914 (0.0006) [2023-12-27 13:10:47,061][20381] Updated weights for policy 1, policy_version 1085904 (0.0007) [2023-12-27 13:10:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 556318720. Throughput: 0: 10133.1, 1: 10368.6. Samples: 556327172. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:47,364][19170] Avg episode reward: [(0, '960.840'), (1, '351.300')] [2023-12-27 13:10:47,689][20347] Updated weights for policy 0, policy_version 1086924 (0.0007) [2023-12-27 13:10:47,741][20347] Updated weights for policy 0, policy_version 1086934 (0.0007) [2023-12-27 13:10:47,752][20381] Updated weights for policy 1, policy_version 1085914 (0.0007) [2023-12-27 13:10:47,800][20347] Updated weights for policy 0, policy_version 1086944 (0.0010) [2023-12-27 13:10:47,813][20381] Updated weights for policy 1, policy_version 1085924 (0.0006) [2023-12-27 13:10:47,866][20381] Updated weights for policy 1, policy_version 1085934 (0.0006) [2023-12-27 13:10:47,923][20381] Updated weights for policy 1, policy_version 1085944 (0.0008) [2023-12-27 13:10:48,482][20347] Updated weights for policy 0, policy_version 1086954 (0.0010) [2023-12-27 13:10:48,545][20347] Updated weights for policy 0, policy_version 1086964 (0.0008) [2023-12-27 13:10:48,570][20381] Updated weights for policy 1, policy_version 1085954 (0.0006) [2023-12-27 13:10:48,597][20347] Updated weights for policy 0, policy_version 1086974 (0.0010) [2023-12-27 13:10:48,625][20381] Updated weights for policy 1, policy_version 1085964 (0.0005) [2023-12-27 13:10:48,651][20347] Updated weights for policy 0, policy_version 1086984 (0.0010) [2023-12-27 13:10:48,688][20381] Updated weights for policy 1, policy_version 1085974 (0.0007) [2023-12-27 13:10:49,358][20347] Updated weights for policy 0, policy_version 1086994 (0.0010) [2023-12-27 13:10:49,410][20381] Updated weights for policy 1, policy_version 1085984 (0.0008) [2023-12-27 13:10:49,421][20347] Updated weights for policy 0, policy_version 1087004 (0.0009) [2023-12-27 13:10:49,464][20381] Updated weights for policy 1, policy_version 1085994 (0.0007) [2023-12-27 13:10:49,477][20347] Updated weights for policy 0, policy_version 1087014 (0.0007) [2023-12-27 13:10:49,521][20381] Updated weights for policy 1, policy_version 1086004 (0.0008) [2023-12-27 13:10:50,191][20381] Updated weights for policy 1, policy_version 1086014 (0.0008) [2023-12-27 13:10:50,208][20347] Updated weights for policy 0, policy_version 1087024 (0.0006) [2023-12-27 13:10:50,246][20381] Updated weights for policy 1, policy_version 1086024 (0.0005) [2023-12-27 13:10:50,271][20347] Updated weights for policy 0, policy_version 1087034 (0.0007) [2023-12-27 13:10:50,307][20381] Updated weights for policy 1, policy_version 1086034 (0.0006) [2023-12-27 13:10:50,329][20347] Updated weights for policy 0, policy_version 1087044 (0.0008) [2023-12-27 13:10:50,969][20381] Updated weights for policy 1, policy_version 1086044 (0.0006) [2023-12-27 13:10:51,020][20347] Updated weights for policy 0, policy_version 1087054 (0.0008) [2023-12-27 13:10:51,029][20381] Updated weights for policy 1, policy_version 1086054 (0.0007) [2023-12-27 13:10:51,086][20347] Updated weights for policy 0, policy_version 1087064 (0.0008) [2023-12-27 13:10:51,097][20381] Updated weights for policy 1, policy_version 1086064 (0.0007) [2023-12-27 13:10:51,148][20347] Updated weights for policy 0, policy_version 1087074 (0.0007) [2023-12-27 13:10:51,807][20381] Updated weights for policy 1, policy_version 1086074 (0.0007) [2023-12-27 13:10:51,861][20381] Updated weights for policy 1, policy_version 1086084 (0.0008) [2023-12-27 13:10:51,891][20347] Updated weights for policy 0, policy_version 1087084 (0.0009) [2023-12-27 13:10:51,917][20381] Updated weights for policy 1, policy_version 1086094 (0.0007) [2023-12-27 13:10:51,942][20347] Updated weights for policy 0, policy_version 1087094 (0.0008) [2023-12-27 13:10:51,966][20381] Updated weights for policy 1, policy_version 1086104 (0.0007) [2023-12-27 13:10:51,991][20347] Updated weights for policy 0, policy_version 1087104 (0.0008) [2023-12-27 13:10:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 556417024. Throughput: 0: 10147.1, 1: 10344.1. Samples: 556387836. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:52,363][19170] Avg episode reward: [(0, '980.440'), (1, '399.170')] [2023-12-27 13:10:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001086104_278069248.pth... [2023-12-27 13:10:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001087112_278347776.pth... [2023-12-27 13:10:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001085928_278044672.pth [2023-12-27 13:10:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001084888_277757952.pth [2023-12-27 13:10:52,676][20347] Updated weights for policy 0, policy_version 1087114 (0.0008) [2023-12-27 13:10:52,721][20381] Updated weights for policy 1, policy_version 1086114 (0.0006) [2023-12-27 13:10:52,726][20347] Updated weights for policy 0, policy_version 1087124 (0.0006) [2023-12-27 13:10:52,777][20381] Updated weights for policy 1, policy_version 1086124 (0.0006) [2023-12-27 13:10:52,783][20347] Updated weights for policy 0, policy_version 1087134 (0.0006) [2023-12-27 13:10:52,835][20381] Updated weights for policy 1, policy_version 1086134 (0.0006) [2023-12-27 13:10:52,839][20347] Updated weights for policy 0, policy_version 1087144 (0.0006) [2023-12-27 13:10:53,444][20381] Updated weights for policy 1, policy_version 1086144 (0.0008) [2023-12-27 13:10:53,489][20381] Updated weights for policy 1, policy_version 1086154 (0.0007) [2023-12-27 13:10:53,533][20381] Updated weights for policy 1, policy_version 1086164 (0.0006) [2023-12-27 13:10:53,569][20347] Updated weights for policy 0, policy_version 1087154 (0.0006) [2023-12-27 13:10:53,629][20347] Updated weights for policy 0, policy_version 1087164 (0.0007) [2023-12-27 13:10:53,681][20347] Updated weights for policy 0, policy_version 1087174 (0.0007) [2023-12-27 13:10:54,252][20381] Updated weights for policy 1, policy_version 1086174 (0.0007) [2023-12-27 13:10:54,307][20381] Updated weights for policy 1, policy_version 1086184 (0.0008) [2023-12-27 13:10:54,350][20347] Updated weights for policy 0, policy_version 1087184 (0.0006) [2023-12-27 13:10:54,363][20381] Updated weights for policy 1, policy_version 1086194 (0.0007) [2023-12-27 13:10:54,406][20347] Updated weights for policy 0, policy_version 1087194 (0.0006) [2023-12-27 13:10:54,472][20347] Updated weights for policy 0, policy_version 1087205 (0.0008) [2023-12-27 13:10:54,984][20381] Updated weights for policy 1, policy_version 1086204 (0.0006) [2023-12-27 13:10:55,043][20381] Updated weights for policy 1, policy_version 1086214 (0.0011) [2023-12-27 13:10:55,108][20381] Updated weights for policy 1, policy_version 1086224 (0.0011) [2023-12-27 13:10:55,226][20347] Updated weights for policy 0, policy_version 1087215 (0.0006) [2023-12-27 13:10:55,285][20347] Updated weights for policy 0, policy_version 1087225 (0.0005) [2023-12-27 13:10:55,339][20347] Updated weights for policy 0, policy_version 1087235 (0.0006) [2023-12-27 13:10:55,746][20381] Updated weights for policy 1, policy_version 1086234 (0.0011) [2023-12-27 13:10:55,812][20381] Updated weights for policy 1, policy_version 1086244 (0.0009) [2023-12-27 13:10:55,874][20381] Updated weights for policy 1, policy_version 1086254 (0.0008) [2023-12-27 13:10:55,919][20347] Updated weights for policy 0, policy_version 1087245 (0.0006) [2023-12-27 13:10:55,935][20381] Updated weights for policy 1, policy_version 1086264 (0.0007) [2023-12-27 13:10:55,971][20347] Updated weights for policy 0, policy_version 1087255 (0.0005) [2023-12-27 13:10:56,023][20347] Updated weights for policy 0, policy_version 1087265 (0.0006) [2023-12-27 13:10:56,533][20381] Updated weights for policy 1, policy_version 1086274 (0.0011) [2023-12-27 13:10:56,580][20381] Updated weights for policy 1, policy_version 1086284 (0.0010) [2023-12-27 13:10:56,624][20381] Updated weights for policy 1, policy_version 1086294 (0.0010) [2023-12-27 13:10:56,659][20347] Updated weights for policy 0, policy_version 1087275 (0.0010) [2023-12-27 13:10:56,713][20347] Updated weights for policy 0, policy_version 1087285 (0.0010) [2023-12-27 13:10:56,766][20347] Updated weights for policy 0, policy_version 1087295 (0.0010) [2023-12-27 13:10:57,319][20381] Updated weights for policy 1, policy_version 1086304 (0.0010) [2023-12-27 13:10:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 556515328. Throughput: 0: 10150.7, 1: 10345.7. Samples: 556510384. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:10:57,363][19170] Avg episode reward: [(0, '980.460'), (1, '346.810')] [2023-12-27 13:10:57,376][20381] Updated weights for policy 1, policy_version 1086314 (0.0010) [2023-12-27 13:10:57,424][20347] Updated weights for policy 0, policy_version 1087305 (0.0010) [2023-12-27 13:10:57,435][20381] Updated weights for policy 1, policy_version 1086324 (0.0010) [2023-12-27 13:10:57,481][20347] Updated weights for policy 0, policy_version 1087315 (0.0010) [2023-12-27 13:10:57,534][20347] Updated weights for policy 0, policy_version 1087325 (0.0010) [2023-12-27 13:10:57,585][20347] Updated weights for policy 0, policy_version 1087335 (0.0010) [2023-12-27 13:10:58,094][20381] Updated weights for policy 1, policy_version 1086334 (0.0008) [2023-12-27 13:10:58,148][20381] Updated weights for policy 1, policy_version 1086344 (0.0007) [2023-12-27 13:10:58,211][20381] Updated weights for policy 1, policy_version 1086354 (0.0008) [2023-12-27 13:10:58,235][20347] Updated weights for policy 0, policy_version 1087345 (0.0010) [2023-12-27 13:10:58,292][20347] Updated weights for policy 0, policy_version 1087355 (0.0011) [2023-12-27 13:10:58,364][20347] Updated weights for policy 0, policy_version 1087365 (0.0010) [2023-12-27 13:10:58,945][20381] Updated weights for policy 1, policy_version 1086364 (0.0007) [2023-12-27 13:10:59,001][20381] Updated weights for policy 1, policy_version 1086374 (0.0008) [2023-12-27 13:10:59,055][20381] Updated weights for policy 1, policy_version 1086384 (0.0007) [2023-12-27 13:10:59,129][20347] Updated weights for policy 0, policy_version 1087375 (0.0008) [2023-12-27 13:10:59,187][20347] Updated weights for policy 0, policy_version 1087385 (0.0007) [2023-12-27 13:10:59,251][20347] Updated weights for policy 0, policy_version 1087395 (0.0008) [2023-12-27 13:10:59,792][20381] Updated weights for policy 1, policy_version 1086394 (0.0007) [2023-12-27 13:10:59,856][20381] Updated weights for policy 1, policy_version 1086404 (0.0008) [2023-12-27 13:10:59,916][20381] Updated weights for policy 1, policy_version 1086414 (0.0006) [2023-12-27 13:10:59,978][20381] Updated weights for policy 1, policy_version 1086424 (0.0006) [2023-12-27 13:10:59,988][20347] Updated weights for policy 0, policy_version 1087405 (0.0008) [2023-12-27 13:11:00,050][20347] Updated weights for policy 0, policy_version 1087415 (0.0008) [2023-12-27 13:11:00,111][20347] Updated weights for policy 0, policy_version 1087425 (0.0008) [2023-12-27 13:11:00,613][20381] Updated weights for policy 1, policy_version 1086434 (0.0008) [2023-12-27 13:11:00,665][20381] Updated weights for policy 1, policy_version 1086444 (0.0008) [2023-12-27 13:11:00,710][20381] Updated weights for policy 1, policy_version 1086454 (0.0008) [2023-12-27 13:11:00,799][20347] Updated weights for policy 0, policy_version 1087435 (0.0008) [2023-12-27 13:11:00,850][20347] Updated weights for policy 0, policy_version 1087445 (0.0010) [2023-12-27 13:11:00,906][20347] Updated weights for policy 0, policy_version 1087455 (0.0010) [2023-12-27 13:11:01,388][20381] Updated weights for policy 1, policy_version 1086464 (0.0008) [2023-12-27 13:11:01,444][20381] Updated weights for policy 1, policy_version 1086474 (0.0007) [2023-12-27 13:11:01,500][20381] Updated weights for policy 1, policy_version 1086484 (0.0008) [2023-12-27 13:11:01,587][20347] Updated weights for policy 0, policy_version 1087465 (0.0007) [2023-12-27 13:11:01,648][20347] Updated weights for policy 0, policy_version 1087475 (0.0008) [2023-12-27 13:11:01,708][20347] Updated weights for policy 0, policy_version 1087485 (0.0007) [2023-12-27 13:11:01,773][20347] Updated weights for policy 0, policy_version 1087495 (0.0008) [2023-12-27 13:11:02,205][20381] Updated weights for policy 1, policy_version 1086494 (0.0007) [2023-12-27 13:11:02,267][20381] Updated weights for policy 1, policy_version 1086504 (0.0007) [2023-12-27 13:11:02,324][20381] Updated weights for policy 1, policy_version 1086514 (0.0006) [2023-12-27 13:11:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 556621824. Throughput: 0: 10113.3, 1: 10333.1. Samples: 556629916. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:02,363][19170] Avg episode reward: [(0, '970.660'), (1, '169.470')] [2023-12-27 13:11:02,501][20347] Updated weights for policy 0, policy_version 1087505 (0.0008) [2023-12-27 13:11:02,552][20347] Updated weights for policy 0, policy_version 1087515 (0.0008) [2023-12-27 13:11:02,605][20347] Updated weights for policy 0, policy_version 1087525 (0.0008) [2023-12-27 13:11:02,937][20381] Updated weights for policy 1, policy_version 1086524 (0.0007) [2023-12-27 13:11:02,983][20381] Updated weights for policy 1, policy_version 1086534 (0.0007) [2023-12-27 13:11:03,050][20381] Updated weights for policy 1, policy_version 1086544 (0.0008) [2023-12-27 13:11:03,349][20347] Updated weights for policy 0, policy_version 1087535 (0.0008) [2023-12-27 13:11:03,406][20347] Updated weights for policy 0, policy_version 1087545 (0.0008) [2023-12-27 13:11:03,459][20347] Updated weights for policy 0, policy_version 1087555 (0.0008) [2023-12-27 13:11:03,474][20291] KL-divergence is very high: 267.6204 [2023-12-27 13:11:03,688][20381] Updated weights for policy 1, policy_version 1086554 (0.0005) [2023-12-27 13:11:03,744][20381] Updated weights for policy 1, policy_version 1086564 (0.0008) [2023-12-27 13:11:03,809][20381] Updated weights for policy 1, policy_version 1086574 (0.0007) [2023-12-27 13:11:03,874][20381] Updated weights for policy 1, policy_version 1086584 (0.0008) [2023-12-27 13:11:04,227][20347] Updated weights for policy 0, policy_version 1087565 (0.0008) [2023-12-27 13:11:04,283][20347] Updated weights for policy 0, policy_version 1087575 (0.0008) [2023-12-27 13:11:04,350][20347] Updated weights for policy 0, policy_version 1087585 (0.0008) [2023-12-27 13:11:04,532][20381] Updated weights for policy 1, policy_version 1086594 (0.0008) [2023-12-27 13:11:04,580][20381] Updated weights for policy 1, policy_version 1086604 (0.0008) [2023-12-27 13:11:04,638][20381] Updated weights for policy 1, policy_version 1086614 (0.0008) [2023-12-27 13:11:04,950][20347] Updated weights for policy 0, policy_version 1087595 (0.0007) [2023-12-27 13:11:05,006][20347] Updated weights for policy 0, policy_version 1087605 (0.0006) [2023-12-27 13:11:05,060][20347] Updated weights for policy 0, policy_version 1087615 (0.0007) [2023-12-27 13:11:05,361][20381] Updated weights for policy 1, policy_version 1086624 (0.0008) [2023-12-27 13:11:05,415][20381] Updated weights for policy 1, policy_version 1086634 (0.0008) [2023-12-27 13:11:05,461][20381] Updated weights for policy 1, policy_version 1086644 (0.0008) [2023-12-27 13:11:05,721][20347] Updated weights for policy 0, policy_version 1087625 (0.0007) [2023-12-27 13:11:05,772][20347] Updated weights for policy 0, policy_version 1087635 (0.0007) [2023-12-27 13:11:05,822][20347] Updated weights for policy 0, policy_version 1087645 (0.0007) [2023-12-27 13:11:05,878][20347] Updated weights for policy 0, policy_version 1087655 (0.0008) [2023-12-27 13:11:06,188][20381] Updated weights for policy 1, policy_version 1086654 (0.0007) [2023-12-27 13:11:06,250][20381] Updated weights for policy 1, policy_version 1086664 (0.0009) [2023-12-27 13:11:06,310][20381] Updated weights for policy 1, policy_version 1086674 (0.0008) [2023-12-27 13:11:06,544][20347] Updated weights for policy 0, policy_version 1087665 (0.0011) [2023-12-27 13:11:06,604][20347] Updated weights for policy 0, policy_version 1087675 (0.0008) [2023-12-27 13:11:06,663][20347] Updated weights for policy 0, policy_version 1087685 (0.0008) [2023-12-27 13:11:07,022][20381] Updated weights for policy 1, policy_version 1086684 (0.0008) [2023-12-27 13:11:07,073][20381] Updated weights for policy 1, policy_version 1086694 (0.0008) [2023-12-27 13:11:07,123][20381] Updated weights for policy 1, policy_version 1086704 (0.0007) [2023-12-27 13:11:07,319][20347] Updated weights for policy 0, policy_version 1087695 (0.0008) [2023-12-27 13:11:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 556720128. Throughput: 0: 10103.0, 1: 10364.4. Samples: 556690972. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:07,363][19170] Avg episode reward: [(0, '931.630'), (1, '315.150')] [2023-12-27 13:11:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001086712_278224896.pth... [2023-12-27 13:11:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001085496_277913600.pth [2023-12-27 13:11:07,376][20347] Updated weights for policy 0, policy_version 1087705 (0.0007) [2023-12-27 13:11:07,439][20347] Updated weights for policy 0, policy_version 1087715 (0.0005) [2023-12-27 13:11:07,471][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001087720_278503424.pth... [2023-12-27 13:11:07,476][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001086536_278200320.pth [2023-12-27 13:11:07,795][20381] Updated weights for policy 1, policy_version 1086714 (0.0009) [2023-12-27 13:11:07,846][20381] Updated weights for policy 1, policy_version 1086724 (0.0010) [2023-12-27 13:11:07,895][20381] Updated weights for policy 1, policy_version 1086734 (0.0010) [2023-12-27 13:11:07,944][20381] Updated weights for policy 1, policy_version 1086744 (0.0007) [2023-12-27 13:11:08,022][20347] Updated weights for policy 0, policy_version 1087725 (0.0006) [2023-12-27 13:11:08,075][20347] Updated weights for policy 0, policy_version 1087735 (0.0007) [2023-12-27 13:11:08,142][20347] Updated weights for policy 0, policy_version 1087745 (0.0006) [2023-12-27 13:11:08,584][20381] Updated weights for policy 1, policy_version 1086754 (0.0008) [2023-12-27 13:11:08,639][20381] Updated weights for policy 1, policy_version 1086764 (0.0008) [2023-12-27 13:11:08,686][20381] Updated weights for policy 1, policy_version 1086774 (0.0008) [2023-12-27 13:11:08,811][20347] Updated weights for policy 0, policy_version 1087755 (0.0006) [2023-12-27 13:11:08,853][20291] KL-divergence is very high: 105.9681 [2023-12-27 13:11:08,865][20291] KL-divergence is very high: 118.0195 [2023-12-27 13:11:08,870][20347] Updated weights for policy 0, policy_version 1087765 (0.0008) [2023-12-27 13:11:08,930][20347] Updated weights for policy 0, policy_version 1087775 (0.0008) [2023-12-27 13:11:09,414][20381] Updated weights for policy 1, policy_version 1086784 (0.0008) [2023-12-27 13:11:09,468][20381] Updated weights for policy 1, policy_version 1086794 (0.0006) [2023-12-27 13:11:09,520][20381] Updated weights for policy 1, policy_version 1086804 (0.0006) [2023-12-27 13:11:09,632][20347] Updated weights for policy 0, policy_version 1087785 (0.0007) [2023-12-27 13:11:09,691][20347] Updated weights for policy 0, policy_version 1087795 (0.0008) [2023-12-27 13:11:09,752][20347] Updated weights for policy 0, policy_version 1087805 (0.0008) [2023-12-27 13:11:09,814][20347] Updated weights for policy 0, policy_version 1087815 (0.0008) [2023-12-27 13:11:10,249][20381] Updated weights for policy 1, policy_version 1086814 (0.0006) [2023-12-27 13:11:10,313][20381] Updated weights for policy 1, policy_version 1086824 (0.0007) [2023-12-27 13:11:10,375][20381] Updated weights for policy 1, policy_version 1086834 (0.0008) [2023-12-27 13:11:10,510][20347] Updated weights for policy 0, policy_version 1087825 (0.0008) [2023-12-27 13:11:10,567][20347] Updated weights for policy 0, policy_version 1087835 (0.0009) [2023-12-27 13:11:10,616][20347] Updated weights for policy 0, policy_version 1087845 (0.0007) [2023-12-27 13:11:10,995][20381] Updated weights for policy 1, policy_version 1086844 (0.0007) [2023-12-27 13:11:11,061][20381] Updated weights for policy 1, policy_version 1086854 (0.0007) [2023-12-27 13:11:11,130][20381] Updated weights for policy 1, policy_version 1086864 (0.0007) [2023-12-27 13:11:11,335][20347] Updated weights for policy 0, policy_version 1087855 (0.0007) [2023-12-27 13:11:11,408][20347] Updated weights for policy 0, policy_version 1087865 (0.0007) [2023-12-27 13:11:11,465][20347] Updated weights for policy 0, policy_version 1087875 (0.0008) [2023-12-27 13:11:11,811][20381] Updated weights for policy 1, policy_version 1086874 (0.0007) [2023-12-27 13:11:11,858][20381] Updated weights for policy 1, policy_version 1086884 (0.0008) [2023-12-27 13:11:11,918][20381] Updated weights for policy 1, policy_version 1086894 (0.0008) [2023-12-27 13:11:11,978][20381] Updated weights for policy 1, policy_version 1086904 (0.0008) [2023-12-27 13:11:12,232][20347] Updated weights for policy 0, policy_version 1087885 (0.0008) [2023-12-27 13:11:12,291][20347] Updated weights for policy 0, policy_version 1087895 (0.0010) [2023-12-27 13:11:12,355][20347] Updated weights for policy 0, policy_version 1087905 (0.0007) [2023-12-27 13:11:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 556818432. Throughput: 0: 10129.5, 1: 10344.7. Samples: 556813344. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:12,363][19170] Avg episode reward: [(0, '882.630'), (1, '448.510')] [2023-12-27 13:11:12,635][20381] Updated weights for policy 1, policy_version 1086914 (0.0008) [2023-12-27 13:11:12,691][20381] Updated weights for policy 1, policy_version 1086924 (0.0005) [2023-12-27 13:11:12,751][20381] Updated weights for policy 1, policy_version 1086934 (0.0009) [2023-12-27 13:11:13,080][20347] Updated weights for policy 0, policy_version 1087915 (0.0007) [2023-12-27 13:11:13,140][20347] Updated weights for policy 0, policy_version 1087925 (0.0008) [2023-12-27 13:11:13,196][20347] Updated weights for policy 0, policy_version 1087935 (0.0006) [2023-12-27 13:11:13,336][20381] Updated weights for policy 1, policy_version 1086944 (0.0006) [2023-12-27 13:11:13,393][20381] Updated weights for policy 1, policy_version 1086954 (0.0006) [2023-12-27 13:11:13,448][20381] Updated weights for policy 1, policy_version 1086964 (0.0008) [2023-12-27 13:11:13,896][20347] Updated weights for policy 0, policy_version 1087945 (0.0008) [2023-12-27 13:11:13,946][20347] Updated weights for policy 0, policy_version 1087955 (0.0008) [2023-12-27 13:11:14,010][20347] Updated weights for policy 0, policy_version 1087965 (0.0008) [2023-12-27 13:11:14,060][20347] Updated weights for policy 0, policy_version 1087975 (0.0007) [2023-12-27 13:11:14,082][20381] Updated weights for policy 1, policy_version 1086974 (0.0007) [2023-12-27 13:11:14,135][20381] Updated weights for policy 1, policy_version 1086984 (0.0007) [2023-12-27 13:11:14,191][20381] Updated weights for policy 1, policy_version 1086994 (0.0008) [2023-12-27 13:11:14,779][20347] Updated weights for policy 0, policy_version 1087985 (0.0007) [2023-12-27 13:11:14,843][20347] Updated weights for policy 0, policy_version 1087995 (0.0007) [2023-12-27 13:11:14,902][20347] Updated weights for policy 0, policy_version 1088005 (0.0008) [2023-12-27 13:11:14,906][20381] Updated weights for policy 1, policy_version 1087004 (0.0009) [2023-12-27 13:11:14,969][20381] Updated weights for policy 1, policy_version 1087014 (0.0011) [2023-12-27 13:11:15,037][20381] Updated weights for policy 1, policy_version 1087024 (0.0011) [2023-12-27 13:11:15,639][20347] Updated weights for policy 0, policy_version 1088015 (0.0008) [2023-12-27 13:11:15,697][20347] Updated weights for policy 0, policy_version 1088025 (0.0007) [2023-12-27 13:11:15,727][20381] Updated weights for policy 1, policy_version 1087034 (0.0011) [2023-12-27 13:11:15,753][20347] Updated weights for policy 0, policy_version 1088035 (0.0009) [2023-12-27 13:11:15,779][20381] Updated weights for policy 1, policy_version 1087044 (0.0010) [2023-12-27 13:11:15,824][20381] Updated weights for policy 1, policy_version 1087054 (0.0010) [2023-12-27 13:11:15,878][20381] Updated weights for policy 1, policy_version 1087064 (0.0010) [2023-12-27 13:11:16,448][20347] Updated weights for policy 0, policy_version 1088045 (0.0008) [2023-12-27 13:11:16,501][20347] Updated weights for policy 0, policy_version 1088055 (0.0007) [2023-12-27 13:11:16,556][20347] Updated weights for policy 0, policy_version 1088065 (0.0008) [2023-12-27 13:11:16,567][20381] Updated weights for policy 1, policy_version 1087074 (0.0010) [2023-12-27 13:11:16,614][20381] Updated weights for policy 1, policy_version 1087084 (0.0008) [2023-12-27 13:11:16,664][20381] Updated weights for policy 1, policy_version 1087094 (0.0006) [2023-12-27 13:11:17,264][20347] Updated weights for policy 0, policy_version 1088075 (0.0006) [2023-12-27 13:11:17,289][20381] Updated weights for policy 1, policy_version 1087104 (0.0005) [2023-12-27 13:11:17,325][20347] Updated weights for policy 0, policy_version 1088085 (0.0006) [2023-12-27 13:11:17,349][20381] Updated weights for policy 1, policy_version 1087114 (0.0006) [2023-12-27 13:11:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 556916736. Throughput: 0: 10018.2, 1: 10331.2. Samples: 556934048. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:17,363][19170] Avg episode reward: [(0, '853.160'), (1, '45.030')] [2023-12-27 13:11:17,388][20347] Updated weights for policy 0, policy_version 1088095 (0.0006) [2023-12-27 13:11:17,409][20381] Updated weights for policy 1, policy_version 1087124 (0.0005) [2023-12-27 13:11:17,946][20381] Updated weights for policy 1, policy_version 1087134 (0.0006) [2023-12-27 13:11:17,999][20347] Updated weights for policy 0, policy_version 1088105 (0.0006) [2023-12-27 13:11:18,010][20381] Updated weights for policy 1, policy_version 1087144 (0.0007) [2023-12-27 13:11:18,059][20347] Updated weights for policy 0, policy_version 1088115 (0.0008) [2023-12-27 13:11:18,060][20381] Updated weights for policy 1, policy_version 1087154 (0.0006) [2023-12-27 13:11:18,119][20347] Updated weights for policy 0, policy_version 1088125 (0.0007) [2023-12-27 13:11:18,177][20347] Updated weights for policy 0, policy_version 1088135 (0.0008) [2023-12-27 13:11:18,669][20381] Updated weights for policy 1, policy_version 1087164 (0.0007) [2023-12-27 13:11:18,729][20381] Updated weights for policy 1, policy_version 1087174 (0.0008) [2023-12-27 13:11:18,787][20381] Updated weights for policy 1, policy_version 1087184 (0.0006) [2023-12-27 13:11:18,828][20347] Updated weights for policy 0, policy_version 1088145 (0.0006) [2023-12-27 13:11:18,892][20347] Updated weights for policy 0, policy_version 1088155 (0.0007) [2023-12-27 13:11:18,957][20347] Updated weights for policy 0, policy_version 1088165 (0.0005) [2023-12-27 13:11:19,438][20381] Updated weights for policy 1, policy_version 1087194 (0.0007) [2023-12-27 13:11:19,499][20381] Updated weights for policy 1, policy_version 1087204 (0.0007) [2023-12-27 13:11:19,567][20347] Updated weights for policy 0, policy_version 1088175 (0.0007) [2023-12-27 13:11:19,582][20381] Updated weights for policy 1, policy_version 1087214 (0.0006) [2023-12-27 13:11:19,634][20347] Updated weights for policy 0, policy_version 1088185 (0.0008) [2023-12-27 13:11:19,640][20381] Updated weights for policy 1, policy_version 1087224 (0.0007) [2023-12-27 13:11:19,695][20347] Updated weights for policy 0, policy_version 1088195 (0.0008) [2023-12-27 13:11:20,337][20381] Updated weights for policy 1, policy_version 1087234 (0.0008) [2023-12-27 13:11:20,396][20347] Updated weights for policy 0, policy_version 1088205 (0.0009) [2023-12-27 13:11:20,398][20381] Updated weights for policy 1, policy_version 1087244 (0.0007) [2023-12-27 13:11:20,455][20347] Updated weights for policy 0, policy_version 1088215 (0.0007) [2023-12-27 13:11:20,459][20381] Updated weights for policy 1, policy_version 1087254 (0.0006) [2023-12-27 13:11:20,514][20347] Updated weights for policy 0, policy_version 1088225 (0.0008) [2023-12-27 13:11:21,106][20381] Updated weights for policy 1, policy_version 1087264 (0.0008) [2023-12-27 13:11:21,169][20381] Updated weights for policy 1, policy_version 1087274 (0.0007) [2023-12-27 13:11:21,232][20381] Updated weights for policy 1, policy_version 1087284 (0.0008) [2023-12-27 13:11:21,266][20347] Updated weights for policy 0, policy_version 1088235 (0.0008) [2023-12-27 13:11:21,331][20347] Updated weights for policy 0, policy_version 1088245 (0.0008) [2023-12-27 13:11:21,401][20347] Updated weights for policy 0, policy_version 1088255 (0.0008) [2023-12-27 13:11:21,940][20381] Updated weights for policy 1, policy_version 1087294 (0.0007) [2023-12-27 13:11:21,990][20381] Updated weights for policy 1, policy_version 1087304 (0.0008) [2023-12-27 13:11:22,040][20381] Updated weights for policy 1, policy_version 1087314 (0.0010) [2023-12-27 13:11:22,120][20347] Updated weights for policy 0, policy_version 1088265 (0.0008) [2023-12-27 13:11:22,181][20347] Updated weights for policy 0, policy_version 1088275 (0.0006) [2023-12-27 13:11:22,237][20347] Updated weights for policy 0, policy_version 1088285 (0.0008) [2023-12-27 13:11:22,305][20347] Updated weights for policy 0, policy_version 1088295 (0.0008) [2023-12-27 13:11:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 557031424. Throughput: 0: 10073.5, 1: 10333.2. Samples: 556997616. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:22,363][19170] Avg episode reward: [(0, '882.660'), (1, '66.740')] [2023-12-27 13:11:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001087320_278380544.pth... [2023-12-27 13:11:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001088296_278650880.pth... [2023-12-27 13:11:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001086104_278069248.pth [2023-12-27 13:11:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001087112_278347776.pth [2023-12-27 13:11:22,771][20381] Updated weights for policy 1, policy_version 1087324 (0.0008) [2023-12-27 13:11:22,831][20381] Updated weights for policy 1, policy_version 1087334 (0.0008) [2023-12-27 13:11:22,881][20381] Updated weights for policy 1, policy_version 1087344 (0.0008) [2023-12-27 13:11:23,009][20347] Updated weights for policy 0, policy_version 1088305 (0.0006) [2023-12-27 13:11:23,065][20347] Updated weights for policy 0, policy_version 1088315 (0.0006) [2023-12-27 13:11:23,116][20347] Updated weights for policy 0, policy_version 1088325 (0.0005) [2023-12-27 13:11:23,586][20381] Updated weights for policy 1, policy_version 1087354 (0.0008) [2023-12-27 13:11:23,635][20381] Updated weights for policy 1, policy_version 1087364 (0.0010) [2023-12-27 13:11:23,685][20381] Updated weights for policy 1, policy_version 1087374 (0.0010) [2023-12-27 13:11:23,697][20347] Updated weights for policy 0, policy_version 1088335 (0.0005) [2023-12-27 13:11:23,733][20381] Updated weights for policy 1, policy_version 1087384 (0.0010) [2023-12-27 13:11:23,743][20347] Updated weights for policy 0, policy_version 1088345 (0.0006) [2023-12-27 13:11:23,795][20347] Updated weights for policy 0, policy_version 1088355 (0.0006) [2023-12-27 13:11:24,411][20381] Updated weights for policy 1, policy_version 1087394 (0.0006) [2023-12-27 13:11:24,470][20381] Updated weights for policy 1, policy_version 1087404 (0.0008) [2023-12-27 13:11:24,510][20347] Updated weights for policy 0, policy_version 1088365 (0.0006) [2023-12-27 13:11:24,523][20381] Updated weights for policy 1, policy_version 1087414 (0.0011) [2023-12-27 13:11:24,568][20347] Updated weights for policy 0, policy_version 1088375 (0.0005) [2023-12-27 13:11:24,625][20347] Updated weights for policy 0, policy_version 1088385 (0.0005) [2023-12-27 13:11:25,228][20381] Updated weights for policy 1, policy_version 1087424 (0.0011) [2023-12-27 13:11:25,282][20347] Updated weights for policy 0, policy_version 1088395 (0.0005) [2023-12-27 13:11:25,284][20381] Updated weights for policy 1, policy_version 1087434 (0.0011) [2023-12-27 13:11:25,337][20347] Updated weights for policy 0, policy_version 1088405 (0.0005) [2023-12-27 13:11:25,343][20381] Updated weights for policy 1, policy_version 1087444 (0.0011) [2023-12-27 13:11:25,392][20347] Updated weights for policy 0, policy_version 1088415 (0.0006) [2023-12-27 13:11:26,006][20381] Updated weights for policy 1, policy_version 1087454 (0.0011) [2023-12-27 13:11:26,010][20347] Updated weights for policy 0, policy_version 1088425 (0.0007) [2023-12-27 13:11:26,056][20347] Updated weights for policy 0, policy_version 1088435 (0.0006) [2023-12-27 13:11:26,057][20381] Updated weights for policy 1, policy_version 1087464 (0.0010) [2023-12-27 13:11:26,109][20347] Updated weights for policy 0, policy_version 1088445 (0.0006) [2023-12-27 13:11:26,113][20381] Updated weights for policy 1, policy_version 1087474 (0.0008) [2023-12-27 13:11:26,163][20347] Updated weights for policy 0, policy_version 1088455 (0.0005) [2023-12-27 13:11:26,647][20381] Updated weights for policy 1, policy_version 1087484 (0.0005) [2023-12-27 13:11:26,695][20381] Updated weights for policy 1, policy_version 1087494 (0.0005) [2023-12-27 13:11:26,735][20347] Updated weights for policy 0, policy_version 1088465 (0.0007) [2023-12-27 13:11:26,738][20381] Updated weights for policy 1, policy_version 1087504 (0.0005) [2023-12-27 13:11:26,780][20347] Updated weights for policy 0, policy_version 1088475 (0.0007) [2023-12-27 13:11:26,837][20347] Updated weights for policy 0, policy_version 1088485 (0.0007) [2023-12-27 13:11:27,311][20381] Updated weights for policy 1, policy_version 1087514 (0.0006) [2023-12-27 13:11:27,360][20381] Updated weights for policy 1, policy_version 1087524 (0.0006) [2023-12-27 13:11:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 557129728. Throughput: 0: 10088.6, 1: 10305.8. Samples: 557119316. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-12-27 13:11:27,363][19170] Avg episode reward: [(0, '853.000'), (1, '258.730')] [2023-12-27 13:11:27,404][20381] Updated weights for policy 1, policy_version 1087534 (0.0005) [2023-12-27 13:11:27,458][20381] Updated weights for policy 1, policy_version 1087544 (0.0005) [2023-12-27 13:11:27,483][20347] Updated weights for policy 0, policy_version 1088495 (0.0008) [2023-12-27 13:11:27,535][20347] Updated weights for policy 0, policy_version 1088505 (0.0008) [2023-12-27 13:11:27,583][20347] Updated weights for policy 0, policy_version 1088515 (0.0010) [2023-12-27 13:11:28,063][20381] Updated weights for policy 1, policy_version 1087554 (0.0005) [2023-12-27 13:11:28,117][20381] Updated weights for policy 1, policy_version 1087564 (0.0005) [2023-12-27 13:11:28,169][20381] Updated weights for policy 1, policy_version 1087574 (0.0007) [2023-12-27 13:11:28,212][20291] KL-divergence is very high: 189.2687 [2023-12-27 13:11:28,217][20347] Updated weights for policy 0, policy_version 1088525 (0.0010) [2023-12-27 13:11:28,249][20291] KL-divergence is very high: 147.2020 [2023-12-27 13:11:28,265][20347] Updated weights for policy 0, policy_version 1088535 (0.0010) [2023-12-27 13:11:28,312][20347] Updated weights for policy 0, policy_version 1088545 (0.0007) [2023-12-27 13:11:28,762][20381] Updated weights for policy 1, policy_version 1087584 (0.0010) [2023-12-27 13:11:28,814][20381] Updated weights for policy 1, policy_version 1087594 (0.0011) [2023-12-27 13:11:28,872][20381] Updated weights for policy 1, policy_version 1087604 (0.0011) [2023-12-27 13:11:29,002][20347] Updated weights for policy 0, policy_version 1088555 (0.0008) [2023-12-27 13:11:29,073][20347] Updated weights for policy 0, policy_version 1088566 (0.0008) [2023-12-27 13:11:29,127][20347] Updated weights for policy 0, policy_version 1088576 (0.0007) [2023-12-27 13:11:29,629][20381] Updated weights for policy 1, policy_version 1087614 (0.0010) [2023-12-27 13:11:29,681][20381] Updated weights for policy 1, policy_version 1087624 (0.0010) [2023-12-27 13:11:29,733][20381] Updated weights for policy 1, policy_version 1087634 (0.0010) [2023-12-27 13:11:29,828][20347] Updated weights for policy 0, policy_version 1088586 (0.0007) [2023-12-27 13:11:29,893][20347] Updated weights for policy 0, policy_version 1088596 (0.0007) [2023-12-27 13:11:29,955][20347] Updated weights for policy 0, policy_version 1088606 (0.0007) [2023-12-27 13:11:29,963][20291] KL-divergence is very high: 122.9285 [2023-12-27 13:11:30,002][20291] KL-divergence is very high: 108.9654 [2023-12-27 13:11:30,011][20347] Updated weights for policy 0, policy_version 1088616 (0.0005) [2023-12-27 13:11:30,437][20381] Updated weights for policy 1, policy_version 1087644 (0.0009) [2023-12-27 13:11:30,496][20381] Updated weights for policy 1, policy_version 1087654 (0.0007) [2023-12-27 13:11:30,545][20381] Updated weights for policy 1, policy_version 1087664 (0.0006) [2023-12-27 13:11:30,677][20347] Updated weights for policy 0, policy_version 1088626 (0.0008) [2023-12-27 13:11:30,736][20347] Updated weights for policy 0, policy_version 1088636 (0.0005) [2023-12-27 13:11:30,792][20347] Updated weights for policy 0, policy_version 1088646 (0.0007) [2023-12-27 13:11:31,214][20381] Updated weights for policy 1, policy_version 1087674 (0.0008) [2023-12-27 13:11:31,279][20381] Updated weights for policy 1, policy_version 1087684 (0.0007) [2023-12-27 13:11:31,341][20381] Updated weights for policy 1, policy_version 1087694 (0.0006) [2023-12-27 13:11:31,411][20381] Updated weights for policy 1, policy_version 1087704 (0.0007) [2023-12-27 13:11:31,434][20347] Updated weights for policy 0, policy_version 1088656 (0.0008) [2023-12-27 13:11:31,495][20347] Updated weights for policy 0, policy_version 1088666 (0.0007) [2023-12-27 13:11:31,552][20347] Updated weights for policy 0, policy_version 1088676 (0.0008) [2023-12-27 13:11:31,982][20381] Updated weights for policy 1, policy_version 1087714 (0.0005) [2023-12-27 13:11:32,040][20381] Updated weights for policy 1, policy_version 1087724 (0.0005) [2023-12-27 13:11:32,101][20381] Updated weights for policy 1, policy_version 1087734 (0.0006) [2023-12-27 13:11:32,246][20347] Updated weights for policy 0, policy_version 1088686 (0.0007) [2023-12-27 13:11:32,306][20347] Updated weights for policy 0, policy_version 1088696 (0.0007) [2023-12-27 13:11:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 557236224. Throughput: 0: 10124.6, 1: 10378.5. Samples: 557249808. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:32,363][19170] Avg episode reward: [(0, '823.840'), (1, '188.110')] [2023-12-27 13:11:32,371][20347] Updated weights for policy 0, policy_version 1088706 (0.0008) [2023-12-27 13:11:32,694][20381] Updated weights for policy 1, policy_version 1087744 (0.0005) [2023-12-27 13:11:32,752][20381] Updated weights for policy 1, policy_version 1087754 (0.0008) [2023-12-27 13:11:32,805][20381] Updated weights for policy 1, policy_version 1087764 (0.0008) [2023-12-27 13:11:33,088][20347] Updated weights for policy 0, policy_version 1088716 (0.0008) [2023-12-27 13:11:33,142][20347] Updated weights for policy 0, policy_version 1088726 (0.0010) [2023-12-27 13:11:33,196][20347] Updated weights for policy 0, policy_version 1088736 (0.0010) [2023-12-27 13:11:33,467][20381] Updated weights for policy 1, policy_version 1087774 (0.0007) [2023-12-27 13:11:33,514][20381] Updated weights for policy 1, policy_version 1087784 (0.0007) [2023-12-27 13:11:33,568][20381] Updated weights for policy 1, policy_version 1087794 (0.0008) [2023-12-27 13:11:33,827][20347] Updated weights for policy 0, policy_version 1088746 (0.0009) [2023-12-27 13:11:33,888][20347] Updated weights for policy 0, policy_version 1088756 (0.0005) [2023-12-27 13:11:33,946][20347] Updated weights for policy 0, policy_version 1088766 (0.0006) [2023-12-27 13:11:34,004][20347] Updated weights for policy 0, policy_version 1088776 (0.0006) [2023-12-27 13:11:34,352][20381] Updated weights for policy 1, policy_version 1087804 (0.0007) [2023-12-27 13:11:34,406][20381] Updated weights for policy 1, policy_version 1087814 (0.0009) [2023-12-27 13:11:34,469][20381] Updated weights for policy 1, policy_version 1087824 (0.0007) [2023-12-27 13:11:34,673][20347] Updated weights for policy 0, policy_version 1088786 (0.0006) [2023-12-27 13:11:34,738][20347] Updated weights for policy 0, policy_version 1088796 (0.0007) [2023-12-27 13:11:34,798][20347] Updated weights for policy 0, policy_version 1088806 (0.0009) [2023-12-27 13:11:35,157][20381] Updated weights for policy 1, policy_version 1087834 (0.0006) [2023-12-27 13:11:35,209][20381] Updated weights for policy 1, policy_version 1087844 (0.0005) [2023-12-27 13:11:35,258][20381] Updated weights for policy 1, policy_version 1087854 (0.0005) [2023-12-27 13:11:35,313][20381] Updated weights for policy 1, policy_version 1087864 (0.0008) [2023-12-27 13:11:35,347][20347] Updated weights for policy 0, policy_version 1088816 (0.0006) [2023-12-27 13:11:35,410][20347] Updated weights for policy 0, policy_version 1088826 (0.0005) [2023-12-27 13:11:35,462][20347] Updated weights for policy 0, policy_version 1088836 (0.0006) [2023-12-27 13:11:36,009][20381] Updated weights for policy 1, policy_version 1087874 (0.0006) [2023-12-27 13:11:36,031][20347] Updated weights for policy 0, policy_version 1088846 (0.0008) [2023-12-27 13:11:36,062][20381] Updated weights for policy 1, policy_version 1087884 (0.0007) [2023-12-27 13:11:36,088][20347] Updated weights for policy 0, policy_version 1088856 (0.0006) [2023-12-27 13:11:36,113][20381] Updated weights for policy 1, policy_version 1087894 (0.0007) [2023-12-27 13:11:36,154][20347] Updated weights for policy 0, policy_version 1088866 (0.0008) [2023-12-27 13:11:36,823][20381] Updated weights for policy 1, policy_version 1087904 (0.0007) [2023-12-27 13:11:36,877][20381] Updated weights for policy 1, policy_version 1087914 (0.0006) [2023-12-27 13:11:36,878][20347] Updated weights for policy 0, policy_version 1088876 (0.0007) [2023-12-27 13:11:36,926][20381] Updated weights for policy 1, policy_version 1087924 (0.0005) [2023-12-27 13:11:36,937][20347] Updated weights for policy 0, policy_version 1088886 (0.0007) [2023-12-27 13:11:37,001][20347] Updated weights for policy 0, policy_version 1088896 (0.0008) [2023-12-27 13:11:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 557342720. Throughput: 0: 10144.7, 1: 10377.6. Samples: 557311348. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:37,364][19170] Avg episode reward: [(0, '921.810'), (1, '252.950')] [2023-12-27 13:11:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001087928_278536192.pth... [2023-12-27 13:11:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001088904_278806528.pth... [2023-12-27 13:11:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001086712_278224896.pth [2023-12-27 13:11:37,396][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001087720_278503424.pth [2023-12-27 13:11:37,499][20381] Updated weights for policy 1, policy_version 1087934 (0.0007) [2023-12-27 13:11:37,552][20381] Updated weights for policy 1, policy_version 1087944 (0.0008) [2023-12-27 13:11:37,607][20381] Updated weights for policy 1, policy_version 1087954 (0.0008) [2023-12-27 13:11:37,689][20347] Updated weights for policy 0, policy_version 1088906 (0.0008) [2023-12-27 13:11:37,735][20347] Updated weights for policy 0, policy_version 1088916 (0.0006) [2023-12-27 13:11:37,780][20347] Updated weights for policy 0, policy_version 1088926 (0.0007) [2023-12-27 13:11:37,828][20347] Updated weights for policy 0, policy_version 1088936 (0.0007) [2023-12-27 13:11:38,238][20381] Updated weights for policy 1, policy_version 1087964 (0.0006) [2023-12-27 13:11:38,292][20381] Updated weights for policy 1, policy_version 1087974 (0.0008) [2023-12-27 13:11:38,355][20381] Updated weights for policy 1, policy_version 1087984 (0.0008) [2023-12-27 13:11:38,423][20347] Updated weights for policy 0, policy_version 1088946 (0.0007) [2023-12-27 13:11:38,483][20347] Updated weights for policy 0, policy_version 1088956 (0.0006) [2023-12-27 13:11:38,544][20347] Updated weights for policy 0, policy_version 1088966 (0.0007) [2023-12-27 13:11:39,002][20381] Updated weights for policy 1, policy_version 1087994 (0.0008) [2023-12-27 13:11:39,047][20381] Updated weights for policy 1, policy_version 1088004 (0.0007) [2023-12-27 13:11:39,090][20381] Updated weights for policy 1, policy_version 1088014 (0.0007) [2023-12-27 13:11:39,152][20381] Updated weights for policy 1, policy_version 1088024 (0.0008) [2023-12-27 13:11:39,241][20347] Updated weights for policy 0, policy_version 1088976 (0.0008) [2023-12-27 13:11:39,299][20347] Updated weights for policy 0, policy_version 1088986 (0.0008) [2023-12-27 13:11:39,361][20347] Updated weights for policy 0, policy_version 1088996 (0.0008) [2023-12-27 13:11:39,934][20381] Updated weights for policy 1, policy_version 1088034 (0.0006) [2023-12-27 13:11:39,996][20381] Updated weights for policy 1, policy_version 1088044 (0.0008) [2023-12-27 13:11:40,014][20347] Updated weights for policy 0, policy_version 1089006 (0.0007) [2023-12-27 13:11:40,058][20381] Updated weights for policy 1, policy_version 1088054 (0.0008) [2023-12-27 13:11:40,068][20347] Updated weights for policy 0, policy_version 1089016 (0.0007) [2023-12-27 13:11:40,119][20347] Updated weights for policy 0, policy_version 1089026 (0.0008) [2023-12-27 13:11:40,741][20381] Updated weights for policy 1, policy_version 1088064 (0.0009) [2023-12-27 13:11:40,801][20381] Updated weights for policy 1, policy_version 1088074 (0.0008) [2023-12-27 13:11:40,807][20347] Updated weights for policy 0, policy_version 1089036 (0.0007) [2023-12-27 13:11:40,860][20381] Updated weights for policy 1, policy_version 1088084 (0.0007) [2023-12-27 13:11:40,862][20347] Updated weights for policy 0, policy_version 1089046 (0.0007) [2023-12-27 13:11:40,916][20347] Updated weights for policy 0, policy_version 1089056 (0.0007) [2023-12-27 13:11:41,558][20381] Updated weights for policy 1, policy_version 1088094 (0.0009) [2023-12-27 13:11:41,618][20381] Updated weights for policy 1, policy_version 1088104 (0.0010) [2023-12-27 13:11:41,676][20347] Updated weights for policy 0, policy_version 1089066 (0.0008) [2023-12-27 13:11:41,682][20381] Updated weights for policy 1, policy_version 1088114 (0.0008) [2023-12-27 13:11:41,735][20347] Updated weights for policy 0, policy_version 1089076 (0.0008) [2023-12-27 13:11:41,802][20347] Updated weights for policy 0, policy_version 1089086 (0.0008) [2023-12-27 13:11:41,866][20347] Updated weights for policy 0, policy_version 1089096 (0.0008) [2023-12-27 13:11:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 557441024. Throughput: 0: 10207.8, 1: 10364.4. Samples: 557436136. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:42,363][19170] Avg episode reward: [(0, '951.000'), (1, '425.690')] [2023-12-27 13:11:42,410][20381] Updated weights for policy 1, policy_version 1088124 (0.0007) [2023-12-27 13:11:42,469][20381] Updated weights for policy 1, policy_version 1088134 (0.0005) [2023-12-27 13:11:42,529][20381] Updated weights for policy 1, policy_version 1088144 (0.0010) [2023-12-27 13:11:42,613][20347] Updated weights for policy 0, policy_version 1089106 (0.0008) [2023-12-27 13:11:42,669][20347] Updated weights for policy 0, policy_version 1089116 (0.0008) [2023-12-27 13:11:42,730][20347] Updated weights for policy 0, policy_version 1089126 (0.0007) [2023-12-27 13:11:43,127][20381] Updated weights for policy 1, policy_version 1088154 (0.0005) [2023-12-27 13:11:43,190][20381] Updated weights for policy 1, policy_version 1088164 (0.0006) [2023-12-27 13:11:43,246][20381] Updated weights for policy 1, policy_version 1088174 (0.0006) [2023-12-27 13:11:43,296][20381] Updated weights for policy 1, policy_version 1088184 (0.0006) [2023-12-27 13:11:43,419][20347] Updated weights for policy 0, policy_version 1089136 (0.0008) [2023-12-27 13:11:43,472][20347] Updated weights for policy 0, policy_version 1089146 (0.0007) [2023-12-27 13:11:43,525][20347] Updated weights for policy 0, policy_version 1089156 (0.0007) [2023-12-27 13:11:43,917][20381] Updated weights for policy 1, policy_version 1088194 (0.0010) [2023-12-27 13:11:43,963][20381] Updated weights for policy 1, policy_version 1088204 (0.0009) [2023-12-27 13:11:44,013][20381] Updated weights for policy 1, policy_version 1088214 (0.0009) [2023-12-27 13:11:44,139][20347] Updated weights for policy 0, policy_version 1089166 (0.0006) [2023-12-27 13:11:44,192][20347] Updated weights for policy 0, policy_version 1089176 (0.0005) [2023-12-27 13:11:44,247][20347] Updated weights for policy 0, policy_version 1089186 (0.0006) [2023-12-27 13:11:44,703][20381] Updated weights for policy 1, policy_version 1088224 (0.0007) [2023-12-27 13:11:44,755][20381] Updated weights for policy 1, policy_version 1088234 (0.0007) [2023-12-27 13:11:44,815][20381] Updated weights for policy 1, policy_version 1088244 (0.0008) [2023-12-27 13:11:44,851][20347] Updated weights for policy 0, policy_version 1089196 (0.0006) [2023-12-27 13:11:44,904][20347] Updated weights for policy 0, policy_version 1089206 (0.0008) [2023-12-27 13:11:44,971][20347] Updated weights for policy 0, policy_version 1089216 (0.0007) [2023-12-27 13:11:45,514][20381] Updated weights for policy 1, policy_version 1088254 (0.0007) [2023-12-27 13:11:45,565][20381] Updated weights for policy 1, policy_version 1088264 (0.0008) [2023-12-27 13:11:45,609][20381] Updated weights for policy 1, policy_version 1088274 (0.0007) [2023-12-27 13:11:45,655][20347] Updated weights for policy 0, policy_version 1089226 (0.0008) [2023-12-27 13:11:45,713][20347] Updated weights for policy 0, policy_version 1089236 (0.0008) [2023-12-27 13:11:45,774][20347] Updated weights for policy 0, policy_version 1089246 (0.0007) [2023-12-27 13:11:45,822][20347] Updated weights for policy 0, policy_version 1089256 (0.0007) [2023-12-27 13:11:46,289][20381] Updated weights for policy 1, policy_version 1088284 (0.0007) [2023-12-27 13:11:46,347][20381] Updated weights for policy 1, policy_version 1088294 (0.0008) [2023-12-27 13:11:46,411][20381] Updated weights for policy 1, policy_version 1088304 (0.0007) [2023-12-27 13:11:46,460][20347] Updated weights for policy 0, policy_version 1089266 (0.0010) [2023-12-27 13:11:46,513][20347] Updated weights for policy 0, policy_version 1089276 (0.0010) [2023-12-27 13:11:46,565][20347] Updated weights for policy 0, policy_version 1089286 (0.0010) [2023-12-27 13:11:47,005][20381] Updated weights for policy 1, policy_version 1088314 (0.0006) [2023-12-27 13:11:47,061][20381] Updated weights for policy 1, policy_version 1088324 (0.0010) [2023-12-27 13:11:47,109][20381] Updated weights for policy 1, policy_version 1088334 (0.0010) [2023-12-27 13:11:47,164][20381] Updated weights for policy 1, policy_version 1088344 (0.0010) [2023-12-27 13:11:47,185][20347] Updated weights for policy 0, policy_version 1089296 (0.0009) [2023-12-27 13:11:47,247][20347] Updated weights for policy 0, policy_version 1089306 (0.0010) [2023-12-27 13:11:47,310][20347] Updated weights for policy 0, policy_version 1089316 (0.0010) [2023-12-27 13:11:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 557555712. Throughput: 0: 10255.7, 1: 10416.0. Samples: 557560140. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:47,363][19170] Avg episode reward: [(0, '941.260'), (1, '354.010')] [2023-12-27 13:11:47,853][20381] Updated weights for policy 1, policy_version 1088354 (0.0008) [2023-12-27 13:11:47,905][20381] Updated weights for policy 1, policy_version 1088364 (0.0007) [2023-12-27 13:11:47,931][20347] Updated weights for policy 0, policy_version 1089326 (0.0010) [2023-12-27 13:11:47,963][20381] Updated weights for policy 1, policy_version 1088374 (0.0007) [2023-12-27 13:11:47,983][20347] Updated weights for policy 0, policy_version 1089336 (0.0010) [2023-12-27 13:11:48,044][20347] Updated weights for policy 0, policy_version 1089346 (0.0010) [2023-12-27 13:11:48,605][20381] Updated weights for policy 1, policy_version 1088384 (0.0005) [2023-12-27 13:11:48,663][20381] Updated weights for policy 1, policy_version 1088394 (0.0009) [2023-12-27 13:11:48,719][20381] Updated weights for policy 1, policy_version 1088404 (0.0009) [2023-12-27 13:11:48,741][20347] Updated weights for policy 0, policy_version 1089356 (0.0009) [2023-12-27 13:11:48,806][20347] Updated weights for policy 0, policy_version 1089366 (0.0007) [2023-12-27 13:11:48,867][20347] Updated weights for policy 0, policy_version 1089376 (0.0007) [2023-12-27 13:11:49,399][20381] Updated weights for policy 1, policy_version 1088414 (0.0008) [2023-12-27 13:11:49,461][20381] Updated weights for policy 1, policy_version 1088424 (0.0006) [2023-12-27 13:11:49,518][20381] Updated weights for policy 1, policy_version 1088434 (0.0006) [2023-12-27 13:11:49,547][20347] Updated weights for policy 0, policy_version 1089386 (0.0007) [2023-12-27 13:11:49,598][20347] Updated weights for policy 0, policy_version 1089396 (0.0008) [2023-12-27 13:11:49,648][20347] Updated weights for policy 0, policy_version 1089406 (0.0007) [2023-12-27 13:11:49,706][20347] Updated weights for policy 0, policy_version 1089416 (0.0007) [2023-12-27 13:11:50,165][20381] Updated weights for policy 1, policy_version 1088444 (0.0008) [2023-12-27 13:11:50,219][20381] Updated weights for policy 1, policy_version 1088454 (0.0010) [2023-12-27 13:11:50,270][20381] Updated weights for policy 1, policy_version 1088464 (0.0009) [2023-12-27 13:11:50,484][20347] Updated weights for policy 0, policy_version 1089426 (0.0007) [2023-12-27 13:11:50,529][20347] Updated weights for policy 0, policy_version 1089436 (0.0007) [2023-12-27 13:11:50,587][20347] Updated weights for policy 0, policy_version 1089446 (0.0008) [2023-12-27 13:11:50,988][20381] Updated weights for policy 1, policy_version 1088474 (0.0007) [2023-12-27 13:11:51,061][20381] Updated weights for policy 1, policy_version 1088484 (0.0006) [2023-12-27 13:11:51,134][20381] Updated weights for policy 1, policy_version 1088494 (0.0006) [2023-12-27 13:11:51,208][20381] Updated weights for policy 1, policy_version 1088504 (0.0007) [2023-12-27 13:11:51,248][20347] Updated weights for policy 0, policy_version 1089456 (0.0008) [2023-12-27 13:11:51,312][20347] Updated weights for policy 0, policy_version 1089466 (0.0007) [2023-12-27 13:11:51,369][20347] Updated weights for policy 0, policy_version 1089476 (0.0008) [2023-12-27 13:11:51,865][20381] Updated weights for policy 1, policy_version 1088514 (0.0007) [2023-12-27 13:11:51,920][20381] Updated weights for policy 1, policy_version 1088524 (0.0007) [2023-12-27 13:11:51,980][20381] Updated weights for policy 1, policy_version 1088534 (0.0008) [2023-12-27 13:11:52,094][20347] Updated weights for policy 0, policy_version 1089486 (0.0007) [2023-12-27 13:11:52,151][20347] Updated weights for policy 0, policy_version 1089496 (0.0007) [2023-12-27 13:11:52,210][20347] Updated weights for policy 0, policy_version 1089506 (0.0008) [2023-12-27 13:11:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 557654016. Throughput: 0: 10258.6, 1: 10411.3. Samples: 557621120. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:52,363][19170] Avg episode reward: [(0, '960.910'), (1, '498.330')] [2023-12-27 13:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001088536_278691840.pth... [2023-12-27 13:11:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001089512_278962176.pth... [2023-12-27 13:11:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001088296_278650880.pth [2023-12-27 13:11:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001087320_278380544.pth [2023-12-27 13:11:52,701][20381] Updated weights for policy 1, policy_version 1088544 (0.0006) [2023-12-27 13:11:52,759][20381] Updated weights for policy 1, policy_version 1088554 (0.0007) [2023-12-27 13:11:52,810][20381] Updated weights for policy 1, policy_version 1088564 (0.0008) [2023-12-27 13:11:52,924][20347] Updated weights for policy 0, policy_version 1089516 (0.0008) [2023-12-27 13:11:52,970][20347] Updated weights for policy 0, policy_version 1089526 (0.0008) [2023-12-27 13:11:53,023][20347] Updated weights for policy 0, policy_version 1089536 (0.0006) [2023-12-27 13:11:53,485][20381] Updated weights for policy 1, policy_version 1088574 (0.0006) [2023-12-27 13:11:53,540][20381] Updated weights for policy 1, policy_version 1088584 (0.0005) [2023-12-27 13:11:53,595][20381] Updated weights for policy 1, policy_version 1088594 (0.0005) [2023-12-27 13:11:53,611][20347] Updated weights for policy 0, policy_version 1089546 (0.0006) [2023-12-27 13:11:53,655][20347] Updated weights for policy 0, policy_version 1089556 (0.0006) [2023-12-27 13:11:53,709][20347] Updated weights for policy 0, policy_version 1089566 (0.0005) [2023-12-27 13:11:53,755][20347] Updated weights for policy 0, policy_version 1089576 (0.0005) [2023-12-27 13:11:54,173][20381] Updated weights for policy 1, policy_version 1088604 (0.0005) [2023-12-27 13:11:54,238][20381] Updated weights for policy 1, policy_version 1088614 (0.0006) [2023-12-27 13:11:54,297][20381] Updated weights for policy 1, policy_version 1088624 (0.0008) [2023-12-27 13:11:54,369][20347] Updated weights for policy 0, policy_version 1089586 (0.0008) [2023-12-27 13:11:54,424][20347] Updated weights for policy 0, policy_version 1089596 (0.0006) [2023-12-27 13:11:54,475][20347] Updated weights for policy 0, policy_version 1089606 (0.0007) [2023-12-27 13:11:54,989][20381] Updated weights for policy 1, policy_version 1088634 (0.0007) [2023-12-27 13:11:55,044][20381] Updated weights for policy 1, policy_version 1088644 (0.0008) [2023-12-27 13:11:55,103][20381] Updated weights for policy 1, policy_version 1088654 (0.0007) [2023-12-27 13:11:55,103][20347] Updated weights for policy 0, policy_version 1089616 (0.0006) [2023-12-27 13:11:55,160][20381] Updated weights for policy 1, policy_version 1088664 (0.0007) [2023-12-27 13:11:55,163][20347] Updated weights for policy 0, policy_version 1089626 (0.0006) [2023-12-27 13:11:55,223][20347] Updated weights for policy 0, policy_version 1089636 (0.0007) [2023-12-27 13:11:55,771][20347] Updated weights for policy 0, policy_version 1089646 (0.0007) [2023-12-27 13:11:55,817][20347] Updated weights for policy 0, policy_version 1089656 (0.0005) [2023-12-27 13:11:55,818][20381] Updated weights for policy 1, policy_version 1088674 (0.0005) [2023-12-27 13:11:55,873][20347] Updated weights for policy 0, policy_version 1089666 (0.0006) [2023-12-27 13:11:55,875][20381] Updated weights for policy 1, policy_version 1088684 (0.0008) [2023-12-27 13:11:55,926][20381] Updated weights for policy 1, policy_version 1088694 (0.0007) [2023-12-27 13:11:56,576][20347] Updated weights for policy 0, policy_version 1089676 (0.0006) [2023-12-27 13:11:56,631][20347] Updated weights for policy 0, policy_version 1089686 (0.0006) [2023-12-27 13:11:56,632][20381] Updated weights for policy 1, policy_version 1088704 (0.0008) [2023-12-27 13:11:56,681][20347] Updated weights for policy 0, policy_version 1089696 (0.0005) [2023-12-27 13:11:56,693][20381] Updated weights for policy 1, policy_version 1088714 (0.0007) [2023-12-27 13:11:56,753][20381] Updated weights for policy 1, policy_version 1088724 (0.0006) [2023-12-27 13:11:57,344][20381] Updated weights for policy 1, policy_version 1088734 (0.0006) [2023-12-27 13:11:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 557752320. Throughput: 0: 10316.3, 1: 10433.3. Samples: 557747076. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:11:57,363][19170] Avg episode reward: [(0, '960.920'), (1, '523.790')] [2023-12-27 13:11:57,372][20347] Updated weights for policy 0, policy_version 1089706 (0.0006) [2023-12-27 13:11:57,403][20381] Updated weights for policy 1, policy_version 1088744 (0.0006) [2023-12-27 13:11:57,431][20347] Updated weights for policy 0, policy_version 1089716 (0.0007) [2023-12-27 13:11:57,452][20381] Updated weights for policy 1, policy_version 1088754 (0.0007) [2023-12-27 13:11:57,481][20347] Updated weights for policy 0, policy_version 1089726 (0.0007) [2023-12-27 13:11:57,531][20347] Updated weights for policy 0, policy_version 1089736 (0.0008) [2023-12-27 13:11:58,124][20381] Updated weights for policy 1, policy_version 1088764 (0.0008) [2023-12-27 13:11:58,140][20347] Updated weights for policy 0, policy_version 1089746 (0.0007) [2023-12-27 13:11:58,180][20381] Updated weights for policy 1, policy_version 1088774 (0.0010) [2023-12-27 13:11:58,192][20347] Updated weights for policy 0, policy_version 1089756 (0.0006) [2023-12-27 13:11:58,240][20381] Updated weights for policy 1, policy_version 1088784 (0.0010) [2023-12-27 13:11:58,255][20347] Updated weights for policy 0, policy_version 1089766 (0.0010) [2023-12-27 13:11:59,066][20347] Updated weights for policy 0, policy_version 1089776 (0.0009) [2023-12-27 13:11:59,085][20381] Updated weights for policy 1, policy_version 1088794 (0.0010) [2023-12-27 13:11:59,124][20347] Updated weights for policy 0, policy_version 1089786 (0.0007) [2023-12-27 13:11:59,138][20381] Updated weights for policy 1, policy_version 1088804 (0.0006) [2023-12-27 13:11:59,180][20347] Updated weights for policy 0, policy_version 1089796 (0.0006) [2023-12-27 13:11:59,197][20381] Updated weights for policy 1, policy_version 1088814 (0.0006) [2023-12-27 13:11:59,262][20381] Updated weights for policy 1, policy_version 1088824 (0.0008) [2023-12-27 13:11:59,910][20347] Updated weights for policy 0, policy_version 1089806 (0.0008) [2023-12-27 13:11:59,927][20381] Updated weights for policy 1, policy_version 1088834 (0.0008) [2023-12-27 13:11:59,976][20347] Updated weights for policy 0, policy_version 1089816 (0.0008) [2023-12-27 13:11:59,989][20381] Updated weights for policy 1, policy_version 1088844 (0.0008) [2023-12-27 13:12:00,038][20347] Updated weights for policy 0, policy_version 1089826 (0.0009) [2023-12-27 13:12:00,048][20381] Updated weights for policy 1, policy_version 1088854 (0.0008) [2023-12-27 13:12:00,647][20347] Updated weights for policy 0, policy_version 1089836 (0.0008) [2023-12-27 13:12:00,694][20381] Updated weights for policy 1, policy_version 1088864 (0.0006) [2023-12-27 13:12:00,696][20347] Updated weights for policy 0, policy_version 1089846 (0.0009) [2023-12-27 13:12:00,747][20347] Updated weights for policy 0, policy_version 1089856 (0.0010) [2023-12-27 13:12:00,749][20381] Updated weights for policy 1, policy_version 1088874 (0.0006) [2023-12-27 13:12:00,801][20381] Updated weights for policy 1, policy_version 1088884 (0.0007) [2023-12-27 13:12:01,400][20381] Updated weights for policy 1, policy_version 1088894 (0.0007) [2023-12-27 13:12:01,445][20381] Updated weights for policy 1, policy_version 1088904 (0.0009) [2023-12-27 13:12:01,453][20347] Updated weights for policy 0, policy_version 1089866 (0.0009) [2023-12-27 13:12:01,504][20381] Updated weights for policy 1, policy_version 1088914 (0.0007) [2023-12-27 13:12:01,510][20347] Updated weights for policy 0, policy_version 1089876 (0.0005) [2023-12-27 13:12:01,565][20347] Updated weights for policy 0, policy_version 1089886 (0.0009) [2023-12-27 13:12:01,619][20347] Updated weights for policy 0, policy_version 1089896 (0.0008) [2023-12-27 13:12:02,255][20381] Updated weights for policy 1, policy_version 1088924 (0.0008) [2023-12-27 13:12:02,313][20381] Updated weights for policy 1, policy_version 1088934 (0.0007) [2023-12-27 13:12:02,352][20347] Updated weights for policy 0, policy_version 1089906 (0.0006) [2023-12-27 13:12:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 557850624. Throughput: 0: 10360.3, 1: 10421.7. Samples: 557869236. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:02,363][19170] Avg episode reward: [(0, '970.730'), (1, '387.160')] [2023-12-27 13:12:02,374][20381] Updated weights for policy 1, policy_version 1088944 (0.0009) [2023-12-27 13:12:02,414][20347] Updated weights for policy 0, policy_version 1089916 (0.0008) [2023-12-27 13:12:02,466][20347] Updated weights for policy 0, policy_version 1089926 (0.0007) [2023-12-27 13:12:03,058][20381] Updated weights for policy 1, policy_version 1088954 (0.0008) [2023-12-27 13:12:03,105][20381] Updated weights for policy 1, policy_version 1088964 (0.0010) [2023-12-27 13:12:03,155][20381] Updated weights for policy 1, policy_version 1088974 (0.0010) [2023-12-27 13:12:03,161][20347] Updated weights for policy 0, policy_version 1089936 (0.0010) [2023-12-27 13:12:03,210][20381] Updated weights for policy 1, policy_version 1088984 (0.0010) [2023-12-27 13:12:03,216][20347] Updated weights for policy 0, policy_version 1089946 (0.0010) [2023-12-27 13:12:03,276][20347] Updated weights for policy 0, policy_version 1089956 (0.0010) [2023-12-27 13:12:03,874][20381] Updated weights for policy 1, policy_version 1088994 (0.0007) [2023-12-27 13:12:03,908][20347] Updated weights for policy 0, policy_version 1089966 (0.0010) [2023-12-27 13:12:03,932][20381] Updated weights for policy 1, policy_version 1089004 (0.0007) [2023-12-27 13:12:03,962][20347] Updated weights for policy 0, policy_version 1089976 (0.0010) [2023-12-27 13:12:03,996][20381] Updated weights for policy 1, policy_version 1089014 (0.0010) [2023-12-27 13:12:04,019][20347] Updated weights for policy 0, policy_version 1089986 (0.0010) [2023-12-27 13:12:04,648][20381] Updated weights for policy 1, policy_version 1089024 (0.0011) [2023-12-27 13:12:04,699][20381] Updated weights for policy 1, policy_version 1089034 (0.0011) [2023-12-27 13:12:04,706][20347] Updated weights for policy 0, policy_version 1089996 (0.0010) [2023-12-27 13:12:04,752][20381] Updated weights for policy 1, policy_version 1089044 (0.0011) [2023-12-27 13:12:04,764][20347] Updated weights for policy 0, policy_version 1090006 (0.0005) [2023-12-27 13:12:04,826][20347] Updated weights for policy 0, policy_version 1090016 (0.0006) [2023-12-27 13:12:05,413][20347] Updated weights for policy 0, policy_version 1090026 (0.0010) [2023-12-27 13:12:05,451][20381] Updated weights for policy 1, policy_version 1089054 (0.0007) [2023-12-27 13:12:05,471][20347] Updated weights for policy 0, policy_version 1090036 (0.0010) [2023-12-27 13:12:05,502][20381] Updated weights for policy 1, policy_version 1089064 (0.0006) [2023-12-27 13:12:05,517][20347] Updated weights for policy 0, policy_version 1090046 (0.0010) [2023-12-27 13:12:05,551][20381] Updated weights for policy 1, policy_version 1089074 (0.0007) [2023-12-27 13:12:05,572][20347] Updated weights for policy 0, policy_version 1090056 (0.0010) [2023-12-27 13:12:06,212][20347] Updated weights for policy 0, policy_version 1090066 (0.0009) [2023-12-27 13:12:06,257][20381] Updated weights for policy 1, policy_version 1089084 (0.0007) [2023-12-27 13:12:06,274][20347] Updated weights for policy 0, policy_version 1090076 (0.0007) [2023-12-27 13:12:06,307][20381] Updated weights for policy 1, policy_version 1089094 (0.0008) [2023-12-27 13:12:06,333][20347] Updated weights for policy 0, policy_version 1090086 (0.0010) [2023-12-27 13:12:06,366][20381] Updated weights for policy 1, policy_version 1089104 (0.0008) [2023-12-27 13:12:06,927][20347] Updated weights for policy 0, policy_version 1090096 (0.0006) [2023-12-27 13:12:06,980][20347] Updated weights for policy 0, policy_version 1090106 (0.0009) [2023-12-27 13:12:07,039][20347] Updated weights for policy 0, policy_version 1090116 (0.0010) [2023-12-27 13:12:07,056][20381] Updated weights for policy 1, policy_version 1089114 (0.0009) [2023-12-27 13:12:07,102][20381] Updated weights for policy 1, policy_version 1089124 (0.0010) [2023-12-27 13:12:07,164][20381] Updated weights for policy 1, policy_version 1089134 (0.0011) [2023-12-27 13:12:07,224][20381] Updated weights for policy 1, policy_version 1089144 (0.0010) [2023-12-27 13:12:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 557965312. Throughput: 0: 10360.4, 1: 10380.6. Samples: 557930968. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:07,364][19170] Avg episode reward: [(0, '970.750'), (1, '452.760')] [2023-12-27 13:12:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001089144_278847488.pth... [2023-12-27 13:12:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001090120_279117824.pth... [2023-12-27 13:12:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001087928_278536192.pth [2023-12-27 13:12:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001088904_278806528.pth [2023-12-27 13:12:07,670][20347] Updated weights for policy 0, policy_version 1090126 (0.0009) [2023-12-27 13:12:07,725][20347] Updated weights for policy 0, policy_version 1090136 (0.0007) [2023-12-27 13:12:07,776][20347] Updated weights for policy 0, policy_version 1090146 (0.0006) [2023-12-27 13:12:07,917][20381] Updated weights for policy 1, policy_version 1089154 (0.0006) [2023-12-27 13:12:07,965][20381] Updated weights for policy 1, policy_version 1089164 (0.0006) [2023-12-27 13:12:08,017][20381] Updated weights for policy 1, policy_version 1089174 (0.0009) [2023-12-27 13:12:08,338][20347] Updated weights for policy 0, policy_version 1090156 (0.0007) [2023-12-27 13:12:08,399][20347] Updated weights for policy 0, policy_version 1090166 (0.0010) [2023-12-27 13:12:08,457][20347] Updated weights for policy 0, policy_version 1090176 (0.0010) [2023-12-27 13:12:08,620][20381] Updated weights for policy 1, policy_version 1089184 (0.0008) [2023-12-27 13:12:08,679][20381] Updated weights for policy 1, policy_version 1089194 (0.0007) [2023-12-27 13:12:08,731][20381] Updated weights for policy 1, policy_version 1089204 (0.0008) [2023-12-27 13:12:09,136][20347] Updated weights for policy 0, policy_version 1090186 (0.0011) [2023-12-27 13:12:09,195][20347] Updated weights for policy 0, policy_version 1090196 (0.0011) [2023-12-27 13:12:09,260][20347] Updated weights for policy 0, policy_version 1090206 (0.0008) [2023-12-27 13:12:09,324][20347] Updated weights for policy 0, policy_version 1090216 (0.0006) [2023-12-27 13:12:09,448][20381] Updated weights for policy 1, policy_version 1089214 (0.0007) [2023-12-27 13:12:09,504][20381] Updated weights for policy 1, policy_version 1089224 (0.0009) [2023-12-27 13:12:09,563][20381] Updated weights for policy 1, policy_version 1089234 (0.0007) [2023-12-27 13:12:10,014][20347] Updated weights for policy 0, policy_version 1090226 (0.0008) [2023-12-27 13:12:10,076][20347] Updated weights for policy 0, policy_version 1090236 (0.0007) [2023-12-27 13:12:10,138][20347] Updated weights for policy 0, policy_version 1090246 (0.0008) [2023-12-27 13:12:10,250][20381] Updated weights for policy 1, policy_version 1089244 (0.0007) [2023-12-27 13:12:10,311][20381] Updated weights for policy 1, policy_version 1089254 (0.0007) [2023-12-27 13:12:10,371][20381] Updated weights for policy 1, policy_version 1089264 (0.0008) [2023-12-27 13:12:10,769][20347] Updated weights for policy 0, policy_version 1090256 (0.0010) [2023-12-27 13:12:10,830][20347] Updated weights for policy 0, policy_version 1090266 (0.0010) [2023-12-27 13:12:10,886][20347] Updated weights for policy 0, policy_version 1090276 (0.0010) [2023-12-27 13:12:11,058][20381] Updated weights for policy 1, policy_version 1089274 (0.0008) [2023-12-27 13:12:11,120][20381] Updated weights for policy 1, policy_version 1089284 (0.0008) [2023-12-27 13:12:11,180][20381] Updated weights for policy 1, policy_version 1089294 (0.0008) [2023-12-27 13:12:11,242][20381] Updated weights for policy 1, policy_version 1089304 (0.0007) [2023-12-27 13:12:11,630][20347] Updated weights for policy 0, policy_version 1090286 (0.0009) [2023-12-27 13:12:11,694][20347] Updated weights for policy 0, policy_version 1090296 (0.0009) [2023-12-27 13:12:11,757][20347] Updated weights for policy 0, policy_version 1090306 (0.0008) [2023-12-27 13:12:11,987][20381] Updated weights for policy 1, policy_version 1089314 (0.0008) [2023-12-27 13:12:12,035][20381] Updated weights for policy 1, policy_version 1089324 (0.0008) [2023-12-27 13:12:12,082][20381] Updated weights for policy 1, policy_version 1089334 (0.0008) [2023-12-27 13:12:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 558063616. Throughput: 0: 10444.6, 1: 10373.5. Samples: 558056132. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:12,363][19170] Avg episode reward: [(0, '990.210'), (1, '357.490')] [2023-12-27 13:12:12,440][20347] Updated weights for policy 0, policy_version 1090316 (0.0009) [2023-12-27 13:12:12,496][20347] Updated weights for policy 0, policy_version 1090326 (0.0008) [2023-12-27 13:12:12,548][20347] Updated weights for policy 0, policy_version 1090336 (0.0008) [2023-12-27 13:12:12,823][20381] Updated weights for policy 1, policy_version 1089344 (0.0008) [2023-12-27 13:12:12,879][20381] Updated weights for policy 1, policy_version 1089354 (0.0007) [2023-12-27 13:12:12,931][20381] Updated weights for policy 1, policy_version 1089364 (0.0007) [2023-12-27 13:12:13,223][20347] Updated weights for policy 0, policy_version 1090346 (0.0007) [2023-12-27 13:12:13,284][20347] Updated weights for policy 0, policy_version 1090356 (0.0005) [2023-12-27 13:12:13,333][20347] Updated weights for policy 0, policy_version 1090366 (0.0005) [2023-12-27 13:12:13,380][20347] Updated weights for policy 0, policy_version 1090376 (0.0005) [2023-12-27 13:12:13,522][20381] Updated weights for policy 1, policy_version 1089374 (0.0009) [2023-12-27 13:12:13,583][20381] Updated weights for policy 1, policy_version 1089384 (0.0010) [2023-12-27 13:12:13,643][20381] Updated weights for policy 1, policy_version 1089394 (0.0008) [2023-12-27 13:12:13,951][20347] Updated weights for policy 0, policy_version 1090386 (0.0010) [2023-12-27 13:12:14,007][20347] Updated weights for policy 0, policy_version 1090396 (0.0008) [2023-12-27 13:12:14,061][20347] Updated weights for policy 0, policy_version 1090406 (0.0005) [2023-12-27 13:12:14,262][20381] Updated weights for policy 1, policy_version 1089404 (0.0008) [2023-12-27 13:12:14,319][20381] Updated weights for policy 1, policy_version 1089414 (0.0007) [2023-12-27 13:12:14,374][20381] Updated weights for policy 1, policy_version 1089424 (0.0005) [2023-12-27 13:12:14,652][20347] Updated weights for policy 0, policy_version 1090416 (0.0005) [2023-12-27 13:12:14,704][20347] Updated weights for policy 0, policy_version 1090426 (0.0005) [2023-12-27 13:12:14,763][20347] Updated weights for policy 0, policy_version 1090436 (0.0007) [2023-12-27 13:12:14,989][20381] Updated weights for policy 1, policy_version 1089434 (0.0006) [2023-12-27 13:12:15,055][20381] Updated weights for policy 1, policy_version 1089444 (0.0010) [2023-12-27 13:12:15,113][20381] Updated weights for policy 1, policy_version 1089454 (0.0006) [2023-12-27 13:12:15,177][20381] Updated weights for policy 1, policy_version 1089464 (0.0006) [2023-12-27 13:12:15,463][20347] Updated weights for policy 0, policy_version 1090446 (0.0009) [2023-12-27 13:12:15,518][20347] Updated weights for policy 0, policy_version 1090456 (0.0010) [2023-12-27 13:12:15,573][20347] Updated weights for policy 0, policy_version 1090466 (0.0010) [2023-12-27 13:12:15,813][20381] Updated weights for policy 1, policy_version 1089474 (0.0011) [2023-12-27 13:12:15,861][20381] Updated weights for policy 1, policy_version 1089484 (0.0008) [2023-12-27 13:12:15,910][20381] Updated weights for policy 1, policy_version 1089494 (0.0010) [2023-12-27 13:12:16,198][20347] Updated weights for policy 0, policy_version 1090476 (0.0011) [2023-12-27 13:12:16,262][20347] Updated weights for policy 0, policy_version 1090486 (0.0010) [2023-12-27 13:12:16,321][20347] Updated weights for policy 0, policy_version 1090496 (0.0010) [2023-12-27 13:12:16,531][20381] Updated weights for policy 1, policy_version 1089504 (0.0010) [2023-12-27 13:12:16,587][20381] Updated weights for policy 1, policy_version 1089514 (0.0010) [2023-12-27 13:12:16,648][20381] Updated weights for policy 1, policy_version 1089524 (0.0009) [2023-12-27 13:12:16,951][20347] Updated weights for policy 0, policy_version 1090506 (0.0009) [2023-12-27 13:12:17,007][20347] Updated weights for policy 0, policy_version 1090516 (0.0006) [2023-12-27 13:12:17,068][20347] Updated weights for policy 0, policy_version 1090526 (0.0010) [2023-12-27 13:12:17,125][20347] Updated weights for policy 0, policy_version 1090536 (0.0010) [2023-12-27 13:12:17,279][20381] Updated weights for policy 1, policy_version 1089534 (0.0008) [2023-12-27 13:12:17,341][20381] Updated weights for policy 1, policy_version 1089544 (0.0007) [2023-12-27 13:12:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 558170112. Throughput: 0: 10436.7, 1: 10316.9. Samples: 558183720. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:17,363][19170] Avg episode reward: [(0, '970.660'), (1, '407.010')] [2023-12-27 13:12:17,392][20381] Updated weights for policy 1, policy_version 1089554 (0.0005) [2023-12-27 13:12:17,702][20347] Updated weights for policy 0, policy_version 1090546 (0.0010) [2023-12-27 13:12:17,760][20347] Updated weights for policy 0, policy_version 1090556 (0.0011) [2023-12-27 13:12:17,809][20347] Updated weights for policy 0, policy_version 1090566 (0.0010) [2023-12-27 13:12:18,026][20381] Updated weights for policy 1, policy_version 1089564 (0.0007) [2023-12-27 13:12:18,079][20381] Updated weights for policy 1, policy_version 1089574 (0.0010) [2023-12-27 13:12:18,128][20381] Updated weights for policy 1, policy_version 1089584 (0.0011) [2023-12-27 13:12:18,524][20347] Updated weights for policy 0, policy_version 1090576 (0.0010) [2023-12-27 13:12:18,587][20347] Updated weights for policy 0, policy_version 1090586 (0.0010) [2023-12-27 13:12:18,644][20347] Updated weights for policy 0, policy_version 1090596 (0.0010) [2023-12-27 13:12:18,791][20381] Updated weights for policy 1, policy_version 1089594 (0.0010) [2023-12-27 13:12:18,846][20381] Updated weights for policy 1, policy_version 1089604 (0.0006) [2023-12-27 13:12:18,898][20381] Updated weights for policy 1, policy_version 1089614 (0.0005) [2023-12-27 13:12:18,958][20381] Updated weights for policy 1, policy_version 1089624 (0.0009) [2023-12-27 13:12:19,331][20347] Updated weights for policy 0, policy_version 1090606 (0.0009) [2023-12-27 13:12:19,390][20347] Updated weights for policy 0, policy_version 1090616 (0.0008) [2023-12-27 13:12:19,455][20347] Updated weights for policy 0, policy_version 1090626 (0.0011) [2023-12-27 13:12:19,657][20381] Updated weights for policy 1, policy_version 1089634 (0.0007) [2023-12-27 13:12:19,724][20381] Updated weights for policy 1, policy_version 1089644 (0.0007) [2023-12-27 13:12:19,788][20381] Updated weights for policy 1, policy_version 1089654 (0.0008) [2023-12-27 13:12:20,141][20347] Updated weights for policy 0, policy_version 1090636 (0.0008) [2023-12-27 13:12:20,204][20347] Updated weights for policy 0, policy_version 1090646 (0.0008) [2023-12-27 13:12:20,260][20347] Updated weights for policy 0, policy_version 1090656 (0.0008) [2023-12-27 13:12:20,493][20381] Updated weights for policy 1, policy_version 1089664 (0.0007) [2023-12-27 13:12:20,551][20381] Updated weights for policy 1, policy_version 1089674 (0.0006) [2023-12-27 13:12:20,616][20381] Updated weights for policy 1, policy_version 1089684 (0.0007) [2023-12-27 13:12:20,944][20347] Updated weights for policy 0, policy_version 1090666 (0.0007) [2023-12-27 13:12:21,004][20347] Updated weights for policy 0, policy_version 1090676 (0.0008) [2023-12-27 13:12:21,067][20347] Updated weights for policy 0, policy_version 1090686 (0.0008) [2023-12-27 13:12:21,122][20347] Updated weights for policy 0, policy_version 1090696 (0.0006) [2023-12-27 13:12:21,311][20381] Updated weights for policy 1, policy_version 1089694 (0.0009) [2023-12-27 13:12:21,374][20381] Updated weights for policy 1, policy_version 1089704 (0.0009) [2023-12-27 13:12:21,448][20381] Updated weights for policy 1, policy_version 1089714 (0.0009) [2023-12-27 13:12:21,864][20347] Updated weights for policy 0, policy_version 1090706 (0.0005) [2023-12-27 13:12:21,929][20347] Updated weights for policy 0, policy_version 1090716 (0.0006) [2023-12-27 13:12:21,996][20347] Updated weights for policy 0, policy_version 1090726 (0.0006) [2023-12-27 13:12:22,198][20381] Updated weights for policy 1, policy_version 1089724 (0.0007) [2023-12-27 13:12:22,257][20381] Updated weights for policy 1, policy_version 1089734 (0.0006) [2023-12-27 13:12:22,319][20381] Updated weights for policy 1, policy_version 1089744 (0.0008) [2023-12-27 13:12:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 558268416. Throughput: 0: 10429.6, 1: 10319.2. Samples: 558245044. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:22,363][19170] Avg episode reward: [(0, '960.900'), (1, '544.030')] [2023-12-27 13:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001090728_279273472.pth... [2023-12-27 13:12:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001089752_279003136.pth... [2023-12-27 13:12:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001089512_278962176.pth [2023-12-27 13:12:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001088536_278691840.pth [2023-12-27 13:12:22,550][20347] Updated weights for policy 0, policy_version 1090736 (0.0007) [2023-12-27 13:12:22,607][20347] Updated weights for policy 0, policy_version 1090746 (0.0008) [2023-12-27 13:12:22,667][20347] Updated weights for policy 0, policy_version 1090756 (0.0008) [2023-12-27 13:12:22,998][20381] Updated weights for policy 1, policy_version 1089754 (0.0007) [2023-12-27 13:12:23,059][20381] Updated weights for policy 1, policy_version 1089764 (0.0006) [2023-12-27 13:12:23,123][20381] Updated weights for policy 1, policy_version 1089774 (0.0007) [2023-12-27 13:12:23,183][20381] Updated weights for policy 1, policy_version 1089784 (0.0008) [2023-12-27 13:12:23,324][20347] Updated weights for policy 0, policy_version 1090766 (0.0006) [2023-12-27 13:12:23,379][20347] Updated weights for policy 0, policy_version 1090776 (0.0005) [2023-12-27 13:12:23,439][20347] Updated weights for policy 0, policy_version 1090786 (0.0005) [2023-12-27 13:12:23,771][20381] Updated weights for policy 1, policy_version 1089794 (0.0011) [2023-12-27 13:12:23,827][20381] Updated weights for policy 1, policy_version 1089804 (0.0011) [2023-12-27 13:12:23,882][20381] Updated weights for policy 1, policy_version 1089814 (0.0010) [2023-12-27 13:12:24,045][20347] Updated weights for policy 0, policy_version 1090796 (0.0007) [2023-12-27 13:12:24,095][20347] Updated weights for policy 0, policy_version 1090806 (0.0007) [2023-12-27 13:12:24,154][20347] Updated weights for policy 0, policy_version 1090816 (0.0008) [2023-12-27 13:12:24,560][20381] Updated weights for policy 1, policy_version 1089824 (0.0011) [2023-12-27 13:12:24,616][20381] Updated weights for policy 1, policy_version 1089834 (0.0011) [2023-12-27 13:12:24,671][20381] Updated weights for policy 1, policy_version 1089844 (0.0010) [2023-12-27 13:12:24,839][20347] Updated weights for policy 0, policy_version 1090826 (0.0008) [2023-12-27 13:12:24,897][20347] Updated weights for policy 0, policy_version 1090836 (0.0008) [2023-12-27 13:12:24,947][20347] Updated weights for policy 0, policy_version 1090846 (0.0008) [2023-12-27 13:12:24,991][20347] Updated weights for policy 0, policy_version 1090856 (0.0007) [2023-12-27 13:12:25,335][20381] Updated weights for policy 1, policy_version 1089854 (0.0010) [2023-12-27 13:12:25,395][20381] Updated weights for policy 1, policy_version 1089864 (0.0011) [2023-12-27 13:12:25,447][20381] Updated weights for policy 1, policy_version 1089874 (0.0008) [2023-12-27 13:12:25,700][20347] Updated weights for policy 0, policy_version 1090866 (0.0008) [2023-12-27 13:12:25,748][20347] Updated weights for policy 0, policy_version 1090876 (0.0007) [2023-12-27 13:12:25,796][20347] Updated weights for policy 0, policy_version 1090886 (0.0008) [2023-12-27 13:12:26,100][20381] Updated weights for policy 1, policy_version 1089884 (0.0008) [2023-12-27 13:12:26,152][20381] Updated weights for policy 1, policy_version 1089894 (0.0006) [2023-12-27 13:12:26,200][20381] Updated weights for policy 1, policy_version 1089904 (0.0010) [2023-12-27 13:12:26,518][20347] Updated weights for policy 0, policy_version 1090896 (0.0010) [2023-12-27 13:12:26,573][20347] Updated weights for policy 0, policy_version 1090906 (0.0010) [2023-12-27 13:12:26,622][20347] Updated weights for policy 0, policy_version 1090916 (0.0010) [2023-12-27 13:12:26,787][20381] Updated weights for policy 1, policy_version 1089914 (0.0010) [2023-12-27 13:12:26,834][20381] Updated weights for policy 1, policy_version 1089924 (0.0008) [2023-12-27 13:12:26,891][20381] Updated weights for policy 1, policy_version 1089934 (0.0007) [2023-12-27 13:12:26,947][20381] Updated weights for policy 1, policy_version 1089944 (0.0010) [2023-12-27 13:12:27,272][20347] Updated weights for policy 0, policy_version 1090926 (0.0007) [2023-12-27 13:12:27,331][20347] Updated weights for policy 0, policy_version 1090936 (0.0007) [2023-12-27 13:12:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 558374912. Throughput: 0: 10395.0, 1: 10293.2. Samples: 558367108. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:27,363][19170] Avg episode reward: [(0, '980.600'), (1, '341.330')] [2023-12-27 13:12:27,376][20347] Updated weights for policy 0, policy_version 1090946 (0.0008) [2023-12-27 13:12:27,500][20381] Updated weights for policy 1, policy_version 1089954 (0.0010) [2023-12-27 13:12:27,553][20381] Updated weights for policy 1, policy_version 1089964 (0.0009) [2023-12-27 13:12:27,614][20381] Updated weights for policy 1, policy_version 1089974 (0.0005) [2023-12-27 13:12:28,003][20347] Updated weights for policy 0, policy_version 1090956 (0.0007) [2023-12-27 13:12:28,052][20347] Updated weights for policy 0, policy_version 1090966 (0.0009) [2023-12-27 13:12:28,110][20347] Updated weights for policy 0, policy_version 1090976 (0.0006) [2023-12-27 13:12:28,125][20291] KL-divergence is very high: 543.4841 [2023-12-27 13:12:28,177][20381] Updated weights for policy 1, policy_version 1089984 (0.0007) [2023-12-27 13:12:28,232][20381] Updated weights for policy 1, policy_version 1089994 (0.0007) [2023-12-27 13:12:28,293][20381] Updated weights for policy 1, policy_version 1090004 (0.0008) [2023-12-27 13:12:28,773][20347] Updated weights for policy 0, policy_version 1090986 (0.0011) [2023-12-27 13:12:28,833][20347] Updated weights for policy 0, policy_version 1090996 (0.0010) [2023-12-27 13:12:28,888][20347] Updated weights for policy 0, policy_version 1091006 (0.0010) [2023-12-27 13:12:28,945][20347] Updated weights for policy 0, policy_version 1091016 (0.0010) [2023-12-27 13:12:28,963][20381] Updated weights for policy 1, policy_version 1090014 (0.0011) [2023-12-27 13:12:29,014][20381] Updated weights for policy 1, policy_version 1090024 (0.0010) [2023-12-27 13:12:29,065][20381] Updated weights for policy 1, policy_version 1090034 (0.0010) [2023-12-27 13:12:29,652][20347] Updated weights for policy 0, policy_version 1091026 (0.0008) [2023-12-27 13:12:29,715][20347] Updated weights for policy 0, policy_version 1091036 (0.0007) [2023-12-27 13:12:29,740][20381] Updated weights for policy 1, policy_version 1090044 (0.0008) [2023-12-27 13:12:29,776][20347] Updated weights for policy 0, policy_version 1091046 (0.0008) [2023-12-27 13:12:29,799][20381] Updated weights for policy 1, policy_version 1090054 (0.0006) [2023-12-27 13:12:29,864][20381] Updated weights for policy 1, policy_version 1090064 (0.0006) [2023-12-27 13:12:30,441][20347] Updated weights for policy 0, policy_version 1091056 (0.0009) [2023-12-27 13:12:30,495][20347] Updated weights for policy 0, policy_version 1091066 (0.0010) [2023-12-27 13:12:30,497][20381] Updated weights for policy 1, policy_version 1090074 (0.0007) [2023-12-27 13:12:30,542][20347] Updated weights for policy 0, policy_version 1091076 (0.0010) [2023-12-27 13:12:30,553][20381] Updated weights for policy 1, policy_version 1090084 (0.0005) [2023-12-27 13:12:30,610][20381] Updated weights for policy 1, policy_version 1090094 (0.0006) [2023-12-27 13:12:30,661][20381] Updated weights for policy 1, policy_version 1090104 (0.0010) [2023-12-27 13:12:31,243][20347] Updated weights for policy 0, policy_version 1091086 (0.0010) [2023-12-27 13:12:31,284][20381] Updated weights for policy 1, policy_version 1090114 (0.0008) [2023-12-27 13:12:31,303][20347] Updated weights for policy 0, policy_version 1091096 (0.0010) [2023-12-27 13:12:31,341][20381] Updated weights for policy 1, policy_version 1090124 (0.0006) [2023-12-27 13:12:31,373][20347] Updated weights for policy 0, policy_version 1091106 (0.0008) [2023-12-27 13:12:31,406][20381] Updated weights for policy 1, policy_version 1090134 (0.0007) [2023-12-27 13:12:32,065][20347] Updated weights for policy 0, policy_version 1091116 (0.0009) [2023-12-27 13:12:32,094][20381] Updated weights for policy 1, policy_version 1090144 (0.0007) [2023-12-27 13:12:32,120][20347] Updated weights for policy 0, policy_version 1091126 (0.0011) [2023-12-27 13:12:32,152][20381] Updated weights for policy 1, policy_version 1090154 (0.0007) [2023-12-27 13:12:32,173][20347] Updated weights for policy 0, policy_version 1091136 (0.0011) [2023-12-27 13:12:32,214][20381] Updated weights for policy 1, policy_version 1090164 (0.0006) [2023-12-27 13:12:32,362][19170] Fps is (10 sec: 22118.6, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 558489600. Throughput: 0: 10397.3, 1: 10361.6. Samples: 558494292. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:32,363][19170] Avg episode reward: [(0, '980.450'), (1, '484.170')] [2023-12-27 13:12:32,807][20381] Updated weights for policy 1, policy_version 1090174 (0.0007) [2023-12-27 13:12:32,871][20381] Updated weights for policy 1, policy_version 1090184 (0.0007) [2023-12-27 13:12:32,928][20381] Updated weights for policy 1, policy_version 1090194 (0.0005) [2023-12-27 13:12:32,933][20347] Updated weights for policy 0, policy_version 1091146 (0.0010) [2023-12-27 13:12:32,988][20347] Updated weights for policy 0, policy_version 1091156 (0.0008) [2023-12-27 13:12:33,033][20347] Updated weights for policy 0, policy_version 1091166 (0.0007) [2023-12-27 13:12:33,083][20347] Updated weights for policy 0, policy_version 1091176 (0.0007) [2023-12-27 13:12:33,554][20381] Updated weights for policy 1, policy_version 1090204 (0.0006) [2023-12-27 13:12:33,601][20381] Updated weights for policy 1, policy_version 1090214 (0.0008) [2023-12-27 13:12:33,656][20381] Updated weights for policy 1, policy_version 1090224 (0.0007) [2023-12-27 13:12:33,789][20347] Updated weights for policy 0, policy_version 1091186 (0.0007) [2023-12-27 13:12:33,842][20347] Updated weights for policy 0, policy_version 1091196 (0.0007) [2023-12-27 13:12:33,898][20347] Updated weights for policy 0, policy_version 1091206 (0.0007) [2023-12-27 13:12:34,354][20381] Updated weights for policy 1, policy_version 1090234 (0.0008) [2023-12-27 13:12:34,427][20381] Updated weights for policy 1, policy_version 1090244 (0.0008) [2023-12-27 13:12:34,493][20381] Updated weights for policy 1, policy_version 1090254 (0.0007) [2023-12-27 13:12:34,562][20381] Updated weights for policy 1, policy_version 1090264 (0.0007) [2023-12-27 13:12:34,576][20347] Updated weights for policy 0, policy_version 1091216 (0.0006) [2023-12-27 13:12:34,641][20347] Updated weights for policy 0, policy_version 1091226 (0.0005) [2023-12-27 13:12:34,697][20347] Updated weights for policy 0, policy_version 1091236 (0.0005) [2023-12-27 13:12:35,257][20381] Updated weights for policy 1, policy_version 1090274 (0.0008) [2023-12-27 13:12:35,318][20381] Updated weights for policy 1, policy_version 1090284 (0.0007) [2023-12-27 13:12:35,322][20347] Updated weights for policy 0, policy_version 1091246 (0.0008) [2023-12-27 13:12:35,374][20381] Updated weights for policy 1, policy_version 1090294 (0.0008) [2023-12-27 13:12:35,376][20347] Updated weights for policy 0, policy_version 1091256 (0.0009) [2023-12-27 13:12:35,433][20347] Updated weights for policy 0, policy_version 1091266 (0.0006) [2023-12-27 13:12:35,956][20381] Updated weights for policy 1, policy_version 1090304 (0.0005) [2023-12-27 13:12:36,005][20381] Updated weights for policy 1, policy_version 1090314 (0.0005) [2023-12-27 13:12:36,052][20381] Updated weights for policy 1, policy_version 1090324 (0.0005) [2023-12-27 13:12:36,119][20347] Updated weights for policy 0, policy_version 1091276 (0.0008) [2023-12-27 13:12:36,184][20347] Updated weights for policy 0, policy_version 1091286 (0.0008) [2023-12-27 13:12:36,234][20347] Updated weights for policy 0, policy_version 1091296 (0.0008) [2023-12-27 13:12:36,748][20381] Updated weights for policy 1, policy_version 1090334 (0.0007) [2023-12-27 13:12:36,801][20381] Updated weights for policy 1, policy_version 1090344 (0.0008) [2023-12-27 13:12:36,858][20381] Updated weights for policy 1, policy_version 1090354 (0.0006) [2023-12-27 13:12:36,924][20347] Updated weights for policy 0, policy_version 1091306 (0.0007) [2023-12-27 13:12:36,980][20347] Updated weights for policy 0, policy_version 1091316 (0.0008) [2023-12-27 13:12:37,040][20347] Updated weights for policy 0, policy_version 1091326 (0.0008) [2023-12-27 13:12:37,102][20347] Updated weights for policy 0, policy_version 1091336 (0.0008) [2023-12-27 13:12:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20577.2). Total num frames: 558587904. Throughput: 0: 10403.2, 1: 10364.0. Samples: 558555644. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:37,363][19170] Avg episode reward: [(0, '961.150'), (1, '446.270')] [2023-12-27 13:12:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001090360_279158784.pth... [2023-12-27 13:12:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001091336_279429120.pth... [2023-12-27 13:12:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001089144_278847488.pth [2023-12-27 13:12:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001090120_279117824.pth [2023-12-27 13:12:37,507][20381] Updated weights for policy 1, policy_version 1090364 (0.0006) [2023-12-27 13:12:37,568][20381] Updated weights for policy 1, policy_version 1090374 (0.0007) [2023-12-27 13:12:37,625][20381] Updated weights for policy 1, policy_version 1090384 (0.0007) [2023-12-27 13:12:37,793][20347] Updated weights for policy 0, policy_version 1091346 (0.0007) [2023-12-27 13:12:37,843][20347] Updated weights for policy 0, policy_version 1091356 (0.0008) [2023-12-27 13:12:37,894][20347] Updated weights for policy 0, policy_version 1091366 (0.0007) [2023-12-27 13:12:38,302][20381] Updated weights for policy 1, policy_version 1090394 (0.0008) [2023-12-27 13:12:38,367][20381] Updated weights for policy 1, policy_version 1090404 (0.0007) [2023-12-27 13:12:38,425][20381] Updated weights for policy 1, policy_version 1090414 (0.0006) [2023-12-27 13:12:38,486][20381] Updated weights for policy 1, policy_version 1090424 (0.0007) [2023-12-27 13:12:38,583][20347] Updated weights for policy 0, policy_version 1091376 (0.0008) [2023-12-27 13:12:38,651][20347] Updated weights for policy 0, policy_version 1091386 (0.0008) [2023-12-27 13:12:38,713][20347] Updated weights for policy 0, policy_version 1091396 (0.0008) [2023-12-27 13:12:39,085][20381] Updated weights for policy 1, policy_version 1090434 (0.0005) [2023-12-27 13:12:39,133][20381] Updated weights for policy 1, policy_version 1090444 (0.0005) [2023-12-27 13:12:39,187][20381] Updated weights for policy 1, policy_version 1090454 (0.0005) [2023-12-27 13:12:39,453][20347] Updated weights for policy 0, policy_version 1091406 (0.0008) [2023-12-27 13:12:39,506][20347] Updated weights for policy 0, policy_version 1091416 (0.0008) [2023-12-27 13:12:39,565][20347] Updated weights for policy 0, policy_version 1091426 (0.0011) [2023-12-27 13:12:39,959][20381] Updated weights for policy 1, policy_version 1090464 (0.0007) [2023-12-27 13:12:40,013][20381] Updated weights for policy 1, policy_version 1090474 (0.0007) [2023-12-27 13:12:40,075][20381] Updated weights for policy 1, policy_version 1090484 (0.0007) [2023-12-27 13:12:40,276][20347] Updated weights for policy 0, policy_version 1091436 (0.0010) [2023-12-27 13:12:40,338][20347] Updated weights for policy 0, policy_version 1091446 (0.0011) [2023-12-27 13:12:40,402][20347] Updated weights for policy 0, policy_version 1091456 (0.0011) [2023-12-27 13:12:40,797][20381] Updated weights for policy 1, policy_version 1090494 (0.0008) [2023-12-27 13:12:40,857][20381] Updated weights for policy 1, policy_version 1090504 (0.0008) [2023-12-27 13:12:40,913][20381] Updated weights for policy 1, policy_version 1090514 (0.0008) [2023-12-27 13:12:41,125][20347] Updated weights for policy 0, policy_version 1091466 (0.0010) [2023-12-27 13:12:41,189][20347] Updated weights for policy 0, policy_version 1091476 (0.0006) [2023-12-27 13:12:41,214][20291] KL-divergence is very high: 201.1757 [2023-12-27 13:12:41,220][20291] KL-divergence is very high: 204.1576 [2023-12-27 13:12:41,252][20347] Updated weights for policy 0, policy_version 1091486 (0.0006) [2023-12-27 13:12:41,265][20291] KL-divergence is very high: 173.8313 [2023-12-27 13:12:41,271][20291] KL-divergence is very high: 144.4353 [2023-12-27 13:12:41,314][20347] Updated weights for policy 0, policy_version 1091496 (0.0008) [2023-12-27 13:12:41,674][20381] Updated weights for policy 1, policy_version 1090524 (0.0008) [2023-12-27 13:12:41,738][20381] Updated weights for policy 1, policy_version 1090534 (0.0006) [2023-12-27 13:12:41,801][20381] Updated weights for policy 1, policy_version 1090544 (0.0008) [2023-12-27 13:12:41,974][20347] Updated weights for policy 0, policy_version 1091506 (0.0006) [2023-12-27 13:12:42,026][20347] Updated weights for policy 0, policy_version 1091516 (0.0007) [2023-12-27 13:12:42,078][20347] Updated weights for policy 0, policy_version 1091526 (0.0008) [2023-12-27 13:12:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 558686208. Throughput: 0: 10303.3, 1: 10368.0. Samples: 558677284. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:42,363][19170] Avg episode reward: [(0, '941.510'), (1, '486.840')] [2023-12-27 13:12:42,502][20381] Updated weights for policy 1, policy_version 1090554 (0.0008) [2023-12-27 13:12:42,560][20381] Updated weights for policy 1, policy_version 1090564 (0.0007) [2023-12-27 13:12:42,615][20381] Updated weights for policy 1, policy_version 1090574 (0.0007) [2023-12-27 13:12:42,674][20381] Updated weights for policy 1, policy_version 1090584 (0.0007) [2023-12-27 13:12:42,790][20347] Updated weights for policy 0, policy_version 1091536 (0.0007) [2023-12-27 13:12:42,847][20347] Updated weights for policy 0, policy_version 1091546 (0.0007) [2023-12-27 13:12:42,902][20347] Updated weights for policy 0, policy_version 1091556 (0.0007) [2023-12-27 13:12:43,314][20381] Updated weights for policy 1, policy_version 1090594 (0.0007) [2023-12-27 13:12:43,368][20381] Updated weights for policy 1, policy_version 1090604 (0.0007) [2023-12-27 13:12:43,423][20381] Updated weights for policy 1, policy_version 1090614 (0.0008) [2023-12-27 13:12:43,533][20347] Updated weights for policy 0, policy_version 1091566 (0.0008) [2023-12-27 13:12:43,588][20347] Updated weights for policy 0, policy_version 1091576 (0.0007) [2023-12-27 13:12:43,646][20347] Updated weights for policy 0, policy_version 1091586 (0.0008) [2023-12-27 13:12:44,068][20381] Updated weights for policy 1, policy_version 1090624 (0.0010) [2023-12-27 13:12:44,135][20381] Updated weights for policy 1, policy_version 1090634 (0.0010) [2023-12-27 13:12:44,186][20381] Updated weights for policy 1, policy_version 1090644 (0.0010) [2023-12-27 13:12:44,334][20347] Updated weights for policy 0, policy_version 1091596 (0.0008) [2023-12-27 13:12:44,383][20347] Updated weights for policy 0, policy_version 1091606 (0.0007) [2023-12-27 13:12:44,444][20347] Updated weights for policy 0, policy_version 1091616 (0.0006) [2023-12-27 13:12:44,915][20381] Updated weights for policy 1, policy_version 1090654 (0.0007) [2023-12-27 13:12:44,984][20381] Updated weights for policy 1, policy_version 1090664 (0.0007) [2023-12-27 13:12:45,046][20381] Updated weights for policy 1, policy_version 1090674 (0.0006) [2023-12-27 13:12:45,083][20347] Updated weights for policy 0, policy_version 1091626 (0.0007) [2023-12-27 13:12:45,149][20347] Updated weights for policy 0, policy_version 1091636 (0.0007) [2023-12-27 13:12:45,212][20347] Updated weights for policy 0, policy_version 1091646 (0.0006) [2023-12-27 13:12:45,287][20347] Updated weights for policy 0, policy_version 1091656 (0.0006) [2023-12-27 13:12:45,661][20381] Updated weights for policy 1, policy_version 1090684 (0.0006) [2023-12-27 13:12:45,710][20381] Updated weights for policy 1, policy_version 1090694 (0.0007) [2023-12-27 13:12:45,753][20381] Updated weights for policy 1, policy_version 1090704 (0.0010) [2023-12-27 13:12:45,791][20347] Updated weights for policy 0, policy_version 1091666 (0.0006) [2023-12-27 13:12:45,842][20347] Updated weights for policy 0, policy_version 1091676 (0.0007) [2023-12-27 13:12:45,894][20347] Updated weights for policy 0, policy_version 1091686 (0.0005) [2023-12-27 13:12:46,401][20381] Updated weights for policy 1, policy_version 1090714 (0.0010) [2023-12-27 13:12:46,468][20381] Updated weights for policy 1, policy_version 1090724 (0.0009) [2023-12-27 13:12:46,475][20347] Updated weights for policy 0, policy_version 1091696 (0.0006) [2023-12-27 13:12:46,520][20381] Updated weights for policy 1, policy_version 1090734 (0.0011) [2023-12-27 13:12:46,526][20347] Updated weights for policy 0, policy_version 1091706 (0.0005) [2023-12-27 13:12:46,579][20381] Updated weights for policy 1, policy_version 1090744 (0.0011) [2023-12-27 13:12:46,580][20347] Updated weights for policy 0, policy_version 1091716 (0.0005) [2023-12-27 13:12:47,201][20347] Updated weights for policy 0, policy_version 1091726 (0.0006) [2023-12-27 13:12:47,246][20381] Updated weights for policy 1, policy_version 1090754 (0.0005) [2023-12-27 13:12:47,256][20347] Updated weights for policy 0, policy_version 1091736 (0.0005) [2023-12-27 13:12:47,301][20381] Updated weights for policy 1, policy_version 1090764 (0.0005) [2023-12-27 13:12:47,315][20347] Updated weights for policy 0, policy_version 1091746 (0.0005) [2023-12-27 13:12:47,355][20381] Updated weights for policy 1, policy_version 1090774 (0.0010) [2023-12-27 13:12:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 558792704. Throughput: 0: 10398.4, 1: 10360.6. Samples: 558803392. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:47,363][19170] Avg episode reward: [(0, '951.200'), (1, '457.120')] [2023-12-27 13:12:47,941][20381] Updated weights for policy 1, policy_version 1090784 (0.0007) [2023-12-27 13:12:47,984][20347] Updated weights for policy 0, policy_version 1091756 (0.0006) [2023-12-27 13:12:47,999][20381] Updated weights for policy 1, policy_version 1090794 (0.0007) [2023-12-27 13:12:48,040][20347] Updated weights for policy 0, policy_version 1091766 (0.0008) [2023-12-27 13:12:48,054][20381] Updated weights for policy 1, policy_version 1090804 (0.0008) [2023-12-27 13:12:48,094][20347] Updated weights for policy 0, policy_version 1091776 (0.0006) [2023-12-27 13:12:48,685][20381] Updated weights for policy 1, policy_version 1090814 (0.0010) [2023-12-27 13:12:48,716][20347] Updated weights for policy 0, policy_version 1091786 (0.0007) [2023-12-27 13:12:48,742][20381] Updated weights for policy 1, policy_version 1090824 (0.0011) [2023-12-27 13:12:48,770][20347] Updated weights for policy 0, policy_version 1091796 (0.0006) [2023-12-27 13:12:48,796][20381] Updated weights for policy 1, policy_version 1090834 (0.0008) [2023-12-27 13:12:48,831][20347] Updated weights for policy 0, policy_version 1091806 (0.0008) [2023-12-27 13:12:48,891][20347] Updated weights for policy 0, policy_version 1091816 (0.0008) [2023-12-27 13:12:49,499][20381] Updated weights for policy 1, policy_version 1090844 (0.0007) [2023-12-27 13:12:49,558][20381] Updated weights for policy 1, policy_version 1090854 (0.0007) [2023-12-27 13:12:49,619][20381] Updated weights for policy 1, policy_version 1090864 (0.0007) [2023-12-27 13:12:49,672][20347] Updated weights for policy 0, policy_version 1091826 (0.0007) [2023-12-27 13:12:49,735][20347] Updated weights for policy 0, policy_version 1091836 (0.0007) [2023-12-27 13:12:49,796][20347] Updated weights for policy 0, policy_version 1091846 (0.0008) [2023-12-27 13:12:50,286][20381] Updated weights for policy 1, policy_version 1090874 (0.0006) [2023-12-27 13:12:50,340][20381] Updated weights for policy 1, policy_version 1090884 (0.0007) [2023-12-27 13:12:50,392][20381] Updated weights for policy 1, policy_version 1090894 (0.0008) [2023-12-27 13:12:50,444][20381] Updated weights for policy 1, policy_version 1090904 (0.0007) [2023-12-27 13:12:50,525][20347] Updated weights for policy 0, policy_version 1091856 (0.0008) [2023-12-27 13:12:50,588][20347] Updated weights for policy 0, policy_version 1091866 (0.0007) [2023-12-27 13:12:50,649][20347] Updated weights for policy 0, policy_version 1091876 (0.0008) [2023-12-27 13:12:51,179][20381] Updated weights for policy 1, policy_version 1090914 (0.0008) [2023-12-27 13:12:51,241][20381] Updated weights for policy 1, policy_version 1090924 (0.0008) [2023-12-27 13:12:51,302][20381] Updated weights for policy 1, policy_version 1090934 (0.0007) [2023-12-27 13:12:51,338][20347] Updated weights for policy 0, policy_version 1091886 (0.0009) [2023-12-27 13:12:51,409][20347] Updated weights for policy 0, policy_version 1091896 (0.0008) [2023-12-27 13:12:51,460][20347] Updated weights for policy 0, policy_version 1091906 (0.0006) [2023-12-27 13:12:52,021][20381] Updated weights for policy 1, policy_version 1090944 (0.0008) [2023-12-27 13:12:52,074][20381] Updated weights for policy 1, policy_version 1090954 (0.0008) [2023-12-27 13:12:52,127][20347] Updated weights for policy 0, policy_version 1091916 (0.0005) [2023-12-27 13:12:52,132][20381] Updated weights for policy 1, policy_version 1090964 (0.0009) [2023-12-27 13:12:52,185][20347] Updated weights for policy 0, policy_version 1091926 (0.0008) [2023-12-27 13:12:52,236][20347] Updated weights for policy 0, policy_version 1091936 (0.0008) [2023-12-27 13:12:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 558899200. Throughput: 0: 10367.9, 1: 10383.5. Samples: 558864776. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:52,363][19170] Avg episode reward: [(0, '990.220'), (1, '394.560')] [2023-12-27 13:12:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001090968_279314432.pth... [2023-12-27 13:12:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001091944_279584768.pth... [2023-12-27 13:12:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001089752_279003136.pth [2023-12-27 13:12:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001090728_279273472.pth [2023-12-27 13:12:52,862][20381] Updated weights for policy 1, policy_version 1090974 (0.0008) [2023-12-27 13:12:52,907][20381] Updated weights for policy 1, policy_version 1090984 (0.0008) [2023-12-27 13:12:52,931][20347] Updated weights for policy 0, policy_version 1091946 (0.0006) [2023-12-27 13:12:52,960][20381] Updated weights for policy 1, policy_version 1090994 (0.0008) [2023-12-27 13:12:52,985][20347] Updated weights for policy 0, policy_version 1091956 (0.0008) [2023-12-27 13:12:53,044][20347] Updated weights for policy 0, policy_version 1091966 (0.0007) [2023-12-27 13:12:53,095][20347] Updated weights for policy 0, policy_version 1091976 (0.0008) [2023-12-27 13:12:53,587][20381] Updated weights for policy 1, policy_version 1091004 (0.0010) [2023-12-27 13:12:53,636][20381] Updated weights for policy 1, policy_version 1091014 (0.0010) [2023-12-27 13:12:53,687][20381] Updated weights for policy 1, policy_version 1091024 (0.0010) [2023-12-27 13:12:53,731][20347] Updated weights for policy 0, policy_version 1091986 (0.0005) [2023-12-27 13:12:53,787][20347] Updated weights for policy 0, policy_version 1091996 (0.0007) [2023-12-27 13:12:53,841][20347] Updated weights for policy 0, policy_version 1092006 (0.0006) [2023-12-27 13:12:54,387][20381] Updated weights for policy 1, policy_version 1091034 (0.0009) [2023-12-27 13:12:54,443][20381] Updated weights for policy 1, policy_version 1091044 (0.0005) [2023-12-27 13:12:54,460][20347] Updated weights for policy 0, policy_version 1092016 (0.0010) [2023-12-27 13:12:54,500][20381] Updated weights for policy 1, policy_version 1091054 (0.0005) [2023-12-27 13:12:54,519][20347] Updated weights for policy 0, policy_version 1092026 (0.0010) [2023-12-27 13:12:54,559][20381] Updated weights for policy 1, policy_version 1091064 (0.0006) [2023-12-27 13:12:54,580][20347] Updated weights for policy 0, policy_version 1092036 (0.0011) [2023-12-27 13:12:55,223][20381] Updated weights for policy 1, policy_version 1091074 (0.0008) [2023-12-27 13:12:55,256][20347] Updated weights for policy 0, policy_version 1092046 (0.0008) [2023-12-27 13:12:55,279][20381] Updated weights for policy 1, policy_version 1091084 (0.0008) [2023-12-27 13:12:55,309][20347] Updated weights for policy 0, policy_version 1092056 (0.0007) [2023-12-27 13:12:55,340][20381] Updated weights for policy 1, policy_version 1091094 (0.0006) [2023-12-27 13:12:55,364][20347] Updated weights for policy 0, policy_version 1092066 (0.0008) [2023-12-27 13:12:55,958][20381] Updated weights for policy 1, policy_version 1091104 (0.0007) [2023-12-27 13:12:56,013][20381] Updated weights for policy 1, policy_version 1091114 (0.0008) [2023-12-27 13:12:56,071][20381] Updated weights for policy 1, policy_version 1091124 (0.0007) [2023-12-27 13:12:56,094][20347] Updated weights for policy 0, policy_version 1092076 (0.0008) [2023-12-27 13:12:56,152][20347] Updated weights for policy 0, policy_version 1092086 (0.0008) [2023-12-27 13:12:56,200][20347] Updated weights for policy 0, policy_version 1092096 (0.0007) [2023-12-27 13:12:56,753][20381] Updated weights for policy 1, policy_version 1091134 (0.0008) [2023-12-27 13:12:56,809][20381] Updated weights for policy 1, policy_version 1091144 (0.0007) [2023-12-27 13:12:56,866][20381] Updated weights for policy 1, policy_version 1091154 (0.0007) [2023-12-27 13:12:56,875][20347] Updated weights for policy 0, policy_version 1092106 (0.0008) [2023-12-27 13:12:56,923][20347] Updated weights for policy 0, policy_version 1092116 (0.0007) [2023-12-27 13:12:56,975][20347] Updated weights for policy 0, policy_version 1092126 (0.0006) [2023-12-27 13:12:57,019][20347] Updated weights for policy 0, policy_version 1092136 (0.0007) [2023-12-27 13:12:57,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 558997504. Throughput: 0: 10291.3, 1: 10387.0. Samples: 558986660. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:12:57,364][19170] Avg episode reward: [(0, '990.210'), (1, '218.130')] [2023-12-27 13:12:57,497][20381] Updated weights for policy 1, policy_version 1091164 (0.0006) [2023-12-27 13:12:57,547][20381] Updated weights for policy 1, policy_version 1091174 (0.0007) [2023-12-27 13:12:57,604][20381] Updated weights for policy 1, policy_version 1091184 (0.0006) [2023-12-27 13:12:57,684][20347] Updated weights for policy 0, policy_version 1092146 (0.0005) [2023-12-27 13:12:57,730][20347] Updated weights for policy 0, policy_version 1092156 (0.0005) [2023-12-27 13:12:57,777][20347] Updated weights for policy 0, policy_version 1092166 (0.0005) [2023-12-27 13:12:58,275][20381] Updated weights for policy 1, policy_version 1091194 (0.0006) [2023-12-27 13:12:58,325][20381] Updated weights for policy 1, policy_version 1091204 (0.0007) [2023-12-27 13:12:58,394][20381] Updated weights for policy 1, policy_version 1091214 (0.0007) [2023-12-27 13:12:58,453][20381] Updated weights for policy 1, policy_version 1091224 (0.0008) [2023-12-27 13:12:58,487][20347] Updated weights for policy 0, policy_version 1092176 (0.0007) [2023-12-27 13:12:58,541][20347] Updated weights for policy 0, policy_version 1092186 (0.0007) [2023-12-27 13:12:58,603][20347] Updated weights for policy 0, policy_version 1092196 (0.0008) [2023-12-27 13:12:59,231][20381] Updated weights for policy 1, policy_version 1091234 (0.0010) [2023-12-27 13:12:59,297][20381] Updated weights for policy 1, policy_version 1091244 (0.0009) [2023-12-27 13:12:59,361][20381] Updated weights for policy 1, policy_version 1091254 (0.0010) [2023-12-27 13:12:59,390][20347] Updated weights for policy 0, policy_version 1092206 (0.0007) [2023-12-27 13:12:59,444][20347] Updated weights for policy 0, policy_version 1092216 (0.0007) [2023-12-27 13:12:59,505][20347] Updated weights for policy 0, policy_version 1092226 (0.0007) [2023-12-27 13:13:00,077][20381] Updated weights for policy 1, policy_version 1091264 (0.0006) [2023-12-27 13:13:00,136][20381] Updated weights for policy 1, policy_version 1091274 (0.0008) [2023-12-27 13:13:00,195][20381] Updated weights for policy 1, policy_version 1091284 (0.0010) [2023-12-27 13:13:00,223][20347] Updated weights for policy 0, policy_version 1092236 (0.0007) [2023-12-27 13:13:00,282][20347] Updated weights for policy 0, policy_version 1092246 (0.0007) [2023-12-27 13:13:00,302][20291] KL-divergence is very high: 108.1143 [2023-12-27 13:13:00,308][20291] KL-divergence is very high: 105.9433 [2023-12-27 13:13:00,334][20347] Updated weights for policy 0, policy_version 1092256 (0.0007) [2023-12-27 13:13:00,852][20381] Updated weights for policy 1, policy_version 1091294 (0.0011) [2023-12-27 13:13:00,895][20381] Updated weights for policy 1, policy_version 1091304 (0.0008) [2023-12-27 13:13:00,940][20381] Updated weights for policy 1, policy_version 1091314 (0.0005) [2023-12-27 13:13:00,958][20347] Updated weights for policy 0, policy_version 1092266 (0.0008) [2023-12-27 13:13:01,009][20347] Updated weights for policy 0, policy_version 1092276 (0.0010) [2023-12-27 13:13:01,071][20347] Updated weights for policy 0, policy_version 1092286 (0.0008) [2023-12-27 13:13:01,135][20347] Updated weights for policy 0, policy_version 1092296 (0.0007) [2023-12-27 13:13:01,620][20381] Updated weights for policy 1, policy_version 1091324 (0.0007) [2023-12-27 13:13:01,686][20381] Updated weights for policy 1, policy_version 1091334 (0.0008) [2023-12-27 13:13:01,750][20381] Updated weights for policy 1, policy_version 1091344 (0.0010) [2023-12-27 13:13:01,820][20347] Updated weights for policy 0, policy_version 1092306 (0.0006) [2023-12-27 13:13:01,870][20347] Updated weights for policy 0, policy_version 1092316 (0.0008) [2023-12-27 13:13:01,917][20347] Updated weights for policy 0, policy_version 1092326 (0.0007) [2023-12-27 13:13:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 559095808. Throughput: 0: 10207.1, 1: 10331.4. Samples: 559107952. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:02,363][19170] Avg episode reward: [(0, '951.360'), (1, '521.770')] [2023-12-27 13:13:02,377][20381] Updated weights for policy 1, policy_version 1091354 (0.0011) [2023-12-27 13:13:02,437][20381] Updated weights for policy 1, policy_version 1091364 (0.0010) [2023-12-27 13:13:02,492][20381] Updated weights for policy 1, policy_version 1091374 (0.0010) [2023-12-27 13:13:02,540][20381] Updated weights for policy 1, policy_version 1091384 (0.0010) [2023-12-27 13:13:02,644][20347] Updated weights for policy 0, policy_version 1092336 (0.0007) [2023-12-27 13:13:02,699][20347] Updated weights for policy 0, policy_version 1092346 (0.0007) [2023-12-27 13:13:02,761][20347] Updated weights for policy 0, policy_version 1092356 (0.0007) [2023-12-27 13:13:03,205][20381] Updated weights for policy 1, policy_version 1091394 (0.0010) [2023-12-27 13:13:03,253][20381] Updated weights for policy 1, policy_version 1091404 (0.0010) [2023-12-27 13:13:03,306][20381] Updated weights for policy 1, policy_version 1091414 (0.0010) [2023-12-27 13:13:03,461][20347] Updated weights for policy 0, policy_version 1092366 (0.0006) [2023-12-27 13:13:03,514][20347] Updated weights for policy 0, policy_version 1092376 (0.0006) [2023-12-27 13:13:03,567][20347] Updated weights for policy 0, policy_version 1092386 (0.0007) [2023-12-27 13:13:03,992][20381] Updated weights for policy 1, policy_version 1091424 (0.0009) [2023-12-27 13:13:04,048][20381] Updated weights for policy 1, policy_version 1091434 (0.0007) [2023-12-27 13:13:04,093][20381] Updated weights for policy 1, policy_version 1091444 (0.0006) [2023-12-27 13:13:04,185][20347] Updated weights for policy 0, policy_version 1092396 (0.0006) [2023-12-27 13:13:04,249][20347] Updated weights for policy 0, policy_version 1092406 (0.0008) [2023-12-27 13:13:04,314][20347] Updated weights for policy 0, policy_version 1092416 (0.0009) [2023-12-27 13:13:04,804][20381] Updated weights for policy 1, policy_version 1091454 (0.0009) [2023-12-27 13:13:04,853][20381] Updated weights for policy 1, policy_version 1091464 (0.0009) [2023-12-27 13:13:04,910][20381] Updated weights for policy 1, policy_version 1091474 (0.0008) [2023-12-27 13:13:04,921][20347] Updated weights for policy 0, policy_version 1092426 (0.0007) [2023-12-27 13:13:04,974][20347] Updated weights for policy 0, policy_version 1092436 (0.0007) [2023-12-27 13:13:05,024][20347] Updated weights for policy 0, policy_version 1092446 (0.0008) [2023-12-27 13:13:05,074][20347] Updated weights for policy 0, policy_version 1092456 (0.0008) [2023-12-27 13:13:05,621][20381] Updated weights for policy 1, policy_version 1091484 (0.0007) [2023-12-27 13:13:05,672][20381] Updated weights for policy 1, policy_version 1091494 (0.0006) [2023-12-27 13:13:05,724][20381] Updated weights for policy 1, policy_version 1091504 (0.0005) [2023-12-27 13:13:05,737][20347] Updated weights for policy 0, policy_version 1092466 (0.0007) [2023-12-27 13:13:05,799][20347] Updated weights for policy 0, policy_version 1092476 (0.0005) [2023-12-27 13:13:05,855][20347] Updated weights for policy 0, policy_version 1092486 (0.0008) [2023-12-27 13:13:06,340][20381] Updated weights for policy 1, policy_version 1091514 (0.0005) [2023-12-27 13:13:06,399][20381] Updated weights for policy 1, policy_version 1091524 (0.0008) [2023-12-27 13:13:06,460][20381] Updated weights for policy 1, policy_version 1091534 (0.0007) [2023-12-27 13:13:06,523][20381] Updated weights for policy 1, policy_version 1091544 (0.0007) [2023-12-27 13:13:06,569][20347] Updated weights for policy 0, policy_version 1092496 (0.0010) [2023-12-27 13:13:06,627][20347] Updated weights for policy 0, policy_version 1092506 (0.0010) [2023-12-27 13:13:06,680][20347] Updated weights for policy 0, policy_version 1092516 (0.0010) [2023-12-27 13:13:07,237][20381] Updated weights for policy 1, policy_version 1091554 (0.0008) [2023-12-27 13:13:07,287][20381] Updated weights for policy 1, policy_version 1091564 (0.0008) [2023-12-27 13:13:07,343][20381] Updated weights for policy 1, policy_version 1091574 (0.0007) [2023-12-27 13:13:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 559202304. Throughput: 0: 10221.9, 1: 10323.7. Samples: 559169592. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:07,363][19170] Avg episode reward: [(0, '912.550'), (1, '352.100')] [2023-12-27 13:13:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001091576_279470080.pth... [2023-12-27 13:13:07,368][20347] Updated weights for policy 0, policy_version 1092526 (0.0007) [2023-12-27 13:13:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001090360_279158784.pth [2023-12-27 13:13:07,419][20347] Updated weights for policy 0, policy_version 1092536 (0.0008) [2023-12-27 13:13:07,471][20347] Updated weights for policy 0, policy_version 1092546 (0.0007) [2023-12-27 13:13:07,497][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001092552_279740416.pth... [2023-12-27 13:13:07,500][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001091336_279429120.pth [2023-12-27 13:13:08,029][20381] Updated weights for policy 1, policy_version 1091584 (0.0009) [2023-12-27 13:13:08,087][20381] Updated weights for policy 1, policy_version 1091594 (0.0010) [2023-12-27 13:13:08,152][20381] Updated weights for policy 1, policy_version 1091604 (0.0010) [2023-12-27 13:13:08,187][20347] Updated weights for policy 0, policy_version 1092556 (0.0008) [2023-12-27 13:13:08,236][20347] Updated weights for policy 0, policy_version 1092566 (0.0008) [2023-12-27 13:13:08,285][20347] Updated weights for policy 0, policy_version 1092576 (0.0010) [2023-12-27 13:13:08,867][20381] Updated weights for policy 1, policy_version 1091614 (0.0011) [2023-12-27 13:13:08,912][20347] Updated weights for policy 0, policy_version 1092586 (0.0010) [2023-12-27 13:13:08,923][20381] Updated weights for policy 1, policy_version 1091624 (0.0011) [2023-12-27 13:13:08,963][20347] Updated weights for policy 0, policy_version 1092596 (0.0010) [2023-12-27 13:13:08,979][20381] Updated weights for policy 1, policy_version 1091634 (0.0011) [2023-12-27 13:13:09,018][20347] Updated weights for policy 0, policy_version 1092606 (0.0010) [2023-12-27 13:13:09,075][20347] Updated weights for policy 0, policy_version 1092616 (0.0010) [2023-12-27 13:13:09,743][20381] Updated weights for policy 1, policy_version 1091644 (0.0011) [2023-12-27 13:13:09,767][20347] Updated weights for policy 0, policy_version 1092626 (0.0006) [2023-12-27 13:13:09,809][20381] Updated weights for policy 1, policy_version 1091654 (0.0011) [2023-12-27 13:13:09,829][20347] Updated weights for policy 0, policy_version 1092636 (0.0006) [2023-12-27 13:13:09,871][20381] Updated weights for policy 1, policy_version 1091664 (0.0011) [2023-12-27 13:13:09,891][20347] Updated weights for policy 0, policy_version 1092646 (0.0007) [2023-12-27 13:13:10,567][20381] Updated weights for policy 1, policy_version 1091674 (0.0011) [2023-12-27 13:13:10,598][20347] Updated weights for policy 0, policy_version 1092656 (0.0006) [2023-12-27 13:13:10,615][20381] Updated weights for policy 1, policy_version 1091684 (0.0010) [2023-12-27 13:13:10,636][20291] KL-divergence is very high: 117.7197 [2023-12-27 13:13:10,654][20347] Updated weights for policy 0, policy_version 1092666 (0.0005) [2023-12-27 13:13:10,674][20381] Updated weights for policy 1, policy_version 1091694 (0.0010) [2023-12-27 13:13:10,712][20347] Updated weights for policy 0, policy_version 1092676 (0.0005) [2023-12-27 13:13:10,733][20381] Updated weights for policy 1, policy_version 1091704 (0.0010) [2023-12-27 13:13:11,430][20347] Updated weights for policy 0, policy_version 1092686 (0.0007) [2023-12-27 13:13:11,492][20347] Updated weights for policy 0, policy_version 1092696 (0.0006) [2023-12-27 13:13:11,518][20381] Updated weights for policy 1, policy_version 1091714 (0.0008) [2023-12-27 13:13:11,557][20347] Updated weights for policy 0, policy_version 1092706 (0.0008) [2023-12-27 13:13:11,579][20381] Updated weights for policy 1, policy_version 1091724 (0.0006) [2023-12-27 13:13:11,639][20381] Updated weights for policy 1, policy_version 1091734 (0.0008) [2023-12-27 13:13:12,267][20347] Updated weights for policy 0, policy_version 1092716 (0.0007) [2023-12-27 13:13:12,330][20347] Updated weights for policy 0, policy_version 1092726 (0.0008) [2023-12-27 13:13:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 559292416. Throughput: 0: 10206.0, 1: 10313.2. Samples: 559290468. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:12,363][19170] Avg episode reward: [(0, '922.010'), (1, '489.610')] [2023-12-27 13:13:12,394][20347] Updated weights for policy 0, policy_version 1092736 (0.0008) [2023-12-27 13:13:12,407][20381] Updated weights for policy 1, policy_version 1091744 (0.0007) [2023-12-27 13:13:12,465][20381] Updated weights for policy 1, policy_version 1091754 (0.0008) [2023-12-27 13:13:12,521][20381] Updated weights for policy 1, policy_version 1091764 (0.0008) [2023-12-27 13:13:13,101][20347] Updated weights for policy 0, policy_version 1092746 (0.0007) [2023-12-27 13:13:13,155][20347] Updated weights for policy 0, policy_version 1092756 (0.0008) [2023-12-27 13:13:13,201][20381] Updated weights for policy 1, policy_version 1091774 (0.0006) [2023-12-27 13:13:13,209][20347] Updated weights for policy 0, policy_version 1092766 (0.0007) [2023-12-27 13:13:13,252][20381] Updated weights for policy 1, policy_version 1091784 (0.0005) [2023-12-27 13:13:13,260][20347] Updated weights for policy 0, policy_version 1092776 (0.0010) [2023-12-27 13:13:13,297][20381] Updated weights for policy 1, policy_version 1091794 (0.0005) [2023-12-27 13:13:13,872][20381] Updated weights for policy 1, policy_version 1091804 (0.0005) [2023-12-27 13:13:13,920][20381] Updated weights for policy 1, policy_version 1091814 (0.0006) [2023-12-27 13:13:13,928][20347] Updated weights for policy 0, policy_version 1092786 (0.0010) [2023-12-27 13:13:13,973][20381] Updated weights for policy 1, policy_version 1091824 (0.0008) [2023-12-27 13:13:13,979][20347] Updated weights for policy 0, policy_version 1092796 (0.0010) [2023-12-27 13:13:14,026][20347] Updated weights for policy 0, policy_version 1092806 (0.0010) [2023-12-27 13:13:14,647][20381] Updated weights for policy 1, policy_version 1091834 (0.0006) [2023-12-27 13:13:14,694][20347] Updated weights for policy 0, policy_version 1092816 (0.0010) [2023-12-27 13:13:14,705][20381] Updated weights for policy 1, policy_version 1091844 (0.0006) [2023-12-27 13:13:14,738][20347] Updated weights for policy 0, policy_version 1092826 (0.0010) [2023-12-27 13:13:14,761][20381] Updated weights for policy 1, policy_version 1091854 (0.0006) [2023-12-27 13:13:14,806][20347] Updated weights for policy 0, policy_version 1092836 (0.0010) [2023-12-27 13:13:14,826][20381] Updated weights for policy 1, policy_version 1091864 (0.0006) [2023-12-27 13:13:15,502][20347] Updated weights for policy 0, policy_version 1092846 (0.0010) [2023-12-27 13:13:15,546][20381] Updated weights for policy 1, policy_version 1091874 (0.0005) [2023-12-27 13:13:15,550][20347] Updated weights for policy 0, policy_version 1092856 (0.0010) [2023-12-27 13:13:15,603][20381] Updated weights for policy 1, policy_version 1091884 (0.0006) [2023-12-27 13:13:15,606][20347] Updated weights for policy 0, policy_version 1092866 (0.0010) [2023-12-27 13:13:15,664][20381] Updated weights for policy 1, policy_version 1091894 (0.0008) [2023-12-27 13:13:16,207][20347] Updated weights for policy 0, policy_version 1092876 (0.0007) [2023-12-27 13:13:16,261][20381] Updated weights for policy 1, policy_version 1091904 (0.0010) [2023-12-27 13:13:16,261][20347] Updated weights for policy 0, policy_version 1092886 (0.0010) [2023-12-27 13:13:16,319][20381] Updated weights for policy 1, policy_version 1091914 (0.0011) [2023-12-27 13:13:16,324][20347] Updated weights for policy 0, policy_version 1092896 (0.0006) [2023-12-27 13:13:16,375][20381] Updated weights for policy 1, policy_version 1091924 (0.0011) [2023-12-27 13:13:16,861][20347] Updated weights for policy 0, policy_version 1092906 (0.0005) [2023-12-27 13:13:16,910][20347] Updated weights for policy 0, policy_version 1092916 (0.0005) [2023-12-27 13:13:16,955][20347] Updated weights for policy 0, policy_version 1092926 (0.0005) [2023-12-27 13:13:16,997][20381] Updated weights for policy 1, policy_version 1091934 (0.0007) [2023-12-27 13:13:17,012][20347] Updated weights for policy 0, policy_version 1092936 (0.0005) [2023-12-27 13:13:17,055][20381] Updated weights for policy 1, policy_version 1091944 (0.0006) [2023-12-27 13:13:17,120][20381] Updated weights for policy 1, policy_version 1091954 (0.0011) [2023-12-27 13:13:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 559407104. Throughput: 0: 10238.8, 1: 10249.1. Samples: 559416244. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:17,363][19170] Avg episode reward: [(0, '970.610'), (1, '586.550')] [2023-12-27 13:13:17,540][20347] Updated weights for policy 0, policy_version 1092946 (0.0005) [2023-12-27 13:13:17,596][20347] Updated weights for policy 0, policy_version 1092956 (0.0005) [2023-12-27 13:13:17,646][20347] Updated weights for policy 0, policy_version 1092966 (0.0005) [2023-12-27 13:13:17,745][20381] Updated weights for policy 1, policy_version 1091964 (0.0009) [2023-12-27 13:13:17,793][20381] Updated weights for policy 1, policy_version 1091974 (0.0007) [2023-12-27 13:13:17,850][20381] Updated weights for policy 1, policy_version 1091984 (0.0007) [2023-12-27 13:13:18,204][20347] Updated weights for policy 0, policy_version 1092976 (0.0010) [2023-12-27 13:13:18,261][20347] Updated weights for policy 0, policy_version 1092986 (0.0010) [2023-12-27 13:13:18,315][20347] Updated weights for policy 0, policy_version 1092996 (0.0010) [2023-12-27 13:13:18,541][20381] Updated weights for policy 1, policy_version 1091994 (0.0007) [2023-12-27 13:13:18,599][20381] Updated weights for policy 1, policy_version 1092004 (0.0007) [2023-12-27 13:13:18,661][20381] Updated weights for policy 1, policy_version 1092014 (0.0008) [2023-12-27 13:13:18,722][20381] Updated weights for policy 1, policy_version 1092024 (0.0007) [2023-12-27 13:13:19,028][20347] Updated weights for policy 0, policy_version 1093006 (0.0007) [2023-12-27 13:13:19,079][20347] Updated weights for policy 0, policy_version 1093016 (0.0008) [2023-12-27 13:13:19,131][20347] Updated weights for policy 0, policy_version 1093026 (0.0010) [2023-12-27 13:13:19,362][20381] Updated weights for policy 1, policy_version 1092034 (0.0007) [2023-12-27 13:13:19,418][20381] Updated weights for policy 1, policy_version 1092044 (0.0007) [2023-12-27 13:13:19,476][20381] Updated weights for policy 1, policy_version 1092054 (0.0007) [2023-12-27 13:13:19,803][20347] Updated weights for policy 0, policy_version 1093036 (0.0007) [2023-12-27 13:13:19,873][20347] Updated weights for policy 0, policy_version 1093046 (0.0010) [2023-12-27 13:13:19,927][20347] Updated weights for policy 0, policy_version 1093056 (0.0010) [2023-12-27 13:13:20,196][20381] Updated weights for policy 1, policy_version 1092064 (0.0008) [2023-12-27 13:13:20,262][20381] Updated weights for policy 1, policy_version 1092074 (0.0008) [2023-12-27 13:13:20,318][20381] Updated weights for policy 1, policy_version 1092084 (0.0007) [2023-12-27 13:13:20,650][20347] Updated weights for policy 0, policy_version 1093066 (0.0010) [2023-12-27 13:13:20,701][20347] Updated weights for policy 0, policy_version 1093076 (0.0010) [2023-12-27 13:13:20,754][20347] Updated weights for policy 0, policy_version 1093086 (0.0011) [2023-12-27 13:13:20,772][20291] KL-divergence is very high: 129.7348 [2023-12-27 13:13:20,811][20347] Updated weights for policy 0, policy_version 1093096 (0.0010) [2023-12-27 13:13:21,107][20381] Updated weights for policy 1, policy_version 1092094 (0.0008) [2023-12-27 13:13:21,171][20381] Updated weights for policy 1, policy_version 1092104 (0.0009) [2023-12-27 13:13:21,239][20381] Updated weights for policy 1, policy_version 1092114 (0.0008) [2023-12-27 13:13:21,582][20347] Updated weights for policy 0, policy_version 1093106 (0.0011) [2023-12-27 13:13:21,644][20347] Updated weights for policy 0, policy_version 1093116 (0.0010) [2023-12-27 13:13:21,714][20347] Updated weights for policy 0, policy_version 1093126 (0.0008) [2023-12-27 13:13:21,963][20381] Updated weights for policy 1, policy_version 1092124 (0.0008) [2023-12-27 13:13:22,026][20381] Updated weights for policy 1, policy_version 1092134 (0.0007) [2023-12-27 13:13:22,089][20381] Updated weights for policy 1, policy_version 1092144 (0.0010) [2023-12-27 13:13:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 559505408. Throughput: 0: 10287.5, 1: 10222.2. Samples: 559478584. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:22,363][19170] Avg episode reward: [(0, '970.720'), (1, '584.670')] [2023-12-27 13:13:22,366][20347] Updated weights for policy 0, policy_version 1093136 (0.0008) [2023-12-27 13:13:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001092152_279617536.pth... [2023-12-27 13:13:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001090968_279314432.pth [2023-12-27 13:13:22,427][20347] Updated weights for policy 0, policy_version 1093146 (0.0008) [2023-12-27 13:13:22,490][20347] Updated weights for policy 0, policy_version 1093156 (0.0007) [2023-12-27 13:13:22,514][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001093160_279896064.pth... [2023-12-27 13:13:22,519][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001091944_279584768.pth [2023-12-27 13:13:22,785][20381] Updated weights for policy 1, policy_version 1092154 (0.0009) [2023-12-27 13:13:22,846][20381] Updated weights for policy 1, policy_version 1092164 (0.0005) [2023-12-27 13:13:22,893][20381] Updated weights for policy 1, policy_version 1092174 (0.0006) [2023-12-27 13:13:22,941][20381] Updated weights for policy 1, policy_version 1092184 (0.0005) [2023-12-27 13:13:23,233][20347] Updated weights for policy 0, policy_version 1093166 (0.0006) [2023-12-27 13:13:23,296][20347] Updated weights for policy 0, policy_version 1093176 (0.0005) [2023-12-27 13:13:23,352][20347] Updated weights for policy 0, policy_version 1093186 (0.0005) [2023-12-27 13:13:23,592][20381] Updated weights for policy 1, policy_version 1092194 (0.0007) [2023-12-27 13:13:23,643][20381] Updated weights for policy 1, policy_version 1092204 (0.0007) [2023-12-27 13:13:23,689][20381] Updated weights for policy 1, policy_version 1092214 (0.0006) [2023-12-27 13:13:23,933][20347] Updated weights for policy 0, policy_version 1093196 (0.0007) [2023-12-27 13:13:23,995][20347] Updated weights for policy 0, policy_version 1093206 (0.0005) [2023-12-27 13:13:24,045][20347] Updated weights for policy 0, policy_version 1093216 (0.0010) [2023-12-27 13:13:24,328][20381] Updated weights for policy 1, policy_version 1092224 (0.0007) [2023-12-27 13:13:24,376][20381] Updated weights for policy 1, policy_version 1092234 (0.0007) [2023-12-27 13:13:24,426][20381] Updated weights for policy 1, policy_version 1092244 (0.0007) [2023-12-27 13:13:24,633][20347] Updated weights for policy 0, policy_version 1093226 (0.0005) [2023-12-27 13:13:24,682][20347] Updated weights for policy 0, policy_version 1093236 (0.0005) [2023-12-27 13:13:24,742][20347] Updated weights for policy 0, policy_version 1093246 (0.0006) [2023-12-27 13:13:24,807][20347] Updated weights for policy 0, policy_version 1093256 (0.0009) [2023-12-27 13:13:25,142][20381] Updated weights for policy 1, policy_version 1092254 (0.0007) [2023-12-27 13:13:25,192][20381] Updated weights for policy 1, policy_version 1092264 (0.0007) [2023-12-27 13:13:25,247][20381] Updated weights for policy 1, policy_version 1092274 (0.0007) [2023-12-27 13:13:25,424][20347] Updated weights for policy 0, policy_version 1093266 (0.0008) [2023-12-27 13:13:25,481][20347] Updated weights for policy 0, policy_version 1093276 (0.0007) [2023-12-27 13:13:25,538][20347] Updated weights for policy 0, policy_version 1093286 (0.0007) [2023-12-27 13:13:25,930][20381] Updated weights for policy 1, policy_version 1092284 (0.0007) [2023-12-27 13:13:25,993][20381] Updated weights for policy 1, policy_version 1092294 (0.0008) [2023-12-27 13:13:26,044][20381] Updated weights for policy 1, policy_version 1092304 (0.0007) [2023-12-27 13:13:26,225][20347] Updated weights for policy 0, policy_version 1093296 (0.0008) [2023-12-27 13:13:26,281][20347] Updated weights for policy 0, policy_version 1093306 (0.0009) [2023-12-27 13:13:26,339][20347] Updated weights for policy 0, policy_version 1093316 (0.0008) [2023-12-27 13:13:26,638][20381] Updated weights for policy 1, policy_version 1092314 (0.0007) [2023-12-27 13:13:26,683][20381] Updated weights for policy 1, policy_version 1092324 (0.0007) [2023-12-27 13:13:26,732][20381] Updated weights for policy 1, policy_version 1092334 (0.0007) [2023-12-27 13:13:26,784][20381] Updated weights for policy 1, policy_version 1092344 (0.0008) [2023-12-27 13:13:27,015][20347] Updated weights for policy 0, policy_version 1093326 (0.0007) [2023-12-27 13:13:27,064][20347] Updated weights for policy 0, policy_version 1093336 (0.0010) [2023-12-27 13:13:27,111][20347] Updated weights for policy 0, policy_version 1093346 (0.0010) [2023-12-27 13:13:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 559611904. Throughput: 0: 10347.1, 1: 10188.4. Samples: 559601384. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-12-27 13:13:27,363][19170] Avg episode reward: [(0, '941.400'), (1, '509.620')] [2023-12-27 13:13:27,431][20381] Updated weights for policy 1, policy_version 1092354 (0.0007) [2023-12-27 13:13:27,484][20381] Updated weights for policy 1, policy_version 1092364 (0.0007) [2023-12-27 13:13:27,531][20381] Updated weights for policy 1, policy_version 1092374 (0.0007) [2023-12-27 13:13:27,802][20347] Updated weights for policy 0, policy_version 1093356 (0.0010) [2023-12-27 13:13:27,829][20291] KL-divergence is very high: 344.4208 [2023-12-27 13:13:27,853][20347] Updated weights for policy 0, policy_version 1093366 (0.0010) [2023-12-27 13:13:27,867][20291] KL-divergence is very high: 329.3213 [2023-12-27 13:13:27,903][20347] Updated weights for policy 0, policy_version 1093376 (0.0010) [2023-12-27 13:13:27,907][20291] KL-divergence is very high: 168.6164 [2023-12-27 13:13:28,219][20381] Updated weights for policy 1, policy_version 1092384 (0.0007) [2023-12-27 13:13:28,262][20381] Updated weights for policy 1, policy_version 1092394 (0.0007) [2023-12-27 13:13:28,312][20381] Updated weights for policy 1, policy_version 1092404 (0.0007) [2023-12-27 13:13:28,558][20347] Updated weights for policy 0, policy_version 1093386 (0.0009) [2023-12-27 13:13:28,606][20347] Updated weights for policy 0, policy_version 1093396 (0.0010) [2023-12-27 13:13:28,655][20347] Updated weights for policy 0, policy_version 1093406 (0.0010) [2023-12-27 13:13:28,710][20347] Updated weights for policy 0, policy_version 1093416 (0.0010) [2023-12-27 13:13:28,969][20381] Updated weights for policy 1, policy_version 1092415 (0.0008) [2023-12-27 13:13:29,017][20381] Updated weights for policy 1, policy_version 1092425 (0.0007) [2023-12-27 13:13:29,068][20381] Updated weights for policy 1, policy_version 1092435 (0.0008) [2023-12-27 13:13:29,400][20347] Updated weights for policy 0, policy_version 1093426 (0.0006) [2023-12-27 13:13:29,457][20291] KL-divergence is very high: 736.1064 [2023-12-27 13:13:29,461][20347] Updated weights for policy 0, policy_version 1093436 (0.0005) [2023-12-27 13:13:29,463][20291] KL-divergence is very high: 590.9233 [2023-12-27 13:13:29,524][20347] Updated weights for policy 0, policy_version 1093446 (0.0005) [2023-12-27 13:13:29,777][20381] Updated weights for policy 1, policy_version 1092445 (0.0006) [2023-12-27 13:13:29,832][20381] Updated weights for policy 1, policy_version 1092455 (0.0007) [2023-12-27 13:13:29,898][20381] Updated weights for policy 1, policy_version 1092465 (0.0008) [2023-12-27 13:13:30,138][20347] Updated weights for policy 0, policy_version 1093456 (0.0008) [2023-12-27 13:13:30,201][20347] Updated weights for policy 0, policy_version 1093466 (0.0009) [2023-12-27 13:13:30,256][20347] Updated weights for policy 0, policy_version 1093476 (0.0010) [2023-12-27 13:13:30,551][20381] Updated weights for policy 1, policy_version 1092475 (0.0007) [2023-12-27 13:13:30,614][20381] Updated weights for policy 1, policy_version 1092485 (0.0007) [2023-12-27 13:13:30,663][20381] Updated weights for policy 1, policy_version 1092495 (0.0007) [2023-12-27 13:13:30,880][20347] Updated weights for policy 0, policy_version 1093486 (0.0007) [2023-12-27 13:13:30,910][20291] KL-divergence is very high: 581.4086 [2023-12-27 13:13:30,916][20291] KL-divergence is very high: 495.2173 [2023-12-27 13:13:30,936][20347] Updated weights for policy 0, policy_version 1093496 (0.0008) [2023-12-27 13:13:30,980][20347] Updated weights for policy 0, policy_version 1093506 (0.0010) [2023-12-27 13:13:31,322][20381] Updated weights for policy 1, policy_version 1092505 (0.0007) [2023-12-27 13:13:31,390][20381] Updated weights for policy 1, policy_version 1092515 (0.0008) [2023-12-27 13:13:31,447][20381] Updated weights for policy 1, policy_version 1092525 (0.0007) [2023-12-27 13:13:31,508][20381] Updated weights for policy 1, policy_version 1092535 (0.0008) [2023-12-27 13:13:31,688][20291] KL-divergence is very high: 556.5825 [2023-12-27 13:13:31,710][20347] Updated weights for policy 0, policy_version 1093516 (0.0008) [2023-12-27 13:13:31,743][20291] KL-divergence is very high: 1217.3528 [2023-12-27 13:13:31,767][20347] Updated weights for policy 0, policy_version 1093526 (0.0010) [2023-12-27 13:13:31,781][20291] KL-divergence is very high: 802.0844 [2023-12-27 13:13:31,817][20347] Updated weights for policy 0, policy_version 1093536 (0.0009) [2023-12-27 13:13:31,822][20291] KL-divergence is very high: 402.5497 [2023-12-27 13:13:32,201][20381] Updated weights for policy 1, policy_version 1092545 (0.0006) [2023-12-27 13:13:32,250][20381] Updated weights for policy 1, policy_version 1092555 (0.0008) [2023-12-27 13:13:32,304][20381] Updated weights for policy 1, policy_version 1092565 (0.0008) [2023-12-27 13:13:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 559718400. Throughput: 0: 10298.3, 1: 10226.1. Samples: 559726988. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:32,363][19170] Avg episode reward: [(0, '882.630'), (1, '500.010')] [2023-12-27 13:13:32,520][20347] Updated weights for policy 0, policy_version 1093546 (0.0008) [2023-12-27 13:13:32,542][20291] KL-divergence is very high: 828.2328 [2023-12-27 13:13:32,572][20347] Updated weights for policy 0, policy_version 1093556 (0.0008) [2023-12-27 13:13:32,580][20291] KL-divergence is very high: 279.1482 [2023-12-27 13:13:32,617][20347] Updated weights for policy 0, policy_version 1093566 (0.0010) [2023-12-27 13:13:32,617][20291] KL-divergence is very high: 104.2824 [2023-12-27 13:13:32,655][20291] KL-divergence is very high: 151.3659 [2023-12-27 13:13:32,666][20347] Updated weights for policy 0, policy_version 1093576 (0.0010) [2023-12-27 13:13:33,035][20381] Updated weights for policy 1, policy_version 1092575 (0.0006) [2023-12-27 13:13:33,091][20381] Updated weights for policy 1, policy_version 1092585 (0.0005) [2023-12-27 13:13:33,143][20381] Updated weights for policy 1, policy_version 1092595 (0.0005) [2023-12-27 13:13:33,303][20291] KL-divergence is very high: 333.7482 [2023-12-27 13:13:33,349][20347] Updated weights for policy 0, policy_version 1093586 (0.0005) [2023-12-27 13:13:33,355][20291] KL-divergence is very high: 240.6873 [2023-12-27 13:13:33,400][20291] KL-divergence is very high: 169.8947 [2023-12-27 13:13:33,407][20347] Updated weights for policy 0, policy_version 1093596 (0.0006) [2023-12-27 13:13:33,444][20291] KL-divergence is very high: 120.4124 [2023-12-27 13:13:33,459][20347] Updated weights for policy 0, policy_version 1093606 (0.0007) [2023-12-27 13:13:33,725][20381] Updated weights for policy 1, policy_version 1092605 (0.0007) [2023-12-27 13:13:33,772][20381] Updated weights for policy 1, policy_version 1092615 (0.0007) [2023-12-27 13:13:33,824][20381] Updated weights for policy 1, policy_version 1092625 (0.0008) [2023-12-27 13:13:34,068][20347] Updated weights for policy 0, policy_version 1093616 (0.0006) [2023-12-27 13:13:34,122][20347] Updated weights for policy 0, policy_version 1093626 (0.0006) [2023-12-27 13:13:34,186][20347] Updated weights for policy 0, policy_version 1093636 (0.0008) [2023-12-27 13:13:34,584][20381] Updated weights for policy 1, policy_version 1092635 (0.0008) [2023-12-27 13:13:34,640][20381] Updated weights for policy 1, policy_version 1092645 (0.0008) [2023-12-27 13:13:34,693][20381] Updated weights for policy 1, policy_version 1092655 (0.0007) [2023-12-27 13:13:34,779][20347] Updated weights for policy 0, policy_version 1093646 (0.0008) [2023-12-27 13:13:34,831][20347] Updated weights for policy 0, policy_version 1093656 (0.0008) [2023-12-27 13:13:34,874][20347] Updated weights for policy 0, policy_version 1093666 (0.0005) [2023-12-27 13:13:35,331][20381] Updated weights for policy 1, policy_version 1092665 (0.0006) [2023-12-27 13:13:35,387][20381] Updated weights for policy 1, policy_version 1092675 (0.0008) [2023-12-27 13:13:35,450][20381] Updated weights for policy 1, policy_version 1092685 (0.0005) [2023-12-27 13:13:35,507][20381] Updated weights for policy 1, policy_version 1092695 (0.0005) [2023-12-27 13:13:35,598][20347] Updated weights for policy 0, policy_version 1093676 (0.0006) [2023-12-27 13:13:35,652][20347] Updated weights for policy 0, policy_version 1093686 (0.0007) [2023-12-27 13:13:35,707][20347] Updated weights for policy 0, policy_version 1093696 (0.0007) [2023-12-27 13:13:36,139][20381] Updated weights for policy 1, policy_version 1092705 (0.0008) [2023-12-27 13:13:36,196][20381] Updated weights for policy 1, policy_version 1092715 (0.0006) [2023-12-27 13:13:36,247][20381] Updated weights for policy 1, policy_version 1092725 (0.0008) [2023-12-27 13:13:36,449][20347] Updated weights for policy 0, policy_version 1093706 (0.0009) [2023-12-27 13:13:36,513][20347] Updated weights for policy 0, policy_version 1093716 (0.0009) [2023-12-27 13:13:36,570][20347] Updated weights for policy 0, policy_version 1093726 (0.0009) [2023-12-27 13:13:36,619][20347] Updated weights for policy 0, policy_version 1093736 (0.0008) [2023-12-27 13:13:36,884][20381] Updated weights for policy 1, policy_version 1092735 (0.0007) [2023-12-27 13:13:36,941][20381] Updated weights for policy 1, policy_version 1092745 (0.0007) [2023-12-27 13:13:37,000][20381] Updated weights for policy 1, policy_version 1092755 (0.0006) [2023-12-27 13:13:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 559816704. Throughput: 0: 10357.2, 1: 10201.7. Samples: 559789928. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:37,363][19170] Avg episode reward: [(0, '872.840'), (1, '558.990')] [2023-12-27 13:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001092760_279773184.pth... [2023-12-27 13:13:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001091576_279470080.pth [2023-12-27 13:13:37,373][20347] Updated weights for policy 0, policy_version 1093746 (0.0007) [2023-12-27 13:13:37,439][20347] Updated weights for policy 0, policy_version 1093756 (0.0007) [2023-12-27 13:13:37,495][20347] Updated weights for policy 0, policy_version 1093766 (0.0007) [2023-12-27 13:13:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001093768_280051712.pth... [2023-12-27 13:13:37,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001092552_279740416.pth [2023-12-27 13:13:37,668][20381] Updated weights for policy 1, policy_version 1092765 (0.0006) [2023-12-27 13:13:37,724][20381] Updated weights for policy 1, policy_version 1092775 (0.0007) [2023-12-27 13:13:37,785][20381] Updated weights for policy 1, policy_version 1092785 (0.0006) [2023-12-27 13:13:38,162][20347] Updated weights for policy 0, policy_version 1093776 (0.0008) [2023-12-27 13:13:38,211][20347] Updated weights for policy 0, policy_version 1093786 (0.0008) [2023-12-27 13:13:38,259][20347] Updated weights for policy 0, policy_version 1093796 (0.0007) [2023-12-27 13:13:38,479][20381] Updated weights for policy 1, policy_version 1092795 (0.0009) [2023-12-27 13:13:38,538][20381] Updated weights for policy 1, policy_version 1092805 (0.0011) [2023-12-27 13:13:38,598][20381] Updated weights for policy 1, policy_version 1092815 (0.0011) [2023-12-27 13:13:38,960][20347] Updated weights for policy 0, policy_version 1093806 (0.0007) [2023-12-27 13:13:39,014][20347] Updated weights for policy 0, policy_version 1093816 (0.0007) [2023-12-27 13:13:39,071][20347] Updated weights for policy 0, policy_version 1093826 (0.0007) [2023-12-27 13:13:39,249][20381] Updated weights for policy 1, policy_version 1092825 (0.0010) [2023-12-27 13:13:39,316][20381] Updated weights for policy 1, policy_version 1092835 (0.0008) [2023-12-27 13:13:39,380][20381] Updated weights for policy 1, policy_version 1092845 (0.0010) [2023-12-27 13:13:39,451][20381] Updated weights for policy 1, policy_version 1092855 (0.0010) [2023-12-27 13:13:39,848][20347] Updated weights for policy 0, policy_version 1093836 (0.0007) [2023-12-27 13:13:39,909][20347] Updated weights for policy 0, policy_version 1093846 (0.0008) [2023-12-27 13:13:39,975][20347] Updated weights for policy 0, policy_version 1093856 (0.0007) [2023-12-27 13:13:40,138][20381] Updated weights for policy 1, policy_version 1092865 (0.0008) [2023-12-27 13:13:40,198][20381] Updated weights for policy 1, policy_version 1092875 (0.0008) [2023-12-27 13:13:40,252][20381] Updated weights for policy 1, policy_version 1092885 (0.0007) [2023-12-27 13:13:40,589][20347] Updated weights for policy 0, policy_version 1093866 (0.0006) [2023-12-27 13:13:40,639][20347] Updated weights for policy 0, policy_version 1093876 (0.0005) [2023-12-27 13:13:40,688][20347] Updated weights for policy 0, policy_version 1093886 (0.0005) [2023-12-27 13:13:40,742][20347] Updated weights for policy 0, policy_version 1093896 (0.0006) [2023-12-27 13:13:40,948][20381] Updated weights for policy 1, policy_version 1092895 (0.0007) [2023-12-27 13:13:41,010][20381] Updated weights for policy 1, policy_version 1092905 (0.0008) [2023-12-27 13:13:41,078][20381] Updated weights for policy 1, policy_version 1092915 (0.0008) [2023-12-27 13:13:41,433][20347] Updated weights for policy 0, policy_version 1093906 (0.0008) [2023-12-27 13:13:41,490][20347] Updated weights for policy 0, policy_version 1093916 (0.0008) [2023-12-27 13:13:41,550][20347] Updated weights for policy 0, policy_version 1093926 (0.0007) [2023-12-27 13:13:41,829][20381] Updated weights for policy 1, policy_version 1092925 (0.0009) [2023-12-27 13:13:41,890][20381] Updated weights for policy 1, policy_version 1092935 (0.0011) [2023-12-27 13:13:41,954][20381] Updated weights for policy 1, policy_version 1092945 (0.0011) [2023-12-27 13:13:42,276][20347] Updated weights for policy 0, policy_version 1093936 (0.0008) [2023-12-27 13:13:42,335][20347] Updated weights for policy 0, policy_version 1093946 (0.0008) [2023-12-27 13:13:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 559915008. Throughput: 0: 10317.7, 1: 10211.7. Samples: 559910480. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:42,363][19170] Avg episode reward: [(0, '892.290'), (1, '480.010')] [2023-12-27 13:13:42,396][20347] Updated weights for policy 0, policy_version 1093956 (0.0008) [2023-12-27 13:13:42,645][20381] Updated weights for policy 1, policy_version 1092955 (0.0010) [2023-12-27 13:13:42,703][20381] Updated weights for policy 1, policy_version 1092965 (0.0010) [2023-12-27 13:13:42,765][20381] Updated weights for policy 1, policy_version 1092975 (0.0011) [2023-12-27 13:13:43,100][20347] Updated weights for policy 0, policy_version 1093966 (0.0008) [2023-12-27 13:13:43,157][20347] Updated weights for policy 0, policy_version 1093976 (0.0007) [2023-12-27 13:13:43,209][20347] Updated weights for policy 0, policy_version 1093986 (0.0007) [2023-12-27 13:13:43,444][20381] Updated weights for policy 1, policy_version 1092985 (0.0011) [2023-12-27 13:13:43,501][20381] Updated weights for policy 1, policy_version 1092995 (0.0010) [2023-12-27 13:13:43,559][20381] Updated weights for policy 1, policy_version 1093005 (0.0010) [2023-12-27 13:13:43,614][20381] Updated weights for policy 1, policy_version 1093015 (0.0010) [2023-12-27 13:13:43,872][20347] Updated weights for policy 0, policy_version 1093996 (0.0006) [2023-12-27 13:13:43,930][20347] Updated weights for policy 0, policy_version 1094006 (0.0007) [2023-12-27 13:13:43,990][20347] Updated weights for policy 0, policy_version 1094016 (0.0007) [2023-12-27 13:13:44,297][20381] Updated weights for policy 1, policy_version 1093025 (0.0010) [2023-12-27 13:13:44,345][20381] Updated weights for policy 1, policy_version 1093035 (0.0010) [2023-12-27 13:13:44,390][20381] Updated weights for policy 1, policy_version 1093045 (0.0010) [2023-12-27 13:13:44,557][20347] Updated weights for policy 0, policy_version 1094026 (0.0006) [2023-12-27 13:13:44,619][20347] Updated weights for policy 0, policy_version 1094036 (0.0007) [2023-12-27 13:13:44,675][20347] Updated weights for policy 0, policy_version 1094046 (0.0007) [2023-12-27 13:13:44,728][20347] Updated weights for policy 0, policy_version 1094056 (0.0007) [2023-12-27 13:13:45,109][20381] Updated weights for policy 1, policy_version 1093055 (0.0011) [2023-12-27 13:13:45,169][20381] Updated weights for policy 1, policy_version 1093065 (0.0011) [2023-12-27 13:13:45,229][20381] Updated weights for policy 1, policy_version 1093075 (0.0011) [2023-12-27 13:13:45,433][20347] Updated weights for policy 0, policy_version 1094066 (0.0008) [2023-12-27 13:13:45,492][20347] Updated weights for policy 0, policy_version 1094076 (0.0007) [2023-12-27 13:13:45,548][20347] Updated weights for policy 0, policy_version 1094086 (0.0008) [2023-12-27 13:13:45,875][20381] Updated weights for policy 1, policy_version 1093085 (0.0008) [2023-12-27 13:13:45,927][20381] Updated weights for policy 1, policy_version 1093095 (0.0005) [2023-12-27 13:13:45,980][20381] Updated weights for policy 1, policy_version 1093105 (0.0005) [2023-12-27 13:13:46,286][20347] Updated weights for policy 0, policy_version 1094096 (0.0008) [2023-12-27 13:13:46,346][20347] Updated weights for policy 0, policy_version 1094106 (0.0008) [2023-12-27 13:13:46,401][20347] Updated weights for policy 0, policy_version 1094116 (0.0007) [2023-12-27 13:13:46,491][20381] Updated weights for policy 1, policy_version 1093115 (0.0006) [2023-12-27 13:13:46,536][20381] Updated weights for policy 1, policy_version 1093125 (0.0010) [2023-12-27 13:13:46,584][20381] Updated weights for policy 1, policy_version 1093135 (0.0010) [2023-12-27 13:13:47,051][20347] Updated weights for policy 0, policy_version 1094126 (0.0007) [2023-12-27 13:13:47,099][20347] Updated weights for policy 0, policy_version 1094136 (0.0007) [2023-12-27 13:13:47,155][20347] Updated weights for policy 0, policy_version 1094146 (0.0008) [2023-12-27 13:13:47,232][20381] Updated weights for policy 1, policy_version 1093145 (0.0010) [2023-12-27 13:13:47,290][20381] Updated weights for policy 1, policy_version 1093155 (0.0010) [2023-12-27 13:13:47,347][20381] Updated weights for policy 1, policy_version 1093165 (0.0010) [2023-12-27 13:13:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 560021504. Throughput: 0: 10338.9, 1: 10244.9. Samples: 560034220. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:47,363][19170] Avg episode reward: [(0, '892.150'), (1, '391.980')] [2023-12-27 13:13:47,432][20381] Updated weights for policy 1, policy_version 1093175 (0.0007) [2023-12-27 13:13:47,858][20347] Updated weights for policy 0, policy_version 1094156 (0.0008) [2023-12-27 13:13:47,915][20347] Updated weights for policy 0, policy_version 1094166 (0.0007) [2023-12-27 13:13:47,971][20347] Updated weights for policy 0, policy_version 1094176 (0.0007) [2023-12-27 13:13:48,067][20381] Updated weights for policy 1, policy_version 1093185 (0.0010) [2023-12-27 13:13:48,122][20381] Updated weights for policy 1, policy_version 1093195 (0.0010) [2023-12-27 13:13:48,174][20381] Updated weights for policy 1, policy_version 1093205 (0.0010) [2023-12-27 13:13:48,652][20347] Updated weights for policy 0, policy_version 1094186 (0.0007) [2023-12-27 13:13:48,704][20347] Updated weights for policy 0, policy_version 1094196 (0.0009) [2023-12-27 13:13:48,766][20347] Updated weights for policy 0, policy_version 1094206 (0.0010) [2023-12-27 13:13:48,824][20347] Updated weights for policy 0, policy_version 1094216 (0.0010) [2023-12-27 13:13:48,869][20381] Updated weights for policy 1, policy_version 1093215 (0.0011) [2023-12-27 13:13:48,923][20381] Updated weights for policy 1, policy_version 1093225 (0.0010) [2023-12-27 13:13:48,984][20381] Updated weights for policy 1, policy_version 1093235 (0.0008) [2023-12-27 13:13:49,532][20347] Updated weights for policy 0, policy_version 1094226 (0.0008) [2023-12-27 13:13:49,591][20347] Updated weights for policy 0, policy_version 1094236 (0.0007) [2023-12-27 13:13:49,639][20381] Updated weights for policy 1, policy_version 1093245 (0.0006) [2023-12-27 13:13:49,653][20347] Updated weights for policy 0, policy_version 1094246 (0.0006) [2023-12-27 13:13:49,689][20381] Updated weights for policy 1, policy_version 1093255 (0.0007) [2023-12-27 13:13:49,745][20381] Updated weights for policy 1, policy_version 1093265 (0.0007) [2023-12-27 13:13:50,391][20347] Updated weights for policy 0, policy_version 1094256 (0.0007) [2023-12-27 13:13:50,427][20381] Updated weights for policy 1, policy_version 1093275 (0.0007) [2023-12-27 13:13:50,450][20347] Updated weights for policy 0, policy_version 1094266 (0.0008) [2023-12-27 13:13:50,481][20381] Updated weights for policy 1, policy_version 1093285 (0.0006) [2023-12-27 13:13:50,505][20347] Updated weights for policy 0, policy_version 1094276 (0.0006) [2023-12-27 13:13:50,533][20381] Updated weights for policy 1, policy_version 1093295 (0.0006) [2023-12-27 13:13:51,215][20347] Updated weights for policy 0, policy_version 1094286 (0.0007) [2023-12-27 13:13:51,246][20381] Updated weights for policy 1, policy_version 1093305 (0.0008) [2023-12-27 13:13:51,275][20347] Updated weights for policy 0, policy_version 1094296 (0.0007) [2023-12-27 13:13:51,299][20381] Updated weights for policy 1, policy_version 1093315 (0.0007) [2023-12-27 13:13:51,330][20347] Updated weights for policy 0, policy_version 1094306 (0.0006) [2023-12-27 13:13:51,356][20381] Updated weights for policy 1, policy_version 1093325 (0.0007) [2023-12-27 13:13:51,422][20381] Updated weights for policy 1, policy_version 1093335 (0.0007) [2023-12-27 13:13:52,080][20347] Updated weights for policy 0, policy_version 1094316 (0.0008) [2023-12-27 13:13:52,108][20381] Updated weights for policy 1, policy_version 1093345 (0.0006) [2023-12-27 13:13:52,127][20347] Updated weights for policy 0, policy_version 1094326 (0.0008) [2023-12-27 13:13:52,158][20381] Updated weights for policy 1, policy_version 1093355 (0.0007) [2023-12-27 13:13:52,194][20347] Updated weights for policy 0, policy_version 1094336 (0.0008) [2023-12-27 13:13:52,213][20381] Updated weights for policy 1, policy_version 1093365 (0.0007) [2023-12-27 13:13:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 560128000. Throughput: 0: 10295.2, 1: 10247.1. Samples: 560093996. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:52,363][19170] Avg episode reward: [(0, '872.790'), (1, '489.640')] [2023-12-27 13:13:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001093368_279928832.pth... [2023-12-27 13:13:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001094344_280199168.pth... [2023-12-27 13:13:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001093160_279896064.pth [2023-12-27 13:13:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001092152_279617536.pth [2023-12-27 13:13:52,872][20347] Updated weights for policy 0, policy_version 1094346 (0.0006) [2023-12-27 13:13:52,920][20347] Updated weights for policy 0, policy_version 1094356 (0.0007) [2023-12-27 13:13:52,981][20347] Updated weights for policy 0, policy_version 1094366 (0.0008) [2023-12-27 13:13:52,995][20381] Updated weights for policy 1, policy_version 1093375 (0.0010) [2023-12-27 13:13:53,035][20347] Updated weights for policy 0, policy_version 1094376 (0.0007) [2023-12-27 13:13:53,050][20381] Updated weights for policy 1, policy_version 1093385 (0.0010) [2023-12-27 13:13:53,098][20381] Updated weights for policy 1, policy_version 1093395 (0.0010) [2023-12-27 13:13:53,697][20347] Updated weights for policy 0, policy_version 1094386 (0.0005) [2023-12-27 13:13:53,744][20381] Updated weights for policy 1, policy_version 1093405 (0.0011) [2023-12-27 13:13:53,747][20347] Updated weights for policy 0, policy_version 1094396 (0.0005) [2023-12-27 13:13:53,799][20381] Updated weights for policy 1, policy_version 1093415 (0.0010) [2023-12-27 13:13:53,804][20347] Updated weights for policy 0, policy_version 1094406 (0.0005) [2023-12-27 13:13:53,856][20381] Updated weights for policy 1, policy_version 1093425 (0.0010) [2023-12-27 13:13:54,438][20347] Updated weights for policy 0, policy_version 1094416 (0.0007) [2023-12-27 13:13:54,492][20347] Updated weights for policy 0, policy_version 1094426 (0.0007) [2023-12-27 13:13:54,514][20381] Updated weights for policy 1, policy_version 1093435 (0.0010) [2023-12-27 13:13:54,555][20347] Updated weights for policy 0, policy_version 1094436 (0.0006) [2023-12-27 13:13:54,572][20381] Updated weights for policy 1, policy_version 1093445 (0.0010) [2023-12-27 13:13:54,623][20381] Updated weights for policy 1, policy_version 1093455 (0.0009) [2023-12-27 13:13:55,211][20347] Updated weights for policy 0, policy_version 1094446 (0.0006) [2023-12-27 13:13:55,257][20347] Updated weights for policy 0, policy_version 1094456 (0.0005) [2023-12-27 13:13:55,303][20347] Updated weights for policy 0, policy_version 1094466 (0.0005) [2023-12-27 13:13:55,313][20381] Updated weights for policy 1, policy_version 1093465 (0.0010) [2023-12-27 13:13:55,377][20381] Updated weights for policy 1, policy_version 1093475 (0.0010) [2023-12-27 13:13:55,422][20381] Updated weights for policy 1, policy_version 1093485 (0.0010) [2023-12-27 13:13:55,474][20381] Updated weights for policy 1, policy_version 1093495 (0.0010) [2023-12-27 13:13:55,930][20347] Updated weights for policy 0, policy_version 1094476 (0.0006) [2023-12-27 13:13:55,979][20347] Updated weights for policy 0, policy_version 1094486 (0.0007) [2023-12-27 13:13:56,027][20347] Updated weights for policy 0, policy_version 1094496 (0.0007) [2023-12-27 13:13:56,189][20381] Updated weights for policy 1, policy_version 1093505 (0.0011) [2023-12-27 13:13:56,251][20381] Updated weights for policy 1, policy_version 1093515 (0.0009) [2023-12-27 13:13:56,306][20381] Updated weights for policy 1, policy_version 1093525 (0.0008) [2023-12-27 13:13:56,690][20347] Updated weights for policy 0, policy_version 1094506 (0.0007) [2023-12-27 13:13:56,742][20347] Updated weights for policy 0, policy_version 1094516 (0.0007) [2023-12-27 13:13:56,791][20347] Updated weights for policy 0, policy_version 1094526 (0.0008) [2023-12-27 13:13:56,845][20347] Updated weights for policy 0, policy_version 1094536 (0.0007) [2023-12-27 13:13:56,969][20381] Updated weights for policy 1, policy_version 1093535 (0.0007) [2023-12-27 13:13:57,013][20381] Updated weights for policy 1, policy_version 1093545 (0.0005) [2023-12-27 13:13:57,063][20381] Updated weights for policy 1, policy_version 1093555 (0.0006) [2023-12-27 13:13:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 560226304. Throughput: 0: 10299.8, 1: 10255.6. Samples: 560215460. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:13:57,363][19170] Avg episode reward: [(0, '843.580'), (1, '519.290')] [2023-12-27 13:13:57,506][20347] Updated weights for policy 0, policy_version 1094546 (0.0007) [2023-12-27 13:13:57,554][20347] Updated weights for policy 0, policy_version 1094556 (0.0008) [2023-12-27 13:13:57,603][20347] Updated weights for policy 0, policy_version 1094566 (0.0008) [2023-12-27 13:13:57,654][20381] Updated weights for policy 1, policy_version 1093565 (0.0007) [2023-12-27 13:13:57,715][20381] Updated weights for policy 1, policy_version 1093575 (0.0008) [2023-12-27 13:13:57,765][20381] Updated weights for policy 1, policy_version 1093585 (0.0008) [2023-12-27 13:13:58,254][20347] Updated weights for policy 0, policy_version 1094576 (0.0007) [2023-12-27 13:13:58,314][20347] Updated weights for policy 0, policy_version 1094586 (0.0008) [2023-12-27 13:13:58,384][20347] Updated weights for policy 0, policy_version 1094596 (0.0008) [2023-12-27 13:13:58,487][20381] Updated weights for policy 1, policy_version 1093595 (0.0008) [2023-12-27 13:13:58,542][20381] Updated weights for policy 1, policy_version 1093605 (0.0008) [2023-12-27 13:13:58,603][20381] Updated weights for policy 1, policy_version 1093615 (0.0007) [2023-12-27 13:13:59,148][20347] Updated weights for policy 0, policy_version 1094606 (0.0008) [2023-12-27 13:13:59,207][20347] Updated weights for policy 0, policy_version 1094616 (0.0007) [2023-12-27 13:13:59,277][20347] Updated weights for policy 0, policy_version 1094626 (0.0008) [2023-12-27 13:13:59,403][20381] Updated weights for policy 1, policy_version 1093625 (0.0009) [2023-12-27 13:13:59,458][20381] Updated weights for policy 1, policy_version 1093635 (0.0006) [2023-12-27 13:13:59,505][20381] Updated weights for policy 1, policy_version 1093645 (0.0005) [2023-12-27 13:13:59,565][20381] Updated weights for policy 1, policy_version 1093655 (0.0006) [2023-12-27 13:14:00,055][20347] Updated weights for policy 0, policy_version 1094636 (0.0007) [2023-12-27 13:14:00,113][20347] Updated weights for policy 0, policy_version 1094646 (0.0008) [2023-12-27 13:14:00,172][20347] Updated weights for policy 0, policy_version 1094656 (0.0008) [2023-12-27 13:14:00,193][20381] Updated weights for policy 1, policy_version 1093665 (0.0005) [2023-12-27 13:14:00,249][20381] Updated weights for policy 1, policy_version 1093675 (0.0007) [2023-12-27 13:14:00,308][20381] Updated weights for policy 1, policy_version 1093685 (0.0007) [2023-12-27 13:14:00,891][20381] Updated weights for policy 1, policy_version 1093695 (0.0007) [2023-12-27 13:14:00,909][20347] Updated weights for policy 0, policy_version 1094666 (0.0006) [2023-12-27 13:14:00,937][20381] Updated weights for policy 1, policy_version 1093705 (0.0006) [2023-12-27 13:14:00,957][20347] Updated weights for policy 0, policy_version 1094676 (0.0005) [2023-12-27 13:14:00,988][20381] Updated weights for policy 1, policy_version 1093715 (0.0005) [2023-12-27 13:14:01,005][20347] Updated weights for policy 0, policy_version 1094686 (0.0005) [2023-12-27 13:14:01,066][20347] Updated weights for policy 0, policy_version 1094696 (0.0009) [2023-12-27 13:14:01,664][20381] Updated weights for policy 1, policy_version 1093725 (0.0006) [2023-12-27 13:14:01,733][20381] Updated weights for policy 1, policy_version 1093735 (0.0008) [2023-12-27 13:14:01,788][20347] Updated weights for policy 0, policy_version 1094706 (0.0008) [2023-12-27 13:14:01,792][20381] Updated weights for policy 1, policy_version 1093745 (0.0008) [2023-12-27 13:14:01,842][20347] Updated weights for policy 0, policy_version 1094716 (0.0005) [2023-12-27 13:14:01,895][20347] Updated weights for policy 0, policy_version 1094726 (0.0005) [2023-12-27 13:14:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 560324608. Throughput: 0: 10213.4, 1: 10231.5. Samples: 560336268. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:02,363][19170] Avg episode reward: [(0, '833.700'), (1, '499.550')] [2023-12-27 13:14:02,496][20381] Updated weights for policy 1, policy_version 1093755 (0.0007) [2023-12-27 13:14:02,551][20347] Updated weights for policy 0, policy_version 1094736 (0.0006) [2023-12-27 13:14:02,556][20381] Updated weights for policy 1, policy_version 1093765 (0.0007) [2023-12-27 13:14:02,579][20291] KL-divergence is very high: 117.1707 [2023-12-27 13:14:02,606][20347] Updated weights for policy 0, policy_version 1094746 (0.0005) [2023-12-27 13:14:02,607][20291] KL-divergence is very high: 118.2723 [2023-12-27 13:14:02,613][20381] Updated weights for policy 1, policy_version 1093775 (0.0008) [2023-12-27 13:14:02,664][20347] Updated weights for policy 0, policy_version 1094756 (0.0005) [2023-12-27 13:14:03,286][20381] Updated weights for policy 1, policy_version 1093785 (0.0008) [2023-12-27 13:14:03,300][20347] Updated weights for policy 0, policy_version 1094766 (0.0006) [2023-12-27 13:14:03,338][20381] Updated weights for policy 1, policy_version 1093795 (0.0006) [2023-12-27 13:14:03,352][20347] Updated weights for policy 0, policy_version 1094776 (0.0005) [2023-12-27 13:14:03,386][20381] Updated weights for policy 1, policy_version 1093805 (0.0005) [2023-12-27 13:14:03,406][20347] Updated weights for policy 0, policy_version 1094786 (0.0006) [2023-12-27 13:14:03,434][20381] Updated weights for policy 1, policy_version 1093815 (0.0005) [2023-12-27 13:14:04,066][20347] Updated weights for policy 0, policy_version 1094796 (0.0007) [2023-12-27 13:14:04,095][20381] Updated weights for policy 1, policy_version 1093825 (0.0006) [2023-12-27 13:14:04,133][20347] Updated weights for policy 0, policy_version 1094806 (0.0008) [2023-12-27 13:14:04,158][20381] Updated weights for policy 1, policy_version 1093835 (0.0006) [2023-12-27 13:14:04,183][20347] Updated weights for policy 0, policy_version 1094816 (0.0007) [2023-12-27 13:14:04,220][20381] Updated weights for policy 1, policy_version 1093845 (0.0006) [2023-12-27 13:14:04,886][20381] Updated weights for policy 1, policy_version 1093855 (0.0008) [2023-12-27 13:14:04,894][20347] Updated weights for policy 0, policy_version 1094826 (0.0008) [2023-12-27 13:14:04,936][20381] Updated weights for policy 1, policy_version 1093865 (0.0007) [2023-12-27 13:14:04,945][20347] Updated weights for policy 0, policy_version 1094836 (0.0006) [2023-12-27 13:14:04,989][20381] Updated weights for policy 1, policy_version 1093875 (0.0005) [2023-12-27 13:14:04,996][20347] Updated weights for policy 0, policy_version 1094846 (0.0006) [2023-12-27 13:14:05,048][20347] Updated weights for policy 0, policy_version 1094856 (0.0007) [2023-12-27 13:14:05,660][20381] Updated weights for policy 1, policy_version 1093885 (0.0006) [2023-12-27 13:14:05,710][20347] Updated weights for policy 0, policy_version 1094866 (0.0006) [2023-12-27 13:14:05,711][20381] Updated weights for policy 1, policy_version 1093895 (0.0005) [2023-12-27 13:14:05,736][20291] KL-divergence is very high: 186.8851 [2023-12-27 13:14:05,759][20291] KL-divergence is very high: 244.8867 [2023-12-27 13:14:05,767][20381] Updated weights for policy 1, policy_version 1093905 (0.0006) [2023-12-27 13:14:05,769][20347] Updated weights for policy 0, policy_version 1094876 (0.0007) [2023-12-27 13:14:05,783][20291] KL-divergence is very high: 236.6630 [2023-12-27 13:14:05,809][20291] KL-divergence is very high: 201.4590 [2023-12-27 13:14:05,832][20347] Updated weights for policy 0, policy_version 1094886 (0.0006) [2023-12-27 13:14:05,834][20291] KL-divergence is very high: 135.1434 [2023-12-27 13:14:06,410][20381] Updated weights for policy 1, policy_version 1093915 (0.0006) [2023-12-27 13:14:06,467][20291] KL-divergence is very high: 129.4003 [2023-12-27 13:14:06,467][20381] Updated weights for policy 1, policy_version 1093925 (0.0007) [2023-12-27 13:14:06,473][20291] KL-divergence is very high: 115.1665 [2023-12-27 13:14:06,515][20347] Updated weights for policy 0, policy_version 1094896 (0.0006) [2023-12-27 13:14:06,521][20381] Updated weights for policy 1, policy_version 1093935 (0.0008) [2023-12-27 13:14:06,572][20347] Updated weights for policy 0, policy_version 1094906 (0.0006) [2023-12-27 13:14:06,636][20347] Updated weights for policy 0, policy_version 1094916 (0.0008) [2023-12-27 13:14:07,232][20381] Updated weights for policy 1, policy_version 1093945 (0.0006) [2023-12-27 13:14:07,285][20381] Updated weights for policy 1, policy_version 1093955 (0.0007) [2023-12-27 13:14:07,332][20381] Updated weights for policy 1, policy_version 1093965 (0.0007) [2023-12-27 13:14:07,333][20347] Updated weights for policy 0, policy_version 1094926 (0.0007) [2023-12-27 13:14:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 560422912. Throughput: 0: 10179.7, 1: 10255.4. Samples: 560398160. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:07,363][19170] Avg episode reward: [(0, '843.380'), (1, '578.270')] [2023-12-27 13:14:07,389][20381] Updated weights for policy 1, policy_version 1093975 (0.0007) [2023-12-27 13:14:07,395][20347] Updated weights for policy 0, policy_version 1094936 (0.0006) [2023-12-27 13:14:07,395][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001093976_280084480.pth... [2023-12-27 13:14:07,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001092760_279773184.pth [2023-12-27 13:14:07,461][20347] Updated weights for policy 0, policy_version 1094946 (0.0008) [2023-12-27 13:14:07,500][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001094952_280354816.pth... [2023-12-27 13:14:07,504][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001093768_280051712.pth [2023-12-27 13:14:08,097][20347] Updated weights for policy 0, policy_version 1094956 (0.0008) [2023-12-27 13:14:08,118][20381] Updated weights for policy 1, policy_version 1093985 (0.0007) [2023-12-27 13:14:08,156][20347] Updated weights for policy 0, policy_version 1094966 (0.0008) [2023-12-27 13:14:08,166][20381] Updated weights for policy 1, policy_version 1093995 (0.0005) [2023-12-27 13:14:08,206][20347] Updated weights for policy 0, policy_version 1094976 (0.0007) [2023-12-27 13:14:08,213][20381] Updated weights for policy 1, policy_version 1094005 (0.0006) [2023-12-27 13:14:08,867][20347] Updated weights for policy 0, policy_version 1094986 (0.0005) [2023-12-27 13:14:08,919][20347] Updated weights for policy 0, policy_version 1094996 (0.0005) [2023-12-27 13:14:08,942][20381] Updated weights for policy 1, policy_version 1094015 (0.0007) [2023-12-27 13:14:08,973][20347] Updated weights for policy 0, policy_version 1095006 (0.0006) [2023-12-27 13:14:08,997][20381] Updated weights for policy 1, policy_version 1094025 (0.0007) [2023-12-27 13:14:09,022][20347] Updated weights for policy 0, policy_version 1095016 (0.0005) [2023-12-27 13:14:09,051][20381] Updated weights for policy 1, policy_version 1094035 (0.0008) [2023-12-27 13:14:09,722][20347] Updated weights for policy 0, policy_version 1095026 (0.0010) [2023-12-27 13:14:09,780][20347] Updated weights for policy 0, policy_version 1095036 (0.0010) [2023-12-27 13:14:09,783][20381] Updated weights for policy 1, policy_version 1094045 (0.0008) [2023-12-27 13:14:09,838][20347] Updated weights for policy 0, policy_version 1095046 (0.0007) [2023-12-27 13:14:09,841][20381] Updated weights for policy 1, policy_version 1094055 (0.0007) [2023-12-27 13:14:09,898][20381] Updated weights for policy 1, policy_version 1094065 (0.0008) [2023-12-27 13:14:10,516][20347] Updated weights for policy 0, policy_version 1095056 (0.0007) [2023-12-27 13:14:10,562][20347] Updated weights for policy 0, policy_version 1095066 (0.0008) [2023-12-27 13:14:10,616][20347] Updated weights for policy 0, policy_version 1095076 (0.0008) [2023-12-27 13:14:10,640][20381] Updated weights for policy 1, policy_version 1094075 (0.0007) [2023-12-27 13:14:10,699][20381] Updated weights for policy 1, policy_version 1094085 (0.0008) [2023-12-27 13:14:10,767][20381] Updated weights for policy 1, policy_version 1094095 (0.0008) [2023-12-27 13:14:11,326][20347] Updated weights for policy 0, policy_version 1095086 (0.0007) [2023-12-27 13:14:11,400][20347] Updated weights for policy 0, policy_version 1095096 (0.0008) [2023-12-27 13:14:11,454][20291] KL-divergence is very high: 129.3979 [2023-12-27 13:14:11,461][20347] Updated weights for policy 0, policy_version 1095106 (0.0008) [2023-12-27 13:14:11,463][20381] Updated weights for policy 1, policy_version 1094105 (0.0008) [2023-12-27 13:14:11,524][20381] Updated weights for policy 1, policy_version 1094115 (0.0006) [2023-12-27 13:14:11,588][20381] Updated weights for policy 1, policy_version 1094125 (0.0007) [2023-12-27 13:14:11,658][20381] Updated weights for policy 1, policy_version 1094135 (0.0008) [2023-12-27 13:14:12,219][20347] Updated weights for policy 0, policy_version 1095116 (0.0008) [2023-12-27 13:14:12,241][20291] KL-divergence is very high: 110.3002 [2023-12-27 13:14:12,247][20291] KL-divergence is very high: 118.6810 [2023-12-27 13:14:12,279][20347] Updated weights for policy 0, policy_version 1095126 (0.0008) [2023-12-27 13:14:12,332][20347] Updated weights for policy 0, policy_version 1095136 (0.0007) [2023-12-27 13:14:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 560521216. Throughput: 0: 10185.3, 1: 10229.4. Samples: 560520048. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:12,363][19170] Avg episode reward: [(0, '472.070'), (1, '617.530')] [2023-12-27 13:14:12,382][20381] Updated weights for policy 1, policy_version 1094145 (0.0008) [2023-12-27 13:14:12,438][20381] Updated weights for policy 1, policy_version 1094155 (0.0008) [2023-12-27 13:14:12,496][20381] Updated weights for policy 1, policy_version 1094165 (0.0007) [2023-12-27 13:14:12,980][20347] Updated weights for policy 0, policy_version 1095146 (0.0008) [2023-12-27 13:14:12,997][20291] KL-divergence is very high: 103.3141 [2023-12-27 13:14:13,034][20347] Updated weights for policy 0, policy_version 1095156 (0.0010) [2023-12-27 13:14:13,082][20347] Updated weights for policy 0, policy_version 1095166 (0.0010) [2023-12-27 13:14:13,134][20347] Updated weights for policy 0, policy_version 1095176 (0.0007) [2023-12-27 13:14:13,177][20381] Updated weights for policy 1, policy_version 1094175 (0.0008) [2023-12-27 13:14:13,225][20381] Updated weights for policy 1, policy_version 1094185 (0.0008) [2023-12-27 13:14:13,274][20381] Updated weights for policy 1, policy_version 1094195 (0.0007) [2023-12-27 13:14:13,724][20347] Updated weights for policy 0, policy_version 1095186 (0.0010) [2023-12-27 13:14:13,774][20347] Updated weights for policy 0, policy_version 1095196 (0.0010) [2023-12-27 13:14:13,834][20347] Updated weights for policy 0, policy_version 1095206 (0.0005) [2023-12-27 13:14:14,003][20381] Updated weights for policy 1, policy_version 1094205 (0.0008) [2023-12-27 13:14:14,054][20381] Updated weights for policy 1, policy_version 1094215 (0.0005) [2023-12-27 13:14:14,116][20381] Updated weights for policy 1, policy_version 1094225 (0.0007) [2023-12-27 13:14:14,425][20347] Updated weights for policy 0, policy_version 1095216 (0.0005) [2023-12-27 13:14:14,476][20347] Updated weights for policy 0, policy_version 1095226 (0.0009) [2023-12-27 13:14:14,528][20347] Updated weights for policy 0, policy_version 1095236 (0.0010) [2023-12-27 13:14:14,761][20381] Updated weights for policy 1, policy_version 1094235 (0.0009) [2023-12-27 13:14:14,827][20381] Updated weights for policy 1, policy_version 1094245 (0.0010) [2023-12-27 13:14:14,883][20381] Updated weights for policy 1, policy_version 1094255 (0.0010) [2023-12-27 13:14:15,166][20347] Updated weights for policy 0, policy_version 1095246 (0.0011) [2023-12-27 13:14:15,227][20347] Updated weights for policy 0, policy_version 1095256 (0.0010) [2023-12-27 13:14:15,284][20347] Updated weights for policy 0, policy_version 1095266 (0.0010) [2023-12-27 13:14:15,569][20381] Updated weights for policy 1, policy_version 1094265 (0.0010) [2023-12-27 13:14:15,627][20381] Updated weights for policy 1, policy_version 1094275 (0.0010) [2023-12-27 13:14:15,688][20381] Updated weights for policy 1, policy_version 1094285 (0.0010) [2023-12-27 13:14:15,750][20381] Updated weights for policy 1, policy_version 1094295 (0.0010) [2023-12-27 13:14:15,970][20347] Updated weights for policy 0, policy_version 1095276 (0.0007) [2023-12-27 13:14:16,013][20347] Updated weights for policy 0, policy_version 1095286 (0.0007) [2023-12-27 13:14:16,067][20347] Updated weights for policy 0, policy_version 1095296 (0.0007) [2023-12-27 13:14:16,418][20381] Updated weights for policy 1, policy_version 1094305 (0.0011) [2023-12-27 13:14:16,470][20381] Updated weights for policy 1, policy_version 1094315 (0.0010) [2023-12-27 13:14:16,524][20381] Updated weights for policy 1, policy_version 1094325 (0.0010) [2023-12-27 13:14:16,731][20347] Updated weights for policy 0, policy_version 1095306 (0.0007) [2023-12-27 13:14:16,791][20347] Updated weights for policy 0, policy_version 1095316 (0.0010) [2023-12-27 13:14:16,851][20347] Updated weights for policy 0, policy_version 1095326 (0.0010) [2023-12-27 13:14:16,903][20347] Updated weights for policy 0, policy_version 1095336 (0.0010) [2023-12-27 13:14:17,216][20381] Updated weights for policy 1, policy_version 1094335 (0.0010) [2023-12-27 13:14:17,265][20381] Updated weights for policy 1, policy_version 1094345 (0.0010) [2023-12-27 13:14:17,316][20381] Updated weights for policy 1, policy_version 1094355 (0.0010) [2023-12-27 13:14:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 560635904. Throughput: 0: 10196.9, 1: 10158.0. Samples: 560642956. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:17,363][19170] Avg episode reward: [(0, '659.690'), (1, '666.600')] [2023-12-27 13:14:17,581][20347] Updated weights for policy 0, policy_version 1095346 (0.0010) [2023-12-27 13:14:17,637][20347] Updated weights for policy 0, policy_version 1095356 (0.0010) [2023-12-27 13:14:17,691][20347] Updated weights for policy 0, policy_version 1095366 (0.0010) [2023-12-27 13:14:17,987][20381] Updated weights for policy 1, policy_version 1094365 (0.0010) [2023-12-27 13:14:18,039][20381] Updated weights for policy 1, policy_version 1094375 (0.0010) [2023-12-27 13:14:18,094][20381] Updated weights for policy 1, policy_version 1094385 (0.0010) [2023-12-27 13:14:18,306][20347] Updated weights for policy 0, policy_version 1095376 (0.0010) [2023-12-27 13:14:18,370][20347] Updated weights for policy 0, policy_version 1095386 (0.0010) [2023-12-27 13:14:18,432][20347] Updated weights for policy 0, policy_version 1095396 (0.0010) [2023-12-27 13:14:18,818][20381] Updated weights for policy 1, policy_version 1094395 (0.0010) [2023-12-27 13:14:18,874][20381] Updated weights for policy 1, policy_version 1094405 (0.0007) [2023-12-27 13:14:18,923][20381] Updated weights for policy 1, policy_version 1094415 (0.0007) [2023-12-27 13:14:19,138][20347] Updated weights for policy 0, policy_version 1095406 (0.0010) [2023-12-27 13:14:19,195][20347] Updated weights for policy 0, policy_version 1095416 (0.0010) [2023-12-27 13:14:19,257][20347] Updated weights for policy 0, policy_version 1095426 (0.0010) [2023-12-27 13:14:19,687][20381] Updated weights for policy 1, policy_version 1094425 (0.0008) [2023-12-27 13:14:19,743][20381] Updated weights for policy 1, policy_version 1094435 (0.0008) [2023-12-27 13:14:19,803][20381] Updated weights for policy 1, policy_version 1094445 (0.0008) [2023-12-27 13:14:19,864][20381] Updated weights for policy 1, policy_version 1094455 (0.0007) [2023-12-27 13:14:19,937][20347] Updated weights for policy 0, policy_version 1095436 (0.0010) [2023-12-27 13:14:19,993][20347] Updated weights for policy 0, policy_version 1095446 (0.0007) [2023-12-27 13:14:20,043][20347] Updated weights for policy 0, policy_version 1095456 (0.0008) [2023-12-27 13:14:20,612][20381] Updated weights for policy 1, policy_version 1094465 (0.0008) [2023-12-27 13:14:20,662][20381] Updated weights for policy 1, policy_version 1094475 (0.0008) [2023-12-27 13:14:20,709][20381] Updated weights for policy 1, policy_version 1094485 (0.0006) [2023-12-27 13:14:20,765][20347] Updated weights for policy 0, policy_version 1095466 (0.0007) [2023-12-27 13:14:20,822][20347] Updated weights for policy 0, policy_version 1095476 (0.0008) [2023-12-27 13:14:20,875][20347] Updated weights for policy 0, policy_version 1095486 (0.0008) [2023-12-27 13:14:20,932][20347] Updated weights for policy 0, policy_version 1095496 (0.0008) [2023-12-27 13:14:21,510][20381] Updated weights for policy 1, policy_version 1094495 (0.0008) [2023-12-27 13:14:21,573][20381] Updated weights for policy 1, policy_version 1094505 (0.0008) [2023-12-27 13:14:21,638][20381] Updated weights for policy 1, policy_version 1094515 (0.0008) [2023-12-27 13:14:21,652][20347] Updated weights for policy 0, policy_version 1095506 (0.0008) [2023-12-27 13:14:21,716][20347] Updated weights for policy 0, policy_version 1095516 (0.0007) [2023-12-27 13:14:21,785][20347] Updated weights for policy 0, policy_version 1095526 (0.0007) [2023-12-27 13:14:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 560726016. Throughput: 0: 10155.7, 1: 10130.6. Samples: 560702812. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:22,363][19170] Avg episode reward: [(0, '639.450'), (1, '646.690')] [2023-12-27 13:14:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001095528_280502272.pth... [2023-12-27 13:14:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001094344_280199168.pth [2023-12-27 13:14:22,376][20381] Updated weights for policy 1, policy_version 1094525 (0.0008) [2023-12-27 13:14:22,432][20381] Updated weights for policy 1, policy_version 1094535 (0.0007) [2023-12-27 13:14:22,491][20381] Updated weights for policy 1, policy_version 1094545 (0.0007) [2023-12-27 13:14:22,493][20347] Updated weights for policy 0, policy_version 1095536 (0.0008) [2023-12-27 13:14:22,532][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001094552_280231936.pth... [2023-12-27 13:14:22,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001093368_279928832.pth [2023-12-27 13:14:22,552][20347] Updated weights for policy 0, policy_version 1095546 (0.0008) [2023-12-27 13:14:22,610][20347] Updated weights for policy 0, policy_version 1095556 (0.0008) [2023-12-27 13:14:23,211][20381] Updated weights for policy 1, policy_version 1094555 (0.0007) [2023-12-27 13:14:23,260][20381] Updated weights for policy 1, policy_version 1094565 (0.0008) [2023-12-27 13:14:23,298][20347] Updated weights for policy 0, policy_version 1095566 (0.0009) [2023-12-27 13:14:23,320][20381] Updated weights for policy 1, policy_version 1094575 (0.0009) [2023-12-27 13:14:23,353][20347] Updated weights for policy 0, policy_version 1095576 (0.0008) [2023-12-27 13:14:23,404][20347] Updated weights for policy 0, policy_version 1095586 (0.0009) [2023-12-27 13:14:24,021][20347] Updated weights for policy 0, policy_version 1095596 (0.0007) [2023-12-27 13:14:24,024][20381] Updated weights for policy 1, policy_version 1094585 (0.0007) [2023-12-27 13:14:24,080][20347] Updated weights for policy 0, policy_version 1095606 (0.0006) [2023-12-27 13:14:24,082][20381] Updated weights for policy 1, policy_version 1094595 (0.0006) [2023-12-27 13:14:24,138][20347] Updated weights for policy 0, policy_version 1095616 (0.0006) [2023-12-27 13:14:24,144][20381] Updated weights for policy 1, policy_version 1094605 (0.0006) [2023-12-27 13:14:24,205][20381] Updated weights for policy 1, policy_version 1094615 (0.0006) [2023-12-27 13:14:24,826][20347] Updated weights for policy 0, policy_version 1095626 (0.0006) [2023-12-27 13:14:24,883][20347] Updated weights for policy 0, policy_version 1095636 (0.0006) [2023-12-27 13:14:24,892][20381] Updated weights for policy 1, policy_version 1094625 (0.0006) [2023-12-27 13:14:24,937][20347] Updated weights for policy 0, policy_version 1095646 (0.0007) [2023-12-27 13:14:24,938][20381] Updated weights for policy 1, policy_version 1094635 (0.0006) [2023-12-27 13:14:24,986][20347] Updated weights for policy 0, policy_version 1095656 (0.0008) [2023-12-27 13:14:24,997][20381] Updated weights for policy 1, policy_version 1094645 (0.0005) [2023-12-27 13:14:25,535][20381] Updated weights for policy 1, policy_version 1094655 (0.0005) [2023-12-27 13:14:25,591][20381] Updated weights for policy 1, policy_version 1094665 (0.0005) [2023-12-27 13:14:25,641][20381] Updated weights for policy 1, policy_version 1094675 (0.0006) [2023-12-27 13:14:25,678][20347] Updated weights for policy 0, policy_version 1095666 (0.0005) [2023-12-27 13:14:25,736][20347] Updated weights for policy 0, policy_version 1095676 (0.0005) [2023-12-27 13:14:25,801][20347] Updated weights for policy 0, policy_version 1095686 (0.0005) [2023-12-27 13:14:26,335][20381] Updated weights for policy 1, policy_version 1094685 (0.0008) [2023-12-27 13:14:26,349][20347] Updated weights for policy 0, policy_version 1095696 (0.0005) [2023-12-27 13:14:26,380][20381] Updated weights for policy 1, policy_version 1094695 (0.0007) [2023-12-27 13:14:26,398][20347] Updated weights for policy 0, policy_version 1095706 (0.0005) [2023-12-27 13:14:26,437][20381] Updated weights for policy 1, policy_version 1094705 (0.0008) [2023-12-27 13:14:26,462][20347] Updated weights for policy 0, policy_version 1095716 (0.0007) [2023-12-27 13:14:27,114][20381] Updated weights for policy 1, policy_version 1094715 (0.0007) [2023-12-27 13:14:27,166][20347] Updated weights for policy 0, policy_version 1095726 (0.0007) [2023-12-27 13:14:27,170][20381] Updated weights for policy 1, policy_version 1094725 (0.0008) [2023-12-27 13:14:27,222][20381] Updated weights for policy 1, policy_version 1094735 (0.0010) [2023-12-27 13:14:27,227][20347] Updated weights for policy 0, policy_version 1095736 (0.0005) [2023-12-27 13:14:27,285][20347] Updated weights for policy 0, policy_version 1095746 (0.0006) [2023-12-27 13:14:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 560840704. Throughput: 0: 10205.7, 1: 10096.7. Samples: 560824088. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:27,363][19170] Avg episode reward: [(0, '760.380'), (1, '552.010')] [2023-12-27 13:14:27,875][20381] Updated weights for policy 1, policy_version 1094745 (0.0010) [2023-12-27 13:14:27,923][20381] Updated weights for policy 1, policy_version 1094755 (0.0010) [2023-12-27 13:14:27,936][20347] Updated weights for policy 0, policy_version 1095756 (0.0006) [2023-12-27 13:14:27,974][20381] Updated weights for policy 1, policy_version 1094765 (0.0010) [2023-12-27 13:14:27,987][20347] Updated weights for policy 0, policy_version 1095766 (0.0005) [2023-12-27 13:14:28,028][20381] Updated weights for policy 1, policy_version 1094775 (0.0010) [2023-12-27 13:14:28,042][20347] Updated weights for policy 0, policy_version 1095776 (0.0006) [2023-12-27 13:14:28,680][20381] Updated weights for policy 1, policy_version 1094785 (0.0008) [2023-12-27 13:14:28,736][20381] Updated weights for policy 1, policy_version 1094795 (0.0007) [2023-12-27 13:14:28,740][20347] Updated weights for policy 0, policy_version 1095786 (0.0007) [2023-12-27 13:14:28,790][20381] Updated weights for policy 1, policy_version 1094805 (0.0006) [2023-12-27 13:14:28,799][20347] Updated weights for policy 0, policy_version 1095796 (0.0007) [2023-12-27 13:14:28,856][20347] Updated weights for policy 0, policy_version 1095806 (0.0008) [2023-12-27 13:14:28,916][20347] Updated weights for policy 0, policy_version 1095816 (0.0008) [2023-12-27 13:14:29,386][20381] Updated weights for policy 1, policy_version 1094815 (0.0008) [2023-12-27 13:14:29,450][20381] Updated weights for policy 1, policy_version 1094825 (0.0007) [2023-12-27 13:14:29,510][20381] Updated weights for policy 1, policy_version 1094835 (0.0008) [2023-12-27 13:14:29,648][20347] Updated weights for policy 0, policy_version 1095826 (0.0008) [2023-12-27 13:14:29,706][20347] Updated weights for policy 0, policy_version 1095836 (0.0008) [2023-12-27 13:14:29,762][20347] Updated weights for policy 0, policy_version 1095846 (0.0008) [2023-12-27 13:14:30,226][20381] Updated weights for policy 1, policy_version 1094845 (0.0008) [2023-12-27 13:14:30,278][20381] Updated weights for policy 1, policy_version 1094855 (0.0007) [2023-12-27 13:14:30,337][20381] Updated weights for policy 1, policy_version 1094865 (0.0008) [2023-12-27 13:14:30,464][20347] Updated weights for policy 0, policy_version 1095856 (0.0007) [2023-12-27 13:14:30,519][20347] Updated weights for policy 0, policy_version 1095866 (0.0008) [2023-12-27 13:14:30,564][20347] Updated weights for policy 0, policy_version 1095876 (0.0007) [2023-12-27 13:14:30,948][20381] Updated weights for policy 1, policy_version 1094875 (0.0007) [2023-12-27 13:14:31,002][20381] Updated weights for policy 1, policy_version 1094885 (0.0007) [2023-12-27 13:14:31,059][20381] Updated weights for policy 1, policy_version 1094895 (0.0008) [2023-12-27 13:14:31,185][20347] Updated weights for policy 0, policy_version 1095886 (0.0008) [2023-12-27 13:14:31,236][20347] Updated weights for policy 0, policy_version 1095896 (0.0007) [2023-12-27 13:14:31,294][20347] Updated weights for policy 0, policy_version 1095906 (0.0007) [2023-12-27 13:14:31,716][20381] Updated weights for policy 1, policy_version 1094905 (0.0007) [2023-12-27 13:14:31,779][20381] Updated weights for policy 1, policy_version 1094915 (0.0007) [2023-12-27 13:14:31,835][20381] Updated weights for policy 1, policy_version 1094925 (0.0008) [2023-12-27 13:14:31,889][20381] Updated weights for policy 1, policy_version 1094935 (0.0007) [2023-12-27 13:14:32,060][20347] Updated weights for policy 0, policy_version 1095916 (0.0006) [2023-12-27 13:14:32,117][20347] Updated weights for policy 0, policy_version 1095926 (0.0007) [2023-12-27 13:14:32,182][20347] Updated weights for policy 0, policy_version 1095936 (0.0007) [2023-12-27 13:14:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 560939008. Throughput: 0: 10186.7, 1: 10104.9. Samples: 560947344. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:32,363][19170] Avg episode reward: [(0, '616.440'), (1, '565.020')] [2023-12-27 13:14:32,560][20381] Updated weights for policy 1, policy_version 1094945 (0.0007) [2023-12-27 13:14:32,620][20381] Updated weights for policy 1, policy_version 1094955 (0.0005) [2023-12-27 13:14:32,681][20381] Updated weights for policy 1, policy_version 1094965 (0.0008) [2023-12-27 13:14:32,822][20347] Updated weights for policy 0, policy_version 1095946 (0.0006) [2023-12-27 13:14:32,877][20347] Updated weights for policy 0, policy_version 1095956 (0.0008) [2023-12-27 13:14:32,935][20347] Updated weights for policy 0, policy_version 1095966 (0.0008) [2023-12-27 13:14:32,992][20347] Updated weights for policy 0, policy_version 1095976 (0.0008) [2023-12-27 13:14:33,330][20381] Updated weights for policy 1, policy_version 1094975 (0.0007) [2023-12-27 13:14:33,383][20381] Updated weights for policy 1, policy_version 1094985 (0.0005) [2023-12-27 13:14:33,445][20381] Updated weights for policy 1, policy_version 1094995 (0.0005) [2023-12-27 13:14:33,699][20347] Updated weights for policy 0, policy_version 1095986 (0.0007) [2023-12-27 13:14:33,769][20347] Updated weights for policy 0, policy_version 1095996 (0.0007) [2023-12-27 13:14:33,825][20347] Updated weights for policy 0, policy_version 1096006 (0.0008) [2023-12-27 13:14:34,031][20381] Updated weights for policy 1, policy_version 1095005 (0.0005) [2023-12-27 13:14:34,089][20381] Updated weights for policy 1, policy_version 1095015 (0.0005) [2023-12-27 13:14:34,148][20381] Updated weights for policy 1, policy_version 1095025 (0.0007) [2023-12-27 13:14:34,558][20347] Updated weights for policy 0, policy_version 1096016 (0.0008) [2023-12-27 13:14:34,617][20347] Updated weights for policy 0, policy_version 1096026 (0.0009) [2023-12-27 13:14:34,678][20347] Updated weights for policy 0, policy_version 1096036 (0.0008) [2023-12-27 13:14:34,758][20381] Updated weights for policy 1, policy_version 1095035 (0.0008) [2023-12-27 13:14:34,818][20381] Updated weights for policy 1, policy_version 1095045 (0.0009) [2023-12-27 13:14:34,877][20381] Updated weights for policy 1, policy_version 1095055 (0.0005) [2023-12-27 13:14:35,339][20347] Updated weights for policy 0, policy_version 1096046 (0.0007) [2023-12-27 13:14:35,386][20347] Updated weights for policy 0, policy_version 1096056 (0.0007) [2023-12-27 13:14:35,434][20347] Updated weights for policy 0, policy_version 1096066 (0.0007) [2023-12-27 13:14:35,444][20291] KL-divergence is very high: 113.2473 [2023-12-27 13:14:35,455][20291] KL-divergence is very high: 167.1607 [2023-12-27 13:14:35,460][20291] KL-divergence is very high: 189.4457 [2023-12-27 13:14:35,533][20381] Updated weights for policy 1, policy_version 1095065 (0.0006) [2023-12-27 13:14:35,587][20381] Updated weights for policy 1, policy_version 1095075 (0.0010) [2023-12-27 13:14:35,641][20381] Updated weights for policy 1, policy_version 1095085 (0.0010) [2023-12-27 13:14:35,694][20381] Updated weights for policy 1, policy_version 1095095 (0.0010) [2023-12-27 13:14:36,181][20347] Updated weights for policy 0, policy_version 1096076 (0.0008) [2023-12-27 13:14:36,244][20347] Updated weights for policy 0, policy_version 1096086 (0.0007) [2023-12-27 13:14:36,305][20347] Updated weights for policy 0, policy_version 1096096 (0.0006) [2023-12-27 13:14:36,315][20381] Updated weights for policy 1, policy_version 1095105 (0.0011) [2023-12-27 13:14:36,377][20381] Updated weights for policy 1, policy_version 1095115 (0.0006) [2023-12-27 13:14:36,443][20381] Updated weights for policy 1, policy_version 1095125 (0.0011) [2023-12-27 13:14:37,016][20347] Updated weights for policy 0, policy_version 1096106 (0.0007) [2023-12-27 13:14:37,072][20347] Updated weights for policy 0, policy_version 1096116 (0.0007) [2023-12-27 13:14:37,089][20381] Updated weights for policy 1, policy_version 1095135 (0.0011) [2023-12-27 13:14:37,124][20347] Updated weights for policy 0, policy_version 1096126 (0.0005) [2023-12-27 13:14:37,148][20381] Updated weights for policy 1, policy_version 1095145 (0.0011) [2023-12-27 13:14:37,176][20347] Updated weights for policy 0, policy_version 1096136 (0.0006) [2023-12-27 13:14:37,200][20381] Updated weights for policy 1, policy_version 1095155 (0.0010) [2023-12-27 13:14:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 561045504. Throughput: 0: 10187.3, 1: 10155.1. Samples: 561009400. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:37,363][19170] Avg episode reward: [(0, '723.930'), (1, '654.850')] [2023-12-27 13:14:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001095160_280387584.pth... [2023-12-27 13:14:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001096136_280657920.pth... [2023-12-27 13:14:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001094952_280354816.pth [2023-12-27 13:14:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001093976_280084480.pth [2023-12-27 13:14:37,879][20347] Updated weights for policy 0, policy_version 1096146 (0.0008) [2023-12-27 13:14:37,904][20381] Updated weights for policy 1, policy_version 1095165 (0.0010) [2023-12-27 13:14:37,924][20347] Updated weights for policy 0, policy_version 1096156 (0.0005) [2023-12-27 13:14:37,949][20381] Updated weights for policy 1, policy_version 1095175 (0.0010) [2023-12-27 13:14:37,973][20347] Updated weights for policy 0, policy_version 1096166 (0.0005) [2023-12-27 13:14:37,995][20381] Updated weights for policy 1, policy_version 1095185 (0.0010) [2023-12-27 13:14:38,720][20347] Updated weights for policy 0, policy_version 1096176 (0.0007) [2023-12-27 13:14:38,733][20381] Updated weights for policy 1, policy_version 1095195 (0.0008) [2023-12-27 13:14:38,777][20347] Updated weights for policy 0, policy_version 1096186 (0.0006) [2023-12-27 13:14:38,783][20381] Updated weights for policy 1, policy_version 1095205 (0.0011) [2023-12-27 13:14:38,831][20347] Updated weights for policy 0, policy_version 1096196 (0.0006) [2023-12-27 13:14:38,837][20381] Updated weights for policy 1, policy_version 1095215 (0.0011) [2023-12-27 13:14:39,531][20381] Updated weights for policy 1, policy_version 1095225 (0.0010) [2023-12-27 13:14:39,582][20347] Updated weights for policy 0, policy_version 1096206 (0.0007) [2023-12-27 13:14:39,595][20381] Updated weights for policy 1, policy_version 1095235 (0.0006) [2023-12-27 13:14:39,643][20347] Updated weights for policy 0, policy_version 1096216 (0.0006) [2023-12-27 13:14:39,654][20381] Updated weights for policy 1, policy_version 1095245 (0.0008) [2023-12-27 13:14:39,701][20347] Updated weights for policy 0, policy_version 1096226 (0.0008) [2023-12-27 13:14:39,718][20381] Updated weights for policy 1, policy_version 1095255 (0.0011) [2023-12-27 13:14:40,404][20381] Updated weights for policy 1, policy_version 1095265 (0.0011) [2023-12-27 13:14:40,444][20347] Updated weights for policy 0, policy_version 1096236 (0.0006) [2023-12-27 13:14:40,451][20381] Updated weights for policy 1, policy_version 1095275 (0.0011) [2023-12-27 13:14:40,504][20347] Updated weights for policy 0, policy_version 1096246 (0.0007) [2023-12-27 13:14:40,508][20381] Updated weights for policy 1, policy_version 1095285 (0.0010) [2023-12-27 13:14:40,560][20347] Updated weights for policy 0, policy_version 1096256 (0.0008) [2023-12-27 13:14:41,232][20381] Updated weights for policy 1, policy_version 1095295 (0.0010) [2023-12-27 13:14:41,295][20381] Updated weights for policy 1, policy_version 1095305 (0.0011) [2023-12-27 13:14:41,311][20347] Updated weights for policy 0, policy_version 1096266 (0.0007) [2023-12-27 13:14:41,359][20381] Updated weights for policy 1, policy_version 1095315 (0.0009) [2023-12-27 13:14:41,374][20347] Updated weights for policy 0, policy_version 1096276 (0.0008) [2023-12-27 13:14:41,436][20347] Updated weights for policy 0, policy_version 1096286 (0.0007) [2023-12-27 13:14:41,484][20347] Updated weights for policy 0, policy_version 1096296 (0.0006) [2023-12-27 13:14:42,049][20381] Updated weights for policy 1, policy_version 1095325 (0.0008) [2023-12-27 13:14:42,109][20381] Updated weights for policy 1, policy_version 1095335 (0.0006) [2023-12-27 13:14:42,170][20381] Updated weights for policy 1, policy_version 1095345 (0.0005) [2023-12-27 13:14:42,250][20347] Updated weights for policy 0, policy_version 1096306 (0.0006) [2023-12-27 13:14:42,315][20347] Updated weights for policy 0, policy_version 1096316 (0.0007) [2023-12-27 13:14:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 561135616. Throughput: 0: 10125.1, 1: 10174.7. Samples: 561128956. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:42,363][19170] Avg episode reward: [(0, '854.330'), (1, '695.580')] [2023-12-27 13:14:42,384][20347] Updated weights for policy 0, policy_version 1096326 (0.0007) [2023-12-27 13:14:42,872][20381] Updated weights for policy 1, policy_version 1095355 (0.0007) [2023-12-27 13:14:42,923][20381] Updated weights for policy 1, policy_version 1095365 (0.0006) [2023-12-27 13:14:42,946][20347] Updated weights for policy 0, policy_version 1096336 (0.0005) [2023-12-27 13:14:42,983][20381] Updated weights for policy 1, policy_version 1095375 (0.0005) [2023-12-27 13:14:42,991][20347] Updated weights for policy 0, policy_version 1096346 (0.0006) [2023-12-27 13:14:43,044][20347] Updated weights for policy 0, policy_version 1096356 (0.0007) [2023-12-27 13:14:43,586][20347] Updated weights for policy 0, policy_version 1096366 (0.0005) [2023-12-27 13:14:43,631][20347] Updated weights for policy 0, policy_version 1096376 (0.0005) [2023-12-27 13:14:43,680][20347] Updated weights for policy 0, policy_version 1096386 (0.0005) [2023-12-27 13:14:43,713][20381] Updated weights for policy 1, policy_version 1095385 (0.0007) [2023-12-27 13:14:43,768][20381] Updated weights for policy 1, policy_version 1095395 (0.0007) [2023-12-27 13:14:43,819][20381] Updated weights for policy 1, policy_version 1095405 (0.0008) [2023-12-27 13:14:43,867][20381] Updated weights for policy 1, policy_version 1095415 (0.0006) [2023-12-27 13:14:44,283][20347] Updated weights for policy 0, policy_version 1096396 (0.0006) [2023-12-27 13:14:44,342][20347] Updated weights for policy 0, policy_version 1096406 (0.0009) [2023-12-27 13:14:44,402][20347] Updated weights for policy 0, policy_version 1096416 (0.0010) [2023-12-27 13:14:44,441][20381] Updated weights for policy 1, policy_version 1095425 (0.0006) [2023-12-27 13:14:44,491][20381] Updated weights for policy 1, policy_version 1095435 (0.0007) [2023-12-27 13:14:44,544][20381] Updated weights for policy 1, policy_version 1095445 (0.0008) [2023-12-27 13:14:45,116][20347] Updated weights for policy 0, policy_version 1096426 (0.0010) [2023-12-27 13:14:45,180][20347] Updated weights for policy 0, policy_version 1096436 (0.0007) [2023-12-27 13:14:45,191][20381] Updated weights for policy 1, policy_version 1095455 (0.0007) [2023-12-27 13:14:45,241][20347] Updated weights for policy 0, policy_version 1096446 (0.0006) [2023-12-27 13:14:45,247][20381] Updated weights for policy 1, policy_version 1095465 (0.0008) [2023-12-27 13:14:45,300][20347] Updated weights for policy 0, policy_version 1096456 (0.0006) [2023-12-27 13:14:45,303][20381] Updated weights for policy 1, policy_version 1095475 (0.0006) [2023-12-27 13:14:45,948][20347] Updated weights for policy 0, policy_version 1096466 (0.0007) [2023-12-27 13:14:46,000][20381] Updated weights for policy 1, policy_version 1095485 (0.0009) [2023-12-27 13:14:46,002][20347] Updated weights for policy 0, policy_version 1096476 (0.0007) [2023-12-27 13:14:46,050][20347] Updated weights for policy 0, policy_version 1096486 (0.0005) [2023-12-27 13:14:46,051][20381] Updated weights for policy 1, policy_version 1095495 (0.0010) [2023-12-27 13:14:46,096][20381] Updated weights for policy 1, policy_version 1095505 (0.0010) [2023-12-27 13:14:46,623][20347] Updated weights for policy 0, policy_version 1096496 (0.0007) [2023-12-27 13:14:46,683][20347] Updated weights for policy 0, policy_version 1096506 (0.0008) [2023-12-27 13:14:46,742][20347] Updated weights for policy 0, policy_version 1096516 (0.0008) [2023-12-27 13:14:46,799][20381] Updated weights for policy 1, policy_version 1095515 (0.0009) [2023-12-27 13:14:46,847][20381] Updated weights for policy 1, policy_version 1095525 (0.0010) [2023-12-27 13:14:46,896][20381] Updated weights for policy 1, policy_version 1095535 (0.0010) [2023-12-27 13:14:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 561242112. Throughput: 0: 10237.7, 1: 10188.4. Samples: 561255440. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:47,363][19170] Avg episode reward: [(0, '835.320'), (1, '627.000')] [2023-12-27 13:14:47,389][20347] Updated weights for policy 0, policy_version 1096526 (0.0006) [2023-12-27 13:14:47,444][20347] Updated weights for policy 0, policy_version 1096536 (0.0005) [2023-12-27 13:14:47,505][20347] Updated weights for policy 0, policy_version 1096546 (0.0007) [2023-12-27 13:14:47,539][20381] Updated weights for policy 1, policy_version 1095545 (0.0010) [2023-12-27 13:14:47,594][20381] Updated weights for policy 1, policy_version 1095555 (0.0011) [2023-12-27 13:14:47,644][20381] Updated weights for policy 1, policy_version 1095565 (0.0008) [2023-12-27 13:14:47,697][20381] Updated weights for policy 1, policy_version 1095575 (0.0005) [2023-12-27 13:14:48,165][20347] Updated weights for policy 0, policy_version 1096556 (0.0007) [2023-12-27 13:14:48,218][20347] Updated weights for policy 0, policy_version 1096566 (0.0007) [2023-12-27 13:14:48,272][20347] Updated weights for policy 0, policy_version 1096576 (0.0007) [2023-12-27 13:14:48,383][20381] Updated weights for policy 1, policy_version 1095585 (0.0008) [2023-12-27 13:14:48,450][20381] Updated weights for policy 1, policy_version 1095595 (0.0008) [2023-12-27 13:14:48,506][20381] Updated weights for policy 1, policy_version 1095605 (0.0009) [2023-12-27 13:14:48,966][20347] Updated weights for policy 0, policy_version 1096586 (0.0008) [2023-12-27 13:14:49,026][20347] Updated weights for policy 0, policy_version 1096596 (0.0007) [2023-12-27 13:14:49,073][20291] KL-divergence is very high: 109.5974 [2023-12-27 13:14:49,080][20347] Updated weights for policy 0, policy_version 1096606 (0.0010) [2023-12-27 13:14:49,128][20347] Updated weights for policy 0, policy_version 1096616 (0.0010) [2023-12-27 13:14:49,211][20381] Updated weights for policy 1, policy_version 1095615 (0.0010) [2023-12-27 13:14:49,274][20381] Updated weights for policy 1, policy_version 1095625 (0.0010) [2023-12-27 13:14:49,336][20381] Updated weights for policy 1, policy_version 1095635 (0.0010) [2023-12-27 13:14:49,852][20347] Updated weights for policy 0, policy_version 1096626 (0.0007) [2023-12-27 13:14:49,911][20347] Updated weights for policy 0, policy_version 1096636 (0.0008) [2023-12-27 13:14:49,974][20347] Updated weights for policy 0, policy_version 1096646 (0.0008) [2023-12-27 13:14:50,046][20381] Updated weights for policy 1, policy_version 1095645 (0.0008) [2023-12-27 13:14:50,107][20381] Updated weights for policy 1, policy_version 1095655 (0.0008) [2023-12-27 13:14:50,163][20381] Updated weights for policy 1, policy_version 1095665 (0.0008) [2023-12-27 13:14:50,677][20347] Updated weights for policy 0, policy_version 1096656 (0.0007) [2023-12-27 13:14:50,736][20347] Updated weights for policy 0, policy_version 1096666 (0.0007) [2023-12-27 13:14:50,788][20347] Updated weights for policy 0, policy_version 1096676 (0.0008) [2023-12-27 13:14:50,861][20381] Updated weights for policy 1, policy_version 1095675 (0.0008) [2023-12-27 13:14:50,921][20381] Updated weights for policy 1, policy_version 1095685 (0.0008) [2023-12-27 13:14:50,970][20381] Updated weights for policy 1, policy_version 1095695 (0.0008) [2023-12-27 13:14:51,519][20347] Updated weights for policy 0, policy_version 1096686 (0.0008) [2023-12-27 13:14:51,574][20347] Updated weights for policy 0, policy_version 1096696 (0.0008) [2023-12-27 13:14:51,628][20347] Updated weights for policy 0, policy_version 1096706 (0.0008) [2023-12-27 13:14:51,743][20381] Updated weights for policy 1, policy_version 1095705 (0.0008) [2023-12-27 13:14:51,794][20381] Updated weights for policy 1, policy_version 1095715 (0.0008) [2023-12-27 13:14:51,844][20381] Updated weights for policy 1, policy_version 1095725 (0.0008) [2023-12-27 13:14:51,897][20381] Updated weights for policy 1, policy_version 1095735 (0.0008) [2023-12-27 13:14:52,330][20347] Updated weights for policy 0, policy_version 1096716 (0.0007) [2023-12-27 13:14:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 561340416. Throughput: 0: 10220.4, 1: 10166.2. Samples: 561315560. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:52,363][19170] Avg episode reward: [(0, '845.190'), (1, '636.970')] [2023-12-27 13:14:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001095736_280535040.pth... [2023-12-27 13:14:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001094552_280231936.pth [2023-12-27 13:14:52,395][20347] Updated weights for policy 0, policy_version 1096726 (0.0007) [2023-12-27 13:14:52,460][20347] Updated weights for policy 0, policy_version 1096736 (0.0008) [2023-12-27 13:14:52,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001096744_280813568.pth... [2023-12-27 13:14:52,511][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001095528_280502272.pth [2023-12-27 13:14:52,642][20381] Updated weights for policy 1, policy_version 1095745 (0.0008) [2023-12-27 13:14:52,703][20381] Updated weights for policy 1, policy_version 1095755 (0.0007) [2023-12-27 13:14:52,767][20381] Updated weights for policy 1, policy_version 1095765 (0.0006) [2023-12-27 13:14:53,104][20347] Updated weights for policy 0, policy_version 1096746 (0.0008) [2023-12-27 13:14:53,149][20347] Updated weights for policy 0, policy_version 1096756 (0.0010) [2023-12-27 13:14:53,209][20347] Updated weights for policy 0, policy_version 1096766 (0.0008) [2023-12-27 13:14:53,273][20347] Updated weights for policy 0, policy_version 1096776 (0.0008) [2023-12-27 13:14:53,356][20381] Updated weights for policy 1, policy_version 1095775 (0.0007) [2023-12-27 13:14:53,410][20381] Updated weights for policy 1, policy_version 1095785 (0.0007) [2023-12-27 13:14:53,465][20381] Updated weights for policy 1, policy_version 1095795 (0.0008) [2023-12-27 13:14:53,960][20347] Updated weights for policy 0, policy_version 1096786 (0.0006) [2023-12-27 13:14:54,012][20347] Updated weights for policy 0, policy_version 1096796 (0.0008) [2023-12-27 13:14:54,067][20347] Updated weights for policy 0, policy_version 1096806 (0.0008) [2023-12-27 13:14:54,182][20381] Updated weights for policy 1, policy_version 1095805 (0.0007) [2023-12-27 13:14:54,249][20381] Updated weights for policy 1, policy_version 1095815 (0.0006) [2023-12-27 13:14:54,311][20381] Updated weights for policy 1, policy_version 1095825 (0.0006) [2023-12-27 13:14:54,733][20347] Updated weights for policy 0, policy_version 1096816 (0.0008) [2023-12-27 13:14:54,791][20347] Updated weights for policy 0, policy_version 1096826 (0.0007) [2023-12-27 13:14:54,854][20347] Updated weights for policy 0, policy_version 1096836 (0.0008) [2023-12-27 13:14:54,982][20381] Updated weights for policy 1, policy_version 1095835 (0.0007) [2023-12-27 13:14:55,034][20381] Updated weights for policy 1, policy_version 1095845 (0.0009) [2023-12-27 13:14:55,091][20381] Updated weights for policy 1, policy_version 1095855 (0.0008) [2023-12-27 13:14:55,493][20347] Updated weights for policy 0, policy_version 1096846 (0.0007) [2023-12-27 13:14:55,556][20347] Updated weights for policy 0, policy_version 1096856 (0.0005) [2023-12-27 13:14:55,605][20347] Updated weights for policy 0, policy_version 1096866 (0.0005) [2023-12-27 13:14:55,722][20381] Updated weights for policy 1, policy_version 1095865 (0.0007) [2023-12-27 13:14:55,783][20381] Updated weights for policy 1, policy_version 1095875 (0.0007) [2023-12-27 13:14:55,849][20381] Updated weights for policy 1, policy_version 1095885 (0.0007) [2023-12-27 13:14:55,903][20381] Updated weights for policy 1, policy_version 1095895 (0.0007) [2023-12-27 13:14:56,220][20347] Updated weights for policy 0, policy_version 1096876 (0.0006) [2023-12-27 13:14:56,272][20347] Updated weights for policy 0, policy_version 1096886 (0.0007) [2023-12-27 13:14:56,331][20347] Updated weights for policy 0, policy_version 1096896 (0.0009) [2023-12-27 13:14:56,560][20381] Updated weights for policy 1, policy_version 1095905 (0.0006) [2023-12-27 13:14:56,620][20381] Updated weights for policy 1, policy_version 1095915 (0.0005) [2023-12-27 13:14:56,678][20381] Updated weights for policy 1, policy_version 1095925 (0.0005) [2023-12-27 13:14:56,987][20347] Updated weights for policy 0, policy_version 1096906 (0.0010) [2023-12-27 13:14:57,046][20347] Updated weights for policy 0, policy_version 1096916 (0.0008) [2023-12-27 13:14:57,103][20347] Updated weights for policy 0, policy_version 1096926 (0.0007) [2023-12-27 13:14:57,166][20347] Updated weights for policy 0, policy_version 1096936 (0.0007) [2023-12-27 13:14:57,279][20381] Updated weights for policy 1, policy_version 1095935 (0.0007) [2023-12-27 13:14:57,343][20381] Updated weights for policy 1, policy_version 1095945 (0.0007) [2023-12-27 13:14:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 561446912. Throughput: 0: 10200.0, 1: 10185.0. Samples: 561437372. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:14:57,363][19170] Avg episode reward: [(0, '834.990'), (1, '607.330')] [2023-12-27 13:14:57,409][20381] Updated weights for policy 1, policy_version 1095955 (0.0007) [2023-12-27 13:14:57,862][20347] Updated weights for policy 0, policy_version 1096946 (0.0007) [2023-12-27 13:14:57,920][20347] Updated weights for policy 0, policy_version 1096956 (0.0006) [2023-12-27 13:14:57,963][20347] Updated weights for policy 0, policy_version 1096966 (0.0005) [2023-12-27 13:14:58,045][20381] Updated weights for policy 1, policy_version 1095965 (0.0008) [2023-12-27 13:14:58,100][20381] Updated weights for policy 1, policy_version 1095975 (0.0007) [2023-12-27 13:14:58,155][20381] Updated weights for policy 1, policy_version 1095985 (0.0008) [2023-12-27 13:14:58,697][20347] Updated weights for policy 0, policy_version 1096976 (0.0009) [2023-12-27 13:14:58,759][20347] Updated weights for policy 0, policy_version 1096986 (0.0008) [2023-12-27 13:14:58,824][20347] Updated weights for policy 0, policy_version 1096996 (0.0009) [2023-12-27 13:14:58,944][20381] Updated weights for policy 1, policy_version 1095995 (0.0009) [2023-12-27 13:14:59,003][20381] Updated weights for policy 1, policy_version 1096005 (0.0008) [2023-12-27 13:14:59,060][20381] Updated weights for policy 1, policy_version 1096015 (0.0009) [2023-12-27 13:14:59,560][20347] Updated weights for policy 0, policy_version 1097006 (0.0008) [2023-12-27 13:14:59,621][20347] Updated weights for policy 0, policy_version 1097016 (0.0008) [2023-12-27 13:14:59,681][20347] Updated weights for policy 0, policy_version 1097026 (0.0007) [2023-12-27 13:14:59,768][20381] Updated weights for policy 1, policy_version 1096025 (0.0008) [2023-12-27 13:14:59,832][20381] Updated weights for policy 1, policy_version 1096035 (0.0008) [2023-12-27 13:14:59,886][20381] Updated weights for policy 1, policy_version 1096045 (0.0007) [2023-12-27 13:14:59,954][20381] Updated weights for policy 1, policy_version 1096055 (0.0006) [2023-12-27 13:15:00,374][20347] Updated weights for policy 0, policy_version 1097036 (0.0007) [2023-12-27 13:15:00,430][20347] Updated weights for policy 0, policy_version 1097046 (0.0008) [2023-12-27 13:15:00,484][20347] Updated weights for policy 0, policy_version 1097056 (0.0007) [2023-12-27 13:15:00,619][20381] Updated weights for policy 1, policy_version 1096065 (0.0005) [2023-12-27 13:15:00,677][20381] Updated weights for policy 1, policy_version 1096075 (0.0005) [2023-12-27 13:15:00,734][20381] Updated weights for policy 1, policy_version 1096085 (0.0007) [2023-12-27 13:15:01,119][20347] Updated weights for policy 0, policy_version 1097066 (0.0006) [2023-12-27 13:15:01,191][20347] Updated weights for policy 0, policy_version 1097076 (0.0007) [2023-12-27 13:15:01,256][20347] Updated weights for policy 0, policy_version 1097086 (0.0007) [2023-12-27 13:15:01,313][20347] Updated weights for policy 0, policy_version 1097096 (0.0007) [2023-12-27 13:15:01,318][20381] Updated weights for policy 1, policy_version 1096095 (0.0006) [2023-12-27 13:15:01,395][20381] Updated weights for policy 1, policy_version 1096105 (0.0008) [2023-12-27 13:15:01,449][20381] Updated weights for policy 1, policy_version 1096115 (0.0006) [2023-12-27 13:15:01,978][20347] Updated weights for policy 0, policy_version 1097106 (0.0010) [2023-12-27 13:15:02,031][20347] Updated weights for policy 0, policy_version 1097116 (0.0010) [2023-12-27 13:15:02,077][20347] Updated weights for policy 0, policy_version 1097126 (0.0010) [2023-12-27 13:15:02,182][20381] Updated weights for policy 1, policy_version 1096125 (0.0007) [2023-12-27 13:15:02,238][20381] Updated weights for policy 1, policy_version 1096135 (0.0007) [2023-12-27 13:15:02,302][20381] Updated weights for policy 1, policy_version 1096145 (0.0008) [2023-12-27 13:15:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 561553408. Throughput: 0: 10133.1, 1: 10226.8. Samples: 561559156. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:02,363][19170] Avg episode reward: [(0, '834.560'), (1, '627.130')] [2023-12-27 13:15:02,764][20347] Updated weights for policy 0, policy_version 1097136 (0.0008) [2023-12-27 13:15:02,816][20347] Updated weights for policy 0, policy_version 1097146 (0.0008) [2023-12-27 13:15:02,871][20347] Updated weights for policy 0, policy_version 1097156 (0.0008) [2023-12-27 13:15:02,974][20381] Updated weights for policy 1, policy_version 1096155 (0.0007) [2023-12-27 13:15:03,022][20381] Updated weights for policy 1, policy_version 1096165 (0.0005) [2023-12-27 13:15:03,078][20381] Updated weights for policy 1, policy_version 1096175 (0.0005) [2023-12-27 13:15:03,576][20347] Updated weights for policy 0, policy_version 1097166 (0.0008) [2023-12-27 13:15:03,627][20347] Updated weights for policy 0, policy_version 1097176 (0.0007) [2023-12-27 13:15:03,675][20347] Updated weights for policy 0, policy_version 1097186 (0.0008) [2023-12-27 13:15:03,713][20381] Updated weights for policy 1, policy_version 1096185 (0.0006) [2023-12-27 13:15:03,770][20381] Updated weights for policy 1, policy_version 1096195 (0.0008) [2023-12-27 13:15:03,821][20381] Updated weights for policy 1, policy_version 1096205 (0.0008) [2023-12-27 13:15:03,889][20381] Updated weights for policy 1, policy_version 1096215 (0.0008) [2023-12-27 13:15:04,399][20347] Updated weights for policy 0, policy_version 1097196 (0.0008) [2023-12-27 13:15:04,452][20347] Updated weights for policy 0, policy_version 1097206 (0.0011) [2023-12-27 13:15:04,504][20347] Updated weights for policy 0, policy_version 1097216 (0.0010) [2023-12-27 13:15:04,611][20381] Updated weights for policy 1, policy_version 1096225 (0.0008) [2023-12-27 13:15:04,667][20381] Updated weights for policy 1, policy_version 1096235 (0.0007) [2023-12-27 13:15:04,721][20381] Updated weights for policy 1, policy_version 1096245 (0.0008) [2023-12-27 13:15:05,145][20347] Updated weights for policy 0, policy_version 1097226 (0.0011) [2023-12-27 13:15:05,194][20347] Updated weights for policy 0, policy_version 1097236 (0.0009) [2023-12-27 13:15:05,245][20347] Updated weights for policy 0, policy_version 1097246 (0.0006) [2023-12-27 13:15:05,303][20347] Updated weights for policy 0, policy_version 1097256 (0.0005) [2023-12-27 13:15:05,384][20381] Updated weights for policy 1, policy_version 1096255 (0.0007) [2023-12-27 13:15:05,443][20381] Updated weights for policy 1, policy_version 1096265 (0.0008) [2023-12-27 13:15:05,499][20381] Updated weights for policy 1, policy_version 1096275 (0.0009) [2023-12-27 13:15:05,850][20347] Updated weights for policy 0, policy_version 1097266 (0.0005) [2023-12-27 13:15:05,915][20347] Updated weights for policy 0, policy_version 1097276 (0.0006) [2023-12-27 13:15:05,979][20347] Updated weights for policy 0, policy_version 1097286 (0.0006) [2023-12-27 13:15:06,250][20381] Updated weights for policy 1, policy_version 1096285 (0.0008) [2023-12-27 13:15:06,308][20381] Updated weights for policy 1, policy_version 1096295 (0.0006) [2023-12-27 13:15:06,364][20381] Updated weights for policy 1, policy_version 1096305 (0.0007) [2023-12-27 13:15:06,630][20347] Updated weights for policy 0, policy_version 1097296 (0.0006) [2023-12-27 13:15:06,632][20291] KL-divergence is very high: 303.5321 [2023-12-27 13:15:06,680][20291] KL-divergence is very high: 247.6444 [2023-12-27 13:15:06,693][20347] Updated weights for policy 0, policy_version 1097306 (0.0007) [2023-12-27 13:15:06,753][20347] Updated weights for policy 0, policy_version 1097316 (0.0006) [2023-12-27 13:15:06,926][20381] Updated weights for policy 1, policy_version 1096315 (0.0007) [2023-12-27 13:15:06,976][20381] Updated weights for policy 1, policy_version 1096325 (0.0008) [2023-12-27 13:15:07,032][20381] Updated weights for policy 1, policy_version 1096335 (0.0008) [2023-12-27 13:15:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 561651712. Throughput: 0: 10144.1, 1: 10247.2. Samples: 561620420. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:07,363][19170] Avg episode reward: [(0, '756.150'), (1, '705.490')] [2023-12-27 13:15:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001096344_280690688.pth... [2023-12-27 13:15:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001095160_280387584.pth [2023-12-27 13:15:07,401][20347] Updated weights for policy 0, policy_version 1097326 (0.0007) [2023-12-27 13:15:07,449][20347] Updated weights for policy 0, policy_version 1097336 (0.0008) [2023-12-27 13:15:07,459][20291] KL-divergence is very high: 103.8740 [2023-12-27 13:15:07,502][20291] KL-divergence is very high: 168.9939 [2023-12-27 13:15:07,503][20347] Updated weights for policy 0, policy_version 1097346 (0.0007) [2023-12-27 13:15:07,508][20291] KL-divergence is very high: 114.5222 [2023-12-27 13:15:07,535][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001097352_280969216.pth... [2023-12-27 13:15:07,539][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001096136_280657920.pth [2023-12-27 13:15:07,704][20381] Updated weights for policy 1, policy_version 1096345 (0.0008) [2023-12-27 13:15:07,756][20381] Updated weights for policy 1, policy_version 1096355 (0.0008) [2023-12-27 13:15:07,811][20381] Updated weights for policy 1, policy_version 1096365 (0.0007) [2023-12-27 13:15:07,868][20381] Updated weights for policy 1, policy_version 1096375 (0.0008) [2023-12-27 13:15:08,184][20347] Updated weights for policy 0, policy_version 1097356 (0.0008) [2023-12-27 13:15:08,241][20347] Updated weights for policy 0, policy_version 1097366 (0.0007) [2023-12-27 13:15:08,286][20347] Updated weights for policy 0, policy_version 1097376 (0.0007) [2023-12-27 13:15:08,504][20381] Updated weights for policy 1, policy_version 1096385 (0.0008) [2023-12-27 13:15:08,560][20381] Updated weights for policy 1, policy_version 1096395 (0.0007) [2023-12-27 13:15:08,623][20381] Updated weights for policy 1, policy_version 1096405 (0.0008) [2023-12-27 13:15:09,016][20347] Updated weights for policy 0, policy_version 1097386 (0.0007) [2023-12-27 13:15:09,075][20347] Updated weights for policy 0, policy_version 1097396 (0.0007) [2023-12-27 13:15:09,130][20347] Updated weights for policy 0, policy_version 1097406 (0.0008) [2023-12-27 13:15:09,183][20347] Updated weights for policy 0, policy_version 1097416 (0.0007) [2023-12-27 13:15:09,338][20381] Updated weights for policy 1, policy_version 1096415 (0.0007) [2023-12-27 13:15:09,413][20381] Updated weights for policy 1, policy_version 1096425 (0.0008) [2023-12-27 13:15:09,480][20381] Updated weights for policy 1, policy_version 1096435 (0.0008) [2023-12-27 13:15:10,036][20347] Updated weights for policy 0, policy_version 1097426 (0.0007) [2023-12-27 13:15:10,042][20291] KL-divergence is very high: 169.5737 [2023-12-27 13:15:10,103][20347] Updated weights for policy 0, policy_version 1097436 (0.0009) [2023-12-27 13:15:10,164][20381] Updated weights for policy 1, policy_version 1096445 (0.0007) [2023-12-27 13:15:10,166][20347] Updated weights for policy 0, policy_version 1097446 (0.0007) [2023-12-27 13:15:10,220][20381] Updated weights for policy 1, policy_version 1096455 (0.0008) [2023-12-27 13:15:10,271][20381] Updated weights for policy 1, policy_version 1096465 (0.0008) [2023-12-27 13:15:10,800][20291] KL-divergence is very high: 268.6963 [2023-12-27 13:15:10,846][20347] Updated weights for policy 0, policy_version 1097456 (0.0009) [2023-12-27 13:15:10,847][20291] KL-divergence is very high: 185.3613 [2023-12-27 13:15:10,899][20291] KL-divergence is very high: 196.8734 [2023-12-27 13:15:10,913][20347] Updated weights for policy 0, policy_version 1097466 (0.0009) [2023-12-27 13:15:10,955][20291] KL-divergence is very high: 198.3796 [2023-12-27 13:15:10,969][20381] Updated weights for policy 1, policy_version 1096475 (0.0007) [2023-12-27 13:15:10,981][20347] Updated weights for policy 0, policy_version 1097476 (0.0008) [2023-12-27 13:15:11,022][20381] Updated weights for policy 1, policy_version 1096485 (0.0007) [2023-12-27 13:15:11,083][20381] Updated weights for policy 1, policy_version 1096495 (0.0011) [2023-12-27 13:15:11,764][20347] Updated weights for policy 0, policy_version 1097486 (0.0008) [2023-12-27 13:15:11,816][20381] Updated weights for policy 1, policy_version 1096505 (0.0009) [2023-12-27 13:15:11,826][20347] Updated weights for policy 0, policy_version 1097496 (0.0009) [2023-12-27 13:15:11,880][20381] Updated weights for policy 1, policy_version 1096515 (0.0007) [2023-12-27 13:15:11,886][20347] Updated weights for policy 0, policy_version 1097506 (0.0008) [2023-12-27 13:15:11,936][20381] Updated weights for policy 1, policy_version 1096525 (0.0006) [2023-12-27 13:15:11,997][20381] Updated weights for policy 1, policy_version 1096535 (0.0008) [2023-12-27 13:15:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 561750016. Throughput: 0: 10120.0, 1: 10294.4. Samples: 561742736. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:12,363][19170] Avg episode reward: [(0, '736.370'), (1, '627.210')] [2023-12-27 13:15:12,563][20347] Updated weights for policy 0, policy_version 1097516 (0.0008) [2023-12-27 13:15:12,612][20347] Updated weights for policy 0, policy_version 1097526 (0.0009) [2023-12-27 13:15:12,630][20381] Updated weights for policy 1, policy_version 1096545 (0.0005) [2023-12-27 13:15:12,665][20347] Updated weights for policy 0, policy_version 1097536 (0.0009) [2023-12-27 13:15:12,691][20381] Updated weights for policy 1, policy_version 1096555 (0.0006) [2023-12-27 13:15:12,751][20381] Updated weights for policy 1, policy_version 1096565 (0.0009) [2023-12-27 13:15:13,322][20381] Updated weights for policy 1, policy_version 1096575 (0.0006) [2023-12-27 13:15:13,372][20381] Updated weights for policy 1, policy_version 1096585 (0.0006) [2023-12-27 13:15:13,393][20347] Updated weights for policy 0, policy_version 1097546 (0.0005) [2023-12-27 13:15:13,422][20381] Updated weights for policy 1, policy_version 1096595 (0.0006) [2023-12-27 13:15:13,443][20347] Updated weights for policy 0, policy_version 1097556 (0.0005) [2023-12-27 13:15:13,498][20347] Updated weights for policy 0, policy_version 1097566 (0.0007) [2023-12-27 13:15:13,554][20347] Updated weights for policy 0, policy_version 1097576 (0.0008) [2023-12-27 13:15:14,133][20381] Updated weights for policy 1, policy_version 1096605 (0.0007) [2023-12-27 13:15:14,166][20347] Updated weights for policy 0, policy_version 1097586 (0.0005) [2023-12-27 13:15:14,190][20381] Updated weights for policy 1, policy_version 1096615 (0.0006) [2023-12-27 13:15:14,219][20347] Updated weights for policy 0, policy_version 1097596 (0.0006) [2023-12-27 13:15:14,247][20381] Updated weights for policy 1, policy_version 1096625 (0.0006) [2023-12-27 13:15:14,271][20347] Updated weights for policy 0, policy_version 1097606 (0.0006) [2023-12-27 13:15:14,871][20381] Updated weights for policy 1, policy_version 1096635 (0.0007) [2023-12-27 13:15:14,920][20347] Updated weights for policy 0, policy_version 1097616 (0.0008) [2023-12-27 13:15:14,920][20381] Updated weights for policy 1, policy_version 1096645 (0.0006) [2023-12-27 13:15:14,979][20381] Updated weights for policy 1, policy_version 1096655 (0.0008) [2023-12-27 13:15:14,980][20347] Updated weights for policy 0, policy_version 1097626 (0.0008) [2023-12-27 13:15:14,985][20291] KL-divergence is very high: 124.6284 [2023-12-27 13:15:15,034][20291] KL-divergence is very high: 172.5483 [2023-12-27 13:15:15,041][20347] Updated weights for policy 0, policy_version 1097636 (0.0010) [2023-12-27 13:15:15,640][20381] Updated weights for policy 1, policy_version 1096665 (0.0007) [2023-12-27 13:15:15,687][20347] Updated weights for policy 0, policy_version 1097646 (0.0007) [2023-12-27 13:15:15,700][20381] Updated weights for policy 1, policy_version 1096675 (0.0007) [2023-12-27 13:15:15,737][20347] Updated weights for policy 0, policy_version 1097656 (0.0005) [2023-12-27 13:15:15,758][20381] Updated weights for policy 1, policy_version 1096685 (0.0006) [2023-12-27 13:15:15,795][20347] Updated weights for policy 0, policy_version 1097666 (0.0008) [2023-12-27 13:15:15,815][20381] Updated weights for policy 1, policy_version 1096695 (0.0007) [2023-12-27 13:15:16,453][20347] Updated weights for policy 0, policy_version 1097676 (0.0010) [2023-12-27 13:15:16,496][20381] Updated weights for policy 1, policy_version 1096705 (0.0005) [2023-12-27 13:15:16,501][20347] Updated weights for policy 0, policy_version 1097686 (0.0005) [2023-12-27 13:15:16,549][20381] Updated weights for policy 1, policy_version 1096715 (0.0007) [2023-12-27 13:15:16,565][20347] Updated weights for policy 0, policy_version 1097696 (0.0008) [2023-12-27 13:15:16,611][20381] Updated weights for policy 1, policy_version 1096725 (0.0007) [2023-12-27 13:15:17,163][20347] Updated weights for policy 0, policy_version 1097706 (0.0006) [2023-12-27 13:15:17,213][20347] Updated weights for policy 0, policy_version 1097716 (0.0008) [2023-12-27 13:15:17,214][20381] Updated weights for policy 1, policy_version 1096735 (0.0008) [2023-12-27 13:15:17,273][20347] Updated weights for policy 0, policy_version 1097726 (0.0007) [2023-12-27 13:15:17,277][20381] Updated weights for policy 1, policy_version 1096745 (0.0009) [2023-12-27 13:15:17,324][20347] Updated weights for policy 0, policy_version 1097736 (0.0007) [2023-12-27 13:15:17,341][20381] Updated weights for policy 1, policy_version 1096755 (0.0008) [2023-12-27 13:15:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 561856512. Throughput: 0: 10168.7, 1: 10286.5. Samples: 561867828. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:17,363][19170] Avg episode reward: [(0, '863.370'), (1, '744.820')] [2023-12-27 13:15:17,922][20347] Updated weights for policy 0, policy_version 1097746 (0.0006) [2023-12-27 13:15:17,975][20381] Updated weights for policy 1, policy_version 1096765 (0.0008) [2023-12-27 13:15:17,976][20347] Updated weights for policy 0, policy_version 1097756 (0.0006) [2023-12-27 13:15:18,023][20381] Updated weights for policy 1, policy_version 1096775 (0.0010) [2023-12-27 13:15:18,029][20347] Updated weights for policy 0, policy_version 1097766 (0.0005) [2023-12-27 13:15:18,072][20381] Updated weights for policy 1, policy_version 1096785 (0.0010) [2023-12-27 13:15:18,686][20347] Updated weights for policy 0, policy_version 1097776 (0.0006) [2023-12-27 13:15:18,732][20347] Updated weights for policy 0, policy_version 1097786 (0.0007) [2023-12-27 13:15:18,767][20381] Updated weights for policy 1, policy_version 1096795 (0.0010) [2023-12-27 13:15:18,784][20347] Updated weights for policy 0, policy_version 1097796 (0.0006) [2023-12-27 13:15:18,825][20381] Updated weights for policy 1, policy_version 1096805 (0.0011) [2023-12-27 13:15:18,885][20381] Updated weights for policy 1, policy_version 1096815 (0.0011) [2023-12-27 13:15:19,529][20347] Updated weights for policy 0, policy_version 1097806 (0.0007) [2023-12-27 13:15:19,567][20381] Updated weights for policy 1, policy_version 1096825 (0.0010) [2023-12-27 13:15:19,584][20347] Updated weights for policy 0, policy_version 1097816 (0.0008) [2023-12-27 13:15:19,628][20381] Updated weights for policy 1, policy_version 1096835 (0.0006) [2023-12-27 13:15:19,638][20347] Updated weights for policy 0, policy_version 1097826 (0.0007) [2023-12-27 13:15:19,686][20381] Updated weights for policy 1, policy_version 1096845 (0.0007) [2023-12-27 13:15:19,747][20381] Updated weights for policy 1, policy_version 1096855 (0.0007) [2023-12-27 13:15:20,340][20347] Updated weights for policy 0, policy_version 1097836 (0.0007) [2023-12-27 13:15:20,397][20347] Updated weights for policy 0, policy_version 1097846 (0.0008) [2023-12-27 13:15:20,456][20347] Updated weights for policy 0, policy_version 1097856 (0.0007) [2023-12-27 13:15:20,466][20381] Updated weights for policy 1, policy_version 1096865 (0.0006) [2023-12-27 13:15:20,530][20381] Updated weights for policy 1, policy_version 1096875 (0.0007) [2023-12-27 13:15:20,599][20381] Updated weights for policy 1, policy_version 1096885 (0.0008) [2023-12-27 13:15:21,171][20347] Updated weights for policy 0, policy_version 1097866 (0.0007) [2023-12-27 13:15:21,227][20347] Updated weights for policy 0, policy_version 1097876 (0.0008) [2023-12-27 13:15:21,263][20381] Updated weights for policy 1, policy_version 1096895 (0.0007) [2023-12-27 13:15:21,292][20347] Updated weights for policy 0, policy_version 1097886 (0.0008) [2023-12-27 13:15:21,330][20381] Updated weights for policy 1, policy_version 1096905 (0.0007) [2023-12-27 13:15:21,358][20347] Updated weights for policy 0, policy_version 1097896 (0.0009) [2023-12-27 13:15:21,403][20381] Updated weights for policy 1, policy_version 1096915 (0.0007) [2023-12-27 13:15:22,071][20381] Updated weights for policy 1, policy_version 1096925 (0.0006) [2023-12-27 13:15:22,119][20347] Updated weights for policy 0, policy_version 1097906 (0.0008) [2023-12-27 13:15:22,126][20381] Updated weights for policy 1, policy_version 1096935 (0.0007) [2023-12-27 13:15:22,182][20381] Updated weights for policy 1, policy_version 1096945 (0.0006) [2023-12-27 13:15:22,183][20347] Updated weights for policy 0, policy_version 1097916 (0.0007) [2023-12-27 13:15:22,245][20347] Updated weights for policy 0, policy_version 1097926 (0.0007) [2023-12-27 13:15:22,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 561963008. Throughput: 0: 10201.2, 1: 10236.9. Samples: 561929120. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:22,363][19170] Avg episode reward: [(0, '863.870'), (1, '715.200')] [2023-12-27 13:15:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001096952_280846336.pth... [2023-12-27 13:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001097928_281116672.pth... [2023-12-27 13:15:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001096744_280813568.pth [2023-12-27 13:15:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001095736_280535040.pth [2023-12-27 13:15:22,891][20381] Updated weights for policy 1, policy_version 1096955 (0.0009) [2023-12-27 13:15:22,947][20381] Updated weights for policy 1, policy_version 1096965 (0.0008) [2023-12-27 13:15:22,984][20347] Updated weights for policy 0, policy_version 1097936 (0.0006) [2023-12-27 13:15:23,012][20381] Updated weights for policy 1, policy_version 1096975 (0.0007) [2023-12-27 13:15:23,043][20347] Updated weights for policy 0, policy_version 1097946 (0.0006) [2023-12-27 13:15:23,101][20347] Updated weights for policy 0, policy_version 1097956 (0.0007) [2023-12-27 13:15:23,701][20381] Updated weights for policy 1, policy_version 1096985 (0.0008) [2023-12-27 13:15:23,754][20381] Updated weights for policy 1, policy_version 1096995 (0.0005) [2023-12-27 13:15:23,798][20347] Updated weights for policy 0, policy_version 1097966 (0.0007) [2023-12-27 13:15:23,812][20381] Updated weights for policy 1, policy_version 1097005 (0.0009) [2023-12-27 13:15:23,850][20347] Updated weights for policy 0, policy_version 1097976 (0.0007) [2023-12-27 13:15:23,870][20381] Updated weights for policy 1, policy_version 1097015 (0.0010) [2023-12-27 13:15:23,907][20347] Updated weights for policy 0, policy_version 1097986 (0.0007) [2023-12-27 13:15:24,446][20381] Updated weights for policy 1, policy_version 1097025 (0.0006) [2023-12-27 13:15:24,496][20381] Updated weights for policy 1, policy_version 1097035 (0.0009) [2023-12-27 13:15:24,554][20381] Updated weights for policy 1, policy_version 1097045 (0.0007) [2023-12-27 13:15:24,595][20347] Updated weights for policy 0, policy_version 1097996 (0.0007) [2023-12-27 13:15:24,652][20347] Updated weights for policy 0, policy_version 1098006 (0.0010) [2023-12-27 13:15:24,714][20347] Updated weights for policy 0, policy_version 1098016 (0.0010) [2023-12-27 13:15:25,186][20381] Updated weights for policy 1, policy_version 1097055 (0.0011) [2023-12-27 13:15:25,246][20381] Updated weights for policy 1, policy_version 1097065 (0.0011) [2023-12-27 13:15:25,303][20381] Updated weights for policy 1, policy_version 1097075 (0.0011) [2023-12-27 13:15:25,391][20347] Updated weights for policy 0, policy_version 1098026 (0.0009) [2023-12-27 13:15:25,442][20347] Updated weights for policy 0, policy_version 1098036 (0.0010) [2023-12-27 13:15:25,491][20347] Updated weights for policy 0, policy_version 1098046 (0.0010) [2023-12-27 13:15:25,556][20347] Updated weights for policy 0, policy_version 1098056 (0.0010) [2023-12-27 13:15:25,916][20381] Updated weights for policy 1, policy_version 1097085 (0.0010) [2023-12-27 13:15:25,970][20381] Updated weights for policy 1, policy_version 1097095 (0.0011) [2023-12-27 13:15:26,026][20381] Updated weights for policy 1, policy_version 1097105 (0.0008) [2023-12-27 13:15:26,235][20347] Updated weights for policy 0, policy_version 1098066 (0.0006) [2023-12-27 13:15:26,293][20347] Updated weights for policy 0, policy_version 1098076 (0.0011) [2023-12-27 13:15:26,357][20347] Updated weights for policy 0, policy_version 1098086 (0.0010) [2023-12-27 13:15:26,636][20381] Updated weights for policy 1, policy_version 1097115 (0.0008) [2023-12-27 13:15:26,685][20381] Updated weights for policy 1, policy_version 1097125 (0.0005) [2023-12-27 13:15:26,743][20381] Updated weights for policy 1, policy_version 1097135 (0.0005) [2023-12-27 13:15:27,023][20347] Updated weights for policy 0, policy_version 1098096 (0.0010) [2023-12-27 13:15:27,091][20347] Updated weights for policy 0, policy_version 1098106 (0.0010) [2023-12-27 13:15:27,154][20347] Updated weights for policy 0, policy_version 1098116 (0.0007) [2023-12-27 13:15:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 562061312. Throughput: 0: 10212.2, 1: 10261.9. Samples: 562050288. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-12-27 13:15:27,363][19170] Avg episode reward: [(0, '902.640'), (1, '774.330')] [2023-12-27 13:15:27,442][20381] Updated weights for policy 1, policy_version 1097145 (0.0005) [2023-12-27 13:15:27,510][20381] Updated weights for policy 1, policy_version 1097155 (0.0006) [2023-12-27 13:15:27,573][20381] Updated weights for policy 1, policy_version 1097165 (0.0010) [2023-12-27 13:15:27,629][20381] Updated weights for policy 1, policy_version 1097175 (0.0011) [2023-12-27 13:15:27,815][20347] Updated weights for policy 0, policy_version 1098126 (0.0009) [2023-12-27 13:15:27,872][20347] Updated weights for policy 0, policy_version 1098136 (0.0010) [2023-12-27 13:15:27,921][20347] Updated weights for policy 0, policy_version 1098146 (0.0005) [2023-12-27 13:15:28,286][20381] Updated weights for policy 1, policy_version 1097185 (0.0008) [2023-12-27 13:15:28,352][20381] Updated weights for policy 1, policy_version 1097195 (0.0008) [2023-12-27 13:15:28,411][20381] Updated weights for policy 1, policy_version 1097205 (0.0007) [2023-12-27 13:15:28,563][20347] Updated weights for policy 0, policy_version 1098156 (0.0007) [2023-12-27 13:15:28,628][20347] Updated weights for policy 0, policy_version 1098166 (0.0010) [2023-12-27 13:15:28,689][20347] Updated weights for policy 0, policy_version 1098176 (0.0008) [2023-12-27 13:15:29,112][20381] Updated weights for policy 1, policy_version 1097215 (0.0008) [2023-12-27 13:15:29,167][20381] Updated weights for policy 1, policy_version 1097225 (0.0007) [2023-12-27 13:15:29,221][20381] Updated weights for policy 1, policy_version 1097235 (0.0008) [2023-12-27 13:15:29,446][20347] Updated weights for policy 0, policy_version 1098186 (0.0009) [2023-12-27 13:15:29,498][20347] Updated weights for policy 0, policy_version 1098196 (0.0010) [2023-12-27 13:15:29,554][20347] Updated weights for policy 0, policy_version 1098206 (0.0010) [2023-12-27 13:15:29,609][20347] Updated weights for policy 0, policy_version 1098216 (0.0010) [2023-12-27 13:15:29,929][20381] Updated weights for policy 1, policy_version 1097245 (0.0009) [2023-12-27 13:15:30,002][20381] Updated weights for policy 1, policy_version 1097255 (0.0010) [2023-12-27 13:15:30,055][20381] Updated weights for policy 1, policy_version 1097265 (0.0011) [2023-12-27 13:15:30,342][20347] Updated weights for policy 0, policy_version 1098226 (0.0009) [2023-12-27 13:15:30,406][20347] Updated weights for policy 0, policy_version 1098236 (0.0007) [2023-12-27 13:15:30,466][20347] Updated weights for policy 0, policy_version 1098246 (0.0007) [2023-12-27 13:15:30,714][20381] Updated weights for policy 1, policy_version 1097275 (0.0010) [2023-12-27 13:15:30,760][20381] Updated weights for policy 1, policy_version 1097285 (0.0010) [2023-12-27 13:15:30,815][20381] Updated weights for policy 1, policy_version 1097295 (0.0010) [2023-12-27 13:15:31,145][20347] Updated weights for policy 0, policy_version 1098256 (0.0008) [2023-12-27 13:15:31,207][20347] Updated weights for policy 0, policy_version 1098266 (0.0006) [2023-12-27 13:15:31,275][20347] Updated weights for policy 0, policy_version 1098276 (0.0007) [2023-12-27 13:15:31,576][20381] Updated weights for policy 1, policy_version 1097305 (0.0010) [2023-12-27 13:15:31,641][20381] Updated weights for policy 1, policy_version 1097315 (0.0006) [2023-12-27 13:15:31,703][20381] Updated weights for policy 1, policy_version 1097325 (0.0008) [2023-12-27 13:15:31,767][20381] Updated weights for policy 1, policy_version 1097335 (0.0009) [2023-12-27 13:15:32,003][20347] Updated weights for policy 0, policy_version 1098286 (0.0008) [2023-12-27 13:15:32,054][20347] Updated weights for policy 0, policy_version 1098296 (0.0008) [2023-12-27 13:15:32,109][20347] Updated weights for policy 0, policy_version 1098306 (0.0007) [2023-12-27 13:15:32,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 562159616. Throughput: 0: 10096.5, 1: 10255.8. Samples: 562171296. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:32,363][19170] Avg episode reward: [(0, '960.980'), (1, '764.100')] [2023-12-27 13:15:32,377][20381] Updated weights for policy 1, policy_version 1097345 (0.0008) [2023-12-27 13:15:32,435][20381] Updated weights for policy 1, policy_version 1097355 (0.0008) [2023-12-27 13:15:32,490][20381] Updated weights for policy 1, policy_version 1097365 (0.0006) [2023-12-27 13:15:32,812][20347] Updated weights for policy 0, policy_version 1098316 (0.0008) [2023-12-27 13:15:32,873][20347] Updated weights for policy 0, policy_version 1098326 (0.0006) [2023-12-27 13:15:32,940][20347] Updated weights for policy 0, policy_version 1098336 (0.0011) [2023-12-27 13:15:33,150][20381] Updated weights for policy 1, policy_version 1097375 (0.0005) [2023-12-27 13:15:33,207][20381] Updated weights for policy 1, policy_version 1097385 (0.0008) [2023-12-27 13:15:33,266][20381] Updated weights for policy 1, policy_version 1097395 (0.0007) [2023-12-27 13:15:33,578][20347] Updated weights for policy 0, policy_version 1098346 (0.0010) [2023-12-27 13:15:33,635][20347] Updated weights for policy 0, policy_version 1098356 (0.0007) [2023-12-27 13:15:33,692][20347] Updated weights for policy 0, policy_version 1098366 (0.0008) [2023-12-27 13:15:33,753][20347] Updated weights for policy 0, policy_version 1098376 (0.0008) [2023-12-27 13:15:33,898][20381] Updated weights for policy 1, policy_version 1097405 (0.0009) [2023-12-27 13:15:33,956][20381] Updated weights for policy 1, policy_version 1097415 (0.0009) [2023-12-27 13:15:34,009][20381] Updated weights for policy 1, policy_version 1097425 (0.0008) [2023-12-27 13:15:34,418][20347] Updated weights for policy 0, policy_version 1098386 (0.0007) [2023-12-27 13:15:34,473][20347] Updated weights for policy 0, policy_version 1098396 (0.0008) [2023-12-27 13:15:34,535][20347] Updated weights for policy 0, policy_version 1098406 (0.0007) [2023-12-27 13:15:34,727][20381] Updated weights for policy 1, policy_version 1097435 (0.0009) [2023-12-27 13:15:34,791][20381] Updated weights for policy 1, policy_version 1097445 (0.0011) [2023-12-27 13:15:34,845][20381] Updated weights for policy 1, policy_version 1097455 (0.0010) [2023-12-27 13:15:35,147][20347] Updated weights for policy 0, policy_version 1098416 (0.0007) [2023-12-27 13:15:35,205][20347] Updated weights for policy 0, policy_version 1098426 (0.0007) [2023-12-27 13:15:35,254][20347] Updated weights for policy 0, policy_version 1098436 (0.0007) [2023-12-27 13:15:35,531][20381] Updated weights for policy 1, policy_version 1097465 (0.0010) [2023-12-27 13:15:35,582][20381] Updated weights for policy 1, policy_version 1097475 (0.0008) [2023-12-27 13:15:35,630][20381] Updated weights for policy 1, policy_version 1097485 (0.0007) [2023-12-27 13:15:35,675][20381] Updated weights for policy 1, policy_version 1097495 (0.0007) [2023-12-27 13:15:35,965][20347] Updated weights for policy 0, policy_version 1098446 (0.0007) [2023-12-27 13:15:36,019][20347] Updated weights for policy 0, policy_version 1098456 (0.0007) [2023-12-27 13:15:36,075][20347] Updated weights for policy 0, policy_version 1098466 (0.0007) [2023-12-27 13:15:36,342][20381] Updated weights for policy 1, policy_version 1097505 (0.0007) [2023-12-27 13:15:36,401][20381] Updated weights for policy 1, policy_version 1097515 (0.0006) [2023-12-27 13:15:36,460][20381] Updated weights for policy 1, policy_version 1097525 (0.0008) [2023-12-27 13:15:36,793][20347] Updated weights for policy 0, policy_version 1098476 (0.0008) [2023-12-27 13:15:36,853][20347] Updated weights for policy 0, policy_version 1098486 (0.0008) [2023-12-27 13:15:36,911][20347] Updated weights for policy 0, policy_version 1098496 (0.0008) [2023-12-27 13:15:37,086][20381] Updated weights for policy 1, policy_version 1097535 (0.0008) [2023-12-27 13:15:37,144][20381] Updated weights for policy 1, policy_version 1097545 (0.0009) [2023-12-27 13:15:37,200][20381] Updated weights for policy 1, policy_version 1097555 (0.0009) [2023-12-27 13:15:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 562266112. Throughput: 0: 10128.2, 1: 10268.1. Samples: 562233396. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:37,363][19170] Avg episode reward: [(0, '931.930'), (1, '725.030')] [2023-12-27 13:15:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001097560_281001984.pth... [2023-12-27 13:15:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001098504_281264128.pth... [2023-12-27 13:15:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001096344_280690688.pth [2023-12-27 13:15:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001097352_280969216.pth [2023-12-27 13:15:37,528][20347] Updated weights for policy 0, policy_version 1098506 (0.0008) [2023-12-27 13:15:37,581][20347] Updated weights for policy 0, policy_version 1098516 (0.0008) [2023-12-27 13:15:37,639][20347] Updated weights for policy 0, policy_version 1098526 (0.0006) [2023-12-27 13:15:37,692][20347] Updated weights for policy 0, policy_version 1098536 (0.0008) [2023-12-27 13:15:37,856][20381] Updated weights for policy 1, policy_version 1097565 (0.0009) [2023-12-27 13:15:37,909][20381] Updated weights for policy 1, policy_version 1097575 (0.0007) [2023-12-27 13:15:37,954][20381] Updated weights for policy 1, policy_version 1097585 (0.0008) [2023-12-27 13:15:38,395][20347] Updated weights for policy 0, policy_version 1098546 (0.0008) [2023-12-27 13:15:38,459][20347] Updated weights for policy 0, policy_version 1098556 (0.0008) [2023-12-27 13:15:38,522][20347] Updated weights for policy 0, policy_version 1098566 (0.0007) [2023-12-27 13:15:38,631][20381] Updated weights for policy 1, policy_version 1097595 (0.0007) [2023-12-27 13:15:38,685][20381] Updated weights for policy 1, policy_version 1097605 (0.0008) [2023-12-27 13:15:38,743][20381] Updated weights for policy 1, policy_version 1097615 (0.0007) [2023-12-27 13:15:39,167][20347] Updated weights for policy 0, policy_version 1098576 (0.0006) [2023-12-27 13:15:39,219][20347] Updated weights for policy 0, policy_version 1098586 (0.0005) [2023-12-27 13:15:39,274][20347] Updated weights for policy 0, policy_version 1098596 (0.0006) [2023-12-27 13:15:39,475][20381] Updated weights for policy 1, policy_version 1097625 (0.0009) [2023-12-27 13:15:39,527][20381] Updated weights for policy 1, policy_version 1097635 (0.0009) [2023-12-27 13:15:39,582][20381] Updated weights for policy 1, policy_version 1097645 (0.0009) [2023-12-27 13:15:39,636][20381] Updated weights for policy 1, policy_version 1097655 (0.0007) [2023-12-27 13:15:40,017][20347] Updated weights for policy 0, policy_version 1098606 (0.0008) [2023-12-27 13:15:40,071][20347] Updated weights for policy 0, policy_version 1098616 (0.0009) [2023-12-27 13:15:40,143][20347] Updated weights for policy 0, policy_version 1098626 (0.0008) [2023-12-27 13:15:40,408][20381] Updated weights for policy 1, policy_version 1097665 (0.0007) [2023-12-27 13:15:40,462][20381] Updated weights for policy 1, policy_version 1097675 (0.0008) [2023-12-27 13:15:40,513][20381] Updated weights for policy 1, policy_version 1097685 (0.0008) [2023-12-27 13:15:40,845][20347] Updated weights for policy 0, policy_version 1098636 (0.0008) [2023-12-27 13:15:40,890][20347] Updated weights for policy 0, policy_version 1098646 (0.0007) [2023-12-27 13:15:40,937][20347] Updated weights for policy 0, policy_version 1098656 (0.0007) [2023-12-27 13:15:41,287][20381] Updated weights for policy 1, policy_version 1097695 (0.0009) [2023-12-27 13:15:41,356][20381] Updated weights for policy 1, policy_version 1097705 (0.0008) [2023-12-27 13:15:41,429][20381] Updated weights for policy 1, policy_version 1097715 (0.0008) [2023-12-27 13:15:41,728][20347] Updated weights for policy 0, policy_version 1098666 (0.0008) [2023-12-27 13:15:41,793][20347] Updated weights for policy 0, policy_version 1098676 (0.0008) [2023-12-27 13:15:41,855][20347] Updated weights for policy 0, policy_version 1098686 (0.0006) [2023-12-27 13:15:41,918][20347] Updated weights for policy 0, policy_version 1098696 (0.0005) [2023-12-27 13:15:42,133][20381] Updated weights for policy 1, policy_version 1097725 (0.0008) [2023-12-27 13:15:42,189][20381] Updated weights for policy 1, policy_version 1097735 (0.0011) [2023-12-27 13:15:42,251][20381] Updated weights for policy 1, policy_version 1097745 (0.0009) [2023-12-27 13:15:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 562364416. Throughput: 0: 10094.4, 1: 10275.8. Samples: 562354032. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:42,363][19170] Avg episode reward: [(0, '902.650'), (1, '705.490')] [2023-12-27 13:15:42,599][20347] Updated weights for policy 0, policy_version 1098706 (0.0007) [2023-12-27 13:15:42,655][20347] Updated weights for policy 0, policy_version 1098716 (0.0007) [2023-12-27 13:15:42,718][20347] Updated weights for policy 0, policy_version 1098726 (0.0007) [2023-12-27 13:15:42,946][20381] Updated weights for policy 1, policy_version 1097755 (0.0010) [2023-12-27 13:15:43,012][20381] Updated weights for policy 1, policy_version 1097765 (0.0008) [2023-12-27 13:15:43,073][20381] Updated weights for policy 1, policy_version 1097775 (0.0008) [2023-12-27 13:15:43,395][20347] Updated weights for policy 0, policy_version 1098736 (0.0006) [2023-12-27 13:15:43,448][20347] Updated weights for policy 0, policy_version 1098746 (0.0005) [2023-12-27 13:15:43,497][20347] Updated weights for policy 0, policy_version 1098756 (0.0006) [2023-12-27 13:15:43,738][20381] Updated weights for policy 1, policy_version 1097785 (0.0008) [2023-12-27 13:15:43,791][20381] Updated weights for policy 1, policy_version 1097795 (0.0005) [2023-12-27 13:15:43,851][20381] Updated weights for policy 1, policy_version 1097805 (0.0005) [2023-12-27 13:15:43,908][20381] Updated weights for policy 1, policy_version 1097815 (0.0005) [2023-12-27 13:15:44,101][20347] Updated weights for policy 0, policy_version 1098766 (0.0008) [2023-12-27 13:15:44,159][20347] Updated weights for policy 0, policy_version 1098776 (0.0006) [2023-12-27 13:15:44,217][20347] Updated weights for policy 0, policy_version 1098786 (0.0006) [2023-12-27 13:15:44,540][20381] Updated weights for policy 1, policy_version 1097825 (0.0005) [2023-12-27 13:15:44,597][20381] Updated weights for policy 1, policy_version 1097835 (0.0005) [2023-12-27 13:15:44,648][20381] Updated weights for policy 1, policy_version 1097845 (0.0005) [2023-12-27 13:15:44,768][20347] Updated weights for policy 0, policy_version 1098796 (0.0007) [2023-12-27 13:15:44,831][20347] Updated weights for policy 0, policy_version 1098806 (0.0009) [2023-12-27 13:15:44,884][20347] Updated weights for policy 0, policy_version 1098816 (0.0011) [2023-12-27 13:15:45,288][20381] Updated weights for policy 1, policy_version 1097855 (0.0006) [2023-12-27 13:15:45,350][20381] Updated weights for policy 1, policy_version 1097865 (0.0006) [2023-12-27 13:15:45,405][20381] Updated weights for policy 1, policy_version 1097875 (0.0008) [2023-12-27 13:15:45,587][20347] Updated weights for policy 0, policy_version 1098826 (0.0010) [2023-12-27 13:15:45,636][20347] Updated weights for policy 0, policy_version 1098836 (0.0010) [2023-12-27 13:15:45,681][20347] Updated weights for policy 0, policy_version 1098846 (0.0010) [2023-12-27 13:15:45,729][20347] Updated weights for policy 0, policy_version 1098856 (0.0009) [2023-12-27 13:15:46,046][20381] Updated weights for policy 1, policy_version 1097885 (0.0008) [2023-12-27 13:15:46,094][20381] Updated weights for policy 1, policy_version 1097895 (0.0010) [2023-12-27 13:15:46,141][20381] Updated weights for policy 1, policy_version 1097905 (0.0005) [2023-12-27 13:15:46,440][20347] Updated weights for policy 0, policy_version 1098866 (0.0009) [2023-12-27 13:15:46,489][20347] Updated weights for policy 0, policy_version 1098876 (0.0010) [2023-12-27 13:15:46,534][20347] Updated weights for policy 0, policy_version 1098886 (0.0006) [2023-12-27 13:15:46,754][20381] Updated weights for policy 1, policy_version 1097915 (0.0007) [2023-12-27 13:15:46,806][20381] Updated weights for policy 1, policy_version 1097925 (0.0010) [2023-12-27 13:15:46,863][20381] Updated weights for policy 1, policy_version 1097935 (0.0009) [2023-12-27 13:15:47,158][20347] Updated weights for policy 0, policy_version 1098896 (0.0005) [2023-12-27 13:15:47,207][20347] Updated weights for policy 0, policy_version 1098906 (0.0005) [2023-12-27 13:15:47,260][20347] Updated weights for policy 0, policy_version 1098916 (0.0007) [2023-12-27 13:15:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 562470912. Throughput: 0: 10161.9, 1: 10298.7. Samples: 562479880. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:47,363][19170] Avg episode reward: [(0, '931.740'), (1, '646.730')] [2023-12-27 13:15:47,475][20381] Updated weights for policy 1, policy_version 1097945 (0.0008) [2023-12-27 13:15:47,535][20381] Updated weights for policy 1, policy_version 1097955 (0.0007) [2023-12-27 13:15:47,593][20381] Updated weights for policy 1, policy_version 1097965 (0.0007) [2023-12-27 13:15:47,643][20381] Updated weights for policy 1, policy_version 1097975 (0.0007) [2023-12-27 13:15:47,929][20347] Updated weights for policy 0, policy_version 1098926 (0.0010) [2023-12-27 13:15:47,975][20347] Updated weights for policy 0, policy_version 1098936 (0.0008) [2023-12-27 13:15:48,021][20347] Updated weights for policy 0, policy_version 1098946 (0.0009) [2023-12-27 13:15:48,364][20381] Updated weights for policy 1, policy_version 1097985 (0.0008) [2023-12-27 13:15:48,416][20381] Updated weights for policy 1, policy_version 1097995 (0.0009) [2023-12-27 13:15:48,469][20381] Updated weights for policy 1, policy_version 1098005 (0.0008) [2023-12-27 13:15:48,617][20347] Updated weights for policy 0, policy_version 1098956 (0.0009) [2023-12-27 13:15:48,675][20347] Updated weights for policy 0, policy_version 1098966 (0.0006) [2023-12-27 13:15:48,738][20347] Updated weights for policy 0, policy_version 1098976 (0.0008) [2023-12-27 13:15:49,245][20381] Updated weights for policy 1, policy_version 1098015 (0.0009) [2023-12-27 13:15:49,308][20381] Updated weights for policy 1, policy_version 1098025 (0.0008) [2023-12-27 13:15:49,349][20347] Updated weights for policy 0, policy_version 1098986 (0.0008) [2023-12-27 13:15:49,376][20381] Updated weights for policy 1, policy_version 1098035 (0.0010) [2023-12-27 13:15:49,410][20347] Updated weights for policy 0, policy_version 1098996 (0.0009) [2023-12-27 13:15:49,458][20347] Updated weights for policy 0, policy_version 1099006 (0.0007) [2023-12-27 13:15:49,503][20347] Updated weights for policy 0, policy_version 1099016 (0.0007) [2023-12-27 13:15:50,092][20381] Updated weights for policy 1, policy_version 1098045 (0.0009) [2023-12-27 13:15:50,153][20381] Updated weights for policy 1, policy_version 1098055 (0.0008) [2023-12-27 13:15:50,211][20381] Updated weights for policy 1, policy_version 1098065 (0.0006) [2023-12-27 13:15:50,231][20347] Updated weights for policy 0, policy_version 1099026 (0.0007) [2023-12-27 13:15:50,294][20347] Updated weights for policy 0, policy_version 1099036 (0.0007) [2023-12-27 13:15:50,353][20347] Updated weights for policy 0, policy_version 1099046 (0.0008) [2023-12-27 13:15:50,862][20381] Updated weights for policy 1, policy_version 1098075 (0.0007) [2023-12-27 13:15:50,925][20381] Updated weights for policy 1, policy_version 1098085 (0.0008) [2023-12-27 13:15:50,984][20381] Updated weights for policy 1, policy_version 1098095 (0.0008) [2023-12-27 13:15:51,028][20347] Updated weights for policy 0, policy_version 1099056 (0.0008) [2023-12-27 13:15:51,087][20347] Updated weights for policy 0, policy_version 1099066 (0.0007) [2023-12-27 13:15:51,143][20347] Updated weights for policy 0, policy_version 1099076 (0.0008) [2023-12-27 13:15:51,728][20381] Updated weights for policy 1, policy_version 1098105 (0.0008) [2023-12-27 13:15:51,787][20381] Updated weights for policy 1, policy_version 1098115 (0.0009) [2023-12-27 13:15:51,836][20381] Updated weights for policy 1, policy_version 1098125 (0.0007) [2023-12-27 13:15:51,890][20381] Updated weights for policy 1, policy_version 1098135 (0.0006) [2023-12-27 13:15:51,896][20347] Updated weights for policy 0, policy_version 1099086 (0.0007) [2023-12-27 13:15:51,946][20347] Updated weights for policy 0, policy_version 1099096 (0.0008) [2023-12-27 13:15:51,993][20347] Updated weights for policy 0, policy_version 1099106 (0.0007) [2023-12-27 13:15:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 562569216. Throughput: 0: 10192.4, 1: 10265.4. Samples: 562541020. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:52,363][19170] Avg episode reward: [(0, '921.920'), (1, '587.950')] [2023-12-27 13:15:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001098136_281149440.pth... [2023-12-27 13:15:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001099112_281419776.pth... [2023-12-27 13:15:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001096952_280846336.pth [2023-12-27 13:15:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001097928_281116672.pth [2023-12-27 13:15:52,611][20381] Updated weights for policy 1, policy_version 1098145 (0.0007) [2023-12-27 13:15:52,668][20381] Updated weights for policy 1, policy_version 1098155 (0.0008) [2023-12-27 13:15:52,725][20381] Updated weights for policy 1, policy_version 1098165 (0.0007) [2023-12-27 13:15:52,753][20347] Updated weights for policy 0, policy_version 1099116 (0.0008) [2023-12-27 13:15:52,805][20347] Updated weights for policy 0, policy_version 1099126 (0.0009) [2023-12-27 13:15:52,855][20347] Updated weights for policy 0, policy_version 1099136 (0.0009) [2023-12-27 13:15:53,366][20381] Updated weights for policy 1, policy_version 1098175 (0.0007) [2023-12-27 13:15:53,424][20381] Updated weights for policy 1, policy_version 1098185 (0.0007) [2023-12-27 13:15:53,479][20381] Updated weights for policy 1, policy_version 1098195 (0.0008) [2023-12-27 13:15:53,520][20347] Updated weights for policy 0, policy_version 1099146 (0.0009) [2023-12-27 13:15:53,574][20347] Updated weights for policy 0, policy_version 1099156 (0.0007) [2023-12-27 13:15:53,636][20347] Updated weights for policy 0, policy_version 1099166 (0.0007) [2023-12-27 13:15:53,692][20347] Updated weights for policy 0, policy_version 1099176 (0.0008) [2023-12-27 13:15:54,187][20381] Updated weights for policy 1, policy_version 1098205 (0.0009) [2023-12-27 13:15:54,252][20381] Updated weights for policy 1, policy_version 1098215 (0.0011) [2023-12-27 13:15:54,308][20381] Updated weights for policy 1, policy_version 1098225 (0.0010) [2023-12-27 13:15:54,353][20347] Updated weights for policy 0, policy_version 1099186 (0.0008) [2023-12-27 13:15:54,410][20347] Updated weights for policy 0, policy_version 1099196 (0.0006) [2023-12-27 13:15:54,470][20347] Updated weights for policy 0, policy_version 1099206 (0.0007) [2023-12-27 13:15:54,986][20381] Updated weights for policy 1, policy_version 1098235 (0.0009) [2023-12-27 13:15:55,036][20381] Updated weights for policy 1, policy_version 1098245 (0.0007) [2023-12-27 13:15:55,089][20381] Updated weights for policy 1, policy_version 1098255 (0.0006) [2023-12-27 13:15:55,146][20347] Updated weights for policy 0, policy_version 1099216 (0.0006) [2023-12-27 13:15:55,197][20347] Updated weights for policy 0, policy_version 1099226 (0.0008) [2023-12-27 13:15:55,251][20347] Updated weights for policy 0, policy_version 1099236 (0.0007) [2023-12-27 13:15:55,673][20381] Updated weights for policy 1, policy_version 1098265 (0.0007) [2023-12-27 13:15:55,732][20381] Updated weights for policy 1, policy_version 1098275 (0.0005) [2023-12-27 13:15:55,789][20381] Updated weights for policy 1, policy_version 1098285 (0.0007) [2023-12-27 13:15:55,838][20381] Updated weights for policy 1, policy_version 1098295 (0.0006) [2023-12-27 13:15:55,952][20347] Updated weights for policy 0, policy_version 1099246 (0.0007) [2023-12-27 13:15:56,003][20347] Updated weights for policy 0, policy_version 1099256 (0.0007) [2023-12-27 13:15:56,050][20347] Updated weights for policy 0, policy_version 1099266 (0.0008) [2023-12-27 13:15:56,491][20381] Updated weights for policy 1, policy_version 1098305 (0.0007) [2023-12-27 13:15:56,539][20381] Updated weights for policy 1, policy_version 1098315 (0.0007) [2023-12-27 13:15:56,595][20381] Updated weights for policy 1, policy_version 1098325 (0.0007) [2023-12-27 13:15:56,737][20347] Updated weights for policy 0, policy_version 1099276 (0.0007) [2023-12-27 13:15:56,788][20347] Updated weights for policy 0, policy_version 1099286 (0.0007) [2023-12-27 13:15:56,836][20347] Updated weights for policy 0, policy_version 1099296 (0.0007) [2023-12-27 13:15:57,252][20381] Updated weights for policy 1, policy_version 1098335 (0.0006) [2023-12-27 13:15:57,314][20381] Updated weights for policy 1, policy_version 1098345 (0.0008) [2023-12-27 13:15:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 562667520. Throughput: 0: 10179.0, 1: 10261.7. Samples: 562662568. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:15:57,363][19170] Avg episode reward: [(0, '843.660'), (1, '636.970')] [2023-12-27 13:15:57,376][20381] Updated weights for policy 1, policy_version 1098355 (0.0007) [2023-12-27 13:15:57,530][20347] Updated weights for policy 0, policy_version 1099306 (0.0008) [2023-12-27 13:15:57,588][20347] Updated weights for policy 0, policy_version 1099316 (0.0010) [2023-12-27 13:15:57,643][20347] Updated weights for policy 0, policy_version 1099326 (0.0010) [2023-12-27 13:15:57,691][20347] Updated weights for policy 0, policy_version 1099336 (0.0010) [2023-12-27 13:15:57,994][20381] Updated weights for policy 1, policy_version 1098365 (0.0006) [2023-12-27 13:15:58,052][20381] Updated weights for policy 1, policy_version 1098375 (0.0007) [2023-12-27 13:15:58,108][20381] Updated weights for policy 1, policy_version 1098385 (0.0007) [2023-12-27 13:15:58,400][20347] Updated weights for policy 0, policy_version 1099346 (0.0010) [2023-12-27 13:15:58,466][20347] Updated weights for policy 0, policy_version 1099356 (0.0010) [2023-12-27 13:15:58,531][20347] Updated weights for policy 0, policy_version 1099366 (0.0011) [2023-12-27 13:15:58,866][20381] Updated weights for policy 1, policy_version 1098395 (0.0008) [2023-12-27 13:15:58,924][20381] Updated weights for policy 1, policy_version 1098405 (0.0006) [2023-12-27 13:15:58,988][20381] Updated weights for policy 1, policy_version 1098415 (0.0010) [2023-12-27 13:15:59,449][20347] Updated weights for policy 0, policy_version 1099376 (0.0008) [2023-12-27 13:15:59,509][20347] Updated weights for policy 0, policy_version 1099386 (0.0007) [2023-12-27 13:15:59,567][20347] Updated weights for policy 0, policy_version 1099396 (0.0008) [2023-12-27 13:15:59,810][20381] Updated weights for policy 1, policy_version 1098425 (0.0009) [2023-12-27 13:15:59,876][20381] Updated weights for policy 1, policy_version 1098435 (0.0010) [2023-12-27 13:15:59,943][20381] Updated weights for policy 1, policy_version 1098445 (0.0010) [2023-12-27 13:16:00,001][20381] Updated weights for policy 1, policy_version 1098455 (0.0006) [2023-12-27 13:16:00,310][20347] Updated weights for policy 0, policy_version 1099406 (0.0007) [2023-12-27 13:16:00,366][20347] Updated weights for policy 0, policy_version 1099416 (0.0008) [2023-12-27 13:16:00,417][20347] Updated weights for policy 0, policy_version 1099426 (0.0007) [2023-12-27 13:16:00,605][20381] Updated weights for policy 1, policy_version 1098465 (0.0007) [2023-12-27 13:16:00,654][20381] Updated weights for policy 1, policy_version 1098475 (0.0008) [2023-12-27 13:16:00,703][20381] Updated weights for policy 1, policy_version 1098485 (0.0008) [2023-12-27 13:16:01,071][20347] Updated weights for policy 0, policy_version 1099436 (0.0006) [2023-12-27 13:16:01,123][20347] Updated weights for policy 0, policy_version 1099446 (0.0008) [2023-12-27 13:16:01,196][20347] Updated weights for policy 0, policy_version 1099456 (0.0008) [2023-12-27 13:16:01,346][20381] Updated weights for policy 1, policy_version 1098495 (0.0008) [2023-12-27 13:16:01,419][20381] Updated weights for policy 1, policy_version 1098505 (0.0008) [2023-12-27 13:16:01,465][20381] Updated weights for policy 1, policy_version 1098515 (0.0008) [2023-12-27 13:16:01,930][20347] Updated weights for policy 0, policy_version 1099466 (0.0008) [2023-12-27 13:16:01,986][20347] Updated weights for policy 0, policy_version 1099476 (0.0007) [2023-12-27 13:16:01,998][20291] KL-divergence is very high: 102.0732 [2023-12-27 13:16:02,029][20291] KL-divergence is very high: 326.2749 [2023-12-27 13:16:02,035][20291] KL-divergence is very high: 256.7610 [2023-12-27 13:16:02,040][20291] KL-divergence is very high: 217.3262 [2023-12-27 13:16:02,045][20347] Updated weights for policy 0, policy_version 1099486 (0.0008) [2023-12-27 13:16:02,045][20291] KL-divergence is very high: 132.2411 [2023-12-27 13:16:02,097][20347] Updated weights for policy 0, policy_version 1099496 (0.0008) [2023-12-27 13:16:02,165][20381] Updated weights for policy 1, policy_version 1098525 (0.0008) [2023-12-27 13:16:02,229][20381] Updated weights for policy 1, policy_version 1098535 (0.0008) [2023-12-27 13:16:02,289][20381] Updated weights for policy 1, policy_version 1098545 (0.0008) [2023-12-27 13:16:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 562774016. Throughput: 0: 10067.6, 1: 10200.5. Samples: 562779896. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:02,363][19170] Avg episode reward: [(0, '804.770'), (1, '754.720')] [2023-12-27 13:16:02,800][20347] Updated weights for policy 0, policy_version 1099506 (0.0008) [2023-12-27 13:16:02,859][20347] Updated weights for policy 0, policy_version 1099516 (0.0008) [2023-12-27 13:16:02,914][20347] Updated weights for policy 0, policy_version 1099526 (0.0010) [2023-12-27 13:16:02,969][20381] Updated weights for policy 1, policy_version 1098555 (0.0007) [2023-12-27 13:16:03,020][20381] Updated weights for policy 1, policy_version 1098565 (0.0006) [2023-12-27 13:16:03,071][20381] Updated weights for policy 1, policy_version 1098575 (0.0007) [2023-12-27 13:16:03,558][20347] Updated weights for policy 0, policy_version 1099536 (0.0010) [2023-12-27 13:16:03,612][20347] Updated weights for policy 0, policy_version 1099546 (0.0010) [2023-12-27 13:16:03,661][20347] Updated weights for policy 0, policy_version 1099556 (0.0010) [2023-12-27 13:16:03,694][20381] Updated weights for policy 1, policy_version 1098585 (0.0008) [2023-12-27 13:16:03,748][20381] Updated weights for policy 1, policy_version 1098595 (0.0010) [2023-12-27 13:16:03,799][20381] Updated weights for policy 1, policy_version 1098605 (0.0010) [2023-12-27 13:16:03,863][20381] Updated weights for policy 1, policy_version 1098615 (0.0009) [2023-12-27 13:16:04,327][20291] KL-divergence is very high: 193.6385 [2023-12-27 13:16:04,334][20347] Updated weights for policy 0, policy_version 1099566 (0.0008) [2023-12-27 13:16:04,338][20291] KL-divergence is very high: 272.9715 [2023-12-27 13:16:04,345][20291] KL-divergence is very high: 187.0775 [2023-12-27 13:16:04,373][20291] KL-divergence is very high: 106.6581 [2023-12-27 13:16:04,389][20347] Updated weights for policy 0, policy_version 1099576 (0.0010) [2023-12-27 13:16:04,422][20291] KL-divergence is very high: 121.0598 [2023-12-27 13:16:04,429][20291] KL-divergence is very high: 188.9824 [2023-12-27 13:16:04,455][20347] Updated weights for policy 0, policy_version 1099586 (0.0010) [2023-12-27 13:16:04,471][20291] KL-divergence is very high: 256.6711 [2023-12-27 13:16:04,477][20291] KL-divergence is very high: 285.7825 [2023-12-27 13:16:04,483][20291] KL-divergence is very high: 199.0722 [2023-12-27 13:16:04,576][20381] Updated weights for policy 1, policy_version 1098625 (0.0007) [2023-12-27 13:16:04,621][20381] Updated weights for policy 1, policy_version 1098635 (0.0006) [2023-12-27 13:16:04,669][20381] Updated weights for policy 1, policy_version 1098645 (0.0007) [2023-12-27 13:16:05,115][20291] KL-divergence is very high: 367.4190 [2023-12-27 13:16:05,143][20347] Updated weights for policy 0, policy_version 1099596 (0.0010) [2023-12-27 13:16:05,167][20291] KL-divergence is very high: 268.2457 [2023-12-27 13:16:05,201][20347] Updated weights for policy 0, policy_version 1099606 (0.0010) [2023-12-27 13:16:05,218][20291] KL-divergence is very high: 202.3781 [2023-12-27 13:16:05,255][20347] Updated weights for policy 0, policy_version 1099616 (0.0005) [2023-12-27 13:16:05,256][20291] KL-divergence is very high: 234.1369 [2023-12-27 13:16:05,293][20381] Updated weights for policy 1, policy_version 1098655 (0.0007) [2023-12-27 13:16:05,344][20381] Updated weights for policy 1, policy_version 1098665 (0.0005) [2023-12-27 13:16:05,403][20381] Updated weights for policy 1, policy_version 1098675 (0.0007) [2023-12-27 13:16:05,878][20347] Updated weights for policy 0, policy_version 1099626 (0.0009) [2023-12-27 13:16:05,927][20347] Updated weights for policy 0, policy_version 1099636 (0.0010) [2023-12-27 13:16:05,977][20347] Updated weights for policy 0, policy_version 1099646 (0.0011) [2023-12-27 13:16:05,980][20381] Updated weights for policy 1, policy_version 1098685 (0.0008) [2023-12-27 13:16:06,030][20347] Updated weights for policy 0, policy_version 1099656 (0.0009) [2023-12-27 13:16:06,031][20381] Updated weights for policy 1, policy_version 1098695 (0.0005) [2023-12-27 13:16:06,082][20381] Updated weights for policy 1, policy_version 1098705 (0.0007) [2023-12-27 13:16:06,729][20347] Updated weights for policy 0, policy_version 1099666 (0.0005) [2023-12-27 13:16:06,774][20381] Updated weights for policy 1, policy_version 1098715 (0.0008) [2023-12-27 13:16:06,781][20347] Updated weights for policy 0, policy_version 1099676 (0.0005) [2023-12-27 13:16:06,822][20291] KL-divergence is very high: 158.4719 [2023-12-27 13:16:06,828][20291] KL-divergence is very high: 195.1113 [2023-12-27 13:16:06,830][20381] Updated weights for policy 1, policy_version 1098725 (0.0007) [2023-12-27 13:16:06,839][20347] Updated weights for policy 0, policy_version 1099686 (0.0006) [2023-12-27 13:16:06,885][20381] Updated weights for policy 1, policy_version 1098735 (0.0005) [2023-12-27 13:16:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 562872320. Throughput: 0: 10061.7, 1: 10230.3. Samples: 562842260. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:07,364][19170] Avg episode reward: [(0, '766.180'), (1, '656.670')] [2023-12-27 13:16:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001098744_281305088.pth... [2023-12-27 13:16:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001099688_281567232.pth... [2023-12-27 13:16:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001098504_281264128.pth [2023-12-27 13:16:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001097560_281001984.pth [2023-12-27 13:16:07,436][20381] Updated weights for policy 1, policy_version 1098745 (0.0006) [2023-12-27 13:16:07,485][20381] Updated weights for policy 1, policy_version 1098755 (0.0007) [2023-12-27 13:16:07,486][20347] Updated weights for policy 0, policy_version 1099696 (0.0006) [2023-12-27 13:16:07,531][20381] Updated weights for policy 1, policy_version 1098765 (0.0007) [2023-12-27 13:16:07,541][20347] Updated weights for policy 0, policy_version 1099706 (0.0006) [2023-12-27 13:16:07,580][20381] Updated weights for policy 1, policy_version 1098775 (0.0010) [2023-12-27 13:16:07,598][20347] Updated weights for policy 0, policy_version 1099716 (0.0005) [2023-12-27 13:16:08,185][20347] Updated weights for policy 0, policy_version 1099726 (0.0008) [2023-12-27 13:16:08,224][20381] Updated weights for policy 1, policy_version 1098785 (0.0010) [2023-12-27 13:16:08,236][20347] Updated weights for policy 0, policy_version 1099736 (0.0010) [2023-12-27 13:16:08,278][20381] Updated weights for policy 1, policy_version 1098795 (0.0010) [2023-12-27 13:16:08,293][20347] Updated weights for policy 0, policy_version 1099746 (0.0010) [2023-12-27 13:16:08,339][20381] Updated weights for policy 1, policy_version 1098805 (0.0011) [2023-12-27 13:16:08,935][20381] Updated weights for policy 1, policy_version 1098815 (0.0011) [2023-12-27 13:16:08,987][20381] Updated weights for policy 1, policy_version 1098825 (0.0010) [2023-12-27 13:16:08,988][20347] Updated weights for policy 0, policy_version 1099756 (0.0010) [2023-12-27 13:16:09,039][20381] Updated weights for policy 1, policy_version 1098835 (0.0011) [2023-12-27 13:16:09,043][20347] Updated weights for policy 0, policy_version 1099766 (0.0010) [2023-12-27 13:16:09,097][20347] Updated weights for policy 0, policy_version 1099776 (0.0009) [2023-12-27 13:16:09,770][20347] Updated weights for policy 0, policy_version 1099786 (0.0010) [2023-12-27 13:16:09,810][20381] Updated weights for policy 1, policy_version 1098845 (0.0008) [2023-12-27 13:16:09,837][20347] Updated weights for policy 0, policy_version 1099796 (0.0011) [2023-12-27 13:16:09,879][20381] Updated weights for policy 1, policy_version 1098855 (0.0008) [2023-12-27 13:16:09,898][20347] Updated weights for policy 0, policy_version 1099806 (0.0009) [2023-12-27 13:16:09,946][20381] Updated weights for policy 1, policy_version 1098865 (0.0007) [2023-12-27 13:16:09,960][20347] Updated weights for policy 0, policy_version 1099816 (0.0008) [2023-12-27 13:16:10,567][20381] Updated weights for policy 1, policy_version 1098875 (0.0007) [2023-12-27 13:16:10,626][20381] Updated weights for policy 1, policy_version 1098885 (0.0008) [2023-12-27 13:16:10,644][20291] KL-divergence is very high: 208.9596 [2023-12-27 13:16:10,662][20291] KL-divergence is very high: 271.5352 [2023-12-27 13:16:10,682][20381] Updated weights for policy 1, policy_version 1098895 (0.0006) [2023-12-27 13:16:10,695][20347] Updated weights for policy 0, policy_version 1099826 (0.0010) [2023-12-27 13:16:10,708][20291] KL-divergence is very high: 112.9801 [2023-12-27 13:16:10,724][20291] KL-divergence is very high: 138.8282 [2023-12-27 13:16:10,753][20347] Updated weights for policy 0, policy_version 1099836 (0.0010) [2023-12-27 13:16:10,779][20291] KL-divergence is very high: 181.3162 [2023-12-27 13:16:10,795][20291] KL-divergence is very high: 160.9884 [2023-12-27 13:16:10,807][20347] Updated weights for policy 0, policy_version 1099846 (0.0011) [2023-12-27 13:16:11,490][20381] Updated weights for policy 1, policy_version 1098905 (0.0006) [2023-12-27 13:16:11,517][20291] KL-divergence is very high: 101.1967 [2023-12-27 13:16:11,552][20381] Updated weights for policy 1, policy_version 1098915 (0.0008) [2023-12-27 13:16:11,557][20347] Updated weights for policy 0, policy_version 1099856 (0.0007) [2023-12-27 13:16:11,612][20381] Updated weights for policy 1, policy_version 1098925 (0.0007) [2023-12-27 13:16:11,617][20347] Updated weights for policy 0, policy_version 1099866 (0.0007) [2023-12-27 13:16:11,679][20381] Updated weights for policy 1, policy_version 1098935 (0.0009) [2023-12-27 13:16:11,685][20347] Updated weights for policy 0, policy_version 1099876 (0.0007) [2023-12-27 13:16:12,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 562970624. Throughput: 0: 10122.5, 1: 10263.3. Samples: 562967656. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:12,363][19170] Avg episode reward: [(0, '768.420'), (1, '647.070')] [2023-12-27 13:16:12,381][20381] Updated weights for policy 1, policy_version 1098945 (0.0009) [2023-12-27 13:16:12,411][20347] Updated weights for policy 0, policy_version 1099886 (0.0009) [2023-12-27 13:16:12,448][20381] Updated weights for policy 1, policy_version 1098955 (0.0008) [2023-12-27 13:16:12,475][20347] Updated weights for policy 0, policy_version 1099896 (0.0006) [2023-12-27 13:16:12,519][20381] Updated weights for policy 1, policy_version 1098965 (0.0006) [2023-12-27 13:16:12,538][20347] Updated weights for policy 0, policy_version 1099906 (0.0008) [2023-12-27 13:16:13,141][20381] Updated weights for policy 1, policy_version 1098975 (0.0006) [2023-12-27 13:16:13,191][20381] Updated weights for policy 1, policy_version 1098985 (0.0005) [2023-12-27 13:16:13,250][20381] Updated weights for policy 1, policy_version 1098995 (0.0007) [2023-12-27 13:16:13,258][20347] Updated weights for policy 0, policy_version 1099916 (0.0009) [2023-12-27 13:16:13,314][20347] Updated weights for policy 0, policy_version 1099926 (0.0007) [2023-12-27 13:16:13,373][20347] Updated weights for policy 0, policy_version 1099936 (0.0008) [2023-12-27 13:16:13,833][20381] Updated weights for policy 1, policy_version 1099005 (0.0008) [2023-12-27 13:16:13,884][20381] Updated weights for policy 1, policy_version 1099015 (0.0008) [2023-12-27 13:16:13,936][20381] Updated weights for policy 1, policy_version 1099025 (0.0007) [2023-12-27 13:16:14,067][20347] Updated weights for policy 0, policy_version 1099946 (0.0008) [2023-12-27 13:16:14,115][20347] Updated weights for policy 0, policy_version 1099956 (0.0008) [2023-12-27 13:16:14,174][20347] Updated weights for policy 0, policy_version 1099966 (0.0008) [2023-12-27 13:16:14,221][20347] Updated weights for policy 0, policy_version 1099976 (0.0007) [2023-12-27 13:16:14,576][20381] Updated weights for policy 1, policy_version 1099035 (0.0008) [2023-12-27 13:16:14,622][20381] Updated weights for policy 1, policy_version 1099045 (0.0008) [2023-12-27 13:16:14,671][20381] Updated weights for policy 1, policy_version 1099055 (0.0007) [2023-12-27 13:16:14,951][20347] Updated weights for policy 0, policy_version 1099986 (0.0009) [2023-12-27 13:16:15,015][20347] Updated weights for policy 0, policy_version 1099996 (0.0008) [2023-12-27 13:16:15,072][20347] Updated weights for policy 0, policy_version 1100006 (0.0008) [2023-12-27 13:16:15,380][20381] Updated weights for policy 1, policy_version 1099065 (0.0007) [2023-12-27 13:16:15,439][20381] Updated weights for policy 1, policy_version 1099075 (0.0008) [2023-12-27 13:16:15,485][20381] Updated weights for policy 1, policy_version 1099085 (0.0007) [2023-12-27 13:16:15,539][20381] Updated weights for policy 1, policy_version 1099095 (0.0008) [2023-12-27 13:16:15,767][20347] Updated weights for policy 0, policy_version 1100016 (0.0008) [2023-12-27 13:16:15,828][20347] Updated weights for policy 0, policy_version 1100026 (0.0007) [2023-12-27 13:16:15,883][20347] Updated weights for policy 0, policy_version 1100036 (0.0008) [2023-12-27 13:16:16,244][20381] Updated weights for policy 1, policy_version 1099105 (0.0008) [2023-12-27 13:16:16,302][20381] Updated weights for policy 1, policy_version 1099115 (0.0008) [2023-12-27 13:16:16,361][20381] Updated weights for policy 1, policy_version 1099125 (0.0008) [2023-12-27 13:16:16,561][20347] Updated weights for policy 0, policy_version 1100046 (0.0008) [2023-12-27 13:16:16,615][20347] Updated weights for policy 0, policy_version 1100056 (0.0008) [2023-12-27 13:16:16,674][20347] Updated weights for policy 0, policy_version 1100066 (0.0008) [2023-12-27 13:16:17,048][20381] Updated weights for policy 1, policy_version 1099135 (0.0008) [2023-12-27 13:16:17,109][20381] Updated weights for policy 1, policy_version 1099145 (0.0008) [2023-12-27 13:16:17,172][20381] Updated weights for policy 1, policy_version 1099155 (0.0007) [2023-12-27 13:16:17,351][20347] Updated weights for policy 0, policy_version 1100076 (0.0007) [2023-12-27 13:16:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 563077120. Throughput: 0: 10103.5, 1: 10254.9. Samples: 563087420. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:17,363][19170] Avg episode reward: [(0, '772.200'), (1, '715.520')] [2023-12-27 13:16:17,409][20347] Updated weights for policy 0, policy_version 1100086 (0.0007) [2023-12-27 13:16:17,463][20347] Updated weights for policy 0, policy_version 1100096 (0.0007) [2023-12-27 13:16:17,864][20381] Updated weights for policy 1, policy_version 1099165 (0.0007) [2023-12-27 13:16:17,913][20381] Updated weights for policy 1, policy_version 1099175 (0.0008) [2023-12-27 13:16:17,964][20381] Updated weights for policy 1, policy_version 1099185 (0.0008) [2023-12-27 13:16:18,162][20347] Updated weights for policy 0, policy_version 1100106 (0.0008) [2023-12-27 13:16:18,222][20347] Updated weights for policy 0, policy_version 1100116 (0.0008) [2023-12-27 13:16:18,280][20347] Updated weights for policy 0, policy_version 1100126 (0.0007) [2023-12-27 13:16:18,339][20347] Updated weights for policy 0, policy_version 1100136 (0.0008) [2023-12-27 13:16:18,590][20381] Updated weights for policy 1, policy_version 1099195 (0.0005) [2023-12-27 13:16:18,650][20381] Updated weights for policy 1, policy_version 1099205 (0.0008) [2023-12-27 13:16:18,707][20381] Updated weights for policy 1, policy_version 1099215 (0.0008) [2023-12-27 13:16:18,977][20347] Updated weights for policy 0, policy_version 1100146 (0.0008) [2023-12-27 13:16:19,032][20347] Updated weights for policy 0, policy_version 1100156 (0.0008) [2023-12-27 13:16:19,084][20347] Updated weights for policy 0, policy_version 1100166 (0.0007) [2023-12-27 13:16:19,340][20381] Updated weights for policy 1, policy_version 1099225 (0.0008) [2023-12-27 13:16:19,405][20381] Updated weights for policy 1, policy_version 1099235 (0.0008) [2023-12-27 13:16:19,460][20381] Updated weights for policy 1, policy_version 1099245 (0.0008) [2023-12-27 13:16:19,521][20381] Updated weights for policy 1, policy_version 1099255 (0.0008) [2023-12-27 13:16:19,802][20347] Updated weights for policy 0, policy_version 1100176 (0.0006) [2023-12-27 13:16:19,879][20347] Updated weights for policy 0, policy_version 1100186 (0.0009) [2023-12-27 13:16:19,938][20347] Updated weights for policy 0, policy_version 1100196 (0.0010) [2023-12-27 13:16:20,277][20381] Updated weights for policy 1, policy_version 1099265 (0.0011) [2023-12-27 13:16:20,340][20381] Updated weights for policy 1, policy_version 1099275 (0.0009) [2023-12-27 13:16:20,403][20381] Updated weights for policy 1, policy_version 1099285 (0.0011) [2023-12-27 13:16:20,613][20347] Updated weights for policy 0, policy_version 1100206 (0.0011) [2023-12-27 13:16:20,667][20347] Updated weights for policy 0, policy_version 1100216 (0.0008) [2023-12-27 13:16:20,726][20347] Updated weights for policy 0, policy_version 1100226 (0.0007) [2023-12-27 13:16:21,056][20381] Updated weights for policy 1, policy_version 1099295 (0.0009) [2023-12-27 13:16:21,109][20381] Updated weights for policy 1, policy_version 1099305 (0.0009) [2023-12-27 13:16:21,169][20381] Updated weights for policy 1, policy_version 1099315 (0.0009) [2023-12-27 13:16:21,527][20347] Updated weights for policy 0, policy_version 1100236 (0.0007) [2023-12-27 13:16:21,579][20347] Updated weights for policy 0, policy_version 1100246 (0.0008) [2023-12-27 13:16:21,636][20347] Updated weights for policy 0, policy_version 1100256 (0.0008) [2023-12-27 13:16:21,942][20381] Updated weights for policy 1, policy_version 1099325 (0.0008) [2023-12-27 13:16:21,998][20381] Updated weights for policy 1, policy_version 1099335 (0.0007) [2023-12-27 13:16:22,061][20381] Updated weights for policy 1, policy_version 1099345 (0.0011) [2023-12-27 13:16:22,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 563175424. Throughput: 0: 10079.9, 1: 10262.7. Samples: 563148808. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:22,363][19170] Avg episode reward: [(0, '787.750'), (1, '656.700')] [2023-12-27 13:16:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001099352_281460736.pth... [2023-12-27 13:16:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001100264_281714688.pth... [2023-12-27 13:16:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001099112_281419776.pth [2023-12-27 13:16:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001098136_281149440.pth [2023-12-27 13:16:22,449][20347] Updated weights for policy 0, policy_version 1100266 (0.0008) [2023-12-27 13:16:22,515][20347] Updated weights for policy 0, policy_version 1100276 (0.0008) [2023-12-27 13:16:22,582][20347] Updated weights for policy 0, policy_version 1100286 (0.0008) [2023-12-27 13:16:22,642][20347] Updated weights for policy 0, policy_version 1100296 (0.0008) [2023-12-27 13:16:22,692][20381] Updated weights for policy 1, policy_version 1099355 (0.0010) [2023-12-27 13:16:22,752][20381] Updated weights for policy 1, policy_version 1099365 (0.0008) [2023-12-27 13:16:22,813][20381] Updated weights for policy 1, policy_version 1099375 (0.0008) [2023-12-27 13:16:23,276][20347] Updated weights for policy 0, policy_version 1100306 (0.0010) [2023-12-27 13:16:23,328][20347] Updated weights for policy 0, policy_version 1100316 (0.0010) [2023-12-27 13:16:23,376][20347] Updated weights for policy 0, policy_version 1100326 (0.0010) [2023-12-27 13:16:23,442][20381] Updated weights for policy 1, policy_version 1099385 (0.0008) [2023-12-27 13:16:23,501][20381] Updated weights for policy 1, policy_version 1099395 (0.0005) [2023-12-27 13:16:23,567][20381] Updated weights for policy 1, policy_version 1099405 (0.0005) [2023-12-27 13:16:23,623][20381] Updated weights for policy 1, policy_version 1099415 (0.0005) [2023-12-27 13:16:24,050][20347] Updated weights for policy 0, policy_version 1100336 (0.0010) [2023-12-27 13:16:24,105][20347] Updated weights for policy 0, policy_version 1100346 (0.0010) [2023-12-27 13:16:24,161][20347] Updated weights for policy 0, policy_version 1100356 (0.0010) [2023-12-27 13:16:24,177][20381] Updated weights for policy 1, policy_version 1099425 (0.0007) [2023-12-27 13:16:24,230][20381] Updated weights for policy 1, policy_version 1099435 (0.0006) [2023-12-27 13:16:24,291][20381] Updated weights for policy 1, policy_version 1099445 (0.0006) [2023-12-27 13:16:24,776][20347] Updated weights for policy 0, policy_version 1100366 (0.0008) [2023-12-27 13:16:24,838][20347] Updated weights for policy 0, policy_version 1100376 (0.0008) [2023-12-27 13:16:24,884][20381] Updated weights for policy 1, policy_version 1099455 (0.0010) [2023-12-27 13:16:24,894][20347] Updated weights for policy 0, policy_version 1100386 (0.0006) [2023-12-27 13:16:24,930][20381] Updated weights for policy 1, policy_version 1099465 (0.0011) [2023-12-27 13:16:24,976][20381] Updated weights for policy 1, policy_version 1099475 (0.0010) [2023-12-27 13:16:25,425][20347] Updated weights for policy 0, policy_version 1100396 (0.0008) [2023-12-27 13:16:25,481][20347] Updated weights for policy 0, policy_version 1100406 (0.0005) [2023-12-27 13:16:25,533][20347] Updated weights for policy 0, policy_version 1100416 (0.0009) [2023-12-27 13:16:25,662][20381] Updated weights for policy 1, policy_version 1099485 (0.0008) [2023-12-27 13:16:25,713][20381] Updated weights for policy 1, policy_version 1099495 (0.0005) [2023-12-27 13:16:25,765][20381] Updated weights for policy 1, policy_version 1099505 (0.0009) [2023-12-27 13:16:26,099][20347] Updated weights for policy 0, policy_version 1100426 (0.0007) [2023-12-27 13:16:26,148][20347] Updated weights for policy 0, policy_version 1100436 (0.0005) [2023-12-27 13:16:26,195][20347] Updated weights for policy 0, policy_version 1100446 (0.0005) [2023-12-27 13:16:26,252][20347] Updated weights for policy 0, policy_version 1100456 (0.0005) [2023-12-27 13:16:26,433][20381] Updated weights for policy 1, policy_version 1099515 (0.0009) [2023-12-27 13:16:26,491][20381] Updated weights for policy 1, policy_version 1099525 (0.0005) [2023-12-27 13:16:26,539][20381] Updated weights for policy 1, policy_version 1099535 (0.0008) [2023-12-27 13:16:26,820][20347] Updated weights for policy 0, policy_version 1100466 (0.0008) [2023-12-27 13:16:26,875][20347] Updated weights for policy 0, policy_version 1100476 (0.0006) [2023-12-27 13:16:26,933][20347] Updated weights for policy 0, policy_version 1100486 (0.0005) [2023-12-27 13:16:27,150][20381] Updated weights for policy 1, policy_version 1099545 (0.0009) [2023-12-27 13:16:27,210][20381] Updated weights for policy 1, policy_version 1099555 (0.0005) [2023-12-27 13:16:27,257][20381] Updated weights for policy 1, policy_version 1099565 (0.0005) [2023-12-27 13:16:27,314][20381] Updated weights for policy 1, policy_version 1099575 (0.0005) [2023-12-27 13:16:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 563290112. Throughput: 0: 10126.1, 1: 10315.9. Samples: 563273924. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:27,363][19170] Avg episode reward: [(0, '748.130'), (1, '656.890')] [2023-12-27 13:16:27,490][20347] Updated weights for policy 0, policy_version 1100496 (0.0007) [2023-12-27 13:16:27,540][20347] Updated weights for policy 0, policy_version 1100506 (0.0006) [2023-12-27 13:16:27,593][20347] Updated weights for policy 0, policy_version 1100516 (0.0006) [2023-12-27 13:16:27,959][20381] Updated weights for policy 1, policy_version 1099585 (0.0008) [2023-12-27 13:16:28,016][20381] Updated weights for policy 1, policy_version 1099595 (0.0007) [2023-12-27 13:16:28,075][20381] Updated weights for policy 1, policy_version 1099605 (0.0008) [2023-12-27 13:16:28,141][20347] Updated weights for policy 0, policy_version 1100526 (0.0005) [2023-12-27 13:16:28,165][20291] KL-divergence is very high: 148.9354 [2023-12-27 13:16:28,192][20347] Updated weights for policy 0, policy_version 1100536 (0.0005) [2023-12-27 13:16:28,211][20291] KL-divergence is very high: 151.9437 [2023-12-27 13:16:28,253][20347] Updated weights for policy 0, policy_version 1100546 (0.0005) [2023-12-27 13:16:28,260][20291] KL-divergence is very high: 108.7698 [2023-12-27 13:16:28,715][20381] Updated weights for policy 1, policy_version 1099615 (0.0007) [2023-12-27 13:16:28,778][20381] Updated weights for policy 1, policy_version 1099625 (0.0007) [2023-12-27 13:16:28,812][20347] Updated weights for policy 0, policy_version 1100556 (0.0005) [2023-12-27 13:16:28,837][20381] Updated weights for policy 1, policy_version 1099635 (0.0007) [2023-12-27 13:16:28,868][20347] Updated weights for policy 0, policy_version 1100566 (0.0005) [2023-12-27 13:16:28,923][20347] Updated weights for policy 0, policy_version 1100576 (0.0007) [2023-12-27 13:16:29,511][20381] Updated weights for policy 1, policy_version 1099645 (0.0007) [2023-12-27 13:16:29,570][20381] Updated weights for policy 1, policy_version 1099655 (0.0006) [2023-12-27 13:16:29,577][20347] Updated weights for policy 0, policy_version 1100586 (0.0007) [2023-12-27 13:16:29,626][20381] Updated weights for policy 1, policy_version 1099665 (0.0007) [2023-12-27 13:16:29,631][20347] Updated weights for policy 0, policy_version 1100596 (0.0005) [2023-12-27 13:16:29,692][20347] Updated weights for policy 0, policy_version 1100606 (0.0006) [2023-12-27 13:16:29,752][20347] Updated weights for policy 0, policy_version 1100616 (0.0006) [2023-12-27 13:16:30,218][20381] Updated weights for policy 1, policy_version 1099675 (0.0009) [2023-12-27 13:16:30,276][20381] Updated weights for policy 1, policy_version 1099685 (0.0006) [2023-12-27 13:16:30,325][20381] Updated weights for policy 1, policy_version 1099695 (0.0007) [2023-12-27 13:16:30,477][20347] Updated weights for policy 0, policy_version 1100626 (0.0008) [2023-12-27 13:16:30,530][20347] Updated weights for policy 0, policy_version 1100636 (0.0008) [2023-12-27 13:16:30,576][20347] Updated weights for policy 0, policy_version 1100646 (0.0009) [2023-12-27 13:16:30,894][20381] Updated weights for policy 1, policy_version 1099705 (0.0008) [2023-12-27 13:16:30,958][20381] Updated weights for policy 1, policy_version 1099715 (0.0009) [2023-12-27 13:16:31,018][20381] Updated weights for policy 1, policy_version 1099725 (0.0008) [2023-12-27 13:16:31,079][20381] Updated weights for policy 1, policy_version 1099735 (0.0007) [2023-12-27 13:16:31,303][20347] Updated weights for policy 0, policy_version 1100656 (0.0006) [2023-12-27 13:16:31,370][20347] Updated weights for policy 0, policy_version 1100666 (0.0006) [2023-12-27 13:16:31,435][20347] Updated weights for policy 0, policy_version 1100676 (0.0007) [2023-12-27 13:16:31,893][20381] Updated weights for policy 1, policy_version 1099745 (0.0008) [2023-12-27 13:16:31,949][20381] Updated weights for policy 1, policy_version 1099755 (0.0008) [2023-12-27 13:16:32,001][20381] Updated weights for policy 1, policy_version 1099765 (0.0006) [2023-12-27 13:16:32,023][20347] Updated weights for policy 0, policy_version 1100686 (0.0006) [2023-12-27 13:16:32,077][20347] Updated weights for policy 0, policy_version 1100696 (0.0007) [2023-12-27 13:16:32,133][20347] Updated weights for policy 0, policy_version 1100706 (0.0007) [2023-12-27 13:16:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 563396608. Throughput: 0: 10209.0, 1: 10321.5. Samples: 563403756. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:32,363][19170] Avg episode reward: [(0, '776.530'), (1, '574.920')] [2023-12-27 13:16:32,696][20381] Updated weights for policy 1, policy_version 1099775 (0.0007) [2023-12-27 13:16:32,761][20381] Updated weights for policy 1, policy_version 1099785 (0.0007) [2023-12-27 13:16:32,806][20347] Updated weights for policy 0, policy_version 1100716 (0.0009) [2023-12-27 13:16:32,820][20381] Updated weights for policy 1, policy_version 1099795 (0.0006) [2023-12-27 13:16:32,861][20347] Updated weights for policy 0, policy_version 1100726 (0.0010) [2023-12-27 13:16:32,919][20347] Updated weights for policy 0, policy_version 1100736 (0.0010) [2023-12-27 13:16:33,454][20381] Updated weights for policy 1, policy_version 1099805 (0.0008) [2023-12-27 13:16:33,504][20381] Updated weights for policy 1, policy_version 1099815 (0.0008) [2023-12-27 13:16:33,559][20381] Updated weights for policy 1, policy_version 1099825 (0.0009) [2023-12-27 13:16:33,572][20347] Updated weights for policy 0, policy_version 1100746 (0.0009) [2023-12-27 13:16:33,628][20347] Updated weights for policy 0, policy_version 1100756 (0.0007) [2023-12-27 13:16:33,686][20347] Updated weights for policy 0, policy_version 1100766 (0.0008) [2023-12-27 13:16:33,739][20347] Updated weights for policy 0, policy_version 1100776 (0.0008) [2023-12-27 13:16:34,287][20381] Updated weights for policy 1, policy_version 1099835 (0.0007) [2023-12-27 13:16:34,355][20381] Updated weights for policy 1, policy_version 1099845 (0.0009) [2023-12-27 13:16:34,404][20347] Updated weights for policy 0, policy_version 1100786 (0.0009) [2023-12-27 13:16:34,410][20381] Updated weights for policy 1, policy_version 1099855 (0.0007) [2023-12-27 13:16:34,460][20347] Updated weights for policy 0, policy_version 1100796 (0.0008) [2023-12-27 13:16:34,521][20347] Updated weights for policy 0, policy_version 1100806 (0.0005) [2023-12-27 13:16:35,125][20347] Updated weights for policy 0, policy_version 1100816 (0.0006) [2023-12-27 13:16:35,142][20381] Updated weights for policy 1, policy_version 1099865 (0.0007) [2023-12-27 13:16:35,177][20347] Updated weights for policy 0, policy_version 1100826 (0.0008) [2023-12-27 13:16:35,188][20381] Updated weights for policy 1, policy_version 1099875 (0.0007) [2023-12-27 13:16:35,238][20381] Updated weights for policy 1, policy_version 1099885 (0.0009) [2023-12-27 13:16:35,256][20347] Updated weights for policy 0, policy_version 1100836 (0.0005) [2023-12-27 13:16:35,287][20381] Updated weights for policy 1, policy_version 1099895 (0.0007) [2023-12-27 13:16:35,789][20347] Updated weights for policy 0, policy_version 1100846 (0.0005) [2023-12-27 13:16:35,842][20347] Updated weights for policy 0, policy_version 1100856 (0.0007) [2023-12-27 13:16:35,906][20347] Updated weights for policy 0, policy_version 1100866 (0.0007) [2023-12-27 13:16:36,053][20381] Updated weights for policy 1, policy_version 1099905 (0.0009) [2023-12-27 13:16:36,115][20381] Updated weights for policy 1, policy_version 1099915 (0.0009) [2023-12-27 13:16:36,170][20381] Updated weights for policy 1, policy_version 1099925 (0.0008) [2023-12-27 13:16:36,490][20347] Updated weights for policy 0, policy_version 1100876 (0.0005) [2023-12-27 13:16:36,555][20347] Updated weights for policy 0, policy_version 1100886 (0.0006) [2023-12-27 13:16:36,617][20347] Updated weights for policy 0, policy_version 1100896 (0.0006) [2023-12-27 13:16:36,891][20381] Updated weights for policy 1, policy_version 1099935 (0.0010) [2023-12-27 13:16:36,937][20381] Updated weights for policy 1, policy_version 1099945 (0.0010) [2023-12-27 13:16:36,986][20381] Updated weights for policy 1, policy_version 1099955 (0.0010) [2023-12-27 13:16:37,212][20347] Updated weights for policy 0, policy_version 1100906 (0.0006) [2023-12-27 13:16:37,263][20347] Updated weights for policy 0, policy_version 1100916 (0.0005) [2023-12-27 13:16:37,312][20347] Updated weights for policy 0, policy_version 1100926 (0.0007) [2023-12-27 13:16:37,362][20347] Updated weights for policy 0, policy_version 1100936 (0.0007) [2023-12-27 13:16:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 563503104. Throughput: 0: 10195.3, 1: 10346.5. Samples: 563465400. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:37,363][19170] Avg episode reward: [(0, '785.980'), (1, '576.410')] [2023-12-27 13:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001100936_281886720.pth... [2023-12-27 13:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001099960_281616384.pth... [2023-12-27 13:16:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001099688_281567232.pth [2023-12-27 13:16:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001098744_281305088.pth [2023-12-27 13:16:37,701][20381] Updated weights for policy 1, policy_version 1099965 (0.0008) [2023-12-27 13:16:37,758][20381] Updated weights for policy 1, policy_version 1099975 (0.0005) [2023-12-27 13:16:37,817][20381] Updated weights for policy 1, policy_version 1099985 (0.0005) [2023-12-27 13:16:38,048][20347] Updated weights for policy 0, policy_version 1100946 (0.0005) [2023-12-27 13:16:38,105][20347] Updated weights for policy 0, policy_version 1100956 (0.0006) [2023-12-27 13:16:38,165][20347] Updated weights for policy 0, policy_version 1100966 (0.0006) [2023-12-27 13:16:38,411][20381] Updated weights for policy 1, policy_version 1099995 (0.0006) [2023-12-27 13:16:38,471][20381] Updated weights for policy 1, policy_version 1100005 (0.0008) [2023-12-27 13:16:38,529][20381] Updated weights for policy 1, policy_version 1100015 (0.0008) [2023-12-27 13:16:38,841][20347] Updated weights for policy 0, policy_version 1100976 (0.0007) [2023-12-27 13:16:38,904][20347] Updated weights for policy 0, policy_version 1100986 (0.0006) [2023-12-27 13:16:38,965][20347] Updated weights for policy 0, policy_version 1100996 (0.0008) [2023-12-27 13:16:39,139][20381] Updated weights for policy 1, policy_version 1100025 (0.0008) [2023-12-27 13:16:39,187][20381] Updated weights for policy 1, policy_version 1100035 (0.0009) [2023-12-27 13:16:39,248][20381] Updated weights for policy 1, policy_version 1100045 (0.0008) [2023-12-27 13:16:39,304][20381] Updated weights for policy 1, policy_version 1100055 (0.0005) [2023-12-27 13:16:39,655][20347] Updated weights for policy 0, policy_version 1101006 (0.0008) [2023-12-27 13:16:39,706][20347] Updated weights for policy 0, policy_version 1101016 (0.0008) [2023-12-27 13:16:39,756][20347] Updated weights for policy 0, policy_version 1101026 (0.0007) [2023-12-27 13:16:39,964][20381] Updated weights for policy 1, policy_version 1100065 (0.0010) [2023-12-27 13:16:40,024][20381] Updated weights for policy 1, policy_version 1100075 (0.0010) [2023-12-27 13:16:40,080][20381] Updated weights for policy 1, policy_version 1100085 (0.0011) [2023-12-27 13:16:40,558][20347] Updated weights for policy 0, policy_version 1101036 (0.0008) [2023-12-27 13:16:40,615][20347] Updated weights for policy 0, policy_version 1101046 (0.0008) [2023-12-27 13:16:40,680][20347] Updated weights for policy 0, policy_version 1101056 (0.0007) [2023-12-27 13:16:40,735][20381] Updated weights for policy 1, policy_version 1100095 (0.0011) [2023-12-27 13:16:40,790][20381] Updated weights for policy 1, policy_version 1100105 (0.0010) [2023-12-27 13:16:40,846][20381] Updated weights for policy 1, policy_version 1100115 (0.0010) [2023-12-27 13:16:41,476][20347] Updated weights for policy 0, policy_version 1101066 (0.0006) [2023-12-27 13:16:41,534][20347] Updated weights for policy 0, policy_version 1101076 (0.0006) [2023-12-27 13:16:41,550][20381] Updated weights for policy 1, policy_version 1100125 (0.0009) [2023-12-27 13:16:41,593][20347] Updated weights for policy 0, policy_version 1101086 (0.0007) [2023-12-27 13:16:41,610][20381] Updated weights for policy 1, policy_version 1100135 (0.0009) [2023-12-27 13:16:41,662][20347] Updated weights for policy 0, policy_version 1101096 (0.0008) [2023-12-27 13:16:41,676][20381] Updated weights for policy 1, policy_version 1100145 (0.0009) [2023-12-27 13:16:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 563593216. Throughput: 0: 10253.3, 1: 10347.0. Samples: 563589584. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:42,363][19170] Avg episode reward: [(0, '853.350'), (1, '640.020')] [2023-12-27 13:16:42,409][20347] Updated weights for policy 0, policy_version 1101106 (0.0008) [2023-12-27 13:16:42,473][20347] Updated weights for policy 0, policy_version 1101116 (0.0009) [2023-12-27 13:16:42,530][20347] Updated weights for policy 0, policy_version 1101126 (0.0006) [2023-12-27 13:16:42,532][20381] Updated weights for policy 1, policy_version 1100155 (0.0009) [2023-12-27 13:16:42,596][20381] Updated weights for policy 1, policy_version 1100165 (0.0008) [2023-12-27 13:16:42,661][20381] Updated weights for policy 1, policy_version 1100175 (0.0008) [2023-12-27 13:16:43,191][20347] Updated weights for policy 0, policy_version 1101136 (0.0010) [2023-12-27 13:16:43,243][20347] Updated weights for policy 0, policy_version 1101146 (0.0010) [2023-12-27 13:16:43,294][20347] Updated weights for policy 0, policy_version 1101156 (0.0007) [2023-12-27 13:16:43,385][20381] Updated weights for policy 1, policy_version 1100185 (0.0008) [2023-12-27 13:16:43,433][20381] Updated weights for policy 1, policy_version 1100195 (0.0008) [2023-12-27 13:16:43,481][20381] Updated weights for policy 1, policy_version 1100205 (0.0007) [2023-12-27 13:16:43,535][20381] Updated weights for policy 1, policy_version 1100215 (0.0007) [2023-12-27 13:16:43,901][20347] Updated weights for policy 0, policy_version 1101166 (0.0008) [2023-12-27 13:16:43,968][20347] Updated weights for policy 0, policy_version 1101176 (0.0007) [2023-12-27 13:16:44,015][20347] Updated weights for policy 0, policy_version 1101186 (0.0007) [2023-12-27 13:16:44,282][20381] Updated weights for policy 1, policy_version 1100225 (0.0007) [2023-12-27 13:16:44,341][20381] Updated weights for policy 1, policy_version 1100235 (0.0007) [2023-12-27 13:16:44,404][20381] Updated weights for policy 1, policy_version 1100245 (0.0008) [2023-12-27 13:16:44,679][20347] Updated weights for policy 0, policy_version 1101196 (0.0007) [2023-12-27 13:16:44,731][20347] Updated weights for policy 0, policy_version 1101206 (0.0008) [2023-12-27 13:16:44,796][20347] Updated weights for policy 0, policy_version 1101216 (0.0008) [2023-12-27 13:16:45,091][20381] Updated weights for policy 1, policy_version 1100255 (0.0007) [2023-12-27 13:16:45,142][20381] Updated weights for policy 1, policy_version 1100265 (0.0008) [2023-12-27 13:16:45,193][20381] Updated weights for policy 1, policy_version 1100275 (0.0007) [2023-12-27 13:16:45,493][20347] Updated weights for policy 0, policy_version 1101226 (0.0006) [2023-12-27 13:16:45,548][20347] Updated weights for policy 0, policy_version 1101236 (0.0006) [2023-12-27 13:16:45,597][20347] Updated weights for policy 0, policy_version 1101246 (0.0008) [2023-12-27 13:16:45,646][20347] Updated weights for policy 0, policy_version 1101256 (0.0008) [2023-12-27 13:16:45,940][20381] Updated weights for policy 1, policy_version 1100285 (0.0008) [2023-12-27 13:16:45,998][20381] Updated weights for policy 1, policy_version 1100295 (0.0007) [2023-12-27 13:16:46,052][20381] Updated weights for policy 1, policy_version 1100305 (0.0008) [2023-12-27 13:16:46,257][20347] Updated weights for policy 0, policy_version 1101266 (0.0008) [2023-12-27 13:16:46,312][20347] Updated weights for policy 0, policy_version 1101276 (0.0010) [2023-12-27 13:16:46,363][20347] Updated weights for policy 0, policy_version 1101286 (0.0010) [2023-12-27 13:16:46,778][20381] Updated weights for policy 1, policy_version 1100315 (0.0008) [2023-12-27 13:16:46,831][20381] Updated weights for policy 1, policy_version 1100325 (0.0008) [2023-12-27 13:16:46,885][20381] Updated weights for policy 1, policy_version 1100335 (0.0007) [2023-12-27 13:16:47,016][20347] Updated weights for policy 0, policy_version 1101296 (0.0007) [2023-12-27 13:16:47,068][20347] Updated weights for policy 0, policy_version 1101306 (0.0007) [2023-12-27 13:16:47,120][20347] Updated weights for policy 0, policy_version 1101316 (0.0006) [2023-12-27 13:16:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 563699712. Throughput: 0: 10355.8, 1: 10270.7. Samples: 563708088. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:47,363][19170] Avg episode reward: [(0, '853.430'), (1, '587.810')] [2023-12-27 13:16:47,572][20381] Updated weights for policy 1, policy_version 1100345 (0.0007) [2023-12-27 13:16:47,616][20381] Updated weights for policy 1, policy_version 1100355 (0.0007) [2023-12-27 13:16:47,663][20381] Updated weights for policy 1, policy_version 1100365 (0.0008) [2023-12-27 13:16:47,719][20381] Updated weights for policy 1, policy_version 1100375 (0.0008) [2023-12-27 13:16:47,819][20347] Updated weights for policy 0, policy_version 1101326 (0.0007) [2023-12-27 13:16:47,884][20347] Updated weights for policy 0, policy_version 1101336 (0.0008) [2023-12-27 13:16:47,942][20347] Updated weights for policy 0, policy_version 1101346 (0.0008) [2023-12-27 13:16:48,452][20381] Updated weights for policy 1, policy_version 1100385 (0.0006) [2023-12-27 13:16:48,512][20381] Updated weights for policy 1, policy_version 1100395 (0.0005) [2023-12-27 13:16:48,565][20381] Updated weights for policy 1, policy_version 1100405 (0.0006) [2023-12-27 13:16:48,567][20347] Updated weights for policy 0, policy_version 1101356 (0.0008) [2023-12-27 13:16:48,623][20347] Updated weights for policy 0, policy_version 1101366 (0.0008) [2023-12-27 13:16:48,682][20347] Updated weights for policy 0, policy_version 1101376 (0.0008) [2023-12-27 13:16:49,202][20381] Updated weights for policy 1, policy_version 1100415 (0.0007) [2023-12-27 13:16:49,266][20381] Updated weights for policy 1, policy_version 1100425 (0.0008) [2023-12-27 13:16:49,316][20381] Updated weights for policy 1, policy_version 1100435 (0.0007) [2023-12-27 13:16:49,388][20347] Updated weights for policy 0, policy_version 1101386 (0.0007) [2023-12-27 13:16:49,440][20347] Updated weights for policy 0, policy_version 1101396 (0.0005) [2023-12-27 13:16:49,497][20347] Updated weights for policy 0, policy_version 1101406 (0.0006) [2023-12-27 13:16:49,557][20347] Updated weights for policy 0, policy_version 1101416 (0.0005) [2023-12-27 13:16:50,105][20381] Updated weights for policy 1, policy_version 1100445 (0.0008) [2023-12-27 13:16:50,162][20381] Updated weights for policy 1, policy_version 1100455 (0.0006) [2023-12-27 13:16:50,195][20347] Updated weights for policy 0, policy_version 1101426 (0.0007) [2023-12-27 13:16:50,214][20381] Updated weights for policy 1, policy_version 1100465 (0.0006) [2023-12-27 13:16:50,251][20347] Updated weights for policy 0, policy_version 1101436 (0.0006) [2023-12-27 13:16:50,305][20347] Updated weights for policy 0, policy_version 1101446 (0.0008) [2023-12-27 13:16:50,889][20381] Updated weights for policy 1, policy_version 1100475 (0.0007) [2023-12-27 13:16:50,942][20381] Updated weights for policy 1, policy_version 1100485 (0.0008) [2023-12-27 13:16:50,994][20381] Updated weights for policy 1, policy_version 1100495 (0.0007) [2023-12-27 13:16:51,019][20347] Updated weights for policy 0, policy_version 1101456 (0.0007) [2023-12-27 13:16:51,079][20347] Updated weights for policy 0, policy_version 1101466 (0.0008) [2023-12-27 13:16:51,139][20347] Updated weights for policy 0, policy_version 1101476 (0.0008) [2023-12-27 13:16:51,754][20381] Updated weights for policy 1, policy_version 1100505 (0.0007) [2023-12-27 13:16:51,818][20381] Updated weights for policy 1, policy_version 1100515 (0.0009) [2023-12-27 13:16:51,875][20347] Updated weights for policy 0, policy_version 1101486 (0.0007) [2023-12-27 13:16:51,879][20381] Updated weights for policy 1, policy_version 1100525 (0.0008) [2023-12-27 13:16:51,934][20347] Updated weights for policy 0, policy_version 1101496 (0.0009) [2023-12-27 13:16:51,940][20381] Updated weights for policy 1, policy_version 1100535 (0.0009) [2023-12-27 13:16:51,996][20347] Updated weights for policy 0, policy_version 1101506 (0.0007) [2023-12-27 13:16:52,002][20291] KL-divergence is very high: 356.4251 [2023-12-27 13:16:52,019][20291] KL-divergence is very high: 256.7674 [2023-12-27 13:16:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 563798016. Throughput: 0: 10365.0, 1: 10221.2. Samples: 563768636. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:52,363][19170] Avg episode reward: [(0, '834.540'), (1, '558.550')] [2023-12-27 13:16:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001100536_281763840.pth... [2023-12-27 13:16:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001101512_282034176.pth... [2023-12-27 13:16:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001100264_281714688.pth [2023-12-27 13:16:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001099352_281460736.pth [2023-12-27 13:16:52,642][20381] Updated weights for policy 1, policy_version 1100545 (0.0006) [2023-12-27 13:16:52,696][20347] Updated weights for policy 0, policy_version 1101516 (0.0007) [2023-12-27 13:16:52,705][20381] Updated weights for policy 1, policy_version 1100555 (0.0011) [2023-12-27 13:16:52,734][20291] KL-divergence is very high: 113.5155 [2023-12-27 13:16:52,750][20347] Updated weights for policy 0, policy_version 1101526 (0.0006) [2023-12-27 13:16:52,768][20381] Updated weights for policy 1, policy_version 1100565 (0.0011) [2023-12-27 13:16:52,807][20347] Updated weights for policy 0, policy_version 1101536 (0.0006) [2023-12-27 13:16:53,378][20381] Updated weights for policy 1, policy_version 1100575 (0.0009) [2023-12-27 13:16:53,382][20347] Updated weights for policy 0, policy_version 1101546 (0.0005) [2023-12-27 13:16:53,413][20291] KL-divergence is very high: 104.4411 [2023-12-27 13:16:53,425][20381] Updated weights for policy 1, policy_version 1100585 (0.0009) [2023-12-27 13:16:53,434][20347] Updated weights for policy 0, policy_version 1101556 (0.0005) [2023-12-27 13:16:53,477][20381] Updated weights for policy 1, policy_version 1100595 (0.0009) [2023-12-27 13:16:53,490][20347] Updated weights for policy 0, policy_version 1101566 (0.0005) [2023-12-27 13:16:53,540][20347] Updated weights for policy 0, policy_version 1101576 (0.0006) [2023-12-27 13:16:54,136][20381] Updated weights for policy 1, policy_version 1100605 (0.0011) [2023-12-27 13:16:54,177][20291] KL-divergence is very high: 108.4426 [2023-12-27 13:16:54,190][20381] Updated weights for policy 1, policy_version 1100615 (0.0010) [2023-12-27 13:16:54,219][20347] Updated weights for policy 0, policy_version 1101586 (0.0007) [2023-12-27 13:16:54,219][20291] KL-divergence is very high: 117.8737 [2023-12-27 13:16:54,245][20381] Updated weights for policy 1, policy_version 1100625 (0.0007) [2023-12-27 13:16:54,274][20347] Updated weights for policy 0, policy_version 1101596 (0.0007) [2023-12-27 13:16:54,326][20347] Updated weights for policy 0, policy_version 1101606 (0.0005) [2023-12-27 13:16:54,853][20381] Updated weights for policy 1, policy_version 1100635 (0.0006) [2023-12-27 13:16:54,911][20381] Updated weights for policy 1, policy_version 1100645 (0.0008) [2023-12-27 13:16:54,934][20347] Updated weights for policy 0, policy_version 1101616 (0.0007) [2023-12-27 13:16:54,964][20381] Updated weights for policy 1, policy_version 1100655 (0.0008) [2023-12-27 13:16:54,988][20347] Updated weights for policy 0, policy_version 1101626 (0.0008) [2023-12-27 13:16:55,041][20347] Updated weights for policy 0, policy_version 1101636 (0.0009) [2023-12-27 13:16:55,647][20347] Updated weights for policy 0, policy_version 1101646 (0.0008) [2023-12-27 13:16:55,668][20381] Updated weights for policy 1, policy_version 1100665 (0.0009) [2023-12-27 13:16:55,710][20347] Updated weights for policy 0, policy_version 1101656 (0.0007) [2023-12-27 13:16:55,727][20381] Updated weights for policy 1, policy_version 1100675 (0.0007) [2023-12-27 13:16:55,760][20347] Updated weights for policy 0, policy_version 1101666 (0.0006) [2023-12-27 13:16:55,779][20381] Updated weights for policy 1, policy_version 1100685 (0.0006) [2023-12-27 13:16:55,839][20381] Updated weights for policy 1, policy_version 1100695 (0.0005) [2023-12-27 13:16:56,432][20347] Updated weights for policy 0, policy_version 1101676 (0.0008) [2023-12-27 13:16:56,451][20291] KL-divergence is very high: 107.5572 [2023-12-27 13:16:56,464][20291] KL-divergence is very high: 180.1803 [2023-12-27 13:16:56,477][20347] Updated weights for policy 0, policy_version 1101686 (0.0005) [2023-12-27 13:16:56,482][20381] Updated weights for policy 1, policy_version 1100705 (0.0006) [2023-12-27 13:16:56,489][20291] KL-divergence is very high: 188.8623 [2023-12-27 13:16:56,502][20291] KL-divergence is very high: 143.9780 [2023-12-27 13:16:56,525][20347] Updated weights for policy 0, policy_version 1101696 (0.0006) [2023-12-27 13:16:56,535][20381] Updated weights for policy 1, policy_version 1100715 (0.0008) [2023-12-27 13:16:56,593][20381] Updated weights for policy 1, policy_version 1100725 (0.0006) [2023-12-27 13:16:57,224][20381] Updated weights for policy 1, policy_version 1100735 (0.0007) [2023-12-27 13:16:57,238][20347] Updated weights for policy 0, policy_version 1101706 (0.0007) [2023-12-27 13:16:57,287][20381] Updated weights for policy 1, policy_version 1100745 (0.0008) [2023-12-27 13:16:57,295][20347] Updated weights for policy 0, policy_version 1101716 (0.0007) [2023-12-27 13:16:57,344][20381] Updated weights for policy 1, policy_version 1100755 (0.0006) [2023-12-27 13:16:57,349][20347] Updated weights for policy 0, policy_version 1101726 (0.0008) [2023-12-27 13:16:57,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 563896320. Throughput: 0: 10391.8, 1: 10186.7. Samples: 563893688. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:16:57,363][19170] Avg episode reward: [(0, '814.590'), (1, '666.440')] [2023-12-27 13:16:57,401][20347] Updated weights for policy 0, policy_version 1101736 (0.0007) [2023-12-27 13:16:57,975][20381] Updated weights for policy 1, policy_version 1100765 (0.0006) [2023-12-27 13:16:58,026][20381] Updated weights for policy 1, policy_version 1100775 (0.0005) [2023-12-27 13:16:58,076][20381] Updated weights for policy 1, policy_version 1100785 (0.0008) [2023-12-27 13:16:58,104][20347] Updated weights for policy 0, policy_version 1101746 (0.0008) [2023-12-27 13:16:58,163][20347] Updated weights for policy 0, policy_version 1101756 (0.0005) [2023-12-27 13:16:58,232][20347] Updated weights for policy 0, policy_version 1101766 (0.0007) [2023-12-27 13:16:58,901][20381] Updated weights for policy 1, policy_version 1100795 (0.0009) [2023-12-27 13:16:58,913][20347] Updated weights for policy 0, policy_version 1101776 (0.0007) [2023-12-27 13:16:58,968][20381] Updated weights for policy 1, policy_version 1100805 (0.0008) [2023-12-27 13:16:58,975][20347] Updated weights for policy 0, policy_version 1101786 (0.0007) [2023-12-27 13:16:59,024][20381] Updated weights for policy 1, policy_version 1100815 (0.0005) [2023-12-27 13:16:59,032][20347] Updated weights for policy 0, policy_version 1101796 (0.0006) [2023-12-27 13:16:59,675][20381] Updated weights for policy 1, policy_version 1100825 (0.0008) [2023-12-27 13:16:59,731][20381] Updated weights for policy 1, policy_version 1100835 (0.0006) [2023-12-27 13:16:59,731][20347] Updated weights for policy 0, policy_version 1101806 (0.0007) [2023-12-27 13:16:59,779][20347] Updated weights for policy 0, policy_version 1101816 (0.0006) [2023-12-27 13:16:59,784][20381] Updated weights for policy 1, policy_version 1100845 (0.0008) [2023-12-27 13:16:59,838][20347] Updated weights for policy 0, policy_version 1101826 (0.0007) [2023-12-27 13:16:59,846][20381] Updated weights for policy 1, policy_version 1100855 (0.0007) [2023-12-27 13:17:00,526][20347] Updated weights for policy 0, policy_version 1101836 (0.0008) [2023-12-27 13:17:00,536][20381] Updated weights for policy 1, policy_version 1100865 (0.0006) [2023-12-27 13:17:00,583][20347] Updated weights for policy 0, policy_version 1101846 (0.0006) [2023-12-27 13:17:00,596][20381] Updated weights for policy 1, policy_version 1100875 (0.0006) [2023-12-27 13:17:00,640][20347] Updated weights for policy 0, policy_version 1101856 (0.0006) [2023-12-27 13:17:00,650][20381] Updated weights for policy 1, policy_version 1100885 (0.0006) [2023-12-27 13:17:01,324][20347] Updated weights for policy 0, policy_version 1101866 (0.0007) [2023-12-27 13:17:01,383][20381] Updated weights for policy 1, policy_version 1100895 (0.0008) [2023-12-27 13:17:01,392][20347] Updated weights for policy 0, policy_version 1101876 (0.0009) [2023-12-27 13:17:01,447][20381] Updated weights for policy 1, policy_version 1100905 (0.0007) [2023-12-27 13:17:01,453][20347] Updated weights for policy 0, policy_version 1101886 (0.0007) [2023-12-27 13:17:01,504][20347] Updated weights for policy 0, policy_version 1101896 (0.0007) [2023-12-27 13:17:01,506][20381] Updated weights for policy 1, policy_version 1100915 (0.0006) [2023-12-27 13:17:02,185][20381] Updated weights for policy 1, policy_version 1100925 (0.0007) [2023-12-27 13:17:02,222][20347] Updated weights for policy 0, policy_version 1101906 (0.0010) [2023-12-27 13:17:02,239][20381] Updated weights for policy 1, policy_version 1100935 (0.0005) [2023-12-27 13:17:02,289][20347] Updated weights for policy 0, policy_version 1101916 (0.0010) [2023-12-27 13:17:02,302][20381] Updated weights for policy 1, policy_version 1100945 (0.0006) [2023-12-27 13:17:02,344][20347] Updated weights for policy 0, policy_version 1101926 (0.0010) [2023-12-27 13:17:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 564011008. Throughput: 0: 10419.6, 1: 10155.6. Samples: 564013304. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:02,363][19170] Avg episode reward: [(0, '813.930'), (1, '745.290')] [2023-12-27 13:17:02,951][20381] Updated weights for policy 1, policy_version 1100955 (0.0007) [2023-12-27 13:17:02,998][20381] Updated weights for policy 1, policy_version 1100965 (0.0007) [2023-12-27 13:17:03,049][20381] Updated weights for policy 1, policy_version 1100975 (0.0006) [2023-12-27 13:17:03,070][20347] Updated weights for policy 0, policy_version 1101936 (0.0010) [2023-12-27 13:17:03,115][20347] Updated weights for policy 0, policy_version 1101946 (0.0010) [2023-12-27 13:17:03,160][20347] Updated weights for policy 0, policy_version 1101956 (0.0010) [2023-12-27 13:17:03,729][20381] Updated weights for policy 1, policy_version 1100985 (0.0006) [2023-12-27 13:17:03,782][20381] Updated weights for policy 1, policy_version 1100995 (0.0007) [2023-12-27 13:17:03,838][20381] Updated weights for policy 1, policy_version 1101005 (0.0008) [2023-12-27 13:17:03,861][20347] Updated weights for policy 0, policy_version 1101966 (0.0009) [2023-12-27 13:17:03,898][20381] Updated weights for policy 1, policy_version 1101015 (0.0006) [2023-12-27 13:17:03,922][20347] Updated weights for policy 0, policy_version 1101976 (0.0010) [2023-12-27 13:17:03,974][20347] Updated weights for policy 0, policy_version 1101986 (0.0010) [2023-12-27 13:17:04,621][20381] Updated weights for policy 1, policy_version 1101025 (0.0008) [2023-12-27 13:17:04,635][20347] Updated weights for policy 0, policy_version 1101996 (0.0008) [2023-12-27 13:17:04,670][20381] Updated weights for policy 1, policy_version 1101035 (0.0007) [2023-12-27 13:17:04,685][20347] Updated weights for policy 0, policy_version 1102006 (0.0005) [2023-12-27 13:17:04,723][20381] Updated weights for policy 1, policy_version 1101045 (0.0006) [2023-12-27 13:17:04,733][20347] Updated weights for policy 0, policy_version 1102016 (0.0006) [2023-12-27 13:17:05,360][20347] Updated weights for policy 0, policy_version 1102026 (0.0007) [2023-12-27 13:17:05,395][20381] Updated weights for policy 1, policy_version 1101055 (0.0007) [2023-12-27 13:17:05,419][20347] Updated weights for policy 0, policy_version 1102036 (0.0005) [2023-12-27 13:17:05,444][20381] Updated weights for policy 1, policy_version 1101065 (0.0007) [2023-12-27 13:17:05,473][20347] Updated weights for policy 0, policy_version 1102046 (0.0005) [2023-12-27 13:17:05,499][20381] Updated weights for policy 1, policy_version 1101075 (0.0008) [2023-12-27 13:17:05,523][20347] Updated weights for policy 0, policy_version 1102056 (0.0007) [2023-12-27 13:17:06,129][20347] Updated weights for policy 0, policy_version 1102066 (0.0007) [2023-12-27 13:17:06,184][20347] Updated weights for policy 0, policy_version 1102076 (0.0006) [2023-12-27 13:17:06,207][20381] Updated weights for policy 1, policy_version 1101085 (0.0008) [2023-12-27 13:17:06,245][20347] Updated weights for policy 0, policy_version 1102086 (0.0006) [2023-12-27 13:17:06,268][20381] Updated weights for policy 1, policy_version 1101095 (0.0007) [2023-12-27 13:17:06,333][20381] Updated weights for policy 1, policy_version 1101105 (0.0009) [2023-12-27 13:17:06,851][20347] Updated weights for policy 0, policy_version 1102096 (0.0007) [2023-12-27 13:17:06,910][20347] Updated weights for policy 0, policy_version 1102106 (0.0008) [2023-12-27 13:17:06,973][20347] Updated weights for policy 0, policy_version 1102116 (0.0005) [2023-12-27 13:17:07,048][20381] Updated weights for policy 1, policy_version 1101115 (0.0009) [2023-12-27 13:17:07,096][20381] Updated weights for policy 1, policy_version 1101125 (0.0008) [2023-12-27 13:17:07,151][20381] Updated weights for policy 1, policy_version 1101135 (0.0009) [2023-12-27 13:17:07,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 564109312. Throughput: 0: 10427.2, 1: 10153.7. Samples: 564074956. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:07,364][19170] Avg episode reward: [(0, '862.860'), (1, '676.440')] [2023-12-27 13:17:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001101144_281919488.pth... [2023-12-27 13:17:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001102120_282189824.pth... [2023-12-27 13:17:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001099960_281616384.pth [2023-12-27 13:17:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001100936_281886720.pth [2023-12-27 13:17:07,551][20347] Updated weights for policy 0, policy_version 1102126 (0.0007) [2023-12-27 13:17:07,601][20347] Updated weights for policy 0, policy_version 1102136 (0.0007) [2023-12-27 13:17:07,659][20347] Updated weights for policy 0, policy_version 1102146 (0.0005) [2023-12-27 13:17:07,913][20381] Updated weights for policy 1, policy_version 1101145 (0.0007) [2023-12-27 13:17:07,961][20381] Updated weights for policy 1, policy_version 1101155 (0.0007) [2023-12-27 13:17:08,009][20381] Updated weights for policy 1, policy_version 1101165 (0.0007) [2023-12-27 13:17:08,054][20381] Updated weights for policy 1, policy_version 1101175 (0.0005) [2023-12-27 13:17:08,271][20347] Updated weights for policy 0, policy_version 1102156 (0.0005) [2023-12-27 13:17:08,323][20347] Updated weights for policy 0, policy_version 1102166 (0.0006) [2023-12-27 13:17:08,385][20347] Updated weights for policy 0, policy_version 1102176 (0.0008) [2023-12-27 13:17:08,776][20381] Updated weights for policy 1, policy_version 1101185 (0.0008) [2023-12-27 13:17:08,832][20381] Updated weights for policy 1, policy_version 1101195 (0.0009) [2023-12-27 13:17:08,889][20381] Updated weights for policy 1, policy_version 1101205 (0.0007) [2023-12-27 13:17:09,050][20347] Updated weights for policy 0, policy_version 1102186 (0.0006) [2023-12-27 13:17:09,100][20347] Updated weights for policy 0, policy_version 1102196 (0.0007) [2023-12-27 13:17:09,159][20347] Updated weights for policy 0, policy_version 1102206 (0.0007) [2023-12-27 13:17:09,217][20347] Updated weights for policy 0, policy_version 1102216 (0.0008) [2023-12-27 13:17:09,618][20381] Updated weights for policy 1, policy_version 1101215 (0.0007) [2023-12-27 13:17:09,671][20381] Updated weights for policy 1, policy_version 1101225 (0.0008) [2023-12-27 13:17:09,725][20381] Updated weights for policy 1, policy_version 1101235 (0.0008) [2023-12-27 13:17:09,983][20347] Updated weights for policy 0, policy_version 1102226 (0.0008) [2023-12-27 13:17:10,040][20347] Updated weights for policy 0, policy_version 1102236 (0.0007) [2023-12-27 13:17:10,092][20347] Updated weights for policy 0, policy_version 1102246 (0.0008) [2023-12-27 13:17:10,491][20381] Updated weights for policy 1, policy_version 1101245 (0.0008) [2023-12-27 13:17:10,548][20381] Updated weights for policy 1, policy_version 1101255 (0.0008) [2023-12-27 13:17:10,604][20381] Updated weights for policy 1, policy_version 1101265 (0.0009) [2023-12-27 13:17:10,814][20347] Updated weights for policy 0, policy_version 1102256 (0.0007) [2023-12-27 13:17:10,875][20347] Updated weights for policy 0, policy_version 1102266 (0.0008) [2023-12-27 13:17:10,927][20347] Updated weights for policy 0, policy_version 1102276 (0.0008) [2023-12-27 13:17:11,300][20381] Updated weights for policy 1, policy_version 1101275 (0.0008) [2023-12-27 13:17:11,359][20381] Updated weights for policy 1, policy_version 1101285 (0.0008) [2023-12-27 13:17:11,425][20381] Updated weights for policy 1, policy_version 1101295 (0.0008) [2023-12-27 13:17:11,646][20347] Updated weights for policy 0, policy_version 1102286 (0.0008) [2023-12-27 13:17:11,705][20347] Updated weights for policy 0, policy_version 1102296 (0.0008) [2023-12-27 13:17:11,778][20347] Updated weights for policy 0, policy_version 1102306 (0.0008) [2023-12-27 13:17:12,168][20381] Updated weights for policy 1, policy_version 1101305 (0.0008) [2023-12-27 13:17:12,224][20381] Updated weights for policy 1, policy_version 1101315 (0.0008) [2023-12-27 13:17:12,287][20381] Updated weights for policy 1, policy_version 1101325 (0.0007) [2023-12-27 13:17:12,354][20381] Updated weights for policy 1, policy_version 1101335 (0.0008) [2023-12-27 13:17:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 564207616. Throughput: 0: 10450.4, 1: 10053.2. Samples: 564196584. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:12,366][19170] Avg episode reward: [(0, '853.010'), (1, '666.560')] [2023-12-27 13:17:12,520][20347] Updated weights for policy 0, policy_version 1102316 (0.0008) [2023-12-27 13:17:12,580][20347] Updated weights for policy 0, policy_version 1102326 (0.0008) [2023-12-27 13:17:12,638][20347] Updated weights for policy 0, policy_version 1102336 (0.0008) [2023-12-27 13:17:13,032][20381] Updated weights for policy 1, policy_version 1101345 (0.0008) [2023-12-27 13:17:13,093][20381] Updated weights for policy 1, policy_version 1101355 (0.0008) [2023-12-27 13:17:13,154][20381] Updated weights for policy 1, policy_version 1101365 (0.0008) [2023-12-27 13:17:13,360][20347] Updated weights for policy 0, policy_version 1102346 (0.0008) [2023-12-27 13:17:13,384][20291] KL-divergence is very high: 113.7255 [2023-12-27 13:17:13,418][20347] Updated weights for policy 0, policy_version 1102356 (0.0008) [2023-12-27 13:17:13,474][20347] Updated weights for policy 0, policy_version 1102366 (0.0009) [2023-12-27 13:17:13,533][20347] Updated weights for policy 0, policy_version 1102376 (0.0010) [2023-12-27 13:17:13,856][20381] Updated weights for policy 1, policy_version 1101375 (0.0007) [2023-12-27 13:17:13,910][20381] Updated weights for policy 1, policy_version 1101385 (0.0007) [2023-12-27 13:17:13,968][20381] Updated weights for policy 1, policy_version 1101395 (0.0007) [2023-12-27 13:17:14,228][20347] Updated weights for policy 0, policy_version 1102386 (0.0010) [2023-12-27 13:17:14,280][20347] Updated weights for policy 0, policy_version 1102396 (0.0010) [2023-12-27 13:17:14,331][20347] Updated weights for policy 0, policy_version 1102406 (0.0010) [2023-12-27 13:17:14,676][20381] Updated weights for policy 1, policy_version 1101405 (0.0007) [2023-12-27 13:17:14,734][20381] Updated weights for policy 1, policy_version 1101415 (0.0007) [2023-12-27 13:17:14,809][20381] Updated weights for policy 1, policy_version 1101425 (0.0008) [2023-12-27 13:17:15,016][20347] Updated weights for policy 0, policy_version 1102416 (0.0007) [2023-12-27 13:17:15,084][20347] Updated weights for policy 0, policy_version 1102426 (0.0008) [2023-12-27 13:17:15,147][20347] Updated weights for policy 0, policy_version 1102436 (0.0011) [2023-12-27 13:17:15,560][20381] Updated weights for policy 1, policy_version 1101435 (0.0007) [2023-12-27 13:17:15,619][20381] Updated weights for policy 1, policy_version 1101445 (0.0005) [2023-12-27 13:17:15,680][20381] Updated weights for policy 1, policy_version 1101455 (0.0007) [2023-12-27 13:17:15,718][20347] Updated weights for policy 0, policy_version 1102446 (0.0008) [2023-12-27 13:17:15,768][20347] Updated weights for policy 0, policy_version 1102456 (0.0006) [2023-12-27 13:17:15,827][20347] Updated weights for policy 0, policy_version 1102466 (0.0005) [2023-12-27 13:17:16,358][20381] Updated weights for policy 1, policy_version 1101465 (0.0007) [2023-12-27 13:17:16,394][20347] Updated weights for policy 0, policy_version 1102476 (0.0008) [2023-12-27 13:17:16,428][20381] Updated weights for policy 1, policy_version 1101475 (0.0007) [2023-12-27 13:17:16,446][20347] Updated weights for policy 0, policy_version 1102486 (0.0006) [2023-12-27 13:17:16,478][20381] Updated weights for policy 1, policy_version 1101485 (0.0006) [2023-12-27 13:17:16,505][20347] Updated weights for policy 0, policy_version 1102496 (0.0006) [2023-12-27 13:17:16,532][20381] Updated weights for policy 1, policy_version 1101495 (0.0005) [2023-12-27 13:17:17,113][20381] Updated weights for policy 1, policy_version 1101505 (0.0008) [2023-12-27 13:17:17,163][20347] Updated weights for policy 0, policy_version 1102506 (0.0005) [2023-12-27 13:17:17,172][20381] Updated weights for policy 1, policy_version 1101515 (0.0008) [2023-12-27 13:17:17,218][20347] Updated weights for policy 0, policy_version 1102516 (0.0005) [2023-12-27 13:17:17,228][20381] Updated weights for policy 1, policy_version 1101525 (0.0010) [2023-12-27 13:17:17,271][20347] Updated weights for policy 0, policy_version 1102526 (0.0006) [2023-12-27 13:17:17,329][20347] Updated weights for policy 0, policy_version 1102536 (0.0007) [2023-12-27 13:17:17,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 564314112. Throughput: 0: 10357.4, 1: 9973.1. Samples: 564318628. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:17,363][19170] Avg episode reward: [(0, '862.880'), (1, '695.910')] [2023-12-27 13:17:17,888][20381] Updated weights for policy 1, policy_version 1101535 (0.0010) [2023-12-27 13:17:17,943][20381] Updated weights for policy 1, policy_version 1101546 (0.0010) [2023-12-27 13:17:17,989][20381] Updated weights for policy 1, policy_version 1101556 (0.0006) [2023-12-27 13:17:18,008][20347] Updated weights for policy 0, policy_version 1102546 (0.0007) [2023-12-27 13:17:18,068][20347] Updated weights for policy 0, policy_version 1102556 (0.0006) [2023-12-27 13:17:18,130][20347] Updated weights for policy 0, policy_version 1102566 (0.0006) [2023-12-27 13:17:18,615][20381] Updated weights for policy 1, policy_version 1101566 (0.0006) [2023-12-27 13:17:18,674][20381] Updated weights for policy 1, policy_version 1101576 (0.0006) [2023-12-27 13:17:18,702][20347] Updated weights for policy 0, policy_version 1102576 (0.0006) [2023-12-27 13:17:18,739][20381] Updated weights for policy 1, policy_version 1101586 (0.0008) [2023-12-27 13:17:18,752][20347] Updated weights for policy 0, policy_version 1102586 (0.0005) [2023-12-27 13:17:18,809][20347] Updated weights for policy 0, policy_version 1102596 (0.0007) [2023-12-27 13:17:19,342][20381] Updated weights for policy 1, policy_version 1101596 (0.0008) [2023-12-27 13:17:19,406][20381] Updated weights for policy 1, policy_version 1101606 (0.0008) [2023-12-27 13:17:19,460][20347] Updated weights for policy 0, policy_version 1102606 (0.0006) [2023-12-27 13:17:19,466][20381] Updated weights for policy 1, policy_version 1101616 (0.0007) [2023-12-27 13:17:19,528][20347] Updated weights for policy 0, policy_version 1102616 (0.0009) [2023-12-27 13:17:19,585][20347] Updated weights for policy 0, policy_version 1102626 (0.0008) [2023-12-27 13:17:20,114][20381] Updated weights for policy 1, policy_version 1101626 (0.0006) [2023-12-27 13:17:20,171][20381] Updated weights for policy 1, policy_version 1101636 (0.0006) [2023-12-27 13:17:20,232][20381] Updated weights for policy 1, policy_version 1101646 (0.0007) [2023-12-27 13:17:20,290][20381] Updated weights for policy 1, policy_version 1101656 (0.0008) [2023-12-27 13:17:20,333][20347] Updated weights for policy 0, policy_version 1102636 (0.0008) [2023-12-27 13:17:20,394][20347] Updated weights for policy 0, policy_version 1102646 (0.0008) [2023-12-27 13:17:20,454][20347] Updated weights for policy 0, policy_version 1102656 (0.0008) [2023-12-27 13:17:21,035][20381] Updated weights for policy 1, policy_version 1101666 (0.0007) [2023-12-27 13:17:21,087][20347] Updated weights for policy 0, policy_version 1102666 (0.0007) [2023-12-27 13:17:21,098][20381] Updated weights for policy 1, policy_version 1101676 (0.0008) [2023-12-27 13:17:21,151][20347] Updated weights for policy 0, policy_version 1102676 (0.0010) [2023-12-27 13:17:21,157][20381] Updated weights for policy 1, policy_version 1101686 (0.0007) [2023-12-27 13:17:21,215][20347] Updated weights for policy 0, policy_version 1102686 (0.0007) [2023-12-27 13:17:21,277][20347] Updated weights for policy 0, policy_version 1102696 (0.0007) [2023-12-27 13:17:21,838][20381] Updated weights for policy 1, policy_version 1101696 (0.0007) [2023-12-27 13:17:21,893][20381] Updated weights for policy 1, policy_version 1101706 (0.0009) [2023-12-27 13:17:21,948][20381] Updated weights for policy 1, policy_version 1101716 (0.0009) [2023-12-27 13:17:21,979][20347] Updated weights for policy 0, policy_version 1102706 (0.0006) [2023-12-27 13:17:22,040][20347] Updated weights for policy 0, policy_version 1102716 (0.0006) [2023-12-27 13:17:22,099][20347] Updated weights for policy 0, policy_version 1102726 (0.0006) [2023-12-27 13:17:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 564412416. Throughput: 0: 10343.9, 1: 10021.9. Samples: 564381864. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:22,363][19170] Avg episode reward: [(0, '882.580'), (1, '666.410')] [2023-12-27 13:17:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001101720_282066944.pth... [2023-12-27 13:17:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001102728_282345472.pth... [2023-12-27 13:17:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001101512_282034176.pth [2023-12-27 13:17:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001100536_281763840.pth [2023-12-27 13:17:22,656][20381] Updated weights for policy 1, policy_version 1101726 (0.0008) [2023-12-27 13:17:22,710][20381] Updated weights for policy 1, policy_version 1101736 (0.0007) [2023-12-27 13:17:22,766][20347] Updated weights for policy 0, policy_version 1102736 (0.0006) [2023-12-27 13:17:22,771][20381] Updated weights for policy 1, policy_version 1101746 (0.0006) [2023-12-27 13:17:22,821][20347] Updated weights for policy 0, policy_version 1102746 (0.0007) [2023-12-27 13:17:22,881][20347] Updated weights for policy 0, policy_version 1102756 (0.0008) [2023-12-27 13:17:23,492][20381] Updated weights for policy 1, policy_version 1101756 (0.0009) [2023-12-27 13:17:23,537][20347] Updated weights for policy 0, policy_version 1102766 (0.0007) [2023-12-27 13:17:23,547][20381] Updated weights for policy 1, policy_version 1101766 (0.0007) [2023-12-27 13:17:23,587][20347] Updated weights for policy 0, policy_version 1102776 (0.0006) [2023-12-27 13:17:23,606][20381] Updated weights for policy 1, policy_version 1101776 (0.0008) [2023-12-27 13:17:23,643][20347] Updated weights for policy 0, policy_version 1102786 (0.0007) [2023-12-27 13:17:24,280][20347] Updated weights for policy 0, policy_version 1102796 (0.0006) [2023-12-27 13:17:24,331][20347] Updated weights for policy 0, policy_version 1102806 (0.0006) [2023-12-27 13:17:24,333][20381] Updated weights for policy 1, policy_version 1101786 (0.0007) [2023-12-27 13:17:24,389][20347] Updated weights for policy 0, policy_version 1102816 (0.0005) [2023-12-27 13:17:24,400][20381] Updated weights for policy 1, policy_version 1101796 (0.0007) [2023-12-27 13:17:24,464][20381] Updated weights for policy 1, policy_version 1101806 (0.0007) [2023-12-27 13:17:24,513][20381] Updated weights for policy 1, policy_version 1101816 (0.0009) [2023-12-27 13:17:24,928][20347] Updated weights for policy 0, policy_version 1102826 (0.0005) [2023-12-27 13:17:24,989][20347] Updated weights for policy 0, policy_version 1102836 (0.0005) [2023-12-27 13:17:25,048][20347] Updated weights for policy 0, policy_version 1102846 (0.0009) [2023-12-27 13:17:25,106][20347] Updated weights for policy 0, policy_version 1102856 (0.0010) [2023-12-27 13:17:25,162][20381] Updated weights for policy 1, policy_version 1101826 (0.0005) [2023-12-27 13:17:25,207][20381] Updated weights for policy 1, policy_version 1101836 (0.0005) [2023-12-27 13:17:25,256][20381] Updated weights for policy 1, policy_version 1101846 (0.0005) [2023-12-27 13:17:25,756][20347] Updated weights for policy 0, policy_version 1102866 (0.0010) [2023-12-27 13:17:25,813][20347] Updated weights for policy 0, policy_version 1102876 (0.0010) [2023-12-27 13:17:25,842][20381] Updated weights for policy 1, policy_version 1101856 (0.0007) [2023-12-27 13:17:25,869][20347] Updated weights for policy 0, policy_version 1102886 (0.0008) [2023-12-27 13:17:25,903][20381] Updated weights for policy 1, policy_version 1101866 (0.0008) [2023-12-27 13:17:25,947][20381] Updated weights for policy 1, policy_version 1101876 (0.0009) [2023-12-27 13:17:26,470][20347] Updated weights for policy 0, policy_version 1102896 (0.0009) [2023-12-27 13:17:26,520][20347] Updated weights for policy 0, policy_version 1102906 (0.0010) [2023-12-27 13:17:26,577][20347] Updated weights for policy 0, policy_version 1102916 (0.0010) [2023-12-27 13:17:26,630][20381] Updated weights for policy 1, policy_version 1101886 (0.0007) [2023-12-27 13:17:26,680][20381] Updated weights for policy 1, policy_version 1101896 (0.0007) [2023-12-27 13:17:26,733][20381] Updated weights for policy 1, policy_version 1101906 (0.0007) [2023-12-27 13:17:27,242][20347] Updated weights for policy 0, policy_version 1102926 (0.0010) [2023-12-27 13:17:27,295][20347] Updated weights for policy 0, policy_version 1102936 (0.0010) [2023-12-27 13:17:27,356][20347] Updated weights for policy 0, policy_version 1102946 (0.0010) [2023-12-27 13:17:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 564510720. Throughput: 0: 10363.6, 1: 9989.6. Samples: 564505480. Policy #0 lag: (min: 31.0, avg: 35.0, max: 63.0) [2023-12-27 13:17:27,363][19170] Avg episode reward: [(0, '872.830'), (1, '725.340')] [2023-12-27 13:17:27,408][20381] Updated weights for policy 1, policy_version 1101916 (0.0007) [2023-12-27 13:17:27,452][20381] Updated weights for policy 1, policy_version 1101926 (0.0007) [2023-12-27 13:17:27,496][20381] Updated weights for policy 1, policy_version 1101936 (0.0007) [2023-12-27 13:17:28,021][20347] Updated weights for policy 0, policy_version 1102956 (0.0009) [2023-12-27 13:17:28,080][20347] Updated weights for policy 0, policy_version 1102966 (0.0007) [2023-12-27 13:17:28,139][20347] Updated weights for policy 0, policy_version 1102976 (0.0008) [2023-12-27 13:17:28,211][20381] Updated weights for policy 1, policy_version 1101946 (0.0008) [2023-12-27 13:17:28,259][20381] Updated weights for policy 1, policy_version 1101956 (0.0008) [2023-12-27 13:17:28,308][20381] Updated weights for policy 1, policy_version 1101966 (0.0007) [2023-12-27 13:17:28,369][20381] Updated weights for policy 1, policy_version 1101976 (0.0008) [2023-12-27 13:17:28,739][20347] Updated weights for policy 0, policy_version 1102986 (0.0005) [2023-12-27 13:17:28,796][20347] Updated weights for policy 0, policy_version 1102996 (0.0005) [2023-12-27 13:17:28,850][20347] Updated weights for policy 0, policy_version 1103006 (0.0005) [2023-12-27 13:17:28,910][20347] Updated weights for policy 0, policy_version 1103016 (0.0008) [2023-12-27 13:17:29,119][20381] Updated weights for policy 1, policy_version 1101986 (0.0008) [2023-12-27 13:17:29,170][20381] Updated weights for policy 1, policy_version 1101996 (0.0007) [2023-12-27 13:17:29,227][20381] Updated weights for policy 1, policy_version 1102006 (0.0008) [2023-12-27 13:17:29,535][20347] Updated weights for policy 0, policy_version 1103026 (0.0009) [2023-12-27 13:17:29,583][20347] Updated weights for policy 0, policy_version 1103036 (0.0010) [2023-12-27 13:17:29,638][20347] Updated weights for policy 0, policy_version 1103046 (0.0010) [2023-12-27 13:17:29,976][20381] Updated weights for policy 1, policy_version 1102016 (0.0008) [2023-12-27 13:17:30,038][20381] Updated weights for policy 1, policy_version 1102026 (0.0008) [2023-12-27 13:17:30,092][20381] Updated weights for policy 1, policy_version 1102036 (0.0007) [2023-12-27 13:17:30,296][20347] Updated weights for policy 0, policy_version 1103056 (0.0008) [2023-12-27 13:17:30,351][20347] Updated weights for policy 0, policy_version 1103066 (0.0008) [2023-12-27 13:17:30,409][20347] Updated weights for policy 0, policy_version 1103076 (0.0007) [2023-12-27 13:17:30,801][20381] Updated weights for policy 1, policy_version 1102046 (0.0008) [2023-12-27 13:17:30,853][20381] Updated weights for policy 1, policy_version 1102056 (0.0007) [2023-12-27 13:17:30,902][20381] Updated weights for policy 1, policy_version 1102066 (0.0007) [2023-12-27 13:17:31,069][20347] Updated weights for policy 0, policy_version 1103086 (0.0008) [2023-12-27 13:17:31,128][20347] Updated weights for policy 0, policy_version 1103096 (0.0008) [2023-12-27 13:17:31,193][20347] Updated weights for policy 0, policy_version 1103106 (0.0008) [2023-12-27 13:17:31,580][20381] Updated weights for policy 1, policy_version 1102076 (0.0008) [2023-12-27 13:17:31,664][20381] Updated weights for policy 1, policy_version 1102086 (0.0008) [2023-12-27 13:17:31,731][20381] Updated weights for policy 1, policy_version 1102096 (0.0008) [2023-12-27 13:17:31,910][20347] Updated weights for policy 0, policy_version 1103116 (0.0008) [2023-12-27 13:17:31,966][20347] Updated weights for policy 0, policy_version 1103126 (0.0008) [2023-12-27 13:17:32,022][20347] Updated weights for policy 0, policy_version 1103136 (0.0007) [2023-12-27 13:17:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 564617216. Throughput: 0: 10383.7, 1: 10051.9. Samples: 564627692. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:32,363][19170] Avg episode reward: [(0, '794.490'), (1, '784.340')] [2023-12-27 13:17:32,455][20381] Updated weights for policy 1, policy_version 1102106 (0.0008) [2023-12-27 13:17:32,517][20381] Updated weights for policy 1, policy_version 1102116 (0.0010) [2023-12-27 13:17:32,579][20381] Updated weights for policy 1, policy_version 1102126 (0.0010) [2023-12-27 13:17:32,636][20381] Updated weights for policy 1, policy_version 1102136 (0.0008) [2023-12-27 13:17:32,789][20347] Updated weights for policy 0, policy_version 1103146 (0.0008) [2023-12-27 13:17:32,839][20347] Updated weights for policy 0, policy_version 1103156 (0.0008) [2023-12-27 13:17:32,888][20347] Updated weights for policy 0, policy_version 1103166 (0.0007) [2023-12-27 13:17:32,942][20347] Updated weights for policy 0, policy_version 1103176 (0.0006) [2023-12-27 13:17:33,342][20381] Updated weights for policy 1, policy_version 1102146 (0.0009) [2023-12-27 13:17:33,404][20381] Updated weights for policy 1, policy_version 1102156 (0.0010) [2023-12-27 13:17:33,462][20381] Updated weights for policy 1, policy_version 1102166 (0.0010) [2023-12-27 13:17:33,597][20347] Updated weights for policy 0, policy_version 1103186 (0.0005) [2023-12-27 13:17:33,642][20347] Updated weights for policy 0, policy_version 1103196 (0.0005) [2023-12-27 13:17:33,686][20347] Updated weights for policy 0, policy_version 1103206 (0.0006) [2023-12-27 13:17:34,123][20381] Updated weights for policy 1, policy_version 1102176 (0.0009) [2023-12-27 13:17:34,158][20292] KL-divergence is very high: 105.6952 [2023-12-27 13:17:34,193][20381] Updated weights for policy 1, policy_version 1102186 (0.0009) [2023-12-27 13:17:34,251][20381] Updated weights for policy 1, policy_version 1102196 (0.0010) [2023-12-27 13:17:34,326][20347] Updated weights for policy 0, policy_version 1103216 (0.0008) [2023-12-27 13:17:34,381][20347] Updated weights for policy 0, policy_version 1103226 (0.0008) [2023-12-27 13:17:34,440][20347] Updated weights for policy 0, policy_version 1103236 (0.0007) [2023-12-27 13:17:34,911][20381] Updated weights for policy 1, policy_version 1102206 (0.0010) [2023-12-27 13:17:34,974][20381] Updated weights for policy 1, policy_version 1102216 (0.0010) [2023-12-27 13:17:35,012][20347] Updated weights for policy 0, policy_version 1103246 (0.0008) [2023-12-27 13:17:35,027][20381] Updated weights for policy 1, policy_version 1102226 (0.0010) [2023-12-27 13:17:35,071][20347] Updated weights for policy 0, policy_version 1103256 (0.0007) [2023-12-27 13:17:35,121][20347] Updated weights for policy 0, policy_version 1103266 (0.0007) [2023-12-27 13:17:35,640][20381] Updated weights for policy 1, policy_version 1102236 (0.0011) [2023-12-27 13:17:35,703][20381] Updated weights for policy 1, policy_version 1102246 (0.0010) [2023-12-27 13:17:35,760][20381] Updated weights for policy 1, policy_version 1102256 (0.0010) [2023-12-27 13:17:35,844][20347] Updated weights for policy 0, policy_version 1103276 (0.0007) [2023-12-27 13:17:35,901][20347] Updated weights for policy 0, policy_version 1103286 (0.0007) [2023-12-27 13:17:35,959][20347] Updated weights for policy 0, policy_version 1103296 (0.0006) [2023-12-27 13:17:36,386][20381] Updated weights for policy 1, policy_version 1102266 (0.0010) [2023-12-27 13:17:36,451][20381] Updated weights for policy 1, policy_version 1102276 (0.0008) [2023-12-27 13:17:36,512][20381] Updated weights for policy 1, policy_version 1102286 (0.0008) [2023-12-27 13:17:36,571][20381] Updated weights for policy 1, policy_version 1102296 (0.0008) [2023-12-27 13:17:36,662][20347] Updated weights for policy 0, policy_version 1103306 (0.0006) [2023-12-27 13:17:36,721][20347] Updated weights for policy 0, policy_version 1103316 (0.0007) [2023-12-27 13:17:36,782][20347] Updated weights for policy 0, policy_version 1103326 (0.0005) [2023-12-27 13:17:36,843][20347] Updated weights for policy 0, policy_version 1103336 (0.0005) [2023-12-27 13:17:37,251][20381] Updated weights for policy 1, policy_version 1102306 (0.0008) [2023-12-27 13:17:37,281][20292] KL-divergence is very high: 202.1071 [2023-12-27 13:17:37,312][20381] Updated weights for policy 1, policy_version 1102316 (0.0008) [2023-12-27 13:17:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 564715520. Throughput: 0: 10423.6, 1: 10061.2. Samples: 564690452. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:37,363][19170] Avg episode reward: [(0, '814.410'), (1, '764.860')] [2023-12-27 13:17:37,369][20381] Updated weights for policy 1, policy_version 1102326 (0.0007) [2023-12-27 13:17:37,378][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001102328_282222592.pth... [2023-12-27 13:17:37,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001101144_281919488.pth [2023-12-27 13:17:37,391][20347] Updated weights for policy 0, policy_version 1103346 (0.0006) [2023-12-27 13:17:37,450][20347] Updated weights for policy 0, policy_version 1103356 (0.0008) [2023-12-27 13:17:37,503][20347] Updated weights for policy 0, policy_version 1103366 (0.0008) [2023-12-27 13:17:37,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001103368_282509312.pth... [2023-12-27 13:17:37,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001102120_282189824.pth [2023-12-27 13:17:37,979][20381] Updated weights for policy 1, policy_version 1102336 (0.0007) [2023-12-27 13:17:38,041][20381] Updated weights for policy 1, policy_version 1102346 (0.0008) [2023-12-27 13:17:38,108][20381] Updated weights for policy 1, policy_version 1102356 (0.0007) [2023-12-27 13:17:38,152][20347] Updated weights for policy 0, policy_version 1103376 (0.0006) [2023-12-27 13:17:38,197][20347] Updated weights for policy 0, policy_version 1103386 (0.0005) [2023-12-27 13:17:38,249][20347] Updated weights for policy 0, policy_version 1103396 (0.0005) [2023-12-27 13:17:38,769][20381] Updated weights for policy 1, policy_version 1102366 (0.0007) [2023-12-27 13:17:38,830][20381] Updated weights for policy 1, policy_version 1102376 (0.0009) [2023-12-27 13:17:38,845][20347] Updated weights for policy 0, policy_version 1103406 (0.0008) [2023-12-27 13:17:38,887][20381] Updated weights for policy 1, policy_version 1102386 (0.0009) [2023-12-27 13:17:38,898][20347] Updated weights for policy 0, policy_version 1103416 (0.0010) [2023-12-27 13:17:38,953][20347] Updated weights for policy 0, policy_version 1103426 (0.0010) [2023-12-27 13:17:39,586][20381] Updated weights for policy 1, policy_version 1102396 (0.0008) [2023-12-27 13:17:39,649][20381] Updated weights for policy 1, policy_version 1102406 (0.0008) [2023-12-27 13:17:39,686][20347] Updated weights for policy 0, policy_version 1103436 (0.0009) [2023-12-27 13:17:39,707][20381] Updated weights for policy 1, policy_version 1102416 (0.0007) [2023-12-27 13:17:39,747][20347] Updated weights for policy 0, policy_version 1103446 (0.0006) [2023-12-27 13:17:39,806][20347] Updated weights for policy 0, policy_version 1103456 (0.0007) [2023-12-27 13:17:40,379][20381] Updated weights for policy 1, policy_version 1102426 (0.0006) [2023-12-27 13:17:40,446][20381] Updated weights for policy 1, policy_version 1102436 (0.0007) [2023-12-27 13:17:40,510][20381] Updated weights for policy 1, policy_version 1102446 (0.0008) [2023-12-27 13:17:40,573][20381] Updated weights for policy 1, policy_version 1102456 (0.0008) [2023-12-27 13:17:40,589][20347] Updated weights for policy 0, policy_version 1103466 (0.0007) [2023-12-27 13:17:40,639][20347] Updated weights for policy 0, policy_version 1103476 (0.0008) [2023-12-27 13:17:40,689][20347] Updated weights for policy 0, policy_version 1103486 (0.0008) [2023-12-27 13:17:40,718][20291] KL-divergence is very high: 168.2663 [2023-12-27 13:17:40,750][20347] Updated weights for policy 0, policy_version 1103496 (0.0008) [2023-12-27 13:17:41,288][20381] Updated weights for policy 1, policy_version 1102466 (0.0008) [2023-12-27 13:17:41,353][20381] Updated weights for policy 1, policy_version 1102476 (0.0007) [2023-12-27 13:17:41,417][20381] Updated weights for policy 1, policy_version 1102486 (0.0008) [2023-12-27 13:17:41,451][20347] Updated weights for policy 0, policy_version 1103506 (0.0006) [2023-12-27 13:17:41,513][20347] Updated weights for policy 0, policy_version 1103516 (0.0008) [2023-12-27 13:17:41,570][20347] Updated weights for policy 0, policy_version 1103526 (0.0005) [2023-12-27 13:17:42,158][20381] Updated weights for policy 1, policy_version 1102496 (0.0007) [2023-12-27 13:17:42,202][20347] Updated weights for policy 0, policy_version 1103536 (0.0005) [2023-12-27 13:17:42,215][20381] Updated weights for policy 1, policy_version 1102506 (0.0006) [2023-12-27 13:17:42,264][20347] Updated weights for policy 0, policy_version 1103546 (0.0006) [2023-12-27 13:17:42,278][20381] Updated weights for policy 1, policy_version 1102516 (0.0006) [2023-12-27 13:17:42,333][20347] Updated weights for policy 0, policy_version 1103556 (0.0007) [2023-12-27 13:17:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 564830208. Throughput: 0: 10407.7, 1: 10054.7. Samples: 564814496. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:42,363][19170] Avg episode reward: [(0, '824.670'), (1, '833.290')] [2023-12-27 13:17:42,938][20381] Updated weights for policy 1, policy_version 1102526 (0.0007) [2023-12-27 13:17:42,993][20381] Updated weights for policy 1, policy_version 1102536 (0.0008) [2023-12-27 13:17:43,034][20347] Updated weights for policy 0, policy_version 1103566 (0.0008) [2023-12-27 13:17:43,048][20381] Updated weights for policy 1, policy_version 1102546 (0.0008) [2023-12-27 13:17:43,085][20347] Updated weights for policy 0, policy_version 1103576 (0.0007) [2023-12-27 13:17:43,145][20347] Updated weights for policy 0, policy_version 1103586 (0.0008) [2023-12-27 13:17:43,760][20381] Updated weights for policy 1, policy_version 1102556 (0.0008) [2023-12-27 13:17:43,818][20381] Updated weights for policy 1, policy_version 1102566 (0.0009) [2023-12-27 13:17:43,827][20347] Updated weights for policy 0, policy_version 1103596 (0.0007) [2023-12-27 13:17:43,863][20381] Updated weights for policy 1, policy_version 1102576 (0.0007) [2023-12-27 13:17:43,875][20347] Updated weights for policy 0, policy_version 1103606 (0.0005) [2023-12-27 13:17:43,924][20347] Updated weights for policy 0, policy_version 1103616 (0.0005) [2023-12-27 13:17:44,482][20347] Updated weights for policy 0, policy_version 1103626 (0.0005) [2023-12-27 13:17:44,525][20347] Updated weights for policy 0, policy_version 1103636 (0.0005) [2023-12-27 13:17:44,583][20381] Updated weights for policy 1, policy_version 1102586 (0.0008) [2023-12-27 13:17:44,589][20347] Updated weights for policy 0, policy_version 1103646 (0.0006) [2023-12-27 13:17:44,634][20381] Updated weights for policy 1, policy_version 1102596 (0.0007) [2023-12-27 13:17:44,644][20347] Updated weights for policy 0, policy_version 1103656 (0.0009) [2023-12-27 13:17:44,690][20381] Updated weights for policy 1, policy_version 1102606 (0.0008) [2023-12-27 13:17:44,736][20381] Updated weights for policy 1, policy_version 1102616 (0.0010) [2023-12-27 13:17:45,357][20347] Updated weights for policy 0, policy_version 1103666 (0.0007) [2023-12-27 13:17:45,417][20347] Updated weights for policy 0, policy_version 1103676 (0.0007) [2023-12-27 13:17:45,456][20381] Updated weights for policy 1, policy_version 1102626 (0.0011) [2023-12-27 13:17:45,476][20347] Updated weights for policy 0, policy_version 1103686 (0.0005) [2023-12-27 13:17:45,504][20381] Updated weights for policy 1, policy_version 1102636 (0.0010) [2023-12-27 13:17:45,553][20381] Updated weights for policy 1, policy_version 1102646 (0.0010) [2023-12-27 13:17:46,156][20347] Updated weights for policy 0, policy_version 1103696 (0.0005) [2023-12-27 13:17:46,204][20347] Updated weights for policy 0, policy_version 1103706 (0.0005) [2023-12-27 13:17:46,256][20381] Updated weights for policy 1, policy_version 1102656 (0.0008) [2023-12-27 13:17:46,261][20347] Updated weights for policy 0, policy_version 1103716 (0.0005) [2023-12-27 13:17:46,319][20381] Updated weights for policy 1, policy_version 1102666 (0.0005) [2023-12-27 13:17:46,381][20381] Updated weights for policy 1, policy_version 1102676 (0.0005) [2023-12-27 13:17:46,919][20347] Updated weights for policy 0, policy_version 1103726 (0.0005) [2023-12-27 13:17:46,953][20381] Updated weights for policy 1, policy_version 1102686 (0.0008) [2023-12-27 13:17:46,974][20347] Updated weights for policy 0, policy_version 1103736 (0.0005) [2023-12-27 13:17:46,998][20381] Updated weights for policy 1, policy_version 1102696 (0.0010) [2023-12-27 13:17:47,032][20347] Updated weights for policy 0, policy_version 1103746 (0.0005) [2023-12-27 13:17:47,057][20381] Updated weights for policy 1, policy_version 1102706 (0.0010) [2023-12-27 13:17:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 564928512. Throughput: 0: 10467.2, 1: 10075.4. Samples: 564937720. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:47,363][19170] Avg episode reward: [(0, '834.370'), (1, '813.590')] [2023-12-27 13:17:47,655][20347] Updated weights for policy 0, policy_version 1103756 (0.0006) [2023-12-27 13:17:47,671][20381] Updated weights for policy 1, policy_version 1102716 (0.0009) [2023-12-27 13:17:47,708][20347] Updated weights for policy 0, policy_version 1103766 (0.0008) [2023-12-27 13:17:47,727][20381] Updated weights for policy 1, policy_version 1102726 (0.0005) [2023-12-27 13:17:47,765][20347] Updated weights for policy 0, policy_version 1103776 (0.0007) [2023-12-27 13:17:47,785][20381] Updated weights for policy 1, policy_version 1102736 (0.0006) [2023-12-27 13:17:48,334][20381] Updated weights for policy 1, policy_version 1102746 (0.0006) [2023-12-27 13:17:48,402][20381] Updated weights for policy 1, policy_version 1102756 (0.0006) [2023-12-27 13:17:48,466][20381] Updated weights for policy 1, policy_version 1102766 (0.0008) [2023-12-27 13:17:48,500][20347] Updated weights for policy 0, policy_version 1103786 (0.0006) [2023-12-27 13:17:48,517][20381] Updated weights for policy 1, policy_version 1102776 (0.0005) [2023-12-27 13:17:48,557][20347] Updated weights for policy 0, policy_version 1103796 (0.0008) [2023-12-27 13:17:48,612][20347] Updated weights for policy 0, policy_version 1103806 (0.0008) [2023-12-27 13:17:48,674][20347] Updated weights for policy 0, policy_version 1103816 (0.0008) [2023-12-27 13:17:49,096][20381] Updated weights for policy 1, policy_version 1102786 (0.0007) [2023-12-27 13:17:49,146][20381] Updated weights for policy 1, policy_version 1102796 (0.0008) [2023-12-27 13:17:49,188][20381] Updated weights for policy 1, policy_version 1102806 (0.0007) [2023-12-27 13:17:49,448][20347] Updated weights for policy 0, policy_version 1103826 (0.0008) [2023-12-27 13:17:49,503][20347] Updated weights for policy 0, policy_version 1103836 (0.0006) [2023-12-27 13:17:49,565][20347] Updated weights for policy 0, policy_version 1103846 (0.0005) [2023-12-27 13:17:49,969][20381] Updated weights for policy 1, policy_version 1102816 (0.0008) [2023-12-27 13:17:50,031][20381] Updated weights for policy 1, policy_version 1102826 (0.0008) [2023-12-27 13:17:50,097][20381] Updated weights for policy 1, policy_version 1102836 (0.0007) [2023-12-27 13:17:50,184][20347] Updated weights for policy 0, policy_version 1103856 (0.0006) [2023-12-27 13:17:50,234][20347] Updated weights for policy 0, policy_version 1103866 (0.0008) [2023-12-27 13:17:50,288][20347] Updated weights for policy 0, policy_version 1103876 (0.0007) [2023-12-27 13:17:50,786][20381] Updated weights for policy 1, policy_version 1102846 (0.0006) [2023-12-27 13:17:50,835][20381] Updated weights for policy 1, policy_version 1102856 (0.0007) [2023-12-27 13:17:50,889][20381] Updated weights for policy 1, policy_version 1102866 (0.0007) [2023-12-27 13:17:50,959][20347] Updated weights for policy 0, policy_version 1103886 (0.0007) [2023-12-27 13:17:51,007][20347] Updated weights for policy 0, policy_version 1103896 (0.0007) [2023-12-27 13:17:51,074][20347] Updated weights for policy 0, policy_version 1103906 (0.0009) [2023-12-27 13:17:51,631][20381] Updated weights for policy 1, policy_version 1102876 (0.0007) [2023-12-27 13:17:51,693][20381] Updated weights for policy 1, policy_version 1102886 (0.0006) [2023-12-27 13:17:51,757][20381] Updated weights for policy 1, policy_version 1102896 (0.0008) [2023-12-27 13:17:51,806][20347] Updated weights for policy 0, policy_version 1103916 (0.0009) [2023-12-27 13:17:51,862][20347] Updated weights for policy 0, policy_version 1103926 (0.0008) [2023-12-27 13:17:51,921][20347] Updated weights for policy 0, policy_version 1103936 (0.0008) [2023-12-27 13:17:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 565026816. Throughput: 0: 10448.7, 1: 10107.5. Samples: 564999976. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:52,363][19170] Avg episode reward: [(0, '863.600'), (1, '715.660')] [2023-12-27 13:17:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001103944_282656768.pth... [2023-12-27 13:17:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001102904_282370048.pth... [2023-12-27 13:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001102728_282345472.pth [2023-12-27 13:17:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001101720_282066944.pth [2023-12-27 13:17:52,425][20381] Updated weights for policy 1, policy_version 1102906 (0.0008) [2023-12-27 13:17:52,475][20381] Updated weights for policy 1, policy_version 1102916 (0.0008) [2023-12-27 13:17:52,522][20381] Updated weights for policy 1, policy_version 1102926 (0.0008) [2023-12-27 13:17:52,574][20381] Updated weights for policy 1, policy_version 1102936 (0.0008) [2023-12-27 13:17:52,629][20347] Updated weights for policy 0, policy_version 1103946 (0.0008) [2023-12-27 13:17:52,672][20291] KL-divergence is very high: 164.3935 [2023-12-27 13:17:52,689][20347] Updated weights for policy 0, policy_version 1103956 (0.0007) [2023-12-27 13:17:52,696][20291] KL-divergence is very high: 159.0699 [2023-12-27 13:17:52,722][20291] KL-divergence is very high: 160.0523 [2023-12-27 13:17:52,750][20347] Updated weights for policy 0, policy_version 1103966 (0.0009) [2023-12-27 13:17:52,787][20291] KL-divergence is very high: 230.2569 [2023-12-27 13:17:52,803][20347] Updated weights for policy 0, policy_version 1103976 (0.0008) [2023-12-27 13:17:53,274][20381] Updated weights for policy 1, policy_version 1102946 (0.0010) [2023-12-27 13:17:53,338][20381] Updated weights for policy 1, policy_version 1102956 (0.0008) [2023-12-27 13:17:53,396][20381] Updated weights for policy 1, policy_version 1102966 (0.0008) [2023-12-27 13:17:53,435][20347] Updated weights for policy 0, policy_version 1103986 (0.0008) [2023-12-27 13:17:53,487][20347] Updated weights for policy 0, policy_version 1103996 (0.0007) [2023-12-27 13:17:53,535][20347] Updated weights for policy 0, policy_version 1104006 (0.0008) [2023-12-27 13:17:54,102][20381] Updated weights for policy 1, policy_version 1102976 (0.0007) [2023-12-27 13:17:54,160][20381] Updated weights for policy 1, policy_version 1102986 (0.0006) [2023-12-27 13:17:54,209][20381] Updated weights for policy 1, policy_version 1102996 (0.0007) [2023-12-27 13:17:54,247][20347] Updated weights for policy 0, policy_version 1104016 (0.0007) [2023-12-27 13:17:54,285][20291] KL-divergence is very high: 256.3714 [2023-12-27 13:17:54,302][20347] Updated weights for policy 0, policy_version 1104026 (0.0007) [2023-12-27 13:17:54,355][20347] Updated weights for policy 0, policy_version 1104036 (0.0007) [2023-12-27 13:17:54,916][20381] Updated weights for policy 1, policy_version 1103006 (0.0007) [2023-12-27 13:17:54,988][20381] Updated weights for policy 1, policy_version 1103016 (0.0008) [2023-12-27 13:17:54,994][20347] Updated weights for policy 0, policy_version 1104046 (0.0007) [2023-12-27 13:17:55,047][20347] Updated weights for policy 0, policy_version 1104056 (0.0005) [2023-12-27 13:17:55,050][20381] Updated weights for policy 1, policy_version 1103026 (0.0007) [2023-12-27 13:17:55,100][20347] Updated weights for policy 0, policy_version 1104066 (0.0006) [2023-12-27 13:17:55,725][20347] Updated weights for policy 0, policy_version 1104076 (0.0007) [2023-12-27 13:17:55,784][20381] Updated weights for policy 1, policy_version 1103036 (0.0006) [2023-12-27 13:17:55,786][20347] Updated weights for policy 0, policy_version 1104086 (0.0007) [2023-12-27 13:17:55,835][20381] Updated weights for policy 1, policy_version 1103046 (0.0006) [2023-12-27 13:17:55,836][20347] Updated weights for policy 0, policy_version 1104096 (0.0006) [2023-12-27 13:17:55,899][20381] Updated weights for policy 1, policy_version 1103056 (0.0006) [2023-12-27 13:17:56,449][20347] Updated weights for policy 0, policy_version 1104106 (0.0006) [2023-12-27 13:17:56,499][20347] Updated weights for policy 0, policy_version 1104116 (0.0009) [2023-12-27 13:17:56,537][20291] KL-divergence is very high: 101.7226 [2023-12-27 13:17:56,547][20347] Updated weights for policy 0, policy_version 1104126 (0.0006) [2023-12-27 13:17:56,598][20347] Updated weights for policy 0, policy_version 1104136 (0.0005) [2023-12-27 13:17:56,608][20381] Updated weights for policy 1, policy_version 1103066 (0.0008) [2023-12-27 13:17:56,659][20381] Updated weights for policy 1, policy_version 1103076 (0.0008) [2023-12-27 13:17:56,707][20381] Updated weights for policy 1, policy_version 1103086 (0.0007) [2023-12-27 13:17:56,757][20381] Updated weights for policy 1, policy_version 1103096 (0.0007) [2023-12-27 13:17:57,226][20347] Updated weights for policy 0, policy_version 1104146 (0.0005) [2023-12-27 13:17:57,284][20347] Updated weights for policy 0, policy_version 1104156 (0.0005) [2023-12-27 13:17:57,337][20347] Updated weights for policy 0, policy_version 1104166 (0.0008) [2023-12-27 13:17:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 565133312. Throughput: 0: 10416.7, 1: 10127.5. Samples: 565121072. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:17:57,363][19170] Avg episode reward: [(0, '824.370'), (1, '686.070')] [2023-12-27 13:17:57,490][20381] Updated weights for policy 1, policy_version 1103106 (0.0005) [2023-12-27 13:17:57,539][20381] Updated weights for policy 1, policy_version 1103116 (0.0005) [2023-12-27 13:17:57,596][20381] Updated weights for policy 1, policy_version 1103126 (0.0005) [2023-12-27 13:17:57,971][20347] Updated weights for policy 0, policy_version 1104176 (0.0010) [2023-12-27 13:17:58,016][20347] Updated weights for policy 0, policy_version 1104186 (0.0010) [2023-12-27 13:17:58,063][20347] Updated weights for policy 0, policy_version 1104196 (0.0010) [2023-12-27 13:17:58,220][20381] Updated weights for policy 1, policy_version 1103136 (0.0007) [2023-12-27 13:17:58,272][20381] Updated weights for policy 1, policy_version 1103146 (0.0007) [2023-12-27 13:17:58,330][20381] Updated weights for policy 1, policy_version 1103156 (0.0008) [2023-12-27 13:17:58,894][20347] Updated weights for policy 0, policy_version 1104206 (0.0010) [2023-12-27 13:17:58,954][20347] Updated weights for policy 0, policy_version 1104216 (0.0011) [2023-12-27 13:17:59,013][20347] Updated weights for policy 0, policy_version 1104226 (0.0011) [2023-12-27 13:17:59,108][20381] Updated weights for policy 1, policy_version 1103166 (0.0007) [2023-12-27 13:17:59,168][20381] Updated weights for policy 1, policy_version 1103176 (0.0006) [2023-12-27 13:17:59,234][20381] Updated weights for policy 1, policy_version 1103186 (0.0007) [2023-12-27 13:17:59,678][20347] Updated weights for policy 0, policy_version 1104236 (0.0008) [2023-12-27 13:17:59,737][20347] Updated weights for policy 0, policy_version 1104246 (0.0005) [2023-12-27 13:17:59,801][20347] Updated weights for policy 0, policy_version 1104256 (0.0005) [2023-12-27 13:17:59,918][20381] Updated weights for policy 1, policy_version 1103196 (0.0008) [2023-12-27 13:17:59,983][20381] Updated weights for policy 1, policy_version 1103206 (0.0007) [2023-12-27 13:18:00,038][20381] Updated weights for policy 1, policy_version 1103216 (0.0008) [2023-12-27 13:18:00,416][20347] Updated weights for policy 0, policy_version 1104266 (0.0007) [2023-12-27 13:18:00,470][20347] Updated weights for policy 0, policy_version 1104276 (0.0007) [2023-12-27 13:18:00,522][20347] Updated weights for policy 0, policy_version 1104286 (0.0008) [2023-12-27 13:18:00,580][20347] Updated weights for policy 0, policy_version 1104296 (0.0008) [2023-12-27 13:18:00,646][20381] Updated weights for policy 1, policy_version 1103226 (0.0008) [2023-12-27 13:18:00,702][20381] Updated weights for policy 1, policy_version 1103236 (0.0008) [2023-12-27 13:18:00,757][20381] Updated weights for policy 1, policy_version 1103246 (0.0007) [2023-12-27 13:18:00,811][20381] Updated weights for policy 1, policy_version 1103256 (0.0007) [2023-12-27 13:18:01,250][20347] Updated weights for policy 0, policy_version 1104306 (0.0008) [2023-12-27 13:18:01,300][20347] Updated weights for policy 0, policy_version 1104316 (0.0007) [2023-12-27 13:18:01,355][20347] Updated weights for policy 0, policy_version 1104326 (0.0008) [2023-12-27 13:18:01,565][20381] Updated weights for policy 1, policy_version 1103266 (0.0007) [2023-12-27 13:18:01,627][20381] Updated weights for policy 1, policy_version 1103276 (0.0007) [2023-12-27 13:18:01,684][20381] Updated weights for policy 1, policy_version 1103286 (0.0006) [2023-12-27 13:18:02,090][20347] Updated weights for policy 0, policy_version 1104336 (0.0006) [2023-12-27 13:18:02,091][20291] KL-divergence is very high: 106.3396 [2023-12-27 13:18:02,097][20291] KL-divergence is very high: 153.1620 [2023-12-27 13:18:02,112][20291] KL-divergence is very high: 103.3549 [2023-12-27 13:18:02,145][20347] Updated weights for policy 0, policy_version 1104346 (0.0005) [2023-12-27 13:18:02,196][20347] Updated weights for policy 0, policy_version 1104356 (0.0007) [2023-12-27 13:18:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 565231616. Throughput: 0: 10400.5, 1: 10125.8. Samples: 565242312. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:02,363][19170] Avg episode reward: [(0, '863.770'), (1, '754.910')] [2023-12-27 13:18:02,394][20381] Updated weights for policy 1, policy_version 1103296 (0.0009) [2023-12-27 13:18:02,452][20381] Updated weights for policy 1, policy_version 1103306 (0.0007) [2023-12-27 13:18:02,507][20381] Updated weights for policy 1, policy_version 1103316 (0.0009) [2023-12-27 13:18:02,879][20347] Updated weights for policy 0, policy_version 1104366 (0.0007) [2023-12-27 13:18:02,939][20347] Updated weights for policy 0, policy_version 1104376 (0.0008) [2023-12-27 13:18:02,990][20347] Updated weights for policy 0, policy_version 1104386 (0.0007) [2023-12-27 13:18:03,214][20381] Updated weights for policy 1, policy_version 1103326 (0.0008) [2023-12-27 13:18:03,265][20381] Updated weights for policy 1, policy_version 1103336 (0.0008) [2023-12-27 13:18:03,327][20381] Updated weights for policy 1, policy_version 1103346 (0.0006) [2023-12-27 13:18:03,679][20347] Updated weights for policy 0, policy_version 1104396 (0.0008) [2023-12-27 13:18:03,724][20347] Updated weights for policy 0, policy_version 1104406 (0.0008) [2023-12-27 13:18:03,777][20347] Updated weights for policy 0, policy_version 1104416 (0.0006) [2023-12-27 13:18:03,916][20381] Updated weights for policy 1, policy_version 1103356 (0.0005) [2023-12-27 13:18:03,975][20381] Updated weights for policy 1, policy_version 1103366 (0.0006) [2023-12-27 13:18:04,031][20381] Updated weights for policy 1, policy_version 1103376 (0.0008) [2023-12-27 13:18:04,530][20347] Updated weights for policy 0, policy_version 1104426 (0.0006) [2023-12-27 13:18:04,589][20347] Updated weights for policy 0, policy_version 1104436 (0.0008) [2023-12-27 13:18:04,649][20347] Updated weights for policy 0, policy_version 1104446 (0.0006) [2023-12-27 13:18:04,702][20347] Updated weights for policy 0, policy_version 1104456 (0.0007) [2023-12-27 13:18:04,719][20381] Updated weights for policy 1, policy_version 1103386 (0.0007) [2023-12-27 13:18:04,777][20381] Updated weights for policy 1, policy_version 1103396 (0.0008) [2023-12-27 13:18:04,834][20381] Updated weights for policy 1, policy_version 1103406 (0.0008) [2023-12-27 13:18:04,901][20381] Updated weights for policy 1, policy_version 1103416 (0.0007) [2023-12-27 13:18:05,385][20347] Updated weights for policy 0, policy_version 1104466 (0.0007) [2023-12-27 13:18:05,439][20347] Updated weights for policy 0, policy_version 1104476 (0.0007) [2023-12-27 13:18:05,493][20347] Updated weights for policy 0, policy_version 1104486 (0.0007) [2023-12-27 13:18:05,585][20381] Updated weights for policy 1, policy_version 1103426 (0.0007) [2023-12-27 13:18:05,650][20381] Updated weights for policy 1, policy_version 1103436 (0.0008) [2023-12-27 13:18:05,711][20381] Updated weights for policy 1, policy_version 1103446 (0.0008) [2023-12-27 13:18:06,116][20347] Updated weights for policy 0, policy_version 1104496 (0.0007) [2023-12-27 13:18:06,175][20347] Updated weights for policy 0, policy_version 1104506 (0.0008) [2023-12-27 13:18:06,238][20347] Updated weights for policy 0, policy_version 1104516 (0.0008) [2023-12-27 13:18:06,350][20381] Updated weights for policy 1, policy_version 1103456 (0.0007) [2023-12-27 13:18:06,417][20381] Updated weights for policy 1, policy_version 1103466 (0.0007) [2023-12-27 13:18:06,480][20381] Updated weights for policy 1, policy_version 1103476 (0.0007) [2023-12-27 13:18:06,988][20347] Updated weights for policy 0, policy_version 1104526 (0.0008) [2023-12-27 13:18:07,046][20347] Updated weights for policy 0, policy_version 1104536 (0.0008) [2023-12-27 13:18:07,098][20347] Updated weights for policy 0, policy_version 1104546 (0.0007) [2023-12-27 13:18:07,104][20381] Updated weights for policy 1, policy_version 1103486 (0.0007) [2023-12-27 13:18:07,161][20381] Updated weights for policy 1, policy_version 1103496 (0.0007) [2023-12-27 13:18:07,221][20381] Updated weights for policy 1, policy_version 1103506 (0.0007) [2023-12-27 13:18:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 565338112. Throughput: 0: 10372.5, 1: 10089.1. Samples: 565302632. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:07,363][19170] Avg episode reward: [(0, '853.940'), (1, '695.910')] [2023-12-27 13:18:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001103512_282525696.pth... [2023-12-27 13:18:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001104552_282812416.pth... [2023-12-27 13:18:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001103368_282509312.pth [2023-12-27 13:18:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001102328_282222592.pth [2023-12-27 13:18:07,802][20347] Updated weights for policy 0, policy_version 1104556 (0.0007) [2023-12-27 13:18:07,856][20347] Updated weights for policy 0, policy_version 1104566 (0.0010) [2023-12-27 13:18:07,882][20381] Updated weights for policy 1, policy_version 1103516 (0.0007) [2023-12-27 13:18:07,911][20347] Updated weights for policy 0, policy_version 1104576 (0.0010) [2023-12-27 13:18:07,932][20381] Updated weights for policy 1, policy_version 1103526 (0.0005) [2023-12-27 13:18:07,979][20381] Updated weights for policy 1, policy_version 1103536 (0.0005) [2023-12-27 13:18:08,582][20347] Updated weights for policy 0, policy_version 1104586 (0.0007) [2023-12-27 13:18:08,604][20381] Updated weights for policy 1, policy_version 1103546 (0.0006) [2023-12-27 13:18:08,643][20347] Updated weights for policy 0, policy_version 1104596 (0.0006) [2023-12-27 13:18:08,660][20381] Updated weights for policy 1, policy_version 1103556 (0.0011) [2023-12-27 13:18:08,661][20291] KL-divergence is very high: 110.5534 [2023-12-27 13:18:08,694][20291] KL-divergence is very high: 106.1642 [2023-12-27 13:18:08,700][20347] Updated weights for policy 0, policy_version 1104606 (0.0006) [2023-12-27 13:18:08,717][20381] Updated weights for policy 1, policy_version 1103566 (0.0011) [2023-12-27 13:18:08,761][20347] Updated weights for policy 0, policy_version 1104616 (0.0006) [2023-12-27 13:18:08,777][20381] Updated weights for policy 1, policy_version 1103576 (0.0011) [2023-12-27 13:18:09,460][20347] Updated weights for policy 0, policy_version 1104626 (0.0010) [2023-12-27 13:18:09,507][20381] Updated weights for policy 1, policy_version 1103586 (0.0007) [2023-12-27 13:18:09,519][20347] Updated weights for policy 0, policy_version 1104636 (0.0010) [2023-12-27 13:18:09,563][20381] Updated weights for policy 1, policy_version 1103596 (0.0006) [2023-12-27 13:18:09,585][20347] Updated weights for policy 0, policy_version 1104646 (0.0010) [2023-12-27 13:18:09,622][20381] Updated weights for policy 1, policy_version 1103606 (0.0006) [2023-12-27 13:18:10,202][20381] Updated weights for policy 1, policy_version 1103616 (0.0007) [2023-12-27 13:18:10,259][20381] Updated weights for policy 1, policy_version 1103626 (0.0006) [2023-12-27 13:18:10,265][20347] Updated weights for policy 0, policy_version 1104656 (0.0011) [2023-12-27 13:18:10,320][20381] Updated weights for policy 1, policy_version 1103636 (0.0006) [2023-12-27 13:18:10,325][20347] Updated weights for policy 0, policy_version 1104666 (0.0010) [2023-12-27 13:18:10,385][20347] Updated weights for policy 0, policy_version 1104676 (0.0010) [2023-12-27 13:18:11,026][20381] Updated weights for policy 1, policy_version 1103646 (0.0010) [2023-12-27 13:18:11,080][20347] Updated weights for policy 0, policy_version 1104686 (0.0010) [2023-12-27 13:18:11,089][20381] Updated weights for policy 1, policy_version 1103656 (0.0008) [2023-12-27 13:18:11,149][20347] Updated weights for policy 0, policy_version 1104696 (0.0008) [2023-12-27 13:18:11,162][20381] Updated weights for policy 1, policy_version 1103666 (0.0009) [2023-12-27 13:18:11,215][20347] Updated weights for policy 0, policy_version 1104706 (0.0008) [2023-12-27 13:18:11,941][20381] Updated weights for policy 1, policy_version 1103676 (0.0008) [2023-12-27 13:18:11,973][20347] Updated weights for policy 0, policy_version 1104716 (0.0008) [2023-12-27 13:18:11,995][20381] Updated weights for policy 1, policy_version 1103686 (0.0006) [2023-12-27 13:18:12,022][20347] Updated weights for policy 0, policy_version 1104726 (0.0009) [2023-12-27 13:18:12,048][20381] Updated weights for policy 1, policy_version 1103696 (0.0007) [2023-12-27 13:18:12,072][20347] Updated weights for policy 0, policy_version 1104736 (0.0009) [2023-12-27 13:18:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 565436416. Throughput: 0: 10290.6, 1: 10138.8. Samples: 565424804. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:12,363][19170] Avg episode reward: [(0, '863.290'), (1, '715.680')] [2023-12-27 13:18:12,783][20381] Updated weights for policy 1, policy_version 1103706 (0.0007) [2023-12-27 13:18:12,838][20347] Updated weights for policy 0, policy_version 1104746 (0.0009) [2023-12-27 13:18:12,847][20381] Updated weights for policy 1, policy_version 1103716 (0.0009) [2023-12-27 13:18:12,898][20347] Updated weights for policy 0, policy_version 1104756 (0.0011) [2023-12-27 13:18:12,905][20381] Updated weights for policy 1, policy_version 1103726 (0.0010) [2023-12-27 13:18:12,946][20347] Updated weights for policy 0, policy_version 1104766 (0.0010) [2023-12-27 13:18:12,958][20381] Updated weights for policy 1, policy_version 1103736 (0.0010) [2023-12-27 13:18:12,992][20347] Updated weights for policy 0, policy_version 1104776 (0.0010) [2023-12-27 13:18:13,627][20381] Updated weights for policy 1, policy_version 1103746 (0.0007) [2023-12-27 13:18:13,661][20347] Updated weights for policy 0, policy_version 1104786 (0.0010) [2023-12-27 13:18:13,685][20381] Updated weights for policy 1, policy_version 1103756 (0.0006) [2023-12-27 13:18:13,724][20347] Updated weights for policy 0, policy_version 1104796 (0.0008) [2023-12-27 13:18:13,743][20381] Updated weights for policy 1, policy_version 1103766 (0.0006) [2023-12-27 13:18:13,774][20347] Updated weights for policy 0, policy_version 1104806 (0.0007) [2023-12-27 13:18:14,338][20381] Updated weights for policy 1, policy_version 1103776 (0.0008) [2023-12-27 13:18:14,399][20381] Updated weights for policy 1, policy_version 1103786 (0.0007) [2023-12-27 13:18:14,469][20381] Updated weights for policy 1, policy_version 1103796 (0.0008) [2023-12-27 13:18:14,518][20347] Updated weights for policy 0, policy_version 1104816 (0.0006) [2023-12-27 13:18:14,571][20347] Updated weights for policy 0, policy_version 1104826 (0.0006) [2023-12-27 13:18:14,629][20347] Updated weights for policy 0, policy_version 1104836 (0.0006) [2023-12-27 13:18:15,154][20381] Updated weights for policy 1, policy_version 1103806 (0.0008) [2023-12-27 13:18:15,211][20381] Updated weights for policy 1, policy_version 1103816 (0.0008) [2023-12-27 13:18:15,271][20381] Updated weights for policy 1, policy_version 1103826 (0.0007) [2023-12-27 13:18:15,323][20347] Updated weights for policy 0, policy_version 1104846 (0.0007) [2023-12-27 13:18:15,367][20347] Updated weights for policy 0, policy_version 1104856 (0.0008) [2023-12-27 13:18:15,411][20347] Updated weights for policy 0, policy_version 1104866 (0.0008) [2023-12-27 13:18:15,920][20381] Updated weights for policy 1, policy_version 1103836 (0.0007) [2023-12-27 13:18:15,968][20381] Updated weights for policy 1, policy_version 1103846 (0.0008) [2023-12-27 13:18:16,019][20381] Updated weights for policy 1, policy_version 1103856 (0.0007) [2023-12-27 13:18:16,134][20347] Updated weights for policy 0, policy_version 1104876 (0.0008) [2023-12-27 13:18:16,181][20347] Updated weights for policy 0, policy_version 1104886 (0.0010) [2023-12-27 13:18:16,230][20347] Updated weights for policy 0, policy_version 1104896 (0.0005) [2023-12-27 13:18:16,794][20381] Updated weights for policy 1, policy_version 1103866 (0.0007) [2023-12-27 13:18:16,799][20347] Updated weights for policy 0, policy_version 1104906 (0.0006) [2023-12-27 13:18:16,851][20347] Updated weights for policy 0, policy_version 1104916 (0.0005) [2023-12-27 13:18:16,852][20381] Updated weights for policy 1, policy_version 1103876 (0.0007) [2023-12-27 13:18:16,903][20347] Updated weights for policy 0, policy_version 1104926 (0.0005) [2023-12-27 13:18:16,912][20381] Updated weights for policy 1, policy_version 1103886 (0.0007) [2023-12-27 13:18:16,959][20347] Updated weights for policy 0, policy_version 1104936 (0.0005) [2023-12-27 13:18:16,964][20381] Updated weights for policy 1, policy_version 1103896 (0.0005) [2023-12-27 13:18:17,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 565534720. Throughput: 0: 10225.9, 1: 10146.2. Samples: 565544444. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:17,364][19170] Avg episode reward: [(0, '853.700'), (1, '695.400')] [2023-12-27 13:18:17,562][20347] Updated weights for policy 0, policy_version 1104946 (0.0008) [2023-12-27 13:18:17,618][20347] Updated weights for policy 0, policy_version 1104956 (0.0008) [2023-12-27 13:18:17,672][20381] Updated weights for policy 1, policy_version 1103906 (0.0007) [2023-12-27 13:18:17,682][20347] Updated weights for policy 0, policy_version 1104966 (0.0007) [2023-12-27 13:18:17,728][20381] Updated weights for policy 1, policy_version 1103916 (0.0009) [2023-12-27 13:18:17,780][20381] Updated weights for policy 1, policy_version 1103926 (0.0008) [2023-12-27 13:18:18,261][20347] Updated weights for policy 0, policy_version 1104976 (0.0007) [2023-12-27 13:18:18,306][20347] Updated weights for policy 0, policy_version 1104986 (0.0007) [2023-12-27 13:18:18,365][20347] Updated weights for policy 0, policy_version 1104996 (0.0007) [2023-12-27 13:18:18,502][20381] Updated weights for policy 1, policy_version 1103936 (0.0008) [2023-12-27 13:18:18,562][20381] Updated weights for policy 1, policy_version 1103946 (0.0008) [2023-12-27 13:18:18,617][20381] Updated weights for policy 1, policy_version 1103956 (0.0008) [2023-12-27 13:18:19,092][20347] Updated weights for policy 0, policy_version 1105006 (0.0008) [2023-12-27 13:18:19,143][20347] Updated weights for policy 0, policy_version 1105016 (0.0007) [2023-12-27 13:18:19,198][20347] Updated weights for policy 0, policy_version 1105026 (0.0007) [2023-12-27 13:18:19,327][20381] Updated weights for policy 1, policy_version 1103966 (0.0008) [2023-12-27 13:18:19,397][20381] Updated weights for policy 1, policy_version 1103976 (0.0009) [2023-12-27 13:18:19,451][20381] Updated weights for policy 1, policy_version 1103986 (0.0008) [2023-12-27 13:18:19,943][20347] Updated weights for policy 0, policy_version 1105036 (0.0008) [2023-12-27 13:18:20,006][20347] Updated weights for policy 0, policy_version 1105046 (0.0008) [2023-12-27 13:18:20,070][20347] Updated weights for policy 0, policy_version 1105056 (0.0009) [2023-12-27 13:18:20,201][20381] Updated weights for policy 1, policy_version 1103996 (0.0007) [2023-12-27 13:18:20,259][20381] Updated weights for policy 1, policy_version 1104006 (0.0008) [2023-12-27 13:18:20,312][20381] Updated weights for policy 1, policy_version 1104016 (0.0008) [2023-12-27 13:18:20,749][20347] Updated weights for policy 0, policy_version 1105066 (0.0008) [2023-12-27 13:18:20,810][20347] Updated weights for policy 0, policy_version 1105076 (0.0008) [2023-12-27 13:18:20,871][20347] Updated weights for policy 0, policy_version 1105086 (0.0007) [2023-12-27 13:18:20,931][20347] Updated weights for policy 0, policy_version 1105096 (0.0007) [2023-12-27 13:18:20,999][20381] Updated weights for policy 1, policy_version 1104026 (0.0008) [2023-12-27 13:18:21,068][20381] Updated weights for policy 1, policy_version 1104036 (0.0009) [2023-12-27 13:18:21,136][20381] Updated weights for policy 1, policy_version 1104046 (0.0008) [2023-12-27 13:18:21,201][20381] Updated weights for policy 1, policy_version 1104056 (0.0008) [2023-12-27 13:18:21,690][20347] Updated weights for policy 0, policy_version 1105106 (0.0008) [2023-12-27 13:18:21,760][20347] Updated weights for policy 0, policy_version 1105116 (0.0008) [2023-12-27 13:18:21,825][20347] Updated weights for policy 0, policy_version 1105126 (0.0006) [2023-12-27 13:18:21,883][20381] Updated weights for policy 1, policy_version 1104066 (0.0008) [2023-12-27 13:18:21,936][20381] Updated weights for policy 1, policy_version 1104076 (0.0008) [2023-12-27 13:18:21,988][20381] Updated weights for policy 1, policy_version 1104086 (0.0008) [2023-12-27 13:18:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 565633024. Throughput: 0: 10196.4, 1: 10129.9. Samples: 565605132. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:22,363][19170] Avg episode reward: [(0, '873.380'), (1, '705.310')] [2023-12-27 13:18:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001104088_282673152.pth... [2023-12-27 13:18:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001105128_282959872.pth... [2023-12-27 13:18:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001103944_282656768.pth [2023-12-27 13:18:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001102904_282370048.pth [2023-12-27 13:18:22,530][20347] Updated weights for policy 0, policy_version 1105136 (0.0006) [2023-12-27 13:18:22,587][20347] Updated weights for policy 0, policy_version 1105146 (0.0007) [2023-12-27 13:18:22,641][20347] Updated weights for policy 0, policy_version 1105156 (0.0008) [2023-12-27 13:18:22,712][20381] Updated weights for policy 1, policy_version 1104096 (0.0007) [2023-12-27 13:18:22,770][20381] Updated weights for policy 1, policy_version 1104106 (0.0008) [2023-12-27 13:18:22,830][20381] Updated weights for policy 1, policy_version 1104116 (0.0008) [2023-12-27 13:18:23,354][20347] Updated weights for policy 0, policy_version 1105166 (0.0008) [2023-12-27 13:18:23,405][20381] Updated weights for policy 1, policy_version 1104126 (0.0006) [2023-12-27 13:18:23,412][20347] Updated weights for policy 0, policy_version 1105176 (0.0007) [2023-12-27 13:18:23,463][20381] Updated weights for policy 1, policy_version 1104136 (0.0005) [2023-12-27 13:18:23,469][20347] Updated weights for policy 0, policy_version 1105186 (0.0008) [2023-12-27 13:18:23,522][20381] Updated weights for policy 1, policy_version 1104146 (0.0006) [2023-12-27 13:18:24,166][20347] Updated weights for policy 0, policy_version 1105196 (0.0006) [2023-12-27 13:18:24,198][20381] Updated weights for policy 1, policy_version 1104156 (0.0007) [2023-12-27 13:18:24,227][20347] Updated weights for policy 0, policy_version 1105206 (0.0007) [2023-12-27 13:18:24,251][20381] Updated weights for policy 1, policy_version 1104166 (0.0006) [2023-12-27 13:18:24,279][20347] Updated weights for policy 0, policy_version 1105216 (0.0007) [2023-12-27 13:18:24,303][20381] Updated weights for policy 1, policy_version 1104176 (0.0006) [2023-12-27 13:18:24,882][20347] Updated weights for policy 0, policy_version 1105226 (0.0006) [2023-12-27 13:18:24,933][20347] Updated weights for policy 0, policy_version 1105236 (0.0007) [2023-12-27 13:18:24,982][20347] Updated weights for policy 0, policy_version 1105246 (0.0006) [2023-12-27 13:18:24,984][20381] Updated weights for policy 1, policy_version 1104186 (0.0007) [2023-12-27 13:18:25,032][20381] Updated weights for policy 1, policy_version 1104196 (0.0006) [2023-12-27 13:18:25,035][20347] Updated weights for policy 0, policy_version 1105256 (0.0005) [2023-12-27 13:18:25,084][20381] Updated weights for policy 1, policy_version 1104206 (0.0006) [2023-12-27 13:18:25,132][20381] Updated weights for policy 1, policy_version 1104216 (0.0008) [2023-12-27 13:18:25,597][20347] Updated weights for policy 0, policy_version 1105266 (0.0006) [2023-12-27 13:18:25,648][20347] Updated weights for policy 0, policy_version 1105276 (0.0010) [2023-12-27 13:18:25,700][20347] Updated weights for policy 0, policy_version 1105286 (0.0010) [2023-12-27 13:18:25,766][20381] Updated weights for policy 1, policy_version 1104226 (0.0005) [2023-12-27 13:18:25,818][20381] Updated weights for policy 1, policy_version 1104236 (0.0007) [2023-12-27 13:18:25,874][20381] Updated weights for policy 1, policy_version 1104246 (0.0005) [2023-12-27 13:18:26,368][20347] Updated weights for policy 0, policy_version 1105296 (0.0008) [2023-12-27 13:18:26,411][20381] Updated weights for policy 1, policy_version 1104256 (0.0005) [2023-12-27 13:18:26,423][20347] Updated weights for policy 0, policy_version 1105306 (0.0007) [2023-12-27 13:18:26,469][20381] Updated weights for policy 1, policy_version 1104266 (0.0005) [2023-12-27 13:18:26,484][20347] Updated weights for policy 0, policy_version 1105316 (0.0008) [2023-12-27 13:18:26,524][20381] Updated weights for policy 1, policy_version 1104276 (0.0005) [2023-12-27 13:18:27,137][20381] Updated weights for policy 1, policy_version 1104286 (0.0007) [2023-12-27 13:18:27,158][20347] Updated weights for policy 0, policy_version 1105326 (0.0010) [2023-12-27 13:18:27,197][20381] Updated weights for policy 1, policy_version 1104296 (0.0006) [2023-12-27 13:18:27,214][20347] Updated weights for policy 0, policy_version 1105336 (0.0010) [2023-12-27 13:18:27,248][20381] Updated weights for policy 1, policy_version 1104306 (0.0005) [2023-12-27 13:18:27,265][20347] Updated weights for policy 0, policy_version 1105346 (0.0010) [2023-12-27 13:18:27,362][19170] Fps is (10 sec: 21300.1, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 565747712. Throughput: 0: 10192.4, 1: 10146.5. Samples: 565729744. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:27,363][19170] Avg episode reward: [(0, '864.040'), (1, '764.650')] [2023-12-27 13:18:27,891][20347] Updated weights for policy 0, policy_version 1105356 (0.0009) [2023-12-27 13:18:27,944][20347] Updated weights for policy 0, policy_version 1105366 (0.0008) [2023-12-27 13:18:27,954][20381] Updated weights for policy 1, policy_version 1104316 (0.0005) [2023-12-27 13:18:27,993][20347] Updated weights for policy 0, policy_version 1105376 (0.0006) [2023-12-27 13:18:28,012][20381] Updated weights for policy 1, policy_version 1104326 (0.0007) [2023-12-27 13:18:28,067][20381] Updated weights for policy 1, policy_version 1104336 (0.0007) [2023-12-27 13:18:28,669][20347] Updated weights for policy 0, policy_version 1105386 (0.0006) [2023-12-27 13:18:28,715][20381] Updated weights for policy 1, policy_version 1104346 (0.0007) [2023-12-27 13:18:28,727][20347] Updated weights for policy 0, policy_version 1105396 (0.0008) [2023-12-27 13:18:28,765][20381] Updated weights for policy 1, policy_version 1104356 (0.0008) [2023-12-27 13:18:28,788][20347] Updated weights for policy 0, policy_version 1105406 (0.0006) [2023-12-27 13:18:28,814][20381] Updated weights for policy 1, policy_version 1104366 (0.0008) [2023-12-27 13:18:28,848][20347] Updated weights for policy 0, policy_version 1105416 (0.0010) [2023-12-27 13:18:28,869][20381] Updated weights for policy 1, policy_version 1104376 (0.0007) [2023-12-27 13:18:29,448][20347] Updated weights for policy 0, policy_version 1105426 (0.0006) [2023-12-27 13:18:29,511][20347] Updated weights for policy 0, policy_version 1105436 (0.0006) [2023-12-27 13:18:29,566][20347] Updated weights for policy 0, policy_version 1105446 (0.0010) [2023-12-27 13:18:29,607][20381] Updated weights for policy 1, policy_version 1104386 (0.0007) [2023-12-27 13:18:29,654][20381] Updated weights for policy 1, policy_version 1104396 (0.0008) [2023-12-27 13:18:29,702][20381] Updated weights for policy 1, policy_version 1104406 (0.0007) [2023-12-27 13:18:30,305][20347] Updated weights for policy 0, policy_version 1105456 (0.0008) [2023-12-27 13:18:30,363][20347] Updated weights for policy 0, policy_version 1105466 (0.0008) [2023-12-27 13:18:30,403][20381] Updated weights for policy 1, policy_version 1104416 (0.0008) [2023-12-27 13:18:30,413][20347] Updated weights for policy 0, policy_version 1105476 (0.0007) [2023-12-27 13:18:30,461][20381] Updated weights for policy 1, policy_version 1104426 (0.0005) [2023-12-27 13:18:30,514][20381] Updated weights for policy 1, policy_version 1104436 (0.0006) [2023-12-27 13:18:31,044][20381] Updated weights for policy 1, policy_version 1104446 (0.0007) [2023-12-27 13:18:31,106][20381] Updated weights for policy 1, policy_version 1104456 (0.0007) [2023-12-27 13:18:31,118][20347] Updated weights for policy 0, policy_version 1105486 (0.0008) [2023-12-27 13:18:31,170][20381] Updated weights for policy 1, policy_version 1104466 (0.0007) [2023-12-27 13:18:31,182][20347] Updated weights for policy 0, policy_version 1105496 (0.0008) [2023-12-27 13:18:31,241][20347] Updated weights for policy 0, policy_version 1105506 (0.0008) [2023-12-27 13:18:31,829][20381] Updated weights for policy 1, policy_version 1104476 (0.0007) [2023-12-27 13:18:31,888][20381] Updated weights for policy 1, policy_version 1104486 (0.0008) [2023-12-27 13:18:31,925][20347] Updated weights for policy 0, policy_version 1105516 (0.0007) [2023-12-27 13:18:31,945][20381] Updated weights for policy 1, policy_version 1104496 (0.0007) [2023-12-27 13:18:31,981][20347] Updated weights for policy 0, policy_version 1105526 (0.0005) [2023-12-27 13:18:32,037][20347] Updated weights for policy 0, policy_version 1105536 (0.0006) [2023-12-27 13:18:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 565846016. Throughput: 0: 10182.8, 1: 10201.6. Samples: 565855020. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:32,363][19170] Avg episode reward: [(0, '864.400'), (1, '646.860')] [2023-12-27 13:18:32,648][20381] Updated weights for policy 1, policy_version 1104506 (0.0008) [2023-12-27 13:18:32,671][20347] Updated weights for policy 0, policy_version 1105546 (0.0008) [2023-12-27 13:18:32,707][20381] Updated weights for policy 1, policy_version 1104516 (0.0008) [2023-12-27 13:18:32,731][20347] Updated weights for policy 0, policy_version 1105556 (0.0005) [2023-12-27 13:18:32,736][20291] KL-divergence is very high: 100.7025 [2023-12-27 13:18:32,758][20291] KL-divergence is very high: 242.6131 [2023-12-27 13:18:32,765][20381] Updated weights for policy 1, policy_version 1104526 (0.0007) [2023-12-27 13:18:32,783][20291] KL-divergence is very high: 142.1358 [2023-12-27 13:18:32,787][20347] Updated weights for policy 0, policy_version 1105566 (0.0005) [2023-12-27 13:18:32,807][20291] KL-divergence is very high: 125.1103 [2023-12-27 13:18:32,825][20381] Updated weights for policy 1, policy_version 1104536 (0.0007) [2023-12-27 13:18:32,847][20347] Updated weights for policy 0, policy_version 1105576 (0.0007) [2023-12-27 13:18:33,389][20347] Updated weights for policy 0, policy_version 1105586 (0.0007) [2023-12-27 13:18:33,442][20347] Updated weights for policy 0, policy_version 1105596 (0.0006) [2023-12-27 13:18:33,491][20347] Updated weights for policy 0, policy_version 1105606 (0.0005) [2023-12-27 13:18:33,537][20381] Updated weights for policy 1, policy_version 1104546 (0.0007) [2023-12-27 13:18:33,589][20381] Updated weights for policy 1, policy_version 1104556 (0.0008) [2023-12-27 13:18:33,647][20381] Updated weights for policy 1, policy_version 1104566 (0.0009) [2023-12-27 13:18:34,045][20347] Updated weights for policy 0, policy_version 1105616 (0.0005) [2023-12-27 13:18:34,093][20347] Updated weights for policy 0, policy_version 1105626 (0.0005) [2023-12-27 13:18:34,156][20347] Updated weights for policy 0, policy_version 1105636 (0.0006) [2023-12-27 13:18:34,386][20381] Updated weights for policy 1, policy_version 1104576 (0.0007) [2023-12-27 13:18:34,439][20381] Updated weights for policy 1, policy_version 1104586 (0.0007) [2023-12-27 13:18:34,502][20381] Updated weights for policy 1, policy_version 1104596 (0.0007) [2023-12-27 13:18:34,847][20347] Updated weights for policy 0, policy_version 1105646 (0.0008) [2023-12-27 13:18:34,904][20347] Updated weights for policy 0, policy_version 1105656 (0.0007) [2023-12-27 13:18:34,960][20347] Updated weights for policy 0, policy_version 1105666 (0.0008) [2023-12-27 13:18:35,172][20381] Updated weights for policy 1, policy_version 1104606 (0.0007) [2023-12-27 13:18:35,230][20381] Updated weights for policy 1, policy_version 1104616 (0.0009) [2023-12-27 13:18:35,274][20381] Updated weights for policy 1, policy_version 1104626 (0.0008) [2023-12-27 13:18:35,634][20347] Updated weights for policy 0, policy_version 1105676 (0.0007) [2023-12-27 13:18:35,692][20347] Updated weights for policy 0, policy_version 1105686 (0.0010) [2023-12-27 13:18:35,740][20347] Updated weights for policy 0, policy_version 1105696 (0.0010) [2023-12-27 13:18:36,033][20381] Updated weights for policy 1, policy_version 1104636 (0.0008) [2023-12-27 13:18:36,092][20381] Updated weights for policy 1, policy_version 1104646 (0.0007) [2023-12-27 13:18:36,155][20381] Updated weights for policy 1, policy_version 1104656 (0.0009) [2023-12-27 13:18:36,386][20347] Updated weights for policy 0, policy_version 1105706 (0.0010) [2023-12-27 13:18:36,444][20347] Updated weights for policy 0, policy_version 1105716 (0.0009) [2023-12-27 13:18:36,507][20347] Updated weights for policy 0, policy_version 1105726 (0.0011) [2023-12-27 13:18:36,567][20347] Updated weights for policy 0, policy_version 1105736 (0.0011) [2023-12-27 13:18:36,890][20381] Updated weights for policy 1, policy_version 1104666 (0.0008) [2023-12-27 13:18:36,945][20381] Updated weights for policy 1, policy_version 1104676 (0.0008) [2023-12-27 13:18:37,008][20381] Updated weights for policy 1, policy_version 1104686 (0.0008) [2023-12-27 13:18:37,064][20381] Updated weights for policy 1, policy_version 1104696 (0.0008) [2023-12-27 13:18:37,239][20347] Updated weights for policy 0, policy_version 1105746 (0.0005) [2023-12-27 13:18:37,292][20347] Updated weights for policy 0, policy_version 1105756 (0.0007) [2023-12-27 13:18:37,341][20347] Updated weights for policy 0, policy_version 1105766 (0.0008) [2023-12-27 13:18:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 565952512. Throughput: 0: 10254.6, 1: 10129.0. Samples: 565917236. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:37,363][19170] Avg episode reward: [(0, '854.450'), (1, '725.100')] [2023-12-27 13:18:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001104696_282828800.pth... [2023-12-27 13:18:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001105768_283123712.pth... [2023-12-27 13:18:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001103512_282525696.pth [2023-12-27 13:18:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001104552_282812416.pth [2023-12-27 13:18:37,798][20381] Updated weights for policy 1, policy_version 1104706 (0.0006) [2023-12-27 13:18:37,866][20381] Updated weights for policy 1, policy_version 1104716 (0.0005) [2023-12-27 13:18:37,928][20381] Updated weights for policy 1, policy_version 1104726 (0.0005) [2023-12-27 13:18:37,967][20347] Updated weights for policy 0, policy_version 1105776 (0.0007) [2023-12-27 13:18:38,021][20347] Updated weights for policy 0, policy_version 1105786 (0.0008) [2023-12-27 13:18:38,078][20347] Updated weights for policy 0, policy_version 1105796 (0.0008) [2023-12-27 13:18:38,520][20381] Updated weights for policy 1, policy_version 1104736 (0.0007) [2023-12-27 13:18:38,576][20381] Updated weights for policy 1, policy_version 1104746 (0.0008) [2023-12-27 13:18:38,636][20381] Updated weights for policy 1, policy_version 1104756 (0.0008) [2023-12-27 13:18:38,786][20347] Updated weights for policy 0, policy_version 1105806 (0.0008) [2023-12-27 13:18:38,841][20347] Updated weights for policy 0, policy_version 1105816 (0.0008) [2023-12-27 13:18:38,894][20347] Updated weights for policy 0, policy_version 1105826 (0.0008) [2023-12-27 13:18:39,262][20381] Updated weights for policy 1, policy_version 1104766 (0.0008) [2023-12-27 13:18:39,323][20381] Updated weights for policy 1, policy_version 1104776 (0.0008) [2023-12-27 13:18:39,392][20381] Updated weights for policy 1, policy_version 1104786 (0.0008) [2023-12-27 13:18:39,651][20347] Updated weights for policy 0, policy_version 1105836 (0.0009) [2023-12-27 13:18:39,705][20347] Updated weights for policy 0, policy_version 1105846 (0.0007) [2023-12-27 13:18:39,763][20347] Updated weights for policy 0, policy_version 1105856 (0.0008) [2023-12-27 13:18:40,114][20381] Updated weights for policy 1, policy_version 1104797 (0.0009) [2023-12-27 13:18:40,172][20381] Updated weights for policy 1, policy_version 1104807 (0.0008) [2023-12-27 13:18:40,233][20381] Updated weights for policy 1, policy_version 1104817 (0.0007) [2023-12-27 13:18:40,488][20347] Updated weights for policy 0, policy_version 1105866 (0.0008) [2023-12-27 13:18:40,553][20347] Updated weights for policy 0, policy_version 1105876 (0.0005) [2023-12-27 13:18:40,601][20347] Updated weights for policy 0, policy_version 1105886 (0.0005) [2023-12-27 13:18:40,650][20347] Updated weights for policy 0, policy_version 1105896 (0.0005) [2023-12-27 13:18:40,934][20381] Updated weights for policy 1, policy_version 1104827 (0.0007) [2023-12-27 13:18:40,984][20381] Updated weights for policy 1, policy_version 1104837 (0.0006) [2023-12-27 13:18:41,035][20381] Updated weights for policy 1, policy_version 1104847 (0.0007) [2023-12-27 13:18:41,332][20347] Updated weights for policy 0, policy_version 1105906 (0.0006) [2023-12-27 13:18:41,402][20347] Updated weights for policy 0, policy_version 1105916 (0.0008) [2023-12-27 13:18:41,465][20347] Updated weights for policy 0, policy_version 1105926 (0.0011) [2023-12-27 13:18:41,803][20381] Updated weights for policy 1, policy_version 1104857 (0.0008) [2023-12-27 13:18:41,867][20381] Updated weights for policy 1, policy_version 1104867 (0.0007) [2023-12-27 13:18:41,924][20381] Updated weights for policy 1, policy_version 1104877 (0.0005) [2023-12-27 13:18:41,988][20381] Updated weights for policy 1, policy_version 1104887 (0.0007) [2023-12-27 13:18:42,171][20347] Updated weights for policy 0, policy_version 1105936 (0.0011) [2023-12-27 13:18:42,229][20347] Updated weights for policy 0, policy_version 1105946 (0.0010) [2023-12-27 13:18:42,288][20291] KL-divergence is very high: 103.3538 [2023-12-27 13:18:42,289][20347] Updated weights for policy 0, policy_version 1105956 (0.0008) [2023-12-27 13:18:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 566050816. Throughput: 0: 10253.2, 1: 10150.2. Samples: 566039228. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:42,363][19170] Avg episode reward: [(0, '747.230'), (1, '646.680')] [2023-12-27 13:18:42,629][20381] Updated weights for policy 1, policy_version 1104897 (0.0007) [2023-12-27 13:18:42,689][20381] Updated weights for policy 1, policy_version 1104907 (0.0008) [2023-12-27 13:18:42,756][20381] Updated weights for policy 1, policy_version 1104917 (0.0009) [2023-12-27 13:18:42,925][20347] Updated weights for policy 0, policy_version 1105966 (0.0008) [2023-12-27 13:18:42,974][20347] Updated weights for policy 0, policy_version 1105976 (0.0007) [2023-12-27 13:18:43,039][20347] Updated weights for policy 0, policy_version 1105986 (0.0008) [2023-12-27 13:18:43,349][20381] Updated weights for policy 1, policy_version 1104927 (0.0005) [2023-12-27 13:18:43,408][20381] Updated weights for policy 1, policy_version 1104937 (0.0005) [2023-12-27 13:18:43,456][20381] Updated weights for policy 1, policy_version 1104947 (0.0005) [2023-12-27 13:18:43,712][20347] Updated weights for policy 0, policy_version 1105996 (0.0009) [2023-12-27 13:18:43,770][20347] Updated weights for policy 0, policy_version 1106006 (0.0009) [2023-12-27 13:18:43,831][20347] Updated weights for policy 0, policy_version 1106016 (0.0006) [2023-12-27 13:18:44,113][20381] Updated weights for policy 1, policy_version 1104957 (0.0006) [2023-12-27 13:18:44,161][20381] Updated weights for policy 1, policy_version 1104967 (0.0008) [2023-12-27 13:18:44,225][20381] Updated weights for policy 1, policy_version 1104977 (0.0007) [2023-12-27 13:18:44,522][20347] Updated weights for policy 0, policy_version 1106026 (0.0008) [2023-12-27 13:18:44,571][20347] Updated weights for policy 0, policy_version 1106036 (0.0007) [2023-12-27 13:18:44,625][20347] Updated weights for policy 0, policy_version 1106046 (0.0007) [2023-12-27 13:18:44,680][20347] Updated weights for policy 0, policy_version 1106056 (0.0008) [2023-12-27 13:18:44,938][20381] Updated weights for policy 1, policy_version 1104987 (0.0007) [2023-12-27 13:18:44,994][20381] Updated weights for policy 1, policy_version 1104997 (0.0008) [2023-12-27 13:18:45,050][20381] Updated weights for policy 1, policy_version 1105007 (0.0008) [2023-12-27 13:18:45,436][20347] Updated weights for policy 0, policy_version 1106066 (0.0010) [2023-12-27 13:18:45,485][20347] Updated weights for policy 0, policy_version 1106076 (0.0008) [2023-12-27 13:18:45,544][20347] Updated weights for policy 0, policy_version 1106086 (0.0008) [2023-12-27 13:18:45,749][20381] Updated weights for policy 1, policy_version 1105017 (0.0008) [2023-12-27 13:18:45,805][20381] Updated weights for policy 1, policy_version 1105027 (0.0007) [2023-12-27 13:18:45,855][20381] Updated weights for policy 1, policy_version 1105037 (0.0010) [2023-12-27 13:18:45,914][20381] Updated weights for policy 1, policy_version 1105047 (0.0008) [2023-12-27 13:18:46,136][20347] Updated weights for policy 0, policy_version 1106096 (0.0006) [2023-12-27 13:18:46,198][20347] Updated weights for policy 0, policy_version 1106106 (0.0005) [2023-12-27 13:18:46,259][20347] Updated weights for policy 0, policy_version 1106116 (0.0005) [2023-12-27 13:18:46,613][20381] Updated weights for policy 1, policy_version 1105057 (0.0006) [2023-12-27 13:18:46,682][20381] Updated weights for policy 1, policy_version 1105067 (0.0007) [2023-12-27 13:18:46,736][20381] Updated weights for policy 1, policy_version 1105077 (0.0011) [2023-12-27 13:18:46,831][20347] Updated weights for policy 0, policy_version 1106126 (0.0006) [2023-12-27 13:18:46,886][20347] Updated weights for policy 0, policy_version 1106136 (0.0007) [2023-12-27 13:18:46,942][20347] Updated weights for policy 0, policy_version 1106146 (0.0007) [2023-12-27 13:18:47,350][20381] Updated weights for policy 1, policy_version 1105087 (0.0007) [2023-12-27 13:18:47,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 566149120. Throughput: 0: 10253.0, 1: 10196.3. Samples: 566162536. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:47,364][19170] Avg episode reward: [(0, '750.520'), (1, '715.680')] [2023-12-27 13:18:47,417][20381] Updated weights for policy 1, policy_version 1105097 (0.0005) [2023-12-27 13:18:47,474][20381] Updated weights for policy 1, policy_version 1105107 (0.0006) [2023-12-27 13:18:47,521][20347] Updated weights for policy 0, policy_version 1106156 (0.0006) [2023-12-27 13:18:47,583][20347] Updated weights for policy 0, policy_version 1106166 (0.0008) [2023-12-27 13:18:47,631][20347] Updated weights for policy 0, policy_version 1106176 (0.0008) [2023-12-27 13:18:48,068][20381] Updated weights for policy 1, policy_version 1105117 (0.0008) [2023-12-27 13:18:48,118][20381] Updated weights for policy 1, policy_version 1105127 (0.0006) [2023-12-27 13:18:48,177][20381] Updated weights for policy 1, policy_version 1105137 (0.0007) [2023-12-27 13:18:48,322][20347] Updated weights for policy 0, policy_version 1106186 (0.0009) [2023-12-27 13:18:48,386][20347] Updated weights for policy 0, policy_version 1106196 (0.0008) [2023-12-27 13:18:48,437][20347] Updated weights for policy 0, policy_version 1106206 (0.0008) [2023-12-27 13:18:48,494][20347] Updated weights for policy 0, policy_version 1106216 (0.0006) [2023-12-27 13:18:48,778][20381] Updated weights for policy 1, policy_version 1105147 (0.0007) [2023-12-27 13:18:48,840][20381] Updated weights for policy 1, policy_version 1105157 (0.0006) [2023-12-27 13:18:48,892][20381] Updated weights for policy 1, policy_version 1105167 (0.0008) [2023-12-27 13:18:49,138][20347] Updated weights for policy 0, policy_version 1106226 (0.0010) [2023-12-27 13:18:49,188][20347] Updated weights for policy 0, policy_version 1106236 (0.0007) [2023-12-27 13:18:49,259][20347] Updated weights for policy 0, policy_version 1106246 (0.0007) [2023-12-27 13:18:49,579][20381] Updated weights for policy 1, policy_version 1105177 (0.0007) [2023-12-27 13:18:49,632][20381] Updated weights for policy 1, policy_version 1105187 (0.0007) [2023-12-27 13:18:49,695][20381] Updated weights for policy 1, policy_version 1105197 (0.0006) [2023-12-27 13:18:49,754][20381] Updated weights for policy 1, policy_version 1105207 (0.0006) [2023-12-27 13:18:49,981][20347] Updated weights for policy 0, policy_version 1106256 (0.0006) [2023-12-27 13:18:50,050][20347] Updated weights for policy 0, policy_version 1106266 (0.0007) [2023-12-27 13:18:50,116][20347] Updated weights for policy 0, policy_version 1106276 (0.0008) [2023-12-27 13:18:50,121][20291] KL-divergence is very high: 137.1889 [2023-12-27 13:18:50,391][20381] Updated weights for policy 1, policy_version 1105217 (0.0007) [2023-12-27 13:18:50,454][20381] Updated weights for policy 1, policy_version 1105227 (0.0008) [2023-12-27 13:18:50,505][20381] Updated weights for policy 1, policy_version 1105237 (0.0008) [2023-12-27 13:18:50,715][20291] KL-divergence is very high: 193.6201 [2023-12-27 13:18:50,739][20291] KL-divergence is very high: 120.9346 [2023-12-27 13:18:50,752][20347] Updated weights for policy 0, policy_version 1106286 (0.0009) [2023-12-27 13:18:50,758][20291] KL-divergence is very high: 215.7179 [2023-12-27 13:18:50,811][20291] KL-divergence is very high: 168.5376 [2023-12-27 13:18:50,818][20347] Updated weights for policy 0, policy_version 1106296 (0.0008) [2023-12-27 13:18:50,883][20347] Updated weights for policy 0, policy_version 1106306 (0.0008) [2023-12-27 13:18:51,147][20381] Updated weights for policy 1, policy_version 1105247 (0.0008) [2023-12-27 13:18:51,214][20381] Updated weights for policy 1, policy_version 1105257 (0.0007) [2023-12-27 13:18:51,275][20381] Updated weights for policy 1, policy_version 1105267 (0.0008) [2023-12-27 13:18:51,622][20347] Updated weights for policy 0, policy_version 1106316 (0.0008) [2023-12-27 13:18:51,685][20347] Updated weights for policy 0, policy_version 1106326 (0.0007) [2023-12-27 13:18:51,747][20347] Updated weights for policy 0, policy_version 1106336 (0.0008) [2023-12-27 13:18:51,973][20381] Updated weights for policy 1, policy_version 1105277 (0.0008) [2023-12-27 13:18:52,033][20381] Updated weights for policy 1, policy_version 1105287 (0.0008) [2023-12-27 13:18:52,093][20381] Updated weights for policy 1, policy_version 1105297 (0.0007) [2023-12-27 13:18:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 566255616. Throughput: 0: 10284.3, 1: 10242.1. Samples: 566226316. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:52,363][19170] Avg episode reward: [(0, '749.480'), (1, '705.570')] [2023-12-27 13:18:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001105304_282984448.pth... [2023-12-27 13:18:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001106344_283271168.pth... [2023-12-27 13:18:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001104088_282673152.pth [2023-12-27 13:18:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001105128_282959872.pth [2023-12-27 13:18:52,446][20347] Updated weights for policy 0, policy_version 1106346 (0.0007) [2023-12-27 13:18:52,497][20347] Updated weights for policy 0, policy_version 1106356 (0.0007) [2023-12-27 13:18:52,555][20347] Updated weights for policy 0, policy_version 1106366 (0.0007) [2023-12-27 13:18:52,607][20347] Updated weights for policy 0, policy_version 1106376 (0.0008) [2023-12-27 13:18:52,772][20381] Updated weights for policy 1, policy_version 1105307 (0.0007) [2023-12-27 13:18:52,821][20381] Updated weights for policy 1, policy_version 1105317 (0.0005) [2023-12-27 13:18:52,871][20381] Updated weights for policy 1, policy_version 1105327 (0.0006) [2023-12-27 13:18:53,324][20347] Updated weights for policy 0, policy_version 1106386 (0.0007) [2023-12-27 13:18:53,373][20347] Updated weights for policy 0, policy_version 1106396 (0.0007) [2023-12-27 13:18:53,428][20347] Updated weights for policy 0, policy_version 1106406 (0.0008) [2023-12-27 13:18:53,469][20381] Updated weights for policy 1, policy_version 1105337 (0.0006) [2023-12-27 13:18:53,522][20381] Updated weights for policy 1, policy_version 1105347 (0.0007) [2023-12-27 13:18:53,575][20381] Updated weights for policy 1, policy_version 1105357 (0.0005) [2023-12-27 13:18:53,620][20381] Updated weights for policy 1, policy_version 1105367 (0.0005) [2023-12-27 13:18:54,152][20347] Updated weights for policy 0, policy_version 1106416 (0.0008) [2023-12-27 13:18:54,176][20381] Updated weights for policy 1, policy_version 1105377 (0.0006) [2023-12-27 13:18:54,198][20347] Updated weights for policy 0, policy_version 1106426 (0.0007) [2023-12-27 13:18:54,231][20381] Updated weights for policy 1, policy_version 1105387 (0.0007) [2023-12-27 13:18:54,261][20347] Updated weights for policy 0, policy_version 1106436 (0.0007) [2023-12-27 13:18:54,282][20381] Updated weights for policy 1, policy_version 1105397 (0.0007) [2023-12-27 13:18:54,882][20381] Updated weights for policy 1, policy_version 1105407 (0.0006) [2023-12-27 13:18:54,932][20381] Updated weights for policy 1, policy_version 1105417 (0.0005) [2023-12-27 13:18:54,985][20381] Updated weights for policy 1, policy_version 1105427 (0.0006) [2023-12-27 13:18:54,998][20347] Updated weights for policy 0, policy_version 1106446 (0.0007) [2023-12-27 13:18:55,057][20347] Updated weights for policy 0, policy_version 1106456 (0.0007) [2023-12-27 13:18:55,116][20347] Updated weights for policy 0, policy_version 1106466 (0.0007) [2023-12-27 13:18:55,610][20381] Updated weights for policy 1, policy_version 1105437 (0.0007) [2023-12-27 13:18:55,671][20381] Updated weights for policy 1, policy_version 1105447 (0.0008) [2023-12-27 13:18:55,717][20381] Updated weights for policy 1, policy_version 1105457 (0.0007) [2023-12-27 13:18:55,801][20347] Updated weights for policy 0, policy_version 1106476 (0.0007) [2023-12-27 13:18:55,856][20347] Updated weights for policy 0, policy_version 1106486 (0.0007) [2023-12-27 13:18:55,915][20347] Updated weights for policy 0, policy_version 1106496 (0.0007) [2023-12-27 13:18:56,366][20381] Updated weights for policy 1, policy_version 1105467 (0.0007) [2023-12-27 13:18:56,424][20381] Updated weights for policy 1, policy_version 1105477 (0.0005) [2023-12-27 13:18:56,476][20381] Updated weights for policy 1, policy_version 1105487 (0.0005) [2023-12-27 13:18:56,648][20347] Updated weights for policy 0, policy_version 1106506 (0.0007) [2023-12-27 13:18:56,705][20347] Updated weights for policy 0, policy_version 1106516 (0.0009) [2023-12-27 13:18:56,761][20347] Updated weights for policy 0, policy_version 1106526 (0.0007) [2023-12-27 13:18:56,816][20347] Updated weights for policy 0, policy_version 1106536 (0.0008) [2023-12-27 13:18:57,044][20381] Updated weights for policy 1, policy_version 1105497 (0.0006) [2023-12-27 13:18:57,105][20381] Updated weights for policy 1, policy_version 1105507 (0.0008) [2023-12-27 13:18:57,167][20381] Updated weights for policy 1, policy_version 1105517 (0.0008) [2023-12-27 13:18:57,221][20381] Updated weights for policy 1, policy_version 1105527 (0.0008) [2023-12-27 13:18:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 566362112. Throughput: 0: 10261.7, 1: 10298.9. Samples: 566350032. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:18:57,363][19170] Avg episode reward: [(0, '786.150'), (1, '646.800')] [2023-12-27 13:18:57,495][20347] Updated weights for policy 0, policy_version 1106546 (0.0007) [2023-12-27 13:18:57,543][20347] Updated weights for policy 0, policy_version 1106556 (0.0007) [2023-12-27 13:18:57,605][20347] Updated weights for policy 0, policy_version 1106566 (0.0007) [2023-12-27 13:18:57,896][20381] Updated weights for policy 1, policy_version 1105537 (0.0007) [2023-12-27 13:18:57,947][20381] Updated weights for policy 1, policy_version 1105547 (0.0007) [2023-12-27 13:18:58,000][20381] Updated weights for policy 1, policy_version 1105557 (0.0007) [2023-12-27 13:18:58,295][20347] Updated weights for policy 0, policy_version 1106576 (0.0006) [2023-12-27 13:18:58,361][20347] Updated weights for policy 0, policy_version 1106586 (0.0008) [2023-12-27 13:18:58,427][20347] Updated weights for policy 0, policy_version 1106596 (0.0008) [2023-12-27 13:18:58,768][20381] Updated weights for policy 1, policy_version 1105567 (0.0009) [2023-12-27 13:18:58,844][20381] Updated weights for policy 1, policy_version 1105577 (0.0009) [2023-12-27 13:18:58,924][20381] Updated weights for policy 1, policy_version 1105587 (0.0010) [2023-12-27 13:18:59,208][20347] Updated weights for policy 0, policy_version 1106606 (0.0008) [2023-12-27 13:18:59,280][20347] Updated weights for policy 0, policy_version 1106616 (0.0008) [2023-12-27 13:18:59,341][20347] Updated weights for policy 0, policy_version 1106626 (0.0007) [2023-12-27 13:18:59,724][20381] Updated weights for policy 1, policy_version 1105597 (0.0008) [2023-12-27 13:18:59,773][20381] Updated weights for policy 1, policy_version 1105607 (0.0008) [2023-12-27 13:18:59,827][20381] Updated weights for policy 1, policy_version 1105617 (0.0008) [2023-12-27 13:19:00,014][20347] Updated weights for policy 0, policy_version 1106638 (0.0008) [2023-12-27 13:19:00,067][20347] Updated weights for policy 0, policy_version 1106648 (0.0008) [2023-12-27 13:19:00,126][20347] Updated weights for policy 0, policy_version 1106658 (0.0008) [2023-12-27 13:19:00,434][20381] Updated weights for policy 1, policy_version 1105627 (0.0008) [2023-12-27 13:19:00,492][20381] Updated weights for policy 1, policy_version 1105637 (0.0007) [2023-12-27 13:19:00,541][20381] Updated weights for policy 1, policy_version 1105647 (0.0008) [2023-12-27 13:19:00,901][20347] Updated weights for policy 0, policy_version 1106668 (0.0008) [2023-12-27 13:19:00,948][20347] Updated weights for policy 0, policy_version 1106678 (0.0009) [2023-12-27 13:19:01,001][20347] Updated weights for policy 0, policy_version 1106688 (0.0007) [2023-12-27 13:19:01,165][20381] Updated weights for policy 1, policy_version 1105657 (0.0007) [2023-12-27 13:19:01,230][20381] Updated weights for policy 1, policy_version 1105667 (0.0006) [2023-12-27 13:19:01,293][20381] Updated weights for policy 1, policy_version 1105677 (0.0006) [2023-12-27 13:19:01,355][20381] Updated weights for policy 1, policy_version 1105687 (0.0007) [2023-12-27 13:19:01,823][20347] Updated weights for policy 0, policy_version 1106698 (0.0009) [2023-12-27 13:19:01,883][20347] Updated weights for policy 0, policy_version 1106708 (0.0007) [2023-12-27 13:19:01,929][20347] Updated weights for policy 0, policy_version 1106718 (0.0007) [2023-12-27 13:19:01,980][20347] Updated weights for policy 0, policy_version 1106728 (0.0006) [2023-12-27 13:19:02,028][20381] Updated weights for policy 1, policy_version 1105697 (0.0008) [2023-12-27 13:19:02,074][20381] Updated weights for policy 1, policy_version 1105707 (0.0009) [2023-12-27 13:19:02,130][20381] Updated weights for policy 1, policy_version 1105717 (0.0008) [2023-12-27 13:19:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 566460416. Throughput: 0: 10205.7, 1: 10317.5. Samples: 566467980. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:02,363][19170] Avg episode reward: [(0, '873.110'), (1, '705.630')] [2023-12-27 13:19:02,608][20347] Updated weights for policy 0, policy_version 1106738 (0.0009) [2023-12-27 13:19:02,665][20347] Updated weights for policy 0, policy_version 1106748 (0.0007) [2023-12-27 13:19:02,727][20347] Updated weights for policy 0, policy_version 1106758 (0.0007) [2023-12-27 13:19:02,832][20381] Updated weights for policy 1, policy_version 1105727 (0.0008) [2023-12-27 13:19:02,881][20381] Updated weights for policy 1, policy_version 1105737 (0.0008) [2023-12-27 13:19:02,928][20381] Updated weights for policy 1, policy_version 1105747 (0.0007) [2023-12-27 13:19:03,473][20347] Updated weights for policy 0, policy_version 1106768 (0.0008) [2023-12-27 13:19:03,504][20381] Updated weights for policy 1, policy_version 1105757 (0.0007) [2023-12-27 13:19:03,524][20347] Updated weights for policy 0, policy_version 1106778 (0.0006) [2023-12-27 13:19:03,560][20381] Updated weights for policy 1, policy_version 1105767 (0.0005) [2023-12-27 13:19:03,580][20347] Updated weights for policy 0, policy_version 1106788 (0.0006) [2023-12-27 13:19:03,616][20381] Updated weights for policy 1, policy_version 1105777 (0.0006) [2023-12-27 13:19:04,238][20381] Updated weights for policy 1, policy_version 1105787 (0.0006) [2023-12-27 13:19:04,299][20381] Updated weights for policy 1, policy_version 1105797 (0.0006) [2023-12-27 13:19:04,315][20347] Updated weights for policy 0, policy_version 1106798 (0.0006) [2023-12-27 13:19:04,358][20381] Updated weights for policy 1, policy_version 1105807 (0.0006) [2023-12-27 13:19:04,374][20347] Updated weights for policy 0, policy_version 1106808 (0.0007) [2023-12-27 13:19:04,432][20347] Updated weights for policy 0, policy_version 1106818 (0.0009) [2023-12-27 13:19:05,039][20381] Updated weights for policy 1, policy_version 1105817 (0.0006) [2023-12-27 13:19:05,087][20347] Updated weights for policy 0, policy_version 1106828 (0.0008) [2023-12-27 13:19:05,096][20381] Updated weights for policy 1, policy_version 1105827 (0.0006) [2023-12-27 13:19:05,140][20347] Updated weights for policy 0, policy_version 1106838 (0.0008) [2023-12-27 13:19:05,154][20381] Updated weights for policy 1, policy_version 1105837 (0.0010) [2023-12-27 13:19:05,198][20347] Updated weights for policy 0, policy_version 1106848 (0.0007) [2023-12-27 13:19:05,207][20381] Updated weights for policy 1, policy_version 1105847 (0.0010) [2023-12-27 13:19:05,829][20381] Updated weights for policy 1, policy_version 1105857 (0.0008) [2023-12-27 13:19:05,887][20381] Updated weights for policy 1, policy_version 1105867 (0.0008) [2023-12-27 13:19:05,895][20347] Updated weights for policy 0, policy_version 1106858 (0.0007) [2023-12-27 13:19:05,941][20347] Updated weights for policy 0, policy_version 1106868 (0.0006) [2023-12-27 13:19:05,947][20381] Updated weights for policy 1, policy_version 1105877 (0.0007) [2023-12-27 13:19:06,001][20347] Updated weights for policy 0, policy_version 1106878 (0.0007) [2023-12-27 13:19:06,056][20347] Updated weights for policy 0, policy_version 1106888 (0.0007) [2023-12-27 13:19:06,668][20381] Updated weights for policy 1, policy_version 1105887 (0.0007) [2023-12-27 13:19:06,734][20381] Updated weights for policy 1, policy_version 1105897 (0.0006) [2023-12-27 13:19:06,795][20347] Updated weights for policy 0, policy_version 1106898 (0.0009) [2023-12-27 13:19:06,801][20381] Updated weights for policy 1, policy_version 1105907 (0.0008) [2023-12-27 13:19:06,858][20347] Updated weights for policy 0, policy_version 1106908 (0.0007) [2023-12-27 13:19:06,922][20347] Updated weights for policy 0, policy_version 1106918 (0.0008) [2023-12-27 13:19:07,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 566558720. Throughput: 0: 10177.8, 1: 10398.2. Samples: 566531052. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:07,363][19170] Avg episode reward: [(0, '912.460'), (1, '646.820')] [2023-12-27 13:19:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001106920_283418624.pth... [2023-12-27 13:19:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001105912_283140096.pth... [2023-12-27 13:19:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001105768_283123712.pth [2023-12-27 13:19:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001104696_282828800.pth [2023-12-27 13:19:07,471][20381] Updated weights for policy 1, policy_version 1105917 (0.0007) [2023-12-27 13:19:07,521][20381] Updated weights for policy 1, policy_version 1105927 (0.0007) [2023-12-27 13:19:07,572][20381] Updated weights for policy 1, policy_version 1105937 (0.0010) [2023-12-27 13:19:07,612][20347] Updated weights for policy 0, policy_version 1106928 (0.0006) [2023-12-27 13:19:07,668][20347] Updated weights for policy 0, policy_version 1106938 (0.0005) [2023-12-27 13:19:07,724][20347] Updated weights for policy 0, policy_version 1106948 (0.0006) [2023-12-27 13:19:08,222][20381] Updated weights for policy 1, policy_version 1105947 (0.0009) [2023-12-27 13:19:08,275][20381] Updated weights for policy 1, policy_version 1105957 (0.0006) [2023-12-27 13:19:08,289][20347] Updated weights for policy 0, policy_version 1106958 (0.0005) [2023-12-27 13:19:08,336][20381] Updated weights for policy 1, policy_version 1105967 (0.0007) [2023-12-27 13:19:08,353][20347] Updated weights for policy 0, policy_version 1106968 (0.0006) [2023-12-27 13:19:08,409][20347] Updated weights for policy 0, policy_version 1106978 (0.0006) [2023-12-27 13:19:08,956][20347] Updated weights for policy 0, policy_version 1106988 (0.0006) [2023-12-27 13:19:08,966][20381] Updated weights for policy 1, policy_version 1105977 (0.0007) [2023-12-27 13:19:09,009][20347] Updated weights for policy 0, policy_version 1106998 (0.0006) [2023-12-27 13:19:09,018][20381] Updated weights for policy 1, policy_version 1105987 (0.0007) [2023-12-27 13:19:09,065][20347] Updated weights for policy 0, policy_version 1107008 (0.0006) [2023-12-27 13:19:09,071][20381] Updated weights for policy 1, policy_version 1105997 (0.0005) [2023-12-27 13:19:09,126][20381] Updated weights for policy 1, policy_version 1106007 (0.0008) [2023-12-27 13:19:09,816][20381] Updated weights for policy 1, policy_version 1106017 (0.0006) [2023-12-27 13:19:09,838][20347] Updated weights for policy 0, policy_version 1107018 (0.0009) [2023-12-27 13:19:09,877][20381] Updated weights for policy 1, policy_version 1106027 (0.0009) [2023-12-27 13:19:09,899][20347] Updated weights for policy 0, policy_version 1107028 (0.0005) [2023-12-27 13:19:09,937][20381] Updated weights for policy 1, policy_version 1106037 (0.0008) [2023-12-27 13:19:09,962][20347] Updated weights for policy 0, policy_version 1107038 (0.0006) [2023-12-27 13:19:10,022][20347] Updated weights for policy 0, policy_version 1107048 (0.0007) [2023-12-27 13:19:10,594][20381] Updated weights for policy 1, policy_version 1106047 (0.0008) [2023-12-27 13:19:10,645][20381] Updated weights for policy 1, policy_version 1106057 (0.0007) [2023-12-27 13:19:10,691][20381] Updated weights for policy 1, policy_version 1106067 (0.0005) [2023-12-27 13:19:10,725][20347] Updated weights for policy 0, policy_version 1107058 (0.0007) [2023-12-27 13:19:10,773][20347] Updated weights for policy 0, policy_version 1107068 (0.0007) [2023-12-27 13:19:10,831][20347] Updated weights for policy 0, policy_version 1107078 (0.0006) [2023-12-27 13:19:11,441][20381] Updated weights for policy 1, policy_version 1106077 (0.0007) [2023-12-27 13:19:11,497][20381] Updated weights for policy 1, policy_version 1106087 (0.0008) [2023-12-27 13:19:11,497][20347] Updated weights for policy 0, policy_version 1107088 (0.0007) [2023-12-27 13:19:11,545][20381] Updated weights for policy 1, policy_version 1106097 (0.0008) [2023-12-27 13:19:11,546][20347] Updated weights for policy 0, policy_version 1107098 (0.0006) [2023-12-27 13:19:11,601][20347] Updated weights for policy 0, policy_version 1107108 (0.0006) [2023-12-27 13:19:12,290][20381] Updated weights for policy 1, policy_version 1106107 (0.0008) [2023-12-27 13:19:12,351][20381] Updated weights for policy 1, policy_version 1106117 (0.0007) [2023-12-27 13:19:12,361][20347] Updated weights for policy 0, policy_version 1107118 (0.0008) [2023-12-27 13:19:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 566657024. Throughput: 0: 10172.5, 1: 10380.5. Samples: 566654632. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:12,363][19170] Avg episode reward: [(0, '922.450'), (1, '617.370')] [2023-12-27 13:19:12,418][20381] Updated weights for policy 1, policy_version 1106127 (0.0008) [2023-12-27 13:19:12,425][20347] Updated weights for policy 0, policy_version 1107128 (0.0008) [2023-12-27 13:19:12,482][20347] Updated weights for policy 0, policy_version 1107138 (0.0006) [2023-12-27 13:19:13,161][20347] Updated weights for policy 0, policy_version 1107148 (0.0007) [2023-12-27 13:19:13,172][20381] Updated weights for policy 1, policy_version 1106137 (0.0007) [2023-12-27 13:19:13,218][20347] Updated weights for policy 0, policy_version 1107158 (0.0008) [2023-12-27 13:19:13,231][20381] Updated weights for policy 1, policy_version 1106147 (0.0011) [2023-12-27 13:19:13,268][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000002 [2023-12-27 13:19:13,271][20347] Updated weights for policy 0, policy_version 1107168 (0.0005) [2023-12-27 13:19:13,283][20381] Updated weights for policy 1, policy_version 1106157 (0.0011) [2023-12-27 13:19:13,342][20381] Updated weights for policy 1, policy_version 1106167 (0.0010) [2023-12-27 13:19:13,840][20347] Updated weights for policy 0, policy_version 1107178 (0.0007) [2023-12-27 13:19:13,894][20347] Updated weights for policy 0, policy_version 1107188 (0.0007) [2023-12-27 13:19:13,948][20347] Updated weights for policy 0, policy_version 1107198 (0.0007) [2023-12-27 13:19:13,986][20381] Updated weights for policy 1, policy_version 1106177 (0.0011) [2023-12-27 13:19:14,034][20381] Updated weights for policy 1, policy_version 1106187 (0.0010) [2023-12-27 13:19:14,090][20381] Updated weights for policy 1, policy_version 1106197 (0.0010) [2023-12-27 13:19:14,712][20347] Updated weights for policy 0, policy_version 1107208 (0.0007) [2023-12-27 13:19:14,777][20347] Updated weights for policy 0, policy_version 1107218 (0.0007) [2023-12-27 13:19:14,811][20381] Updated weights for policy 1, policy_version 1106207 (0.0009) [2023-12-27 13:19:14,839][20347] Updated weights for policy 0, policy_version 1107228 (0.0006) [2023-12-27 13:19:14,868][20381] Updated weights for policy 1, policy_version 1106217 (0.0007) [2023-12-27 13:19:14,920][20381] Updated weights for policy 1, policy_version 1106227 (0.0007) [2023-12-27 13:19:15,589][20347] Updated weights for policy 0, policy_version 1107238 (0.0007) [2023-12-27 13:19:15,642][20381] Updated weights for policy 1, policy_version 1106237 (0.0008) [2023-12-27 13:19:15,648][20347] Updated weights for policy 0, policy_version 1107248 (0.0006) [2023-12-27 13:19:15,696][20347] Updated weights for policy 0, policy_version 1107258 (0.0006) [2023-12-27 13:19:15,697][20381] Updated weights for policy 1, policy_version 1106247 (0.0007) [2023-12-27 13:19:15,747][20381] Updated weights for policy 1, policy_version 1106257 (0.0008) [2023-12-27 13:19:16,335][20381] Updated weights for policy 1, policy_version 1106267 (0.0007) [2023-12-27 13:19:16,359][20347] Updated weights for policy 0, policy_version 1107268 (0.0007) [2023-12-27 13:19:16,396][20381] Updated weights for policy 1, policy_version 1106277 (0.0006) [2023-12-27 13:19:16,417][20347] Updated weights for policy 0, policy_version 1107278 (0.0009) [2023-12-27 13:19:16,452][20381] Updated weights for policy 1, policy_version 1106287 (0.0006) [2023-12-27 13:19:16,476][20347] Updated weights for policy 0, policy_version 1107288 (0.0010) [2023-12-27 13:19:17,095][20381] Updated weights for policy 1, policy_version 1106297 (0.0006) [2023-12-27 13:19:17,134][20347] Updated weights for policy 0, policy_version 1107298 (0.0010) [2023-12-27 13:19:17,146][20381] Updated weights for policy 1, policy_version 1106307 (0.0008) [2023-12-27 13:19:17,183][20347] Updated weights for policy 0, policy_version 1107308 (0.0010) [2023-12-27 13:19:17,207][20381] Updated weights for policy 1, policy_version 1106317 (0.0007) [2023-12-27 13:19:17,231][20347] Updated weights for policy 0, policy_version 1107318 (0.0010) [2023-12-27 13:19:17,262][20381] Updated weights for policy 1, policy_version 1106327 (0.0006) [2023-12-27 13:19:17,285][20347] Updated weights for policy 0, policy_version 1107328 (0.0010) [2023-12-27 13:19:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.7, 300 sec: 20493.9). Total num frames: 566771712. Throughput: 0: 10148.1, 1: 10312.7. Samples: 566775752. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:17,363][19170] Avg episode reward: [(0, '931.830'), (1, '695.640')] [2023-12-27 13:19:17,963][20347] Updated weights for policy 0, policy_version 1107338 (0.0007) [2023-12-27 13:19:17,968][20381] Updated weights for policy 1, policy_version 1106337 (0.0008) [2023-12-27 13:19:18,008][20347] Updated weights for policy 0, policy_version 1107348 (0.0007) [2023-12-27 13:19:18,017][20381] Updated weights for policy 1, policy_version 1106347 (0.0007) [2023-12-27 13:19:18,059][20347] Updated weights for policy 0, policy_version 1107358 (0.0008) [2023-12-27 13:19:18,068][20381] Updated weights for policy 1, policy_version 1106357 (0.0005) [2023-12-27 13:19:18,762][20381] Updated weights for policy 1, policy_version 1106367 (0.0007) [2023-12-27 13:19:18,793][20347] Updated weights for policy 0, policy_version 1107368 (0.0010) [2023-12-27 13:19:18,818][20381] Updated weights for policy 1, policy_version 1106377 (0.0005) [2023-12-27 13:19:18,846][20347] Updated weights for policy 0, policy_version 1107378 (0.0010) [2023-12-27 13:19:18,874][20381] Updated weights for policy 1, policy_version 1106387 (0.0006) [2023-12-27 13:19:18,901][20347] Updated weights for policy 0, policy_version 1107388 (0.0010) [2023-12-27 13:19:19,594][20381] Updated weights for policy 1, policy_version 1106397 (0.0006) [2023-12-27 13:19:19,596][20347] Updated weights for policy 0, policy_version 1107398 (0.0008) [2023-12-27 13:19:19,655][20347] Updated weights for policy 0, policy_version 1107408 (0.0006) [2023-12-27 13:19:19,656][20381] Updated weights for policy 1, policy_version 1106407 (0.0008) [2023-12-27 13:19:19,714][20347] Updated weights for policy 0, policy_version 1107418 (0.0006) [2023-12-27 13:19:19,716][20381] Updated weights for policy 1, policy_version 1106417 (0.0006) [2023-12-27 13:19:20,357][20347] Updated weights for policy 0, policy_version 1107428 (0.0007) [2023-12-27 13:19:20,411][20347] Updated weights for policy 0, policy_version 1107438 (0.0008) [2023-12-27 13:19:20,471][20347] Updated weights for policy 0, policy_version 1107448 (0.0008) [2023-12-27 13:19:20,490][20381] Updated weights for policy 1, policy_version 1106427 (0.0007) [2023-12-27 13:19:20,544][20381] Updated weights for policy 1, policy_version 1106437 (0.0007) [2023-12-27 13:19:20,619][20381] Updated weights for policy 1, policy_version 1106447 (0.0009) [2023-12-27 13:19:21,168][20347] Updated weights for policy 0, policy_version 1107458 (0.0006) [2023-12-27 13:19:21,222][20347] Updated weights for policy 0, policy_version 1107468 (0.0008) [2023-12-27 13:19:21,281][20347] Updated weights for policy 0, policy_version 1107478 (0.0008) [2023-12-27 13:19:21,301][20381] Updated weights for policy 1, policy_version 1106457 (0.0008) [2023-12-27 13:19:21,344][20347] Updated weights for policy 0, policy_version 1107488 (0.0007) [2023-12-27 13:19:21,359][20381] Updated weights for policy 1, policy_version 1106467 (0.0007) [2023-12-27 13:19:21,431][20381] Updated weights for policy 1, policy_version 1106477 (0.0009) [2023-12-27 13:19:21,496][20381] Updated weights for policy 1, policy_version 1106487 (0.0007) [2023-12-27 13:19:22,109][20347] Updated weights for policy 0, policy_version 1107498 (0.0006) [2023-12-27 13:19:22,165][20347] Updated weights for policy 0, policy_version 1107508 (0.0005) [2023-12-27 13:19:22,219][20347] Updated weights for policy 0, policy_version 1107518 (0.0006) [2023-12-27 13:19:22,235][20381] Updated weights for policy 1, policy_version 1106497 (0.0009) [2023-12-27 13:19:22,300][20381] Updated weights for policy 1, policy_version 1106507 (0.0008) [2023-12-27 13:19:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 566861824. Throughput: 0: 10076.6, 1: 10324.4. Samples: 566835284. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:22,363][19170] Avg episode reward: [(0, '902.570'), (1, '724.730')] [2023-12-27 13:19:22,366][20381] Updated weights for policy 1, policy_version 1106517 (0.0008) [2023-12-27 13:19:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001107520_283574272.pth... [2023-12-27 13:19:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001106344_283271168.pth [2023-12-27 13:19:22,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001106520_283295744.pth... [2023-12-27 13:19:22,387][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001105304_282984448.pth [2023-12-27 13:19:22,890][20347] Updated weights for policy 0, policy_version 1107528 (0.0008) [2023-12-27 13:19:22,948][20347] Updated weights for policy 0, policy_version 1107538 (0.0009) [2023-12-27 13:19:23,009][20347] Updated weights for policy 0, policy_version 1107548 (0.0008) [2023-12-27 13:19:23,022][20381] Updated weights for policy 1, policy_version 1106527 (0.0007) [2023-12-27 13:19:23,076][20381] Updated weights for policy 1, policy_version 1106537 (0.0006) [2023-12-27 13:19:23,141][20381] Updated weights for policy 1, policy_version 1106547 (0.0011) [2023-12-27 13:19:23,629][20347] Updated weights for policy 0, policy_version 1107558 (0.0006) [2023-12-27 13:19:23,682][20347] Updated weights for policy 0, policy_version 1107568 (0.0006) [2023-12-27 13:19:23,725][20381] Updated weights for policy 1, policy_version 1106557 (0.0008) [2023-12-27 13:19:23,732][20347] Updated weights for policy 0, policy_version 1107578 (0.0008) [2023-12-27 13:19:23,782][20381] Updated weights for policy 1, policy_version 1106567 (0.0006) [2023-12-27 13:19:23,835][20381] Updated weights for policy 1, policy_version 1106577 (0.0010) [2023-12-27 13:19:24,393][20347] Updated weights for policy 0, policy_version 1107588 (0.0006) [2023-12-27 13:19:24,442][20347] Updated weights for policy 0, policy_version 1107598 (0.0007) [2023-12-27 13:19:24,467][20381] Updated weights for policy 1, policy_version 1106587 (0.0009) [2023-12-27 13:19:24,493][20347] Updated weights for policy 0, policy_version 1107608 (0.0006) [2023-12-27 13:19:24,524][20381] Updated weights for policy 1, policy_version 1106597 (0.0005) [2023-12-27 13:19:24,584][20381] Updated weights for policy 1, policy_version 1106607 (0.0005) [2023-12-27 13:19:25,118][20347] Updated weights for policy 0, policy_version 1107618 (0.0006) [2023-12-27 13:19:25,172][20347] Updated weights for policy 0, policy_version 1107628 (0.0010) [2023-12-27 13:19:25,209][20381] Updated weights for policy 1, policy_version 1106617 (0.0010) [2023-12-27 13:19:25,232][20347] Updated weights for policy 0, policy_version 1107638 (0.0010) [2023-12-27 13:19:25,268][20381] Updated weights for policy 1, policy_version 1106627 (0.0005) [2023-12-27 13:19:25,290][20347] Updated weights for policy 0, policy_version 1107648 (0.0010) [2023-12-27 13:19:25,327][20381] Updated weights for policy 1, policy_version 1106637 (0.0007) [2023-12-27 13:19:25,383][20381] Updated weights for policy 1, policy_version 1106647 (0.0010) [2023-12-27 13:19:25,958][20347] Updated weights for policy 0, policy_version 1107658 (0.0010) [2023-12-27 13:19:25,989][20381] Updated weights for policy 1, policy_version 1106657 (0.0010) [2023-12-27 13:19:26,014][20347] Updated weights for policy 0, policy_version 1107668 (0.0010) [2023-12-27 13:19:26,051][20381] Updated weights for policy 1, policy_version 1106667 (0.0005) [2023-12-27 13:19:26,072][20347] Updated weights for policy 0, policy_version 1107678 (0.0010) [2023-12-27 13:19:26,100][20381] Updated weights for policy 1, policy_version 1106677 (0.0006) [2023-12-27 13:19:26,703][20381] Updated weights for policy 1, policy_version 1106687 (0.0006) [2023-12-27 13:19:26,749][20347] Updated weights for policy 0, policy_version 1107688 (0.0007) [2023-12-27 13:19:26,757][20381] Updated weights for policy 1, policy_version 1106697 (0.0010) [2023-12-27 13:19:26,799][20347] Updated weights for policy 0, policy_version 1107698 (0.0006) [2023-12-27 13:19:26,805][20381] Updated weights for policy 1, policy_version 1106707 (0.0010) [2023-12-27 13:19:26,855][20347] Updated weights for policy 0, policy_version 1107708 (0.0006) [2023-12-27 13:19:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 566968320. Throughput: 0: 10077.0, 1: 10382.7. Samples: 566959912. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-12-27 13:19:27,363][19170] Avg episode reward: [(0, '863.930'), (1, '754.480')] [2023-12-27 13:19:27,506][20381] Updated weights for policy 1, policy_version 1106717 (0.0009) [2023-12-27 13:19:27,544][20347] Updated weights for policy 0, policy_version 1107718 (0.0006) [2023-12-27 13:19:27,557][20381] Updated weights for policy 1, policy_version 1106727 (0.0010) [2023-12-27 13:19:27,598][20347] Updated weights for policy 0, policy_version 1107728 (0.0008) [2023-12-27 13:19:27,602][20381] Updated weights for policy 1, policy_version 1106737 (0.0010) [2023-12-27 13:19:27,648][20347] Updated weights for policy 0, policy_version 1107738 (0.0007) [2023-12-27 13:19:28,290][20381] Updated weights for policy 1, policy_version 1106747 (0.0010) [2023-12-27 13:19:28,345][20381] Updated weights for policy 1, policy_version 1106757 (0.0011) [2023-12-27 13:19:28,347][20347] Updated weights for policy 0, policy_version 1107748 (0.0007) [2023-12-27 13:19:28,402][20347] Updated weights for policy 0, policy_version 1107758 (0.0008) [2023-12-27 13:19:28,407][20381] Updated weights for policy 1, policy_version 1106767 (0.0009) [2023-12-27 13:19:28,451][20347] Updated weights for policy 0, policy_version 1107768 (0.0006) [2023-12-27 13:19:29,100][20381] Updated weights for policy 1, policy_version 1106777 (0.0011) [2023-12-27 13:19:29,135][20347] Updated weights for policy 0, policy_version 1107778 (0.0007) [2023-12-27 13:19:29,152][20381] Updated weights for policy 1, policy_version 1106787 (0.0010) [2023-12-27 13:19:29,186][20347] Updated weights for policy 0, policy_version 1107788 (0.0006) [2023-12-27 13:19:29,210][20381] Updated weights for policy 1, policy_version 1106797 (0.0008) [2023-12-27 13:19:29,251][20347] Updated weights for policy 0, policy_version 1107798 (0.0008) [2023-12-27 13:19:29,271][20381] Updated weights for policy 1, policy_version 1106807 (0.0008) [2023-12-27 13:19:29,315][20347] Updated weights for policy 0, policy_version 1107808 (0.0007) [2023-12-27 13:19:29,980][20347] Updated weights for policy 0, policy_version 1107818 (0.0011) [2023-12-27 13:19:30,020][20381] Updated weights for policy 1, policy_version 1106817 (0.0007) [2023-12-27 13:19:30,026][20347] Updated weights for policy 0, policy_version 1107828 (0.0010) [2023-12-27 13:19:30,075][20347] Updated weights for policy 0, policy_version 1107838 (0.0010) [2023-12-27 13:19:30,079][20381] Updated weights for policy 1, policy_version 1106827 (0.0006) [2023-12-27 13:19:30,140][20381] Updated weights for policy 1, policy_version 1106837 (0.0008) [2023-12-27 13:19:30,681][20347] Updated weights for policy 0, policy_version 1107848 (0.0006) [2023-12-27 13:19:30,745][20347] Updated weights for policy 0, policy_version 1107858 (0.0005) [2023-12-27 13:19:30,807][20347] Updated weights for policy 0, policy_version 1107868 (0.0006) [2023-12-27 13:19:30,850][20381] Updated weights for policy 1, policy_version 1106847 (0.0007) [2023-12-27 13:19:30,898][20381] Updated weights for policy 1, policy_version 1106857 (0.0010) [2023-12-27 13:19:30,946][20381] Updated weights for policy 1, policy_version 1106867 (0.0010) [2023-12-27 13:19:31,364][20347] Updated weights for policy 0, policy_version 1107878 (0.0007) [2023-12-27 13:19:31,423][20347] Updated weights for policy 0, policy_version 1107888 (0.0007) [2023-12-27 13:19:31,479][20347] Updated weights for policy 0, policy_version 1107898 (0.0005) [2023-12-27 13:19:31,653][20381] Updated weights for policy 1, policy_version 1106877 (0.0008) [2023-12-27 13:19:31,718][20381] Updated weights for policy 1, policy_version 1106887 (0.0011) [2023-12-27 13:19:31,768][20381] Updated weights for policy 1, policy_version 1106897 (0.0010) [2023-12-27 13:19:32,125][20347] Updated weights for policy 0, policy_version 1107908 (0.0008) [2023-12-27 13:19:32,183][20347] Updated weights for policy 0, policy_version 1107918 (0.0007) [2023-12-27 13:19:32,240][20347] Updated weights for policy 0, policy_version 1107928 (0.0007) [2023-12-27 13:19:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 567074816. Throughput: 0: 10115.0, 1: 10349.0. Samples: 567083412. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:32,363][19170] Avg episode reward: [(0, '864.020'), (1, '764.870')] [2023-12-27 13:19:32,473][20381] Updated weights for policy 1, policy_version 1106907 (0.0009) [2023-12-27 13:19:32,529][20381] Updated weights for policy 1, policy_version 1106917 (0.0008) [2023-12-27 13:19:32,583][20381] Updated weights for policy 1, policy_version 1106927 (0.0007) [2023-12-27 13:19:32,916][20347] Updated weights for policy 0, policy_version 1107938 (0.0007) [2023-12-27 13:19:32,968][20347] Updated weights for policy 0, policy_version 1107948 (0.0006) [2023-12-27 13:19:33,023][20347] Updated weights for policy 0, policy_version 1107958 (0.0007) [2023-12-27 13:19:33,083][20347] Updated weights for policy 0, policy_version 1107968 (0.0008) [2023-12-27 13:19:33,236][20381] Updated weights for policy 1, policy_version 1106937 (0.0008) [2023-12-27 13:19:33,284][20381] Updated weights for policy 1, policy_version 1106947 (0.0007) [2023-12-27 13:19:33,328][20381] Updated weights for policy 1, policy_version 1106957 (0.0008) [2023-12-27 13:19:33,376][20381] Updated weights for policy 1, policy_version 1106967 (0.0007) [2023-12-27 13:19:33,724][20347] Updated weights for policy 0, policy_version 1107978 (0.0005) [2023-12-27 13:19:33,772][20347] Updated weights for policy 0, policy_version 1107988 (0.0005) [2023-12-27 13:19:33,824][20347] Updated weights for policy 0, policy_version 1107998 (0.0005) [2023-12-27 13:19:34,121][20381] Updated weights for policy 1, policy_version 1106977 (0.0008) [2023-12-27 13:19:34,179][20381] Updated weights for policy 1, policy_version 1106987 (0.0008) [2023-12-27 13:19:34,231][20381] Updated weights for policy 1, policy_version 1106997 (0.0008) [2023-12-27 13:19:34,449][20347] Updated weights for policy 0, policy_version 1108008 (0.0009) [2023-12-27 13:19:34,513][20347] Updated weights for policy 0, policy_version 1108018 (0.0011) [2023-12-27 13:19:34,566][20347] Updated weights for policy 0, policy_version 1108028 (0.0010) [2023-12-27 13:19:34,941][20381] Updated weights for policy 1, policy_version 1107007 (0.0007) [2023-12-27 13:19:34,995][20381] Updated weights for policy 1, policy_version 1107017 (0.0008) [2023-12-27 13:19:35,043][20381] Updated weights for policy 1, policy_version 1107027 (0.0007) [2023-12-27 13:19:35,245][20347] Updated weights for policy 0, policy_version 1108038 (0.0010) [2023-12-27 13:19:35,299][20347] Updated weights for policy 0, policy_version 1108048 (0.0010) [2023-12-27 13:19:35,352][20347] Updated weights for policy 0, policy_version 1108058 (0.0008) [2023-12-27 13:19:35,770][20381] Updated weights for policy 1, policy_version 1107037 (0.0008) [2023-12-27 13:19:35,822][20381] Updated weights for policy 1, policy_version 1107047 (0.0007) [2023-12-27 13:19:35,878][20381] Updated weights for policy 1, policy_version 1107057 (0.0007) [2023-12-27 13:19:35,990][20347] Updated weights for policy 0, policy_version 1108068 (0.0006) [2023-12-27 13:19:36,049][20347] Updated weights for policy 0, policy_version 1108078 (0.0006) [2023-12-27 13:19:36,099][20347] Updated weights for policy 0, policy_version 1108088 (0.0007) [2023-12-27 13:19:36,534][20381] Updated weights for policy 1, policy_version 1107067 (0.0007) [2023-12-27 13:19:36,586][20381] Updated weights for policy 1, policy_version 1107077 (0.0007) [2023-12-27 13:19:36,633][20381] Updated weights for policy 1, policy_version 1107087 (0.0008) [2023-12-27 13:19:36,846][20347] Updated weights for policy 0, policy_version 1108098 (0.0007) [2023-12-27 13:19:36,911][20347] Updated weights for policy 0, policy_version 1108108 (0.0008) [2023-12-27 13:19:36,976][20347] Updated weights for policy 0, policy_version 1108118 (0.0009) [2023-12-27 13:19:37,029][20347] Updated weights for policy 0, policy_version 1108128 (0.0011) [2023-12-27 13:19:37,313][20381] Updated weights for policy 1, policy_version 1107097 (0.0006) [2023-12-27 13:19:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 567173120. Throughput: 0: 10128.3, 1: 10282.6. Samples: 567144804. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:37,363][19170] Avg episode reward: [(0, '912.190'), (1, '764.740')] [2023-12-27 13:19:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001108128_283729920.pth... [2023-12-27 13:19:37,369][20381] Updated weights for policy 1, policy_version 1107107 (0.0007) [2023-12-27 13:19:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001106920_283418624.pth [2023-12-27 13:19:37,430][20381] Updated weights for policy 1, policy_version 1107117 (0.0007) [2023-12-27 13:19:37,477][20381] Updated weights for policy 1, policy_version 1107127 (0.0007) [2023-12-27 13:19:37,481][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001107128_283451392.pth... [2023-12-27 13:19:37,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001105912_283140096.pth [2023-12-27 13:19:37,726][20347] Updated weights for policy 0, policy_version 1108138 (0.0005) [2023-12-27 13:19:37,775][20347] Updated weights for policy 0, policy_version 1108148 (0.0005) [2023-12-27 13:19:37,832][20347] Updated weights for policy 0, policy_version 1108158 (0.0005) [2023-12-27 13:19:38,131][20381] Updated weights for policy 1, policy_version 1107137 (0.0005) [2023-12-27 13:19:38,180][20381] Updated weights for policy 1, policy_version 1107147 (0.0006) [2023-12-27 13:19:38,237][20381] Updated weights for policy 1, policy_version 1107157 (0.0007) [2023-12-27 13:19:38,408][20347] Updated weights for policy 0, policy_version 1108168 (0.0007) [2023-12-27 13:19:38,474][20347] Updated weights for policy 0, policy_version 1108178 (0.0007) [2023-12-27 13:19:38,537][20347] Updated weights for policy 0, policy_version 1108188 (0.0008) [2023-12-27 13:19:38,820][20381] Updated weights for policy 1, policy_version 1107167 (0.0007) [2023-12-27 13:19:38,870][20381] Updated weights for policy 1, policy_version 1107177 (0.0008) [2023-12-27 13:19:38,933][20381] Updated weights for policy 1, policy_version 1107187 (0.0008) [2023-12-27 13:19:39,225][20347] Updated weights for policy 0, policy_version 1108198 (0.0008) [2023-12-27 13:19:39,285][20347] Updated weights for policy 0, policy_version 1108208 (0.0008) [2023-12-27 13:19:39,343][20347] Updated weights for policy 0, policy_version 1108218 (0.0008) [2023-12-27 13:19:39,684][20381] Updated weights for policy 1, policy_version 1107197 (0.0008) [2023-12-27 13:19:39,734][20381] Updated weights for policy 1, policy_version 1107207 (0.0008) [2023-12-27 13:19:39,790][20381] Updated weights for policy 1, policy_version 1107217 (0.0009) [2023-12-27 13:19:40,121][20347] Updated weights for policy 0, policy_version 1108228 (0.0008) [2023-12-27 13:19:40,172][20347] Updated weights for policy 0, policy_version 1108238 (0.0007) [2023-12-27 13:19:40,225][20347] Updated weights for policy 0, policy_version 1108248 (0.0005) [2023-12-27 13:19:40,555][20381] Updated weights for policy 1, policy_version 1107227 (0.0007) [2023-12-27 13:19:40,610][20381] Updated weights for policy 1, policy_version 1107237 (0.0007) [2023-12-27 13:19:40,671][20381] Updated weights for policy 1, policy_version 1107247 (0.0009) [2023-12-27 13:19:40,950][20347] Updated weights for policy 0, policy_version 1108258 (0.0007) [2023-12-27 13:19:41,011][20347] Updated weights for policy 0, policy_version 1108268 (0.0009) [2023-12-27 13:19:41,083][20347] Updated weights for policy 0, policy_version 1108278 (0.0010) [2023-12-27 13:19:41,148][20347] Updated weights for policy 0, policy_version 1108288 (0.0010) [2023-12-27 13:19:41,377][20381] Updated weights for policy 1, policy_version 1107257 (0.0006) [2023-12-27 13:19:41,442][20381] Updated weights for policy 1, policy_version 1107267 (0.0007) [2023-12-27 13:19:41,506][20381] Updated weights for policy 1, policy_version 1107277 (0.0005) [2023-12-27 13:19:41,557][20292] KL-divergence is very high: 106.6123 [2023-12-27 13:19:41,571][20381] Updated weights for policy 1, policy_version 1107287 (0.0006) [2023-12-27 13:19:41,874][20347] Updated weights for policy 0, policy_version 1108298 (0.0009) [2023-12-27 13:19:41,934][20347] Updated weights for policy 0, policy_version 1108308 (0.0008) [2023-12-27 13:19:41,993][20347] Updated weights for policy 0, policy_version 1108318 (0.0008) [2023-12-27 13:19:42,242][20381] Updated weights for policy 1, policy_version 1107297 (0.0005) [2023-12-27 13:19:42,308][20381] Updated weights for policy 1, policy_version 1107307 (0.0007) [2023-12-27 13:19:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 567271424. Throughput: 0: 10167.7, 1: 10196.6. Samples: 567266428. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:42,363][19170] Avg episode reward: [(0, '873.190'), (1, '715.200')] [2023-12-27 13:19:42,366][20381] Updated weights for policy 1, policy_version 1107317 (0.0008) [2023-12-27 13:19:42,681][20347] Updated weights for policy 0, policy_version 1108328 (0.0006) [2023-12-27 13:19:42,730][20347] Updated weights for policy 0, policy_version 1108338 (0.0005) [2023-12-27 13:19:42,790][20347] Updated weights for policy 0, policy_version 1108348 (0.0007) [2023-12-27 13:19:43,048][20381] Updated weights for policy 1, policy_version 1107327 (0.0008) [2023-12-27 13:19:43,097][20381] Updated weights for policy 1, policy_version 1107337 (0.0007) [2023-12-27 13:19:43,153][20381] Updated weights for policy 1, policy_version 1107347 (0.0007) [2023-12-27 13:19:43,435][20347] Updated weights for policy 0, policy_version 1108358 (0.0008) [2023-12-27 13:19:43,481][20347] Updated weights for policy 0, policy_version 1108368 (0.0007) [2023-12-27 13:19:43,528][20347] Updated weights for policy 0, policy_version 1108378 (0.0006) [2023-12-27 13:19:43,840][20381] Updated weights for policy 1, policy_version 1107357 (0.0008) [2023-12-27 13:19:43,903][20381] Updated weights for policy 1, policy_version 1107367 (0.0008) [2023-12-27 13:19:43,955][20381] Updated weights for policy 1, policy_version 1107377 (0.0007) [2023-12-27 13:19:44,083][20347] Updated weights for policy 0, policy_version 1108388 (0.0006) [2023-12-27 13:19:44,132][20347] Updated weights for policy 0, policy_version 1108398 (0.0007) [2023-12-27 13:19:44,192][20347] Updated weights for policy 0, policy_version 1108408 (0.0008) [2023-12-27 13:19:44,630][20381] Updated weights for policy 1, policy_version 1107387 (0.0009) [2023-12-27 13:19:44,688][20381] Updated weights for policy 1, policy_version 1107397 (0.0008) [2023-12-27 13:19:44,748][20381] Updated weights for policy 1, policy_version 1107407 (0.0008) [2023-12-27 13:19:44,843][20347] Updated weights for policy 0, policy_version 1108418 (0.0007) [2023-12-27 13:19:44,904][20347] Updated weights for policy 0, policy_version 1108428 (0.0009) [2023-12-27 13:19:44,968][20347] Updated weights for policy 0, policy_version 1108438 (0.0007) [2023-12-27 13:19:45,025][20347] Updated weights for policy 0, policy_version 1108448 (0.0008) [2023-12-27 13:19:45,525][20381] Updated weights for policy 1, policy_version 1107417 (0.0007) [2023-12-27 13:19:45,577][20381] Updated weights for policy 1, policy_version 1107427 (0.0007) [2023-12-27 13:19:45,623][20381] Updated weights for policy 1, policy_version 1107437 (0.0008) [2023-12-27 13:19:45,633][20347] Updated weights for policy 0, policy_version 1108458 (0.0005) [2023-12-27 13:19:45,671][20381] Updated weights for policy 1, policy_version 1107447 (0.0007) [2023-12-27 13:19:45,682][20347] Updated weights for policy 0, policy_version 1108468 (0.0005) [2023-12-27 13:19:45,734][20347] Updated weights for policy 0, policy_version 1108478 (0.0005) [2023-12-27 13:19:46,341][20347] Updated weights for policy 0, policy_version 1108488 (0.0008) [2023-12-27 13:19:46,397][20347] Updated weights for policy 0, policy_version 1108498 (0.0006) [2023-12-27 13:19:46,423][20381] Updated weights for policy 1, policy_version 1107457 (0.0008) [2023-12-27 13:19:46,453][20347] Updated weights for policy 0, policy_version 1108508 (0.0010) [2023-12-27 13:19:46,472][20381] Updated weights for policy 1, policy_version 1107467 (0.0009) [2023-12-27 13:19:46,522][20381] Updated weights for policy 1, policy_version 1107477 (0.0009) [2023-12-27 13:19:47,122][20347] Updated weights for policy 0, policy_version 1108518 (0.0007) [2023-12-27 13:19:47,183][20347] Updated weights for policy 0, policy_version 1108528 (0.0010) [2023-12-27 13:19:47,237][20381] Updated weights for policy 1, policy_version 1107487 (0.0009) [2023-12-27 13:19:47,243][20347] Updated weights for policy 0, policy_version 1108538 (0.0011) [2023-12-27 13:19:47,290][20381] Updated weights for policy 1, policy_version 1107497 (0.0010) [2023-12-27 13:19:47,335][20381] Updated weights for policy 1, policy_version 1107507 (0.0010) [2023-12-27 13:19:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 567386112. Throughput: 0: 10307.1, 1: 10158.9. Samples: 567388948. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:47,363][19170] Avg episode reward: [(0, '912.180'), (1, '597.390')] [2023-12-27 13:19:47,797][20347] Updated weights for policy 0, policy_version 1108548 (0.0007) [2023-12-27 13:19:47,853][20347] Updated weights for policy 0, policy_version 1108558 (0.0005) [2023-12-27 13:19:47,888][20381] Updated weights for policy 1, policy_version 1107517 (0.0008) [2023-12-27 13:19:47,914][20347] Updated weights for policy 0, policy_version 1108568 (0.0008) [2023-12-27 13:19:47,943][20381] Updated weights for policy 1, policy_version 1107527 (0.0009) [2023-12-27 13:19:47,998][20381] Updated weights for policy 1, policy_version 1107537 (0.0010) [2023-12-27 13:19:48,569][20347] Updated weights for policy 0, policy_version 1108578 (0.0006) [2023-12-27 13:19:48,630][20347] Updated weights for policy 0, policy_version 1108588 (0.0005) [2023-12-27 13:19:48,699][20347] Updated weights for policy 0, policy_version 1108598 (0.0006) [2023-12-27 13:19:48,709][20381] Updated weights for policy 1, policy_version 1107547 (0.0009) [2023-12-27 13:19:48,758][20347] Updated weights for policy 0, policy_version 1108608 (0.0007) [2023-12-27 13:19:48,770][20381] Updated weights for policy 1, policy_version 1107557 (0.0005) [2023-12-27 13:19:48,833][20381] Updated weights for policy 1, policy_version 1107567 (0.0006) [2023-12-27 13:19:49,465][20347] Updated weights for policy 0, policy_version 1108618 (0.0007) [2023-12-27 13:19:49,488][20381] Updated weights for policy 1, policy_version 1107577 (0.0010) [2023-12-27 13:19:49,523][20347] Updated weights for policy 0, policy_version 1108628 (0.0006) [2023-12-27 13:19:49,547][20381] Updated weights for policy 1, policy_version 1107587 (0.0010) [2023-12-27 13:19:49,587][20347] Updated weights for policy 0, policy_version 1108638 (0.0007) [2023-12-27 13:19:49,609][20381] Updated weights for policy 1, policy_version 1107597 (0.0005) [2023-12-27 13:19:49,677][20381] Updated weights for policy 1, policy_version 1107607 (0.0007) [2023-12-27 13:19:50,282][20347] Updated weights for policy 0, policy_version 1108648 (0.0008) [2023-12-27 13:19:50,345][20347] Updated weights for policy 0, policy_version 1108658 (0.0008) [2023-12-27 13:19:50,378][20381] Updated weights for policy 1, policy_version 1107617 (0.0006) [2023-12-27 13:19:50,408][20347] Updated weights for policy 0, policy_version 1108668 (0.0006) [2023-12-27 13:19:50,433][20381] Updated weights for policy 1, policy_version 1107627 (0.0008) [2023-12-27 13:19:50,489][20381] Updated weights for policy 1, policy_version 1107637 (0.0008) [2023-12-27 13:19:51,049][20347] Updated weights for policy 0, policy_version 1108678 (0.0007) [2023-12-27 13:19:51,115][20347] Updated weights for policy 0, policy_version 1108688 (0.0008) [2023-12-27 13:19:51,183][20347] Updated weights for policy 0, policy_version 1108698 (0.0009) [2023-12-27 13:19:51,199][20381] Updated weights for policy 1, policy_version 1107647 (0.0010) [2023-12-27 13:19:51,259][20381] Updated weights for policy 1, policy_version 1107657 (0.0007) [2023-12-27 13:19:51,318][20381] Updated weights for policy 1, policy_version 1107667 (0.0011) [2023-12-27 13:19:51,926][20347] Updated weights for policy 0, policy_version 1108708 (0.0007) [2023-12-27 13:19:51,981][20347] Updated weights for policy 0, policy_version 1108718 (0.0008) [2023-12-27 13:19:52,020][20381] Updated weights for policy 1, policy_version 1107677 (0.0009) [2023-12-27 13:19:52,033][20347] Updated weights for policy 0, policy_version 1108728 (0.0008) [2023-12-27 13:19:52,079][20381] Updated weights for policy 1, policy_version 1107687 (0.0007) [2023-12-27 13:19:52,137][20381] Updated weights for policy 1, policy_version 1107697 (0.0007) [2023-12-27 13:19:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 567484416. Throughput: 0: 10318.1, 1: 10146.7. Samples: 567451968. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:52,363][19170] Avg episode reward: [(0, '941.700'), (1, '695.170')] [2023-12-27 13:19:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001107704_283598848.pth... [2023-12-27 13:19:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001108736_283885568.pth... [2023-12-27 13:19:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001106520_283295744.pth [2023-12-27 13:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001107520_283574272.pth [2023-12-27 13:19:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001108736_283885568.pth [2023-12-27 13:19:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001107704_283598848.pth [2023-12-27 13:19:52,811][20381] Updated weights for policy 1, policy_version 1107707 (0.0007) [2023-12-27 13:19:52,849][20347] Updated weights for policy 0, policy_version 1108738 (0.0008) [2023-12-27 13:19:52,866][20381] Updated weights for policy 1, policy_version 1107717 (0.0007) [2023-12-27 13:19:52,911][20347] Updated weights for policy 0, policy_version 1108748 (0.0006) [2023-12-27 13:19:52,924][20381] Updated weights for policy 1, policy_version 1107727 (0.0007) [2023-12-27 13:19:52,964][20347] Updated weights for policy 0, policy_version 1108758 (0.0006) [2023-12-27 13:19:53,018][20347] Updated weights for policy 0, policy_version 1108768 (0.0007) [2023-12-27 13:19:53,643][20381] Updated weights for policy 1, policy_version 1107737 (0.0006) [2023-12-27 13:19:53,681][20347] Updated weights for policy 0, policy_version 1108778 (0.0006) [2023-12-27 13:19:53,694][20381] Updated weights for policy 1, policy_version 1107747 (0.0006) [2023-12-27 13:19:53,736][20347] Updated weights for policy 0, policy_version 1108788 (0.0006) [2023-12-27 13:19:53,742][20381] Updated weights for policy 1, policy_version 1107757 (0.0005) [2023-12-27 13:19:53,785][20347] Updated weights for policy 0, policy_version 1108798 (0.0005) [2023-12-27 13:19:53,794][20381] Updated weights for policy 1, policy_version 1107767 (0.0006) [2023-12-27 13:19:54,494][20347] Updated weights for policy 0, policy_version 1108808 (0.0006) [2023-12-27 13:19:54,495][20381] Updated weights for policy 1, policy_version 1107777 (0.0008) [2023-12-27 13:19:54,547][20347] Updated weights for policy 0, policy_version 1108818 (0.0005) [2023-12-27 13:19:54,550][20381] Updated weights for policy 1, policy_version 1107787 (0.0007) [2023-12-27 13:19:54,606][20381] Updated weights for policy 1, policy_version 1107797 (0.0007) [2023-12-27 13:19:54,610][20347] Updated weights for policy 0, policy_version 1108828 (0.0005) [2023-12-27 13:19:55,174][20347] Updated weights for policy 0, policy_version 1108838 (0.0007) [2023-12-27 13:19:55,224][20347] Updated weights for policy 0, policy_version 1108848 (0.0005) [2023-12-27 13:19:55,275][20347] Updated weights for policy 0, policy_version 1108858 (0.0005) [2023-12-27 13:19:55,301][20381] Updated weights for policy 1, policy_version 1107807 (0.0008) [2023-12-27 13:19:55,353][20381] Updated weights for policy 1, policy_version 1107817 (0.0008) [2023-12-27 13:19:55,411][20381] Updated weights for policy 1, policy_version 1107827 (0.0007) [2023-12-27 13:19:55,902][20347] Updated weights for policy 0, policy_version 1108868 (0.0007) [2023-12-27 13:19:55,957][20347] Updated weights for policy 0, policy_version 1108878 (0.0010) [2023-12-27 13:19:56,006][20347] Updated weights for policy 0, policy_version 1108888 (0.0010) [2023-12-27 13:19:56,097][20381] Updated weights for policy 1, policy_version 1107837 (0.0007) [2023-12-27 13:19:56,154][20381] Updated weights for policy 1, policy_version 1107847 (0.0006) [2023-12-27 13:19:56,208][20381] Updated weights for policy 1, policy_version 1107857 (0.0008) [2023-12-27 13:19:56,659][20347] Updated weights for policy 0, policy_version 1108898 (0.0010) [2023-12-27 13:19:56,716][20347] Updated weights for policy 0, policy_version 1108908 (0.0010) [2023-12-27 13:19:56,769][20347] Updated weights for policy 0, policy_version 1108918 (0.0010) [2023-12-27 13:19:56,822][20347] Updated weights for policy 0, policy_version 1108928 (0.0010) [2023-12-27 13:19:56,896][20381] Updated weights for policy 1, policy_version 1107867 (0.0008) [2023-12-27 13:19:56,952][20381] Updated weights for policy 1, policy_version 1107877 (0.0007) [2023-12-27 13:19:57,004][20381] Updated weights for policy 1, policy_version 1107887 (0.0007) [2023-12-27 13:19:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 567582720. Throughput: 0: 10308.3, 1: 10094.7. Samples: 567572764. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:19:57,363][19170] Avg episode reward: [(0, '931.960'), (1, '732.580')] [2023-12-27 13:19:57,499][20347] Updated weights for policy 0, policy_version 1108938 (0.0010) [2023-12-27 13:19:57,555][20347] Updated weights for policy 0, policy_version 1108948 (0.0010) [2023-12-27 13:19:57,602][20347] Updated weights for policy 0, policy_version 1108958 (0.0010) [2023-12-27 13:19:57,632][20381] Updated weights for policy 1, policy_version 1107897 (0.0007) [2023-12-27 13:19:57,688][20381] Updated weights for policy 1, policy_version 1107907 (0.0007) [2023-12-27 13:19:57,734][20381] Updated weights for policy 1, policy_version 1107917 (0.0007) [2023-12-27 13:19:57,780][20381] Updated weights for policy 1, policy_version 1107927 (0.0007) [2023-12-27 13:19:58,271][20347] Updated weights for policy 0, policy_version 1108968 (0.0008) [2023-12-27 13:19:58,334][20347] Updated weights for policy 0, policy_version 1108978 (0.0007) [2023-12-27 13:19:58,402][20347] Updated weights for policy 0, policy_version 1108988 (0.0007) [2023-12-27 13:19:58,485][20381] Updated weights for policy 1, policy_version 1107937 (0.0007) [2023-12-27 13:19:58,545][20381] Updated weights for policy 1, policy_version 1107947 (0.0008) [2023-12-27 13:19:58,614][20381] Updated weights for policy 1, policy_version 1107958 (0.0007) [2023-12-27 13:19:59,183][20347] Updated weights for policy 0, policy_version 1108998 (0.0011) [2023-12-27 13:19:59,256][20347] Updated weights for policy 0, policy_version 1109008 (0.0010) [2023-12-27 13:19:59,312][20347] Updated weights for policy 0, policy_version 1109018 (0.0006) [2023-12-27 13:19:59,370][20381] Updated weights for policy 1, policy_version 1107968 (0.0009) [2023-12-27 13:19:59,422][20381] Updated weights for policy 1, policy_version 1107978 (0.0008) [2023-12-27 13:19:59,473][20381] Updated weights for policy 1, policy_version 1107988 (0.0008) [2023-12-27 13:19:59,892][20347] Updated weights for policy 0, policy_version 1109028 (0.0006) [2023-12-27 13:19:59,956][20347] Updated weights for policy 0, policy_version 1109038 (0.0008) [2023-12-27 13:20:00,015][20347] Updated weights for policy 0, policy_version 1109048 (0.0007) [2023-12-27 13:20:00,222][20381] Updated weights for policy 1, policy_version 1107998 (0.0008) [2023-12-27 13:20:00,285][20381] Updated weights for policy 1, policy_version 1108008 (0.0007) [2023-12-27 13:20:00,352][20381] Updated weights for policy 1, policy_version 1108018 (0.0006) [2023-12-27 13:20:00,688][20347] Updated weights for policy 0, policy_version 1109058 (0.0008) [2023-12-27 13:20:00,739][20347] Updated weights for policy 0, policy_version 1109068 (0.0007) [2023-12-27 13:20:00,792][20347] Updated weights for policy 0, policy_version 1109078 (0.0008) [2023-12-27 13:20:00,836][20347] Updated weights for policy 0, policy_version 1109088 (0.0008) [2023-12-27 13:20:00,929][20381] Updated weights for policy 1, policy_version 1108028 (0.0007) [2023-12-27 13:20:00,982][20381] Updated weights for policy 1, policy_version 1108038 (0.0007) [2023-12-27 13:20:01,068][20381] Updated weights for policy 1, policy_version 1108048 (0.0007) [2023-12-27 13:20:01,550][20347] Updated weights for policy 0, policy_version 1109098 (0.0007) [2023-12-27 13:20:01,602][20347] Updated weights for policy 0, policy_version 1109108 (0.0007) [2023-12-27 13:20:01,662][20347] Updated weights for policy 0, policy_version 1109118 (0.0008) [2023-12-27 13:20:01,699][20381] Updated weights for policy 1, policy_version 1108058 (0.0007) [2023-12-27 13:20:01,759][20381] Updated weights for policy 1, policy_version 1108068 (0.0007) [2023-12-27 13:20:01,816][20381] Updated weights for policy 1, policy_version 1108078 (0.0007) [2023-12-27 13:20:01,874][20381] Updated weights for policy 1, policy_version 1108088 (0.0008) [2023-12-27 13:20:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 567681024. Throughput: 0: 10320.2, 1: 10100.2. Samples: 567694672. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:02,363][19170] Avg episode reward: [(0, '931.830'), (1, '681.740')] [2023-12-27 13:20:02,387][20347] Updated weights for policy 0, policy_version 1109128 (0.0007) [2023-12-27 13:20:02,446][20347] Updated weights for policy 0, policy_version 1109138 (0.0007) [2023-12-27 13:20:02,499][20347] Updated weights for policy 0, policy_version 1109148 (0.0007) [2023-12-27 13:20:02,559][20381] Updated weights for policy 1, policy_version 1108098 (0.0007) [2023-12-27 13:20:02,608][20381] Updated weights for policy 1, policy_version 1108108 (0.0010) [2023-12-27 13:20:02,667][20381] Updated weights for policy 1, policy_version 1108118 (0.0010) [2023-12-27 13:20:03,217][20347] Updated weights for policy 0, policy_version 1109158 (0.0007) [2023-12-27 13:20:03,265][20347] Updated weights for policy 0, policy_version 1109168 (0.0008) [2023-12-27 13:20:03,306][20381] Updated weights for policy 1, policy_version 1108128 (0.0006) [2023-12-27 13:20:03,316][20347] Updated weights for policy 0, policy_version 1109178 (0.0010) [2023-12-27 13:20:03,359][20381] Updated weights for policy 1, policy_version 1108138 (0.0006) [2023-12-27 13:20:03,416][20381] Updated weights for policy 1, policy_version 1108148 (0.0006) [2023-12-27 13:20:04,022][20381] Updated weights for policy 1, policy_version 1108158 (0.0010) [2023-12-27 13:20:04,022][20347] Updated weights for policy 0, policy_version 1109188 (0.0009) [2023-12-27 13:20:04,078][20381] Updated weights for policy 1, policy_version 1108168 (0.0011) [2023-12-27 13:20:04,087][20347] Updated weights for policy 0, policy_version 1109198 (0.0008) [2023-12-27 13:20:04,132][20381] Updated weights for policy 1, policy_version 1108178 (0.0011) [2023-12-27 13:20:04,151][20347] Updated weights for policy 0, policy_version 1109208 (0.0008) [2023-12-27 13:20:04,755][20381] Updated weights for policy 1, policy_version 1108188 (0.0009) [2023-12-27 13:20:04,763][20347] Updated weights for policy 0, policy_version 1109218 (0.0007) [2023-12-27 13:20:04,803][20381] Updated weights for policy 1, policy_version 1108198 (0.0005) [2023-12-27 13:20:04,812][20347] Updated weights for policy 0, policy_version 1109228 (0.0007) [2023-12-27 13:20:04,849][20381] Updated weights for policy 1, policy_version 1108208 (0.0005) [2023-12-27 13:20:04,868][20347] Updated weights for policy 0, policy_version 1109238 (0.0006) [2023-12-27 13:20:04,930][20347] Updated weights for policy 0, policy_version 1109248 (0.0005) [2023-12-27 13:20:05,481][20381] Updated weights for policy 1, policy_version 1108218 (0.0005) [2023-12-27 13:20:05,483][20347] Updated weights for policy 0, policy_version 1109258 (0.0007) [2023-12-27 13:20:05,526][20381] Updated weights for policy 1, policy_version 1108228 (0.0007) [2023-12-27 13:20:05,536][20347] Updated weights for policy 0, policy_version 1109268 (0.0005) [2023-12-27 13:20:05,588][20381] Updated weights for policy 1, policy_version 1108238 (0.0007) [2023-12-27 13:20:05,593][20347] Updated weights for policy 0, policy_version 1109278 (0.0005) [2023-12-27 13:20:05,633][20381] Updated weights for policy 1, policy_version 1108248 (0.0008) [2023-12-27 13:20:06,221][20347] Updated weights for policy 0, policy_version 1109288 (0.0007) [2023-12-27 13:20:06,280][20347] Updated weights for policy 0, policy_version 1109298 (0.0008) [2023-12-27 13:20:06,339][20347] Updated weights for policy 0, policy_version 1109308 (0.0006) [2023-12-27 13:20:06,356][20381] Updated weights for policy 1, policy_version 1108258 (0.0007) [2023-12-27 13:20:06,415][20381] Updated weights for policy 1, policy_version 1108268 (0.0007) [2023-12-27 13:20:06,477][20381] Updated weights for policy 1, policy_version 1108278 (0.0007) [2023-12-27 13:20:07,045][20347] Updated weights for policy 0, policy_version 1109318 (0.0007) [2023-12-27 13:20:07,107][20347] Updated weights for policy 0, policy_version 1109328 (0.0006) [2023-12-27 13:20:07,168][20347] Updated weights for policy 0, policy_version 1109338 (0.0007) [2023-12-27 13:20:07,200][20381] Updated weights for policy 1, policy_version 1108288 (0.0007) [2023-12-27 13:20:07,258][20381] Updated weights for policy 1, policy_version 1108298 (0.0005) [2023-12-27 13:20:07,312][20381] Updated weights for policy 1, policy_version 1108308 (0.0005) [2023-12-27 13:20:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 567795712. Throughput: 0: 10341.5, 1: 10177.0. Samples: 567758620. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:07,364][19170] Avg episode reward: [(0, '902.970'), (1, '635.300')] [2023-12-27 13:20:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001108312_283754496.pth... [2023-12-27 13:20:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001109344_284041216.pth... [2023-12-27 13:20:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001107128_283451392.pth [2023-12-27 13:20:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001108128_283729920.pth [2023-12-27 13:20:07,768][20347] Updated weights for policy 0, policy_version 1109348 (0.0008) [2023-12-27 13:20:07,813][20347] Updated weights for policy 0, policy_version 1109358 (0.0007) [2023-12-27 13:20:07,869][20347] Updated weights for policy 0, policy_version 1109368 (0.0005) [2023-12-27 13:20:07,902][20381] Updated weights for policy 1, policy_version 1108318 (0.0008) [2023-12-27 13:20:07,958][20381] Updated weights for policy 1, policy_version 1108328 (0.0007) [2023-12-27 13:20:08,012][20381] Updated weights for policy 1, policy_version 1108338 (0.0007) [2023-12-27 13:20:08,546][20347] Updated weights for policy 0, policy_version 1109378 (0.0007) [2023-12-27 13:20:08,603][20347] Updated weights for policy 0, policy_version 1109388 (0.0007) [2023-12-27 13:20:08,653][20347] Updated weights for policy 0, policy_version 1109398 (0.0006) [2023-12-27 13:20:08,705][20347] Updated weights for policy 0, policy_version 1109408 (0.0006) [2023-12-27 13:20:08,711][20381] Updated weights for policy 1, policy_version 1108348 (0.0008) [2023-12-27 13:20:08,760][20381] Updated weights for policy 1, policy_version 1108358 (0.0009) [2023-12-27 13:20:08,809][20381] Updated weights for policy 1, policy_version 1108368 (0.0009) [2023-12-27 13:20:09,426][20347] Updated weights for policy 0, policy_version 1109418 (0.0007) [2023-12-27 13:20:09,482][20347] Updated weights for policy 0, policy_version 1109428 (0.0008) [2023-12-27 13:20:09,541][20347] Updated weights for policy 0, policy_version 1109438 (0.0007) [2023-12-27 13:20:09,588][20381] Updated weights for policy 1, policy_version 1108378 (0.0009) [2023-12-27 13:20:09,649][20381] Updated weights for policy 1, policy_version 1108388 (0.0007) [2023-12-27 13:20:09,717][20381] Updated weights for policy 1, policy_version 1108398 (0.0006) [2023-12-27 13:20:09,782][20381] Updated weights for policy 1, policy_version 1108408 (0.0010) [2023-12-27 13:20:10,326][20347] Updated weights for policy 0, policy_version 1109448 (0.0008) [2023-12-27 13:20:10,391][20347] Updated weights for policy 0, policy_version 1109458 (0.0007) [2023-12-27 13:20:10,400][20381] Updated weights for policy 1, policy_version 1108418 (0.0010) [2023-12-27 13:20:10,452][20347] Updated weights for policy 0, policy_version 1109468 (0.0006) [2023-12-27 13:20:10,458][20381] Updated weights for policy 1, policy_version 1108428 (0.0008) [2023-12-27 13:20:10,510][20381] Updated weights for policy 1, policy_version 1108438 (0.0008) [2023-12-27 13:20:11,153][20347] Updated weights for policy 0, policy_version 1109478 (0.0007) [2023-12-27 13:20:11,170][20381] Updated weights for policy 1, policy_version 1108448 (0.0008) [2023-12-27 13:20:11,217][20347] Updated weights for policy 0, policy_version 1109488 (0.0010) [2023-12-27 13:20:11,230][20381] Updated weights for policy 1, policy_version 1108458 (0.0006) [2023-12-27 13:20:11,278][20347] Updated weights for policy 0, policy_version 1109498 (0.0011) [2023-12-27 13:20:11,293][20381] Updated weights for policy 1, policy_version 1108468 (0.0009) [2023-12-27 13:20:11,988][20347] Updated weights for policy 0, policy_version 1109508 (0.0008) [2023-12-27 13:20:12,042][20347] Updated weights for policy 0, policy_version 1109518 (0.0008) [2023-12-27 13:20:12,087][20381] Updated weights for policy 1, policy_version 1108478 (0.0008) [2023-12-27 13:20:12,100][20347] Updated weights for policy 0, policy_version 1109528 (0.0008) [2023-12-27 13:20:12,148][20381] Updated weights for policy 1, policy_version 1108488 (0.0007) [2023-12-27 13:20:12,209][20381] Updated weights for policy 1, policy_version 1108498 (0.0006) [2023-12-27 13:20:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 567894016. Throughput: 0: 10330.6, 1: 10135.7. Samples: 567880896. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:12,363][19170] Avg episode reward: [(0, '883.610'), (1, '555.690')] [2023-12-27 13:20:12,831][20347] Updated weights for policy 0, policy_version 1109538 (0.0009) [2023-12-27 13:20:12,847][20381] Updated weights for policy 1, policy_version 1108508 (0.0007) [2023-12-27 13:20:12,890][20347] Updated weights for policy 0, policy_version 1109548 (0.0007) [2023-12-27 13:20:12,906][20381] Updated weights for policy 1, policy_version 1108518 (0.0008) [2023-12-27 13:20:12,946][20347] Updated weights for policy 0, policy_version 1109558 (0.0006) [2023-12-27 13:20:12,959][20381] Updated weights for policy 1, policy_version 1108528 (0.0006) [2023-12-27 13:20:12,998][20347] Updated weights for policy 0, policy_version 1109568 (0.0007) [2023-12-27 13:20:13,598][20347] Updated weights for policy 0, policy_version 1109578 (0.0005) [2023-12-27 13:20:13,658][20347] Updated weights for policy 0, policy_version 1109588 (0.0006) [2023-12-27 13:20:13,696][20381] Updated weights for policy 1, policy_version 1108538 (0.0006) [2023-12-27 13:20:13,714][20347] Updated weights for policy 0, policy_version 1109598 (0.0007) [2023-12-27 13:20:13,745][20381] Updated weights for policy 1, policy_version 1108548 (0.0008) [2023-12-27 13:20:13,800][20381] Updated weights for policy 1, policy_version 1108558 (0.0007) [2023-12-27 13:20:13,853][20381] Updated weights for policy 1, policy_version 1108568 (0.0005) [2023-12-27 13:20:14,335][20347] Updated weights for policy 0, policy_version 1109608 (0.0007) [2023-12-27 13:20:14,383][20347] Updated weights for policy 0, policy_version 1109618 (0.0007) [2023-12-27 13:20:14,436][20347] Updated weights for policy 0, policy_version 1109628 (0.0008) [2023-12-27 13:20:14,526][20381] Updated weights for policy 1, policy_version 1108578 (0.0008) [2023-12-27 13:20:14,582][20381] Updated weights for policy 1, policy_version 1108588 (0.0008) [2023-12-27 13:20:14,644][20381] Updated weights for policy 1, policy_version 1108598 (0.0008) [2023-12-27 13:20:15,088][20347] Updated weights for policy 0, policy_version 1109638 (0.0006) [2023-12-27 13:20:15,150][20347] Updated weights for policy 0, policy_version 1109648 (0.0006) [2023-12-27 13:20:15,210][20347] Updated weights for policy 0, policy_version 1109658 (0.0007) [2023-12-27 13:20:15,433][20381] Updated weights for policy 1, policy_version 1108608 (0.0007) [2023-12-27 13:20:15,477][20381] Updated weights for policy 1, policy_version 1108618 (0.0007) [2023-12-27 13:20:15,525][20381] Updated weights for policy 1, policy_version 1108628 (0.0008) [2023-12-27 13:20:15,857][20347] Updated weights for policy 0, policy_version 1109668 (0.0007) [2023-12-27 13:20:15,908][20347] Updated weights for policy 0, policy_version 1109678 (0.0008) [2023-12-27 13:20:15,954][20347] Updated weights for policy 0, policy_version 1109688 (0.0005) [2023-12-27 13:20:16,256][20381] Updated weights for policy 1, policy_version 1108638 (0.0006) [2023-12-27 13:20:16,302][20381] Updated weights for policy 1, policy_version 1108648 (0.0005) [2023-12-27 13:20:16,351][20381] Updated weights for policy 1, policy_version 1108658 (0.0005) [2023-12-27 13:20:16,584][20347] Updated weights for policy 0, policy_version 1109698 (0.0008) [2023-12-27 13:20:16,633][20347] Updated weights for policy 0, policy_version 1109708 (0.0009) [2023-12-27 13:20:16,686][20347] Updated weights for policy 0, policy_version 1109718 (0.0009) [2023-12-27 13:20:16,745][20347] Updated weights for policy 0, policy_version 1109728 (0.0007) [2023-12-27 13:20:16,908][20381] Updated weights for policy 1, policy_version 1108668 (0.0006) [2023-12-27 13:20:16,959][20381] Updated weights for policy 1, policy_version 1108678 (0.0007) [2023-12-27 13:20:17,023][20381] Updated weights for policy 1, policy_version 1108688 (0.0008) [2023-12-27 13:20:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.4, 300 sec: 20438.4). Total num frames: 567992320. Throughput: 0: 10305.0, 1: 10132.1. Samples: 568003080. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:17,363][19170] Avg episode reward: [(0, '893.330'), (1, '568.220')] [2023-12-27 13:20:17,405][20347] Updated weights for policy 0, policy_version 1109738 (0.0010) [2023-12-27 13:20:17,461][20347] Updated weights for policy 0, policy_version 1109748 (0.0010) [2023-12-27 13:20:17,513][20347] Updated weights for policy 0, policy_version 1109758 (0.0010) [2023-12-27 13:20:17,740][20381] Updated weights for policy 1, policy_version 1108698 (0.0007) [2023-12-27 13:20:17,796][20381] Updated weights for policy 1, policy_version 1108708 (0.0007) [2023-12-27 13:20:17,851][20381] Updated weights for policy 1, policy_version 1108718 (0.0007) [2023-12-27 13:20:17,902][20381] Updated weights for policy 1, policy_version 1108728 (0.0007) [2023-12-27 13:20:18,189][20347] Updated weights for policy 0, policy_version 1109768 (0.0011) [2023-12-27 13:20:18,238][20347] Updated weights for policy 0, policy_version 1109778 (0.0010) [2023-12-27 13:20:18,292][20347] Updated weights for policy 0, policy_version 1109788 (0.0010) [2023-12-27 13:20:18,635][20381] Updated weights for policy 1, policy_version 1108738 (0.0008) [2023-12-27 13:20:18,687][20381] Updated weights for policy 1, policy_version 1108748 (0.0007) [2023-12-27 13:20:18,739][20381] Updated weights for policy 1, policy_version 1108758 (0.0007) [2023-12-27 13:20:18,949][20347] Updated weights for policy 0, policy_version 1109798 (0.0008) [2023-12-27 13:20:18,999][20347] Updated weights for policy 0, policy_version 1109808 (0.0005) [2023-12-27 13:20:19,042][20347] Updated weights for policy 0, policy_version 1109818 (0.0005) [2023-12-27 13:20:19,417][20381] Updated weights for policy 1, policy_version 1108768 (0.0010) [2023-12-27 13:20:19,470][20381] Updated weights for policy 1, policy_version 1108778 (0.0011) [2023-12-27 13:20:19,533][20381] Updated weights for policy 1, policy_version 1108788 (0.0010) [2023-12-27 13:20:19,747][20347] Updated weights for policy 0, policy_version 1109828 (0.0007) [2023-12-27 13:20:19,792][20347] Updated weights for policy 0, policy_version 1109838 (0.0007) [2023-12-27 13:20:19,851][20347] Updated weights for policy 0, policy_version 1109848 (0.0008) [2023-12-27 13:20:20,261][20381] Updated weights for policy 1, policy_version 1108798 (0.0008) [2023-12-27 13:20:20,316][20381] Updated weights for policy 1, policy_version 1108808 (0.0005) [2023-12-27 13:20:20,371][20381] Updated weights for policy 1, policy_version 1108818 (0.0006) [2023-12-27 13:20:20,633][20347] Updated weights for policy 0, policy_version 1109858 (0.0007) [2023-12-27 13:20:20,685][20347] Updated weights for policy 0, policy_version 1109868 (0.0008) [2023-12-27 13:20:20,743][20347] Updated weights for policy 0, policy_version 1109878 (0.0008) [2023-12-27 13:20:20,800][20347] Updated weights for policy 0, policy_version 1109888 (0.0008) [2023-12-27 13:20:21,064][20381] Updated weights for policy 1, policy_version 1108828 (0.0008) [2023-12-27 13:20:21,123][20381] Updated weights for policy 1, policy_version 1108838 (0.0008) [2023-12-27 13:20:21,192][20381] Updated weights for policy 1, policy_version 1108848 (0.0008) [2023-12-27 13:20:21,556][20347] Updated weights for policy 0, policy_version 1109898 (0.0007) [2023-12-27 13:20:21,625][20347] Updated weights for policy 0, policy_version 1109908 (0.0007) [2023-12-27 13:20:21,686][20347] Updated weights for policy 0, policy_version 1109918 (0.0008) [2023-12-27 13:20:21,875][20381] Updated weights for policy 1, policy_version 1108858 (0.0008) [2023-12-27 13:20:21,925][20381] Updated weights for policy 1, policy_version 1108868 (0.0008) [2023-12-27 13:20:21,975][20381] Updated weights for policy 1, policy_version 1108878 (0.0008) [2023-12-27 13:20:22,032][20381] Updated weights for policy 1, policy_version 1108888 (0.0007) [2023-12-27 13:20:22,329][20347] Updated weights for policy 0, policy_version 1109928 (0.0008) [2023-12-27 13:20:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 568090624. Throughput: 0: 10282.6, 1: 10136.7. Samples: 568063676. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:22,363][19170] Avg episode reward: [(0, '942.020'), (1, '594.420')] [2023-12-27 13:20:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001108888_283901952.pth... [2023-12-27 13:20:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001107704_283598848.pth [2023-12-27 13:20:22,394][20347] Updated weights for policy 0, policy_version 1109938 (0.0008) [2023-12-27 13:20:22,454][20347] Updated weights for policy 0, policy_version 1109948 (0.0009) [2023-12-27 13:20:22,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001109952_284196864.pth... [2023-12-27 13:20:22,476][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001108736_283885568.pth [2023-12-27 13:20:22,756][20381] Updated weights for policy 1, policy_version 1108898 (0.0007) [2023-12-27 13:20:22,807][20381] Updated weights for policy 1, policy_version 1108908 (0.0007) [2023-12-27 13:20:22,867][20381] Updated weights for policy 1, policy_version 1108918 (0.0006) [2023-12-27 13:20:23,149][20347] Updated weights for policy 0, policy_version 1109958 (0.0007) [2023-12-27 13:20:23,198][20347] Updated weights for policy 0, policy_version 1109968 (0.0006) [2023-12-27 13:20:23,245][20347] Updated weights for policy 0, policy_version 1109978 (0.0007) [2023-12-27 13:20:23,512][20381] Updated weights for policy 1, policy_version 1108928 (0.0010) [2023-12-27 13:20:23,574][20381] Updated weights for policy 1, policy_version 1108938 (0.0011) [2023-12-27 13:20:23,639][20381] Updated weights for policy 1, policy_version 1108948 (0.0011) [2023-12-27 13:20:23,935][20347] Updated weights for policy 0, policy_version 1109988 (0.0007) [2023-12-27 13:20:23,984][20347] Updated weights for policy 0, policy_version 1109998 (0.0005) [2023-12-27 13:20:24,043][20347] Updated weights for policy 0, policy_version 1110008 (0.0006) [2023-12-27 13:20:24,291][20381] Updated weights for policy 1, policy_version 1108958 (0.0011) [2023-12-27 13:20:24,343][20381] Updated weights for policy 1, policy_version 1108968 (0.0008) [2023-12-27 13:20:24,387][20381] Updated weights for policy 1, policy_version 1108978 (0.0008) [2023-12-27 13:20:24,576][20347] Updated weights for policy 0, policy_version 1110018 (0.0006) [2023-12-27 13:20:24,633][20347] Updated weights for policy 0, policy_version 1110028 (0.0008) [2023-12-27 13:20:24,694][20347] Updated weights for policy 0, policy_version 1110038 (0.0007) [2023-12-27 13:20:24,749][20347] Updated weights for policy 0, policy_version 1110048 (0.0009) [2023-12-27 13:20:25,003][20381] Updated weights for policy 1, policy_version 1108988 (0.0007) [2023-12-27 13:20:25,063][20381] Updated weights for policy 1, policy_version 1108998 (0.0006) [2023-12-27 13:20:25,131][20381] Updated weights for policy 1, policy_version 1109008 (0.0011) [2023-12-27 13:20:25,442][20347] Updated weights for policy 0, policy_version 1110058 (0.0005) [2023-12-27 13:20:25,502][20347] Updated weights for policy 0, policy_version 1110068 (0.0005) [2023-12-27 13:20:25,553][20347] Updated weights for policy 0, policy_version 1110078 (0.0005) [2023-12-27 13:20:25,781][20381] Updated weights for policy 1, policy_version 1109018 (0.0011) [2023-12-27 13:20:25,829][20381] Updated weights for policy 1, policy_version 1109028 (0.0010) [2023-12-27 13:20:25,874][20381] Updated weights for policy 1, policy_version 1109038 (0.0010) [2023-12-27 13:20:25,925][20381] Updated weights for policy 1, policy_version 1109048 (0.0011) [2023-12-27 13:20:26,124][20347] Updated weights for policy 0, policy_version 1110088 (0.0009) [2023-12-27 13:20:26,178][20347] Updated weights for policy 0, policy_version 1110098 (0.0010) [2023-12-27 13:20:26,236][20347] Updated weights for policy 0, policy_version 1110108 (0.0010) [2023-12-27 13:20:26,608][20381] Updated weights for policy 1, policy_version 1109058 (0.0005) [2023-12-27 13:20:26,660][20381] Updated weights for policy 1, policy_version 1109068 (0.0006) [2023-12-27 13:20:26,708][20381] Updated weights for policy 1, policy_version 1109078 (0.0010) [2023-12-27 13:20:26,909][20347] Updated weights for policy 0, policy_version 1110118 (0.0009) [2023-12-27 13:20:26,969][20347] Updated weights for policy 0, policy_version 1110128 (0.0005) [2023-12-27 13:20:27,027][20347] Updated weights for policy 0, policy_version 1110138 (0.0005) [2023-12-27 13:20:27,332][20381] Updated weights for policy 1, policy_version 1109088 (0.0010) [2023-12-27 13:20:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 568197120. Throughput: 0: 10320.6, 1: 10150.9. Samples: 568187644. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:27,363][19170] Avg episode reward: [(0, '961.060'), (1, '636.880')] [2023-12-27 13:20:27,395][20381] Updated weights for policy 1, policy_version 1109098 (0.0010) [2023-12-27 13:20:27,455][20381] Updated weights for policy 1, policy_version 1109108 (0.0010) [2023-12-27 13:20:27,589][20347] Updated weights for policy 0, policy_version 1110148 (0.0007) [2023-12-27 13:20:27,643][20347] Updated weights for policy 0, policy_version 1110158 (0.0010) [2023-12-27 13:20:27,691][20347] Updated weights for policy 0, policy_version 1110168 (0.0010) [2023-12-27 13:20:28,085][20381] Updated weights for policy 1, policy_version 1109118 (0.0007) [2023-12-27 13:20:28,132][20381] Updated weights for policy 1, policy_version 1109128 (0.0011) [2023-12-27 13:20:28,178][20381] Updated weights for policy 1, policy_version 1109138 (0.0010) [2023-12-27 13:20:28,292][20347] Updated weights for policy 0, policy_version 1110178 (0.0006) [2023-12-27 13:20:28,349][20347] Updated weights for policy 0, policy_version 1110188 (0.0008) [2023-12-27 13:20:28,409][20347] Updated weights for policy 0, policy_version 1110198 (0.0011) [2023-12-27 13:20:28,466][20347] Updated weights for policy 0, policy_version 1110208 (0.0008) [2023-12-27 13:20:28,840][20381] Updated weights for policy 1, policy_version 1109148 (0.0008) [2023-12-27 13:20:28,897][20381] Updated weights for policy 1, policy_version 1109158 (0.0006) [2023-12-27 13:20:28,958][20381] Updated weights for policy 1, policy_version 1109168 (0.0006) [2023-12-27 13:20:29,108][20347] Updated weights for policy 0, policy_version 1110218 (0.0009) [2023-12-27 13:20:29,163][20347] Updated weights for policy 0, policy_version 1110228 (0.0007) [2023-12-27 13:20:29,217][20347] Updated weights for policy 0, policy_version 1110238 (0.0010) [2023-12-27 13:20:29,597][20381] Updated weights for policy 1, policy_version 1109178 (0.0008) [2023-12-27 13:20:29,657][20381] Updated weights for policy 1, policy_version 1109188 (0.0010) [2023-12-27 13:20:29,717][20381] Updated weights for policy 1, policy_version 1109198 (0.0007) [2023-12-27 13:20:29,776][20381] Updated weights for policy 1, policy_version 1109208 (0.0006) [2023-12-27 13:20:29,926][20347] Updated weights for policy 0, policy_version 1110248 (0.0007) [2023-12-27 13:20:29,987][20347] Updated weights for policy 0, policy_version 1110258 (0.0009) [2023-12-27 13:20:30,046][20347] Updated weights for policy 0, policy_version 1110268 (0.0010) [2023-12-27 13:20:30,462][20381] Updated weights for policy 1, policy_version 1109218 (0.0007) [2023-12-27 13:20:30,527][20381] Updated weights for policy 1, policy_version 1109228 (0.0009) [2023-12-27 13:20:30,576][20381] Updated weights for policy 1, policy_version 1109238 (0.0007) [2023-12-27 13:20:30,666][20347] Updated weights for policy 0, policy_version 1110278 (0.0007) [2023-12-27 13:20:30,729][20347] Updated weights for policy 0, policy_version 1110288 (0.0007) [2023-12-27 13:20:30,787][20347] Updated weights for policy 0, policy_version 1110298 (0.0008) [2023-12-27 13:20:31,277][20381] Updated weights for policy 1, policy_version 1109248 (0.0008) [2023-12-27 13:20:31,277][20292] KL-divergence is very high: 475.9329 [2023-12-27 13:20:31,315][20292] KL-divergence is very high: 184.6966 [2023-12-27 13:20:31,328][20381] Updated weights for policy 1, policy_version 1109258 (0.0008) [2023-12-27 13:20:31,388][20381] Updated weights for policy 1, policy_version 1109268 (0.0006) [2023-12-27 13:20:31,476][20347] Updated weights for policy 0, policy_version 1110308 (0.0008) [2023-12-27 13:20:31,535][20347] Updated weights for policy 0, policy_version 1110318 (0.0008) [2023-12-27 13:20:31,607][20347] Updated weights for policy 0, policy_version 1110328 (0.0008) [2023-12-27 13:20:32,140][20381] Updated weights for policy 1, policy_version 1109278 (0.0007) [2023-12-27 13:20:32,190][20381] Updated weights for policy 1, policy_version 1109288 (0.0008) [2023-12-27 13:20:32,240][20381] Updated weights for policy 1, policy_version 1109298 (0.0007) [2023-12-27 13:20:32,279][20347] Updated weights for policy 0, policy_version 1110338 (0.0008) [2023-12-27 13:20:32,334][20347] Updated weights for policy 0, policy_version 1110348 (0.0007) [2023-12-27 13:20:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 568303616. Throughput: 0: 10328.9, 1: 10226.7. Samples: 568313952. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:32,363][19170] Avg episode reward: [(0, '970.800'), (1, '646.530')] [2023-12-27 13:20:32,396][20347] Updated weights for policy 0, policy_version 1110358 (0.0007) [2023-12-27 13:20:32,457][20347] Updated weights for policy 0, policy_version 1110368 (0.0008) [2023-12-27 13:20:32,879][20381] Updated weights for policy 1, policy_version 1109308 (0.0007) [2023-12-27 13:20:32,937][20381] Updated weights for policy 1, policy_version 1109318 (0.0007) [2023-12-27 13:20:32,989][20381] Updated weights for policy 1, policy_version 1109328 (0.0007) [2023-12-27 13:20:33,158][20347] Updated weights for policy 0, policy_version 1110378 (0.0005) [2023-12-27 13:20:33,210][20347] Updated weights for policy 0, policy_version 1110388 (0.0008) [2023-12-27 13:20:33,258][20347] Updated weights for policy 0, policy_version 1110398 (0.0010) [2023-12-27 13:20:33,592][20381] Updated weights for policy 1, policy_version 1109338 (0.0007) [2023-12-27 13:20:33,640][20381] Updated weights for policy 1, policy_version 1109348 (0.0008) [2023-12-27 13:20:33,689][20381] Updated weights for policy 1, policy_version 1109358 (0.0008) [2023-12-27 13:20:33,739][20381] Updated weights for policy 1, policy_version 1109368 (0.0006) [2023-12-27 13:20:33,886][20347] Updated weights for policy 0, policy_version 1110408 (0.0008) [2023-12-27 13:20:33,945][20347] Updated weights for policy 0, policy_version 1110418 (0.0007) [2023-12-27 13:20:34,003][20347] Updated weights for policy 0, policy_version 1110428 (0.0008) [2023-12-27 13:20:34,385][20381] Updated weights for policy 1, policy_version 1109378 (0.0008) [2023-12-27 13:20:34,439][20381] Updated weights for policy 1, policy_version 1109388 (0.0008) [2023-12-27 13:20:34,494][20381] Updated weights for policy 1, policy_version 1109398 (0.0008) [2023-12-27 13:20:34,701][20347] Updated weights for policy 0, policy_version 1110438 (0.0006) [2023-12-27 13:20:34,755][20347] Updated weights for policy 0, policy_version 1110448 (0.0006) [2023-12-27 13:20:34,804][20347] Updated weights for policy 0, policy_version 1110458 (0.0007) [2023-12-27 13:20:35,169][20381] Updated weights for policy 1, policy_version 1109408 (0.0006) [2023-12-27 13:20:35,225][20381] Updated weights for policy 1, policy_version 1109418 (0.0006) [2023-12-27 13:20:35,284][20381] Updated weights for policy 1, policy_version 1109428 (0.0006) [2023-12-27 13:20:35,442][20347] Updated weights for policy 0, policy_version 1110468 (0.0010) [2023-12-27 13:20:35,491][20347] Updated weights for policy 0, policy_version 1110478 (0.0010) [2023-12-27 13:20:35,544][20347] Updated weights for policy 0, policy_version 1110488 (0.0010) [2023-12-27 13:20:35,898][20381] Updated weights for policy 1, policy_version 1109438 (0.0006) [2023-12-27 13:20:35,952][20381] Updated weights for policy 1, policy_version 1109448 (0.0008) [2023-12-27 13:20:36,000][20381] Updated weights for policy 1, policy_version 1109458 (0.0010) [2023-12-27 13:20:36,204][20347] Updated weights for policy 0, policy_version 1110498 (0.0010) [2023-12-27 13:20:36,265][20347] Updated weights for policy 0, policy_version 1110508 (0.0008) [2023-12-27 13:20:36,328][20347] Updated weights for policy 0, policy_version 1110518 (0.0009) [2023-12-27 13:20:36,384][20347] Updated weights for policy 0, policy_version 1110528 (0.0006) [2023-12-27 13:20:36,692][20381] Updated weights for policy 1, policy_version 1109468 (0.0008) [2023-12-27 13:20:36,751][20381] Updated weights for policy 1, policy_version 1109478 (0.0008) [2023-12-27 13:20:36,805][20381] Updated weights for policy 1, policy_version 1109488 (0.0008) [2023-12-27 13:20:37,045][20347] Updated weights for policy 0, policy_version 1110538 (0.0010) [2023-12-27 13:20:37,104][20347] Updated weights for policy 0, policy_version 1110548 (0.0007) [2023-12-27 13:20:37,158][20347] Updated weights for policy 0, policy_version 1110558 (0.0007) [2023-12-27 13:20:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 568410112. Throughput: 0: 10333.0, 1: 10240.8. Samples: 568377788. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:37,364][19170] Avg episode reward: [(0, '970.760'), (1, '646.280')] [2023-12-27 13:20:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001110560_284352512.pth... [2023-12-27 13:20:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001109344_284041216.pth [2023-12-27 13:20:37,389][20381] Updated weights for policy 1, policy_version 1109498 (0.0006) [2023-12-27 13:20:37,451][20381] Updated weights for policy 1, policy_version 1109508 (0.0011) [2023-12-27 13:20:37,507][20381] Updated weights for policy 1, policy_version 1109518 (0.0011) [2023-12-27 13:20:37,554][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001109528_284065792.pth... [2023-12-27 13:20:37,556][20381] Updated weights for policy 1, policy_version 1109528 (0.0011) [2023-12-27 13:20:37,558][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001108312_283754496.pth [2023-12-27 13:20:37,821][20347] Updated weights for policy 0, policy_version 1110568 (0.0006) [2023-12-27 13:20:37,879][20347] Updated weights for policy 0, policy_version 1110578 (0.0007) [2023-12-27 13:20:37,927][20347] Updated weights for policy 0, policy_version 1110588 (0.0007) [2023-12-27 13:20:38,291][20381] Updated weights for policy 1, policy_version 1109538 (0.0010) [2023-12-27 13:20:38,354][20381] Updated weights for policy 1, policy_version 1109548 (0.0010) [2023-12-27 13:20:38,412][20381] Updated weights for policy 1, policy_version 1109558 (0.0009) [2023-12-27 13:20:38,612][20347] Updated weights for policy 0, policy_version 1110598 (0.0006) [2023-12-27 13:20:38,656][20291] KL-divergence is very high: 113.5836 [2023-12-27 13:20:38,663][20291] KL-divergence is very high: 108.5404 [2023-12-27 13:20:38,675][20347] Updated weights for policy 0, policy_version 1110608 (0.0007) [2023-12-27 13:20:38,731][20347] Updated weights for policy 0, policy_version 1110618 (0.0007) [2023-12-27 13:20:39,057][20381] Updated weights for policy 1, policy_version 1109568 (0.0008) [2023-12-27 13:20:39,109][20381] Updated weights for policy 1, policy_version 1109578 (0.0007) [2023-12-27 13:20:39,165][20381] Updated weights for policy 1, policy_version 1109588 (0.0007) [2023-12-27 13:20:39,471][20347] Updated weights for policy 0, policy_version 1110628 (0.0007) [2023-12-27 13:20:39,529][20347] Updated weights for policy 0, policy_version 1110638 (0.0008) [2023-12-27 13:20:39,584][20347] Updated weights for policy 0, policy_version 1110648 (0.0008) [2023-12-27 13:20:39,927][20381] Updated weights for policy 1, policy_version 1109598 (0.0008) [2023-12-27 13:20:39,995][20381] Updated weights for policy 1, policy_version 1109608 (0.0008) [2023-12-27 13:20:40,056][20381] Updated weights for policy 1, policy_version 1109618 (0.0008) [2023-12-27 13:20:40,254][20347] Updated weights for policy 0, policy_version 1110658 (0.0007) [2023-12-27 13:20:40,310][20347] Updated weights for policy 0, policy_version 1110668 (0.0007) [2023-12-27 13:20:40,375][20347] Updated weights for policy 0, policy_version 1110678 (0.0007) [2023-12-27 13:20:40,413][20291] KL-divergence is very high: 187.9552 [2023-12-27 13:20:40,425][20347] Updated weights for policy 0, policy_version 1110688 (0.0008) [2023-12-27 13:20:40,756][20381] Updated weights for policy 1, policy_version 1109628 (0.0007) [2023-12-27 13:20:40,818][20381] Updated weights for policy 1, policy_version 1109638 (0.0006) [2023-12-27 13:20:40,879][20381] Updated weights for policy 1, policy_version 1109648 (0.0006) [2023-12-27 13:20:41,034][20347] Updated weights for policy 0, policy_version 1110698 (0.0008) [2023-12-27 13:20:41,102][20347] Updated weights for policy 0, policy_version 1110708 (0.0008) [2023-12-27 13:20:41,167][20347] Updated weights for policy 0, policy_version 1110718 (0.0008) [2023-12-27 13:20:41,591][20381] Updated weights for policy 1, policy_version 1109658 (0.0009) [2023-12-27 13:20:41,666][20381] Updated weights for policy 1, policy_version 1109668 (0.0009) [2023-12-27 13:20:41,737][20381] Updated weights for policy 1, policy_version 1109678 (0.0009) [2023-12-27 13:20:41,807][20381] Updated weights for policy 1, policy_version 1109688 (0.0008) [2023-12-27 13:20:41,905][20347] Updated weights for policy 0, policy_version 1110728 (0.0007) [2023-12-27 13:20:41,965][20347] Updated weights for policy 0, policy_version 1110738 (0.0006) [2023-12-27 13:20:42,024][20347] Updated weights for policy 0, policy_version 1110748 (0.0006) [2023-12-27 13:20:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 568508416. Throughput: 0: 10374.1, 1: 10269.3. Samples: 568501720. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:42,365][19170] Avg episode reward: [(0, '922.060'), (1, '676.050')] [2023-12-27 13:20:42,565][20381] Updated weights for policy 1, policy_version 1109698 (0.0008) [2023-12-27 13:20:42,624][20381] Updated weights for policy 1, policy_version 1109708 (0.0006) [2023-12-27 13:20:42,661][20347] Updated weights for policy 0, policy_version 1110758 (0.0007) [2023-12-27 13:20:42,689][20381] Updated weights for policy 1, policy_version 1109718 (0.0006) [2023-12-27 13:20:42,726][20347] Updated weights for policy 0, policy_version 1110768 (0.0008) [2023-12-27 13:20:42,799][20347] Updated weights for policy 0, policy_version 1110778 (0.0008) [2023-12-27 13:20:43,335][20381] Updated weights for policy 1, policy_version 1109728 (0.0010) [2023-12-27 13:20:43,390][20381] Updated weights for policy 1, policy_version 1109738 (0.0010) [2023-12-27 13:20:43,444][20381] Updated weights for policy 1, policy_version 1109748 (0.0010) [2023-12-27 13:20:43,475][20347] Updated weights for policy 0, policy_version 1110788 (0.0007) [2023-12-27 13:20:43,531][20347] Updated weights for policy 0, policy_version 1110798 (0.0008) [2023-12-27 13:20:43,582][20347] Updated weights for policy 0, policy_version 1110808 (0.0010) [2023-12-27 13:20:44,116][20381] Updated weights for policy 1, policy_version 1109758 (0.0011) [2023-12-27 13:20:44,161][20381] Updated weights for policy 1, policy_version 1109768 (0.0010) [2023-12-27 13:20:44,213][20381] Updated weights for policy 1, policy_version 1109778 (0.0010) [2023-12-27 13:20:44,249][20347] Updated weights for policy 0, policy_version 1110818 (0.0009) [2023-12-27 13:20:44,306][20347] Updated weights for policy 0, policy_version 1110828 (0.0008) [2023-12-27 13:20:44,359][20347] Updated weights for policy 0, policy_version 1110838 (0.0005) [2023-12-27 13:20:44,415][20347] Updated weights for policy 0, policy_version 1110848 (0.0005) [2023-12-27 13:20:44,926][20381] Updated weights for policy 1, policy_version 1109788 (0.0009) [2023-12-27 13:20:44,989][20381] Updated weights for policy 1, policy_version 1109798 (0.0009) [2023-12-27 13:20:45,030][20347] Updated weights for policy 0, policy_version 1110858 (0.0008) [2023-12-27 13:20:45,039][20291] KL-divergence is very high: 175.9806 [2023-12-27 13:20:45,045][20291] KL-divergence is very high: 188.2137 [2023-12-27 13:20:45,051][20291] KL-divergence is very high: 177.8109 [2023-12-27 13:20:45,052][20381] Updated weights for policy 1, policy_version 1109808 (0.0007) [2023-12-27 13:20:45,093][20347] Updated weights for policy 0, policy_version 1110868 (0.0007) [2023-12-27 13:20:45,147][20347] Updated weights for policy 0, policy_version 1110878 (0.0007) [2023-12-27 13:20:45,713][20381] Updated weights for policy 1, policy_version 1109818 (0.0009) [2023-12-27 13:20:45,741][20347] Updated weights for policy 0, policy_version 1110888 (0.0005) [2023-12-27 13:20:45,769][20381] Updated weights for policy 1, policy_version 1109828 (0.0011) [2023-12-27 13:20:45,797][20347] Updated weights for policy 0, policy_version 1110898 (0.0005) [2023-12-27 13:20:45,817][20381] Updated weights for policy 1, policy_version 1109838 (0.0011) [2023-12-27 13:20:45,850][20347] Updated weights for policy 0, policy_version 1110908 (0.0005) [2023-12-27 13:20:45,866][20381] Updated weights for policy 1, policy_version 1109848 (0.0009) [2023-12-27 13:20:46,429][20347] Updated weights for policy 0, policy_version 1110918 (0.0007) [2023-12-27 13:20:46,468][20381] Updated weights for policy 1, policy_version 1109858 (0.0006) [2023-12-27 13:20:46,484][20347] Updated weights for policy 0, policy_version 1110928 (0.0006) [2023-12-27 13:20:46,530][20381] Updated weights for policy 1, policy_version 1109868 (0.0008) [2023-12-27 13:20:46,541][20347] Updated weights for policy 0, policy_version 1110938 (0.0006) [2023-12-27 13:20:46,588][20381] Updated weights for policy 1, policy_version 1109878 (0.0007) [2023-12-27 13:20:47,204][20347] Updated weights for policy 0, policy_version 1110948 (0.0006) [2023-12-27 13:20:47,228][20381] Updated weights for policy 1, policy_version 1109888 (0.0006) [2023-12-27 13:20:47,256][20347] Updated weights for policy 0, policy_version 1110958 (0.0006) [2023-12-27 13:20:47,287][20381] Updated weights for policy 1, policy_version 1109898 (0.0005) [2023-12-27 13:20:47,311][20347] Updated weights for policy 0, policy_version 1110968 (0.0007) [2023-12-27 13:20:47,340][20381] Updated weights for policy 1, policy_version 1109908 (0.0005) [2023-12-27 13:20:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 568623104. Throughput: 0: 10411.4, 1: 10286.0. Samples: 568626060. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:47,364][19170] Avg episode reward: [(0, '883.640'), (1, '617.160')] [2023-12-27 13:20:47,967][20381] Updated weights for policy 1, policy_version 1109918 (0.0009) [2023-12-27 13:20:48,015][20347] Updated weights for policy 0, policy_version 1110978 (0.0008) [2023-12-27 13:20:48,015][20381] Updated weights for policy 1, policy_version 1109928 (0.0008) [2023-12-27 13:20:48,064][20347] Updated weights for policy 0, policy_version 1110988 (0.0010) [2023-12-27 13:20:48,071][20381] Updated weights for policy 1, policy_version 1109938 (0.0011) [2023-12-27 13:20:48,116][20347] Updated weights for policy 0, policy_version 1110998 (0.0010) [2023-12-27 13:20:48,175][20347] Updated weights for policy 0, policy_version 1111008 (0.0010) [2023-12-27 13:20:48,832][20381] Updated weights for policy 1, policy_version 1109948 (0.0009) [2023-12-27 13:20:48,859][20347] Updated weights for policy 0, policy_version 1111018 (0.0006) [2023-12-27 13:20:48,888][20381] Updated weights for policy 1, policy_version 1109958 (0.0007) [2023-12-27 13:20:48,913][20347] Updated weights for policy 0, policy_version 1111028 (0.0006) [2023-12-27 13:20:48,937][20381] Updated weights for policy 1, policy_version 1109968 (0.0008) [2023-12-27 13:20:48,957][20347] Updated weights for policy 0, policy_version 1111038 (0.0005) [2023-12-27 13:20:49,586][20381] Updated weights for policy 1, policy_version 1109978 (0.0007) [2023-12-27 13:20:49,647][20381] Updated weights for policy 1, policy_version 1109988 (0.0009) [2023-12-27 13:20:49,660][20347] Updated weights for policy 0, policy_version 1111048 (0.0007) [2023-12-27 13:20:49,707][20381] Updated weights for policy 1, policy_version 1109998 (0.0006) [2023-12-27 13:20:49,721][20347] Updated weights for policy 0, policy_version 1111058 (0.0005) [2023-12-27 13:20:49,765][20381] Updated weights for policy 1, policy_version 1110008 (0.0006) [2023-12-27 13:20:49,784][20347] Updated weights for policy 0, policy_version 1111068 (0.0006) [2023-12-27 13:20:50,409][20347] Updated weights for policy 0, policy_version 1111078 (0.0008) [2023-12-27 13:20:50,465][20381] Updated weights for policy 1, policy_version 1110018 (0.0006) [2023-12-27 13:20:50,467][20347] Updated weights for policy 0, policy_version 1111088 (0.0007) [2023-12-27 13:20:50,516][20381] Updated weights for policy 1, policy_version 1110028 (0.0005) [2023-12-27 13:20:50,522][20347] Updated weights for policy 0, policy_version 1111098 (0.0006) [2023-12-27 13:20:50,562][20381] Updated weights for policy 1, policy_version 1110038 (0.0007) [2023-12-27 13:20:51,191][20347] Updated weights for policy 0, policy_version 1111108 (0.0007) [2023-12-27 13:20:51,250][20347] Updated weights for policy 0, policy_version 1111118 (0.0006) [2023-12-27 13:20:51,263][20381] Updated weights for policy 1, policy_version 1110048 (0.0007) [2023-12-27 13:20:51,310][20347] Updated weights for policy 0, policy_version 1111128 (0.0006) [2023-12-27 13:20:51,320][20381] Updated weights for policy 1, policy_version 1110058 (0.0006) [2023-12-27 13:20:51,387][20381] Updated weights for policy 1, policy_version 1110068 (0.0008) [2023-12-27 13:20:52,042][20347] Updated weights for policy 0, policy_version 1111138 (0.0010) [2023-12-27 13:20:52,101][20381] Updated weights for policy 1, policy_version 1110078 (0.0010) [2023-12-27 13:20:52,101][20347] Updated weights for policy 0, policy_version 1111148 (0.0010) [2023-12-27 13:20:52,157][20347] Updated weights for policy 0, policy_version 1111158 (0.0010) [2023-12-27 13:20:52,160][20381] Updated weights for policy 1, policy_version 1110088 (0.0010) [2023-12-27 13:20:52,206][20347] Updated weights for policy 0, policy_version 1111168 (0.0010) [2023-12-27 13:20:52,215][20381] Updated weights for policy 1, policy_version 1110098 (0.0011) [2023-12-27 13:20:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 568721408. Throughput: 0: 10400.1, 1: 10226.6. Samples: 568686816. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:52,363][19170] Avg episode reward: [(0, '895.910'), (1, '606.800')] [2023-12-27 13:20:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001110104_284213248.pth... [2023-12-27 13:20:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001111168_284508160.pth... [2023-12-27 13:20:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001109952_284196864.pth [2023-12-27 13:20:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001108888_283901952.pth [2023-12-27 13:20:52,873][20381] Updated weights for policy 1, policy_version 1110108 (0.0008) [2023-12-27 13:20:52,928][20347] Updated weights for policy 0, policy_version 1111178 (0.0005) [2023-12-27 13:20:52,935][20381] Updated weights for policy 1, policy_version 1110118 (0.0006) [2023-12-27 13:20:52,984][20381] Updated weights for policy 1, policy_version 1110128 (0.0007) [2023-12-27 13:20:52,986][20347] Updated weights for policy 0, policy_version 1111188 (0.0007) [2023-12-27 13:20:53,042][20347] Updated weights for policy 0, policy_version 1111198 (0.0008) [2023-12-27 13:20:53,561][20381] Updated weights for policy 1, policy_version 1110138 (0.0008) [2023-12-27 13:20:53,597][20347] Updated weights for policy 0, policy_version 1111208 (0.0007) [2023-12-27 13:20:53,616][20381] Updated weights for policy 1, policy_version 1110148 (0.0010) [2023-12-27 13:20:53,654][20347] Updated weights for policy 0, policy_version 1111218 (0.0005) [2023-12-27 13:20:53,677][20381] Updated weights for policy 1, policy_version 1110158 (0.0010) [2023-12-27 13:20:53,704][20347] Updated weights for policy 0, policy_version 1111228 (0.0007) [2023-12-27 13:20:53,738][20381] Updated weights for policy 1, policy_version 1110168 (0.0010) [2023-12-27 13:20:54,286][20347] Updated weights for policy 0, policy_version 1111238 (0.0006) [2023-12-27 13:20:54,338][20347] Updated weights for policy 0, policy_version 1111248 (0.0007) [2023-12-27 13:20:54,375][20381] Updated weights for policy 1, policy_version 1110178 (0.0008) [2023-12-27 13:20:54,401][20347] Updated weights for policy 0, policy_version 1111258 (0.0009) [2023-12-27 13:20:54,427][20381] Updated weights for policy 1, policy_version 1110188 (0.0010) [2023-12-27 13:20:54,476][20381] Updated weights for policy 1, policy_version 1110198 (0.0010) [2023-12-27 13:20:55,008][20347] Updated weights for policy 0, policy_version 1111268 (0.0010) [2023-12-27 13:20:55,073][20347] Updated weights for policy 0, policy_version 1111278 (0.0009) [2023-12-27 13:20:55,127][20347] Updated weights for policy 0, policy_version 1111288 (0.0007) [2023-12-27 13:20:55,154][20381] Updated weights for policy 1, policy_version 1110208 (0.0010) [2023-12-27 13:20:55,200][20381] Updated weights for policy 1, policy_version 1110218 (0.0010) [2023-12-27 13:20:55,245][20381] Updated weights for policy 1, policy_version 1110228 (0.0010) [2023-12-27 13:20:55,815][20347] Updated weights for policy 0, policy_version 1111298 (0.0005) [2023-12-27 13:20:55,867][20347] Updated weights for policy 0, policy_version 1111308 (0.0007) [2023-12-27 13:20:55,884][20381] Updated weights for policy 1, policy_version 1110238 (0.0008) [2023-12-27 13:20:55,916][20347] Updated weights for policy 0, policy_version 1111318 (0.0005) [2023-12-27 13:20:55,933][20381] Updated weights for policy 1, policy_version 1110248 (0.0006) [2023-12-27 13:20:55,969][20347] Updated weights for policy 0, policy_version 1111328 (0.0006) [2023-12-27 13:20:55,983][20381] Updated weights for policy 1, policy_version 1110258 (0.0006) [2023-12-27 13:20:56,645][20381] Updated weights for policy 1, policy_version 1110268 (0.0007) [2023-12-27 13:20:56,664][20347] Updated weights for policy 0, policy_version 1111338 (0.0007) [2023-12-27 13:20:56,698][20381] Updated weights for policy 1, policy_version 1110278 (0.0005) [2023-12-27 13:20:56,722][20347] Updated weights for policy 0, policy_version 1111348 (0.0005) [2023-12-27 13:20:56,757][20381] Updated weights for policy 1, policy_version 1110288 (0.0009) [2023-12-27 13:20:56,773][20347] Updated weights for policy 0, policy_version 1111358 (0.0007) [2023-12-27 13:20:57,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 568819712. Throughput: 0: 10457.0, 1: 10266.2. Samples: 568813440. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:20:57,363][19170] Avg episode reward: [(0, '943.980'), (1, '494.490')] [2023-12-27 13:20:57,411][20381] Updated weights for policy 1, policy_version 1110298 (0.0007) [2023-12-27 13:20:57,434][20347] Updated weights for policy 0, policy_version 1111368 (0.0006) [2023-12-27 13:20:57,462][20381] Updated weights for policy 1, policy_version 1110308 (0.0005) [2023-12-27 13:20:57,489][20347] Updated weights for policy 0, policy_version 1111378 (0.0006) [2023-12-27 13:20:57,516][20381] Updated weights for policy 1, policy_version 1110318 (0.0005) [2023-12-27 13:20:57,547][20347] Updated weights for policy 0, policy_version 1111388 (0.0006) [2023-12-27 13:20:57,568][20381] Updated weights for policy 1, policy_version 1110328 (0.0005) [2023-12-27 13:20:58,120][20347] Updated weights for policy 0, policy_version 1111398 (0.0006) [2023-12-27 13:20:58,168][20347] Updated weights for policy 0, policy_version 1111408 (0.0005) [2023-12-27 13:20:58,229][20347] Updated weights for policy 0, policy_version 1111418 (0.0007) [2023-12-27 13:20:58,248][20381] Updated weights for policy 1, policy_version 1110338 (0.0008) [2023-12-27 13:20:58,307][20381] Updated weights for policy 1, policy_version 1110348 (0.0007) [2023-12-27 13:20:58,375][20381] Updated weights for policy 1, policy_version 1110358 (0.0007) [2023-12-27 13:20:58,993][20347] Updated weights for policy 0, policy_version 1111428 (0.0008) [2023-12-27 13:20:59,043][20347] Updated weights for policy 0, policy_version 1111438 (0.0009) [2023-12-27 13:20:59,102][20347] Updated weights for policy 0, policy_version 1111448 (0.0008) [2023-12-27 13:20:59,134][20381] Updated weights for policy 1, policy_version 1110368 (0.0008) [2023-12-27 13:20:59,192][20381] Updated weights for policy 1, policy_version 1110378 (0.0008) [2023-12-27 13:20:59,274][20381] Updated weights for policy 1, policy_version 1110388 (0.0008) [2023-12-27 13:20:59,757][20347] Updated weights for policy 0, policy_version 1111458 (0.0006) [2023-12-27 13:20:59,811][20347] Updated weights for policy 0, policy_version 1111468 (0.0006) [2023-12-27 13:20:59,880][20347] Updated weights for policy 0, policy_version 1111478 (0.0008) [2023-12-27 13:20:59,946][20347] Updated weights for policy 0, policy_version 1111488 (0.0008) [2023-12-27 13:20:59,999][20381] Updated weights for policy 1, policy_version 1110398 (0.0008) [2023-12-27 13:21:00,061][20381] Updated weights for policy 1, policy_version 1110408 (0.0008) [2023-12-27 13:21:00,115][20381] Updated weights for policy 1, policy_version 1110418 (0.0008) [2023-12-27 13:21:00,624][20347] Updated weights for policy 0, policy_version 1111498 (0.0008) [2023-12-27 13:21:00,677][20347] Updated weights for policy 0, policy_version 1111508 (0.0007) [2023-12-27 13:21:00,734][20347] Updated weights for policy 0, policy_version 1111518 (0.0008) [2023-12-27 13:21:00,819][20381] Updated weights for policy 1, policy_version 1110428 (0.0008) [2023-12-27 13:21:00,875][20381] Updated weights for policy 1, policy_version 1110438 (0.0008) [2023-12-27 13:21:00,925][20381] Updated weights for policy 1, policy_version 1110448 (0.0007) [2023-12-27 13:21:01,426][20347] Updated weights for policy 0, policy_version 1111528 (0.0008) [2023-12-27 13:21:01,487][20347] Updated weights for policy 0, policy_version 1111538 (0.0007) [2023-12-27 13:21:01,539][20347] Updated weights for policy 0, policy_version 1111548 (0.0006) [2023-12-27 13:21:01,677][20381] Updated weights for policy 1, policy_version 1110458 (0.0006) [2023-12-27 13:21:01,741][20381] Updated weights for policy 1, policy_version 1110468 (0.0008) [2023-12-27 13:21:01,803][20381] Updated weights for policy 1, policy_version 1110478 (0.0008) [2023-12-27 13:21:01,863][20381] Updated weights for policy 1, policy_version 1110488 (0.0006) [2023-12-27 13:21:02,292][20347] Updated weights for policy 0, policy_version 1111558 (0.0008) [2023-12-27 13:21:02,355][20347] Updated weights for policy 0, policy_version 1111568 (0.0008) [2023-12-27 13:21:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 568918016. Throughput: 0: 10431.7, 1: 10252.7. Samples: 568933880. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:02,363][19170] Avg episode reward: [(0, '902.730'), (1, '627.610')] [2023-12-27 13:21:02,414][20347] Updated weights for policy 0, policy_version 1111578 (0.0007) [2023-12-27 13:21:02,541][20381] Updated weights for policy 1, policy_version 1110498 (0.0008) [2023-12-27 13:21:02,591][20381] Updated weights for policy 1, policy_version 1110508 (0.0008) [2023-12-27 13:21:02,648][20381] Updated weights for policy 1, policy_version 1110518 (0.0007) [2023-12-27 13:21:03,091][20347] Updated weights for policy 0, policy_version 1111588 (0.0007) [2023-12-27 13:21:03,146][20347] Updated weights for policy 0, policy_version 1111598 (0.0007) [2023-12-27 13:21:03,204][20347] Updated weights for policy 0, policy_version 1111608 (0.0007) [2023-12-27 13:21:03,342][20381] Updated weights for policy 1, policy_version 1110528 (0.0006) [2023-12-27 13:21:03,396][20381] Updated weights for policy 1, policy_version 1110538 (0.0005) [2023-12-27 13:21:03,454][20381] Updated weights for policy 1, policy_version 1110548 (0.0006) [2023-12-27 13:21:03,903][20347] Updated weights for policy 0, policy_version 1111618 (0.0008) [2023-12-27 13:21:03,955][20291] KL-divergence is very high: 112.7877 [2023-12-27 13:21:03,960][20347] Updated weights for policy 0, policy_version 1111628 (0.0006) [2023-12-27 13:21:03,967][20291] KL-divergence is very high: 111.9872 [2023-12-27 13:21:04,007][20291] KL-divergence is very high: 247.2613 [2023-12-27 13:21:04,020][20347] Updated weights for policy 0, policy_version 1111638 (0.0007) [2023-12-27 13:21:04,054][20291] KL-divergence is very high: 178.5388 [2023-12-27 13:21:04,075][20347] Updated weights for policy 0, policy_version 1111648 (0.0007) [2023-12-27 13:21:04,124][20381] Updated weights for policy 1, policy_version 1110558 (0.0010) [2023-12-27 13:21:04,181][20381] Updated weights for policy 1, policy_version 1110568 (0.0011) [2023-12-27 13:21:04,242][20381] Updated weights for policy 1, policy_version 1110578 (0.0011) [2023-12-27 13:21:04,797][20347] Updated weights for policy 0, policy_version 1111658 (0.0008) [2023-12-27 13:21:04,846][20347] Updated weights for policy 0, policy_version 1111668 (0.0007) [2023-12-27 13:21:04,891][20381] Updated weights for policy 1, policy_version 1110588 (0.0009) [2023-12-27 13:21:04,897][20347] Updated weights for policy 0, policy_version 1111678 (0.0007) [2023-12-27 13:21:04,943][20381] Updated weights for policy 1, policy_version 1110598 (0.0007) [2023-12-27 13:21:05,001][20381] Updated weights for policy 1, policy_version 1110608 (0.0010) [2023-12-27 13:21:05,592][20347] Updated weights for policy 0, policy_version 1111688 (0.0007) [2023-12-27 13:21:05,634][20381] Updated weights for policy 1, policy_version 1110618 (0.0011) [2023-12-27 13:21:05,642][20347] Updated weights for policy 0, policy_version 1111698 (0.0008) [2023-12-27 13:21:05,685][20381] Updated weights for policy 1, policy_version 1110628 (0.0010) [2023-12-27 13:21:05,686][20347] Updated weights for policy 0, policy_version 1111708 (0.0005) [2023-12-27 13:21:05,732][20381] Updated weights for policy 1, policy_version 1110638 (0.0006) [2023-12-27 13:21:05,787][20381] Updated weights for policy 1, policy_version 1110648 (0.0006) [2023-12-27 13:21:06,425][20381] Updated weights for policy 1, policy_version 1110658 (0.0011) [2023-12-27 13:21:06,467][20347] Updated weights for policy 0, policy_version 1111718 (0.0007) [2023-12-27 13:21:06,485][20381] Updated weights for policy 1, policy_version 1110668 (0.0010) [2023-12-27 13:21:06,527][20347] Updated weights for policy 0, policy_version 1111728 (0.0009) [2023-12-27 13:21:06,544][20381] Updated weights for policy 1, policy_version 1110678 (0.0010) [2023-12-27 13:21:06,582][20347] Updated weights for policy 0, policy_version 1111738 (0.0007) [2023-12-27 13:21:07,197][20381] Updated weights for policy 1, policy_version 1110688 (0.0010) [2023-12-27 13:21:07,233][20347] Updated weights for policy 0, policy_version 1111748 (0.0008) [2023-12-27 13:21:07,247][20381] Updated weights for policy 1, policy_version 1110698 (0.0010) [2023-12-27 13:21:07,289][20347] Updated weights for policy 0, policy_version 1111758 (0.0010) [2023-12-27 13:21:07,299][20381] Updated weights for policy 1, policy_version 1110708 (0.0010) [2023-12-27 13:21:07,346][20347] Updated weights for policy 0, policy_version 1111768 (0.0010) [2023-12-27 13:21:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 569024512. Throughput: 0: 10399.1, 1: 10279.0. Samples: 568994192. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:07,363][19170] Avg episode reward: [(0, '884.850'), (1, '641.760')] [2023-12-27 13:21:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001110712_284368896.pth... [2023-12-27 13:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001109528_284065792.pth [2023-12-27 13:21:07,389][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001111776_284663808.pth... [2023-12-27 13:21:07,393][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001110560_284352512.pth [2023-12-27 13:21:07,919][20381] Updated weights for policy 1, policy_version 1110718 (0.0007) [2023-12-27 13:21:07,981][20381] Updated weights for policy 1, policy_version 1110728 (0.0005) [2023-12-27 13:21:08,002][20347] Updated weights for policy 0, policy_version 1111778 (0.0010) [2023-12-27 13:21:08,040][20381] Updated weights for policy 1, policy_version 1110738 (0.0008) [2023-12-27 13:21:08,052][20347] Updated weights for policy 0, policy_version 1111788 (0.0010) [2023-12-27 13:21:08,112][20347] Updated weights for policy 0, policy_version 1111798 (0.0009) [2023-12-27 13:21:08,175][20347] Updated weights for policy 0, policy_version 1111808 (0.0010) [2023-12-27 13:21:08,685][20381] Updated weights for policy 1, policy_version 1110748 (0.0011) [2023-12-27 13:21:08,741][20381] Updated weights for policy 1, policy_version 1110758 (0.0011) [2023-12-27 13:21:08,800][20381] Updated weights for policy 1, policy_version 1110768 (0.0011) [2023-12-27 13:21:08,819][20347] Updated weights for policy 0, policy_version 1111818 (0.0010) [2023-12-27 13:21:08,877][20347] Updated weights for policy 0, policy_version 1111828 (0.0011) [2023-12-27 13:21:08,929][20347] Updated weights for policy 0, policy_version 1111838 (0.0010) [2023-12-27 13:21:09,513][20381] Updated weights for policy 1, policy_version 1110778 (0.0010) [2023-12-27 13:21:09,579][20381] Updated weights for policy 1, policy_version 1110788 (0.0007) [2023-12-27 13:21:09,638][20381] Updated weights for policy 1, policy_version 1110798 (0.0007) [2023-12-27 13:21:09,644][20347] Updated weights for policy 0, policy_version 1111848 (0.0007) [2023-12-27 13:21:09,697][20347] Updated weights for policy 0, policy_version 1111858 (0.0005) [2023-12-27 13:21:09,701][20381] Updated weights for policy 1, policy_version 1110808 (0.0007) [2023-12-27 13:21:09,753][20347] Updated weights for policy 0, policy_version 1111868 (0.0008) [2023-12-27 13:21:10,430][20347] Updated weights for policy 0, policy_version 1111878 (0.0006) [2023-12-27 13:21:10,455][20381] Updated weights for policy 1, policy_version 1110818 (0.0008) [2023-12-27 13:21:10,485][20347] Updated weights for policy 0, policy_version 1111888 (0.0006) [2023-12-27 13:21:10,507][20381] Updated weights for policy 1, policy_version 1110828 (0.0007) [2023-12-27 13:21:10,540][20347] Updated weights for policy 0, policy_version 1111898 (0.0006) [2023-12-27 13:21:10,560][20381] Updated weights for policy 1, policy_version 1110838 (0.0006) [2023-12-27 13:21:11,218][20347] Updated weights for policy 0, policy_version 1111908 (0.0006) [2023-12-27 13:21:11,259][20381] Updated weights for policy 1, policy_version 1110848 (0.0008) [2023-12-27 13:21:11,276][20347] Updated weights for policy 0, policy_version 1111918 (0.0007) [2023-12-27 13:21:11,313][20381] Updated weights for policy 1, policy_version 1110858 (0.0006) [2023-12-27 13:21:11,339][20347] Updated weights for policy 0, policy_version 1111928 (0.0006) [2023-12-27 13:21:11,379][20381] Updated weights for policy 1, policy_version 1110868 (0.0008) [2023-12-27 13:21:12,018][20347] Updated weights for policy 0, policy_version 1111938 (0.0009) [2023-12-27 13:21:12,080][20347] Updated weights for policy 0, policy_version 1111948 (0.0005) [2023-12-27 13:21:12,113][20381] Updated weights for policy 1, policy_version 1110878 (0.0008) [2023-12-27 13:21:12,142][20347] Updated weights for policy 0, policy_version 1111958 (0.0007) [2023-12-27 13:21:12,173][20381] Updated weights for policy 1, policy_version 1110888 (0.0008) [2023-12-27 13:21:12,206][20347] Updated weights for policy 0, policy_version 1111968 (0.0006) [2023-12-27 13:21:12,237][20381] Updated weights for policy 1, policy_version 1110898 (0.0008) [2023-12-27 13:21:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 569131008. Throughput: 0: 10366.5, 1: 10280.6. Samples: 569116764. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:12,363][19170] Avg episode reward: [(0, '894.910'), (1, '627.060')] [2023-12-27 13:21:12,840][20347] Updated weights for policy 0, policy_version 1111978 (0.0007) [2023-12-27 13:21:12,909][20347] Updated weights for policy 0, policy_version 1111988 (0.0007) [2023-12-27 13:21:12,976][20347] Updated weights for policy 0, policy_version 1111998 (0.0006) [2023-12-27 13:21:12,994][20381] Updated weights for policy 1, policy_version 1110908 (0.0008) [2023-12-27 13:21:13,051][20381] Updated weights for policy 1, policy_version 1110918 (0.0007) [2023-12-27 13:21:13,110][20381] Updated weights for policy 1, policy_version 1110928 (0.0008) [2023-12-27 13:21:13,600][20347] Updated weights for policy 0, policy_version 1112008 (0.0006) [2023-12-27 13:21:13,654][20347] Updated weights for policy 0, policy_version 1112018 (0.0007) [2023-12-27 13:21:13,708][20347] Updated weights for policy 0, policy_version 1112028 (0.0007) [2023-12-27 13:21:13,802][20381] Updated weights for policy 1, policy_version 1110938 (0.0008) [2023-12-27 13:21:13,851][20381] Updated weights for policy 1, policy_version 1110948 (0.0007) [2023-12-27 13:21:13,908][20381] Updated weights for policy 1, policy_version 1110958 (0.0008) [2023-12-27 13:21:13,960][20381] Updated weights for policy 1, policy_version 1110968 (0.0009) [2023-12-27 13:21:14,379][20347] Updated weights for policy 0, policy_version 1112038 (0.0008) [2023-12-27 13:21:14,438][20347] Updated weights for policy 0, policy_version 1112048 (0.0007) [2023-12-27 13:21:14,496][20347] Updated weights for policy 0, policy_version 1112058 (0.0008) [2023-12-27 13:21:14,648][20381] Updated weights for policy 1, policy_version 1110978 (0.0007) [2023-12-27 13:21:14,692][20381] Updated weights for policy 1, policy_version 1110988 (0.0008) [2023-12-27 13:21:14,737][20381] Updated weights for policy 1, policy_version 1110998 (0.0007) [2023-12-27 13:21:15,208][20347] Updated weights for policy 0, policy_version 1112068 (0.0008) [2023-12-27 13:21:15,266][20347] Updated weights for policy 0, policy_version 1112078 (0.0009) [2023-12-27 13:21:15,319][20347] Updated weights for policy 0, policy_version 1112088 (0.0007) [2023-12-27 13:21:15,490][20381] Updated weights for policy 1, policy_version 1111008 (0.0007) [2023-12-27 13:21:15,552][20381] Updated weights for policy 1, policy_version 1111018 (0.0008) [2023-12-27 13:21:15,613][20381] Updated weights for policy 1, policy_version 1111028 (0.0008) [2023-12-27 13:21:16,013][20347] Updated weights for policy 0, policy_version 1112098 (0.0007) [2023-12-27 13:21:16,071][20347] Updated weights for policy 0, policy_version 1112108 (0.0005) [2023-12-27 13:21:16,129][20347] Updated weights for policy 0, policy_version 1112118 (0.0006) [2023-12-27 13:21:16,188][20347] Updated weights for policy 0, policy_version 1112128 (0.0005) [2023-12-27 13:21:16,324][20381] Updated weights for policy 1, policy_version 1111038 (0.0008) [2023-12-27 13:21:16,387][20381] Updated weights for policy 1, policy_version 1111048 (0.0007) [2023-12-27 13:21:16,450][20381] Updated weights for policy 1, policy_version 1111058 (0.0008) [2023-12-27 13:21:16,724][20347] Updated weights for policy 0, policy_version 1112138 (0.0006) [2023-12-27 13:21:16,778][20347] Updated weights for policy 0, policy_version 1112148 (0.0005) [2023-12-27 13:21:16,838][20347] Updated weights for policy 0, policy_version 1112158 (0.0005) [2023-12-27 13:21:17,197][20381] Updated weights for policy 1, policy_version 1111068 (0.0007) [2023-12-27 13:21:17,241][20381] Updated weights for policy 1, policy_version 1111078 (0.0005) [2023-12-27 13:21:17,298][20381] Updated weights for policy 1, policy_version 1111088 (0.0005) [2023-12-27 13:21:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 569229312. Throughput: 0: 10343.7, 1: 10179.9. Samples: 569237516. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:17,364][19170] Avg episode reward: [(0, '941.910'), (1, '609.390')] [2023-12-27 13:21:17,395][20347] Updated weights for policy 0, policy_version 1112168 (0.0009) [2023-12-27 13:21:17,455][20347] Updated weights for policy 0, policy_version 1112178 (0.0009) [2023-12-27 13:21:17,514][20347] Updated weights for policy 0, policy_version 1112188 (0.0006) [2023-12-27 13:21:17,940][20381] Updated weights for policy 1, policy_version 1111098 (0.0006) [2023-12-27 13:21:18,000][20381] Updated weights for policy 1, policy_version 1111108 (0.0009) [2023-12-27 13:21:18,063][20381] Updated weights for policy 1, policy_version 1111118 (0.0007) [2023-12-27 13:21:18,097][20347] Updated weights for policy 0, policy_version 1112198 (0.0005) [2023-12-27 13:21:18,114][20381] Updated weights for policy 1, policy_version 1111128 (0.0007) [2023-12-27 13:21:18,157][20347] Updated weights for policy 0, policy_version 1112208 (0.0006) [2023-12-27 13:21:18,216][20347] Updated weights for policy 0, policy_version 1112218 (0.0008) [2023-12-27 13:21:18,834][20381] Updated weights for policy 1, policy_version 1111138 (0.0006) [2023-12-27 13:21:18,891][20381] Updated weights for policy 1, policy_version 1111148 (0.0008) [2023-12-27 13:21:18,930][20347] Updated weights for policy 0, policy_version 1112228 (0.0008) [2023-12-27 13:21:18,953][20381] Updated weights for policy 1, policy_version 1111158 (0.0007) [2023-12-27 13:21:18,983][20347] Updated weights for policy 0, policy_version 1112238 (0.0006) [2023-12-27 13:21:19,039][20347] Updated weights for policy 0, policy_version 1112248 (0.0005) [2023-12-27 13:21:19,709][20381] Updated weights for policy 1, policy_version 1111168 (0.0008) [2023-12-27 13:21:19,728][20347] Updated weights for policy 0, policy_version 1112258 (0.0008) [2023-12-27 13:21:19,775][20381] Updated weights for policy 1, policy_version 1111178 (0.0008) [2023-12-27 13:21:19,788][20347] Updated weights for policy 0, policy_version 1112268 (0.0007) [2023-12-27 13:21:19,839][20381] Updated weights for policy 1, policy_version 1111188 (0.0008) [2023-12-27 13:21:19,854][20347] Updated weights for policy 0, policy_version 1112278 (0.0008) [2023-12-27 13:21:19,915][20347] Updated weights for policy 0, policy_version 1112288 (0.0009) [2023-12-27 13:21:20,558][20381] Updated weights for policy 1, policy_version 1111198 (0.0008) [2023-12-27 13:21:20,622][20381] Updated weights for policy 1, policy_version 1111208 (0.0007) [2023-12-27 13:21:20,638][20347] Updated weights for policy 0, policy_version 1112298 (0.0008) [2023-12-27 13:21:20,685][20381] Updated weights for policy 1, policy_version 1111218 (0.0005) [2023-12-27 13:21:20,697][20347] Updated weights for policy 0, policy_version 1112308 (0.0008) [2023-12-27 13:21:20,757][20347] Updated weights for policy 0, policy_version 1112318 (0.0008) [2023-12-27 13:21:21,419][20381] Updated weights for policy 1, policy_version 1111228 (0.0006) [2023-12-27 13:21:21,476][20381] Updated weights for policy 1, policy_version 1111238 (0.0008) [2023-12-27 13:21:21,532][20381] Updated weights for policy 1, policy_version 1111248 (0.0007) [2023-12-27 13:21:21,554][20347] Updated weights for policy 0, policy_version 1112328 (0.0010) [2023-12-27 13:21:21,611][20347] Updated weights for policy 0, policy_version 1112338 (0.0011) [2023-12-27 13:21:21,675][20347] Updated weights for policy 0, policy_version 1112348 (0.0011) [2023-12-27 13:21:22,229][20381] Updated weights for policy 1, policy_version 1111258 (0.0007) [2023-12-27 13:21:22,291][20381] Updated weights for policy 1, policy_version 1111268 (0.0007) [2023-12-27 13:21:22,355][20381] Updated weights for policy 1, policy_version 1111278 (0.0007) [2023-12-27 13:21:22,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 569319424. Throughput: 0: 10338.2, 1: 10096.7. Samples: 569297356. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:22,363][19170] Avg episode reward: [(0, '961.580'), (1, '624.890')] [2023-12-27 13:21:22,394][20347] Updated weights for policy 0, policy_version 1112358 (0.0012) [2023-12-27 13:21:22,423][20381] Updated weights for policy 1, policy_version 1111288 (0.0007) [2023-12-27 13:21:22,423][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001111288_284516352.pth... [2023-12-27 13:21:22,428][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001110104_284213248.pth [2023-12-27 13:21:22,455][20347] Updated weights for policy 0, policy_version 1112368 (0.0009) [2023-12-27 13:21:22,509][20347] Updated weights for policy 0, policy_version 1112378 (0.0009) [2023-12-27 13:21:22,545][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001112384_284819456.pth... [2023-12-27 13:21:22,550][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001111168_284508160.pth [2023-12-27 13:21:23,113][20347] Updated weights for policy 0, policy_version 1112388 (0.0005) [2023-12-27 13:21:23,150][20381] Updated weights for policy 1, policy_version 1111298 (0.0007) [2023-12-27 13:21:23,167][20347] Updated weights for policy 0, policy_version 1112398 (0.0005) [2023-12-27 13:21:23,202][20381] Updated weights for policy 1, policy_version 1111308 (0.0006) [2023-12-27 13:21:23,218][20347] Updated weights for policy 0, policy_version 1112408 (0.0005) [2023-12-27 13:21:23,255][20381] Updated weights for policy 1, policy_version 1111318 (0.0006) [2023-12-27 13:21:23,872][20347] Updated weights for policy 0, policy_version 1112418 (0.0007) [2023-12-27 13:21:23,927][20347] Updated weights for policy 0, policy_version 1112428 (0.0007) [2023-12-27 13:21:23,958][20381] Updated weights for policy 1, policy_version 1111328 (0.0006) [2023-12-27 13:21:23,982][20347] Updated weights for policy 0, policy_version 1112438 (0.0008) [2023-12-27 13:21:24,006][20381] Updated weights for policy 1, policy_version 1111338 (0.0006) [2023-12-27 13:21:24,033][20347] Updated weights for policy 0, policy_version 1112448 (0.0006) [2023-12-27 13:21:24,060][20381] Updated weights for policy 1, policy_version 1111348 (0.0007) [2023-12-27 13:21:24,682][20347] Updated weights for policy 0, policy_version 1112458 (0.0007) [2023-12-27 13:21:24,727][20347] Updated weights for policy 0, policy_version 1112468 (0.0007) [2023-12-27 13:21:24,767][20381] Updated weights for policy 1, policy_version 1111358 (0.0007) [2023-12-27 13:21:24,769][20347] Updated weights for policy 0, policy_version 1112478 (0.0006) [2023-12-27 13:21:24,818][20381] Updated weights for policy 1, policy_version 1111368 (0.0007) [2023-12-27 13:21:24,876][20381] Updated weights for policy 1, policy_version 1111378 (0.0008) [2023-12-27 13:21:25,464][20381] Updated weights for policy 1, policy_version 1111388 (0.0007) [2023-12-27 13:21:25,510][20347] Updated weights for policy 0, policy_version 1112488 (0.0006) [2023-12-27 13:21:25,520][20381] Updated weights for policy 1, policy_version 1111398 (0.0007) [2023-12-27 13:21:25,569][20347] Updated weights for policy 0, policy_version 1112498 (0.0006) [2023-12-27 13:21:25,572][20381] Updated weights for policy 1, policy_version 1111408 (0.0006) [2023-12-27 13:21:25,631][20347] Updated weights for policy 0, policy_version 1112508 (0.0007) [2023-12-27 13:21:26,178][20381] Updated weights for policy 1, policy_version 1111418 (0.0006) [2023-12-27 13:21:26,212][20347] Updated weights for policy 0, policy_version 1112518 (0.0008) [2023-12-27 13:21:26,226][20381] Updated weights for policy 1, policy_version 1111428 (0.0005) [2023-12-27 13:21:26,265][20347] Updated weights for policy 0, policy_version 1112528 (0.0008) [2023-12-27 13:21:26,278][20381] Updated weights for policy 1, policy_version 1111438 (0.0006) [2023-12-27 13:21:26,311][20347] Updated weights for policy 0, policy_version 1112538 (0.0007) [2023-12-27 13:21:26,323][20381] Updated weights for policy 1, policy_version 1111448 (0.0008) [2023-12-27 13:21:26,953][20381] Updated weights for policy 1, policy_version 1111458 (0.0005) [2023-12-27 13:21:26,995][20347] Updated weights for policy 0, policy_version 1112548 (0.0006) [2023-12-27 13:21:27,005][20381] Updated weights for policy 1, policy_version 1111468 (0.0005) [2023-12-27 13:21:27,053][20347] Updated weights for policy 0, policy_version 1112558 (0.0006) [2023-12-27 13:21:27,065][20381] Updated weights for policy 1, policy_version 1111478 (0.0006) [2023-12-27 13:21:27,110][20347] Updated weights for policy 0, policy_version 1112568 (0.0006) [2023-12-27 13:21:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 569434112. Throughput: 0: 10306.2, 1: 10091.2. Samples: 569419604. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 13:21:27,363][19170] Avg episode reward: [(0, '970.980'), (1, '517.060')] [2023-12-27 13:21:27,681][20347] Updated weights for policy 0, policy_version 1112578 (0.0007) [2023-12-27 13:21:27,739][20347] Updated weights for policy 0, policy_version 1112588 (0.0007) [2023-12-27 13:21:27,760][20381] Updated weights for policy 1, policy_version 1111488 (0.0005) [2023-12-27 13:21:27,793][20347] Updated weights for policy 0, policy_version 1112598 (0.0006) [2023-12-27 13:21:27,802][20381] Updated weights for policy 1, policy_version 1111498 (0.0007) [2023-12-27 13:21:27,841][20347] Updated weights for policy 0, policy_version 1112608 (0.0005) [2023-12-27 13:21:27,845][20381] Updated weights for policy 1, policy_version 1111508 (0.0007) [2023-12-27 13:21:28,493][20347] Updated weights for policy 0, policy_version 1112618 (0.0005) [2023-12-27 13:21:28,542][20347] Updated weights for policy 0, policy_version 1112628 (0.0005) [2023-12-27 13:21:28,584][20381] Updated weights for policy 1, policy_version 1111518 (0.0007) [2023-12-27 13:21:28,596][20347] Updated weights for policy 0, policy_version 1112638 (0.0006) [2023-12-27 13:21:28,647][20381] Updated weights for policy 1, policy_version 1111528 (0.0008) [2023-12-27 13:21:28,707][20381] Updated weights for policy 1, policy_version 1111538 (0.0007) [2023-12-27 13:21:29,238][20347] Updated weights for policy 0, policy_version 1112648 (0.0009) [2023-12-27 13:21:29,291][20347] Updated weights for policy 0, policy_version 1112658 (0.0007) [2023-12-27 13:21:29,359][20347] Updated weights for policy 0, policy_version 1112668 (0.0007) [2023-12-27 13:21:29,391][20381] Updated weights for policy 1, policy_version 1111548 (0.0007) [2023-12-27 13:21:29,454][20381] Updated weights for policy 1, policy_version 1111558 (0.0009) [2023-12-27 13:21:29,515][20381] Updated weights for policy 1, policy_version 1111568 (0.0008) [2023-12-27 13:21:30,056][20347] Updated weights for policy 0, policy_version 1112678 (0.0006) [2023-12-27 13:21:30,113][20347] Updated weights for policy 0, policy_version 1112688 (0.0008) [2023-12-27 13:21:30,169][20347] Updated weights for policy 0, policy_version 1112698 (0.0007) [2023-12-27 13:21:30,238][20381] Updated weights for policy 1, policy_version 1111578 (0.0008) [2023-12-27 13:21:30,292][20381] Updated weights for policy 1, policy_version 1111588 (0.0007) [2023-12-27 13:21:30,354][20381] Updated weights for policy 1, policy_version 1111598 (0.0007) [2023-12-27 13:21:30,407][20381] Updated weights for policy 1, policy_version 1111608 (0.0007) [2023-12-27 13:21:30,766][20347] Updated weights for policy 0, policy_version 1112708 (0.0007) [2023-12-27 13:21:30,819][20347] Updated weights for policy 0, policy_version 1112718 (0.0006) [2023-12-27 13:21:30,864][20347] Updated weights for policy 0, policy_version 1112728 (0.0006) [2023-12-27 13:21:31,130][20381] Updated weights for policy 1, policy_version 1111618 (0.0008) [2023-12-27 13:21:31,187][20381] Updated weights for policy 1, policy_version 1111628 (0.0008) [2023-12-27 13:21:31,249][20381] Updated weights for policy 1, policy_version 1111638 (0.0008) [2023-12-27 13:21:31,529][20347] Updated weights for policy 0, policy_version 1112738 (0.0005) [2023-12-27 13:21:31,589][20347] Updated weights for policy 0, policy_version 1112748 (0.0007) [2023-12-27 13:21:31,683][20347] Updated weights for policy 0, policy_version 1112758 (0.0008) [2023-12-27 13:21:31,752][20347] Updated weights for policy 0, policy_version 1112768 (0.0008) [2023-12-27 13:21:31,998][20381] Updated weights for policy 1, policy_version 1111648 (0.0008) [2023-12-27 13:21:32,053][20381] Updated weights for policy 1, policy_version 1111658 (0.0006) [2023-12-27 13:21:32,114][20381] Updated weights for policy 1, policy_version 1111668 (0.0006) [2023-12-27 13:21:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 569532416. Throughput: 0: 10340.1, 1: 10043.1. Samples: 569543304. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:32,363][19170] Avg episode reward: [(0, '970.790'), (1, '302.950')] [2023-12-27 13:21:32,380][20347] Updated weights for policy 0, policy_version 1112778 (0.0008) [2023-12-27 13:21:32,441][20347] Updated weights for policy 0, policy_version 1112788 (0.0009) [2023-12-27 13:21:32,494][20347] Updated weights for policy 0, policy_version 1112798 (0.0008) [2023-12-27 13:21:32,791][20381] Updated weights for policy 1, policy_version 1111678 (0.0008) [2023-12-27 13:21:32,844][20381] Updated weights for policy 1, policy_version 1111688 (0.0006) [2023-12-27 13:21:32,898][20381] Updated weights for policy 1, policy_version 1111698 (0.0007) [2023-12-27 13:21:33,160][20347] Updated weights for policy 0, policy_version 1112808 (0.0006) [2023-12-27 13:21:33,213][20347] Updated weights for policy 0, policy_version 1112818 (0.0005) [2023-12-27 13:21:33,271][20347] Updated weights for policy 0, policy_version 1112828 (0.0006) [2023-12-27 13:21:33,520][20381] Updated weights for policy 1, policy_version 1111708 (0.0009) [2023-12-27 13:21:33,568][20381] Updated weights for policy 1, policy_version 1111718 (0.0009) [2023-12-27 13:21:33,625][20381] Updated weights for policy 1, policy_version 1111728 (0.0009) [2023-12-27 13:21:33,803][20347] Updated weights for policy 0, policy_version 1112838 (0.0006) [2023-12-27 13:21:33,856][20347] Updated weights for policy 0, policy_version 1112848 (0.0005) [2023-12-27 13:21:33,904][20347] Updated weights for policy 0, policy_version 1112858 (0.0005) [2023-12-27 13:21:34,341][20381] Updated weights for policy 1, policy_version 1111738 (0.0007) [2023-12-27 13:21:34,400][20381] Updated weights for policy 1, policy_version 1111748 (0.0010) [2023-12-27 13:21:34,458][20381] Updated weights for policy 1, policy_version 1111758 (0.0010) [2023-12-27 13:21:34,510][20381] Updated weights for policy 1, policy_version 1111768 (0.0011) [2023-12-27 13:21:34,521][20347] Updated weights for policy 0, policy_version 1112868 (0.0006) [2023-12-27 13:21:34,553][20291] KL-divergence is very high: 131.4404 [2023-12-27 13:21:34,584][20347] Updated weights for policy 0, policy_version 1112878 (0.0008) [2023-12-27 13:21:34,641][20347] Updated weights for policy 0, policy_version 1112888 (0.0006) [2023-12-27 13:21:35,191][20381] Updated weights for policy 1, policy_version 1111778 (0.0006) [2023-12-27 13:21:35,244][20381] Updated weights for policy 1, policy_version 1111788 (0.0007) [2023-12-27 13:21:35,252][20347] Updated weights for policy 0, policy_version 1112898 (0.0006) [2023-12-27 13:21:35,293][20381] Updated weights for policy 1, policy_version 1111798 (0.0005) [2023-12-27 13:21:35,310][20347] Updated weights for policy 0, policy_version 1112908 (0.0009) [2023-12-27 13:21:35,360][20347] Updated weights for policy 0, policy_version 1112918 (0.0007) [2023-12-27 13:21:35,418][20347] Updated weights for policy 0, policy_version 1112928 (0.0008) [2023-12-27 13:21:35,824][20381] Updated weights for policy 1, policy_version 1111808 (0.0005) [2023-12-27 13:21:35,885][20381] Updated weights for policy 1, policy_version 1111818 (0.0005) [2023-12-27 13:21:35,941][20381] Updated weights for policy 1, policy_version 1111828 (0.0008) [2023-12-27 13:21:36,139][20347] Updated weights for policy 0, policy_version 1112938 (0.0009) [2023-12-27 13:21:36,202][20347] Updated weights for policy 0, policy_version 1112948 (0.0007) [2023-12-27 13:21:36,259][20347] Updated weights for policy 0, policy_version 1112958 (0.0008) [2023-12-27 13:21:36,597][20381] Updated weights for policy 1, policy_version 1111838 (0.0006) [2023-12-27 13:21:36,663][20381] Updated weights for policy 1, policy_version 1111848 (0.0007) [2023-12-27 13:21:36,723][20381] Updated weights for policy 1, policy_version 1111858 (0.0008) [2023-12-27 13:21:36,969][20347] Updated weights for policy 0, policy_version 1112968 (0.0007) [2023-12-27 13:21:37,026][20347] Updated weights for policy 0, policy_version 1112978 (0.0008) [2023-12-27 13:21:37,089][20347] Updated weights for policy 0, policy_version 1112988 (0.0007) [2023-12-27 13:21:37,324][20381] Updated weights for policy 1, policy_version 1111868 (0.0007) [2023-12-27 13:21:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 569638912. Throughput: 0: 10420.5, 1: 10066.9. Samples: 569608752. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:37,363][19170] Avg episode reward: [(0, '951.270'), (1, '535.790')] [2023-12-27 13:21:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001112992_284975104.pth... [2023-12-27 13:21:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001111776_284663808.pth [2023-12-27 13:21:37,384][20381] Updated weights for policy 1, policy_version 1111878 (0.0005) [2023-12-27 13:21:37,435][20381] Updated weights for policy 1, policy_version 1111888 (0.0005) [2023-12-27 13:21:37,479][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001111896_284672000.pth... [2023-12-27 13:21:37,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001110712_284368896.pth [2023-12-27 13:21:37,800][20347] Updated weights for policy 0, policy_version 1112998 (0.0006) [2023-12-27 13:21:37,853][20347] Updated weights for policy 0, policy_version 1113008 (0.0005) [2023-12-27 13:21:37,909][20347] Updated weights for policy 0, policy_version 1113018 (0.0005) [2023-12-27 13:21:38,050][20381] Updated weights for policy 1, policy_version 1111898 (0.0005) [2023-12-27 13:21:38,096][20381] Updated weights for policy 1, policy_version 1111908 (0.0005) [2023-12-27 13:21:38,140][20381] Updated weights for policy 1, policy_version 1111918 (0.0005) [2023-12-27 13:21:38,193][20381] Updated weights for policy 1, policy_version 1111928 (0.0005) [2023-12-27 13:21:38,458][20347] Updated weights for policy 0, policy_version 1113028 (0.0006) [2023-12-27 13:21:38,531][20347] Updated weights for policy 0, policy_version 1113038 (0.0005) [2023-12-27 13:21:38,586][20347] Updated weights for policy 0, policy_version 1113048 (0.0007) [2023-12-27 13:21:38,849][20381] Updated weights for policy 1, policy_version 1111938 (0.0011) [2023-12-27 13:21:38,909][20381] Updated weights for policy 1, policy_version 1111948 (0.0011) [2023-12-27 13:21:38,969][20381] Updated weights for policy 1, policy_version 1111958 (0.0011) [2023-12-27 13:21:39,232][20347] Updated weights for policy 0, policy_version 1113058 (0.0007) [2023-12-27 13:21:39,282][20347] Updated weights for policy 0, policy_version 1113068 (0.0010) [2023-12-27 13:21:39,335][20347] Updated weights for policy 0, policy_version 1113078 (0.0010) [2023-12-27 13:21:39,403][20347] Updated weights for policy 0, policy_version 1113088 (0.0011) [2023-12-27 13:21:39,660][20381] Updated weights for policy 1, policy_version 1111968 (0.0011) [2023-12-27 13:21:39,718][20381] Updated weights for policy 1, policy_version 1111978 (0.0011) [2023-12-27 13:21:39,781][20381] Updated weights for policy 1, policy_version 1111988 (0.0011) [2023-12-27 13:21:40,187][20347] Updated weights for policy 0, policy_version 1113098 (0.0011) [2023-12-27 13:21:40,253][20347] Updated weights for policy 0, policy_version 1113108 (0.0011) [2023-12-27 13:21:40,312][20347] Updated weights for policy 0, policy_version 1113118 (0.0010) [2023-12-27 13:21:40,506][20381] Updated weights for policy 1, policy_version 1111998 (0.0011) [2023-12-27 13:21:40,559][20381] Updated weights for policy 1, policy_version 1112008 (0.0011) [2023-12-27 13:21:40,617][20381] Updated weights for policy 1, policy_version 1112018 (0.0010) [2023-12-27 13:21:40,995][20347] Updated weights for policy 0, policy_version 1113128 (0.0010) [2023-12-27 13:21:41,056][20347] Updated weights for policy 0, policy_version 1113138 (0.0010) [2023-12-27 13:21:41,122][20347] Updated weights for policy 0, policy_version 1113148 (0.0011) [2023-12-27 13:21:41,388][20381] Updated weights for policy 1, policy_version 1112028 (0.0010) [2023-12-27 13:21:41,456][20381] Updated weights for policy 1, policy_version 1112038 (0.0011) [2023-12-27 13:21:41,520][20381] Updated weights for policy 1, policy_version 1112048 (0.0011) [2023-12-27 13:21:41,883][20347] Updated weights for policy 0, policy_version 1113158 (0.0010) [2023-12-27 13:21:41,936][20347] Updated weights for policy 0, policy_version 1113168 (0.0009) [2023-12-27 13:21:41,990][20347] Updated weights for policy 0, policy_version 1113178 (0.0005) [2023-12-27 13:21:42,212][20381] Updated weights for policy 1, policy_version 1112058 (0.0010) [2023-12-27 13:21:42,280][20381] Updated weights for policy 1, policy_version 1112068 (0.0007) [2023-12-27 13:21:42,337][20381] Updated weights for policy 1, policy_version 1112078 (0.0008) [2023-12-27 13:21:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 569737216. Throughput: 0: 10341.0, 1: 10082.7. Samples: 569732508. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:42,364][19170] Avg episode reward: [(0, '941.400'), (1, '595.540')] [2023-12-27 13:21:42,402][20381] Updated weights for policy 1, policy_version 1112088 (0.0008) [2023-12-27 13:21:42,653][20347] Updated weights for policy 0, policy_version 1113188 (0.0006) [2023-12-27 13:21:42,715][20347] Updated weights for policy 0, policy_version 1113198 (0.0009) [2023-12-27 13:21:42,774][20347] Updated weights for policy 0, policy_version 1113208 (0.0011) [2023-12-27 13:21:43,110][20381] Updated weights for policy 1, policy_version 1112098 (0.0007) [2023-12-27 13:21:43,170][20381] Updated weights for policy 1, policy_version 1112108 (0.0006) [2023-12-27 13:21:43,217][20381] Updated weights for policy 1, policy_version 1112118 (0.0007) [2023-12-27 13:21:43,412][20347] Updated weights for policy 0, policy_version 1113218 (0.0009) [2023-12-27 13:21:43,468][20347] Updated weights for policy 0, policy_version 1113228 (0.0006) [2023-12-27 13:21:43,520][20347] Updated weights for policy 0, policy_version 1113238 (0.0007) [2023-12-27 13:21:43,579][20347] Updated weights for policy 0, policy_version 1113248 (0.0006) [2023-12-27 13:21:43,899][20381] Updated weights for policy 1, policy_version 1112128 (0.0007) [2023-12-27 13:21:43,943][20381] Updated weights for policy 1, policy_version 1112138 (0.0007) [2023-12-27 13:21:43,995][20381] Updated weights for policy 1, policy_version 1112148 (0.0007) [2023-12-27 13:21:44,159][20347] Updated weights for policy 0, policy_version 1113258 (0.0006) [2023-12-27 13:21:44,216][20347] Updated weights for policy 0, policy_version 1113268 (0.0006) [2023-12-27 13:21:44,280][20347] Updated weights for policy 0, policy_version 1113278 (0.0007) [2023-12-27 13:21:44,593][20381] Updated weights for policy 1, policy_version 1112158 (0.0008) [2023-12-27 13:21:44,658][20381] Updated weights for policy 1, policy_version 1112168 (0.0008) [2023-12-27 13:21:44,720][20381] Updated weights for policy 1, policy_version 1112178 (0.0008) [2023-12-27 13:21:44,943][20347] Updated weights for policy 0, policy_version 1113288 (0.0008) [2023-12-27 13:21:45,000][20347] Updated weights for policy 0, policy_version 1113298 (0.0008) [2023-12-27 13:21:45,055][20347] Updated weights for policy 0, policy_version 1113308 (0.0008) [2023-12-27 13:21:45,452][20381] Updated weights for policy 1, policy_version 1112188 (0.0008) [2023-12-27 13:21:45,506][20381] Updated weights for policy 1, policy_version 1112198 (0.0008) [2023-12-27 13:21:45,556][20381] Updated weights for policy 1, policy_version 1112208 (0.0008) [2023-12-27 13:21:45,779][20347] Updated weights for policy 0, policy_version 1113318 (0.0007) [2023-12-27 13:21:45,815][20291] KL-divergence is very high: 254.7597 [2023-12-27 13:21:45,820][20291] KL-divergence is very high: 568.3104 [2023-12-27 13:21:45,827][20291] KL-divergence is very high: 172.3510 [2023-12-27 13:21:45,839][20347] Updated weights for policy 0, policy_version 1113328 (0.0009) [2023-12-27 13:21:45,862][20291] KL-divergence is very high: 390.7026 [2023-12-27 13:21:45,870][20291] KL-divergence is very high: 568.7408 [2023-12-27 13:21:45,876][20291] KL-divergence is very high: 111.0236 [2023-12-27 13:21:45,898][20347] Updated weights for policy 0, policy_version 1113338 (0.0007) [2023-12-27 13:21:45,910][20291] KL-divergence is very high: 167.9213 [2023-12-27 13:21:45,915][20291] KL-divergence is very high: 193.9578 [2023-12-27 13:21:46,259][20381] Updated weights for policy 1, policy_version 1112218 (0.0008) [2023-12-27 13:21:46,311][20381] Updated weights for policy 1, policy_version 1112228 (0.0008) [2023-12-27 13:21:46,363][20381] Updated weights for policy 1, policy_version 1112238 (0.0008) [2023-12-27 13:21:46,410][20381] Updated weights for policy 1, policy_version 1112248 (0.0007) [2023-12-27 13:21:46,584][20347] Updated weights for policy 0, policy_version 1113348 (0.0008) [2023-12-27 13:21:46,641][20347] Updated weights for policy 0, policy_version 1113358 (0.0007) [2023-12-27 13:21:46,686][20347] Updated weights for policy 0, policy_version 1113368 (0.0008) [2023-12-27 13:21:47,031][20381] Updated weights for policy 1, policy_version 1112258 (0.0005) [2023-12-27 13:21:47,092][20381] Updated weights for policy 1, policy_version 1112268 (0.0005) [2023-12-27 13:21:47,156][20381] Updated weights for policy 1, policy_version 1112278 (0.0005) [2023-12-27 13:21:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 569843712. Throughput: 0: 10336.0, 1: 10121.8. Samples: 569854480. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:47,363][19170] Avg episode reward: [(0, '931.850'), (1, '564.980')] [2023-12-27 13:21:47,421][20347] Updated weights for policy 0, policy_version 1113378 (0.0007) [2023-12-27 13:21:47,476][20347] Updated weights for policy 0, policy_version 1113388 (0.0005) [2023-12-27 13:21:47,523][20347] Updated weights for policy 0, policy_version 1113398 (0.0006) [2023-12-27 13:21:47,576][20347] Updated weights for policy 0, policy_version 1113408 (0.0006) [2023-12-27 13:21:47,736][20381] Updated weights for policy 1, policy_version 1112288 (0.0007) [2023-12-27 13:21:47,789][20381] Updated weights for policy 1, policy_version 1112298 (0.0009) [2023-12-27 13:21:47,840][20381] Updated weights for policy 1, policy_version 1112308 (0.0007) [2023-12-27 13:21:48,165][20347] Updated weights for policy 0, policy_version 1113418 (0.0005) [2023-12-27 13:21:48,212][20347] Updated weights for policy 0, policy_version 1113428 (0.0006) [2023-12-27 13:21:48,263][20347] Updated weights for policy 0, policy_version 1113438 (0.0007) [2023-12-27 13:21:48,581][20381] Updated weights for policy 1, policy_version 1112318 (0.0007) [2023-12-27 13:21:48,628][20381] Updated weights for policy 1, policy_version 1112328 (0.0006) [2023-12-27 13:21:48,677][20381] Updated weights for policy 1, policy_version 1112338 (0.0008) [2023-12-27 13:21:48,968][20347] Updated weights for policy 0, policy_version 1113448 (0.0007) [2023-12-27 13:21:49,023][20347] Updated weights for policy 0, policy_version 1113458 (0.0007) [2023-12-27 13:21:49,072][20347] Updated weights for policy 0, policy_version 1113468 (0.0008) [2023-12-27 13:21:49,464][20381] Updated weights for policy 1, policy_version 1112348 (0.0008) [2023-12-27 13:21:49,509][20381] Updated weights for policy 1, policy_version 1112358 (0.0008) [2023-12-27 13:21:49,555][20381] Updated weights for policy 1, policy_version 1112368 (0.0008) [2023-12-27 13:21:49,877][20347] Updated weights for policy 0, policy_version 1113478 (0.0007) [2023-12-27 13:21:49,937][20347] Updated weights for policy 0, policy_version 1113488 (0.0008) [2023-12-27 13:21:49,999][20347] Updated weights for policy 0, policy_version 1113498 (0.0009) [2023-12-27 13:21:50,409][20381] Updated weights for policy 1, policy_version 1112378 (0.0009) [2023-12-27 13:21:50,468][20381] Updated weights for policy 1, policy_version 1112388 (0.0008) [2023-12-27 13:21:50,526][20381] Updated weights for policy 1, policy_version 1112398 (0.0008) [2023-12-27 13:21:50,586][20381] Updated weights for policy 1, policy_version 1112408 (0.0009) [2023-12-27 13:21:50,802][20347] Updated weights for policy 0, policy_version 1113508 (0.0008) [2023-12-27 13:21:50,866][20347] Updated weights for policy 0, policy_version 1113518 (0.0007) [2023-12-27 13:21:50,930][20347] Updated weights for policy 0, policy_version 1113528 (0.0008) [2023-12-27 13:21:51,298][20381] Updated weights for policy 1, policy_version 1112418 (0.0011) [2023-12-27 13:21:51,353][20381] Updated weights for policy 1, policy_version 1112428 (0.0010) [2023-12-27 13:21:51,416][20381] Updated weights for policy 1, policy_version 1112438 (0.0009) [2023-12-27 13:21:51,672][20347] Updated weights for policy 0, policy_version 1113538 (0.0007) [2023-12-27 13:21:51,738][20347] Updated weights for policy 0, policy_version 1113548 (0.0008) [2023-12-27 13:21:51,800][20347] Updated weights for policy 0, policy_version 1113558 (0.0008) [2023-12-27 13:21:51,856][20347] Updated weights for policy 0, policy_version 1113568 (0.0007) [2023-12-27 13:21:52,122][20381] Updated weights for policy 1, policy_version 1112448 (0.0007) [2023-12-27 13:21:52,175][20381] Updated weights for policy 1, policy_version 1112458 (0.0005) [2023-12-27 13:21:52,229][20381] Updated weights for policy 1, policy_version 1112468 (0.0006) [2023-12-27 13:21:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 569942016. Throughput: 0: 10346.5, 1: 10069.5. Samples: 569912912. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:52,363][19170] Avg episode reward: [(0, '922.130'), (1, '549.630')] [2023-12-27 13:21:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001112472_284819456.pth... [2023-12-27 13:21:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001113568_285122560.pth... [2023-12-27 13:21:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001111288_284516352.pth [2023-12-27 13:21:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001112384_284819456.pth [2023-12-27 13:21:52,613][20347] Updated weights for policy 0, policy_version 1113578 (0.0006) [2023-12-27 13:21:52,668][20347] Updated weights for policy 0, policy_version 1113588 (0.0008) [2023-12-27 13:21:52,726][20347] Updated weights for policy 0, policy_version 1113598 (0.0008) [2023-12-27 13:21:52,878][20381] Updated weights for policy 1, policy_version 1112478 (0.0007) [2023-12-27 13:21:52,941][20381] Updated weights for policy 1, policy_version 1112488 (0.0008) [2023-12-27 13:21:53,000][20381] Updated weights for policy 1, policy_version 1112498 (0.0008) [2023-12-27 13:21:53,339][20347] Updated weights for policy 0, policy_version 1113608 (0.0005) [2023-12-27 13:21:53,390][20347] Updated weights for policy 0, policy_version 1113618 (0.0007) [2023-12-27 13:21:53,439][20347] Updated weights for policy 0, policy_version 1113628 (0.0007) [2023-12-27 13:21:53,702][20381] Updated weights for policy 1, policy_version 1112508 (0.0008) [2023-12-27 13:21:53,753][20381] Updated weights for policy 1, policy_version 1112518 (0.0007) [2023-12-27 13:21:53,802][20381] Updated weights for policy 1, policy_version 1112528 (0.0006) [2023-12-27 13:21:54,113][20347] Updated weights for policy 0, policy_version 1113638 (0.0007) [2023-12-27 13:21:54,172][20347] Updated weights for policy 0, policy_version 1113648 (0.0007) [2023-12-27 13:21:54,224][20347] Updated weights for policy 0, policy_version 1113658 (0.0007) [2023-12-27 13:21:54,403][20381] Updated weights for policy 1, policy_version 1112538 (0.0006) [2023-12-27 13:21:54,453][20381] Updated weights for policy 1, policy_version 1112548 (0.0010) [2023-12-27 13:21:54,513][20381] Updated weights for policy 1, policy_version 1112558 (0.0007) [2023-12-27 13:21:54,570][20381] Updated weights for policy 1, policy_version 1112568 (0.0005) [2023-12-27 13:21:54,853][20347] Updated weights for policy 0, policy_version 1113668 (0.0006) [2023-12-27 13:21:54,908][20347] Updated weights for policy 0, policy_version 1113678 (0.0008) [2023-12-27 13:21:54,958][20347] Updated weights for policy 0, policy_version 1113688 (0.0010) [2023-12-27 13:21:55,186][20381] Updated weights for policy 1, policy_version 1112578 (0.0010) [2023-12-27 13:21:55,234][20381] Updated weights for policy 1, policy_version 1112588 (0.0010) [2023-12-27 13:21:55,278][20381] Updated weights for policy 1, policy_version 1112598 (0.0010) [2023-12-27 13:21:55,617][20347] Updated weights for policy 0, policy_version 1113698 (0.0007) [2023-12-27 13:21:55,674][20347] Updated weights for policy 0, policy_version 1113708 (0.0010) [2023-12-27 13:21:55,720][20347] Updated weights for policy 0, policy_version 1113718 (0.0010) [2023-12-27 13:21:55,771][20347] Updated weights for policy 0, policy_version 1113728 (0.0009) [2023-12-27 13:21:55,891][20381] Updated weights for policy 1, policy_version 1112608 (0.0006) [2023-12-27 13:21:55,952][20381] Updated weights for policy 1, policy_version 1112618 (0.0006) [2023-12-27 13:21:56,002][20381] Updated weights for policy 1, policy_version 1112628 (0.0005) [2023-12-27 13:21:56,366][20347] Updated weights for policy 0, policy_version 1113738 (0.0005) [2023-12-27 13:21:56,421][20347] Updated weights for policy 0, policy_version 1113748 (0.0005) [2023-12-27 13:21:56,478][20347] Updated weights for policy 0, policy_version 1113758 (0.0005) [2023-12-27 13:21:56,534][20381] Updated weights for policy 1, policy_version 1112638 (0.0008) [2023-12-27 13:21:56,578][20381] Updated weights for policy 1, policy_version 1112648 (0.0010) [2023-12-27 13:21:56,622][20381] Updated weights for policy 1, policy_version 1112658 (0.0010) [2023-12-27 13:21:56,984][20347] Updated weights for policy 0, policy_version 1113768 (0.0009) [2023-12-27 13:21:57,030][20347] Updated weights for policy 0, policy_version 1113778 (0.0010) [2023-12-27 13:21:57,075][20347] Updated weights for policy 0, policy_version 1113788 (0.0010) [2023-12-27 13:21:57,323][20381] Updated weights for policy 1, policy_version 1112668 (0.0010) [2023-12-27 13:21:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 570048512. Throughput: 0: 10342.6, 1: 10110.6. Samples: 570037156. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:21:57,363][19170] Avg episode reward: [(0, '902.620'), (1, '513.170')] [2023-12-27 13:21:57,377][20381] Updated weights for policy 1, policy_version 1112678 (0.0010) [2023-12-27 13:21:57,431][20381] Updated weights for policy 1, policy_version 1112688 (0.0010) [2023-12-27 13:21:57,714][20347] Updated weights for policy 0, policy_version 1113798 (0.0007) [2023-12-27 13:21:57,762][20347] Updated weights for policy 0, policy_version 1113808 (0.0005) [2023-12-27 13:21:57,818][20347] Updated weights for policy 0, policy_version 1113818 (0.0005) [2023-12-27 13:21:58,106][20381] Updated weights for policy 1, policy_version 1112698 (0.0010) [2023-12-27 13:21:58,162][20381] Updated weights for policy 1, policy_version 1112708 (0.0011) [2023-12-27 13:21:58,222][20381] Updated weights for policy 1, policy_version 1112718 (0.0011) [2023-12-27 13:21:58,276][20381] Updated weights for policy 1, policy_version 1112728 (0.0011) [2023-12-27 13:21:58,452][20347] Updated weights for policy 0, policy_version 1113828 (0.0007) [2023-12-27 13:21:58,514][20347] Updated weights for policy 0, policy_version 1113838 (0.0008) [2023-12-27 13:21:58,575][20347] Updated weights for policy 0, policy_version 1113848 (0.0008) [2023-12-27 13:21:59,079][20381] Updated weights for policy 1, policy_version 1112738 (0.0008) [2023-12-27 13:21:59,150][20381] Updated weights for policy 1, policy_version 1112748 (0.0009) [2023-12-27 13:21:59,216][20381] Updated weights for policy 1, policy_version 1112758 (0.0007) [2023-12-27 13:21:59,314][20347] Updated weights for policy 0, policy_version 1113858 (0.0007) [2023-12-27 13:21:59,377][20347] Updated weights for policy 0, policy_version 1113868 (0.0008) [2023-12-27 13:21:59,436][20347] Updated weights for policy 0, policy_version 1113878 (0.0009) [2023-12-27 13:21:59,490][20347] Updated weights for policy 0, policy_version 1113888 (0.0008) [2023-12-27 13:21:59,950][20381] Updated weights for policy 1, policy_version 1112768 (0.0011) [2023-12-27 13:22:00,004][20381] Updated weights for policy 1, policy_version 1112778 (0.0008) [2023-12-27 13:22:00,067][20381] Updated weights for policy 1, policy_version 1112788 (0.0006) [2023-12-27 13:22:00,185][20347] Updated weights for policy 0, policy_version 1113898 (0.0008) [2023-12-27 13:22:00,234][20347] Updated weights for policy 0, policy_version 1113908 (0.0006) [2023-12-27 13:22:00,282][20347] Updated weights for policy 0, policy_version 1113918 (0.0008) [2023-12-27 13:22:00,711][20381] Updated weights for policy 1, policy_version 1112798 (0.0005) [2023-12-27 13:22:00,762][20381] Updated weights for policy 1, policy_version 1112808 (0.0005) [2023-12-27 13:22:00,811][20381] Updated weights for policy 1, policy_version 1112818 (0.0005) [2023-12-27 13:22:00,978][20347] Updated weights for policy 0, policy_version 1113928 (0.0007) [2023-12-27 13:22:01,038][20347] Updated weights for policy 0, policy_version 1113939 (0.0007) [2023-12-27 13:22:01,091][20347] Updated weights for policy 0, policy_version 1113949 (0.0007) [2023-12-27 13:22:01,502][20381] Updated weights for policy 1, policy_version 1112828 (0.0007) [2023-12-27 13:22:01,562][20381] Updated weights for policy 1, policy_version 1112838 (0.0009) [2023-12-27 13:22:01,629][20381] Updated weights for policy 1, policy_version 1112848 (0.0007) [2023-12-27 13:22:01,828][20347] Updated weights for policy 0, policy_version 1113959 (0.0007) [2023-12-27 13:22:01,872][20347] Updated weights for policy 0, policy_version 1113969 (0.0007) [2023-12-27 13:22:01,919][20347] Updated weights for policy 0, policy_version 1113979 (0.0008) [2023-12-27 13:22:02,268][20381] Updated weights for policy 1, policy_version 1112858 (0.0007) [2023-12-27 13:22:02,330][20381] Updated weights for policy 1, policy_version 1112868 (0.0006) [2023-12-27 13:22:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 570146816. Throughput: 0: 10327.1, 1: 10172.8. Samples: 570160012. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:02,363][19170] Avg episode reward: [(0, '893.040'), (1, '446.910')] [2023-12-27 13:22:02,392][20381] Updated weights for policy 1, policy_version 1112878 (0.0007) [2023-12-27 13:22:02,437][20381] Updated weights for policy 1, policy_version 1112888 (0.0008) [2023-12-27 13:22:02,646][20347] Updated weights for policy 0, policy_version 1113989 (0.0009) [2023-12-27 13:22:02,702][20347] Updated weights for policy 0, policy_version 1113999 (0.0008) [2023-12-27 13:22:02,763][20347] Updated weights for policy 0, policy_version 1114009 (0.0009) [2023-12-27 13:22:03,073][20381] Updated weights for policy 1, policy_version 1112898 (0.0008) [2023-12-27 13:22:03,132][20381] Updated weights for policy 1, policy_version 1112908 (0.0009) [2023-12-27 13:22:03,186][20381] Updated weights for policy 1, policy_version 1112918 (0.0008) [2023-12-27 13:22:03,462][20347] Updated weights for policy 0, policy_version 1114019 (0.0008) [2023-12-27 13:22:03,521][20347] Updated weights for policy 0, policy_version 1114029 (0.0008) [2023-12-27 13:22:03,574][20347] Updated weights for policy 0, policy_version 1114039 (0.0007) [2023-12-27 13:22:03,890][20381] Updated weights for policy 1, policy_version 1112928 (0.0008) [2023-12-27 13:22:03,941][20381] Updated weights for policy 1, policy_version 1112938 (0.0008) [2023-12-27 13:22:03,997][20381] Updated weights for policy 1, policy_version 1112948 (0.0008) [2023-12-27 13:22:04,268][20347] Updated weights for policy 0, policy_version 1114049 (0.0007) [2023-12-27 13:22:04,332][20347] Updated weights for policy 0, policy_version 1114059 (0.0008) [2023-12-27 13:22:04,388][20347] Updated weights for policy 0, policy_version 1114069 (0.0007) [2023-12-27 13:22:04,435][20347] Updated weights for policy 0, policy_version 1114079 (0.0008) [2023-12-27 13:22:04,734][20381] Updated weights for policy 1, policy_version 1112958 (0.0008) [2023-12-27 13:22:04,789][20381] Updated weights for policy 1, policy_version 1112968 (0.0008) [2023-12-27 13:22:04,847][20381] Updated weights for policy 1, policy_version 1112978 (0.0007) [2023-12-27 13:22:05,116][20347] Updated weights for policy 0, policy_version 1114089 (0.0006) [2023-12-27 13:22:05,173][20347] Updated weights for policy 0, policy_version 1114099 (0.0005) [2023-12-27 13:22:05,230][20347] Updated weights for policy 0, policy_version 1114109 (0.0005) [2023-12-27 13:22:05,576][20381] Updated weights for policy 1, policy_version 1112988 (0.0008) [2023-12-27 13:22:05,633][20381] Updated weights for policy 1, policy_version 1112998 (0.0010) [2023-12-27 13:22:05,687][20381] Updated weights for policy 1, policy_version 1113008 (0.0010) [2023-12-27 13:22:05,794][20347] Updated weights for policy 0, policy_version 1114119 (0.0005) [2023-12-27 13:22:05,858][20347] Updated weights for policy 0, policy_version 1114129 (0.0006) [2023-12-27 13:22:05,908][20347] Updated weights for policy 0, policy_version 1114139 (0.0005) [2023-12-27 13:22:06,354][20381] Updated weights for policy 1, policy_version 1113018 (0.0010) [2023-12-27 13:22:06,413][20381] Updated weights for policy 1, policy_version 1113028 (0.0006) [2023-12-27 13:22:06,473][20381] Updated weights for policy 1, policy_version 1113038 (0.0007) [2023-12-27 13:22:06,529][20381] Updated weights for policy 1, policy_version 1113048 (0.0005) [2023-12-27 13:22:06,579][20347] Updated weights for policy 0, policy_version 1114149 (0.0006) [2023-12-27 13:22:06,636][20347] Updated weights for policy 0, policy_version 1114159 (0.0008) [2023-12-27 13:22:06,696][20347] Updated weights for policy 0, policy_version 1114169 (0.0008) [2023-12-27 13:22:07,145][20381] Updated weights for policy 1, policy_version 1113058 (0.0009) [2023-12-27 13:22:07,200][20381] Updated weights for policy 1, policy_version 1113068 (0.0009) [2023-12-27 13:22:07,247][20381] Updated weights for policy 1, policy_version 1113078 (0.0008) [2023-12-27 13:22:07,331][20347] Updated weights for policy 0, policy_version 1114179 (0.0006) [2023-12-27 13:22:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 570253312. Throughput: 0: 10324.0, 1: 10189.2. Samples: 570220448. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:07,363][19170] Avg episode reward: [(0, '922.250'), (1, '547.700')] [2023-12-27 13:22:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001113080_284975104.pth... [2023-12-27 13:22:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001111896_284672000.pth [2023-12-27 13:22:07,392][20347] Updated weights for policy 0, policy_version 1114189 (0.0005) [2023-12-27 13:22:07,455][20347] Updated weights for policy 0, policy_version 1114199 (0.0005) [2023-12-27 13:22:07,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001114208_285286400.pth... [2023-12-27 13:22:07,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001112992_284975104.pth [2023-12-27 13:22:07,878][20381] Updated weights for policy 1, policy_version 1113088 (0.0005) [2023-12-27 13:22:07,924][20381] Updated weights for policy 1, policy_version 1113098 (0.0005) [2023-12-27 13:22:07,980][20381] Updated weights for policy 1, policy_version 1113108 (0.0006) [2023-12-27 13:22:07,980][20347] Updated weights for policy 0, policy_version 1114209 (0.0006) [2023-12-27 13:22:08,039][20347] Updated weights for policy 0, policy_version 1114219 (0.0008) [2023-12-27 13:22:08,091][20347] Updated weights for policy 0, policy_version 1114229 (0.0007) [2023-12-27 13:22:08,144][20347] Updated weights for policy 0, policy_version 1114239 (0.0008) [2023-12-27 13:22:08,633][20381] Updated weights for policy 1, policy_version 1113118 (0.0008) [2023-12-27 13:22:08,691][20381] Updated weights for policy 1, policy_version 1113128 (0.0005) [2023-12-27 13:22:08,750][20381] Updated weights for policy 1, policy_version 1113138 (0.0006) [2023-12-27 13:22:08,870][20347] Updated weights for policy 0, policy_version 1114249 (0.0008) [2023-12-27 13:22:08,921][20347] Updated weights for policy 0, policy_version 1114259 (0.0008) [2023-12-27 13:22:08,983][20347] Updated weights for policy 0, policy_version 1114269 (0.0008) [2023-12-27 13:22:09,410][20381] Updated weights for policy 1, policy_version 1113148 (0.0007) [2023-12-27 13:22:09,474][20381] Updated weights for policy 1, policy_version 1113158 (0.0007) [2023-12-27 13:22:09,534][20381] Updated weights for policy 1, policy_version 1113168 (0.0008) [2023-12-27 13:22:09,761][20347] Updated weights for policy 0, policy_version 1114279 (0.0006) [2023-12-27 13:22:09,821][20347] Updated weights for policy 0, policy_version 1114289 (0.0006) [2023-12-27 13:22:09,892][20347] Updated weights for policy 0, policy_version 1114299 (0.0007) [2023-12-27 13:22:10,221][20381] Updated weights for policy 1, policy_version 1113178 (0.0007) [2023-12-27 13:22:10,277][20381] Updated weights for policy 1, policy_version 1113188 (0.0006) [2023-12-27 13:22:10,334][20381] Updated weights for policy 1, policy_version 1113198 (0.0008) [2023-12-27 13:22:10,397][20381] Updated weights for policy 1, policy_version 1113208 (0.0008) [2023-12-27 13:22:10,601][20347] Updated weights for policy 0, policy_version 1114309 (0.0007) [2023-12-27 13:22:10,655][20347] Updated weights for policy 0, policy_version 1114319 (0.0008) [2023-12-27 13:22:10,712][20347] Updated weights for policy 0, policy_version 1114329 (0.0007) [2023-12-27 13:22:11,068][20381] Updated weights for policy 1, policy_version 1113218 (0.0008) [2023-12-27 13:22:11,126][20381] Updated weights for policy 1, policy_version 1113228 (0.0006) [2023-12-27 13:22:11,194][20381] Updated weights for policy 1, policy_version 1113238 (0.0007) [2023-12-27 13:22:11,457][20347] Updated weights for policy 0, policy_version 1114339 (0.0006) [2023-12-27 13:22:11,501][20347] Updated weights for policy 0, policy_version 1114349 (0.0008) [2023-12-27 13:22:11,557][20347] Updated weights for policy 0, policy_version 1114359 (0.0007) [2023-12-27 13:22:11,882][20381] Updated weights for policy 1, policy_version 1113248 (0.0006) [2023-12-27 13:22:11,941][20381] Updated weights for policy 1, policy_version 1113258 (0.0008) [2023-12-27 13:22:11,998][20381] Updated weights for policy 1, policy_version 1113268 (0.0008) [2023-12-27 13:22:12,307][20347] Updated weights for policy 0, policy_version 1114369 (0.0009) [2023-12-27 13:22:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 570351616. Throughput: 0: 10352.4, 1: 10230.0. Samples: 570345816. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:12,363][19170] Avg episode reward: [(0, '941.820'), (1, '570.160')] [2023-12-27 13:22:12,375][20347] Updated weights for policy 0, policy_version 1114379 (0.0008) [2023-12-27 13:22:12,438][20347] Updated weights for policy 0, policy_version 1114389 (0.0008) [2023-12-27 13:22:12,449][20291] KL-divergence is very high: 109.6071 [2023-12-27 13:22:12,494][20347] Updated weights for policy 0, policy_version 1114399 (0.0008) [2023-12-27 13:22:12,674][20381] Updated weights for policy 1, policy_version 1113278 (0.0006) [2023-12-27 13:22:12,724][20381] Updated weights for policy 1, policy_version 1113288 (0.0006) [2023-12-27 13:22:12,775][20381] Updated weights for policy 1, policy_version 1113298 (0.0007) [2023-12-27 13:22:13,182][20347] Updated weights for policy 0, policy_version 1114409 (0.0007) [2023-12-27 13:22:13,236][20347] Updated weights for policy 0, policy_version 1114419 (0.0007) [2023-12-27 13:22:13,286][20347] Updated weights for policy 0, policy_version 1114429 (0.0008) [2023-12-27 13:22:13,444][20381] Updated weights for policy 1, policy_version 1113308 (0.0006) [2023-12-27 13:22:13,501][20381] Updated weights for policy 1, policy_version 1113318 (0.0008) [2023-12-27 13:22:13,557][20381] Updated weights for policy 1, policy_version 1113328 (0.0007) [2023-12-27 13:22:13,985][20347] Updated weights for policy 0, policy_version 1114439 (0.0008) [2023-12-27 13:22:14,043][20347] Updated weights for policy 0, policy_version 1114449 (0.0008) [2023-12-27 13:22:14,104][20347] Updated weights for policy 0, policy_version 1114459 (0.0008) [2023-12-27 13:22:14,252][20381] Updated weights for policy 1, policy_version 1113338 (0.0008) [2023-12-27 13:22:14,304][20381] Updated weights for policy 1, policy_version 1113348 (0.0007) [2023-12-27 13:22:14,355][20381] Updated weights for policy 1, policy_version 1113358 (0.0007) [2023-12-27 13:22:14,405][20381] Updated weights for policy 1, policy_version 1113368 (0.0008) [2023-12-27 13:22:14,855][20347] Updated weights for policy 0, policy_version 1114469 (0.0008) [2023-12-27 13:22:14,919][20347] Updated weights for policy 0, policy_version 1114479 (0.0007) [2023-12-27 13:22:14,989][20347] Updated weights for policy 0, policy_version 1114489 (0.0007) [2023-12-27 13:22:15,080][20381] Updated weights for policy 1, policy_version 1113378 (0.0010) [2023-12-27 13:22:15,138][20381] Updated weights for policy 1, policy_version 1113388 (0.0011) [2023-12-27 13:22:15,192][20381] Updated weights for policy 1, policy_version 1113398 (0.0011) [2023-12-27 13:22:15,598][20347] Updated weights for policy 0, policy_version 1114499 (0.0007) [2023-12-27 13:22:15,656][20347] Updated weights for policy 0, policy_version 1114509 (0.0006) [2023-12-27 13:22:15,707][20347] Updated weights for policy 0, policy_version 1114519 (0.0008) [2023-12-27 13:22:15,908][20381] Updated weights for policy 1, policy_version 1113408 (0.0009) [2023-12-27 13:22:15,970][20381] Updated weights for policy 1, policy_version 1113418 (0.0007) [2023-12-27 13:22:16,027][20381] Updated weights for policy 1, policy_version 1113428 (0.0008) [2023-12-27 13:22:16,318][20347] Updated weights for policy 0, policy_version 1114529 (0.0008) [2023-12-27 13:22:16,370][20347] Updated weights for policy 0, policy_version 1114539 (0.0006) [2023-12-27 13:22:16,426][20347] Updated weights for policy 0, policy_version 1114549 (0.0005) [2023-12-27 13:22:16,484][20347] Updated weights for policy 0, policy_version 1114559 (0.0006) [2023-12-27 13:22:16,775][20381] Updated weights for policy 1, policy_version 1113438 (0.0008) [2023-12-27 13:22:16,829][20381] Updated weights for policy 1, policy_version 1113448 (0.0005) [2023-12-27 13:22:16,882][20381] Updated weights for policy 1, policy_version 1113458 (0.0005) [2023-12-27 13:22:17,086][20347] Updated weights for policy 0, policy_version 1114569 (0.0007) [2023-12-27 13:22:17,140][20347] Updated weights for policy 0, policy_version 1114579 (0.0005) [2023-12-27 13:22:17,191][20347] Updated weights for policy 0, policy_version 1114589 (0.0006) [2023-12-27 13:22:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 570458112. Throughput: 0: 10264.7, 1: 10276.5. Samples: 570467660. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:17,363][19170] Avg episode reward: [(0, '990.270'), (1, '627.350')] [2023-12-27 13:22:17,470][20381] Updated weights for policy 1, policy_version 1113468 (0.0007) [2023-12-27 13:22:17,535][20381] Updated weights for policy 1, policy_version 1113478 (0.0008) [2023-12-27 13:22:17,594][20381] Updated weights for policy 1, policy_version 1113488 (0.0006) [2023-12-27 13:22:17,841][20347] Updated weights for policy 0, policy_version 1114599 (0.0009) [2023-12-27 13:22:17,889][20347] Updated weights for policy 0, policy_version 1114609 (0.0008) [2023-12-27 13:22:17,942][20347] Updated weights for policy 0, policy_version 1114619 (0.0006) [2023-12-27 13:22:18,135][20381] Updated weights for policy 1, policy_version 1113498 (0.0006) [2023-12-27 13:22:18,184][20381] Updated weights for policy 1, policy_version 1113508 (0.0005) [2023-12-27 13:22:18,234][20381] Updated weights for policy 1, policy_version 1113518 (0.0009) [2023-12-27 13:22:18,280][20381] Updated weights for policy 1, policy_version 1113528 (0.0008) [2023-12-27 13:22:18,552][20347] Updated weights for policy 0, policy_version 1114629 (0.0006) [2023-12-27 13:22:18,614][20347] Updated weights for policy 0, policy_version 1114639 (0.0009) [2023-12-27 13:22:18,667][20347] Updated weights for policy 0, policy_version 1114649 (0.0007) [2023-12-27 13:22:18,970][20381] Updated weights for policy 1, policy_version 1113538 (0.0008) [2023-12-27 13:22:19,019][20381] Updated weights for policy 1, policy_version 1113548 (0.0008) [2023-12-27 13:22:19,081][20381] Updated weights for policy 1, policy_version 1113558 (0.0007) [2023-12-27 13:22:19,349][20347] Updated weights for policy 0, policy_version 1114659 (0.0008) [2023-12-27 13:22:19,412][20347] Updated weights for policy 0, policy_version 1114669 (0.0007) [2023-12-27 13:22:19,473][20347] Updated weights for policy 0, policy_version 1114679 (0.0007) [2023-12-27 13:22:19,828][20381] Updated weights for policy 1, policy_version 1113568 (0.0008) [2023-12-27 13:22:19,891][20381] Updated weights for policy 1, policy_version 1113578 (0.0008) [2023-12-27 13:22:19,957][20381] Updated weights for policy 1, policy_version 1113588 (0.0007) [2023-12-27 13:22:20,255][20347] Updated weights for policy 0, policy_version 1114689 (0.0008) [2023-12-27 13:22:20,313][20347] Updated weights for policy 0, policy_version 1114699 (0.0009) [2023-12-27 13:22:20,371][20347] Updated weights for policy 0, policy_version 1114709 (0.0008) [2023-12-27 13:22:20,432][20347] Updated weights for policy 0, policy_version 1114719 (0.0007) [2023-12-27 13:22:20,684][20381] Updated weights for policy 1, policy_version 1113598 (0.0007) [2023-12-27 13:22:20,755][20381] Updated weights for policy 1, policy_version 1113608 (0.0006) [2023-12-27 13:22:20,818][20381] Updated weights for policy 1, policy_version 1113618 (0.0008) [2023-12-27 13:22:21,177][20347] Updated weights for policy 0, policy_version 1114729 (0.0008) [2023-12-27 13:22:21,241][20347] Updated weights for policy 0, policy_version 1114739 (0.0008) [2023-12-27 13:22:21,311][20347] Updated weights for policy 0, policy_version 1114749 (0.0009) [2023-12-27 13:22:21,524][20381] Updated weights for policy 1, policy_version 1113628 (0.0008) [2023-12-27 13:22:21,575][20381] Updated weights for policy 1, policy_version 1113638 (0.0007) [2023-12-27 13:22:21,637][20381] Updated weights for policy 1, policy_version 1113648 (0.0008) [2023-12-27 13:22:22,069][20347] Updated weights for policy 0, policy_version 1114759 (0.0007) [2023-12-27 13:22:22,125][20347] Updated weights for policy 0, policy_version 1114769 (0.0008) [2023-12-27 13:22:22,182][20347] Updated weights for policy 0, policy_version 1114779 (0.0008) [2023-12-27 13:22:22,324][20381] Updated weights for policy 1, policy_version 1113658 (0.0008) [2023-12-27 13:22:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 570556416. Throughput: 0: 10192.1, 1: 10269.5. Samples: 570529524. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:22,363][19170] Avg episode reward: [(0, '970.790'), (1, '596.470')] [2023-12-27 13:22:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001114784_285433856.pth... [2023-12-27 13:22:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001113568_285122560.pth [2023-12-27 13:22:22,393][20381] Updated weights for policy 1, policy_version 1113668 (0.0008) [2023-12-27 13:22:22,453][20381] Updated weights for policy 1, policy_version 1113678 (0.0008) [2023-12-27 13:22:22,517][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001113688_285130752.pth... [2023-12-27 13:22:22,519][20381] Updated weights for policy 1, policy_version 1113688 (0.0008) [2023-12-27 13:22:22,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001112472_284819456.pth [2023-12-27 13:22:22,826][20347] Updated weights for policy 0, policy_version 1114789 (0.0007) [2023-12-27 13:22:22,888][20347] Updated weights for policy 0, policy_version 1114799 (0.0007) [2023-12-27 13:22:22,946][20347] Updated weights for policy 0, policy_version 1114809 (0.0008) [2023-12-27 13:22:23,224][20381] Updated weights for policy 1, policy_version 1113698 (0.0008) [2023-12-27 13:22:23,285][20381] Updated weights for policy 1, policy_version 1113708 (0.0007) [2023-12-27 13:22:23,341][20381] Updated weights for policy 1, policy_version 1113718 (0.0008) [2023-12-27 13:22:23,555][20347] Updated weights for policy 0, policy_version 1114819 (0.0008) [2023-12-27 13:22:23,619][20347] Updated weights for policy 0, policy_version 1114829 (0.0007) [2023-12-27 13:22:23,674][20347] Updated weights for policy 0, policy_version 1114839 (0.0005) [2023-12-27 13:22:24,061][20381] Updated weights for policy 1, policy_version 1113728 (0.0008) [2023-12-27 13:22:24,123][20381] Updated weights for policy 1, policy_version 1113738 (0.0007) [2023-12-27 13:22:24,175][20381] Updated weights for policy 1, policy_version 1113748 (0.0008) [2023-12-27 13:22:24,324][20347] Updated weights for policy 0, policy_version 1114849 (0.0008) [2023-12-27 13:22:24,372][20347] Updated weights for policy 0, policy_version 1114859 (0.0011) [2023-12-27 13:22:24,417][20347] Updated weights for policy 0, policy_version 1114869 (0.0010) [2023-12-27 13:22:24,469][20347] Updated weights for policy 0, policy_version 1114879 (0.0010) [2023-12-27 13:22:24,848][20381] Updated weights for policy 1, policy_version 1113758 (0.0006) [2023-12-27 13:22:24,905][20381] Updated weights for policy 1, policy_version 1113768 (0.0007) [2023-12-27 13:22:24,954][20381] Updated weights for policy 1, policy_version 1113778 (0.0010) [2023-12-27 13:22:25,145][20347] Updated weights for policy 0, policy_version 1114889 (0.0007) [2023-12-27 13:22:25,192][20347] Updated weights for policy 0, policy_version 1114899 (0.0007) [2023-12-27 13:22:25,239][20347] Updated weights for policy 0, policy_version 1114909 (0.0007) [2023-12-27 13:22:25,616][20381] Updated weights for policy 1, policy_version 1113788 (0.0010) [2023-12-27 13:22:25,671][20381] Updated weights for policy 1, policy_version 1113798 (0.0010) [2023-12-27 13:22:25,728][20381] Updated weights for policy 1, policy_version 1113808 (0.0010) [2023-12-27 13:22:25,947][20347] Updated weights for policy 0, policy_version 1114919 (0.0006) [2023-12-27 13:22:25,996][20347] Updated weights for policy 0, policy_version 1114929 (0.0006) [2023-12-27 13:22:26,041][20347] Updated weights for policy 0, policy_version 1114939 (0.0007) [2023-12-27 13:22:26,379][20381] Updated weights for policy 1, policy_version 1113818 (0.0008) [2023-12-27 13:22:26,424][20381] Updated weights for policy 1, policy_version 1113828 (0.0010) [2023-12-27 13:22:26,468][20381] Updated weights for policy 1, policy_version 1113838 (0.0010) [2023-12-27 13:22:26,513][20381] Updated weights for policy 1, policy_version 1113848 (0.0010) [2023-12-27 13:22:26,738][20347] Updated weights for policy 0, policy_version 1114949 (0.0007) [2023-12-27 13:22:26,794][20347] Updated weights for policy 0, policy_version 1114959 (0.0007) [2023-12-27 13:22:26,849][20347] Updated weights for policy 0, policy_version 1114969 (0.0007) [2023-12-27 13:22:27,140][20381] Updated weights for policy 1, policy_version 1113858 (0.0005) [2023-12-27 13:22:27,194][20381] Updated weights for policy 1, policy_version 1113868 (0.0010) [2023-12-27 13:22:27,246][20381] Updated weights for policy 1, policy_version 1113878 (0.0010) [2023-12-27 13:22:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 570662912. Throughput: 0: 10198.1, 1: 10202.9. Samples: 570650548. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:27,363][19170] Avg episode reward: [(0, '961.110'), (1, '655.760')] [2023-12-27 13:22:27,521][20347] Updated weights for policy 0, policy_version 1114979 (0.0008) [2023-12-27 13:22:27,583][20347] Updated weights for policy 0, policy_version 1114989 (0.0006) [2023-12-27 13:22:27,638][20347] Updated weights for policy 0, policy_version 1114999 (0.0007) [2023-12-27 13:22:27,892][20381] Updated weights for policy 1, policy_version 1113888 (0.0006) [2023-12-27 13:22:27,947][20381] Updated weights for policy 1, policy_version 1113898 (0.0007) [2023-12-27 13:22:28,006][20381] Updated weights for policy 1, policy_version 1113908 (0.0008) [2023-12-27 13:22:28,299][20347] Updated weights for policy 0, policy_version 1115009 (0.0007) [2023-12-27 13:22:28,358][20347] Updated weights for policy 0, policy_version 1115019 (0.0007) [2023-12-27 13:22:28,412][20347] Updated weights for policy 0, policy_version 1115029 (0.0007) [2023-12-27 13:22:28,464][20347] Updated weights for policy 0, policy_version 1115039 (0.0007) [2023-12-27 13:22:28,712][20381] Updated weights for policy 1, policy_version 1113918 (0.0008) [2023-12-27 13:22:28,773][20381] Updated weights for policy 1, policy_version 1113928 (0.0008) [2023-12-27 13:22:28,826][20381] Updated weights for policy 1, policy_version 1113938 (0.0009) [2023-12-27 13:22:29,108][20347] Updated weights for policy 0, policy_version 1115049 (0.0008) [2023-12-27 13:22:29,156][20347] Updated weights for policy 0, policy_version 1115059 (0.0010) [2023-12-27 13:22:29,203][20347] Updated weights for policy 0, policy_version 1115070 (0.0007) [2023-12-27 13:22:29,578][20381] Updated weights for policy 1, policy_version 1113948 (0.0009) [2023-12-27 13:22:29,628][20381] Updated weights for policy 1, policy_version 1113958 (0.0008) [2023-12-27 13:22:29,677][20381] Updated weights for policy 1, policy_version 1113968 (0.0007) [2023-12-27 13:22:29,927][20347] Updated weights for policy 0, policy_version 1115080 (0.0007) [2023-12-27 13:22:29,991][20347] Updated weights for policy 0, policy_version 1115090 (0.0007) [2023-12-27 13:22:30,054][20347] Updated weights for policy 0, policy_version 1115100 (0.0007) [2023-12-27 13:22:30,392][20381] Updated weights for policy 1, policy_version 1113978 (0.0008) [2023-12-27 13:22:30,450][20381] Updated weights for policy 1, policy_version 1113988 (0.0009) [2023-12-27 13:22:30,508][20381] Updated weights for policy 1, policy_version 1113998 (0.0008) [2023-12-27 13:22:30,587][20381] Updated weights for policy 1, policy_version 1114008 (0.0008) [2023-12-27 13:22:30,709][20347] Updated weights for policy 0, policy_version 1115110 (0.0006) [2023-12-27 13:22:30,769][20347] Updated weights for policy 0, policy_version 1115120 (0.0005) [2023-12-27 13:22:30,825][20347] Updated weights for policy 0, policy_version 1115130 (0.0005) [2023-12-27 13:22:31,224][20381] Updated weights for policy 1, policy_version 1114018 (0.0007) [2023-12-27 13:22:31,286][20381] Updated weights for policy 1, policy_version 1114028 (0.0008) [2023-12-27 13:22:31,338][20381] Updated weights for policy 1, policy_version 1114038 (0.0007) [2023-12-27 13:22:31,492][20347] Updated weights for policy 0, policy_version 1115140 (0.0006) [2023-12-27 13:22:31,553][20347] Updated weights for policy 0, policy_version 1115150 (0.0005) [2023-12-27 13:22:31,609][20347] Updated weights for policy 0, policy_version 1115160 (0.0007) [2023-12-27 13:22:32,054][20381] Updated weights for policy 1, policy_version 1114048 (0.0010) [2023-12-27 13:22:32,102][20381] Updated weights for policy 1, policy_version 1114058 (0.0010) [2023-12-27 13:22:32,147][20381] Updated weights for policy 1, policy_version 1114068 (0.0010) [2023-12-27 13:22:32,235][20347] Updated weights for policy 0, policy_version 1115170 (0.0007) [2023-12-27 13:22:32,293][20347] Updated weights for policy 0, policy_version 1115180 (0.0007) [2023-12-27 13:22:32,346][20347] Updated weights for policy 0, policy_version 1115190 (0.0008) [2023-12-27 13:22:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 570761216. Throughput: 0: 10210.5, 1: 10197.8. Samples: 570772852. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:32,363][19170] Avg episode reward: [(0, '970.840'), (1, '617.510')] [2023-12-27 13:22:32,407][20347] Updated weights for policy 0, policy_version 1115200 (0.0007) [2023-12-27 13:22:32,828][20381] Updated weights for policy 1, policy_version 1114078 (0.0009) [2023-12-27 13:22:32,885][20381] Updated weights for policy 1, policy_version 1114088 (0.0007) [2023-12-27 13:22:32,946][20381] Updated weights for policy 1, policy_version 1114098 (0.0007) [2023-12-27 13:22:33,123][20347] Updated weights for policy 0, policy_version 1115210 (0.0007) [2023-12-27 13:22:33,172][20347] Updated weights for policy 0, policy_version 1115220 (0.0008) [2023-12-27 13:22:33,221][20347] Updated weights for policy 0, policy_version 1115230 (0.0008) [2023-12-27 13:22:33,600][20381] Updated weights for policy 1, policy_version 1114108 (0.0008) [2023-12-27 13:22:33,658][20381] Updated weights for policy 1, policy_version 1114118 (0.0007) [2023-12-27 13:22:33,712][20381] Updated weights for policy 1, policy_version 1114128 (0.0007) [2023-12-27 13:22:33,932][20347] Updated weights for policy 0, policy_version 1115240 (0.0006) [2023-12-27 13:22:33,991][20347] Updated weights for policy 0, policy_version 1115250 (0.0008) [2023-12-27 13:22:34,049][20347] Updated weights for policy 0, policy_version 1115260 (0.0010) [2023-12-27 13:22:34,336][20381] Updated weights for policy 1, policy_version 1114138 (0.0008) [2023-12-27 13:22:34,403][20381] Updated weights for policy 1, policy_version 1114148 (0.0007) [2023-12-27 13:22:34,465][20381] Updated weights for policy 1, policy_version 1114158 (0.0007) [2023-12-27 13:22:34,517][20381] Updated weights for policy 1, policy_version 1114168 (0.0008) [2023-12-27 13:22:34,690][20347] Updated weights for policy 0, policy_version 1115270 (0.0007) [2023-12-27 13:22:34,740][20347] Updated weights for policy 0, policy_version 1115280 (0.0008) [2023-12-27 13:22:34,786][20347] Updated weights for policy 0, policy_version 1115290 (0.0006) [2023-12-27 13:22:34,797][20291] KL-divergence is very high: 282.5923 [2023-12-27 13:22:35,262][20381] Updated weights for policy 1, policy_version 1114178 (0.0007) [2023-12-27 13:22:35,317][20381] Updated weights for policy 1, policy_version 1114188 (0.0006) [2023-12-27 13:22:35,348][20347] Updated weights for policy 0, policy_version 1115300 (0.0007) [2023-12-27 13:22:35,374][20381] Updated weights for policy 1, policy_version 1114198 (0.0007) [2023-12-27 13:22:35,406][20347] Updated weights for policy 0, policy_version 1115310 (0.0010) [2023-12-27 13:22:35,458][20347] Updated weights for policy 0, policy_version 1115320 (0.0010) [2023-12-27 13:22:36,079][20381] Updated weights for policy 1, policy_version 1114208 (0.0006) [2023-12-27 13:22:36,089][20347] Updated weights for policy 0, policy_version 1115330 (0.0009) [2023-12-27 13:22:36,142][20381] Updated weights for policy 1, policy_version 1114218 (0.0008) [2023-12-27 13:22:36,157][20347] Updated weights for policy 0, policy_version 1115340 (0.0008) [2023-12-27 13:22:36,205][20381] Updated weights for policy 1, policy_version 1114228 (0.0006) [2023-12-27 13:22:36,224][20347] Updated weights for policy 0, policy_version 1115350 (0.0008) [2023-12-27 13:22:36,291][20347] Updated weights for policy 0, policy_version 1115360 (0.0008) [2023-12-27 13:22:36,870][20381] Updated weights for policy 1, policy_version 1114238 (0.0007) [2023-12-27 13:22:36,933][20381] Updated weights for policy 1, policy_version 1114248 (0.0007) [2023-12-27 13:22:36,968][20347] Updated weights for policy 0, policy_version 1115370 (0.0010) [2023-12-27 13:22:36,985][20381] Updated weights for policy 1, policy_version 1114258 (0.0005) [2023-12-27 13:22:37,020][20347] Updated weights for policy 0, policy_version 1115380 (0.0010) [2023-12-27 13:22:37,078][20347] Updated weights for policy 0, policy_version 1115390 (0.0010) [2023-12-27 13:22:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 570867712. Throughput: 0: 10245.6, 1: 10253.1. Samples: 570835356. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:37,364][19170] Avg episode reward: [(0, '932.050'), (1, '666.660')] [2023-12-27 13:22:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001114264_285278208.pth... [2023-12-27 13:22:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001115392_285589504.pth... [2023-12-27 13:22:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001113080_284975104.pth [2023-12-27 13:22:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001114208_285286400.pth [2023-12-27 13:22:37,667][20381] Updated weights for policy 1, policy_version 1114268 (0.0006) [2023-12-27 13:22:37,726][20381] Updated weights for policy 1, policy_version 1114278 (0.0008) [2023-12-27 13:22:37,768][20347] Updated weights for policy 0, policy_version 1115400 (0.0008) [2023-12-27 13:22:37,781][20381] Updated weights for policy 1, policy_version 1114288 (0.0006) [2023-12-27 13:22:37,812][20347] Updated weights for policy 0, policy_version 1115410 (0.0007) [2023-12-27 13:22:37,856][20347] Updated weights for policy 0, policy_version 1115420 (0.0007) [2023-12-27 13:22:38,386][20381] Updated weights for policy 1, policy_version 1114298 (0.0006) [2023-12-27 13:22:38,449][20381] Updated weights for policy 1, policy_version 1114308 (0.0006) [2023-12-27 13:22:38,512][20381] Updated weights for policy 1, policy_version 1114318 (0.0007) [2023-12-27 13:22:38,539][20347] Updated weights for policy 0, policy_version 1115430 (0.0006) [2023-12-27 13:22:38,577][20381] Updated weights for policy 1, policy_version 1114328 (0.0007) [2023-12-27 13:22:38,584][20347] Updated weights for policy 0, policy_version 1115440 (0.0007) [2023-12-27 13:22:38,636][20347] Updated weights for policy 0, policy_version 1115450 (0.0006) [2023-12-27 13:22:39,243][20381] Updated weights for policy 1, policy_version 1114338 (0.0008) [2023-12-27 13:22:39,302][20381] Updated weights for policy 1, policy_version 1114348 (0.0008) [2023-12-27 13:22:39,343][20347] Updated weights for policy 0, policy_version 1115460 (0.0005) [2023-12-27 13:22:39,362][20381] Updated weights for policy 1, policy_version 1114358 (0.0008) [2023-12-27 13:22:39,412][20347] Updated weights for policy 0, policy_version 1115470 (0.0008) [2023-12-27 13:22:39,467][20347] Updated weights for policy 0, policy_version 1115480 (0.0008) [2023-12-27 13:22:40,160][20381] Updated weights for policy 1, policy_version 1114368 (0.0010) [2023-12-27 13:22:40,217][20381] Updated weights for policy 1, policy_version 1114378 (0.0011) [2023-12-27 13:22:40,223][20347] Updated weights for policy 0, policy_version 1115490 (0.0008) [2023-12-27 13:22:40,275][20381] Updated weights for policy 1, policy_version 1114388 (0.0011) [2023-12-27 13:22:40,283][20347] Updated weights for policy 0, policy_version 1115500 (0.0006) [2023-12-27 13:22:40,342][20347] Updated weights for policy 0, policy_version 1115510 (0.0006) [2023-12-27 13:22:40,407][20347] Updated weights for policy 0, policy_version 1115520 (0.0008) [2023-12-27 13:22:40,993][20381] Updated weights for policy 1, policy_version 1114398 (0.0008) [2023-12-27 13:22:41,047][20347] Updated weights for policy 0, policy_version 1115530 (0.0007) [2023-12-27 13:22:41,056][20381] Updated weights for policy 1, policy_version 1114408 (0.0007) [2023-12-27 13:22:41,107][20347] Updated weights for policy 0, policy_version 1115540 (0.0006) [2023-12-27 13:22:41,117][20381] Updated weights for policy 1, policy_version 1114418 (0.0007) [2023-12-27 13:22:41,174][20347] Updated weights for policy 0, policy_version 1115550 (0.0008) [2023-12-27 13:22:41,862][20381] Updated weights for policy 1, policy_version 1114428 (0.0008) [2023-12-27 13:22:41,926][20347] Updated weights for policy 0, policy_version 1115560 (0.0006) [2023-12-27 13:22:41,928][20381] Updated weights for policy 1, policy_version 1114438 (0.0007) [2023-12-27 13:22:41,989][20347] Updated weights for policy 0, policy_version 1115570 (0.0008) [2023-12-27 13:22:41,990][20381] Updated weights for policy 1, policy_version 1114448 (0.0006) [2023-12-27 13:22:42,052][20347] Updated weights for policy 0, policy_version 1115580 (0.0008) [2023-12-27 13:22:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 570966016. Throughput: 0: 10270.0, 1: 10166.8. Samples: 570956808. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:42,363][19170] Avg episode reward: [(0, '941.780'), (1, '676.610')] [2023-12-27 13:22:42,626][20381] Updated weights for policy 1, policy_version 1114458 (0.0007) [2023-12-27 13:22:42,685][20381] Updated weights for policy 1, policy_version 1114468 (0.0007) [2023-12-27 13:22:42,717][20347] Updated weights for policy 0, policy_version 1115590 (0.0006) [2023-12-27 13:22:42,745][20381] Updated weights for policy 1, policy_version 1114478 (0.0008) [2023-12-27 13:22:42,771][20347] Updated weights for policy 0, policy_version 1115600 (0.0006) [2023-12-27 13:22:42,802][20381] Updated weights for policy 1, policy_version 1114488 (0.0008) [2023-12-27 13:22:42,821][20347] Updated weights for policy 0, policy_version 1115610 (0.0007) [2023-12-27 13:22:43,499][20381] Updated weights for policy 1, policy_version 1114498 (0.0008) [2023-12-27 13:22:43,523][20347] Updated weights for policy 0, policy_version 1115620 (0.0008) [2023-12-27 13:22:43,551][20381] Updated weights for policy 1, policy_version 1114508 (0.0006) [2023-12-27 13:22:43,584][20347] Updated weights for policy 0, policy_version 1115630 (0.0007) [2023-12-27 13:22:43,604][20381] Updated weights for policy 1, policy_version 1114518 (0.0008) [2023-12-27 13:22:43,641][20347] Updated weights for policy 0, policy_version 1115640 (0.0007) [2023-12-27 13:22:44,295][20381] Updated weights for policy 1, policy_version 1114528 (0.0008) [2023-12-27 13:22:44,329][20347] Updated weights for policy 0, policy_version 1115650 (0.0007) [2023-12-27 13:22:44,348][20381] Updated weights for policy 1, policy_version 1114538 (0.0007) [2023-12-27 13:22:44,390][20347] Updated weights for policy 0, policy_version 1115660 (0.0005) [2023-12-27 13:22:44,399][20381] Updated weights for policy 1, policy_version 1114548 (0.0006) [2023-12-27 13:22:44,443][20347] Updated weights for policy 0, policy_version 1115670 (0.0007) [2023-12-27 13:22:44,494][20347] Updated weights for policy 0, policy_version 1115680 (0.0007) [2023-12-27 13:22:45,048][20381] Updated weights for policy 1, policy_version 1114558 (0.0008) [2023-12-27 13:22:45,106][20381] Updated weights for policy 1, policy_version 1114568 (0.0007) [2023-12-27 13:22:45,141][20347] Updated weights for policy 0, policy_version 1115690 (0.0006) [2023-12-27 13:22:45,166][20381] Updated weights for policy 1, policy_version 1114578 (0.0007) [2023-12-27 13:22:45,200][20347] Updated weights for policy 0, policy_version 1115700 (0.0006) [2023-12-27 13:22:45,258][20347] Updated weights for policy 0, policy_version 1115710 (0.0006) [2023-12-27 13:22:45,834][20347] Updated weights for policy 0, policy_version 1115720 (0.0005) [2023-12-27 13:22:45,835][20381] Updated weights for policy 1, policy_version 1114588 (0.0007) [2023-12-27 13:22:45,885][20347] Updated weights for policy 0, policy_version 1115730 (0.0005) [2023-12-27 13:22:45,888][20381] Updated weights for policy 1, policy_version 1114598 (0.0008) [2023-12-27 13:22:45,942][20347] Updated weights for policy 0, policy_version 1115740 (0.0005) [2023-12-27 13:22:45,947][20381] Updated weights for policy 1, policy_version 1114608 (0.0008) [2023-12-27 13:22:46,630][20381] Updated weights for policy 1, policy_version 1114618 (0.0008) [2023-12-27 13:22:46,635][20347] Updated weights for policy 0, policy_version 1115750 (0.0007) [2023-12-27 13:22:46,686][20381] Updated weights for policy 1, policy_version 1114628 (0.0006) [2023-12-27 13:22:46,692][20347] Updated weights for policy 0, policy_version 1115760 (0.0005) [2023-12-27 13:22:46,742][20347] Updated weights for policy 0, policy_version 1115770 (0.0006) [2023-12-27 13:22:46,744][20381] Updated weights for policy 1, policy_version 1114638 (0.0006) [2023-12-27 13:22:46,796][20381] Updated weights for policy 1, policy_version 1114648 (0.0006) [2023-12-27 13:22:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 571064320. Throughput: 0: 10250.0, 1: 10164.7. Samples: 571078672. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:47,363][19170] Avg episode reward: [(0, '970.720'), (1, '538.940')] [2023-12-27 13:22:47,412][20347] Updated weights for policy 0, policy_version 1115780 (0.0005) [2023-12-27 13:22:47,474][20347] Updated weights for policy 0, policy_version 1115790 (0.0006) [2023-12-27 13:22:47,506][20381] Updated weights for policy 1, policy_version 1114658 (0.0006) [2023-12-27 13:22:47,528][20347] Updated weights for policy 0, policy_version 1115800 (0.0007) [2023-12-27 13:22:47,568][20381] Updated weights for policy 1, policy_version 1114668 (0.0007) [2023-12-27 13:22:47,626][20381] Updated weights for policy 1, policy_version 1114678 (0.0007) [2023-12-27 13:22:48,214][20347] Updated weights for policy 0, policy_version 1115810 (0.0008) [2023-12-27 13:22:48,216][20381] Updated weights for policy 1, policy_version 1114688 (0.0007) [2023-12-27 13:22:48,265][20347] Updated weights for policy 0, policy_version 1115820 (0.0007) [2023-12-27 13:22:48,274][20381] Updated weights for policy 1, policy_version 1114698 (0.0005) [2023-12-27 13:22:48,319][20347] Updated weights for policy 0, policy_version 1115830 (0.0006) [2023-12-27 13:22:48,334][20381] Updated weights for policy 1, policy_version 1114708 (0.0008) [2023-12-27 13:22:48,384][20347] Updated weights for policy 0, policy_version 1115840 (0.0008) [2023-12-27 13:22:48,986][20381] Updated weights for policy 1, policy_version 1114718 (0.0008) [2023-12-27 13:22:49,042][20381] Updated weights for policy 1, policy_version 1114728 (0.0008) [2023-12-27 13:22:49,091][20381] Updated weights for policy 1, policy_version 1114738 (0.0006) [2023-12-27 13:22:49,106][20347] Updated weights for policy 0, policy_version 1115850 (0.0008) [2023-12-27 13:22:49,153][20347] Updated weights for policy 0, policy_version 1115860 (0.0009) [2023-12-27 13:22:49,199][20347] Updated weights for policy 0, policy_version 1115870 (0.0010) [2023-12-27 13:22:49,793][20381] Updated weights for policy 1, policy_version 1114748 (0.0006) [2023-12-27 13:22:49,858][20381] Updated weights for policy 1, policy_version 1114758 (0.0007) [2023-12-27 13:22:49,924][20381] Updated weights for policy 1, policy_version 1114768 (0.0007) [2023-12-27 13:22:49,924][20347] Updated weights for policy 0, policy_version 1115880 (0.0007) [2023-12-27 13:22:49,985][20347] Updated weights for policy 0, policy_version 1115890 (0.0007) [2023-12-27 13:22:50,049][20347] Updated weights for policy 0, policy_version 1115900 (0.0008) [2023-12-27 13:22:50,656][20381] Updated weights for policy 1, policy_version 1114778 (0.0008) [2023-12-27 13:22:50,706][20381] Updated weights for policy 1, policy_version 1114788 (0.0008) [2023-12-27 13:22:50,751][20347] Updated weights for policy 0, policy_version 1115910 (0.0007) [2023-12-27 13:22:50,768][20381] Updated weights for policy 1, policy_version 1114798 (0.0007) [2023-12-27 13:22:50,776][20291] KL-divergence is very high: 112.9184 [2023-12-27 13:22:50,813][20347] Updated weights for policy 0, policy_version 1115920 (0.0007) [2023-12-27 13:22:50,830][20381] Updated weights for policy 1, policy_version 1114808 (0.0006) [2023-12-27 13:22:50,872][20347] Updated weights for policy 0, policy_version 1115930 (0.0007) [2023-12-27 13:22:51,540][20381] Updated weights for policy 1, policy_version 1114818 (0.0007) [2023-12-27 13:22:51,597][20347] Updated weights for policy 0, policy_version 1115940 (0.0007) [2023-12-27 13:22:51,599][20381] Updated weights for policy 1, policy_version 1114828 (0.0007) [2023-12-27 13:22:51,659][20347] Updated weights for policy 0, policy_version 1115950 (0.0009) [2023-12-27 13:22:51,664][20381] Updated weights for policy 1, policy_version 1114838 (0.0008) [2023-12-27 13:22:51,727][20347] Updated weights for policy 0, policy_version 1115960 (0.0008) [2023-12-27 13:22:52,319][20381] Updated weights for policy 1, policy_version 1114848 (0.0007) [2023-12-27 13:22:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 571162624. Throughput: 0: 10224.0, 1: 10200.0. Samples: 571139528. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:52,363][19170] Avg episode reward: [(0, '921.970'), (1, '652.600')] [2023-12-27 13:22:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001115968_285736960.pth... [2023-12-27 13:22:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001114784_285433856.pth [2023-12-27 13:22:52,386][20381] Updated weights for policy 1, policy_version 1114858 (0.0009) [2023-12-27 13:22:52,447][20381] Updated weights for policy 1, policy_version 1114868 (0.0008) [2023-12-27 13:22:52,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001114872_285433856.pth... [2023-12-27 13:22:52,473][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001113688_285130752.pth [2023-12-27 13:22:52,497][20347] Updated weights for policy 0, policy_version 1115970 (0.0007) [2023-12-27 13:22:52,557][20347] Updated weights for policy 0, policy_version 1115980 (0.0006) [2023-12-27 13:22:52,615][20347] Updated weights for policy 0, policy_version 1115990 (0.0007) [2023-12-27 13:22:52,673][20347] Updated weights for policy 0, policy_version 1116000 (0.0008) [2023-12-27 13:22:53,158][20381] Updated weights for policy 1, policy_version 1114878 (0.0007) [2023-12-27 13:22:53,215][20381] Updated weights for policy 1, policy_version 1114888 (0.0009) [2023-12-27 13:22:53,267][20381] Updated weights for policy 1, policy_version 1114898 (0.0008) [2023-12-27 13:22:53,326][20347] Updated weights for policy 0, policy_version 1116010 (0.0005) [2023-12-27 13:22:53,390][20347] Updated weights for policy 0, policy_version 1116020 (0.0006) [2023-12-27 13:22:53,441][20347] Updated weights for policy 0, policy_version 1116030 (0.0009) [2023-12-27 13:22:53,976][20381] Updated weights for policy 1, policy_version 1114908 (0.0008) [2023-12-27 13:22:54,036][20381] Updated weights for policy 1, policy_version 1114918 (0.0007) [2023-12-27 13:22:54,075][20347] Updated weights for policy 0, policy_version 1116040 (0.0006) [2023-12-27 13:22:54,101][20381] Updated weights for policy 1, policy_version 1114928 (0.0008) [2023-12-27 13:22:54,133][20347] Updated weights for policy 0, policy_version 1116050 (0.0010) [2023-12-27 13:22:54,194][20347] Updated weights for policy 0, policy_version 1116060 (0.0008) [2023-12-27 13:22:54,732][20381] Updated weights for policy 1, policy_version 1114938 (0.0007) [2023-12-27 13:22:54,750][20347] Updated weights for policy 0, policy_version 1116070 (0.0006) [2023-12-27 13:22:54,790][20381] Updated weights for policy 1, policy_version 1114948 (0.0006) [2023-12-27 13:22:54,807][20347] Updated weights for policy 0, policy_version 1116080 (0.0009) [2023-12-27 13:22:54,842][20381] Updated weights for policy 1, policy_version 1114958 (0.0006) [2023-12-27 13:22:54,857][20347] Updated weights for policy 0, policy_version 1116090 (0.0010) [2023-12-27 13:22:54,896][20381] Updated weights for policy 1, policy_version 1114968 (0.0006) [2023-12-27 13:22:55,508][20347] Updated weights for policy 0, policy_version 1116100 (0.0010) [2023-12-27 13:22:55,511][20381] Updated weights for policy 1, policy_version 1114978 (0.0005) [2023-12-27 13:22:55,561][20347] Updated weights for policy 0, policy_version 1116110 (0.0008) [2023-12-27 13:22:55,567][20381] Updated weights for policy 1, policy_version 1114988 (0.0005) [2023-12-27 13:22:55,617][20347] Updated weights for policy 0, policy_version 1116120 (0.0005) [2023-12-27 13:22:55,621][20381] Updated weights for policy 1, policy_version 1114998 (0.0005) [2023-12-27 13:22:56,236][20347] Updated weights for policy 0, policy_version 1116130 (0.0006) [2023-12-27 13:22:56,243][20381] Updated weights for policy 1, policy_version 1115008 (0.0007) [2023-12-27 13:22:56,287][20347] Updated weights for policy 0, policy_version 1116140 (0.0010) [2023-12-27 13:22:56,293][20381] Updated weights for policy 1, policy_version 1115018 (0.0006) [2023-12-27 13:22:56,335][20347] Updated weights for policy 0, policy_version 1116150 (0.0010) [2023-12-27 13:22:56,340][20381] Updated weights for policy 1, policy_version 1115028 (0.0005) [2023-12-27 13:22:56,371][20291] KL-divergence is very high: 115.2298 [2023-12-27 13:22:56,387][20347] Updated weights for policy 0, policy_version 1116160 (0.0010) [2023-12-27 13:22:56,963][20347] Updated weights for policy 0, policy_version 1116170 (0.0005) [2023-12-27 13:22:57,010][20381] Updated weights for policy 1, policy_version 1115038 (0.0005) [2023-12-27 13:22:57,021][20347] Updated weights for policy 0, policy_version 1116180 (0.0006) [2023-12-27 13:22:57,068][20347] Updated weights for policy 0, policy_version 1116190 (0.0006) [2023-12-27 13:22:57,073][20381] Updated weights for policy 1, policy_version 1115048 (0.0005) [2023-12-27 13:22:57,139][20381] Updated weights for policy 1, policy_version 1115058 (0.0005) [2023-12-27 13:22:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 571277312. Throughput: 0: 10212.8, 1: 10178.2. Samples: 571263416. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:22:57,364][19170] Avg episode reward: [(0, '931.790'), (1, '654.710')] [2023-12-27 13:22:57,699][20347] Updated weights for policy 0, policy_version 1116200 (0.0009) [2023-12-27 13:22:57,723][20381] Updated weights for policy 1, policy_version 1115068 (0.0005) [2023-12-27 13:22:57,754][20347] Updated weights for policy 0, policy_version 1116210 (0.0010) [2023-12-27 13:22:57,776][20381] Updated weights for policy 1, policy_version 1115078 (0.0005) [2023-12-27 13:22:57,808][20347] Updated weights for policy 0, policy_version 1116220 (0.0009) [2023-12-27 13:22:57,835][20381] Updated weights for policy 1, policy_version 1115088 (0.0005) [2023-12-27 13:22:58,442][20381] Updated weights for policy 1, policy_version 1115098 (0.0008) [2023-12-27 13:22:58,490][20347] Updated weights for policy 0, policy_version 1116230 (0.0007) [2023-12-27 13:22:58,501][20381] Updated weights for policy 1, policy_version 1115108 (0.0008) [2023-12-27 13:22:58,550][20347] Updated weights for policy 0, policy_version 1116240 (0.0006) [2023-12-27 13:22:58,560][20381] Updated weights for policy 1, policy_version 1115118 (0.0008) [2023-12-27 13:22:58,611][20347] Updated weights for policy 0, policy_version 1116250 (0.0006) [2023-12-27 13:22:58,616][20381] Updated weights for policy 1, policy_version 1115128 (0.0007) [2023-12-27 13:22:59,374][20347] Updated weights for policy 0, policy_version 1116260 (0.0008) [2023-12-27 13:22:59,428][20381] Updated weights for policy 1, policy_version 1115138 (0.0008) [2023-12-27 13:22:59,429][20347] Updated weights for policy 0, policy_version 1116270 (0.0008) [2023-12-27 13:22:59,477][20381] Updated weights for policy 1, policy_version 1115148 (0.0005) [2023-12-27 13:22:59,489][20347] Updated weights for policy 0, policy_version 1116280 (0.0006) [2023-12-27 13:22:59,525][20381] Updated weights for policy 1, policy_version 1115158 (0.0008) [2023-12-27 13:23:00,152][20381] Updated weights for policy 1, policy_version 1115168 (0.0008) [2023-12-27 13:23:00,204][20381] Updated weights for policy 1, policy_version 1115178 (0.0007) [2023-12-27 13:23:00,219][20347] Updated weights for policy 0, policy_version 1116290 (0.0006) [2023-12-27 13:23:00,263][20381] Updated weights for policy 1, policy_version 1115188 (0.0006) [2023-12-27 13:23:00,272][20347] Updated weights for policy 0, policy_version 1116300 (0.0008) [2023-12-27 13:23:00,324][20347] Updated weights for policy 0, policy_version 1116310 (0.0008) [2023-12-27 13:23:00,380][20347] Updated weights for policy 0, policy_version 1116320 (0.0007) [2023-12-27 13:23:00,897][20381] Updated weights for policy 1, policy_version 1115198 (0.0006) [2023-12-27 13:23:00,957][20381] Updated weights for policy 1, policy_version 1115208 (0.0005) [2023-12-27 13:23:01,012][20381] Updated weights for policy 1, policy_version 1115218 (0.0005) [2023-12-27 13:23:01,112][20347] Updated weights for policy 0, policy_version 1116330 (0.0008) [2023-12-27 13:23:01,174][20347] Updated weights for policy 0, policy_version 1116340 (0.0008) [2023-12-27 13:23:01,223][20347] Updated weights for policy 0, policy_version 1116350 (0.0008) [2023-12-27 13:23:01,694][20381] Updated weights for policy 1, policy_version 1115228 (0.0007) [2023-12-27 13:23:01,756][20381] Updated weights for policy 1, policy_version 1115238 (0.0008) [2023-12-27 13:23:01,817][20381] Updated weights for policy 1, policy_version 1115248 (0.0007) [2023-12-27 13:23:01,954][20347] Updated weights for policy 0, policy_version 1116360 (0.0007) [2023-12-27 13:23:02,012][20347] Updated weights for policy 0, policy_version 1116370 (0.0007) [2023-12-27 13:23:02,073][20347] Updated weights for policy 0, policy_version 1116380 (0.0008) [2023-12-27 13:23:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 571375616. Throughput: 0: 10210.3, 1: 10199.6. Samples: 571386104. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:02,363][19170] Avg episode reward: [(0, '921.870'), (1, '647.010')] [2023-12-27 13:23:02,477][20381] Updated weights for policy 1, policy_version 1115258 (0.0007) [2023-12-27 13:23:02,537][20381] Updated weights for policy 1, policy_version 1115268 (0.0005) [2023-12-27 13:23:02,596][20381] Updated weights for policy 1, policy_version 1115278 (0.0007) [2023-12-27 13:23:02,659][20381] Updated weights for policy 1, policy_version 1115288 (0.0007) [2023-12-27 13:23:02,698][20347] Updated weights for policy 0, policy_version 1116390 (0.0007) [2023-12-27 13:23:02,747][20347] Updated weights for policy 0, policy_version 1116400 (0.0009) [2023-12-27 13:23:02,804][20347] Updated weights for policy 0, policy_version 1116410 (0.0008) [2023-12-27 13:23:03,265][20381] Updated weights for policy 1, policy_version 1115298 (0.0008) [2023-12-27 13:23:03,318][20381] Updated weights for policy 1, policy_version 1115308 (0.0008) [2023-12-27 13:23:03,372][20381] Updated weights for policy 1, policy_version 1115318 (0.0006) [2023-12-27 13:23:03,380][20347] Updated weights for policy 0, policy_version 1116420 (0.0009) [2023-12-27 13:23:03,427][20347] Updated weights for policy 0, policy_version 1116430 (0.0010) [2023-12-27 13:23:03,475][20347] Updated weights for policy 0, policy_version 1116440 (0.0007) [2023-12-27 13:23:04,061][20347] Updated weights for policy 0, policy_version 1116450 (0.0006) [2023-12-27 13:23:04,114][20381] Updated weights for policy 1, policy_version 1115328 (0.0007) [2023-12-27 13:23:04,127][20347] Updated weights for policy 0, policy_version 1116460 (0.0007) [2023-12-27 13:23:04,176][20381] Updated weights for policy 1, policy_version 1115338 (0.0006) [2023-12-27 13:23:04,189][20347] Updated weights for policy 0, policy_version 1116470 (0.0007) [2023-12-27 13:23:04,245][20381] Updated weights for policy 1, policy_version 1115348 (0.0006) [2023-12-27 13:23:04,253][20347] Updated weights for policy 0, policy_version 1116480 (0.0008) [2023-12-27 13:23:04,829][20381] Updated weights for policy 1, policy_version 1115358 (0.0007) [2023-12-27 13:23:04,885][20381] Updated weights for policy 1, policy_version 1115368 (0.0008) [2023-12-27 13:23:04,942][20381] Updated weights for policy 1, policy_version 1115378 (0.0008) [2023-12-27 13:23:04,959][20347] Updated weights for policy 0, policy_version 1116490 (0.0006) [2023-12-27 13:23:05,014][20347] Updated weights for policy 0, policy_version 1116500 (0.0007) [2023-12-27 13:23:05,073][20347] Updated weights for policy 0, policy_version 1116510 (0.0005) [2023-12-27 13:23:05,615][20381] Updated weights for policy 1, policy_version 1115388 (0.0006) [2023-12-27 13:23:05,664][20381] Updated weights for policy 1, policy_version 1115398 (0.0007) [2023-12-27 13:23:05,689][20347] Updated weights for policy 0, policy_version 1116520 (0.0006) [2023-12-27 13:23:05,713][20381] Updated weights for policy 1, policy_version 1115408 (0.0007) [2023-12-27 13:23:05,742][20347] Updated weights for policy 0, policy_version 1116530 (0.0006) [2023-12-27 13:23:05,795][20347] Updated weights for policy 0, policy_version 1116540 (0.0006) [2023-12-27 13:23:06,407][20381] Updated weights for policy 1, policy_version 1115418 (0.0008) [2023-12-27 13:23:06,447][20347] Updated weights for policy 0, policy_version 1116550 (0.0008) [2023-12-27 13:23:06,471][20381] Updated weights for policy 1, policy_version 1115428 (0.0006) [2023-12-27 13:23:06,506][20347] Updated weights for policy 0, policy_version 1116560 (0.0011) [2023-12-27 13:23:06,537][20381] Updated weights for policy 1, policy_version 1115438 (0.0006) [2023-12-27 13:23:06,569][20347] Updated weights for policy 0, policy_version 1116570 (0.0010) [2023-12-27 13:23:06,600][20381] Updated weights for policy 1, policy_version 1115448 (0.0006) [2023-12-27 13:23:07,241][20381] Updated weights for policy 1, policy_version 1115458 (0.0005) [2023-12-27 13:23:07,253][20347] Updated weights for policy 0, policy_version 1116580 (0.0011) [2023-12-27 13:23:07,296][20381] Updated weights for policy 1, policy_version 1115468 (0.0006) [2023-12-27 13:23:07,309][20347] Updated weights for policy 0, policy_version 1116590 (0.0011) [2023-12-27 13:23:07,350][20291] KL-divergence is very high: 100.9012 [2023-12-27 13:23:07,352][20381] Updated weights for policy 1, policy_version 1115478 (0.0005) [2023-12-27 13:23:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 571482112. Throughput: 0: 10259.9, 1: 10214.8. Samples: 571450884. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:07,363][19170] Avg episode reward: [(0, '902.110'), (1, '656.990')] [2023-12-27 13:23:07,368][20347] Updated weights for policy 0, policy_version 1116600 (0.0010) [2023-12-27 13:23:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001115480_285589504.pth... [2023-12-27 13:23:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001114264_285278208.pth [2023-12-27 13:23:07,405][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001116608_285900800.pth... [2023-12-27 13:23:07,408][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001115392_285589504.pth [2023-12-27 13:23:08,029][20347] Updated weights for policy 0, policy_version 1116610 (0.0010) [2023-12-27 13:23:08,031][20381] Updated weights for policy 1, policy_version 1115488 (0.0009) [2023-12-27 13:23:08,081][20347] Updated weights for policy 0, policy_version 1116620 (0.0010) [2023-12-27 13:23:08,086][20381] Updated weights for policy 1, policy_version 1115498 (0.0008) [2023-12-27 13:23:08,135][20347] Updated weights for policy 0, policy_version 1116630 (0.0010) [2023-12-27 13:23:08,137][20381] Updated weights for policy 1, policy_version 1115508 (0.0005) [2023-12-27 13:23:08,190][20347] Updated weights for policy 0, policy_version 1116640 (0.0010) [2023-12-27 13:23:08,814][20347] Updated weights for policy 0, policy_version 1116650 (0.0008) [2023-12-27 13:23:08,835][20381] Updated weights for policy 1, policy_version 1115518 (0.0010) [2023-12-27 13:23:08,864][20347] Updated weights for policy 0, policy_version 1116660 (0.0005) [2023-12-27 13:23:08,893][20381] Updated weights for policy 1, policy_version 1115528 (0.0011) [2023-12-27 13:23:08,924][20347] Updated weights for policy 0, policy_version 1116670 (0.0006) [2023-12-27 13:23:08,951][20381] Updated weights for policy 1, policy_version 1115538 (0.0011) [2023-12-27 13:23:09,626][20347] Updated weights for policy 0, policy_version 1116680 (0.0005) [2023-12-27 13:23:09,688][20347] Updated weights for policy 0, policy_version 1116690 (0.0005) [2023-12-27 13:23:09,728][20381] Updated weights for policy 1, policy_version 1115548 (0.0009) [2023-12-27 13:23:09,745][20347] Updated weights for policy 0, policy_version 1116700 (0.0008) [2023-12-27 13:23:09,794][20381] Updated weights for policy 1, policy_version 1115558 (0.0006) [2023-12-27 13:23:09,859][20381] Updated weights for policy 1, policy_version 1115568 (0.0009) [2023-12-27 13:23:10,369][20347] Updated weights for policy 0, policy_version 1116710 (0.0006) [2023-12-27 13:23:10,426][20347] Updated weights for policy 0, policy_version 1116720 (0.0005) [2023-12-27 13:23:10,480][20347] Updated weights for policy 0, policy_version 1116730 (0.0005) [2023-12-27 13:23:10,617][20381] Updated weights for policy 1, policy_version 1115578 (0.0007) [2023-12-27 13:23:10,684][20381] Updated weights for policy 1, policy_version 1115588 (0.0007) [2023-12-27 13:23:10,750][20381] Updated weights for policy 1, policy_version 1115598 (0.0008) [2023-12-27 13:23:10,815][20381] Updated weights for policy 1, policy_version 1115608 (0.0008) [2023-12-27 13:23:11,067][20347] Updated weights for policy 0, policy_version 1116740 (0.0006) [2023-12-27 13:23:11,127][20347] Updated weights for policy 0, policy_version 1116750 (0.0007) [2023-12-27 13:23:11,192][20347] Updated weights for policy 0, policy_version 1116760 (0.0008) [2023-12-27 13:23:11,470][20381] Updated weights for policy 1, policy_version 1115618 (0.0011) [2023-12-27 13:23:11,529][20381] Updated weights for policy 1, policy_version 1115628 (0.0010) [2023-12-27 13:23:11,592][20381] Updated weights for policy 1, policy_version 1115638 (0.0011) [2023-12-27 13:23:11,898][20347] Updated weights for policy 0, policy_version 1116770 (0.0008) [2023-12-27 13:23:11,958][20347] Updated weights for policy 0, policy_version 1116780 (0.0006) [2023-12-27 13:23:12,020][20347] Updated weights for policy 0, policy_version 1116790 (0.0006) [2023-12-27 13:23:12,077][20347] Updated weights for policy 0, policy_version 1116800 (0.0008) [2023-12-27 13:23:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 571580416. Throughput: 0: 10340.8, 1: 10191.2. Samples: 571574488. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:12,363][19170] Avg episode reward: [(0, '911.990'), (1, '617.480')] [2023-12-27 13:23:12,390][20381] Updated weights for policy 1, policy_version 1115648 (0.0009) [2023-12-27 13:23:12,448][20381] Updated weights for policy 1, policy_version 1115658 (0.0008) [2023-12-27 13:23:12,499][20381] Updated weights for policy 1, policy_version 1115668 (0.0008) [2023-12-27 13:23:12,758][20347] Updated weights for policy 0, policy_version 1116810 (0.0007) [2023-12-27 13:23:12,816][20347] Updated weights for policy 0, policy_version 1116820 (0.0006) [2023-12-27 13:23:12,870][20347] Updated weights for policy 0, policy_version 1116830 (0.0007) [2023-12-27 13:23:13,201][20381] Updated weights for policy 1, policy_version 1115678 (0.0007) [2023-12-27 13:23:13,244][20381] Updated weights for policy 1, policy_version 1115688 (0.0006) [2023-12-27 13:23:13,293][20381] Updated weights for policy 1, policy_version 1115698 (0.0005) [2023-12-27 13:23:13,430][20347] Updated weights for policy 0, policy_version 1116840 (0.0006) [2023-12-27 13:23:13,489][20347] Updated weights for policy 0, policy_version 1116850 (0.0005) [2023-12-27 13:23:13,553][20347] Updated weights for policy 0, policy_version 1116860 (0.0005) [2023-12-27 13:23:13,860][20381] Updated weights for policy 1, policy_version 1115708 (0.0006) [2023-12-27 13:23:13,908][20381] Updated weights for policy 1, policy_version 1115718 (0.0005) [2023-12-27 13:23:13,964][20381] Updated weights for policy 1, policy_version 1115728 (0.0005) [2023-12-27 13:23:14,040][20347] Updated weights for policy 0, policy_version 1116870 (0.0005) [2023-12-27 13:23:14,091][20347] Updated weights for policy 0, policy_version 1116880 (0.0006) [2023-12-27 13:23:14,140][20347] Updated weights for policy 0, policy_version 1116890 (0.0008) [2023-12-27 13:23:14,540][20381] Updated weights for policy 1, policy_version 1115738 (0.0005) [2023-12-27 13:23:14,595][20381] Updated weights for policy 1, policy_version 1115748 (0.0006) [2023-12-27 13:23:14,650][20381] Updated weights for policy 1, policy_version 1115758 (0.0006) [2023-12-27 13:23:14,707][20381] Updated weights for policy 1, policy_version 1115768 (0.0009) [2023-12-27 13:23:14,842][20347] Updated weights for policy 0, policy_version 1116900 (0.0008) [2023-12-27 13:23:14,910][20347] Updated weights for policy 0, policy_version 1116910 (0.0008) [2023-12-27 13:23:14,971][20347] Updated weights for policy 0, policy_version 1116920 (0.0007) [2023-12-27 13:23:15,424][20381] Updated weights for policy 1, policy_version 1115778 (0.0008) [2023-12-27 13:23:15,489][20381] Updated weights for policy 1, policy_version 1115788 (0.0008) [2023-12-27 13:23:15,553][20381] Updated weights for policy 1, policy_version 1115798 (0.0008) [2023-12-27 13:23:15,564][20347] Updated weights for policy 0, policy_version 1116930 (0.0007) [2023-12-27 13:23:15,622][20347] Updated weights for policy 0, policy_version 1116940 (0.0007) [2023-12-27 13:23:15,676][20347] Updated weights for policy 0, policy_version 1116950 (0.0007) [2023-12-27 13:23:15,731][20347] Updated weights for policy 0, policy_version 1116960 (0.0007) [2023-12-27 13:23:16,132][20381] Updated weights for policy 1, policy_version 1115808 (0.0007) [2023-12-27 13:23:16,185][20381] Updated weights for policy 1, policy_version 1115818 (0.0011) [2023-12-27 13:23:16,247][20381] Updated weights for policy 1, policy_version 1115828 (0.0008) [2023-12-27 13:23:16,365][20347] Updated weights for policy 0, policy_version 1116970 (0.0007) [2023-12-27 13:23:16,426][20347] Updated weights for policy 0, policy_version 1116980 (0.0007) [2023-12-27 13:23:16,495][20347] Updated weights for policy 0, policy_version 1116990 (0.0010) [2023-12-27 13:23:16,896][20381] Updated weights for policy 1, policy_version 1115838 (0.0006) [2023-12-27 13:23:16,947][20381] Updated weights for policy 1, policy_version 1115848 (0.0006) [2023-12-27 13:23:16,995][20381] Updated weights for policy 1, policy_version 1115858 (0.0007) [2023-12-27 13:23:17,130][20347] Updated weights for policy 0, policy_version 1117000 (0.0010) [2023-12-27 13:23:17,181][20347] Updated weights for policy 0, policy_version 1117010 (0.0008) [2023-12-27 13:23:17,242][20347] Updated weights for policy 0, policy_version 1117020 (0.0005) [2023-12-27 13:23:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 571695104. Throughput: 0: 10418.6, 1: 10239.6. Samples: 571702468. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:17,363][19170] Avg episode reward: [(0, '951.180'), (1, '656.910')] [2023-12-27 13:23:17,715][20381] Updated weights for policy 1, policy_version 1115868 (0.0007) [2023-12-27 13:23:17,771][20381] Updated weights for policy 1, policy_version 1115878 (0.0008) [2023-12-27 13:23:17,825][20381] Updated weights for policy 1, policy_version 1115888 (0.0008) [2023-12-27 13:23:17,853][20347] Updated weights for policy 0, policy_version 1117030 (0.0006) [2023-12-27 13:23:17,910][20347] Updated weights for policy 0, policy_version 1117040 (0.0007) [2023-12-27 13:23:17,962][20347] Updated weights for policy 0, policy_version 1117050 (0.0008) [2023-12-27 13:23:18,562][20381] Updated weights for policy 1, policy_version 1115898 (0.0006) [2023-12-27 13:23:18,605][20347] Updated weights for policy 0, policy_version 1117060 (0.0008) [2023-12-27 13:23:18,618][20381] Updated weights for policy 1, policy_version 1115908 (0.0006) [2023-12-27 13:23:18,660][20347] Updated weights for policy 0, policy_version 1117070 (0.0009) [2023-12-27 13:23:18,676][20381] Updated weights for policy 1, policy_version 1115918 (0.0005) [2023-12-27 13:23:18,718][20347] Updated weights for policy 0, policy_version 1117080 (0.0008) [2023-12-27 13:23:18,740][20381] Updated weights for policy 1, policy_version 1115928 (0.0006) [2023-12-27 13:23:19,291][20381] Updated weights for policy 1, policy_version 1115938 (0.0007) [2023-12-27 13:23:19,350][20381] Updated weights for policy 1, policy_version 1115948 (0.0007) [2023-12-27 13:23:19,418][20381] Updated weights for policy 1, policy_version 1115958 (0.0008) [2023-12-27 13:23:19,476][20347] Updated weights for policy 0, policy_version 1117090 (0.0008) [2023-12-27 13:23:19,549][20347] Updated weights for policy 0, policy_version 1117100 (0.0008) [2023-12-27 13:23:19,609][20347] Updated weights for policy 0, policy_version 1117110 (0.0008) [2023-12-27 13:23:19,669][20347] Updated weights for policy 0, policy_version 1117120 (0.0008) [2023-12-27 13:23:20,146][20381] Updated weights for policy 1, policy_version 1115968 (0.0008) [2023-12-27 13:23:20,203][20381] Updated weights for policy 1, policy_version 1115978 (0.0008) [2023-12-27 13:23:20,261][20381] Updated weights for policy 1, policy_version 1115988 (0.0008) [2023-12-27 13:23:20,396][20347] Updated weights for policy 0, policy_version 1117130 (0.0007) [2023-12-27 13:23:20,457][20347] Updated weights for policy 0, policy_version 1117140 (0.0008) [2023-12-27 13:23:20,516][20347] Updated weights for policy 0, policy_version 1117150 (0.0007) [2023-12-27 13:23:20,951][20381] Updated weights for policy 1, policy_version 1115998 (0.0008) [2023-12-27 13:23:21,008][20381] Updated weights for policy 1, policy_version 1116008 (0.0006) [2023-12-27 13:23:21,096][20381] Updated weights for policy 1, policy_version 1116018 (0.0007) [2023-12-27 13:23:21,234][20347] Updated weights for policy 0, policy_version 1117160 (0.0008) [2023-12-27 13:23:21,297][20347] Updated weights for policy 0, policy_version 1117170 (0.0008) [2023-12-27 13:23:21,358][20347] Updated weights for policy 0, policy_version 1117180 (0.0008) [2023-12-27 13:23:21,809][20381] Updated weights for policy 1, policy_version 1116028 (0.0007) [2023-12-27 13:23:21,872][20381] Updated weights for policy 1, policy_version 1116038 (0.0007) [2023-12-27 13:23:21,935][20381] Updated weights for policy 1, policy_version 1116048 (0.0008) [2023-12-27 13:23:22,123][20347] Updated weights for policy 0, policy_version 1117190 (0.0010) [2023-12-27 13:23:22,182][20347] Updated weights for policy 0, policy_version 1117200 (0.0010) [2023-12-27 13:23:22,239][20347] Updated weights for policy 0, policy_version 1117210 (0.0010) [2023-12-27 13:23:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 571793408. Throughput: 0: 10396.4, 1: 10224.0. Samples: 571763268. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:22,363][19170] Avg episode reward: [(0, '951.290'), (1, '578.420')] [2023-12-27 13:23:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001116056_285736960.pth... [2023-12-27 13:23:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001117216_286056448.pth... [2023-12-27 13:23:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001115968_285736960.pth [2023-12-27 13:23:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001114872_285433856.pth [2023-12-27 13:23:22,623][20381] Updated weights for policy 1, policy_version 1116058 (0.0008) [2023-12-27 13:23:22,690][20381] Updated weights for policy 1, policy_version 1116068 (0.0008) [2023-12-27 13:23:22,746][20381] Updated weights for policy 1, policy_version 1116078 (0.0008) [2023-12-27 13:23:22,810][20381] Updated weights for policy 1, policy_version 1116088 (0.0008) [2023-12-27 13:23:22,980][20347] Updated weights for policy 0, policy_version 1117220 (0.0010) [2023-12-27 13:23:23,036][20347] Updated weights for policy 0, policy_version 1117230 (0.0010) [2023-12-27 13:23:23,081][20347] Updated weights for policy 0, policy_version 1117240 (0.0010) [2023-12-27 13:23:23,525][20381] Updated weights for policy 1, policy_version 1116098 (0.0006) [2023-12-27 13:23:23,576][20381] Updated weights for policy 1, policy_version 1116108 (0.0007) [2023-12-27 13:23:23,623][20381] Updated weights for policy 1, policy_version 1116118 (0.0006) [2023-12-27 13:23:23,671][20347] Updated weights for policy 0, policy_version 1117250 (0.0009) [2023-12-27 13:23:23,726][20347] Updated weights for policy 0, policy_version 1117260 (0.0007) [2023-12-27 13:23:23,773][20347] Updated weights for policy 0, policy_version 1117270 (0.0008) [2023-12-27 13:23:23,820][20347] Updated weights for policy 0, policy_version 1117280 (0.0007) [2023-12-27 13:23:24,304][20381] Updated weights for policy 1, policy_version 1116128 (0.0008) [2023-12-27 13:23:24,356][20381] Updated weights for policy 1, policy_version 1116138 (0.0008) [2023-12-27 13:23:24,380][20347] Updated weights for policy 0, policy_version 1117290 (0.0005) [2023-12-27 13:23:24,407][20381] Updated weights for policy 1, policy_version 1116148 (0.0005) [2023-12-27 13:23:24,428][20347] Updated weights for policy 0, policy_version 1117300 (0.0009) [2023-12-27 13:23:24,473][20347] Updated weights for policy 0, policy_version 1117310 (0.0010) [2023-12-27 13:23:25,073][20347] Updated weights for policy 0, policy_version 1117320 (0.0006) [2023-12-27 13:23:25,092][20381] Updated weights for policy 1, policy_version 1116158 (0.0008) [2023-12-27 13:23:25,121][20347] Updated weights for policy 0, policy_version 1117330 (0.0005) [2023-12-27 13:23:25,137][20381] Updated weights for policy 1, policy_version 1116168 (0.0010) [2023-12-27 13:23:25,165][20347] Updated weights for policy 0, policy_version 1117340 (0.0005) [2023-12-27 13:23:25,192][20381] Updated weights for policy 1, policy_version 1116178 (0.0008) [2023-12-27 13:23:25,736][20347] Updated weights for policy 0, policy_version 1117350 (0.0009) [2023-12-27 13:23:25,790][20347] Updated weights for policy 0, policy_version 1117360 (0.0010) [2023-12-27 13:23:25,836][20347] Updated weights for policy 0, policy_version 1117370 (0.0010) [2023-12-27 13:23:25,839][20381] Updated weights for policy 1, policy_version 1116188 (0.0009) [2023-12-27 13:23:25,894][20381] Updated weights for policy 1, policy_version 1116198 (0.0011) [2023-12-27 13:23:25,945][20381] Updated weights for policy 1, policy_version 1116208 (0.0010) [2023-12-27 13:23:26,534][20347] Updated weights for policy 0, policy_version 1117380 (0.0010) [2023-12-27 13:23:26,579][20347] Updated weights for policy 0, policy_version 1117390 (0.0010) [2023-12-27 13:23:26,623][20347] Updated weights for policy 0, policy_version 1117400 (0.0010) [2023-12-27 13:23:26,640][20381] Updated weights for policy 1, policy_version 1116218 (0.0011) [2023-12-27 13:23:26,694][20381] Updated weights for policy 1, policy_version 1116228 (0.0010) [2023-12-27 13:23:26,751][20381] Updated weights for policy 1, policy_version 1116238 (0.0010) [2023-12-27 13:23:26,811][20381] Updated weights for policy 1, policy_version 1116248 (0.0011) [2023-12-27 13:23:27,300][20347] Updated weights for policy 0, policy_version 1117410 (0.0010) [2023-12-27 13:23:27,351][20347] Updated weights for policy 0, policy_version 1117420 (0.0010) [2023-12-27 13:23:27,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 571891712. Throughput: 0: 10432.5, 1: 10223.3. Samples: 571886320. Policy #0 lag: (min: 26.0, avg: 41.8, max: 58.0) [2023-12-27 13:23:27,363][19170] Avg episode reward: [(0, '961.110'), (1, '695.890')] [2023-12-27 13:23:27,402][20347] Updated weights for policy 0, policy_version 1117430 (0.0005) [2023-12-27 13:23:27,460][20347] Updated weights for policy 0, policy_version 1117440 (0.0005) [2023-12-27 13:23:27,490][20381] Updated weights for policy 1, policy_version 1116258 (0.0011) [2023-12-27 13:23:27,544][20381] Updated weights for policy 1, policy_version 1116268 (0.0010) [2023-12-27 13:23:27,598][20381] Updated weights for policy 1, policy_version 1116278 (0.0010) [2023-12-27 13:23:28,070][20347] Updated weights for policy 0, policy_version 1117450 (0.0011) [2023-12-27 13:23:28,125][20347] Updated weights for policy 0, policy_version 1117460 (0.0010) [2023-12-27 13:23:28,173][20347] Updated weights for policy 0, policy_version 1117470 (0.0010) [2023-12-27 13:23:28,250][20381] Updated weights for policy 1, policy_version 1116288 (0.0006) [2023-12-27 13:23:28,303][20381] Updated weights for policy 1, policy_version 1116298 (0.0005) [2023-12-27 13:23:28,366][20381] Updated weights for policy 1, policy_version 1116308 (0.0009) [2023-12-27 13:23:28,848][20347] Updated weights for policy 0, policy_version 1117480 (0.0006) [2023-12-27 13:23:28,905][20347] Updated weights for policy 0, policy_version 1117490 (0.0005) [2023-12-27 13:23:28,953][20347] Updated weights for policy 0, policy_version 1117500 (0.0006) [2023-12-27 13:23:29,031][20381] Updated weights for policy 1, policy_version 1116318 (0.0010) [2023-12-27 13:23:29,078][20381] Updated weights for policy 1, policy_version 1116328 (0.0010) [2023-12-27 13:23:29,137][20381] Updated weights for policy 1, policy_version 1116338 (0.0009) [2023-12-27 13:23:29,633][20347] Updated weights for policy 0, policy_version 1117510 (0.0010) [2023-12-27 13:23:29,692][20347] Updated weights for policy 0, policy_version 1117520 (0.0010) [2023-12-27 13:23:29,763][20347] Updated weights for policy 0, policy_version 1117530 (0.0010) [2023-12-27 13:23:29,808][20381] Updated weights for policy 1, policy_version 1116348 (0.0008) [2023-12-27 13:23:29,866][20381] Updated weights for policy 1, policy_version 1116358 (0.0006) [2023-12-27 13:23:29,918][20381] Updated weights for policy 1, policy_version 1116368 (0.0007) [2023-12-27 13:23:30,402][20347] Updated weights for policy 0, policy_version 1117540 (0.0009) [2023-12-27 13:23:30,458][20347] Updated weights for policy 0, policy_version 1117550 (0.0007) [2023-12-27 13:23:30,510][20347] Updated weights for policy 0, policy_version 1117560 (0.0007) [2023-12-27 13:23:30,555][20381] Updated weights for policy 1, policy_version 1116378 (0.0009) [2023-12-27 13:23:30,615][20381] Updated weights for policy 1, policy_version 1116388 (0.0005) [2023-12-27 13:23:30,672][20381] Updated weights for policy 1, policy_version 1116398 (0.0006) [2023-12-27 13:23:30,722][20381] Updated weights for policy 1, policy_version 1116408 (0.0009) [2023-12-27 13:23:31,173][20347] Updated weights for policy 0, policy_version 1117570 (0.0008) [2023-12-27 13:23:31,227][20347] Updated weights for policy 0, policy_version 1117580 (0.0010) [2023-12-27 13:23:31,288][20347] Updated weights for policy 0, policy_version 1117590 (0.0011) [2023-12-27 13:23:31,347][20347] Updated weights for policy 0, policy_version 1117600 (0.0009) [2023-12-27 13:23:31,393][20381] Updated weights for policy 1, policy_version 1116418 (0.0010) [2023-12-27 13:23:31,398][20292] KL-divergence is very high: 214.4459 [2023-12-27 13:23:31,447][20381] Updated weights for policy 1, policy_version 1116428 (0.0010) [2023-12-27 13:23:31,499][20381] Updated weights for policy 1, policy_version 1116438 (0.0011) [2023-12-27 13:23:32,083][20347] Updated weights for policy 0, policy_version 1117610 (0.0008) [2023-12-27 13:23:32,137][20347] Updated weights for policy 0, policy_version 1117620 (0.0007) [2023-12-27 13:23:32,182][20381] Updated weights for policy 1, policy_version 1116448 (0.0009) [2023-12-27 13:23:32,197][20347] Updated weights for policy 0, policy_version 1117630 (0.0007) [2023-12-27 13:23:32,239][20381] Updated weights for policy 1, policy_version 1116458 (0.0005) [2023-12-27 13:23:32,298][20381] Updated weights for policy 1, policy_version 1116468 (0.0006) [2023-12-27 13:23:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 572006400. Throughput: 0: 10462.8, 1: 10261.8. Samples: 572011276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:32,363][19170] Avg episode reward: [(0, '990.260'), (1, '735.160')] [2023-12-27 13:23:32,792][20347] Updated weights for policy 0, policy_version 1117640 (0.0006) [2023-12-27 13:23:32,843][20347] Updated weights for policy 0, policy_version 1117650 (0.0005) [2023-12-27 13:23:32,900][20347] Updated weights for policy 0, policy_version 1117660 (0.0006) [2023-12-27 13:23:32,934][20381] Updated weights for policy 1, policy_version 1116478 (0.0008) [2023-12-27 13:23:32,990][20381] Updated weights for policy 1, policy_version 1116488 (0.0010) [2023-12-27 13:23:33,044][20381] Updated weights for policy 1, policy_version 1116498 (0.0010) [2023-12-27 13:23:33,560][20347] Updated weights for policy 0, policy_version 1117670 (0.0007) [2023-12-27 13:23:33,606][20347] Updated weights for policy 0, policy_version 1117680 (0.0007) [2023-12-27 13:23:33,647][20347] Updated weights for policy 0, policy_version 1117690 (0.0007) [2023-12-27 13:23:33,668][20381] Updated weights for policy 1, policy_version 1116508 (0.0008) [2023-12-27 13:23:33,718][20381] Updated weights for policy 1, policy_version 1116518 (0.0005) [2023-12-27 13:23:33,723][20292] KL-divergence is very high: 232.3902 [2023-12-27 13:23:33,768][20381] Updated weights for policy 1, policy_version 1116528 (0.0006) [2023-12-27 13:23:34,396][20347] Updated weights for policy 0, policy_version 1117700 (0.0006) [2023-12-27 13:23:34,409][20381] Updated weights for policy 1, policy_version 1116538 (0.0010) [2023-12-27 13:23:34,454][20347] Updated weights for policy 0, policy_version 1117710 (0.0007) [2023-12-27 13:23:34,468][20381] Updated weights for policy 1, policy_version 1116548 (0.0008) [2023-12-27 13:23:34,516][20347] Updated weights for policy 0, policy_version 1117720 (0.0007) [2023-12-27 13:23:34,527][20381] Updated weights for policy 1, policy_version 1116558 (0.0008) [2023-12-27 13:23:34,583][20381] Updated weights for policy 1, policy_version 1116568 (0.0008) [2023-12-27 13:23:35,211][20347] Updated weights for policy 0, policy_version 1117730 (0.0007) [2023-12-27 13:23:35,262][20347] Updated weights for policy 0, policy_version 1117740 (0.0007) [2023-12-27 13:23:35,292][20381] Updated weights for policy 1, policy_version 1116578 (0.0006) [2023-12-27 13:23:35,315][20347] Updated weights for policy 0, policy_version 1117750 (0.0007) [2023-12-27 13:23:35,355][20381] Updated weights for policy 1, policy_version 1116588 (0.0006) [2023-12-27 13:23:35,368][20347] Updated weights for policy 0, policy_version 1117760 (0.0007) [2023-12-27 13:23:35,420][20381] Updated weights for policy 1, policy_version 1116598 (0.0006) [2023-12-27 13:23:35,919][20347] Updated weights for policy 0, policy_version 1117770 (0.0005) [2023-12-27 13:23:35,970][20347] Updated weights for policy 0, policy_version 1117780 (0.0005) [2023-12-27 13:23:36,020][20347] Updated weights for policy 0, policy_version 1117790 (0.0006) [2023-12-27 13:23:36,025][20381] Updated weights for policy 1, policy_version 1116608 (0.0007) [2023-12-27 13:23:36,073][20381] Updated weights for policy 1, policy_version 1116618 (0.0007) [2023-12-27 13:23:36,142][20381] Updated weights for policy 1, policy_version 1116628 (0.0009) [2023-12-27 13:23:36,688][20347] Updated weights for policy 0, policy_version 1117800 (0.0007) [2023-12-27 13:23:36,740][20347] Updated weights for policy 0, policy_version 1117810 (0.0007) [2023-12-27 13:23:36,799][20347] Updated weights for policy 0, policy_version 1117820 (0.0007) [2023-12-27 13:23:36,859][20381] Updated weights for policy 1, policy_version 1116638 (0.0007) [2023-12-27 13:23:36,910][20381] Updated weights for policy 1, policy_version 1116648 (0.0008) [2023-12-27 13:23:36,964][20381] Updated weights for policy 1, policy_version 1116658 (0.0010) [2023-12-27 13:23:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 572104704. Throughput: 0: 10493.2, 1: 10265.3. Samples: 572073656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:37,363][19170] Avg episode reward: [(0, '990.260'), (1, '755.060')] [2023-12-27 13:23:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001117824_286212096.pth... [2023-12-27 13:23:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001116664_285892608.pth... [2023-12-27 13:23:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001116608_285900800.pth [2023-12-27 13:23:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001115480_285589504.pth [2023-12-27 13:23:37,447][20347] Updated weights for policy 0, policy_version 1117830 (0.0010) [2023-12-27 13:23:37,504][20347] Updated weights for policy 0, policy_version 1117840 (0.0011) [2023-12-27 13:23:37,560][20347] Updated weights for policy 0, policy_version 1117850 (0.0011) [2023-12-27 13:23:37,667][20381] Updated weights for policy 1, policy_version 1116668 (0.0010) [2023-12-27 13:23:37,720][20381] Updated weights for policy 1, policy_version 1116678 (0.0010) [2023-12-27 13:23:37,775][20381] Updated weights for policy 1, policy_version 1116688 (0.0010) [2023-12-27 13:23:38,247][20347] Updated weights for policy 0, policy_version 1117860 (0.0010) [2023-12-27 13:23:38,305][20347] Updated weights for policy 0, policy_version 1117870 (0.0010) [2023-12-27 13:23:38,370][20347] Updated weights for policy 0, policy_version 1117880 (0.0010) [2023-12-27 13:23:38,450][20381] Updated weights for policy 1, policy_version 1116698 (0.0007) [2023-12-27 13:23:38,505][20381] Updated weights for policy 1, policy_version 1116708 (0.0008) [2023-12-27 13:23:38,567][20381] Updated weights for policy 1, policy_version 1116718 (0.0008) [2023-12-27 13:23:38,624][20381] Updated weights for policy 1, policy_version 1116728 (0.0007) [2023-12-27 13:23:39,073][20347] Updated weights for policy 0, policy_version 1117890 (0.0007) [2023-12-27 13:23:39,124][20347] Updated weights for policy 0, policy_version 1117900 (0.0009) [2023-12-27 13:23:39,181][20347] Updated weights for policy 0, policy_version 1117910 (0.0007) [2023-12-27 13:23:39,237][20347] Updated weights for policy 0, policy_version 1117920 (0.0007) [2023-12-27 13:23:39,279][20381] Updated weights for policy 1, policy_version 1116738 (0.0007) [2023-12-27 13:23:39,346][20381] Updated weights for policy 1, policy_version 1116748 (0.0007) [2023-12-27 13:23:39,412][20381] Updated weights for policy 1, policy_version 1116758 (0.0007) [2023-12-27 13:23:39,909][20347] Updated weights for policy 0, policy_version 1117930 (0.0006) [2023-12-27 13:23:39,973][20347] Updated weights for policy 0, policy_version 1117940 (0.0007) [2023-12-27 13:23:40,026][20347] Updated weights for policy 0, policy_version 1117950 (0.0006) [2023-12-27 13:23:40,143][20381] Updated weights for policy 1, policy_version 1116768 (0.0008) [2023-12-27 13:23:40,196][20381] Updated weights for policy 1, policy_version 1116778 (0.0008) [2023-12-27 13:23:40,253][20381] Updated weights for policy 1, policy_version 1116788 (0.0007) [2023-12-27 13:23:40,717][20347] Updated weights for policy 0, policy_version 1117960 (0.0007) [2023-12-27 13:23:40,775][20347] Updated weights for policy 0, policy_version 1117970 (0.0006) [2023-12-27 13:23:40,830][20347] Updated weights for policy 0, policy_version 1117980 (0.0006) [2023-12-27 13:23:40,941][20381] Updated weights for policy 1, policy_version 1116798 (0.0009) [2023-12-27 13:23:40,995][20381] Updated weights for policy 1, policy_version 1116808 (0.0010) [2023-12-27 13:23:41,059][20381] Updated weights for policy 1, policy_version 1116818 (0.0009) [2023-12-27 13:23:41,548][20347] Updated weights for policy 0, policy_version 1117990 (0.0008) [2023-12-27 13:23:41,620][20347] Updated weights for policy 0, policy_version 1118000 (0.0006) [2023-12-27 13:23:41,685][20347] Updated weights for policy 0, policy_version 1118010 (0.0009) [2023-12-27 13:23:41,767][20381] Updated weights for policy 1, policy_version 1116828 (0.0009) [2023-12-27 13:23:41,823][20381] Updated weights for policy 1, policy_version 1116838 (0.0011) [2023-12-27 13:23:41,879][20381] Updated weights for policy 1, policy_version 1116848 (0.0011) [2023-12-27 13:23:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 572203008. Throughput: 0: 10499.4, 1: 10231.9. Samples: 572196320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '676.480')] [2023-12-27 13:23:42,383][20347] Updated weights for policy 0, policy_version 1118020 (0.0007) [2023-12-27 13:23:42,432][20347] Updated weights for policy 0, policy_version 1118030 (0.0007) [2023-12-27 13:23:42,489][20347] Updated weights for policy 0, policy_version 1118040 (0.0008) [2023-12-27 13:23:42,574][20381] Updated weights for policy 1, policy_version 1116858 (0.0010) [2023-12-27 13:23:42,631][20381] Updated weights for policy 1, policy_version 1116868 (0.0008) [2023-12-27 13:23:42,690][20381] Updated weights for policy 1, policy_version 1116878 (0.0007) [2023-12-27 13:23:42,753][20381] Updated weights for policy 1, policy_version 1116888 (0.0007) [2023-12-27 13:23:43,167][20347] Updated weights for policy 0, policy_version 1118050 (0.0007) [2023-12-27 13:23:43,222][20347] Updated weights for policy 0, policy_version 1118060 (0.0010) [2023-12-27 13:23:43,280][20347] Updated weights for policy 0, policy_version 1118070 (0.0010) [2023-12-27 13:23:43,333][20347] Updated weights for policy 0, policy_version 1118080 (0.0010) [2023-12-27 13:23:43,432][20381] Updated weights for policy 1, policy_version 1116898 (0.0007) [2023-12-27 13:23:43,487][20381] Updated weights for policy 1, policy_version 1116908 (0.0007) [2023-12-27 13:23:43,545][20381] Updated weights for policy 1, policy_version 1116918 (0.0006) [2023-12-27 13:23:44,008][20347] Updated weights for policy 0, policy_version 1118090 (0.0010) [2023-12-27 13:23:44,066][20347] Updated weights for policy 0, policy_version 1118100 (0.0010) [2023-12-27 13:23:44,117][20347] Updated weights for policy 0, policy_version 1118110 (0.0010) [2023-12-27 13:23:44,143][20381] Updated weights for policy 1, policy_version 1116928 (0.0005) [2023-12-27 13:23:44,200][20381] Updated weights for policy 1, policy_version 1116938 (0.0005) [2023-12-27 13:23:44,263][20381] Updated weights for policy 1, policy_version 1116948 (0.0005) [2023-12-27 13:23:44,762][20347] Updated weights for policy 0, policy_version 1118120 (0.0010) [2023-12-27 13:23:44,819][20347] Updated weights for policy 0, policy_version 1118130 (0.0007) [2023-12-27 13:23:44,882][20347] Updated weights for policy 0, policy_version 1118140 (0.0010) [2023-12-27 13:23:44,909][20381] Updated weights for policy 1, policy_version 1116958 (0.0009) [2023-12-27 13:23:44,969][20381] Updated weights for policy 1, policy_version 1116968 (0.0011) [2023-12-27 13:23:45,032][20381] Updated weights for policy 1, policy_version 1116978 (0.0011) [2023-12-27 13:23:45,560][20347] Updated weights for policy 0, policy_version 1118150 (0.0010) [2023-12-27 13:23:45,617][20347] Updated weights for policy 0, policy_version 1118160 (0.0010) [2023-12-27 13:23:45,671][20347] Updated weights for policy 0, policy_version 1118170 (0.0010) [2023-12-27 13:23:45,699][20381] Updated weights for policy 1, policy_version 1116988 (0.0010) [2023-12-27 13:23:45,748][20381] Updated weights for policy 1, policy_version 1116998 (0.0010) [2023-12-27 13:23:45,798][20381] Updated weights for policy 1, policy_version 1117008 (0.0009) [2023-12-27 13:23:46,306][20347] Updated weights for policy 0, policy_version 1118180 (0.0010) [2023-12-27 13:23:46,356][20347] Updated weights for policy 0, policy_version 1118190 (0.0010) [2023-12-27 13:23:46,427][20347] Updated weights for policy 0, policy_version 1118200 (0.0009) [2023-12-27 13:23:46,478][20381] Updated weights for policy 1, policy_version 1117018 (0.0008) [2023-12-27 13:23:46,534][20381] Updated weights for policy 1, policy_version 1117028 (0.0011) [2023-12-27 13:23:46,591][20381] Updated weights for policy 1, policy_version 1117038 (0.0011) [2023-12-27 13:23:46,653][20381] Updated weights for policy 1, policy_version 1117048 (0.0011) [2023-12-27 13:23:47,122][20347] Updated weights for policy 0, policy_version 1118210 (0.0007) [2023-12-27 13:23:47,173][20347] Updated weights for policy 0, policy_version 1118220 (0.0010) [2023-12-27 13:23:47,226][20347] Updated weights for policy 0, policy_version 1118230 (0.0010) [2023-12-27 13:23:47,270][20347] Updated weights for policy 0, policy_version 1118240 (0.0010) [2023-12-27 13:23:47,346][20381] Updated weights for policy 1, policy_version 1117058 (0.0010) [2023-12-27 13:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 572309504. Throughput: 0: 10512.3, 1: 10232.6. Samples: 572319628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '754.770')] [2023-12-27 13:23:47,400][20381] Updated weights for policy 1, policy_version 1117068 (0.0010) [2023-12-27 13:23:47,452][20381] Updated weights for policy 1, policy_version 1117078 (0.0010) [2023-12-27 13:23:47,943][20347] Updated weights for policy 0, policy_version 1118250 (0.0010) [2023-12-27 13:23:47,991][20347] Updated weights for policy 0, policy_version 1118260 (0.0010) [2023-12-27 13:23:48,042][20347] Updated weights for policy 0, policy_version 1118270 (0.0010) [2023-12-27 13:23:48,104][20381] Updated weights for policy 1, policy_version 1117088 (0.0006) [2023-12-27 13:23:48,152][20381] Updated weights for policy 1, policy_version 1117098 (0.0006) [2023-12-27 13:23:48,204][20381] Updated weights for policy 1, policy_version 1117108 (0.0005) [2023-12-27 13:23:48,789][20347] Updated weights for policy 0, policy_version 1118280 (0.0010) [2023-12-27 13:23:48,844][20347] Updated weights for policy 0, policy_version 1118290 (0.0010) [2023-12-27 13:23:48,881][20381] Updated weights for policy 1, policy_version 1117118 (0.0009) [2023-12-27 13:23:48,899][20347] Updated weights for policy 0, policy_version 1118300 (0.0010) [2023-12-27 13:23:48,938][20381] Updated weights for policy 1, policy_version 1117128 (0.0006) [2023-12-27 13:23:48,992][20381] Updated weights for policy 1, policy_version 1117138 (0.0007) [2023-12-27 13:23:49,632][20347] Updated weights for policy 0, policy_version 1118310 (0.0010) [2023-12-27 13:23:49,693][20381] Updated weights for policy 1, policy_version 1117148 (0.0007) [2023-12-27 13:23:49,693][20347] Updated weights for policy 0, policy_version 1118320 (0.0010) [2023-12-27 13:23:49,755][20381] Updated weights for policy 1, policy_version 1117158 (0.0007) [2023-12-27 13:23:49,760][20347] Updated weights for policy 0, policy_version 1118330 (0.0011) [2023-12-27 13:23:49,811][20381] Updated weights for policy 1, policy_version 1117168 (0.0006) [2023-12-27 13:23:50,445][20381] Updated weights for policy 1, policy_version 1117178 (0.0008) [2023-12-27 13:23:50,451][20347] Updated weights for policy 0, policy_version 1118340 (0.0010) [2023-12-27 13:23:50,499][20381] Updated weights for policy 1, policy_version 1117188 (0.0006) [2023-12-27 13:23:50,501][20347] Updated weights for policy 0, policy_version 1118350 (0.0008) [2023-12-27 13:23:50,559][20381] Updated weights for policy 1, policy_version 1117198 (0.0007) [2023-12-27 13:23:50,559][20347] Updated weights for policy 0, policy_version 1118360 (0.0006) [2023-12-27 13:23:50,625][20381] Updated weights for policy 1, policy_version 1117208 (0.0008) [2023-12-27 13:23:51,246][20347] Updated weights for policy 0, policy_version 1118370 (0.0009) [2023-12-27 13:23:51,306][20347] Updated weights for policy 0, policy_version 1118380 (0.0008) [2023-12-27 13:23:51,311][20381] Updated weights for policy 1, policy_version 1117218 (0.0007) [2023-12-27 13:23:51,311][20291] KL-divergence is very high: 218.8667 [2023-12-27 13:23:51,370][20347] Updated weights for policy 0, policy_version 1118390 (0.0008) [2023-12-27 13:23:51,374][20381] Updated weights for policy 1, policy_version 1117228 (0.0009) [2023-12-27 13:23:51,430][20347] Updated weights for policy 0, policy_version 1118400 (0.0008) [2023-12-27 13:23:51,444][20381] Updated weights for policy 1, policy_version 1117238 (0.0008) [2023-12-27 13:23:52,129][20381] Updated weights for policy 1, policy_version 1117248 (0.0007) [2023-12-27 13:23:52,186][20347] Updated weights for policy 0, policy_version 1118410 (0.0006) [2023-12-27 13:23:52,188][20381] Updated weights for policy 1, policy_version 1117258 (0.0006) [2023-12-27 13:23:52,244][20347] Updated weights for policy 0, policy_version 1118420 (0.0006) [2023-12-27 13:23:52,250][20381] Updated weights for policy 1, policy_version 1117268 (0.0006) [2023-12-27 13:23:52,311][20347] Updated weights for policy 0, policy_version 1118430 (0.0008) [2023-12-27 13:23:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.6, 300 sec: 20521.6). Total num frames: 572416000. Throughput: 0: 10441.1, 1: 10213.7. Samples: 572380356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:52,363][19170] Avg episode reward: [(0, '990.260'), (1, '676.340')] [2023-12-27 13:23:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001117272_286048256.pth... [2023-12-27 13:23:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001118432_286367744.pth... [2023-12-27 13:23:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001116056_285736960.pth [2023-12-27 13:23:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001117216_286056448.pth [2023-12-27 13:23:52,916][20381] Updated weights for policy 1, policy_version 1117278 (0.0007) [2023-12-27 13:23:52,970][20381] Updated weights for policy 1, policy_version 1117288 (0.0007) [2023-12-27 13:23:53,002][20347] Updated weights for policy 0, policy_version 1118440 (0.0006) [2023-12-27 13:23:53,026][20381] Updated weights for policy 1, policy_version 1117298 (0.0006) [2023-12-27 13:23:53,057][20347] Updated weights for policy 0, policy_version 1118450 (0.0006) [2023-12-27 13:23:53,115][20347] Updated weights for policy 0, policy_version 1118460 (0.0008) [2023-12-27 13:23:53,669][20381] Updated weights for policy 1, policy_version 1117308 (0.0007) [2023-12-27 13:23:53,720][20381] Updated weights for policy 1, policy_version 1117318 (0.0007) [2023-12-27 13:23:53,779][20381] Updated weights for policy 1, policy_version 1117328 (0.0007) [2023-12-27 13:23:53,781][20347] Updated weights for policy 0, policy_version 1118470 (0.0008) [2023-12-27 13:23:53,834][20347] Updated weights for policy 0, policy_version 1118480 (0.0006) [2023-12-27 13:23:53,879][20347] Updated weights for policy 0, policy_version 1118490 (0.0007) [2023-12-27 13:23:54,474][20381] Updated weights for policy 1, policy_version 1117338 (0.0006) [2023-12-27 13:23:54,534][20381] Updated weights for policy 1, policy_version 1117348 (0.0008) [2023-12-27 13:23:54,587][20347] Updated weights for policy 0, policy_version 1118500 (0.0007) [2023-12-27 13:23:54,599][20381] Updated weights for policy 1, policy_version 1117358 (0.0009) [2023-12-27 13:23:54,639][20347] Updated weights for policy 0, policy_version 1118510 (0.0006) [2023-12-27 13:23:54,653][20381] Updated weights for policy 1, policy_version 1117368 (0.0006) [2023-12-27 13:23:54,692][20347] Updated weights for policy 0, policy_version 1118520 (0.0009) [2023-12-27 13:23:55,306][20381] Updated weights for policy 1, policy_version 1117378 (0.0010) [2023-12-27 13:23:55,362][20381] Updated weights for policy 1, policy_version 1117388 (0.0010) [2023-12-27 13:23:55,379][20347] Updated weights for policy 0, policy_version 1118530 (0.0009) [2023-12-27 13:23:55,420][20381] Updated weights for policy 1, policy_version 1117398 (0.0010) [2023-12-27 13:23:55,435][20347] Updated weights for policy 0, policy_version 1118540 (0.0007) [2023-12-27 13:23:55,479][20347] Updated weights for policy 0, policy_version 1118550 (0.0007) [2023-12-27 13:23:55,525][20347] Updated weights for policy 0, policy_version 1118560 (0.0007) [2023-12-27 13:23:56,143][20381] Updated weights for policy 1, policy_version 1117408 (0.0011) [2023-12-27 13:23:56,202][20381] Updated weights for policy 1, policy_version 1117418 (0.0010) [2023-12-27 13:23:56,235][20347] Updated weights for policy 0, policy_version 1118570 (0.0005) [2023-12-27 13:23:56,255][20381] Updated weights for policy 1, policy_version 1117428 (0.0009) [2023-12-27 13:23:56,291][20347] Updated weights for policy 0, policy_version 1118580 (0.0006) [2023-12-27 13:23:56,343][20347] Updated weights for policy 0, policy_version 1118590 (0.0008) [2023-12-27 13:23:56,838][20381] Updated weights for policy 1, policy_version 1117438 (0.0007) [2023-12-27 13:23:56,891][20381] Updated weights for policy 1, policy_version 1117448 (0.0008) [2023-12-27 13:23:56,940][20381] Updated weights for policy 1, policy_version 1117458 (0.0008) [2023-12-27 13:23:56,980][20347] Updated weights for policy 0, policy_version 1118600 (0.0007) [2023-12-27 13:23:57,043][20347] Updated weights for policy 0, policy_version 1118610 (0.0006) [2023-12-27 13:23:57,105][20347] Updated weights for policy 0, policy_version 1118620 (0.0005) [2023-12-27 13:23:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 572514304. Throughput: 0: 10339.9, 1: 10251.7. Samples: 572501112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:23:57,363][19170] Avg episode reward: [(0, '990.260'), (1, '676.410')] [2023-12-27 13:23:57,693][20381] Updated weights for policy 1, policy_version 1117468 (0.0006) [2023-12-27 13:23:57,697][20347] Updated weights for policy 0, policy_version 1118630 (0.0008) [2023-12-27 13:23:57,746][20381] Updated weights for policy 1, policy_version 1117478 (0.0006) [2023-12-27 13:23:57,752][20347] Updated weights for policy 0, policy_version 1118640 (0.0010) [2023-12-27 13:23:57,804][20381] Updated weights for policy 1, policy_version 1117488 (0.0005) [2023-12-27 13:23:57,807][20347] Updated weights for policy 0, policy_version 1118650 (0.0008) [2023-12-27 13:23:58,447][20381] Updated weights for policy 1, policy_version 1117498 (0.0006) [2023-12-27 13:23:58,478][20347] Updated weights for policy 0, policy_version 1118660 (0.0008) [2023-12-27 13:23:58,507][20381] Updated weights for policy 1, policy_version 1117508 (0.0008) [2023-12-27 13:23:58,542][20347] Updated weights for policy 0, policy_version 1118670 (0.0008) [2023-12-27 13:23:58,569][20381] Updated weights for policy 1, policy_version 1117518 (0.0008) [2023-12-27 13:23:58,602][20347] Updated weights for policy 0, policy_version 1118680 (0.0007) [2023-12-27 13:23:58,629][20381] Updated weights for policy 1, policy_version 1117528 (0.0007) [2023-12-27 13:23:59,403][20347] Updated weights for policy 0, policy_version 1118690 (0.0008) [2023-12-27 13:23:59,409][20381] Updated weights for policy 1, policy_version 1117538 (0.0010) [2023-12-27 13:23:59,455][20381] Updated weights for policy 1, policy_version 1117548 (0.0010) [2023-12-27 13:23:59,457][20347] Updated weights for policy 0, policy_version 1118700 (0.0008) [2023-12-27 13:23:59,500][20381] Updated weights for policy 1, policy_version 1117558 (0.0010) [2023-12-27 13:23:59,508][20347] Updated weights for policy 0, policy_version 1118710 (0.0008) [2023-12-27 13:23:59,570][20347] Updated weights for policy 0, policy_version 1118720 (0.0006) [2023-12-27 13:24:00,182][20381] Updated weights for policy 1, policy_version 1117568 (0.0009) [2023-12-27 13:24:00,239][20381] Updated weights for policy 1, policy_version 1117578 (0.0010) [2023-12-27 13:24:00,274][20347] Updated weights for policy 0, policy_version 1118730 (0.0006) [2023-12-27 13:24:00,298][20381] Updated weights for policy 1, policy_version 1117588 (0.0007) [2023-12-27 13:24:00,333][20347] Updated weights for policy 0, policy_version 1118740 (0.0006) [2023-12-27 13:24:00,391][20347] Updated weights for policy 0, policy_version 1118750 (0.0007) [2023-12-27 13:24:00,939][20381] Updated weights for policy 1, policy_version 1117598 (0.0006) [2023-12-27 13:24:00,997][20381] Updated weights for policy 1, policy_version 1117608 (0.0008) [2023-12-27 13:24:01,060][20381] Updated weights for policy 1, policy_version 1117618 (0.0007) [2023-12-27 13:24:01,063][20347] Updated weights for policy 0, policy_version 1118760 (0.0007) [2023-12-27 13:24:01,126][20347] Updated weights for policy 0, policy_version 1118770 (0.0008) [2023-12-27 13:24:01,185][20347] Updated weights for policy 0, policy_version 1118780 (0.0008) [2023-12-27 13:24:01,767][20381] Updated weights for policy 1, policy_version 1117628 (0.0007) [2023-12-27 13:24:01,814][20381] Updated weights for policy 1, policy_version 1117638 (0.0005) [2023-12-27 13:24:01,864][20381] Updated weights for policy 1, policy_version 1117648 (0.0005) [2023-12-27 13:24:01,992][20347] Updated weights for policy 0, policy_version 1118790 (0.0008) [2023-12-27 13:24:02,057][20347] Updated weights for policy 0, policy_version 1118800 (0.0008) [2023-12-27 13:24:02,114][20347] Updated weights for policy 0, policy_version 1118810 (0.0007) [2023-12-27 13:24:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 572612608. Throughput: 0: 10231.1, 1: 10219.7. Samples: 572622756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '735.210')] [2023-12-27 13:24:02,456][20381] Updated weights for policy 1, policy_version 1117658 (0.0006) [2023-12-27 13:24:02,517][20381] Updated weights for policy 1, policy_version 1117668 (0.0007) [2023-12-27 13:24:02,577][20381] Updated weights for policy 1, policy_version 1117678 (0.0008) [2023-12-27 13:24:02,637][20381] Updated weights for policy 1, policy_version 1117688 (0.0008) [2023-12-27 13:24:02,785][20347] Updated weights for policy 0, policy_version 1118820 (0.0006) [2023-12-27 13:24:02,840][20347] Updated weights for policy 0, policy_version 1118830 (0.0007) [2023-12-27 13:24:02,894][20347] Updated weights for policy 0, policy_version 1118840 (0.0007) [2023-12-27 13:24:03,333][20381] Updated weights for policy 1, policy_version 1117698 (0.0007) [2023-12-27 13:24:03,389][20381] Updated weights for policy 1, policy_version 1117708 (0.0005) [2023-12-27 13:24:03,450][20381] Updated weights for policy 1, policy_version 1117718 (0.0007) [2023-12-27 13:24:03,601][20347] Updated weights for policy 0, policy_version 1118850 (0.0007) [2023-12-27 13:24:03,652][20347] Updated weights for policy 0, policy_version 1118860 (0.0008) [2023-12-27 13:24:03,699][20347] Updated weights for policy 0, policy_version 1118870 (0.0007) [2023-12-27 13:24:03,749][20347] Updated weights for policy 0, policy_version 1118880 (0.0007) [2023-12-27 13:24:04,093][20381] Updated weights for policy 1, policy_version 1117728 (0.0008) [2023-12-27 13:24:04,151][20381] Updated weights for policy 1, policy_version 1117738 (0.0008) [2023-12-27 13:24:04,211][20381] Updated weights for policy 1, policy_version 1117748 (0.0007) [2023-12-27 13:24:04,467][20347] Updated weights for policy 0, policy_version 1118890 (0.0007) [2023-12-27 13:24:04,524][20347] Updated weights for policy 0, policy_version 1118900 (0.0007) [2023-12-27 13:24:04,581][20347] Updated weights for policy 0, policy_version 1118910 (0.0007) [2023-12-27 13:24:04,882][20381] Updated weights for policy 1, policy_version 1117758 (0.0006) [2023-12-27 13:24:04,946][20381] Updated weights for policy 1, policy_version 1117768 (0.0005) [2023-12-27 13:24:05,007][20381] Updated weights for policy 1, policy_version 1117778 (0.0008) [2023-12-27 13:24:05,265][20347] Updated weights for policy 0, policy_version 1118920 (0.0009) [2023-12-27 13:24:05,325][20347] Updated weights for policy 0, policy_version 1118930 (0.0008) [2023-12-27 13:24:05,377][20347] Updated weights for policy 0, policy_version 1118940 (0.0010) [2023-12-27 13:24:05,616][20381] Updated weights for policy 1, policy_version 1117788 (0.0007) [2023-12-27 13:24:05,665][20381] Updated weights for policy 1, policy_version 1117798 (0.0005) [2023-12-27 13:24:05,711][20381] Updated weights for policy 1, policy_version 1117808 (0.0007) [2023-12-27 13:24:06,034][20347] Updated weights for policy 0, policy_version 1118950 (0.0007) [2023-12-27 13:24:06,094][20347] Updated weights for policy 0, policy_version 1118960 (0.0007) [2023-12-27 13:24:06,152][20347] Updated weights for policy 0, policy_version 1118970 (0.0007) [2023-12-27 13:24:06,394][20381] Updated weights for policy 1, policy_version 1117818 (0.0007) [2023-12-27 13:24:06,454][20381] Updated weights for policy 1, policy_version 1117828 (0.0008) [2023-12-27 13:24:06,508][20381] Updated weights for policy 1, policy_version 1117838 (0.0008) [2023-12-27 13:24:06,567][20381] Updated weights for policy 1, policy_version 1117848 (0.0008) [2023-12-27 13:24:06,802][20347] Updated weights for policy 0, policy_version 1118980 (0.0007) [2023-12-27 13:24:06,858][20347] Updated weights for policy 0, policy_version 1118990 (0.0007) [2023-12-27 13:24:06,903][20347] Updated weights for policy 0, policy_version 1119000 (0.0007) [2023-12-27 13:24:07,287][20381] Updated weights for policy 1, policy_version 1117858 (0.0009) [2023-12-27 13:24:07,356][20381] Updated weights for policy 1, policy_version 1117868 (0.0006) [2023-12-27 13:24:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 572710912. Throughput: 0: 10221.9, 1: 10224.2. Samples: 572683340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '754.780')] [2023-12-27 13:24:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001119008_286515200.pth... [2023-12-27 13:24:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001117824_286212096.pth [2023-12-27 13:24:07,424][20381] Updated weights for policy 1, policy_version 1117878 (0.0006) [2023-12-27 13:24:07,436][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001117880_286203904.pth... [2023-12-27 13:24:07,439][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001116664_285892608.pth [2023-12-27 13:24:07,577][20347] Updated weights for policy 0, policy_version 1119010 (0.0007) [2023-12-27 13:24:07,626][20347] Updated weights for policy 0, policy_version 1119020 (0.0010) [2023-12-27 13:24:07,674][20347] Updated weights for policy 0, policy_version 1119030 (0.0010) [2023-12-27 13:24:07,730][20347] Updated weights for policy 0, policy_version 1119040 (0.0010) [2023-12-27 13:24:08,072][20381] Updated weights for policy 1, policy_version 1117888 (0.0010) [2023-12-27 13:24:08,124][20381] Updated weights for policy 1, policy_version 1117898 (0.0011) [2023-12-27 13:24:08,182][20381] Updated weights for policy 1, policy_version 1117908 (0.0009) [2023-12-27 13:24:08,334][20347] Updated weights for policy 0, policy_version 1119050 (0.0006) [2023-12-27 13:24:08,397][20347] Updated weights for policy 0, policy_version 1119060 (0.0006) [2023-12-27 13:24:08,456][20347] Updated weights for policy 0, policy_version 1119070 (0.0006) [2023-12-27 13:24:08,816][20381] Updated weights for policy 1, policy_version 1117918 (0.0009) [2023-12-27 13:24:08,874][20381] Updated weights for policy 1, policy_version 1117928 (0.0011) [2023-12-27 13:24:08,931][20381] Updated weights for policy 1, policy_version 1117938 (0.0011) [2023-12-27 13:24:09,057][20347] Updated weights for policy 0, policy_version 1119080 (0.0010) [2023-12-27 13:24:09,105][20347] Updated weights for policy 0, policy_version 1119090 (0.0007) [2023-12-27 13:24:09,156][20347] Updated weights for policy 0, policy_version 1119100 (0.0006) [2023-12-27 13:24:09,647][20381] Updated weights for policy 1, policy_version 1117948 (0.0009) [2023-12-27 13:24:09,702][20381] Updated weights for policy 1, policy_version 1117958 (0.0007) [2023-12-27 13:24:09,760][20381] Updated weights for policy 1, policy_version 1117968 (0.0009) [2023-12-27 13:24:09,867][20347] Updated weights for policy 0, policy_version 1119110 (0.0009) [2023-12-27 13:24:09,934][20347] Updated weights for policy 0, policy_version 1119120 (0.0007) [2023-12-27 13:24:09,994][20347] Updated weights for policy 0, policy_version 1119130 (0.0006) [2023-12-27 13:24:10,463][20381] Updated weights for policy 1, policy_version 1117978 (0.0007) [2023-12-27 13:24:10,519][20381] Updated weights for policy 1, policy_version 1117988 (0.0006) [2023-12-27 13:24:10,581][20381] Updated weights for policy 1, policy_version 1117998 (0.0005) [2023-12-27 13:24:10,643][20381] Updated weights for policy 1, policy_version 1118008 (0.0006) [2023-12-27 13:24:10,646][20347] Updated weights for policy 0, policy_version 1119140 (0.0006) [2023-12-27 13:24:10,708][20347] Updated weights for policy 0, policy_version 1119150 (0.0008) [2023-12-27 13:24:10,763][20347] Updated weights for policy 0, policy_version 1119160 (0.0008) [2023-12-27 13:24:11,308][20381] Updated weights for policy 1, policy_version 1118018 (0.0008) [2023-12-27 13:24:11,379][20381] Updated weights for policy 1, policy_version 1118028 (0.0009) [2023-12-27 13:24:11,439][20381] Updated weights for policy 1, policy_version 1118038 (0.0006) [2023-12-27 13:24:11,496][20347] Updated weights for policy 0, policy_version 1119170 (0.0007) [2023-12-27 13:24:11,553][20347] Updated weights for policy 0, policy_version 1119180 (0.0008) [2023-12-27 13:24:11,608][20347] Updated weights for policy 0, policy_version 1119190 (0.0008) [2023-12-27 13:24:11,677][20347] Updated weights for policy 0, policy_version 1119200 (0.0008) [2023-12-27 13:24:12,068][20381] Updated weights for policy 1, policy_version 1118048 (0.0006) [2023-12-27 13:24:12,130][20381] Updated weights for policy 1, policy_version 1118058 (0.0008) [2023-12-27 13:24:12,194][20381] Updated weights for policy 1, policy_version 1118068 (0.0007) [2023-12-27 13:24:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 572817408. Throughput: 0: 10220.8, 1: 10268.5. Samples: 572808336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '764.550')] [2023-12-27 13:24:12,436][20347] Updated weights for policy 0, policy_version 1119210 (0.0007) [2023-12-27 13:24:12,498][20347] Updated weights for policy 0, policy_version 1119220 (0.0008) [2023-12-27 13:24:12,551][20347] Updated weights for policy 0, policy_version 1119230 (0.0008) [2023-12-27 13:24:12,900][20381] Updated weights for policy 1, policy_version 1118078 (0.0008) [2023-12-27 13:24:12,952][20381] Updated weights for policy 1, policy_version 1118088 (0.0008) [2023-12-27 13:24:13,010][20381] Updated weights for policy 1, policy_version 1118098 (0.0008) [2023-12-27 13:24:13,271][20347] Updated weights for policy 0, policy_version 1119240 (0.0006) [2023-12-27 13:24:13,332][20347] Updated weights for policy 0, policy_version 1119250 (0.0008) [2023-12-27 13:24:13,392][20347] Updated weights for policy 0, policy_version 1119260 (0.0008) [2023-12-27 13:24:13,595][20381] Updated weights for policy 1, policy_version 1118108 (0.0007) [2023-12-27 13:24:13,647][20381] Updated weights for policy 1, policy_version 1118118 (0.0007) [2023-12-27 13:24:13,698][20381] Updated weights for policy 1, policy_version 1118128 (0.0007) [2023-12-27 13:24:14,086][20347] Updated weights for policy 0, policy_version 1119270 (0.0009) [2023-12-27 13:24:14,141][20347] Updated weights for policy 0, policy_version 1119280 (0.0008) [2023-12-27 13:24:14,195][20347] Updated weights for policy 0, policy_version 1119290 (0.0010) [2023-12-27 13:24:14,244][20381] Updated weights for policy 1, policy_version 1118138 (0.0006) [2023-12-27 13:24:14,301][20381] Updated weights for policy 1, policy_version 1118148 (0.0006) [2023-12-27 13:24:14,354][20381] Updated weights for policy 1, policy_version 1118158 (0.0008) [2023-12-27 13:24:14,401][20381] Updated weights for policy 1, policy_version 1118168 (0.0005) [2023-12-27 13:24:14,948][20347] Updated weights for policy 0, policy_version 1119300 (0.0009) [2023-12-27 13:24:15,015][20347] Updated weights for policy 0, policy_version 1119310 (0.0008) [2023-12-27 13:24:15,052][20381] Updated weights for policy 1, policy_version 1118178 (0.0008) [2023-12-27 13:24:15,075][20347] Updated weights for policy 0, policy_version 1119320 (0.0007) [2023-12-27 13:24:15,108][20381] Updated weights for policy 1, policy_version 1118188 (0.0008) [2023-12-27 13:24:15,165][20381] Updated weights for policy 1, policy_version 1118198 (0.0008) [2023-12-27 13:24:15,771][20347] Updated weights for policy 0, policy_version 1119330 (0.0008) [2023-12-27 13:24:15,794][20381] Updated weights for policy 1, policy_version 1118208 (0.0008) [2023-12-27 13:24:15,829][20347] Updated weights for policy 0, policy_version 1119340 (0.0010) [2023-12-27 13:24:15,842][20381] Updated weights for policy 1, policy_version 1118218 (0.0005) [2023-12-27 13:24:15,884][20347] Updated weights for policy 0, policy_version 1119350 (0.0010) [2023-12-27 13:24:15,894][20381] Updated weights for policy 1, policy_version 1118228 (0.0005) [2023-12-27 13:24:15,936][20347] Updated weights for policy 0, policy_version 1119360 (0.0006) [2023-12-27 13:24:16,479][20381] Updated weights for policy 1, policy_version 1118238 (0.0007) [2023-12-27 13:24:16,526][20381] Updated weights for policy 1, policy_version 1118248 (0.0005) [2023-12-27 13:24:16,570][20381] Updated weights for policy 1, policy_version 1118258 (0.0005) [2023-12-27 13:24:16,595][20347] Updated weights for policy 0, policy_version 1119370 (0.0007) [2023-12-27 13:24:16,649][20347] Updated weights for policy 0, policy_version 1119380 (0.0007) [2023-12-27 13:24:16,714][20347] Updated weights for policy 0, policy_version 1119390 (0.0007) [2023-12-27 13:24:17,222][20381] Updated weights for policy 1, policy_version 1118268 (0.0007) [2023-12-27 13:24:17,280][20381] Updated weights for policy 1, policy_version 1118278 (0.0011) [2023-12-27 13:24:17,341][20381] Updated weights for policy 1, policy_version 1118288 (0.0008) [2023-12-27 13:24:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 572915712. Throughput: 0: 10127.7, 1: 10356.5. Samples: 572933068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:17,363][19170] Avg episode reward: [(0, '990.280'), (1, '725.580')] [2023-12-27 13:24:17,398][20347] Updated weights for policy 0, policy_version 1119400 (0.0010) [2023-12-27 13:24:17,443][20347] Updated weights for policy 0, policy_version 1119410 (0.0010) [2023-12-27 13:24:17,496][20347] Updated weights for policy 0, policy_version 1119420 (0.0010) [2023-12-27 13:24:18,009][20381] Updated weights for policy 1, policy_version 1118298 (0.0007) [2023-12-27 13:24:18,064][20381] Updated weights for policy 1, policy_version 1118308 (0.0005) [2023-12-27 13:24:18,124][20381] Updated weights for policy 1, policy_version 1118318 (0.0008) [2023-12-27 13:24:18,177][20347] Updated weights for policy 0, policy_version 1119430 (0.0010) [2023-12-27 13:24:18,182][20381] Updated weights for policy 1, policy_version 1118328 (0.0011) [2023-12-27 13:24:18,241][20347] Updated weights for policy 0, policy_version 1119440 (0.0010) [2023-12-27 13:24:18,292][20347] Updated weights for policy 0, policy_version 1119450 (0.0010) [2023-12-27 13:24:18,875][20381] Updated weights for policy 1, policy_version 1118338 (0.0011) [2023-12-27 13:24:18,937][20381] Updated weights for policy 1, policy_version 1118348 (0.0011) [2023-12-27 13:24:18,954][20347] Updated weights for policy 0, policy_version 1119460 (0.0008) [2023-12-27 13:24:18,984][20381] Updated weights for policy 1, policy_version 1118358 (0.0010) [2023-12-27 13:24:19,010][20347] Updated weights for policy 0, policy_version 1119470 (0.0010) [2023-12-27 13:24:19,068][20347] Updated weights for policy 0, policy_version 1119480 (0.0010) [2023-12-27 13:24:19,685][20381] Updated weights for policy 1, policy_version 1118368 (0.0008) [2023-12-27 13:24:19,735][20381] Updated weights for policy 1, policy_version 1118378 (0.0006) [2023-12-27 13:24:19,797][20381] Updated weights for policy 1, policy_version 1118388 (0.0006) [2023-12-27 13:24:19,810][20347] Updated weights for policy 0, policy_version 1119490 (0.0010) [2023-12-27 13:24:19,882][20347] Updated weights for policy 0, policy_version 1119501 (0.0008) [2023-12-27 13:24:19,946][20347] Updated weights for policy 0, policy_version 1119511 (0.0008) [2023-12-27 13:24:20,497][20381] Updated weights for policy 1, policy_version 1118398 (0.0008) [2023-12-27 13:24:20,551][20381] Updated weights for policy 1, policy_version 1118408 (0.0007) [2023-12-27 13:24:20,565][20347] Updated weights for policy 0, policy_version 1119521 (0.0008) [2023-12-27 13:24:20,608][20381] Updated weights for policy 1, policy_version 1118418 (0.0008) [2023-12-27 13:24:20,628][20347] Updated weights for policy 0, policy_version 1119531 (0.0007) [2023-12-27 13:24:20,690][20347] Updated weights for policy 0, policy_version 1119541 (0.0007) [2023-12-27 13:24:20,755][20347] Updated weights for policy 0, policy_version 1119551 (0.0008) [2023-12-27 13:24:21,313][20381] Updated weights for policy 1, policy_version 1118428 (0.0008) [2023-12-27 13:24:21,374][20381] Updated weights for policy 1, policy_version 1118438 (0.0008) [2023-12-27 13:24:21,445][20381] Updated weights for policy 1, policy_version 1118448 (0.0008) [2023-12-27 13:24:21,518][20347] Updated weights for policy 0, policy_version 1119561 (0.0006) [2023-12-27 13:24:21,578][20347] Updated weights for policy 0, policy_version 1119571 (0.0008) [2023-12-27 13:24:21,643][20347] Updated weights for policy 0, policy_version 1119581 (0.0008) [2023-12-27 13:24:22,160][20381] Updated weights for policy 1, policy_version 1118458 (0.0007) [2023-12-27 13:24:22,216][20381] Updated weights for policy 1, policy_version 1118468 (0.0007) [2023-12-27 13:24:22,275][20381] Updated weights for policy 1, policy_version 1118478 (0.0007) [2023-12-27 13:24:22,339][20381] Updated weights for policy 1, policy_version 1118488 (0.0007) [2023-12-27 13:24:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 573022208. Throughput: 0: 10101.8, 1: 10323.2. Samples: 572992780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:22,363][19170] Avg episode reward: [(0, '980.520'), (1, '735.260')] [2023-12-27 13:24:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001118488_286359552.pth... [2023-12-27 13:24:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001117272_286048256.pth [2023-12-27 13:24:22,396][20347] Updated weights for policy 0, policy_version 1119591 (0.0007) [2023-12-27 13:24:22,459][20347] Updated weights for policy 0, policy_version 1119601 (0.0007) [2023-12-27 13:24:22,524][20347] Updated weights for policy 0, policy_version 1119611 (0.0008) [2023-12-27 13:24:22,555][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001119616_286670848.pth... [2023-12-27 13:24:22,558][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001118432_286367744.pth [2023-12-27 13:24:23,105][20381] Updated weights for policy 1, policy_version 1118498 (0.0009) [2023-12-27 13:24:23,150][20381] Updated weights for policy 1, policy_version 1118508 (0.0009) [2023-12-27 13:24:23,203][20381] Updated weights for policy 1, policy_version 1118518 (0.0008) [2023-12-27 13:24:23,231][20347] Updated weights for policy 0, policy_version 1119621 (0.0008) [2023-12-27 13:24:23,280][20347] Updated weights for policy 0, policy_version 1119631 (0.0005) [2023-12-27 13:24:23,337][20347] Updated weights for policy 0, policy_version 1119641 (0.0005) [2023-12-27 13:24:23,861][20347] Updated weights for policy 0, policy_version 1119651 (0.0005) [2023-12-27 13:24:23,919][20347] Updated weights for policy 0, policy_version 1119661 (0.0005) [2023-12-27 13:24:23,969][20347] Updated weights for policy 0, policy_version 1119671 (0.0006) [2023-12-27 13:24:23,983][20381] Updated weights for policy 1, policy_version 1118528 (0.0007) [2023-12-27 13:24:24,032][20381] Updated weights for policy 1, policy_version 1118538 (0.0007) [2023-12-27 13:24:24,084][20381] Updated weights for policy 1, policy_version 1118548 (0.0008) [2023-12-27 13:24:24,584][20347] Updated weights for policy 0, policy_version 1119681 (0.0006) [2023-12-27 13:24:24,643][20347] Updated weights for policy 0, policy_version 1119691 (0.0010) [2023-12-27 13:24:24,704][20347] Updated weights for policy 0, policy_version 1119701 (0.0010) [2023-12-27 13:24:24,730][20381] Updated weights for policy 1, policy_version 1118558 (0.0008) [2023-12-27 13:24:24,756][20347] Updated weights for policy 0, policy_version 1119711 (0.0007) [2023-12-27 13:24:24,786][20381] Updated weights for policy 1, policy_version 1118568 (0.0007) [2023-12-27 13:24:24,850][20381] Updated weights for policy 1, policy_version 1118578 (0.0006) [2023-12-27 13:24:25,419][20347] Updated weights for policy 0, policy_version 1119721 (0.0010) [2023-12-27 13:24:25,469][20381] Updated weights for policy 1, policy_version 1118588 (0.0008) [2023-12-27 13:24:25,475][20347] Updated weights for policy 0, policy_version 1119731 (0.0007) [2023-12-27 13:24:25,524][20381] Updated weights for policy 1, policy_version 1118598 (0.0006) [2023-12-27 13:24:25,529][20347] Updated weights for policy 0, policy_version 1119741 (0.0005) [2023-12-27 13:24:25,571][20381] Updated weights for policy 1, policy_version 1118608 (0.0007) [2023-12-27 13:24:26,228][20347] Updated weights for policy 0, policy_version 1119751 (0.0007) [2023-12-27 13:24:26,234][20381] Updated weights for policy 1, policy_version 1118618 (0.0008) [2023-12-27 13:24:26,282][20381] Updated weights for policy 1, policy_version 1118628 (0.0007) [2023-12-27 13:24:26,292][20347] Updated weights for policy 0, policy_version 1119761 (0.0007) [2023-12-27 13:24:26,332][20381] Updated weights for policy 1, policy_version 1118638 (0.0006) [2023-12-27 13:24:26,347][20347] Updated weights for policy 0, policy_version 1119771 (0.0005) [2023-12-27 13:24:26,384][20381] Updated weights for policy 1, policy_version 1118648 (0.0008) [2023-12-27 13:24:26,933][20347] Updated weights for policy 0, policy_version 1119781 (0.0005) [2023-12-27 13:24:26,987][20347] Updated weights for policy 0, policy_version 1119791 (0.0006) [2023-12-27 13:24:27,040][20347] Updated weights for policy 0, policy_version 1119801 (0.0007) [2023-12-27 13:24:27,120][20381] Updated weights for policy 1, policy_version 1118658 (0.0010) [2023-12-27 13:24:27,171][20381] Updated weights for policy 1, policy_version 1118668 (0.0010) [2023-12-27 13:24:27,225][20381] Updated weights for policy 1, policy_version 1118678 (0.0010) [2023-12-27 13:24:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 573128704. Throughput: 0: 10097.8, 1: 10310.6. Samples: 573114696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '666.520')] [2023-12-27 13:24:27,761][20347] Updated weights for policy 0, policy_version 1119811 (0.0008) [2023-12-27 13:24:27,790][20381] Updated weights for policy 1, policy_version 1118688 (0.0007) [2023-12-27 13:24:27,819][20347] Updated weights for policy 0, policy_version 1119821 (0.0005) [2023-12-27 13:24:27,852][20381] Updated weights for policy 1, policy_version 1118698 (0.0005) [2023-12-27 13:24:27,881][20347] Updated weights for policy 0, policy_version 1119831 (0.0008) [2023-12-27 13:24:27,899][20381] Updated weights for policy 1, policy_version 1118708 (0.0005) [2023-12-27 13:24:28,429][20381] Updated weights for policy 1, policy_version 1118718 (0.0007) [2023-12-27 13:24:28,480][20381] Updated weights for policy 1, policy_version 1118728 (0.0007) [2023-12-27 13:24:28,509][20347] Updated weights for policy 0, policy_version 1119841 (0.0007) [2023-12-27 13:24:28,533][20381] Updated weights for policy 1, policy_version 1118738 (0.0007) [2023-12-27 13:24:28,567][20347] Updated weights for policy 0, policy_version 1119851 (0.0007) [2023-12-27 13:24:28,616][20347] Updated weights for policy 0, policy_version 1119861 (0.0008) [2023-12-27 13:24:28,665][20347] Updated weights for policy 0, policy_version 1119871 (0.0009) [2023-12-27 13:24:29,131][20381] Updated weights for policy 1, policy_version 1118748 (0.0007) [2023-12-27 13:24:29,186][20381] Updated weights for policy 1, policy_version 1118758 (0.0008) [2023-12-27 13:24:29,248][20381] Updated weights for policy 1, policy_version 1118768 (0.0008) [2023-12-27 13:24:29,394][20347] Updated weights for policy 0, policy_version 1119881 (0.0008) [2023-12-27 13:24:29,451][20347] Updated weights for policy 0, policy_version 1119891 (0.0008) [2023-12-27 13:24:29,505][20347] Updated weights for policy 0, policy_version 1119901 (0.0007) [2023-12-27 13:24:30,007][20381] Updated weights for policy 1, policy_version 1118778 (0.0007) [2023-12-27 13:24:30,067][20381] Updated weights for policy 1, policy_version 1118788 (0.0008) [2023-12-27 13:24:30,123][20381] Updated weights for policy 1, policy_version 1118798 (0.0008) [2023-12-27 13:24:30,176][20381] Updated weights for policy 1, policy_version 1118808 (0.0008) [2023-12-27 13:24:30,202][20347] Updated weights for policy 0, policy_version 1119911 (0.0007) [2023-12-27 13:24:30,254][20347] Updated weights for policy 0, policy_version 1119921 (0.0008) [2023-12-27 13:24:30,303][20347] Updated weights for policy 0, policy_version 1119931 (0.0007) [2023-12-27 13:24:30,755][20381] Updated weights for policy 1, policy_version 1118818 (0.0008) [2023-12-27 13:24:30,808][20381] Updated weights for policy 1, policy_version 1118828 (0.0007) [2023-12-27 13:24:30,856][20381] Updated weights for policy 1, policy_version 1118838 (0.0005) [2023-12-27 13:24:31,030][20347] Updated weights for policy 0, policy_version 1119941 (0.0008) [2023-12-27 13:24:31,087][20347] Updated weights for policy 0, policy_version 1119951 (0.0007) [2023-12-27 13:24:31,145][20347] Updated weights for policy 0, policy_version 1119961 (0.0008) [2023-12-27 13:24:31,520][20381] Updated weights for policy 1, policy_version 1118848 (0.0007) [2023-12-27 13:24:31,575][20381] Updated weights for policy 1, policy_version 1118858 (0.0009) [2023-12-27 13:24:31,638][20381] Updated weights for policy 1, policy_version 1118868 (0.0008) [2023-12-27 13:24:31,801][20347] Updated weights for policy 0, policy_version 1119971 (0.0007) [2023-12-27 13:24:31,861][20347] Updated weights for policy 0, policy_version 1119981 (0.0007) [2023-12-27 13:24:31,919][20347] Updated weights for policy 0, policy_version 1119991 (0.0006) [2023-12-27 13:24:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 573227008. Throughput: 0: 10097.9, 1: 10357.6. Samples: 573240124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '666.550')] [2023-12-27 13:24:32,382][20381] Updated weights for policy 1, policy_version 1118878 (0.0008) [2023-12-27 13:24:32,434][20381] Updated weights for policy 1, policy_version 1118888 (0.0008) [2023-12-27 13:24:32,486][20381] Updated weights for policy 1, policy_version 1118898 (0.0008) [2023-12-27 13:24:32,590][20347] Updated weights for policy 0, policy_version 1120001 (0.0007) [2023-12-27 13:24:32,654][20347] Updated weights for policy 0, policy_version 1120011 (0.0007) [2023-12-27 13:24:32,714][20347] Updated weights for policy 0, policy_version 1120021 (0.0007) [2023-12-27 13:24:32,782][20347] Updated weights for policy 0, policy_version 1120031 (0.0006) [2023-12-27 13:24:33,180][20381] Updated weights for policy 1, policy_version 1118908 (0.0008) [2023-12-27 13:24:33,233][20381] Updated weights for policy 1, policy_version 1118918 (0.0008) [2023-12-27 13:24:33,292][20381] Updated weights for policy 1, policy_version 1118928 (0.0005) [2023-12-27 13:24:33,345][20347] Updated weights for policy 0, policy_version 1120041 (0.0005) [2023-12-27 13:24:33,391][20347] Updated weights for policy 0, policy_version 1120051 (0.0008) [2023-12-27 13:24:33,440][20347] Updated weights for policy 0, policy_version 1120061 (0.0010) [2023-12-27 13:24:33,837][20381] Updated weights for policy 1, policy_version 1118938 (0.0005) [2023-12-27 13:24:33,890][20381] Updated weights for policy 1, policy_version 1118948 (0.0008) [2023-12-27 13:24:33,936][20381] Updated weights for policy 1, policy_version 1118958 (0.0008) [2023-12-27 13:24:33,982][20381] Updated weights for policy 1, policy_version 1118968 (0.0008) [2023-12-27 13:24:34,039][20347] Updated weights for policy 0, policy_version 1120071 (0.0007) [2023-12-27 13:24:34,088][20347] Updated weights for policy 0, policy_version 1120081 (0.0007) [2023-12-27 13:24:34,138][20347] Updated weights for policy 0, policy_version 1120091 (0.0006) [2023-12-27 13:24:34,733][20381] Updated weights for policy 1, policy_version 1118978 (0.0007) [2023-12-27 13:24:34,786][20381] Updated weights for policy 1, policy_version 1118988 (0.0007) [2023-12-27 13:24:34,787][20347] Updated weights for policy 0, policy_version 1120101 (0.0007) [2023-12-27 13:24:34,833][20347] Updated weights for policy 0, policy_version 1120111 (0.0005) [2023-12-27 13:24:34,835][20381] Updated weights for policy 1, policy_version 1118998 (0.0006) [2023-12-27 13:24:34,882][20347] Updated weights for policy 0, policy_version 1120121 (0.0007) [2023-12-27 13:24:35,540][20381] Updated weights for policy 1, policy_version 1119008 (0.0006) [2023-12-27 13:24:35,577][20347] Updated weights for policy 0, policy_version 1120131 (0.0007) [2023-12-27 13:24:35,592][20381] Updated weights for policy 1, policy_version 1119018 (0.0007) [2023-12-27 13:24:35,627][20347] Updated weights for policy 0, policy_version 1120141 (0.0005) [2023-12-27 13:24:35,643][20381] Updated weights for policy 1, policy_version 1119028 (0.0008) [2023-12-27 13:24:35,676][20347] Updated weights for policy 0, policy_version 1120151 (0.0005) [2023-12-27 13:24:36,260][20347] Updated weights for policy 0, policy_version 1120161 (0.0005) [2023-12-27 13:24:36,324][20347] Updated weights for policy 0, policy_version 1120171 (0.0007) [2023-12-27 13:24:36,377][20381] Updated weights for policy 1, policy_version 1119038 (0.0006) [2023-12-27 13:24:36,386][20347] Updated weights for policy 0, policy_version 1120181 (0.0006) [2023-12-27 13:24:36,438][20381] Updated weights for policy 1, policy_version 1119048 (0.0006) [2023-12-27 13:24:36,448][20347] Updated weights for policy 0, policy_version 1120191 (0.0006) [2023-12-27 13:24:36,504][20381] Updated weights for policy 1, policy_version 1119058 (0.0007) [2023-12-27 13:24:37,126][20347] Updated weights for policy 0, policy_version 1120201 (0.0007) [2023-12-27 13:24:37,183][20347] Updated weights for policy 0, policy_version 1120211 (0.0007) [2023-12-27 13:24:37,189][20381] Updated weights for policy 1, policy_version 1119068 (0.0007) [2023-12-27 13:24:37,244][20347] Updated weights for policy 0, policy_version 1120221 (0.0008) [2023-12-27 13:24:37,249][20381] Updated weights for policy 1, policy_version 1119078 (0.0005) [2023-12-27 13:24:37,306][20381] Updated weights for policy 1, policy_version 1119088 (0.0005) [2023-12-27 13:24:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 573341696. Throughput: 0: 10162.0, 1: 10362.7. Samples: 573303964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:37,363][19170] Avg episode reward: [(0, '990.260'), (1, '646.970')] [2023-12-27 13:24:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001119096_286515200.pth... [2023-12-27 13:24:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001120224_286826496.pth... [2023-12-27 13:24:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001117880_286203904.pth [2023-12-27 13:24:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001119008_286515200.pth [2023-12-27 13:24:37,902][20347] Updated weights for policy 0, policy_version 1120231 (0.0006) [2023-12-27 13:24:37,955][20381] Updated weights for policy 1, policy_version 1119098 (0.0006) [2023-12-27 13:24:37,961][20347] Updated weights for policy 0, policy_version 1120241 (0.0005) [2023-12-27 13:24:38,000][20381] Updated weights for policy 1, policy_version 1119108 (0.0006) [2023-12-27 13:24:38,013][20347] Updated weights for policy 0, policy_version 1120251 (0.0005) [2023-12-27 13:24:38,057][20381] Updated weights for policy 1, policy_version 1119118 (0.0005) [2023-12-27 13:24:38,110][20381] Updated weights for policy 1, policy_version 1119128 (0.0007) [2023-12-27 13:24:38,665][20347] Updated weights for policy 0, policy_version 1120261 (0.0006) [2023-12-27 13:24:38,720][20347] Updated weights for policy 0, policy_version 1120271 (0.0006) [2023-12-27 13:24:38,777][20347] Updated weights for policy 0, policy_version 1120281 (0.0007) [2023-12-27 13:24:38,835][20381] Updated weights for policy 1, policy_version 1119138 (0.0011) [2023-12-27 13:24:38,901][20381] Updated weights for policy 1, policy_version 1119148 (0.0010) [2023-12-27 13:24:38,949][20381] Updated weights for policy 1, policy_version 1119158 (0.0010) [2023-12-27 13:24:39,484][20347] Updated weights for policy 0, policy_version 1120291 (0.0006) [2023-12-27 13:24:39,552][20347] Updated weights for policy 0, policy_version 1120301 (0.0006) [2023-12-27 13:24:39,619][20347] Updated weights for policy 0, policy_version 1120311 (0.0006) [2023-12-27 13:24:39,639][20381] Updated weights for policy 1, policy_version 1119168 (0.0008) [2023-12-27 13:24:39,693][20381] Updated weights for policy 1, policy_version 1119178 (0.0007) [2023-12-27 13:24:39,746][20381] Updated weights for policy 1, policy_version 1119188 (0.0005) [2023-12-27 13:24:40,256][20347] Updated weights for policy 0, policy_version 1120321 (0.0006) [2023-12-27 13:24:40,313][20347] Updated weights for policy 0, policy_version 1120331 (0.0007) [2023-12-27 13:24:40,373][20347] Updated weights for policy 0, policy_version 1120341 (0.0008) [2023-12-27 13:24:40,431][20347] Updated weights for policy 0, policy_version 1120351 (0.0007) [2023-12-27 13:24:40,455][20381] Updated weights for policy 1, policy_version 1119198 (0.0007) [2023-12-27 13:24:40,516][20381] Updated weights for policy 1, policy_version 1119208 (0.0007) [2023-12-27 13:24:40,567][20381] Updated weights for policy 1, policy_version 1119218 (0.0008) [2023-12-27 13:24:41,070][20347] Updated weights for policy 0, policy_version 1120361 (0.0008) [2023-12-27 13:24:41,131][20347] Updated weights for policy 0, policy_version 1120371 (0.0006) [2023-12-27 13:24:41,200][20347] Updated weights for policy 0, policy_version 1120381 (0.0008) [2023-12-27 13:24:41,318][20381] Updated weights for policy 1, policy_version 1119228 (0.0008) [2023-12-27 13:24:41,391][20381] Updated weights for policy 1, policy_version 1119238 (0.0008) [2023-12-27 13:24:41,465][20381] Updated weights for policy 1, policy_version 1119248 (0.0008) [2023-12-27 13:24:41,925][20347] Updated weights for policy 0, policy_version 1120391 (0.0007) [2023-12-27 13:24:41,985][20347] Updated weights for policy 0, policy_version 1120401 (0.0007) [2023-12-27 13:24:42,048][20347] Updated weights for policy 0, policy_version 1120411 (0.0008) [2023-12-27 13:24:42,191][20381] Updated weights for policy 1, policy_version 1119258 (0.0008) [2023-12-27 13:24:42,251][20381] Updated weights for policy 1, policy_version 1119268 (0.0007) [2023-12-27 13:24:42,318][20381] Updated weights for policy 1, policy_version 1119278 (0.0007) [2023-12-27 13:24:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 573431808. Throughput: 0: 10240.8, 1: 10330.6. Samples: 573426824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:42,363][19170] Avg episode reward: [(0, '990.260'), (1, '617.230')] [2023-12-27 13:24:42,380][20381] Updated weights for policy 1, policy_version 1119288 (0.0009) [2023-12-27 13:24:42,762][20347] Updated weights for policy 0, policy_version 1120421 (0.0008) [2023-12-27 13:24:42,813][20347] Updated weights for policy 0, policy_version 1120431 (0.0006) [2023-12-27 13:24:42,864][20347] Updated weights for policy 0, policy_version 1120441 (0.0005) [2023-12-27 13:24:43,075][20381] Updated weights for policy 1, policy_version 1119298 (0.0007) [2023-12-27 13:24:43,127][20381] Updated weights for policy 1, policy_version 1119308 (0.0007) [2023-12-27 13:24:43,184][20381] Updated weights for policy 1, policy_version 1119318 (0.0009) [2023-12-27 13:24:43,481][20347] Updated weights for policy 0, policy_version 1120451 (0.0007) [2023-12-27 13:24:43,528][20347] Updated weights for policy 0, policy_version 1120461 (0.0010) [2023-12-27 13:24:43,585][20347] Updated weights for policy 0, policy_version 1120471 (0.0010) [2023-12-27 13:24:43,877][20381] Updated weights for policy 1, policy_version 1119328 (0.0010) [2023-12-27 13:24:43,926][20381] Updated weights for policy 1, policy_version 1119338 (0.0010) [2023-12-27 13:24:43,970][20381] Updated weights for policy 1, policy_version 1119348 (0.0010) [2023-12-27 13:24:44,230][20347] Updated weights for policy 0, policy_version 1120481 (0.0010) [2023-12-27 13:24:44,293][20347] Updated weights for policy 0, policy_version 1120491 (0.0005) [2023-12-27 13:24:44,357][20347] Updated weights for policy 0, policy_version 1120501 (0.0006) [2023-12-27 13:24:44,419][20347] Updated weights for policy 0, policy_version 1120511 (0.0010) [2023-12-27 13:24:44,703][20381] Updated weights for policy 1, policy_version 1119358 (0.0010) [2023-12-27 13:24:44,748][20381] Updated weights for policy 1, policy_version 1119368 (0.0010) [2023-12-27 13:24:44,812][20381] Updated weights for policy 1, policy_version 1119378 (0.0008) [2023-12-27 13:24:45,014][20347] Updated weights for policy 0, policy_version 1120521 (0.0008) [2023-12-27 13:24:45,069][20347] Updated weights for policy 0, policy_version 1120531 (0.0008) [2023-12-27 13:24:45,121][20347] Updated weights for policy 0, policy_version 1120541 (0.0007) [2023-12-27 13:24:45,468][20381] Updated weights for policy 1, policy_version 1119388 (0.0006) [2023-12-27 13:24:45,524][20381] Updated weights for policy 1, policy_version 1119398 (0.0006) [2023-12-27 13:24:45,580][20381] Updated weights for policy 1, policy_version 1119408 (0.0006) [2023-12-27 13:24:45,731][20347] Updated weights for policy 0, policy_version 1120551 (0.0007) [2023-12-27 13:24:45,786][20347] Updated weights for policy 0, policy_version 1120561 (0.0006) [2023-12-27 13:24:45,845][20347] Updated weights for policy 0, policy_version 1120571 (0.0005) [2023-12-27 13:24:46,117][20381] Updated weights for policy 1, policy_version 1119418 (0.0009) [2023-12-27 13:24:46,164][20381] Updated weights for policy 1, policy_version 1119428 (0.0005) [2023-12-27 13:24:46,227][20381] Updated weights for policy 1, policy_version 1119438 (0.0005) [2023-12-27 13:24:46,272][20381] Updated weights for policy 1, policy_version 1119448 (0.0005) [2023-12-27 13:24:46,525][20347] Updated weights for policy 0, policy_version 1120581 (0.0007) [2023-12-27 13:24:46,572][20347] Updated weights for policy 0, policy_version 1120591 (0.0006) [2023-12-27 13:24:46,627][20347] Updated weights for policy 0, policy_version 1120601 (0.0006) [2023-12-27 13:24:46,875][20381] Updated weights for policy 1, policy_version 1119458 (0.0005) [2023-12-27 13:24:46,931][20381] Updated weights for policy 1, policy_version 1119468 (0.0006) [2023-12-27 13:24:47,004][20381] Updated weights for policy 1, policy_version 1119478 (0.0006) [2023-12-27 13:24:47,234][20347] Updated weights for policy 0, policy_version 1120611 (0.0007) [2023-12-27 13:24:47,287][20347] Updated weights for policy 0, policy_version 1120621 (0.0007) [2023-12-27 13:24:47,343][20347] Updated weights for policy 0, policy_version 1120631 (0.0007) [2023-12-27 13:24:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 573538304. Throughput: 0: 10311.7, 1: 10339.7. Samples: 573552072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '666.860')] [2023-12-27 13:24:47,704][20381] Updated weights for policy 1, policy_version 1119488 (0.0006) [2023-12-27 13:24:47,767][20381] Updated weights for policy 1, policy_version 1119498 (0.0005) [2023-12-27 13:24:47,828][20381] Updated weights for policy 1, policy_version 1119508 (0.0007) [2023-12-27 13:24:48,041][20347] Updated weights for policy 0, policy_version 1120641 (0.0008) [2023-12-27 13:24:48,094][20347] Updated weights for policy 0, policy_version 1120651 (0.0007) [2023-12-27 13:24:48,142][20347] Updated weights for policy 0, policy_version 1120661 (0.0007) [2023-12-27 13:24:48,199][20347] Updated weights for policy 0, policy_version 1120671 (0.0007) [2023-12-27 13:24:48,401][20381] Updated weights for policy 1, policy_version 1119518 (0.0007) [2023-12-27 13:24:48,460][20381] Updated weights for policy 1, policy_version 1119528 (0.0007) [2023-12-27 13:24:48,514][20381] Updated weights for policy 1, policy_version 1119538 (0.0007) [2023-12-27 13:24:48,965][20347] Updated weights for policy 0, policy_version 1120681 (0.0008) [2023-12-27 13:24:49,020][20347] Updated weights for policy 0, policy_version 1120691 (0.0007) [2023-12-27 13:24:49,071][20347] Updated weights for policy 0, policy_version 1120701 (0.0006) [2023-12-27 13:24:49,224][20381] Updated weights for policy 1, policy_version 1119548 (0.0008) [2023-12-27 13:24:49,281][20381] Updated weights for policy 1, policy_version 1119558 (0.0007) [2023-12-27 13:24:49,342][20381] Updated weights for policy 1, policy_version 1119568 (0.0007) [2023-12-27 13:24:49,737][20347] Updated weights for policy 0, policy_version 1120711 (0.0007) [2023-12-27 13:24:49,796][20347] Updated weights for policy 0, policy_version 1120721 (0.0006) [2023-12-27 13:24:49,863][20347] Updated weights for policy 0, policy_version 1120731 (0.0007) [2023-12-27 13:24:50,052][20381] Updated weights for policy 1, policy_version 1119578 (0.0009) [2023-12-27 13:24:50,114][20381] Updated weights for policy 1, policy_version 1119588 (0.0009) [2023-12-27 13:24:50,179][20381] Updated weights for policy 1, policy_version 1119598 (0.0007) [2023-12-27 13:24:50,244][20381] Updated weights for policy 1, policy_version 1119608 (0.0006) [2023-12-27 13:24:50,525][20347] Updated weights for policy 0, policy_version 1120741 (0.0009) [2023-12-27 13:24:50,585][20347] Updated weights for policy 0, policy_version 1120751 (0.0008) [2023-12-27 13:24:50,639][20347] Updated weights for policy 0, policy_version 1120761 (0.0007) [2023-12-27 13:24:50,851][20381] Updated weights for policy 1, policy_version 1119618 (0.0006) [2023-12-27 13:24:50,912][20381] Updated weights for policy 1, policy_version 1119628 (0.0007) [2023-12-27 13:24:50,971][20381] Updated weights for policy 1, policy_version 1119638 (0.0008) [2023-12-27 13:24:51,379][20347] Updated weights for policy 0, policy_version 1120771 (0.0007) [2023-12-27 13:24:51,435][20347] Updated weights for policy 0, policy_version 1120781 (0.0007) [2023-12-27 13:24:51,487][20347] Updated weights for policy 0, policy_version 1120791 (0.0007) [2023-12-27 13:24:51,697][20381] Updated weights for policy 1, policy_version 1119648 (0.0008) [2023-12-27 13:24:51,759][20381] Updated weights for policy 1, policy_version 1119658 (0.0007) [2023-12-27 13:24:51,820][20381] Updated weights for policy 1, policy_version 1119668 (0.0005) [2023-12-27 13:24:52,225][20347] Updated weights for policy 0, policy_version 1120801 (0.0008) [2023-12-27 13:24:52,289][20347] Updated weights for policy 0, policy_version 1120811 (0.0009) [2023-12-27 13:24:52,355][20347] Updated weights for policy 0, policy_version 1120821 (0.0008) [2023-12-27 13:24:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 573636608. Throughput: 0: 10312.4, 1: 10331.7. Samples: 573612328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:52,363][19170] Avg episode reward: [(0, '990.360'), (1, '637.290')] [2023-12-27 13:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001119672_286662656.pth... [2023-12-27 13:24:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001118488_286359552.pth [2023-12-27 13:24:52,421][20347] Updated weights for policy 0, policy_version 1120831 (0.0007) [2023-12-27 13:24:52,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001120832_286982144.pth... [2023-12-27 13:24:52,426][20381] Updated weights for policy 1, policy_version 1119678 (0.0006) [2023-12-27 13:24:52,431][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001119616_286670848.pth [2023-12-27 13:24:52,485][20381] Updated weights for policy 1, policy_version 1119688 (0.0007) [2023-12-27 13:24:52,542][20381] Updated weights for policy 1, policy_version 1119698 (0.0008) [2023-12-27 13:24:53,079][20347] Updated weights for policy 0, policy_version 1120841 (0.0005) [2023-12-27 13:24:53,130][20347] Updated weights for policy 0, policy_version 1120851 (0.0006) [2023-12-27 13:24:53,187][20347] Updated weights for policy 0, policy_version 1120861 (0.0006) [2023-12-27 13:24:53,290][20381] Updated weights for policy 1, policy_version 1119708 (0.0009) [2023-12-27 13:24:53,347][20381] Updated weights for policy 1, policy_version 1119718 (0.0006) [2023-12-27 13:24:53,397][20381] Updated weights for policy 1, policy_version 1119728 (0.0008) [2023-12-27 13:24:53,787][20347] Updated weights for policy 0, policy_version 1120871 (0.0009) [2023-12-27 13:24:53,841][20347] Updated weights for policy 0, policy_version 1120881 (0.0011) [2023-12-27 13:24:53,896][20347] Updated weights for policy 0, policy_version 1120891 (0.0010) [2023-12-27 13:24:54,164][20381] Updated weights for policy 1, policy_version 1119738 (0.0008) [2023-12-27 13:24:54,226][20381] Updated weights for policy 1, policy_version 1119748 (0.0008) [2023-12-27 13:24:54,282][20381] Updated weights for policy 1, policy_version 1119758 (0.0007) [2023-12-27 13:24:54,331][20381] Updated weights for policy 1, policy_version 1119768 (0.0006) [2023-12-27 13:24:54,527][20347] Updated weights for policy 0, policy_version 1120901 (0.0008) [2023-12-27 13:24:54,586][20347] Updated weights for policy 0, policy_version 1120911 (0.0010) [2023-12-27 13:24:54,636][20347] Updated weights for policy 0, policy_version 1120921 (0.0010) [2023-12-27 13:24:55,027][20381] Updated weights for policy 1, policy_version 1119778 (0.0008) [2023-12-27 13:24:55,089][20381] Updated weights for policy 1, policy_version 1119788 (0.0007) [2023-12-27 13:24:55,150][20381] Updated weights for policy 1, policy_version 1119798 (0.0007) [2023-12-27 13:24:55,319][20347] Updated weights for policy 0, policy_version 1120931 (0.0010) [2023-12-27 13:24:55,367][20347] Updated weights for policy 0, policy_version 1120941 (0.0010) [2023-12-27 13:24:55,414][20347] Updated weights for policy 0, policy_version 1120951 (0.0010) [2023-12-27 13:24:55,838][20381] Updated weights for policy 1, policy_version 1119808 (0.0007) [2023-12-27 13:24:55,893][20381] Updated weights for policy 1, policy_version 1119818 (0.0008) [2023-12-27 13:24:55,945][20381] Updated weights for policy 1, policy_version 1119828 (0.0007) [2023-12-27 13:24:56,075][20347] Updated weights for policy 0, policy_version 1120961 (0.0010) [2023-12-27 13:24:56,125][20347] Updated weights for policy 0, policy_version 1120971 (0.0010) [2023-12-27 13:24:56,177][20347] Updated weights for policy 0, policy_version 1120981 (0.0010) [2023-12-27 13:24:56,228][20347] Updated weights for policy 0, policy_version 1120991 (0.0010) [2023-12-27 13:24:56,539][20381] Updated weights for policy 1, policy_version 1119838 (0.0007) [2023-12-27 13:24:56,592][20381] Updated weights for policy 1, policy_version 1119848 (0.0008) [2023-12-27 13:24:56,644][20381] Updated weights for policy 1, policy_version 1119858 (0.0009) [2023-12-27 13:24:56,841][20347] Updated weights for policy 0, policy_version 1121001 (0.0006) [2023-12-27 13:24:56,897][20347] Updated weights for policy 0, policy_version 1121011 (0.0008) [2023-12-27 13:24:56,952][20347] Updated weights for policy 0, policy_version 1121021 (0.0008) [2023-12-27 13:24:57,337][20381] Updated weights for policy 1, policy_version 1119868 (0.0009) [2023-12-27 13:24:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 573743104. Throughput: 0: 10281.8, 1: 10300.9. Samples: 573734556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:24:57,363][19170] Avg episode reward: [(0, '990.360'), (1, '686.390')] [2023-12-27 13:24:57,392][20381] Updated weights for policy 1, policy_version 1119878 (0.0010) [2023-12-27 13:24:57,440][20381] Updated weights for policy 1, policy_version 1119888 (0.0010) [2023-12-27 13:24:57,462][20347] Updated weights for policy 0, policy_version 1121031 (0.0005) [2023-12-27 13:24:57,516][20347] Updated weights for policy 0, policy_version 1121041 (0.0005) [2023-12-27 13:24:57,576][20347] Updated weights for policy 0, policy_version 1121051 (0.0005) [2023-12-27 13:24:58,119][20381] Updated weights for policy 1, policy_version 1119898 (0.0010) [2023-12-27 13:24:58,121][20347] Updated weights for policy 0, policy_version 1121061 (0.0005) [2023-12-27 13:24:58,164][20381] Updated weights for policy 1, policy_version 1119908 (0.0010) [2023-12-27 13:24:58,176][20347] Updated weights for policy 0, policy_version 1121071 (0.0005) [2023-12-27 13:24:58,221][20381] Updated weights for policy 1, policy_version 1119918 (0.0009) [2023-12-27 13:24:58,240][20347] Updated weights for policy 0, policy_version 1121081 (0.0010) [2023-12-27 13:24:58,278][20381] Updated weights for policy 1, policy_version 1119928 (0.0007) [2023-12-27 13:24:58,979][20347] Updated weights for policy 0, policy_version 1121091 (0.0009) [2023-12-27 13:24:59,043][20347] Updated weights for policy 0, policy_version 1121101 (0.0006) [2023-12-27 13:24:59,077][20381] Updated weights for policy 1, policy_version 1119938 (0.0008) [2023-12-27 13:24:59,102][20347] Updated weights for policy 0, policy_version 1121111 (0.0008) [2023-12-27 13:24:59,132][20381] Updated weights for policy 1, policy_version 1119948 (0.0007) [2023-12-27 13:24:59,189][20381] Updated weights for policy 1, policy_version 1119958 (0.0008) [2023-12-27 13:24:59,788][20347] Updated weights for policy 0, policy_version 1121121 (0.0010) [2023-12-27 13:24:59,857][20347] Updated weights for policy 0, policy_version 1121131 (0.0007) [2023-12-27 13:24:59,906][20381] Updated weights for policy 1, policy_version 1119968 (0.0008) [2023-12-27 13:24:59,908][20347] Updated weights for policy 0, policy_version 1121141 (0.0007) [2023-12-27 13:24:59,970][20347] Updated weights for policy 0, policy_version 1121151 (0.0006) [2023-12-27 13:24:59,971][20381] Updated weights for policy 1, policy_version 1119978 (0.0008) [2023-12-27 13:25:00,023][20381] Updated weights for policy 1, policy_version 1119988 (0.0009) [2023-12-27 13:25:00,544][20347] Updated weights for policy 0, policy_version 1121161 (0.0005) [2023-12-27 13:25:00,589][20347] Updated weights for policy 0, policy_version 1121171 (0.0005) [2023-12-27 13:25:00,644][20347] Updated weights for policy 0, policy_version 1121181 (0.0005) [2023-12-27 13:25:00,753][20381] Updated weights for policy 1, policy_version 1119998 (0.0009) [2023-12-27 13:25:00,803][20381] Updated weights for policy 1, policy_version 1120008 (0.0009) [2023-12-27 13:25:00,852][20381] Updated weights for policy 1, policy_version 1120018 (0.0009) [2023-12-27 13:25:01,178][20347] Updated weights for policy 0, policy_version 1121191 (0.0007) [2023-12-27 13:25:01,236][20347] Updated weights for policy 0, policy_version 1121201 (0.0007) [2023-12-27 13:25:01,290][20347] Updated weights for policy 0, policy_version 1121211 (0.0008) [2023-12-27 13:25:01,631][20381] Updated weights for policy 1, policy_version 1120028 (0.0009) [2023-12-27 13:25:01,692][20381] Updated weights for policy 1, policy_version 1120038 (0.0008) [2023-12-27 13:25:01,752][20381] Updated weights for policy 1, policy_version 1120048 (0.0008) [2023-12-27 13:25:01,999][20347] Updated weights for policy 0, policy_version 1121221 (0.0006) [2023-12-27 13:25:02,046][20347] Updated weights for policy 0, policy_version 1121231 (0.0005) [2023-12-27 13:25:02,099][20347] Updated weights for policy 0, policy_version 1121241 (0.0006) [2023-12-27 13:25:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 573849600. Throughput: 0: 10466.6, 1: 10124.6. Samples: 573859676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '657.040')] [2023-12-27 13:25:02,507][20381] Updated weights for policy 1, policy_version 1120058 (0.0008) [2023-12-27 13:25:02,559][20381] Updated weights for policy 1, policy_version 1120068 (0.0008) [2023-12-27 13:25:02,616][20381] Updated weights for policy 1, policy_version 1120078 (0.0008) [2023-12-27 13:25:02,674][20381] Updated weights for policy 1, policy_version 1120088 (0.0007) [2023-12-27 13:25:02,737][20347] Updated weights for policy 0, policy_version 1121251 (0.0006) [2023-12-27 13:25:02,802][20347] Updated weights for policy 0, policy_version 1121261 (0.0005) [2023-12-27 13:25:02,869][20347] Updated weights for policy 0, policy_version 1121271 (0.0005) [2023-12-27 13:25:03,354][20381] Updated weights for policy 1, policy_version 1120098 (0.0009) [2023-12-27 13:25:03,400][20381] Updated weights for policy 1, policy_version 1120108 (0.0006) [2023-12-27 13:25:03,414][20347] Updated weights for policy 0, policy_version 1121281 (0.0007) [2023-12-27 13:25:03,451][20381] Updated weights for policy 1, policy_version 1120118 (0.0005) [2023-12-27 13:25:03,476][20347] Updated weights for policy 0, policy_version 1121291 (0.0005) [2023-12-27 13:25:03,526][20347] Updated weights for policy 0, policy_version 1121301 (0.0006) [2023-12-27 13:25:03,573][20347] Updated weights for policy 0, policy_version 1121311 (0.0007) [2023-12-27 13:25:04,171][20381] Updated weights for policy 1, policy_version 1120128 (0.0009) [2023-12-27 13:25:04,176][20347] Updated weights for policy 0, policy_version 1121321 (0.0010) [2023-12-27 13:25:04,222][20381] Updated weights for policy 1, policy_version 1120138 (0.0009) [2023-12-27 13:25:04,229][20347] Updated weights for policy 0, policy_version 1121331 (0.0010) [2023-12-27 13:25:04,278][20381] Updated weights for policy 1, policy_version 1120148 (0.0006) [2023-12-27 13:25:04,289][20347] Updated weights for policy 0, policy_version 1121341 (0.0010) [2023-12-27 13:25:04,952][20381] Updated weights for policy 1, policy_version 1120158 (0.0008) [2023-12-27 13:25:04,987][20347] Updated weights for policy 0, policy_version 1121351 (0.0010) [2023-12-27 13:25:05,001][20381] Updated weights for policy 1, policy_version 1120168 (0.0010) [2023-12-27 13:25:05,037][20347] Updated weights for policy 0, policy_version 1121361 (0.0010) [2023-12-27 13:25:05,054][20381] Updated weights for policy 1, policy_version 1120178 (0.0010) [2023-12-27 13:25:05,098][20347] Updated weights for policy 0, policy_version 1121371 (0.0009) [2023-12-27 13:25:05,675][20381] Updated weights for policy 1, policy_version 1120188 (0.0009) [2023-12-27 13:25:05,732][20381] Updated weights for policy 1, policy_version 1120198 (0.0009) [2023-12-27 13:25:05,784][20381] Updated weights for policy 1, policy_version 1120208 (0.0010) [2023-12-27 13:25:05,788][20347] Updated weights for policy 0, policy_version 1121381 (0.0009) [2023-12-27 13:25:05,845][20347] Updated weights for policy 0, policy_version 1121391 (0.0010) [2023-12-27 13:25:05,906][20347] Updated weights for policy 0, policy_version 1121401 (0.0010) [2023-12-27 13:25:06,470][20381] Updated weights for policy 1, policy_version 1120218 (0.0009) [2023-12-27 13:25:06,530][20381] Updated weights for policy 1, policy_version 1120228 (0.0008) [2023-12-27 13:25:06,579][20347] Updated weights for policy 0, policy_version 1121411 (0.0009) [2023-12-27 13:25:06,592][20381] Updated weights for policy 1, policy_version 1120238 (0.0009) [2023-12-27 13:25:06,643][20347] Updated weights for policy 0, policy_version 1121421 (0.0006) [2023-12-27 13:25:06,652][20381] Updated weights for policy 1, policy_version 1120248 (0.0006) [2023-12-27 13:25:06,706][20347] Updated weights for policy 0, policy_version 1121431 (0.0007) [2023-12-27 13:25:07,348][20347] Updated weights for policy 0, policy_version 1121441 (0.0008) [2023-12-27 13:25:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 573947904. Throughput: 0: 10536.8, 1: 10126.5. Samples: 573922628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:07,363][19170] Avg episode reward: [(0, '980.560'), (1, '637.330')] [2023-12-27 13:25:07,374][20381] Updated weights for policy 1, policy_version 1120258 (0.0007) [2023-12-27 13:25:07,413][20347] Updated weights for policy 0, policy_version 1121451 (0.0008) [2023-12-27 13:25:07,422][20381] Updated weights for policy 1, policy_version 1120268 (0.0006) [2023-12-27 13:25:07,473][20381] Updated weights for policy 1, policy_version 1120278 (0.0006) [2023-12-27 13:25:07,475][20347] Updated weights for policy 0, policy_version 1121461 (0.0009) [2023-12-27 13:25:07,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001120280_286818304.pth... [2023-12-27 13:25:07,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001119096_286515200.pth [2023-12-27 13:25:07,523][20347] Updated weights for policy 0, policy_version 1121471 (0.0010) [2023-12-27 13:25:07,526][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001121472_287145984.pth... [2023-12-27 13:25:07,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001120224_286826496.pth [2023-12-27 13:25:08,042][20381] Updated weights for policy 1, policy_version 1120288 (0.0007) [2023-12-27 13:25:08,092][20381] Updated weights for policy 1, policy_version 1120298 (0.0008) [2023-12-27 13:25:08,151][20381] Updated weights for policy 1, policy_version 1120308 (0.0006) [2023-12-27 13:25:08,157][20347] Updated weights for policy 0, policy_version 1121481 (0.0010) [2023-12-27 13:25:08,217][20347] Updated weights for policy 0, policy_version 1121491 (0.0010) [2023-12-27 13:25:08,277][20347] Updated weights for policy 0, policy_version 1121502 (0.0010) [2023-12-27 13:25:08,851][20381] Updated weights for policy 1, policy_version 1120318 (0.0007) [2023-12-27 13:25:08,906][20381] Updated weights for policy 1, policy_version 1120328 (0.0010) [2023-12-27 13:25:08,964][20381] Updated weights for policy 1, policy_version 1120338 (0.0011) [2023-12-27 13:25:08,969][20347] Updated weights for policy 0, policy_version 1121512 (0.0010) [2023-12-27 13:25:09,027][20347] Updated weights for policy 0, policy_version 1121522 (0.0010) [2023-12-27 13:25:09,075][20347] Updated weights for policy 0, policy_version 1121532 (0.0010) [2023-12-27 13:25:09,708][20381] Updated weights for policy 1, policy_version 1120348 (0.0009) [2023-12-27 13:25:09,768][20381] Updated weights for policy 1, policy_version 1120358 (0.0006) [2023-12-27 13:25:09,793][20347] Updated weights for policy 0, policy_version 1121542 (0.0009) [2023-12-27 13:25:09,835][20381] Updated weights for policy 1, policy_version 1120368 (0.0006) [2023-12-27 13:25:09,862][20347] Updated weights for policy 0, policy_version 1121552 (0.0008) [2023-12-27 13:25:09,925][20347] Updated weights for policy 0, policy_version 1121562 (0.0007) [2023-12-27 13:25:10,431][20381] Updated weights for policy 1, policy_version 1120378 (0.0007) [2023-12-27 13:25:10,486][20381] Updated weights for policy 1, policy_version 1120388 (0.0007) [2023-12-27 13:25:10,534][20381] Updated weights for policy 1, policy_version 1120398 (0.0008) [2023-12-27 13:25:10,586][20381] Updated weights for policy 1, policy_version 1120408 (0.0008) [2023-12-27 13:25:10,696][20347] Updated weights for policy 0, policy_version 1121572 (0.0009) [2023-12-27 13:25:10,758][20347] Updated weights for policy 0, policy_version 1121582 (0.0010) [2023-12-27 13:25:10,818][20347] Updated weights for policy 0, policy_version 1121592 (0.0010) [2023-12-27 13:25:11,298][20381] Updated weights for policy 1, policy_version 1120418 (0.0009) [2023-12-27 13:25:11,359][20381] Updated weights for policy 1, policy_version 1120428 (0.0008) [2023-12-27 13:25:11,429][20381] Updated weights for policy 1, policy_version 1120438 (0.0011) [2023-12-27 13:25:11,589][20347] Updated weights for policy 0, policy_version 1121602 (0.0010) [2023-12-27 13:25:11,657][20347] Updated weights for policy 0, policy_version 1121612 (0.0009) [2023-12-27 13:25:11,724][20347] Updated weights for policy 0, policy_version 1121622 (0.0009) [2023-12-27 13:25:11,786][20347] Updated weights for policy 0, policy_version 1121632 (0.0009) [2023-12-27 13:25:12,179][20381] Updated weights for policy 1, policy_version 1120448 (0.0008) [2023-12-27 13:25:12,236][20381] Updated weights for policy 1, policy_version 1120458 (0.0008) [2023-12-27 13:25:12,299][20381] Updated weights for policy 1, policy_version 1120468 (0.0008) [2023-12-27 13:25:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 574054400. Throughput: 0: 10480.2, 1: 10201.4. Samples: 574045368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:12,363][19170] Avg episode reward: [(0, '990.230'), (1, '637.430')] [2023-12-27 13:25:12,497][20347] Updated weights for policy 0, policy_version 1121642 (0.0008) [2023-12-27 13:25:12,558][20347] Updated weights for policy 0, policy_version 1121652 (0.0007) [2023-12-27 13:25:12,622][20347] Updated weights for policy 0, policy_version 1121662 (0.0006) [2023-12-27 13:25:13,005][20381] Updated weights for policy 1, policy_version 1120478 (0.0008) [2023-12-27 13:25:13,062][20381] Updated weights for policy 1, policy_version 1120488 (0.0007) [2023-12-27 13:25:13,125][20381] Updated weights for policy 1, policy_version 1120498 (0.0006) [2023-12-27 13:25:13,246][20347] Updated weights for policy 0, policy_version 1121672 (0.0009) [2023-12-27 13:25:13,294][20347] Updated weights for policy 0, policy_version 1121682 (0.0010) [2023-12-27 13:25:13,342][20347] Updated weights for policy 0, policy_version 1121692 (0.0010) [2023-12-27 13:25:13,702][20381] Updated weights for policy 1, policy_version 1120508 (0.0006) [2023-12-27 13:25:13,763][20381] Updated weights for policy 1, policy_version 1120518 (0.0007) [2023-12-27 13:25:13,826][20381] Updated weights for policy 1, policy_version 1120528 (0.0006) [2023-12-27 13:25:13,970][20347] Updated weights for policy 0, policy_version 1121702 (0.0010) [2023-12-27 13:25:14,021][20347] Updated weights for policy 0, policy_version 1121712 (0.0010) [2023-12-27 13:25:14,069][20347] Updated weights for policy 0, policy_version 1121722 (0.0010) [2023-12-27 13:25:14,412][20381] Updated weights for policy 1, policy_version 1120538 (0.0005) [2023-12-27 13:25:14,466][20381] Updated weights for policy 1, policy_version 1120548 (0.0005) [2023-12-27 13:25:14,514][20381] Updated weights for policy 1, policy_version 1120558 (0.0007) [2023-12-27 13:25:14,567][20381] Updated weights for policy 1, policy_version 1120568 (0.0008) [2023-12-27 13:25:14,752][20347] Updated weights for policy 0, policy_version 1121732 (0.0009) [2023-12-27 13:25:14,824][20347] Updated weights for policy 0, policy_version 1121742 (0.0009) [2023-12-27 13:25:14,882][20347] Updated weights for policy 0, policy_version 1121752 (0.0010) [2023-12-27 13:25:15,247][20381] Updated weights for policy 1, policy_version 1120578 (0.0006) [2023-12-27 13:25:15,305][20381] Updated weights for policy 1, policy_version 1120588 (0.0006) [2023-12-27 13:25:15,354][20381] Updated weights for policy 1, policy_version 1120598 (0.0007) [2023-12-27 13:25:15,562][20347] Updated weights for policy 0, policy_version 1121762 (0.0009) [2023-12-27 13:25:15,616][20347] Updated weights for policy 0, policy_version 1121772 (0.0005) [2023-12-27 13:25:15,676][20347] Updated weights for policy 0, policy_version 1121782 (0.0006) [2023-12-27 13:25:15,735][20347] Updated weights for policy 0, policy_version 1121792 (0.0008) [2023-12-27 13:25:15,957][20381] Updated weights for policy 1, policy_version 1120608 (0.0006) [2023-12-27 13:25:16,008][20381] Updated weights for policy 1, policy_version 1120618 (0.0007) [2023-12-27 13:25:16,061][20381] Updated weights for policy 1, policy_version 1120628 (0.0007) [2023-12-27 13:25:16,332][20347] Updated weights for policy 0, policy_version 1121802 (0.0005) [2023-12-27 13:25:16,382][20347] Updated weights for policy 0, policy_version 1121812 (0.0005) [2023-12-27 13:25:16,435][20347] Updated weights for policy 0, policy_version 1121822 (0.0005) [2023-12-27 13:25:16,709][20381] Updated weights for policy 1, policy_version 1120638 (0.0010) [2023-12-27 13:25:16,767][20381] Updated weights for policy 1, policy_version 1120648 (0.0010) [2023-12-27 13:25:16,827][20381] Updated weights for policy 1, policy_version 1120658 (0.0010) [2023-12-27 13:25:17,071][20347] Updated weights for policy 0, policy_version 1121832 (0.0007) [2023-12-27 13:25:17,124][20347] Updated weights for policy 0, policy_version 1121842 (0.0007) [2023-12-27 13:25:17,175][20347] Updated weights for policy 0, policy_version 1121852 (0.0007) [2023-12-27 13:25:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 574160896. Throughput: 0: 10499.5, 1: 10169.0. Samples: 574170204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:17,363][19170] Avg episode reward: [(0, '990.400'), (1, '676.610')] [2023-12-27 13:25:17,523][20381] Updated weights for policy 1, policy_version 1120668 (0.0011) [2023-12-27 13:25:17,579][20381] Updated weights for policy 1, policy_version 1120678 (0.0011) [2023-12-27 13:25:17,635][20381] Updated weights for policy 1, policy_version 1120688 (0.0011) [2023-12-27 13:25:17,883][20347] Updated weights for policy 0, policy_version 1121862 (0.0007) [2023-12-27 13:25:17,941][20347] Updated weights for policy 0, policy_version 1121872 (0.0010) [2023-12-27 13:25:18,003][20347] Updated weights for policy 0, policy_version 1121882 (0.0010) [2023-12-27 13:25:18,288][20381] Updated weights for policy 1, policy_version 1120698 (0.0010) [2023-12-27 13:25:18,352][20381] Updated weights for policy 1, policy_version 1120708 (0.0008) [2023-12-27 13:25:18,417][20381] Updated weights for policy 1, policy_version 1120718 (0.0008) [2023-12-27 13:25:18,479][20381] Updated weights for policy 1, policy_version 1120728 (0.0007) [2023-12-27 13:25:18,669][20347] Updated weights for policy 0, policy_version 1121892 (0.0010) [2023-12-27 13:25:18,729][20347] Updated weights for policy 0, policy_version 1121902 (0.0011) [2023-12-27 13:25:18,799][20347] Updated weights for policy 0, policy_version 1121912 (0.0008) [2023-12-27 13:25:19,092][20381] Updated weights for policy 1, policy_version 1120738 (0.0006) [2023-12-27 13:25:19,147][20381] Updated weights for policy 1, policy_version 1120748 (0.0007) [2023-12-27 13:25:19,200][20381] Updated weights for policy 1, policy_version 1120758 (0.0005) [2023-12-27 13:25:19,459][20347] Updated weights for policy 0, policy_version 1121922 (0.0010) [2023-12-27 13:25:19,519][20347] Updated weights for policy 0, policy_version 1121932 (0.0007) [2023-12-27 13:25:19,578][20347] Updated weights for policy 0, policy_version 1121942 (0.0009) [2023-12-27 13:25:19,634][20347] Updated weights for policy 0, policy_version 1121952 (0.0008) [2023-12-27 13:25:19,876][20381] Updated weights for policy 1, policy_version 1120768 (0.0007) [2023-12-27 13:25:19,943][20381] Updated weights for policy 1, policy_version 1120778 (0.0006) [2023-12-27 13:25:20,008][20381] Updated weights for policy 1, policy_version 1120788 (0.0006) [2023-12-27 13:25:20,366][20347] Updated weights for policy 0, policy_version 1121962 (0.0006) [2023-12-27 13:25:20,431][20347] Updated weights for policy 0, policy_version 1121972 (0.0007) [2023-12-27 13:25:20,488][20347] Updated weights for policy 0, policy_version 1121982 (0.0006) [2023-12-27 13:25:20,676][20381] Updated weights for policy 1, policy_version 1120798 (0.0007) [2023-12-27 13:25:20,727][20381] Updated weights for policy 1, policy_version 1120808 (0.0007) [2023-12-27 13:25:20,783][20381] Updated weights for policy 1, policy_version 1120818 (0.0008) [2023-12-27 13:25:21,106][20347] Updated weights for policy 0, policy_version 1121992 (0.0007) [2023-12-27 13:25:21,174][20347] Updated weights for policy 0, policy_version 1122002 (0.0008) [2023-12-27 13:25:21,244][20347] Updated weights for policy 0, policy_version 1122012 (0.0007) [2023-12-27 13:25:21,519][20381] Updated weights for policy 1, policy_version 1120828 (0.0008) [2023-12-27 13:25:21,575][20381] Updated weights for policy 1, policy_version 1120838 (0.0006) [2023-12-27 13:25:21,633][20381] Updated weights for policy 1, policy_version 1120848 (0.0007) [2023-12-27 13:25:22,000][20347] Updated weights for policy 0, policy_version 1122022 (0.0009) [2023-12-27 13:25:22,059][20347] Updated weights for policy 0, policy_version 1122032 (0.0010) [2023-12-27 13:25:22,119][20347] Updated weights for policy 0, policy_version 1122042 (0.0010) [2023-12-27 13:25:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 574259200. Throughput: 0: 10436.4, 1: 10172.2. Samples: 574231352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:22,363][20381] Updated weights for policy 1, policy_version 1120858 (0.0008) [2023-12-27 13:25:22,363][19170] Avg episode reward: [(0, '980.740'), (1, '686.490')] [2023-12-27 13:25:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001122048_287293440.pth... [2023-12-27 13:25:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001120832_286982144.pth [2023-12-27 13:25:22,427][20381] Updated weights for policy 1, policy_version 1120868 (0.0007) [2023-12-27 13:25:22,486][20381] Updated weights for policy 1, policy_version 1120878 (0.0008) [2023-12-27 13:25:22,544][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001120888_286973952.pth... [2023-12-27 13:25:22,546][20381] Updated weights for policy 1, policy_version 1120888 (0.0008) [2023-12-27 13:25:22,550][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001119672_286662656.pth [2023-12-27 13:25:22,817][20347] Updated weights for policy 0, policy_version 1122052 (0.0010) [2023-12-27 13:25:22,873][20347] Updated weights for policy 0, policy_version 1122062 (0.0010) [2023-12-27 13:25:22,928][20347] Updated weights for policy 0, policy_version 1122072 (0.0010) [2023-12-27 13:25:23,202][20381] Updated weights for policy 1, policy_version 1120898 (0.0005) [2023-12-27 13:25:23,248][20381] Updated weights for policy 1, policy_version 1120908 (0.0005) [2023-12-27 13:25:23,297][20381] Updated weights for policy 1, policy_version 1120918 (0.0005) [2023-12-27 13:25:23,617][20347] Updated weights for policy 0, policy_version 1122082 (0.0009) [2023-12-27 13:25:23,675][20347] Updated weights for policy 0, policy_version 1122092 (0.0008) [2023-12-27 13:25:23,738][20347] Updated weights for policy 0, policy_version 1122102 (0.0008) [2023-12-27 13:25:23,791][20347] Updated weights for policy 0, policy_version 1122112 (0.0008) [2023-12-27 13:25:23,838][20381] Updated weights for policy 1, policy_version 1120928 (0.0007) [2023-12-27 13:25:23,894][20381] Updated weights for policy 1, policy_version 1120938 (0.0008) [2023-12-27 13:25:23,948][20381] Updated weights for policy 1, policy_version 1120948 (0.0008) [2023-12-27 13:25:24,499][20347] Updated weights for policy 0, policy_version 1122122 (0.0008) [2023-12-27 13:25:24,542][20291] KL-divergence is very high: 102.6864 [2023-12-27 13:25:24,561][20347] Updated weights for policy 0, policy_version 1122132 (0.0007) [2023-12-27 13:25:24,621][20347] Updated weights for policy 0, policy_version 1122142 (0.0006) [2023-12-27 13:25:24,657][20381] Updated weights for policy 1, policy_version 1120958 (0.0008) [2023-12-27 13:25:24,712][20381] Updated weights for policy 1, policy_version 1120968 (0.0008) [2023-12-27 13:25:24,769][20381] Updated weights for policy 1, policy_version 1120978 (0.0008) [2023-12-27 13:25:25,225][20347] Updated weights for policy 0, policy_version 1122152 (0.0007) [2023-12-27 13:25:25,280][20347] Updated weights for policy 0, policy_version 1122162 (0.0006) [2023-12-27 13:25:25,332][20347] Updated weights for policy 0, policy_version 1122172 (0.0006) [2023-12-27 13:25:25,359][20381] Updated weights for policy 1, policy_version 1120988 (0.0008) [2023-12-27 13:25:25,418][20381] Updated weights for policy 1, policy_version 1120998 (0.0008) [2023-12-27 13:25:25,470][20381] Updated weights for policy 1, policy_version 1121008 (0.0007) [2023-12-27 13:25:25,982][20347] Updated weights for policy 0, policy_version 1122182 (0.0010) [2023-12-27 13:25:26,035][20347] Updated weights for policy 0, policy_version 1122192 (0.0008) [2023-12-27 13:25:26,091][20347] Updated weights for policy 0, policy_version 1122202 (0.0006) [2023-12-27 13:25:26,155][20381] Updated weights for policy 1, policy_version 1121018 (0.0007) [2023-12-27 13:25:26,206][20381] Updated weights for policy 1, policy_version 1121028 (0.0005) [2023-12-27 13:25:26,252][20381] Updated weights for policy 1, policy_version 1121038 (0.0006) [2023-12-27 13:25:26,297][20381] Updated weights for policy 1, policy_version 1121048 (0.0005) [2023-12-27 13:25:26,644][20347] Updated weights for policy 0, policy_version 1122212 (0.0010) [2023-12-27 13:25:26,694][20347] Updated weights for policy 0, policy_version 1122222 (0.0006) [2023-12-27 13:25:26,738][20347] Updated weights for policy 0, policy_version 1122232 (0.0005) [2023-12-27 13:25:26,830][20381] Updated weights for policy 1, policy_version 1121058 (0.0008) [2023-12-27 13:25:26,889][20381] Updated weights for policy 1, policy_version 1121068 (0.0008) [2023-12-27 13:25:26,954][20381] Updated weights for policy 1, policy_version 1121078 (0.0008) [2023-12-27 13:25:27,309][20347] Updated weights for policy 0, policy_version 1122242 (0.0008) [2023-12-27 13:25:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 574365696. Throughput: 0: 10386.5, 1: 10238.4. Samples: 574354944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:27,363][19170] Avg episode reward: [(0, '970.890'), (1, '715.720')] [2023-12-27 13:25:27,366][20347] Updated weights for policy 0, policy_version 1122252 (0.0010) [2023-12-27 13:25:27,410][20347] Updated weights for policy 0, policy_version 1122262 (0.0010) [2023-12-27 13:25:27,470][20347] Updated weights for policy 0, policy_version 1122272 (0.0008) [2023-12-27 13:25:27,613][20381] Updated weights for policy 1, policy_version 1121088 (0.0010) [2023-12-27 13:25:27,664][20381] Updated weights for policy 1, policy_version 1121098 (0.0010) [2023-12-27 13:25:27,705][20381] Updated weights for policy 1, policy_version 1121108 (0.0010) [2023-12-27 13:25:28,122][20347] Updated weights for policy 0, policy_version 1122282 (0.0010) [2023-12-27 13:25:28,175][20347] Updated weights for policy 0, policy_version 1122292 (0.0010) [2023-12-27 13:25:28,228][20347] Updated weights for policy 0, policy_version 1122302 (0.0010) [2023-12-27 13:25:28,326][20381] Updated weights for policy 1, policy_version 1121118 (0.0009) [2023-12-27 13:25:28,392][20381] Updated weights for policy 1, policy_version 1121128 (0.0009) [2023-12-27 13:25:28,443][20381] Updated weights for policy 1, policy_version 1121138 (0.0007) [2023-12-27 13:25:28,906][20347] Updated weights for policy 0, policy_version 1122312 (0.0010) [2023-12-27 13:25:28,931][20291] KL-divergence is very high: 356.0623 [2023-12-27 13:25:28,936][20291] KL-divergence is very high: 1392.4697 [2023-12-27 13:25:28,956][20347] Updated weights for policy 0, policy_version 1122322 (0.0010) [2023-12-27 13:25:28,973][20291] KL-divergence is very high: 125.4622 [2023-12-27 13:25:29,009][20347] Updated weights for policy 0, policy_version 1122332 (0.0010) [2023-12-27 13:25:29,015][20291] KL-divergence is very high: 280.7179 [2023-12-27 13:25:29,020][20291] KL-divergence is very high: 316.6719 [2023-12-27 13:25:29,142][20381] Updated weights for policy 1, policy_version 1121148 (0.0009) [2023-12-27 13:25:29,193][20381] Updated weights for policy 1, policy_version 1121158 (0.0010) [2023-12-27 13:25:29,269][20381] Updated weights for policy 1, policy_version 1121168 (0.0010) [2023-12-27 13:25:29,683][20291] KL-divergence is very high: 111.4653 [2023-12-27 13:25:29,696][20291] KL-divergence is very high: 264.3287 [2023-12-27 13:25:29,720][20347] Updated weights for policy 0, policy_version 1122342 (0.0010) [2023-12-27 13:25:29,733][20291] KL-divergence is very high: 182.1449 [2023-12-27 13:25:29,746][20291] KL-divergence is very high: 471.8742 [2023-12-27 13:25:29,783][20347] Updated weights for policy 0, policy_version 1122352 (0.0010) [2023-12-27 13:25:29,794][20291] KL-divergence is very high: 144.1111 [2023-12-27 13:25:29,848][20347] Updated weights for policy 0, policy_version 1122362 (0.0009) [2023-12-27 13:25:29,849][20291] KL-divergence is very high: 229.9357 [2023-12-27 13:25:29,882][20381] Updated weights for policy 1, policy_version 1121178 (0.0009) [2023-12-27 13:25:29,947][20381] Updated weights for policy 1, policy_version 1121188 (0.0007) [2023-12-27 13:25:30,012][20381] Updated weights for policy 1, policy_version 1121198 (0.0008) [2023-12-27 13:25:30,060][20381] Updated weights for policy 1, policy_version 1121208 (0.0006) [2023-12-27 13:25:30,511][20347] Updated weights for policy 0, policy_version 1122372 (0.0007) [2023-12-27 13:25:30,568][20347] Updated weights for policy 0, policy_version 1122382 (0.0006) [2023-12-27 13:25:30,625][20347] Updated weights for policy 0, policy_version 1122392 (0.0006) [2023-12-27 13:25:30,778][20381] Updated weights for policy 1, policy_version 1121218 (0.0007) [2023-12-27 13:25:30,831][20381] Updated weights for policy 1, policy_version 1121228 (0.0007) [2023-12-27 13:25:30,888][20381] Updated weights for policy 1, policy_version 1121238 (0.0008) [2023-12-27 13:25:31,242][20347] Updated weights for policy 0, policy_version 1122402 (0.0006) [2023-12-27 13:25:31,305][20347] Updated weights for policy 0, policy_version 1122412 (0.0011) [2023-12-27 13:25:31,366][20347] Updated weights for policy 0, policy_version 1122422 (0.0010) [2023-12-27 13:25:31,429][20347] Updated weights for policy 0, policy_version 1122432 (0.0007) [2023-12-27 13:25:31,614][20381] Updated weights for policy 1, policy_version 1121248 (0.0008) [2023-12-27 13:25:31,674][20381] Updated weights for policy 1, policy_version 1121258 (0.0008) [2023-12-27 13:25:31,732][20381] Updated weights for policy 1, policy_version 1121268 (0.0007) [2023-12-27 13:25:32,123][20347] Updated weights for policy 0, policy_version 1122442 (0.0011) [2023-12-27 13:25:32,177][20347] Updated weights for policy 0, policy_version 1122452 (0.0010) [2023-12-27 13:25:32,231][20347] Updated weights for policy 0, policy_version 1122462 (0.0008) [2023-12-27 13:25:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 574472192. Throughput: 0: 10418.2, 1: 10262.7. Samples: 574482716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:25:32,363][19170] Avg episode reward: [(0, '951.460'), (1, '597.880')] [2023-12-27 13:25:32,389][20381] Updated weights for policy 1, policy_version 1121278 (0.0008) [2023-12-27 13:25:32,452][20381] Updated weights for policy 1, policy_version 1121288 (0.0008) [2023-12-27 13:25:32,508][20381] Updated weights for policy 1, policy_version 1121298 (0.0008) [2023-12-27 13:25:32,854][20347] Updated weights for policy 0, policy_version 1122472 (0.0008) [2023-12-27 13:25:32,900][20347] Updated weights for policy 0, policy_version 1122482 (0.0008) [2023-12-27 13:25:32,951][20347] Updated weights for policy 0, policy_version 1122492 (0.0008) [2023-12-27 13:25:33,112][20381] Updated weights for policy 1, policy_version 1121308 (0.0007) [2023-12-27 13:25:33,137][20292] KL-divergence is very high: 106.8222 [2023-12-27 13:25:33,159][20292] KL-divergence is very high: 415.0558 [2023-12-27 13:25:33,170][20381] Updated weights for policy 1, policy_version 1121318 (0.0007) [2023-12-27 13:25:33,181][20292] KL-divergence is very high: 198.7712 [2023-12-27 13:25:33,203][20292] KL-divergence is very high: 255.2726 [2023-12-27 13:25:33,223][20381] Updated weights for policy 1, policy_version 1121328 (0.0007) [2023-12-27 13:25:33,244][20292] KL-divergence is very high: 108.2397 [2023-12-27 13:25:33,633][20347] Updated weights for policy 0, policy_version 1122502 (0.0006) [2023-12-27 13:25:33,686][20347] Updated weights for policy 0, policy_version 1122512 (0.0005) [2023-12-27 13:25:33,736][20347] Updated weights for policy 0, policy_version 1122522 (0.0005) [2023-12-27 13:25:33,889][20381] Updated weights for policy 1, policy_version 1121338 (0.0007) [2023-12-27 13:25:33,936][20381] Updated weights for policy 1, policy_version 1121348 (0.0005) [2023-12-27 13:25:33,984][20381] Updated weights for policy 1, policy_version 1121358 (0.0006) [2023-12-27 13:25:34,027][20381] Updated weights for policy 1, policy_version 1121368 (0.0007) [2023-12-27 13:25:34,397][20347] Updated weights for policy 0, policy_version 1122532 (0.0009) [2023-12-27 13:25:34,452][20347] Updated weights for policy 0, policy_version 1122542 (0.0008) [2023-12-27 13:25:34,512][20347] Updated weights for policy 0, policy_version 1122552 (0.0007) [2023-12-27 13:25:34,740][20381] Updated weights for policy 1, policy_version 1121378 (0.0009) [2023-12-27 13:25:34,792][20381] Updated weights for policy 1, policy_version 1121388 (0.0008) [2023-12-27 13:25:34,837][20381] Updated weights for policy 1, policy_version 1121398 (0.0008) [2023-12-27 13:25:35,190][20347] Updated weights for policy 0, policy_version 1122562 (0.0008) [2023-12-27 13:25:35,245][20347] Updated weights for policy 0, policy_version 1122572 (0.0008) [2023-12-27 13:25:35,304][20347] Updated weights for policy 0, policy_version 1122582 (0.0007) [2023-12-27 13:25:35,360][20347] Updated weights for policy 0, policy_version 1122592 (0.0008) [2023-12-27 13:25:35,574][20381] Updated weights for policy 1, policy_version 1121408 (0.0008) [2023-12-27 13:25:35,636][20381] Updated weights for policy 1, policy_version 1121418 (0.0008) [2023-12-27 13:25:35,690][20381] Updated weights for policy 1, policy_version 1121428 (0.0008) [2023-12-27 13:25:36,044][20347] Updated weights for policy 0, policy_version 1122602 (0.0008) [2023-12-27 13:25:36,110][20347] Updated weights for policy 0, policy_version 1122612 (0.0023) [2023-12-27 13:25:36,167][20347] Updated weights for policy 0, policy_version 1122622 (0.0009) [2023-12-27 13:25:36,378][20381] Updated weights for policy 1, policy_version 1121438 (0.0008) [2023-12-27 13:25:36,438][20381] Updated weights for policy 1, policy_version 1121448 (0.0008) [2023-12-27 13:25:36,497][20381] Updated weights for policy 1, policy_version 1121458 (0.0008) [2023-12-27 13:25:36,897][20347] Updated weights for policy 0, policy_version 1122632 (0.0008) [2023-12-27 13:25:36,965][20347] Updated weights for policy 0, policy_version 1122642 (0.0008) [2023-12-27 13:25:37,030][20347] Updated weights for policy 0, policy_version 1122652 (0.0008) [2023-12-27 13:25:37,187][20381] Updated weights for policy 1, policy_version 1121468 (0.0007) [2023-12-27 13:25:37,242][20381] Updated weights for policy 1, policy_version 1121478 (0.0007) [2023-12-27 13:25:37,302][20381] Updated weights for policy 1, policy_version 1121488 (0.0008) [2023-12-27 13:25:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 574578688. Throughput: 0: 10453.7, 1: 10278.7. Samples: 574545284. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:25:37,363][19170] Avg episode reward: [(0, '951.380'), (1, '637.430')] [2023-12-27 13:25:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001121496_287129600.pth... [2023-12-27 13:25:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001122656_287449088.pth... [2023-12-27 13:25:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001121472_287145984.pth [2023-12-27 13:25:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001120280_286818304.pth [2023-12-27 13:25:37,661][20347] Updated weights for policy 0, policy_version 1122662 (0.0007) [2023-12-27 13:25:37,718][20347] Updated weights for policy 0, policy_version 1122672 (0.0007) [2023-12-27 13:25:37,767][20347] Updated weights for policy 0, policy_version 1122682 (0.0008) [2023-12-27 13:25:38,044][20381] Updated weights for policy 1, policy_version 1121498 (0.0008) [2023-12-27 13:25:38,100][20381] Updated weights for policy 1, policy_version 1121508 (0.0008) [2023-12-27 13:25:38,156][20381] Updated weights for policy 1, policy_version 1121518 (0.0008) [2023-12-27 13:25:38,217][20381] Updated weights for policy 1, policy_version 1121528 (0.0008) [2023-12-27 13:25:38,370][20347] Updated weights for policy 0, policy_version 1122692 (0.0007) [2023-12-27 13:25:38,422][20347] Updated weights for policy 0, policy_version 1122702 (0.0008) [2023-12-27 13:25:38,479][20347] Updated weights for policy 0, policy_version 1122712 (0.0008) [2023-12-27 13:25:38,902][20381] Updated weights for policy 1, policy_version 1121538 (0.0009) [2023-12-27 13:25:38,957][20381] Updated weights for policy 1, policy_version 1121548 (0.0007) [2023-12-27 13:25:39,010][20381] Updated weights for policy 1, policy_version 1121558 (0.0009) [2023-12-27 13:25:39,078][20347] Updated weights for policy 0, policy_version 1122722 (0.0005) [2023-12-27 13:25:39,139][20347] Updated weights for policy 0, policy_version 1122732 (0.0005) [2023-12-27 13:25:39,199][20347] Updated weights for policy 0, policy_version 1122742 (0.0008) [2023-12-27 13:25:39,266][20347] Updated weights for policy 0, policy_version 1122752 (0.0007) [2023-12-27 13:25:39,711][20381] Updated weights for policy 1, policy_version 1121568 (0.0008) [2023-12-27 13:25:39,770][20381] Updated weights for policy 1, policy_version 1121578 (0.0008) [2023-12-27 13:25:39,832][20381] Updated weights for policy 1, policy_version 1121588 (0.0008) [2023-12-27 13:25:39,961][20347] Updated weights for policy 0, policy_version 1122762 (0.0011) [2023-12-27 13:25:40,012][20347] Updated weights for policy 0, policy_version 1122772 (0.0010) [2023-12-27 13:25:40,072][20347] Updated weights for policy 0, policy_version 1122782 (0.0010) [2023-12-27 13:25:40,537][20381] Updated weights for policy 1, policy_version 1121598 (0.0009) [2023-12-27 13:25:40,596][20381] Updated weights for policy 1, policy_version 1121608 (0.0009) [2023-12-27 13:25:40,658][20381] Updated weights for policy 1, policy_version 1121618 (0.0005) [2023-12-27 13:25:40,794][20347] Updated weights for policy 0, policy_version 1122792 (0.0011) [2023-12-27 13:25:40,847][20347] Updated weights for policy 0, policy_version 1122802 (0.0010) [2023-12-27 13:25:40,889][20291] KL-divergence is very high: 143.2067 [2023-12-27 13:25:40,900][20347] Updated weights for policy 0, policy_version 1122812 (0.0010) [2023-12-27 13:25:41,320][20381] Updated weights for policy 1, policy_version 1121628 (0.0008) [2023-12-27 13:25:41,398][20381] Updated weights for policy 1, policy_version 1121638 (0.0009) [2023-12-27 13:25:41,451][20381] Updated weights for policy 1, policy_version 1121648 (0.0007) [2023-12-27 13:25:41,652][20347] Updated weights for policy 0, policy_version 1122822 (0.0010) [2023-12-27 13:25:41,717][20347] Updated weights for policy 0, policy_version 1122832 (0.0007) [2023-12-27 13:25:41,790][20347] Updated weights for policy 0, policy_version 1122842 (0.0008) [2023-12-27 13:25:42,126][20381] Updated weights for policy 1, policy_version 1121658 (0.0008) [2023-12-27 13:25:42,190][20381] Updated weights for policy 1, policy_version 1121668 (0.0006) [2023-12-27 13:25:42,255][20381] Updated weights for policy 1, policy_version 1121678 (0.0006) [2023-12-27 13:25:42,312][20381] Updated weights for policy 1, policy_version 1121688 (0.0008) [2023-12-27 13:25:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 574676992. Throughput: 0: 10444.3, 1: 10278.4. Samples: 574667076. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:25:42,363][19170] Avg episode reward: [(0, '931.760'), (1, '705.700')] [2023-12-27 13:25:42,452][20347] Updated weights for policy 0, policy_version 1122852 (0.0009) [2023-12-27 13:25:42,512][20347] Updated weights for policy 0, policy_version 1122862 (0.0007) [2023-12-27 13:25:42,571][20347] Updated weights for policy 0, policy_version 1122872 (0.0007) [2023-12-27 13:25:42,965][20381] Updated weights for policy 1, policy_version 1121698 (0.0005) [2023-12-27 13:25:43,024][20381] Updated weights for policy 1, policy_version 1121708 (0.0010) [2023-12-27 13:25:43,080][20381] Updated weights for policy 1, policy_version 1121718 (0.0010) [2023-12-27 13:25:43,263][20347] Updated weights for policy 0, policy_version 1122882 (0.0007) [2023-12-27 13:25:43,311][20347] Updated weights for policy 0, policy_version 1122892 (0.0007) [2023-12-27 13:25:43,368][20347] Updated weights for policy 0, policy_version 1122902 (0.0007) [2023-12-27 13:25:43,426][20347] Updated weights for policy 0, policy_version 1122912 (0.0007) [2023-12-27 13:25:43,756][20381] Updated weights for policy 1, policy_version 1121728 (0.0007) [2023-12-27 13:25:43,822][20381] Updated weights for policy 1, policy_version 1121738 (0.0005) [2023-12-27 13:25:43,881][20381] Updated weights for policy 1, policy_version 1121748 (0.0005) [2023-12-27 13:25:44,152][20347] Updated weights for policy 0, policy_version 1122922 (0.0008) [2023-12-27 13:25:44,196][20347] Updated weights for policy 0, policy_version 1122932 (0.0007) [2023-12-27 13:25:44,246][20347] Updated weights for policy 0, policy_version 1122942 (0.0006) [2023-12-27 13:25:44,419][20381] Updated weights for policy 1, policy_version 1121758 (0.0007) [2023-12-27 13:25:44,481][20381] Updated weights for policy 1, policy_version 1121768 (0.0007) [2023-12-27 13:25:44,543][20381] Updated weights for policy 1, policy_version 1121778 (0.0006) [2023-12-27 13:25:44,915][20347] Updated weights for policy 0, policy_version 1122952 (0.0006) [2023-12-27 13:25:44,974][20347] Updated weights for policy 0, policy_version 1122962 (0.0006) [2023-12-27 13:25:45,030][20347] Updated weights for policy 0, policy_version 1122972 (0.0006) [2023-12-27 13:25:45,239][20381] Updated weights for policy 1, policy_version 1121788 (0.0008) [2023-12-27 13:25:45,301][20381] Updated weights for policy 1, policy_version 1121798 (0.0008) [2023-12-27 13:25:45,352][20381] Updated weights for policy 1, policy_version 1121808 (0.0008) [2023-12-27 13:25:45,687][20347] Updated weights for policy 0, policy_version 1122982 (0.0008) [2023-12-27 13:25:45,737][20347] Updated weights for policy 0, policy_version 1122992 (0.0008) [2023-12-27 13:25:45,792][20347] Updated weights for policy 0, policy_version 1123002 (0.0008) [2023-12-27 13:25:46,015][20381] Updated weights for policy 1, policy_version 1121818 (0.0007) [2023-12-27 13:25:46,076][20381] Updated weights for policy 1, policy_version 1121828 (0.0007) [2023-12-27 13:25:46,128][20381] Updated weights for policy 1, policy_version 1121838 (0.0008) [2023-12-27 13:25:46,176][20381] Updated weights for policy 1, policy_version 1121848 (0.0008) [2023-12-27 13:25:46,474][20347] Updated weights for policy 0, policy_version 1123012 (0.0006) [2023-12-27 13:25:46,535][20347] Updated weights for policy 0, policy_version 1123022 (0.0006) [2023-12-27 13:25:46,591][20347] Updated weights for policy 0, policy_version 1123032 (0.0008) [2023-12-27 13:25:46,832][20381] Updated weights for policy 1, policy_version 1121858 (0.0007) [2023-12-27 13:25:46,879][20381] Updated weights for policy 1, policy_version 1121868 (0.0007) [2023-12-27 13:25:46,931][20381] Updated weights for policy 1, policy_version 1121878 (0.0007) [2023-12-27 13:25:47,252][20347] Updated weights for policy 0, policy_version 1123042 (0.0008) [2023-12-27 13:25:47,304][20347] Updated weights for policy 0, policy_version 1123052 (0.0006) [2023-12-27 13:25:47,356][20347] Updated weights for policy 0, policy_version 1123062 (0.0007) [2023-12-27 13:25:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 574775296. Throughput: 0: 10311.1, 1: 10363.8. Samples: 574790044. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:25:47,363][19170] Avg episode reward: [(0, '892.600'), (1, '695.980')] [2023-12-27 13:25:47,403][20347] Updated weights for policy 0, policy_version 1123072 (0.0008) [2023-12-27 13:25:47,653][20381] Updated weights for policy 1, policy_version 1121888 (0.0008) [2023-12-27 13:25:47,710][20381] Updated weights for policy 1, policy_version 1121898 (0.0007) [2023-12-27 13:25:47,762][20381] Updated weights for policy 1, policy_version 1121908 (0.0008) [2023-12-27 13:25:48,057][20347] Updated weights for policy 0, policy_version 1123082 (0.0008) [2023-12-27 13:25:48,117][20347] Updated weights for policy 0, policy_version 1123092 (0.0008) [2023-12-27 13:25:48,168][20347] Updated weights for policy 0, policy_version 1123102 (0.0008) [2023-12-27 13:25:48,471][20381] Updated weights for policy 1, policy_version 1121918 (0.0010) [2023-12-27 13:25:48,526][20381] Updated weights for policy 1, policy_version 1121928 (0.0010) [2023-12-27 13:25:48,585][20381] Updated weights for policy 1, policy_version 1121938 (0.0011) [2023-12-27 13:25:48,874][20347] Updated weights for policy 0, policy_version 1123112 (0.0006) [2023-12-27 13:25:48,934][20347] Updated weights for policy 0, policy_version 1123122 (0.0006) [2023-12-27 13:25:48,992][20347] Updated weights for policy 0, policy_version 1123132 (0.0008) [2023-12-27 13:25:49,179][20381] Updated weights for policy 1, policy_version 1121948 (0.0008) [2023-12-27 13:25:49,231][20381] Updated weights for policy 1, policy_version 1121958 (0.0006) [2023-12-27 13:25:49,293][20381] Updated weights for policy 1, policy_version 1121968 (0.0010) [2023-12-27 13:25:49,742][20347] Updated weights for policy 0, policy_version 1123142 (0.0009) [2023-12-27 13:25:49,795][20347] Updated weights for policy 0, policy_version 1123152 (0.0008) [2023-12-27 13:25:49,854][20347] Updated weights for policy 0, policy_version 1123162 (0.0008) [2023-12-27 13:25:49,914][20381] Updated weights for policy 1, policy_version 1121978 (0.0010) [2023-12-27 13:25:49,973][20381] Updated weights for policy 1, policy_version 1121988 (0.0008) [2023-12-27 13:25:50,030][20381] Updated weights for policy 1, policy_version 1121998 (0.0007) [2023-12-27 13:25:50,092][20381] Updated weights for policy 1, policy_version 1122008 (0.0008) [2023-12-27 13:25:50,517][20291] KL-divergence is very high: 167.5456 [2023-12-27 13:25:50,537][20347] Updated weights for policy 0, policy_version 1123172 (0.0008) [2023-12-27 13:25:50,603][20347] Updated weights for policy 0, policy_version 1123182 (0.0008) [2023-12-27 13:25:50,665][20347] Updated weights for policy 0, policy_version 1123192 (0.0006) [2023-12-27 13:25:50,750][20381] Updated weights for policy 1, policy_version 1122018 (0.0007) [2023-12-27 13:25:50,812][20381] Updated weights for policy 1, policy_version 1122028 (0.0006) [2023-12-27 13:25:50,871][20381] Updated weights for policy 1, policy_version 1122038 (0.0007) [2023-12-27 13:25:51,378][20347] Updated weights for policy 0, policy_version 1123202 (0.0008) [2023-12-27 13:25:51,443][20347] Updated weights for policy 0, policy_version 1123212 (0.0008) [2023-12-27 13:25:51,494][20347] Updated weights for policy 0, policy_version 1123222 (0.0008) [2023-12-27 13:25:51,546][20347] Updated weights for policy 0, policy_version 1123232 (0.0008) [2023-12-27 13:25:51,574][20381] Updated weights for policy 1, policy_version 1122048 (0.0007) [2023-12-27 13:25:51,633][20381] Updated weights for policy 1, policy_version 1122058 (0.0006) [2023-12-27 13:25:51,693][20381] Updated weights for policy 1, policy_version 1122068 (0.0006) [2023-12-27 13:25:52,299][20347] Updated weights for policy 0, policy_version 1123242 (0.0008) [2023-12-27 13:25:52,304][20381] Updated weights for policy 1, policy_version 1122078 (0.0007) [2023-12-27 13:25:52,353][20347] Updated weights for policy 0, policy_version 1123252 (0.0005) [2023-12-27 13:25:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 574873600. Throughput: 0: 10242.8, 1: 10383.7. Samples: 574850824. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:25:52,363][20381] Updated weights for policy 1, policy_version 1122088 (0.0006) [2023-12-27 13:25:52,363][19170] Avg episode reward: [(0, '902.420'), (1, '715.510')] [2023-12-27 13:25:52,417][20347] Updated weights for policy 0, policy_version 1123262 (0.0008) [2023-12-27 13:25:52,424][20381] Updated weights for policy 1, policy_version 1122098 (0.0008) [2023-12-27 13:25:52,427][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001123264_287604736.pth... [2023-12-27 13:25:52,431][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001122048_287293440.pth [2023-12-27 13:25:52,456][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001122104_287285248.pth... [2023-12-27 13:25:52,467][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001120888_286973952.pth [2023-12-27 13:25:53,037][20347] Updated weights for policy 0, policy_version 1123272 (0.0007) [2023-12-27 13:25:53,091][20381] Updated weights for policy 1, policy_version 1122108 (0.0008) [2023-12-27 13:25:53,097][20347] Updated weights for policy 0, policy_version 1123282 (0.0006) [2023-12-27 13:25:53,142][20381] Updated weights for policy 1, policy_version 1122118 (0.0009) [2023-12-27 13:25:53,155][20347] Updated weights for policy 0, policy_version 1123292 (0.0007) [2023-12-27 13:25:53,198][20381] Updated weights for policy 1, policy_version 1122128 (0.0005) [2023-12-27 13:25:53,728][20381] Updated weights for policy 1, policy_version 1122138 (0.0006) [2023-12-27 13:25:53,776][20347] Updated weights for policy 0, policy_version 1123302 (0.0008) [2023-12-27 13:25:53,784][20381] Updated weights for policy 1, policy_version 1122148 (0.0011) [2023-12-27 13:25:53,826][20347] Updated weights for policy 0, policy_version 1123312 (0.0006) [2023-12-27 13:25:53,828][20381] Updated weights for policy 1, policy_version 1122158 (0.0010) [2023-12-27 13:25:53,873][20381] Updated weights for policy 1, policy_version 1122168 (0.0010) [2023-12-27 13:25:53,878][20347] Updated weights for policy 0, policy_version 1123322 (0.0006) [2023-12-27 13:25:54,555][20347] Updated weights for policy 0, policy_version 1123332 (0.0007) [2023-12-27 13:25:54,562][20381] Updated weights for policy 1, policy_version 1122178 (0.0011) [2023-12-27 13:25:54,611][20347] Updated weights for policy 0, policy_version 1123342 (0.0009) [2023-12-27 13:25:54,621][20381] Updated weights for policy 1, policy_version 1122188 (0.0010) [2023-12-27 13:25:54,667][20347] Updated weights for policy 0, policy_version 1123352 (0.0007) [2023-12-27 13:25:54,671][20381] Updated weights for policy 1, policy_version 1122198 (0.0011) [2023-12-27 13:25:55,326][20347] Updated weights for policy 0, policy_version 1123362 (0.0005) [2023-12-27 13:25:55,375][20381] Updated weights for policy 1, policy_version 1122208 (0.0008) [2023-12-27 13:25:55,384][20347] Updated weights for policy 0, policy_version 1123372 (0.0006) [2023-12-27 13:25:55,430][20381] Updated weights for policy 1, policy_version 1122218 (0.0009) [2023-12-27 13:25:55,438][20347] Updated weights for policy 0, policy_version 1123382 (0.0006) [2023-12-27 13:25:55,486][20381] Updated weights for policy 1, policy_version 1122228 (0.0010) [2023-12-27 13:25:55,496][20347] Updated weights for policy 0, policy_version 1123392 (0.0009) [2023-12-27 13:25:56,173][20347] Updated weights for policy 0, policy_version 1123402 (0.0008) [2023-12-27 13:25:56,193][20381] Updated weights for policy 1, policy_version 1122238 (0.0011) [2023-12-27 13:25:56,234][20347] Updated weights for policy 0, policy_version 1123412 (0.0006) [2023-12-27 13:25:56,251][20381] Updated weights for policy 1, policy_version 1122248 (0.0010) [2023-12-27 13:25:56,291][20347] Updated weights for policy 0, policy_version 1123422 (0.0006) [2023-12-27 13:25:56,309][20381] Updated weights for policy 1, policy_version 1122258 (0.0011) [2023-12-27 13:25:56,942][20381] Updated weights for policy 1, policy_version 1122268 (0.0010) [2023-12-27 13:25:56,998][20381] Updated weights for policy 1, policy_version 1122278 (0.0009) [2023-12-27 13:25:57,000][20347] Updated weights for policy 0, policy_version 1123432 (0.0007) [2023-12-27 13:25:57,052][20347] Updated weights for policy 0, policy_version 1123442 (0.0007) [2023-12-27 13:25:57,056][20381] Updated weights for policy 1, policy_version 1122288 (0.0010) [2023-12-27 13:25:57,105][20347] Updated weights for policy 0, policy_version 1123452 (0.0007) [2023-12-27 13:25:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 574988288. Throughput: 0: 10286.3, 1: 10385.8. Samples: 574975612. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:25:57,363][19170] Avg episode reward: [(0, '912.150'), (1, '627.470')] [2023-12-27 13:25:57,654][20381] Updated weights for policy 1, policy_version 1122298 (0.0009) [2023-12-27 13:25:57,704][20381] Updated weights for policy 1, policy_version 1122308 (0.0006) [2023-12-27 13:25:57,759][20381] Updated weights for policy 1, policy_version 1122318 (0.0011) [2023-12-27 13:25:57,795][20347] Updated weights for policy 0, policy_version 1123462 (0.0009) [2023-12-27 13:25:57,813][20381] Updated weights for policy 1, policy_version 1122328 (0.0010) [2023-12-27 13:25:57,841][20347] Updated weights for policy 0, policy_version 1123472 (0.0006) [2023-12-27 13:25:57,884][20347] Updated weights for policy 0, policy_version 1123482 (0.0007) [2023-12-27 13:25:58,455][20381] Updated weights for policy 1, policy_version 1122338 (0.0008) [2023-12-27 13:25:58,517][20381] Updated weights for policy 1, policy_version 1122348 (0.0009) [2023-12-27 13:25:58,581][20381] Updated weights for policy 1, policy_version 1122358 (0.0008) [2023-12-27 13:25:58,631][20347] Updated weights for policy 0, policy_version 1123492 (0.0008) [2023-12-27 13:25:58,691][20347] Updated weights for policy 0, policy_version 1123502 (0.0010) [2023-12-27 13:25:58,762][20347] Updated weights for policy 0, policy_version 1123512 (0.0008) [2023-12-27 13:25:59,332][20381] Updated weights for policy 1, policy_version 1122368 (0.0007) [2023-12-27 13:25:59,393][20381] Updated weights for policy 1, policy_version 1122378 (0.0008) [2023-12-27 13:25:59,444][20381] Updated weights for policy 1, policy_version 1122388 (0.0006) [2023-12-27 13:25:59,514][20347] Updated weights for policy 0, policy_version 1123522 (0.0007) [2023-12-27 13:25:59,564][20347] Updated weights for policy 0, policy_version 1123532 (0.0007) [2023-12-27 13:25:59,617][20347] Updated weights for policy 0, policy_version 1123542 (0.0007) [2023-12-27 13:25:59,670][20347] Updated weights for policy 0, policy_version 1123552 (0.0007) [2023-12-27 13:26:00,131][20381] Updated weights for policy 1, policy_version 1122398 (0.0009) [2023-12-27 13:26:00,193][20381] Updated weights for policy 1, policy_version 1122408 (0.0010) [2023-12-27 13:26:00,252][20381] Updated weights for policy 1, policy_version 1122418 (0.0009) [2023-12-27 13:26:00,418][20347] Updated weights for policy 0, policy_version 1123562 (0.0007) [2023-12-27 13:26:00,500][20347] Updated weights for policy 0, policy_version 1123572 (0.0007) [2023-12-27 13:26:00,547][20347] Updated weights for policy 0, policy_version 1123582 (0.0006) [2023-12-27 13:26:00,894][20381] Updated weights for policy 1, policy_version 1122428 (0.0007) [2023-12-27 13:26:00,942][20381] Updated weights for policy 1, policy_version 1122438 (0.0006) [2023-12-27 13:26:01,000][20381] Updated weights for policy 1, policy_version 1122448 (0.0005) [2023-12-27 13:26:01,144][20347] Updated weights for policy 0, policy_version 1123592 (0.0008) [2023-12-27 13:26:01,201][20347] Updated weights for policy 0, policy_version 1123602 (0.0008) [2023-12-27 13:26:01,257][20347] Updated weights for policy 0, policy_version 1123612 (0.0011) [2023-12-27 13:26:01,642][20381] Updated weights for policy 1, policy_version 1122458 (0.0007) [2023-12-27 13:26:01,696][20381] Updated weights for policy 1, policy_version 1122468 (0.0007) [2023-12-27 13:26:01,759][20381] Updated weights for policy 1, policy_version 1122478 (0.0008) [2023-12-27 13:26:01,803][20381] Updated weights for policy 1, policy_version 1122488 (0.0007) [2023-12-27 13:26:01,999][20347] Updated weights for policy 0, policy_version 1123622 (0.0008) [2023-12-27 13:26:02,051][20347] Updated weights for policy 0, policy_version 1123632 (0.0007) [2023-12-27 13:26:02,102][20347] Updated weights for policy 0, policy_version 1123642 (0.0007) [2023-12-27 13:26:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 575086592. Throughput: 0: 10222.2, 1: 10366.3. Samples: 575096688. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:02,363][19170] Avg episode reward: [(0, '951.280'), (1, '646.120')] [2023-12-27 13:26:02,555][20381] Updated weights for policy 1, policy_version 1122498 (0.0006) [2023-12-27 13:26:02,614][20381] Updated weights for policy 1, policy_version 1122508 (0.0008) [2023-12-27 13:26:02,674][20381] Updated weights for policy 1, policy_version 1122518 (0.0006) [2023-12-27 13:26:02,768][20347] Updated weights for policy 0, policy_version 1123652 (0.0009) [2023-12-27 13:26:02,817][20347] Updated weights for policy 0, policy_version 1123662 (0.0010) [2023-12-27 13:26:02,869][20347] Updated weights for policy 0, policy_version 1123672 (0.0011) [2023-12-27 13:26:03,314][20381] Updated weights for policy 1, policy_version 1122528 (0.0007) [2023-12-27 13:26:03,361][20381] Updated weights for policy 1, policy_version 1122538 (0.0007) [2023-12-27 13:26:03,423][20381] Updated weights for policy 1, policy_version 1122548 (0.0008) [2023-12-27 13:26:03,535][20347] Updated weights for policy 0, policy_version 1123682 (0.0010) [2023-12-27 13:26:03,586][20347] Updated weights for policy 0, policy_version 1123692 (0.0005) [2023-12-27 13:26:03,638][20347] Updated weights for policy 0, policy_version 1123702 (0.0007) [2023-12-27 13:26:03,692][20347] Updated weights for policy 0, policy_version 1123712 (0.0010) [2023-12-27 13:26:04,034][20381] Updated weights for policy 1, policy_version 1122558 (0.0006) [2023-12-27 13:26:04,095][20381] Updated weights for policy 1, policy_version 1122568 (0.0006) [2023-12-27 13:26:04,160][20381] Updated weights for policy 1, policy_version 1122578 (0.0006) [2023-12-27 13:26:04,429][20347] Updated weights for policy 0, policy_version 1123722 (0.0008) [2023-12-27 13:26:04,486][20347] Updated weights for policy 0, policy_version 1123732 (0.0007) [2023-12-27 13:26:04,543][20347] Updated weights for policy 0, policy_version 1123742 (0.0006) [2023-12-27 13:26:04,783][20381] Updated weights for policy 1, policy_version 1122588 (0.0007) [2023-12-27 13:26:04,834][20381] Updated weights for policy 1, policy_version 1122598 (0.0005) [2023-12-27 13:26:04,883][20381] Updated weights for policy 1, policy_version 1122608 (0.0007) [2023-12-27 13:26:05,175][20347] Updated weights for policy 0, policy_version 1123752 (0.0009) [2023-12-27 13:26:05,224][20347] Updated weights for policy 0, policy_version 1123762 (0.0010) [2023-12-27 13:26:05,270][20347] Updated weights for policy 0, policy_version 1123772 (0.0005) [2023-12-27 13:26:05,529][20381] Updated weights for policy 1, policy_version 1122618 (0.0009) [2023-12-27 13:26:05,589][20381] Updated weights for policy 1, policy_version 1122628 (0.0006) [2023-12-27 13:26:05,640][20381] Updated weights for policy 1, policy_version 1122638 (0.0010) [2023-12-27 13:26:05,701][20381] Updated weights for policy 1, policy_version 1122648 (0.0010) [2023-12-27 13:26:05,899][20347] Updated weights for policy 0, policy_version 1123782 (0.0009) [2023-12-27 13:26:05,954][20347] Updated weights for policy 0, policy_version 1123792 (0.0010) [2023-12-27 13:26:06,012][20347] Updated weights for policy 0, policy_version 1123802 (0.0010) [2023-12-27 13:26:06,271][20381] Updated weights for policy 1, policy_version 1122658 (0.0011) [2023-12-27 13:26:06,335][20381] Updated weights for policy 1, policy_version 1122668 (0.0009) [2023-12-27 13:26:06,392][20381] Updated weights for policy 1, policy_version 1122678 (0.0006) [2023-12-27 13:26:06,743][20347] Updated weights for policy 0, policy_version 1123812 (0.0009) [2023-12-27 13:26:06,810][20347] Updated weights for policy 0, policy_version 1123822 (0.0008) [2023-12-27 13:26:06,873][20347] Updated weights for policy 0, policy_version 1123832 (0.0008) [2023-12-27 13:26:07,029][20381] Updated weights for policy 1, policy_version 1122688 (0.0005) [2023-12-27 13:26:07,078][20381] Updated weights for policy 1, policy_version 1122698 (0.0005) [2023-12-27 13:26:07,130][20381] Updated weights for policy 1, policy_version 1122708 (0.0008) [2023-12-27 13:26:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 575193088. Throughput: 0: 10252.6, 1: 10380.4. Samples: 575159840. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:07,363][19170] Avg episode reward: [(0, '961.010'), (1, '675.830')] [2023-12-27 13:26:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001122712_287440896.pth... [2023-12-27 13:26:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001123840_287752192.pth... [2023-12-27 13:26:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001122656_287449088.pth [2023-12-27 13:26:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001121496_287129600.pth [2023-12-27 13:26:07,576][20347] Updated weights for policy 0, policy_version 1123842 (0.0009) [2023-12-27 13:26:07,637][20347] Updated weights for policy 0, policy_version 1123852 (0.0010) [2023-12-27 13:26:07,706][20347] Updated weights for policy 0, policy_version 1123862 (0.0010) [2023-12-27 13:26:07,754][20381] Updated weights for policy 1, policy_version 1122718 (0.0007) [2023-12-27 13:26:07,762][20347] Updated weights for policy 0, policy_version 1123872 (0.0010) [2023-12-27 13:26:07,800][20381] Updated weights for policy 1, policy_version 1122728 (0.0005) [2023-12-27 13:26:07,858][20381] Updated weights for policy 1, policy_version 1122738 (0.0005) [2023-12-27 13:26:08,437][20381] Updated weights for policy 1, policy_version 1122748 (0.0007) [2023-12-27 13:26:08,454][20347] Updated weights for policy 0, policy_version 1123882 (0.0010) [2023-12-27 13:26:08,493][20381] Updated weights for policy 1, policy_version 1122758 (0.0008) [2023-12-27 13:26:08,510][20347] Updated weights for policy 0, policy_version 1123892 (0.0010) [2023-12-27 13:26:08,553][20381] Updated weights for policy 1, policy_version 1122768 (0.0008) [2023-12-27 13:26:08,570][20347] Updated weights for policy 0, policy_version 1123902 (0.0011) [2023-12-27 13:26:09,181][20381] Updated weights for policy 1, policy_version 1122778 (0.0006) [2023-12-27 13:26:09,197][20347] Updated weights for policy 0, policy_version 1123912 (0.0007) [2023-12-27 13:26:09,240][20381] Updated weights for policy 1, policy_version 1122788 (0.0007) [2023-12-27 13:26:09,262][20347] Updated weights for policy 0, policy_version 1123922 (0.0008) [2023-12-27 13:26:09,300][20381] Updated weights for policy 1, policy_version 1122798 (0.0006) [2023-12-27 13:26:09,328][20347] Updated weights for policy 0, policy_version 1123932 (0.0008) [2023-12-27 13:26:09,359][20381] Updated weights for policy 1, policy_version 1122808 (0.0007) [2023-12-27 13:26:10,033][20347] Updated weights for policy 0, policy_version 1123942 (0.0010) [2023-12-27 13:26:10,076][20381] Updated weights for policy 1, policy_version 1122818 (0.0006) [2023-12-27 13:26:10,094][20347] Updated weights for policy 0, policy_version 1123952 (0.0011) [2023-12-27 13:26:10,133][20381] Updated weights for policy 1, policy_version 1122828 (0.0006) [2023-12-27 13:26:10,150][20347] Updated weights for policy 0, policy_version 1123962 (0.0010) [2023-12-27 13:26:10,191][20381] Updated weights for policy 1, policy_version 1122838 (0.0008) [2023-12-27 13:26:10,775][20347] Updated weights for policy 0, policy_version 1123972 (0.0010) [2023-12-27 13:26:10,828][20381] Updated weights for policy 1, policy_version 1122848 (0.0009) [2023-12-27 13:26:10,830][20347] Updated weights for policy 0, policy_version 1123982 (0.0008) [2023-12-27 13:26:10,882][20381] Updated weights for policy 1, policy_version 1122858 (0.0008) [2023-12-27 13:26:10,886][20347] Updated weights for policy 0, policy_version 1123992 (0.0005) [2023-12-27 13:26:10,925][20381] Updated weights for policy 1, policy_version 1122868 (0.0007) [2023-12-27 13:26:11,582][20347] Updated weights for policy 0, policy_version 1124002 (0.0006) [2023-12-27 13:26:11,629][20381] Updated weights for policy 1, policy_version 1122878 (0.0008) [2023-12-27 13:26:11,649][20347] Updated weights for policy 0, policy_version 1124012 (0.0009) [2023-12-27 13:26:11,697][20381] Updated weights for policy 1, policy_version 1122888 (0.0007) [2023-12-27 13:26:11,713][20347] Updated weights for policy 0, policy_version 1124022 (0.0009) [2023-12-27 13:26:11,769][20381] Updated weights for policy 1, policy_version 1122898 (0.0009) [2023-12-27 13:26:11,783][20347] Updated weights for policy 0, policy_version 1124032 (0.0007) [2023-12-27 13:26:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 575291392. Throughput: 0: 10263.3, 1: 10447.7. Samples: 575286940. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:12,365][19170] Avg episode reward: [(0, '980.410'), (1, '703.030')] [2023-12-27 13:26:12,466][20347] Updated weights for policy 0, policy_version 1124042 (0.0007) [2023-12-27 13:26:12,480][20381] Updated weights for policy 1, policy_version 1122908 (0.0006) [2023-12-27 13:26:12,531][20347] Updated weights for policy 0, policy_version 1124052 (0.0007) [2023-12-27 13:26:12,544][20381] Updated weights for policy 1, policy_version 1122918 (0.0007) [2023-12-27 13:26:12,583][20347] Updated weights for policy 0, policy_version 1124062 (0.0005) [2023-12-27 13:26:12,602][20381] Updated weights for policy 1, policy_version 1122928 (0.0006) [2023-12-27 13:26:13,170][20347] Updated weights for policy 0, policy_version 1124072 (0.0006) [2023-12-27 13:26:13,218][20347] Updated weights for policy 0, policy_version 1124082 (0.0007) [2023-12-27 13:26:13,270][20347] Updated weights for policy 0, policy_version 1124092 (0.0007) [2023-12-27 13:26:13,346][20381] Updated weights for policy 1, policy_version 1122938 (0.0008) [2023-12-27 13:26:13,400][20381] Updated weights for policy 1, policy_version 1122948 (0.0008) [2023-12-27 13:26:13,452][20381] Updated weights for policy 1, policy_version 1122958 (0.0008) [2023-12-27 13:26:13,508][20381] Updated weights for policy 1, policy_version 1122968 (0.0008) [2023-12-27 13:26:13,941][20347] Updated weights for policy 0, policy_version 1124102 (0.0007) [2023-12-27 13:26:13,988][20347] Updated weights for policy 0, policy_version 1124112 (0.0007) [2023-12-27 13:26:14,036][20347] Updated weights for policy 0, policy_version 1124122 (0.0007) [2023-12-27 13:26:14,192][20381] Updated weights for policy 1, policy_version 1122978 (0.0005) [2023-12-27 13:26:14,251][20381] Updated weights for policy 1, policy_version 1122988 (0.0005) [2023-12-27 13:26:14,309][20381] Updated weights for policy 1, policy_version 1122998 (0.0006) [2023-12-27 13:26:14,732][20347] Updated weights for policy 0, policy_version 1124132 (0.0007) [2023-12-27 13:26:14,801][20347] Updated weights for policy 0, policy_version 1124142 (0.0008) [2023-12-27 13:26:14,829][20381] Updated weights for policy 1, policy_version 1123008 (0.0008) [2023-12-27 13:26:14,849][20347] Updated weights for policy 0, policy_version 1124152 (0.0007) [2023-12-27 13:26:14,884][20381] Updated weights for policy 1, policy_version 1123018 (0.0008) [2023-12-27 13:26:14,940][20381] Updated weights for policy 1, policy_version 1123028 (0.0008) [2023-12-27 13:26:15,510][20347] Updated weights for policy 0, policy_version 1124162 (0.0006) [2023-12-27 13:26:15,567][20347] Updated weights for policy 0, policy_version 1124172 (0.0007) [2023-12-27 13:26:15,621][20347] Updated weights for policy 0, policy_version 1124182 (0.0007) [2023-12-27 13:26:15,673][20347] Updated weights for policy 0, policy_version 1124192 (0.0007) [2023-12-27 13:26:15,693][20381] Updated weights for policy 1, policy_version 1123038 (0.0008) [2023-12-27 13:26:15,747][20381] Updated weights for policy 1, policy_version 1123048 (0.0006) [2023-12-27 13:26:15,805][20381] Updated weights for policy 1, policy_version 1123058 (0.0008) [2023-12-27 13:26:16,300][20347] Updated weights for policy 0, policy_version 1124202 (0.0007) [2023-12-27 13:26:16,350][20347] Updated weights for policy 0, policy_version 1124212 (0.0008) [2023-12-27 13:26:16,406][20347] Updated weights for policy 0, policy_version 1124222 (0.0007) [2023-12-27 13:26:16,424][20381] Updated weights for policy 1, policy_version 1123068 (0.0007) [2023-12-27 13:26:16,469][20381] Updated weights for policy 1, policy_version 1123078 (0.0008) [2023-12-27 13:26:16,515][20381] Updated weights for policy 1, policy_version 1123088 (0.0008) [2023-12-27 13:26:17,081][20347] Updated weights for policy 0, policy_version 1124232 (0.0005) [2023-12-27 13:26:17,130][20347] Updated weights for policy 0, policy_version 1124242 (0.0005) [2023-12-27 13:26:17,191][20347] Updated weights for policy 0, policy_version 1124252 (0.0005) [2023-12-27 13:26:17,219][20381] Updated weights for policy 1, policy_version 1123098 (0.0007) [2023-12-27 13:26:17,272][20381] Updated weights for policy 1, policy_version 1123108 (0.0006) [2023-12-27 13:26:17,333][20381] Updated weights for policy 1, policy_version 1123118 (0.0006) [2023-12-27 13:26:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 575397888. Throughput: 0: 10209.0, 1: 10407.2. Samples: 575410444. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:17,363][19170] Avg episode reward: [(0, '980.470'), (1, '696.240')] [2023-12-27 13:26:17,387][20381] Updated weights for policy 1, policy_version 1123128 (0.0005) [2023-12-27 13:26:17,824][20347] Updated weights for policy 0, policy_version 1124262 (0.0006) [2023-12-27 13:26:17,875][20347] Updated weights for policy 0, policy_version 1124272 (0.0007) [2023-12-27 13:26:17,923][20347] Updated weights for policy 0, policy_version 1124282 (0.0007) [2023-12-27 13:26:18,047][20381] Updated weights for policy 1, policy_version 1123138 (0.0011) [2023-12-27 13:26:18,107][20381] Updated weights for policy 1, policy_version 1123148 (0.0011) [2023-12-27 13:26:18,160][20381] Updated weights for policy 1, policy_version 1123158 (0.0011) [2023-12-27 13:26:18,567][20347] Updated weights for policy 0, policy_version 1124292 (0.0009) [2023-12-27 13:26:18,636][20347] Updated weights for policy 0, policy_version 1124302 (0.0011) [2023-12-27 13:26:18,703][20347] Updated weights for policy 0, policy_version 1124312 (0.0011) [2023-12-27 13:26:18,809][20381] Updated weights for policy 1, policy_version 1123168 (0.0008) [2023-12-27 13:26:18,869][20381] Updated weights for policy 1, policy_version 1123178 (0.0008) [2023-12-27 13:26:18,921][20381] Updated weights for policy 1, policy_version 1123188 (0.0007) [2023-12-27 13:26:19,327][20347] Updated weights for policy 0, policy_version 1124322 (0.0010) [2023-12-27 13:26:19,389][20347] Updated weights for policy 0, policy_version 1124332 (0.0007) [2023-12-27 13:26:19,449][20347] Updated weights for policy 0, policy_version 1124342 (0.0007) [2023-12-27 13:26:19,512][20347] Updated weights for policy 0, policy_version 1124352 (0.0007) [2023-12-27 13:26:19,614][20381] Updated weights for policy 1, policy_version 1123198 (0.0007) [2023-12-27 13:26:19,677][20381] Updated weights for policy 1, policy_version 1123208 (0.0006) [2023-12-27 13:26:19,738][20381] Updated weights for policy 1, policy_version 1123218 (0.0007) [2023-12-27 13:26:20,168][20347] Updated weights for policy 0, policy_version 1124362 (0.0010) [2023-12-27 13:26:20,230][20347] Updated weights for policy 0, policy_version 1124372 (0.0010) [2023-12-27 13:26:20,290][20347] Updated weights for policy 0, policy_version 1124382 (0.0010) [2023-12-27 13:26:20,400][20381] Updated weights for policy 1, policy_version 1123228 (0.0007) [2023-12-27 13:26:20,458][20381] Updated weights for policy 1, policy_version 1123238 (0.0006) [2023-12-27 13:26:20,522][20381] Updated weights for policy 1, policy_version 1123248 (0.0006) [2023-12-27 13:26:20,969][20347] Updated weights for policy 0, policy_version 1124392 (0.0007) [2023-12-27 13:26:21,027][20347] Updated weights for policy 0, policy_version 1124402 (0.0007) [2023-12-27 13:26:21,094][20347] Updated weights for policy 0, policy_version 1124412 (0.0008) [2023-12-27 13:26:21,203][20381] Updated weights for policy 1, policy_version 1123258 (0.0007) [2023-12-27 13:26:21,268][20381] Updated weights for policy 1, policy_version 1123268 (0.0008) [2023-12-27 13:26:21,335][20381] Updated weights for policy 1, policy_version 1123278 (0.0007) [2023-12-27 13:26:21,410][20381] Updated weights for policy 1, policy_version 1123288 (0.0008) [2023-12-27 13:26:21,893][20347] Updated weights for policy 0, policy_version 1124422 (0.0008) [2023-12-27 13:26:21,952][20347] Updated weights for policy 0, policy_version 1124432 (0.0008) [2023-12-27 13:26:21,958][20291] KL-divergence is very high: 122.5686 [2023-12-27 13:26:21,977][20291] KL-divergence is very high: 458.7978 [2023-12-27 13:26:22,005][20291] KL-divergence is very high: 141.0899 [2023-12-27 13:26:22,012][20347] Updated weights for policy 0, policy_version 1124442 (0.0008) [2023-12-27 13:26:22,021][20291] KL-divergence is very high: 273.0479 [2023-12-27 13:26:22,039][20381] Updated weights for policy 1, policy_version 1123298 (0.0007) [2023-12-27 13:26:22,099][20381] Updated weights for policy 1, policy_version 1123308 (0.0007) [2023-12-27 13:26:22,156][20381] Updated weights for policy 1, policy_version 1123318 (0.0007) [2023-12-27 13:26:22,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 575504384. Throughput: 0: 10216.3, 1: 10403.1. Samples: 575473156. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:22,364][19170] Avg episode reward: [(0, '980.500'), (1, '637.490')] [2023-12-27 13:26:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001123320_287596544.pth... [2023-12-27 13:26:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001122104_287285248.pth [2023-12-27 13:26:22,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001124448_287907840.pth... [2023-12-27 13:26:22,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001123264_287604736.pth [2023-12-27 13:26:22,845][20347] Updated weights for policy 0, policy_version 1124452 (0.0008) [2023-12-27 13:26:22,868][20381] Updated weights for policy 1, policy_version 1123328 (0.0009) [2023-12-27 13:26:22,902][20347] Updated weights for policy 0, policy_version 1124462 (0.0008) [2023-12-27 13:26:22,926][20381] Updated weights for policy 1, policy_version 1123338 (0.0008) [2023-12-27 13:26:22,960][20347] Updated weights for policy 0, policy_version 1124472 (0.0006) [2023-12-27 13:26:22,981][20381] Updated weights for policy 1, policy_version 1123348 (0.0006) [2023-12-27 13:26:23,626][20381] Updated weights for policy 1, policy_version 1123358 (0.0007) [2023-12-27 13:26:23,672][20347] Updated weights for policy 0, policy_version 1124482 (0.0006) [2023-12-27 13:26:23,681][20381] Updated weights for policy 1, policy_version 1123368 (0.0006) [2023-12-27 13:26:23,724][20347] Updated weights for policy 0, policy_version 1124492 (0.0007) [2023-12-27 13:26:23,750][20381] Updated weights for policy 1, policy_version 1123378 (0.0005) [2023-12-27 13:26:23,777][20347] Updated weights for policy 0, policy_version 1124502 (0.0008) [2023-12-27 13:26:23,829][20347] Updated weights for policy 0, policy_version 1124512 (0.0008) [2023-12-27 13:26:24,322][20381] Updated weights for policy 1, policy_version 1123388 (0.0007) [2023-12-27 13:26:24,380][20381] Updated weights for policy 1, policy_version 1123398 (0.0007) [2023-12-27 13:26:24,435][20381] Updated weights for policy 1, policy_version 1123408 (0.0006) [2023-12-27 13:26:24,526][20347] Updated weights for policy 0, policy_version 1124522 (0.0009) [2023-12-27 13:26:24,571][20347] Updated weights for policy 0, policy_version 1124532 (0.0010) [2023-12-27 13:26:24,616][20347] Updated weights for policy 0, policy_version 1124542 (0.0010) [2023-12-27 13:26:25,029][20381] Updated weights for policy 1, policy_version 1123418 (0.0008) [2023-12-27 13:26:25,077][20381] Updated weights for policy 1, policy_version 1123428 (0.0007) [2023-12-27 13:26:25,132][20381] Updated weights for policy 1, policy_version 1123438 (0.0008) [2023-12-27 13:26:25,190][20381] Updated weights for policy 1, policy_version 1123448 (0.0007) [2023-12-27 13:26:25,310][20347] Updated weights for policy 0, policy_version 1124552 (0.0009) [2023-12-27 13:26:25,357][20347] Updated weights for policy 0, policy_version 1124562 (0.0010) [2023-12-27 13:26:25,404][20347] Updated weights for policy 0, policy_version 1124572 (0.0009) [2023-12-27 13:26:25,835][20381] Updated weights for policy 1, policy_version 1123458 (0.0007) [2023-12-27 13:26:25,888][20381] Updated weights for policy 1, policy_version 1123468 (0.0007) [2023-12-27 13:26:25,936][20381] Updated weights for policy 1, policy_version 1123478 (0.0008) [2023-12-27 13:26:26,136][20347] Updated weights for policy 0, policy_version 1124582 (0.0009) [2023-12-27 13:26:26,193][20347] Updated weights for policy 0, policy_version 1124592 (0.0010) [2023-12-27 13:26:26,253][20347] Updated weights for policy 0, policy_version 1124602 (0.0010) [2023-12-27 13:26:26,641][20381] Updated weights for policy 1, policy_version 1123488 (0.0010) [2023-12-27 13:26:26,702][20381] Updated weights for policy 1, policy_version 1123498 (0.0010) [2023-12-27 13:26:26,755][20381] Updated weights for policy 1, policy_version 1123508 (0.0010) [2023-12-27 13:26:26,811][20347] Updated weights for policy 0, policy_version 1124612 (0.0008) [2023-12-27 13:26:26,872][20347] Updated weights for policy 0, policy_version 1124622 (0.0007) [2023-12-27 13:26:26,928][20347] Updated weights for policy 0, policy_version 1124632 (0.0007) [2023-12-27 13:26:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 575602688. Throughput: 0: 10138.6, 1: 10462.0. Samples: 575594104. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:27,363][19170] Avg episode reward: [(0, '960.890'), (1, '627.360')] [2023-12-27 13:26:27,374][20381] Updated weights for policy 1, policy_version 1123518 (0.0008) [2023-12-27 13:26:27,448][20381] Updated weights for policy 1, policy_version 1123529 (0.0007) [2023-12-27 13:26:27,518][20381] Updated weights for policy 1, policy_version 1123539 (0.0008) [2023-12-27 13:26:27,533][20347] Updated weights for policy 0, policy_version 1124642 (0.0009) [2023-12-27 13:26:27,598][20347] Updated weights for policy 0, policy_version 1124652 (0.0006) [2023-12-27 13:26:27,657][20347] Updated weights for policy 0, policy_version 1124662 (0.0005) [2023-12-27 13:26:27,709][20347] Updated weights for policy 0, policy_version 1124672 (0.0007) [2023-12-27 13:26:28,201][20381] Updated weights for policy 1, policy_version 1123549 (0.0008) [2023-12-27 13:26:28,260][20381] Updated weights for policy 1, policy_version 1123559 (0.0007) [2023-12-27 13:26:28,285][20347] Updated weights for policy 0, policy_version 1124682 (0.0006) [2023-12-27 13:26:28,317][20381] Updated weights for policy 1, policy_version 1123569 (0.0005) [2023-12-27 13:26:28,346][20347] Updated weights for policy 0, policy_version 1124692 (0.0007) [2023-12-27 13:26:28,408][20347] Updated weights for policy 0, policy_version 1124702 (0.0008) [2023-12-27 13:26:28,947][20381] Updated weights for policy 1, policy_version 1123579 (0.0008) [2023-12-27 13:26:29,004][20381] Updated weights for policy 1, policy_version 1123589 (0.0006) [2023-12-27 13:26:29,061][20381] Updated weights for policy 1, policy_version 1123599 (0.0005) [2023-12-27 13:26:29,151][20347] Updated weights for policy 0, policy_version 1124712 (0.0008) [2023-12-27 13:26:29,210][20347] Updated weights for policy 0, policy_version 1124722 (0.0008) [2023-12-27 13:26:29,268][20347] Updated weights for policy 0, policy_version 1124732 (0.0009) [2023-12-27 13:26:29,697][20381] Updated weights for policy 1, policy_version 1123609 (0.0006) [2023-12-27 13:26:29,759][20381] Updated weights for policy 1, policy_version 1123619 (0.0007) [2023-12-27 13:26:29,813][20381] Updated weights for policy 1, policy_version 1123629 (0.0007) [2023-12-27 13:26:29,883][20381] Updated weights for policy 1, policy_version 1123639 (0.0007) [2023-12-27 13:26:29,979][20347] Updated weights for policy 0, policy_version 1124742 (0.0008) [2023-12-27 13:26:30,028][20347] Updated weights for policy 0, policy_version 1124752 (0.0008) [2023-12-27 13:26:30,078][20347] Updated weights for policy 0, policy_version 1124762 (0.0007) [2023-12-27 13:26:30,500][20381] Updated weights for policy 1, policy_version 1123649 (0.0006) [2023-12-27 13:26:30,556][20381] Updated weights for policy 1, policy_version 1123659 (0.0006) [2023-12-27 13:26:30,602][20381] Updated weights for policy 1, policy_version 1123669 (0.0006) [2023-12-27 13:26:30,818][20347] Updated weights for policy 0, policy_version 1124772 (0.0008) [2023-12-27 13:26:30,878][20347] Updated weights for policy 0, policy_version 1124782 (0.0007) [2023-12-27 13:26:30,944][20347] Updated weights for policy 0, policy_version 1124792 (0.0008) [2023-12-27 13:26:31,276][20381] Updated weights for policy 1, policy_version 1123679 (0.0009) [2023-12-27 13:26:31,338][20381] Updated weights for policy 1, policy_version 1123689 (0.0008) [2023-12-27 13:26:31,401][20381] Updated weights for policy 1, policy_version 1123699 (0.0009) [2023-12-27 13:26:31,582][20347] Updated weights for policy 0, policy_version 1124802 (0.0008) [2023-12-27 13:26:31,647][20347] Updated weights for policy 0, policy_version 1124812 (0.0007) [2023-12-27 13:26:31,708][20347] Updated weights for policy 0, policy_version 1124822 (0.0008) [2023-12-27 13:26:31,775][20347] Updated weights for policy 0, policy_version 1124832 (0.0008) [2023-12-27 13:26:32,115][20381] Updated weights for policy 1, policy_version 1123709 (0.0008) [2023-12-27 13:26:32,173][20381] Updated weights for policy 1, policy_version 1123719 (0.0007) [2023-12-27 13:26:32,234][20381] Updated weights for policy 1, policy_version 1123729 (0.0005) [2023-12-27 13:26:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 575709184. Throughput: 0: 10164.9, 1: 10467.2. Samples: 575718488. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:32,363][19170] Avg episode reward: [(0, '941.410'), (1, '598.140')] [2023-12-27 13:26:32,429][20347] Updated weights for policy 0, policy_version 1124842 (0.0008) [2023-12-27 13:26:32,485][20347] Updated weights for policy 0, policy_version 1124852 (0.0007) [2023-12-27 13:26:32,537][20347] Updated weights for policy 0, policy_version 1124862 (0.0007) [2023-12-27 13:26:32,903][20381] Updated weights for policy 1, policy_version 1123739 (0.0008) [2023-12-27 13:26:32,957][20381] Updated weights for policy 1, policy_version 1123749 (0.0008) [2023-12-27 13:26:33,006][20381] Updated weights for policy 1, policy_version 1123759 (0.0008) [2023-12-27 13:26:33,235][20347] Updated weights for policy 0, policy_version 1124872 (0.0007) [2023-12-27 13:26:33,293][20347] Updated weights for policy 0, policy_version 1124882 (0.0010) [2023-12-27 13:26:33,353][20347] Updated weights for policy 0, policy_version 1124892 (0.0010) [2023-12-27 13:26:33,740][20381] Updated weights for policy 1, policy_version 1123769 (0.0008) [2023-12-27 13:26:33,793][20381] Updated weights for policy 1, policy_version 1123779 (0.0009) [2023-12-27 13:26:33,848][20381] Updated weights for policy 1, policy_version 1123789 (0.0006) [2023-12-27 13:26:33,895][20381] Updated weights for policy 1, policy_version 1123799 (0.0007) [2023-12-27 13:26:33,933][20347] Updated weights for policy 0, policy_version 1124902 (0.0007) [2023-12-27 13:26:33,987][20347] Updated weights for policy 0, policy_version 1124912 (0.0010) [2023-12-27 13:26:34,038][20347] Updated weights for policy 0, policy_version 1124922 (0.0010) [2023-12-27 13:26:34,502][20381] Updated weights for policy 1, policy_version 1123809 (0.0007) [2023-12-27 13:26:34,559][20381] Updated weights for policy 1, policy_version 1123819 (0.0007) [2023-12-27 13:26:34,619][20381] Updated weights for policy 1, policy_version 1123829 (0.0008) [2023-12-27 13:26:34,718][20347] Updated weights for policy 0, policy_version 1124932 (0.0009) [2023-12-27 13:26:34,785][20347] Updated weights for policy 0, policy_version 1124942 (0.0008) [2023-12-27 13:26:34,851][20347] Updated weights for policy 0, policy_version 1124952 (0.0008) [2023-12-27 13:26:35,194][20381] Updated weights for policy 1, policy_version 1123839 (0.0006) [2023-12-27 13:26:35,242][20381] Updated weights for policy 1, policy_version 1123849 (0.0005) [2023-12-27 13:26:35,290][20381] Updated weights for policy 1, policy_version 1123859 (0.0005) [2023-12-27 13:26:35,583][20347] Updated weights for policy 0, policy_version 1124962 (0.0008) [2023-12-27 13:26:35,631][20347] Updated weights for policy 0, policy_version 1124972 (0.0008) [2023-12-27 13:26:35,681][20347] Updated weights for policy 0, policy_version 1124982 (0.0008) [2023-12-27 13:26:35,731][20347] Updated weights for policy 0, policy_version 1124992 (0.0007) [2023-12-27 13:26:35,844][20381] Updated weights for policy 1, policy_version 1123869 (0.0007) [2023-12-27 13:26:35,902][20381] Updated weights for policy 1, policy_version 1123879 (0.0007) [2023-12-27 13:26:35,956][20381] Updated weights for policy 1, policy_version 1123889 (0.0007) [2023-12-27 13:26:36,433][20347] Updated weights for policy 0, policy_version 1125002 (0.0010) [2023-12-27 13:26:36,487][20347] Updated weights for policy 0, policy_version 1125012 (0.0008) [2023-12-27 13:26:36,547][20347] Updated weights for policy 0, policy_version 1125022 (0.0008) [2023-12-27 13:26:36,606][20381] Updated weights for policy 1, policy_version 1123899 (0.0008) [2023-12-27 13:26:36,664][20381] Updated weights for policy 1, policy_version 1123909 (0.0008) [2023-12-27 13:26:36,725][20381] Updated weights for policy 1, policy_version 1123919 (0.0008) [2023-12-27 13:26:37,195][20347] Updated weights for policy 0, policy_version 1125032 (0.0006) [2023-12-27 13:26:37,247][20347] Updated weights for policy 0, policy_version 1125042 (0.0005) [2023-12-27 13:26:37,299][20347] Updated weights for policy 0, policy_version 1125052 (0.0005) [2023-12-27 13:26:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 575815680. Throughput: 0: 10195.3, 1: 10495.0. Samples: 575781888. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:37,363][19170] Avg episode reward: [(0, '931.730'), (1, '686.450')] [2023-12-27 13:26:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001123928_287752192.pth... [2023-12-27 13:26:37,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001125056_288063488.pth... [2023-12-27 13:26:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001122712_287440896.pth [2023-12-27 13:26:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001123840_287752192.pth [2023-12-27 13:26:37,451][20381] Updated weights for policy 1, policy_version 1123929 (0.0007) [2023-12-27 13:26:37,518][20381] Updated weights for policy 1, policy_version 1123939 (0.0010) [2023-12-27 13:26:37,584][20381] Updated weights for policy 1, policy_version 1123949 (0.0011) [2023-12-27 13:26:37,650][20381] Updated weights for policy 1, policy_version 1123959 (0.0011) [2023-12-27 13:26:37,844][20347] Updated weights for policy 0, policy_version 1125062 (0.0006) [2023-12-27 13:26:37,899][20347] Updated weights for policy 0, policy_version 1125072 (0.0005) [2023-12-27 13:26:37,961][20347] Updated weights for policy 0, policy_version 1125082 (0.0006) [2023-12-27 13:26:38,325][20381] Updated weights for policy 1, policy_version 1123969 (0.0011) [2023-12-27 13:26:38,386][20381] Updated weights for policy 1, policy_version 1123979 (0.0011) [2023-12-27 13:26:38,437][20381] Updated weights for policy 1, policy_version 1123989 (0.0011) [2023-12-27 13:26:38,504][20347] Updated weights for policy 0, policy_version 1125092 (0.0005) [2023-12-27 13:26:38,566][20347] Updated weights for policy 0, policy_version 1125102 (0.0008) [2023-12-27 13:26:38,626][20347] Updated weights for policy 0, policy_version 1125112 (0.0008) [2023-12-27 13:26:39,140][20381] Updated weights for policy 1, policy_version 1123999 (0.0011) [2023-12-27 13:26:39,184][20381] Updated weights for policy 1, policy_version 1124009 (0.0010) [2023-12-27 13:26:39,242][20381] Updated weights for policy 1, policy_version 1124019 (0.0010) [2023-12-27 13:26:39,313][20347] Updated weights for policy 0, policy_version 1125122 (0.0008) [2023-12-27 13:26:39,381][20347] Updated weights for policy 0, policy_version 1125132 (0.0010) [2023-12-27 13:26:39,446][20347] Updated weights for policy 0, policy_version 1125142 (0.0007) [2023-12-27 13:26:39,506][20347] Updated weights for policy 0, policy_version 1125152 (0.0006) [2023-12-27 13:26:40,035][20381] Updated weights for policy 1, policy_version 1124029 (0.0011) [2023-12-27 13:26:40,099][20381] Updated weights for policy 1, policy_version 1124039 (0.0011) [2023-12-27 13:26:40,150][20381] Updated weights for policy 1, policy_version 1124049 (0.0011) [2023-12-27 13:26:40,153][20347] Updated weights for policy 0, policy_version 1125162 (0.0008) [2023-12-27 13:26:40,214][20347] Updated weights for policy 0, policy_version 1125172 (0.0007) [2023-12-27 13:26:40,262][20347] Updated weights for policy 0, policy_version 1125182 (0.0006) [2023-12-27 13:26:40,867][20381] Updated weights for policy 1, policy_version 1124059 (0.0011) [2023-12-27 13:26:40,905][20347] Updated weights for policy 0, policy_version 1125192 (0.0005) [2023-12-27 13:26:40,922][20381] Updated weights for policy 1, policy_version 1124069 (0.0011) [2023-12-27 13:26:40,962][20347] Updated weights for policy 0, policy_version 1125202 (0.0006) [2023-12-27 13:26:40,976][20381] Updated weights for policy 1, policy_version 1124079 (0.0011) [2023-12-27 13:26:41,018][20347] Updated weights for policy 0, policy_version 1125212 (0.0006) [2023-12-27 13:26:41,706][20381] Updated weights for policy 1, policy_version 1124089 (0.0011) [2023-12-27 13:26:41,770][20381] Updated weights for policy 1, policy_version 1124099 (0.0009) [2023-12-27 13:26:41,831][20381] Updated weights for policy 1, policy_version 1124109 (0.0011) [2023-12-27 13:26:41,833][20347] Updated weights for policy 0, policy_version 1125222 (0.0006) [2023-12-27 13:26:41,889][20381] Updated weights for policy 1, policy_version 1124119 (0.0011) [2023-12-27 13:26:41,890][20347] Updated weights for policy 0, policy_version 1125232 (0.0006) [2023-12-27 13:26:41,943][20347] Updated weights for policy 0, policy_version 1125242 (0.0007) [2023-12-27 13:26:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 575913984. Throughput: 0: 10242.0, 1: 10424.8. Samples: 575905620. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:42,363][19170] Avg episode reward: [(0, '941.420'), (1, '774.690')] [2023-12-27 13:26:42,579][20381] Updated weights for policy 1, policy_version 1124129 (0.0010) [2023-12-27 13:26:42,639][20381] Updated weights for policy 1, policy_version 1124139 (0.0007) [2023-12-27 13:26:42,644][20347] Updated weights for policy 0, policy_version 1125252 (0.0008) [2023-12-27 13:26:42,698][20381] Updated weights for policy 1, policy_version 1124149 (0.0006) [2023-12-27 13:26:42,698][20347] Updated weights for policy 0, policy_version 1125262 (0.0008) [2023-12-27 13:26:42,762][20347] Updated weights for policy 0, policy_version 1125272 (0.0008) [2023-12-27 13:26:43,371][20381] Updated weights for policy 1, policy_version 1124159 (0.0007) [2023-12-27 13:26:43,420][20381] Updated weights for policy 1, policy_version 1124169 (0.0006) [2023-12-27 13:26:43,470][20381] Updated weights for policy 1, policy_version 1124179 (0.0005) [2023-12-27 13:26:43,480][20347] Updated weights for policy 0, policy_version 1125282 (0.0007) [2023-12-27 13:26:43,532][20347] Updated weights for policy 0, policy_version 1125292 (0.0007) [2023-12-27 13:26:43,585][20347] Updated weights for policy 0, policy_version 1125302 (0.0007) [2023-12-27 13:26:43,643][20347] Updated weights for policy 0, policy_version 1125312 (0.0007) [2023-12-27 13:26:44,107][20381] Updated weights for policy 1, policy_version 1124189 (0.0007) [2023-12-27 13:26:44,167][20381] Updated weights for policy 1, policy_version 1124199 (0.0007) [2023-12-27 13:26:44,220][20381] Updated weights for policy 1, policy_version 1124209 (0.0007) [2023-12-27 13:26:44,348][20347] Updated weights for policy 0, policy_version 1125322 (0.0007) [2023-12-27 13:26:44,399][20347] Updated weights for policy 0, policy_version 1125332 (0.0008) [2023-12-27 13:26:44,444][20347] Updated weights for policy 0, policy_version 1125342 (0.0007) [2023-12-27 13:26:44,892][20381] Updated weights for policy 1, policy_version 1124219 (0.0008) [2023-12-27 13:26:44,950][20381] Updated weights for policy 1, policy_version 1124229 (0.0008) [2023-12-27 13:26:45,001][20381] Updated weights for policy 1, policy_version 1124239 (0.0008) [2023-12-27 13:26:45,176][20347] Updated weights for policy 0, policy_version 1125352 (0.0007) [2023-12-27 13:26:45,231][20347] Updated weights for policy 0, policy_version 1125362 (0.0007) [2023-12-27 13:26:45,285][20347] Updated weights for policy 0, policy_version 1125372 (0.0007) [2023-12-27 13:26:45,720][20381] Updated weights for policy 1, policy_version 1124249 (0.0008) [2023-12-27 13:26:45,775][20381] Updated weights for policy 1, policy_version 1124259 (0.0008) [2023-12-27 13:26:45,827][20381] Updated weights for policy 1, policy_version 1124269 (0.0008) [2023-12-27 13:26:45,886][20381] Updated weights for policy 1, policy_version 1124279 (0.0008) [2023-12-27 13:26:46,002][20347] Updated weights for policy 0, policy_version 1125382 (0.0008) [2023-12-27 13:26:46,048][20347] Updated weights for policy 0, policy_version 1125392 (0.0007) [2023-12-27 13:26:46,092][20347] Updated weights for policy 0, policy_version 1125402 (0.0005) [2023-12-27 13:26:46,468][20381] Updated weights for policy 1, policy_version 1124289 (0.0007) [2023-12-27 13:26:46,525][20381] Updated weights for policy 1, policy_version 1124299 (0.0007) [2023-12-27 13:26:46,583][20381] Updated weights for policy 1, policy_version 1124309 (0.0008) [2023-12-27 13:26:46,829][20347] Updated weights for policy 0, policy_version 1125412 (0.0008) [2023-12-27 13:26:46,875][20347] Updated weights for policy 0, policy_version 1125422 (0.0010) [2023-12-27 13:26:46,929][20347] Updated weights for policy 0, policy_version 1125432 (0.0009) [2023-12-27 13:26:47,218][20381] Updated weights for policy 1, policy_version 1124319 (0.0008) [2023-12-27 13:26:47,267][20381] Updated weights for policy 1, policy_version 1124329 (0.0007) [2023-12-27 13:26:47,312][20381] Updated weights for policy 1, policy_version 1124339 (0.0007) [2023-12-27 13:26:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 576020480. Throughput: 0: 10218.6, 1: 10440.9. Samples: 576026364. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:47,363][19170] Avg episode reward: [(0, '951.290'), (1, '735.540')] [2023-12-27 13:26:47,609][20347] Updated weights for policy 0, policy_version 1125442 (0.0008) [2023-12-27 13:26:47,671][20347] Updated weights for policy 0, policy_version 1125452 (0.0008) [2023-12-27 13:26:47,729][20347] Updated weights for policy 0, policy_version 1125462 (0.0008) [2023-12-27 13:26:47,775][20347] Updated weights for policy 0, policy_version 1125472 (0.0007) [2023-12-27 13:26:48,039][20381] Updated weights for policy 1, policy_version 1124349 (0.0008) [2023-12-27 13:26:48,097][20381] Updated weights for policy 1, policy_version 1124359 (0.0008) [2023-12-27 13:26:48,148][20381] Updated weights for policy 1, policy_version 1124369 (0.0008) [2023-12-27 13:26:48,535][20347] Updated weights for policy 0, policy_version 1125482 (0.0008) [2023-12-27 13:26:48,596][20347] Updated weights for policy 0, policy_version 1125492 (0.0007) [2023-12-27 13:26:48,653][20347] Updated weights for policy 0, policy_version 1125502 (0.0007) [2023-12-27 13:26:48,859][20381] Updated weights for policy 1, policy_version 1124379 (0.0007) [2023-12-27 13:26:48,926][20381] Updated weights for policy 1, policy_version 1124389 (0.0008) [2023-12-27 13:26:48,985][20381] Updated weights for policy 1, policy_version 1124399 (0.0006) [2023-12-27 13:26:49,371][20347] Updated weights for policy 0, policy_version 1125512 (0.0009) [2023-12-27 13:26:49,435][20347] Updated weights for policy 0, policy_version 1125522 (0.0010) [2023-12-27 13:26:49,489][20347] Updated weights for policy 0, policy_version 1125532 (0.0010) [2023-12-27 13:26:49,668][20381] Updated weights for policy 1, policy_version 1124409 (0.0008) [2023-12-27 13:26:49,727][20381] Updated weights for policy 1, policy_version 1124419 (0.0007) [2023-12-27 13:26:49,789][20381] Updated weights for policy 1, policy_version 1124429 (0.0006) [2023-12-27 13:26:49,853][20381] Updated weights for policy 1, policy_version 1124439 (0.0007) [2023-12-27 13:26:50,235][20347] Updated weights for policy 0, policy_version 1125542 (0.0009) [2023-12-27 13:26:50,295][20347] Updated weights for policy 0, policy_version 1125552 (0.0008) [2023-12-27 13:26:50,354][20347] Updated weights for policy 0, policy_version 1125562 (0.0008) [2023-12-27 13:26:50,511][20381] Updated weights for policy 1, policy_version 1124449 (0.0007) [2023-12-27 13:26:50,567][20381] Updated weights for policy 1, policy_version 1124459 (0.0006) [2023-12-27 13:26:50,628][20381] Updated weights for policy 1, policy_version 1124469 (0.0006) [2023-12-27 13:26:51,062][20347] Updated weights for policy 0, policy_version 1125572 (0.0008) [2023-12-27 13:26:51,120][20347] Updated weights for policy 0, policy_version 1125582 (0.0006) [2023-12-27 13:26:51,182][20347] Updated weights for policy 0, policy_version 1125592 (0.0005) [2023-12-27 13:26:51,331][20381] Updated weights for policy 1, policy_version 1124479 (0.0007) [2023-12-27 13:26:51,398][20381] Updated weights for policy 1, policy_version 1124489 (0.0008) [2023-12-27 13:26:51,453][20381] Updated weights for policy 1, policy_version 1124499 (0.0008) [2023-12-27 13:26:51,912][20347] Updated weights for policy 0, policy_version 1125602 (0.0007) [2023-12-27 13:26:51,967][20347] Updated weights for policy 0, policy_version 1125612 (0.0007) [2023-12-27 13:26:52,029][20347] Updated weights for policy 0, policy_version 1125622 (0.0008) [2023-12-27 13:26:52,089][20347] Updated weights for policy 0, policy_version 1125632 (0.0008) [2023-12-27 13:26:52,130][20381] Updated weights for policy 1, policy_version 1124509 (0.0007) [2023-12-27 13:26:52,184][20381] Updated weights for policy 1, policy_version 1124519 (0.0008) [2023-12-27 13:26:52,249][20381] Updated weights for policy 1, policy_version 1124529 (0.0008) [2023-12-27 13:26:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 576118784. Throughput: 0: 10181.9, 1: 10393.2. Samples: 576085720. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:52,363][19170] Avg episode reward: [(0, '951.120'), (1, '657.100')] [2023-12-27 13:26:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001124536_287907840.pth... [2023-12-27 13:26:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001125632_288210944.pth... [2023-12-27 13:26:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001123320_287596544.pth [2023-12-27 13:26:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001124448_287907840.pth [2023-12-27 13:26:52,828][20347] Updated weights for policy 0, policy_version 1125642 (0.0008) [2023-12-27 13:26:52,830][20291] KL-divergence is very high: 137.6535 [2023-12-27 13:26:52,876][20291] KL-divergence is very high: 115.8578 [2023-12-27 13:26:52,887][20347] Updated weights for policy 0, policy_version 1125652 (0.0008) [2023-12-27 13:26:52,906][20381] Updated weights for policy 1, policy_version 1124539 (0.0008) [2023-12-27 13:26:52,946][20347] Updated weights for policy 0, policy_version 1125662 (0.0008) [2023-12-27 13:26:52,961][20381] Updated weights for policy 1, policy_version 1124549 (0.0008) [2023-12-27 13:26:53,019][20381] Updated weights for policy 1, policy_version 1124559 (0.0009) [2023-12-27 13:26:53,577][20347] Updated weights for policy 0, policy_version 1125672 (0.0008) [2023-12-27 13:26:53,623][20347] Updated weights for policy 0, policy_version 1125682 (0.0009) [2023-12-27 13:26:53,668][20347] Updated weights for policy 0, policy_version 1125692 (0.0009) [2023-12-27 13:26:53,693][20381] Updated weights for policy 1, policy_version 1124569 (0.0008) [2023-12-27 13:26:53,757][20381] Updated weights for policy 1, policy_version 1124579 (0.0005) [2023-12-27 13:26:53,821][20381] Updated weights for policy 1, policy_version 1124589 (0.0006) [2023-12-27 13:26:53,882][20381] Updated weights for policy 1, policy_version 1124599 (0.0008) [2023-12-27 13:26:54,356][20347] Updated weights for policy 0, policy_version 1125702 (0.0007) [2023-12-27 13:26:54,418][20347] Updated weights for policy 0, policy_version 1125712 (0.0007) [2023-12-27 13:26:54,473][20381] Updated weights for policy 1, policy_version 1124609 (0.0007) [2023-12-27 13:26:54,475][20347] Updated weights for policy 0, policy_version 1125722 (0.0006) [2023-12-27 13:26:54,528][20381] Updated weights for policy 1, policy_version 1124619 (0.0005) [2023-12-27 13:26:54,576][20381] Updated weights for policy 1, policy_version 1124629 (0.0005) [2023-12-27 13:26:55,173][20347] Updated weights for policy 0, policy_version 1125732 (0.0006) [2023-12-27 13:26:55,189][20381] Updated weights for policy 1, policy_version 1124639 (0.0005) [2023-12-27 13:26:55,226][20347] Updated weights for policy 0, policy_version 1125742 (0.0006) [2023-12-27 13:26:55,245][20381] Updated weights for policy 1, policy_version 1124649 (0.0007) [2023-12-27 13:26:55,278][20347] Updated weights for policy 0, policy_version 1125752 (0.0006) [2023-12-27 13:26:55,304][20381] Updated weights for policy 1, policy_version 1124659 (0.0007) [2023-12-27 13:26:55,879][20347] Updated weights for policy 0, policy_version 1125762 (0.0006) [2023-12-27 13:26:55,919][20381] Updated weights for policy 1, policy_version 1124669 (0.0008) [2023-12-27 13:26:55,938][20347] Updated weights for policy 0, policy_version 1125772 (0.0006) [2023-12-27 13:26:55,978][20381] Updated weights for policy 1, policy_version 1124679 (0.0007) [2023-12-27 13:26:55,996][20347] Updated weights for policy 0, policy_version 1125782 (0.0006) [2023-12-27 13:26:56,031][20381] Updated weights for policy 1, policy_version 1124689 (0.0006) [2023-12-27 13:26:56,049][20347] Updated weights for policy 0, policy_version 1125792 (0.0006) [2023-12-27 13:26:56,638][20347] Updated weights for policy 0, policy_version 1125802 (0.0010) [2023-12-27 13:26:56,692][20347] Updated weights for policy 0, policy_version 1125812 (0.0010) [2023-12-27 13:26:56,694][20381] Updated weights for policy 1, policy_version 1124699 (0.0006) [2023-12-27 13:26:56,748][20381] Updated weights for policy 1, policy_version 1124709 (0.0005) [2023-12-27 13:26:56,749][20347] Updated weights for policy 0, policy_version 1125822 (0.0010) [2023-12-27 13:26:56,797][20381] Updated weights for policy 1, policy_version 1124719 (0.0007) [2023-12-27 13:26:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 576217088. Throughput: 0: 10153.5, 1: 10340.4. Samples: 576209160. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:26:57,363][19170] Avg episode reward: [(0, '931.570'), (1, '676.870')] [2023-12-27 13:26:57,383][20347] Updated weights for policy 0, policy_version 1125832 (0.0010) [2023-12-27 13:26:57,434][20347] Updated weights for policy 0, policy_version 1125842 (0.0010) [2023-12-27 13:26:57,482][20347] Updated weights for policy 0, policy_version 1125852 (0.0010) [2023-12-27 13:26:57,513][20381] Updated weights for policy 1, policy_version 1124729 (0.0007) [2023-12-27 13:26:57,566][20381] Updated weights for policy 1, policy_version 1124739 (0.0005) [2023-12-27 13:26:57,619][20381] Updated weights for policy 1, policy_version 1124749 (0.0005) [2023-12-27 13:26:57,669][20381] Updated weights for policy 1, policy_version 1124759 (0.0005) [2023-12-27 13:26:58,167][20347] Updated weights for policy 0, policy_version 1125862 (0.0010) [2023-12-27 13:26:58,230][20347] Updated weights for policy 0, policy_version 1125872 (0.0010) [2023-12-27 13:26:58,291][20347] Updated weights for policy 0, policy_version 1125882 (0.0006) [2023-12-27 13:26:58,331][20381] Updated weights for policy 1, policy_version 1124769 (0.0007) [2023-12-27 13:26:58,390][20381] Updated weights for policy 1, policy_version 1124779 (0.0008) [2023-12-27 13:26:58,452][20381] Updated weights for policy 1, policy_version 1124789 (0.0008) [2023-12-27 13:26:59,133][20347] Updated weights for policy 0, policy_version 1125892 (0.0009) [2023-12-27 13:26:59,192][20347] Updated weights for policy 0, policy_version 1125902 (0.0007) [2023-12-27 13:26:59,247][20381] Updated weights for policy 1, policy_version 1124799 (0.0011) [2023-12-27 13:26:59,262][20347] Updated weights for policy 0, policy_version 1125913 (0.0010) [2023-12-27 13:26:59,308][20381] Updated weights for policy 1, policy_version 1124809 (0.0011) [2023-12-27 13:26:59,372][20381] Updated weights for policy 1, policy_version 1124819 (0.0007) [2023-12-27 13:27:00,046][20347] Updated weights for policy 0, policy_version 1125923 (0.0008) [2023-12-27 13:27:00,100][20347] Updated weights for policy 0, policy_version 1125933 (0.0006) [2023-12-27 13:27:00,123][20381] Updated weights for policy 1, policy_version 1124829 (0.0010) [2023-12-27 13:27:00,154][20347] Updated weights for policy 0, policy_version 1125943 (0.0005) [2023-12-27 13:27:00,173][20381] Updated weights for policy 1, policy_version 1124839 (0.0011) [2023-12-27 13:27:00,238][20381] Updated weights for policy 1, policy_version 1124850 (0.0010) [2023-12-27 13:27:00,806][20347] Updated weights for policy 0, policy_version 1125953 (0.0007) [2023-12-27 13:27:00,854][20347] Updated weights for policy 0, policy_version 1125963 (0.0005) [2023-12-27 13:27:00,909][20347] Updated weights for policy 0, policy_version 1125973 (0.0007) [2023-12-27 13:27:00,930][20381] Updated weights for policy 1, policy_version 1124860 (0.0007) [2023-12-27 13:27:00,960][20347] Updated weights for policy 0, policy_version 1125983 (0.0010) [2023-12-27 13:27:00,985][20381] Updated weights for policy 1, policy_version 1124870 (0.0006) [2023-12-27 13:27:01,049][20381] Updated weights for policy 1, policy_version 1124880 (0.0006) [2023-12-27 13:27:01,674][20347] Updated weights for policy 0, policy_version 1125993 (0.0008) [2023-12-27 13:27:01,707][20381] Updated weights for policy 1, policy_version 1124890 (0.0007) [2023-12-27 13:27:01,737][20347] Updated weights for policy 0, policy_version 1126003 (0.0008) [2023-12-27 13:27:01,765][20381] Updated weights for policy 1, policy_version 1124900 (0.0006) [2023-12-27 13:27:01,793][20347] Updated weights for policy 0, policy_version 1126013 (0.0007) [2023-12-27 13:27:01,810][20381] Updated weights for policy 1, policy_version 1124910 (0.0006) [2023-12-27 13:27:01,862][20381] Updated weights for policy 1, policy_version 1124920 (0.0008) [2023-12-27 13:27:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 576315392. Throughput: 0: 10105.8, 1: 10300.0. Samples: 576328708. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:02,363][19170] Avg episode reward: [(0, '912.160'), (1, '676.660')] [2023-12-27 13:27:02,466][20347] Updated weights for policy 0, policy_version 1126023 (0.0006) [2023-12-27 13:27:02,522][20347] Updated weights for policy 0, policy_version 1126033 (0.0006) [2023-12-27 13:27:02,577][20347] Updated weights for policy 0, policy_version 1126043 (0.0006) [2023-12-27 13:27:02,591][20381] Updated weights for policy 1, policy_version 1124930 (0.0008) [2023-12-27 13:27:02,648][20381] Updated weights for policy 1, policy_version 1124940 (0.0008) [2023-12-27 13:27:02,703][20381] Updated weights for policy 1, policy_version 1124950 (0.0007) [2023-12-27 13:27:03,200][20347] Updated weights for policy 0, policy_version 1126053 (0.0008) [2023-12-27 13:27:03,256][20347] Updated weights for policy 0, policy_version 1126063 (0.0006) [2023-12-27 13:27:03,308][20347] Updated weights for policy 0, policy_version 1126073 (0.0006) [2023-12-27 13:27:03,413][20381] Updated weights for policy 1, policy_version 1124960 (0.0006) [2023-12-27 13:27:03,469][20381] Updated weights for policy 1, policy_version 1124970 (0.0005) [2023-12-27 13:27:03,522][20381] Updated weights for policy 1, policy_version 1124980 (0.0005) [2023-12-27 13:27:03,992][20347] Updated weights for policy 0, policy_version 1126083 (0.0009) [2023-12-27 13:27:04,043][20347] Updated weights for policy 0, policy_version 1126093 (0.0010) [2023-12-27 13:27:04,095][20347] Updated weights for policy 0, policy_version 1126103 (0.0010) [2023-12-27 13:27:04,190][20381] Updated weights for policy 1, policy_version 1124990 (0.0007) [2023-12-27 13:27:04,247][20381] Updated weights for policy 1, policy_version 1125000 (0.0007) [2023-12-27 13:27:04,292][20292] KL-divergence is very high: 130.9786 [2023-12-27 13:27:04,297][20381] Updated weights for policy 1, policy_version 1125010 (0.0008) [2023-12-27 13:27:04,316][20292] KL-divergence is very high: 189.8833 [2023-12-27 13:27:04,809][20347] Updated weights for policy 0, policy_version 1126113 (0.0010) [2023-12-27 13:27:04,856][20347] Updated weights for policy 0, policy_version 1126123 (0.0010) [2023-12-27 13:27:04,905][20347] Updated weights for policy 0, policy_version 1126133 (0.0010) [2023-12-27 13:27:04,937][20381] Updated weights for policy 1, policy_version 1125020 (0.0007) [2023-12-27 13:27:04,955][20347] Updated weights for policy 0, policy_version 1126143 (0.0010) [2023-12-27 13:27:04,998][20381] Updated weights for policy 1, policy_version 1125030 (0.0006) [2023-12-27 13:27:05,054][20381] Updated weights for policy 1, policy_version 1125040 (0.0008) [2023-12-27 13:27:05,626][20381] Updated weights for policy 1, policy_version 1125050 (0.0008) [2023-12-27 13:27:05,673][20381] Updated weights for policy 1, policy_version 1125060 (0.0008) [2023-12-27 13:27:05,694][20347] Updated weights for policy 0, policy_version 1126153 (0.0008) [2023-12-27 13:27:05,718][20381] Updated weights for policy 1, policy_version 1125070 (0.0005) [2023-12-27 13:27:05,751][20347] Updated weights for policy 0, policy_version 1126163 (0.0008) [2023-12-27 13:27:05,768][20381] Updated weights for policy 1, policy_version 1125080 (0.0008) [2023-12-27 13:27:05,805][20347] Updated weights for policy 0, policy_version 1126173 (0.0007) [2023-12-27 13:27:06,460][20347] Updated weights for policy 0, policy_version 1126183 (0.0008) [2023-12-27 13:27:06,514][20381] Updated weights for policy 1, policy_version 1125090 (0.0007) [2023-12-27 13:27:06,519][20347] Updated weights for policy 0, policy_version 1126193 (0.0006) [2023-12-27 13:27:06,574][20381] Updated weights for policy 1, policy_version 1125100 (0.0006) [2023-12-27 13:27:06,580][20347] Updated weights for policy 0, policy_version 1126203 (0.0006) [2023-12-27 13:27:06,636][20381] Updated weights for policy 1, policy_version 1125110 (0.0007) [2023-12-27 13:27:07,270][20347] Updated weights for policy 0, policy_version 1126213 (0.0006) [2023-12-27 13:27:07,334][20347] Updated weights for policy 0, policy_version 1126223 (0.0005) [2023-12-27 13:27:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 576413696. Throughput: 0: 10086.6, 1: 10293.6. Samples: 576390264. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:07,363][19170] Avg episode reward: [(0, '912.120'), (1, '588.880')] [2023-12-27 13:27:07,367][20381] Updated weights for policy 1, policy_version 1125120 (0.0007) [2023-12-27 13:27:07,390][20347] Updated weights for policy 0, policy_version 1126233 (0.0006) [2023-12-27 13:27:07,418][20381] Updated weights for policy 1, policy_version 1125130 (0.0005) [2023-12-27 13:27:07,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001126240_288366592.pth... [2023-12-27 13:27:07,435][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001125056_288063488.pth [2023-12-27 13:27:07,482][20381] Updated weights for policy 1, policy_version 1125140 (0.0008) [2023-12-27 13:27:07,507][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001125144_288063488.pth... [2023-12-27 13:27:07,512][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001123928_287752192.pth [2023-12-27 13:27:07,961][20347] Updated weights for policy 0, policy_version 1126243 (0.0007) [2023-12-27 13:27:08,015][20347] Updated weights for policy 0, policy_version 1126253 (0.0007) [2023-12-27 13:27:08,064][20347] Updated weights for policy 0, policy_version 1126263 (0.0009) [2023-12-27 13:27:08,152][20381] Updated weights for policy 1, policy_version 1125150 (0.0009) [2023-12-27 13:27:08,208][20381] Updated weights for policy 1, policy_version 1125160 (0.0009) [2023-12-27 13:27:08,256][20381] Updated weights for policy 1, policy_version 1125170 (0.0008) [2023-12-27 13:27:08,670][20347] Updated weights for policy 0, policy_version 1126273 (0.0007) [2023-12-27 13:27:08,730][20347] Updated weights for policy 0, policy_version 1126283 (0.0007) [2023-12-27 13:27:08,780][20347] Updated weights for policy 0, policy_version 1126293 (0.0005) [2023-12-27 13:27:08,829][20347] Updated weights for policy 0, policy_version 1126303 (0.0007) [2023-12-27 13:27:09,024][20381] Updated weights for policy 1, policy_version 1125180 (0.0008) [2023-12-27 13:27:09,077][20381] Updated weights for policy 1, policy_version 1125190 (0.0008) [2023-12-27 13:27:09,133][20381] Updated weights for policy 1, policy_version 1125200 (0.0008) [2023-12-27 13:27:09,481][20347] Updated weights for policy 0, policy_version 1126313 (0.0007) [2023-12-27 13:27:09,545][20347] Updated weights for policy 0, policy_version 1126323 (0.0008) [2023-12-27 13:27:09,607][20347] Updated weights for policy 0, policy_version 1126333 (0.0008) [2023-12-27 13:27:09,930][20381] Updated weights for policy 1, policy_version 1125210 (0.0008) [2023-12-27 13:27:09,990][20381] Updated weights for policy 1, policy_version 1125220 (0.0007) [2023-12-27 13:27:10,050][20381] Updated weights for policy 1, policy_version 1125230 (0.0008) [2023-12-27 13:27:10,107][20381] Updated weights for policy 1, policy_version 1125240 (0.0007) [2023-12-27 13:27:10,329][20347] Updated weights for policy 0, policy_version 1126343 (0.0007) [2023-12-27 13:27:10,384][20347] Updated weights for policy 0, policy_version 1126353 (0.0008) [2023-12-27 13:27:10,441][20347] Updated weights for policy 0, policy_version 1126363 (0.0008) [2023-12-27 13:27:10,790][20381] Updated weights for policy 1, policy_version 1125250 (0.0010) [2023-12-27 13:27:10,842][20381] Updated weights for policy 1, policy_version 1125260 (0.0008) [2023-12-27 13:27:10,901][20381] Updated weights for policy 1, policy_version 1125270 (0.0009) [2023-12-27 13:27:11,166][20347] Updated weights for policy 0, policy_version 1126373 (0.0008) [2023-12-27 13:27:11,217][20347] Updated weights for policy 0, policy_version 1126383 (0.0007) [2023-12-27 13:27:11,284][20347] Updated weights for policy 0, policy_version 1126393 (0.0008) [2023-12-27 13:27:11,628][20381] Updated weights for policy 1, policy_version 1125280 (0.0008) [2023-12-27 13:27:11,685][20381] Updated weights for policy 1, policy_version 1125290 (0.0008) [2023-12-27 13:27:11,755][20381] Updated weights for policy 1, policy_version 1125300 (0.0008) [2023-12-27 13:27:12,051][20347] Updated weights for policy 0, policy_version 1126403 (0.0007) [2023-12-27 13:27:12,120][20347] Updated weights for policy 0, policy_version 1126413 (0.0008) [2023-12-27 13:27:12,169][20347] Updated weights for policy 0, policy_version 1126423 (0.0009) [2023-12-27 13:27:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 576520192. Throughput: 0: 10185.3, 1: 10215.4. Samples: 576512136. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:12,363][19170] Avg episode reward: [(0, '863.030'), (1, '647.400')] [2023-12-27 13:27:12,413][20381] Updated weights for policy 1, policy_version 1125310 (0.0010) [2023-12-27 13:27:12,473][20381] Updated weights for policy 1, policy_version 1125320 (0.0011) [2023-12-27 13:27:12,531][20381] Updated weights for policy 1, policy_version 1125330 (0.0010) [2023-12-27 13:27:12,831][20347] Updated weights for policy 0, policy_version 1126433 (0.0007) [2023-12-27 13:27:12,888][20347] Updated weights for policy 0, policy_version 1126443 (0.0007) [2023-12-27 13:27:12,943][20347] Updated weights for policy 0, policy_version 1126453 (0.0007) [2023-12-27 13:27:13,009][20347] Updated weights for policy 0, policy_version 1126463 (0.0007) [2023-12-27 13:27:13,197][20381] Updated weights for policy 1, policy_version 1125340 (0.0011) [2023-12-27 13:27:13,252][20381] Updated weights for policy 1, policy_version 1125350 (0.0010) [2023-12-27 13:27:13,311][20381] Updated weights for policy 1, policy_version 1125360 (0.0006) [2023-12-27 13:27:13,668][20347] Updated weights for policy 0, policy_version 1126473 (0.0010) [2023-12-27 13:27:13,719][20347] Updated weights for policy 0, policy_version 1126483 (0.0010) [2023-12-27 13:27:13,771][20347] Updated weights for policy 0, policy_version 1126493 (0.0010) [2023-12-27 13:27:13,926][20381] Updated weights for policy 1, policy_version 1125370 (0.0006) [2023-12-27 13:27:13,981][20381] Updated weights for policy 1, policy_version 1125380 (0.0011) [2023-12-27 13:27:14,044][20381] Updated weights for policy 1, policy_version 1125390 (0.0008) [2023-12-27 13:27:14,096][20381] Updated weights for policy 1, policy_version 1125400 (0.0011) [2023-12-27 13:27:14,379][20347] Updated weights for policy 0, policy_version 1126503 (0.0007) [2023-12-27 13:27:14,433][20347] Updated weights for policy 0, policy_version 1126513 (0.0005) [2023-12-27 13:27:14,485][20347] Updated weights for policy 0, policy_version 1126523 (0.0005) [2023-12-27 13:27:14,775][20381] Updated weights for policy 1, policy_version 1125410 (0.0011) [2023-12-27 13:27:14,838][20381] Updated weights for policy 1, policy_version 1125420 (0.0009) [2023-12-27 13:27:14,903][20381] Updated weights for policy 1, policy_version 1125430 (0.0008) [2023-12-27 13:27:15,106][20347] Updated weights for policy 0, policy_version 1126533 (0.0007) [2023-12-27 13:27:15,173][20347] Updated weights for policy 0, policy_version 1126543 (0.0007) [2023-12-27 13:27:15,237][20347] Updated weights for policy 0, policy_version 1126553 (0.0008) [2023-12-27 13:27:15,581][20381] Updated weights for policy 1, policy_version 1125440 (0.0009) [2023-12-27 13:27:15,633][20381] Updated weights for policy 1, policy_version 1125450 (0.0010) [2023-12-27 13:27:15,681][20381] Updated weights for policy 1, policy_version 1125460 (0.0010) [2023-12-27 13:27:15,917][20347] Updated weights for policy 0, policy_version 1126563 (0.0008) [2023-12-27 13:27:15,964][20347] Updated weights for policy 0, policy_version 1126573 (0.0006) [2023-12-27 13:27:16,010][20347] Updated weights for policy 0, policy_version 1126583 (0.0005) [2023-12-27 13:27:16,377][20381] Updated weights for policy 1, policy_version 1125470 (0.0010) [2023-12-27 13:27:16,425][20381] Updated weights for policy 1, policy_version 1125480 (0.0010) [2023-12-27 13:27:16,480][20381] Updated weights for policy 1, policy_version 1125490 (0.0010) [2023-12-27 13:27:16,643][20347] Updated weights for policy 0, policy_version 1126593 (0.0007) [2023-12-27 13:27:16,699][20347] Updated weights for policy 0, policy_version 1126603 (0.0007) [2023-12-27 13:27:16,748][20347] Updated weights for policy 0, policy_version 1126613 (0.0006) [2023-12-27 13:27:16,794][20347] Updated weights for policy 0, policy_version 1126623 (0.0006) [2023-12-27 13:27:17,179][20381] Updated weights for policy 1, policy_version 1125500 (0.0009) [2023-12-27 13:27:17,237][20381] Updated weights for policy 1, policy_version 1125510 (0.0009) [2023-12-27 13:27:17,294][20381] Updated weights for policy 1, policy_version 1125520 (0.0010) [2023-12-27 13:27:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 576626688. Throughput: 0: 10178.1, 1: 10190.9. Samples: 576635088. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:17,363][19170] Avg episode reward: [(0, '872.990'), (1, '549.380')] [2023-12-27 13:27:17,458][20347] Updated weights for policy 0, policy_version 1126633 (0.0010) [2023-12-27 13:27:17,511][20347] Updated weights for policy 0, policy_version 1126643 (0.0006) [2023-12-27 13:27:17,546][20291] KL-divergence is very high: 100.7080 [2023-12-27 13:27:17,567][20347] Updated weights for policy 0, policy_version 1126653 (0.0005) [2023-12-27 13:27:17,915][20381] Updated weights for policy 1, policy_version 1125530 (0.0010) [2023-12-27 13:27:17,970][20381] Updated weights for policy 1, policy_version 1125540 (0.0010) [2023-12-27 13:27:18,022][20381] Updated weights for policy 1, policy_version 1125550 (0.0010) [2023-12-27 13:27:18,079][20381] Updated weights for policy 1, policy_version 1125560 (0.0010) [2023-12-27 13:27:18,203][20347] Updated weights for policy 0, policy_version 1126663 (0.0007) [2023-12-27 13:27:18,263][20347] Updated weights for policy 0, policy_version 1126673 (0.0008) [2023-12-27 13:27:18,329][20347] Updated weights for policy 0, policy_version 1126683 (0.0008) [2023-12-27 13:27:18,756][20381] Updated weights for policy 1, policy_version 1125570 (0.0008) [2023-12-27 13:27:18,813][20381] Updated weights for policy 1, policy_version 1125580 (0.0009) [2023-12-27 13:27:18,874][20381] Updated weights for policy 1, policy_version 1125590 (0.0008) [2023-12-27 13:27:18,960][20347] Updated weights for policy 0, policy_version 1126693 (0.0008) [2023-12-27 13:27:19,018][20347] Updated weights for policy 0, policy_version 1126703 (0.0008) [2023-12-27 13:27:19,070][20347] Updated weights for policy 0, policy_version 1126713 (0.0008) [2023-12-27 13:27:19,596][20381] Updated weights for policy 1, policy_version 1125600 (0.0008) [2023-12-27 13:27:19,649][20381] Updated weights for policy 1, policy_version 1125610 (0.0007) [2023-12-27 13:27:19,707][20381] Updated weights for policy 1, policy_version 1125620 (0.0008) [2023-12-27 13:27:19,766][20347] Updated weights for policy 0, policy_version 1126723 (0.0008) [2023-12-27 13:27:19,834][20347] Updated weights for policy 0, policy_version 1126733 (0.0008) [2023-12-27 13:27:19,901][20347] Updated weights for policy 0, policy_version 1126743 (0.0008) [2023-12-27 13:27:20,465][20381] Updated weights for policy 1, policy_version 1125630 (0.0007) [2023-12-27 13:27:20,519][20381] Updated weights for policy 1, policy_version 1125640 (0.0008) [2023-12-27 13:27:20,544][20347] Updated weights for policy 0, policy_version 1126753 (0.0007) [2023-12-27 13:27:20,573][20381] Updated weights for policy 1, policy_version 1125650 (0.0007) [2023-12-27 13:27:20,607][20347] Updated weights for policy 0, policy_version 1126763 (0.0008) [2023-12-27 13:27:20,664][20347] Updated weights for policy 0, policy_version 1126773 (0.0005) [2023-12-27 13:27:20,729][20347] Updated weights for policy 0, policy_version 1126783 (0.0005) [2023-12-27 13:27:21,260][20381] Updated weights for policy 1, policy_version 1125660 (0.0008) [2023-12-27 13:27:21,325][20381] Updated weights for policy 1, policy_version 1125670 (0.0006) [2023-12-27 13:27:21,396][20381] Updated weights for policy 1, policy_version 1125680 (0.0008) [2023-12-27 13:27:21,401][20347] Updated weights for policy 0, policy_version 1126793 (0.0008) [2023-12-27 13:27:21,461][20347] Updated weights for policy 0, policy_version 1126803 (0.0007) [2023-12-27 13:27:21,524][20347] Updated weights for policy 0, policy_version 1126813 (0.0007) [2023-12-27 13:27:22,101][20381] Updated weights for policy 1, policy_version 1125690 (0.0006) [2023-12-27 13:27:22,159][20381] Updated weights for policy 1, policy_version 1125700 (0.0006) [2023-12-27 13:27:22,187][20347] Updated weights for policy 0, policy_version 1126823 (0.0008) [2023-12-27 13:27:22,220][20381] Updated weights for policy 1, policy_version 1125710 (0.0006) [2023-12-27 13:27:22,243][20347] Updated weights for policy 0, policy_version 1126833 (0.0007) [2023-12-27 13:27:22,287][20381] Updated weights for policy 1, policy_version 1125720 (0.0008) [2023-12-27 13:27:22,310][20347] Updated weights for policy 0, policy_version 1126843 (0.0007) [2023-12-27 13:27:22,364][19170] Fps is (10 sec: 21296.0, 60 sec: 20479.5, 300 sec: 20577.1). Total num frames: 576733184. Throughput: 0: 10189.2, 1: 10141.7. Samples: 576696808. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:22,365][19170] Avg episode reward: [(0, '892.390'), (1, '647.240')] [2023-12-27 13:27:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001125720_288210944.pth... [2023-12-27 13:27:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001126848_288522240.pth... [2023-12-27 13:27:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001124536_287907840.pth [2023-12-27 13:27:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001125632_288210944.pth [2023-12-27 13:27:22,965][20381] Updated weights for policy 1, policy_version 1125730 (0.0005) [2023-12-27 13:27:23,023][20381] Updated weights for policy 1, policy_version 1125740 (0.0006) [2023-12-27 13:27:23,030][20347] Updated weights for policy 0, policy_version 1126853 (0.0007) [2023-12-27 13:27:23,080][20381] Updated weights for policy 1, policy_version 1125750 (0.0006) [2023-12-27 13:27:23,089][20347] Updated weights for policy 0, policy_version 1126863 (0.0005) [2023-12-27 13:27:23,148][20347] Updated weights for policy 0, policy_version 1126873 (0.0007) [2023-12-27 13:27:23,695][20381] Updated weights for policy 1, policy_version 1125760 (0.0005) [2023-12-27 13:27:23,757][20381] Updated weights for policy 1, policy_version 1125770 (0.0008) [2023-12-27 13:27:23,806][20381] Updated weights for policy 1, policy_version 1125780 (0.0011) [2023-12-27 13:27:23,821][20347] Updated weights for policy 0, policy_version 1126883 (0.0007) [2023-12-27 13:27:23,877][20347] Updated weights for policy 0, policy_version 1126893 (0.0007) [2023-12-27 13:27:23,930][20347] Updated weights for policy 0, policy_version 1126903 (0.0006) [2023-12-27 13:27:24,428][20381] Updated weights for policy 1, policy_version 1125790 (0.0010) [2023-12-27 13:27:24,489][20381] Updated weights for policy 1, policy_version 1125800 (0.0010) [2023-12-27 13:27:24,543][20381] Updated weights for policy 1, policy_version 1125810 (0.0010) [2023-12-27 13:27:24,629][20347] Updated weights for policy 0, policy_version 1126913 (0.0006) [2023-12-27 13:27:24,686][20347] Updated weights for policy 0, policy_version 1126923 (0.0007) [2023-12-27 13:27:24,744][20347] Updated weights for policy 0, policy_version 1126933 (0.0008) [2023-12-27 13:27:24,795][20347] Updated weights for policy 0, policy_version 1126943 (0.0007) [2023-12-27 13:27:25,209][20381] Updated weights for policy 1, policy_version 1125820 (0.0010) [2023-12-27 13:27:25,272][20381] Updated weights for policy 1, policy_version 1125830 (0.0010) [2023-12-27 13:27:25,322][20381] Updated weights for policy 1, policy_version 1125840 (0.0006) [2023-12-27 13:27:25,500][20347] Updated weights for policy 0, policy_version 1126953 (0.0005) [2023-12-27 13:27:25,545][20347] Updated weights for policy 0, policy_version 1126963 (0.0008) [2023-12-27 13:27:25,593][20347] Updated weights for policy 0, policy_version 1126973 (0.0007) [2023-12-27 13:27:25,935][20381] Updated weights for policy 1, policy_version 1125850 (0.0006) [2023-12-27 13:27:25,994][20381] Updated weights for policy 1, policy_version 1125860 (0.0011) [2023-12-27 13:27:26,052][20381] Updated weights for policy 1, policy_version 1125870 (0.0011) [2023-12-27 13:27:26,107][20381] Updated weights for policy 1, policy_version 1125880 (0.0010) [2023-12-27 13:27:26,258][20347] Updated weights for policy 0, policy_version 1126983 (0.0008) [2023-12-27 13:27:26,311][20347] Updated weights for policy 0, policy_version 1126993 (0.0010) [2023-12-27 13:27:26,367][20347] Updated weights for policy 0, policy_version 1127003 (0.0010) [2023-12-27 13:27:26,802][20381] Updated weights for policy 1, policy_version 1125890 (0.0011) [2023-12-27 13:27:26,859][20381] Updated weights for policy 1, policy_version 1125900 (0.0010) [2023-12-27 13:27:26,910][20381] Updated weights for policy 1, policy_version 1125910 (0.0010) [2023-12-27 13:27:27,042][20347] Updated weights for policy 0, policy_version 1127013 (0.0009) [2023-12-27 13:27:27,089][20347] Updated weights for policy 0, policy_version 1127023 (0.0007) [2023-12-27 13:27:27,139][20347] Updated weights for policy 0, policy_version 1127033 (0.0007) [2023-12-27 13:27:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 576831488. Throughput: 0: 10102.1, 1: 10184.6. Samples: 576818524. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:27,363][19170] Avg episode reward: [(0, '862.970'), (1, '725.770')] [2023-12-27 13:27:27,587][20381] Updated weights for policy 1, policy_version 1125920 (0.0006) [2023-12-27 13:27:27,644][20381] Updated weights for policy 1, policy_version 1125930 (0.0006) [2023-12-27 13:27:27,693][20381] Updated weights for policy 1, policy_version 1125940 (0.0008) [2023-12-27 13:27:27,775][20347] Updated weights for policy 0, policy_version 1127043 (0.0006) [2023-12-27 13:27:27,819][20347] Updated weights for policy 0, policy_version 1127053 (0.0005) [2023-12-27 13:27:27,865][20347] Updated weights for policy 0, policy_version 1127063 (0.0005) [2023-12-27 13:27:28,296][20381] Updated weights for policy 1, policy_version 1125950 (0.0007) [2023-12-27 13:27:28,350][20381] Updated weights for policy 1, policy_version 1125960 (0.0007) [2023-12-27 13:27:28,410][20381] Updated weights for policy 1, policy_version 1125970 (0.0006) [2023-12-27 13:27:28,521][20347] Updated weights for policy 0, policy_version 1127073 (0.0006) [2023-12-27 13:27:28,578][20347] Updated weights for policy 0, policy_version 1127083 (0.0010) [2023-12-27 13:27:28,630][20347] Updated weights for policy 0, policy_version 1127093 (0.0010) [2023-12-27 13:27:28,694][20347] Updated weights for policy 0, policy_version 1127103 (0.0010) [2023-12-27 13:27:29,064][20381] Updated weights for policy 1, policy_version 1125980 (0.0006) [2023-12-27 13:27:29,112][20381] Updated weights for policy 1, policy_version 1125990 (0.0009) [2023-12-27 13:27:29,161][20381] Updated weights for policy 1, policy_version 1126000 (0.0008) [2023-12-27 13:27:29,284][20347] Updated weights for policy 0, policy_version 1127113 (0.0010) [2023-12-27 13:27:29,367][20347] Updated weights for policy 0, policy_version 1127123 (0.0010) [2023-12-27 13:27:29,416][20347] Updated weights for policy 0, policy_version 1127133 (0.0010) [2023-12-27 13:27:29,877][20381] Updated weights for policy 1, policy_version 1126010 (0.0008) [2023-12-27 13:27:29,933][20381] Updated weights for policy 1, policy_version 1126020 (0.0008) [2023-12-27 13:27:29,998][20381] Updated weights for policy 1, policy_version 1126030 (0.0008) [2023-12-27 13:27:30,053][20381] Updated weights for policy 1, policy_version 1126040 (0.0008) [2023-12-27 13:27:30,083][20347] Updated weights for policy 0, policy_version 1127143 (0.0007) [2023-12-27 13:27:30,138][20347] Updated weights for policy 0, policy_version 1127153 (0.0007) [2023-12-27 13:27:30,193][20347] Updated weights for policy 0, policy_version 1127163 (0.0008) [2023-12-27 13:27:30,781][20381] Updated weights for policy 1, policy_version 1126050 (0.0007) [2023-12-27 13:27:30,814][20347] Updated weights for policy 0, policy_version 1127173 (0.0007) [2023-12-27 13:27:30,842][20381] Updated weights for policy 1, policy_version 1126060 (0.0007) [2023-12-27 13:27:30,867][20347] Updated weights for policy 0, policy_version 1127183 (0.0007) [2023-12-27 13:27:30,897][20381] Updated weights for policy 1, policy_version 1126070 (0.0006) [2023-12-27 13:27:30,918][20347] Updated weights for policy 0, policy_version 1127193 (0.0008) [2023-12-27 13:27:31,641][20381] Updated weights for policy 1, policy_version 1126080 (0.0007) [2023-12-27 13:27:31,664][20347] Updated weights for policy 0, policy_version 1127203 (0.0009) [2023-12-27 13:27:31,695][20381] Updated weights for policy 1, policy_version 1126090 (0.0006) [2023-12-27 13:27:31,713][20347] Updated weights for policy 0, policy_version 1127213 (0.0010) [2023-12-27 13:27:31,759][20381] Updated weights for policy 1, policy_version 1126100 (0.0008) [2023-12-27 13:27:31,780][20347] Updated weights for policy 0, policy_version 1127223 (0.0008) [2023-12-27 13:27:32,362][19170] Fps is (10 sec: 19663.5, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 576929792. Throughput: 0: 10225.9, 1: 10150.6. Samples: 576943312. Policy #0 lag: (min: 31.0, avg: 32.6, max: 63.0) [2023-12-27 13:27:32,363][19170] Avg episode reward: [(0, '843.460'), (1, '666.990')] [2023-12-27 13:27:32,425][20381] Updated weights for policy 1, policy_version 1126110 (0.0006) [2023-12-27 13:27:32,441][20347] Updated weights for policy 0, policy_version 1127233 (0.0006) [2023-12-27 13:27:32,485][20381] Updated weights for policy 1, policy_version 1126120 (0.0006) [2023-12-27 13:27:32,504][20347] Updated weights for policy 0, policy_version 1127243 (0.0006) [2023-12-27 13:27:32,535][20381] Updated weights for policy 1, policy_version 1126130 (0.0006) [2023-12-27 13:27:32,555][20347] Updated weights for policy 0, policy_version 1127253 (0.0007) [2023-12-27 13:27:32,609][20347] Updated weights for policy 0, policy_version 1127263 (0.0006) [2023-12-27 13:27:33,238][20381] Updated weights for policy 1, policy_version 1126140 (0.0007) [2023-12-27 13:27:33,243][20347] Updated weights for policy 0, policy_version 1127273 (0.0007) [2023-12-27 13:27:33,285][20381] Updated weights for policy 1, policy_version 1126150 (0.0005) [2023-12-27 13:27:33,293][20347] Updated weights for policy 0, policy_version 1127283 (0.0006) [2023-12-27 13:27:33,337][20381] Updated weights for policy 1, policy_version 1126161 (0.0008) [2023-12-27 13:27:33,349][20347] Updated weights for policy 0, policy_version 1127293 (0.0006) [2023-12-27 13:27:33,913][20381] Updated weights for policy 1, policy_version 1126171 (0.0009) [2023-12-27 13:27:33,968][20381] Updated weights for policy 1, policy_version 1126181 (0.0010) [2023-12-27 13:27:34,020][20381] Updated weights for policy 1, policy_version 1126191 (0.0010) [2023-12-27 13:27:34,078][20347] Updated weights for policy 0, policy_version 1127303 (0.0007) [2023-12-27 13:27:34,135][20347] Updated weights for policy 0, policy_version 1127313 (0.0007) [2023-12-27 13:27:34,194][20347] Updated weights for policy 0, policy_version 1127323 (0.0008) [2023-12-27 13:27:34,683][20381] Updated weights for policy 1, policy_version 1126201 (0.0010) [2023-12-27 13:27:34,745][20381] Updated weights for policy 1, policy_version 1126211 (0.0006) [2023-12-27 13:27:34,796][20381] Updated weights for policy 1, policy_version 1126221 (0.0006) [2023-12-27 13:27:34,850][20381] Updated weights for policy 1, policy_version 1126231 (0.0005) [2023-12-27 13:27:34,930][20347] Updated weights for policy 0, policy_version 1127333 (0.0007) [2023-12-27 13:27:34,994][20347] Updated weights for policy 0, policy_version 1127343 (0.0007) [2023-12-27 13:27:35,053][20347] Updated weights for policy 0, policy_version 1127353 (0.0007) [2023-12-27 13:27:35,496][20381] Updated weights for policy 1, policy_version 1126241 (0.0007) [2023-12-27 13:27:35,545][20381] Updated weights for policy 1, policy_version 1126251 (0.0007) [2023-12-27 13:27:35,600][20381] Updated weights for policy 1, policy_version 1126261 (0.0008) [2023-12-27 13:27:35,695][20347] Updated weights for policy 0, policy_version 1127363 (0.0007) [2023-12-27 13:27:35,750][20347] Updated weights for policy 0, policy_version 1127373 (0.0008) [2023-12-27 13:27:35,793][20347] Updated weights for policy 0, policy_version 1127383 (0.0008) [2023-12-27 13:27:36,298][20381] Updated weights for policy 1, policy_version 1126271 (0.0008) [2023-12-27 13:27:36,357][20381] Updated weights for policy 1, policy_version 1126281 (0.0008) [2023-12-27 13:27:36,416][20381] Updated weights for policy 1, policy_version 1126291 (0.0007) [2023-12-27 13:27:36,518][20347] Updated weights for policy 0, policy_version 1127393 (0.0008) [2023-12-27 13:27:36,578][20347] Updated weights for policy 0, policy_version 1127403 (0.0008) [2023-12-27 13:27:36,637][20347] Updated weights for policy 0, policy_version 1127413 (0.0008) [2023-12-27 13:27:36,693][20347] Updated weights for policy 0, policy_version 1127423 (0.0008) [2023-12-27 13:27:37,104][20381] Updated weights for policy 1, policy_version 1126301 (0.0008) [2023-12-27 13:27:37,161][20381] Updated weights for policy 1, policy_version 1126311 (0.0008) [2023-12-27 13:27:37,213][20381] Updated weights for policy 1, policy_version 1126321 (0.0007) [2023-12-27 13:27:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 577036288. Throughput: 0: 10234.3, 1: 10201.7. Samples: 577005340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:27:37,363][19170] Avg episode reward: [(0, '911.870'), (1, '657.190')] [2023-12-27 13:27:37,364][20347] Updated weights for policy 0, policy_version 1127433 (0.0007) [2023-12-27 13:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001126328_288366592.pth... [2023-12-27 13:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001125144_288063488.pth [2023-12-27 13:27:37,412][20347] Updated weights for policy 0, policy_version 1127443 (0.0007) [2023-12-27 13:27:37,467][20347] Updated weights for policy 0, policy_version 1127453 (0.0008) [2023-12-27 13:27:37,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001127456_288677888.pth... [2023-12-27 13:27:37,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001126240_288366592.pth [2023-12-27 13:27:37,829][20381] Updated weights for policy 1, policy_version 1126331 (0.0008) [2023-12-27 13:27:37,888][20381] Updated weights for policy 1, policy_version 1126341 (0.0009) [2023-12-27 13:27:37,943][20381] Updated weights for policy 1, policy_version 1126351 (0.0007) [2023-12-27 13:27:38,192][20347] Updated weights for policy 0, policy_version 1127463 (0.0006) [2023-12-27 13:27:38,244][20347] Updated weights for policy 0, policy_version 1127473 (0.0006) [2023-12-27 13:27:38,288][20347] Updated weights for policy 0, policy_version 1127483 (0.0007) [2023-12-27 13:27:38,678][20381] Updated weights for policy 1, policy_version 1126361 (0.0008) [2023-12-27 13:27:38,742][20381] Updated weights for policy 1, policy_version 1126371 (0.0008) [2023-12-27 13:27:38,802][20381] Updated weights for policy 1, policy_version 1126381 (0.0007) [2023-12-27 13:27:38,856][20381] Updated weights for policy 1, policy_version 1126391 (0.0005) [2023-12-27 13:27:39,040][20347] Updated weights for policy 0, policy_version 1127493 (0.0009) [2023-12-27 13:27:39,088][20347] Updated weights for policy 0, policy_version 1127503 (0.0007) [2023-12-27 13:27:39,143][20347] Updated weights for policy 0, policy_version 1127513 (0.0007) [2023-12-27 13:27:39,508][20381] Updated weights for policy 1, policy_version 1126401 (0.0010) [2023-12-27 13:27:39,565][20381] Updated weights for policy 1, policy_version 1126411 (0.0011) [2023-12-27 13:27:39,626][20381] Updated weights for policy 1, policy_version 1126421 (0.0011) [2023-12-27 13:27:39,852][20347] Updated weights for policy 0, policy_version 1127523 (0.0008) [2023-12-27 13:27:39,920][20347] Updated weights for policy 0, policy_version 1127533 (0.0012) [2023-12-27 13:27:39,990][20347] Updated weights for policy 0, policy_version 1127543 (0.0007) [2023-12-27 13:27:40,341][20381] Updated weights for policy 1, policy_version 1126431 (0.0009) [2023-12-27 13:27:40,403][20381] Updated weights for policy 1, policy_version 1126441 (0.0011) [2023-12-27 13:27:40,460][20381] Updated weights for policy 1, policy_version 1126451 (0.0011) [2023-12-27 13:27:40,679][20347] Updated weights for policy 0, policy_version 1127553 (0.0008) [2023-12-27 13:27:40,729][20347] Updated weights for policy 0, policy_version 1127563 (0.0009) [2023-12-27 13:27:40,781][20347] Updated weights for policy 0, policy_version 1127573 (0.0005) [2023-12-27 13:27:40,833][20347] Updated weights for policy 0, policy_version 1127583 (0.0005) [2023-12-27 13:27:41,127][20381] Updated weights for policy 1, policy_version 1126461 (0.0011) [2023-12-27 13:27:41,194][20381] Updated weights for policy 1, policy_version 1126471 (0.0009) [2023-12-27 13:27:41,261][20381] Updated weights for policy 1, policy_version 1126481 (0.0010) [2023-12-27 13:27:41,541][20347] Updated weights for policy 0, policy_version 1127593 (0.0008) [2023-12-27 13:27:41,602][20347] Updated weights for policy 0, policy_version 1127603 (0.0011) [2023-12-27 13:27:41,662][20347] Updated weights for policy 0, policy_version 1127613 (0.0008) [2023-12-27 13:27:42,009][20381] Updated weights for policy 1, policy_version 1126491 (0.0011) [2023-12-27 13:27:42,069][20381] Updated weights for policy 1, policy_version 1126501 (0.0011) [2023-12-27 13:27:42,126][20381] Updated weights for policy 1, policy_version 1126511 (0.0011) [2023-12-27 13:27:42,289][20347] Updated weights for policy 0, policy_version 1127623 (0.0009) [2023-12-27 13:27:42,352][20347] Updated weights for policy 0, policy_version 1127633 (0.0010) [2023-12-27 13:27:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 577134592. Throughput: 0: 10230.5, 1: 10152.6. Samples: 577126400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:27:42,363][19170] Avg episode reward: [(0, '951.080'), (1, '706.170')] [2023-12-27 13:27:42,420][20347] Updated weights for policy 0, policy_version 1127643 (0.0008) [2023-12-27 13:27:42,760][20381] Updated weights for policy 1, policy_version 1126521 (0.0009) [2023-12-27 13:27:42,823][20381] Updated weights for policy 1, policy_version 1126531 (0.0011) [2023-12-27 13:27:42,880][20381] Updated weights for policy 1, policy_version 1126541 (0.0011) [2023-12-27 13:27:42,929][20381] Updated weights for policy 1, policy_version 1126551 (0.0011) [2023-12-27 13:27:43,016][20347] Updated weights for policy 0, policy_version 1127653 (0.0008) [2023-12-27 13:27:43,072][20347] Updated weights for policy 0, policy_version 1127663 (0.0007) [2023-12-27 13:27:43,129][20347] Updated weights for policy 0, policy_version 1127673 (0.0007) [2023-12-27 13:27:43,576][20381] Updated weights for policy 1, policy_version 1126561 (0.0011) [2023-12-27 13:27:43,637][20381] Updated weights for policy 1, policy_version 1126571 (0.0011) [2023-12-27 13:27:43,695][20381] Updated weights for policy 1, policy_version 1126581 (0.0010) [2023-12-27 13:27:43,754][20347] Updated weights for policy 0, policy_version 1127683 (0.0006) [2023-12-27 13:27:43,815][20347] Updated weights for policy 0, policy_version 1127693 (0.0005) [2023-12-27 13:27:43,882][20347] Updated weights for policy 0, policy_version 1127703 (0.0007) [2023-12-27 13:27:44,275][20381] Updated weights for policy 1, policy_version 1126591 (0.0007) [2023-12-27 13:27:44,335][20381] Updated weights for policy 1, policy_version 1126601 (0.0005) [2023-12-27 13:27:44,400][20381] Updated weights for policy 1, policy_version 1126611 (0.0005) [2023-12-27 13:27:44,535][20347] Updated weights for policy 0, policy_version 1127713 (0.0007) [2023-12-27 13:27:44,597][20347] Updated weights for policy 0, policy_version 1127723 (0.0006) [2023-12-27 13:27:44,642][20347] Updated weights for policy 0, policy_version 1127733 (0.0005) [2023-12-27 13:27:44,700][20347] Updated weights for policy 0, policy_version 1127743 (0.0005) [2023-12-27 13:27:45,031][20381] Updated weights for policy 1, policy_version 1126621 (0.0008) [2023-12-27 13:27:45,054][20292] KL-divergence is very high: 203.8219 [2023-12-27 13:27:45,068][20292] KL-divergence is very high: 138.1335 [2023-12-27 13:27:45,094][20381] Updated weights for policy 1, policy_version 1126631 (0.0011) [2023-12-27 13:27:45,159][20381] Updated weights for policy 1, policy_version 1126641 (0.0011) [2023-12-27 13:27:45,403][20347] Updated weights for policy 0, policy_version 1127753 (0.0010) [2023-12-27 13:27:45,459][20347] Updated weights for policy 0, policy_version 1127763 (0.0010) [2023-12-27 13:27:45,510][20347] Updated weights for policy 0, policy_version 1127773 (0.0005) [2023-12-27 13:27:45,805][20381] Updated weights for policy 1, policy_version 1126651 (0.0008) [2023-12-27 13:27:45,850][20381] Updated weights for policy 1, policy_version 1126661 (0.0010) [2023-12-27 13:27:45,898][20381] Updated weights for policy 1, policy_version 1126671 (0.0010) [2023-12-27 13:27:46,125][20347] Updated weights for policy 0, policy_version 1127783 (0.0007) [2023-12-27 13:27:46,173][20347] Updated weights for policy 0, policy_version 1127793 (0.0007) [2023-12-27 13:27:46,223][20347] Updated weights for policy 0, policy_version 1127803 (0.0008) [2023-12-27 13:27:46,603][20381] Updated weights for policy 1, policy_version 1126681 (0.0010) [2023-12-27 13:27:46,652][20381] Updated weights for policy 1, policy_version 1126691 (0.0011) [2023-12-27 13:27:46,704][20381] Updated weights for policy 1, policy_version 1126701 (0.0011) [2023-12-27 13:27:46,751][20381] Updated weights for policy 1, policy_version 1126711 (0.0011) [2023-12-27 13:27:46,843][20347] Updated weights for policy 0, policy_version 1127813 (0.0009) [2023-12-27 13:27:46,897][20347] Updated weights for policy 0, policy_version 1127823 (0.0010) [2023-12-27 13:27:46,961][20347] Updated weights for policy 0, policy_version 1127833 (0.0010) [2023-12-27 13:27:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 577241088. Throughput: 0: 10317.6, 1: 10215.6. Samples: 577252700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:27:47,363][19170] Avg episode reward: [(0, '951.160'), (1, '627.940')] [2023-12-27 13:27:47,444][20381] Updated weights for policy 1, policy_version 1126721 (0.0006) [2023-12-27 13:27:47,494][20381] Updated weights for policy 1, policy_version 1126731 (0.0006) [2023-12-27 13:27:47,549][20381] Updated weights for policy 1, policy_version 1126741 (0.0007) [2023-12-27 13:27:47,625][20347] Updated weights for policy 0, policy_version 1127843 (0.0010) [2023-12-27 13:27:47,688][20347] Updated weights for policy 0, policy_version 1127853 (0.0005) [2023-12-27 13:27:47,745][20347] Updated weights for policy 0, policy_version 1127863 (0.0005) [2023-12-27 13:27:48,216][20381] Updated weights for policy 1, policy_version 1126751 (0.0007) [2023-12-27 13:27:48,274][20381] Updated weights for policy 1, policy_version 1126761 (0.0008) [2023-12-27 13:27:48,313][20347] Updated weights for policy 0, policy_version 1127873 (0.0005) [2023-12-27 13:27:48,334][20381] Updated weights for policy 1, policy_version 1126771 (0.0009) [2023-12-27 13:27:48,379][20347] Updated weights for policy 0, policy_version 1127883 (0.0007) [2023-12-27 13:27:48,443][20347] Updated weights for policy 0, policy_version 1127893 (0.0008) [2023-12-27 13:27:48,516][20347] Updated weights for policy 0, policy_version 1127903 (0.0006) [2023-12-27 13:27:49,009][20381] Updated weights for policy 1, policy_version 1126781 (0.0009) [2023-12-27 13:27:49,054][20381] Updated weights for policy 1, policy_version 1126791 (0.0007) [2023-12-27 13:27:49,102][20381] Updated weights for policy 1, policy_version 1126801 (0.0007) [2023-12-27 13:27:49,174][20347] Updated weights for policy 0, policy_version 1127913 (0.0005) [2023-12-27 13:27:49,233][20347] Updated weights for policy 0, policy_version 1127923 (0.0008) [2023-12-27 13:27:49,285][20347] Updated weights for policy 0, policy_version 1127933 (0.0005) [2023-12-27 13:27:49,759][20381] Updated weights for policy 1, policy_version 1126811 (0.0006) [2023-12-27 13:27:49,839][20381] Updated weights for policy 1, policy_version 1126821 (0.0010) [2023-12-27 13:27:49,900][20381] Updated weights for policy 1, policy_version 1126831 (0.0008) [2023-12-27 13:27:49,956][20347] Updated weights for policy 0, policy_version 1127943 (0.0007) [2023-12-27 13:27:50,014][20347] Updated weights for policy 0, policy_version 1127953 (0.0007) [2023-12-27 13:27:50,065][20347] Updated weights for policy 0, policy_version 1127963 (0.0008) [2023-12-27 13:27:50,547][20381] Updated weights for policy 1, policy_version 1126841 (0.0010) [2023-12-27 13:27:50,611][20381] Updated weights for policy 1, policy_version 1126851 (0.0009) [2023-12-27 13:27:50,668][20381] Updated weights for policy 1, policy_version 1126861 (0.0011) [2023-12-27 13:27:50,730][20381] Updated weights for policy 1, policy_version 1126871 (0.0010) [2023-12-27 13:27:50,751][20347] Updated weights for policy 0, policy_version 1127973 (0.0008) [2023-12-27 13:27:50,805][20347] Updated weights for policy 0, policy_version 1127983 (0.0011) [2023-12-27 13:27:50,864][20347] Updated weights for policy 0, policy_version 1127993 (0.0011) [2023-12-27 13:27:51,440][20381] Updated weights for policy 1, policy_version 1126881 (0.0008) [2023-12-27 13:27:51,495][20381] Updated weights for policy 1, policy_version 1126891 (0.0008) [2023-12-27 13:27:51,539][20347] Updated weights for policy 0, policy_version 1128003 (0.0009) [2023-12-27 13:27:51,551][20381] Updated weights for policy 1, policy_version 1126901 (0.0008) [2023-12-27 13:27:51,589][20347] Updated weights for policy 0, policy_version 1128013 (0.0006) [2023-12-27 13:27:51,652][20347] Updated weights for policy 0, policy_version 1128023 (0.0008) [2023-12-27 13:27:52,321][20381] Updated weights for policy 1, policy_version 1126911 (0.0006) [2023-12-27 13:27:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 577339392. Throughput: 0: 10321.9, 1: 10225.1. Samples: 577314884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:27:52,363][19170] Avg episode reward: [(0, '892.360'), (1, '627.840')] [2023-12-27 13:27:52,377][20347] Updated weights for policy 0, policy_version 1128033 (0.0008) [2023-12-27 13:27:52,389][20381] Updated weights for policy 1, policy_version 1126921 (0.0007) [2023-12-27 13:27:52,441][20347] Updated weights for policy 0, policy_version 1128043 (0.0005) [2023-12-27 13:27:52,448][20381] Updated weights for policy 1, policy_version 1126931 (0.0006) [2023-12-27 13:27:52,477][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001126936_288522240.pth... [2023-12-27 13:27:52,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001125720_288210944.pth [2023-12-27 13:27:52,504][20347] Updated weights for policy 0, policy_version 1128053 (0.0006) [2023-12-27 13:27:52,562][20347] Updated weights for policy 0, policy_version 1128063 (0.0007) [2023-12-27 13:27:52,567][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001128064_288833536.pth... [2023-12-27 13:27:52,571][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001126848_288522240.pth [2023-12-27 13:27:53,071][20381] Updated weights for policy 1, policy_version 1126941 (0.0006) [2023-12-27 13:27:53,133][20381] Updated weights for policy 1, policy_version 1126951 (0.0005) [2023-12-27 13:27:53,191][20381] Updated weights for policy 1, policy_version 1126961 (0.0005) [2023-12-27 13:27:53,280][20347] Updated weights for policy 0, policy_version 1128073 (0.0007) [2023-12-27 13:27:53,340][20347] Updated weights for policy 0, policy_version 1128083 (0.0005) [2023-12-27 13:27:53,394][20347] Updated weights for policy 0, policy_version 1128093 (0.0006) [2023-12-27 13:27:53,827][20381] Updated weights for policy 1, policy_version 1126971 (0.0006) [2023-12-27 13:27:53,878][20381] Updated weights for policy 1, policy_version 1126981 (0.0007) [2023-12-27 13:27:53,939][20381] Updated weights for policy 1, policy_version 1126991 (0.0008) [2023-12-27 13:27:53,971][20347] Updated weights for policy 0, policy_version 1128103 (0.0006) [2023-12-27 13:27:54,022][20347] Updated weights for policy 0, policy_version 1128113 (0.0007) [2023-12-27 13:27:54,066][20347] Updated weights for policy 0, policy_version 1128123 (0.0007) [2023-12-27 13:27:54,553][20381] Updated weights for policy 1, policy_version 1127001 (0.0005) [2023-12-27 13:27:54,615][20381] Updated weights for policy 1, policy_version 1127011 (0.0005) [2023-12-27 13:27:54,673][20381] Updated weights for policy 1, policy_version 1127021 (0.0008) [2023-12-27 13:27:54,723][20347] Updated weights for policy 0, policy_version 1128133 (0.0007) [2023-12-27 13:27:54,725][20381] Updated weights for policy 1, policy_version 1127031 (0.0008) [2023-12-27 13:27:54,784][20347] Updated weights for policy 0, policy_version 1128143 (0.0005) [2023-12-27 13:27:54,844][20347] Updated weights for policy 0, policy_version 1128153 (0.0005) [2023-12-27 13:27:55,424][20381] Updated weights for policy 1, policy_version 1127041 (0.0007) [2023-12-27 13:27:55,472][20347] Updated weights for policy 0, policy_version 1128163 (0.0006) [2023-12-27 13:27:55,478][20381] Updated weights for policy 1, policy_version 1127051 (0.0006) [2023-12-27 13:27:55,520][20347] Updated weights for policy 0, policy_version 1128173 (0.0006) [2023-12-27 13:27:55,533][20381] Updated weights for policy 1, policy_version 1127061 (0.0005) [2023-12-27 13:27:55,572][20347] Updated weights for policy 0, policy_version 1128183 (0.0008) [2023-12-27 13:27:56,175][20381] Updated weights for policy 1, policy_version 1127071 (0.0008) [2023-12-27 13:27:56,228][20381] Updated weights for policy 1, policy_version 1127081 (0.0006) [2023-12-27 13:27:56,232][20347] Updated weights for policy 0, policy_version 1128193 (0.0007) [2023-12-27 13:27:56,282][20381] Updated weights for policy 1, policy_version 1127091 (0.0007) [2023-12-27 13:27:56,283][20347] Updated weights for policy 0, policy_version 1128203 (0.0005) [2023-12-27 13:27:56,345][20347] Updated weights for policy 0, policy_version 1128213 (0.0005) [2023-12-27 13:27:56,410][20347] Updated weights for policy 0, policy_version 1128223 (0.0005) [2023-12-27 13:27:56,951][20381] Updated weights for policy 1, policy_version 1127101 (0.0006) [2023-12-27 13:27:56,964][20347] Updated weights for policy 0, policy_version 1128233 (0.0005) [2023-12-27 13:27:57,003][20381] Updated weights for policy 1, policy_version 1127111 (0.0005) [2023-12-27 13:27:57,024][20347] Updated weights for policy 0, policy_version 1128243 (0.0010) [2023-12-27 13:27:57,060][20381] Updated weights for policy 1, policy_version 1127121 (0.0007) [2023-12-27 13:27:57,075][20347] Updated weights for policy 0, policy_version 1128253 (0.0009) [2023-12-27 13:27:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 577454080. Throughput: 0: 10325.7, 1: 10281.3. Samples: 577439452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:27:57,363][19170] Avg episode reward: [(0, '882.790'), (1, '715.970')] [2023-12-27 13:27:57,722][20347] Updated weights for policy 0, policy_version 1128263 (0.0010) [2023-12-27 13:27:57,724][20381] Updated weights for policy 1, policy_version 1127131 (0.0009) [2023-12-27 13:27:57,774][20381] Updated weights for policy 1, policy_version 1127141 (0.0005) [2023-12-27 13:27:57,776][20347] Updated weights for policy 0, policy_version 1128273 (0.0010) [2023-12-27 13:27:57,825][20381] Updated weights for policy 1, policy_version 1127151 (0.0005) [2023-12-27 13:27:57,830][20347] Updated weights for policy 0, policy_version 1128283 (0.0010) [2023-12-27 13:27:58,516][20381] Updated weights for policy 1, policy_version 1127161 (0.0006) [2023-12-27 13:27:58,526][20347] Updated weights for policy 0, policy_version 1128293 (0.0010) [2023-12-27 13:27:58,583][20381] Updated weights for policy 1, policy_version 1127171 (0.0008) [2023-12-27 13:27:58,590][20347] Updated weights for policy 0, policy_version 1128303 (0.0010) [2023-12-27 13:27:58,645][20381] Updated weights for policy 1, policy_version 1127181 (0.0006) [2023-12-27 13:27:58,652][20347] Updated weights for policy 0, policy_version 1128313 (0.0011) [2023-12-27 13:27:58,708][20381] Updated weights for policy 1, policy_version 1127191 (0.0009) [2023-12-27 13:27:59,469][20347] Updated weights for policy 0, policy_version 1128323 (0.0007) [2023-12-27 13:27:59,499][20381] Updated weights for policy 1, policy_version 1127201 (0.0010) [2023-12-27 13:27:59,522][20347] Updated weights for policy 0, policy_version 1128333 (0.0009) [2023-12-27 13:27:59,551][20381] Updated weights for policy 1, policy_version 1127211 (0.0010) [2023-12-27 13:27:59,576][20347] Updated weights for policy 0, policy_version 1128343 (0.0007) [2023-12-27 13:27:59,608][20381] Updated weights for policy 1, policy_version 1127221 (0.0011) [2023-12-27 13:28:00,270][20347] Updated weights for policy 0, policy_version 1128353 (0.0006) [2023-12-27 13:28:00,327][20381] Updated weights for policy 1, policy_version 1127231 (0.0009) [2023-12-27 13:28:00,335][20347] Updated weights for policy 0, policy_version 1128363 (0.0008) [2023-12-27 13:28:00,381][20381] Updated weights for policy 1, policy_version 1127241 (0.0007) [2023-12-27 13:28:00,387][20347] Updated weights for policy 0, policy_version 1128373 (0.0008) [2023-12-27 13:28:00,434][20381] Updated weights for policy 1, policy_version 1127251 (0.0005) [2023-12-27 13:28:00,440][20347] Updated weights for policy 0, policy_version 1128383 (0.0006) [2023-12-27 13:28:01,067][20381] Updated weights for policy 1, policy_version 1127261 (0.0007) [2023-12-27 13:28:01,096][20347] Updated weights for policy 0, policy_version 1128393 (0.0007) [2023-12-27 13:28:01,128][20381] Updated weights for policy 1, policy_version 1127271 (0.0008) [2023-12-27 13:28:01,146][20347] Updated weights for policy 0, policy_version 1128403 (0.0006) [2023-12-27 13:28:01,195][20381] Updated weights for policy 1, policy_version 1127281 (0.0006) [2023-12-27 13:28:01,212][20347] Updated weights for policy 0, policy_version 1128413 (0.0008) [2023-12-27 13:28:01,875][20381] Updated weights for policy 1, policy_version 1127291 (0.0006) [2023-12-27 13:28:01,937][20381] Updated weights for policy 1, policy_version 1127301 (0.0007) [2023-12-27 13:28:01,973][20347] Updated weights for policy 0, policy_version 1128423 (0.0006) [2023-12-27 13:28:01,998][20381] Updated weights for policy 1, policy_version 1127311 (0.0006) [2023-12-27 13:28:02,031][20347] Updated weights for policy 0, policy_version 1128433 (0.0006) [2023-12-27 13:28:02,088][20347] Updated weights for policy 0, policy_version 1128443 (0.0008) [2023-12-27 13:28:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 577552384. Throughput: 0: 10316.7, 1: 10253.1. Samples: 577560732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:02,363][19170] Avg episode reward: [(0, '941.400'), (1, '676.590')] [2023-12-27 13:28:02,695][20381] Updated weights for policy 1, policy_version 1127321 (0.0007) [2023-12-27 13:28:02,754][20381] Updated weights for policy 1, policy_version 1127331 (0.0007) [2023-12-27 13:28:02,810][20347] Updated weights for policy 0, policy_version 1128453 (0.0007) [2023-12-27 13:28:02,812][20381] Updated weights for policy 1, policy_version 1127341 (0.0007) [2023-12-27 13:28:02,866][20347] Updated weights for policy 0, policy_version 1128463 (0.0005) [2023-12-27 13:28:02,868][20381] Updated weights for policy 1, policy_version 1127351 (0.0005) [2023-12-27 13:28:02,924][20347] Updated weights for policy 0, policy_version 1128473 (0.0007) [2023-12-27 13:28:03,539][20381] Updated weights for policy 1, policy_version 1127361 (0.0005) [2023-12-27 13:28:03,583][20381] Updated weights for policy 1, policy_version 1127371 (0.0005) [2023-12-27 13:28:03,614][20347] Updated weights for policy 0, policy_version 1128483 (0.0007) [2023-12-27 13:28:03,633][20381] Updated weights for policy 1, policy_version 1127381 (0.0005) [2023-12-27 13:28:03,669][20347] Updated weights for policy 0, policy_version 1128493 (0.0007) [2023-12-27 13:28:03,734][20347] Updated weights for policy 0, policy_version 1128503 (0.0008) [2023-12-27 13:28:04,311][20381] Updated weights for policy 1, policy_version 1127391 (0.0006) [2023-12-27 13:28:04,374][20381] Updated weights for policy 1, policy_version 1127401 (0.0008) [2023-12-27 13:28:04,429][20381] Updated weights for policy 1, policy_version 1127411 (0.0007) [2023-12-27 13:28:04,445][20347] Updated weights for policy 0, policy_version 1128513 (0.0006) [2023-12-27 13:28:04,494][20347] Updated weights for policy 0, policy_version 1128523 (0.0007) [2023-12-27 13:28:04,543][20347] Updated weights for policy 0, policy_version 1128533 (0.0008) [2023-12-27 13:28:04,600][20347] Updated weights for policy 0, policy_version 1128543 (0.0008) [2023-12-27 13:28:05,097][20381] Updated weights for policy 1, policy_version 1127421 (0.0007) [2023-12-27 13:28:05,139][20381] Updated weights for policy 1, policy_version 1127431 (0.0007) [2023-12-27 13:28:05,183][20381] Updated weights for policy 1, policy_version 1127441 (0.0007) [2023-12-27 13:28:05,307][20347] Updated weights for policy 0, policy_version 1128553 (0.0007) [2023-12-27 13:28:05,361][20347] Updated weights for policy 0, policy_version 1128563 (0.0008) [2023-12-27 13:28:05,410][20347] Updated weights for policy 0, policy_version 1128573 (0.0008) [2023-12-27 13:28:05,852][20381] Updated weights for policy 1, policy_version 1127451 (0.0008) [2023-12-27 13:28:05,906][20381] Updated weights for policy 1, policy_version 1127461 (0.0007) [2023-12-27 13:28:05,958][20381] Updated weights for policy 1, policy_version 1127471 (0.0008) [2023-12-27 13:28:06,052][20347] Updated weights for policy 0, policy_version 1128583 (0.0008) [2023-12-27 13:28:06,104][20347] Updated weights for policy 0, policy_version 1128593 (0.0009) [2023-12-27 13:28:06,168][20347] Updated weights for policy 0, policy_version 1128603 (0.0007) [2023-12-27 13:28:06,630][20381] Updated weights for policy 1, policy_version 1127481 (0.0006) [2023-12-27 13:28:06,694][20381] Updated weights for policy 1, policy_version 1127491 (0.0006) [2023-12-27 13:28:06,752][20381] Updated weights for policy 1, policy_version 1127501 (0.0005) [2023-12-27 13:28:06,785][20347] Updated weights for policy 0, policy_version 1128613 (0.0007) [2023-12-27 13:28:06,813][20381] Updated weights for policy 1, policy_version 1127511 (0.0007) [2023-12-27 13:28:06,843][20347] Updated weights for policy 0, policy_version 1128623 (0.0006) [2023-12-27 13:28:06,898][20347] Updated weights for policy 0, policy_version 1128633 (0.0007) [2023-12-27 13:28:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 577650688. Throughput: 0: 10269.6, 1: 10264.6. Samples: 577620816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:07,363][19170] Avg episode reward: [(0, '921.880'), (1, '696.310')] [2023-12-27 13:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001128640_288980992.pth... [2023-12-27 13:28:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001127456_288677888.pth [2023-12-27 13:28:07,421][20381] Updated weights for policy 1, policy_version 1127521 (0.0007) [2023-12-27 13:28:07,478][20381] Updated weights for policy 1, policy_version 1127531 (0.0008) [2023-12-27 13:28:07,532][20381] Updated weights for policy 1, policy_version 1127541 (0.0009) [2023-12-27 13:28:07,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001127544_288677888.pth... [2023-12-27 13:28:07,551][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001126328_288366592.pth [2023-12-27 13:28:07,614][20347] Updated weights for policy 0, policy_version 1128643 (0.0007) [2023-12-27 13:28:07,666][20347] Updated weights for policy 0, policy_version 1128653 (0.0008) [2023-12-27 13:28:07,723][20347] Updated weights for policy 0, policy_version 1128663 (0.0010) [2023-12-27 13:28:07,723][20291] KL-divergence is very high: 222.2203 [2023-12-27 13:28:07,741][20291] KL-divergence is very high: 199.1073 [2023-12-27 13:28:08,185][20381] Updated weights for policy 1, policy_version 1127551 (0.0010) [2023-12-27 13:28:08,245][20381] Updated weights for policy 1, policy_version 1127561 (0.0008) [2023-12-27 13:28:08,300][20381] Updated weights for policy 1, policy_version 1127571 (0.0007) [2023-12-27 13:28:08,411][20347] Updated weights for policy 0, policy_version 1128673 (0.0008) [2023-12-27 13:28:08,470][20347] Updated weights for policy 0, policy_version 1128683 (0.0007) [2023-12-27 13:28:08,524][20347] Updated weights for policy 0, policy_version 1128693 (0.0008) [2023-12-27 13:28:08,584][20347] Updated weights for policy 0, policy_version 1128703 (0.0007) [2023-12-27 13:28:08,984][20381] Updated weights for policy 1, policy_version 1127581 (0.0010) [2023-12-27 13:28:09,031][20381] Updated weights for policy 1, policy_version 1127591 (0.0007) [2023-12-27 13:28:09,087][20381] Updated weights for policy 1, policy_version 1127601 (0.0005) [2023-12-27 13:28:09,269][20347] Updated weights for policy 0, policy_version 1128713 (0.0008) [2023-12-27 13:28:09,334][20347] Updated weights for policy 0, policy_version 1128723 (0.0008) [2023-12-27 13:28:09,405][20347] Updated weights for policy 0, policy_version 1128733 (0.0008) [2023-12-27 13:28:09,801][20381] Updated weights for policy 1, policy_version 1127611 (0.0005) [2023-12-27 13:28:09,860][20381] Updated weights for policy 1, policy_version 1127621 (0.0009) [2023-12-27 13:28:09,923][20381] Updated weights for policy 1, policy_version 1127631 (0.0009) [2023-12-27 13:28:10,102][20347] Updated weights for policy 0, policy_version 1128743 (0.0008) [2023-12-27 13:28:10,154][20347] Updated weights for policy 0, policy_version 1128753 (0.0008) [2023-12-27 13:28:10,206][20347] Updated weights for policy 0, policy_version 1128763 (0.0008) [2023-12-27 13:28:10,669][20381] Updated weights for policy 1, policy_version 1127641 (0.0008) [2023-12-27 13:28:10,725][20381] Updated weights for policy 1, policy_version 1127651 (0.0010) [2023-12-27 13:28:10,778][20381] Updated weights for policy 1, policy_version 1127661 (0.0009) [2023-12-27 13:28:10,829][20381] Updated weights for policy 1, policy_version 1127671 (0.0006) [2023-12-27 13:28:10,909][20347] Updated weights for policy 0, policy_version 1128773 (0.0009) [2023-12-27 13:28:10,962][20347] Updated weights for policy 0, policy_version 1128783 (0.0010) [2023-12-27 13:28:11,010][20347] Updated weights for policy 0, policy_version 1128793 (0.0010) [2023-12-27 13:28:11,498][20381] Updated weights for policy 1, policy_version 1127681 (0.0008) [2023-12-27 13:28:11,556][20381] Updated weights for policy 1, policy_version 1127691 (0.0008) [2023-12-27 13:28:11,625][20381] Updated weights for policy 1, policy_version 1127702 (0.0008) [2023-12-27 13:28:11,791][20347] Updated weights for policy 0, policy_version 1128803 (0.0008) [2023-12-27 13:28:11,850][20347] Updated weights for policy 0, policy_version 1128813 (0.0008) [2023-12-27 13:28:11,910][20347] Updated weights for policy 0, policy_version 1128823 (0.0007) [2023-12-27 13:28:12,338][20381] Updated weights for policy 1, policy_version 1127712 (0.0010) [2023-12-27 13:28:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 577748992. Throughput: 0: 10286.6, 1: 10282.1. Samples: 577744116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:12,363][19170] Avg episode reward: [(0, '872.860'), (1, '676.850')] [2023-12-27 13:28:12,403][20381] Updated weights for policy 1, policy_version 1127722 (0.0009) [2023-12-27 13:28:12,462][20381] Updated weights for policy 1, policy_version 1127732 (0.0008) [2023-12-27 13:28:12,569][20347] Updated weights for policy 0, policy_version 1128833 (0.0005) [2023-12-27 13:28:12,620][20347] Updated weights for policy 0, policy_version 1128843 (0.0008) [2023-12-27 13:28:12,674][20347] Updated weights for policy 0, policy_version 1128853 (0.0008) [2023-12-27 13:28:12,728][20347] Updated weights for policy 0, policy_version 1128863 (0.0008) [2023-12-27 13:28:13,064][20381] Updated weights for policy 1, policy_version 1127742 (0.0008) [2023-12-27 13:28:13,111][20381] Updated weights for policy 1, policy_version 1127752 (0.0008) [2023-12-27 13:28:13,161][20381] Updated weights for policy 1, policy_version 1127762 (0.0007) [2023-12-27 13:28:13,452][20347] Updated weights for policy 0, policy_version 1128873 (0.0007) [2023-12-27 13:28:13,463][20291] KL-divergence is very high: 121.8570 [2023-12-27 13:28:13,469][20291] KL-divergence is very high: 110.8162 [2023-12-27 13:28:13,510][20347] Updated weights for policy 0, policy_version 1128883 (0.0007) [2023-12-27 13:28:13,565][20347] Updated weights for policy 0, policy_version 1128893 (0.0007) [2023-12-27 13:28:13,852][20381] Updated weights for policy 1, policy_version 1127772 (0.0006) [2023-12-27 13:28:13,903][20381] Updated weights for policy 1, policy_version 1127782 (0.0005) [2023-12-27 13:28:13,961][20381] Updated weights for policy 1, policy_version 1127792 (0.0005) [2023-12-27 13:28:14,278][20347] Updated weights for policy 0, policy_version 1128903 (0.0008) [2023-12-27 13:28:14,327][20347] Updated weights for policy 0, policy_version 1128913 (0.0009) [2023-12-27 13:28:14,388][20347] Updated weights for policy 0, policy_version 1128923 (0.0008) [2023-12-27 13:28:14,546][20381] Updated weights for policy 1, policy_version 1127802 (0.0006) [2023-12-27 13:28:14,594][20381] Updated weights for policy 1, policy_version 1127812 (0.0008) [2023-12-27 13:28:14,642][20381] Updated weights for policy 1, policy_version 1127822 (0.0007) [2023-12-27 13:28:14,690][20381] Updated weights for policy 1, policy_version 1127832 (0.0007) [2023-12-27 13:28:15,082][20347] Updated weights for policy 0, policy_version 1128933 (0.0009) [2023-12-27 13:28:15,143][20347] Updated weights for policy 0, policy_version 1128943 (0.0011) [2023-12-27 13:28:15,206][20347] Updated weights for policy 0, policy_version 1128953 (0.0011) [2023-12-27 13:28:15,380][20381] Updated weights for policy 1, policy_version 1127842 (0.0007) [2023-12-27 13:28:15,440][20381] Updated weights for policy 1, policy_version 1127852 (0.0008) [2023-12-27 13:28:15,497][20381] Updated weights for policy 1, policy_version 1127862 (0.0007) [2023-12-27 13:28:15,837][20347] Updated weights for policy 0, policy_version 1128963 (0.0010) [2023-12-27 13:28:15,892][20347] Updated weights for policy 0, policy_version 1128973 (0.0010) [2023-12-27 13:28:15,951][20347] Updated weights for policy 0, policy_version 1128983 (0.0011) [2023-12-27 13:28:16,247][20381] Updated weights for policy 1, policy_version 1127872 (0.0008) [2023-12-27 13:28:16,304][20381] Updated weights for policy 1, policy_version 1127882 (0.0008) [2023-12-27 13:28:16,353][20381] Updated weights for policy 1, policy_version 1127892 (0.0008) [2023-12-27 13:28:16,504][20347] Updated weights for policy 0, policy_version 1128993 (0.0010) [2023-12-27 13:28:16,558][20347] Updated weights for policy 0, policy_version 1129003 (0.0005) [2023-12-27 13:28:16,617][20347] Updated weights for policy 0, policy_version 1129013 (0.0005) [2023-12-27 13:28:16,674][20347] Updated weights for policy 0, policy_version 1129023 (0.0006) [2023-12-27 13:28:17,124][20381] Updated weights for policy 1, policy_version 1127902 (0.0008) [2023-12-27 13:28:17,176][20381] Updated weights for policy 1, policy_version 1127912 (0.0008) [2023-12-27 13:28:17,227][20381] Updated weights for policy 1, policy_version 1127922 (0.0007) [2023-12-27 13:28:17,269][20347] Updated weights for policy 0, policy_version 1129033 (0.0010) [2023-12-27 13:28:17,333][20347] Updated weights for policy 0, policy_version 1129043 (0.0010) [2023-12-27 13:28:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 577855488. Throughput: 0: 10252.2, 1: 10265.1. Samples: 577866588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:17,363][19170] Avg episode reward: [(0, '872.820'), (1, '745.530')] [2023-12-27 13:28:17,394][20347] Updated weights for policy 0, policy_version 1129053 (0.0010) [2023-12-27 13:28:17,893][20381] Updated weights for policy 1, policy_version 1127932 (0.0007) [2023-12-27 13:28:17,946][20381] Updated weights for policy 1, policy_version 1127942 (0.0006) [2023-12-27 13:28:18,009][20381] Updated weights for policy 1, policy_version 1127952 (0.0007) [2023-12-27 13:28:18,058][20347] Updated weights for policy 0, policy_version 1129063 (0.0007) [2023-12-27 13:28:18,113][20347] Updated weights for policy 0, policy_version 1129073 (0.0007) [2023-12-27 13:28:18,165][20347] Updated weights for policy 0, policy_version 1129083 (0.0007) [2023-12-27 13:28:18,664][20381] Updated weights for policy 1, policy_version 1127962 (0.0008) [2023-12-27 13:28:18,730][20381] Updated weights for policy 1, policy_version 1127972 (0.0011) [2023-12-27 13:28:18,798][20381] Updated weights for policy 1, policy_version 1127982 (0.0011) [2023-12-27 13:28:18,863][20381] Updated weights for policy 1, policy_version 1127992 (0.0011) [2023-12-27 13:28:18,916][20347] Updated weights for policy 0, policy_version 1129093 (0.0008) [2023-12-27 13:28:18,974][20347] Updated weights for policy 0, policy_version 1129103 (0.0006) [2023-12-27 13:28:19,034][20347] Updated weights for policy 0, policy_version 1129113 (0.0006) [2023-12-27 13:28:19,577][20381] Updated weights for policy 1, policy_version 1128002 (0.0006) [2023-12-27 13:28:19,642][20381] Updated weights for policy 1, policy_version 1128012 (0.0006) [2023-12-27 13:28:19,701][20381] Updated weights for policy 1, policy_version 1128022 (0.0010) [2023-12-27 13:28:19,771][20347] Updated weights for policy 0, policy_version 1129123 (0.0007) [2023-12-27 13:28:19,836][20347] Updated weights for policy 0, policy_version 1129133 (0.0007) [2023-12-27 13:28:19,886][20347] Updated weights for policy 0, policy_version 1129143 (0.0006) [2023-12-27 13:28:20,391][20381] Updated weights for policy 1, policy_version 1128032 (0.0007) [2023-12-27 13:28:20,454][20381] Updated weights for policy 1, policy_version 1128042 (0.0007) [2023-12-27 13:28:20,506][20381] Updated weights for policy 1, policy_version 1128052 (0.0008) [2023-12-27 13:28:20,584][20347] Updated weights for policy 0, policy_version 1129153 (0.0007) [2023-12-27 13:28:20,639][20347] Updated weights for policy 0, policy_version 1129163 (0.0008) [2023-12-27 13:28:20,695][20347] Updated weights for policy 0, policy_version 1129173 (0.0008) [2023-12-27 13:28:20,747][20347] Updated weights for policy 0, policy_version 1129183 (0.0005) [2023-12-27 13:28:21,247][20381] Updated weights for policy 1, policy_version 1128062 (0.0008) [2023-12-27 13:28:21,331][20381] Updated weights for policy 1, policy_version 1128072 (0.0009) [2023-12-27 13:28:21,401][20381] Updated weights for policy 1, policy_version 1128082 (0.0009) [2023-12-27 13:28:21,459][20347] Updated weights for policy 0, policy_version 1129193 (0.0007) [2023-12-27 13:28:21,524][20347] Updated weights for policy 0, policy_version 1129203 (0.0008) [2023-12-27 13:28:21,592][20347] Updated weights for policy 0, policy_version 1129213 (0.0007) [2023-12-27 13:28:22,119][20381] Updated weights for policy 1, policy_version 1128092 (0.0009) [2023-12-27 13:28:22,178][20381] Updated weights for policy 1, policy_version 1128102 (0.0010) [2023-12-27 13:28:22,241][20381] Updated weights for policy 1, policy_version 1128112 (0.0008) [2023-12-27 13:28:22,282][20347] Updated weights for policy 0, policy_version 1129223 (0.0007) [2023-12-27 13:28:22,342][20347] Updated weights for policy 0, policy_version 1129233 (0.0007) [2023-12-27 13:28:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.9, 300 sec: 20549.4). Total num frames: 577953792. Throughput: 0: 10250.3, 1: 10230.8. Samples: 577926992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:22,363][19170] Avg episode reward: [(0, '873.060'), (1, '755.420')] [2023-12-27 13:28:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001128120_288825344.pth... [2023-12-27 13:28:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001126936_288522240.pth [2023-12-27 13:28:22,409][20347] Updated weights for policy 0, policy_version 1129243 (0.0009) [2023-12-27 13:28:22,438][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001129248_289136640.pth... [2023-12-27 13:28:22,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001128064_288833536.pth [2023-12-27 13:28:22,935][20381] Updated weights for policy 1, policy_version 1128122 (0.0008) [2023-12-27 13:28:22,988][20381] Updated weights for policy 1, policy_version 1128132 (0.0009) [2023-12-27 13:28:23,041][20381] Updated weights for policy 1, policy_version 1128142 (0.0011) [2023-12-27 13:28:23,074][20347] Updated weights for policy 0, policy_version 1129253 (0.0007) [2023-12-27 13:28:23,087][20381] Updated weights for policy 1, policy_version 1128152 (0.0010) [2023-12-27 13:28:23,127][20347] Updated weights for policy 0, policy_version 1129263 (0.0006) [2023-12-27 13:28:23,177][20347] Updated weights for policy 0, policy_version 1129273 (0.0007) [2023-12-27 13:28:23,756][20381] Updated weights for policy 1, policy_version 1128162 (0.0006) [2023-12-27 13:28:23,804][20381] Updated weights for policy 1, policy_version 1128172 (0.0005) [2023-12-27 13:28:23,855][20381] Updated weights for policy 1, policy_version 1128182 (0.0005) [2023-12-27 13:28:23,886][20347] Updated weights for policy 0, policy_version 1129283 (0.0007) [2023-12-27 13:28:23,938][20347] Updated weights for policy 0, policy_version 1129293 (0.0007) [2023-12-27 13:28:23,984][20347] Updated weights for policy 0, policy_version 1129303 (0.0007) [2023-12-27 13:28:24,511][20381] Updated weights for policy 1, policy_version 1128192 (0.0010) [2023-12-27 13:28:24,570][20381] Updated weights for policy 1, policy_version 1128202 (0.0011) [2023-12-27 13:28:24,628][20347] Updated weights for policy 0, policy_version 1129313 (0.0007) [2023-12-27 13:28:24,629][20381] Updated weights for policy 1, policy_version 1128212 (0.0011) [2023-12-27 13:28:24,678][20347] Updated weights for policy 0, policy_version 1129323 (0.0005) [2023-12-27 13:28:24,735][20347] Updated weights for policy 0, policy_version 1129333 (0.0007) [2023-12-27 13:28:24,784][20347] Updated weights for policy 0, policy_version 1129343 (0.0007) [2023-12-27 13:28:25,283][20381] Updated weights for policy 1, policy_version 1128222 (0.0010) [2023-12-27 13:28:25,323][20347] Updated weights for policy 0, policy_version 1129353 (0.0005) [2023-12-27 13:28:25,347][20381] Updated weights for policy 1, policy_version 1128232 (0.0010) [2023-12-27 13:28:25,383][20347] Updated weights for policy 0, policy_version 1129363 (0.0008) [2023-12-27 13:28:25,399][20381] Updated weights for policy 1, policy_version 1128242 (0.0010) [2023-12-27 13:28:25,436][20347] Updated weights for policy 0, policy_version 1129373 (0.0010) [2023-12-27 13:28:26,053][20347] Updated weights for policy 0, policy_version 1129383 (0.0007) [2023-12-27 13:28:26,087][20381] Updated weights for policy 1, policy_version 1128252 (0.0010) [2023-12-27 13:28:26,114][20347] Updated weights for policy 0, policy_version 1129393 (0.0005) [2023-12-27 13:28:26,138][20381] Updated weights for policy 1, policy_version 1128262 (0.0009) [2023-12-27 13:28:26,161][20347] Updated weights for policy 0, policy_version 1129403 (0.0005) [2023-12-27 13:28:26,190][20381] Updated weights for policy 1, policy_version 1128272 (0.0010) [2023-12-27 13:28:26,719][20347] Updated weights for policy 0, policy_version 1129413 (0.0006) [2023-12-27 13:28:26,772][20347] Updated weights for policy 0, policy_version 1129423 (0.0006) [2023-12-27 13:28:26,807][20381] Updated weights for policy 1, policy_version 1128282 (0.0009) [2023-12-27 13:28:26,817][20347] Updated weights for policy 0, policy_version 1129433 (0.0007) [2023-12-27 13:28:26,861][20381] Updated weights for policy 1, policy_version 1128292 (0.0005) [2023-12-27 13:28:26,918][20381] Updated weights for policy 1, policy_version 1128302 (0.0005) [2023-12-27 13:28:26,984][20381] Updated weights for policy 1, policy_version 1128312 (0.0010) [2023-12-27 13:28:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 578060288. Throughput: 0: 10318.1, 1: 10209.9. Samples: 578050160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:27,363][19170] Avg episode reward: [(0, '882.990'), (1, '735.810')] [2023-12-27 13:28:27,386][20347] Updated weights for policy 0, policy_version 1129443 (0.0009) [2023-12-27 13:28:27,449][20347] Updated weights for policy 0, policy_version 1129453 (0.0005) [2023-12-27 13:28:27,507][20347] Updated weights for policy 0, policy_version 1129463 (0.0005) [2023-12-27 13:28:27,640][20381] Updated weights for policy 1, policy_version 1128322 (0.0008) [2023-12-27 13:28:27,696][20381] Updated weights for policy 1, policy_version 1128332 (0.0007) [2023-12-27 13:28:27,758][20381] Updated weights for policy 1, policy_version 1128342 (0.0007) [2023-12-27 13:28:28,061][20347] Updated weights for policy 0, policy_version 1129473 (0.0006) [2023-12-27 13:28:28,105][20347] Updated weights for policy 0, policy_version 1129483 (0.0010) [2023-12-27 13:28:28,155][20347] Updated weights for policy 0, policy_version 1129493 (0.0010) [2023-12-27 13:28:28,212][20347] Updated weights for policy 0, policy_version 1129503 (0.0010) [2023-12-27 13:28:28,501][20381] Updated weights for policy 1, policy_version 1128352 (0.0010) [2023-12-27 13:28:28,554][20381] Updated weights for policy 1, policy_version 1128362 (0.0007) [2023-12-27 13:28:28,599][20381] Updated weights for policy 1, policy_version 1128372 (0.0005) [2023-12-27 13:28:28,934][20347] Updated weights for policy 0, policy_version 1129513 (0.0009) [2023-12-27 13:28:28,984][20347] Updated weights for policy 0, policy_version 1129523 (0.0006) [2023-12-27 13:28:29,059][20347] Updated weights for policy 0, policy_version 1129533 (0.0008) [2023-12-27 13:28:29,255][20381] Updated weights for policy 1, policy_version 1128382 (0.0007) [2023-12-27 13:28:29,306][20381] Updated weights for policy 1, policy_version 1128392 (0.0008) [2023-12-27 13:28:29,370][20381] Updated weights for policy 1, policy_version 1128402 (0.0008) [2023-12-27 13:28:29,693][20347] Updated weights for policy 0, policy_version 1129543 (0.0008) [2023-12-27 13:28:29,750][20347] Updated weights for policy 0, policy_version 1129553 (0.0007) [2023-12-27 13:28:29,801][20347] Updated weights for policy 0, policy_version 1129563 (0.0006) [2023-12-27 13:28:30,042][20381] Updated weights for policy 1, policy_version 1128412 (0.0009) [2023-12-27 13:28:30,098][20381] Updated weights for policy 1, policy_version 1128422 (0.0011) [2023-12-27 13:28:30,144][20381] Updated weights for policy 1, policy_version 1128432 (0.0010) [2023-12-27 13:28:30,434][20347] Updated weights for policy 0, policy_version 1129573 (0.0007) [2023-12-27 13:28:30,469][20291] KL-divergence is very high: 145.0320 [2023-12-27 13:28:30,489][20347] Updated weights for policy 0, policy_version 1129583 (0.0007) [2023-12-27 13:28:30,533][20347] Updated weights for policy 0, policy_version 1129593 (0.0007) [2023-12-27 13:28:30,852][20381] Updated weights for policy 1, policy_version 1128442 (0.0011) [2023-12-27 13:28:30,895][20381] Updated weights for policy 1, policy_version 1128452 (0.0008) [2023-12-27 13:28:30,938][20381] Updated weights for policy 1, policy_version 1128462 (0.0005) [2023-12-27 13:28:30,983][20381] Updated weights for policy 1, policy_version 1128472 (0.0006) [2023-12-27 13:28:31,162][20347] Updated weights for policy 0, policy_version 1129603 (0.0005) [2023-12-27 13:28:31,222][20347] Updated weights for policy 0, policy_version 1129613 (0.0007) [2023-12-27 13:28:31,288][20347] Updated weights for policy 0, policy_version 1129623 (0.0007) [2023-12-27 13:28:31,696][20381] Updated weights for policy 1, policy_version 1128482 (0.0008) [2023-12-27 13:28:31,761][20381] Updated weights for policy 1, policy_version 1128492 (0.0007) [2023-12-27 13:28:31,824][20381] Updated weights for policy 1, policy_version 1128502 (0.0006) [2023-12-27 13:28:32,001][20347] Updated weights for policy 0, policy_version 1129633 (0.0007) [2023-12-27 13:28:32,048][20291] KL-divergence is very high: 109.4121 [2023-12-27 13:28:32,060][20347] Updated weights for policy 0, policy_version 1129643 (0.0008) [2023-12-27 13:28:32,118][20347] Updated weights for policy 0, policy_version 1129653 (0.0008) [2023-12-27 13:28:32,143][20291] KL-divergence is very high: 168.8833 [2023-12-27 13:28:32,149][20291] KL-divergence is very high: 133.9899 [2023-12-27 13:28:32,180][20347] Updated weights for policy 0, policy_version 1129663 (0.0008) [2023-12-27 13:28:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 578166784. Throughput: 0: 10348.6, 1: 10196.5. Samples: 578177232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:32,363][19170] Avg episode reward: [(0, '823.800'), (1, '676.510')] [2023-12-27 13:28:32,490][20381] Updated weights for policy 1, policy_version 1128512 (0.0007) [2023-12-27 13:28:32,555][20381] Updated weights for policy 1, policy_version 1128522 (0.0009) [2023-12-27 13:28:32,615][20381] Updated weights for policy 1, policy_version 1128532 (0.0009) [2023-12-27 13:28:32,797][20291] KL-divergence is very high: 250.9250 [2023-12-27 13:28:32,830][20291] KL-divergence is very high: 118.4481 [2023-12-27 13:28:32,836][20347] Updated weights for policy 0, policy_version 1129673 (0.0007) [2023-12-27 13:28:32,851][20291] KL-divergence is very high: 118.5907 [2023-12-27 13:28:32,882][20291] KL-divergence is very high: 216.0113 [2023-12-27 13:28:32,901][20291] KL-divergence is very high: 179.2480 [2023-12-27 13:28:32,902][20347] Updated weights for policy 0, policy_version 1129683 (0.0007) [2023-12-27 13:28:32,913][20291] KL-divergence is very high: 151.8851 [2023-12-27 13:28:32,932][20291] KL-divergence is very high: 146.1357 [2023-12-27 13:28:32,948][20291] KL-divergence is very high: 191.8945 [2023-12-27 13:28:32,960][20291] KL-divergence is very high: 122.5073 [2023-12-27 13:28:32,960][20347] Updated weights for policy 0, policy_version 1129693 (0.0010) [2023-12-27 13:28:33,285][20381] Updated weights for policy 1, policy_version 1128542 (0.0007) [2023-12-27 13:28:33,333][20381] Updated weights for policy 1, policy_version 1128552 (0.0007) [2023-12-27 13:28:33,383][20381] Updated weights for policy 1, policy_version 1128562 (0.0008) [2023-12-27 13:28:33,596][20347] Updated weights for policy 0, policy_version 1129703 (0.0010) [2023-12-27 13:28:33,646][20347] Updated weights for policy 0, policy_version 1129713 (0.0007) [2023-12-27 13:28:33,701][20347] Updated weights for policy 0, policy_version 1129723 (0.0007) [2023-12-27 13:28:34,077][20381] Updated weights for policy 1, policy_version 1128572 (0.0008) [2023-12-27 13:28:34,144][20381] Updated weights for policy 1, policy_version 1128582 (0.0008) [2023-12-27 13:28:34,208][20381] Updated weights for policy 1, policy_version 1128592 (0.0007) [2023-12-27 13:28:34,367][20347] Updated weights for policy 0, policy_version 1129733 (0.0007) [2023-12-27 13:28:34,415][20347] Updated weights for policy 0, policy_version 1129743 (0.0006) [2023-12-27 13:28:34,487][20347] Updated weights for policy 0, policy_version 1129753 (0.0010) [2023-12-27 13:28:34,940][20381] Updated weights for policy 1, policy_version 1128602 (0.0008) [2023-12-27 13:28:34,993][20381] Updated weights for policy 1, policy_version 1128612 (0.0009) [2023-12-27 13:28:35,043][20381] Updated weights for policy 1, policy_version 1128623 (0.0007) [2023-12-27 13:28:35,205][20347] Updated weights for policy 0, policy_version 1129763 (0.0009) [2023-12-27 13:28:35,260][20347] Updated weights for policy 0, policy_version 1129773 (0.0007) [2023-12-27 13:28:35,305][20347] Updated weights for policy 0, policy_version 1129783 (0.0008) [2023-12-27 13:28:35,698][20381] Updated weights for policy 1, policy_version 1128633 (0.0009) [2023-12-27 13:28:35,748][20381] Updated weights for policy 1, policy_version 1128643 (0.0007) [2023-12-27 13:28:35,807][20381] Updated weights for policy 1, policy_version 1128653 (0.0010) [2023-12-27 13:28:35,865][20381] Updated weights for policy 1, policy_version 1128663 (0.0010) [2023-12-27 13:28:35,998][20347] Updated weights for policy 0, policy_version 1129793 (0.0008) [2023-12-27 13:28:36,060][20347] Updated weights for policy 0, policy_version 1129803 (0.0010) [2023-12-27 13:28:36,117][20347] Updated weights for policy 0, policy_version 1129813 (0.0010) [2023-12-27 13:28:36,181][20347] Updated weights for policy 0, policy_version 1129823 (0.0010) [2023-12-27 13:28:36,538][20381] Updated weights for policy 1, policy_version 1128673 (0.0011) [2023-12-27 13:28:36,591][20381] Updated weights for policy 1, policy_version 1128683 (0.0011) [2023-12-27 13:28:36,644][20381] Updated weights for policy 1, policy_version 1128693 (0.0011) [2023-12-27 13:28:36,870][20347] Updated weights for policy 0, policy_version 1129833 (0.0010) [2023-12-27 13:28:36,925][20347] Updated weights for policy 0, policy_version 1129843 (0.0005) [2023-12-27 13:28:36,976][20347] Updated weights for policy 0, policy_version 1129853 (0.0005) [2023-12-27 13:28:37,355][20381] Updated weights for policy 1, policy_version 1128703 (0.0008) [2023-12-27 13:28:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 578265088. Throughput: 0: 10334.4, 1: 10178.5. Samples: 578237964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:37,363][19170] Avg episode reward: [(0, '794.240'), (1, '657.110')] [2023-12-27 13:28:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001129856_289292288.pth... [2023-12-27 13:28:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001128640_288980992.pth [2023-12-27 13:28:37,412][20381] Updated weights for policy 1, policy_version 1128713 (0.0010) [2023-12-27 13:28:37,474][20381] Updated weights for policy 1, policy_version 1128723 (0.0010) [2023-12-27 13:28:37,496][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001128728_288980992.pth... [2023-12-27 13:28:37,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001127544_288677888.pth [2023-12-27 13:28:37,576][20347] Updated weights for policy 0, policy_version 1129863 (0.0005) [2023-12-27 13:28:37,636][20347] Updated weights for policy 0, policy_version 1129873 (0.0005) [2023-12-27 13:28:37,696][20347] Updated weights for policy 0, policy_version 1129883 (0.0005) [2023-12-27 13:28:38,122][20381] Updated weights for policy 1, policy_version 1128733 (0.0010) [2023-12-27 13:28:38,173][20381] Updated weights for policy 1, policy_version 1128743 (0.0008) [2023-12-27 13:28:38,223][20381] Updated weights for policy 1, policy_version 1128753 (0.0005) [2023-12-27 13:28:38,270][20347] Updated weights for policy 0, policy_version 1129893 (0.0008) [2023-12-27 13:28:38,319][20347] Updated weights for policy 0, policy_version 1129903 (0.0010) [2023-12-27 13:28:38,385][20347] Updated weights for policy 0, policy_version 1129913 (0.0010) [2023-12-27 13:28:38,892][20381] Updated weights for policy 1, policy_version 1128763 (0.0008) [2023-12-27 13:28:38,948][20381] Updated weights for policy 1, policy_version 1128773 (0.0011) [2023-12-27 13:28:39,004][20381] Updated weights for policy 1, policy_version 1128783 (0.0011) [2023-12-27 13:28:39,079][20347] Updated weights for policy 0, policy_version 1129923 (0.0010) [2023-12-27 13:28:39,139][20347] Updated weights for policy 0, policy_version 1129933 (0.0010) [2023-12-27 13:28:39,172][20291] KL-divergence is very high: 112.1083 [2023-12-27 13:28:39,197][20347] Updated weights for policy 0, policy_version 1129943 (0.0010) [2023-12-27 13:28:39,742][20381] Updated weights for policy 1, policy_version 1128793 (0.0007) [2023-12-27 13:28:39,805][20381] Updated weights for policy 1, policy_version 1128803 (0.0008) [2023-12-27 13:28:39,869][20381] Updated weights for policy 1, policy_version 1128813 (0.0007) [2023-12-27 13:28:39,932][20381] Updated weights for policy 1, policy_version 1128823 (0.0008) [2023-12-27 13:28:39,938][20347] Updated weights for policy 0, policy_version 1129953 (0.0010) [2023-12-27 13:28:40,006][20347] Updated weights for policy 0, policy_version 1129963 (0.0007) [2023-12-27 13:28:40,065][20347] Updated weights for policy 0, policy_version 1129973 (0.0007) [2023-12-27 13:28:40,131][20347] Updated weights for policy 0, policy_version 1129983 (0.0008) [2023-12-27 13:28:40,589][20381] Updated weights for policy 1, policy_version 1128833 (0.0008) [2023-12-27 13:28:40,641][20381] Updated weights for policy 1, policy_version 1128843 (0.0005) [2023-12-27 13:28:40,689][20381] Updated weights for policy 1, policy_version 1128853 (0.0005) [2023-12-27 13:28:40,801][20347] Updated weights for policy 0, policy_version 1129993 (0.0008) [2023-12-27 13:28:40,851][20347] Updated weights for policy 0, policy_version 1130003 (0.0009) [2023-12-27 13:28:40,901][20347] Updated weights for policy 0, policy_version 1130013 (0.0008) [2023-12-27 13:28:41,357][20381] Updated weights for policy 1, policy_version 1128863 (0.0008) [2023-12-27 13:28:41,424][20381] Updated weights for policy 1, policy_version 1128873 (0.0008) [2023-12-27 13:28:41,484][20381] Updated weights for policy 1, policy_version 1128883 (0.0008) [2023-12-27 13:28:41,671][20347] Updated weights for policy 0, policy_version 1130023 (0.0006) [2023-12-27 13:28:41,734][20347] Updated weights for policy 0, policy_version 1130033 (0.0009) [2023-12-27 13:28:41,789][20347] Updated weights for policy 0, policy_version 1130043 (0.0008) [2023-12-27 13:28:42,141][20381] Updated weights for policy 1, policy_version 1128893 (0.0008) [2023-12-27 13:28:42,190][20381] Updated weights for policy 1, policy_version 1128903 (0.0007) [2023-12-27 13:28:42,239][20381] Updated weights for policy 1, policy_version 1128913 (0.0007) [2023-12-27 13:28:42,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 578371584. Throughput: 0: 10306.5, 1: 10175.4. Samples: 578361144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:42,363][19170] Avg episode reward: [(0, '784.830'), (1, '657.120')] [2023-12-27 13:28:42,561][20347] Updated weights for policy 0, policy_version 1130053 (0.0008) [2023-12-27 13:28:42,626][20347] Updated weights for policy 0, policy_version 1130063 (0.0008) [2023-12-27 13:28:42,684][20347] Updated weights for policy 0, policy_version 1130073 (0.0007) [2023-12-27 13:28:42,899][20381] Updated weights for policy 1, policy_version 1128923 (0.0006) [2023-12-27 13:28:42,950][20381] Updated weights for policy 1, policy_version 1128933 (0.0005) [2023-12-27 13:28:43,008][20381] Updated weights for policy 1, policy_version 1128943 (0.0005) [2023-12-27 13:28:43,434][20347] Updated weights for policy 0, policy_version 1130083 (0.0008) [2023-12-27 13:28:43,495][20347] Updated weights for policy 0, policy_version 1130093 (0.0008) [2023-12-27 13:28:43,565][20347] Updated weights for policy 0, policy_version 1130103 (0.0008) [2023-12-27 13:28:43,574][20381] Updated weights for policy 1, policy_version 1128953 (0.0006) [2023-12-27 13:28:43,632][20381] Updated weights for policy 1, policy_version 1128963 (0.0007) [2023-12-27 13:28:43,688][20381] Updated weights for policy 1, policy_version 1128973 (0.0008) [2023-12-27 13:28:43,736][20381] Updated weights for policy 1, policy_version 1128983 (0.0008) [2023-12-27 13:28:44,244][20347] Updated weights for policy 0, policy_version 1130113 (0.0008) [2023-12-27 13:28:44,293][20347] Updated weights for policy 0, policy_version 1130123 (0.0009) [2023-12-27 13:28:44,299][20291] KL-divergence is very high: 123.8200 [2023-12-27 13:28:44,344][20347] Updated weights for policy 0, policy_version 1130133 (0.0008) [2023-12-27 13:28:44,351][20381] Updated weights for policy 1, policy_version 1128993 (0.0006) [2023-12-27 13:28:44,400][20347] Updated weights for policy 0, policy_version 1130143 (0.0008) [2023-12-27 13:28:44,414][20381] Updated weights for policy 1, policy_version 1129003 (0.0007) [2023-12-27 13:28:44,477][20381] Updated weights for policy 1, policy_version 1129013 (0.0006) [2023-12-27 13:28:45,115][20381] Updated weights for policy 1, policy_version 1129023 (0.0007) [2023-12-27 13:28:45,138][20347] Updated weights for policy 0, policy_version 1130153 (0.0008) [2023-12-27 13:28:45,175][20381] Updated weights for policy 1, policy_version 1129033 (0.0007) [2023-12-27 13:28:45,194][20347] Updated weights for policy 0, policy_version 1130163 (0.0008) [2023-12-27 13:28:45,231][20381] Updated weights for policy 1, policy_version 1129043 (0.0006) [2023-12-27 13:28:45,245][20347] Updated weights for policy 0, policy_version 1130173 (0.0008) [2023-12-27 13:28:45,799][20381] Updated weights for policy 1, policy_version 1129053 (0.0007) [2023-12-27 13:28:45,856][20381] Updated weights for policy 1, policy_version 1129063 (0.0008) [2023-12-27 13:28:45,915][20381] Updated weights for policy 1, policy_version 1129073 (0.0007) [2023-12-27 13:28:45,958][20347] Updated weights for policy 0, policy_version 1130183 (0.0007) [2023-12-27 13:28:46,011][20347] Updated weights for policy 0, policy_version 1130193 (0.0008) [2023-12-27 13:28:46,067][20347] Updated weights for policy 0, policy_version 1130203 (0.0008) [2023-12-27 13:28:46,574][20381] Updated weights for policy 1, policy_version 1129083 (0.0007) [2023-12-27 13:28:46,631][20381] Updated weights for policy 1, policy_version 1129093 (0.0008) [2023-12-27 13:28:46,687][20381] Updated weights for policy 1, policy_version 1129103 (0.0008) [2023-12-27 13:28:46,727][20347] Updated weights for policy 0, policy_version 1130213 (0.0008) [2023-12-27 13:28:46,791][20347] Updated weights for policy 0, policy_version 1130223 (0.0009) [2023-12-27 13:28:46,855][20347] Updated weights for policy 0, policy_version 1130233 (0.0010) [2023-12-27 13:28:47,343][20381] Updated weights for policy 1, policy_version 1129113 (0.0007) [2023-12-27 13:28:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 578469888. Throughput: 0: 10232.3, 1: 10292.8. Samples: 578484360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:47,363][19170] Avg episode reward: [(0, '823.980'), (1, '667.100')] [2023-12-27 13:28:47,405][20381] Updated weights for policy 1, policy_version 1129123 (0.0007) [2023-12-27 13:28:47,430][20347] Updated weights for policy 0, policy_version 1130243 (0.0010) [2023-12-27 13:28:47,461][20381] Updated weights for policy 1, policy_version 1129133 (0.0006) [2023-12-27 13:28:47,484][20347] Updated weights for policy 0, policy_version 1130253 (0.0010) [2023-12-27 13:28:47,516][20381] Updated weights for policy 1, policy_version 1129143 (0.0006) [2023-12-27 13:28:47,536][20347] Updated weights for policy 0, policy_version 1130263 (0.0010) [2023-12-27 13:28:48,194][20347] Updated weights for policy 0, policy_version 1130273 (0.0010) [2023-12-27 13:28:48,226][20381] Updated weights for policy 1, policy_version 1129153 (0.0007) [2023-12-27 13:28:48,249][20347] Updated weights for policy 0, policy_version 1130283 (0.0005) [2023-12-27 13:28:48,276][20381] Updated weights for policy 1, policy_version 1129163 (0.0007) [2023-12-27 13:28:48,303][20347] Updated weights for policy 0, policy_version 1130293 (0.0005) [2023-12-27 13:28:48,333][20381] Updated weights for policy 1, policy_version 1129173 (0.0007) [2023-12-27 13:28:48,363][20347] Updated weights for policy 0, policy_version 1130303 (0.0008) [2023-12-27 13:28:49,035][20347] Updated weights for policy 0, policy_version 1130313 (0.0006) [2023-12-27 13:28:49,046][20381] Updated weights for policy 1, policy_version 1129183 (0.0009) [2023-12-27 13:28:49,094][20347] Updated weights for policy 0, policy_version 1130323 (0.0008) [2023-12-27 13:28:49,106][20381] Updated weights for policy 1, policy_version 1129193 (0.0009) [2023-12-27 13:28:49,155][20347] Updated weights for policy 0, policy_version 1130333 (0.0007) [2023-12-27 13:28:49,164][20381] Updated weights for policy 1, policy_version 1129203 (0.0007) [2023-12-27 13:28:49,884][20381] Updated weights for policy 1, policy_version 1129213 (0.0009) [2023-12-27 13:28:49,886][20347] Updated weights for policy 0, policy_version 1130343 (0.0007) [2023-12-27 13:28:49,945][20381] Updated weights for policy 1, policy_version 1129223 (0.0010) [2023-12-27 13:28:49,948][20347] Updated weights for policy 0, policy_version 1130353 (0.0008) [2023-12-27 13:28:50,011][20347] Updated weights for policy 0, policy_version 1130363 (0.0008) [2023-12-27 13:28:50,013][20381] Updated weights for policy 1, policy_version 1129233 (0.0008) [2023-12-27 13:28:50,736][20381] Updated weights for policy 1, policy_version 1129243 (0.0011) [2023-12-27 13:28:50,761][20347] Updated weights for policy 0, policy_version 1130373 (0.0008) [2023-12-27 13:28:50,794][20381] Updated weights for policy 1, policy_version 1129253 (0.0008) [2023-12-27 13:28:50,827][20347] Updated weights for policy 0, policy_version 1130383 (0.0011) [2023-12-27 13:28:50,852][20381] Updated weights for policy 1, policy_version 1129263 (0.0008) [2023-12-27 13:28:50,883][20347] Updated weights for policy 0, policy_version 1130393 (0.0011) [2023-12-27 13:28:51,559][20381] Updated weights for policy 1, policy_version 1129273 (0.0010) [2023-12-27 13:28:51,570][20347] Updated weights for policy 0, policy_version 1130403 (0.0010) [2023-12-27 13:28:51,620][20381] Updated weights for policy 1, policy_version 1129283 (0.0007) [2023-12-27 13:28:51,636][20347] Updated weights for policy 0, policy_version 1130413 (0.0012) [2023-12-27 13:28:51,686][20381] Updated weights for policy 1, policy_version 1129293 (0.0008) [2023-12-27 13:28:51,696][20347] Updated weights for policy 0, policy_version 1130423 (0.0008) [2023-12-27 13:28:51,756][20381] Updated weights for policy 1, policy_version 1129303 (0.0008) [2023-12-27 13:28:52,358][20347] Updated weights for policy 0, policy_version 1130433 (0.0007) [2023-12-27 13:28:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 578568192. Throughput: 0: 10258.9, 1: 10254.7. Samples: 578543928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:52,363][19170] Avg episode reward: [(0, '794.960'), (1, '745.290')] [2023-12-27 13:28:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001129304_289128448.pth... [2023-12-27 13:28:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001128120_288825344.pth [2023-12-27 13:28:52,419][20347] Updated weights for policy 0, policy_version 1130443 (0.0009) [2023-12-27 13:28:52,456][20291] KL-divergence is very high: 129.7403 [2023-12-27 13:28:52,473][20347] Updated weights for policy 0, policy_version 1130453 (0.0008) [2023-12-27 13:28:52,485][20381] Updated weights for policy 1, policy_version 1129313 (0.0006) [2023-12-27 13:28:52,529][20347] Updated weights for policy 0, policy_version 1130463 (0.0007) [2023-12-27 13:28:52,535][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001130464_289447936.pth... [2023-12-27 13:28:52,539][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001129248_289136640.pth [2023-12-27 13:28:52,549][20381] Updated weights for policy 1, policy_version 1129323 (0.0006) [2023-12-27 13:28:52,623][20381] Updated weights for policy 1, policy_version 1129333 (0.0007) [2023-12-27 13:28:53,233][20381] Updated weights for policy 1, policy_version 1129343 (0.0006) [2023-12-27 13:28:53,235][20347] Updated weights for policy 0, policy_version 1130473 (0.0010) [2023-12-27 13:28:53,285][20381] Updated weights for policy 1, policy_version 1129353 (0.0006) [2023-12-27 13:28:53,290][20347] Updated weights for policy 0, policy_version 1130483 (0.0009) [2023-12-27 13:28:53,333][20381] Updated weights for policy 1, policy_version 1129363 (0.0006) [2023-12-27 13:28:53,338][20347] Updated weights for policy 0, policy_version 1130493 (0.0010) [2023-12-27 13:28:53,924][20347] Updated weights for policy 0, policy_version 1130503 (0.0010) [2023-12-27 13:28:53,959][20381] Updated weights for policy 1, policy_version 1129373 (0.0008) [2023-12-27 13:28:53,986][20347] Updated weights for policy 0, policy_version 1130513 (0.0010) [2023-12-27 13:28:54,013][20381] Updated weights for policy 1, policy_version 1129383 (0.0007) [2023-12-27 13:28:54,041][20347] Updated weights for policy 0, policy_version 1130523 (0.0010) [2023-12-27 13:28:54,061][20381] Updated weights for policy 1, policy_version 1129393 (0.0005) [2023-12-27 13:28:54,666][20347] Updated weights for policy 0, policy_version 1130533 (0.0010) [2023-12-27 13:28:54,691][20381] Updated weights for policy 1, policy_version 1129403 (0.0005) [2023-12-27 13:28:54,727][20347] Updated weights for policy 0, policy_version 1130543 (0.0010) [2023-12-27 13:28:54,755][20381] Updated weights for policy 1, policy_version 1129413 (0.0007) [2023-12-27 13:28:54,789][20347] Updated weights for policy 0, policy_version 1130553 (0.0010) [2023-12-27 13:28:54,808][20381] Updated weights for policy 1, policy_version 1129423 (0.0007) [2023-12-27 13:28:55,470][20347] Updated weights for policy 0, policy_version 1130563 (0.0007) [2023-12-27 13:28:55,502][20381] Updated weights for policy 1, policy_version 1129433 (0.0007) [2023-12-27 13:28:55,528][20347] Updated weights for policy 0, policy_version 1130573 (0.0007) [2023-12-27 13:28:55,549][20381] Updated weights for policy 1, policy_version 1129443 (0.0006) [2023-12-27 13:28:55,590][20347] Updated weights for policy 0, policy_version 1130583 (0.0007) [2023-12-27 13:28:55,597][20381] Updated weights for policy 1, policy_version 1129453 (0.0006) [2023-12-27 13:28:55,643][20381] Updated weights for policy 1, policy_version 1129463 (0.0007) [2023-12-27 13:28:56,185][20347] Updated weights for policy 0, policy_version 1130593 (0.0007) [2023-12-27 13:28:56,246][20347] Updated weights for policy 0, policy_version 1130603 (0.0005) [2023-12-27 13:28:56,300][20347] Updated weights for policy 0, policy_version 1130613 (0.0006) [2023-12-27 13:28:56,348][20381] Updated weights for policy 1, policy_version 1129473 (0.0008) [2023-12-27 13:28:56,361][20347] Updated weights for policy 0, policy_version 1130623 (0.0006) [2023-12-27 13:28:56,406][20381] Updated weights for policy 1, policy_version 1129483 (0.0007) [2023-12-27 13:28:56,457][20381] Updated weights for policy 1, policy_version 1129493 (0.0008) [2023-12-27 13:28:56,920][20347] Updated weights for policy 0, policy_version 1130633 (0.0005) [2023-12-27 13:28:56,978][20347] Updated weights for policy 0, policy_version 1130643 (0.0008) [2023-12-27 13:28:57,032][20347] Updated weights for policy 0, policy_version 1130653 (0.0010) [2023-12-27 13:28:57,133][20381] Updated weights for policy 1, policy_version 1129503 (0.0007) [2023-12-27 13:28:57,190][20381] Updated weights for policy 1, policy_version 1129513 (0.0007) [2023-12-27 13:28:57,237][20381] Updated weights for policy 1, policy_version 1129523 (0.0007) [2023-12-27 13:28:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 578682880. Throughput: 0: 10296.2, 1: 10225.2. Samples: 578667580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:28:57,363][19170] Avg episode reward: [(0, '853.510'), (1, '735.350')] [2023-12-27 13:28:57,620][20347] Updated weights for policy 0, policy_version 1130663 (0.0010) [2023-12-27 13:28:57,667][20347] Updated weights for policy 0, policy_version 1130673 (0.0010) [2023-12-27 13:28:57,722][20347] Updated weights for policy 0, policy_version 1130683 (0.0010) [2023-12-27 13:28:57,948][20381] Updated weights for policy 1, policy_version 1129533 (0.0007) [2023-12-27 13:28:58,010][20381] Updated weights for policy 1, policy_version 1129543 (0.0006) [2023-12-27 13:28:58,073][20381] Updated weights for policy 1, policy_version 1129553 (0.0006) [2023-12-27 13:28:58,419][20347] Updated weights for policy 0, policy_version 1130693 (0.0011) [2023-12-27 13:28:58,486][20347] Updated weights for policy 0, policy_version 1130703 (0.0009) [2023-12-27 13:28:58,553][20347] Updated weights for policy 0, policy_version 1130713 (0.0010) [2023-12-27 13:28:58,889][20381] Updated weights for policy 1, policy_version 1129563 (0.0008) [2023-12-27 13:28:58,955][20381] Updated weights for policy 1, policy_version 1129573 (0.0006) [2023-12-27 13:28:59,013][20381] Updated weights for policy 1, policy_version 1129583 (0.0008) [2023-12-27 13:28:59,386][20347] Updated weights for policy 0, policy_version 1130723 (0.0007) [2023-12-27 13:28:59,450][20347] Updated weights for policy 0, policy_version 1130733 (0.0008) [2023-12-27 13:28:59,501][20347] Updated weights for policy 0, policy_version 1130743 (0.0008) [2023-12-27 13:28:59,704][20381] Updated weights for policy 1, policy_version 1129593 (0.0007) [2023-12-27 13:28:59,760][20381] Updated weights for policy 1, policy_version 1129603 (0.0007) [2023-12-27 13:28:59,815][20381] Updated weights for policy 1, policy_version 1129613 (0.0006) [2023-12-27 13:28:59,881][20381] Updated weights for policy 1, policy_version 1129623 (0.0009) [2023-12-27 13:29:00,236][20347] Updated weights for policy 0, policy_version 1130753 (0.0007) [2023-12-27 13:29:00,295][20347] Updated weights for policy 0, policy_version 1130763 (0.0007) [2023-12-27 13:29:00,352][20347] Updated weights for policy 0, policy_version 1130773 (0.0007) [2023-12-27 13:29:00,413][20347] Updated weights for policy 0, policy_version 1130783 (0.0008) [2023-12-27 13:29:00,531][20381] Updated weights for policy 1, policy_version 1129633 (0.0008) [2023-12-27 13:29:00,581][20381] Updated weights for policy 1, policy_version 1129643 (0.0007) [2023-12-27 13:29:00,639][20381] Updated weights for policy 1, policy_version 1129653 (0.0008) [2023-12-27 13:29:01,033][20347] Updated weights for policy 0, policy_version 1130793 (0.0006) [2023-12-27 13:29:01,094][20347] Updated weights for policy 0, policy_version 1130803 (0.0007) [2023-12-27 13:29:01,156][20347] Updated weights for policy 0, policy_version 1130813 (0.0009) [2023-12-27 13:29:01,325][20381] Updated weights for policy 1, policy_version 1129663 (0.0008) [2023-12-27 13:29:01,382][20381] Updated weights for policy 1, policy_version 1129673 (0.0008) [2023-12-27 13:29:01,440][20381] Updated weights for policy 1, policy_version 1129683 (0.0008) [2023-12-27 13:29:01,790][20347] Updated weights for policy 0, policy_version 1130823 (0.0007) [2023-12-27 13:29:01,852][20347] Updated weights for policy 0, policy_version 1130833 (0.0007) [2023-12-27 13:29:01,917][20347] Updated weights for policy 0, policy_version 1130843 (0.0008) [2023-12-27 13:29:02,196][20381] Updated weights for policy 1, policy_version 1129693 (0.0009) [2023-12-27 13:29:02,249][20381] Updated weights for policy 1, policy_version 1129703 (0.0010) [2023-12-27 13:29:02,315][20381] Updated weights for policy 1, policy_version 1129713 (0.0009) [2023-12-27 13:29:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 578781184. Throughput: 0: 10292.3, 1: 10179.2. Samples: 578787808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:02,363][19170] Avg episode reward: [(0, '941.500'), (1, '755.010')] [2023-12-27 13:29:02,634][20347] Updated weights for policy 0, policy_version 1130853 (0.0007) [2023-12-27 13:29:02,696][20347] Updated weights for policy 0, policy_version 1130863 (0.0008) [2023-12-27 13:29:02,753][20347] Updated weights for policy 0, policy_version 1130873 (0.0008) [2023-12-27 13:29:02,972][20381] Updated weights for policy 1, policy_version 1129723 (0.0007) [2023-12-27 13:29:03,016][20381] Updated weights for policy 1, policy_version 1129733 (0.0007) [2023-12-27 13:29:03,078][20381] Updated weights for policy 1, policy_version 1129743 (0.0007) [2023-12-27 13:29:03,395][20347] Updated weights for policy 0, policy_version 1130883 (0.0008) [2023-12-27 13:29:03,450][20347] Updated weights for policy 0, policy_version 1130893 (0.0007) [2023-12-27 13:29:03,504][20347] Updated weights for policy 0, policy_version 1130903 (0.0005) [2023-12-27 13:29:03,758][20381] Updated weights for policy 1, policy_version 1129753 (0.0008) [2023-12-27 13:29:03,822][20381] Updated weights for policy 1, policy_version 1129763 (0.0006) [2023-12-27 13:29:03,893][20381] Updated weights for policy 1, policy_version 1129773 (0.0007) [2023-12-27 13:29:03,959][20381] Updated weights for policy 1, policy_version 1129783 (0.0007) [2023-12-27 13:29:04,165][20347] Updated weights for policy 0, policy_version 1130913 (0.0006) [2023-12-27 13:29:04,223][20347] Updated weights for policy 0, policy_version 1130923 (0.0007) [2023-12-27 13:29:04,283][20347] Updated weights for policy 0, policy_version 1130933 (0.0008) [2023-12-27 13:29:04,345][20347] Updated weights for policy 0, policy_version 1130943 (0.0008) [2023-12-27 13:29:04,621][20381] Updated weights for policy 1, policy_version 1129793 (0.0008) [2023-12-27 13:29:04,682][20381] Updated weights for policy 1, policy_version 1129803 (0.0007) [2023-12-27 13:29:04,734][20381] Updated weights for policy 1, policy_version 1129813 (0.0008) [2023-12-27 13:29:05,058][20347] Updated weights for policy 0, policy_version 1130953 (0.0008) [2023-12-27 13:29:05,129][20347] Updated weights for policy 0, policy_version 1130963 (0.0006) [2023-12-27 13:29:05,194][20347] Updated weights for policy 0, policy_version 1130973 (0.0008) [2023-12-27 13:29:05,462][20381] Updated weights for policy 1, policy_version 1129823 (0.0009) [2023-12-27 13:29:05,516][20381] Updated weights for policy 1, policy_version 1129833 (0.0008) [2023-12-27 13:29:05,569][20381] Updated weights for policy 1, policy_version 1129843 (0.0007) [2023-12-27 13:29:05,804][20347] Updated weights for policy 0, policy_version 1130983 (0.0008) [2023-12-27 13:29:05,869][20347] Updated weights for policy 0, policy_version 1130993 (0.0005) [2023-12-27 13:29:05,931][20347] Updated weights for policy 0, policy_version 1131003 (0.0006) [2023-12-27 13:29:06,297][20381] Updated weights for policy 1, policy_version 1129853 (0.0007) [2023-12-27 13:29:06,348][20381] Updated weights for policy 1, policy_version 1129863 (0.0007) [2023-12-27 13:29:06,399][20381] Updated weights for policy 1, policy_version 1129873 (0.0007) [2023-12-27 13:29:06,604][20347] Updated weights for policy 0, policy_version 1131013 (0.0007) [2023-12-27 13:29:06,663][20347] Updated weights for policy 0, policy_version 1131023 (0.0008) [2023-12-27 13:29:06,714][20347] Updated weights for policy 0, policy_version 1131033 (0.0008) [2023-12-27 13:29:07,143][20381] Updated weights for policy 1, policy_version 1129883 (0.0008) [2023-12-27 13:29:07,204][20381] Updated weights for policy 1, policy_version 1129893 (0.0009) [2023-12-27 13:29:07,262][20381] Updated weights for policy 1, policy_version 1129903 (0.0008) [2023-12-27 13:29:07,352][20347] Updated weights for policy 0, policy_version 1131043 (0.0009) [2023-12-27 13:29:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 578879488. Throughput: 0: 10308.9, 1: 10177.2. Samples: 578848868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:07,363][19170] Avg episode reward: [(0, '951.270'), (1, '725.580')] [2023-12-27 13:29:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001129912_289284096.pth... [2023-12-27 13:29:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001128728_288980992.pth [2023-12-27 13:29:07,412][20347] Updated weights for policy 0, policy_version 1131053 (0.0006) [2023-12-27 13:29:07,472][20347] Updated weights for policy 0, policy_version 1131063 (0.0010) [2023-12-27 13:29:07,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001131072_289603584.pth... [2023-12-27 13:29:07,526][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001129856_289292288.pth [2023-12-27 13:29:07,957][20381] Updated weights for policy 1, policy_version 1129913 (0.0008) [2023-12-27 13:29:08,009][20381] Updated weights for policy 1, policy_version 1129923 (0.0006) [2023-12-27 13:29:08,068][20381] Updated weights for policy 1, policy_version 1129933 (0.0008) [2023-12-27 13:29:08,098][20347] Updated weights for policy 0, policy_version 1131073 (0.0010) [2023-12-27 13:29:08,120][20381] Updated weights for policy 1, policy_version 1129943 (0.0009) [2023-12-27 13:29:08,155][20347] Updated weights for policy 0, policy_version 1131083 (0.0005) [2023-12-27 13:29:08,210][20347] Updated weights for policy 0, policy_version 1131093 (0.0005) [2023-12-27 13:29:08,267][20347] Updated weights for policy 0, policy_version 1131103 (0.0005) [2023-12-27 13:29:08,818][20381] Updated weights for policy 1, policy_version 1129953 (0.0008) [2023-12-27 13:29:08,869][20381] Updated weights for policy 1, policy_version 1129963 (0.0007) [2023-12-27 13:29:08,926][20347] Updated weights for policy 0, policy_version 1131113 (0.0007) [2023-12-27 13:29:08,926][20381] Updated weights for policy 1, policy_version 1129973 (0.0007) [2023-12-27 13:29:08,985][20347] Updated weights for policy 0, policy_version 1131123 (0.0007) [2023-12-27 13:29:09,047][20347] Updated weights for policy 0, policy_version 1131133 (0.0007) [2023-12-27 13:29:09,712][20381] Updated weights for policy 1, policy_version 1129983 (0.0008) [2023-12-27 13:29:09,719][20347] Updated weights for policy 0, policy_version 1131143 (0.0007) [2023-12-27 13:29:09,769][20381] Updated weights for policy 1, policy_version 1129993 (0.0008) [2023-12-27 13:29:09,783][20347] Updated weights for policy 0, policy_version 1131153 (0.0008) [2023-12-27 13:29:09,826][20381] Updated weights for policy 1, policy_version 1130003 (0.0006) [2023-12-27 13:29:09,840][20347] Updated weights for policy 0, policy_version 1131163 (0.0007) [2023-12-27 13:29:10,503][20381] Updated weights for policy 1, policy_version 1130013 (0.0007) [2023-12-27 13:29:10,515][20347] Updated weights for policy 0, policy_version 1131173 (0.0010) [2023-12-27 13:29:10,556][20381] Updated weights for policy 1, policy_version 1130023 (0.0009) [2023-12-27 13:29:10,567][20347] Updated weights for policy 0, policy_version 1131183 (0.0010) [2023-12-27 13:29:10,609][20381] Updated weights for policy 1, policy_version 1130033 (0.0007) [2023-12-27 13:29:10,620][20347] Updated weights for policy 0, policy_version 1131193 (0.0005) [2023-12-27 13:29:11,292][20381] Updated weights for policy 1, policy_version 1130043 (0.0008) [2023-12-27 13:29:11,322][20347] Updated weights for policy 0, policy_version 1131203 (0.0006) [2023-12-27 13:29:11,357][20381] Updated weights for policy 1, policy_version 1130053 (0.0007) [2023-12-27 13:29:11,382][20347] Updated weights for policy 0, policy_version 1131213 (0.0008) [2023-12-27 13:29:11,423][20381] Updated weights for policy 1, policy_version 1130063 (0.0008) [2023-12-27 13:29:11,451][20347] Updated weights for policy 0, policy_version 1131223 (0.0010) [2023-12-27 13:29:12,147][20347] Updated weights for policy 0, policy_version 1131233 (0.0009) [2023-12-27 13:29:12,168][20381] Updated weights for policy 1, policy_version 1130073 (0.0008) [2023-12-27 13:29:12,205][20347] Updated weights for policy 0, policy_version 1131243 (0.0007) [2023-12-27 13:29:12,231][20381] Updated weights for policy 1, policy_version 1130083 (0.0010) [2023-12-27 13:29:12,265][20347] Updated weights for policy 0, policy_version 1131253 (0.0007) [2023-12-27 13:29:12,295][20381] Updated weights for policy 1, policy_version 1130093 (0.0009) [2023-12-27 13:29:12,325][20347] Updated weights for policy 0, policy_version 1131263 (0.0008) [2023-12-27 13:29:12,358][20381] Updated weights for policy 1, policy_version 1130103 (0.0009) [2023-12-27 13:29:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 578977792. Throughput: 0: 10303.4, 1: 10140.4. Samples: 578970128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:12,363][19170] Avg episode reward: [(0, '882.880'), (1, '686.640')] [2023-12-27 13:29:13,024][20381] Updated weights for policy 1, policy_version 1130113 (0.0006) [2023-12-27 13:29:13,061][20347] Updated weights for policy 0, policy_version 1131273 (0.0007) [2023-12-27 13:29:13,080][20381] Updated weights for policy 1, policy_version 1130123 (0.0005) [2023-12-27 13:29:13,118][20347] Updated weights for policy 0, policy_version 1131283 (0.0008) [2023-12-27 13:29:13,138][20381] Updated weights for policy 1, policy_version 1130133 (0.0005) [2023-12-27 13:29:13,178][20347] Updated weights for policy 0, policy_version 1131293 (0.0008) [2023-12-27 13:29:13,665][20381] Updated weights for policy 1, policy_version 1130143 (0.0009) [2023-12-27 13:29:13,722][20381] Updated weights for policy 1, policy_version 1130153 (0.0010) [2023-12-27 13:29:13,769][20381] Updated weights for policy 1, policy_version 1130163 (0.0006) [2023-12-27 13:29:13,889][20347] Updated weights for policy 0, policy_version 1131303 (0.0010) [2023-12-27 13:29:13,938][20347] Updated weights for policy 0, policy_version 1131313 (0.0009) [2023-12-27 13:29:13,997][20347] Updated weights for policy 0, policy_version 1131323 (0.0007) [2023-12-27 13:29:14,368][20381] Updated weights for policy 1, policy_version 1130173 (0.0008) [2023-12-27 13:29:14,422][20381] Updated weights for policy 1, policy_version 1130183 (0.0010) [2023-12-27 13:29:14,481][20381] Updated weights for policy 1, policy_version 1130193 (0.0010) [2023-12-27 13:29:14,705][20347] Updated weights for policy 0, policy_version 1131333 (0.0007) [2023-12-27 13:29:14,767][20347] Updated weights for policy 0, policy_version 1131343 (0.0008) [2023-12-27 13:29:14,829][20347] Updated weights for policy 0, policy_version 1131353 (0.0008) [2023-12-27 13:29:15,098][20381] Updated weights for policy 1, policy_version 1130203 (0.0011) [2023-12-27 13:29:15,155][20381] Updated weights for policy 1, policy_version 1130213 (0.0011) [2023-12-27 13:29:15,209][20381] Updated weights for policy 1, policy_version 1130223 (0.0011) [2023-12-27 13:29:15,573][20347] Updated weights for policy 0, policy_version 1131363 (0.0008) [2023-12-27 13:29:15,617][20347] Updated weights for policy 0, policy_version 1131373 (0.0007) [2023-12-27 13:29:15,672][20347] Updated weights for policy 0, policy_version 1131383 (0.0006) [2023-12-27 13:29:15,895][20381] Updated weights for policy 1, policy_version 1130233 (0.0010) [2023-12-27 13:29:15,958][20381] Updated weights for policy 1, policy_version 1130243 (0.0005) [2023-12-27 13:29:16,020][20381] Updated weights for policy 1, policy_version 1130253 (0.0005) [2023-12-27 13:29:16,080][20381] Updated weights for policy 1, policy_version 1130263 (0.0005) [2023-12-27 13:29:16,318][20347] Updated weights for policy 0, policy_version 1131393 (0.0006) [2023-12-27 13:29:16,372][20347] Updated weights for policy 0, policy_version 1131403 (0.0010) [2023-12-27 13:29:16,435][20347] Updated weights for policy 0, policy_version 1131413 (0.0010) [2023-12-27 13:29:16,486][20347] Updated weights for policy 0, policy_version 1131423 (0.0010) [2023-12-27 13:29:16,614][20381] Updated weights for policy 1, policy_version 1130273 (0.0010) [2023-12-27 13:29:16,675][20381] Updated weights for policy 1, policy_version 1130283 (0.0010) [2023-12-27 13:29:16,728][20381] Updated weights for policy 1, policy_version 1130293 (0.0010) [2023-12-27 13:29:17,126][20347] Updated weights for policy 0, policy_version 1131433 (0.0008) [2023-12-27 13:29:17,183][20347] Updated weights for policy 0, policy_version 1131443 (0.0005) [2023-12-27 13:29:17,237][20347] Updated weights for policy 0, policy_version 1131453 (0.0006) [2023-12-27 13:29:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 579084288. Throughput: 0: 10163.2, 1: 10212.5. Samples: 579094136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:17,363][19170] Avg episode reward: [(0, '921.950'), (1, '608.170')] [2023-12-27 13:29:17,364][20381] Updated weights for policy 1, policy_version 1130303 (0.0010) [2023-12-27 13:29:17,427][20381] Updated weights for policy 1, policy_version 1130313 (0.0008) [2023-12-27 13:29:17,484][20381] Updated weights for policy 1, policy_version 1130323 (0.0008) [2023-12-27 13:29:17,783][20347] Updated weights for policy 0, policy_version 1131463 (0.0005) [2023-12-27 13:29:17,845][20347] Updated weights for policy 0, policy_version 1131473 (0.0010) [2023-12-27 13:29:17,897][20347] Updated weights for policy 0, policy_version 1131483 (0.0007) [2023-12-27 13:29:18,246][20381] Updated weights for policy 1, policy_version 1130333 (0.0006) [2023-12-27 13:29:18,300][20381] Updated weights for policy 1, policy_version 1130343 (0.0006) [2023-12-27 13:29:18,359][20381] Updated weights for policy 1, policy_version 1130353 (0.0009) [2023-12-27 13:29:18,477][20347] Updated weights for policy 0, policy_version 1131493 (0.0007) [2023-12-27 13:29:18,528][20347] Updated weights for policy 0, policy_version 1131503 (0.0008) [2023-12-27 13:29:18,582][20347] Updated weights for policy 0, policy_version 1131513 (0.0008) [2023-12-27 13:29:19,005][20381] Updated weights for policy 1, policy_version 1130363 (0.0007) [2023-12-27 13:29:19,053][20381] Updated weights for policy 1, policy_version 1130373 (0.0008) [2023-12-27 13:29:19,102][20381] Updated weights for policy 1, policy_version 1130383 (0.0007) [2023-12-27 13:29:19,311][20347] Updated weights for policy 0, policy_version 1131523 (0.0007) [2023-12-27 13:29:19,382][20347] Updated weights for policy 0, policy_version 1131533 (0.0007) [2023-12-27 13:29:19,446][20347] Updated weights for policy 0, policy_version 1131543 (0.0007) [2023-12-27 13:29:19,890][20381] Updated weights for policy 1, policy_version 1130393 (0.0008) [2023-12-27 13:29:19,959][20381] Updated weights for policy 1, policy_version 1130403 (0.0008) [2023-12-27 13:29:20,017][20381] Updated weights for policy 1, policy_version 1130413 (0.0008) [2023-12-27 13:29:20,079][20381] Updated weights for policy 1, policy_version 1130423 (0.0008) [2023-12-27 13:29:20,161][20347] Updated weights for policy 0, policy_version 1131553 (0.0008) [2023-12-27 13:29:20,222][20347] Updated weights for policy 0, policy_version 1131563 (0.0008) [2023-12-27 13:29:20,279][20347] Updated weights for policy 0, policy_version 1131573 (0.0009) [2023-12-27 13:29:20,340][20347] Updated weights for policy 0, policy_version 1131583 (0.0007) [2023-12-27 13:29:20,795][20381] Updated weights for policy 1, policy_version 1130433 (0.0008) [2023-12-27 13:29:20,854][20381] Updated weights for policy 1, policy_version 1130443 (0.0008) [2023-12-27 13:29:20,911][20381] Updated weights for policy 1, policy_version 1130453 (0.0008) [2023-12-27 13:29:21,055][20347] Updated weights for policy 0, policy_version 1131593 (0.0008) [2023-12-27 13:29:21,111][20347] Updated weights for policy 0, policy_version 1131603 (0.0007) [2023-12-27 13:29:21,177][20347] Updated weights for policy 0, policy_version 1131613 (0.0008) [2023-12-27 13:29:21,707][20381] Updated weights for policy 1, policy_version 1130463 (0.0008) [2023-12-27 13:29:21,783][20381] Updated weights for policy 1, policy_version 1130473 (0.0008) [2023-12-27 13:29:21,844][20381] Updated weights for policy 1, policy_version 1130483 (0.0008) [2023-12-27 13:29:21,930][20347] Updated weights for policy 0, policy_version 1131623 (0.0007) [2023-12-27 13:29:21,983][20347] Updated weights for policy 0, policy_version 1131633 (0.0008) [2023-12-27 13:29:22,044][20347] Updated weights for policy 0, policy_version 1131643 (0.0008) [2023-12-27 13:29:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 579182592. Throughput: 0: 10194.5, 1: 10177.0. Samples: 579154680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:22,363][19170] Avg episode reward: [(0, '951.090'), (1, '608.030')] [2023-12-27 13:29:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001130488_289431552.pth... [2023-12-27 13:29:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001131648_289751040.pth... [2023-12-27 13:29:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001130464_289447936.pth [2023-12-27 13:29:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001129304_289128448.pth [2023-12-27 13:29:22,557][20381] Updated weights for policy 1, policy_version 1130493 (0.0006) [2023-12-27 13:29:22,612][20381] Updated weights for policy 1, policy_version 1130503 (0.0006) [2023-12-27 13:29:22,670][20381] Updated weights for policy 1, policy_version 1130513 (0.0008) [2023-12-27 13:29:22,769][20347] Updated weights for policy 0, policy_version 1131653 (0.0008) [2023-12-27 13:29:22,826][20347] Updated weights for policy 0, policy_version 1131663 (0.0007) [2023-12-27 13:29:22,882][20347] Updated weights for policy 0, policy_version 1131673 (0.0007) [2023-12-27 13:29:23,324][20381] Updated weights for policy 1, policy_version 1130523 (0.0008) [2023-12-27 13:29:23,373][20381] Updated weights for policy 1, policy_version 1130533 (0.0008) [2023-12-27 13:29:23,428][20381] Updated weights for policy 1, policy_version 1130543 (0.0008) [2023-12-27 13:29:23,600][20347] Updated weights for policy 0, policy_version 1131683 (0.0007) [2023-12-27 13:29:23,649][20347] Updated weights for policy 0, policy_version 1131693 (0.0008) [2023-12-27 13:29:23,695][20347] Updated weights for policy 0, policy_version 1131703 (0.0008) [2023-12-27 13:29:24,165][20381] Updated weights for policy 1, policy_version 1130553 (0.0007) [2023-12-27 13:29:24,218][20381] Updated weights for policy 1, policy_version 1130563 (0.0009) [2023-12-27 13:29:24,264][20381] Updated weights for policy 1, policy_version 1130573 (0.0008) [2023-12-27 13:29:24,299][20347] Updated weights for policy 0, policy_version 1131713 (0.0007) [2023-12-27 13:29:24,310][20381] Updated weights for policy 1, policy_version 1130583 (0.0007) [2023-12-27 13:29:24,348][20347] Updated weights for policy 0, policy_version 1131723 (0.0007) [2023-12-27 13:29:24,405][20347] Updated weights for policy 0, policy_version 1131733 (0.0008) [2023-12-27 13:29:24,460][20347] Updated weights for policy 0, policy_version 1131743 (0.0008) [2023-12-27 13:29:24,979][20381] Updated weights for policy 1, policy_version 1130593 (0.0006) [2023-12-27 13:29:25,037][20381] Updated weights for policy 1, policy_version 1130603 (0.0007) [2023-12-27 13:29:25,093][20381] Updated weights for policy 1, policy_version 1130613 (0.0007) [2023-12-27 13:29:25,183][20347] Updated weights for policy 0, policy_version 1131753 (0.0007) [2023-12-27 13:29:25,240][20347] Updated weights for policy 0, policy_version 1131763 (0.0006) [2023-12-27 13:29:25,301][20347] Updated weights for policy 0, policy_version 1131773 (0.0006) [2023-12-27 13:29:25,719][20381] Updated weights for policy 1, policy_version 1130623 (0.0007) [2023-12-27 13:29:25,774][20381] Updated weights for policy 1, policy_version 1130633 (0.0007) [2023-12-27 13:29:25,827][20381] Updated weights for policy 1, policy_version 1130643 (0.0007) [2023-12-27 13:29:25,968][20347] Updated weights for policy 0, policy_version 1131783 (0.0010) [2023-12-27 13:29:26,013][20347] Updated weights for policy 0, policy_version 1131793 (0.0010) [2023-12-27 13:29:26,063][20347] Updated weights for policy 0, policy_version 1131803 (0.0010) [2023-12-27 13:29:26,567][20381] Updated weights for policy 1, policy_version 1130653 (0.0007) [2023-12-27 13:29:26,619][20381] Updated weights for policy 1, policy_version 1130663 (0.0008) [2023-12-27 13:29:26,648][20347] Updated weights for policy 0, policy_version 1131813 (0.0008) [2023-12-27 13:29:26,671][20381] Updated weights for policy 1, policy_version 1130673 (0.0007) [2023-12-27 13:29:26,696][20347] Updated weights for policy 0, policy_version 1131823 (0.0005) [2023-12-27 13:29:26,745][20347] Updated weights for policy 0, policy_version 1131833 (0.0005) [2023-12-27 13:29:27,353][20347] Updated weights for policy 0, policy_version 1131843 (0.0005) [2023-12-27 13:29:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 579280896. Throughput: 0: 10156.1, 1: 10128.0. Samples: 579273928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:27,363][19170] Avg episode reward: [(0, '941.750'), (1, '735.590')] [2023-12-27 13:29:27,401][20347] Updated weights for policy 0, policy_version 1131853 (0.0005) [2023-12-27 13:29:27,426][20381] Updated weights for policy 1, policy_version 1130683 (0.0007) [2023-12-27 13:29:27,462][20347] Updated weights for policy 0, policy_version 1131863 (0.0007) [2023-12-27 13:29:27,482][20381] Updated weights for policy 1, policy_version 1130693 (0.0006) [2023-12-27 13:29:27,538][20381] Updated weights for policy 1, policy_version 1130703 (0.0007) [2023-12-27 13:29:28,044][20347] Updated weights for policy 0, policy_version 1131873 (0.0005) [2023-12-27 13:29:28,090][20347] Updated weights for policy 0, policy_version 1131883 (0.0005) [2023-12-27 13:29:28,147][20347] Updated weights for policy 0, policy_version 1131893 (0.0005) [2023-12-27 13:29:28,205][20347] Updated weights for policy 0, policy_version 1131903 (0.0005) [2023-12-27 13:29:28,219][20381] Updated weights for policy 1, policy_version 1130713 (0.0008) [2023-12-27 13:29:28,271][20381] Updated weights for policy 1, policy_version 1130723 (0.0007) [2023-12-27 13:29:28,325][20381] Updated weights for policy 1, policy_version 1130733 (0.0008) [2023-12-27 13:29:28,384][20381] Updated weights for policy 1, policy_version 1130743 (0.0007) [2023-12-27 13:29:28,847][20347] Updated weights for policy 0, policy_version 1131913 (0.0007) [2023-12-27 13:29:28,901][20347] Updated weights for policy 0, policy_version 1131923 (0.0008) [2023-12-27 13:29:28,957][20347] Updated weights for policy 0, policy_version 1131933 (0.0008) [2023-12-27 13:29:29,085][20381] Updated weights for policy 1, policy_version 1130753 (0.0005) [2023-12-27 13:29:29,148][20381] Updated weights for policy 1, policy_version 1130763 (0.0005) [2023-12-27 13:29:29,210][20381] Updated weights for policy 1, policy_version 1130773 (0.0005) [2023-12-27 13:29:29,687][20347] Updated weights for policy 0, policy_version 1131943 (0.0008) [2023-12-27 13:29:29,745][20347] Updated weights for policy 0, policy_version 1131953 (0.0009) [2023-12-27 13:29:29,754][20381] Updated weights for policy 1, policy_version 1130783 (0.0008) [2023-12-27 13:29:29,796][20381] Updated weights for policy 1, policy_version 1130793 (0.0006) [2023-12-27 13:29:29,801][20347] Updated weights for policy 0, policy_version 1131963 (0.0010) [2023-12-27 13:29:29,862][20381] Updated weights for policy 1, policy_version 1130803 (0.0008) [2023-12-27 13:29:30,494][20347] Updated weights for policy 0, policy_version 1131973 (0.0008) [2023-12-27 13:29:30,515][20381] Updated weights for policy 1, policy_version 1130813 (0.0007) [2023-12-27 13:29:30,546][20347] Updated weights for policy 0, policy_version 1131983 (0.0005) [2023-12-27 13:29:30,564][20381] Updated weights for policy 1, policy_version 1130823 (0.0006) [2023-12-27 13:29:30,595][20347] Updated weights for policy 0, policy_version 1131993 (0.0006) [2023-12-27 13:29:30,616][20381] Updated weights for policy 1, policy_version 1130833 (0.0006) [2023-12-27 13:29:31,228][20347] Updated weights for policy 0, policy_version 1132003 (0.0006) [2023-12-27 13:29:31,288][20347] Updated weights for policy 0, policy_version 1132013 (0.0009) [2023-12-27 13:29:31,326][20381] Updated weights for policy 1, policy_version 1130843 (0.0008) [2023-12-27 13:29:31,351][20347] Updated weights for policy 0, policy_version 1132023 (0.0011) [2023-12-27 13:29:31,398][20381] Updated weights for policy 1, policy_version 1130853 (0.0008) [2023-12-27 13:29:31,452][20381] Updated weights for policy 1, policy_version 1130863 (0.0008) [2023-12-27 13:29:32,069][20347] Updated weights for policy 0, policy_version 1132033 (0.0009) [2023-12-27 13:29:32,133][20347] Updated weights for policy 0, policy_version 1132043 (0.0008) [2023-12-27 13:29:32,187][20347] Updated weights for policy 0, policy_version 1132053 (0.0007) [2023-12-27 13:29:32,204][20381] Updated weights for policy 1, policy_version 1130873 (0.0007) [2023-12-27 13:29:32,246][20347] Updated weights for policy 0, policy_version 1132063 (0.0008) [2023-12-27 13:29:32,260][20381] Updated weights for policy 1, policy_version 1130883 (0.0007) [2023-12-27 13:29:32,309][20381] Updated weights for policy 1, policy_version 1130893 (0.0007) [2023-12-27 13:29:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 579387392. Throughput: 0: 10302.9, 1: 10022.0. Samples: 579398984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:29:32,363][19170] Avg episode reward: [(0, '932.150'), (1, '696.400')] [2023-12-27 13:29:32,375][20381] Updated weights for policy 1, policy_version 1130903 (0.0009) [2023-12-27 13:29:32,923][20347] Updated weights for policy 0, policy_version 1132073 (0.0006) [2023-12-27 13:29:32,929][20291] KL-divergence is very high: 524.8282 [2023-12-27 13:29:32,973][20291] KL-divergence is very high: 266.7714 [2023-12-27 13:29:32,978][20347] Updated weights for policy 0, policy_version 1132083 (0.0005) [2023-12-27 13:29:33,029][20347] Updated weights for policy 0, policy_version 1132093 (0.0007) [2023-12-27 13:29:33,059][20381] Updated weights for policy 1, policy_version 1130913 (0.0010) [2023-12-27 13:29:33,114][20381] Updated weights for policy 1, policy_version 1130923 (0.0011) [2023-12-27 13:29:33,171][20381] Updated weights for policy 1, policy_version 1130933 (0.0010) [2023-12-27 13:29:33,681][20347] Updated weights for policy 0, policy_version 1132103 (0.0009) [2023-12-27 13:29:33,736][20347] Updated weights for policy 0, policy_version 1132113 (0.0010) [2023-12-27 13:29:33,772][20381] Updated weights for policy 1, policy_version 1130943 (0.0009) [2023-12-27 13:29:33,809][20347] Updated weights for policy 0, policy_version 1132123 (0.0010) [2023-12-27 13:29:33,834][20381] Updated weights for policy 1, policy_version 1130953 (0.0011) [2023-12-27 13:29:33,884][20381] Updated weights for policy 1, policy_version 1130963 (0.0008) [2023-12-27 13:29:34,438][20347] Updated weights for policy 0, policy_version 1132133 (0.0010) [2023-12-27 13:29:34,476][20381] Updated weights for policy 1, policy_version 1130973 (0.0005) [2023-12-27 13:29:34,498][20347] Updated weights for policy 0, policy_version 1132143 (0.0011) [2023-12-27 13:29:34,530][20381] Updated weights for policy 1, policy_version 1130983 (0.0006) [2023-12-27 13:29:34,555][20347] Updated weights for policy 0, policy_version 1132153 (0.0010) [2023-12-27 13:29:34,592][20381] Updated weights for policy 1, policy_version 1130993 (0.0010) [2023-12-27 13:29:35,125][20347] Updated weights for policy 0, policy_version 1132163 (0.0009) [2023-12-27 13:29:35,180][20347] Updated weights for policy 0, policy_version 1132173 (0.0010) [2023-12-27 13:29:35,231][20347] Updated weights for policy 0, policy_version 1132183 (0.0010) [2023-12-27 13:29:35,297][20381] Updated weights for policy 1, policy_version 1131003 (0.0008) [2023-12-27 13:29:35,344][20381] Updated weights for policy 1, policy_version 1131013 (0.0007) [2023-12-27 13:29:35,398][20381] Updated weights for policy 1, policy_version 1131023 (0.0007) [2023-12-27 13:29:35,912][20347] Updated weights for policy 0, policy_version 1132193 (0.0010) [2023-12-27 13:29:35,972][20347] Updated weights for policy 0, policy_version 1132203 (0.0010) [2023-12-27 13:29:35,997][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-27 13:29:36,035][20381] Updated weights for policy 1, policy_version 1131033 (0.0007) [2023-12-27 13:29:36,085][20381] Updated weights for policy 1, policy_version 1131043 (0.0008) [2023-12-27 13:29:36,149][20381] Updated weights for policy 1, policy_version 1131053 (0.0007) [2023-12-27 13:29:36,211][20381] Updated weights for policy 1, policy_version 1131063 (0.0007) [2023-12-27 13:29:36,827][20347] Updated weights for policy 0, policy_version 1132213 (0.0010) [2023-12-27 13:29:36,846][20381] Updated weights for policy 1, policy_version 1131073 (0.0010) [2023-12-27 13:29:36,880][20347] Updated weights for policy 0, policy_version 1132223 (0.0010) [2023-12-27 13:29:36,898][20381] Updated weights for policy 1, policy_version 1131083 (0.0008) [2023-12-27 13:29:36,941][20347] Updated weights for policy 0, policy_version 1132233 (0.0010) [2023-12-27 13:29:36,950][20381] Updated weights for policy 1, policy_version 1131093 (0.0005) [2023-12-27 13:29:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 579493888. Throughput: 0: 10330.6, 1: 10095.3. Samples: 579463092. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:29:37,363][19170] Avg episode reward: [(0, '951.350'), (1, '715.810')] [2023-12-27 13:29:37,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001131096_289587200.pth... [2023-12-27 13:29:37,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001132240_289906688.pth... [2023-12-27 13:29:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001131072_289603584.pth [2023-12-27 13:29:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001129912_289284096.pth [2023-12-27 13:29:37,615][20347] Updated weights for policy 0, policy_version 1132243 (0.0009) [2023-12-27 13:29:37,634][20381] Updated weights for policy 1, policy_version 1131103 (0.0006) [2023-12-27 13:29:37,675][20347] Updated weights for policy 0, policy_version 1132253 (0.0011) [2023-12-27 13:29:37,694][20381] Updated weights for policy 1, policy_version 1131113 (0.0006) [2023-12-27 13:29:37,739][20347] Updated weights for policy 0, policy_version 1132263 (0.0011) [2023-12-27 13:29:37,756][20381] Updated weights for policy 1, policy_version 1131123 (0.0009) [2023-12-27 13:29:38,366][20347] Updated weights for policy 0, policy_version 1132273 (0.0011) [2023-12-27 13:29:38,417][20347] Updated weights for policy 0, policy_version 1132283 (0.0009) [2023-12-27 13:29:38,465][20347] Updated weights for policy 0, policy_version 1132293 (0.0008) [2023-12-27 13:29:38,483][20381] Updated weights for policy 1, policy_version 1131133 (0.0008) [2023-12-27 13:29:38,513][20347] Updated weights for policy 0, policy_version 1132303 (0.0008) [2023-12-27 13:29:38,530][20381] Updated weights for policy 1, policy_version 1131143 (0.0011) [2023-12-27 13:29:38,578][20381] Updated weights for policy 1, policy_version 1131153 (0.0010) [2023-12-27 13:29:39,207][20347] Updated weights for policy 0, policy_version 1132313 (0.0007) [2023-12-27 13:29:39,271][20347] Updated weights for policy 0, policy_version 1132323 (0.0008) [2023-12-27 13:29:39,322][20381] Updated weights for policy 1, policy_version 1131163 (0.0010) [2023-12-27 13:29:39,327][20347] Updated weights for policy 0, policy_version 1132333 (0.0006) [2023-12-27 13:29:39,386][20381] Updated weights for policy 1, policy_version 1131173 (0.0008) [2023-12-27 13:29:39,454][20381] Updated weights for policy 1, policy_version 1131183 (0.0010) [2023-12-27 13:29:40,034][20347] Updated weights for policy 0, policy_version 1132343 (0.0009) [2023-12-27 13:29:40,096][20347] Updated weights for policy 0, policy_version 1132353 (0.0010) [2023-12-27 13:29:40,159][20347] Updated weights for policy 0, policy_version 1132363 (0.0009) [2023-12-27 13:29:40,182][20381] Updated weights for policy 1, policy_version 1131193 (0.0011) [2023-12-27 13:29:40,243][20381] Updated weights for policy 1, policy_version 1131203 (0.0011) [2023-12-27 13:29:40,307][20381] Updated weights for policy 1, policy_version 1131213 (0.0011) [2023-12-27 13:29:40,369][20381] Updated weights for policy 1, policy_version 1131223 (0.0010) [2023-12-27 13:29:40,792][20347] Updated weights for policy 0, policy_version 1132373 (0.0008) [2023-12-27 13:29:40,854][20347] Updated weights for policy 0, policy_version 1132383 (0.0010) [2023-12-27 13:29:40,912][20347] Updated weights for policy 0, policy_version 1132393 (0.0010) [2023-12-27 13:29:40,957][20381] Updated weights for policy 1, policy_version 1131233 (0.0008) [2023-12-27 13:29:41,017][20381] Updated weights for policy 1, policy_version 1131243 (0.0008) [2023-12-27 13:29:41,076][20381] Updated weights for policy 1, policy_version 1131253 (0.0009) [2023-12-27 13:29:41,694][20347] Updated weights for policy 0, policy_version 1132403 (0.0010) [2023-12-27 13:29:41,763][20347] Updated weights for policy 0, policy_version 1132413 (0.0007) [2023-12-27 13:29:41,784][20381] Updated weights for policy 1, policy_version 1131263 (0.0007) [2023-12-27 13:29:41,829][20347] Updated weights for policy 0, policy_version 1132423 (0.0007) [2023-12-27 13:29:41,843][20381] Updated weights for policy 1, policy_version 1131273 (0.0007) [2023-12-27 13:29:41,901][20381] Updated weights for policy 1, policy_version 1131283 (0.0009) [2023-12-27 13:29:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 579592192. Throughput: 0: 10290.1, 1: 10095.5. Samples: 579584928. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:29:42,363][19170] Avg episode reward: [(0, '990.290'), (1, '716.250')] [2023-12-27 13:29:42,525][20347] Updated weights for policy 0, policy_version 1132433 (0.0007) [2023-12-27 13:29:42,584][20347] Updated weights for policy 0, policy_version 1132443 (0.0008) [2023-12-27 13:29:42,621][20381] Updated weights for policy 1, policy_version 1131293 (0.0007) [2023-12-27 13:29:42,646][20347] Updated weights for policy 0, policy_version 1132453 (0.0008) [2023-12-27 13:29:42,670][20381] Updated weights for policy 1, policy_version 1131303 (0.0006) [2023-12-27 13:29:42,711][20347] Updated weights for policy 0, policy_version 1132463 (0.0007) [2023-12-27 13:29:42,731][20381] Updated weights for policy 1, policy_version 1131313 (0.0006) [2023-12-27 13:29:43,349][20381] Updated weights for policy 1, policy_version 1131323 (0.0006) [2023-12-27 13:29:43,350][20347] Updated weights for policy 0, policy_version 1132473 (0.0010) [2023-12-27 13:29:43,397][20381] Updated weights for policy 1, policy_version 1131333 (0.0010) [2023-12-27 13:29:43,401][20347] Updated weights for policy 0, policy_version 1132483 (0.0010) [2023-12-27 13:29:43,445][20381] Updated weights for policy 1, policy_version 1131343 (0.0010) [2023-12-27 13:29:43,455][20347] Updated weights for policy 0, policy_version 1132493 (0.0010) [2023-12-27 13:29:44,030][20381] Updated weights for policy 1, policy_version 1131353 (0.0010) [2023-12-27 13:29:44,062][20347] Updated weights for policy 0, policy_version 1132503 (0.0010) [2023-12-27 13:29:44,086][20381] Updated weights for policy 1, policy_version 1131363 (0.0006) [2023-12-27 13:29:44,117][20347] Updated weights for policy 0, policy_version 1132513 (0.0008) [2023-12-27 13:29:44,141][20381] Updated weights for policy 1, policy_version 1131373 (0.0006) [2023-12-27 13:29:44,171][20347] Updated weights for policy 0, policy_version 1132523 (0.0005) [2023-12-27 13:29:44,189][20381] Updated weights for policy 1, policy_version 1131383 (0.0007) [2023-12-27 13:29:44,727][20347] Updated weights for policy 0, policy_version 1132533 (0.0005) [2023-12-27 13:29:44,793][20347] Updated weights for policy 0, policy_version 1132543 (0.0008) [2023-12-27 13:29:44,848][20347] Updated weights for policy 0, policy_version 1132553 (0.0008) [2023-12-27 13:29:44,941][20381] Updated weights for policy 1, policy_version 1131393 (0.0007) [2023-12-27 13:29:44,996][20381] Updated weights for policy 1, policy_version 1131403 (0.0007) [2023-12-27 13:29:45,055][20381] Updated weights for policy 1, policy_version 1131413 (0.0008) [2023-12-27 13:29:45,514][20347] Updated weights for policy 0, policy_version 1132563 (0.0008) [2023-12-27 13:29:45,578][20347] Updated weights for policy 0, policy_version 1132573 (0.0010) [2023-12-27 13:29:45,630][20347] Updated weights for policy 0, policy_version 1132583 (0.0010) [2023-12-27 13:29:45,714][20381] Updated weights for policy 1, policy_version 1131423 (0.0006) [2023-12-27 13:29:45,772][20381] Updated weights for policy 1, policy_version 1131433 (0.0006) [2023-12-27 13:29:45,825][20381] Updated weights for policy 1, policy_version 1131443 (0.0007) [2023-12-27 13:29:46,306][20347] Updated weights for policy 0, policy_version 1132593 (0.0010) [2023-12-27 13:29:46,363][20347] Updated weights for policy 0, policy_version 1132603 (0.0010) [2023-12-27 13:29:46,421][20347] Updated weights for policy 0, policy_version 1132613 (0.0010) [2023-12-27 13:29:46,479][20347] Updated weights for policy 0, policy_version 1132623 (0.0010) [2023-12-27 13:29:46,480][20381] Updated weights for policy 1, policy_version 1131453 (0.0007) [2023-12-27 13:29:46,533][20381] Updated weights for policy 1, policy_version 1131463 (0.0005) [2023-12-27 13:29:46,588][20381] Updated weights for policy 1, policy_version 1131473 (0.0005) [2023-12-27 13:29:47,102][20347] Updated weights for policy 0, policy_version 1132633 (0.0008) [2023-12-27 13:29:47,152][20347] Updated weights for policy 0, policy_version 1132643 (0.0007) [2023-12-27 13:29:47,193][20381] Updated weights for policy 1, policy_version 1131483 (0.0007) [2023-12-27 13:29:47,207][20347] Updated weights for policy 0, policy_version 1132653 (0.0009) [2023-12-27 13:29:47,246][20381] Updated weights for policy 1, policy_version 1131493 (0.0009) [2023-12-27 13:29:47,304][20381] Updated weights for policy 1, policy_version 1131503 (0.0005) [2023-12-27 13:29:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 579706880. Throughput: 0: 10302.5, 1: 10198.7. Samples: 579710360. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:29:47,363][19170] Avg episode reward: [(0, '970.840'), (1, '617.420')] [2023-12-27 13:29:47,821][20347] Updated weights for policy 0, policy_version 1132663 (0.0009) [2023-12-27 13:29:47,844][20381] Updated weights for policy 1, policy_version 1131513 (0.0005) [2023-12-27 13:29:47,882][20347] Updated weights for policy 0, policy_version 1132673 (0.0008) [2023-12-27 13:29:47,899][20381] Updated weights for policy 1, policy_version 1131523 (0.0006) [2023-12-27 13:29:47,941][20347] Updated weights for policy 0, policy_version 1132683 (0.0008) [2023-12-27 13:29:47,948][20381] Updated weights for policy 1, policy_version 1131533 (0.0005) [2023-12-27 13:29:47,995][20381] Updated weights for policy 1, policy_version 1131543 (0.0005) [2023-12-27 13:29:48,643][20347] Updated weights for policy 0, policy_version 1132693 (0.0009) [2023-12-27 13:29:48,652][20381] Updated weights for policy 1, policy_version 1131553 (0.0010) [2023-12-27 13:29:48,690][20347] Updated weights for policy 0, policy_version 1132703 (0.0010) [2023-12-27 13:29:48,708][20381] Updated weights for policy 1, policy_version 1131563 (0.0011) [2023-12-27 13:29:48,737][20347] Updated weights for policy 0, policy_version 1132713 (0.0010) [2023-12-27 13:29:48,764][20381] Updated weights for policy 1, policy_version 1131573 (0.0011) [2023-12-27 13:29:49,463][20347] Updated weights for policy 0, policy_version 1132723 (0.0011) [2023-12-27 13:29:49,502][20381] Updated weights for policy 1, policy_version 1131583 (0.0008) [2023-12-27 13:29:49,525][20347] Updated weights for policy 0, policy_version 1132733 (0.0011) [2023-12-27 13:29:49,550][20381] Updated weights for policy 1, policy_version 1131593 (0.0007) [2023-12-27 13:29:49,584][20347] Updated weights for policy 0, policy_version 1132743 (0.0009) [2023-12-27 13:29:49,613][20381] Updated weights for policy 1, policy_version 1131603 (0.0010) [2023-12-27 13:29:50,211][20347] Updated weights for policy 0, policy_version 1132753 (0.0008) [2023-12-27 13:29:50,275][20347] Updated weights for policy 0, policy_version 1132763 (0.0005) [2023-12-27 13:29:50,301][20381] Updated weights for policy 1, policy_version 1131613 (0.0010) [2023-12-27 13:29:50,330][20347] Updated weights for policy 0, policy_version 1132773 (0.0006) [2023-12-27 13:29:50,357][20381] Updated weights for policy 1, policy_version 1131623 (0.0010) [2023-12-27 13:29:50,389][20347] Updated weights for policy 0, policy_version 1132783 (0.0006) [2023-12-27 13:29:50,411][20381] Updated weights for policy 1, policy_version 1131633 (0.0008) [2023-12-27 13:29:51,007][20347] Updated weights for policy 0, policy_version 1132793 (0.0006) [2023-12-27 13:29:51,021][20381] Updated weights for policy 1, policy_version 1131643 (0.0008) [2023-12-27 13:29:51,075][20347] Updated weights for policy 0, policy_version 1132803 (0.0009) [2023-12-27 13:29:51,082][20381] Updated weights for policy 1, policy_version 1131653 (0.0010) [2023-12-27 13:29:51,136][20347] Updated weights for policy 0, policy_version 1132813 (0.0009) [2023-12-27 13:29:51,140][20381] Updated weights for policy 1, policy_version 1131663 (0.0008) [2023-12-27 13:29:51,853][20347] Updated weights for policy 0, policy_version 1132823 (0.0008) [2023-12-27 13:29:51,889][20381] Updated weights for policy 1, policy_version 1131673 (0.0008) [2023-12-27 13:29:51,906][20347] Updated weights for policy 0, policy_version 1132833 (0.0008) [2023-12-27 13:29:51,946][20381] Updated weights for policy 1, policy_version 1131683 (0.0007) [2023-12-27 13:29:51,960][20347] Updated weights for policy 0, policy_version 1132843 (0.0011) [2023-12-27 13:29:52,001][20381] Updated weights for policy 1, policy_version 1131693 (0.0006) [2023-12-27 13:29:52,054][20381] Updated weights for policy 1, policy_version 1131703 (0.0008) [2023-12-27 13:29:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 579805184. Throughput: 0: 10309.8, 1: 10226.8. Samples: 579773012. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:29:52,363][19170] Avg episode reward: [(0, '980.490'), (1, '656.950')] [2023-12-27 13:29:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001131704_289742848.pth... [2023-12-27 13:29:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001132848_290062336.pth... [2023-12-27 13:29:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001131648_289751040.pth [2023-12-27 13:29:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001130488_289431552.pth [2023-12-27 13:29:52,663][20347] Updated weights for policy 0, policy_version 1132853 (0.0009) [2023-12-27 13:29:52,698][20381] Updated weights for policy 1, policy_version 1131713 (0.0006) [2023-12-27 13:29:52,727][20347] Updated weights for policy 0, policy_version 1132863 (0.0007) [2023-12-27 13:29:52,748][20381] Updated weights for policy 1, policy_version 1131723 (0.0006) [2023-12-27 13:29:52,784][20347] Updated weights for policy 0, policy_version 1132873 (0.0006) [2023-12-27 13:29:52,802][20381] Updated weights for policy 1, policy_version 1131733 (0.0005) [2023-12-27 13:29:53,430][20381] Updated weights for policy 1, policy_version 1131743 (0.0007) [2023-12-27 13:29:53,484][20347] Updated weights for policy 0, policy_version 1132883 (0.0007) [2023-12-27 13:29:53,487][20381] Updated weights for policy 1, policy_version 1131753 (0.0007) [2023-12-27 13:29:53,530][20347] Updated weights for policy 0, policy_version 1132893 (0.0006) [2023-12-27 13:29:53,543][20381] Updated weights for policy 1, policy_version 1131763 (0.0006) [2023-12-27 13:29:53,583][20347] Updated weights for policy 0, policy_version 1132903 (0.0006) [2023-12-27 13:29:54,222][20381] Updated weights for policy 1, policy_version 1131773 (0.0006) [2023-12-27 13:29:54,239][20347] Updated weights for policy 0, policy_version 1132913 (0.0006) [2023-12-27 13:29:54,266][20381] Updated weights for policy 1, policy_version 1131783 (0.0008) [2023-12-27 13:29:54,297][20347] Updated weights for policy 0, policy_version 1132923 (0.0008) [2023-12-27 13:29:54,314][20381] Updated weights for policy 1, policy_version 1131793 (0.0007) [2023-12-27 13:29:54,352][20347] Updated weights for policy 0, policy_version 1132933 (0.0007) [2023-12-27 13:29:54,406][20347] Updated weights for policy 0, policy_version 1132943 (0.0007) [2023-12-27 13:29:54,987][20381] Updated weights for policy 1, policy_version 1131803 (0.0007) [2023-12-27 13:29:55,043][20381] Updated weights for policy 1, policy_version 1131813 (0.0010) [2023-12-27 13:29:55,065][20347] Updated weights for policy 0, policy_version 1132953 (0.0010) [2023-12-27 13:29:55,098][20381] Updated weights for policy 1, policy_version 1131823 (0.0010) [2023-12-27 13:29:55,117][20347] Updated weights for policy 0, policy_version 1132963 (0.0010) [2023-12-27 13:29:55,175][20347] Updated weights for policy 0, policy_version 1132973 (0.0010) [2023-12-27 13:29:55,728][20381] Updated weights for policy 1, policy_version 1131833 (0.0006) [2023-12-27 13:29:55,788][20381] Updated weights for policy 1, policy_version 1131843 (0.0009) [2023-12-27 13:29:55,841][20381] Updated weights for policy 1, policy_version 1131853 (0.0005) [2023-12-27 13:29:55,860][20347] Updated weights for policy 0, policy_version 1132983 (0.0010) [2023-12-27 13:29:55,891][20381] Updated weights for policy 1, policy_version 1131863 (0.0009) [2023-12-27 13:29:55,909][20347] Updated weights for policy 0, policy_version 1132993 (0.0010) [2023-12-27 13:29:55,958][20347] Updated weights for policy 0, policy_version 1133003 (0.0010) [2023-12-27 13:29:56,517][20381] Updated weights for policy 1, policy_version 1131873 (0.0010) [2023-12-27 13:29:56,571][20381] Updated weights for policy 1, policy_version 1131883 (0.0010) [2023-12-27 13:29:56,580][20347] Updated weights for policy 0, policy_version 1133013 (0.0008) [2023-12-27 13:29:56,619][20381] Updated weights for policy 1, policy_version 1131893 (0.0007) [2023-12-27 13:29:56,623][20347] Updated weights for policy 0, policy_version 1133023 (0.0005) [2023-12-27 13:29:56,670][20347] Updated weights for policy 0, policy_version 1133033 (0.0006) [2023-12-27 13:29:57,248][20347] Updated weights for policy 0, policy_version 1133043 (0.0005) [2023-12-27 13:29:57,256][20381] Updated weights for policy 1, policy_version 1131903 (0.0007) [2023-12-27 13:29:57,299][20347] Updated weights for policy 0, policy_version 1133053 (0.0007) [2023-12-27 13:29:57,306][20381] Updated weights for policy 1, policy_version 1131913 (0.0007) [2023-12-27 13:29:57,351][20347] Updated weights for policy 0, policy_version 1133063 (0.0008) [2023-12-27 13:29:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 579903488. Throughput: 0: 10278.7, 1: 10336.9. Samples: 579897828. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:29:57,363][19170] Avg episode reward: [(0, '980.470'), (1, '559.130')] [2023-12-27 13:29:57,364][20381] Updated weights for policy 1, policy_version 1131923 (0.0007) [2023-12-27 13:29:57,951][20381] Updated weights for policy 1, policy_version 1131933 (0.0006) [2023-12-27 13:29:57,998][20347] Updated weights for policy 0, policy_version 1133073 (0.0007) [2023-12-27 13:29:58,000][20381] Updated weights for policy 1, policy_version 1131943 (0.0007) [2023-12-27 13:29:58,053][20381] Updated weights for policy 1, policy_version 1131953 (0.0006) [2023-12-27 13:29:58,058][20347] Updated weights for policy 0, policy_version 1133083 (0.0007) [2023-12-27 13:29:58,115][20347] Updated weights for policy 0, policy_version 1133093 (0.0006) [2023-12-27 13:29:58,172][20347] Updated weights for policy 0, policy_version 1133103 (0.0007) [2023-12-27 13:29:58,801][20381] Updated weights for policy 1, policy_version 1131963 (0.0008) [2023-12-27 13:29:58,865][20381] Updated weights for policy 1, policy_version 1131973 (0.0008) [2023-12-27 13:29:58,921][20381] Updated weights for policy 1, policy_version 1131983 (0.0008) [2023-12-27 13:29:58,993][20347] Updated weights for policy 0, policy_version 1133113 (0.0007) [2023-12-27 13:29:59,054][20347] Updated weights for policy 0, policy_version 1133123 (0.0008) [2023-12-27 13:29:59,118][20347] Updated weights for policy 0, policy_version 1133133 (0.0008) [2023-12-27 13:29:59,667][20381] Updated weights for policy 1, policy_version 1131993 (0.0008) [2023-12-27 13:29:59,724][20381] Updated weights for policy 1, policy_version 1132003 (0.0008) [2023-12-27 13:29:59,782][20381] Updated weights for policy 1, policy_version 1132013 (0.0008) [2023-12-27 13:29:59,843][20347] Updated weights for policy 0, policy_version 1133143 (0.0008) [2023-12-27 13:29:59,844][20381] Updated weights for policy 1, policy_version 1132023 (0.0007) [2023-12-27 13:29:59,902][20347] Updated weights for policy 0, policy_version 1133153 (0.0007) [2023-12-27 13:29:59,973][20347] Updated weights for policy 0, policy_version 1133163 (0.0007) [2023-12-27 13:30:00,517][20381] Updated weights for policy 1, policy_version 1132033 (0.0007) [2023-12-27 13:30:00,565][20381] Updated weights for policy 1, policy_version 1132043 (0.0007) [2023-12-27 13:30:00,615][20381] Updated weights for policy 1, policy_version 1132053 (0.0007) [2023-12-27 13:30:00,630][20347] Updated weights for policy 0, policy_version 1133173 (0.0007) [2023-12-27 13:30:00,680][20347] Updated weights for policy 0, policy_version 1133183 (0.0007) [2023-12-27 13:30:00,735][20347] Updated weights for policy 0, policy_version 1133193 (0.0007) [2023-12-27 13:30:01,370][20381] Updated weights for policy 1, policy_version 1132063 (0.0008) [2023-12-27 13:30:01,422][20381] Updated weights for policy 1, policy_version 1132073 (0.0008) [2023-12-27 13:30:01,475][20381] Updated weights for policy 1, policy_version 1132083 (0.0007) [2023-12-27 13:30:01,475][20347] Updated weights for policy 0, policy_version 1133203 (0.0007) [2023-12-27 13:30:01,537][20347] Updated weights for policy 0, policy_version 1133213 (0.0008) [2023-12-27 13:30:01,597][20347] Updated weights for policy 0, policy_version 1133223 (0.0008) [2023-12-27 13:30:02,114][20381] Updated weights for policy 1, policy_version 1132093 (0.0008) [2023-12-27 13:30:02,176][20381] Updated weights for policy 1, policy_version 1132103 (0.0011) [2023-12-27 13:30:02,238][20381] Updated weights for policy 1, policy_version 1132113 (0.0010) [2023-12-27 13:30:02,341][20347] Updated weights for policy 0, policy_version 1133233 (0.0008) [2023-12-27 13:30:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 580009984. Throughput: 0: 10310.5, 1: 10243.2. Samples: 580019056. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:02,363][19170] Avg episode reward: [(0, '980.470'), (1, '686.280')] [2023-12-27 13:30:02,408][20347] Updated weights for policy 0, policy_version 1133244 (0.0011) [2023-12-27 13:30:02,459][20347] Updated weights for policy 0, policy_version 1133254 (0.0010) [2023-12-27 13:30:02,521][20347] Updated weights for policy 0, policy_version 1133264 (0.0006) [2023-12-27 13:30:02,876][20381] Updated weights for policy 1, policy_version 1132123 (0.0010) [2023-12-27 13:30:02,920][20381] Updated weights for policy 1, policy_version 1132133 (0.0010) [2023-12-27 13:30:02,975][20381] Updated weights for policy 1, policy_version 1132143 (0.0010) [2023-12-27 13:30:03,192][20347] Updated weights for policy 0, policy_version 1133274 (0.0010) [2023-12-27 13:30:03,243][20347] Updated weights for policy 0, policy_version 1133284 (0.0010) [2023-12-27 13:30:03,287][20347] Updated weights for policy 0, policy_version 1133294 (0.0010) [2023-12-27 13:30:03,572][20381] Updated weights for policy 1, policy_version 1132153 (0.0010) [2023-12-27 13:30:03,632][20381] Updated weights for policy 1, policy_version 1132163 (0.0006) [2023-12-27 13:30:03,682][20381] Updated weights for policy 1, policy_version 1132173 (0.0005) [2023-12-27 13:30:03,734][20381] Updated weights for policy 1, policy_version 1132183 (0.0005) [2023-12-27 13:30:03,983][20347] Updated weights for policy 0, policy_version 1133304 (0.0008) [2023-12-27 13:30:04,043][20347] Updated weights for policy 0, policy_version 1133314 (0.0008) [2023-12-27 13:30:04,100][20347] Updated weights for policy 0, policy_version 1133324 (0.0007) [2023-12-27 13:30:04,347][20381] Updated weights for policy 1, policy_version 1132193 (0.0010) [2023-12-27 13:30:04,407][20381] Updated weights for policy 1, policy_version 1132203 (0.0011) [2023-12-27 13:30:04,463][20381] Updated weights for policy 1, policy_version 1132213 (0.0011) [2023-12-27 13:30:04,721][20347] Updated weights for policy 0, policy_version 1133334 (0.0007) [2023-12-27 13:30:04,773][20347] Updated weights for policy 0, policy_version 1133344 (0.0008) [2023-12-27 13:30:04,825][20347] Updated weights for policy 0, policy_version 1133354 (0.0007) [2023-12-27 13:30:05,093][20381] Updated weights for policy 1, policy_version 1132223 (0.0008) [2023-12-27 13:30:05,150][20381] Updated weights for policy 1, policy_version 1132233 (0.0008) [2023-12-27 13:30:05,203][20381] Updated weights for policy 1, policy_version 1132243 (0.0008) [2023-12-27 13:30:05,583][20347] Updated weights for policy 0, policy_version 1133364 (0.0008) [2023-12-27 13:30:05,641][20347] Updated weights for policy 0, policy_version 1133374 (0.0007) [2023-12-27 13:30:05,690][20347] Updated weights for policy 0, policy_version 1133384 (0.0007) [2023-12-27 13:30:05,914][20381] Updated weights for policy 1, policy_version 1132253 (0.0008) [2023-12-27 13:30:05,961][20381] Updated weights for policy 1, policy_version 1132263 (0.0005) [2023-12-27 13:30:06,009][20381] Updated weights for policy 1, policy_version 1132273 (0.0006) [2023-12-27 13:30:06,319][20347] Updated weights for policy 0, policy_version 1133394 (0.0006) [2023-12-27 13:30:06,371][20347] Updated weights for policy 0, policy_version 1133404 (0.0008) [2023-12-27 13:30:06,421][20347] Updated weights for policy 0, policy_version 1133414 (0.0007) [2023-12-27 13:30:06,475][20291] KL-divergence is very high: 443.6090 [2023-12-27 13:30:06,481][20347] Updated weights for policy 0, policy_version 1133424 (0.0008) [2023-12-27 13:30:06,699][20381] Updated weights for policy 1, policy_version 1132283 (0.0006) [2023-12-27 13:30:06,749][20381] Updated weights for policy 1, policy_version 1132293 (0.0007) [2023-12-27 13:30:06,801][20381] Updated weights for policy 1, policy_version 1132303 (0.0007) [2023-12-27 13:30:07,237][20347] Updated weights for policy 0, policy_version 1133434 (0.0007) [2023-12-27 13:30:07,297][20347] Updated weights for policy 0, policy_version 1133444 (0.0008) [2023-12-27 13:30:07,320][20291] KL-divergence is very high: 111.8564 [2023-12-27 13:30:07,343][20347] Updated weights for policy 0, policy_version 1133454 (0.0007) [2023-12-27 13:30:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 580116480. Throughput: 0: 10282.3, 1: 10345.0. Samples: 580082908. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:07,363][19170] Avg episode reward: [(0, '970.930'), (1, '683.750')] [2023-12-27 13:30:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001133456_290217984.pth... [2023-12-27 13:30:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001132312_289898496.pth... [2023-12-27 13:30:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001132240_289906688.pth [2023-12-27 13:30:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001131096_289587200.pth [2023-12-27 13:30:07,404][20381] Updated weights for policy 1, policy_version 1132313 (0.0007) [2023-12-27 13:30:07,459][20381] Updated weights for policy 1, policy_version 1132323 (0.0006) [2023-12-27 13:30:07,510][20381] Updated weights for policy 1, policy_version 1132333 (0.0005) [2023-12-27 13:30:07,565][20381] Updated weights for policy 1, policy_version 1132343 (0.0005) [2023-12-27 13:30:08,106][20347] Updated weights for policy 0, policy_version 1133464 (0.0008) [2023-12-27 13:30:08,122][20381] Updated weights for policy 1, policy_version 1132353 (0.0005) [2023-12-27 13:30:08,168][20347] Updated weights for policy 0, policy_version 1133474 (0.0008) [2023-12-27 13:30:08,182][20381] Updated weights for policy 1, policy_version 1132363 (0.0005) [2023-12-27 13:30:08,235][20347] Updated weights for policy 0, policy_version 1133484 (0.0009) [2023-12-27 13:30:08,242][20381] Updated weights for policy 1, policy_version 1132373 (0.0006) [2023-12-27 13:30:08,881][20381] Updated weights for policy 1, policy_version 1132383 (0.0006) [2023-12-27 13:30:08,907][20347] Updated weights for policy 0, policy_version 1133494 (0.0009) [2023-12-27 13:30:08,941][20381] Updated weights for policy 1, policy_version 1132393 (0.0008) [2023-12-27 13:30:08,961][20347] Updated weights for policy 0, policy_version 1133504 (0.0012) [2023-12-27 13:30:08,993][20381] Updated weights for policy 1, policy_version 1132403 (0.0006) [2023-12-27 13:30:09,015][20347] Updated weights for policy 0, policy_version 1133514 (0.0006) [2023-12-27 13:30:09,699][20381] Updated weights for policy 1, policy_version 1132413 (0.0007) [2023-12-27 13:30:09,755][20381] Updated weights for policy 1, policy_version 1132423 (0.0007) [2023-12-27 13:30:09,757][20347] Updated weights for policy 0, policy_version 1133524 (0.0009) [2023-12-27 13:30:09,814][20381] Updated weights for policy 1, policy_version 1132433 (0.0006) [2023-12-27 13:30:09,816][20347] Updated weights for policy 0, policy_version 1133534 (0.0010) [2023-12-27 13:30:09,879][20347] Updated weights for policy 0, policy_version 1133544 (0.0010) [2023-12-27 13:30:10,568][20381] Updated weights for policy 1, policy_version 1132443 (0.0006) [2023-12-27 13:30:10,586][20347] Updated weights for policy 0, policy_version 1133554 (0.0008) [2023-12-27 13:30:10,624][20381] Updated weights for policy 1, policy_version 1132453 (0.0008) [2023-12-27 13:30:10,650][20347] Updated weights for policy 0, policy_version 1133564 (0.0008) [2023-12-27 13:30:10,676][20381] Updated weights for policy 1, policy_version 1132463 (0.0007) [2023-12-27 13:30:10,713][20347] Updated weights for policy 0, policy_version 1133574 (0.0011) [2023-12-27 13:30:10,772][20347] Updated weights for policy 0, policy_version 1133584 (0.0008) [2023-12-27 13:30:11,424][20381] Updated weights for policy 1, policy_version 1132473 (0.0007) [2023-12-27 13:30:11,447][20347] Updated weights for policy 0, policy_version 1133594 (0.0009) [2023-12-27 13:30:11,481][20381] Updated weights for policy 1, policy_version 1132483 (0.0008) [2023-12-27 13:30:11,498][20347] Updated weights for policy 0, policy_version 1133604 (0.0008) [2023-12-27 13:30:11,529][20381] Updated weights for policy 1, policy_version 1132493 (0.0008) [2023-12-27 13:30:11,555][20347] Updated weights for policy 0, policy_version 1133614 (0.0008) [2023-12-27 13:30:11,584][20381] Updated weights for policy 1, policy_version 1132503 (0.0008) [2023-12-27 13:30:12,317][20347] Updated weights for policy 0, policy_version 1133624 (0.0007) [2023-12-27 13:30:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 580206592. Throughput: 0: 10286.5, 1: 10406.2. Samples: 580205100. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:12,363][19170] Avg episode reward: [(0, '961.170'), (1, '628.100')] [2023-12-27 13:30:12,381][20381] Updated weights for policy 1, policy_version 1132513 (0.0007) [2023-12-27 13:30:12,383][20347] Updated weights for policy 0, policy_version 1133634 (0.0008) [2023-12-27 13:30:12,431][20381] Updated weights for policy 1, policy_version 1132523 (0.0008) [2023-12-27 13:30:12,438][20347] Updated weights for policy 0, policy_version 1133644 (0.0008) [2023-12-27 13:30:12,483][20381] Updated weights for policy 1, policy_version 1132533 (0.0006) [2023-12-27 13:30:13,124][20347] Updated weights for policy 0, policy_version 1133654 (0.0008) [2023-12-27 13:30:13,181][20347] Updated weights for policy 0, policy_version 1133664 (0.0007) [2023-12-27 13:30:13,216][20381] Updated weights for policy 1, policy_version 1132543 (0.0006) [2023-12-27 13:30:13,238][20347] Updated weights for policy 0, policy_version 1133674 (0.0006) [2023-12-27 13:30:13,274][20381] Updated weights for policy 1, policy_version 1132553 (0.0006) [2023-12-27 13:30:13,331][20381] Updated weights for policy 1, policy_version 1132563 (0.0005) [2023-12-27 13:30:13,878][20381] Updated weights for policy 1, policy_version 1132573 (0.0007) [2023-12-27 13:30:13,901][20347] Updated weights for policy 0, policy_version 1133684 (0.0007) [2023-12-27 13:30:13,934][20381] Updated weights for policy 1, policy_version 1132583 (0.0007) [2023-12-27 13:30:13,958][20347] Updated weights for policy 0, policy_version 1133694 (0.0008) [2023-12-27 13:30:13,990][20381] Updated weights for policy 1, policy_version 1132593 (0.0007) [2023-12-27 13:30:14,007][20347] Updated weights for policy 0, policy_version 1133704 (0.0006) [2023-12-27 13:30:14,643][20347] Updated weights for policy 0, policy_version 1133714 (0.0007) [2023-12-27 13:30:14,680][20381] Updated weights for policy 1, policy_version 1132603 (0.0006) [2023-12-27 13:30:14,704][20347] Updated weights for policy 0, policy_version 1133724 (0.0005) [2023-12-27 13:30:14,740][20381] Updated weights for policy 1, policy_version 1132613 (0.0008) [2023-12-27 13:30:14,760][20347] Updated weights for policy 0, policy_version 1133734 (0.0006) [2023-12-27 13:30:14,810][20381] Updated weights for policy 1, policy_version 1132623 (0.0008) [2023-12-27 13:30:14,828][20347] Updated weights for policy 0, policy_version 1133744 (0.0008) [2023-12-27 13:30:15,504][20381] Updated weights for policy 1, policy_version 1132633 (0.0008) [2023-12-27 13:30:15,515][20347] Updated weights for policy 0, policy_version 1133754 (0.0006) [2023-12-27 13:30:15,564][20381] Updated weights for policy 1, policy_version 1132643 (0.0008) [2023-12-27 13:30:15,567][20347] Updated weights for policy 0, policy_version 1133764 (0.0006) [2023-12-27 13:30:15,620][20381] Updated weights for policy 1, policy_version 1132653 (0.0006) [2023-12-27 13:30:15,621][20347] Updated weights for policy 0, policy_version 1133774 (0.0005) [2023-12-27 13:30:15,674][20381] Updated weights for policy 1, policy_version 1132663 (0.0007) [2023-12-27 13:30:16,277][20381] Updated weights for policy 1, policy_version 1132673 (0.0006) [2023-12-27 13:30:16,325][20347] Updated weights for policy 0, policy_version 1133784 (0.0009) [2023-12-27 13:30:16,331][20381] Updated weights for policy 1, policy_version 1132683 (0.0006) [2023-12-27 13:30:16,379][20347] Updated weights for policy 0, policy_version 1133794 (0.0010) [2023-12-27 13:30:16,389][20381] Updated weights for policy 1, policy_version 1132693 (0.0010) [2023-12-27 13:30:16,431][20347] Updated weights for policy 0, policy_version 1133804 (0.0006) [2023-12-27 13:30:17,060][20347] Updated weights for policy 0, policy_version 1133814 (0.0007) [2023-12-27 13:30:17,085][20381] Updated weights for policy 1, policy_version 1132703 (0.0008) [2023-12-27 13:30:17,112][20347] Updated weights for policy 0, policy_version 1133824 (0.0007) [2023-12-27 13:30:17,148][20381] Updated weights for policy 1, policy_version 1132713 (0.0007) [2023-12-27 13:30:17,165][20347] Updated weights for policy 0, policy_version 1133834 (0.0007) [2023-12-27 13:30:17,214][20381] Updated weights for policy 1, policy_version 1132723 (0.0006) [2023-12-27 13:30:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 580321280. Throughput: 0: 10203.7, 1: 10426.7. Samples: 580327348. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:17,363][19170] Avg episode reward: [(0, '980.580'), (1, '426.460')] [2023-12-27 13:30:17,756][20381] Updated weights for policy 1, policy_version 1132733 (0.0007) [2023-12-27 13:30:17,809][20381] Updated weights for policy 1, policy_version 1132743 (0.0008) [2023-12-27 13:30:17,838][20347] Updated weights for policy 0, policy_version 1133844 (0.0007) [2023-12-27 13:30:17,861][20381] Updated weights for policy 1, policy_version 1132753 (0.0007) [2023-12-27 13:30:17,902][20347] Updated weights for policy 0, policy_version 1133854 (0.0006) [2023-12-27 13:30:17,966][20347] Updated weights for policy 0, policy_version 1133864 (0.0005) [2023-12-27 13:30:18,562][20381] Updated weights for policy 1, policy_version 1132763 (0.0007) [2023-12-27 13:30:18,618][20381] Updated weights for policy 1, policy_version 1132773 (0.0008) [2023-12-27 13:30:18,644][20347] Updated weights for policy 0, policy_version 1133874 (0.0008) [2023-12-27 13:30:18,671][20381] Updated weights for policy 1, policy_version 1132783 (0.0008) [2023-12-27 13:30:18,697][20347] Updated weights for policy 0, policy_version 1133884 (0.0005) [2023-12-27 13:30:18,759][20347] Updated weights for policy 0, policy_version 1133894 (0.0007) [2023-12-27 13:30:18,824][20347] Updated weights for policy 0, policy_version 1133904 (0.0008) [2023-12-27 13:30:19,349][20381] Updated weights for policy 1, policy_version 1132793 (0.0007) [2023-12-27 13:30:19,408][20381] Updated weights for policy 1, policy_version 1132803 (0.0007) [2023-12-27 13:30:19,459][20381] Updated weights for policy 1, policy_version 1132813 (0.0007) [2023-12-27 13:30:19,519][20381] Updated weights for policy 1, policy_version 1132823 (0.0007) [2023-12-27 13:30:19,619][20347] Updated weights for policy 0, policy_version 1133914 (0.0008) [2023-12-27 13:30:19,683][20347] Updated weights for policy 0, policy_version 1133924 (0.0007) [2023-12-27 13:30:19,739][20347] Updated weights for policy 0, policy_version 1133934 (0.0005) [2023-12-27 13:30:20,291][20381] Updated weights for policy 1, policy_version 1132833 (0.0008) [2023-12-27 13:30:20,352][20381] Updated weights for policy 1, policy_version 1132843 (0.0008) [2023-12-27 13:30:20,412][20381] Updated weights for policy 1, policy_version 1132853 (0.0008) [2023-12-27 13:30:20,455][20347] Updated weights for policy 0, policy_version 1133944 (0.0008) [2023-12-27 13:30:20,509][20347] Updated weights for policy 0, policy_version 1133954 (0.0007) [2023-12-27 13:30:20,563][20347] Updated weights for policy 0, policy_version 1133964 (0.0007) [2023-12-27 13:30:21,109][20381] Updated weights for policy 1, policy_version 1132863 (0.0007) [2023-12-27 13:30:21,173][20381] Updated weights for policy 1, policy_version 1132873 (0.0006) [2023-12-27 13:30:21,238][20381] Updated weights for policy 1, policy_version 1132883 (0.0008) [2023-12-27 13:30:21,329][20347] Updated weights for policy 0, policy_version 1133974 (0.0008) [2023-12-27 13:30:21,404][20347] Updated weights for policy 0, policy_version 1133984 (0.0008) [2023-12-27 13:30:21,459][20347] Updated weights for policy 0, policy_version 1133994 (0.0006) [2023-12-27 13:30:21,949][20381] Updated weights for policy 1, policy_version 1132893 (0.0009) [2023-12-27 13:30:22,008][20381] Updated weights for policy 1, policy_version 1132903 (0.0011) [2023-12-27 13:30:22,064][20381] Updated weights for policy 1, policy_version 1132913 (0.0011) [2023-12-27 13:30:22,242][20347] Updated weights for policy 0, policy_version 1134004 (0.0007) [2023-12-27 13:30:22,308][20347] Updated weights for policy 0, policy_version 1134014 (0.0007) [2023-12-27 13:30:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 580411392. Throughput: 0: 10157.9, 1: 10397.5. Samples: 580388084. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:22,363][19170] Avg episode reward: [(0, '990.340'), (1, '144.970')] [2023-12-27 13:30:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001132920_290054144.pth... [2023-12-27 13:30:22,369][20347] Updated weights for policy 0, policy_version 1134024 (0.0008) [2023-12-27 13:30:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001131704_289742848.pth [2023-12-27 13:30:22,413][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001134032_290365440.pth... [2023-12-27 13:30:22,418][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001132848_290062336.pth [2023-12-27 13:30:22,809][20381] Updated weights for policy 1, policy_version 1132923 (0.0010) [2023-12-27 13:30:22,867][20381] Updated weights for policy 1, policy_version 1132933 (0.0007) [2023-12-27 13:30:22,927][20381] Updated weights for policy 1, policy_version 1132943 (0.0008) [2023-12-27 13:30:23,009][20347] Updated weights for policy 0, policy_version 1134034 (0.0005) [2023-12-27 13:30:23,073][20347] Updated weights for policy 0, policy_version 1134044 (0.0006) [2023-12-27 13:30:23,129][20347] Updated weights for policy 0, policy_version 1134054 (0.0006) [2023-12-27 13:30:23,189][20347] Updated weights for policy 0, policy_version 1134064 (0.0005) [2023-12-27 13:30:23,565][20381] Updated weights for policy 1, policy_version 1132953 (0.0008) [2023-12-27 13:30:23,617][20381] Updated weights for policy 1, policy_version 1132963 (0.0008) [2023-12-27 13:30:23,663][20381] Updated weights for policy 1, policy_version 1132973 (0.0008) [2023-12-27 13:30:23,711][20347] Updated weights for policy 0, policy_version 1134074 (0.0005) [2023-12-27 13:30:23,727][20381] Updated weights for policy 1, policy_version 1132983 (0.0006) [2023-12-27 13:30:23,775][20347] Updated weights for policy 0, policy_version 1134084 (0.0005) [2023-12-27 13:30:23,826][20347] Updated weights for policy 0, policy_version 1134094 (0.0005) [2023-12-27 13:30:24,414][20347] Updated weights for policy 0, policy_version 1134104 (0.0008) [2023-12-27 13:30:24,417][20381] Updated weights for policy 1, policy_version 1132993 (0.0007) [2023-12-27 13:30:24,465][20381] Updated weights for policy 1, policy_version 1133003 (0.0005) [2023-12-27 13:30:24,472][20347] Updated weights for policy 0, policy_version 1134114 (0.0005) [2023-12-27 13:30:24,524][20381] Updated weights for policy 1, policy_version 1133013 (0.0006) [2023-12-27 13:30:24,528][20347] Updated weights for policy 0, policy_version 1134124 (0.0006) [2023-12-27 13:30:25,174][20347] Updated weights for policy 0, policy_version 1134134 (0.0007) [2023-12-27 13:30:25,193][20381] Updated weights for policy 1, policy_version 1133023 (0.0007) [2023-12-27 13:30:25,229][20347] Updated weights for policy 0, policy_version 1134144 (0.0007) [2023-12-27 13:30:25,254][20381] Updated weights for policy 1, policy_version 1133033 (0.0007) [2023-12-27 13:30:25,292][20347] Updated weights for policy 0, policy_version 1134154 (0.0006) [2023-12-27 13:30:25,313][20381] Updated weights for policy 1, policy_version 1133043 (0.0007) [2023-12-27 13:30:25,972][20347] Updated weights for policy 0, policy_version 1134164 (0.0007) [2023-12-27 13:30:25,988][20381] Updated weights for policy 1, policy_version 1133053 (0.0009) [2023-12-27 13:30:26,023][20347] Updated weights for policy 0, policy_version 1134174 (0.0007) [2023-12-27 13:30:26,041][20381] Updated weights for policy 1, policy_version 1133063 (0.0006) [2023-12-27 13:30:26,083][20347] Updated weights for policy 0, policy_version 1134184 (0.0006) [2023-12-27 13:30:26,098][20381] Updated weights for policy 1, policy_version 1133073 (0.0006) [2023-12-27 13:30:26,704][20381] Updated weights for policy 1, policy_version 1133083 (0.0007) [2023-12-27 13:30:26,737][20347] Updated weights for policy 0, policy_version 1134194 (0.0006) [2023-12-27 13:30:26,764][20381] Updated weights for policy 1, policy_version 1133093 (0.0005) [2023-12-27 13:30:26,793][20347] Updated weights for policy 0, policy_version 1134204 (0.0006) [2023-12-27 13:30:26,817][20381] Updated weights for policy 1, policy_version 1133103 (0.0006) [2023-12-27 13:30:26,850][20347] Updated weights for policy 0, policy_version 1134214 (0.0008) [2023-12-27 13:30:26,901][20347] Updated weights for policy 0, policy_version 1134224 (0.0008) [2023-12-27 13:30:27,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 580517888. Throughput: 0: 10186.0, 1: 10368.2. Samples: 580509872. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:27,363][19170] Avg episode reward: [(0, '942.110'), (1, '170.870')] [2023-12-27 13:30:27,452][20381] Updated weights for policy 1, policy_version 1133113 (0.0005) [2023-12-27 13:30:27,501][20381] Updated weights for policy 1, policy_version 1133123 (0.0007) [2023-12-27 13:30:27,554][20381] Updated weights for policy 1, policy_version 1133133 (0.0008) [2023-12-27 13:30:27,599][20381] Updated weights for policy 1, policy_version 1133143 (0.0008) [2023-12-27 13:30:27,621][20347] Updated weights for policy 0, policy_version 1134234 (0.0005) [2023-12-27 13:30:27,666][20347] Updated weights for policy 0, policy_version 1134244 (0.0008) [2023-12-27 13:30:27,713][20347] Updated weights for policy 0, policy_version 1134254 (0.0010) [2023-12-27 13:30:28,274][20381] Updated weights for policy 1, policy_version 1133153 (0.0007) [2023-12-27 13:30:28,332][20381] Updated weights for policy 1, policy_version 1133163 (0.0007) [2023-12-27 13:30:28,358][20347] Updated weights for policy 0, policy_version 1134264 (0.0009) [2023-12-27 13:30:28,379][20291] KL-divergence is very high: 133.0427 [2023-12-27 13:30:28,396][20381] Updated weights for policy 1, policy_version 1133173 (0.0009) [2023-12-27 13:30:28,418][20347] Updated weights for policy 0, policy_version 1134274 (0.0006) [2023-12-27 13:30:28,426][20291] KL-divergence is very high: 101.1266 [2023-12-27 13:30:28,479][20347] Updated weights for policy 0, policy_version 1134284 (0.0007) [2023-12-27 13:30:29,081][20381] Updated weights for policy 1, policy_version 1133183 (0.0006) [2023-12-27 13:30:29,103][20347] Updated weights for policy 0, policy_version 1134294 (0.0009) [2023-12-27 13:30:29,132][20381] Updated weights for policy 1, policy_version 1133193 (0.0008) [2023-12-27 13:30:29,153][20347] Updated weights for policy 0, policy_version 1134304 (0.0010) [2023-12-27 13:30:29,182][20381] Updated weights for policy 1, policy_version 1133203 (0.0009) [2023-12-27 13:30:29,203][20347] Updated weights for policy 0, policy_version 1134314 (0.0010) [2023-12-27 13:30:29,908][20381] Updated weights for policy 1, policy_version 1133213 (0.0009) [2023-12-27 13:30:29,934][20347] Updated weights for policy 0, policy_version 1134324 (0.0009) [2023-12-27 13:30:29,977][20381] Updated weights for policy 1, policy_version 1133223 (0.0009) [2023-12-27 13:30:29,994][20347] Updated weights for policy 0, policy_version 1134334 (0.0007) [2023-12-27 13:30:30,032][20381] Updated weights for policy 1, policy_version 1133233 (0.0011) [2023-12-27 13:30:30,053][20347] Updated weights for policy 0, policy_version 1134344 (0.0008) [2023-12-27 13:30:30,753][20381] Updated weights for policy 1, policy_version 1133243 (0.0009) [2023-12-27 13:30:30,781][20347] Updated weights for policy 0, policy_version 1134354 (0.0008) [2023-12-27 13:30:30,809][20381] Updated weights for policy 1, policy_version 1133253 (0.0008) [2023-12-27 13:30:30,843][20347] Updated weights for policy 0, policy_version 1134364 (0.0007) [2023-12-27 13:30:30,864][20381] Updated weights for policy 1, policy_version 1133263 (0.0007) [2023-12-27 13:30:30,890][20347] Updated weights for policy 0, policy_version 1134374 (0.0006) [2023-12-27 13:30:30,942][20347] Updated weights for policy 0, policy_version 1134384 (0.0006) [2023-12-27 13:30:31,594][20381] Updated weights for policy 1, policy_version 1133273 (0.0007) [2023-12-27 13:30:31,659][20381] Updated weights for policy 1, policy_version 1133283 (0.0009) [2023-12-27 13:30:31,676][20347] Updated weights for policy 0, policy_version 1134394 (0.0006) [2023-12-27 13:30:31,716][20381] Updated weights for policy 1, policy_version 1133293 (0.0010) [2023-12-27 13:30:31,735][20347] Updated weights for policy 0, policy_version 1134404 (0.0006) [2023-12-27 13:30:31,776][20381] Updated weights for policy 1, policy_version 1133303 (0.0011) [2023-12-27 13:30:31,796][20347] Updated weights for policy 0, policy_version 1134414 (0.0008) [2023-12-27 13:30:32,367][19170] Fps is (10 sec: 20470.4, 60 sec: 20478.4, 300 sec: 20465.8). Total num frames: 580616192. Throughput: 0: 10139.7, 1: 10325.4. Samples: 580631388. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:32,368][19170] Avg episode reward: [(0, '864.580'), (1, '297.750')] [2023-12-27 13:30:32,516][20381] Updated weights for policy 1, policy_version 1133313 (0.0008) [2023-12-27 13:30:32,530][20347] Updated weights for policy 0, policy_version 1134424 (0.0009) [2023-12-27 13:30:32,570][20381] Updated weights for policy 1, policy_version 1133323 (0.0007) [2023-12-27 13:30:32,591][20347] Updated weights for policy 0, policy_version 1134434 (0.0006) [2023-12-27 13:30:32,627][20381] Updated weights for policy 1, policy_version 1133333 (0.0007) [2023-12-27 13:30:32,645][20347] Updated weights for policy 0, policy_version 1134444 (0.0006) [2023-12-27 13:30:33,290][20347] Updated weights for policy 0, policy_version 1134454 (0.0007) [2023-12-27 13:30:33,342][20347] Updated weights for policy 0, policy_version 1134464 (0.0006) [2023-12-27 13:30:33,347][20381] Updated weights for policy 1, policy_version 1133343 (0.0010) [2023-12-27 13:30:33,396][20347] Updated weights for policy 0, policy_version 1134474 (0.0008) [2023-12-27 13:30:33,403][20381] Updated weights for policy 1, policy_version 1133353 (0.0011) [2023-12-27 13:30:33,453][20381] Updated weights for policy 1, policy_version 1133363 (0.0011) [2023-12-27 13:30:34,055][20347] Updated weights for policy 0, policy_version 1134484 (0.0006) [2023-12-27 13:30:34,089][20381] Updated weights for policy 1, policy_version 1133373 (0.0010) [2023-12-27 13:30:34,125][20347] Updated weights for policy 0, policy_version 1134494 (0.0009) [2023-12-27 13:30:34,150][20381] Updated weights for policy 1, policy_version 1133383 (0.0008) [2023-12-27 13:30:34,194][20347] Updated weights for policy 0, policy_version 1134504 (0.0009) [2023-12-27 13:30:34,209][20381] Updated weights for policy 1, policy_version 1133393 (0.0009) [2023-12-27 13:30:34,845][20381] Updated weights for policy 1, policy_version 1133403 (0.0007) [2023-12-27 13:30:34,888][20347] Updated weights for policy 0, policy_version 1134514 (0.0009) [2023-12-27 13:30:34,903][20381] Updated weights for policy 1, policy_version 1133413 (0.0006) [2023-12-27 13:30:34,943][20347] Updated weights for policy 0, policy_version 1134524 (0.0007) [2023-12-27 13:30:34,958][20381] Updated weights for policy 1, policy_version 1133423 (0.0007) [2023-12-27 13:30:34,987][20347] Updated weights for policy 0, policy_version 1134534 (0.0006) [2023-12-27 13:30:35,037][20347] Updated weights for policy 0, policy_version 1134544 (0.0007) [2023-12-27 13:30:35,662][20381] Updated weights for policy 1, policy_version 1133433 (0.0006) [2023-12-27 13:30:35,720][20381] Updated weights for policy 1, policy_version 1133443 (0.0006) [2023-12-27 13:30:35,753][20347] Updated weights for policy 0, policy_version 1134554 (0.0007) [2023-12-27 13:30:35,771][20381] Updated weights for policy 1, policy_version 1133453 (0.0006) [2023-12-27 13:30:35,816][20347] Updated weights for policy 0, policy_version 1134564 (0.0008) [2023-12-27 13:30:35,818][20381] Updated weights for policy 1, policy_version 1133463 (0.0006) [2023-12-27 13:30:35,883][20347] Updated weights for policy 0, policy_version 1134574 (0.0008) [2023-12-27 13:30:36,463][20381] Updated weights for policy 1, policy_version 1133473 (0.0010) [2023-12-27 13:30:36,510][20381] Updated weights for policy 1, policy_version 1133483 (0.0011) [2023-12-27 13:30:36,567][20381] Updated weights for policy 1, policy_version 1133493 (0.0011) [2023-12-27 13:30:36,609][20347] Updated weights for policy 0, policy_version 1134584 (0.0008) [2023-12-27 13:30:36,668][20347] Updated weights for policy 0, policy_version 1134594 (0.0008) [2023-12-27 13:30:36,727][20347] Updated weights for policy 0, policy_version 1134604 (0.0007) [2023-12-27 13:30:37,235][20381] Updated weights for policy 1, policy_version 1133503 (0.0007) [2023-12-27 13:30:37,296][20381] Updated weights for policy 1, policy_version 1133513 (0.0008) [2023-12-27 13:30:37,354][20381] Updated weights for policy 1, policy_version 1133523 (0.0010) [2023-12-27 13:30:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 580714496. Throughput: 0: 10130.8, 1: 10293.5. Samples: 580692104. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:37,363][19170] Avg episode reward: [(0, '853.990'), (1, '325.990')] [2023-12-27 13:30:37,381][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001133528_290209792.pth... [2023-12-27 13:30:37,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001132312_289898496.pth [2023-12-27 13:30:37,401][20347] Updated weights for policy 0, policy_version 1134614 (0.0006) [2023-12-27 13:30:37,454][20347] Updated weights for policy 0, policy_version 1134624 (0.0005) [2023-12-27 13:30:37,515][20347] Updated weights for policy 0, policy_version 1134634 (0.0006) [2023-12-27 13:30:37,545][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001134640_290521088.pth... [2023-12-27 13:30:37,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001133456_290217984.pth [2023-12-27 13:30:37,980][20381] Updated weights for policy 1, policy_version 1133533 (0.0008) [2023-12-27 13:30:38,034][20381] Updated weights for policy 1, policy_version 1133543 (0.0010) [2023-12-27 13:30:38,089][20381] Updated weights for policy 1, policy_version 1133553 (0.0010) [2023-12-27 13:30:38,115][20347] Updated weights for policy 0, policy_version 1134644 (0.0008) [2023-12-27 13:30:38,175][20347] Updated weights for policy 0, policy_version 1134654 (0.0008) [2023-12-27 13:30:38,221][20347] Updated weights for policy 0, policy_version 1134664 (0.0007) [2023-12-27 13:30:38,783][20381] Updated weights for policy 1, policy_version 1133563 (0.0010) [2023-12-27 13:30:38,838][20381] Updated weights for policy 1, policy_version 1133573 (0.0010) [2023-12-27 13:30:38,893][20381] Updated weights for policy 1, policy_version 1133583 (0.0010) [2023-12-27 13:30:38,917][20347] Updated weights for policy 0, policy_version 1134674 (0.0007) [2023-12-27 13:30:38,969][20347] Updated weights for policy 0, policy_version 1134684 (0.0006) [2023-12-27 13:30:39,028][20347] Updated weights for policy 0, policy_version 1134694 (0.0006) [2023-12-27 13:30:39,086][20347] Updated weights for policy 0, policy_version 1134704 (0.0005) [2023-12-27 13:30:39,621][20381] Updated weights for policy 1, policy_version 1133593 (0.0011) [2023-12-27 13:30:39,684][20381] Updated weights for policy 1, policy_version 1133603 (0.0010) [2023-12-27 13:30:39,715][20347] Updated weights for policy 0, policy_version 1134714 (0.0008) [2023-12-27 13:30:39,744][20381] Updated weights for policy 1, policy_version 1133613 (0.0010) [2023-12-27 13:30:39,775][20347] Updated weights for policy 0, policy_version 1134724 (0.0005) [2023-12-27 13:30:39,803][20381] Updated weights for policy 1, policy_version 1133623 (0.0011) [2023-12-27 13:30:39,843][20347] Updated weights for policy 0, policy_version 1134734 (0.0007) [2023-12-27 13:30:40,523][20381] Updated weights for policy 1, policy_version 1133633 (0.0011) [2023-12-27 13:30:40,574][20347] Updated weights for policy 0, policy_version 1134744 (0.0006) [2023-12-27 13:30:40,579][20381] Updated weights for policy 1, policy_version 1133643 (0.0010) [2023-12-27 13:30:40,630][20347] Updated weights for policy 0, policy_version 1134754 (0.0006) [2023-12-27 13:30:40,635][20381] Updated weights for policy 1, policy_version 1133653 (0.0011) [2023-12-27 13:30:40,689][20347] Updated weights for policy 0, policy_version 1134764 (0.0006) [2023-12-27 13:30:41,368][20347] Updated weights for policy 0, policy_version 1134774 (0.0008) [2023-12-27 13:30:41,382][20381] Updated weights for policy 1, policy_version 1133663 (0.0009) [2023-12-27 13:30:41,425][20347] Updated weights for policy 0, policy_version 1134784 (0.0008) [2023-12-27 13:30:41,442][20381] Updated weights for policy 1, policy_version 1133673 (0.0008) [2023-12-27 13:30:41,478][20347] Updated weights for policy 0, policy_version 1134794 (0.0006) [2023-12-27 13:30:41,495][20381] Updated weights for policy 1, policy_version 1133683 (0.0007) [2023-12-27 13:30:42,230][20381] Updated weights for policy 1, policy_version 1133693 (0.0007) [2023-12-27 13:30:42,231][20347] Updated weights for policy 0, policy_version 1134804 (0.0006) [2023-12-27 13:30:42,292][20381] Updated weights for policy 1, policy_version 1133703 (0.0007) [2023-12-27 13:30:42,296][20347] Updated weights for policy 0, policy_version 1134814 (0.0008) [2023-12-27 13:30:42,347][20381] Updated weights for policy 1, policy_version 1133713 (0.0007) [2023-12-27 13:30:42,354][20347] Updated weights for policy 0, policy_version 1134824 (0.0006) [2023-12-27 13:30:42,362][19170] Fps is (10 sec: 19670.3, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 580812800. Throughput: 0: 10117.7, 1: 10236.9. Samples: 580813784. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:42,370][19170] Avg episode reward: [(0, '863.270'), (1, '348.880')] [2023-12-27 13:30:43,008][20347] Updated weights for policy 0, policy_version 1134834 (0.0008) [2023-12-27 13:30:43,057][20347] Updated weights for policy 0, policy_version 1134844 (0.0007) [2023-12-27 13:30:43,082][20381] Updated weights for policy 1, policy_version 1133723 (0.0008) [2023-12-27 13:30:43,104][20347] Updated weights for policy 0, policy_version 1134854 (0.0008) [2023-12-27 13:30:43,140][20381] Updated weights for policy 1, policy_version 1133733 (0.0007) [2023-12-27 13:30:43,150][20347] Updated weights for policy 0, policy_version 1134864 (0.0005) [2023-12-27 13:30:43,187][20381] Updated weights for policy 1, policy_version 1133743 (0.0007) [2023-12-27 13:30:43,792][20347] Updated weights for policy 0, policy_version 1134874 (0.0008) [2023-12-27 13:30:43,842][20347] Updated weights for policy 0, policy_version 1134884 (0.0009) [2023-12-27 13:30:43,897][20347] Updated weights for policy 0, policy_version 1134894 (0.0010) [2023-12-27 13:30:43,898][20381] Updated weights for policy 1, policy_version 1133753 (0.0007) [2023-12-27 13:30:43,959][20381] Updated weights for policy 1, policy_version 1133763 (0.0005) [2023-12-27 13:30:44,014][20381] Updated weights for policy 1, policy_version 1133773 (0.0005) [2023-12-27 13:30:44,066][20381] Updated weights for policy 1, policy_version 1133783 (0.0006) [2023-12-27 13:30:44,534][20347] Updated weights for policy 0, policy_version 1134904 (0.0007) [2023-12-27 13:30:44,586][20347] Updated weights for policy 0, policy_version 1134914 (0.0008) [2023-12-27 13:30:44,638][20347] Updated weights for policy 0, policy_version 1134924 (0.0006) [2023-12-27 13:30:44,639][20381] Updated weights for policy 1, policy_version 1133793 (0.0007) [2023-12-27 13:30:44,691][20381] Updated weights for policy 1, policy_version 1133803 (0.0008) [2023-12-27 13:30:44,736][20381] Updated weights for policy 1, policy_version 1133813 (0.0009) [2023-12-27 13:30:45,311][20347] Updated weights for policy 0, policy_version 1134934 (0.0007) [2023-12-27 13:30:45,349][20291] KL-divergence is very high: 127.3082 [2023-12-27 13:30:45,362][20347] Updated weights for policy 0, policy_version 1134944 (0.0008) [2023-12-27 13:30:45,390][20291] KL-divergence is very high: 130.5677 [2023-12-27 13:30:45,410][20347] Updated weights for policy 0, policy_version 1134954 (0.0007) [2023-12-27 13:30:45,475][20381] Updated weights for policy 1, policy_version 1133823 (0.0008) [2023-12-27 13:30:45,532][20381] Updated weights for policy 1, policy_version 1133833 (0.0008) [2023-12-27 13:30:45,584][20381] Updated weights for policy 1, policy_version 1133843 (0.0008) [2023-12-27 13:30:46,101][20347] Updated weights for policy 0, policy_version 1134964 (0.0008) [2023-12-27 13:30:46,147][20347] Updated weights for policy 0, policy_version 1134974 (0.0008) [2023-12-27 13:30:46,209][20347] Updated weights for policy 0, policy_version 1134984 (0.0008) [2023-12-27 13:30:46,275][20381] Updated weights for policy 1, policy_version 1133853 (0.0007) [2023-12-27 13:30:46,337][20381] Updated weights for policy 1, policy_version 1133863 (0.0008) [2023-12-27 13:30:46,398][20381] Updated weights for policy 1, policy_version 1133873 (0.0008) [2023-12-27 13:30:46,874][20347] Updated weights for policy 0, policy_version 1134994 (0.0006) [2023-12-27 13:30:46,934][20347] Updated weights for policy 0, policy_version 1135004 (0.0006) [2023-12-27 13:30:46,996][20347] Updated weights for policy 0, policy_version 1135014 (0.0005) [2023-12-27 13:30:47,060][20347] Updated weights for policy 0, policy_version 1135024 (0.0008) [2023-12-27 13:30:47,103][20381] Updated weights for policy 1, policy_version 1133883 (0.0008) [2023-12-27 13:30:47,153][20381] Updated weights for policy 1, policy_version 1133893 (0.0006) [2023-12-27 13:30:47,201][20381] Updated weights for policy 1, policy_version 1133903 (0.0006) [2023-12-27 13:30:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 580927488. Throughput: 0: 10134.0, 1: 10213.9. Samples: 580934716. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:47,364][19170] Avg episode reward: [(0, '902.960'), (1, '505.920')] [2023-12-27 13:30:47,634][20347] Updated weights for policy 0, policy_version 1135034 (0.0005) [2023-12-27 13:30:47,691][20347] Updated weights for policy 0, policy_version 1135044 (0.0007) [2023-12-27 13:30:47,750][20347] Updated weights for policy 0, policy_version 1135054 (0.0008) [2023-12-27 13:30:47,911][20381] Updated weights for policy 1, policy_version 1133913 (0.0009) [2023-12-27 13:30:47,959][20381] Updated weights for policy 1, policy_version 1133923 (0.0007) [2023-12-27 13:30:48,009][20381] Updated weights for policy 1, policy_version 1133933 (0.0007) [2023-12-27 13:30:48,063][20381] Updated weights for policy 1, policy_version 1133943 (0.0007) [2023-12-27 13:30:48,397][20347] Updated weights for policy 0, policy_version 1135064 (0.0007) [2023-12-27 13:30:48,461][20347] Updated weights for policy 0, policy_version 1135074 (0.0006) [2023-12-27 13:30:48,521][20347] Updated weights for policy 0, policy_version 1135084 (0.0006) [2023-12-27 13:30:48,801][20381] Updated weights for policy 1, policy_version 1133953 (0.0008) [2023-12-27 13:30:48,859][20381] Updated weights for policy 1, policy_version 1133963 (0.0008) [2023-12-27 13:30:48,919][20381] Updated weights for policy 1, policy_version 1133973 (0.0010) [2023-12-27 13:30:49,092][20347] Updated weights for policy 0, policy_version 1135094 (0.0007) [2023-12-27 13:30:49,138][20347] Updated weights for policy 0, policy_version 1135104 (0.0010) [2023-12-27 13:30:49,187][20347] Updated weights for policy 0, policy_version 1135114 (0.0010) [2023-12-27 13:30:49,666][20381] Updated weights for policy 1, policy_version 1133983 (0.0010) [2023-12-27 13:30:49,719][20381] Updated weights for policy 1, policy_version 1133993 (0.0011) [2023-12-27 13:30:49,775][20381] Updated weights for policy 1, policy_version 1134003 (0.0011) [2023-12-27 13:30:49,895][20347] Updated weights for policy 0, policy_version 1135124 (0.0009) [2023-12-27 13:30:49,958][20347] Updated weights for policy 0, policy_version 1135134 (0.0009) [2023-12-27 13:30:50,013][20347] Updated weights for policy 0, policy_version 1135144 (0.0008) [2023-12-27 13:30:50,467][20381] Updated weights for policy 1, policy_version 1134013 (0.0008) [2023-12-27 13:30:50,528][20381] Updated weights for policy 1, policy_version 1134023 (0.0007) [2023-12-27 13:30:50,616][20381] Updated weights for policy 1, policy_version 1134033 (0.0010) [2023-12-27 13:30:50,711][20347] Updated weights for policy 0, policy_version 1135154 (0.0007) [2023-12-27 13:30:50,775][20347] Updated weights for policy 0, policy_version 1135164 (0.0006) [2023-12-27 13:30:50,836][20347] Updated weights for policy 0, policy_version 1135174 (0.0010) [2023-12-27 13:30:50,900][20347] Updated weights for policy 0, policy_version 1135184 (0.0010) [2023-12-27 13:30:51,287][20381] Updated weights for policy 1, policy_version 1134043 (0.0009) [2023-12-27 13:30:51,361][20381] Updated weights for policy 1, policy_version 1134053 (0.0008) [2023-12-27 13:30:51,413][20381] Updated weights for policy 1, policy_version 1134063 (0.0007) [2023-12-27 13:30:51,576][20347] Updated weights for policy 0, policy_version 1135194 (0.0008) [2023-12-27 13:30:51,645][20347] Updated weights for policy 0, policy_version 1135204 (0.0007) [2023-12-27 13:30:51,706][20347] Updated weights for policy 0, policy_version 1135214 (0.0009) [2023-12-27 13:30:52,182][20381] Updated weights for policy 1, policy_version 1134073 (0.0008) [2023-12-27 13:30:52,244][20381] Updated weights for policy 1, policy_version 1134083 (0.0007) [2023-12-27 13:30:52,304][20381] Updated weights for policy 1, policy_version 1134093 (0.0008) [2023-12-27 13:30:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 581017600. Throughput: 0: 10156.0, 1: 10124.9. Samples: 580995548. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:52,363][19170] Avg episode reward: [(0, '873.540'), (1, '597.060')] [2023-12-27 13:30:52,367][20381] Updated weights for policy 1, policy_version 1134103 (0.0007) [2023-12-27 13:30:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001134104_290357248.pth... [2023-12-27 13:30:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001132920_290054144.pth [2023-12-27 13:30:52,383][20347] Updated weights for policy 0, policy_version 1135224 (0.0010) [2023-12-27 13:30:52,445][20347] Updated weights for policy 0, policy_version 1135234 (0.0010) [2023-12-27 13:30:52,500][20347] Updated weights for policy 0, policy_version 1135244 (0.0010) [2023-12-27 13:30:52,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001135248_290676736.pth... [2023-12-27 13:30:52,530][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001134032_290365440.pth [2023-12-27 13:30:53,075][20381] Updated weights for policy 1, policy_version 1134113 (0.0007) [2023-12-27 13:30:53,128][20381] Updated weights for policy 1, policy_version 1134123 (0.0007) [2023-12-27 13:30:53,182][20347] Updated weights for policy 0, policy_version 1135254 (0.0010) [2023-12-27 13:30:53,193][20381] Updated weights for policy 1, policy_version 1134133 (0.0007) [2023-12-27 13:30:53,239][20347] Updated weights for policy 0, policy_version 1135264 (0.0010) [2023-12-27 13:30:53,297][20347] Updated weights for policy 0, policy_version 1135274 (0.0010) [2023-12-27 13:30:53,832][20381] Updated weights for policy 1, policy_version 1134143 (0.0005) [2023-12-27 13:30:53,883][20347] Updated weights for policy 0, policy_version 1135284 (0.0008) [2023-12-27 13:30:53,887][20381] Updated weights for policy 1, policy_version 1134153 (0.0006) [2023-12-27 13:30:53,941][20381] Updated weights for policy 1, policy_version 1134163 (0.0006) [2023-12-27 13:30:53,943][20347] Updated weights for policy 0, policy_version 1135294 (0.0007) [2023-12-27 13:30:53,998][20347] Updated weights for policy 0, policy_version 1135304 (0.0010) [2023-12-27 13:30:54,578][20381] Updated weights for policy 1, policy_version 1134173 (0.0009) [2023-12-27 13:30:54,594][20347] Updated weights for policy 0, policy_version 1135314 (0.0010) [2023-12-27 13:30:54,631][20381] Updated weights for policy 1, policy_version 1134183 (0.0009) [2023-12-27 13:30:54,656][20347] Updated weights for policy 0, policy_version 1135324 (0.0005) [2023-12-27 13:30:54,689][20381] Updated weights for policy 1, policy_version 1134193 (0.0010) [2023-12-27 13:30:54,713][20347] Updated weights for policy 0, policy_version 1135334 (0.0005) [2023-12-27 13:30:54,769][20347] Updated weights for policy 0, policy_version 1135344 (0.0006) [2023-12-27 13:30:55,283][20381] Updated weights for policy 1, policy_version 1134203 (0.0010) [2023-12-27 13:30:55,335][20381] Updated weights for policy 1, policy_version 1134213 (0.0005) [2023-12-27 13:30:55,339][20347] Updated weights for policy 0, policy_version 1135354 (0.0007) [2023-12-27 13:30:55,389][20381] Updated weights for policy 1, policy_version 1134223 (0.0005) [2023-12-27 13:30:55,398][20347] Updated weights for policy 0, policy_version 1135364 (0.0006) [2023-12-27 13:30:55,453][20347] Updated weights for policy 0, policy_version 1135374 (0.0008) [2023-12-27 13:30:56,028][20381] Updated weights for policy 1, policy_version 1134233 (0.0007) [2023-12-27 13:30:56,061][20347] Updated weights for policy 0, policy_version 1135384 (0.0007) [2023-12-27 13:30:56,082][20381] Updated weights for policy 1, policy_version 1134243 (0.0006) [2023-12-27 13:30:56,118][20347] Updated weights for policy 0, policy_version 1135394 (0.0007) [2023-12-27 13:30:56,135][20381] Updated weights for policy 1, policy_version 1134253 (0.0005) [2023-12-27 13:30:56,167][20347] Updated weights for policy 0, policy_version 1135404 (0.0006) [2023-12-27 13:30:56,184][20381] Updated weights for policy 1, policy_version 1134263 (0.0005) [2023-12-27 13:30:56,817][20347] Updated weights for policy 0, policy_version 1135414 (0.0006) [2023-12-27 13:30:56,833][20381] Updated weights for policy 1, policy_version 1134273 (0.0007) [2023-12-27 13:30:56,877][20347] Updated weights for policy 0, policy_version 1135424 (0.0007) [2023-12-27 13:30:56,880][20381] Updated weights for policy 1, policy_version 1134283 (0.0007) [2023-12-27 13:30:56,926][20381] Updated weights for policy 1, policy_version 1134293 (0.0006) [2023-12-27 13:30:56,927][20347] Updated weights for policy 0, policy_version 1135434 (0.0007) [2023-12-27 13:30:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 581132288. Throughput: 0: 10249.2, 1: 10108.0. Samples: 581121176. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:30:57,363][19170] Avg episode reward: [(0, '863.280'), (1, '608.160')] [2023-12-27 13:30:57,516][20347] Updated weights for policy 0, policy_version 1135444 (0.0008) [2023-12-27 13:30:57,570][20347] Updated weights for policy 0, policy_version 1135454 (0.0008) [2023-12-27 13:30:57,611][20381] Updated weights for policy 1, policy_version 1134303 (0.0008) [2023-12-27 13:30:57,631][20347] Updated weights for policy 0, policy_version 1135464 (0.0005) [2023-12-27 13:30:57,668][20381] Updated weights for policy 1, policy_version 1134313 (0.0005) [2023-12-27 13:30:57,724][20381] Updated weights for policy 1, policy_version 1134323 (0.0005) [2023-12-27 13:30:58,212][20347] Updated weights for policy 0, policy_version 1135474 (0.0007) [2023-12-27 13:30:58,274][20347] Updated weights for policy 0, policy_version 1135484 (0.0007) [2023-12-27 13:30:58,335][20347] Updated weights for policy 0, policy_version 1135494 (0.0008) [2023-12-27 13:30:58,402][20347] Updated weights for policy 0, policy_version 1135504 (0.0007) [2023-12-27 13:30:58,435][20381] Updated weights for policy 1, policy_version 1134333 (0.0007) [2023-12-27 13:30:58,498][20381] Updated weights for policy 1, policy_version 1134343 (0.0010) [2023-12-27 13:30:58,558][20381] Updated weights for policy 1, policy_version 1134353 (0.0011) [2023-12-27 13:30:59,230][20347] Updated weights for policy 0, policy_version 1135515 (0.0009) [2023-12-27 13:30:59,292][20347] Updated weights for policy 0, policy_version 1135525 (0.0010) [2023-12-27 13:30:59,334][20381] Updated weights for policy 1, policy_version 1134363 (0.0010) [2023-12-27 13:30:59,360][20347] Updated weights for policy 0, policy_version 1135535 (0.0007) [2023-12-27 13:30:59,400][20381] Updated weights for policy 1, policy_version 1134373 (0.0007) [2023-12-27 13:30:59,452][20381] Updated weights for policy 1, policy_version 1134383 (0.0008) [2023-12-27 13:31:00,068][20347] Updated weights for policy 0, policy_version 1135545 (0.0007) [2023-12-27 13:31:00,124][20347] Updated weights for policy 0, policy_version 1135555 (0.0008) [2023-12-27 13:31:00,152][20381] Updated weights for policy 1, policy_version 1134393 (0.0007) [2023-12-27 13:31:00,180][20347] Updated weights for policy 0, policy_version 1135565 (0.0007) [2023-12-27 13:31:00,215][20381] Updated weights for policy 1, policy_version 1134403 (0.0008) [2023-12-27 13:31:00,274][20381] Updated weights for policy 1, policy_version 1134413 (0.0008) [2023-12-27 13:31:00,328][20381] Updated weights for policy 1, policy_version 1134423 (0.0009) [2023-12-27 13:31:00,805][20347] Updated weights for policy 0, policy_version 1135575 (0.0005) [2023-12-27 13:31:00,863][20347] Updated weights for policy 0, policy_version 1135585 (0.0005) [2023-12-27 13:31:00,908][20347] Updated weights for policy 0, policy_version 1135595 (0.0007) [2023-12-27 13:31:00,997][20381] Updated weights for policy 1, policy_version 1134433 (0.0006) [2023-12-27 13:31:01,065][20381] Updated weights for policy 1, policy_version 1134443 (0.0008) [2023-12-27 13:31:01,129][20381] Updated weights for policy 1, policy_version 1134453 (0.0008) [2023-12-27 13:31:01,614][20347] Updated weights for policy 0, policy_version 1135605 (0.0008) [2023-12-27 13:31:01,679][20347] Updated weights for policy 0, policy_version 1135615 (0.0007) [2023-12-27 13:31:01,737][20347] Updated weights for policy 0, policy_version 1135625 (0.0009) [2023-12-27 13:31:01,807][20381] Updated weights for policy 1, policy_version 1134463 (0.0006) [2023-12-27 13:31:01,866][20381] Updated weights for policy 1, policy_version 1134473 (0.0008) [2023-12-27 13:31:01,920][20381] Updated weights for policy 1, policy_version 1134483 (0.0007) [2023-12-27 13:31:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 581230592. Throughput: 0: 10260.4, 1: 10083.3. Samples: 581242816. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:02,363][19170] Avg episode reward: [(0, '872.990'), (1, '579.120')] [2023-12-27 13:31:02,479][20347] Updated weights for policy 0, policy_version 1135635 (0.0007) [2023-12-27 13:31:02,538][20347] Updated weights for policy 0, policy_version 1135645 (0.0008) [2023-12-27 13:31:02,548][20381] Updated weights for policy 1, policy_version 1134493 (0.0008) [2023-12-27 13:31:02,592][20347] Updated weights for policy 0, policy_version 1135655 (0.0007) [2023-12-27 13:31:02,595][20381] Updated weights for policy 1, policy_version 1134503 (0.0009) [2023-12-27 13:31:02,645][20381] Updated weights for policy 1, policy_version 1134513 (0.0010) [2023-12-27 13:31:03,232][20347] Updated weights for policy 0, policy_version 1135665 (0.0008) [2023-12-27 13:31:03,280][20347] Updated weights for policy 0, policy_version 1135675 (0.0010) [2023-12-27 13:31:03,317][20381] Updated weights for policy 1, policy_version 1134523 (0.0009) [2023-12-27 13:31:03,325][20347] Updated weights for policy 0, policy_version 1135685 (0.0010) [2023-12-27 13:31:03,363][20381] Updated weights for policy 1, policy_version 1134533 (0.0010) [2023-12-27 13:31:03,371][20347] Updated weights for policy 0, policy_version 1135695 (0.0010) [2023-12-27 13:31:03,414][20381] Updated weights for policy 1, policy_version 1134543 (0.0006) [2023-12-27 13:31:04,080][20347] Updated weights for policy 0, policy_version 1135705 (0.0010) [2023-12-27 13:31:04,088][20381] Updated weights for policy 1, policy_version 1134553 (0.0008) [2023-12-27 13:31:04,140][20347] Updated weights for policy 0, policy_version 1135715 (0.0011) [2023-12-27 13:31:04,145][20381] Updated weights for policy 1, policy_version 1134563 (0.0008) [2023-12-27 13:31:04,197][20347] Updated weights for policy 0, policy_version 1135725 (0.0010) [2023-12-27 13:31:04,206][20381] Updated weights for policy 1, policy_version 1134573 (0.0007) [2023-12-27 13:31:04,261][20381] Updated weights for policy 1, policy_version 1134583 (0.0008) [2023-12-27 13:31:04,877][20347] Updated weights for policy 0, policy_version 1135735 (0.0010) [2023-12-27 13:31:04,929][20347] Updated weights for policy 0, policy_version 1135745 (0.0010) [2023-12-27 13:31:04,959][20381] Updated weights for policy 1, policy_version 1134593 (0.0008) [2023-12-27 13:31:04,979][20347] Updated weights for policy 0, policy_version 1135755 (0.0009) [2023-12-27 13:31:05,014][20381] Updated weights for policy 1, policy_version 1134603 (0.0006) [2023-12-27 13:31:05,073][20381] Updated weights for policy 1, policy_version 1134613 (0.0011) [2023-12-27 13:31:05,546][20347] Updated weights for policy 0, policy_version 1135765 (0.0005) [2023-12-27 13:31:05,601][20347] Updated weights for policy 0, policy_version 1135775 (0.0006) [2023-12-27 13:31:05,658][20347] Updated weights for policy 0, policy_version 1135785 (0.0006) [2023-12-27 13:31:05,763][20381] Updated weights for policy 1, policy_version 1134623 (0.0011) [2023-12-27 13:31:05,826][20381] Updated weights for policy 1, policy_version 1134633 (0.0011) [2023-12-27 13:31:05,878][20381] Updated weights for policy 1, policy_version 1134643 (0.0011) [2023-12-27 13:31:06,290][20347] Updated weights for policy 0, policy_version 1135795 (0.0007) [2023-12-27 13:31:06,346][20347] Updated weights for policy 0, policy_version 1135805 (0.0010) [2023-12-27 13:31:06,405][20347] Updated weights for policy 0, policy_version 1135815 (0.0011) [2023-12-27 13:31:06,620][20381] Updated weights for policy 1, policy_version 1134653 (0.0008) [2023-12-27 13:31:06,691][20381] Updated weights for policy 1, policy_version 1134663 (0.0006) [2023-12-27 13:31:06,752][20381] Updated weights for policy 1, policy_version 1134673 (0.0008) [2023-12-27 13:31:07,030][20347] Updated weights for policy 0, policy_version 1135825 (0.0010) [2023-12-27 13:31:07,085][20347] Updated weights for policy 0, policy_version 1135835 (0.0007) [2023-12-27 13:31:07,141][20347] Updated weights for policy 0, policy_version 1135845 (0.0008) [2023-12-27 13:31:07,202][20347] Updated weights for policy 0, policy_version 1135855 (0.0010) [2023-12-27 13:31:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 581337088. Throughput: 0: 10278.4, 1: 10085.6. Samples: 581304464. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:07,363][19170] Avg episode reward: [(0, '902.250'), (1, '657.330')] [2023-12-27 13:31:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001135856_290832384.pth... [2023-12-27 13:31:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001134680_290504704.pth... [2023-12-27 13:31:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001134640_290521088.pth [2023-12-27 13:31:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001133528_290209792.pth [2023-12-27 13:31:07,421][20381] Updated weights for policy 1, policy_version 1134683 (0.0009) [2023-12-27 13:31:07,480][20381] Updated weights for policy 1, policy_version 1134693 (0.0010) [2023-12-27 13:31:07,532][20381] Updated weights for policy 1, policy_version 1134703 (0.0010) [2023-12-27 13:31:07,857][20347] Updated weights for policy 0, policy_version 1135865 (0.0006) [2023-12-27 13:31:07,912][20347] Updated weights for policy 0, policy_version 1135875 (0.0006) [2023-12-27 13:31:07,972][20347] Updated weights for policy 0, policy_version 1135885 (0.0005) [2023-12-27 13:31:08,199][20381] Updated weights for policy 1, policy_version 1134713 (0.0010) [2023-12-27 13:31:08,251][20381] Updated weights for policy 1, policy_version 1134723 (0.0010) [2023-12-27 13:31:08,306][20381] Updated weights for policy 1, policy_version 1134733 (0.0010) [2023-12-27 13:31:08,365][20381] Updated weights for policy 1, policy_version 1134743 (0.0009) [2023-12-27 13:31:08,615][20347] Updated weights for policy 0, policy_version 1135895 (0.0007) [2023-12-27 13:31:08,669][20347] Updated weights for policy 0, policy_version 1135905 (0.0007) [2023-12-27 13:31:08,727][20347] Updated weights for policy 0, policy_version 1135915 (0.0008) [2023-12-27 13:31:08,965][20381] Updated weights for policy 1, policy_version 1134753 (0.0009) [2023-12-27 13:31:09,014][20381] Updated weights for policy 1, policy_version 1134763 (0.0010) [2023-12-27 13:31:09,066][20381] Updated weights for policy 1, policy_version 1134773 (0.0010) [2023-12-27 13:31:09,568][20347] Updated weights for policy 0, policy_version 1135925 (0.0007) [2023-12-27 13:31:09,622][20347] Updated weights for policy 0, policy_version 1135935 (0.0007) [2023-12-27 13:31:09,681][20347] Updated weights for policy 0, policy_version 1135945 (0.0007) [2023-12-27 13:31:09,797][20381] Updated weights for policy 1, policy_version 1134783 (0.0011) [2023-12-27 13:31:09,860][20381] Updated weights for policy 1, policy_version 1134793 (0.0011) [2023-12-27 13:31:09,931][20381] Updated weights for policy 1, policy_version 1134803 (0.0010) [2023-12-27 13:31:10,338][20347] Updated weights for policy 0, policy_version 1135955 (0.0008) [2023-12-27 13:31:10,398][20347] Updated weights for policy 0, policy_version 1135965 (0.0007) [2023-12-27 13:31:10,451][20347] Updated weights for policy 0, policy_version 1135975 (0.0007) [2023-12-27 13:31:10,643][20381] Updated weights for policy 1, policy_version 1134813 (0.0010) [2023-12-27 13:31:10,695][20381] Updated weights for policy 1, policy_version 1134823 (0.0010) [2023-12-27 13:31:10,744][20381] Updated weights for policy 1, policy_version 1134833 (0.0010) [2023-12-27 13:31:11,212][20347] Updated weights for policy 0, policy_version 1135985 (0.0007) [2023-12-27 13:31:11,272][20347] Updated weights for policy 0, policy_version 1135995 (0.0009) [2023-12-27 13:31:11,338][20347] Updated weights for policy 0, policy_version 1136005 (0.0006) [2023-12-27 13:31:11,414][20347] Updated weights for policy 0, policy_version 1136015 (0.0010) [2023-12-27 13:31:11,510][20381] Updated weights for policy 1, policy_version 1134843 (0.0010) [2023-12-27 13:31:11,562][20381] Updated weights for policy 1, policy_version 1134853 (0.0008) [2023-12-27 13:31:11,620][20381] Updated weights for policy 1, policy_version 1134863 (0.0008) [2023-12-27 13:31:12,131][20347] Updated weights for policy 0, policy_version 1136025 (0.0010) [2023-12-27 13:31:12,194][20347] Updated weights for policy 0, policy_version 1136035 (0.0010) [2023-12-27 13:31:12,254][20347] Updated weights for policy 0, policy_version 1136045 (0.0010) [2023-12-27 13:31:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 581435392. Throughput: 0: 10307.2, 1: 10091.5. Samples: 581427812. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:12,364][19170] Avg episode reward: [(0, '882.560'), (1, '628.540')] [2023-12-27 13:31:12,377][20381] Updated weights for policy 1, policy_version 1134873 (0.0008) [2023-12-27 13:31:12,444][20381] Updated weights for policy 1, policy_version 1134883 (0.0007) [2023-12-27 13:31:12,506][20381] Updated weights for policy 1, policy_version 1134893 (0.0008) [2023-12-27 13:31:12,571][20381] Updated weights for policy 1, policy_version 1134903 (0.0008) [2023-12-27 13:31:12,966][20347] Updated weights for policy 0, policy_version 1136055 (0.0010) [2023-12-27 13:31:13,028][20347] Updated weights for policy 0, policy_version 1136065 (0.0010) [2023-12-27 13:31:13,084][20347] Updated weights for policy 0, policy_version 1136075 (0.0010) [2023-12-27 13:31:13,270][20381] Updated weights for policy 1, policy_version 1134913 (0.0010) [2023-12-27 13:31:13,322][20381] Updated weights for policy 1, policy_version 1134923 (0.0010) [2023-12-27 13:31:13,376][20381] Updated weights for policy 1, policy_version 1134933 (0.0010) [2023-12-27 13:31:13,695][20347] Updated weights for policy 0, policy_version 1136085 (0.0008) [2023-12-27 13:31:13,768][20347] Updated weights for policy 0, policy_version 1136095 (0.0006) [2023-12-27 13:31:13,829][20347] Updated weights for policy 0, policy_version 1136105 (0.0007) [2023-12-27 13:31:13,851][20291] KL-divergence is very high: 251.3128 [2023-12-27 13:31:13,857][20291] KL-divergence is very high: 270.8670 [2023-12-27 13:31:14,020][20381] Updated weights for policy 1, policy_version 1134943 (0.0010) [2023-12-27 13:31:14,070][20381] Updated weights for policy 1, policy_version 1134953 (0.0005) [2023-12-27 13:31:14,124][20381] Updated weights for policy 1, policy_version 1134963 (0.0006) [2023-12-27 13:31:14,417][20291] KL-divergence is very high: 111.2550 [2023-12-27 13:31:14,427][20347] Updated weights for policy 0, policy_version 1136115 (0.0006) [2023-12-27 13:31:14,432][20291] KL-divergence is very high: 163.1510 [2023-12-27 13:31:14,457][20291] KL-divergence is very high: 133.5267 [2023-12-27 13:31:14,463][20291] KL-divergence is very high: 206.0298 [2023-12-27 13:31:14,473][20291] KL-divergence is very high: 101.3834 [2023-12-27 13:31:14,478][20347] Updated weights for policy 0, policy_version 1136125 (0.0007) [2023-12-27 13:31:14,497][20291] KL-divergence is very high: 121.1044 [2023-12-27 13:31:14,527][20347] Updated weights for policy 0, policy_version 1136135 (0.0007) [2023-12-27 13:31:14,541][20291] KL-divergence is very high: 112.4510 [2023-12-27 13:31:14,551][20291] KL-divergence is very high: 146.7454 [2023-12-27 13:31:14,747][20381] Updated weights for policy 1, policy_version 1134973 (0.0006) [2023-12-27 13:31:14,813][20381] Updated weights for policy 1, policy_version 1134983 (0.0008) [2023-12-27 13:31:14,879][20381] Updated weights for policy 1, policy_version 1134993 (0.0011) [2023-12-27 13:31:15,235][20347] Updated weights for policy 0, policy_version 1136145 (0.0006) [2023-12-27 13:31:15,295][20347] Updated weights for policy 0, policy_version 1136155 (0.0007) [2023-12-27 13:31:15,352][20347] Updated weights for policy 0, policy_version 1136165 (0.0008) [2023-12-27 13:31:15,408][20347] Updated weights for policy 0, policy_version 1136175 (0.0008) [2023-12-27 13:31:15,506][20381] Updated weights for policy 1, policy_version 1135003 (0.0009) [2023-12-27 13:31:15,563][20381] Updated weights for policy 1, policy_version 1135013 (0.0007) [2023-12-27 13:31:15,611][20381] Updated weights for policy 1, policy_version 1135023 (0.0009) [2023-12-27 13:31:16,041][20347] Updated weights for policy 0, policy_version 1136185 (0.0007) [2023-12-27 13:31:16,094][20347] Updated weights for policy 0, policy_version 1136195 (0.0005) [2023-12-27 13:31:16,146][20347] Updated weights for policy 0, policy_version 1136205 (0.0006) [2023-12-27 13:31:16,275][20381] Updated weights for policy 1, policy_version 1135033 (0.0008) [2023-12-27 13:31:16,320][20381] Updated weights for policy 1, policy_version 1135043 (0.0010) [2023-12-27 13:31:16,365][20381] Updated weights for policy 1, policy_version 1135053 (0.0010) [2023-12-27 13:31:16,410][20381] Updated weights for policy 1, policy_version 1135063 (0.0010) [2023-12-27 13:31:16,816][20347] Updated weights for policy 0, policy_version 1136215 (0.0007) [2023-12-27 13:31:16,870][20347] Updated weights for policy 0, policy_version 1136225 (0.0007) [2023-12-27 13:31:16,921][20347] Updated weights for policy 0, policy_version 1136235 (0.0006) [2023-12-27 13:31:17,082][20381] Updated weights for policy 1, policy_version 1135073 (0.0006) [2023-12-27 13:31:17,139][20381] Updated weights for policy 1, policy_version 1135083 (0.0005) [2023-12-27 13:31:17,204][20381] Updated weights for policy 1, policy_version 1135093 (0.0005) [2023-12-27 13:31:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 581541888. Throughput: 0: 10335.0, 1: 10102.5. Samples: 581550976. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:17,363][19170] Avg episode reward: [(0, '872.910'), (1, '628.670')] [2023-12-27 13:31:17,528][20347] Updated weights for policy 0, policy_version 1136245 (0.0008) [2023-12-27 13:31:17,579][20347] Updated weights for policy 0, policy_version 1136255 (0.0010) [2023-12-27 13:31:17,633][20347] Updated weights for policy 0, policy_version 1136265 (0.0010) [2023-12-27 13:31:17,787][20381] Updated weights for policy 1, policy_version 1135103 (0.0009) [2023-12-27 13:31:17,839][20381] Updated weights for policy 1, policy_version 1135113 (0.0010) [2023-12-27 13:31:17,894][20381] Updated weights for policy 1, policy_version 1135123 (0.0010) [2023-12-27 13:31:18,262][20347] Updated weights for policy 0, policy_version 1136275 (0.0009) [2023-12-27 13:31:18,331][20347] Updated weights for policy 0, policy_version 1136285 (0.0006) [2023-12-27 13:31:18,384][20347] Updated weights for policy 0, policy_version 1136295 (0.0007) [2023-12-27 13:31:18,612][20381] Updated weights for policy 1, policy_version 1135133 (0.0011) [2023-12-27 13:31:18,664][20381] Updated weights for policy 1, policy_version 1135143 (0.0010) [2023-12-27 13:31:18,712][20381] Updated weights for policy 1, policy_version 1135153 (0.0010) [2023-12-27 13:31:19,070][20347] Updated weights for policy 0, policy_version 1136305 (0.0007) [2023-12-27 13:31:19,121][20347] Updated weights for policy 0, policy_version 1136315 (0.0010) [2023-12-27 13:31:19,166][20347] Updated weights for policy 0, policy_version 1136325 (0.0009) [2023-12-27 13:31:19,226][20347] Updated weights for policy 0, policy_version 1136335 (0.0006) [2023-12-27 13:31:19,382][20381] Updated weights for policy 1, policy_version 1135163 (0.0009) [2023-12-27 13:31:19,448][20381] Updated weights for policy 1, policy_version 1135173 (0.0005) [2023-12-27 13:31:19,515][20381] Updated weights for policy 1, policy_version 1135183 (0.0009) [2023-12-27 13:31:19,944][20347] Updated weights for policy 0, policy_version 1136345 (0.0009) [2023-12-27 13:31:20,002][20347] Updated weights for policy 0, policy_version 1136355 (0.0010) [2023-12-27 13:31:20,008][20291] KL-divergence is very high: 144.3606 [2023-12-27 13:31:20,014][20291] KL-divergence is very high: 137.8236 [2023-12-27 13:31:20,025][20291] KL-divergence is very high: 110.5135 [2023-12-27 13:31:20,058][20347] Updated weights for policy 0, policy_version 1136365 (0.0008) [2023-12-27 13:31:20,170][20381] Updated weights for policy 1, policy_version 1135193 (0.0011) [2023-12-27 13:31:20,234][20381] Updated weights for policy 1, policy_version 1135203 (0.0010) [2023-12-27 13:31:20,295][20381] Updated weights for policy 1, policy_version 1135213 (0.0008) [2023-12-27 13:31:20,356][20381] Updated weights for policy 1, policy_version 1135223 (0.0006) [2023-12-27 13:31:20,755][20347] Updated weights for policy 0, policy_version 1136375 (0.0006) [2023-12-27 13:31:20,822][20347] Updated weights for policy 0, policy_version 1136385 (0.0006) [2023-12-27 13:31:20,888][20347] Updated weights for policy 0, policy_version 1136395 (0.0007) [2023-12-27 13:31:21,039][20381] Updated weights for policy 1, policy_version 1135233 (0.0008) [2023-12-27 13:31:21,103][20381] Updated weights for policy 1, policy_version 1135243 (0.0007) [2023-12-27 13:31:21,165][20381] Updated weights for policy 1, policy_version 1135253 (0.0007) [2023-12-27 13:31:21,596][20347] Updated weights for policy 0, policy_version 1136405 (0.0009) [2023-12-27 13:31:21,661][20347] Updated weights for policy 0, policy_version 1136415 (0.0009) [2023-12-27 13:31:21,724][20347] Updated weights for policy 0, policy_version 1136425 (0.0010) [2023-12-27 13:31:21,784][20381] Updated weights for policy 1, policy_version 1135263 (0.0006) [2023-12-27 13:31:21,847][20381] Updated weights for policy 1, policy_version 1135273 (0.0006) [2023-12-27 13:31:21,914][20381] Updated weights for policy 1, policy_version 1135283 (0.0005) [2023-12-27 13:31:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 581640192. Throughput: 0: 10354.7, 1: 10116.5. Samples: 581613308. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:22,363][19170] Avg episode reward: [(0, '814.080'), (1, '686.800')] [2023-12-27 13:31:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001135288_290660352.pth... [2023-12-27 13:31:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001136432_290979840.pth... [2023-12-27 13:31:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001134104_290357248.pth [2023-12-27 13:31:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001135248_290676736.pth [2023-12-27 13:31:22,447][20347] Updated weights for policy 0, policy_version 1136435 (0.0009) [2023-12-27 13:31:22,514][20347] Updated weights for policy 0, policy_version 1136445 (0.0010) [2023-12-27 13:31:22,530][20381] Updated weights for policy 1, policy_version 1135293 (0.0007) [2023-12-27 13:31:22,574][20347] Updated weights for policy 0, policy_version 1136455 (0.0011) [2023-12-27 13:31:22,584][20381] Updated weights for policy 1, policy_version 1135303 (0.0006) [2023-12-27 13:31:22,642][20381] Updated weights for policy 1, policy_version 1135313 (0.0006) [2023-12-27 13:31:23,239][20381] Updated weights for policy 1, policy_version 1135323 (0.0008) [2023-12-27 13:31:23,251][20347] Updated weights for policy 0, policy_version 1136465 (0.0010) [2023-12-27 13:31:23,291][20381] Updated weights for policy 1, policy_version 1135333 (0.0007) [2023-12-27 13:31:23,314][20347] Updated weights for policy 0, policy_version 1136475 (0.0008) [2023-12-27 13:31:23,343][20381] Updated weights for policy 1, policy_version 1135343 (0.0007) [2023-12-27 13:31:23,371][20347] Updated weights for policy 0, policy_version 1136485 (0.0007) [2023-12-27 13:31:23,427][20347] Updated weights for policy 0, policy_version 1136495 (0.0007) [2023-12-27 13:31:23,956][20381] Updated weights for policy 1, policy_version 1135353 (0.0005) [2023-12-27 13:31:24,007][20381] Updated weights for policy 1, policy_version 1135363 (0.0007) [2023-12-27 13:31:24,058][20381] Updated weights for policy 1, policy_version 1135373 (0.0007) [2023-12-27 13:31:24,096][20347] Updated weights for policy 0, policy_version 1136505 (0.0006) [2023-12-27 13:31:24,115][20381] Updated weights for policy 1, policy_version 1135383 (0.0007) [2023-12-27 13:31:24,150][20347] Updated weights for policy 0, policy_version 1136515 (0.0008) [2023-12-27 13:31:24,210][20347] Updated weights for policy 0, policy_version 1136525 (0.0008) [2023-12-27 13:31:24,815][20347] Updated weights for policy 0, policy_version 1136535 (0.0007) [2023-12-27 13:31:24,851][20381] Updated weights for policy 1, policy_version 1135393 (0.0007) [2023-12-27 13:31:24,866][20347] Updated weights for policy 0, policy_version 1136545 (0.0007) [2023-12-27 13:31:24,898][20381] Updated weights for policy 1, policy_version 1135403 (0.0007) [2023-12-27 13:31:24,918][20347] Updated weights for policy 0, policy_version 1136555 (0.0005) [2023-12-27 13:31:24,946][20381] Updated weights for policy 1, policy_version 1135413 (0.0005) [2023-12-27 13:31:25,522][20381] Updated weights for policy 1, policy_version 1135423 (0.0006) [2023-12-27 13:31:25,541][20347] Updated weights for policy 0, policy_version 1136565 (0.0007) [2023-12-27 13:31:25,578][20381] Updated weights for policy 1, policy_version 1135433 (0.0005) [2023-12-27 13:31:25,590][20347] Updated weights for policy 0, policy_version 1136575 (0.0010) [2023-12-27 13:31:25,634][20381] Updated weights for policy 1, policy_version 1135443 (0.0005) [2023-12-27 13:31:25,639][20347] Updated weights for policy 0, policy_version 1136585 (0.0010) [2023-12-27 13:31:26,224][20381] Updated weights for policy 1, policy_version 1135453 (0.0005) [2023-12-27 13:31:26,268][20381] Updated weights for policy 1, policy_version 1135463 (0.0005) [2023-12-27 13:31:26,299][20347] Updated weights for policy 0, policy_version 1136595 (0.0010) [2023-12-27 13:31:26,316][20381] Updated weights for policy 1, policy_version 1135473 (0.0005) [2023-12-27 13:31:26,353][20347] Updated weights for policy 0, policy_version 1136605 (0.0010) [2023-12-27 13:31:26,401][20347] Updated weights for policy 0, policy_version 1136615 (0.0007) [2023-12-27 13:31:26,920][20381] Updated weights for policy 1, policy_version 1135483 (0.0008) [2023-12-27 13:31:26,968][20381] Updated weights for policy 1, policy_version 1135493 (0.0005) [2023-12-27 13:31:27,017][20381] Updated weights for policy 1, policy_version 1135503 (0.0005) [2023-12-27 13:31:27,065][20347] Updated weights for policy 0, policy_version 1136625 (0.0006) [2023-12-27 13:31:27,128][20347] Updated weights for policy 0, policy_version 1136635 (0.0006) [2023-12-27 13:31:27,181][20347] Updated weights for policy 0, policy_version 1136645 (0.0005) [2023-12-27 13:31:27,237][20347] Updated weights for policy 0, policy_version 1136655 (0.0006) [2023-12-27 13:31:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 581754880. Throughput: 0: 10369.4, 1: 10211.3. Samples: 581739916. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:27,363][19170] Avg episode reward: [(0, '824.710'), (1, '637.700')] [2023-12-27 13:31:27,627][20381] Updated weights for policy 1, policy_version 1135513 (0.0007) [2023-12-27 13:31:27,677][20381] Updated weights for policy 1, policy_version 1135523 (0.0005) [2023-12-27 13:31:27,730][20381] Updated weights for policy 1, policy_version 1135533 (0.0005) [2023-12-27 13:31:27,785][20381] Updated weights for policy 1, policy_version 1135543 (0.0005) [2023-12-27 13:31:27,926][20347] Updated weights for policy 0, policy_version 1136665 (0.0007) [2023-12-27 13:31:27,974][20347] Updated weights for policy 0, policy_version 1136675 (0.0007) [2023-12-27 13:31:28,021][20347] Updated weights for policy 0, policy_version 1136685 (0.0007) [2023-12-27 13:31:28,365][20381] Updated weights for policy 1, policy_version 1135553 (0.0007) [2023-12-27 13:31:28,419][20381] Updated weights for policy 1, policy_version 1135563 (0.0007) [2023-12-27 13:31:28,481][20381] Updated weights for policy 1, policy_version 1135573 (0.0008) [2023-12-27 13:31:28,774][20347] Updated weights for policy 0, policy_version 1136695 (0.0008) [2023-12-27 13:31:28,833][20347] Updated weights for policy 0, policy_version 1136705 (0.0008) [2023-12-27 13:31:28,897][20347] Updated weights for policy 0, policy_version 1136715 (0.0008) [2023-12-27 13:31:29,114][20381] Updated weights for policy 1, policy_version 1135583 (0.0007) [2023-12-27 13:31:29,168][20381] Updated weights for policy 1, policy_version 1135593 (0.0007) [2023-12-27 13:31:29,230][20381] Updated weights for policy 1, policy_version 1135603 (0.0008) [2023-12-27 13:31:29,554][20347] Updated weights for policy 0, policy_version 1136725 (0.0008) [2023-12-27 13:31:29,604][20347] Updated weights for policy 0, policy_version 1136735 (0.0006) [2023-12-27 13:31:29,661][20347] Updated weights for policy 0, policy_version 1136745 (0.0008) [2023-12-27 13:31:29,949][20381] Updated weights for policy 1, policy_version 1135613 (0.0008) [2023-12-27 13:31:30,013][20381] Updated weights for policy 1, policy_version 1135623 (0.0006) [2023-12-27 13:31:30,073][20381] Updated weights for policy 1, policy_version 1135633 (0.0007) [2023-12-27 13:31:30,287][20347] Updated weights for policy 0, policy_version 1136755 (0.0008) [2023-12-27 13:31:30,336][20347] Updated weights for policy 0, policy_version 1136765 (0.0008) [2023-12-27 13:31:30,393][20347] Updated weights for policy 0, policy_version 1136775 (0.0008) [2023-12-27 13:31:30,709][20381] Updated weights for policy 1, policy_version 1135643 (0.0008) [2023-12-27 13:31:30,764][20381] Updated weights for policy 1, policy_version 1135653 (0.0007) [2023-12-27 13:31:30,814][20381] Updated weights for policy 1, policy_version 1135663 (0.0007) [2023-12-27 13:31:31,098][20347] Updated weights for policy 0, policy_version 1136785 (0.0008) [2023-12-27 13:31:31,170][20347] Updated weights for policy 0, policy_version 1136795 (0.0007) [2023-12-27 13:31:31,219][20347] Updated weights for policy 0, policy_version 1136805 (0.0005) [2023-12-27 13:31:31,279][20347] Updated weights for policy 0, policy_version 1136815 (0.0008) [2023-12-27 13:31:31,538][20381] Updated weights for policy 1, policy_version 1135673 (0.0007) [2023-12-27 13:31:31,584][20381] Updated weights for policy 1, policy_version 1135683 (0.0008) [2023-12-27 13:31:31,643][20381] Updated weights for policy 1, policy_version 1135693 (0.0007) [2023-12-27 13:31:31,697][20381] Updated weights for policy 1, policy_version 1135703 (0.0009) [2023-12-27 13:31:31,968][20347] Updated weights for policy 0, policy_version 1136825 (0.0007) [2023-12-27 13:31:32,031][20347] Updated weights for policy 0, policy_version 1136835 (0.0008) [2023-12-27 13:31:32,092][20347] Updated weights for policy 0, policy_version 1136845 (0.0007) [2023-12-27 13:31:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20618.2, 300 sec: 20466.1). Total num frames: 581853184. Throughput: 0: 10366.2, 1: 10297.4. Samples: 581864572. Policy #0 lag: (min: 18.0, avg: 27.2, max: 50.0) [2023-12-27 13:31:32,363][19170] Avg episode reward: [(0, '777.220'), (1, '676.770')] [2023-12-27 13:31:32,468][20381] Updated weights for policy 1, policy_version 1135713 (0.0008) [2023-12-27 13:31:32,532][20381] Updated weights for policy 1, policy_version 1135723 (0.0008) [2023-12-27 13:31:32,587][20381] Updated weights for policy 1, policy_version 1135733 (0.0008) [2023-12-27 13:31:32,811][20347] Updated weights for policy 0, policy_version 1136855 (0.0007) [2023-12-27 13:31:32,869][20347] Updated weights for policy 0, policy_version 1136865 (0.0007) [2023-12-27 13:31:32,917][20347] Updated weights for policy 0, policy_version 1136875 (0.0007) [2023-12-27 13:31:33,263][20381] Updated weights for policy 1, policy_version 1135743 (0.0008) [2023-12-27 13:31:33,315][20381] Updated weights for policy 1, policy_version 1135753 (0.0008) [2023-12-27 13:31:33,374][20381] Updated weights for policy 1, policy_version 1135763 (0.0008) [2023-12-27 13:31:33,531][20347] Updated weights for policy 0, policy_version 1136885 (0.0008) [2023-12-27 13:31:33,584][20347] Updated weights for policy 0, policy_version 1136895 (0.0008) [2023-12-27 13:31:33,639][20347] Updated weights for policy 0, policy_version 1136905 (0.0007) [2023-12-27 13:31:34,046][20381] Updated weights for policy 1, policy_version 1135773 (0.0008) [2023-12-27 13:31:34,102][20381] Updated weights for policy 1, policy_version 1135783 (0.0007) [2023-12-27 13:31:34,165][20381] Updated weights for policy 1, policy_version 1135793 (0.0008) [2023-12-27 13:31:34,348][20347] Updated weights for policy 0, policy_version 1136915 (0.0008) [2023-12-27 13:31:34,410][20347] Updated weights for policy 0, policy_version 1136925 (0.0010) [2023-12-27 13:31:34,489][20347] Updated weights for policy 0, policy_version 1136935 (0.0008) [2023-12-27 13:31:34,864][20381] Updated weights for policy 1, policy_version 1135803 (0.0007) [2023-12-27 13:31:34,908][20381] Updated weights for policy 1, policy_version 1135813 (0.0008) [2023-12-27 13:31:34,955][20381] Updated weights for policy 1, policy_version 1135823 (0.0007) [2023-12-27 13:31:35,140][20347] Updated weights for policy 0, policy_version 1136945 (0.0007) [2023-12-27 13:31:35,189][20347] Updated weights for policy 0, policy_version 1136955 (0.0008) [2023-12-27 13:31:35,235][20347] Updated weights for policy 0, policy_version 1136965 (0.0008) [2023-12-27 13:31:35,283][20347] Updated weights for policy 0, policy_version 1136975 (0.0007) [2023-12-27 13:31:35,624][20381] Updated weights for policy 1, policy_version 1135833 (0.0008) [2023-12-27 13:31:35,682][20381] Updated weights for policy 1, policy_version 1135843 (0.0007) [2023-12-27 13:31:35,744][20381] Updated weights for policy 1, policy_version 1135853 (0.0007) [2023-12-27 13:31:35,801][20381] Updated weights for policy 1, policy_version 1135863 (0.0008) [2023-12-27 13:31:36,017][20347] Updated weights for policy 0, policy_version 1136985 (0.0007) [2023-12-27 13:31:36,070][20347] Updated weights for policy 0, policy_version 1136995 (0.0006) [2023-12-27 13:31:36,125][20347] Updated weights for policy 0, policy_version 1137005 (0.0007) [2023-12-27 13:31:36,518][20381] Updated weights for policy 1, policy_version 1135873 (0.0008) [2023-12-27 13:31:36,574][20381] Updated weights for policy 1, policy_version 1135883 (0.0009) [2023-12-27 13:31:36,620][20381] Updated weights for policy 1, policy_version 1135893 (0.0006) [2023-12-27 13:31:36,845][20347] Updated weights for policy 0, policy_version 1137015 (0.0006) [2023-12-27 13:31:36,900][20347] Updated weights for policy 0, policy_version 1137025 (0.0008) [2023-12-27 13:31:36,951][20347] Updated weights for policy 0, policy_version 1137035 (0.0007) [2023-12-27 13:31:37,284][20381] Updated weights for policy 1, policy_version 1135903 (0.0007) [2023-12-27 13:31:37,333][20381] Updated weights for policy 1, policy_version 1135913 (0.0008) [2023-12-27 13:31:37,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 581951488. Throughput: 0: 10349.4, 1: 10313.5. Samples: 581925380. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:31:37,363][19170] Avg episode reward: [(0, '776.430'), (1, '706.120')] [2023-12-27 13:31:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001137040_291135488.pth... [2023-12-27 13:31:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001135856_290832384.pth [2023-12-27 13:31:37,386][20381] Updated weights for policy 1, policy_version 1135923 (0.0008) [2023-12-27 13:31:37,416][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001135928_290824192.pth... [2023-12-27 13:31:37,420][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001134680_290504704.pth [2023-12-27 13:31:37,636][20347] Updated weights for policy 0, policy_version 1137045 (0.0007) [2023-12-27 13:31:37,696][20347] Updated weights for policy 0, policy_version 1137055 (0.0009) [2023-12-27 13:31:37,759][20347] Updated weights for policy 0, policy_version 1137065 (0.0007) [2023-12-27 13:31:38,095][20381] Updated weights for policy 1, policy_version 1135933 (0.0007) [2023-12-27 13:31:38,148][20381] Updated weights for policy 1, policy_version 1135943 (0.0008) [2023-12-27 13:31:38,196][20381] Updated weights for policy 1, policy_version 1135953 (0.0007) [2023-12-27 13:31:38,329][20347] Updated weights for policy 0, policy_version 1137075 (0.0008) [2023-12-27 13:31:38,392][20347] Updated weights for policy 0, policy_version 1137085 (0.0009) [2023-12-27 13:31:38,459][20347] Updated weights for policy 0, policy_version 1137095 (0.0010) [2023-12-27 13:31:38,938][20381] Updated weights for policy 1, policy_version 1135963 (0.0008) [2023-12-27 13:31:38,991][20381] Updated weights for policy 1, policy_version 1135973 (0.0007) [2023-12-27 13:31:39,044][20381] Updated weights for policy 1, policy_version 1135983 (0.0008) [2023-12-27 13:31:39,117][20347] Updated weights for policy 0, policy_version 1137105 (0.0008) [2023-12-27 13:31:39,166][20347] Updated weights for policy 0, policy_version 1137115 (0.0008) [2023-12-27 13:31:39,230][20347] Updated weights for policy 0, policy_version 1137125 (0.0008) [2023-12-27 13:31:39,289][20347] Updated weights for policy 0, policy_version 1137135 (0.0007) [2023-12-27 13:31:39,827][20381] Updated weights for policy 1, policy_version 1135993 (0.0008) [2023-12-27 13:31:39,892][20381] Updated weights for policy 1, policy_version 1136003 (0.0007) [2023-12-27 13:31:39,959][20381] Updated weights for policy 1, policy_version 1136013 (0.0007) [2023-12-27 13:31:39,984][20347] Updated weights for policy 0, policy_version 1137145 (0.0008) [2023-12-27 13:31:40,013][20381] Updated weights for policy 1, policy_version 1136023 (0.0006) [2023-12-27 13:31:40,048][20347] Updated weights for policy 0, policy_version 1137155 (0.0007) [2023-12-27 13:31:40,129][20347] Updated weights for policy 0, policy_version 1137165 (0.0009) [2023-12-27 13:31:40,657][20381] Updated weights for policy 1, policy_version 1136033 (0.0009) [2023-12-27 13:31:40,713][20381] Updated weights for policy 1, policy_version 1136043 (0.0008) [2023-12-27 13:31:40,768][20381] Updated weights for policy 1, policy_version 1136053 (0.0008) [2023-12-27 13:31:40,805][20347] Updated weights for policy 0, policy_version 1137175 (0.0007) [2023-12-27 13:31:40,863][20347] Updated weights for policy 0, policy_version 1137185 (0.0005) [2023-12-27 13:31:40,922][20347] Updated weights for policy 0, policy_version 1137195 (0.0007) [2023-12-27 13:31:41,490][20381] Updated weights for policy 1, policy_version 1136063 (0.0008) [2023-12-27 13:31:41,549][20381] Updated weights for policy 1, policy_version 1136073 (0.0008) [2023-12-27 13:31:41,616][20381] Updated weights for policy 1, policy_version 1136083 (0.0008) [2023-12-27 13:31:41,632][20347] Updated weights for policy 0, policy_version 1137205 (0.0007) [2023-12-27 13:31:41,693][20347] Updated weights for policy 0, policy_version 1137215 (0.0008) [2023-12-27 13:31:41,763][20347] Updated weights for policy 0, policy_version 1137225 (0.0008) [2023-12-27 13:31:42,302][20381] Updated weights for policy 1, policy_version 1136093 (0.0007) [2023-12-27 13:31:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 582049792. Throughput: 0: 10285.4, 1: 10285.1. Samples: 582046844. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:31:42,363][19170] Avg episode reward: [(0, '796.370'), (1, '656.710')] [2023-12-27 13:31:42,371][20381] Updated weights for policy 1, policy_version 1136103 (0.0009) [2023-12-27 13:31:42,437][20381] Updated weights for policy 1, policy_version 1136113 (0.0008) [2023-12-27 13:31:42,512][20347] Updated weights for policy 0, policy_version 1137235 (0.0007) [2023-12-27 13:31:42,570][20347] Updated weights for policy 0, policy_version 1137245 (0.0006) [2023-12-27 13:31:42,621][20347] Updated weights for policy 0, policy_version 1137255 (0.0007) [2023-12-27 13:31:43,173][20381] Updated weights for policy 1, policy_version 1136123 (0.0008) [2023-12-27 13:31:43,237][20381] Updated weights for policy 1, policy_version 1136133 (0.0006) [2023-12-27 13:31:43,241][20347] Updated weights for policy 0, policy_version 1137265 (0.0006) [2023-12-27 13:31:43,291][20347] Updated weights for policy 0, policy_version 1137275 (0.0008) [2023-12-27 13:31:43,300][20381] Updated weights for policy 1, policy_version 1136143 (0.0005) [2023-12-27 13:31:43,348][20347] Updated weights for policy 0, policy_version 1137285 (0.0007) [2023-12-27 13:31:43,411][20347] Updated weights for policy 0, policy_version 1137295 (0.0010) [2023-12-27 13:31:43,893][20381] Updated weights for policy 1, policy_version 1136153 (0.0006) [2023-12-27 13:31:43,947][20381] Updated weights for policy 1, policy_version 1136163 (0.0007) [2023-12-27 13:31:43,992][20381] Updated weights for policy 1, policy_version 1136173 (0.0007) [2023-12-27 13:31:44,040][20381] Updated weights for policy 1, policy_version 1136183 (0.0007) [2023-12-27 13:31:44,114][20347] Updated weights for policy 0, policy_version 1137305 (0.0008) [2023-12-27 13:31:44,116][20291] KL-divergence is very high: 125.8419 [2023-12-27 13:31:44,166][20347] Updated weights for policy 0, policy_version 1137315 (0.0007) [2023-12-27 13:31:44,205][20291] KL-divergence is very high: 139.9549 [2023-12-27 13:31:44,220][20347] Updated weights for policy 0, policy_version 1137325 (0.0007) [2023-12-27 13:31:44,781][20381] Updated weights for policy 1, policy_version 1136193 (0.0007) [2023-12-27 13:31:44,839][20381] Updated weights for policy 1, policy_version 1136203 (0.0006) [2023-12-27 13:31:44,839][20347] Updated weights for policy 0, policy_version 1137335 (0.0007) [2023-12-27 13:31:44,894][20347] Updated weights for policy 0, policy_version 1137345 (0.0008) [2023-12-27 13:31:44,896][20381] Updated weights for policy 1, policy_version 1136213 (0.0006) [2023-12-27 13:31:44,947][20347] Updated weights for policy 0, policy_version 1137355 (0.0008) [2023-12-27 13:31:45,567][20381] Updated weights for policy 1, policy_version 1136223 (0.0006) [2023-12-27 13:31:45,618][20381] Updated weights for policy 1, policy_version 1136233 (0.0007) [2023-12-27 13:31:45,633][20347] Updated weights for policy 0, policy_version 1137365 (0.0007) [2023-12-27 13:31:45,676][20381] Updated weights for policy 1, policy_version 1136243 (0.0006) [2023-12-27 13:31:45,689][20347] Updated weights for policy 0, policy_version 1137375 (0.0010) [2023-12-27 13:31:45,734][20347] Updated weights for policy 0, policy_version 1137385 (0.0009) [2023-12-27 13:31:46,314][20381] Updated weights for policy 1, policy_version 1136253 (0.0006) [2023-12-27 13:31:46,347][20347] Updated weights for policy 0, policy_version 1137395 (0.0007) [2023-12-27 13:31:46,371][20381] Updated weights for policy 1, policy_version 1136263 (0.0006) [2023-12-27 13:31:46,405][20347] Updated weights for policy 0, policy_version 1137405 (0.0010) [2023-12-27 13:31:46,414][20381] Updated weights for policy 1, policy_version 1136273 (0.0007) [2023-12-27 13:31:46,456][20347] Updated weights for policy 0, policy_version 1137415 (0.0010) [2023-12-27 13:31:47,043][20347] Updated weights for policy 0, policy_version 1137425 (0.0010) [2023-12-27 13:31:47,102][20347] Updated weights for policy 0, policy_version 1137435 (0.0007) [2023-12-27 13:31:47,118][20381] Updated weights for policy 1, policy_version 1136283 (0.0006) [2023-12-27 13:31:47,159][20347] Updated weights for policy 0, policy_version 1137445 (0.0010) [2023-12-27 13:31:47,173][20381] Updated weights for policy 1, policy_version 1136293 (0.0006) [2023-12-27 13:31:47,217][20347] Updated weights for policy 0, policy_version 1137455 (0.0010) [2023-12-27 13:31:47,230][20381] Updated weights for policy 1, policy_version 1136303 (0.0007) [2023-12-27 13:31:47,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 582164480. Throughput: 0: 10320.5, 1: 10297.2. Samples: 582170612. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:31:47,363][19170] Avg episode reward: [(0, '796.390'), (1, '696.150')] [2023-12-27 13:31:47,780][20347] Updated weights for policy 0, policy_version 1137465 (0.0006) [2023-12-27 13:31:47,782][20291] KL-divergence is very high: 149.4108 [2023-12-27 13:31:47,787][20291] KL-divergence is very high: 167.9365 [2023-12-27 13:31:47,796][20381] Updated weights for policy 1, policy_version 1136313 (0.0005) [2023-12-27 13:31:47,827][20347] Updated weights for policy 0, policy_version 1137475 (0.0005) [2023-12-27 13:31:47,855][20381] Updated weights for policy 1, policy_version 1136323 (0.0006) [2023-12-27 13:31:47,880][20347] Updated weights for policy 0, policy_version 1137485 (0.0009) [2023-12-27 13:31:47,910][20381] Updated weights for policy 1, policy_version 1136333 (0.0007) [2023-12-27 13:31:47,964][20381] Updated weights for policy 1, policy_version 1136343 (0.0007) [2023-12-27 13:31:48,500][20291] KL-divergence is very high: 103.1249 [2023-12-27 13:31:48,524][20347] Updated weights for policy 0, policy_version 1137495 (0.0006) [2023-12-27 13:31:48,584][20347] Updated weights for policy 0, policy_version 1137505 (0.0005) [2023-12-27 13:31:48,653][20347] Updated weights for policy 0, policy_version 1137515 (0.0007) [2023-12-27 13:31:48,660][20381] Updated weights for policy 1, policy_version 1136353 (0.0007) [2023-12-27 13:31:48,717][20381] Updated weights for policy 1, policy_version 1136363 (0.0005) [2023-12-27 13:31:48,773][20381] Updated weights for policy 1, policy_version 1136373 (0.0005) [2023-12-27 13:31:49,279][20347] Updated weights for policy 0, policy_version 1137525 (0.0006) [2023-12-27 13:31:49,334][20347] Updated weights for policy 0, policy_version 1137535 (0.0007) [2023-12-27 13:31:49,402][20347] Updated weights for policy 0, policy_version 1137545 (0.0007) [2023-12-27 13:31:49,445][20381] Updated weights for policy 1, policy_version 1136383 (0.0007) [2023-12-27 13:31:49,501][20381] Updated weights for policy 1, policy_version 1136393 (0.0009) [2023-12-27 13:31:49,559][20381] Updated weights for policy 1, policy_version 1136403 (0.0008) [2023-12-27 13:31:50,097][20347] Updated weights for policy 0, policy_version 1137555 (0.0007) [2023-12-27 13:31:50,155][20347] Updated weights for policy 0, policy_version 1137565 (0.0011) [2023-12-27 13:31:50,220][20347] Updated weights for policy 0, policy_version 1137575 (0.0010) [2023-12-27 13:31:50,284][20381] Updated weights for policy 1, policy_version 1136413 (0.0008) [2023-12-27 13:31:50,342][20381] Updated weights for policy 1, policy_version 1136423 (0.0009) [2023-12-27 13:31:50,400][20381] Updated weights for policy 1, policy_version 1136433 (0.0007) [2023-12-27 13:31:50,828][20347] Updated weights for policy 0, policy_version 1137585 (0.0008) [2023-12-27 13:31:50,892][20347] Updated weights for policy 0, policy_version 1137595 (0.0011) [2023-12-27 13:31:50,957][20347] Updated weights for policy 0, policy_version 1137605 (0.0010) [2023-12-27 13:31:51,017][20347] Updated weights for policy 0, policy_version 1137615 (0.0011) [2023-12-27 13:31:51,097][20381] Updated weights for policy 1, policy_version 1136443 (0.0008) [2023-12-27 13:31:51,159][20381] Updated weights for policy 1, policy_version 1136453 (0.0008) [2023-12-27 13:31:51,213][20381] Updated weights for policy 1, policy_version 1136463 (0.0009) [2023-12-27 13:31:51,756][20347] Updated weights for policy 0, policy_version 1137625 (0.0007) [2023-12-27 13:31:51,816][20347] Updated weights for policy 0, policy_version 1137635 (0.0007) [2023-12-27 13:31:51,870][20347] Updated weights for policy 0, policy_version 1137645 (0.0010) [2023-12-27 13:31:52,017][20381] Updated weights for policy 1, policy_version 1136473 (0.0009) [2023-12-27 13:31:52,076][20381] Updated weights for policy 1, policy_version 1136483 (0.0006) [2023-12-27 13:31:52,134][20381] Updated weights for policy 1, policy_version 1136493 (0.0005) [2023-12-27 13:31:52,192][20381] Updated weights for policy 1, policy_version 1136503 (0.0005) [2023-12-27 13:31:52,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 582262784. Throughput: 0: 10351.9, 1: 10300.0. Samples: 582233800. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:31:52,363][19170] Avg episode reward: [(0, '743.660'), (1, '637.440')] [2023-12-27 13:31:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001136504_290971648.pth... [2023-12-27 13:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001137648_291291136.pth... [2023-12-27 13:31:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001135288_290660352.pth [2023-12-27 13:31:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001136432_290979840.pth [2023-12-27 13:31:52,520][20347] Updated weights for policy 0, policy_version 1137655 (0.0010) [2023-12-27 13:31:52,577][20347] Updated weights for policy 0, policy_version 1137665 (0.0008) [2023-12-27 13:31:52,637][20347] Updated weights for policy 0, policy_version 1137675 (0.0008) [2023-12-27 13:31:52,798][20381] Updated weights for policy 1, policy_version 1136513 (0.0007) [2023-12-27 13:31:52,851][20381] Updated weights for policy 1, policy_version 1136523 (0.0008) [2023-12-27 13:31:52,906][20381] Updated weights for policy 1, policy_version 1136533 (0.0008) [2023-12-27 13:31:53,325][20347] Updated weights for policy 0, policy_version 1137685 (0.0007) [2023-12-27 13:31:53,382][20347] Updated weights for policy 0, policy_version 1137695 (0.0008) [2023-12-27 13:31:53,438][20347] Updated weights for policy 0, policy_version 1137705 (0.0008) [2023-12-27 13:31:53,603][20381] Updated weights for policy 1, policy_version 1136543 (0.0007) [2023-12-27 13:31:53,650][20381] Updated weights for policy 1, policy_version 1136553 (0.0008) [2023-12-27 13:31:53,695][20381] Updated weights for policy 1, policy_version 1136563 (0.0008) [2023-12-27 13:31:54,138][20347] Updated weights for policy 0, policy_version 1137715 (0.0007) [2023-12-27 13:31:54,188][20347] Updated weights for policy 0, policy_version 1137725 (0.0008) [2023-12-27 13:31:54,235][20347] Updated weights for policy 0, policy_version 1137735 (0.0010) [2023-12-27 13:31:54,358][20381] Updated weights for policy 1, policy_version 1136573 (0.0006) [2023-12-27 13:31:54,420][20381] Updated weights for policy 1, policy_version 1136583 (0.0005) [2023-12-27 13:31:54,487][20381] Updated weights for policy 1, policy_version 1136593 (0.0005) [2023-12-27 13:31:54,886][20347] Updated weights for policy 0, policy_version 1137745 (0.0008) [2023-12-27 13:31:54,939][20347] Updated weights for policy 0, policy_version 1137755 (0.0008) [2023-12-27 13:31:54,999][20347] Updated weights for policy 0, policy_version 1137765 (0.0011) [2023-12-27 13:31:55,058][20347] Updated weights for policy 0, policy_version 1137775 (0.0007) [2023-12-27 13:31:55,125][20381] Updated weights for policy 1, policy_version 1136603 (0.0007) [2023-12-27 13:31:55,181][20381] Updated weights for policy 1, policy_version 1136613 (0.0005) [2023-12-27 13:31:55,237][20381] Updated weights for policy 1, policy_version 1136623 (0.0005) [2023-12-27 13:31:55,698][20347] Updated weights for policy 0, policy_version 1137785 (0.0007) [2023-12-27 13:31:55,758][20347] Updated weights for policy 0, policy_version 1137795 (0.0005) [2023-12-27 13:31:55,764][20381] Updated weights for policy 1, policy_version 1136633 (0.0005) [2023-12-27 13:31:55,813][20347] Updated weights for policy 0, policy_version 1137805 (0.0006) [2023-12-27 13:31:55,819][20381] Updated weights for policy 1, policy_version 1136643 (0.0009) [2023-12-27 13:31:55,874][20381] Updated weights for policy 1, policy_version 1136653 (0.0010) [2023-12-27 13:31:55,931][20381] Updated weights for policy 1, policy_version 1136663 (0.0010) [2023-12-27 13:31:56,392][20347] Updated weights for policy 0, policy_version 1137815 (0.0006) [2023-12-27 13:31:56,454][20347] Updated weights for policy 0, policy_version 1137825 (0.0005) [2023-12-27 13:31:56,515][20347] Updated weights for policy 0, policy_version 1137835 (0.0006) [2023-12-27 13:31:56,618][20381] Updated weights for policy 1, policy_version 1136673 (0.0010) [2023-12-27 13:31:56,669][20381] Updated weights for policy 1, policy_version 1136683 (0.0010) [2023-12-27 13:31:56,725][20381] Updated weights for policy 1, policy_version 1136693 (0.0010) [2023-12-27 13:31:57,026][20347] Updated weights for policy 0, policy_version 1137845 (0.0006) [2023-12-27 13:31:57,073][20347] Updated weights for policy 0, policy_version 1137855 (0.0005) [2023-12-27 13:31:57,124][20347] Updated weights for policy 0, policy_version 1137865 (0.0005) [2023-12-27 13:31:57,323][20381] Updated weights for policy 1, policy_version 1136703 (0.0006) [2023-12-27 13:31:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 582369280. Throughput: 0: 10323.9, 1: 10332.5. Samples: 582357352. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:31:57,364][19170] Avg episode reward: [(0, '646.950'), (1, '627.850')] [2023-12-27 13:31:57,374][20381] Updated weights for policy 1, policy_version 1136713 (0.0005) [2023-12-27 13:31:57,421][20381] Updated weights for policy 1, policy_version 1136723 (0.0005) [2023-12-27 13:31:57,728][20347] Updated weights for policy 0, policy_version 1137875 (0.0006) [2023-12-27 13:31:57,773][20347] Updated weights for policy 0, policy_version 1137885 (0.0008) [2023-12-27 13:31:57,818][20347] Updated weights for policy 0, policy_version 1137895 (0.0009) [2023-12-27 13:31:57,943][20381] Updated weights for policy 1, policy_version 1136733 (0.0005) [2023-12-27 13:31:57,988][20381] Updated weights for policy 1, policy_version 1136743 (0.0005) [2023-12-27 13:31:58,044][20381] Updated weights for policy 1, policy_version 1136753 (0.0006) [2023-12-27 13:31:58,613][20347] Updated weights for policy 0, policy_version 1137905 (0.0009) [2023-12-27 13:31:58,618][20291] KL-divergence is very high: 137.2163 [2023-12-27 13:31:58,674][20291] KL-divergence is very high: 104.6027 [2023-12-27 13:31:58,681][20347] Updated weights for policy 0, policy_version 1137915 (0.0009) [2023-12-27 13:31:58,761][20347] Updated weights for policy 0, policy_version 1137927 (0.0011) [2023-12-27 13:31:58,784][20381] Updated weights for policy 1, policy_version 1136763 (0.0008) [2023-12-27 13:31:58,851][20381] Updated weights for policy 1, policy_version 1136773 (0.0008) [2023-12-27 13:31:58,906][20381] Updated weights for policy 1, policy_version 1136783 (0.0007) [2023-12-27 13:31:59,550][20347] Updated weights for policy 0, policy_version 1137937 (0.0008) [2023-12-27 13:31:59,608][20347] Updated weights for policy 0, policy_version 1137947 (0.0007) [2023-12-27 13:31:59,634][20381] Updated weights for policy 1, policy_version 1136793 (0.0009) [2023-12-27 13:31:59,667][20347] Updated weights for policy 0, policy_version 1137957 (0.0006) [2023-12-27 13:31:59,688][20381] Updated weights for policy 1, policy_version 1136803 (0.0007) [2023-12-27 13:31:59,729][20347] Updated weights for policy 0, policy_version 1137967 (0.0007) [2023-12-27 13:31:59,742][20381] Updated weights for policy 1, policy_version 1136813 (0.0006) [2023-12-27 13:31:59,787][20381] Updated weights for policy 1, policy_version 1136823 (0.0009) [2023-12-27 13:32:00,398][20347] Updated weights for policy 0, policy_version 1137977 (0.0006) [2023-12-27 13:32:00,456][20347] Updated weights for policy 0, policy_version 1137987 (0.0008) [2023-12-27 13:32:00,513][20347] Updated weights for policy 0, policy_version 1137997 (0.0007) [2023-12-27 13:32:00,557][20381] Updated weights for policy 1, policy_version 1136833 (0.0007) [2023-12-27 13:32:00,609][20381] Updated weights for policy 1, policy_version 1136843 (0.0008) [2023-12-27 13:32:00,657][20381] Updated weights for policy 1, policy_version 1136853 (0.0007) [2023-12-27 13:32:01,246][20347] Updated weights for policy 0, policy_version 1138007 (0.0007) [2023-12-27 13:32:01,301][20381] Updated weights for policy 1, policy_version 1136863 (0.0008) [2023-12-27 13:32:01,317][20347] Updated weights for policy 0, policy_version 1138017 (0.0007) [2023-12-27 13:32:01,358][20381] Updated weights for policy 1, policy_version 1136873 (0.0006) [2023-12-27 13:32:01,389][20347] Updated weights for policy 0, policy_version 1138027 (0.0007) [2023-12-27 13:32:01,422][20381] Updated weights for policy 1, policy_version 1136883 (0.0006) [2023-12-27 13:32:02,018][20347] Updated weights for policy 0, policy_version 1138037 (0.0008) [2023-12-27 13:32:02,070][20347] Updated weights for policy 0, policy_version 1138047 (0.0008) [2023-12-27 13:32:02,103][20381] Updated weights for policy 1, policy_version 1136893 (0.0006) [2023-12-27 13:32:02,124][20347] Updated weights for policy 0, policy_version 1138057 (0.0006) [2023-12-27 13:32:02,156][20381] Updated weights for policy 1, policy_version 1136903 (0.0005) [2023-12-27 13:32:02,205][20381] Updated weights for policy 1, policy_version 1136913 (0.0005) [2023-12-27 13:32:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 582475776. Throughput: 0: 10335.3, 1: 10351.7. Samples: 582481896. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:02,363][19170] Avg episode reward: [(0, '611.820'), (1, '578.830')] [2023-12-27 13:32:02,818][20347] Updated weights for policy 0, policy_version 1138067 (0.0008) [2023-12-27 13:32:02,867][20381] Updated weights for policy 1, policy_version 1136923 (0.0007) [2023-12-27 13:32:02,882][20347] Updated weights for policy 0, policy_version 1138077 (0.0007) [2023-12-27 13:32:02,929][20381] Updated weights for policy 1, policy_version 1136933 (0.0010) [2023-12-27 13:32:02,935][20347] Updated weights for policy 0, policy_version 1138087 (0.0007) [2023-12-27 13:32:02,986][20381] Updated weights for policy 1, policy_version 1136943 (0.0011) [2023-12-27 13:32:03,575][20347] Updated weights for policy 0, policy_version 1138097 (0.0006) [2023-12-27 13:32:03,622][20381] Updated weights for policy 1, policy_version 1136953 (0.0010) [2023-12-27 13:32:03,630][20347] Updated weights for policy 0, policy_version 1138107 (0.0006) [2023-12-27 13:32:03,676][20381] Updated weights for policy 1, policy_version 1136963 (0.0008) [2023-12-27 13:32:03,685][20347] Updated weights for policy 0, policy_version 1138117 (0.0007) [2023-12-27 13:32:03,728][20381] Updated weights for policy 1, policy_version 1136973 (0.0010) [2023-12-27 13:32:03,739][20347] Updated weights for policy 0, policy_version 1138127 (0.0006) [2023-12-27 13:32:03,775][20381] Updated weights for policy 1, policy_version 1136983 (0.0010) [2023-12-27 13:32:04,459][20347] Updated weights for policy 0, policy_version 1138137 (0.0006) [2023-12-27 13:32:04,478][20381] Updated weights for policy 1, policy_version 1136993 (0.0010) [2023-12-27 13:32:04,525][20347] Updated weights for policy 0, policy_version 1138147 (0.0005) [2023-12-27 13:32:04,544][20381] Updated weights for policy 1, policy_version 1137003 (0.0010) [2023-12-27 13:32:04,576][20347] Updated weights for policy 0, policy_version 1138157 (0.0005) [2023-12-27 13:32:04,599][20381] Updated weights for policy 1, policy_version 1137013 (0.0010) [2023-12-27 13:32:05,254][20347] Updated weights for policy 0, policy_version 1138167 (0.0007) [2023-12-27 13:32:05,288][20381] Updated weights for policy 1, policy_version 1137023 (0.0011) [2023-12-27 13:32:05,309][20347] Updated weights for policy 0, policy_version 1138177 (0.0007) [2023-12-27 13:32:05,334][20381] Updated weights for policy 1, policy_version 1137033 (0.0010) [2023-12-27 13:32:05,361][20347] Updated weights for policy 0, policy_version 1138187 (0.0005) [2023-12-27 13:32:05,383][20381] Updated weights for policy 1, policy_version 1137043 (0.0011) [2023-12-27 13:32:06,079][20381] Updated weights for policy 1, policy_version 1137053 (0.0009) [2023-12-27 13:32:06,091][20347] Updated weights for policy 0, policy_version 1138197 (0.0007) [2023-12-27 13:32:06,145][20381] Updated weights for policy 1, policy_version 1137063 (0.0007) [2023-12-27 13:32:06,149][20291] KL-divergence is very high: 213.3117 [2023-12-27 13:32:06,154][20347] Updated weights for policy 0, policy_version 1138207 (0.0007) [2023-12-27 13:32:06,194][20291] KL-divergence is very high: 153.1195 [2023-12-27 13:32:06,206][20381] Updated weights for policy 1, policy_version 1137073 (0.0007) [2023-12-27 13:32:06,210][20347] Updated weights for policy 0, policy_version 1138217 (0.0009) [2023-12-27 13:32:06,865][20381] Updated weights for policy 1, policy_version 1137083 (0.0008) [2023-12-27 13:32:06,882][20347] Updated weights for policy 0, policy_version 1138227 (0.0006) [2023-12-27 13:32:06,915][20381] Updated weights for policy 1, policy_version 1137093 (0.0010) [2023-12-27 13:32:06,938][20347] Updated weights for policy 0, policy_version 1138237 (0.0007) [2023-12-27 13:32:06,964][20381] Updated weights for policy 1, policy_version 1137103 (0.0010) [2023-12-27 13:32:06,988][20347] Updated weights for policy 0, policy_version 1138247 (0.0006) [2023-12-27 13:32:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 582574080. Throughput: 0: 10315.7, 1: 10351.6. Samples: 582543340. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:07,364][19170] Avg episode reward: [(0, '694.610'), (1, '666.870')] [2023-12-27 13:32:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001137112_291127296.pth... [2023-12-27 13:32:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001138256_291446784.pth... [2023-12-27 13:32:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001135928_290824192.pth [2023-12-27 13:32:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001137040_291135488.pth [2023-12-27 13:32:07,566][20381] Updated weights for policy 1, policy_version 1137113 (0.0010) [2023-12-27 13:32:07,621][20381] Updated weights for policy 1, policy_version 1137123 (0.0005) [2023-12-27 13:32:07,669][20381] Updated weights for policy 1, policy_version 1137133 (0.0005) [2023-12-27 13:32:07,726][20381] Updated weights for policy 1, policy_version 1137143 (0.0007) [2023-12-27 13:32:07,735][20347] Updated weights for policy 0, policy_version 1138257 (0.0006) [2023-12-27 13:32:07,797][20347] Updated weights for policy 0, policy_version 1138267 (0.0008) [2023-12-27 13:32:07,860][20347] Updated weights for policy 0, policy_version 1138277 (0.0008) [2023-12-27 13:32:07,916][20347] Updated weights for policy 0, policy_version 1138287 (0.0007) [2023-12-27 13:32:08,246][20381] Updated weights for policy 1, policy_version 1137153 (0.0008) [2023-12-27 13:32:08,291][20381] Updated weights for policy 1, policy_version 1137163 (0.0008) [2023-12-27 13:32:08,338][20381] Updated weights for policy 1, policy_version 1137173 (0.0008) [2023-12-27 13:32:08,625][20347] Updated weights for policy 0, policy_version 1138297 (0.0007) [2023-12-27 13:32:08,674][20347] Updated weights for policy 0, policy_version 1138307 (0.0008) [2023-12-27 13:32:08,697][20291] KL-divergence is very high: 109.5196 [2023-12-27 13:32:08,711][20291] KL-divergence is very high: 112.4760 [2023-12-27 13:32:08,734][20347] Updated weights for policy 0, policy_version 1138317 (0.0008) [2023-12-27 13:32:09,024][20381] Updated weights for policy 1, policy_version 1137183 (0.0008) [2023-12-27 13:32:09,081][20381] Updated weights for policy 1, policy_version 1137193 (0.0007) [2023-12-27 13:32:09,139][20381] Updated weights for policy 1, policy_version 1137203 (0.0007) [2023-12-27 13:32:09,476][20347] Updated weights for policy 0, policy_version 1138327 (0.0008) [2023-12-27 13:32:09,528][20347] Updated weights for policy 0, policy_version 1138337 (0.0007) [2023-12-27 13:32:09,586][20347] Updated weights for policy 0, policy_version 1138347 (0.0007) [2023-12-27 13:32:09,824][20381] Updated weights for policy 1, policy_version 1137213 (0.0008) [2023-12-27 13:32:09,887][20381] Updated weights for policy 1, policy_version 1137223 (0.0011) [2023-12-27 13:32:09,942][20381] Updated weights for policy 1, policy_version 1137233 (0.0010) [2023-12-27 13:32:10,359][20347] Updated weights for policy 0, policy_version 1138357 (0.0008) [2023-12-27 13:32:10,421][20347] Updated weights for policy 0, policy_version 1138367 (0.0007) [2023-12-27 13:32:10,472][20347] Updated weights for policy 0, policy_version 1138377 (0.0007) [2023-12-27 13:32:10,675][20381] Updated weights for policy 1, policy_version 1137243 (0.0009) [2023-12-27 13:32:10,725][20381] Updated weights for policy 1, policy_version 1137253 (0.0009) [2023-12-27 13:32:10,774][20381] Updated weights for policy 1, policy_version 1137263 (0.0009) [2023-12-27 13:32:11,297][20347] Updated weights for policy 0, policy_version 1138387 (0.0007) [2023-12-27 13:32:11,363][20347] Updated weights for policy 0, policy_version 1138397 (0.0007) [2023-12-27 13:32:11,426][20347] Updated weights for policy 0, policy_version 1138407 (0.0009) [2023-12-27 13:32:11,621][20381] Updated weights for policy 1, policy_version 1137273 (0.0009) [2023-12-27 13:32:11,682][20381] Updated weights for policy 1, policy_version 1137283 (0.0007) [2023-12-27 13:32:11,752][20381] Updated weights for policy 1, policy_version 1137293 (0.0008) [2023-12-27 13:32:11,811][20381] Updated weights for policy 1, policy_version 1137303 (0.0007) [2023-12-27 13:32:12,144][20347] Updated weights for policy 0, policy_version 1138417 (0.0007) [2023-12-27 13:32:12,196][20347] Updated weights for policy 0, policy_version 1138427 (0.0008) [2023-12-27 13:32:12,254][20347] Updated weights for policy 0, policy_version 1138437 (0.0008) [2023-12-27 13:32:12,310][20347] Updated weights for policy 0, policy_version 1138447 (0.0008) [2023-12-27 13:32:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 582672384. Throughput: 0: 10215.5, 1: 10282.0. Samples: 582662304. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:12,363][19170] Avg episode reward: [(0, '624.690'), (1, '696.240')] [2023-12-27 13:32:12,491][20381] Updated weights for policy 1, policy_version 1137313 (0.0008) [2023-12-27 13:32:12,551][20381] Updated weights for policy 1, policy_version 1137323 (0.0008) [2023-12-27 13:32:12,610][20381] Updated weights for policy 1, policy_version 1137333 (0.0008) [2023-12-27 13:32:13,039][20347] Updated weights for policy 0, policy_version 1138457 (0.0008) [2023-12-27 13:32:13,088][20347] Updated weights for policy 0, policy_version 1138467 (0.0008) [2023-12-27 13:32:13,141][20347] Updated weights for policy 0, policy_version 1138477 (0.0009) [2023-12-27 13:32:13,257][20381] Updated weights for policy 1, policy_version 1137343 (0.0006) [2023-12-27 13:32:13,313][20381] Updated weights for policy 1, policy_version 1137353 (0.0008) [2023-12-27 13:32:13,355][20381] Updated weights for policy 1, policy_version 1137363 (0.0008) [2023-12-27 13:32:13,867][20347] Updated weights for policy 0, policy_version 1138487 (0.0008) [2023-12-27 13:32:13,919][20347] Updated weights for policy 0, policy_version 1138497 (0.0007) [2023-12-27 13:32:13,975][20347] Updated weights for policy 0, policy_version 1138507 (0.0008) [2023-12-27 13:32:14,024][20381] Updated weights for policy 1, policy_version 1137373 (0.0009) [2023-12-27 13:32:14,077][20381] Updated weights for policy 1, policy_version 1137383 (0.0009) [2023-12-27 13:32:14,123][20381] Updated weights for policy 1, policy_version 1137393 (0.0009) [2023-12-27 13:32:14,640][20347] Updated weights for policy 0, policy_version 1138517 (0.0007) [2023-12-27 13:32:14,693][20347] Updated weights for policy 0, policy_version 1138527 (0.0005) [2023-12-27 13:32:14,745][20347] Updated weights for policy 0, policy_version 1138537 (0.0006) [2023-12-27 13:32:14,893][20381] Updated weights for policy 1, policy_version 1137403 (0.0008) [2023-12-27 13:32:14,953][20381] Updated weights for policy 1, policy_version 1137413 (0.0007) [2023-12-27 13:32:15,019][20381] Updated weights for policy 1, policy_version 1137423 (0.0007) [2023-12-27 13:32:15,431][20347] Updated weights for policy 0, policy_version 1138547 (0.0008) [2023-12-27 13:32:15,490][20347] Updated weights for policy 0, policy_version 1138557 (0.0007) [2023-12-27 13:32:15,545][20347] Updated weights for policy 0, policy_version 1138567 (0.0007) [2023-12-27 13:32:15,719][20381] Updated weights for policy 1, policy_version 1137433 (0.0007) [2023-12-27 13:32:15,788][20381] Updated weights for policy 1, policy_version 1137443 (0.0007) [2023-12-27 13:32:15,853][20381] Updated weights for policy 1, policy_version 1137453 (0.0007) [2023-12-27 13:32:15,918][20381] Updated weights for policy 1, policy_version 1137463 (0.0007) [2023-12-27 13:32:16,255][20347] Updated weights for policy 0, policy_version 1138577 (0.0007) [2023-12-27 13:32:16,308][20347] Updated weights for policy 0, policy_version 1138587 (0.0007) [2023-12-27 13:32:16,360][20347] Updated weights for policy 0, policy_version 1138597 (0.0005) [2023-12-27 13:32:16,415][20347] Updated weights for policy 0, policy_version 1138607 (0.0006) [2023-12-27 13:32:16,604][20381] Updated weights for policy 1, policy_version 1137473 (0.0008) [2023-12-27 13:32:16,656][20381] Updated weights for policy 1, policy_version 1137483 (0.0008) [2023-12-27 13:32:16,708][20381] Updated weights for policy 1, policy_version 1137493 (0.0007) [2023-12-27 13:32:17,112][20347] Updated weights for policy 0, policy_version 1138617 (0.0008) [2023-12-27 13:32:17,168][20347] Updated weights for policy 0, policy_version 1138627 (0.0009) [2023-12-27 13:32:17,224][20347] Updated weights for policy 0, policy_version 1138637 (0.0006) [2023-12-27 13:32:17,354][20381] Updated weights for policy 1, policy_version 1137503 (0.0007) [2023-12-27 13:32:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20466.2). Total num frames: 582770688. Throughput: 0: 10173.9, 1: 10206.4. Samples: 582781684. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:17,363][19170] Avg episode reward: [(0, '600.750'), (1, '657.370')] [2023-12-27 13:32:17,399][20381] Updated weights for policy 1, policy_version 1137513 (0.0007) [2023-12-27 13:32:17,462][20381] Updated weights for policy 1, policy_version 1137523 (0.0008) [2023-12-27 13:32:17,855][20347] Updated weights for policy 0, policy_version 1138647 (0.0006) [2023-12-27 13:32:17,905][20347] Updated weights for policy 0, policy_version 1138657 (0.0006) [2023-12-27 13:32:17,959][20347] Updated weights for policy 0, policy_version 1138667 (0.0007) [2023-12-27 13:32:18,159][20381] Updated weights for policy 1, policy_version 1137533 (0.0009) [2023-12-27 13:32:18,212][20381] Updated weights for policy 1, policy_version 1137543 (0.0008) [2023-12-27 13:32:18,262][20381] Updated weights for policy 1, policy_version 1137553 (0.0008) [2023-12-27 13:32:18,594][20347] Updated weights for policy 0, policy_version 1138677 (0.0007) [2023-12-27 13:32:18,646][20347] Updated weights for policy 0, policy_version 1138687 (0.0005) [2023-12-27 13:32:18,703][20347] Updated weights for policy 0, policy_version 1138697 (0.0005) [2023-12-27 13:32:19,009][20381] Updated weights for policy 1, policy_version 1137563 (0.0009) [2023-12-27 13:32:19,056][20381] Updated weights for policy 1, policy_version 1137573 (0.0008) [2023-12-27 13:32:19,113][20381] Updated weights for policy 1, policy_version 1137583 (0.0005) [2023-12-27 13:32:19,344][20347] Updated weights for policy 0, policy_version 1138707 (0.0006) [2023-12-27 13:32:19,410][20347] Updated weights for policy 0, policy_version 1138717 (0.0008) [2023-12-27 13:32:19,465][20347] Updated weights for policy 0, policy_version 1138727 (0.0007) [2023-12-27 13:32:19,776][20381] Updated weights for policy 1, policy_version 1137593 (0.0006) [2023-12-27 13:32:19,841][20381] Updated weights for policy 1, policy_version 1137603 (0.0011) [2023-12-27 13:32:19,904][20381] Updated weights for policy 1, policy_version 1137613 (0.0008) [2023-12-27 13:32:19,964][20381] Updated weights for policy 1, policy_version 1137623 (0.0008) [2023-12-27 13:32:20,208][20347] Updated weights for policy 0, policy_version 1138737 (0.0008) [2023-12-27 13:32:20,275][20347] Updated weights for policy 0, policy_version 1138747 (0.0007) [2023-12-27 13:32:20,341][20347] Updated weights for policy 0, policy_version 1138757 (0.0005) [2023-12-27 13:32:20,405][20347] Updated weights for policy 0, policy_version 1138767 (0.0007) [2023-12-27 13:32:20,740][20381] Updated weights for policy 1, policy_version 1137633 (0.0009) [2023-12-27 13:32:20,800][20381] Updated weights for policy 1, policy_version 1137643 (0.0010) [2023-12-27 13:32:20,863][20381] Updated weights for policy 1, policy_version 1137653 (0.0010) [2023-12-27 13:32:21,048][20347] Updated weights for policy 0, policy_version 1138777 (0.0007) [2023-12-27 13:32:21,112][20347] Updated weights for policy 0, policy_version 1138787 (0.0009) [2023-12-27 13:32:21,185][20347] Updated weights for policy 0, policy_version 1138797 (0.0008) [2023-12-27 13:32:21,639][20381] Updated weights for policy 1, policy_version 1137663 (0.0009) [2023-12-27 13:32:21,704][20381] Updated weights for policy 1, policy_version 1137673 (0.0007) [2023-12-27 13:32:21,774][20381] Updated weights for policy 1, policy_version 1137683 (0.0007) [2023-12-27 13:32:21,885][20347] Updated weights for policy 0, policy_version 1138807 (0.0007) [2023-12-27 13:32:21,945][20347] Updated weights for policy 0, policy_version 1138817 (0.0007) [2023-12-27 13:32:22,005][20347] Updated weights for policy 0, policy_version 1138827 (0.0007) [2023-12-27 13:32:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 582868992. Throughput: 0: 10181.0, 1: 10191.7. Samples: 582842148. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:22,363][19170] Avg episode reward: [(0, '537.790'), (1, '608.510')] [2023-12-27 13:32:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001137688_291274752.pth... [2023-12-27 13:32:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001138832_291594240.pth... [2023-12-27 13:32:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001137648_291291136.pth [2023-12-27 13:32:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001136504_290971648.pth [2023-12-27 13:32:22,424][20381] Updated weights for policy 1, policy_version 1137693 (0.0008) [2023-12-27 13:32:22,476][20381] Updated weights for policy 1, policy_version 1137703 (0.0009) [2023-12-27 13:32:22,524][20381] Updated weights for policy 1, policy_version 1137713 (0.0009) [2023-12-27 13:32:22,719][20347] Updated weights for policy 0, policy_version 1138837 (0.0008) [2023-12-27 13:32:22,777][20347] Updated weights for policy 0, policy_version 1138847 (0.0008) [2023-12-27 13:32:22,839][20347] Updated weights for policy 0, policy_version 1138857 (0.0007) [2023-12-27 13:32:23,252][20381] Updated weights for policy 1, policy_version 1137723 (0.0008) [2023-12-27 13:32:23,310][20381] Updated weights for policy 1, policy_version 1137733 (0.0008) [2023-12-27 13:32:23,364][20381] Updated weights for policy 1, policy_version 1137743 (0.0007) [2023-12-27 13:32:23,510][20347] Updated weights for policy 0, policy_version 1138867 (0.0008) [2023-12-27 13:32:23,565][20347] Updated weights for policy 0, policy_version 1138877 (0.0008) [2023-12-27 13:32:23,618][20347] Updated weights for policy 0, policy_version 1138887 (0.0007) [2023-12-27 13:32:24,058][20381] Updated weights for policy 1, policy_version 1137753 (0.0008) [2023-12-27 13:32:24,111][20381] Updated weights for policy 1, policy_version 1137763 (0.0008) [2023-12-27 13:32:24,164][20381] Updated weights for policy 1, policy_version 1137773 (0.0008) [2023-12-27 13:32:24,217][20381] Updated weights for policy 1, policy_version 1137783 (0.0008) [2023-12-27 13:32:24,244][20347] Updated weights for policy 0, policy_version 1138897 (0.0007) [2023-12-27 13:32:24,305][20347] Updated weights for policy 0, policy_version 1138907 (0.0006) [2023-12-27 13:32:24,357][20347] Updated weights for policy 0, policy_version 1138917 (0.0008) [2023-12-27 13:32:24,403][20347] Updated weights for policy 0, policy_version 1138927 (0.0008) [2023-12-27 13:32:24,915][20381] Updated weights for policy 1, policy_version 1137793 (0.0008) [2023-12-27 13:32:24,970][20381] Updated weights for policy 1, policy_version 1137803 (0.0008) [2023-12-27 13:32:25,024][20381] Updated weights for policy 1, policy_version 1137813 (0.0007) [2023-12-27 13:32:25,083][20347] Updated weights for policy 0, policy_version 1138937 (0.0006) [2023-12-27 13:32:25,135][20347] Updated weights for policy 0, policy_version 1138947 (0.0005) [2023-12-27 13:32:25,190][20347] Updated weights for policy 0, policy_version 1138957 (0.0005) [2023-12-27 13:32:25,736][20381] Updated weights for policy 1, policy_version 1137823 (0.0008) [2023-12-27 13:32:25,756][20347] Updated weights for policy 0, policy_version 1138967 (0.0005) [2023-12-27 13:32:25,799][20381] Updated weights for policy 1, policy_version 1137833 (0.0008) [2023-12-27 13:32:25,815][20347] Updated weights for policy 0, policy_version 1138977 (0.0006) [2023-12-27 13:32:25,859][20381] Updated weights for policy 1, policy_version 1137843 (0.0008) [2023-12-27 13:32:25,870][20347] Updated weights for policy 0, policy_version 1138987 (0.0005) [2023-12-27 13:32:26,479][20347] Updated weights for policy 0, policy_version 1138997 (0.0006) [2023-12-27 13:32:26,538][20347] Updated weights for policy 0, policy_version 1139007 (0.0005) [2023-12-27 13:32:26,551][20381] Updated weights for policy 1, policy_version 1137853 (0.0008) [2023-12-27 13:32:26,598][20347] Updated weights for policy 0, policy_version 1139017 (0.0006) [2023-12-27 13:32:26,613][20381] Updated weights for policy 1, policy_version 1137863 (0.0006) [2023-12-27 13:32:26,663][20381] Updated weights for policy 1, policy_version 1137873 (0.0007) [2023-12-27 13:32:27,257][20347] Updated weights for policy 0, policy_version 1139027 (0.0007) [2023-12-27 13:32:27,282][20381] Updated weights for policy 1, policy_version 1137883 (0.0007) [2023-12-27 13:32:27,310][20347] Updated weights for policy 0, policy_version 1139037 (0.0006) [2023-12-27 13:32:27,335][20381] Updated weights for policy 1, policy_version 1137893 (0.0006) [2023-12-27 13:32:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 582967296. Throughput: 0: 10213.8, 1: 10149.9. Samples: 582963208. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:27,363][19170] Avg episode reward: [(0, '589.970'), (1, '588.950')] [2023-12-27 13:32:27,368][20347] Updated weights for policy 0, policy_version 1139047 (0.0007) [2023-12-27 13:32:27,388][20381] Updated weights for policy 1, policy_version 1137903 (0.0008) [2023-12-27 13:32:27,979][20347] Updated weights for policy 0, policy_version 1139057 (0.0006) [2023-12-27 13:32:28,041][20347] Updated weights for policy 0, policy_version 1139067 (0.0006) [2023-12-27 13:32:28,086][20347] Updated weights for policy 0, policy_version 1139077 (0.0007) [2023-12-27 13:32:28,113][20381] Updated weights for policy 1, policy_version 1137913 (0.0006) [2023-12-27 13:32:28,137][20347] Updated weights for policy 0, policy_version 1139087 (0.0006) [2023-12-27 13:32:28,169][20381] Updated weights for policy 1, policy_version 1137923 (0.0007) [2023-12-27 13:32:28,220][20381] Updated weights for policy 1, policy_version 1137933 (0.0007) [2023-12-27 13:32:28,268][20381] Updated weights for policy 1, policy_version 1137943 (0.0007) [2023-12-27 13:32:28,749][20347] Updated weights for policy 0, policy_version 1139097 (0.0007) [2023-12-27 13:32:28,800][20347] Updated weights for policy 0, policy_version 1139107 (0.0006) [2023-12-27 13:32:28,857][20347] Updated weights for policy 0, policy_version 1139117 (0.0006) [2023-12-27 13:32:28,979][20381] Updated weights for policy 1, policy_version 1137953 (0.0006) [2023-12-27 13:32:29,042][20381] Updated weights for policy 1, policy_version 1137963 (0.0007) [2023-12-27 13:32:29,101][20381] Updated weights for policy 1, policy_version 1137973 (0.0008) [2023-12-27 13:32:29,538][20347] Updated weights for policy 0, policy_version 1139127 (0.0006) [2023-12-27 13:32:29,598][20347] Updated weights for policy 0, policy_version 1139137 (0.0006) [2023-12-27 13:32:29,659][20347] Updated weights for policy 0, policy_version 1139147 (0.0005) [2023-12-27 13:32:29,737][20381] Updated weights for policy 1, policy_version 1137983 (0.0006) [2023-12-27 13:32:29,792][20381] Updated weights for policy 1, policy_version 1137993 (0.0006) [2023-12-27 13:32:29,852][20381] Updated weights for policy 1, policy_version 1138003 (0.0008) [2023-12-27 13:32:30,301][20347] Updated weights for policy 0, policy_version 1139157 (0.0007) [2023-12-27 13:32:30,362][20347] Updated weights for policy 0, policy_version 1139167 (0.0007) [2023-12-27 13:32:30,414][20347] Updated weights for policy 0, policy_version 1139177 (0.0007) [2023-12-27 13:32:30,492][20381] Updated weights for policy 1, policy_version 1138013 (0.0010) [2023-12-27 13:32:30,537][20381] Updated weights for policy 1, policy_version 1138023 (0.0010) [2023-12-27 13:32:30,585][20381] Updated weights for policy 1, policy_version 1138033 (0.0010) [2023-12-27 13:32:31,077][20347] Updated weights for policy 0, policy_version 1139187 (0.0007) [2023-12-27 13:32:31,142][20347] Updated weights for policy 0, policy_version 1139197 (0.0008) [2023-12-27 13:32:31,206][20347] Updated weights for policy 0, policy_version 1139207 (0.0007) [2023-12-27 13:32:31,211][20381] Updated weights for policy 1, policy_version 1138043 (0.0009) [2023-12-27 13:32:31,273][20381] Updated weights for policy 1, policy_version 1138053 (0.0006) [2023-12-27 13:32:31,340][20381] Updated weights for policy 1, policy_version 1138063 (0.0007) [2023-12-27 13:32:31,931][20347] Updated weights for policy 0, policy_version 1139217 (0.0007) [2023-12-27 13:32:31,972][20381] Updated weights for policy 1, policy_version 1138073 (0.0007) [2023-12-27 13:32:31,990][20347] Updated weights for policy 0, policy_version 1139227 (0.0008) [2023-12-27 13:32:32,033][20381] Updated weights for policy 1, policy_version 1138083 (0.0006) [2023-12-27 13:32:32,053][20347] Updated weights for policy 0, policy_version 1139237 (0.0007) [2023-12-27 13:32:32,086][20381] Updated weights for policy 1, policy_version 1138093 (0.0007) [2023-12-27 13:32:32,109][20347] Updated weights for policy 0, policy_version 1139247 (0.0007) [2023-12-27 13:32:32,134][20381] Updated weights for policy 1, policy_version 1138103 (0.0006) [2023-12-27 13:32:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 583081984. Throughput: 0: 10206.2, 1: 10199.7. Samples: 583088880. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:32,363][19170] Avg episode reward: [(0, '647.790'), (1, '726.170')] [2023-12-27 13:32:32,785][20347] Updated weights for policy 0, policy_version 1139257 (0.0006) [2023-12-27 13:32:32,802][20381] Updated weights for policy 1, policy_version 1138113 (0.0006) [2023-12-27 13:32:32,847][20347] Updated weights for policy 0, policy_version 1139267 (0.0007) [2023-12-27 13:32:32,849][20381] Updated weights for policy 1, policy_version 1138123 (0.0007) [2023-12-27 13:32:32,894][20381] Updated weights for policy 1, policy_version 1138133 (0.0007) [2023-12-27 13:32:32,899][20347] Updated weights for policy 0, policy_version 1139277 (0.0008) [2023-12-27 13:32:33,520][20381] Updated weights for policy 1, policy_version 1138143 (0.0005) [2023-12-27 13:32:33,570][20381] Updated weights for policy 1, policy_version 1138153 (0.0006) [2023-12-27 13:32:33,620][20381] Updated weights for policy 1, policy_version 1138163 (0.0007) [2023-12-27 13:32:33,648][20347] Updated weights for policy 0, policy_version 1139287 (0.0007) [2023-12-27 13:32:33,700][20347] Updated weights for policy 0, policy_version 1139297 (0.0008) [2023-12-27 13:32:33,758][20347] Updated weights for policy 0, policy_version 1139307 (0.0008) [2023-12-27 13:32:34,267][20381] Updated weights for policy 1, policy_version 1138173 (0.0007) [2023-12-27 13:32:34,326][20381] Updated weights for policy 1, policy_version 1138183 (0.0008) [2023-12-27 13:32:34,385][20381] Updated weights for policy 1, policy_version 1138193 (0.0007) [2023-12-27 13:32:34,489][20347] Updated weights for policy 0, policy_version 1139317 (0.0008) [2023-12-27 13:32:34,550][20347] Updated weights for policy 0, policy_version 1139327 (0.0007) [2023-12-27 13:32:34,614][20347] Updated weights for policy 0, policy_version 1139337 (0.0007) [2023-12-27 13:32:35,077][20381] Updated weights for policy 1, policy_version 1138203 (0.0006) [2023-12-27 13:32:35,121][20381] Updated weights for policy 1, policy_version 1138213 (0.0008) [2023-12-27 13:32:35,168][20381] Updated weights for policy 1, policy_version 1138223 (0.0008) [2023-12-27 13:32:35,284][20347] Updated weights for policy 0, policy_version 1139347 (0.0008) [2023-12-27 13:32:35,335][20347] Updated weights for policy 0, policy_version 1139357 (0.0006) [2023-12-27 13:32:35,386][20347] Updated weights for policy 0, policy_version 1139367 (0.0005) [2023-12-27 13:32:35,851][20381] Updated weights for policy 1, policy_version 1138233 (0.0007) [2023-12-27 13:32:35,908][20381] Updated weights for policy 1, policy_version 1138243 (0.0008) [2023-12-27 13:32:35,956][20381] Updated weights for policy 1, policy_version 1138253 (0.0009) [2023-12-27 13:32:35,994][20347] Updated weights for policy 0, policy_version 1139377 (0.0007) [2023-12-27 13:32:36,014][20381] Updated weights for policy 1, policy_version 1138263 (0.0007) [2023-12-27 13:32:36,040][20347] Updated weights for policy 0, policy_version 1139387 (0.0006) [2023-12-27 13:32:36,092][20347] Updated weights for policy 0, policy_version 1139397 (0.0008) [2023-12-27 13:32:36,155][20347] Updated weights for policy 0, policy_version 1139407 (0.0011) [2023-12-27 13:32:36,731][20381] Updated weights for policy 1, policy_version 1138273 (0.0008) [2023-12-27 13:32:36,780][20381] Updated weights for policy 1, policy_version 1138283 (0.0006) [2023-12-27 13:32:36,834][20381] Updated weights for policy 1, policy_version 1138293 (0.0005) [2023-12-27 13:32:36,850][20347] Updated weights for policy 0, policy_version 1139417 (0.0007) [2023-12-27 13:32:36,877][20291] KL-divergence is very high: 140.9108 [2023-12-27 13:32:36,883][20291] KL-divergence is very high: 161.9325 [2023-12-27 13:32:36,907][20347] Updated weights for policy 0, policy_version 1139427 (0.0008) [2023-12-27 13:32:36,926][20291] KL-divergence is very high: 102.4380 [2023-12-27 13:32:36,932][20291] KL-divergence is very high: 104.3761 [2023-12-27 13:32:36,964][20347] Updated weights for policy 0, policy_version 1139437 (0.0008) [2023-12-27 13:32:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 583180288. Throughput: 0: 10148.5, 1: 10206.0. Samples: 583149756. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:37,363][19170] Avg episode reward: [(0, '654.390'), (1, '628.080')] [2023-12-27 13:32:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001138296_291430400.pth... [2023-12-27 13:32:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001139440_291749888.pth... [2023-12-27 13:32:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001137112_291127296.pth [2023-12-27 13:32:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001138256_291446784.pth [2023-12-27 13:32:37,507][20381] Updated weights for policy 1, policy_version 1138303 (0.0007) [2023-12-27 13:32:37,553][20381] Updated weights for policy 1, policy_version 1138313 (0.0007) [2023-12-27 13:32:37,604][20381] Updated weights for policy 1, policy_version 1138323 (0.0008) [2023-12-27 13:32:37,670][20347] Updated weights for policy 0, policy_version 1139447 (0.0007) [2023-12-27 13:32:37,732][20347] Updated weights for policy 0, policy_version 1139457 (0.0008) [2023-12-27 13:32:37,787][20347] Updated weights for policy 0, policy_version 1139467 (0.0008) [2023-12-27 13:32:38,342][20381] Updated weights for policy 1, policy_version 1138333 (0.0008) [2023-12-27 13:32:38,398][20381] Updated weights for policy 1, policy_version 1138343 (0.0007) [2023-12-27 13:32:38,402][20347] Updated weights for policy 0, policy_version 1139477 (0.0008) [2023-12-27 13:32:38,452][20381] Updated weights for policy 1, policy_version 1138353 (0.0006) [2023-12-27 13:32:38,456][20347] Updated weights for policy 0, policy_version 1139487 (0.0008) [2023-12-27 13:32:38,510][20347] Updated weights for policy 0, policy_version 1139497 (0.0008) [2023-12-27 13:32:39,101][20381] Updated weights for policy 1, policy_version 1138363 (0.0006) [2023-12-27 13:32:39,159][20381] Updated weights for policy 1, policy_version 1138373 (0.0010) [2023-12-27 13:32:39,207][20381] Updated weights for policy 1, policy_version 1138383 (0.0010) [2023-12-27 13:32:39,244][20347] Updated weights for policy 0, policy_version 1139507 (0.0009) [2023-12-27 13:32:39,305][20347] Updated weights for policy 0, policy_version 1139517 (0.0007) [2023-12-27 13:32:39,375][20347] Updated weights for policy 0, policy_version 1139527 (0.0009) [2023-12-27 13:32:39,989][20381] Updated weights for policy 1, policy_version 1138393 (0.0008) [2023-12-27 13:32:40,051][20381] Updated weights for policy 1, policy_version 1138403 (0.0006) [2023-12-27 13:32:40,110][20381] Updated weights for policy 1, policy_version 1138413 (0.0008) [2023-12-27 13:32:40,121][20347] Updated weights for policy 0, policy_version 1139537 (0.0008) [2023-12-27 13:32:40,168][20381] Updated weights for policy 1, policy_version 1138423 (0.0007) [2023-12-27 13:32:40,188][20347] Updated weights for policy 0, policy_version 1139547 (0.0007) [2023-12-27 13:32:40,244][20347] Updated weights for policy 0, policy_version 1139557 (0.0008) [2023-12-27 13:32:40,300][20347] Updated weights for policy 0, policy_version 1139567 (0.0007) [2023-12-27 13:32:40,808][20381] Updated weights for policy 1, policy_version 1138433 (0.0007) [2023-12-27 13:32:40,861][20381] Updated weights for policy 1, policy_version 1138443 (0.0008) [2023-12-27 13:32:40,908][20381] Updated weights for policy 1, policy_version 1138453 (0.0008) [2023-12-27 13:32:40,985][20347] Updated weights for policy 0, policy_version 1139577 (0.0007) [2023-12-27 13:32:41,044][20347] Updated weights for policy 0, policy_version 1139587 (0.0007) [2023-12-27 13:32:41,100][20347] Updated weights for policy 0, policy_version 1139597 (0.0008) [2023-12-27 13:32:41,672][20381] Updated weights for policy 1, policy_version 1138463 (0.0008) [2023-12-27 13:32:41,736][20381] Updated weights for policy 1, policy_version 1138473 (0.0007) [2023-12-27 13:32:41,794][20381] Updated weights for policy 1, policy_version 1138483 (0.0008) [2023-12-27 13:32:41,839][20347] Updated weights for policy 0, policy_version 1139607 (0.0008) [2023-12-27 13:32:41,896][20347] Updated weights for policy 0, policy_version 1139617 (0.0008) [2023-12-27 13:32:41,957][20347] Updated weights for policy 0, policy_version 1139627 (0.0008) [2023-12-27 13:32:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 583278592. Throughput: 0: 10131.8, 1: 10183.8. Samples: 583271552. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:42,363][19170] Avg episode reward: [(0, '383.970'), (1, '696.520')] [2023-12-27 13:32:42,572][20381] Updated weights for policy 1, policy_version 1138493 (0.0008) [2023-12-27 13:32:42,631][20381] Updated weights for policy 1, policy_version 1138503 (0.0011) [2023-12-27 13:32:42,669][20347] Updated weights for policy 0, policy_version 1139637 (0.0009) [2023-12-27 13:32:42,690][20381] Updated weights for policy 1, policy_version 1138513 (0.0011) [2023-12-27 13:32:42,724][20347] Updated weights for policy 0, policy_version 1139647 (0.0011) [2023-12-27 13:32:42,785][20347] Updated weights for policy 0, policy_version 1139657 (0.0009) [2023-12-27 13:32:43,323][20381] Updated weights for policy 1, policy_version 1138523 (0.0010) [2023-12-27 13:32:43,381][20381] Updated weights for policy 1, policy_version 1138533 (0.0009) [2023-12-27 13:32:43,424][20381] Updated weights for policy 1, policy_version 1138543 (0.0007) [2023-12-27 13:32:43,474][20347] Updated weights for policy 0, policy_version 1139667 (0.0008) [2023-12-27 13:32:43,523][20347] Updated weights for policy 0, policy_version 1139677 (0.0008) [2023-12-27 13:32:43,571][20347] Updated weights for policy 0, policy_version 1139687 (0.0010) [2023-12-27 13:32:44,085][20381] Updated weights for policy 1, policy_version 1138553 (0.0008) [2023-12-27 13:32:44,131][20347] Updated weights for policy 0, policy_version 1139697 (0.0010) [2023-12-27 13:32:44,135][20381] Updated weights for policy 1, policy_version 1138563 (0.0009) [2023-12-27 13:32:44,184][20381] Updated weights for policy 1, policy_version 1138573 (0.0008) [2023-12-27 13:32:44,194][20347] Updated weights for policy 0, policy_version 1139707 (0.0006) [2023-12-27 13:32:44,234][20381] Updated weights for policy 1, policy_version 1138583 (0.0009) [2023-12-27 13:32:44,256][20347] Updated weights for policy 0, policy_version 1139717 (0.0006) [2023-12-27 13:32:44,309][20347] Updated weights for policy 0, policy_version 1139727 (0.0009) [2023-12-27 13:32:44,847][20347] Updated weights for policy 0, policy_version 1139737 (0.0007) [2023-12-27 13:32:44,859][20381] Updated weights for policy 1, policy_version 1138593 (0.0007) [2023-12-27 13:32:44,911][20347] Updated weights for policy 0, policy_version 1139747 (0.0006) [2023-12-27 13:32:44,920][20381] Updated weights for policy 1, policy_version 1138603 (0.0006) [2023-12-27 13:32:44,980][20347] Updated weights for policy 0, policy_version 1139757 (0.0008) [2023-12-27 13:32:44,981][20381] Updated weights for policy 1, policy_version 1138613 (0.0008) [2023-12-27 13:32:45,651][20347] Updated weights for policy 0, policy_version 1139767 (0.0006) [2023-12-27 13:32:45,695][20381] Updated weights for policy 1, policy_version 1138623 (0.0006) [2023-12-27 13:32:45,708][20347] Updated weights for policy 0, policy_version 1139777 (0.0006) [2023-12-27 13:32:45,751][20381] Updated weights for policy 1, policy_version 1138633 (0.0005) [2023-12-27 13:32:45,766][20347] Updated weights for policy 0, policy_version 1139787 (0.0006) [2023-12-27 13:32:45,802][20381] Updated weights for policy 1, policy_version 1138643 (0.0007) [2023-12-27 13:32:46,401][20381] Updated weights for policy 1, policy_version 1138653 (0.0009) [2023-12-27 13:32:46,451][20381] Updated weights for policy 1, policy_version 1138663 (0.0008) [2023-12-27 13:32:46,494][20381] Updated weights for policy 1, policy_version 1138673 (0.0005) [2023-12-27 13:32:46,497][20347] Updated weights for policy 0, policy_version 1139797 (0.0007) [2023-12-27 13:32:46,547][20347] Updated weights for policy 0, policy_version 1139807 (0.0008) [2023-12-27 13:32:46,601][20347] Updated weights for policy 0, policy_version 1139817 (0.0008) [2023-12-27 13:32:47,167][20381] Updated weights for policy 1, policy_version 1138683 (0.0006) [2023-12-27 13:32:47,221][20381] Updated weights for policy 1, policy_version 1138693 (0.0006) [2023-12-27 13:32:47,274][20381] Updated weights for policy 1, policy_version 1138703 (0.0008) [2023-12-27 13:32:47,305][20347] Updated weights for policy 0, policy_version 1139827 (0.0008) [2023-12-27 13:32:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 583385088. Throughput: 0: 10111.9, 1: 10185.5. Samples: 583395276. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:47,363][19170] Avg episode reward: [(0, '239.530'), (1, '696.590')] [2023-12-27 13:32:47,369][20347] Updated weights for policy 0, policy_version 1139837 (0.0006) [2023-12-27 13:32:47,427][20347] Updated weights for policy 0, policy_version 1139847 (0.0008) [2023-12-27 13:32:47,960][20381] Updated weights for policy 1, policy_version 1138713 (0.0006) [2023-12-27 13:32:48,009][20381] Updated weights for policy 1, policy_version 1138723 (0.0010) [2023-12-27 13:32:48,067][20381] Updated weights for policy 1, policy_version 1138733 (0.0011) [2023-12-27 13:32:48,111][20347] Updated weights for policy 0, policy_version 1139857 (0.0007) [2023-12-27 13:32:48,116][20381] Updated weights for policy 1, policy_version 1138743 (0.0010) [2023-12-27 13:32:48,162][20347] Updated weights for policy 0, policy_version 1139867 (0.0008) [2023-12-27 13:32:48,214][20347] Updated weights for policy 0, policy_version 1139877 (0.0008) [2023-12-27 13:32:48,273][20347] Updated weights for policy 0, policy_version 1139887 (0.0007) [2023-12-27 13:32:48,823][20381] Updated weights for policy 1, policy_version 1138753 (0.0008) [2023-12-27 13:32:48,874][20381] Updated weights for policy 1, policy_version 1138763 (0.0007) [2023-12-27 13:32:48,926][20381] Updated weights for policy 1, policy_version 1138773 (0.0007) [2023-12-27 13:32:48,970][20347] Updated weights for policy 0, policy_version 1139897 (0.0010) [2023-12-27 13:32:49,024][20347] Updated weights for policy 0, policy_version 1139907 (0.0010) [2023-12-27 13:32:49,070][20347] Updated weights for policy 0, policy_version 1139917 (0.0010) [2023-12-27 13:32:49,643][20381] Updated weights for policy 1, policy_version 1138783 (0.0007) [2023-12-27 13:32:49,695][20381] Updated weights for policy 1, policy_version 1138793 (0.0006) [2023-12-27 13:32:49,750][20381] Updated weights for policy 1, policy_version 1138803 (0.0007) [2023-12-27 13:32:49,792][20347] Updated weights for policy 0, policy_version 1139927 (0.0010) [2023-12-27 13:32:49,850][20347] Updated weights for policy 0, policy_version 1139937 (0.0011) [2023-12-27 13:32:49,914][20347] Updated weights for policy 0, policy_version 1139947 (0.0010) [2023-12-27 13:32:50,499][20381] Updated weights for policy 1, policy_version 1138813 (0.0007) [2023-12-27 13:32:50,544][20381] Updated weights for policy 1, policy_version 1138823 (0.0007) [2023-12-27 13:32:50,602][20381] Updated weights for policy 1, policy_version 1138833 (0.0009) [2023-12-27 13:32:50,693][20347] Updated weights for policy 0, policy_version 1139957 (0.0007) [2023-12-27 13:32:50,759][20347] Updated weights for policy 0, policy_version 1139967 (0.0006) [2023-12-27 13:32:50,819][20347] Updated weights for policy 0, policy_version 1139977 (0.0009) [2023-12-27 13:32:51,254][20381] Updated weights for policy 1, policy_version 1138843 (0.0008) [2023-12-27 13:32:51,311][20381] Updated weights for policy 1, policy_version 1138853 (0.0007) [2023-12-27 13:32:51,376][20381] Updated weights for policy 1, policy_version 1138863 (0.0009) [2023-12-27 13:32:51,491][20347] Updated weights for policy 0, policy_version 1139987 (0.0008) [2023-12-27 13:32:51,550][20347] Updated weights for policy 0, policy_version 1139997 (0.0008) [2023-12-27 13:32:51,610][20347] Updated weights for policy 0, policy_version 1140007 (0.0008) [2023-12-27 13:32:52,133][20381] Updated weights for policy 1, policy_version 1138873 (0.0008) [2023-12-27 13:32:52,182][20381] Updated weights for policy 1, policy_version 1138883 (0.0007) [2023-12-27 13:32:52,232][20381] Updated weights for policy 1, policy_version 1138893 (0.0008) [2023-12-27 13:32:52,277][20347] Updated weights for policy 0, policy_version 1140017 (0.0008) [2023-12-27 13:32:52,296][20381] Updated weights for policy 1, policy_version 1138903 (0.0009) [2023-12-27 13:32:52,333][20347] Updated weights for policy 0, policy_version 1140027 (0.0009) [2023-12-27 13:32:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 583483392. Throughput: 0: 10093.0, 1: 10157.1. Samples: 583454592. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:52,363][19170] Avg episode reward: [(0, '237.530'), (1, '608.390')] [2023-12-27 13:32:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001138904_291586048.pth... [2023-12-27 13:32:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001137688_291274752.pth [2023-12-27 13:32:52,397][20347] Updated weights for policy 0, policy_version 1140037 (0.0011) [2023-12-27 13:32:52,447][20347] Updated weights for policy 0, policy_version 1140047 (0.0010) [2023-12-27 13:32:52,450][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001140048_291905536.pth... [2023-12-27 13:32:52,453][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001138832_291594240.pth [2023-12-27 13:32:52,988][20381] Updated weights for policy 1, policy_version 1138913 (0.0007) [2023-12-27 13:32:53,047][20381] Updated weights for policy 1, policy_version 1138923 (0.0006) [2023-12-27 13:32:53,105][20381] Updated weights for policy 1, policy_version 1138933 (0.0007) [2023-12-27 13:32:53,171][20347] Updated weights for policy 0, policy_version 1140057 (0.0010) [2023-12-27 13:32:53,229][20347] Updated weights for policy 0, policy_version 1140067 (0.0010) [2023-12-27 13:32:53,289][20347] Updated weights for policy 0, policy_version 1140077 (0.0010) [2023-12-27 13:32:53,651][20381] Updated weights for policy 1, policy_version 1138943 (0.0007) [2023-12-27 13:32:53,698][20381] Updated weights for policy 1, policy_version 1138953 (0.0007) [2023-12-27 13:32:53,755][20381] Updated weights for policy 1, policy_version 1138963 (0.0006) [2023-12-27 13:32:53,962][20347] Updated weights for policy 0, policy_version 1140087 (0.0009) [2023-12-27 13:32:54,013][20347] Updated weights for policy 0, policy_version 1140097 (0.0010) [2023-12-27 13:32:54,064][20347] Updated weights for policy 0, policy_version 1140107 (0.0010) [2023-12-27 13:32:54,402][20381] Updated weights for policy 1, policy_version 1138973 (0.0006) [2023-12-27 13:32:54,453][20381] Updated weights for policy 1, policy_version 1138983 (0.0005) [2023-12-27 13:32:54,510][20381] Updated weights for policy 1, policy_version 1138993 (0.0006) [2023-12-27 13:32:54,754][20347] Updated weights for policy 0, policy_version 1140117 (0.0010) [2023-12-27 13:32:54,814][20347] Updated weights for policy 0, policy_version 1140127 (0.0010) [2023-12-27 13:32:54,865][20347] Updated weights for policy 0, policy_version 1140137 (0.0010) [2023-12-27 13:32:55,115][20381] Updated weights for policy 1, policy_version 1139003 (0.0007) [2023-12-27 13:32:55,168][20381] Updated weights for policy 1, policy_version 1139013 (0.0006) [2023-12-27 13:32:55,219][20381] Updated weights for policy 1, policy_version 1139023 (0.0005) [2023-12-27 13:32:55,542][20347] Updated weights for policy 0, policy_version 1140147 (0.0009) [2023-12-27 13:32:55,589][20347] Updated weights for policy 0, policy_version 1140157 (0.0005) [2023-12-27 13:32:55,638][20347] Updated weights for policy 0, policy_version 1140167 (0.0005) [2023-12-27 13:32:55,815][20381] Updated weights for policy 1, policy_version 1139033 (0.0005) [2023-12-27 13:32:55,865][20381] Updated weights for policy 1, policy_version 1139043 (0.0005) [2023-12-27 13:32:55,932][20381] Updated weights for policy 1, policy_version 1139053 (0.0007) [2023-12-27 13:32:55,977][20381] Updated weights for policy 1, policy_version 1139063 (0.0010) [2023-12-27 13:32:56,283][20347] Updated weights for policy 0, policy_version 1140177 (0.0010) [2023-12-27 13:32:56,341][20347] Updated weights for policy 0, policy_version 1140187 (0.0008) [2023-12-27 13:32:56,392][20347] Updated weights for policy 0, policy_version 1140197 (0.0007) [2023-12-27 13:32:56,452][20347] Updated weights for policy 0, policy_version 1140207 (0.0008) [2023-12-27 13:32:56,536][20381] Updated weights for policy 1, policy_version 1139073 (0.0010) [2023-12-27 13:32:56,591][20381] Updated weights for policy 1, policy_version 1139083 (0.0009) [2023-12-27 13:32:56,644][20381] Updated weights for policy 1, policy_version 1139093 (0.0009) [2023-12-27 13:32:57,147][20347] Updated weights for policy 0, policy_version 1140217 (0.0010) [2023-12-27 13:32:57,211][20347] Updated weights for policy 0, policy_version 1140227 (0.0010) [2023-12-27 13:32:57,261][20347] Updated weights for policy 0, policy_version 1140237 (0.0010) [2023-12-27 13:32:57,284][20381] Updated weights for policy 1, policy_version 1139103 (0.0007) [2023-12-27 13:32:57,338][20381] Updated weights for policy 1, policy_version 1139113 (0.0007) [2023-12-27 13:32:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 583589888. Throughput: 0: 10181.2, 1: 10216.8. Samples: 583580212. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:32:57,363][19170] Avg episode reward: [(0, '346.040'), (1, '608.680')] [2023-12-27 13:32:57,393][20381] Updated weights for policy 1, policy_version 1139123 (0.0005) [2023-12-27 13:32:57,873][20347] Updated weights for policy 0, policy_version 1140247 (0.0007) [2023-12-27 13:32:57,921][20347] Updated weights for policy 0, policy_version 1140257 (0.0005) [2023-12-27 13:32:57,966][20347] Updated weights for policy 0, policy_version 1140267 (0.0005) [2023-12-27 13:32:58,044][20381] Updated weights for policy 1, policy_version 1139133 (0.0007) [2023-12-27 13:32:58,100][20381] Updated weights for policy 1, policy_version 1139143 (0.0005) [2023-12-27 13:32:58,152][20381] Updated weights for policy 1, policy_version 1139153 (0.0005) [2023-12-27 13:32:58,671][20347] Updated weights for policy 0, policy_version 1140277 (0.0006) [2023-12-27 13:32:58,753][20347] Updated weights for policy 0, policy_version 1140287 (0.0007) [2023-12-27 13:32:58,821][20347] Updated weights for policy 0, policy_version 1140297 (0.0008) [2023-12-27 13:32:58,911][20381] Updated weights for policy 1, policy_version 1139163 (0.0007) [2023-12-27 13:32:58,976][20381] Updated weights for policy 1, policy_version 1139173 (0.0008) [2023-12-27 13:32:59,029][20381] Updated weights for policy 1, policy_version 1139183 (0.0008) [2023-12-27 13:32:59,520][20347] Updated weights for policy 0, policy_version 1140307 (0.0007) [2023-12-27 13:32:59,579][20347] Updated weights for policy 0, policy_version 1140317 (0.0005) [2023-12-27 13:32:59,638][20347] Updated weights for policy 0, policy_version 1140327 (0.0008) [2023-12-27 13:32:59,692][20381] Updated weights for policy 1, policy_version 1139193 (0.0008) [2023-12-27 13:32:59,758][20381] Updated weights for policy 1, policy_version 1139203 (0.0005) [2023-12-27 13:32:59,821][20381] Updated weights for policy 1, policy_version 1139213 (0.0008) [2023-12-27 13:32:59,880][20381] Updated weights for policy 1, policy_version 1139223 (0.0011) [2023-12-27 13:33:00,310][20347] Updated weights for policy 0, policy_version 1140337 (0.0008) [2023-12-27 13:33:00,365][20347] Updated weights for policy 0, policy_version 1140347 (0.0007) [2023-12-27 13:33:00,414][20347] Updated weights for policy 0, policy_version 1140357 (0.0007) [2023-12-27 13:33:00,469][20347] Updated weights for policy 0, policy_version 1140367 (0.0007) [2023-12-27 13:33:00,549][20381] Updated weights for policy 1, policy_version 1139233 (0.0010) [2023-12-27 13:33:00,598][20381] Updated weights for policy 1, policy_version 1139243 (0.0010) [2023-12-27 13:33:00,649][20381] Updated weights for policy 1, policy_version 1139253 (0.0006) [2023-12-27 13:33:01,110][20347] Updated weights for policy 0, policy_version 1140377 (0.0008) [2023-12-27 13:33:01,169][20347] Updated weights for policy 0, policy_version 1140387 (0.0008) [2023-12-27 13:33:01,237][20347] Updated weights for policy 0, policy_version 1140397 (0.0009) [2023-12-27 13:33:01,305][20381] Updated weights for policy 1, policy_version 1139263 (0.0007) [2023-12-27 13:33:01,368][20381] Updated weights for policy 1, policy_version 1139273 (0.0008) [2023-12-27 13:33:01,431][20381] Updated weights for policy 1, policy_version 1139283 (0.0007) [2023-12-27 13:33:01,979][20347] Updated weights for policy 0, policy_version 1140407 (0.0008) [2023-12-27 13:33:02,031][20347] Updated weights for policy 0, policy_version 1140417 (0.0008) [2023-12-27 13:33:02,083][20347] Updated weights for policy 0, policy_version 1140427 (0.0007) [2023-12-27 13:33:02,096][20381] Updated weights for policy 1, policy_version 1139293 (0.0006) [2023-12-27 13:33:02,147][20381] Updated weights for policy 1, policy_version 1139303 (0.0007) [2023-12-27 13:33:02,207][20381] Updated weights for policy 1, policy_version 1139313 (0.0008) [2023-12-27 13:33:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 583696384. Throughput: 0: 10217.2, 1: 10259.5. Samples: 583703136. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:02,363][19170] Avg episode reward: [(0, '341.860'), (1, '588.520')] [2023-12-27 13:33:02,770][20347] Updated weights for policy 0, policy_version 1140437 (0.0007) [2023-12-27 13:33:02,831][20347] Updated weights for policy 0, policy_version 1140447 (0.0008) [2023-12-27 13:33:02,880][20347] Updated weights for policy 0, policy_version 1140457 (0.0007) [2023-12-27 13:33:02,936][20381] Updated weights for policy 1, policy_version 1139323 (0.0007) [2023-12-27 13:33:02,988][20381] Updated weights for policy 1, policy_version 1139333 (0.0008) [2023-12-27 13:33:02,989][20292] KL-divergence is very high: 154.4411 [2023-12-27 13:33:02,998][20292] KL-divergence is very high: 141.0618 [2023-12-27 13:33:03,042][20381] Updated weights for policy 1, policy_version 1139343 (0.0008) [2023-12-27 13:33:03,560][20347] Updated weights for policy 0, policy_version 1140467 (0.0007) [2023-12-27 13:33:03,609][20347] Updated weights for policy 0, policy_version 1140477 (0.0008) [2023-12-27 13:33:03,663][20347] Updated weights for policy 0, policy_version 1140487 (0.0008) [2023-12-27 13:33:03,737][20381] Updated weights for policy 1, policy_version 1139353 (0.0007) [2023-12-27 13:33:03,786][20381] Updated weights for policy 1, policy_version 1139363 (0.0008) [2023-12-27 13:33:03,832][20381] Updated weights for policy 1, policy_version 1139373 (0.0007) [2023-12-27 13:33:03,896][20381] Updated weights for policy 1, policy_version 1139383 (0.0008) [2023-12-27 13:33:04,418][20347] Updated weights for policy 0, policy_version 1140497 (0.0008) [2023-12-27 13:33:04,477][20347] Updated weights for policy 0, policy_version 1140507 (0.0007) [2023-12-27 13:33:04,538][20347] Updated weights for policy 0, policy_version 1140517 (0.0008) [2023-12-27 13:33:04,591][20381] Updated weights for policy 1, policy_version 1139393 (0.0005) [2023-12-27 13:33:04,596][20347] Updated weights for policy 0, policy_version 1140527 (0.0007) [2023-12-27 13:33:04,641][20381] Updated weights for policy 1, policy_version 1139403 (0.0007) [2023-12-27 13:33:04,692][20381] Updated weights for policy 1, policy_version 1139413 (0.0007) [2023-12-27 13:33:05,239][20381] Updated weights for policy 1, policy_version 1139423 (0.0006) [2023-12-27 13:33:05,292][20381] Updated weights for policy 1, policy_version 1139433 (0.0007) [2023-12-27 13:33:05,314][20347] Updated weights for policy 0, policy_version 1140537 (0.0008) [2023-12-27 13:33:05,348][20381] Updated weights for policy 1, policy_version 1139443 (0.0007) [2023-12-27 13:33:05,379][20347] Updated weights for policy 0, policy_version 1140547 (0.0008) [2023-12-27 13:33:05,434][20347] Updated weights for policy 0, policy_version 1140557 (0.0008) [2023-12-27 13:33:05,961][20381] Updated weights for policy 1, policy_version 1139453 (0.0009) [2023-12-27 13:33:06,017][20381] Updated weights for policy 1, policy_version 1139463 (0.0008) [2023-12-27 13:33:06,065][20347] Updated weights for policy 0, policy_version 1140567 (0.0006) [2023-12-27 13:33:06,070][20381] Updated weights for policy 1, policy_version 1139473 (0.0006) [2023-12-27 13:33:06,124][20347] Updated weights for policy 0, policy_version 1140577 (0.0007) [2023-12-27 13:33:06,184][20347] Updated weights for policy 0, policy_version 1140587 (0.0009) [2023-12-27 13:33:06,758][20381] Updated weights for policy 1, policy_version 1139483 (0.0008) [2023-12-27 13:33:06,803][20381] Updated weights for policy 1, policy_version 1139493 (0.0007) [2023-12-27 13:33:06,858][20381] Updated weights for policy 1, policy_version 1139503 (0.0007) [2023-12-27 13:33:06,898][20347] Updated weights for policy 0, policy_version 1140597 (0.0011) [2023-12-27 13:33:06,947][20347] Updated weights for policy 0, policy_version 1140607 (0.0010) [2023-12-27 13:33:07,003][20347] Updated weights for policy 0, policy_version 1140617 (0.0010) [2023-12-27 13:33:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.6, 300 sec: 20493.9). Total num frames: 583794688. Throughput: 0: 10174.8, 1: 10322.2. Samples: 583764512. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:07,363][19170] Avg episode reward: [(0, '496.960'), (1, '637.610')] [2023-12-27 13:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001139512_291741696.pth... [2023-12-27 13:33:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001140624_292052992.pth... [2023-12-27 13:33:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001138296_291430400.pth [2023-12-27 13:33:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001139440_291749888.pth [2023-12-27 13:33:07,566][20381] Updated weights for policy 1, policy_version 1139513 (0.0007) [2023-12-27 13:33:07,622][20381] Updated weights for policy 1, policy_version 1139523 (0.0010) [2023-12-27 13:33:07,676][20381] Updated weights for policy 1, policy_version 1139533 (0.0010) [2023-12-27 13:33:07,705][20347] Updated weights for policy 0, policy_version 1140627 (0.0010) [2023-12-27 13:33:07,734][20381] Updated weights for policy 1, policy_version 1139543 (0.0010) [2023-12-27 13:33:07,760][20347] Updated weights for policy 0, policy_version 1140637 (0.0008) [2023-12-27 13:33:07,813][20347] Updated weights for policy 0, policy_version 1140647 (0.0005) [2023-12-27 13:33:08,387][20381] Updated weights for policy 1, policy_version 1139553 (0.0008) [2023-12-27 13:33:08,417][20347] Updated weights for policy 0, policy_version 1140657 (0.0006) [2023-12-27 13:33:08,442][20381] Updated weights for policy 1, policy_version 1139563 (0.0007) [2023-12-27 13:33:08,469][20347] Updated weights for policy 0, policy_version 1140667 (0.0010) [2023-12-27 13:33:08,490][20381] Updated weights for policy 1, policy_version 1139573 (0.0007) [2023-12-27 13:33:08,527][20347] Updated weights for policy 0, policy_version 1140677 (0.0010) [2023-12-27 13:33:08,583][20347] Updated weights for policy 0, policy_version 1140687 (0.0010) [2023-12-27 13:33:09,180][20381] Updated weights for policy 1, policy_version 1139583 (0.0010) [2023-12-27 13:33:09,233][20347] Updated weights for policy 0, policy_version 1140697 (0.0008) [2023-12-27 13:33:09,241][20381] Updated weights for policy 1, policy_version 1139593 (0.0010) [2023-12-27 13:33:09,291][20347] Updated weights for policy 0, policy_version 1140707 (0.0008) [2023-12-27 13:33:09,299][20381] Updated weights for policy 1, policy_version 1139603 (0.0008) [2023-12-27 13:33:09,350][20347] Updated weights for policy 0, policy_version 1140717 (0.0008) [2023-12-27 13:33:09,987][20381] Updated weights for policy 1, policy_version 1139613 (0.0008) [2023-12-27 13:33:10,045][20381] Updated weights for policy 1, policy_version 1139623 (0.0008) [2023-12-27 13:33:10,077][20347] Updated weights for policy 0, policy_version 1140727 (0.0006) [2023-12-27 13:33:10,102][20381] Updated weights for policy 1, policy_version 1139633 (0.0006) [2023-12-27 13:33:10,130][20347] Updated weights for policy 0, policy_version 1140737 (0.0005) [2023-12-27 13:33:10,181][20347] Updated weights for policy 0, policy_version 1140747 (0.0007) [2023-12-27 13:33:10,834][20381] Updated weights for policy 1, policy_version 1139643 (0.0007) [2023-12-27 13:33:10,889][20381] Updated weights for policy 1, policy_version 1139653 (0.0007) [2023-12-27 13:33:10,893][20347] Updated weights for policy 0, policy_version 1140757 (0.0007) [2023-12-27 13:33:10,942][20347] Updated weights for policy 0, policy_version 1140767 (0.0005) [2023-12-27 13:33:10,951][20381] Updated weights for policy 1, policy_version 1139663 (0.0008) [2023-12-27 13:33:10,992][20347] Updated weights for policy 0, policy_version 1140777 (0.0008) [2023-12-27 13:33:11,660][20381] Updated weights for policy 1, policy_version 1139673 (0.0008) [2023-12-27 13:33:11,714][20347] Updated weights for policy 0, policy_version 1140787 (0.0007) [2023-12-27 13:33:11,725][20381] Updated weights for policy 1, policy_version 1139683 (0.0007) [2023-12-27 13:33:11,777][20347] Updated weights for policy 0, policy_version 1140797 (0.0008) [2023-12-27 13:33:11,791][20381] Updated weights for policy 1, policy_version 1139693 (0.0009) [2023-12-27 13:33:11,828][20347] Updated weights for policy 0, policy_version 1140807 (0.0006) [2023-12-27 13:33:11,850][20381] Updated weights for policy 1, policy_version 1139703 (0.0006) [2023-12-27 13:33:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 583892992. Throughput: 0: 10142.5, 1: 10366.1. Samples: 583886096. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:12,363][19170] Avg episode reward: [(0, '358.530'), (1, '666.650')] [2023-12-27 13:33:12,533][20381] Updated weights for policy 1, policy_version 1139713 (0.0008) [2023-12-27 13:33:12,580][20347] Updated weights for policy 0, policy_version 1140817 (0.0008) [2023-12-27 13:33:12,585][20381] Updated weights for policy 1, policy_version 1139723 (0.0009) [2023-12-27 13:33:12,637][20347] Updated weights for policy 0, policy_version 1140827 (0.0006) [2023-12-27 13:33:12,642][20381] Updated weights for policy 1, policy_version 1139733 (0.0007) [2023-12-27 13:33:12,695][20347] Updated weights for policy 0, policy_version 1140837 (0.0006) [2023-12-27 13:33:12,757][20347] Updated weights for policy 0, policy_version 1140847 (0.0006) [2023-12-27 13:33:13,376][20381] Updated weights for policy 1, policy_version 1139743 (0.0006) [2023-12-27 13:33:13,431][20347] Updated weights for policy 0, policy_version 1140857 (0.0008) [2023-12-27 13:33:13,433][20381] Updated weights for policy 1, policy_version 1139753 (0.0006) [2023-12-27 13:33:13,482][20347] Updated weights for policy 0, policy_version 1140867 (0.0005) [2023-12-27 13:33:13,484][20381] Updated weights for policy 1, policy_version 1139763 (0.0007) [2023-12-27 13:33:13,540][20347] Updated weights for policy 0, policy_version 1140877 (0.0007) [2023-12-27 13:33:14,144][20381] Updated weights for policy 1, policy_version 1139773 (0.0009) [2023-12-27 13:33:14,203][20381] Updated weights for policy 1, policy_version 1139783 (0.0011) [2023-12-27 13:33:14,228][20347] Updated weights for policy 0, policy_version 1140887 (0.0007) [2023-12-27 13:33:14,250][20381] Updated weights for policy 1, policy_version 1139793 (0.0010) [2023-12-27 13:33:14,278][20347] Updated weights for policy 0, policy_version 1140897 (0.0008) [2023-12-27 13:33:14,329][20347] Updated weights for policy 0, policy_version 1140907 (0.0007) [2023-12-27 13:33:14,893][20381] Updated weights for policy 1, policy_version 1139803 (0.0010) [2023-12-27 13:33:14,953][20381] Updated weights for policy 1, policy_version 1139813 (0.0011) [2023-12-27 13:33:14,984][20347] Updated weights for policy 0, policy_version 1140917 (0.0007) [2023-12-27 13:33:15,007][20381] Updated weights for policy 1, policy_version 1139823 (0.0011) [2023-12-27 13:33:15,046][20347] Updated weights for policy 0, policy_version 1140927 (0.0008) [2023-12-27 13:33:15,099][20347] Updated weights for policy 0, policy_version 1140937 (0.0006) [2023-12-27 13:33:15,701][20381] Updated weights for policy 1, policy_version 1139833 (0.0010) [2023-12-27 13:33:15,756][20381] Updated weights for policy 1, policy_version 1139843 (0.0008) [2023-12-27 13:33:15,781][20347] Updated weights for policy 0, policy_version 1140947 (0.0009) [2023-12-27 13:33:15,811][20381] Updated weights for policy 1, policy_version 1139853 (0.0007) [2023-12-27 13:33:15,844][20347] Updated weights for policy 0, policy_version 1140957 (0.0006) [2023-12-27 13:33:15,868][20381] Updated weights for policy 1, policy_version 1139863 (0.0006) [2023-12-27 13:33:15,908][20347] Updated weights for policy 0, policy_version 1140967 (0.0006) [2023-12-27 13:33:16,544][20347] Updated weights for policy 0, policy_version 1140977 (0.0006) [2023-12-27 13:33:16,576][20381] Updated weights for policy 1, policy_version 1139873 (0.0007) [2023-12-27 13:33:16,600][20347] Updated weights for policy 0, policy_version 1140987 (0.0006) [2023-12-27 13:33:16,628][20381] Updated weights for policy 1, policy_version 1139883 (0.0006) [2023-12-27 13:33:16,655][20347] Updated weights for policy 0, policy_version 1140997 (0.0006) [2023-12-27 13:33:16,680][20381] Updated weights for policy 1, policy_version 1139893 (0.0005) [2023-12-27 13:33:16,713][20347] Updated weights for policy 0, policy_version 1141007 (0.0007) [2023-12-27 13:33:17,357][20381] Updated weights for policy 1, policy_version 1139903 (0.0008) [2023-12-27 13:33:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 583991296. Throughput: 0: 10100.7, 1: 10310.0. Samples: 584007360. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:17,363][19170] Avg episode reward: [(0, '431.780'), (1, '715.770')] [2023-12-27 13:33:17,403][20381] Updated weights for policy 1, policy_version 1139913 (0.0006) [2023-12-27 13:33:17,410][20347] Updated weights for policy 0, policy_version 1141017 (0.0010) [2023-12-27 13:33:17,453][20381] Updated weights for policy 1, policy_version 1139923 (0.0007) [2023-12-27 13:33:17,461][20347] Updated weights for policy 0, policy_version 1141027 (0.0009) [2023-12-27 13:33:17,524][20347] Updated weights for policy 0, policy_version 1141037 (0.0005) [2023-12-27 13:33:18,085][20381] Updated weights for policy 1, policy_version 1139933 (0.0006) [2023-12-27 13:33:18,140][20381] Updated weights for policy 1, policy_version 1139943 (0.0007) [2023-12-27 13:33:18,145][20347] Updated weights for policy 0, policy_version 1141047 (0.0005) [2023-12-27 13:33:18,191][20347] Updated weights for policy 0, policy_version 1141057 (0.0005) [2023-12-27 13:33:18,196][20381] Updated weights for policy 1, policy_version 1139953 (0.0006) [2023-12-27 13:33:18,237][20347] Updated weights for policy 0, policy_version 1141067 (0.0006) [2023-12-27 13:33:18,847][20347] Updated weights for policy 0, policy_version 1141077 (0.0006) [2023-12-27 13:33:18,909][20347] Updated weights for policy 0, policy_version 1141087 (0.0008) [2023-12-27 13:33:18,937][20381] Updated weights for policy 1, policy_version 1139963 (0.0007) [2023-12-27 13:33:18,967][20347] Updated weights for policy 0, policy_version 1141097 (0.0008) [2023-12-27 13:33:18,982][20381] Updated weights for policy 1, policy_version 1139973 (0.0006) [2023-12-27 13:33:19,034][20381] Updated weights for policy 1, policy_version 1139983 (0.0007) [2023-12-27 13:33:19,699][20347] Updated weights for policy 0, policy_version 1141107 (0.0008) [2023-12-27 13:33:19,765][20347] Updated weights for policy 0, policy_version 1141117 (0.0008) [2023-12-27 13:33:19,782][20381] Updated weights for policy 1, policy_version 1139993 (0.0007) [2023-12-27 13:33:19,837][20347] Updated weights for policy 0, policy_version 1141127 (0.0007) [2023-12-27 13:33:19,885][20381] Updated weights for policy 1, policy_version 1140003 (0.0007) [2023-12-27 13:33:19,951][20381] Updated weights for policy 1, policy_version 1140013 (0.0008) [2023-12-27 13:33:20,007][20381] Updated weights for policy 1, policy_version 1140023 (0.0007) [2023-12-27 13:33:20,448][20347] Updated weights for policy 0, policy_version 1141137 (0.0007) [2023-12-27 13:33:20,501][20347] Updated weights for policy 0, policy_version 1141147 (0.0007) [2023-12-27 13:33:20,566][20347] Updated weights for policy 0, policy_version 1141157 (0.0008) [2023-12-27 13:33:20,628][20347] Updated weights for policy 0, policy_version 1141167 (0.0007) [2023-12-27 13:33:20,682][20381] Updated weights for policy 1, policy_version 1140033 (0.0008) [2023-12-27 13:33:20,736][20381] Updated weights for policy 1, policy_version 1140043 (0.0007) [2023-12-27 13:33:20,795][20381] Updated weights for policy 1, policy_version 1140053 (0.0008) [2023-12-27 13:33:21,353][20347] Updated weights for policy 0, policy_version 1141177 (0.0007) [2023-12-27 13:33:21,420][20347] Updated weights for policy 0, policy_version 1141187 (0.0008) [2023-12-27 13:33:21,473][20347] Updated weights for policy 0, policy_version 1141197 (0.0008) [2023-12-27 13:33:21,530][20381] Updated weights for policy 1, policy_version 1140063 (0.0007) [2023-12-27 13:33:21,585][20381] Updated weights for policy 1, policy_version 1140073 (0.0009) [2023-12-27 13:33:21,648][20381] Updated weights for policy 1, policy_version 1140083 (0.0008) [2023-12-27 13:33:22,235][20347] Updated weights for policy 0, policy_version 1141207 (0.0006) [2023-12-27 13:33:22,305][20347] Updated weights for policy 0, policy_version 1141217 (0.0010) [2023-12-27 13:33:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 584089600. Throughput: 0: 10143.1, 1: 10272.6. Samples: 584068464. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:22,363][19170] Avg episode reward: [(0, '204.050'), (1, '646.880')] [2023-12-27 13:33:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001140088_291889152.pth... [2023-12-27 13:33:22,371][20347] Updated weights for policy 0, policy_version 1141227 (0.0011) [2023-12-27 13:33:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001138904_291586048.pth [2023-12-27 13:33:22,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001141232_292208640.pth... [2023-12-27 13:33:22,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001140048_291905536.pth [2023-12-27 13:33:22,419][20381] Updated weights for policy 1, policy_version 1140093 (0.0010) [2023-12-27 13:33:22,466][20381] Updated weights for policy 1, policy_version 1140103 (0.0011) [2023-12-27 13:33:22,515][20381] Updated weights for policy 1, policy_version 1140113 (0.0010) [2023-12-27 13:33:23,026][20347] Updated weights for policy 0, policy_version 1141237 (0.0007) [2023-12-27 13:33:23,087][20347] Updated weights for policy 0, policy_version 1141247 (0.0006) [2023-12-27 13:33:23,143][20347] Updated weights for policy 0, policy_version 1141257 (0.0005) [2023-12-27 13:33:23,226][20381] Updated weights for policy 1, policy_version 1140123 (0.0009) [2023-12-27 13:33:23,280][20381] Updated weights for policy 1, policy_version 1140133 (0.0007) [2023-12-27 13:33:23,333][20381] Updated weights for policy 1, policy_version 1140143 (0.0007) [2023-12-27 13:33:23,696][20347] Updated weights for policy 0, policy_version 1141267 (0.0005) [2023-12-27 13:33:23,747][20347] Updated weights for policy 0, policy_version 1141277 (0.0005) [2023-12-27 13:33:23,813][20347] Updated weights for policy 0, policy_version 1141287 (0.0005) [2023-12-27 13:33:23,907][20381] Updated weights for policy 1, policy_version 1140153 (0.0007) [2023-12-27 13:33:23,969][20381] Updated weights for policy 1, policy_version 1140163 (0.0007) [2023-12-27 13:33:24,029][20381] Updated weights for policy 1, policy_version 1140173 (0.0008) [2023-12-27 13:33:24,100][20381] Updated weights for policy 1, policy_version 1140183 (0.0008) [2023-12-27 13:33:24,484][20347] Updated weights for policy 0, policy_version 1141297 (0.0009) [2023-12-27 13:33:24,543][20347] Updated weights for policy 0, policy_version 1141307 (0.0008) [2023-12-27 13:33:24,597][20347] Updated weights for policy 0, policy_version 1141317 (0.0007) [2023-12-27 13:33:24,638][20381] Updated weights for policy 1, policy_version 1140193 (0.0007) [2023-12-27 13:33:24,659][20347] Updated weights for policy 0, policy_version 1141327 (0.0007) [2023-12-27 13:33:24,695][20381] Updated weights for policy 1, policy_version 1140203 (0.0007) [2023-12-27 13:33:24,754][20381] Updated weights for policy 1, policy_version 1140213 (0.0006) [2023-12-27 13:33:25,375][20381] Updated weights for policy 1, policy_version 1140223 (0.0009) [2023-12-27 13:33:25,377][20347] Updated weights for policy 0, policy_version 1141337 (0.0006) [2023-12-27 13:33:25,420][20381] Updated weights for policy 1, policy_version 1140233 (0.0010) [2023-12-27 13:33:25,432][20347] Updated weights for policy 0, policy_version 1141347 (0.0005) [2023-12-27 13:33:25,469][20381] Updated weights for policy 1, policy_version 1140243 (0.0009) [2023-12-27 13:33:25,485][20347] Updated weights for policy 0, policy_version 1141357 (0.0005) [2023-12-27 13:33:26,104][20381] Updated weights for policy 1, policy_version 1140253 (0.0008) [2023-12-27 13:33:26,151][20381] Updated weights for policy 1, policy_version 1140263 (0.0005) [2023-12-27 13:33:26,204][20347] Updated weights for policy 0, policy_version 1141367 (0.0006) [2023-12-27 13:33:26,210][20381] Updated weights for policy 1, policy_version 1140273 (0.0005) [2023-12-27 13:33:26,259][20347] Updated weights for policy 0, policy_version 1141377 (0.0005) [2023-12-27 13:33:26,310][20347] Updated weights for policy 0, policy_version 1141387 (0.0008) [2023-12-27 13:33:26,790][20381] Updated weights for policy 1, policy_version 1140283 (0.0006) [2023-12-27 13:33:26,842][20381] Updated weights for policy 1, policy_version 1140293 (0.0007) [2023-12-27 13:33:26,901][20381] Updated weights for policy 1, policy_version 1140303 (0.0007) [2023-12-27 13:33:26,977][20347] Updated weights for policy 0, policy_version 1141397 (0.0008) [2023-12-27 13:33:27,036][20347] Updated weights for policy 0, policy_version 1141407 (0.0007) [2023-12-27 13:33:27,087][20347] Updated weights for policy 0, policy_version 1141417 (0.0005) [2023-12-27 13:33:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 584204288. Throughput: 0: 10141.6, 1: 10329.0. Samples: 584192728. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:27,363][19170] Avg episode reward: [(0, '240.190'), (1, '647.290')] [2023-12-27 13:33:27,530][20381] Updated weights for policy 1, policy_version 1140313 (0.0008) [2023-12-27 13:33:27,573][20381] Updated weights for policy 1, policy_version 1140323 (0.0008) [2023-12-27 13:33:27,618][20381] Updated weights for policy 1, policy_version 1140333 (0.0007) [2023-12-27 13:33:27,660][20381] Updated weights for policy 1, policy_version 1140343 (0.0007) [2023-12-27 13:33:27,738][20347] Updated weights for policy 0, policy_version 1141427 (0.0008) [2023-12-27 13:33:27,792][20347] Updated weights for policy 0, policy_version 1141437 (0.0007) [2023-12-27 13:33:27,841][20347] Updated weights for policy 0, policy_version 1141447 (0.0008) [2023-12-27 13:33:28,335][20381] Updated weights for policy 1, policy_version 1140353 (0.0008) [2023-12-27 13:33:28,386][20381] Updated weights for policy 1, policy_version 1140363 (0.0006) [2023-12-27 13:33:28,444][20381] Updated weights for policy 1, policy_version 1140373 (0.0008) [2023-12-27 13:33:28,462][20347] Updated weights for policy 0, policy_version 1141457 (0.0007) [2023-12-27 13:33:28,507][20347] Updated weights for policy 0, policy_version 1141467 (0.0007) [2023-12-27 13:33:28,565][20347] Updated weights for policy 0, policy_version 1141477 (0.0007) [2023-12-27 13:33:28,624][20347] Updated weights for policy 0, policy_version 1141487 (0.0008) [2023-12-27 13:33:29,146][20381] Updated weights for policy 1, policy_version 1140383 (0.0008) [2023-12-27 13:33:29,199][20381] Updated weights for policy 1, policy_version 1140393 (0.0007) [2023-12-27 13:33:29,264][20381] Updated weights for policy 1, policy_version 1140403 (0.0007) [2023-12-27 13:33:29,264][20347] Updated weights for policy 0, policy_version 1141497 (0.0008) [2023-12-27 13:33:29,317][20347] Updated weights for policy 0, policy_version 1141507 (0.0005) [2023-12-27 13:33:29,376][20347] Updated weights for policy 0, policy_version 1141517 (0.0010) [2023-12-27 13:33:29,906][20381] Updated weights for policy 1, policy_version 1140413 (0.0006) [2023-12-27 13:33:29,969][20347] Updated weights for policy 0, policy_version 1141527 (0.0010) [2023-12-27 13:33:29,972][20381] Updated weights for policy 1, policy_version 1140423 (0.0008) [2023-12-27 13:33:30,026][20347] Updated weights for policy 0, policy_version 1141537 (0.0006) [2023-12-27 13:33:30,030][20381] Updated weights for policy 1, policy_version 1140433 (0.0008) [2023-12-27 13:33:30,087][20347] Updated weights for policy 0, policy_version 1141547 (0.0006) [2023-12-27 13:33:30,627][20381] Updated weights for policy 1, policy_version 1140443 (0.0008) [2023-12-27 13:33:30,688][20381] Updated weights for policy 1, policy_version 1140453 (0.0008) [2023-12-27 13:33:30,746][20381] Updated weights for policy 1, policy_version 1140463 (0.0007) [2023-12-27 13:33:30,772][20347] Updated weights for policy 0, policy_version 1141557 (0.0007) [2023-12-27 13:33:30,822][20347] Updated weights for policy 0, policy_version 1141567 (0.0006) [2023-12-27 13:33:30,867][20347] Updated weights for policy 0, policy_version 1141577 (0.0008) [2023-12-27 13:33:31,346][20381] Updated weights for policy 1, policy_version 1140473 (0.0006) [2023-12-27 13:33:31,411][20381] Updated weights for policy 1, policy_version 1140483 (0.0007) [2023-12-27 13:33:31,463][20381] Updated weights for policy 1, policy_version 1140493 (0.0007) [2023-12-27 13:33:31,515][20381] Updated weights for policy 1, policy_version 1140503 (0.0008) [2023-12-27 13:33:31,601][20347] Updated weights for policy 0, policy_version 1141587 (0.0007) [2023-12-27 13:33:31,670][20347] Updated weights for policy 0, policy_version 1141597 (0.0008) [2023-12-27 13:33:31,741][20347] Updated weights for policy 0, policy_version 1141607 (0.0007) [2023-12-27 13:33:32,190][20381] Updated weights for policy 1, policy_version 1140513 (0.0007) [2023-12-27 13:33:32,251][20381] Updated weights for policy 1, policy_version 1140523 (0.0008) [2023-12-27 13:33:32,313][20381] Updated weights for policy 1, policy_version 1140533 (0.0006) [2023-12-27 13:33:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 584310784. Throughput: 0: 10168.2, 1: 10369.5. Samples: 584319476. Policy #0 lag: (min: 18.0, avg: 31.1, max: 50.0) [2023-12-27 13:33:32,363][19170] Avg episode reward: [(0, '317.740'), (1, '647.190')] [2023-12-27 13:33:32,427][20347] Updated weights for policy 0, policy_version 1141617 (0.0007) [2023-12-27 13:33:32,486][20347] Updated weights for policy 0, policy_version 1141627 (0.0005) [2023-12-27 13:33:32,544][20347] Updated weights for policy 0, policy_version 1141637 (0.0005) [2023-12-27 13:33:32,597][20347] Updated weights for policy 0, policy_version 1141647 (0.0007) [2023-12-27 13:33:32,981][20381] Updated weights for policy 1, policy_version 1140543 (0.0008) [2023-12-27 13:33:33,031][20381] Updated weights for policy 1, policy_version 1140553 (0.0008) [2023-12-27 13:33:33,088][20381] Updated weights for policy 1, policy_version 1140563 (0.0011) [2023-12-27 13:33:33,239][20347] Updated weights for policy 0, policy_version 1141657 (0.0007) [2023-12-27 13:33:33,286][20347] Updated weights for policy 0, policy_version 1141667 (0.0007) [2023-12-27 13:33:33,340][20347] Updated weights for policy 0, policy_version 1141677 (0.0007) [2023-12-27 13:33:33,774][20381] Updated weights for policy 1, policy_version 1140573 (0.0010) [2023-12-27 13:33:33,825][20381] Updated weights for policy 1, policy_version 1140583 (0.0010) [2023-12-27 13:33:33,878][20381] Updated weights for policy 1, policy_version 1140593 (0.0010) [2023-12-27 13:33:34,046][20347] Updated weights for policy 0, policy_version 1141687 (0.0007) [2023-12-27 13:33:34,098][20347] Updated weights for policy 0, policy_version 1141697 (0.0007) [2023-12-27 13:33:34,153][20347] Updated weights for policy 0, policy_version 1141707 (0.0007) [2023-12-27 13:33:34,548][20381] Updated weights for policy 1, policy_version 1140603 (0.0010) [2023-12-27 13:33:34,609][20381] Updated weights for policy 1, policy_version 1140613 (0.0011) [2023-12-27 13:33:34,666][20381] Updated weights for policy 1, policy_version 1140623 (0.0011) [2023-12-27 13:33:34,851][20347] Updated weights for policy 0, policy_version 1141717 (0.0007) [2023-12-27 13:33:34,913][20347] Updated weights for policy 0, policy_version 1141727 (0.0008) [2023-12-27 13:33:34,977][20347] Updated weights for policy 0, policy_version 1141737 (0.0007) [2023-12-27 13:33:35,382][20381] Updated weights for policy 1, policy_version 1140633 (0.0010) [2023-12-27 13:33:35,433][20381] Updated weights for policy 1, policy_version 1140643 (0.0007) [2023-12-27 13:33:35,486][20381] Updated weights for policy 1, policy_version 1140653 (0.0007) [2023-12-27 13:33:35,540][20381] Updated weights for policy 1, policy_version 1140663 (0.0007) [2023-12-27 13:33:35,690][20347] Updated weights for policy 0, policy_version 1141747 (0.0008) [2023-12-27 13:33:35,740][20347] Updated weights for policy 0, policy_version 1141757 (0.0008) [2023-12-27 13:33:35,787][20347] Updated weights for policy 0, policy_version 1141767 (0.0008) [2023-12-27 13:33:36,159][20381] Updated weights for policy 1, policy_version 1140673 (0.0007) [2023-12-27 13:33:36,219][20381] Updated weights for policy 1, policy_version 1140683 (0.0008) [2023-12-27 13:33:36,280][20381] Updated weights for policy 1, policy_version 1140693 (0.0008) [2023-12-27 13:33:36,513][20347] Updated weights for policy 0, policy_version 1141777 (0.0007) [2023-12-27 13:33:36,567][20347] Updated weights for policy 0, policy_version 1141787 (0.0006) [2023-12-27 13:33:36,621][20347] Updated weights for policy 0, policy_version 1141797 (0.0007) [2023-12-27 13:33:36,683][20347] Updated weights for policy 0, policy_version 1141807 (0.0008) [2023-12-27 13:33:36,987][20381] Updated weights for policy 1, policy_version 1140703 (0.0008) [2023-12-27 13:33:37,053][20381] Updated weights for policy 1, policy_version 1140713 (0.0008) [2023-12-27 13:33:37,120][20381] Updated weights for policy 1, policy_version 1140723 (0.0008) [2023-12-27 13:33:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 584409088. Throughput: 0: 10185.7, 1: 10386.6. Samples: 584380344. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:33:37,363][19170] Avg episode reward: [(0, '340.700'), (1, '696.320')] [2023-12-27 13:33:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001140728_292052992.pth... [2023-12-27 13:33:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001139512_291741696.pth [2023-12-27 13:33:37,394][20347] Updated weights for policy 0, policy_version 1141817 (0.0006) [2023-12-27 13:33:37,444][20347] Updated weights for policy 0, policy_version 1141827 (0.0007) [2023-12-27 13:33:37,493][20347] Updated weights for policy 0, policy_version 1141837 (0.0008) [2023-12-27 13:33:37,515][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001141840_292364288.pth... [2023-12-27 13:33:37,519][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001140624_292052992.pth [2023-12-27 13:33:37,815][20381] Updated weights for policy 1, policy_version 1140733 (0.0008) [2023-12-27 13:33:37,865][20381] Updated weights for policy 1, policy_version 1140743 (0.0008) [2023-12-27 13:33:37,922][20381] Updated weights for policy 1, policy_version 1140753 (0.0007) [2023-12-27 13:33:38,148][20347] Updated weights for policy 0, policy_version 1141847 (0.0007) [2023-12-27 13:33:38,207][20347] Updated weights for policy 0, policy_version 1141857 (0.0008) [2023-12-27 13:33:38,257][20347] Updated weights for policy 0, policy_version 1141867 (0.0010) [2023-12-27 13:33:38,516][20381] Updated weights for policy 1, policy_version 1140763 (0.0005) [2023-12-27 13:33:38,577][20381] Updated weights for policy 1, policy_version 1140773 (0.0010) [2023-12-27 13:33:38,636][20381] Updated weights for policy 1, policy_version 1140783 (0.0008) [2023-12-27 13:33:38,893][20347] Updated weights for policy 0, policy_version 1141877 (0.0008) [2023-12-27 13:33:38,960][20347] Updated weights for policy 0, policy_version 1141887 (0.0008) [2023-12-27 13:33:39,012][20347] Updated weights for policy 0, policy_version 1141897 (0.0008) [2023-12-27 13:33:39,177][20381] Updated weights for policy 1, policy_version 1140793 (0.0006) [2023-12-27 13:33:39,233][20381] Updated weights for policy 1, policy_version 1140803 (0.0010) [2023-12-27 13:33:39,294][20381] Updated weights for policy 1, policy_version 1140813 (0.0007) [2023-12-27 13:33:39,361][20381] Updated weights for policy 1, policy_version 1140823 (0.0009) [2023-12-27 13:33:39,819][20347] Updated weights for policy 0, policy_version 1141907 (0.0008) [2023-12-27 13:33:39,881][20347] Updated weights for policy 0, policy_version 1141917 (0.0007) [2023-12-27 13:33:39,945][20347] Updated weights for policy 0, policy_version 1141927 (0.0007) [2023-12-27 13:33:40,125][20381] Updated weights for policy 1, policy_version 1140833 (0.0007) [2023-12-27 13:33:40,183][20381] Updated weights for policy 1, policy_version 1140843 (0.0006) [2023-12-27 13:33:40,243][20381] Updated weights for policy 1, policy_version 1140853 (0.0006) [2023-12-27 13:33:40,617][20347] Updated weights for policy 0, policy_version 1141937 (0.0007) [2023-12-27 13:33:40,673][20347] Updated weights for policy 0, policy_version 1141947 (0.0008) [2023-12-27 13:33:40,723][20347] Updated weights for policy 0, policy_version 1141957 (0.0008) [2023-12-27 13:33:40,769][20347] Updated weights for policy 0, policy_version 1141967 (0.0007) [2023-12-27 13:33:40,946][20381] Updated weights for policy 1, policy_version 1140863 (0.0007) [2023-12-27 13:33:41,011][20381] Updated weights for policy 1, policy_version 1140873 (0.0006) [2023-12-27 13:33:41,070][20381] Updated weights for policy 1, policy_version 1140883 (0.0007) [2023-12-27 13:33:41,565][20347] Updated weights for policy 0, policy_version 1141977 (0.0008) [2023-12-27 13:33:41,633][20347] Updated weights for policy 0, policy_version 1141987 (0.0010) [2023-12-27 13:33:41,701][20347] Updated weights for policy 0, policy_version 1141997 (0.0007) [2023-12-27 13:33:41,863][20381] Updated weights for policy 1, policy_version 1140893 (0.0008) [2023-12-27 13:33:41,920][20381] Updated weights for policy 1, policy_version 1140903 (0.0008) [2023-12-27 13:33:41,971][20381] Updated weights for policy 1, policy_version 1140913 (0.0009) [2023-12-27 13:33:42,332][20347] Updated weights for policy 0, policy_version 1142007 (0.0008) [2023-12-27 13:33:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 584507392. Throughput: 0: 10155.0, 1: 10336.4. Samples: 584502328. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:33:42,363][19170] Avg episode reward: [(0, '347.960'), (1, '716.200')] [2023-12-27 13:33:42,396][20347] Updated weights for policy 0, policy_version 1142017 (0.0008) [2023-12-27 13:33:42,455][20347] Updated weights for policy 0, policy_version 1142027 (0.0008) [2023-12-27 13:33:42,742][20381] Updated weights for policy 1, policy_version 1140923 (0.0008) [2023-12-27 13:33:42,799][20381] Updated weights for policy 1, policy_version 1140933 (0.0007) [2023-12-27 13:33:42,853][20381] Updated weights for policy 1, policy_version 1140943 (0.0008) [2023-12-27 13:33:43,123][20347] Updated weights for policy 0, policy_version 1142037 (0.0008) [2023-12-27 13:33:43,175][20347] Updated weights for policy 0, policy_version 1142047 (0.0006) [2023-12-27 13:33:43,234][20347] Updated weights for policy 0, policy_version 1142057 (0.0009) [2023-12-27 13:33:43,490][20381] Updated weights for policy 1, policy_version 1140953 (0.0007) [2023-12-27 13:33:43,538][20381] Updated weights for policy 1, policy_version 1140963 (0.0007) [2023-12-27 13:33:43,587][20381] Updated weights for policy 1, policy_version 1140973 (0.0008) [2023-12-27 13:33:43,649][20381] Updated weights for policy 1, policy_version 1140983 (0.0008) [2023-12-27 13:33:43,935][20347] Updated weights for policy 0, policy_version 1142067 (0.0007) [2023-12-27 13:33:43,993][20347] Updated weights for policy 0, policy_version 1142077 (0.0008) [2023-12-27 13:33:44,054][20347] Updated weights for policy 0, policy_version 1142087 (0.0008) [2023-12-27 13:33:44,255][20381] Updated weights for policy 1, policy_version 1140993 (0.0007) [2023-12-27 13:33:44,313][20381] Updated weights for policy 1, policy_version 1141003 (0.0010) [2023-12-27 13:33:44,369][20381] Updated weights for policy 1, policy_version 1141013 (0.0009) [2023-12-27 13:33:44,733][20347] Updated weights for policy 0, policy_version 1142097 (0.0008) [2023-12-27 13:33:44,790][20347] Updated weights for policy 0, policy_version 1142107 (0.0014) [2023-12-27 13:33:44,852][20347] Updated weights for policy 0, policy_version 1142117 (0.0007) [2023-12-27 13:33:44,908][20347] Updated weights for policy 0, policy_version 1142127 (0.0008) [2023-12-27 13:33:44,984][20381] Updated weights for policy 1, policy_version 1141024 (0.0007) [2023-12-27 13:33:45,048][20381] Updated weights for policy 1, policy_version 1141034 (0.0011) [2023-12-27 13:33:45,111][20381] Updated weights for policy 1, policy_version 1141044 (0.0009) [2023-12-27 13:33:45,614][20347] Updated weights for policy 0, policy_version 1142137 (0.0006) [2023-12-27 13:33:45,665][20347] Updated weights for policy 0, policy_version 1142147 (0.0006) [2023-12-27 13:33:45,710][20347] Updated weights for policy 0, policy_version 1142157 (0.0007) [2023-12-27 13:33:45,750][20381] Updated weights for policy 1, policy_version 1141054 (0.0010) [2023-12-27 13:33:45,810][20381] Updated weights for policy 1, policy_version 1141064 (0.0011) [2023-12-27 13:33:45,868][20381] Updated weights for policy 1, policy_version 1141074 (0.0010) [2023-12-27 13:33:46,364][20347] Updated weights for policy 0, policy_version 1142167 (0.0009) [2023-12-27 13:33:46,418][20347] Updated weights for policy 0, policy_version 1142177 (0.0010) [2023-12-27 13:33:46,472][20347] Updated weights for policy 0, policy_version 1142187 (0.0010) [2023-12-27 13:33:46,492][20381] Updated weights for policy 1, policy_version 1141084 (0.0010) [2023-12-27 13:33:46,550][20381] Updated weights for policy 1, policy_version 1141094 (0.0010) [2023-12-27 13:33:46,609][20381] Updated weights for policy 1, policy_version 1141104 (0.0005) [2023-12-27 13:33:47,153][20347] Updated weights for policy 0, policy_version 1142197 (0.0009) [2023-12-27 13:33:47,174][20381] Updated weights for policy 1, policy_version 1141114 (0.0005) [2023-12-27 13:33:47,219][20347] Updated weights for policy 0, policy_version 1142207 (0.0007) [2023-12-27 13:33:47,230][20381] Updated weights for policy 1, policy_version 1141124 (0.0007) [2023-12-27 13:33:47,276][20347] Updated weights for policy 0, policy_version 1142217 (0.0005) [2023-12-27 13:33:47,283][20381] Updated weights for policy 1, policy_version 1141134 (0.0007) [2023-12-27 13:33:47,336][20381] Updated weights for policy 1, policy_version 1141144 (0.0008) [2023-12-27 13:33:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 584622080. Throughput: 0: 10150.1, 1: 10367.1. Samples: 584626412. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:33:47,364][19170] Avg episode reward: [(0, '384.820'), (1, '647.230')] [2023-12-27 13:33:47,877][20347] Updated weights for policy 0, policy_version 1142227 (0.0005) [2023-12-27 13:33:47,929][20347] Updated weights for policy 0, policy_version 1142237 (0.0005) [2023-12-27 13:33:47,984][20347] Updated weights for policy 0, policy_version 1142247 (0.0007) [2023-12-27 13:33:47,988][20381] Updated weights for policy 1, policy_version 1141154 (0.0009) [2023-12-27 13:33:48,037][20381] Updated weights for policy 1, policy_version 1141164 (0.0011) [2023-12-27 13:33:48,086][20381] Updated weights for policy 1, policy_version 1141174 (0.0010) [2023-12-27 13:33:48,627][20347] Updated weights for policy 0, policy_version 1142257 (0.0009) [2023-12-27 13:33:48,689][20347] Updated weights for policy 0, policy_version 1142267 (0.0008) [2023-12-27 13:33:48,751][20347] Updated weights for policy 0, policy_version 1142277 (0.0006) [2023-12-27 13:33:48,766][20381] Updated weights for policy 1, policy_version 1141184 (0.0011) [2023-12-27 13:33:48,812][20347] Updated weights for policy 0, policy_version 1142287 (0.0005) [2023-12-27 13:33:48,828][20381] Updated weights for policy 1, policy_version 1141194 (0.0007) [2023-12-27 13:33:48,890][20381] Updated weights for policy 1, policy_version 1141204 (0.0007) [2023-12-27 13:33:49,500][20381] Updated weights for policy 1, policy_version 1141214 (0.0011) [2023-12-27 13:33:49,511][20347] Updated weights for policy 0, policy_version 1142297 (0.0006) [2023-12-27 13:33:49,559][20381] Updated weights for policy 1, policy_version 1141224 (0.0007) [2023-12-27 13:33:49,579][20347] Updated weights for policy 0, policy_version 1142307 (0.0008) [2023-12-27 13:33:49,620][20381] Updated weights for policy 1, policy_version 1141234 (0.0010) [2023-12-27 13:33:49,638][20347] Updated weights for policy 0, policy_version 1142317 (0.0006) [2023-12-27 13:33:50,254][20347] Updated weights for policy 0, policy_version 1142327 (0.0006) [2023-12-27 13:33:50,279][20381] Updated weights for policy 1, policy_version 1141244 (0.0009) [2023-12-27 13:33:50,316][20347] Updated weights for policy 0, policy_version 1142337 (0.0011) [2023-12-27 13:33:50,338][20381] Updated weights for policy 1, policy_version 1141254 (0.0008) [2023-12-27 13:33:50,381][20347] Updated weights for policy 0, policy_version 1142347 (0.0007) [2023-12-27 13:33:50,402][20381] Updated weights for policy 1, policy_version 1141264 (0.0011) [2023-12-27 13:33:51,086][20381] Updated weights for policy 1, policy_version 1141274 (0.0009) [2023-12-27 13:33:51,099][20347] Updated weights for policy 0, policy_version 1142357 (0.0007) [2023-12-27 13:33:51,149][20381] Updated weights for policy 1, policy_version 1141284 (0.0006) [2023-12-27 13:33:51,164][20347] Updated weights for policy 0, policy_version 1142367 (0.0007) [2023-12-27 13:33:51,210][20381] Updated weights for policy 1, policy_version 1141294 (0.0008) [2023-12-27 13:33:51,216][20347] Updated weights for policy 0, policy_version 1142377 (0.0006) [2023-12-27 13:33:51,273][20381] Updated weights for policy 1, policy_version 1141304 (0.0007) [2023-12-27 13:33:51,964][20347] Updated weights for policy 0, policy_version 1142387 (0.0006) [2023-12-27 13:33:51,997][20381] Updated weights for policy 1, policy_version 1141314 (0.0008) [2023-12-27 13:33:52,022][20347] Updated weights for policy 0, policy_version 1142397 (0.0009) [2023-12-27 13:33:52,055][20381] Updated weights for policy 1, policy_version 1141324 (0.0008) [2023-12-27 13:33:52,079][20347] Updated weights for policy 0, policy_version 1142407 (0.0008) [2023-12-27 13:33:52,108][20381] Updated weights for policy 1, policy_version 1141334 (0.0007) [2023-12-27 13:33:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 584720384. Throughput: 0: 10204.8, 1: 10352.9. Samples: 584689612. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:33:52,363][19170] Avg episode reward: [(0, '192.340'), (1, '647.520')] [2023-12-27 13:33:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001141336_292208640.pth... [2023-12-27 13:33:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001142416_292511744.pth... [2023-12-27 13:33:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001141232_292208640.pth [2023-12-27 13:33:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001140088_291889152.pth [2023-12-27 13:33:52,759][20347] Updated weights for policy 0, policy_version 1142417 (0.0008) [2023-12-27 13:33:52,819][20347] Updated weights for policy 0, policy_version 1142427 (0.0007) [2023-12-27 13:33:52,880][20347] Updated weights for policy 0, policy_version 1142437 (0.0007) [2023-12-27 13:33:52,888][20381] Updated weights for policy 1, policy_version 1141344 (0.0006) [2023-12-27 13:33:52,941][20381] Updated weights for policy 1, policy_version 1141354 (0.0007) [2023-12-27 13:33:52,943][20347] Updated weights for policy 0, policy_version 1142447 (0.0007) [2023-12-27 13:33:52,993][20381] Updated weights for policy 1, policy_version 1141364 (0.0005) [2023-12-27 13:33:53,559][20381] Updated weights for policy 1, policy_version 1141374 (0.0007) [2023-12-27 13:33:53,606][20381] Updated weights for policy 1, policy_version 1141384 (0.0007) [2023-12-27 13:33:53,608][20347] Updated weights for policy 0, policy_version 1142457 (0.0010) [2023-12-27 13:33:53,658][20381] Updated weights for policy 1, policy_version 1141394 (0.0005) [2023-12-27 13:33:53,659][20347] Updated weights for policy 0, policy_version 1142467 (0.0010) [2023-12-27 13:33:53,718][20347] Updated weights for policy 0, policy_version 1142477 (0.0005) [2023-12-27 13:33:54,315][20347] Updated weights for policy 0, policy_version 1142487 (0.0009) [2023-12-27 13:33:54,375][20347] Updated weights for policy 0, policy_version 1142497 (0.0008) [2023-12-27 13:33:54,412][20381] Updated weights for policy 1, policy_version 1141404 (0.0007) [2023-12-27 13:33:54,437][20347] Updated weights for policy 0, policy_version 1142507 (0.0008) [2023-12-27 13:33:54,469][20381] Updated weights for policy 1, policy_version 1141414 (0.0008) [2023-12-27 13:33:54,526][20381] Updated weights for policy 1, policy_version 1141424 (0.0008) [2023-12-27 13:33:55,110][20347] Updated weights for policy 0, policy_version 1142517 (0.0008) [2023-12-27 13:33:55,163][20347] Updated weights for policy 0, policy_version 1142527 (0.0007) [2023-12-27 13:33:55,218][20347] Updated weights for policy 0, policy_version 1142537 (0.0007) [2023-12-27 13:33:55,234][20381] Updated weights for policy 1, policy_version 1141434 (0.0008) [2023-12-27 13:33:55,288][20381] Updated weights for policy 1, policy_version 1141444 (0.0007) [2023-12-27 13:33:55,341][20381] Updated weights for policy 1, policy_version 1141454 (0.0008) [2023-12-27 13:33:55,390][20381] Updated weights for policy 1, policy_version 1141464 (0.0007) [2023-12-27 13:33:55,802][20347] Updated weights for policy 0, policy_version 1142547 (0.0005) [2023-12-27 13:33:55,854][20347] Updated weights for policy 0, policy_version 1142557 (0.0005) [2023-12-27 13:33:55,903][20347] Updated weights for policy 0, policy_version 1142567 (0.0006) [2023-12-27 13:33:56,121][20381] Updated weights for policy 1, policy_version 1141474 (0.0011) [2023-12-27 13:33:56,167][20381] Updated weights for policy 1, policy_version 1141484 (0.0010) [2023-12-27 13:33:56,214][20381] Updated weights for policy 1, policy_version 1141494 (0.0010) [2023-12-27 13:33:56,570][20347] Updated weights for policy 0, policy_version 1142577 (0.0007) [2023-12-27 13:33:56,628][20347] Updated weights for policy 0, policy_version 1142587 (0.0008) [2023-12-27 13:33:56,681][20347] Updated weights for policy 0, policy_version 1142597 (0.0005) [2023-12-27 13:33:56,741][20347] Updated weights for policy 0, policy_version 1142607 (0.0007) [2023-12-27 13:33:56,927][20381] Updated weights for policy 1, policy_version 1141504 (0.0011) [2023-12-27 13:33:56,981][20381] Updated weights for policy 1, policy_version 1141514 (0.0010) [2023-12-27 13:33:57,038][20381] Updated weights for policy 1, policy_version 1141524 (0.0010) [2023-12-27 13:33:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 584818688. Throughput: 0: 10217.8, 1: 10325.3. Samples: 584810532. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:33:57,363][19170] Avg episode reward: [(0, '270.330'), (1, '647.690')] [2023-12-27 13:33:57,379][20347] Updated weights for policy 0, policy_version 1142617 (0.0007) [2023-12-27 13:33:57,432][20347] Updated weights for policy 0, policy_version 1142627 (0.0007) [2023-12-27 13:33:57,482][20347] Updated weights for policy 0, policy_version 1142637 (0.0007) [2023-12-27 13:33:57,710][20381] Updated weights for policy 1, policy_version 1141534 (0.0010) [2023-12-27 13:33:57,767][20381] Updated weights for policy 1, policy_version 1141544 (0.0010) [2023-12-27 13:33:57,821][20381] Updated weights for policy 1, policy_version 1141554 (0.0010) [2023-12-27 13:33:58,152][20347] Updated weights for policy 0, policy_version 1142647 (0.0008) [2023-12-27 13:33:58,208][20347] Updated weights for policy 0, policy_version 1142657 (0.0008) [2023-12-27 13:33:58,266][20347] Updated weights for policy 0, policy_version 1142667 (0.0008) [2023-12-27 13:33:58,579][20381] Updated weights for policy 1, policy_version 1141564 (0.0009) [2023-12-27 13:33:58,640][20381] Updated weights for policy 1, policy_version 1141574 (0.0009) [2023-12-27 13:33:58,700][20381] Updated weights for policy 1, policy_version 1141584 (0.0009) [2023-12-27 13:33:59,136][20347] Updated weights for policy 0, policy_version 1142677 (0.0008) [2023-12-27 13:33:59,187][20347] Updated weights for policy 0, policy_version 1142687 (0.0008) [2023-12-27 13:33:59,251][20347] Updated weights for policy 0, policy_version 1142697 (0.0009) [2023-12-27 13:33:59,454][20381] Updated weights for policy 1, policy_version 1141594 (0.0007) [2023-12-27 13:33:59,509][20381] Updated weights for policy 1, policy_version 1141604 (0.0006) [2023-12-27 13:33:59,568][20381] Updated weights for policy 1, policy_version 1141614 (0.0007) [2023-12-27 13:33:59,625][20381] Updated weights for policy 1, policy_version 1141624 (0.0008) [2023-12-27 13:33:59,945][20347] Updated weights for policy 0, policy_version 1142707 (0.0010) [2023-12-27 13:34:00,008][20347] Updated weights for policy 0, policy_version 1142717 (0.0010) [2023-12-27 13:34:00,067][20347] Updated weights for policy 0, policy_version 1142727 (0.0010) [2023-12-27 13:34:00,316][20381] Updated weights for policy 1, policy_version 1141634 (0.0007) [2023-12-27 13:34:00,373][20381] Updated weights for policy 1, policy_version 1141644 (0.0007) [2023-12-27 13:34:00,426][20381] Updated weights for policy 1, policy_version 1141654 (0.0006) [2023-12-27 13:34:00,745][20347] Updated weights for policy 0, policy_version 1142737 (0.0010) [2023-12-27 13:34:00,792][20347] Updated weights for policy 0, policy_version 1142747 (0.0010) [2023-12-27 13:34:00,840][20347] Updated weights for policy 0, policy_version 1142757 (0.0010) [2023-12-27 13:34:00,900][20347] Updated weights for policy 0, policy_version 1142767 (0.0010) [2023-12-27 13:34:01,063][20381] Updated weights for policy 1, policy_version 1141664 (0.0008) [2023-12-27 13:34:01,122][20381] Updated weights for policy 1, policy_version 1141674 (0.0007) [2023-12-27 13:34:01,180][20381] Updated weights for policy 1, policy_version 1141684 (0.0008) [2023-12-27 13:34:01,615][20347] Updated weights for policy 0, policy_version 1142777 (0.0008) [2023-12-27 13:34:01,677][20347] Updated weights for policy 0, policy_version 1142787 (0.0008) [2023-12-27 13:34:01,743][20347] Updated weights for policy 0, policy_version 1142797 (0.0008) [2023-12-27 13:34:01,898][20381] Updated weights for policy 1, policy_version 1141694 (0.0008) [2023-12-27 13:34:01,961][20381] Updated weights for policy 1, policy_version 1141704 (0.0008) [2023-12-27 13:34:02,016][20381] Updated weights for policy 1, policy_version 1141714 (0.0008) [2023-12-27 13:34:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 584916992. Throughput: 0: 10175.9, 1: 10310.8. Samples: 584929264. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:02,363][19170] Avg episode reward: [(0, '206.490'), (1, '725.800')] [2023-12-27 13:34:02,484][20347] Updated weights for policy 0, policy_version 1142807 (0.0008) [2023-12-27 13:34:02,541][20347] Updated weights for policy 0, policy_version 1142817 (0.0008) [2023-12-27 13:34:02,600][20347] Updated weights for policy 0, policy_version 1142827 (0.0007) [2023-12-27 13:34:02,614][20381] Updated weights for policy 1, policy_version 1141724 (0.0005) [2023-12-27 13:34:02,666][20381] Updated weights for policy 1, policy_version 1141734 (0.0005) [2023-12-27 13:34:02,726][20381] Updated weights for policy 1, policy_version 1141744 (0.0006) [2023-12-27 13:34:03,259][20347] Updated weights for policy 0, policy_version 1142837 (0.0007) [2023-12-27 13:34:03,307][20381] Updated weights for policy 1, policy_version 1141754 (0.0006) [2023-12-27 13:34:03,309][20347] Updated weights for policy 0, policy_version 1142847 (0.0008) [2023-12-27 13:34:03,361][20347] Updated weights for policy 0, policy_version 1142857 (0.0007) [2023-12-27 13:34:03,363][20381] Updated weights for policy 1, policy_version 1141764 (0.0006) [2023-12-27 13:34:03,419][20381] Updated weights for policy 1, policy_version 1141774 (0.0007) [2023-12-27 13:34:03,478][20381] Updated weights for policy 1, policy_version 1141784 (0.0007) [2023-12-27 13:34:03,995][20347] Updated weights for policy 0, policy_version 1142867 (0.0006) [2023-12-27 13:34:04,052][20347] Updated weights for policy 0, policy_version 1142877 (0.0008) [2023-12-27 13:34:04,110][20347] Updated weights for policy 0, policy_version 1142887 (0.0008) [2023-12-27 13:34:04,191][20381] Updated weights for policy 1, policy_version 1141794 (0.0007) [2023-12-27 13:34:04,246][20381] Updated weights for policy 1, policy_version 1141804 (0.0008) [2023-12-27 13:34:04,303][20381] Updated weights for policy 1, policy_version 1141814 (0.0005) [2023-12-27 13:34:04,814][20347] Updated weights for policy 0, policy_version 1142897 (0.0007) [2023-12-27 13:34:04,871][20347] Updated weights for policy 0, policy_version 1142907 (0.0008) [2023-12-27 13:34:04,929][20347] Updated weights for policy 0, policy_version 1142917 (0.0008) [2023-12-27 13:34:04,955][20381] Updated weights for policy 1, policy_version 1141824 (0.0005) [2023-12-27 13:34:04,979][20347] Updated weights for policy 0, policy_version 1142927 (0.0006) [2023-12-27 13:34:05,013][20381] Updated weights for policy 1, policy_version 1141834 (0.0007) [2023-12-27 13:34:05,070][20381] Updated weights for policy 1, policy_version 1141844 (0.0008) [2023-12-27 13:34:05,700][20347] Updated weights for policy 0, policy_version 1142937 (0.0007) [2023-12-27 13:34:05,716][20381] Updated weights for policy 1, policy_version 1141854 (0.0006) [2023-12-27 13:34:05,752][20347] Updated weights for policy 0, policy_version 1142947 (0.0007) [2023-12-27 13:34:05,762][20381] Updated weights for policy 1, policy_version 1141864 (0.0005) [2023-12-27 13:34:05,806][20347] Updated weights for policy 0, policy_version 1142957 (0.0008) [2023-12-27 13:34:05,821][20381] Updated weights for policy 1, policy_version 1141874 (0.0005) [2023-12-27 13:34:06,459][20381] Updated weights for policy 1, policy_version 1141884 (0.0007) [2023-12-27 13:34:06,518][20381] Updated weights for policy 1, policy_version 1141894 (0.0007) [2023-12-27 13:34:06,521][20347] Updated weights for policy 0, policy_version 1142967 (0.0007) [2023-12-27 13:34:06,580][20347] Updated weights for policy 0, policy_version 1142977 (0.0007) [2023-12-27 13:34:06,581][20381] Updated weights for policy 1, policy_version 1141904 (0.0007) [2023-12-27 13:34:06,643][20347] Updated weights for policy 0, policy_version 1142987 (0.0007) [2023-12-27 13:34:07,258][20381] Updated weights for policy 1, policy_version 1141914 (0.0008) [2023-12-27 13:34:07,318][20381] Updated weights for policy 1, policy_version 1141924 (0.0007) [2023-12-27 13:34:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 585015296. Throughput: 0: 10154.6, 1: 10362.4. Samples: 584991728. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:07,363][19170] Avg episode reward: [(0, '371.980'), (1, '755.210')] [2023-12-27 13:34:07,364][20347] Updated weights for policy 0, policy_version 1142997 (0.0008) [2023-12-27 13:34:07,380][20381] Updated weights for policy 1, policy_version 1141934 (0.0009) [2023-12-27 13:34:07,425][20347] Updated weights for policy 0, policy_version 1143007 (0.0006) [2023-12-27 13:34:07,439][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001141944_292364288.pth... [2023-12-27 13:34:07,440][20381] Updated weights for policy 1, policy_version 1141944 (0.0007) [2023-12-27 13:34:07,443][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001140728_292052992.pth [2023-12-27 13:34:07,492][20347] Updated weights for policy 0, policy_version 1143017 (0.0008) [2023-12-27 13:34:07,530][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001143024_292667392.pth... [2023-12-27 13:34:07,534][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001141840_292364288.pth [2023-12-27 13:34:08,048][20381] Updated weights for policy 1, policy_version 1141954 (0.0008) [2023-12-27 13:34:08,096][20381] Updated weights for policy 1, policy_version 1141964 (0.0007) [2023-12-27 13:34:08,118][20347] Updated weights for policy 0, policy_version 1143027 (0.0007) [2023-12-27 13:34:08,154][20381] Updated weights for policy 1, policy_version 1141974 (0.0006) [2023-12-27 13:34:08,174][20347] Updated weights for policy 0, policy_version 1143037 (0.0006) [2023-12-27 13:34:08,231][20347] Updated weights for policy 0, policy_version 1143047 (0.0006) [2023-12-27 13:34:08,808][20381] Updated weights for policy 1, policy_version 1141984 (0.0007) [2023-12-27 13:34:08,862][20381] Updated weights for policy 1, policy_version 1141994 (0.0008) [2023-12-27 13:34:08,911][20381] Updated weights for policy 1, policy_version 1142004 (0.0008) [2023-12-27 13:34:08,917][20347] Updated weights for policy 0, policy_version 1143057 (0.0007) [2023-12-27 13:34:08,978][20347] Updated weights for policy 0, policy_version 1143067 (0.0007) [2023-12-27 13:34:09,030][20347] Updated weights for policy 0, policy_version 1143077 (0.0007) [2023-12-27 13:34:09,090][20347] Updated weights for policy 0, policy_version 1143087 (0.0008) [2023-12-27 13:34:09,563][20381] Updated weights for policy 1, policy_version 1142014 (0.0007) [2023-12-27 13:34:09,617][20381] Updated weights for policy 1, policy_version 1142024 (0.0009) [2023-12-27 13:34:09,674][20381] Updated weights for policy 1, policy_version 1142034 (0.0008) [2023-12-27 13:34:09,800][20347] Updated weights for policy 0, policy_version 1143097 (0.0007) [2023-12-27 13:34:09,866][20347] Updated weights for policy 0, policy_version 1143107 (0.0008) [2023-12-27 13:34:09,927][20347] Updated weights for policy 0, policy_version 1143117 (0.0006) [2023-12-27 13:34:10,410][20381] Updated weights for policy 1, policy_version 1142044 (0.0008) [2023-12-27 13:34:10,467][20381] Updated weights for policy 1, policy_version 1142054 (0.0008) [2023-12-27 13:34:10,527][20381] Updated weights for policy 1, policy_version 1142064 (0.0007) [2023-12-27 13:34:10,570][20347] Updated weights for policy 0, policy_version 1143127 (0.0006) [2023-12-27 13:34:10,627][20347] Updated weights for policy 0, policy_version 1143137 (0.0008) [2023-12-27 13:34:10,686][20347] Updated weights for policy 0, policy_version 1143147 (0.0007) [2023-12-27 13:34:11,166][20381] Updated weights for policy 1, policy_version 1142074 (0.0007) [2023-12-27 13:34:11,234][20381] Updated weights for policy 1, policy_version 1142084 (0.0006) [2023-12-27 13:34:11,303][20381] Updated weights for policy 1, policy_version 1142094 (0.0006) [2023-12-27 13:34:11,366][20381] Updated weights for policy 1, policy_version 1142104 (0.0010) [2023-12-27 13:34:11,478][20347] Updated weights for policy 0, policy_version 1143157 (0.0009) [2023-12-27 13:34:11,539][20347] Updated weights for policy 0, policy_version 1143167 (0.0010) [2023-12-27 13:34:11,601][20347] Updated weights for policy 0, policy_version 1143177 (0.0006) [2023-12-27 13:34:12,027][20381] Updated weights for policy 1, policy_version 1142114 (0.0008) [2023-12-27 13:34:12,089][20381] Updated weights for policy 1, policy_version 1142124 (0.0008) [2023-12-27 13:34:12,156][20381] Updated weights for policy 1, policy_version 1142134 (0.0008) [2023-12-27 13:34:12,302][20347] Updated weights for policy 0, policy_version 1143187 (0.0008) [2023-12-27 13:34:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 585121792. Throughput: 0: 10117.4, 1: 10359.9. Samples: 585114208. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:12,363][19170] Avg episode reward: [(0, '347.220'), (1, '745.470')] [2023-12-27 13:34:12,376][20347] Updated weights for policy 0, policy_version 1143197 (0.0007) [2023-12-27 13:34:12,447][20347] Updated weights for policy 0, policy_version 1143207 (0.0007) [2023-12-27 13:34:12,877][20381] Updated weights for policy 1, policy_version 1142144 (0.0007) [2023-12-27 13:34:12,932][20381] Updated weights for policy 1, policy_version 1142154 (0.0009) [2023-12-27 13:34:12,987][20381] Updated weights for policy 1, policy_version 1142164 (0.0005) [2023-12-27 13:34:13,108][20347] Updated weights for policy 0, policy_version 1143217 (0.0008) [2023-12-27 13:34:13,163][20347] Updated weights for policy 0, policy_version 1143227 (0.0008) [2023-12-27 13:34:13,214][20347] Updated weights for policy 0, policy_version 1143237 (0.0009) [2023-12-27 13:34:13,275][20347] Updated weights for policy 0, policy_version 1143247 (0.0007) [2023-12-27 13:34:13,643][20381] Updated weights for policy 1, policy_version 1142174 (0.0007) [2023-12-27 13:34:13,702][20381] Updated weights for policy 1, policy_version 1142184 (0.0008) [2023-12-27 13:34:13,757][20381] Updated weights for policy 1, policy_version 1142194 (0.0007) [2023-12-27 13:34:13,956][20347] Updated weights for policy 0, policy_version 1143257 (0.0007) [2023-12-27 13:34:14,008][20347] Updated weights for policy 0, policy_version 1143267 (0.0007) [2023-12-27 13:34:14,059][20347] Updated weights for policy 0, policy_version 1143277 (0.0006) [2023-12-27 13:34:14,457][20381] Updated weights for policy 1, policy_version 1142204 (0.0006) [2023-12-27 13:34:14,504][20381] Updated weights for policy 1, policy_version 1142214 (0.0005) [2023-12-27 13:34:14,550][20381] Updated weights for policy 1, policy_version 1142224 (0.0006) [2023-12-27 13:34:14,701][20347] Updated weights for policy 0, policy_version 1143287 (0.0008) [2023-12-27 13:34:14,757][20347] Updated weights for policy 0, policy_version 1143297 (0.0008) [2023-12-27 13:34:14,823][20347] Updated weights for policy 0, policy_version 1143307 (0.0007) [2023-12-27 13:34:15,193][20381] Updated weights for policy 1, policy_version 1142234 (0.0008) [2023-12-27 13:34:15,252][20381] Updated weights for policy 1, policy_version 1142244 (0.0008) [2023-12-27 13:34:15,308][20381] Updated weights for policy 1, policy_version 1142254 (0.0007) [2023-12-27 13:34:15,365][20381] Updated weights for policy 1, policy_version 1142264 (0.0005) [2023-12-27 13:34:15,486][20347] Updated weights for policy 0, policy_version 1143317 (0.0008) [2023-12-27 13:34:15,533][20347] Updated weights for policy 0, policy_version 1143327 (0.0005) [2023-12-27 13:34:15,586][20347] Updated weights for policy 0, policy_version 1143337 (0.0005) [2023-12-27 13:34:16,085][20381] Updated weights for policy 1, policy_version 1142274 (0.0007) [2023-12-27 13:34:16,140][20381] Updated weights for policy 1, policy_version 1142284 (0.0007) [2023-12-27 13:34:16,187][20381] Updated weights for policy 1, policy_version 1142294 (0.0007) [2023-12-27 13:34:16,217][20347] Updated weights for policy 0, policy_version 1143347 (0.0007) [2023-12-27 13:34:16,273][20347] Updated weights for policy 0, policy_version 1143357 (0.0010) [2023-12-27 13:34:16,337][20347] Updated weights for policy 0, policy_version 1143367 (0.0010) [2023-12-27 13:34:16,887][20381] Updated weights for policy 1, policy_version 1142304 (0.0007) [2023-12-27 13:34:16,946][20381] Updated weights for policy 1, policy_version 1142314 (0.0010) [2023-12-27 13:34:16,960][20347] Updated weights for policy 0, policy_version 1143377 (0.0010) [2023-12-27 13:34:16,996][20381] Updated weights for policy 1, policy_version 1142324 (0.0008) [2023-12-27 13:34:17,007][20347] Updated weights for policy 0, policy_version 1143387 (0.0005) [2023-12-27 13:34:17,051][20347] Updated weights for policy 0, policy_version 1143397 (0.0005) [2023-12-27 13:34:17,108][20347] Updated weights for policy 0, policy_version 1143407 (0.0007) [2023-12-27 13:34:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 585228288. Throughput: 0: 10112.1, 1: 10259.7. Samples: 585236204. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:17,363][19170] Avg episode reward: [(0, '344.760'), (1, '667.390')] [2023-12-27 13:34:17,733][20381] Updated weights for policy 1, policy_version 1142334 (0.0008) [2023-12-27 13:34:17,771][20347] Updated weights for policy 0, policy_version 1143417 (0.0010) [2023-12-27 13:34:17,792][20381] Updated weights for policy 1, policy_version 1142344 (0.0006) [2023-12-27 13:34:17,817][20347] Updated weights for policy 0, policy_version 1143427 (0.0010) [2023-12-27 13:34:17,852][20381] Updated weights for policy 1, policy_version 1142354 (0.0007) [2023-12-27 13:34:17,870][20347] Updated weights for policy 0, policy_version 1143437 (0.0010) [2023-12-27 13:34:18,554][20381] Updated weights for policy 1, policy_version 1142364 (0.0007) [2023-12-27 13:34:18,571][20347] Updated weights for policy 0, policy_version 1143447 (0.0008) [2023-12-27 13:34:18,607][20381] Updated weights for policy 1, policy_version 1142374 (0.0006) [2023-12-27 13:34:18,624][20347] Updated weights for policy 0, policy_version 1143457 (0.0007) [2023-12-27 13:34:18,660][20381] Updated weights for policy 1, policy_version 1142384 (0.0005) [2023-12-27 13:34:18,679][20347] Updated weights for policy 0, policy_version 1143467 (0.0008) [2023-12-27 13:34:19,395][20347] Updated weights for policy 0, policy_version 1143477 (0.0008) [2023-12-27 13:34:19,395][20381] Updated weights for policy 1, policy_version 1142394 (0.0006) [2023-12-27 13:34:19,451][20381] Updated weights for policy 1, policy_version 1142404 (0.0007) [2023-12-27 13:34:19,458][20347] Updated weights for policy 0, policy_version 1143487 (0.0008) [2023-12-27 13:34:19,511][20381] Updated weights for policy 1, policy_version 1142414 (0.0007) [2023-12-27 13:34:19,525][20347] Updated weights for policy 0, policy_version 1143497 (0.0007) [2023-12-27 13:34:19,567][20381] Updated weights for policy 1, policy_version 1142424 (0.0007) [2023-12-27 13:34:20,278][20347] Updated weights for policy 0, policy_version 1143507 (0.0007) [2023-12-27 13:34:20,313][20381] Updated weights for policy 1, policy_version 1142434 (0.0008) [2023-12-27 13:34:20,335][20347] Updated weights for policy 0, policy_version 1143517 (0.0005) [2023-12-27 13:34:20,373][20381] Updated weights for policy 1, policy_version 1142444 (0.0006) [2023-12-27 13:34:20,397][20347] Updated weights for policy 0, policy_version 1143527 (0.0007) [2023-12-27 13:34:20,435][20381] Updated weights for policy 1, policy_version 1142454 (0.0007) [2023-12-27 13:34:21,109][20347] Updated weights for policy 0, policy_version 1143537 (0.0008) [2023-12-27 13:34:21,155][20381] Updated weights for policy 1, policy_version 1142464 (0.0009) [2023-12-27 13:34:21,173][20347] Updated weights for policy 0, policy_version 1143547 (0.0009) [2023-12-27 13:34:21,214][20381] Updated weights for policy 1, policy_version 1142474 (0.0007) [2023-12-27 13:34:21,240][20347] Updated weights for policy 0, policy_version 1143557 (0.0009) [2023-12-27 13:34:21,279][20381] Updated weights for policy 1, policy_version 1142484 (0.0010) [2023-12-27 13:34:21,304][20347] Updated weights for policy 0, policy_version 1143567 (0.0009) [2023-12-27 13:34:22,077][20381] Updated weights for policy 1, policy_version 1142494 (0.0008) [2023-12-27 13:34:22,091][20347] Updated weights for policy 0, policy_version 1143577 (0.0008) [2023-12-27 13:34:22,136][20381] Updated weights for policy 1, policy_version 1142504 (0.0006) [2023-12-27 13:34:22,152][20347] Updated weights for policy 0, policy_version 1143587 (0.0008) [2023-12-27 13:34:22,193][20381] Updated weights for policy 1, policy_version 1142514 (0.0007) [2023-12-27 13:34:22,212][20347] Updated weights for policy 0, policy_version 1143597 (0.0008) [2023-12-27 13:34:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 585326592. Throughput: 0: 10093.7, 1: 10225.3. Samples: 585294700. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:22,363][19170] Avg episode reward: [(0, '316.550'), (1, '657.330')] [2023-12-27 13:34:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001142520_292511744.pth... [2023-12-27 13:34:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001143600_292814848.pth... [2023-12-27 13:34:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001141336_292208640.pth [2023-12-27 13:34:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001142416_292511744.pth [2023-12-27 13:34:22,895][20381] Updated weights for policy 1, policy_version 1142524 (0.0008) [2023-12-27 13:34:22,926][20347] Updated weights for policy 0, policy_version 1143607 (0.0008) [2023-12-27 13:34:22,957][20381] Updated weights for policy 1, policy_version 1142534 (0.0006) [2023-12-27 13:34:22,985][20347] Updated weights for policy 0, policy_version 1143617 (0.0008) [2023-12-27 13:34:23,015][20381] Updated weights for policy 1, policy_version 1142544 (0.0006) [2023-12-27 13:34:23,049][20347] Updated weights for policy 0, policy_version 1143627 (0.0007) [2023-12-27 13:34:23,610][20381] Updated weights for policy 1, policy_version 1142554 (0.0007) [2023-12-27 13:34:23,659][20381] Updated weights for policy 1, policy_version 1142564 (0.0010) [2023-12-27 13:34:23,702][20347] Updated weights for policy 0, policy_version 1143637 (0.0007) [2023-12-27 13:34:23,707][20381] Updated weights for policy 1, policy_version 1142574 (0.0005) [2023-12-27 13:34:23,757][20381] Updated weights for policy 1, policy_version 1142584 (0.0009) [2023-12-27 13:34:23,761][20347] Updated weights for policy 0, policy_version 1143647 (0.0005) [2023-12-27 13:34:23,814][20347] Updated weights for policy 0, policy_version 1143657 (0.0005) [2023-12-27 13:34:24,384][20347] Updated weights for policy 0, policy_version 1143667 (0.0006) [2023-12-27 13:34:24,404][20381] Updated weights for policy 1, policy_version 1142594 (0.0010) [2023-12-27 13:34:24,435][20347] Updated weights for policy 0, policy_version 1143677 (0.0006) [2023-12-27 13:34:24,459][20381] Updated weights for policy 1, policy_version 1142604 (0.0009) [2023-12-27 13:34:24,484][20347] Updated weights for policy 0, policy_version 1143687 (0.0007) [2023-12-27 13:34:24,507][20381] Updated weights for policy 1, policy_version 1142614 (0.0008) [2023-12-27 13:34:25,063][20347] Updated weights for policy 0, policy_version 1143697 (0.0005) [2023-12-27 13:34:25,114][20347] Updated weights for policy 0, policy_version 1143707 (0.0005) [2023-12-27 13:34:25,163][20347] Updated weights for policy 0, policy_version 1143717 (0.0006) [2023-12-27 13:34:25,210][20347] Updated weights for policy 0, policy_version 1143727 (0.0006) [2023-12-27 13:34:25,229][20381] Updated weights for policy 1, policy_version 1142624 (0.0010) [2023-12-27 13:34:25,274][20381] Updated weights for policy 1, policy_version 1142634 (0.0010) [2023-12-27 13:34:25,325][20381] Updated weights for policy 1, policy_version 1142644 (0.0010) [2023-12-27 13:34:25,842][20347] Updated weights for policy 0, policy_version 1143737 (0.0009) [2023-12-27 13:34:25,899][20347] Updated weights for policy 0, policy_version 1143747 (0.0010) [2023-12-27 13:34:25,953][20347] Updated weights for policy 0, policy_version 1143757 (0.0010) [2023-12-27 13:34:25,974][20381] Updated weights for policy 1, policy_version 1142654 (0.0010) [2023-12-27 13:34:26,021][20381] Updated weights for policy 1, policy_version 1142664 (0.0010) [2023-12-27 13:34:26,076][20381] Updated weights for policy 1, policy_version 1142674 (0.0010) [2023-12-27 13:34:26,622][20347] Updated weights for policy 0, policy_version 1143767 (0.0010) [2023-12-27 13:34:26,682][20347] Updated weights for policy 0, policy_version 1143777 (0.0010) [2023-12-27 13:34:26,736][20381] Updated weights for policy 1, policy_version 1142684 (0.0008) [2023-12-27 13:34:26,743][20347] Updated weights for policy 0, policy_version 1143787 (0.0010) [2023-12-27 13:34:26,792][20381] Updated weights for policy 1, policy_version 1142694 (0.0006) [2023-12-27 13:34:26,847][20381] Updated weights for policy 1, policy_version 1142704 (0.0010) [2023-12-27 13:34:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 585424896. Throughput: 0: 10135.5, 1: 10185.6. Samples: 585416776. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:27,363][19170] Avg episode reward: [(0, '290.050'), (1, '686.770')] [2023-12-27 13:34:27,371][20347] Updated weights for policy 0, policy_version 1143797 (0.0010) [2023-12-27 13:34:27,418][20347] Updated weights for policy 0, policy_version 1143807 (0.0010) [2023-12-27 13:34:27,469][20347] Updated weights for policy 0, policy_version 1143817 (0.0010) [2023-12-27 13:34:27,477][20381] Updated weights for policy 1, policy_version 1142714 (0.0009) [2023-12-27 13:34:27,531][20381] Updated weights for policy 1, policy_version 1142724 (0.0005) [2023-12-27 13:34:27,590][20381] Updated weights for policy 1, policy_version 1142734 (0.0005) [2023-12-27 13:34:27,650][20381] Updated weights for policy 1, policy_version 1142744 (0.0006) [2023-12-27 13:34:28,155][20347] Updated weights for policy 0, policy_version 1143827 (0.0009) [2023-12-27 13:34:28,203][20381] Updated weights for policy 1, policy_version 1142754 (0.0007) [2023-12-27 13:34:28,212][20347] Updated weights for policy 0, policy_version 1143837 (0.0009) [2023-12-27 13:34:28,262][20381] Updated weights for policy 1, policy_version 1142764 (0.0008) [2023-12-27 13:34:28,267][20347] Updated weights for policy 0, policy_version 1143847 (0.0008) [2023-12-27 13:34:28,325][20381] Updated weights for policy 1, policy_version 1142774 (0.0007) [2023-12-27 13:34:28,959][20347] Updated weights for policy 0, policy_version 1143857 (0.0008) [2023-12-27 13:34:28,981][20381] Updated weights for policy 1, policy_version 1142784 (0.0006) [2023-12-27 13:34:29,011][20347] Updated weights for policy 0, policy_version 1143867 (0.0007) [2023-12-27 13:34:29,046][20381] Updated weights for policy 1, policy_version 1142794 (0.0005) [2023-12-27 13:34:29,069][20347] Updated weights for policy 0, policy_version 1143877 (0.0006) [2023-12-27 13:34:29,109][20381] Updated weights for policy 1, policy_version 1142804 (0.0006) [2023-12-27 13:34:29,117][20347] Updated weights for policy 0, policy_version 1143887 (0.0007) [2023-12-27 13:34:29,760][20381] Updated weights for policy 1, policy_version 1142814 (0.0008) [2023-12-27 13:34:29,813][20381] Updated weights for policy 1, policy_version 1142824 (0.0008) [2023-12-27 13:34:29,849][20347] Updated weights for policy 0, policy_version 1143897 (0.0007) [2023-12-27 13:34:29,874][20381] Updated weights for policy 1, policy_version 1142834 (0.0007) [2023-12-27 13:34:29,905][20347] Updated weights for policy 0, policy_version 1143907 (0.0007) [2023-12-27 13:34:29,972][20347] Updated weights for policy 0, policy_version 1143917 (0.0007) [2023-12-27 13:34:30,532][20381] Updated weights for policy 1, policy_version 1142844 (0.0006) [2023-12-27 13:34:30,581][20381] Updated weights for policy 1, policy_version 1142854 (0.0008) [2023-12-27 13:34:30,610][20347] Updated weights for policy 0, policy_version 1143927 (0.0007) [2023-12-27 13:34:30,632][20381] Updated weights for policy 1, policy_version 1142864 (0.0007) [2023-12-27 13:34:30,664][20347] Updated weights for policy 0, policy_version 1143937 (0.0006) [2023-12-27 13:34:30,719][20347] Updated weights for policy 0, policy_version 1143947 (0.0007) [2023-12-27 13:34:31,345][20381] Updated weights for policy 1, policy_version 1142874 (0.0006) [2023-12-27 13:34:31,399][20347] Updated weights for policy 0, policy_version 1143957 (0.0007) [2023-12-27 13:34:31,411][20381] Updated weights for policy 1, policy_version 1142884 (0.0009) [2023-12-27 13:34:31,445][20347] Updated weights for policy 0, policy_version 1143967 (0.0005) [2023-12-27 13:34:31,471][20381] Updated weights for policy 1, policy_version 1142894 (0.0008) [2023-12-27 13:34:31,498][20347] Updated weights for policy 0, policy_version 1143977 (0.0006) [2023-12-27 13:34:31,527][20381] Updated weights for policy 1, policy_version 1142904 (0.0006) [2023-12-27 13:34:32,147][20347] Updated weights for policy 0, policy_version 1143987 (0.0006) [2023-12-27 13:34:32,197][20347] Updated weights for policy 0, policy_version 1143997 (0.0009) [2023-12-27 13:34:32,246][20347] Updated weights for policy 0, policy_version 1144007 (0.0007) [2023-12-27 13:34:32,271][20381] Updated weights for policy 1, policy_version 1142914 (0.0008) [2023-12-27 13:34:32,320][20381] Updated weights for policy 1, policy_version 1142924 (0.0007) [2023-12-27 13:34:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 585531392. Throughput: 0: 10178.2, 1: 10176.1. Samples: 585542356. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:32,363][19170] Avg episode reward: [(0, '308.530'), (1, '608.250')] [2023-12-27 13:34:32,383][20381] Updated weights for policy 1, policy_version 1142934 (0.0007) [2023-12-27 13:34:32,926][20347] Updated weights for policy 0, policy_version 1144017 (0.0006) [2023-12-27 13:34:32,977][20347] Updated weights for policy 0, policy_version 1144027 (0.0005) [2023-12-27 13:34:33,027][20347] Updated weights for policy 0, policy_version 1144037 (0.0007) [2023-12-27 13:34:33,051][20381] Updated weights for policy 1, policy_version 1142944 (0.0007) [2023-12-27 13:34:33,077][20347] Updated weights for policy 0, policy_version 1144047 (0.0005) [2023-12-27 13:34:33,096][20381] Updated weights for policy 1, policy_version 1142954 (0.0007) [2023-12-27 13:34:33,150][20381] Updated weights for policy 1, policy_version 1142964 (0.0007) [2023-12-27 13:34:33,685][20347] Updated weights for policy 0, policy_version 1144057 (0.0010) [2023-12-27 13:34:33,737][20347] Updated weights for policy 0, policy_version 1144067 (0.0010) [2023-12-27 13:34:33,782][20347] Updated weights for policy 0, policy_version 1144077 (0.0007) [2023-12-27 13:34:33,784][20381] Updated weights for policy 1, policy_version 1142974 (0.0007) [2023-12-27 13:34:33,828][20381] Updated weights for policy 1, policy_version 1142984 (0.0009) [2023-12-27 13:34:33,882][20381] Updated weights for policy 1, policy_version 1142994 (0.0007) [2023-12-27 13:34:34,475][20347] Updated weights for policy 0, policy_version 1144087 (0.0009) [2023-12-27 13:34:34,525][20347] Updated weights for policy 0, policy_version 1144097 (0.0010) [2023-12-27 13:34:34,567][20381] Updated weights for policy 1, policy_version 1143004 (0.0007) [2023-12-27 13:34:34,583][20347] Updated weights for policy 0, policy_version 1144107 (0.0008) [2023-12-27 13:34:34,620][20381] Updated weights for policy 1, policy_version 1143014 (0.0006) [2023-12-27 13:34:34,678][20381] Updated weights for policy 1, policy_version 1143024 (0.0008) [2023-12-27 13:34:35,239][20347] Updated weights for policy 0, policy_version 1144117 (0.0005) [2023-12-27 13:34:35,297][20347] Updated weights for policy 0, policy_version 1144127 (0.0006) [2023-12-27 13:34:35,341][20347] Updated weights for policy 0, policy_version 1144137 (0.0007) [2023-12-27 13:34:35,398][20381] Updated weights for policy 1, policy_version 1143034 (0.0007) [2023-12-27 13:34:35,454][20381] Updated weights for policy 1, policy_version 1143044 (0.0008) [2023-12-27 13:34:35,516][20381] Updated weights for policy 1, policy_version 1143054 (0.0007) [2023-12-27 13:34:35,571][20381] Updated weights for policy 1, policy_version 1143064 (0.0007) [2023-12-27 13:34:35,978][20347] Updated weights for policy 0, policy_version 1144147 (0.0007) [2023-12-27 13:34:36,035][20347] Updated weights for policy 0, policy_version 1144157 (0.0007) [2023-12-27 13:34:36,093][20347] Updated weights for policy 0, policy_version 1144167 (0.0008) [2023-12-27 13:34:36,285][20381] Updated weights for policy 1, policy_version 1143074 (0.0007) [2023-12-27 13:34:36,348][20381] Updated weights for policy 1, policy_version 1143084 (0.0005) [2023-12-27 13:34:36,414][20381] Updated weights for policy 1, policy_version 1143094 (0.0006) [2023-12-27 13:34:36,757][20347] Updated weights for policy 0, policy_version 1144177 (0.0008) [2023-12-27 13:34:36,817][20347] Updated weights for policy 0, policy_version 1144187 (0.0009) [2023-12-27 13:34:36,867][20347] Updated weights for policy 0, policy_version 1144197 (0.0010) [2023-12-27 13:34:36,921][20347] Updated weights for policy 0, policy_version 1144207 (0.0006) [2023-12-27 13:34:36,977][20381] Updated weights for policy 1, policy_version 1143104 (0.0008) [2023-12-27 13:34:37,035][20381] Updated weights for policy 1, policy_version 1143114 (0.0005) [2023-12-27 13:34:37,093][20381] Updated weights for policy 1, policy_version 1143124 (0.0007) [2023-12-27 13:34:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 585637888. Throughput: 0: 10173.1, 1: 10165.5. Samples: 585604848. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:37,363][19170] Avg episode reward: [(0, '478.690'), (1, '627.720')] [2023-12-27 13:34:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001143128_292667392.pth... [2023-12-27 13:34:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001144208_292970496.pth... [2023-12-27 13:34:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001141944_292364288.pth [2023-12-27 13:34:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001143024_292667392.pth [2023-12-27 13:34:37,609][20347] Updated weights for policy 0, policy_version 1144217 (0.0007) [2023-12-27 13:34:37,671][20347] Updated weights for policy 0, policy_version 1144227 (0.0008) [2023-12-27 13:34:37,735][20347] Updated weights for policy 0, policy_version 1144237 (0.0008) [2023-12-27 13:34:37,745][20381] Updated weights for policy 1, policy_version 1143134 (0.0008) [2023-12-27 13:34:37,802][20381] Updated weights for policy 1, policy_version 1143144 (0.0007) [2023-12-27 13:34:37,859][20381] Updated weights for policy 1, policy_version 1143154 (0.0007) [2023-12-27 13:34:38,439][20347] Updated weights for policy 0, policy_version 1144247 (0.0008) [2023-12-27 13:34:38,503][20347] Updated weights for policy 0, policy_version 1144257 (0.0007) [2023-12-27 13:34:38,565][20381] Updated weights for policy 1, policy_version 1143164 (0.0008) [2023-12-27 13:34:38,566][20347] Updated weights for policy 0, policy_version 1144267 (0.0007) [2023-12-27 13:34:38,624][20381] Updated weights for policy 1, policy_version 1143174 (0.0006) [2023-12-27 13:34:38,687][20381] Updated weights for policy 1, policy_version 1143184 (0.0005) [2023-12-27 13:34:39,221][20381] Updated weights for policy 1, policy_version 1143194 (0.0006) [2023-12-27 13:34:39,282][20381] Updated weights for policy 1, policy_version 1143204 (0.0008) [2023-12-27 13:34:39,324][20347] Updated weights for policy 0, policy_version 1144277 (0.0008) [2023-12-27 13:34:39,340][20381] Updated weights for policy 1, policy_version 1143214 (0.0008) [2023-12-27 13:34:39,390][20347] Updated weights for policy 0, policy_version 1144287 (0.0008) [2023-12-27 13:34:39,413][20381] Updated weights for policy 1, policy_version 1143224 (0.0008) [2023-12-27 13:34:39,452][20347] Updated weights for policy 0, policy_version 1144297 (0.0008) [2023-12-27 13:34:40,134][20381] Updated weights for policy 1, policy_version 1143234 (0.0008) [2023-12-27 13:34:40,178][20347] Updated weights for policy 0, policy_version 1144307 (0.0008) [2023-12-27 13:34:40,196][20381] Updated weights for policy 1, policy_version 1143244 (0.0008) [2023-12-27 13:34:40,231][20347] Updated weights for policy 0, policy_version 1144317 (0.0008) [2023-12-27 13:34:40,255][20381] Updated weights for policy 1, policy_version 1143254 (0.0007) [2023-12-27 13:34:40,288][20347] Updated weights for policy 0, policy_version 1144327 (0.0006) [2023-12-27 13:34:40,877][20381] Updated weights for policy 1, policy_version 1143264 (0.0007) [2023-12-27 13:34:40,935][20381] Updated weights for policy 1, policy_version 1143274 (0.0005) [2023-12-27 13:34:40,989][20347] Updated weights for policy 0, policy_version 1144337 (0.0007) [2023-12-27 13:34:40,994][20381] Updated weights for policy 1, policy_version 1143284 (0.0007) [2023-12-27 13:34:41,049][20347] Updated weights for policy 0, policy_version 1144347 (0.0007) [2023-12-27 13:34:41,116][20347] Updated weights for policy 0, policy_version 1144357 (0.0007) [2023-12-27 13:34:41,180][20347] Updated weights for policy 0, policy_version 1144367 (0.0007) [2023-12-27 13:34:41,707][20381] Updated weights for policy 1, policy_version 1143294 (0.0007) [2023-12-27 13:34:41,773][20381] Updated weights for policy 1, policy_version 1143304 (0.0007) [2023-12-27 13:34:41,839][20381] Updated weights for policy 1, policy_version 1143314 (0.0008) [2023-12-27 13:34:41,946][20347] Updated weights for policy 0, policy_version 1144377 (0.0007) [2023-12-27 13:34:42,008][20347] Updated weights for policy 0, policy_version 1144387 (0.0007) [2023-12-27 13:34:42,066][20347] Updated weights for policy 0, policy_version 1144397 (0.0006) [2023-12-27 13:34:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 585736192. Throughput: 0: 10134.5, 1: 10234.5. Samples: 585727140. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:42,363][19170] Avg episode reward: [(0, '434.460'), (1, '666.970')] [2023-12-27 13:34:42,501][20381] Updated weights for policy 1, policy_version 1143324 (0.0008) [2023-12-27 13:34:42,555][20381] Updated weights for policy 1, policy_version 1143334 (0.0007) [2023-12-27 13:34:42,612][20381] Updated weights for policy 1, policy_version 1143344 (0.0005) [2023-12-27 13:34:42,785][20347] Updated weights for policy 0, policy_version 1144407 (0.0007) [2023-12-27 13:34:42,843][20347] Updated weights for policy 0, policy_version 1144417 (0.0007) [2023-12-27 13:34:42,894][20347] Updated weights for policy 0, policy_version 1144427 (0.0007) [2023-12-27 13:34:43,292][20381] Updated weights for policy 1, policy_version 1143354 (0.0010) [2023-12-27 13:34:43,353][20381] Updated weights for policy 1, policy_version 1143364 (0.0010) [2023-12-27 13:34:43,423][20381] Updated weights for policy 1, policy_version 1143375 (0.0008) [2023-12-27 13:34:43,552][20347] Updated weights for policy 0, policy_version 1144437 (0.0006) [2023-12-27 13:34:43,600][20347] Updated weights for policy 0, policy_version 1144447 (0.0005) [2023-12-27 13:34:43,649][20347] Updated weights for policy 0, policy_version 1144457 (0.0005) [2023-12-27 13:34:44,004][20381] Updated weights for policy 1, policy_version 1143385 (0.0006) [2023-12-27 13:34:44,066][20381] Updated weights for policy 1, policy_version 1143395 (0.0008) [2023-12-27 13:34:44,134][20381] Updated weights for policy 1, policy_version 1143405 (0.0008) [2023-12-27 13:34:44,196][20381] Updated weights for policy 1, policy_version 1143415 (0.0008) [2023-12-27 13:34:44,331][20347] Updated weights for policy 0, policy_version 1144467 (0.0007) [2023-12-27 13:34:44,386][20347] Updated weights for policy 0, policy_version 1144477 (0.0008) [2023-12-27 13:34:44,438][20347] Updated weights for policy 0, policy_version 1144487 (0.0008) [2023-12-27 13:34:44,832][20381] Updated weights for policy 1, policy_version 1143425 (0.0008) [2023-12-27 13:34:44,892][20381] Updated weights for policy 1, policy_version 1143435 (0.0007) [2023-12-27 13:34:44,951][20381] Updated weights for policy 1, policy_version 1143445 (0.0007) [2023-12-27 13:34:45,164][20347] Updated weights for policy 0, policy_version 1144497 (0.0008) [2023-12-27 13:34:45,224][20347] Updated weights for policy 0, policy_version 1144507 (0.0008) [2023-12-27 13:34:45,286][20347] Updated weights for policy 0, policy_version 1144517 (0.0008) [2023-12-27 13:34:45,339][20347] Updated weights for policy 0, policy_version 1144527 (0.0008) [2023-12-27 13:34:45,551][20381] Updated weights for policy 1, policy_version 1143455 (0.0008) [2023-12-27 13:34:45,607][20381] Updated weights for policy 1, policy_version 1143465 (0.0008) [2023-12-27 13:34:45,660][20381] Updated weights for policy 1, policy_version 1143475 (0.0007) [2023-12-27 13:34:46,056][20347] Updated weights for policy 0, policy_version 1144537 (0.0008) [2023-12-27 13:34:46,112][20347] Updated weights for policy 0, policy_version 1144547 (0.0007) [2023-12-27 13:34:46,164][20347] Updated weights for policy 0, policy_version 1144557 (0.0008) [2023-12-27 13:34:46,380][20381] Updated weights for policy 1, policy_version 1143485 (0.0008) [2023-12-27 13:34:46,428][20381] Updated weights for policy 1, policy_version 1143495 (0.0007) [2023-12-27 13:34:46,481][20381] Updated weights for policy 1, policy_version 1143505 (0.0007) [2023-12-27 13:34:46,812][20347] Updated weights for policy 0, policy_version 1144567 (0.0007) [2023-12-27 13:34:46,882][20347] Updated weights for policy 0, policy_version 1144577 (0.0009) [2023-12-27 13:34:46,943][20347] Updated weights for policy 0, policy_version 1144587 (0.0009) [2023-12-27 13:34:47,142][20381] Updated weights for policy 1, policy_version 1143515 (0.0008) [2023-12-27 13:34:47,192][20381] Updated weights for policy 1, policy_version 1143525 (0.0006) [2023-12-27 13:34:47,239][20381] Updated weights for policy 1, policy_version 1143535 (0.0007) [2023-12-27 13:34:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 585842688. Throughput: 0: 10139.0, 1: 10290.3. Samples: 585848584. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:47,363][19170] Avg episode reward: [(0, '470.760'), (1, '686.850')] [2023-12-27 13:34:47,655][20347] Updated weights for policy 0, policy_version 1144597 (0.0008) [2023-12-27 13:34:47,721][20347] Updated weights for policy 0, policy_version 1144607 (0.0007) [2023-12-27 13:34:47,781][20347] Updated weights for policy 0, policy_version 1144617 (0.0008) [2023-12-27 13:34:47,917][20381] Updated weights for policy 1, policy_version 1143545 (0.0007) [2023-12-27 13:34:47,965][20381] Updated weights for policy 1, policy_version 1143555 (0.0005) [2023-12-27 13:34:48,010][20381] Updated weights for policy 1, policy_version 1143565 (0.0006) [2023-12-27 13:34:48,058][20381] Updated weights for policy 1, policy_version 1143575 (0.0007) [2023-12-27 13:34:48,517][20347] Updated weights for policy 0, policy_version 1144627 (0.0007) [2023-12-27 13:34:48,572][20347] Updated weights for policy 0, policy_version 1144637 (0.0007) [2023-12-27 13:34:48,627][20347] Updated weights for policy 0, policy_version 1144647 (0.0007) [2023-12-27 13:34:48,716][20381] Updated weights for policy 1, policy_version 1143585 (0.0010) [2023-12-27 13:34:48,765][20381] Updated weights for policy 1, policy_version 1143595 (0.0010) [2023-12-27 13:34:48,819][20381] Updated weights for policy 1, policy_version 1143605 (0.0009) [2023-12-27 13:34:49,296][20347] Updated weights for policy 0, policy_version 1144657 (0.0008) [2023-12-27 13:34:49,362][20347] Updated weights for policy 0, policy_version 1144667 (0.0008) [2023-12-27 13:34:49,420][20347] Updated weights for policy 0, policy_version 1144677 (0.0012) [2023-12-27 13:34:49,476][20347] Updated weights for policy 0, policy_version 1144687 (0.0011) [2023-12-27 13:34:49,589][20381] Updated weights for policy 1, policy_version 1143615 (0.0008) [2023-12-27 13:34:49,649][20381] Updated weights for policy 1, policy_version 1143625 (0.0007) [2023-12-27 13:34:49,701][20381] Updated weights for policy 1, policy_version 1143635 (0.0007) [2023-12-27 13:34:50,163][20347] Updated weights for policy 0, policy_version 1144697 (0.0008) [2023-12-27 13:34:50,230][20347] Updated weights for policy 0, policy_version 1144707 (0.0008) [2023-12-27 13:34:50,286][20347] Updated weights for policy 0, policy_version 1144717 (0.0007) [2023-12-27 13:34:50,442][20381] Updated weights for policy 1, policy_version 1143645 (0.0008) [2023-12-27 13:34:50,490][20381] Updated weights for policy 1, policy_version 1143655 (0.0008) [2023-12-27 13:34:50,538][20381] Updated weights for policy 1, policy_version 1143665 (0.0009) [2023-12-27 13:34:50,864][20347] Updated weights for policy 0, policy_version 1144727 (0.0008) [2023-12-27 13:34:50,909][20347] Updated weights for policy 0, policy_version 1144737 (0.0007) [2023-12-27 13:34:50,958][20347] Updated weights for policy 0, policy_version 1144747 (0.0008) [2023-12-27 13:34:51,305][20381] Updated weights for policy 1, policy_version 1143675 (0.0007) [2023-12-27 13:34:51,371][20381] Updated weights for policy 1, policy_version 1143685 (0.0008) [2023-12-27 13:34:51,434][20381] Updated weights for policy 1, policy_version 1143695 (0.0006) [2023-12-27 13:34:51,741][20347] Updated weights for policy 0, policy_version 1144757 (0.0008) [2023-12-27 13:34:51,804][20347] Updated weights for policy 0, policy_version 1144767 (0.0009) [2023-12-27 13:34:51,861][20347] Updated weights for policy 0, policy_version 1144777 (0.0006) [2023-12-27 13:34:52,109][20381] Updated weights for policy 1, policy_version 1143705 (0.0006) [2023-12-27 13:34:52,163][20381] Updated weights for policy 1, policy_version 1143715 (0.0007) [2023-12-27 13:34:52,218][20381] Updated weights for policy 1, policy_version 1143725 (0.0007) [2023-12-27 13:34:52,282][20381] Updated weights for policy 1, policy_version 1143735 (0.0008) [2023-12-27 13:34:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 585940992. Throughput: 0: 10127.3, 1: 10246.2. Samples: 585908536. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:52,363][19170] Avg episode reward: [(0, '481.590'), (1, '686.410')] [2023-12-27 13:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001143736_292823040.pth... [2023-12-27 13:34:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001144784_293117952.pth... [2023-12-27 13:34:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001142520_292511744.pth [2023-12-27 13:34:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001143600_292814848.pth [2023-12-27 13:34:52,538][20347] Updated weights for policy 0, policy_version 1144787 (0.0007) [2023-12-27 13:34:52,593][20347] Updated weights for policy 0, policy_version 1144797 (0.0006) [2023-12-27 13:34:52,652][20347] Updated weights for policy 0, policy_version 1144807 (0.0006) [2023-12-27 13:34:53,011][20381] Updated weights for policy 1, policy_version 1143745 (0.0008) [2023-12-27 13:34:53,056][20381] Updated weights for policy 1, policy_version 1143755 (0.0008) [2023-12-27 13:34:53,107][20381] Updated weights for policy 1, policy_version 1143765 (0.0005) [2023-12-27 13:34:53,306][20347] Updated weights for policy 0, policy_version 1144817 (0.0010) [2023-12-27 13:34:53,359][20347] Updated weights for policy 0, policy_version 1144827 (0.0007) [2023-12-27 13:34:53,413][20347] Updated weights for policy 0, policy_version 1144837 (0.0007) [2023-12-27 13:34:53,462][20347] Updated weights for policy 0, policy_version 1144847 (0.0007) [2023-12-27 13:34:53,706][20381] Updated weights for policy 1, policy_version 1143775 (0.0006) [2023-12-27 13:34:53,754][20381] Updated weights for policy 1, policy_version 1143785 (0.0010) [2023-12-27 13:34:53,798][20381] Updated weights for policy 1, policy_version 1143795 (0.0010) [2023-12-27 13:34:54,123][20347] Updated weights for policy 0, policy_version 1144857 (0.0007) [2023-12-27 13:34:54,178][20347] Updated weights for policy 0, policy_version 1144867 (0.0007) [2023-12-27 13:34:54,227][20347] Updated weights for policy 0, policy_version 1144877 (0.0008) [2023-12-27 13:34:54,449][20381] Updated weights for policy 1, policy_version 1143805 (0.0011) [2023-12-27 13:34:54,511][20381] Updated weights for policy 1, policy_version 1143815 (0.0010) [2023-12-27 13:34:54,572][20381] Updated weights for policy 1, policy_version 1143825 (0.0010) [2023-12-27 13:34:54,920][20347] Updated weights for policy 0, policy_version 1144887 (0.0009) [2023-12-27 13:34:54,979][20347] Updated weights for policy 0, policy_version 1144897 (0.0006) [2023-12-27 13:34:55,039][20347] Updated weights for policy 0, policy_version 1144907 (0.0007) [2023-12-27 13:34:55,225][20381] Updated weights for policy 1, policy_version 1143835 (0.0011) [2023-12-27 13:34:55,274][20381] Updated weights for policy 1, policy_version 1143845 (0.0008) [2023-12-27 13:34:55,324][20381] Updated weights for policy 1, policy_version 1143855 (0.0007) [2023-12-27 13:34:55,670][20347] Updated weights for policy 0, policy_version 1144917 (0.0006) [2023-12-27 13:34:55,715][20347] Updated weights for policy 0, policy_version 1144927 (0.0006) [2023-12-27 13:34:55,760][20347] Updated weights for policy 0, policy_version 1144937 (0.0010) [2023-12-27 13:34:55,970][20381] Updated weights for policy 1, policy_version 1143865 (0.0008) [2023-12-27 13:34:56,021][20381] Updated weights for policy 1, policy_version 1143875 (0.0009) [2023-12-27 13:34:56,068][20381] Updated weights for policy 1, policy_version 1143885 (0.0007) [2023-12-27 13:34:56,124][20381] Updated weights for policy 1, policy_version 1143895 (0.0005) [2023-12-27 13:34:56,442][20347] Updated weights for policy 0, policy_version 1144947 (0.0009) [2023-12-27 13:34:56,501][20347] Updated weights for policy 0, policy_version 1144957 (0.0008) [2023-12-27 13:34:56,554][20347] Updated weights for policy 0, policy_version 1144967 (0.0010) [2023-12-27 13:34:56,715][20381] Updated weights for policy 1, policy_version 1143905 (0.0010) [2023-12-27 13:34:56,763][20381] Updated weights for policy 1, policy_version 1143915 (0.0010) [2023-12-27 13:34:56,814][20381] Updated weights for policy 1, policy_version 1143925 (0.0010) [2023-12-27 13:34:57,202][20347] Updated weights for policy 0, policy_version 1144977 (0.0010) [2023-12-27 13:34:57,251][20347] Updated weights for policy 0, policy_version 1144987 (0.0010) [2023-12-27 13:34:57,302][20347] Updated weights for policy 0, policy_version 1144997 (0.0010) [2023-12-27 13:34:57,357][20347] Updated weights for policy 0, policy_version 1145007 (0.0010) [2023-12-27 13:34:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 586047488. Throughput: 0: 10185.3, 1: 10233.8. Samples: 586033068. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:34:57,363][19170] Avg episode reward: [(0, '365.890'), (1, '725.700')] [2023-12-27 13:34:57,467][20381] Updated weights for policy 1, policy_version 1143935 (0.0010) [2023-12-27 13:34:57,514][20381] Updated weights for policy 1, policy_version 1143945 (0.0010) [2023-12-27 13:34:57,565][20381] Updated weights for policy 1, policy_version 1143955 (0.0007) [2023-12-27 13:34:58,038][20347] Updated weights for policy 0, policy_version 1145017 (0.0010) [2023-12-27 13:34:58,085][20347] Updated weights for policy 0, policy_version 1145027 (0.0010) [2023-12-27 13:34:58,139][20347] Updated weights for policy 0, policy_version 1145037 (0.0010) [2023-12-27 13:34:58,224][20381] Updated weights for policy 1, policy_version 1143965 (0.0009) [2023-12-27 13:34:58,287][20381] Updated weights for policy 1, policy_version 1143975 (0.0009) [2023-12-27 13:34:58,362][20381] Updated weights for policy 1, policy_version 1143985 (0.0009) [2023-12-27 13:34:58,951][20347] Updated weights for policy 0, policy_version 1145047 (0.0010) [2023-12-27 13:34:59,005][20347] Updated weights for policy 0, policy_version 1145057 (0.0006) [2023-12-27 13:34:59,062][20347] Updated weights for policy 0, policy_version 1145067 (0.0005) [2023-12-27 13:34:59,112][20381] Updated weights for policy 1, policy_version 1143995 (0.0008) [2023-12-27 13:34:59,171][20381] Updated weights for policy 1, policy_version 1144005 (0.0008) [2023-12-27 13:34:59,236][20381] Updated weights for policy 1, policy_version 1144015 (0.0008) [2023-12-27 13:34:59,790][20347] Updated weights for policy 0, policy_version 1145077 (0.0007) [2023-12-27 13:34:59,855][20347] Updated weights for policy 0, policy_version 1145087 (0.0008) [2023-12-27 13:34:59,889][20381] Updated weights for policy 1, policy_version 1144025 (0.0009) [2023-12-27 13:34:59,910][20347] Updated weights for policy 0, policy_version 1145097 (0.0006) [2023-12-27 13:34:59,948][20381] Updated weights for policy 1, policy_version 1144035 (0.0008) [2023-12-27 13:35:00,012][20381] Updated weights for policy 1, policy_version 1144045 (0.0008) [2023-12-27 13:35:00,063][20381] Updated weights for policy 1, policy_version 1144055 (0.0008) [2023-12-27 13:35:00,547][20347] Updated weights for policy 0, policy_version 1145107 (0.0008) [2023-12-27 13:35:00,605][20347] Updated weights for policy 0, policy_version 1145117 (0.0010) [2023-12-27 13:35:00,661][20347] Updated weights for policy 0, policy_version 1145127 (0.0010) [2023-12-27 13:35:00,785][20381] Updated weights for policy 1, policy_version 1144065 (0.0008) [2023-12-27 13:35:00,833][20381] Updated weights for policy 1, policy_version 1144075 (0.0007) [2023-12-27 13:35:00,880][20381] Updated weights for policy 1, policy_version 1144085 (0.0007) [2023-12-27 13:35:01,360][20347] Updated weights for policy 0, policy_version 1145137 (0.0010) [2023-12-27 13:35:01,420][20347] Updated weights for policy 0, policy_version 1145147 (0.0006) [2023-12-27 13:35:01,470][20347] Updated weights for policy 0, policy_version 1145157 (0.0005) [2023-12-27 13:35:01,518][20347] Updated weights for policy 0, policy_version 1145167 (0.0006) [2023-12-27 13:35:01,688][20381] Updated weights for policy 1, policy_version 1144095 (0.0007) [2023-12-27 13:35:01,752][20381] Updated weights for policy 1, policy_version 1144105 (0.0008) [2023-12-27 13:35:01,808][20381] Updated weights for policy 1, policy_version 1144115 (0.0007) [2023-12-27 13:35:02,229][20347] Updated weights for policy 0, policy_version 1145177 (0.0007) [2023-12-27 13:35:02,291][20347] Updated weights for policy 0, policy_version 1145187 (0.0009) [2023-12-27 13:35:02,349][20347] Updated weights for policy 0, policy_version 1145197 (0.0010) [2023-12-27 13:35:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 586137600. Throughput: 0: 10142.6, 1: 10240.6. Samples: 586153452. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:02,363][19170] Avg episode reward: [(0, '493.620'), (1, '696.520')] [2023-12-27 13:35:02,533][20381] Updated weights for policy 1, policy_version 1144125 (0.0007) [2023-12-27 13:35:02,581][20381] Updated weights for policy 1, policy_version 1144135 (0.0008) [2023-12-27 13:35:02,630][20381] Updated weights for policy 1, policy_version 1144145 (0.0009) [2023-12-27 13:35:03,003][20347] Updated weights for policy 0, policy_version 1145207 (0.0011) [2023-12-27 13:35:03,058][20347] Updated weights for policy 0, policy_version 1145217 (0.0010) [2023-12-27 13:35:03,110][20347] Updated weights for policy 0, policy_version 1145227 (0.0010) [2023-12-27 13:35:03,331][20381] Updated weights for policy 1, policy_version 1144155 (0.0006) [2023-12-27 13:35:03,376][20381] Updated weights for policy 1, policy_version 1144165 (0.0008) [2023-12-27 13:35:03,430][20381] Updated weights for policy 1, policy_version 1144175 (0.0007) [2023-12-27 13:35:03,783][20347] Updated weights for policy 0, policy_version 1145237 (0.0010) [2023-12-27 13:35:03,856][20347] Updated weights for policy 0, policy_version 1145249 (0.0006) [2023-12-27 13:35:03,917][20347] Updated weights for policy 0, policy_version 1145259 (0.0009) [2023-12-27 13:35:04,100][20381] Updated weights for policy 1, policy_version 1144185 (0.0007) [2023-12-27 13:35:04,159][20381] Updated weights for policy 1, policy_version 1144195 (0.0008) [2023-12-27 13:35:04,222][20381] Updated weights for policy 1, policy_version 1144205 (0.0008) [2023-12-27 13:35:04,286][20381] Updated weights for policy 1, policy_version 1144215 (0.0009) [2023-12-27 13:35:04,589][20347] Updated weights for policy 0, policy_version 1145269 (0.0007) [2023-12-27 13:35:04,650][20347] Updated weights for policy 0, policy_version 1145279 (0.0006) [2023-12-27 13:35:04,707][20347] Updated weights for policy 0, policy_version 1145289 (0.0007) [2023-12-27 13:35:04,946][20381] Updated weights for policy 1, policy_version 1144225 (0.0007) [2023-12-27 13:35:05,000][20381] Updated weights for policy 1, policy_version 1144235 (0.0008) [2023-12-27 13:35:05,058][20381] Updated weights for policy 1, policy_version 1144245 (0.0007) [2023-12-27 13:35:05,352][20347] Updated weights for policy 0, policy_version 1145299 (0.0005) [2023-12-27 13:35:05,412][20347] Updated weights for policy 0, policy_version 1145309 (0.0006) [2023-12-27 13:35:05,468][20347] Updated weights for policy 0, policy_version 1145319 (0.0008) [2023-12-27 13:35:05,701][20381] Updated weights for policy 1, policy_version 1144255 (0.0005) [2023-12-27 13:35:05,746][20381] Updated weights for policy 1, policy_version 1144265 (0.0005) [2023-12-27 13:35:05,797][20381] Updated weights for policy 1, policy_version 1144275 (0.0005) [2023-12-27 13:35:06,177][20347] Updated weights for policy 0, policy_version 1145329 (0.0007) [2023-12-27 13:35:06,237][20347] Updated weights for policy 0, policy_version 1145339 (0.0007) [2023-12-27 13:35:06,301][20347] Updated weights for policy 0, policy_version 1145349 (0.0008) [2023-12-27 13:35:06,362][20347] Updated weights for policy 0, policy_version 1145359 (0.0007) [2023-12-27 13:35:06,475][20381] Updated weights for policy 1, policy_version 1144285 (0.0007) [2023-12-27 13:35:06,531][20381] Updated weights for policy 1, policy_version 1144295 (0.0008) [2023-12-27 13:35:06,595][20381] Updated weights for policy 1, policy_version 1144305 (0.0008) [2023-12-27 13:35:06,965][20347] Updated weights for policy 0, policy_version 1145369 (0.0008) [2023-12-27 13:35:07,017][20347] Updated weights for policy 0, policy_version 1145379 (0.0008) [2023-12-27 13:35:07,075][20347] Updated weights for policy 0, policy_version 1145389 (0.0006) [2023-12-27 13:35:07,315][20381] Updated weights for policy 1, policy_version 1144315 (0.0008) [2023-12-27 13:35:07,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 586244096. Throughput: 0: 10175.5, 1: 10285.5. Samples: 586215452. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:07,364][19170] Avg episode reward: [(0, '403.260'), (1, '706.260')] [2023-12-27 13:35:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001145392_293273600.pth... [2023-12-27 13:35:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001144208_292970496.pth [2023-12-27 13:35:07,383][20381] Updated weights for policy 1, policy_version 1144325 (0.0007) [2023-12-27 13:35:07,443][20381] Updated weights for policy 1, policy_version 1144335 (0.0008) [2023-12-27 13:35:07,488][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001144344_292978688.pth... [2023-12-27 13:35:07,492][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001143128_292667392.pth [2023-12-27 13:35:07,780][20347] Updated weights for policy 0, policy_version 1145399 (0.0006) [2023-12-27 13:35:07,841][20347] Updated weights for policy 0, policy_version 1145409 (0.0007) [2023-12-27 13:35:07,902][20347] Updated weights for policy 0, policy_version 1145419 (0.0007) [2023-12-27 13:35:08,084][20381] Updated weights for policy 1, policy_version 1144345 (0.0008) [2023-12-27 13:35:08,144][20381] Updated weights for policy 1, policy_version 1144355 (0.0005) [2023-12-27 13:35:08,201][20381] Updated weights for policy 1, policy_version 1144365 (0.0006) [2023-12-27 13:35:08,253][20381] Updated weights for policy 1, policy_version 1144375 (0.0009) [2023-12-27 13:35:08,629][20347] Updated weights for policy 0, policy_version 1145429 (0.0007) [2023-12-27 13:35:08,687][20347] Updated weights for policy 0, policy_version 1145439 (0.0007) [2023-12-27 13:35:08,741][20347] Updated weights for policy 0, policy_version 1145449 (0.0008) [2023-12-27 13:35:08,906][20381] Updated weights for policy 1, policy_version 1144385 (0.0010) [2023-12-27 13:35:08,959][20381] Updated weights for policy 1, policy_version 1144395 (0.0007) [2023-12-27 13:35:09,016][20381] Updated weights for policy 1, policy_version 1144405 (0.0010) [2023-12-27 13:35:09,478][20347] Updated weights for policy 0, policy_version 1145459 (0.0007) [2023-12-27 13:35:09,538][20347] Updated weights for policy 0, policy_version 1145469 (0.0007) [2023-12-27 13:35:09,595][20347] Updated weights for policy 0, policy_version 1145479 (0.0008) [2023-12-27 13:35:09,727][20381] Updated weights for policy 1, policy_version 1144415 (0.0008) [2023-12-27 13:35:09,736][20292] KL-divergence is very high: 129.4201 [2023-12-27 13:35:09,793][20381] Updated weights for policy 1, policy_version 1144425 (0.0007) [2023-12-27 13:35:09,857][20381] Updated weights for policy 1, policy_version 1144435 (0.0007) [2023-12-27 13:35:10,319][20347] Updated weights for policy 0, policy_version 1145489 (0.0008) [2023-12-27 13:35:10,379][20347] Updated weights for policy 0, policy_version 1145499 (0.0008) [2023-12-27 13:35:10,441][20347] Updated weights for policy 0, policy_version 1145509 (0.0008) [2023-12-27 13:35:10,498][20347] Updated weights for policy 0, policy_version 1145519 (0.0008) [2023-12-27 13:35:10,534][20381] Updated weights for policy 1, policy_version 1144445 (0.0008) [2023-12-27 13:35:10,593][20381] Updated weights for policy 1, policy_version 1144455 (0.0006) [2023-12-27 13:35:10,655][20381] Updated weights for policy 1, policy_version 1144465 (0.0007) [2023-12-27 13:35:11,114][20347] Updated weights for policy 0, policy_version 1145529 (0.0008) [2023-12-27 13:35:11,183][20347] Updated weights for policy 0, policy_version 1145539 (0.0008) [2023-12-27 13:35:11,239][20347] Updated weights for policy 0, policy_version 1145549 (0.0008) [2023-12-27 13:35:11,390][20381] Updated weights for policy 1, policy_version 1144475 (0.0008) [2023-12-27 13:35:11,454][20381] Updated weights for policy 1, policy_version 1144485 (0.0008) [2023-12-27 13:35:11,524][20381] Updated weights for policy 1, policy_version 1144495 (0.0008) [2023-12-27 13:35:11,926][20347] Updated weights for policy 0, policy_version 1145559 (0.0008) [2023-12-27 13:35:11,984][20347] Updated weights for policy 0, policy_version 1145569 (0.0007) [2023-12-27 13:35:12,043][20347] Updated weights for policy 0, policy_version 1145579 (0.0007) [2023-12-27 13:35:12,262][20381] Updated weights for policy 1, policy_version 1144505 (0.0009) [2023-12-27 13:35:12,325][20381] Updated weights for policy 1, policy_version 1144515 (0.0008) [2023-12-27 13:35:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 586342400. Throughput: 0: 10145.1, 1: 10299.4. Samples: 586336780. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:12,363][19170] Avg episode reward: [(0, '383.650'), (1, '637.680')] [2023-12-27 13:35:12,384][20381] Updated weights for policy 1, policy_version 1144525 (0.0008) [2023-12-27 13:35:12,446][20381] Updated weights for policy 1, policy_version 1144535 (0.0007) [2023-12-27 13:35:12,791][20347] Updated weights for policy 0, policy_version 1145589 (0.0008) [2023-12-27 13:35:12,847][20347] Updated weights for policy 0, policy_version 1145599 (0.0009) [2023-12-27 13:35:12,897][20347] Updated weights for policy 0, policy_version 1145609 (0.0007) [2023-12-27 13:35:13,027][20381] Updated weights for policy 1, policy_version 1144545 (0.0005) [2023-12-27 13:35:13,089][20381] Updated weights for policy 1, policy_version 1144555 (0.0005) [2023-12-27 13:35:13,152][20381] Updated weights for policy 1, policy_version 1144565 (0.0005) [2023-12-27 13:35:13,617][20347] Updated weights for policy 0, policy_version 1145619 (0.0007) [2023-12-27 13:35:13,677][20347] Updated weights for policy 0, policy_version 1145629 (0.0008) [2023-12-27 13:35:13,731][20347] Updated weights for policy 0, policy_version 1145639 (0.0008) [2023-12-27 13:35:13,769][20381] Updated weights for policy 1, policy_version 1144575 (0.0008) [2023-12-27 13:35:13,824][20381] Updated weights for policy 1, policy_version 1144585 (0.0006) [2023-12-27 13:35:13,871][20381] Updated weights for policy 1, policy_version 1144595 (0.0007) [2023-12-27 13:35:14,394][20347] Updated weights for policy 0, policy_version 1145649 (0.0010) [2023-12-27 13:35:14,455][20347] Updated weights for policy 0, policy_version 1145659 (0.0010) [2023-12-27 13:35:14,516][20347] Updated weights for policy 0, policy_version 1145669 (0.0010) [2023-12-27 13:35:14,522][20381] Updated weights for policy 1, policy_version 1144605 (0.0007) [2023-12-27 13:35:14,565][20347] Updated weights for policy 0, policy_version 1145679 (0.0010) [2023-12-27 13:35:14,570][20381] Updated weights for policy 1, policy_version 1144615 (0.0005) [2023-12-27 13:35:14,622][20381] Updated weights for policy 1, policy_version 1144625 (0.0007) [2023-12-27 13:35:15,272][20347] Updated weights for policy 0, policy_version 1145689 (0.0010) [2023-12-27 13:35:15,333][20347] Updated weights for policy 0, policy_version 1145699 (0.0010) [2023-12-27 13:35:15,334][20381] Updated weights for policy 1, policy_version 1144635 (0.0009) [2023-12-27 13:35:15,389][20381] Updated weights for policy 1, policy_version 1144645 (0.0009) [2023-12-27 13:35:15,394][20347] Updated weights for policy 0, policy_version 1145709 (0.0010) [2023-12-27 13:35:15,441][20381] Updated weights for policy 1, policy_version 1144655 (0.0005) [2023-12-27 13:35:16,059][20347] Updated weights for policy 0, policy_version 1145719 (0.0008) [2023-12-27 13:35:16,104][20381] Updated weights for policy 1, policy_version 1144665 (0.0007) [2023-12-27 13:35:16,106][20347] Updated weights for policy 0, policy_version 1145729 (0.0007) [2023-12-27 13:35:16,156][20347] Updated weights for policy 0, policy_version 1145739 (0.0006) [2023-12-27 13:35:16,162][20381] Updated weights for policy 1, policy_version 1144675 (0.0006) [2023-12-27 13:35:16,224][20381] Updated weights for policy 1, policy_version 1144685 (0.0007) [2023-12-27 13:35:16,280][20381] Updated weights for policy 1, policy_version 1144695 (0.0007) [2023-12-27 13:35:16,847][20347] Updated weights for policy 0, policy_version 1145749 (0.0006) [2023-12-27 13:35:16,893][20347] Updated weights for policy 0, policy_version 1145759 (0.0005) [2023-12-27 13:35:16,946][20347] Updated weights for policy 0, policy_version 1145769 (0.0005) [2023-12-27 13:35:16,963][20381] Updated weights for policy 1, policy_version 1144705 (0.0007) [2023-12-27 13:35:17,023][20381] Updated weights for policy 1, policy_version 1144715 (0.0007) [2023-12-27 13:35:17,077][20381] Updated weights for policy 1, policy_version 1144725 (0.0008) [2023-12-27 13:35:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 586448896. Throughput: 0: 10087.9, 1: 10269.2. Samples: 586458424. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:17,363][19170] Avg episode reward: [(0, '567.010'), (1, '657.150')] [2023-12-27 13:35:17,524][20347] Updated weights for policy 0, policy_version 1145779 (0.0005) [2023-12-27 13:35:17,570][20347] Updated weights for policy 0, policy_version 1145789 (0.0005) [2023-12-27 13:35:17,619][20347] Updated weights for policy 0, policy_version 1145799 (0.0005) [2023-12-27 13:35:17,822][20381] Updated weights for policy 1, policy_version 1144735 (0.0007) [2023-12-27 13:35:17,881][20381] Updated weights for policy 1, policy_version 1144745 (0.0007) [2023-12-27 13:35:17,931][20381] Updated weights for policy 1, policy_version 1144755 (0.0009) [2023-12-27 13:35:18,220][20347] Updated weights for policy 0, policy_version 1145809 (0.0008) [2023-12-27 13:35:18,264][20347] Updated weights for policy 0, policy_version 1145819 (0.0010) [2023-12-27 13:35:18,311][20347] Updated weights for policy 0, policy_version 1145829 (0.0009) [2023-12-27 13:35:18,375][20347] Updated weights for policy 0, policy_version 1145839 (0.0008) [2023-12-27 13:35:18,605][20381] Updated weights for policy 1, policy_version 1144765 (0.0007) [2023-12-27 13:35:18,656][20381] Updated weights for policy 1, policy_version 1144775 (0.0008) [2023-12-27 13:35:18,704][20381] Updated weights for policy 1, policy_version 1144785 (0.0008) [2023-12-27 13:35:19,061][20347] Updated weights for policy 0, policy_version 1145849 (0.0007) [2023-12-27 13:35:19,117][20347] Updated weights for policy 0, policy_version 1145859 (0.0006) [2023-12-27 13:35:19,175][20347] Updated weights for policy 0, policy_version 1145869 (0.0008) [2023-12-27 13:35:19,459][20381] Updated weights for policy 1, policy_version 1144795 (0.0008) [2023-12-27 13:35:19,529][20381] Updated weights for policy 1, policy_version 1144805 (0.0006) [2023-12-27 13:35:19,597][20381] Updated weights for policy 1, policy_version 1144815 (0.0007) [2023-12-27 13:35:19,940][20347] Updated weights for policy 0, policy_version 1145879 (0.0007) [2023-12-27 13:35:20,009][20347] Updated weights for policy 0, policy_version 1145889 (0.0009) [2023-12-27 13:35:20,073][20347] Updated weights for policy 0, policy_version 1145899 (0.0008) [2023-12-27 13:35:20,309][20381] Updated weights for policy 1, policy_version 1144825 (0.0008) [2023-12-27 13:35:20,368][20381] Updated weights for policy 1, policy_version 1144835 (0.0008) [2023-12-27 13:35:20,422][20381] Updated weights for policy 1, policy_version 1144845 (0.0006) [2023-12-27 13:35:20,485][20381] Updated weights for policy 1, policy_version 1144855 (0.0008) [2023-12-27 13:35:20,773][20347] Updated weights for policy 0, policy_version 1145909 (0.0008) [2023-12-27 13:35:20,834][20347] Updated weights for policy 0, policy_version 1145919 (0.0008) [2023-12-27 13:35:20,894][20347] Updated weights for policy 0, policy_version 1145929 (0.0008) [2023-12-27 13:35:21,212][20381] Updated weights for policy 1, policy_version 1144865 (0.0007) [2023-12-27 13:35:21,280][20381] Updated weights for policy 1, policy_version 1144875 (0.0007) [2023-12-27 13:35:21,344][20381] Updated weights for policy 1, policy_version 1144885 (0.0008) [2023-12-27 13:35:21,628][20347] Updated weights for policy 0, policy_version 1145939 (0.0010) [2023-12-27 13:35:21,689][20347] Updated weights for policy 0, policy_version 1145949 (0.0007) [2023-12-27 13:35:21,762][20347] Updated weights for policy 0, policy_version 1145959 (0.0010) [2023-12-27 13:35:22,050][20381] Updated weights for policy 1, policy_version 1144895 (0.0007) [2023-12-27 13:35:22,104][20381] Updated weights for policy 1, policy_version 1144905 (0.0008) [2023-12-27 13:35:22,157][20381] Updated weights for policy 1, policy_version 1144915 (0.0007) [2023-12-27 13:35:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 586547200. Throughput: 0: 10087.0, 1: 10227.6. Samples: 586519008. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:22,363][19170] Avg episode reward: [(0, '483.360'), (1, '745.280')] [2023-12-27 13:35:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001144920_293126144.pth... [2023-12-27 13:35:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001145968_293421056.pth... [2023-12-27 13:35:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001143736_292823040.pth [2023-12-27 13:35:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001144784_293117952.pth [2023-12-27 13:35:22,466][20347] Updated weights for policy 0, policy_version 1145969 (0.0010) [2023-12-27 13:35:22,521][20347] Updated weights for policy 0, policy_version 1145979 (0.0006) [2023-12-27 13:35:22,580][20347] Updated weights for policy 0, policy_version 1145989 (0.0008) [2023-12-27 13:35:22,647][20347] Updated weights for policy 0, policy_version 1145999 (0.0008) [2023-12-27 13:35:22,910][20381] Updated weights for policy 1, policy_version 1144925 (0.0006) [2023-12-27 13:35:22,970][20381] Updated weights for policy 1, policy_version 1144935 (0.0007) [2023-12-27 13:35:23,027][20381] Updated weights for policy 1, policy_version 1144945 (0.0005) [2023-12-27 13:35:23,286][20347] Updated weights for policy 0, policy_version 1146009 (0.0008) [2023-12-27 13:35:23,338][20347] Updated weights for policy 0, policy_version 1146019 (0.0008) [2023-12-27 13:35:23,398][20347] Updated weights for policy 0, policy_version 1146029 (0.0007) [2023-12-27 13:35:23,700][20381] Updated weights for policy 1, policy_version 1144955 (0.0007) [2023-12-27 13:35:23,762][20381] Updated weights for policy 1, policy_version 1144965 (0.0005) [2023-12-27 13:35:23,811][20381] Updated weights for policy 1, policy_version 1144975 (0.0005) [2023-12-27 13:35:24,145][20347] Updated weights for policy 0, policy_version 1146039 (0.0007) [2023-12-27 13:35:24,205][20347] Updated weights for policy 0, policy_version 1146049 (0.0008) [2023-12-27 13:35:24,260][20347] Updated weights for policy 0, policy_version 1146059 (0.0005) [2023-12-27 13:35:24,395][20381] Updated weights for policy 1, policy_version 1144985 (0.0005) [2023-12-27 13:35:24,452][20381] Updated weights for policy 1, policy_version 1144995 (0.0006) [2023-12-27 13:35:24,509][20381] Updated weights for policy 1, policy_version 1145005 (0.0006) [2023-12-27 13:35:24,574][20381] Updated weights for policy 1, policy_version 1145015 (0.0007) [2023-12-27 13:35:24,877][20347] Updated weights for policy 0, policy_version 1146069 (0.0006) [2023-12-27 13:35:24,935][20347] Updated weights for policy 0, policy_version 1146079 (0.0009) [2023-12-27 13:35:24,994][20347] Updated weights for policy 0, policy_version 1146089 (0.0008) [2023-12-27 13:35:25,103][20381] Updated weights for policy 1, policy_version 1145025 (0.0006) [2023-12-27 13:35:25,157][20381] Updated weights for policy 1, policy_version 1145035 (0.0011) [2023-12-27 13:35:25,207][20381] Updated weights for policy 1, policy_version 1145045 (0.0010) [2023-12-27 13:35:25,701][20347] Updated weights for policy 0, policy_version 1146099 (0.0008) [2023-12-27 13:35:25,758][20347] Updated weights for policy 0, policy_version 1146109 (0.0006) [2023-12-27 13:35:25,805][20381] Updated weights for policy 1, policy_version 1145055 (0.0008) [2023-12-27 13:35:25,818][20347] Updated weights for policy 0, policy_version 1146119 (0.0005) [2023-12-27 13:35:25,854][20381] Updated weights for policy 1, policy_version 1145065 (0.0005) [2023-12-27 13:35:25,904][20381] Updated weights for policy 1, policy_version 1145075 (0.0006) [2023-12-27 13:35:26,456][20347] Updated weights for policy 0, policy_version 1146129 (0.0006) [2023-12-27 13:35:26,505][20381] Updated weights for policy 1, policy_version 1145085 (0.0007) [2023-12-27 13:35:26,508][20347] Updated weights for policy 0, policy_version 1146139 (0.0007) [2023-12-27 13:35:26,556][20381] Updated weights for policy 1, policy_version 1145095 (0.0006) [2023-12-27 13:35:26,558][20347] Updated weights for policy 0, policy_version 1146149 (0.0006) [2023-12-27 13:35:26,610][20381] Updated weights for policy 1, policy_version 1145105 (0.0005) [2023-12-27 13:35:26,617][20347] Updated weights for policy 0, policy_version 1146159 (0.0007) [2023-12-27 13:35:27,190][20347] Updated weights for policy 0, policy_version 1146169 (0.0006) [2023-12-27 13:35:27,242][20347] Updated weights for policy 0, policy_version 1146179 (0.0007) [2023-12-27 13:35:27,298][20381] Updated weights for policy 1, policy_version 1145115 (0.0006) [2023-12-27 13:35:27,299][20347] Updated weights for policy 0, policy_version 1146189 (0.0008) [2023-12-27 13:35:27,351][20381] Updated weights for policy 1, policy_version 1145125 (0.0005) [2023-12-27 13:35:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.5). Total num frames: 586653696. Throughput: 0: 10078.0, 1: 10259.9. Samples: 586642344. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:27,363][19170] Avg episode reward: [(0, '589.550'), (1, '598.500')] [2023-12-27 13:35:27,405][20381] Updated weights for policy 1, policy_version 1145135 (0.0005) [2023-12-27 13:35:27,924][20347] Updated weights for policy 0, policy_version 1146199 (0.0007) [2023-12-27 13:35:27,978][20347] Updated weights for policy 0, policy_version 1146209 (0.0007) [2023-12-27 13:35:28,025][20381] Updated weights for policy 1, policy_version 1145145 (0.0006) [2023-12-27 13:35:28,029][20347] Updated weights for policy 0, policy_version 1146219 (0.0006) [2023-12-27 13:35:28,082][20381] Updated weights for policy 1, policy_version 1145155 (0.0010) [2023-12-27 13:35:28,140][20381] Updated weights for policy 1, policy_version 1145165 (0.0010) [2023-12-27 13:35:28,195][20381] Updated weights for policy 1, policy_version 1145175 (0.0010) [2023-12-27 13:35:28,674][20347] Updated weights for policy 0, policy_version 1146229 (0.0006) [2023-12-27 13:35:28,720][20347] Updated weights for policy 0, policy_version 1146239 (0.0007) [2023-12-27 13:35:28,775][20347] Updated weights for policy 0, policy_version 1146249 (0.0007) [2023-12-27 13:35:28,798][20381] Updated weights for policy 1, policy_version 1145185 (0.0006) [2023-12-27 13:35:28,860][20381] Updated weights for policy 1, policy_version 1145195 (0.0005) [2023-12-27 13:35:28,912][20381] Updated weights for policy 1, policy_version 1145205 (0.0005) [2023-12-27 13:35:29,474][20381] Updated weights for policy 1, policy_version 1145215 (0.0007) [2023-12-27 13:35:29,522][20381] Updated weights for policy 1, policy_version 1145225 (0.0007) [2023-12-27 13:35:29,534][20347] Updated weights for policy 0, policy_version 1146259 (0.0008) [2023-12-27 13:35:29,574][20381] Updated weights for policy 1, policy_version 1145235 (0.0008) [2023-12-27 13:35:29,587][20347] Updated weights for policy 0, policy_version 1146269 (0.0005) [2023-12-27 13:35:29,644][20347] Updated weights for policy 0, policy_version 1146279 (0.0005) [2023-12-27 13:35:30,219][20347] Updated weights for policy 0, policy_version 1146289 (0.0006) [2023-12-27 13:35:30,275][20347] Updated weights for policy 0, policy_version 1146299 (0.0008) [2023-12-27 13:35:30,334][20347] Updated weights for policy 0, policy_version 1146309 (0.0008) [2023-12-27 13:35:30,355][20381] Updated weights for policy 1, policy_version 1145245 (0.0006) [2023-12-27 13:35:30,387][20347] Updated weights for policy 0, policy_version 1146319 (0.0006) [2023-12-27 13:35:30,412][20381] Updated weights for policy 1, policy_version 1145255 (0.0006) [2023-12-27 13:35:30,465][20381] Updated weights for policy 1, policy_version 1145265 (0.0007) [2023-12-27 13:35:31,071][20381] Updated weights for policy 1, policy_version 1145275 (0.0008) [2023-12-27 13:35:31,109][20347] Updated weights for policy 0, policy_version 1146329 (0.0006) [2023-12-27 13:35:31,128][20381] Updated weights for policy 1, policy_version 1145285 (0.0007) [2023-12-27 13:35:31,177][20347] Updated weights for policy 0, policy_version 1146339 (0.0008) [2023-12-27 13:35:31,183][20381] Updated weights for policy 1, policy_version 1145295 (0.0007) [2023-12-27 13:35:31,236][20347] Updated weights for policy 0, policy_version 1146349 (0.0007) [2023-12-27 13:35:31,864][20381] Updated weights for policy 1, policy_version 1145305 (0.0006) [2023-12-27 13:35:31,903][20347] Updated weights for policy 0, policy_version 1146359 (0.0006) [2023-12-27 13:35:31,914][20381] Updated weights for policy 1, policy_version 1145315 (0.0008) [2023-12-27 13:35:31,961][20347] Updated weights for policy 0, policy_version 1146369 (0.0005) [2023-12-27 13:35:31,973][20381] Updated weights for policy 1, policy_version 1145325 (0.0011) [2023-12-27 13:35:32,017][20347] Updated weights for policy 0, policy_version 1146379 (0.0007) [2023-12-27 13:35:32,025][20381] Updated weights for policy 1, policy_version 1145335 (0.0011) [2023-12-27 13:35:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 586760192. Throughput: 0: 10175.9, 1: 10278.0. Samples: 586769012. Policy #0 lag: (min: 31.0, avg: 41.3, max: 63.0) [2023-12-27 13:35:32,363][19170] Avg episode reward: [(0, '707.250'), (1, '677.150')] [2023-12-27 13:35:32,608][20347] Updated weights for policy 0, policy_version 1146389 (0.0005) [2023-12-27 13:35:32,663][20347] Updated weights for policy 0, policy_version 1146399 (0.0006) [2023-12-27 13:35:32,722][20347] Updated weights for policy 0, policy_version 1146409 (0.0007) [2023-12-27 13:35:32,750][20381] Updated weights for policy 1, policy_version 1145345 (0.0007) [2023-12-27 13:35:32,795][20381] Updated weights for policy 1, policy_version 1145355 (0.0007) [2023-12-27 13:35:32,846][20381] Updated weights for policy 1, policy_version 1145365 (0.0007) [2023-12-27 13:35:33,345][20347] Updated weights for policy 0, policy_version 1146419 (0.0006) [2023-12-27 13:35:33,400][20347] Updated weights for policy 0, policy_version 1146429 (0.0008) [2023-12-27 13:35:33,458][20347] Updated weights for policy 0, policy_version 1146439 (0.0007) [2023-12-27 13:35:33,472][20381] Updated weights for policy 1, policy_version 1145375 (0.0006) [2023-12-27 13:35:33,522][20381] Updated weights for policy 1, policy_version 1145385 (0.0005) [2023-12-27 13:35:33,571][20381] Updated weights for policy 1, policy_version 1145395 (0.0005) [2023-12-27 13:35:34,163][20381] Updated weights for policy 1, policy_version 1145405 (0.0006) [2023-12-27 13:35:34,185][20347] Updated weights for policy 0, policy_version 1146449 (0.0008) [2023-12-27 13:35:34,228][20381] Updated weights for policy 1, policy_version 1145415 (0.0007) [2023-12-27 13:35:34,242][20347] Updated weights for policy 0, policy_version 1146459 (0.0007) [2023-12-27 13:35:34,285][20381] Updated weights for policy 1, policy_version 1145425 (0.0005) [2023-12-27 13:35:34,303][20347] Updated weights for policy 0, policy_version 1146469 (0.0007) [2023-12-27 13:35:34,362][20347] Updated weights for policy 0, policy_version 1146479 (0.0007) [2023-12-27 13:35:34,857][20381] Updated weights for policy 1, policy_version 1145435 (0.0006) [2023-12-27 13:35:34,917][20381] Updated weights for policy 1, policy_version 1145445 (0.0007) [2023-12-27 13:35:34,967][20381] Updated weights for policy 1, policy_version 1145455 (0.0008) [2023-12-27 13:35:35,134][20347] Updated weights for policy 0, policy_version 1146489 (0.0008) [2023-12-27 13:35:35,190][20347] Updated weights for policy 0, policy_version 1146499 (0.0008) [2023-12-27 13:35:35,245][20347] Updated weights for policy 0, policy_version 1146509 (0.0008) [2023-12-27 13:35:35,591][20381] Updated weights for policy 1, policy_version 1145465 (0.0008) [2023-12-27 13:35:35,641][20381] Updated weights for policy 1, policy_version 1145475 (0.0008) [2023-12-27 13:35:35,700][20381] Updated weights for policy 1, policy_version 1145485 (0.0008) [2023-12-27 13:35:35,750][20381] Updated weights for policy 1, policy_version 1145495 (0.0008) [2023-12-27 13:35:35,959][20347] Updated weights for policy 0, policy_version 1146519 (0.0009) [2023-12-27 13:35:36,005][20347] Updated weights for policy 0, policy_version 1146529 (0.0008) [2023-12-27 13:35:36,050][20347] Updated weights for policy 0, policy_version 1146539 (0.0007) [2023-12-27 13:35:36,411][20381] Updated weights for policy 1, policy_version 1145505 (0.0008) [2023-12-27 13:35:36,475][20381] Updated weights for policy 1, policy_version 1145515 (0.0007) [2023-12-27 13:35:36,540][20381] Updated weights for policy 1, policy_version 1145525 (0.0005) [2023-12-27 13:35:36,764][20347] Updated weights for policy 0, policy_version 1146549 (0.0008) [2023-12-27 13:35:36,830][20347] Updated weights for policy 0, policy_version 1146559 (0.0007) [2023-12-27 13:35:36,893][20347] Updated weights for policy 0, policy_version 1146569 (0.0007) [2023-12-27 13:35:37,142][20381] Updated weights for policy 1, policy_version 1145535 (0.0008) [2023-12-27 13:35:37,196][20381] Updated weights for policy 1, policy_version 1145545 (0.0008) [2023-12-27 13:35:37,252][20381] Updated weights for policy 1, policy_version 1145555 (0.0006) [2023-12-27 13:35:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 586866688. Throughput: 0: 10190.2, 1: 10348.6. Samples: 586832776. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:35:37,363][19170] Avg episode reward: [(0, '737.630'), (1, '745.470')] [2023-12-27 13:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001145560_293289984.pth... [2023-12-27 13:35:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001146576_293576704.pth... [2023-12-27 13:35:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001144344_292978688.pth [2023-12-27 13:35:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001145392_293273600.pth [2023-12-27 13:35:37,553][20347] Updated weights for policy 0, policy_version 1146579 (0.0007) [2023-12-27 13:35:37,619][20347] Updated weights for policy 0, policy_version 1146589 (0.0006) [2023-12-27 13:35:37,690][20347] Updated weights for policy 0, policy_version 1146600 (0.0006) [2023-12-27 13:35:37,919][20381] Updated weights for policy 1, policy_version 1145565 (0.0008) [2023-12-27 13:35:37,976][20381] Updated weights for policy 1, policy_version 1145575 (0.0011) [2023-12-27 13:35:38,031][20381] Updated weights for policy 1, policy_version 1145585 (0.0010) [2023-12-27 13:35:38,396][20347] Updated weights for policy 0, policy_version 1146610 (0.0009) [2023-12-27 13:35:38,453][20347] Updated weights for policy 0, policy_version 1146620 (0.0008) [2023-12-27 13:35:38,509][20347] Updated weights for policy 0, policy_version 1146630 (0.0008) [2023-12-27 13:35:38,563][20347] Updated weights for policy 0, policy_version 1146640 (0.0007) [2023-12-27 13:35:38,729][20381] Updated weights for policy 1, policy_version 1145595 (0.0010) [2023-12-27 13:35:38,784][20381] Updated weights for policy 1, policy_version 1145605 (0.0007) [2023-12-27 13:35:38,844][20381] Updated weights for policy 1, policy_version 1145615 (0.0008) [2023-12-27 13:35:39,285][20347] Updated weights for policy 0, policy_version 1146650 (0.0011) [2023-12-27 13:35:39,348][20347] Updated weights for policy 0, policy_version 1146660 (0.0009) [2023-12-27 13:35:39,417][20347] Updated weights for policy 0, policy_version 1146670 (0.0008) [2023-12-27 13:35:39,537][20381] Updated weights for policy 1, policy_version 1145625 (0.0007) [2023-12-27 13:35:39,593][20381] Updated weights for policy 1, policy_version 1145635 (0.0006) [2023-12-27 13:35:39,646][20381] Updated weights for policy 1, policy_version 1145645 (0.0007) [2023-12-27 13:35:39,693][20381] Updated weights for policy 1, policy_version 1145655 (0.0007) [2023-12-27 13:35:40,184][20347] Updated weights for policy 0, policy_version 1146680 (0.0008) [2023-12-27 13:35:40,240][20347] Updated weights for policy 0, policy_version 1146690 (0.0007) [2023-12-27 13:35:40,300][20347] Updated weights for policy 0, policy_version 1146700 (0.0008) [2023-12-27 13:35:40,436][20381] Updated weights for policy 1, policy_version 1145665 (0.0007) [2023-12-27 13:35:40,499][20381] Updated weights for policy 1, policy_version 1145675 (0.0007) [2023-12-27 13:35:40,558][20381] Updated weights for policy 1, policy_version 1145685 (0.0008) [2023-12-27 13:35:40,893][20347] Updated weights for policy 0, policy_version 1146710 (0.0006) [2023-12-27 13:35:40,951][20347] Updated weights for policy 0, policy_version 1146720 (0.0005) [2023-12-27 13:35:41,022][20347] Updated weights for policy 0, policy_version 1146730 (0.0006) [2023-12-27 13:35:41,255][20381] Updated weights for policy 1, policy_version 1145695 (0.0007) [2023-12-27 13:35:41,338][20381] Updated weights for policy 1, policy_version 1145705 (0.0008) [2023-12-27 13:35:41,390][20381] Updated weights for policy 1, policy_version 1145715 (0.0008) [2023-12-27 13:35:41,714][20347] Updated weights for policy 0, policy_version 1146740 (0.0009) [2023-12-27 13:35:41,786][20347] Updated weights for policy 0, policy_version 1146750 (0.0009) [2023-12-27 13:35:41,852][20347] Updated weights for policy 0, policy_version 1146760 (0.0005) [2023-12-27 13:35:42,143][20381] Updated weights for policy 1, policy_version 1145725 (0.0007) [2023-12-27 13:35:42,201][20381] Updated weights for policy 1, policy_version 1145735 (0.0008) [2023-12-27 13:35:42,262][20381] Updated weights for policy 1, policy_version 1145745 (0.0007) [2023-12-27 13:35:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 586964992. Throughput: 0: 10140.3, 1: 10331.8. Samples: 586954312. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:35:42,363][19170] Avg episode reward: [(0, '660.230'), (1, '657.250')] [2023-12-27 13:35:42,576][20347] Updated weights for policy 0, policy_version 1146770 (0.0008) [2023-12-27 13:35:42,625][20347] Updated weights for policy 0, policy_version 1146780 (0.0008) [2023-12-27 13:35:42,687][20347] Updated weights for policy 0, policy_version 1146790 (0.0007) [2023-12-27 13:35:42,743][20347] Updated weights for policy 0, policy_version 1146800 (0.0009) [2023-12-27 13:35:42,940][20381] Updated weights for policy 1, policy_version 1145755 (0.0013) [2023-12-27 13:35:42,988][20381] Updated weights for policy 1, policy_version 1145765 (0.0010) [2023-12-27 13:35:43,040][20381] Updated weights for policy 1, policy_version 1145775 (0.0011) [2023-12-27 13:35:43,383][20347] Updated weights for policy 0, policy_version 1146810 (0.0008) [2023-12-27 13:35:43,441][20347] Updated weights for policy 0, policy_version 1146820 (0.0008) [2023-12-27 13:35:43,498][20347] Updated weights for policy 0, policy_version 1146830 (0.0007) [2023-12-27 13:35:43,696][20381] Updated weights for policy 1, policy_version 1145785 (0.0010) [2023-12-27 13:35:43,748][20381] Updated weights for policy 1, policy_version 1145795 (0.0011) [2023-12-27 13:35:43,810][20381] Updated weights for policy 1, policy_version 1145805 (0.0011) [2023-12-27 13:35:43,858][20381] Updated weights for policy 1, policy_version 1145815 (0.0010) [2023-12-27 13:35:44,076][20347] Updated weights for policy 0, policy_version 1146840 (0.0006) [2023-12-27 13:35:44,133][20347] Updated weights for policy 0, policy_version 1146850 (0.0005) [2023-12-27 13:35:44,187][20347] Updated weights for policy 0, policy_version 1146860 (0.0005) [2023-12-27 13:35:44,546][20381] Updated weights for policy 1, policy_version 1145825 (0.0010) [2023-12-27 13:35:44,594][20381] Updated weights for policy 1, policy_version 1145835 (0.0010) [2023-12-27 13:35:44,652][20381] Updated weights for policy 1, policy_version 1145845 (0.0011) [2023-12-27 13:35:44,826][20347] Updated weights for policy 0, policy_version 1146870 (0.0007) [2023-12-27 13:35:44,890][20347] Updated weights for policy 0, policy_version 1146880 (0.0006) [2023-12-27 13:35:44,953][20347] Updated weights for policy 0, policy_version 1146890 (0.0006) [2023-12-27 13:35:45,376][20381] Updated weights for policy 1, policy_version 1145855 (0.0009) [2023-12-27 13:35:45,425][20381] Updated weights for policy 1, policy_version 1145865 (0.0008) [2023-12-27 13:35:45,470][20381] Updated weights for policy 1, policy_version 1145875 (0.0009) [2023-12-27 13:35:45,525][20347] Updated weights for policy 0, policy_version 1146900 (0.0005) [2023-12-27 13:35:45,583][20347] Updated weights for policy 0, policy_version 1146910 (0.0005) [2023-12-27 13:35:45,642][20347] Updated weights for policy 0, policy_version 1146920 (0.0005) [2023-12-27 13:35:46,215][20347] Updated weights for policy 0, policy_version 1146930 (0.0005) [2023-12-27 13:35:46,228][20381] Updated weights for policy 1, policy_version 1145885 (0.0009) [2023-12-27 13:35:46,265][20347] Updated weights for policy 0, policy_version 1146940 (0.0005) [2023-12-27 13:35:46,285][20381] Updated weights for policy 1, policy_version 1145895 (0.0008) [2023-12-27 13:35:46,316][20347] Updated weights for policy 0, policy_version 1146950 (0.0005) [2023-12-27 13:35:46,335][20381] Updated weights for policy 1, policy_version 1145905 (0.0006) [2023-12-27 13:35:46,375][20347] Updated weights for policy 0, policy_version 1146960 (0.0005) [2023-12-27 13:35:46,986][20347] Updated weights for policy 0, policy_version 1146970 (0.0007) [2023-12-27 13:35:47,016][20381] Updated weights for policy 1, policy_version 1145915 (0.0006) [2023-12-27 13:35:47,041][20347] Updated weights for policy 0, policy_version 1146980 (0.0008) [2023-12-27 13:35:47,069][20381] Updated weights for policy 1, policy_version 1145925 (0.0006) [2023-12-27 13:35:47,090][20347] Updated weights for policy 0, policy_version 1146990 (0.0005) [2023-12-27 13:35:47,118][20381] Updated weights for policy 1, policy_version 1145935 (0.0006) [2023-12-27 13:35:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 587071488. Throughput: 0: 10238.2, 1: 10309.3. Samples: 587078092. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:35:47,363][19170] Avg episode reward: [(0, '718.570'), (1, '647.360')] [2023-12-27 13:35:47,756][20347] Updated weights for policy 0, policy_version 1147000 (0.0005) [2023-12-27 13:35:47,811][20347] Updated weights for policy 0, policy_version 1147010 (0.0005) [2023-12-27 13:35:47,845][20381] Updated weights for policy 1, policy_version 1145945 (0.0008) [2023-12-27 13:35:47,864][20347] Updated weights for policy 0, policy_version 1147020 (0.0008) [2023-12-27 13:35:47,934][20381] Updated weights for policy 1, policy_version 1145955 (0.0005) [2023-12-27 13:35:47,996][20381] Updated weights for policy 1, policy_version 1145965 (0.0006) [2023-12-27 13:35:48,053][20381] Updated weights for policy 1, policy_version 1145975 (0.0008) [2023-12-27 13:35:48,538][20347] Updated weights for policy 0, policy_version 1147030 (0.0009) [2023-12-27 13:35:48,600][20347] Updated weights for policy 0, policy_version 1147040 (0.0010) [2023-12-27 13:35:48,647][20381] Updated weights for policy 1, policy_version 1145985 (0.0008) [2023-12-27 13:35:48,655][20347] Updated weights for policy 0, policy_version 1147050 (0.0010) [2023-12-27 13:35:48,709][20381] Updated weights for policy 1, policy_version 1145995 (0.0006) [2023-12-27 13:35:48,772][20381] Updated weights for policy 1, policy_version 1146005 (0.0007) [2023-12-27 13:35:49,349][20347] Updated weights for policy 0, policy_version 1147060 (0.0009) [2023-12-27 13:35:49,412][20347] Updated weights for policy 0, policy_version 1147070 (0.0008) [2023-12-27 13:35:49,436][20381] Updated weights for policy 1, policy_version 1146015 (0.0007) [2023-12-27 13:35:49,471][20347] Updated weights for policy 0, policy_version 1147080 (0.0008) [2023-12-27 13:35:49,500][20381] Updated weights for policy 1, policy_version 1146025 (0.0006) [2023-12-27 13:35:49,567][20381] Updated weights for policy 1, policy_version 1146035 (0.0007) [2023-12-27 13:35:50,134][20347] Updated weights for policy 0, policy_version 1147090 (0.0007) [2023-12-27 13:35:50,195][20347] Updated weights for policy 0, policy_version 1147100 (0.0008) [2023-12-27 13:35:50,243][20347] Updated weights for policy 0, policy_version 1147110 (0.0007) [2023-12-27 13:35:50,296][20347] Updated weights for policy 0, policy_version 1147120 (0.0006) [2023-12-27 13:35:50,299][20381] Updated weights for policy 1, policy_version 1146045 (0.0007) [2023-12-27 13:35:50,354][20381] Updated weights for policy 1, policy_version 1146055 (0.0008) [2023-12-27 13:35:50,417][20381] Updated weights for policy 1, policy_version 1146065 (0.0006) [2023-12-27 13:35:50,995][20347] Updated weights for policy 0, policy_version 1147130 (0.0008) [2023-12-27 13:35:51,059][20381] Updated weights for policy 1, policy_version 1146075 (0.0009) [2023-12-27 13:35:51,060][20347] Updated weights for policy 0, policy_version 1147140 (0.0009) [2023-12-27 13:35:51,117][20381] Updated weights for policy 1, policy_version 1146085 (0.0008) [2023-12-27 13:35:51,121][20347] Updated weights for policy 0, policy_version 1147150 (0.0006) [2023-12-27 13:35:51,180][20381] Updated weights for policy 1, policy_version 1146095 (0.0009) [2023-12-27 13:35:51,899][20347] Updated weights for policy 0, policy_version 1147160 (0.0007) [2023-12-27 13:35:51,903][20381] Updated weights for policy 1, policy_version 1146105 (0.0011) [2023-12-27 13:35:51,955][20347] Updated weights for policy 0, policy_version 1147170 (0.0007) [2023-12-27 13:35:51,964][20381] Updated weights for policy 1, policy_version 1146115 (0.0010) [2023-12-27 13:35:52,010][20347] Updated weights for policy 0, policy_version 1147180 (0.0005) [2023-12-27 13:35:52,020][20381] Updated weights for policy 1, policy_version 1146125 (0.0010) [2023-12-27 13:35:52,077][20381] Updated weights for policy 1, policy_version 1146135 (0.0010) [2023-12-27 13:35:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 587169792. Throughput: 0: 10242.6, 1: 10295.2. Samples: 587139652. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:35:52,363][19170] Avg episode reward: [(0, '776.120'), (1, '627.690')] [2023-12-27 13:35:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001147184_293732352.pth... [2023-12-27 13:35:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001146136_293437440.pth... [2023-12-27 13:35:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001145968_293421056.pth [2023-12-27 13:35:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001144920_293126144.pth [2023-12-27 13:35:52,664][20347] Updated weights for policy 0, policy_version 1147190 (0.0008) [2023-12-27 13:35:52,718][20347] Updated weights for policy 0, policy_version 1147200 (0.0008) [2023-12-27 13:35:52,780][20347] Updated weights for policy 0, policy_version 1147210 (0.0007) [2023-12-27 13:35:52,832][20381] Updated weights for policy 1, policy_version 1146145 (0.0008) [2023-12-27 13:35:52,898][20381] Updated weights for policy 1, policy_version 1146155 (0.0009) [2023-12-27 13:35:52,960][20381] Updated weights for policy 1, policy_version 1146165 (0.0008) [2023-12-27 13:35:53,444][20347] Updated weights for policy 0, policy_version 1147220 (0.0006) [2023-12-27 13:35:53,506][20347] Updated weights for policy 0, policy_version 1147230 (0.0008) [2023-12-27 13:35:53,562][20347] Updated weights for policy 0, policy_version 1147240 (0.0008) [2023-12-27 13:35:53,647][20381] Updated weights for policy 1, policy_version 1146175 (0.0008) [2023-12-27 13:35:53,702][20381] Updated weights for policy 1, policy_version 1146185 (0.0008) [2023-12-27 13:35:53,761][20381] Updated weights for policy 1, policy_version 1146195 (0.0008) [2023-12-27 13:35:54,220][20347] Updated weights for policy 0, policy_version 1147250 (0.0007) [2023-12-27 13:35:54,275][20347] Updated weights for policy 0, policy_version 1147260 (0.0010) [2023-12-27 13:35:54,332][20347] Updated weights for policy 0, policy_version 1147270 (0.0010) [2023-12-27 13:35:54,391][20347] Updated weights for policy 0, policy_version 1147280 (0.0010) [2023-12-27 13:35:54,459][20381] Updated weights for policy 1, policy_version 1146205 (0.0007) [2023-12-27 13:35:54,524][20381] Updated weights for policy 1, policy_version 1146215 (0.0008) [2023-12-27 13:35:54,585][20381] Updated weights for policy 1, policy_version 1146225 (0.0006) [2023-12-27 13:35:55,029][20347] Updated weights for policy 0, policy_version 1147290 (0.0006) [2023-12-27 13:35:55,084][20347] Updated weights for policy 0, policy_version 1147300 (0.0005) [2023-12-27 13:35:55,141][20347] Updated weights for policy 0, policy_version 1147310 (0.0008) [2023-12-27 13:35:55,169][20381] Updated weights for policy 1, policy_version 1146235 (0.0006) [2023-12-27 13:35:55,228][20381] Updated weights for policy 1, policy_version 1146245 (0.0007) [2023-12-27 13:35:55,285][20381] Updated weights for policy 1, policy_version 1146255 (0.0008) [2023-12-27 13:35:55,738][20347] Updated weights for policy 0, policy_version 1147320 (0.0010) [2023-12-27 13:35:55,788][20347] Updated weights for policy 0, policy_version 1147330 (0.0005) [2023-12-27 13:35:55,837][20347] Updated weights for policy 0, policy_version 1147340 (0.0005) [2023-12-27 13:35:56,016][20381] Updated weights for policy 1, policy_version 1146265 (0.0008) [2023-12-27 13:35:56,064][20381] Updated weights for policy 1, policy_version 1146275 (0.0007) [2023-12-27 13:35:56,118][20381] Updated weights for policy 1, policy_version 1146285 (0.0008) [2023-12-27 13:35:56,167][20381] Updated weights for policy 1, policy_version 1146295 (0.0005) [2023-12-27 13:35:56,476][20347] Updated weights for policy 0, policy_version 1147350 (0.0008) [2023-12-27 13:35:56,523][20347] Updated weights for policy 0, policy_version 1147360 (0.0010) [2023-12-27 13:35:56,571][20347] Updated weights for policy 0, policy_version 1147370 (0.0006) [2023-12-27 13:35:56,754][20381] Updated weights for policy 1, policy_version 1146305 (0.0007) [2023-12-27 13:35:56,804][20381] Updated weights for policy 1, policy_version 1146315 (0.0007) [2023-12-27 13:35:56,852][20381] Updated weights for policy 1, policy_version 1146325 (0.0007) [2023-12-27 13:35:57,235][20347] Updated weights for policy 0, policy_version 1147380 (0.0008) [2023-12-27 13:35:57,283][20347] Updated weights for policy 0, policy_version 1147390 (0.0005) [2023-12-27 13:35:57,340][20347] Updated weights for policy 0, policy_version 1147400 (0.0005) [2023-12-27 13:35:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 587268096. Throughput: 0: 10289.7, 1: 10263.2. Samples: 587261660. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:35:57,363][19170] Avg episode reward: [(0, '775.050'), (1, '707.280')] [2023-12-27 13:35:57,517][20381] Updated weights for policy 1, policy_version 1146335 (0.0009) [2023-12-27 13:35:57,566][20381] Updated weights for policy 1, policy_version 1146345 (0.0009) [2023-12-27 13:35:57,610][20381] Updated weights for policy 1, policy_version 1146355 (0.0008) [2023-12-27 13:35:57,841][20347] Updated weights for policy 0, policy_version 1147410 (0.0005) [2023-12-27 13:35:57,899][20347] Updated weights for policy 0, policy_version 1147420 (0.0010) [2023-12-27 13:35:57,949][20347] Updated weights for policy 0, policy_version 1147430 (0.0010) [2023-12-27 13:35:57,998][20347] Updated weights for policy 0, policy_version 1147440 (0.0010) [2023-12-27 13:35:58,361][20381] Updated weights for policy 1, policy_version 1146365 (0.0009) [2023-12-27 13:35:58,419][20381] Updated weights for policy 1, policy_version 1146375 (0.0009) [2023-12-27 13:35:58,484][20381] Updated weights for policy 1, policy_version 1146385 (0.0011) [2023-12-27 13:35:58,725][20347] Updated weights for policy 0, policy_version 1147450 (0.0007) [2023-12-27 13:35:58,791][20347] Updated weights for policy 0, policy_version 1147460 (0.0008) [2023-12-27 13:35:58,858][20347] Updated weights for policy 0, policy_version 1147470 (0.0010) [2023-12-27 13:35:59,228][20381] Updated weights for policy 1, policy_version 1146395 (0.0010) [2023-12-27 13:35:59,296][20381] Updated weights for policy 1, policy_version 1146405 (0.0009) [2023-12-27 13:35:59,356][20381] Updated weights for policy 1, policy_version 1146415 (0.0011) [2023-12-27 13:35:59,594][20347] Updated weights for policy 0, policy_version 1147480 (0.0006) [2023-12-27 13:35:59,643][20347] Updated weights for policy 0, policy_version 1147490 (0.0008) [2023-12-27 13:35:59,697][20347] Updated weights for policy 0, policy_version 1147500 (0.0007) [2023-12-27 13:36:00,101][20381] Updated weights for policy 1, policy_version 1146425 (0.0009) [2023-12-27 13:36:00,154][20381] Updated weights for policy 1, policy_version 1146435 (0.0008) [2023-12-27 13:36:00,216][20381] Updated weights for policy 1, policy_version 1146445 (0.0007) [2023-12-27 13:36:00,280][20381] Updated weights for policy 1, policy_version 1146455 (0.0008) [2023-12-27 13:36:00,366][20347] Updated weights for policy 0, policy_version 1147510 (0.0006) [2023-12-27 13:36:00,424][20347] Updated weights for policy 0, policy_version 1147520 (0.0007) [2023-12-27 13:36:00,481][20347] Updated weights for policy 0, policy_version 1147530 (0.0005) [2023-12-27 13:36:00,907][20381] Updated weights for policy 1, policy_version 1146465 (0.0008) [2023-12-27 13:36:00,956][20381] Updated weights for policy 1, policy_version 1146475 (0.0008) [2023-12-27 13:36:01,007][20381] Updated weights for policy 1, policy_version 1146485 (0.0007) [2023-12-27 13:36:01,068][20347] Updated weights for policy 0, policy_version 1147540 (0.0006) [2023-12-27 13:36:01,131][20347] Updated weights for policy 0, policy_version 1147550 (0.0006) [2023-12-27 13:36:01,193][20347] Updated weights for policy 0, policy_version 1147560 (0.0008) [2023-12-27 13:36:01,733][20381] Updated weights for policy 1, policy_version 1146495 (0.0008) [2023-12-27 13:36:01,789][20381] Updated weights for policy 1, policy_version 1146505 (0.0008) [2023-12-27 13:36:01,850][20381] Updated weights for policy 1, policy_version 1146515 (0.0007) [2023-12-27 13:36:01,905][20347] Updated weights for policy 0, policy_version 1147570 (0.0008) [2023-12-27 13:36:01,965][20347] Updated weights for policy 0, policy_version 1147580 (0.0008) [2023-12-27 13:36:02,017][20347] Updated weights for policy 0, policy_version 1147590 (0.0008) [2023-12-27 13:36:02,074][20347] Updated weights for policy 0, policy_version 1147600 (0.0008) [2023-12-27 13:36:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 587374592. Throughput: 0: 10354.9, 1: 10230.0. Samples: 587384748. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:02,363][19170] Avg episode reward: [(0, '752.810'), (1, '647.650')] [2023-12-27 13:36:02,525][20381] Updated weights for policy 1, policy_version 1146525 (0.0006) [2023-12-27 13:36:02,576][20381] Updated weights for policy 1, policy_version 1146535 (0.0007) [2023-12-27 13:36:02,631][20381] Updated weights for policy 1, policy_version 1146545 (0.0007) [2023-12-27 13:36:02,773][20347] Updated weights for policy 0, policy_version 1147610 (0.0007) [2023-12-27 13:36:02,834][20347] Updated weights for policy 0, policy_version 1147620 (0.0006) [2023-12-27 13:36:02,887][20347] Updated weights for policy 0, policy_version 1147630 (0.0005) [2023-12-27 13:36:03,293][20381] Updated weights for policy 1, policy_version 1146555 (0.0008) [2023-12-27 13:36:03,352][20381] Updated weights for policy 1, policy_version 1146565 (0.0011) [2023-12-27 13:36:03,408][20381] Updated weights for policy 1, policy_version 1146575 (0.0011) [2023-12-27 13:36:03,468][20347] Updated weights for policy 0, policy_version 1147640 (0.0005) [2023-12-27 13:36:03,521][20347] Updated weights for policy 0, policy_version 1147650 (0.0005) [2023-12-27 13:36:03,574][20347] Updated weights for policy 0, policy_version 1147660 (0.0007) [2023-12-27 13:36:04,102][20381] Updated weights for policy 1, policy_version 1146585 (0.0010) [2023-12-27 13:36:04,163][20381] Updated weights for policy 1, policy_version 1146595 (0.0011) [2023-12-27 13:36:04,170][20347] Updated weights for policy 0, policy_version 1147670 (0.0008) [2023-12-27 13:36:04,228][20381] Updated weights for policy 1, policy_version 1146605 (0.0009) [2023-12-27 13:36:04,234][20347] Updated weights for policy 0, policy_version 1147680 (0.0010) [2023-12-27 13:36:04,283][20381] Updated weights for policy 1, policy_version 1146615 (0.0008) [2023-12-27 13:36:04,291][20347] Updated weights for policy 0, policy_version 1147690 (0.0010) [2023-12-27 13:36:04,963][20347] Updated weights for policy 0, policy_version 1147700 (0.0009) [2023-12-27 13:36:04,963][20381] Updated weights for policy 1, policy_version 1146625 (0.0006) [2023-12-27 13:36:05,019][20381] Updated weights for policy 1, policy_version 1146635 (0.0008) [2023-12-27 13:36:05,021][20347] Updated weights for policy 0, policy_version 1147710 (0.0010) [2023-12-27 13:36:05,066][20347] Updated weights for policy 0, policy_version 1147720 (0.0010) [2023-12-27 13:36:05,067][20381] Updated weights for policy 1, policy_version 1146645 (0.0007) [2023-12-27 13:36:05,078][20292] KL-divergence is very high: 205.1469 [2023-12-27 13:36:05,721][20347] Updated weights for policy 0, policy_version 1147730 (0.0010) [2023-12-27 13:36:05,727][20381] Updated weights for policy 1, policy_version 1146655 (0.0009) [2023-12-27 13:36:05,775][20347] Updated weights for policy 0, policy_version 1147740 (0.0010) [2023-12-27 13:36:05,777][20381] Updated weights for policy 1, policy_version 1146665 (0.0010) [2023-12-27 13:36:05,833][20347] Updated weights for policy 0, policy_version 1147750 (0.0010) [2023-12-27 13:36:05,836][20381] Updated weights for policy 1, policy_version 1146675 (0.0010) [2023-12-27 13:36:05,888][20347] Updated weights for policy 0, policy_version 1147760 (0.0010) [2023-12-27 13:36:06,568][20381] Updated weights for policy 1, policy_version 1146685 (0.0011) [2023-12-27 13:36:06,576][20347] Updated weights for policy 0, policy_version 1147770 (0.0010) [2023-12-27 13:36:06,629][20381] Updated weights for policy 1, policy_version 1146695 (0.0009) [2023-12-27 13:36:06,635][20347] Updated weights for policy 0, policy_version 1147780 (0.0009) [2023-12-27 13:36:06,691][20381] Updated weights for policy 1, policy_version 1146705 (0.0006) [2023-12-27 13:36:06,694][20347] Updated weights for policy 0, policy_version 1147790 (0.0010) [2023-12-27 13:36:07,279][20347] Updated weights for policy 0, policy_version 1147800 (0.0011) [2023-12-27 13:36:07,338][20347] Updated weights for policy 0, policy_version 1147810 (0.0010) [2023-12-27 13:36:07,355][20381] Updated weights for policy 1, policy_version 1146715 (0.0009) [2023-12-27 13:36:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 587472896. Throughput: 0: 10377.8, 1: 10263.6. Samples: 587447872. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:07,363][19170] Avg episode reward: [(0, '799.450'), (1, '667.190')] [2023-12-27 13:36:07,397][20347] Updated weights for policy 0, policy_version 1147820 (0.0011) [2023-12-27 13:36:07,411][20381] Updated weights for policy 1, policy_version 1146725 (0.0006) [2023-12-27 13:36:07,414][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001147824_293896192.pth... [2023-12-27 13:36:07,417][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001146576_293576704.pth [2023-12-27 13:36:07,467][20381] Updated weights for policy 1, policy_version 1146735 (0.0007) [2023-12-27 13:36:07,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001146744_293593088.pth... [2023-12-27 13:36:07,512][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001145560_293289984.pth [2023-12-27 13:36:08,060][20347] Updated weights for policy 0, policy_version 1147830 (0.0009) [2023-12-27 13:36:08,120][20347] Updated weights for policy 0, policy_version 1147840 (0.0010) [2023-12-27 13:36:08,187][20347] Updated weights for policy 0, policy_version 1147850 (0.0010) [2023-12-27 13:36:08,202][20381] Updated weights for policy 1, policy_version 1146745 (0.0007) [2023-12-27 13:36:08,262][20381] Updated weights for policy 1, policy_version 1146755 (0.0007) [2023-12-27 13:36:08,325][20381] Updated weights for policy 1, policy_version 1146765 (0.0007) [2023-12-27 13:36:08,387][20381] Updated weights for policy 1, policy_version 1146775 (0.0007) [2023-12-27 13:36:08,916][20347] Updated weights for policy 0, policy_version 1147860 (0.0010) [2023-12-27 13:36:08,973][20347] Updated weights for policy 0, policy_version 1147870 (0.0007) [2023-12-27 13:36:09,027][20347] Updated weights for policy 0, policy_version 1147880 (0.0008) [2023-12-27 13:36:09,078][20381] Updated weights for policy 1, policy_version 1146785 (0.0007) [2023-12-27 13:36:09,122][20381] Updated weights for policy 1, policy_version 1146795 (0.0007) [2023-12-27 13:36:09,170][20381] Updated weights for policy 1, policy_version 1146805 (0.0007) [2023-12-27 13:36:09,720][20347] Updated weights for policy 0, policy_version 1147890 (0.0006) [2023-12-27 13:36:09,783][20347] Updated weights for policy 0, policy_version 1147900 (0.0006) [2023-12-27 13:36:09,851][20347] Updated weights for policy 0, policy_version 1147910 (0.0006) [2023-12-27 13:36:09,911][20381] Updated weights for policy 1, policy_version 1146815 (0.0009) [2023-12-27 13:36:09,914][20347] Updated weights for policy 0, policy_version 1147920 (0.0007) [2023-12-27 13:36:09,969][20381] Updated weights for policy 1, policy_version 1146825 (0.0009) [2023-12-27 13:36:10,024][20381] Updated weights for policy 1, policy_version 1146835 (0.0008) [2023-12-27 13:36:10,611][20347] Updated weights for policy 0, policy_version 1147930 (0.0009) [2023-12-27 13:36:10,666][20347] Updated weights for policy 0, policy_version 1147940 (0.0008) [2023-12-27 13:36:10,713][20347] Updated weights for policy 0, policy_version 1147950 (0.0007) [2023-12-27 13:36:10,724][20381] Updated weights for policy 1, policy_version 1146845 (0.0008) [2023-12-27 13:36:10,776][20381] Updated weights for policy 1, policy_version 1146855 (0.0009) [2023-12-27 13:36:10,831][20381] Updated weights for policy 1, policy_version 1146865 (0.0007) [2023-12-27 13:36:11,490][20347] Updated weights for policy 0, policy_version 1147960 (0.0008) [2023-12-27 13:36:11,549][20347] Updated weights for policy 0, policy_version 1147970 (0.0008) [2023-12-27 13:36:11,583][20381] Updated weights for policy 1, policy_version 1146875 (0.0007) [2023-12-27 13:36:11,608][20347] Updated weights for policy 0, policy_version 1147980 (0.0008) [2023-12-27 13:36:11,652][20381] Updated weights for policy 1, policy_version 1146885 (0.0008) [2023-12-27 13:36:11,706][20381] Updated weights for policy 1, policy_version 1146895 (0.0007) [2023-12-27 13:36:12,342][20347] Updated weights for policy 0, policy_version 1147990 (0.0010) [2023-12-27 13:36:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 587571200. Throughput: 0: 10419.0, 1: 10168.9. Samples: 587568800. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:12,363][19170] Avg episode reward: [(0, '840.330'), (1, '676.800')] [2023-12-27 13:36:12,407][20347] Updated weights for policy 0, policy_version 1148000 (0.0010) [2023-12-27 13:36:12,433][20381] Updated weights for policy 1, policy_version 1146905 (0.0007) [2023-12-27 13:36:12,468][20347] Updated weights for policy 0, policy_version 1148010 (0.0008) [2023-12-27 13:36:12,487][20381] Updated weights for policy 1, policy_version 1146915 (0.0009) [2023-12-27 13:36:12,546][20381] Updated weights for policy 1, policy_version 1146925 (0.0008) [2023-12-27 13:36:12,605][20381] Updated weights for policy 1, policy_version 1146935 (0.0008) [2023-12-27 13:36:13,165][20347] Updated weights for policy 0, policy_version 1148020 (0.0006) [2023-12-27 13:36:13,215][20347] Updated weights for policy 0, policy_version 1148030 (0.0005) [2023-12-27 13:36:13,268][20347] Updated weights for policy 0, policy_version 1148040 (0.0005) [2023-12-27 13:36:13,306][20381] Updated weights for policy 1, policy_version 1146945 (0.0008) [2023-12-27 13:36:13,353][20381] Updated weights for policy 1, policy_version 1146955 (0.0007) [2023-12-27 13:36:13,409][20381] Updated weights for policy 1, policy_version 1146965 (0.0008) [2023-12-27 13:36:13,865][20347] Updated weights for policy 0, policy_version 1148050 (0.0006) [2023-12-27 13:36:13,921][20347] Updated weights for policy 0, policy_version 1148060 (0.0008) [2023-12-27 13:36:13,971][20347] Updated weights for policy 0, policy_version 1148070 (0.0007) [2023-12-27 13:36:14,016][20347] Updated weights for policy 0, policy_version 1148080 (0.0005) [2023-12-27 13:36:14,165][20381] Updated weights for policy 1, policy_version 1146975 (0.0007) [2023-12-27 13:36:14,223][20381] Updated weights for policy 1, policy_version 1146985 (0.0008) [2023-12-27 13:36:14,275][20381] Updated weights for policy 1, policy_version 1146995 (0.0009) [2023-12-27 13:36:14,590][20347] Updated weights for policy 0, policy_version 1148090 (0.0005) [2023-12-27 13:36:14,645][20347] Updated weights for policy 0, policy_version 1148100 (0.0006) [2023-12-27 13:36:14,704][20347] Updated weights for policy 0, policy_version 1148110 (0.0008) [2023-12-27 13:36:14,991][20381] Updated weights for policy 1, policy_version 1147005 (0.0008) [2023-12-27 13:36:15,056][20381] Updated weights for policy 1, policy_version 1147015 (0.0008) [2023-12-27 13:36:15,117][20381] Updated weights for policy 1, policy_version 1147025 (0.0005) [2023-12-27 13:36:15,334][20347] Updated weights for policy 0, policy_version 1148120 (0.0008) [2023-12-27 13:36:15,390][20347] Updated weights for policy 0, policy_version 1148130 (0.0008) [2023-12-27 13:36:15,444][20347] Updated weights for policy 0, policy_version 1148140 (0.0007) [2023-12-27 13:36:15,753][20381] Updated weights for policy 1, policy_version 1147035 (0.0005) [2023-12-27 13:36:15,812][20381] Updated weights for policy 1, policy_version 1147045 (0.0005) [2023-12-27 13:36:15,874][20381] Updated weights for policy 1, policy_version 1147055 (0.0006) [2023-12-27 13:36:16,150][20347] Updated weights for policy 0, policy_version 1148150 (0.0007) [2023-12-27 13:36:16,195][20347] Updated weights for policy 0, policy_version 1148160 (0.0008) [2023-12-27 13:36:16,243][20347] Updated weights for policy 0, policy_version 1148170 (0.0008) [2023-12-27 13:36:16,501][20381] Updated weights for policy 1, policy_version 1147065 (0.0007) [2023-12-27 13:36:16,554][20381] Updated weights for policy 1, policy_version 1147075 (0.0006) [2023-12-27 13:36:16,606][20381] Updated weights for policy 1, policy_version 1147085 (0.0005) [2023-12-27 13:36:16,664][20381] Updated weights for policy 1, policy_version 1147095 (0.0008) [2023-12-27 13:36:16,947][20347] Updated weights for policy 0, policy_version 1148180 (0.0008) [2023-12-27 13:36:17,008][20347] Updated weights for policy 0, policy_version 1148190 (0.0007) [2023-12-27 13:36:17,070][20347] Updated weights for policy 0, policy_version 1148200 (0.0008) [2023-12-27 13:36:17,269][20381] Updated weights for policy 1, policy_version 1147105 (0.0007) [2023-12-27 13:36:17,326][20381] Updated weights for policy 1, policy_version 1147115 (0.0005) [2023-12-27 13:36:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 587677696. Throughput: 0: 10381.6, 1: 10114.6. Samples: 587691340. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:17,363][19170] Avg episode reward: [(0, '843.310'), (1, '628.380')] [2023-12-27 13:36:17,374][20381] Updated weights for policy 1, policy_version 1147125 (0.0005) [2023-12-27 13:36:17,798][20347] Updated weights for policy 0, policy_version 1148210 (0.0008) [2023-12-27 13:36:17,858][20347] Updated weights for policy 0, policy_version 1148220 (0.0008) [2023-12-27 13:36:17,920][20347] Updated weights for policy 0, policy_version 1148230 (0.0007) [2023-12-27 13:36:17,925][20381] Updated weights for policy 1, policy_version 1147135 (0.0007) [2023-12-27 13:36:17,969][20347] Updated weights for policy 0, policy_version 1148240 (0.0007) [2023-12-27 13:36:17,978][20381] Updated weights for policy 1, policy_version 1147145 (0.0005) [2023-12-27 13:36:18,034][20381] Updated weights for policy 1, policy_version 1147155 (0.0005) [2023-12-27 13:36:18,631][20347] Updated weights for policy 0, policy_version 1148250 (0.0007) [2023-12-27 13:36:18,688][20347] Updated weights for policy 0, policy_version 1148260 (0.0008) [2023-12-27 13:36:18,699][20381] Updated weights for policy 1, policy_version 1147165 (0.0009) [2023-12-27 13:36:18,746][20347] Updated weights for policy 0, policy_version 1148270 (0.0007) [2023-12-27 13:36:18,748][20381] Updated weights for policy 1, policy_version 1147175 (0.0010) [2023-12-27 13:36:18,800][20381] Updated weights for policy 1, policy_version 1147185 (0.0010) [2023-12-27 13:36:19,479][20347] Updated weights for policy 0, policy_version 1148280 (0.0007) [2023-12-27 13:36:19,492][20381] Updated weights for policy 1, policy_version 1147195 (0.0010) [2023-12-27 13:36:19,543][20347] Updated weights for policy 0, policy_version 1148290 (0.0007) [2023-12-27 13:36:19,556][20381] Updated weights for policy 1, policy_version 1147205 (0.0010) [2023-12-27 13:36:19,602][20347] Updated weights for policy 0, policy_version 1148300 (0.0005) [2023-12-27 13:36:19,616][20381] Updated weights for policy 1, policy_version 1147215 (0.0011) [2023-12-27 13:36:20,325][20347] Updated weights for policy 0, policy_version 1148310 (0.0006) [2023-12-27 13:36:20,340][20381] Updated weights for policy 1, policy_version 1147225 (0.0011) [2023-12-27 13:36:20,385][20347] Updated weights for policy 0, policy_version 1148320 (0.0009) [2023-12-27 13:36:20,399][20381] Updated weights for policy 1, policy_version 1147235 (0.0011) [2023-12-27 13:36:20,446][20347] Updated weights for policy 0, policy_version 1148330 (0.0006) [2023-12-27 13:36:20,446][20381] Updated weights for policy 1, policy_version 1147245 (0.0011) [2023-12-27 13:36:20,493][20381] Updated weights for policy 1, policy_version 1147255 (0.0010) [2023-12-27 13:36:21,195][20347] Updated weights for policy 0, policy_version 1148340 (0.0007) [2023-12-27 13:36:21,218][20381] Updated weights for policy 1, policy_version 1147265 (0.0006) [2023-12-27 13:36:21,252][20347] Updated weights for policy 0, policy_version 1148350 (0.0008) [2023-12-27 13:36:21,279][20381] Updated weights for policy 1, policy_version 1147275 (0.0009) [2023-12-27 13:36:21,317][20347] Updated weights for policy 0, policy_version 1148360 (0.0006) [2023-12-27 13:36:21,341][20381] Updated weights for policy 1, policy_version 1147285 (0.0011) [2023-12-27 13:36:22,060][20347] Updated weights for policy 0, policy_version 1148370 (0.0007) [2023-12-27 13:36:22,078][20381] Updated weights for policy 1, policy_version 1147295 (0.0011) [2023-12-27 13:36:22,121][20347] Updated weights for policy 0, policy_version 1148380 (0.0008) [2023-12-27 13:36:22,130][20381] Updated weights for policy 1, policy_version 1147305 (0.0005) [2023-12-27 13:36:22,170][20347] Updated weights for policy 0, policy_version 1148390 (0.0007) [2023-12-27 13:36:22,180][20381] Updated weights for policy 1, policy_version 1147315 (0.0005) [2023-12-27 13:36:22,221][20347] Updated weights for policy 0, policy_version 1148400 (0.0008) [2023-12-27 13:36:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 587784192. Throughput: 0: 10356.6, 1: 10068.1. Samples: 587751888. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:22,363][19170] Avg episode reward: [(0, '872.880'), (1, '657.780')] [2023-12-27 13:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001147320_293740544.pth... [2023-12-27 13:36:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001148400_294043648.pth... [2023-12-27 13:36:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001147184_293732352.pth [2023-12-27 13:36:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001146136_293437440.pth [2023-12-27 13:36:22,836][20381] Updated weights for policy 1, policy_version 1147325 (0.0008) [2023-12-27 13:36:22,891][20381] Updated weights for policy 1, policy_version 1147335 (0.0008) [2023-12-27 13:36:22,944][20347] Updated weights for policy 0, policy_version 1148410 (0.0007) [2023-12-27 13:36:22,947][20381] Updated weights for policy 1, policy_version 1147345 (0.0007) [2023-12-27 13:36:22,995][20347] Updated weights for policy 0, policy_version 1148420 (0.0006) [2023-12-27 13:36:23,046][20347] Updated weights for policy 0, policy_version 1148430 (0.0008) [2023-12-27 13:36:23,611][20381] Updated weights for policy 1, policy_version 1147355 (0.0007) [2023-12-27 13:36:23,666][20381] Updated weights for policy 1, policy_version 1147365 (0.0006) [2023-12-27 13:36:23,713][20347] Updated weights for policy 0, policy_version 1148440 (0.0006) [2023-12-27 13:36:23,723][20381] Updated weights for policy 1, policy_version 1147375 (0.0006) [2023-12-27 13:36:23,768][20347] Updated weights for policy 0, policy_version 1148450 (0.0006) [2023-12-27 13:36:23,813][20347] Updated weights for policy 0, policy_version 1148460 (0.0006) [2023-12-27 13:36:24,403][20381] Updated weights for policy 1, policy_version 1147385 (0.0006) [2023-12-27 13:36:24,441][20347] Updated weights for policy 0, policy_version 1148470 (0.0006) [2023-12-27 13:36:24,457][20381] Updated weights for policy 1, policy_version 1147395 (0.0008) [2023-12-27 13:36:24,500][20347] Updated weights for policy 0, policy_version 1148480 (0.0007) [2023-12-27 13:36:24,513][20381] Updated weights for policy 1, policy_version 1147405 (0.0007) [2023-12-27 13:36:24,554][20347] Updated weights for policy 0, policy_version 1148490 (0.0006) [2023-12-27 13:36:24,571][20381] Updated weights for policy 1, policy_version 1147415 (0.0007) [2023-12-27 13:36:25,130][20347] Updated weights for policy 0, policy_version 1148500 (0.0006) [2023-12-27 13:36:25,176][20347] Updated weights for policy 0, policy_version 1148510 (0.0005) [2023-12-27 13:36:25,222][20347] Updated weights for policy 0, policy_version 1148520 (0.0005) [2023-12-27 13:36:25,305][20381] Updated weights for policy 1, policy_version 1147425 (0.0007) [2023-12-27 13:36:25,360][20381] Updated weights for policy 1, policy_version 1147435 (0.0008) [2023-12-27 13:36:25,413][20381] Updated weights for policy 1, policy_version 1147445 (0.0007) [2023-12-27 13:36:25,924][20347] Updated weights for policy 0, policy_version 1148530 (0.0006) [2023-12-27 13:36:25,972][20347] Updated weights for policy 0, policy_version 1148540 (0.0007) [2023-12-27 13:36:25,993][20381] Updated weights for policy 1, policy_version 1147455 (0.0010) [2023-12-27 13:36:26,036][20347] Updated weights for policy 0, policy_version 1148550 (0.0005) [2023-12-27 13:36:26,042][20381] Updated weights for policy 1, policy_version 1147465 (0.0011) [2023-12-27 13:36:26,088][20347] Updated weights for policy 0, policy_version 1148560 (0.0005) [2023-12-27 13:36:26,095][20381] Updated weights for policy 1, policy_version 1147475 (0.0011) [2023-12-27 13:36:26,714][20347] Updated weights for policy 0, policy_version 1148570 (0.0005) [2023-12-27 13:36:26,763][20347] Updated weights for policy 0, policy_version 1148580 (0.0006) [2023-12-27 13:36:26,794][20381] Updated weights for policy 1, policy_version 1147485 (0.0010) [2023-12-27 13:36:26,813][20347] Updated weights for policy 0, policy_version 1148590 (0.0007) [2023-12-27 13:36:26,853][20381] Updated weights for policy 1, policy_version 1147495 (0.0010) [2023-12-27 13:36:26,908][20381] Updated weights for policy 1, policy_version 1147505 (0.0010) [2023-12-27 13:36:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 587882496. Throughput: 0: 10378.1, 1: 10054.9. Samples: 587873800. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:27,363][19170] Avg episode reward: [(0, '882.550'), (1, '736.650')] [2023-12-27 13:36:27,397][20347] Updated weights for policy 0, policy_version 1148600 (0.0006) [2023-12-27 13:36:27,459][20347] Updated weights for policy 0, policy_version 1148610 (0.0005) [2023-12-27 13:36:27,516][20347] Updated weights for policy 0, policy_version 1148620 (0.0005) [2023-12-27 13:36:27,519][20381] Updated weights for policy 1, policy_version 1147515 (0.0009) [2023-12-27 13:36:27,574][20381] Updated weights for policy 1, policy_version 1147525 (0.0005) [2023-12-27 13:36:27,629][20381] Updated weights for policy 1, policy_version 1147535 (0.0005) [2023-12-27 13:36:28,101][20347] Updated weights for policy 0, policy_version 1148630 (0.0005) [2023-12-27 13:36:28,155][20381] Updated weights for policy 1, policy_version 1147545 (0.0006) [2023-12-27 13:36:28,159][20347] Updated weights for policy 0, policy_version 1148640 (0.0006) [2023-12-27 13:36:28,210][20381] Updated weights for policy 1, policy_version 1147555 (0.0010) [2023-12-27 13:36:28,211][20347] Updated weights for policy 0, policy_version 1148650 (0.0006) [2023-12-27 13:36:28,265][20381] Updated weights for policy 1, policy_version 1147565 (0.0010) [2023-12-27 13:36:28,316][20381] Updated weights for policy 1, policy_version 1147575 (0.0010) [2023-12-27 13:36:28,884][20347] Updated weights for policy 0, policy_version 1148660 (0.0006) [2023-12-27 13:36:28,932][20347] Updated weights for policy 0, policy_version 1148670 (0.0007) [2023-12-27 13:36:28,983][20347] Updated weights for policy 0, policy_version 1148680 (0.0007) [2023-12-27 13:36:29,014][20381] Updated weights for policy 1, policy_version 1147585 (0.0007) [2023-12-27 13:36:29,057][20381] Updated weights for policy 1, policy_version 1147595 (0.0007) [2023-12-27 13:36:29,109][20381] Updated weights for policy 1, policy_version 1147605 (0.0005) [2023-12-27 13:36:29,638][20347] Updated weights for policy 0, policy_version 1148690 (0.0006) [2023-12-27 13:36:29,698][20347] Updated weights for policy 0, policy_version 1148700 (0.0007) [2023-12-27 13:36:29,720][20381] Updated weights for policy 1, policy_version 1147615 (0.0006) [2023-12-27 13:36:29,756][20347] Updated weights for policy 0, policy_version 1148710 (0.0010) [2023-12-27 13:36:29,772][20381] Updated weights for policy 1, policy_version 1147625 (0.0006) [2023-12-27 13:36:29,810][20347] Updated weights for policy 0, policy_version 1148720 (0.0010) [2023-12-27 13:36:29,835][20381] Updated weights for policy 1, policy_version 1147635 (0.0006) [2023-12-27 13:36:30,475][20347] Updated weights for policy 0, policy_version 1148730 (0.0010) [2023-12-27 13:36:30,534][20347] Updated weights for policy 0, policy_version 1148740 (0.0007) [2023-12-27 13:36:30,544][20381] Updated weights for policy 1, policy_version 1147645 (0.0008) [2023-12-27 13:36:30,586][20347] Updated weights for policy 0, policy_version 1148750 (0.0005) [2023-12-27 13:36:30,601][20381] Updated weights for policy 1, policy_version 1147655 (0.0007) [2023-12-27 13:36:30,659][20381] Updated weights for policy 1, policy_version 1147665 (0.0006) [2023-12-27 13:36:31,186][20347] Updated weights for policy 0, policy_version 1148760 (0.0008) [2023-12-27 13:36:31,232][20347] Updated weights for policy 0, policy_version 1148770 (0.0008) [2023-12-27 13:36:31,290][20347] Updated weights for policy 0, policy_version 1148780 (0.0008) [2023-12-27 13:36:31,312][20381] Updated weights for policy 1, policy_version 1147675 (0.0005) [2023-12-27 13:36:31,373][20381] Updated weights for policy 1, policy_version 1147685 (0.0008) [2023-12-27 13:36:31,426][20381] Updated weights for policy 1, policy_version 1147695 (0.0008) [2023-12-27 13:36:32,005][20347] Updated weights for policy 0, policy_version 1148790 (0.0006) [2023-12-27 13:36:32,065][20347] Updated weights for policy 0, policy_version 1148800 (0.0005) [2023-12-27 13:36:32,087][20381] Updated weights for policy 1, policy_version 1147705 (0.0008) [2023-12-27 13:36:32,122][20347] Updated weights for policy 0, policy_version 1148810 (0.0006) [2023-12-27 13:36:32,147][20381] Updated weights for policy 1, policy_version 1147715 (0.0007) [2023-12-27 13:36:32,206][20381] Updated weights for policy 1, policy_version 1147725 (0.0008) [2023-12-27 13:36:32,267][20381] Updated weights for policy 1, policy_version 1147735 (0.0009) [2023-12-27 13:36:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 587997184. Throughput: 0: 10384.5, 1: 10169.8. Samples: 588003036. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:32,363][19170] Avg episode reward: [(0, '863.210'), (1, '716.180')] [2023-12-27 13:36:32,721][20347] Updated weights for policy 0, policy_version 1148820 (0.0006) [2023-12-27 13:36:32,770][20347] Updated weights for policy 0, policy_version 1148830 (0.0005) [2023-12-27 13:36:32,822][20347] Updated weights for policy 0, policy_version 1148840 (0.0007) [2023-12-27 13:36:32,951][20381] Updated weights for policy 1, policy_version 1147745 (0.0008) [2023-12-27 13:36:32,996][20381] Updated weights for policy 1, policy_version 1147755 (0.0007) [2023-12-27 13:36:33,040][20381] Updated weights for policy 1, policy_version 1147765 (0.0009) [2023-12-27 13:36:33,475][20347] Updated weights for policy 0, policy_version 1148850 (0.0008) [2023-12-27 13:36:33,549][20347] Updated weights for policy 0, policy_version 1148860 (0.0008) [2023-12-27 13:36:33,603][20347] Updated weights for policy 0, policy_version 1148870 (0.0005) [2023-12-27 13:36:33,648][20347] Updated weights for policy 0, policy_version 1148880 (0.0005) [2023-12-27 13:36:33,754][20381] Updated weights for policy 1, policy_version 1147775 (0.0008) [2023-12-27 13:36:33,799][20381] Updated weights for policy 1, policy_version 1147785 (0.0007) [2023-12-27 13:36:33,843][20381] Updated weights for policy 1, policy_version 1147795 (0.0007) [2023-12-27 13:36:34,236][20347] Updated weights for policy 0, policy_version 1148890 (0.0008) [2023-12-27 13:36:34,295][20347] Updated weights for policy 0, policy_version 1148900 (0.0007) [2023-12-27 13:36:34,355][20347] Updated weights for policy 0, policy_version 1148910 (0.0008) [2023-12-27 13:36:34,573][20381] Updated weights for policy 1, policy_version 1147805 (0.0008) [2023-12-27 13:36:34,627][20381] Updated weights for policy 1, policy_version 1147815 (0.0007) [2023-12-27 13:36:34,676][20381] Updated weights for policy 1, policy_version 1147825 (0.0007) [2023-12-27 13:36:35,004][20347] Updated weights for policy 0, policy_version 1148920 (0.0010) [2023-12-27 13:36:35,060][20347] Updated weights for policy 0, policy_version 1148930 (0.0010) [2023-12-27 13:36:35,117][20347] Updated weights for policy 0, policy_version 1148940 (0.0010) [2023-12-27 13:36:35,368][20381] Updated weights for policy 1, policy_version 1147835 (0.0007) [2023-12-27 13:36:35,429][20381] Updated weights for policy 1, policy_version 1147845 (0.0007) [2023-12-27 13:36:35,485][20381] Updated weights for policy 1, policy_version 1147855 (0.0008) [2023-12-27 13:36:35,742][20347] Updated weights for policy 0, policy_version 1148950 (0.0008) [2023-12-27 13:36:35,805][20347] Updated weights for policy 0, policy_version 1148960 (0.0005) [2023-12-27 13:36:35,867][20347] Updated weights for policy 0, policy_version 1148970 (0.0006) [2023-12-27 13:36:36,158][20381] Updated weights for policy 1, policy_version 1147865 (0.0009) [2023-12-27 13:36:36,208][20381] Updated weights for policy 1, policy_version 1147875 (0.0007) [2023-12-27 13:36:36,262][20381] Updated weights for policy 1, policy_version 1147885 (0.0008) [2023-12-27 13:36:36,321][20381] Updated weights for policy 1, policy_version 1147895 (0.0008) [2023-12-27 13:36:36,528][20347] Updated weights for policy 0, policy_version 1148980 (0.0006) [2023-12-27 13:36:36,590][20347] Updated weights for policy 0, policy_version 1148990 (0.0007) [2023-12-27 13:36:36,650][20347] Updated weights for policy 0, policy_version 1149000 (0.0006) [2023-12-27 13:36:37,024][20381] Updated weights for policy 1, policy_version 1147905 (0.0009) [2023-12-27 13:36:37,073][20381] Updated weights for policy 1, policy_version 1147915 (0.0010) [2023-12-27 13:36:37,130][20381] Updated weights for policy 1, policy_version 1147925 (0.0011) [2023-12-27 13:36:37,236][20347] Updated weights for policy 0, policy_version 1149010 (0.0007) [2023-12-27 13:36:37,294][20347] Updated weights for policy 0, policy_version 1149020 (0.0007) [2023-12-27 13:36:37,347][20347] Updated weights for policy 0, policy_version 1149030 (0.0007) [2023-12-27 13:36:37,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 588095488. Throughput: 0: 10415.8, 1: 10163.4. Samples: 588065716. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:37,363][19170] Avg episode reward: [(0, '902.330'), (1, '580.900')] [2023-12-27 13:36:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001147928_293896192.pth... [2023-12-27 13:36:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001146744_293593088.pth [2023-12-27 13:36:37,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001149040_294207488.pth... [2023-12-27 13:36:37,407][20347] Updated weights for policy 0, policy_version 1149040 (0.0006) [2023-12-27 13:36:37,410][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001147824_293896192.pth [2023-12-27 13:36:37,878][20381] Updated weights for policy 1, policy_version 1147935 (0.0009) [2023-12-27 13:36:37,925][20381] Updated weights for policy 1, policy_version 1147945 (0.0008) [2023-12-27 13:36:37,944][20347] Updated weights for policy 0, policy_version 1149050 (0.0006) [2023-12-27 13:36:37,982][20381] Updated weights for policy 1, policy_version 1147955 (0.0008) [2023-12-27 13:36:37,999][20347] Updated weights for policy 0, policy_version 1149060 (0.0005) [2023-12-27 13:36:38,050][20347] Updated weights for policy 0, policy_version 1149070 (0.0005) [2023-12-27 13:36:38,682][20347] Updated weights for policy 0, policy_version 1149080 (0.0007) [2023-12-27 13:36:38,700][20381] Updated weights for policy 1, policy_version 1147965 (0.0007) [2023-12-27 13:36:38,733][20347] Updated weights for policy 0, policy_version 1149090 (0.0006) [2023-12-27 13:36:38,764][20381] Updated weights for policy 1, policy_version 1147975 (0.0007) [2023-12-27 13:36:38,794][20347] Updated weights for policy 0, policy_version 1149100 (0.0007) [2023-12-27 13:36:38,819][20381] Updated weights for policy 1, policy_version 1147985 (0.0007) [2023-12-27 13:36:39,454][20347] Updated weights for policy 0, policy_version 1149110 (0.0008) [2023-12-27 13:36:39,508][20347] Updated weights for policy 0, policy_version 1149120 (0.0009) [2023-12-27 13:36:39,565][20347] Updated weights for policy 0, policy_version 1149130 (0.0009) [2023-12-27 13:36:39,579][20381] Updated weights for policy 1, policy_version 1147995 (0.0007) [2023-12-27 13:36:39,631][20381] Updated weights for policy 1, policy_version 1148005 (0.0009) [2023-12-27 13:36:39,691][20381] Updated weights for policy 1, policy_version 1148015 (0.0008) [2023-12-27 13:36:40,401][20347] Updated weights for policy 0, policy_version 1149140 (0.0008) [2023-12-27 13:36:40,419][20381] Updated weights for policy 1, policy_version 1148025 (0.0008) [2023-12-27 13:36:40,462][20347] Updated weights for policy 0, policy_version 1149150 (0.0006) [2023-12-27 13:36:40,474][20381] Updated weights for policy 1, policy_version 1148035 (0.0005) [2023-12-27 13:36:40,519][20347] Updated weights for policy 0, policy_version 1149160 (0.0009) [2023-12-27 13:36:40,526][20381] Updated weights for policy 1, policy_version 1148045 (0.0006) [2023-12-27 13:36:40,577][20381] Updated weights for policy 1, policy_version 1148055 (0.0006) [2023-12-27 13:36:41,143][20347] Updated weights for policy 0, policy_version 1149170 (0.0010) [2023-12-27 13:36:41,211][20347] Updated weights for policy 0, policy_version 1149180 (0.0007) [2023-12-27 13:36:41,246][20381] Updated weights for policy 1, policy_version 1148065 (0.0008) [2023-12-27 13:36:41,272][20347] Updated weights for policy 0, policy_version 1149190 (0.0007) [2023-12-27 13:36:41,303][20381] Updated weights for policy 1, policy_version 1148075 (0.0008) [2023-12-27 13:36:41,333][20347] Updated weights for policy 0, policy_version 1149200 (0.0008) [2023-12-27 13:36:41,356][20381] Updated weights for policy 1, policy_version 1148085 (0.0008) [2023-12-27 13:36:42,001][20347] Updated weights for policy 0, policy_version 1149210 (0.0006) [2023-12-27 13:36:42,051][20347] Updated weights for policy 0, policy_version 1149220 (0.0005) [2023-12-27 13:36:42,111][20347] Updated weights for policy 0, policy_version 1149230 (0.0005) [2023-12-27 13:36:42,119][20381] Updated weights for policy 1, policy_version 1148095 (0.0008) [2023-12-27 13:36:42,183][20381] Updated weights for policy 1, policy_version 1148105 (0.0008) [2023-12-27 13:36:42,244][20381] Updated weights for policy 1, policy_version 1148115 (0.0008) [2023-12-27 13:36:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 588201984. Throughput: 0: 10456.2, 1: 10148.7. Samples: 588188880. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:42,363][19170] Avg episode reward: [(0, '941.400'), (1, '549.480')] [2023-12-27 13:36:42,820][20347] Updated weights for policy 0, policy_version 1149240 (0.0007) [2023-12-27 13:36:42,883][20347] Updated weights for policy 0, policy_version 1149250 (0.0008) [2023-12-27 13:36:42,940][20347] Updated weights for policy 0, policy_version 1149260 (0.0006) [2023-12-27 13:36:42,948][20381] Updated weights for policy 1, policy_version 1148125 (0.0007) [2023-12-27 13:36:43,007][20381] Updated weights for policy 1, policy_version 1148135 (0.0010) [2023-12-27 13:36:43,055][20381] Updated weights for policy 1, policy_version 1148145 (0.0010) [2023-12-27 13:36:43,504][20347] Updated weights for policy 0, policy_version 1149270 (0.0005) [2023-12-27 13:36:43,560][20347] Updated weights for policy 0, policy_version 1149280 (0.0010) [2023-12-27 13:36:43,616][20347] Updated weights for policy 0, policy_version 1149290 (0.0010) [2023-12-27 13:36:43,745][20381] Updated weights for policy 1, policy_version 1148155 (0.0011) [2023-12-27 13:36:43,797][20381] Updated weights for policy 1, policy_version 1148165 (0.0010) [2023-12-27 13:36:43,845][20381] Updated weights for policy 1, policy_version 1148175 (0.0010) [2023-12-27 13:36:44,248][20347] Updated weights for policy 0, policy_version 1149300 (0.0010) [2023-12-27 13:36:44,298][20347] Updated weights for policy 0, policy_version 1149310 (0.0010) [2023-12-27 13:36:44,353][20347] Updated weights for policy 0, policy_version 1149320 (0.0010) [2023-12-27 13:36:44,514][20381] Updated weights for policy 1, policy_version 1148185 (0.0010) [2023-12-27 13:36:44,571][20381] Updated weights for policy 1, policy_version 1148195 (0.0010) [2023-12-27 13:36:44,629][20381] Updated weights for policy 1, policy_version 1148205 (0.0010) [2023-12-27 13:36:44,679][20381] Updated weights for policy 1, policy_version 1148215 (0.0010) [2023-12-27 13:36:45,049][20347] Updated weights for policy 0, policy_version 1149330 (0.0009) [2023-12-27 13:36:45,111][20347] Updated weights for policy 0, policy_version 1149340 (0.0008) [2023-12-27 13:36:45,176][20347] Updated weights for policy 0, policy_version 1149350 (0.0008) [2023-12-27 13:36:45,243][20347] Updated weights for policy 0, policy_version 1149360 (0.0007) [2023-12-27 13:36:45,321][20381] Updated weights for policy 1, policy_version 1148225 (0.0007) [2023-12-27 13:36:45,370][20381] Updated weights for policy 1, policy_version 1148235 (0.0007) [2023-12-27 13:36:45,427][20381] Updated weights for policy 1, policy_version 1148245 (0.0007) [2023-12-27 13:36:45,943][20347] Updated weights for policy 0, policy_version 1149370 (0.0008) [2023-12-27 13:36:46,007][20347] Updated weights for policy 0, policy_version 1149380 (0.0008) [2023-12-27 13:36:46,068][20347] Updated weights for policy 0, policy_version 1149390 (0.0008) [2023-12-27 13:36:46,115][20381] Updated weights for policy 1, policy_version 1148255 (0.0009) [2023-12-27 13:36:46,166][20381] Updated weights for policy 1, policy_version 1148265 (0.0008) [2023-12-27 13:36:46,211][20381] Updated weights for policy 1, policy_version 1148275 (0.0009) [2023-12-27 13:36:46,646][20347] Updated weights for policy 0, policy_version 1149400 (0.0007) [2023-12-27 13:36:46,713][20347] Updated weights for policy 0, policy_version 1149410 (0.0010) [2023-12-27 13:36:46,762][20347] Updated weights for policy 0, policy_version 1149420 (0.0006) [2023-12-27 13:36:46,828][20381] Updated weights for policy 1, policy_version 1148285 (0.0006) [2023-12-27 13:36:46,894][20381] Updated weights for policy 1, policy_version 1148295 (0.0008) [2023-12-27 13:36:46,939][20381] Updated weights for policy 1, policy_version 1148305 (0.0010) [2023-12-27 13:36:47,325][20347] Updated weights for policy 0, policy_version 1149430 (0.0006) [2023-12-27 13:36:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 588300288. Throughput: 0: 10465.2, 1: 10182.1. Samples: 588313876. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:47,363][19170] Avg episode reward: [(0, '941.300'), (1, '686.420')] [2023-12-27 13:36:47,382][20347] Updated weights for policy 0, policy_version 1149440 (0.0007) [2023-12-27 13:36:47,441][20347] Updated weights for policy 0, policy_version 1149450 (0.0007) [2023-12-27 13:36:47,592][20381] Updated weights for policy 1, policy_version 1148315 (0.0007) [2023-12-27 13:36:47,640][20381] Updated weights for policy 1, policy_version 1148325 (0.0008) [2023-12-27 13:36:47,691][20381] Updated weights for policy 1, policy_version 1148335 (0.0008) [2023-12-27 13:36:48,016][20347] Updated weights for policy 0, policy_version 1149460 (0.0006) [2023-12-27 13:36:48,078][20347] Updated weights for policy 0, policy_version 1149470 (0.0005) [2023-12-27 13:36:48,131][20347] Updated weights for policy 0, policy_version 1149480 (0.0006) [2023-12-27 13:36:48,379][20381] Updated weights for policy 1, policy_version 1148345 (0.0008) [2023-12-27 13:36:48,441][20381] Updated weights for policy 1, policy_version 1148355 (0.0008) [2023-12-27 13:36:48,501][20381] Updated weights for policy 1, policy_version 1148365 (0.0009) [2023-12-27 13:36:48,557][20381] Updated weights for policy 1, policy_version 1148375 (0.0011) [2023-12-27 13:36:48,800][20347] Updated weights for policy 0, policy_version 1149490 (0.0007) [2023-12-27 13:36:48,853][20347] Updated weights for policy 0, policy_version 1149500 (0.0008) [2023-12-27 13:36:48,905][20347] Updated weights for policy 0, policy_version 1149510 (0.0007) [2023-12-27 13:36:48,957][20347] Updated weights for policy 0, policy_version 1149520 (0.0007) [2023-12-27 13:36:49,279][20381] Updated weights for policy 1, policy_version 1148385 (0.0007) [2023-12-27 13:36:49,331][20381] Updated weights for policy 1, policy_version 1148395 (0.0006) [2023-12-27 13:36:49,395][20381] Updated weights for policy 1, policy_version 1148405 (0.0008) [2023-12-27 13:36:49,621][20347] Updated weights for policy 0, policy_version 1149530 (0.0008) [2023-12-27 13:36:49,680][20347] Updated weights for policy 0, policy_version 1149540 (0.0008) [2023-12-27 13:36:49,736][20347] Updated weights for policy 0, policy_version 1149550 (0.0007) [2023-12-27 13:36:50,090][20381] Updated weights for policy 1, policy_version 1148415 (0.0008) [2023-12-27 13:36:50,159][20381] Updated weights for policy 1, policy_version 1148425 (0.0008) [2023-12-27 13:36:50,229][20381] Updated weights for policy 1, policy_version 1148435 (0.0007) [2023-12-27 13:36:50,475][20347] Updated weights for policy 0, policy_version 1149560 (0.0007) [2023-12-27 13:36:50,534][20347] Updated weights for policy 0, policy_version 1149570 (0.0008) [2023-12-27 13:36:50,597][20347] Updated weights for policy 0, policy_version 1149580 (0.0008) [2023-12-27 13:36:50,893][20381] Updated weights for policy 1, policy_version 1148445 (0.0007) [2023-12-27 13:36:50,950][20381] Updated weights for policy 1, policy_version 1148455 (0.0007) [2023-12-27 13:36:51,004][20381] Updated weights for policy 1, policy_version 1148465 (0.0007) [2023-12-27 13:36:51,354][20347] Updated weights for policy 0, policy_version 1149590 (0.0009) [2023-12-27 13:36:51,415][20347] Updated weights for policy 0, policy_version 1149600 (0.0007) [2023-12-27 13:36:51,467][20347] Updated weights for policy 0, policy_version 1149610 (0.0010) [2023-12-27 13:36:51,745][20381] Updated weights for policy 1, policy_version 1148475 (0.0008) [2023-12-27 13:36:51,813][20381] Updated weights for policy 1, policy_version 1148485 (0.0008) [2023-12-27 13:36:51,879][20381] Updated weights for policy 1, policy_version 1148495 (0.0008) [2023-12-27 13:36:52,171][20347] Updated weights for policy 0, policy_version 1149620 (0.0006) [2023-12-27 13:36:52,224][20347] Updated weights for policy 0, policy_version 1149630 (0.0007) [2023-12-27 13:36:52,278][20347] Updated weights for policy 0, policy_version 1149640 (0.0007) [2023-12-27 13:36:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 588406784. Throughput: 0: 10442.7, 1: 10173.3. Samples: 588375592. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:52,363][19170] Avg episode reward: [(0, '941.180'), (1, '696.510')] [2023-12-27 13:36:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001148504_294043648.pth... [2023-12-27 13:36:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001149648_294363136.pth... [2023-12-27 13:36:52,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001147320_293740544.pth [2023-12-27 13:36:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001148400_294043648.pth [2023-12-27 13:36:52,507][20381] Updated weights for policy 1, policy_version 1148505 (0.0007) [2023-12-27 13:36:52,567][20381] Updated weights for policy 1, policy_version 1148515 (0.0008) [2023-12-27 13:36:52,628][20381] Updated weights for policy 1, policy_version 1148525 (0.0008) [2023-12-27 13:36:52,684][20381] Updated weights for policy 1, policy_version 1148535 (0.0008) [2023-12-27 13:36:53,005][20347] Updated weights for policy 0, policy_version 1149650 (0.0007) [2023-12-27 13:36:53,053][20347] Updated weights for policy 0, policy_version 1149660 (0.0007) [2023-12-27 13:36:53,108][20347] Updated weights for policy 0, policy_version 1149670 (0.0008) [2023-12-27 13:36:53,166][20347] Updated weights for policy 0, policy_version 1149680 (0.0008) [2023-12-27 13:36:53,330][20381] Updated weights for policy 1, policy_version 1148545 (0.0008) [2023-12-27 13:36:53,390][20381] Updated weights for policy 1, policy_version 1148555 (0.0007) [2023-12-27 13:36:53,451][20381] Updated weights for policy 1, policy_version 1148565 (0.0007) [2023-12-27 13:36:53,853][20347] Updated weights for policy 0, policy_version 1149690 (0.0010) [2023-12-27 13:36:53,914][20347] Updated weights for policy 0, policy_version 1149700 (0.0010) [2023-12-27 13:36:53,969][20347] Updated weights for policy 0, policy_version 1149710 (0.0010) [2023-12-27 13:36:54,101][20381] Updated weights for policy 1, policy_version 1148575 (0.0006) [2023-12-27 13:36:54,160][20381] Updated weights for policy 1, policy_version 1148585 (0.0007) [2023-12-27 13:36:54,212][20381] Updated weights for policy 1, policy_version 1148595 (0.0007) [2023-12-27 13:36:54,631][20347] Updated weights for policy 0, policy_version 1149720 (0.0010) [2023-12-27 13:36:54,679][20347] Updated weights for policy 0, policy_version 1149730 (0.0010) [2023-12-27 13:36:54,727][20347] Updated weights for policy 0, policy_version 1149740 (0.0010) [2023-12-27 13:36:54,901][20381] Updated weights for policy 1, policy_version 1148605 (0.0007) [2023-12-27 13:36:54,953][20381] Updated weights for policy 1, policy_version 1148615 (0.0008) [2023-12-27 13:36:55,005][20381] Updated weights for policy 1, policy_version 1148625 (0.0008) [2023-12-27 13:36:55,375][20347] Updated weights for policy 0, policy_version 1149750 (0.0010) [2023-12-27 13:36:55,427][20347] Updated weights for policy 0, policy_version 1149760 (0.0008) [2023-12-27 13:36:55,483][20347] Updated weights for policy 0, policy_version 1149770 (0.0005) [2023-12-27 13:36:55,763][20381] Updated weights for policy 1, policy_version 1148635 (0.0009) [2023-12-27 13:36:55,825][20381] Updated weights for policy 1, policy_version 1148645 (0.0009) [2023-12-27 13:36:55,875][20381] Updated weights for policy 1, policy_version 1148655 (0.0009) [2023-12-27 13:36:56,028][20347] Updated weights for policy 0, policy_version 1149780 (0.0006) [2023-12-27 13:36:56,083][20347] Updated weights for policy 0, policy_version 1149790 (0.0006) [2023-12-27 13:36:56,134][20347] Updated weights for policy 0, policy_version 1149800 (0.0005) [2023-12-27 13:36:56,655][20381] Updated weights for policy 1, policy_version 1148665 (0.0009) [2023-12-27 13:36:56,660][20347] Updated weights for policy 0, policy_version 1149810 (0.0005) [2023-12-27 13:36:56,704][20381] Updated weights for policy 1, policy_version 1148675 (0.0007) [2023-12-27 13:36:56,708][20347] Updated weights for policy 0, policy_version 1149820 (0.0005) [2023-12-27 13:36:56,754][20347] Updated weights for policy 0, policy_version 1149830 (0.0005) [2023-12-27 13:36:56,764][20381] Updated weights for policy 1, policy_version 1148685 (0.0007) [2023-12-27 13:36:56,799][20347] Updated weights for policy 0, policy_version 1149840 (0.0005) [2023-12-27 13:36:56,820][20381] Updated weights for policy 1, policy_version 1148695 (0.0008) [2023-12-27 13:36:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 588505088. Throughput: 0: 10457.9, 1: 10183.3. Samples: 588497652. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:36:57,363][19170] Avg episode reward: [(0, '941.290'), (1, '657.340')] [2023-12-27 13:36:57,430][20347] Updated weights for policy 0, policy_version 1149850 (0.0010) [2023-12-27 13:36:57,454][20381] Updated weights for policy 1, policy_version 1148705 (0.0005) [2023-12-27 13:36:57,477][20347] Updated weights for policy 0, policy_version 1149860 (0.0010) [2023-12-27 13:36:57,509][20381] Updated weights for policy 1, policy_version 1148715 (0.0005) [2023-12-27 13:36:57,528][20347] Updated weights for policy 0, policy_version 1149870 (0.0010) [2023-12-27 13:36:57,564][20381] Updated weights for policy 1, policy_version 1148725 (0.0007) [2023-12-27 13:36:58,174][20347] Updated weights for policy 0, policy_version 1149880 (0.0008) [2023-12-27 13:36:58,238][20347] Updated weights for policy 0, policy_version 1149890 (0.0008) [2023-12-27 13:36:58,298][20347] Updated weights for policy 0, policy_version 1149900 (0.0007) [2023-12-27 13:36:58,299][20381] Updated weights for policy 1, policy_version 1148735 (0.0007) [2023-12-27 13:36:58,387][20381] Updated weights for policy 1, policy_version 1148745 (0.0007) [2023-12-27 13:36:58,455][20381] Updated weights for policy 1, policy_version 1148755 (0.0008) [2023-12-27 13:36:59,112][20347] Updated weights for policy 0, policy_version 1149910 (0.0007) [2023-12-27 13:36:59,164][20347] Updated weights for policy 0, policy_version 1149920 (0.0007) [2023-12-27 13:36:59,227][20347] Updated weights for policy 0, policy_version 1149930 (0.0008) [2023-12-27 13:36:59,244][20381] Updated weights for policy 1, policy_version 1148765 (0.0008) [2023-12-27 13:36:59,307][20381] Updated weights for policy 1, policy_version 1148775 (0.0010) [2023-12-27 13:36:59,371][20381] Updated weights for policy 1, policy_version 1148785 (0.0008) [2023-12-27 13:36:59,989][20347] Updated weights for policy 0, policy_version 1149940 (0.0008) [2023-12-27 13:37:00,027][20381] Updated weights for policy 1, policy_version 1148795 (0.0008) [2023-12-27 13:37:00,052][20347] Updated weights for policy 0, policy_version 1149950 (0.0009) [2023-12-27 13:37:00,085][20381] Updated weights for policy 1, policy_version 1148805 (0.0005) [2023-12-27 13:37:00,118][20347] Updated weights for policy 0, policy_version 1149960 (0.0008) [2023-12-27 13:37:00,142][20381] Updated weights for policy 1, policy_version 1148815 (0.0005) [2023-12-27 13:37:00,735][20347] Updated weights for policy 0, policy_version 1149970 (0.0007) [2023-12-27 13:37:00,791][20381] Updated weights for policy 1, policy_version 1148825 (0.0007) [2023-12-27 13:37:00,792][20347] Updated weights for policy 0, policy_version 1149980 (0.0007) [2023-12-27 13:37:00,840][20381] Updated weights for policy 1, policy_version 1148835 (0.0007) [2023-12-27 13:37:00,847][20347] Updated weights for policy 0, policy_version 1149990 (0.0005) [2023-12-27 13:37:00,893][20381] Updated weights for policy 1, policy_version 1148845 (0.0006) [2023-12-27 13:37:00,902][20347] Updated weights for policy 0, policy_version 1150000 (0.0009) [2023-12-27 13:37:00,942][20381] Updated weights for policy 1, policy_version 1148855 (0.0007) [2023-12-27 13:37:01,614][20347] Updated weights for policy 0, policy_version 1150010 (0.0008) [2023-12-27 13:37:01,647][20381] Updated weights for policy 1, policy_version 1148865 (0.0006) [2023-12-27 13:37:01,675][20347] Updated weights for policy 0, policy_version 1150020 (0.0008) [2023-12-27 13:37:01,714][20381] Updated weights for policy 1, policy_version 1148875 (0.0006) [2023-12-27 13:37:01,739][20347] Updated weights for policy 0, policy_version 1150030 (0.0007) [2023-12-27 13:37:01,765][20381] Updated weights for policy 1, policy_version 1148885 (0.0008) [2023-12-27 13:37:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 588603392. Throughput: 0: 10473.4, 1: 10150.9. Samples: 588619432. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:02,363][19170] Avg episode reward: [(0, '941.350'), (1, '647.830')] [2023-12-27 13:37:02,375][20347] Updated weights for policy 0, policy_version 1150040 (0.0008) [2023-12-27 13:37:02,433][20347] Updated weights for policy 0, policy_version 1150050 (0.0007) [2023-12-27 13:37:02,493][20381] Updated weights for policy 1, policy_version 1148895 (0.0006) [2023-12-27 13:37:02,494][20347] Updated weights for policy 0, policy_version 1150060 (0.0008) [2023-12-27 13:37:02,548][20381] Updated weights for policy 1, policy_version 1148905 (0.0007) [2023-12-27 13:37:02,604][20381] Updated weights for policy 1, policy_version 1148915 (0.0006) [2023-12-27 13:37:03,152][20381] Updated weights for policy 1, policy_version 1148925 (0.0005) [2023-12-27 13:37:03,168][20347] Updated weights for policy 0, policy_version 1150070 (0.0007) [2023-12-27 13:37:03,202][20381] Updated weights for policy 1, policy_version 1148935 (0.0005) [2023-12-27 13:37:03,226][20347] Updated weights for policy 0, policy_version 1150080 (0.0007) [2023-12-27 13:37:03,250][20381] Updated weights for policy 1, policy_version 1148945 (0.0005) [2023-12-27 13:37:03,281][20347] Updated weights for policy 0, policy_version 1150090 (0.0007) [2023-12-27 13:37:03,893][20347] Updated weights for policy 0, policy_version 1150100 (0.0008) [2023-12-27 13:37:03,948][20347] Updated weights for policy 0, policy_version 1150110 (0.0007) [2023-12-27 13:37:03,964][20381] Updated weights for policy 1, policy_version 1148955 (0.0007) [2023-12-27 13:37:04,001][20347] Updated weights for policy 0, policy_version 1150120 (0.0006) [2023-12-27 13:37:04,015][20381] Updated weights for policy 1, policy_version 1148965 (0.0005) [2023-12-27 13:37:04,061][20381] Updated weights for policy 1, policy_version 1148975 (0.0008) [2023-12-27 13:37:04,595][20347] Updated weights for policy 0, policy_version 1150130 (0.0005) [2023-12-27 13:37:04,651][20347] Updated weights for policy 0, policy_version 1150140 (0.0005) [2023-12-27 13:37:04,708][20347] Updated weights for policy 0, policy_version 1150150 (0.0005) [2023-12-27 13:37:04,738][20381] Updated weights for policy 1, policy_version 1148985 (0.0008) [2023-12-27 13:37:04,758][20347] Updated weights for policy 0, policy_version 1150160 (0.0005) [2023-12-27 13:37:04,786][20381] Updated weights for policy 1, policy_version 1148995 (0.0007) [2023-12-27 13:37:04,834][20381] Updated weights for policy 1, policy_version 1149005 (0.0008) [2023-12-27 13:37:04,881][20381] Updated weights for policy 1, policy_version 1149015 (0.0009) [2023-12-27 13:37:05,299][20347] Updated weights for policy 0, policy_version 1150170 (0.0007) [2023-12-27 13:37:05,357][20347] Updated weights for policy 0, policy_version 1150180 (0.0006) [2023-12-27 13:37:05,404][20347] Updated weights for policy 0, policy_version 1150190 (0.0007) [2023-12-27 13:37:05,627][20381] Updated weights for policy 1, policy_version 1149025 (0.0006) [2023-12-27 13:37:05,670][20381] Updated weights for policy 1, policy_version 1149035 (0.0005) [2023-12-27 13:37:05,719][20381] Updated weights for policy 1, policy_version 1149045 (0.0005) [2023-12-27 13:37:06,071][20347] Updated weights for policy 0, policy_version 1150200 (0.0008) [2023-12-27 13:37:06,133][20347] Updated weights for policy 0, policy_version 1150210 (0.0008) [2023-12-27 13:37:06,192][20347] Updated weights for policy 0, policy_version 1150220 (0.0007) [2023-12-27 13:37:06,307][20381] Updated weights for policy 1, policy_version 1149055 (0.0007) [2023-12-27 13:37:06,369][20381] Updated weights for policy 1, policy_version 1149065 (0.0007) [2023-12-27 13:37:06,429][20381] Updated weights for policy 1, policy_version 1149075 (0.0008) [2023-12-27 13:37:06,885][20347] Updated weights for policy 0, policy_version 1150230 (0.0008) [2023-12-27 13:37:06,933][20347] Updated weights for policy 0, policy_version 1150240 (0.0008) [2023-12-27 13:37:06,982][20347] Updated weights for policy 0, policy_version 1150250 (0.0007) [2023-12-27 13:37:07,115][20381] Updated weights for policy 1, policy_version 1149085 (0.0008) [2023-12-27 13:37:07,174][20381] Updated weights for policy 1, policy_version 1149095 (0.0007) [2023-12-27 13:37:07,231][20381] Updated weights for policy 1, policy_version 1149105 (0.0008) [2023-12-27 13:37:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 588718080. Throughput: 0: 10558.6, 1: 10162.8. Samples: 588684348. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:07,363][19170] Avg episode reward: [(0, '941.350'), (1, '716.220')] [2023-12-27 13:37:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001149112_294199296.pth... [2023-12-27 13:37:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001150256_294518784.pth... [2023-12-27 13:37:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001149040_294207488.pth [2023-12-27 13:37:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001147928_293896192.pth [2023-12-27 13:37:07,620][20347] Updated weights for policy 0, policy_version 1150260 (0.0007) [2023-12-27 13:37:07,669][20347] Updated weights for policy 0, policy_version 1150270 (0.0005) [2023-12-27 13:37:07,726][20347] Updated weights for policy 0, policy_version 1150280 (0.0005) [2023-12-27 13:37:07,965][20381] Updated weights for policy 1, policy_version 1149115 (0.0007) [2023-12-27 13:37:08,017][20381] Updated weights for policy 1, policy_version 1149125 (0.0007) [2023-12-27 13:37:08,082][20381] Updated weights for policy 1, policy_version 1149135 (0.0008) [2023-12-27 13:37:08,273][20347] Updated weights for policy 0, policy_version 1150290 (0.0005) [2023-12-27 13:37:08,337][20347] Updated weights for policy 0, policy_version 1150300 (0.0006) [2023-12-27 13:37:08,400][20347] Updated weights for policy 0, policy_version 1150310 (0.0006) [2023-12-27 13:37:08,461][20347] Updated weights for policy 0, policy_version 1150320 (0.0007) [2023-12-27 13:37:08,828][20381] Updated weights for policy 1, policy_version 1149145 (0.0008) [2023-12-27 13:37:08,887][20381] Updated weights for policy 1, policy_version 1149155 (0.0009) [2023-12-27 13:37:08,952][20381] Updated weights for policy 1, policy_version 1149165 (0.0010) [2023-12-27 13:37:09,005][20381] Updated weights for policy 1, policy_version 1149175 (0.0007) [2023-12-27 13:37:09,049][20347] Updated weights for policy 0, policy_version 1150330 (0.0008) [2023-12-27 13:37:09,103][20347] Updated weights for policy 0, policy_version 1150340 (0.0007) [2023-12-27 13:37:09,164][20347] Updated weights for policy 0, policy_version 1150350 (0.0006) [2023-12-27 13:37:09,743][20381] Updated weights for policy 1, policy_version 1149185 (0.0008) [2023-12-27 13:37:09,800][20381] Updated weights for policy 1, policy_version 1149195 (0.0008) [2023-12-27 13:37:09,867][20381] Updated weights for policy 1, policy_version 1149205 (0.0009) [2023-12-27 13:37:09,931][20347] Updated weights for policy 0, policy_version 1150360 (0.0007) [2023-12-27 13:37:09,997][20347] Updated weights for policy 0, policy_version 1150370 (0.0008) [2023-12-27 13:37:10,061][20347] Updated weights for policy 0, policy_version 1150380 (0.0009) [2023-12-27 13:37:10,570][20381] Updated weights for policy 1, policy_version 1149215 (0.0008) [2023-12-27 13:37:10,620][20381] Updated weights for policy 1, policy_version 1149225 (0.0007) [2023-12-27 13:37:10,670][20381] Updated weights for policy 1, policy_version 1149235 (0.0006) [2023-12-27 13:37:10,694][20347] Updated weights for policy 0, policy_version 1150390 (0.0006) [2023-12-27 13:37:10,749][20347] Updated weights for policy 0, policy_version 1150400 (0.0005) [2023-12-27 13:37:10,807][20347] Updated weights for policy 0, policy_version 1150410 (0.0005) [2023-12-27 13:37:11,456][20347] Updated weights for policy 0, policy_version 1150420 (0.0006) [2023-12-27 13:37:11,458][20381] Updated weights for policy 1, policy_version 1149245 (0.0007) [2023-12-27 13:37:11,520][20381] Updated weights for policy 1, policy_version 1149255 (0.0007) [2023-12-27 13:37:11,522][20347] Updated weights for policy 0, policy_version 1150430 (0.0008) [2023-12-27 13:37:11,579][20381] Updated weights for policy 1, policy_version 1149265 (0.0007) [2023-12-27 13:37:11,580][20347] Updated weights for policy 0, policy_version 1150440 (0.0006) [2023-12-27 13:37:12,274][20347] Updated weights for policy 0, policy_version 1150450 (0.0007) [2023-12-27 13:37:12,299][20381] Updated weights for policy 1, policy_version 1149275 (0.0007) [2023-12-27 13:37:12,339][20347] Updated weights for policy 0, policy_version 1150460 (0.0008) [2023-12-27 13:37:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 588808192. Throughput: 0: 10640.7, 1: 10110.0. Samples: 588807576. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:12,363][19170] Avg episode reward: [(0, '931.740'), (1, '667.550')] [2023-12-27 13:37:12,373][20381] Updated weights for policy 1, policy_version 1149285 (0.0009) [2023-12-27 13:37:12,403][20347] Updated weights for policy 0, policy_version 1150470 (0.0007) [2023-12-27 13:37:12,428][20381] Updated weights for policy 1, policy_version 1149295 (0.0007) [2023-12-27 13:37:12,458][20347] Updated weights for policy 0, policy_version 1150480 (0.0007) [2023-12-27 13:37:13,130][20347] Updated weights for policy 0, policy_version 1150490 (0.0008) [2023-12-27 13:37:13,131][20381] Updated weights for policy 1, policy_version 1149305 (0.0007) [2023-12-27 13:37:13,185][20347] Updated weights for policy 0, policy_version 1150500 (0.0006) [2023-12-27 13:37:13,187][20381] Updated weights for policy 1, policy_version 1149315 (0.0008) [2023-12-27 13:37:13,236][20347] Updated weights for policy 0, policy_version 1150510 (0.0007) [2023-12-27 13:37:13,237][20381] Updated weights for policy 1, policy_version 1149325 (0.0007) [2023-12-27 13:37:13,284][20381] Updated weights for policy 1, policy_version 1149335 (0.0007) [2023-12-27 13:37:13,901][20347] Updated weights for policy 0, policy_version 1150520 (0.0009) [2023-12-27 13:37:13,956][20347] Updated weights for policy 0, policy_version 1150530 (0.0010) [2023-12-27 13:37:13,982][20381] Updated weights for policy 1, policy_version 1149345 (0.0006) [2023-12-27 13:37:14,006][20347] Updated weights for policy 0, policy_version 1150540 (0.0010) [2023-12-27 13:37:14,033][20381] Updated weights for policy 1, policy_version 1149355 (0.0005) [2023-12-27 13:37:14,080][20381] Updated weights for policy 1, policy_version 1149365 (0.0008) [2023-12-27 13:37:14,687][20347] Updated weights for policy 0, policy_version 1150550 (0.0010) [2023-12-27 13:37:14,741][20347] Updated weights for policy 0, policy_version 1150560 (0.0010) [2023-12-27 13:37:14,802][20381] Updated weights for policy 1, policy_version 1149375 (0.0007) [2023-12-27 13:37:14,808][20347] Updated weights for policy 0, policy_version 1150570 (0.0011) [2023-12-27 13:37:14,862][20381] Updated weights for policy 1, policy_version 1149385 (0.0006) [2023-12-27 13:37:14,923][20381] Updated weights for policy 1, policy_version 1149395 (0.0007) [2023-12-27 13:37:15,460][20347] Updated weights for policy 0, policy_version 1150580 (0.0011) [2023-12-27 13:37:15,518][20347] Updated weights for policy 0, policy_version 1150590 (0.0010) [2023-12-27 13:37:15,567][20381] Updated weights for policy 1, policy_version 1149405 (0.0006) [2023-12-27 13:37:15,576][20347] Updated weights for policy 0, policy_version 1150600 (0.0010) [2023-12-27 13:37:15,614][20381] Updated weights for policy 1, policy_version 1149415 (0.0005) [2023-12-27 13:37:15,678][20381] Updated weights for policy 1, policy_version 1149425 (0.0007) [2023-12-27 13:37:16,252][20347] Updated weights for policy 0, policy_version 1150610 (0.0010) [2023-12-27 13:37:16,307][20347] Updated weights for policy 0, policy_version 1150620 (0.0010) [2023-12-27 13:37:16,352][20347] Updated weights for policy 0, policy_version 1150630 (0.0010) [2023-12-27 13:37:16,353][20381] Updated weights for policy 1, policy_version 1149435 (0.0007) [2023-12-27 13:37:16,402][20381] Updated weights for policy 1, policy_version 1149445 (0.0005) [2023-12-27 13:37:16,404][20347] Updated weights for policy 0, policy_version 1150640 (0.0010) [2023-12-27 13:37:16,449][20381] Updated weights for policy 1, policy_version 1149455 (0.0007) [2023-12-27 13:37:17,038][20381] Updated weights for policy 1, policy_version 1149465 (0.0006) [2023-12-27 13:37:17,094][20381] Updated weights for policy 1, policy_version 1149475 (0.0005) [2023-12-27 13:37:17,100][20347] Updated weights for policy 0, policy_version 1150650 (0.0010) [2023-12-27 13:37:17,146][20381] Updated weights for policy 1, policy_version 1149485 (0.0005) [2023-12-27 13:37:17,153][20347] Updated weights for policy 0, policy_version 1150660 (0.0010) [2023-12-27 13:37:17,198][20381] Updated weights for policy 1, policy_version 1149495 (0.0005) [2023-12-27 13:37:17,202][20347] Updated weights for policy 0, policy_version 1150670 (0.0010) [2023-12-27 13:37:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 588922880. Throughput: 0: 10546.7, 1: 10038.1. Samples: 588929352. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:17,363][19170] Avg episode reward: [(0, '951.260'), (1, '726.220')] [2023-12-27 13:37:17,827][20381] Updated weights for policy 1, policy_version 1149505 (0.0007) [2023-12-27 13:37:17,879][20381] Updated weights for policy 1, policy_version 1149515 (0.0007) [2023-12-27 13:37:17,898][20347] Updated weights for policy 0, policy_version 1150680 (0.0010) [2023-12-27 13:37:17,929][20381] Updated weights for policy 1, policy_version 1149525 (0.0006) [2023-12-27 13:37:17,953][20347] Updated weights for policy 0, policy_version 1150690 (0.0010) [2023-12-27 13:37:18,013][20347] Updated weights for policy 0, policy_version 1150700 (0.0010) [2023-12-27 13:37:18,641][20381] Updated weights for policy 1, policy_version 1149535 (0.0007) [2023-12-27 13:37:18,686][20347] Updated weights for policy 0, policy_version 1150710 (0.0008) [2023-12-27 13:37:18,698][20381] Updated weights for policy 1, policy_version 1149545 (0.0007) [2023-12-27 13:37:18,746][20347] Updated weights for policy 0, policy_version 1150720 (0.0007) [2023-12-27 13:37:18,761][20381] Updated weights for policy 1, policy_version 1149555 (0.0008) [2023-12-27 13:37:18,805][20347] Updated weights for policy 0, policy_version 1150730 (0.0008) [2023-12-27 13:37:19,464][20347] Updated weights for policy 0, policy_version 1150740 (0.0008) [2023-12-27 13:37:19,481][20381] Updated weights for policy 1, policy_version 1149565 (0.0007) [2023-12-27 13:37:19,526][20347] Updated weights for policy 0, policy_version 1150750 (0.0010) [2023-12-27 13:37:19,549][20381] Updated weights for policy 1, policy_version 1149575 (0.0007) [2023-12-27 13:37:19,584][20347] Updated weights for policy 0, policy_version 1150760 (0.0010) [2023-12-27 13:37:19,609][20381] Updated weights for policy 1, policy_version 1149585 (0.0006) [2023-12-27 13:37:20,232][20347] Updated weights for policy 0, policy_version 1150770 (0.0010) [2023-12-27 13:37:20,294][20347] Updated weights for policy 0, policy_version 1150780 (0.0009) [2023-12-27 13:37:20,353][20347] Updated weights for policy 0, policy_version 1150790 (0.0010) [2023-12-27 13:37:20,362][20381] Updated weights for policy 1, policy_version 1149595 (0.0006) [2023-12-27 13:37:20,409][20347] Updated weights for policy 0, policy_version 1150800 (0.0011) [2023-12-27 13:37:20,419][20381] Updated weights for policy 1, policy_version 1149605 (0.0006) [2023-12-27 13:37:20,450][20292] KL-divergence is very high: 192.5613 [2023-12-27 13:37:20,469][20381] Updated weights for policy 1, policy_version 1149615 (0.0007) [2023-12-27 13:37:21,088][20347] Updated weights for policy 0, policy_version 1150810 (0.0009) [2023-12-27 13:37:21,153][20381] Updated weights for policy 1, policy_version 1149625 (0.0007) [2023-12-27 13:37:21,157][20347] Updated weights for policy 0, policy_version 1150820 (0.0007) [2023-12-27 13:37:21,214][20381] Updated weights for policy 1, policy_version 1149635 (0.0006) [2023-12-27 13:37:21,222][20347] Updated weights for policy 0, policy_version 1150830 (0.0007) [2023-12-27 13:37:21,279][20381] Updated weights for policy 1, policy_version 1149645 (0.0007) [2023-12-27 13:37:21,332][20381] Updated weights for policy 1, policy_version 1149655 (0.0008) [2023-12-27 13:37:21,930][20347] Updated weights for policy 0, policy_version 1150840 (0.0006) [2023-12-27 13:37:21,995][20347] Updated weights for policy 0, policy_version 1150850 (0.0006) [2023-12-27 13:37:22,059][20347] Updated weights for policy 0, policy_version 1150860 (0.0005) [2023-12-27 13:37:22,097][20381] Updated weights for policy 1, policy_version 1149665 (0.0010) [2023-12-27 13:37:22,156][20381] Updated weights for policy 1, policy_version 1149675 (0.0010) [2023-12-27 13:37:22,220][20381] Updated weights for policy 1, policy_version 1149685 (0.0011) [2023-12-27 13:37:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 589021184. Throughput: 0: 10502.1, 1: 10023.4. Samples: 588989364. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:22,363][19170] Avg episode reward: [(0, '922.040'), (1, '657.480')] [2023-12-27 13:37:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001149688_294346752.pth... [2023-12-27 13:37:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001150864_294674432.pth... [2023-12-27 13:37:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001148504_294043648.pth [2023-12-27 13:37:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001149648_294363136.pth [2023-12-27 13:37:22,766][20347] Updated weights for policy 0, policy_version 1150870 (0.0005) [2023-12-27 13:37:22,827][20347] Updated weights for policy 0, policy_version 1150880 (0.0005) [2023-12-27 13:37:22,886][20347] Updated weights for policy 0, policy_version 1150890 (0.0008) [2023-12-27 13:37:22,967][20381] Updated weights for policy 1, policy_version 1149695 (0.0009) [2023-12-27 13:37:23,023][20381] Updated weights for policy 1, policy_version 1149705 (0.0007) [2023-12-27 13:37:23,070][20381] Updated weights for policy 1, policy_version 1149715 (0.0008) [2023-12-27 13:37:23,587][20347] Updated weights for policy 0, policy_version 1150900 (0.0009) [2023-12-27 13:37:23,642][20347] Updated weights for policy 0, policy_version 1150910 (0.0010) [2023-12-27 13:37:23,704][20347] Updated weights for policy 0, policy_version 1150920 (0.0010) [2023-12-27 13:37:23,724][20381] Updated weights for policy 1, policy_version 1149725 (0.0009) [2023-12-27 13:37:23,778][20381] Updated weights for policy 1, policy_version 1149735 (0.0008) [2023-12-27 13:37:23,832][20381] Updated weights for policy 1, policy_version 1149745 (0.0008) [2023-12-27 13:37:24,404][20347] Updated weights for policy 0, policy_version 1150930 (0.0010) [2023-12-27 13:37:24,465][20347] Updated weights for policy 0, policy_version 1150940 (0.0010) [2023-12-27 13:37:24,519][20347] Updated weights for policy 0, policy_version 1150950 (0.0010) [2023-12-27 13:37:24,568][20381] Updated weights for policy 1, policy_version 1149755 (0.0007) [2023-12-27 13:37:24,578][20347] Updated weights for policy 0, policy_version 1150960 (0.0010) [2023-12-27 13:37:24,615][20381] Updated weights for policy 1, policy_version 1149765 (0.0007) [2023-12-27 13:37:24,662][20381] Updated weights for policy 1, policy_version 1149775 (0.0007) [2023-12-27 13:37:25,216][20347] Updated weights for policy 0, policy_version 1150970 (0.0005) [2023-12-27 13:37:25,265][20347] Updated weights for policy 0, policy_version 1150980 (0.0005) [2023-12-27 13:37:25,311][20347] Updated weights for policy 0, policy_version 1150990 (0.0005) [2023-12-27 13:37:25,395][20381] Updated weights for policy 1, policy_version 1149785 (0.0008) [2023-12-27 13:37:25,450][20381] Updated weights for policy 1, policy_version 1149795 (0.0008) [2023-12-27 13:37:25,512][20381] Updated weights for policy 1, policy_version 1149805 (0.0007) [2023-12-27 13:37:25,574][20381] Updated weights for policy 1, policy_version 1149815 (0.0007) [2023-12-27 13:37:25,849][20347] Updated weights for policy 0, policy_version 1151000 (0.0005) [2023-12-27 13:37:25,905][20347] Updated weights for policy 0, policy_version 1151010 (0.0009) [2023-12-27 13:37:25,954][20347] Updated weights for policy 0, policy_version 1151020 (0.0005) [2023-12-27 13:37:26,240][20381] Updated weights for policy 1, policy_version 1149825 (0.0010) [2023-12-27 13:37:26,289][20381] Updated weights for policy 1, policy_version 1149835 (0.0010) [2023-12-27 13:37:26,336][20381] Updated weights for policy 1, policy_version 1149845 (0.0010) [2023-12-27 13:37:26,505][20347] Updated weights for policy 0, policy_version 1151030 (0.0008) [2023-12-27 13:37:26,561][20347] Updated weights for policy 0, policy_version 1151040 (0.0005) [2023-12-27 13:37:26,608][20347] Updated weights for policy 0, policy_version 1151050 (0.0005) [2023-12-27 13:37:26,925][20381] Updated weights for policy 1, policy_version 1149855 (0.0008) [2023-12-27 13:37:26,979][20381] Updated weights for policy 1, policy_version 1149865 (0.0006) [2023-12-27 13:37:27,034][20381] Updated weights for policy 1, policy_version 1149875 (0.0010) [2023-12-27 13:37:27,182][20347] Updated weights for policy 0, policy_version 1151060 (0.0006) [2023-12-27 13:37:27,234][20347] Updated weights for policy 0, policy_version 1151070 (0.0007) [2023-12-27 13:37:27,284][20347] Updated weights for policy 0, policy_version 1151080 (0.0008) [2023-12-27 13:37:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 589127680. Throughput: 0: 10464.9, 1: 10032.5. Samples: 589111268. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:27,364][19170] Avg episode reward: [(0, '921.970'), (1, '618.120')] [2023-12-27 13:37:27,697][20381] Updated weights for policy 1, policy_version 1149885 (0.0009) [2023-12-27 13:37:27,757][20381] Updated weights for policy 1, policy_version 1149895 (0.0008) [2023-12-27 13:37:27,814][20381] Updated weights for policy 1, policy_version 1149905 (0.0008) [2023-12-27 13:37:27,914][20347] Updated weights for policy 0, policy_version 1151090 (0.0006) [2023-12-27 13:37:27,971][20347] Updated weights for policy 0, policy_version 1151100 (0.0005) [2023-12-27 13:37:28,035][20347] Updated weights for policy 0, policy_version 1151110 (0.0006) [2023-12-27 13:37:28,094][20347] Updated weights for policy 0, policy_version 1151120 (0.0005) [2023-12-27 13:37:28,384][20381] Updated weights for policy 1, policy_version 1149915 (0.0008) [2023-12-27 13:37:28,441][20381] Updated weights for policy 1, policy_version 1149925 (0.0006) [2023-12-27 13:37:28,505][20381] Updated weights for policy 1, policy_version 1149935 (0.0006) [2023-12-27 13:37:28,705][20347] Updated weights for policy 0, policy_version 1151130 (0.0011) [2023-12-27 13:37:28,766][20347] Updated weights for policy 0, policy_version 1151140 (0.0010) [2023-12-27 13:37:28,825][20347] Updated weights for policy 0, policy_version 1151150 (0.0010) [2023-12-27 13:37:29,168][20381] Updated weights for policy 1, policy_version 1149945 (0.0008) [2023-12-27 13:37:29,222][20381] Updated weights for policy 1, policy_version 1149955 (0.0010) [2023-12-27 13:37:29,278][20381] Updated weights for policy 1, policy_version 1149965 (0.0008) [2023-12-27 13:37:29,340][20381] Updated weights for policy 1, policy_version 1149975 (0.0011) [2023-12-27 13:37:29,483][20347] Updated weights for policy 0, policy_version 1151160 (0.0007) [2023-12-27 13:37:29,542][20347] Updated weights for policy 0, policy_version 1151170 (0.0011) [2023-12-27 13:37:29,601][20347] Updated weights for policy 0, policy_version 1151180 (0.0010) [2023-12-27 13:37:29,988][20381] Updated weights for policy 1, policy_version 1149985 (0.0009) [2023-12-27 13:37:30,044][20381] Updated weights for policy 1, policy_version 1149995 (0.0008) [2023-12-27 13:37:30,099][20381] Updated weights for policy 1, policy_version 1150005 (0.0008) [2023-12-27 13:37:30,267][20347] Updated weights for policy 0, policy_version 1151190 (0.0006) [2023-12-27 13:37:30,331][20347] Updated weights for policy 0, policy_version 1151200 (0.0005) [2023-12-27 13:37:30,388][20347] Updated weights for policy 0, policy_version 1151210 (0.0005) [2023-12-27 13:37:30,741][20381] Updated weights for policy 1, policy_version 1150015 (0.0006) [2023-12-27 13:37:30,794][20381] Updated weights for policy 1, policy_version 1150025 (0.0005) [2023-12-27 13:37:30,849][20381] Updated weights for policy 1, policy_version 1150035 (0.0005) [2023-12-27 13:37:30,955][20347] Updated weights for policy 0, policy_version 1151220 (0.0007) [2023-12-27 13:37:31,011][20347] Updated weights for policy 0, policy_version 1151230 (0.0005) [2023-12-27 13:37:31,076][20347] Updated weights for policy 0, policy_version 1151240 (0.0010) [2023-12-27 13:37:31,433][20381] Updated weights for policy 1, policy_version 1150045 (0.0008) [2023-12-27 13:37:31,492][20381] Updated weights for policy 1, policy_version 1150055 (0.0010) [2023-12-27 13:37:31,548][20381] Updated weights for policy 1, policy_version 1150065 (0.0010) [2023-12-27 13:37:31,758][20347] Updated weights for policy 0, policy_version 1151250 (0.0009) [2023-12-27 13:37:31,818][20347] Updated weights for policy 0, policy_version 1151260 (0.0005) [2023-12-27 13:37:31,876][20347] Updated weights for policy 0, policy_version 1151270 (0.0006) [2023-12-27 13:37:31,933][20347] Updated weights for policy 0, policy_version 1151280 (0.0006) [2023-12-27 13:37:32,272][20381] Updated weights for policy 1, policy_version 1150075 (0.0010) [2023-12-27 13:37:32,335][20381] Updated weights for policy 1, policy_version 1150085 (0.0008) [2023-12-27 13:37:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 589225984. Throughput: 0: 10523.0, 1: 10094.2. Samples: 589241648. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 13:37:32,363][19170] Avg episode reward: [(0, '941.360'), (1, '725.940')] [2023-12-27 13:37:32,401][20381] Updated weights for policy 1, policy_version 1150095 (0.0009) [2023-12-27 13:37:32,554][20347] Updated weights for policy 0, policy_version 1151290 (0.0006) [2023-12-27 13:37:32,613][20347] Updated weights for policy 0, policy_version 1151300 (0.0006) [2023-12-27 13:37:32,668][20347] Updated weights for policy 0, policy_version 1151310 (0.0006) [2023-12-27 13:37:33,040][20381] Updated weights for policy 1, policy_version 1150105 (0.0007) [2023-12-27 13:37:33,090][20381] Updated weights for policy 1, policy_version 1150115 (0.0005) [2023-12-27 13:37:33,143][20381] Updated weights for policy 1, policy_version 1150125 (0.0005) [2023-12-27 13:37:33,186][20381] Updated weights for policy 1, policy_version 1150135 (0.0005) [2023-12-27 13:37:33,211][20347] Updated weights for policy 0, policy_version 1151320 (0.0006) [2023-12-27 13:37:33,267][20347] Updated weights for policy 0, policy_version 1151330 (0.0006) [2023-12-27 13:37:33,315][20347] Updated weights for policy 0, policy_version 1151340 (0.0006) [2023-12-27 13:37:33,807][20381] Updated weights for policy 1, policy_version 1150145 (0.0007) [2023-12-27 13:37:33,850][20381] Updated weights for policy 1, policy_version 1150155 (0.0007) [2023-12-27 13:37:33,903][20381] Updated weights for policy 1, policy_version 1150165 (0.0008) [2023-12-27 13:37:33,990][20347] Updated weights for policy 0, policy_version 1151350 (0.0010) [2023-12-27 13:37:34,043][20347] Updated weights for policy 0, policy_version 1151360 (0.0010) [2023-12-27 13:37:34,104][20347] Updated weights for policy 0, policy_version 1151370 (0.0010) [2023-12-27 13:37:34,595][20381] Updated weights for policy 1, policy_version 1150175 (0.0010) [2023-12-27 13:37:34,648][20381] Updated weights for policy 1, policy_version 1150185 (0.0011) [2023-12-27 13:37:34,707][20381] Updated weights for policy 1, policy_version 1150195 (0.0011) [2023-12-27 13:37:34,782][20347] Updated weights for policy 0, policy_version 1151380 (0.0010) [2023-12-27 13:37:34,839][20347] Updated weights for policy 0, policy_version 1151390 (0.0010) [2023-12-27 13:37:34,885][20347] Updated weights for policy 0, policy_version 1151400 (0.0010) [2023-12-27 13:37:35,336][20381] Updated weights for policy 1, policy_version 1150205 (0.0008) [2023-12-27 13:37:35,383][20381] Updated weights for policy 1, policy_version 1150215 (0.0010) [2023-12-27 13:37:35,434][20381] Updated weights for policy 1, policy_version 1150225 (0.0010) [2023-12-27 13:37:35,566][20347] Updated weights for policy 0, policy_version 1151410 (0.0010) [2023-12-27 13:37:35,624][20347] Updated weights for policy 0, policy_version 1151420 (0.0008) [2023-12-27 13:37:35,669][20347] Updated weights for policy 0, policy_version 1151430 (0.0006) [2023-12-27 13:37:35,719][20347] Updated weights for policy 0, policy_version 1151440 (0.0010) [2023-12-27 13:37:36,072][20381] Updated weights for policy 1, policy_version 1150235 (0.0009) [2023-12-27 13:37:36,129][20381] Updated weights for policy 1, policy_version 1150245 (0.0008) [2023-12-27 13:37:36,190][20381] Updated weights for policy 1, policy_version 1150255 (0.0007) [2023-12-27 13:37:36,409][20347] Updated weights for policy 0, policy_version 1151450 (0.0006) [2023-12-27 13:37:36,470][20347] Updated weights for policy 0, policy_version 1151460 (0.0006) [2023-12-27 13:37:36,536][20347] Updated weights for policy 0, policy_version 1151470 (0.0006) [2023-12-27 13:37:36,938][20381] Updated weights for policy 1, policy_version 1150265 (0.0008) [2023-12-27 13:37:36,995][20381] Updated weights for policy 1, policy_version 1150275 (0.0008) [2023-12-27 13:37:37,057][20381] Updated weights for policy 1, policy_version 1150285 (0.0009) [2023-12-27 13:37:37,068][20347] Updated weights for policy 0, policy_version 1151480 (0.0006) [2023-12-27 13:37:37,111][20381] Updated weights for policy 1, policy_version 1150295 (0.0006) [2023-12-27 13:37:37,124][20347] Updated weights for policy 0, policy_version 1151490 (0.0006) [2023-12-27 13:37:37,180][20347] Updated weights for policy 0, policy_version 1151500 (0.0008) [2023-12-27 13:37:37,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 589340672. Throughput: 0: 10546.1, 1: 10130.7. Samples: 589306048. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:37:37,363][19170] Avg episode reward: [(0, '990.190'), (1, '667.000')] [2023-12-27 13:37:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001150296_294502400.pth... [2023-12-27 13:37:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001151504_294838272.pth... [2023-12-27 13:37:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001149112_294199296.pth [2023-12-27 13:37:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001150256_294518784.pth [2023-12-27 13:37:37,765][20347] Updated weights for policy 0, policy_version 1151510 (0.0006) [2023-12-27 13:37:37,820][20347] Updated weights for policy 0, policy_version 1151520 (0.0007) [2023-12-27 13:37:37,853][20381] Updated weights for policy 1, policy_version 1150305 (0.0005) [2023-12-27 13:37:37,866][20347] Updated weights for policy 0, policy_version 1151530 (0.0005) [2023-12-27 13:37:37,898][20381] Updated weights for policy 1, policy_version 1150315 (0.0006) [2023-12-27 13:37:37,950][20381] Updated weights for policy 1, policy_version 1150325 (0.0008) [2023-12-27 13:37:38,522][20347] Updated weights for policy 0, policy_version 1151540 (0.0006) [2023-12-27 13:37:38,581][20347] Updated weights for policy 0, policy_version 1151550 (0.0008) [2023-12-27 13:37:38,644][20347] Updated weights for policy 0, policy_version 1151560 (0.0007) [2023-12-27 13:37:38,676][20381] Updated weights for policy 1, policy_version 1150335 (0.0006) [2023-12-27 13:37:38,736][20381] Updated weights for policy 1, policy_version 1150345 (0.0005) [2023-12-27 13:37:38,789][20381] Updated weights for policy 1, policy_version 1150355 (0.0007) [2023-12-27 13:37:39,328][20347] Updated weights for policy 0, policy_version 1151570 (0.0007) [2023-12-27 13:37:39,403][20347] Updated weights for policy 0, policy_version 1151580 (0.0007) [2023-12-27 13:37:39,467][20347] Updated weights for policy 0, policy_version 1151590 (0.0009) [2023-12-27 13:37:39,472][20381] Updated weights for policy 1, policy_version 1150365 (0.0009) [2023-12-27 13:37:39,520][20347] Updated weights for policy 0, policy_version 1151600 (0.0009) [2023-12-27 13:37:39,525][20381] Updated weights for policy 1, policy_version 1150375 (0.0009) [2023-12-27 13:37:39,578][20381] Updated weights for policy 1, policy_version 1150385 (0.0009) [2023-12-27 13:37:40,297][20347] Updated weights for policy 0, policy_version 1151610 (0.0008) [2023-12-27 13:37:40,360][20347] Updated weights for policy 0, policy_version 1151620 (0.0007) [2023-12-27 13:37:40,409][20381] Updated weights for policy 1, policy_version 1150395 (0.0008) [2023-12-27 13:37:40,419][20347] Updated weights for policy 0, policy_version 1151630 (0.0006) [2023-12-27 13:37:40,465][20381] Updated weights for policy 1, policy_version 1150405 (0.0007) [2023-12-27 13:37:40,522][20381] Updated weights for policy 1, policy_version 1150415 (0.0008) [2023-12-27 13:37:41,072][20347] Updated weights for policy 0, policy_version 1151640 (0.0008) [2023-12-27 13:37:41,131][20347] Updated weights for policy 0, policy_version 1151650 (0.0006) [2023-12-27 13:37:41,198][20347] Updated weights for policy 0, policy_version 1151660 (0.0008) [2023-12-27 13:37:41,221][20381] Updated weights for policy 1, policy_version 1150425 (0.0008) [2023-12-27 13:37:41,287][20381] Updated weights for policy 1, policy_version 1150435 (0.0006) [2023-12-27 13:37:41,346][20381] Updated weights for policy 1, policy_version 1150445 (0.0008) [2023-12-27 13:37:41,413][20381] Updated weights for policy 1, policy_version 1150455 (0.0008) [2023-12-27 13:37:41,916][20347] Updated weights for policy 0, policy_version 1151670 (0.0007) [2023-12-27 13:37:41,982][20347] Updated weights for policy 0, policy_version 1151680 (0.0008) [2023-12-27 13:37:42,039][20381] Updated weights for policy 1, policy_version 1150465 (0.0008) [2023-12-27 13:37:42,050][20347] Updated weights for policy 0, policy_version 1151690 (0.0007) [2023-12-27 13:37:42,095][20381] Updated weights for policy 1, policy_version 1150475 (0.0007) [2023-12-27 13:37:42,154][20381] Updated weights for policy 1, policy_version 1150485 (0.0008) [2023-12-27 13:37:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 589438976. Throughput: 0: 10561.5, 1: 10090.2. Samples: 589426980. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:37:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '666.930')] [2023-12-27 13:37:42,795][20347] Updated weights for policy 0, policy_version 1151700 (0.0006) [2023-12-27 13:37:42,803][20381] Updated weights for policy 1, policy_version 1150495 (0.0007) [2023-12-27 13:37:42,853][20347] Updated weights for policy 0, policy_version 1151710 (0.0007) [2023-12-27 13:37:42,855][20381] Updated weights for policy 1, policy_version 1150505 (0.0006) [2023-12-27 13:37:42,911][20381] Updated weights for policy 1, policy_version 1150515 (0.0006) [2023-12-27 13:37:42,912][20347] Updated weights for policy 0, policy_version 1151720 (0.0009) [2023-12-27 13:37:43,606][20347] Updated weights for policy 0, policy_version 1151730 (0.0008) [2023-12-27 13:37:43,610][20381] Updated weights for policy 1, policy_version 1150525 (0.0008) [2023-12-27 13:37:43,661][20347] Updated weights for policy 0, policy_version 1151740 (0.0005) [2023-12-27 13:37:43,674][20381] Updated weights for policy 1, policy_version 1150535 (0.0007) [2023-12-27 13:37:43,719][20347] Updated weights for policy 0, policy_version 1151750 (0.0006) [2023-12-27 13:37:43,733][20381] Updated weights for policy 1, policy_version 1150545 (0.0005) [2023-12-27 13:37:43,776][20347] Updated weights for policy 0, policy_version 1151760 (0.0005) [2023-12-27 13:37:44,329][20381] Updated weights for policy 1, policy_version 1150555 (0.0006) [2023-12-27 13:37:44,384][20381] Updated weights for policy 1, policy_version 1150565 (0.0009) [2023-12-27 13:37:44,413][20347] Updated weights for policy 0, policy_version 1151770 (0.0005) [2023-12-27 13:37:44,441][20381] Updated weights for policy 1, policy_version 1150575 (0.0008) [2023-12-27 13:37:44,472][20347] Updated weights for policy 0, policy_version 1151780 (0.0005) [2023-12-27 13:37:44,528][20347] Updated weights for policy 0, policy_version 1151790 (0.0005) [2023-12-27 13:37:45,146][20347] Updated weights for policy 0, policy_version 1151800 (0.0010) [2023-12-27 13:37:45,160][20381] Updated weights for policy 1, policy_version 1150585 (0.0007) [2023-12-27 13:37:45,203][20347] Updated weights for policy 0, policy_version 1151810 (0.0011) [2023-12-27 13:37:45,216][20381] Updated weights for policy 1, policy_version 1150595 (0.0006) [2023-12-27 13:37:45,260][20347] Updated weights for policy 0, policy_version 1151820 (0.0010) [2023-12-27 13:37:45,277][20381] Updated weights for policy 1, policy_version 1150605 (0.0006) [2023-12-27 13:37:45,341][20381] Updated weights for policy 1, policy_version 1150615 (0.0009) [2023-12-27 13:37:45,936][20347] Updated weights for policy 0, policy_version 1151830 (0.0009) [2023-12-27 13:37:45,967][20381] Updated weights for policy 1, policy_version 1150625 (0.0006) [2023-12-27 13:37:45,982][20347] Updated weights for policy 0, policy_version 1151840 (0.0010) [2023-12-27 13:37:46,019][20381] Updated weights for policy 1, policy_version 1150635 (0.0005) [2023-12-27 13:37:46,034][20347] Updated weights for policy 0, policy_version 1151850 (0.0010) [2023-12-27 13:37:46,077][20381] Updated weights for policy 1, policy_version 1150645 (0.0005) [2023-12-27 13:37:46,667][20347] Updated weights for policy 0, policy_version 1151860 (0.0010) [2023-12-27 13:37:46,703][20381] Updated weights for policy 1, policy_version 1150655 (0.0006) [2023-12-27 13:37:46,719][20347] Updated weights for policy 0, policy_version 1151870 (0.0010) [2023-12-27 13:37:46,752][20381] Updated weights for policy 1, policy_version 1150665 (0.0010) [2023-12-27 13:37:46,770][20347] Updated weights for policy 0, policy_version 1151880 (0.0010) [2023-12-27 13:37:46,813][20381] Updated weights for policy 1, policy_version 1150675 (0.0007) [2023-12-27 13:37:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 589537280. Throughput: 0: 10540.8, 1: 10181.7. Samples: 589551944. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:37:47,363][19170] Avg episode reward: [(0, '990.190'), (1, '716.220')] [2023-12-27 13:37:47,428][20347] Updated weights for policy 0, policy_version 1151890 (0.0011) [2023-12-27 13:37:47,440][20381] Updated weights for policy 1, policy_version 1150685 (0.0008) [2023-12-27 13:37:47,483][20347] Updated weights for policy 0, policy_version 1151900 (0.0010) [2023-12-27 13:37:47,493][20381] Updated weights for policy 1, policy_version 1150695 (0.0010) [2023-12-27 13:37:47,534][20347] Updated weights for policy 0, policy_version 1151910 (0.0010) [2023-12-27 13:37:47,548][20381] Updated weights for policy 1, policy_version 1150705 (0.0010) [2023-12-27 13:37:47,590][20347] Updated weights for policy 0, policy_version 1151920 (0.0010) [2023-12-27 13:37:48,166][20347] Updated weights for policy 0, policy_version 1151930 (0.0005) [2023-12-27 13:37:48,215][20347] Updated weights for policy 0, policy_version 1151940 (0.0006) [2023-12-27 13:37:48,270][20347] Updated weights for policy 0, policy_version 1151950 (0.0010) [2023-12-27 13:37:48,286][20381] Updated weights for policy 1, policy_version 1150715 (0.0009) [2023-12-27 13:37:48,350][20381] Updated weights for policy 1, policy_version 1150725 (0.0007) [2023-12-27 13:37:48,414][20381] Updated weights for policy 1, policy_version 1150735 (0.0008) [2023-12-27 13:37:48,913][20347] Updated weights for policy 0, policy_version 1151960 (0.0010) [2023-12-27 13:37:48,972][20347] Updated weights for policy 0, policy_version 1151970 (0.0010) [2023-12-27 13:37:49,018][20347] Updated weights for policy 0, policy_version 1151980 (0.0010) [2023-12-27 13:37:49,115][20381] Updated weights for policy 1, policy_version 1150745 (0.0007) [2023-12-27 13:37:49,164][20381] Updated weights for policy 1, policy_version 1150755 (0.0006) [2023-12-27 13:37:49,225][20381] Updated weights for policy 1, policy_version 1150765 (0.0006) [2023-12-27 13:37:49,283][20381] Updated weights for policy 1, policy_version 1150775 (0.0008) [2023-12-27 13:37:49,762][20347] Updated weights for policy 0, policy_version 1151990 (0.0008) [2023-12-27 13:37:49,829][20347] Updated weights for policy 0, policy_version 1152000 (0.0009) [2023-12-27 13:37:49,893][20347] Updated weights for policy 0, policy_version 1152010 (0.0008) [2023-12-27 13:37:49,976][20381] Updated weights for policy 1, policy_version 1150785 (0.0006) [2023-12-27 13:37:50,039][20381] Updated weights for policy 1, policy_version 1150795 (0.0007) [2023-12-27 13:37:50,101][20381] Updated weights for policy 1, policy_version 1150805 (0.0007) [2023-12-27 13:37:50,640][20347] Updated weights for policy 0, policy_version 1152020 (0.0008) [2023-12-27 13:37:50,698][20347] Updated weights for policy 0, policy_version 1152030 (0.0008) [2023-12-27 13:37:50,702][20381] Updated weights for policy 1, policy_version 1150815 (0.0009) [2023-12-27 13:37:50,756][20381] Updated weights for policy 1, policy_version 1150825 (0.0006) [2023-12-27 13:37:50,758][20347] Updated weights for policy 0, policy_version 1152040 (0.0006) [2023-12-27 13:37:50,811][20381] Updated weights for policy 1, policy_version 1150835 (0.0006) [2023-12-27 13:37:51,473][20347] Updated weights for policy 0, policy_version 1152050 (0.0006) [2023-12-27 13:37:51,526][20381] Updated weights for policy 1, policy_version 1150845 (0.0007) [2023-12-27 13:37:51,528][20347] Updated weights for policy 0, policy_version 1152060 (0.0007) [2023-12-27 13:37:51,580][20381] Updated weights for policy 1, policy_version 1150855 (0.0006) [2023-12-27 13:37:51,585][20347] Updated weights for policy 0, policy_version 1152070 (0.0008) [2023-12-27 13:37:51,645][20347] Updated weights for policy 0, policy_version 1152080 (0.0008) [2023-12-27 13:37:51,653][20381] Updated weights for policy 1, policy_version 1150865 (0.0007) [2023-12-27 13:37:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 589635584. Throughput: 0: 10501.8, 1: 10130.6. Samples: 589612808. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:37:52,363][19170] Avg episode reward: [(0, '990.270'), (1, '707.390')] [2023-12-27 13:37:52,368][20381] Updated weights for policy 1, policy_version 1150875 (0.0008) [2023-12-27 13:37:52,371][20347] Updated weights for policy 0, policy_version 1152090 (0.0008) [2023-12-27 13:37:52,430][20381] Updated weights for policy 1, policy_version 1150885 (0.0007) [2023-12-27 13:37:52,434][20347] Updated weights for policy 0, policy_version 1152100 (0.0006) [2023-12-27 13:37:52,487][20381] Updated weights for policy 1, policy_version 1150895 (0.0007) [2023-12-27 13:37:52,493][20347] Updated weights for policy 0, policy_version 1152110 (0.0005) [2023-12-27 13:37:52,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001152112_294993920.pth... [2023-12-27 13:37:52,508][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001150864_294674432.pth [2023-12-27 13:37:52,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001150904_294658048.pth... [2023-12-27 13:37:52,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001149688_294346752.pth [2023-12-27 13:37:53,127][20347] Updated weights for policy 0, policy_version 1152120 (0.0008) [2023-12-27 13:37:53,180][20347] Updated weights for policy 0, policy_version 1152130 (0.0008) [2023-12-27 13:37:53,181][20381] Updated weights for policy 1, policy_version 1150905 (0.0007) [2023-12-27 13:37:53,225][20347] Updated weights for policy 0, policy_version 1152140 (0.0005) [2023-12-27 13:37:53,231][20381] Updated weights for policy 1, policy_version 1150915 (0.0005) [2023-12-27 13:37:53,281][20381] Updated weights for policy 1, policy_version 1150925 (0.0007) [2023-12-27 13:37:53,329][20381] Updated weights for policy 1, policy_version 1150935 (0.0007) [2023-12-27 13:37:53,925][20347] Updated weights for policy 0, policy_version 1152150 (0.0008) [2023-12-27 13:37:53,957][20381] Updated weights for policy 1, policy_version 1150945 (0.0006) [2023-12-27 13:37:53,977][20347] Updated weights for policy 0, policy_version 1152160 (0.0010) [2023-12-27 13:37:54,019][20381] Updated weights for policy 1, policy_version 1150955 (0.0007) [2023-12-27 13:37:54,044][20347] Updated weights for policy 0, policy_version 1152170 (0.0009) [2023-12-27 13:37:54,078][20381] Updated weights for policy 1, policy_version 1150965 (0.0006) [2023-12-27 13:37:54,734][20347] Updated weights for policy 0, policy_version 1152180 (0.0007) [2023-12-27 13:37:54,789][20347] Updated weights for policy 0, policy_version 1152190 (0.0005) [2023-12-27 13:37:54,812][20381] Updated weights for policy 1, policy_version 1150975 (0.0008) [2023-12-27 13:37:54,838][20347] Updated weights for policy 0, policy_version 1152200 (0.0005) [2023-12-27 13:37:54,865][20381] Updated weights for policy 1, policy_version 1150985 (0.0007) [2023-12-27 13:37:54,921][20381] Updated weights for policy 1, policy_version 1150995 (0.0006) [2023-12-27 13:37:55,436][20347] Updated weights for policy 0, policy_version 1152210 (0.0006) [2023-12-27 13:37:55,490][20347] Updated weights for policy 0, policy_version 1152220 (0.0007) [2023-12-27 13:37:55,553][20347] Updated weights for policy 0, policy_version 1152230 (0.0008) [2023-12-27 13:37:55,604][20347] Updated weights for policy 0, policy_version 1152240 (0.0007) [2023-12-27 13:37:55,610][20381] Updated weights for policy 1, policy_version 1151005 (0.0005) [2023-12-27 13:37:55,661][20381] Updated weights for policy 1, policy_version 1151015 (0.0008) [2023-12-27 13:37:55,717][20381] Updated weights for policy 1, policy_version 1151025 (0.0008) [2023-12-27 13:37:56,257][20347] Updated weights for policy 0, policy_version 1152250 (0.0009) [2023-12-27 13:37:56,310][20347] Updated weights for policy 0, policy_version 1152260 (0.0008) [2023-12-27 13:37:56,356][20347] Updated weights for policy 0, policy_version 1152270 (0.0008) [2023-12-27 13:37:56,381][20381] Updated weights for policy 1, policy_version 1151035 (0.0008) [2023-12-27 13:37:56,436][20381] Updated weights for policy 1, policy_version 1151045 (0.0008) [2023-12-27 13:37:56,487][20381] Updated weights for policy 1, policy_version 1151055 (0.0007) [2023-12-27 13:37:57,019][20347] Updated weights for policy 0, policy_version 1152280 (0.0006) [2023-12-27 13:37:57,069][20347] Updated weights for policy 0, policy_version 1152290 (0.0008) [2023-12-27 13:37:57,124][20347] Updated weights for policy 0, policy_version 1152300 (0.0007) [2023-12-27 13:37:57,185][20381] Updated weights for policy 1, policy_version 1151065 (0.0009) [2023-12-27 13:37:57,228][20381] Updated weights for policy 1, policy_version 1151075 (0.0007) [2023-12-27 13:37:57,272][20381] Updated weights for policy 1, policy_version 1151085 (0.0008) [2023-12-27 13:37:57,324][20381] Updated weights for policy 1, policy_version 1151095 (0.0007) [2023-12-27 13:37:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 589750272. Throughput: 0: 10428.1, 1: 10183.1. Samples: 589735080. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:37:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '717.130')] [2023-12-27 13:37:57,703][20347] Updated weights for policy 0, policy_version 1152310 (0.0007) [2023-12-27 13:37:57,752][20347] Updated weights for policy 0, policy_version 1152320 (0.0005) [2023-12-27 13:37:57,801][20347] Updated weights for policy 0, policy_version 1152330 (0.0005) [2023-12-27 13:37:58,085][20381] Updated weights for policy 1, policy_version 1151105 (0.0007) [2023-12-27 13:37:58,139][20381] Updated weights for policy 1, policy_version 1151115 (0.0007) [2023-12-27 13:37:58,194][20381] Updated weights for policy 1, policy_version 1151125 (0.0009) [2023-12-27 13:37:58,396][20347] Updated weights for policy 0, policy_version 1152340 (0.0006) [2023-12-27 13:37:58,456][20347] Updated weights for policy 0, policy_version 1152350 (0.0008) [2023-12-27 13:37:58,516][20347] Updated weights for policy 0, policy_version 1152360 (0.0008) [2023-12-27 13:37:59,031][20381] Updated weights for policy 1, policy_version 1151135 (0.0007) [2023-12-27 13:37:59,081][20381] Updated weights for policy 1, policy_version 1151145 (0.0007) [2023-12-27 13:37:59,147][20381] Updated weights for policy 1, policy_version 1151155 (0.0008) [2023-12-27 13:37:59,330][20347] Updated weights for policy 0, policy_version 1152370 (0.0008) [2023-12-27 13:37:59,399][20347] Updated weights for policy 0, policy_version 1152380 (0.0008) [2023-12-27 13:37:59,463][20347] Updated weights for policy 0, policy_version 1152390 (0.0008) [2023-12-27 13:37:59,520][20347] Updated weights for policy 0, policy_version 1152400 (0.0007) [2023-12-27 13:37:59,832][20381] Updated weights for policy 1, policy_version 1151165 (0.0008) [2023-12-27 13:37:59,894][20381] Updated weights for policy 1, policy_version 1151175 (0.0007) [2023-12-27 13:37:59,961][20381] Updated weights for policy 1, policy_version 1151185 (0.0007) [2023-12-27 13:38:00,257][20347] Updated weights for policy 0, policy_version 1152410 (0.0008) [2023-12-27 13:38:00,314][20347] Updated weights for policy 0, policy_version 1152420 (0.0008) [2023-12-27 13:38:00,372][20347] Updated weights for policy 0, policy_version 1152430 (0.0008) [2023-12-27 13:38:00,620][20381] Updated weights for policy 1, policy_version 1151195 (0.0007) [2023-12-27 13:38:00,665][20381] Updated weights for policy 1, policy_version 1151205 (0.0007) [2023-12-27 13:38:00,714][20381] Updated weights for policy 1, policy_version 1151215 (0.0008) [2023-12-27 13:38:01,062][20347] Updated weights for policy 0, policy_version 1152440 (0.0008) [2023-12-27 13:38:01,145][20347] Updated weights for policy 0, policy_version 1152450 (0.0007) [2023-12-27 13:38:01,195][20347] Updated weights for policy 0, policy_version 1152460 (0.0007) [2023-12-27 13:38:01,399][20381] Updated weights for policy 1, policy_version 1151225 (0.0007) [2023-12-27 13:38:01,452][20381] Updated weights for policy 1, policy_version 1151235 (0.0006) [2023-12-27 13:38:01,503][20381] Updated weights for policy 1, policy_version 1151245 (0.0006) [2023-12-27 13:38:01,557][20381] Updated weights for policy 1, policy_version 1151255 (0.0008) [2023-12-27 13:38:01,914][20347] Updated weights for policy 0, policy_version 1152470 (0.0008) [2023-12-27 13:38:01,966][20347] Updated weights for policy 0, policy_version 1152480 (0.0006) [2023-12-27 13:38:02,027][20347] Updated weights for policy 0, policy_version 1152490 (0.0010) [2023-12-27 13:38:02,266][20381] Updated weights for policy 1, policy_version 1151265 (0.0007) [2023-12-27 13:38:02,326][20381] Updated weights for policy 1, policy_version 1151275 (0.0007) [2023-12-27 13:38:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 589840384. Throughput: 0: 10422.9, 1: 10129.5. Samples: 589854212. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:02,363][19170] Avg episode reward: [(0, '980.490'), (1, '725.800')] [2023-12-27 13:38:02,391][20381] Updated weights for policy 1, policy_version 1151285 (0.0008) [2023-12-27 13:38:02,680][20347] Updated weights for policy 0, policy_version 1152500 (0.0010) [2023-12-27 13:38:02,734][20347] Updated weights for policy 0, policy_version 1152510 (0.0010) [2023-12-27 13:38:02,795][20347] Updated weights for policy 0, policy_version 1152520 (0.0009) [2023-12-27 13:38:03,089][20381] Updated weights for policy 1, policy_version 1151295 (0.0009) [2023-12-27 13:38:03,146][20381] Updated weights for policy 1, policy_version 1151305 (0.0010) [2023-12-27 13:38:03,202][20381] Updated weights for policy 1, policy_version 1151315 (0.0011) [2023-12-27 13:38:03,438][20347] Updated weights for policy 0, policy_version 1152530 (0.0011) [2023-12-27 13:38:03,496][20347] Updated weights for policy 0, policy_version 1152540 (0.0009) [2023-12-27 13:38:03,554][20347] Updated weights for policy 0, policy_version 1152550 (0.0005) [2023-12-27 13:38:03,613][20347] Updated weights for policy 0, policy_version 1152560 (0.0007) [2023-12-27 13:38:03,884][20381] Updated weights for policy 1, policy_version 1151325 (0.0009) [2023-12-27 13:38:03,942][20381] Updated weights for policy 1, policy_version 1151335 (0.0008) [2023-12-27 13:38:04,004][20381] Updated weights for policy 1, policy_version 1151345 (0.0008) [2023-12-27 13:38:04,234][20347] Updated weights for policy 0, policy_version 1152570 (0.0008) [2023-12-27 13:38:04,288][20347] Updated weights for policy 0, policy_version 1152580 (0.0005) [2023-12-27 13:38:04,349][20347] Updated weights for policy 0, policy_version 1152590 (0.0006) [2023-12-27 13:38:04,724][20381] Updated weights for policy 1, policy_version 1151355 (0.0007) [2023-12-27 13:38:04,776][20381] Updated weights for policy 1, policy_version 1151365 (0.0005) [2023-12-27 13:38:04,836][20381] Updated weights for policy 1, policy_version 1151375 (0.0008) [2023-12-27 13:38:04,942][20347] Updated weights for policy 0, policy_version 1152600 (0.0007) [2023-12-27 13:38:04,994][20347] Updated weights for policy 0, policy_version 1152610 (0.0005) [2023-12-27 13:38:05,059][20347] Updated weights for policy 0, policy_version 1152620 (0.0005) [2023-12-27 13:38:05,401][20381] Updated weights for policy 1, policy_version 1151385 (0.0010) [2023-12-27 13:38:05,462][20381] Updated weights for policy 1, policy_version 1151395 (0.0009) [2023-12-27 13:38:05,524][20381] Updated weights for policy 1, policy_version 1151405 (0.0010) [2023-12-27 13:38:05,585][20381] Updated weights for policy 1, policy_version 1151415 (0.0011) [2023-12-27 13:38:05,623][20347] Updated weights for policy 0, policy_version 1152630 (0.0008) [2023-12-27 13:38:05,675][20347] Updated weights for policy 0, policy_version 1152640 (0.0010) [2023-12-27 13:38:05,727][20347] Updated weights for policy 0, policy_version 1152650 (0.0010) [2023-12-27 13:38:06,186][20381] Updated weights for policy 1, policy_version 1151425 (0.0010) [2023-12-27 13:38:06,243][20381] Updated weights for policy 1, policy_version 1151435 (0.0011) [2023-12-27 13:38:06,295][20381] Updated weights for policy 1, policy_version 1151445 (0.0010) [2023-12-27 13:38:06,445][20347] Updated weights for policy 0, policy_version 1152660 (0.0008) [2023-12-27 13:38:06,507][20347] Updated weights for policy 0, policy_version 1152670 (0.0008) [2023-12-27 13:38:06,572][20347] Updated weights for policy 0, policy_version 1152680 (0.0007) [2023-12-27 13:38:06,958][20381] Updated weights for policy 1, policy_version 1151455 (0.0009) [2023-12-27 13:38:07,008][20381] Updated weights for policy 1, policy_version 1151465 (0.0011) [2023-12-27 13:38:07,066][20381] Updated weights for policy 1, policy_version 1151475 (0.0011) [2023-12-27 13:38:07,110][20347] Updated weights for policy 0, policy_version 1152690 (0.0005) [2023-12-27 13:38:07,165][20347] Updated weights for policy 0, policy_version 1152700 (0.0009) [2023-12-27 13:38:07,219][20347] Updated weights for policy 0, policy_version 1152710 (0.0010) [2023-12-27 13:38:07,280][20347] Updated weights for policy 0, policy_version 1152720 (0.0010) [2023-12-27 13:38:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 589955072. Throughput: 0: 10467.0, 1: 10152.4. Samples: 589917236. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:07,363][19170] Avg episode reward: [(0, '980.450'), (1, '716.020')] [2023-12-27 13:38:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001151480_294805504.pth... [2023-12-27 13:38:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001152720_295149568.pth... [2023-12-27 13:38:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001150296_294502400.pth [2023-12-27 13:38:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001151504_294838272.pth [2023-12-27 13:38:07,691][20381] Updated weights for policy 1, policy_version 1151485 (0.0008) [2023-12-27 13:38:07,743][20381] Updated weights for policy 1, policy_version 1151495 (0.0005) [2023-12-27 13:38:07,799][20381] Updated weights for policy 1, policy_version 1151505 (0.0007) [2023-12-27 13:38:07,950][20347] Updated weights for policy 0, policy_version 1152730 (0.0010) [2023-12-27 13:38:07,995][20347] Updated weights for policy 0, policy_version 1152740 (0.0010) [2023-12-27 13:38:08,038][20347] Updated weights for policy 0, policy_version 1152750 (0.0010) [2023-12-27 13:38:08,373][20381] Updated weights for policy 1, policy_version 1151515 (0.0007) [2023-12-27 13:38:08,438][20381] Updated weights for policy 1, policy_version 1151525 (0.0008) [2023-12-27 13:38:08,495][20381] Updated weights for policy 1, policy_version 1151535 (0.0008) [2023-12-27 13:38:08,740][20347] Updated weights for policy 0, policy_version 1152760 (0.0010) [2023-12-27 13:38:08,789][20347] Updated weights for policy 0, policy_version 1152770 (0.0010) [2023-12-27 13:38:08,851][20347] Updated weights for policy 0, policy_version 1152780 (0.0009) [2023-12-27 13:38:09,199][20381] Updated weights for policy 1, policy_version 1151545 (0.0007) [2023-12-27 13:38:09,262][20381] Updated weights for policy 1, policy_version 1151555 (0.0007) [2023-12-27 13:38:09,318][20381] Updated weights for policy 1, policy_version 1151565 (0.0008) [2023-12-27 13:38:09,391][20381] Updated weights for policy 1, policy_version 1151575 (0.0007) [2023-12-27 13:38:09,569][20347] Updated weights for policy 0, policy_version 1152790 (0.0006) [2023-12-27 13:38:09,624][20347] Updated weights for policy 0, policy_version 1152800 (0.0006) [2023-12-27 13:38:09,680][20347] Updated weights for policy 0, policy_version 1152810 (0.0005) [2023-12-27 13:38:10,142][20381] Updated weights for policy 1, policy_version 1151585 (0.0010) [2023-12-27 13:38:10,204][20381] Updated weights for policy 1, policy_version 1151595 (0.0011) [2023-12-27 13:38:10,264][20381] Updated weights for policy 1, policy_version 1151605 (0.0009) [2023-12-27 13:38:10,338][20347] Updated weights for policy 0, policy_version 1152820 (0.0005) [2023-12-27 13:38:10,400][20347] Updated weights for policy 0, policy_version 1152830 (0.0007) [2023-12-27 13:38:10,455][20347] Updated weights for policy 0, policy_version 1152840 (0.0007) [2023-12-27 13:38:10,895][20381] Updated weights for policy 1, policy_version 1151615 (0.0007) [2023-12-27 13:38:10,945][20381] Updated weights for policy 1, policy_version 1151625 (0.0008) [2023-12-27 13:38:10,998][20381] Updated weights for policy 1, policy_version 1151635 (0.0008) [2023-12-27 13:38:11,064][20347] Updated weights for policy 0, policy_version 1152850 (0.0008) [2023-12-27 13:38:11,124][20347] Updated weights for policy 0, policy_version 1152860 (0.0008) [2023-12-27 13:38:11,194][20347] Updated weights for policy 0, policy_version 1152870 (0.0006) [2023-12-27 13:38:11,258][20347] Updated weights for policy 0, policy_version 1152880 (0.0007) [2023-12-27 13:38:11,813][20381] Updated weights for policy 1, policy_version 1151645 (0.0010) [2023-12-27 13:38:11,870][20381] Updated weights for policy 1, policy_version 1151655 (0.0011) [2023-12-27 13:38:11,924][20381] Updated weights for policy 1, policy_version 1151665 (0.0011) [2023-12-27 13:38:11,993][20347] Updated weights for policy 0, policy_version 1152890 (0.0011) [2023-12-27 13:38:12,048][20347] Updated weights for policy 0, policy_version 1152900 (0.0010) [2023-12-27 13:38:12,101][20347] Updated weights for policy 0, policy_version 1152910 (0.0010) [2023-12-27 13:38:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 590053376. Throughput: 0: 10493.4, 1: 10219.0. Samples: 590043324. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:12,363][19170] Avg episode reward: [(0, '960.840'), (1, '531.110')] [2023-12-27 13:38:12,610][20381] Updated weights for policy 1, policy_version 1151675 (0.0010) [2023-12-27 13:38:12,662][20381] Updated weights for policy 1, policy_version 1151685 (0.0007) [2023-12-27 13:38:12,723][20381] Updated weights for policy 1, policy_version 1151695 (0.0008) [2023-12-27 13:38:12,824][20347] Updated weights for policy 0, policy_version 1152920 (0.0008) [2023-12-27 13:38:12,879][20347] Updated weights for policy 0, policy_version 1152930 (0.0008) [2023-12-27 13:38:12,927][20347] Updated weights for policy 0, policy_version 1152940 (0.0007) [2023-12-27 13:38:13,435][20381] Updated weights for policy 1, policy_version 1151705 (0.0007) [2023-12-27 13:38:13,493][20381] Updated weights for policy 1, policy_version 1151715 (0.0008) [2023-12-27 13:38:13,554][20381] Updated weights for policy 1, policy_version 1151725 (0.0008) [2023-12-27 13:38:13,582][20347] Updated weights for policy 0, policy_version 1152950 (0.0006) [2023-12-27 13:38:13,613][20381] Updated weights for policy 1, policy_version 1151735 (0.0007) [2023-12-27 13:38:13,637][20347] Updated weights for policy 0, policy_version 1152960 (0.0006) [2023-12-27 13:38:13,690][20347] Updated weights for policy 0, policy_version 1152970 (0.0007) [2023-12-27 13:38:14,274][20381] Updated weights for policy 1, policy_version 1151745 (0.0007) [2023-12-27 13:38:14,326][20347] Updated weights for policy 0, policy_version 1152980 (0.0007) [2023-12-27 13:38:14,332][20381] Updated weights for policy 1, policy_version 1151755 (0.0006) [2023-12-27 13:38:14,382][20347] Updated weights for policy 0, policy_version 1152990 (0.0006) [2023-12-27 13:38:14,387][20381] Updated weights for policy 1, policy_version 1151765 (0.0006) [2023-12-27 13:38:14,440][20347] Updated weights for policy 0, policy_version 1153000 (0.0007) [2023-12-27 13:38:15,057][20381] Updated weights for policy 1, policy_version 1151775 (0.0006) [2023-12-27 13:38:15,125][20381] Updated weights for policy 1, policy_version 1151785 (0.0006) [2023-12-27 13:38:15,155][20347] Updated weights for policy 0, policy_version 1153010 (0.0007) [2023-12-27 13:38:15,188][20381] Updated weights for policy 1, policy_version 1151795 (0.0007) [2023-12-27 13:38:15,222][20347] Updated weights for policy 0, policy_version 1153020 (0.0007) [2023-12-27 13:38:15,284][20347] Updated weights for policy 0, policy_version 1153030 (0.0008) [2023-12-27 13:38:15,352][20347] Updated weights for policy 0, policy_version 1153040 (0.0008) [2023-12-27 13:38:15,796][20381] Updated weights for policy 1, policy_version 1151805 (0.0006) [2023-12-27 13:38:15,858][20381] Updated weights for policy 1, policy_version 1151815 (0.0005) [2023-12-27 13:38:15,916][20381] Updated weights for policy 1, policy_version 1151825 (0.0005) [2023-12-27 13:38:15,971][20347] Updated weights for policy 0, policy_version 1153050 (0.0006) [2023-12-27 13:38:16,017][20347] Updated weights for policy 0, policy_version 1153060 (0.0005) [2023-12-27 13:38:16,069][20347] Updated weights for policy 0, policy_version 1153070 (0.0005) [2023-12-27 13:38:16,572][20381] Updated weights for policy 1, policy_version 1151835 (0.0006) [2023-12-27 13:38:16,627][20381] Updated weights for policy 1, policy_version 1151845 (0.0006) [2023-12-27 13:38:16,632][20347] Updated weights for policy 0, policy_version 1153080 (0.0009) [2023-12-27 13:38:16,670][20381] Updated weights for policy 1, policy_version 1151855 (0.0005) [2023-12-27 13:38:16,683][20347] Updated weights for policy 0, policy_version 1153090 (0.0010) [2023-12-27 13:38:16,738][20347] Updated weights for policy 0, policy_version 1153100 (0.0010) [2023-12-27 13:38:17,242][20381] Updated weights for policy 1, policy_version 1151865 (0.0006) [2023-12-27 13:38:17,302][20381] Updated weights for policy 1, policy_version 1151875 (0.0007) [2023-12-27 13:38:17,346][20381] Updated weights for policy 1, policy_version 1151885 (0.0007) [2023-12-27 13:38:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 590151680. Throughput: 0: 10403.6, 1: 10175.3. Samples: 590167704. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:17,364][19170] Avg episode reward: [(0, '970.770'), (1, '598.890')] [2023-12-27 13:38:17,396][20381] Updated weights for policy 1, policy_version 1151895 (0.0008) [2023-12-27 13:38:17,416][20347] Updated weights for policy 0, policy_version 1153110 (0.0010) [2023-12-27 13:38:17,473][20347] Updated weights for policy 0, policy_version 1153120 (0.0010) [2023-12-27 13:38:17,536][20347] Updated weights for policy 0, policy_version 1153130 (0.0010) [2023-12-27 13:38:18,026][20381] Updated weights for policy 1, policy_version 1151905 (0.0007) [2023-12-27 13:38:18,070][20381] Updated weights for policy 1, policy_version 1151915 (0.0008) [2023-12-27 13:38:18,155][20381] Updated weights for policy 1, policy_version 1151925 (0.0007) [2023-12-27 13:38:18,206][20347] Updated weights for policy 0, policy_version 1153140 (0.0009) [2023-12-27 13:38:18,258][20347] Updated weights for policy 0, policy_version 1153150 (0.0008) [2023-12-27 13:38:18,309][20347] Updated weights for policy 0, policy_version 1153160 (0.0007) [2023-12-27 13:38:18,826][20381] Updated weights for policy 1, policy_version 1151935 (0.0007) [2023-12-27 13:38:18,890][20381] Updated weights for policy 1, policy_version 1151945 (0.0007) [2023-12-27 13:38:18,949][20381] Updated weights for policy 1, policy_version 1151955 (0.0008) [2023-12-27 13:38:18,972][20347] Updated weights for policy 0, policy_version 1153170 (0.0009) [2023-12-27 13:38:19,021][20347] Updated weights for policy 0, policy_version 1153180 (0.0009) [2023-12-27 13:38:19,077][20347] Updated weights for policy 0, policy_version 1153190 (0.0009) [2023-12-27 13:38:19,135][20347] Updated weights for policy 0, policy_version 1153200 (0.0010) [2023-12-27 13:38:19,705][20381] Updated weights for policy 1, policy_version 1151965 (0.0010) [2023-12-27 13:38:19,765][20381] Updated weights for policy 1, policy_version 1151975 (0.0011) [2023-12-27 13:38:19,826][20381] Updated weights for policy 1, policy_version 1151985 (0.0011) [2023-12-27 13:38:19,830][20347] Updated weights for policy 0, policy_version 1153210 (0.0010) [2023-12-27 13:38:19,897][20347] Updated weights for policy 0, policy_version 1153220 (0.0009) [2023-12-27 13:38:19,958][20347] Updated weights for policy 0, policy_version 1153230 (0.0008) [2023-12-27 13:38:20,508][20381] Updated weights for policy 1, policy_version 1151995 (0.0009) [2023-12-27 13:38:20,579][20381] Updated weights for policy 1, policy_version 1152005 (0.0010) [2023-12-27 13:38:20,644][20381] Updated weights for policy 1, policy_version 1152015 (0.0009) [2023-12-27 13:38:20,693][20347] Updated weights for policy 0, policy_version 1153240 (0.0007) [2023-12-27 13:38:20,758][20347] Updated weights for policy 0, policy_version 1153250 (0.0008) [2023-12-27 13:38:20,825][20347] Updated weights for policy 0, policy_version 1153260 (0.0007) [2023-12-27 13:38:21,385][20381] Updated weights for policy 1, policy_version 1152025 (0.0008) [2023-12-27 13:38:21,445][20381] Updated weights for policy 1, policy_version 1152035 (0.0009) [2023-12-27 13:38:21,500][20381] Updated weights for policy 1, policy_version 1152045 (0.0007) [2023-12-27 13:38:21,552][20347] Updated weights for policy 0, policy_version 1153270 (0.0010) [2023-12-27 13:38:21,559][20381] Updated weights for policy 1, policy_version 1152055 (0.0007) [2023-12-27 13:38:21,612][20347] Updated weights for policy 0, policy_version 1153280 (0.0010) [2023-12-27 13:38:21,679][20347] Updated weights for policy 0, policy_version 1153290 (0.0009) [2023-12-27 13:38:22,286][20381] Updated weights for policy 1, policy_version 1152065 (0.0008) [2023-12-27 13:38:22,352][20381] Updated weights for policy 1, policy_version 1152075 (0.0008) [2023-12-27 13:38:22,362][20347] Updated weights for policy 0, policy_version 1153300 (0.0008) [2023-12-27 13:38:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 590249984. Throughput: 0: 10360.0, 1: 10134.1. Samples: 590228284. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:22,363][19170] Avg episode reward: [(0, '990.370'), (1, '598.670')] [2023-12-27 13:38:22,418][20347] Updated weights for policy 0, policy_version 1153310 (0.0007) [2023-12-27 13:38:22,421][20381] Updated weights for policy 1, policy_version 1152085 (0.0007) [2023-12-27 13:38:22,440][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001152088_294961152.pth... [2023-12-27 13:38:22,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001150904_294658048.pth [2023-12-27 13:38:22,481][20347] Updated weights for policy 0, policy_version 1153320 (0.0007) [2023-12-27 13:38:22,530][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001153328_295305216.pth... [2023-12-27 13:38:22,534][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001152112_294993920.pth [2023-12-27 13:38:23,076][20381] Updated weights for policy 1, policy_version 1152095 (0.0009) [2023-12-27 13:38:23,089][20347] Updated weights for policy 0, policy_version 1153330 (0.0007) [2023-12-27 13:38:23,135][20381] Updated weights for policy 1, policy_version 1152105 (0.0006) [2023-12-27 13:38:23,150][20347] Updated weights for policy 0, policy_version 1153340 (0.0008) [2023-12-27 13:38:23,190][20381] Updated weights for policy 1, policy_version 1152115 (0.0005) [2023-12-27 13:38:23,208][20347] Updated weights for policy 0, policy_version 1153350 (0.0008) [2023-12-27 13:38:23,255][20347] Updated weights for policy 0, policy_version 1153360 (0.0010) [2023-12-27 13:38:23,832][20381] Updated weights for policy 1, policy_version 1152125 (0.0006) [2023-12-27 13:38:23,890][20381] Updated weights for policy 1, policy_version 1152135 (0.0006) [2023-12-27 13:38:23,897][20347] Updated weights for policy 0, policy_version 1153370 (0.0005) [2023-12-27 13:38:23,945][20347] Updated weights for policy 0, policy_version 1153380 (0.0010) [2023-12-27 13:38:23,950][20381] Updated weights for policy 1, policy_version 1152145 (0.0005) [2023-12-27 13:38:23,997][20347] Updated weights for policy 0, policy_version 1153390 (0.0010) [2023-12-27 13:38:24,568][20381] Updated weights for policy 1, policy_version 1152155 (0.0005) [2023-12-27 13:38:24,629][20381] Updated weights for policy 1, policy_version 1152165 (0.0007) [2023-12-27 13:38:24,676][20347] Updated weights for policy 0, policy_version 1153400 (0.0009) [2023-12-27 13:38:24,690][20381] Updated weights for policy 1, policy_version 1152175 (0.0011) [2023-12-27 13:38:24,737][20347] Updated weights for policy 0, policy_version 1153410 (0.0006) [2023-12-27 13:38:24,793][20347] Updated weights for policy 0, policy_version 1153420 (0.0006) [2023-12-27 13:38:25,332][20381] Updated weights for policy 1, policy_version 1152185 (0.0009) [2023-12-27 13:38:25,389][20381] Updated weights for policy 1, policy_version 1152195 (0.0006) [2023-12-27 13:38:25,429][20347] Updated weights for policy 0, policy_version 1153430 (0.0006) [2023-12-27 13:38:25,446][20381] Updated weights for policy 1, policy_version 1152205 (0.0007) [2023-12-27 13:38:25,479][20347] Updated weights for policy 0, policy_version 1153440 (0.0007) [2023-12-27 13:38:25,505][20381] Updated weights for policy 1, policy_version 1152215 (0.0006) [2023-12-27 13:38:25,533][20347] Updated weights for policy 0, policy_version 1153450 (0.0006) [2023-12-27 13:38:26,134][20347] Updated weights for policy 0, policy_version 1153460 (0.0006) [2023-12-27 13:38:26,175][20381] Updated weights for policy 1, policy_version 1152225 (0.0008) [2023-12-27 13:38:26,179][20347] Updated weights for policy 0, policy_version 1153470 (0.0005) [2023-12-27 13:38:26,224][20381] Updated weights for policy 1, policy_version 1152235 (0.0009) [2023-12-27 13:38:26,229][20347] Updated weights for policy 0, policy_version 1153480 (0.0005) [2023-12-27 13:38:26,281][20381] Updated weights for policy 1, policy_version 1152245 (0.0006) [2023-12-27 13:38:26,870][20347] Updated weights for policy 0, policy_version 1153490 (0.0005) [2023-12-27 13:38:26,915][20347] Updated weights for policy 0, policy_version 1153500 (0.0005) [2023-12-27 13:38:26,964][20381] Updated weights for policy 1, policy_version 1152255 (0.0006) [2023-12-27 13:38:26,966][20347] Updated weights for policy 0, policy_version 1153510 (0.0009) [2023-12-27 13:38:27,016][20347] Updated weights for policy 0, policy_version 1153520 (0.0010) [2023-12-27 13:38:27,018][20381] Updated weights for policy 1, policy_version 1152265 (0.0006) [2023-12-27 13:38:27,070][20381] Updated weights for policy 1, policy_version 1152275 (0.0006) [2023-12-27 13:38:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 590364672. Throughput: 0: 10357.2, 1: 10200.0. Samples: 590352052. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '501.000')] [2023-12-27 13:38:27,593][20347] Updated weights for policy 0, policy_version 1153530 (0.0008) [2023-12-27 13:38:27,640][20381] Updated weights for policy 1, policy_version 1152285 (0.0008) [2023-12-27 13:38:27,643][20347] Updated weights for policy 0, policy_version 1153540 (0.0010) [2023-12-27 13:38:27,685][20381] Updated weights for policy 1, policy_version 1152295 (0.0010) [2023-12-27 13:38:27,691][20347] Updated weights for policy 0, policy_version 1153550 (0.0010) [2023-12-27 13:38:27,736][20381] Updated weights for policy 1, policy_version 1152305 (0.0010) [2023-12-27 13:38:28,379][20381] Updated weights for policy 1, policy_version 1152315 (0.0009) [2023-12-27 13:38:28,381][20347] Updated weights for policy 0, policy_version 1153560 (0.0010) [2023-12-27 13:38:28,433][20347] Updated weights for policy 0, policy_version 1153570 (0.0010) [2023-12-27 13:38:28,442][20381] Updated weights for policy 1, policy_version 1152325 (0.0006) [2023-12-27 13:38:28,485][20347] Updated weights for policy 0, policy_version 1153580 (0.0010) [2023-12-27 13:38:28,495][20381] Updated weights for policy 1, policy_version 1152335 (0.0006) [2023-12-27 13:38:29,071][20381] Updated weights for policy 1, policy_version 1152345 (0.0007) [2023-12-27 13:38:29,126][20381] Updated weights for policy 1, policy_version 1152355 (0.0007) [2023-12-27 13:38:29,166][20347] Updated weights for policy 0, policy_version 1153590 (0.0009) [2023-12-27 13:38:29,179][20381] Updated weights for policy 1, policy_version 1152365 (0.0007) [2023-12-27 13:38:29,220][20347] Updated weights for policy 0, policy_version 1153600 (0.0006) [2023-12-27 13:38:29,235][20381] Updated weights for policy 1, policy_version 1152375 (0.0008) [2023-12-27 13:38:29,278][20347] Updated weights for policy 0, policy_version 1153610 (0.0009) [2023-12-27 13:38:29,918][20347] Updated weights for policy 0, policy_version 1153620 (0.0006) [2023-12-27 13:38:29,929][20381] Updated weights for policy 1, policy_version 1152385 (0.0007) [2023-12-27 13:38:29,985][20347] Updated weights for policy 0, policy_version 1153630 (0.0008) [2023-12-27 13:38:29,995][20381] Updated weights for policy 1, policy_version 1152395 (0.0008) [2023-12-27 13:38:30,039][20347] Updated weights for policy 0, policy_version 1153640 (0.0006) [2023-12-27 13:38:30,051][20381] Updated weights for policy 1, policy_version 1152405 (0.0007) [2023-12-27 13:38:30,718][20347] Updated weights for policy 0, policy_version 1153650 (0.0006) [2023-12-27 13:38:30,728][20381] Updated weights for policy 1, policy_version 1152415 (0.0007) [2023-12-27 13:38:30,777][20381] Updated weights for policy 1, policy_version 1152425 (0.0007) [2023-12-27 13:38:30,783][20347] Updated weights for policy 0, policy_version 1153660 (0.0006) [2023-12-27 13:38:30,823][20381] Updated weights for policy 1, policy_version 1152435 (0.0008) [2023-12-27 13:38:30,839][20347] Updated weights for policy 0, policy_version 1153670 (0.0005) [2023-12-27 13:38:30,894][20347] Updated weights for policy 0, policy_version 1153680 (0.0005) [2023-12-27 13:38:31,534][20381] Updated weights for policy 1, policy_version 1152445 (0.0007) [2023-12-27 13:38:31,543][20347] Updated weights for policy 0, policy_version 1153690 (0.0005) [2023-12-27 13:38:31,593][20381] Updated weights for policy 1, policy_version 1152455 (0.0007) [2023-12-27 13:38:31,595][20347] Updated weights for policy 0, policy_version 1153700 (0.0005) [2023-12-27 13:38:31,657][20381] Updated weights for policy 1, policy_version 1152465 (0.0008) [2023-12-27 13:38:31,658][20347] Updated weights for policy 0, policy_version 1153710 (0.0008) [2023-12-27 13:38:32,308][20347] Updated weights for policy 0, policy_version 1153720 (0.0009) [2023-12-27 13:38:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 590462976. Throughput: 0: 10401.6, 1: 10198.8. Samples: 590478960. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:32,363][19170] Avg episode reward: [(0, '990.230'), (1, '588.980')] [2023-12-27 13:38:32,371][20347] Updated weights for policy 0, policy_version 1153730 (0.0010) [2023-12-27 13:38:32,400][20381] Updated weights for policy 1, policy_version 1152475 (0.0009) [2023-12-27 13:38:32,433][20347] Updated weights for policy 0, policy_version 1153740 (0.0006) [2023-12-27 13:38:32,459][20381] Updated weights for policy 1, policy_version 1152485 (0.0007) [2023-12-27 13:38:32,520][20381] Updated weights for policy 1, policy_version 1152495 (0.0009) [2023-12-27 13:38:33,025][20347] Updated weights for policy 0, policy_version 1153750 (0.0007) [2023-12-27 13:38:33,077][20347] Updated weights for policy 0, policy_version 1153760 (0.0007) [2023-12-27 13:38:33,132][20347] Updated weights for policy 0, policy_version 1153770 (0.0007) [2023-12-27 13:38:33,196][20381] Updated weights for policy 1, policy_version 1152505 (0.0007) [2023-12-27 13:38:33,242][20381] Updated weights for policy 1, policy_version 1152515 (0.0005) [2023-12-27 13:38:33,302][20381] Updated weights for policy 1, policy_version 1152525 (0.0005) [2023-12-27 13:38:33,349][20381] Updated weights for policy 1, policy_version 1152535 (0.0010) [2023-12-27 13:38:33,823][20347] Updated weights for policy 0, policy_version 1153780 (0.0008) [2023-12-27 13:38:33,868][20347] Updated weights for policy 0, policy_version 1153790 (0.0007) [2023-12-27 13:38:33,923][20347] Updated weights for policy 0, policy_version 1153800 (0.0008) [2023-12-27 13:38:33,993][20381] Updated weights for policy 1, policy_version 1152545 (0.0006) [2023-12-27 13:38:34,047][20381] Updated weights for policy 1, policy_version 1152555 (0.0009) [2023-12-27 13:38:34,093][20381] Updated weights for policy 1, policy_version 1152565 (0.0010) [2023-12-27 13:38:34,626][20347] Updated weights for policy 0, policy_version 1153810 (0.0006) [2023-12-27 13:38:34,688][20347] Updated weights for policy 0, policy_version 1153820 (0.0005) [2023-12-27 13:38:34,743][20347] Updated weights for policy 0, policy_version 1153830 (0.0006) [2023-12-27 13:38:34,769][20381] Updated weights for policy 1, policy_version 1152575 (0.0009) [2023-12-27 13:38:34,793][20347] Updated weights for policy 0, policy_version 1153840 (0.0005) [2023-12-27 13:38:34,820][20381] Updated weights for policy 1, policy_version 1152585 (0.0008) [2023-12-27 13:38:34,867][20381] Updated weights for policy 1, policy_version 1152595 (0.0007) [2023-12-27 13:38:35,389][20347] Updated weights for policy 0, policy_version 1153850 (0.0007) [2023-12-27 13:38:35,455][20347] Updated weights for policy 0, policy_version 1153860 (0.0007) [2023-12-27 13:38:35,508][20381] Updated weights for policy 1, policy_version 1152605 (0.0008) [2023-12-27 13:38:35,510][20347] Updated weights for policy 0, policy_version 1153870 (0.0006) [2023-12-27 13:38:35,557][20381] Updated weights for policy 1, policy_version 1152615 (0.0010) [2023-12-27 13:38:35,605][20381] Updated weights for policy 1, policy_version 1152625 (0.0010) [2023-12-27 13:38:36,165][20347] Updated weights for policy 0, policy_version 1153880 (0.0007) [2023-12-27 13:38:36,215][20347] Updated weights for policy 0, policy_version 1153890 (0.0010) [2023-12-27 13:38:36,251][20381] Updated weights for policy 1, policy_version 1152635 (0.0009) [2023-12-27 13:38:36,275][20347] Updated weights for policy 0, policy_version 1153900 (0.0011) [2023-12-27 13:38:36,316][20381] Updated weights for policy 1, policy_version 1152645 (0.0006) [2023-12-27 13:38:36,369][20381] Updated weights for policy 1, policy_version 1152655 (0.0008) [2023-12-27 13:38:36,986][20347] Updated weights for policy 0, policy_version 1153910 (0.0008) [2023-12-27 13:38:37,039][20347] Updated weights for policy 0, policy_version 1153920 (0.0006) [2023-12-27 13:38:37,104][20347] Updated weights for policy 0, policy_version 1153930 (0.0005) [2023-12-27 13:38:37,108][20381] Updated weights for policy 1, policy_version 1152665 (0.0008) [2023-12-27 13:38:37,159][20381] Updated weights for policy 1, policy_version 1152675 (0.0008) [2023-12-27 13:38:37,213][20381] Updated weights for policy 1, policy_version 1152685 (0.0009) [2023-12-27 13:38:37,262][20381] Updated weights for policy 1, policy_version 1152695 (0.0009) [2023-12-27 13:38:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 590577664. Throughput: 0: 10412.6, 1: 10245.9. Samples: 590542440. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:37,363][19170] Avg episode reward: [(0, '980.490'), (1, '559.510')] [2023-12-27 13:38:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001153936_295460864.pth... [2023-12-27 13:38:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001152696_295116800.pth... [2023-12-27 13:38:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001152720_295149568.pth [2023-12-27 13:38:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001151480_294805504.pth [2023-12-27 13:38:37,664][20347] Updated weights for policy 0, policy_version 1153940 (0.0007) [2023-12-27 13:38:37,716][20347] Updated weights for policy 0, policy_version 1153950 (0.0010) [2023-12-27 13:38:37,762][20347] Updated weights for policy 0, policy_version 1153960 (0.0010) [2023-12-27 13:38:37,960][20381] Updated weights for policy 1, policy_version 1152705 (0.0007) [2023-12-27 13:38:38,017][20381] Updated weights for policy 1, policy_version 1152715 (0.0005) [2023-12-27 13:38:38,074][20381] Updated weights for policy 1, policy_version 1152725 (0.0008) [2023-12-27 13:38:38,375][20347] Updated weights for policy 0, policy_version 1153970 (0.0010) [2023-12-27 13:38:38,426][20347] Updated weights for policy 0, policy_version 1153980 (0.0008) [2023-12-27 13:38:38,482][20347] Updated weights for policy 0, policy_version 1153990 (0.0008) [2023-12-27 13:38:38,538][20347] Updated weights for policy 0, policy_version 1154000 (0.0008) [2023-12-27 13:38:38,689][20381] Updated weights for policy 1, policy_version 1152735 (0.0007) [2023-12-27 13:38:38,747][20381] Updated weights for policy 1, policy_version 1152745 (0.0006) [2023-12-27 13:38:38,806][20381] Updated weights for policy 1, policy_version 1152755 (0.0006) [2023-12-27 13:38:39,194][20347] Updated weights for policy 0, policy_version 1154010 (0.0008) [2023-12-27 13:38:39,263][20347] Updated weights for policy 0, policy_version 1154020 (0.0010) [2023-12-27 13:38:39,330][20347] Updated weights for policy 0, policy_version 1154030 (0.0008) [2023-12-27 13:38:39,514][20381] Updated weights for policy 1, policy_version 1152765 (0.0006) [2023-12-27 13:38:39,569][20381] Updated weights for policy 1, policy_version 1152775 (0.0006) [2023-12-27 13:38:39,639][20381] Updated weights for policy 1, policy_version 1152785 (0.0006) [2023-12-27 13:38:40,049][20347] Updated weights for policy 0, policy_version 1154040 (0.0007) [2023-12-27 13:38:40,103][20347] Updated weights for policy 0, policy_version 1154050 (0.0007) [2023-12-27 13:38:40,162][20347] Updated weights for policy 0, policy_version 1154060 (0.0005) [2023-12-27 13:38:40,323][20381] Updated weights for policy 1, policy_version 1152795 (0.0007) [2023-12-27 13:38:40,387][20381] Updated weights for policy 1, policy_version 1152805 (0.0007) [2023-12-27 13:38:40,443][20381] Updated weights for policy 1, policy_version 1152815 (0.0008) [2023-12-27 13:38:40,820][20347] Updated weights for policy 0, policy_version 1154070 (0.0007) [2023-12-27 13:38:40,877][20347] Updated weights for policy 0, policy_version 1154080 (0.0008) [2023-12-27 13:38:40,932][20347] Updated weights for policy 0, policy_version 1154090 (0.0007) [2023-12-27 13:38:41,151][20381] Updated weights for policy 1, policy_version 1152825 (0.0008) [2023-12-27 13:38:41,212][20381] Updated weights for policy 1, policy_version 1152835 (0.0008) [2023-12-27 13:38:41,269][20381] Updated weights for policy 1, policy_version 1152845 (0.0008) [2023-12-27 13:38:41,327][20381] Updated weights for policy 1, policy_version 1152855 (0.0008) [2023-12-27 13:38:41,723][20347] Updated weights for policy 0, policy_version 1154100 (0.0008) [2023-12-27 13:38:41,800][20347] Updated weights for policy 0, policy_version 1154110 (0.0008) [2023-12-27 13:38:41,867][20347] Updated weights for policy 0, policy_version 1154120 (0.0006) [2023-12-27 13:38:42,090][20381] Updated weights for policy 1, policy_version 1152865 (0.0008) [2023-12-27 13:38:42,152][20381] Updated weights for policy 1, policy_version 1152875 (0.0007) [2023-12-27 13:38:42,207][20381] Updated weights for policy 1, policy_version 1152885 (0.0008) [2023-12-27 13:38:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 590675968. Throughput: 0: 10438.2, 1: 10237.6. Samples: 590665492. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '579.620')] [2023-12-27 13:38:42,625][20347] Updated weights for policy 0, policy_version 1154130 (0.0007) [2023-12-27 13:38:42,684][20347] Updated weights for policy 0, policy_version 1154140 (0.0007) [2023-12-27 13:38:42,737][20347] Updated weights for policy 0, policy_version 1154150 (0.0008) [2023-12-27 13:38:42,796][20347] Updated weights for policy 0, policy_version 1154160 (0.0007) [2023-12-27 13:38:42,892][20381] Updated weights for policy 1, policy_version 1152895 (0.0007) [2023-12-27 13:38:42,951][20381] Updated weights for policy 1, policy_version 1152905 (0.0005) [2023-12-27 13:38:43,008][20381] Updated weights for policy 1, policy_version 1152915 (0.0005) [2023-12-27 13:38:43,397][20347] Updated weights for policy 0, policy_version 1154170 (0.0006) [2023-12-27 13:38:43,444][20347] Updated weights for policy 0, policy_version 1154180 (0.0009) [2023-12-27 13:38:43,501][20347] Updated weights for policy 0, policy_version 1154190 (0.0005) [2023-12-27 13:38:43,704][20381] Updated weights for policy 1, policy_version 1152925 (0.0006) [2023-12-27 13:38:43,764][20381] Updated weights for policy 1, policy_version 1152935 (0.0008) [2023-12-27 13:38:43,813][20381] Updated weights for policy 1, policy_version 1152945 (0.0007) [2023-12-27 13:38:44,143][20347] Updated weights for policy 0, policy_version 1154200 (0.0009) [2023-12-27 13:38:44,198][20347] Updated weights for policy 0, policy_version 1154210 (0.0010) [2023-12-27 13:38:44,254][20347] Updated weights for policy 0, policy_version 1154220 (0.0011) [2023-12-27 13:38:44,552][20381] Updated weights for policy 1, policy_version 1152955 (0.0007) [2023-12-27 13:38:44,609][20381] Updated weights for policy 1, policy_version 1152965 (0.0008) [2023-12-27 13:38:44,659][20381] Updated weights for policy 1, policy_version 1152975 (0.0007) [2023-12-27 13:38:44,897][20347] Updated weights for policy 0, policy_version 1154230 (0.0010) [2023-12-27 13:38:44,954][20347] Updated weights for policy 0, policy_version 1154240 (0.0011) [2023-12-27 13:38:45,012][20347] Updated weights for policy 0, policy_version 1154250 (0.0009) [2023-12-27 13:38:45,347][20381] Updated weights for policy 1, policy_version 1152985 (0.0006) [2023-12-27 13:38:45,406][20381] Updated weights for policy 1, policy_version 1152995 (0.0006) [2023-12-27 13:38:45,466][20381] Updated weights for policy 1, policy_version 1153005 (0.0006) [2023-12-27 13:38:45,530][20381] Updated weights for policy 1, policy_version 1153015 (0.0005) [2023-12-27 13:38:45,680][20347] Updated weights for policy 0, policy_version 1154260 (0.0007) [2023-12-27 13:38:45,743][20347] Updated weights for policy 0, policy_version 1154270 (0.0006) [2023-12-27 13:38:45,800][20347] Updated weights for policy 0, policy_version 1154280 (0.0006) [2023-12-27 13:38:46,137][20381] Updated weights for policy 1, policy_version 1153025 (0.0006) [2023-12-27 13:38:46,196][20381] Updated weights for policy 1, policy_version 1153035 (0.0005) [2023-12-27 13:38:46,256][20381] Updated weights for policy 1, policy_version 1153045 (0.0005) [2023-12-27 13:38:46,404][20347] Updated weights for policy 0, policy_version 1154290 (0.0006) [2023-12-27 13:38:46,456][20347] Updated weights for policy 0, policy_version 1154300 (0.0008) [2023-12-27 13:38:46,501][20347] Updated weights for policy 0, policy_version 1154310 (0.0007) [2023-12-27 13:38:46,547][20347] Updated weights for policy 0, policy_version 1154320 (0.0008) [2023-12-27 13:38:46,858][20381] Updated weights for policy 1, policy_version 1153055 (0.0007) [2023-12-27 13:38:46,907][20381] Updated weights for policy 1, policy_version 1153065 (0.0008) [2023-12-27 13:38:46,950][20381] Updated weights for policy 1, policy_version 1153075 (0.0008) [2023-12-27 13:38:47,284][20347] Updated weights for policy 0, policy_version 1154330 (0.0007) [2023-12-27 13:38:47,342][20347] Updated weights for policy 0, policy_version 1154340 (0.0007) [2023-12-27 13:38:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 590774272. Throughput: 0: 10476.9, 1: 10286.8. Samples: 590788576. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '579.390')] [2023-12-27 13:38:47,391][20347] Updated weights for policy 0, policy_version 1154350 (0.0008) [2023-12-27 13:38:47,644][20381] Updated weights for policy 1, policy_version 1153085 (0.0007) [2023-12-27 13:38:47,701][20381] Updated weights for policy 1, policy_version 1153095 (0.0007) [2023-12-27 13:38:47,760][20381] Updated weights for policy 1, policy_version 1153105 (0.0008) [2023-12-27 13:38:48,072][20347] Updated weights for policy 0, policy_version 1154360 (0.0010) [2023-12-27 13:38:48,127][20347] Updated weights for policy 0, policy_version 1154370 (0.0010) [2023-12-27 13:38:48,178][20347] Updated weights for policy 0, policy_version 1154380 (0.0010) [2023-12-27 13:38:48,342][20381] Updated weights for policy 1, policy_version 1153115 (0.0008) [2023-12-27 13:38:48,402][20381] Updated weights for policy 1, policy_version 1153125 (0.0011) [2023-12-27 13:38:48,459][20381] Updated weights for policy 1, policy_version 1153135 (0.0011) [2023-12-27 13:38:48,899][20347] Updated weights for policy 0, policy_version 1154390 (0.0009) [2023-12-27 13:38:48,956][20347] Updated weights for policy 0, policy_version 1154400 (0.0007) [2023-12-27 13:38:49,011][20347] Updated weights for policy 0, policy_version 1154410 (0.0007) [2023-12-27 13:38:49,143][20381] Updated weights for policy 1, policy_version 1153145 (0.0011) [2023-12-27 13:38:49,202][20381] Updated weights for policy 1, policy_version 1153155 (0.0010) [2023-12-27 13:38:49,276][20381] Updated weights for policy 1, policy_version 1153165 (0.0011) [2023-12-27 13:38:49,336][20381] Updated weights for policy 1, policy_version 1153175 (0.0008) [2023-12-27 13:38:49,672][20347] Updated weights for policy 0, policy_version 1154420 (0.0007) [2023-12-27 13:38:49,741][20347] Updated weights for policy 0, policy_version 1154430 (0.0007) [2023-12-27 13:38:49,809][20347] Updated weights for policy 0, policy_version 1154440 (0.0007) [2023-12-27 13:38:49,995][20381] Updated weights for policy 1, policy_version 1153185 (0.0008) [2023-12-27 13:38:50,047][20381] Updated weights for policy 1, policy_version 1153195 (0.0007) [2023-12-27 13:38:50,106][20381] Updated weights for policy 1, policy_version 1153205 (0.0007) [2023-12-27 13:38:50,509][20347] Updated weights for policy 0, policy_version 1154450 (0.0007) [2023-12-27 13:38:50,571][20347] Updated weights for policy 0, policy_version 1154460 (0.0008) [2023-12-27 13:38:50,628][20347] Updated weights for policy 0, policy_version 1154470 (0.0008) [2023-12-27 13:38:50,687][20347] Updated weights for policy 0, policy_version 1154480 (0.0007) [2023-12-27 13:38:50,785][20381] Updated weights for policy 1, policy_version 1153215 (0.0008) [2023-12-27 13:38:50,845][20381] Updated weights for policy 1, policy_version 1153225 (0.0007) [2023-12-27 13:38:50,907][20381] Updated weights for policy 1, policy_version 1153235 (0.0007) [2023-12-27 13:38:51,381][20347] Updated weights for policy 0, policy_version 1154490 (0.0008) [2023-12-27 13:38:51,433][20347] Updated weights for policy 0, policy_version 1154500 (0.0007) [2023-12-27 13:38:51,483][20347] Updated weights for policy 0, policy_version 1154510 (0.0008) [2023-12-27 13:38:51,597][20381] Updated weights for policy 1, policy_version 1153245 (0.0008) [2023-12-27 13:38:51,659][20381] Updated weights for policy 1, policy_version 1153255 (0.0007) [2023-12-27 13:38:51,726][20381] Updated weights for policy 1, policy_version 1153265 (0.0007) [2023-12-27 13:38:52,262][20347] Updated weights for policy 0, policy_version 1154520 (0.0006) [2023-12-27 13:38:52,318][20347] Updated weights for policy 0, policy_version 1154530 (0.0008) [2023-12-27 13:38:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 590872576. Throughput: 0: 10415.5, 1: 10304.9. Samples: 590849656. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '589.240')] [2023-12-27 13:38:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001153272_295264256.pth... [2023-12-27 13:38:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001152088_294961152.pth [2023-12-27 13:38:52,383][20347] Updated weights for policy 0, policy_version 1154540 (0.0007) [2023-12-27 13:38:52,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001154544_295616512.pth... [2023-12-27 13:38:52,406][20381] Updated weights for policy 1, policy_version 1153275 (0.0008) [2023-12-27 13:38:52,409][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001153328_295305216.pth [2023-12-27 13:38:52,472][20381] Updated weights for policy 1, policy_version 1153285 (0.0007) [2023-12-27 13:38:52,533][20381] Updated weights for policy 1, policy_version 1153295 (0.0007) [2023-12-27 13:38:53,050][20347] Updated weights for policy 0, policy_version 1154550 (0.0010) [2023-12-27 13:38:53,102][20347] Updated weights for policy 0, policy_version 1154560 (0.0010) [2023-12-27 13:38:53,148][20381] Updated weights for policy 1, policy_version 1153305 (0.0009) [2023-12-27 13:38:53,157][20347] Updated weights for policy 0, policy_version 1154570 (0.0010) [2023-12-27 13:38:53,215][20381] Updated weights for policy 1, policy_version 1153315 (0.0006) [2023-12-27 13:38:53,267][20381] Updated weights for policy 1, policy_version 1153325 (0.0008) [2023-12-27 13:38:53,320][20381] Updated weights for policy 1, policy_version 1153335 (0.0008) [2023-12-27 13:38:53,832][20347] Updated weights for policy 0, policy_version 1154580 (0.0009) [2023-12-27 13:38:53,879][20347] Updated weights for policy 0, policy_version 1154590 (0.0007) [2023-12-27 13:38:53,930][20381] Updated weights for policy 1, policy_version 1153345 (0.0006) [2023-12-27 13:38:53,935][20347] Updated weights for policy 0, policy_version 1154600 (0.0007) [2023-12-27 13:38:53,983][20381] Updated weights for policy 1, policy_version 1153355 (0.0006) [2023-12-27 13:38:54,031][20381] Updated weights for policy 1, policy_version 1153365 (0.0007) [2023-12-27 13:38:54,585][20347] Updated weights for policy 0, policy_version 1154610 (0.0006) [2023-12-27 13:38:54,619][20291] KL-divergence is very high: 213.4786 [2023-12-27 13:38:54,634][20347] Updated weights for policy 0, policy_version 1154620 (0.0008) [2023-12-27 13:38:54,659][20291] KL-divergence is very high: 119.2046 [2023-12-27 13:38:54,686][20347] Updated weights for policy 0, policy_version 1154630 (0.0007) [2023-12-27 13:38:54,710][20381] Updated weights for policy 1, policy_version 1153375 (0.0007) [2023-12-27 13:38:54,739][20347] Updated weights for policy 0, policy_version 1154640 (0.0006) [2023-12-27 13:38:54,767][20381] Updated weights for policy 1, policy_version 1153385 (0.0007) [2023-12-27 13:38:54,820][20381] Updated weights for policy 1, policy_version 1153395 (0.0007) [2023-12-27 13:38:55,381][20347] Updated weights for policy 0, policy_version 1154650 (0.0008) [2023-12-27 13:38:55,439][20347] Updated weights for policy 0, policy_version 1154660 (0.0007) [2023-12-27 13:38:55,491][20381] Updated weights for policy 1, policy_version 1153405 (0.0008) [2023-12-27 13:38:55,494][20347] Updated weights for policy 0, policy_version 1154670 (0.0007) [2023-12-27 13:38:55,561][20381] Updated weights for policy 1, policy_version 1153415 (0.0009) [2023-12-27 13:38:55,629][20381] Updated weights for policy 1, policy_version 1153425 (0.0008) [2023-12-27 13:38:56,058][20347] Updated weights for policy 0, policy_version 1154680 (0.0005) [2023-12-27 13:38:56,113][20347] Updated weights for policy 0, policy_version 1154690 (0.0006) [2023-12-27 13:38:56,166][20347] Updated weights for policy 0, policy_version 1154700 (0.0006) [2023-12-27 13:38:56,363][20381] Updated weights for policy 1, policy_version 1153435 (0.0009) [2023-12-27 13:38:56,411][20381] Updated weights for policy 1, policy_version 1153445 (0.0008) [2023-12-27 13:38:56,458][20381] Updated weights for policy 1, policy_version 1153455 (0.0005) [2023-12-27 13:38:56,759][20347] Updated weights for policy 0, policy_version 1154710 (0.0006) [2023-12-27 13:38:56,813][20347] Updated weights for policy 0, policy_version 1154720 (0.0008) [2023-12-27 13:38:56,870][20347] Updated weights for policy 0, policy_version 1154730 (0.0008) [2023-12-27 13:38:57,060][20381] Updated weights for policy 1, policy_version 1153465 (0.0005) [2023-12-27 13:38:57,111][20381] Updated weights for policy 1, policy_version 1153475 (0.0005) [2023-12-27 13:38:57,166][20381] Updated weights for policy 1, policy_version 1153485 (0.0006) [2023-12-27 13:38:57,214][20381] Updated weights for policy 1, policy_version 1153495 (0.0006) [2023-12-27 13:38:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 590987264. Throughput: 0: 10384.1, 1: 10293.6. Samples: 590973820. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:38:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '647.900')] [2023-12-27 13:38:57,595][20347] Updated weights for policy 0, policy_version 1154740 (0.0007) [2023-12-27 13:38:57,645][20347] Updated weights for policy 0, policy_version 1154750 (0.0008) [2023-12-27 13:38:57,700][20347] Updated weights for policy 0, policy_version 1154760 (0.0008) [2023-12-27 13:38:57,790][20381] Updated weights for policy 1, policy_version 1153505 (0.0006) [2023-12-27 13:38:57,845][20381] Updated weights for policy 1, policy_version 1153515 (0.0006) [2023-12-27 13:38:57,894][20381] Updated weights for policy 1, policy_version 1153525 (0.0008) [2023-12-27 13:38:58,397][20347] Updated weights for policy 0, policy_version 1154770 (0.0008) [2023-12-27 13:38:58,457][20347] Updated weights for policy 0, policy_version 1154780 (0.0008) [2023-12-27 13:38:58,520][20347] Updated weights for policy 0, policy_version 1154790 (0.0006) [2023-12-27 13:38:58,583][20381] Updated weights for policy 1, policy_version 1153535 (0.0006) [2023-12-27 13:38:58,584][20347] Updated weights for policy 0, policy_version 1154800 (0.0007) [2023-12-27 13:38:58,646][20381] Updated weights for policy 1, policy_version 1153545 (0.0008) [2023-12-27 13:38:58,711][20381] Updated weights for policy 1, policy_version 1153555 (0.0009) [2023-12-27 13:38:59,282][20347] Updated weights for policy 0, policy_version 1154810 (0.0008) [2023-12-27 13:38:59,338][20347] Updated weights for policy 0, policy_version 1154820 (0.0007) [2023-12-27 13:38:59,427][20347] Updated weights for policy 0, policy_version 1154830 (0.0007) [2023-12-27 13:38:59,465][20381] Updated weights for policy 1, policy_version 1153565 (0.0009) [2023-12-27 13:38:59,517][20381] Updated weights for policy 1, policy_version 1153575 (0.0010) [2023-12-27 13:38:59,574][20381] Updated weights for policy 1, policy_version 1153585 (0.0009) [2023-12-27 13:39:00,085][20347] Updated weights for policy 0, policy_version 1154840 (0.0009) [2023-12-27 13:39:00,134][20347] Updated weights for policy 0, policy_version 1154850 (0.0007) [2023-12-27 13:39:00,181][20347] Updated weights for policy 0, policy_version 1154860 (0.0008) [2023-12-27 13:39:00,271][20381] Updated weights for policy 1, policy_version 1153595 (0.0010) [2023-12-27 13:39:00,321][20381] Updated weights for policy 1, policy_version 1153605 (0.0009) [2023-12-27 13:39:00,375][20381] Updated weights for policy 1, policy_version 1153615 (0.0008) [2023-12-27 13:39:00,873][20347] Updated weights for policy 0, policy_version 1154870 (0.0006) [2023-12-27 13:39:00,930][20347] Updated weights for policy 0, policy_version 1154880 (0.0006) [2023-12-27 13:39:00,988][20347] Updated weights for policy 0, policy_version 1154890 (0.0010) [2023-12-27 13:39:01,056][20381] Updated weights for policy 1, policy_version 1153625 (0.0008) [2023-12-27 13:39:01,116][20381] Updated weights for policy 1, policy_version 1153635 (0.0012) [2023-12-27 13:39:01,170][20381] Updated weights for policy 1, policy_version 1153645 (0.0008) [2023-12-27 13:39:01,234][20381] Updated weights for policy 1, policy_version 1153655 (0.0007) [2023-12-27 13:39:01,654][20347] Updated weights for policy 0, policy_version 1154900 (0.0009) [2023-12-27 13:39:01,707][20347] Updated weights for policy 0, policy_version 1154910 (0.0009) [2023-12-27 13:39:01,766][20347] Updated weights for policy 0, policy_version 1154920 (0.0008) [2023-12-27 13:39:01,934][20381] Updated weights for policy 1, policy_version 1153665 (0.0008) [2023-12-27 13:39:01,996][20381] Updated weights for policy 1, policy_version 1153675 (0.0010) [2023-12-27 13:39:02,055][20381] Updated weights for policy 1, policy_version 1153685 (0.0011) [2023-12-27 13:39:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 591085568. Throughput: 0: 10357.3, 1: 10282.5. Samples: 591096492. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:02,363][19170] Avg episode reward: [(0, '980.450'), (1, '667.410')] [2023-12-27 13:39:02,446][20347] Updated weights for policy 0, policy_version 1154930 (0.0008) [2023-12-27 13:39:02,507][20347] Updated weights for policy 0, policy_version 1154940 (0.0007) [2023-12-27 13:39:02,565][20347] Updated weights for policy 0, policy_version 1154950 (0.0009) [2023-12-27 13:39:02,620][20347] Updated weights for policy 0, policy_version 1154960 (0.0005) [2023-12-27 13:39:02,721][20381] Updated weights for policy 1, policy_version 1153695 (0.0010) [2023-12-27 13:39:02,770][20381] Updated weights for policy 1, policy_version 1153705 (0.0010) [2023-12-27 13:39:02,820][20381] Updated weights for policy 1, policy_version 1153715 (0.0008) [2023-12-27 13:39:03,251][20347] Updated weights for policy 0, policy_version 1154970 (0.0008) [2023-12-27 13:39:03,306][20347] Updated weights for policy 0, policy_version 1154980 (0.0007) [2023-12-27 13:39:03,360][20347] Updated weights for policy 0, policy_version 1154990 (0.0007) [2023-12-27 13:39:03,496][20381] Updated weights for policy 1, policy_version 1153725 (0.0008) [2023-12-27 13:39:03,543][20381] Updated weights for policy 1, policy_version 1153735 (0.0006) [2023-12-27 13:39:03,591][20381] Updated weights for policy 1, policy_version 1153745 (0.0009) [2023-12-27 13:39:04,083][20347] Updated weights for policy 0, policy_version 1155000 (0.0008) [2023-12-27 13:39:04,148][20347] Updated weights for policy 0, policy_version 1155010 (0.0008) [2023-12-27 13:39:04,211][20347] Updated weights for policy 0, policy_version 1155020 (0.0008) [2023-12-27 13:39:04,270][20381] Updated weights for policy 1, policy_version 1153755 (0.0009) [2023-12-27 13:39:04,331][20381] Updated weights for policy 1, policy_version 1153765 (0.0006) [2023-12-27 13:39:04,393][20381] Updated weights for policy 1, policy_version 1153775 (0.0007) [2023-12-27 13:39:04,938][20347] Updated weights for policy 0, policy_version 1155030 (0.0008) [2023-12-27 13:39:04,982][20381] Updated weights for policy 1, policy_version 1153785 (0.0007) [2023-12-27 13:39:05,000][20347] Updated weights for policy 0, policy_version 1155040 (0.0007) [2023-12-27 13:39:05,034][20381] Updated weights for policy 1, policy_version 1153795 (0.0006) [2023-12-27 13:39:05,060][20347] Updated weights for policy 0, policy_version 1155050 (0.0008) [2023-12-27 13:39:05,079][20381] Updated weights for policy 1, policy_version 1153805 (0.0005) [2023-12-27 13:39:05,127][20381] Updated weights for policy 1, policy_version 1153815 (0.0005) [2023-12-27 13:39:05,659][20381] Updated weights for policy 1, policy_version 1153825 (0.0006) [2023-12-27 13:39:05,704][20381] Updated weights for policy 1, policy_version 1153835 (0.0005) [2023-12-27 13:39:05,753][20347] Updated weights for policy 0, policy_version 1155060 (0.0008) [2023-12-27 13:39:05,759][20381] Updated weights for policy 1, policy_version 1153845 (0.0005) [2023-12-27 13:39:05,810][20347] Updated weights for policy 0, policy_version 1155070 (0.0007) [2023-12-27 13:39:05,865][20347] Updated weights for policy 0, policy_version 1155080 (0.0010) [2023-12-27 13:39:06,309][20381] Updated weights for policy 1, policy_version 1153855 (0.0007) [2023-12-27 13:39:06,371][20381] Updated weights for policy 1, policy_version 1153865 (0.0006) [2023-12-27 13:39:06,427][20381] Updated weights for policy 1, policy_version 1153875 (0.0007) [2023-12-27 13:39:06,534][20347] Updated weights for policy 0, policy_version 1155090 (0.0006) [2023-12-27 13:39:06,593][20347] Updated weights for policy 0, policy_version 1155100 (0.0011) [2023-12-27 13:39:06,653][20347] Updated weights for policy 0, policy_version 1155110 (0.0011) [2023-12-27 13:39:06,715][20347] Updated weights for policy 0, policy_version 1155120 (0.0010) [2023-12-27 13:39:07,110][20381] Updated weights for policy 1, policy_version 1153885 (0.0006) [2023-12-27 13:39:07,165][20381] Updated weights for policy 1, policy_version 1153895 (0.0007) [2023-12-27 13:39:07,215][20381] Updated weights for policy 1, policy_version 1153905 (0.0006) [2023-12-27 13:39:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 591192064. Throughput: 0: 10339.2, 1: 10327.2. Samples: 591158272. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:07,363][19170] Avg episode reward: [(0, '990.200'), (1, '686.680')] [2023-12-27 13:39:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001153912_295428096.pth... [2023-12-27 13:39:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001152696_295116800.pth [2023-12-27 13:39:07,400][20347] Updated weights for policy 0, policy_version 1155130 (0.0010) [2023-12-27 13:39:07,455][20347] Updated weights for policy 0, policy_version 1155140 (0.0010) [2023-12-27 13:39:07,514][20347] Updated weights for policy 0, policy_version 1155150 (0.0010) [2023-12-27 13:39:07,523][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001155152_295772160.pth... [2023-12-27 13:39:07,527][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001153936_295460864.pth [2023-12-27 13:39:07,799][20381] Updated weights for policy 1, policy_version 1153915 (0.0007) [2023-12-27 13:39:07,843][20381] Updated weights for policy 1, policy_version 1153925 (0.0005) [2023-12-27 13:39:07,896][20381] Updated weights for policy 1, policy_version 1153935 (0.0005) [2023-12-27 13:39:08,188][20347] Updated weights for policy 0, policy_version 1155160 (0.0006) [2023-12-27 13:39:08,241][20347] Updated weights for policy 0, policy_version 1155170 (0.0005) [2023-12-27 13:39:08,292][20347] Updated weights for policy 0, policy_version 1155180 (0.0005) [2023-12-27 13:39:08,553][20381] Updated weights for policy 1, policy_version 1153945 (0.0006) [2023-12-27 13:39:08,602][20381] Updated weights for policy 1, policy_version 1153955 (0.0008) [2023-12-27 13:39:08,652][20381] Updated weights for policy 1, policy_version 1153965 (0.0007) [2023-12-27 13:39:08,705][20381] Updated weights for policy 1, policy_version 1153975 (0.0008) [2023-12-27 13:39:08,943][20347] Updated weights for policy 0, policy_version 1155190 (0.0005) [2023-12-27 13:39:09,001][20347] Updated weights for policy 0, policy_version 1155200 (0.0005) [2023-12-27 13:39:09,062][20347] Updated weights for policy 0, policy_version 1155210 (0.0007) [2023-12-27 13:39:09,462][20381] Updated weights for policy 1, policy_version 1153985 (0.0006) [2023-12-27 13:39:09,525][20381] Updated weights for policy 1, policy_version 1153995 (0.0008) [2023-12-27 13:39:09,590][20381] Updated weights for policy 1, policy_version 1154005 (0.0007) [2023-12-27 13:39:09,770][20347] Updated weights for policy 0, policy_version 1155220 (0.0007) [2023-12-27 13:39:09,832][20347] Updated weights for policy 0, policy_version 1155230 (0.0008) [2023-12-27 13:39:09,896][20347] Updated weights for policy 0, policy_version 1155240 (0.0007) [2023-12-27 13:39:10,288][20381] Updated weights for policy 1, policy_version 1154015 (0.0008) [2023-12-27 13:39:10,350][20381] Updated weights for policy 1, policy_version 1154025 (0.0008) [2023-12-27 13:39:10,414][20381] Updated weights for policy 1, policy_version 1154035 (0.0007) [2023-12-27 13:39:10,618][20347] Updated weights for policy 0, policy_version 1155250 (0.0007) [2023-12-27 13:39:10,675][20347] Updated weights for policy 0, policy_version 1155260 (0.0005) [2023-12-27 13:39:10,728][20347] Updated weights for policy 0, policy_version 1155270 (0.0007) [2023-12-27 13:39:10,780][20347] Updated weights for policy 0, policy_version 1155280 (0.0006) [2023-12-27 13:39:11,118][20381] Updated weights for policy 1, policy_version 1154045 (0.0008) [2023-12-27 13:39:11,188][20381] Updated weights for policy 1, policy_version 1154055 (0.0009) [2023-12-27 13:39:11,249][20381] Updated weights for policy 1, policy_version 1154065 (0.0008) [2023-12-27 13:39:11,426][20347] Updated weights for policy 0, policy_version 1155290 (0.0007) [2023-12-27 13:39:11,485][20347] Updated weights for policy 0, policy_version 1155300 (0.0006) [2023-12-27 13:39:11,543][20347] Updated weights for policy 0, policy_version 1155310 (0.0007) [2023-12-27 13:39:12,025][20381] Updated weights for policy 1, policy_version 1154075 (0.0008) [2023-12-27 13:39:12,080][20381] Updated weights for policy 1, policy_version 1154085 (0.0006) [2023-12-27 13:39:12,139][20381] Updated weights for policy 1, policy_version 1154095 (0.0008) [2023-12-27 13:39:12,313][20347] Updated weights for policy 0, policy_version 1155320 (0.0008) [2023-12-27 13:39:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 591290368. Throughput: 0: 10302.0, 1: 10371.3. Samples: 591282352. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:12,363][19170] Avg episode reward: [(0, '980.530'), (1, '608.720')] [2023-12-27 13:39:12,380][20347] Updated weights for policy 0, policy_version 1155330 (0.0008) [2023-12-27 13:39:12,448][20347] Updated weights for policy 0, policy_version 1155340 (0.0007) [2023-12-27 13:39:12,855][20381] Updated weights for policy 1, policy_version 1154105 (0.0006) [2023-12-27 13:39:12,912][20381] Updated weights for policy 1, policy_version 1154115 (0.0007) [2023-12-27 13:39:12,968][20381] Updated weights for policy 1, policy_version 1154125 (0.0007) [2023-12-27 13:39:13,025][20381] Updated weights for policy 1, policy_version 1154135 (0.0008) [2023-12-27 13:39:13,084][20347] Updated weights for policy 0, policy_version 1155350 (0.0007) [2023-12-27 13:39:13,147][20347] Updated weights for policy 0, policy_version 1155360 (0.0007) [2023-12-27 13:39:13,198][20347] Updated weights for policy 0, policy_version 1155370 (0.0008) [2023-12-27 13:39:13,704][20381] Updated weights for policy 1, policy_version 1154145 (0.0006) [2023-12-27 13:39:13,753][20381] Updated weights for policy 1, policy_version 1154155 (0.0005) [2023-12-27 13:39:13,801][20381] Updated weights for policy 1, policy_version 1154165 (0.0005) [2023-12-27 13:39:13,869][20347] Updated weights for policy 0, policy_version 1155380 (0.0008) [2023-12-27 13:39:13,929][20347] Updated weights for policy 0, policy_version 1155390 (0.0008) [2023-12-27 13:39:13,985][20347] Updated weights for policy 0, policy_version 1155400 (0.0007) [2023-12-27 13:39:14,363][20381] Updated weights for policy 1, policy_version 1154175 (0.0007) [2023-12-27 13:39:14,420][20381] Updated weights for policy 1, policy_version 1154185 (0.0008) [2023-12-27 13:39:14,479][20381] Updated weights for policy 1, policy_version 1154195 (0.0006) [2023-12-27 13:39:14,728][20347] Updated weights for policy 0, policy_version 1155410 (0.0007) [2023-12-27 13:39:14,799][20347] Updated weights for policy 0, policy_version 1155420 (0.0008) [2023-12-27 13:39:14,859][20347] Updated weights for policy 0, policy_version 1155430 (0.0009) [2023-12-27 13:39:14,917][20347] Updated weights for policy 0, policy_version 1155440 (0.0008) [2023-12-27 13:39:15,067][20381] Updated weights for policy 1, policy_version 1154205 (0.0006) [2023-12-27 13:39:15,118][20381] Updated weights for policy 1, policy_version 1154215 (0.0008) [2023-12-27 13:39:15,168][20381] Updated weights for policy 1, policy_version 1154225 (0.0007) [2023-12-27 13:39:15,629][20347] Updated weights for policy 0, policy_version 1155450 (0.0006) [2023-12-27 13:39:15,678][20347] Updated weights for policy 0, policy_version 1155460 (0.0006) [2023-12-27 13:39:15,730][20347] Updated weights for policy 0, policy_version 1155470 (0.0006) [2023-12-27 13:39:15,891][20381] Updated weights for policy 1, policy_version 1154235 (0.0006) [2023-12-27 13:39:15,941][20381] Updated weights for policy 1, policy_version 1154245 (0.0008) [2023-12-27 13:39:15,985][20381] Updated weights for policy 1, policy_version 1154255 (0.0007) [2023-12-27 13:39:16,351][20347] Updated weights for policy 0, policy_version 1155480 (0.0009) [2023-12-27 13:39:16,413][20347] Updated weights for policy 0, policy_version 1155490 (0.0010) [2023-12-27 13:39:16,465][20347] Updated weights for policy 0, policy_version 1155500 (0.0010) [2023-12-27 13:39:16,655][20381] Updated weights for policy 1, policy_version 1154265 (0.0007) [2023-12-27 13:39:16,712][20381] Updated weights for policy 1, policy_version 1154275 (0.0005) [2023-12-27 13:39:16,773][20381] Updated weights for policy 1, policy_version 1154285 (0.0005) [2023-12-27 13:39:16,830][20381] Updated weights for policy 1, policy_version 1154295 (0.0007) [2023-12-27 13:39:17,111][20347] Updated weights for policy 0, policy_version 1155510 (0.0010) [2023-12-27 13:39:17,159][20347] Updated weights for policy 0, policy_version 1155520 (0.0010) [2023-12-27 13:39:17,206][20347] Updated weights for policy 0, policy_version 1155530 (0.0010) [2023-12-27 13:39:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 591396864. Throughput: 0: 10232.7, 1: 10363.6. Samples: 591405796. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:17,363][19170] Avg episode reward: [(0, '970.740'), (1, '667.320')] [2023-12-27 13:39:17,456][20381] Updated weights for policy 1, policy_version 1154305 (0.0006) [2023-12-27 13:39:17,504][20381] Updated weights for policy 1, policy_version 1154315 (0.0005) [2023-12-27 13:39:17,565][20381] Updated weights for policy 1, policy_version 1154325 (0.0006) [2023-12-27 13:39:17,880][20347] Updated weights for policy 0, policy_version 1155540 (0.0008) [2023-12-27 13:39:17,930][20347] Updated weights for policy 0, policy_version 1155550 (0.0010) [2023-12-27 13:39:17,980][20347] Updated weights for policy 0, policy_version 1155560 (0.0007) [2023-12-27 13:39:18,236][20381] Updated weights for policy 1, policy_version 1154335 (0.0008) [2023-12-27 13:39:18,291][20381] Updated weights for policy 1, policy_version 1154345 (0.0008) [2023-12-27 13:39:18,353][20381] Updated weights for policy 1, policy_version 1154355 (0.0007) [2023-12-27 13:39:18,688][20347] Updated weights for policy 0, policy_version 1155570 (0.0009) [2023-12-27 13:39:18,752][20347] Updated weights for policy 0, policy_version 1155580 (0.0006) [2023-12-27 13:39:18,815][20347] Updated weights for policy 0, policy_version 1155590 (0.0006) [2023-12-27 13:39:18,873][20347] Updated weights for policy 0, policy_version 1155600 (0.0007) [2023-12-27 13:39:19,059][20381] Updated weights for policy 1, policy_version 1154365 (0.0006) [2023-12-27 13:39:19,114][20381] Updated weights for policy 1, policy_version 1154375 (0.0009) [2023-12-27 13:39:19,167][20381] Updated weights for policy 1, policy_version 1154385 (0.0010) [2023-12-27 13:39:19,534][20347] Updated weights for policy 0, policy_version 1155610 (0.0011) [2023-12-27 13:39:19,594][20347] Updated weights for policy 0, policy_version 1155620 (0.0010) [2023-12-27 13:39:19,651][20347] Updated weights for policy 0, policy_version 1155630 (0.0010) [2023-12-27 13:39:19,917][20381] Updated weights for policy 1, policy_version 1154395 (0.0010) [2023-12-27 13:39:19,986][20381] Updated weights for policy 1, policy_version 1154405 (0.0009) [2023-12-27 13:39:20,045][20381] Updated weights for policy 1, policy_version 1154415 (0.0009) [2023-12-27 13:39:20,279][20347] Updated weights for policy 0, policy_version 1155640 (0.0006) [2023-12-27 13:39:20,336][20347] Updated weights for policy 0, policy_version 1155650 (0.0009) [2023-12-27 13:39:20,388][20347] Updated weights for policy 0, policy_version 1155660 (0.0010) [2023-12-27 13:39:20,717][20381] Updated weights for policy 1, policy_version 1154425 (0.0011) [2023-12-27 13:39:20,770][20381] Updated weights for policy 1, policy_version 1154435 (0.0011) [2023-12-27 13:39:20,830][20381] Updated weights for policy 1, policy_version 1154445 (0.0011) [2023-12-27 13:39:20,890][20381] Updated weights for policy 1, policy_version 1154455 (0.0011) [2023-12-27 13:39:21,069][20347] Updated weights for policy 0, policy_version 1155670 (0.0007) [2023-12-27 13:39:21,130][20347] Updated weights for policy 0, policy_version 1155680 (0.0010) [2023-12-27 13:39:21,196][20347] Updated weights for policy 0, policy_version 1155690 (0.0010) [2023-12-27 13:39:21,618][20381] Updated weights for policy 1, policy_version 1154465 (0.0008) [2023-12-27 13:39:21,687][20381] Updated weights for policy 1, policy_version 1154475 (0.0008) [2023-12-27 13:39:21,748][20381] Updated weights for policy 1, policy_version 1154485 (0.0008) [2023-12-27 13:39:21,928][20347] Updated weights for policy 0, policy_version 1155700 (0.0010) [2023-12-27 13:39:21,981][20347] Updated weights for policy 0, policy_version 1155710 (0.0008) [2023-12-27 13:39:22,045][20347] Updated weights for policy 0, policy_version 1155720 (0.0008) [2023-12-27 13:39:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 591495168. Throughput: 0: 10204.8, 1: 10323.3. Samples: 591466204. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:22,363][19170] Avg episode reward: [(0, '970.630'), (1, '696.580')] [2023-12-27 13:39:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001154488_295575552.pth... [2023-12-27 13:39:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001155728_295919616.pth... [2023-12-27 13:39:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001154544_295616512.pth [2023-12-27 13:39:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001153272_295264256.pth [2023-12-27 13:39:22,433][20381] Updated weights for policy 1, policy_version 1154495 (0.0007) [2023-12-27 13:39:22,497][20381] Updated weights for policy 1, policy_version 1154505 (0.0007) [2023-12-27 13:39:22,558][20381] Updated weights for policy 1, policy_version 1154515 (0.0008) [2023-12-27 13:39:22,772][20347] Updated weights for policy 0, policy_version 1155730 (0.0008) [2023-12-27 13:39:22,832][20347] Updated weights for policy 0, policy_version 1155740 (0.0008) [2023-12-27 13:39:22,885][20347] Updated weights for policy 0, policy_version 1155750 (0.0008) [2023-12-27 13:39:22,940][20347] Updated weights for policy 0, policy_version 1155760 (0.0007) [2023-12-27 13:39:23,254][20381] Updated weights for policy 1, policy_version 1154525 (0.0008) [2023-12-27 13:39:23,301][20381] Updated weights for policy 1, policy_version 1154535 (0.0005) [2023-12-27 13:39:23,345][20381] Updated weights for policy 1, policy_version 1154545 (0.0005) [2023-12-27 13:39:23,661][20347] Updated weights for policy 0, policy_version 1155770 (0.0006) [2023-12-27 13:39:23,712][20347] Updated weights for policy 0, policy_version 1155780 (0.0007) [2023-12-27 13:39:23,765][20347] Updated weights for policy 0, policy_version 1155790 (0.0007) [2023-12-27 13:39:23,983][20381] Updated weights for policy 1, policy_version 1154555 (0.0005) [2023-12-27 13:39:24,048][20381] Updated weights for policy 1, policy_version 1154565 (0.0006) [2023-12-27 13:39:24,103][20381] Updated weights for policy 1, policy_version 1154575 (0.0008) [2023-12-27 13:39:24,460][20347] Updated weights for policy 0, policy_version 1155800 (0.0008) [2023-12-27 13:39:24,507][20347] Updated weights for policy 0, policy_version 1155810 (0.0007) [2023-12-27 13:39:24,566][20347] Updated weights for policy 0, policy_version 1155820 (0.0008) [2023-12-27 13:39:24,744][20381] Updated weights for policy 1, policy_version 1154585 (0.0008) [2023-12-27 13:39:24,798][20381] Updated weights for policy 1, policy_version 1154595 (0.0007) [2023-12-27 13:39:24,850][20381] Updated weights for policy 1, policy_version 1154605 (0.0007) [2023-12-27 13:39:24,896][20381] Updated weights for policy 1, policy_version 1154615 (0.0009) [2023-12-27 13:39:25,221][20347] Updated weights for policy 0, policy_version 1155830 (0.0008) [2023-12-27 13:39:25,281][20347] Updated weights for policy 0, policy_version 1155840 (0.0008) [2023-12-27 13:39:25,329][20347] Updated weights for policy 0, policy_version 1155850 (0.0007) [2023-12-27 13:39:25,569][20381] Updated weights for policy 1, policy_version 1154625 (0.0007) [2023-12-27 13:39:25,621][20381] Updated weights for policy 1, policy_version 1154635 (0.0007) [2023-12-27 13:39:25,671][20381] Updated weights for policy 1, policy_version 1154645 (0.0008) [2023-12-27 13:39:25,970][20347] Updated weights for policy 0, policy_version 1155860 (0.0007) [2023-12-27 13:39:26,030][20347] Updated weights for policy 0, policy_version 1155870 (0.0007) [2023-12-27 13:39:26,084][20347] Updated weights for policy 0, policy_version 1155880 (0.0010) [2023-12-27 13:39:26,376][20381] Updated weights for policy 1, policy_version 1154655 (0.0008) [2023-12-27 13:39:26,431][20381] Updated weights for policy 1, policy_version 1154665 (0.0008) [2023-12-27 13:39:26,486][20381] Updated weights for policy 1, policy_version 1154675 (0.0008) [2023-12-27 13:39:26,742][20347] Updated weights for policy 0, policy_version 1155890 (0.0010) [2023-12-27 13:39:26,802][20347] Updated weights for policy 0, policy_version 1155900 (0.0010) [2023-12-27 13:39:26,862][20347] Updated weights for policy 0, policy_version 1155910 (0.0010) [2023-12-27 13:39:26,917][20347] Updated weights for policy 0, policy_version 1155920 (0.0009) [2023-12-27 13:39:27,179][20381] Updated weights for policy 1, policy_version 1154685 (0.0008) [2023-12-27 13:39:27,231][20381] Updated weights for policy 1, policy_version 1154695 (0.0009) [2023-12-27 13:39:27,281][20381] Updated weights for policy 1, policy_version 1154705 (0.0008) [2023-12-27 13:39:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 591601664. Throughput: 0: 10176.3, 1: 10322.9. Samples: 591587952. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:27,363][19170] Avg episode reward: [(0, '960.840'), (1, '647.570')] [2023-12-27 13:39:27,503][20347] Updated weights for policy 0, policy_version 1155930 (0.0006) [2023-12-27 13:39:27,568][20347] Updated weights for policy 0, policy_version 1155940 (0.0008) [2023-12-27 13:39:27,625][20347] Updated weights for policy 0, policy_version 1155950 (0.0009) [2023-12-27 13:39:27,908][20381] Updated weights for policy 1, policy_version 1154715 (0.0007) [2023-12-27 13:39:27,960][20381] Updated weights for policy 1, policy_version 1154725 (0.0007) [2023-12-27 13:39:28,009][20381] Updated weights for policy 1, policy_version 1154735 (0.0008) [2023-12-27 13:39:28,274][20347] Updated weights for policy 0, policy_version 1155960 (0.0008) [2023-12-27 13:39:28,332][20347] Updated weights for policy 0, policy_version 1155970 (0.0007) [2023-12-27 13:39:28,389][20347] Updated weights for policy 0, policy_version 1155980 (0.0008) [2023-12-27 13:39:28,774][20381] Updated weights for policy 1, policy_version 1154745 (0.0008) [2023-12-27 13:39:28,831][20381] Updated weights for policy 1, policy_version 1154755 (0.0007) [2023-12-27 13:39:28,886][20381] Updated weights for policy 1, policy_version 1154765 (0.0007) [2023-12-27 13:39:28,941][20381] Updated weights for policy 1, policy_version 1154775 (0.0007) [2023-12-27 13:39:29,081][20347] Updated weights for policy 0, policy_version 1155990 (0.0009) [2023-12-27 13:39:29,132][20347] Updated weights for policy 0, policy_version 1156000 (0.0010) [2023-12-27 13:39:29,184][20347] Updated weights for policy 0, policy_version 1156010 (0.0010) [2023-12-27 13:39:29,700][20381] Updated weights for policy 1, policy_version 1154785 (0.0006) [2023-12-27 13:39:29,754][20381] Updated weights for policy 1, policy_version 1154795 (0.0007) [2023-12-27 13:39:29,810][20381] Updated weights for policy 1, policy_version 1154805 (0.0008) [2023-12-27 13:39:29,890][20347] Updated weights for policy 0, policy_version 1156020 (0.0009) [2023-12-27 13:39:29,949][20347] Updated weights for policy 0, policy_version 1156030 (0.0008) [2023-12-27 13:39:30,008][20347] Updated weights for policy 0, policy_version 1156040 (0.0006) [2023-12-27 13:39:30,545][20381] Updated weights for policy 1, policy_version 1154815 (0.0008) [2023-12-27 13:39:30,596][20381] Updated weights for policy 1, policy_version 1154825 (0.0007) [2023-12-27 13:39:30,634][20347] Updated weights for policy 0, policy_version 1156050 (0.0006) [2023-12-27 13:39:30,643][20381] Updated weights for policy 1, policy_version 1154835 (0.0006) [2023-12-27 13:39:30,681][20347] Updated weights for policy 0, policy_version 1156060 (0.0010) [2023-12-27 13:39:30,730][20347] Updated weights for policy 0, policy_version 1156070 (0.0010) [2023-12-27 13:39:30,784][20347] Updated weights for policy 0, policy_version 1156080 (0.0010) [2023-12-27 13:39:31,267][20381] Updated weights for policy 1, policy_version 1154845 (0.0006) [2023-12-27 13:39:31,327][20381] Updated weights for policy 1, policy_version 1154855 (0.0008) [2023-12-27 13:39:31,391][20381] Updated weights for policy 1, policy_version 1154865 (0.0009) [2023-12-27 13:39:31,515][20347] Updated weights for policy 0, policy_version 1156090 (0.0010) [2023-12-27 13:39:31,576][20347] Updated weights for policy 0, policy_version 1156100 (0.0010) [2023-12-27 13:39:31,637][20347] Updated weights for policy 0, policy_version 1156110 (0.0009) [2023-12-27 13:39:32,056][20381] Updated weights for policy 1, policy_version 1154875 (0.0007) [2023-12-27 13:39:32,106][20381] Updated weights for policy 1, policy_version 1154885 (0.0005) [2023-12-27 13:39:32,158][20381] Updated weights for policy 1, policy_version 1154895 (0.0005) [2023-12-27 13:39:32,315][20347] Updated weights for policy 0, policy_version 1156120 (0.0009) [2023-12-27 13:39:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 591699968. Throughput: 0: 10168.6, 1: 10325.1. Samples: 591710796. Policy #0 lag: (min: 12.0, avg: 22.8, max: 44.0) [2023-12-27 13:39:32,363][19170] Avg episode reward: [(0, '990.220'), (1, '628.050')] [2023-12-27 13:39:32,375][20347] Updated weights for policy 0, policy_version 1156130 (0.0008) [2023-12-27 13:39:32,427][20347] Updated weights for policy 0, policy_version 1156140 (0.0008) [2023-12-27 13:39:32,793][20381] Updated weights for policy 1, policy_version 1154905 (0.0005) [2023-12-27 13:39:32,859][20381] Updated weights for policy 1, policy_version 1154915 (0.0005) [2023-12-27 13:39:32,924][20381] Updated weights for policy 1, policy_version 1154925 (0.0007) [2023-12-27 13:39:32,979][20381] Updated weights for policy 1, policy_version 1154935 (0.0007) [2023-12-27 13:39:33,032][20347] Updated weights for policy 0, policy_version 1156150 (0.0007) [2023-12-27 13:39:33,080][20347] Updated weights for policy 0, policy_version 1156160 (0.0005) [2023-12-27 13:39:33,134][20347] Updated weights for policy 0, policy_version 1156170 (0.0005) [2023-12-27 13:39:33,540][20381] Updated weights for policy 1, policy_version 1154945 (0.0005) [2023-12-27 13:39:33,597][20381] Updated weights for policy 1, policy_version 1154955 (0.0005) [2023-12-27 13:39:33,643][20381] Updated weights for policy 1, policy_version 1154965 (0.0005) [2023-12-27 13:39:33,668][20347] Updated weights for policy 0, policy_version 1156180 (0.0006) [2023-12-27 13:39:33,722][20347] Updated weights for policy 0, policy_version 1156190 (0.0010) [2023-12-27 13:39:33,770][20347] Updated weights for policy 0, policy_version 1156200 (0.0010) [2023-12-27 13:39:34,192][20381] Updated weights for policy 1, policy_version 1154975 (0.0007) [2023-12-27 13:39:34,252][20381] Updated weights for policy 1, policy_version 1154985 (0.0007) [2023-12-27 13:39:34,310][20381] Updated weights for policy 1, policy_version 1154995 (0.0006) [2023-12-27 13:39:34,480][20347] Updated weights for policy 0, policy_version 1156210 (0.0010) [2023-12-27 13:39:34,536][20347] Updated weights for policy 0, policy_version 1156220 (0.0008) [2023-12-27 13:39:34,604][20347] Updated weights for policy 0, policy_version 1156230 (0.0008) [2023-12-27 13:39:34,668][20347] Updated weights for policy 0, policy_version 1156240 (0.0008) [2023-12-27 13:39:35,030][20381] Updated weights for policy 1, policy_version 1155005 (0.0009) [2023-12-27 13:39:35,093][20381] Updated weights for policy 1, policy_version 1155015 (0.0011) [2023-12-27 13:39:35,155][20381] Updated weights for policy 1, policy_version 1155025 (0.0011) [2023-12-27 13:39:35,392][20347] Updated weights for policy 0, policy_version 1156250 (0.0007) [2023-12-27 13:39:35,449][20347] Updated weights for policy 0, policy_version 1156260 (0.0009) [2023-12-27 13:39:35,507][20347] Updated weights for policy 0, policy_version 1156270 (0.0008) [2023-12-27 13:39:35,808][20381] Updated weights for policy 1, policy_version 1155035 (0.0011) [2023-12-27 13:39:35,866][20381] Updated weights for policy 1, policy_version 1155045 (0.0009) [2023-12-27 13:39:35,930][20381] Updated weights for policy 1, policy_version 1155055 (0.0005) [2023-12-27 13:39:36,218][20347] Updated weights for policy 0, policy_version 1156280 (0.0006) [2023-12-27 13:39:36,270][20347] Updated weights for policy 0, policy_version 1156290 (0.0007) [2023-12-27 13:39:36,322][20347] Updated weights for policy 0, policy_version 1156300 (0.0007) [2023-12-27 13:39:36,572][20381] Updated weights for policy 1, policy_version 1155065 (0.0006) [2023-12-27 13:39:36,629][20381] Updated weights for policy 1, policy_version 1155075 (0.0007) [2023-12-27 13:39:36,685][20381] Updated weights for policy 1, policy_version 1155085 (0.0008) [2023-12-27 13:39:36,741][20381] Updated weights for policy 1, policy_version 1155095 (0.0008) [2023-12-27 13:39:37,065][20347] Updated weights for policy 0, policy_version 1156310 (0.0008) [2023-12-27 13:39:37,115][20347] Updated weights for policy 0, policy_version 1156320 (0.0006) [2023-12-27 13:39:37,165][20347] Updated weights for policy 0, policy_version 1156330 (0.0008) [2023-12-27 13:39:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 591806464. Throughput: 0: 10220.5, 1: 10359.1. Samples: 591775740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:39:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '637.640')] [2023-12-27 13:39:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001156336_296075264.pth... [2023-12-27 13:39:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001155152_295772160.pth [2023-12-27 13:39:37,392][20381] Updated weights for policy 1, policy_version 1155105 (0.0008) [2023-12-27 13:39:37,442][20381] Updated weights for policy 1, policy_version 1155115 (0.0009) [2023-12-27 13:39:37,489][20381] Updated weights for policy 1, policy_version 1155125 (0.0007) [2023-12-27 13:39:37,500][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001155128_295739392.pth... [2023-12-27 13:39:37,503][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001153912_295428096.pth [2023-12-27 13:39:37,885][20347] Updated weights for policy 0, policy_version 1156340 (0.0008) [2023-12-27 13:39:37,933][20347] Updated weights for policy 0, policy_version 1156350 (0.0008) [2023-12-27 13:39:37,977][20347] Updated weights for policy 0, policy_version 1156360 (0.0008) [2023-12-27 13:39:38,042][20381] Updated weights for policy 1, policy_version 1155135 (0.0007) [2023-12-27 13:39:38,100][20381] Updated weights for policy 1, policy_version 1155145 (0.0007) [2023-12-27 13:39:38,162][20381] Updated weights for policy 1, policy_version 1155155 (0.0008) [2023-12-27 13:39:38,730][20347] Updated weights for policy 0, policy_version 1156370 (0.0007) [2023-12-27 13:39:38,789][20347] Updated weights for policy 0, policy_version 1156380 (0.0008) [2023-12-27 13:39:38,803][20381] Updated weights for policy 1, policy_version 1155165 (0.0007) [2023-12-27 13:39:38,843][20347] Updated weights for policy 0, policy_version 1156390 (0.0006) [2023-12-27 13:39:38,852][20381] Updated weights for policy 1, policy_version 1155175 (0.0006) [2023-12-27 13:39:38,867][20292] KL-divergence is very high: 111.7887 [2023-12-27 13:39:38,899][20347] Updated weights for policy 0, policy_version 1156400 (0.0007) [2023-12-27 13:39:38,905][20381] Updated weights for policy 1, policy_version 1155185 (0.0006) [2023-12-27 13:39:39,589][20381] Updated weights for policy 1, policy_version 1155195 (0.0008) [2023-12-27 13:39:39,648][20381] Updated weights for policy 1, policy_version 1155205 (0.0006) [2023-12-27 13:39:39,669][20347] Updated weights for policy 0, policy_version 1156410 (0.0006) [2023-12-27 13:39:39,710][20381] Updated weights for policy 1, policy_version 1155215 (0.0006) [2023-12-27 13:39:39,727][20347] Updated weights for policy 0, policy_version 1156420 (0.0006) [2023-12-27 13:39:39,791][20347] Updated weights for policy 0, policy_version 1156430 (0.0006) [2023-12-27 13:39:40,452][20381] Updated weights for policy 1, policy_version 1155225 (0.0006) [2023-12-27 13:39:40,501][20347] Updated weights for policy 0, policy_version 1156440 (0.0007) [2023-12-27 13:39:40,511][20381] Updated weights for policy 1, policy_version 1155235 (0.0006) [2023-12-27 13:39:40,557][20347] Updated weights for policy 0, policy_version 1156450 (0.0006) [2023-12-27 13:39:40,572][20381] Updated weights for policy 1, policy_version 1155245 (0.0007) [2023-12-27 13:39:40,617][20347] Updated weights for policy 0, policy_version 1156460 (0.0006) [2023-12-27 13:39:40,631][20381] Updated weights for policy 1, policy_version 1155255 (0.0007) [2023-12-27 13:39:41,286][20347] Updated weights for policy 0, policy_version 1156470 (0.0008) [2023-12-27 13:39:41,321][20381] Updated weights for policy 1, policy_version 1155265 (0.0006) [2023-12-27 13:39:41,349][20347] Updated weights for policy 0, policy_version 1156480 (0.0007) [2023-12-27 13:39:41,388][20381] Updated weights for policy 1, policy_version 1155275 (0.0009) [2023-12-27 13:39:41,409][20347] Updated weights for policy 0, policy_version 1156490 (0.0008) [2023-12-27 13:39:41,446][20381] Updated weights for policy 1, policy_version 1155285 (0.0006) [2023-12-27 13:39:42,098][20347] Updated weights for policy 0, policy_version 1156500 (0.0009) [2023-12-27 13:39:42,157][20347] Updated weights for policy 0, policy_version 1156510 (0.0007) [2023-12-27 13:39:42,219][20381] Updated weights for policy 1, policy_version 1155295 (0.0007) [2023-12-27 13:39:42,220][20347] Updated weights for policy 0, policy_version 1156520 (0.0006) [2023-12-27 13:39:42,280][20381] Updated weights for policy 1, policy_version 1155305 (0.0006) [2023-12-27 13:39:42,338][20381] Updated weights for policy 1, policy_version 1155315 (0.0008) [2023-12-27 13:39:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 591904768. Throughput: 0: 10134.8, 1: 10376.8. Samples: 591896844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:39:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '618.360')] [2023-12-27 13:39:42,849][20347] Updated weights for policy 0, policy_version 1156530 (0.0009) [2023-12-27 13:39:42,918][20347] Updated weights for policy 0, policy_version 1156540 (0.0010) [2023-12-27 13:39:42,983][20347] Updated weights for policy 0, policy_version 1156550 (0.0011) [2023-12-27 13:39:43,045][20347] Updated weights for policy 0, policy_version 1156560 (0.0009) [2023-12-27 13:39:43,073][20381] Updated weights for policy 1, policy_version 1155325 (0.0008) [2023-12-27 13:39:43,139][20381] Updated weights for policy 1, policy_version 1155335 (0.0008) [2023-12-27 13:39:43,202][20381] Updated weights for policy 1, policy_version 1155345 (0.0007) [2023-12-27 13:39:43,669][20347] Updated weights for policy 0, policy_version 1156570 (0.0011) [2023-12-27 13:39:43,717][20347] Updated weights for policy 0, policy_version 1156580 (0.0008) [2023-12-27 13:39:43,768][20347] Updated weights for policy 0, policy_version 1156590 (0.0007) [2023-12-27 13:39:43,912][20381] Updated weights for policy 1, policy_version 1155355 (0.0008) [2023-12-27 13:39:43,968][20381] Updated weights for policy 1, policy_version 1155365 (0.0008) [2023-12-27 13:39:44,022][20381] Updated weights for policy 1, policy_version 1155375 (0.0008) [2023-12-27 13:39:44,472][20347] Updated weights for policy 0, policy_version 1156600 (0.0008) [2023-12-27 13:39:44,529][20347] Updated weights for policy 0, policy_version 1156610 (0.0007) [2023-12-27 13:39:44,591][20347] Updated weights for policy 0, policy_version 1156620 (0.0008) [2023-12-27 13:39:44,727][20381] Updated weights for policy 1, policy_version 1155385 (0.0007) [2023-12-27 13:39:44,783][20381] Updated weights for policy 1, policy_version 1155395 (0.0008) [2023-12-27 13:39:44,839][20381] Updated weights for policy 1, policy_version 1155405 (0.0007) [2023-12-27 13:39:44,899][20381] Updated weights for policy 1, policy_version 1155415 (0.0008) [2023-12-27 13:39:45,287][20347] Updated weights for policy 0, policy_version 1156630 (0.0007) [2023-12-27 13:39:45,354][20347] Updated weights for policy 0, policy_version 1156640 (0.0007) [2023-12-27 13:39:45,405][20347] Updated weights for policy 0, policy_version 1156650 (0.0007) [2023-12-27 13:39:45,635][20381] Updated weights for policy 1, policy_version 1155425 (0.0005) [2023-12-27 13:39:45,701][20381] Updated weights for policy 1, policy_version 1155435 (0.0005) [2023-12-27 13:39:45,752][20381] Updated weights for policy 1, policy_version 1155445 (0.0005) [2023-12-27 13:39:46,059][20347] Updated weights for policy 0, policy_version 1156660 (0.0006) [2023-12-27 13:39:46,109][20347] Updated weights for policy 0, policy_version 1156670 (0.0008) [2023-12-27 13:39:46,155][20347] Updated weights for policy 0, policy_version 1156680 (0.0008) [2023-12-27 13:39:46,308][20381] Updated weights for policy 1, policy_version 1155455 (0.0007) [2023-12-27 13:39:46,356][20381] Updated weights for policy 1, policy_version 1155465 (0.0007) [2023-12-27 13:39:46,405][20381] Updated weights for policy 1, policy_version 1155475 (0.0007) [2023-12-27 13:39:46,806][20347] Updated weights for policy 0, policy_version 1156690 (0.0008) [2023-12-27 13:39:46,866][20347] Updated weights for policy 0, policy_version 1156700 (0.0007) [2023-12-27 13:39:46,921][20347] Updated weights for policy 0, policy_version 1156710 (0.0005) [2023-12-27 13:39:46,967][20347] Updated weights for policy 0, policy_version 1156720 (0.0005) [2023-12-27 13:39:47,134][20381] Updated weights for policy 1, policy_version 1155485 (0.0009) [2023-12-27 13:39:47,192][20381] Updated weights for policy 1, policy_version 1155495 (0.0010) [2023-12-27 13:39:47,257][20381] Updated weights for policy 1, policy_version 1155505 (0.0011) [2023-12-27 13:39:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 592011264. Throughput: 0: 10162.3, 1: 10314.0. Samples: 592017924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:39:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '725.880')] [2023-12-27 13:39:47,613][20347] Updated weights for policy 0, policy_version 1156730 (0.0007) [2023-12-27 13:39:47,658][20347] Updated weights for policy 0, policy_version 1156740 (0.0006) [2023-12-27 13:39:47,702][20347] Updated weights for policy 0, policy_version 1156750 (0.0007) [2023-12-27 13:39:47,925][20381] Updated weights for policy 1, policy_version 1155515 (0.0010) [2023-12-27 13:39:47,985][20381] Updated weights for policy 1, policy_version 1155525 (0.0010) [2023-12-27 13:39:48,044][20381] Updated weights for policy 1, policy_version 1155535 (0.0010) [2023-12-27 13:39:48,322][20347] Updated weights for policy 0, policy_version 1156760 (0.0006) [2023-12-27 13:39:48,392][20347] Updated weights for policy 0, policy_version 1156770 (0.0008) [2023-12-27 13:39:48,459][20347] Updated weights for policy 0, policy_version 1156780 (0.0008) [2023-12-27 13:39:48,691][20381] Updated weights for policy 1, policy_version 1155545 (0.0010) [2023-12-27 13:39:48,752][20381] Updated weights for policy 1, policy_version 1155555 (0.0007) [2023-12-27 13:39:48,808][20381] Updated weights for policy 1, policy_version 1155565 (0.0010) [2023-12-27 13:39:48,873][20381] Updated weights for policy 1, policy_version 1155575 (0.0011) [2023-12-27 13:39:49,056][20347] Updated weights for policy 0, policy_version 1156790 (0.0006) [2023-12-27 13:39:49,110][20347] Updated weights for policy 0, policy_version 1156800 (0.0005) [2023-12-27 13:39:49,162][20347] Updated weights for policy 0, policy_version 1156810 (0.0005) [2023-12-27 13:39:49,598][20381] Updated weights for policy 1, policy_version 1155585 (0.0010) [2023-12-27 13:39:49,660][20381] Updated weights for policy 1, policy_version 1155595 (0.0009) [2023-12-27 13:39:49,711][20381] Updated weights for policy 1, policy_version 1155605 (0.0005) [2023-12-27 13:39:49,820][20347] Updated weights for policy 0, policy_version 1156820 (0.0007) [2023-12-27 13:39:49,884][20347] Updated weights for policy 0, policy_version 1156830 (0.0007) [2023-12-27 13:39:49,948][20347] Updated weights for policy 0, policy_version 1156840 (0.0008) [2023-12-27 13:39:50,468][20381] Updated weights for policy 1, policy_version 1155615 (0.0008) [2023-12-27 13:39:50,521][20381] Updated weights for policy 1, policy_version 1155625 (0.0008) [2023-12-27 13:39:50,579][20381] Updated weights for policy 1, policy_version 1155635 (0.0008) [2023-12-27 13:39:50,631][20347] Updated weights for policy 0, policy_version 1156850 (0.0007) [2023-12-27 13:39:50,693][20347] Updated weights for policy 0, policy_version 1156860 (0.0008) [2023-12-27 13:39:50,751][20347] Updated weights for policy 0, policy_version 1156870 (0.0006) [2023-12-27 13:39:50,803][20347] Updated weights for policy 0, policy_version 1156880 (0.0006) [2023-12-27 13:39:51,323][20381] Updated weights for policy 1, policy_version 1155645 (0.0008) [2023-12-27 13:39:51,385][20381] Updated weights for policy 1, policy_version 1155655 (0.0008) [2023-12-27 13:39:51,454][20381] Updated weights for policy 1, policy_version 1155665 (0.0008) [2023-12-27 13:39:51,503][20347] Updated weights for policy 0, policy_version 1156890 (0.0006) [2023-12-27 13:39:51,554][20347] Updated weights for policy 0, policy_version 1156900 (0.0008) [2023-12-27 13:39:51,619][20347] Updated weights for policy 0, policy_version 1156910 (0.0008) [2023-12-27 13:39:52,131][20381] Updated weights for policy 1, policy_version 1155675 (0.0007) [2023-12-27 13:39:52,180][20381] Updated weights for policy 1, policy_version 1155685 (0.0007) [2023-12-27 13:39:52,229][20381] Updated weights for policy 1, policy_version 1155695 (0.0008) [2023-12-27 13:39:52,354][20347] Updated weights for policy 0, policy_version 1156920 (0.0010) [2023-12-27 13:39:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 592109568. Throughput: 0: 10212.1, 1: 10272.3. Samples: 592080068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:39:52,363][19170] Avg episode reward: [(0, '990.240'), (1, '716.130')] [2023-12-27 13:39:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001155704_295886848.pth... [2023-12-27 13:39:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001154488_295575552.pth [2023-12-27 13:39:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001155704_295886848.pth [2023-12-27 13:39:52,413][20347] Updated weights for policy 0, policy_version 1156930 (0.0011) [2023-12-27 13:39:52,476][20347] Updated weights for policy 0, policy_version 1156940 (0.0009) [2023-12-27 13:39:52,498][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001156944_296230912.pth... [2023-12-27 13:39:52,501][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001155728_295919616.pth [2023-12-27 13:39:52,501][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001156944_296230912.pth [2023-12-27 13:39:52,972][20381] Updated weights for policy 1, policy_version 1155705 (0.0008) [2023-12-27 13:39:53,026][20381] Updated weights for policy 1, policy_version 1155715 (0.0010) [2023-12-27 13:39:53,077][20381] Updated weights for policy 1, policy_version 1155725 (0.0009) [2023-12-27 13:39:53,137][20381] Updated weights for policy 1, policy_version 1155735 (0.0009) [2023-12-27 13:39:53,156][20347] Updated weights for policy 0, policy_version 1156950 (0.0010) [2023-12-27 13:39:53,198][20347] Updated weights for policy 0, policy_version 1156960 (0.0010) [2023-12-27 13:39:53,246][20347] Updated weights for policy 0, policy_version 1156970 (0.0010) [2023-12-27 13:39:53,726][20381] Updated weights for policy 1, policy_version 1155745 (0.0010) [2023-12-27 13:39:53,788][20381] Updated weights for policy 1, policy_version 1155755 (0.0009) [2023-12-27 13:39:53,846][20381] Updated weights for policy 1, policy_version 1155765 (0.0006) [2023-12-27 13:39:53,927][20347] Updated weights for policy 0, policy_version 1156980 (0.0010) [2023-12-27 13:39:53,982][20347] Updated weights for policy 0, policy_version 1156990 (0.0010) [2023-12-27 13:39:54,042][20347] Updated weights for policy 0, policy_version 1157000 (0.0010) [2023-12-27 13:39:54,384][20381] Updated weights for policy 1, policy_version 1155775 (0.0005) [2023-12-27 13:39:54,429][20381] Updated weights for policy 1, policy_version 1155785 (0.0005) [2023-12-27 13:39:54,481][20381] Updated weights for policy 1, policy_version 1155795 (0.0007) [2023-12-27 13:39:54,674][20347] Updated weights for policy 0, policy_version 1157010 (0.0010) [2023-12-27 13:39:54,736][20347] Updated weights for policy 0, policy_version 1157020 (0.0008) [2023-12-27 13:39:54,794][20347] Updated weights for policy 0, policy_version 1157030 (0.0006) [2023-12-27 13:39:54,840][20347] Updated weights for policy 0, policy_version 1157040 (0.0005) [2023-12-27 13:39:55,105][20381] Updated weights for policy 1, policy_version 1155805 (0.0008) [2023-12-27 13:39:55,159][20381] Updated weights for policy 1, policy_version 1155815 (0.0007) [2023-12-27 13:39:55,217][20381] Updated weights for policy 1, policy_version 1155825 (0.0008) [2023-12-27 13:39:55,533][20347] Updated weights for policy 0, policy_version 1157050 (0.0008) [2023-12-27 13:39:55,588][20347] Updated weights for policy 0, policy_version 1157060 (0.0008) [2023-12-27 13:39:55,639][20347] Updated weights for policy 0, policy_version 1157070 (0.0008) [2023-12-27 13:39:55,917][20381] Updated weights for policy 1, policy_version 1155835 (0.0006) [2023-12-27 13:39:55,967][20381] Updated weights for policy 1, policy_version 1155845 (0.0009) [2023-12-27 13:39:56,013][20381] Updated weights for policy 1, policy_version 1155855 (0.0005) [2023-12-27 13:39:56,228][20347] Updated weights for policy 0, policy_version 1157080 (0.0010) [2023-12-27 13:39:56,276][20347] Updated weights for policy 0, policy_version 1157090 (0.0007) [2023-12-27 13:39:56,337][20347] Updated weights for policy 0, policy_version 1157100 (0.0005) [2023-12-27 13:39:56,667][20381] Updated weights for policy 1, policy_version 1155865 (0.0006) [2023-12-27 13:39:56,720][20381] Updated weights for policy 1, policy_version 1155875 (0.0009) [2023-12-27 13:39:56,774][20381] Updated weights for policy 1, policy_version 1155885 (0.0010) [2023-12-27 13:39:56,831][20381] Updated weights for policy 1, policy_version 1155895 (0.0010) [2023-12-27 13:39:56,921][20347] Updated weights for policy 0, policy_version 1157110 (0.0008) [2023-12-27 13:39:56,969][20347] Updated weights for policy 0, policy_version 1157120 (0.0010) [2023-12-27 13:39:57,024][20347] Updated weights for policy 0, policy_version 1157130 (0.0010) [2023-12-27 13:39:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 592216064. Throughput: 0: 10224.0, 1: 10268.9. Samples: 592204532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:39:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '687.000')] [2023-12-27 13:39:57,434][20381] Updated weights for policy 1, policy_version 1155905 (0.0010) [2023-12-27 13:39:57,491][20381] Updated weights for policy 1, policy_version 1155915 (0.0010) [2023-12-27 13:39:57,541][20381] Updated weights for policy 1, policy_version 1155925 (0.0010) [2023-12-27 13:39:57,669][20347] Updated weights for policy 0, policy_version 1157140 (0.0010) [2023-12-27 13:39:57,713][20347] Updated weights for policy 0, policy_version 1157150 (0.0010) [2023-12-27 13:39:57,766][20347] Updated weights for policy 0, policy_version 1157160 (0.0010) [2023-12-27 13:39:58,149][20381] Updated weights for policy 1, policy_version 1155935 (0.0008) [2023-12-27 13:39:58,216][20381] Updated weights for policy 1, policy_version 1155945 (0.0006) [2023-12-27 13:39:58,272][20381] Updated weights for policy 1, policy_version 1155955 (0.0007) [2023-12-27 13:39:58,493][20347] Updated weights for policy 0, policy_version 1157170 (0.0009) [2023-12-27 13:39:58,559][20347] Updated weights for policy 0, policy_version 1157180 (0.0008) [2023-12-27 13:39:58,623][20347] Updated weights for policy 0, policy_version 1157190 (0.0008) [2023-12-27 13:39:58,686][20347] Updated weights for policy 0, policy_version 1157200 (0.0008) [2023-12-27 13:39:59,068][20381] Updated weights for policy 1, policy_version 1155965 (0.0008) [2023-12-27 13:39:59,121][20381] Updated weights for policy 1, policy_version 1155975 (0.0008) [2023-12-27 13:39:59,206][20381] Updated weights for policy 1, policy_version 1155985 (0.0006) [2023-12-27 13:39:59,528][20347] Updated weights for policy 0, policy_version 1157210 (0.0008) [2023-12-27 13:39:59,588][20347] Updated weights for policy 0, policy_version 1157220 (0.0008) [2023-12-27 13:39:59,646][20347] Updated weights for policy 0, policy_version 1157230 (0.0008) [2023-12-27 13:39:59,892][20381] Updated weights for policy 1, policy_version 1155995 (0.0010) [2023-12-27 13:39:59,952][20381] Updated weights for policy 1, policy_version 1156005 (0.0009) [2023-12-27 13:40:00,013][20381] Updated weights for policy 1, policy_version 1156015 (0.0007) [2023-12-27 13:40:00,365][20347] Updated weights for policy 0, policy_version 1157240 (0.0006) [2023-12-27 13:40:00,416][20347] Updated weights for policy 0, policy_version 1157250 (0.0005) [2023-12-27 13:40:00,475][20347] Updated weights for policy 0, policy_version 1157260 (0.0005) [2023-12-27 13:40:00,616][20381] Updated weights for policy 1, policy_version 1156025 (0.0006) [2023-12-27 13:40:00,666][20381] Updated weights for policy 1, policy_version 1156035 (0.0007) [2023-12-27 13:40:00,717][20381] Updated weights for policy 1, policy_version 1156045 (0.0008) [2023-12-27 13:40:00,778][20381] Updated weights for policy 1, policy_version 1156055 (0.0007) [2023-12-27 13:40:01,036][20347] Updated weights for policy 0, policy_version 1157270 (0.0005) [2023-12-27 13:40:01,097][20347] Updated weights for policy 0, policy_version 1157280 (0.0008) [2023-12-27 13:40:01,160][20347] Updated weights for policy 0, policy_version 1157290 (0.0008) [2023-12-27 13:40:01,487][20381] Updated weights for policy 1, policy_version 1156065 (0.0007) [2023-12-27 13:40:01,544][20381] Updated weights for policy 1, policy_version 1156075 (0.0006) [2023-12-27 13:40:01,600][20381] Updated weights for policy 1, policy_version 1156085 (0.0005) [2023-12-27 13:40:01,898][20347] Updated weights for policy 0, policy_version 1157300 (0.0008) [2023-12-27 13:40:01,955][20347] Updated weights for policy 0, policy_version 1157310 (0.0008) [2023-12-27 13:40:02,009][20347] Updated weights for policy 0, policy_version 1157320 (0.0008) [2023-12-27 13:40:02,235][20381] Updated weights for policy 1, policy_version 1156095 (0.0006) [2023-12-27 13:40:02,295][20381] Updated weights for policy 1, policy_version 1156105 (0.0007) [2023-12-27 13:40:02,359][20381] Updated weights for policy 1, policy_version 1156115 (0.0008) [2023-12-27 13:40:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 592314368. Throughput: 0: 10226.8, 1: 10252.8. Samples: 592327376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '706.360')] [2023-12-27 13:40:02,756][20347] Updated weights for policy 0, policy_version 1157330 (0.0008) [2023-12-27 13:40:02,806][20347] Updated weights for policy 0, policy_version 1157340 (0.0007) [2023-12-27 13:40:02,857][20347] Updated weights for policy 0, policy_version 1157350 (0.0007) [2023-12-27 13:40:02,913][20347] Updated weights for policy 0, policy_version 1157360 (0.0008) [2023-12-27 13:40:03,035][20381] Updated weights for policy 1, policy_version 1156125 (0.0008) [2023-12-27 13:40:03,089][20381] Updated weights for policy 1, policy_version 1156135 (0.0008) [2023-12-27 13:40:03,137][20381] Updated weights for policy 1, policy_version 1156145 (0.0007) [2023-12-27 13:40:03,601][20347] Updated weights for policy 0, policy_version 1157370 (0.0007) [2023-12-27 13:40:03,649][20347] Updated weights for policy 0, policy_version 1157380 (0.0007) [2023-12-27 13:40:03,706][20347] Updated weights for policy 0, policy_version 1157390 (0.0008) [2023-12-27 13:40:03,786][20381] Updated weights for policy 1, policy_version 1156155 (0.0007) [2023-12-27 13:40:03,847][20381] Updated weights for policy 1, policy_version 1156165 (0.0007) [2023-12-27 13:40:03,904][20381] Updated weights for policy 1, policy_version 1156175 (0.0007) [2023-12-27 13:40:04,423][20347] Updated weights for policy 0, policy_version 1157400 (0.0008) [2023-12-27 13:40:04,478][20347] Updated weights for policy 0, policy_version 1157410 (0.0008) [2023-12-27 13:40:04,527][20347] Updated weights for policy 0, policy_version 1157420 (0.0007) [2023-12-27 13:40:04,598][20381] Updated weights for policy 1, policy_version 1156185 (0.0008) [2023-12-27 13:40:04,656][20381] Updated weights for policy 1, policy_version 1156195 (0.0008) [2023-12-27 13:40:04,702][20381] Updated weights for policy 1, policy_version 1156205 (0.0008) [2023-12-27 13:40:04,750][20381] Updated weights for policy 1, policy_version 1156215 (0.0007) [2023-12-27 13:40:05,229][20347] Updated weights for policy 0, policy_version 1157430 (0.0007) [2023-12-27 13:40:05,293][20347] Updated weights for policy 0, policy_version 1157440 (0.0008) [2023-12-27 13:40:05,345][20347] Updated weights for policy 0, policy_version 1157450 (0.0007) [2023-12-27 13:40:05,443][20381] Updated weights for policy 1, policy_version 1156225 (0.0008) [2023-12-27 13:40:05,509][20381] Updated weights for policy 1, policy_version 1156235 (0.0008) [2023-12-27 13:40:05,556][20381] Updated weights for policy 1, policy_version 1156245 (0.0007) [2023-12-27 13:40:06,026][20347] Updated weights for policy 0, policy_version 1157460 (0.0008) [2023-12-27 13:40:06,084][20347] Updated weights for policy 0, policy_version 1157470 (0.0008) [2023-12-27 13:40:06,159][20347] Updated weights for policy 0, policy_version 1157480 (0.0007) [2023-12-27 13:40:06,278][20381] Updated weights for policy 1, policy_version 1156255 (0.0008) [2023-12-27 13:40:06,337][20381] Updated weights for policy 1, policy_version 1156265 (0.0007) [2023-12-27 13:40:06,395][20381] Updated weights for policy 1, policy_version 1156275 (0.0008) [2023-12-27 13:40:06,856][20347] Updated weights for policy 0, policy_version 1157490 (0.0007) [2023-12-27 13:40:06,921][20347] Updated weights for policy 0, policy_version 1157500 (0.0008) [2023-12-27 13:40:06,984][20347] Updated weights for policy 0, policy_version 1157510 (0.0007) [2023-12-27 13:40:07,044][20347] Updated weights for policy 0, policy_version 1157520 (0.0008) [2023-12-27 13:40:07,101][20381] Updated weights for policy 1, policy_version 1156285 (0.0009) [2023-12-27 13:40:07,158][20381] Updated weights for policy 1, policy_version 1156295 (0.0008) [2023-12-27 13:40:07,214][20381] Updated weights for policy 1, policy_version 1156305 (0.0007) [2023-12-27 13:40:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 592420864. Throughput: 0: 10212.1, 1: 10270.5. Samples: 592387920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '657.430')] [2023-12-27 13:40:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001156312_296042496.pth... [2023-12-27 13:40:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001157520_296378368.pth... [2023-12-27 13:40:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001155128_295739392.pth [2023-12-27 13:40:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001156336_296075264.pth [2023-12-27 13:40:07,685][20347] Updated weights for policy 0, policy_version 1157530 (0.0011) [2023-12-27 13:40:07,744][20347] Updated weights for policy 0, policy_version 1157540 (0.0011) [2023-12-27 13:40:07,804][20347] Updated weights for policy 0, policy_version 1157550 (0.0011) [2023-12-27 13:40:07,916][20381] Updated weights for policy 1, policy_version 1156315 (0.0007) [2023-12-27 13:40:07,972][20381] Updated weights for policy 1, policy_version 1156325 (0.0006) [2023-12-27 13:40:08,021][20381] Updated weights for policy 1, policy_version 1156335 (0.0007) [2023-12-27 13:40:08,445][20347] Updated weights for policy 0, policy_version 1157560 (0.0009) [2023-12-27 13:40:08,508][20347] Updated weights for policy 0, policy_version 1157570 (0.0008) [2023-12-27 13:40:08,566][20347] Updated weights for policy 0, policy_version 1157580 (0.0007) [2023-12-27 13:40:08,699][20381] Updated weights for policy 1, policy_version 1156345 (0.0008) [2023-12-27 13:40:08,751][20381] Updated weights for policy 1, policy_version 1156355 (0.0008) [2023-12-27 13:40:08,799][20381] Updated weights for policy 1, policy_version 1156365 (0.0007) [2023-12-27 13:40:08,848][20381] Updated weights for policy 1, policy_version 1156375 (0.0007) [2023-12-27 13:40:09,276][20347] Updated weights for policy 0, policy_version 1157590 (0.0008) [2023-12-27 13:40:09,335][20347] Updated weights for policy 0, policy_version 1157600 (0.0010) [2023-12-27 13:40:09,371][20291] KL-divergence is very high: 195.7305 [2023-12-27 13:40:09,379][20291] KL-divergence is very high: 135.6391 [2023-12-27 13:40:09,406][20347] Updated weights for policy 0, policy_version 1157610 (0.0009) [2023-12-27 13:40:09,429][20291] KL-divergence is very high: 150.3410 [2023-12-27 13:40:09,557][20381] Updated weights for policy 1, policy_version 1156385 (0.0007) [2023-12-27 13:40:09,617][20381] Updated weights for policy 1, policy_version 1156395 (0.0008) [2023-12-27 13:40:09,678][20381] Updated weights for policy 1, policy_version 1156405 (0.0008) [2023-12-27 13:40:10,151][20347] Updated weights for policy 0, policy_version 1157620 (0.0009) [2023-12-27 13:40:10,214][20347] Updated weights for policy 0, policy_version 1157630 (0.0007) [2023-12-27 13:40:10,271][20347] Updated weights for policy 0, policy_version 1157640 (0.0008) [2023-12-27 13:40:10,437][20381] Updated weights for policy 1, policy_version 1156415 (0.0008) [2023-12-27 13:40:10,469][20292] KL-divergence is very high: 229.2088 [2023-12-27 13:40:10,482][20292] KL-divergence is very high: 257.7675 [2023-12-27 13:40:10,500][20381] Updated weights for policy 1, policy_version 1156425 (0.0007) [2023-12-27 13:40:10,556][20381] Updated weights for policy 1, policy_version 1156435 (0.0008) [2023-12-27 13:40:10,960][20347] Updated weights for policy 0, policy_version 1157650 (0.0008) [2023-12-27 13:40:11,007][20347] Updated weights for policy 0, policy_version 1157660 (0.0007) [2023-12-27 13:40:11,074][20347] Updated weights for policy 0, policy_version 1157670 (0.0008) [2023-12-27 13:40:11,133][20347] Updated weights for policy 0, policy_version 1157680 (0.0008) [2023-12-27 13:40:11,292][20381] Updated weights for policy 1, policy_version 1156445 (0.0008) [2023-12-27 13:40:11,362][20381] Updated weights for policy 1, policy_version 1156455 (0.0008) [2023-12-27 13:40:11,427][20381] Updated weights for policy 1, policy_version 1156465 (0.0009) [2023-12-27 13:40:11,823][20347] Updated weights for policy 0, policy_version 1157690 (0.0007) [2023-12-27 13:40:11,873][20347] Updated weights for policy 0, policy_version 1157700 (0.0007) [2023-12-27 13:40:11,930][20347] Updated weights for policy 0, policy_version 1157710 (0.0008) [2023-12-27 13:40:12,141][20381] Updated weights for policy 1, policy_version 1156475 (0.0007) [2023-12-27 13:40:12,200][20381] Updated weights for policy 1, policy_version 1156485 (0.0008) [2023-12-27 13:40:12,259][20381] Updated weights for policy 1, policy_version 1156495 (0.0007) [2023-12-27 13:40:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 592519168. Throughput: 0: 10179.1, 1: 10229.6. Samples: 592506344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:12,363][19170] Avg episode reward: [(0, '990.240'), (1, '725.980')] [2023-12-27 13:40:12,689][20347] Updated weights for policy 0, policy_version 1157720 (0.0008) [2023-12-27 13:40:12,748][20347] Updated weights for policy 0, policy_version 1157730 (0.0008) [2023-12-27 13:40:12,801][20347] Updated weights for policy 0, policy_version 1157740 (0.0007) [2023-12-27 13:40:12,960][20381] Updated weights for policy 1, policy_version 1156505 (0.0007) [2023-12-27 13:40:13,006][20381] Updated weights for policy 1, policy_version 1156515 (0.0007) [2023-12-27 13:40:13,056][20381] Updated weights for policy 1, policy_version 1156525 (0.0008) [2023-12-27 13:40:13,106][20381] Updated weights for policy 1, policy_version 1156535 (0.0008) [2023-12-27 13:40:13,500][20347] Updated weights for policy 0, policy_version 1157750 (0.0007) [2023-12-27 13:40:13,551][20347] Updated weights for policy 0, policy_version 1157760 (0.0007) [2023-12-27 13:40:13,607][20347] Updated weights for policy 0, policy_version 1157770 (0.0008) [2023-12-27 13:40:13,795][20381] Updated weights for policy 1, policy_version 1156545 (0.0006) [2023-12-27 13:40:13,853][20381] Updated weights for policy 1, policy_version 1156555 (0.0005) [2023-12-27 13:40:13,905][20381] Updated weights for policy 1, policy_version 1156565 (0.0006) [2023-12-27 13:40:14,374][20347] Updated weights for policy 0, policy_version 1157780 (0.0006) [2023-12-27 13:40:14,418][20291] KL-divergence is very high: 280.8150 [2023-12-27 13:40:14,432][20347] Updated weights for policy 0, policy_version 1157790 (0.0007) [2023-12-27 13:40:14,462][20291] KL-divergence is very high: 120.4569 [2023-12-27 13:40:14,482][20347] Updated weights for policy 0, policy_version 1157800 (0.0007) [2023-12-27 13:40:14,544][20381] Updated weights for policy 1, policy_version 1156575 (0.0007) [2023-12-27 13:40:14,605][20381] Updated weights for policy 1, policy_version 1156585 (0.0007) [2023-12-27 13:40:14,666][20381] Updated weights for policy 1, policy_version 1156595 (0.0005) [2023-12-27 13:40:15,154][20347] Updated weights for policy 0, policy_version 1157810 (0.0007) [2023-12-27 13:40:15,209][20347] Updated weights for policy 0, policy_version 1157820 (0.0005) [2023-12-27 13:40:15,260][20347] Updated weights for policy 0, policy_version 1157830 (0.0007) [2023-12-27 13:40:15,311][20381] Updated weights for policy 1, policy_version 1156605 (0.0008) [2023-12-27 13:40:15,317][20347] Updated weights for policy 0, policy_version 1157840 (0.0006) [2023-12-27 13:40:15,360][20381] Updated weights for policy 1, policy_version 1156615 (0.0010) [2023-12-27 13:40:15,412][20381] Updated weights for policy 1, policy_version 1156625 (0.0010) [2023-12-27 13:40:15,970][20347] Updated weights for policy 0, policy_version 1157850 (0.0008) [2023-12-27 13:40:16,015][20381] Updated weights for policy 1, policy_version 1156635 (0.0008) [2023-12-27 13:40:16,021][20347] Updated weights for policy 0, policy_version 1157860 (0.0007) [2023-12-27 13:40:16,071][20381] Updated weights for policy 1, policy_version 1156645 (0.0006) [2023-12-27 13:40:16,073][20347] Updated weights for policy 0, policy_version 1157870 (0.0007) [2023-12-27 13:40:16,131][20381] Updated weights for policy 1, policy_version 1156655 (0.0008) [2023-12-27 13:40:16,704][20381] Updated weights for policy 1, policy_version 1156665 (0.0007) [2023-12-27 13:40:16,755][20381] Updated weights for policy 1, policy_version 1156675 (0.0006) [2023-12-27 13:40:16,763][20347] Updated weights for policy 0, policy_version 1157880 (0.0006) [2023-12-27 13:40:16,810][20381] Updated weights for policy 1, policy_version 1156685 (0.0009) [2023-12-27 13:40:16,818][20347] Updated weights for policy 0, policy_version 1157890 (0.0005) [2023-12-27 13:40:16,866][20381] Updated weights for policy 1, policy_version 1156695 (0.0011) [2023-12-27 13:40:16,871][20347] Updated weights for policy 0, policy_version 1157900 (0.0005) [2023-12-27 13:40:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 592617472. Throughput: 0: 10134.8, 1: 10286.2. Samples: 592629744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '755.500')] [2023-12-27 13:40:17,440][20381] Updated weights for policy 1, policy_version 1156705 (0.0006) [2023-12-27 13:40:17,490][20381] Updated weights for policy 1, policy_version 1156715 (0.0006) [2023-12-27 13:40:17,548][20381] Updated weights for policy 1, policy_version 1156725 (0.0010) [2023-12-27 13:40:17,563][20347] Updated weights for policy 0, policy_version 1157910 (0.0006) [2023-12-27 13:40:17,609][20347] Updated weights for policy 0, policy_version 1157920 (0.0006) [2023-12-27 13:40:17,665][20347] Updated weights for policy 0, policy_version 1157930 (0.0006) [2023-12-27 13:40:18,175][20381] Updated weights for policy 1, policy_version 1156735 (0.0011) [2023-12-27 13:40:18,227][20381] Updated weights for policy 1, policy_version 1156745 (0.0010) [2023-12-27 13:40:18,272][20381] Updated weights for policy 1, policy_version 1156755 (0.0010) [2023-12-27 13:40:18,329][20347] Updated weights for policy 0, policy_version 1157940 (0.0007) [2023-12-27 13:40:18,394][20347] Updated weights for policy 0, policy_version 1157950 (0.0008) [2023-12-27 13:40:18,449][20347] Updated weights for policy 0, policy_version 1157960 (0.0007) [2023-12-27 13:40:18,963][20381] Updated weights for policy 1, policy_version 1156765 (0.0010) [2023-12-27 13:40:19,008][20381] Updated weights for policy 1, policy_version 1156775 (0.0010) [2023-12-27 13:40:19,054][20381] Updated weights for policy 1, policy_version 1156785 (0.0006) [2023-12-27 13:40:19,139][20347] Updated weights for policy 0, policy_version 1157970 (0.0007) [2023-12-27 13:40:19,188][20347] Updated weights for policy 0, policy_version 1157980 (0.0007) [2023-12-27 13:40:19,254][20347] Updated weights for policy 0, policy_version 1157990 (0.0007) [2023-12-27 13:40:19,317][20347] Updated weights for policy 0, policy_version 1158000 (0.0008) [2023-12-27 13:40:19,796][20381] Updated weights for policy 1, policy_version 1156795 (0.0005) [2023-12-27 13:40:19,866][20381] Updated weights for policy 1, policy_version 1156805 (0.0007) [2023-12-27 13:40:19,927][20381] Updated weights for policy 1, policy_version 1156815 (0.0007) [2023-12-27 13:40:20,027][20347] Updated weights for policy 0, policy_version 1158010 (0.0008) [2023-12-27 13:40:20,087][20347] Updated weights for policy 0, policy_version 1158020 (0.0007) [2023-12-27 13:40:20,147][20347] Updated weights for policy 0, policy_version 1158030 (0.0008) [2023-12-27 13:40:20,597][20381] Updated weights for policy 1, policy_version 1156825 (0.0007) [2023-12-27 13:40:20,655][20381] Updated weights for policy 1, policy_version 1156835 (0.0007) [2023-12-27 13:40:20,715][20381] Updated weights for policy 1, policy_version 1156845 (0.0006) [2023-12-27 13:40:20,772][20381] Updated weights for policy 1, policy_version 1156855 (0.0006) [2023-12-27 13:40:20,904][20347] Updated weights for policy 0, policy_version 1158040 (0.0008) [2023-12-27 13:40:20,954][20347] Updated weights for policy 0, policy_version 1158050 (0.0006) [2023-12-27 13:40:21,022][20347] Updated weights for policy 0, policy_version 1158060 (0.0008) [2023-12-27 13:40:21,552][20381] Updated weights for policy 1, policy_version 1156865 (0.0008) [2023-12-27 13:40:21,621][20381] Updated weights for policy 1, policy_version 1156875 (0.0007) [2023-12-27 13:40:21,689][20381] Updated weights for policy 1, policy_version 1156885 (0.0007) [2023-12-27 13:40:21,710][20347] Updated weights for policy 0, policy_version 1158070 (0.0007) [2023-12-27 13:40:21,781][20347] Updated weights for policy 0, policy_version 1158080 (0.0008) [2023-12-27 13:40:21,848][20347] Updated weights for policy 0, policy_version 1158090 (0.0006) [2023-12-27 13:40:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 592715776. Throughput: 0: 10083.4, 1: 10256.3. Samples: 592691024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '777.210')] [2023-12-27 13:40:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001158096_296525824.pth... [2023-12-27 13:40:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001156944_296230912.pth [2023-12-27 13:40:22,405][20381] Updated weights for policy 1, policy_version 1156895 (0.0007) [2023-12-27 13:40:22,465][20381] Updated weights for policy 1, policy_version 1156905 (0.0008) [2023-12-27 13:40:22,524][20381] Updated weights for policy 1, policy_version 1156915 (0.0007) [2023-12-27 13:40:22,537][20347] Updated weights for policy 0, policy_version 1158100 (0.0007) [2023-12-27 13:40:22,552][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001156920_296198144.pth... [2023-12-27 13:40:22,556][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001155704_295886848.pth [2023-12-27 13:40:22,591][20347] Updated weights for policy 0, policy_version 1158110 (0.0008) [2023-12-27 13:40:22,641][20347] Updated weights for policy 0, policy_version 1158120 (0.0007) [2023-12-27 13:40:23,209][20381] Updated weights for policy 1, policy_version 1156925 (0.0008) [2023-12-27 13:40:23,264][20381] Updated weights for policy 1, policy_version 1156935 (0.0010) [2023-12-27 13:40:23,294][20347] Updated weights for policy 0, policy_version 1158130 (0.0007) [2023-12-27 13:40:23,316][20381] Updated weights for policy 1, policy_version 1156945 (0.0011) [2023-12-27 13:40:23,354][20347] Updated weights for policy 0, policy_version 1158140 (0.0006) [2023-12-27 13:40:23,414][20347] Updated weights for policy 0, policy_version 1158150 (0.0007) [2023-12-27 13:40:23,475][20347] Updated weights for policy 0, policy_version 1158160 (0.0007) [2023-12-27 13:40:24,009][20381] Updated weights for policy 1, policy_version 1156955 (0.0011) [2023-12-27 13:40:24,057][20347] Updated weights for policy 0, policy_version 1158170 (0.0005) [2023-12-27 13:40:24,068][20381] Updated weights for policy 1, policy_version 1156965 (0.0010) [2023-12-27 13:40:24,105][20347] Updated weights for policy 0, policy_version 1158180 (0.0006) [2023-12-27 13:40:24,123][20381] Updated weights for policy 1, policy_version 1156975 (0.0011) [2023-12-27 13:40:24,156][20347] Updated weights for policy 0, policy_version 1158190 (0.0005) [2023-12-27 13:40:24,752][20347] Updated weights for policy 0, policy_version 1158200 (0.0005) [2023-12-27 13:40:24,785][20381] Updated weights for policy 1, policy_version 1156985 (0.0011) [2023-12-27 13:40:24,800][20347] Updated weights for policy 0, policy_version 1158210 (0.0005) [2023-12-27 13:40:24,844][20381] Updated weights for policy 1, policy_version 1156995 (0.0010) [2023-12-27 13:40:24,849][20347] Updated weights for policy 0, policy_version 1158220 (0.0005) [2023-12-27 13:40:24,896][20381] Updated weights for policy 1, policy_version 1157005 (0.0011) [2023-12-27 13:40:24,946][20381] Updated weights for policy 1, policy_version 1157015 (0.0011) [2023-12-27 13:40:25,511][20347] Updated weights for policy 0, policy_version 1158230 (0.0005) [2023-12-27 13:40:25,572][20347] Updated weights for policy 0, policy_version 1158240 (0.0006) [2023-12-27 13:40:25,608][20381] Updated weights for policy 1, policy_version 1157025 (0.0010) [2023-12-27 13:40:25,625][20347] Updated weights for policy 0, policy_version 1158250 (0.0006) [2023-12-27 13:40:25,661][20381] Updated weights for policy 1, policy_version 1157035 (0.0007) [2023-12-27 13:40:25,714][20381] Updated weights for policy 1, policy_version 1157045 (0.0007) [2023-12-27 13:40:26,306][20347] Updated weights for policy 0, policy_version 1158260 (0.0006) [2023-12-27 13:40:26,345][20381] Updated weights for policy 1, policy_version 1157055 (0.0006) [2023-12-27 13:40:26,358][20347] Updated weights for policy 0, policy_version 1158270 (0.0006) [2023-12-27 13:40:26,397][20381] Updated weights for policy 1, policy_version 1157065 (0.0006) [2023-12-27 13:40:26,413][20347] Updated weights for policy 0, policy_version 1158280 (0.0005) [2023-12-27 13:40:26,451][20381] Updated weights for policy 1, policy_version 1157075 (0.0005) [2023-12-27 13:40:27,016][20347] Updated weights for policy 0, policy_version 1158290 (0.0007) [2023-12-27 13:40:27,073][20347] Updated weights for policy 0, policy_version 1158300 (0.0008) [2023-12-27 13:40:27,098][20381] Updated weights for policy 1, policy_version 1157085 (0.0006) [2023-12-27 13:40:27,131][20347] Updated weights for policy 0, policy_version 1158310 (0.0006) [2023-12-27 13:40:27,151][20381] Updated weights for policy 1, policy_version 1157095 (0.0008) [2023-12-27 13:40:27,187][20347] Updated weights for policy 0, policy_version 1158320 (0.0006) [2023-12-27 13:40:27,205][20381] Updated weights for policy 1, policy_version 1157105 (0.0006) [2023-12-27 13:40:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 592830464. Throughput: 0: 10177.4, 1: 10198.2. Samples: 592813744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '756.990')] [2023-12-27 13:40:27,808][20347] Updated weights for policy 0, policy_version 1158330 (0.0005) [2023-12-27 13:40:27,863][20347] Updated weights for policy 0, policy_version 1158340 (0.0008) [2023-12-27 13:40:27,896][20381] Updated weights for policy 1, policy_version 1157115 (0.0009) [2023-12-27 13:40:27,917][20347] Updated weights for policy 0, policy_version 1158350 (0.0010) [2023-12-27 13:40:27,951][20381] Updated weights for policy 1, policy_version 1157125 (0.0010) [2023-12-27 13:40:28,011][20381] Updated weights for policy 1, policy_version 1157135 (0.0009) [2023-12-27 13:40:28,613][20381] Updated weights for policy 1, policy_version 1157145 (0.0006) [2023-12-27 13:40:28,618][20347] Updated weights for policy 0, policy_version 1158360 (0.0007) [2023-12-27 13:40:28,667][20347] Updated weights for policy 0, policy_version 1158370 (0.0006) [2023-12-27 13:40:28,668][20381] Updated weights for policy 1, policy_version 1157155 (0.0011) [2023-12-27 13:40:28,716][20347] Updated weights for policy 0, policy_version 1158380 (0.0006) [2023-12-27 13:40:28,721][20381] Updated weights for policy 1, policy_version 1157165 (0.0010) [2023-12-27 13:40:28,776][20381] Updated weights for policy 1, policy_version 1157175 (0.0011) [2023-12-27 13:40:29,398][20381] Updated weights for policy 1, policy_version 1157185 (0.0008) [2023-12-27 13:40:29,462][20381] Updated weights for policy 1, policy_version 1157195 (0.0008) [2023-12-27 13:40:29,500][20347] Updated weights for policy 0, policy_version 1158390 (0.0008) [2023-12-27 13:40:29,516][20381] Updated weights for policy 1, policy_version 1157205 (0.0006) [2023-12-27 13:40:29,554][20347] Updated weights for policy 0, policy_version 1158400 (0.0008) [2023-12-27 13:40:29,604][20347] Updated weights for policy 0, policy_version 1158410 (0.0009) [2023-12-27 13:40:30,124][20381] Updated weights for policy 1, policy_version 1157215 (0.0007) [2023-12-27 13:40:30,180][20381] Updated weights for policy 1, policy_version 1157225 (0.0007) [2023-12-27 13:40:30,229][20381] Updated weights for policy 1, policy_version 1157235 (0.0008) [2023-12-27 13:40:30,348][20347] Updated weights for policy 0, policy_version 1158420 (0.0007) [2023-12-27 13:40:30,405][20347] Updated weights for policy 0, policy_version 1158430 (0.0005) [2023-12-27 13:40:30,467][20347] Updated weights for policy 0, policy_version 1158440 (0.0006) [2023-12-27 13:40:30,949][20381] Updated weights for policy 1, policy_version 1157245 (0.0007) [2023-12-27 13:40:31,002][20381] Updated weights for policy 1, policy_version 1157255 (0.0005) [2023-12-27 13:40:31,069][20381] Updated weights for policy 1, policy_version 1157265 (0.0007) [2023-12-27 13:40:31,137][20347] Updated weights for policy 0, policy_version 1158450 (0.0008) [2023-12-27 13:40:31,201][20347] Updated weights for policy 0, policy_version 1158460 (0.0008) [2023-12-27 13:40:31,263][20347] Updated weights for policy 0, policy_version 1158470 (0.0008) [2023-12-27 13:40:31,322][20347] Updated weights for policy 0, policy_version 1158480 (0.0008) [2023-12-27 13:40:31,773][20381] Updated weights for policy 1, policy_version 1157275 (0.0007) [2023-12-27 13:40:31,838][20381] Updated weights for policy 1, policy_version 1157285 (0.0006) [2023-12-27 13:40:31,893][20381] Updated weights for policy 1, policy_version 1157295 (0.0007) [2023-12-27 13:40:31,992][20347] Updated weights for policy 0, policy_version 1158490 (0.0005) [2023-12-27 13:40:32,045][20347] Updated weights for policy 0, policy_version 1158500 (0.0005) [2023-12-27 13:40:32,099][20347] Updated weights for policy 0, policy_version 1158510 (0.0005) [2023-12-27 13:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 592928768. Throughput: 0: 10168.2, 1: 10284.2. Samples: 592938284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '764.930')] [2023-12-27 13:40:32,546][20381] Updated weights for policy 1, policy_version 1157305 (0.0007) [2023-12-27 13:40:32,605][20381] Updated weights for policy 1, policy_version 1157315 (0.0006) [2023-12-27 13:40:32,664][20381] Updated weights for policy 1, policy_version 1157325 (0.0008) [2023-12-27 13:40:32,727][20381] Updated weights for policy 1, policy_version 1157335 (0.0007) [2023-12-27 13:40:32,737][20347] Updated weights for policy 0, policy_version 1158520 (0.0007) [2023-12-27 13:40:32,796][20347] Updated weights for policy 0, policy_version 1158530 (0.0006) [2023-12-27 13:40:32,849][20347] Updated weights for policy 0, policy_version 1158540 (0.0007) [2023-12-27 13:40:33,354][20381] Updated weights for policy 1, policy_version 1157345 (0.0008) [2023-12-27 13:40:33,410][20381] Updated weights for policy 1, policy_version 1157355 (0.0010) [2023-12-27 13:40:33,465][20381] Updated weights for policy 1, policy_version 1157365 (0.0006) [2023-12-27 13:40:33,479][20347] Updated weights for policy 0, policy_version 1158550 (0.0007) [2023-12-27 13:40:33,527][20347] Updated weights for policy 0, policy_version 1158560 (0.0007) [2023-12-27 13:40:33,579][20347] Updated weights for policy 0, policy_version 1158570 (0.0009) [2023-12-27 13:40:34,023][20381] Updated weights for policy 1, policy_version 1157375 (0.0009) [2023-12-27 13:40:34,072][20381] Updated weights for policy 1, policy_version 1157385 (0.0009) [2023-12-27 13:40:34,122][20381] Updated weights for policy 1, policy_version 1157395 (0.0010) [2023-12-27 13:40:34,271][20347] Updated weights for policy 0, policy_version 1158580 (0.0008) [2023-12-27 13:40:34,331][20347] Updated weights for policy 0, policy_version 1158590 (0.0007) [2023-12-27 13:40:34,386][20347] Updated weights for policy 0, policy_version 1158600 (0.0008) [2023-12-27 13:40:34,878][20381] Updated weights for policy 1, policy_version 1157405 (0.0008) [2023-12-27 13:40:34,937][20381] Updated weights for policy 1, policy_version 1157415 (0.0006) [2023-12-27 13:40:34,992][20381] Updated weights for policy 1, policy_version 1157425 (0.0008) [2023-12-27 13:40:35,046][20347] Updated weights for policy 0, policy_version 1158610 (0.0008) [2023-12-27 13:40:35,100][20347] Updated weights for policy 0, policy_version 1158620 (0.0008) [2023-12-27 13:40:35,149][20347] Updated weights for policy 0, policy_version 1158630 (0.0008) [2023-12-27 13:40:35,193][20347] Updated weights for policy 0, policy_version 1158640 (0.0008) [2023-12-27 13:40:35,625][20381] Updated weights for policy 1, policy_version 1157435 (0.0007) [2023-12-27 13:40:35,677][20381] Updated weights for policy 1, policy_version 1157445 (0.0008) [2023-12-27 13:40:35,730][20381] Updated weights for policy 1, policy_version 1157455 (0.0008) [2023-12-27 13:40:35,905][20347] Updated weights for policy 0, policy_version 1158650 (0.0007) [2023-12-27 13:40:35,953][20347] Updated weights for policy 0, policy_version 1158660 (0.0007) [2023-12-27 13:40:36,001][20347] Updated weights for policy 0, policy_version 1158670 (0.0006) [2023-12-27 13:40:36,447][20381] Updated weights for policy 1, policy_version 1157465 (0.0006) [2023-12-27 13:40:36,511][20381] Updated weights for policy 1, policy_version 1157475 (0.0007) [2023-12-27 13:40:36,572][20381] Updated weights for policy 1, policy_version 1157485 (0.0009) [2023-12-27 13:40:36,622][20381] Updated weights for policy 1, policy_version 1157495 (0.0008) [2023-12-27 13:40:36,743][20347] Updated weights for policy 0, policy_version 1158680 (0.0007) [2023-12-27 13:40:36,800][20347] Updated weights for policy 0, policy_version 1158690 (0.0007) [2023-12-27 13:40:36,858][20347] Updated weights for policy 0, policy_version 1158700 (0.0008) [2023-12-27 13:40:37,334][20381] Updated weights for policy 1, policy_version 1157505 (0.0006) [2023-12-27 13:40:37,356][20292] KL-divergence is very high: 165.2382 [2023-12-27 13:40:37,362][20292] KL-divergence is very high: 174.9452 [2023-12-27 13:40:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 593027072. Throughput: 0: 10157.7, 1: 10332.8. Samples: 593002140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:37,363][19170] Avg episode reward: [(0, '990.250'), (1, '804.120')] [2023-12-27 13:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001158704_296681472.pth... [2023-12-27 13:40:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001157520_296378368.pth [2023-12-27 13:40:37,401][20381] Updated weights for policy 1, policy_version 1157515 (0.0005) [2023-12-27 13:40:37,465][20381] Updated weights for policy 1, policy_version 1157525 (0.0005) [2023-12-27 13:40:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001157528_296353792.pth... [2023-12-27 13:40:37,486][20347] Updated weights for policy 0, policy_version 1158710 (0.0007) [2023-12-27 13:40:37,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001156312_296042496.pth [2023-12-27 13:40:37,538][20347] Updated weights for policy 0, policy_version 1158720 (0.0008) [2023-12-27 13:40:37,594][20347] Updated weights for policy 0, policy_version 1158730 (0.0008) [2023-12-27 13:40:38,088][20381] Updated weights for policy 1, policy_version 1157535 (0.0007) [2023-12-27 13:40:38,144][20381] Updated weights for policy 1, policy_version 1157545 (0.0008) [2023-12-27 13:40:38,199][20381] Updated weights for policy 1, policy_version 1157555 (0.0005) [2023-12-27 13:40:38,325][20347] Updated weights for policy 0, policy_version 1158740 (0.0009) [2023-12-27 13:40:38,387][20347] Updated weights for policy 0, policy_version 1158750 (0.0007) [2023-12-27 13:40:38,442][20347] Updated weights for policy 0, policy_version 1158760 (0.0006) [2023-12-27 13:40:38,912][20381] Updated weights for policy 1, policy_version 1157565 (0.0007) [2023-12-27 13:40:38,965][20381] Updated weights for policy 1, policy_version 1157575 (0.0007) [2023-12-27 13:40:39,013][20381] Updated weights for policy 1, policy_version 1157585 (0.0008) [2023-12-27 13:40:39,075][20347] Updated weights for policy 0, policy_version 1158770 (0.0006) [2023-12-27 13:40:39,136][20347] Updated weights for policy 0, policy_version 1158780 (0.0006) [2023-12-27 13:40:39,202][20347] Updated weights for policy 0, policy_version 1158790 (0.0005) [2023-12-27 13:40:39,270][20347] Updated weights for policy 0, policy_version 1158800 (0.0008) [2023-12-27 13:40:39,731][20381] Updated weights for policy 1, policy_version 1157595 (0.0009) [2023-12-27 13:40:39,791][20381] Updated weights for policy 1, policy_version 1157605 (0.0009) [2023-12-27 13:40:39,857][20381] Updated weights for policy 1, policy_version 1157615 (0.0009) [2023-12-27 13:40:40,006][20347] Updated weights for policy 0, policy_version 1158810 (0.0006) [2023-12-27 13:40:40,069][20347] Updated weights for policy 0, policy_version 1158820 (0.0009) [2023-12-27 13:40:40,122][20347] Updated weights for policy 0, policy_version 1158830 (0.0011) [2023-12-27 13:40:40,541][20381] Updated weights for policy 1, policy_version 1157625 (0.0008) [2023-12-27 13:40:40,597][20381] Updated weights for policy 1, policy_version 1157635 (0.0011) [2023-12-27 13:40:40,654][20381] Updated weights for policy 1, policy_version 1157645 (0.0011) [2023-12-27 13:40:40,714][20381] Updated weights for policy 1, policy_version 1157655 (0.0011) [2023-12-27 13:40:40,759][20347] Updated weights for policy 0, policy_version 1158840 (0.0006) [2023-12-27 13:40:40,822][20347] Updated weights for policy 0, policy_version 1158850 (0.0006) [2023-12-27 13:40:40,882][20347] Updated weights for policy 0, policy_version 1158860 (0.0008) [2023-12-27 13:40:41,463][20381] Updated weights for policy 1, policy_version 1157665 (0.0007) [2023-12-27 13:40:41,523][20381] Updated weights for policy 1, policy_version 1157675 (0.0008) [2023-12-27 13:40:41,583][20381] Updated weights for policy 1, policy_version 1157685 (0.0009) [2023-12-27 13:40:41,651][20347] Updated weights for policy 0, policy_version 1158870 (0.0008) [2023-12-27 13:40:41,712][20347] Updated weights for policy 0, policy_version 1158880 (0.0007) [2023-12-27 13:40:41,779][20347] Updated weights for policy 0, policy_version 1158890 (0.0006) [2023-12-27 13:40:42,263][20381] Updated weights for policy 1, policy_version 1157695 (0.0010) [2023-12-27 13:40:42,323][20381] Updated weights for policy 1, policy_version 1157705 (0.0011) [2023-12-27 13:40:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 593125376. Throughput: 0: 10138.1, 1: 10271.5. Samples: 593122964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '823.880')] [2023-12-27 13:40:42,393][20381] Updated weights for policy 1, policy_version 1157715 (0.0009) [2023-12-27 13:40:42,420][20347] Updated weights for policy 0, policy_version 1158900 (0.0006) [2023-12-27 13:40:42,478][20347] Updated weights for policy 0, policy_version 1158910 (0.0007) [2023-12-27 13:40:42,540][20347] Updated weights for policy 0, policy_version 1158920 (0.0008) [2023-12-27 13:40:43,041][20381] Updated weights for policy 1, policy_version 1157725 (0.0008) [2023-12-27 13:40:43,092][20381] Updated weights for policy 1, policy_version 1157735 (0.0005) [2023-12-27 13:40:43,148][20381] Updated weights for policy 1, policy_version 1157745 (0.0005) [2023-12-27 13:40:43,288][20347] Updated weights for policy 0, policy_version 1158930 (0.0007) [2023-12-27 13:40:43,336][20347] Updated weights for policy 0, policy_version 1158940 (0.0008) [2023-12-27 13:40:43,381][20347] Updated weights for policy 0, policy_version 1158950 (0.0008) [2023-12-27 13:40:43,427][20347] Updated weights for policy 0, policy_version 1158960 (0.0008) [2023-12-27 13:40:43,685][20381] Updated weights for policy 1, policy_version 1157755 (0.0005) [2023-12-27 13:40:43,731][20381] Updated weights for policy 1, policy_version 1157765 (0.0005) [2023-12-27 13:40:43,784][20381] Updated weights for policy 1, policy_version 1157775 (0.0006) [2023-12-27 13:40:44,112][20347] Updated weights for policy 0, policy_version 1158970 (0.0005) [2023-12-27 13:40:44,159][20347] Updated weights for policy 0, policy_version 1158980 (0.0007) [2023-12-27 13:40:44,208][20347] Updated weights for policy 0, policy_version 1158990 (0.0007) [2023-12-27 13:40:44,488][20381] Updated weights for policy 1, policy_version 1157785 (0.0006) [2023-12-27 13:40:44,550][20381] Updated weights for policy 1, policy_version 1157795 (0.0008) [2023-12-27 13:40:44,611][20381] Updated weights for policy 1, policy_version 1157805 (0.0008) [2023-12-27 13:40:44,672][20381] Updated weights for policy 1, policy_version 1157815 (0.0007) [2023-12-27 13:40:44,824][20347] Updated weights for policy 0, policy_version 1159000 (0.0009) [2023-12-27 13:40:44,884][20347] Updated weights for policy 0, policy_version 1159010 (0.0008) [2023-12-27 13:40:44,938][20347] Updated weights for policy 0, policy_version 1159020 (0.0008) [2023-12-27 13:40:44,955][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-27 13:40:45,394][20381] Updated weights for policy 1, policy_version 1157825 (0.0006) [2023-12-27 13:40:45,451][20381] Updated weights for policy 1, policy_version 1157835 (0.0005) [2023-12-27 13:40:45,504][20381] Updated weights for policy 1, policy_version 1157845 (0.0008) [2023-12-27 13:40:45,652][20347] Updated weights for policy 0, policy_version 1159030 (0.0008) [2023-12-27 13:40:45,700][20347] Updated weights for policy 0, policy_version 1159040 (0.0007) [2023-12-27 13:40:45,757][20347] Updated weights for policy 0, policy_version 1159050 (0.0008) [2023-12-27 13:40:46,175][20381] Updated weights for policy 1, policy_version 1157855 (0.0007) [2023-12-27 13:40:46,235][20381] Updated weights for policy 1, policy_version 1157865 (0.0008) [2023-12-27 13:40:46,291][20381] Updated weights for policy 1, policy_version 1157875 (0.0008) [2023-12-27 13:40:46,465][20347] Updated weights for policy 0, policy_version 1159060 (0.0007) [2023-12-27 13:40:46,517][20347] Updated weights for policy 0, policy_version 1159070 (0.0007) [2023-12-27 13:40:46,581][20347] Updated weights for policy 0, policy_version 1159080 (0.0008) [2023-12-27 13:40:46,886][20381] Updated weights for policy 1, policy_version 1157885 (0.0007) [2023-12-27 13:40:46,939][20381] Updated weights for policy 1, policy_version 1157895 (0.0005) [2023-12-27 13:40:46,993][20381] Updated weights for policy 1, policy_version 1157905 (0.0005) [2023-12-27 13:40:47,141][20347] Updated weights for policy 0, policy_version 1159090 (0.0006) [2023-12-27 13:40:47,202][20347] Updated weights for policy 0, policy_version 1159100 (0.0006) [2023-12-27 13:40:47,257][20347] Updated weights for policy 0, policy_version 1159110 (0.0009) [2023-12-27 13:40:47,314][20347] Updated weights for policy 0, policy_version 1159120 (0.0008) [2023-12-27 13:40:47,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 593240064. Throughput: 0: 10162.9, 1: 10288.2. Samples: 593247672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '765.120')] [2023-12-27 13:40:47,572][20381] Updated weights for policy 1, policy_version 1157915 (0.0006) [2023-12-27 13:40:47,616][20381] Updated weights for policy 1, policy_version 1157925 (0.0007) [2023-12-27 13:40:47,667][20381] Updated weights for policy 1, policy_version 1157935 (0.0008) [2023-12-27 13:40:47,972][20347] Updated weights for policy 0, policy_version 1159130 (0.0005) [2023-12-27 13:40:48,021][20347] Updated weights for policy 0, policy_version 1159140 (0.0005) [2023-12-27 13:40:48,079][20347] Updated weights for policy 0, policy_version 1159150 (0.0009) [2023-12-27 13:40:48,237][20381] Updated weights for policy 1, policy_version 1157945 (0.0005) [2023-12-27 13:40:48,297][20381] Updated weights for policy 1, policy_version 1157955 (0.0005) [2023-12-27 13:40:48,357][20381] Updated weights for policy 1, policy_version 1157965 (0.0007) [2023-12-27 13:40:48,413][20381] Updated weights for policy 1, policy_version 1157975 (0.0008) [2023-12-27 13:40:48,732][20347] Updated weights for policy 0, policy_version 1159160 (0.0008) [2023-12-27 13:40:48,785][20347] Updated weights for policy 0, policy_version 1159170 (0.0010) [2023-12-27 13:40:48,842][20347] Updated weights for policy 0, policy_version 1159180 (0.0010) [2023-12-27 13:40:49,017][20381] Updated weights for policy 1, policy_version 1157985 (0.0008) [2023-12-27 13:40:49,071][20381] Updated weights for policy 1, policy_version 1157995 (0.0008) [2023-12-27 13:40:49,127][20381] Updated weights for policy 1, policy_version 1158005 (0.0007) [2023-12-27 13:40:49,577][20347] Updated weights for policy 0, policy_version 1159190 (0.0009) [2023-12-27 13:40:49,642][20347] Updated weights for policy 0, policy_version 1159200 (0.0008) [2023-12-27 13:40:49,702][20347] Updated weights for policy 0, policy_version 1159210 (0.0008) [2023-12-27 13:40:49,838][20381] Updated weights for policy 1, policy_version 1158015 (0.0008) [2023-12-27 13:40:49,898][20381] Updated weights for policy 1, policy_version 1158025 (0.0009) [2023-12-27 13:40:49,956][20381] Updated weights for policy 1, policy_version 1158035 (0.0008) [2023-12-27 13:40:50,332][20347] Updated weights for policy 0, policy_version 1159220 (0.0008) [2023-12-27 13:40:50,394][20347] Updated weights for policy 0, policy_version 1159230 (0.0008) [2023-12-27 13:40:50,450][20347] Updated weights for policy 0, policy_version 1159240 (0.0008) [2023-12-27 13:40:50,654][20381] Updated weights for policy 1, policy_version 1158045 (0.0008) [2023-12-27 13:40:50,713][20381] Updated weights for policy 1, policy_version 1158055 (0.0007) [2023-12-27 13:40:50,770][20381] Updated weights for policy 1, policy_version 1158065 (0.0006) [2023-12-27 13:40:51,149][20347] Updated weights for policy 0, policy_version 1159250 (0.0008) [2023-12-27 13:40:51,200][20347] Updated weights for policy 0, policy_version 1159260 (0.0008) [2023-12-27 13:40:51,254][20347] Updated weights for policy 0, policy_version 1159270 (0.0008) [2023-12-27 13:40:51,316][20347] Updated weights for policy 0, policy_version 1159280 (0.0009) [2023-12-27 13:40:51,453][20381] Updated weights for policy 1, policy_version 1158075 (0.0007) [2023-12-27 13:40:51,521][20381] Updated weights for policy 1, policy_version 1158085 (0.0005) [2023-12-27 13:40:51,586][20381] Updated weights for policy 1, policy_version 1158095 (0.0005) [2023-12-27 13:40:52,131][20347] Updated weights for policy 0, policy_version 1159290 (0.0008) [2023-12-27 13:40:52,182][20347] Updated weights for policy 0, policy_version 1159300 (0.0006) [2023-12-27 13:40:52,248][20347] Updated weights for policy 0, policy_version 1159310 (0.0006) [2023-12-27 13:40:52,250][20381] Updated weights for policy 1, policy_version 1158105 (0.0006) [2023-12-27 13:40:52,315][20381] Updated weights for policy 1, policy_version 1158115 (0.0008) [2023-12-27 13:40:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 593338368. Throughput: 0: 10167.9, 1: 10330.3. Samples: 593310340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:52,363][19170] Avg episode reward: [(0, '990.270'), (1, '745.470')] [2023-12-27 13:40:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001159312_296837120.pth... [2023-12-27 13:40:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001158096_296525824.pth [2023-12-27 13:40:52,384][20381] Updated weights for policy 1, policy_version 1158125 (0.0008) [2023-12-27 13:40:52,440][20381] Updated weights for policy 1, policy_version 1158135 (0.0008) [2023-12-27 13:40:52,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001158136_296509440.pth... [2023-12-27 13:40:52,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001156920_296198144.pth [2023-12-27 13:40:52,942][20347] Updated weights for policy 0, policy_version 1159320 (0.0007) [2023-12-27 13:40:53,001][20347] Updated weights for policy 0, policy_version 1159330 (0.0008) [2023-12-27 13:40:53,051][20347] Updated weights for policy 0, policy_version 1159340 (0.0008) [2023-12-27 13:40:53,089][20381] Updated weights for policy 1, policy_version 1158145 (0.0006) [2023-12-27 13:40:53,142][20381] Updated weights for policy 1, policy_version 1158155 (0.0005) [2023-12-27 13:40:53,199][20381] Updated weights for policy 1, policy_version 1158165 (0.0006) [2023-12-27 13:40:53,729][20381] Updated weights for policy 1, policy_version 1158175 (0.0005) [2023-12-27 13:40:53,796][20381] Updated weights for policy 1, policy_version 1158185 (0.0005) [2023-12-27 13:40:53,809][20347] Updated weights for policy 0, policy_version 1159350 (0.0008) [2023-12-27 13:40:53,849][20381] Updated weights for policy 1, policy_version 1158195 (0.0005) [2023-12-27 13:40:53,855][20347] Updated weights for policy 0, policy_version 1159360 (0.0005) [2023-12-27 13:40:53,902][20347] Updated weights for policy 0, policy_version 1159370 (0.0007) [2023-12-27 13:40:54,489][20381] Updated weights for policy 1, policy_version 1158205 (0.0007) [2023-12-27 13:40:54,537][20381] Updated weights for policy 1, policy_version 1158215 (0.0008) [2023-12-27 13:40:54,588][20381] Updated weights for policy 1, policy_version 1158225 (0.0007) [2023-12-27 13:40:54,610][20347] Updated weights for policy 0, policy_version 1159380 (0.0007) [2023-12-27 13:40:54,665][20347] Updated weights for policy 0, policy_version 1159390 (0.0008) [2023-12-27 13:40:54,720][20347] Updated weights for policy 0, policy_version 1159400 (0.0008) [2023-12-27 13:40:55,277][20381] Updated weights for policy 1, policy_version 1158235 (0.0009) [2023-12-27 13:40:55,329][20381] Updated weights for policy 1, policy_version 1158245 (0.0007) [2023-12-27 13:40:55,377][20381] Updated weights for policy 1, policy_version 1158255 (0.0008) [2023-12-27 13:40:55,423][20347] Updated weights for policy 0, policy_version 1159410 (0.0007) [2023-12-27 13:40:55,470][20347] Updated weights for policy 0, policy_version 1159420 (0.0007) [2023-12-27 13:40:55,516][20347] Updated weights for policy 0, policy_version 1159430 (0.0008) [2023-12-27 13:40:55,579][20347] Updated weights for policy 0, policy_version 1159440 (0.0008) [2023-12-27 13:40:56,036][20381] Updated weights for policy 1, policy_version 1158265 (0.0007) [2023-12-27 13:40:56,088][20381] Updated weights for policy 1, policy_version 1158275 (0.0007) [2023-12-27 13:40:56,145][20381] Updated weights for policy 1, policy_version 1158285 (0.0008) [2023-12-27 13:40:56,202][20381] Updated weights for policy 1, policy_version 1158295 (0.0008) [2023-12-27 13:40:56,317][20347] Updated weights for policy 0, policy_version 1159450 (0.0008) [2023-12-27 13:40:56,363][20347] Updated weights for policy 0, policy_version 1159460 (0.0008) [2023-12-27 13:40:56,419][20347] Updated weights for policy 0, policy_version 1159470 (0.0008) [2023-12-27 13:40:56,899][20381] Updated weights for policy 1, policy_version 1158305 (0.0006) [2023-12-27 13:40:56,948][20381] Updated weights for policy 1, policy_version 1158315 (0.0005) [2023-12-27 13:40:57,010][20381] Updated weights for policy 1, policy_version 1158325 (0.0007) [2023-12-27 13:40:57,115][20347] Updated weights for policy 0, policy_version 1159480 (0.0007) [2023-12-27 13:40:57,168][20347] Updated weights for policy 0, policy_version 1159490 (0.0008) [2023-12-27 13:40:57,222][20347] Updated weights for policy 0, policy_version 1159500 (0.0007) [2023-12-27 13:40:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 593444864. Throughput: 0: 10137.4, 1: 10421.0. Samples: 593431472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:40:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '774.960')] [2023-12-27 13:40:57,650][20381] Updated weights for policy 1, policy_version 1158335 (0.0008) [2023-12-27 13:40:57,694][20381] Updated weights for policy 1, policy_version 1158345 (0.0007) [2023-12-27 13:40:57,744][20381] Updated weights for policy 1, policy_version 1158355 (0.0006) [2023-12-27 13:40:57,932][20347] Updated weights for policy 0, policy_version 1159510 (0.0007) [2023-12-27 13:40:57,986][20347] Updated weights for policy 0, policy_version 1159520 (0.0007) [2023-12-27 13:40:58,044][20347] Updated weights for policy 0, policy_version 1159530 (0.0007) [2023-12-27 13:40:58,432][20381] Updated weights for policy 1, policy_version 1158365 (0.0007) [2023-12-27 13:40:58,496][20381] Updated weights for policy 1, policy_version 1158375 (0.0007) [2023-12-27 13:40:58,556][20381] Updated weights for policy 1, policy_version 1158385 (0.0008) [2023-12-27 13:40:58,821][20347] Updated weights for policy 0, policy_version 1159540 (0.0008) [2023-12-27 13:40:58,889][20347] Updated weights for policy 0, policy_version 1159550 (0.0007) [2023-12-27 13:40:58,944][20347] Updated weights for policy 0, policy_version 1159560 (0.0008) [2023-12-27 13:40:59,376][20381] Updated weights for policy 1, policy_version 1158396 (0.0009) [2023-12-27 13:40:59,438][20381] Updated weights for policy 1, policy_version 1158406 (0.0007) [2023-12-27 13:40:59,499][20381] Updated weights for policy 1, policy_version 1158416 (0.0008) [2023-12-27 13:40:59,704][20347] Updated weights for policy 0, policy_version 1159570 (0.0008) [2023-12-27 13:40:59,753][20347] Updated weights for policy 0, policy_version 1159580 (0.0007) [2023-12-27 13:40:59,800][20347] Updated weights for policy 0, policy_version 1159590 (0.0007) [2023-12-27 13:40:59,862][20347] Updated weights for policy 0, policy_version 1159600 (0.0007) [2023-12-27 13:41:00,156][20381] Updated weights for policy 1, policy_version 1158426 (0.0008) [2023-12-27 13:41:00,209][20381] Updated weights for policy 1, policy_version 1158436 (0.0009) [2023-12-27 13:41:00,257][20381] Updated weights for policy 1, policy_version 1158446 (0.0005) [2023-12-27 13:41:00,312][20381] Updated weights for policy 1, policy_version 1158456 (0.0005) [2023-12-27 13:41:00,618][20347] Updated weights for policy 0, policy_version 1159610 (0.0008) [2023-12-27 13:41:00,667][20347] Updated weights for policy 0, policy_version 1159620 (0.0007) [2023-12-27 13:41:00,715][20347] Updated weights for policy 0, policy_version 1159630 (0.0007) [2023-12-27 13:41:00,960][20381] Updated weights for policy 1, policy_version 1158466 (0.0009) [2023-12-27 13:41:01,019][20381] Updated weights for policy 1, policy_version 1158476 (0.0008) [2023-12-27 13:41:01,075][20381] Updated weights for policy 1, policy_version 1158486 (0.0008) [2023-12-27 13:41:01,390][20347] Updated weights for policy 0, policy_version 1159640 (0.0008) [2023-12-27 13:41:01,448][20347] Updated weights for policy 0, policy_version 1159650 (0.0007) [2023-12-27 13:41:01,506][20347] Updated weights for policy 0, policy_version 1159660 (0.0007) [2023-12-27 13:41:01,772][20381] Updated weights for policy 1, policy_version 1158496 (0.0008) [2023-12-27 13:41:01,834][20381] Updated weights for policy 1, policy_version 1158506 (0.0008) [2023-12-27 13:41:01,896][20381] Updated weights for policy 1, policy_version 1158516 (0.0008) [2023-12-27 13:41:02,098][20347] Updated weights for policy 0, policy_version 1159670 (0.0006) [2023-12-27 13:41:02,150][20347] Updated weights for policy 0, policy_version 1159680 (0.0005) [2023-12-27 13:41:02,198][20347] Updated weights for policy 0, policy_version 1159690 (0.0006) [2023-12-27 13:41:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 593543168. Throughput: 0: 10113.1, 1: 10356.4. Samples: 593550872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '765.060')] [2023-12-27 13:41:02,607][20381] Updated weights for policy 1, policy_version 1158526 (0.0008) [2023-12-27 13:41:02,662][20381] Updated weights for policy 1, policy_version 1158536 (0.0007) [2023-12-27 13:41:02,720][20381] Updated weights for policy 1, policy_version 1158546 (0.0007) [2023-12-27 13:41:02,895][20347] Updated weights for policy 0, policy_version 1159700 (0.0007) [2023-12-27 13:41:02,949][20347] Updated weights for policy 0, policy_version 1159710 (0.0008) [2023-12-27 13:41:02,999][20347] Updated weights for policy 0, policy_version 1159720 (0.0008) [2023-12-27 13:41:03,400][20381] Updated weights for policy 1, policy_version 1158556 (0.0007) [2023-12-27 13:41:03,449][20381] Updated weights for policy 1, policy_version 1158566 (0.0007) [2023-12-27 13:41:03,500][20381] Updated weights for policy 1, policy_version 1158576 (0.0005) [2023-12-27 13:41:03,711][20347] Updated weights for policy 0, policy_version 1159730 (0.0007) [2023-12-27 13:41:03,758][20347] Updated weights for policy 0, policy_version 1159740 (0.0007) [2023-12-27 13:41:03,805][20347] Updated weights for policy 0, policy_version 1159750 (0.0008) [2023-12-27 13:41:03,864][20347] Updated weights for policy 0, policy_version 1159760 (0.0008) [2023-12-27 13:41:04,086][20381] Updated weights for policy 1, policy_version 1158586 (0.0006) [2023-12-27 13:41:04,149][20381] Updated weights for policy 1, policy_version 1158596 (0.0008) [2023-12-27 13:41:04,212][20381] Updated weights for policy 1, policy_version 1158606 (0.0007) [2023-12-27 13:41:04,269][20381] Updated weights for policy 1, policy_version 1158616 (0.0006) [2023-12-27 13:41:04,627][20347] Updated weights for policy 0, policy_version 1159770 (0.0008) [2023-12-27 13:41:04,681][20347] Updated weights for policy 0, policy_version 1159780 (0.0008) [2023-12-27 13:41:04,737][20347] Updated weights for policy 0, policy_version 1159790 (0.0007) [2023-12-27 13:41:04,874][20381] Updated weights for policy 1, policy_version 1158626 (0.0008) [2023-12-27 13:41:04,923][20381] Updated weights for policy 1, policy_version 1158636 (0.0007) [2023-12-27 13:41:04,968][20381] Updated weights for policy 1, policy_version 1158646 (0.0007) [2023-12-27 13:41:05,386][20347] Updated weights for policy 0, policy_version 1159800 (0.0008) [2023-12-27 13:41:05,432][20347] Updated weights for policy 0, policy_version 1159810 (0.0010) [2023-12-27 13:41:05,476][20347] Updated weights for policy 0, policy_version 1159820 (0.0009) [2023-12-27 13:41:05,649][20381] Updated weights for policy 1, policy_version 1158656 (0.0009) [2023-12-27 13:41:05,698][20381] Updated weights for policy 1, policy_version 1158666 (0.0005) [2023-12-27 13:41:05,748][20381] Updated weights for policy 1, policy_version 1158676 (0.0008) [2023-12-27 13:41:06,072][20347] Updated weights for policy 0, policy_version 1159830 (0.0008) [2023-12-27 13:41:06,132][20347] Updated weights for policy 0, policy_version 1159840 (0.0010) [2023-12-27 13:41:06,196][20347] Updated weights for policy 0, policy_version 1159850 (0.0011) [2023-12-27 13:41:06,434][20381] Updated weights for policy 1, policy_version 1158686 (0.0007) [2023-12-27 13:41:06,493][20381] Updated weights for policy 1, policy_version 1158696 (0.0007) [2023-12-27 13:41:06,555][20381] Updated weights for policy 1, policy_version 1158706 (0.0008) [2023-12-27 13:41:06,886][20347] Updated weights for policy 0, policy_version 1159860 (0.0011) [2023-12-27 13:41:06,935][20347] Updated weights for policy 0, policy_version 1159870 (0.0010) [2023-12-27 13:41:06,981][20347] Updated weights for policy 0, policy_version 1159880 (0.0010) [2023-12-27 13:41:07,234][20381] Updated weights for policy 1, policy_version 1158716 (0.0008) [2023-12-27 13:41:07,285][20381] Updated weights for policy 1, policy_version 1158726 (0.0007) [2023-12-27 13:41:07,332][20381] Updated weights for policy 1, policy_version 1158736 (0.0007) [2023-12-27 13:41:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 593641472. Throughput: 0: 10110.7, 1: 10376.6. Samples: 593612952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:07,363][19170] Avg episode reward: [(0, '980.650'), (1, '804.290')] [2023-12-27 13:41:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001158744_296665088.pth... [2023-12-27 13:41:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001159888_296984576.pth... [2023-12-27 13:41:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001158704_296681472.pth [2023-12-27 13:41:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001157528_296353792.pth [2023-12-27 13:41:07,669][20347] Updated weights for policy 0, policy_version 1159890 (0.0009) [2023-12-27 13:41:07,721][20347] Updated weights for policy 0, policy_version 1159900 (0.0006) [2023-12-27 13:41:07,775][20347] Updated weights for policy 0, policy_version 1159910 (0.0010) [2023-12-27 13:41:07,825][20347] Updated weights for policy 0, policy_version 1159920 (0.0009) [2023-12-27 13:41:08,006][20381] Updated weights for policy 1, policy_version 1158746 (0.0007) [2023-12-27 13:41:08,054][20381] Updated weights for policy 1, policy_version 1158756 (0.0005) [2023-12-27 13:41:08,099][20381] Updated weights for policy 1, policy_version 1158766 (0.0005) [2023-12-27 13:41:08,149][20381] Updated weights for policy 1, policy_version 1158776 (0.0007) [2023-12-27 13:41:08,464][20347] Updated weights for policy 0, policy_version 1159930 (0.0007) [2023-12-27 13:41:08,519][20347] Updated weights for policy 0, policy_version 1159940 (0.0008) [2023-12-27 13:41:08,568][20347] Updated weights for policy 0, policy_version 1159950 (0.0006) [2023-12-27 13:41:08,834][20381] Updated weights for policy 1, policy_version 1158786 (0.0007) [2023-12-27 13:41:08,893][20381] Updated weights for policy 1, policy_version 1158796 (0.0008) [2023-12-27 13:41:08,948][20381] Updated weights for policy 1, policy_version 1158806 (0.0008) [2023-12-27 13:41:09,273][20347] Updated weights for policy 0, policy_version 1159960 (0.0008) [2023-12-27 13:41:09,335][20347] Updated weights for policy 0, policy_version 1159970 (0.0011) [2023-12-27 13:41:09,410][20347] Updated weights for policy 0, policy_version 1159980 (0.0009) [2023-12-27 13:41:09,739][20381] Updated weights for policy 1, policy_version 1158816 (0.0008) [2023-12-27 13:41:09,798][20381] Updated weights for policy 1, policy_version 1158826 (0.0008) [2023-12-27 13:41:09,875][20381] Updated weights for policy 1, policy_version 1158836 (0.0008) [2023-12-27 13:41:10,114][20347] Updated weights for policy 0, policy_version 1159990 (0.0011) [2023-12-27 13:41:10,174][20347] Updated weights for policy 0, policy_version 1160000 (0.0010) [2023-12-27 13:41:10,235][20347] Updated weights for policy 0, policy_version 1160010 (0.0011) [2023-12-27 13:41:10,579][20381] Updated weights for policy 1, policy_version 1158846 (0.0009) [2023-12-27 13:41:10,628][20381] Updated weights for policy 1, policy_version 1158856 (0.0007) [2023-12-27 13:41:10,686][20381] Updated weights for policy 1, policy_version 1158866 (0.0007) [2023-12-27 13:41:10,882][20347] Updated weights for policy 0, policy_version 1160020 (0.0010) [2023-12-27 13:41:10,944][20347] Updated weights for policy 0, policy_version 1160030 (0.0008) [2023-12-27 13:41:11,006][20347] Updated weights for policy 0, policy_version 1160040 (0.0008) [2023-12-27 13:41:11,489][20381] Updated weights for policy 1, policy_version 1158876 (0.0007) [2023-12-27 13:41:11,551][20381] Updated weights for policy 1, policy_version 1158886 (0.0008) [2023-12-27 13:41:11,617][20381] Updated weights for policy 1, policy_version 1158896 (0.0008) [2023-12-27 13:41:11,763][20347] Updated weights for policy 0, policy_version 1160050 (0.0008) [2023-12-27 13:41:11,823][20347] Updated weights for policy 0, policy_version 1160060 (0.0008) [2023-12-27 13:41:11,881][20347] Updated weights for policy 0, policy_version 1160070 (0.0007) [2023-12-27 13:41:11,941][20347] Updated weights for policy 0, policy_version 1160080 (0.0007) [2023-12-27 13:41:12,311][20381] Updated weights for policy 1, policy_version 1158906 (0.0008) [2023-12-27 13:41:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 593739776. Throughput: 0: 10099.6, 1: 10354.0. Samples: 593734160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:12,363][19170] Avg episode reward: [(0, '990.380'), (1, '794.560')] [2023-12-27 13:41:12,374][20381] Updated weights for policy 1, policy_version 1158916 (0.0006) [2023-12-27 13:41:12,422][20381] Updated weights for policy 1, policy_version 1158926 (0.0008) [2023-12-27 13:41:12,465][20381] Updated weights for policy 1, policy_version 1158936 (0.0008) [2023-12-27 13:41:12,702][20347] Updated weights for policy 0, policy_version 1160090 (0.0008) [2023-12-27 13:41:12,762][20347] Updated weights for policy 0, policy_version 1160100 (0.0008) [2023-12-27 13:41:12,825][20347] Updated weights for policy 0, policy_version 1160110 (0.0008) [2023-12-27 13:41:13,199][20381] Updated weights for policy 1, policy_version 1158946 (0.0008) [2023-12-27 13:41:13,252][20381] Updated weights for policy 1, policy_version 1158956 (0.0008) [2023-12-27 13:41:13,302][20381] Updated weights for policy 1, policy_version 1158966 (0.0008) [2023-12-27 13:41:13,437][20347] Updated weights for policy 0, policy_version 1160120 (0.0007) [2023-12-27 13:41:13,488][20347] Updated weights for policy 0, policy_version 1160130 (0.0007) [2023-12-27 13:41:13,540][20347] Updated weights for policy 0, policy_version 1160140 (0.0007) [2023-12-27 13:41:14,015][20381] Updated weights for policy 1, policy_version 1158976 (0.0006) [2023-12-27 13:41:14,061][20381] Updated weights for policy 1, policy_version 1158986 (0.0008) [2023-12-27 13:41:14,114][20381] Updated weights for policy 1, policy_version 1158996 (0.0007) [2023-12-27 13:41:14,242][20347] Updated weights for policy 0, policy_version 1160150 (0.0009) [2023-12-27 13:41:14,286][20347] Updated weights for policy 0, policy_version 1160160 (0.0008) [2023-12-27 13:41:14,333][20347] Updated weights for policy 0, policy_version 1160170 (0.0007) [2023-12-27 13:41:14,786][20381] Updated weights for policy 1, policy_version 1159006 (0.0007) [2023-12-27 13:41:14,849][20381] Updated weights for policy 1, policy_version 1159016 (0.0007) [2023-12-27 13:41:14,910][20381] Updated weights for policy 1, policy_version 1159026 (0.0007) [2023-12-27 13:41:15,096][20347] Updated weights for policy 0, policy_version 1160180 (0.0007) [2023-12-27 13:41:15,160][20347] Updated weights for policy 0, policy_version 1160190 (0.0008) [2023-12-27 13:41:15,223][20347] Updated weights for policy 0, policy_version 1160200 (0.0008) [2023-12-27 13:41:15,546][20381] Updated weights for policy 1, policy_version 1159036 (0.0007) [2023-12-27 13:41:15,596][20381] Updated weights for policy 1, policy_version 1159046 (0.0008) [2023-12-27 13:41:15,646][20381] Updated weights for policy 1, policy_version 1159056 (0.0007) [2023-12-27 13:41:15,852][20347] Updated weights for policy 0, policy_version 1160210 (0.0008) [2023-12-27 13:41:15,904][20347] Updated weights for policy 0, policy_version 1160220 (0.0005) [2023-12-27 13:41:15,964][20347] Updated weights for policy 0, policy_version 1160230 (0.0009) [2023-12-27 13:41:16,021][20347] Updated weights for policy 0, policy_version 1160240 (0.0010) [2023-12-27 13:41:16,349][20381] Updated weights for policy 1, policy_version 1159066 (0.0009) [2023-12-27 13:41:16,397][20381] Updated weights for policy 1, policy_version 1159076 (0.0010) [2023-12-27 13:41:16,449][20381] Updated weights for policy 1, policy_version 1159086 (0.0010) [2023-12-27 13:41:16,507][20381] Updated weights for policy 1, policy_version 1159096 (0.0010) [2023-12-27 13:41:16,638][20347] Updated weights for policy 0, policy_version 1160250 (0.0008) [2023-12-27 13:41:16,685][20347] Updated weights for policy 0, policy_version 1160260 (0.0010) [2023-12-27 13:41:16,746][20347] Updated weights for policy 0, policy_version 1160270 (0.0010) [2023-12-27 13:41:17,161][20381] Updated weights for policy 1, policy_version 1159106 (0.0007) [2023-12-27 13:41:17,215][20381] Updated weights for policy 1, policy_version 1159116 (0.0005) [2023-12-27 13:41:17,270][20381] Updated weights for policy 1, policy_version 1159126 (0.0006) [2023-12-27 13:41:17,340][20347] Updated weights for policy 0, policy_version 1160280 (0.0006) [2023-12-27 13:41:17,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 593846272. Throughput: 0: 10097.0, 1: 10301.7. Samples: 593856228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:17,364][19170] Avg episode reward: [(0, '990.380'), (1, '784.830')] [2023-12-27 13:41:17,396][20347] Updated weights for policy 0, policy_version 1160290 (0.0005) [2023-12-27 13:41:17,455][20347] Updated weights for policy 0, policy_version 1160300 (0.0010) [2023-12-27 13:41:17,828][20381] Updated weights for policy 1, policy_version 1159136 (0.0005) [2023-12-27 13:41:17,877][20381] Updated weights for policy 1, policy_version 1159146 (0.0005) [2023-12-27 13:41:17,939][20381] Updated weights for policy 1, policy_version 1159156 (0.0006) [2023-12-27 13:41:18,078][20347] Updated weights for policy 0, policy_version 1160310 (0.0007) [2023-12-27 13:41:18,132][20347] Updated weights for policy 0, policy_version 1160320 (0.0005) [2023-12-27 13:41:18,180][20347] Updated weights for policy 0, policy_version 1160330 (0.0007) [2023-12-27 13:41:18,559][20381] Updated weights for policy 1, policy_version 1159166 (0.0009) [2023-12-27 13:41:18,621][20381] Updated weights for policy 1, policy_version 1159176 (0.0010) [2023-12-27 13:41:18,679][20381] Updated weights for policy 1, policy_version 1159186 (0.0010) [2023-12-27 13:41:18,817][20347] Updated weights for policy 0, policy_version 1160340 (0.0008) [2023-12-27 13:41:18,828][20291] KL-divergence is very high: 386.1400 [2023-12-27 13:41:18,873][20347] Updated weights for policy 0, policy_version 1160350 (0.0007) [2023-12-27 13:41:18,879][20291] KL-divergence is very high: 123.0863 [2023-12-27 13:41:18,896][20291] KL-divergence is very high: 127.7978 [2023-12-27 13:41:18,923][20291] KL-divergence is very high: 188.8915 [2023-12-27 13:41:18,935][20347] Updated weights for policy 0, policy_version 1160360 (0.0008) [2023-12-27 13:41:18,976][20291] KL-divergence is very high: 574.8945 [2023-12-27 13:41:19,340][20381] Updated weights for policy 1, policy_version 1159196 (0.0009) [2023-12-27 13:41:19,408][20381] Updated weights for policy 1, policy_version 1159206 (0.0008) [2023-12-27 13:41:19,467][20381] Updated weights for policy 1, policy_version 1159216 (0.0007) [2023-12-27 13:41:19,715][20347] Updated weights for policy 0, policy_version 1160370 (0.0008) [2023-12-27 13:41:19,775][20347] Updated weights for policy 0, policy_version 1160380 (0.0008) [2023-12-27 13:41:19,830][20347] Updated weights for policy 0, policy_version 1160390 (0.0008) [2023-12-27 13:41:19,893][20347] Updated weights for policy 0, policy_version 1160400 (0.0007) [2023-12-27 13:41:20,187][20381] Updated weights for policy 1, policy_version 1159226 (0.0008) [2023-12-27 13:41:20,248][20381] Updated weights for policy 1, policy_version 1159236 (0.0010) [2023-12-27 13:41:20,308][20381] Updated weights for policy 1, policy_version 1159246 (0.0010) [2023-12-27 13:41:20,364][20381] Updated weights for policy 1, policy_version 1159256 (0.0010) [2023-12-27 13:41:20,650][20347] Updated weights for policy 0, policy_version 1160410 (0.0007) [2023-12-27 13:41:20,709][20347] Updated weights for policy 0, policy_version 1160420 (0.0008) [2023-12-27 13:41:20,771][20347] Updated weights for policy 0, policy_version 1160430 (0.0007) [2023-12-27 13:41:21,026][20381] Updated weights for policy 1, policy_version 1159266 (0.0009) [2023-12-27 13:41:21,089][20381] Updated weights for policy 1, policy_version 1159276 (0.0010) [2023-12-27 13:41:21,155][20381] Updated weights for policy 1, policy_version 1159286 (0.0009) [2023-12-27 13:41:21,493][20347] Updated weights for policy 0, policy_version 1160440 (0.0007) [2023-12-27 13:41:21,547][20347] Updated weights for policy 0, policy_version 1160450 (0.0007) [2023-12-27 13:41:21,606][20347] Updated weights for policy 0, policy_version 1160460 (0.0007) [2023-12-27 13:41:21,929][20381] Updated weights for policy 1, policy_version 1159296 (0.0009) [2023-12-27 13:41:21,991][20381] Updated weights for policy 1, policy_version 1159306 (0.0009) [2023-12-27 13:41:22,040][20381] Updated weights for policy 1, policy_version 1159316 (0.0009) [2023-12-27 13:41:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 593944576. Throughput: 0: 10065.2, 1: 10297.8. Samples: 593918476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:22,363][19170] Avg episode reward: [(0, '960.830'), (1, '784.660')] [2023-12-27 13:41:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001159320_296812544.pth... [2023-12-27 13:41:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001160464_297132032.pth... [2023-12-27 13:41:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001158136_296509440.pth [2023-12-27 13:41:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001159312_296837120.pth [2023-12-27 13:41:22,440][20347] Updated weights for policy 0, policy_version 1160470 (0.0008) [2023-12-27 13:41:22,491][20347] Updated weights for policy 0, policy_version 1160480 (0.0008) [2023-12-27 13:41:22,547][20347] Updated weights for policy 0, policy_version 1160490 (0.0008) [2023-12-27 13:41:22,830][20381] Updated weights for policy 1, policy_version 1159326 (0.0008) [2023-12-27 13:41:22,884][20381] Updated weights for policy 1, policy_version 1159336 (0.0008) [2023-12-27 13:41:22,944][20381] Updated weights for policy 1, policy_version 1159346 (0.0007) [2023-12-27 13:41:23,285][20347] Updated weights for policy 0, policy_version 1160500 (0.0008) [2023-12-27 13:41:23,346][20347] Updated weights for policy 0, policy_version 1160510 (0.0008) [2023-12-27 13:41:23,396][20347] Updated weights for policy 0, policy_version 1160520 (0.0008) [2023-12-27 13:41:23,593][20381] Updated weights for policy 1, policy_version 1159356 (0.0007) [2023-12-27 13:41:23,638][20381] Updated weights for policy 1, policy_version 1159366 (0.0008) [2023-12-27 13:41:23,684][20381] Updated weights for policy 1, policy_version 1159376 (0.0008) [2023-12-27 13:41:24,092][20347] Updated weights for policy 0, policy_version 1160530 (0.0008) [2023-12-27 13:41:24,146][20347] Updated weights for policy 0, policy_version 1160540 (0.0007) [2023-12-27 13:41:24,197][20347] Updated weights for policy 0, policy_version 1160550 (0.0006) [2023-12-27 13:41:24,255][20347] Updated weights for policy 0, policy_version 1160560 (0.0007) [2023-12-27 13:41:24,358][20381] Updated weights for policy 1, policy_version 1159386 (0.0007) [2023-12-27 13:41:24,420][20381] Updated weights for policy 1, policy_version 1159396 (0.0008) [2023-12-27 13:41:24,476][20381] Updated weights for policy 1, policy_version 1159406 (0.0008) [2023-12-27 13:41:24,531][20381] Updated weights for policy 1, policy_version 1159416 (0.0007) [2023-12-27 13:41:24,973][20347] Updated weights for policy 0, policy_version 1160570 (0.0008) [2023-12-27 13:41:25,030][20347] Updated weights for policy 0, policy_version 1160580 (0.0007) [2023-12-27 13:41:25,083][20381] Updated weights for policy 1, policy_version 1159426 (0.0007) [2023-12-27 13:41:25,089][20347] Updated weights for policy 0, policy_version 1160590 (0.0006) [2023-12-27 13:41:25,136][20381] Updated weights for policy 1, policy_version 1159436 (0.0007) [2023-12-27 13:41:25,191][20381] Updated weights for policy 1, policy_version 1159446 (0.0007) [2023-12-27 13:41:25,780][20347] Updated weights for policy 0, policy_version 1160600 (0.0007) [2023-12-27 13:41:25,822][20291] KL-divergence is very high: 496.2903 [2023-12-27 13:41:25,841][20347] Updated weights for policy 0, policy_version 1160610 (0.0007) [2023-12-27 13:41:25,850][20291] KL-divergence is very high: 676.5023 [2023-12-27 13:41:25,874][20291] KL-divergence is very high: 302.0915 [2023-12-27 13:41:25,897][20381] Updated weights for policy 1, policy_version 1159456 (0.0008) [2023-12-27 13:41:25,904][20347] Updated weights for policy 0, policy_version 1160620 (0.0007) [2023-12-27 13:41:25,954][20381] Updated weights for policy 1, policy_version 1159466 (0.0006) [2023-12-27 13:41:26,004][20381] Updated weights for policy 1, policy_version 1159476 (0.0006) [2023-12-27 13:41:26,607][20381] Updated weights for policy 1, policy_version 1159486 (0.0005) [2023-12-27 13:41:26,651][20347] Updated weights for policy 0, policy_version 1160630 (0.0007) [2023-12-27 13:41:26,657][20381] Updated weights for policy 1, policy_version 1159496 (0.0007) [2023-12-27 13:41:26,706][20347] Updated weights for policy 0, policy_version 1160640 (0.0007) [2023-12-27 13:41:26,715][20381] Updated weights for policy 1, policy_version 1159506 (0.0005) [2023-12-27 13:41:26,764][20347] Updated weights for policy 0, policy_version 1160650 (0.0007) [2023-12-27 13:41:27,313][20381] Updated weights for policy 1, policy_version 1159516 (0.0008) [2023-12-27 13:41:27,358][20381] Updated weights for policy 1, policy_version 1159526 (0.0010) [2023-12-27 13:41:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 594042880. Throughput: 0: 9994.5, 1: 10307.7. Samples: 594036564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:27,363][19170] Avg episode reward: [(0, '921.780'), (1, '755.310')] [2023-12-27 13:41:27,410][20381] Updated weights for policy 1, policy_version 1159536 (0.0010) [2023-12-27 13:41:27,490][20347] Updated weights for policy 0, policy_version 1160660 (0.0008) [2023-12-27 13:41:27,545][20347] Updated weights for policy 0, policy_version 1160670 (0.0007) [2023-12-27 13:41:27,602][20347] Updated weights for policy 0, policy_version 1160680 (0.0008) [2023-12-27 13:41:28,026][20381] Updated weights for policy 1, policy_version 1159546 (0.0011) [2023-12-27 13:41:28,085][20381] Updated weights for policy 1, policy_version 1159556 (0.0006) [2023-12-27 13:41:28,138][20381] Updated weights for policy 1, policy_version 1159566 (0.0005) [2023-12-27 13:41:28,189][20381] Updated weights for policy 1, policy_version 1159576 (0.0005) [2023-12-27 13:41:28,272][20347] Updated weights for policy 0, policy_version 1160690 (0.0008) [2023-12-27 13:41:28,328][20347] Updated weights for policy 0, policy_version 1160700 (0.0005) [2023-12-27 13:41:28,395][20347] Updated weights for policy 0, policy_version 1160710 (0.0008) [2023-12-27 13:41:28,451][20347] Updated weights for policy 0, policy_version 1160720 (0.0008) [2023-12-27 13:41:28,826][20381] Updated weights for policy 1, policy_version 1159586 (0.0007) [2023-12-27 13:41:28,874][20381] Updated weights for policy 1, policy_version 1159596 (0.0007) [2023-12-27 13:41:28,933][20381] Updated weights for policy 1, policy_version 1159606 (0.0008) [2023-12-27 13:41:29,100][20347] Updated weights for policy 0, policy_version 1160730 (0.0010) [2023-12-27 13:41:29,155][20347] Updated weights for policy 0, policy_version 1160740 (0.0010) [2023-12-27 13:41:29,207][20347] Updated weights for policy 0, policy_version 1160750 (0.0010) [2023-12-27 13:41:29,600][20381] Updated weights for policy 1, policy_version 1159616 (0.0007) [2023-12-27 13:41:29,656][20381] Updated weights for policy 1, policy_version 1159626 (0.0006) [2023-12-27 13:41:29,701][20381] Updated weights for policy 1, policy_version 1159636 (0.0005) [2023-12-27 13:41:29,942][20347] Updated weights for policy 0, policy_version 1160760 (0.0009) [2023-12-27 13:41:30,002][20347] Updated weights for policy 0, policy_version 1160771 (0.0009) [2023-12-27 13:41:30,051][20347] Updated weights for policy 0, policy_version 1160781 (0.0010) [2023-12-27 13:41:30,341][20381] Updated weights for policy 1, policy_version 1159646 (0.0007) [2023-12-27 13:41:30,400][20381] Updated weights for policy 1, policy_version 1159656 (0.0007) [2023-12-27 13:41:30,453][20381] Updated weights for policy 1, policy_version 1159666 (0.0009) [2023-12-27 13:41:30,769][20347] Updated weights for policy 0, policy_version 1160791 (0.0008) [2023-12-27 13:41:30,816][20347] Updated weights for policy 0, policy_version 1160801 (0.0006) [2023-12-27 13:41:30,857][20347] Updated weights for policy 0, policy_version 1160811 (0.0008) [2023-12-27 13:41:31,111][20381] Updated weights for policy 1, policy_version 1159676 (0.0007) [2023-12-27 13:41:31,171][20381] Updated weights for policy 1, policy_version 1159686 (0.0007) [2023-12-27 13:41:31,232][20381] Updated weights for policy 1, policy_version 1159696 (0.0008) [2023-12-27 13:41:31,573][20347] Updated weights for policy 0, policy_version 1160821 (0.0010) [2023-12-27 13:41:31,629][20347] Updated weights for policy 0, policy_version 1160831 (0.0008) [2023-12-27 13:41:31,691][20347] Updated weights for policy 0, policy_version 1160841 (0.0006) [2023-12-27 13:41:31,970][20381] Updated weights for policy 1, policy_version 1159706 (0.0007) [2023-12-27 13:41:32,021][20381] Updated weights for policy 1, policy_version 1159716 (0.0007) [2023-12-27 13:41:32,081][20381] Updated weights for policy 1, policy_version 1159726 (0.0006) [2023-12-27 13:41:32,141][20381] Updated weights for policy 1, policy_version 1159736 (0.0006) [2023-12-27 13:41:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 594149376. Throughput: 0: 9958.8, 1: 10327.3. Samples: 594160548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:41:32,363][19170] Avg episode reward: [(0, '980.500'), (1, '764.930')] [2023-12-27 13:41:32,375][20347] Updated weights for policy 0, policy_version 1160851 (0.0009) [2023-12-27 13:41:32,433][20347] Updated weights for policy 0, policy_version 1160861 (0.0010) [2023-12-27 13:41:32,490][20347] Updated weights for policy 0, policy_version 1160871 (0.0010) [2023-12-27 13:41:32,838][20381] Updated weights for policy 1, policy_version 1159746 (0.0008) [2023-12-27 13:41:32,885][20381] Updated weights for policy 1, policy_version 1159756 (0.0007) [2023-12-27 13:41:32,930][20381] Updated weights for policy 1, policy_version 1159766 (0.0008) [2023-12-27 13:41:33,089][20347] Updated weights for policy 0, policy_version 1160881 (0.0010) [2023-12-27 13:41:33,144][20347] Updated weights for policy 0, policy_version 1160891 (0.0005) [2023-12-27 13:41:33,201][20347] Updated weights for policy 0, policy_version 1160901 (0.0005) [2023-12-27 13:41:33,254][20347] Updated weights for policy 0, policy_version 1160911 (0.0010) [2023-12-27 13:41:33,591][20381] Updated weights for policy 1, policy_version 1159776 (0.0007) [2023-12-27 13:41:33,641][20381] Updated weights for policy 1, policy_version 1159786 (0.0005) [2023-12-27 13:41:33,696][20381] Updated weights for policy 1, policy_version 1159796 (0.0005) [2023-12-27 13:41:33,903][20347] Updated weights for policy 0, policy_version 1160921 (0.0008) [2023-12-27 13:41:33,954][20347] Updated weights for policy 0, policy_version 1160931 (0.0007) [2023-12-27 13:41:34,010][20347] Updated weights for policy 0, policy_version 1160941 (0.0008) [2023-12-27 13:41:34,355][20381] Updated weights for policy 1, policy_version 1159806 (0.0008) [2023-12-27 13:41:34,412][20381] Updated weights for policy 1, policy_version 1159816 (0.0007) [2023-12-27 13:41:34,464][20381] Updated weights for policy 1, policy_version 1159826 (0.0007) [2023-12-27 13:41:34,791][20347] Updated weights for policy 0, policy_version 1160951 (0.0006) [2023-12-27 13:41:34,850][20347] Updated weights for policy 0, policy_version 1160961 (0.0005) [2023-12-27 13:41:34,910][20347] Updated weights for policy 0, policy_version 1160971 (0.0008) [2023-12-27 13:41:35,120][20381] Updated weights for policy 1, policy_version 1159836 (0.0008) [2023-12-27 13:41:35,174][20381] Updated weights for policy 1, policy_version 1159846 (0.0007) [2023-12-27 13:41:35,231][20381] Updated weights for policy 1, policy_version 1159856 (0.0007) [2023-12-27 13:41:35,502][20347] Updated weights for policy 0, policy_version 1160981 (0.0008) [2023-12-27 13:41:35,558][20347] Updated weights for policy 0, policy_version 1160991 (0.0008) [2023-12-27 13:41:35,616][20347] Updated weights for policy 0, policy_version 1161001 (0.0010) [2023-12-27 13:41:35,937][20381] Updated weights for policy 1, policy_version 1159866 (0.0007) [2023-12-27 13:41:35,992][20381] Updated weights for policy 1, policy_version 1159876 (0.0008) [2023-12-27 13:41:36,051][20381] Updated weights for policy 1, policy_version 1159886 (0.0005) [2023-12-27 13:41:36,119][20381] Updated weights for policy 1, policy_version 1159896 (0.0006) [2023-12-27 13:41:36,191][20347] Updated weights for policy 0, policy_version 1161011 (0.0009) [2023-12-27 13:41:36,248][20347] Updated weights for policy 0, policy_version 1161021 (0.0009) [2023-12-27 13:41:36,311][20347] Updated weights for policy 0, policy_version 1161031 (0.0011) [2023-12-27 13:41:36,771][20381] Updated weights for policy 1, policy_version 1159906 (0.0006) [2023-12-27 13:41:36,832][20381] Updated weights for policy 1, policy_version 1159916 (0.0005) [2023-12-27 13:41:36,894][20381] Updated weights for policy 1, policy_version 1159926 (0.0006) [2023-12-27 13:41:37,023][20347] Updated weights for policy 0, policy_version 1161041 (0.0011) [2023-12-27 13:41:37,074][20347] Updated weights for policy 0, policy_version 1161051 (0.0010) [2023-12-27 13:41:37,129][20347] Updated weights for policy 0, policy_version 1161061 (0.0010) [2023-12-27 13:41:37,185][20347] Updated weights for policy 0, policy_version 1161071 (0.0010) [2023-12-27 13:41:37,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 594255872. Throughput: 0: 9973.8, 1: 10307.7. Samples: 594223016. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:41:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '735.730')] [2023-12-27 13:41:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001159928_296968192.pth... [2023-12-27 13:41:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001161072_297287680.pth... [2023-12-27 13:41:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001158744_296665088.pth [2023-12-27 13:41:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001159888_296984576.pth [2023-12-27 13:41:37,486][20381] Updated weights for policy 1, policy_version 1159936 (0.0007) [2023-12-27 13:41:37,545][20381] Updated weights for policy 1, policy_version 1159946 (0.0008) [2023-12-27 13:41:37,608][20381] Updated weights for policy 1, policy_version 1159956 (0.0007) [2023-12-27 13:41:37,874][20347] Updated weights for policy 0, policy_version 1161081 (0.0010) [2023-12-27 13:41:37,919][20347] Updated weights for policy 0, policy_version 1161091 (0.0010) [2023-12-27 13:41:37,964][20347] Updated weights for policy 0, policy_version 1161101 (0.0008) [2023-12-27 13:41:38,225][20381] Updated weights for policy 1, policy_version 1159966 (0.0007) [2023-12-27 13:41:38,278][20381] Updated weights for policy 1, policy_version 1159976 (0.0008) [2023-12-27 13:41:38,341][20381] Updated weights for policy 1, policy_version 1159986 (0.0007) [2023-12-27 13:41:38,637][20347] Updated weights for policy 0, policy_version 1161111 (0.0007) [2023-12-27 13:41:38,696][20347] Updated weights for policy 0, policy_version 1161121 (0.0008) [2023-12-27 13:41:38,756][20347] Updated weights for policy 0, policy_version 1161131 (0.0008) [2023-12-27 13:41:39,072][20381] Updated weights for policy 1, policy_version 1159996 (0.0008) [2023-12-27 13:41:39,120][20381] Updated weights for policy 1, policy_version 1160006 (0.0009) [2023-12-27 13:41:39,172][20381] Updated weights for policy 1, policy_version 1160016 (0.0008) [2023-12-27 13:41:39,448][20347] Updated weights for policy 0, policy_version 1161141 (0.0007) [2023-12-27 13:41:39,507][20347] Updated weights for policy 0, policy_version 1161151 (0.0007) [2023-12-27 13:41:39,560][20347] Updated weights for policy 0, policy_version 1161161 (0.0008) [2023-12-27 13:41:39,923][20381] Updated weights for policy 1, policy_version 1160026 (0.0008) [2023-12-27 13:41:39,980][20381] Updated weights for policy 1, policy_version 1160036 (0.0008) [2023-12-27 13:41:40,036][20381] Updated weights for policy 1, policy_version 1160046 (0.0008) [2023-12-27 13:41:40,092][20381] Updated weights for policy 1, policy_version 1160056 (0.0008) [2023-12-27 13:41:40,292][20347] Updated weights for policy 0, policy_version 1161171 (0.0010) [2023-12-27 13:41:40,353][20347] Updated weights for policy 0, policy_version 1161181 (0.0011) [2023-12-27 13:41:40,412][20347] Updated weights for policy 0, policy_version 1161191 (0.0010) [2023-12-27 13:41:40,774][20381] Updated weights for policy 1, policy_version 1160066 (0.0007) [2023-12-27 13:41:40,823][20381] Updated weights for policy 1, policy_version 1160076 (0.0008) [2023-12-27 13:41:40,875][20381] Updated weights for policy 1, policy_version 1160086 (0.0008) [2023-12-27 13:41:41,114][20347] Updated weights for policy 0, policy_version 1161201 (0.0010) [2023-12-27 13:41:41,181][20347] Updated weights for policy 0, policy_version 1161211 (0.0009) [2023-12-27 13:41:41,241][20347] Updated weights for policy 0, policy_version 1161221 (0.0008) [2023-12-27 13:41:41,305][20347] Updated weights for policy 0, policy_version 1161231 (0.0011) [2023-12-27 13:41:41,676][20381] Updated weights for policy 1, policy_version 1160096 (0.0008) [2023-12-27 13:41:41,742][20381] Updated weights for policy 1, policy_version 1160106 (0.0008) [2023-12-27 13:41:41,803][20381] Updated weights for policy 1, policy_version 1160116 (0.0008) [2023-12-27 13:41:42,036][20347] Updated weights for policy 0, policy_version 1161241 (0.0011) [2023-12-27 13:41:42,093][20347] Updated weights for policy 0, policy_version 1161251 (0.0010) [2023-12-27 13:41:42,155][20347] Updated weights for policy 0, policy_version 1161261 (0.0010) [2023-12-27 13:41:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 594354176. Throughput: 0: 10053.4, 1: 10256.5. Samples: 594345420. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:41:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '755.460')] [2023-12-27 13:41:42,519][20381] Updated weights for policy 1, policy_version 1160126 (0.0007) [2023-12-27 13:41:42,581][20381] Updated weights for policy 1, policy_version 1160136 (0.0006) [2023-12-27 13:41:42,640][20381] Updated weights for policy 1, policy_version 1160146 (0.0006) [2023-12-27 13:41:42,855][20347] Updated weights for policy 0, policy_version 1161271 (0.0009) [2023-12-27 13:41:42,916][20347] Updated weights for policy 0, policy_version 1161281 (0.0010) [2023-12-27 13:41:42,971][20347] Updated weights for policy 0, policy_version 1161291 (0.0010) [2023-12-27 13:41:43,264][20381] Updated weights for policy 1, policy_version 1160156 (0.0006) [2023-12-27 13:41:43,328][20381] Updated weights for policy 1, policy_version 1160166 (0.0008) [2023-12-27 13:41:43,375][20381] Updated weights for policy 1, policy_version 1160176 (0.0007) [2023-12-27 13:41:43,656][20347] Updated weights for policy 0, policy_version 1161301 (0.0009) [2023-12-27 13:41:43,712][20347] Updated weights for policy 0, policy_version 1161311 (0.0006) [2023-12-27 13:41:43,766][20347] Updated weights for policy 0, policy_version 1161321 (0.0005) [2023-12-27 13:41:43,974][20381] Updated weights for policy 1, policy_version 1160186 (0.0007) [2023-12-27 13:41:44,024][20381] Updated weights for policy 1, policy_version 1160196 (0.0007) [2023-12-27 13:41:44,072][20381] Updated weights for policy 1, policy_version 1160206 (0.0005) [2023-12-27 13:41:44,121][20381] Updated weights for policy 1, policy_version 1160216 (0.0005) [2023-12-27 13:41:44,334][20347] Updated weights for policy 0, policy_version 1161331 (0.0006) [2023-12-27 13:41:44,385][20347] Updated weights for policy 0, policy_version 1161341 (0.0008) [2023-12-27 13:41:44,395][20291] KL-divergence is very high: 117.3259 [2023-12-27 13:41:44,420][20291] KL-divergence is very high: 244.7944 [2023-12-27 13:41:44,426][20291] KL-divergence is very high: 104.0494 [2023-12-27 13:41:44,442][20347] Updated weights for policy 0, policy_version 1161351 (0.0007) [2023-12-27 13:41:44,807][20381] Updated weights for policy 1, policy_version 1160226 (0.0008) [2023-12-27 13:41:44,870][20381] Updated weights for policy 1, policy_version 1160236 (0.0008) [2023-12-27 13:41:44,938][20381] Updated weights for policy 1, policy_version 1160246 (0.0008) [2023-12-27 13:41:45,060][20347] Updated weights for policy 0, policy_version 1161361 (0.0005) [2023-12-27 13:41:45,113][20347] Updated weights for policy 0, policy_version 1161371 (0.0008) [2023-12-27 13:41:45,169][20347] Updated weights for policy 0, policy_version 1161381 (0.0008) [2023-12-27 13:41:45,220][20347] Updated weights for policy 0, policy_version 1161391 (0.0008) [2023-12-27 13:41:45,640][20381] Updated weights for policy 1, policy_version 1160256 (0.0007) [2023-12-27 13:41:45,694][20381] Updated weights for policy 1, policy_version 1160266 (0.0009) [2023-12-27 13:41:45,743][20381] Updated weights for policy 1, policy_version 1160276 (0.0009) [2023-12-27 13:41:45,885][20347] Updated weights for policy 0, policy_version 1161401 (0.0008) [2023-12-27 13:41:45,942][20347] Updated weights for policy 0, policy_version 1161411 (0.0005) [2023-12-27 13:41:46,003][20347] Updated weights for policy 0, policy_version 1161421 (0.0005) [2023-12-27 13:41:46,392][20381] Updated weights for policy 1, policy_version 1160286 (0.0008) [2023-12-27 13:41:46,442][20381] Updated weights for policy 1, policy_version 1160296 (0.0009) [2023-12-27 13:41:46,498][20381] Updated weights for policy 1, policy_version 1160306 (0.0009) [2023-12-27 13:41:46,635][20347] Updated weights for policy 0, policy_version 1161431 (0.0007) [2023-12-27 13:41:46,698][20347] Updated weights for policy 0, policy_version 1161441 (0.0008) [2023-12-27 13:41:46,758][20347] Updated weights for policy 0, policy_version 1161451 (0.0006) [2023-12-27 13:41:47,160][20381] Updated weights for policy 1, policy_version 1160316 (0.0008) [2023-12-27 13:41:47,211][20381] Updated weights for policy 1, policy_version 1160326 (0.0007) [2023-12-27 13:41:47,258][20381] Updated weights for policy 1, policy_version 1160336 (0.0009) [2023-12-27 13:41:47,363][19170] Fps is (10 sec: 20480.6, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 594460672. Throughput: 0: 10128.8, 1: 10284.8. Samples: 594469492. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:41:47,364][19170] Avg episode reward: [(0, '912.010'), (1, '794.530')] [2023-12-27 13:41:47,390][20347] Updated weights for policy 0, policy_version 1161461 (0.0007) [2023-12-27 13:41:47,441][20347] Updated weights for policy 0, policy_version 1161471 (0.0008) [2023-12-27 13:41:47,494][20347] Updated weights for policy 0, policy_version 1161481 (0.0005) [2023-12-27 13:41:47,960][20381] Updated weights for policy 1, policy_version 1160346 (0.0008) [2023-12-27 13:41:48,016][20381] Updated weights for policy 1, policy_version 1160356 (0.0008) [2023-12-27 13:41:48,070][20381] Updated weights for policy 1, policy_version 1160366 (0.0008) [2023-12-27 13:41:48,122][20347] Updated weights for policy 0, policy_version 1161491 (0.0006) [2023-12-27 13:41:48,131][20381] Updated weights for policy 1, policy_version 1160376 (0.0008) [2023-12-27 13:41:48,173][20347] Updated weights for policy 0, policy_version 1161501 (0.0007) [2023-12-27 13:41:48,222][20347] Updated weights for policy 0, policy_version 1161511 (0.0006) [2023-12-27 13:41:48,847][20381] Updated weights for policy 1, policy_version 1160386 (0.0005) [2023-12-27 13:41:48,907][20381] Updated weights for policy 1, policy_version 1160396 (0.0005) [2023-12-27 13:41:48,927][20347] Updated weights for policy 0, policy_version 1161521 (0.0005) [2023-12-27 13:41:48,963][20381] Updated weights for policy 1, policy_version 1160406 (0.0005) [2023-12-27 13:41:48,985][20347] Updated weights for policy 0, policy_version 1161531 (0.0008) [2023-12-27 13:41:49,044][20347] Updated weights for policy 0, policy_version 1161541 (0.0008) [2023-12-27 13:41:49,103][20347] Updated weights for policy 0, policy_version 1161551 (0.0008) [2023-12-27 13:41:49,545][20381] Updated weights for policy 1, policy_version 1160416 (0.0007) [2023-12-27 13:41:49,600][20381] Updated weights for policy 1, policy_version 1160426 (0.0006) [2023-12-27 13:41:49,658][20381] Updated weights for policy 1, policy_version 1160436 (0.0006) [2023-12-27 13:41:49,841][20347] Updated weights for policy 0, policy_version 1161561 (0.0008) [2023-12-27 13:41:49,906][20347] Updated weights for policy 0, policy_version 1161571 (0.0008) [2023-12-27 13:41:49,969][20347] Updated weights for policy 0, policy_version 1161581 (0.0008) [2023-12-27 13:41:50,262][20381] Updated weights for policy 1, policy_version 1160446 (0.0006) [2023-12-27 13:41:50,315][20381] Updated weights for policy 1, policy_version 1160456 (0.0006) [2023-12-27 13:41:50,368][20381] Updated weights for policy 1, policy_version 1160466 (0.0008) [2023-12-27 13:41:50,687][20347] Updated weights for policy 0, policy_version 1161591 (0.0007) [2023-12-27 13:41:50,743][20347] Updated weights for policy 0, policy_version 1161601 (0.0008) [2023-12-27 13:41:50,798][20347] Updated weights for policy 0, policy_version 1161611 (0.0008) [2023-12-27 13:41:51,001][20381] Updated weights for policy 1, policy_version 1160476 (0.0009) [2023-12-27 13:41:51,071][20381] Updated weights for policy 1, policy_version 1160486 (0.0010) [2023-12-27 13:41:51,137][20381] Updated weights for policy 1, policy_version 1160496 (0.0010) [2023-12-27 13:41:51,507][20347] Updated weights for policy 0, policy_version 1161621 (0.0007) [2023-12-27 13:41:51,561][20347] Updated weights for policy 0, policy_version 1161631 (0.0007) [2023-12-27 13:41:51,631][20347] Updated weights for policy 0, policy_version 1161641 (0.0008) [2023-12-27 13:41:51,955][20381] Updated weights for policy 1, policy_version 1160506 (0.0010) [2023-12-27 13:41:52,018][20381] Updated weights for policy 1, policy_version 1160516 (0.0008) [2023-12-27 13:41:52,080][20381] Updated weights for policy 1, policy_version 1160526 (0.0008) [2023-12-27 13:41:52,148][20381] Updated weights for policy 1, policy_version 1160536 (0.0007) [2023-12-27 13:41:52,325][20347] Updated weights for policy 0, policy_version 1161651 (0.0008) [2023-12-27 13:41:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 594558976. Throughput: 0: 10133.5, 1: 10285.4. Samples: 594531804. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:41:52,363][19170] Avg episode reward: [(0, '863.330'), (1, '765.040')] [2023-12-27 13:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001160536_297123840.pth... [2023-12-27 13:41:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001159320_296812544.pth [2023-12-27 13:41:52,389][20347] Updated weights for policy 0, policy_version 1161661 (0.0008) [2023-12-27 13:41:52,451][20347] Updated weights for policy 0, policy_version 1161671 (0.0008) [2023-12-27 13:41:52,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001161680_297443328.pth... [2023-12-27 13:41:52,506][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001160464_297132032.pth [2023-12-27 13:41:52,793][20381] Updated weights for policy 1, policy_version 1160546 (0.0007) [2023-12-27 13:41:52,857][20381] Updated weights for policy 1, policy_version 1160556 (0.0005) [2023-12-27 13:41:52,916][20381] Updated weights for policy 1, policy_version 1160566 (0.0006) [2023-12-27 13:41:53,124][20347] Updated weights for policy 0, policy_version 1161681 (0.0010) [2023-12-27 13:41:53,181][20347] Updated weights for policy 0, policy_version 1161691 (0.0010) [2023-12-27 13:41:53,237][20347] Updated weights for policy 0, policy_version 1161701 (0.0008) [2023-12-27 13:41:53,290][20347] Updated weights for policy 0, policy_version 1161711 (0.0005) [2023-12-27 13:41:53,556][20381] Updated weights for policy 1, policy_version 1160576 (0.0006) [2023-12-27 13:41:53,617][20381] Updated weights for policy 1, policy_version 1160586 (0.0005) [2023-12-27 13:41:53,682][20381] Updated weights for policy 1, policy_version 1160596 (0.0010) [2023-12-27 13:41:53,913][20347] Updated weights for policy 0, policy_version 1161721 (0.0008) [2023-12-27 13:41:53,964][20347] Updated weights for policy 0, policy_version 1161731 (0.0007) [2023-12-27 13:41:54,018][20347] Updated weights for policy 0, policy_version 1161741 (0.0007) [2023-12-27 13:41:54,285][20381] Updated weights for policy 1, policy_version 1160606 (0.0009) [2023-12-27 13:41:54,347][20381] Updated weights for policy 1, policy_version 1160616 (0.0008) [2023-12-27 13:41:54,416][20381] Updated weights for policy 1, policy_version 1160626 (0.0008) [2023-12-27 13:41:54,741][20347] Updated weights for policy 0, policy_version 1161751 (0.0008) [2023-12-27 13:41:54,786][20347] Updated weights for policy 0, policy_version 1161761 (0.0008) [2023-12-27 13:41:54,842][20347] Updated weights for policy 0, policy_version 1161771 (0.0007) [2023-12-27 13:41:55,017][20381] Updated weights for policy 1, policy_version 1160636 (0.0008) [2023-12-27 13:41:55,062][20381] Updated weights for policy 1, policy_version 1160646 (0.0010) [2023-12-27 13:41:55,114][20381] Updated weights for policy 1, policy_version 1160656 (0.0010) [2023-12-27 13:41:55,449][20347] Updated weights for policy 0, policy_version 1161781 (0.0006) [2023-12-27 13:41:55,495][20347] Updated weights for policy 0, policy_version 1161791 (0.0005) [2023-12-27 13:41:55,558][20347] Updated weights for policy 0, policy_version 1161801 (0.0005) [2023-12-27 13:41:55,766][20381] Updated weights for policy 1, policy_version 1160666 (0.0009) [2023-12-27 13:41:55,817][20381] Updated weights for policy 1, policy_version 1160676 (0.0005) [2023-12-27 13:41:55,867][20381] Updated weights for policy 1, policy_version 1160686 (0.0010) [2023-12-27 13:41:55,918][20381] Updated weights for policy 1, policy_version 1160696 (0.0009) [2023-12-27 13:41:56,160][20347] Updated weights for policy 0, policy_version 1161811 (0.0006) [2023-12-27 13:41:56,227][20347] Updated weights for policy 0, policy_version 1161821 (0.0006) [2023-12-27 13:41:56,284][20347] Updated weights for policy 0, policy_version 1161831 (0.0008) [2023-12-27 13:41:56,301][20291] KL-divergence is very high: 125.6254 [2023-12-27 13:41:56,325][20291] KL-divergence is very high: 179.7192 [2023-12-27 13:41:56,500][20381] Updated weights for policy 1, policy_version 1160706 (0.0005) [2023-12-27 13:41:56,557][20381] Updated weights for policy 1, policy_version 1160716 (0.0006) [2023-12-27 13:41:56,613][20381] Updated weights for policy 1, policy_version 1160726 (0.0006) [2023-12-27 13:41:57,018][20347] Updated weights for policy 0, policy_version 1161841 (0.0008) [2023-12-27 13:41:57,068][20347] Updated weights for policy 0, policy_version 1161851 (0.0009) [2023-12-27 13:41:57,082][20291] KL-divergence is very high: 170.3797 [2023-12-27 13:41:57,124][20347] Updated weights for policy 0, policy_version 1161861 (0.0007) [2023-12-27 13:41:57,139][20381] Updated weights for policy 1, policy_version 1160736 (0.0006) [2023-12-27 13:41:57,177][20347] Updated weights for policy 0, policy_version 1161871 (0.0007) [2023-12-27 13:41:57,194][20381] Updated weights for policy 1, policy_version 1160746 (0.0006) [2023-12-27 13:41:57,253][20381] Updated weights for policy 1, policy_version 1160756 (0.0005) [2023-12-27 13:41:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 594673664. Throughput: 0: 10148.3, 1: 10358.7. Samples: 594656980. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:41:57,363][19170] Avg episode reward: [(0, '902.410'), (1, '716.090')] [2023-12-27 13:41:57,785][20381] Updated weights for policy 1, policy_version 1160766 (0.0006) [2023-12-27 13:41:57,842][20381] Updated weights for policy 1, policy_version 1160776 (0.0005) [2023-12-27 13:41:57,902][20381] Updated weights for policy 1, policy_version 1160786 (0.0005) [2023-12-27 13:41:57,936][20347] Updated weights for policy 0, policy_version 1161881 (0.0007) [2023-12-27 13:41:57,981][20347] Updated weights for policy 0, policy_version 1161891 (0.0007) [2023-12-27 13:41:58,034][20347] Updated weights for policy 0, policy_version 1161901 (0.0007) [2023-12-27 13:41:58,582][20381] Updated weights for policy 1, policy_version 1160796 (0.0006) [2023-12-27 13:41:58,645][20381] Updated weights for policy 1, policy_version 1160806 (0.0008) [2023-12-27 13:41:58,708][20381] Updated weights for policy 1, policy_version 1160816 (0.0007) [2023-12-27 13:41:58,857][20347] Updated weights for policy 0, policy_version 1161911 (0.0009) [2023-12-27 13:41:58,917][20347] Updated weights for policy 0, policy_version 1161921 (0.0010) [2023-12-27 13:41:58,975][20347] Updated weights for policy 0, policy_version 1161931 (0.0009) [2023-12-27 13:41:59,496][20381] Updated weights for policy 1, policy_version 1160826 (0.0008) [2023-12-27 13:41:59,558][20381] Updated weights for policy 1, policy_version 1160836 (0.0011) [2023-12-27 13:41:59,617][20381] Updated weights for policy 1, policy_version 1160846 (0.0011) [2023-12-27 13:41:59,680][20381] Updated weights for policy 1, policy_version 1160856 (0.0008) [2023-12-27 13:41:59,702][20347] Updated weights for policy 0, policy_version 1161941 (0.0007) [2023-12-27 13:41:59,759][20347] Updated weights for policy 0, policy_version 1161951 (0.0007) [2023-12-27 13:41:59,820][20347] Updated weights for policy 0, policy_version 1161961 (0.0008) [2023-12-27 13:42:00,352][20381] Updated weights for policy 1, policy_version 1160866 (0.0005) [2023-12-27 13:42:00,416][20381] Updated weights for policy 1, policy_version 1160876 (0.0008) [2023-12-27 13:42:00,461][20347] Updated weights for policy 0, policy_version 1161971 (0.0007) [2023-12-27 13:42:00,470][20381] Updated weights for policy 1, policy_version 1160886 (0.0010) [2023-12-27 13:42:00,515][20347] Updated weights for policy 0, policy_version 1161981 (0.0005) [2023-12-27 13:42:00,572][20347] Updated weights for policy 0, policy_version 1161991 (0.0006) [2023-12-27 13:42:01,077][20381] Updated weights for policy 1, policy_version 1160896 (0.0011) [2023-12-27 13:42:01,138][20381] Updated weights for policy 1, policy_version 1160906 (0.0009) [2023-12-27 13:42:01,171][20347] Updated weights for policy 0, policy_version 1162001 (0.0007) [2023-12-27 13:42:01,204][20381] Updated weights for policy 1, policy_version 1160916 (0.0011) [2023-12-27 13:42:01,228][20347] Updated weights for policy 0, policy_version 1162011 (0.0006) [2023-12-27 13:42:01,286][20347] Updated weights for policy 0, policy_version 1162021 (0.0008) [2023-12-27 13:42:01,332][20347] Updated weights for policy 0, policy_version 1162031 (0.0008) [2023-12-27 13:42:01,923][20381] Updated weights for policy 1, policy_version 1160926 (0.0009) [2023-12-27 13:42:01,972][20381] Updated weights for policy 1, policy_version 1160936 (0.0007) [2023-12-27 13:42:02,029][20381] Updated weights for policy 1, policy_version 1160946 (0.0011) [2023-12-27 13:42:02,043][20347] Updated weights for policy 0, policy_version 1162041 (0.0006) [2023-12-27 13:42:02,104][20347] Updated weights for policy 0, policy_version 1162051 (0.0008) [2023-12-27 13:42:02,158][20347] Updated weights for policy 0, policy_version 1162061 (0.0007) [2023-12-27 13:42:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 594771968. Throughput: 0: 10070.7, 1: 10423.1. Samples: 594778444. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:02,363][19170] Avg episode reward: [(0, '902.490'), (1, '774.880')] [2023-12-27 13:42:02,754][20381] Updated weights for policy 1, policy_version 1160956 (0.0010) [2023-12-27 13:42:02,812][20381] Updated weights for policy 1, policy_version 1160966 (0.0010) [2023-12-27 13:42:02,860][20347] Updated weights for policy 0, policy_version 1162071 (0.0007) [2023-12-27 13:42:02,863][20381] Updated weights for policy 1, policy_version 1160976 (0.0010) [2023-12-27 13:42:02,909][20347] Updated weights for policy 0, policy_version 1162081 (0.0005) [2023-12-27 13:42:02,968][20347] Updated weights for policy 0, policy_version 1162091 (0.0007) [2023-12-27 13:42:03,502][20381] Updated weights for policy 1, policy_version 1160986 (0.0009) [2023-12-27 13:42:03,560][20381] Updated weights for policy 1, policy_version 1160996 (0.0005) [2023-12-27 13:42:03,616][20381] Updated weights for policy 1, policy_version 1161006 (0.0005) [2023-12-27 13:42:03,675][20381] Updated weights for policy 1, policy_version 1161016 (0.0006) [2023-12-27 13:42:03,679][20347] Updated weights for policy 0, policy_version 1162101 (0.0008) [2023-12-27 13:42:03,720][20347] Updated weights for policy 0, policy_version 1162111 (0.0008) [2023-12-27 13:42:03,766][20347] Updated weights for policy 0, policy_version 1162121 (0.0007) [2023-12-27 13:42:04,300][20381] Updated weights for policy 1, policy_version 1161026 (0.0011) [2023-12-27 13:42:04,356][20381] Updated weights for policy 1, policy_version 1161036 (0.0011) [2023-12-27 13:42:04,416][20381] Updated weights for policy 1, policy_version 1161046 (0.0011) [2023-12-27 13:42:04,498][20347] Updated weights for policy 0, policy_version 1162131 (0.0006) [2023-12-27 13:42:04,511][20291] KL-divergence is very high: 246.0537 [2023-12-27 13:42:04,549][20347] Updated weights for policy 0, policy_version 1162141 (0.0007) [2023-12-27 13:42:04,553][20291] KL-divergence is very high: 190.7191 [2023-12-27 13:42:04,597][20347] Updated weights for policy 0, policy_version 1162151 (0.0007) [2023-12-27 13:42:05,088][20381] Updated weights for policy 1, policy_version 1161056 (0.0009) [2023-12-27 13:42:05,145][20381] Updated weights for policy 1, policy_version 1161066 (0.0009) [2023-12-27 13:42:05,194][20381] Updated weights for policy 1, policy_version 1161076 (0.0009) [2023-12-27 13:42:05,295][20347] Updated weights for policy 0, policy_version 1162161 (0.0008) [2023-12-27 13:42:05,339][20347] Updated weights for policy 0, policy_version 1162171 (0.0008) [2023-12-27 13:42:05,388][20347] Updated weights for policy 0, policy_version 1162181 (0.0009) [2023-12-27 13:42:05,448][20347] Updated weights for policy 0, policy_version 1162191 (0.0008) [2023-12-27 13:42:05,786][20381] Updated weights for policy 1, policy_version 1161086 (0.0010) [2023-12-27 13:42:05,843][20381] Updated weights for policy 1, policy_version 1161096 (0.0010) [2023-12-27 13:42:05,906][20381] Updated weights for policy 1, policy_version 1161106 (0.0010) [2023-12-27 13:42:06,238][20347] Updated weights for policy 0, policy_version 1162201 (0.0007) [2023-12-27 13:42:06,292][20347] Updated weights for policy 0, policy_version 1162211 (0.0010) [2023-12-27 13:42:06,353][20347] Updated weights for policy 0, policy_version 1162221 (0.0007) [2023-12-27 13:42:06,514][20381] Updated weights for policy 1, policy_version 1161116 (0.0009) [2023-12-27 13:42:06,570][20381] Updated weights for policy 1, policy_version 1161126 (0.0008) [2023-12-27 13:42:06,630][20381] Updated weights for policy 1, policy_version 1161136 (0.0008) [2023-12-27 13:42:06,999][20347] Updated weights for policy 0, policy_version 1162231 (0.0007) [2023-12-27 13:42:07,057][20347] Updated weights for policy 0, policy_version 1162241 (0.0008) [2023-12-27 13:42:07,114][20347] Updated weights for policy 0, policy_version 1162251 (0.0008) [2023-12-27 13:42:07,317][20381] Updated weights for policy 1, policy_version 1161146 (0.0008) [2023-12-27 13:42:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 594870272. Throughput: 0: 10054.7, 1: 10404.9. Samples: 594839156. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:07,363][19170] Avg episode reward: [(0, '833.900'), (1, '804.220')] [2023-12-27 13:42:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001162256_297590784.pth... [2023-12-27 13:42:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001161072_297287680.pth [2023-12-27 13:42:07,378][20381] Updated weights for policy 1, policy_version 1161156 (0.0007) [2023-12-27 13:42:07,437][20381] Updated weights for policy 1, policy_version 1161166 (0.0006) [2023-12-27 13:42:07,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001161176_297287680.pth... [2023-12-27 13:42:07,496][20381] Updated weights for policy 1, policy_version 1161176 (0.0005) [2023-12-27 13:42:07,497][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001159928_296968192.pth [2023-12-27 13:42:07,799][20347] Updated weights for policy 0, policy_version 1162261 (0.0010) [2023-12-27 13:42:07,844][20347] Updated weights for policy 0, policy_version 1162271 (0.0010) [2023-12-27 13:42:07,895][20347] Updated weights for policy 0, policy_version 1162281 (0.0010) [2023-12-27 13:42:08,041][20381] Updated weights for policy 1, policy_version 1161186 (0.0007) [2023-12-27 13:42:08,087][20381] Updated weights for policy 1, policy_version 1161196 (0.0008) [2023-12-27 13:42:08,141][20381] Updated weights for policy 1, policy_version 1161206 (0.0009) [2023-12-27 13:42:08,593][20347] Updated weights for policy 0, policy_version 1162291 (0.0009) [2023-12-27 13:42:08,651][20347] Updated weights for policy 0, policy_version 1162301 (0.0007) [2023-12-27 13:42:08,706][20347] Updated weights for policy 0, policy_version 1162311 (0.0008) [2023-12-27 13:42:08,849][20381] Updated weights for policy 1, policy_version 1161216 (0.0009) [2023-12-27 13:42:08,899][20381] Updated weights for policy 1, policy_version 1161226 (0.0006) [2023-12-27 13:42:08,950][20381] Updated weights for policy 1, policy_version 1161236 (0.0005) [2023-12-27 13:42:09,426][20347] Updated weights for policy 0, policy_version 1162321 (0.0008) [2023-12-27 13:42:09,485][20347] Updated weights for policy 0, policy_version 1162331 (0.0008) [2023-12-27 13:42:09,545][20347] Updated weights for policy 0, policy_version 1162341 (0.0008) [2023-12-27 13:42:09,597][20347] Updated weights for policy 0, policy_version 1162351 (0.0007) [2023-12-27 13:42:09,611][20381] Updated weights for policy 1, policy_version 1161246 (0.0007) [2023-12-27 13:42:09,678][20381] Updated weights for policy 1, policy_version 1161256 (0.0007) [2023-12-27 13:42:09,743][20381] Updated weights for policy 1, policy_version 1161266 (0.0007) [2023-12-27 13:42:10,339][20347] Updated weights for policy 0, policy_version 1162361 (0.0007) [2023-12-27 13:42:10,397][20347] Updated weights for policy 0, policy_version 1162371 (0.0007) [2023-12-27 13:42:10,431][20381] Updated weights for policy 1, policy_version 1161276 (0.0006) [2023-12-27 13:42:10,452][20347] Updated weights for policy 0, policy_version 1162381 (0.0008) [2023-12-27 13:42:10,484][20381] Updated weights for policy 1, policy_version 1161286 (0.0006) [2023-12-27 13:42:10,547][20381] Updated weights for policy 1, policy_version 1161296 (0.0006) [2023-12-27 13:42:11,099][20347] Updated weights for policy 0, policy_version 1162391 (0.0007) [2023-12-27 13:42:11,160][20347] Updated weights for policy 0, policy_version 1162401 (0.0009) [2023-12-27 13:42:11,207][20381] Updated weights for policy 1, policy_version 1161306 (0.0008) [2023-12-27 13:42:11,216][20347] Updated weights for policy 0, policy_version 1162411 (0.0007) [2023-12-27 13:42:11,277][20381] Updated weights for policy 1, policy_version 1161316 (0.0011) [2023-12-27 13:42:11,341][20381] Updated weights for policy 1, policy_version 1161326 (0.0011) [2023-12-27 13:42:11,414][20381] Updated weights for policy 1, policy_version 1161336 (0.0010) [2023-12-27 13:42:11,970][20347] Updated weights for policy 0, policy_version 1162421 (0.0007) [2023-12-27 13:42:12,022][20347] Updated weights for policy 0, policy_version 1162431 (0.0007) [2023-12-27 13:42:12,065][20381] Updated weights for policy 1, policy_version 1161346 (0.0011) [2023-12-27 13:42:12,082][20347] Updated weights for policy 0, policy_version 1162441 (0.0006) [2023-12-27 13:42:12,124][20381] Updated weights for policy 1, policy_version 1161356 (0.0011) [2023-12-27 13:42:12,181][20381] Updated weights for policy 1, policy_version 1161366 (0.0011) [2023-12-27 13:42:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 594976768. Throughput: 0: 10110.0, 1: 10478.7. Samples: 594963052. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:12,363][19170] Avg episode reward: [(0, '902.260'), (1, '784.520')] [2023-12-27 13:42:12,839][20347] Updated weights for policy 0, policy_version 1162451 (0.0008) [2023-12-27 13:42:12,850][20381] Updated weights for policy 1, policy_version 1161376 (0.0011) [2023-12-27 13:42:12,887][20347] Updated weights for policy 0, policy_version 1162461 (0.0006) [2023-12-27 13:42:12,908][20381] Updated weights for policy 1, policy_version 1161386 (0.0009) [2023-12-27 13:42:12,936][20347] Updated weights for policy 0, policy_version 1162471 (0.0005) [2023-12-27 13:42:12,962][20381] Updated weights for policy 1, policy_version 1161396 (0.0005) [2023-12-27 13:42:13,578][20381] Updated weights for policy 1, policy_version 1161406 (0.0007) [2023-12-27 13:42:13,636][20381] Updated weights for policy 1, policy_version 1161416 (0.0007) [2023-12-27 13:42:13,683][20347] Updated weights for policy 0, policy_version 1162481 (0.0007) [2023-12-27 13:42:13,693][20381] Updated weights for policy 1, policy_version 1161426 (0.0007) [2023-12-27 13:42:13,746][20347] Updated weights for policy 0, policy_version 1162491 (0.0007) [2023-12-27 13:42:13,802][20347] Updated weights for policy 0, policy_version 1162501 (0.0008) [2023-12-27 13:42:13,854][20347] Updated weights for policy 0, policy_version 1162511 (0.0008) [2023-12-27 13:42:14,375][20381] Updated weights for policy 1, policy_version 1161436 (0.0006) [2023-12-27 13:42:14,427][20381] Updated weights for policy 1, policy_version 1161446 (0.0005) [2023-12-27 13:42:14,479][20381] Updated weights for policy 1, policy_version 1161456 (0.0006) [2023-12-27 13:42:14,541][20347] Updated weights for policy 0, policy_version 1162521 (0.0005) [2023-12-27 13:42:14,600][20347] Updated weights for policy 0, policy_version 1162531 (0.0005) [2023-12-27 13:42:14,656][20347] Updated weights for policy 0, policy_version 1162541 (0.0005) [2023-12-27 13:42:15,202][20381] Updated weights for policy 1, policy_version 1161466 (0.0006) [2023-12-27 13:42:15,251][20347] Updated weights for policy 0, policy_version 1162551 (0.0006) [2023-12-27 13:42:15,260][20381] Updated weights for policy 1, policy_version 1161476 (0.0006) [2023-12-27 13:42:15,309][20347] Updated weights for policy 0, policy_version 1162561 (0.0007) [2023-12-27 13:42:15,321][20381] Updated weights for policy 1, policy_version 1161486 (0.0009) [2023-12-27 13:42:15,360][20347] Updated weights for policy 0, policy_version 1162571 (0.0007) [2023-12-27 13:42:15,379][20381] Updated weights for policy 1, policy_version 1161496 (0.0008) [2023-12-27 13:42:15,963][20347] Updated weights for policy 0, policy_version 1162581 (0.0006) [2023-12-27 13:42:16,023][20347] Updated weights for policy 0, policy_version 1162591 (0.0005) [2023-12-27 13:42:16,073][20381] Updated weights for policy 1, policy_version 1161506 (0.0007) [2023-12-27 13:42:16,080][20347] Updated weights for policy 0, policy_version 1162601 (0.0005) [2023-12-27 13:42:16,128][20381] Updated weights for policy 1, policy_version 1161516 (0.0007) [2023-12-27 13:42:16,176][20381] Updated weights for policy 1, policy_version 1161526 (0.0007) [2023-12-27 13:42:16,709][20347] Updated weights for policy 0, policy_version 1162611 (0.0007) [2023-12-27 13:42:16,769][20347] Updated weights for policy 0, policy_version 1162621 (0.0006) [2023-12-27 13:42:16,832][20347] Updated weights for policy 0, policy_version 1162631 (0.0007) [2023-12-27 13:42:16,878][20381] Updated weights for policy 1, policy_version 1161536 (0.0008) [2023-12-27 13:42:16,934][20381] Updated weights for policy 1, policy_version 1161546 (0.0008) [2023-12-27 13:42:16,998][20381] Updated weights for policy 1, policy_version 1161556 (0.0008) [2023-12-27 13:42:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 595075072. Throughput: 0: 10135.4, 1: 10411.9. Samples: 595085176. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:17,363][19170] Avg episode reward: [(0, '893.210'), (1, '853.160')] [2023-12-27 13:42:17,450][20347] Updated weights for policy 0, policy_version 1162641 (0.0006) [2023-12-27 13:42:17,503][20347] Updated weights for policy 0, policy_version 1162651 (0.0008) [2023-12-27 13:42:17,556][20347] Updated weights for policy 0, policy_version 1162661 (0.0009) [2023-12-27 13:42:17,605][20347] Updated weights for policy 0, policy_version 1162671 (0.0009) [2023-12-27 13:42:17,654][20381] Updated weights for policy 1, policy_version 1161566 (0.0008) [2023-12-27 13:42:17,705][20381] Updated weights for policy 1, policy_version 1161576 (0.0008) [2023-12-27 13:42:17,757][20381] Updated weights for policy 1, policy_version 1161586 (0.0008) [2023-12-27 13:42:18,246][20347] Updated weights for policy 0, policy_version 1162681 (0.0008) [2023-12-27 13:42:18,298][20347] Updated weights for policy 0, policy_version 1162691 (0.0009) [2023-12-27 13:42:18,358][20381] Updated weights for policy 1, policy_version 1161596 (0.0008) [2023-12-27 13:42:18,366][20347] Updated weights for policy 0, policy_version 1162701 (0.0009) [2023-12-27 13:42:18,422][20381] Updated weights for policy 1, policy_version 1161606 (0.0008) [2023-12-27 13:42:18,483][20381] Updated weights for policy 1, policy_version 1161616 (0.0008) [2023-12-27 13:42:19,012][20347] Updated weights for policy 0, policy_version 1162711 (0.0007) [2023-12-27 13:42:19,058][20347] Updated weights for policy 0, policy_version 1162721 (0.0007) [2023-12-27 13:42:19,102][20347] Updated weights for policy 0, policy_version 1162731 (0.0007) [2023-12-27 13:42:19,123][20381] Updated weights for policy 1, policy_version 1161626 (0.0008) [2023-12-27 13:42:19,185][20381] Updated weights for policy 1, policy_version 1161636 (0.0007) [2023-12-27 13:42:19,250][20381] Updated weights for policy 1, policy_version 1161646 (0.0007) [2023-12-27 13:42:19,310][20381] Updated weights for policy 1, policy_version 1161656 (0.0008) [2023-12-27 13:42:19,853][20347] Updated weights for policy 0, policy_version 1162741 (0.0007) [2023-12-27 13:42:19,909][20347] Updated weights for policy 0, policy_version 1162751 (0.0008) [2023-12-27 13:42:19,970][20347] Updated weights for policy 0, policy_version 1162761 (0.0008) [2023-12-27 13:42:20,045][20381] Updated weights for policy 1, policy_version 1161666 (0.0006) [2023-12-27 13:42:20,104][20381] Updated weights for policy 1, policy_version 1161676 (0.0005) [2023-12-27 13:42:20,162][20381] Updated weights for policy 1, policy_version 1161686 (0.0008) [2023-12-27 13:42:20,700][20347] Updated weights for policy 0, policy_version 1162771 (0.0008) [2023-12-27 13:42:20,762][20347] Updated weights for policy 0, policy_version 1162781 (0.0008) [2023-12-27 13:42:20,822][20347] Updated weights for policy 0, policy_version 1162791 (0.0007) [2023-12-27 13:42:20,847][20381] Updated weights for policy 1, policy_version 1161696 (0.0005) [2023-12-27 13:42:20,907][20381] Updated weights for policy 1, policy_version 1161706 (0.0007) [2023-12-27 13:42:20,970][20381] Updated weights for policy 1, policy_version 1161716 (0.0007) [2023-12-27 13:42:21,594][20347] Updated weights for policy 0, policy_version 1162801 (0.0007) [2023-12-27 13:42:21,644][20381] Updated weights for policy 1, policy_version 1161726 (0.0008) [2023-12-27 13:42:21,653][20347] Updated weights for policy 0, policy_version 1162811 (0.0007) [2023-12-27 13:42:21,709][20381] Updated weights for policy 1, policy_version 1161736 (0.0008) [2023-12-27 13:42:21,727][20347] Updated weights for policy 0, policy_version 1162821 (0.0008) [2023-12-27 13:42:21,775][20381] Updated weights for policy 1, policy_version 1161746 (0.0007) [2023-12-27 13:42:21,789][20347] Updated weights for policy 0, policy_version 1162831 (0.0008) [2023-12-27 13:42:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 595173376. Throughput: 0: 10132.5, 1: 10403.9. Samples: 595147148. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:22,363][19170] Avg episode reward: [(0, '874.080'), (1, '911.920')] [2023-12-27 13:42:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001162832_297738240.pth... [2023-12-27 13:42:22,371][20381] Updated weights for policy 1, policy_version 1161756 (0.0008) [2023-12-27 13:42:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001161680_297443328.pth [2023-12-27 13:42:22,431][20381] Updated weights for policy 1, policy_version 1161766 (0.0008) [2023-12-27 13:42:22,496][20381] Updated weights for policy 1, policy_version 1161776 (0.0007) [2023-12-27 13:42:22,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001161784_297443328.pth... [2023-12-27 13:42:22,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001160536_297123840.pth [2023-12-27 13:42:22,597][20347] Updated weights for policy 0, policy_version 1162841 (0.0008) [2023-12-27 13:42:22,654][20347] Updated weights for policy 0, policy_version 1162851 (0.0008) [2023-12-27 13:42:22,708][20347] Updated weights for policy 0, policy_version 1162861 (0.0007) [2023-12-27 13:42:23,120][20381] Updated weights for policy 1, policy_version 1161786 (0.0008) [2023-12-27 13:42:23,171][20381] Updated weights for policy 1, policy_version 1161796 (0.0007) [2023-12-27 13:42:23,229][20381] Updated weights for policy 1, policy_version 1161806 (0.0005) [2023-12-27 13:42:23,280][20381] Updated weights for policy 1, policy_version 1161816 (0.0008) [2023-12-27 13:42:23,447][20347] Updated weights for policy 0, policy_version 1162871 (0.0008) [2023-12-27 13:42:23,485][20291] KL-divergence is very high: 308.9346 [2023-12-27 13:42:23,493][20347] Updated weights for policy 0, policy_version 1162881 (0.0008) [2023-12-27 13:42:23,495][20291] KL-divergence is very high: 201.9115 [2023-12-27 13:42:23,526][20291] KL-divergence is very high: 226.6868 [2023-12-27 13:42:23,546][20347] Updated weights for policy 0, policy_version 1162891 (0.0008) [2023-12-27 13:42:23,952][20381] Updated weights for policy 1, policy_version 1161826 (0.0008) [2023-12-27 13:42:24,005][20381] Updated weights for policy 1, policy_version 1161836 (0.0008) [2023-12-27 13:42:24,059][20381] Updated weights for policy 1, policy_version 1161846 (0.0008) [2023-12-27 13:42:24,247][20347] Updated weights for policy 0, policy_version 1162901 (0.0006) [2023-12-27 13:42:24,303][20347] Updated weights for policy 0, policy_version 1162911 (0.0005) [2023-12-27 13:42:24,360][20347] Updated weights for policy 0, policy_version 1162921 (0.0005) [2023-12-27 13:42:24,791][20381] Updated weights for policy 1, policy_version 1161856 (0.0010) [2023-12-27 13:42:24,842][20381] Updated weights for policy 1, policy_version 1161866 (0.0010) [2023-12-27 13:42:24,888][20381] Updated weights for policy 1, policy_version 1161876 (0.0010) [2023-12-27 13:42:24,918][20347] Updated weights for policy 0, policy_version 1162931 (0.0006) [2023-12-27 13:42:24,969][20347] Updated weights for policy 0, policy_version 1162941 (0.0007) [2023-12-27 13:42:25,016][20347] Updated weights for policy 0, policy_version 1162951 (0.0007) [2023-12-27 13:42:25,587][20381] Updated weights for policy 1, policy_version 1161886 (0.0007) [2023-12-27 13:42:25,627][20347] Updated weights for policy 0, policy_version 1162961 (0.0006) [2023-12-27 13:42:25,647][20381] Updated weights for policy 1, policy_version 1161896 (0.0008) [2023-12-27 13:42:25,673][20347] Updated weights for policy 0, policy_version 1162971 (0.0008) [2023-12-27 13:42:25,701][20381] Updated weights for policy 1, policy_version 1161906 (0.0010) [2023-12-27 13:42:25,718][20347] Updated weights for policy 0, policy_version 1162981 (0.0005) [2023-12-27 13:42:25,764][20347] Updated weights for policy 0, policy_version 1162991 (0.0007) [2023-12-27 13:42:26,327][20381] Updated weights for policy 1, policy_version 1161916 (0.0010) [2023-12-27 13:42:26,379][20381] Updated weights for policy 1, policy_version 1161926 (0.0010) [2023-12-27 13:42:26,435][20347] Updated weights for policy 0, policy_version 1163001 (0.0008) [2023-12-27 13:42:26,442][20381] Updated weights for policy 1, policy_version 1161936 (0.0011) [2023-12-27 13:42:26,495][20347] Updated weights for policy 0, policy_version 1163011 (0.0006) [2023-12-27 13:42:26,554][20347] Updated weights for policy 0, policy_version 1163021 (0.0007) [2023-12-27 13:42:27,147][20381] Updated weights for policy 1, policy_version 1161946 (0.0010) [2023-12-27 13:42:27,196][20381] Updated weights for policy 1, policy_version 1161956 (0.0010) [2023-12-27 13:42:27,199][20347] Updated weights for policy 0, policy_version 1163031 (0.0006) [2023-12-27 13:42:27,245][20381] Updated weights for policy 1, policy_version 1161966 (0.0010) [2023-12-27 13:42:27,258][20347] Updated weights for policy 0, policy_version 1163041 (0.0006) [2023-12-27 13:42:27,294][20381] Updated weights for policy 1, policy_version 1161976 (0.0008) [2023-12-27 13:42:27,314][20347] Updated weights for policy 0, policy_version 1163051 (0.0008) [2023-12-27 13:42:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 595288064. Throughput: 0: 10113.7, 1: 10429.8. Samples: 595269876. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:27,363][19170] Avg episode reward: [(0, '864.520'), (1, '902.250')] [2023-12-27 13:42:27,953][20347] Updated weights for policy 0, policy_version 1163061 (0.0006) [2023-12-27 13:42:27,971][20381] Updated weights for policy 1, policy_version 1161986 (0.0011) [2023-12-27 13:42:28,007][20347] Updated weights for policy 0, policy_version 1163071 (0.0005) [2023-12-27 13:42:28,020][20381] Updated weights for policy 1, policy_version 1161996 (0.0010) [2023-12-27 13:42:28,055][20347] Updated weights for policy 0, policy_version 1163081 (0.0005) [2023-12-27 13:42:28,064][20381] Updated weights for policy 1, policy_version 1162006 (0.0010) [2023-12-27 13:42:28,643][20347] Updated weights for policy 0, policy_version 1163091 (0.0005) [2023-12-27 13:42:28,655][20291] KL-divergence is very high: 417.1826 [2023-12-27 13:42:28,669][20291] KL-divergence is very high: 234.0475 [2023-12-27 13:42:28,675][20291] KL-divergence is very high: 219.2506 [2023-12-27 13:42:28,704][20291] KL-divergence is very high: 248.9912 [2023-12-27 13:42:28,705][20347] Updated weights for policy 0, policy_version 1163101 (0.0006) [2023-12-27 13:42:28,722][20381] Updated weights for policy 1, policy_version 1162016 (0.0006) [2023-12-27 13:42:28,762][20347] Updated weights for policy 0, policy_version 1163111 (0.0007) [2023-12-27 13:42:28,778][20381] Updated weights for policy 1, policy_version 1162026 (0.0005) [2023-12-27 13:42:28,837][20381] Updated weights for policy 1, policy_version 1162036 (0.0005) [2023-12-27 13:42:29,457][20381] Updated weights for policy 1, policy_version 1162046 (0.0007) [2023-12-27 13:42:29,465][20347] Updated weights for policy 0, policy_version 1163121 (0.0008) [2023-12-27 13:42:29,514][20381] Updated weights for policy 1, policy_version 1162056 (0.0009) [2023-12-27 13:42:29,517][20347] Updated weights for policy 0, policy_version 1163131 (0.0005) [2023-12-27 13:42:29,573][20381] Updated weights for policy 1, policy_version 1162066 (0.0007) [2023-12-27 13:42:29,573][20347] Updated weights for policy 0, policy_version 1163141 (0.0005) [2023-12-27 13:42:29,633][20347] Updated weights for policy 0, policy_version 1163151 (0.0005) [2023-12-27 13:42:30,237][20381] Updated weights for policy 1, policy_version 1162076 (0.0008) [2023-12-27 13:42:30,287][20347] Updated weights for policy 0, policy_version 1163161 (0.0007) [2023-12-27 13:42:30,293][20381] Updated weights for policy 1, policy_version 1162086 (0.0006) [2023-12-27 13:42:30,337][20347] Updated weights for policy 0, policy_version 1163171 (0.0006) [2023-12-27 13:42:30,348][20381] Updated weights for policy 1, policy_version 1162096 (0.0006) [2023-12-27 13:42:30,396][20347] Updated weights for policy 0, policy_version 1163181 (0.0006) [2023-12-27 13:42:31,051][20381] Updated weights for policy 1, policy_version 1162106 (0.0007) [2023-12-27 13:42:31,084][20347] Updated weights for policy 0, policy_version 1163191 (0.0010) [2023-12-27 13:42:31,108][20381] Updated weights for policy 1, policy_version 1162116 (0.0006) [2023-12-27 13:42:31,144][20347] Updated weights for policy 0, policy_version 1163201 (0.0011) [2023-12-27 13:42:31,171][20381] Updated weights for policy 1, policy_version 1162126 (0.0007) [2023-12-27 13:42:31,203][20347] Updated weights for policy 0, policy_version 1163211 (0.0006) [2023-12-27 13:42:31,216][20381] Updated weights for policy 1, policy_version 1162136 (0.0006) [2023-12-27 13:42:31,932][20347] Updated weights for policy 0, policy_version 1163221 (0.0009) [2023-12-27 13:42:31,970][20381] Updated weights for policy 1, policy_version 1162146 (0.0008) [2023-12-27 13:42:31,984][20347] Updated weights for policy 0, policy_version 1163231 (0.0010) [2023-12-27 13:42:32,027][20381] Updated weights for policy 1, policy_version 1162156 (0.0006) [2023-12-27 13:42:32,037][20347] Updated weights for policy 0, policy_version 1163241 (0.0010) [2023-12-27 13:42:32,089][20381] Updated weights for policy 1, policy_version 1162166 (0.0007) [2023-12-27 13:42:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 595386368. Throughput: 0: 10109.7, 1: 10421.4. Samples: 595393384. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:32,363][19170] Avg episode reward: [(0, '825.500'), (1, '921.600')] [2023-12-27 13:42:32,697][20347] Updated weights for policy 0, policy_version 1163251 (0.0007) [2023-12-27 13:42:32,750][20347] Updated weights for policy 0, policy_version 1163261 (0.0010) [2023-12-27 13:42:32,789][20381] Updated weights for policy 1, policy_version 1162176 (0.0006) [2023-12-27 13:42:32,803][20347] Updated weights for policy 0, policy_version 1163271 (0.0010) [2023-12-27 13:42:32,836][20381] Updated weights for policy 1, policy_version 1162186 (0.0005) [2023-12-27 13:42:32,879][20381] Updated weights for policy 1, policy_version 1162196 (0.0007) [2023-12-27 13:42:33,419][20347] Updated weights for policy 0, policy_version 1163281 (0.0010) [2023-12-27 13:42:33,469][20347] Updated weights for policy 0, policy_version 1163291 (0.0009) [2023-12-27 13:42:33,523][20347] Updated weights for policy 0, policy_version 1163301 (0.0010) [2023-12-27 13:42:33,578][20347] Updated weights for policy 0, policy_version 1163311 (0.0010) [2023-12-27 13:42:33,608][20381] Updated weights for policy 1, policy_version 1162206 (0.0006) [2023-12-27 13:42:33,665][20381] Updated weights for policy 1, policy_version 1162216 (0.0005) [2023-12-27 13:42:33,718][20381] Updated weights for policy 1, policy_version 1162226 (0.0007) [2023-12-27 13:42:34,211][20347] Updated weights for policy 0, policy_version 1163321 (0.0009) [2023-12-27 13:42:34,263][20347] Updated weights for policy 0, policy_version 1163331 (0.0008) [2023-12-27 13:42:34,318][20347] Updated weights for policy 0, policy_version 1163341 (0.0009) [2023-12-27 13:42:34,470][20381] Updated weights for policy 1, policy_version 1162236 (0.0007) [2023-12-27 13:42:34,528][20381] Updated weights for policy 1, policy_version 1162246 (0.0007) [2023-12-27 13:42:34,584][20381] Updated weights for policy 1, policy_version 1162256 (0.0007) [2023-12-27 13:42:35,022][20347] Updated weights for policy 0, policy_version 1163351 (0.0008) [2023-12-27 13:42:35,085][20347] Updated weights for policy 0, policy_version 1163361 (0.0006) [2023-12-27 13:42:35,143][20347] Updated weights for policy 0, policy_version 1163371 (0.0010) [2023-12-27 13:42:35,284][20381] Updated weights for policy 1, policy_version 1162266 (0.0007) [2023-12-27 13:42:35,331][20381] Updated weights for policy 1, policy_version 1162276 (0.0008) [2023-12-27 13:42:35,384][20381] Updated weights for policy 1, policy_version 1162286 (0.0007) [2023-12-27 13:42:35,432][20381] Updated weights for policy 1, policy_version 1162296 (0.0007) [2023-12-27 13:42:35,751][20347] Updated weights for policy 0, policy_version 1163381 (0.0008) [2023-12-27 13:42:35,816][20347] Updated weights for policy 0, policy_version 1163391 (0.0005) [2023-12-27 13:42:35,881][20347] Updated weights for policy 0, policy_version 1163401 (0.0006) [2023-12-27 13:42:36,167][20381] Updated weights for policy 1, policy_version 1162306 (0.0007) [2023-12-27 13:42:36,219][20381] Updated weights for policy 1, policy_version 1162316 (0.0008) [2023-12-27 13:42:36,279][20381] Updated weights for policy 1, policy_version 1162326 (0.0008) [2023-12-27 13:42:36,587][20347] Updated weights for policy 0, policy_version 1163411 (0.0008) [2023-12-27 13:42:36,645][20347] Updated weights for policy 0, policy_version 1163421 (0.0007) [2023-12-27 13:42:36,703][20347] Updated weights for policy 0, policy_version 1163431 (0.0008) [2023-12-27 13:42:37,002][20381] Updated weights for policy 1, policy_version 1162336 (0.0008) [2023-12-27 13:42:37,052][20381] Updated weights for policy 1, policy_version 1162346 (0.0008) [2023-12-27 13:42:37,106][20381] Updated weights for policy 1, policy_version 1162356 (0.0007) [2023-12-27 13:42:37,319][20347] Updated weights for policy 0, policy_version 1163441 (0.0006) [2023-12-27 13:42:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.2, 300 sec: 20493.9). Total num frames: 595484672. Throughput: 0: 10156.0, 1: 10363.1. Samples: 595455164. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:37,363][19170] Avg episode reward: [(0, '912.960'), (1, '872.850')] [2023-12-27 13:42:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001162360_297590784.pth... [2023-12-27 13:42:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001161176_297287680.pth [2023-12-27 13:42:37,376][20347] Updated weights for policy 0, policy_version 1163451 (0.0007) [2023-12-27 13:42:37,427][20347] Updated weights for policy 0, policy_version 1163461 (0.0007) [2023-12-27 13:42:37,483][20347] Updated weights for policy 0, policy_version 1163471 (0.0006) [2023-12-27 13:42:37,489][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001163472_297902080.pth... [2023-12-27 13:42:37,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001162256_297590784.pth [2023-12-27 13:42:37,848][20381] Updated weights for policy 1, policy_version 1162366 (0.0007) [2023-12-27 13:42:37,904][20381] Updated weights for policy 1, policy_version 1162376 (0.0007) [2023-12-27 13:42:37,959][20381] Updated weights for policy 1, policy_version 1162386 (0.0007) [2023-12-27 13:42:38,063][20347] Updated weights for policy 0, policy_version 1163481 (0.0005) [2023-12-27 13:42:38,115][20347] Updated weights for policy 0, policy_version 1163491 (0.0005) [2023-12-27 13:42:38,173][20347] Updated weights for policy 0, policy_version 1163501 (0.0005) [2023-12-27 13:42:38,699][20381] Updated weights for policy 1, policy_version 1162396 (0.0007) [2023-12-27 13:42:38,745][20381] Updated weights for policy 1, policy_version 1162406 (0.0007) [2023-12-27 13:42:38,795][20381] Updated weights for policy 1, policy_version 1162416 (0.0006) [2023-12-27 13:42:38,825][20347] Updated weights for policy 0, policy_version 1163511 (0.0006) [2023-12-27 13:42:38,875][20347] Updated weights for policy 0, policy_version 1163521 (0.0007) [2023-12-27 13:42:38,933][20347] Updated weights for policy 0, policy_version 1163531 (0.0008) [2023-12-27 13:42:39,480][20381] Updated weights for policy 1, policy_version 1162426 (0.0005) [2023-12-27 13:42:39,531][20381] Updated weights for policy 1, policy_version 1162436 (0.0007) [2023-12-27 13:42:39,582][20381] Updated weights for policy 1, policy_version 1162446 (0.0007) [2023-12-27 13:42:39,632][20381] Updated weights for policy 1, policy_version 1162456 (0.0008) [2023-12-27 13:42:39,706][20347] Updated weights for policy 0, policy_version 1163541 (0.0006) [2023-12-27 13:42:39,768][20347] Updated weights for policy 0, policy_version 1163551 (0.0007) [2023-12-27 13:42:39,838][20347] Updated weights for policy 0, policy_version 1163561 (0.0008) [2023-12-27 13:42:40,423][20381] Updated weights for policy 1, policy_version 1162466 (0.0010) [2023-12-27 13:42:40,489][20381] Updated weights for policy 1, policy_version 1162476 (0.0011) [2023-12-27 13:42:40,504][20347] Updated weights for policy 0, policy_version 1163571 (0.0006) [2023-12-27 13:42:40,544][20381] Updated weights for policy 1, policy_version 1162486 (0.0010) [2023-12-27 13:42:40,560][20347] Updated weights for policy 0, policy_version 1163581 (0.0006) [2023-12-27 13:42:40,615][20347] Updated weights for policy 0, policy_version 1163591 (0.0007) [2023-12-27 13:42:41,250][20381] Updated weights for policy 1, policy_version 1162496 (0.0007) [2023-12-27 13:42:41,311][20347] Updated weights for policy 0, policy_version 1163601 (0.0007) [2023-12-27 13:42:41,311][20381] Updated weights for policy 1, policy_version 1162506 (0.0008) [2023-12-27 13:42:41,382][20347] Updated weights for policy 0, policy_version 1163611 (0.0006) [2023-12-27 13:42:41,392][20381] Updated weights for policy 1, policy_version 1162516 (0.0008) [2023-12-27 13:42:41,444][20347] Updated weights for policy 0, policy_version 1163621 (0.0008) [2023-12-27 13:42:41,500][20347] Updated weights for policy 0, policy_version 1163631 (0.0008) [2023-12-27 13:42:42,120][20381] Updated weights for policy 1, policy_version 1162526 (0.0008) [2023-12-27 13:42:42,167][20381] Updated weights for policy 1, policy_version 1162536 (0.0007) [2023-12-27 13:42:42,212][20347] Updated weights for policy 0, policy_version 1163641 (0.0008) [2023-12-27 13:42:42,218][20381] Updated weights for policy 1, policy_version 1162546 (0.0006) [2023-12-27 13:42:42,278][20347] Updated weights for policy 0, policy_version 1163651 (0.0008) [2023-12-27 13:42:42,344][20347] Updated weights for policy 0, policy_version 1163661 (0.0008) [2023-12-27 13:42:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 595582976. Throughput: 0: 10172.4, 1: 10265.6. Samples: 595576692. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:42,363][19170] Avg episode reward: [(0, '844.830'), (1, '872.790')] [2023-12-27 13:42:42,963][20381] Updated weights for policy 1, policy_version 1162556 (0.0007) [2023-12-27 13:42:43,008][20347] Updated weights for policy 0, policy_version 1163671 (0.0008) [2023-12-27 13:42:43,017][20381] Updated weights for policy 1, policy_version 1162566 (0.0006) [2023-12-27 13:42:43,063][20347] Updated weights for policy 0, policy_version 1163681 (0.0006) [2023-12-27 13:42:43,072][20381] Updated weights for policy 1, policy_version 1162576 (0.0005) [2023-12-27 13:42:43,115][20347] Updated weights for policy 0, policy_version 1163691 (0.0006) [2023-12-27 13:42:43,730][20347] Updated weights for policy 0, policy_version 1163701 (0.0007) [2023-12-27 13:42:43,745][20381] Updated weights for policy 1, policy_version 1162586 (0.0006) [2023-12-27 13:42:43,790][20347] Updated weights for policy 0, policy_version 1163711 (0.0005) [2023-12-27 13:42:43,791][20381] Updated weights for policy 1, policy_version 1162596 (0.0007) [2023-12-27 13:42:43,846][20347] Updated weights for policy 0, policy_version 1163721 (0.0005) [2023-12-27 13:42:43,847][20381] Updated weights for policy 1, policy_version 1162606 (0.0007) [2023-12-27 13:42:43,899][20381] Updated weights for policy 1, policy_version 1162616 (0.0005) [2023-12-27 13:42:44,415][20347] Updated weights for policy 0, policy_version 1163731 (0.0007) [2023-12-27 13:42:44,472][20347] Updated weights for policy 0, policy_version 1163741 (0.0007) [2023-12-27 13:42:44,529][20347] Updated weights for policy 0, policy_version 1163751 (0.0010) [2023-12-27 13:42:44,632][20381] Updated weights for policy 1, policy_version 1162626 (0.0007) [2023-12-27 13:42:44,695][20381] Updated weights for policy 1, policy_version 1162636 (0.0008) [2023-12-27 13:42:44,758][20381] Updated weights for policy 1, policy_version 1162646 (0.0008) [2023-12-27 13:42:45,242][20347] Updated weights for policy 0, policy_version 1163761 (0.0010) [2023-12-27 13:42:45,298][20347] Updated weights for policy 0, policy_version 1163771 (0.0007) [2023-12-27 13:42:45,356][20347] Updated weights for policy 0, policy_version 1163781 (0.0007) [2023-12-27 13:42:45,383][20381] Updated weights for policy 1, policy_version 1162656 (0.0007) [2023-12-27 13:42:45,413][20347] Updated weights for policy 0, policy_version 1163791 (0.0006) [2023-12-27 13:42:45,440][20381] Updated weights for policy 1, policy_version 1162666 (0.0008) [2023-12-27 13:42:45,496][20381] Updated weights for policy 1, policy_version 1162676 (0.0006) [2023-12-27 13:42:46,054][20347] Updated weights for policy 0, policy_version 1163801 (0.0008) [2023-12-27 13:42:46,097][20347] Updated weights for policy 0, policy_version 1163811 (0.0006) [2023-12-27 13:42:46,110][20381] Updated weights for policy 1, policy_version 1162686 (0.0006) [2023-12-27 13:42:46,146][20347] Updated weights for policy 0, policy_version 1163821 (0.0008) [2023-12-27 13:42:46,164][20381] Updated weights for policy 1, policy_version 1162696 (0.0005) [2023-12-27 13:42:46,225][20381] Updated weights for policy 1, policy_version 1162706 (0.0005) [2023-12-27 13:42:46,793][20381] Updated weights for policy 1, policy_version 1162716 (0.0006) [2023-12-27 13:42:46,854][20381] Updated weights for policy 1, policy_version 1162726 (0.0008) [2023-12-27 13:42:46,893][20347] Updated weights for policy 0, policy_version 1163831 (0.0007) [2023-12-27 13:42:46,910][20381] Updated weights for policy 1, policy_version 1162736 (0.0006) [2023-12-27 13:42:46,938][20347] Updated weights for policy 0, policy_version 1163841 (0.0005) [2023-12-27 13:42:46,988][20347] Updated weights for policy 0, policy_version 1163851 (0.0007) [2023-12-27 13:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 595689472. Throughput: 0: 10258.9, 1: 10218.0. Samples: 595699900. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:47,363][19170] Avg episode reward: [(0, '834.780'), (1, '882.430')] [2023-12-27 13:42:47,585][20381] Updated weights for policy 1, policy_version 1162746 (0.0009) [2023-12-27 13:42:47,641][20381] Updated weights for policy 1, policy_version 1162756 (0.0008) [2023-12-27 13:42:47,693][20381] Updated weights for policy 1, policy_version 1162766 (0.0010) [2023-12-27 13:42:47,707][20347] Updated weights for policy 0, policy_version 1163861 (0.0006) [2023-12-27 13:42:47,745][20381] Updated weights for policy 1, policy_version 1162776 (0.0010) [2023-12-27 13:42:47,756][20347] Updated weights for policy 0, policy_version 1163871 (0.0007) [2023-12-27 13:42:47,815][20347] Updated weights for policy 0, policy_version 1163881 (0.0007) [2023-12-27 13:42:48,412][20381] Updated weights for policy 1, policy_version 1162786 (0.0008) [2023-12-27 13:42:48,469][20381] Updated weights for policy 1, policy_version 1162796 (0.0008) [2023-12-27 13:42:48,515][20347] Updated weights for policy 0, policy_version 1163891 (0.0006) [2023-12-27 13:42:48,527][20381] Updated weights for policy 1, policy_version 1162806 (0.0007) [2023-12-27 13:42:48,575][20347] Updated weights for policy 0, policy_version 1163901 (0.0007) [2023-12-27 13:42:48,634][20347] Updated weights for policy 0, policy_version 1163911 (0.0008) [2023-12-27 13:42:49,227][20381] Updated weights for policy 1, policy_version 1162816 (0.0010) [2023-12-27 13:42:49,292][20381] Updated weights for policy 1, policy_version 1162826 (0.0008) [2023-12-27 13:42:49,349][20347] Updated weights for policy 0, policy_version 1163921 (0.0008) [2023-12-27 13:42:49,352][20381] Updated weights for policy 1, policy_version 1162836 (0.0008) [2023-12-27 13:42:49,407][20347] Updated weights for policy 0, policy_version 1163931 (0.0007) [2023-12-27 13:42:49,467][20347] Updated weights for policy 0, policy_version 1163941 (0.0008) [2023-12-27 13:42:49,523][20347] Updated weights for policy 0, policy_version 1163951 (0.0007) [2023-12-27 13:42:50,001][20381] Updated weights for policy 1, policy_version 1162846 (0.0008) [2023-12-27 13:42:50,059][20381] Updated weights for policy 1, policy_version 1162856 (0.0006) [2023-12-27 13:42:50,122][20381] Updated weights for policy 1, policy_version 1162866 (0.0008) [2023-12-27 13:42:50,285][20347] Updated weights for policy 0, policy_version 1163961 (0.0008) [2023-12-27 13:42:50,340][20347] Updated weights for policy 0, policy_version 1163971 (0.0007) [2023-12-27 13:42:50,398][20347] Updated weights for policy 0, policy_version 1163981 (0.0008) [2023-12-27 13:42:50,842][20381] Updated weights for policy 1, policy_version 1162876 (0.0008) [2023-12-27 13:42:50,894][20381] Updated weights for policy 1, policy_version 1162886 (0.0008) [2023-12-27 13:42:50,956][20381] Updated weights for policy 1, policy_version 1162896 (0.0007) [2023-12-27 13:42:51,088][20347] Updated weights for policy 0, policy_version 1163991 (0.0009) [2023-12-27 13:42:51,155][20347] Updated weights for policy 0, policy_version 1164001 (0.0011) [2023-12-27 13:42:51,220][20347] Updated weights for policy 0, policy_version 1164011 (0.0010) [2023-12-27 13:42:51,695][20381] Updated weights for policy 1, policy_version 1162906 (0.0007) [2023-12-27 13:42:51,759][20381] Updated weights for policy 1, policy_version 1162916 (0.0009) [2023-12-27 13:42:51,805][20381] Updated weights for policy 1, policy_version 1162926 (0.0009) [2023-12-27 13:42:51,862][20381] Updated weights for policy 1, policy_version 1162936 (0.0008) [2023-12-27 13:42:51,899][20347] Updated weights for policy 0, policy_version 1164021 (0.0009) [2023-12-27 13:42:51,959][20347] Updated weights for policy 0, policy_version 1164031 (0.0007) [2023-12-27 13:42:52,011][20347] Updated weights for policy 0, policy_version 1164041 (0.0007) [2023-12-27 13:42:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 595787776. Throughput: 0: 10245.9, 1: 10200.0. Samples: 595759224. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:52,363][19170] Avg episode reward: [(0, '863.820'), (1, '823.910')] [2023-12-27 13:42:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001162936_297738240.pth... [2023-12-27 13:42:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001164048_298049536.pth... [2023-12-27 13:42:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001161784_297443328.pth [2023-12-27 13:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001162832_297738240.pth [2023-12-27 13:42:52,562][20381] Updated weights for policy 1, policy_version 1162946 (0.0008) [2023-12-27 13:42:52,622][20381] Updated weights for policy 1, policy_version 1162956 (0.0008) [2023-12-27 13:42:52,675][20381] Updated weights for policy 1, policy_version 1162966 (0.0008) [2023-12-27 13:42:52,740][20347] Updated weights for policy 0, policy_version 1164051 (0.0008) [2023-12-27 13:42:52,798][20347] Updated weights for policy 0, policy_version 1164061 (0.0007) [2023-12-27 13:42:52,854][20347] Updated weights for policy 0, policy_version 1164071 (0.0008) [2023-12-27 13:42:53,383][20381] Updated weights for policy 1, policy_version 1162976 (0.0008) [2023-12-27 13:42:53,443][20381] Updated weights for policy 1, policy_version 1162986 (0.0007) [2023-12-27 13:42:53,497][20381] Updated weights for policy 1, policy_version 1162996 (0.0008) [2023-12-27 13:42:53,512][20347] Updated weights for policy 0, policy_version 1164081 (0.0007) [2023-12-27 13:42:53,564][20347] Updated weights for policy 0, policy_version 1164091 (0.0007) [2023-12-27 13:42:53,617][20347] Updated weights for policy 0, policy_version 1164101 (0.0008) [2023-12-27 13:42:53,662][20347] Updated weights for policy 0, policy_version 1164111 (0.0007) [2023-12-27 13:42:54,149][20381] Updated weights for policy 1, policy_version 1163006 (0.0009) [2023-12-27 13:42:54,205][20381] Updated weights for policy 1, policy_version 1163016 (0.0010) [2023-12-27 13:42:54,253][20381] Updated weights for policy 1, policy_version 1163026 (0.0010) [2023-12-27 13:42:54,395][20347] Updated weights for policy 0, policy_version 1164121 (0.0007) [2023-12-27 13:42:54,439][20347] Updated weights for policy 0, policy_version 1164131 (0.0007) [2023-12-27 13:42:54,487][20347] Updated weights for policy 0, policy_version 1164141 (0.0007) [2023-12-27 13:42:54,884][20381] Updated weights for policy 1, policy_version 1163036 (0.0006) [2023-12-27 13:42:54,939][20381] Updated weights for policy 1, policy_version 1163046 (0.0006) [2023-12-27 13:42:55,001][20381] Updated weights for policy 1, policy_version 1163056 (0.0006) [2023-12-27 13:42:55,228][20347] Updated weights for policy 0, policy_version 1164151 (0.0006) [2023-12-27 13:42:55,289][20347] Updated weights for policy 0, policy_version 1164161 (0.0006) [2023-12-27 13:42:55,348][20347] Updated weights for policy 0, policy_version 1164171 (0.0006) [2023-12-27 13:42:55,634][20381] Updated weights for policy 1, policy_version 1163066 (0.0007) [2023-12-27 13:42:55,686][20381] Updated weights for policy 1, policy_version 1163076 (0.0010) [2023-12-27 13:42:55,734][20381] Updated weights for policy 1, policy_version 1163086 (0.0010) [2023-12-27 13:42:55,780][20381] Updated weights for policy 1, policy_version 1163096 (0.0010) [2023-12-27 13:42:55,929][20347] Updated weights for policy 0, policy_version 1164181 (0.0008) [2023-12-27 13:42:55,978][20347] Updated weights for policy 0, policy_version 1164191 (0.0008) [2023-12-27 13:42:56,035][20347] Updated weights for policy 0, policy_version 1164201 (0.0007) [2023-12-27 13:42:56,385][20381] Updated weights for policy 1, policy_version 1163106 (0.0007) [2023-12-27 13:42:56,447][20381] Updated weights for policy 1, policy_version 1163116 (0.0008) [2023-12-27 13:42:56,502][20381] Updated weights for policy 1, policy_version 1163126 (0.0008) [2023-12-27 13:42:56,754][20347] Updated weights for policy 0, policy_version 1164211 (0.0008) [2023-12-27 13:42:56,802][20347] Updated weights for policy 0, policy_version 1164221 (0.0008) [2023-12-27 13:42:56,851][20347] Updated weights for policy 0, policy_version 1164231 (0.0007) [2023-12-27 13:42:57,178][20381] Updated weights for policy 1, policy_version 1163136 (0.0008) [2023-12-27 13:42:57,236][20381] Updated weights for policy 1, policy_version 1163146 (0.0007) [2023-12-27 13:42:57,297][20381] Updated weights for policy 1, policy_version 1163156 (0.0008) [2023-12-27 13:42:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 595894272. Throughput: 0: 10244.0, 1: 10149.0. Samples: 595880736. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:42:57,363][19170] Avg episode reward: [(0, '863.510'), (1, '843.550')] [2023-12-27 13:42:57,580][20347] Updated weights for policy 0, policy_version 1164241 (0.0008) [2023-12-27 13:42:57,633][20347] Updated weights for policy 0, policy_version 1164251 (0.0008) [2023-12-27 13:42:57,686][20347] Updated weights for policy 0, policy_version 1164261 (0.0007) [2023-12-27 13:42:57,736][20347] Updated weights for policy 0, policy_version 1164271 (0.0007) [2023-12-27 13:42:57,904][20381] Updated weights for policy 1, policy_version 1163166 (0.0008) [2023-12-27 13:42:57,956][20381] Updated weights for policy 1, policy_version 1163176 (0.0008) [2023-12-27 13:42:58,005][20381] Updated weights for policy 1, policy_version 1163186 (0.0008) [2023-12-27 13:42:58,469][20347] Updated weights for policy 0, policy_version 1164281 (0.0008) [2023-12-27 13:42:58,522][20347] Updated weights for policy 0, policy_version 1164291 (0.0008) [2023-12-27 13:42:58,574][20347] Updated weights for policy 0, policy_version 1164301 (0.0008) [2023-12-27 13:42:58,779][20381] Updated weights for policy 1, policy_version 1163196 (0.0008) [2023-12-27 13:42:58,846][20381] Updated weights for policy 1, policy_version 1163206 (0.0013) [2023-12-27 13:42:58,905][20381] Updated weights for policy 1, policy_version 1163216 (0.0008) [2023-12-27 13:42:59,302][20347] Updated weights for policy 0, policy_version 1164311 (0.0008) [2023-12-27 13:42:59,357][20347] Updated weights for policy 0, policy_version 1164321 (0.0008) [2023-12-27 13:42:59,420][20347] Updated weights for policy 0, policy_version 1164331 (0.0008) [2023-12-27 13:42:59,572][20381] Updated weights for policy 1, policy_version 1163226 (0.0006) [2023-12-27 13:42:59,636][20381] Updated weights for policy 1, policy_version 1163236 (0.0008) [2023-12-27 13:42:59,696][20381] Updated weights for policy 1, policy_version 1163246 (0.0008) [2023-12-27 13:42:59,753][20381] Updated weights for policy 1, policy_version 1163256 (0.0011) [2023-12-27 13:43:00,110][20347] Updated weights for policy 0, policy_version 1164341 (0.0008) [2023-12-27 13:43:00,162][20347] Updated weights for policy 0, policy_version 1164351 (0.0008) [2023-12-27 13:43:00,217][20347] Updated weights for policy 0, policy_version 1164361 (0.0009) [2023-12-27 13:43:00,401][20381] Updated weights for policy 1, policy_version 1163266 (0.0009) [2023-12-27 13:43:00,456][20381] Updated weights for policy 1, policy_version 1163276 (0.0008) [2023-12-27 13:43:00,519][20381] Updated weights for policy 1, policy_version 1163286 (0.0006) [2023-12-27 13:43:00,980][20347] Updated weights for policy 0, policy_version 1164371 (0.0008) [2023-12-27 13:43:01,051][20347] Updated weights for policy 0, policy_version 1164381 (0.0007) [2023-12-27 13:43:01,081][20381] Updated weights for policy 1, policy_version 1163296 (0.0008) [2023-12-27 13:43:01,111][20347] Updated weights for policy 0, policy_version 1164391 (0.0007) [2023-12-27 13:43:01,145][20381] Updated weights for policy 1, policy_version 1163306 (0.0008) [2023-12-27 13:43:01,204][20381] Updated weights for policy 1, policy_version 1163316 (0.0008) [2023-12-27 13:43:01,818][20347] Updated weights for policy 0, policy_version 1164401 (0.0008) [2023-12-27 13:43:01,870][20347] Updated weights for policy 0, policy_version 1164411 (0.0008) [2023-12-27 13:43:01,907][20291] KL-divergence is very high: 118.3169 [2023-12-27 13:43:01,933][20347] Updated weights for policy 0, policy_version 1164421 (0.0008) [2023-12-27 13:43:01,974][20381] Updated weights for policy 1, policy_version 1163326 (0.0006) [2023-12-27 13:43:01,989][20347] Updated weights for policy 0, policy_version 1164431 (0.0007) [2023-12-27 13:43:02,028][20381] Updated weights for policy 1, policy_version 1163336 (0.0007) [2023-12-27 13:43:02,082][20381] Updated weights for policy 1, policy_version 1163346 (0.0007) [2023-12-27 13:43:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 595992576. Throughput: 0: 10162.4, 1: 10165.3. Samples: 595999920. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:02,363][19170] Avg episode reward: [(0, '853.340'), (1, '843.520')] [2023-12-27 13:43:02,723][20347] Updated weights for policy 0, policy_version 1164441 (0.0006) [2023-12-27 13:43:02,768][20381] Updated weights for policy 1, policy_version 1163356 (0.0008) [2023-12-27 13:43:02,781][20347] Updated weights for policy 0, policy_version 1164451 (0.0005) [2023-12-27 13:43:02,820][20381] Updated weights for policy 1, policy_version 1163366 (0.0008) [2023-12-27 13:43:02,830][20347] Updated weights for policy 0, policy_version 1164461 (0.0005) [2023-12-27 13:43:02,868][20381] Updated weights for policy 1, policy_version 1163376 (0.0008) [2023-12-27 13:43:03,442][20347] Updated weights for policy 0, policy_version 1164471 (0.0007) [2023-12-27 13:43:03,497][20347] Updated weights for policy 0, policy_version 1164481 (0.0008) [2023-12-27 13:43:03,515][20381] Updated weights for policy 1, policy_version 1163386 (0.0008) [2023-12-27 13:43:03,549][20347] Updated weights for policy 0, policy_version 1164491 (0.0006) [2023-12-27 13:43:03,563][20381] Updated weights for policy 1, policy_version 1163396 (0.0005) [2023-12-27 13:43:03,624][20381] Updated weights for policy 1, policy_version 1163406 (0.0007) [2023-12-27 13:43:03,683][20381] Updated weights for policy 1, policy_version 1163416 (0.0006) [2023-12-27 13:43:04,289][20347] Updated weights for policy 0, policy_version 1164501 (0.0007) [2023-12-27 13:43:04,323][20381] Updated weights for policy 1, policy_version 1163426 (0.0007) [2023-12-27 13:43:04,340][20347] Updated weights for policy 0, policy_version 1164511 (0.0007) [2023-12-27 13:43:04,382][20381] Updated weights for policy 1, policy_version 1163436 (0.0008) [2023-12-27 13:43:04,401][20347] Updated weights for policy 0, policy_version 1164521 (0.0008) [2023-12-27 13:43:04,436][20381] Updated weights for policy 1, policy_version 1163446 (0.0007) [2023-12-27 13:43:04,977][20381] Updated weights for policy 1, policy_version 1163456 (0.0006) [2023-12-27 13:43:05,027][20381] Updated weights for policy 1, policy_version 1163466 (0.0007) [2023-12-27 13:43:05,043][20347] Updated weights for policy 0, policy_version 1164531 (0.0007) [2023-12-27 13:43:05,079][20381] Updated weights for policy 1, policy_version 1163476 (0.0007) [2023-12-27 13:43:05,095][20347] Updated weights for policy 0, policy_version 1164541 (0.0005) [2023-12-27 13:43:05,150][20347] Updated weights for policy 0, policy_version 1164551 (0.0005) [2023-12-27 13:43:05,776][20347] Updated weights for policy 0, policy_version 1164561 (0.0006) [2023-12-27 13:43:05,777][20381] Updated weights for policy 1, policy_version 1163486 (0.0008) [2023-12-27 13:43:05,818][20292] KL-divergence is very high: 116.6914 [2023-12-27 13:43:05,831][20381] Updated weights for policy 1, policy_version 1163496 (0.0007) [2023-12-27 13:43:05,832][20347] Updated weights for policy 0, policy_version 1164571 (0.0005) [2023-12-27 13:43:05,884][20381] Updated weights for policy 1, policy_version 1163506 (0.0006) [2023-12-27 13:43:05,892][20347] Updated weights for policy 0, policy_version 1164581 (0.0010) [2023-12-27 13:43:05,946][20347] Updated weights for policy 0, policy_version 1164591 (0.0010) [2023-12-27 13:43:06,561][20381] Updated weights for policy 1, policy_version 1163516 (0.0007) [2023-12-27 13:43:06,568][20347] Updated weights for policy 0, policy_version 1164601 (0.0008) [2023-12-27 13:43:06,624][20381] Updated weights for policy 1, policy_version 1163526 (0.0008) [2023-12-27 13:43:06,628][20347] Updated weights for policy 0, policy_version 1164611 (0.0008) [2023-12-27 13:43:06,682][20381] Updated weights for policy 1, policy_version 1163536 (0.0008) [2023-12-27 13:43:06,687][20347] Updated weights for policy 0, policy_version 1164621 (0.0006) [2023-12-27 13:43:07,274][20347] Updated weights for policy 0, policy_version 1164631 (0.0005) [2023-12-27 13:43:07,337][20347] Updated weights for policy 0, policy_version 1164641 (0.0005) [2023-12-27 13:43:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 596090880. Throughput: 0: 10169.6, 1: 10207.3. Samples: 596064108. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:07,363][19170] Avg episode reward: [(0, '843.820'), (1, '833.460')] [2023-12-27 13:43:07,365][20381] Updated weights for policy 1, policy_version 1163546 (0.0008) [2023-12-27 13:43:07,395][20347] Updated weights for policy 0, policy_version 1164651 (0.0007) [2023-12-27 13:43:07,419][20381] Updated weights for policy 1, policy_version 1163556 (0.0005) [2023-12-27 13:43:07,424][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001164656_298205184.pth... [2023-12-27 13:43:07,428][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001163472_297902080.pth [2023-12-27 13:43:07,468][20381] Updated weights for policy 1, policy_version 1163566 (0.0010) [2023-12-27 13:43:07,513][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001163576_297902080.pth... [2023-12-27 13:43:07,514][20381] Updated weights for policy 1, policy_version 1163576 (0.0005) [2023-12-27 13:43:07,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001162360_297590784.pth [2023-12-27 13:43:07,979][20347] Updated weights for policy 0, policy_version 1164661 (0.0007) [2023-12-27 13:43:08,030][20347] Updated weights for policy 0, policy_version 1164671 (0.0007) [2023-12-27 13:43:08,073][20347] Updated weights for policy 0, policy_version 1164681 (0.0007) [2023-12-27 13:43:08,170][20381] Updated weights for policy 1, policy_version 1163586 (0.0010) [2023-12-27 13:43:08,230][20381] Updated weights for policy 1, policy_version 1163596 (0.0010) [2023-12-27 13:43:08,287][20381] Updated weights for policy 1, policy_version 1163606 (0.0010) [2023-12-27 13:43:08,756][20347] Updated weights for policy 0, policy_version 1164691 (0.0008) [2023-12-27 13:43:08,821][20347] Updated weights for policy 0, policy_version 1164701 (0.0006) [2023-12-27 13:43:08,882][20347] Updated weights for policy 0, policy_version 1164711 (0.0009) [2023-12-27 13:43:08,995][20381] Updated weights for policy 1, policy_version 1163616 (0.0011) [2023-12-27 13:43:09,043][20381] Updated weights for policy 1, policy_version 1163626 (0.0010) [2023-12-27 13:43:09,092][20381] Updated weights for policy 1, policy_version 1163636 (0.0010) [2023-12-27 13:43:09,545][20347] Updated weights for policy 0, policy_version 1164721 (0.0009) [2023-12-27 13:43:09,608][20347] Updated weights for policy 0, policy_version 1164731 (0.0007) [2023-12-27 13:43:09,670][20347] Updated weights for policy 0, policy_version 1164741 (0.0008) [2023-12-27 13:43:09,734][20347] Updated weights for policy 0, policy_version 1164751 (0.0009) [2023-12-27 13:43:09,805][20381] Updated weights for policy 1, policy_version 1163646 (0.0009) [2023-12-27 13:43:09,873][20381] Updated weights for policy 1, policy_version 1163656 (0.0010) [2023-12-27 13:43:09,926][20381] Updated weights for policy 1, policy_version 1163666 (0.0010) [2023-12-27 13:43:10,405][20347] Updated weights for policy 0, policy_version 1164761 (0.0007) [2023-12-27 13:43:10,457][20347] Updated weights for policy 0, policy_version 1164771 (0.0007) [2023-12-27 13:43:10,512][20347] Updated weights for policy 0, policy_version 1164781 (0.0010) [2023-12-27 13:43:10,647][20381] Updated weights for policy 1, policy_version 1163676 (0.0009) [2023-12-27 13:43:10,709][20381] Updated weights for policy 1, policy_version 1163686 (0.0011) [2023-12-27 13:43:10,768][20381] Updated weights for policy 1, policy_version 1163696 (0.0008) [2023-12-27 13:43:11,223][20347] Updated weights for policy 0, policy_version 1164791 (0.0011) [2023-12-27 13:43:11,287][20347] Updated weights for policy 0, policy_version 1164801 (0.0009) [2023-12-27 13:43:11,343][20347] Updated weights for policy 0, policy_version 1164811 (0.0008) [2023-12-27 13:43:11,498][20381] Updated weights for policy 1, policy_version 1163706 (0.0006) [2023-12-27 13:43:11,562][20381] Updated weights for policy 1, policy_version 1163716 (0.0009) [2023-12-27 13:43:11,577][20292] KL-divergence is very high: 137.8811 [2023-12-27 13:43:11,626][20381] Updated weights for policy 1, policy_version 1163726 (0.0010) [2023-12-27 13:43:11,694][20381] Updated weights for policy 1, policy_version 1163736 (0.0009) [2023-12-27 13:43:12,064][20347] Updated weights for policy 0, policy_version 1164821 (0.0008) [2023-12-27 13:43:12,121][20347] Updated weights for policy 0, policy_version 1164831 (0.0006) [2023-12-27 13:43:12,173][20347] Updated weights for policy 0, policy_version 1164841 (0.0010) [2023-12-27 13:43:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 596197376. Throughput: 0: 10237.4, 1: 10173.7. Samples: 596188372. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:12,363][19170] Avg episode reward: [(0, '814.570'), (1, '774.720')] [2023-12-27 13:43:12,423][20381] Updated weights for policy 1, policy_version 1163746 (0.0007) [2023-12-27 13:43:12,487][20381] Updated weights for policy 1, policy_version 1163756 (0.0007) [2023-12-27 13:43:12,550][20381] Updated weights for policy 1, policy_version 1163766 (0.0007) [2023-12-27 13:43:12,821][20347] Updated weights for policy 0, policy_version 1164851 (0.0010) [2023-12-27 13:43:12,876][20347] Updated weights for policy 0, policy_version 1164861 (0.0009) [2023-12-27 13:43:12,930][20347] Updated weights for policy 0, policy_version 1164871 (0.0005) [2023-12-27 13:43:13,222][20381] Updated weights for policy 1, policy_version 1163776 (0.0009) [2023-12-27 13:43:13,277][20381] Updated weights for policy 1, policy_version 1163786 (0.0010) [2023-12-27 13:43:13,334][20381] Updated weights for policy 1, policy_version 1163796 (0.0010) [2023-12-27 13:43:13,490][20347] Updated weights for policy 0, policy_version 1164881 (0.0007) [2023-12-27 13:43:13,564][20347] Updated weights for policy 0, policy_version 1164891 (0.0005) [2023-12-27 13:43:13,623][20347] Updated weights for policy 0, policy_version 1164901 (0.0006) [2023-12-27 13:43:13,677][20347] Updated weights for policy 0, policy_version 1164911 (0.0009) [2023-12-27 13:43:13,929][20381] Updated weights for policy 1, policy_version 1163806 (0.0010) [2023-12-27 13:43:13,987][20381] Updated weights for policy 1, policy_version 1163816 (0.0010) [2023-12-27 13:43:14,049][20381] Updated weights for policy 1, policy_version 1163826 (0.0010) [2023-12-27 13:43:14,244][20347] Updated weights for policy 0, policy_version 1164921 (0.0005) [2023-12-27 13:43:14,301][20347] Updated weights for policy 0, policy_version 1164931 (0.0008) [2023-12-27 13:43:14,354][20347] Updated weights for policy 0, policy_version 1164941 (0.0007) [2023-12-27 13:43:14,705][20381] Updated weights for policy 1, policy_version 1163836 (0.0009) [2023-12-27 13:43:14,770][20381] Updated weights for policy 1, policy_version 1163846 (0.0010) [2023-12-27 13:43:14,831][20381] Updated weights for policy 1, policy_version 1163856 (0.0008) [2023-12-27 13:43:14,986][20347] Updated weights for policy 0, policy_version 1164951 (0.0007) [2023-12-27 13:43:15,034][20347] Updated weights for policy 0, policy_version 1164961 (0.0006) [2023-12-27 13:43:15,081][20347] Updated weights for policy 0, policy_version 1164971 (0.0006) [2023-12-27 13:43:15,526][20381] Updated weights for policy 1, policy_version 1163866 (0.0008) [2023-12-27 13:43:15,582][20381] Updated weights for policy 1, policy_version 1163876 (0.0011) [2023-12-27 13:43:15,640][20381] Updated weights for policy 1, policy_version 1163886 (0.0011) [2023-12-27 13:43:15,699][20381] Updated weights for policy 1, policy_version 1163896 (0.0009) [2023-12-27 13:43:15,734][20347] Updated weights for policy 0, policy_version 1164981 (0.0007) [2023-12-27 13:43:15,791][20347] Updated weights for policy 0, policy_version 1164991 (0.0007) [2023-12-27 13:43:15,847][20347] Updated weights for policy 0, policy_version 1165001 (0.0008) [2023-12-27 13:43:16,260][20381] Updated weights for policy 1, policy_version 1163906 (0.0005) [2023-12-27 13:43:16,314][20381] Updated weights for policy 1, policy_version 1163916 (0.0005) [2023-12-27 13:43:16,376][20381] Updated weights for policy 1, policy_version 1163926 (0.0006) [2023-12-27 13:43:16,544][20347] Updated weights for policy 0, policy_version 1165011 (0.0009) [2023-12-27 13:43:16,608][20347] Updated weights for policy 0, policy_version 1165021 (0.0010) [2023-12-27 13:43:16,668][20347] Updated weights for policy 0, policy_version 1165031 (0.0010) [2023-12-27 13:43:16,902][20381] Updated weights for policy 1, policy_version 1163936 (0.0009) [2023-12-27 13:43:16,954][20381] Updated weights for policy 1, policy_version 1163946 (0.0011) [2023-12-27 13:43:17,009][20381] Updated weights for policy 1, policy_version 1163956 (0.0010) [2023-12-27 13:43:17,347][20347] Updated weights for policy 0, policy_version 1165041 (0.0010) [2023-12-27 13:43:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 596303872. Throughput: 0: 10277.1, 1: 10208.4. Samples: 596315232. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:17,363][19170] Avg episode reward: [(0, '863.280'), (1, '765.220')] [2023-12-27 13:43:17,407][20347] Updated weights for policy 0, policy_version 1165051 (0.0009) [2023-12-27 13:43:17,468][20347] Updated weights for policy 0, policy_version 1165061 (0.0010) [2023-12-27 13:43:17,522][20347] Updated weights for policy 0, policy_version 1165071 (0.0010) [2023-12-27 13:43:17,688][20381] Updated weights for policy 1, policy_version 1163966 (0.0011) [2023-12-27 13:43:17,743][20381] Updated weights for policy 1, policy_version 1163976 (0.0006) [2023-12-27 13:43:17,799][20381] Updated weights for policy 1, policy_version 1163986 (0.0005) [2023-12-27 13:43:18,078][20347] Updated weights for policy 0, policy_version 1165081 (0.0010) [2023-12-27 13:43:18,135][20347] Updated weights for policy 0, policy_version 1165091 (0.0010) [2023-12-27 13:43:18,194][20347] Updated weights for policy 0, policy_version 1165101 (0.0010) [2023-12-27 13:43:18,410][20381] Updated weights for policy 1, policy_version 1163996 (0.0006) [2023-12-27 13:43:18,473][20381] Updated weights for policy 1, policy_version 1164006 (0.0006) [2023-12-27 13:43:18,536][20381] Updated weights for policy 1, policy_version 1164016 (0.0009) [2023-12-27 13:43:18,902][20347] Updated weights for policy 0, policy_version 1165111 (0.0008) [2023-12-27 13:43:18,956][20347] Updated weights for policy 0, policy_version 1165121 (0.0007) [2023-12-27 13:43:19,007][20347] Updated weights for policy 0, policy_version 1165131 (0.0007) [2023-12-27 13:43:19,190][20381] Updated weights for policy 1, policy_version 1164026 (0.0010) [2023-12-27 13:43:19,252][20381] Updated weights for policy 1, policy_version 1164036 (0.0009) [2023-12-27 13:43:19,315][20381] Updated weights for policy 1, policy_version 1164046 (0.0010) [2023-12-27 13:43:19,381][20381] Updated weights for policy 1, policy_version 1164056 (0.0008) [2023-12-27 13:43:19,725][20347] Updated weights for policy 0, policy_version 1165141 (0.0007) [2023-12-27 13:43:19,790][20347] Updated weights for policy 0, policy_version 1165151 (0.0007) [2023-12-27 13:43:19,853][20347] Updated weights for policy 0, policy_version 1165161 (0.0008) [2023-12-27 13:43:20,081][20381] Updated weights for policy 1, policy_version 1164066 (0.0010) [2023-12-27 13:43:20,138][20381] Updated weights for policy 1, policy_version 1164076 (0.0007) [2023-12-27 13:43:20,200][20381] Updated weights for policy 1, policy_version 1164086 (0.0010) [2023-12-27 13:43:20,578][20347] Updated weights for policy 0, policy_version 1165171 (0.0008) [2023-12-27 13:43:20,645][20347] Updated weights for policy 0, policy_version 1165181 (0.0006) [2023-12-27 13:43:20,707][20347] Updated weights for policy 0, policy_version 1165191 (0.0007) [2023-12-27 13:43:20,915][20381] Updated weights for policy 1, policy_version 1164096 (0.0007) [2023-12-27 13:43:20,971][20381] Updated weights for policy 1, policy_version 1164106 (0.0008) [2023-12-27 13:43:21,033][20381] Updated weights for policy 1, policy_version 1164116 (0.0010) [2023-12-27 13:43:21,462][20347] Updated weights for policy 0, policy_version 1165201 (0.0008) [2023-12-27 13:43:21,523][20347] Updated weights for policy 0, policy_version 1165211 (0.0008) [2023-12-27 13:43:21,583][20347] Updated weights for policy 0, policy_version 1165221 (0.0008) [2023-12-27 13:43:21,644][20347] Updated weights for policy 0, policy_version 1165231 (0.0008) [2023-12-27 13:43:21,753][20381] Updated weights for policy 1, policy_version 1164126 (0.0009) [2023-12-27 13:43:21,821][20381] Updated weights for policy 1, policy_version 1164136 (0.0009) [2023-12-27 13:43:21,886][20381] Updated weights for policy 1, policy_version 1164146 (0.0006) [2023-12-27 13:43:22,339][20347] Updated weights for policy 0, policy_version 1165241 (0.0008) [2023-12-27 13:43:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 596402176. Throughput: 0: 10243.9, 1: 10238.2. Samples: 596376860. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:22,363][19170] Avg episode reward: [(0, '883.050'), (1, '745.650')] [2023-12-27 13:43:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001164152_298049536.pth... [2023-12-27 13:43:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001162936_297738240.pth [2023-12-27 13:43:22,407][20347] Updated weights for policy 0, policy_version 1165251 (0.0007) [2023-12-27 13:43:22,464][20347] Updated weights for policy 0, policy_version 1165261 (0.0008) [2023-12-27 13:43:22,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001165264_298360832.pth... [2023-12-27 13:43:22,487][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001164048_298049536.pth [2023-12-27 13:43:22,511][20381] Updated weights for policy 1, policy_version 1164156 (0.0009) [2023-12-27 13:43:22,570][20381] Updated weights for policy 1, policy_version 1164166 (0.0009) [2023-12-27 13:43:22,634][20381] Updated weights for policy 1, policy_version 1164176 (0.0010) [2023-12-27 13:43:23,129][20347] Updated weights for policy 0, policy_version 1165271 (0.0006) [2023-12-27 13:43:23,183][20347] Updated weights for policy 0, policy_version 1165281 (0.0005) [2023-12-27 13:43:23,227][20347] Updated weights for policy 0, policy_version 1165291 (0.0005) [2023-12-27 13:43:23,313][20381] Updated weights for policy 1, policy_version 1164186 (0.0011) [2023-12-27 13:43:23,370][20381] Updated weights for policy 1, policy_version 1164196 (0.0010) [2023-12-27 13:43:23,423][20381] Updated weights for policy 1, policy_version 1164206 (0.0005) [2023-12-27 13:43:23,480][20381] Updated weights for policy 1, policy_version 1164216 (0.0006) [2023-12-27 13:43:23,842][20347] Updated weights for policy 0, policy_version 1165301 (0.0007) [2023-12-27 13:43:23,895][20347] Updated weights for policy 0, policy_version 1165311 (0.0008) [2023-12-27 13:43:23,946][20347] Updated weights for policy 0, policy_version 1165321 (0.0007) [2023-12-27 13:43:24,105][20381] Updated weights for policy 1, policy_version 1164226 (0.0007) [2023-12-27 13:43:24,153][20381] Updated weights for policy 1, policy_version 1164236 (0.0008) [2023-12-27 13:43:24,196][20381] Updated weights for policy 1, policy_version 1164246 (0.0007) [2023-12-27 13:43:24,659][20347] Updated weights for policy 0, policy_version 1165331 (0.0007) [2023-12-27 13:43:24,724][20347] Updated weights for policy 0, policy_version 1165341 (0.0005) [2023-12-27 13:43:24,790][20347] Updated weights for policy 0, policy_version 1165351 (0.0007) [2023-12-27 13:43:24,823][20381] Updated weights for policy 1, policy_version 1164256 (0.0006) [2023-12-27 13:43:24,876][20381] Updated weights for policy 1, policy_version 1164266 (0.0005) [2023-12-27 13:43:24,929][20381] Updated weights for policy 1, policy_version 1164276 (0.0005) [2023-12-27 13:43:25,456][20381] Updated weights for policy 1, policy_version 1164286 (0.0005) [2023-12-27 13:43:25,486][20347] Updated weights for policy 0, policy_version 1165361 (0.0008) [2023-12-27 13:43:25,512][20381] Updated weights for policy 1, policy_version 1164296 (0.0005) [2023-12-27 13:43:25,539][20347] Updated weights for policy 0, policy_version 1165371 (0.0007) [2023-12-27 13:43:25,566][20381] Updated weights for policy 1, policy_version 1164306 (0.0005) [2023-12-27 13:43:25,595][20347] Updated weights for policy 0, policy_version 1165381 (0.0005) [2023-12-27 13:43:25,645][20347] Updated weights for policy 0, policy_version 1165391 (0.0007) [2023-12-27 13:43:26,154][20381] Updated weights for policy 1, policy_version 1164316 (0.0006) [2023-12-27 13:43:26,203][20381] Updated weights for policy 1, policy_version 1164326 (0.0005) [2023-12-27 13:43:26,262][20381] Updated weights for policy 1, policy_version 1164336 (0.0005) [2023-12-27 13:43:26,340][20347] Updated weights for policy 0, policy_version 1165401 (0.0009) [2023-12-27 13:43:26,389][20347] Updated weights for policy 0, policy_version 1165411 (0.0010) [2023-12-27 13:43:26,440][20347] Updated weights for policy 0, policy_version 1165421 (0.0010) [2023-12-27 13:43:26,893][20381] Updated weights for policy 1, policy_version 1164346 (0.0007) [2023-12-27 13:43:26,950][20381] Updated weights for policy 1, policy_version 1164356 (0.0008) [2023-12-27 13:43:27,009][20381] Updated weights for policy 1, policy_version 1164366 (0.0007) [2023-12-27 13:43:27,066][20381] Updated weights for policy 1, policy_version 1164376 (0.0007) [2023-12-27 13:43:27,114][20347] Updated weights for policy 0, policy_version 1165431 (0.0007) [2023-12-27 13:43:27,172][20347] Updated weights for policy 0, policy_version 1165441 (0.0008) [2023-12-27 13:43:27,236][20347] Updated weights for policy 0, policy_version 1165451 (0.0006) [2023-12-27 13:43:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 596516864. Throughput: 0: 10180.0, 1: 10390.3. Samples: 596502356. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:27,363][19170] Avg episode reward: [(0, '844.080'), (1, '794.760')] [2023-12-27 13:43:27,642][20381] Updated weights for policy 1, policy_version 1164386 (0.0008) [2023-12-27 13:43:27,702][20381] Updated weights for policy 1, policy_version 1164396 (0.0007) [2023-12-27 13:43:27,764][20381] Updated weights for policy 1, policy_version 1164406 (0.0006) [2023-12-27 13:43:27,876][20347] Updated weights for policy 0, policy_version 1165461 (0.0009) [2023-12-27 13:43:27,921][20347] Updated weights for policy 0, policy_version 1165471 (0.0007) [2023-12-27 13:43:27,972][20347] Updated weights for policy 0, policy_version 1165481 (0.0006) [2023-12-27 13:43:28,432][20381] Updated weights for policy 1, policy_version 1164416 (0.0008) [2023-12-27 13:43:28,491][20381] Updated weights for policy 1, policy_version 1164426 (0.0008) [2023-12-27 13:43:28,549][20381] Updated weights for policy 1, policy_version 1164436 (0.0008) [2023-12-27 13:43:28,567][20292] KL-divergence is very high: 116.3233 [2023-12-27 13:43:28,663][20347] Updated weights for policy 0, policy_version 1165491 (0.0006) [2023-12-27 13:43:28,709][20347] Updated weights for policy 0, policy_version 1165501 (0.0005) [2023-12-27 13:43:28,762][20347] Updated weights for policy 0, policy_version 1165511 (0.0007) [2023-12-27 13:43:29,233][20381] Updated weights for policy 1, policy_version 1164446 (0.0008) [2023-12-27 13:43:29,295][20381] Updated weights for policy 1, policy_version 1164456 (0.0007) [2023-12-27 13:43:29,358][20381] Updated weights for policy 1, policy_version 1164466 (0.0008) [2023-12-27 13:43:29,449][20347] Updated weights for policy 0, policy_version 1165521 (0.0008) [2023-12-27 13:43:29,508][20347] Updated weights for policy 0, policy_version 1165531 (0.0008) [2023-12-27 13:43:29,565][20347] Updated weights for policy 0, policy_version 1165541 (0.0008) [2023-12-27 13:43:29,622][20347] Updated weights for policy 0, policy_version 1165551 (0.0007) [2023-12-27 13:43:30,041][20381] Updated weights for policy 1, policy_version 1164476 (0.0006) [2023-12-27 13:43:30,102][20381] Updated weights for policy 1, policy_version 1164486 (0.0006) [2023-12-27 13:43:30,164][20381] Updated weights for policy 1, policy_version 1164496 (0.0007) [2023-12-27 13:43:30,375][20347] Updated weights for policy 0, policy_version 1165561 (0.0008) [2023-12-27 13:43:30,429][20347] Updated weights for policy 0, policy_version 1165571 (0.0008) [2023-12-27 13:43:30,484][20347] Updated weights for policy 0, policy_version 1165581 (0.0007) [2023-12-27 13:43:30,824][20381] Updated weights for policy 1, policy_version 1164506 (0.0007) [2023-12-27 13:43:30,874][20381] Updated weights for policy 1, policy_version 1164516 (0.0005) [2023-12-27 13:43:30,928][20381] Updated weights for policy 1, policy_version 1164526 (0.0006) [2023-12-27 13:43:30,972][20381] Updated weights for policy 1, policy_version 1164536 (0.0008) [2023-12-27 13:43:31,102][20347] Updated weights for policy 0, policy_version 1165591 (0.0007) [2023-12-27 13:43:31,165][20347] Updated weights for policy 0, policy_version 1165601 (0.0008) [2023-12-27 13:43:31,216][20347] Updated weights for policy 0, policy_version 1165611 (0.0007) [2023-12-27 13:43:31,604][20381] Updated weights for policy 1, policy_version 1164546 (0.0008) [2023-12-27 13:43:31,672][20381] Updated weights for policy 1, policy_version 1164556 (0.0008) [2023-12-27 13:43:31,730][20381] Updated weights for policy 1, policy_version 1164566 (0.0008) [2023-12-27 13:43:31,957][20347] Updated weights for policy 0, policy_version 1165621 (0.0007) [2023-12-27 13:43:32,023][20347] Updated weights for policy 0, policy_version 1165631 (0.0008) [2023-12-27 13:43:32,078][20347] Updated weights for policy 0, policy_version 1165641 (0.0007) [2023-12-27 13:43:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 596615168. Throughput: 0: 10168.4, 1: 10413.2. Samples: 596626068. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:32,363][19170] Avg episode reward: [(0, '873.250'), (1, '716.160')] [2023-12-27 13:43:32,451][20381] Updated weights for policy 1, policy_version 1164576 (0.0008) [2023-12-27 13:43:32,509][20381] Updated weights for policy 1, policy_version 1164586 (0.0007) [2023-12-27 13:43:32,563][20381] Updated weights for policy 1, policy_version 1164596 (0.0005) [2023-12-27 13:43:32,763][20347] Updated weights for policy 0, policy_version 1165651 (0.0007) [2023-12-27 13:43:32,822][20347] Updated weights for policy 0, policy_version 1165661 (0.0008) [2023-12-27 13:43:32,879][20347] Updated weights for policy 0, policy_version 1165671 (0.0007) [2023-12-27 13:43:33,190][20381] Updated weights for policy 1, policy_version 1164606 (0.0006) [2023-12-27 13:43:33,249][20381] Updated weights for policy 1, policy_version 1164616 (0.0005) [2023-12-27 13:43:33,302][20381] Updated weights for policy 1, policy_version 1164626 (0.0008) [2023-12-27 13:43:33,499][20347] Updated weights for policy 0, policy_version 1165681 (0.0008) [2023-12-27 13:43:33,558][20347] Updated weights for policy 0, policy_version 1165691 (0.0005) [2023-12-27 13:43:33,614][20347] Updated weights for policy 0, policy_version 1165701 (0.0007) [2023-12-27 13:43:33,665][20347] Updated weights for policy 0, policy_version 1165711 (0.0007) [2023-12-27 13:43:33,906][20381] Updated weights for policy 1, policy_version 1164636 (0.0006) [2023-12-27 13:43:33,964][20381] Updated weights for policy 1, policy_version 1164646 (0.0006) [2023-12-27 13:43:34,011][20381] Updated weights for policy 1, policy_version 1164656 (0.0006) [2023-12-27 13:43:34,307][20347] Updated weights for policy 0, policy_version 1165721 (0.0008) [2023-12-27 13:43:34,361][20347] Updated weights for policy 0, policy_version 1165731 (0.0007) [2023-12-27 13:43:34,424][20347] Updated weights for policy 0, policy_version 1165741 (0.0007) [2023-12-27 13:43:34,717][20381] Updated weights for policy 1, policy_version 1164666 (0.0007) [2023-12-27 13:43:34,772][20381] Updated weights for policy 1, policy_version 1164676 (0.0008) [2023-12-27 13:43:34,824][20381] Updated weights for policy 1, policy_version 1164686 (0.0007) [2023-12-27 13:43:34,879][20381] Updated weights for policy 1, policy_version 1164696 (0.0007) [2023-12-27 13:43:35,120][20347] Updated weights for policy 0, policy_version 1165751 (0.0009) [2023-12-27 13:43:35,179][20347] Updated weights for policy 0, policy_version 1165762 (0.0010) [2023-12-27 13:43:35,237][20347] Updated weights for policy 0, policy_version 1165772 (0.0008) [2023-12-27 13:43:35,546][20381] Updated weights for policy 1, policy_version 1164706 (0.0008) [2023-12-27 13:43:35,621][20381] Updated weights for policy 1, policy_version 1164716 (0.0007) [2023-12-27 13:43:35,674][20381] Updated weights for policy 1, policy_version 1164726 (0.0007) [2023-12-27 13:43:35,941][20347] Updated weights for policy 0, policy_version 1165782 (0.0007) [2023-12-27 13:43:35,999][20347] Updated weights for policy 0, policy_version 1165792 (0.0007) [2023-12-27 13:43:36,054][20347] Updated weights for policy 0, policy_version 1165802 (0.0007) [2023-12-27 13:43:36,350][20381] Updated weights for policy 1, policy_version 1164736 (0.0010) [2023-12-27 13:43:36,417][20381] Updated weights for policy 1, policy_version 1164746 (0.0011) [2023-12-27 13:43:36,481][20381] Updated weights for policy 1, policy_version 1164756 (0.0011) [2023-12-27 13:43:36,750][20347] Updated weights for policy 0, policy_version 1165812 (0.0009) [2023-12-27 13:43:36,812][20347] Updated weights for policy 0, policy_version 1165822 (0.0011) [2023-12-27 13:43:36,861][20347] Updated weights for policy 0, policy_version 1165832 (0.0010) [2023-12-27 13:43:37,171][20381] Updated weights for policy 1, policy_version 1164766 (0.0011) [2023-12-27 13:43:37,238][20381] Updated weights for policy 1, policy_version 1164776 (0.0011) [2023-12-27 13:43:37,298][20381] Updated weights for policy 1, policy_version 1164786 (0.0006) [2023-12-27 13:43:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 596721664. Throughput: 0: 10223.3, 1: 10435.0. Samples: 596688848. Policy #0 lag: (min: 17.0, avg: 34.7, max: 49.0) [2023-12-27 13:43:37,364][19170] Avg episode reward: [(0, '902.300'), (1, '706.310')] [2023-12-27 13:43:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001164792_298213376.pth... [2023-12-27 13:43:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001165840_298508288.pth... [2023-12-27 13:43:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001163576_297902080.pth [2023-12-27 13:43:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001164656_298205184.pth [2023-12-27 13:43:37,495][20347] Updated weights for policy 0, policy_version 1165842 (0.0010) [2023-12-27 13:43:37,544][20347] Updated weights for policy 0, policy_version 1165852 (0.0010) [2023-12-27 13:43:37,594][20347] Updated weights for policy 0, policy_version 1165862 (0.0011) [2023-12-27 13:43:37,657][20347] Updated weights for policy 0, policy_version 1165872 (0.0010) [2023-12-27 13:43:37,890][20381] Updated weights for policy 1, policy_version 1164796 (0.0008) [2023-12-27 13:43:37,943][20381] Updated weights for policy 1, policy_version 1164806 (0.0010) [2023-12-27 13:43:37,991][20381] Updated weights for policy 1, policy_version 1164816 (0.0010) [2023-12-27 13:43:38,332][20347] Updated weights for policy 0, policy_version 1165882 (0.0010) [2023-12-27 13:43:38,396][20347] Updated weights for policy 0, policy_version 1165892 (0.0011) [2023-12-27 13:43:38,451][20347] Updated weights for policy 0, policy_version 1165902 (0.0010) [2023-12-27 13:43:38,721][20381] Updated weights for policy 1, policy_version 1164826 (0.0010) [2023-12-27 13:43:38,785][20381] Updated weights for policy 1, policy_version 1164836 (0.0005) [2023-12-27 13:43:38,849][20381] Updated weights for policy 1, policy_version 1164846 (0.0007) [2023-12-27 13:43:38,914][20381] Updated weights for policy 1, policy_version 1164856 (0.0008) [2023-12-27 13:43:39,101][20347] Updated weights for policy 0, policy_version 1165912 (0.0010) [2023-12-27 13:43:39,152][20347] Updated weights for policy 0, policy_version 1165922 (0.0010) [2023-12-27 13:43:39,200][20347] Updated weights for policy 0, policy_version 1165932 (0.0010) [2023-12-27 13:43:39,641][20381] Updated weights for policy 1, policy_version 1164866 (0.0006) [2023-12-27 13:43:39,699][20381] Updated weights for policy 1, policy_version 1164876 (0.0008) [2023-12-27 13:43:39,754][20381] Updated weights for policy 1, policy_version 1164886 (0.0008) [2023-12-27 13:43:39,916][20347] Updated weights for policy 0, policy_version 1165942 (0.0009) [2023-12-27 13:43:39,978][20347] Updated weights for policy 0, policy_version 1165952 (0.0010) [2023-12-27 13:43:40,038][20347] Updated weights for policy 0, policy_version 1165962 (0.0011) [2023-12-27 13:43:40,457][20381] Updated weights for policy 1, policy_version 1164896 (0.0007) [2023-12-27 13:43:40,512][20381] Updated weights for policy 1, policy_version 1164906 (0.0009) [2023-12-27 13:43:40,561][20381] Updated weights for policy 1, policy_version 1164916 (0.0007) [2023-12-27 13:43:40,755][20347] Updated weights for policy 0, policy_version 1165972 (0.0010) [2023-12-27 13:43:40,820][20347] Updated weights for policy 0, policy_version 1165982 (0.0010) [2023-12-27 13:43:40,878][20347] Updated weights for policy 0, policy_version 1165992 (0.0010) [2023-12-27 13:43:41,266][20381] Updated weights for policy 1, policy_version 1164926 (0.0008) [2023-12-27 13:43:41,325][20381] Updated weights for policy 1, policy_version 1164936 (0.0008) [2023-12-27 13:43:41,394][20381] Updated weights for policy 1, policy_version 1164946 (0.0007) [2023-12-27 13:43:41,573][20347] Updated weights for policy 0, policy_version 1166002 (0.0010) [2023-12-27 13:43:41,637][20347] Updated weights for policy 0, policy_version 1166012 (0.0010) [2023-12-27 13:43:41,697][20347] Updated weights for policy 0, policy_version 1166022 (0.0008) [2023-12-27 13:43:41,760][20347] Updated weights for policy 0, policy_version 1166032 (0.0009) [2023-12-27 13:43:42,178][20381] Updated weights for policy 1, policy_version 1164956 (0.0008) [2023-12-27 13:43:42,226][20381] Updated weights for policy 1, policy_version 1164966 (0.0007) [2023-12-27 13:43:42,282][20381] Updated weights for policy 1, policy_version 1164976 (0.0007) [2023-12-27 13:43:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 596819968. Throughput: 0: 10236.4, 1: 10418.5. Samples: 596810204. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:43:42,363][19170] Avg episode reward: [(0, '941.310'), (1, '765.100')] [2023-12-27 13:43:42,484][20347] Updated weights for policy 0, policy_version 1166042 (0.0005) [2023-12-27 13:43:42,549][20347] Updated weights for policy 0, policy_version 1166052 (0.0007) [2023-12-27 13:43:42,612][20347] Updated weights for policy 0, policy_version 1166062 (0.0009) [2023-12-27 13:43:43,053][20381] Updated weights for policy 1, policy_version 1164986 (0.0008) [2023-12-27 13:43:43,102][20381] Updated weights for policy 1, policy_version 1164996 (0.0009) [2023-12-27 13:43:43,152][20381] Updated weights for policy 1, policy_version 1165006 (0.0008) [2023-12-27 13:43:43,180][20347] Updated weights for policy 0, policy_version 1166072 (0.0009) [2023-12-27 13:43:43,200][20381] Updated weights for policy 1, policy_version 1165016 (0.0006) [2023-12-27 13:43:43,234][20347] Updated weights for policy 0, policy_version 1166082 (0.0010) [2023-12-27 13:43:43,282][20347] Updated weights for policy 0, policy_version 1166092 (0.0010) [2023-12-27 13:43:43,855][20381] Updated weights for policy 1, policy_version 1165026 (0.0005) [2023-12-27 13:43:43,910][20381] Updated weights for policy 1, policy_version 1165036 (0.0005) [2023-12-27 13:43:43,962][20347] Updated weights for policy 0, policy_version 1166102 (0.0010) [2023-12-27 13:43:43,969][20381] Updated weights for policy 1, policy_version 1165046 (0.0009) [2023-12-27 13:43:43,974][20291] KL-divergence is very high: 110.2140 [2023-12-27 13:43:44,014][20347] Updated weights for policy 0, policy_version 1166112 (0.0010) [2023-12-27 13:43:44,015][20291] KL-divergence is very high: 136.2962 [2023-12-27 13:43:44,020][20291] KL-divergence is very high: 278.2765 [2023-12-27 13:43:44,028][20291] KL-divergence is very high: 497.6172 [2023-12-27 13:43:44,050][20291] KL-divergence is very high: 356.9622 [2023-12-27 13:43:44,065][20291] KL-divergence is very high: 237.7047 [2023-12-27 13:43:44,070][20291] KL-divergence is very high: 324.7780 [2023-12-27 13:43:44,072][20347] Updated weights for policy 0, policy_version 1166122 (0.0010) [2023-12-27 13:43:44,621][20381] Updated weights for policy 1, policy_version 1165056 (0.0008) [2023-12-27 13:43:44,668][20381] Updated weights for policy 1, policy_version 1165066 (0.0007) [2023-12-27 13:43:44,704][20291] KL-divergence is very high: 203.8873 [2023-12-27 13:43:44,714][20381] Updated weights for policy 1, policy_version 1165076 (0.0006) [2023-12-27 13:43:44,720][20347] Updated weights for policy 0, policy_version 1166132 (0.0010) [2023-12-27 13:43:44,767][20347] Updated weights for policy 0, policy_version 1166142 (0.0008) [2023-12-27 13:43:44,832][20347] Updated weights for policy 0, policy_version 1166152 (0.0008) [2023-12-27 13:43:45,457][20381] Updated weights for policy 1, policy_version 1165086 (0.0008) [2023-12-27 13:43:45,502][20381] Updated weights for policy 1, policy_version 1165096 (0.0010) [2023-12-27 13:43:45,546][20347] Updated weights for policy 0, policy_version 1166162 (0.0011) [2023-12-27 13:43:45,559][20381] Updated weights for policy 1, policy_version 1165106 (0.0008) [2023-12-27 13:43:45,595][20347] Updated weights for policy 0, policy_version 1166172 (0.0010) [2023-12-27 13:43:45,644][20347] Updated weights for policy 0, policy_version 1166182 (0.0010) [2023-12-27 13:43:45,701][20347] Updated weights for policy 0, policy_version 1166192 (0.0010) [2023-12-27 13:43:46,290][20381] Updated weights for policy 1, policy_version 1165116 (0.0009) [2023-12-27 13:43:46,340][20381] Updated weights for policy 1, policy_version 1165126 (0.0010) [2023-12-27 13:43:46,381][20347] Updated weights for policy 0, policy_version 1166202 (0.0008) [2023-12-27 13:43:46,407][20381] Updated weights for policy 1, policy_version 1165136 (0.0005) [2023-12-27 13:43:46,442][20347] Updated weights for policy 0, policy_version 1166212 (0.0005) [2023-12-27 13:43:46,492][20347] Updated weights for policy 0, policy_version 1166222 (0.0006) [2023-12-27 13:43:47,009][20381] Updated weights for policy 1, policy_version 1165146 (0.0010) [2023-12-27 13:43:47,059][20381] Updated weights for policy 1, policy_version 1165156 (0.0011) [2023-12-27 13:43:47,107][20347] Updated weights for policy 0, policy_version 1166232 (0.0006) [2023-12-27 13:43:47,116][20381] Updated weights for policy 1, policy_version 1165166 (0.0011) [2023-12-27 13:43:47,161][20347] Updated weights for policy 0, policy_version 1166242 (0.0009) [2023-12-27 13:43:47,173][20381] Updated weights for policy 1, policy_version 1165176 (0.0011) [2023-12-27 13:43:47,219][20347] Updated weights for policy 0, policy_version 1166252 (0.0010) [2023-12-27 13:43:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 596926464. Throughput: 0: 10344.8, 1: 10364.4. Samples: 596931836. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:43:47,363][19170] Avg episode reward: [(0, '863.100'), (1, '745.580')] [2023-12-27 13:43:47,846][20347] Updated weights for policy 0, policy_version 1166262 (0.0010) [2023-12-27 13:43:47,873][20381] Updated weights for policy 1, policy_version 1165186 (0.0008) [2023-12-27 13:43:47,907][20347] Updated weights for policy 0, policy_version 1166272 (0.0010) [2023-12-27 13:43:47,928][20381] Updated weights for policy 1, policy_version 1165196 (0.0008) [2023-12-27 13:43:47,966][20347] Updated weights for policy 0, policy_version 1166282 (0.0011) [2023-12-27 13:43:47,981][20381] Updated weights for policy 1, policy_version 1165206 (0.0008) [2023-12-27 13:43:48,650][20347] Updated weights for policy 0, policy_version 1166292 (0.0011) [2023-12-27 13:43:48,701][20381] Updated weights for policy 1, policy_version 1165216 (0.0011) [2023-12-27 13:43:48,710][20347] Updated weights for policy 0, policy_version 1166302 (0.0011) [2023-12-27 13:43:48,762][20381] Updated weights for policy 1, policy_version 1165226 (0.0011) [2023-12-27 13:43:48,771][20347] Updated weights for policy 0, policy_version 1166312 (0.0011) [2023-12-27 13:43:48,817][20381] Updated weights for policy 1, policy_version 1165236 (0.0010) [2023-12-27 13:43:49,516][20347] Updated weights for policy 0, policy_version 1166322 (0.0009) [2023-12-27 13:43:49,542][20381] Updated weights for policy 1, policy_version 1165246 (0.0008) [2023-12-27 13:43:49,560][20291] KL-divergence is very high: 102.7709 [2023-12-27 13:43:49,579][20347] Updated weights for policy 0, policy_version 1166332 (0.0005) [2023-12-27 13:43:49,605][20381] Updated weights for policy 1, policy_version 1165256 (0.0007) [2023-12-27 13:43:49,635][20347] Updated weights for policy 0, policy_version 1166342 (0.0005) [2023-12-27 13:43:49,663][20381] Updated weights for policy 1, policy_version 1165266 (0.0007) [2023-12-27 13:43:49,695][20347] Updated weights for policy 0, policy_version 1166352 (0.0005) [2023-12-27 13:43:50,264][20381] Updated weights for policy 1, policy_version 1165276 (0.0006) [2023-12-27 13:43:50,329][20381] Updated weights for policy 1, policy_version 1165286 (0.0006) [2023-12-27 13:43:50,381][20347] Updated weights for policy 0, policy_version 1166362 (0.0007) [2023-12-27 13:43:50,388][20381] Updated weights for policy 1, policy_version 1165296 (0.0007) [2023-12-27 13:43:50,441][20347] Updated weights for policy 0, policy_version 1166372 (0.0009) [2023-12-27 13:43:50,499][20347] Updated weights for policy 0, policy_version 1166382 (0.0008) [2023-12-27 13:43:51,076][20381] Updated weights for policy 1, policy_version 1165306 (0.0008) [2023-12-27 13:43:51,136][20381] Updated weights for policy 1, policy_version 1165316 (0.0010) [2023-12-27 13:43:51,196][20381] Updated weights for policy 1, policy_version 1165326 (0.0007) [2023-12-27 13:43:51,205][20347] Updated weights for policy 0, policy_version 1166392 (0.0010) [2023-12-27 13:43:51,261][20381] Updated weights for policy 1, policy_version 1165336 (0.0006) [2023-12-27 13:43:51,263][20347] Updated weights for policy 0, policy_version 1166402 (0.0009) [2023-12-27 13:43:51,323][20347] Updated weights for policy 0, policy_version 1166412 (0.0009) [2023-12-27 13:43:51,925][20381] Updated weights for policy 1, policy_version 1165346 (0.0008) [2023-12-27 13:43:51,988][20381] Updated weights for policy 1, policy_version 1165356 (0.0007) [2023-12-27 13:43:52,014][20347] Updated weights for policy 0, policy_version 1166422 (0.0007) [2023-12-27 13:43:52,045][20381] Updated weights for policy 1, policy_version 1165366 (0.0007) [2023-12-27 13:43:52,069][20347] Updated weights for policy 0, policy_version 1166432 (0.0005) [2023-12-27 13:43:52,129][20347] Updated weights for policy 0, policy_version 1166442 (0.0008) [2023-12-27 13:43:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 597024768. Throughput: 0: 10330.5, 1: 10312.9. Samples: 596993060. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:43:52,363][19170] Avg episode reward: [(0, '873.090'), (1, '726.010')] [2023-12-27 13:43:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001165368_298360832.pth... [2023-12-27 13:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001166448_298663936.pth... [2023-12-27 13:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001164152_298049536.pth [2023-12-27 13:43:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001165264_298360832.pth [2023-12-27 13:43:52,707][20381] Updated weights for policy 1, policy_version 1165376 (0.0007) [2023-12-27 13:43:52,769][20381] Updated weights for policy 1, policy_version 1165386 (0.0008) [2023-12-27 13:43:52,800][20347] Updated weights for policy 0, policy_version 1166452 (0.0008) [2023-12-27 13:43:52,828][20381] Updated weights for policy 1, policy_version 1165396 (0.0006) [2023-12-27 13:43:52,853][20347] Updated weights for policy 0, policy_version 1166462 (0.0005) [2023-12-27 13:43:52,912][20347] Updated weights for policy 0, policy_version 1166472 (0.0009) [2023-12-27 13:43:53,484][20381] Updated weights for policy 1, policy_version 1165406 (0.0009) [2023-12-27 13:43:53,538][20381] Updated weights for policy 1, policy_version 1165416 (0.0010) [2023-12-27 13:43:53,570][20347] Updated weights for policy 0, policy_version 1166482 (0.0010) [2023-12-27 13:43:53,593][20381] Updated weights for policy 1, policy_version 1165426 (0.0010) [2023-12-27 13:43:53,628][20347] Updated weights for policy 0, policy_version 1166492 (0.0010) [2023-12-27 13:43:53,678][20347] Updated weights for policy 0, policy_version 1166502 (0.0010) [2023-12-27 13:43:53,735][20347] Updated weights for policy 0, policy_version 1166512 (0.0010) [2023-12-27 13:43:54,240][20381] Updated weights for policy 1, policy_version 1165436 (0.0008) [2023-12-27 13:43:54,299][20381] Updated weights for policy 1, policy_version 1165446 (0.0007) [2023-12-27 13:43:54,360][20381] Updated weights for policy 1, policy_version 1165456 (0.0005) [2023-12-27 13:43:54,411][20347] Updated weights for policy 0, policy_version 1166522 (0.0010) [2023-12-27 13:43:54,475][20347] Updated weights for policy 0, policy_version 1166532 (0.0010) [2023-12-27 13:43:54,523][20347] Updated weights for policy 0, policy_version 1166542 (0.0010) [2023-12-27 13:43:54,964][20381] Updated weights for policy 1, policy_version 1165466 (0.0007) [2023-12-27 13:43:55,020][20381] Updated weights for policy 1, policy_version 1165476 (0.0005) [2023-12-27 13:43:55,069][20381] Updated weights for policy 1, policy_version 1165486 (0.0005) [2023-12-27 13:43:55,126][20381] Updated weights for policy 1, policy_version 1165496 (0.0005) [2023-12-27 13:43:55,219][20347] Updated weights for policy 0, policy_version 1166552 (0.0010) [2023-12-27 13:43:55,270][20347] Updated weights for policy 0, policy_version 1166562 (0.0010) [2023-12-27 13:43:55,316][20347] Updated weights for policy 0, policy_version 1166572 (0.0010) [2023-12-27 13:43:55,734][20381] Updated weights for policy 1, policy_version 1165506 (0.0010) [2023-12-27 13:43:55,782][20381] Updated weights for policy 1, policy_version 1165516 (0.0010) [2023-12-27 13:43:55,836][20381] Updated weights for policy 1, policy_version 1165526 (0.0010) [2023-12-27 13:43:56,002][20347] Updated weights for policy 0, policy_version 1166582 (0.0010) [2023-12-27 13:43:56,062][20347] Updated weights for policy 0, policy_version 1166592 (0.0010) [2023-12-27 13:43:56,116][20347] Updated weights for policy 0, policy_version 1166602 (0.0010) [2023-12-27 13:43:56,505][20381] Updated weights for policy 1, policy_version 1165536 (0.0006) [2023-12-27 13:43:56,552][20381] Updated weights for policy 1, policy_version 1165546 (0.0006) [2023-12-27 13:43:56,598][20381] Updated weights for policy 1, policy_version 1165556 (0.0005) [2023-12-27 13:43:56,798][20347] Updated weights for policy 0, policy_version 1166612 (0.0010) [2023-12-27 13:43:56,849][20347] Updated weights for policy 0, policy_version 1166622 (0.0010) [2023-12-27 13:43:56,924][20347] Updated weights for policy 0, policy_version 1166632 (0.0010) [2023-12-27 13:43:57,164][20381] Updated weights for policy 1, policy_version 1165566 (0.0007) [2023-12-27 13:43:57,225][20381] Updated weights for policy 1, policy_version 1165576 (0.0009) [2023-12-27 13:43:57,287][20381] Updated weights for policy 1, policy_version 1165586 (0.0009) [2023-12-27 13:43:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 597131264. Throughput: 0: 10253.6, 1: 10366.6. Samples: 597116284. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:43:57,363][19170] Avg episode reward: [(0, '912.110'), (1, '784.630')] [2023-12-27 13:43:57,608][20347] Updated weights for policy 0, policy_version 1166642 (0.0009) [2023-12-27 13:43:57,658][20347] Updated weights for policy 0, policy_version 1166652 (0.0006) [2023-12-27 13:43:57,711][20347] Updated weights for policy 0, policy_version 1166662 (0.0007) [2023-12-27 13:43:57,765][20347] Updated weights for policy 0, policy_version 1166672 (0.0007) [2023-12-27 13:43:57,942][20381] Updated weights for policy 1, policy_version 1165596 (0.0008) [2023-12-27 13:43:57,990][20381] Updated weights for policy 1, policy_version 1165606 (0.0009) [2023-12-27 13:43:58,042][20381] Updated weights for policy 1, policy_version 1165616 (0.0010) [2023-12-27 13:43:58,497][20347] Updated weights for policy 0, policy_version 1166682 (0.0008) [2023-12-27 13:43:58,559][20347] Updated weights for policy 0, policy_version 1166692 (0.0007) [2023-12-27 13:43:58,619][20347] Updated weights for policy 0, policy_version 1166702 (0.0008) [2023-12-27 13:43:58,754][20381] Updated weights for policy 1, policy_version 1165626 (0.0010) [2023-12-27 13:43:58,819][20381] Updated weights for policy 1, policy_version 1165636 (0.0007) [2023-12-27 13:43:58,889][20381] Updated weights for policy 1, policy_version 1165646 (0.0010) [2023-12-27 13:43:58,950][20381] Updated weights for policy 1, policy_version 1165656 (0.0011) [2023-12-27 13:43:59,410][20347] Updated weights for policy 0, policy_version 1166712 (0.0009) [2023-12-27 13:43:59,464][20347] Updated weights for policy 0, policy_version 1166722 (0.0007) [2023-12-27 13:43:59,519][20347] Updated weights for policy 0, policy_version 1166732 (0.0007) [2023-12-27 13:43:59,673][20381] Updated weights for policy 1, policy_version 1165666 (0.0009) [2023-12-27 13:43:59,724][20381] Updated weights for policy 1, policy_version 1165676 (0.0007) [2023-12-27 13:43:59,776][20381] Updated weights for policy 1, policy_version 1165686 (0.0008) [2023-12-27 13:44:00,221][20347] Updated weights for policy 0, policy_version 1166742 (0.0007) [2023-12-27 13:44:00,274][20347] Updated weights for policy 0, policy_version 1166752 (0.0007) [2023-12-27 13:44:00,336][20347] Updated weights for policy 0, policy_version 1166762 (0.0009) [2023-12-27 13:44:00,462][20381] Updated weights for policy 1, policy_version 1165696 (0.0008) [2023-12-27 13:44:00,525][20381] Updated weights for policy 1, policy_version 1165706 (0.0008) [2023-12-27 13:44:00,580][20381] Updated weights for policy 1, policy_version 1165716 (0.0007) [2023-12-27 13:44:01,003][20347] Updated weights for policy 0, policy_version 1166772 (0.0009) [2023-12-27 13:44:01,062][20347] Updated weights for policy 0, policy_version 1166782 (0.0008) [2023-12-27 13:44:01,122][20347] Updated weights for policy 0, policy_version 1166792 (0.0007) [2023-12-27 13:44:01,280][20381] Updated weights for policy 1, policy_version 1165726 (0.0008) [2023-12-27 13:44:01,337][20381] Updated weights for policy 1, policy_version 1165736 (0.0008) [2023-12-27 13:44:01,402][20381] Updated weights for policy 1, policy_version 1165746 (0.0012) [2023-12-27 13:44:01,848][20347] Updated weights for policy 0, policy_version 1166802 (0.0008) [2023-12-27 13:44:01,908][20347] Updated weights for policy 0, policy_version 1166812 (0.0009) [2023-12-27 13:44:01,964][20347] Updated weights for policy 0, policy_version 1166822 (0.0010) [2023-12-27 13:44:02,016][20347] Updated weights for policy 0, policy_version 1166832 (0.0010) [2023-12-27 13:44:02,093][20381] Updated weights for policy 1, policy_version 1165756 (0.0006) [2023-12-27 13:44:02,161][20381] Updated weights for policy 1, policy_version 1165766 (0.0009) [2023-12-27 13:44:02,227][20381] Updated weights for policy 1, policy_version 1165776 (0.0010) [2023-12-27 13:44:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 597229568. Throughput: 0: 10142.6, 1: 10336.8. Samples: 597236808. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:02,363][19170] Avg episode reward: [(0, '931.630'), (1, '784.690')] [2023-12-27 13:44:02,661][20347] Updated weights for policy 0, policy_version 1166842 (0.0007) [2023-12-27 13:44:02,714][20347] Updated weights for policy 0, policy_version 1166852 (0.0007) [2023-12-27 13:44:02,765][20347] Updated weights for policy 0, policy_version 1166862 (0.0007) [2023-12-27 13:44:02,823][20381] Updated weights for policy 1, policy_version 1165786 (0.0009) [2023-12-27 13:44:02,878][20381] Updated weights for policy 1, policy_version 1165796 (0.0008) [2023-12-27 13:44:02,928][20381] Updated weights for policy 1, policy_version 1165806 (0.0007) [2023-12-27 13:44:02,982][20381] Updated weights for policy 1, policy_version 1165816 (0.0008) [2023-12-27 13:44:03,461][20347] Updated weights for policy 0, policy_version 1166872 (0.0007) [2023-12-27 13:44:03,516][20347] Updated weights for policy 0, policy_version 1166882 (0.0007) [2023-12-27 13:44:03,575][20347] Updated weights for policy 0, policy_version 1166892 (0.0008) [2023-12-27 13:44:03,670][20381] Updated weights for policy 1, policy_version 1165826 (0.0011) [2023-12-27 13:44:03,718][20381] Updated weights for policy 1, policy_version 1165836 (0.0010) [2023-12-27 13:44:03,763][20381] Updated weights for policy 1, policy_version 1165846 (0.0010) [2023-12-27 13:44:04,247][20347] Updated weights for policy 0, policy_version 1166902 (0.0007) [2023-12-27 13:44:04,312][20347] Updated weights for policy 0, policy_version 1166912 (0.0008) [2023-12-27 13:44:04,370][20347] Updated weights for policy 0, policy_version 1166922 (0.0007) [2023-12-27 13:44:04,448][20381] Updated weights for policy 1, policy_version 1165856 (0.0011) [2023-12-27 13:44:04,505][20381] Updated weights for policy 1, policy_version 1165866 (0.0011) [2023-12-27 13:44:04,569][20381] Updated weights for policy 1, policy_version 1165876 (0.0010) [2023-12-27 13:44:05,044][20347] Updated weights for policy 0, policy_version 1166932 (0.0007) [2023-12-27 13:44:05,100][20347] Updated weights for policy 0, policy_version 1166942 (0.0007) [2023-12-27 13:44:05,153][20347] Updated weights for policy 0, policy_version 1166952 (0.0007) [2023-12-27 13:44:05,256][20381] Updated weights for policy 1, policy_version 1165886 (0.0011) [2023-12-27 13:44:05,313][20381] Updated weights for policy 1, policy_version 1165896 (0.0010) [2023-12-27 13:44:05,368][20381] Updated weights for policy 1, policy_version 1165906 (0.0010) [2023-12-27 13:44:05,832][20347] Updated weights for policy 0, policy_version 1166962 (0.0007) [2023-12-27 13:44:05,890][20347] Updated weights for policy 0, policy_version 1166972 (0.0007) [2023-12-27 13:44:05,947][20347] Updated weights for policy 0, policy_version 1166982 (0.0007) [2023-12-27 13:44:06,000][20347] Updated weights for policy 0, policy_version 1166992 (0.0007) [2023-12-27 13:44:06,027][20381] Updated weights for policy 1, policy_version 1165916 (0.0011) [2023-12-27 13:44:06,081][20381] Updated weights for policy 1, policy_version 1165926 (0.0010) [2023-12-27 13:44:06,143][20381] Updated weights for policy 1, policy_version 1165936 (0.0011) [2023-12-27 13:44:06,702][20347] Updated weights for policy 0, policy_version 1167002 (0.0005) [2023-12-27 13:44:06,759][20347] Updated weights for policy 0, policy_version 1167012 (0.0005) [2023-12-27 13:44:06,818][20347] Updated weights for policy 0, policy_version 1167022 (0.0007) [2023-12-27 13:44:06,880][20381] Updated weights for policy 1, policy_version 1165946 (0.0009) [2023-12-27 13:44:06,945][20381] Updated weights for policy 1, policy_version 1165956 (0.0005) [2023-12-27 13:44:07,002][20381] Updated weights for policy 1, policy_version 1165966 (0.0005) [2023-12-27 13:44:07,060][20381] Updated weights for policy 1, policy_version 1165976 (0.0005) [2023-12-27 13:44:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 597327872. Throughput: 0: 10149.6, 1: 10339.8. Samples: 597298880. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:07,363][19170] Avg episode reward: [(0, '912.300'), (1, '765.130')] [2023-12-27 13:44:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001165976_298516480.pth... [2023-12-27 13:44:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001167024_298811392.pth... [2023-12-27 13:44:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001164792_298213376.pth [2023-12-27 13:44:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001165840_298508288.pth [2023-12-27 13:44:07,442][20347] Updated weights for policy 0, policy_version 1167032 (0.0006) [2023-12-27 13:44:07,498][20347] Updated weights for policy 0, policy_version 1167042 (0.0006) [2023-12-27 13:44:07,552][20347] Updated weights for policy 0, policy_version 1167052 (0.0005) [2023-12-27 13:44:07,646][20381] Updated weights for policy 1, policy_version 1165986 (0.0010) [2023-12-27 13:44:07,692][20381] Updated weights for policy 1, policy_version 1165996 (0.0009) [2023-12-27 13:44:07,740][20381] Updated weights for policy 1, policy_version 1166006 (0.0008) [2023-12-27 13:44:08,170][20347] Updated weights for policy 0, policy_version 1167062 (0.0005) [2023-12-27 13:44:08,231][20347] Updated weights for policy 0, policy_version 1167072 (0.0005) [2023-12-27 13:44:08,279][20347] Updated weights for policy 0, policy_version 1167082 (0.0005) [2023-12-27 13:44:08,397][20381] Updated weights for policy 1, policy_version 1166016 (0.0008) [2023-12-27 13:44:08,461][20381] Updated weights for policy 1, policy_version 1166026 (0.0008) [2023-12-27 13:44:08,524][20381] Updated weights for policy 1, policy_version 1166036 (0.0006) [2023-12-27 13:44:08,841][20347] Updated weights for policy 0, policy_version 1167092 (0.0005) [2023-12-27 13:44:08,901][20347] Updated weights for policy 0, policy_version 1167102 (0.0007) [2023-12-27 13:44:08,962][20347] Updated weights for policy 0, policy_version 1167112 (0.0007) [2023-12-27 13:44:08,981][20291] KL-divergence is very high: 175.9166 [2023-12-27 13:44:09,003][20291] KL-divergence is very high: 125.4865 [2023-12-27 13:44:09,187][20381] Updated weights for policy 1, policy_version 1166046 (0.0008) [2023-12-27 13:44:09,245][20381] Updated weights for policy 1, policy_version 1166056 (0.0009) [2023-12-27 13:44:09,309][20381] Updated weights for policy 1, policy_version 1166066 (0.0008) [2023-12-27 13:44:09,576][20347] Updated weights for policy 0, policy_version 1167122 (0.0007) [2023-12-27 13:44:09,629][20347] Updated weights for policy 0, policy_version 1167132 (0.0007) [2023-12-27 13:44:09,686][20347] Updated weights for policy 0, policy_version 1167142 (0.0007) [2023-12-27 13:44:09,740][20347] Updated weights for policy 0, policy_version 1167152 (0.0006) [2023-12-27 13:44:10,068][20381] Updated weights for policy 1, policy_version 1166076 (0.0008) [2023-12-27 13:44:10,131][20381] Updated weights for policy 1, policy_version 1166086 (0.0009) [2023-12-27 13:44:10,191][20381] Updated weights for policy 1, policy_version 1166096 (0.0011) [2023-12-27 13:44:10,435][20347] Updated weights for policy 0, policy_version 1167162 (0.0005) [2023-12-27 13:44:10,492][20347] Updated weights for policy 0, policy_version 1167172 (0.0006) [2023-12-27 13:44:10,545][20347] Updated weights for policy 0, policy_version 1167182 (0.0009) [2023-12-27 13:44:10,822][20381] Updated weights for policy 1, policy_version 1166106 (0.0010) [2023-12-27 13:44:10,879][20381] Updated weights for policy 1, policy_version 1166116 (0.0008) [2023-12-27 13:44:10,941][20381] Updated weights for policy 1, policy_version 1166126 (0.0011) [2023-12-27 13:44:11,004][20381] Updated weights for policy 1, policy_version 1166136 (0.0011) [2023-12-27 13:44:11,246][20347] Updated weights for policy 0, policy_version 1167192 (0.0008) [2023-12-27 13:44:11,311][20347] Updated weights for policy 0, policy_version 1167202 (0.0007) [2023-12-27 13:44:11,370][20347] Updated weights for policy 0, policy_version 1167212 (0.0008) [2023-12-27 13:44:11,773][20381] Updated weights for policy 1, policy_version 1166146 (0.0008) [2023-12-27 13:44:11,827][20381] Updated weights for policy 1, policy_version 1166156 (0.0008) [2023-12-27 13:44:11,881][20381] Updated weights for policy 1, policy_version 1166166 (0.0009) [2023-12-27 13:44:12,094][20347] Updated weights for policy 0, policy_version 1167222 (0.0008) [2023-12-27 13:44:12,149][20347] Updated weights for policy 0, policy_version 1167232 (0.0008) [2023-12-27 13:44:12,211][20347] Updated weights for policy 0, policy_version 1167242 (0.0007) [2023-12-27 13:44:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 597434368. Throughput: 0: 10218.6, 1: 10247.2. Samples: 597423320. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:12,363][19170] Avg episode reward: [(0, '873.400'), (1, '735.670')] [2023-12-27 13:44:12,603][20381] Updated weights for policy 1, policy_version 1166176 (0.0008) [2023-12-27 13:44:12,653][20381] Updated weights for policy 1, policy_version 1166186 (0.0008) [2023-12-27 13:44:12,701][20381] Updated weights for policy 1, policy_version 1166196 (0.0007) [2023-12-27 13:44:12,941][20347] Updated weights for policy 0, policy_version 1167252 (0.0008) [2023-12-27 13:44:12,998][20347] Updated weights for policy 0, policy_version 1167262 (0.0007) [2023-12-27 13:44:13,053][20347] Updated weights for policy 0, policy_version 1167272 (0.0007) [2023-12-27 13:44:13,357][20381] Updated weights for policy 1, policy_version 1166206 (0.0007) [2023-12-27 13:44:13,419][20381] Updated weights for policy 1, policy_version 1166216 (0.0007) [2023-12-27 13:44:13,476][20381] Updated weights for policy 1, policy_version 1166226 (0.0005) [2023-12-27 13:44:13,770][20347] Updated weights for policy 0, policy_version 1167282 (0.0007) [2023-12-27 13:44:13,825][20347] Updated weights for policy 0, policy_version 1167292 (0.0005) [2023-12-27 13:44:13,888][20347] Updated weights for policy 0, policy_version 1167302 (0.0005) [2023-12-27 13:44:13,950][20347] Updated weights for policy 0, policy_version 1167312 (0.0007) [2023-12-27 13:44:14,045][20381] Updated weights for policy 1, policy_version 1166236 (0.0005) [2023-12-27 13:44:14,101][20381] Updated weights for policy 1, policy_version 1166246 (0.0008) [2023-12-27 13:44:14,157][20381] Updated weights for policy 1, policy_version 1166256 (0.0011) [2023-12-27 13:44:14,599][20347] Updated weights for policy 0, policy_version 1167322 (0.0007) [2023-12-27 13:44:14,656][20347] Updated weights for policy 0, policy_version 1167332 (0.0008) [2023-12-27 13:44:14,714][20347] Updated weights for policy 0, policy_version 1167342 (0.0007) [2023-12-27 13:44:14,748][20381] Updated weights for policy 1, policy_version 1166266 (0.0006) [2023-12-27 13:44:14,810][20381] Updated weights for policy 1, policy_version 1166276 (0.0007) [2023-12-27 13:44:14,866][20381] Updated weights for policy 1, policy_version 1166286 (0.0007) [2023-12-27 13:44:14,934][20381] Updated weights for policy 1, policy_version 1166296 (0.0011) [2023-12-27 13:44:15,422][20347] Updated weights for policy 0, policy_version 1167352 (0.0008) [2023-12-27 13:44:15,474][20347] Updated weights for policy 0, policy_version 1167362 (0.0007) [2023-12-27 13:44:15,524][20347] Updated weights for policy 0, policy_version 1167372 (0.0006) [2023-12-27 13:44:15,616][20381] Updated weights for policy 1, policy_version 1166306 (0.0010) [2023-12-27 13:44:15,678][20381] Updated weights for policy 1, policy_version 1166316 (0.0008) [2023-12-27 13:44:15,735][20381] Updated weights for policy 1, policy_version 1166326 (0.0010) [2023-12-27 13:44:16,242][20347] Updated weights for policy 0, policy_version 1167382 (0.0008) [2023-12-27 13:44:16,294][20347] Updated weights for policy 0, policy_version 1167392 (0.0008) [2023-12-27 13:44:16,340][20381] Updated weights for policy 1, policy_version 1166336 (0.0010) [2023-12-27 13:44:16,353][20347] Updated weights for policy 0, policy_version 1167402 (0.0005) [2023-12-27 13:44:16,397][20381] Updated weights for policy 1, policy_version 1166346 (0.0006) [2023-12-27 13:44:16,453][20381] Updated weights for policy 1, policy_version 1166356 (0.0005) [2023-12-27 13:44:17,073][20347] Updated weights for policy 0, policy_version 1167412 (0.0008) [2023-12-27 13:44:17,096][20381] Updated weights for policy 1, policy_version 1166366 (0.0008) [2023-12-27 13:44:17,121][20347] Updated weights for policy 0, policy_version 1167422 (0.0005) [2023-12-27 13:44:17,145][20381] Updated weights for policy 1, policy_version 1166376 (0.0010) [2023-12-27 13:44:17,186][20347] Updated weights for policy 0, policy_version 1167432 (0.0008) [2023-12-27 13:44:17,196][20381] Updated weights for policy 1, policy_version 1166386 (0.0008) [2023-12-27 13:44:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 597540864. Throughput: 0: 10168.1, 1: 10270.7. Samples: 597545812. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:17,363][19170] Avg episode reward: [(0, '854.140'), (1, '804.180')] [2023-12-27 13:44:17,826][20347] Updated weights for policy 0, policy_version 1167442 (0.0006) [2023-12-27 13:44:17,877][20347] Updated weights for policy 0, policy_version 1167452 (0.0005) [2023-12-27 13:44:17,894][20381] Updated weights for policy 1, policy_version 1166396 (0.0010) [2023-12-27 13:44:17,931][20347] Updated weights for policy 0, policy_version 1167462 (0.0008) [2023-12-27 13:44:17,953][20381] Updated weights for policy 1, policy_version 1166406 (0.0011) [2023-12-27 13:44:17,979][20347] Updated weights for policy 0, policy_version 1167472 (0.0010) [2023-12-27 13:44:18,006][20381] Updated weights for policy 1, policy_version 1166416 (0.0011) [2023-12-27 13:44:18,683][20347] Updated weights for policy 0, policy_version 1167482 (0.0010) [2023-12-27 13:44:18,750][20381] Updated weights for policy 1, policy_version 1166426 (0.0011) [2023-12-27 13:44:18,752][20347] Updated weights for policy 0, policy_version 1167492 (0.0010) [2023-12-27 13:44:18,809][20381] Updated weights for policy 1, policy_version 1166436 (0.0011) [2023-12-27 13:44:18,821][20347] Updated weights for policy 0, policy_version 1167502 (0.0011) [2023-12-27 13:44:18,868][20381] Updated weights for policy 1, policy_version 1166446 (0.0010) [2023-12-27 13:44:18,930][20381] Updated weights for policy 1, policy_version 1166456 (0.0007) [2023-12-27 13:44:19,567][20347] Updated weights for policy 0, policy_version 1167512 (0.0008) [2023-12-27 13:44:19,604][20381] Updated weights for policy 1, policy_version 1166466 (0.0009) [2023-12-27 13:44:19,628][20347] Updated weights for policy 0, policy_version 1167522 (0.0011) [2023-12-27 13:44:19,665][20381] Updated weights for policy 1, policy_version 1166476 (0.0008) [2023-12-27 13:44:19,686][20347] Updated weights for policy 0, policy_version 1167532 (0.0011) [2023-12-27 13:44:19,729][20381] Updated weights for policy 1, policy_version 1166486 (0.0008) [2023-12-27 13:44:20,468][20347] Updated weights for policy 0, policy_version 1167542 (0.0009) [2023-12-27 13:44:20,524][20381] Updated weights for policy 1, policy_version 1166496 (0.0010) [2023-12-27 13:44:20,526][20347] Updated weights for policy 0, policy_version 1167552 (0.0010) [2023-12-27 13:44:20,584][20347] Updated weights for policy 0, policy_version 1167562 (0.0009) [2023-12-27 13:44:20,586][20381] Updated weights for policy 1, policy_version 1166506 (0.0010) [2023-12-27 13:44:20,645][20381] Updated weights for policy 1, policy_version 1166516 (0.0011) [2023-12-27 13:44:21,280][20381] Updated weights for policy 1, policy_version 1166526 (0.0011) [2023-12-27 13:44:21,285][20347] Updated weights for policy 0, policy_version 1167572 (0.0010) [2023-12-27 13:44:21,343][20381] Updated weights for policy 1, policy_version 1166536 (0.0010) [2023-12-27 13:44:21,350][20347] Updated weights for policy 0, policy_version 1167582 (0.0008) [2023-12-27 13:44:21,415][20381] Updated weights for policy 1, policy_version 1166546 (0.0009) [2023-12-27 13:44:21,418][20347] Updated weights for policy 0, policy_version 1167592 (0.0010) [2023-12-27 13:44:22,105][20347] Updated weights for policy 0, policy_version 1167602 (0.0010) [2023-12-27 13:44:22,157][20347] Updated weights for policy 0, policy_version 1167612 (0.0007) [2023-12-27 13:44:22,212][20347] Updated weights for policy 0, policy_version 1167622 (0.0007) [2023-12-27 13:44:22,222][20381] Updated weights for policy 1, policy_version 1166556 (0.0009) [2023-12-27 13:44:22,275][20381] Updated weights for policy 1, policy_version 1166566 (0.0008) [2023-12-27 13:44:22,275][20347] Updated weights for policy 0, policy_version 1167632 (0.0007) [2023-12-27 13:44:22,342][20381] Updated weights for policy 1, policy_version 1166576 (0.0009) [2023-12-27 13:44:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 597630976. Throughput: 0: 10127.0, 1: 10218.8. Samples: 597604404. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:22,363][19170] Avg episode reward: [(0, '903.020'), (1, '784.570')] [2023-12-27 13:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001167632_298967040.pth... [2023-12-27 13:44:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001166448_298663936.pth [2023-12-27 13:44:22,400][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001166584_298672128.pth... [2023-12-27 13:44:22,406][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001165368_298360832.pth [2023-12-27 13:44:22,997][20347] Updated weights for policy 0, policy_version 1167642 (0.0008) [2023-12-27 13:44:23,007][20381] Updated weights for policy 1, policy_version 1166586 (0.0009) [2023-12-27 13:44:23,055][20347] Updated weights for policy 0, policy_version 1167652 (0.0009) [2023-12-27 13:44:23,059][20381] Updated weights for policy 1, policy_version 1166596 (0.0009) [2023-12-27 13:44:23,108][20347] Updated weights for policy 0, policy_version 1167662 (0.0008) [2023-12-27 13:44:23,114][20381] Updated weights for policy 1, policy_version 1166606 (0.0006) [2023-12-27 13:44:23,166][20381] Updated weights for policy 1, policy_version 1166616 (0.0005) [2023-12-27 13:44:23,797][20347] Updated weights for policy 0, policy_version 1167672 (0.0007) [2023-12-27 13:44:23,803][20381] Updated weights for policy 1, policy_version 1166626 (0.0005) [2023-12-27 13:44:23,850][20347] Updated weights for policy 0, policy_version 1167682 (0.0007) [2023-12-27 13:44:23,865][20381] Updated weights for policy 1, policy_version 1166636 (0.0005) [2023-12-27 13:44:23,901][20347] Updated weights for policy 0, policy_version 1167692 (0.0007) [2023-12-27 13:44:23,923][20381] Updated weights for policy 1, policy_version 1166646 (0.0006) [2023-12-27 13:44:24,595][20381] Updated weights for policy 1, policy_version 1166656 (0.0007) [2023-12-27 13:44:24,599][20347] Updated weights for policy 0, policy_version 1167702 (0.0008) [2023-12-27 13:44:24,651][20381] Updated weights for policy 1, policy_version 1166666 (0.0005) [2023-12-27 13:44:24,656][20347] Updated weights for policy 0, policy_version 1167712 (0.0008) [2023-12-27 13:44:24,708][20381] Updated weights for policy 1, policy_version 1166676 (0.0006) [2023-12-27 13:44:24,716][20347] Updated weights for policy 0, policy_version 1167722 (0.0007) [2023-12-27 13:44:25,338][20381] Updated weights for policy 1, policy_version 1166686 (0.0008) [2023-12-27 13:44:25,375][20347] Updated weights for policy 0, policy_version 1167732 (0.0007) [2023-12-27 13:44:25,398][20381] Updated weights for policy 1, policy_version 1166696 (0.0009) [2023-12-27 13:44:25,429][20347] Updated weights for policy 0, policy_version 1167742 (0.0006) [2023-12-27 13:44:25,452][20381] Updated weights for policy 1, policy_version 1166706 (0.0007) [2023-12-27 13:44:25,484][20347] Updated weights for policy 0, policy_version 1167752 (0.0005) [2023-12-27 13:44:26,107][20381] Updated weights for policy 1, policy_version 1166716 (0.0007) [2023-12-27 13:44:26,160][20347] Updated weights for policy 0, policy_version 1167762 (0.0009) [2023-12-27 13:44:26,167][20381] Updated weights for policy 1, policy_version 1166726 (0.0005) [2023-12-27 13:44:26,206][20347] Updated weights for policy 0, policy_version 1167772 (0.0010) [2023-12-27 13:44:26,225][20381] Updated weights for policy 1, policy_version 1166736 (0.0005) [2023-12-27 13:44:26,261][20347] Updated weights for policy 0, policy_version 1167782 (0.0010) [2023-12-27 13:44:26,307][20347] Updated weights for policy 0, policy_version 1167792 (0.0010) [2023-12-27 13:44:26,864][20381] Updated weights for policy 1, policy_version 1166746 (0.0005) [2023-12-27 13:44:26,910][20347] Updated weights for policy 0, policy_version 1167802 (0.0005) [2023-12-27 13:44:26,927][20381] Updated weights for policy 1, policy_version 1166756 (0.0006) [2023-12-27 13:44:26,962][20347] Updated weights for policy 0, policy_version 1167812 (0.0006) [2023-12-27 13:44:26,978][20381] Updated weights for policy 1, policy_version 1166766 (0.0006) [2023-12-27 13:44:27,020][20347] Updated weights for policy 0, policy_version 1167822 (0.0005) [2023-12-27 13:44:27,029][20381] Updated weights for policy 1, policy_version 1166776 (0.0006) [2023-12-27 13:44:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 597737472. Throughput: 0: 10102.9, 1: 10244.4. Samples: 597725828. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:27,363][19170] Avg episode reward: [(0, '863.500'), (1, '784.530')] [2023-12-27 13:44:27,677][20381] Updated weights for policy 1, policy_version 1166786 (0.0006) [2023-12-27 13:44:27,682][20347] Updated weights for policy 0, policy_version 1167832 (0.0006) [2023-12-27 13:44:27,733][20381] Updated weights for policy 1, policy_version 1166796 (0.0006) [2023-12-27 13:44:27,736][20347] Updated weights for policy 0, policy_version 1167842 (0.0005) [2023-12-27 13:44:27,781][20381] Updated weights for policy 1, policy_version 1166806 (0.0007) [2023-12-27 13:44:27,786][20347] Updated weights for policy 0, policy_version 1167852 (0.0005) [2023-12-27 13:44:28,436][20347] Updated weights for policy 0, policy_version 1167862 (0.0008) [2023-12-27 13:44:28,486][20347] Updated weights for policy 0, policy_version 1167872 (0.0005) [2023-12-27 13:44:28,488][20381] Updated weights for policy 1, policy_version 1166816 (0.0007) [2023-12-27 13:44:28,531][20347] Updated weights for policy 0, policy_version 1167882 (0.0006) [2023-12-27 13:44:28,551][20381] Updated weights for policy 1, policy_version 1166826 (0.0006) [2023-12-27 13:44:28,612][20381] Updated weights for policy 1, policy_version 1166836 (0.0008) [2023-12-27 13:44:29,135][20347] Updated weights for policy 0, policy_version 1167892 (0.0006) [2023-12-27 13:44:29,187][20347] Updated weights for policy 0, policy_version 1167902 (0.0007) [2023-12-27 13:44:29,245][20347] Updated weights for policy 0, policy_version 1167912 (0.0007) [2023-12-27 13:44:29,316][20381] Updated weights for policy 1, policy_version 1166846 (0.0007) [2023-12-27 13:44:29,330][20292] KL-divergence is very high: 110.0660 [2023-12-27 13:44:29,385][20381] Updated weights for policy 1, policy_version 1166856 (0.0008) [2023-12-27 13:44:29,387][20292] KL-divergence is very high: 104.1684 [2023-12-27 13:44:29,445][20381] Updated weights for policy 1, policy_version 1166866 (0.0008) [2023-12-27 13:44:29,962][20347] Updated weights for policy 0, policy_version 1167922 (0.0007) [2023-12-27 13:44:30,023][20347] Updated weights for policy 0, policy_version 1167932 (0.0008) [2023-12-27 13:44:30,084][20347] Updated weights for policy 0, policy_version 1167942 (0.0009) [2023-12-27 13:44:30,141][20347] Updated weights for policy 0, policy_version 1167952 (0.0008) [2023-12-27 13:44:30,146][20381] Updated weights for policy 1, policy_version 1166876 (0.0008) [2023-12-27 13:44:30,209][20381] Updated weights for policy 1, policy_version 1166886 (0.0008) [2023-12-27 13:44:30,265][20381] Updated weights for policy 1, policy_version 1166896 (0.0007) [2023-12-27 13:44:30,792][20347] Updated weights for policy 0, policy_version 1167962 (0.0007) [2023-12-27 13:44:30,841][20347] Updated weights for policy 0, policy_version 1167972 (0.0005) [2023-12-27 13:44:30,892][20347] Updated weights for policy 0, policy_version 1167982 (0.0005) [2023-12-27 13:44:30,970][20381] Updated weights for policy 1, policy_version 1166906 (0.0008) [2023-12-27 13:44:31,002][20292] KL-divergence is very high: 200.8693 [2023-12-27 13:44:31,025][20381] Updated weights for policy 1, policy_version 1166916 (0.0007) [2023-12-27 13:44:31,026][20292] KL-divergence is very high: 115.2300 [2023-12-27 13:44:31,087][20381] Updated weights for policy 1, policy_version 1166926 (0.0007) [2023-12-27 13:44:31,150][20381] Updated weights for policy 1, policy_version 1166936 (0.0007) [2023-12-27 13:44:31,592][20347] Updated weights for policy 0, policy_version 1167992 (0.0007) [2023-12-27 13:44:31,654][20347] Updated weights for policy 0, policy_version 1168002 (0.0007) [2023-12-27 13:44:31,709][20347] Updated weights for policy 0, policy_version 1168012 (0.0008) [2023-12-27 13:44:31,868][20381] Updated weights for policy 1, policy_version 1166946 (0.0005) [2023-12-27 13:44:31,919][20381] Updated weights for policy 1, policy_version 1166956 (0.0005) [2023-12-27 13:44:31,975][20381] Updated weights for policy 1, policy_version 1166966 (0.0007) [2023-12-27 13:44:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 597835776. Throughput: 0: 10126.7, 1: 10259.3. Samples: 597849208. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:32,363][19170] Avg episode reward: [(0, '824.560'), (1, '725.820')] [2023-12-27 13:44:32,407][20347] Updated weights for policy 0, policy_version 1168022 (0.0008) [2023-12-27 13:44:32,463][20347] Updated weights for policy 0, policy_version 1168032 (0.0011) [2023-12-27 13:44:32,516][20347] Updated weights for policy 0, policy_version 1168042 (0.0007) [2023-12-27 13:44:32,648][20381] Updated weights for policy 1, policy_version 1166976 (0.0006) [2023-12-27 13:44:32,649][20292] KL-divergence is very high: 296.8360 [2023-12-27 13:44:32,654][20292] KL-divergence is very high: 197.6337 [2023-12-27 13:44:32,664][20292] KL-divergence is very high: 236.9017 [2023-12-27 13:44:32,697][20381] Updated weights for policy 1, policy_version 1166986 (0.0007) [2023-12-27 13:44:32,741][20381] Updated weights for policy 1, policy_version 1166996 (0.0005) [2023-12-27 13:44:33,204][20347] Updated weights for policy 0, policy_version 1168052 (0.0008) [2023-12-27 13:44:33,265][20347] Updated weights for policy 0, policy_version 1168062 (0.0010) [2023-12-27 13:44:33,317][20347] Updated weights for policy 0, policy_version 1168072 (0.0010) [2023-12-27 13:44:33,365][20381] Updated weights for policy 1, policy_version 1167006 (0.0007) [2023-12-27 13:44:33,414][20381] Updated weights for policy 1, policy_version 1167016 (0.0005) [2023-12-27 13:44:33,463][20381] Updated weights for policy 1, policy_version 1167026 (0.0005) [2023-12-27 13:44:33,938][20347] Updated weights for policy 0, policy_version 1168082 (0.0010) [2023-12-27 13:44:33,990][20381] Updated weights for policy 1, policy_version 1167036 (0.0005) [2023-12-27 13:44:34,002][20347] Updated weights for policy 0, policy_version 1168092 (0.0007) [2023-12-27 13:44:34,046][20381] Updated weights for policy 1, policy_version 1167046 (0.0005) [2023-12-27 13:44:34,060][20347] Updated weights for policy 0, policy_version 1168102 (0.0005) [2023-12-27 13:44:34,100][20381] Updated weights for policy 1, policy_version 1167056 (0.0005) [2023-12-27 13:44:34,119][20347] Updated weights for policy 0, policy_version 1168112 (0.0008) [2023-12-27 13:44:34,713][20347] Updated weights for policy 0, policy_version 1168122 (0.0010) [2023-12-27 13:44:34,747][20381] Updated weights for policy 1, policy_version 1167066 (0.0009) [2023-12-27 13:44:34,768][20347] Updated weights for policy 0, policy_version 1168132 (0.0010) [2023-12-27 13:44:34,792][20381] Updated weights for policy 1, policy_version 1167076 (0.0010) [2023-12-27 13:44:34,824][20347] Updated weights for policy 0, policy_version 1168142 (0.0009) [2023-12-27 13:44:34,841][20381] Updated weights for policy 1, policy_version 1167086 (0.0010) [2023-12-27 13:44:34,892][20381] Updated weights for policy 1, policy_version 1167096 (0.0010) [2023-12-27 13:44:35,436][20347] Updated weights for policy 0, policy_version 1168152 (0.0009) [2023-12-27 13:44:35,491][20347] Updated weights for policy 0, policy_version 1168162 (0.0010) [2023-12-27 13:44:35,549][20347] Updated weights for policy 0, policy_version 1168172 (0.0010) [2023-12-27 13:44:35,598][20381] Updated weights for policy 1, policy_version 1167106 (0.0010) [2023-12-27 13:44:35,650][20381] Updated weights for policy 1, policy_version 1167116 (0.0010) [2023-12-27 13:44:35,698][20381] Updated weights for policy 1, policy_version 1167126 (0.0010) [2023-12-27 13:44:36,224][20347] Updated weights for policy 0, policy_version 1168182 (0.0007) [2023-12-27 13:44:36,295][20347] Updated weights for policy 0, policy_version 1168192 (0.0007) [2023-12-27 13:44:36,363][20347] Updated weights for policy 0, policy_version 1168202 (0.0008) [2023-12-27 13:44:36,400][20381] Updated weights for policy 1, policy_version 1167136 (0.0009) [2023-12-27 13:44:36,461][20381] Updated weights for policy 1, policy_version 1167146 (0.0008) [2023-12-27 13:44:36,523][20381] Updated weights for policy 1, policy_version 1167156 (0.0007) [2023-12-27 13:44:37,010][20347] Updated weights for policy 0, policy_version 1168212 (0.0007) [2023-12-27 13:44:37,056][20347] Updated weights for policy 0, policy_version 1168222 (0.0010) [2023-12-27 13:44:37,103][20347] Updated weights for policy 0, policy_version 1168232 (0.0008) [2023-12-27 13:44:37,136][20381] Updated weights for policy 1, policy_version 1167166 (0.0007) [2023-12-27 13:44:37,194][20381] Updated weights for policy 1, policy_version 1167176 (0.0006) [2023-12-27 13:44:37,208][20292] KL-divergence is very high: 121.5817 [2023-12-27 13:44:37,220][20292] KL-divergence is very high: 136.7745 [2023-12-27 13:44:37,252][20381] Updated weights for policy 1, policy_version 1167186 (0.0008) [2023-12-27 13:44:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 597950464. Throughput: 0: 10160.8, 1: 10320.7. Samples: 597914728. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:37,363][19170] Avg episode reward: [(0, '775.890'), (1, '774.550')] [2023-12-27 13:44:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001167192_298827776.pth... [2023-12-27 13:44:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001168240_299122688.pth... [2023-12-27 13:44:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001165976_298516480.pth [2023-12-27 13:44:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001167024_298811392.pth [2023-12-27 13:44:37,847][20347] Updated weights for policy 0, policy_version 1168242 (0.0007) [2023-12-27 13:44:37,868][20381] Updated weights for policy 1, policy_version 1167196 (0.0006) [2023-12-27 13:44:37,902][20347] Updated weights for policy 0, policy_version 1168252 (0.0006) [2023-12-27 13:44:37,926][20381] Updated weights for policy 1, policy_version 1167206 (0.0007) [2023-12-27 13:44:37,954][20347] Updated weights for policy 0, policy_version 1168262 (0.0006) [2023-12-27 13:44:37,979][20381] Updated weights for policy 1, policy_version 1167216 (0.0005) [2023-12-27 13:44:38,005][20347] Updated weights for policy 0, policy_version 1168272 (0.0006) [2023-12-27 13:44:38,628][20381] Updated weights for policy 1, policy_version 1167226 (0.0008) [2023-12-27 13:44:38,688][20381] Updated weights for policy 1, policy_version 1167236 (0.0007) [2023-12-27 13:44:38,693][20347] Updated weights for policy 0, policy_version 1168282 (0.0008) [2023-12-27 13:44:38,745][20381] Updated weights for policy 1, policy_version 1167246 (0.0006) [2023-12-27 13:44:38,748][20347] Updated weights for policy 0, policy_version 1168292 (0.0008) [2023-12-27 13:44:38,803][20381] Updated weights for policy 1, policy_version 1167256 (0.0007) [2023-12-27 13:44:38,809][20347] Updated weights for policy 0, policy_version 1168302 (0.0008) [2023-12-27 13:44:39,451][20381] Updated weights for policy 1, policy_version 1167266 (0.0008) [2023-12-27 13:44:39,503][20381] Updated weights for policy 1, policy_version 1167276 (0.0008) [2023-12-27 13:44:39,558][20347] Updated weights for policy 0, policy_version 1168312 (0.0006) [2023-12-27 13:44:39,565][20381] Updated weights for policy 1, policy_version 1167286 (0.0008) [2023-12-27 13:44:39,620][20347] Updated weights for policy 0, policy_version 1168322 (0.0006) [2023-12-27 13:44:39,683][20347] Updated weights for policy 0, policy_version 1168332 (0.0006) [2023-12-27 13:44:40,314][20381] Updated weights for policy 1, policy_version 1167296 (0.0009) [2023-12-27 13:44:40,361][20347] Updated weights for policy 0, policy_version 1168342 (0.0006) [2023-12-27 13:44:40,377][20381] Updated weights for policy 1, policy_version 1167306 (0.0007) [2023-12-27 13:44:40,423][20347] Updated weights for policy 0, policy_version 1168352 (0.0007) [2023-12-27 13:44:40,427][20381] Updated weights for policy 1, policy_version 1167316 (0.0005) [2023-12-27 13:44:40,480][20347] Updated weights for policy 0, policy_version 1168362 (0.0008) [2023-12-27 13:44:41,063][20381] Updated weights for policy 1, policy_version 1167326 (0.0007) [2023-12-27 13:44:41,122][20381] Updated weights for policy 1, policy_version 1167336 (0.0009) [2023-12-27 13:44:41,179][20381] Updated weights for policy 1, policy_version 1167346 (0.0009) [2023-12-27 13:44:41,211][20347] Updated weights for policy 0, policy_version 1168372 (0.0008) [2023-12-27 13:44:41,271][20347] Updated weights for policy 0, policy_version 1168382 (0.0009) [2023-12-27 13:44:41,321][20347] Updated weights for policy 0, policy_version 1168392 (0.0009) [2023-12-27 13:44:42,065][20381] Updated weights for policy 1, policy_version 1167356 (0.0009) [2023-12-27 13:44:42,125][20381] Updated weights for policy 1, policy_version 1167366 (0.0011) [2023-12-27 13:44:42,128][20347] Updated weights for policy 0, policy_version 1168402 (0.0008) [2023-12-27 13:44:42,181][20381] Updated weights for policy 1, policy_version 1167376 (0.0007) [2023-12-27 13:44:42,194][20347] Updated weights for policy 0, policy_version 1168412 (0.0008) [2023-12-27 13:44:42,262][20347] Updated weights for policy 0, policy_version 1168422 (0.0008) [2023-12-27 13:44:42,325][20347] Updated weights for policy 0, policy_version 1168432 (0.0007) [2023-12-27 13:44:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 598048768. Throughput: 0: 10154.2, 1: 10294.4. Samples: 598036472. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:42,363][19170] Avg episode reward: [(0, '766.390'), (1, '872.510')] [2023-12-27 13:44:42,879][20381] Updated weights for policy 1, policy_version 1167386 (0.0008) [2023-12-27 13:44:42,931][20381] Updated weights for policy 1, policy_version 1167396 (0.0006) [2023-12-27 13:44:42,992][20381] Updated weights for policy 1, policy_version 1167406 (0.0005) [2023-12-27 13:44:42,996][20347] Updated weights for policy 0, policy_version 1168442 (0.0007) [2023-12-27 13:44:43,047][20381] Updated weights for policy 1, policy_version 1167416 (0.0005) [2023-12-27 13:44:43,053][20347] Updated weights for policy 0, policy_version 1168452 (0.0008) [2023-12-27 13:44:43,116][20347] Updated weights for policy 0, policy_version 1168462 (0.0008) [2023-12-27 13:44:43,572][20381] Updated weights for policy 1, policy_version 1167426 (0.0008) [2023-12-27 13:44:43,630][20381] Updated weights for policy 1, policy_version 1167436 (0.0008) [2023-12-27 13:44:43,686][20381] Updated weights for policy 1, policy_version 1167446 (0.0007) [2023-12-27 13:44:43,851][20347] Updated weights for policy 0, policy_version 1168472 (0.0008) [2023-12-27 13:44:43,896][20347] Updated weights for policy 0, policy_version 1168482 (0.0007) [2023-12-27 13:44:43,943][20347] Updated weights for policy 0, policy_version 1168492 (0.0007) [2023-12-27 13:44:44,348][20381] Updated weights for policy 1, policy_version 1167456 (0.0007) [2023-12-27 13:44:44,402][20381] Updated weights for policy 1, policy_version 1167466 (0.0007) [2023-12-27 13:44:44,455][20381] Updated weights for policy 1, policy_version 1167476 (0.0007) [2023-12-27 13:44:44,627][20347] Updated weights for policy 0, policy_version 1168502 (0.0007) [2023-12-27 13:44:44,683][20347] Updated weights for policy 0, policy_version 1168512 (0.0008) [2023-12-27 13:44:44,729][20347] Updated weights for policy 0, policy_version 1168522 (0.0010) [2023-12-27 13:44:45,162][20381] Updated weights for policy 1, policy_version 1167486 (0.0008) [2023-12-27 13:44:45,214][20381] Updated weights for policy 1, policy_version 1167496 (0.0008) [2023-12-27 13:44:45,276][20381] Updated weights for policy 1, policy_version 1167506 (0.0008) [2023-12-27 13:44:45,430][20347] Updated weights for policy 0, policy_version 1168532 (0.0010) [2023-12-27 13:44:45,488][20347] Updated weights for policy 0, policy_version 1168542 (0.0007) [2023-12-27 13:44:45,551][20347] Updated weights for policy 0, policy_version 1168552 (0.0007) [2023-12-27 13:44:45,971][20381] Updated weights for policy 1, policy_version 1167516 (0.0007) [2023-12-27 13:44:46,028][20381] Updated weights for policy 1, policy_version 1167526 (0.0005) [2023-12-27 13:44:46,082][20381] Updated weights for policy 1, policy_version 1167536 (0.0005) [2023-12-27 13:44:46,195][20347] Updated weights for policy 0, policy_version 1168562 (0.0007) [2023-12-27 13:44:46,245][20347] Updated weights for policy 0, policy_version 1168572 (0.0005) [2023-12-27 13:44:46,299][20347] Updated weights for policy 0, policy_version 1168582 (0.0005) [2023-12-27 13:44:46,360][20347] Updated weights for policy 0, policy_version 1168592 (0.0005) [2023-12-27 13:44:46,657][20381] Updated weights for policy 1, policy_version 1167546 (0.0006) [2023-12-27 13:44:46,717][20381] Updated weights for policy 1, policy_version 1167556 (0.0007) [2023-12-27 13:44:46,776][20381] Updated weights for policy 1, policy_version 1167566 (0.0009) [2023-12-27 13:44:46,836][20381] Updated weights for policy 1, policy_version 1167576 (0.0007) [2023-12-27 13:44:46,909][20347] Updated weights for policy 0, policy_version 1168602 (0.0010) [2023-12-27 13:44:46,918][20291] KL-divergence is very high: 140.2485 [2023-12-27 13:44:46,922][20291] KL-divergence is very high: 204.9534 [2023-12-27 13:44:46,966][20347] Updated weights for policy 0, policy_version 1168612 (0.0010) [2023-12-27 13:44:47,012][20347] Updated weights for policy 0, policy_version 1168622 (0.0010) [2023-12-27 13:44:47,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 598147072. Throughput: 0: 10183.0, 1: 10303.3. Samples: 598158696. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:47,363][19170] Avg episode reward: [(0, '795.580'), (1, '843.330')] [2023-12-27 13:44:47,483][20381] Updated weights for policy 1, policy_version 1167586 (0.0007) [2023-12-27 13:44:47,531][20381] Updated weights for policy 1, policy_version 1167596 (0.0007) [2023-12-27 13:44:47,588][20381] Updated weights for policy 1, policy_version 1167606 (0.0007) [2023-12-27 13:44:47,660][20347] Updated weights for policy 0, policy_version 1168632 (0.0010) [2023-12-27 13:44:47,715][20347] Updated weights for policy 0, policy_version 1168642 (0.0010) [2023-12-27 13:44:47,764][20347] Updated weights for policy 0, policy_version 1168652 (0.0010) [2023-12-27 13:44:48,290][20381] Updated weights for policy 1, policy_version 1167616 (0.0008) [2023-12-27 13:44:48,343][20381] Updated weights for policy 1, policy_version 1167626 (0.0007) [2023-12-27 13:44:48,395][20381] Updated weights for policy 1, policy_version 1167636 (0.0006) [2023-12-27 13:44:48,481][20347] Updated weights for policy 0, policy_version 1168662 (0.0010) [2023-12-27 13:44:48,542][20347] Updated weights for policy 0, policy_version 1168672 (0.0010) [2023-12-27 13:44:48,604][20347] Updated weights for policy 0, policy_version 1168682 (0.0011) [2023-12-27 13:44:49,024][20381] Updated weights for policy 1, policy_version 1167646 (0.0005) [2023-12-27 13:44:49,076][20381] Updated weights for policy 1, policy_version 1167656 (0.0006) [2023-12-27 13:44:49,130][20381] Updated weights for policy 1, policy_version 1167666 (0.0005) [2023-12-27 13:44:49,254][20347] Updated weights for policy 0, policy_version 1168692 (0.0009) [2023-12-27 13:44:49,310][20347] Updated weights for policy 0, policy_version 1168702 (0.0007) [2023-12-27 13:44:49,382][20347] Updated weights for policy 0, policy_version 1168712 (0.0007) [2023-12-27 13:44:49,813][20381] Updated weights for policy 1, policy_version 1167676 (0.0006) [2023-12-27 13:44:49,883][20381] Updated weights for policy 1, policy_version 1167686 (0.0008) [2023-12-27 13:44:49,951][20381] Updated weights for policy 1, policy_version 1167696 (0.0009) [2023-12-27 13:44:50,056][20347] Updated weights for policy 0, policy_version 1168722 (0.0007) [2023-12-27 13:44:50,117][20347] Updated weights for policy 0, policy_version 1168732 (0.0008) [2023-12-27 13:44:50,169][20347] Updated weights for policy 0, policy_version 1168742 (0.0007) [2023-12-27 13:44:50,226][20347] Updated weights for policy 0, policy_version 1168752 (0.0008) [2023-12-27 13:44:50,614][20381] Updated weights for policy 1, policy_version 1167706 (0.0008) [2023-12-27 13:44:50,676][20381] Updated weights for policy 1, policy_version 1167716 (0.0008) [2023-12-27 13:44:50,733][20381] Updated weights for policy 1, policy_version 1167726 (0.0008) [2023-12-27 13:44:50,799][20381] Updated weights for policy 1, policy_version 1167736 (0.0007) [2023-12-27 13:44:50,914][20347] Updated weights for policy 0, policy_version 1168762 (0.0008) [2023-12-27 13:44:50,973][20347] Updated weights for policy 0, policy_version 1168772 (0.0009) [2023-12-27 13:44:51,031][20347] Updated weights for policy 0, policy_version 1168782 (0.0010) [2023-12-27 13:44:51,556][20381] Updated weights for policy 1, policy_version 1167746 (0.0007) [2023-12-27 13:44:51,612][20381] Updated weights for policy 1, policy_version 1167756 (0.0007) [2023-12-27 13:44:51,677][20381] Updated weights for policy 1, policy_version 1167766 (0.0008) [2023-12-27 13:44:51,826][20347] Updated weights for policy 0, policy_version 1168792 (0.0007) [2023-12-27 13:44:51,887][20347] Updated weights for policy 0, policy_version 1168802 (0.0006) [2023-12-27 13:44:51,941][20347] Updated weights for policy 0, policy_version 1168812 (0.0008) [2023-12-27 13:44:52,316][20381] Updated weights for policy 1, policy_version 1167776 (0.0008) [2023-12-27 13:44:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 598245376. Throughput: 0: 10194.1, 1: 10299.1. Samples: 598221076. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:52,363][19170] Avg episode reward: [(0, '853.550'), (1, '814.040')] [2023-12-27 13:44:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001168816_299270144.pth... [2023-12-27 13:44:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001167632_298967040.pth [2023-12-27 13:44:52,381][20381] Updated weights for policy 1, policy_version 1167786 (0.0007) [2023-12-27 13:44:52,447][20381] Updated weights for policy 1, policy_version 1167796 (0.0008) [2023-12-27 13:44:52,471][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001167800_298983424.pth... [2023-12-27 13:44:52,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001166584_298672128.pth [2023-12-27 13:44:52,642][20347] Updated weights for policy 0, policy_version 1168822 (0.0008) [2023-12-27 13:44:52,702][20347] Updated weights for policy 0, policy_version 1168832 (0.0008) [2023-12-27 13:44:52,759][20347] Updated weights for policy 0, policy_version 1168842 (0.0006) [2023-12-27 13:44:53,122][20381] Updated weights for policy 1, policy_version 1167806 (0.0009) [2023-12-27 13:44:53,175][20381] Updated weights for policy 1, policy_version 1167816 (0.0009) [2023-12-27 13:44:53,223][20381] Updated weights for policy 1, policy_version 1167826 (0.0005) [2023-12-27 13:44:53,395][20347] Updated weights for policy 0, policy_version 1168852 (0.0006) [2023-12-27 13:44:53,453][20347] Updated weights for policy 0, policy_version 1168862 (0.0009) [2023-12-27 13:44:53,515][20347] Updated weights for policy 0, policy_version 1168872 (0.0006) [2023-12-27 13:44:53,797][20381] Updated weights for policy 1, policy_version 1167836 (0.0005) [2023-12-27 13:44:53,859][20381] Updated weights for policy 1, policy_version 1167846 (0.0006) [2023-12-27 13:44:53,912][20381] Updated weights for policy 1, policy_version 1167856 (0.0010) [2023-12-27 13:44:54,099][20347] Updated weights for policy 0, policy_version 1168882 (0.0006) [2023-12-27 13:44:54,150][20347] Updated weights for policy 0, policy_version 1168892 (0.0009) [2023-12-27 13:44:54,203][20347] Updated weights for policy 0, policy_version 1168902 (0.0010) [2023-12-27 13:44:54,261][20347] Updated weights for policy 0, policy_version 1168912 (0.0010) [2023-12-27 13:44:54,572][20381] Updated weights for policy 1, policy_version 1167866 (0.0010) [2023-12-27 13:44:54,628][20381] Updated weights for policy 1, policy_version 1167876 (0.0010) [2023-12-27 13:44:54,687][20381] Updated weights for policy 1, policy_version 1167886 (0.0010) [2023-12-27 13:44:54,751][20381] Updated weights for policy 1, policy_version 1167896 (0.0011) [2023-12-27 13:44:54,852][20347] Updated weights for policy 0, policy_version 1168922 (0.0005) [2023-12-27 13:44:54,909][20347] Updated weights for policy 0, policy_version 1168932 (0.0007) [2023-12-27 13:44:54,972][20347] Updated weights for policy 0, policy_version 1168942 (0.0011) [2023-12-27 13:44:55,425][20381] Updated weights for policy 1, policy_version 1167906 (0.0011) [2023-12-27 13:44:55,470][20381] Updated weights for policy 1, policy_version 1167916 (0.0010) [2023-12-27 13:44:55,514][20381] Updated weights for policy 1, policy_version 1167926 (0.0010) [2023-12-27 13:44:55,604][20347] Updated weights for policy 0, policy_version 1168952 (0.0010) [2023-12-27 13:44:55,662][20347] Updated weights for policy 0, policy_version 1168962 (0.0010) [2023-12-27 13:44:55,727][20347] Updated weights for policy 0, policy_version 1168972 (0.0010) [2023-12-27 13:44:56,069][20381] Updated weights for policy 1, policy_version 1167936 (0.0009) [2023-12-27 13:44:56,124][20381] Updated weights for policy 1, policy_version 1167946 (0.0010) [2023-12-27 13:44:56,182][20381] Updated weights for policy 1, policy_version 1167956 (0.0009) [2023-12-27 13:44:56,397][20347] Updated weights for policy 0, policy_version 1168982 (0.0010) [2023-12-27 13:44:56,455][20347] Updated weights for policy 0, policy_version 1168992 (0.0010) [2023-12-27 13:44:56,510][20347] Updated weights for policy 0, policy_version 1169002 (0.0010) [2023-12-27 13:44:56,797][20381] Updated weights for policy 1, policy_version 1167966 (0.0008) [2023-12-27 13:44:56,847][20381] Updated weights for policy 1, policy_version 1167976 (0.0010) [2023-12-27 13:44:56,901][20381] Updated weights for policy 1, policy_version 1167986 (0.0010) [2023-12-27 13:44:57,175][20347] Updated weights for policy 0, policy_version 1169012 (0.0010) [2023-12-27 13:44:57,225][20347] Updated weights for policy 0, policy_version 1169022 (0.0010) [2023-12-27 13:44:57,275][20347] Updated weights for policy 0, policy_version 1169032 (0.0010) [2023-12-27 13:44:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 598360064. Throughput: 0: 10165.5, 1: 10345.1. Samples: 598346296. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:44:57,363][19170] Avg episode reward: [(0, '843.810'), (1, '843.430')] [2023-12-27 13:44:57,557][20381] Updated weights for policy 1, policy_version 1167996 (0.0010) [2023-12-27 13:44:57,614][20381] Updated weights for policy 1, policy_version 1168006 (0.0010) [2023-12-27 13:44:57,666][20381] Updated weights for policy 1, policy_version 1168016 (0.0011) [2023-12-27 13:44:57,948][20347] Updated weights for policy 0, policy_version 1169042 (0.0010) [2023-12-27 13:44:58,009][20347] Updated weights for policy 0, policy_version 1169052 (0.0010) [2023-12-27 13:44:58,066][20347] Updated weights for policy 0, policy_version 1169062 (0.0010) [2023-12-27 13:44:58,127][20347] Updated weights for policy 0, policy_version 1169072 (0.0010) [2023-12-27 13:44:58,435][20381] Updated weights for policy 1, policy_version 1168026 (0.0011) [2023-12-27 13:44:58,496][20381] Updated weights for policy 1, policy_version 1168036 (0.0006) [2023-12-27 13:44:58,561][20381] Updated weights for policy 1, policy_version 1168046 (0.0006) [2023-12-27 13:44:58,624][20381] Updated weights for policy 1, policy_version 1168056 (0.0007) [2023-12-27 13:44:58,867][20347] Updated weights for policy 0, policy_version 1169082 (0.0008) [2023-12-27 13:44:58,938][20347] Updated weights for policy 0, policy_version 1169092 (0.0008) [2023-12-27 13:44:58,995][20347] Updated weights for policy 0, policy_version 1169102 (0.0008) [2023-12-27 13:44:59,349][20381] Updated weights for policy 1, policy_version 1168066 (0.0008) [2023-12-27 13:44:59,409][20381] Updated weights for policy 1, policy_version 1168076 (0.0008) [2023-12-27 13:44:59,458][20381] Updated weights for policy 1, policy_version 1168086 (0.0008) [2023-12-27 13:44:59,718][20347] Updated weights for policy 0, policy_version 1169112 (0.0008) [2023-12-27 13:44:59,774][20347] Updated weights for policy 0, policy_version 1169122 (0.0006) [2023-12-27 13:44:59,829][20347] Updated weights for policy 0, policy_version 1169132 (0.0006) [2023-12-27 13:45:00,123][20381] Updated weights for policy 1, policy_version 1168096 (0.0006) [2023-12-27 13:45:00,171][20381] Updated weights for policy 1, policy_version 1168106 (0.0006) [2023-12-27 13:45:00,224][20381] Updated weights for policy 1, policy_version 1168116 (0.0007) [2023-12-27 13:45:00,492][20347] Updated weights for policy 0, policy_version 1169142 (0.0008) [2023-12-27 13:45:00,536][20347] Updated weights for policy 0, policy_version 1169152 (0.0008) [2023-12-27 13:45:00,596][20347] Updated weights for policy 0, policy_version 1169162 (0.0008) [2023-12-27 13:45:00,762][20381] Updated weights for policy 1, policy_version 1168126 (0.0007) [2023-12-27 13:45:00,807][20381] Updated weights for policy 1, policy_version 1168136 (0.0007) [2023-12-27 13:45:00,858][20381] Updated weights for policy 1, policy_version 1168146 (0.0008) [2023-12-27 13:45:01,338][20347] Updated weights for policy 0, policy_version 1169172 (0.0008) [2023-12-27 13:45:01,408][20347] Updated weights for policy 0, policy_version 1169182 (0.0008) [2023-12-27 13:45:01,461][20347] Updated weights for policy 0, policy_version 1169192 (0.0007) [2023-12-27 13:45:01,623][20381] Updated weights for policy 1, policy_version 1168156 (0.0008) [2023-12-27 13:45:01,675][20381] Updated weights for policy 1, policy_version 1168166 (0.0009) [2023-12-27 13:45:01,738][20381] Updated weights for policy 1, policy_version 1168176 (0.0008) [2023-12-27 13:45:02,181][20347] Updated weights for policy 0, policy_version 1169202 (0.0008) [2023-12-27 13:45:02,229][20347] Updated weights for policy 0, policy_version 1169212 (0.0005) [2023-12-27 13:45:02,286][20347] Updated weights for policy 0, policy_version 1169222 (0.0007) [2023-12-27 13:45:02,346][20347] Updated weights for policy 0, policy_version 1169232 (0.0010) [2023-12-27 13:45:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 598458368. Throughput: 0: 10173.6, 1: 10311.0. Samples: 598467620. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:02,363][19170] Avg episode reward: [(0, '902.600'), (1, '804.200')] [2023-12-27 13:45:02,402][20381] Updated weights for policy 1, policy_version 1168186 (0.0009) [2023-12-27 13:45:02,463][20381] Updated weights for policy 1, policy_version 1168196 (0.0008) [2023-12-27 13:45:02,523][20381] Updated weights for policy 1, policy_version 1168206 (0.0008) [2023-12-27 13:45:02,583][20381] Updated weights for policy 1, policy_version 1168216 (0.0005) [2023-12-27 13:45:03,035][20347] Updated weights for policy 0, policy_version 1169242 (0.0007) [2023-12-27 13:45:03,082][20347] Updated weights for policy 0, policy_version 1169252 (0.0007) [2023-12-27 13:45:03,135][20347] Updated weights for policy 0, policy_version 1169262 (0.0007) [2023-12-27 13:45:03,181][20381] Updated weights for policy 1, policy_version 1168226 (0.0007) [2023-12-27 13:45:03,235][20381] Updated weights for policy 1, policy_version 1168236 (0.0007) [2023-12-27 13:45:03,290][20381] Updated weights for policy 1, policy_version 1168246 (0.0007) [2023-12-27 13:45:03,829][20347] Updated weights for policy 0, policy_version 1169272 (0.0007) [2023-12-27 13:45:03,879][20381] Updated weights for policy 1, policy_version 1168256 (0.0006) [2023-12-27 13:45:03,892][20347] Updated weights for policy 0, policy_version 1169282 (0.0006) [2023-12-27 13:45:03,927][20381] Updated weights for policy 1, policy_version 1168266 (0.0005) [2023-12-27 13:45:03,956][20347] Updated weights for policy 0, policy_version 1169292 (0.0008) [2023-12-27 13:45:03,985][20381] Updated weights for policy 1, policy_version 1168276 (0.0006) [2023-12-27 13:45:04,664][20347] Updated weights for policy 0, policy_version 1169302 (0.0007) [2023-12-27 13:45:04,674][20381] Updated weights for policy 1, policy_version 1168286 (0.0006) [2023-12-27 13:45:04,717][20347] Updated weights for policy 0, policy_version 1169312 (0.0006) [2023-12-27 13:45:04,730][20381] Updated weights for policy 1, policy_version 1168296 (0.0007) [2023-12-27 13:45:04,773][20347] Updated weights for policy 0, policy_version 1169322 (0.0006) [2023-12-27 13:45:04,789][20381] Updated weights for policy 1, policy_version 1168306 (0.0007) [2023-12-27 13:45:05,411][20347] Updated weights for policy 0, policy_version 1169332 (0.0007) [2023-12-27 13:45:05,455][20347] Updated weights for policy 0, policy_version 1169342 (0.0008) [2023-12-27 13:45:05,469][20381] Updated weights for policy 1, policy_version 1168316 (0.0007) [2023-12-27 13:45:05,504][20347] Updated weights for policy 0, policy_version 1169352 (0.0007) [2023-12-27 13:45:05,535][20381] Updated weights for policy 1, policy_version 1168326 (0.0005) [2023-12-27 13:45:05,588][20381] Updated weights for policy 1, policy_version 1168336 (0.0005) [2023-12-27 13:45:06,120][20381] Updated weights for policy 1, policy_version 1168346 (0.0006) [2023-12-27 13:45:06,176][20381] Updated weights for policy 1, policy_version 1168356 (0.0006) [2023-12-27 13:45:06,234][20381] Updated weights for policy 1, policy_version 1168366 (0.0009) [2023-12-27 13:45:06,252][20347] Updated weights for policy 0, policy_version 1169362 (0.0007) [2023-12-27 13:45:06,298][20381] Updated weights for policy 1, policy_version 1168376 (0.0011) [2023-12-27 13:45:06,310][20347] Updated weights for policy 0, policy_version 1169372 (0.0006) [2023-12-27 13:45:06,373][20347] Updated weights for policy 0, policy_version 1169382 (0.0007) [2023-12-27 13:45:06,428][20347] Updated weights for policy 0, policy_version 1169392 (0.0008) [2023-12-27 13:45:06,872][20381] Updated weights for policy 1, policy_version 1168386 (0.0009) [2023-12-27 13:45:06,927][20381] Updated weights for policy 1, policy_version 1168396 (0.0011) [2023-12-27 13:45:06,987][20381] Updated weights for policy 1, policy_version 1168406 (0.0010) [2023-12-27 13:45:07,142][20347] Updated weights for policy 0, policy_version 1169402 (0.0007) [2023-12-27 13:45:07,207][20347] Updated weights for policy 0, policy_version 1169412 (0.0007) [2023-12-27 13:45:07,272][20347] Updated weights for policy 0, policy_version 1169422 (0.0007) [2023-12-27 13:45:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 598564864. Throughput: 0: 10189.4, 1: 10379.5. Samples: 598530004. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:07,363][19170] Avg episode reward: [(0, '902.630'), (1, '853.110')] [2023-12-27 13:45:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001168408_299139072.pth... [2023-12-27 13:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001169424_299425792.pth... [2023-12-27 13:45:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001167192_298827776.pth [2023-12-27 13:45:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001168240_299122688.pth [2023-12-27 13:45:07,604][20381] Updated weights for policy 1, policy_version 1168416 (0.0009) [2023-12-27 13:45:07,658][20381] Updated weights for policy 1, policy_version 1168426 (0.0010) [2023-12-27 13:45:07,709][20381] Updated weights for policy 1, policy_version 1168436 (0.0010) [2023-12-27 13:45:07,881][20347] Updated weights for policy 0, policy_version 1169432 (0.0007) [2023-12-27 13:45:07,935][20347] Updated weights for policy 0, policy_version 1169442 (0.0007) [2023-12-27 13:45:07,995][20347] Updated weights for policy 0, policy_version 1169452 (0.0007) [2023-12-27 13:45:08,417][20381] Updated weights for policy 1, policy_version 1168446 (0.0008) [2023-12-27 13:45:08,484][20381] Updated weights for policy 1, policy_version 1168456 (0.0007) [2023-12-27 13:45:08,532][20381] Updated weights for policy 1, policy_version 1168466 (0.0007) [2023-12-27 13:45:08,672][20347] Updated weights for policy 0, policy_version 1169462 (0.0007) [2023-12-27 13:45:08,728][20347] Updated weights for policy 0, policy_version 1169472 (0.0007) [2023-12-27 13:45:08,783][20347] Updated weights for policy 0, policy_version 1169482 (0.0007) [2023-12-27 13:45:09,202][20381] Updated weights for policy 1, policy_version 1168476 (0.0008) [2023-12-27 13:45:09,266][20381] Updated weights for policy 1, policy_version 1168486 (0.0008) [2023-12-27 13:45:09,321][20381] Updated weights for policy 1, policy_version 1168496 (0.0008) [2023-12-27 13:45:09,509][20347] Updated weights for policy 0, policy_version 1169492 (0.0006) [2023-12-27 13:45:09,573][20347] Updated weights for policy 0, policy_version 1169502 (0.0006) [2023-12-27 13:45:09,639][20347] Updated weights for policy 0, policy_version 1169512 (0.0007) [2023-12-27 13:45:10,066][20381] Updated weights for policy 1, policy_version 1168506 (0.0008) [2023-12-27 13:45:10,120][20381] Updated weights for policy 1, policy_version 1168516 (0.0011) [2023-12-27 13:45:10,171][20381] Updated weights for policy 1, policy_version 1168526 (0.0010) [2023-12-27 13:45:10,221][20381] Updated weights for policy 1, policy_version 1168536 (0.0011) [2023-12-27 13:45:10,311][20347] Updated weights for policy 0, policy_version 1169522 (0.0008) [2023-12-27 13:45:10,366][20347] Updated weights for policy 0, policy_version 1169532 (0.0008) [2023-12-27 13:45:10,421][20347] Updated weights for policy 0, policy_version 1169542 (0.0007) [2023-12-27 13:45:10,476][20347] Updated weights for policy 0, policy_version 1169552 (0.0005) [2023-12-27 13:45:10,883][20381] Updated weights for policy 1, policy_version 1168546 (0.0006) [2023-12-27 13:45:10,943][20381] Updated weights for policy 1, policy_version 1168556 (0.0005) [2023-12-27 13:45:11,000][20381] Updated weights for policy 1, policy_version 1168566 (0.0007) [2023-12-27 13:45:11,122][20347] Updated weights for policy 0, policy_version 1169562 (0.0006) [2023-12-27 13:45:11,185][20347] Updated weights for policy 0, policy_version 1169572 (0.0008) [2023-12-27 13:45:11,241][20347] Updated weights for policy 0, policy_version 1169582 (0.0010) [2023-12-27 13:45:11,641][20381] Updated weights for policy 1, policy_version 1168576 (0.0007) [2023-12-27 13:45:11,705][20381] Updated weights for policy 1, policy_version 1168586 (0.0007) [2023-12-27 13:45:11,771][20381] Updated weights for policy 1, policy_version 1168596 (0.0008) [2023-12-27 13:45:11,970][20347] Updated weights for policy 0, policy_version 1169592 (0.0009) [2023-12-27 13:45:12,018][20347] Updated weights for policy 0, policy_version 1169602 (0.0008) [2023-12-27 13:45:12,064][20347] Updated weights for policy 0, policy_version 1169612 (0.0006) [2023-12-27 13:45:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 598663168. Throughput: 0: 10220.0, 1: 10421.0. Samples: 598654672. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:12,363][19170] Avg episode reward: [(0, '892.870'), (1, '794.660')] [2023-12-27 13:45:12,487][20381] Updated weights for policy 1, policy_version 1168606 (0.0007) [2023-12-27 13:45:12,549][20381] Updated weights for policy 1, policy_version 1168616 (0.0007) [2023-12-27 13:45:12,607][20381] Updated weights for policy 1, policy_version 1168626 (0.0007) [2023-12-27 13:45:12,830][20347] Updated weights for policy 0, policy_version 1169622 (0.0008) [2023-12-27 13:45:12,888][20347] Updated weights for policy 0, policy_version 1169632 (0.0007) [2023-12-27 13:45:12,938][20347] Updated weights for policy 0, policy_version 1169642 (0.0008) [2023-12-27 13:45:13,248][20381] Updated weights for policy 1, policy_version 1168636 (0.0006) [2023-12-27 13:45:13,306][20381] Updated weights for policy 1, policy_version 1168646 (0.0008) [2023-12-27 13:45:13,358][20381] Updated weights for policy 1, policy_version 1168656 (0.0008) [2023-12-27 13:45:13,641][20347] Updated weights for policy 0, policy_version 1169652 (0.0008) [2023-12-27 13:45:13,695][20347] Updated weights for policy 0, policy_version 1169662 (0.0008) [2023-12-27 13:45:13,744][20347] Updated weights for policy 0, policy_version 1169672 (0.0008) [2023-12-27 13:45:13,982][20381] Updated weights for policy 1, policy_version 1168666 (0.0008) [2023-12-27 13:45:14,031][20381] Updated weights for policy 1, policy_version 1168676 (0.0009) [2023-12-27 13:45:14,084][20381] Updated weights for policy 1, policy_version 1168686 (0.0007) [2023-12-27 13:45:14,139][20381] Updated weights for policy 1, policy_version 1168696 (0.0006) [2023-12-27 13:45:14,458][20347] Updated weights for policy 0, policy_version 1169682 (0.0007) [2023-12-27 13:45:14,525][20347] Updated weights for policy 0, policy_version 1169692 (0.0007) [2023-12-27 13:45:14,559][20291] KL-divergence is very high: 150.9043 [2023-12-27 13:45:14,587][20347] Updated weights for policy 0, policy_version 1169702 (0.0008) [2023-12-27 13:45:14,639][20347] Updated weights for policy 0, policy_version 1169712 (0.0005) [2023-12-27 13:45:14,739][20381] Updated weights for policy 1, policy_version 1168706 (0.0005) [2023-12-27 13:45:14,807][20381] Updated weights for policy 1, policy_version 1168716 (0.0008) [2023-12-27 13:45:14,869][20381] Updated weights for policy 1, policy_version 1168726 (0.0007) [2023-12-27 13:45:15,320][20347] Updated weights for policy 0, policy_version 1169722 (0.0007) [2023-12-27 13:45:15,370][20347] Updated weights for policy 0, policy_version 1169732 (0.0005) [2023-12-27 13:45:15,419][20347] Updated weights for policy 0, policy_version 1169742 (0.0005) [2023-12-27 13:45:15,535][20381] Updated weights for policy 1, policy_version 1168736 (0.0006) [2023-12-27 13:45:15,583][20381] Updated weights for policy 1, policy_version 1168746 (0.0005) [2023-12-27 13:45:15,634][20381] Updated weights for policy 1, policy_version 1168756 (0.0006) [2023-12-27 13:45:16,086][20347] Updated weights for policy 0, policy_version 1169752 (0.0006) [2023-12-27 13:45:16,147][20347] Updated weights for policy 0, policy_version 1169762 (0.0005) [2023-12-27 13:45:16,209][20347] Updated weights for policy 0, policy_version 1169772 (0.0005) [2023-12-27 13:45:16,243][20381] Updated weights for policy 1, policy_version 1168766 (0.0006) [2023-12-27 13:45:16,288][20381] Updated weights for policy 1, policy_version 1168776 (0.0007) [2023-12-27 13:45:16,341][20381] Updated weights for policy 1, policy_version 1168786 (0.0009) [2023-12-27 13:45:16,797][20347] Updated weights for policy 0, policy_version 1169782 (0.0006) [2023-12-27 13:45:16,847][20347] Updated weights for policy 0, policy_version 1169792 (0.0007) [2023-12-27 13:45:16,894][20347] Updated weights for policy 0, policy_version 1169802 (0.0008) [2023-12-27 13:45:16,983][20381] Updated weights for policy 1, policy_version 1168796 (0.0007) [2023-12-27 13:45:17,031][20381] Updated weights for policy 1, policy_version 1168806 (0.0007) [2023-12-27 13:45:17,079][20381] Updated weights for policy 1, policy_version 1168816 (0.0008) [2023-12-27 13:45:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 598769664. Throughput: 0: 10140.3, 1: 10510.2. Samples: 598778476. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:17,363][19170] Avg episode reward: [(0, '863.840'), (1, '804.370')] [2023-12-27 13:45:17,585][20347] Updated weights for policy 0, policy_version 1169812 (0.0007) [2023-12-27 13:45:17,641][20347] Updated weights for policy 0, policy_version 1169822 (0.0009) [2023-12-27 13:45:17,700][20347] Updated weights for policy 0, policy_version 1169832 (0.0011) [2023-12-27 13:45:17,801][20381] Updated weights for policy 1, policy_version 1168826 (0.0008) [2023-12-27 13:45:17,853][20381] Updated weights for policy 1, policy_version 1168836 (0.0007) [2023-12-27 13:45:17,910][20381] Updated weights for policy 1, policy_version 1168846 (0.0007) [2023-12-27 13:45:17,958][20381] Updated weights for policy 1, policy_version 1168856 (0.0007) [2023-12-27 13:45:18,375][20347] Updated weights for policy 0, policy_version 1169842 (0.0010) [2023-12-27 13:45:18,429][20347] Updated weights for policy 0, policy_version 1169852 (0.0008) [2023-12-27 13:45:18,489][20347] Updated weights for policy 0, policy_version 1169862 (0.0007) [2023-12-27 13:45:18,536][20347] Updated weights for policy 0, policy_version 1169872 (0.0005) [2023-12-27 13:45:18,676][20381] Updated weights for policy 1, policy_version 1168866 (0.0007) [2023-12-27 13:45:18,737][20381] Updated weights for policy 1, policy_version 1168876 (0.0008) [2023-12-27 13:45:18,793][20381] Updated weights for policy 1, policy_version 1168886 (0.0006) [2023-12-27 13:45:19,252][20347] Updated weights for policy 0, policy_version 1169882 (0.0008) [2023-12-27 13:45:19,305][20347] Updated weights for policy 0, policy_version 1169892 (0.0009) [2023-12-27 13:45:19,372][20347] Updated weights for policy 0, policy_version 1169902 (0.0008) [2023-12-27 13:45:19,517][20381] Updated weights for policy 1, policy_version 1168896 (0.0006) [2023-12-27 13:45:19,579][20381] Updated weights for policy 1, policy_version 1168906 (0.0007) [2023-12-27 13:45:19,643][20381] Updated weights for policy 1, policy_version 1168916 (0.0008) [2023-12-27 13:45:20,069][20347] Updated weights for policy 0, policy_version 1169912 (0.0007) [2023-12-27 13:45:20,133][20347] Updated weights for policy 0, policy_version 1169922 (0.0008) [2023-12-27 13:45:20,190][20347] Updated weights for policy 0, policy_version 1169932 (0.0007) [2023-12-27 13:45:20,348][20381] Updated weights for policy 1, policy_version 1168926 (0.0009) [2023-12-27 13:45:20,407][20381] Updated weights for policy 1, policy_version 1168936 (0.0007) [2023-12-27 13:45:20,466][20381] Updated weights for policy 1, policy_version 1168946 (0.0008) [2023-12-27 13:45:20,896][20347] Updated weights for policy 0, policy_version 1169942 (0.0008) [2023-12-27 13:45:20,958][20347] Updated weights for policy 0, policy_version 1169952 (0.0008) [2023-12-27 13:45:21,017][20347] Updated weights for policy 0, policy_version 1169962 (0.0008) [2023-12-27 13:45:21,079][20381] Updated weights for policy 1, policy_version 1168956 (0.0007) [2023-12-27 13:45:21,145][20381] Updated weights for policy 1, policy_version 1168966 (0.0008) [2023-12-27 13:45:21,211][20381] Updated weights for policy 1, policy_version 1168976 (0.0008) [2023-12-27 13:45:21,845][20347] Updated weights for policy 0, policy_version 1169972 (0.0007) [2023-12-27 13:45:21,915][20347] Updated weights for policy 0, policy_version 1169982 (0.0007) [2023-12-27 13:45:21,972][20347] Updated weights for policy 0, policy_version 1169992 (0.0008) [2023-12-27 13:45:22,017][20381] Updated weights for policy 1, policy_version 1168986 (0.0008) [2023-12-27 13:45:22,077][20381] Updated weights for policy 1, policy_version 1168996 (0.0007) [2023-12-27 13:45:22,132][20381] Updated weights for policy 1, policy_version 1169006 (0.0010) [2023-12-27 13:45:22,192][20381] Updated weights for policy 1, policy_version 1169016 (0.0011) [2023-12-27 13:45:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 598867968. Throughput: 0: 10106.5, 1: 10420.8. Samples: 598838460. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:22,363][19170] Avg episode reward: [(0, '903.070'), (1, '853.200')] [2023-12-27 13:45:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001169016_299294720.pth... [2023-12-27 13:45:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001170000_299573248.pth... [2023-12-27 13:45:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001168816_299270144.pth [2023-12-27 13:45:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001167800_298983424.pth [2023-12-27 13:45:22,686][20347] Updated weights for policy 0, policy_version 1170002 (0.0007) [2023-12-27 13:45:22,751][20347] Updated weights for policy 0, policy_version 1170012 (0.0007) [2023-12-27 13:45:22,819][20347] Updated weights for policy 0, policy_version 1170022 (0.0008) [2023-12-27 13:45:22,875][20381] Updated weights for policy 1, policy_version 1169026 (0.0011) [2023-12-27 13:45:22,881][20347] Updated weights for policy 0, policy_version 1170032 (0.0007) [2023-12-27 13:45:22,931][20381] Updated weights for policy 1, policy_version 1169036 (0.0009) [2023-12-27 13:45:22,993][20381] Updated weights for policy 1, policy_version 1169046 (0.0008) [2023-12-27 13:45:23,469][20347] Updated weights for policy 0, policy_version 1170042 (0.0005) [2023-12-27 13:45:23,521][20347] Updated weights for policy 0, policy_version 1170052 (0.0005) [2023-12-27 13:45:23,573][20347] Updated weights for policy 0, policy_version 1170062 (0.0005) [2023-12-27 13:45:23,625][20381] Updated weights for policy 1, policy_version 1169056 (0.0005) [2023-12-27 13:45:23,674][20381] Updated weights for policy 1, policy_version 1169066 (0.0005) [2023-12-27 13:45:23,723][20381] Updated weights for policy 1, policy_version 1169076 (0.0005) [2023-12-27 13:45:24,145][20347] Updated weights for policy 0, policy_version 1170072 (0.0005) [2023-12-27 13:45:24,203][20347] Updated weights for policy 0, policy_version 1170082 (0.0006) [2023-12-27 13:45:24,268][20347] Updated weights for policy 0, policy_version 1170092 (0.0006) [2023-12-27 13:45:24,305][20381] Updated weights for policy 1, policy_version 1169086 (0.0009) [2023-12-27 13:45:24,357][20381] Updated weights for policy 1, policy_version 1169096 (0.0011) [2023-12-27 13:45:24,413][20381] Updated weights for policy 1, policy_version 1169106 (0.0011) [2023-12-27 13:45:24,869][20347] Updated weights for policy 0, policy_version 1170102 (0.0006) [2023-12-27 13:45:24,921][20347] Updated weights for policy 0, policy_version 1170112 (0.0007) [2023-12-27 13:45:24,971][20347] Updated weights for policy 0, policy_version 1170122 (0.0007) [2023-12-27 13:45:25,065][20381] Updated weights for policy 1, policy_version 1169116 (0.0009) [2023-12-27 13:45:25,114][20381] Updated weights for policy 1, policy_version 1169126 (0.0010) [2023-12-27 13:45:25,164][20381] Updated weights for policy 1, policy_version 1169136 (0.0010) [2023-12-27 13:45:25,642][20347] Updated weights for policy 0, policy_version 1170132 (0.0007) [2023-12-27 13:45:25,702][20347] Updated weights for policy 0, policy_version 1170142 (0.0007) [2023-12-27 13:45:25,754][20347] Updated weights for policy 0, policy_version 1170152 (0.0008) [2023-12-27 13:45:25,816][20381] Updated weights for policy 1, policy_version 1169146 (0.0009) [2023-12-27 13:45:25,870][20381] Updated weights for policy 1, policy_version 1169156 (0.0005) [2023-12-27 13:45:25,919][20381] Updated weights for policy 1, policy_version 1169166 (0.0005) [2023-12-27 13:45:25,972][20381] Updated weights for policy 1, policy_version 1169176 (0.0006) [2023-12-27 13:45:26,458][20347] Updated weights for policy 0, policy_version 1170162 (0.0007) [2023-12-27 13:45:26,516][20347] Updated weights for policy 0, policy_version 1170172 (0.0005) [2023-12-27 13:45:26,577][20347] Updated weights for policy 0, policy_version 1170182 (0.0005) [2023-12-27 13:45:26,629][20347] Updated weights for policy 0, policy_version 1170192 (0.0005) [2023-12-27 13:45:26,638][20381] Updated weights for policy 1, policy_version 1169186 (0.0008) [2023-12-27 13:45:26,695][20381] Updated weights for policy 1, policy_version 1169196 (0.0007) [2023-12-27 13:45:26,745][20381] Updated weights for policy 1, policy_version 1169206 (0.0008) [2023-12-27 13:45:27,207][20347] Updated weights for policy 0, policy_version 1170202 (0.0010) [2023-12-27 13:45:27,261][20347] Updated weights for policy 0, policy_version 1170212 (0.0010) [2023-12-27 13:45:27,309][20347] Updated weights for policy 0, policy_version 1170222 (0.0010) [2023-12-27 13:45:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 598974464. Throughput: 0: 10115.6, 1: 10438.5. Samples: 598961404. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:27,363][19170] Avg episode reward: [(0, '894.050'), (1, '843.300')] [2023-12-27 13:45:27,440][20381] Updated weights for policy 1, policy_version 1169216 (0.0007) [2023-12-27 13:45:27,497][20381] Updated weights for policy 1, policy_version 1169226 (0.0009) [2023-12-27 13:45:27,551][20381] Updated weights for policy 1, policy_version 1169236 (0.0006) [2023-12-27 13:45:27,918][20347] Updated weights for policy 0, policy_version 1170232 (0.0008) [2023-12-27 13:45:27,984][20347] Updated weights for policy 0, policy_version 1170242 (0.0007) [2023-12-27 13:45:28,044][20347] Updated weights for policy 0, policy_version 1170252 (0.0008) [2023-12-27 13:45:28,209][20381] Updated weights for policy 1, policy_version 1169246 (0.0007) [2023-12-27 13:45:28,265][20381] Updated weights for policy 1, policy_version 1169256 (0.0008) [2023-12-27 13:45:28,324][20381] Updated weights for policy 1, policy_version 1169266 (0.0007) [2023-12-27 13:45:28,752][20347] Updated weights for policy 0, policy_version 1170262 (0.0009) [2023-12-27 13:45:28,802][20347] Updated weights for policy 0, policy_version 1170272 (0.0010) [2023-12-27 13:45:28,861][20347] Updated weights for policy 0, policy_version 1170282 (0.0011) [2023-12-27 13:45:29,060][20381] Updated weights for policy 1, policy_version 1169276 (0.0007) [2023-12-27 13:45:29,113][20381] Updated weights for policy 1, policy_version 1169286 (0.0009) [2023-12-27 13:45:29,172][20381] Updated weights for policy 1, policy_version 1169296 (0.0010) [2023-12-27 13:45:29,583][20347] Updated weights for policy 0, policy_version 1170292 (0.0009) [2023-12-27 13:45:29,638][20347] Updated weights for policy 0, policy_version 1170302 (0.0007) [2023-12-27 13:45:29,697][20347] Updated weights for policy 0, policy_version 1170312 (0.0006) [2023-12-27 13:45:29,923][20381] Updated weights for policy 1, policy_version 1169306 (0.0008) [2023-12-27 13:45:29,987][20381] Updated weights for policy 1, policy_version 1169316 (0.0008) [2023-12-27 13:45:30,044][20381] Updated weights for policy 1, policy_version 1169326 (0.0008) [2023-12-27 13:45:30,100][20381] Updated weights for policy 1, policy_version 1169336 (0.0008) [2023-12-27 13:45:30,376][20347] Updated weights for policy 0, policy_version 1170322 (0.0007) [2023-12-27 13:45:30,420][20291] KL-divergence is very high: 209.5832 [2023-12-27 13:45:30,429][20347] Updated weights for policy 0, policy_version 1170332 (0.0009) [2023-12-27 13:45:30,436][20291] KL-divergence is very high: 107.5774 [2023-12-27 13:45:30,486][20347] Updated weights for policy 0, policy_version 1170342 (0.0005) [2023-12-27 13:45:30,535][20347] Updated weights for policy 0, policy_version 1170352 (0.0008) [2023-12-27 13:45:30,830][20381] Updated weights for policy 1, policy_version 1169346 (0.0006) [2023-12-27 13:45:30,884][20381] Updated weights for policy 1, policy_version 1169356 (0.0005) [2023-12-27 13:45:30,942][20381] Updated weights for policy 1, policy_version 1169366 (0.0005) [2023-12-27 13:45:31,186][20347] Updated weights for policy 0, policy_version 1170362 (0.0007) [2023-12-27 13:45:31,245][20347] Updated weights for policy 0, policy_version 1170372 (0.0010) [2023-12-27 13:45:31,258][20291] KL-divergence is very high: 136.7356 [2023-12-27 13:45:31,312][20347] Updated weights for policy 0, policy_version 1170382 (0.0010) [2023-12-27 13:45:31,641][20381] Updated weights for policy 1, policy_version 1169376 (0.0008) [2023-12-27 13:45:31,702][20381] Updated weights for policy 1, policy_version 1169386 (0.0008) [2023-12-27 13:45:31,770][20381] Updated weights for policy 1, policy_version 1169396 (0.0009) [2023-12-27 13:45:32,025][20347] Updated weights for policy 0, policy_version 1170392 (0.0009) [2023-12-27 13:45:32,079][20347] Updated weights for policy 0, policy_version 1170402 (0.0009) [2023-12-27 13:45:32,127][20347] Updated weights for policy 0, policy_version 1170412 (0.0009) [2023-12-27 13:45:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 599072768. Throughput: 0: 10166.0, 1: 10341.5. Samples: 599081528. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:32,363][19170] Avg episode reward: [(0, '854.680'), (1, '804.060')] [2023-12-27 13:45:32,552][20381] Updated weights for policy 1, policy_version 1169406 (0.0009) [2023-12-27 13:45:32,605][20381] Updated weights for policy 1, policy_version 1169416 (0.0008) [2023-12-27 13:45:32,665][20381] Updated weights for policy 1, policy_version 1169426 (0.0008) [2023-12-27 13:45:32,875][20347] Updated weights for policy 0, policy_version 1170422 (0.0007) [2023-12-27 13:45:32,924][20347] Updated weights for policy 0, policy_version 1170432 (0.0005) [2023-12-27 13:45:32,970][20347] Updated weights for policy 0, policy_version 1170442 (0.0005) [2023-12-27 13:45:33,366][20381] Updated weights for policy 1, policy_version 1169436 (0.0009) [2023-12-27 13:45:33,417][20381] Updated weights for policy 1, policy_version 1169446 (0.0008) [2023-12-27 13:45:33,473][20381] Updated weights for policy 1, policy_version 1169456 (0.0007) [2023-12-27 13:45:33,627][20347] Updated weights for policy 0, policy_version 1170452 (0.0006) [2023-12-27 13:45:33,678][20347] Updated weights for policy 0, policy_version 1170462 (0.0006) [2023-12-27 13:45:33,734][20347] Updated weights for policy 0, policy_version 1170472 (0.0008) [2023-12-27 13:45:34,155][20381] Updated weights for policy 1, policy_version 1169466 (0.0008) [2023-12-27 13:45:34,219][20381] Updated weights for policy 1, policy_version 1169476 (0.0010) [2023-12-27 13:45:34,285][20381] Updated weights for policy 1, policy_version 1169486 (0.0008) [2023-12-27 13:45:34,344][20381] Updated weights for policy 1, policy_version 1169496 (0.0010) [2023-12-27 13:45:34,415][20347] Updated weights for policy 0, policy_version 1170482 (0.0007) [2023-12-27 13:45:34,478][20347] Updated weights for policy 0, policy_version 1170492 (0.0009) [2023-12-27 13:45:34,535][20347] Updated weights for policy 0, policy_version 1170502 (0.0009) [2023-12-27 13:45:34,591][20347] Updated weights for policy 0, policy_version 1170512 (0.0007) [2023-12-27 13:45:34,947][20381] Updated weights for policy 1, policy_version 1169506 (0.0005) [2023-12-27 13:45:35,004][20381] Updated weights for policy 1, policy_version 1169516 (0.0009) [2023-12-27 13:45:35,059][20381] Updated weights for policy 1, policy_version 1169526 (0.0010) [2023-12-27 13:45:35,290][20347] Updated weights for policy 0, policy_version 1170522 (0.0008) [2023-12-27 13:45:35,341][20347] Updated weights for policy 0, policy_version 1170532 (0.0008) [2023-12-27 13:45:35,404][20347] Updated weights for policy 0, policy_version 1170542 (0.0008) [2023-12-27 13:45:35,712][20381] Updated weights for policy 1, policy_version 1169536 (0.0010) [2023-12-27 13:45:35,777][20381] Updated weights for policy 1, policy_version 1169546 (0.0011) [2023-12-27 13:45:35,831][20381] Updated weights for policy 1, policy_version 1169556 (0.0010) [2023-12-27 13:45:36,107][20347] Updated weights for policy 0, policy_version 1170552 (0.0007) [2023-12-27 13:45:36,167][20347] Updated weights for policy 0, policy_version 1170562 (0.0007) [2023-12-27 13:45:36,221][20347] Updated weights for policy 0, policy_version 1170572 (0.0007) [2023-12-27 13:45:36,509][20381] Updated weights for policy 1, policy_version 1169566 (0.0009) [2023-12-27 13:45:36,568][20381] Updated weights for policy 1, policy_version 1169576 (0.0008) [2023-12-27 13:45:36,623][20381] Updated weights for policy 1, policy_version 1169586 (0.0008) [2023-12-27 13:45:36,925][20347] Updated weights for policy 0, policy_version 1170582 (0.0006) [2023-12-27 13:45:36,979][20347] Updated weights for policy 0, policy_version 1170592 (0.0009) [2023-12-27 13:45:37,027][20347] Updated weights for policy 0, policy_version 1170602 (0.0009) [2023-12-27 13:45:37,205][20381] Updated weights for policy 1, policy_version 1169596 (0.0010) [2023-12-27 13:45:37,251][20381] Updated weights for policy 1, policy_version 1169606 (0.0010) [2023-12-27 13:45:37,296][20381] Updated weights for policy 1, policy_version 1169616 (0.0010) [2023-12-27 13:45:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 599179264. Throughput: 0: 10146.4, 1: 10347.4. Samples: 599143296. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 13:45:37,363][19170] Avg episode reward: [(0, '922.550'), (1, '852.990')] [2023-12-27 13:45:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001169624_299450368.pth... [2023-12-27 13:45:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001170608_299728896.pth... [2023-12-27 13:45:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001168408_299139072.pth [2023-12-27 13:45:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001169424_299425792.pth [2023-12-27 13:45:37,733][20347] Updated weights for policy 0, policy_version 1170612 (0.0008) [2023-12-27 13:45:37,790][20347] Updated weights for policy 0, policy_version 1170622 (0.0008) [2023-12-27 13:45:37,835][20347] Updated weights for policy 0, policy_version 1170632 (0.0009) [2023-12-27 13:45:37,955][20381] Updated weights for policy 1, policy_version 1169626 (0.0009) [2023-12-27 13:45:38,011][20381] Updated weights for policy 1, policy_version 1169636 (0.0007) [2023-12-27 13:45:38,066][20381] Updated weights for policy 1, policy_version 1169646 (0.0010) [2023-12-27 13:45:38,117][20381] Updated weights for policy 1, policy_version 1169656 (0.0010) [2023-12-27 13:45:38,566][20347] Updated weights for policy 0, policy_version 1170642 (0.0007) [2023-12-27 13:45:38,621][20347] Updated weights for policy 0, policy_version 1170652 (0.0007) [2023-12-27 13:45:38,672][20347] Updated weights for policy 0, policy_version 1170662 (0.0007) [2023-12-27 13:45:38,720][20347] Updated weights for policy 0, policy_version 1170672 (0.0006) [2023-12-27 13:45:38,841][20381] Updated weights for policy 1, policy_version 1169666 (0.0006) [2023-12-27 13:45:38,892][20381] Updated weights for policy 1, policy_version 1169676 (0.0010) [2023-12-27 13:45:38,945][20381] Updated weights for policy 1, policy_version 1169686 (0.0011) [2023-12-27 13:45:39,512][20347] Updated weights for policy 0, policy_version 1170682 (0.0007) [2023-12-27 13:45:39,565][20347] Updated weights for policy 0, policy_version 1170692 (0.0008) [2023-12-27 13:45:39,617][20347] Updated weights for policy 0, policy_version 1170702 (0.0007) [2023-12-27 13:45:39,696][20381] Updated weights for policy 1, policy_version 1169696 (0.0011) [2023-12-27 13:45:39,757][20381] Updated weights for policy 1, policy_version 1169706 (0.0011) [2023-12-27 13:45:39,814][20381] Updated weights for policy 1, policy_version 1169716 (0.0011) [2023-12-27 13:45:40,360][20347] Updated weights for policy 0, policy_version 1170712 (0.0007) [2023-12-27 13:45:40,416][20347] Updated weights for policy 0, policy_version 1170722 (0.0007) [2023-12-27 13:45:40,474][20347] Updated weights for policy 0, policy_version 1170732 (0.0008) [2023-12-27 13:45:40,491][20381] Updated weights for policy 1, policy_version 1169726 (0.0007) [2023-12-27 13:45:40,553][20381] Updated weights for policy 1, policy_version 1169736 (0.0008) [2023-12-27 13:45:40,613][20381] Updated weights for policy 1, policy_version 1169746 (0.0008) [2023-12-27 13:45:41,146][20347] Updated weights for policy 0, policy_version 1170742 (0.0007) [2023-12-27 13:45:41,210][20347] Updated weights for policy 0, policy_version 1170752 (0.0010) [2023-12-27 13:45:41,276][20347] Updated weights for policy 0, policy_version 1170762 (0.0010) [2023-12-27 13:45:41,328][20381] Updated weights for policy 1, policy_version 1169756 (0.0010) [2023-12-27 13:45:41,393][20381] Updated weights for policy 1, policy_version 1169766 (0.0010) [2023-12-27 13:45:41,454][20381] Updated weights for policy 1, policy_version 1169776 (0.0011) [2023-12-27 13:45:41,971][20347] Updated weights for policy 0, policy_version 1170772 (0.0010) [2023-12-27 13:45:42,031][20347] Updated weights for policy 0, policy_version 1170782 (0.0011) [2023-12-27 13:45:42,094][20347] Updated weights for policy 0, policy_version 1170792 (0.0008) [2023-12-27 13:45:42,195][20381] Updated weights for policy 1, policy_version 1169786 (0.0010) [2023-12-27 13:45:42,256][20381] Updated weights for policy 1, policy_version 1169796 (0.0006) [2023-12-27 13:45:42,321][20381] Updated weights for policy 1, policy_version 1169806 (0.0007) [2023-12-27 13:45:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 599269376. Throughput: 0: 10065.4, 1: 10317.3. Samples: 599263520. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:45:42,363][19170] Avg episode reward: [(0, '951.600'), (1, '833.480')] [2023-12-27 13:45:42,390][20381] Updated weights for policy 1, policy_version 1169816 (0.0009) [2023-12-27 13:45:42,827][20347] Updated weights for policy 0, policy_version 1170802 (0.0010) [2023-12-27 13:45:42,884][20347] Updated weights for policy 0, policy_version 1170812 (0.0008) [2023-12-27 13:45:42,932][20347] Updated weights for policy 0, policy_version 1170822 (0.0007) [2023-12-27 13:45:42,979][20347] Updated weights for policy 0, policy_version 1170832 (0.0006) [2023-12-27 13:45:43,097][20381] Updated weights for policy 1, policy_version 1169826 (0.0008) [2023-12-27 13:45:43,162][20381] Updated weights for policy 1, policy_version 1169836 (0.0008) [2023-12-27 13:45:43,226][20381] Updated weights for policy 1, policy_version 1169846 (0.0008) [2023-12-27 13:45:43,698][20347] Updated weights for policy 0, policy_version 1170842 (0.0008) [2023-12-27 13:45:43,747][20347] Updated weights for policy 0, policy_version 1170852 (0.0008) [2023-12-27 13:45:43,794][20347] Updated weights for policy 0, policy_version 1170862 (0.0006) [2023-12-27 13:45:43,929][20381] Updated weights for policy 1, policy_version 1169856 (0.0007) [2023-12-27 13:45:43,979][20381] Updated weights for policy 1, policy_version 1169866 (0.0008) [2023-12-27 13:45:44,025][20381] Updated weights for policy 1, policy_version 1169876 (0.0007) [2023-12-27 13:45:44,529][20347] Updated weights for policy 0, policy_version 1170872 (0.0006) [2023-12-27 13:45:44,542][20291] KL-divergence is very high: 333.3484 [2023-12-27 13:45:44,597][20347] Updated weights for policy 0, policy_version 1170882 (0.0007) [2023-12-27 13:45:44,597][20291] KL-divergence is very high: 340.4865 [2023-12-27 13:45:44,628][20381] Updated weights for policy 1, policy_version 1169886 (0.0007) [2023-12-27 13:45:44,643][20291] KL-divergence is very high: 101.2266 [2023-12-27 13:45:44,651][20347] Updated weights for policy 0, policy_version 1170892 (0.0005) [2023-12-27 13:45:44,682][20381] Updated weights for policy 1, policy_version 1169896 (0.0005) [2023-12-27 13:45:44,730][20381] Updated weights for policy 1, policy_version 1169906 (0.0008) [2023-12-27 13:45:45,333][20347] Updated weights for policy 0, policy_version 1170902 (0.0006) [2023-12-27 13:45:45,384][20347] Updated weights for policy 0, policy_version 1170912 (0.0007) [2023-12-27 13:45:45,394][20381] Updated weights for policy 1, policy_version 1169916 (0.0008) [2023-12-27 13:45:45,432][20347] Updated weights for policy 0, policy_version 1170922 (0.0005) [2023-12-27 13:45:45,448][20381] Updated weights for policy 1, policy_version 1169926 (0.0010) [2023-12-27 13:45:45,504][20381] Updated weights for policy 1, policy_version 1169936 (0.0005) [2023-12-27 13:45:46,097][20381] Updated weights for policy 1, policy_version 1169946 (0.0006) [2023-12-27 13:45:46,155][20381] Updated weights for policy 1, policy_version 1169956 (0.0010) [2023-12-27 13:45:46,192][20347] Updated weights for policy 0, policy_version 1170932 (0.0006) [2023-12-27 13:45:46,209][20381] Updated weights for policy 1, policy_version 1169966 (0.0011) [2023-12-27 13:45:46,244][20347] Updated weights for policy 0, policy_version 1170942 (0.0006) [2023-12-27 13:45:46,264][20381] Updated weights for policy 1, policy_version 1169976 (0.0010) [2023-12-27 13:45:46,295][20291] KL-divergence is very high: 101.2897 [2023-12-27 13:45:46,296][20347] Updated weights for policy 0, policy_version 1170952 (0.0006) [2023-12-27 13:45:46,845][20381] Updated weights for policy 1, policy_version 1169986 (0.0005) [2023-12-27 13:45:46,901][20381] Updated weights for policy 1, policy_version 1169996 (0.0006) [2023-12-27 13:45:46,962][20381] Updated weights for policy 1, policy_version 1170006 (0.0005) [2023-12-27 13:45:47,003][20347] Updated weights for policy 0, policy_version 1170962 (0.0008) [2023-12-27 13:45:47,062][20347] Updated weights for policy 0, policy_version 1170972 (0.0008) [2023-12-27 13:45:47,113][20347] Updated weights for policy 0, policy_version 1170982 (0.0006) [2023-12-27 13:45:47,172][20347] Updated weights for policy 0, policy_version 1170992 (0.0005) [2023-12-27 13:45:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 599375872. Throughput: 0: 10035.5, 1: 10344.2. Samples: 599384712. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:45:47,364][19170] Avg episode reward: [(0, '951.230'), (1, '833.410')] [2023-12-27 13:45:47,516][20381] Updated weights for policy 1, policy_version 1170016 (0.0005) [2023-12-27 13:45:47,569][20381] Updated weights for policy 1, policy_version 1170026 (0.0005) [2023-12-27 13:45:47,622][20381] Updated weights for policy 1, policy_version 1170036 (0.0008) [2023-12-27 13:45:47,790][20347] Updated weights for policy 0, policy_version 1171002 (0.0007) [2023-12-27 13:45:47,843][20347] Updated weights for policy 0, policy_version 1171012 (0.0007) [2023-12-27 13:45:47,896][20347] Updated weights for policy 0, policy_version 1171022 (0.0007) [2023-12-27 13:45:48,263][20381] Updated weights for policy 1, policy_version 1170046 (0.0007) [2023-12-27 13:45:48,322][20381] Updated weights for policy 1, policy_version 1170056 (0.0006) [2023-12-27 13:45:48,386][20381] Updated weights for policy 1, policy_version 1170066 (0.0007) [2023-12-27 13:45:48,602][20347] Updated weights for policy 0, policy_version 1171032 (0.0010) [2023-12-27 13:45:48,661][20347] Updated weights for policy 0, policy_version 1171042 (0.0010) [2023-12-27 13:45:48,723][20347] Updated weights for policy 0, policy_version 1171052 (0.0010) [2023-12-27 13:45:49,040][20381] Updated weights for policy 1, policy_version 1170076 (0.0008) [2023-12-27 13:45:49,086][20381] Updated weights for policy 1, policy_version 1170086 (0.0010) [2023-12-27 13:45:49,132][20381] Updated weights for policy 1, policy_version 1170096 (0.0010) [2023-12-27 13:45:49,434][20291] KL-divergence is very high: 174.9617 [2023-12-27 13:45:49,452][20347] Updated weights for policy 0, policy_version 1171062 (0.0007) [2023-12-27 13:45:49,482][20291] KL-divergence is very high: 140.8133 [2023-12-27 13:45:49,508][20347] Updated weights for policy 0, policy_version 1171072 (0.0007) [2023-12-27 13:45:49,562][20347] Updated weights for policy 0, policy_version 1171082 (0.0005) [2023-12-27 13:45:49,878][20381] Updated weights for policy 1, policy_version 1170106 (0.0009) [2023-12-27 13:45:49,940][20381] Updated weights for policy 1, policy_version 1170116 (0.0008) [2023-12-27 13:45:50,003][20381] Updated weights for policy 1, policy_version 1170126 (0.0007) [2023-12-27 13:45:50,068][20381] Updated weights for policy 1, policy_version 1170136 (0.0008) [2023-12-27 13:45:50,230][20347] Updated weights for policy 0, policy_version 1171092 (0.0006) [2023-12-27 13:45:50,277][20347] Updated weights for policy 0, policy_version 1171102 (0.0005) [2023-12-27 13:45:50,323][20347] Updated weights for policy 0, policy_version 1171112 (0.0005) [2023-12-27 13:45:50,800][20381] Updated weights for policy 1, policy_version 1170146 (0.0009) [2023-12-27 13:45:50,860][20381] Updated weights for policy 1, policy_version 1170156 (0.0007) [2023-12-27 13:45:50,917][20381] Updated weights for policy 1, policy_version 1170166 (0.0007) [2023-12-27 13:45:50,988][20347] Updated weights for policy 0, policy_version 1171122 (0.0005) [2023-12-27 13:45:51,054][20347] Updated weights for policy 0, policy_version 1171132 (0.0007) [2023-12-27 13:45:51,115][20347] Updated weights for policy 0, policy_version 1171142 (0.0010) [2023-12-27 13:45:51,177][20347] Updated weights for policy 0, policy_version 1171152 (0.0010) [2023-12-27 13:45:51,638][20381] Updated weights for policy 1, policy_version 1170176 (0.0007) [2023-12-27 13:45:51,697][20381] Updated weights for policy 1, policy_version 1170186 (0.0008) [2023-12-27 13:45:51,765][20381] Updated weights for policy 1, policy_version 1170196 (0.0008) [2023-12-27 13:45:51,906][20347] Updated weights for policy 0, policy_version 1171162 (0.0006) [2023-12-27 13:45:51,963][20347] Updated weights for policy 0, policy_version 1171172 (0.0007) [2023-12-27 13:45:52,023][20347] Updated weights for policy 0, policy_version 1171182 (0.0008) [2023-12-27 13:45:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 599474176. Throughput: 0: 10040.3, 1: 10316.9. Samples: 599446076. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:45:52,363][19170] Avg episode reward: [(0, '941.520'), (1, '754.950')] [2023-12-27 13:45:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001170200_299597824.pth... [2023-12-27 13:45:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001171184_299876352.pth... [2023-12-27 13:45:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001169016_299294720.pth [2023-12-27 13:45:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001170000_299573248.pth [2023-12-27 13:45:52,494][20381] Updated weights for policy 1, policy_version 1170206 (0.0007) [2023-12-27 13:45:52,553][20381] Updated weights for policy 1, policy_version 1170216 (0.0007) [2023-12-27 13:45:52,609][20381] Updated weights for policy 1, policy_version 1170226 (0.0007) [2023-12-27 13:45:52,741][20347] Updated weights for policy 0, policy_version 1171192 (0.0010) [2023-12-27 13:45:52,792][20347] Updated weights for policy 0, policy_version 1171202 (0.0011) [2023-12-27 13:45:52,849][20347] Updated weights for policy 0, policy_version 1171212 (0.0011) [2023-12-27 13:45:53,310][20381] Updated weights for policy 1, policy_version 1170236 (0.0007) [2023-12-27 13:45:53,362][20381] Updated weights for policy 1, policy_version 1170246 (0.0007) [2023-12-27 13:45:53,412][20381] Updated weights for policy 1, policy_version 1170256 (0.0008) [2023-12-27 13:45:53,558][20347] Updated weights for policy 0, policy_version 1171222 (0.0010) [2023-12-27 13:45:53,607][20347] Updated weights for policy 0, policy_version 1171232 (0.0008) [2023-12-27 13:45:53,653][20347] Updated weights for policy 0, policy_version 1171242 (0.0008) [2023-12-27 13:45:54,032][20381] Updated weights for policy 1, policy_version 1170266 (0.0008) [2023-12-27 13:45:54,088][20381] Updated weights for policy 1, policy_version 1170276 (0.0008) [2023-12-27 13:45:54,146][20381] Updated weights for policy 1, policy_version 1170286 (0.0006) [2023-12-27 13:45:54,200][20381] Updated weights for policy 1, policy_version 1170296 (0.0007) [2023-12-27 13:45:54,327][20347] Updated weights for policy 0, policy_version 1171252 (0.0006) [2023-12-27 13:45:54,386][20347] Updated weights for policy 0, policy_version 1171262 (0.0007) [2023-12-27 13:45:54,445][20347] Updated weights for policy 0, policy_version 1171272 (0.0009) [2023-12-27 13:45:54,858][20381] Updated weights for policy 1, policy_version 1170306 (0.0007) [2023-12-27 13:45:54,908][20381] Updated weights for policy 1, policy_version 1170316 (0.0005) [2023-12-27 13:45:54,957][20381] Updated weights for policy 1, policy_version 1170326 (0.0007) [2023-12-27 13:45:55,081][20347] Updated weights for policy 0, policy_version 1171282 (0.0008) [2023-12-27 13:45:55,142][20347] Updated weights for policy 0, policy_version 1171292 (0.0005) [2023-12-27 13:45:55,199][20347] Updated weights for policy 0, policy_version 1171302 (0.0005) [2023-12-27 13:45:55,242][20347] Updated weights for policy 0, policy_version 1171312 (0.0005) [2023-12-27 13:45:55,542][20381] Updated weights for policy 1, policy_version 1170336 (0.0010) [2023-12-27 13:45:55,593][20381] Updated weights for policy 1, policy_version 1170346 (0.0010) [2023-12-27 13:45:55,649][20381] Updated weights for policy 1, policy_version 1170356 (0.0006) [2023-12-27 13:45:55,843][20347] Updated weights for policy 0, policy_version 1171322 (0.0006) [2023-12-27 13:45:55,898][20347] Updated weights for policy 0, policy_version 1171332 (0.0010) [2023-12-27 13:45:55,960][20347] Updated weights for policy 0, policy_version 1171342 (0.0010) [2023-12-27 13:45:56,190][20381] Updated weights for policy 1, policy_version 1170366 (0.0007) [2023-12-27 13:45:56,236][20381] Updated weights for policy 1, policy_version 1170376 (0.0006) [2023-12-27 13:45:56,290][20381] Updated weights for policy 1, policy_version 1170386 (0.0005) [2023-12-27 13:45:56,624][20347] Updated weights for policy 0, policy_version 1171352 (0.0010) [2023-12-27 13:45:56,671][20347] Updated weights for policy 0, policy_version 1171362 (0.0010) [2023-12-27 13:45:56,729][20347] Updated weights for policy 0, policy_version 1171372 (0.0006) [2023-12-27 13:45:56,890][20381] Updated weights for policy 1, policy_version 1170396 (0.0006) [2023-12-27 13:45:56,937][20381] Updated weights for policy 1, policy_version 1170406 (0.0008) [2023-12-27 13:45:56,982][20381] Updated weights for policy 1, policy_version 1170416 (0.0008) [2023-12-27 13:45:57,309][20347] Updated weights for policy 0, policy_version 1171382 (0.0005) [2023-12-27 13:45:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 599580672. Throughput: 0: 10058.8, 1: 10296.4. Samples: 599570660. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:45:57,363][20347] Updated weights for policy 0, policy_version 1171392 (0.0007) [2023-12-27 13:45:57,363][19170] Avg episode reward: [(0, '960.910'), (1, '676.610')] [2023-12-27 13:45:57,413][20347] Updated weights for policy 0, policy_version 1171402 (0.0007) [2023-12-27 13:45:57,652][20381] Updated weights for policy 1, policy_version 1170426 (0.0007) [2023-12-27 13:45:57,695][20381] Updated weights for policy 1, policy_version 1170436 (0.0005) [2023-12-27 13:45:57,748][20381] Updated weights for policy 1, policy_version 1170446 (0.0005) [2023-12-27 13:45:57,793][20381] Updated weights for policy 1, policy_version 1170456 (0.0005) [2023-12-27 13:45:58,016][20347] Updated weights for policy 0, policy_version 1171412 (0.0008) [2023-12-27 13:45:58,070][20347] Updated weights for policy 0, policy_version 1171422 (0.0010) [2023-12-27 13:45:58,122][20347] Updated weights for policy 0, policy_version 1171432 (0.0010) [2023-12-27 13:45:58,486][20381] Updated weights for policy 1, policy_version 1170466 (0.0008) [2023-12-27 13:45:58,552][20381] Updated weights for policy 1, policy_version 1170476 (0.0008) [2023-12-27 13:45:58,620][20381] Updated weights for policy 1, policy_version 1170486 (0.0009) [2023-12-27 13:45:58,930][20347] Updated weights for policy 0, policy_version 1171442 (0.0010) [2023-12-27 13:45:58,989][20347] Updated weights for policy 0, policy_version 1171452 (0.0010) [2023-12-27 13:45:59,049][20347] Updated weights for policy 0, policy_version 1171462 (0.0010) [2023-12-27 13:45:59,104][20347] Updated weights for policy 0, policy_version 1171472 (0.0010) [2023-12-27 13:45:59,415][20381] Updated weights for policy 1, policy_version 1170496 (0.0009) [2023-12-27 13:45:59,475][20381] Updated weights for policy 1, policy_version 1170506 (0.0008) [2023-12-27 13:45:59,522][20381] Updated weights for policy 1, policy_version 1170516 (0.0008) [2023-12-27 13:45:59,775][20347] Updated weights for policy 0, policy_version 1171482 (0.0007) [2023-12-27 13:45:59,845][20347] Updated weights for policy 0, policy_version 1171492 (0.0007) [2023-12-27 13:45:59,907][20347] Updated weights for policy 0, policy_version 1171502 (0.0007) [2023-12-27 13:46:00,280][20381] Updated weights for policy 1, policy_version 1170526 (0.0007) [2023-12-27 13:46:00,335][20381] Updated weights for policy 1, policy_version 1170536 (0.0006) [2023-12-27 13:46:00,396][20381] Updated weights for policy 1, policy_version 1170546 (0.0006) [2023-12-27 13:46:00,540][20347] Updated weights for policy 0, policy_version 1171512 (0.0007) [2023-12-27 13:46:00,587][20347] Updated weights for policy 0, policy_version 1171522 (0.0007) [2023-12-27 13:46:00,638][20347] Updated weights for policy 0, policy_version 1171532 (0.0008) [2023-12-27 13:46:01,041][20381] Updated weights for policy 1, policy_version 1170556 (0.0006) [2023-12-27 13:46:01,101][20381] Updated weights for policy 1, policy_version 1170566 (0.0006) [2023-12-27 13:46:01,166][20381] Updated weights for policy 1, policy_version 1170576 (0.0008) [2023-12-27 13:46:01,288][20347] Updated weights for policy 0, policy_version 1171542 (0.0008) [2023-12-27 13:46:01,351][20347] Updated weights for policy 0, policy_version 1171552 (0.0007) [2023-12-27 13:46:01,420][20347] Updated weights for policy 0, policy_version 1171562 (0.0008) [2023-12-27 13:46:01,896][20381] Updated weights for policy 1, policy_version 1170586 (0.0007) [2023-12-27 13:46:01,946][20381] Updated weights for policy 1, policy_version 1170596 (0.0006) [2023-12-27 13:46:02,008][20347] Updated weights for policy 0, policy_version 1171572 (0.0006) [2023-12-27 13:46:02,009][20381] Updated weights for policy 1, policy_version 1170606 (0.0009) [2023-12-27 13:46:02,061][20381] Updated weights for policy 1, policy_version 1170616 (0.0007) [2023-12-27 13:46:02,065][20347] Updated weights for policy 0, policy_version 1171582 (0.0005) [2023-12-27 13:46:02,121][20347] Updated weights for policy 0, policy_version 1171592 (0.0006) [2023-12-27 13:46:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 599687168. Throughput: 0: 10135.0, 1: 10214.0. Samples: 599694184. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:02,363][19170] Avg episode reward: [(0, '941.410'), (1, '794.290')] [2023-12-27 13:46:02,767][20381] Updated weights for policy 1, policy_version 1170626 (0.0011) [2023-12-27 13:46:02,791][20347] Updated weights for policy 0, policy_version 1171602 (0.0007) [2023-12-27 13:46:02,820][20381] Updated weights for policy 1, policy_version 1170636 (0.0011) [2023-12-27 13:46:02,851][20347] Updated weights for policy 0, policy_version 1171612 (0.0005) [2023-12-27 13:46:02,876][20381] Updated weights for policy 1, policy_version 1170646 (0.0011) [2023-12-27 13:46:02,908][20347] Updated weights for policy 0, policy_version 1171622 (0.0005) [2023-12-27 13:46:02,955][20347] Updated weights for policy 0, policy_version 1171632 (0.0005) [2023-12-27 13:46:03,473][20381] Updated weights for policy 1, policy_version 1170656 (0.0007) [2023-12-27 13:46:03,527][20381] Updated weights for policy 1, policy_version 1170666 (0.0008) [2023-12-27 13:46:03,534][20347] Updated weights for policy 0, policy_version 1171642 (0.0006) [2023-12-27 13:46:03,583][20381] Updated weights for policy 1, policy_version 1170676 (0.0009) [2023-12-27 13:46:03,587][20347] Updated weights for policy 0, policy_version 1171652 (0.0005) [2023-12-27 13:46:03,646][20347] Updated weights for policy 0, policy_version 1171662 (0.0006) [2023-12-27 13:46:04,257][20381] Updated weights for policy 1, policy_version 1170686 (0.0007) [2023-12-27 13:46:04,293][20347] Updated weights for policy 0, policy_version 1171672 (0.0010) [2023-12-27 13:46:04,316][20381] Updated weights for policy 1, policy_version 1170696 (0.0008) [2023-12-27 13:46:04,351][20347] Updated weights for policy 0, policy_version 1171682 (0.0010) [2023-12-27 13:46:04,371][20381] Updated weights for policy 1, policy_version 1170706 (0.0008) [2023-12-27 13:46:04,404][20347] Updated weights for policy 0, policy_version 1171692 (0.0010) [2023-12-27 13:46:05,052][20381] Updated weights for policy 1, policy_version 1170716 (0.0007) [2023-12-27 13:46:05,099][20347] Updated weights for policy 0, policy_version 1171702 (0.0011) [2023-12-27 13:46:05,110][20381] Updated weights for policy 1, policy_version 1170726 (0.0005) [2023-12-27 13:46:05,163][20347] Updated weights for policy 0, policy_version 1171712 (0.0010) [2023-12-27 13:46:05,170][20381] Updated weights for policy 1, policy_version 1170736 (0.0006) [2023-12-27 13:46:05,215][20347] Updated weights for policy 0, policy_version 1171722 (0.0009) [2023-12-27 13:46:05,764][20381] Updated weights for policy 1, policy_version 1170746 (0.0009) [2023-12-27 13:46:05,826][20381] Updated weights for policy 1, policy_version 1170756 (0.0008) [2023-12-27 13:46:05,833][20347] Updated weights for policy 0, policy_version 1171732 (0.0007) [2023-12-27 13:46:05,885][20381] Updated weights for policy 1, policy_version 1170766 (0.0006) [2023-12-27 13:46:05,885][20347] Updated weights for policy 0, policy_version 1171742 (0.0010) [2023-12-27 13:46:05,935][20347] Updated weights for policy 0, policy_version 1171752 (0.0011) [2023-12-27 13:46:05,942][20381] Updated weights for policy 1, policy_version 1170776 (0.0005) [2023-12-27 13:46:06,610][20381] Updated weights for policy 1, policy_version 1170786 (0.0011) [2023-12-27 13:46:06,654][20347] Updated weights for policy 0, policy_version 1171762 (0.0010) [2023-12-27 13:46:06,674][20381] Updated weights for policy 1, policy_version 1170796 (0.0011) [2023-12-27 13:46:06,713][20347] Updated weights for policy 0, policy_version 1171772 (0.0006) [2023-12-27 13:46:06,730][20381] Updated weights for policy 1, policy_version 1170806 (0.0011) [2023-12-27 13:46:06,771][20347] Updated weights for policy 0, policy_version 1171782 (0.0007) [2023-12-27 13:46:06,830][20347] Updated weights for policy 0, policy_version 1171792 (0.0006) [2023-12-27 13:46:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 599785472. Throughput: 0: 10162.0, 1: 10260.1. Samples: 599757452. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:07,363][19170] Avg episode reward: [(0, '960.910'), (1, '833.410')] [2023-12-27 13:46:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001171792_300032000.pth... [2023-12-27 13:46:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001170608_299728896.pth [2023-12-27 13:46:07,401][20381] Updated weights for policy 1, policy_version 1170816 (0.0010) [2023-12-27 13:46:07,460][20381] Updated weights for policy 1, policy_version 1170826 (0.0010) [2023-12-27 13:46:07,490][20347] Updated weights for policy 0, policy_version 1171802 (0.0005) [2023-12-27 13:46:07,512][20381] Updated weights for policy 1, policy_version 1170837 (0.0008) [2023-12-27 13:46:07,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001170840_299761664.pth... [2023-12-27 13:46:07,526][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001169624_299450368.pth [2023-12-27 13:46:07,539][20347] Updated weights for policy 0, policy_version 1171812 (0.0008) [2023-12-27 13:46:07,588][20347] Updated weights for policy 0, policy_version 1171822 (0.0008) [2023-12-27 13:46:08,096][20381] Updated weights for policy 1, policy_version 1170847 (0.0005) [2023-12-27 13:46:08,151][20381] Updated weights for policy 1, policy_version 1170857 (0.0007) [2023-12-27 13:46:08,207][20381] Updated weights for policy 1, policy_version 1170867 (0.0008) [2023-12-27 13:46:08,351][20347] Updated weights for policy 0, policy_version 1171832 (0.0007) [2023-12-27 13:46:08,406][20347] Updated weights for policy 0, policy_version 1171842 (0.0006) [2023-12-27 13:46:08,470][20347] Updated weights for policy 0, policy_version 1171852 (0.0006) [2023-12-27 13:46:08,809][20381] Updated weights for policy 1, policy_version 1170877 (0.0006) [2023-12-27 13:46:08,874][20381] Updated weights for policy 1, policy_version 1170887 (0.0007) [2023-12-27 13:46:08,931][20381] Updated weights for policy 1, policy_version 1170897 (0.0008) [2023-12-27 13:46:09,142][20291] KL-divergence is very high: 107.1130 [2023-12-27 13:46:09,147][20347] Updated weights for policy 0, policy_version 1171862 (0.0007) [2023-12-27 13:46:09,204][20347] Updated weights for policy 0, policy_version 1171872 (0.0007) [2023-12-27 13:46:09,268][20347] Updated weights for policy 0, policy_version 1171882 (0.0008) [2023-12-27 13:46:09,602][20381] Updated weights for policy 1, policy_version 1170907 (0.0007) [2023-12-27 13:46:09,663][20292] KL-divergence is very high: 207.1170 [2023-12-27 13:46:09,668][20381] Updated weights for policy 1, policy_version 1170917 (0.0006) [2023-12-27 13:46:09,734][20381] Updated weights for policy 1, policy_version 1170927 (0.0006) [2023-12-27 13:46:10,007][20347] Updated weights for policy 0, policy_version 1171892 (0.0008) [2023-12-27 13:46:10,066][20347] Updated weights for policy 0, policy_version 1171902 (0.0008) [2023-12-27 13:46:10,134][20347] Updated weights for policy 0, policy_version 1171912 (0.0008) [2023-12-27 13:46:10,327][20381] Updated weights for policy 1, policy_version 1170937 (0.0006) [2023-12-27 13:46:10,388][20381] Updated weights for policy 1, policy_version 1170947 (0.0008) [2023-12-27 13:46:10,449][20381] Updated weights for policy 1, policy_version 1170957 (0.0008) [2023-12-27 13:46:10,512][20381] Updated weights for policy 1, policy_version 1170967 (0.0008) [2023-12-27 13:46:10,871][20347] Updated weights for policy 0, policy_version 1171922 (0.0007) [2023-12-27 13:46:10,915][20347] Updated weights for policy 0, policy_version 1171932 (0.0008) [2023-12-27 13:46:10,968][20347] Updated weights for policy 0, policy_version 1171942 (0.0008) [2023-12-27 13:46:11,021][20347] Updated weights for policy 0, policy_version 1171952 (0.0008) [2023-12-27 13:46:11,176][20381] Updated weights for policy 1, policy_version 1170977 (0.0008) [2023-12-27 13:46:11,229][20381] Updated weights for policy 1, policy_version 1170987 (0.0008) [2023-12-27 13:46:11,293][20381] Updated weights for policy 1, policy_version 1170997 (0.0006) [2023-12-27 13:46:11,808][20347] Updated weights for policy 0, policy_version 1171962 (0.0008) [2023-12-27 13:46:11,865][20347] Updated weights for policy 0, policy_version 1171972 (0.0007) [2023-12-27 13:46:11,924][20347] Updated weights for policy 0, policy_version 1171982 (0.0008) [2023-12-27 13:46:11,962][20381] Updated weights for policy 1, policy_version 1171007 (0.0007) [2023-12-27 13:46:12,023][20381] Updated weights for policy 1, policy_version 1171017 (0.0009) [2023-12-27 13:46:12,077][20381] Updated weights for policy 1, policy_version 1171027 (0.0008) [2023-12-27 13:46:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 599891968. Throughput: 0: 10136.2, 1: 10298.8. Samples: 599880976. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:12,363][19170] Avg episode reward: [(0, '921.750'), (1, '725.970')] [2023-12-27 13:46:12,588][20347] Updated weights for policy 0, policy_version 1171992 (0.0008) [2023-12-27 13:46:12,648][20347] Updated weights for policy 0, policy_version 1172002 (0.0008) [2023-12-27 13:46:12,707][20347] Updated weights for policy 0, policy_version 1172012 (0.0008) [2023-12-27 13:46:12,816][20381] Updated weights for policy 1, policy_version 1171037 (0.0008) [2023-12-27 13:46:12,876][20381] Updated weights for policy 1, policy_version 1171047 (0.0008) [2023-12-27 13:46:12,931][20381] Updated weights for policy 1, policy_version 1171057 (0.0007) [2023-12-27 13:46:13,412][20347] Updated weights for policy 0, policy_version 1172022 (0.0007) [2023-12-27 13:46:13,475][20347] Updated weights for policy 0, policy_version 1172032 (0.0007) [2023-12-27 13:46:13,537][20347] Updated weights for policy 0, policy_version 1172042 (0.0008) [2023-12-27 13:46:13,578][20381] Updated weights for policy 1, policy_version 1171067 (0.0006) [2023-12-27 13:46:13,634][20381] Updated weights for policy 1, policy_version 1171077 (0.0007) [2023-12-27 13:46:13,694][20381] Updated weights for policy 1, policy_version 1171087 (0.0008) [2023-12-27 13:46:14,219][20347] Updated weights for policy 0, policy_version 1172052 (0.0006) [2023-12-27 13:46:14,282][20347] Updated weights for policy 0, policy_version 1172062 (0.0005) [2023-12-27 13:46:14,342][20347] Updated weights for policy 0, policy_version 1172072 (0.0008) [2023-12-27 13:46:14,398][20381] Updated weights for policy 1, policy_version 1171097 (0.0008) [2023-12-27 13:46:14,451][20381] Updated weights for policy 1, policy_version 1171107 (0.0008) [2023-12-27 13:46:14,507][20381] Updated weights for policy 1, policy_version 1171117 (0.0007) [2023-12-27 13:46:14,552][20381] Updated weights for policy 1, policy_version 1171127 (0.0007) [2023-12-27 13:46:14,929][20347] Updated weights for policy 0, policy_version 1172082 (0.0008) [2023-12-27 13:46:14,992][20347] Updated weights for policy 0, policy_version 1172092 (0.0006) [2023-12-27 13:46:15,060][20347] Updated weights for policy 0, policy_version 1172102 (0.0008) [2023-12-27 13:46:15,128][20347] Updated weights for policy 0, policy_version 1172112 (0.0010) [2023-12-27 13:46:15,278][20381] Updated weights for policy 1, policy_version 1171137 (0.0009) [2023-12-27 13:46:15,336][20381] Updated weights for policy 1, policy_version 1171147 (0.0010) [2023-12-27 13:46:15,395][20381] Updated weights for policy 1, policy_version 1171157 (0.0008) [2023-12-27 13:46:15,693][20347] Updated weights for policy 0, policy_version 1172122 (0.0010) [2023-12-27 13:46:15,745][20347] Updated weights for policy 0, policy_version 1172132 (0.0010) [2023-12-27 13:46:15,790][20347] Updated weights for policy 0, policy_version 1172142 (0.0010) [2023-12-27 13:46:16,058][20381] Updated weights for policy 1, policy_version 1171167 (0.0006) [2023-12-27 13:46:16,110][20381] Updated weights for policy 1, policy_version 1171177 (0.0005) [2023-12-27 13:46:16,160][20381] Updated weights for policy 1, policy_version 1171187 (0.0005) [2023-12-27 13:46:16,497][20347] Updated weights for policy 0, policy_version 1172152 (0.0010) [2023-12-27 13:46:16,542][20347] Updated weights for policy 0, policy_version 1172162 (0.0010) [2023-12-27 13:46:16,596][20347] Updated weights for policy 0, policy_version 1172172 (0.0009) [2023-12-27 13:46:16,771][20381] Updated weights for policy 1, policy_version 1171197 (0.0006) [2023-12-27 13:46:16,816][20381] Updated weights for policy 1, policy_version 1171207 (0.0007) [2023-12-27 13:46:16,862][20381] Updated weights for policy 1, policy_version 1171217 (0.0007) [2023-12-27 13:46:17,256][20347] Updated weights for policy 0, policy_version 1172182 (0.0010) [2023-12-27 13:46:17,305][20347] Updated weights for policy 0, policy_version 1172192 (0.0010) [2023-12-27 13:46:17,362][20347] Updated weights for policy 0, policy_version 1172202 (0.0010) [2023-12-27 13:46:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 599990272. Throughput: 0: 10132.7, 1: 10373.4. Samples: 600004304. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:17,363][19170] Avg episode reward: [(0, '960.900'), (1, '813.980')] [2023-12-27 13:46:17,579][20381] Updated weights for policy 1, policy_version 1171227 (0.0007) [2023-12-27 13:46:17,636][20381] Updated weights for policy 1, policy_version 1171237 (0.0007) [2023-12-27 13:46:17,690][20381] Updated weights for policy 1, policy_version 1171247 (0.0008) [2023-12-27 13:46:18,035][20347] Updated weights for policy 0, policy_version 1172212 (0.0010) [2023-12-27 13:46:18,095][20347] Updated weights for policy 0, policy_version 1172222 (0.0010) [2023-12-27 13:46:18,160][20347] Updated weights for policy 0, policy_version 1172232 (0.0010) [2023-12-27 13:46:18,312][20381] Updated weights for policy 1, policy_version 1171257 (0.0007) [2023-12-27 13:46:18,376][20381] Updated weights for policy 1, policy_version 1171267 (0.0008) [2023-12-27 13:46:18,432][20381] Updated weights for policy 1, policy_version 1171277 (0.0007) [2023-12-27 13:46:18,480][20381] Updated weights for policy 1, policy_version 1171287 (0.0008) [2023-12-27 13:46:18,834][20347] Updated weights for policy 0, policy_version 1172242 (0.0010) [2023-12-27 13:46:18,890][20347] Updated weights for policy 0, policy_version 1172252 (0.0008) [2023-12-27 13:46:18,945][20347] Updated weights for policy 0, policy_version 1172262 (0.0007) [2023-12-27 13:46:18,995][20347] Updated weights for policy 0, policy_version 1172272 (0.0007) [2023-12-27 13:46:19,157][20381] Updated weights for policy 1, policy_version 1171297 (0.0007) [2023-12-27 13:46:19,220][20381] Updated weights for policy 1, policy_version 1171307 (0.0007) [2023-12-27 13:46:19,281][20381] Updated weights for policy 1, policy_version 1171317 (0.0007) [2023-12-27 13:46:19,678][20347] Updated weights for policy 0, policy_version 1172282 (0.0007) [2023-12-27 13:46:19,743][20347] Updated weights for policy 0, policy_version 1172292 (0.0008) [2023-12-27 13:46:19,806][20347] Updated weights for policy 0, policy_version 1172302 (0.0009) [2023-12-27 13:46:20,000][20381] Updated weights for policy 1, policy_version 1171327 (0.0008) [2023-12-27 13:46:20,059][20381] Updated weights for policy 1, policy_version 1171337 (0.0008) [2023-12-27 13:46:20,119][20381] Updated weights for policy 1, policy_version 1171347 (0.0009) [2023-12-27 13:46:20,456][20347] Updated weights for policy 0, policy_version 1172312 (0.0008) [2023-12-27 13:46:20,458][20291] KL-divergence is very high: 367.9556 [2023-12-27 13:46:20,474][20291] KL-divergence is very high: 523.5922 [2023-12-27 13:46:20,500][20291] KL-divergence is very high: 323.7582 [2023-12-27 13:46:20,511][20347] Updated weights for policy 0, policy_version 1172322 (0.0008) [2023-12-27 13:46:20,515][20291] KL-divergence is very high: 226.6229 [2023-12-27 13:46:20,541][20291] KL-divergence is very high: 348.1581 [2023-12-27 13:46:20,564][20347] Updated weights for policy 0, policy_version 1172332 (0.0007) [2023-12-27 13:46:20,823][20381] Updated weights for policy 1, policy_version 1171357 (0.0008) [2023-12-27 13:46:20,880][20381] Updated weights for policy 1, policy_version 1171367 (0.0007) [2023-12-27 13:46:20,943][20381] Updated weights for policy 1, policy_version 1171377 (0.0008) [2023-12-27 13:46:21,274][20347] Updated weights for policy 0, policy_version 1172342 (0.0007) [2023-12-27 13:46:21,333][20347] Updated weights for policy 0, policy_version 1172352 (0.0008) [2023-12-27 13:46:21,402][20347] Updated weights for policy 0, policy_version 1172362 (0.0009) [2023-12-27 13:46:21,652][20381] Updated weights for policy 1, policy_version 1171387 (0.0008) [2023-12-27 13:46:21,715][20381] Updated weights for policy 1, policy_version 1171397 (0.0008) [2023-12-27 13:46:21,772][20381] Updated weights for policy 1, policy_version 1171407 (0.0007) [2023-12-27 13:46:22,133][20347] Updated weights for policy 0, policy_version 1172372 (0.0008) [2023-12-27 13:46:22,182][20347] Updated weights for policy 0, policy_version 1172382 (0.0008) [2023-12-27 13:46:22,242][20347] Updated weights for policy 0, policy_version 1172392 (0.0007) [2023-12-27 13:46:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 600096768. Throughput: 0: 10136.0, 1: 10351.7. Samples: 600065244. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:22,363][19170] Avg episode reward: [(0, '902.520'), (1, '833.620')] [2023-12-27 13:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001171416_299909120.pth... [2023-12-27 13:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001172400_300187648.pth... [2023-12-27 13:46:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001171184_299876352.pth [2023-12-27 13:46:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001170200_299597824.pth [2023-12-27 13:46:22,512][20381] Updated weights for policy 1, policy_version 1171417 (0.0011) [2023-12-27 13:46:22,559][20381] Updated weights for policy 1, policy_version 1171427 (0.0007) [2023-12-27 13:46:22,620][20381] Updated weights for policy 1, policy_version 1171437 (0.0008) [2023-12-27 13:46:22,674][20381] Updated weights for policy 1, policy_version 1171447 (0.0008) [2023-12-27 13:46:22,992][20347] Updated weights for policy 0, policy_version 1172402 (0.0007) [2023-12-27 13:46:23,044][20347] Updated weights for policy 0, policy_version 1172412 (0.0007) [2023-12-27 13:46:23,102][20347] Updated weights for policy 0, policy_version 1172422 (0.0005) [2023-12-27 13:46:23,164][20347] Updated weights for policy 0, policy_version 1172432 (0.0006) [2023-12-27 13:46:23,327][20381] Updated weights for policy 1, policy_version 1171457 (0.0010) [2023-12-27 13:46:23,372][20381] Updated weights for policy 1, policy_version 1171467 (0.0010) [2023-12-27 13:46:23,425][20381] Updated weights for policy 1, policy_version 1171477 (0.0011) [2023-12-27 13:46:23,736][20347] Updated weights for policy 0, policy_version 1172442 (0.0007) [2023-12-27 13:46:23,792][20347] Updated weights for policy 0, policy_version 1172452 (0.0007) [2023-12-27 13:46:23,847][20347] Updated weights for policy 0, policy_version 1172462 (0.0007) [2023-12-27 13:46:24,073][20381] Updated weights for policy 1, policy_version 1171487 (0.0011) [2023-12-27 13:46:24,124][20381] Updated weights for policy 1, policy_version 1171497 (0.0010) [2023-12-27 13:46:24,178][20381] Updated weights for policy 1, policy_version 1171507 (0.0010) [2023-12-27 13:46:24,564][20347] Updated weights for policy 0, policy_version 1172472 (0.0007) [2023-12-27 13:46:24,623][20347] Updated weights for policy 0, policy_version 1172482 (0.0006) [2023-12-27 13:46:24,683][20347] Updated weights for policy 0, policy_version 1172492 (0.0007) [2023-12-27 13:46:24,901][20381] Updated weights for policy 1, policy_version 1171517 (0.0009) [2023-12-27 13:46:24,960][20381] Updated weights for policy 1, policy_version 1171527 (0.0006) [2023-12-27 13:46:25,014][20381] Updated weights for policy 1, policy_version 1171537 (0.0008) [2023-12-27 13:46:25,355][20347] Updated weights for policy 0, policy_version 1172502 (0.0007) [2023-12-27 13:46:25,409][20347] Updated weights for policy 0, policy_version 1172512 (0.0007) [2023-12-27 13:46:25,465][20347] Updated weights for policy 0, policy_version 1172522 (0.0007) [2023-12-27 13:46:25,634][20381] Updated weights for policy 1, policy_version 1171547 (0.0008) [2023-12-27 13:46:25,692][20381] Updated weights for policy 1, policy_version 1171557 (0.0006) [2023-12-27 13:46:25,751][20381] Updated weights for policy 1, policy_version 1171567 (0.0007) [2023-12-27 13:46:26,205][20347] Updated weights for policy 0, policy_version 1172532 (0.0008) [2023-12-27 13:46:26,261][20347] Updated weights for policy 0, policy_version 1172542 (0.0008) [2023-12-27 13:46:26,315][20347] Updated weights for policy 0, policy_version 1172552 (0.0009) [2023-12-27 13:46:26,332][20381] Updated weights for policy 1, policy_version 1171577 (0.0006) [2023-12-27 13:46:26,381][20381] Updated weights for policy 1, policy_version 1171587 (0.0008) [2023-12-27 13:46:26,429][20381] Updated weights for policy 1, policy_version 1171597 (0.0008) [2023-12-27 13:46:26,486][20381] Updated weights for policy 1, policy_version 1171607 (0.0007) [2023-12-27 13:46:26,996][20347] Updated weights for policy 0, policy_version 1172562 (0.0007) [2023-12-27 13:46:27,048][20347] Updated weights for policy 0, policy_version 1172572 (0.0005) [2023-12-27 13:46:27,098][20347] Updated weights for policy 0, policy_version 1172582 (0.0005) [2023-12-27 13:46:27,141][20381] Updated weights for policy 1, policy_version 1171617 (0.0007) [2023-12-27 13:46:27,150][20347] Updated weights for policy 0, policy_version 1172592 (0.0005) [2023-12-27 13:46:27,199][20381] Updated weights for policy 1, policy_version 1171627 (0.0005) [2023-12-27 13:46:27,259][20381] Updated weights for policy 1, policy_version 1171637 (0.0005) [2023-12-27 13:46:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 600203264. Throughput: 0: 10159.5, 1: 10363.1. Samples: 600187036. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:27,363][19170] Avg episode reward: [(0, '902.510'), (1, '843.360')] [2023-12-27 13:46:27,696][20347] Updated weights for policy 0, policy_version 1172602 (0.0008) [2023-12-27 13:46:27,751][20347] Updated weights for policy 0, policy_version 1172612 (0.0007) [2023-12-27 13:46:27,807][20347] Updated weights for policy 0, policy_version 1172622 (0.0007) [2023-12-27 13:46:27,876][20381] Updated weights for policy 1, policy_version 1171647 (0.0005) [2023-12-27 13:46:27,929][20381] Updated weights for policy 1, policy_version 1171657 (0.0007) [2023-12-27 13:46:27,976][20381] Updated weights for policy 1, policy_version 1171667 (0.0005) [2023-12-27 13:46:28,541][20347] Updated weights for policy 0, policy_version 1172632 (0.0007) [2023-12-27 13:46:28,578][20381] Updated weights for policy 1, policy_version 1171677 (0.0006) [2023-12-27 13:46:28,599][20347] Updated weights for policy 0, policy_version 1172642 (0.0006) [2023-12-27 13:46:28,637][20381] Updated weights for policy 1, policy_version 1171687 (0.0007) [2023-12-27 13:46:28,650][20347] Updated weights for policy 0, policy_version 1172652 (0.0006) [2023-12-27 13:46:28,693][20381] Updated weights for policy 1, policy_version 1171697 (0.0006) [2023-12-27 13:46:29,313][20381] Updated weights for policy 1, policy_version 1171707 (0.0008) [2023-12-27 13:46:29,365][20347] Updated weights for policy 0, policy_version 1172662 (0.0007) [2023-12-27 13:46:29,382][20381] Updated weights for policy 1, policy_version 1171717 (0.0007) [2023-12-27 13:46:29,426][20347] Updated weights for policy 0, policy_version 1172672 (0.0006) [2023-12-27 13:46:29,440][20381] Updated weights for policy 1, policy_version 1171727 (0.0008) [2023-12-27 13:46:29,479][20347] Updated weights for policy 0, policy_version 1172682 (0.0005) [2023-12-27 13:46:30,160][20381] Updated weights for policy 1, policy_version 1171737 (0.0007) [2023-12-27 13:46:30,162][20347] Updated weights for policy 0, policy_version 1172692 (0.0006) [2023-12-27 13:46:30,210][20381] Updated weights for policy 1, policy_version 1171747 (0.0006) [2023-12-27 13:46:30,211][20347] Updated weights for policy 0, policy_version 1172702 (0.0005) [2023-12-27 13:46:30,263][20381] Updated weights for policy 1, policy_version 1171757 (0.0006) [2023-12-27 13:46:30,264][20347] Updated weights for policy 0, policy_version 1172712 (0.0005) [2023-12-27 13:46:30,319][20381] Updated weights for policy 1, policy_version 1171767 (0.0006) [2023-12-27 13:46:30,952][20381] Updated weights for policy 1, policy_version 1171777 (0.0006) [2023-12-27 13:46:30,977][20347] Updated weights for policy 0, policy_version 1172722 (0.0006) [2023-12-27 13:46:31,009][20381] Updated weights for policy 1, policy_version 1171787 (0.0010) [2023-12-27 13:46:31,030][20347] Updated weights for policy 0, policy_version 1172732 (0.0010) [2023-12-27 13:46:31,068][20381] Updated weights for policy 1, policy_version 1171797 (0.0009) [2023-12-27 13:46:31,085][20347] Updated weights for policy 0, policy_version 1172742 (0.0010) [2023-12-27 13:46:31,131][20347] Updated weights for policy 0, policy_version 1172752 (0.0010) [2023-12-27 13:46:31,762][20381] Updated weights for policy 1, policy_version 1171807 (0.0009) [2023-12-27 13:46:31,811][20381] Updated weights for policy 1, policy_version 1171817 (0.0010) [2023-12-27 13:46:31,873][20381] Updated weights for policy 1, policy_version 1171827 (0.0011) [2023-12-27 13:46:31,900][20347] Updated weights for policy 0, policy_version 1172762 (0.0006) [2023-12-27 13:46:31,951][20347] Updated weights for policy 0, policy_version 1172772 (0.0007) [2023-12-27 13:46:32,005][20347] Updated weights for policy 0, policy_version 1172782 (0.0007) [2023-12-27 13:46:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 600301568. Throughput: 0: 10220.6, 1: 10372.9. Samples: 600311416. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:32,363][19170] Avg episode reward: [(0, '912.360'), (1, '774.900')] [2023-12-27 13:46:32,599][20381] Updated weights for policy 1, policy_version 1171837 (0.0010) [2023-12-27 13:46:32,658][20381] Updated weights for policy 1, policy_version 1171847 (0.0008) [2023-12-27 13:46:32,708][20381] Updated weights for policy 1, policy_version 1171857 (0.0006) [2023-12-27 13:46:32,710][20347] Updated weights for policy 0, policy_version 1172792 (0.0007) [2023-12-27 13:46:32,766][20347] Updated weights for policy 0, policy_version 1172802 (0.0006) [2023-12-27 13:46:32,827][20347] Updated weights for policy 0, policy_version 1172812 (0.0005) [2023-12-27 13:46:33,247][20381] Updated weights for policy 1, policy_version 1171867 (0.0005) [2023-12-27 13:46:33,295][20381] Updated weights for policy 1, policy_version 1171877 (0.0006) [2023-12-27 13:46:33,341][20381] Updated weights for policy 1, policy_version 1171887 (0.0008) [2023-12-27 13:46:33,368][20347] Updated weights for policy 0, policy_version 1172822 (0.0009) [2023-12-27 13:46:33,426][20347] Updated weights for policy 0, policy_version 1172832 (0.0010) [2023-12-27 13:46:33,474][20347] Updated weights for policy 0, policy_version 1172842 (0.0008) [2023-12-27 13:46:33,953][20381] Updated weights for policy 1, policy_version 1171897 (0.0010) [2023-12-27 13:46:34,008][20381] Updated weights for policy 1, policy_version 1171907 (0.0005) [2023-12-27 13:46:34,066][20381] Updated weights for policy 1, policy_version 1171917 (0.0005) [2023-12-27 13:46:34,072][20347] Updated weights for policy 0, policy_version 1172852 (0.0006) [2023-12-27 13:46:34,127][20381] Updated weights for policy 1, policy_version 1171927 (0.0005) [2023-12-27 13:46:34,127][20347] Updated weights for policy 0, policy_version 1172862 (0.0005) [2023-12-27 13:46:34,193][20347] Updated weights for policy 0, policy_version 1172872 (0.0008) [2023-12-27 13:46:34,788][20381] Updated weights for policy 1, policy_version 1171937 (0.0007) [2023-12-27 13:46:34,839][20381] Updated weights for policy 1, policy_version 1171947 (0.0007) [2023-12-27 13:46:34,873][20347] Updated weights for policy 0, policy_version 1172882 (0.0011) [2023-12-27 13:46:34,890][20381] Updated weights for policy 1, policy_version 1171957 (0.0007) [2023-12-27 13:46:34,928][20347] Updated weights for policy 0, policy_version 1172892 (0.0010) [2023-12-27 13:46:34,982][20347] Updated weights for policy 0, policy_version 1172902 (0.0010) [2023-12-27 13:46:35,032][20347] Updated weights for policy 0, policy_version 1172912 (0.0010) [2023-12-27 13:46:35,482][20381] Updated weights for policy 1, policy_version 1171967 (0.0006) [2023-12-27 13:46:35,540][20381] Updated weights for policy 1, policy_version 1171977 (0.0006) [2023-12-27 13:46:35,591][20381] Updated weights for policy 1, policy_version 1171987 (0.0005) [2023-12-27 13:46:35,718][20347] Updated weights for policy 0, policy_version 1172922 (0.0005) [2023-12-27 13:46:35,779][20347] Updated weights for policy 0, policy_version 1172932 (0.0006) [2023-12-27 13:46:35,838][20347] Updated weights for policy 0, policy_version 1172942 (0.0008) [2023-12-27 13:46:36,256][20381] Updated weights for policy 1, policy_version 1171997 (0.0007) [2023-12-27 13:46:36,310][20381] Updated weights for policy 1, policy_version 1172007 (0.0009) [2023-12-27 13:46:36,370][20381] Updated weights for policy 1, policy_version 1172017 (0.0010) [2023-12-27 13:46:36,490][20347] Updated weights for policy 0, policy_version 1172952 (0.0006) [2023-12-27 13:46:36,562][20347] Updated weights for policy 0, policy_version 1172962 (0.0006) [2023-12-27 13:46:36,626][20347] Updated weights for policy 0, policy_version 1172972 (0.0011) [2023-12-27 13:46:37,117][20381] Updated weights for policy 1, policy_version 1172027 (0.0009) [2023-12-27 13:46:37,178][20381] Updated weights for policy 1, policy_version 1172037 (0.0006) [2023-12-27 13:46:37,216][20347] Updated weights for policy 0, policy_version 1172982 (0.0007) [2023-12-27 13:46:37,227][20381] Updated weights for policy 1, policy_version 1172047 (0.0008) [2023-12-27 13:46:37,274][20347] Updated weights for policy 0, policy_version 1172992 (0.0010) [2023-12-27 13:46:37,329][20347] Updated weights for policy 0, policy_version 1173002 (0.0010) [2023-12-27 13:46:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 600408064. Throughput: 0: 10270.6, 1: 10430.7. Samples: 600377632. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:37,363][19170] Avg episode reward: [(0, '931.830'), (1, '725.850')] [2023-12-27 13:46:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001172056_300072960.pth... [2023-12-27 13:46:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001173008_300343296.pth... [2023-12-27 13:46:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001171792_300032000.pth [2023-12-27 13:46:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001170840_299761664.pth [2023-12-27 13:46:37,854][20381] Updated weights for policy 1, policy_version 1172057 (0.0005) [2023-12-27 13:46:37,909][20381] Updated weights for policy 1, policy_version 1172067 (0.0006) [2023-12-27 13:46:37,967][20381] Updated weights for policy 1, policy_version 1172077 (0.0005) [2023-12-27 13:46:38,001][20347] Updated weights for policy 0, policy_version 1173012 (0.0010) [2023-12-27 13:46:38,024][20381] Updated weights for policy 1, policy_version 1172087 (0.0005) [2023-12-27 13:46:38,058][20347] Updated weights for policy 0, policy_version 1173022 (0.0010) [2023-12-27 13:46:38,112][20347] Updated weights for policy 0, policy_version 1173032 (0.0010) [2023-12-27 13:46:38,700][20381] Updated weights for policy 1, policy_version 1172097 (0.0006) [2023-12-27 13:46:38,750][20381] Updated weights for policy 1, policy_version 1172107 (0.0007) [2023-12-27 13:46:38,801][20381] Updated weights for policy 1, policy_version 1172117 (0.0008) [2023-12-27 13:46:38,803][20347] Updated weights for policy 0, policy_version 1173042 (0.0009) [2023-12-27 13:46:38,858][20347] Updated weights for policy 0, policy_version 1173052 (0.0005) [2023-12-27 13:46:38,916][20347] Updated weights for policy 0, policy_version 1173062 (0.0006) [2023-12-27 13:46:38,972][20347] Updated weights for policy 0, policy_version 1173072 (0.0005) [2023-12-27 13:46:39,560][20381] Updated weights for policy 1, policy_version 1172127 (0.0006) [2023-12-27 13:46:39,573][20347] Updated weights for policy 0, policy_version 1173082 (0.0010) [2023-12-27 13:46:39,619][20381] Updated weights for policy 1, policy_version 1172137 (0.0008) [2023-12-27 13:46:39,632][20347] Updated weights for policy 0, policy_version 1173092 (0.0011) [2023-12-27 13:46:39,678][20381] Updated weights for policy 1, policy_version 1172147 (0.0007) [2023-12-27 13:46:39,698][20347] Updated weights for policy 0, policy_version 1173102 (0.0010) [2023-12-27 13:46:40,400][20347] Updated weights for policy 0, policy_version 1173112 (0.0011) [2023-12-27 13:46:40,450][20381] Updated weights for policy 1, policy_version 1172157 (0.0007) [2023-12-27 13:46:40,456][20347] Updated weights for policy 0, policy_version 1173122 (0.0010) [2023-12-27 13:46:40,510][20381] Updated weights for policy 1, policy_version 1172167 (0.0006) [2023-12-27 13:46:40,512][20347] Updated weights for policy 0, policy_version 1173132 (0.0010) [2023-12-27 13:46:40,567][20381] Updated weights for policy 1, policy_version 1172177 (0.0005) [2023-12-27 13:46:41,210][20381] Updated weights for policy 1, policy_version 1172187 (0.0006) [2023-12-27 13:46:41,229][20347] Updated weights for policy 0, policy_version 1173142 (0.0008) [2023-12-27 13:46:41,273][20381] Updated weights for policy 1, policy_version 1172197 (0.0007) [2023-12-27 13:46:41,289][20347] Updated weights for policy 0, policy_version 1173152 (0.0008) [2023-12-27 13:46:41,327][20381] Updated weights for policy 1, policy_version 1172207 (0.0006) [2023-12-27 13:46:41,338][20347] Updated weights for policy 0, policy_version 1173162 (0.0007) [2023-12-27 13:46:41,966][20381] Updated weights for policy 1, policy_version 1172217 (0.0008) [2023-12-27 13:46:42,031][20381] Updated weights for policy 1, policy_version 1172227 (0.0009) [2023-12-27 13:46:42,068][20347] Updated weights for policy 0, policy_version 1173172 (0.0007) [2023-12-27 13:46:42,092][20381] Updated weights for policy 1, policy_version 1172237 (0.0008) [2023-12-27 13:46:42,124][20347] Updated weights for policy 0, policy_version 1173182 (0.0008) [2023-12-27 13:46:42,148][20381] Updated weights for policy 1, policy_version 1172247 (0.0007) [2023-12-27 13:46:42,182][20347] Updated weights for policy 0, policy_version 1173192 (0.0006) [2023-12-27 13:46:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 600514560. Throughput: 0: 10295.8, 1: 10385.9. Samples: 600501336. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:42,363][19170] Avg episode reward: [(0, '903.000'), (1, '794.380')] [2023-12-27 13:46:42,786][20381] Updated weights for policy 1, policy_version 1172257 (0.0008) [2023-12-27 13:46:42,839][20381] Updated weights for policy 1, policy_version 1172267 (0.0008) [2023-12-27 13:46:42,887][20381] Updated weights for policy 1, policy_version 1172277 (0.0008) [2023-12-27 13:46:42,900][20347] Updated weights for policy 0, policy_version 1173202 (0.0008) [2023-12-27 13:46:42,948][20347] Updated weights for policy 0, policy_version 1173212 (0.0007) [2023-12-27 13:46:42,993][20347] Updated weights for policy 0, policy_version 1173222 (0.0007) [2023-12-27 13:46:43,041][20347] Updated weights for policy 0, policy_version 1173232 (0.0008) [2023-12-27 13:46:43,613][20381] Updated weights for policy 1, policy_version 1172287 (0.0006) [2023-12-27 13:46:43,680][20381] Updated weights for policy 1, policy_version 1172297 (0.0008) [2023-12-27 13:46:43,705][20347] Updated weights for policy 0, policy_version 1173242 (0.0007) [2023-12-27 13:46:43,738][20381] Updated weights for policy 1, policy_version 1172307 (0.0007) [2023-12-27 13:46:43,767][20347] Updated weights for policy 0, policy_version 1173252 (0.0006) [2023-12-27 13:46:43,821][20347] Updated weights for policy 0, policy_version 1173262 (0.0007) [2023-12-27 13:46:44,408][20381] Updated weights for policy 1, policy_version 1172317 (0.0007) [2023-12-27 13:46:44,462][20381] Updated weights for policy 1, policy_version 1172327 (0.0007) [2023-12-27 13:46:44,504][20347] Updated weights for policy 0, policy_version 1173272 (0.0007) [2023-12-27 13:46:44,513][20381] Updated weights for policy 1, policy_version 1172337 (0.0007) [2023-12-27 13:46:44,556][20347] Updated weights for policy 0, policy_version 1173282 (0.0006) [2023-12-27 13:46:44,609][20347] Updated weights for policy 0, policy_version 1173292 (0.0007) [2023-12-27 13:46:45,201][20381] Updated weights for policy 1, policy_version 1172347 (0.0005) [2023-12-27 13:46:45,257][20381] Updated weights for policy 1, policy_version 1172357 (0.0006) [2023-12-27 13:46:45,305][20381] Updated weights for policy 1, policy_version 1172367 (0.0010) [2023-12-27 13:46:45,326][20347] Updated weights for policy 0, policy_version 1173302 (0.0008) [2023-12-27 13:46:45,388][20347] Updated weights for policy 0, policy_version 1173312 (0.0011) [2023-12-27 13:46:45,449][20347] Updated weights for policy 0, policy_version 1173322 (0.0008) [2023-12-27 13:46:45,916][20381] Updated weights for policy 1, policy_version 1172377 (0.0010) [2023-12-27 13:46:45,968][20381] Updated weights for policy 1, policy_version 1172387 (0.0005) [2023-12-27 13:46:46,018][20381] Updated weights for policy 1, policy_version 1172397 (0.0005) [2023-12-27 13:46:46,038][20347] Updated weights for policy 0, policy_version 1173332 (0.0007) [2023-12-27 13:46:46,078][20381] Updated weights for policy 1, policy_version 1172407 (0.0005) [2023-12-27 13:46:46,097][20347] Updated weights for policy 0, policy_version 1173342 (0.0011) [2023-12-27 13:46:46,159][20347] Updated weights for policy 0, policy_version 1173352 (0.0010) [2023-12-27 13:46:46,615][20381] Updated weights for policy 1, policy_version 1172417 (0.0008) [2023-12-27 13:46:46,665][20381] Updated weights for policy 1, policy_version 1172427 (0.0010) [2023-12-27 13:46:46,714][20381] Updated weights for policy 1, policy_version 1172437 (0.0007) [2023-12-27 13:46:46,765][20347] Updated weights for policy 0, policy_version 1173362 (0.0009) [2023-12-27 13:46:46,824][20347] Updated weights for policy 0, policy_version 1173372 (0.0006) [2023-12-27 13:46:46,875][20347] Updated weights for policy 0, policy_version 1173382 (0.0005) [2023-12-27 13:46:46,922][20347] Updated weights for policy 0, policy_version 1173392 (0.0005) [2023-12-27 13:46:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 600612864. Throughput: 0: 10267.8, 1: 10469.9. Samples: 600627380. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:47,363][19170] Avg episode reward: [(0, '912.310'), (1, '814.080')] [2023-12-27 13:46:47,368][20381] Updated weights for policy 1, policy_version 1172447 (0.0009) [2023-12-27 13:46:47,419][20381] Updated weights for policy 1, policy_version 1172457 (0.0010) [2023-12-27 13:46:47,463][20347] Updated weights for policy 0, policy_version 1173402 (0.0005) [2023-12-27 13:46:47,478][20381] Updated weights for policy 1, policy_version 1172467 (0.0011) [2023-12-27 13:46:47,512][20347] Updated weights for policy 0, policy_version 1173412 (0.0005) [2023-12-27 13:46:47,566][20347] Updated weights for policy 0, policy_version 1173422 (0.0005) [2023-12-27 13:46:48,116][20347] Updated weights for policy 0, policy_version 1173432 (0.0008) [2023-12-27 13:46:48,142][20381] Updated weights for policy 1, policy_version 1172477 (0.0011) [2023-12-27 13:46:48,171][20347] Updated weights for policy 0, policy_version 1173442 (0.0010) [2023-12-27 13:46:48,197][20381] Updated weights for policy 1, policy_version 1172487 (0.0010) [2023-12-27 13:46:48,223][20347] Updated weights for policy 0, policy_version 1173452 (0.0007) [2023-12-27 13:46:48,255][20381] Updated weights for policy 1, policy_version 1172497 (0.0010) [2023-12-27 13:46:48,825][20347] Updated weights for policy 0, policy_version 1173462 (0.0008) [2023-12-27 13:46:48,879][20347] Updated weights for policy 0, policy_version 1173472 (0.0008) [2023-12-27 13:46:48,933][20347] Updated weights for policy 0, policy_version 1173482 (0.0005) [2023-12-27 13:46:48,955][20381] Updated weights for policy 1, policy_version 1172507 (0.0010) [2023-12-27 13:46:49,004][20381] Updated weights for policy 1, policy_version 1172517 (0.0010) [2023-12-27 13:46:49,052][20381] Updated weights for policy 1, policy_version 1172527 (0.0010) [2023-12-27 13:46:49,566][20347] Updated weights for policy 0, policy_version 1173492 (0.0007) [2023-12-27 13:46:49,617][20347] Updated weights for policy 0, policy_version 1173502 (0.0007) [2023-12-27 13:46:49,672][20347] Updated weights for policy 0, policy_version 1173512 (0.0007) [2023-12-27 13:46:49,742][20381] Updated weights for policy 1, policy_version 1172537 (0.0010) [2023-12-27 13:46:49,795][20381] Updated weights for policy 1, policy_version 1172547 (0.0007) [2023-12-27 13:46:49,860][20381] Updated weights for policy 1, policy_version 1172557 (0.0007) [2023-12-27 13:46:49,918][20381] Updated weights for policy 1, policy_version 1172567 (0.0006) [2023-12-27 13:46:50,359][20347] Updated weights for policy 0, policy_version 1173522 (0.0007) [2023-12-27 13:46:50,420][20347] Updated weights for policy 0, policy_version 1173532 (0.0008) [2023-12-27 13:46:50,474][20347] Updated weights for policy 0, policy_version 1173542 (0.0006) [2023-12-27 13:46:50,530][20347] Updated weights for policy 0, policy_version 1173552 (0.0005) [2023-12-27 13:46:50,640][20381] Updated weights for policy 1, policy_version 1172580 (0.0009) [2023-12-27 13:46:50,705][20381] Updated weights for policy 1, policy_version 1172590 (0.0008) [2023-12-27 13:46:50,771][20381] Updated weights for policy 1, policy_version 1172600 (0.0008) [2023-12-27 13:46:51,178][20347] Updated weights for policy 0, policy_version 1173562 (0.0008) [2023-12-27 13:46:51,224][20347] Updated weights for policy 0, policy_version 1173572 (0.0008) [2023-12-27 13:46:51,282][20347] Updated weights for policy 0, policy_version 1173582 (0.0008) [2023-12-27 13:46:51,461][20381] Updated weights for policy 1, policy_version 1172610 (0.0007) [2023-12-27 13:46:51,518][20381] Updated weights for policy 1, policy_version 1172620 (0.0008) [2023-12-27 13:46:51,574][20381] Updated weights for policy 1, policy_version 1172630 (0.0008) [2023-12-27 13:46:52,037][20347] Updated weights for policy 0, policy_version 1173592 (0.0007) [2023-12-27 13:46:52,096][20347] Updated weights for policy 0, policy_version 1173602 (0.0008) [2023-12-27 13:46:52,153][20347] Updated weights for policy 0, policy_version 1173612 (0.0008) [2023-12-27 13:46:52,287][20381] Updated weights for policy 1, policy_version 1172640 (0.0009) [2023-12-27 13:46:52,344][20381] Updated weights for policy 1, policy_version 1172650 (0.0008) [2023-12-27 13:46:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 600719360. Throughput: 0: 10322.3, 1: 10436.5. Samples: 600691600. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:52,363][19170] Avg episode reward: [(0, '922.110'), (1, '755.160')] [2023-12-27 13:46:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001173616_300498944.pth... [2023-12-27 13:46:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001172400_300187648.pth [2023-12-27 13:46:52,403][20381] Updated weights for policy 1, policy_version 1172660 (0.0008) [2023-12-27 13:46:52,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001172664_300228608.pth... [2023-12-27 13:46:52,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001171416_299909120.pth [2023-12-27 13:46:52,923][20347] Updated weights for policy 0, policy_version 1173622 (0.0007) [2023-12-27 13:46:52,987][20347] Updated weights for policy 0, policy_version 1173632 (0.0007) [2023-12-27 13:46:53,051][20347] Updated weights for policy 0, policy_version 1173642 (0.0008) [2023-12-27 13:46:53,118][20381] Updated weights for policy 1, policy_version 1172670 (0.0011) [2023-12-27 13:46:53,174][20381] Updated weights for policy 1, policy_version 1172680 (0.0011) [2023-12-27 13:46:53,228][20381] Updated weights for policy 1, policy_version 1172690 (0.0010) [2023-12-27 13:46:53,741][20347] Updated weights for policy 0, policy_version 1173652 (0.0007) [2023-12-27 13:46:53,791][20347] Updated weights for policy 0, policy_version 1173662 (0.0007) [2023-12-27 13:46:53,844][20347] Updated weights for policy 0, policy_version 1173672 (0.0007) [2023-12-27 13:46:53,897][20381] Updated weights for policy 1, policy_version 1172700 (0.0011) [2023-12-27 13:46:53,959][20381] Updated weights for policy 1, policy_version 1172710 (0.0011) [2023-12-27 13:46:54,018][20381] Updated weights for policy 1, policy_version 1172720 (0.0011) [2023-12-27 13:46:54,573][20347] Updated weights for policy 0, policy_version 1173682 (0.0007) [2023-12-27 13:46:54,620][20347] Updated weights for policy 0, policy_version 1173692 (0.0007) [2023-12-27 13:46:54,642][20381] Updated weights for policy 1, policy_version 1172730 (0.0008) [2023-12-27 13:46:54,666][20347] Updated weights for policy 0, policy_version 1173702 (0.0006) [2023-12-27 13:46:54,697][20381] Updated weights for policy 1, policy_version 1172740 (0.0010) [2023-12-27 13:46:54,722][20347] Updated weights for policy 0, policy_version 1173712 (0.0006) [2023-12-27 13:46:54,759][20381] Updated weights for policy 1, policy_version 1172750 (0.0011) [2023-12-27 13:46:54,827][20381] Updated weights for policy 1, policy_version 1172760 (0.0011) [2023-12-27 13:46:55,365][20347] Updated weights for policy 0, policy_version 1173722 (0.0010) [2023-12-27 13:46:55,425][20347] Updated weights for policy 0, policy_version 1173732 (0.0010) [2023-12-27 13:46:55,486][20347] Updated weights for policy 0, policy_version 1173742 (0.0010) [2023-12-27 13:46:55,496][20381] Updated weights for policy 1, policy_version 1172770 (0.0006) [2023-12-27 13:46:55,541][20381] Updated weights for policy 1, policy_version 1172780 (0.0010) [2023-12-27 13:46:55,586][20381] Updated weights for policy 1, policy_version 1172790 (0.0010) [2023-12-27 13:46:56,085][20347] Updated weights for policy 0, policy_version 1173752 (0.0010) [2023-12-27 13:46:56,137][20347] Updated weights for policy 0, policy_version 1173762 (0.0010) [2023-12-27 13:46:56,189][20347] Updated weights for policy 0, policy_version 1173772 (0.0010) [2023-12-27 13:46:56,273][20381] Updated weights for policy 1, policy_version 1172800 (0.0011) [2023-12-27 13:46:56,330][20381] Updated weights for policy 1, policy_version 1172810 (0.0008) [2023-12-27 13:46:56,377][20381] Updated weights for policy 1, policy_version 1172820 (0.0007) [2023-12-27 13:46:56,808][20347] Updated weights for policy 0, policy_version 1173782 (0.0007) [2023-12-27 13:46:56,866][20347] Updated weights for policy 0, policy_version 1173792 (0.0005) [2023-12-27 13:46:56,923][20347] Updated weights for policy 0, policy_version 1173802 (0.0007) [2023-12-27 13:46:57,115][20381] Updated weights for policy 1, policy_version 1172830 (0.0008) [2023-12-27 13:46:57,167][20381] Updated weights for policy 1, policy_version 1172840 (0.0008) [2023-12-27 13:46:57,225][20381] Updated weights for policy 1, policy_version 1172850 (0.0008) [2023-12-27 13:46:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 600825856. Throughput: 0: 10355.4, 1: 10362.2. Samples: 600813268. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:46:57,363][19170] Avg episode reward: [(0, '922.030'), (1, '676.890')] [2023-12-27 13:46:57,556][20347] Updated weights for policy 0, policy_version 1173812 (0.0007) [2023-12-27 13:46:57,612][20347] Updated weights for policy 0, policy_version 1173822 (0.0007) [2023-12-27 13:46:57,667][20347] Updated weights for policy 0, policy_version 1173832 (0.0007) [2023-12-27 13:46:57,903][20381] Updated weights for policy 1, policy_version 1172860 (0.0007) [2023-12-27 13:46:57,954][20381] Updated weights for policy 1, policy_version 1172870 (0.0007) [2023-12-27 13:46:58,009][20381] Updated weights for policy 1, policy_version 1172880 (0.0008) [2023-12-27 13:46:58,317][20347] Updated weights for policy 0, policy_version 1173842 (0.0007) [2023-12-27 13:46:58,380][20347] Updated weights for policy 0, policy_version 1173852 (0.0008) [2023-12-27 13:46:58,441][20347] Updated weights for policy 0, policy_version 1173862 (0.0008) [2023-12-27 13:46:58,507][20347] Updated weights for policy 0, policy_version 1173872 (0.0008) [2023-12-27 13:46:58,758][20381] Updated weights for policy 1, policy_version 1172890 (0.0006) [2023-12-27 13:46:58,829][20381] Updated weights for policy 1, policy_version 1172900 (0.0008) [2023-12-27 13:46:58,883][20381] Updated weights for policy 1, policy_version 1172910 (0.0009) [2023-12-27 13:46:58,938][20381] Updated weights for policy 1, policy_version 1172920 (0.0011) [2023-12-27 13:46:59,251][20347] Updated weights for policy 0, policy_version 1173882 (0.0008) [2023-12-27 13:46:59,306][20347] Updated weights for policy 0, policy_version 1173892 (0.0007) [2023-12-27 13:46:59,367][20347] Updated weights for policy 0, policy_version 1173902 (0.0006) [2023-12-27 13:46:59,649][20381] Updated weights for policy 1, policy_version 1172930 (0.0007) [2023-12-27 13:46:59,699][20381] Updated weights for policy 1, policy_version 1172940 (0.0009) [2023-12-27 13:46:59,759][20381] Updated weights for policy 1, policy_version 1172950 (0.0007) [2023-12-27 13:47:00,051][20347] Updated weights for policy 0, policy_version 1173912 (0.0007) [2023-12-27 13:47:00,095][20347] Updated weights for policy 0, policy_version 1173922 (0.0007) [2023-12-27 13:47:00,153][20347] Updated weights for policy 0, policy_version 1173932 (0.0008) [2023-12-27 13:47:00,450][20381] Updated weights for policy 1, policy_version 1172960 (0.0008) [2023-12-27 13:47:00,506][20381] Updated weights for policy 1, policy_version 1172970 (0.0008) [2023-12-27 13:47:00,551][20381] Updated weights for policy 1, policy_version 1172980 (0.0006) [2023-12-27 13:47:00,817][20347] Updated weights for policy 0, policy_version 1173942 (0.0006) [2023-12-27 13:47:00,872][20347] Updated weights for policy 0, policy_version 1173952 (0.0005) [2023-12-27 13:47:00,927][20347] Updated weights for policy 0, policy_version 1173962 (0.0005) [2023-12-27 13:47:01,232][20381] Updated weights for policy 1, policy_version 1172990 (0.0006) [2023-12-27 13:47:01,291][20381] Updated weights for policy 1, policy_version 1173000 (0.0007) [2023-12-27 13:47:01,349][20381] Updated weights for policy 1, policy_version 1173010 (0.0008) [2023-12-27 13:47:01,567][20347] Updated weights for policy 0, policy_version 1173972 (0.0006) [2023-12-27 13:47:01,623][20347] Updated weights for policy 0, policy_version 1173982 (0.0007) [2023-12-27 13:47:01,690][20347] Updated weights for policy 0, policy_version 1173992 (0.0008) [2023-12-27 13:47:02,062][20381] Updated weights for policy 1, policy_version 1173020 (0.0007) [2023-12-27 13:47:02,119][20381] Updated weights for policy 1, policy_version 1173030 (0.0005) [2023-12-27 13:47:02,178][20381] Updated weights for policy 1, policy_version 1173040 (0.0006) [2023-12-27 13:47:02,352][20347] Updated weights for policy 0, policy_version 1174002 (0.0008) [2023-12-27 13:47:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 600924160. Throughput: 0: 10353.2, 1: 10320.7. Samples: 600934628. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:02,363][19170] Avg episode reward: [(0, '951.310'), (1, '755.420')] [2023-12-27 13:47:02,418][20347] Updated weights for policy 0, policy_version 1174012 (0.0010) [2023-12-27 13:47:02,477][20347] Updated weights for policy 0, policy_version 1174022 (0.0005) [2023-12-27 13:47:02,525][20347] Updated weights for policy 0, policy_version 1174032 (0.0005) [2023-12-27 13:47:02,796][20381] Updated weights for policy 1, policy_version 1173050 (0.0008) [2023-12-27 13:47:02,852][20381] Updated weights for policy 1, policy_version 1173060 (0.0008) [2023-12-27 13:47:02,912][20381] Updated weights for policy 1, policy_version 1173070 (0.0008) [2023-12-27 13:47:02,965][20381] Updated weights for policy 1, policy_version 1173080 (0.0008) [2023-12-27 13:47:03,123][20347] Updated weights for policy 0, policy_version 1174042 (0.0008) [2023-12-27 13:47:03,170][20347] Updated weights for policy 0, policy_version 1174052 (0.0007) [2023-12-27 13:47:03,214][20347] Updated weights for policy 0, policy_version 1174062 (0.0008) [2023-12-27 13:47:03,643][20381] Updated weights for policy 1, policy_version 1173090 (0.0008) [2023-12-27 13:47:03,699][20381] Updated weights for policy 1, policy_version 1173100 (0.0009) [2023-12-27 13:47:03,753][20381] Updated weights for policy 1, policy_version 1173110 (0.0007) [2023-12-27 13:47:03,842][20347] Updated weights for policy 0, policy_version 1174072 (0.0007) [2023-12-27 13:47:03,903][20347] Updated weights for policy 0, policy_version 1174082 (0.0008) [2023-12-27 13:47:03,955][20347] Updated weights for policy 0, policy_version 1174092 (0.0007) [2023-12-27 13:47:04,506][20381] Updated weights for policy 1, policy_version 1173120 (0.0008) [2023-12-27 13:47:04,563][20381] Updated weights for policy 1, policy_version 1173130 (0.0008) [2023-12-27 13:47:04,617][20381] Updated weights for policy 1, policy_version 1173140 (0.0006) [2023-12-27 13:47:04,619][20347] Updated weights for policy 0, policy_version 1174102 (0.0007) [2023-12-27 13:47:04,678][20347] Updated weights for policy 0, policy_version 1174112 (0.0007) [2023-12-27 13:47:04,724][20347] Updated weights for policy 0, policy_version 1174122 (0.0008) [2023-12-27 13:47:05,315][20381] Updated weights for policy 1, policy_version 1173150 (0.0007) [2023-12-27 13:47:05,329][20347] Updated weights for policy 0, policy_version 1174132 (0.0007) [2023-12-27 13:47:05,367][20381] Updated weights for policy 1, policy_version 1173160 (0.0005) [2023-12-27 13:47:05,384][20347] Updated weights for policy 0, policy_version 1174142 (0.0006) [2023-12-27 13:47:05,423][20381] Updated weights for policy 1, policy_version 1173170 (0.0006) [2023-12-27 13:47:05,439][20347] Updated weights for policy 0, policy_version 1174152 (0.0007) [2023-12-27 13:47:06,118][20381] Updated weights for policy 1, policy_version 1173180 (0.0006) [2023-12-27 13:47:06,180][20381] Updated weights for policy 1, policy_version 1173190 (0.0007) [2023-12-27 13:47:06,180][20347] Updated weights for policy 0, policy_version 1174162 (0.0006) [2023-12-27 13:47:06,237][20381] Updated weights for policy 1, policy_version 1173200 (0.0007) [2023-12-27 13:47:06,244][20347] Updated weights for policy 0, policy_version 1174172 (0.0007) [2023-12-27 13:47:06,300][20347] Updated weights for policy 0, policy_version 1174182 (0.0009) [2023-12-27 13:47:06,352][20347] Updated weights for policy 0, policy_version 1174192 (0.0009) [2023-12-27 13:47:06,791][20381] Updated weights for policy 1, policy_version 1173210 (0.0006) [2023-12-27 13:47:06,848][20381] Updated weights for policy 1, policy_version 1173220 (0.0007) [2023-12-27 13:47:06,904][20381] Updated weights for policy 1, policy_version 1173230 (0.0008) [2023-12-27 13:47:06,965][20381] Updated weights for policy 1, policy_version 1173240 (0.0008) [2023-12-27 13:47:07,112][20347] Updated weights for policy 0, policy_version 1174202 (0.0007) [2023-12-27 13:47:07,161][20347] Updated weights for policy 0, policy_version 1174212 (0.0007) [2023-12-27 13:47:07,227][20347] Updated weights for policy 0, policy_version 1174222 (0.0006) [2023-12-27 13:47:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 601030656. Throughput: 0: 10395.1, 1: 10327.0. Samples: 600997740. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:07,363][19170] Avg episode reward: [(0, '941.590'), (1, '647.620')] [2023-12-27 13:47:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001173240_300376064.pth... [2023-12-27 13:47:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001174224_300654592.pth... [2023-12-27 13:47:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001172056_300072960.pth [2023-12-27 13:47:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001173008_300343296.pth [2023-12-27 13:47:07,685][20381] Updated weights for policy 1, policy_version 1173250 (0.0009) [2023-12-27 13:47:07,735][20381] Updated weights for policy 1, policy_version 1173260 (0.0007) [2023-12-27 13:47:07,779][20381] Updated weights for policy 1, policy_version 1173270 (0.0007) [2023-12-27 13:47:07,799][20347] Updated weights for policy 0, policy_version 1174232 (0.0006) [2023-12-27 13:47:07,855][20347] Updated weights for policy 0, policy_version 1174242 (0.0007) [2023-12-27 13:47:07,914][20347] Updated weights for policy 0, policy_version 1174252 (0.0006) [2023-12-27 13:47:08,516][20381] Updated weights for policy 1, policy_version 1173280 (0.0007) [2023-12-27 13:47:08,572][20381] Updated weights for policy 1, policy_version 1173290 (0.0009) [2023-12-27 13:47:08,578][20347] Updated weights for policy 0, policy_version 1174262 (0.0007) [2023-12-27 13:47:08,628][20381] Updated weights for policy 1, policy_version 1173300 (0.0006) [2023-12-27 13:47:08,634][20347] Updated weights for policy 0, policy_version 1174272 (0.0005) [2023-12-27 13:47:08,690][20347] Updated weights for policy 0, policy_version 1174282 (0.0006) [2023-12-27 13:47:09,303][20381] Updated weights for policy 1, policy_version 1173310 (0.0007) [2023-12-27 13:47:09,371][20381] Updated weights for policy 1, policy_version 1173320 (0.0007) [2023-12-27 13:47:09,410][20347] Updated weights for policy 0, policy_version 1174292 (0.0008) [2023-12-27 13:47:09,432][20381] Updated weights for policy 1, policy_version 1173330 (0.0007) [2023-12-27 13:47:09,473][20347] Updated weights for policy 0, policy_version 1174302 (0.0008) [2023-12-27 13:47:09,533][20347] Updated weights for policy 0, policy_version 1174312 (0.0008) [2023-12-27 13:47:10,181][20381] Updated weights for policy 1, policy_version 1173340 (0.0007) [2023-12-27 13:47:10,217][20347] Updated weights for policy 0, policy_version 1174322 (0.0008) [2023-12-27 13:47:10,231][20381] Updated weights for policy 1, policy_version 1173350 (0.0007) [2023-12-27 13:47:10,270][20347] Updated weights for policy 0, policy_version 1174332 (0.0006) [2023-12-27 13:47:10,279][20381] Updated weights for policy 1, policy_version 1173360 (0.0007) [2023-12-27 13:47:10,328][20347] Updated weights for policy 0, policy_version 1174342 (0.0009) [2023-12-27 13:47:10,388][20347] Updated weights for policy 0, policy_version 1174352 (0.0008) [2023-12-27 13:47:10,910][20381] Updated weights for policy 1, policy_version 1173370 (0.0008) [2023-12-27 13:47:10,962][20381] Updated weights for policy 1, policy_version 1173380 (0.0008) [2023-12-27 13:47:11,025][20381] Updated weights for policy 1, policy_version 1173390 (0.0008) [2023-12-27 13:47:11,087][20381] Updated weights for policy 1, policy_version 1173400 (0.0008) [2023-12-27 13:47:11,128][20347] Updated weights for policy 0, policy_version 1174362 (0.0009) [2023-12-27 13:47:11,186][20347] Updated weights for policy 0, policy_version 1174372 (0.0006) [2023-12-27 13:47:11,242][20347] Updated weights for policy 0, policy_version 1174382 (0.0008) [2023-12-27 13:47:11,906][20381] Updated weights for policy 1, policy_version 1173410 (0.0008) [2023-12-27 13:47:11,963][20381] Updated weights for policy 1, policy_version 1173420 (0.0007) [2023-12-27 13:47:12,000][20347] Updated weights for policy 0, policy_version 1174392 (0.0007) [2023-12-27 13:47:12,026][20381] Updated weights for policy 1, policy_version 1173430 (0.0008) [2023-12-27 13:47:12,056][20347] Updated weights for policy 0, policy_version 1174402 (0.0006) [2023-12-27 13:47:12,111][20347] Updated weights for policy 0, policy_version 1174412 (0.0006) [2023-12-27 13:47:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 601128960. Throughput: 0: 10412.3, 1: 10299.5. Samples: 601119072. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:12,363][19170] Avg episode reward: [(0, '912.490'), (1, '657.460')] [2023-12-27 13:47:12,704][20347] Updated weights for policy 0, policy_version 1174422 (0.0007) [2023-12-27 13:47:12,762][20381] Updated weights for policy 1, policy_version 1173440 (0.0007) [2023-12-27 13:47:12,763][20347] Updated weights for policy 0, policy_version 1174432 (0.0007) [2023-12-27 13:47:12,819][20381] Updated weights for policy 1, policy_version 1173450 (0.0006) [2023-12-27 13:47:12,823][20347] Updated weights for policy 0, policy_version 1174442 (0.0008) [2023-12-27 13:47:12,876][20381] Updated weights for policy 1, policy_version 1173460 (0.0006) [2023-12-27 13:47:13,448][20381] Updated weights for policy 1, policy_version 1173470 (0.0007) [2023-12-27 13:47:13,505][20381] Updated weights for policy 1, policy_version 1173480 (0.0008) [2023-12-27 13:47:13,544][20347] Updated weights for policy 0, policy_version 1174452 (0.0007) [2023-12-27 13:47:13,553][20381] Updated weights for policy 1, policy_version 1173490 (0.0008) [2023-12-27 13:47:13,586][20347] Updated weights for policy 0, policy_version 1174462 (0.0005) [2023-12-27 13:47:13,637][20347] Updated weights for policy 0, policy_version 1174472 (0.0006) [2023-12-27 13:47:14,298][20347] Updated weights for policy 0, policy_version 1174482 (0.0008) [2023-12-27 13:47:14,307][20381] Updated weights for policy 1, policy_version 1173500 (0.0006) [2023-12-27 13:47:14,354][20347] Updated weights for policy 0, policy_version 1174492 (0.0006) [2023-12-27 13:47:14,367][20381] Updated weights for policy 1, policy_version 1173510 (0.0006) [2023-12-27 13:47:14,415][20347] Updated weights for policy 0, policy_version 1174502 (0.0006) [2023-12-27 13:47:14,428][20381] Updated weights for policy 1, policy_version 1173520 (0.0006) [2023-12-27 13:47:14,476][20347] Updated weights for policy 0, policy_version 1174512 (0.0008) [2023-12-27 13:47:15,126][20347] Updated weights for policy 0, policy_version 1174522 (0.0011) [2023-12-27 13:47:15,162][20381] Updated weights for policy 1, policy_version 1173530 (0.0007) [2023-12-27 13:47:15,184][20347] Updated weights for policy 0, policy_version 1174532 (0.0007) [2023-12-27 13:47:15,213][20381] Updated weights for policy 1, policy_version 1173540 (0.0007) [2023-12-27 13:47:15,245][20347] Updated weights for policy 0, policy_version 1174542 (0.0010) [2023-12-27 13:47:15,268][20381] Updated weights for policy 1, policy_version 1173550 (0.0007) [2023-12-27 13:47:15,327][20381] Updated weights for policy 1, policy_version 1173560 (0.0008) [2023-12-27 13:47:15,903][20347] Updated weights for policy 0, policy_version 1174552 (0.0010) [2023-12-27 13:47:15,957][20347] Updated weights for policy 0, policy_version 1174562 (0.0006) [2023-12-27 13:47:16,016][20347] Updated weights for policy 0, policy_version 1174572 (0.0010) [2023-12-27 13:47:16,036][20381] Updated weights for policy 1, policy_version 1173570 (0.0006) [2023-12-27 13:47:16,083][20381] Updated weights for policy 1, policy_version 1173580 (0.0007) [2023-12-27 13:47:16,131][20381] Updated weights for policy 1, policy_version 1173590 (0.0008) [2023-12-27 13:47:16,630][20347] Updated weights for policy 0, policy_version 1174582 (0.0007) [2023-12-27 13:47:16,695][20347] Updated weights for policy 0, policy_version 1174592 (0.0009) [2023-12-27 13:47:16,755][20347] Updated weights for policy 0, policy_version 1174602 (0.0010) [2023-12-27 13:47:16,771][20381] Updated weights for policy 1, policy_version 1173600 (0.0007) [2023-12-27 13:47:16,831][20381] Updated weights for policy 1, policy_version 1173610 (0.0007) [2023-12-27 13:47:16,887][20381] Updated weights for policy 1, policy_version 1173620 (0.0008) [2023-12-27 13:47:17,306][20347] Updated weights for policy 0, policy_version 1174612 (0.0007) [2023-12-27 13:47:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 601227264. Throughput: 0: 10454.0, 1: 10195.2. Samples: 601240628. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:17,363][19170] Avg episode reward: [(0, '922.610'), (1, '617.940')] [2023-12-27 13:47:17,365][20347] Updated weights for policy 0, policy_version 1174622 (0.0008) [2023-12-27 13:47:17,421][20347] Updated weights for policy 0, policy_version 1174632 (0.0008) [2023-12-27 13:47:17,631][20381] Updated weights for policy 1, policy_version 1173630 (0.0008) [2023-12-27 13:47:17,684][20381] Updated weights for policy 1, policy_version 1173640 (0.0009) [2023-12-27 13:47:17,737][20381] Updated weights for policy 1, policy_version 1173650 (0.0008) [2023-12-27 13:47:18,036][20347] Updated weights for policy 0, policy_version 1174642 (0.0008) [2023-12-27 13:47:18,090][20347] Updated weights for policy 0, policy_version 1174652 (0.0008) [2023-12-27 13:47:18,138][20347] Updated weights for policy 0, policy_version 1174662 (0.0005) [2023-12-27 13:47:18,185][20347] Updated weights for policy 0, policy_version 1174672 (0.0005) [2023-12-27 13:47:18,424][20381] Updated weights for policy 1, policy_version 1173660 (0.0008) [2023-12-27 13:47:18,486][20381] Updated weights for policy 1, policy_version 1173670 (0.0006) [2023-12-27 13:47:18,544][20381] Updated weights for policy 1, policy_version 1173680 (0.0008) [2023-12-27 13:47:18,875][20347] Updated weights for policy 0, policy_version 1174682 (0.0008) [2023-12-27 13:47:18,938][20347] Updated weights for policy 0, policy_version 1174692 (0.0007) [2023-12-27 13:47:19,001][20347] Updated weights for policy 0, policy_version 1174702 (0.0008) [2023-12-27 13:47:19,137][20381] Updated weights for policy 1, policy_version 1173690 (0.0008) [2023-12-27 13:47:19,200][20381] Updated weights for policy 1, policy_version 1173700 (0.0008) [2023-12-27 13:47:19,272][20381] Updated weights for policy 1, policy_version 1173710 (0.0008) [2023-12-27 13:47:19,334][20381] Updated weights for policy 1, policy_version 1173720 (0.0007) [2023-12-27 13:47:19,653][20347] Updated weights for policy 0, policy_version 1174712 (0.0007) [2023-12-27 13:47:19,709][20347] Updated weights for policy 0, policy_version 1174722 (0.0008) [2023-12-27 13:47:19,775][20347] Updated weights for policy 0, policy_version 1174732 (0.0008) [2023-12-27 13:47:20,078][20381] Updated weights for policy 1, policy_version 1173730 (0.0007) [2023-12-27 13:47:20,135][20381] Updated weights for policy 1, policy_version 1173740 (0.0009) [2023-12-27 13:47:20,185][20381] Updated weights for policy 1, policy_version 1173750 (0.0008) [2023-12-27 13:47:20,432][20347] Updated weights for policy 0, policy_version 1174742 (0.0008) [2023-12-27 13:47:20,490][20347] Updated weights for policy 0, policy_version 1174752 (0.0007) [2023-12-27 13:47:20,542][20347] Updated weights for policy 0, policy_version 1174762 (0.0008) [2023-12-27 13:47:20,894][20381] Updated weights for policy 1, policy_version 1173760 (0.0006) [2023-12-27 13:47:20,952][20381] Updated weights for policy 1, policy_version 1173770 (0.0005) [2023-12-27 13:47:21,012][20381] Updated weights for policy 1, policy_version 1173780 (0.0006) [2023-12-27 13:47:21,287][20347] Updated weights for policy 0, policy_version 1174772 (0.0008) [2023-12-27 13:47:21,352][20347] Updated weights for policy 0, policy_version 1174782 (0.0007) [2023-12-27 13:47:21,411][20347] Updated weights for policy 0, policy_version 1174792 (0.0007) [2023-12-27 13:47:21,742][20381] Updated weights for policy 1, policy_version 1173790 (0.0009) [2023-12-27 13:47:21,805][20381] Updated weights for policy 1, policy_version 1173800 (0.0007) [2023-12-27 13:47:21,862][20381] Updated weights for policy 1, policy_version 1173810 (0.0008) [2023-12-27 13:47:22,147][20347] Updated weights for policy 0, policy_version 1174802 (0.0008) [2023-12-27 13:47:22,208][20347] Updated weights for policy 0, policy_version 1174812 (0.0007) [2023-12-27 13:47:22,267][20347] Updated weights for policy 0, policy_version 1174822 (0.0008) [2023-12-27 13:47:22,331][20347] Updated weights for policy 0, policy_version 1174832 (0.0007) [2023-12-27 13:47:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 601333760. Throughput: 0: 10427.9, 1: 10115.7. Samples: 601302096. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:22,363][19170] Avg episode reward: [(0, '951.830'), (1, '666.770')] [2023-12-27 13:47:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001173816_300523520.pth... [2023-12-27 13:47:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001174832_300810240.pth... [2023-12-27 13:47:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001172664_300228608.pth [2023-12-27 13:47:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001173616_300498944.pth [2023-12-27 13:47:22,586][20381] Updated weights for policy 1, policy_version 1173820 (0.0008) [2023-12-27 13:47:22,643][20381] Updated weights for policy 1, policy_version 1173830 (0.0008) [2023-12-27 13:47:22,702][20381] Updated weights for policy 1, policy_version 1173840 (0.0008) [2023-12-27 13:47:23,071][20347] Updated weights for policy 0, policy_version 1174842 (0.0008) [2023-12-27 13:47:23,095][20291] KL-divergence is very high: 142.1436 [2023-12-27 13:47:23,131][20347] Updated weights for policy 0, policy_version 1174852 (0.0008) [2023-12-27 13:47:23,165][20291] KL-divergence is very high: 139.4775 [2023-12-27 13:47:23,184][20347] Updated weights for policy 0, policy_version 1174862 (0.0005) [2023-12-27 13:47:23,184][20291] KL-divergence is very high: 169.4258 [2023-12-27 13:47:23,371][20381] Updated weights for policy 1, policy_version 1173850 (0.0007) [2023-12-27 13:47:23,424][20381] Updated weights for policy 1, policy_version 1173860 (0.0006) [2023-12-27 13:47:23,474][20381] Updated weights for policy 1, policy_version 1173870 (0.0005) [2023-12-27 13:47:23,524][20381] Updated weights for policy 1, policy_version 1173880 (0.0009) [2023-12-27 13:47:23,865][20347] Updated weights for policy 0, policy_version 1174872 (0.0007) [2023-12-27 13:47:23,922][20347] Updated weights for policy 0, policy_version 1174882 (0.0008) [2023-12-27 13:47:23,985][20347] Updated weights for policy 0, policy_version 1174892 (0.0009) [2023-12-27 13:47:24,153][20381] Updated weights for policy 1, policy_version 1173890 (0.0010) [2023-12-27 13:47:24,213][20381] Updated weights for policy 1, policy_version 1173900 (0.0011) [2023-12-27 13:47:24,263][20381] Updated weights for policy 1, policy_version 1173910 (0.0010) [2023-12-27 13:47:24,661][20347] Updated weights for policy 0, policy_version 1174902 (0.0007) [2023-12-27 13:47:24,661][20291] KL-divergence is very high: 115.1751 [2023-12-27 13:47:24,711][20347] Updated weights for policy 0, policy_version 1174912 (0.0005) [2023-12-27 13:47:24,760][20347] Updated weights for policy 0, policy_version 1174922 (0.0005) [2023-12-27 13:47:24,940][20381] Updated weights for policy 1, policy_version 1173920 (0.0010) [2023-12-27 13:47:24,991][20381] Updated weights for policy 1, policy_version 1173930 (0.0006) [2023-12-27 13:47:25,040][20381] Updated weights for policy 1, policy_version 1173940 (0.0005) [2023-12-27 13:47:25,404][20347] Updated weights for policy 0, policy_version 1174932 (0.0006) [2023-12-27 13:47:25,455][20347] Updated weights for policy 0, policy_version 1174942 (0.0007) [2023-12-27 13:47:25,509][20347] Updated weights for policy 0, policy_version 1174952 (0.0007) [2023-12-27 13:47:25,680][20381] Updated weights for policy 1, policy_version 1173950 (0.0008) [2023-12-27 13:47:25,725][20381] Updated weights for policy 1, policy_version 1173960 (0.0010) [2023-12-27 13:47:25,786][20381] Updated weights for policy 1, policy_version 1173970 (0.0010) [2023-12-27 13:47:26,217][20347] Updated weights for policy 0, policy_version 1174962 (0.0007) [2023-12-27 13:47:26,278][20347] Updated weights for policy 0, policy_version 1174972 (0.0007) [2023-12-27 13:47:26,329][20347] Updated weights for policy 0, policy_version 1174982 (0.0007) [2023-12-27 13:47:26,379][20347] Updated weights for policy 0, policy_version 1174992 (0.0007) [2023-12-27 13:47:26,489][20381] Updated weights for policy 1, policy_version 1173980 (0.0010) [2023-12-27 13:47:26,542][20381] Updated weights for policy 1, policy_version 1173990 (0.0010) [2023-12-27 13:47:26,592][20381] Updated weights for policy 1, policy_version 1174000 (0.0006) [2023-12-27 13:47:27,019][20347] Updated weights for policy 0, policy_version 1175002 (0.0005) [2023-12-27 13:47:27,064][20347] Updated weights for policy 0, policy_version 1175012 (0.0005) [2023-12-27 13:47:27,121][20347] Updated weights for policy 0, policy_version 1175022 (0.0005) [2023-12-27 13:47:27,147][20381] Updated weights for policy 1, policy_version 1174010 (0.0007) [2023-12-27 13:47:27,204][20381] Updated weights for policy 1, policy_version 1174020 (0.0008) [2023-12-27 13:47:27,259][20381] Updated weights for policy 1, policy_version 1174030 (0.0008) [2023-12-27 13:47:27,314][20381] Updated weights for policy 1, policy_version 1174040 (0.0007) [2023-12-27 13:47:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 601440256. Throughput: 0: 10370.9, 1: 10118.2. Samples: 601423340. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:27,363][19170] Avg episode reward: [(0, '912.590'), (1, '715.510')] [2023-12-27 13:47:27,726][20347] Updated weights for policy 0, policy_version 1175032 (0.0007) [2023-12-27 13:47:27,783][20347] Updated weights for policy 0, policy_version 1175042 (0.0007) [2023-12-27 13:47:27,845][20347] Updated weights for policy 0, policy_version 1175052 (0.0008) [2023-12-27 13:47:28,026][20381] Updated weights for policy 1, policy_version 1174050 (0.0007) [2023-12-27 13:47:28,080][20381] Updated weights for policy 1, policy_version 1174060 (0.0010) [2023-12-27 13:47:28,132][20381] Updated weights for policy 1, policy_version 1174070 (0.0007) [2023-12-27 13:47:28,581][20347] Updated weights for policy 0, policy_version 1175062 (0.0008) [2023-12-27 13:47:28,629][20347] Updated weights for policy 0, policy_version 1175072 (0.0008) [2023-12-27 13:47:28,691][20347] Updated weights for policy 0, policy_version 1175082 (0.0006) [2023-12-27 13:47:28,715][20381] Updated weights for policy 1, policy_version 1174080 (0.0006) [2023-12-27 13:47:28,770][20381] Updated weights for policy 1, policy_version 1174090 (0.0007) [2023-12-27 13:47:28,824][20381] Updated weights for policy 1, policy_version 1174100 (0.0005) [2023-12-27 13:47:29,396][20347] Updated weights for policy 0, policy_version 1175092 (0.0007) [2023-12-27 13:47:29,449][20347] Updated weights for policy 0, policy_version 1175102 (0.0008) [2023-12-27 13:47:29,472][20381] Updated weights for policy 1, policy_version 1174110 (0.0007) [2023-12-27 13:47:29,497][20347] Updated weights for policy 0, policy_version 1175112 (0.0008) [2023-12-27 13:47:29,520][20381] Updated weights for policy 1, policy_version 1174120 (0.0008) [2023-12-27 13:47:29,574][20381] Updated weights for policy 1, policy_version 1174130 (0.0008) [2023-12-27 13:47:30,192][20347] Updated weights for policy 0, policy_version 1175122 (0.0007) [2023-12-27 13:47:30,234][20291] KL-divergence is very high: 130.0007 [2023-12-27 13:47:30,244][20347] Updated weights for policy 0, policy_version 1175132 (0.0005) [2023-12-27 13:47:30,293][20347] Updated weights for policy 0, policy_version 1175142 (0.0005) [2023-12-27 13:47:30,341][20347] Updated weights for policy 0, policy_version 1175152 (0.0006) [2023-12-27 13:47:30,357][20381] Updated weights for policy 1, policy_version 1174140 (0.0008) [2023-12-27 13:47:30,413][20381] Updated weights for policy 1, policy_version 1174150 (0.0007) [2023-12-27 13:47:30,459][20381] Updated weights for policy 1, policy_version 1174160 (0.0007) [2023-12-27 13:47:30,994][20347] Updated weights for policy 0, policy_version 1175162 (0.0007) [2023-12-27 13:47:31,054][20347] Updated weights for policy 0, policy_version 1175172 (0.0008) [2023-12-27 13:47:31,113][20347] Updated weights for policy 0, policy_version 1175182 (0.0008) [2023-12-27 13:47:31,190][20381] Updated weights for policy 1, policy_version 1174170 (0.0007) [2023-12-27 13:47:31,249][20381] Updated weights for policy 1, policy_version 1174180 (0.0007) [2023-12-27 13:47:31,305][20381] Updated weights for policy 1, policy_version 1174190 (0.0006) [2023-12-27 13:47:31,369][20381] Updated weights for policy 1, policy_version 1174200 (0.0006) [2023-12-27 13:47:31,845][20347] Updated weights for policy 0, policy_version 1175192 (0.0008) [2023-12-27 13:47:31,903][20347] Updated weights for policy 0, policy_version 1175202 (0.0007) [2023-12-27 13:47:31,961][20347] Updated weights for policy 0, policy_version 1175212 (0.0007) [2023-12-27 13:47:32,088][20381] Updated weights for policy 1, policy_version 1174210 (0.0008) [2023-12-27 13:47:32,146][20381] Updated weights for policy 1, policy_version 1174220 (0.0008) [2023-12-27 13:47:32,203][20381] Updated weights for policy 1, policy_version 1174230 (0.0008) [2023-12-27 13:47:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 601538560. Throughput: 0: 10346.5, 1: 10065.0. Samples: 601545896. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:32,363][19170] Avg episode reward: [(0, '931.800'), (1, '715.720')] [2023-12-27 13:47:32,674][20347] Updated weights for policy 0, policy_version 1175222 (0.0008) [2023-12-27 13:47:32,733][20347] Updated weights for policy 0, policy_version 1175232 (0.0008) [2023-12-27 13:47:32,786][20347] Updated weights for policy 0, policy_version 1175242 (0.0010) [2023-12-27 13:47:32,886][20381] Updated weights for policy 1, policy_version 1174240 (0.0009) [2023-12-27 13:47:32,944][20381] Updated weights for policy 1, policy_version 1174250 (0.0007) [2023-12-27 13:47:33,007][20381] Updated weights for policy 1, policy_version 1174260 (0.0005) [2023-12-27 13:47:33,473][20347] Updated weights for policy 0, policy_version 1175252 (0.0010) [2023-12-27 13:47:33,511][20381] Updated weights for policy 1, policy_version 1174270 (0.0005) [2023-12-27 13:47:33,534][20347] Updated weights for policy 0, policy_version 1175262 (0.0010) [2023-12-27 13:47:33,567][20381] Updated weights for policy 1, policy_version 1174280 (0.0006) [2023-12-27 13:47:33,592][20347] Updated weights for policy 0, policy_version 1175272 (0.0010) [2023-12-27 13:47:33,625][20381] Updated weights for policy 1, policy_version 1174290 (0.0010) [2023-12-27 13:47:34,201][20347] Updated weights for policy 0, policy_version 1175282 (0.0010) [2023-12-27 13:47:34,215][20381] Updated weights for policy 1, policy_version 1174300 (0.0010) [2023-12-27 13:47:34,259][20347] Updated weights for policy 0, policy_version 1175292 (0.0010) [2023-12-27 13:47:34,280][20381] Updated weights for policy 1, policy_version 1174310 (0.0008) [2023-12-27 13:47:34,324][20347] Updated weights for policy 0, policy_version 1175302 (0.0010) [2023-12-27 13:47:34,347][20381] Updated weights for policy 1, policy_version 1174320 (0.0008) [2023-12-27 13:47:34,383][20347] Updated weights for policy 0, policy_version 1175312 (0.0010) [2023-12-27 13:47:35,025][20381] Updated weights for policy 1, policy_version 1174330 (0.0007) [2023-12-27 13:47:35,081][20381] Updated weights for policy 1, policy_version 1174340 (0.0005) [2023-12-27 13:47:35,105][20347] Updated weights for policy 0, policy_version 1175322 (0.0009) [2023-12-27 13:47:35,135][20381] Updated weights for policy 1, policy_version 1174350 (0.0010) [2023-12-27 13:47:35,157][20347] Updated weights for policy 0, policy_version 1175332 (0.0008) [2023-12-27 13:47:35,185][20381] Updated weights for policy 1, policy_version 1174360 (0.0010) [2023-12-27 13:47:35,209][20347] Updated weights for policy 0, policy_version 1175342 (0.0006) [2023-12-27 13:47:35,863][20381] Updated weights for policy 1, policy_version 1174370 (0.0011) [2023-12-27 13:47:35,893][20347] Updated weights for policy 0, policy_version 1175352 (0.0006) [2023-12-27 13:47:35,911][20381] Updated weights for policy 1, policy_version 1174380 (0.0010) [2023-12-27 13:47:35,952][20347] Updated weights for policy 0, policy_version 1175362 (0.0005) [2023-12-27 13:47:35,959][20381] Updated weights for policy 1, policy_version 1174390 (0.0010) [2023-12-27 13:47:36,013][20347] Updated weights for policy 0, policy_version 1175372 (0.0006) [2023-12-27 13:47:36,667][20381] Updated weights for policy 1, policy_version 1174400 (0.0011) [2023-12-27 13:47:36,693][20347] Updated weights for policy 0, policy_version 1175382 (0.0010) [2023-12-27 13:47:36,726][20381] Updated weights for policy 1, policy_version 1174410 (0.0010) [2023-12-27 13:47:36,746][20347] Updated weights for policy 0, policy_version 1175392 (0.0010) [2023-12-27 13:47:36,778][20381] Updated weights for policy 1, policy_version 1174420 (0.0007) [2023-12-27 13:47:36,801][20347] Updated weights for policy 0, policy_version 1175402 (0.0010) [2023-12-27 13:47:37,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 601636864. Throughput: 0: 10264.6, 1: 10137.4. Samples: 601609692. Policy #0 lag: (min: 8.0, avg: 33.3, max: 40.0) [2023-12-27 13:47:37,364][19170] Avg episode reward: [(0, '941.940'), (1, '735.460')] [2023-12-27 13:47:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001174424_300679168.pth... [2023-12-27 13:47:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001175408_300957696.pth... [2023-12-27 13:47:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001174224_300654592.pth [2023-12-27 13:47:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001173240_300376064.pth [2023-12-27 13:47:37,433][20381] Updated weights for policy 1, policy_version 1174430 (0.0006) [2023-12-27 13:47:37,495][20381] Updated weights for policy 1, policy_version 1174440 (0.0006) [2023-12-27 13:47:37,506][20347] Updated weights for policy 0, policy_version 1175412 (0.0010) [2023-12-27 13:47:37,551][20381] Updated weights for policy 1, policy_version 1174450 (0.0006) [2023-12-27 13:47:37,561][20347] Updated weights for policy 0, policy_version 1175422 (0.0011) [2023-12-27 13:47:37,620][20347] Updated weights for policy 0, policy_version 1175432 (0.0010) [2023-12-27 13:47:38,174][20381] Updated weights for policy 1, policy_version 1174460 (0.0006) [2023-12-27 13:47:38,231][20381] Updated weights for policy 1, policy_version 1174470 (0.0006) [2023-12-27 13:47:38,278][20381] Updated weights for policy 1, policy_version 1174480 (0.0007) [2023-12-27 13:47:38,315][20347] Updated weights for policy 0, policy_version 1175442 (0.0011) [2023-12-27 13:47:38,376][20347] Updated weights for policy 0, policy_version 1175452 (0.0013) [2023-12-27 13:47:38,425][20291] KL-divergence is very high: 277.8951 [2023-12-27 13:47:38,426][20347] Updated weights for policy 0, policy_version 1175462 (0.0010) [2023-12-27 13:47:38,472][20347] Updated weights for policy 0, policy_version 1175472 (0.0010) [2023-12-27 13:47:38,943][20381] Updated weights for policy 1, policy_version 1174490 (0.0007) [2023-12-27 13:47:38,997][20381] Updated weights for policy 1, policy_version 1174500 (0.0006) [2023-12-27 13:47:39,056][20381] Updated weights for policy 1, policy_version 1174510 (0.0007) [2023-12-27 13:47:39,117][20381] Updated weights for policy 1, policy_version 1174520 (0.0008) [2023-12-27 13:47:39,180][20347] Updated weights for policy 0, policy_version 1175482 (0.0008) [2023-12-27 13:47:39,243][20347] Updated weights for policy 0, policy_version 1175492 (0.0007) [2023-12-27 13:47:39,305][20347] Updated weights for policy 0, policy_version 1175502 (0.0007) [2023-12-27 13:47:39,826][20381] Updated weights for policy 1, policy_version 1174530 (0.0008) [2023-12-27 13:47:39,888][20381] Updated weights for policy 1, policy_version 1174540 (0.0008) [2023-12-27 13:47:39,955][20381] Updated weights for policy 1, policy_version 1174550 (0.0008) [2023-12-27 13:47:40,050][20347] Updated weights for policy 0, policy_version 1175512 (0.0007) [2023-12-27 13:47:40,111][20347] Updated weights for policy 0, policy_version 1175522 (0.0008) [2023-12-27 13:47:40,165][20347] Updated weights for policy 0, policy_version 1175532 (0.0009) [2023-12-27 13:47:40,650][20381] Updated weights for policy 1, policy_version 1174560 (0.0008) [2023-12-27 13:47:40,708][20381] Updated weights for policy 1, policy_version 1174570 (0.0007) [2023-12-27 13:47:40,762][20381] Updated weights for policy 1, policy_version 1174580 (0.0005) [2023-12-27 13:47:40,877][20347] Updated weights for policy 0, policy_version 1175542 (0.0010) [2023-12-27 13:47:40,931][20347] Updated weights for policy 0, policy_version 1175552 (0.0009) [2023-12-27 13:47:40,990][20347] Updated weights for policy 0, policy_version 1175562 (0.0007) [2023-12-27 13:47:41,474][20381] Updated weights for policy 1, policy_version 1174590 (0.0007) [2023-12-27 13:47:41,529][20381] Updated weights for policy 1, policy_version 1174600 (0.0005) [2023-12-27 13:47:41,588][20381] Updated weights for policy 1, policy_version 1174610 (0.0006) [2023-12-27 13:47:41,775][20347] Updated weights for policy 0, policy_version 1175572 (0.0008) [2023-12-27 13:47:41,839][20347] Updated weights for policy 0, policy_version 1175582 (0.0006) [2023-12-27 13:47:41,897][20347] Updated weights for policy 0, policy_version 1175592 (0.0005) [2023-12-27 13:47:42,306][20381] Updated weights for policy 1, policy_version 1174620 (0.0007) [2023-12-27 13:47:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 601735168. Throughput: 0: 10230.8, 1: 10145.0. Samples: 601730176. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:47:42,363][19170] Avg episode reward: [(0, '903.100'), (1, '696.380')] [2023-12-27 13:47:42,364][20381] Updated weights for policy 1, policy_version 1174630 (0.0008) [2023-12-27 13:47:42,430][20381] Updated weights for policy 1, policy_version 1174640 (0.0008) [2023-12-27 13:47:42,606][20347] Updated weights for policy 0, policy_version 1175602 (0.0006) [2023-12-27 13:47:42,659][20347] Updated weights for policy 0, policy_version 1175612 (0.0008) [2023-12-27 13:47:42,715][20347] Updated weights for policy 0, policy_version 1175622 (0.0007) [2023-12-27 13:47:42,777][20347] Updated weights for policy 0, policy_version 1175632 (0.0006) [2023-12-27 13:47:43,110][20381] Updated weights for policy 1, policy_version 1174650 (0.0008) [2023-12-27 13:47:43,169][20381] Updated weights for policy 1, policy_version 1174660 (0.0008) [2023-12-27 13:47:43,230][20381] Updated weights for policy 1, policy_version 1174670 (0.0005) [2023-12-27 13:47:43,286][20381] Updated weights for policy 1, policy_version 1174680 (0.0005) [2023-12-27 13:47:43,369][20347] Updated weights for policy 0, policy_version 1175642 (0.0007) [2023-12-27 13:47:43,431][20347] Updated weights for policy 0, policy_version 1175652 (0.0005) [2023-12-27 13:47:43,492][20347] Updated weights for policy 0, policy_version 1175662 (0.0006) [2023-12-27 13:47:43,843][20381] Updated weights for policy 1, policy_version 1174690 (0.0009) [2023-12-27 13:47:43,892][20381] Updated weights for policy 1, policy_version 1174700 (0.0010) [2023-12-27 13:47:43,938][20381] Updated weights for policy 1, policy_version 1174710 (0.0010) [2023-12-27 13:47:44,034][20347] Updated weights for policy 0, policy_version 1175672 (0.0005) [2023-12-27 13:47:44,092][20347] Updated weights for policy 0, policy_version 1175682 (0.0005) [2023-12-27 13:47:44,151][20347] Updated weights for policy 0, policy_version 1175692 (0.0007) [2023-12-27 13:47:44,704][20381] Updated weights for policy 1, policy_version 1174720 (0.0008) [2023-12-27 13:47:44,764][20381] Updated weights for policy 1, policy_version 1174730 (0.0008) [2023-12-27 13:47:44,790][20347] Updated weights for policy 0, policy_version 1175702 (0.0008) [2023-12-27 13:47:44,827][20381] Updated weights for policy 1, policy_version 1174740 (0.0008) [2023-12-27 13:47:44,847][20347] Updated weights for policy 0, policy_version 1175712 (0.0008) [2023-12-27 13:47:44,903][20347] Updated weights for policy 0, policy_version 1175722 (0.0007) [2023-12-27 13:47:45,522][20381] Updated weights for policy 1, policy_version 1174750 (0.0006) [2023-12-27 13:47:45,538][20347] Updated weights for policy 0, policy_version 1175732 (0.0007) [2023-12-27 13:47:45,581][20381] Updated weights for policy 1, policy_version 1174760 (0.0005) [2023-12-27 13:47:45,591][20347] Updated weights for policy 0, policy_version 1175742 (0.0008) [2023-12-27 13:47:45,636][20381] Updated weights for policy 1, policy_version 1174770 (0.0006) [2023-12-27 13:47:45,648][20347] Updated weights for policy 0, policy_version 1175752 (0.0009) [2023-12-27 13:47:46,269][20381] Updated weights for policy 1, policy_version 1174780 (0.0006) [2023-12-27 13:47:46,279][20347] Updated weights for policy 0, policy_version 1175762 (0.0007) [2023-12-27 13:47:46,327][20381] Updated weights for policy 1, policy_version 1174790 (0.0006) [2023-12-27 13:47:46,336][20347] Updated weights for policy 0, policy_version 1175772 (0.0006) [2023-12-27 13:47:46,383][20381] Updated weights for policy 1, policy_version 1174800 (0.0007) [2023-12-27 13:47:46,393][20347] Updated weights for policy 0, policy_version 1175782 (0.0007) [2023-12-27 13:47:46,442][20347] Updated weights for policy 0, policy_version 1175792 (0.0005) [2023-12-27 13:47:47,020][20347] Updated weights for policy 0, policy_version 1175802 (0.0010) [2023-12-27 13:47:47,027][20381] Updated weights for policy 1, policy_version 1174810 (0.0008) [2023-12-27 13:47:47,076][20347] Updated weights for policy 0, policy_version 1175812 (0.0010) [2023-12-27 13:47:47,085][20381] Updated weights for policy 1, policy_version 1174820 (0.0010) [2023-12-27 13:47:47,130][20347] Updated weights for policy 0, policy_version 1175822 (0.0010) [2023-12-27 13:47:47,139][20381] Updated weights for policy 1, policy_version 1174830 (0.0010) [2023-12-27 13:47:47,193][20381] Updated weights for policy 1, policy_version 1174840 (0.0010) [2023-12-27 13:47:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 601849856. Throughput: 0: 10280.5, 1: 10180.5. Samples: 601855368. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:47:47,363][19170] Avg episode reward: [(0, '807.500'), (1, '774.660')] [2023-12-27 13:47:47,754][20347] Updated weights for policy 0, policy_version 1175832 (0.0010) [2023-12-27 13:47:47,787][20381] Updated weights for policy 1, policy_version 1174850 (0.0007) [2023-12-27 13:47:47,810][20347] Updated weights for policy 0, policy_version 1175842 (0.0010) [2023-12-27 13:47:47,839][20381] Updated weights for policy 1, policy_version 1174860 (0.0011) [2023-12-27 13:47:47,867][20347] Updated weights for policy 0, policy_version 1175852 (0.0010) [2023-12-27 13:47:47,892][20381] Updated weights for policy 1, policy_version 1174870 (0.0010) [2023-12-27 13:47:48,568][20381] Updated weights for policy 1, policy_version 1174880 (0.0009) [2023-12-27 13:47:48,578][20347] Updated weights for policy 0, policy_version 1175862 (0.0010) [2023-12-27 13:47:48,622][20381] Updated weights for policy 1, policy_version 1174890 (0.0006) [2023-12-27 13:47:48,634][20347] Updated weights for policy 0, policy_version 1175872 (0.0010) [2023-12-27 13:47:48,680][20381] Updated weights for policy 1, policy_version 1174900 (0.0006) [2023-12-27 13:47:48,687][20347] Updated weights for policy 0, policy_version 1175882 (0.0010) [2023-12-27 13:47:49,242][20381] Updated weights for policy 1, policy_version 1174910 (0.0008) [2023-12-27 13:47:49,303][20381] Updated weights for policy 1, policy_version 1174920 (0.0007) [2023-12-27 13:47:49,372][20347] Updated weights for policy 0, policy_version 1175892 (0.0008) [2023-12-27 13:47:49,372][20381] Updated weights for policy 1, policy_version 1174930 (0.0008) [2023-12-27 13:47:49,430][20347] Updated weights for policy 0, policy_version 1175902 (0.0006) [2023-12-27 13:47:49,493][20347] Updated weights for policy 0, policy_version 1175912 (0.0011) [2023-12-27 13:47:50,104][20381] Updated weights for policy 1, policy_version 1174940 (0.0008) [2023-12-27 13:47:50,145][20347] Updated weights for policy 0, policy_version 1175922 (0.0010) [2023-12-27 13:47:50,163][20381] Updated weights for policy 1, policy_version 1174950 (0.0006) [2023-12-27 13:47:50,195][20347] Updated weights for policy 0, policy_version 1175932 (0.0006) [2023-12-27 13:47:50,220][20381] Updated weights for policy 1, policy_version 1174960 (0.0006) [2023-12-27 13:47:50,247][20347] Updated weights for policy 0, policy_version 1175942 (0.0006) [2023-12-27 13:47:50,302][20347] Updated weights for policy 0, policy_version 1175952 (0.0007) [2023-12-27 13:47:50,910][20381] Updated weights for policy 1, policy_version 1174970 (0.0005) [2023-12-27 13:47:50,973][20381] Updated weights for policy 1, policy_version 1174980 (0.0006) [2023-12-27 13:47:51,027][20347] Updated weights for policy 0, policy_version 1175962 (0.0007) [2023-12-27 13:47:51,040][20381] Updated weights for policy 1, policy_version 1174990 (0.0007) [2023-12-27 13:47:51,090][20347] Updated weights for policy 0, policy_version 1175972 (0.0008) [2023-12-27 13:47:51,099][20381] Updated weights for policy 1, policy_version 1175000 (0.0007) [2023-12-27 13:47:51,157][20347] Updated weights for policy 0, policy_version 1175982 (0.0007) [2023-12-27 13:47:51,775][20381] Updated weights for policy 1, policy_version 1175010 (0.0008) [2023-12-27 13:47:51,839][20381] Updated weights for policy 1, policy_version 1175020 (0.0007) [2023-12-27 13:47:51,883][20347] Updated weights for policy 0, policy_version 1175992 (0.0007) [2023-12-27 13:47:51,896][20381] Updated weights for policy 1, policy_version 1175030 (0.0006) [2023-12-27 13:47:51,937][20347] Updated weights for policy 0, policy_version 1176002 (0.0007) [2023-12-27 13:47:51,985][20347] Updated weights for policy 0, policy_version 1176012 (0.0007) [2023-12-27 13:47:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 601948160. Throughput: 0: 10237.3, 1: 10219.5. Samples: 601918300. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:47:52,363][19170] Avg episode reward: [(0, '611.610'), (1, '706.060')] [2023-12-27 13:47:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001175032_300834816.pth... [2023-12-27 13:47:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001176016_301113344.pth... [2023-12-27 13:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001173816_300523520.pth [2023-12-27 13:47:52,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001174832_300810240.pth [2023-12-27 13:47:52,567][20381] Updated weights for policy 1, policy_version 1175040 (0.0006) [2023-12-27 13:47:52,631][20381] Updated weights for policy 1, policy_version 1175050 (0.0006) [2023-12-27 13:47:52,681][20381] Updated weights for policy 1, policy_version 1175060 (0.0007) [2023-12-27 13:47:52,759][20347] Updated weights for policy 0, policy_version 1176022 (0.0009) [2023-12-27 13:47:52,820][20347] Updated weights for policy 0, policy_version 1176032 (0.0008) [2023-12-27 13:47:52,881][20347] Updated weights for policy 0, policy_version 1176042 (0.0007) [2023-12-27 13:47:53,250][20381] Updated weights for policy 1, policy_version 1175070 (0.0008) [2023-12-27 13:47:53,303][20381] Updated weights for policy 1, policy_version 1175080 (0.0007) [2023-12-27 13:47:53,364][20381] Updated weights for policy 1, policy_version 1175090 (0.0008) [2023-12-27 13:47:53,566][20347] Updated weights for policy 0, policy_version 1176052 (0.0008) [2023-12-27 13:47:53,622][20347] Updated weights for policy 0, policy_version 1176062 (0.0007) [2023-12-27 13:47:53,675][20347] Updated weights for policy 0, policy_version 1176072 (0.0009) [2023-12-27 13:47:54,055][20381] Updated weights for policy 1, policy_version 1175100 (0.0006) [2023-12-27 13:47:54,104][20381] Updated weights for policy 1, policy_version 1175110 (0.0007) [2023-12-27 13:47:54,161][20381] Updated weights for policy 1, policy_version 1175120 (0.0008) [2023-12-27 13:47:54,347][20347] Updated weights for policy 0, policy_version 1176082 (0.0008) [2023-12-27 13:47:54,399][20347] Updated weights for policy 0, policy_version 1176092 (0.0010) [2023-12-27 13:47:54,445][20347] Updated weights for policy 0, policy_version 1176102 (0.0009) [2023-12-27 13:47:54,491][20347] Updated weights for policy 0, policy_version 1176112 (0.0007) [2023-12-27 13:47:54,864][20381] Updated weights for policy 1, policy_version 1175130 (0.0007) [2023-12-27 13:47:54,921][20381] Updated weights for policy 1, policy_version 1175140 (0.0011) [2023-12-27 13:47:54,981][20381] Updated weights for policy 1, policy_version 1175150 (0.0011) [2023-12-27 13:47:55,039][20381] Updated weights for policy 1, policy_version 1175160 (0.0011) [2023-12-27 13:47:55,190][20347] Updated weights for policy 0, policy_version 1176122 (0.0011) [2023-12-27 13:47:55,246][20347] Updated weights for policy 0, policy_version 1176132 (0.0010) [2023-12-27 13:47:55,301][20347] Updated weights for policy 0, policy_version 1176142 (0.0010) [2023-12-27 13:47:55,738][20381] Updated weights for policy 1, policy_version 1175170 (0.0010) [2023-12-27 13:47:55,792][20381] Updated weights for policy 1, policy_version 1175180 (0.0011) [2023-12-27 13:47:55,846][20381] Updated weights for policy 1, policy_version 1175190 (0.0010) [2023-12-27 13:47:55,973][20347] Updated weights for policy 0, policy_version 1176152 (0.0006) [2023-12-27 13:47:56,022][20347] Updated weights for policy 0, policy_version 1176162 (0.0005) [2023-12-27 13:47:56,088][20347] Updated weights for policy 0, policy_version 1176172 (0.0005) [2023-12-27 13:47:56,427][20381] Updated weights for policy 1, policy_version 1175200 (0.0006) [2023-12-27 13:47:56,473][20381] Updated weights for policy 1, policy_version 1175210 (0.0005) [2023-12-27 13:47:56,518][20381] Updated weights for policy 1, policy_version 1175220 (0.0006) [2023-12-27 13:47:56,706][20347] Updated weights for policy 0, policy_version 1176182 (0.0007) [2023-12-27 13:47:56,761][20347] Updated weights for policy 0, policy_version 1176192 (0.0009) [2023-12-27 13:47:56,813][20347] Updated weights for policy 0, policy_version 1176202 (0.0009) [2023-12-27 13:47:57,092][20381] Updated weights for policy 1, policy_version 1175230 (0.0007) [2023-12-27 13:47:57,145][20381] Updated weights for policy 1, policy_version 1175240 (0.0007) [2023-12-27 13:47:57,195][20381] Updated weights for policy 1, policy_version 1175250 (0.0007) [2023-12-27 13:47:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 602054656. Throughput: 0: 10224.1, 1: 10265.7. Samples: 602041112. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:47:57,363][19170] Avg episode reward: [(0, '267.810'), (1, '735.400')] [2023-12-27 13:47:57,440][20347] Updated weights for policy 0, policy_version 1176212 (0.0008) [2023-12-27 13:47:57,502][20347] Updated weights for policy 0, policy_version 1176222 (0.0005) [2023-12-27 13:47:57,560][20347] Updated weights for policy 0, policy_version 1176232 (0.0006) [2023-12-27 13:47:57,826][20381] Updated weights for policy 1, policy_version 1175260 (0.0007) [2023-12-27 13:47:57,877][20381] Updated weights for policy 1, policy_version 1175270 (0.0007) [2023-12-27 13:47:57,922][20381] Updated weights for policy 1, policy_version 1175280 (0.0007) [2023-12-27 13:47:58,154][20347] Updated weights for policy 0, policy_version 1176242 (0.0009) [2023-12-27 13:47:58,216][20347] Updated weights for policy 0, policy_version 1176252 (0.0008) [2023-12-27 13:47:58,270][20347] Updated weights for policy 0, policy_version 1176262 (0.0008) [2023-12-27 13:47:58,317][20347] Updated weights for policy 0, policy_version 1176272 (0.0008) [2023-12-27 13:47:58,680][20381] Updated weights for policy 1, policy_version 1175290 (0.0007) [2023-12-27 13:47:58,746][20381] Updated weights for policy 1, policy_version 1175300 (0.0008) [2023-12-27 13:47:58,814][20381] Updated weights for policy 1, policy_version 1175310 (0.0008) [2023-12-27 13:47:58,881][20381] Updated weights for policy 1, policy_version 1175320 (0.0007) [2023-12-27 13:47:59,153][20347] Updated weights for policy 0, policy_version 1176282 (0.0008) [2023-12-27 13:47:59,209][20347] Updated weights for policy 0, policy_version 1176292 (0.0008) [2023-12-27 13:47:59,281][20347] Updated weights for policy 0, policy_version 1176302 (0.0009) [2023-12-27 13:47:59,622][20381] Updated weights for policy 1, policy_version 1175330 (0.0008) [2023-12-27 13:47:59,691][20381] Updated weights for policy 1, policy_version 1175340 (0.0005) [2023-12-27 13:47:59,760][20381] Updated weights for policy 1, policy_version 1175350 (0.0006) [2023-12-27 13:47:59,955][20347] Updated weights for policy 0, policy_version 1176312 (0.0008) [2023-12-27 13:48:00,015][20347] Updated weights for policy 0, policy_version 1176322 (0.0007) [2023-12-27 13:48:00,079][20347] Updated weights for policy 0, policy_version 1176332 (0.0007) [2023-12-27 13:48:00,450][20381] Updated weights for policy 1, policy_version 1175360 (0.0007) [2023-12-27 13:48:00,513][20381] Updated weights for policy 1, policy_version 1175370 (0.0007) [2023-12-27 13:48:00,569][20381] Updated weights for policy 1, policy_version 1175380 (0.0009) [2023-12-27 13:48:00,746][20347] Updated weights for policy 0, policy_version 1176342 (0.0008) [2023-12-27 13:48:00,812][20347] Updated weights for policy 0, policy_version 1176352 (0.0008) [2023-12-27 13:48:00,863][20347] Updated weights for policy 0, policy_version 1176362 (0.0008) [2023-12-27 13:48:01,227][20381] Updated weights for policy 1, policy_version 1175390 (0.0007) [2023-12-27 13:48:01,290][20381] Updated weights for policy 1, policy_version 1175400 (0.0010) [2023-12-27 13:48:01,345][20381] Updated weights for policy 1, policy_version 1175410 (0.0010) [2023-12-27 13:48:01,607][20347] Updated weights for policy 0, policy_version 1176372 (0.0008) [2023-12-27 13:48:01,670][20347] Updated weights for policy 0, policy_version 1176382 (0.0007) [2023-12-27 13:48:01,731][20347] Updated weights for policy 0, policy_version 1176392 (0.0007) [2023-12-27 13:48:02,037][20381] Updated weights for policy 1, policy_version 1175420 (0.0009) [2023-12-27 13:48:02,101][20381] Updated weights for policy 1, policy_version 1175430 (0.0008) [2023-12-27 13:48:02,158][20381] Updated weights for policy 1, policy_version 1175440 (0.0008) [2023-12-27 13:48:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 602152960. Throughput: 0: 10179.4, 1: 10310.5. Samples: 602162672. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:02,363][19170] Avg episode reward: [(0, '449.310'), (1, '676.840')] [2023-12-27 13:48:02,410][20347] Updated weights for policy 0, policy_version 1176402 (0.0007) [2023-12-27 13:48:02,476][20347] Updated weights for policy 0, policy_version 1176412 (0.0006) [2023-12-27 13:48:02,534][20347] Updated weights for policy 0, policy_version 1176422 (0.0005) [2023-12-27 13:48:02,584][20347] Updated weights for policy 0, policy_version 1176432 (0.0005) [2023-12-27 13:48:02,772][20381] Updated weights for policy 1, policy_version 1175450 (0.0007) [2023-12-27 13:48:02,836][20381] Updated weights for policy 1, policy_version 1175460 (0.0005) [2023-12-27 13:48:02,894][20381] Updated weights for policy 1, policy_version 1175470 (0.0007) [2023-12-27 13:48:02,953][20381] Updated weights for policy 1, policy_version 1175480 (0.0008) [2023-12-27 13:48:03,194][20347] Updated weights for policy 0, policy_version 1176442 (0.0005) [2023-12-27 13:48:03,238][20347] Updated weights for policy 0, policy_version 1176452 (0.0005) [2023-12-27 13:48:03,284][20347] Updated weights for policy 0, policy_version 1176462 (0.0005) [2023-12-27 13:48:03,612][20381] Updated weights for policy 1, policy_version 1175490 (0.0008) [2023-12-27 13:48:03,674][20381] Updated weights for policy 1, policy_version 1175500 (0.0008) [2023-12-27 13:48:03,729][20381] Updated weights for policy 1, policy_version 1175510 (0.0008) [2023-12-27 13:48:03,951][20347] Updated weights for policy 0, policy_version 1176472 (0.0010) [2023-12-27 13:48:04,008][20347] Updated weights for policy 0, policy_version 1176482 (0.0011) [2023-12-27 13:48:04,066][20347] Updated weights for policy 0, policy_version 1176492 (0.0010) [2023-12-27 13:48:04,444][20381] Updated weights for policy 1, policy_version 1175520 (0.0007) [2023-12-27 13:48:04,503][20381] Updated weights for policy 1, policy_version 1175530 (0.0007) [2023-12-27 13:48:04,562][20381] Updated weights for policy 1, policy_version 1175540 (0.0008) [2023-12-27 13:48:04,754][20347] Updated weights for policy 0, policy_version 1176502 (0.0010) [2023-12-27 13:48:04,799][20347] Updated weights for policy 0, policy_version 1176512 (0.0010) [2023-12-27 13:48:04,843][20347] Updated weights for policy 0, policy_version 1176522 (0.0010) [2023-12-27 13:48:05,220][20381] Updated weights for policy 1, policy_version 1175550 (0.0008) [2023-12-27 13:48:05,283][20381] Updated weights for policy 1, policy_version 1175560 (0.0008) [2023-12-27 13:48:05,344][20381] Updated weights for policy 1, policy_version 1175570 (0.0009) [2023-12-27 13:48:05,433][20347] Updated weights for policy 0, policy_version 1176532 (0.0006) [2023-12-27 13:48:05,484][20347] Updated weights for policy 0, policy_version 1176542 (0.0010) [2023-12-27 13:48:05,537][20347] Updated weights for policy 0, policy_version 1176552 (0.0010) [2023-12-27 13:48:06,037][20381] Updated weights for policy 1, policy_version 1175580 (0.0008) [2023-12-27 13:48:06,086][20381] Updated weights for policy 1, policy_version 1175590 (0.0008) [2023-12-27 13:48:06,164][20381] Updated weights for policy 1, policy_version 1175600 (0.0007) [2023-12-27 13:48:06,220][20347] Updated weights for policy 0, policy_version 1176562 (0.0011) [2023-12-27 13:48:06,284][20347] Updated weights for policy 0, policy_version 1176572 (0.0011) [2023-12-27 13:48:06,348][20347] Updated weights for policy 0, policy_version 1176582 (0.0009) [2023-12-27 13:48:06,415][20347] Updated weights for policy 0, policy_version 1176592 (0.0009) [2023-12-27 13:48:06,925][20381] Updated weights for policy 1, policy_version 1175610 (0.0007) [2023-12-27 13:48:06,981][20381] Updated weights for policy 1, policy_version 1175620 (0.0007) [2023-12-27 13:48:07,023][20347] Updated weights for policy 0, policy_version 1176602 (0.0007) [2023-12-27 13:48:07,037][20381] Updated weights for policy 1, policy_version 1175630 (0.0005) [2023-12-27 13:48:07,082][20347] Updated weights for policy 0, policy_version 1176612 (0.0006) [2023-12-27 13:48:07,095][20381] Updated weights for policy 1, policy_version 1175640 (0.0006) [2023-12-27 13:48:07,139][20347] Updated weights for policy 0, policy_version 1176622 (0.0005) [2023-12-27 13:48:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 602259456. Throughput: 0: 10180.4, 1: 10341.6. Samples: 602225592. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:07,363][19170] Avg episode reward: [(0, '536.310'), (1, '696.380')] [2023-12-27 13:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001175640_300990464.pth... [2023-12-27 13:48:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001176624_301268992.pth... [2023-12-27 13:48:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001174424_300679168.pth [2023-12-27 13:48:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001175408_300957696.pth [2023-12-27 13:48:07,738][20381] Updated weights for policy 1, policy_version 1175650 (0.0007) [2023-12-27 13:48:07,795][20381] Updated weights for policy 1, policy_version 1175660 (0.0008) [2023-12-27 13:48:07,824][20347] Updated weights for policy 0, policy_version 1176632 (0.0009) [2023-12-27 13:48:07,849][20381] Updated weights for policy 1, policy_version 1175670 (0.0008) [2023-12-27 13:48:07,882][20347] Updated weights for policy 0, policy_version 1176642 (0.0010) [2023-12-27 13:48:07,939][20347] Updated weights for policy 0, policy_version 1176652 (0.0010) [2023-12-27 13:48:08,557][20381] Updated weights for policy 1, policy_version 1175680 (0.0007) [2023-12-27 13:48:08,613][20381] Updated weights for policy 1, policy_version 1175690 (0.0005) [2023-12-27 13:48:08,653][20347] Updated weights for policy 0, policy_version 1176662 (0.0011) [2023-12-27 13:48:08,678][20381] Updated weights for policy 1, policy_version 1175700 (0.0005) [2023-12-27 13:48:08,709][20347] Updated weights for policy 0, policy_version 1176672 (0.0010) [2023-12-27 13:48:08,764][20347] Updated weights for policy 0, policy_version 1176682 (0.0010) [2023-12-27 13:48:09,380][20347] Updated weights for policy 0, policy_version 1176692 (0.0009) [2023-12-27 13:48:09,393][20381] Updated weights for policy 1, policy_version 1175710 (0.0006) [2023-12-27 13:48:09,448][20347] Updated weights for policy 0, policy_version 1176702 (0.0010) [2023-12-27 13:48:09,463][20381] Updated weights for policy 1, policy_version 1175720 (0.0006) [2023-12-27 13:48:09,505][20347] Updated weights for policy 0, policy_version 1176712 (0.0009) [2023-12-27 13:48:09,531][20381] Updated weights for policy 1, policy_version 1175730 (0.0006) [2023-12-27 13:48:10,132][20381] Updated weights for policy 1, policy_version 1175740 (0.0006) [2023-12-27 13:48:10,186][20347] Updated weights for policy 0, policy_version 1176722 (0.0007) [2023-12-27 13:48:10,194][20381] Updated weights for policy 1, policy_version 1175750 (0.0009) [2023-12-27 13:48:10,248][20347] Updated weights for policy 0, policy_version 1176732 (0.0010) [2023-12-27 13:48:10,253][20381] Updated weights for policy 1, policy_version 1175760 (0.0007) [2023-12-27 13:48:10,308][20347] Updated weights for policy 0, policy_version 1176742 (0.0010) [2023-12-27 13:48:10,366][20347] Updated weights for policy 0, policy_version 1176752 (0.0009) [2023-12-27 13:48:10,937][20381] Updated weights for policy 1, policy_version 1175770 (0.0006) [2023-12-27 13:48:11,001][20347] Updated weights for policy 0, policy_version 1176762 (0.0007) [2023-12-27 13:48:11,004][20381] Updated weights for policy 1, policy_version 1175780 (0.0006) [2023-12-27 13:48:11,068][20381] Updated weights for policy 1, policy_version 1175790 (0.0009) [2023-12-27 13:48:11,070][20347] Updated weights for policy 0, policy_version 1176773 (0.0007) [2023-12-27 13:48:11,127][20381] Updated weights for policy 1, policy_version 1175800 (0.0011) [2023-12-27 13:48:11,138][20347] Updated weights for policy 0, policy_version 1176783 (0.0007) [2023-12-27 13:48:11,864][20347] Updated weights for policy 0, policy_version 1176793 (0.0006) [2023-12-27 13:48:11,892][20381] Updated weights for policy 1, policy_version 1175810 (0.0007) [2023-12-27 13:48:11,921][20347] Updated weights for policy 0, policy_version 1176803 (0.0006) [2023-12-27 13:48:11,945][20381] Updated weights for policy 1, policy_version 1175820 (0.0005) [2023-12-27 13:48:11,981][20347] Updated weights for policy 0, policy_version 1176813 (0.0006) [2023-12-27 13:48:12,002][20381] Updated weights for policy 1, policy_version 1175830 (0.0007) [2023-12-27 13:48:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 602357760. Throughput: 0: 10246.5, 1: 10303.5. Samples: 602348096. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:12,363][19170] Avg episode reward: [(0, '316.560'), (1, '764.860')] [2023-12-27 13:48:12,683][20347] Updated weights for policy 0, policy_version 1176823 (0.0007) [2023-12-27 13:48:12,737][20381] Updated weights for policy 1, policy_version 1175840 (0.0010) [2023-12-27 13:48:12,749][20347] Updated weights for policy 0, policy_version 1176833 (0.0008) [2023-12-27 13:48:12,794][20381] Updated weights for policy 1, policy_version 1175850 (0.0011) [2023-12-27 13:48:12,815][20347] Updated weights for policy 0, policy_version 1176843 (0.0008) [2023-12-27 13:48:12,845][20381] Updated weights for policy 1, policy_version 1175860 (0.0011) [2023-12-27 13:48:13,386][20347] Updated weights for policy 0, policy_version 1176853 (0.0006) [2023-12-27 13:48:13,439][20347] Updated weights for policy 0, policy_version 1176863 (0.0008) [2023-12-27 13:48:13,503][20347] Updated weights for policy 0, policy_version 1176873 (0.0008) [2023-12-27 13:48:13,503][20381] Updated weights for policy 1, policy_version 1175870 (0.0010) [2023-12-27 13:48:13,552][20381] Updated weights for policy 1, policy_version 1175880 (0.0010) [2023-12-27 13:48:13,596][20381] Updated weights for policy 1, policy_version 1175890 (0.0010) [2023-12-27 13:48:14,164][20347] Updated weights for policy 0, policy_version 1176883 (0.0006) [2023-12-27 13:48:14,217][20347] Updated weights for policy 0, policy_version 1176893 (0.0005) [2023-12-27 13:48:14,268][20381] Updated weights for policy 1, policy_version 1175900 (0.0011) [2023-12-27 13:48:14,277][20347] Updated weights for policy 0, policy_version 1176903 (0.0005) [2023-12-27 13:48:14,342][20381] Updated weights for policy 1, policy_version 1175910 (0.0010) [2023-12-27 13:48:14,400][20381] Updated weights for policy 1, policy_version 1175920 (0.0010) [2023-12-27 13:48:14,974][20347] Updated weights for policy 0, policy_version 1176913 (0.0005) [2023-12-27 13:48:15,040][20347] Updated weights for policy 0, policy_version 1176923 (0.0007) [2023-12-27 13:48:15,098][20347] Updated weights for policy 0, policy_version 1176933 (0.0008) [2023-12-27 13:48:15,118][20381] Updated weights for policy 1, policy_version 1175930 (0.0010) [2023-12-27 13:48:15,154][20347] Updated weights for policy 0, policy_version 1176943 (0.0008) [2023-12-27 13:48:15,173][20381] Updated weights for policy 1, policy_version 1175940 (0.0008) [2023-12-27 13:48:15,233][20381] Updated weights for policy 1, policy_version 1175950 (0.0008) [2023-12-27 13:48:15,292][20381] Updated weights for policy 1, policy_version 1175960 (0.0008) [2023-12-27 13:48:15,777][20347] Updated weights for policy 0, policy_version 1176953 (0.0010) [2023-12-27 13:48:15,822][20347] Updated weights for policy 0, policy_version 1176963 (0.0010) [2023-12-27 13:48:15,869][20347] Updated weights for policy 0, policy_version 1176973 (0.0010) [2023-12-27 13:48:15,935][20381] Updated weights for policy 1, policy_version 1175970 (0.0007) [2023-12-27 13:48:15,993][20381] Updated weights for policy 1, policy_version 1175980 (0.0008) [2023-12-27 13:48:16,056][20381] Updated weights for policy 1, policy_version 1175990 (0.0008) [2023-12-27 13:48:16,535][20347] Updated weights for policy 0, policy_version 1176983 (0.0008) [2023-12-27 13:48:16,591][20347] Updated weights for policy 0, policy_version 1176993 (0.0007) [2023-12-27 13:48:16,652][20347] Updated weights for policy 0, policy_version 1177003 (0.0008) [2023-12-27 13:48:16,692][20381] Updated weights for policy 1, policy_version 1176000 (0.0009) [2023-12-27 13:48:16,748][20381] Updated weights for policy 1, policy_version 1176010 (0.0008) [2023-12-27 13:48:16,799][20381] Updated weights for policy 1, policy_version 1176020 (0.0008) [2023-12-27 13:48:17,306][20347] Updated weights for policy 0, policy_version 1177013 (0.0007) [2023-12-27 13:48:17,350][20347] Updated weights for policy 0, policy_version 1177023 (0.0009) [2023-12-27 13:48:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 602456064. Throughput: 0: 10277.8, 1: 10275.6. Samples: 602470800. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:17,363][19170] Avg episode reward: [(0, '605.610'), (1, '666.890')] [2023-12-27 13:48:17,394][20347] Updated weights for policy 0, policy_version 1177033 (0.0007) [2023-12-27 13:48:17,471][20381] Updated weights for policy 1, policy_version 1176030 (0.0008) [2023-12-27 13:48:17,524][20381] Updated weights for policy 1, policy_version 1176040 (0.0008) [2023-12-27 13:48:17,588][20381] Updated weights for policy 1, policy_version 1176050 (0.0008) [2023-12-27 13:48:18,097][20347] Updated weights for policy 0, policy_version 1177043 (0.0009) [2023-12-27 13:48:18,149][20347] Updated weights for policy 0, policy_version 1177053 (0.0010) [2023-12-27 13:48:18,191][20381] Updated weights for policy 1, policy_version 1176060 (0.0010) [2023-12-27 13:48:18,204][20347] Updated weights for policy 0, policy_version 1177063 (0.0006) [2023-12-27 13:48:18,240][20381] Updated weights for policy 1, policy_version 1176070 (0.0006) [2023-12-27 13:48:18,287][20381] Updated weights for policy 1, policy_version 1176080 (0.0005) [2023-12-27 13:48:18,925][20347] Updated weights for policy 0, policy_version 1177073 (0.0008) [2023-12-27 13:48:18,946][20381] Updated weights for policy 1, policy_version 1176090 (0.0007) [2023-12-27 13:48:18,984][20347] Updated weights for policy 0, policy_version 1177083 (0.0008) [2023-12-27 13:48:18,994][20381] Updated weights for policy 1, policy_version 1176100 (0.0007) [2023-12-27 13:48:19,040][20381] Updated weights for policy 1, policy_version 1176110 (0.0005) [2023-12-27 13:48:19,041][20347] Updated weights for policy 0, policy_version 1177093 (0.0006) [2023-12-27 13:48:19,097][20381] Updated weights for policy 1, policy_version 1176120 (0.0005) [2023-12-27 13:48:19,099][20347] Updated weights for policy 0, policy_version 1177103 (0.0007) [2023-12-27 13:48:19,823][20381] Updated weights for policy 1, policy_version 1176130 (0.0007) [2023-12-27 13:48:19,892][20381] Updated weights for policy 1, policy_version 1176140 (0.0007) [2023-12-27 13:48:19,901][20347] Updated weights for policy 0, policy_version 1177113 (0.0006) [2023-12-27 13:48:19,957][20381] Updated weights for policy 1, policy_version 1176150 (0.0007) [2023-12-27 13:48:19,964][20347] Updated weights for policy 0, policy_version 1177123 (0.0008) [2023-12-27 13:48:20,021][20347] Updated weights for policy 0, policy_version 1177133 (0.0007) [2023-12-27 13:48:20,636][20381] Updated weights for policy 1, policy_version 1176160 (0.0007) [2023-12-27 13:48:20,684][20347] Updated weights for policy 0, policy_version 1177143 (0.0006) [2023-12-27 13:48:20,688][20381] Updated weights for policy 1, policy_version 1176170 (0.0006) [2023-12-27 13:48:20,748][20347] Updated weights for policy 0, policy_version 1177153 (0.0006) [2023-12-27 13:48:20,750][20381] Updated weights for policy 1, policy_version 1176180 (0.0008) [2023-12-27 13:48:20,814][20347] Updated weights for policy 0, policy_version 1177163 (0.0006) [2023-12-27 13:48:21,454][20347] Updated weights for policy 0, policy_version 1177173 (0.0006) [2023-12-27 13:48:21,487][20291] KL-divergence is very high: 123.0503 [2023-12-27 13:48:21,510][20347] Updated weights for policy 0, policy_version 1177183 (0.0006) [2023-12-27 13:48:21,512][20381] Updated weights for policy 1, policy_version 1176190 (0.0008) [2023-12-27 13:48:21,569][20347] Updated weights for policy 0, policy_version 1177193 (0.0007) [2023-12-27 13:48:21,571][20381] Updated weights for policy 1, policy_version 1176200 (0.0008) [2023-12-27 13:48:21,635][20381] Updated weights for policy 1, policy_version 1176210 (0.0007) [2023-12-27 13:48:22,211][20347] Updated weights for policy 0, policy_version 1177203 (0.0007) [2023-12-27 13:48:22,272][20347] Updated weights for policy 0, policy_version 1177213 (0.0008) [2023-12-27 13:48:22,336][20347] Updated weights for policy 0, policy_version 1177223 (0.0007) [2023-12-27 13:48:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 602554368. Throughput: 0: 10238.7, 1: 10230.3. Samples: 602530792. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:22,363][19170] Avg episode reward: [(0, '609.730'), (1, '666.920')] [2023-12-27 13:48:22,367][20381] Updated weights for policy 1, policy_version 1176220 (0.0008) [2023-12-27 13:48:22,391][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001177232_301424640.pth... [2023-12-27 13:48:22,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001176016_301113344.pth [2023-12-27 13:48:22,424][20381] Updated weights for policy 1, policy_version 1176230 (0.0008) [2023-12-27 13:48:22,481][20381] Updated weights for policy 1, policy_version 1176240 (0.0008) [2023-12-27 13:48:22,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001176248_301146112.pth... [2023-12-27 13:48:22,526][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001175032_300834816.pth [2023-12-27 13:48:23,049][20347] Updated weights for policy 0, policy_version 1177233 (0.0008) [2023-12-27 13:48:23,104][20347] Updated weights for policy 0, policy_version 1177243 (0.0008) [2023-12-27 13:48:23,161][20347] Updated weights for policy 0, policy_version 1177253 (0.0010) [2023-12-27 13:48:23,195][20381] Updated weights for policy 1, policy_version 1176250 (0.0008) [2023-12-27 13:48:23,213][20347] Updated weights for policy 0, policy_version 1177263 (0.0010) [2023-12-27 13:48:23,254][20381] Updated weights for policy 1, policy_version 1176260 (0.0007) [2023-12-27 13:48:23,313][20381] Updated weights for policy 1, policy_version 1176270 (0.0007) [2023-12-27 13:48:23,362][20381] Updated weights for policy 1, policy_version 1176280 (0.0008) [2023-12-27 13:48:23,856][20347] Updated weights for policy 0, policy_version 1177273 (0.0007) [2023-12-27 13:48:23,914][20347] Updated weights for policy 0, policy_version 1177283 (0.0008) [2023-12-27 13:48:23,966][20347] Updated weights for policy 0, policy_version 1177293 (0.0006) [2023-12-27 13:48:24,082][20381] Updated weights for policy 1, policy_version 1176290 (0.0008) [2023-12-27 13:48:24,144][20381] Updated weights for policy 1, policy_version 1176300 (0.0008) [2023-12-27 13:48:24,200][20381] Updated weights for policy 1, policy_version 1176310 (0.0008) [2023-12-27 13:48:24,663][20347] Updated weights for policy 0, policy_version 1177303 (0.0008) [2023-12-27 13:48:24,713][20347] Updated weights for policy 0, policy_version 1177313 (0.0007) [2023-12-27 13:48:24,765][20347] Updated weights for policy 0, policy_version 1177323 (0.0006) [2023-12-27 13:48:24,841][20381] Updated weights for policy 1, policy_version 1176320 (0.0007) [2023-12-27 13:48:24,893][20381] Updated weights for policy 1, policy_version 1176330 (0.0006) [2023-12-27 13:48:24,949][20381] Updated weights for policy 1, policy_version 1176340 (0.0005) [2023-12-27 13:48:25,453][20347] Updated weights for policy 0, policy_version 1177333 (0.0008) [2023-12-27 13:48:25,504][20381] Updated weights for policy 1, policy_version 1176350 (0.0007) [2023-12-27 13:48:25,515][20347] Updated weights for policy 0, policy_version 1177343 (0.0008) [2023-12-27 13:48:25,563][20381] Updated weights for policy 1, policy_version 1176360 (0.0006) [2023-12-27 13:48:25,576][20347] Updated weights for policy 0, policy_version 1177353 (0.0006) [2023-12-27 13:48:25,621][20381] Updated weights for policy 1, policy_version 1176370 (0.0006) [2023-12-27 13:48:26,174][20381] Updated weights for policy 1, policy_version 1176380 (0.0008) [2023-12-27 13:48:26,231][20347] Updated weights for policy 0, policy_version 1177363 (0.0007) [2023-12-27 13:48:26,231][20381] Updated weights for policy 1, policy_version 1176390 (0.0011) [2023-12-27 13:48:26,284][20347] Updated weights for policy 0, policy_version 1177373 (0.0009) [2023-12-27 13:48:26,285][20381] Updated weights for policy 1, policy_version 1176400 (0.0011) [2023-12-27 13:48:26,343][20347] Updated weights for policy 0, policy_version 1177383 (0.0007) [2023-12-27 13:48:26,927][20381] Updated weights for policy 1, policy_version 1176410 (0.0011) [2023-12-27 13:48:26,961][20347] Updated weights for policy 0, policy_version 1177393 (0.0007) [2023-12-27 13:48:26,976][20381] Updated weights for policy 1, policy_version 1176420 (0.0010) [2023-12-27 13:48:27,012][20347] Updated weights for policy 0, policy_version 1177403 (0.0005) [2023-12-27 13:48:27,025][20381] Updated weights for policy 1, policy_version 1176430 (0.0010) [2023-12-27 13:48:27,067][20347] Updated weights for policy 0, policy_version 1177413 (0.0005) [2023-12-27 13:48:27,072][20381] Updated weights for policy 1, policy_version 1176440 (0.0010) [2023-12-27 13:48:27,115][20347] Updated weights for policy 0, policy_version 1177423 (0.0006) [2023-12-27 13:48:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 602669056. Throughput: 0: 10297.6, 1: 10252.6. Samples: 602654936. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:27,363][19170] Avg episode reward: [(0, '504.680'), (1, '764.740')] [2023-12-27 13:48:27,749][20347] Updated weights for policy 0, policy_version 1177433 (0.0007) [2023-12-27 13:48:27,755][20381] Updated weights for policy 1, policy_version 1176450 (0.0010) [2023-12-27 13:48:27,805][20347] Updated weights for policy 0, policy_version 1177443 (0.0005) [2023-12-27 13:48:27,809][20381] Updated weights for policy 1, policy_version 1176460 (0.0010) [2023-12-27 13:48:27,854][20347] Updated weights for policy 0, policy_version 1177453 (0.0005) [2023-12-27 13:48:27,855][20381] Updated weights for policy 1, policy_version 1176470 (0.0010) [2023-12-27 13:48:28,416][20347] Updated weights for policy 0, policy_version 1177463 (0.0007) [2023-12-27 13:48:28,480][20347] Updated weights for policy 0, policy_version 1177473 (0.0007) [2023-12-27 13:48:28,541][20347] Updated weights for policy 0, policy_version 1177483 (0.0005) [2023-12-27 13:48:28,562][20381] Updated weights for policy 1, policy_version 1176480 (0.0011) [2023-12-27 13:48:28,621][20381] Updated weights for policy 1, policy_version 1176490 (0.0011) [2023-12-27 13:48:28,680][20381] Updated weights for policy 1, policy_version 1176500 (0.0011) [2023-12-27 13:48:29,122][20347] Updated weights for policy 0, policy_version 1177493 (0.0008) [2023-12-27 13:48:29,166][20347] Updated weights for policy 0, policy_version 1177503 (0.0008) [2023-12-27 13:48:29,214][20347] Updated weights for policy 0, policy_version 1177513 (0.0008) [2023-12-27 13:48:29,239][20381] Updated weights for policy 1, policy_version 1176510 (0.0009) [2023-12-27 13:48:29,298][20381] Updated weights for policy 1, policy_version 1176520 (0.0008) [2023-12-27 13:48:29,359][20381] Updated weights for policy 1, policy_version 1176530 (0.0008) [2023-12-27 13:48:29,970][20347] Updated weights for policy 0, policy_version 1177523 (0.0008) [2023-12-27 13:48:30,026][20347] Updated weights for policy 0, policy_version 1177533 (0.0008) [2023-12-27 13:48:30,028][20381] Updated weights for policy 1, policy_version 1176540 (0.0007) [2023-12-27 13:48:30,082][20347] Updated weights for policy 0, policy_version 1177543 (0.0008) [2023-12-27 13:48:30,085][20381] Updated weights for policy 1, policy_version 1176550 (0.0006) [2023-12-27 13:48:30,144][20381] Updated weights for policy 1, policy_version 1176560 (0.0007) [2023-12-27 13:48:30,777][20347] Updated weights for policy 0, policy_version 1177553 (0.0007) [2023-12-27 13:48:30,838][20381] Updated weights for policy 1, policy_version 1176570 (0.0007) [2023-12-27 13:48:30,840][20347] Updated weights for policy 0, policy_version 1177563 (0.0006) [2023-12-27 13:48:30,896][20381] Updated weights for policy 1, policy_version 1176580 (0.0006) [2023-12-27 13:48:30,899][20347] Updated weights for policy 0, policy_version 1177573 (0.0005) [2023-12-27 13:48:30,954][20381] Updated weights for policy 1, policy_version 1176590 (0.0006) [2023-12-27 13:48:30,963][20347] Updated weights for policy 0, policy_version 1177583 (0.0006) [2023-12-27 13:48:31,021][20381] Updated weights for policy 1, policy_version 1176600 (0.0007) [2023-12-27 13:48:31,677][20347] Updated weights for policy 0, policy_version 1177593 (0.0009) [2023-12-27 13:48:31,740][20347] Updated weights for policy 0, policy_version 1177603 (0.0008) [2023-12-27 13:48:31,774][20381] Updated weights for policy 1, policy_version 1176610 (0.0007) [2023-12-27 13:48:31,806][20347] Updated weights for policy 0, policy_version 1177613 (0.0006) [2023-12-27 13:48:31,833][20381] Updated weights for policy 1, policy_version 1176620 (0.0007) [2023-12-27 13:48:31,879][20381] Updated weights for policy 1, policy_version 1176630 (0.0008) [2023-12-27 13:48:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 602767360. Throughput: 0: 10280.2, 1: 10279.5. Samples: 602780552. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:32,363][19170] Avg episode reward: [(0, '380.480'), (1, '813.770')] [2023-12-27 13:48:32,481][20347] Updated weights for policy 0, policy_version 1177623 (0.0007) [2023-12-27 13:48:32,534][20347] Updated weights for policy 0, policy_version 1177633 (0.0007) [2023-12-27 13:48:32,574][20381] Updated weights for policy 1, policy_version 1176640 (0.0009) [2023-12-27 13:48:32,584][20347] Updated weights for policy 0, policy_version 1177643 (0.0007) [2023-12-27 13:48:32,628][20381] Updated weights for policy 1, policy_version 1176650 (0.0006) [2023-12-27 13:48:32,683][20381] Updated weights for policy 1, policy_version 1176660 (0.0008) [2023-12-27 13:48:33,312][20381] Updated weights for policy 1, policy_version 1176670 (0.0006) [2023-12-27 13:48:33,315][20347] Updated weights for policy 0, policy_version 1177653 (0.0007) [2023-12-27 13:48:33,358][20381] Updated weights for policy 1, policy_version 1176680 (0.0005) [2023-12-27 13:48:33,373][20347] Updated weights for policy 0, policy_version 1177663 (0.0008) [2023-12-27 13:48:33,406][20381] Updated weights for policy 1, policy_version 1176690 (0.0007) [2023-12-27 13:48:33,426][20347] Updated weights for policy 0, policy_version 1177673 (0.0006) [2023-12-27 13:48:34,041][20381] Updated weights for policy 1, policy_version 1176700 (0.0007) [2023-12-27 13:48:34,102][20381] Updated weights for policy 1, policy_version 1176710 (0.0010) [2023-12-27 13:48:34,125][20347] Updated weights for policy 0, policy_version 1177683 (0.0008) [2023-12-27 13:48:34,163][20381] Updated weights for policy 1, policy_version 1176720 (0.0012) [2023-12-27 13:48:34,194][20347] Updated weights for policy 0, policy_version 1177693 (0.0006) [2023-12-27 13:48:34,252][20347] Updated weights for policy 0, policy_version 1177703 (0.0007) [2023-12-27 13:48:34,868][20381] Updated weights for policy 1, policy_version 1176730 (0.0008) [2023-12-27 13:48:34,917][20381] Updated weights for policy 1, policy_version 1176740 (0.0008) [2023-12-27 13:48:34,922][20347] Updated weights for policy 0, policy_version 1177713 (0.0007) [2023-12-27 13:48:34,970][20381] Updated weights for policy 1, policy_version 1176750 (0.0006) [2023-12-27 13:48:34,972][20347] Updated weights for policy 0, policy_version 1177723 (0.0007) [2023-12-27 13:48:35,023][20347] Updated weights for policy 0, policy_version 1177733 (0.0008) [2023-12-27 13:48:35,023][20381] Updated weights for policy 1, policy_version 1176760 (0.0009) [2023-12-27 13:48:35,070][20347] Updated weights for policy 0, policy_version 1177743 (0.0008) [2023-12-27 13:48:35,718][20381] Updated weights for policy 1, policy_version 1176770 (0.0007) [2023-12-27 13:48:35,727][20347] Updated weights for policy 0, policy_version 1177753 (0.0006) [2023-12-27 13:48:35,770][20381] Updated weights for policy 1, policy_version 1176780 (0.0005) [2023-12-27 13:48:35,784][20347] Updated weights for policy 0, policy_version 1177763 (0.0006) [2023-12-27 13:48:35,829][20381] Updated weights for policy 1, policy_version 1176790 (0.0005) [2023-12-27 13:48:35,840][20347] Updated weights for policy 0, policy_version 1177773 (0.0006) [2023-12-27 13:48:36,468][20347] Updated weights for policy 0, policy_version 1177783 (0.0007) [2023-12-27 13:48:36,501][20381] Updated weights for policy 1, policy_version 1176800 (0.0006) [2023-12-27 13:48:36,529][20347] Updated weights for policy 0, policy_version 1177793 (0.0008) [2023-12-27 13:48:36,558][20381] Updated weights for policy 1, policy_version 1176810 (0.0008) [2023-12-27 13:48:36,590][20347] Updated weights for policy 0, policy_version 1177803 (0.0008) [2023-12-27 13:48:36,616][20381] Updated weights for policy 1, policy_version 1176820 (0.0006) [2023-12-27 13:48:37,275][20381] Updated weights for policy 1, policy_version 1176830 (0.0008) [2023-12-27 13:48:37,323][20381] Updated weights for policy 1, policy_version 1176840 (0.0007) [2023-12-27 13:48:37,333][20347] Updated weights for policy 0, policy_version 1177813 (0.0007) [2023-12-27 13:48:37,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 602865664. Throughput: 0: 10253.1, 1: 10263.4. Samples: 602841540. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:37,363][19170] Avg episode reward: [(0, '452.200'), (1, '754.880')] [2023-12-27 13:48:37,373][20381] Updated weights for policy 1, policy_version 1176850 (0.0006) [2023-12-27 13:48:37,387][20347] Updated weights for policy 0, policy_version 1177823 (0.0007) [2023-12-27 13:48:37,403][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001176856_301301760.pth... [2023-12-27 13:48:37,408][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001175640_300990464.pth [2023-12-27 13:48:37,436][20347] Updated weights for policy 0, policy_version 1177833 (0.0007) [2023-12-27 13:48:37,469][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001177840_301580288.pth... [2023-12-27 13:48:37,472][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001176624_301268992.pth [2023-12-27 13:48:38,046][20381] Updated weights for policy 1, policy_version 1176860 (0.0006) [2023-12-27 13:48:38,104][20381] Updated weights for policy 1, policy_version 1176870 (0.0007) [2023-12-27 13:48:38,155][20381] Updated weights for policy 1, policy_version 1176880 (0.0007) [2023-12-27 13:48:38,157][20347] Updated weights for policy 0, policy_version 1177843 (0.0007) [2023-12-27 13:48:38,214][20347] Updated weights for policy 0, policy_version 1177853 (0.0006) [2023-12-27 13:48:38,271][20347] Updated weights for policy 0, policy_version 1177863 (0.0006) [2023-12-27 13:48:38,878][20381] Updated weights for policy 1, policy_version 1176890 (0.0006) [2023-12-27 13:48:38,938][20381] Updated weights for policy 1, policy_version 1176900 (0.0008) [2023-12-27 13:48:38,956][20347] Updated weights for policy 0, policy_version 1177873 (0.0006) [2023-12-27 13:48:38,981][20381] Updated weights for policy 1, policy_version 1176910 (0.0006) [2023-12-27 13:48:39,007][20347] Updated weights for policy 0, policy_version 1177883 (0.0007) [2023-12-27 13:48:39,033][20381] Updated weights for policy 1, policy_version 1176920 (0.0009) [2023-12-27 13:48:39,055][20347] Updated weights for policy 0, policy_version 1177893 (0.0009) [2023-12-27 13:48:39,102][20347] Updated weights for policy 0, policy_version 1177903 (0.0008) [2023-12-27 13:48:39,812][20347] Updated weights for policy 0, policy_version 1177913 (0.0006) [2023-12-27 13:48:39,814][20381] Updated weights for policy 1, policy_version 1176930 (0.0006) [2023-12-27 13:48:39,876][20381] Updated weights for policy 1, policy_version 1176940 (0.0008) [2023-12-27 13:48:39,881][20347] Updated weights for policy 0, policy_version 1177923 (0.0008) [2023-12-27 13:48:39,941][20381] Updated weights for policy 1, policy_version 1176950 (0.0007) [2023-12-27 13:48:39,946][20347] Updated weights for policy 0, policy_version 1177933 (0.0006) [2023-12-27 13:48:40,576][20381] Updated weights for policy 1, policy_version 1176960 (0.0006) [2023-12-27 13:48:40,630][20347] Updated weights for policy 0, policy_version 1177943 (0.0006) [2023-12-27 13:48:40,635][20381] Updated weights for policy 1, policy_version 1176970 (0.0005) [2023-12-27 13:48:40,684][20347] Updated weights for policy 0, policy_version 1177953 (0.0008) [2023-12-27 13:48:40,693][20381] Updated weights for policy 1, policy_version 1176980 (0.0005) [2023-12-27 13:48:40,745][20347] Updated weights for policy 0, policy_version 1177963 (0.0007) [2023-12-27 13:48:41,359][20381] Updated weights for policy 1, policy_version 1176990 (0.0009) [2023-12-27 13:48:41,431][20381] Updated weights for policy 1, policy_version 1177000 (0.0010) [2023-12-27 13:48:41,455][20347] Updated weights for policy 0, policy_version 1177973 (0.0009) [2023-12-27 13:48:41,492][20381] Updated weights for policy 1, policy_version 1177010 (0.0011) [2023-12-27 13:48:41,512][20347] Updated weights for policy 0, policy_version 1177983 (0.0008) [2023-12-27 13:48:41,563][20347] Updated weights for policy 0, policy_version 1177993 (0.0011) [2023-12-27 13:48:42,208][20381] Updated weights for policy 1, policy_version 1177020 (0.0011) [2023-12-27 13:48:42,225][20347] Updated weights for policy 0, policy_version 1178003 (0.0007) [2023-12-27 13:48:42,268][20381] Updated weights for policy 1, policy_version 1177030 (0.0011) [2023-12-27 13:48:42,288][20347] Updated weights for policy 0, policy_version 1178013 (0.0007) [2023-12-27 13:48:42,325][20381] Updated weights for policy 1, policy_version 1177040 (0.0011) [2023-12-27 13:48:42,346][20347] Updated weights for policy 0, policy_version 1178023 (0.0006) [2023-12-27 13:48:42,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 602963968. Throughput: 0: 10262.7, 1: 10228.4. Samples: 602963216. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:42,363][19170] Avg episode reward: [(0, '463.990'), (1, '764.880')] [2023-12-27 13:48:42,981][20347] Updated weights for policy 0, policy_version 1178033 (0.0007) [2023-12-27 13:48:43,013][20381] Updated weights for policy 1, policy_version 1177050 (0.0010) [2023-12-27 13:48:43,047][20347] Updated weights for policy 0, policy_version 1178043 (0.0009) [2023-12-27 13:48:43,067][20381] Updated weights for policy 1, policy_version 1177060 (0.0008) [2023-12-27 13:48:43,108][20347] Updated weights for policy 0, policy_version 1178053 (0.0009) [2023-12-27 13:48:43,120][20381] Updated weights for policy 1, policy_version 1177070 (0.0006) [2023-12-27 13:48:43,169][20347] Updated weights for policy 0, policy_version 1178063 (0.0009) [2023-12-27 13:48:43,179][20381] Updated weights for policy 1, policy_version 1177080 (0.0007) [2023-12-27 13:48:43,841][20347] Updated weights for policy 0, policy_version 1178073 (0.0007) [2023-12-27 13:48:43,871][20381] Updated weights for policy 1, policy_version 1177090 (0.0007) [2023-12-27 13:48:43,895][20347] Updated weights for policy 0, policy_version 1178083 (0.0005) [2023-12-27 13:48:43,931][20381] Updated weights for policy 1, policy_version 1177100 (0.0007) [2023-12-27 13:48:43,945][20347] Updated weights for policy 0, policy_version 1178093 (0.0006) [2023-12-27 13:48:43,983][20381] Updated weights for policy 1, policy_version 1177110 (0.0007) [2023-12-27 13:48:44,598][20381] Updated weights for policy 1, policy_version 1177120 (0.0007) [2023-12-27 13:48:44,639][20347] Updated weights for policy 0, policy_version 1178103 (0.0005) [2023-12-27 13:48:44,650][20381] Updated weights for policy 1, policy_version 1177130 (0.0010) [2023-12-27 13:48:44,691][20347] Updated weights for policy 0, policy_version 1178113 (0.0008) [2023-12-27 13:48:44,709][20381] Updated weights for policy 1, policy_version 1177140 (0.0011) [2023-12-27 13:48:44,752][20347] Updated weights for policy 0, policy_version 1178123 (0.0008) [2023-12-27 13:48:45,369][20381] Updated weights for policy 1, policy_version 1177150 (0.0006) [2023-12-27 13:48:45,429][20381] Updated weights for policy 1, policy_version 1177160 (0.0006) [2023-12-27 13:48:45,450][20347] Updated weights for policy 0, policy_version 1178133 (0.0007) [2023-12-27 13:48:45,478][20381] Updated weights for policy 1, policy_version 1177170 (0.0007) [2023-12-27 13:48:45,499][20347] Updated weights for policy 0, policy_version 1178143 (0.0006) [2023-12-27 13:48:45,555][20347] Updated weights for policy 0, policy_version 1178153 (0.0007) [2023-12-27 13:48:46,100][20381] Updated weights for policy 1, policy_version 1177180 (0.0006) [2023-12-27 13:48:46,147][20381] Updated weights for policy 1, policy_version 1177190 (0.0005) [2023-12-27 13:48:46,193][20381] Updated weights for policy 1, policy_version 1177200 (0.0005) [2023-12-27 13:48:46,262][20347] Updated weights for policy 0, policy_version 1178163 (0.0007) [2023-12-27 13:48:46,320][20347] Updated weights for policy 0, policy_version 1178173 (0.0008) [2023-12-27 13:48:46,368][20347] Updated weights for policy 0, policy_version 1178183 (0.0007) [2023-12-27 13:48:46,808][20381] Updated weights for policy 1, policy_version 1177210 (0.0005) [2023-12-27 13:48:46,864][20381] Updated weights for policy 1, policy_version 1177220 (0.0007) [2023-12-27 13:48:46,915][20381] Updated weights for policy 1, policy_version 1177230 (0.0008) [2023-12-27 13:48:46,940][20347] Updated weights for policy 0, policy_version 1178193 (0.0005) [2023-12-27 13:48:46,970][20381] Updated weights for policy 1, policy_version 1177240 (0.0008) [2023-12-27 13:48:47,002][20347] Updated weights for policy 0, policy_version 1178203 (0.0007) [2023-12-27 13:48:47,058][20347] Updated weights for policy 0, policy_version 1178213 (0.0008) [2023-12-27 13:48:47,114][20347] Updated weights for policy 0, policy_version 1178223 (0.0007) [2023-12-27 13:48:47,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 603078656. Throughput: 0: 10303.9, 1: 10262.7. Samples: 603088168. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:47,363][19170] Avg episode reward: [(0, '516.120'), (1, '735.290')] [2023-12-27 13:48:47,615][20381] Updated weights for policy 1, policy_version 1177250 (0.0010) [2023-12-27 13:48:47,669][20381] Updated weights for policy 1, policy_version 1177260 (0.0006) [2023-12-27 13:48:47,715][20381] Updated weights for policy 1, policy_version 1177270 (0.0005) [2023-12-27 13:48:47,803][20347] Updated weights for policy 0, policy_version 1178233 (0.0005) [2023-12-27 13:48:47,858][20347] Updated weights for policy 0, policy_version 1178243 (0.0010) [2023-12-27 13:48:47,910][20347] Updated weights for policy 0, policy_version 1178253 (0.0006) [2023-12-27 13:48:48,358][20381] Updated weights for policy 1, policy_version 1177280 (0.0009) [2023-12-27 13:48:48,420][20381] Updated weights for policy 1, policy_version 1177290 (0.0011) [2023-12-27 13:48:48,483][20381] Updated weights for policy 1, policy_version 1177300 (0.0009) [2023-12-27 13:48:48,532][20347] Updated weights for policy 0, policy_version 1178263 (0.0009) [2023-12-27 13:48:48,592][20347] Updated weights for policy 0, policy_version 1178273 (0.0010) [2023-12-27 13:48:48,650][20347] Updated weights for policy 0, policy_version 1178283 (0.0009) [2023-12-27 13:48:49,099][20381] Updated weights for policy 1, policy_version 1177310 (0.0011) [2023-12-27 13:48:49,158][20381] Updated weights for policy 1, policy_version 1177320 (0.0011) [2023-12-27 13:48:49,220][20381] Updated weights for policy 1, policy_version 1177330 (0.0011) [2023-12-27 13:48:49,334][20347] Updated weights for policy 0, policy_version 1178293 (0.0007) [2023-12-27 13:48:49,405][20347] Updated weights for policy 0, policy_version 1178303 (0.0008) [2023-12-27 13:48:49,467][20347] Updated weights for policy 0, policy_version 1178313 (0.0008) [2023-12-27 13:48:49,899][20381] Updated weights for policy 1, policy_version 1177340 (0.0009) [2023-12-27 13:48:49,959][20381] Updated weights for policy 1, policy_version 1177350 (0.0009) [2023-12-27 13:48:50,023][20381] Updated weights for policy 1, policy_version 1177360 (0.0011) [2023-12-27 13:48:50,182][20347] Updated weights for policy 0, policy_version 1178323 (0.0009) [2023-12-27 13:48:50,238][20347] Updated weights for policy 0, policy_version 1178333 (0.0010) [2023-12-27 13:48:50,299][20347] Updated weights for policy 0, policy_version 1178343 (0.0005) [2023-12-27 13:48:50,704][20381] Updated weights for policy 1, policy_version 1177370 (0.0009) [2023-12-27 13:48:50,759][20381] Updated weights for policy 1, policy_version 1177380 (0.0011) [2023-12-27 13:48:50,825][20381] Updated weights for policy 1, policy_version 1177390 (0.0011) [2023-12-27 13:48:50,881][20381] Updated weights for policy 1, policy_version 1177400 (0.0010) [2023-12-27 13:48:50,945][20347] Updated weights for policy 0, policy_version 1178353 (0.0009) [2023-12-27 13:48:51,003][20347] Updated weights for policy 0, policy_version 1178363 (0.0010) [2023-12-27 13:48:51,064][20347] Updated weights for policy 0, policy_version 1178373 (0.0008) [2023-12-27 13:48:51,126][20347] Updated weights for policy 0, policy_version 1178383 (0.0009) [2023-12-27 13:48:51,607][20381] Updated weights for policy 1, policy_version 1177410 (0.0006) [2023-12-27 13:48:51,675][20381] Updated weights for policy 1, policy_version 1177420 (0.0008) [2023-12-27 13:48:51,739][20381] Updated weights for policy 1, policy_version 1177430 (0.0009) [2023-12-27 13:48:51,852][20347] Updated weights for policy 0, policy_version 1178393 (0.0006) [2023-12-27 13:48:51,909][20347] Updated weights for policy 0, policy_version 1178403 (0.0008) [2023-12-27 13:48:51,972][20347] Updated weights for policy 0, policy_version 1178413 (0.0008) [2023-12-27 13:48:52,357][20381] Updated weights for policy 1, policy_version 1177440 (0.0008) [2023-12-27 13:48:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 603176960. Throughput: 0: 10280.2, 1: 10269.7. Samples: 603150336. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:52,363][19170] Avg episode reward: [(0, '493.860'), (1, '666.950')] [2023-12-27 13:48:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001178416_301727744.pth... [2023-12-27 13:48:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001177232_301424640.pth [2023-12-27 13:48:52,420][20381] Updated weights for policy 1, policy_version 1177450 (0.0010) [2023-12-27 13:48:52,482][20381] Updated weights for policy 1, policy_version 1177460 (0.0010) [2023-12-27 13:48:52,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001177464_301457408.pth... [2023-12-27 13:48:52,509][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001176248_301146112.pth [2023-12-27 13:48:52,690][20347] Updated weights for policy 0, policy_version 1178423 (0.0008) [2023-12-27 13:48:52,750][20347] Updated weights for policy 0, policy_version 1178433 (0.0008) [2023-12-27 13:48:52,811][20347] Updated weights for policy 0, policy_version 1178443 (0.0007) [2023-12-27 13:48:53,121][20381] Updated weights for policy 1, policy_version 1177470 (0.0010) [2023-12-27 13:48:53,170][20381] Updated weights for policy 1, policy_version 1177480 (0.0008) [2023-12-27 13:48:53,216][20381] Updated weights for policy 1, policy_version 1177490 (0.0005) [2023-12-27 13:48:53,386][20347] Updated weights for policy 0, policy_version 1178453 (0.0006) [2023-12-27 13:48:53,440][20347] Updated weights for policy 0, policy_version 1178463 (0.0005) [2023-12-27 13:48:53,497][20347] Updated weights for policy 0, policy_version 1178473 (0.0005) [2023-12-27 13:48:53,885][20381] Updated weights for policy 1, policy_version 1177500 (0.0006) [2023-12-27 13:48:53,945][20381] Updated weights for policy 1, policy_version 1177510 (0.0008) [2023-12-27 13:48:54,002][20381] Updated weights for policy 1, policy_version 1177520 (0.0008) [2023-12-27 13:48:54,104][20347] Updated weights for policy 0, policy_version 1178483 (0.0006) [2023-12-27 13:48:54,150][20347] Updated weights for policy 0, policy_version 1178493 (0.0008) [2023-12-27 13:48:54,204][20347] Updated weights for policy 0, policy_version 1178503 (0.0008) [2023-12-27 13:48:54,677][20381] Updated weights for policy 1, policy_version 1177530 (0.0007) [2023-12-27 13:48:54,732][20381] Updated weights for policy 1, policy_version 1177540 (0.0007) [2023-12-27 13:48:54,781][20381] Updated weights for policy 1, policy_version 1177550 (0.0009) [2023-12-27 13:48:54,832][20381] Updated weights for policy 1, policy_version 1177560 (0.0007) [2023-12-27 13:48:54,893][20347] Updated weights for policy 0, policy_version 1178513 (0.0008) [2023-12-27 13:48:54,949][20347] Updated weights for policy 0, policy_version 1178523 (0.0007) [2023-12-27 13:48:55,014][20347] Updated weights for policy 0, policy_version 1178533 (0.0005) [2023-12-27 13:48:55,067][20347] Updated weights for policy 0, policy_version 1178543 (0.0010) [2023-12-27 13:48:55,549][20381] Updated weights for policy 1, policy_version 1177570 (0.0006) [2023-12-27 13:48:55,606][20381] Updated weights for policy 1, policy_version 1177580 (0.0007) [2023-12-27 13:48:55,656][20381] Updated weights for policy 1, policy_version 1177590 (0.0006) [2023-12-27 13:48:55,675][20347] Updated weights for policy 0, policy_version 1178553 (0.0010) [2023-12-27 13:48:55,736][20347] Updated weights for policy 0, policy_version 1178563 (0.0009) [2023-12-27 13:48:55,797][20347] Updated weights for policy 0, policy_version 1178573 (0.0010) [2023-12-27 13:48:56,327][20381] Updated weights for policy 1, policy_version 1177600 (0.0007) [2023-12-27 13:48:56,383][20381] Updated weights for policy 1, policy_version 1177610 (0.0007) [2023-12-27 13:48:56,437][20381] Updated weights for policy 1, policy_version 1177620 (0.0008) [2023-12-27 13:48:56,477][20347] Updated weights for policy 0, policy_version 1178583 (0.0007) [2023-12-27 13:48:56,528][20347] Updated weights for policy 0, policy_version 1178593 (0.0005) [2023-12-27 13:48:56,574][20347] Updated weights for policy 0, policy_version 1178603 (0.0005) [2023-12-27 13:48:57,074][20381] Updated weights for policy 1, policy_version 1177630 (0.0007) [2023-12-27 13:48:57,127][20381] Updated weights for policy 1, policy_version 1177640 (0.0007) [2023-12-27 13:48:57,191][20381] Updated weights for policy 1, policy_version 1177650 (0.0006) [2023-12-27 13:48:57,226][20347] Updated weights for policy 0, policy_version 1178613 (0.0007) [2023-12-27 13:48:57,274][20347] Updated weights for policy 0, policy_version 1178623 (0.0005) [2023-12-27 13:48:57,326][20347] Updated weights for policy 0, policy_version 1178633 (0.0007) [2023-12-27 13:48:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 603291648. Throughput: 0: 10267.0, 1: 10320.5. Samples: 603274528. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:48:57,363][19170] Avg episode reward: [(0, '523.580'), (1, '764.770')] [2023-12-27 13:48:57,867][20381] Updated weights for policy 1, policy_version 1177660 (0.0006) [2023-12-27 13:48:57,929][20381] Updated weights for policy 1, policy_version 1177670 (0.0008) [2023-12-27 13:48:57,964][20347] Updated weights for policy 0, policy_version 1178643 (0.0009) [2023-12-27 13:48:57,980][20381] Updated weights for policy 1, policy_version 1177680 (0.0006) [2023-12-27 13:48:58,022][20347] Updated weights for policy 0, policy_version 1178653 (0.0010) [2023-12-27 13:48:58,067][20347] Updated weights for policy 0, policy_version 1178663 (0.0010) [2023-12-27 13:48:58,669][20381] Updated weights for policy 1, policy_version 1177690 (0.0005) [2023-12-27 13:48:58,734][20381] Updated weights for policy 1, policy_version 1177700 (0.0008) [2023-12-27 13:48:58,799][20381] Updated weights for policy 1, policy_version 1177710 (0.0008) [2023-12-27 13:48:58,868][20381] Updated weights for policy 1, policy_version 1177720 (0.0008) [2023-12-27 13:48:58,900][20347] Updated weights for policy 0, policy_version 1178673 (0.0010) [2023-12-27 13:48:58,962][20347] Updated weights for policy 0, policy_version 1178683 (0.0010) [2023-12-27 13:48:59,025][20347] Updated weights for policy 0, policy_version 1178693 (0.0008) [2023-12-27 13:48:59,083][20347] Updated weights for policy 0, policy_version 1178703 (0.0008) [2023-12-27 13:48:59,647][20381] Updated weights for policy 1, policy_version 1177730 (0.0006) [2023-12-27 13:48:59,703][20381] Updated weights for policy 1, policy_version 1177740 (0.0007) [2023-12-27 13:48:59,758][20381] Updated weights for policy 1, policy_version 1177750 (0.0006) [2023-12-27 13:48:59,771][20347] Updated weights for policy 0, policy_version 1178713 (0.0007) [2023-12-27 13:48:59,819][20347] Updated weights for policy 0, policy_version 1178723 (0.0007) [2023-12-27 13:48:59,879][20347] Updated weights for policy 0, policy_version 1178733 (0.0007) [2023-12-27 13:49:00,408][20381] Updated weights for policy 1, policy_version 1177760 (0.0005) [2023-12-27 13:49:00,470][20381] Updated weights for policy 1, policy_version 1177770 (0.0005) [2023-12-27 13:49:00,523][20381] Updated weights for policy 1, policy_version 1177780 (0.0005) [2023-12-27 13:49:00,585][20347] Updated weights for policy 0, policy_version 1178743 (0.0008) [2023-12-27 13:49:00,643][20347] Updated weights for policy 0, policy_version 1178753 (0.0007) [2023-12-27 13:49:00,705][20347] Updated weights for policy 0, policy_version 1178763 (0.0008) [2023-12-27 13:49:01,140][20381] Updated weights for policy 1, policy_version 1177790 (0.0008) [2023-12-27 13:49:01,180][20292] KL-divergence is very high: 102.3251 [2023-12-27 13:49:01,197][20381] Updated weights for policy 1, policy_version 1177800 (0.0011) [2023-12-27 13:49:01,250][20381] Updated weights for policy 1, policy_version 1177810 (0.0010) [2023-12-27 13:49:01,365][20347] Updated weights for policy 0, policy_version 1178773 (0.0007) [2023-12-27 13:49:01,421][20347] Updated weights for policy 0, policy_version 1178783 (0.0007) [2023-12-27 13:49:01,482][20347] Updated weights for policy 0, policy_version 1178793 (0.0006) [2023-12-27 13:49:01,988][20381] Updated weights for policy 1, policy_version 1177820 (0.0010) [2023-12-27 13:49:02,041][20381] Updated weights for policy 1, policy_version 1177830 (0.0011) [2023-12-27 13:49:02,107][20381] Updated weights for policy 1, policy_version 1177840 (0.0011) [2023-12-27 13:49:02,189][20347] Updated weights for policy 0, policy_version 1178803 (0.0006) [2023-12-27 13:49:02,247][20347] Updated weights for policy 0, policy_version 1178813 (0.0008) [2023-12-27 13:49:02,308][20347] Updated weights for policy 0, policy_version 1178823 (0.0007) [2023-12-27 13:49:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 603381760. Throughput: 0: 10223.8, 1: 10315.1. Samples: 603395052. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:02,363][19170] Avg episode reward: [(0, '423.590'), (1, '784.280')] [2023-12-27 13:49:02,792][20381] Updated weights for policy 1, policy_version 1177850 (0.0010) [2023-12-27 13:49:02,848][20381] Updated weights for policy 1, policy_version 1177860 (0.0007) [2023-12-27 13:49:02,898][20381] Updated weights for policy 1, policy_version 1177870 (0.0007) [2023-12-27 13:49:02,952][20381] Updated weights for policy 1, policy_version 1177880 (0.0009) [2023-12-27 13:49:02,963][20347] Updated weights for policy 0, policy_version 1178833 (0.0008) [2023-12-27 13:49:03,024][20347] Updated weights for policy 0, policy_version 1178843 (0.0007) [2023-12-27 13:49:03,067][20347] Updated weights for policy 0, policy_version 1178853 (0.0005) [2023-12-27 13:49:03,123][20347] Updated weights for policy 0, policy_version 1178863 (0.0006) [2023-12-27 13:49:03,563][20381] Updated weights for policy 1, policy_version 1177890 (0.0005) [2023-12-27 13:49:03,619][20381] Updated weights for policy 1, policy_version 1177900 (0.0005) [2023-12-27 13:49:03,674][20381] Updated weights for policy 1, policy_version 1177910 (0.0005) [2023-12-27 13:49:03,772][20347] Updated weights for policy 0, policy_version 1178873 (0.0008) [2023-12-27 13:49:03,818][20347] Updated weights for policy 0, policy_version 1178883 (0.0005) [2023-12-27 13:49:03,882][20347] Updated weights for policy 0, policy_version 1178893 (0.0008) [2023-12-27 13:49:04,333][20381] Updated weights for policy 1, policy_version 1177920 (0.0008) [2023-12-27 13:49:04,393][20381] Updated weights for policy 1, policy_version 1177930 (0.0007) [2023-12-27 13:49:04,453][20381] Updated weights for policy 1, policy_version 1177940 (0.0008) [2023-12-27 13:49:04,576][20347] Updated weights for policy 0, policy_version 1178903 (0.0007) [2023-12-27 13:49:04,637][20347] Updated weights for policy 0, policy_version 1178913 (0.0008) [2023-12-27 13:49:04,697][20347] Updated weights for policy 0, policy_version 1178923 (0.0008) [2023-12-27 13:49:05,104][20381] Updated weights for policy 1, policy_version 1177950 (0.0008) [2023-12-27 13:49:05,162][20381] Updated weights for policy 1, policy_version 1177960 (0.0008) [2023-12-27 13:49:05,221][20381] Updated weights for policy 1, policy_version 1177970 (0.0008) [2023-12-27 13:49:05,385][20347] Updated weights for policy 0, policy_version 1178933 (0.0006) [2023-12-27 13:49:05,444][20347] Updated weights for policy 0, policy_version 1178943 (0.0005) [2023-12-27 13:49:05,497][20347] Updated weights for policy 0, policy_version 1178953 (0.0005) [2023-12-27 13:49:05,919][20381] Updated weights for policy 1, policy_version 1177980 (0.0009) [2023-12-27 13:49:05,968][20381] Updated weights for policy 1, policy_version 1177990 (0.0010) [2023-12-27 13:49:06,016][20381] Updated weights for policy 1, policy_version 1178000 (0.0010) [2023-12-27 13:49:06,095][20347] Updated weights for policy 0, policy_version 1178963 (0.0005) [2023-12-27 13:49:06,158][20347] Updated weights for policy 0, policy_version 1178973 (0.0008) [2023-12-27 13:49:06,212][20347] Updated weights for policy 0, policy_version 1178983 (0.0008) [2023-12-27 13:49:06,730][20381] Updated weights for policy 1, policy_version 1178010 (0.0011) [2023-12-27 13:49:06,802][20381] Updated weights for policy 1, policy_version 1178020 (0.0010) [2023-12-27 13:49:06,851][20347] Updated weights for policy 0, policy_version 1178993 (0.0008) [2023-12-27 13:49:06,873][20381] Updated weights for policy 1, policy_version 1178030 (0.0010) [2023-12-27 13:49:06,912][20347] Updated weights for policy 0, policy_version 1179003 (0.0006) [2023-12-27 13:49:06,946][20381] Updated weights for policy 1, policy_version 1178040 (0.0010) [2023-12-27 13:49:06,975][20347] Updated weights for policy 0, policy_version 1179013 (0.0007) [2023-12-27 13:49:07,035][20347] Updated weights for policy 0, policy_version 1179023 (0.0006) [2023-12-27 13:49:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 603488256. Throughput: 0: 10262.0, 1: 10326.2. Samples: 603457260. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:07,363][19170] Avg episode reward: [(0, '284.630'), (1, '813.770')] [2023-12-27 13:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001179024_301883392.pth... [2023-12-27 13:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001178040_301604864.pth... [2023-12-27 13:49:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001177840_301580288.pth [2023-12-27 13:49:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001176856_301301760.pth [2023-12-27 13:49:07,481][20381] Updated weights for policy 1, policy_version 1178050 (0.0005) [2023-12-27 13:49:07,533][20381] Updated weights for policy 1, policy_version 1178060 (0.0005) [2023-12-27 13:49:07,543][20292] KL-divergence is very high: 118.7289 [2023-12-27 13:49:07,549][20292] KL-divergence is very high: 121.1491 [2023-12-27 13:49:07,587][20381] Updated weights for policy 1, policy_version 1178070 (0.0007) [2023-12-27 13:49:07,691][20347] Updated weights for policy 0, policy_version 1179033 (0.0008) [2023-12-27 13:49:07,744][20347] Updated weights for policy 0, policy_version 1179043 (0.0007) [2023-12-27 13:49:07,798][20347] Updated weights for policy 0, policy_version 1179053 (0.0007) [2023-12-27 13:49:08,188][20381] Updated weights for policy 1, policy_version 1178080 (0.0010) [2023-12-27 13:49:08,246][20381] Updated weights for policy 1, policy_version 1178090 (0.0006) [2023-12-27 13:49:08,307][20381] Updated weights for policy 1, policy_version 1178100 (0.0009) [2023-12-27 13:49:08,481][20347] Updated weights for policy 0, policy_version 1179063 (0.0008) [2023-12-27 13:49:08,535][20347] Updated weights for policy 0, policy_version 1179073 (0.0007) [2023-12-27 13:49:08,585][20347] Updated weights for policy 0, policy_version 1179083 (0.0007) [2023-12-27 13:49:08,995][20381] Updated weights for policy 1, policy_version 1178110 (0.0010) [2023-12-27 13:49:09,047][20381] Updated weights for policy 1, policy_version 1178120 (0.0010) [2023-12-27 13:49:09,102][20381] Updated weights for policy 1, policy_version 1178130 (0.0010) [2023-12-27 13:49:09,271][20347] Updated weights for policy 0, policy_version 1179093 (0.0007) [2023-12-27 13:49:09,327][20347] Updated weights for policy 0, policy_version 1179103 (0.0006) [2023-12-27 13:49:09,393][20347] Updated weights for policy 0, policy_version 1179113 (0.0008) [2023-12-27 13:49:09,911][20381] Updated weights for policy 1, policy_version 1178140 (0.0009) [2023-12-27 13:49:09,977][20381] Updated weights for policy 1, policy_version 1178150 (0.0008) [2023-12-27 13:49:10,043][20381] Updated weights for policy 1, policy_version 1178160 (0.0007) [2023-12-27 13:49:10,083][20347] Updated weights for policy 0, policy_version 1179123 (0.0007) [2023-12-27 13:49:10,145][20347] Updated weights for policy 0, policy_version 1179133 (0.0006) [2023-12-27 13:49:10,208][20347] Updated weights for policy 0, policy_version 1179143 (0.0006) [2023-12-27 13:49:10,733][20381] Updated weights for policy 1, policy_version 1178170 (0.0008) [2023-12-27 13:49:10,788][20381] Updated weights for policy 1, policy_version 1178180 (0.0007) [2023-12-27 13:49:10,820][20347] Updated weights for policy 0, policy_version 1179153 (0.0008) [2023-12-27 13:49:10,845][20381] Updated weights for policy 1, policy_version 1178190 (0.0007) [2023-12-27 13:49:10,876][20347] Updated weights for policy 0, policy_version 1179163 (0.0006) [2023-12-27 13:49:10,897][20381] Updated weights for policy 1, policy_version 1178200 (0.0006) [2023-12-27 13:49:10,927][20347] Updated weights for policy 0, policy_version 1179173 (0.0007) [2023-12-27 13:49:10,984][20347] Updated weights for policy 0, policy_version 1179183 (0.0007) [2023-12-27 13:49:11,612][20381] Updated weights for policy 1, policy_version 1178210 (0.0007) [2023-12-27 13:49:11,682][20381] Updated weights for policy 1, policy_version 1178220 (0.0007) [2023-12-27 13:49:11,750][20381] Updated weights for policy 1, policy_version 1178230 (0.0008) [2023-12-27 13:49:11,767][20347] Updated weights for policy 0, policy_version 1179193 (0.0009) [2023-12-27 13:49:11,829][20347] Updated weights for policy 0, policy_version 1179203 (0.0010) [2023-12-27 13:49:11,887][20347] Updated weights for policy 0, policy_version 1179213 (0.0008) [2023-12-27 13:49:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 603586560. Throughput: 0: 10266.7, 1: 10305.6. Samples: 603580688. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:12,363][19170] Avg episode reward: [(0, '191.720'), (1, '706.040')] [2023-12-27 13:49:12,404][20381] Updated weights for policy 1, policy_version 1178240 (0.0009) [2023-12-27 13:49:12,457][20381] Updated weights for policy 1, policy_version 1178250 (0.0010) [2023-12-27 13:49:12,510][20381] Updated weights for policy 1, policy_version 1178260 (0.0010) [2023-12-27 13:49:12,601][20347] Updated weights for policy 0, policy_version 1179223 (0.0008) [2023-12-27 13:49:12,657][20347] Updated weights for policy 0, policy_version 1179233 (0.0007) [2023-12-27 13:49:12,709][20347] Updated weights for policy 0, policy_version 1179243 (0.0007) [2023-12-27 13:49:13,196][20381] Updated weights for policy 1, policy_version 1178270 (0.0010) [2023-12-27 13:49:13,241][20381] Updated weights for policy 1, policy_version 1178280 (0.0010) [2023-12-27 13:49:13,299][20381] Updated weights for policy 1, policy_version 1178290 (0.0010) [2023-12-27 13:49:13,411][20347] Updated weights for policy 0, policy_version 1179253 (0.0010) [2023-12-27 13:49:13,464][20347] Updated weights for policy 0, policy_version 1179263 (0.0009) [2023-12-27 13:49:13,526][20347] Updated weights for policy 0, policy_version 1179273 (0.0005) [2023-12-27 13:49:13,953][20381] Updated weights for policy 1, policy_version 1178300 (0.0008) [2023-12-27 13:49:14,007][20381] Updated weights for policy 1, policy_version 1178310 (0.0005) [2023-12-27 13:49:14,044][20347] Updated weights for policy 0, policy_version 1179283 (0.0005) [2023-12-27 13:49:14,067][20381] Updated weights for policy 1, policy_version 1178320 (0.0006) [2023-12-27 13:49:14,087][20347] Updated weights for policy 0, policy_version 1179293 (0.0007) [2023-12-27 13:49:14,133][20347] Updated weights for policy 0, policy_version 1179303 (0.0006) [2023-12-27 13:49:14,626][20381] Updated weights for policy 1, policy_version 1178330 (0.0007) [2023-12-27 13:49:14,683][20381] Updated weights for policy 1, policy_version 1178340 (0.0005) [2023-12-27 13:49:14,740][20381] Updated weights for policy 1, policy_version 1178350 (0.0006) [2023-12-27 13:49:14,803][20381] Updated weights for policy 1, policy_version 1178360 (0.0009) [2023-12-27 13:49:14,822][20347] Updated weights for policy 0, policy_version 1179313 (0.0008) [2023-12-27 13:49:14,880][20347] Updated weights for policy 0, policy_version 1179323 (0.0008) [2023-12-27 13:49:14,939][20347] Updated weights for policy 0, policy_version 1179333 (0.0008) [2023-12-27 13:49:15,002][20347] Updated weights for policy 0, policy_version 1179343 (0.0007) [2023-12-27 13:49:15,451][20381] Updated weights for policy 1, policy_version 1178370 (0.0007) [2023-12-27 13:49:15,503][20381] Updated weights for policy 1, policy_version 1178380 (0.0007) [2023-12-27 13:49:15,559][20381] Updated weights for policy 1, policy_version 1178390 (0.0009) [2023-12-27 13:49:15,622][20347] Updated weights for policy 0, policy_version 1179353 (0.0008) [2023-12-27 13:49:15,671][20347] Updated weights for policy 0, policy_version 1179363 (0.0008) [2023-12-27 13:49:15,716][20347] Updated weights for policy 0, policy_version 1179373 (0.0008) [2023-12-27 13:49:16,272][20381] Updated weights for policy 1, policy_version 1178400 (0.0007) [2023-12-27 13:49:16,324][20381] Updated weights for policy 1, policy_version 1178410 (0.0008) [2023-12-27 13:49:16,371][20381] Updated weights for policy 1, policy_version 1178420 (0.0007) [2023-12-27 13:49:16,419][20347] Updated weights for policy 0, policy_version 1179383 (0.0007) [2023-12-27 13:49:16,470][20347] Updated weights for policy 0, policy_version 1179393 (0.0007) [2023-12-27 13:49:16,515][20347] Updated weights for policy 0, policy_version 1179403 (0.0008) [2023-12-27 13:49:17,064][20381] Updated weights for policy 1, policy_version 1178430 (0.0006) [2023-12-27 13:49:17,136][20381] Updated weights for policy 1, policy_version 1178440 (0.0008) [2023-12-27 13:49:17,193][20347] Updated weights for policy 0, policy_version 1179413 (0.0006) [2023-12-27 13:49:17,194][20381] Updated weights for policy 1, policy_version 1178450 (0.0008) [2023-12-27 13:49:17,252][20347] Updated weights for policy 0, policy_version 1179423 (0.0005) [2023-12-27 13:49:17,317][20347] Updated weights for policy 0, policy_version 1179433 (0.0005) [2023-12-27 13:49:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 603701248. Throughput: 0: 10243.3, 1: 10289.6. Samples: 603704532. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:17,363][19170] Avg episode reward: [(0, '306.090'), (1, '735.560')] [2023-12-27 13:49:17,875][20381] Updated weights for policy 1, policy_version 1178460 (0.0008) [2023-12-27 13:49:17,892][20347] Updated weights for policy 0, policy_version 1179443 (0.0005) [2023-12-27 13:49:17,934][20381] Updated weights for policy 1, policy_version 1178470 (0.0006) [2023-12-27 13:49:17,952][20347] Updated weights for policy 0, policy_version 1179453 (0.0008) [2023-12-27 13:49:17,983][20381] Updated weights for policy 1, policy_version 1178480 (0.0008) [2023-12-27 13:49:18,008][20347] Updated weights for policy 0, policy_version 1179463 (0.0006) [2023-12-27 13:49:18,620][20347] Updated weights for policy 0, policy_version 1179473 (0.0006) [2023-12-27 13:49:18,670][20381] Updated weights for policy 1, policy_version 1178490 (0.0005) [2023-12-27 13:49:18,685][20347] Updated weights for policy 0, policy_version 1179483 (0.0006) [2023-12-27 13:49:18,728][20381] Updated weights for policy 1, policy_version 1178500 (0.0006) [2023-12-27 13:49:18,742][20347] Updated weights for policy 0, policy_version 1179493 (0.0006) [2023-12-27 13:49:18,783][20381] Updated weights for policy 1, policy_version 1178510 (0.0007) [2023-12-27 13:49:18,801][20347] Updated weights for policy 0, policy_version 1179503 (0.0005) [2023-12-27 13:49:18,847][20381] Updated weights for policy 1, policy_version 1178520 (0.0009) [2023-12-27 13:49:19,403][20347] Updated weights for policy 0, policy_version 1179513 (0.0008) [2023-12-27 13:49:19,464][20347] Updated weights for policy 0, policy_version 1179523 (0.0008) [2023-12-27 13:49:19,529][20347] Updated weights for policy 0, policy_version 1179533 (0.0007) [2023-12-27 13:49:19,584][20381] Updated weights for policy 1, policy_version 1178530 (0.0010) [2023-12-27 13:49:19,641][20381] Updated weights for policy 1, policy_version 1178540 (0.0011) [2023-12-27 13:49:19,703][20381] Updated weights for policy 1, policy_version 1178550 (0.0011) [2023-12-27 13:49:20,249][20347] Updated weights for policy 0, policy_version 1179543 (0.0006) [2023-12-27 13:49:20,313][20347] Updated weights for policy 0, policy_version 1179553 (0.0008) [2023-12-27 13:49:20,377][20347] Updated weights for policy 0, policy_version 1179563 (0.0008) [2023-12-27 13:49:20,396][20381] Updated weights for policy 1, policy_version 1178560 (0.0007) [2023-12-27 13:49:20,448][20381] Updated weights for policy 1, policy_version 1178570 (0.0006) [2023-12-27 13:49:20,506][20381] Updated weights for policy 1, policy_version 1178580 (0.0005) [2023-12-27 13:49:21,046][20347] Updated weights for policy 0, policy_version 1179573 (0.0008) [2023-12-27 13:49:21,114][20347] Updated weights for policy 0, policy_version 1179583 (0.0009) [2023-12-27 13:49:21,180][20347] Updated weights for policy 0, policy_version 1179593 (0.0007) [2023-12-27 13:49:21,204][20381] Updated weights for policy 1, policy_version 1178590 (0.0009) [2023-12-27 13:49:21,269][20381] Updated weights for policy 1, policy_version 1178600 (0.0011) [2023-12-27 13:49:21,332][20381] Updated weights for policy 1, policy_version 1178610 (0.0009) [2023-12-27 13:49:21,902][20347] Updated weights for policy 0, policy_version 1179603 (0.0007) [2023-12-27 13:49:21,963][20347] Updated weights for policy 0, policy_version 1179613 (0.0007) [2023-12-27 13:49:22,021][20347] Updated weights for policy 0, policy_version 1179623 (0.0007) [2023-12-27 13:49:22,088][20381] Updated weights for policy 1, policy_version 1178620 (0.0008) [2023-12-27 13:49:22,147][20381] Updated weights for policy 1, policy_version 1178630 (0.0008) [2023-12-27 13:49:22,192][20381] Updated weights for policy 1, policy_version 1178640 (0.0008) [2023-12-27 13:49:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 603799552. Throughput: 0: 10313.7, 1: 10247.2. Samples: 603766780. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:22,363][19170] Avg episode reward: [(0, '372.950'), (1, '745.190')] [2023-12-27 13:49:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001178648_301760512.pth... [2023-12-27 13:49:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001179632_302039040.pth... [2023-12-27 13:49:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001177464_301457408.pth [2023-12-27 13:49:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001178416_301727744.pth [2023-12-27 13:49:22,676][20347] Updated weights for policy 0, policy_version 1179633 (0.0008) [2023-12-27 13:49:22,738][20347] Updated weights for policy 0, policy_version 1179643 (0.0007) [2023-12-27 13:49:22,799][20347] Updated weights for policy 0, policy_version 1179653 (0.0007) [2023-12-27 13:49:22,855][20347] Updated weights for policy 0, policy_version 1179663 (0.0009) [2023-12-27 13:49:22,866][20381] Updated weights for policy 1, policy_version 1178650 (0.0007) [2023-12-27 13:49:22,925][20381] Updated weights for policy 1, policy_version 1178660 (0.0007) [2023-12-27 13:49:22,982][20381] Updated weights for policy 1, policy_version 1178670 (0.0008) [2023-12-27 13:49:23,039][20381] Updated weights for policy 1, policy_version 1178680 (0.0007) [2023-12-27 13:49:23,545][20347] Updated weights for policy 0, policy_version 1179673 (0.0006) [2023-12-27 13:49:23,592][20347] Updated weights for policy 0, policy_version 1179683 (0.0005) [2023-12-27 13:49:23,647][20347] Updated weights for policy 0, policy_version 1179693 (0.0008) [2023-12-27 13:49:23,721][20381] Updated weights for policy 1, policy_version 1178690 (0.0005) [2023-12-27 13:49:23,776][20381] Updated weights for policy 1, policy_version 1178700 (0.0007) [2023-12-27 13:49:23,828][20381] Updated weights for policy 1, policy_version 1178710 (0.0010) [2023-12-27 13:49:24,305][20347] Updated weights for policy 0, policy_version 1179703 (0.0008) [2023-12-27 13:49:24,355][20347] Updated weights for policy 0, policy_version 1179713 (0.0007) [2023-12-27 13:49:24,407][20347] Updated weights for policy 0, policy_version 1179723 (0.0005) [2023-12-27 13:49:24,446][20381] Updated weights for policy 1, policy_version 1178720 (0.0011) [2023-12-27 13:49:24,505][20381] Updated weights for policy 1, policy_version 1178730 (0.0011) [2023-12-27 13:49:24,563][20381] Updated weights for policy 1, policy_version 1178740 (0.0010) [2023-12-27 13:49:25,075][20347] Updated weights for policy 0, policy_version 1179733 (0.0006) [2023-12-27 13:49:25,126][20347] Updated weights for policy 0, policy_version 1179743 (0.0007) [2023-12-27 13:49:25,182][20347] Updated weights for policy 0, policy_version 1179753 (0.0008) [2023-12-27 13:49:25,189][20381] Updated weights for policy 1, policy_version 1178750 (0.0011) [2023-12-27 13:49:25,237][20381] Updated weights for policy 1, policy_version 1178760 (0.0010) [2023-12-27 13:49:25,288][20381] Updated weights for policy 1, policy_version 1178770 (0.0010) [2023-12-27 13:49:25,855][20347] Updated weights for policy 0, policy_version 1179763 (0.0007) [2023-12-27 13:49:25,899][20347] Updated weights for policy 0, policy_version 1179773 (0.0007) [2023-12-27 13:49:25,953][20347] Updated weights for policy 0, policy_version 1179783 (0.0007) [2023-12-27 13:49:25,984][20381] Updated weights for policy 1, policy_version 1178780 (0.0010) [2023-12-27 13:49:26,036][20381] Updated weights for policy 1, policy_version 1178790 (0.0008) [2023-12-27 13:49:26,091][20381] Updated weights for policy 1, policy_version 1178800 (0.0009) [2023-12-27 13:49:26,678][20347] Updated weights for policy 0, policy_version 1179793 (0.0007) [2023-12-27 13:49:26,712][20381] Updated weights for policy 1, policy_version 1178810 (0.0008) [2023-12-27 13:49:26,739][20347] Updated weights for policy 0, policy_version 1179803 (0.0007) [2023-12-27 13:49:26,762][20381] Updated weights for policy 1, policy_version 1178820 (0.0007) [2023-12-27 13:49:26,792][20347] Updated weights for policy 0, policy_version 1179813 (0.0007) [2023-12-27 13:49:26,817][20381] Updated weights for policy 1, policy_version 1178830 (0.0006) [2023-12-27 13:49:26,846][20347] Updated weights for policy 0, policy_version 1179823 (0.0008) [2023-12-27 13:49:26,876][20381] Updated weights for policy 1, policy_version 1178840 (0.0008) [2023-12-27 13:49:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 603897856. Throughput: 0: 10322.7, 1: 10254.3. Samples: 603889180. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:27,363][19170] Avg episode reward: [(0, '510.960'), (1, '696.340')] [2023-12-27 13:49:27,522][20347] Updated weights for policy 0, policy_version 1179833 (0.0007) [2023-12-27 13:49:27,540][20381] Updated weights for policy 1, policy_version 1178850 (0.0005) [2023-12-27 13:49:27,541][20292] KL-divergence is very high: 155.3643 [2023-12-27 13:49:27,572][20347] Updated weights for policy 0, policy_version 1179843 (0.0007) [2023-12-27 13:49:27,584][20292] KL-divergence is very high: 145.5553 [2023-12-27 13:49:27,596][20381] Updated weights for policy 1, policy_version 1178860 (0.0007) [2023-12-27 13:49:27,617][20347] Updated weights for policy 0, policy_version 1179853 (0.0005) [2023-12-27 13:49:27,628][20292] KL-divergence is very high: 111.9358 [2023-12-27 13:49:27,648][20381] Updated weights for policy 1, policy_version 1178870 (0.0006) [2023-12-27 13:49:28,217][20381] Updated weights for policy 1, policy_version 1178880 (0.0010) [2023-12-27 13:49:28,254][20347] Updated weights for policy 0, policy_version 1179863 (0.0008) [2023-12-27 13:49:28,265][20381] Updated weights for policy 1, policy_version 1178890 (0.0010) [2023-12-27 13:49:28,306][20347] Updated weights for policy 0, policy_version 1179873 (0.0008) [2023-12-27 13:49:28,317][20381] Updated weights for policy 1, policy_version 1178900 (0.0005) [2023-12-27 13:49:28,369][20347] Updated weights for policy 0, policy_version 1179883 (0.0007) [2023-12-27 13:49:28,929][20381] Updated weights for policy 1, policy_version 1178910 (0.0009) [2023-12-27 13:49:28,983][20381] Updated weights for policy 1, policy_version 1178920 (0.0010) [2023-12-27 13:49:29,037][20381] Updated weights for policy 1, policy_version 1178930 (0.0010) [2023-12-27 13:49:29,082][20347] Updated weights for policy 0, policy_version 1179893 (0.0007) [2023-12-27 13:49:29,144][20347] Updated weights for policy 0, policy_version 1179903 (0.0007) [2023-12-27 13:49:29,195][20347] Updated weights for policy 0, policy_version 1179913 (0.0008) [2023-12-27 13:49:29,736][20381] Updated weights for policy 1, policy_version 1178940 (0.0010) [2023-12-27 13:49:29,792][20381] Updated weights for policy 1, policy_version 1178950 (0.0009) [2023-12-27 13:49:29,828][20347] Updated weights for policy 0, policy_version 1179923 (0.0006) [2023-12-27 13:49:29,848][20381] Updated weights for policy 1, policy_version 1178960 (0.0007) [2023-12-27 13:49:29,890][20347] Updated weights for policy 0, policy_version 1179933 (0.0006) [2023-12-27 13:49:29,955][20347] Updated weights for policy 0, policy_version 1179943 (0.0008) [2023-12-27 13:49:30,570][20381] Updated weights for policy 1, policy_version 1178970 (0.0007) [2023-12-27 13:49:30,616][20347] Updated weights for policy 0, policy_version 1179953 (0.0008) [2023-12-27 13:49:30,618][20381] Updated weights for policy 1, policy_version 1178980 (0.0007) [2023-12-27 13:49:30,667][20381] Updated weights for policy 1, policy_version 1178990 (0.0006) [2023-12-27 13:49:30,669][20347] Updated weights for policy 0, policy_version 1179963 (0.0007) [2023-12-27 13:49:30,722][20347] Updated weights for policy 0, policy_version 1179973 (0.0005) [2023-12-27 13:49:30,724][20381] Updated weights for policy 1, policy_version 1179000 (0.0006) [2023-12-27 13:49:30,780][20347] Updated weights for policy 0, policy_version 1179983 (0.0007) [2023-12-27 13:49:31,430][20381] Updated weights for policy 1, policy_version 1179010 (0.0008) [2023-12-27 13:49:31,477][20381] Updated weights for policy 1, policy_version 1179020 (0.0008) [2023-12-27 13:49:31,526][20347] Updated weights for policy 0, policy_version 1179993 (0.0006) [2023-12-27 13:49:31,528][20381] Updated weights for policy 1, policy_version 1179030 (0.0007) [2023-12-27 13:49:31,584][20347] Updated weights for policy 0, policy_version 1180003 (0.0006) [2023-12-27 13:49:31,643][20347] Updated weights for policy 0, policy_version 1180013 (0.0007) [2023-12-27 13:49:32,293][20381] Updated weights for policy 1, policy_version 1179040 (0.0007) [2023-12-27 13:49:32,299][20347] Updated weights for policy 0, policy_version 1180023 (0.0006) [2023-12-27 13:49:32,348][20381] Updated weights for policy 1, policy_version 1179050 (0.0008) [2023-12-27 13:49:32,352][20347] Updated weights for policy 0, policy_version 1180033 (0.0008) [2023-12-27 13:49:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 603996160. Throughput: 0: 10290.1, 1: 10265.9. Samples: 604013192. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:32,363][19170] Avg episode reward: [(0, '727.100'), (1, '745.350')] [2023-12-27 13:49:32,409][20381] Updated weights for policy 1, policy_version 1179060 (0.0007) [2023-12-27 13:49:32,412][20347] Updated weights for policy 0, policy_version 1180043 (0.0007) [2023-12-27 13:49:33,063][20347] Updated weights for policy 0, policy_version 1180053 (0.0006) [2023-12-27 13:49:33,074][20381] Updated weights for policy 1, policy_version 1179070 (0.0008) [2023-12-27 13:49:33,119][20347] Updated weights for policy 0, policy_version 1180063 (0.0005) [2023-12-27 13:49:33,129][20381] Updated weights for policy 1, policy_version 1179080 (0.0007) [2023-12-27 13:49:33,177][20347] Updated weights for policy 0, policy_version 1180073 (0.0005) [2023-12-27 13:49:33,180][20381] Updated weights for policy 1, policy_version 1179090 (0.0008) [2023-12-27 13:49:33,765][20347] Updated weights for policy 0, policy_version 1180083 (0.0006) [2023-12-27 13:49:33,816][20347] Updated weights for policy 0, policy_version 1180093 (0.0006) [2023-12-27 13:49:33,828][20381] Updated weights for policy 1, policy_version 1179100 (0.0008) [2023-12-27 13:49:33,861][20347] Updated weights for policy 0, policy_version 1180103 (0.0006) [2023-12-27 13:49:33,885][20381] Updated weights for policy 1, policy_version 1179110 (0.0008) [2023-12-27 13:49:33,947][20381] Updated weights for policy 1, policy_version 1179120 (0.0008) [2023-12-27 13:49:34,462][20347] Updated weights for policy 0, policy_version 1180113 (0.0005) [2023-12-27 13:49:34,524][20347] Updated weights for policy 0, policy_version 1180123 (0.0007) [2023-12-27 13:49:34,586][20347] Updated weights for policy 0, policy_version 1180133 (0.0008) [2023-12-27 13:49:34,650][20347] Updated weights for policy 0, policy_version 1180143 (0.0007) [2023-12-27 13:49:34,651][20381] Updated weights for policy 1, policy_version 1179130 (0.0008) [2023-12-27 13:49:34,706][20381] Updated weights for policy 1, policy_version 1179140 (0.0009) [2023-12-27 13:49:34,764][20381] Updated weights for policy 1, policy_version 1179150 (0.0007) [2023-12-27 13:49:34,819][20381] Updated weights for policy 1, policy_version 1179160 (0.0009) [2023-12-27 13:49:35,278][20347] Updated weights for policy 0, policy_version 1180153 (0.0007) [2023-12-27 13:49:35,338][20347] Updated weights for policy 0, policy_version 1180163 (0.0007) [2023-12-27 13:49:35,397][20347] Updated weights for policy 0, policy_version 1180173 (0.0007) [2023-12-27 13:49:35,467][20381] Updated weights for policy 1, policy_version 1179170 (0.0010) [2023-12-27 13:49:35,526][20381] Updated weights for policy 1, policy_version 1179180 (0.0009) [2023-12-27 13:49:35,583][20381] Updated weights for policy 1, policy_version 1179190 (0.0005) [2023-12-27 13:49:36,141][20347] Updated weights for policy 0, policy_version 1180183 (0.0007) [2023-12-27 13:49:36,147][20381] Updated weights for policy 1, policy_version 1179200 (0.0008) [2023-12-27 13:49:36,198][20347] Updated weights for policy 0, policy_version 1180193 (0.0007) [2023-12-27 13:49:36,210][20381] Updated weights for policy 1, policy_version 1179210 (0.0007) [2023-12-27 13:49:36,258][20347] Updated weights for policy 0, policy_version 1180203 (0.0007) [2023-12-27 13:49:36,269][20381] Updated weights for policy 1, policy_version 1179220 (0.0006) [2023-12-27 13:49:36,846][20381] Updated weights for policy 1, policy_version 1179230 (0.0006) [2023-12-27 13:49:36,901][20381] Updated weights for policy 1, policy_version 1179240 (0.0005) [2023-12-27 13:49:36,934][20347] Updated weights for policy 0, policy_version 1180213 (0.0007) [2023-12-27 13:49:36,961][20381] Updated weights for policy 1, policy_version 1179250 (0.0005) [2023-12-27 13:49:36,992][20347] Updated weights for policy 0, policy_version 1180223 (0.0005) [2023-12-27 13:49:37,051][20347] Updated weights for policy 0, policy_version 1180233 (0.0006) [2023-12-27 13:49:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 604110848. Throughput: 0: 10358.6, 1: 10252.4. Samples: 604077828. Policy #0 lag: (min: 2.0, avg: 14.7, max: 34.0) [2023-12-27 13:49:37,363][19170] Avg episode reward: [(0, '746.910'), (1, '754.900')] [2023-12-27 13:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001179256_301916160.pth... [2023-12-27 13:49:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001180240_302194688.pth... [2023-12-27 13:49:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001179024_301883392.pth [2023-12-27 13:49:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001178040_301604864.pth [2023-12-27 13:49:37,529][20381] Updated weights for policy 1, policy_version 1179260 (0.0008) [2023-12-27 13:49:37,585][20381] Updated weights for policy 1, policy_version 1179270 (0.0007) [2023-12-27 13:49:37,640][20381] Updated weights for policy 1, policy_version 1179280 (0.0007) [2023-12-27 13:49:37,650][20347] Updated weights for policy 0, policy_version 1180243 (0.0005) [2023-12-27 13:49:37,702][20347] Updated weights for policy 0, policy_version 1180253 (0.0006) [2023-12-27 13:49:37,761][20347] Updated weights for policy 0, policy_version 1180263 (0.0008) [2023-12-27 13:49:38,237][20381] Updated weights for policy 1, policy_version 1179290 (0.0007) [2023-12-27 13:49:38,295][20381] Updated weights for policy 1, policy_version 1179300 (0.0010) [2023-12-27 13:49:38,351][20292] KL-divergence is very high: 415.4560 [2023-12-27 13:49:38,358][20381] Updated weights for policy 1, policy_version 1179310 (0.0010) [2023-12-27 13:49:38,406][20292] KL-divergence is very high: 252.5951 [2023-12-27 13:49:38,426][20381] Updated weights for policy 1, policy_version 1179320 (0.0006) [2023-12-27 13:49:38,433][20347] Updated weights for policy 0, policy_version 1180273 (0.0008) [2023-12-27 13:49:38,501][20347] Updated weights for policy 0, policy_version 1180283 (0.0009) [2023-12-27 13:49:38,566][20347] Updated weights for policy 0, policy_version 1180293 (0.0011) [2023-12-27 13:49:38,628][20347] Updated weights for policy 0, policy_version 1180303 (0.0011) [2023-12-27 13:49:39,060][20381] Updated weights for policy 1, policy_version 1179330 (0.0006) [2023-12-27 13:49:39,124][20381] Updated weights for policy 1, policy_version 1179340 (0.0006) [2023-12-27 13:49:39,177][20381] Updated weights for policy 1, policy_version 1179350 (0.0008) [2023-12-27 13:49:39,231][20347] Updated weights for policy 0, policy_version 1180313 (0.0008) [2023-12-27 13:49:39,289][20347] Updated weights for policy 0, policy_version 1180323 (0.0007) [2023-12-27 13:49:39,348][20347] Updated weights for policy 0, policy_version 1180333 (0.0010) [2023-12-27 13:49:39,904][20381] Updated weights for policy 1, policy_version 1179360 (0.0007) [2023-12-27 13:49:39,966][20381] Updated weights for policy 1, policy_version 1179370 (0.0008) [2023-12-27 13:49:40,025][20381] Updated weights for policy 1, policy_version 1179380 (0.0008) [2023-12-27 13:49:40,045][20347] Updated weights for policy 0, policy_version 1180343 (0.0008) [2023-12-27 13:49:40,111][20347] Updated weights for policy 0, policy_version 1180353 (0.0008) [2023-12-27 13:49:40,179][20347] Updated weights for policy 0, policy_version 1180363 (0.0008) [2023-12-27 13:49:40,748][20381] Updated weights for policy 1, policy_version 1179390 (0.0007) [2023-12-27 13:49:40,794][20381] Updated weights for policy 1, policy_version 1179400 (0.0007) [2023-12-27 13:49:40,846][20381] Updated weights for policy 1, policy_version 1179410 (0.0007) [2023-12-27 13:49:40,851][20347] Updated weights for policy 0, policy_version 1180373 (0.0007) [2023-12-27 13:49:40,908][20347] Updated weights for policy 0, policy_version 1180383 (0.0007) [2023-12-27 13:49:40,962][20347] Updated weights for policy 0, policy_version 1180393 (0.0007) [2023-12-27 13:49:41,631][20381] Updated weights for policy 1, policy_version 1179420 (0.0006) [2023-12-27 13:49:41,694][20381] Updated weights for policy 1, policy_version 1179430 (0.0008) [2023-12-27 13:49:41,742][20347] Updated weights for policy 0, policy_version 1180403 (0.0009) [2023-12-27 13:49:41,765][20381] Updated weights for policy 1, policy_version 1179440 (0.0007) [2023-12-27 13:49:41,807][20347] Updated weights for policy 0, policy_version 1180413 (0.0008) [2023-12-27 13:49:41,867][20347] Updated weights for policy 0, policy_version 1180423 (0.0010) [2023-12-27 13:49:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 604209152. Throughput: 0: 10321.6, 1: 10297.8. Samples: 604202404. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:49:42,363][19170] Avg episode reward: [(0, '738.320'), (1, '755.430')] [2023-12-27 13:49:42,409][20381] Updated weights for policy 1, policy_version 1179450 (0.0006) [2023-12-27 13:49:42,471][20381] Updated weights for policy 1, policy_version 1179460 (0.0008) [2023-12-27 13:49:42,527][20381] Updated weights for policy 1, policy_version 1179470 (0.0007) [2023-12-27 13:49:42,557][20347] Updated weights for policy 0, policy_version 1180433 (0.0010) [2023-12-27 13:49:42,588][20381] Updated weights for policy 1, policy_version 1179480 (0.0006) [2023-12-27 13:49:42,619][20347] Updated weights for policy 0, policy_version 1180443 (0.0011) [2023-12-27 13:49:42,665][20347] Updated weights for policy 0, policy_version 1180453 (0.0010) [2023-12-27 13:49:42,721][20347] Updated weights for policy 0, policy_version 1180463 (0.0010) [2023-12-27 13:49:43,300][20381] Updated weights for policy 1, policy_version 1179490 (0.0008) [2023-12-27 13:49:43,362][20381] Updated weights for policy 1, policy_version 1179500 (0.0008) [2023-12-27 13:49:43,398][20347] Updated weights for policy 0, policy_version 1180473 (0.0006) [2023-12-27 13:49:43,419][20381] Updated weights for policy 1, policy_version 1179510 (0.0007) [2023-12-27 13:49:43,459][20347] Updated weights for policy 0, policy_version 1180483 (0.0005) [2023-12-27 13:49:43,515][20347] Updated weights for policy 0, policy_version 1180493 (0.0006) [2023-12-27 13:49:44,078][20347] Updated weights for policy 0, policy_version 1180503 (0.0006) [2023-12-27 13:49:44,132][20347] Updated weights for policy 0, policy_version 1180513 (0.0005) [2023-12-27 13:49:44,156][20381] Updated weights for policy 1, policy_version 1179520 (0.0009) [2023-12-27 13:49:44,188][20347] Updated weights for policy 0, policy_version 1180523 (0.0005) [2023-12-27 13:49:44,214][20381] Updated weights for policy 1, policy_version 1179530 (0.0007) [2023-12-27 13:49:44,265][20381] Updated weights for policy 1, policy_version 1179540 (0.0007) [2023-12-27 13:49:44,827][20347] Updated weights for policy 0, policy_version 1180533 (0.0007) [2023-12-27 13:49:44,887][20347] Updated weights for policy 0, policy_version 1180543 (0.0007) [2023-12-27 13:49:44,948][20347] Updated weights for policy 0, policy_version 1180553 (0.0008) [2023-12-27 13:49:44,996][20381] Updated weights for policy 1, policy_version 1179550 (0.0007) [2023-12-27 13:49:45,057][20381] Updated weights for policy 1, policy_version 1179560 (0.0008) [2023-12-27 13:49:45,116][20381] Updated weights for policy 1, policy_version 1179570 (0.0006) [2023-12-27 13:49:45,703][20347] Updated weights for policy 0, policy_version 1180563 (0.0006) [2023-12-27 13:49:45,747][20347] Updated weights for policy 0, policy_version 1180573 (0.0007) [2023-12-27 13:49:45,782][20381] Updated weights for policy 1, policy_version 1179580 (0.0005) [2023-12-27 13:49:45,798][20347] Updated weights for policy 0, policy_version 1180583 (0.0007) [2023-12-27 13:49:45,840][20381] Updated weights for policy 1, policy_version 1179590 (0.0007) [2023-12-27 13:49:45,895][20381] Updated weights for policy 1, policy_version 1179600 (0.0008) [2023-12-27 13:49:46,442][20347] Updated weights for policy 0, policy_version 1180593 (0.0006) [2023-12-27 13:49:46,500][20347] Updated weights for policy 0, policy_version 1180603 (0.0010) [2023-12-27 13:49:46,556][20347] Updated weights for policy 0, policy_version 1180613 (0.0008) [2023-12-27 13:49:46,599][20381] Updated weights for policy 1, policy_version 1179610 (0.0007) [2023-12-27 13:49:46,616][20347] Updated weights for policy 0, policy_version 1180623 (0.0010) [2023-12-27 13:49:46,657][20381] Updated weights for policy 1, policy_version 1179620 (0.0005) [2023-12-27 13:49:46,713][20381] Updated weights for policy 1, policy_version 1179630 (0.0005) [2023-12-27 13:49:46,770][20381] Updated weights for policy 1, policy_version 1179640 (0.0007) [2023-12-27 13:49:47,265][20347] Updated weights for policy 0, policy_version 1180633 (0.0008) [2023-12-27 13:49:47,310][20347] Updated weights for policy 0, policy_version 1180643 (0.0008) [2023-12-27 13:49:47,355][20347] Updated weights for policy 0, policy_version 1180653 (0.0008) [2023-12-27 13:49:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 604307456. Throughput: 0: 10362.0, 1: 10276.9. Samples: 604323800. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:49:47,363][19170] Avg episode reward: [(0, '778.810'), (1, '735.360')] [2023-12-27 13:49:47,435][20381] Updated weights for policy 1, policy_version 1179650 (0.0009) [2023-12-27 13:49:47,483][20381] Updated weights for policy 1, policy_version 1179660 (0.0008) [2023-12-27 13:49:47,533][20381] Updated weights for policy 1, policy_version 1179670 (0.0009) [2023-12-27 13:49:47,949][20347] Updated weights for policy 0, policy_version 1180663 (0.0010) [2023-12-27 13:49:48,009][20347] Updated weights for policy 0, policy_version 1180673 (0.0010) [2023-12-27 13:49:48,067][20347] Updated weights for policy 0, policy_version 1180683 (0.0010) [2023-12-27 13:49:48,228][20381] Updated weights for policy 1, policy_version 1179680 (0.0008) [2023-12-27 13:49:48,280][20381] Updated weights for policy 1, policy_version 1179690 (0.0007) [2023-12-27 13:49:48,331][20381] Updated weights for policy 1, policy_version 1179700 (0.0008) [2023-12-27 13:49:48,803][20347] Updated weights for policy 0, policy_version 1180693 (0.0009) [2023-12-27 13:49:48,856][20347] Updated weights for policy 0, policy_version 1180703 (0.0007) [2023-12-27 13:49:48,916][20347] Updated weights for policy 0, policy_version 1180713 (0.0007) [2023-12-27 13:49:48,995][20381] Updated weights for policy 1, policy_version 1179710 (0.0008) [2023-12-27 13:49:49,040][20381] Updated weights for policy 1, policy_version 1179720 (0.0007) [2023-12-27 13:49:49,095][20381] Updated weights for policy 1, policy_version 1179730 (0.0007) [2023-12-27 13:49:49,648][20347] Updated weights for policy 0, policy_version 1180723 (0.0007) [2023-12-27 13:49:49,705][20347] Updated weights for policy 0, policy_version 1180733 (0.0007) [2023-12-27 13:49:49,742][20381] Updated weights for policy 1, policy_version 1179740 (0.0007) [2023-12-27 13:49:49,755][20347] Updated weights for policy 0, policy_version 1180743 (0.0006) [2023-12-27 13:49:49,792][20381] Updated weights for policy 1, policy_version 1179750 (0.0006) [2023-12-27 13:49:49,846][20381] Updated weights for policy 1, policy_version 1179760 (0.0007) [2023-12-27 13:49:50,496][20347] Updated weights for policy 0, policy_version 1180753 (0.0008) [2023-12-27 13:49:50,552][20381] Updated weights for policy 1, policy_version 1179770 (0.0008) [2023-12-27 13:49:50,558][20347] Updated weights for policy 0, policy_version 1180763 (0.0007) [2023-12-27 13:49:50,615][20347] Updated weights for policy 0, policy_version 1180773 (0.0008) [2023-12-27 13:49:50,620][20381] Updated weights for policy 1, policy_version 1179780 (0.0007) [2023-12-27 13:49:50,671][20347] Updated weights for policy 0, policy_version 1180783 (0.0006) [2023-12-27 13:49:50,683][20381] Updated weights for policy 1, policy_version 1179790 (0.0008) [2023-12-27 13:49:50,748][20381] Updated weights for policy 1, policy_version 1179800 (0.0008) [2023-12-27 13:49:51,388][20347] Updated weights for policy 0, policy_version 1180793 (0.0007) [2023-12-27 13:49:51,428][20381] Updated weights for policy 1, policy_version 1179810 (0.0006) [2023-12-27 13:49:51,448][20347] Updated weights for policy 0, policy_version 1180803 (0.0007) [2023-12-27 13:49:51,489][20381] Updated weights for policy 1, policy_version 1179820 (0.0006) [2023-12-27 13:49:51,503][20347] Updated weights for policy 0, policy_version 1180813 (0.0008) [2023-12-27 13:49:51,555][20381] Updated weights for policy 1, policy_version 1179830 (0.0006) [2023-12-27 13:49:52,186][20381] Updated weights for policy 1, policy_version 1179840 (0.0008) [2023-12-27 13:49:52,247][20381] Updated weights for policy 1, policy_version 1179850 (0.0007) [2023-12-27 13:49:52,267][20347] Updated weights for policy 0, policy_version 1180823 (0.0008) [2023-12-27 13:49:52,311][20381] Updated weights for policy 1, policy_version 1179860 (0.0006) [2023-12-27 13:49:52,321][20347] Updated weights for policy 0, policy_version 1180833 (0.0005) [2023-12-27 13:49:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 604413952. Throughput: 0: 10352.9, 1: 10272.0. Samples: 604385380. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:49:52,363][19170] Avg episode reward: [(0, '851.730'), (1, '725.870')] [2023-12-27 13:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001179864_302071808.pth... [2023-12-27 13:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001178648_301760512.pth [2023-12-27 13:49:52,386][20347] Updated weights for policy 0, policy_version 1180843 (0.0008) [2023-12-27 13:49:52,417][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001180848_302350336.pth... [2023-12-27 13:49:52,422][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001179632_302039040.pth [2023-12-27 13:49:53,014][20381] Updated weights for policy 1, policy_version 1179870 (0.0007) [2023-12-27 13:49:53,044][20347] Updated weights for policy 0, policy_version 1180853 (0.0007) [2023-12-27 13:49:53,069][20381] Updated weights for policy 1, policy_version 1179880 (0.0008) [2023-12-27 13:49:53,106][20347] Updated weights for policy 0, policy_version 1180863 (0.0005) [2023-12-27 13:49:53,126][20381] Updated weights for policy 1, policy_version 1179890 (0.0007) [2023-12-27 13:49:53,169][20347] Updated weights for policy 0, policy_version 1180873 (0.0006) [2023-12-27 13:49:53,741][20381] Updated weights for policy 1, policy_version 1179900 (0.0008) [2023-12-27 13:49:53,791][20347] Updated weights for policy 0, policy_version 1180883 (0.0007) [2023-12-27 13:49:53,793][20381] Updated weights for policy 1, policy_version 1179910 (0.0007) [2023-12-27 13:49:53,843][20347] Updated weights for policy 0, policy_version 1180893 (0.0005) [2023-12-27 13:49:53,851][20381] Updated weights for policy 1, policy_version 1179920 (0.0007) [2023-12-27 13:49:53,900][20347] Updated weights for policy 0, policy_version 1180903 (0.0006) [2023-12-27 13:49:54,537][20381] Updated weights for policy 1, policy_version 1179930 (0.0008) [2023-12-27 13:49:54,547][20347] Updated weights for policy 0, policy_version 1180913 (0.0010) [2023-12-27 13:49:54,585][20381] Updated weights for policy 1, policy_version 1179940 (0.0006) [2023-12-27 13:49:54,612][20347] Updated weights for policy 0, policy_version 1180923 (0.0010) [2023-12-27 13:49:54,635][20381] Updated weights for policy 1, policy_version 1179950 (0.0008) [2023-12-27 13:49:54,671][20347] Updated weights for policy 0, policy_version 1180933 (0.0008) [2023-12-27 13:49:54,684][20381] Updated weights for policy 1, policy_version 1179960 (0.0006) [2023-12-27 13:49:54,723][20347] Updated weights for policy 0, policy_version 1180943 (0.0006) [2023-12-27 13:49:55,333][20381] Updated weights for policy 1, policy_version 1179970 (0.0008) [2023-12-27 13:49:55,339][20347] Updated weights for policy 0, policy_version 1180953 (0.0007) [2023-12-27 13:49:55,387][20381] Updated weights for policy 1, policy_version 1179980 (0.0010) [2023-12-27 13:49:55,403][20347] Updated weights for policy 0, policy_version 1180963 (0.0008) [2023-12-27 13:49:55,438][20381] Updated weights for policy 1, policy_version 1179990 (0.0010) [2023-12-27 13:49:55,461][20347] Updated weights for policy 0, policy_version 1180973 (0.0006) [2023-12-27 13:49:56,083][20381] Updated weights for policy 1, policy_version 1180000 (0.0010) [2023-12-27 13:49:56,110][20347] Updated weights for policy 0, policy_version 1180983 (0.0006) [2023-12-27 13:49:56,138][20381] Updated weights for policy 1, policy_version 1180010 (0.0010) [2023-12-27 13:49:56,163][20347] Updated weights for policy 0, policy_version 1180993 (0.0006) [2023-12-27 13:49:56,197][20381] Updated weights for policy 1, policy_version 1180020 (0.0010) [2023-12-27 13:49:56,221][20347] Updated weights for policy 0, policy_version 1181003 (0.0005) [2023-12-27 13:49:56,779][20381] Updated weights for policy 1, policy_version 1180030 (0.0007) [2023-12-27 13:49:56,842][20381] Updated weights for policy 1, policy_version 1180040 (0.0005) [2023-12-27 13:49:56,869][20347] Updated weights for policy 0, policy_version 1181013 (0.0008) [2023-12-27 13:49:56,902][20381] Updated weights for policy 1, policy_version 1180050 (0.0006) [2023-12-27 13:49:56,928][20347] Updated weights for policy 0, policy_version 1181023 (0.0007) [2023-12-27 13:49:56,980][20347] Updated weights for policy 0, policy_version 1181033 (0.0008) [2023-12-27 13:49:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 604520448. Throughput: 0: 10334.8, 1: 10294.2. Samples: 604508992. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:49:57,363][19170] Avg episode reward: [(0, '844.750'), (1, '755.170')] [2023-12-27 13:49:57,513][20381] Updated weights for policy 1, policy_version 1180060 (0.0007) [2023-12-27 13:49:57,577][20381] Updated weights for policy 1, policy_version 1180070 (0.0008) [2023-12-27 13:49:57,614][20347] Updated weights for policy 0, policy_version 1181043 (0.0007) [2023-12-27 13:49:57,638][20381] Updated weights for policy 1, policy_version 1180080 (0.0008) [2023-12-27 13:49:57,662][20347] Updated weights for policy 0, policy_version 1181053 (0.0005) [2023-12-27 13:49:57,712][20347] Updated weights for policy 0, policy_version 1181063 (0.0006) [2023-12-27 13:49:58,284][20381] Updated weights for policy 1, policy_version 1180090 (0.0007) [2023-12-27 13:49:58,353][20381] Updated weights for policy 1, policy_version 1180100 (0.0007) [2023-12-27 13:49:58,420][20381] Updated weights for policy 1, policy_version 1180110 (0.0008) [2023-12-27 13:49:58,442][20347] Updated weights for policy 0, policy_version 1181073 (0.0008) [2023-12-27 13:49:58,481][20381] Updated weights for policy 1, policy_version 1180120 (0.0008) [2023-12-27 13:49:58,500][20347] Updated weights for policy 0, policy_version 1181083 (0.0008) [2023-12-27 13:49:58,563][20347] Updated weights for policy 0, policy_version 1181093 (0.0008) [2023-12-27 13:49:58,630][20347] Updated weights for policy 0, policy_version 1181103 (0.0009) [2023-12-27 13:49:59,188][20381] Updated weights for policy 1, policy_version 1180130 (0.0008) [2023-12-27 13:49:59,255][20381] Updated weights for policy 1, policy_version 1180140 (0.0008) [2023-12-27 13:49:59,318][20381] Updated weights for policy 1, policy_version 1180150 (0.0007) [2023-12-27 13:49:59,463][20347] Updated weights for policy 0, policy_version 1181113 (0.0007) [2023-12-27 13:49:59,511][20347] Updated weights for policy 0, policy_version 1181123 (0.0006) [2023-12-27 13:49:59,569][20347] Updated weights for policy 0, policy_version 1181133 (0.0006) [2023-12-27 13:50:00,105][20381] Updated weights for policy 1, policy_version 1180160 (0.0010) [2023-12-27 13:50:00,164][20381] Updated weights for policy 1, policy_version 1180170 (0.0010) [2023-12-27 13:50:00,218][20381] Updated weights for policy 1, policy_version 1180180 (0.0006) [2023-12-27 13:50:00,287][20347] Updated weights for policy 0, policy_version 1181143 (0.0007) [2023-12-27 13:50:00,342][20347] Updated weights for policy 0, policy_version 1181153 (0.0007) [2023-12-27 13:50:00,399][20347] Updated weights for policy 0, policy_version 1181163 (0.0007) [2023-12-27 13:50:00,840][20381] Updated weights for policy 1, policy_version 1180190 (0.0010) [2023-12-27 13:50:00,892][20381] Updated weights for policy 1, policy_version 1180200 (0.0010) [2023-12-27 13:50:00,953][20381] Updated weights for policy 1, policy_version 1180210 (0.0010) [2023-12-27 13:50:01,144][20347] Updated weights for policy 0, policy_version 1181173 (0.0007) [2023-12-27 13:50:01,213][20347] Updated weights for policy 0, policy_version 1181183 (0.0006) [2023-12-27 13:50:01,284][20347] Updated weights for policy 0, policy_version 1181193 (0.0008) [2023-12-27 13:50:01,688][20381] Updated weights for policy 1, policy_version 1180220 (0.0008) [2023-12-27 13:50:01,757][20381] Updated weights for policy 1, policy_version 1180230 (0.0008) [2023-12-27 13:50:01,809][20381] Updated weights for policy 1, policy_version 1180240 (0.0008) [2023-12-27 13:50:01,947][20347] Updated weights for policy 0, policy_version 1181203 (0.0009) [2023-12-27 13:50:02,009][20347] Updated weights for policy 0, policy_version 1181213 (0.0008) [2023-12-27 13:50:02,073][20347] Updated weights for policy 0, policy_version 1181223 (0.0008) [2023-12-27 13:50:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 604618752. Throughput: 0: 10251.3, 1: 10312.5. Samples: 604629908. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:02,363][19170] Avg episode reward: [(0, '815.100'), (1, '706.580')] [2023-12-27 13:50:02,481][20381] Updated weights for policy 1, policy_version 1180250 (0.0007) [2023-12-27 13:50:02,538][20381] Updated weights for policy 1, policy_version 1180260 (0.0008) [2023-12-27 13:50:02,592][20381] Updated weights for policy 1, policy_version 1180270 (0.0008) [2023-12-27 13:50:02,649][20381] Updated weights for policy 1, policy_version 1180280 (0.0008) [2023-12-27 13:50:02,759][20347] Updated weights for policy 0, policy_version 1181233 (0.0007) [2023-12-27 13:50:02,820][20347] Updated weights for policy 0, policy_version 1181243 (0.0005) [2023-12-27 13:50:02,871][20347] Updated weights for policy 0, policy_version 1181253 (0.0005) [2023-12-27 13:50:02,924][20347] Updated weights for policy 0, policy_version 1181263 (0.0006) [2023-12-27 13:50:03,364][20381] Updated weights for policy 1, policy_version 1180290 (0.0009) [2023-12-27 13:50:03,409][20381] Updated weights for policy 1, policy_version 1180300 (0.0010) [2023-12-27 13:50:03,453][20381] Updated weights for policy 1, policy_version 1180310 (0.0010) [2023-12-27 13:50:03,566][20347] Updated weights for policy 0, policy_version 1181273 (0.0005) [2023-12-27 13:50:03,616][20347] Updated weights for policy 0, policy_version 1181283 (0.0006) [2023-12-27 13:50:03,672][20347] Updated weights for policy 0, policy_version 1181293 (0.0007) [2023-12-27 13:50:04,068][20381] Updated weights for policy 1, policy_version 1180320 (0.0006) [2023-12-27 13:50:04,127][20381] Updated weights for policy 1, policy_version 1180330 (0.0006) [2023-12-27 13:50:04,190][20381] Updated weights for policy 1, policy_version 1180340 (0.0006) [2023-12-27 13:50:04,387][20347] Updated weights for policy 0, policy_version 1181303 (0.0009) [2023-12-27 13:50:04,443][20347] Updated weights for policy 0, policy_version 1181313 (0.0007) [2023-12-27 13:50:04,496][20347] Updated weights for policy 0, policy_version 1181323 (0.0007) [2023-12-27 13:50:04,806][20381] Updated weights for policy 1, policy_version 1180350 (0.0006) [2023-12-27 13:50:04,856][20381] Updated weights for policy 1, policy_version 1180360 (0.0005) [2023-12-27 13:50:04,905][20381] Updated weights for policy 1, policy_version 1180370 (0.0005) [2023-12-27 13:50:05,238][20347] Updated weights for policy 0, policy_version 1181333 (0.0008) [2023-12-27 13:50:05,296][20347] Updated weights for policy 0, policy_version 1181343 (0.0008) [2023-12-27 13:50:05,346][20347] Updated weights for policy 0, policy_version 1181353 (0.0008) [2023-12-27 13:50:05,500][20381] Updated weights for policy 1, policy_version 1180380 (0.0006) [2023-12-27 13:50:05,555][20381] Updated weights for policy 1, policy_version 1180390 (0.0007) [2023-12-27 13:50:05,606][20381] Updated weights for policy 1, policy_version 1180400 (0.0007) [2023-12-27 13:50:05,997][20347] Updated weights for policy 0, policy_version 1181363 (0.0007) [2023-12-27 13:50:06,046][20347] Updated weights for policy 0, policy_version 1181373 (0.0005) [2023-12-27 13:50:06,109][20347] Updated weights for policy 0, policy_version 1181383 (0.0006) [2023-12-27 13:50:06,326][20381] Updated weights for policy 1, policy_version 1180410 (0.0007) [2023-12-27 13:50:06,386][20381] Updated weights for policy 1, policy_version 1180420 (0.0007) [2023-12-27 13:50:06,451][20381] Updated weights for policy 1, policy_version 1180430 (0.0006) [2023-12-27 13:50:06,510][20381] Updated weights for policy 1, policy_version 1180440 (0.0007) [2023-12-27 13:50:06,817][20347] Updated weights for policy 0, policy_version 1181393 (0.0008) [2023-12-27 13:50:06,870][20347] Updated weights for policy 0, policy_version 1181403 (0.0007) [2023-12-27 13:50:06,921][20347] Updated weights for policy 0, policy_version 1181413 (0.0009) [2023-12-27 13:50:06,973][20347] Updated weights for policy 0, policy_version 1181423 (0.0007) [2023-12-27 13:50:07,103][20381] Updated weights for policy 1, policy_version 1180450 (0.0008) [2023-12-27 13:50:07,162][20381] Updated weights for policy 1, policy_version 1180460 (0.0008) [2023-12-27 13:50:07,216][20381] Updated weights for policy 1, policy_version 1180470 (0.0009) [2023-12-27 13:50:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 604725248. Throughput: 0: 10195.7, 1: 10371.9. Samples: 604692324. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:07,364][19170] Avg episode reward: [(0, '784.990'), (1, '716.110')] [2023-12-27 13:50:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001180472_302227456.pth... [2023-12-27 13:50:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001181424_302497792.pth... [2023-12-27 13:50:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001179256_301916160.pth [2023-12-27 13:50:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001180240_302194688.pth [2023-12-27 13:50:07,649][20347] Updated weights for policy 0, policy_version 1181433 (0.0008) [2023-12-27 13:50:07,712][20347] Updated weights for policy 0, policy_version 1181443 (0.0007) [2023-12-27 13:50:07,771][20347] Updated weights for policy 0, policy_version 1181453 (0.0007) [2023-12-27 13:50:07,862][20381] Updated weights for policy 1, policy_version 1180480 (0.0010) [2023-12-27 13:50:07,914][20381] Updated weights for policy 1, policy_version 1180490 (0.0010) [2023-12-27 13:50:07,966][20381] Updated weights for policy 1, policy_version 1180500 (0.0010) [2023-12-27 13:50:08,457][20347] Updated weights for policy 0, policy_version 1181463 (0.0008) [2023-12-27 13:50:08,508][20347] Updated weights for policy 0, policy_version 1181473 (0.0008) [2023-12-27 13:50:08,565][20347] Updated weights for policy 0, policy_version 1181483 (0.0005) [2023-12-27 13:50:08,639][20381] Updated weights for policy 1, policy_version 1180510 (0.0008) [2023-12-27 13:50:08,701][20381] Updated weights for policy 1, policy_version 1180520 (0.0006) [2023-12-27 13:50:08,760][20381] Updated weights for policy 1, policy_version 1180530 (0.0006) [2023-12-27 13:50:09,227][20347] Updated weights for policy 0, policy_version 1181493 (0.0006) [2023-12-27 13:50:09,289][20347] Updated weights for policy 0, policy_version 1181503 (0.0007) [2023-12-27 13:50:09,349][20347] Updated weights for policy 0, policy_version 1181513 (0.0005) [2023-12-27 13:50:09,434][20381] Updated weights for policy 1, policy_version 1180540 (0.0008) [2023-12-27 13:50:09,497][20381] Updated weights for policy 1, policy_version 1180550 (0.0008) [2023-12-27 13:50:09,554][20381] Updated weights for policy 1, policy_version 1180560 (0.0008) [2023-12-27 13:50:09,993][20347] Updated weights for policy 0, policy_version 1181523 (0.0008) [2023-12-27 13:50:10,058][20347] Updated weights for policy 0, policy_version 1181533 (0.0008) [2023-12-27 13:50:10,129][20347] Updated weights for policy 0, policy_version 1181543 (0.0008) [2023-12-27 13:50:10,268][20381] Updated weights for policy 1, policy_version 1180570 (0.0007) [2023-12-27 13:50:10,336][20381] Updated weights for policy 1, policy_version 1180580 (0.0007) [2023-12-27 13:50:10,403][20381] Updated weights for policy 1, policy_version 1180590 (0.0007) [2023-12-27 13:50:10,463][20381] Updated weights for policy 1, policy_version 1180600 (0.0007) [2023-12-27 13:50:10,845][20347] Updated weights for policy 0, policy_version 1181553 (0.0008) [2023-12-27 13:50:10,901][20347] Updated weights for policy 0, policy_version 1181563 (0.0008) [2023-12-27 13:50:10,955][20347] Updated weights for policy 0, policy_version 1181573 (0.0008) [2023-12-27 13:50:11,005][20347] Updated weights for policy 0, policy_version 1181583 (0.0007) [2023-12-27 13:50:11,087][20381] Updated weights for policy 1, policy_version 1180610 (0.0008) [2023-12-27 13:50:11,152][20381] Updated weights for policy 1, policy_version 1180620 (0.0008) [2023-12-27 13:50:11,213][20381] Updated weights for policy 1, policy_version 1180630 (0.0006) [2023-12-27 13:50:11,792][20347] Updated weights for policy 0, policy_version 1181593 (0.0008) [2023-12-27 13:50:11,858][20347] Updated weights for policy 0, policy_version 1181603 (0.0007) [2023-12-27 13:50:11,912][20347] Updated weights for policy 0, policy_version 1181613 (0.0008) [2023-12-27 13:50:11,922][20291] KL-divergence is very high: 104.6876 [2023-12-27 13:50:12,012][20381] Updated weights for policy 1, policy_version 1180640 (0.0008) [2023-12-27 13:50:12,064][20381] Updated weights for policy 1, policy_version 1180650 (0.0009) [2023-12-27 13:50:12,114][20381] Updated weights for policy 1, policy_version 1180660 (0.0008) [2023-12-27 13:50:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 604823552. Throughput: 0: 10196.2, 1: 10371.1. Samples: 604814712. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:12,363][19170] Avg episode reward: [(0, '795.360'), (1, '657.570')] [2023-12-27 13:50:12,619][20347] Updated weights for policy 0, policy_version 1181623 (0.0007) [2023-12-27 13:50:12,693][20347] Updated weights for policy 0, policy_version 1181633 (0.0007) [2023-12-27 13:50:12,756][20347] Updated weights for policy 0, policy_version 1181643 (0.0007) [2023-12-27 13:50:12,788][20381] Updated weights for policy 1, policy_version 1180670 (0.0006) [2023-12-27 13:50:12,846][20381] Updated weights for policy 1, policy_version 1180680 (0.0006) [2023-12-27 13:50:12,901][20381] Updated weights for policy 1, policy_version 1180690 (0.0006) [2023-12-27 13:50:13,367][20347] Updated weights for policy 0, policy_version 1181653 (0.0006) [2023-12-27 13:50:13,421][20347] Updated weights for policy 0, policy_version 1181663 (0.0005) [2023-12-27 13:50:13,474][20347] Updated weights for policy 0, policy_version 1181673 (0.0005) [2023-12-27 13:50:13,575][20381] Updated weights for policy 1, policy_version 1180700 (0.0007) [2023-12-27 13:50:13,623][20381] Updated weights for policy 1, policy_version 1180710 (0.0008) [2023-12-27 13:50:13,685][20381] Updated weights for policy 1, policy_version 1180720 (0.0005) [2023-12-27 13:50:13,713][20292] KL-divergence is very high: 155.4488 [2023-12-27 13:50:14,071][20347] Updated weights for policy 0, policy_version 1181683 (0.0005) [2023-12-27 13:50:14,133][20347] Updated weights for policy 0, policy_version 1181693 (0.0005) [2023-12-27 13:50:14,192][20347] Updated weights for policy 0, policy_version 1181703 (0.0005) [2023-12-27 13:50:14,346][20381] Updated weights for policy 1, policy_version 1180730 (0.0008) [2023-12-27 13:50:14,397][20381] Updated weights for policy 1, policy_version 1180740 (0.0008) [2023-12-27 13:50:14,402][20292] KL-divergence is very high: 101.6577 [2023-12-27 13:50:14,454][20381] Updated weights for policy 1, policy_version 1180750 (0.0006) [2023-12-27 13:50:14,512][20381] Updated weights for policy 1, policy_version 1180760 (0.0008) [2023-12-27 13:50:14,753][20347] Updated weights for policy 0, policy_version 1181713 (0.0006) [2023-12-27 13:50:14,821][20347] Updated weights for policy 0, policy_version 1181723 (0.0008) [2023-12-27 13:50:14,881][20347] Updated weights for policy 0, policy_version 1181733 (0.0007) [2023-12-27 13:50:14,948][20347] Updated weights for policy 0, policy_version 1181743 (0.0006) [2023-12-27 13:50:15,258][20381] Updated weights for policy 1, policy_version 1180770 (0.0006) [2023-12-27 13:50:15,317][20381] Updated weights for policy 1, policy_version 1180780 (0.0007) [2023-12-27 13:50:15,361][20292] KL-divergence is very high: 104.6022 [2023-12-27 13:50:15,379][20381] Updated weights for policy 1, policy_version 1180790 (0.0007) [2023-12-27 13:50:15,636][20347] Updated weights for policy 0, policy_version 1181753 (0.0007) [2023-12-27 13:50:15,694][20347] Updated weights for policy 0, policy_version 1181763 (0.0008) [2023-12-27 13:50:15,755][20347] Updated weights for policy 0, policy_version 1181773 (0.0007) [2023-12-27 13:50:16,052][20381] Updated weights for policy 1, policy_version 1180800 (0.0006) [2023-12-27 13:50:16,103][20381] Updated weights for policy 1, policy_version 1180810 (0.0005) [2023-12-27 13:50:16,156][20381] Updated weights for policy 1, policy_version 1180820 (0.0006) [2023-12-27 13:50:16,423][20347] Updated weights for policy 0, policy_version 1181783 (0.0006) [2023-12-27 13:50:16,472][20347] Updated weights for policy 0, policy_version 1181793 (0.0005) [2023-12-27 13:50:16,525][20347] Updated weights for policy 0, policy_version 1181803 (0.0006) [2023-12-27 13:50:16,855][20381] Updated weights for policy 1, policy_version 1180830 (0.0007) [2023-12-27 13:50:16,912][20381] Updated weights for policy 1, policy_version 1180840 (0.0007) [2023-12-27 13:50:16,972][20381] Updated weights for policy 1, policy_version 1180850 (0.0006) [2023-12-27 13:50:17,118][20347] Updated weights for policy 0, policy_version 1181813 (0.0006) [2023-12-27 13:50:17,170][20347] Updated weights for policy 0, policy_version 1181823 (0.0005) [2023-12-27 13:50:17,221][20347] Updated weights for policy 0, policy_version 1181833 (0.0007) [2023-12-27 13:50:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 604930048. Throughput: 0: 10249.9, 1: 10296.1. Samples: 604937760. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:17,363][19170] Avg episode reward: [(0, '776.140'), (1, '687.740')] [2023-12-27 13:50:17,672][20381] Updated weights for policy 1, policy_version 1180860 (0.0008) [2023-12-27 13:50:17,726][20381] Updated weights for policy 1, policy_version 1180870 (0.0008) [2023-12-27 13:50:17,785][20381] Updated weights for policy 1, policy_version 1180880 (0.0007) [2023-12-27 13:50:17,853][20347] Updated weights for policy 0, policy_version 1181843 (0.0008) [2023-12-27 13:50:17,904][20347] Updated weights for policy 0, policy_version 1181853 (0.0006) [2023-12-27 13:50:17,952][20347] Updated weights for policy 0, policy_version 1181863 (0.0005) [2023-12-27 13:50:18,548][20347] Updated weights for policy 0, policy_version 1181873 (0.0005) [2023-12-27 13:50:18,578][20381] Updated weights for policy 1, policy_version 1180890 (0.0007) [2023-12-27 13:50:18,605][20347] Updated weights for policy 0, policy_version 1181883 (0.0007) [2023-12-27 13:50:18,635][20381] Updated weights for policy 1, policy_version 1180900 (0.0007) [2023-12-27 13:50:18,661][20347] Updated weights for policy 0, policy_version 1181893 (0.0006) [2023-12-27 13:50:18,692][20381] Updated weights for policy 1, policy_version 1180910 (0.0007) [2023-12-27 13:50:18,720][20347] Updated weights for policy 0, policy_version 1181903 (0.0005) [2023-12-27 13:50:18,751][20381] Updated weights for policy 1, policy_version 1180920 (0.0008) [2023-12-27 13:50:19,363][20347] Updated weights for policy 0, policy_version 1181913 (0.0007) [2023-12-27 13:50:19,419][20347] Updated weights for policy 0, policy_version 1181923 (0.0006) [2023-12-27 13:50:19,471][20347] Updated weights for policy 0, policy_version 1181933 (0.0005) [2023-12-27 13:50:19,503][20381] Updated weights for policy 1, policy_version 1180930 (0.0007) [2023-12-27 13:50:19,569][20381] Updated weights for policy 1, policy_version 1180940 (0.0008) [2023-12-27 13:50:19,635][20381] Updated weights for policy 1, policy_version 1180950 (0.0008) [2023-12-27 13:50:20,162][20347] Updated weights for policy 0, policy_version 1181943 (0.0010) [2023-12-27 13:50:20,222][20347] Updated weights for policy 0, policy_version 1181953 (0.0008) [2023-12-27 13:50:20,281][20347] Updated weights for policy 0, policy_version 1181963 (0.0007) [2023-12-27 13:50:20,384][20381] Updated weights for policy 1, policy_version 1180960 (0.0008) [2023-12-27 13:50:20,437][20381] Updated weights for policy 1, policy_version 1180970 (0.0007) [2023-12-27 13:50:20,490][20381] Updated weights for policy 1, policy_version 1180980 (0.0008) [2023-12-27 13:50:20,929][20347] Updated weights for policy 0, policy_version 1181973 (0.0009) [2023-12-27 13:50:20,988][20347] Updated weights for policy 0, policy_version 1181983 (0.0007) [2023-12-27 13:50:21,075][20347] Updated weights for policy 0, policy_version 1181993 (0.0007) [2023-12-27 13:50:21,252][20381] Updated weights for policy 1, policy_version 1180990 (0.0008) [2023-12-27 13:50:21,316][20381] Updated weights for policy 1, policy_version 1181000 (0.0007) [2023-12-27 13:50:21,388][20381] Updated weights for policy 1, policy_version 1181010 (0.0009) [2023-12-27 13:50:21,792][20347] Updated weights for policy 0, policy_version 1182003 (0.0008) [2023-12-27 13:50:21,854][20347] Updated weights for policy 0, policy_version 1182013 (0.0007) [2023-12-27 13:50:21,918][20347] Updated weights for policy 0, policy_version 1182023 (0.0007) [2023-12-27 13:50:22,097][20381] Updated weights for policy 1, policy_version 1181020 (0.0007) [2023-12-27 13:50:22,107][20292] KL-divergence is very high: 133.1069 [2023-12-27 13:50:22,117][20292] KL-divergence is very high: 129.0581 [2023-12-27 13:50:22,132][20292] KL-divergence is very high: 144.4401 [2023-12-27 13:50:22,147][20381] Updated weights for policy 1, policy_version 1181030 (0.0007) [2023-12-27 13:50:22,199][20381] Updated weights for policy 1, policy_version 1181040 (0.0008) [2023-12-27 13:50:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 605028352. Throughput: 0: 10229.3, 1: 10226.9. Samples: 604998360. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:22,363][19170] Avg episode reward: [(0, '845.160'), (1, '804.060')] [2023-12-27 13:50:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001181048_302374912.pth... [2023-12-27 13:50:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001182032_302653440.pth... [2023-12-27 13:50:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001180848_302350336.pth [2023-12-27 13:50:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001179864_302071808.pth [2023-12-27 13:50:22,626][20347] Updated weights for policy 0, policy_version 1182033 (0.0008) [2023-12-27 13:50:22,687][20347] Updated weights for policy 0, policy_version 1182043 (0.0009) [2023-12-27 13:50:22,755][20347] Updated weights for policy 0, policy_version 1182053 (0.0008) [2023-12-27 13:50:22,811][20347] Updated weights for policy 0, policy_version 1182063 (0.0009) [2023-12-27 13:50:22,838][20381] Updated weights for policy 1, policy_version 1181050 (0.0007) [2023-12-27 13:50:22,903][20381] Updated weights for policy 1, policy_version 1181060 (0.0007) [2023-12-27 13:50:22,953][20381] Updated weights for policy 1, policy_version 1181070 (0.0006) [2023-12-27 13:50:23,014][20381] Updated weights for policy 1, policy_version 1181080 (0.0007) [2023-12-27 13:50:23,498][20347] Updated weights for policy 0, policy_version 1182073 (0.0009) [2023-12-27 13:50:23,552][20347] Updated weights for policy 0, policy_version 1182083 (0.0005) [2023-12-27 13:50:23,609][20347] Updated weights for policy 0, policy_version 1182093 (0.0006) [2023-12-27 13:50:23,618][20381] Updated weights for policy 1, policy_version 1181090 (0.0007) [2023-12-27 13:50:23,676][20381] Updated weights for policy 1, policy_version 1181100 (0.0006) [2023-12-27 13:50:23,732][20381] Updated weights for policy 1, policy_version 1181110 (0.0005) [2023-12-27 13:50:24,246][20347] Updated weights for policy 0, policy_version 1182103 (0.0009) [2023-12-27 13:50:24,304][20347] Updated weights for policy 0, policy_version 1182113 (0.0010) [2023-12-27 13:50:24,352][20381] Updated weights for policy 1, policy_version 1181120 (0.0005) [2023-12-27 13:50:24,361][20347] Updated weights for policy 0, policy_version 1182123 (0.0010) [2023-12-27 13:50:24,406][20381] Updated weights for policy 1, policy_version 1181130 (0.0006) [2023-12-27 13:50:24,459][20381] Updated weights for policy 1, policy_version 1181140 (0.0007) [2023-12-27 13:50:24,976][20347] Updated weights for policy 0, policy_version 1182133 (0.0010) [2023-12-27 13:50:25,031][20347] Updated weights for policy 0, policy_version 1182143 (0.0008) [2023-12-27 13:50:25,074][20347] Updated weights for policy 0, policy_version 1182153 (0.0007) [2023-12-27 13:50:25,129][20381] Updated weights for policy 1, policy_version 1181150 (0.0007) [2023-12-27 13:50:25,178][20381] Updated weights for policy 1, policy_version 1181160 (0.0007) [2023-12-27 13:50:25,230][20381] Updated weights for policy 1, policy_version 1181170 (0.0006) [2023-12-27 13:50:25,724][20347] Updated weights for policy 0, policy_version 1182163 (0.0007) [2023-12-27 13:50:25,779][20347] Updated weights for policy 0, policy_version 1182173 (0.0008) [2023-12-27 13:50:25,837][20347] Updated weights for policy 0, policy_version 1182183 (0.0007) [2023-12-27 13:50:25,936][20381] Updated weights for policy 1, policy_version 1181180 (0.0007) [2023-12-27 13:50:25,985][20381] Updated weights for policy 1, policy_version 1181190 (0.0008) [2023-12-27 13:50:25,991][20292] KL-divergence is very high: 130.5024 [2023-12-27 13:50:26,044][20381] Updated weights for policy 1, policy_version 1181200 (0.0007) [2023-12-27 13:50:26,523][20347] Updated weights for policy 0, policy_version 1182193 (0.0007) [2023-12-27 13:50:26,580][20347] Updated weights for policy 0, policy_version 1182203 (0.0007) [2023-12-27 13:50:26,632][20347] Updated weights for policy 0, policy_version 1182213 (0.0007) [2023-12-27 13:50:26,686][20347] Updated weights for policy 0, policy_version 1182223 (0.0007) [2023-12-27 13:50:26,703][20381] Updated weights for policy 1, policy_version 1181210 (0.0007) [2023-12-27 13:50:26,751][20381] Updated weights for policy 1, policy_version 1181220 (0.0007) [2023-12-27 13:50:26,797][20381] Updated weights for policy 1, policy_version 1181230 (0.0007) [2023-12-27 13:50:26,848][20381] Updated weights for policy 1, policy_version 1181240 (0.0007) [2023-12-27 13:50:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 605126656. Throughput: 0: 10240.9, 1: 10163.6. Samples: 605120608. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:27,363][19170] Avg episode reward: [(0, '816.880'), (1, '804.220')] [2023-12-27 13:50:27,370][20347] Updated weights for policy 0, policy_version 1182233 (0.0008) [2023-12-27 13:50:27,424][20347] Updated weights for policy 0, policy_version 1182243 (0.0008) [2023-12-27 13:50:27,483][20347] Updated weights for policy 0, policy_version 1182253 (0.0007) [2023-12-27 13:50:27,526][20381] Updated weights for policy 1, policy_version 1181250 (0.0007) [2023-12-27 13:50:27,585][20381] Updated weights for policy 1, policy_version 1181260 (0.0008) [2023-12-27 13:50:27,641][20381] Updated weights for policy 1, policy_version 1181270 (0.0008) [2023-12-27 13:50:28,185][20347] Updated weights for policy 0, policy_version 1182263 (0.0007) [2023-12-27 13:50:28,232][20347] Updated weights for policy 0, policy_version 1182273 (0.0009) [2023-12-27 13:50:28,280][20347] Updated weights for policy 0, policy_version 1182283 (0.0008) [2023-12-27 13:50:28,355][20381] Updated weights for policy 1, policy_version 1181285 (0.0006) [2023-12-27 13:50:28,417][20381] Updated weights for policy 1, policy_version 1181295 (0.0007) [2023-12-27 13:50:28,984][20347] Updated weights for policy 0, policy_version 1182293 (0.0008) [2023-12-27 13:50:29,047][20347] Updated weights for policy 0, policy_version 1182303 (0.0008) [2023-12-27 13:50:29,106][20347] Updated weights for policy 0, policy_version 1182313 (0.0008) [2023-12-27 13:50:29,113][20381] Updated weights for policy 1, policy_version 1181305 (0.0007) [2023-12-27 13:50:29,170][20381] Updated weights for policy 1, policy_version 1181315 (0.0008) [2023-12-27 13:50:29,224][20381] Updated weights for policy 1, policy_version 1181325 (0.0008) [2023-12-27 13:50:29,282][20381] Updated weights for policy 1, policy_version 1181335 (0.0008) [2023-12-27 13:50:29,827][20347] Updated weights for policy 0, policy_version 1182323 (0.0007) [2023-12-27 13:50:29,886][20347] Updated weights for policy 0, policy_version 1182333 (0.0008) [2023-12-27 13:50:29,949][20347] Updated weights for policy 0, policy_version 1182343 (0.0007) [2023-12-27 13:50:29,956][20381] Updated weights for policy 1, policy_version 1181345 (0.0009) [2023-12-27 13:50:30,016][20381] Updated weights for policy 1, policy_version 1181355 (0.0008) [2023-12-27 13:50:30,072][20381] Updated weights for policy 1, policy_version 1181365 (0.0007) [2023-12-27 13:50:30,681][20347] Updated weights for policy 0, policy_version 1182353 (0.0008) [2023-12-27 13:50:30,732][20347] Updated weights for policy 0, policy_version 1182363 (0.0007) [2023-12-27 13:50:30,736][20381] Updated weights for policy 1, policy_version 1181375 (0.0007) [2023-12-27 13:50:30,788][20347] Updated weights for policy 0, policy_version 1182373 (0.0009) [2023-12-27 13:50:30,794][20381] Updated weights for policy 1, policy_version 1181385 (0.0007) [2023-12-27 13:50:30,835][20347] Updated weights for policy 0, policy_version 1182383 (0.0007) [2023-12-27 13:50:30,845][20381] Updated weights for policy 1, policy_version 1181395 (0.0009) [2023-12-27 13:50:31,571][20347] Updated weights for policy 0, policy_version 1182393 (0.0005) [2023-12-27 13:50:31,576][20381] Updated weights for policy 1, policy_version 1181405 (0.0009) [2023-12-27 13:50:31,636][20347] Updated weights for policy 0, policy_version 1182403 (0.0007) [2023-12-27 13:50:31,638][20381] Updated weights for policy 1, policy_version 1181415 (0.0008) [2023-12-27 13:50:31,691][20347] Updated weights for policy 0, policy_version 1182413 (0.0006) [2023-12-27 13:50:31,699][20381] Updated weights for policy 1, policy_version 1181425 (0.0007) [2023-12-27 13:50:32,325][20381] Updated weights for policy 1, policy_version 1181436 (0.0008) [2023-12-27 13:50:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 605224960. Throughput: 0: 10164.4, 1: 10229.6. Samples: 605241532. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:32,363][19170] Avg episode reward: [(0, '806.150'), (1, '804.520')] [2023-12-27 13:50:32,390][20381] Updated weights for policy 1, policy_version 1181446 (0.0007) [2023-12-27 13:50:32,425][20347] Updated weights for policy 0, policy_version 1182423 (0.0007) [2023-12-27 13:50:32,452][20381] Updated weights for policy 1, policy_version 1181456 (0.0005) [2023-12-27 13:50:32,478][20347] Updated weights for policy 0, policy_version 1182433 (0.0007) [2023-12-27 13:50:32,536][20347] Updated weights for policy 0, policy_version 1182443 (0.0008) [2023-12-27 13:50:33,074][20381] Updated weights for policy 1, policy_version 1181466 (0.0007) [2023-12-27 13:50:33,126][20381] Updated weights for policy 1, policy_version 1181476 (0.0008) [2023-12-27 13:50:33,171][20381] Updated weights for policy 1, policy_version 1181486 (0.0008) [2023-12-27 13:50:33,208][20347] Updated weights for policy 0, policy_version 1182453 (0.0007) [2023-12-27 13:50:33,217][20381] Updated weights for policy 1, policy_version 1181496 (0.0006) [2023-12-27 13:50:33,257][20347] Updated weights for policy 0, policy_version 1182463 (0.0007) [2023-12-27 13:50:33,305][20347] Updated weights for policy 0, policy_version 1182473 (0.0008) [2023-12-27 13:50:33,879][20381] Updated weights for policy 1, policy_version 1181506 (0.0008) [2023-12-27 13:50:33,906][20347] Updated weights for policy 0, policy_version 1182483 (0.0007) [2023-12-27 13:50:33,926][20381] Updated weights for policy 1, policy_version 1181516 (0.0007) [2023-12-27 13:50:33,954][20347] Updated weights for policy 0, policy_version 1182493 (0.0005) [2023-12-27 13:50:33,978][20381] Updated weights for policy 1, policy_version 1181526 (0.0006) [2023-12-27 13:50:34,004][20347] Updated weights for policy 0, policy_version 1182503 (0.0006) [2023-12-27 13:50:34,686][20347] Updated weights for policy 0, policy_version 1182513 (0.0007) [2023-12-27 13:50:34,713][20381] Updated weights for policy 1, policy_version 1181536 (0.0008) [2023-12-27 13:50:34,745][20347] Updated weights for policy 0, policy_version 1182523 (0.0007) [2023-12-27 13:50:34,766][20381] Updated weights for policy 1, policy_version 1181546 (0.0005) [2023-12-27 13:50:34,802][20347] Updated weights for policy 0, policy_version 1182533 (0.0007) [2023-12-27 13:50:34,828][20381] Updated weights for policy 1, policy_version 1181556 (0.0006) [2023-12-27 13:50:34,861][20347] Updated weights for policy 0, policy_version 1182543 (0.0007) [2023-12-27 13:50:35,478][20381] Updated weights for policy 1, policy_version 1181566 (0.0007) [2023-12-27 13:50:35,480][20347] Updated weights for policy 0, policy_version 1182553 (0.0005) [2023-12-27 13:50:35,523][20381] Updated weights for policy 1, policy_version 1181576 (0.0008) [2023-12-27 13:50:35,531][20347] Updated weights for policy 0, policy_version 1182563 (0.0005) [2023-12-27 13:50:35,565][20381] Updated weights for policy 1, policy_version 1181586 (0.0007) [2023-12-27 13:50:35,589][20347] Updated weights for policy 0, policy_version 1182573 (0.0006) [2023-12-27 13:50:36,245][20347] Updated weights for policy 0, policy_version 1182583 (0.0008) [2023-12-27 13:50:36,265][20381] Updated weights for policy 1, policy_version 1181596 (0.0006) [2023-12-27 13:50:36,305][20347] Updated weights for policy 0, policy_version 1182593 (0.0008) [2023-12-27 13:50:36,323][20381] Updated weights for policy 1, policy_version 1181606 (0.0006) [2023-12-27 13:50:36,368][20347] Updated weights for policy 0, policy_version 1182603 (0.0008) [2023-12-27 13:50:36,386][20381] Updated weights for policy 1, policy_version 1181616 (0.0008) [2023-12-27 13:50:37,069][20381] Updated weights for policy 1, policy_version 1181626 (0.0007) [2023-12-27 13:50:37,117][20347] Updated weights for policy 0, policy_version 1182613 (0.0007) [2023-12-27 13:50:37,133][20381] Updated weights for policy 1, policy_version 1181636 (0.0007) [2023-12-27 13:50:37,176][20347] Updated weights for policy 0, policy_version 1182623 (0.0005) [2023-12-27 13:50:37,205][20381] Updated weights for policy 1, policy_version 1181646 (0.0007) [2023-12-27 13:50:37,236][20347] Updated weights for policy 0, policy_version 1182633 (0.0006) [2023-12-27 13:50:37,271][20381] Updated weights for policy 1, policy_version 1181656 (0.0008) [2023-12-27 13:50:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 605339648. Throughput: 0: 10186.3, 1: 10230.6. Samples: 605304140. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:37,363][19170] Avg episode reward: [(0, '893.310'), (1, '725.950')] [2023-12-27 13:50:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001181656_302530560.pth... [2023-12-27 13:50:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001182640_302809088.pth... [2023-12-27 13:50:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001180472_302227456.pth [2023-12-27 13:50:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001181424_302497792.pth [2023-12-27 13:50:37,834][20347] Updated weights for policy 0, policy_version 1182643 (0.0006) [2023-12-27 13:50:37,889][20347] Updated weights for policy 0, policy_version 1182653 (0.0005) [2023-12-27 13:50:37,948][20347] Updated weights for policy 0, policy_version 1182663 (0.0005) [2023-12-27 13:50:37,971][20381] Updated weights for policy 1, policy_version 1181666 (0.0008) [2023-12-27 13:50:38,019][20381] Updated weights for policy 1, policy_version 1181676 (0.0007) [2023-12-27 13:50:38,070][20381] Updated weights for policy 1, policy_version 1181686 (0.0007) [2023-12-27 13:50:38,583][20347] Updated weights for policy 0, policy_version 1182673 (0.0006) [2023-12-27 13:50:38,636][20347] Updated weights for policy 0, policy_version 1182683 (0.0007) [2023-12-27 13:50:38,691][20347] Updated weights for policy 0, policy_version 1182693 (0.0007) [2023-12-27 13:50:38,758][20347] Updated weights for policy 0, policy_version 1182703 (0.0006) [2023-12-27 13:50:38,784][20381] Updated weights for policy 1, policy_version 1181696 (0.0008) [2023-12-27 13:50:38,847][20381] Updated weights for policy 1, policy_version 1181706 (0.0008) [2023-12-27 13:50:38,904][20381] Updated weights for policy 1, policy_version 1181716 (0.0008) [2023-12-27 13:50:39,432][20347] Updated weights for policy 0, policy_version 1182713 (0.0009) [2023-12-27 13:50:39,495][20347] Updated weights for policy 0, policy_version 1182723 (0.0007) [2023-12-27 13:50:39,556][20347] Updated weights for policy 0, policy_version 1182733 (0.0006) [2023-12-27 13:50:39,632][20381] Updated weights for policy 1, policy_version 1181726 (0.0006) [2023-12-27 13:50:39,696][20381] Updated weights for policy 1, policy_version 1181736 (0.0006) [2023-12-27 13:50:39,760][20381] Updated weights for policy 1, policy_version 1181746 (0.0008) [2023-12-27 13:50:40,307][20347] Updated weights for policy 0, policy_version 1182743 (0.0008) [2023-12-27 13:50:40,370][20347] Updated weights for policy 0, policy_version 1182753 (0.0008) [2023-12-27 13:50:40,411][20381] Updated weights for policy 1, policy_version 1181756 (0.0007) [2023-12-27 13:50:40,434][20347] Updated weights for policy 0, policy_version 1182763 (0.0009) [2023-12-27 13:50:40,471][20381] Updated weights for policy 1, policy_version 1181766 (0.0008) [2023-12-27 13:50:40,526][20381] Updated weights for policy 1, policy_version 1181776 (0.0008) [2023-12-27 13:50:41,022][20347] Updated weights for policy 0, policy_version 1182773 (0.0008) [2023-12-27 13:50:41,083][20347] Updated weights for policy 0, policy_version 1182783 (0.0009) [2023-12-27 13:50:41,159][20347] Updated weights for policy 0, policy_version 1182793 (0.0009) [2023-12-27 13:50:41,237][20381] Updated weights for policy 1, policy_version 1181786 (0.0007) [2023-12-27 13:50:41,305][20381] Updated weights for policy 1, policy_version 1181796 (0.0007) [2023-12-27 13:50:41,371][20381] Updated weights for policy 1, policy_version 1181806 (0.0007) [2023-12-27 13:50:41,438][20381] Updated weights for policy 1, policy_version 1181816 (0.0008) [2023-12-27 13:50:41,889][20347] Updated weights for policy 0, policy_version 1182803 (0.0007) [2023-12-27 13:50:41,937][20347] Updated weights for policy 0, policy_version 1182813 (0.0007) [2023-12-27 13:50:41,995][20347] Updated weights for policy 0, policy_version 1182823 (0.0008) [2023-12-27 13:50:42,171][20381] Updated weights for policy 1, policy_version 1181826 (0.0009) [2023-12-27 13:50:42,231][20381] Updated weights for policy 1, policy_version 1181836 (0.0011) [2023-12-27 13:50:42,290][20381] Updated weights for policy 1, policy_version 1181846 (0.0011) [2023-12-27 13:50:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 605437952. Throughput: 0: 10205.9, 1: 10151.5. Samples: 605425076. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:42,363][19170] Avg episode reward: [(0, '902.400'), (1, '726.060')] [2023-12-27 13:50:42,729][20347] Updated weights for policy 0, policy_version 1182833 (0.0008) [2023-12-27 13:50:42,786][20347] Updated weights for policy 0, policy_version 1182843 (0.0008) [2023-12-27 13:50:42,846][20347] Updated weights for policy 0, policy_version 1182853 (0.0008) [2023-12-27 13:50:42,905][20347] Updated weights for policy 0, policy_version 1182863 (0.0007) [2023-12-27 13:50:42,938][20381] Updated weights for policy 1, policy_version 1181856 (0.0009) [2023-12-27 13:50:43,000][20381] Updated weights for policy 1, policy_version 1181866 (0.0007) [2023-12-27 13:50:43,084][20381] Updated weights for policy 1, policy_version 1181876 (0.0007) [2023-12-27 13:50:43,526][20347] Updated weights for policy 0, policy_version 1182873 (0.0008) [2023-12-27 13:50:43,580][20347] Updated weights for policy 0, policy_version 1182883 (0.0008) [2023-12-27 13:50:43,638][20347] Updated weights for policy 0, policy_version 1182893 (0.0008) [2023-12-27 13:50:43,706][20381] Updated weights for policy 1, policy_version 1181886 (0.0008) [2023-12-27 13:50:43,761][20381] Updated weights for policy 1, policy_version 1181896 (0.0008) [2023-12-27 13:50:43,810][20381] Updated weights for policy 1, policy_version 1181906 (0.0007) [2023-12-27 13:50:44,338][20347] Updated weights for policy 0, policy_version 1182903 (0.0008) [2023-12-27 13:50:44,398][20347] Updated weights for policy 0, policy_version 1182913 (0.0010) [2023-12-27 13:50:44,458][20347] Updated weights for policy 0, policy_version 1182923 (0.0010) [2023-12-27 13:50:44,499][20381] Updated weights for policy 1, policy_version 1181916 (0.0007) [2023-12-27 13:50:44,556][20381] Updated weights for policy 1, policy_version 1181926 (0.0007) [2023-12-27 13:50:44,622][20381] Updated weights for policy 1, policy_version 1181936 (0.0006) [2023-12-27 13:50:45,171][20347] Updated weights for policy 0, policy_version 1182933 (0.0010) [2023-12-27 13:50:45,238][20347] Updated weights for policy 0, policy_version 1182943 (0.0006) [2023-12-27 13:50:45,287][20381] Updated weights for policy 1, policy_version 1181946 (0.0007) [2023-12-27 13:50:45,298][20347] Updated weights for policy 0, policy_version 1182953 (0.0006) [2023-12-27 13:50:45,339][20381] Updated weights for policy 1, policy_version 1181956 (0.0005) [2023-12-27 13:50:45,401][20381] Updated weights for policy 1, policy_version 1181966 (0.0005) [2023-12-27 13:50:45,462][20381] Updated weights for policy 1, policy_version 1181976 (0.0007) [2023-12-27 13:50:45,963][20347] Updated weights for policy 0, policy_version 1182963 (0.0008) [2023-12-27 13:50:46,018][20347] Updated weights for policy 0, policy_version 1182973 (0.0008) [2023-12-27 13:50:46,071][20347] Updated weights for policy 0, policy_version 1182983 (0.0008) [2023-12-27 13:50:46,119][20381] Updated weights for policy 1, policy_version 1181986 (0.0006) [2023-12-27 13:50:46,175][20381] Updated weights for policy 1, policy_version 1181996 (0.0007) [2023-12-27 13:50:46,226][20381] Updated weights for policy 1, policy_version 1182006 (0.0007) [2023-12-27 13:50:46,769][20347] Updated weights for policy 0, policy_version 1182993 (0.0006) [2023-12-27 13:50:46,828][20347] Updated weights for policy 0, policy_version 1183003 (0.0005) [2023-12-27 13:50:46,886][20347] Updated weights for policy 0, policy_version 1183013 (0.0006) [2023-12-27 13:50:46,922][20381] Updated weights for policy 1, policy_version 1182016 (0.0006) [2023-12-27 13:50:46,942][20347] Updated weights for policy 0, policy_version 1183023 (0.0006) [2023-12-27 13:50:46,968][20381] Updated weights for policy 1, policy_version 1182026 (0.0006) [2023-12-27 13:50:47,018][20381] Updated weights for policy 1, policy_version 1182036 (0.0008) [2023-12-27 13:50:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 605536256. Throughput: 0: 10236.2, 1: 10115.8. Samples: 605545748. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:47,363][19170] Avg episode reward: [(0, '863.610'), (1, '833.730')] [2023-12-27 13:50:47,550][20347] Updated weights for policy 0, policy_version 1183033 (0.0007) [2023-12-27 13:50:47,601][20347] Updated weights for policy 0, policy_version 1183043 (0.0007) [2023-12-27 13:50:47,656][20347] Updated weights for policy 0, policy_version 1183053 (0.0007) [2023-12-27 13:50:47,738][20381] Updated weights for policy 1, policy_version 1182046 (0.0006) [2023-12-27 13:50:47,796][20381] Updated weights for policy 1, policy_version 1182056 (0.0005) [2023-12-27 13:50:47,850][20381] Updated weights for policy 1, policy_version 1182066 (0.0008) [2023-12-27 13:50:48,386][20347] Updated weights for policy 0, policy_version 1183063 (0.0007) [2023-12-27 13:50:48,446][20347] Updated weights for policy 0, policy_version 1183073 (0.0008) [2023-12-27 13:50:48,506][20347] Updated weights for policy 0, policy_version 1183083 (0.0007) [2023-12-27 13:50:48,522][20381] Updated weights for policy 1, policy_version 1182076 (0.0008) [2023-12-27 13:50:48,585][20381] Updated weights for policy 1, policy_version 1182086 (0.0007) [2023-12-27 13:50:48,646][20381] Updated weights for policy 1, policy_version 1182096 (0.0008) [2023-12-27 13:50:49,210][20347] Updated weights for policy 0, policy_version 1183093 (0.0006) [2023-12-27 13:50:49,278][20347] Updated weights for policy 0, policy_version 1183103 (0.0008) [2023-12-27 13:50:49,341][20347] Updated weights for policy 0, policy_version 1183113 (0.0008) [2023-12-27 13:50:49,344][20381] Updated weights for policy 1, policy_version 1182106 (0.0008) [2023-12-27 13:50:49,411][20381] Updated weights for policy 1, policy_version 1182116 (0.0008) [2023-12-27 13:50:49,466][20381] Updated weights for policy 1, policy_version 1182126 (0.0007) [2023-12-27 13:50:49,526][20381] Updated weights for policy 1, policy_version 1182136 (0.0009) [2023-12-27 13:50:50,007][20347] Updated weights for policy 0, policy_version 1183123 (0.0008) [2023-12-27 13:50:50,073][20347] Updated weights for policy 0, policy_version 1183133 (0.0007) [2023-12-27 13:50:50,135][20347] Updated weights for policy 0, policy_version 1183143 (0.0008) [2023-12-27 13:50:50,257][20381] Updated weights for policy 1, policy_version 1182146 (0.0006) [2023-12-27 13:50:50,323][20381] Updated weights for policy 1, policy_version 1182156 (0.0005) [2023-12-27 13:50:50,394][20381] Updated weights for policy 1, policy_version 1182166 (0.0006) [2023-12-27 13:50:50,802][20347] Updated weights for policy 0, policy_version 1183153 (0.0008) [2023-12-27 13:50:50,865][20347] Updated weights for policy 0, policy_version 1183163 (0.0007) [2023-12-27 13:50:50,925][20347] Updated weights for policy 0, policy_version 1183173 (0.0008) [2023-12-27 13:50:50,982][20347] Updated weights for policy 0, policy_version 1183183 (0.0007) [2023-12-27 13:50:51,027][20381] Updated weights for policy 1, policy_version 1182176 (0.0008) [2023-12-27 13:50:51,087][20381] Updated weights for policy 1, policy_version 1182186 (0.0008) [2023-12-27 13:50:51,157][20381] Updated weights for policy 1, policy_version 1182197 (0.0007) [2023-12-27 13:50:51,768][20347] Updated weights for policy 0, policy_version 1183193 (0.0007) [2023-12-27 13:50:51,821][20347] Updated weights for policy 0, policy_version 1183203 (0.0005) [2023-12-27 13:50:51,864][20381] Updated weights for policy 1, policy_version 1182207 (0.0007) [2023-12-27 13:50:51,882][20347] Updated weights for policy 0, policy_version 1183213 (0.0005) [2023-12-27 13:50:51,921][20381] Updated weights for policy 1, policy_version 1182217 (0.0008) [2023-12-27 13:50:51,980][20381] Updated weights for policy 1, policy_version 1182227 (0.0008) [2023-12-27 13:50:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 605634560. Throughput: 0: 10237.8, 1: 10060.4. Samples: 605605740. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:52,363][19170] Avg episode reward: [(0, '824.360'), (1, '814.130')] [2023-12-27 13:50:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001183216_302956544.pth... [2023-12-27 13:50:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001182232_302678016.pth... [2023-12-27 13:50:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001182032_302653440.pth [2023-12-27 13:50:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001181048_302374912.pth [2023-12-27 13:50:52,538][20347] Updated weights for policy 0, policy_version 1183223 (0.0007) [2023-12-27 13:50:52,591][20347] Updated weights for policy 0, policy_version 1183233 (0.0008) [2023-12-27 13:50:52,649][20347] Updated weights for policy 0, policy_version 1183243 (0.0007) [2023-12-27 13:50:52,679][20381] Updated weights for policy 1, policy_version 1182237 (0.0008) [2023-12-27 13:50:52,740][20381] Updated weights for policy 1, policy_version 1182247 (0.0006) [2023-12-27 13:50:52,792][20381] Updated weights for policy 1, policy_version 1182257 (0.0007) [2023-12-27 13:50:53,364][20347] Updated weights for policy 0, policy_version 1183253 (0.0007) [2023-12-27 13:50:53,374][20381] Updated weights for policy 1, policy_version 1182267 (0.0006) [2023-12-27 13:50:53,409][20347] Updated weights for policy 0, policy_version 1183263 (0.0007) [2023-12-27 13:50:53,417][20291] KL-divergence is very high: 220.2445 [2023-12-27 13:50:53,422][20291] KL-divergence is very high: 226.2275 [2023-12-27 13:50:53,427][20291] KL-divergence is very high: 194.1578 [2023-12-27 13:50:53,432][20291] KL-divergence is very high: 165.6235 [2023-12-27 13:50:53,433][20381] Updated weights for policy 1, policy_version 1182277 (0.0007) [2023-12-27 13:50:53,455][20291] KL-divergence is very high: 143.7307 [2023-12-27 13:50:53,456][20347] Updated weights for policy 0, policy_version 1183273 (0.0006) [2023-12-27 13:50:53,460][20291] KL-divergence is very high: 128.7178 [2023-12-27 13:50:53,465][20291] KL-divergence is very high: 115.7332 [2023-12-27 13:50:53,470][20291] KL-divergence is very high: 101.8403 [2023-12-27 13:50:53,489][20381] Updated weights for policy 1, policy_version 1182287 (0.0006) [2023-12-27 13:50:54,125][20291] KL-divergence is very high: 278.4547 [2023-12-27 13:50:54,128][20381] Updated weights for policy 1, policy_version 1182297 (0.0007) [2023-12-27 13:50:54,143][20347] Updated weights for policy 0, policy_version 1183283 (0.0010) [2023-12-27 13:50:54,149][20291] KL-divergence is very high: 104.0421 [2023-12-27 13:50:54,173][20291] KL-divergence is very high: 186.5790 [2023-12-27 13:50:54,194][20381] Updated weights for policy 1, policy_version 1182307 (0.0007) [2023-12-27 13:50:54,197][20291] KL-divergence is very high: 105.6917 [2023-12-27 13:50:54,202][20347] Updated weights for policy 0, policy_version 1183293 (0.0010) [2023-12-27 13:50:54,254][20381] Updated weights for policy 1, policy_version 1182317 (0.0010) [2023-12-27 13:50:54,261][20347] Updated weights for policy 0, policy_version 1183303 (0.0011) [2023-12-27 13:50:54,292][20291] KL-divergence is very high: 126.8978 [2023-12-27 13:50:54,314][20381] Updated weights for policy 1, policy_version 1182327 (0.0008) [2023-12-27 13:50:54,946][20347] Updated weights for policy 0, policy_version 1183313 (0.0010) [2023-12-27 13:50:54,961][20381] Updated weights for policy 1, policy_version 1182337 (0.0007) [2023-12-27 13:50:55,006][20347] Updated weights for policy 0, policy_version 1183323 (0.0008) [2023-12-27 13:50:55,024][20381] Updated weights for policy 1, policy_version 1182347 (0.0006) [2023-12-27 13:50:55,059][20347] Updated weights for policy 0, policy_version 1183333 (0.0010) [2023-12-27 13:50:55,073][20381] Updated weights for policy 1, policy_version 1182357 (0.0005) [2023-12-27 13:50:55,118][20347] Updated weights for policy 0, policy_version 1183343 (0.0010) [2023-12-27 13:50:55,724][20347] Updated weights for policy 0, policy_version 1183353 (0.0008) [2023-12-27 13:50:55,749][20381] Updated weights for policy 1, policy_version 1182367 (0.0008) [2023-12-27 13:50:55,771][20347] Updated weights for policy 0, policy_version 1183363 (0.0007) [2023-12-27 13:50:55,805][20381] Updated weights for policy 1, policy_version 1182377 (0.0008) [2023-12-27 13:50:55,822][20347] Updated weights for policy 0, policy_version 1183373 (0.0008) [2023-12-27 13:50:55,859][20381] Updated weights for policy 1, policy_version 1182387 (0.0010) [2023-12-27 13:50:56,459][20347] Updated weights for policy 0, policy_version 1183383 (0.0009) [2023-12-27 13:50:56,510][20347] Updated weights for policy 0, policy_version 1183393 (0.0010) [2023-12-27 13:50:56,515][20381] Updated weights for policy 1, policy_version 1182397 (0.0010) [2023-12-27 13:50:56,560][20347] Updated weights for policy 0, policy_version 1183403 (0.0009) [2023-12-27 13:50:56,563][20381] Updated weights for policy 1, policy_version 1182407 (0.0010) [2023-12-27 13:50:56,614][20381] Updated weights for policy 1, policy_version 1182417 (0.0010) [2023-12-27 13:50:57,230][20347] Updated weights for policy 0, policy_version 1183413 (0.0010) [2023-12-27 13:50:57,289][20347] Updated weights for policy 0, policy_version 1183423 (0.0008) [2023-12-27 13:50:57,309][20381] Updated weights for policy 1, policy_version 1182427 (0.0011) [2023-12-27 13:50:57,340][20347] Updated weights for policy 0, policy_version 1183433 (0.0005) [2023-12-27 13:50:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 605732864. Throughput: 0: 10238.0, 1: 10088.9. Samples: 605729424. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:50:57,363][20381] Updated weights for policy 1, policy_version 1182437 (0.0010) [2023-12-27 13:50:57,363][19170] Avg episode reward: [(0, '796.200'), (1, '872.820')] [2023-12-27 13:50:57,418][20381] Updated weights for policy 1, policy_version 1182447 (0.0010) [2023-12-27 13:50:57,944][20347] Updated weights for policy 0, policy_version 1183443 (0.0007) [2023-12-27 13:50:58,002][20347] Updated weights for policy 0, policy_version 1183453 (0.0006) [2023-12-27 13:50:58,063][20347] Updated weights for policy 0, policy_version 1183463 (0.0008) [2023-12-27 13:50:58,130][20381] Updated weights for policy 1, policy_version 1182457 (0.0007) [2023-12-27 13:50:58,192][20381] Updated weights for policy 1, policy_version 1182467 (0.0008) [2023-12-27 13:50:58,259][20381] Updated weights for policy 1, policy_version 1182477 (0.0008) [2023-12-27 13:50:58,317][20381] Updated weights for policy 1, policy_version 1182487 (0.0007) [2023-12-27 13:50:58,803][20347] Updated weights for policy 0, policy_version 1183473 (0.0008) [2023-12-27 13:50:58,879][20347] Updated weights for policy 0, policy_version 1183483 (0.0008) [2023-12-27 13:50:58,931][20347] Updated weights for policy 0, policy_version 1183493 (0.0008) [2023-12-27 13:50:58,988][20347] Updated weights for policy 0, policy_version 1183503 (0.0007) [2023-12-27 13:50:59,108][20381] Updated weights for policy 1, policy_version 1182497 (0.0008) [2023-12-27 13:50:59,161][20381] Updated weights for policy 1, policy_version 1182507 (0.0008) [2023-12-27 13:50:59,220][20381] Updated weights for policy 1, policy_version 1182517 (0.0008) [2023-12-27 13:50:59,716][20347] Updated weights for policy 0, policy_version 1183513 (0.0008) [2023-12-27 13:50:59,772][20347] Updated weights for policy 0, policy_version 1183523 (0.0007) [2023-12-27 13:50:59,830][20347] Updated weights for policy 0, policy_version 1183533 (0.0007) [2023-12-27 13:50:59,904][20381] Updated weights for policy 1, policy_version 1182527 (0.0008) [2023-12-27 13:50:59,970][20381] Updated weights for policy 1, policy_version 1182537 (0.0008) [2023-12-27 13:51:00,033][20381] Updated weights for policy 1, policy_version 1182547 (0.0008) [2023-12-27 13:51:00,528][20347] Updated weights for policy 0, policy_version 1183543 (0.0008) [2023-12-27 13:51:00,584][20347] Updated weights for policy 0, policy_version 1183553 (0.0008) [2023-12-27 13:51:00,634][20347] Updated weights for policy 0, policy_version 1183563 (0.0007) [2023-12-27 13:51:00,773][20381] Updated weights for policy 1, policy_version 1182557 (0.0008) [2023-12-27 13:51:00,823][20381] Updated weights for policy 1, policy_version 1182567 (0.0007) [2023-12-27 13:51:00,871][20381] Updated weights for policy 1, policy_version 1182577 (0.0008) [2023-12-27 13:51:01,355][20347] Updated weights for policy 0, policy_version 1183573 (0.0009) [2023-12-27 13:51:01,424][20347] Updated weights for policy 0, policy_version 1183583 (0.0008) [2023-12-27 13:51:01,492][20347] Updated weights for policy 0, policy_version 1183593 (0.0006) [2023-12-27 13:51:01,571][20381] Updated weights for policy 1, policy_version 1182587 (0.0007) [2023-12-27 13:51:01,641][20381] Updated weights for policy 1, policy_version 1182597 (0.0008) [2023-12-27 13:51:01,702][20381] Updated weights for policy 1, policy_version 1182607 (0.0009) [2023-12-27 13:51:02,198][20347] Updated weights for policy 0, policy_version 1183603 (0.0007) [2023-12-27 13:51:02,247][20347] Updated weights for policy 0, policy_version 1183613 (0.0008) [2023-12-27 13:51:02,312][20347] Updated weights for policy 0, policy_version 1183623 (0.0008) [2023-12-27 13:51:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 605831168. Throughput: 0: 10160.4, 1: 10049.8. Samples: 605847216. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:02,363][19170] Avg episode reward: [(0, '863.900'), (1, '872.880')] [2023-12-27 13:51:02,427][20381] Updated weights for policy 1, policy_version 1182617 (0.0009) [2023-12-27 13:51:02,488][20381] Updated weights for policy 1, policy_version 1182627 (0.0008) [2023-12-27 13:51:02,537][20381] Updated weights for policy 1, policy_version 1182637 (0.0008) [2023-12-27 13:51:02,588][20381] Updated weights for policy 1, policy_version 1182647 (0.0007) [2023-12-27 13:51:02,926][20347] Updated weights for policy 0, policy_version 1183633 (0.0007) [2023-12-27 13:51:02,977][20347] Updated weights for policy 0, policy_version 1183643 (0.0007) [2023-12-27 13:51:03,029][20347] Updated weights for policy 0, policy_version 1183653 (0.0008) [2023-12-27 13:51:03,075][20347] Updated weights for policy 0, policy_version 1183663 (0.0007) [2023-12-27 13:51:03,300][20381] Updated weights for policy 1, policy_version 1182657 (0.0008) [2023-12-27 13:51:03,351][20381] Updated weights for policy 1, policy_version 1182667 (0.0007) [2023-12-27 13:51:03,407][20381] Updated weights for policy 1, policy_version 1182677 (0.0007) [2023-12-27 13:51:03,751][20347] Updated weights for policy 0, policy_version 1183673 (0.0007) [2023-12-27 13:51:03,803][20347] Updated weights for policy 0, policy_version 1183683 (0.0007) [2023-12-27 13:51:03,857][20347] Updated weights for policy 0, policy_version 1183693 (0.0009) [2023-12-27 13:51:04,146][20381] Updated weights for policy 1, policy_version 1182687 (0.0007) [2023-12-27 13:51:04,202][20381] Updated weights for policy 1, policy_version 1182697 (0.0008) [2023-12-27 13:51:04,251][20381] Updated weights for policy 1, policy_version 1182707 (0.0008) [2023-12-27 13:51:04,601][20347] Updated weights for policy 0, policy_version 1183703 (0.0011) [2023-12-27 13:51:04,656][20347] Updated weights for policy 0, policy_version 1183713 (0.0010) [2023-12-27 13:51:04,711][20347] Updated weights for policy 0, policy_version 1183723 (0.0010) [2023-12-27 13:51:04,958][20381] Updated weights for policy 1, policy_version 1182717 (0.0007) [2023-12-27 13:51:05,012][20381] Updated weights for policy 1, policy_version 1182727 (0.0007) [2023-12-27 13:51:05,056][20381] Updated weights for policy 1, policy_version 1182737 (0.0007) [2023-12-27 13:51:05,391][20347] Updated weights for policy 0, policy_version 1183733 (0.0010) [2023-12-27 13:51:05,445][20347] Updated weights for policy 0, policy_version 1183743 (0.0010) [2023-12-27 13:51:05,506][20347] Updated weights for policy 0, policy_version 1183753 (0.0010) [2023-12-27 13:51:05,746][20381] Updated weights for policy 1, policy_version 1182747 (0.0008) [2023-12-27 13:51:05,799][20381] Updated weights for policy 1, policy_version 1182757 (0.0011) [2023-12-27 13:51:05,862][20381] Updated weights for policy 1, policy_version 1182767 (0.0009) [2023-12-27 13:51:06,186][20347] Updated weights for policy 0, policy_version 1183763 (0.0010) [2023-12-27 13:51:06,248][20347] Updated weights for policy 0, policy_version 1183773 (0.0011) [2023-12-27 13:51:06,311][20347] Updated weights for policy 0, policy_version 1183783 (0.0010) [2023-12-27 13:51:06,567][20381] Updated weights for policy 1, policy_version 1182777 (0.0011) [2023-12-27 13:51:06,620][20381] Updated weights for policy 1, policy_version 1182787 (0.0011) [2023-12-27 13:51:06,674][20381] Updated weights for policy 1, policy_version 1182797 (0.0011) [2023-12-27 13:51:06,732][20381] Updated weights for policy 1, policy_version 1182807 (0.0011) [2023-12-27 13:51:06,984][20347] Updated weights for policy 0, policy_version 1183793 (0.0010) [2023-12-27 13:51:07,042][20347] Updated weights for policy 0, policy_version 1183803 (0.0010) [2023-12-27 13:51:07,104][20347] Updated weights for policy 0, policy_version 1183813 (0.0010) [2023-12-27 13:51:07,165][20347] Updated weights for policy 0, policy_version 1183823 (0.0010) [2023-12-27 13:51:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 605937664. Throughput: 0: 10122.0, 1: 10081.0. Samples: 605907492. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:07,363][19170] Avg episode reward: [(0, '864.140'), (1, '843.370')] [2023-12-27 13:51:07,364][20381] Updated weights for policy 1, policy_version 1182817 (0.0006) [2023-12-27 13:51:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001183824_303112192.pth... [2023-12-27 13:51:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001182640_302809088.pth [2023-12-27 13:51:07,416][20381] Updated weights for policy 1, policy_version 1182827 (0.0008) [2023-12-27 13:51:07,465][20381] Updated weights for policy 1, policy_version 1182837 (0.0007) [2023-12-27 13:51:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001182840_302833664.pth... [2023-12-27 13:51:07,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001181656_302530560.pth [2023-12-27 13:51:07,811][20347] Updated weights for policy 0, policy_version 1183833 (0.0007) [2023-12-27 13:51:07,872][20347] Updated weights for policy 0, policy_version 1183843 (0.0007) [2023-12-27 13:51:07,924][20347] Updated weights for policy 0, policy_version 1183853 (0.0007) [2023-12-27 13:51:08,156][20381] Updated weights for policy 1, policy_version 1182847 (0.0009) [2023-12-27 13:51:08,202][20381] Updated weights for policy 1, policy_version 1182857 (0.0010) [2023-12-27 13:51:08,255][20381] Updated weights for policy 1, policy_version 1182867 (0.0011) [2023-12-27 13:51:08,544][20347] Updated weights for policy 0, policy_version 1183863 (0.0008) [2023-12-27 13:51:08,602][20347] Updated weights for policy 0, policy_version 1183873 (0.0008) [2023-12-27 13:51:08,658][20347] Updated weights for policy 0, policy_version 1183883 (0.0008) [2023-12-27 13:51:08,943][20381] Updated weights for policy 1, policy_version 1182877 (0.0009) [2023-12-27 13:51:08,993][20381] Updated weights for policy 1, policy_version 1182887 (0.0007) [2023-12-27 13:51:09,049][20381] Updated weights for policy 1, policy_version 1182897 (0.0008) [2023-12-27 13:51:09,340][20347] Updated weights for policy 0, policy_version 1183893 (0.0008) [2023-12-27 13:51:09,410][20347] Updated weights for policy 0, policy_version 1183903 (0.0014) [2023-12-27 13:51:09,470][20347] Updated weights for policy 0, policy_version 1183913 (0.0009) [2023-12-27 13:51:09,811][20381] Updated weights for policy 1, policy_version 1182907 (0.0008) [2023-12-27 13:51:09,885][20381] Updated weights for policy 1, policy_version 1182917 (0.0008) [2023-12-27 13:51:09,944][20381] Updated weights for policy 1, policy_version 1182927 (0.0008) [2023-12-27 13:51:10,255][20347] Updated weights for policy 0, policy_version 1183923 (0.0005) [2023-12-27 13:51:10,315][20347] Updated weights for policy 0, policy_version 1183933 (0.0009) [2023-12-27 13:51:10,380][20347] Updated weights for policy 0, policy_version 1183943 (0.0007) [2023-12-27 13:51:10,710][20381] Updated weights for policy 1, policy_version 1182937 (0.0008) [2023-12-27 13:51:10,773][20381] Updated weights for policy 1, policy_version 1182947 (0.0007) [2023-12-27 13:51:10,835][20381] Updated weights for policy 1, policy_version 1182957 (0.0006) [2023-12-27 13:51:10,897][20381] Updated weights for policy 1, policy_version 1182967 (0.0007) [2023-12-27 13:51:11,072][20347] Updated weights for policy 0, policy_version 1183953 (0.0008) [2023-12-27 13:51:11,135][20347] Updated weights for policy 0, policy_version 1183963 (0.0010) [2023-12-27 13:51:11,200][20347] Updated weights for policy 0, policy_version 1183973 (0.0010) [2023-12-27 13:51:11,270][20347] Updated weights for policy 0, policy_version 1183983 (0.0009) [2023-12-27 13:51:11,600][20381] Updated weights for policy 1, policy_version 1182977 (0.0007) [2023-12-27 13:51:11,661][20381] Updated weights for policy 1, policy_version 1182987 (0.0009) [2023-12-27 13:51:11,716][20381] Updated weights for policy 1, policy_version 1182997 (0.0008) [2023-12-27 13:51:12,036][20347] Updated weights for policy 0, policy_version 1183993 (0.0008) [2023-12-27 13:51:12,093][20347] Updated weights for policy 0, policy_version 1184003 (0.0006) [2023-12-27 13:51:12,150][20347] Updated weights for policy 0, policy_version 1184013 (0.0008) [2023-12-27 13:51:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 606035968. Throughput: 0: 10119.0, 1: 10059.3. Samples: 606028632. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:12,363][19170] Avg episode reward: [(0, '931.660'), (1, '862.860')] [2023-12-27 13:51:12,474][20381] Updated weights for policy 1, policy_version 1183007 (0.0006) [2023-12-27 13:51:12,534][20381] Updated weights for policy 1, policy_version 1183017 (0.0005) [2023-12-27 13:51:12,593][20381] Updated weights for policy 1, policy_version 1183027 (0.0006) [2023-12-27 13:51:12,883][20347] Updated weights for policy 0, policy_version 1184023 (0.0006) [2023-12-27 13:51:12,940][20347] Updated weights for policy 0, policy_version 1184033 (0.0006) [2023-12-27 13:51:12,994][20347] Updated weights for policy 0, policy_version 1184043 (0.0005) [2023-12-27 13:51:13,239][20381] Updated weights for policy 1, policy_version 1183037 (0.0007) [2023-12-27 13:51:13,288][20381] Updated weights for policy 1, policy_version 1183047 (0.0005) [2023-12-27 13:51:13,332][20381] Updated weights for policy 1, policy_version 1183057 (0.0006) [2023-12-27 13:51:13,627][20347] Updated weights for policy 0, policy_version 1184053 (0.0008) [2023-12-27 13:51:13,680][20347] Updated weights for policy 0, policy_version 1184063 (0.0009) [2023-12-27 13:51:13,729][20347] Updated weights for policy 0, policy_version 1184073 (0.0008) [2023-12-27 13:51:13,940][20381] Updated weights for policy 1, policy_version 1183067 (0.0008) [2023-12-27 13:51:13,988][20381] Updated weights for policy 1, policy_version 1183077 (0.0009) [2023-12-27 13:51:14,038][20381] Updated weights for policy 1, policy_version 1183087 (0.0008) [2023-12-27 13:51:14,403][20347] Updated weights for policy 0, policy_version 1184083 (0.0006) [2023-12-27 13:51:14,463][20347] Updated weights for policy 0, policy_version 1184093 (0.0007) [2023-12-27 13:51:14,522][20347] Updated weights for policy 0, policy_version 1184103 (0.0008) [2023-12-27 13:51:14,753][20381] Updated weights for policy 1, policy_version 1183097 (0.0008) [2023-12-27 13:51:14,824][20381] Updated weights for policy 1, policy_version 1183107 (0.0008) [2023-12-27 13:51:14,883][20381] Updated weights for policy 1, policy_version 1183117 (0.0008) [2023-12-27 13:51:14,944][20381] Updated weights for policy 1, policy_version 1183127 (0.0006) [2023-12-27 13:51:15,192][20347] Updated weights for policy 0, policy_version 1184113 (0.0007) [2023-12-27 13:51:15,245][20347] Updated weights for policy 0, policy_version 1184123 (0.0008) [2023-12-27 13:51:15,298][20347] Updated weights for policy 0, policy_version 1184133 (0.0008) [2023-12-27 13:51:15,347][20347] Updated weights for policy 0, policy_version 1184143 (0.0007) [2023-12-27 13:51:15,574][20381] Updated weights for policy 1, policy_version 1183137 (0.0008) [2023-12-27 13:51:15,637][20381] Updated weights for policy 1, policy_version 1183147 (0.0007) [2023-12-27 13:51:15,703][20381] Updated weights for policy 1, policy_version 1183157 (0.0006) [2023-12-27 13:51:16,037][20347] Updated weights for policy 0, policy_version 1184153 (0.0007) [2023-12-27 13:51:16,091][20347] Updated weights for policy 0, policy_version 1184163 (0.0007) [2023-12-27 13:51:16,158][20347] Updated weights for policy 0, policy_version 1184173 (0.0009) [2023-12-27 13:51:16,373][20381] Updated weights for policy 1, policy_version 1183167 (0.0007) [2023-12-27 13:51:16,437][20381] Updated weights for policy 1, policy_version 1183177 (0.0008) [2023-12-27 13:51:16,497][20381] Updated weights for policy 1, policy_version 1183187 (0.0008) [2023-12-27 13:51:16,812][20347] Updated weights for policy 0, policy_version 1184183 (0.0009) [2023-12-27 13:51:16,881][20347] Updated weights for policy 0, policy_version 1184193 (0.0008) [2023-12-27 13:51:16,939][20347] Updated weights for policy 0, policy_version 1184203 (0.0010) [2023-12-27 13:51:17,123][20381] Updated weights for policy 1, policy_version 1183197 (0.0007) [2023-12-27 13:51:17,189][20381] Updated weights for policy 1, policy_version 1183207 (0.0006) [2023-12-27 13:51:17,239][20381] Updated weights for policy 1, policy_version 1183217 (0.0005) [2023-12-27 13:51:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 606142464. Throughput: 0: 10140.9, 1: 10049.5. Samples: 606150096. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:17,363][19170] Avg episode reward: [(0, '960.980'), (1, '862.980')] [2023-12-27 13:51:17,595][20347] Updated weights for policy 0, policy_version 1184213 (0.0010) [2023-12-27 13:51:17,647][20347] Updated weights for policy 0, policy_version 1184223 (0.0008) [2023-12-27 13:51:17,705][20347] Updated weights for policy 0, policy_version 1184233 (0.0007) [2023-12-27 13:51:17,850][20381] Updated weights for policy 1, policy_version 1183227 (0.0006) [2023-12-27 13:51:17,910][20381] Updated weights for policy 1, policy_version 1183237 (0.0007) [2023-12-27 13:51:17,959][20381] Updated weights for policy 1, policy_version 1183247 (0.0008) [2023-12-27 13:51:18,365][20347] Updated weights for policy 0, policy_version 1184243 (0.0008) [2023-12-27 13:51:18,427][20347] Updated weights for policy 0, policy_version 1184253 (0.0008) [2023-12-27 13:51:18,489][20347] Updated weights for policy 0, policy_version 1184263 (0.0008) [2023-12-27 13:51:18,637][20381] Updated weights for policy 1, policy_version 1183257 (0.0008) [2023-12-27 13:51:18,688][20381] Updated weights for policy 1, policy_version 1183267 (0.0007) [2023-12-27 13:51:18,743][20381] Updated weights for policy 1, policy_version 1183277 (0.0007) [2023-12-27 13:51:18,803][20381] Updated weights for policy 1, policy_version 1183287 (0.0007) [2023-12-27 13:51:19,143][20347] Updated weights for policy 0, policy_version 1184273 (0.0008) [2023-12-27 13:51:19,201][20347] Updated weights for policy 0, policy_version 1184283 (0.0010) [2023-12-27 13:51:19,265][20347] Updated weights for policy 0, policy_version 1184293 (0.0010) [2023-12-27 13:51:19,322][20347] Updated weights for policy 0, policy_version 1184303 (0.0011) [2023-12-27 13:51:19,549][20381] Updated weights for policy 1, policy_version 1183297 (0.0008) [2023-12-27 13:51:19,611][20381] Updated weights for policy 1, policy_version 1183307 (0.0009) [2023-12-27 13:51:19,674][20381] Updated weights for policy 1, policy_version 1183317 (0.0008) [2023-12-27 13:51:19,978][20347] Updated weights for policy 0, policy_version 1184313 (0.0008) [2023-12-27 13:51:20,036][20347] Updated weights for policy 0, policy_version 1184323 (0.0007) [2023-12-27 13:51:20,094][20347] Updated weights for policy 0, policy_version 1184333 (0.0005) [2023-12-27 13:51:20,365][20381] Updated weights for policy 1, policy_version 1183327 (0.0007) [2023-12-27 13:51:20,423][20381] Updated weights for policy 1, policy_version 1183337 (0.0006) [2023-12-27 13:51:20,480][20381] Updated weights for policy 1, policy_version 1183347 (0.0005) [2023-12-27 13:51:20,811][20347] Updated weights for policy 0, policy_version 1184343 (0.0008) [2023-12-27 13:51:20,825][20291] KL-divergence is very high: 265.9928 [2023-12-27 13:51:20,843][20291] KL-divergence is very high: 225.9602 [2023-12-27 13:51:20,874][20347] Updated weights for policy 0, policy_version 1184353 (0.0008) [2023-12-27 13:51:20,931][20347] Updated weights for policy 0, policy_version 1184363 (0.0008) [2023-12-27 13:51:21,076][20381] Updated weights for policy 1, policy_version 1183357 (0.0008) [2023-12-27 13:51:21,132][20381] Updated weights for policy 1, policy_version 1183367 (0.0008) [2023-12-27 13:51:21,194][20381] Updated weights for policy 1, policy_version 1183377 (0.0009) [2023-12-27 13:51:21,711][20347] Updated weights for policy 0, policy_version 1184373 (0.0009) [2023-12-27 13:51:21,789][20347] Updated weights for policy 0, policy_version 1184383 (0.0008) [2023-12-27 13:51:21,854][20347] Updated weights for policy 0, policy_version 1184393 (0.0007) [2023-12-27 13:51:21,917][20381] Updated weights for policy 1, policy_version 1183387 (0.0007) [2023-12-27 13:51:21,979][20381] Updated weights for policy 1, policy_version 1183397 (0.0006) [2023-12-27 13:51:22,043][20381] Updated weights for policy 1, policy_version 1183407 (0.0006) [2023-12-27 13:51:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 606240768. Throughput: 0: 10141.1, 1: 10018.6. Samples: 606211332. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:22,363][19170] Avg episode reward: [(0, '921.780'), (1, '843.610')] [2023-12-27 13:51:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001183416_302981120.pth... [2023-12-27 13:51:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001184400_303259648.pth... [2023-12-27 13:51:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001182232_302678016.pth [2023-12-27 13:51:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001183216_302956544.pth [2023-12-27 13:51:22,555][20347] Updated weights for policy 0, policy_version 1184403 (0.0007) [2023-12-27 13:51:22,616][20347] Updated weights for policy 0, policy_version 1184413 (0.0007) [2023-12-27 13:51:22,673][20347] Updated weights for policy 0, policy_version 1184423 (0.0008) [2023-12-27 13:51:22,707][20381] Updated weights for policy 1, policy_version 1183417 (0.0006) [2023-12-27 13:51:22,768][20381] Updated weights for policy 1, policy_version 1183427 (0.0008) [2023-12-27 13:51:22,825][20381] Updated weights for policy 1, policy_version 1183437 (0.0008) [2023-12-27 13:51:22,892][20381] Updated weights for policy 1, policy_version 1183447 (0.0008) [2023-12-27 13:51:23,302][20347] Updated weights for policy 0, policy_version 1184433 (0.0007) [2023-12-27 13:51:23,362][20347] Updated weights for policy 0, policy_version 1184443 (0.0008) [2023-12-27 13:51:23,414][20347] Updated weights for policy 0, policy_version 1184453 (0.0007) [2023-12-27 13:51:23,478][20347] Updated weights for policy 0, policy_version 1184463 (0.0007) [2023-12-27 13:51:23,549][20381] Updated weights for policy 1, policy_version 1183457 (0.0007) [2023-12-27 13:51:23,593][20292] KL-divergence is very high: 207.6844 [2023-12-27 13:51:23,598][20381] Updated weights for policy 1, policy_version 1183467 (0.0007) [2023-12-27 13:51:23,605][20292] KL-divergence is very high: 267.6756 [2023-12-27 13:51:23,611][20292] KL-divergence is very high: 258.4016 [2023-12-27 13:51:23,639][20292] KL-divergence is very high: 185.1386 [2023-12-27 13:51:23,652][20292] KL-divergence is very high: 149.9569 [2023-12-27 13:51:23,657][20381] Updated weights for policy 1, policy_version 1183477 (0.0005) [2023-12-27 13:51:23,658][20292] KL-divergence is very high: 123.8214 [2023-12-27 13:51:24,117][20347] Updated weights for policy 0, policy_version 1184473 (0.0006) [2023-12-27 13:51:24,171][20347] Updated weights for policy 0, policy_version 1184483 (0.0005) [2023-12-27 13:51:24,228][20347] Updated weights for policy 0, policy_version 1184493 (0.0005) [2023-12-27 13:51:24,251][20381] Updated weights for policy 1, policy_version 1183487 (0.0007) [2023-12-27 13:51:24,259][20292] KL-divergence is very high: 161.8582 [2023-12-27 13:51:24,307][20292] KL-divergence is very high: 126.6540 [2023-12-27 13:51:24,313][20381] Updated weights for policy 1, policy_version 1183497 (0.0005) [2023-12-27 13:51:24,371][20381] Updated weights for policy 1, policy_version 1183507 (0.0006) [2023-12-27 13:51:24,834][20347] Updated weights for policy 0, policy_version 1184503 (0.0005) [2023-12-27 13:51:24,898][20347] Updated weights for policy 0, policy_version 1184513 (0.0005) [2023-12-27 13:51:24,944][20347] Updated weights for policy 0, policy_version 1184523 (0.0005) [2023-12-27 13:51:24,977][20381] Updated weights for policy 1, policy_version 1183517 (0.0007) [2023-12-27 13:51:25,027][20381] Updated weights for policy 1, policy_version 1183527 (0.0005) [2023-12-27 13:51:25,082][20381] Updated weights for policy 1, policy_version 1183537 (0.0005) [2023-12-27 13:51:25,563][20347] Updated weights for policy 0, policy_version 1184533 (0.0008) [2023-12-27 13:51:25,621][20347] Updated weights for policy 0, policy_version 1184543 (0.0009) [2023-12-27 13:51:25,658][20381] Updated weights for policy 1, policy_version 1183547 (0.0005) [2023-12-27 13:51:25,671][20347] Updated weights for policy 0, policy_version 1184553 (0.0007) [2023-12-27 13:51:25,703][20381] Updated weights for policy 1, policy_version 1183557 (0.0007) [2023-12-27 13:51:25,748][20381] Updated weights for policy 1, policy_version 1183567 (0.0007) [2023-12-27 13:51:26,259][20347] Updated weights for policy 0, policy_version 1184563 (0.0005) [2023-12-27 13:51:26,311][20347] Updated weights for policy 0, policy_version 1184573 (0.0005) [2023-12-27 13:51:26,360][20347] Updated weights for policy 0, policy_version 1184583 (0.0005) [2023-12-27 13:51:26,425][20381] Updated weights for policy 1, policy_version 1183577 (0.0008) [2023-12-27 13:51:26,485][20381] Updated weights for policy 1, policy_version 1183587 (0.0010) [2023-12-27 13:51:26,533][20381] Updated weights for policy 1, policy_version 1183597 (0.0008) [2023-12-27 13:51:26,588][20381] Updated weights for policy 1, policy_version 1183607 (0.0009) [2023-12-27 13:51:26,934][20347] Updated weights for policy 0, policy_version 1184593 (0.0005) [2023-12-27 13:51:26,987][20347] Updated weights for policy 0, policy_version 1184603 (0.0005) [2023-12-27 13:51:27,041][20347] Updated weights for policy 0, policy_version 1184613 (0.0005) [2023-12-27 13:51:27,103][20347] Updated weights for policy 0, policy_version 1184623 (0.0008) [2023-12-27 13:51:27,212][20381] Updated weights for policy 1, policy_version 1183617 (0.0007) [2023-12-27 13:51:27,270][20381] Updated weights for policy 1, policy_version 1183627 (0.0008) [2023-12-27 13:51:27,321][20381] Updated weights for policy 1, policy_version 1183637 (0.0010) [2023-12-27 13:51:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 606355456. Throughput: 0: 10140.6, 1: 10141.5. Samples: 606337772. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:27,364][19170] Avg episode reward: [(0, '921.910'), (1, '794.730')] [2023-12-27 13:51:27,706][20347] Updated weights for policy 0, policy_version 1184633 (0.0008) [2023-12-27 13:51:27,766][20347] Updated weights for policy 0, policy_version 1184643 (0.0008) [2023-12-27 13:51:27,826][20347] Updated weights for policy 0, policy_version 1184653 (0.0007) [2023-12-27 13:51:27,912][20381] Updated weights for policy 1, policy_version 1183647 (0.0010) [2023-12-27 13:51:27,965][20381] Updated weights for policy 1, policy_version 1183657 (0.0010) [2023-12-27 13:51:28,024][20381] Updated weights for policy 1, policy_version 1183667 (0.0008) [2023-12-27 13:51:28,447][20347] Updated weights for policy 0, policy_version 1184663 (0.0009) [2023-12-27 13:51:28,505][20347] Updated weights for policy 0, policy_version 1184673 (0.0008) [2023-12-27 13:51:28,563][20347] Updated weights for policy 0, policy_version 1184683 (0.0010) [2023-12-27 13:51:28,677][20381] Updated weights for policy 1, policy_version 1183677 (0.0008) [2023-12-27 13:51:28,726][20381] Updated weights for policy 1, policy_version 1183687 (0.0010) [2023-12-27 13:51:28,775][20381] Updated weights for policy 1, policy_version 1183697 (0.0010) [2023-12-27 13:51:29,146][20347] Updated weights for policy 0, policy_version 1184693 (0.0008) [2023-12-27 13:51:29,200][20347] Updated weights for policy 0, policy_version 1184703 (0.0006) [2023-12-27 13:51:29,270][20347] Updated weights for policy 0, policy_version 1184713 (0.0009) [2023-12-27 13:51:29,498][20381] Updated weights for policy 1, policy_version 1183707 (0.0009) [2023-12-27 13:51:29,552][20381] Updated weights for policy 1, policy_version 1183717 (0.0005) [2023-12-27 13:51:29,604][20381] Updated weights for policy 1, policy_version 1183727 (0.0005) [2023-12-27 13:51:29,956][20347] Updated weights for policy 0, policy_version 1184723 (0.0009) [2023-12-27 13:51:30,014][20347] Updated weights for policy 0, policy_version 1184733 (0.0007) [2023-12-27 13:51:30,072][20347] Updated weights for policy 0, policy_version 1184743 (0.0005) [2023-12-27 13:51:30,247][20381] Updated weights for policy 1, policy_version 1183737 (0.0005) [2023-12-27 13:51:30,301][20381] Updated weights for policy 1, policy_version 1183747 (0.0008) [2023-12-27 13:51:30,364][20381] Updated weights for policy 1, policy_version 1183757 (0.0008) [2023-12-27 13:51:30,430][20381] Updated weights for policy 1, policy_version 1183767 (0.0008) [2023-12-27 13:51:30,711][20347] Updated weights for policy 0, policy_version 1184753 (0.0008) [2023-12-27 13:51:30,767][20347] Updated weights for policy 0, policy_version 1184763 (0.0005) [2023-12-27 13:51:30,812][20347] Updated weights for policy 0, policy_version 1184773 (0.0005) [2023-12-27 13:51:30,863][20347] Updated weights for policy 0, policy_version 1184783 (0.0005) [2023-12-27 13:51:31,051][20381] Updated weights for policy 1, policy_version 1183777 (0.0007) [2023-12-27 13:51:31,105][20381] Updated weights for policy 1, policy_version 1183787 (0.0008) [2023-12-27 13:51:31,167][20381] Updated weights for policy 1, policy_version 1183797 (0.0007) [2023-12-27 13:51:31,540][20347] Updated weights for policy 0, policy_version 1184793 (0.0007) [2023-12-27 13:51:31,596][20347] Updated weights for policy 0, policy_version 1184803 (0.0008) [2023-12-27 13:51:31,655][20347] Updated weights for policy 0, policy_version 1184813 (0.0007) [2023-12-27 13:51:31,867][20381] Updated weights for policy 1, policy_version 1183807 (0.0008) [2023-12-27 13:51:31,926][20381] Updated weights for policy 1, policy_version 1183817 (0.0008) [2023-12-27 13:51:31,975][20381] Updated weights for policy 1, policy_version 1183827 (0.0005) [2023-12-27 13:51:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 606453760. Throughput: 0: 10238.1, 1: 10230.9. Samples: 606466852. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:32,363][19170] Avg episode reward: [(0, '961.010'), (1, '902.310')] [2023-12-27 13:51:32,412][20347] Updated weights for policy 0, policy_version 1184823 (0.0007) [2023-12-27 13:51:32,471][20347] Updated weights for policy 0, policy_version 1184833 (0.0008) [2023-12-27 13:51:32,532][20347] Updated weights for policy 0, policy_version 1184843 (0.0008) [2023-12-27 13:51:32,602][20381] Updated weights for policy 1, policy_version 1183837 (0.0007) [2023-12-27 13:51:32,655][20381] Updated weights for policy 1, policy_version 1183847 (0.0008) [2023-12-27 13:51:32,711][20381] Updated weights for policy 1, policy_version 1183857 (0.0007) [2023-12-27 13:51:33,195][20347] Updated weights for policy 0, policy_version 1184853 (0.0008) [2023-12-27 13:51:33,246][20347] Updated weights for policy 0, policy_version 1184863 (0.0008) [2023-12-27 13:51:33,295][20347] Updated weights for policy 0, policy_version 1184873 (0.0008) [2023-12-27 13:51:33,340][20381] Updated weights for policy 1, policy_version 1183867 (0.0006) [2023-12-27 13:51:33,392][20381] Updated weights for policy 1, policy_version 1183877 (0.0007) [2023-12-27 13:51:33,439][20381] Updated weights for policy 1, policy_version 1183887 (0.0008) [2023-12-27 13:51:33,907][20347] Updated weights for policy 0, policy_version 1184883 (0.0007) [2023-12-27 13:51:33,963][20347] Updated weights for policy 0, policy_version 1184893 (0.0005) [2023-12-27 13:51:34,013][20347] Updated weights for policy 0, policy_version 1184903 (0.0008) [2023-12-27 13:51:34,056][20381] Updated weights for policy 1, policy_version 1183897 (0.0009) [2023-12-27 13:51:34,112][20381] Updated weights for policy 1, policy_version 1183907 (0.0005) [2023-12-27 13:51:34,172][20381] Updated weights for policy 1, policy_version 1183917 (0.0007) [2023-12-27 13:51:34,230][20381] Updated weights for policy 1, policy_version 1183927 (0.0009) [2023-12-27 13:51:34,624][20347] Updated weights for policy 0, policy_version 1184913 (0.0007) [2023-12-27 13:51:34,680][20347] Updated weights for policy 0, policy_version 1184923 (0.0008) [2023-12-27 13:51:34,737][20347] Updated weights for policy 0, policy_version 1184933 (0.0007) [2023-12-27 13:51:34,793][20347] Updated weights for policy 0, policy_version 1184943 (0.0008) [2023-12-27 13:51:34,945][20381] Updated weights for policy 1, policy_version 1183937 (0.0008) [2023-12-27 13:51:34,999][20381] Updated weights for policy 1, policy_version 1183947 (0.0009) [2023-12-27 13:51:35,054][20381] Updated weights for policy 1, policy_version 1183957 (0.0007) [2023-12-27 13:51:35,451][20347] Updated weights for policy 0, policy_version 1184953 (0.0007) [2023-12-27 13:51:35,516][20347] Updated weights for policy 0, policy_version 1184963 (0.0007) [2023-12-27 13:51:35,567][20347] Updated weights for policy 0, policy_version 1184973 (0.0007) [2023-12-27 13:51:35,752][20381] Updated weights for policy 1, policy_version 1183967 (0.0007) [2023-12-27 13:51:35,813][20381] Updated weights for policy 1, policy_version 1183977 (0.0005) [2023-12-27 13:51:35,870][20381] Updated weights for policy 1, policy_version 1183987 (0.0005) [2023-12-27 13:51:36,317][20347] Updated weights for policy 0, policy_version 1184983 (0.0007) [2023-12-27 13:51:36,380][20347] Updated weights for policy 0, policy_version 1184993 (0.0008) [2023-12-27 13:51:36,421][20381] Updated weights for policy 1, policy_version 1183997 (0.0006) [2023-12-27 13:51:36,438][20347] Updated weights for policy 0, policy_version 1185003 (0.0006) [2023-12-27 13:51:36,478][20381] Updated weights for policy 1, policy_version 1184007 (0.0006) [2023-12-27 13:51:36,537][20381] Updated weights for policy 1, policy_version 1184017 (0.0008) [2023-12-27 13:51:37,045][20347] Updated weights for policy 0, policy_version 1185013 (0.0006) [2023-12-27 13:51:37,096][20347] Updated weights for policy 0, policy_version 1185023 (0.0005) [2023-12-27 13:51:37,150][20347] Updated weights for policy 0, policy_version 1185033 (0.0005) [2023-12-27 13:51:37,247][20381] Updated weights for policy 1, policy_version 1184027 (0.0008) [2023-12-27 13:51:37,301][20381] Updated weights for policy 1, policy_version 1184037 (0.0008) [2023-12-27 13:51:37,358][20381] Updated weights for policy 1, policy_version 1184047 (0.0009) [2023-12-27 13:51:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 606560256. Throughput: 0: 10281.5, 1: 10260.6. Samples: 606530136. Policy #0 lag: (min: 3.0, avg: 12.4, max: 35.0) [2023-12-27 13:51:37,363][19170] Avg episode reward: [(0, '931.650'), (1, '912.010')] [2023-12-27 13:51:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001185040_303423488.pth... [2023-12-27 13:51:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001183824_303112192.pth [2023-12-27 13:51:37,408][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001184056_303144960.pth... [2023-12-27 13:51:37,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001182840_302833664.pth [2023-12-27 13:51:37,756][20347] Updated weights for policy 0, policy_version 1185043 (0.0006) [2023-12-27 13:51:37,810][20347] Updated weights for policy 0, policy_version 1185053 (0.0007) [2023-12-27 13:51:37,866][20347] Updated weights for policy 0, policy_version 1185063 (0.0006) [2023-12-27 13:51:38,047][20381] Updated weights for policy 1, policy_version 1184057 (0.0008) [2023-12-27 13:51:38,103][20381] Updated weights for policy 1, policy_version 1184067 (0.0008) [2023-12-27 13:51:38,161][20381] Updated weights for policy 1, policy_version 1184077 (0.0008) [2023-12-27 13:51:38,222][20381] Updated weights for policy 1, policy_version 1184087 (0.0006) [2023-12-27 13:51:38,582][20347] Updated weights for policy 0, policy_version 1185073 (0.0008) [2023-12-27 13:51:38,640][20347] Updated weights for policy 0, policy_version 1185083 (0.0007) [2023-12-27 13:51:38,686][20347] Updated weights for policy 0, policy_version 1185093 (0.0008) [2023-12-27 13:51:38,743][20347] Updated weights for policy 0, policy_version 1185103 (0.0006) [2023-12-27 13:51:38,881][20381] Updated weights for policy 1, policy_version 1184097 (0.0007) [2023-12-27 13:51:38,929][20381] Updated weights for policy 1, policy_version 1184107 (0.0008) [2023-12-27 13:51:38,984][20381] Updated weights for policy 1, policy_version 1184117 (0.0007) [2023-12-27 13:51:39,473][20347] Updated weights for policy 0, policy_version 1185113 (0.0009) [2023-12-27 13:51:39,534][20347] Updated weights for policy 0, policy_version 1185123 (0.0008) [2023-12-27 13:51:39,595][20347] Updated weights for policy 0, policy_version 1185133 (0.0008) [2023-12-27 13:51:39,707][20381] Updated weights for policy 1, policy_version 1184127 (0.0008) [2023-12-27 13:51:39,770][20381] Updated weights for policy 1, policy_version 1184137 (0.0006) [2023-12-27 13:51:39,834][20381] Updated weights for policy 1, policy_version 1184147 (0.0008) [2023-12-27 13:51:40,348][20347] Updated weights for policy 0, policy_version 1185143 (0.0007) [2023-12-27 13:51:40,414][20347] Updated weights for policy 0, policy_version 1185153 (0.0007) [2023-12-27 13:51:40,470][20347] Updated weights for policy 0, policy_version 1185163 (0.0008) [2023-12-27 13:51:40,511][20381] Updated weights for policy 1, policy_version 1184157 (0.0008) [2023-12-27 13:51:40,571][20381] Updated weights for policy 1, policy_version 1184167 (0.0007) [2023-12-27 13:51:40,628][20381] Updated weights for policy 1, policy_version 1184177 (0.0008) [2023-12-27 13:51:41,193][20347] Updated weights for policy 0, policy_version 1185173 (0.0007) [2023-12-27 13:51:41,253][20347] Updated weights for policy 0, policy_version 1185183 (0.0006) [2023-12-27 13:51:41,309][20347] Updated weights for policy 0, policy_version 1185193 (0.0008) [2023-12-27 13:51:41,384][20381] Updated weights for policy 1, policy_version 1184187 (0.0009) [2023-12-27 13:51:41,441][20381] Updated weights for policy 1, policy_version 1184197 (0.0011) [2023-12-27 13:51:41,503][20381] Updated weights for policy 1, policy_version 1184207 (0.0011) [2023-12-27 13:51:42,036][20347] Updated weights for policy 0, policy_version 1185203 (0.0008) [2023-12-27 13:51:42,097][20347] Updated weights for policy 0, policy_version 1185213 (0.0009) [2023-12-27 13:51:42,151][20347] Updated weights for policy 0, policy_version 1185223 (0.0007) [2023-12-27 13:51:42,219][20381] Updated weights for policy 1, policy_version 1184217 (0.0009) [2023-12-27 13:51:42,281][20381] Updated weights for policy 1, policy_version 1184227 (0.0007) [2023-12-27 13:51:42,343][20381] Updated weights for policy 1, policy_version 1184237 (0.0009) [2023-12-27 13:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 606658560. Throughput: 0: 10270.0, 1: 10228.8. Samples: 606651868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:51:42,363][19170] Avg episode reward: [(0, '941.440'), (1, '882.690')] [2023-12-27 13:51:42,408][20381] Updated weights for policy 1, policy_version 1184247 (0.0009) [2023-12-27 13:51:42,825][20347] Updated weights for policy 0, policy_version 1185233 (0.0007) [2023-12-27 13:51:42,886][20347] Updated weights for policy 0, policy_version 1185243 (0.0006) [2023-12-27 13:51:42,940][20347] Updated weights for policy 0, policy_version 1185253 (0.0009) [2023-12-27 13:51:43,006][20347] Updated weights for policy 0, policy_version 1185263 (0.0008) [2023-12-27 13:51:43,063][20381] Updated weights for policy 1, policy_version 1184257 (0.0008) [2023-12-27 13:51:43,116][20381] Updated weights for policy 1, policy_version 1184267 (0.0008) [2023-12-27 13:51:43,167][20381] Updated weights for policy 1, policy_version 1184277 (0.0008) [2023-12-27 13:51:43,676][20347] Updated weights for policy 0, policy_version 1185273 (0.0007) [2023-12-27 13:51:43,727][20347] Updated weights for policy 0, policy_version 1185283 (0.0008) [2023-12-27 13:51:43,788][20347] Updated weights for policy 0, policy_version 1185293 (0.0007) [2023-12-27 13:51:43,857][20381] Updated weights for policy 1, policy_version 1184287 (0.0008) [2023-12-27 13:51:43,908][20381] Updated weights for policy 1, policy_version 1184297 (0.0007) [2023-12-27 13:51:43,955][20381] Updated weights for policy 1, policy_version 1184307 (0.0007) [2023-12-27 13:51:44,350][20347] Updated weights for policy 0, policy_version 1185303 (0.0007) [2023-12-27 13:51:44,408][20347] Updated weights for policy 0, policy_version 1185313 (0.0008) [2023-12-27 13:51:44,466][20347] Updated weights for policy 0, policy_version 1185323 (0.0008) [2023-12-27 13:51:44,701][20381] Updated weights for policy 1, policy_version 1184317 (0.0007) [2023-12-27 13:51:44,748][20381] Updated weights for policy 1, policy_version 1184327 (0.0007) [2023-12-27 13:51:44,810][20381] Updated weights for policy 1, policy_version 1184337 (0.0007) [2023-12-27 13:51:45,178][20347] Updated weights for policy 0, policy_version 1185333 (0.0009) [2023-12-27 13:51:45,237][20347] Updated weights for policy 0, policy_version 1185343 (0.0009) [2023-12-27 13:51:45,302][20347] Updated weights for policy 0, policy_version 1185353 (0.0009) [2023-12-27 13:51:45,522][20381] Updated weights for policy 1, policy_version 1184347 (0.0009) [2023-12-27 13:51:45,573][20381] Updated weights for policy 1, policy_version 1184357 (0.0007) [2023-12-27 13:51:45,620][20381] Updated weights for policy 1, policy_version 1184367 (0.0007) [2023-12-27 13:51:45,953][20347] Updated weights for policy 0, policy_version 1185363 (0.0009) [2023-12-27 13:51:46,011][20347] Updated weights for policy 0, policy_version 1185373 (0.0007) [2023-12-27 13:51:46,066][20347] Updated weights for policy 0, policy_version 1185383 (0.0008) [2023-12-27 13:51:46,329][20381] Updated weights for policy 1, policy_version 1184377 (0.0007) [2023-12-27 13:51:46,394][20381] Updated weights for policy 1, policy_version 1184387 (0.0008) [2023-12-27 13:51:46,446][20381] Updated weights for policy 1, policy_version 1184397 (0.0008) [2023-12-27 13:51:46,508][20381] Updated weights for policy 1, policy_version 1184407 (0.0007) [2023-12-27 13:51:46,758][20347] Updated weights for policy 0, policy_version 1185393 (0.0007) [2023-12-27 13:51:46,820][20347] Updated weights for policy 0, policy_version 1185403 (0.0011) [2023-12-27 13:51:46,877][20347] Updated weights for policy 0, policy_version 1185413 (0.0011) [2023-12-27 13:51:46,939][20347] Updated weights for policy 0, policy_version 1185423 (0.0010) [2023-12-27 13:51:47,192][20381] Updated weights for policy 1, policy_version 1184417 (0.0008) [2023-12-27 13:51:47,239][20381] Updated weights for policy 1, policy_version 1184427 (0.0007) [2023-12-27 13:51:47,294][20381] Updated weights for policy 1, policy_version 1184437 (0.0007) [2023-12-27 13:51:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 606765056. Throughput: 0: 10295.5, 1: 10260.5. Samples: 606772236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:51:47,363][19170] Avg episode reward: [(0, '951.270'), (1, '921.780')] [2023-12-27 13:51:47,564][20347] Updated weights for policy 0, policy_version 1185433 (0.0006) [2023-12-27 13:51:47,622][20347] Updated weights for policy 0, policy_version 1185443 (0.0005) [2023-12-27 13:51:47,669][20347] Updated weights for policy 0, policy_version 1185453 (0.0008) [2023-12-27 13:51:47,974][20381] Updated weights for policy 1, policy_version 1184447 (0.0009) [2023-12-27 13:51:48,029][20381] Updated weights for policy 1, policy_version 1184457 (0.0010) [2023-12-27 13:51:48,088][20381] Updated weights for policy 1, policy_version 1184467 (0.0008) [2023-12-27 13:51:48,276][20347] Updated weights for policy 0, policy_version 1185463 (0.0007) [2023-12-27 13:51:48,340][20347] Updated weights for policy 0, policy_version 1185473 (0.0009) [2023-12-27 13:51:48,402][20347] Updated weights for policy 0, policy_version 1185483 (0.0007) [2023-12-27 13:51:48,781][20381] Updated weights for policy 1, policy_version 1184477 (0.0007) [2023-12-27 13:51:48,839][20381] Updated weights for policy 1, policy_version 1184487 (0.0007) [2023-12-27 13:51:48,891][20381] Updated weights for policy 1, policy_version 1184497 (0.0007) [2023-12-27 13:51:49,075][20347] Updated weights for policy 0, policy_version 1185493 (0.0010) [2023-12-27 13:51:49,127][20347] Updated weights for policy 0, policy_version 1185503 (0.0010) [2023-12-27 13:51:49,183][20347] Updated weights for policy 0, policy_version 1185513 (0.0010) [2023-12-27 13:51:49,633][20381] Updated weights for policy 1, policy_version 1184507 (0.0009) [2023-12-27 13:51:49,680][20381] Updated weights for policy 1, policy_version 1184517 (0.0011) [2023-12-27 13:51:49,733][20381] Updated weights for policy 1, policy_version 1184527 (0.0010) [2023-12-27 13:51:49,924][20347] Updated weights for policy 0, policy_version 1185523 (0.0009) [2023-12-27 13:51:49,982][20347] Updated weights for policy 0, policy_version 1185533 (0.0006) [2023-12-27 13:51:50,038][20347] Updated weights for policy 0, policy_version 1185543 (0.0006) [2023-12-27 13:51:50,409][20381] Updated weights for policy 1, policy_version 1184537 (0.0005) [2023-12-27 13:51:50,469][20381] Updated weights for policy 1, policy_version 1184547 (0.0008) [2023-12-27 13:51:50,524][20381] Updated weights for policy 1, policy_version 1184557 (0.0007) [2023-12-27 13:51:50,581][20381] Updated weights for policy 1, policy_version 1184567 (0.0006) [2023-12-27 13:51:50,731][20347] Updated weights for policy 0, policy_version 1185553 (0.0006) [2023-12-27 13:51:50,778][20347] Updated weights for policy 0, policy_version 1185563 (0.0007) [2023-12-27 13:51:50,836][20347] Updated weights for policy 0, policy_version 1185573 (0.0007) [2023-12-27 13:51:50,881][20347] Updated weights for policy 0, policy_version 1185583 (0.0007) [2023-12-27 13:51:51,267][20381] Updated weights for policy 1, policy_version 1184577 (0.0007) [2023-12-27 13:51:51,326][20381] Updated weights for policy 1, policy_version 1184587 (0.0006) [2023-12-27 13:51:51,391][20381] Updated weights for policy 1, policy_version 1184597 (0.0008) [2023-12-27 13:51:51,686][20347] Updated weights for policy 0, policy_version 1185593 (0.0011) [2023-12-27 13:51:51,747][20347] Updated weights for policy 0, policy_version 1185603 (0.0008) [2023-12-27 13:51:51,813][20347] Updated weights for policy 0, policy_version 1185613 (0.0007) [2023-12-27 13:51:52,013][20381] Updated weights for policy 1, policy_version 1184607 (0.0005) [2023-12-27 13:51:52,061][20381] Updated weights for policy 1, policy_version 1184617 (0.0005) [2023-12-27 13:51:52,110][20381] Updated weights for policy 1, policy_version 1184627 (0.0005) [2023-12-27 13:51:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 606863360. Throughput: 0: 10304.5, 1: 10281.0. Samples: 606833844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:51:52,363][19170] Avg episode reward: [(0, '951.380'), (1, '882.560')] [2023-12-27 13:51:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001184632_303292416.pth... [2023-12-27 13:51:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001185616_303570944.pth... [2023-12-27 13:51:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001183416_302981120.pth [2023-12-27 13:51:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001184400_303259648.pth [2023-12-27 13:51:52,573][20347] Updated weights for policy 0, policy_version 1185623 (0.0007) [2023-12-27 13:51:52,629][20347] Updated weights for policy 0, policy_version 1185633 (0.0009) [2023-12-27 13:51:52,680][20347] Updated weights for policy 0, policy_version 1185643 (0.0007) [2023-12-27 13:51:52,709][20381] Updated weights for policy 1, policy_version 1184637 (0.0007) [2023-12-27 13:51:52,763][20381] Updated weights for policy 1, policy_version 1184647 (0.0007) [2023-12-27 13:51:52,812][20381] Updated weights for policy 1, policy_version 1184657 (0.0005) [2023-12-27 13:51:53,339][20347] Updated weights for policy 0, policy_version 1185653 (0.0007) [2023-12-27 13:51:53,385][20347] Updated weights for policy 0, policy_version 1185663 (0.0007) [2023-12-27 13:51:53,431][20347] Updated weights for policy 0, policy_version 1185673 (0.0008) [2023-12-27 13:51:53,480][20381] Updated weights for policy 1, policy_version 1184667 (0.0006) [2023-12-27 13:51:53,540][20381] Updated weights for policy 1, policy_version 1184677 (0.0008) [2023-12-27 13:51:53,591][20381] Updated weights for policy 1, policy_version 1184687 (0.0010) [2023-12-27 13:51:54,013][20347] Updated weights for policy 0, policy_version 1185683 (0.0007) [2023-12-27 13:51:54,068][20347] Updated weights for policy 0, policy_version 1185693 (0.0007) [2023-12-27 13:51:54,125][20347] Updated weights for policy 0, policy_version 1185703 (0.0007) [2023-12-27 13:51:54,252][20381] Updated weights for policy 1, policy_version 1184697 (0.0009) [2023-12-27 13:51:54,306][20381] Updated weights for policy 1, policy_version 1184707 (0.0009) [2023-12-27 13:51:54,364][20381] Updated weights for policy 1, policy_version 1184717 (0.0007) [2023-12-27 13:51:54,409][20381] Updated weights for policy 1, policy_version 1184727 (0.0007) [2023-12-27 13:51:54,767][20347] Updated weights for policy 0, policy_version 1185713 (0.0006) [2023-12-27 13:51:54,825][20347] Updated weights for policy 0, policy_version 1185723 (0.0010) [2023-12-27 13:51:54,882][20347] Updated weights for policy 0, policy_version 1185733 (0.0010) [2023-12-27 13:51:54,940][20347] Updated weights for policy 0, policy_version 1185743 (0.0011) [2023-12-27 13:51:55,114][20381] Updated weights for policy 1, policy_version 1184737 (0.0011) [2023-12-27 13:51:55,173][20381] Updated weights for policy 1, policy_version 1184747 (0.0011) [2023-12-27 13:51:55,219][20381] Updated weights for policy 1, policy_version 1184757 (0.0010) [2023-12-27 13:51:55,514][20347] Updated weights for policy 0, policy_version 1185753 (0.0008) [2023-12-27 13:51:55,568][20347] Updated weights for policy 0, policy_version 1185763 (0.0006) [2023-12-27 13:51:55,619][20347] Updated weights for policy 0, policy_version 1185773 (0.0005) [2023-12-27 13:51:55,856][20381] Updated weights for policy 1, policy_version 1184767 (0.0006) [2023-12-27 13:51:55,907][20381] Updated weights for policy 1, policy_version 1184777 (0.0005) [2023-12-27 13:51:55,958][20381] Updated weights for policy 1, policy_version 1184787 (0.0005) [2023-12-27 13:51:56,176][20347] Updated weights for policy 0, policy_version 1185783 (0.0006) [2023-12-27 13:51:56,230][20347] Updated weights for policy 0, policy_version 1185793 (0.0007) [2023-12-27 13:51:56,285][20347] Updated weights for policy 0, policy_version 1185803 (0.0008) [2023-12-27 13:51:56,582][20381] Updated weights for policy 1, policy_version 1184797 (0.0008) [2023-12-27 13:51:56,637][20381] Updated weights for policy 1, policy_version 1184807 (0.0010) [2023-12-27 13:51:56,691][20381] Updated weights for policy 1, policy_version 1184817 (0.0010) [2023-12-27 13:51:56,958][20347] Updated weights for policy 0, policy_version 1185813 (0.0006) [2023-12-27 13:51:57,013][20347] Updated weights for policy 0, policy_version 1185823 (0.0005) [2023-12-27 13:51:57,069][20347] Updated weights for policy 0, policy_version 1185833 (0.0005) [2023-12-27 13:51:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 606969856. Throughput: 0: 10355.1, 1: 10365.3. Samples: 606961052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:51:57,363][19170] Avg episode reward: [(0, '941.740'), (1, '843.470')] [2023-12-27 13:51:57,374][20381] Updated weights for policy 1, policy_version 1184827 (0.0010) [2023-12-27 13:51:57,426][20381] Updated weights for policy 1, policy_version 1184837 (0.0010) [2023-12-27 13:51:57,480][20381] Updated weights for policy 1, policy_version 1184847 (0.0010) [2023-12-27 13:51:57,649][20347] Updated weights for policy 0, policy_version 1185843 (0.0006) [2023-12-27 13:51:57,694][20347] Updated weights for policy 0, policy_version 1185853 (0.0005) [2023-12-27 13:51:57,742][20347] Updated weights for policy 0, policy_version 1185863 (0.0005) [2023-12-27 13:51:58,142][20381] Updated weights for policy 1, policy_version 1184857 (0.0010) [2023-12-27 13:51:58,203][20381] Updated weights for policy 1, policy_version 1184867 (0.0007) [2023-12-27 13:51:58,259][20381] Updated weights for policy 1, policy_version 1184877 (0.0010) [2023-12-27 13:51:58,298][20347] Updated weights for policy 0, policy_version 1185873 (0.0006) [2023-12-27 13:51:58,326][20381] Updated weights for policy 1, policy_version 1184887 (0.0010) [2023-12-27 13:51:58,365][20347] Updated weights for policy 0, policy_version 1185883 (0.0008) [2023-12-27 13:51:58,433][20347] Updated weights for policy 0, policy_version 1185893 (0.0010) [2023-12-27 13:51:58,496][20347] Updated weights for policy 0, policy_version 1185903 (0.0008) [2023-12-27 13:51:59,145][20381] Updated weights for policy 1, policy_version 1184897 (0.0007) [2023-12-27 13:51:59,206][20381] Updated weights for policy 1, policy_version 1184907 (0.0007) [2023-12-27 13:51:59,259][20347] Updated weights for policy 0, policy_version 1185913 (0.0009) [2023-12-27 13:51:59,280][20381] Updated weights for policy 1, policy_version 1184917 (0.0008) [2023-12-27 13:51:59,321][20347] Updated weights for policy 0, policy_version 1185923 (0.0006) [2023-12-27 13:51:59,381][20347] Updated weights for policy 0, policy_version 1185933 (0.0008) [2023-12-27 13:51:59,993][20381] Updated weights for policy 1, policy_version 1184927 (0.0008) [2023-12-27 13:52:00,050][20381] Updated weights for policy 1, policy_version 1184937 (0.0006) [2023-12-27 13:52:00,108][20381] Updated weights for policy 1, policy_version 1184947 (0.0008) [2023-12-27 13:52:00,110][20347] Updated weights for policy 0, policy_version 1185943 (0.0007) [2023-12-27 13:52:00,161][20347] Updated weights for policy 0, policy_version 1185953 (0.0007) [2023-12-27 13:52:00,217][20347] Updated weights for policy 0, policy_version 1185963 (0.0007) [2023-12-27 13:52:00,788][20381] Updated weights for policy 1, policy_version 1184957 (0.0010) [2023-12-27 13:52:00,836][20381] Updated weights for policy 1, policy_version 1184967 (0.0010) [2023-12-27 13:52:00,860][20347] Updated weights for policy 0, policy_version 1185973 (0.0006) [2023-12-27 13:52:00,885][20381] Updated weights for policy 1, policy_version 1184977 (0.0010) [2023-12-27 13:52:00,909][20347] Updated weights for policy 0, policy_version 1185983 (0.0005) [2023-12-27 13:52:00,966][20347] Updated weights for policy 0, policy_version 1185993 (0.0007) [2023-12-27 13:52:01,538][20381] Updated weights for policy 1, policy_version 1184987 (0.0009) [2023-12-27 13:52:01,595][20381] Updated weights for policy 1, policy_version 1184997 (0.0008) [2023-12-27 13:52:01,655][20381] Updated weights for policy 1, policy_version 1185007 (0.0008) [2023-12-27 13:52:01,671][20347] Updated weights for policy 0, policy_version 1186003 (0.0007) [2023-12-27 13:52:01,726][20347] Updated weights for policy 0, policy_version 1186013 (0.0007) [2023-12-27 13:52:01,785][20347] Updated weights for policy 0, policy_version 1186023 (0.0009) [2023-12-27 13:52:02,232][20381] Updated weights for policy 1, policy_version 1185017 (0.0006) [2023-12-27 13:52:02,290][20381] Updated weights for policy 1, policy_version 1185027 (0.0007) [2023-12-27 13:52:02,343][20381] Updated weights for policy 1, policy_version 1185037 (0.0005) [2023-12-27 13:52:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 607068160. Throughput: 0: 10399.7, 1: 10345.9. Samples: 607083652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:02,363][19170] Avg episode reward: [(0, '883.110'), (1, '853.400')] [2023-12-27 13:52:02,410][20381] Updated weights for policy 1, policy_version 1185047 (0.0008) [2023-12-27 13:52:02,518][20347] Updated weights for policy 0, policy_version 1186033 (0.0009) [2023-12-27 13:52:02,574][20347] Updated weights for policy 0, policy_version 1186043 (0.0006) [2023-12-27 13:52:02,632][20347] Updated weights for policy 0, policy_version 1186053 (0.0007) [2023-12-27 13:52:02,688][20347] Updated weights for policy 0, policy_version 1186063 (0.0008) [2023-12-27 13:52:03,062][20381] Updated weights for policy 1, policy_version 1185057 (0.0007) [2023-12-27 13:52:03,107][20381] Updated weights for policy 1, policy_version 1185067 (0.0009) [2023-12-27 13:52:03,152][20381] Updated weights for policy 1, policy_version 1185077 (0.0008) [2023-12-27 13:52:03,160][20292] KL-divergence is very high: 123.9179 [2023-12-27 13:52:03,296][20347] Updated weights for policy 0, policy_version 1186073 (0.0006) [2023-12-27 13:52:03,343][20347] Updated weights for policy 0, policy_version 1186083 (0.0006) [2023-12-27 13:52:03,401][20347] Updated weights for policy 0, policy_version 1186093 (0.0008) [2023-12-27 13:52:03,885][20381] Updated weights for policy 1, policy_version 1185087 (0.0007) [2023-12-27 13:52:03,940][20381] Updated weights for policy 1, policy_version 1185097 (0.0007) [2023-12-27 13:52:04,000][20381] Updated weights for policy 1, policy_version 1185107 (0.0007) [2023-12-27 13:52:04,088][20347] Updated weights for policy 0, policy_version 1186103 (0.0008) [2023-12-27 13:52:04,146][20347] Updated weights for policy 0, policy_version 1186113 (0.0007) [2023-12-27 13:52:04,207][20347] Updated weights for policy 0, policy_version 1186123 (0.0007) [2023-12-27 13:52:04,699][20381] Updated weights for policy 1, policy_version 1185117 (0.0009) [2023-12-27 13:52:04,754][20381] Updated weights for policy 1, policy_version 1185127 (0.0010) [2023-12-27 13:52:04,809][20381] Updated weights for policy 1, policy_version 1185137 (0.0008) [2023-12-27 13:52:04,849][20347] Updated weights for policy 0, policy_version 1186133 (0.0007) [2023-12-27 13:52:04,911][20347] Updated weights for policy 0, policy_version 1186143 (0.0006) [2023-12-27 13:52:04,977][20347] Updated weights for policy 0, policy_version 1186153 (0.0008) [2023-12-27 13:52:05,530][20381] Updated weights for policy 1, policy_version 1185147 (0.0007) [2023-12-27 13:52:05,580][20381] Updated weights for policy 1, policy_version 1185157 (0.0007) [2023-12-27 13:52:05,610][20347] Updated weights for policy 0, policy_version 1186163 (0.0007) [2023-12-27 13:52:05,633][20381] Updated weights for policy 1, policy_version 1185167 (0.0006) [2023-12-27 13:52:05,648][20291] KL-divergence is very high: 174.5961 [2023-12-27 13:52:05,658][20291] KL-divergence is very high: 174.4432 [2023-12-27 13:52:05,663][20347] Updated weights for policy 0, policy_version 1186173 (0.0008) [2023-12-27 13:52:05,690][20291] KL-divergence is very high: 106.8580 [2023-12-27 13:52:05,715][20347] Updated weights for policy 0, policy_version 1186183 (0.0009) [2023-12-27 13:52:06,261][20381] Updated weights for policy 1, policy_version 1185177 (0.0005) [2023-12-27 13:52:06,315][20381] Updated weights for policy 1, policy_version 1185187 (0.0009) [2023-12-27 13:52:06,369][20381] Updated weights for policy 1, policy_version 1185197 (0.0010) [2023-12-27 13:52:06,435][20381] Updated weights for policy 1, policy_version 1185207 (0.0011) [2023-12-27 13:52:06,459][20347] Updated weights for policy 0, policy_version 1186193 (0.0007) [2023-12-27 13:52:06,523][20347] Updated weights for policy 0, policy_version 1186203 (0.0008) [2023-12-27 13:52:06,584][20347] Updated weights for policy 0, policy_version 1186213 (0.0008) [2023-12-27 13:52:06,647][20347] Updated weights for policy 0, policy_version 1186223 (0.0008) [2023-12-27 13:52:07,066][20381] Updated weights for policy 1, policy_version 1185217 (0.0010) [2023-12-27 13:52:07,113][20381] Updated weights for policy 1, policy_version 1185227 (0.0008) [2023-12-27 13:52:07,158][20381] Updated weights for policy 1, policy_version 1185237 (0.0005) [2023-12-27 13:52:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 607174656. Throughput: 0: 10408.0, 1: 10337.3. Samples: 607144864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:07,363][19170] Avg episode reward: [(0, '931.700'), (1, '902.270')] [2023-12-27 13:52:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001185240_303448064.pth... [2023-12-27 13:52:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001184056_303144960.pth [2023-12-27 13:52:07,399][20347] Updated weights for policy 0, policy_version 1186233 (0.0008) [2023-12-27 13:52:07,451][20347] Updated weights for policy 0, policy_version 1186243 (0.0009) [2023-12-27 13:52:07,507][20347] Updated weights for policy 0, policy_version 1186253 (0.0009) [2023-12-27 13:52:07,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001186256_303734784.pth... [2023-12-27 13:52:07,530][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001185040_303423488.pth [2023-12-27 13:52:07,728][20381] Updated weights for policy 1, policy_version 1185247 (0.0008) [2023-12-27 13:52:07,774][20381] Updated weights for policy 1, policy_version 1185257 (0.0008) [2023-12-27 13:52:07,819][20381] Updated weights for policy 1, policy_version 1185267 (0.0007) [2023-12-27 13:52:08,220][20347] Updated weights for policy 0, policy_version 1186263 (0.0007) [2023-12-27 13:52:08,274][20347] Updated weights for policy 0, policy_version 1186273 (0.0007) [2023-12-27 13:52:08,326][20347] Updated weights for policy 0, policy_version 1186283 (0.0007) [2023-12-27 13:52:08,536][20381] Updated weights for policy 1, policy_version 1185277 (0.0008) [2023-12-27 13:52:08,603][20381] Updated weights for policy 1, policy_version 1185287 (0.0008) [2023-12-27 13:52:08,666][20381] Updated weights for policy 1, policy_version 1185297 (0.0008) [2023-12-27 13:52:09,021][20347] Updated weights for policy 0, policy_version 1186293 (0.0008) [2023-12-27 13:52:09,067][20347] Updated weights for policy 0, policy_version 1186303 (0.0008) [2023-12-27 13:52:09,118][20347] Updated weights for policy 0, policy_version 1186313 (0.0008) [2023-12-27 13:52:09,351][20381] Updated weights for policy 1, policy_version 1185307 (0.0008) [2023-12-27 13:52:09,423][20381] Updated weights for policy 1, policy_version 1185317 (0.0008) [2023-12-27 13:52:09,483][20381] Updated weights for policy 1, policy_version 1185327 (0.0008) [2023-12-27 13:52:09,894][20347] Updated weights for policy 0, policy_version 1186323 (0.0008) [2023-12-27 13:52:09,953][20347] Updated weights for policy 0, policy_version 1186333 (0.0008) [2023-12-27 13:52:10,011][20347] Updated weights for policy 0, policy_version 1186343 (0.0008) [2023-12-27 13:52:10,201][20381] Updated weights for policy 1, policy_version 1185337 (0.0008) [2023-12-27 13:52:10,251][20381] Updated weights for policy 1, policy_version 1185347 (0.0008) [2023-12-27 13:52:10,298][20381] Updated weights for policy 1, policy_version 1185357 (0.0008) [2023-12-27 13:52:10,351][20381] Updated weights for policy 1, policy_version 1185367 (0.0009) [2023-12-27 13:52:10,729][20347] Updated weights for policy 0, policy_version 1186353 (0.0008) [2023-12-27 13:52:10,792][20347] Updated weights for policy 0, policy_version 1186363 (0.0009) [2023-12-27 13:52:10,852][20347] Updated weights for policy 0, policy_version 1186373 (0.0010) [2023-12-27 13:52:10,907][20347] Updated weights for policy 0, policy_version 1186383 (0.0010) [2023-12-27 13:52:11,066][20381] Updated weights for policy 1, policy_version 1185377 (0.0007) [2023-12-27 13:52:11,138][20381] Updated weights for policy 1, policy_version 1185388 (0.0007) [2023-12-27 13:52:11,195][20381] Updated weights for policy 1, policy_version 1185398 (0.0006) [2023-12-27 13:52:11,721][20347] Updated weights for policy 0, policy_version 1186393 (0.0008) [2023-12-27 13:52:11,791][20347] Updated weights for policy 0, policy_version 1186403 (0.0007) [2023-12-27 13:52:11,851][20347] Updated weights for policy 0, policy_version 1186413 (0.0006) [2023-12-27 13:52:11,955][20381] Updated weights for policy 1, policy_version 1185408 (0.0008) [2023-12-27 13:52:12,003][20381] Updated weights for policy 1, policy_version 1185418 (0.0010) [2023-12-27 13:52:12,061][20381] Updated weights for policy 1, policy_version 1185428 (0.0009) [2023-12-27 13:52:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 607272960. Throughput: 0: 10314.3, 1: 10286.8. Samples: 607264820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:12,363][19170] Avg episode reward: [(0, '941.540'), (1, '892.350')] [2023-12-27 13:52:12,485][20347] Updated weights for policy 0, policy_version 1186423 (0.0007) [2023-12-27 13:52:12,537][20347] Updated weights for policy 0, policy_version 1186433 (0.0007) [2023-12-27 13:52:12,582][20347] Updated weights for policy 0, policy_version 1186443 (0.0007) [2023-12-27 13:52:12,830][20381] Updated weights for policy 1, policy_version 1185438 (0.0009) [2023-12-27 13:52:12,888][20381] Updated weights for policy 1, policy_version 1185448 (0.0008) [2023-12-27 13:52:12,945][20381] Updated weights for policy 1, policy_version 1185458 (0.0007) [2023-12-27 13:52:13,272][20347] Updated weights for policy 0, policy_version 1186453 (0.0008) [2023-12-27 13:52:13,330][20347] Updated weights for policy 0, policy_version 1186463 (0.0007) [2023-12-27 13:52:13,389][20347] Updated weights for policy 0, policy_version 1186473 (0.0008) [2023-12-27 13:52:13,684][20381] Updated weights for policy 1, policy_version 1185468 (0.0008) [2023-12-27 13:52:13,734][20381] Updated weights for policy 1, policy_version 1185478 (0.0010) [2023-12-27 13:52:13,784][20381] Updated weights for policy 1, policy_version 1185488 (0.0008) [2023-12-27 13:52:13,961][20347] Updated weights for policy 0, policy_version 1186483 (0.0007) [2023-12-27 13:52:14,007][20347] Updated weights for policy 0, policy_version 1186493 (0.0005) [2023-12-27 13:52:14,059][20347] Updated weights for policy 0, policy_version 1186503 (0.0008) [2023-12-27 13:52:14,418][20381] Updated weights for policy 1, policy_version 1185498 (0.0005) [2023-12-27 13:52:14,471][20381] Updated weights for policy 1, policy_version 1185508 (0.0007) [2023-12-27 13:52:14,523][20381] Updated weights for policy 1, policy_version 1185518 (0.0008) [2023-12-27 13:52:14,567][20381] Updated weights for policy 1, policy_version 1185528 (0.0009) [2023-12-27 13:52:14,695][20347] Updated weights for policy 0, policy_version 1186514 (0.0008) [2023-12-27 13:52:14,747][20347] Updated weights for policy 0, policy_version 1186524 (0.0009) [2023-12-27 13:52:14,829][20347] Updated weights for policy 0, policy_version 1186539 (0.0007) [2023-12-27 13:52:15,257][20381] Updated weights for policy 1, policy_version 1185538 (0.0007) [2023-12-27 13:52:15,311][20381] Updated weights for policy 1, policy_version 1185548 (0.0005) [2023-12-27 13:52:15,368][20381] Updated weights for policy 1, policy_version 1185558 (0.0005) [2023-12-27 13:52:15,504][20347] Updated weights for policy 0, policy_version 1186549 (0.0010) [2023-12-27 13:52:15,562][20347] Updated weights for policy 0, policy_version 1186559 (0.0009) [2023-12-27 13:52:15,623][20347] Updated weights for policy 0, policy_version 1186569 (0.0005) [2023-12-27 13:52:15,905][20381] Updated weights for policy 1, policy_version 1185568 (0.0006) [2023-12-27 13:52:15,960][20381] Updated weights for policy 1, policy_version 1185578 (0.0008) [2023-12-27 13:52:16,019][20381] Updated weights for policy 1, policy_version 1185588 (0.0008) [2023-12-27 13:52:16,196][20347] Updated weights for policy 0, policy_version 1186579 (0.0007) [2023-12-27 13:52:16,248][20347] Updated weights for policy 0, policy_version 1186589 (0.0010) [2023-12-27 13:52:16,295][20347] Updated weights for policy 0, policy_version 1186599 (0.0010) [2023-12-27 13:52:16,695][20381] Updated weights for policy 1, policy_version 1185598 (0.0007) [2023-12-27 13:52:16,754][20381] Updated weights for policy 1, policy_version 1185608 (0.0008) [2023-12-27 13:52:16,804][20381] Updated weights for policy 1, policy_version 1185618 (0.0007) [2023-12-27 13:52:17,007][20347] Updated weights for policy 0, policy_version 1186609 (0.0010) [2023-12-27 13:52:17,060][20347] Updated weights for policy 0, policy_version 1186619 (0.0010) [2023-12-27 13:52:17,120][20347] Updated weights for policy 0, policy_version 1186629 (0.0009) [2023-12-27 13:52:17,172][20347] Updated weights for policy 0, policy_version 1186639 (0.0010) [2023-12-27 13:52:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 607379456. Throughput: 0: 10287.8, 1: 10202.1. Samples: 607388892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:17,363][19170] Avg episode reward: [(0, '931.760'), (1, '872.780')] [2023-12-27 13:52:17,511][20381] Updated weights for policy 1, policy_version 1185628 (0.0008) [2023-12-27 13:52:17,557][20381] Updated weights for policy 1, policy_version 1185638 (0.0008) [2023-12-27 13:52:17,605][20381] Updated weights for policy 1, policy_version 1185648 (0.0009) [2023-12-27 13:52:17,812][20347] Updated weights for policy 0, policy_version 1186649 (0.0008) [2023-12-27 13:52:17,860][20347] Updated weights for policy 0, policy_version 1186659 (0.0008) [2023-12-27 13:52:17,912][20347] Updated weights for policy 0, policy_version 1186669 (0.0008) [2023-12-27 13:52:18,327][20381] Updated weights for policy 1, policy_version 1185658 (0.0008) [2023-12-27 13:52:18,386][20381] Updated weights for policy 1, policy_version 1185668 (0.0006) [2023-12-27 13:52:18,446][20381] Updated weights for policy 1, policy_version 1185678 (0.0006) [2023-12-27 13:52:18,498][20381] Updated weights for policy 1, policy_version 1185688 (0.0006) [2023-12-27 13:52:18,654][20347] Updated weights for policy 0, policy_version 1186679 (0.0008) [2023-12-27 13:52:18,706][20347] Updated weights for policy 0, policy_version 1186689 (0.0008) [2023-12-27 13:52:18,768][20347] Updated weights for policy 0, policy_version 1186699 (0.0009) [2023-12-27 13:52:19,118][20292] KL-divergence is very high: 170.6651 [2023-12-27 13:52:19,128][20381] Updated weights for policy 1, policy_version 1185698 (0.0007) [2023-12-27 13:52:19,175][20381] Updated weights for policy 1, policy_version 1185708 (0.0008) [2023-12-27 13:52:19,231][20381] Updated weights for policy 1, policy_version 1185718 (0.0006) [2023-12-27 13:52:19,492][20347] Updated weights for policy 0, policy_version 1186709 (0.0008) [2023-12-27 13:52:19,551][20347] Updated weights for policy 0, policy_version 1186719 (0.0008) [2023-12-27 13:52:19,604][20347] Updated weights for policy 0, policy_version 1186729 (0.0008) [2023-12-27 13:52:19,986][20381] Updated weights for policy 1, policy_version 1185728 (0.0011) [2023-12-27 13:52:20,037][20381] Updated weights for policy 1, policy_version 1185738 (0.0011) [2023-12-27 13:52:20,088][20381] Updated weights for policy 1, policy_version 1185748 (0.0009) [2023-12-27 13:52:20,377][20347] Updated weights for policy 0, policy_version 1186739 (0.0008) [2023-12-27 13:52:20,430][20347] Updated weights for policy 0, policy_version 1186749 (0.0008) [2023-12-27 13:52:20,480][20347] Updated weights for policy 0, policy_version 1186759 (0.0008) [2023-12-27 13:52:20,851][20381] Updated weights for policy 1, policy_version 1185758 (0.0007) [2023-12-27 13:52:20,927][20381] Updated weights for policy 1, policy_version 1185768 (0.0008) [2023-12-27 13:52:20,989][20381] Updated weights for policy 1, policy_version 1185778 (0.0008) [2023-12-27 13:52:21,253][20347] Updated weights for policy 0, policy_version 1186769 (0.0008) [2023-12-27 13:52:21,318][20347] Updated weights for policy 0, policy_version 1186779 (0.0010) [2023-12-27 13:52:21,389][20347] Updated weights for policy 0, policy_version 1186789 (0.0011) [2023-12-27 13:52:21,450][20347] Updated weights for policy 0, policy_version 1186799 (0.0010) [2023-12-27 13:52:21,734][20381] Updated weights for policy 1, policy_version 1185788 (0.0007) [2023-12-27 13:52:21,802][20381] Updated weights for policy 1, policy_version 1185798 (0.0007) [2023-12-27 13:52:21,863][20381] Updated weights for policy 1, policy_version 1185808 (0.0007) [2023-12-27 13:52:22,200][20347] Updated weights for policy 0, policy_version 1186809 (0.0010) [2023-12-27 13:52:22,261][20347] Updated weights for policy 0, policy_version 1186819 (0.0012) [2023-12-27 13:52:22,322][20347] Updated weights for policy 0, policy_version 1186829 (0.0007) [2023-12-27 13:52:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 607477760. Throughput: 0: 10227.3, 1: 10171.8. Samples: 607448096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:22,363][19170] Avg episode reward: [(0, '912.770'), (1, '872.890')] [2023-12-27 13:52:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001185816_303595520.pth... [2023-12-27 13:52:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001186832_303882240.pth... [2023-12-27 13:52:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001184632_303292416.pth [2023-12-27 13:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001185616_303570944.pth [2023-12-27 13:52:22,531][20381] Updated weights for policy 1, policy_version 1185818 (0.0007) [2023-12-27 13:52:22,587][20381] Updated weights for policy 1, policy_version 1185828 (0.0007) [2023-12-27 13:52:22,644][20381] Updated weights for policy 1, policy_version 1185838 (0.0008) [2023-12-27 13:52:22,705][20381] Updated weights for policy 1, policy_version 1185848 (0.0008) [2023-12-27 13:52:23,014][20347] Updated weights for policy 0, policy_version 1186839 (0.0007) [2023-12-27 13:52:23,071][20347] Updated weights for policy 0, policy_version 1186849 (0.0008) [2023-12-27 13:52:23,122][20347] Updated weights for policy 0, policy_version 1186859 (0.0010) [2023-12-27 13:52:23,335][20381] Updated weights for policy 1, policy_version 1185858 (0.0005) [2023-12-27 13:52:23,387][20381] Updated weights for policy 1, policy_version 1185868 (0.0005) [2023-12-27 13:52:23,433][20381] Updated weights for policy 1, policy_version 1185878 (0.0005) [2023-12-27 13:52:23,823][20347] Updated weights for policy 0, policy_version 1186869 (0.0009) [2023-12-27 13:52:23,868][20347] Updated weights for policy 0, policy_version 1186879 (0.0009) [2023-12-27 13:52:23,923][20347] Updated weights for policy 0, policy_version 1186889 (0.0008) [2023-12-27 13:52:23,994][20381] Updated weights for policy 1, policy_version 1185888 (0.0011) [2023-12-27 13:52:24,049][20381] Updated weights for policy 1, policy_version 1185898 (0.0010) [2023-12-27 13:52:24,104][20381] Updated weights for policy 1, policy_version 1185908 (0.0010) [2023-12-27 13:52:24,602][20347] Updated weights for policy 0, policy_version 1186899 (0.0007) [2023-12-27 13:52:24,662][20347] Updated weights for policy 0, policy_version 1186909 (0.0006) [2023-12-27 13:52:24,681][20381] Updated weights for policy 1, policy_version 1185918 (0.0008) [2023-12-27 13:52:24,710][20347] Updated weights for policy 0, policy_version 1186919 (0.0008) [2023-12-27 13:52:24,734][20381] Updated weights for policy 1, policy_version 1185928 (0.0010) [2023-12-27 13:52:24,788][20381] Updated weights for policy 1, policy_version 1185938 (0.0009) [2023-12-27 13:52:25,364][20381] Updated weights for policy 1, policy_version 1185948 (0.0007) [2023-12-27 13:52:25,391][20347] Updated weights for policy 0, policy_version 1186929 (0.0006) [2023-12-27 13:52:25,413][20381] Updated weights for policy 1, policy_version 1185958 (0.0010) [2023-12-27 13:52:25,440][20347] Updated weights for policy 0, policy_version 1186939 (0.0006) [2023-12-27 13:52:25,468][20381] Updated weights for policy 1, policy_version 1185968 (0.0011) [2023-12-27 13:52:25,484][20347] Updated weights for policy 0, policy_version 1186949 (0.0005) [2023-12-27 13:52:25,528][20347] Updated weights for policy 0, policy_version 1186959 (0.0008) [2023-12-27 13:52:26,059][20381] Updated weights for policy 1, policy_version 1185978 (0.0009) [2023-12-27 13:52:26,109][20381] Updated weights for policy 1, policy_version 1185988 (0.0006) [2023-12-27 13:52:26,156][20381] Updated weights for policy 1, policy_version 1185998 (0.0006) [2023-12-27 13:52:26,198][20347] Updated weights for policy 0, policy_version 1186969 (0.0008) [2023-12-27 13:52:26,205][20381] Updated weights for policy 1, policy_version 1186008 (0.0005) [2023-12-27 13:52:26,256][20347] Updated weights for policy 0, policy_version 1186979 (0.0009) [2023-12-27 13:52:26,304][20347] Updated weights for policy 0, policy_version 1186989 (0.0008) [2023-12-27 13:52:26,834][20381] Updated weights for policy 1, policy_version 1186018 (0.0009) [2023-12-27 13:52:26,893][20381] Updated weights for policy 1, policy_version 1186028 (0.0008) [2023-12-27 13:52:26,919][20347] Updated weights for policy 0, policy_version 1186999 (0.0008) [2023-12-27 13:52:26,947][20381] Updated weights for policy 1, policy_version 1186038 (0.0005) [2023-12-27 13:52:26,970][20347] Updated weights for policy 0, policy_version 1187009 (0.0008) [2023-12-27 13:52:27,014][20347] Updated weights for policy 0, policy_version 1187019 (0.0007) [2023-12-27 13:52:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 607584256. Throughput: 0: 10197.8, 1: 10252.8. Samples: 607572144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:27,363][19170] Avg episode reward: [(0, '903.250'), (1, '921.890')] [2023-12-27 13:52:27,612][20381] Updated weights for policy 1, policy_version 1186048 (0.0010) [2023-12-27 13:52:27,658][20381] Updated weights for policy 1, policy_version 1186058 (0.0005) [2023-12-27 13:52:27,680][20347] Updated weights for policy 0, policy_version 1187029 (0.0006) [2023-12-27 13:52:27,708][20381] Updated weights for policy 1, policy_version 1186068 (0.0006) [2023-12-27 13:52:27,739][20347] Updated weights for policy 0, policy_version 1187039 (0.0008) [2023-12-27 13:52:27,800][20347] Updated weights for policy 0, policy_version 1187049 (0.0010) [2023-12-27 13:52:28,293][20381] Updated weights for policy 1, policy_version 1186078 (0.0007) [2023-12-27 13:52:28,362][20381] Updated weights for policy 1, policy_version 1186088 (0.0006) [2023-12-27 13:52:28,419][20347] Updated weights for policy 0, policy_version 1187059 (0.0009) [2023-12-27 13:52:28,427][20381] Updated weights for policy 1, policy_version 1186098 (0.0009) [2023-12-27 13:52:28,473][20347] Updated weights for policy 0, policy_version 1187069 (0.0009) [2023-12-27 13:52:28,527][20347] Updated weights for policy 0, policy_version 1187079 (0.0006) [2023-12-27 13:52:29,071][20381] Updated weights for policy 1, policy_version 1186108 (0.0007) [2023-12-27 13:52:29,126][20381] Updated weights for policy 1, policy_version 1186118 (0.0006) [2023-12-27 13:52:29,140][20347] Updated weights for policy 0, policy_version 1187089 (0.0010) [2023-12-27 13:52:29,188][20381] Updated weights for policy 1, policy_version 1186128 (0.0005) [2023-12-27 13:52:29,189][20347] Updated weights for policy 0, policy_version 1187099 (0.0005) [2023-12-27 13:52:29,251][20347] Updated weights for policy 0, policy_version 1187109 (0.0007) [2023-12-27 13:52:29,313][20347] Updated weights for policy 0, policy_version 1187119 (0.0006) [2023-12-27 13:52:29,793][20381] Updated weights for policy 1, policy_version 1186138 (0.0008) [2023-12-27 13:52:29,855][20381] Updated weights for policy 1, policy_version 1186148 (0.0007) [2023-12-27 13:52:29,898][20347] Updated weights for policy 0, policy_version 1187129 (0.0010) [2023-12-27 13:52:29,914][20381] Updated weights for policy 1, policy_version 1186158 (0.0007) [2023-12-27 13:52:29,962][20347] Updated weights for policy 0, policy_version 1187139 (0.0011) [2023-12-27 13:52:29,972][20381] Updated weights for policy 1, policy_version 1186168 (0.0006) [2023-12-27 13:52:30,018][20347] Updated weights for policy 0, policy_version 1187149 (0.0009) [2023-12-27 13:52:30,611][20381] Updated weights for policy 1, policy_version 1186178 (0.0010) [2023-12-27 13:52:30,662][20381] Updated weights for policy 1, policy_version 1186188 (0.0011) [2023-12-27 13:52:30,679][20347] Updated weights for policy 0, policy_version 1187159 (0.0006) [2023-12-27 13:52:30,714][20381] Updated weights for policy 1, policy_version 1186198 (0.0011) [2023-12-27 13:52:30,731][20347] Updated weights for policy 0, policy_version 1187169 (0.0005) [2023-12-27 13:52:30,796][20347] Updated weights for policy 0, policy_version 1187179 (0.0006) [2023-12-27 13:52:31,400][20381] Updated weights for policy 1, policy_version 1186208 (0.0010) [2023-12-27 13:52:31,449][20347] Updated weights for policy 0, policy_version 1187189 (0.0008) [2023-12-27 13:52:31,457][20381] Updated weights for policy 1, policy_version 1186218 (0.0011) [2023-12-27 13:52:31,502][20347] Updated weights for policy 0, policy_version 1187199 (0.0010) [2023-12-27 13:52:31,518][20381] Updated weights for policy 1, policy_version 1186228 (0.0010) [2023-12-27 13:52:31,567][20347] Updated weights for policy 0, policy_version 1187209 (0.0010) [2023-12-27 13:52:32,177][20381] Updated weights for policy 1, policy_version 1186238 (0.0008) [2023-12-27 13:52:32,223][20381] Updated weights for policy 1, policy_version 1186248 (0.0008) [2023-12-27 13:52:32,287][20381] Updated weights for policy 1, policy_version 1186258 (0.0008) [2023-12-27 13:52:32,304][20347] Updated weights for policy 0, policy_version 1187219 (0.0010) [2023-12-27 13:52:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 607690752. Throughput: 0: 10254.2, 1: 10361.5. Samples: 607699944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:32,363][19170] Avg episode reward: [(0, '922.130'), (1, '902.330')] [2023-12-27 13:52:32,367][20347] Updated weights for policy 0, policy_version 1187229 (0.0007) [2023-12-27 13:52:32,424][20347] Updated weights for policy 0, policy_version 1187239 (0.0006) [2023-12-27 13:52:33,009][20381] Updated weights for policy 1, policy_version 1186268 (0.0007) [2023-12-27 13:52:33,058][20381] Updated weights for policy 1, policy_version 1186278 (0.0007) [2023-12-27 13:52:33,085][20347] Updated weights for policy 0, policy_version 1187249 (0.0007) [2023-12-27 13:52:33,113][20381] Updated weights for policy 1, policy_version 1186288 (0.0006) [2023-12-27 13:52:33,139][20347] Updated weights for policy 0, policy_version 1187259 (0.0010) [2023-12-27 13:52:33,186][20347] Updated weights for policy 0, policy_version 1187269 (0.0010) [2023-12-27 13:52:33,227][20347] Updated weights for policy 0, policy_version 1187279 (0.0006) [2023-12-27 13:52:33,821][20381] Updated weights for policy 1, policy_version 1186298 (0.0006) [2023-12-27 13:52:33,839][20347] Updated weights for policy 0, policy_version 1187289 (0.0005) [2023-12-27 13:52:33,869][20381] Updated weights for policy 1, policy_version 1186308 (0.0007) [2023-12-27 13:52:33,882][20347] Updated weights for policy 0, policy_version 1187299 (0.0005) [2023-12-27 13:52:33,910][20381] Updated weights for policy 1, policy_version 1186318 (0.0007) [2023-12-27 13:52:33,927][20347] Updated weights for policy 0, policy_version 1187309 (0.0006) [2023-12-27 13:52:33,956][20381] Updated weights for policy 1, policy_version 1186328 (0.0006) [2023-12-27 13:52:34,641][20347] Updated weights for policy 0, policy_version 1187319 (0.0011) [2023-12-27 13:52:34,675][20381] Updated weights for policy 1, policy_version 1186338 (0.0008) [2023-12-27 13:52:34,697][20347] Updated weights for policy 0, policy_version 1187329 (0.0007) [2023-12-27 13:52:34,715][20291] KL-divergence is very high: 179.4143 [2023-12-27 13:52:34,722][20291] KL-divergence is very high: 187.6723 [2023-12-27 13:52:34,733][20381] Updated weights for policy 1, policy_version 1186348 (0.0008) [2023-12-27 13:52:34,754][20347] Updated weights for policy 0, policy_version 1187339 (0.0007) [2023-12-27 13:52:34,781][20381] Updated weights for policy 1, policy_version 1186358 (0.0008) [2023-12-27 13:52:35,332][20347] Updated weights for policy 0, policy_version 1187349 (0.0008) [2023-12-27 13:52:35,378][20347] Updated weights for policy 0, policy_version 1187359 (0.0010) [2023-12-27 13:52:35,426][20347] Updated weights for policy 0, policy_version 1187369 (0.0010) [2023-12-27 13:52:35,460][20381] Updated weights for policy 1, policy_version 1186368 (0.0008) [2023-12-27 13:52:35,507][20381] Updated weights for policy 1, policy_version 1186378 (0.0007) [2023-12-27 13:52:35,551][20381] Updated weights for policy 1, policy_version 1186388 (0.0007) [2023-12-27 13:52:36,113][20347] Updated weights for policy 0, policy_version 1187379 (0.0010) [2023-12-27 13:52:36,173][20347] Updated weights for policy 0, policy_version 1187389 (0.0009) [2023-12-27 13:52:36,236][20347] Updated weights for policy 0, policy_version 1187399 (0.0011) [2023-12-27 13:52:36,249][20381] Updated weights for policy 1, policy_version 1186398 (0.0006) [2023-12-27 13:52:36,309][20381] Updated weights for policy 1, policy_version 1186408 (0.0006) [2023-12-27 13:52:36,376][20381] Updated weights for policy 1, policy_version 1186418 (0.0008) [2023-12-27 13:52:36,896][20347] Updated weights for policy 0, policy_version 1187409 (0.0010) [2023-12-27 13:52:36,967][20347] Updated weights for policy 0, policy_version 1187419 (0.0010) [2023-12-27 13:52:37,019][20381] Updated weights for policy 1, policy_version 1186428 (0.0006) [2023-12-27 13:52:37,028][20347] Updated weights for policy 0, policy_version 1187429 (0.0005) [2023-12-27 13:52:37,073][20381] Updated weights for policy 1, policy_version 1186438 (0.0005) [2023-12-27 13:52:37,085][20347] Updated weights for policy 0, policy_version 1187439 (0.0005) [2023-12-27 13:52:37,125][20381] Updated weights for policy 1, policy_version 1186448 (0.0005) [2023-12-27 13:52:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 607797248. Throughput: 0: 10266.6, 1: 10349.0. Samples: 607761544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:37,363][19170] Avg episode reward: [(0, '833.910'), (1, '863.210')] [2023-12-27 13:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001186456_303759360.pth... [2023-12-27 13:52:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001187440_304037888.pth... [2023-12-27 13:52:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001185240_303448064.pth [2023-12-27 13:52:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001186256_303734784.pth [2023-12-27 13:52:37,609][20347] Updated weights for policy 0, policy_version 1187449 (0.0010) [2023-12-27 13:52:37,665][20347] Updated weights for policy 0, policy_version 1187459 (0.0009) [2023-12-27 13:52:37,730][20381] Updated weights for policy 1, policy_version 1186458 (0.0006) [2023-12-27 13:52:37,731][20347] Updated weights for policy 0, policy_version 1187469 (0.0009) [2023-12-27 13:52:37,779][20381] Updated weights for policy 1, policy_version 1186468 (0.0009) [2023-12-27 13:52:37,829][20381] Updated weights for policy 1, policy_version 1186478 (0.0009) [2023-12-27 13:52:37,882][20381] Updated weights for policy 1, policy_version 1186488 (0.0007) [2023-12-27 13:52:38,346][20347] Updated weights for policy 0, policy_version 1187479 (0.0009) [2023-12-27 13:52:38,402][20347] Updated weights for policy 0, policy_version 1187489 (0.0006) [2023-12-27 13:52:38,459][20347] Updated weights for policy 0, policy_version 1187499 (0.0007) [2023-12-27 13:52:38,570][20381] Updated weights for policy 1, policy_version 1186498 (0.0008) [2023-12-27 13:52:38,629][20381] Updated weights for policy 1, policy_version 1186508 (0.0007) [2023-12-27 13:52:38,685][20381] Updated weights for policy 1, policy_version 1186518 (0.0008) [2023-12-27 13:52:39,105][20347] Updated weights for policy 0, policy_version 1187509 (0.0007) [2023-12-27 13:52:39,167][20347] Updated weights for policy 0, policy_version 1187519 (0.0007) [2023-12-27 13:52:39,221][20347] Updated weights for policy 0, policy_version 1187529 (0.0007) [2023-12-27 13:52:39,416][20381] Updated weights for policy 1, policy_version 1186528 (0.0008) [2023-12-27 13:52:39,474][20381] Updated weights for policy 1, policy_version 1186538 (0.0006) [2023-12-27 13:52:39,531][20381] Updated weights for policy 1, policy_version 1186548 (0.0006) [2023-12-27 13:52:39,934][20347] Updated weights for policy 0, policy_version 1187539 (0.0009) [2023-12-27 13:52:39,991][20347] Updated weights for policy 0, policy_version 1187549 (0.0008) [2023-12-27 13:52:40,050][20347] Updated weights for policy 0, policy_version 1187559 (0.0007) [2023-12-27 13:52:40,222][20381] Updated weights for policy 1, policy_version 1186558 (0.0007) [2023-12-27 13:52:40,278][20381] Updated weights for policy 1, policy_version 1186568 (0.0006) [2023-12-27 13:52:40,339][20381] Updated weights for policy 1, policy_version 1186578 (0.0008) [2023-12-27 13:52:40,728][20347] Updated weights for policy 0, policy_version 1187569 (0.0008) [2023-12-27 13:52:40,782][20347] Updated weights for policy 0, policy_version 1187579 (0.0009) [2023-12-27 13:52:40,833][20347] Updated weights for policy 0, policy_version 1187589 (0.0005) [2023-12-27 13:52:40,882][20347] Updated weights for policy 0, policy_version 1187599 (0.0006) [2023-12-27 13:52:41,022][20381] Updated weights for policy 1, policy_version 1186588 (0.0008) [2023-12-27 13:52:41,087][20381] Updated weights for policy 1, policy_version 1186598 (0.0008) [2023-12-27 13:52:41,152][20381] Updated weights for policy 1, policy_version 1186608 (0.0008) [2023-12-27 13:52:41,586][20347] Updated weights for policy 0, policy_version 1187609 (0.0006) [2023-12-27 13:52:41,657][20347] Updated weights for policy 0, policy_version 1187619 (0.0008) [2023-12-27 13:52:41,721][20347] Updated weights for policy 0, policy_version 1187629 (0.0006) [2023-12-27 13:52:41,934][20381] Updated weights for policy 1, policy_version 1186618 (0.0008) [2023-12-27 13:52:42,000][20381] Updated weights for policy 1, policy_version 1186628 (0.0006) [2023-12-27 13:52:42,066][20381] Updated weights for policy 1, policy_version 1186638 (0.0006) [2023-12-27 13:52:42,130][20381] Updated weights for policy 1, policy_version 1186648 (0.0007) [2023-12-27 13:52:42,330][20347] Updated weights for policy 0, policy_version 1187639 (0.0007) [2023-12-27 13:52:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 607895552. Throughput: 0: 10275.9, 1: 10295.3. Samples: 607886756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:42,363][19170] Avg episode reward: [(0, '824.180'), (1, '872.870')] [2023-12-27 13:52:42,395][20347] Updated weights for policy 0, policy_version 1187649 (0.0007) [2023-12-27 13:52:42,445][20347] Updated weights for policy 0, policy_version 1187659 (0.0007) [2023-12-27 13:52:42,818][20381] Updated weights for policy 1, policy_version 1186658 (0.0006) [2023-12-27 13:52:42,873][20381] Updated weights for policy 1, policy_version 1186668 (0.0006) [2023-12-27 13:52:42,928][20381] Updated weights for policy 1, policy_version 1186678 (0.0007) [2023-12-27 13:52:43,190][20347] Updated weights for policy 0, policy_version 1187669 (0.0006) [2023-12-27 13:52:43,248][20347] Updated weights for policy 0, policy_version 1187679 (0.0006) [2023-12-27 13:52:43,307][20347] Updated weights for policy 0, policy_version 1187689 (0.0005) [2023-12-27 13:52:43,555][20381] Updated weights for policy 1, policy_version 1186688 (0.0008) [2023-12-27 13:52:43,581][20292] KL-divergence is very high: 453.7414 [2023-12-27 13:52:43,595][20292] KL-divergence is very high: 459.3970 [2023-12-27 13:52:43,617][20381] Updated weights for policy 1, policy_version 1186698 (0.0007) [2023-12-27 13:52:43,627][20292] KL-divergence is very high: 251.3463 [2023-12-27 13:52:43,639][20292] KL-divergence is very high: 138.7652 [2023-12-27 13:52:43,676][20381] Updated weights for policy 1, policy_version 1186708 (0.0005) [2023-12-27 13:52:43,961][20347] Updated weights for policy 0, policy_version 1187699 (0.0005) [2023-12-27 13:52:44,012][20347] Updated weights for policy 0, policy_version 1187709 (0.0005) [2023-12-27 13:52:44,058][20347] Updated weights for policy 0, policy_version 1187719 (0.0005) [2023-12-27 13:52:44,307][20292] KL-divergence is very high: 437.5402 [2023-12-27 13:52:44,317][20381] Updated weights for policy 1, policy_version 1186718 (0.0009) [2023-12-27 13:52:44,349][20292] KL-divergence is very high: 110.3514 [2023-12-27 13:52:44,371][20381] Updated weights for policy 1, policy_version 1186728 (0.0008) [2023-12-27 13:52:44,394][20292] KL-divergence is very high: 120.6760 [2023-12-27 13:52:44,424][20381] Updated weights for policy 1, policy_version 1186738 (0.0009) [2023-12-27 13:52:44,433][20292] KL-divergence is very high: 119.0704 [2023-12-27 13:52:44,706][20347] Updated weights for policy 0, policy_version 1187729 (0.0005) [2023-12-27 13:52:44,779][20347] Updated weights for policy 0, policy_version 1187739 (0.0008) [2023-12-27 13:52:44,836][20347] Updated weights for policy 0, policy_version 1187749 (0.0008) [2023-12-27 13:52:44,900][20347] Updated weights for policy 0, policy_version 1187759 (0.0007) [2023-12-27 13:52:45,071][20381] Updated weights for policy 1, policy_version 1186748 (0.0010) [2023-12-27 13:52:45,122][20381] Updated weights for policy 1, policy_version 1186758 (0.0011) [2023-12-27 13:52:45,184][20381] Updated weights for policy 1, policy_version 1186768 (0.0011) [2023-12-27 13:52:45,603][20347] Updated weights for policy 0, policy_version 1187769 (0.0007) [2023-12-27 13:52:45,667][20347] Updated weights for policy 0, policy_version 1187779 (0.0007) [2023-12-27 13:52:45,728][20347] Updated weights for policy 0, policy_version 1187789 (0.0006) [2023-12-27 13:52:45,849][20381] Updated weights for policy 1, policy_version 1186778 (0.0006) [2023-12-27 13:52:45,907][20381] Updated weights for policy 1, policy_version 1186788 (0.0005) [2023-12-27 13:52:45,964][20381] Updated weights for policy 1, policy_version 1186798 (0.0007) [2023-12-27 13:52:46,024][20381] Updated weights for policy 1, policy_version 1186808 (0.0008) [2023-12-27 13:52:46,384][20347] Updated weights for policy 0, policy_version 1187799 (0.0006) [2023-12-27 13:52:46,433][20347] Updated weights for policy 0, policy_version 1187809 (0.0008) [2023-12-27 13:52:46,487][20347] Updated weights for policy 0, policy_version 1187819 (0.0009) [2023-12-27 13:52:46,582][20381] Updated weights for policy 1, policy_version 1186818 (0.0007) [2023-12-27 13:52:46,637][20381] Updated weights for policy 1, policy_version 1186828 (0.0007) [2023-12-27 13:52:46,699][20381] Updated weights for policy 1, policy_version 1186838 (0.0008) [2023-12-27 13:52:47,180][20347] Updated weights for policy 0, policy_version 1187829 (0.0008) [2023-12-27 13:52:47,239][20347] Updated weights for policy 0, policy_version 1187839 (0.0007) [2023-12-27 13:52:47,298][20347] Updated weights for policy 0, policy_version 1187849 (0.0008) [2023-12-27 13:52:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 608002048. Throughput: 0: 10258.7, 1: 10346.1. Samples: 608010864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:47,363][19170] Avg episode reward: [(0, '902.370'), (1, '814.980')] [2023-12-27 13:52:47,365][20381] Updated weights for policy 1, policy_version 1186848 (0.0007) [2023-12-27 13:52:47,425][20381] Updated weights for policy 1, policy_version 1186858 (0.0008) [2023-12-27 13:52:47,484][20381] Updated weights for policy 1, policy_version 1186868 (0.0008) [2023-12-27 13:52:47,935][20347] Updated weights for policy 0, policy_version 1187859 (0.0006) [2023-12-27 13:52:47,995][20347] Updated weights for policy 0, policy_version 1187869 (0.0006) [2023-12-27 13:52:48,051][20347] Updated weights for policy 0, policy_version 1187879 (0.0006) [2023-12-27 13:52:48,126][20381] Updated weights for policy 1, policy_version 1186878 (0.0007) [2023-12-27 13:52:48,189][20381] Updated weights for policy 1, policy_version 1186888 (0.0006) [2023-12-27 13:52:48,249][20381] Updated weights for policy 1, policy_version 1186898 (0.0006) [2023-12-27 13:52:48,756][20347] Updated weights for policy 0, policy_version 1187889 (0.0007) [2023-12-27 13:52:48,815][20347] Updated weights for policy 0, policy_version 1187899 (0.0008) [2023-12-27 13:52:48,874][20347] Updated weights for policy 0, policy_version 1187909 (0.0007) [2023-12-27 13:52:48,876][20381] Updated weights for policy 1, policy_version 1186908 (0.0007) [2023-12-27 13:52:48,928][20347] Updated weights for policy 0, policy_version 1187919 (0.0006) [2023-12-27 13:52:48,930][20381] Updated weights for policy 1, policy_version 1186918 (0.0006) [2023-12-27 13:52:48,987][20381] Updated weights for policy 1, policy_version 1186928 (0.0007) [2023-12-27 13:52:49,636][20347] Updated weights for policy 0, policy_version 1187929 (0.0010) [2023-12-27 13:52:49,641][20381] Updated weights for policy 1, policy_version 1186938 (0.0007) [2023-12-27 13:52:49,688][20347] Updated weights for policy 0, policy_version 1187939 (0.0010) [2023-12-27 13:52:49,701][20381] Updated weights for policy 1, policy_version 1186948 (0.0006) [2023-12-27 13:52:49,754][20381] Updated weights for policy 1, policy_version 1186958 (0.0006) [2023-12-27 13:52:49,773][20347] Updated weights for policy 0, policy_version 1187949 (0.0011) [2023-12-27 13:52:49,816][20381] Updated weights for policy 1, policy_version 1186968 (0.0006) [2023-12-27 13:52:50,467][20347] Updated weights for policy 0, policy_version 1187959 (0.0008) [2023-12-27 13:52:50,503][20381] Updated weights for policy 1, policy_version 1186978 (0.0005) [2023-12-27 13:52:50,514][20347] Updated weights for policy 0, policy_version 1187969 (0.0005) [2023-12-27 13:52:50,552][20381] Updated weights for policy 1, policy_version 1186988 (0.0007) [2023-12-27 13:52:50,563][20347] Updated weights for policy 0, policy_version 1187979 (0.0006) [2023-12-27 13:52:50,610][20381] Updated weights for policy 1, policy_version 1186998 (0.0006) [2023-12-27 13:52:51,314][20347] Updated weights for policy 0, policy_version 1187989 (0.0009) [2023-12-27 13:52:51,338][20381] Updated weights for policy 1, policy_version 1187008 (0.0006) [2023-12-27 13:52:51,386][20347] Updated weights for policy 0, policy_version 1188000 (0.0008) [2023-12-27 13:52:51,404][20381] Updated weights for policy 1, policy_version 1187018 (0.0008) [2023-12-27 13:52:51,452][20347] Updated weights for policy 0, policy_version 1188010 (0.0010) [2023-12-27 13:52:51,462][20381] Updated weights for policy 1, policy_version 1187028 (0.0008) [2023-12-27 13:52:52,117][20347] Updated weights for policy 0, policy_version 1188020 (0.0010) [2023-12-27 13:52:52,159][20381] Updated weights for policy 1, policy_version 1187038 (0.0008) [2023-12-27 13:52:52,175][20347] Updated weights for policy 0, policy_version 1188030 (0.0008) [2023-12-27 13:52:52,215][20381] Updated weights for policy 1, policy_version 1187048 (0.0007) [2023-12-27 13:52:52,220][20347] Updated weights for policy 0, policy_version 1188040 (0.0005) [2023-12-27 13:52:52,273][20381] Updated weights for policy 1, policy_version 1187058 (0.0007) [2023-12-27 13:52:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 608108544. Throughput: 0: 10222.7, 1: 10377.5. Samples: 608071876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:52,363][19170] Avg episode reward: [(0, '921.780'), (1, '853.450')] [2023-12-27 13:52:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001187064_303915008.pth... [2023-12-27 13:52:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001188048_304193536.pth... [2023-12-27 13:52:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001185816_303595520.pth [2023-12-27 13:52:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001186832_303882240.pth [2023-12-27 13:52:52,863][20347] Updated weights for policy 0, policy_version 1188050 (0.0008) [2023-12-27 13:52:52,921][20347] Updated weights for policy 0, policy_version 1188060 (0.0009) [2023-12-27 13:52:52,976][20347] Updated weights for policy 0, policy_version 1188070 (0.0010) [2023-12-27 13:52:52,980][20381] Updated weights for policy 1, policy_version 1187068 (0.0008) [2023-12-27 13:52:53,024][20347] Updated weights for policy 0, policy_version 1188080 (0.0010) [2023-12-27 13:52:53,027][20381] Updated weights for policy 1, policy_version 1187078 (0.0009) [2023-12-27 13:52:53,079][20381] Updated weights for policy 1, policy_version 1187088 (0.0010) [2023-12-27 13:52:53,629][20347] Updated weights for policy 0, policy_version 1188090 (0.0010) [2023-12-27 13:52:53,679][20347] Updated weights for policy 0, policy_version 1188100 (0.0006) [2023-12-27 13:52:53,736][20347] Updated weights for policy 0, policy_version 1188110 (0.0005) [2023-12-27 13:52:53,758][20381] Updated weights for policy 1, policy_version 1187098 (0.0009) [2023-12-27 13:52:53,811][20381] Updated weights for policy 1, policy_version 1187108 (0.0007) [2023-12-27 13:52:53,863][20381] Updated weights for policy 1, policy_version 1187118 (0.0006) [2023-12-27 13:52:53,911][20381] Updated weights for policy 1, policy_version 1187128 (0.0007) [2023-12-27 13:52:54,404][20347] Updated weights for policy 0, policy_version 1188120 (0.0005) [2023-12-27 13:52:54,452][20347] Updated weights for policy 0, policy_version 1188130 (0.0005) [2023-12-27 13:52:54,509][20347] Updated weights for policy 0, policy_version 1188140 (0.0006) [2023-12-27 13:52:54,565][20381] Updated weights for policy 1, policy_version 1187138 (0.0006) [2023-12-27 13:52:54,623][20381] Updated weights for policy 1, policy_version 1187148 (0.0006) [2023-12-27 13:52:54,688][20381] Updated weights for policy 1, policy_version 1187158 (0.0005) [2023-12-27 13:52:55,072][20347] Updated weights for policy 0, policy_version 1188150 (0.0008) [2023-12-27 13:52:55,135][20347] Updated weights for policy 0, policy_version 1188160 (0.0008) [2023-12-27 13:52:55,188][20347] Updated weights for policy 0, policy_version 1188170 (0.0007) [2023-12-27 13:52:55,245][20381] Updated weights for policy 1, policy_version 1187168 (0.0007) [2023-12-27 13:52:55,296][20381] Updated weights for policy 1, policy_version 1187178 (0.0009) [2023-12-27 13:52:55,352][20381] Updated weights for policy 1, policy_version 1187188 (0.0008) [2023-12-27 13:52:55,757][20347] Updated weights for policy 0, policy_version 1188180 (0.0006) [2023-12-27 13:52:55,809][20347] Updated weights for policy 0, policy_version 1188190 (0.0007) [2023-12-27 13:52:55,862][20347] Updated weights for policy 0, policy_version 1188200 (0.0005) [2023-12-27 13:52:56,105][20381] Updated weights for policy 1, policy_version 1187198 (0.0008) [2023-12-27 13:52:56,162][20381] Updated weights for policy 1, policy_version 1187208 (0.0008) [2023-12-27 13:52:56,216][20381] Updated weights for policy 1, policy_version 1187218 (0.0008) [2023-12-27 13:52:56,491][20347] Updated weights for policy 0, policy_version 1188210 (0.0006) [2023-12-27 13:52:56,548][20347] Updated weights for policy 0, policy_version 1188220 (0.0010) [2023-12-27 13:52:56,609][20347] Updated weights for policy 0, policy_version 1188230 (0.0008) [2023-12-27 13:52:56,669][20347] Updated weights for policy 0, policy_version 1188240 (0.0005) [2023-12-27 13:52:56,809][20381] Updated weights for policy 1, policy_version 1187228 (0.0007) [2023-12-27 13:52:56,870][20381] Updated weights for policy 1, policy_version 1187238 (0.0005) [2023-12-27 13:52:56,930][20381] Updated weights for policy 1, policy_version 1187248 (0.0005) [2023-12-27 13:52:57,251][20347] Updated weights for policy 0, policy_version 1188250 (0.0008) [2023-12-27 13:52:57,313][20347] Updated weights for policy 0, policy_version 1188260 (0.0011) [2023-12-27 13:52:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 608206848. Throughput: 0: 10372.3, 1: 10358.0. Samples: 608197684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:52:57,363][19170] Avg episode reward: [(0, '941.350'), (1, '843.600')] [2023-12-27 13:52:57,376][20347] Updated weights for policy 0, policy_version 1188270 (0.0006) [2023-12-27 13:52:57,445][20381] Updated weights for policy 1, policy_version 1187258 (0.0006) [2023-12-27 13:52:57,512][20381] Updated weights for policy 1, policy_version 1187268 (0.0006) [2023-12-27 13:52:57,578][20381] Updated weights for policy 1, policy_version 1187278 (0.0005) [2023-12-27 13:52:57,638][20381] Updated weights for policy 1, policy_version 1187288 (0.0006) [2023-12-27 13:52:57,947][20347] Updated weights for policy 0, policy_version 1188280 (0.0008) [2023-12-27 13:52:57,995][20347] Updated weights for policy 0, policy_version 1188290 (0.0005) [2023-12-27 13:52:58,039][20347] Updated weights for policy 0, policy_version 1188300 (0.0005) [2023-12-27 13:52:58,248][20381] Updated weights for policy 1, policy_version 1187298 (0.0009) [2023-12-27 13:52:58,304][20381] Updated weights for policy 1, policy_version 1187308 (0.0008) [2023-12-27 13:52:58,382][20381] Updated weights for policy 1, policy_version 1187318 (0.0024) [2023-12-27 13:52:58,754][20347] Updated weights for policy 0, policy_version 1188310 (0.0007) [2023-12-27 13:52:58,817][20347] Updated weights for policy 0, policy_version 1188320 (0.0008) [2023-12-27 13:52:58,861][20291] KL-divergence is very high: 294.6188 [2023-12-27 13:52:58,888][20347] Updated weights for policy 0, policy_version 1188330 (0.0007) [2023-12-27 13:52:58,894][20291] KL-divergence is very high: 705.3069 [2023-12-27 13:52:58,912][20291] KL-divergence is very high: 747.7323 [2023-12-27 13:52:59,133][20381] Updated weights for policy 1, policy_version 1187328 (0.0010) [2023-12-27 13:52:59,194][20381] Updated weights for policy 1, policy_version 1187338 (0.0008) [2023-12-27 13:52:59,269][20381] Updated weights for policy 1, policy_version 1187348 (0.0009) [2023-12-27 13:52:59,664][20347] Updated weights for policy 0, policy_version 1188340 (0.0008) [2023-12-27 13:52:59,720][20347] Updated weights for policy 0, policy_version 1188350 (0.0007) [2023-12-27 13:52:59,777][20347] Updated weights for policy 0, policy_version 1188360 (0.0007) [2023-12-27 13:52:59,938][20381] Updated weights for policy 1, policy_version 1187358 (0.0008) [2023-12-27 13:53:00,000][20381] Updated weights for policy 1, policy_version 1187368 (0.0008) [2023-12-27 13:53:00,048][20381] Updated weights for policy 1, policy_version 1187378 (0.0008) [2023-12-27 13:53:00,515][20347] Updated weights for policy 0, policy_version 1188370 (0.0007) [2023-12-27 13:53:00,568][20347] Updated weights for policy 0, policy_version 1188380 (0.0006) [2023-12-27 13:53:00,627][20347] Updated weights for policy 0, policy_version 1188390 (0.0006) [2023-12-27 13:53:00,674][20381] Updated weights for policy 1, policy_version 1187388 (0.0007) [2023-12-27 13:53:00,686][20347] Updated weights for policy 0, policy_version 1188400 (0.0005) [2023-12-27 13:53:00,730][20381] Updated weights for policy 1, policy_version 1187398 (0.0005) [2023-12-27 13:53:00,774][20381] Updated weights for policy 1, policy_version 1187408 (0.0005) [2023-12-27 13:53:01,388][20381] Updated weights for policy 1, policy_version 1187418 (0.0005) [2023-12-27 13:53:01,388][20347] Updated weights for policy 0, policy_version 1188410 (0.0010) [2023-12-27 13:53:01,442][20347] Updated weights for policy 0, policy_version 1188420 (0.0006) [2023-12-27 13:53:01,446][20381] Updated weights for policy 1, policy_version 1187428 (0.0007) [2023-12-27 13:53:01,492][20347] Updated weights for policy 0, policy_version 1188430 (0.0007) [2023-12-27 13:53:01,503][20381] Updated weights for policy 1, policy_version 1187438 (0.0006) [2023-12-27 13:53:01,558][20381] Updated weights for policy 1, policy_version 1187448 (0.0006) [2023-12-27 13:53:02,232][20347] Updated weights for policy 0, policy_version 1188440 (0.0008) [2023-12-27 13:53:02,295][20347] Updated weights for policy 0, policy_version 1188450 (0.0008) [2023-12-27 13:53:02,298][20381] Updated weights for policy 1, policy_version 1187458 (0.0007) [2023-12-27 13:53:02,357][20347] Updated weights for policy 0, policy_version 1188460 (0.0008) [2023-12-27 13:53:02,360][20381] Updated weights for policy 1, policy_version 1187468 (0.0008) [2023-12-27 13:53:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 608305152. Throughput: 0: 10321.3, 1: 10431.3. Samples: 608322764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:02,363][19170] Avg episode reward: [(0, '941.320'), (1, '853.430')] [2023-12-27 13:53:02,419][20381] Updated weights for policy 1, policy_version 1187478 (0.0008) [2023-12-27 13:53:02,972][20347] Updated weights for policy 0, policy_version 1188470 (0.0007) [2023-12-27 13:53:03,015][20347] Updated weights for policy 0, policy_version 1188480 (0.0007) [2023-12-27 13:53:03,072][20347] Updated weights for policy 0, policy_version 1188490 (0.0007) [2023-12-27 13:53:03,163][20381] Updated weights for policy 1, policy_version 1187488 (0.0007) [2023-12-27 13:53:03,211][20381] Updated weights for policy 1, policy_version 1187498 (0.0006) [2023-12-27 13:53:03,268][20381] Updated weights for policy 1, policy_version 1187508 (0.0006) [2023-12-27 13:53:03,688][20347] Updated weights for policy 0, policy_version 1188500 (0.0007) [2023-12-27 13:53:03,742][20347] Updated weights for policy 0, policy_version 1188510 (0.0006) [2023-12-27 13:53:03,793][20347] Updated weights for policy 0, policy_version 1188520 (0.0007) [2023-12-27 13:53:03,890][20381] Updated weights for policy 1, policy_version 1187518 (0.0007) [2023-12-27 13:53:03,944][20381] Updated weights for policy 1, policy_version 1187528 (0.0007) [2023-12-27 13:53:03,993][20381] Updated weights for policy 1, policy_version 1187538 (0.0007) [2023-12-27 13:53:04,485][20347] Updated weights for policy 0, policy_version 1188530 (0.0007) [2023-12-27 13:53:04,540][20347] Updated weights for policy 0, policy_version 1188540 (0.0010) [2023-12-27 13:53:04,594][20347] Updated weights for policy 0, policy_version 1188550 (0.0010) [2023-12-27 13:53:04,648][20347] Updated weights for policy 0, policy_version 1188560 (0.0010) [2023-12-27 13:53:04,716][20381] Updated weights for policy 1, policy_version 1187548 (0.0008) [2023-12-27 13:53:04,776][20381] Updated weights for policy 1, policy_version 1187558 (0.0007) [2023-12-27 13:53:04,828][20381] Updated weights for policy 1, policy_version 1187568 (0.0007) [2023-12-27 13:53:05,310][20347] Updated weights for policy 0, policy_version 1188570 (0.0007) [2023-12-27 13:53:05,360][20347] Updated weights for policy 0, policy_version 1188580 (0.0010) [2023-12-27 13:53:05,392][20381] Updated weights for policy 1, policy_version 1187578 (0.0007) [2023-12-27 13:53:05,415][20347] Updated weights for policy 0, policy_version 1188590 (0.0010) [2023-12-27 13:53:05,447][20381] Updated weights for policy 1, policy_version 1187588 (0.0006) [2023-12-27 13:53:05,512][20381] Updated weights for policy 1, policy_version 1187598 (0.0008) [2023-12-27 13:53:05,573][20381] Updated weights for policy 1, policy_version 1187608 (0.0009) [2023-12-27 13:53:06,034][20347] Updated weights for policy 0, policy_version 1188600 (0.0010) [2023-12-27 13:53:06,079][20347] Updated weights for policy 0, policy_version 1188610 (0.0010) [2023-12-27 13:53:06,130][20347] Updated weights for policy 0, policy_version 1188620 (0.0010) [2023-12-27 13:53:06,203][20381] Updated weights for policy 1, policy_version 1187618 (0.0005) [2023-12-27 13:53:06,262][20381] Updated weights for policy 1, policy_version 1187628 (0.0011) [2023-12-27 13:53:06,329][20381] Updated weights for policy 1, policy_version 1187638 (0.0010) [2023-12-27 13:53:06,779][20347] Updated weights for policy 0, policy_version 1188630 (0.0006) [2023-12-27 13:53:06,838][20347] Updated weights for policy 0, policy_version 1188640 (0.0006) [2023-12-27 13:53:06,898][20347] Updated weights for policy 0, policy_version 1188650 (0.0006) [2023-12-27 13:53:06,903][20381] Updated weights for policy 1, policy_version 1187648 (0.0006) [2023-12-27 13:53:06,958][20381] Updated weights for policy 1, policy_version 1187658 (0.0009) [2023-12-27 13:53:07,010][20381] Updated weights for policy 1, policy_version 1187668 (0.0010) [2023-12-27 13:53:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 608419840. Throughput: 0: 10370.5, 1: 10461.7. Samples: 608385552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:07,364][19170] Avg episode reward: [(0, '931.480'), (1, '844.420')] [2023-12-27 13:53:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001187672_304070656.pth... [2023-12-27 13:53:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001188656_304349184.pth... [2023-12-27 13:53:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001186456_303759360.pth [2023-12-27 13:53:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001187440_304037888.pth [2023-12-27 13:53:07,487][20347] Updated weights for policy 0, policy_version 1188660 (0.0008) [2023-12-27 13:53:07,541][20347] Updated weights for policy 0, policy_version 1188670 (0.0008) [2023-12-27 13:53:07,594][20347] Updated weights for policy 0, policy_version 1188680 (0.0008) [2023-12-27 13:53:07,707][20381] Updated weights for policy 1, policy_version 1187678 (0.0010) [2023-12-27 13:53:07,756][20381] Updated weights for policy 1, policy_version 1187688 (0.0010) [2023-12-27 13:53:07,801][20381] Updated weights for policy 1, policy_version 1187698 (0.0010) [2023-12-27 13:53:08,217][20347] Updated weights for policy 0, policy_version 1188690 (0.0005) [2023-12-27 13:53:08,269][20347] Updated weights for policy 0, policy_version 1188700 (0.0007) [2023-12-27 13:53:08,316][20347] Updated weights for policy 0, policy_version 1188710 (0.0007) [2023-12-27 13:53:08,380][20347] Updated weights for policy 0, policy_version 1188720 (0.0008) [2023-12-27 13:53:08,560][20381] Updated weights for policy 1, policy_version 1187708 (0.0010) [2023-12-27 13:53:08,608][20381] Updated weights for policy 1, policy_version 1187718 (0.0010) [2023-12-27 13:53:08,661][20381] Updated weights for policy 1, policy_version 1187728 (0.0010) [2023-12-27 13:53:09,096][20347] Updated weights for policy 0, policy_version 1188730 (0.0008) [2023-12-27 13:53:09,153][20347] Updated weights for policy 0, policy_version 1188740 (0.0006) [2023-12-27 13:53:09,212][20347] Updated weights for policy 0, policy_version 1188750 (0.0006) [2023-12-27 13:53:09,355][20381] Updated weights for policy 1, policy_version 1187738 (0.0010) [2023-12-27 13:53:09,420][20381] Updated weights for policy 1, policy_version 1187748 (0.0009) [2023-12-27 13:53:09,483][20381] Updated weights for policy 1, policy_version 1187758 (0.0008) [2023-12-27 13:53:09,551][20381] Updated weights for policy 1, policy_version 1187768 (0.0008) [2023-12-27 13:53:09,896][20347] Updated weights for policy 0, policy_version 1188760 (0.0007) [2023-12-27 13:53:09,956][20347] Updated weights for policy 0, policy_version 1188770 (0.0007) [2023-12-27 13:53:10,012][20347] Updated weights for policy 0, policy_version 1188780 (0.0006) [2023-12-27 13:53:10,247][20381] Updated weights for policy 1, policy_version 1187778 (0.0007) [2023-12-27 13:53:10,305][20381] Updated weights for policy 1, policy_version 1187788 (0.0006) [2023-12-27 13:53:10,372][20381] Updated weights for policy 1, policy_version 1187798 (0.0011) [2023-12-27 13:53:10,671][20347] Updated weights for policy 0, policy_version 1188790 (0.0006) [2023-12-27 13:53:10,725][20347] Updated weights for policy 0, policy_version 1188800 (0.0007) [2023-12-27 13:53:10,783][20347] Updated weights for policy 0, policy_version 1188810 (0.0006) [2023-12-27 13:53:11,028][20381] Updated weights for policy 1, policy_version 1187808 (0.0010) [2023-12-27 13:53:11,096][20381] Updated weights for policy 1, policy_version 1187818 (0.0010) [2023-12-27 13:53:11,158][20381] Updated weights for policy 1, policy_version 1187828 (0.0011) [2023-12-27 13:53:11,553][20347] Updated weights for policy 0, policy_version 1188820 (0.0008) [2023-12-27 13:53:11,620][20347] Updated weights for policy 0, policy_version 1188830 (0.0008) [2023-12-27 13:53:11,688][20347] Updated weights for policy 0, policy_version 1188840 (0.0009) [2023-12-27 13:53:11,864][20381] Updated weights for policy 1, policy_version 1187838 (0.0009) [2023-12-27 13:53:11,923][20381] Updated weights for policy 1, policy_version 1187848 (0.0010) [2023-12-27 13:53:11,976][20381] Updated weights for policy 1, policy_version 1187858 (0.0011) [2023-12-27 13:53:12,338][20347] Updated weights for policy 0, policy_version 1188850 (0.0007) [2023-12-27 13:53:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 608518144. Throughput: 0: 10479.4, 1: 10385.3. Samples: 608511056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:12,363][19170] Avg episode reward: [(0, '960.810'), (1, '883.310')] [2023-12-27 13:53:12,405][20347] Updated weights for policy 0, policy_version 1188860 (0.0007) [2023-12-27 13:53:12,459][20347] Updated weights for policy 0, policy_version 1188870 (0.0008) [2023-12-27 13:53:12,521][20347] Updated weights for policy 0, policy_version 1188880 (0.0006) [2023-12-27 13:53:12,656][20381] Updated weights for policy 1, policy_version 1187868 (0.0010) [2023-12-27 13:53:12,716][20381] Updated weights for policy 1, policy_version 1187878 (0.0008) [2023-12-27 13:53:12,771][20381] Updated weights for policy 1, policy_version 1187888 (0.0008) [2023-12-27 13:53:13,184][20347] Updated weights for policy 0, policy_version 1188890 (0.0007) [2023-12-27 13:53:13,248][20347] Updated weights for policy 0, policy_version 1188900 (0.0005) [2023-12-27 13:53:13,311][20347] Updated weights for policy 0, policy_version 1188910 (0.0005) [2023-12-27 13:53:13,374][20381] Updated weights for policy 1, policy_version 1187898 (0.0008) [2023-12-27 13:53:13,438][20381] Updated weights for policy 1, policy_version 1187908 (0.0005) [2023-12-27 13:53:13,503][20381] Updated weights for policy 1, policy_version 1187918 (0.0006) [2023-12-27 13:53:13,560][20381] Updated weights for policy 1, policy_version 1187928 (0.0008) [2023-12-27 13:53:13,861][20347] Updated weights for policy 0, policy_version 1188920 (0.0005) [2023-12-27 13:53:13,924][20347] Updated weights for policy 0, policy_version 1188930 (0.0010) [2023-12-27 13:53:13,980][20347] Updated weights for policy 0, policy_version 1188940 (0.0011) [2023-12-27 13:53:14,174][20381] Updated weights for policy 1, policy_version 1187938 (0.0011) [2023-12-27 13:53:14,232][20381] Updated weights for policy 1, policy_version 1187948 (0.0011) [2023-12-27 13:53:14,292][20381] Updated weights for policy 1, policy_version 1187958 (0.0009) [2023-12-27 13:53:14,631][20347] Updated weights for policy 0, policy_version 1188950 (0.0010) [2023-12-27 13:53:14,685][20347] Updated weights for policy 0, policy_version 1188960 (0.0010) [2023-12-27 13:53:14,745][20347] Updated weights for policy 0, policy_version 1188970 (0.0008) [2023-12-27 13:53:14,956][20381] Updated weights for policy 1, policy_version 1187968 (0.0009) [2023-12-27 13:53:15,011][20381] Updated weights for policy 1, policy_version 1187978 (0.0008) [2023-12-27 13:53:15,064][20381] Updated weights for policy 1, policy_version 1187988 (0.0008) [2023-12-27 13:53:15,458][20347] Updated weights for policy 0, policy_version 1188980 (0.0007) [2023-12-27 13:53:15,509][20347] Updated weights for policy 0, policy_version 1188990 (0.0007) [2023-12-27 13:53:15,561][20347] Updated weights for policy 0, policy_version 1189000 (0.0008) [2023-12-27 13:53:15,734][20381] Updated weights for policy 1, policy_version 1187998 (0.0007) [2023-12-27 13:53:15,790][20381] Updated weights for policy 1, policy_version 1188008 (0.0007) [2023-12-27 13:53:15,846][20381] Updated weights for policy 1, policy_version 1188018 (0.0008) [2023-12-27 13:53:16,187][20347] Updated weights for policy 0, policy_version 1189010 (0.0007) [2023-12-27 13:53:16,248][20347] Updated weights for policy 0, policy_version 1189020 (0.0005) [2023-12-27 13:53:16,308][20347] Updated weights for policy 0, policy_version 1189030 (0.0005) [2023-12-27 13:53:16,358][20347] Updated weights for policy 0, policy_version 1189040 (0.0005) [2023-12-27 13:53:16,457][20381] Updated weights for policy 1, policy_version 1188028 (0.0006) [2023-12-27 13:53:16,505][20381] Updated weights for policy 1, policy_version 1188038 (0.0005) [2023-12-27 13:53:16,563][20381] Updated weights for policy 1, policy_version 1188048 (0.0006) [2023-12-27 13:53:16,936][20347] Updated weights for policy 0, policy_version 1189050 (0.0007) [2023-12-27 13:53:16,979][20291] KL-divergence is very high: 143.6579 [2023-12-27 13:53:16,986][20347] Updated weights for policy 0, policy_version 1189060 (0.0009) [2023-12-27 13:53:17,052][20347] Updated weights for policy 0, policy_version 1189070 (0.0006) [2023-12-27 13:53:17,159][20381] Updated weights for policy 1, policy_version 1188058 (0.0009) [2023-12-27 13:53:17,218][20381] Updated weights for policy 1, policy_version 1188068 (0.0008) [2023-12-27 13:53:17,273][20381] Updated weights for policy 1, policy_version 1188078 (0.0007) [2023-12-27 13:53:17,335][20381] Updated weights for policy 1, policy_version 1188088 (0.0007) [2023-12-27 13:53:17,363][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.5, 300 sec: 20605.0). Total num frames: 608632832. Throughput: 0: 10467.9, 1: 10369.6. Samples: 608637632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:17,364][19170] Avg episode reward: [(0, '863.350'), (1, '844.260')] [2023-12-27 13:53:17,751][20347] Updated weights for policy 0, policy_version 1189080 (0.0005) [2023-12-27 13:53:17,814][20347] Updated weights for policy 0, policy_version 1189090 (0.0007) [2023-12-27 13:53:17,869][20347] Updated weights for policy 0, policy_version 1189100 (0.0005) [2023-12-27 13:53:18,028][20381] Updated weights for policy 1, policy_version 1188098 (0.0006) [2023-12-27 13:53:18,080][20381] Updated weights for policy 1, policy_version 1188108 (0.0005) [2023-12-27 13:53:18,130][20381] Updated weights for policy 1, policy_version 1188118 (0.0007) [2023-12-27 13:53:18,485][20347] Updated weights for policy 0, policy_version 1189110 (0.0009) [2023-12-27 13:53:18,543][20347] Updated weights for policy 0, policy_version 1189120 (0.0010) [2023-12-27 13:53:18,600][20347] Updated weights for policy 0, policy_version 1189130 (0.0010) [2023-12-27 13:53:18,823][20381] Updated weights for policy 1, policy_version 1188128 (0.0009) [2023-12-27 13:53:18,881][20381] Updated weights for policy 1, policy_version 1188138 (0.0010) [2023-12-27 13:53:18,940][20381] Updated weights for policy 1, policy_version 1188148 (0.0010) [2023-12-27 13:53:19,284][20347] Updated weights for policy 0, policy_version 1189140 (0.0009) [2023-12-27 13:53:19,335][20347] Updated weights for policy 0, policy_version 1189150 (0.0008) [2023-12-27 13:53:19,397][20347] Updated weights for policy 0, policy_version 1189160 (0.0010) [2023-12-27 13:53:19,576][20381] Updated weights for policy 1, policy_version 1188158 (0.0007) [2023-12-27 13:53:19,636][20381] Updated weights for policy 1, policy_version 1188168 (0.0008) [2023-12-27 13:53:19,692][20381] Updated weights for policy 1, policy_version 1188178 (0.0008) [2023-12-27 13:53:20,153][20347] Updated weights for policy 0, policy_version 1189170 (0.0010) [2023-12-27 13:53:20,216][20347] Updated weights for policy 0, policy_version 1189180 (0.0008) [2023-12-27 13:53:20,278][20347] Updated weights for policy 0, policy_version 1189190 (0.0008) [2023-12-27 13:53:20,341][20347] Updated weights for policy 0, policy_version 1189200 (0.0008) [2023-12-27 13:53:20,441][20381] Updated weights for policy 1, policy_version 1188188 (0.0009) [2023-12-27 13:53:20,499][20381] Updated weights for policy 1, policy_version 1188198 (0.0010) [2023-12-27 13:53:20,557][20381] Updated weights for policy 1, policy_version 1188208 (0.0010) [2023-12-27 13:53:21,069][20347] Updated weights for policy 0, policy_version 1189210 (0.0007) [2023-12-27 13:53:21,069][20291] KL-divergence is very high: 282.7489 [2023-12-27 13:53:21,082][20291] KL-divergence is very high: 270.4037 [2023-12-27 13:53:21,118][20291] KL-divergence is very high: 216.1626 [2023-12-27 13:53:21,131][20347] Updated weights for policy 0, policy_version 1189220 (0.0007) [2023-12-27 13:53:21,132][20291] KL-divergence is very high: 145.8807 [2023-12-27 13:53:21,195][20347] Updated weights for policy 0, policy_version 1189230 (0.0009) [2023-12-27 13:53:21,303][20381] Updated weights for policy 1, policy_version 1188218 (0.0009) [2023-12-27 13:53:21,367][20381] Updated weights for policy 1, policy_version 1188228 (0.0009) [2023-12-27 13:53:21,425][20381] Updated weights for policy 1, policy_version 1188238 (0.0009) [2023-12-27 13:53:21,482][20381] Updated weights for policy 1, policy_version 1188248 (0.0009) [2023-12-27 13:53:21,943][20291] KL-divergence is very high: 229.7542 [2023-12-27 13:53:21,974][20347] Updated weights for policy 0, policy_version 1189240 (0.0008) [2023-12-27 13:53:21,993][20291] KL-divergence is very high: 313.1885 [2023-12-27 13:53:22,036][20347] Updated weights for policy 0, policy_version 1189250 (0.0007) [2023-12-27 13:53:22,040][20291] KL-divergence is very high: 328.8739 [2023-12-27 13:53:22,087][20291] KL-divergence is very high: 185.4569 [2023-12-27 13:53:22,093][20347] Updated weights for policy 0, policy_version 1189260 (0.0008) [2023-12-27 13:53:22,239][20381] Updated weights for policy 1, policy_version 1188258 (0.0007) [2023-12-27 13:53:22,303][20381] Updated weights for policy 1, policy_version 1188268 (0.0007) [2023-12-27 13:53:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 608722944. Throughput: 0: 10431.5, 1: 10380.6. Samples: 608698092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:22,363][19170] Avg episode reward: [(0, '814.270'), (1, '824.840')] [2023-12-27 13:53:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001189264_304504832.pth... [2023-12-27 13:53:22,370][20381] Updated weights for policy 1, policy_version 1188278 (0.0009) [2023-12-27 13:53:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001188048_304193536.pth [2023-12-27 13:53:22,379][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001188280_304226304.pth... [2023-12-27 13:53:22,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001187064_303915008.pth [2023-12-27 13:53:22,812][20347] Updated weights for policy 0, policy_version 1189270 (0.0008) [2023-12-27 13:53:22,872][20347] Updated weights for policy 0, policy_version 1189280 (0.0007) [2023-12-27 13:53:22,936][20347] Updated weights for policy 0, policy_version 1189290 (0.0008) [2023-12-27 13:53:23,051][20381] Updated weights for policy 1, policy_version 1188288 (0.0010) [2023-12-27 13:53:23,100][20381] Updated weights for policy 1, policy_version 1188298 (0.0010) [2023-12-27 13:53:23,156][20381] Updated weights for policy 1, policy_version 1188308 (0.0007) [2023-12-27 13:53:23,559][20347] Updated weights for policy 0, policy_version 1189300 (0.0005) [2023-12-27 13:53:23,614][20347] Updated weights for policy 0, policy_version 1189310 (0.0011) [2023-12-27 13:53:23,660][20347] Updated weights for policy 0, policy_version 1189320 (0.0010) [2023-12-27 13:53:23,792][20381] Updated weights for policy 1, policy_version 1188318 (0.0006) [2023-12-27 13:53:23,850][20381] Updated weights for policy 1, policy_version 1188328 (0.0006) [2023-12-27 13:53:23,901][20381] Updated weights for policy 1, policy_version 1188338 (0.0005) [2023-12-27 13:53:24,283][20347] Updated weights for policy 0, policy_version 1189330 (0.0009) [2023-12-27 13:53:24,352][20347] Updated weights for policy 0, policy_version 1189340 (0.0008) [2023-12-27 13:53:24,415][20347] Updated weights for policy 0, policy_version 1189350 (0.0009) [2023-12-27 13:53:24,467][20347] Updated weights for policy 0, policy_version 1189360 (0.0010) [2023-12-27 13:53:24,529][20381] Updated weights for policy 1, policy_version 1188348 (0.0006) [2023-12-27 13:53:24,581][20381] Updated weights for policy 1, policy_version 1188358 (0.0007) [2023-12-27 13:53:24,634][20381] Updated weights for policy 1, policy_version 1188368 (0.0007) [2023-12-27 13:53:25,133][20347] Updated weights for policy 0, policy_version 1189370 (0.0009) [2023-12-27 13:53:25,189][20347] Updated weights for policy 0, policy_version 1189380 (0.0007) [2023-12-27 13:53:25,251][20347] Updated weights for policy 0, policy_version 1189390 (0.0008) [2023-12-27 13:53:25,334][20381] Updated weights for policy 1, policy_version 1188378 (0.0008) [2023-12-27 13:53:25,384][20381] Updated weights for policy 1, policy_version 1188388 (0.0008) [2023-12-27 13:53:25,439][20381] Updated weights for policy 1, policy_version 1188398 (0.0006) [2023-12-27 13:53:25,485][20381] Updated weights for policy 1, policy_version 1188408 (0.0005) [2023-12-27 13:53:25,892][20347] Updated weights for policy 0, policy_version 1189400 (0.0006) [2023-12-27 13:53:25,953][20347] Updated weights for policy 0, policy_version 1189410 (0.0005) [2023-12-27 13:53:26,012][20347] Updated weights for policy 0, policy_version 1189420 (0.0008) [2023-12-27 13:53:26,049][20381] Updated weights for policy 1, policy_version 1188418 (0.0005) [2023-12-27 13:53:26,098][20381] Updated weights for policy 1, policy_version 1188428 (0.0005) [2023-12-27 13:53:26,147][20381] Updated weights for policy 1, policy_version 1188438 (0.0005) [2023-12-27 13:53:26,581][20347] Updated weights for policy 0, policy_version 1189430 (0.0006) [2023-12-27 13:53:26,634][20347] Updated weights for policy 0, policy_version 1189440 (0.0005) [2023-12-27 13:53:26,689][20347] Updated weights for policy 0, policy_version 1189450 (0.0005) [2023-12-27 13:53:26,728][20381] Updated weights for policy 1, policy_version 1188448 (0.0007) [2023-12-27 13:53:26,783][20381] Updated weights for policy 1, policy_version 1188458 (0.0009) [2023-12-27 13:53:26,829][20381] Updated weights for policy 1, policy_version 1188468 (0.0008) [2023-12-27 13:53:27,279][20347] Updated weights for policy 0, policy_version 1189460 (0.0005) [2023-12-27 13:53:27,334][20347] Updated weights for policy 0, policy_version 1189470 (0.0006) [2023-12-27 13:53:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 608829440. Throughput: 0: 10358.4, 1: 10399.9. Samples: 608820880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:27,363][19170] Avg episode reward: [(0, '843.230'), (1, '843.530')] [2023-12-27 13:53:27,383][20347] Updated weights for policy 0, policy_version 1189480 (0.0005) [2023-12-27 13:53:27,515][20381] Updated weights for policy 1, policy_version 1188478 (0.0007) [2023-12-27 13:53:27,574][20381] Updated weights for policy 1, policy_version 1188488 (0.0008) [2023-12-27 13:53:27,628][20381] Updated weights for policy 1, policy_version 1188498 (0.0007) [2023-12-27 13:53:28,022][20347] Updated weights for policy 0, policy_version 1189490 (0.0007) [2023-12-27 13:53:28,078][20347] Updated weights for policy 0, policy_version 1189500 (0.0008) [2023-12-27 13:53:28,136][20347] Updated weights for policy 0, policy_version 1189510 (0.0008) [2023-12-27 13:53:28,196][20347] Updated weights for policy 0, policy_version 1189520 (0.0007) [2023-12-27 13:53:28,310][20292] KL-divergence is very high: 102.6648 [2023-12-27 13:53:28,320][20381] Updated weights for policy 1, policy_version 1188508 (0.0007) [2023-12-27 13:53:28,357][20292] KL-divergence is very high: 187.8383 [2023-12-27 13:53:28,364][20292] KL-divergence is very high: 192.8377 [2023-12-27 13:53:28,383][20381] Updated weights for policy 1, policy_version 1188518 (0.0009) [2023-12-27 13:53:28,434][20381] Updated weights for policy 1, policy_version 1188528 (0.0007) [2023-12-27 13:53:28,439][20292] KL-divergence is very high: 126.3680 [2023-12-27 13:53:28,443][20292] KL-divergence is very high: 112.5439 [2023-12-27 13:53:28,871][20347] Updated weights for policy 0, policy_version 1189530 (0.0008) [2023-12-27 13:53:28,920][20347] Updated weights for policy 0, policy_version 1189540 (0.0007) [2023-12-27 13:53:28,972][20347] Updated weights for policy 0, policy_version 1189550 (0.0008) [2023-12-27 13:53:29,127][20381] Updated weights for policy 1, policy_version 1188538 (0.0008) [2023-12-27 13:53:29,183][20381] Updated weights for policy 1, policy_version 1188548 (0.0007) [2023-12-27 13:53:29,246][20381] Updated weights for policy 1, policy_version 1188558 (0.0008) [2023-12-27 13:53:29,301][20381] Updated weights for policy 1, policy_version 1188568 (0.0008) [2023-12-27 13:53:29,699][20347] Updated weights for policy 0, policy_version 1189560 (0.0008) [2023-12-27 13:53:29,758][20347] Updated weights for policy 0, policy_version 1189570 (0.0008) [2023-12-27 13:53:29,819][20347] Updated weights for policy 0, policy_version 1189580 (0.0008) [2023-12-27 13:53:30,023][20381] Updated weights for policy 1, policy_version 1188578 (0.0007) [2023-12-27 13:53:30,083][20381] Updated weights for policy 1, policy_version 1188588 (0.0008) [2023-12-27 13:53:30,142][20381] Updated weights for policy 1, policy_version 1188598 (0.0008) [2023-12-27 13:53:30,525][20347] Updated weights for policy 0, policy_version 1189590 (0.0007) [2023-12-27 13:53:30,580][20347] Updated weights for policy 0, policy_version 1189600 (0.0005) [2023-12-27 13:53:30,645][20347] Updated weights for policy 0, policy_version 1189610 (0.0007) [2023-12-27 13:53:30,826][20381] Updated weights for policy 1, policy_version 1188608 (0.0008) [2023-12-27 13:53:30,881][20381] Updated weights for policy 1, policy_version 1188618 (0.0008) [2023-12-27 13:53:30,933][20381] Updated weights for policy 1, policy_version 1188628 (0.0008) [2023-12-27 13:53:31,318][20347] Updated weights for policy 0, policy_version 1189620 (0.0008) [2023-12-27 13:53:31,382][20347] Updated weights for policy 0, policy_version 1189630 (0.0008) [2023-12-27 13:53:31,443][20347] Updated weights for policy 0, policy_version 1189640 (0.0008) [2023-12-27 13:53:31,669][20381] Updated weights for policy 1, policy_version 1188638 (0.0008) [2023-12-27 13:53:31,730][20381] Updated weights for policy 1, policy_version 1188648 (0.0008) [2023-12-27 13:53:31,793][20381] Updated weights for policy 1, policy_version 1188658 (0.0007) [2023-12-27 13:53:32,136][20347] Updated weights for policy 0, policy_version 1189650 (0.0008) [2023-12-27 13:53:32,195][20347] Updated weights for policy 0, policy_version 1189660 (0.0008) [2023-12-27 13:53:32,256][20347] Updated weights for policy 0, policy_version 1189670 (0.0008) [2023-12-27 13:53:32,312][20347] Updated weights for policy 0, policy_version 1189680 (0.0008) [2023-12-27 13:53:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 608935936. Throughput: 0: 10394.1, 1: 10350.8. Samples: 608944388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:32,363][19170] Avg episode reward: [(0, '882.320'), (1, '795.300')] [2023-12-27 13:53:32,442][20381] Updated weights for policy 1, policy_version 1188668 (0.0006) [2023-12-27 13:53:32,504][20381] Updated weights for policy 1, policy_version 1188678 (0.0009) [2023-12-27 13:53:32,563][20381] Updated weights for policy 1, policy_version 1188688 (0.0008) [2023-12-27 13:53:32,982][20347] Updated weights for policy 0, policy_version 1189690 (0.0005) [2023-12-27 13:53:33,048][20347] Updated weights for policy 0, policy_version 1189700 (0.0009) [2023-12-27 13:53:33,102][20347] Updated weights for policy 0, policy_version 1189710 (0.0010) [2023-12-27 13:53:33,229][20381] Updated weights for policy 1, policy_version 1188698 (0.0008) [2023-12-27 13:53:33,286][20381] Updated weights for policy 1, policy_version 1188708 (0.0010) [2023-12-27 13:53:33,344][20381] Updated weights for policy 1, policy_version 1188718 (0.0006) [2023-12-27 13:53:33,396][20381] Updated weights for policy 1, policy_version 1188728 (0.0007) [2023-12-27 13:53:33,742][20347] Updated weights for policy 0, policy_version 1189720 (0.0010) [2023-12-27 13:53:33,793][20347] Updated weights for policy 0, policy_version 1189730 (0.0009) [2023-12-27 13:53:33,850][20347] Updated weights for policy 0, policy_version 1189740 (0.0006) [2023-12-27 13:53:34,029][20381] Updated weights for policy 1, policy_version 1188738 (0.0010) [2023-12-27 13:53:34,076][20381] Updated weights for policy 1, policy_version 1188748 (0.0010) [2023-12-27 13:53:34,121][20381] Updated weights for policy 1, policy_version 1188758 (0.0010) [2023-12-27 13:53:34,486][20347] Updated weights for policy 0, policy_version 1189750 (0.0008) [2023-12-27 13:53:34,532][20347] Updated weights for policy 0, policy_version 1189760 (0.0010) [2023-12-27 13:53:34,588][20347] Updated weights for policy 0, policy_version 1189770 (0.0010) [2023-12-27 13:53:34,772][20381] Updated weights for policy 1, policy_version 1188768 (0.0010) [2023-12-27 13:53:34,816][20381] Updated weights for policy 1, policy_version 1188778 (0.0010) [2023-12-27 13:53:34,863][20381] Updated weights for policy 1, policy_version 1188788 (0.0009) [2023-12-27 13:53:35,285][20347] Updated weights for policy 0, policy_version 1189780 (0.0010) [2023-12-27 13:53:35,339][20347] Updated weights for policy 0, policy_version 1189790 (0.0010) [2023-12-27 13:53:35,396][20347] Updated weights for policy 0, policy_version 1189800 (0.0010) [2023-12-27 13:53:35,548][20381] Updated weights for policy 1, policy_version 1188798 (0.0007) [2023-12-27 13:53:35,590][20381] Updated weights for policy 1, policy_version 1188808 (0.0005) [2023-12-27 13:53:35,650][20381] Updated weights for policy 1, policy_version 1188818 (0.0007) [2023-12-27 13:53:36,109][20347] Updated weights for policy 0, policy_version 1189810 (0.0006) [2023-12-27 13:53:36,168][20347] Updated weights for policy 0, policy_version 1189820 (0.0007) [2023-12-27 13:53:36,226][20347] Updated weights for policy 0, policy_version 1189830 (0.0006) [2023-12-27 13:53:36,289][20347] Updated weights for policy 0, policy_version 1189840 (0.0007) [2023-12-27 13:53:36,334][20381] Updated weights for policy 1, policy_version 1188828 (0.0009) [2023-12-27 13:53:36,400][20381] Updated weights for policy 1, policy_version 1188838 (0.0006) [2023-12-27 13:53:36,466][20381] Updated weights for policy 1, policy_version 1188848 (0.0008) [2023-12-27 13:53:36,916][20347] Updated weights for policy 0, policy_version 1189850 (0.0010) [2023-12-27 13:53:36,977][20347] Updated weights for policy 0, policy_version 1189860 (0.0010) [2023-12-27 13:53:37,035][20347] Updated weights for policy 0, policy_version 1189870 (0.0010) [2023-12-27 13:53:37,086][20381] Updated weights for policy 1, policy_version 1188858 (0.0007) [2023-12-27 13:53:37,147][20381] Updated weights for policy 1, policy_version 1188868 (0.0011) [2023-12-27 13:53:37,197][20381] Updated weights for policy 1, policy_version 1188878 (0.0010) [2023-12-27 13:53:37,242][20381] Updated weights for policy 1, policy_version 1188888 (0.0010) [2023-12-27 13:53:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 609042432. Throughput: 0: 10418.8, 1: 10354.5. Samples: 609006672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 13:53:37,363][19170] Avg episode reward: [(0, '911.790'), (1, '787.550')] [2023-12-27 13:53:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001188888_304381952.pth... [2023-12-27 13:53:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001189872_304660480.pth... [2023-12-27 13:53:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001187672_304070656.pth [2023-12-27 13:53:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001188656_304349184.pth [2023-12-27 13:53:37,675][20347] Updated weights for policy 0, policy_version 1189880 (0.0007) [2023-12-27 13:53:37,737][20347] Updated weights for policy 0, policy_version 1189890 (0.0006) [2023-12-27 13:53:37,800][20347] Updated weights for policy 0, policy_version 1189900 (0.0006) [2023-12-27 13:53:37,928][20381] Updated weights for policy 1, policy_version 1188898 (0.0005) [2023-12-27 13:53:37,983][20381] Updated weights for policy 1, policy_version 1188908 (0.0005) [2023-12-27 13:53:38,043][20381] Updated weights for policy 1, policy_version 1188918 (0.0005) [2023-12-27 13:53:38,494][20347] Updated weights for policy 0, policy_version 1189910 (0.0007) [2023-12-27 13:53:38,556][20347] Updated weights for policy 0, policy_version 1189920 (0.0006) [2023-12-27 13:53:38,600][20381] Updated weights for policy 1, policy_version 1188928 (0.0005) [2023-12-27 13:53:38,618][20347] Updated weights for policy 0, policy_version 1189930 (0.0007) [2023-12-27 13:53:38,661][20381] Updated weights for policy 1, policy_version 1188938 (0.0010) [2023-12-27 13:53:38,718][20381] Updated weights for policy 1, policy_version 1188948 (0.0011) [2023-12-27 13:53:39,258][20347] Updated weights for policy 0, policy_version 1189940 (0.0007) [2023-12-27 13:53:39,316][20347] Updated weights for policy 0, policy_version 1189950 (0.0006) [2023-12-27 13:53:39,361][20381] Updated weights for policy 1, policy_version 1188958 (0.0009) [2023-12-27 13:53:39,392][20347] Updated weights for policy 0, policy_version 1189960 (0.0008) [2023-12-27 13:53:39,424][20381] Updated weights for policy 1, policy_version 1188968 (0.0010) [2023-12-27 13:53:39,484][20381] Updated weights for policy 1, policy_version 1188978 (0.0009) [2023-12-27 13:53:40,069][20347] Updated weights for policy 0, policy_version 1189970 (0.0007) [2023-12-27 13:53:40,129][20347] Updated weights for policy 0, policy_version 1189980 (0.0007) [2023-12-27 13:53:40,188][20347] Updated weights for policy 0, policy_version 1189990 (0.0007) [2023-12-27 13:53:40,238][20381] Updated weights for policy 1, policy_version 1188988 (0.0009) [2023-12-27 13:53:40,253][20347] Updated weights for policy 0, policy_version 1190000 (0.0009) [2023-12-27 13:53:40,298][20381] Updated weights for policy 1, policy_version 1188998 (0.0011) [2023-12-27 13:53:40,363][20381] Updated weights for policy 1, policy_version 1189008 (0.0011) [2023-12-27 13:53:40,915][20347] Updated weights for policy 0, policy_version 1190010 (0.0007) [2023-12-27 13:53:40,967][20347] Updated weights for policy 0, policy_version 1190020 (0.0007) [2023-12-27 13:53:41,018][20347] Updated weights for policy 0, policy_version 1190030 (0.0007) [2023-12-27 13:53:41,031][20381] Updated weights for policy 1, policy_version 1189018 (0.0011) [2023-12-27 13:53:41,086][20381] Updated weights for policy 1, policy_version 1189028 (0.0011) [2023-12-27 13:53:41,145][20381] Updated weights for policy 1, policy_version 1189038 (0.0011) [2023-12-27 13:53:41,201][20381] Updated weights for policy 1, policy_version 1189048 (0.0008) [2023-12-27 13:53:41,836][20347] Updated weights for policy 0, policy_version 1190040 (0.0008) [2023-12-27 13:53:41,889][20347] Updated weights for policy 0, policy_version 1190050 (0.0008) [2023-12-27 13:53:41,942][20347] Updated weights for policy 0, policy_version 1190060 (0.0008) [2023-12-27 13:53:41,962][20381] Updated weights for policy 1, policy_version 1189058 (0.0006) [2023-12-27 13:53:42,021][20381] Updated weights for policy 1, policy_version 1189068 (0.0007) [2023-12-27 13:53:42,075][20381] Updated weights for policy 1, policy_version 1189078 (0.0006) [2023-12-27 13:53:42,374][19170] Fps is (10 sec: 20456.6, 60 sec: 20749.1, 300 sec: 20548.6). Total num frames: 609140736. Throughput: 0: 10341.4, 1: 10379.9. Samples: 609130376. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:53:42,375][19170] Avg episode reward: [(0, '882.460'), (1, '827.470')] [2023-12-27 13:53:42,659][20347] Updated weights for policy 0, policy_version 1190070 (0.0006) [2023-12-27 13:53:42,712][20347] Updated weights for policy 0, policy_version 1190080 (0.0006) [2023-12-27 13:53:42,765][20347] Updated weights for policy 0, policy_version 1190090 (0.0008) [2023-12-27 13:53:42,802][20381] Updated weights for policy 1, policy_version 1189088 (0.0007) [2023-12-27 13:53:42,863][20381] Updated weights for policy 1, policy_version 1189098 (0.0009) [2023-12-27 13:53:42,916][20381] Updated weights for policy 1, policy_version 1189108 (0.0008) [2023-12-27 13:53:43,370][20347] Updated weights for policy 0, policy_version 1190100 (0.0007) [2023-12-27 13:53:43,430][20347] Updated weights for policy 0, policy_version 1190110 (0.0006) [2023-12-27 13:53:43,495][20347] Updated weights for policy 0, policy_version 1190120 (0.0005) [2023-12-27 13:53:43,507][20381] Updated weights for policy 1, policy_version 1189118 (0.0007) [2023-12-27 13:53:43,561][20381] Updated weights for policy 1, policy_version 1189128 (0.0008) [2023-12-27 13:53:43,609][20381] Updated weights for policy 1, policy_version 1189138 (0.0009) [2023-12-27 13:53:44,015][20347] Updated weights for policy 0, policy_version 1190130 (0.0006) [2023-12-27 13:53:44,082][20347] Updated weights for policy 0, policy_version 1190140 (0.0006) [2023-12-27 13:53:44,138][20347] Updated weights for policy 0, policy_version 1190150 (0.0006) [2023-12-27 13:53:44,194][20347] Updated weights for policy 0, policy_version 1190160 (0.0008) [2023-12-27 13:53:44,271][20381] Updated weights for policy 1, policy_version 1189148 (0.0010) [2023-12-27 13:53:44,335][20381] Updated weights for policy 1, policy_version 1189158 (0.0009) [2023-12-27 13:53:44,396][20381] Updated weights for policy 1, policy_version 1189168 (0.0010) [2023-12-27 13:53:44,785][20347] Updated weights for policy 0, policy_version 1190170 (0.0008) [2023-12-27 13:53:44,842][20347] Updated weights for policy 0, policy_version 1190180 (0.0007) [2023-12-27 13:53:44,900][20347] Updated weights for policy 0, policy_version 1190190 (0.0008) [2023-12-27 13:53:45,122][20381] Updated weights for policy 1, policy_version 1189178 (0.0010) [2023-12-27 13:53:45,159][20292] KL-divergence is very high: 168.8726 [2023-12-27 13:53:45,185][20381] Updated weights for policy 1, policy_version 1189188 (0.0008) [2023-12-27 13:53:45,244][20381] Updated weights for policy 1, policy_version 1189198 (0.0008) [2023-12-27 13:53:45,307][20381] Updated weights for policy 1, policy_version 1189208 (0.0008) [2023-12-27 13:53:45,560][20347] Updated weights for policy 0, policy_version 1190200 (0.0005) [2023-12-27 13:53:45,591][20291] KL-divergence is very high: 109.7102 [2023-12-27 13:53:45,616][20347] Updated weights for policy 0, policy_version 1190210 (0.0009) [2023-12-27 13:53:45,680][20347] Updated weights for policy 0, policy_version 1190220 (0.0008) [2023-12-27 13:53:45,952][20381] Updated weights for policy 1, policy_version 1189218 (0.0008) [2023-12-27 13:53:46,004][20381] Updated weights for policy 1, policy_version 1189228 (0.0007) [2023-12-27 13:53:46,059][20381] Updated weights for policy 1, policy_version 1189238 (0.0008) [2023-12-27 13:53:46,331][20347] Updated weights for policy 0, policy_version 1190230 (0.0008) [2023-12-27 13:53:46,380][20347] Updated weights for policy 0, policy_version 1190240 (0.0005) [2023-12-27 13:53:46,426][20347] Updated weights for policy 0, policy_version 1190250 (0.0005) [2023-12-27 13:53:46,766][20381] Updated weights for policy 1, policy_version 1189248 (0.0008) [2023-12-27 13:53:46,825][20381] Updated weights for policy 1, policy_version 1189258 (0.0009) [2023-12-27 13:53:46,888][20381] Updated weights for policy 1, policy_version 1189268 (0.0007) [2023-12-27 13:53:47,080][20347] Updated weights for policy 0, policy_version 1190260 (0.0007) [2023-12-27 13:53:47,122][20347] Updated weights for policy 0, policy_version 1190270 (0.0008) [2023-12-27 13:53:47,170][20347] Updated weights for policy 0, policy_version 1190280 (0.0007) [2023-12-27 13:53:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 609247232. Throughput: 0: 10409.3, 1: 10308.8. Samples: 609255080. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:53:47,364][19170] Avg episode reward: [(0, '705.920'), (1, '796.250')] [2023-12-27 13:53:47,614][20381] Updated weights for policy 1, policy_version 1189278 (0.0008) [2023-12-27 13:53:47,671][20381] Updated weights for policy 1, policy_version 1189288 (0.0008) [2023-12-27 13:53:47,724][20381] Updated weights for policy 1, policy_version 1189298 (0.0008) [2023-12-27 13:53:47,858][20347] Updated weights for policy 0, policy_version 1190290 (0.0006) [2023-12-27 13:53:47,918][20347] Updated weights for policy 0, policy_version 1190300 (0.0007) [2023-12-27 13:53:47,975][20347] Updated weights for policy 0, policy_version 1190310 (0.0007) [2023-12-27 13:53:48,023][20347] Updated weights for policy 0, policy_version 1190320 (0.0008) [2023-12-27 13:53:48,464][20381] Updated weights for policy 1, policy_version 1189308 (0.0007) [2023-12-27 13:53:48,523][20381] Updated weights for policy 1, policy_version 1189318 (0.0007) [2023-12-27 13:53:48,579][20381] Updated weights for policy 1, policy_version 1189328 (0.0008) [2023-12-27 13:53:48,725][20347] Updated weights for policy 0, policy_version 1190330 (0.0008) [2023-12-27 13:53:48,777][20347] Updated weights for policy 0, policy_version 1190340 (0.0007) [2023-12-27 13:53:48,837][20347] Updated weights for policy 0, policy_version 1190350 (0.0006) [2023-12-27 13:53:49,292][20381] Updated weights for policy 1, policy_version 1189338 (0.0008) [2023-12-27 13:53:49,352][20381] Updated weights for policy 1, policy_version 1189348 (0.0009) [2023-12-27 13:53:49,421][20381] Updated weights for policy 1, policy_version 1189358 (0.0011) [2023-12-27 13:53:49,467][20347] Updated weights for policy 0, policy_version 1190360 (0.0008) [2023-12-27 13:53:49,483][20381] Updated weights for policy 1, policy_version 1189368 (0.0007) [2023-12-27 13:53:49,524][20347] Updated weights for policy 0, policy_version 1190370 (0.0006) [2023-12-27 13:53:49,589][20347] Updated weights for policy 0, policy_version 1190380 (0.0005) [2023-12-27 13:53:50,229][20347] Updated weights for policy 0, policy_version 1190390 (0.0006) [2023-12-27 13:53:50,229][20381] Updated weights for policy 1, policy_version 1189378 (0.0008) [2023-12-27 13:53:50,284][20381] Updated weights for policy 1, policy_version 1189388 (0.0007) [2023-12-27 13:53:50,291][20347] Updated weights for policy 0, policy_version 1190400 (0.0005) [2023-12-27 13:53:50,340][20381] Updated weights for policy 1, policy_version 1189398 (0.0008) [2023-12-27 13:53:50,355][20347] Updated weights for policy 0, policy_version 1190410 (0.0005) [2023-12-27 13:53:50,982][20347] Updated weights for policy 0, policy_version 1190420 (0.0007) [2023-12-27 13:53:51,017][20381] Updated weights for policy 1, policy_version 1189408 (0.0006) [2023-12-27 13:53:51,041][20347] Updated weights for policy 0, policy_version 1190430 (0.0006) [2023-12-27 13:53:51,080][20381] Updated weights for policy 1, policy_version 1189418 (0.0008) [2023-12-27 13:53:51,098][20347] Updated weights for policy 0, policy_version 1190440 (0.0007) [2023-12-27 13:53:51,139][20381] Updated weights for policy 1, policy_version 1189428 (0.0008) [2023-12-27 13:53:51,823][20347] Updated weights for policy 0, policy_version 1190450 (0.0009) [2023-12-27 13:53:51,880][20347] Updated weights for policy 0, policy_version 1190460 (0.0008) [2023-12-27 13:53:51,882][20381] Updated weights for policy 1, policy_version 1189438 (0.0008) [2023-12-27 13:53:51,938][20347] Updated weights for policy 0, policy_version 1190470 (0.0008) [2023-12-27 13:53:51,943][20381] Updated weights for policy 1, policy_version 1189448 (0.0007) [2023-12-27 13:53:51,996][20347] Updated weights for policy 0, policy_version 1190480 (0.0008) [2023-12-27 13:53:52,002][20381] Updated weights for policy 1, policy_version 1189458 (0.0007) [2023-12-27 13:53:52,362][19170] Fps is (10 sec: 20503.3, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 609345536. Throughput: 0: 10395.4, 1: 10253.1. Samples: 609314732. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:53:52,363][19170] Avg episode reward: [(0, '696.340'), (1, '815.950')] [2023-12-27 13:53:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001189464_304529408.pth... [2023-12-27 13:53:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001190480_304816128.pth... [2023-12-27 13:53:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001188280_304226304.pth [2023-12-27 13:53:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001189264_304504832.pth [2023-12-27 13:53:52,675][20347] Updated weights for policy 0, policy_version 1190490 (0.0011) [2023-12-27 13:53:52,719][20381] Updated weights for policy 1, policy_version 1189468 (0.0007) [2023-12-27 13:53:52,733][20347] Updated weights for policy 0, policy_version 1190500 (0.0010) [2023-12-27 13:53:52,775][20381] Updated weights for policy 1, policy_version 1189478 (0.0005) [2023-12-27 13:53:52,793][20347] Updated weights for policy 0, policy_version 1190510 (0.0010) [2023-12-27 13:53:52,822][20381] Updated weights for policy 1, policy_version 1189488 (0.0006) [2023-12-27 13:53:53,369][20347] Updated weights for policy 0, policy_version 1190520 (0.0008) [2023-12-27 13:53:53,426][20347] Updated weights for policy 0, policy_version 1190530 (0.0007) [2023-12-27 13:53:53,484][20347] Updated weights for policy 0, policy_version 1190540 (0.0007) [2023-12-27 13:53:53,529][20381] Updated weights for policy 1, policy_version 1189498 (0.0007) [2023-12-27 13:53:53,580][20381] Updated weights for policy 1, policy_version 1189508 (0.0007) [2023-12-27 13:53:53,641][20381] Updated weights for policy 1, policy_version 1189518 (0.0008) [2023-12-27 13:53:53,700][20381] Updated weights for policy 1, policy_version 1189528 (0.0007) [2023-12-27 13:53:54,112][20347] Updated weights for policy 0, policy_version 1190550 (0.0007) [2023-12-27 13:53:54,160][20347] Updated weights for policy 0, policy_version 1190560 (0.0006) [2023-12-27 13:53:54,212][20347] Updated weights for policy 0, policy_version 1190570 (0.0008) [2023-12-27 13:53:54,402][20381] Updated weights for policy 1, policy_version 1189538 (0.0007) [2023-12-27 13:53:54,460][20381] Updated weights for policy 1, policy_version 1189548 (0.0007) [2023-12-27 13:53:54,511][20381] Updated weights for policy 1, policy_version 1189558 (0.0006) [2023-12-27 13:53:54,845][20347] Updated weights for policy 0, policy_version 1190580 (0.0007) [2023-12-27 13:53:54,901][20347] Updated weights for policy 0, policy_version 1190590 (0.0005) [2023-12-27 13:53:54,955][20347] Updated weights for policy 0, policy_version 1190600 (0.0007) [2023-12-27 13:53:55,075][20381] Updated weights for policy 1, policy_version 1189568 (0.0008) [2023-12-27 13:53:55,122][20381] Updated weights for policy 1, policy_version 1189578 (0.0009) [2023-12-27 13:53:55,176][20381] Updated weights for policy 1, policy_version 1189588 (0.0008) [2023-12-27 13:53:55,593][20347] Updated weights for policy 0, policy_version 1190610 (0.0008) [2023-12-27 13:53:55,649][20347] Updated weights for policy 0, policy_version 1190620 (0.0007) [2023-12-27 13:53:55,700][20347] Updated weights for policy 0, policy_version 1190630 (0.0008) [2023-12-27 13:53:55,891][20381] Updated weights for policy 1, policy_version 1189598 (0.0010) [2023-12-27 13:53:55,947][20381] Updated weights for policy 1, policy_version 1189608 (0.0010) [2023-12-27 13:53:56,002][20381] Updated weights for policy 1, policy_version 1189618 (0.0010) [2023-12-27 13:53:56,333][20347] Updated weights for policy 0, policy_version 1190641 (0.0006) [2023-12-27 13:53:56,396][20347] Updated weights for policy 0, policy_version 1190651 (0.0010) [2023-12-27 13:53:56,452][20347] Updated weights for policy 0, policy_version 1190661 (0.0006) [2023-12-27 13:53:56,505][20347] Updated weights for policy 0, policy_version 1190671 (0.0005) [2023-12-27 13:53:56,640][20381] Updated weights for policy 1, policy_version 1189628 (0.0010) [2023-12-27 13:53:56,703][20381] Updated weights for policy 1, policy_version 1189638 (0.0010) [2023-12-27 13:53:56,766][20381] Updated weights for policy 1, policy_version 1189648 (0.0010) [2023-12-27 13:53:57,098][20347] Updated weights for policy 0, policy_version 1190681 (0.0008) [2023-12-27 13:53:57,149][20347] Updated weights for policy 0, policy_version 1190691 (0.0007) [2023-12-27 13:53:57,208][20347] Updated weights for policy 0, policy_version 1190701 (0.0008) [2023-12-27 13:53:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 609452032. Throughput: 0: 10403.9, 1: 10234.2. Samples: 609439768. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:53:57,363][19170] Avg episode reward: [(0, '784.590'), (1, '813.730')] [2023-12-27 13:53:57,382][20381] Updated weights for policy 1, policy_version 1189658 (0.0010) [2023-12-27 13:53:57,435][20381] Updated weights for policy 1, policy_version 1189668 (0.0010) [2023-12-27 13:53:57,490][20381] Updated weights for policy 1, policy_version 1189678 (0.0010) [2023-12-27 13:53:57,542][20381] Updated weights for policy 1, policy_version 1189688 (0.0007) [2023-12-27 13:53:57,917][20347] Updated weights for policy 0, policy_version 1190711 (0.0006) [2023-12-27 13:53:57,973][20347] Updated weights for policy 0, policy_version 1190721 (0.0005) [2023-12-27 13:53:57,999][20291] KL-divergence is very high: 171.1598 [2023-12-27 13:53:58,010][20291] KL-divergence is very high: 210.5706 [2023-12-27 13:53:58,015][20291] KL-divergence is very high: 209.4626 [2023-12-27 13:53:58,020][20291] KL-divergence is very high: 166.9479 [2023-12-27 13:53:58,031][20347] Updated weights for policy 0, policy_version 1190731 (0.0006) [2023-12-27 13:53:58,081][20381] Updated weights for policy 1, policy_version 1189698 (0.0007) [2023-12-27 13:53:58,136][20381] Updated weights for policy 1, policy_version 1189708 (0.0010) [2023-12-27 13:53:58,193][20381] Updated weights for policy 1, policy_version 1189718 (0.0010) [2023-12-27 13:53:58,739][20347] Updated weights for policy 0, policy_version 1190741 (0.0008) [2023-12-27 13:53:58,768][20291] KL-divergence is very high: 173.7423 [2023-12-27 13:53:58,804][20347] Updated weights for policy 0, policy_version 1190751 (0.0007) [2023-12-27 13:53:58,827][20291] KL-divergence is very high: 160.4005 [2023-12-27 13:53:58,883][20347] Updated weights for policy 0, policy_version 1190761 (0.0009) [2023-12-27 13:53:58,884][20291] KL-divergence is very high: 104.9201 [2023-12-27 13:53:59,041][20381] Updated weights for policy 1, policy_version 1189728 (0.0009) [2023-12-27 13:53:59,101][20381] Updated weights for policy 1, policy_version 1189738 (0.0008) [2023-12-27 13:53:59,166][20381] Updated weights for policy 1, policy_version 1189748 (0.0008) [2023-12-27 13:53:59,636][20347] Updated weights for policy 0, policy_version 1190771 (0.0008) [2023-12-27 13:53:59,685][20347] Updated weights for policy 0, policy_version 1190781 (0.0008) [2023-12-27 13:53:59,732][20347] Updated weights for policy 0, policy_version 1190791 (0.0008) [2023-12-27 13:53:59,807][20381] Updated weights for policy 1, policy_version 1189758 (0.0006) [2023-12-27 13:53:59,870][20381] Updated weights for policy 1, policy_version 1189768 (0.0008) [2023-12-27 13:53:59,922][20381] Updated weights for policy 1, policy_version 1189778 (0.0008) [2023-12-27 13:54:00,492][20347] Updated weights for policy 0, policy_version 1190801 (0.0007) [2023-12-27 13:54:00,543][20347] Updated weights for policy 0, policy_version 1190811 (0.0009) [2023-12-27 13:54:00,588][20347] Updated weights for policy 0, policy_version 1190821 (0.0008) [2023-12-27 13:54:00,629][20381] Updated weights for policy 1, policy_version 1189788 (0.0007) [2023-12-27 13:54:00,635][20347] Updated weights for policy 0, policy_version 1190831 (0.0005) [2023-12-27 13:54:00,684][20381] Updated weights for policy 1, policy_version 1189798 (0.0005) [2023-12-27 13:54:00,738][20381] Updated weights for policy 1, policy_version 1189808 (0.0007) [2023-12-27 13:54:01,333][20347] Updated weights for policy 0, policy_version 1190841 (0.0006) [2023-12-27 13:54:01,398][20347] Updated weights for policy 0, policy_version 1190851 (0.0010) [2023-12-27 13:54:01,399][20381] Updated weights for policy 1, policy_version 1189818 (0.0009) [2023-12-27 13:54:01,445][20381] Updated weights for policy 1, policy_version 1189828 (0.0006) [2023-12-27 13:54:01,447][20347] Updated weights for policy 0, policy_version 1190861 (0.0010) [2023-12-27 13:54:01,495][20381] Updated weights for policy 1, policy_version 1189838 (0.0007) [2023-12-27 13:54:01,546][20381] Updated weights for policy 1, policy_version 1189848 (0.0008) [2023-12-27 13:54:02,132][20347] Updated weights for policy 0, policy_version 1190871 (0.0010) [2023-12-27 13:54:02,191][20347] Updated weights for policy 0, policy_version 1190881 (0.0010) [2023-12-27 13:54:02,225][20381] Updated weights for policy 1, policy_version 1189858 (0.0010) [2023-12-27 13:54:02,240][20347] Updated weights for policy 0, policy_version 1190891 (0.0010) [2023-12-27 13:54:02,283][20381] Updated weights for policy 1, policy_version 1189868 (0.0009) [2023-12-27 13:54:02,339][20381] Updated weights for policy 1, policy_version 1189878 (0.0008) [2023-12-27 13:54:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 609558528. Throughput: 0: 10337.3, 1: 10211.0. Samples: 609562300. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:02,363][19170] Avg episode reward: [(0, '765.040'), (1, '758.490')] [2023-12-27 13:54:02,890][20347] Updated weights for policy 0, policy_version 1190901 (0.0007) [2023-12-27 13:54:02,948][20347] Updated weights for policy 0, policy_version 1190911 (0.0007) [2023-12-27 13:54:02,998][20347] Updated weights for policy 0, policy_version 1190921 (0.0007) [2023-12-27 13:54:03,070][20381] Updated weights for policy 1, policy_version 1189888 (0.0007) [2023-12-27 13:54:03,132][20381] Updated weights for policy 1, policy_version 1189898 (0.0009) [2023-12-27 13:54:03,190][20381] Updated weights for policy 1, policy_version 1189908 (0.0010) [2023-12-27 13:54:03,597][20347] Updated weights for policy 0, policy_version 1190931 (0.0006) [2023-12-27 13:54:03,655][20347] Updated weights for policy 0, policy_version 1190941 (0.0007) [2023-12-27 13:54:03,716][20347] Updated weights for policy 0, policy_version 1190951 (0.0008) [2023-12-27 13:54:03,875][20381] Updated weights for policy 1, policy_version 1189918 (0.0009) [2023-12-27 13:54:03,926][20381] Updated weights for policy 1, policy_version 1189928 (0.0008) [2023-12-27 13:54:03,980][20381] Updated weights for policy 1, policy_version 1189938 (0.0009) [2023-12-27 13:54:04,384][20347] Updated weights for policy 0, policy_version 1190961 (0.0007) [2023-12-27 13:54:04,443][20347] Updated weights for policy 0, policy_version 1190971 (0.0008) [2023-12-27 13:54:04,501][20347] Updated weights for policy 0, policy_version 1190981 (0.0007) [2023-12-27 13:54:04,554][20347] Updated weights for policy 0, policy_version 1190991 (0.0008) [2023-12-27 13:54:04,706][20381] Updated weights for policy 1, policy_version 1189948 (0.0009) [2023-12-27 13:54:04,760][20381] Updated weights for policy 1, policy_version 1189958 (0.0006) [2023-12-27 13:54:04,818][20381] Updated weights for policy 1, policy_version 1189968 (0.0005) [2023-12-27 13:54:05,241][20347] Updated weights for policy 0, policy_version 1191001 (0.0008) [2023-12-27 13:54:05,292][20347] Updated weights for policy 0, policy_version 1191011 (0.0008) [2023-12-27 13:54:05,348][20347] Updated weights for policy 0, policy_version 1191021 (0.0007) [2023-12-27 13:54:05,462][20381] Updated weights for policy 1, policy_version 1189978 (0.0006) [2023-12-27 13:54:05,526][20381] Updated weights for policy 1, policy_version 1189988 (0.0008) [2023-12-27 13:54:05,579][20381] Updated weights for policy 1, policy_version 1189998 (0.0008) [2023-12-27 13:54:05,629][20381] Updated weights for policy 1, policy_version 1190008 (0.0008) [2023-12-27 13:54:06,009][20347] Updated weights for policy 0, policy_version 1191031 (0.0006) [2023-12-27 13:54:06,067][20347] Updated weights for policy 0, policy_version 1191041 (0.0006) [2023-12-27 13:54:06,127][20347] Updated weights for policy 0, policy_version 1191051 (0.0007) [2023-12-27 13:54:06,359][20381] Updated weights for policy 1, policy_version 1190018 (0.0008) [2023-12-27 13:54:06,415][20381] Updated weights for policy 1, policy_version 1190028 (0.0006) [2023-12-27 13:54:06,461][20381] Updated weights for policy 1, policy_version 1190038 (0.0005) [2023-12-27 13:54:06,884][20347] Updated weights for policy 0, policy_version 1191061 (0.0007) [2023-12-27 13:54:06,941][20347] Updated weights for policy 0, policy_version 1191071 (0.0006) [2023-12-27 13:54:06,996][20347] Updated weights for policy 0, policy_version 1191081 (0.0006) [2023-12-27 13:54:07,080][20381] Updated weights for policy 1, policy_version 1190048 (0.0010) [2023-12-27 13:54:07,153][20381] Updated weights for policy 1, policy_version 1190058 (0.0011) [2023-12-27 13:54:07,210][20381] Updated weights for policy 1, policy_version 1190068 (0.0011) [2023-12-27 13:54:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 609656832. Throughput: 0: 10366.2, 1: 10198.7. Samples: 609623512. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:07,363][19170] Avg episode reward: [(0, '804.340'), (1, '692.550')] [2023-12-27 13:54:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001190072_304685056.pth... [2023-12-27 13:54:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001191088_304971776.pth... [2023-12-27 13:54:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001189872_304660480.pth [2023-12-27 13:54:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001188888_304381952.pth [2023-12-27 13:54:07,630][20347] Updated weights for policy 0, policy_version 1191091 (0.0006) [2023-12-27 13:54:07,685][20347] Updated weights for policy 0, policy_version 1191101 (0.0008) [2023-12-27 13:54:07,740][20347] Updated weights for policy 0, policy_version 1191111 (0.0005) [2023-12-27 13:54:07,867][20381] Updated weights for policy 1, policy_version 1190078 (0.0010) [2023-12-27 13:54:07,919][20381] Updated weights for policy 1, policy_version 1190088 (0.0010) [2023-12-27 13:54:07,971][20381] Updated weights for policy 1, policy_version 1190098 (0.0010) [2023-12-27 13:54:08,371][20347] Updated weights for policy 0, policy_version 1191121 (0.0010) [2023-12-27 13:54:08,425][20347] Updated weights for policy 0, policy_version 1191131 (0.0005) [2023-12-27 13:54:08,482][20347] Updated weights for policy 0, policy_version 1191141 (0.0007) [2023-12-27 13:54:08,538][20347] Updated weights for policy 0, policy_version 1191151 (0.0007) [2023-12-27 13:54:08,614][20381] Updated weights for policy 1, policy_version 1190108 (0.0008) [2023-12-27 13:54:08,681][20381] Updated weights for policy 1, policy_version 1190118 (0.0005) [2023-12-27 13:54:08,727][20381] Updated weights for policy 1, policy_version 1190128 (0.0005) [2023-12-27 13:54:09,176][20347] Updated weights for policy 0, policy_version 1191161 (0.0007) [2023-12-27 13:54:09,234][20347] Updated weights for policy 0, policy_version 1191171 (0.0007) [2023-12-27 13:54:09,295][20347] Updated weights for policy 0, policy_version 1191181 (0.0007) [2023-12-27 13:54:09,459][20381] Updated weights for policy 1, policy_version 1190138 (0.0006) [2023-12-27 13:54:09,518][20381] Updated weights for policy 1, policy_version 1190148 (0.0008) [2023-12-27 13:54:09,577][20381] Updated weights for policy 1, policy_version 1190158 (0.0006) [2023-12-27 13:54:09,637][20381] Updated weights for policy 1, policy_version 1190168 (0.0006) [2023-12-27 13:54:10,042][20347] Updated weights for policy 0, policy_version 1191191 (0.0008) [2023-12-27 13:54:10,097][20347] Updated weights for policy 0, policy_version 1191201 (0.0007) [2023-12-27 13:54:10,155][20347] Updated weights for policy 0, policy_version 1191211 (0.0008) [2023-12-27 13:54:10,262][20381] Updated weights for policy 1, policy_version 1190178 (0.0007) [2023-12-27 13:54:10,324][20381] Updated weights for policy 1, policy_version 1190188 (0.0008) [2023-12-27 13:54:10,387][20381] Updated weights for policy 1, policy_version 1190198 (0.0007) [2023-12-27 13:54:10,881][20347] Updated weights for policy 0, policy_version 1191221 (0.0008) [2023-12-27 13:54:10,941][20347] Updated weights for policy 0, policy_version 1191231 (0.0008) [2023-12-27 13:54:10,966][20381] Updated weights for policy 1, policy_version 1190208 (0.0007) [2023-12-27 13:54:10,999][20347] Updated weights for policy 0, policy_version 1191241 (0.0007) [2023-12-27 13:54:11,027][20381] Updated weights for policy 1, policy_version 1190218 (0.0006) [2023-12-27 13:54:11,094][20381] Updated weights for policy 1, policy_version 1190228 (0.0008) [2023-12-27 13:54:11,742][20347] Updated weights for policy 0, policy_version 1191251 (0.0010) [2023-12-27 13:54:11,800][20381] Updated weights for policy 1, policy_version 1190238 (0.0007) [2023-12-27 13:54:11,806][20347] Updated weights for policy 0, policy_version 1191261 (0.0007) [2023-12-27 13:54:11,856][20347] Updated weights for policy 0, policy_version 1191271 (0.0006) [2023-12-27 13:54:11,858][20381] Updated weights for policy 1, policy_version 1190248 (0.0007) [2023-12-27 13:54:11,915][20381] Updated weights for policy 1, policy_version 1190258 (0.0007) [2023-12-27 13:54:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 609755136. Throughput: 0: 10370.4, 1: 10215.0. Samples: 609747224. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:12,363][19170] Avg episode reward: [(0, '823.460'), (1, '759.730')] [2023-12-27 13:54:12,543][20347] Updated weights for policy 0, policy_version 1191281 (0.0006) [2023-12-27 13:54:12,599][20347] Updated weights for policy 0, policy_version 1191291 (0.0006) [2023-12-27 13:54:12,599][20291] KL-divergence is very high: 155.7359 [2023-12-27 13:54:12,605][20291] KL-divergence is very high: 164.6273 [2023-12-27 13:54:12,612][20381] Updated weights for policy 1, policy_version 1190268 (0.0007) [2023-12-27 13:54:12,620][20291] KL-divergence is very high: 141.5177 [2023-12-27 13:54:12,652][20347] Updated weights for policy 0, policy_version 1191301 (0.0006) [2023-12-27 13:54:12,674][20381] Updated weights for policy 1, policy_version 1190278 (0.0006) [2023-12-27 13:54:12,704][20347] Updated weights for policy 0, policy_version 1191311 (0.0007) [2023-12-27 13:54:12,726][20381] Updated weights for policy 1, policy_version 1190288 (0.0007) [2023-12-27 13:54:13,349][20347] Updated weights for policy 0, policy_version 1191321 (0.0005) [2023-12-27 13:54:13,389][20381] Updated weights for policy 1, policy_version 1190298 (0.0008) [2023-12-27 13:54:13,406][20347] Updated weights for policy 0, policy_version 1191331 (0.0005) [2023-12-27 13:54:13,441][20381] Updated weights for policy 1, policy_version 1190308 (0.0006) [2023-12-27 13:54:13,453][20347] Updated weights for policy 0, policy_version 1191341 (0.0005) [2023-12-27 13:54:13,493][20381] Updated weights for policy 1, policy_version 1190318 (0.0007) [2023-12-27 13:54:13,546][20381] Updated weights for policy 1, policy_version 1190328 (0.0008) [2023-12-27 13:54:14,119][20347] Updated weights for policy 0, policy_version 1191351 (0.0007) [2023-12-27 13:54:14,166][20347] Updated weights for policy 0, policy_version 1191361 (0.0007) [2023-12-27 13:54:14,209][20381] Updated weights for policy 1, policy_version 1190338 (0.0007) [2023-12-27 13:54:14,215][20347] Updated weights for policy 0, policy_version 1191371 (0.0006) [2023-12-27 13:54:14,266][20381] Updated weights for policy 1, policy_version 1190348 (0.0006) [2023-12-27 13:54:14,320][20381] Updated weights for policy 1, policy_version 1190358 (0.0007) [2023-12-27 13:54:14,949][20347] Updated weights for policy 0, policy_version 1191381 (0.0006) [2023-12-27 13:54:15,005][20347] Updated weights for policy 0, policy_version 1191391 (0.0009) [2023-12-27 13:54:15,023][20381] Updated weights for policy 1, policy_version 1190368 (0.0008) [2023-12-27 13:54:15,059][20347] Updated weights for policy 0, policy_version 1191401 (0.0006) [2023-12-27 13:54:15,086][20381] Updated weights for policy 1, policy_version 1190378 (0.0007) [2023-12-27 13:54:15,148][20381] Updated weights for policy 1, policy_version 1190388 (0.0007) [2023-12-27 13:54:15,765][20347] Updated weights for policy 0, policy_version 1191411 (0.0006) [2023-12-27 13:54:15,775][20381] Updated weights for policy 1, policy_version 1190398 (0.0009) [2023-12-27 13:54:15,810][20291] KL-divergence is very high: 266.5108 [2023-12-27 13:54:15,817][20291] KL-divergence is very high: 280.6722 [2023-12-27 13:54:15,823][20347] Updated weights for policy 0, policy_version 1191421 (0.0007) [2023-12-27 13:54:15,835][20381] Updated weights for policy 1, policy_version 1190408 (0.0009) [2023-12-27 13:54:15,849][20291] KL-divergence is very high: 124.9477 [2023-12-27 13:54:15,862][20291] KL-divergence is very high: 173.2552 [2023-12-27 13:54:15,869][20291] KL-divergence is very high: 135.4568 [2023-12-27 13:54:15,888][20347] Updated weights for policy 0, policy_version 1191431 (0.0006) [2023-12-27 13:54:15,898][20381] Updated weights for policy 1, policy_version 1190418 (0.0010) [2023-12-27 13:54:16,489][20381] Updated weights for policy 1, policy_version 1190428 (0.0008) [2023-12-27 13:54:16,539][20381] Updated weights for policy 1, policy_version 1190438 (0.0007) [2023-12-27 13:54:16,590][20291] KL-divergence is very high: 974.3282 [2023-12-27 13:54:16,593][20381] Updated weights for policy 1, policy_version 1190448 (0.0008) [2023-12-27 13:54:16,594][20347] Updated weights for policy 0, policy_version 1191441 (0.0007) [2023-12-27 13:54:16,607][20291] KL-divergence is very high: 904.1306 [2023-12-27 13:54:16,634][20291] KL-divergence is very high: 1390.3588 [2023-12-27 13:54:16,648][20347] Updated weights for policy 0, policy_version 1191451 (0.0006) [2023-12-27 13:54:16,648][20291] KL-divergence is very high: 315.9745 [2023-12-27 13:54:16,673][20291] KL-divergence is very high: 876.6375 [2023-12-27 13:54:16,689][20291] KL-divergence is very high: 176.7186 [2023-12-27 13:54:16,699][20347] Updated weights for policy 0, policy_version 1191461 (0.0007) [2023-12-27 13:54:16,717][20291] KL-divergence is very high: 880.7300 [2023-12-27 13:54:16,737][20291] KL-divergence is very high: 229.5385 [2023-12-27 13:54:16,763][20347] Updated weights for policy 0, policy_version 1191471 (0.0008) [2023-12-27 13:54:17,348][20381] Updated weights for policy 1, policy_version 1190458 (0.0008) [2023-12-27 13:54:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 609853440. Throughput: 0: 10330.0, 1: 10227.6. Samples: 609869480. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:17,363][19170] Avg episode reward: [(0, '754.890'), (1, '749.830')] [2023-12-27 13:54:17,393][20347] Updated weights for policy 0, policy_version 1191481 (0.0007) [2023-12-27 13:54:17,395][20381] Updated weights for policy 1, policy_version 1190468 (0.0006) [2023-12-27 13:54:17,400][20291] KL-divergence is very high: 103.6069 [2023-12-27 13:54:17,448][20347] Updated weights for policy 0, policy_version 1191491 (0.0009) [2023-12-27 13:54:17,455][20381] Updated weights for policy 1, policy_version 1190478 (0.0008) [2023-12-27 13:54:17,507][20347] Updated weights for policy 0, policy_version 1191501 (0.0007) [2023-12-27 13:54:17,508][20381] Updated weights for policy 1, policy_version 1190488 (0.0006) [2023-12-27 13:54:18,193][20381] Updated weights for policy 1, policy_version 1190498 (0.0007) [2023-12-27 13:54:18,207][20347] Updated weights for policy 0, policy_version 1191511 (0.0007) [2023-12-27 13:54:18,254][20381] Updated weights for policy 1, policy_version 1190508 (0.0006) [2023-12-27 13:54:18,259][20347] Updated weights for policy 0, policy_version 1191521 (0.0007) [2023-12-27 13:54:18,312][20347] Updated weights for policy 0, policy_version 1191531 (0.0008) [2023-12-27 13:54:18,313][20381] Updated weights for policy 1, policy_version 1190518 (0.0006) [2023-12-27 13:54:18,977][20381] Updated weights for policy 1, policy_version 1190528 (0.0008) [2023-12-27 13:54:19,025][20381] Updated weights for policy 1, policy_version 1190538 (0.0008) [2023-12-27 13:54:19,031][20347] Updated weights for policy 0, policy_version 1191541 (0.0008) [2023-12-27 13:54:19,079][20347] Updated weights for policy 0, policy_version 1191551 (0.0007) [2023-12-27 13:54:19,080][20381] Updated weights for policy 1, policy_version 1190548 (0.0006) [2023-12-27 13:54:19,126][20347] Updated weights for policy 0, policy_version 1191561 (0.0007) [2023-12-27 13:54:19,805][20347] Updated weights for policy 0, policy_version 1191571 (0.0006) [2023-12-27 13:54:19,863][20347] Updated weights for policy 0, policy_version 1191581 (0.0009) [2023-12-27 13:54:19,880][20381] Updated weights for policy 1, policy_version 1190558 (0.0007) [2023-12-27 13:54:19,917][20347] Updated weights for policy 0, policy_version 1191591 (0.0006) [2023-12-27 13:54:19,942][20381] Updated weights for policy 1, policy_version 1190568 (0.0008) [2023-12-27 13:54:20,002][20381] Updated weights for policy 1, policy_version 1190578 (0.0007) [2023-12-27 13:54:20,663][20347] Updated weights for policy 0, policy_version 1191601 (0.0007) [2023-12-27 13:54:20,726][20347] Updated weights for policy 0, policy_version 1191611 (0.0007) [2023-12-27 13:54:20,748][20381] Updated weights for policy 1, policy_version 1190588 (0.0008) [2023-12-27 13:54:20,786][20347] Updated weights for policy 0, policy_version 1191621 (0.0006) [2023-12-27 13:54:20,815][20381] Updated weights for policy 1, policy_version 1190598 (0.0007) [2023-12-27 13:54:20,843][20347] Updated weights for policy 0, policy_version 1191631 (0.0006) [2023-12-27 13:54:20,878][20381] Updated weights for policy 1, policy_version 1190608 (0.0006) [2023-12-27 13:54:21,573][20381] Updated weights for policy 1, policy_version 1190618 (0.0006) [2023-12-27 13:54:21,613][20347] Updated weights for policy 0, policy_version 1191641 (0.0010) [2023-12-27 13:54:21,633][20381] Updated weights for policy 1, policy_version 1190628 (0.0007) [2023-12-27 13:54:21,675][20347] Updated weights for policy 0, policy_version 1191651 (0.0010) [2023-12-27 13:54:21,693][20381] Updated weights for policy 1, policy_version 1190638 (0.0006) [2023-12-27 13:54:21,742][20347] Updated weights for policy 0, policy_version 1191661 (0.0009) [2023-12-27 13:54:21,759][20381] Updated weights for policy 1, policy_version 1190648 (0.0009) [2023-12-27 13:54:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 609951744. Throughput: 0: 10303.4, 1: 10186.6. Samples: 609928724. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:22,363][19170] Avg episode reward: [(0, '794.730'), (1, '797.700')] [2023-12-27 13:54:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001190648_304832512.pth... [2023-12-27 13:54:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001191664_305119232.pth... [2023-12-27 13:54:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001189464_304529408.pth [2023-12-27 13:54:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001190480_304816128.pth [2023-12-27 13:54:22,476][20291] KL-divergence is very high: 212.2049 [2023-12-27 13:54:22,488][20347] Updated weights for policy 0, policy_version 1191671 (0.0007) [2023-12-27 13:54:22,488][20291] KL-divergence is very high: 127.5614 [2023-12-27 13:54:22,542][20347] Updated weights for policy 0, policy_version 1191681 (0.0010) [2023-12-27 13:54:22,547][20381] Updated weights for policy 1, policy_version 1190658 (0.0006) [2023-12-27 13:54:22,565][20291] KL-divergence is very high: 156.8821 [2023-12-27 13:54:22,571][20291] KL-divergence is very high: 195.7484 [2023-12-27 13:54:22,599][20347] Updated weights for policy 0, policy_version 1191691 (0.0008) [2023-12-27 13:54:22,602][20381] Updated weights for policy 1, policy_version 1190668 (0.0006) [2023-12-27 13:54:22,654][20381] Updated weights for policy 1, policy_version 1190678 (0.0007) [2023-12-27 13:54:23,292][20347] Updated weights for policy 0, policy_version 1191701 (0.0008) [2023-12-27 13:54:23,327][20291] KL-divergence is very high: 109.8084 [2023-12-27 13:54:23,345][20347] Updated weights for policy 0, policy_version 1191711 (0.0008) [2023-12-27 13:54:23,386][20381] Updated weights for policy 1, policy_version 1190688 (0.0007) [2023-12-27 13:54:23,402][20347] Updated weights for policy 0, policy_version 1191721 (0.0006) [2023-12-27 13:54:23,446][20381] Updated weights for policy 1, policy_version 1190698 (0.0007) [2023-12-27 13:54:23,509][20381] Updated weights for policy 1, policy_version 1190708 (0.0009) [2023-12-27 13:54:24,019][20347] Updated weights for policy 0, policy_version 1191731 (0.0007) [2023-12-27 13:54:24,076][20347] Updated weights for policy 0, policy_version 1191741 (0.0007) [2023-12-27 13:54:24,135][20347] Updated weights for policy 0, policy_version 1191751 (0.0008) [2023-12-27 13:54:24,204][20381] Updated weights for policy 1, policy_version 1190718 (0.0007) [2023-12-27 13:54:24,252][20381] Updated weights for policy 1, policy_version 1190728 (0.0007) [2023-12-27 13:54:24,300][20381] Updated weights for policy 1, policy_version 1190738 (0.0006) [2023-12-27 13:54:24,809][20347] Updated weights for policy 0, policy_version 1191761 (0.0007) [2023-12-27 13:54:24,866][20347] Updated weights for policy 0, policy_version 1191771 (0.0007) [2023-12-27 13:54:24,928][20347] Updated weights for policy 0, policy_version 1191781 (0.0008) [2023-12-27 13:54:24,934][20381] Updated weights for policy 1, policy_version 1190748 (0.0006) [2023-12-27 13:54:24,982][20381] Updated weights for policy 1, policy_version 1190758 (0.0005) [2023-12-27 13:54:24,992][20347] Updated weights for policy 0, policy_version 1191791 (0.0008) [2023-12-27 13:54:25,040][20381] Updated weights for policy 1, policy_version 1190768 (0.0007) [2023-12-27 13:54:25,636][20347] Updated weights for policy 0, policy_version 1191801 (0.0007) [2023-12-27 13:54:25,641][20381] Updated weights for policy 1, policy_version 1190778 (0.0008) [2023-12-27 13:54:25,688][20381] Updated weights for policy 1, policy_version 1190788 (0.0005) [2023-12-27 13:54:25,691][20347] Updated weights for policy 0, policy_version 1191811 (0.0007) [2023-12-27 13:54:25,733][20381] Updated weights for policy 1, policy_version 1190798 (0.0005) [2023-12-27 13:54:25,750][20347] Updated weights for policy 0, policy_version 1191821 (0.0007) [2023-12-27 13:54:25,789][20381] Updated weights for policy 1, policy_version 1190808 (0.0006) [2023-12-27 13:54:26,397][20381] Updated weights for policy 1, policy_version 1190818 (0.0007) [2023-12-27 13:54:26,453][20381] Updated weights for policy 1, policy_version 1190828 (0.0005) [2023-12-27 13:54:26,456][20347] Updated weights for policy 0, policy_version 1191831 (0.0006) [2023-12-27 13:54:26,513][20381] Updated weights for policy 1, policy_version 1190838 (0.0005) [2023-12-27 13:54:26,514][20347] Updated weights for policy 0, policy_version 1191841 (0.0007) [2023-12-27 13:54:26,572][20347] Updated weights for policy 0, policy_version 1191851 (0.0007) [2023-12-27 13:54:27,068][20381] Updated weights for policy 1, policy_version 1190848 (0.0005) [2023-12-27 13:54:27,120][20381] Updated weights for policy 1, policy_version 1190858 (0.0005) [2023-12-27 13:54:27,174][20381] Updated weights for policy 1, policy_version 1190868 (0.0006) [2023-12-27 13:54:27,311][20347] Updated weights for policy 0, policy_version 1191861 (0.0009) [2023-12-27 13:54:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 610058240. Throughput: 0: 10273.9, 1: 10156.0. Samples: 610049492. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:27,364][19170] Avg episode reward: [(0, '765.540'), (1, '655.660')] [2023-12-27 13:54:27,368][20347] Updated weights for policy 0, policy_version 1191871 (0.0008) [2023-12-27 13:54:27,422][20347] Updated weights for policy 0, policy_version 1191882 (0.0008) [2023-12-27 13:54:27,745][20381] Updated weights for policy 1, policy_version 1190878 (0.0008) [2023-12-27 13:54:27,790][20381] Updated weights for policy 1, policy_version 1190888 (0.0008) [2023-12-27 13:54:27,835][20381] Updated weights for policy 1, policy_version 1190898 (0.0008) [2023-12-27 13:54:27,990][20347] Updated weights for policy 0, policy_version 1191892 (0.0008) [2023-12-27 13:54:28,035][20347] Updated weights for policy 0, policy_version 1191902 (0.0010) [2023-12-27 13:54:28,080][20347] Updated weights for policy 0, policy_version 1191912 (0.0010) [2023-12-27 13:54:28,622][20381] Updated weights for policy 1, policy_version 1190908 (0.0008) [2023-12-27 13:54:28,652][20347] Updated weights for policy 0, policy_version 1191922 (0.0009) [2023-12-27 13:54:28,686][20381] Updated weights for policy 1, policy_version 1190918 (0.0007) [2023-12-27 13:54:28,712][20347] Updated weights for policy 0, policy_version 1191932 (0.0005) [2023-12-27 13:54:28,743][20381] Updated weights for policy 1, policy_version 1190928 (0.0007) [2023-12-27 13:54:28,769][20347] Updated weights for policy 0, policy_version 1191942 (0.0007) [2023-12-27 13:54:28,829][20347] Updated weights for policy 0, policy_version 1191952 (0.0007) [2023-12-27 13:54:29,365][20381] Updated weights for policy 1, policy_version 1190938 (0.0006) [2023-12-27 13:54:29,425][20381] Updated weights for policy 1, policy_version 1190948 (0.0010) [2023-12-27 13:54:29,470][20347] Updated weights for policy 0, policy_version 1191962 (0.0005) [2023-12-27 13:54:29,488][20381] Updated weights for policy 1, policy_version 1190958 (0.0011) [2023-12-27 13:54:29,530][20347] Updated weights for policy 0, policy_version 1191972 (0.0007) [2023-12-27 13:54:29,546][20381] Updated weights for policy 1, policy_version 1190968 (0.0008) [2023-12-27 13:54:29,584][20347] Updated weights for policy 0, policy_version 1191982 (0.0009) [2023-12-27 13:54:30,193][20381] Updated weights for policy 1, policy_version 1190978 (0.0011) [2023-12-27 13:54:30,251][20381] Updated weights for policy 1, policy_version 1190988 (0.0011) [2023-12-27 13:54:30,257][20347] Updated weights for policy 0, policy_version 1191992 (0.0006) [2023-12-27 13:54:30,306][20381] Updated weights for policy 1, policy_version 1190998 (0.0011) [2023-12-27 13:54:30,318][20347] Updated weights for policy 0, policy_version 1192002 (0.0005) [2023-12-27 13:54:30,373][20347] Updated weights for policy 0, policy_version 1192012 (0.0008) [2023-12-27 13:54:30,981][20347] Updated weights for policy 0, policy_version 1192022 (0.0006) [2023-12-27 13:54:30,984][20381] Updated weights for policy 1, policy_version 1191008 (0.0008) [2023-12-27 13:54:31,040][20381] Updated weights for policy 1, policy_version 1191018 (0.0007) [2023-12-27 13:54:31,045][20347] Updated weights for policy 0, policy_version 1192032 (0.0006) [2023-12-27 13:54:31,098][20381] Updated weights for policy 1, policy_version 1191028 (0.0008) [2023-12-27 13:54:31,110][20347] Updated weights for policy 0, policy_version 1192042 (0.0007) [2023-12-27 13:54:31,757][20347] Updated weights for policy 0, policy_version 1192052 (0.0007) [2023-12-27 13:54:31,814][20347] Updated weights for policy 0, policy_version 1192062 (0.0006) [2023-12-27 13:54:31,826][20381] Updated weights for policy 1, policy_version 1191038 (0.0007) [2023-12-27 13:54:31,877][20347] Updated weights for policy 0, policy_version 1192072 (0.0007) [2023-12-27 13:54:31,878][20381] Updated weights for policy 1, policy_version 1191048 (0.0005) [2023-12-27 13:54:31,936][20381] Updated weights for policy 1, policy_version 1191058 (0.0008) [2023-12-27 13:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 610164736. Throughput: 0: 10262.3, 1: 10197.6. Samples: 610175768. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:32,363][19170] Avg episode reward: [(0, '755.610'), (1, '653.310')] [2023-12-27 13:54:32,595][20347] Updated weights for policy 0, policy_version 1192082 (0.0006) [2023-12-27 13:54:32,626][20381] Updated weights for policy 1, policy_version 1191068 (0.0006) [2023-12-27 13:54:32,649][20347] Updated weights for policy 0, policy_version 1192092 (0.0007) [2023-12-27 13:54:32,682][20381] Updated weights for policy 1, policy_version 1191078 (0.0006) [2023-12-27 13:54:32,684][20291] KL-divergence is very high: 143.3210 [2023-12-27 13:54:32,698][20347] Updated weights for policy 0, policy_version 1192102 (0.0009) [2023-12-27 13:54:32,704][20291] KL-divergence is very high: 123.2767 [2023-12-27 13:54:32,740][20381] Updated weights for policy 1, policy_version 1191088 (0.0007) [2023-12-27 13:54:32,749][20347] Updated weights for policy 0, policy_version 1192112 (0.0007) [2023-12-27 13:54:33,354][20347] Updated weights for policy 0, policy_version 1192122 (0.0006) [2023-12-27 13:54:33,413][20347] Updated weights for policy 0, policy_version 1192132 (0.0006) [2023-12-27 13:54:33,415][20381] Updated weights for policy 1, policy_version 1191098 (0.0007) [2023-12-27 13:54:33,468][20347] Updated weights for policy 0, policy_version 1192142 (0.0005) [2023-12-27 13:54:33,470][20381] Updated weights for policy 1, policy_version 1191108 (0.0006) [2023-12-27 13:54:33,516][20381] Updated weights for policy 1, policy_version 1191118 (0.0008) [2023-12-27 13:54:33,571][20381] Updated weights for policy 1, policy_version 1191128 (0.0009) [2023-12-27 13:54:34,094][20347] Updated weights for policy 0, policy_version 1192152 (0.0007) [2023-12-27 13:54:34,108][20291] KL-divergence is very high: 132.4278 [2023-12-27 13:54:34,159][20347] Updated weights for policy 0, policy_version 1192162 (0.0008) [2023-12-27 13:54:34,159][20291] KL-divergence is very high: 152.4479 [2023-12-27 13:54:34,224][20347] Updated weights for policy 0, policy_version 1192172 (0.0008) [2023-12-27 13:54:34,251][20381] Updated weights for policy 1, policy_version 1191138 (0.0006) [2023-12-27 13:54:34,304][20381] Updated weights for policy 1, policy_version 1191148 (0.0008) [2023-12-27 13:54:34,358][20381] Updated weights for policy 1, policy_version 1191158 (0.0007) [2023-12-27 13:54:34,913][20347] Updated weights for policy 0, policy_version 1192182 (0.0007) [2023-12-27 13:54:34,961][20347] Updated weights for policy 0, policy_version 1192192 (0.0007) [2023-12-27 13:54:35,015][20347] Updated weights for policy 0, policy_version 1192202 (0.0008) [2023-12-27 13:54:35,047][20381] Updated weights for policy 1, policy_version 1191168 (0.0008) [2023-12-27 13:54:35,095][20381] Updated weights for policy 1, policy_version 1191178 (0.0007) [2023-12-27 13:54:35,147][20381] Updated weights for policy 1, policy_version 1191188 (0.0007) [2023-12-27 13:54:35,682][20347] Updated weights for policy 0, policy_version 1192212 (0.0007) [2023-12-27 13:54:35,739][20347] Updated weights for policy 0, policy_version 1192222 (0.0007) [2023-12-27 13:54:35,795][20347] Updated weights for policy 0, policy_version 1192232 (0.0008) [2023-12-27 13:54:35,830][20381] Updated weights for policy 1, policy_version 1191198 (0.0006) [2023-12-27 13:54:35,890][20381] Updated weights for policy 1, policy_version 1191208 (0.0007) [2023-12-27 13:54:35,950][20381] Updated weights for policy 1, policy_version 1191218 (0.0010) [2023-12-27 13:54:36,484][20347] Updated weights for policy 0, policy_version 1192242 (0.0007) [2023-12-27 13:54:36,542][20347] Updated weights for policy 0, policy_version 1192252 (0.0005) [2023-12-27 13:54:36,554][20291] KL-divergence is very high: 157.1506 [2023-12-27 13:54:36,560][20291] KL-divergence is very high: 155.8995 [2023-12-27 13:54:36,600][20347] Updated weights for policy 0, policy_version 1192262 (0.0008) [2023-12-27 13:54:36,607][20381] Updated weights for policy 1, policy_version 1191228 (0.0010) [2023-12-27 13:54:36,657][20347] Updated weights for policy 0, policy_version 1192272 (0.0008) [2023-12-27 13:54:36,665][20381] Updated weights for policy 1, policy_version 1191238 (0.0007) [2023-12-27 13:54:36,724][20381] Updated weights for policy 1, policy_version 1191248 (0.0006) [2023-12-27 13:54:37,355][20347] Updated weights for policy 0, policy_version 1192282 (0.0007) [2023-12-27 13:54:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 610263040. Throughput: 0: 10277.2, 1: 10237.3. Samples: 610237884. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:37,363][19170] Avg episode reward: [(0, '746.240'), (1, '786.920')] [2023-12-27 13:54:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001191256_304988160.pth... [2023-12-27 13:54:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001190072_304685056.pth [2023-12-27 13:54:37,399][20381] Updated weights for policy 1, policy_version 1191258 (0.0010) [2023-12-27 13:54:37,408][20347] Updated weights for policy 0, policy_version 1192292 (0.0008) [2023-12-27 13:54:37,454][20381] Updated weights for policy 1, policy_version 1191268 (0.0009) [2023-12-27 13:54:37,460][20347] Updated weights for policy 0, policy_version 1192302 (0.0006) [2023-12-27 13:54:37,467][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001192304_305283072.pth... [2023-12-27 13:54:37,471][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001191088_304971776.pth [2023-12-27 13:54:37,507][20381] Updated weights for policy 1, policy_version 1191278 (0.0010) [2023-12-27 13:54:37,563][20381] Updated weights for policy 1, policy_version 1191288 (0.0010) [2023-12-27 13:54:38,144][20347] Updated weights for policy 0, policy_version 1192312 (0.0008) [2023-12-27 13:54:38,184][20381] Updated weights for policy 1, policy_version 1191298 (0.0006) [2023-12-27 13:54:38,204][20347] Updated weights for policy 0, policy_version 1192322 (0.0007) [2023-12-27 13:54:38,238][20381] Updated weights for policy 1, policy_version 1191308 (0.0005) [2023-12-27 13:54:38,262][20347] Updated weights for policy 0, policy_version 1192332 (0.0008) [2023-12-27 13:54:38,296][20381] Updated weights for policy 1, policy_version 1191318 (0.0006) [2023-12-27 13:54:38,970][20381] Updated weights for policy 1, policy_version 1191328 (0.0007) [2023-12-27 13:54:38,992][20347] Updated weights for policy 0, policy_version 1192342 (0.0006) [2023-12-27 13:54:39,015][20381] Updated weights for policy 1, policy_version 1191338 (0.0008) [2023-12-27 13:54:39,040][20347] Updated weights for policy 0, policy_version 1192352 (0.0006) [2023-12-27 13:54:39,060][20381] Updated weights for policy 1, policy_version 1191348 (0.0008) [2023-12-27 13:54:39,088][20347] Updated weights for policy 0, policy_version 1192362 (0.0006) [2023-12-27 13:54:39,751][20381] Updated weights for policy 1, policy_version 1191358 (0.0006) [2023-12-27 13:54:39,815][20381] Updated weights for policy 1, policy_version 1191368 (0.0006) [2023-12-27 13:54:39,863][20347] Updated weights for policy 0, policy_version 1192372 (0.0007) [2023-12-27 13:54:39,886][20381] Updated weights for policy 1, policy_version 1191378 (0.0007) [2023-12-27 13:54:39,928][20347] Updated weights for policy 0, policy_version 1192382 (0.0007) [2023-12-27 13:54:39,990][20347] Updated weights for policy 0, policy_version 1192392 (0.0006) [2023-12-27 13:54:40,510][20381] Updated weights for policy 1, policy_version 1191388 (0.0006) [2023-12-27 13:54:40,566][20381] Updated weights for policy 1, policy_version 1191398 (0.0005) [2023-12-27 13:54:40,629][20347] Updated weights for policy 0, policy_version 1192402 (0.0007) [2023-12-27 13:54:40,630][20292] KL-divergence is very high: 122.4459 [2023-12-27 13:54:40,630][20381] Updated weights for policy 1, policy_version 1191408 (0.0006) [2023-12-27 13:54:40,636][20292] KL-divergence is very high: 112.2110 [2023-12-27 13:54:40,677][20347] Updated weights for policy 0, policy_version 1192412 (0.0006) [2023-12-27 13:54:40,725][20347] Updated weights for policy 0, policy_version 1192422 (0.0008) [2023-12-27 13:54:40,785][20347] Updated weights for policy 0, policy_version 1192432 (0.0007) [2023-12-27 13:54:41,287][20381] Updated weights for policy 1, policy_version 1191418 (0.0007) [2023-12-27 13:54:41,342][20381] Updated weights for policy 1, policy_version 1191428 (0.0006) [2023-12-27 13:54:41,412][20381] Updated weights for policy 1, policy_version 1191438 (0.0008) [2023-12-27 13:54:41,471][20381] Updated weights for policy 1, policy_version 1191448 (0.0008) [2023-12-27 13:54:41,539][20347] Updated weights for policy 0, policy_version 1192442 (0.0007) [2023-12-27 13:54:41,598][20347] Updated weights for policy 0, policy_version 1192452 (0.0007) [2023-12-27 13:54:41,668][20347] Updated weights for policy 0, policy_version 1192462 (0.0011) [2023-12-27 13:54:42,206][20381] Updated weights for policy 1, policy_version 1191458 (0.0008) [2023-12-27 13:54:42,269][20381] Updated weights for policy 1, policy_version 1191468 (0.0008) [2023-12-27 13:54:42,329][20381] Updated weights for policy 1, policy_version 1191478 (0.0007) [2023-12-27 13:54:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20483.9, 300 sec: 20549.4). Total num frames: 610369536. Throughput: 0: 10180.5, 1: 10285.0. Samples: 610360720. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:42,363][19170] Avg episode reward: [(0, '872.860'), (1, '758.860')] [2023-12-27 13:54:42,424][20347] Updated weights for policy 0, policy_version 1192472 (0.0007) [2023-12-27 13:54:42,484][20347] Updated weights for policy 0, policy_version 1192482 (0.0007) [2023-12-27 13:54:42,541][20347] Updated weights for policy 0, policy_version 1192492 (0.0008) [2023-12-27 13:54:43,066][20381] Updated weights for policy 1, policy_version 1191488 (0.0007) [2023-12-27 13:54:43,126][20381] Updated weights for policy 1, policy_version 1191498 (0.0008) [2023-12-27 13:54:43,144][20291] KL-divergence is very high: 101.3065 [2023-12-27 13:54:43,162][20347] Updated weights for policy 0, policy_version 1192502 (0.0007) [2023-12-27 13:54:43,179][20381] Updated weights for policy 1, policy_version 1191508 (0.0008) [2023-12-27 13:54:43,212][20347] Updated weights for policy 0, policy_version 1192512 (0.0007) [2023-12-27 13:54:43,233][20291] KL-divergence is very high: 125.0087 [2023-12-27 13:54:43,243][20291] KL-divergence is very high: 134.7428 [2023-12-27 13:54:43,263][20347] Updated weights for policy 0, policy_version 1192522 (0.0005) [2023-12-27 13:54:43,819][20347] Updated weights for policy 0, policy_version 1192532 (0.0005) [2023-12-27 13:54:43,868][20347] Updated weights for policy 0, policy_version 1192542 (0.0005) [2023-12-27 13:54:43,872][20381] Updated weights for policy 1, policy_version 1191518 (0.0008) [2023-12-27 13:54:43,920][20347] Updated weights for policy 0, policy_version 1192552 (0.0005) [2023-12-27 13:54:43,924][20381] Updated weights for policy 1, policy_version 1191528 (0.0008) [2023-12-27 13:54:43,980][20381] Updated weights for policy 1, policy_version 1191538 (0.0007) [2023-12-27 13:54:44,453][20347] Updated weights for policy 0, policy_version 1192562 (0.0005) [2023-12-27 13:54:44,508][20347] Updated weights for policy 0, policy_version 1192572 (0.0005) [2023-12-27 13:54:44,562][20347] Updated weights for policy 0, policy_version 1192582 (0.0007) [2023-12-27 13:54:44,622][20347] Updated weights for policy 0, policy_version 1192592 (0.0007) [2023-12-27 13:54:44,691][20381] Updated weights for policy 1, policy_version 1191548 (0.0008) [2023-12-27 13:54:44,746][20381] Updated weights for policy 1, policy_version 1191558 (0.0005) [2023-12-27 13:54:44,811][20381] Updated weights for policy 1, policy_version 1191568 (0.0011) [2023-12-27 13:54:45,271][20347] Updated weights for policy 0, policy_version 1192602 (0.0010) [2023-12-27 13:54:45,332][20347] Updated weights for policy 0, policy_version 1192612 (0.0010) [2023-12-27 13:54:45,396][20347] Updated weights for policy 0, policy_version 1192622 (0.0010) [2023-12-27 13:54:45,442][20381] Updated weights for policy 1, policy_version 1191578 (0.0010) [2023-12-27 13:54:45,490][20381] Updated weights for policy 1, policy_version 1191588 (0.0010) [2023-12-27 13:54:45,538][20381] Updated weights for policy 1, policy_version 1191598 (0.0010) [2023-12-27 13:54:45,602][20381] Updated weights for policy 1, policy_version 1191608 (0.0010) [2023-12-27 13:54:46,055][20347] Updated weights for policy 0, policy_version 1192632 (0.0010) [2023-12-27 13:54:46,105][20347] Updated weights for policy 0, policy_version 1192642 (0.0007) [2023-12-27 13:54:46,153][20347] Updated weights for policy 0, policy_version 1192652 (0.0007) [2023-12-27 13:54:46,225][20381] Updated weights for policy 1, policy_version 1191618 (0.0009) [2023-12-27 13:54:46,281][20381] Updated weights for policy 1, policy_version 1191628 (0.0007) [2023-12-27 13:54:46,332][20381] Updated weights for policy 1, policy_version 1191638 (0.0010) [2023-12-27 13:54:46,761][20347] Updated weights for policy 0, policy_version 1192662 (0.0006) [2023-12-27 13:54:46,825][20347] Updated weights for policy 0, policy_version 1192672 (0.0006) [2023-12-27 13:54:46,876][20347] Updated weights for policy 0, policy_version 1192682 (0.0006) [2023-12-27 13:54:46,992][20381] Updated weights for policy 1, policy_version 1191648 (0.0008) [2023-12-27 13:54:47,048][20381] Updated weights for policy 1, policy_version 1191658 (0.0007) [2023-12-27 13:54:47,108][20381] Updated weights for policy 1, policy_version 1191668 (0.0008) [2023-12-27 13:54:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 610476032. Throughput: 0: 10285.2, 1: 10225.0. Samples: 610485260. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:47,363][19170] Avg episode reward: [(0, '902.430'), (1, '807.170')] [2023-12-27 13:54:47,452][20347] Updated weights for policy 0, policy_version 1192692 (0.0007) [2023-12-27 13:54:47,502][20347] Updated weights for policy 0, policy_version 1192702 (0.0005) [2023-12-27 13:54:47,546][20347] Updated weights for policy 0, policy_version 1192712 (0.0007) [2023-12-27 13:54:47,861][20381] Updated weights for policy 1, policy_version 1191678 (0.0009) [2023-12-27 13:54:47,929][20381] Updated weights for policy 1, policy_version 1191688 (0.0010) [2023-12-27 13:54:48,001][20381] Updated weights for policy 1, policy_version 1191699 (0.0009) [2023-12-27 13:54:48,245][20347] Updated weights for policy 0, policy_version 1192722 (0.0008) [2023-12-27 13:54:48,317][20347] Updated weights for policy 0, policy_version 1192732 (0.0006) [2023-12-27 13:54:48,387][20347] Updated weights for policy 0, policy_version 1192742 (0.0007) [2023-12-27 13:54:48,452][20347] Updated weights for policy 0, policy_version 1192752 (0.0007) [2023-12-27 13:54:48,725][20381] Updated weights for policy 1, policy_version 1191709 (0.0008) [2023-12-27 13:54:48,782][20381] Updated weights for policy 1, policy_version 1191719 (0.0008) [2023-12-27 13:54:48,832][20381] Updated weights for policy 1, policy_version 1191729 (0.0007) [2023-12-27 13:54:49,132][20347] Updated weights for policy 0, policy_version 1192762 (0.0008) [2023-12-27 13:54:49,190][20347] Updated weights for policy 0, policy_version 1192772 (0.0007) [2023-12-27 13:54:49,254][20347] Updated weights for policy 0, policy_version 1192782 (0.0009) [2023-12-27 13:54:49,551][20381] Updated weights for policy 1, policy_version 1191739 (0.0007) [2023-12-27 13:54:49,603][20381] Updated weights for policy 1, policy_version 1191749 (0.0005) [2023-12-27 13:54:49,652][20381] Updated weights for policy 1, policy_version 1191759 (0.0008) [2023-12-27 13:54:50,012][20347] Updated weights for policy 0, policy_version 1192792 (0.0007) [2023-12-27 13:54:50,068][20347] Updated weights for policy 0, policy_version 1192802 (0.0008) [2023-12-27 13:54:50,131][20347] Updated weights for policy 0, policy_version 1192812 (0.0008) [2023-12-27 13:54:50,362][20381] Updated weights for policy 1, policy_version 1191769 (0.0008) [2023-12-27 13:54:50,422][20381] Updated weights for policy 1, policy_version 1191779 (0.0008) [2023-12-27 13:54:50,474][20381] Updated weights for policy 1, policy_version 1191789 (0.0008) [2023-12-27 13:54:50,517][20381] Updated weights for policy 1, policy_version 1191799 (0.0005) [2023-12-27 13:54:50,860][20347] Updated weights for policy 0, policy_version 1192822 (0.0007) [2023-12-27 13:54:50,921][20347] Updated weights for policy 0, policy_version 1192832 (0.0007) [2023-12-27 13:54:50,972][20347] Updated weights for policy 0, policy_version 1192842 (0.0008) [2023-12-27 13:54:51,226][20381] Updated weights for policy 1, policy_version 1191809 (0.0007) [2023-12-27 13:54:51,293][20381] Updated weights for policy 1, policy_version 1191819 (0.0009) [2023-12-27 13:54:51,352][20381] Updated weights for policy 1, policy_version 1191829 (0.0008) [2023-12-27 13:54:51,761][20347] Updated weights for policy 0, policy_version 1192852 (0.0007) [2023-12-27 13:54:51,812][20347] Updated weights for policy 0, policy_version 1192862 (0.0007) [2023-12-27 13:54:51,862][20347] Updated weights for policy 0, policy_version 1192872 (0.0008) [2023-12-27 13:54:52,022][20381] Updated weights for policy 1, policy_version 1191839 (0.0011) [2023-12-27 13:54:52,079][20381] Updated weights for policy 1, policy_version 1191849 (0.0010) [2023-12-27 13:54:52,137][20381] Updated weights for policy 1, policy_version 1191859 (0.0011) [2023-12-27 13:54:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 610574336. Throughput: 0: 10250.7, 1: 10215.3. Samples: 610544484. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:52,363][19170] Avg episode reward: [(0, '893.070'), (1, '806.080')] [2023-12-27 13:54:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001191864_305143808.pth... [2023-12-27 13:54:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001192880_305430528.pth... [2023-12-27 13:54:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001190648_304832512.pth [2023-12-27 13:54:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001191664_305119232.pth [2023-12-27 13:54:52,627][20347] Updated weights for policy 0, policy_version 1192882 (0.0008) [2023-12-27 13:54:52,679][20347] Updated weights for policy 0, policy_version 1192892 (0.0009) [2023-12-27 13:54:52,725][20347] Updated weights for policy 0, policy_version 1192902 (0.0007) [2023-12-27 13:54:52,777][20347] Updated weights for policy 0, policy_version 1192912 (0.0007) [2023-12-27 13:54:52,811][20381] Updated weights for policy 1, policy_version 1191869 (0.0010) [2023-12-27 13:54:52,860][20381] Updated weights for policy 1, policy_version 1191879 (0.0007) [2023-12-27 13:54:52,925][20381] Updated weights for policy 1, policy_version 1191889 (0.0009) [2023-12-27 13:54:53,471][20347] Updated weights for policy 0, policy_version 1192922 (0.0005) [2023-12-27 13:54:53,486][20291] KL-divergence is very high: 115.2756 [2023-12-27 13:54:53,496][20291] KL-divergence is very high: 123.6808 [2023-12-27 13:54:53,521][20347] Updated weights for policy 0, policy_version 1192932 (0.0006) [2023-12-27 13:54:53,570][20381] Updated weights for policy 1, policy_version 1191899 (0.0011) [2023-12-27 13:54:53,580][20347] Updated weights for policy 0, policy_version 1192942 (0.0008) [2023-12-27 13:54:53,621][20381] Updated weights for policy 1, policy_version 1191909 (0.0010) [2023-12-27 13:54:53,682][20381] Updated weights for policy 1, policy_version 1191919 (0.0010) [2023-12-27 13:54:54,244][20381] Updated weights for policy 1, policy_version 1191929 (0.0008) [2023-12-27 13:54:54,275][20291] KL-divergence is very high: 147.3527 [2023-12-27 13:54:54,296][20291] KL-divergence is very high: 187.8834 [2023-12-27 13:54:54,306][20381] Updated weights for policy 1, policy_version 1191939 (0.0005) [2023-12-27 13:54:54,316][20347] Updated weights for policy 0, policy_version 1192952 (0.0007) [2023-12-27 13:54:54,327][20291] KL-divergence is very high: 175.3594 [2023-12-27 13:54:54,344][20291] KL-divergence is very high: 113.5526 [2023-12-27 13:54:54,365][20381] Updated weights for policy 1, policy_version 1191949 (0.0005) [2023-12-27 13:54:54,370][20347] Updated weights for policy 0, policy_version 1192962 (0.0007) [2023-12-27 13:54:54,420][20347] Updated weights for policy 0, policy_version 1192972 (0.0006) [2023-12-27 13:54:54,425][20381] Updated weights for policy 1, policy_version 1191959 (0.0008) [2023-12-27 13:54:54,951][20381] Updated weights for policy 1, policy_version 1191969 (0.0006) [2023-12-27 13:54:55,006][20381] Updated weights for policy 1, policy_version 1191979 (0.0006) [2023-12-27 13:54:55,054][20381] Updated weights for policy 1, policy_version 1191989 (0.0007) [2023-12-27 13:54:55,106][20347] Updated weights for policy 0, policy_version 1192982 (0.0009) [2023-12-27 13:54:55,165][20347] Updated weights for policy 0, policy_version 1192992 (0.0008) [2023-12-27 13:54:55,218][20347] Updated weights for policy 0, policy_version 1193002 (0.0005) [2023-12-27 13:54:55,688][20381] Updated weights for policy 1, policy_version 1191999 (0.0010) [2023-12-27 13:54:55,732][20381] Updated weights for policy 1, policy_version 1192009 (0.0007) [2023-12-27 13:54:55,786][20381] Updated weights for policy 1, policy_version 1192019 (0.0007) [2023-12-27 13:54:55,843][20347] Updated weights for policy 0, policy_version 1193012 (0.0005) [2023-12-27 13:54:55,894][20347] Updated weights for policy 0, policy_version 1193022 (0.0005) [2023-12-27 13:54:55,951][20347] Updated weights for policy 0, policy_version 1193032 (0.0005) [2023-12-27 13:54:56,505][20381] Updated weights for policy 1, policy_version 1192029 (0.0008) [2023-12-27 13:54:56,550][20347] Updated weights for policy 0, policy_version 1193042 (0.0006) [2023-12-27 13:54:56,550][20381] Updated weights for policy 1, policy_version 1192039 (0.0007) [2023-12-27 13:54:56,598][20381] Updated weights for policy 1, policy_version 1192049 (0.0007) [2023-12-27 13:54:56,604][20347] Updated weights for policy 0, policy_version 1193052 (0.0010) [2023-12-27 13:54:56,657][20347] Updated weights for policy 0, policy_version 1193062 (0.0010) [2023-12-27 13:54:56,707][20347] Updated weights for policy 0, policy_version 1193072 (0.0010) [2023-12-27 13:54:57,206][20381] Updated weights for policy 1, policy_version 1192059 (0.0005) [2023-12-27 13:54:57,257][20381] Updated weights for policy 1, policy_version 1192069 (0.0008) [2023-12-27 13:54:57,311][20381] Updated weights for policy 1, policy_version 1192079 (0.0006) [2023-12-27 13:54:57,354][20347] Updated weights for policy 0, policy_version 1193082 (0.0009) [2023-12-27 13:54:57,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 610672640. Throughput: 0: 10223.9, 1: 10245.6. Samples: 610668352. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:54:57,363][19170] Avg episode reward: [(0, '873.280'), (1, '795.860')] [2023-12-27 13:54:57,398][20347] Updated weights for policy 0, policy_version 1193092 (0.0009) [2023-12-27 13:54:57,443][20347] Updated weights for policy 0, policy_version 1193102 (0.0010) [2023-12-27 13:54:57,902][20381] Updated weights for policy 1, policy_version 1192089 (0.0010) [2023-12-27 13:54:57,952][20381] Updated weights for policy 1, policy_version 1192099 (0.0006) [2023-12-27 13:54:58,011][20381] Updated weights for policy 1, policy_version 1192109 (0.0007) [2023-12-27 13:54:58,067][20347] Updated weights for policy 0, policy_version 1193112 (0.0007) [2023-12-27 13:54:58,070][20381] Updated weights for policy 1, policy_version 1192119 (0.0007) [2023-12-27 13:54:58,124][20347] Updated weights for policy 0, policy_version 1193122 (0.0009) [2023-12-27 13:54:58,172][20347] Updated weights for policy 0, policy_version 1193132 (0.0008) [2023-12-27 13:54:58,753][20381] Updated weights for policy 1, policy_version 1192129 (0.0007) [2023-12-27 13:54:58,819][20381] Updated weights for policy 1, policy_version 1192139 (0.0008) [2023-12-27 13:54:58,885][20381] Updated weights for policy 1, policy_version 1192149 (0.0007) [2023-12-27 13:54:58,963][20347] Updated weights for policy 0, policy_version 1193142 (0.0008) [2023-12-27 13:54:59,021][20347] Updated weights for policy 0, policy_version 1193152 (0.0007) [2023-12-27 13:54:59,079][20347] Updated weights for policy 0, policy_version 1193162 (0.0008) [2023-12-27 13:54:59,530][20381] Updated weights for policy 1, policy_version 1192159 (0.0010) [2023-12-27 13:54:59,576][20381] Updated weights for policy 1, policy_version 1192169 (0.0010) [2023-12-27 13:54:59,625][20381] Updated weights for policy 1, policy_version 1192179 (0.0008) [2023-12-27 13:54:59,804][20347] Updated weights for policy 0, policy_version 1193172 (0.0007) [2023-12-27 13:54:59,868][20347] Updated weights for policy 0, policy_version 1193182 (0.0008) [2023-12-27 13:54:59,932][20347] Updated weights for policy 0, policy_version 1193192 (0.0007) [2023-12-27 13:55:00,312][20381] Updated weights for policy 1, policy_version 1192189 (0.0008) [2023-12-27 13:55:00,377][20381] Updated weights for policy 1, policy_version 1192199 (0.0010) [2023-12-27 13:55:00,432][20381] Updated weights for policy 1, policy_version 1192209 (0.0010) [2023-12-27 13:55:00,618][20347] Updated weights for policy 0, policy_version 1193202 (0.0008) [2023-12-27 13:55:00,659][20347] Updated weights for policy 0, policy_version 1193212 (0.0007) [2023-12-27 13:55:00,705][20347] Updated weights for policy 0, policy_version 1193222 (0.0008) [2023-12-27 13:55:00,750][20347] Updated weights for policy 0, policy_version 1193232 (0.0009) [2023-12-27 13:55:01,021][20381] Updated weights for policy 1, policy_version 1192219 (0.0010) [2023-12-27 13:55:01,082][20381] Updated weights for policy 1, policy_version 1192229 (0.0008) [2023-12-27 13:55:01,138][20381] Updated weights for policy 1, policy_version 1192239 (0.0008) [2023-12-27 13:55:01,536][20347] Updated weights for policy 0, policy_version 1193242 (0.0007) [2023-12-27 13:55:01,594][20347] Updated weights for policy 0, policy_version 1193252 (0.0008) [2023-12-27 13:55:01,661][20347] Updated weights for policy 0, policy_version 1193262 (0.0009) [2023-12-27 13:55:01,891][20381] Updated weights for policy 1, policy_version 1192249 (0.0007) [2023-12-27 13:55:01,946][20381] Updated weights for policy 1, policy_version 1192259 (0.0007) [2023-12-27 13:55:02,001][20381] Updated weights for policy 1, policy_version 1192269 (0.0007) [2023-12-27 13:55:02,060][20381] Updated weights for policy 1, policy_version 1192279 (0.0008) [2023-12-27 13:55:02,342][20347] Updated weights for policy 0, policy_version 1193272 (0.0007) [2023-12-27 13:55:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 610779136. Throughput: 0: 10206.2, 1: 10284.3. Samples: 610791552. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:02,363][19170] Avg episode reward: [(0, '853.250'), (1, '852.860')] [2023-12-27 13:55:02,401][20347] Updated weights for policy 0, policy_version 1193282 (0.0008) [2023-12-27 13:55:02,462][20347] Updated weights for policy 0, policy_version 1193292 (0.0007) [2023-12-27 13:55:02,733][20381] Updated weights for policy 1, policy_version 1192289 (0.0008) [2023-12-27 13:55:02,789][20381] Updated weights for policy 1, policy_version 1192299 (0.0007) [2023-12-27 13:55:02,847][20381] Updated weights for policy 1, policy_version 1192309 (0.0006) [2023-12-27 13:55:03,189][20347] Updated weights for policy 0, policy_version 1193302 (0.0007) [2023-12-27 13:55:03,245][20347] Updated weights for policy 0, policy_version 1193312 (0.0007) [2023-12-27 13:55:03,307][20347] Updated weights for policy 0, policy_version 1193322 (0.0009) [2023-12-27 13:55:03,475][20381] Updated weights for policy 1, policy_version 1192319 (0.0006) [2023-12-27 13:55:03,538][20381] Updated weights for policy 1, policy_version 1192329 (0.0009) [2023-12-27 13:55:03,601][20381] Updated weights for policy 1, policy_version 1192339 (0.0008) [2023-12-27 13:55:03,985][20347] Updated weights for policy 0, policy_version 1193332 (0.0007) [2023-12-27 13:55:04,034][20347] Updated weights for policy 0, policy_version 1193342 (0.0008) [2023-12-27 13:55:04,050][20291] KL-divergence is very high: 132.2334 [2023-12-27 13:55:04,066][20291] KL-divergence is very high: 128.4588 [2023-12-27 13:55:04,089][20347] Updated weights for policy 0, policy_version 1193352 (0.0007) [2023-12-27 13:55:04,209][20381] Updated weights for policy 1, policy_version 1192349 (0.0007) [2023-12-27 13:55:04,256][20381] Updated weights for policy 1, policy_version 1192359 (0.0008) [2023-12-27 13:55:04,307][20381] Updated weights for policy 1, policy_version 1192369 (0.0008) [2023-12-27 13:55:04,791][20347] Updated weights for policy 0, policy_version 1193362 (0.0007) [2023-12-27 13:55:04,848][20347] Updated weights for policy 0, policy_version 1193372 (0.0009) [2023-12-27 13:55:04,904][20347] Updated weights for policy 0, policy_version 1193382 (0.0007) [2023-12-27 13:55:04,967][20347] Updated weights for policy 0, policy_version 1193392 (0.0007) [2023-12-27 13:55:04,990][20381] Updated weights for policy 1, policy_version 1192379 (0.0008) [2023-12-27 13:55:05,043][20381] Updated weights for policy 1, policy_version 1192389 (0.0011) [2023-12-27 13:55:05,097][20381] Updated weights for policy 1, policy_version 1192399 (0.0010) [2023-12-27 13:55:05,654][20347] Updated weights for policy 0, policy_version 1193402 (0.0007) [2023-12-27 13:55:05,708][20347] Updated weights for policy 0, policy_version 1193412 (0.0008) [2023-12-27 13:55:05,759][20381] Updated weights for policy 1, policy_version 1192409 (0.0008) [2023-12-27 13:55:05,771][20347] Updated weights for policy 0, policy_version 1193422 (0.0008) [2023-12-27 13:55:05,809][20381] Updated weights for policy 1, policy_version 1192419 (0.0006) [2023-12-27 13:55:05,858][20381] Updated weights for policy 1, policy_version 1192429 (0.0005) [2023-12-27 13:55:05,907][20381] Updated weights for policy 1, policy_version 1192439 (0.0005) [2023-12-27 13:55:06,518][20347] Updated weights for policy 0, policy_version 1193432 (0.0008) [2023-12-27 13:55:06,525][20381] Updated weights for policy 1, policy_version 1192449 (0.0007) [2023-12-27 13:55:06,577][20347] Updated weights for policy 0, policy_version 1193442 (0.0009) [2023-12-27 13:55:06,579][20381] Updated weights for policy 1, policy_version 1192459 (0.0010) [2023-12-27 13:55:06,635][20347] Updated weights for policy 0, policy_version 1193452 (0.0006) [2023-12-27 13:55:06,638][20381] Updated weights for policy 1, policy_version 1192469 (0.0006) [2023-12-27 13:55:07,261][20381] Updated weights for policy 1, policy_version 1192479 (0.0007) [2023-12-27 13:55:07,318][20381] Updated weights for policy 1, policy_version 1192489 (0.0008) [2023-12-27 13:55:07,333][20347] Updated weights for policy 0, policy_version 1193462 (0.0006) [2023-12-27 13:55:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 610877440. Throughput: 0: 10196.3, 1: 10331.9. Samples: 610852496. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:07,363][20381] Updated weights for policy 1, policy_version 1192499 (0.0007) [2023-12-27 13:55:07,363][19170] Avg episode reward: [(0, '853.280'), (1, '706.930')] [2023-12-27 13:55:07,386][20347] Updated weights for policy 0, policy_version 1193472 (0.0006) [2023-12-27 13:55:07,393][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001192504_305307648.pth... [2023-12-27 13:55:07,396][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001191256_304988160.pth [2023-12-27 13:55:07,445][20347] Updated weights for policy 0, policy_version 1193482 (0.0010) [2023-12-27 13:55:07,476][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001193488_305586176.pth... [2023-12-27 13:55:07,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001192304_305283072.pth [2023-12-27 13:55:08,027][20381] Updated weights for policy 1, policy_version 1192509 (0.0006) [2023-12-27 13:55:08,048][20347] Updated weights for policy 0, policy_version 1193492 (0.0008) [2023-12-27 13:55:08,088][20381] Updated weights for policy 1, policy_version 1192519 (0.0007) [2023-12-27 13:55:08,109][20347] Updated weights for policy 0, policy_version 1193502 (0.0006) [2023-12-27 13:55:08,145][20381] Updated weights for policy 1, policy_version 1192529 (0.0008) [2023-12-27 13:55:08,171][20347] Updated weights for policy 0, policy_version 1193512 (0.0005) [2023-12-27 13:55:08,736][20347] Updated weights for policy 0, policy_version 1193522 (0.0006) [2023-12-27 13:55:08,746][20381] Updated weights for policy 1, policy_version 1192539 (0.0007) [2023-12-27 13:55:08,790][20347] Updated weights for policy 0, policy_version 1193532 (0.0010) [2023-12-27 13:55:08,808][20381] Updated weights for policy 1, policy_version 1192549 (0.0007) [2023-12-27 13:55:08,848][20347] Updated weights for policy 0, policy_version 1193542 (0.0011) [2023-12-27 13:55:08,865][20381] Updated weights for policy 1, policy_version 1192559 (0.0006) [2023-12-27 13:55:08,899][20347] Updated weights for policy 0, policy_version 1193552 (0.0011) [2023-12-27 13:55:09,539][20381] Updated weights for policy 1, policy_version 1192569 (0.0006) [2023-12-27 13:55:09,595][20381] Updated weights for policy 1, policy_version 1192579 (0.0008) [2023-12-27 13:55:09,635][20347] Updated weights for policy 0, policy_version 1193562 (0.0006) [2023-12-27 13:55:09,658][20381] Updated weights for policy 1, policy_version 1192589 (0.0007) [2023-12-27 13:55:09,662][20291] KL-divergence is very high: 119.3284 [2023-12-27 13:55:09,690][20347] Updated weights for policy 0, policy_version 1193572 (0.0008) [2023-12-27 13:55:09,720][20381] Updated weights for policy 1, policy_version 1192599 (0.0008) [2023-12-27 13:55:09,753][20347] Updated weights for policy 0, policy_version 1193582 (0.0006) [2023-12-27 13:55:10,422][20347] Updated weights for policy 0, policy_version 1193592 (0.0007) [2023-12-27 13:55:10,453][20381] Updated weights for policy 1, policy_version 1192609 (0.0006) [2023-12-27 13:55:10,476][20347] Updated weights for policy 0, policy_version 1193602 (0.0007) [2023-12-27 13:55:10,515][20381] Updated weights for policy 1, policy_version 1192619 (0.0006) [2023-12-27 13:55:10,535][20347] Updated weights for policy 0, policy_version 1193612 (0.0006) [2023-12-27 13:55:10,563][20381] Updated weights for policy 1, policy_version 1192629 (0.0005) [2023-12-27 13:55:11,209][20381] Updated weights for policy 1, policy_version 1192639 (0.0007) [2023-12-27 13:55:11,275][20381] Updated weights for policy 1, policy_version 1192649 (0.0008) [2023-12-27 13:55:11,309][20347] Updated weights for policy 0, policy_version 1193622 (0.0007) [2023-12-27 13:55:11,331][20381] Updated weights for policy 1, policy_version 1192659 (0.0008) [2023-12-27 13:55:11,375][20347] Updated weights for policy 0, policy_version 1193632 (0.0008) [2023-12-27 13:55:11,439][20347] Updated weights for policy 0, policy_version 1193642 (0.0009) [2023-12-27 13:55:12,085][20381] Updated weights for policy 1, policy_version 1192669 (0.0008) [2023-12-27 13:55:12,134][20347] Updated weights for policy 0, policy_version 1193652 (0.0007) [2023-12-27 13:55:12,154][20381] Updated weights for policy 1, policy_version 1192679 (0.0007) [2023-12-27 13:55:12,191][20347] Updated weights for policy 0, policy_version 1193662 (0.0008) [2023-12-27 13:55:12,211][20381] Updated weights for policy 1, policy_version 1192689 (0.0009) [2023-12-27 13:55:12,248][20347] Updated weights for policy 0, policy_version 1193672 (0.0006) [2023-12-27 13:55:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 610992128. Throughput: 0: 10239.4, 1: 10401.8. Samples: 610978344. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:12,363][19170] Avg episode reward: [(0, '873.100'), (1, '333.720')] [2023-12-27 13:55:12,860][20381] Updated weights for policy 1, policy_version 1192699 (0.0006) [2023-12-27 13:55:12,923][20381] Updated weights for policy 1, policy_version 1192709 (0.0008) [2023-12-27 13:55:12,944][20347] Updated weights for policy 0, policy_version 1193682 (0.0007) [2023-12-27 13:55:12,979][20381] Updated weights for policy 1, policy_version 1192719 (0.0006) [2023-12-27 13:55:13,000][20347] Updated weights for policy 0, policy_version 1193692 (0.0006) [2023-12-27 13:55:13,055][20347] Updated weights for policy 0, policy_version 1193702 (0.0007) [2023-12-27 13:55:13,103][20347] Updated weights for policy 0, policy_version 1193712 (0.0008) [2023-12-27 13:55:13,702][20381] Updated weights for policy 1, policy_version 1192729 (0.0006) [2023-12-27 13:55:13,716][20347] Updated weights for policy 0, policy_version 1193722 (0.0005) [2023-12-27 13:55:13,754][20381] Updated weights for policy 1, policy_version 1192739 (0.0007) [2023-12-27 13:55:13,769][20347] Updated weights for policy 0, policy_version 1193732 (0.0006) [2023-12-27 13:55:13,809][20381] Updated weights for policy 1, policy_version 1192749 (0.0006) [2023-12-27 13:55:13,832][20347] Updated weights for policy 0, policy_version 1193742 (0.0006) [2023-12-27 13:55:13,864][20381] Updated weights for policy 1, policy_version 1192759 (0.0008) [2023-12-27 13:55:14,481][20347] Updated weights for policy 0, policy_version 1193752 (0.0009) [2023-12-27 13:55:14,542][20347] Updated weights for policy 0, policy_version 1193762 (0.0010) [2023-12-27 13:55:14,566][20381] Updated weights for policy 1, policy_version 1192769 (0.0006) [2023-12-27 13:55:14,596][20347] Updated weights for policy 0, policy_version 1193772 (0.0010) [2023-12-27 13:55:14,624][20381] Updated weights for policy 1, policy_version 1192779 (0.0005) [2023-12-27 13:55:14,675][20381] Updated weights for policy 1, policy_version 1192789 (0.0005) [2023-12-27 13:55:15,256][20347] Updated weights for policy 0, policy_version 1193782 (0.0010) [2023-12-27 13:55:15,312][20347] Updated weights for policy 0, policy_version 1193792 (0.0010) [2023-12-27 13:55:15,344][20381] Updated weights for policy 1, policy_version 1192799 (0.0007) [2023-12-27 13:55:15,364][20347] Updated weights for policy 0, policy_version 1193802 (0.0010) [2023-12-27 13:55:15,393][20381] Updated weights for policy 1, policy_version 1192809 (0.0009) [2023-12-27 13:55:15,447][20381] Updated weights for policy 1, policy_version 1192819 (0.0008) [2023-12-27 13:55:15,972][20347] Updated weights for policy 0, policy_version 1193812 (0.0009) [2023-12-27 13:55:16,032][20347] Updated weights for policy 0, policy_version 1193822 (0.0008) [2023-12-27 13:55:16,092][20347] Updated weights for policy 0, policy_version 1193832 (0.0007) [2023-12-27 13:55:16,097][20381] Updated weights for policy 1, policy_version 1192829 (0.0006) [2023-12-27 13:55:16,150][20381] Updated weights for policy 1, policy_version 1192839 (0.0006) [2023-12-27 13:55:16,203][20381] Updated weights for policy 1, policy_version 1192849 (0.0008) [2023-12-27 13:55:16,759][20347] Updated weights for policy 0, policy_version 1193842 (0.0007) [2023-12-27 13:55:16,810][20347] Updated weights for policy 0, policy_version 1193852 (0.0005) [2023-12-27 13:55:16,838][20381] Updated weights for policy 1, policy_version 1192859 (0.0008) [2023-12-27 13:55:16,858][20347] Updated weights for policy 0, policy_version 1193862 (0.0005) [2023-12-27 13:55:16,888][20381] Updated weights for policy 1, policy_version 1192869 (0.0008) [2023-12-27 13:55:16,905][20347] Updated weights for policy 0, policy_version 1193872 (0.0007) [2023-12-27 13:55:16,942][20381] Updated weights for policy 1, policy_version 1192879 (0.0006) [2023-12-27 13:55:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 611090432. Throughput: 0: 10216.3, 1: 10360.9. Samples: 611101740. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:17,363][19170] Avg episode reward: [(0, '805.050'), (1, '523.530')] [2023-12-27 13:55:17,514][20347] Updated weights for policy 0, policy_version 1193882 (0.0010) [2023-12-27 13:55:17,559][20347] Updated weights for policy 0, policy_version 1193892 (0.0010) [2023-12-27 13:55:17,598][20381] Updated weights for policy 1, policy_version 1192889 (0.0007) [2023-12-27 13:55:17,604][20347] Updated weights for policy 0, policy_version 1193902 (0.0010) [2023-12-27 13:55:17,648][20381] Updated weights for policy 1, policy_version 1192899 (0.0007) [2023-12-27 13:55:17,693][20381] Updated weights for policy 1, policy_version 1192909 (0.0008) [2023-12-27 13:55:17,746][20381] Updated weights for policy 1, policy_version 1192919 (0.0008) [2023-12-27 13:55:18,316][20347] Updated weights for policy 0, policy_version 1193912 (0.0010) [2023-12-27 13:55:18,406][20347] Updated weights for policy 0, policy_version 1193922 (0.0006) [2023-12-27 13:55:18,462][20347] Updated weights for policy 0, policy_version 1193932 (0.0006) [2023-12-27 13:55:18,477][20381] Updated weights for policy 1, policy_version 1192929 (0.0007) [2023-12-27 13:55:18,523][20381] Updated weights for policy 1, policy_version 1192939 (0.0007) [2023-12-27 13:55:18,571][20381] Updated weights for policy 1, policy_version 1192949 (0.0009) [2023-12-27 13:55:19,049][20347] Updated weights for policy 0, policy_version 1193942 (0.0007) [2023-12-27 13:55:19,101][20347] Updated weights for policy 0, policy_version 1193952 (0.0006) [2023-12-27 13:55:19,163][20347] Updated weights for policy 0, policy_version 1193962 (0.0006) [2023-12-27 13:55:19,295][20381] Updated weights for policy 1, policy_version 1192959 (0.0010) [2023-12-27 13:55:19,356][20381] Updated weights for policy 1, policy_version 1192969 (0.0010) [2023-12-27 13:55:19,427][20381] Updated weights for policy 1, policy_version 1192979 (0.0008) [2023-12-27 13:55:19,865][20291] KL-divergence is very high: 110.0516 [2023-12-27 13:55:19,889][20347] Updated weights for policy 0, policy_version 1193972 (0.0007) [2023-12-27 13:55:19,896][20291] KL-divergence is very high: 119.2833 [2023-12-27 13:55:19,917][20291] KL-divergence is very high: 111.8857 [2023-12-27 13:55:19,956][20347] Updated weights for policy 0, policy_version 1193982 (0.0007) [2023-12-27 13:55:20,022][20347] Updated weights for policy 0, policy_version 1193992 (0.0008) [2023-12-27 13:55:20,024][20291] KL-divergence is very high: 162.5789 [2023-12-27 13:55:20,035][20291] KL-divergence is very high: 129.8626 [2023-12-27 13:55:20,054][20291] KL-divergence is very high: 240.8860 [2023-12-27 13:55:20,162][20381] Updated weights for policy 1, policy_version 1192989 (0.0009) [2023-12-27 13:55:20,219][20381] Updated weights for policy 1, policy_version 1192999 (0.0008) [2023-12-27 13:55:20,283][20381] Updated weights for policy 1, policy_version 1193009 (0.0007) [2023-12-27 13:55:20,701][20347] Updated weights for policy 0, policy_version 1194002 (0.0008) [2023-12-27 13:55:20,759][20347] Updated weights for policy 0, policy_version 1194012 (0.0008) [2023-12-27 13:55:20,817][20347] Updated weights for policy 0, policy_version 1194022 (0.0008) [2023-12-27 13:55:20,866][20347] Updated weights for policy 0, policy_version 1194032 (0.0006) [2023-12-27 13:55:21,006][20381] Updated weights for policy 1, policy_version 1193019 (0.0007) [2023-12-27 13:55:21,072][20381] Updated weights for policy 1, policy_version 1193029 (0.0008) [2023-12-27 13:55:21,134][20381] Updated weights for policy 1, policy_version 1193039 (0.0008) [2023-12-27 13:55:21,615][20347] Updated weights for policy 0, policy_version 1194042 (0.0006) [2023-12-27 13:55:21,672][20347] Updated weights for policy 0, policy_version 1194052 (0.0006) [2023-12-27 13:55:21,740][20347] Updated weights for policy 0, policy_version 1194062 (0.0008) [2023-12-27 13:55:21,857][20381] Updated weights for policy 1, policy_version 1193049 (0.0008) [2023-12-27 13:55:21,908][20381] Updated weights for policy 1, policy_version 1193059 (0.0008) [2023-12-27 13:55:21,965][20381] Updated weights for policy 1, policy_version 1193069 (0.0008) [2023-12-27 13:55:22,027][20381] Updated weights for policy 1, policy_version 1193079 (0.0008) [2023-12-27 13:55:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 611188736. Throughput: 0: 10201.5, 1: 10332.7. Samples: 611161924. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:22,363][19170] Avg episode reward: [(0, '765.410'), (1, '654.310')] [2023-12-27 13:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001193080_305455104.pth... [2023-12-27 13:55:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001191864_305143808.pth [2023-12-27 13:55:22,420][20347] Updated weights for policy 0, policy_version 1194072 (0.0008) [2023-12-27 13:55:22,476][20347] Updated weights for policy 0, policy_version 1194082 (0.0008) [2023-12-27 13:55:22,537][20347] Updated weights for policy 0, policy_version 1194092 (0.0007) [2023-12-27 13:55:22,559][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001194096_305741824.pth... [2023-12-27 13:55:22,563][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001192880_305430528.pth [2023-12-27 13:55:22,810][20381] Updated weights for policy 1, policy_version 1193089 (0.0007) [2023-12-27 13:55:22,867][20381] Updated weights for policy 1, policy_version 1193099 (0.0008) [2023-12-27 13:55:22,919][20381] Updated weights for policy 1, policy_version 1193109 (0.0007) [2023-12-27 13:55:23,252][20347] Updated weights for policy 0, policy_version 1194102 (0.0007) [2023-12-27 13:55:23,301][20347] Updated weights for policy 0, policy_version 1194112 (0.0007) [2023-12-27 13:55:23,350][20347] Updated weights for policy 0, policy_version 1194122 (0.0007) [2023-12-27 13:55:23,623][20381] Updated weights for policy 1, policy_version 1193119 (0.0010) [2023-12-27 13:55:23,683][20381] Updated weights for policy 1, policy_version 1193129 (0.0010) [2023-12-27 13:55:23,735][20381] Updated weights for policy 1, policy_version 1193139 (0.0011) [2023-12-27 13:55:23,994][20347] Updated weights for policy 0, policy_version 1194132 (0.0007) [2023-12-27 13:55:24,044][20347] Updated weights for policy 0, policy_version 1194142 (0.0007) [2023-12-27 13:55:24,100][20347] Updated weights for policy 0, policy_version 1194152 (0.0007) [2023-12-27 13:55:24,432][20381] Updated weights for policy 1, policy_version 1193149 (0.0008) [2023-12-27 13:55:24,494][20381] Updated weights for policy 1, policy_version 1193159 (0.0010) [2023-12-27 13:55:24,556][20381] Updated weights for policy 1, policy_version 1193169 (0.0011) [2023-12-27 13:55:24,782][20347] Updated weights for policy 0, policy_version 1194162 (0.0008) [2023-12-27 13:55:24,802][20291] KL-divergence is very high: 100.7481 [2023-12-27 13:55:24,824][20291] KL-divergence is very high: 232.5392 [2023-12-27 13:55:24,830][20291] KL-divergence is very high: 255.0361 [2023-12-27 13:55:24,836][20291] KL-divergence is very high: 288.5809 [2023-12-27 13:55:24,841][20347] Updated weights for policy 0, policy_version 1194172 (0.0010) [2023-12-27 13:55:24,848][20291] KL-divergence is very high: 275.9933 [2023-12-27 13:55:24,869][20291] KL-divergence is very high: 231.4288 [2023-12-27 13:55:24,874][20291] KL-divergence is very high: 211.8392 [2023-12-27 13:55:24,880][20291] KL-divergence is very high: 208.3963 [2023-12-27 13:55:24,889][20291] KL-divergence is very high: 145.7218 [2023-12-27 13:55:24,896][20347] Updated weights for policy 0, policy_version 1194182 (0.0007) [2023-12-27 13:55:24,951][20347] Updated weights for policy 0, policy_version 1194192 (0.0010) [2023-12-27 13:55:25,183][20381] Updated weights for policy 1, policy_version 1193179 (0.0009) [2023-12-27 13:55:25,242][20381] Updated weights for policy 1, policy_version 1193189 (0.0011) [2023-12-27 13:55:25,303][20381] Updated weights for policy 1, policy_version 1193199 (0.0008) [2023-12-27 13:55:25,621][20347] Updated weights for policy 0, policy_version 1194202 (0.0010) [2023-12-27 13:55:25,678][20347] Updated weights for policy 0, policy_version 1194212 (0.0010) [2023-12-27 13:55:25,736][20347] Updated weights for policy 0, policy_version 1194222 (0.0010) [2023-12-27 13:55:25,872][20381] Updated weights for policy 1, policy_version 1193209 (0.0005) [2023-12-27 13:55:25,931][20381] Updated weights for policy 1, policy_version 1193219 (0.0010) [2023-12-27 13:55:25,983][20381] Updated weights for policy 1, policy_version 1193229 (0.0010) [2023-12-27 13:55:26,038][20381] Updated weights for policy 1, policy_version 1193239 (0.0011) [2023-12-27 13:55:26,393][20347] Updated weights for policy 0, policy_version 1194232 (0.0007) [2023-12-27 13:55:26,444][20291] KL-divergence is very high: 115.7399 [2023-12-27 13:55:26,449][20347] Updated weights for policy 0, policy_version 1194242 (0.0007) [2023-12-27 13:55:26,501][20347] Updated weights for policy 0, policy_version 1194252 (0.0005) [2023-12-27 13:55:26,673][20381] Updated weights for policy 1, policy_version 1193249 (0.0010) [2023-12-27 13:55:26,727][20381] Updated weights for policy 1, policy_version 1193259 (0.0010) [2023-12-27 13:55:26,787][20381] Updated weights for policy 1, policy_version 1193269 (0.0010) [2023-12-27 13:55:27,011][20291] KL-divergence is very high: 102.6263 [2023-12-27 13:55:27,029][20347] Updated weights for policy 0, policy_version 1194262 (0.0006) [2023-12-27 13:55:27,056][20291] KL-divergence is very high: 112.4505 [2023-12-27 13:55:27,083][20347] Updated weights for policy 0, policy_version 1194272 (0.0007) [2023-12-27 13:55:27,129][20347] Updated weights for policy 0, policy_version 1194282 (0.0005) [2023-12-27 13:55:27,132][20291] KL-divergence is very high: 115.5776 [2023-12-27 13:55:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 611295232. Throughput: 0: 10220.4, 1: 10266.1. Samples: 611282612. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:27,363][19170] Avg episode reward: [(0, '804.880'), (1, '553.350')] [2023-12-27 13:55:27,395][20381] Updated weights for policy 1, policy_version 1193279 (0.0010) [2023-12-27 13:55:27,441][20381] Updated weights for policy 1, policy_version 1193289 (0.0009) [2023-12-27 13:55:27,494][20381] Updated weights for policy 1, policy_version 1193299 (0.0008) [2023-12-27 13:55:27,682][20347] Updated weights for policy 0, policy_version 1194292 (0.0005) [2023-12-27 13:55:27,749][20347] Updated weights for policy 0, policy_version 1194302 (0.0005) [2023-12-27 13:55:27,805][20347] Updated weights for policy 0, policy_version 1194312 (0.0006) [2023-12-27 13:55:28,241][20381] Updated weights for policy 1, policy_version 1193309 (0.0007) [2023-12-27 13:55:28,299][20381] Updated weights for policy 1, policy_version 1193319 (0.0008) [2023-12-27 13:55:28,360][20381] Updated weights for policy 1, policy_version 1193329 (0.0008) [2023-12-27 13:55:28,415][20347] Updated weights for policy 0, policy_version 1194322 (0.0007) [2023-12-27 13:55:28,472][20347] Updated weights for policy 0, policy_version 1194332 (0.0006) [2023-12-27 13:55:28,531][20347] Updated weights for policy 0, policy_version 1194342 (0.0005) [2023-12-27 13:55:28,584][20347] Updated weights for policy 0, policy_version 1194352 (0.0007) [2023-12-27 13:55:29,092][20381] Updated weights for policy 1, policy_version 1193339 (0.0007) [2023-12-27 13:55:29,143][20381] Updated weights for policy 1, policy_version 1193349 (0.0008) [2023-12-27 13:55:29,203][20381] Updated weights for policy 1, policy_version 1193359 (0.0006) [2023-12-27 13:55:29,221][20347] Updated weights for policy 0, policy_version 1194362 (0.0006) [2023-12-27 13:55:29,284][20347] Updated weights for policy 0, policy_version 1194372 (0.0007) [2023-12-27 13:55:29,344][20347] Updated weights for policy 0, policy_version 1194382 (0.0007) [2023-12-27 13:55:29,944][20381] Updated weights for policy 1, policy_version 1193369 (0.0007) [2023-12-27 13:55:30,002][20381] Updated weights for policy 1, policy_version 1193379 (0.0007) [2023-12-27 13:55:30,026][20347] Updated weights for policy 0, policy_version 1194392 (0.0008) [2023-12-27 13:55:30,056][20381] Updated weights for policy 1, policy_version 1193389 (0.0006) [2023-12-27 13:55:30,080][20347] Updated weights for policy 0, policy_version 1194402 (0.0006) [2023-12-27 13:55:30,110][20381] Updated weights for policy 1, policy_version 1193399 (0.0007) [2023-12-27 13:55:30,139][20347] Updated weights for policy 0, policy_version 1194412 (0.0008) [2023-12-27 13:55:30,754][20347] Updated weights for policy 0, policy_version 1194422 (0.0006) [2023-12-27 13:55:30,801][20347] Updated weights for policy 0, policy_version 1194432 (0.0007) [2023-12-27 13:55:30,808][20381] Updated weights for policy 1, policy_version 1193409 (0.0006) [2023-12-27 13:55:30,846][20347] Updated weights for policy 0, policy_version 1194442 (0.0007) [2023-12-27 13:55:30,863][20381] Updated weights for policy 1, policy_version 1193419 (0.0005) [2023-12-27 13:55:30,916][20381] Updated weights for policy 1, policy_version 1193429 (0.0005) [2023-12-27 13:55:31,515][20381] Updated weights for policy 1, policy_version 1193439 (0.0006) [2023-12-27 13:55:31,568][20381] Updated weights for policy 1, policy_version 1193449 (0.0006) [2023-12-27 13:55:31,631][20381] Updated weights for policy 1, policy_version 1193459 (0.0008) [2023-12-27 13:55:31,641][20347] Updated weights for policy 0, policy_version 1194452 (0.0008) [2023-12-27 13:55:31,700][20347] Updated weights for policy 0, policy_version 1194462 (0.0007) [2023-12-27 13:55:31,762][20347] Updated weights for policy 0, policy_version 1194472 (0.0008) [2023-12-27 13:55:32,328][20381] Updated weights for policy 1, policy_version 1193469 (0.0007) [2023-12-27 13:55:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 611393536. Throughput: 0: 10217.1, 1: 10299.2. Samples: 611408492. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:32,363][19170] Avg episode reward: [(0, '814.670'), (1, '126.300')] [2023-12-27 13:55:32,389][20381] Updated weights for policy 1, policy_version 1193479 (0.0008) [2023-12-27 13:55:32,445][20381] Updated weights for policy 1, policy_version 1193489 (0.0007) [2023-12-27 13:55:32,475][20347] Updated weights for policy 0, policy_version 1194482 (0.0008) [2023-12-27 13:55:32,532][20347] Updated weights for policy 0, policy_version 1194492 (0.0008) [2023-12-27 13:55:32,588][20347] Updated weights for policy 0, policy_version 1194502 (0.0007) [2023-12-27 13:55:32,637][20347] Updated weights for policy 0, policy_version 1194512 (0.0009) [2023-12-27 13:55:33,112][20381] Updated weights for policy 1, policy_version 1193499 (0.0008) [2023-12-27 13:55:33,169][20381] Updated weights for policy 1, policy_version 1193509 (0.0006) [2023-12-27 13:55:33,225][20381] Updated weights for policy 1, policy_version 1193519 (0.0005) [2023-12-27 13:55:33,322][20347] Updated weights for policy 0, policy_version 1194522 (0.0010) [2023-12-27 13:55:33,388][20347] Updated weights for policy 0, policy_version 1194532 (0.0011) [2023-12-27 13:55:33,440][20347] Updated weights for policy 0, policy_version 1194542 (0.0011) [2023-12-27 13:55:33,869][20381] Updated weights for policy 1, policy_version 1193529 (0.0006) [2023-12-27 13:55:33,926][20381] Updated weights for policy 1, policy_version 1193539 (0.0007) [2023-12-27 13:55:33,978][20381] Updated weights for policy 1, policy_version 1193549 (0.0007) [2023-12-27 13:55:34,032][20381] Updated weights for policy 1, policy_version 1193559 (0.0007) [2023-12-27 13:55:34,131][20347] Updated weights for policy 0, policy_version 1194552 (0.0010) [2023-12-27 13:55:34,191][20347] Updated weights for policy 0, policy_version 1194562 (0.0007) [2023-12-27 13:55:34,247][20347] Updated weights for policy 0, policy_version 1194572 (0.0010) [2023-12-27 13:55:34,802][20381] Updated weights for policy 1, policy_version 1193569 (0.0007) [2023-12-27 13:55:34,819][20347] Updated weights for policy 0, policy_version 1194582 (0.0008) [2023-12-27 13:55:34,846][20381] Updated weights for policy 1, policy_version 1193579 (0.0007) [2023-12-27 13:55:34,876][20347] Updated weights for policy 0, policy_version 1194592 (0.0006) [2023-12-27 13:55:34,889][20381] Updated weights for policy 1, policy_version 1193589 (0.0006) [2023-12-27 13:55:34,922][20347] Updated weights for policy 0, policy_version 1194602 (0.0008) [2023-12-27 13:55:35,586][20347] Updated weights for policy 0, policy_version 1194612 (0.0007) [2023-12-27 13:55:35,588][20381] Updated weights for policy 1, policy_version 1193599 (0.0005) [2023-12-27 13:55:35,641][20347] Updated weights for policy 0, policy_version 1194622 (0.0007) [2023-12-27 13:55:35,643][20381] Updated weights for policy 1, policy_version 1193609 (0.0006) [2023-12-27 13:55:35,700][20381] Updated weights for policy 1, policy_version 1193619 (0.0006) [2023-12-27 13:55:35,702][20347] Updated weights for policy 0, policy_version 1194632 (0.0006) [2023-12-27 13:55:36,354][20347] Updated weights for policy 0, policy_version 1194642 (0.0006) [2023-12-27 13:55:36,409][20347] Updated weights for policy 0, policy_version 1194652 (0.0006) [2023-12-27 13:55:36,462][20381] Updated weights for policy 1, policy_version 1193629 (0.0007) [2023-12-27 13:55:36,468][20347] Updated weights for policy 0, policy_version 1194662 (0.0007) [2023-12-27 13:55:36,525][20347] Updated weights for policy 0, policy_version 1194672 (0.0007) [2023-12-27 13:55:36,533][20381] Updated weights for policy 1, policy_version 1193639 (0.0007) [2023-12-27 13:55:36,598][20381] Updated weights for policy 1, policy_version 1193649 (0.0008) [2023-12-27 13:55:37,200][20347] Updated weights for policy 0, policy_version 1194682 (0.0005) [2023-12-27 13:55:37,257][20347] Updated weights for policy 0, policy_version 1194692 (0.0005) [2023-12-27 13:55:37,298][20381] Updated weights for policy 1, policy_version 1193659 (0.0007) [2023-12-27 13:55:37,313][20347] Updated weights for policy 0, policy_version 1194702 (0.0005) [2023-12-27 13:55:37,356][20381] Updated weights for policy 1, policy_version 1193669 (0.0007) [2023-12-27 13:55:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 611500032. Throughput: 0: 10244.8, 1: 10319.2. Samples: 611469864. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-12-27 13:55:37,363][19170] Avg episode reward: [(0, '892.480'), (1, '181.920')] [2023-12-27 13:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001194704_305897472.pth... [2023-12-27 13:55:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001193488_305586176.pth [2023-12-27 13:55:37,413][20381] Updated weights for policy 1, policy_version 1193679 (0.0007) [2023-12-27 13:55:37,459][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001193688_305610752.pth... [2023-12-27 13:55:37,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001192504_305307648.pth [2023-12-27 13:55:37,883][20347] Updated weights for policy 0, policy_version 1194712 (0.0009) [2023-12-27 13:55:37,936][20347] Updated weights for policy 0, policy_version 1194722 (0.0009) [2023-12-27 13:55:37,994][20347] Updated weights for policy 0, policy_version 1194732 (0.0005) [2023-12-27 13:55:38,140][20381] Updated weights for policy 1, policy_version 1193689 (0.0006) [2023-12-27 13:55:38,199][20381] Updated weights for policy 1, policy_version 1193699 (0.0008) [2023-12-27 13:55:38,255][20381] Updated weights for policy 1, policy_version 1193709 (0.0009) [2023-12-27 13:55:38,304][20381] Updated weights for policy 1, policy_version 1193719 (0.0007) [2023-12-27 13:55:38,602][20347] Updated weights for policy 0, policy_version 1194742 (0.0009) [2023-12-27 13:55:38,660][20347] Updated weights for policy 0, policy_version 1194752 (0.0010) [2023-12-27 13:55:38,710][20347] Updated weights for policy 0, policy_version 1194762 (0.0010) [2023-12-27 13:55:38,997][20381] Updated weights for policy 1, policy_version 1193729 (0.0007) [2023-12-27 13:55:39,048][20381] Updated weights for policy 1, policy_version 1193739 (0.0007) [2023-12-27 13:55:39,102][20381] Updated weights for policy 1, policy_version 1193749 (0.0007) [2023-12-27 13:55:39,448][20347] Updated weights for policy 0, policy_version 1194772 (0.0009) [2023-12-27 13:55:39,507][20347] Updated weights for policy 0, policy_version 1194782 (0.0010) [2023-12-27 13:55:39,567][20347] Updated weights for policy 0, policy_version 1194792 (0.0011) [2023-12-27 13:55:39,787][20381] Updated weights for policy 1, policy_version 1193759 (0.0005) [2023-12-27 13:55:39,854][20381] Updated weights for policy 1, policy_version 1193769 (0.0007) [2023-12-27 13:55:39,919][20381] Updated weights for policy 1, policy_version 1193779 (0.0007) [2023-12-27 13:55:40,277][20347] Updated weights for policy 0, policy_version 1194802 (0.0010) [2023-12-27 13:55:40,343][20347] Updated weights for policy 0, policy_version 1194812 (0.0011) [2023-12-27 13:55:40,402][20347] Updated weights for policy 0, policy_version 1194822 (0.0011) [2023-12-27 13:55:40,469][20347] Updated weights for policy 0, policy_version 1194832 (0.0011) [2023-12-27 13:55:40,559][20381] Updated weights for policy 1, policy_version 1193789 (0.0009) [2023-12-27 13:55:40,645][20381] Updated weights for policy 1, policy_version 1193799 (0.0010) [2023-12-27 13:55:40,698][20381] Updated weights for policy 1, policy_version 1193809 (0.0009) [2023-12-27 13:55:41,212][20347] Updated weights for policy 0, policy_version 1194842 (0.0009) [2023-12-27 13:55:41,274][20347] Updated weights for policy 0, policy_version 1194852 (0.0008) [2023-12-27 13:55:41,335][20347] Updated weights for policy 0, policy_version 1194862 (0.0009) [2023-12-27 13:55:41,433][20381] Updated weights for policy 1, policy_version 1193819 (0.0008) [2023-12-27 13:55:41,492][20381] Updated weights for policy 1, policy_version 1193829 (0.0005) [2023-12-27 13:55:41,553][20381] Updated weights for policy 1, policy_version 1193839 (0.0006) [2023-12-27 13:55:42,086][20347] Updated weights for policy 0, policy_version 1194872 (0.0008) [2023-12-27 13:55:42,140][20347] Updated weights for policy 0, policy_version 1194882 (0.0010) [2023-12-27 13:55:42,175][20381] Updated weights for policy 1, policy_version 1193849 (0.0007) [2023-12-27 13:55:42,197][20347] Updated weights for policy 0, policy_version 1194892 (0.0007) [2023-12-27 13:55:42,241][20381] Updated weights for policy 1, policy_version 1193859 (0.0008) [2023-12-27 13:55:42,302][20381] Updated weights for policy 1, policy_version 1193869 (0.0008) [2023-12-27 13:55:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 611598336. Throughput: 0: 10298.1, 1: 10207.7. Samples: 611591112. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:55:42,363][19170] Avg episode reward: [(0, '931.600'), (1, '612.090')] [2023-12-27 13:55:42,364][20381] Updated weights for policy 1, policy_version 1193879 (0.0009) [2023-12-27 13:55:42,845][20347] Updated weights for policy 0, policy_version 1194902 (0.0008) [2023-12-27 13:55:42,906][20347] Updated weights for policy 0, policy_version 1194912 (0.0011) [2023-12-27 13:55:42,961][20347] Updated weights for policy 0, policy_version 1194922 (0.0010) [2023-12-27 13:55:43,026][20381] Updated weights for policy 1, policy_version 1193889 (0.0010) [2023-12-27 13:55:43,074][20381] Updated weights for policy 1, policy_version 1193899 (0.0010) [2023-12-27 13:55:43,125][20381] Updated weights for policy 1, policy_version 1193909 (0.0010) [2023-12-27 13:55:43,580][20347] Updated weights for policy 0, policy_version 1194932 (0.0008) [2023-12-27 13:55:43,635][20347] Updated weights for policy 0, policy_version 1194942 (0.0006) [2023-12-27 13:55:43,693][20347] Updated weights for policy 0, policy_version 1194952 (0.0007) [2023-12-27 13:55:43,818][20381] Updated weights for policy 1, policy_version 1193919 (0.0010) [2023-12-27 13:55:43,875][20381] Updated weights for policy 1, policy_version 1193929 (0.0010) [2023-12-27 13:55:43,932][20381] Updated weights for policy 1, policy_version 1193939 (0.0010) [2023-12-27 13:55:44,265][20347] Updated weights for policy 0, policy_version 1194962 (0.0006) [2023-12-27 13:55:44,317][20347] Updated weights for policy 0, policy_version 1194972 (0.0005) [2023-12-27 13:55:44,364][20347] Updated weights for policy 0, policy_version 1194982 (0.0005) [2023-12-27 13:55:44,418][20347] Updated weights for policy 0, policy_version 1194992 (0.0005) [2023-12-27 13:55:44,603][20381] Updated weights for policy 1, policy_version 1193949 (0.0008) [2023-12-27 13:55:44,651][20381] Updated weights for policy 1, policy_version 1193959 (0.0010) [2023-12-27 13:55:44,700][20381] Updated weights for policy 1, policy_version 1193969 (0.0010) [2023-12-27 13:55:44,998][20347] Updated weights for policy 0, policy_version 1195002 (0.0010) [2023-12-27 13:55:45,069][20347] Updated weights for policy 0, policy_version 1195012 (0.0011) [2023-12-27 13:55:45,138][20347] Updated weights for policy 0, policy_version 1195022 (0.0010) [2023-12-27 13:55:45,348][20381] Updated weights for policy 1, policy_version 1193979 (0.0011) [2023-12-27 13:55:45,400][20381] Updated weights for policy 1, policy_version 1193989 (0.0010) [2023-12-27 13:55:45,458][20381] Updated weights for policy 1, policy_version 1193999 (0.0010) [2023-12-27 13:55:45,707][20347] Updated weights for policy 0, policy_version 1195032 (0.0010) [2023-12-27 13:55:45,770][20347] Updated weights for policy 0, policy_version 1195042 (0.0010) [2023-12-27 13:55:45,822][20347] Updated weights for policy 0, policy_version 1195052 (0.0007) [2023-12-27 13:55:46,020][20381] Updated weights for policy 1, policy_version 1194009 (0.0009) [2023-12-27 13:55:46,081][20381] Updated weights for policy 1, policy_version 1194019 (0.0005) [2023-12-27 13:55:46,141][20381] Updated weights for policy 1, policy_version 1194029 (0.0009) [2023-12-27 13:55:46,201][20381] Updated weights for policy 1, policy_version 1194039 (0.0010) [2023-12-27 13:55:46,544][20347] Updated weights for policy 0, policy_version 1195062 (0.0008) [2023-12-27 13:55:46,589][20347] Updated weights for policy 0, policy_version 1195072 (0.0010) [2023-12-27 13:55:46,635][20347] Updated weights for policy 0, policy_version 1195082 (0.0010) [2023-12-27 13:55:46,731][20381] Updated weights for policy 1, policy_version 1194049 (0.0010) [2023-12-27 13:55:46,775][20381] Updated weights for policy 1, policy_version 1194059 (0.0010) [2023-12-27 13:55:46,827][20381] Updated weights for policy 1, policy_version 1194069 (0.0010) [2023-12-27 13:55:47,320][20347] Updated weights for policy 0, policy_version 1195092 (0.0010) [2023-12-27 13:55:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 611704832. Throughput: 0: 10375.4, 1: 10242.5. Samples: 611719356. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:55:47,363][19170] Avg episode reward: [(0, '970.830'), (1, '660.430')] [2023-12-27 13:55:47,365][20347] Updated weights for policy 0, policy_version 1195102 (0.0010) [2023-12-27 13:55:47,409][20347] Updated weights for policy 0, policy_version 1195112 (0.0010) [2023-12-27 13:55:47,536][20381] Updated weights for policy 1, policy_version 1194079 (0.0010) [2023-12-27 13:55:47,590][20381] Updated weights for policy 1, policy_version 1194089 (0.0010) [2023-12-27 13:55:47,648][20381] Updated weights for policy 1, policy_version 1194099 (0.0010) [2023-12-27 13:55:48,022][20347] Updated weights for policy 0, policy_version 1195122 (0.0010) [2023-12-27 13:55:48,078][20347] Updated weights for policy 0, policy_version 1195132 (0.0010) [2023-12-27 13:55:48,133][20347] Updated weights for policy 0, policy_version 1195142 (0.0010) [2023-12-27 13:55:48,184][20347] Updated weights for policy 0, policy_version 1195152 (0.0010) [2023-12-27 13:55:48,335][20381] Updated weights for policy 1, policy_version 1194109 (0.0011) [2023-12-27 13:55:48,397][20381] Updated weights for policy 1, policy_version 1194119 (0.0011) [2023-12-27 13:55:48,453][20381] Updated weights for policy 1, policy_version 1194129 (0.0010) [2023-12-27 13:55:48,802][20347] Updated weights for policy 0, policy_version 1195162 (0.0011) [2023-12-27 13:55:48,864][20347] Updated weights for policy 0, policy_version 1195172 (0.0010) [2023-12-27 13:55:48,926][20347] Updated weights for policy 0, policy_version 1195182 (0.0010) [2023-12-27 13:55:49,154][20381] Updated weights for policy 1, policy_version 1194139 (0.0011) [2023-12-27 13:55:49,206][20381] Updated weights for policy 1, policy_version 1194149 (0.0010) [2023-12-27 13:55:49,252][20292] KL-divergence is very high: 112.9523 [2023-12-27 13:55:49,259][20292] KL-divergence is very high: 125.4069 [2023-12-27 13:55:49,273][20292] KL-divergence is very high: 111.0478 [2023-12-27 13:55:49,282][20381] Updated weights for policy 1, policy_version 1194159 (0.0011) [2023-12-27 13:55:49,621][20347] Updated weights for policy 0, policy_version 1195192 (0.0011) [2023-12-27 13:55:49,676][20347] Updated weights for policy 0, policy_version 1195202 (0.0010) [2023-12-27 13:55:49,741][20347] Updated weights for policy 0, policy_version 1195212 (0.0010) [2023-12-27 13:55:50,030][20381] Updated weights for policy 1, policy_version 1194169 (0.0009) [2023-12-27 13:55:50,066][20292] KL-divergence is very high: 241.8553 [2023-12-27 13:55:50,072][20292] KL-divergence is very high: 115.4100 [2023-12-27 13:55:50,088][20381] Updated weights for policy 1, policy_version 1194179 (0.0008) [2023-12-27 13:55:50,115][20292] KL-divergence is very high: 172.9761 [2023-12-27 13:55:50,150][20381] Updated weights for policy 1, policy_version 1194189 (0.0008) [2023-12-27 13:55:50,160][20292] KL-divergence is very high: 101.1048 [2023-12-27 13:55:50,205][20381] Updated weights for policy 1, policy_version 1194199 (0.0008) [2023-12-27 13:55:50,432][20347] Updated weights for policy 0, policy_version 1195222 (0.0008) [2023-12-27 13:55:50,487][20347] Updated weights for policy 0, policy_version 1195232 (0.0008) [2023-12-27 13:55:50,546][20347] Updated weights for policy 0, policy_version 1195242 (0.0006) [2023-12-27 13:55:50,857][20381] Updated weights for policy 1, policy_version 1194209 (0.0008) [2023-12-27 13:55:50,919][20381] Updated weights for policy 1, policy_version 1194219 (0.0007) [2023-12-27 13:55:50,984][20381] Updated weights for policy 1, policy_version 1194229 (0.0007) [2023-12-27 13:55:51,306][20347] Updated weights for policy 0, policy_version 1195252 (0.0007) [2023-12-27 13:55:51,373][20347] Updated weights for policy 0, policy_version 1195262 (0.0007) [2023-12-27 13:55:51,411][20291] KL-divergence is very high: 135.3824 [2023-12-27 13:55:51,441][20347] Updated weights for policy 0, policy_version 1195272 (0.0009) [2023-12-27 13:55:51,664][20381] Updated weights for policy 1, policy_version 1194239 (0.0008) [2023-12-27 13:55:51,727][20381] Updated weights for policy 1, policy_version 1194249 (0.0009) [2023-12-27 13:55:51,789][20381] Updated weights for policy 1, policy_version 1194259 (0.0006) [2023-12-27 13:55:52,229][20347] Updated weights for policy 0, policy_version 1195282 (0.0008) [2023-12-27 13:55:52,300][20347] Updated weights for policy 0, policy_version 1195292 (0.0009) [2023-12-27 13:55:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 611803136. Throughput: 0: 10429.7, 1: 10191.1. Samples: 611780432. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:55:52,363][19170] Avg episode reward: [(0, '961.290'), (1, '654.120')] [2023-12-27 13:55:52,365][20347] Updated weights for policy 0, policy_version 1195302 (0.0008) [2023-12-27 13:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001194264_305758208.pth... [2023-12-27 13:55:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001193080_305455104.pth [2023-12-27 13:55:52,429][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001195312_306053120.pth... [2023-12-27 13:55:52,430][20381] Updated weights for policy 1, policy_version 1194269 (0.0007) [2023-12-27 13:55:52,430][20347] Updated weights for policy 0, policy_version 1195312 (0.0008) [2023-12-27 13:55:52,434][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001194096_305741824.pth [2023-12-27 13:55:52,486][20381] Updated weights for policy 1, policy_version 1194279 (0.0008) [2023-12-27 13:55:52,533][20381] Updated weights for policy 1, policy_version 1194289 (0.0008) [2023-12-27 13:55:53,099][20347] Updated weights for policy 0, policy_version 1195322 (0.0010) [2023-12-27 13:55:53,153][20347] Updated weights for policy 0, policy_version 1195332 (0.0009) [2023-12-27 13:55:53,211][20347] Updated weights for policy 0, policy_version 1195342 (0.0010) [2023-12-27 13:55:53,225][20381] Updated weights for policy 1, policy_version 1194299 (0.0007) [2023-12-27 13:55:53,274][20381] Updated weights for policy 1, policy_version 1194309 (0.0005) [2023-12-27 13:55:53,320][20381] Updated weights for policy 1, policy_version 1194319 (0.0005) [2023-12-27 13:55:53,880][20347] Updated weights for policy 0, policy_version 1195352 (0.0009) [2023-12-27 13:55:53,933][20347] Updated weights for policy 0, policy_version 1195362 (0.0010) [2023-12-27 13:55:53,947][20381] Updated weights for policy 1, policy_version 1194329 (0.0009) [2023-12-27 13:55:53,994][20347] Updated weights for policy 0, policy_version 1195372 (0.0011) [2023-12-27 13:55:53,999][20381] Updated weights for policy 1, policy_version 1194339 (0.0005) [2023-12-27 13:55:54,049][20381] Updated weights for policy 1, policy_version 1194349 (0.0005) [2023-12-27 13:55:54,105][20381] Updated weights for policy 1, policy_version 1194359 (0.0005) [2023-12-27 13:55:54,672][20347] Updated weights for policy 0, policy_version 1195382 (0.0010) [2023-12-27 13:55:54,726][20347] Updated weights for policy 0, policy_version 1195392 (0.0010) [2023-12-27 13:55:54,745][20381] Updated weights for policy 1, policy_version 1194369 (0.0010) [2023-12-27 13:55:54,788][20347] Updated weights for policy 0, policy_version 1195402 (0.0011) [2023-12-27 13:55:54,800][20381] Updated weights for policy 1, policy_version 1194379 (0.0010) [2023-12-27 13:55:54,849][20381] Updated weights for policy 1, policy_version 1194389 (0.0010) [2023-12-27 13:55:55,454][20347] Updated weights for policy 0, policy_version 1195412 (0.0010) [2023-12-27 13:55:55,505][20347] Updated weights for policy 0, policy_version 1195422 (0.0010) [2023-12-27 13:55:55,524][20381] Updated weights for policy 1, policy_version 1194399 (0.0011) [2023-12-27 13:55:55,560][20347] Updated weights for policy 0, policy_version 1195432 (0.0010) [2023-12-27 13:55:55,576][20381] Updated weights for policy 1, policy_version 1194409 (0.0010) [2023-12-27 13:55:55,630][20381] Updated weights for policy 1, policy_version 1194419 (0.0010) [2023-12-27 13:55:56,245][20347] Updated weights for policy 0, policy_version 1195442 (0.0010) [2023-12-27 13:55:56,300][20347] Updated weights for policy 0, policy_version 1195452 (0.0010) [2023-12-27 13:55:56,301][20381] Updated weights for policy 1, policy_version 1194429 (0.0010) [2023-12-27 13:55:56,357][20381] Updated weights for policy 1, policy_version 1194439 (0.0010) [2023-12-27 13:55:56,358][20347] Updated weights for policy 0, policy_version 1195462 (0.0010) [2023-12-27 13:55:56,406][20381] Updated weights for policy 1, policy_version 1194449 (0.0011) [2023-12-27 13:55:56,414][20347] Updated weights for policy 0, policy_version 1195472 (0.0011) [2023-12-27 13:55:57,078][20381] Updated weights for policy 1, policy_version 1194459 (0.0009) [2023-12-27 13:55:57,081][20347] Updated weights for policy 0, policy_version 1195482 (0.0009) [2023-12-27 13:55:57,126][20381] Updated weights for policy 1, policy_version 1194469 (0.0009) [2023-12-27 13:55:57,129][20347] Updated weights for policy 0, policy_version 1195492 (0.0010) [2023-12-27 13:55:57,176][20381] Updated weights for policy 1, policy_version 1194479 (0.0010) [2023-12-27 13:55:57,183][20347] Updated weights for policy 0, policy_version 1195502 (0.0010) [2023-12-27 13:55:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 611917824. Throughput: 0: 10383.1, 1: 10175.6. Samples: 611903488. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:55:57,363][19170] Avg episode reward: [(0, '970.990'), (1, '527.360')] [2023-12-27 13:55:57,804][20347] Updated weights for policy 0, policy_version 1195512 (0.0006) [2023-12-27 13:55:57,851][20381] Updated weights for policy 1, policy_version 1194489 (0.0010) [2023-12-27 13:55:57,862][20347] Updated weights for policy 0, policy_version 1195522 (0.0007) [2023-12-27 13:55:57,904][20381] Updated weights for policy 1, policy_version 1194499 (0.0010) [2023-12-27 13:55:57,911][20347] Updated weights for policy 0, policy_version 1195532 (0.0008) [2023-12-27 13:55:57,952][20381] Updated weights for policy 1, policy_version 1194509 (0.0010) [2023-12-27 13:55:58,003][20381] Updated weights for policy 1, policy_version 1194519 (0.0010) [2023-12-27 13:55:58,582][20347] Updated weights for policy 0, policy_version 1195542 (0.0008) [2023-12-27 13:55:58,647][20347] Updated weights for policy 0, policy_version 1195552 (0.0007) [2023-12-27 13:55:58,709][20347] Updated weights for policy 0, policy_version 1195562 (0.0008) [2023-12-27 13:55:58,794][20381] Updated weights for policy 1, policy_version 1194529 (0.0010) [2023-12-27 13:55:58,866][20381] Updated weights for policy 1, policy_version 1194539 (0.0009) [2023-12-27 13:55:58,923][20381] Updated weights for policy 1, policy_version 1194549 (0.0011) [2023-12-27 13:55:59,500][20347] Updated weights for policy 0, policy_version 1195572 (0.0008) [2023-12-27 13:55:59,555][20347] Updated weights for policy 0, policy_version 1195582 (0.0010) [2023-12-27 13:55:59,614][20347] Updated weights for policy 0, policy_version 1195592 (0.0010) [2023-12-27 13:55:59,616][20381] Updated weights for policy 1, policy_version 1194559 (0.0011) [2023-12-27 13:55:59,671][20381] Updated weights for policy 1, policy_version 1194569 (0.0010) [2023-12-27 13:55:59,732][20381] Updated weights for policy 1, policy_version 1194579 (0.0011) [2023-12-27 13:56:00,269][20347] Updated weights for policy 0, policy_version 1195602 (0.0009) [2023-12-27 13:56:00,332][20347] Updated weights for policy 0, policy_version 1195612 (0.0007) [2023-12-27 13:56:00,399][20347] Updated weights for policy 0, policy_version 1195622 (0.0007) [2023-12-27 13:56:00,447][20381] Updated weights for policy 1, policy_version 1194589 (0.0008) [2023-12-27 13:56:00,449][20347] Updated weights for policy 0, policy_version 1195632 (0.0007) [2023-12-27 13:56:00,495][20381] Updated weights for policy 1, policy_version 1194599 (0.0007) [2023-12-27 13:56:00,542][20381] Updated weights for policy 1, policy_version 1194609 (0.0008) [2023-12-27 13:56:01,093][20347] Updated weights for policy 0, policy_version 1195642 (0.0007) [2023-12-27 13:56:01,161][20347] Updated weights for policy 0, policy_version 1195652 (0.0007) [2023-12-27 13:56:01,223][20347] Updated weights for policy 0, policy_version 1195662 (0.0007) [2023-12-27 13:56:01,227][20381] Updated weights for policy 1, policy_version 1194619 (0.0008) [2023-12-27 13:56:01,286][20381] Updated weights for policy 1, policy_version 1194629 (0.0009) [2023-12-27 13:56:01,339][20381] Updated weights for policy 1, policy_version 1194639 (0.0007) [2023-12-27 13:56:01,918][20347] Updated weights for policy 0, policy_version 1195672 (0.0006) [2023-12-27 13:56:01,976][20347] Updated weights for policy 0, policy_version 1195682 (0.0006) [2023-12-27 13:56:02,038][20347] Updated weights for policy 0, policy_version 1195692 (0.0007) [2023-12-27 13:56:02,079][20381] Updated weights for policy 1, policy_version 1194649 (0.0009) [2023-12-27 13:56:02,140][20381] Updated weights for policy 1, policy_version 1194659 (0.0008) [2023-12-27 13:56:02,197][20381] Updated weights for policy 1, policy_version 1194669 (0.0008) [2023-12-27 13:56:02,258][20381] Updated weights for policy 1, policy_version 1194679 (0.0008) [2023-12-27 13:56:02,369][19170] Fps is (10 sec: 21286.0, 60 sec: 20614.4, 300 sec: 20604.5). Total num frames: 612016128. Throughput: 0: 10346.0, 1: 10137.9. Samples: 612023644. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:02,370][19170] Avg episode reward: [(0, '980.570'), (1, '367.290')] [2023-12-27 13:56:02,669][20347] Updated weights for policy 0, policy_version 1195702 (0.0007) [2023-12-27 13:56:02,729][20347] Updated weights for policy 0, policy_version 1195712 (0.0007) [2023-12-27 13:56:02,793][20347] Updated weights for policy 0, policy_version 1195722 (0.0007) [2023-12-27 13:56:02,961][20381] Updated weights for policy 1, policy_version 1194689 (0.0010) [2023-12-27 13:56:03,012][20381] Updated weights for policy 1, policy_version 1194699 (0.0010) [2023-12-27 13:56:03,061][20381] Updated weights for policy 1, policy_version 1194709 (0.0005) [2023-12-27 13:56:03,482][20347] Updated weights for policy 0, policy_version 1195732 (0.0007) [2023-12-27 13:56:03,532][20347] Updated weights for policy 0, policy_version 1195742 (0.0007) [2023-12-27 13:56:03,579][20347] Updated weights for policy 0, policy_version 1195752 (0.0007) [2023-12-27 13:56:03,713][20381] Updated weights for policy 1, policy_version 1194719 (0.0009) [2023-12-27 13:56:03,769][20381] Updated weights for policy 1, policy_version 1194729 (0.0010) [2023-12-27 13:56:03,817][20381] Updated weights for policy 1, policy_version 1194739 (0.0007) [2023-12-27 13:56:04,249][20347] Updated weights for policy 0, policy_version 1195762 (0.0007) [2023-12-27 13:56:04,300][20347] Updated weights for policy 0, policy_version 1195772 (0.0009) [2023-12-27 13:56:04,353][20347] Updated weights for policy 0, policy_version 1195782 (0.0008) [2023-12-27 13:56:04,414][20347] Updated weights for policy 0, policy_version 1195792 (0.0008) [2023-12-27 13:56:04,550][20381] Updated weights for policy 1, policy_version 1194749 (0.0009) [2023-12-27 13:56:04,609][20381] Updated weights for policy 1, policy_version 1194759 (0.0011) [2023-12-27 13:56:04,665][20381] Updated weights for policy 1, policy_version 1194769 (0.0011) [2023-12-27 13:56:05,053][20347] Updated weights for policy 0, policy_version 1195802 (0.0008) [2023-12-27 13:56:05,107][20347] Updated weights for policy 0, policy_version 1195812 (0.0006) [2023-12-27 13:56:05,170][20347] Updated weights for policy 0, policy_version 1195822 (0.0006) [2023-12-27 13:56:05,253][20381] Updated weights for policy 1, policy_version 1194779 (0.0007) [2023-12-27 13:56:05,309][20381] Updated weights for policy 1, policy_version 1194789 (0.0010) [2023-12-27 13:56:05,362][20381] Updated weights for policy 1, policy_version 1194799 (0.0005) [2023-12-27 13:56:05,816][20347] Updated weights for policy 0, policy_version 1195832 (0.0005) [2023-12-27 13:56:05,864][20347] Updated weights for policy 0, policy_version 1195842 (0.0008) [2023-12-27 13:56:05,907][20381] Updated weights for policy 1, policy_version 1194809 (0.0006) [2023-12-27 13:56:05,916][20347] Updated weights for policy 0, policy_version 1195852 (0.0007) [2023-12-27 13:56:05,962][20381] Updated weights for policy 1, policy_version 1194819 (0.0011) [2023-12-27 13:56:06,010][20381] Updated weights for policy 1, policy_version 1194829 (0.0010) [2023-12-27 13:56:06,061][20381] Updated weights for policy 1, policy_version 1194839 (0.0010) [2023-12-27 13:56:06,616][20347] Updated weights for policy 0, policy_version 1195862 (0.0006) [2023-12-27 13:56:06,623][20291] KL-divergence is very high: 115.0491 [2023-12-27 13:56:06,630][20291] KL-divergence is very high: 231.5999 [2023-12-27 13:56:06,673][20291] KL-divergence is very high: 170.0934 [2023-12-27 13:56:06,678][20347] Updated weights for policy 0, policy_version 1195872 (0.0008) [2023-12-27 13:56:06,680][20291] KL-divergence is very high: 280.5300 [2023-12-27 13:56:06,720][20291] KL-divergence is very high: 102.5490 [2023-12-27 13:56:06,727][20291] KL-divergence is very high: 152.9647 [2023-12-27 13:56:06,738][20347] Updated weights for policy 0, policy_version 1195882 (0.0007) [2023-12-27 13:56:06,812][20381] Updated weights for policy 1, policy_version 1194849 (0.0006) [2023-12-27 13:56:06,882][20381] Updated weights for policy 1, policy_version 1194859 (0.0007) [2023-12-27 13:56:06,942][20381] Updated weights for policy 1, policy_version 1194869 (0.0008) [2023-12-27 13:56:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 612114432. Throughput: 0: 10352.1, 1: 10189.9. Samples: 612086312. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:07,363][19170] Avg episode reward: [(0, '951.240'), (1, '438.660')] [2023-12-27 13:56:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001194872_305913856.pth... [2023-12-27 13:56:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001195888_306200576.pth... [2023-12-27 13:56:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001193688_305610752.pth [2023-12-27 13:56:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001194704_305897472.pth [2023-12-27 13:56:07,442][20291] KL-divergence is very high: 105.5455 [2023-12-27 13:56:07,448][20347] Updated weights for policy 0, policy_version 1195892 (0.0006) [2023-12-27 13:56:07,514][20347] Updated weights for policy 0, policy_version 1195902 (0.0005) [2023-12-27 13:56:07,571][20347] Updated weights for policy 0, policy_version 1195912 (0.0006) [2023-12-27 13:56:07,574][20381] Updated weights for policy 1, policy_version 1194879 (0.0007) [2023-12-27 13:56:07,630][20381] Updated weights for policy 1, policy_version 1194889 (0.0008) [2023-12-27 13:56:07,682][20381] Updated weights for policy 1, policy_version 1194899 (0.0011) [2023-12-27 13:56:08,192][20347] Updated weights for policy 0, policy_version 1195922 (0.0010) [2023-12-27 13:56:08,247][20347] Updated weights for policy 0, policy_version 1195932 (0.0010) [2023-12-27 13:56:08,307][20347] Updated weights for policy 0, policy_version 1195942 (0.0010) [2023-12-27 13:56:08,333][20381] Updated weights for policy 1, policy_version 1194909 (0.0009) [2023-12-27 13:56:08,369][20347] Updated weights for policy 0, policy_version 1195952 (0.0009) [2023-12-27 13:56:08,391][20381] Updated weights for policy 1, policy_version 1194919 (0.0008) [2023-12-27 13:56:08,447][20381] Updated weights for policy 1, policy_version 1194929 (0.0007) [2023-12-27 13:56:09,010][20347] Updated weights for policy 0, policy_version 1195962 (0.0007) [2023-12-27 13:56:09,055][20347] Updated weights for policy 0, policy_version 1195972 (0.0008) [2023-12-27 13:56:09,100][20347] Updated weights for policy 0, policy_version 1195982 (0.0008) [2023-12-27 13:56:09,193][20381] Updated weights for policy 1, policy_version 1194939 (0.0008) [2023-12-27 13:56:09,255][20381] Updated weights for policy 1, policy_version 1194949 (0.0007) [2023-12-27 13:56:09,313][20381] Updated weights for policy 1, policy_version 1194959 (0.0008) [2023-12-27 13:56:09,822][20347] Updated weights for policy 0, policy_version 1195992 (0.0006) [2023-12-27 13:56:09,878][20347] Updated weights for policy 0, policy_version 1196002 (0.0006) [2023-12-27 13:56:09,933][20347] Updated weights for policy 0, policy_version 1196012 (0.0006) [2023-12-27 13:56:10,071][20381] Updated weights for policy 1, policy_version 1194969 (0.0008) [2023-12-27 13:56:10,134][20381] Updated weights for policy 1, policy_version 1194979 (0.0008) [2023-12-27 13:56:10,191][20381] Updated weights for policy 1, policy_version 1194989 (0.0008) [2023-12-27 13:56:10,251][20381] Updated weights for policy 1, policy_version 1194999 (0.0007) [2023-12-27 13:56:10,629][20347] Updated weights for policy 0, policy_version 1196022 (0.0008) [2023-12-27 13:56:10,687][20347] Updated weights for policy 0, policy_version 1196032 (0.0008) [2023-12-27 13:56:10,743][20347] Updated weights for policy 0, policy_version 1196042 (0.0008) [2023-12-27 13:56:10,868][20381] Updated weights for policy 1, policy_version 1195009 (0.0008) [2023-12-27 13:56:10,922][20381] Updated weights for policy 1, policy_version 1195019 (0.0007) [2023-12-27 13:56:10,969][20381] Updated weights for policy 1, policy_version 1195029 (0.0007) [2023-12-27 13:56:11,493][20347] Updated weights for policy 0, policy_version 1196052 (0.0007) [2023-12-27 13:56:11,552][20347] Updated weights for policy 0, policy_version 1196062 (0.0007) [2023-12-27 13:56:11,607][20347] Updated weights for policy 0, policy_version 1196072 (0.0007) [2023-12-27 13:56:11,768][20381] Updated weights for policy 1, policy_version 1195039 (0.0007) [2023-12-27 13:56:11,832][20381] Updated weights for policy 1, policy_version 1195049 (0.0006) [2023-12-27 13:56:11,896][20381] Updated weights for policy 1, policy_version 1195059 (0.0007) [2023-12-27 13:56:12,276][20347] Updated weights for policy 0, policy_version 1196082 (0.0008) [2023-12-27 13:56:12,334][20347] Updated weights for policy 0, policy_version 1196092 (0.0008) [2023-12-27 13:56:12,362][19170] Fps is (10 sec: 19672.9, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 612212736. Throughput: 0: 10358.5, 1: 10215.0. Samples: 612208420. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:12,364][19170] Avg episode reward: [(0, '873.020'), (1, '605.160')] [2023-12-27 13:56:12,402][20347] Updated weights for policy 0, policy_version 1196102 (0.0010) [2023-12-27 13:56:12,461][20347] Updated weights for policy 0, policy_version 1196112 (0.0009) [2023-12-27 13:56:12,558][20381] Updated weights for policy 1, policy_version 1195069 (0.0006) [2023-12-27 13:56:12,612][20381] Updated weights for policy 1, policy_version 1195079 (0.0006) [2023-12-27 13:56:12,673][20381] Updated weights for policy 1, policy_version 1195089 (0.0011) [2023-12-27 13:56:13,096][20347] Updated weights for policy 0, policy_version 1196122 (0.0005) [2023-12-27 13:56:13,149][20347] Updated weights for policy 0, policy_version 1196132 (0.0005) [2023-12-27 13:56:13,198][20347] Updated weights for policy 0, policy_version 1196142 (0.0005) [2023-12-27 13:56:13,322][20381] Updated weights for policy 1, policy_version 1195099 (0.0009) [2023-12-27 13:56:13,386][20381] Updated weights for policy 1, policy_version 1195109 (0.0005) [2023-12-27 13:56:13,452][20381] Updated weights for policy 1, policy_version 1195119 (0.0006) [2023-12-27 13:56:13,827][20347] Updated weights for policy 0, policy_version 1196152 (0.0005) [2023-12-27 13:56:13,877][20347] Updated weights for policy 0, policy_version 1196162 (0.0009) [2023-12-27 13:56:13,922][20347] Updated weights for policy 0, policy_version 1196172 (0.0010) [2023-12-27 13:56:14,045][20381] Updated weights for policy 1, policy_version 1195129 (0.0006) [2023-12-27 13:56:14,107][20381] Updated weights for policy 1, policy_version 1195139 (0.0007) [2023-12-27 13:56:14,165][20381] Updated weights for policy 1, policy_version 1195149 (0.0006) [2023-12-27 13:56:14,225][20381] Updated weights for policy 1, policy_version 1195159 (0.0009) [2023-12-27 13:56:14,529][20347] Updated weights for policy 0, policy_version 1196182 (0.0007) [2023-12-27 13:56:14,584][20347] Updated weights for policy 0, policy_version 1196192 (0.0006) [2023-12-27 13:56:14,640][20347] Updated weights for policy 0, policy_version 1196202 (0.0006) [2023-12-27 13:56:14,887][20381] Updated weights for policy 1, policy_version 1195169 (0.0006) [2023-12-27 13:56:14,948][20381] Updated weights for policy 1, policy_version 1195179 (0.0006) [2023-12-27 13:56:15,011][20381] Updated weights for policy 1, policy_version 1195189 (0.0008) [2023-12-27 13:56:15,305][20347] Updated weights for policy 0, policy_version 1196212 (0.0006) [2023-12-27 13:56:15,355][20347] Updated weights for policy 0, policy_version 1196222 (0.0008) [2023-12-27 13:56:15,411][20347] Updated weights for policy 0, policy_version 1196232 (0.0009) [2023-12-27 13:56:15,602][20381] Updated weights for policy 1, policy_version 1195199 (0.0006) [2023-12-27 13:56:15,650][20381] Updated weights for policy 1, policy_version 1195209 (0.0005) [2023-12-27 13:56:15,699][20381] Updated weights for policy 1, policy_version 1195219 (0.0005) [2023-12-27 13:56:16,183][20347] Updated weights for policy 0, policy_version 1196242 (0.0008) [2023-12-27 13:56:16,237][20347] Updated weights for policy 0, policy_version 1196252 (0.0007) [2023-12-27 13:56:16,247][20381] Updated weights for policy 1, policy_version 1195229 (0.0006) [2023-12-27 13:56:16,292][20347] Updated weights for policy 0, policy_version 1196262 (0.0007) [2023-12-27 13:56:16,307][20381] Updated weights for policy 1, policy_version 1195239 (0.0005) [2023-12-27 13:56:16,344][20347] Updated weights for policy 0, policy_version 1196272 (0.0007) [2023-12-27 13:56:16,362][20381] Updated weights for policy 1, policy_version 1195249 (0.0005) [2023-12-27 13:56:16,876][20381] Updated weights for policy 1, policy_version 1195259 (0.0006) [2023-12-27 13:56:16,940][20381] Updated weights for policy 1, policy_version 1195269 (0.0005) [2023-12-27 13:56:17,005][20381] Updated weights for policy 1, policy_version 1195279 (0.0005) [2023-12-27 13:56:17,095][20347] Updated weights for policy 0, policy_version 1196282 (0.0006) [2023-12-27 13:56:17,152][20347] Updated weights for policy 0, policy_version 1196292 (0.0005) [2023-12-27 13:56:17,211][20347] Updated weights for policy 0, policy_version 1196302 (0.0007) [2023-12-27 13:56:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 612327424. Throughput: 0: 10290.7, 1: 10319.8. Samples: 612335968. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:17,364][19170] Avg episode reward: [(0, '892.640'), (1, '569.520')] [2023-12-27 13:56:17,533][20381] Updated weights for policy 1, policy_version 1195289 (0.0005) [2023-12-27 13:56:17,598][20381] Updated weights for policy 1, policy_version 1195299 (0.0008) [2023-12-27 13:56:17,649][20381] Updated weights for policy 1, policy_version 1195309 (0.0008) [2023-12-27 13:56:17,700][20381] Updated weights for policy 1, policy_version 1195319 (0.0008) [2023-12-27 13:56:17,844][20347] Updated weights for policy 0, policy_version 1196312 (0.0006) [2023-12-27 13:56:17,900][20347] Updated weights for policy 0, policy_version 1196322 (0.0005) [2023-12-27 13:56:17,959][20347] Updated weights for policy 0, policy_version 1196332 (0.0005) [2023-12-27 13:56:18,353][20381] Updated weights for policy 1, policy_version 1195329 (0.0008) [2023-12-27 13:56:18,406][20381] Updated weights for policy 1, policy_version 1195339 (0.0007) [2023-12-27 13:56:18,458][20381] Updated weights for policy 1, policy_version 1195349 (0.0008) [2023-12-27 13:56:18,551][20347] Updated weights for policy 0, policy_version 1196342 (0.0006) [2023-12-27 13:56:18,603][20347] Updated weights for policy 0, policy_version 1196352 (0.0007) [2023-12-27 13:56:18,659][20347] Updated weights for policy 0, policy_version 1196362 (0.0007) [2023-12-27 13:56:19,192][20381] Updated weights for policy 1, policy_version 1195359 (0.0010) [2023-12-27 13:56:19,254][20381] Updated weights for policy 1, policy_version 1195369 (0.0008) [2023-12-27 13:56:19,314][20381] Updated weights for policy 1, policy_version 1195379 (0.0008) [2023-12-27 13:56:19,353][20347] Updated weights for policy 0, policy_version 1196372 (0.0008) [2023-12-27 13:56:19,414][20347] Updated weights for policy 0, policy_version 1196382 (0.0008) [2023-12-27 13:56:19,473][20347] Updated weights for policy 0, policy_version 1196392 (0.0007) [2023-12-27 13:56:20,008][20381] Updated weights for policy 1, policy_version 1195389 (0.0008) [2023-12-27 13:56:20,064][20381] Updated weights for policy 1, policy_version 1195399 (0.0009) [2023-12-27 13:56:20,127][20381] Updated weights for policy 1, policy_version 1195409 (0.0006) [2023-12-27 13:56:20,230][20347] Updated weights for policy 0, policy_version 1196402 (0.0008) [2023-12-27 13:56:20,294][20347] Updated weights for policy 0, policy_version 1196412 (0.0008) [2023-12-27 13:56:20,357][20347] Updated weights for policy 0, policy_version 1196422 (0.0007) [2023-12-27 13:56:20,420][20347] Updated weights for policy 0, policy_version 1196432 (0.0008) [2023-12-27 13:56:20,801][20381] Updated weights for policy 1, policy_version 1195419 (0.0006) [2023-12-27 13:56:20,848][20381] Updated weights for policy 1, policy_version 1195429 (0.0007) [2023-12-27 13:56:20,902][20381] Updated weights for policy 1, policy_version 1195439 (0.0006) [2023-12-27 13:56:21,095][20347] Updated weights for policy 0, policy_version 1196442 (0.0009) [2023-12-27 13:56:21,154][20347] Updated weights for policy 0, policy_version 1196452 (0.0009) [2023-12-27 13:56:21,205][20347] Updated weights for policy 0, policy_version 1196462 (0.0008) [2023-12-27 13:56:21,644][20381] Updated weights for policy 1, policy_version 1195449 (0.0006) [2023-12-27 13:56:21,710][20381] Updated weights for policy 1, policy_version 1195459 (0.0011) [2023-12-27 13:56:21,774][20381] Updated weights for policy 1, policy_version 1195469 (0.0008) [2023-12-27 13:56:21,831][20381] Updated weights for policy 1, policy_version 1195479 (0.0006) [2023-12-27 13:56:21,955][20347] Updated weights for policy 0, policy_version 1196472 (0.0010) [2023-12-27 13:56:22,011][20347] Updated weights for policy 0, policy_version 1196482 (0.0011) [2023-12-27 13:56:22,074][20347] Updated weights for policy 0, policy_version 1196492 (0.0010) [2023-12-27 13:56:22,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 612425728. Throughput: 0: 10280.7, 1: 10339.1. Samples: 612397756. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:22,363][19170] Avg episode reward: [(0, '941.490'), (1, '574.630')] [2023-12-27 13:56:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001196496_306356224.pth... [2023-12-27 13:56:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001195312_306053120.pth [2023-12-27 13:56:22,423][20381] Updated weights for policy 1, policy_version 1195489 (0.0010) [2023-12-27 13:56:22,480][20381] Updated weights for policy 1, policy_version 1195499 (0.0009) [2023-12-27 13:56:22,541][20381] Updated weights for policy 1, policy_version 1195509 (0.0006) [2023-12-27 13:56:22,560][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001195512_306077696.pth... [2023-12-27 13:56:22,564][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001194264_305758208.pth [2023-12-27 13:56:22,795][20347] Updated weights for policy 0, policy_version 1196502 (0.0010) [2023-12-27 13:56:22,852][20347] Updated weights for policy 0, policy_version 1196512 (0.0010) [2023-12-27 13:56:22,915][20347] Updated weights for policy 0, policy_version 1196522 (0.0009) [2023-12-27 13:56:23,273][20381] Updated weights for policy 1, policy_version 1195519 (0.0006) [2023-12-27 13:56:23,322][20381] Updated weights for policy 1, policy_version 1195529 (0.0005) [2023-12-27 13:56:23,375][20381] Updated weights for policy 1, policy_version 1195539 (0.0006) [2023-12-27 13:56:23,571][20347] Updated weights for policy 0, policy_version 1196532 (0.0008) [2023-12-27 13:56:23,626][20347] Updated weights for policy 0, policy_version 1196542 (0.0010) [2023-12-27 13:56:23,678][20347] Updated weights for policy 0, policy_version 1196552 (0.0010) [2023-12-27 13:56:24,042][20381] Updated weights for policy 1, policy_version 1195549 (0.0007) [2023-12-27 13:56:24,087][20381] Updated weights for policy 1, policy_version 1195559 (0.0007) [2023-12-27 13:56:24,138][20381] Updated weights for policy 1, policy_version 1195569 (0.0010) [2023-12-27 13:56:24,292][20347] Updated weights for policy 0, policy_version 1196562 (0.0006) [2023-12-27 13:56:24,345][20347] Updated weights for policy 0, policy_version 1196572 (0.0005) [2023-12-27 13:56:24,399][20347] Updated weights for policy 0, policy_version 1196582 (0.0010) [2023-12-27 13:56:24,450][20347] Updated weights for policy 0, policy_version 1196592 (0.0010) [2023-12-27 13:56:24,766][20381] Updated weights for policy 1, policy_version 1195579 (0.0010) [2023-12-27 13:56:24,821][20381] Updated weights for policy 1, policy_version 1195589 (0.0010) [2023-12-27 13:56:24,876][20381] Updated weights for policy 1, policy_version 1195599 (0.0010) [2023-12-27 13:56:25,081][20347] Updated weights for policy 0, policy_version 1196602 (0.0007) [2023-12-27 13:56:25,137][20347] Updated weights for policy 0, policy_version 1196612 (0.0007) [2023-12-27 13:56:25,182][20347] Updated weights for policy 0, policy_version 1196622 (0.0007) [2023-12-27 13:56:25,560][20381] Updated weights for policy 1, policy_version 1195609 (0.0010) [2023-12-27 13:56:25,611][20381] Updated weights for policy 1, policy_version 1195619 (0.0010) [2023-12-27 13:56:25,659][20381] Updated weights for policy 1, policy_version 1195629 (0.0010) [2023-12-27 13:56:25,706][20381] Updated weights for policy 1, policy_version 1195639 (0.0010) [2023-12-27 13:56:25,826][20347] Updated weights for policy 0, policy_version 1196632 (0.0006) [2023-12-27 13:56:25,886][20347] Updated weights for policy 0, policy_version 1196642 (0.0005) [2023-12-27 13:56:25,946][20347] Updated weights for policy 0, policy_version 1196652 (0.0007) [2023-12-27 13:56:26,344][20381] Updated weights for policy 1, policy_version 1195649 (0.0009) [2023-12-27 13:56:26,399][20381] Updated weights for policy 1, policy_version 1195659 (0.0010) [2023-12-27 13:56:26,449][20381] Updated weights for policy 1, policy_version 1195669 (0.0006) [2023-12-27 13:56:26,545][20347] Updated weights for policy 0, policy_version 1196662 (0.0008) [2023-12-27 13:56:26,591][20347] Updated weights for policy 0, policy_version 1196672 (0.0007) [2023-12-27 13:56:26,645][20347] Updated weights for policy 0, policy_version 1196682 (0.0008) [2023-12-27 13:56:27,103][20381] Updated weights for policy 1, policy_version 1195679 (0.0009) [2023-12-27 13:56:27,156][20381] Updated weights for policy 1, policy_version 1195689 (0.0010) [2023-12-27 13:56:27,210][20381] Updated weights for policy 1, policy_version 1195699 (0.0010) [2023-12-27 13:56:27,269][20347] Updated weights for policy 0, policy_version 1196692 (0.0007) [2023-12-27 13:56:27,330][20347] Updated weights for policy 0, policy_version 1196702 (0.0006) [2023-12-27 13:56:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 612532224. Throughput: 0: 10261.0, 1: 10405.9. Samples: 612521120. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:27,363][19170] Avg episode reward: [(0, '863.080'), (1, '603.110')] [2023-12-27 13:56:27,393][20347] Updated weights for policy 0, policy_version 1196712 (0.0005) [2023-12-27 13:56:27,904][20381] Updated weights for policy 1, policy_version 1195709 (0.0006) [2023-12-27 13:56:27,946][20381] Updated weights for policy 1, policy_version 1195719 (0.0005) [2023-12-27 13:56:27,990][20381] Updated weights for policy 1, policy_version 1195729 (0.0007) [2023-12-27 13:56:27,993][20347] Updated weights for policy 0, policy_version 1196722 (0.0006) [2023-12-27 13:56:28,048][20347] Updated weights for policy 0, policy_version 1196732 (0.0007) [2023-12-27 13:56:28,104][20347] Updated weights for policy 0, policy_version 1196742 (0.0008) [2023-12-27 13:56:28,160][20347] Updated weights for policy 0, policy_version 1196752 (0.0007) [2023-12-27 13:56:28,608][20381] Updated weights for policy 1, policy_version 1195739 (0.0005) [2023-12-27 13:56:28,655][20381] Updated weights for policy 1, policy_version 1195749 (0.0006) [2023-12-27 13:56:28,700][20381] Updated weights for policy 1, policy_version 1195759 (0.0010) [2023-12-27 13:56:28,792][20347] Updated weights for policy 0, policy_version 1196762 (0.0005) [2023-12-27 13:56:28,843][20347] Updated weights for policy 0, policy_version 1196772 (0.0005) [2023-12-27 13:56:28,889][20347] Updated weights for policy 0, policy_version 1196782 (0.0005) [2023-12-27 13:56:29,387][20381] Updated weights for policy 1, policy_version 1195769 (0.0008) [2023-12-27 13:56:29,436][20381] Updated weights for policy 1, policy_version 1195779 (0.0008) [2023-12-27 13:56:29,489][20381] Updated weights for policy 1, policy_version 1195789 (0.0008) [2023-12-27 13:56:29,543][20347] Updated weights for policy 0, policy_version 1196792 (0.0008) [2023-12-27 13:56:29,544][20381] Updated weights for policy 1, policy_version 1195799 (0.0006) [2023-12-27 13:56:29,601][20347] Updated weights for policy 0, policy_version 1196802 (0.0011) [2023-12-27 13:56:29,666][20347] Updated weights for policy 0, policy_version 1196812 (0.0010) [2023-12-27 13:56:30,210][20381] Updated weights for policy 1, policy_version 1195809 (0.0006) [2023-12-27 13:56:30,258][20381] Updated weights for policy 1, policy_version 1195819 (0.0005) [2023-12-27 13:56:30,311][20381] Updated weights for policy 1, policy_version 1195829 (0.0009) [2023-12-27 13:56:30,335][20347] Updated weights for policy 0, policy_version 1196822 (0.0007) [2023-12-27 13:56:30,401][20347] Updated weights for policy 0, policy_version 1196832 (0.0010) [2023-12-27 13:56:30,455][20347] Updated weights for policy 0, policy_version 1196842 (0.0010) [2023-12-27 13:56:30,974][20381] Updated weights for policy 1, policy_version 1195839 (0.0008) [2023-12-27 13:56:31,025][20381] Updated weights for policy 1, policy_version 1195849 (0.0007) [2023-12-27 13:56:31,043][20347] Updated weights for policy 0, policy_version 1196852 (0.0009) [2023-12-27 13:56:31,084][20381] Updated weights for policy 1, policy_version 1195859 (0.0008) [2023-12-27 13:56:31,100][20347] Updated weights for policy 0, policy_version 1196862 (0.0007) [2023-12-27 13:56:31,155][20347] Updated weights for policy 0, policy_version 1196872 (0.0009) [2023-12-27 13:56:31,816][20381] Updated weights for policy 1, policy_version 1195869 (0.0008) [2023-12-27 13:56:31,862][20381] Updated weights for policy 1, policy_version 1195879 (0.0007) [2023-12-27 13:56:31,902][20347] Updated weights for policy 0, policy_version 1196882 (0.0009) [2023-12-27 13:56:31,915][20381] Updated weights for policy 1, policy_version 1195889 (0.0006) [2023-12-27 13:56:31,945][20347] Updated weights for policy 0, policy_version 1196892 (0.0005) [2023-12-27 13:56:32,004][20347] Updated weights for policy 0, policy_version 1196902 (0.0008) [2023-12-27 13:56:32,059][20347] Updated weights for policy 0, policy_version 1196912 (0.0008) [2023-12-27 13:56:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 612638720. Throughput: 0: 10283.9, 1: 10360.1. Samples: 612648336. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:32,363][19170] Avg episode reward: [(0, '873.340'), (1, '490.300')] [2023-12-27 13:56:32,648][20381] Updated weights for policy 1, policy_version 1195899 (0.0007) [2023-12-27 13:56:32,714][20381] Updated weights for policy 1, policy_version 1195909 (0.0007) [2023-12-27 13:56:32,745][20347] Updated weights for policy 0, policy_version 1196922 (0.0007) [2023-12-27 13:56:32,773][20381] Updated weights for policy 1, policy_version 1195919 (0.0005) [2023-12-27 13:56:32,802][20347] Updated weights for policy 0, policy_version 1196932 (0.0005) [2023-12-27 13:56:32,861][20347] Updated weights for policy 0, policy_version 1196942 (0.0007) [2023-12-27 13:56:33,435][20381] Updated weights for policy 1, policy_version 1195929 (0.0007) [2023-12-27 13:56:33,464][20347] Updated weights for policy 0, policy_version 1196952 (0.0010) [2023-12-27 13:56:33,490][20381] Updated weights for policy 1, policy_version 1195939 (0.0010) [2023-12-27 13:56:33,514][20347] Updated weights for policy 0, policy_version 1196962 (0.0008) [2023-12-27 13:56:33,547][20381] Updated weights for policy 1, policy_version 1195949 (0.0007) [2023-12-27 13:56:33,576][20347] Updated weights for policy 0, policy_version 1196972 (0.0005) [2023-12-27 13:56:33,600][20381] Updated weights for policy 1, policy_version 1195959 (0.0009) [2023-12-27 13:56:34,136][20347] Updated weights for policy 0, policy_version 1196982 (0.0006) [2023-12-27 13:56:34,200][20347] Updated weights for policy 0, policy_version 1196992 (0.0008) [2023-12-27 13:56:34,254][20347] Updated weights for policy 0, policy_version 1197002 (0.0007) [2023-12-27 13:56:34,309][20381] Updated weights for policy 1, policy_version 1195969 (0.0007) [2023-12-27 13:56:34,362][20381] Updated weights for policy 1, policy_version 1195979 (0.0006) [2023-12-27 13:56:34,414][20381] Updated weights for policy 1, policy_version 1195989 (0.0006) [2023-12-27 13:56:34,930][20347] Updated weights for policy 0, policy_version 1197012 (0.0008) [2023-12-27 13:56:34,974][20347] Updated weights for policy 0, policy_version 1197022 (0.0008) [2023-12-27 13:56:35,020][20381] Updated weights for policy 1, policy_version 1195999 (0.0006) [2023-12-27 13:56:35,026][20347] Updated weights for policy 0, policy_version 1197032 (0.0007) [2023-12-27 13:56:35,077][20381] Updated weights for policy 1, policy_version 1196009 (0.0006) [2023-12-27 13:56:35,130][20381] Updated weights for policy 1, policy_version 1196019 (0.0007) [2023-12-27 13:56:35,670][20347] Updated weights for policy 0, policy_version 1197042 (0.0006) [2023-12-27 13:56:35,722][20347] Updated weights for policy 0, policy_version 1197052 (0.0008) [2023-12-27 13:56:35,742][20381] Updated weights for policy 1, policy_version 1196029 (0.0009) [2023-12-27 13:56:35,773][20347] Updated weights for policy 0, policy_version 1197062 (0.0005) [2023-12-27 13:56:35,802][20381] Updated weights for policy 1, policy_version 1196039 (0.0011) [2023-12-27 13:56:35,829][20347] Updated weights for policy 0, policy_version 1197072 (0.0007) [2023-12-27 13:56:35,860][20381] Updated weights for policy 1, policy_version 1196049 (0.0005) [2023-12-27 13:56:36,523][20347] Updated weights for policy 0, policy_version 1197082 (0.0008) [2023-12-27 13:56:36,530][20381] Updated weights for policy 1, policy_version 1196059 (0.0010) [2023-12-27 13:56:36,577][20347] Updated weights for policy 0, policy_version 1197092 (0.0006) [2023-12-27 13:56:36,587][20381] Updated weights for policy 1, policy_version 1196069 (0.0011) [2023-12-27 13:56:36,640][20347] Updated weights for policy 0, policy_version 1197102 (0.0009) [2023-12-27 13:56:36,648][20381] Updated weights for policy 1, policy_version 1196079 (0.0011) [2023-12-27 13:56:37,340][20381] Updated weights for policy 1, policy_version 1196089 (0.0010) [2023-12-27 13:56:37,346][20347] Updated weights for policy 0, policy_version 1197112 (0.0007) [2023-12-27 13:56:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 612737024. Throughput: 0: 10305.7, 1: 10395.7. Samples: 612711992. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:37,363][19170] Avg episode reward: [(0, '893.050'), (1, '514.250')] [2023-12-27 13:56:37,398][20381] Updated weights for policy 1, policy_version 1196099 (0.0006) [2023-12-27 13:56:37,412][20347] Updated weights for policy 0, policy_version 1197122 (0.0007) [2023-12-27 13:56:37,450][20381] Updated weights for policy 1, policy_version 1196109 (0.0007) [2023-12-27 13:56:37,471][20347] Updated weights for policy 0, policy_version 1197132 (0.0007) [2023-12-27 13:56:37,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001197136_306520064.pth... [2023-12-27 13:56:37,494][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001195888_306200576.pth [2023-12-27 13:56:37,504][20381] Updated weights for policy 1, policy_version 1196119 (0.0007) [2023-12-27 13:56:37,509][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001196120_306233344.pth... [2023-12-27 13:56:37,512][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001194872_305913856.pth [2023-12-27 13:56:38,098][20381] Updated weights for policy 1, policy_version 1196129 (0.0006) [2023-12-27 13:56:38,152][20347] Updated weights for policy 0, policy_version 1197142 (0.0007) [2023-12-27 13:56:38,158][20381] Updated weights for policy 1, policy_version 1196139 (0.0010) [2023-12-27 13:56:38,208][20347] Updated weights for policy 0, policy_version 1197152 (0.0007) [2023-12-27 13:56:38,215][20381] Updated weights for policy 1, policy_version 1196149 (0.0010) [2023-12-27 13:56:38,268][20347] Updated weights for policy 0, policy_version 1197162 (0.0006) [2023-12-27 13:56:38,920][20381] Updated weights for policy 1, policy_version 1196159 (0.0010) [2023-12-27 13:56:38,939][20347] Updated weights for policy 0, policy_version 1197172 (0.0007) [2023-12-27 13:56:38,979][20381] Updated weights for policy 1, policy_version 1196169 (0.0010) [2023-12-27 13:56:38,995][20347] Updated weights for policy 0, policy_version 1197182 (0.0005) [2023-12-27 13:56:39,037][20381] Updated weights for policy 1, policy_version 1196179 (0.0010) [2023-12-27 13:56:39,056][20347] Updated weights for policy 0, policy_version 1197192 (0.0007) [2023-12-27 13:56:39,681][20381] Updated weights for policy 1, policy_version 1196189 (0.0010) [2023-12-27 13:56:39,740][20381] Updated weights for policy 1, policy_version 1196199 (0.0011) [2023-12-27 13:56:39,779][20347] Updated weights for policy 0, policy_version 1197202 (0.0006) [2023-12-27 13:56:39,800][20381] Updated weights for policy 1, policy_version 1196209 (0.0010) [2023-12-27 13:56:39,840][20347] Updated weights for policy 0, policy_version 1197212 (0.0007) [2023-12-27 13:56:39,901][20347] Updated weights for policy 0, policy_version 1197222 (0.0009) [2023-12-27 13:56:39,968][20347] Updated weights for policy 0, policy_version 1197232 (0.0007) [2023-12-27 13:56:40,419][20381] Updated weights for policy 1, policy_version 1196219 (0.0008) [2023-12-27 13:56:40,484][20381] Updated weights for policy 1, policy_version 1196229 (0.0008) [2023-12-27 13:56:40,548][20381] Updated weights for policy 1, policy_version 1196239 (0.0010) [2023-12-27 13:56:40,676][20347] Updated weights for policy 0, policy_version 1197242 (0.0005) [2023-12-27 13:56:40,706][20291] KL-divergence is very high: 150.0065 [2023-12-27 13:56:40,736][20347] Updated weights for policy 0, policy_version 1197252 (0.0009) [2023-12-27 13:56:40,797][20347] Updated weights for policy 0, policy_version 1197262 (0.0008) [2023-12-27 13:56:41,200][20381] Updated weights for policy 1, policy_version 1196249 (0.0010) [2023-12-27 13:56:41,270][20381] Updated weights for policy 1, policy_version 1196259 (0.0007) [2023-12-27 13:56:41,336][20381] Updated weights for policy 1, policy_version 1196269 (0.0008) [2023-12-27 13:56:41,404][20381] Updated weights for policy 1, policy_version 1196279 (0.0007) [2023-12-27 13:56:41,470][20347] Updated weights for policy 0, policy_version 1197272 (0.0007) [2023-12-27 13:56:41,531][20347] Updated weights for policy 0, policy_version 1197282 (0.0006) [2023-12-27 13:56:41,592][20347] Updated weights for policy 0, policy_version 1197292 (0.0006) [2023-12-27 13:56:42,117][20381] Updated weights for policy 1, policy_version 1196289 (0.0007) [2023-12-27 13:56:42,172][20381] Updated weights for policy 1, policy_version 1196299 (0.0007) [2023-12-27 13:56:42,228][20381] Updated weights for policy 1, policy_version 1196309 (0.0007) [2023-12-27 13:56:42,246][20347] Updated weights for policy 0, policy_version 1197302 (0.0006) [2023-12-27 13:56:42,305][20347] Updated weights for policy 0, policy_version 1197312 (0.0008) [2023-12-27 13:56:42,357][20347] Updated weights for policy 0, policy_version 1197322 (0.0007) [2023-12-27 13:56:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 612843520. Throughput: 0: 10334.8, 1: 10401.4. Samples: 612836612. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:42,363][19170] Avg episode reward: [(0, '912.300'), (1, '418.550')] [2023-12-27 13:56:42,986][20381] Updated weights for policy 1, policy_version 1196319 (0.0007) [2023-12-27 13:56:43,039][20381] Updated weights for policy 1, policy_version 1196329 (0.0007) [2023-12-27 13:56:43,057][20347] Updated weights for policy 0, policy_version 1197332 (0.0007) [2023-12-27 13:56:43,088][20381] Updated weights for policy 1, policy_version 1196339 (0.0008) [2023-12-27 13:56:43,110][20347] Updated weights for policy 0, policy_version 1197342 (0.0005) [2023-12-27 13:56:43,164][20347] Updated weights for policy 0, policy_version 1197352 (0.0005) [2023-12-27 13:56:43,736][20381] Updated weights for policy 1, policy_version 1196349 (0.0006) [2023-12-27 13:56:43,799][20381] Updated weights for policy 1, policy_version 1196359 (0.0005) [2023-12-27 13:56:43,810][20347] Updated weights for policy 0, policy_version 1197362 (0.0006) [2023-12-27 13:56:43,855][20381] Updated weights for policy 1, policy_version 1196369 (0.0006) [2023-12-27 13:56:43,870][20347] Updated weights for policy 0, policy_version 1197372 (0.0006) [2023-12-27 13:56:43,925][20347] Updated weights for policy 0, policy_version 1197382 (0.0005) [2023-12-27 13:56:43,977][20347] Updated weights for policy 0, policy_version 1197392 (0.0005) [2023-12-27 13:56:44,490][20381] Updated weights for policy 1, policy_version 1196379 (0.0008) [2023-12-27 13:56:44,498][20347] Updated weights for policy 0, policy_version 1197402 (0.0008) [2023-12-27 13:56:44,542][20381] Updated weights for policy 1, policy_version 1196389 (0.0007) [2023-12-27 13:56:44,558][20347] Updated weights for policy 0, policy_version 1197412 (0.0008) [2023-12-27 13:56:44,603][20381] Updated weights for policy 1, policy_version 1196399 (0.0006) [2023-12-27 13:56:44,610][20347] Updated weights for policy 0, policy_version 1197422 (0.0005) [2023-12-27 13:56:45,282][20347] Updated weights for policy 0, policy_version 1197432 (0.0005) [2023-12-27 13:56:45,331][20381] Updated weights for policy 1, policy_version 1196409 (0.0006) [2023-12-27 13:56:45,338][20347] Updated weights for policy 0, policy_version 1197442 (0.0007) [2023-12-27 13:56:45,393][20347] Updated weights for policy 0, policy_version 1197452 (0.0006) [2023-12-27 13:56:45,397][20381] Updated weights for policy 1, policy_version 1196419 (0.0008) [2023-12-27 13:56:45,458][20381] Updated weights for policy 1, policy_version 1196429 (0.0008) [2023-12-27 13:56:45,528][20381] Updated weights for policy 1, policy_version 1196439 (0.0009) [2023-12-27 13:56:46,038][20347] Updated weights for policy 0, policy_version 1197462 (0.0007) [2023-12-27 13:56:46,102][20347] Updated weights for policy 0, policy_version 1197472 (0.0008) [2023-12-27 13:56:46,165][20347] Updated weights for policy 0, policy_version 1197482 (0.0007) [2023-12-27 13:56:46,223][20381] Updated weights for policy 1, policy_version 1196449 (0.0007) [2023-12-27 13:56:46,277][20381] Updated weights for policy 1, policy_version 1196459 (0.0007) [2023-12-27 13:56:46,324][20381] Updated weights for policy 1, policy_version 1196469 (0.0007) [2023-12-27 13:56:46,769][20347] Updated weights for policy 0, policy_version 1197492 (0.0005) [2023-12-27 13:56:46,830][20347] Updated weights for policy 0, policy_version 1197502 (0.0007) [2023-12-27 13:56:46,852][20291] KL-divergence is very high: 118.5640 [2023-12-27 13:56:46,884][20347] Updated weights for policy 0, policy_version 1197512 (0.0007) [2023-12-27 13:56:46,893][20291] KL-divergence is very high: 101.7324 [2023-12-27 13:56:47,003][20381] Updated weights for policy 1, policy_version 1196479 (0.0007) [2023-12-27 13:56:47,048][20381] Updated weights for policy 1, policy_version 1196489 (0.0007) [2023-12-27 13:56:47,096][20381] Updated weights for policy 1, policy_version 1196499 (0.0008) [2023-12-27 13:56:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 612950016. Throughput: 0: 10388.3, 1: 10411.2. Samples: 612959492. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:47,364][19170] Avg episode reward: [(0, '882.640'), (1, '571.460')] [2023-12-27 13:56:47,567][20347] Updated weights for policy 0, policy_version 1197522 (0.0008) [2023-12-27 13:56:47,612][20347] Updated weights for policy 0, policy_version 1197532 (0.0007) [2023-12-27 13:56:47,663][20347] Updated weights for policy 0, policy_version 1197542 (0.0007) [2023-12-27 13:56:47,723][20347] Updated weights for policy 0, policy_version 1197552 (0.0007) [2023-12-27 13:56:47,793][20381] Updated weights for policy 1, policy_version 1196509 (0.0007) [2023-12-27 13:56:47,838][20381] Updated weights for policy 1, policy_version 1196519 (0.0007) [2023-12-27 13:56:47,883][20381] Updated weights for policy 1, policy_version 1196529 (0.0007) [2023-12-27 13:56:48,425][20347] Updated weights for policy 0, policy_version 1197562 (0.0008) [2023-12-27 13:56:48,488][20347] Updated weights for policy 0, policy_version 1197572 (0.0008) [2023-12-27 13:56:48,549][20347] Updated weights for policy 0, policy_version 1197582 (0.0008) [2023-12-27 13:56:48,595][20381] Updated weights for policy 1, policy_version 1196539 (0.0007) [2023-12-27 13:56:48,644][20381] Updated weights for policy 1, policy_version 1196549 (0.0007) [2023-12-27 13:56:48,693][20381] Updated weights for policy 1, policy_version 1196559 (0.0007) [2023-12-27 13:56:49,196][20347] Updated weights for policy 0, policy_version 1197592 (0.0008) [2023-12-27 13:56:49,263][20347] Updated weights for policy 0, policy_version 1197602 (0.0008) [2023-12-27 13:56:49,320][20347] Updated weights for policy 0, policy_version 1197612 (0.0008) [2023-12-27 13:56:49,457][20381] Updated weights for policy 1, policy_version 1196569 (0.0008) [2023-12-27 13:56:49,518][20381] Updated weights for policy 1, policy_version 1196579 (0.0006) [2023-12-27 13:56:49,572][20381] Updated weights for policy 1, policy_version 1196589 (0.0008) [2023-12-27 13:56:49,629][20381] Updated weights for policy 1, policy_version 1196599 (0.0006) [2023-12-27 13:56:50,090][20347] Updated weights for policy 0, policy_version 1197622 (0.0006) [2023-12-27 13:56:50,151][20347] Updated weights for policy 0, policy_version 1197632 (0.0005) [2023-12-27 13:56:50,210][20347] Updated weights for policy 0, policy_version 1197642 (0.0006) [2023-12-27 13:56:50,281][20381] Updated weights for policy 1, policy_version 1196609 (0.0008) [2023-12-27 13:56:50,340][20381] Updated weights for policy 1, policy_version 1196619 (0.0008) [2023-12-27 13:56:50,404][20381] Updated weights for policy 1, policy_version 1196629 (0.0008) [2023-12-27 13:56:50,796][20347] Updated weights for policy 0, policy_version 1197652 (0.0007) [2023-12-27 13:56:50,856][20347] Updated weights for policy 0, policy_version 1197662 (0.0009) [2023-12-27 13:56:50,917][20347] Updated weights for policy 0, policy_version 1197672 (0.0007) [2023-12-27 13:56:51,053][20381] Updated weights for policy 1, policy_version 1196639 (0.0007) [2023-12-27 13:56:51,118][20381] Updated weights for policy 1, policy_version 1196649 (0.0007) [2023-12-27 13:56:51,181][20381] Updated weights for policy 1, policy_version 1196659 (0.0006) [2023-12-27 13:56:51,610][20347] Updated weights for policy 0, policy_version 1197682 (0.0008) [2023-12-27 13:56:51,673][20347] Updated weights for policy 0, policy_version 1197692 (0.0009) [2023-12-27 13:56:51,725][20347] Updated weights for policy 0, policy_version 1197702 (0.0008) [2023-12-27 13:56:51,788][20347] Updated weights for policy 0, policy_version 1197712 (0.0007) [2023-12-27 13:56:51,955][20381] Updated weights for policy 1, policy_version 1196669 (0.0008) [2023-12-27 13:56:52,023][20381] Updated weights for policy 1, policy_version 1196679 (0.0007) [2023-12-27 13:56:52,076][20381] Updated weights for policy 1, policy_version 1196689 (0.0007) [2023-12-27 13:56:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 613048320. Throughput: 0: 10354.6, 1: 10380.7. Samples: 613019400. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:52,363][19170] Avg episode reward: [(0, '873.070'), (1, '367.270')] [2023-12-27 13:56:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001196696_306380800.pth... [2023-12-27 13:56:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001197712_306667520.pth... [2023-12-27 13:56:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001195512_306077696.pth [2023-12-27 13:56:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001196496_306356224.pth [2023-12-27 13:56:52,520][20347] Updated weights for policy 0, policy_version 1197722 (0.0008) [2023-12-27 13:56:52,575][20347] Updated weights for policy 0, policy_version 1197732 (0.0007) [2023-12-27 13:56:52,635][20347] Updated weights for policy 0, policy_version 1197742 (0.0008) [2023-12-27 13:56:52,726][20381] Updated weights for policy 1, policy_version 1196699 (0.0007) [2023-12-27 13:56:52,780][20381] Updated weights for policy 1, policy_version 1196709 (0.0007) [2023-12-27 13:56:52,836][20381] Updated weights for policy 1, policy_version 1196719 (0.0008) [2023-12-27 13:56:53,351][20347] Updated weights for policy 0, policy_version 1197752 (0.0008) [2023-12-27 13:56:53,399][20347] Updated weights for policy 0, policy_version 1197762 (0.0007) [2023-12-27 13:56:53,445][20347] Updated weights for policy 0, policy_version 1197772 (0.0006) [2023-12-27 13:56:53,526][20381] Updated weights for policy 1, policy_version 1196729 (0.0008) [2023-12-27 13:56:53,589][20381] Updated weights for policy 1, policy_version 1196739 (0.0008) [2023-12-27 13:56:53,650][20381] Updated weights for policy 1, policy_version 1196749 (0.0008) [2023-12-27 13:56:53,705][20381] Updated weights for policy 1, policy_version 1196759 (0.0008) [2023-12-27 13:56:54,079][20347] Updated weights for policy 0, policy_version 1197782 (0.0007) [2023-12-27 13:56:54,138][20347] Updated weights for policy 0, policy_version 1197792 (0.0007) [2023-12-27 13:56:54,193][20347] Updated weights for policy 0, policy_version 1197802 (0.0008) [2023-12-27 13:56:54,391][20381] Updated weights for policy 1, policy_version 1196769 (0.0005) [2023-12-27 13:56:54,456][20381] Updated weights for policy 1, policy_version 1196779 (0.0007) [2023-12-27 13:56:54,515][20381] Updated weights for policy 1, policy_version 1196789 (0.0008) [2023-12-27 13:56:54,900][20347] Updated weights for policy 0, policy_version 1197812 (0.0007) [2023-12-27 13:56:54,956][20291] KL-divergence is very high: 172.5735 [2023-12-27 13:56:54,961][20347] Updated weights for policy 0, policy_version 1197822 (0.0008) [2023-12-27 13:56:54,979][20291] KL-divergence is very high: 185.2070 [2023-12-27 13:56:55,019][20347] Updated weights for policy 0, policy_version 1197832 (0.0007) [2023-12-27 13:56:55,181][20381] Updated weights for policy 1, policy_version 1196799 (0.0006) [2023-12-27 13:56:55,241][20381] Updated weights for policy 1, policy_version 1196809 (0.0006) [2023-12-27 13:56:55,292][20381] Updated weights for policy 1, policy_version 1196819 (0.0007) [2023-12-27 13:56:55,680][20347] Updated weights for policy 0, policy_version 1197842 (0.0008) [2023-12-27 13:56:55,740][20347] Updated weights for policy 0, policy_version 1197852 (0.0006) [2023-12-27 13:56:55,796][20347] Updated weights for policy 0, policy_version 1197862 (0.0007) [2023-12-27 13:56:55,848][20347] Updated weights for policy 0, policy_version 1197872 (0.0008) [2023-12-27 13:56:55,977][20381] Updated weights for policy 1, policy_version 1196829 (0.0008) [2023-12-27 13:56:56,036][20381] Updated weights for policy 1, policy_version 1196839 (0.0008) [2023-12-27 13:56:56,090][20381] Updated weights for policy 1, policy_version 1196849 (0.0007) [2023-12-27 13:56:56,494][20347] Updated weights for policy 0, policy_version 1197882 (0.0007) [2023-12-27 13:56:56,536][20347] Updated weights for policy 0, policy_version 1197892 (0.0007) [2023-12-27 13:56:56,580][20347] Updated weights for policy 0, policy_version 1197902 (0.0007) [2023-12-27 13:56:56,748][20381] Updated weights for policy 1, policy_version 1196859 (0.0007) [2023-12-27 13:56:56,798][20381] Updated weights for policy 1, policy_version 1196869 (0.0005) [2023-12-27 13:56:56,842][20381] Updated weights for policy 1, policy_version 1196879 (0.0006) [2023-12-27 13:56:57,335][20347] Updated weights for policy 0, policy_version 1197912 (0.0006) [2023-12-27 13:56:57,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 613146624. Throughput: 0: 10358.6, 1: 10359.9. Samples: 613140756. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:56:57,363][19170] Avg episode reward: [(0, '882.780'), (1, '542.210')] [2023-12-27 13:56:57,396][20347] Updated weights for policy 0, policy_version 1197922 (0.0008) [2023-12-27 13:56:57,435][20381] Updated weights for policy 1, policy_version 1196889 (0.0006) [2023-12-27 13:56:57,459][20347] Updated weights for policy 0, policy_version 1197932 (0.0008) [2023-12-27 13:56:57,493][20381] Updated weights for policy 1, policy_version 1196899 (0.0007) [2023-12-27 13:56:57,545][20381] Updated weights for policy 1, policy_version 1196909 (0.0008) [2023-12-27 13:56:57,593][20381] Updated weights for policy 1, policy_version 1196919 (0.0008) [2023-12-27 13:56:58,127][20347] Updated weights for policy 0, policy_version 1197942 (0.0008) [2023-12-27 13:56:58,183][20347] Updated weights for policy 0, policy_version 1197952 (0.0007) [2023-12-27 13:56:58,239][20347] Updated weights for policy 0, policy_version 1197962 (0.0007) [2023-12-27 13:56:58,277][20381] Updated weights for policy 1, policy_version 1196929 (0.0008) [2023-12-27 13:56:58,346][20381] Updated weights for policy 1, policy_version 1196939 (0.0008) [2023-12-27 13:56:58,408][20381] Updated weights for policy 1, policy_version 1196949 (0.0008) [2023-12-27 13:56:58,976][20347] Updated weights for policy 0, policy_version 1197972 (0.0007) [2023-12-27 13:56:59,035][20347] Updated weights for policy 0, policy_version 1197982 (0.0008) [2023-12-27 13:56:59,099][20347] Updated weights for policy 0, policy_version 1197992 (0.0008) [2023-12-27 13:56:59,196][20381] Updated weights for policy 1, policy_version 1196959 (0.0006) [2023-12-27 13:56:59,263][20381] Updated weights for policy 1, policy_version 1196969 (0.0008) [2023-12-27 13:56:59,318][20381] Updated weights for policy 1, policy_version 1196979 (0.0009) [2023-12-27 13:56:59,812][20347] Updated weights for policy 0, policy_version 1198002 (0.0008) [2023-12-27 13:56:59,873][20347] Updated weights for policy 0, policy_version 1198012 (0.0008) [2023-12-27 13:56:59,922][20347] Updated weights for policy 0, policy_version 1198022 (0.0009) [2023-12-27 13:56:59,998][20347] Updated weights for policy 0, policy_version 1198032 (0.0008) [2023-12-27 13:57:00,047][20381] Updated weights for policy 1, policy_version 1196989 (0.0008) [2023-12-27 13:57:00,113][20381] Updated weights for policy 1, policy_version 1196999 (0.0009) [2023-12-27 13:57:00,160][20381] Updated weights for policy 1, policy_version 1197009 (0.0010) [2023-12-27 13:57:00,706][20347] Updated weights for policy 0, policy_version 1198042 (0.0008) [2023-12-27 13:57:00,744][20381] Updated weights for policy 1, policy_version 1197019 (0.0008) [2023-12-27 13:57:00,763][20347] Updated weights for policy 0, policy_version 1198052 (0.0006) [2023-12-27 13:57:00,796][20381] Updated weights for policy 1, policy_version 1197029 (0.0010) [2023-12-27 13:57:00,813][20347] Updated weights for policy 0, policy_version 1198062 (0.0006) [2023-12-27 13:57:00,857][20381] Updated weights for policy 1, policy_version 1197039 (0.0006) [2023-12-27 13:57:01,481][20347] Updated weights for policy 0, policy_version 1198072 (0.0008) [2023-12-27 13:57:01,532][20347] Updated weights for policy 0, policy_version 1198082 (0.0007) [2023-12-27 13:57:01,585][20347] Updated weights for policy 0, policy_version 1198092 (0.0005) [2023-12-27 13:57:01,639][20381] Updated weights for policy 1, policy_version 1197049 (0.0007) [2023-12-27 13:57:01,696][20381] Updated weights for policy 1, policy_version 1197059 (0.0008) [2023-12-27 13:57:01,764][20381] Updated weights for policy 1, policy_version 1197069 (0.0008) [2023-12-27 13:57:01,829][20381] Updated weights for policy 1, policy_version 1197079 (0.0007) [2023-12-27 13:57:02,209][20347] Updated weights for policy 0, policy_version 1198102 (0.0006) [2023-12-27 13:57:02,264][20347] Updated weights for policy 0, policy_version 1198112 (0.0006) [2023-12-27 13:57:02,322][20347] Updated weights for policy 0, policy_version 1198122 (0.0006) [2023-12-27 13:57:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20618.7, 300 sec: 20605.0). Total num frames: 613253120. Throughput: 0: 10316.4, 1: 10240.9. Samples: 613261040. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:02,363][19170] Avg episode reward: [(0, '824.450'), (1, '577.410')] [2023-12-27 13:57:02,495][20381] Updated weights for policy 1, policy_version 1197089 (0.0006) [2023-12-27 13:57:02,550][20381] Updated weights for policy 1, policy_version 1197099 (0.0008) [2023-12-27 13:57:02,595][20381] Updated weights for policy 1, policy_version 1197109 (0.0007) [2023-12-27 13:57:03,055][20347] Updated weights for policy 0, policy_version 1198132 (0.0005) [2023-12-27 13:57:03,114][20347] Updated weights for policy 0, policy_version 1198142 (0.0006) [2023-12-27 13:57:03,164][20381] Updated weights for policy 1, policy_version 1197119 (0.0009) [2023-12-27 13:57:03,169][20347] Updated weights for policy 0, policy_version 1198152 (0.0006) [2023-12-27 13:57:03,216][20381] Updated weights for policy 1, policy_version 1197129 (0.0010) [2023-12-27 13:57:03,267][20381] Updated weights for policy 1, policy_version 1197139 (0.0010) [2023-12-27 13:57:03,828][20347] Updated weights for policy 0, policy_version 1198162 (0.0006) [2023-12-27 13:57:03,889][20347] Updated weights for policy 0, policy_version 1198172 (0.0008) [2023-12-27 13:57:03,917][20381] Updated weights for policy 1, policy_version 1197149 (0.0009) [2023-12-27 13:57:03,948][20347] Updated weights for policy 0, policy_version 1198182 (0.0007) [2023-12-27 13:57:03,972][20381] Updated weights for policy 1, policy_version 1197159 (0.0006) [2023-12-27 13:57:04,009][20347] Updated weights for policy 0, policy_version 1198192 (0.0006) [2023-12-27 13:57:04,027][20381] Updated weights for policy 1, policy_version 1197169 (0.0006) [2023-12-27 13:57:04,705][20347] Updated weights for policy 0, policy_version 1198202 (0.0008) [2023-12-27 13:57:04,715][20381] Updated weights for policy 1, policy_version 1197179 (0.0007) [2023-12-27 13:57:04,761][20347] Updated weights for policy 0, policy_version 1198212 (0.0008) [2023-12-27 13:57:04,767][20381] Updated weights for policy 1, policy_version 1197189 (0.0006) [2023-12-27 13:57:04,815][20347] Updated weights for policy 0, policy_version 1198222 (0.0008) [2023-12-27 13:57:04,827][20381] Updated weights for policy 1, policy_version 1197199 (0.0006) [2023-12-27 13:57:05,484][20347] Updated weights for policy 0, policy_version 1198232 (0.0008) [2023-12-27 13:57:05,514][20381] Updated weights for policy 1, policy_version 1197209 (0.0005) [2023-12-27 13:57:05,533][20347] Updated weights for policy 0, policy_version 1198242 (0.0007) [2023-12-27 13:57:05,563][20381] Updated weights for policy 1, policy_version 1197219 (0.0005) [2023-12-27 13:57:05,581][20347] Updated weights for policy 0, policy_version 1198252 (0.0007) [2023-12-27 13:57:05,606][20381] Updated weights for policy 1, policy_version 1197229 (0.0005) [2023-12-27 13:57:05,663][20381] Updated weights for policy 1, policy_version 1197239 (0.0005) [2023-12-27 13:57:06,207][20381] Updated weights for policy 1, policy_version 1197249 (0.0008) [2023-12-27 13:57:06,264][20381] Updated weights for policy 1, policy_version 1197259 (0.0007) [2023-12-27 13:57:06,326][20381] Updated weights for policy 1, policy_version 1197269 (0.0008) [2023-12-27 13:57:06,375][20347] Updated weights for policy 0, policy_version 1198262 (0.0007) [2023-12-27 13:57:06,429][20347] Updated weights for policy 0, policy_version 1198272 (0.0008) [2023-12-27 13:57:06,490][20347] Updated weights for policy 0, policy_version 1198282 (0.0008) [2023-12-27 13:57:07,067][20381] Updated weights for policy 1, policy_version 1197279 (0.0008) [2023-12-27 13:57:07,088][20347] Updated weights for policy 0, policy_version 1198292 (0.0007) [2023-12-27 13:57:07,120][20381] Updated weights for policy 1, policy_version 1197289 (0.0006) [2023-12-27 13:57:07,142][20347] Updated weights for policy 0, policy_version 1198302 (0.0006) [2023-12-27 13:57:07,178][20381] Updated weights for policy 1, policy_version 1197299 (0.0007) [2023-12-27 13:57:07,196][20347] Updated weights for policy 0, policy_version 1198312 (0.0006) [2023-12-27 13:57:07,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 613359616. Throughput: 0: 10308.1, 1: 10259.0. Samples: 613323276. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:07,363][19170] Avg episode reward: [(0, '736.500'), (1, '599.540')] [2023-12-27 13:57:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001198320_306823168.pth... [2023-12-27 13:57:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001197304_306536448.pth... [2023-12-27 13:57:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001197136_306520064.pth [2023-12-27 13:57:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001196120_306233344.pth [2023-12-27 13:57:07,821][20381] Updated weights for policy 1, policy_version 1197309 (0.0006) [2023-12-27 13:57:07,878][20381] Updated weights for policy 1, policy_version 1197319 (0.0005) [2023-12-27 13:57:07,921][20347] Updated weights for policy 0, policy_version 1198322 (0.0007) [2023-12-27 13:57:07,943][20381] Updated weights for policy 1, policy_version 1197329 (0.0006) [2023-12-27 13:57:07,981][20347] Updated weights for policy 0, policy_version 1198332 (0.0008) [2023-12-27 13:57:08,035][20347] Updated weights for policy 0, policy_version 1198342 (0.0008) [2023-12-27 13:57:08,087][20347] Updated weights for policy 0, policy_version 1198352 (0.0007) [2023-12-27 13:57:08,578][20381] Updated weights for policy 1, policy_version 1197339 (0.0007) [2023-12-27 13:57:08,636][20381] Updated weights for policy 1, policy_version 1197349 (0.0011) [2023-12-27 13:57:08,695][20381] Updated weights for policy 1, policy_version 1197359 (0.0010) [2023-12-27 13:57:08,796][20347] Updated weights for policy 0, policy_version 1198362 (0.0007) [2023-12-27 13:57:08,859][20347] Updated weights for policy 0, policy_version 1198372 (0.0007) [2023-12-27 13:57:08,913][20347] Updated weights for policy 0, policy_version 1198382 (0.0008) [2023-12-27 13:57:09,267][20381] Updated weights for policy 1, policy_version 1197369 (0.0011) [2023-12-27 13:57:09,346][20381] Updated weights for policy 1, policy_version 1197379 (0.0010) [2023-12-27 13:57:09,412][20381] Updated weights for policy 1, policy_version 1197389 (0.0008) [2023-12-27 13:57:09,479][20381] Updated weights for policy 1, policy_version 1197400 (0.0007) [2023-12-27 13:57:09,686][20347] Updated weights for policy 0, policy_version 1198392 (0.0010) [2023-12-27 13:57:09,744][20347] Updated weights for policy 0, policy_version 1198402 (0.0008) [2023-12-27 13:57:09,807][20347] Updated weights for policy 0, policy_version 1198412 (0.0007) [2023-12-27 13:57:10,176][20381] Updated weights for policy 1, policy_version 1197410 (0.0008) [2023-12-27 13:57:10,231][20381] Updated weights for policy 1, policy_version 1197420 (0.0008) [2023-12-27 13:57:10,285][20381] Updated weights for policy 1, policy_version 1197430 (0.0008) [2023-12-27 13:57:10,493][20347] Updated weights for policy 0, policy_version 1198422 (0.0008) [2023-12-27 13:57:10,548][20347] Updated weights for policy 0, policy_version 1198432 (0.0007) [2023-12-27 13:57:10,603][20347] Updated weights for policy 0, policy_version 1198442 (0.0008) [2023-12-27 13:57:11,011][20381] Updated weights for policy 1, policy_version 1197440 (0.0008) [2023-12-27 13:57:11,076][20381] Updated weights for policy 1, policy_version 1197450 (0.0009) [2023-12-27 13:57:11,134][20381] Updated weights for policy 1, policy_version 1197460 (0.0009) [2023-12-27 13:57:11,323][20347] Updated weights for policy 0, policy_version 1198452 (0.0008) [2023-12-27 13:57:11,393][20347] Updated weights for policy 0, policy_version 1198462 (0.0008) [2023-12-27 13:57:11,462][20347] Updated weights for policy 0, policy_version 1198472 (0.0008) [2023-12-27 13:57:11,831][20381] Updated weights for policy 1, policy_version 1197470 (0.0008) [2023-12-27 13:57:11,887][20381] Updated weights for policy 1, policy_version 1197480 (0.0008) [2023-12-27 13:57:11,947][20381] Updated weights for policy 1, policy_version 1197490 (0.0009) [2023-12-27 13:57:12,138][20347] Updated weights for policy 0, policy_version 1198482 (0.0007) [2023-12-27 13:57:12,193][20347] Updated weights for policy 0, policy_version 1198492 (0.0008) [2023-12-27 13:57:12,249][20347] Updated weights for policy 0, policy_version 1198502 (0.0007) [2023-12-27 13:57:12,307][20347] Updated weights for policy 0, policy_version 1198512 (0.0008) [2023-12-27 13:57:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 613457920. Throughput: 0: 10267.3, 1: 10257.7. Samples: 613444744. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:12,363][19170] Avg episode reward: [(0, '746.030'), (1, '480.700')] [2023-12-27 13:57:12,718][20381] Updated weights for policy 1, policy_version 1197500 (0.0007) [2023-12-27 13:57:12,779][20381] Updated weights for policy 1, policy_version 1197510 (0.0007) [2023-12-27 13:57:12,832][20381] Updated weights for policy 1, policy_version 1197520 (0.0007) [2023-12-27 13:57:12,943][20347] Updated weights for policy 0, policy_version 1198522 (0.0007) [2023-12-27 13:57:12,994][20347] Updated weights for policy 0, policy_version 1198532 (0.0007) [2023-12-27 13:57:13,041][20347] Updated weights for policy 0, policy_version 1198542 (0.0008) [2023-12-27 13:57:13,537][20381] Updated weights for policy 1, policy_version 1197530 (0.0009) [2023-12-27 13:57:13,589][20381] Updated weights for policy 1, policy_version 1197540 (0.0008) [2023-12-27 13:57:13,640][20381] Updated weights for policy 1, policy_version 1197550 (0.0008) [2023-12-27 13:57:13,689][20381] Updated weights for policy 1, policy_version 1197560 (0.0007) [2023-12-27 13:57:13,734][20347] Updated weights for policy 0, policy_version 1198552 (0.0007) [2023-12-27 13:57:13,773][20291] KL-divergence is very high: 135.4585 [2023-12-27 13:57:13,779][20291] KL-divergence is very high: 126.4682 [2023-12-27 13:57:13,789][20347] Updated weights for policy 0, policy_version 1198562 (0.0007) [2023-12-27 13:57:13,819][20291] KL-divergence is very high: 134.8264 [2023-12-27 13:57:13,825][20291] KL-divergence is very high: 111.3898 [2023-12-27 13:57:13,850][20347] Updated weights for policy 0, policy_version 1198572 (0.0007) [2023-12-27 13:57:14,381][20381] Updated weights for policy 1, policy_version 1197570 (0.0005) [2023-12-27 13:57:14,427][20381] Updated weights for policy 1, policy_version 1197580 (0.0005) [2023-12-27 13:57:14,474][20381] Updated weights for policy 1, policy_version 1197590 (0.0005) [2023-12-27 13:57:14,555][20347] Updated weights for policy 0, policy_version 1198582 (0.0008) [2023-12-27 13:57:14,612][20347] Updated weights for policy 0, policy_version 1198592 (0.0007) [2023-12-27 13:57:14,665][20347] Updated weights for policy 0, policy_version 1198602 (0.0007) [2023-12-27 13:57:15,144][20381] Updated weights for policy 1, policy_version 1197600 (0.0006) [2023-12-27 13:57:15,209][20381] Updated weights for policy 1, policy_version 1197610 (0.0007) [2023-12-27 13:57:15,271][20381] Updated weights for policy 1, policy_version 1197620 (0.0010) [2023-12-27 13:57:15,399][20347] Updated weights for policy 0, policy_version 1198612 (0.0008) [2023-12-27 13:57:15,450][20347] Updated weights for policy 0, policy_version 1198622 (0.0010) [2023-12-27 13:57:15,512][20347] Updated weights for policy 0, policy_version 1198632 (0.0010) [2023-12-27 13:57:15,888][20381] Updated weights for policy 1, policy_version 1197630 (0.0007) [2023-12-27 13:57:15,945][20381] Updated weights for policy 1, policy_version 1197640 (0.0007) [2023-12-27 13:57:16,005][20381] Updated weights for policy 1, policy_version 1197650 (0.0007) [2023-12-27 13:57:16,150][20347] Updated weights for policy 0, policy_version 1198642 (0.0010) [2023-12-27 13:57:16,198][20347] Updated weights for policy 0, policy_version 1198652 (0.0010) [2023-12-27 13:57:16,248][20347] Updated weights for policy 0, policy_version 1198662 (0.0006) [2023-12-27 13:57:16,297][20347] Updated weights for policy 0, policy_version 1198672 (0.0005) [2023-12-27 13:57:16,650][20381] Updated weights for policy 1, policy_version 1197660 (0.0009) [2023-12-27 13:57:16,701][20381] Updated weights for policy 1, policy_version 1197670 (0.0010) [2023-12-27 13:57:16,755][20381] Updated weights for policy 1, policy_version 1197680 (0.0010) [2023-12-27 13:57:16,898][20347] Updated weights for policy 0, policy_version 1198682 (0.0010) [2023-12-27 13:57:16,947][20347] Updated weights for policy 0, policy_version 1198692 (0.0010) [2023-12-27 13:57:16,999][20347] Updated weights for policy 0, policy_version 1198702 (0.0011) [2023-12-27 13:57:17,341][20381] Updated weights for policy 1, policy_version 1197690 (0.0008) [2023-12-27 13:57:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 613556224. Throughput: 0: 10196.2, 1: 10248.1. Samples: 613568328. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:17,363][19170] Avg episode reward: [(0, '824.200'), (1, '376.380')] [2023-12-27 13:57:17,407][20381] Updated weights for policy 1, policy_version 1197700 (0.0009) [2023-12-27 13:57:17,468][20381] Updated weights for policy 1, policy_version 1197710 (0.0011) [2023-12-27 13:57:17,526][20381] Updated weights for policy 1, policy_version 1197720 (0.0011) [2023-12-27 13:57:17,626][20347] Updated weights for policy 0, policy_version 1198712 (0.0006) [2023-12-27 13:57:17,682][20347] Updated weights for policy 0, policy_version 1198722 (0.0010) [2023-12-27 13:57:17,737][20347] Updated weights for policy 0, policy_version 1198732 (0.0010) [2023-12-27 13:57:18,162][20381] Updated weights for policy 1, policy_version 1197730 (0.0008) [2023-12-27 13:57:18,223][20381] Updated weights for policy 1, policy_version 1197740 (0.0007) [2023-12-27 13:57:18,287][20381] Updated weights for policy 1, policy_version 1197750 (0.0008) [2023-12-27 13:57:18,424][20347] Updated weights for policy 0, policy_version 1198742 (0.0008) [2023-12-27 13:57:18,481][20347] Updated weights for policy 0, policy_version 1198752 (0.0005) [2023-12-27 13:57:18,537][20347] Updated weights for policy 0, policy_version 1198762 (0.0006) [2023-12-27 13:57:18,550][20291] KL-divergence is very high: 141.0533 [2023-12-27 13:57:18,556][20291] KL-divergence is very high: 113.0936 [2023-12-27 13:57:18,988][20381] Updated weights for policy 1, policy_version 1197760 (0.0006) [2023-12-27 13:57:19,053][20381] Updated weights for policy 1, policy_version 1197770 (0.0007) [2023-12-27 13:57:19,116][20381] Updated weights for policy 1, policy_version 1197780 (0.0008) [2023-12-27 13:57:19,169][20347] Updated weights for policy 0, policy_version 1198772 (0.0009) [2023-12-27 13:57:19,218][20347] Updated weights for policy 0, policy_version 1198782 (0.0006) [2023-12-27 13:57:19,283][20347] Updated weights for policy 0, policy_version 1198792 (0.0007) [2023-12-27 13:57:19,814][20381] Updated weights for policy 1, policy_version 1197790 (0.0008) [2023-12-27 13:57:19,881][20381] Updated weights for policy 1, policy_version 1197800 (0.0007) [2023-12-27 13:57:19,944][20381] Updated weights for policy 1, policy_version 1197810 (0.0008) [2023-12-27 13:57:20,022][20347] Updated weights for policy 0, policy_version 1198802 (0.0012) [2023-12-27 13:57:20,081][20347] Updated weights for policy 0, policy_version 1198812 (0.0008) [2023-12-27 13:57:20,139][20347] Updated weights for policy 0, policy_version 1198822 (0.0008) [2023-12-27 13:57:20,205][20347] Updated weights for policy 0, policy_version 1198832 (0.0008) [2023-12-27 13:57:20,678][20381] Updated weights for policy 1, policy_version 1197820 (0.0010) [2023-12-27 13:57:20,748][20381] Updated weights for policy 1, policy_version 1197830 (0.0011) [2023-12-27 13:57:20,806][20381] Updated weights for policy 1, policy_version 1197840 (0.0011) [2023-12-27 13:57:20,922][20347] Updated weights for policy 0, policy_version 1198842 (0.0008) [2023-12-27 13:57:20,984][20347] Updated weights for policy 0, policy_version 1198852 (0.0007) [2023-12-27 13:57:21,049][20347] Updated weights for policy 0, policy_version 1198862 (0.0008) [2023-12-27 13:57:21,554][20381] Updated weights for policy 1, policy_version 1197850 (0.0011) [2023-12-27 13:57:21,618][20381] Updated weights for policy 1, policy_version 1197860 (0.0010) [2023-12-27 13:57:21,677][20381] Updated weights for policy 1, policy_version 1197870 (0.0009) [2023-12-27 13:57:21,744][20381] Updated weights for policy 1, policy_version 1197880 (0.0008) [2023-12-27 13:57:21,885][20347] Updated weights for policy 0, policy_version 1198872 (0.0008) [2023-12-27 13:57:21,942][20347] Updated weights for policy 0, policy_version 1198882 (0.0007) [2023-12-27 13:57:22,006][20347] Updated weights for policy 0, policy_version 1198892 (0.0008) [2023-12-27 13:57:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 613654528. Throughput: 0: 10171.9, 1: 10219.6. Samples: 613629608. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:22,363][19170] Avg episode reward: [(0, '804.710'), (1, '412.150')] [2023-12-27 13:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001198896_306970624.pth... [2023-12-27 13:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001197712_306667520.pth [2023-12-27 13:57:22,441][20381] Updated weights for policy 1, policy_version 1197890 (0.0011) [2023-12-27 13:57:22,493][20381] Updated weights for policy 1, policy_version 1197900 (0.0009) [2023-12-27 13:57:22,552][20381] Updated weights for policy 1, policy_version 1197910 (0.0011) [2023-12-27 13:57:22,562][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001197912_306692096.pth... [2023-12-27 13:57:22,566][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001196696_306380800.pth [2023-12-27 13:57:22,689][20347] Updated weights for policy 0, policy_version 1198902 (0.0008) [2023-12-27 13:57:22,757][20347] Updated weights for policy 0, policy_version 1198912 (0.0008) [2023-12-27 13:57:22,814][20347] Updated weights for policy 0, policy_version 1198922 (0.0008) [2023-12-27 13:57:23,157][20381] Updated weights for policy 1, policy_version 1197920 (0.0007) [2023-12-27 13:57:23,210][20381] Updated weights for policy 1, policy_version 1197930 (0.0005) [2023-12-27 13:57:23,253][20381] Updated weights for policy 1, policy_version 1197940 (0.0007) [2023-12-27 13:57:23,525][20347] Updated weights for policy 0, policy_version 1198932 (0.0007) [2023-12-27 13:57:23,575][20347] Updated weights for policy 0, policy_version 1198942 (0.0005) [2023-12-27 13:57:23,627][20347] Updated weights for policy 0, policy_version 1198952 (0.0005) [2023-12-27 13:57:23,887][20381] Updated weights for policy 1, policy_version 1197950 (0.0007) [2023-12-27 13:57:23,945][20381] Updated weights for policy 1, policy_version 1197960 (0.0008) [2023-12-27 13:57:24,007][20381] Updated weights for policy 1, policy_version 1197970 (0.0008) [2023-12-27 13:57:24,272][20347] Updated weights for policy 0, policy_version 1198962 (0.0005) [2023-12-27 13:57:24,334][20347] Updated weights for policy 0, policy_version 1198972 (0.0005) [2023-12-27 13:57:24,389][20347] Updated weights for policy 0, policy_version 1198982 (0.0005) [2023-12-27 13:57:24,447][20347] Updated weights for policy 0, policy_version 1198992 (0.0010) [2023-12-27 13:57:24,665][20381] Updated weights for policy 1, policy_version 1197980 (0.0007) [2023-12-27 13:57:24,713][20381] Updated weights for policy 1, policy_version 1197990 (0.0005) [2023-12-27 13:57:24,763][20381] Updated weights for policy 1, policy_version 1198000 (0.0005) [2023-12-27 13:57:24,989][20347] Updated weights for policy 0, policy_version 1199002 (0.0010) [2023-12-27 13:57:25,041][20347] Updated weights for policy 0, policy_version 1199012 (0.0010) [2023-12-27 13:57:25,090][20347] Updated weights for policy 0, policy_version 1199022 (0.0010) [2023-12-27 13:57:25,332][20381] Updated weights for policy 1, policy_version 1198010 (0.0005) [2023-12-27 13:57:25,388][20381] Updated weights for policy 1, policy_version 1198020 (0.0007) [2023-12-27 13:57:25,449][20381] Updated weights for policy 1, policy_version 1198030 (0.0007) [2023-12-27 13:57:25,499][20381] Updated weights for policy 1, policy_version 1198040 (0.0006) [2023-12-27 13:57:25,734][20347] Updated weights for policy 0, policy_version 1199032 (0.0011) [2023-12-27 13:57:25,791][20347] Updated weights for policy 0, policy_version 1199042 (0.0008) [2023-12-27 13:57:25,849][20347] Updated weights for policy 0, policy_version 1199052 (0.0007) [2023-12-27 13:57:26,113][20381] Updated weights for policy 1, policy_version 1198050 (0.0005) [2023-12-27 13:57:26,169][20381] Updated weights for policy 1, policy_version 1198060 (0.0005) [2023-12-27 13:57:26,227][20381] Updated weights for policy 1, policy_version 1198070 (0.0007) [2023-12-27 13:57:26,544][20347] Updated weights for policy 0, policy_version 1199062 (0.0007) [2023-12-27 13:57:26,597][20347] Updated weights for policy 0, policy_version 1199072 (0.0005) [2023-12-27 13:57:26,648][20347] Updated weights for policy 0, policy_version 1199082 (0.0005) [2023-12-27 13:57:26,800][20381] Updated weights for policy 1, policy_version 1198080 (0.0008) [2023-12-27 13:57:26,856][20381] Updated weights for policy 1, policy_version 1198090 (0.0008) [2023-12-27 13:57:26,920][20381] Updated weights for policy 1, policy_version 1198100 (0.0008) [2023-12-27 13:57:27,239][20347] Updated weights for policy 0, policy_version 1199092 (0.0007) [2023-12-27 13:57:27,291][20347] Updated weights for policy 0, policy_version 1199102 (0.0008) [2023-12-27 13:57:27,353][20347] Updated weights for policy 0, policy_version 1199112 (0.0008) [2023-12-27 13:57:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 613761024. Throughput: 0: 10158.9, 1: 10225.4. Samples: 613753908. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:27,363][19170] Avg episode reward: [(0, '853.450'), (1, '259.010')] [2023-12-27 13:57:27,611][20381] Updated weights for policy 1, policy_version 1198110 (0.0007) [2023-12-27 13:57:27,664][20381] Updated weights for policy 1, policy_version 1198120 (0.0007) [2023-12-27 13:57:27,720][20381] Updated weights for policy 1, policy_version 1198130 (0.0007) [2023-12-27 13:57:28,016][20347] Updated weights for policy 0, policy_version 1199122 (0.0007) [2023-12-27 13:57:28,077][20347] Updated weights for policy 0, policy_version 1199132 (0.0009) [2023-12-27 13:57:28,128][20347] Updated weights for policy 0, policy_version 1199142 (0.0009) [2023-12-27 13:57:28,183][20347] Updated weights for policy 0, policy_version 1199152 (0.0005) [2023-12-27 13:57:28,401][20381] Updated weights for policy 1, policy_version 1198140 (0.0007) [2023-12-27 13:57:28,462][20381] Updated weights for policy 1, policy_version 1198150 (0.0007) [2023-12-27 13:57:28,520][20381] Updated weights for policy 1, policy_version 1198160 (0.0009) [2023-12-27 13:57:28,783][20347] Updated weights for policy 0, policy_version 1199162 (0.0007) [2023-12-27 13:57:28,839][20347] Updated weights for policy 0, policy_version 1199172 (0.0008) [2023-12-27 13:57:28,887][20347] Updated weights for policy 0, policy_version 1199182 (0.0008) [2023-12-27 13:57:29,208][20381] Updated weights for policy 1, policy_version 1198170 (0.0008) [2023-12-27 13:57:29,265][20381] Updated weights for policy 1, policy_version 1198180 (0.0008) [2023-12-27 13:57:29,316][20381] Updated weights for policy 1, policy_version 1198190 (0.0006) [2023-12-27 13:57:29,378][20381] Updated weights for policy 1, policy_version 1198200 (0.0008) [2023-12-27 13:57:29,608][20347] Updated weights for policy 0, policy_version 1199192 (0.0009) [2023-12-27 13:57:29,667][20347] Updated weights for policy 0, policy_version 1199202 (0.0006) [2023-12-27 13:57:29,716][20347] Updated weights for policy 0, policy_version 1199212 (0.0005) [2023-12-27 13:57:30,008][20381] Updated weights for policy 1, policy_version 1198210 (0.0006) [2023-12-27 13:57:30,075][20381] Updated weights for policy 1, policy_version 1198220 (0.0005) [2023-12-27 13:57:30,143][20381] Updated weights for policy 1, policy_version 1198230 (0.0006) [2023-12-27 13:57:30,403][20347] Updated weights for policy 0, policy_version 1199222 (0.0007) [2023-12-27 13:57:30,463][20347] Updated weights for policy 0, policy_version 1199232 (0.0007) [2023-12-27 13:57:30,517][20347] Updated weights for policy 0, policy_version 1199242 (0.0007) [2023-12-27 13:57:30,754][20381] Updated weights for policy 1, policy_version 1198240 (0.0009) [2023-12-27 13:57:30,805][20381] Updated weights for policy 1, policy_version 1198250 (0.0010) [2023-12-27 13:57:30,849][20381] Updated weights for policy 1, policy_version 1198260 (0.0010) [2023-12-27 13:57:31,198][20347] Updated weights for policy 0, policy_version 1199252 (0.0008) [2023-12-27 13:57:31,260][20347] Updated weights for policy 0, policy_version 1199262 (0.0008) [2023-12-27 13:57:31,322][20347] Updated weights for policy 0, policy_version 1199272 (0.0007) [2023-12-27 13:57:31,518][20381] Updated weights for policy 1, policy_version 1198270 (0.0007) [2023-12-27 13:57:31,584][20381] Updated weights for policy 1, policy_version 1198280 (0.0006) [2023-12-27 13:57:31,649][20381] Updated weights for policy 1, policy_version 1198290 (0.0008) [2023-12-27 13:57:32,024][20347] Updated weights for policy 0, policy_version 1199282 (0.0009) [2023-12-27 13:57:32,071][20347] Updated weights for policy 0, policy_version 1199292 (0.0005) [2023-12-27 13:57:32,072][20291] KL-divergence is very high: 391.2419 [2023-12-27 13:57:32,078][20291] KL-divergence is very high: 386.0847 [2023-12-27 13:57:32,118][20291] KL-divergence is very high: 163.9552 [2023-12-27 13:57:32,125][20291] KL-divergence is very high: 129.8221 [2023-12-27 13:57:32,130][20347] Updated weights for policy 0, policy_version 1199302 (0.0005) [2023-12-27 13:57:32,187][20347] Updated weights for policy 0, policy_version 1199312 (0.0005) [2023-12-27 13:57:32,271][20381] Updated weights for policy 1, policy_version 1198300 (0.0009) [2023-12-27 13:57:32,319][20381] Updated weights for policy 1, policy_version 1198310 (0.0009) [2023-12-27 13:57:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 613867520. Throughput: 0: 10149.0, 1: 10302.3. Samples: 613879800. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:32,363][19170] Avg episode reward: [(0, '892.350'), (1, '138.230')] [2023-12-27 13:57:32,375][20381] Updated weights for policy 1, policy_version 1198320 (0.0007) [2023-12-27 13:57:32,806][20291] KL-divergence is very high: 193.7236 [2023-12-27 13:57:32,811][20347] Updated weights for policy 0, policy_version 1199322 (0.0010) [2023-12-27 13:57:32,852][20291] KL-divergence is very high: 116.7440 [2023-12-27 13:57:32,869][20347] Updated weights for policy 0, policy_version 1199332 (0.0010) [2023-12-27 13:57:32,929][20347] Updated weights for policy 0, policy_version 1199342 (0.0006) [2023-12-27 13:57:33,041][20381] Updated weights for policy 1, policy_version 1198330 (0.0007) [2023-12-27 13:57:33,094][20381] Updated weights for policy 1, policy_version 1198340 (0.0005) [2023-12-27 13:57:33,147][20381] Updated weights for policy 1, policy_version 1198350 (0.0006) [2023-12-27 13:57:33,211][20381] Updated weights for policy 1, policy_version 1198360 (0.0006) [2023-12-27 13:57:33,560][20347] Updated weights for policy 0, policy_version 1199352 (0.0008) [2023-12-27 13:57:33,622][20347] Updated weights for policy 0, policy_version 1199362 (0.0008) [2023-12-27 13:57:33,665][20347] Updated weights for policy 0, policy_version 1199372 (0.0006) [2023-12-27 13:57:33,761][20381] Updated weights for policy 1, policy_version 1198370 (0.0005) [2023-12-27 13:57:33,817][20381] Updated weights for policy 1, policy_version 1198380 (0.0005) [2023-12-27 13:57:33,872][20381] Updated weights for policy 1, policy_version 1198390 (0.0008) [2023-12-27 13:57:34,338][20347] Updated weights for policy 0, policy_version 1199382 (0.0007) [2023-12-27 13:57:34,404][20347] Updated weights for policy 0, policy_version 1199392 (0.0007) [2023-12-27 13:57:34,464][20347] Updated weights for policy 0, policy_version 1199402 (0.0008) [2023-12-27 13:57:34,545][20381] Updated weights for policy 1, policy_version 1198400 (0.0009) [2023-12-27 13:57:34,602][20381] Updated weights for policy 1, policy_version 1198410 (0.0011) [2023-12-27 13:57:34,662][20381] Updated weights for policy 1, policy_version 1198420 (0.0010) [2023-12-27 13:57:35,139][20347] Updated weights for policy 0, policy_version 1199412 (0.0008) [2023-12-27 13:57:35,193][20347] Updated weights for policy 0, policy_version 1199422 (0.0005) [2023-12-27 13:57:35,244][20347] Updated weights for policy 0, policy_version 1199432 (0.0005) [2023-12-27 13:57:35,358][20381] Updated weights for policy 1, policy_version 1198430 (0.0011) [2023-12-27 13:57:35,406][20381] Updated weights for policy 1, policy_version 1198440 (0.0010) [2023-12-27 13:57:35,461][20381] Updated weights for policy 1, policy_version 1198450 (0.0010) [2023-12-27 13:57:35,800][20347] Updated weights for policy 0, policy_version 1199442 (0.0006) [2023-12-27 13:57:35,851][20347] Updated weights for policy 0, policy_version 1199452 (0.0010) [2023-12-27 13:57:35,903][20347] Updated weights for policy 0, policy_version 1199462 (0.0007) [2023-12-27 13:57:35,961][20347] Updated weights for policy 0, policy_version 1199472 (0.0005) [2023-12-27 13:57:36,100][20381] Updated weights for policy 1, policy_version 1198460 (0.0011) [2023-12-27 13:57:36,165][20381] Updated weights for policy 1, policy_version 1198470 (0.0011) [2023-12-27 13:57:36,221][20381] Updated weights for policy 1, policy_version 1198480 (0.0011) [2023-12-27 13:57:36,626][20347] Updated weights for policy 0, policy_version 1199482 (0.0011) [2023-12-27 13:57:36,686][20347] Updated weights for policy 0, policy_version 1199492 (0.0011) [2023-12-27 13:57:36,738][20347] Updated weights for policy 0, policy_version 1199502 (0.0010) [2023-12-27 13:57:36,887][20381] Updated weights for policy 1, policy_version 1198490 (0.0010) [2023-12-27 13:57:36,941][20381] Updated weights for policy 1, policy_version 1198500 (0.0005) [2023-12-27 13:57:36,987][20381] Updated weights for policy 1, policy_version 1198510 (0.0005) [2023-12-27 13:57:37,045][20381] Updated weights for policy 1, policy_version 1198520 (0.0006) [2023-12-27 13:57:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 613974016. Throughput: 0: 10190.9, 1: 10340.2. Samples: 613943300. Policy #0 lag: (min: 46.0, avg: 55.8, max: 56.0) [2023-12-27 13:57:37,364][19170] Avg episode reward: [(0, '833.340'), (1, '343.760')] [2023-12-27 13:57:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001198520_306847744.pth... [2023-12-27 13:57:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001197304_306536448.pth [2023-12-27 13:57:37,399][20347] Updated weights for policy 0, policy_version 1199512 (0.0010) [2023-12-27 13:57:37,460][20347] Updated weights for policy 0, policy_version 1199522 (0.0008) [2023-12-27 13:57:37,509][20347] Updated weights for policy 0, policy_version 1199532 (0.0009) [2023-12-27 13:57:37,528][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001199536_307134464.pth... [2023-12-27 13:57:37,532][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001198320_306823168.pth [2023-12-27 13:57:37,650][20381] Updated weights for policy 1, policy_version 1198530 (0.0011) [2023-12-27 13:57:37,716][20381] Updated weights for policy 1, policy_version 1198540 (0.0011) [2023-12-27 13:57:37,772][20381] Updated weights for policy 1, policy_version 1198550 (0.0011) [2023-12-27 13:57:38,193][20347] Updated weights for policy 0, policy_version 1199542 (0.0007) [2023-12-27 13:57:38,258][20347] Updated weights for policy 0, policy_version 1199552 (0.0005) [2023-12-27 13:57:38,316][20347] Updated weights for policy 0, policy_version 1199562 (0.0006) [2023-12-27 13:57:38,430][20381] Updated weights for policy 1, policy_version 1198560 (0.0011) [2023-12-27 13:57:38,490][20381] Updated weights for policy 1, policy_version 1198570 (0.0011) [2023-12-27 13:57:38,551][20381] Updated weights for policy 1, policy_version 1198580 (0.0011) [2023-12-27 13:57:39,009][20347] Updated weights for policy 0, policy_version 1199572 (0.0007) [2023-12-27 13:57:39,064][20347] Updated weights for policy 0, policy_version 1199582 (0.0010) [2023-12-27 13:57:39,115][20347] Updated weights for policy 0, policy_version 1199592 (0.0010) [2023-12-27 13:57:39,253][20381] Updated weights for policy 1, policy_version 1198590 (0.0010) [2023-12-27 13:57:39,309][20381] Updated weights for policy 1, policy_version 1198600 (0.0010) [2023-12-27 13:57:39,366][20381] Updated weights for policy 1, policy_version 1198610 (0.0011) [2023-12-27 13:57:39,808][20347] Updated weights for policy 0, policy_version 1199602 (0.0010) [2023-12-27 13:57:39,875][20347] Updated weights for policy 0, policy_version 1199612 (0.0008) [2023-12-27 13:57:39,940][20347] Updated weights for policy 0, policy_version 1199622 (0.0006) [2023-12-27 13:57:39,998][20347] Updated weights for policy 0, policy_version 1199632 (0.0006) [2023-12-27 13:57:40,097][20381] Updated weights for policy 1, policy_version 1198620 (0.0011) [2023-12-27 13:57:40,150][20381] Updated weights for policy 1, policy_version 1198630 (0.0011) [2023-12-27 13:57:40,207][20381] Updated weights for policy 1, policy_version 1198640 (0.0011) [2023-12-27 13:57:40,644][20347] Updated weights for policy 0, policy_version 1199642 (0.0007) [2023-12-27 13:57:40,701][20347] Updated weights for policy 0, policy_version 1199652 (0.0007) [2023-12-27 13:57:40,759][20347] Updated weights for policy 0, policy_version 1199662 (0.0007) [2023-12-27 13:57:40,906][20381] Updated weights for policy 1, policy_version 1198650 (0.0010) [2023-12-27 13:57:40,958][20381] Updated weights for policy 1, policy_version 1198660 (0.0010) [2023-12-27 13:57:41,008][20381] Updated weights for policy 1, policy_version 1198670 (0.0009) [2023-12-27 13:57:41,071][20381] Updated weights for policy 1, policy_version 1198680 (0.0007) [2023-12-27 13:57:41,534][20347] Updated weights for policy 0, policy_version 1199672 (0.0008) [2023-12-27 13:57:41,597][20347] Updated weights for policy 0, policy_version 1199682 (0.0008) [2023-12-27 13:57:41,659][20347] Updated weights for policy 0, policy_version 1199692 (0.0008) [2023-12-27 13:57:41,760][20381] Updated weights for policy 1, policy_version 1198690 (0.0008) [2023-12-27 13:57:41,823][20381] Updated weights for policy 1, policy_version 1198700 (0.0008) [2023-12-27 13:57:41,883][20381] Updated weights for policy 1, policy_version 1198710 (0.0008) [2023-12-27 13:57:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 614072320. Throughput: 0: 10212.5, 1: 10382.2. Samples: 614067512. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:57:42,363][19170] Avg episode reward: [(0, '803.980'), (1, '91.190')] [2023-12-27 13:57:42,377][20347] Updated weights for policy 0, policy_version 1199702 (0.0009) [2023-12-27 13:57:42,442][20347] Updated weights for policy 0, policy_version 1199712 (0.0010) [2023-12-27 13:57:42,497][20347] Updated weights for policy 0, policy_version 1199722 (0.0010) [2023-12-27 13:57:42,634][20381] Updated weights for policy 1, policy_version 1198720 (0.0007) [2023-12-27 13:57:42,693][20381] Updated weights for policy 1, policy_version 1198730 (0.0008) [2023-12-27 13:57:42,745][20381] Updated weights for policy 1, policy_version 1198740 (0.0008) [2023-12-27 13:57:43,193][20347] Updated weights for policy 0, policy_version 1199732 (0.0008) [2023-12-27 13:57:43,245][20347] Updated weights for policy 0, policy_version 1199742 (0.0005) [2023-12-27 13:57:43,296][20347] Updated weights for policy 0, policy_version 1199752 (0.0005) [2023-12-27 13:57:43,415][20381] Updated weights for policy 1, policy_version 1198750 (0.0008) [2023-12-27 13:57:43,471][20381] Updated weights for policy 1, policy_version 1198760 (0.0008) [2023-12-27 13:57:43,529][20381] Updated weights for policy 1, policy_version 1198770 (0.0008) [2023-12-27 13:57:43,898][20347] Updated weights for policy 0, policy_version 1199762 (0.0006) [2023-12-27 13:57:43,947][20347] Updated weights for policy 0, policy_version 1199772 (0.0010) [2023-12-27 13:57:43,993][20347] Updated weights for policy 0, policy_version 1199782 (0.0005) [2023-12-27 13:57:44,046][20347] Updated weights for policy 0, policy_version 1199792 (0.0005) [2023-12-27 13:57:44,116][20381] Updated weights for policy 1, policy_version 1198780 (0.0007) [2023-12-27 13:57:44,172][20381] Updated weights for policy 1, policy_version 1198790 (0.0005) [2023-12-27 13:57:44,231][20381] Updated weights for policy 1, policy_version 1198800 (0.0005) [2023-12-27 13:57:44,613][20347] Updated weights for policy 0, policy_version 1199802 (0.0010) [2023-12-27 13:57:44,658][20347] Updated weights for policy 0, policy_version 1199812 (0.0010) [2023-12-27 13:57:44,713][20347] Updated weights for policy 0, policy_version 1199822 (0.0010) [2023-12-27 13:57:44,838][20381] Updated weights for policy 1, policy_version 1198810 (0.0008) [2023-12-27 13:57:44,901][20381] Updated weights for policy 1, policy_version 1198820 (0.0008) [2023-12-27 13:57:44,967][20381] Updated weights for policy 1, policy_version 1198830 (0.0007) [2023-12-27 13:57:45,024][20381] Updated weights for policy 1, policy_version 1198840 (0.0011) [2023-12-27 13:57:45,453][20347] Updated weights for policy 0, policy_version 1199832 (0.0008) [2023-12-27 13:57:45,517][20347] Updated weights for policy 0, policy_version 1199842 (0.0008) [2023-12-27 13:57:45,571][20347] Updated weights for policy 0, policy_version 1199852 (0.0008) [2023-12-27 13:57:45,635][20381] Updated weights for policy 1, policy_version 1198850 (0.0011) [2023-12-27 13:57:45,690][20381] Updated weights for policy 1, policy_version 1198860 (0.0010) [2023-12-27 13:57:45,749][20381] Updated weights for policy 1, policy_version 1198870 (0.0011) [2023-12-27 13:57:46,156][20347] Updated weights for policy 0, policy_version 1199862 (0.0008) [2023-12-27 13:57:46,210][20347] Updated weights for policy 0, policy_version 1199872 (0.0010) [2023-12-27 13:57:46,295][20347] Updated weights for policy 0, policy_version 1199882 (0.0010) [2023-12-27 13:57:46,435][20381] Updated weights for policy 1, policy_version 1198880 (0.0010) [2023-12-27 13:57:46,487][20381] Updated weights for policy 1, policy_version 1198890 (0.0010) [2023-12-27 13:57:46,532][20381] Updated weights for policy 1, policy_version 1198900 (0.0010) [2023-12-27 13:57:46,864][20347] Updated weights for policy 0, policy_version 1199892 (0.0008) [2023-12-27 13:57:46,919][20347] Updated weights for policy 0, policy_version 1199902 (0.0005) [2023-12-27 13:57:46,965][20347] Updated weights for policy 0, policy_version 1199912 (0.0007) [2023-12-27 13:57:47,237][20381] Updated weights for policy 1, policy_version 1198910 (0.0010) [2023-12-27 13:57:47,296][20381] Updated weights for policy 1, policy_version 1198920 (0.0011) [2023-12-27 13:57:47,360][20381] Updated weights for policy 1, policy_version 1198930 (0.0011) [2023-12-27 13:57:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 614178816. Throughput: 0: 10300.1, 1: 10426.5. Samples: 614193740. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:57:47,363][19170] Avg episode reward: [(0, '794.360'), (1, '82.640')] [2023-12-27 13:57:47,622][20347] Updated weights for policy 0, policy_version 1199922 (0.0010) [2023-12-27 13:57:47,674][20347] Updated weights for policy 0, policy_version 1199932 (0.0010) [2023-12-27 13:57:47,731][20347] Updated weights for policy 0, policy_version 1199942 (0.0010) [2023-12-27 13:57:47,788][20347] Updated weights for policy 0, policy_version 1199952 (0.0010) [2023-12-27 13:57:48,041][20381] Updated weights for policy 1, policy_version 1198940 (0.0011) [2023-12-27 13:57:48,099][20381] Updated weights for policy 1, policy_version 1198950 (0.0010) [2023-12-27 13:57:48,153][20381] Updated weights for policy 1, policy_version 1198960 (0.0010) [2023-12-27 13:57:48,480][20347] Updated weights for policy 0, policy_version 1199962 (0.0010) [2023-12-27 13:57:48,539][20347] Updated weights for policy 0, policy_version 1199972 (0.0010) [2023-12-27 13:57:48,598][20347] Updated weights for policy 0, policy_version 1199982 (0.0011) [2023-12-27 13:57:48,789][20381] Updated weights for policy 1, policy_version 1198970 (0.0010) [2023-12-27 13:57:48,849][20381] Updated weights for policy 1, policy_version 1198980 (0.0005) [2023-12-27 13:57:48,902][20381] Updated weights for policy 1, policy_version 1198990 (0.0005) [2023-12-27 13:57:48,965][20381] Updated weights for policy 1, policy_version 1199000 (0.0010) [2023-12-27 13:57:49,269][20291] KL-divergence is very high: 115.3822 [2023-12-27 13:57:49,289][20291] KL-divergence is very high: 134.4270 [2023-12-27 13:57:49,295][20347] Updated weights for policy 0, policy_version 1199992 (0.0008) [2023-12-27 13:57:49,321][20291] KL-divergence is very high: 100.0169 [2023-12-27 13:57:49,362][20347] Updated weights for policy 0, policy_version 1200002 (0.0009) [2023-12-27 13:57:49,406][20291] KL-divergence is very high: 271.2732 [2023-12-27 13:57:49,423][20347] Updated weights for policy 0, policy_version 1200012 (0.0006) [2023-12-27 13:57:49,667][20381] Updated weights for policy 1, policy_version 1199010 (0.0009) [2023-12-27 13:57:49,727][20381] Updated weights for policy 1, policy_version 1199020 (0.0008) [2023-12-27 13:57:49,787][20381] Updated weights for policy 1, policy_version 1199030 (0.0008) [2023-12-27 13:57:50,090][20347] Updated weights for policy 0, policy_version 1200022 (0.0007) [2023-12-27 13:57:50,156][20347] Updated weights for policy 0, policy_version 1200032 (0.0008) [2023-12-27 13:57:50,226][20347] Updated weights for policy 0, policy_version 1200042 (0.0008) [2023-12-27 13:57:50,442][20381] Updated weights for policy 1, policy_version 1199040 (0.0008) [2023-12-27 13:57:50,497][20381] Updated weights for policy 1, policy_version 1199050 (0.0007) [2023-12-27 13:57:50,557][20381] Updated weights for policy 1, policy_version 1199060 (0.0007) [2023-12-27 13:57:50,877][20347] Updated weights for policy 0, policy_version 1200052 (0.0008) [2023-12-27 13:57:50,931][20347] Updated weights for policy 0, policy_version 1200062 (0.0008) [2023-12-27 13:57:50,991][20347] Updated weights for policy 0, policy_version 1200072 (0.0007) [2023-12-27 13:57:51,228][20381] Updated weights for policy 1, policy_version 1199070 (0.0007) [2023-12-27 13:57:51,294][20381] Updated weights for policy 1, policy_version 1199080 (0.0008) [2023-12-27 13:57:51,359][20381] Updated weights for policy 1, policy_version 1199090 (0.0007) [2023-12-27 13:57:51,720][20347] Updated weights for policy 0, policy_version 1200082 (0.0008) [2023-12-27 13:57:51,780][20347] Updated weights for policy 0, policy_version 1200092 (0.0008) [2023-12-27 13:57:51,837][20347] Updated weights for policy 0, policy_version 1200102 (0.0007) [2023-12-27 13:57:51,898][20347] Updated weights for policy 0, policy_version 1200112 (0.0008) [2023-12-27 13:57:52,048][20381] Updated weights for policy 1, policy_version 1199100 (0.0007) [2023-12-27 13:57:52,099][20381] Updated weights for policy 1, policy_version 1199110 (0.0007) [2023-12-27 13:57:52,162][20381] Updated weights for policy 1, policy_version 1199120 (0.0007) [2023-12-27 13:57:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 614285312. Throughput: 0: 10300.1, 1: 10399.4. Samples: 614254752. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:57:52,363][19170] Avg episode reward: [(0, '862.850'), (1, '85.730')] [2023-12-27 13:57:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001200112_307281920.pth... [2023-12-27 13:57:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001199128_307003392.pth... [2023-12-27 13:57:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001198896_306970624.pth [2023-12-27 13:57:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001197912_306692096.pth [2023-12-27 13:57:52,621][20347] Updated weights for policy 0, policy_version 1200122 (0.0008) [2023-12-27 13:57:52,670][20347] Updated weights for policy 0, policy_version 1200132 (0.0007) [2023-12-27 13:57:52,715][20347] Updated weights for policy 0, policy_version 1200142 (0.0008) [2023-12-27 13:57:52,822][20381] Updated weights for policy 1, policy_version 1199130 (0.0006) [2023-12-27 13:57:52,877][20381] Updated weights for policy 1, policy_version 1199140 (0.0007) [2023-12-27 13:57:52,936][20381] Updated weights for policy 1, policy_version 1199150 (0.0007) [2023-12-27 13:57:52,990][20381] Updated weights for policy 1, policy_version 1199160 (0.0008) [2023-12-27 13:57:53,413][20347] Updated weights for policy 0, policy_version 1200152 (0.0008) [2023-12-27 13:57:53,465][20347] Updated weights for policy 0, policy_version 1200162 (0.0007) [2023-12-27 13:57:53,515][20347] Updated weights for policy 0, policy_version 1200172 (0.0008) [2023-12-27 13:57:53,668][20381] Updated weights for policy 1, policy_version 1199170 (0.0008) [2023-12-27 13:57:53,727][20381] Updated weights for policy 1, policy_version 1199180 (0.0009) [2023-12-27 13:57:53,776][20381] Updated weights for policy 1, policy_version 1199190 (0.0006) [2023-12-27 13:57:54,227][20347] Updated weights for policy 0, policy_version 1200182 (0.0007) [2023-12-27 13:57:54,278][20347] Updated weights for policy 0, policy_version 1200192 (0.0007) [2023-12-27 13:57:54,328][20347] Updated weights for policy 0, policy_version 1200202 (0.0009) [2023-12-27 13:57:54,400][20381] Updated weights for policy 1, policy_version 1199200 (0.0005) [2023-12-27 13:57:54,449][20381] Updated weights for policy 1, policy_version 1199210 (0.0005) [2023-12-27 13:57:54,499][20381] Updated weights for policy 1, policy_version 1199220 (0.0005) [2023-12-27 13:57:55,016][20347] Updated weights for policy 0, policy_version 1200212 (0.0007) [2023-12-27 13:57:55,062][20347] Updated weights for policy 0, policy_version 1200222 (0.0008) [2023-12-27 13:57:55,109][20347] Updated weights for policy 0, policy_version 1200232 (0.0008) [2023-12-27 13:57:55,132][20381] Updated weights for policy 1, policy_version 1199230 (0.0007) [2023-12-27 13:57:55,189][20381] Updated weights for policy 1, policy_version 1199240 (0.0007) [2023-12-27 13:57:55,246][20381] Updated weights for policy 1, policy_version 1199250 (0.0005) [2023-12-27 13:57:55,824][20347] Updated weights for policy 0, policy_version 1200242 (0.0006) [2023-12-27 13:57:55,876][20347] Updated weights for policy 0, policy_version 1200252 (0.0007) [2023-12-27 13:57:55,905][20381] Updated weights for policy 1, policy_version 1199260 (0.0006) [2023-12-27 13:57:55,921][20347] Updated weights for policy 0, policy_version 1200262 (0.0007) [2023-12-27 13:57:55,959][20381] Updated weights for policy 1, policy_version 1199270 (0.0006) [2023-12-27 13:57:55,966][20291] KL-divergence is very high: 142.8479 [2023-12-27 13:57:55,978][20347] Updated weights for policy 0, policy_version 1200272 (0.0008) [2023-12-27 13:57:56,007][20381] Updated weights for policy 1, policy_version 1199280 (0.0007) [2023-12-27 13:57:56,625][20291] KL-divergence is very high: 159.6144 [2023-12-27 13:57:56,669][20291] KL-divergence is very high: 139.5245 [2023-12-27 13:57:56,679][20347] Updated weights for policy 0, policy_version 1200282 (0.0007) [2023-12-27 13:57:56,686][20381] Updated weights for policy 1, policy_version 1199290 (0.0006) [2023-12-27 13:57:56,736][20347] Updated weights for policy 0, policy_version 1200292 (0.0007) [2023-12-27 13:57:56,737][20381] Updated weights for policy 1, policy_version 1199300 (0.0005) [2023-12-27 13:57:56,791][20347] Updated weights for policy 0, policy_version 1200302 (0.0007) [2023-12-27 13:57:56,795][20381] Updated weights for policy 1, policy_version 1199310 (0.0005) [2023-12-27 13:57:56,844][20381] Updated weights for policy 1, policy_version 1199320 (0.0005) [2023-12-27 13:57:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 614383616. Throughput: 0: 10297.3, 1: 10408.2. Samples: 614376488. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:57:57,363][19170] Avg episode reward: [(0, '892.200'), (1, '180.420')] [2023-12-27 13:57:57,384][20381] Updated weights for policy 1, policy_version 1199330 (0.0009) [2023-12-27 13:57:57,428][20381] Updated weights for policy 1, policy_version 1199340 (0.0005) [2023-12-27 13:57:57,464][20347] Updated weights for policy 0, policy_version 1200312 (0.0006) [2023-12-27 13:57:57,482][20381] Updated weights for policy 1, policy_version 1199350 (0.0010) [2023-12-27 13:57:57,509][20347] Updated weights for policy 0, policy_version 1200322 (0.0006) [2023-12-27 13:57:57,565][20347] Updated weights for policy 0, policy_version 1200332 (0.0005) [2023-12-27 13:57:58,205][20381] Updated weights for policy 1, policy_version 1199360 (0.0009) [2023-12-27 13:57:58,225][20347] Updated weights for policy 0, policy_version 1200342 (0.0006) [2023-12-27 13:57:58,264][20381] Updated weights for policy 1, policy_version 1199370 (0.0008) [2023-12-27 13:57:58,286][20347] Updated weights for policy 0, policy_version 1200352 (0.0007) [2023-12-27 13:57:58,320][20381] Updated weights for policy 1, policy_version 1199380 (0.0006) [2023-12-27 13:57:58,346][20347] Updated weights for policy 0, policy_version 1200362 (0.0008) [2023-12-27 13:57:59,076][20381] Updated weights for policy 1, policy_version 1199390 (0.0009) [2023-12-27 13:57:59,119][20347] Updated weights for policy 0, policy_version 1200372 (0.0008) [2023-12-27 13:57:59,139][20381] Updated weights for policy 1, policy_version 1199400 (0.0010) [2023-12-27 13:57:59,171][20347] Updated weights for policy 0, policy_version 1200382 (0.0006) [2023-12-27 13:57:59,190][20381] Updated weights for policy 1, policy_version 1199410 (0.0008) [2023-12-27 13:57:59,235][20347] Updated weights for policy 0, policy_version 1200392 (0.0007) [2023-12-27 13:57:59,847][20347] Updated weights for policy 0, policy_version 1200402 (0.0007) [2023-12-27 13:57:59,904][20347] Updated weights for policy 0, policy_version 1200412 (0.0007) [2023-12-27 13:57:59,907][20381] Updated weights for policy 1, policy_version 1199420 (0.0008) [2023-12-27 13:57:59,967][20381] Updated weights for policy 1, policy_version 1199430 (0.0007) [2023-12-27 13:57:59,970][20347] Updated weights for policy 0, policy_version 1200422 (0.0008) [2023-12-27 13:58:00,019][20381] Updated weights for policy 1, policy_version 1199440 (0.0006) [2023-12-27 13:58:00,022][20347] Updated weights for policy 0, policy_version 1200432 (0.0008) [2023-12-27 13:58:00,614][20381] Updated weights for policy 1, policy_version 1199450 (0.0005) [2023-12-27 13:58:00,666][20381] Updated weights for policy 1, policy_version 1199460 (0.0008) [2023-12-27 13:58:00,679][20347] Updated weights for policy 0, policy_version 1200442 (0.0010) [2023-12-27 13:58:00,722][20381] Updated weights for policy 1, policy_version 1199470 (0.0007) [2023-12-27 13:58:00,727][20347] Updated weights for policy 0, policy_version 1200452 (0.0010) [2023-12-27 13:58:00,774][20381] Updated weights for policy 1, policy_version 1199480 (0.0008) [2023-12-27 13:58:00,782][20347] Updated weights for policy 0, policy_version 1200462 (0.0010) [2023-12-27 13:58:01,449][20347] Updated weights for policy 0, policy_version 1200472 (0.0006) [2023-12-27 13:58:01,478][20381] Updated weights for policy 1, policy_version 1199490 (0.0011) [2023-12-27 13:58:01,511][20347] Updated weights for policy 0, policy_version 1200482 (0.0005) [2023-12-27 13:58:01,534][20381] Updated weights for policy 1, policy_version 1199500 (0.0011) [2023-12-27 13:58:01,568][20347] Updated weights for policy 0, policy_version 1200492 (0.0006) [2023-12-27 13:58:01,593][20381] Updated weights for policy 1, policy_version 1199510 (0.0011) [2023-12-27 13:58:02,206][20347] Updated weights for policy 0, policy_version 1200502 (0.0008) [2023-12-27 13:58:02,257][20347] Updated weights for policy 0, policy_version 1200512 (0.0010) [2023-12-27 13:58:02,283][20381] Updated weights for policy 1, policy_version 1199520 (0.0009) [2023-12-27 13:58:02,317][20347] Updated weights for policy 0, policy_version 1200522 (0.0009) [2023-12-27 13:58:02,338][20381] Updated weights for policy 1, policy_version 1199530 (0.0006) [2023-12-27 13:58:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 614490112. Throughput: 0: 10315.1, 1: 10390.6. Samples: 614500088. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:02,363][19170] Avg episode reward: [(0, '872.640'), (1, '215.390')] [2023-12-27 13:58:02,397][20381] Updated weights for policy 1, policy_version 1199540 (0.0007) [2023-12-27 13:58:03,014][20347] Updated weights for policy 0, policy_version 1200532 (0.0009) [2023-12-27 13:58:03,030][20381] Updated weights for policy 1, policy_version 1199550 (0.0009) [2023-12-27 13:58:03,065][20347] Updated weights for policy 0, policy_version 1200542 (0.0008) [2023-12-27 13:58:03,089][20381] Updated weights for policy 1, policy_version 1199560 (0.0007) [2023-12-27 13:58:03,120][20347] Updated weights for policy 0, policy_version 1200552 (0.0008) [2023-12-27 13:58:03,145][20381] Updated weights for policy 1, policy_version 1199570 (0.0005) [2023-12-27 13:58:03,779][20381] Updated weights for policy 1, policy_version 1199580 (0.0006) [2023-12-27 13:58:03,781][20347] Updated weights for policy 0, policy_version 1200562 (0.0006) [2023-12-27 13:58:03,831][20347] Updated weights for policy 0, policy_version 1200572 (0.0008) [2023-12-27 13:58:03,833][20381] Updated weights for policy 1, policy_version 1199590 (0.0009) [2023-12-27 13:58:03,891][20381] Updated weights for policy 1, policy_version 1199600 (0.0008) [2023-12-27 13:58:03,892][20347] Updated weights for policy 0, policy_version 1200582 (0.0007) [2023-12-27 13:58:03,949][20347] Updated weights for policy 0, policy_version 1200592 (0.0006) [2023-12-27 13:58:04,661][20381] Updated weights for policy 1, policy_version 1199610 (0.0008) [2023-12-27 13:58:04,675][20347] Updated weights for policy 0, policy_version 1200602 (0.0010) [2023-12-27 13:58:04,725][20381] Updated weights for policy 1, policy_version 1199620 (0.0009) [2023-12-27 13:58:04,743][20347] Updated weights for policy 0, policy_version 1200612 (0.0009) [2023-12-27 13:58:04,783][20381] Updated weights for policy 1, policy_version 1199630 (0.0007) [2023-12-27 13:58:04,809][20347] Updated weights for policy 0, policy_version 1200622 (0.0009) [2023-12-27 13:58:04,840][20381] Updated weights for policy 1, policy_version 1199640 (0.0007) [2023-12-27 13:58:05,436][20347] Updated weights for policy 0, policy_version 1200632 (0.0007) [2023-12-27 13:58:05,482][20347] Updated weights for policy 0, policy_version 1200642 (0.0007) [2023-12-27 13:58:05,487][20381] Updated weights for policy 1, policy_version 1199650 (0.0010) [2023-12-27 13:58:05,525][20347] Updated weights for policy 0, policy_version 1200652 (0.0007) [2023-12-27 13:58:05,553][20381] Updated weights for policy 1, policy_version 1199660 (0.0010) [2023-12-27 13:58:05,608][20381] Updated weights for policy 1, policy_version 1199670 (0.0008) [2023-12-27 13:58:06,138][20347] Updated weights for policy 0, policy_version 1200662 (0.0009) [2023-12-27 13:58:06,190][20347] Updated weights for policy 0, policy_version 1200672 (0.0010) [2023-12-27 13:58:06,249][20347] Updated weights for policy 0, policy_version 1200682 (0.0006) [2023-12-27 13:58:06,353][20381] Updated weights for policy 1, policy_version 1199680 (0.0008) [2023-12-27 13:58:06,407][20381] Updated weights for policy 1, policy_version 1199690 (0.0010) [2023-12-27 13:58:06,462][20381] Updated weights for policy 1, policy_version 1199700 (0.0008) [2023-12-27 13:58:06,892][20347] Updated weights for policy 0, policy_version 1200692 (0.0007) [2023-12-27 13:58:06,954][20347] Updated weights for policy 0, policy_version 1200702 (0.0011) [2023-12-27 13:58:07,023][20347] Updated weights for policy 0, policy_version 1200712 (0.0010) [2023-12-27 13:58:07,130][20381] Updated weights for policy 1, policy_version 1199710 (0.0007) [2023-12-27 13:58:07,175][20381] Updated weights for policy 1, policy_version 1199720 (0.0007) [2023-12-27 13:58:07,225][20381] Updated weights for policy 1, policy_version 1199730 (0.0008) [2023-12-27 13:58:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 614596608. Throughput: 0: 10276.3, 1: 10415.8. Samples: 614560752. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:07,363][19170] Avg episode reward: [(0, '921.790'), (1, '376.120')] [2023-12-27 13:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001199736_307159040.pth... [2023-12-27 13:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001200720_307437568.pth... [2023-12-27 13:58:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001198520_306847744.pth [2023-12-27 13:58:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001199536_307134464.pth [2023-12-27 13:58:07,685][20347] Updated weights for policy 0, policy_version 1200722 (0.0010) [2023-12-27 13:58:07,741][20347] Updated weights for policy 0, policy_version 1200732 (0.0010) [2023-12-27 13:58:07,759][20291] KL-divergence is very high: 127.6020 [2023-12-27 13:58:07,782][20291] KL-divergence is very high: 196.3540 [2023-12-27 13:58:07,801][20347] Updated weights for policy 0, policy_version 1200742 (0.0010) [2023-12-27 13:58:07,806][20291] KL-divergence is very high: 160.7504 [2023-12-27 13:58:07,849][20347] Updated weights for policy 0, policy_version 1200752 (0.0010) [2023-12-27 13:58:07,899][20381] Updated weights for policy 1, policy_version 1199740 (0.0007) [2023-12-27 13:58:07,961][20381] Updated weights for policy 1, policy_version 1199750 (0.0008) [2023-12-27 13:58:08,041][20381] Updated weights for policy 1, policy_version 1199760 (0.0008) [2023-12-27 13:58:08,498][20291] KL-divergence is very high: 132.0672 [2023-12-27 13:58:08,536][20347] Updated weights for policy 0, policy_version 1200762 (0.0008) [2023-12-27 13:58:08,542][20291] KL-divergence is very high: 103.3949 [2023-12-27 13:58:08,589][20347] Updated weights for policy 0, policy_version 1200772 (0.0007) [2023-12-27 13:58:08,640][20347] Updated weights for policy 0, policy_version 1200782 (0.0007) [2023-12-27 13:58:08,751][20381] Updated weights for policy 1, policy_version 1199770 (0.0008) [2023-12-27 13:58:08,808][20381] Updated weights for policy 1, policy_version 1199780 (0.0008) [2023-12-27 13:58:08,862][20381] Updated weights for policy 1, policy_version 1199790 (0.0008) [2023-12-27 13:58:08,918][20381] Updated weights for policy 1, policy_version 1199800 (0.0009) [2023-12-27 13:58:09,339][20347] Updated weights for policy 0, policy_version 1200792 (0.0007) [2023-12-27 13:58:09,409][20347] Updated weights for policy 0, policy_version 1200802 (0.0010) [2023-12-27 13:58:09,478][20347] Updated weights for policy 0, policy_version 1200812 (0.0010) [2023-12-27 13:58:09,647][20381] Updated weights for policy 1, policy_version 1199810 (0.0007) [2023-12-27 13:58:09,716][20381] Updated weights for policy 1, policy_version 1199820 (0.0007) [2023-12-27 13:58:09,777][20381] Updated weights for policy 1, policy_version 1199830 (0.0008) [2023-12-27 13:58:10,176][20347] Updated weights for policy 0, policy_version 1200822 (0.0011) [2023-12-27 13:58:10,237][20347] Updated weights for policy 0, policy_version 1200832 (0.0011) [2023-12-27 13:58:10,255][20291] KL-divergence is very high: 206.3024 [2023-12-27 13:58:10,279][20291] KL-divergence is very high: 110.9089 [2023-12-27 13:58:10,296][20347] Updated weights for policy 0, policy_version 1200842 (0.0011) [2023-12-27 13:58:10,494][20381] Updated weights for policy 1, policy_version 1199840 (0.0007) [2023-12-27 13:58:10,557][20381] Updated weights for policy 1, policy_version 1199850 (0.0007) [2023-12-27 13:58:10,607][20381] Updated weights for policy 1, policy_version 1199860 (0.0008) [2023-12-27 13:58:10,932][20347] Updated weights for policy 0, policy_version 1200852 (0.0008) [2023-12-27 13:58:10,983][20347] Updated weights for policy 0, policy_version 1200862 (0.0005) [2023-12-27 13:58:11,067][20347] Updated weights for policy 0, policy_version 1200872 (0.0008) [2023-12-27 13:58:11,338][20381] Updated weights for policy 1, policy_version 1199870 (0.0008) [2023-12-27 13:58:11,407][20381] Updated weights for policy 1, policy_version 1199880 (0.0008) [2023-12-27 13:58:11,452][20381] Updated weights for policy 1, policy_version 1199890 (0.0008) [2023-12-27 13:58:11,791][20347] Updated weights for policy 0, policy_version 1200882 (0.0007) [2023-12-27 13:58:11,852][20347] Updated weights for policy 0, policy_version 1200892 (0.0007) [2023-12-27 13:58:11,910][20347] Updated weights for policy 0, policy_version 1200902 (0.0008) [2023-12-27 13:58:11,973][20347] Updated weights for policy 0, policy_version 1200912 (0.0007) [2023-12-27 13:58:12,222][20381] Updated weights for policy 1, policy_version 1199900 (0.0009) [2023-12-27 13:58:12,272][20381] Updated weights for policy 1, policy_version 1199910 (0.0011) [2023-12-27 13:58:12,325][20381] Updated weights for policy 1, policy_version 1199920 (0.0010) [2023-12-27 13:58:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 614686720. Throughput: 0: 10326.9, 1: 10323.0. Samples: 614683156. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:12,363][19170] Avg episode reward: [(0, '921.590'), (1, '479.930')] [2023-12-27 13:58:12,718][20347] Updated weights for policy 0, policy_version 1200922 (0.0007) [2023-12-27 13:58:12,787][20347] Updated weights for policy 0, policy_version 1200932 (0.0008) [2023-12-27 13:58:12,842][20347] Updated weights for policy 0, policy_version 1200942 (0.0007) [2023-12-27 13:58:13,047][20381] Updated weights for policy 1, policy_version 1199930 (0.0008) [2023-12-27 13:58:13,104][20381] Updated weights for policy 1, policy_version 1199940 (0.0006) [2023-12-27 13:58:13,158][20381] Updated weights for policy 1, policy_version 1199950 (0.0010) [2023-12-27 13:58:13,213][20381] Updated weights for policy 1, policy_version 1199960 (0.0010) [2023-12-27 13:58:13,519][20347] Updated weights for policy 0, policy_version 1200952 (0.0007) [2023-12-27 13:58:13,577][20347] Updated weights for policy 0, policy_version 1200962 (0.0007) [2023-12-27 13:58:13,635][20347] Updated weights for policy 0, policy_version 1200972 (0.0008) [2023-12-27 13:58:13,888][20381] Updated weights for policy 1, policy_version 1199970 (0.0007) [2023-12-27 13:58:13,938][20381] Updated weights for policy 1, policy_version 1199980 (0.0007) [2023-12-27 13:58:13,989][20381] Updated weights for policy 1, policy_version 1199990 (0.0009) [2023-12-27 13:58:14,258][20347] Updated weights for policy 0, policy_version 1200982 (0.0007) [2023-12-27 13:58:14,313][20347] Updated weights for policy 0, policy_version 1200992 (0.0009) [2023-12-27 13:58:14,361][20347] Updated weights for policy 0, policy_version 1201002 (0.0008) [2023-12-27 13:58:14,630][20381] Updated weights for policy 1, policy_version 1200000 (0.0008) [2023-12-27 13:58:14,681][20381] Updated weights for policy 1, policy_version 1200010 (0.0007) [2023-12-27 13:58:14,736][20381] Updated weights for policy 1, policy_version 1200020 (0.0007) [2023-12-27 13:58:15,103][20347] Updated weights for policy 0, policy_version 1201012 (0.0009) [2023-12-27 13:58:15,161][20347] Updated weights for policy 0, policy_version 1201022 (0.0010) [2023-12-27 13:58:15,228][20347] Updated weights for policy 0, policy_version 1201032 (0.0007) [2023-12-27 13:58:15,456][20381] Updated weights for policy 1, policy_version 1200030 (0.0006) [2023-12-27 13:58:15,508][20381] Updated weights for policy 1, policy_version 1200040 (0.0005) [2023-12-27 13:58:15,560][20381] Updated weights for policy 1, policy_version 1200050 (0.0006) [2023-12-27 13:58:15,818][20347] Updated weights for policy 0, policy_version 1201042 (0.0006) [2023-12-27 13:58:15,876][20347] Updated weights for policy 0, policy_version 1201052 (0.0005) [2023-12-27 13:58:15,934][20347] Updated weights for policy 0, policy_version 1201062 (0.0005) [2023-12-27 13:58:15,988][20347] Updated weights for policy 0, policy_version 1201072 (0.0009) [2023-12-27 13:58:16,240][20381] Updated weights for policy 1, policy_version 1200060 (0.0007) [2023-12-27 13:58:16,284][20381] Updated weights for policy 1, policy_version 1200070 (0.0009) [2023-12-27 13:58:16,330][20381] Updated weights for policy 1, policy_version 1200080 (0.0008) [2023-12-27 13:58:16,567][20347] Updated weights for policy 0, policy_version 1201082 (0.0005) [2023-12-27 13:58:16,613][20347] Updated weights for policy 0, policy_version 1201092 (0.0005) [2023-12-27 13:58:16,659][20347] Updated weights for policy 0, policy_version 1201102 (0.0005) [2023-12-27 13:58:17,013][20381] Updated weights for policy 1, policy_version 1200090 (0.0008) [2023-12-27 13:58:17,068][20381] Updated weights for policy 1, policy_version 1200100 (0.0005) [2023-12-27 13:58:17,124][20381] Updated weights for policy 1, policy_version 1200110 (0.0006) [2023-12-27 13:58:17,181][20381] Updated weights for policy 1, policy_version 1200120 (0.0007) [2023-12-27 13:58:17,332][20347] Updated weights for policy 0, policy_version 1201112 (0.0007) [2023-12-27 13:58:17,338][20291] KL-divergence is very high: 230.3701 [2023-12-27 13:58:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 614793216. Throughput: 0: 10298.3, 1: 10265.5. Samples: 614805172. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:17,363][19170] Avg episode reward: [(0, '931.280'), (1, '300.950')] [2023-12-27 13:58:17,376][20291] KL-divergence is very high: 226.5482 [2023-12-27 13:58:17,381][20347] Updated weights for policy 0, policy_version 1201122 (0.0007) [2023-12-27 13:58:17,422][20291] KL-divergence is very high: 125.2811 [2023-12-27 13:58:17,443][20347] Updated weights for policy 0, policy_version 1201132 (0.0008) [2023-12-27 13:58:17,755][20381] Updated weights for policy 1, policy_version 1200130 (0.0006) [2023-12-27 13:58:17,805][20381] Updated weights for policy 1, policy_version 1200140 (0.0005) [2023-12-27 13:58:17,859][20381] Updated weights for policy 1, policy_version 1200150 (0.0005) [2023-12-27 13:58:18,124][20347] Updated weights for policy 0, policy_version 1201142 (0.0008) [2023-12-27 13:58:18,186][20347] Updated weights for policy 0, policy_version 1201152 (0.0008) [2023-12-27 13:58:18,241][20347] Updated weights for policy 0, policy_version 1201162 (0.0008) [2023-12-27 13:58:18,451][20381] Updated weights for policy 1, policy_version 1200160 (0.0007) [2023-12-27 13:58:18,507][20381] Updated weights for policy 1, policy_version 1200170 (0.0008) [2023-12-27 13:58:18,563][20381] Updated weights for policy 1, policy_version 1200180 (0.0007) [2023-12-27 13:58:18,948][20347] Updated weights for policy 0, policy_version 1201172 (0.0009) [2023-12-27 13:58:18,993][20347] Updated weights for policy 0, policy_version 1201182 (0.0007) [2023-12-27 13:58:19,045][20347] Updated weights for policy 0, policy_version 1201192 (0.0007) [2023-12-27 13:58:19,272][20381] Updated weights for policy 1, policy_version 1200190 (0.0009) [2023-12-27 13:58:19,330][20381] Updated weights for policy 1, policy_version 1200200 (0.0010) [2023-12-27 13:58:19,398][20381] Updated weights for policy 1, policy_version 1200210 (0.0011) [2023-12-27 13:58:19,757][20347] Updated weights for policy 0, policy_version 1201202 (0.0007) [2023-12-27 13:58:19,820][20347] Updated weights for policy 0, policy_version 1201212 (0.0008) [2023-12-27 13:58:19,889][20347] Updated weights for policy 0, policy_version 1201222 (0.0007) [2023-12-27 13:58:19,957][20347] Updated weights for policy 0, policy_version 1201232 (0.0008) [2023-12-27 13:58:20,073][20381] Updated weights for policy 1, policy_version 1200220 (0.0010) [2023-12-27 13:58:20,123][20381] Updated weights for policy 1, policy_version 1200230 (0.0011) [2023-12-27 13:58:20,175][20381] Updated weights for policy 1, policy_version 1200240 (0.0007) [2023-12-27 13:58:20,702][20347] Updated weights for policy 0, policy_version 1201242 (0.0009) [2023-12-27 13:58:20,763][20347] Updated weights for policy 0, policy_version 1201252 (0.0009) [2023-12-27 13:58:20,821][20347] Updated weights for policy 0, policy_version 1201262 (0.0007) [2023-12-27 13:58:20,822][20381] Updated weights for policy 1, policy_version 1200250 (0.0006) [2023-12-27 13:58:20,888][20381] Updated weights for policy 1, policy_version 1200260 (0.0007) [2023-12-27 13:58:20,951][20381] Updated weights for policy 1, policy_version 1200270 (0.0007) [2023-12-27 13:58:21,010][20381] Updated weights for policy 1, policy_version 1200280 (0.0006) [2023-12-27 13:58:21,573][20347] Updated weights for policy 0, policy_version 1201272 (0.0008) [2023-12-27 13:58:21,629][20347] Updated weights for policy 0, policy_version 1201282 (0.0007) [2023-12-27 13:58:21,667][20381] Updated weights for policy 1, policy_version 1200290 (0.0008) [2023-12-27 13:58:21,690][20347] Updated weights for policy 0, policy_version 1201292 (0.0008) [2023-12-27 13:58:21,725][20381] Updated weights for policy 1, policy_version 1200300 (0.0006) [2023-12-27 13:58:21,793][20381] Updated weights for policy 1, policy_version 1200310 (0.0008) [2023-12-27 13:58:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 614891520. Throughput: 0: 10252.9, 1: 10266.5. Samples: 614866672. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:22,363][19170] Avg episode reward: [(0, '921.610'), (1, '406.390')] [2023-12-27 13:58:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001200312_307306496.pth... [2023-12-27 13:58:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001199128_307003392.pth [2023-12-27 13:58:22,414][20347] Updated weights for policy 0, policy_version 1201302 (0.0008) [2023-12-27 13:58:22,468][20347] Updated weights for policy 0, policy_version 1201312 (0.0008) [2023-12-27 13:58:22,523][20347] Updated weights for policy 0, policy_version 1201322 (0.0007) [2023-12-27 13:58:22,546][20381] Updated weights for policy 1, policy_version 1200320 (0.0007) [2023-12-27 13:58:22,555][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001201328_307593216.pth... [2023-12-27 13:58:22,559][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001200112_307281920.pth [2023-12-27 13:58:22,604][20381] Updated weights for policy 1, policy_version 1200330 (0.0007) [2023-12-27 13:58:22,667][20381] Updated weights for policy 1, policy_version 1200340 (0.0008) [2023-12-27 13:58:23,228][20347] Updated weights for policy 0, policy_version 1201332 (0.0007) [2023-12-27 13:58:23,272][20347] Updated weights for policy 0, policy_version 1201342 (0.0008) [2023-12-27 13:58:23,326][20347] Updated weights for policy 0, policy_version 1201352 (0.0008) [2023-12-27 13:58:23,331][20381] Updated weights for policy 1, policy_version 1200350 (0.0008) [2023-12-27 13:58:23,384][20381] Updated weights for policy 1, policy_version 1200360 (0.0005) [2023-12-27 13:58:23,440][20381] Updated weights for policy 1, policy_version 1200370 (0.0007) [2023-12-27 13:58:23,963][20381] Updated weights for policy 1, policy_version 1200380 (0.0006) [2023-12-27 13:58:24,021][20381] Updated weights for policy 1, policy_version 1200390 (0.0008) [2023-12-27 13:58:24,079][20381] Updated weights for policy 1, policy_version 1200400 (0.0008) [2023-12-27 13:58:24,100][20347] Updated weights for policy 0, policy_version 1201362 (0.0007) [2023-12-27 13:58:24,155][20347] Updated weights for policy 0, policy_version 1201372 (0.0007) [2023-12-27 13:58:24,205][20347] Updated weights for policy 0, policy_version 1201382 (0.0007) [2023-12-27 13:58:24,263][20347] Updated weights for policy 0, policy_version 1201392 (0.0007) [2023-12-27 13:58:24,787][20381] Updated weights for policy 1, policy_version 1200410 (0.0007) [2023-12-27 13:58:24,832][20381] Updated weights for policy 1, policy_version 1200420 (0.0008) [2023-12-27 13:58:24,881][20381] Updated weights for policy 1, policy_version 1200430 (0.0007) [2023-12-27 13:58:24,934][20381] Updated weights for policy 1, policy_version 1200440 (0.0007) [2023-12-27 13:58:24,951][20347] Updated weights for policy 0, policy_version 1201402 (0.0007) [2023-12-27 13:58:25,003][20347] Updated weights for policy 0, policy_version 1201412 (0.0009) [2023-12-27 13:58:25,057][20347] Updated weights for policy 0, policy_version 1201422 (0.0008) [2023-12-27 13:58:25,615][20381] Updated weights for policy 1, policy_version 1200450 (0.0006) [2023-12-27 13:58:25,669][20381] Updated weights for policy 1, policy_version 1200460 (0.0007) [2023-12-27 13:58:25,713][20381] Updated weights for policy 1, policy_version 1200470 (0.0006) [2023-12-27 13:58:25,725][20347] Updated weights for policy 0, policy_version 1201432 (0.0007) [2023-12-27 13:58:25,771][20347] Updated weights for policy 0, policy_version 1201442 (0.0008) [2023-12-27 13:58:25,823][20347] Updated weights for policy 0, policy_version 1201452 (0.0008) [2023-12-27 13:58:26,373][20381] Updated weights for policy 1, policy_version 1200480 (0.0007) [2023-12-27 13:58:26,417][20381] Updated weights for policy 1, policy_version 1200490 (0.0007) [2023-12-27 13:58:26,463][20381] Updated weights for policy 1, policy_version 1200500 (0.0006) [2023-12-27 13:58:26,511][20347] Updated weights for policy 0, policy_version 1201462 (0.0007) [2023-12-27 13:58:26,562][20347] Updated weights for policy 0, policy_version 1201472 (0.0007) [2023-12-27 13:58:26,610][20347] Updated weights for policy 0, policy_version 1201482 (0.0007) [2023-12-27 13:58:27,183][20381] Updated weights for policy 1, policy_version 1200510 (0.0006) [2023-12-27 13:58:27,232][20381] Updated weights for policy 1, policy_version 1200520 (0.0005) [2023-12-27 13:58:27,291][20381] Updated weights for policy 1, policy_version 1200530 (0.0005) [2023-12-27 13:58:27,300][20347] Updated weights for policy 0, policy_version 1201492 (0.0007) [2023-12-27 13:58:27,356][20347] Updated weights for policy 0, policy_version 1201502 (0.0006) [2023-12-27 13:58:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 614998016. Throughput: 0: 10151.8, 1: 10285.8. Samples: 614987200. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:27,363][19170] Avg episode reward: [(0, '931.530'), (1, '574.320')] [2023-12-27 13:58:27,411][20347] Updated weights for policy 0, policy_version 1201512 (0.0007) [2023-12-27 13:58:27,918][20381] Updated weights for policy 1, policy_version 1200540 (0.0006) [2023-12-27 13:58:27,974][20381] Updated weights for policy 1, policy_version 1200550 (0.0007) [2023-12-27 13:58:28,025][20381] Updated weights for policy 1, policy_version 1200560 (0.0007) [2023-12-27 13:58:28,104][20347] Updated weights for policy 0, policy_version 1201522 (0.0007) [2023-12-27 13:58:28,152][20347] Updated weights for policy 0, policy_version 1201532 (0.0009) [2023-12-27 13:58:28,198][20347] Updated weights for policy 0, policy_version 1201542 (0.0010) [2023-12-27 13:58:28,251][20347] Updated weights for policy 0, policy_version 1201552 (0.0008) [2023-12-27 13:58:28,674][20381] Updated weights for policy 1, policy_version 1200570 (0.0005) [2023-12-27 13:58:28,726][20381] Updated weights for policy 1, policy_version 1200580 (0.0009) [2023-12-27 13:58:28,784][20381] Updated weights for policy 1, policy_version 1200590 (0.0008) [2023-12-27 13:58:28,838][20381] Updated weights for policy 1, policy_version 1200600 (0.0007) [2023-12-27 13:58:28,956][20347] Updated weights for policy 0, policy_version 1201562 (0.0008) [2023-12-27 13:58:29,005][20347] Updated weights for policy 0, policy_version 1201572 (0.0010) [2023-12-27 13:58:29,057][20347] Updated weights for policy 0, policy_version 1201582 (0.0009) [2023-12-27 13:58:29,547][20381] Updated weights for policy 1, policy_version 1200610 (0.0010) [2023-12-27 13:58:29,607][20381] Updated weights for policy 1, policy_version 1200620 (0.0008) [2023-12-27 13:58:29,669][20381] Updated weights for policy 1, policy_version 1200630 (0.0005) [2023-12-27 13:58:29,695][20347] Updated weights for policy 0, policy_version 1201592 (0.0005) [2023-12-27 13:58:29,755][20347] Updated weights for policy 0, policy_version 1201602 (0.0005) [2023-12-27 13:58:29,816][20347] Updated weights for policy 0, policy_version 1201612 (0.0006) [2023-12-27 13:58:30,335][20381] Updated weights for policy 1, policy_version 1200640 (0.0006) [2023-12-27 13:58:30,387][20381] Updated weights for policy 1, policy_version 1200650 (0.0007) [2023-12-27 13:58:30,444][20381] Updated weights for policy 1, policy_version 1200660 (0.0007) [2023-12-27 13:58:30,459][20347] Updated weights for policy 0, policy_version 1201622 (0.0009) [2023-12-27 13:58:30,513][20347] Updated weights for policy 0, policy_version 1201632 (0.0010) [2023-12-27 13:58:30,563][20347] Updated weights for policy 0, policy_version 1201642 (0.0010) [2023-12-27 13:58:31,140][20381] Updated weights for policy 1, policy_version 1200670 (0.0010) [2023-12-27 13:58:31,202][20381] Updated weights for policy 1, policy_version 1200680 (0.0010) [2023-12-27 13:58:31,258][20347] Updated weights for policy 0, policy_version 1201652 (0.0010) [2023-12-27 13:58:31,269][20381] Updated weights for policy 1, policy_version 1200690 (0.0008) [2023-12-27 13:58:31,315][20347] Updated weights for policy 0, policy_version 1201662 (0.0010) [2023-12-27 13:58:31,382][20347] Updated weights for policy 0, policy_version 1201672 (0.0009) [2023-12-27 13:58:31,975][20381] Updated weights for policy 1, policy_version 1200700 (0.0008) [2023-12-27 13:58:32,012][20347] Updated weights for policy 0, policy_version 1201682 (0.0011) [2023-12-27 13:58:32,023][20381] Updated weights for policy 1, policy_version 1200710 (0.0007) [2023-12-27 13:58:32,063][20347] Updated weights for policy 0, policy_version 1201692 (0.0005) [2023-12-27 13:58:32,078][20381] Updated weights for policy 1, policy_version 1200720 (0.0007) [2023-12-27 13:58:32,117][20347] Updated weights for policy 0, policy_version 1201702 (0.0009) [2023-12-27 13:58:32,162][20347] Updated weights for policy 0, policy_version 1201712 (0.0008) [2023-12-27 13:58:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 615104512. Throughput: 0: 10131.2, 1: 10249.4. Samples: 615110868. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:32,363][19170] Avg episode reward: [(0, '941.330'), (1, '622.840')] [2023-12-27 13:58:32,785][20381] Updated weights for policy 1, policy_version 1200730 (0.0009) [2023-12-27 13:58:32,840][20381] Updated weights for policy 1, policy_version 1200740 (0.0006) [2023-12-27 13:58:32,853][20347] Updated weights for policy 0, policy_version 1201722 (0.0010) [2023-12-27 13:58:32,888][20381] Updated weights for policy 1, policy_version 1200750 (0.0005) [2023-12-27 13:58:32,905][20347] Updated weights for policy 0, policy_version 1201732 (0.0010) [2023-12-27 13:58:32,946][20381] Updated weights for policy 1, policy_version 1200760 (0.0005) [2023-12-27 13:58:32,962][20347] Updated weights for policy 0, policy_version 1201742 (0.0010) [2023-12-27 13:58:33,640][20347] Updated weights for policy 0, policy_version 1201752 (0.0009) [2023-12-27 13:58:33,653][20381] Updated weights for policy 1, policy_version 1200770 (0.0008) [2023-12-27 13:58:33,693][20347] Updated weights for policy 0, policy_version 1201762 (0.0005) [2023-12-27 13:58:33,711][20381] Updated weights for policy 1, policy_version 1200780 (0.0008) [2023-12-27 13:58:33,752][20347] Updated weights for policy 0, policy_version 1201772 (0.0005) [2023-12-27 13:58:33,763][20381] Updated weights for policy 1, policy_version 1200790 (0.0008) [2023-12-27 13:58:34,332][20381] Updated weights for policy 1, policy_version 1200800 (0.0006) [2023-12-27 13:58:34,393][20381] Updated weights for policy 1, policy_version 1200810 (0.0007) [2023-12-27 13:58:34,418][20347] Updated weights for policy 0, policy_version 1201782 (0.0007) [2023-12-27 13:58:34,450][20381] Updated weights for policy 1, policy_version 1200820 (0.0005) [2023-12-27 13:58:34,478][20347] Updated weights for policy 0, policy_version 1201792 (0.0006) [2023-12-27 13:58:34,542][20347] Updated weights for policy 0, policy_version 1201802 (0.0008) [2023-12-27 13:58:35,101][20381] Updated weights for policy 1, policy_version 1200830 (0.0006) [2023-12-27 13:58:35,170][20381] Updated weights for policy 1, policy_version 1200840 (0.0007) [2023-12-27 13:58:35,243][20381] Updated weights for policy 1, policy_version 1200850 (0.0007) [2023-12-27 13:58:35,285][20347] Updated weights for policy 0, policy_version 1201812 (0.0008) [2023-12-27 13:58:35,345][20347] Updated weights for policy 0, policy_version 1201822 (0.0007) [2023-12-27 13:58:35,409][20347] Updated weights for policy 0, policy_version 1201832 (0.0005) [2023-12-27 13:58:35,916][20381] Updated weights for policy 1, policy_version 1200860 (0.0006) [2023-12-27 13:58:35,971][20381] Updated weights for policy 1, policy_version 1200870 (0.0009) [2023-12-27 13:58:35,992][20347] Updated weights for policy 0, policy_version 1201842 (0.0006) [2023-12-27 13:58:36,027][20381] Updated weights for policy 1, policy_version 1200880 (0.0008) [2023-12-27 13:58:36,041][20347] Updated weights for policy 0, policy_version 1201852 (0.0007) [2023-12-27 13:58:36,098][20347] Updated weights for policy 0, policy_version 1201862 (0.0007) [2023-12-27 13:58:36,165][20347] Updated weights for policy 0, policy_version 1201872 (0.0008) [2023-12-27 13:58:36,740][20381] Updated weights for policy 1, policy_version 1200890 (0.0006) [2023-12-27 13:58:36,804][20381] Updated weights for policy 1, policy_version 1200900 (0.0008) [2023-12-27 13:58:36,856][20381] Updated weights for policy 1, policy_version 1200910 (0.0008) [2023-12-27 13:58:36,873][20347] Updated weights for policy 0, policy_version 1201882 (0.0006) [2023-12-27 13:58:36,902][20381] Updated weights for policy 1, policy_version 1200920 (0.0006) [2023-12-27 13:58:36,933][20347] Updated weights for policy 0, policy_version 1201892 (0.0007) [2023-12-27 13:58:36,986][20347] Updated weights for policy 0, policy_version 1201902 (0.0008) [2023-12-27 13:58:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20550.2). Total num frames: 615202816. Throughput: 0: 10129.0, 1: 10273.4. Samples: 615172860. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:37,363][19170] Avg episode reward: [(0, '960.910'), (1, '548.850')] [2023-12-27 13:58:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001200920_307462144.pth... [2023-12-27 13:58:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001201904_307740672.pth... [2023-12-27 13:58:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001199736_307159040.pth [2023-12-27 13:58:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001200720_307437568.pth [2023-12-27 13:58:37,613][20381] Updated weights for policy 1, policy_version 1200930 (0.0007) [2023-12-27 13:58:37,614][20347] Updated weights for policy 0, policy_version 1201912 (0.0008) [2023-12-27 13:58:37,668][20381] Updated weights for policy 1, policy_version 1200940 (0.0006) [2023-12-27 13:58:37,671][20347] Updated weights for policy 0, policy_version 1201922 (0.0006) [2023-12-27 13:58:37,728][20381] Updated weights for policy 1, policy_version 1200950 (0.0006) [2023-12-27 13:58:37,730][20347] Updated weights for policy 0, policy_version 1201932 (0.0006) [2023-12-27 13:58:38,314][20347] Updated weights for policy 0, policy_version 1201942 (0.0005) [2023-12-27 13:58:38,373][20347] Updated weights for policy 0, policy_version 1201952 (0.0007) [2023-12-27 13:58:38,411][20381] Updated weights for policy 1, policy_version 1200960 (0.0005) [2023-12-27 13:58:38,434][20347] Updated weights for policy 0, policy_version 1201962 (0.0007) [2023-12-27 13:58:38,466][20381] Updated weights for policy 1, policy_version 1200970 (0.0006) [2023-12-27 13:58:38,523][20381] Updated weights for policy 1, policy_version 1200980 (0.0005) [2023-12-27 13:58:39,040][20347] Updated weights for policy 0, policy_version 1201972 (0.0006) [2023-12-27 13:58:39,090][20347] Updated weights for policy 0, policy_version 1201982 (0.0009) [2023-12-27 13:58:39,140][20347] Updated weights for policy 0, policy_version 1201992 (0.0007) [2023-12-27 13:58:39,158][20381] Updated weights for policy 1, policy_version 1200990 (0.0006) [2023-12-27 13:58:39,209][20381] Updated weights for policy 1, policy_version 1201000 (0.0005) [2023-12-27 13:58:39,271][20381] Updated weights for policy 1, policy_version 1201010 (0.0008) [2023-12-27 13:58:39,987][20347] Updated weights for policy 0, policy_version 1202002 (0.0008) [2023-12-27 13:58:40,028][20381] Updated weights for policy 1, policy_version 1201020 (0.0008) [2023-12-27 13:58:40,053][20347] Updated weights for policy 0, policy_version 1202012 (0.0006) [2023-12-27 13:58:40,093][20381] Updated weights for policy 1, policy_version 1201030 (0.0007) [2023-12-27 13:58:40,118][20347] Updated weights for policy 0, policy_version 1202022 (0.0006) [2023-12-27 13:58:40,156][20381] Updated weights for policy 1, policy_version 1201040 (0.0007) [2023-12-27 13:58:40,182][20347] Updated weights for policy 0, policy_version 1202032 (0.0006) [2023-12-27 13:58:40,813][20347] Updated weights for policy 0, policy_version 1202042 (0.0006) [2023-12-27 13:58:40,830][20381] Updated weights for policy 1, policy_version 1201050 (0.0007) [2023-12-27 13:58:40,871][20347] Updated weights for policy 0, policy_version 1202052 (0.0007) [2023-12-27 13:58:40,887][20381] Updated weights for policy 1, policy_version 1201060 (0.0005) [2023-12-27 13:58:40,928][20347] Updated weights for policy 0, policy_version 1202062 (0.0007) [2023-12-27 13:58:40,944][20381] Updated weights for policy 1, policy_version 1201070 (0.0007) [2023-12-27 13:58:40,989][20381] Updated weights for policy 1, policy_version 1201080 (0.0008) [2023-12-27 13:58:41,653][20347] Updated weights for policy 0, policy_version 1202072 (0.0008) [2023-12-27 13:58:41,721][20347] Updated weights for policy 0, policy_version 1202082 (0.0008) [2023-12-27 13:58:41,758][20381] Updated weights for policy 1, policy_version 1201090 (0.0008) [2023-12-27 13:58:41,779][20347] Updated weights for policy 0, policy_version 1202092 (0.0007) [2023-12-27 13:58:41,819][20381] Updated weights for policy 1, policy_version 1201100 (0.0007) [2023-12-27 13:58:41,886][20381] Updated weights for policy 1, policy_version 1201110 (0.0008) [2023-12-27 13:58:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 615301120. Throughput: 0: 10197.6, 1: 10206.6. Samples: 615294676. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:42,363][19170] Avg episode reward: [(0, '951.030'), (1, '604.180')] [2023-12-27 13:58:42,516][20347] Updated weights for policy 0, policy_version 1202102 (0.0007) [2023-12-27 13:58:42,542][20291] KL-divergence is very high: 217.2428 [2023-12-27 13:58:42,548][20291] KL-divergence is very high: 205.9330 [2023-12-27 13:58:42,577][20347] Updated weights for policy 0, policy_version 1202112 (0.0006) [2023-12-27 13:58:42,587][20291] KL-divergence is very high: 171.1771 [2023-12-27 13:58:42,590][20381] Updated weights for policy 1, policy_version 1201120 (0.0007) [2023-12-27 13:58:42,593][20291] KL-divergence is very high: 128.5392 [2023-12-27 13:58:42,634][20291] KL-divergence is very high: 140.6007 [2023-12-27 13:58:42,635][20347] Updated weights for policy 0, policy_version 1202122 (0.0006) [2023-12-27 13:58:42,640][20291] KL-divergence is very high: 121.2740 [2023-12-27 13:58:42,644][20381] Updated weights for policy 1, policy_version 1201130 (0.0005) [2023-12-27 13:58:42,706][20381] Updated weights for policy 1, policy_version 1201140 (0.0005) [2023-12-27 13:58:43,327][20381] Updated weights for policy 1, policy_version 1201150 (0.0006) [2023-12-27 13:58:43,345][20347] Updated weights for policy 0, policy_version 1202132 (0.0007) [2023-12-27 13:58:43,386][20381] Updated weights for policy 1, policy_version 1201160 (0.0006) [2023-12-27 13:58:43,406][20347] Updated weights for policy 0, policy_version 1202142 (0.0005) [2023-12-27 13:58:43,448][20381] Updated weights for policy 1, policy_version 1201170 (0.0006) [2023-12-27 13:58:43,464][20347] Updated weights for policy 0, policy_version 1202152 (0.0006) [2023-12-27 13:58:44,124][20381] Updated weights for policy 1, policy_version 1201180 (0.0007) [2023-12-27 13:58:44,140][20347] Updated weights for policy 0, policy_version 1202162 (0.0007) [2023-12-27 13:58:44,184][20381] Updated weights for policy 1, policy_version 1201190 (0.0007) [2023-12-27 13:58:44,193][20347] Updated weights for policy 0, policy_version 1202172 (0.0008) [2023-12-27 13:58:44,247][20381] Updated weights for policy 1, policy_version 1201200 (0.0006) [2023-12-27 13:58:44,250][20347] Updated weights for policy 0, policy_version 1202182 (0.0006) [2023-12-27 13:58:44,307][20347] Updated weights for policy 0, policy_version 1202192 (0.0007) [2023-12-27 13:58:44,939][20381] Updated weights for policy 1, policy_version 1201210 (0.0005) [2023-12-27 13:58:45,001][20381] Updated weights for policy 1, policy_version 1201220 (0.0007) [2023-12-27 13:58:45,023][20347] Updated weights for policy 0, policy_version 1202202 (0.0006) [2023-12-27 13:58:45,055][20381] Updated weights for policy 1, policy_version 1201230 (0.0006) [2023-12-27 13:58:45,081][20347] Updated weights for policy 0, policy_version 1202212 (0.0007) [2023-12-27 13:58:45,108][20381] Updated weights for policy 1, policy_version 1201240 (0.0008) [2023-12-27 13:58:45,145][20347] Updated weights for policy 0, policy_version 1202222 (0.0007) [2023-12-27 13:58:45,809][20381] Updated weights for policy 1, policy_version 1201250 (0.0007) [2023-12-27 13:58:45,837][20347] Updated weights for policy 0, policy_version 1202232 (0.0006) [2023-12-27 13:58:45,865][20381] Updated weights for policy 1, policy_version 1201260 (0.0006) [2023-12-27 13:58:45,894][20347] Updated weights for policy 0, policy_version 1202242 (0.0006) [2023-12-27 13:58:45,925][20381] Updated weights for policy 1, policy_version 1201270 (0.0008) [2023-12-27 13:58:45,950][20347] Updated weights for policy 0, policy_version 1202252 (0.0007) [2023-12-27 13:58:46,592][20381] Updated weights for policy 1, policy_version 1201280 (0.0005) [2023-12-27 13:58:46,651][20381] Updated weights for policy 1, policy_version 1201290 (0.0007) [2023-12-27 13:58:46,659][20347] Updated weights for policy 0, policy_version 1202262 (0.0008) [2023-12-27 13:58:46,704][20381] Updated weights for policy 1, policy_version 1201300 (0.0007) [2023-12-27 13:58:46,709][20347] Updated weights for policy 0, policy_version 1202272 (0.0007) [2023-12-27 13:58:46,757][20347] Updated weights for policy 0, policy_version 1202282 (0.0007) [2023-12-27 13:58:47,330][20381] Updated weights for policy 1, policy_version 1201310 (0.0007) [2023-12-27 13:58:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 615399424. Throughput: 0: 10124.3, 1: 10192.4. Samples: 615414336. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:47,363][19170] Avg episode reward: [(0, '931.480'), (1, '574.090')] [2023-12-27 13:58:47,385][20381] Updated weights for policy 1, policy_version 1201320 (0.0007) [2023-12-27 13:58:47,437][20381] Updated weights for policy 1, policy_version 1201330 (0.0008) [2023-12-27 13:58:47,482][20347] Updated weights for policy 0, policy_version 1202292 (0.0008) [2023-12-27 13:58:47,533][20347] Updated weights for policy 0, policy_version 1202302 (0.0006) [2023-12-27 13:58:47,590][20347] Updated weights for policy 0, policy_version 1202312 (0.0005) [2023-12-27 13:58:48,121][20347] Updated weights for policy 0, policy_version 1202322 (0.0006) [2023-12-27 13:58:48,167][20381] Updated weights for policy 1, policy_version 1201340 (0.0006) [2023-12-27 13:58:48,169][20347] Updated weights for policy 0, policy_version 1202332 (0.0007) [2023-12-27 13:58:48,218][20347] Updated weights for policy 0, policy_version 1202342 (0.0008) [2023-12-27 13:58:48,226][20381] Updated weights for policy 1, policy_version 1201350 (0.0006) [2023-12-27 13:58:48,268][20347] Updated weights for policy 0, policy_version 1202352 (0.0008) [2023-12-27 13:58:48,279][20381] Updated weights for policy 1, policy_version 1201360 (0.0005) [2023-12-27 13:58:48,970][20381] Updated weights for policy 1, policy_version 1201370 (0.0006) [2023-12-27 13:58:48,992][20347] Updated weights for policy 0, policy_version 1202362 (0.0009) [2023-12-27 13:58:49,026][20381] Updated weights for policy 1, policy_version 1201380 (0.0005) [2023-12-27 13:58:49,041][20347] Updated weights for policy 0, policy_version 1202372 (0.0006) [2023-12-27 13:58:49,081][20381] Updated weights for policy 1, policy_version 1201390 (0.0005) [2023-12-27 13:58:49,088][20347] Updated weights for policy 0, policy_version 1202382 (0.0005) [2023-12-27 13:58:49,139][20381] Updated weights for policy 1, policy_version 1201400 (0.0005) [2023-12-27 13:58:49,769][20347] Updated weights for policy 0, policy_version 1202392 (0.0009) [2023-12-27 13:58:49,806][20381] Updated weights for policy 1, policy_version 1201410 (0.0011) [2023-12-27 13:58:49,829][20347] Updated weights for policy 0, policy_version 1202402 (0.0010) [2023-12-27 13:58:49,869][20381] Updated weights for policy 1, policy_version 1201420 (0.0010) [2023-12-27 13:58:49,892][20347] Updated weights for policy 0, policy_version 1202412 (0.0007) [2023-12-27 13:58:49,928][20381] Updated weights for policy 1, policy_version 1201430 (0.0009) [2023-12-27 13:58:50,603][20347] Updated weights for policy 0, policy_version 1202422 (0.0010) [2023-12-27 13:58:50,616][20381] Updated weights for policy 1, policy_version 1201440 (0.0008) [2023-12-27 13:58:50,664][20347] Updated weights for policy 0, policy_version 1202432 (0.0008) [2023-12-27 13:58:50,673][20381] Updated weights for policy 1, policy_version 1201450 (0.0006) [2023-12-27 13:58:50,723][20347] Updated weights for policy 0, policy_version 1202442 (0.0008) [2023-12-27 13:58:50,729][20381] Updated weights for policy 1, policy_version 1201460 (0.0007) [2023-12-27 13:58:51,379][20381] Updated weights for policy 1, policy_version 1201470 (0.0008) [2023-12-27 13:58:51,449][20381] Updated weights for policy 1, policy_version 1201480 (0.0008) [2023-12-27 13:58:51,466][20347] Updated weights for policy 0, policy_version 1202452 (0.0006) [2023-12-27 13:58:51,509][20381] Updated weights for policy 1, policy_version 1201490 (0.0008) [2023-12-27 13:58:51,515][20347] Updated weights for policy 0, policy_version 1202462 (0.0007) [2023-12-27 13:58:51,566][20347] Updated weights for policy 0, policy_version 1202472 (0.0007) [2023-12-27 13:58:52,189][20347] Updated weights for policy 0, policy_version 1202482 (0.0007) [2023-12-27 13:58:52,219][20381] Updated weights for policy 1, policy_version 1201500 (0.0007) [2023-12-27 13:58:52,239][20347] Updated weights for policy 0, policy_version 1202492 (0.0005) [2023-12-27 13:58:52,281][20381] Updated weights for policy 1, policy_version 1201510 (0.0007) [2023-12-27 13:58:52,298][20347] Updated weights for policy 0, policy_version 1202502 (0.0007) [2023-12-27 13:58:52,337][20381] Updated weights for policy 1, policy_version 1201520 (0.0008) [2023-12-27 13:58:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 615497728. Throughput: 0: 10165.8, 1: 10176.4. Samples: 615476156. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:52,364][19170] Avg episode reward: [(0, '912.020'), (1, '598.080')] [2023-12-27 13:58:52,368][20347] Updated weights for policy 0, policy_version 1202512 (0.0008) [2023-12-27 13:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001202512_307896320.pth... [2023-12-27 13:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001201328_307593216.pth [2023-12-27 13:58:52,391][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001201528_307617792.pth... [2023-12-27 13:58:52,394][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001200312_307306496.pth [2023-12-27 13:58:52,918][20381] Updated weights for policy 1, policy_version 1201530 (0.0007) [2023-12-27 13:58:52,975][20381] Updated weights for policy 1, policy_version 1201540 (0.0005) [2023-12-27 13:58:53,032][20381] Updated weights for policy 1, policy_version 1201550 (0.0006) [2023-12-27 13:58:53,086][20347] Updated weights for policy 0, policy_version 1202522 (0.0005) [2023-12-27 13:58:53,086][20381] Updated weights for policy 1, policy_version 1201560 (0.0005) [2023-12-27 13:58:53,149][20347] Updated weights for policy 0, policy_version 1202532 (0.0006) [2023-12-27 13:58:53,211][20347] Updated weights for policy 0, policy_version 1202542 (0.0005) [2023-12-27 13:58:53,603][20381] Updated weights for policy 1, policy_version 1201570 (0.0005) [2023-12-27 13:58:53,666][20381] Updated weights for policy 1, policy_version 1201580 (0.0007) [2023-12-27 13:58:53,725][20381] Updated weights for policy 1, policy_version 1201590 (0.0011) [2023-12-27 13:58:53,776][20347] Updated weights for policy 0, policy_version 1202552 (0.0009) [2023-12-27 13:58:53,827][20347] Updated weights for policy 0, policy_version 1202562 (0.0010) [2023-12-27 13:58:53,875][20347] Updated weights for policy 0, policy_version 1202572 (0.0010) [2023-12-27 13:58:54,377][20381] Updated weights for policy 1, policy_version 1201600 (0.0010) [2023-12-27 13:58:54,423][20381] Updated weights for policy 1, policy_version 1201610 (0.0010) [2023-12-27 13:58:54,468][20381] Updated weights for policy 1, policy_version 1201620 (0.0010) [2023-12-27 13:58:54,508][20347] Updated weights for policy 0, policy_version 1202582 (0.0010) [2023-12-27 13:58:54,565][20347] Updated weights for policy 0, policy_version 1202592 (0.0010) [2023-12-27 13:58:54,617][20347] Updated weights for policy 0, policy_version 1202602 (0.0010) [2023-12-27 13:58:55,187][20381] Updated weights for policy 1, policy_version 1201630 (0.0008) [2023-12-27 13:58:55,247][20381] Updated weights for policy 1, policy_version 1201640 (0.0008) [2023-12-27 13:58:55,273][20347] Updated weights for policy 0, policy_version 1202612 (0.0010) [2023-12-27 13:58:55,305][20381] Updated weights for policy 1, policy_version 1201650 (0.0008) [2023-12-27 13:58:55,333][20347] Updated weights for policy 0, policy_version 1202622 (0.0005) [2023-12-27 13:58:55,388][20347] Updated weights for policy 0, policy_version 1202632 (0.0008) [2023-12-27 13:58:55,913][20381] Updated weights for policy 1, policy_version 1201660 (0.0006) [2023-12-27 13:58:55,963][20381] Updated weights for policy 1, policy_version 1201670 (0.0009) [2023-12-27 13:58:56,006][20347] Updated weights for policy 0, policy_version 1202642 (0.0006) [2023-12-27 13:58:56,016][20381] Updated weights for policy 1, policy_version 1201680 (0.0008) [2023-12-27 13:58:56,061][20347] Updated weights for policy 0, policy_version 1202652 (0.0009) [2023-12-27 13:58:56,118][20347] Updated weights for policy 0, policy_version 1202662 (0.0008) [2023-12-27 13:58:56,175][20347] Updated weights for policy 0, policy_version 1202672 (0.0010) [2023-12-27 13:58:56,638][20381] Updated weights for policy 1, policy_version 1201690 (0.0007) [2023-12-27 13:58:56,695][20381] Updated weights for policy 1, policy_version 1201700 (0.0005) [2023-12-27 13:58:56,753][20381] Updated weights for policy 1, policy_version 1201710 (0.0005) [2023-12-27 13:58:56,805][20347] Updated weights for policy 0, policy_version 1202682 (0.0010) [2023-12-27 13:58:56,810][20381] Updated weights for policy 1, policy_version 1201720 (0.0005) [2023-12-27 13:58:56,860][20347] Updated weights for policy 0, policy_version 1202692 (0.0010) [2023-12-27 13:58:56,905][20347] Updated weights for policy 0, policy_version 1202702 (0.0010) [2023-12-27 13:58:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 615612416. Throughput: 0: 10177.2, 1: 10273.9. Samples: 615603456. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:58:57,363][19170] Avg episode reward: [(0, '872.890'), (1, '682.060')] [2023-12-27 13:58:57,399][20381] Updated weights for policy 1, policy_version 1201730 (0.0010) [2023-12-27 13:58:57,447][20381] Updated weights for policy 1, policy_version 1201740 (0.0007) [2023-12-27 13:58:57,500][20381] Updated weights for policy 1, policy_version 1201750 (0.0005) [2023-12-27 13:58:57,588][20347] Updated weights for policy 0, policy_version 1202712 (0.0006) [2023-12-27 13:58:57,633][20347] Updated weights for policy 0, policy_version 1202722 (0.0005) [2023-12-27 13:58:57,680][20347] Updated weights for policy 0, policy_version 1202732 (0.0005) [2023-12-27 13:58:58,149][20381] Updated weights for policy 1, policy_version 1201760 (0.0005) [2023-12-27 13:58:58,209][20381] Updated weights for policy 1, policy_version 1201770 (0.0006) [2023-12-27 13:58:58,269][20381] Updated weights for policy 1, policy_version 1201780 (0.0006) [2023-12-27 13:58:58,322][20347] Updated weights for policy 0, policy_version 1202742 (0.0007) [2023-12-27 13:58:58,395][20347] Updated weights for policy 0, policy_version 1202752 (0.0007) [2023-12-27 13:58:58,462][20347] Updated weights for policy 0, policy_version 1202762 (0.0008) [2023-12-27 13:58:58,951][20381] Updated weights for policy 1, policy_version 1201790 (0.0007) [2023-12-27 13:58:59,007][20381] Updated weights for policy 1, policy_version 1201800 (0.0006) [2023-12-27 13:58:59,065][20381] Updated weights for policy 1, policy_version 1201810 (0.0008) [2023-12-27 13:58:59,152][20347] Updated weights for policy 0, policy_version 1202772 (0.0007) [2023-12-27 13:58:59,207][20347] Updated weights for policy 0, policy_version 1202782 (0.0006) [2023-12-27 13:58:59,276][20347] Updated weights for policy 0, policy_version 1202792 (0.0008) [2023-12-27 13:58:59,722][20381] Updated weights for policy 1, policy_version 1201820 (0.0007) [2023-12-27 13:58:59,783][20381] Updated weights for policy 1, policy_version 1201830 (0.0007) [2023-12-27 13:58:59,849][20381] Updated weights for policy 1, policy_version 1201840 (0.0007) [2023-12-27 13:59:00,029][20347] Updated weights for policy 0, policy_version 1202802 (0.0007) [2023-12-27 13:59:00,089][20347] Updated weights for policy 0, policy_version 1202812 (0.0007) [2023-12-27 13:59:00,157][20347] Updated weights for policy 0, policy_version 1202822 (0.0008) [2023-12-27 13:59:00,203][20347] Updated weights for policy 0, policy_version 1202832 (0.0007) [2023-12-27 13:59:00,600][20381] Updated weights for policy 1, policy_version 1201850 (0.0008) [2023-12-27 13:59:00,659][20381] Updated weights for policy 1, policy_version 1201860 (0.0006) [2023-12-27 13:59:00,720][20381] Updated weights for policy 1, policy_version 1201870 (0.0005) [2023-12-27 13:59:00,784][20381] Updated weights for policy 1, policy_version 1201880 (0.0006) [2023-12-27 13:59:00,866][20347] Updated weights for policy 0, policy_version 1202842 (0.0008) [2023-12-27 13:59:00,916][20347] Updated weights for policy 0, policy_version 1202852 (0.0007) [2023-12-27 13:59:00,969][20347] Updated weights for policy 0, policy_version 1202862 (0.0007) [2023-12-27 13:59:01,430][20381] Updated weights for policy 1, policy_version 1201890 (0.0008) [2023-12-27 13:59:01,487][20381] Updated weights for policy 1, policy_version 1201900 (0.0008) [2023-12-27 13:59:01,545][20381] Updated weights for policy 1, policy_version 1201910 (0.0008) [2023-12-27 13:59:01,698][20347] Updated weights for policy 0, policy_version 1202872 (0.0008) [2023-12-27 13:59:01,758][20347] Updated weights for policy 0, policy_version 1202882 (0.0008) [2023-12-27 13:59:01,812][20347] Updated weights for policy 0, policy_version 1202892 (0.0009) [2023-12-27 13:59:02,238][20381] Updated weights for policy 1, policy_version 1201920 (0.0007) [2023-12-27 13:59:02,299][20381] Updated weights for policy 1, policy_version 1201930 (0.0008) [2023-12-27 13:59:02,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 615710720. Throughput: 0: 10145.4, 1: 10319.9. Samples: 615726112. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:02,363][20381] Updated weights for policy 1, policy_version 1201940 (0.0008) [2023-12-27 13:59:02,363][19170] Avg episode reward: [(0, '872.870'), (1, '675.230')] [2023-12-27 13:59:02,532][20347] Updated weights for policy 0, policy_version 1202902 (0.0008) [2023-12-27 13:59:02,586][20347] Updated weights for policy 0, policy_version 1202912 (0.0007) [2023-12-27 13:59:02,638][20347] Updated weights for policy 0, policy_version 1202922 (0.0007) [2023-12-27 13:59:03,071][20381] Updated weights for policy 1, policy_version 1201950 (0.0007) [2023-12-27 13:59:03,135][20381] Updated weights for policy 1, policy_version 1201960 (0.0007) [2023-12-27 13:59:03,197][20381] Updated weights for policy 1, policy_version 1201970 (0.0007) [2023-12-27 13:59:03,280][20347] Updated weights for policy 0, policy_version 1202932 (0.0006) [2023-12-27 13:59:03,327][20347] Updated weights for policy 0, policy_version 1202942 (0.0005) [2023-12-27 13:59:03,377][20347] Updated weights for policy 0, policy_version 1202952 (0.0005) [2023-12-27 13:59:03,842][20381] Updated weights for policy 1, policy_version 1201980 (0.0007) [2023-12-27 13:59:03,904][20381] Updated weights for policy 1, policy_version 1201990 (0.0006) [2023-12-27 13:59:03,968][20381] Updated weights for policy 1, policy_version 1202000 (0.0007) [2023-12-27 13:59:04,032][20347] Updated weights for policy 0, policy_version 1202962 (0.0006) [2023-12-27 13:59:04,090][20347] Updated weights for policy 0, policy_version 1202972 (0.0007) [2023-12-27 13:59:04,155][20347] Updated weights for policy 0, policy_version 1202982 (0.0008) [2023-12-27 13:59:04,213][20347] Updated weights for policy 0, policy_version 1202992 (0.0006) [2023-12-27 13:59:04,623][20381] Updated weights for policy 1, policy_version 1202010 (0.0008) [2023-12-27 13:59:04,687][20381] Updated weights for policy 1, policy_version 1202020 (0.0006) [2023-12-27 13:59:04,742][20381] Updated weights for policy 1, policy_version 1202030 (0.0007) [2023-12-27 13:59:04,796][20381] Updated weights for policy 1, policy_version 1202040 (0.0007) [2023-12-27 13:59:04,867][20347] Updated weights for policy 0, policy_version 1203002 (0.0010) [2023-12-27 13:59:04,922][20347] Updated weights for policy 0, policy_version 1203012 (0.0010) [2023-12-27 13:59:04,980][20347] Updated weights for policy 0, policy_version 1203022 (0.0010) [2023-12-27 13:59:05,435][20381] Updated weights for policy 1, policy_version 1202050 (0.0005) [2023-12-27 13:59:05,492][20381] Updated weights for policy 1, policy_version 1202060 (0.0005) [2023-12-27 13:59:05,555][20381] Updated weights for policy 1, policy_version 1202070 (0.0005) [2023-12-27 13:59:05,655][20347] Updated weights for policy 0, policy_version 1203032 (0.0008) [2023-12-27 13:59:05,716][20347] Updated weights for policy 0, policy_version 1203042 (0.0008) [2023-12-27 13:59:05,777][20347] Updated weights for policy 0, policy_version 1203052 (0.0007) [2023-12-27 13:59:06,183][20381] Updated weights for policy 1, policy_version 1202080 (0.0009) [2023-12-27 13:59:06,242][20381] Updated weights for policy 1, policy_version 1202090 (0.0009) [2023-12-27 13:59:06,302][20381] Updated weights for policy 1, policy_version 1202100 (0.0008) [2023-12-27 13:59:06,439][20347] Updated weights for policy 0, policy_version 1203062 (0.0008) [2023-12-27 13:59:06,499][20347] Updated weights for policy 0, policy_version 1203072 (0.0007) [2023-12-27 13:59:06,555][20347] Updated weights for policy 0, policy_version 1203082 (0.0008) [2023-12-27 13:59:06,962][20381] Updated weights for policy 1, policy_version 1202110 (0.0006) [2023-12-27 13:59:07,016][20381] Updated weights for policy 1, policy_version 1202120 (0.0006) [2023-12-27 13:59:07,073][20381] Updated weights for policy 1, policy_version 1202130 (0.0008) [2023-12-27 13:59:07,292][20347] Updated weights for policy 0, policy_version 1203092 (0.0007) [2023-12-27 13:59:07,349][20347] Updated weights for policy 0, policy_version 1203102 (0.0005) [2023-12-27 13:59:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 615817216. Throughput: 0: 10183.6, 1: 10288.9. Samples: 615787932. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:07,363][19170] Avg episode reward: [(0, '951.230'), (1, '692.430')] [2023-12-27 13:59:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001202136_307773440.pth... [2023-12-27 13:59:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001200920_307462144.pth [2023-12-27 13:59:07,407][20347] Updated weights for policy 0, policy_version 1203112 (0.0005) [2023-12-27 13:59:07,450][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001203120_308051968.pth... [2023-12-27 13:59:07,454][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001201904_307740672.pth [2023-12-27 13:59:07,719][20381] Updated weights for policy 1, policy_version 1202140 (0.0007) [2023-12-27 13:59:07,768][20381] Updated weights for policy 1, policy_version 1202150 (0.0005) [2023-12-27 13:59:07,826][20381] Updated weights for policy 1, policy_version 1202160 (0.0005) [2023-12-27 13:59:08,089][20347] Updated weights for policy 0, policy_version 1203122 (0.0006) [2023-12-27 13:59:08,144][20347] Updated weights for policy 0, policy_version 1203132 (0.0008) [2023-12-27 13:59:08,198][20347] Updated weights for policy 0, policy_version 1203142 (0.0006) [2023-12-27 13:59:08,249][20347] Updated weights for policy 0, policy_version 1203152 (0.0005) [2023-12-27 13:59:08,434][20381] Updated weights for policy 1, policy_version 1202170 (0.0006) [2023-12-27 13:59:08,492][20381] Updated weights for policy 1, policy_version 1202180 (0.0008) [2023-12-27 13:59:08,544][20381] Updated weights for policy 1, policy_version 1202190 (0.0008) [2023-12-27 13:59:08,597][20381] Updated weights for policy 1, policy_version 1202200 (0.0006) [2023-12-27 13:59:08,870][20347] Updated weights for policy 0, policy_version 1203162 (0.0007) [2023-12-27 13:59:08,935][20347] Updated weights for policy 0, policy_version 1203172 (0.0008) [2023-12-27 13:59:09,004][20347] Updated weights for policy 0, policy_version 1203182 (0.0008) [2023-12-27 13:59:09,285][20381] Updated weights for policy 1, policy_version 1202210 (0.0008) [2023-12-27 13:59:09,344][20381] Updated weights for policy 1, policy_version 1202220 (0.0008) [2023-12-27 13:59:09,409][20381] Updated weights for policy 1, policy_version 1202230 (0.0008) [2023-12-27 13:59:09,674][20347] Updated weights for policy 0, policy_version 1203192 (0.0009) [2023-12-27 13:59:09,732][20347] Updated weights for policy 0, policy_version 1203202 (0.0009) [2023-12-27 13:59:09,794][20347] Updated weights for policy 0, policy_version 1203212 (0.0009) [2023-12-27 13:59:10,079][20381] Updated weights for policy 1, policy_version 1202240 (0.0007) [2023-12-27 13:59:10,132][20381] Updated weights for policy 1, policy_version 1202250 (0.0007) [2023-12-27 13:59:10,192][20381] Updated weights for policy 1, policy_version 1202260 (0.0005) [2023-12-27 13:59:10,569][20347] Updated weights for policy 0, policy_version 1203222 (0.0007) [2023-12-27 13:59:10,627][20347] Updated weights for policy 0, policy_version 1203232 (0.0008) [2023-12-27 13:59:10,677][20347] Updated weights for policy 0, policy_version 1203242 (0.0008) [2023-12-27 13:59:10,908][20381] Updated weights for policy 1, policy_version 1202270 (0.0006) [2023-12-27 13:59:10,974][20381] Updated weights for policy 1, policy_version 1202280 (0.0008) [2023-12-27 13:59:11,037][20381] Updated weights for policy 1, policy_version 1202290 (0.0010) [2023-12-27 13:59:11,441][20347] Updated weights for policy 0, policy_version 1203252 (0.0008) [2023-12-27 13:59:11,507][20347] Updated weights for policy 0, policy_version 1203262 (0.0008) [2023-12-27 13:59:11,571][20347] Updated weights for policy 0, policy_version 1203272 (0.0007) [2023-12-27 13:59:11,838][20381] Updated weights for policy 1, policy_version 1202300 (0.0010) [2023-12-27 13:59:11,898][20381] Updated weights for policy 1, policy_version 1202310 (0.0011) [2023-12-27 13:59:11,958][20381] Updated weights for policy 1, policy_version 1202320 (0.0011) [2023-12-27 13:59:12,266][20347] Updated weights for policy 0, policy_version 1203282 (0.0008) [2023-12-27 13:59:12,325][20347] Updated weights for policy 0, policy_version 1203292 (0.0008) [2023-12-27 13:59:12,343][20291] KL-divergence is very high: 101.1011 [2023-12-27 13:59:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 615915520. Throughput: 0: 10240.2, 1: 10276.2. Samples: 615910436. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:12,363][19170] Avg episode reward: [(0, '931.530'), (1, '713.060')] [2023-12-27 13:59:12,390][20347] Updated weights for policy 0, policy_version 1203302 (0.0008) [2023-12-27 13:59:12,451][20347] Updated weights for policy 0, policy_version 1203312 (0.0008) [2023-12-27 13:59:12,654][20381] Updated weights for policy 1, policy_version 1202330 (0.0008) [2023-12-27 13:59:12,716][20381] Updated weights for policy 1, policy_version 1202340 (0.0011) [2023-12-27 13:59:12,774][20381] Updated weights for policy 1, policy_version 1202350 (0.0010) [2023-12-27 13:59:12,827][20381] Updated weights for policy 1, policy_version 1202360 (0.0010) [2023-12-27 13:59:13,138][20347] Updated weights for policy 0, policy_version 1203322 (0.0008) [2023-12-27 13:59:13,184][20347] Updated weights for policy 0, policy_version 1203332 (0.0009) [2023-12-27 13:59:13,231][20347] Updated weights for policy 0, policy_version 1203342 (0.0010) [2023-12-27 13:59:13,407][20381] Updated weights for policy 1, policy_version 1202370 (0.0005) [2023-12-27 13:59:13,461][20381] Updated weights for policy 1, policy_version 1202380 (0.0005) [2023-12-27 13:59:13,521][20381] Updated weights for policy 1, policy_version 1202390 (0.0005) [2023-12-27 13:59:13,871][20347] Updated weights for policy 0, policy_version 1203352 (0.0006) [2023-12-27 13:59:13,928][20347] Updated weights for policy 0, policy_version 1203362 (0.0005) [2023-12-27 13:59:13,975][20347] Updated weights for policy 0, policy_version 1203372 (0.0005) [2023-12-27 13:59:14,081][20381] Updated weights for policy 1, policy_version 1202400 (0.0005) [2023-12-27 13:59:14,140][20381] Updated weights for policy 1, policy_version 1202410 (0.0005) [2023-12-27 13:59:14,192][20381] Updated weights for policy 1, policy_version 1202420 (0.0005) [2023-12-27 13:59:14,654][20347] Updated weights for policy 0, policy_version 1203382 (0.0006) [2023-12-27 13:59:14,713][20347] Updated weights for policy 0, policy_version 1203392 (0.0008) [2023-12-27 13:59:14,720][20381] Updated weights for policy 1, policy_version 1202430 (0.0008) [2023-12-27 13:59:14,764][20347] Updated weights for policy 0, policy_version 1203402 (0.0006) [2023-12-27 13:59:14,777][20381] Updated weights for policy 1, policy_version 1202440 (0.0010) [2023-12-27 13:59:14,842][20381] Updated weights for policy 1, policy_version 1202450 (0.0008) [2023-12-27 13:59:15,467][20347] Updated weights for policy 0, policy_version 1203412 (0.0007) [2023-12-27 13:59:15,520][20347] Updated weights for policy 0, policy_version 1203422 (0.0007) [2023-12-27 13:59:15,540][20381] Updated weights for policy 1, policy_version 1202460 (0.0009) [2023-12-27 13:59:15,565][20347] Updated weights for policy 0, policy_version 1203432 (0.0007) [2023-12-27 13:59:15,589][20381] Updated weights for policy 1, policy_version 1202470 (0.0010) [2023-12-27 13:59:15,640][20381] Updated weights for policy 1, policy_version 1202480 (0.0010) [2023-12-27 13:59:16,234][20347] Updated weights for policy 0, policy_version 1203442 (0.0006) [2023-12-27 13:59:16,294][20347] Updated weights for policy 0, policy_version 1203452 (0.0007) [2023-12-27 13:59:16,342][20381] Updated weights for policy 1, policy_version 1202490 (0.0010) [2023-12-27 13:59:16,347][20347] Updated weights for policy 0, policy_version 1203462 (0.0007) [2023-12-27 13:59:16,394][20381] Updated weights for policy 1, policy_version 1202500 (0.0010) [2023-12-27 13:59:16,399][20347] Updated weights for policy 0, policy_version 1203472 (0.0005) [2023-12-27 13:59:16,448][20381] Updated weights for policy 1, policy_version 1202510 (0.0010) [2023-12-27 13:59:16,502][20381] Updated weights for policy 1, policy_version 1202520 (0.0010) [2023-12-27 13:59:17,092][20347] Updated weights for policy 0, policy_version 1203482 (0.0007) [2023-12-27 13:59:17,150][20347] Updated weights for policy 0, policy_version 1203492 (0.0007) [2023-12-27 13:59:17,195][20381] Updated weights for policy 1, policy_version 1202530 (0.0010) [2023-12-27 13:59:17,207][20347] Updated weights for policy 0, policy_version 1203502 (0.0006) [2023-12-27 13:59:17,253][20381] Updated weights for policy 1, policy_version 1202540 (0.0008) [2023-12-27 13:59:17,310][20381] Updated weights for policy 1, policy_version 1202550 (0.0007) [2023-12-27 13:59:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 616030208. Throughput: 0: 10193.5, 1: 10322.7. Samples: 616034096. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:17,363][19170] Avg episode reward: [(0, '902.310'), (1, '693.510')] [2023-12-27 13:59:17,806][20347] Updated weights for policy 0, policy_version 1203512 (0.0007) [2023-12-27 13:59:17,854][20347] Updated weights for policy 0, policy_version 1203522 (0.0007) [2023-12-27 13:59:17,901][20347] Updated weights for policy 0, policy_version 1203532 (0.0008) [2023-12-27 13:59:18,001][20381] Updated weights for policy 1, policy_version 1202560 (0.0008) [2023-12-27 13:59:18,050][20381] Updated weights for policy 1, policy_version 1202570 (0.0008) [2023-12-27 13:59:18,105][20381] Updated weights for policy 1, policy_version 1202580 (0.0007) [2023-12-27 13:59:18,638][20347] Updated weights for policy 0, policy_version 1203542 (0.0006) [2023-12-27 13:59:18,690][20347] Updated weights for policy 0, policy_version 1203552 (0.0010) [2023-12-27 13:59:18,735][20381] Updated weights for policy 1, policy_version 1202590 (0.0007) [2023-12-27 13:59:18,741][20347] Updated weights for policy 0, policy_version 1203562 (0.0011) [2023-12-27 13:59:18,795][20381] Updated weights for policy 1, policy_version 1202600 (0.0006) [2023-12-27 13:59:18,845][20381] Updated weights for policy 1, policy_version 1202610 (0.0008) [2023-12-27 13:59:19,385][20347] Updated weights for policy 0, policy_version 1203572 (0.0009) [2023-12-27 13:59:19,451][20347] Updated weights for policy 0, policy_version 1203582 (0.0005) [2023-12-27 13:59:19,521][20347] Updated weights for policy 0, policy_version 1203592 (0.0007) [2023-12-27 13:59:19,628][20381] Updated weights for policy 1, policy_version 1202620 (0.0010) [2023-12-27 13:59:19,703][20381] Updated weights for policy 1, policy_version 1202630 (0.0011) [2023-12-27 13:59:19,767][20381] Updated weights for policy 1, policy_version 1202640 (0.0011) [2023-12-27 13:59:20,106][20347] Updated weights for policy 0, policy_version 1203602 (0.0006) [2023-12-27 13:59:20,165][20347] Updated weights for policy 0, policy_version 1203612 (0.0007) [2023-12-27 13:59:20,228][20347] Updated weights for policy 0, policy_version 1203622 (0.0008) [2023-12-27 13:59:20,291][20347] Updated weights for policy 0, policy_version 1203632 (0.0010) [2023-12-27 13:59:20,470][20381] Updated weights for policy 1, policy_version 1202650 (0.0010) [2023-12-27 13:59:20,528][20381] Updated weights for policy 1, policy_version 1202660 (0.0008) [2023-12-27 13:59:20,588][20381] Updated weights for policy 1, policy_version 1202670 (0.0008) [2023-12-27 13:59:20,648][20381] Updated weights for policy 1, policy_version 1202680 (0.0009) [2023-12-27 13:59:21,037][20347] Updated weights for policy 0, policy_version 1203642 (0.0009) [2023-12-27 13:59:21,090][20347] Updated weights for policy 0, policy_version 1203652 (0.0009) [2023-12-27 13:59:21,154][20347] Updated weights for policy 0, policy_version 1203662 (0.0008) [2023-12-27 13:59:21,478][20381] Updated weights for policy 1, policy_version 1202690 (0.0006) [2023-12-27 13:59:21,536][20381] Updated weights for policy 1, policy_version 1202700 (0.0008) [2023-12-27 13:59:21,596][20381] Updated weights for policy 1, policy_version 1202710 (0.0007) [2023-12-27 13:59:22,037][20347] Updated weights for policy 0, policy_version 1203672 (0.0008) [2023-12-27 13:59:22,102][20347] Updated weights for policy 0, policy_version 1203682 (0.0008) [2023-12-27 13:59:22,165][20347] Updated weights for policy 0, policy_version 1203692 (0.0008) [2023-12-27 13:59:22,310][20381] Updated weights for policy 1, policy_version 1202720 (0.0008) [2023-12-27 13:59:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 616120320. Throughput: 0: 10234.0, 1: 10285.2. Samples: 616096224. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:22,363][19170] Avg episode reward: [(0, '921.910'), (1, '683.270')] [2023-12-27 13:59:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001203696_308199424.pth... [2023-12-27 13:59:22,373][20381] Updated weights for policy 1, policy_version 1202730 (0.0008) [2023-12-27 13:59:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001202512_307896320.pth [2023-12-27 13:59:22,435][20381] Updated weights for policy 1, policy_version 1202740 (0.0006) [2023-12-27 13:59:22,461][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001202744_307929088.pth... [2023-12-27 13:59:22,464][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001201528_307617792.pth [2023-12-27 13:59:22,884][20347] Updated weights for policy 0, policy_version 1203702 (0.0010) [2023-12-27 13:59:22,944][20347] Updated weights for policy 0, policy_version 1203712 (0.0010) [2023-12-27 13:59:22,994][20347] Updated weights for policy 0, policy_version 1203722 (0.0011) [2023-12-27 13:59:23,148][20381] Updated weights for policy 1, policy_version 1202750 (0.0008) [2023-12-27 13:59:23,203][20381] Updated weights for policy 1, policy_version 1202760 (0.0010) [2023-12-27 13:59:23,261][20381] Updated weights for policy 1, policy_version 1202770 (0.0007) [2023-12-27 13:59:23,673][20347] Updated weights for policy 0, policy_version 1203732 (0.0009) [2023-12-27 13:59:23,721][20347] Updated weights for policy 0, policy_version 1203742 (0.0010) [2023-12-27 13:59:23,769][20347] Updated weights for policy 0, policy_version 1203752 (0.0010) [2023-12-27 13:59:23,853][20381] Updated weights for policy 1, policy_version 1202780 (0.0008) [2023-12-27 13:59:23,907][20381] Updated weights for policy 1, policy_version 1202790 (0.0007) [2023-12-27 13:59:23,955][20381] Updated weights for policy 1, policy_version 1202800 (0.0007) [2023-12-27 13:59:24,435][20347] Updated weights for policy 0, policy_version 1203762 (0.0010) [2023-12-27 13:59:24,489][20347] Updated weights for policy 0, policy_version 1203772 (0.0007) [2023-12-27 13:59:24,544][20347] Updated weights for policy 0, policy_version 1203782 (0.0007) [2023-12-27 13:59:24,595][20347] Updated weights for policy 0, policy_version 1203792 (0.0010) [2023-12-27 13:59:24,639][20381] Updated weights for policy 1, policy_version 1202810 (0.0008) [2023-12-27 13:59:24,695][20381] Updated weights for policy 1, policy_version 1202820 (0.0007) [2023-12-27 13:59:24,749][20381] Updated weights for policy 1, policy_version 1202830 (0.0007) [2023-12-27 13:59:24,799][20381] Updated weights for policy 1, policy_version 1202840 (0.0007) [2023-12-27 13:59:25,284][20347] Updated weights for policy 0, policy_version 1203802 (0.0008) [2023-12-27 13:59:25,335][20347] Updated weights for policy 0, policy_version 1203812 (0.0006) [2023-12-27 13:59:25,386][20347] Updated weights for policy 0, policy_version 1203822 (0.0005) [2023-12-27 13:59:25,439][20381] Updated weights for policy 1, policy_version 1202850 (0.0008) [2023-12-27 13:59:25,490][20381] Updated weights for policy 1, policy_version 1202860 (0.0005) [2023-12-27 13:59:25,543][20381] Updated weights for policy 1, policy_version 1202870 (0.0006) [2023-12-27 13:59:26,037][20347] Updated weights for policy 0, policy_version 1203832 (0.0007) [2023-12-27 13:59:26,093][20347] Updated weights for policy 0, policy_version 1203842 (0.0008) [2023-12-27 13:59:26,123][20381] Updated weights for policy 1, policy_version 1202880 (0.0006) [2023-12-27 13:59:26,150][20347] Updated weights for policy 0, policy_version 1203852 (0.0008) [2023-12-27 13:59:26,181][20381] Updated weights for policy 1, policy_version 1202890 (0.0009) [2023-12-27 13:59:26,227][20381] Updated weights for policy 1, policy_version 1202900 (0.0007) [2023-12-27 13:59:26,806][20381] Updated weights for policy 1, policy_version 1202910 (0.0008) [2023-12-27 13:59:26,856][20381] Updated weights for policy 1, policy_version 1202920 (0.0006) [2023-12-27 13:59:26,861][20347] Updated weights for policy 0, policy_version 1203862 (0.0011) [2023-12-27 13:59:26,913][20381] Updated weights for policy 1, policy_version 1202930 (0.0007) [2023-12-27 13:59:26,917][20347] Updated weights for policy 0, policy_version 1203872 (0.0005) [2023-12-27 13:59:26,974][20347] Updated weights for policy 0, policy_version 1203882 (0.0005) [2023-12-27 13:59:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 616226816. Throughput: 0: 10156.9, 1: 10318.2. Samples: 616216052. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:27,363][19170] Avg episode reward: [(0, '912.080'), (1, '694.030')] [2023-12-27 13:59:27,565][20347] Updated weights for policy 0, policy_version 1203892 (0.0007) [2023-12-27 13:59:27,614][20381] Updated weights for policy 1, policy_version 1202940 (0.0008) [2023-12-27 13:59:27,620][20347] Updated weights for policy 0, policy_version 1203902 (0.0010) [2023-12-27 13:59:27,672][20381] Updated weights for policy 1, policy_version 1202950 (0.0007) [2023-12-27 13:59:27,677][20347] Updated weights for policy 0, policy_version 1203912 (0.0005) [2023-12-27 13:59:27,730][20381] Updated weights for policy 1, policy_version 1202960 (0.0008) [2023-12-27 13:59:28,339][20347] Updated weights for policy 0, policy_version 1203922 (0.0008) [2023-12-27 13:59:28,395][20347] Updated weights for policy 0, policy_version 1203932 (0.0006) [2023-12-27 13:59:28,447][20381] Updated weights for policy 1, policy_version 1202970 (0.0007) [2023-12-27 13:59:28,456][20347] Updated weights for policy 0, policy_version 1203942 (0.0006) [2023-12-27 13:59:28,509][20381] Updated weights for policy 1, policy_version 1202980 (0.0009) [2023-12-27 13:59:28,511][20347] Updated weights for policy 0, policy_version 1203952 (0.0006) [2023-12-27 13:59:28,572][20381] Updated weights for policy 1, policy_version 1202990 (0.0009) [2023-12-27 13:59:28,631][20381] Updated weights for policy 1, policy_version 1203000 (0.0007) [2023-12-27 13:59:29,051][20347] Updated weights for policy 0, policy_version 1203962 (0.0005) [2023-12-27 13:59:29,104][20347] Updated weights for policy 0, policy_version 1203972 (0.0006) [2023-12-27 13:59:29,158][20347] Updated weights for policy 0, policy_version 1203982 (0.0007) [2023-12-27 13:59:29,361][20381] Updated weights for policy 1, policy_version 1203010 (0.0008) [2023-12-27 13:59:29,413][20381] Updated weights for policy 1, policy_version 1203020 (0.0008) [2023-12-27 13:59:29,467][20381] Updated weights for policy 1, policy_version 1203030 (0.0008) [2023-12-27 13:59:29,775][20347] Updated weights for policy 0, policy_version 1203992 (0.0006) [2023-12-27 13:59:29,824][20347] Updated weights for policy 0, policy_version 1204002 (0.0008) [2023-12-27 13:59:29,887][20347] Updated weights for policy 0, policy_version 1204012 (0.0008) [2023-12-27 13:59:30,161][20381] Updated weights for policy 1, policy_version 1203040 (0.0006) [2023-12-27 13:59:30,223][20381] Updated weights for policy 1, policy_version 1203050 (0.0011) [2023-12-27 13:59:30,280][20381] Updated weights for policy 1, policy_version 1203060 (0.0007) [2023-12-27 13:59:30,515][20347] Updated weights for policy 0, policy_version 1204022 (0.0007) [2023-12-27 13:59:30,568][20347] Updated weights for policy 0, policy_version 1204032 (0.0005) [2023-12-27 13:59:30,618][20347] Updated weights for policy 0, policy_version 1204042 (0.0006) [2023-12-27 13:59:30,921][20381] Updated weights for policy 1, policy_version 1203070 (0.0007) [2023-12-27 13:59:30,971][20381] Updated weights for policy 1, policy_version 1203080 (0.0005) [2023-12-27 13:59:31,027][20381] Updated weights for policy 1, policy_version 1203090 (0.0008) [2023-12-27 13:59:31,282][20347] Updated weights for policy 0, policy_version 1204053 (0.0008) [2023-12-27 13:59:31,334][20347] Updated weights for policy 0, policy_version 1204063 (0.0008) [2023-12-27 13:59:31,402][20347] Updated weights for policy 0, policy_version 1204073 (0.0009) [2023-12-27 13:59:31,751][20381] Updated weights for policy 1, policy_version 1203100 (0.0009) [2023-12-27 13:59:31,804][20381] Updated weights for policy 1, policy_version 1203110 (0.0006) [2023-12-27 13:59:31,857][20381] Updated weights for policy 1, policy_version 1203120 (0.0007) [2023-12-27 13:59:32,123][20347] Updated weights for policy 0, policy_version 1204083 (0.0010) [2023-12-27 13:59:32,182][20347] Updated weights for policy 0, policy_version 1204093 (0.0011) [2023-12-27 13:59:32,245][20347] Updated weights for policy 0, policy_version 1204103 (0.0011) [2023-12-27 13:59:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 616333312. Throughput: 0: 10277.2, 1: 10324.9. Samples: 616341428. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:32,363][19170] Avg episode reward: [(0, '911.900'), (1, '717.480')] [2023-12-27 13:59:32,542][20381] Updated weights for policy 1, policy_version 1203130 (0.0007) [2023-12-27 13:59:32,598][20381] Updated weights for policy 1, policy_version 1203140 (0.0005) [2023-12-27 13:59:32,662][20381] Updated weights for policy 1, policy_version 1203150 (0.0005) [2023-12-27 13:59:32,720][20381] Updated weights for policy 1, policy_version 1203160 (0.0006) [2023-12-27 13:59:32,941][20347] Updated weights for policy 0, policy_version 1204113 (0.0009) [2023-12-27 13:59:33,002][20347] Updated weights for policy 0, policy_version 1204123 (0.0008) [2023-12-27 13:59:33,050][20347] Updated weights for policy 0, policy_version 1204133 (0.0009) [2023-12-27 13:59:33,101][20347] Updated weights for policy 0, policy_version 1204143 (0.0009) [2023-12-27 13:59:33,239][20381] Updated weights for policy 1, policy_version 1203170 (0.0006) [2023-12-27 13:59:33,286][20381] Updated weights for policy 1, policy_version 1203180 (0.0009) [2023-12-27 13:59:33,334][20381] Updated weights for policy 1, policy_version 1203190 (0.0007) [2023-12-27 13:59:33,798][20347] Updated weights for policy 0, policy_version 1204153 (0.0007) [2023-12-27 13:59:33,851][20347] Updated weights for policy 0, policy_version 1204163 (0.0007) [2023-12-27 13:59:33,896][20347] Updated weights for policy 0, policy_version 1204173 (0.0010) [2023-12-27 13:59:34,019][20381] Updated weights for policy 1, policy_version 1203200 (0.0007) [2023-12-27 13:59:34,066][20381] Updated weights for policy 1, policy_version 1203210 (0.0007) [2023-12-27 13:59:34,117][20381] Updated weights for policy 1, policy_version 1203220 (0.0008) [2023-12-27 13:59:34,589][20347] Updated weights for policy 0, policy_version 1204183 (0.0009) [2023-12-27 13:59:34,648][20347] Updated weights for policy 0, policy_version 1204193 (0.0010) [2023-12-27 13:59:34,710][20347] Updated weights for policy 0, policy_version 1204203 (0.0011) [2023-12-27 13:59:34,774][20381] Updated weights for policy 1, policy_version 1203230 (0.0006) [2023-12-27 13:59:34,825][20381] Updated weights for policy 1, policy_version 1203240 (0.0005) [2023-12-27 13:59:34,886][20381] Updated weights for policy 1, policy_version 1203250 (0.0005) [2023-12-27 13:59:35,365][20347] Updated weights for policy 0, policy_version 1204213 (0.0010) [2023-12-27 13:59:35,412][20291] KL-divergence is very high: 171.5263 [2023-12-27 13:59:35,422][20347] Updated weights for policy 0, policy_version 1204223 (0.0010) [2023-12-27 13:59:35,423][20291] KL-divergence is very high: 161.0417 [2023-12-27 13:59:35,470][20347] Updated weights for policy 0, policy_version 1204233 (0.0010) [2023-12-27 13:59:35,519][20381] Updated weights for policy 1, policy_version 1203260 (0.0006) [2023-12-27 13:59:35,581][20381] Updated weights for policy 1, policy_version 1203270 (0.0008) [2023-12-27 13:59:35,642][20381] Updated weights for policy 1, policy_version 1203280 (0.0008) [2023-12-27 13:59:36,166][20347] Updated weights for policy 0, policy_version 1204243 (0.0007) [2023-12-27 13:59:36,233][20347] Updated weights for policy 0, policy_version 1204253 (0.0006) [2023-12-27 13:59:36,291][20347] Updated weights for policy 0, policy_version 1204263 (0.0007) [2023-12-27 13:59:36,364][20381] Updated weights for policy 1, policy_version 1203290 (0.0007) [2023-12-27 13:59:36,430][20381] Updated weights for policy 1, policy_version 1203300 (0.0007) [2023-12-27 13:59:36,487][20381] Updated weights for policy 1, policy_version 1203310 (0.0009) [2023-12-27 13:59:36,550][20381] Updated weights for policy 1, policy_version 1203320 (0.0007) [2023-12-27 13:59:36,973][20347] Updated weights for policy 0, policy_version 1204273 (0.0008) [2023-12-27 13:59:37,032][20347] Updated weights for policy 0, policy_version 1204283 (0.0010) [2023-12-27 13:59:37,091][20347] Updated weights for policy 0, policy_version 1204293 (0.0011) [2023-12-27 13:59:37,147][20347] Updated weights for policy 0, policy_version 1204303 (0.0010) [2023-12-27 13:59:37,164][20381] Updated weights for policy 1, policy_version 1203330 (0.0006) [2023-12-27 13:59:37,209][20381] Updated weights for policy 1, policy_version 1203340 (0.0007) [2023-12-27 13:59:37,265][20381] Updated weights for policy 1, policy_version 1203350 (0.0008) [2023-12-27 13:59:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 616439808. Throughput: 0: 10247.0, 1: 10391.7. Samples: 616404892. Policy #0 lag: (min: 28.0, avg: 39.1, max: 60.0) [2023-12-27 13:59:37,363][19170] Avg episode reward: [(0, '902.150'), (1, '700.530')] [2023-12-27 13:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001203352_308084736.pth... [2023-12-27 13:59:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001204304_308355072.pth... [2023-12-27 13:59:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001202136_307773440.pth [2023-12-27 13:59:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001203120_308051968.pth [2023-12-27 13:59:37,749][20347] Updated weights for policy 0, policy_version 1204313 (0.0007) [2023-12-27 13:59:37,802][20347] Updated weights for policy 0, policy_version 1204323 (0.0006) [2023-12-27 13:59:37,858][20347] Updated weights for policy 0, policy_version 1204333 (0.0007) [2023-12-27 13:59:37,940][20381] Updated weights for policy 1, policy_version 1203360 (0.0006) [2023-12-27 13:59:37,992][20381] Updated weights for policy 1, policy_version 1203370 (0.0007) [2023-12-27 13:59:38,052][20381] Updated weights for policy 1, policy_version 1203380 (0.0008) [2023-12-27 13:59:38,550][20347] Updated weights for policy 0, policy_version 1204343 (0.0007) [2023-12-27 13:59:38,606][20347] Updated weights for policy 0, policy_version 1204353 (0.0007) [2023-12-27 13:59:38,652][20347] Updated weights for policy 0, policy_version 1204363 (0.0007) [2023-12-27 13:59:38,761][20381] Updated weights for policy 1, policy_version 1203390 (0.0008) [2023-12-27 13:59:38,820][20381] Updated weights for policy 1, policy_version 1203400 (0.0008) [2023-12-27 13:59:38,870][20381] Updated weights for policy 1, policy_version 1203410 (0.0007) [2023-12-27 13:59:39,382][20347] Updated weights for policy 0, policy_version 1204373 (0.0008) [2023-12-27 13:59:39,424][20291] KL-divergence is very high: 408.8138 [2023-12-27 13:59:39,430][20291] KL-divergence is very high: 392.3257 [2023-12-27 13:59:39,447][20347] Updated weights for policy 0, policy_version 1204383 (0.0008) [2023-12-27 13:59:39,464][20291] KL-divergence is very high: 161.8022 [2023-12-27 13:59:39,469][20291] KL-divergence is very high: 127.4846 [2023-12-27 13:59:39,498][20347] Updated weights for policy 0, policy_version 1204393 (0.0008) [2023-12-27 13:59:39,607][20381] Updated weights for policy 1, policy_version 1203420 (0.0008) [2023-12-27 13:59:39,667][20381] Updated weights for policy 1, policy_version 1203430 (0.0008) [2023-12-27 13:59:39,724][20381] Updated weights for policy 1, policy_version 1203440 (0.0007) [2023-12-27 13:59:40,211][20291] KL-divergence is very high: 274.4975 [2023-12-27 13:59:40,222][20347] Updated weights for policy 0, policy_version 1204403 (0.0007) [2023-12-27 13:59:40,249][20291] KL-divergence is very high: 271.4877 [2023-12-27 13:59:40,256][20291] KL-divergence is very high: 113.0784 [2023-12-27 13:59:40,277][20347] Updated weights for policy 0, policy_version 1204413 (0.0005) [2023-12-27 13:59:40,295][20291] KL-divergence is very high: 265.2261 [2023-12-27 13:59:40,337][20347] Updated weights for policy 0, policy_version 1204423 (0.0006) [2023-12-27 13:59:40,343][20291] KL-divergence is very high: 144.9279 [2023-12-27 13:59:40,461][20381] Updated weights for policy 1, policy_version 1203450 (0.0008) [2023-12-27 13:59:40,524][20381] Updated weights for policy 1, policy_version 1203460 (0.0007) [2023-12-27 13:59:40,585][20381] Updated weights for policy 1, policy_version 1203470 (0.0008) [2023-12-27 13:59:40,648][20381] Updated weights for policy 1, policy_version 1203480 (0.0007) [2023-12-27 13:59:40,955][20347] Updated weights for policy 0, policy_version 1204433 (0.0008) [2023-12-27 13:59:41,015][20347] Updated weights for policy 0, policy_version 1204443 (0.0006) [2023-12-27 13:59:41,077][20347] Updated weights for policy 0, policy_version 1204453 (0.0008) [2023-12-27 13:59:41,142][20347] Updated weights for policy 0, policy_version 1204463 (0.0006) [2023-12-27 13:59:41,394][20381] Updated weights for policy 1, policy_version 1203490 (0.0008) [2023-12-27 13:59:41,447][20381] Updated weights for policy 1, policy_version 1203500 (0.0006) [2023-12-27 13:59:41,500][20381] Updated weights for policy 1, policy_version 1203510 (0.0007) [2023-12-27 13:59:41,872][20347] Updated weights for policy 0, policy_version 1204473 (0.0008) [2023-12-27 13:59:41,927][20347] Updated weights for policy 0, policy_version 1204483 (0.0006) [2023-12-27 13:59:41,987][20347] Updated weights for policy 0, policy_version 1204493 (0.0007) [2023-12-27 13:59:42,252][20381] Updated weights for policy 1, policy_version 1203520 (0.0007) [2023-12-27 13:59:42,324][20381] Updated weights for policy 1, policy_version 1203530 (0.0008) [2023-12-27 13:59:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 616529920. Throughput: 0: 10202.9, 1: 10294.2. Samples: 616525828. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 13:59:42,363][19170] Avg episode reward: [(0, '872.730'), (1, '725.010')] [2023-12-27 13:59:42,389][20381] Updated weights for policy 1, policy_version 1203540 (0.0007) [2023-12-27 13:59:42,648][20347] Updated weights for policy 0, policy_version 1204503 (0.0008) [2023-12-27 13:59:42,708][20347] Updated weights for policy 0, policy_version 1204513 (0.0008) [2023-12-27 13:59:42,771][20347] Updated weights for policy 0, policy_version 1204523 (0.0007) [2023-12-27 13:59:43,006][20381] Updated weights for policy 1, policy_version 1203550 (0.0007) [2023-12-27 13:59:43,060][20381] Updated weights for policy 1, policy_version 1203560 (0.0006) [2023-12-27 13:59:43,121][20381] Updated weights for policy 1, policy_version 1203570 (0.0006) [2023-12-27 13:59:43,482][20347] Updated weights for policy 0, policy_version 1204533 (0.0007) [2023-12-27 13:59:43,540][20347] Updated weights for policy 0, policy_version 1204543 (0.0005) [2023-12-27 13:59:43,584][20347] Updated weights for policy 0, policy_version 1204553 (0.0005) [2023-12-27 13:59:43,693][20381] Updated weights for policy 1, policy_version 1203580 (0.0006) [2023-12-27 13:59:43,762][20381] Updated weights for policy 1, policy_version 1203590 (0.0005) [2023-12-27 13:59:43,827][20381] Updated weights for policy 1, policy_version 1203600 (0.0006) [2023-12-27 13:59:44,184][20347] Updated weights for policy 0, policy_version 1204563 (0.0006) [2023-12-27 13:59:44,237][20347] Updated weights for policy 0, policy_version 1204573 (0.0008) [2023-12-27 13:59:44,285][20347] Updated weights for policy 0, policy_version 1204583 (0.0009) [2023-12-27 13:59:44,343][20381] Updated weights for policy 1, policy_version 1203610 (0.0006) [2023-12-27 13:59:44,391][20381] Updated weights for policy 1, policy_version 1203620 (0.0005) [2023-12-27 13:59:44,444][20381] Updated weights for policy 1, policy_version 1203630 (0.0006) [2023-12-27 13:59:44,494][20381] Updated weights for policy 1, policy_version 1203640 (0.0005) [2023-12-27 13:59:45,030][20347] Updated weights for policy 0, policy_version 1204593 (0.0008) [2023-12-27 13:59:45,079][20347] Updated weights for policy 0, policy_version 1204603 (0.0007) [2023-12-27 13:59:45,137][20347] Updated weights for policy 0, policy_version 1204613 (0.0008) [2023-12-27 13:59:45,167][20381] Updated weights for policy 1, policy_version 1203650 (0.0008) [2023-12-27 13:59:45,197][20347] Updated weights for policy 0, policy_version 1204623 (0.0010) [2023-12-27 13:59:45,227][20381] Updated weights for policy 1, policy_version 1203660 (0.0008) [2023-12-27 13:59:45,287][20381] Updated weights for policy 1, policy_version 1203670 (0.0008) [2023-12-27 13:59:45,818][20347] Updated weights for policy 0, policy_version 1204633 (0.0006) [2023-12-27 13:59:45,866][20347] Updated weights for policy 0, policy_version 1204643 (0.0005) [2023-12-27 13:59:45,922][20347] Updated weights for policy 0, policy_version 1204653 (0.0007) [2023-12-27 13:59:46,015][20381] Updated weights for policy 1, policy_version 1203680 (0.0008) [2023-12-27 13:59:46,079][20381] Updated weights for policy 1, policy_version 1203690 (0.0008) [2023-12-27 13:59:46,139][20381] Updated weights for policy 1, policy_version 1203700 (0.0008) [2023-12-27 13:59:46,563][20347] Updated weights for policy 0, policy_version 1204663 (0.0007) [2023-12-27 13:59:46,618][20347] Updated weights for policy 0, policy_version 1204673 (0.0008) [2023-12-27 13:59:46,673][20347] Updated weights for policy 0, policy_version 1204683 (0.0007) [2023-12-27 13:59:46,804][20381] Updated weights for policy 1, policy_version 1203710 (0.0008) [2023-12-27 13:59:46,862][20381] Updated weights for policy 1, policy_version 1203720 (0.0008) [2023-12-27 13:59:46,927][20381] Updated weights for policy 1, policy_version 1203730 (0.0006) [2023-12-27 13:59:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 616636416. Throughput: 0: 10227.4, 1: 10296.6. Samples: 616649696. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 13:59:47,363][19170] Avg episode reward: [(0, '872.660'), (1, '657.340')] [2023-12-27 13:59:47,375][20347] Updated weights for policy 0, policy_version 1204693 (0.0007) [2023-12-27 13:59:47,439][20347] Updated weights for policy 0, policy_version 1204703 (0.0005) [2023-12-27 13:59:47,496][20347] Updated weights for policy 0, policy_version 1204713 (0.0007) [2023-12-27 13:59:47,593][20381] Updated weights for policy 1, policy_version 1203740 (0.0007) [2023-12-27 13:59:47,642][20381] Updated weights for policy 1, policy_version 1203750 (0.0008) [2023-12-27 13:59:47,695][20381] Updated weights for policy 1, policy_version 1203760 (0.0007) [2023-12-27 13:59:48,083][20347] Updated weights for policy 0, policy_version 1204723 (0.0007) [2023-12-27 13:59:48,133][20347] Updated weights for policy 0, policy_version 1204733 (0.0005) [2023-12-27 13:59:48,190][20347] Updated weights for policy 0, policy_version 1204743 (0.0005) [2023-12-27 13:59:48,367][20381] Updated weights for policy 1, policy_version 1203770 (0.0006) [2023-12-27 13:59:48,426][20381] Updated weights for policy 1, policy_version 1203780 (0.0006) [2023-12-27 13:59:48,490][20381] Updated weights for policy 1, policy_version 1203790 (0.0007) [2023-12-27 13:59:48,557][20381] Updated weights for policy 1, policy_version 1203800 (0.0007) [2023-12-27 13:59:48,806][20347] Updated weights for policy 0, policy_version 1204753 (0.0006) [2023-12-27 13:59:48,873][20347] Updated weights for policy 0, policy_version 1204763 (0.0008) [2023-12-27 13:59:48,936][20347] Updated weights for policy 0, policy_version 1204773 (0.0008) [2023-12-27 13:59:49,000][20347] Updated weights for policy 0, policy_version 1204783 (0.0008) [2023-12-27 13:59:49,228][20381] Updated weights for policy 1, policy_version 1203810 (0.0007) [2023-12-27 13:59:49,285][20381] Updated weights for policy 1, policy_version 1203820 (0.0006) [2023-12-27 13:59:49,338][20381] Updated weights for policy 1, policy_version 1203830 (0.0007) [2023-12-27 13:59:49,673][20347] Updated weights for policy 0, policy_version 1204793 (0.0008) [2023-12-27 13:59:49,732][20347] Updated weights for policy 0, policy_version 1204803 (0.0008) [2023-12-27 13:59:49,789][20347] Updated weights for policy 0, policy_version 1204813 (0.0007) [2023-12-27 13:59:50,081][20381] Updated weights for policy 1, policy_version 1203840 (0.0007) [2023-12-27 13:59:50,139][20381] Updated weights for policy 1, policy_version 1203850 (0.0007) [2023-12-27 13:59:50,197][20381] Updated weights for policy 1, policy_version 1203860 (0.0007) [2023-12-27 13:59:50,488][20347] Updated weights for policy 0, policy_version 1204823 (0.0007) [2023-12-27 13:59:50,555][20347] Updated weights for policy 0, policy_version 1204833 (0.0007) [2023-12-27 13:59:50,619][20347] Updated weights for policy 0, policy_version 1204843 (0.0008) [2023-12-27 13:59:50,837][20381] Updated weights for policy 1, policy_version 1203870 (0.0008) [2023-12-27 13:59:50,886][20381] Updated weights for policy 1, policy_version 1203880 (0.0008) [2023-12-27 13:59:50,931][20381] Updated weights for policy 1, policy_version 1203890 (0.0008) [2023-12-27 13:59:51,283][20347] Updated weights for policy 0, policy_version 1204853 (0.0006) [2023-12-27 13:59:51,336][20347] Updated weights for policy 0, policy_version 1204863 (0.0008) [2023-12-27 13:59:51,399][20347] Updated weights for policy 0, policy_version 1204873 (0.0008) [2023-12-27 13:59:51,733][20381] Updated weights for policy 1, policy_version 1203900 (0.0007) [2023-12-27 13:59:51,799][20381] Updated weights for policy 1, policy_version 1203910 (0.0007) [2023-12-27 13:59:51,864][20381] Updated weights for policy 1, policy_version 1203920 (0.0008) [2023-12-27 13:59:52,084][20347] Updated weights for policy 0, policy_version 1204883 (0.0008) [2023-12-27 13:59:52,137][20347] Updated weights for policy 0, policy_version 1204893 (0.0007) [2023-12-27 13:59:52,188][20347] Updated weights for policy 0, policy_version 1204903 (0.0007) [2023-12-27 13:59:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 616742912. Throughput: 0: 10238.1, 1: 10279.5. Samples: 616711228. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 13:59:52,363][19170] Avg episode reward: [(0, '853.160'), (1, '610.940')] [2023-12-27 13:59:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001203928_308232192.pth... [2023-12-27 13:59:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001204912_308510720.pth... [2023-12-27 13:59:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001203696_308199424.pth [2023-12-27 13:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001202744_307929088.pth [2023-12-27 13:59:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001203928_308232192.pth [2023-12-27 13:59:52,377][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001204912_308510720.pth [2023-12-27 13:59:52,538][20381] Updated weights for policy 1, policy_version 1203930 (0.0009) [2023-12-27 13:59:52,600][20381] Updated weights for policy 1, policy_version 1203940 (0.0011) [2023-12-27 13:59:52,658][20381] Updated weights for policy 1, policy_version 1203950 (0.0010) [2023-12-27 13:59:52,717][20381] Updated weights for policy 1, policy_version 1203960 (0.0011) [2023-12-27 13:59:52,876][20347] Updated weights for policy 0, policy_version 1204913 (0.0007) [2023-12-27 13:59:52,940][20347] Updated weights for policy 0, policy_version 1204923 (0.0007) [2023-12-27 13:59:53,001][20347] Updated weights for policy 0, policy_version 1204933 (0.0007) [2023-12-27 13:59:53,061][20347] Updated weights for policy 0, policy_version 1204943 (0.0006) [2023-12-27 13:59:53,391][20381] Updated weights for policy 1, policy_version 1203970 (0.0011) [2023-12-27 13:59:53,442][20381] Updated weights for policy 1, policy_version 1203980 (0.0010) [2023-12-27 13:59:53,491][20381] Updated weights for policy 1, policy_version 1203990 (0.0010) [2023-12-27 13:59:53,717][20347] Updated weights for policy 0, policy_version 1204953 (0.0007) [2023-12-27 13:59:53,768][20347] Updated weights for policy 0, policy_version 1204963 (0.0006) [2023-12-27 13:59:53,827][20347] Updated weights for policy 0, policy_version 1204973 (0.0006) [2023-12-27 13:59:54,230][20381] Updated weights for policy 1, policy_version 1204000 (0.0009) [2023-12-27 13:59:54,286][20381] Updated weights for policy 1, policy_version 1204010 (0.0006) [2023-12-27 13:59:54,337][20381] Updated weights for policy 1, policy_version 1204020 (0.0007) [2023-12-27 13:59:54,404][20347] Updated weights for policy 0, policy_version 1204983 (0.0007) [2023-12-27 13:59:54,419][20291] KL-divergence is very high: 103.9841 [2023-12-27 13:59:54,453][20347] Updated weights for policy 0, policy_version 1204993 (0.0010) [2023-12-27 13:59:54,457][20291] KL-divergence is very high: 105.1427 [2023-12-27 13:59:54,503][20347] Updated weights for policy 0, policy_version 1205003 (0.0010) [2023-12-27 13:59:55,004][20381] Updated weights for policy 1, policy_version 1204030 (0.0006) [2023-12-27 13:59:55,058][20381] Updated weights for policy 1, policy_version 1204040 (0.0008) [2023-12-27 13:59:55,112][20381] Updated weights for policy 1, policy_version 1204050 (0.0006) [2023-12-27 13:59:55,220][20347] Updated weights for policy 0, policy_version 1205013 (0.0010) [2023-12-27 13:59:55,272][20347] Updated weights for policy 0, policy_version 1205023 (0.0010) [2023-12-27 13:59:55,335][20347] Updated weights for policy 0, policy_version 1205033 (0.0008) [2023-12-27 13:59:55,743][20381] Updated weights for policy 1, policy_version 1204060 (0.0006) [2023-12-27 13:59:55,802][20381] Updated weights for policy 1, policy_version 1204070 (0.0008) [2023-12-27 13:59:55,849][20381] Updated weights for policy 1, policy_version 1204080 (0.0007) [2023-12-27 13:59:55,901][20347] Updated weights for policy 0, policy_version 1205043 (0.0009) [2023-12-27 13:59:55,945][20347] Updated weights for policy 0, policy_version 1205053 (0.0010) [2023-12-27 13:59:56,000][20347] Updated weights for policy 0, policy_version 1205063 (0.0010) [2023-12-27 13:59:56,565][20381] Updated weights for policy 1, policy_version 1204090 (0.0007) [2023-12-27 13:59:56,621][20381] Updated weights for policy 1, policy_version 1204100 (0.0007) [2023-12-27 13:59:56,670][20381] Updated weights for policy 1, policy_version 1204110 (0.0007) [2023-12-27 13:59:56,699][20347] Updated weights for policy 0, policy_version 1205073 (0.0010) [2023-12-27 13:59:56,723][20381] Updated weights for policy 1, policy_version 1204120 (0.0006) [2023-12-27 13:59:56,750][20347] Updated weights for policy 0, policy_version 1205083 (0.0010) [2023-12-27 13:59:56,798][20347] Updated weights for policy 0, policy_version 1205093 (0.0010) [2023-12-27 13:59:56,855][20347] Updated weights for policy 0, policy_version 1205103 (0.0010) [2023-12-27 13:59:57,278][20381] Updated weights for policy 1, policy_version 1204130 (0.0005) [2023-12-27 13:59:57,335][20381] Updated weights for policy 1, policy_version 1204140 (0.0006) [2023-12-27 13:59:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 616841216. Throughput: 0: 10302.3, 1: 10244.7. Samples: 616835052. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 13:59:57,363][19170] Avg episode reward: [(0, '882.560'), (1, '692.420')] [2023-12-27 13:59:57,382][20381] Updated weights for policy 1, policy_version 1204150 (0.0007) [2023-12-27 13:59:57,546][20347] Updated weights for policy 0, policy_version 1205113 (0.0010) [2023-12-27 13:59:57,602][20347] Updated weights for policy 0, policy_version 1205123 (0.0010) [2023-12-27 13:59:57,652][20347] Updated weights for policy 0, policy_version 1205133 (0.0010) [2023-12-27 13:59:57,954][20381] Updated weights for policy 1, policy_version 1204160 (0.0005) [2023-12-27 13:59:58,012][20381] Updated weights for policy 1, policy_version 1204170 (0.0005) [2023-12-27 13:59:58,070][20381] Updated weights for policy 1, policy_version 1204180 (0.0005) [2023-12-27 13:59:58,276][20347] Updated weights for policy 0, policy_version 1205143 (0.0010) [2023-12-27 13:59:58,351][20347] Updated weights for policy 0, policy_version 1205153 (0.0009) [2023-12-27 13:59:58,421][20347] Updated weights for policy 0, policy_version 1205163 (0.0007) [2023-12-27 13:59:58,751][20381] Updated weights for policy 1, policy_version 1204190 (0.0007) [2023-12-27 13:59:58,817][20381] Updated weights for policy 1, policy_version 1204200 (0.0009) [2023-12-27 13:59:58,886][20381] Updated weights for policy 1, policy_version 1204210 (0.0011) [2023-12-27 13:59:59,160][20347] Updated weights for policy 0, policy_version 1205173 (0.0007) [2023-12-27 13:59:59,218][20347] Updated weights for policy 0, policy_version 1205183 (0.0007) [2023-12-27 13:59:59,286][20347] Updated weights for policy 0, policy_version 1205193 (0.0008) [2023-12-27 13:59:59,608][20381] Updated weights for policy 1, policy_version 1204220 (0.0010) [2023-12-27 13:59:59,668][20381] Updated weights for policy 1, policy_version 1204230 (0.0008) [2023-12-27 13:59:59,726][20381] Updated weights for policy 1, policy_version 1204240 (0.0007) [2023-12-27 13:59:59,918][20347] Updated weights for policy 0, policy_version 1205203 (0.0008) [2023-12-27 13:59:59,971][20347] Updated weights for policy 0, policy_version 1205213 (0.0008) [2023-12-27 14:00:00,011][20291] KL-divergence is very high: 106.7351 [2023-12-27 14:00:00,022][20347] Updated weights for policy 0, policy_version 1205223 (0.0009) [2023-12-27 14:00:00,342][20381] Updated weights for policy 1, policy_version 1204250 (0.0008) [2023-12-27 14:00:00,405][20381] Updated weights for policy 1, policy_version 1204260 (0.0006) [2023-12-27 14:00:00,458][20381] Updated weights for policy 1, policy_version 1204270 (0.0005) [2023-12-27 14:00:00,513][20381] Updated weights for policy 1, policy_version 1204280 (0.0005) [2023-12-27 14:00:00,765][20347] Updated weights for policy 0, policy_version 1205233 (0.0008) [2023-12-27 14:00:00,814][20347] Updated weights for policy 0, policy_version 1205243 (0.0006) [2023-12-27 14:00:00,862][20347] Updated weights for policy 0, policy_version 1205253 (0.0005) [2023-12-27 14:00:00,906][20347] Updated weights for policy 0, policy_version 1205263 (0.0007) [2023-12-27 14:00:01,158][20381] Updated weights for policy 1, policy_version 1204290 (0.0008) [2023-12-27 14:00:01,213][20381] Updated weights for policy 1, policy_version 1204300 (0.0008) [2023-12-27 14:00:01,266][20381] Updated weights for policy 1, policy_version 1204310 (0.0008) [2023-12-27 14:00:01,588][20347] Updated weights for policy 0, policy_version 1205273 (0.0006) [2023-12-27 14:00:01,653][20347] Updated weights for policy 0, policy_version 1205283 (0.0007) [2023-12-27 14:00:01,720][20347] Updated weights for policy 0, policy_version 1205293 (0.0007) [2023-12-27 14:00:02,020][20381] Updated weights for policy 1, policy_version 1204320 (0.0008) [2023-12-27 14:00:02,081][20381] Updated weights for policy 1, policy_version 1204330 (0.0008) [2023-12-27 14:00:02,141][20381] Updated weights for policy 1, policy_version 1204340 (0.0005) [2023-12-27 14:00:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 616947712. Throughput: 0: 10304.8, 1: 10248.8. Samples: 616959008. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:02,363][19170] Avg episode reward: [(0, '931.400'), (1, '703.970')] [2023-12-27 14:00:02,383][20347] Updated weights for policy 0, policy_version 1205303 (0.0009) [2023-12-27 14:00:02,434][20347] Updated weights for policy 0, policy_version 1205313 (0.0008) [2023-12-27 14:00:02,489][20347] Updated weights for policy 0, policy_version 1205323 (0.0007) [2023-12-27 14:00:02,812][20381] Updated weights for policy 1, policy_version 1204350 (0.0006) [2023-12-27 14:00:02,874][20381] Updated weights for policy 1, policy_version 1204360 (0.0008) [2023-12-27 14:00:02,928][20381] Updated weights for policy 1, policy_version 1204370 (0.0008) [2023-12-27 14:00:03,174][20347] Updated weights for policy 0, policy_version 1205333 (0.0008) [2023-12-27 14:00:03,230][20347] Updated weights for policy 0, policy_version 1205343 (0.0011) [2023-12-27 14:00:03,290][20347] Updated weights for policy 0, policy_version 1205353 (0.0010) [2023-12-27 14:00:03,622][20381] Updated weights for policy 1, policy_version 1204380 (0.0008) [2023-12-27 14:00:03,680][20381] Updated weights for policy 1, policy_version 1204390 (0.0009) [2023-12-27 14:00:03,733][20381] Updated weights for policy 1, policy_version 1204400 (0.0007) [2023-12-27 14:00:03,939][20347] Updated weights for policy 0, policy_version 1205363 (0.0009) [2023-12-27 14:00:03,996][20347] Updated weights for policy 0, policy_version 1205373 (0.0007) [2023-12-27 14:00:04,048][20347] Updated weights for policy 0, policy_version 1205383 (0.0008) [2023-12-27 14:00:04,469][20381] Updated weights for policy 1, policy_version 1204410 (0.0008) [2023-12-27 14:00:04,524][20381] Updated weights for policy 1, policy_version 1204420 (0.0006) [2023-12-27 14:00:04,578][20381] Updated weights for policy 1, policy_version 1204430 (0.0008) [2023-12-27 14:00:04,631][20381] Updated weights for policy 1, policy_version 1204440 (0.0007) [2023-12-27 14:00:04,708][20347] Updated weights for policy 0, policy_version 1205393 (0.0008) [2023-12-27 14:00:04,770][20347] Updated weights for policy 0, policy_version 1205403 (0.0011) [2023-12-27 14:00:04,831][20347] Updated weights for policy 0, policy_version 1205413 (0.0010) [2023-12-27 14:00:04,886][20347] Updated weights for policy 0, policy_version 1205423 (0.0010) [2023-12-27 14:00:05,345][20381] Updated weights for policy 1, policy_version 1204450 (0.0008) [2023-12-27 14:00:05,389][20381] Updated weights for policy 1, policy_version 1204460 (0.0008) [2023-12-27 14:00:05,441][20381] Updated weights for policy 1, policy_version 1204470 (0.0008) [2023-12-27 14:00:05,551][20347] Updated weights for policy 0, policy_version 1205433 (0.0006) [2023-12-27 14:00:05,597][20347] Updated weights for policy 0, policy_version 1205443 (0.0005) [2023-12-27 14:00:05,643][20347] Updated weights for policy 0, policy_version 1205453 (0.0005) [2023-12-27 14:00:06,178][20381] Updated weights for policy 1, policy_version 1204480 (0.0007) [2023-12-27 14:00:06,232][20381] Updated weights for policy 1, policy_version 1204490 (0.0008) [2023-12-27 14:00:06,284][20381] Updated weights for policy 1, policy_version 1204500 (0.0008) [2023-12-27 14:00:06,343][20347] Updated weights for policy 0, policy_version 1205463 (0.0009) [2023-12-27 14:00:06,402][20347] Updated weights for policy 0, policy_version 1205473 (0.0010) [2023-12-27 14:00:06,458][20347] Updated weights for policy 0, policy_version 1205483 (0.0010) [2023-12-27 14:00:07,002][20381] Updated weights for policy 1, policy_version 1204510 (0.0007) [2023-12-27 14:00:07,054][20381] Updated weights for policy 1, policy_version 1204520 (0.0007) [2023-12-27 14:00:07,105][20381] Updated weights for policy 1, policy_version 1204530 (0.0008) [2023-12-27 14:00:07,146][20347] Updated weights for policy 0, policy_version 1205493 (0.0010) [2023-12-27 14:00:07,205][20347] Updated weights for policy 0, policy_version 1205503 (0.0010) [2023-12-27 14:00:07,267][20347] Updated weights for policy 0, policy_version 1205513 (0.0010) [2023-12-27 14:00:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 617054208. Throughput: 0: 10276.7, 1: 10245.4. Samples: 617019720. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:07,363][19170] Avg episode reward: [(0, '872.730'), (1, '717.260')] [2023-12-27 14:00:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001204536_308387840.pth... [2023-12-27 14:00:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001205520_308666368.pth... [2023-12-27 14:00:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001203352_308084736.pth [2023-12-27 14:00:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001204304_308355072.pth [2023-12-27 14:00:07,830][20381] Updated weights for policy 1, policy_version 1204540 (0.0006) [2023-12-27 14:00:07,884][20381] Updated weights for policy 1, policy_version 1204550 (0.0007) [2023-12-27 14:00:07,928][20347] Updated weights for policy 0, policy_version 1205523 (0.0010) [2023-12-27 14:00:07,934][20381] Updated weights for policy 1, policy_version 1204560 (0.0007) [2023-12-27 14:00:07,976][20347] Updated weights for policy 0, policy_version 1205533 (0.0010) [2023-12-27 14:00:08,028][20347] Updated weights for policy 0, policy_version 1205543 (0.0010) [2023-12-27 14:00:08,620][20381] Updated weights for policy 1, policy_version 1204570 (0.0006) [2023-12-27 14:00:08,673][20381] Updated weights for policy 1, policy_version 1204580 (0.0007) [2023-12-27 14:00:08,728][20381] Updated weights for policy 1, policy_version 1204590 (0.0008) [2023-12-27 14:00:08,731][20347] Updated weights for policy 0, policy_version 1205553 (0.0010) [2023-12-27 14:00:08,781][20381] Updated weights for policy 1, policy_version 1204600 (0.0005) [2023-12-27 14:00:08,790][20347] Updated weights for policy 0, policy_version 1205563 (0.0010) [2023-12-27 14:00:08,854][20347] Updated weights for policy 0, policy_version 1205573 (0.0010) [2023-12-27 14:00:08,920][20347] Updated weights for policy 0, policy_version 1205583 (0.0008) [2023-12-27 14:00:09,528][20381] Updated weights for policy 1, policy_version 1204610 (0.0007) [2023-12-27 14:00:09,589][20381] Updated weights for policy 1, policy_version 1204620 (0.0007) [2023-12-27 14:00:09,652][20381] Updated weights for policy 1, policy_version 1204630 (0.0006) [2023-12-27 14:00:09,657][20347] Updated weights for policy 0, policy_version 1205593 (0.0010) [2023-12-27 14:00:09,721][20347] Updated weights for policy 0, policy_version 1205603 (0.0011) [2023-12-27 14:00:09,783][20347] Updated weights for policy 0, policy_version 1205613 (0.0010) [2023-12-27 14:00:10,385][20381] Updated weights for policy 1, policy_version 1204640 (0.0008) [2023-12-27 14:00:10,444][20381] Updated weights for policy 1, policy_version 1204650 (0.0009) [2023-12-27 14:00:10,463][20347] Updated weights for policy 0, policy_version 1205623 (0.0010) [2023-12-27 14:00:10,501][20381] Updated weights for policy 1, policy_version 1204660 (0.0006) [2023-12-27 14:00:10,519][20347] Updated weights for policy 0, policy_version 1205633 (0.0010) [2023-12-27 14:00:10,577][20347] Updated weights for policy 0, policy_version 1205643 (0.0010) [2023-12-27 14:00:11,091][20381] Updated weights for policy 1, policy_version 1204670 (0.0008) [2023-12-27 14:00:11,160][20381] Updated weights for policy 1, policy_version 1204680 (0.0008) [2023-12-27 14:00:11,217][20381] Updated weights for policy 1, policy_version 1204690 (0.0008) [2023-12-27 14:00:11,258][20347] Updated weights for policy 0, policy_version 1205653 (0.0009) [2023-12-27 14:00:11,315][20347] Updated weights for policy 0, policy_version 1205663 (0.0008) [2023-12-27 14:00:11,380][20347] Updated weights for policy 0, policy_version 1205673 (0.0008) [2023-12-27 14:00:12,052][20381] Updated weights for policy 1, policy_version 1204700 (0.0007) [2023-12-27 14:00:12,059][20347] Updated weights for policy 0, policy_version 1205683 (0.0008) [2023-12-27 14:00:12,113][20381] Updated weights for policy 1, policy_version 1204710 (0.0007) [2023-12-27 14:00:12,114][20347] Updated weights for policy 0, policy_version 1205693 (0.0007) [2023-12-27 14:00:12,170][20381] Updated weights for policy 1, policy_version 1204720 (0.0008) [2023-12-27 14:00:12,173][20347] Updated weights for policy 0, policy_version 1205703 (0.0006) [2023-12-27 14:00:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 617152512. Throughput: 0: 10318.9, 1: 10201.7. Samples: 617139480. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:12,363][19170] Avg episode reward: [(0, '882.570'), (1, '762.670')] [2023-12-27 14:00:12,867][20347] Updated weights for policy 0, policy_version 1205713 (0.0007) [2023-12-27 14:00:12,874][20381] Updated weights for policy 1, policy_version 1204730 (0.0007) [2023-12-27 14:00:12,927][20381] Updated weights for policy 1, policy_version 1204740 (0.0011) [2023-12-27 14:00:12,929][20347] Updated weights for policy 0, policy_version 1205723 (0.0007) [2023-12-27 14:00:12,980][20381] Updated weights for policy 1, policy_version 1204750 (0.0011) [2023-12-27 14:00:12,994][20347] Updated weights for policy 0, policy_version 1205733 (0.0006) [2023-12-27 14:00:13,043][20381] Updated weights for policy 1, policy_version 1204760 (0.0011) [2023-12-27 14:00:13,057][20347] Updated weights for policy 0, policy_version 1205743 (0.0006) [2023-12-27 14:00:13,713][20347] Updated weights for policy 0, policy_version 1205753 (0.0006) [2023-12-27 14:00:13,733][20381] Updated weights for policy 1, policy_version 1204770 (0.0006) [2023-12-27 14:00:13,761][20347] Updated weights for policy 0, policy_version 1205763 (0.0006) [2023-12-27 14:00:13,788][20381] Updated weights for policy 1, policy_version 1204780 (0.0007) [2023-12-27 14:00:13,810][20347] Updated weights for policy 0, policy_version 1205773 (0.0006) [2023-12-27 14:00:13,844][20381] Updated weights for policy 1, policy_version 1204790 (0.0006) [2023-12-27 14:00:14,426][20347] Updated weights for policy 0, policy_version 1205783 (0.0007) [2023-12-27 14:00:14,473][20347] Updated weights for policy 0, policy_version 1205793 (0.0008) [2023-12-27 14:00:14,506][20381] Updated weights for policy 1, policy_version 1204800 (0.0007) [2023-12-27 14:00:14,519][20347] Updated weights for policy 0, policy_version 1205803 (0.0005) [2023-12-27 14:00:14,567][20381] Updated weights for policy 1, policy_version 1204810 (0.0007) [2023-12-27 14:00:14,624][20381] Updated weights for policy 1, policy_version 1204820 (0.0008) [2023-12-27 14:00:15,212][20347] Updated weights for policy 0, policy_version 1205813 (0.0008) [2023-12-27 14:00:15,275][20347] Updated weights for policy 0, policy_version 1205823 (0.0006) [2023-12-27 14:00:15,296][20381] Updated weights for policy 1, policy_version 1204830 (0.0008) [2023-12-27 14:00:15,339][20347] Updated weights for policy 0, policy_version 1205833 (0.0005) [2023-12-27 14:00:15,361][20381] Updated weights for policy 1, policy_version 1204840 (0.0008) [2023-12-27 14:00:15,424][20381] Updated weights for policy 1, policy_version 1204850 (0.0008) [2023-12-27 14:00:15,867][20347] Updated weights for policy 0, policy_version 1205843 (0.0006) [2023-12-27 14:00:15,961][20347] Updated weights for policy 0, policy_version 1205853 (0.0007) [2023-12-27 14:00:16,033][20347] Updated weights for policy 0, policy_version 1205863 (0.0009) [2023-12-27 14:00:16,093][20381] Updated weights for policy 1, policy_version 1204860 (0.0005) [2023-12-27 14:00:16,153][20381] Updated weights for policy 1, policy_version 1204870 (0.0005) [2023-12-27 14:00:16,212][20381] Updated weights for policy 1, policy_version 1204880 (0.0006) [2023-12-27 14:00:16,538][20347] Updated weights for policy 0, policy_version 1205873 (0.0007) [2023-12-27 14:00:16,597][20347] Updated weights for policy 0, policy_version 1205883 (0.0006) [2023-12-27 14:00:16,653][20291] KL-divergence is very high: 138.5717 [2023-12-27 14:00:16,658][20347] Updated weights for policy 0, policy_version 1205893 (0.0006) [2023-12-27 14:00:16,665][20291] KL-divergence is very high: 136.7834 [2023-12-27 14:00:16,720][20347] Updated weights for policy 0, policy_version 1205903 (0.0010) [2023-12-27 14:00:16,823][20381] Updated weights for policy 1, policy_version 1204890 (0.0007) [2023-12-27 14:00:16,876][20381] Updated weights for policy 1, policy_version 1204900 (0.0005) [2023-12-27 14:00:16,928][20381] Updated weights for policy 1, policy_version 1204910 (0.0006) [2023-12-27 14:00:16,980][20381] Updated weights for policy 1, policy_version 1204920 (0.0009) [2023-12-27 14:00:17,346][20347] Updated weights for policy 0, policy_version 1205913 (0.0006) [2023-12-27 14:00:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 617250816. Throughput: 0: 10322.1, 1: 10194.7. Samples: 617264688. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:17,364][19170] Avg episode reward: [(0, '882.650'), (1, '664.340')] [2023-12-27 14:00:17,394][20347] Updated weights for policy 0, policy_version 1205923 (0.0005) [2023-12-27 14:00:17,446][20347] Updated weights for policy 0, policy_version 1205933 (0.0005) [2023-12-27 14:00:17,582][20381] Updated weights for policy 1, policy_version 1204930 (0.0010) [2023-12-27 14:00:17,628][20381] Updated weights for policy 1, policy_version 1204940 (0.0010) [2023-12-27 14:00:17,682][20381] Updated weights for policy 1, policy_version 1204950 (0.0010) [2023-12-27 14:00:18,004][20347] Updated weights for policy 0, policy_version 1205943 (0.0005) [2023-12-27 14:00:18,055][20347] Updated weights for policy 0, policy_version 1205953 (0.0005) [2023-12-27 14:00:18,117][20347] Updated weights for policy 0, policy_version 1205963 (0.0005) [2023-12-27 14:00:18,387][20381] Updated weights for policy 1, policy_version 1204960 (0.0010) [2023-12-27 14:00:18,452][20381] Updated weights for policy 1, policy_version 1204970 (0.0011) [2023-12-27 14:00:18,514][20381] Updated weights for policy 1, policy_version 1204980 (0.0010) [2023-12-27 14:00:18,686][20347] Updated weights for policy 0, policy_version 1205973 (0.0006) [2023-12-27 14:00:18,742][20347] Updated weights for policy 0, policy_version 1205983 (0.0008) [2023-12-27 14:00:18,791][20347] Updated weights for policy 0, policy_version 1205993 (0.0008) [2023-12-27 14:00:19,180][20381] Updated weights for policy 1, policy_version 1204990 (0.0010) [2023-12-27 14:00:19,241][20381] Updated weights for policy 1, policy_version 1205000 (0.0010) [2023-12-27 14:00:19,296][20381] Updated weights for policy 1, policy_version 1205010 (0.0010) [2023-12-27 14:00:19,414][20347] Updated weights for policy 0, policy_version 1206003 (0.0009) [2023-12-27 14:00:19,471][20347] Updated weights for policy 0, policy_version 1206013 (0.0010) [2023-12-27 14:00:19,531][20347] Updated weights for policy 0, policy_version 1206023 (0.0008) [2023-12-27 14:00:20,081][20381] Updated weights for policy 1, policy_version 1205020 (0.0011) [2023-12-27 14:00:20,144][20381] Updated weights for policy 1, policy_version 1205030 (0.0011) [2023-12-27 14:00:20,206][20381] Updated weights for policy 1, policy_version 1205040 (0.0010) [2023-12-27 14:00:20,269][20347] Updated weights for policy 0, policy_version 1206033 (0.0008) [2023-12-27 14:00:20,322][20347] Updated weights for policy 0, policy_version 1206043 (0.0008) [2023-12-27 14:00:20,378][20347] Updated weights for policy 0, policy_version 1206053 (0.0006) [2023-12-27 14:00:20,437][20347] Updated weights for policy 0, policy_version 1206063 (0.0007) [2023-12-27 14:00:20,916][20381] Updated weights for policy 1, policy_version 1205050 (0.0008) [2023-12-27 14:00:20,973][20381] Updated weights for policy 1, policy_version 1205060 (0.0008) [2023-12-27 14:00:21,029][20381] Updated weights for policy 1, policy_version 1205070 (0.0006) [2023-12-27 14:00:21,087][20381] Updated weights for policy 1, policy_version 1205080 (0.0007) [2023-12-27 14:00:21,215][20347] Updated weights for policy 0, policy_version 1206073 (0.0007) [2023-12-27 14:00:21,275][20347] Updated weights for policy 0, policy_version 1206083 (0.0008) [2023-12-27 14:00:21,339][20347] Updated weights for policy 0, policy_version 1206093 (0.0008) [2023-12-27 14:00:21,743][20381] Updated weights for policy 1, policy_version 1205090 (0.0008) [2023-12-27 14:00:21,808][20381] Updated weights for policy 1, policy_version 1205100 (0.0007) [2023-12-27 14:00:21,869][20381] Updated weights for policy 1, policy_version 1205110 (0.0007) [2023-12-27 14:00:22,098][20347] Updated weights for policy 0, policy_version 1206103 (0.0008) [2023-12-27 14:00:22,161][20347] Updated weights for policy 0, policy_version 1206113 (0.0008) [2023-12-27 14:00:22,220][20347] Updated weights for policy 0, policy_version 1206123 (0.0009) [2023-12-27 14:00:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 617357312. Throughput: 0: 10382.0, 1: 10123.8. Samples: 617327656. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:22,363][19170] Avg episode reward: [(0, '892.350'), (1, '700.380')] [2023-12-27 14:00:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001205112_308535296.pth... [2023-12-27 14:00:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001206128_308822016.pth... [2023-12-27 14:00:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001203928_308232192.pth [2023-12-27 14:00:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001204912_308510720.pth [2023-12-27 14:00:22,543][20381] Updated weights for policy 1, policy_version 1205120 (0.0008) [2023-12-27 14:00:22,602][20381] Updated weights for policy 1, policy_version 1205130 (0.0008) [2023-12-27 14:00:22,666][20381] Updated weights for policy 1, policy_version 1205140 (0.0008) [2023-12-27 14:00:23,000][20347] Updated weights for policy 0, policy_version 1206133 (0.0007) [2023-12-27 14:00:23,057][20347] Updated weights for policy 0, policy_version 1206143 (0.0006) [2023-12-27 14:00:23,119][20347] Updated weights for policy 0, policy_version 1206153 (0.0007) [2023-12-27 14:00:23,343][20381] Updated weights for policy 1, policy_version 1205150 (0.0008) [2023-12-27 14:00:23,405][20381] Updated weights for policy 1, policy_version 1205160 (0.0008) [2023-12-27 14:00:23,460][20381] Updated weights for policy 1, policy_version 1205170 (0.0007) [2023-12-27 14:00:23,680][20347] Updated weights for policy 0, policy_version 1206163 (0.0007) [2023-12-27 14:00:23,743][20347] Updated weights for policy 0, policy_version 1206173 (0.0007) [2023-12-27 14:00:23,794][20347] Updated weights for policy 0, policy_version 1206183 (0.0008) [2023-12-27 14:00:24,141][20381] Updated weights for policy 1, policy_version 1205180 (0.0008) [2023-12-27 14:00:24,207][20381] Updated weights for policy 1, policy_version 1205190 (0.0008) [2023-12-27 14:00:24,258][20381] Updated weights for policy 1, policy_version 1205200 (0.0008) [2023-12-27 14:00:24,446][20347] Updated weights for policy 0, policy_version 1206193 (0.0008) [2023-12-27 14:00:24,498][20347] Updated weights for policy 0, policy_version 1206203 (0.0009) [2023-12-27 14:00:24,553][20347] Updated weights for policy 0, policy_version 1206213 (0.0007) [2023-12-27 14:00:24,608][20347] Updated weights for policy 0, policy_version 1206223 (0.0009) [2023-12-27 14:00:24,892][20381] Updated weights for policy 1, policy_version 1205210 (0.0006) [2023-12-27 14:00:24,949][20381] Updated weights for policy 1, policy_version 1205220 (0.0007) [2023-12-27 14:00:25,008][20381] Updated weights for policy 1, policy_version 1205230 (0.0008) [2023-12-27 14:00:25,060][20381] Updated weights for policy 1, policy_version 1205240 (0.0007) [2023-12-27 14:00:25,264][20347] Updated weights for policy 0, policy_version 1206233 (0.0006) [2023-12-27 14:00:25,323][20347] Updated weights for policy 0, policy_version 1206243 (0.0006) [2023-12-27 14:00:25,389][20347] Updated weights for policy 0, policy_version 1206253 (0.0006) [2023-12-27 14:00:25,766][20381] Updated weights for policy 1, policy_version 1205250 (0.0007) [2023-12-27 14:00:25,828][20381] Updated weights for policy 1, policy_version 1205260 (0.0008) [2023-12-27 14:00:25,881][20381] Updated weights for policy 1, policy_version 1205270 (0.0008) [2023-12-27 14:00:26,036][20347] Updated weights for policy 0, policy_version 1206263 (0.0007) [2023-12-27 14:00:26,100][20347] Updated weights for policy 0, policy_version 1206273 (0.0007) [2023-12-27 14:00:26,159][20347] Updated weights for policy 0, policy_version 1206283 (0.0007) [2023-12-27 14:00:26,584][20381] Updated weights for policy 1, policy_version 1205280 (0.0009) [2023-12-27 14:00:26,630][20381] Updated weights for policy 1, policy_version 1205290 (0.0010) [2023-12-27 14:00:26,676][20381] Updated weights for policy 1, policy_version 1205300 (0.0010) [2023-12-27 14:00:26,740][20347] Updated weights for policy 0, policy_version 1206293 (0.0007) [2023-12-27 14:00:26,788][20347] Updated weights for policy 0, policy_version 1206303 (0.0008) [2023-12-27 14:00:26,841][20347] Updated weights for policy 0, policy_version 1206313 (0.0008) [2023-12-27 14:00:27,307][20381] Updated weights for policy 1, policy_version 1205310 (0.0008) [2023-12-27 14:00:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 617455616. Throughput: 0: 10359.4, 1: 10146.0. Samples: 617448568. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:27,363][19170] Avg episode reward: [(0, '911.950'), (1, '662.340')] [2023-12-27 14:00:27,365][20381] Updated weights for policy 1, policy_version 1205320 (0.0006) [2023-12-27 14:00:27,408][20381] Updated weights for policy 1, policy_version 1205330 (0.0005) [2023-12-27 14:00:27,519][20347] Updated weights for policy 0, policy_version 1206323 (0.0007) [2023-12-27 14:00:27,567][20347] Updated weights for policy 0, policy_version 1206333 (0.0006) [2023-12-27 14:00:27,624][20347] Updated weights for policy 0, policy_version 1206343 (0.0010) [2023-12-27 14:00:28,006][20381] Updated weights for policy 1, policy_version 1205340 (0.0006) [2023-12-27 14:00:28,059][20381] Updated weights for policy 1, policy_version 1205350 (0.0005) [2023-12-27 14:00:28,112][20381] Updated weights for policy 1, policy_version 1205360 (0.0005) [2023-12-27 14:00:28,286][20347] Updated weights for policy 0, policy_version 1206353 (0.0010) [2023-12-27 14:00:28,332][20291] KL-divergence is very high: 134.5029 [2023-12-27 14:00:28,343][20347] Updated weights for policy 0, policy_version 1206363 (0.0009) [2023-12-27 14:00:28,409][20347] Updated weights for policy 0, policy_version 1206373 (0.0009) [2023-12-27 14:00:28,464][20291] KL-divergence is very high: 117.4987 [2023-12-27 14:00:28,471][20347] Updated weights for policy 0, policy_version 1206383 (0.0010) [2023-12-27 14:00:28,746][20381] Updated weights for policy 1, policy_version 1205370 (0.0007) [2023-12-27 14:00:28,805][20381] Updated weights for policy 1, policy_version 1205380 (0.0008) [2023-12-27 14:00:28,868][20381] Updated weights for policy 1, policy_version 1205390 (0.0008) [2023-12-27 14:00:28,924][20381] Updated weights for policy 1, policy_version 1205400 (0.0008) [2023-12-27 14:00:29,144][20291] KL-divergence is very high: 134.7073 [2023-12-27 14:00:29,194][20291] KL-divergence is very high: 118.9273 [2023-12-27 14:00:29,200][20347] Updated weights for policy 0, policy_version 1206393 (0.0010) [2023-12-27 14:00:29,262][20347] Updated weights for policy 0, policy_version 1206403 (0.0010) [2023-12-27 14:00:29,331][20347] Updated weights for policy 0, policy_version 1206413 (0.0009) [2023-12-27 14:00:29,606][20381] Updated weights for policy 1, policy_version 1205410 (0.0008) [2023-12-27 14:00:29,662][20381] Updated weights for policy 1, policy_version 1205420 (0.0008) [2023-12-27 14:00:29,721][20381] Updated weights for policy 1, policy_version 1205430 (0.0008) [2023-12-27 14:00:30,048][20347] Updated weights for policy 0, policy_version 1206423 (0.0009) [2023-12-27 14:00:30,111][20347] Updated weights for policy 0, policy_version 1206433 (0.0007) [2023-12-27 14:00:30,175][20347] Updated weights for policy 0, policy_version 1206443 (0.0009) [2023-12-27 14:00:30,392][20381] Updated weights for policy 1, policy_version 1205440 (0.0007) [2023-12-27 14:00:30,452][20381] Updated weights for policy 1, policy_version 1205450 (0.0008) [2023-12-27 14:00:30,513][20381] Updated weights for policy 1, policy_version 1205460 (0.0009) [2023-12-27 14:00:30,840][20347] Updated weights for policy 0, policy_version 1206453 (0.0010) [2023-12-27 14:00:30,900][20347] Updated weights for policy 0, policy_version 1206463 (0.0010) [2023-12-27 14:00:30,955][20347] Updated weights for policy 0, policy_version 1206473 (0.0010) [2023-12-27 14:00:31,235][20381] Updated weights for policy 1, policy_version 1205470 (0.0009) [2023-12-27 14:00:31,298][20381] Updated weights for policy 1, policy_version 1205480 (0.0011) [2023-12-27 14:00:31,355][20381] Updated weights for policy 1, policy_version 1205490 (0.0011) [2023-12-27 14:00:31,676][20347] Updated weights for policy 0, policy_version 1206483 (0.0009) [2023-12-27 14:00:31,740][20347] Updated weights for policy 0, policy_version 1206493 (0.0006) [2023-12-27 14:00:31,801][20347] Updated weights for policy 0, policy_version 1206503 (0.0007) [2023-12-27 14:00:32,072][20381] Updated weights for policy 1, policy_version 1205500 (0.0009) [2023-12-27 14:00:32,133][20381] Updated weights for policy 1, policy_version 1205510 (0.0008) [2023-12-27 14:00:32,197][20381] Updated weights for policy 1, policy_version 1205520 (0.0009) [2023-12-27 14:00:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 617562112. Throughput: 0: 10360.0, 1: 10125.8. Samples: 617571556. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:32,363][19170] Avg episode reward: [(0, '882.440'), (1, '605.150')] [2023-12-27 14:00:32,476][20347] Updated weights for policy 0, policy_version 1206513 (0.0007) [2023-12-27 14:00:32,539][20347] Updated weights for policy 0, policy_version 1206523 (0.0008) [2023-12-27 14:00:32,592][20347] Updated weights for policy 0, policy_version 1206533 (0.0007) [2023-12-27 14:00:32,651][20347] Updated weights for policy 0, policy_version 1206543 (0.0007) [2023-12-27 14:00:32,922][20381] Updated weights for policy 1, policy_version 1205530 (0.0008) [2023-12-27 14:00:32,967][20381] Updated weights for policy 1, policy_version 1205540 (0.0007) [2023-12-27 14:00:33,021][20381] Updated weights for policy 1, policy_version 1205550 (0.0008) [2023-12-27 14:00:33,076][20381] Updated weights for policy 1, policy_version 1205560 (0.0007) [2023-12-27 14:00:33,336][20347] Updated weights for policy 0, policy_version 1206553 (0.0008) [2023-12-27 14:00:33,354][20291] KL-divergence is very high: 149.8145 [2023-12-27 14:00:33,393][20347] Updated weights for policy 0, policy_version 1206563 (0.0007) [2023-12-27 14:00:33,399][20291] KL-divergence is very high: 137.5034 [2023-12-27 14:00:33,448][20347] Updated weights for policy 0, policy_version 1206573 (0.0006) [2023-12-27 14:00:33,732][20381] Updated weights for policy 1, policy_version 1205570 (0.0008) [2023-12-27 14:00:33,787][20381] Updated weights for policy 1, policy_version 1205580 (0.0006) [2023-12-27 14:00:33,848][20381] Updated weights for policy 1, policy_version 1205590 (0.0005) [2023-12-27 14:00:34,071][20347] Updated weights for policy 0, policy_version 1206583 (0.0006) [2023-12-27 14:00:34,126][20347] Updated weights for policy 0, policy_version 1206593 (0.0008) [2023-12-27 14:00:34,192][20347] Updated weights for policy 0, policy_version 1206603 (0.0007) [2023-12-27 14:00:34,514][20381] Updated weights for policy 1, policy_version 1205600 (0.0008) [2023-12-27 14:00:34,574][20381] Updated weights for policy 1, policy_version 1205610 (0.0008) [2023-12-27 14:00:34,643][20381] Updated weights for policy 1, policy_version 1205620 (0.0007) [2023-12-27 14:00:34,911][20347] Updated weights for policy 0, policy_version 1206613 (0.0008) [2023-12-27 14:00:34,968][20347] Updated weights for policy 0, policy_version 1206623 (0.0009) [2023-12-27 14:00:35,024][20347] Updated weights for policy 0, policy_version 1206633 (0.0009) [2023-12-27 14:00:35,325][20381] Updated weights for policy 1, policy_version 1205630 (0.0009) [2023-12-27 14:00:35,371][20381] Updated weights for policy 1, policy_version 1205640 (0.0007) [2023-12-27 14:00:35,416][20381] Updated weights for policy 1, policy_version 1205650 (0.0007) [2023-12-27 14:00:35,796][20347] Updated weights for policy 0, policy_version 1206643 (0.0009) [2023-12-27 14:00:35,856][20347] Updated weights for policy 0, policy_version 1206653 (0.0009) [2023-12-27 14:00:35,918][20347] Updated weights for policy 0, policy_version 1206663 (0.0008) [2023-12-27 14:00:36,138][20381] Updated weights for policy 1, policy_version 1205660 (0.0008) [2023-12-27 14:00:36,200][20381] Updated weights for policy 1, policy_version 1205670 (0.0006) [2023-12-27 14:00:36,260][20381] Updated weights for policy 1, policy_version 1205680 (0.0005) [2023-12-27 14:00:36,596][20347] Updated weights for policy 0, policy_version 1206673 (0.0007) [2023-12-27 14:00:36,650][20347] Updated weights for policy 0, policy_version 1206683 (0.0008) [2023-12-27 14:00:36,709][20347] Updated weights for policy 0, policy_version 1206693 (0.0007) [2023-12-27 14:00:36,763][20347] Updated weights for policy 0, policy_version 1206703 (0.0005) [2023-12-27 14:00:36,966][20381] Updated weights for policy 1, policy_version 1205690 (0.0008) [2023-12-27 14:00:37,034][20381] Updated weights for policy 1, policy_version 1205700 (0.0008) [2023-12-27 14:00:37,096][20381] Updated weights for policy 1, policy_version 1205710 (0.0008) [2023-12-27 14:00:37,161][20381] Updated weights for policy 1, policy_version 1205720 (0.0007) [2023-12-27 14:00:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 617660416. Throughput: 0: 10309.7, 1: 10142.5. Samples: 617631576. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:37,363][19170] Avg episode reward: [(0, '892.240'), (1, '719.210')] [2023-12-27 14:00:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001206704_308969472.pth... [2023-12-27 14:00:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001205720_308690944.pth... [2023-12-27 14:00:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001205520_308666368.pth [2023-12-27 14:00:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001204536_308387840.pth [2023-12-27 14:00:37,442][20347] Updated weights for policy 0, policy_version 1206713 (0.0007) [2023-12-27 14:00:37,497][20347] Updated weights for policy 0, policy_version 1206723 (0.0007) [2023-12-27 14:00:37,543][20347] Updated weights for policy 0, policy_version 1206733 (0.0008) [2023-12-27 14:00:37,833][20381] Updated weights for policy 1, policy_version 1205730 (0.0008) [2023-12-27 14:00:37,892][20381] Updated weights for policy 1, policy_version 1205740 (0.0005) [2023-12-27 14:00:37,948][20381] Updated weights for policy 1, policy_version 1205750 (0.0005) [2023-12-27 14:00:38,192][20347] Updated weights for policy 0, policy_version 1206743 (0.0006) [2023-12-27 14:00:38,255][20347] Updated weights for policy 0, policy_version 1206753 (0.0005) [2023-12-27 14:00:38,308][20347] Updated weights for policy 0, policy_version 1206763 (0.0006) [2023-12-27 14:00:38,611][20381] Updated weights for policy 1, policy_version 1205760 (0.0008) [2023-12-27 14:00:38,661][20381] Updated weights for policy 1, policy_version 1205770 (0.0009) [2023-12-27 14:00:38,721][20381] Updated weights for policy 1, policy_version 1205780 (0.0008) [2023-12-27 14:00:38,894][20347] Updated weights for policy 0, policy_version 1206773 (0.0007) [2023-12-27 14:00:38,940][20347] Updated weights for policy 0, policy_version 1206783 (0.0008) [2023-12-27 14:00:38,989][20347] Updated weights for policy 0, policy_version 1206793 (0.0007) [2023-12-27 14:00:39,456][20381] Updated weights for policy 1, policy_version 1205790 (0.0007) [2023-12-27 14:00:39,511][20381] Updated weights for policy 1, policy_version 1205800 (0.0008) [2023-12-27 14:00:39,567][20381] Updated weights for policy 1, policy_version 1205810 (0.0009) [2023-12-27 14:00:39,749][20347] Updated weights for policy 0, policy_version 1206803 (0.0009) [2023-12-27 14:00:39,806][20347] Updated weights for policy 0, policy_version 1206813 (0.0009) [2023-12-27 14:00:39,885][20347] Updated weights for policy 0, policy_version 1206823 (0.0007) [2023-12-27 14:00:40,230][20381] Updated weights for policy 1, policy_version 1205820 (0.0007) [2023-12-27 14:00:40,288][20381] Updated weights for policy 1, policy_version 1205830 (0.0007) [2023-12-27 14:00:40,347][20381] Updated weights for policy 1, policy_version 1205840 (0.0005) [2023-12-27 14:00:40,642][20347] Updated weights for policy 0, policy_version 1206833 (0.0008) [2023-12-27 14:00:40,690][20347] Updated weights for policy 0, policy_version 1206843 (0.0007) [2023-12-27 14:00:40,738][20347] Updated weights for policy 0, policy_version 1206853 (0.0006) [2023-12-27 14:00:40,791][20347] Updated weights for policy 0, policy_version 1206863 (0.0008) [2023-12-27 14:00:41,040][20381] Updated weights for policy 1, policy_version 1205850 (0.0007) [2023-12-27 14:00:41,103][20381] Updated weights for policy 1, policy_version 1205860 (0.0008) [2023-12-27 14:00:41,171][20381] Updated weights for policy 1, policy_version 1205870 (0.0008) [2023-12-27 14:00:41,231][20381] Updated weights for policy 1, policy_version 1205880 (0.0008) [2023-12-27 14:00:41,542][20347] Updated weights for policy 0, policy_version 1206873 (0.0007) [2023-12-27 14:00:41,603][20347] Updated weights for policy 0, policy_version 1206883 (0.0008) [2023-12-27 14:00:41,664][20347] Updated weights for policy 0, policy_version 1206893 (0.0008) [2023-12-27 14:00:41,967][20381] Updated weights for policy 1, policy_version 1205890 (0.0006) [2023-12-27 14:00:42,025][20381] Updated weights for policy 1, policy_version 1205900 (0.0007) [2023-12-27 14:00:42,073][20381] Updated weights for policy 1, policy_version 1205910 (0.0007) [2023-12-27 14:00:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 617758720. Throughput: 0: 10240.3, 1: 10116.7. Samples: 617751116. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:42,363][19170] Avg episode reward: [(0, '911.720'), (1, '654.660')] [2023-12-27 14:00:42,415][20347] Updated weights for policy 0, policy_version 1206903 (0.0007) [2023-12-27 14:00:42,477][20347] Updated weights for policy 0, policy_version 1206913 (0.0007) [2023-12-27 14:00:42,530][20347] Updated weights for policy 0, policy_version 1206923 (0.0007) [2023-12-27 14:00:42,773][20381] Updated weights for policy 1, policy_version 1205920 (0.0006) [2023-12-27 14:00:42,826][20381] Updated weights for policy 1, policy_version 1205930 (0.0011) [2023-12-27 14:00:42,875][20381] Updated weights for policy 1, policy_version 1205940 (0.0010) [2023-12-27 14:00:43,144][20347] Updated weights for policy 0, policy_version 1206933 (0.0006) [2023-12-27 14:00:43,194][20347] Updated weights for policy 0, policy_version 1206943 (0.0007) [2023-12-27 14:00:43,246][20347] Updated weights for policy 0, policy_version 1206953 (0.0007) [2023-12-27 14:00:43,573][20381] Updated weights for policy 1, policy_version 1205950 (0.0011) [2023-12-27 14:00:43,635][20381] Updated weights for policy 1, policy_version 1205960 (0.0011) [2023-12-27 14:00:43,693][20381] Updated weights for policy 1, policy_version 1205970 (0.0010) [2023-12-27 14:00:43,920][20347] Updated weights for policy 0, policy_version 1206963 (0.0008) [2023-12-27 14:00:43,979][20347] Updated weights for policy 0, policy_version 1206973 (0.0007) [2023-12-27 14:00:44,021][20347] Updated weights for policy 0, policy_version 1206983 (0.0007) [2023-12-27 14:00:44,252][20381] Updated weights for policy 1, policy_version 1205980 (0.0007) [2023-12-27 14:00:44,308][20381] Updated weights for policy 1, policy_version 1205990 (0.0009) [2023-12-27 14:00:44,367][20381] Updated weights for policy 1, policy_version 1206000 (0.0010) [2023-12-27 14:00:44,634][20347] Updated weights for policy 0, policy_version 1206993 (0.0006) [2023-12-27 14:00:44,687][20347] Updated weights for policy 0, policy_version 1207003 (0.0007) [2023-12-27 14:00:44,745][20347] Updated weights for policy 0, policy_version 1207013 (0.0006) [2023-12-27 14:00:44,809][20347] Updated weights for policy 0, policy_version 1207023 (0.0008) [2023-12-27 14:00:44,956][20381] Updated weights for policy 1, policy_version 1206010 (0.0009) [2023-12-27 14:00:45,023][20381] Updated weights for policy 1, policy_version 1206020 (0.0009) [2023-12-27 14:00:45,082][20381] Updated weights for policy 1, policy_version 1206030 (0.0009) [2023-12-27 14:00:45,151][20381] Updated weights for policy 1, policy_version 1206040 (0.0008) [2023-12-27 14:00:45,491][20347] Updated weights for policy 0, policy_version 1207033 (0.0009) [2023-12-27 14:00:45,547][20347] Updated weights for policy 0, policy_version 1207043 (0.0005) [2023-12-27 14:00:45,610][20347] Updated weights for policy 0, policy_version 1207053 (0.0005) [2023-12-27 14:00:45,857][20381] Updated weights for policy 1, policy_version 1206050 (0.0008) [2023-12-27 14:00:45,916][20381] Updated weights for policy 1, policy_version 1206060 (0.0008) [2023-12-27 14:00:45,966][20381] Updated weights for policy 1, policy_version 1206070 (0.0008) [2023-12-27 14:00:46,161][20347] Updated weights for policy 0, policy_version 1207063 (0.0005) [2023-12-27 14:00:46,218][20347] Updated weights for policy 0, policy_version 1207073 (0.0005) [2023-12-27 14:00:46,274][20347] Updated weights for policy 0, policy_version 1207083 (0.0009) [2023-12-27 14:00:46,695][20381] Updated weights for policy 1, policy_version 1206080 (0.0008) [2023-12-27 14:00:46,755][20381] Updated weights for policy 1, policy_version 1206090 (0.0007) [2023-12-27 14:00:46,802][20381] Updated weights for policy 1, policy_version 1206100 (0.0007) [2023-12-27 14:00:46,908][20347] Updated weights for policy 0, policy_version 1207093 (0.0010) [2023-12-27 14:00:46,965][20347] Updated weights for policy 0, policy_version 1207103 (0.0010) [2023-12-27 14:00:47,026][20347] Updated weights for policy 0, policy_version 1207113 (0.0009) [2023-12-27 14:00:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 617865216. Throughput: 0: 10298.6, 1: 10091.8. Samples: 617876572. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:47,363][19170] Avg episode reward: [(0, '931.320'), (1, '672.320')] [2023-12-27 14:00:47,467][20381] Updated weights for policy 1, policy_version 1206110 (0.0008) [2023-12-27 14:00:47,517][20381] Updated weights for policy 1, policy_version 1206120 (0.0007) [2023-12-27 14:00:47,568][20381] Updated weights for policy 1, policy_version 1206130 (0.0007) [2023-12-27 14:00:47,694][20347] Updated weights for policy 0, policy_version 1207123 (0.0010) [2023-12-27 14:00:47,746][20347] Updated weights for policy 0, policy_version 1207133 (0.0010) [2023-12-27 14:00:47,800][20347] Updated weights for policy 0, policy_version 1207143 (0.0010) [2023-12-27 14:00:48,264][20381] Updated weights for policy 1, policy_version 1206140 (0.0008) [2023-12-27 14:00:48,325][20381] Updated weights for policy 1, policy_version 1206150 (0.0008) [2023-12-27 14:00:48,393][20381] Updated weights for policy 1, policy_version 1206160 (0.0007) [2023-12-27 14:00:48,486][20347] Updated weights for policy 0, policy_version 1207153 (0.0010) [2023-12-27 14:00:48,539][20347] Updated weights for policy 0, policy_version 1207163 (0.0008) [2023-12-27 14:00:48,590][20291] KL-divergence is very high: 111.7164 [2023-12-27 14:00:48,601][20347] Updated weights for policy 0, policy_version 1207173 (0.0007) [2023-12-27 14:00:48,670][20347] Updated weights for policy 0, policy_version 1207183 (0.0007) [2023-12-27 14:00:49,076][20381] Updated weights for policy 1, policy_version 1206170 (0.0008) [2023-12-27 14:00:49,126][20381] Updated weights for policy 1, policy_version 1206180 (0.0007) [2023-12-27 14:00:49,173][20381] Updated weights for policy 1, policy_version 1206190 (0.0007) [2023-12-27 14:00:49,229][20381] Updated weights for policy 1, policy_version 1206200 (0.0008) [2023-12-27 14:00:49,290][20347] Updated weights for policy 0, policy_version 1207193 (0.0011) [2023-12-27 14:00:49,355][20347] Updated weights for policy 0, policy_version 1207203 (0.0010) [2023-12-27 14:00:49,424][20347] Updated weights for policy 0, policy_version 1207213 (0.0009) [2023-12-27 14:00:49,986][20381] Updated weights for policy 1, policy_version 1206210 (0.0010) [2023-12-27 14:00:50,039][20381] Updated weights for policy 1, policy_version 1206220 (0.0006) [2023-12-27 14:00:50,102][20381] Updated weights for policy 1, policy_version 1206230 (0.0008) [2023-12-27 14:00:50,126][20347] Updated weights for policy 0, policy_version 1207223 (0.0008) [2023-12-27 14:00:50,182][20347] Updated weights for policy 0, policy_version 1207233 (0.0006) [2023-12-27 14:00:50,243][20347] Updated weights for policy 0, policy_version 1207243 (0.0007) [2023-12-27 14:00:50,789][20381] Updated weights for policy 1, policy_version 1206240 (0.0007) [2023-12-27 14:00:50,842][20381] Updated weights for policy 1, policy_version 1206250 (0.0007) [2023-12-27 14:00:50,895][20347] Updated weights for policy 0, policy_version 1207253 (0.0007) [2023-12-27 14:00:50,901][20381] Updated weights for policy 1, policy_version 1206260 (0.0008) [2023-12-27 14:00:50,958][20347] Updated weights for policy 0, policy_version 1207263 (0.0007) [2023-12-27 14:00:51,018][20347] Updated weights for policy 0, policy_version 1207273 (0.0008) [2023-12-27 14:00:51,624][20381] Updated weights for policy 1, policy_version 1206270 (0.0008) [2023-12-27 14:00:51,683][20381] Updated weights for policy 1, policy_version 1206280 (0.0008) [2023-12-27 14:00:51,759][20381] Updated weights for policy 1, policy_version 1206290 (0.0008) [2023-12-27 14:00:51,818][20347] Updated weights for policy 0, policy_version 1207283 (0.0007) [2023-12-27 14:00:51,875][20347] Updated weights for policy 0, policy_version 1207293 (0.0008) [2023-12-27 14:00:51,932][20347] Updated weights for policy 0, policy_version 1207303 (0.0007) [2023-12-27 14:00:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 617963520. Throughput: 0: 10295.1, 1: 10087.3. Samples: 617936928. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:52,363][19170] Avg episode reward: [(0, '950.950'), (1, '731.810')] [2023-12-27 14:00:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001206296_308838400.pth... [2023-12-27 14:00:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001207312_309125120.pth... [2023-12-27 14:00:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001205112_308535296.pth [2023-12-27 14:00:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001206128_308822016.pth [2023-12-27 14:00:52,482][20381] Updated weights for policy 1, policy_version 1206300 (0.0008) [2023-12-27 14:00:52,539][20381] Updated weights for policy 1, policy_version 1206310 (0.0007) [2023-12-27 14:00:52,598][20381] Updated weights for policy 1, policy_version 1206320 (0.0008) [2023-12-27 14:00:52,612][20347] Updated weights for policy 0, policy_version 1207313 (0.0006) [2023-12-27 14:00:52,669][20347] Updated weights for policy 0, policy_version 1207323 (0.0006) [2023-12-27 14:00:52,725][20347] Updated weights for policy 0, policy_version 1207333 (0.0008) [2023-12-27 14:00:52,781][20347] Updated weights for policy 0, policy_version 1207343 (0.0007) [2023-12-27 14:00:53,345][20381] Updated weights for policy 1, policy_version 1206330 (0.0008) [2023-12-27 14:00:53,403][20381] Updated weights for policy 1, policy_version 1206340 (0.0008) [2023-12-27 14:00:53,416][20347] Updated weights for policy 0, policy_version 1207353 (0.0006) [2023-12-27 14:00:53,428][20291] KL-divergence is very high: 215.1357 [2023-12-27 14:00:53,452][20381] Updated weights for policy 1, policy_version 1206350 (0.0007) [2023-12-27 14:00:53,468][20347] Updated weights for policy 0, policy_version 1207363 (0.0005) [2023-12-27 14:00:53,469][20291] KL-divergence is very high: 212.6375 [2023-12-27 14:00:53,511][20381] Updated weights for policy 1, policy_version 1206360 (0.0008) [2023-12-27 14:00:53,520][20347] Updated weights for policy 0, policy_version 1207373 (0.0006) [2023-12-27 14:00:54,128][20347] Updated weights for policy 0, policy_version 1207383 (0.0006) [2023-12-27 14:00:54,142][20381] Updated weights for policy 1, policy_version 1206370 (0.0006) [2023-12-27 14:00:54,177][20347] Updated weights for policy 0, policy_version 1207393 (0.0006) [2023-12-27 14:00:54,204][20381] Updated weights for policy 1, policy_version 1206380 (0.0008) [2023-12-27 14:00:54,230][20347] Updated weights for policy 0, policy_version 1207403 (0.0006) [2023-12-27 14:00:54,248][20381] Updated weights for policy 1, policy_version 1206390 (0.0006) [2023-12-27 14:00:54,841][20347] Updated weights for policy 0, policy_version 1207413 (0.0006) [2023-12-27 14:00:54,891][20347] Updated weights for policy 0, policy_version 1207423 (0.0006) [2023-12-27 14:00:54,941][20381] Updated weights for policy 1, policy_version 1206400 (0.0007) [2023-12-27 14:00:54,959][20347] Updated weights for policy 0, policy_version 1207433 (0.0005) [2023-12-27 14:00:54,992][20381] Updated weights for policy 1, policy_version 1206410 (0.0006) [2023-12-27 14:00:55,042][20381] Updated weights for policy 1, policy_version 1206420 (0.0008) [2023-12-27 14:00:55,634][20347] Updated weights for policy 0, policy_version 1207443 (0.0008) [2023-12-27 14:00:55,686][20347] Updated weights for policy 0, policy_version 1207453 (0.0006) [2023-12-27 14:00:55,710][20381] Updated weights for policy 1, policy_version 1206430 (0.0007) [2023-12-27 14:00:55,738][20347] Updated weights for policy 0, policy_version 1207463 (0.0006) [2023-12-27 14:00:55,752][20381] Updated weights for policy 1, policy_version 1206440 (0.0005) [2023-12-27 14:00:55,802][20381] Updated weights for policy 1, policy_version 1206450 (0.0007) [2023-12-27 14:00:56,400][20347] Updated weights for policy 0, policy_version 1207473 (0.0007) [2023-12-27 14:00:56,449][20347] Updated weights for policy 0, policy_version 1207483 (0.0008) [2023-12-27 14:00:56,493][20347] Updated weights for policy 0, policy_version 1207493 (0.0007) [2023-12-27 14:00:56,530][20381] Updated weights for policy 1, policy_version 1206460 (0.0008) [2023-12-27 14:00:56,540][20347] Updated weights for policy 0, policy_version 1207503 (0.0005) [2023-12-27 14:00:56,585][20381] Updated weights for policy 1, policy_version 1206470 (0.0008) [2023-12-27 14:00:56,642][20381] Updated weights for policy 1, policy_version 1206480 (0.0007) [2023-12-27 14:00:57,157][20347] Updated weights for policy 0, policy_version 1207513 (0.0010) [2023-12-27 14:00:57,213][20347] Updated weights for policy 0, policy_version 1207523 (0.0010) [2023-12-27 14:00:57,244][20381] Updated weights for policy 1, policy_version 1206490 (0.0007) [2023-12-27 14:00:57,261][20347] Updated weights for policy 0, policy_version 1207533 (0.0010) [2023-12-27 14:00:57,297][20381] Updated weights for policy 1, policy_version 1206500 (0.0005) [2023-12-27 14:00:57,350][20381] Updated weights for policy 1, policy_version 1206510 (0.0008) [2023-12-27 14:00:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20522.1). Total num frames: 618070016. Throughput: 0: 10338.9, 1: 10100.4. Samples: 618059248. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:00:57,363][19170] Avg episode reward: [(0, '921.650'), (1, '712.270')] [2023-12-27 14:00:57,410][20381] Updated weights for policy 1, policy_version 1206520 (0.0010) [2023-12-27 14:00:57,921][20347] Updated weights for policy 0, policy_version 1207543 (0.0010) [2023-12-27 14:00:57,977][20347] Updated weights for policy 0, policy_version 1207553 (0.0009) [2023-12-27 14:00:58,005][20381] Updated weights for policy 1, policy_version 1206530 (0.0010) [2023-12-27 14:00:58,036][20347] Updated weights for policy 0, policy_version 1207563 (0.0007) [2023-12-27 14:00:58,060][20381] Updated weights for policy 1, policy_version 1206540 (0.0010) [2023-12-27 14:00:58,108][20381] Updated weights for policy 1, policy_version 1206550 (0.0010) [2023-12-27 14:00:58,760][20347] Updated weights for policy 0, policy_version 1207573 (0.0009) [2023-12-27 14:00:58,823][20347] Updated weights for policy 0, policy_version 1207583 (0.0007) [2023-12-27 14:00:58,891][20347] Updated weights for policy 0, policy_version 1207593 (0.0007) [2023-12-27 14:00:58,901][20381] Updated weights for policy 1, policy_version 1206560 (0.0008) [2023-12-27 14:00:58,906][20292] KL-divergence is very high: 130.3856 [2023-12-27 14:00:58,963][20381] Updated weights for policy 1, policy_version 1206570 (0.0008) [2023-12-27 14:00:59,029][20381] Updated weights for policy 1, policy_version 1206580 (0.0008) [2023-12-27 14:00:59,648][20347] Updated weights for policy 0, policy_version 1207603 (0.0006) [2023-12-27 14:00:59,700][20347] Updated weights for policy 0, policy_version 1207613 (0.0010) [2023-12-27 14:00:59,752][20381] Updated weights for policy 1, policy_version 1206590 (0.0010) [2023-12-27 14:00:59,752][20347] Updated weights for policy 0, policy_version 1207623 (0.0011) [2023-12-27 14:00:59,810][20381] Updated weights for policy 1, policy_version 1206600 (0.0010) [2023-12-27 14:00:59,871][20381] Updated weights for policy 1, policy_version 1206610 (0.0011) [2023-12-27 14:01:00,429][20347] Updated weights for policy 0, policy_version 1207633 (0.0010) [2023-12-27 14:01:00,491][20347] Updated weights for policy 0, policy_version 1207643 (0.0006) [2023-12-27 14:01:00,544][20347] Updated weights for policy 0, policy_version 1207653 (0.0010) [2023-12-27 14:01:00,588][20381] Updated weights for policy 1, policy_version 1206620 (0.0008) [2023-12-27 14:01:00,598][20347] Updated weights for policy 0, policy_version 1207663 (0.0010) [2023-12-27 14:01:00,650][20381] Updated weights for policy 1, policy_version 1206630 (0.0006) [2023-12-27 14:01:00,708][20381] Updated weights for policy 1, policy_version 1206640 (0.0006) [2023-12-27 14:01:01,272][20347] Updated weights for policy 0, policy_version 1207673 (0.0011) [2023-12-27 14:01:01,321][20347] Updated weights for policy 0, policy_version 1207683 (0.0010) [2023-12-27 14:01:01,346][20381] Updated weights for policy 1, policy_version 1206650 (0.0010) [2023-12-27 14:01:01,377][20347] Updated weights for policy 0, policy_version 1207693 (0.0010) [2023-12-27 14:01:01,412][20381] Updated weights for policy 1, policy_version 1206660 (0.0007) [2023-12-27 14:01:01,470][20381] Updated weights for policy 1, policy_version 1206670 (0.0008) [2023-12-27 14:01:01,519][20381] Updated weights for policy 1, policy_version 1206680 (0.0005) [2023-12-27 14:01:02,119][20347] Updated weights for policy 0, policy_version 1207703 (0.0008) [2023-12-27 14:01:02,177][20347] Updated weights for policy 0, policy_version 1207713 (0.0007) [2023-12-27 14:01:02,218][20381] Updated weights for policy 1, policy_version 1206690 (0.0007) [2023-12-27 14:01:02,232][20347] Updated weights for policy 0, policy_version 1207723 (0.0008) [2023-12-27 14:01:02,274][20381] Updated weights for policy 1, policy_version 1206700 (0.0009) [2023-12-27 14:01:02,337][20381] Updated weights for policy 1, policy_version 1206710 (0.0008) [2023-12-27 14:01:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 618176512. Throughput: 0: 10261.8, 1: 10095.8. Samples: 618180780. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:02,363][19170] Avg episode reward: [(0, '862.940'), (1, '699.870')] [2023-12-27 14:01:02,892][20347] Updated weights for policy 0, policy_version 1207733 (0.0010) [2023-12-27 14:01:02,945][20347] Updated weights for policy 0, policy_version 1207743 (0.0010) [2023-12-27 14:01:02,972][20381] Updated weights for policy 1, policy_version 1206720 (0.0006) [2023-12-27 14:01:03,002][20347] Updated weights for policy 0, policy_version 1207753 (0.0009) [2023-12-27 14:01:03,027][20381] Updated weights for policy 1, policy_version 1206730 (0.0005) [2023-12-27 14:01:03,083][20381] Updated weights for policy 1, policy_version 1206740 (0.0005) [2023-12-27 14:01:03,607][20381] Updated weights for policy 1, policy_version 1206750 (0.0007) [2023-12-27 14:01:03,663][20381] Updated weights for policy 1, policy_version 1206760 (0.0008) [2023-12-27 14:01:03,663][20347] Updated weights for policy 0, policy_version 1207763 (0.0007) [2023-12-27 14:01:03,713][20381] Updated weights for policy 1, policy_version 1206770 (0.0005) [2023-12-27 14:01:03,716][20347] Updated weights for policy 0, policy_version 1207773 (0.0007) [2023-12-27 14:01:03,765][20347] Updated weights for policy 0, policy_version 1207783 (0.0005) [2023-12-27 14:01:04,400][20381] Updated weights for policy 1, policy_version 1206780 (0.0007) [2023-12-27 14:01:04,457][20381] Updated weights for policy 1, policy_version 1206790 (0.0010) [2023-12-27 14:01:04,487][20347] Updated weights for policy 0, policy_version 1207793 (0.0006) [2023-12-27 14:01:04,509][20381] Updated weights for policy 1, policy_version 1206800 (0.0010) [2023-12-27 14:01:04,545][20347] Updated weights for policy 0, policy_version 1207803 (0.0007) [2023-12-27 14:01:04,602][20347] Updated weights for policy 0, policy_version 1207813 (0.0007) [2023-12-27 14:01:04,653][20347] Updated weights for policy 0, policy_version 1207823 (0.0007) [2023-12-27 14:01:05,192][20381] Updated weights for policy 1, policy_version 1206810 (0.0010) [2023-12-27 14:01:05,257][20381] Updated weights for policy 1, policy_version 1206820 (0.0008) [2023-12-27 14:01:05,323][20381] Updated weights for policy 1, policy_version 1206830 (0.0008) [2023-12-27 14:01:05,355][20347] Updated weights for policy 0, policy_version 1207833 (0.0006) [2023-12-27 14:01:05,386][20381] Updated weights for policy 1, policy_version 1206840 (0.0008) [2023-12-27 14:01:05,409][20347] Updated weights for policy 0, policy_version 1207843 (0.0006) [2023-12-27 14:01:05,466][20347] Updated weights for policy 0, policy_version 1207853 (0.0007) [2023-12-27 14:01:06,034][20381] Updated weights for policy 1, policy_version 1206850 (0.0010) [2023-12-27 14:01:06,093][20381] Updated weights for policy 1, policy_version 1206860 (0.0010) [2023-12-27 14:01:06,152][20381] Updated weights for policy 1, policy_version 1206870 (0.0009) [2023-12-27 14:01:06,163][20347] Updated weights for policy 0, policy_version 1207863 (0.0007) [2023-12-27 14:01:06,231][20347] Updated weights for policy 0, policy_version 1207873 (0.0007) [2023-12-27 14:01:06,296][20347] Updated weights for policy 0, policy_version 1207883 (0.0007) [2023-12-27 14:01:06,859][20381] Updated weights for policy 1, policy_version 1206880 (0.0010) [2023-12-27 14:01:06,917][20381] Updated weights for policy 1, policy_version 1206890 (0.0011) [2023-12-27 14:01:06,943][20347] Updated weights for policy 0, policy_version 1207893 (0.0010) [2023-12-27 14:01:06,968][20381] Updated weights for policy 1, policy_version 1206900 (0.0011) [2023-12-27 14:01:07,005][20347] Updated weights for policy 0, policy_version 1207903 (0.0006) [2023-12-27 14:01:07,034][20291] KL-divergence is very high: 120.8641 [2023-12-27 14:01:07,060][20291] KL-divergence is very high: 109.2953 [2023-12-27 14:01:07,069][20347] Updated weights for policy 0, policy_version 1207913 (0.0007) [2023-12-27 14:01:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 618274816. Throughput: 0: 10201.6, 1: 10155.4. Samples: 618243724. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:07,364][19170] Avg episode reward: [(0, '871.090'), (1, '689.060')] [2023-12-27 14:01:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001206904_308994048.pth... [2023-12-27 14:01:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001207920_309280768.pth... [2023-12-27 14:01:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001205720_308690944.pth [2023-12-27 14:01:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001206704_308969472.pth [2023-12-27 14:01:07,613][20381] Updated weights for policy 1, policy_version 1206910 (0.0008) [2023-12-27 14:01:07,623][20347] Updated weights for policy 0, policy_version 1207923 (0.0007) [2023-12-27 14:01:07,672][20381] Updated weights for policy 1, policy_version 1206920 (0.0006) [2023-12-27 14:01:07,684][20347] Updated weights for policy 0, policy_version 1207933 (0.0008) [2023-12-27 14:01:07,733][20381] Updated weights for policy 1, policy_version 1206930 (0.0005) [2023-12-27 14:01:07,739][20347] Updated weights for policy 0, policy_version 1207943 (0.0010) [2023-12-27 14:01:08,369][20347] Updated weights for policy 0, policy_version 1207953 (0.0010) [2023-12-27 14:01:08,430][20347] Updated weights for policy 0, policy_version 1207963 (0.0008) [2023-12-27 14:01:08,469][20381] Updated weights for policy 1, policy_version 1206940 (0.0006) [2023-12-27 14:01:08,487][20347] Updated weights for policy 0, policy_version 1207973 (0.0006) [2023-12-27 14:01:08,527][20381] Updated weights for policy 1, policy_version 1206950 (0.0007) [2023-12-27 14:01:08,543][20347] Updated weights for policy 0, policy_version 1207983 (0.0007) [2023-12-27 14:01:08,594][20381] Updated weights for policy 1, policy_version 1206960 (0.0007) [2023-12-27 14:01:09,214][20347] Updated weights for policy 0, policy_version 1207993 (0.0008) [2023-12-27 14:01:09,281][20347] Updated weights for policy 0, policy_version 1208003 (0.0008) [2023-12-27 14:01:09,318][20381] Updated weights for policy 1, policy_version 1206970 (0.0008) [2023-12-27 14:01:09,341][20347] Updated weights for policy 0, policy_version 1208013 (0.0008) [2023-12-27 14:01:09,385][20381] Updated weights for policy 1, policy_version 1206980 (0.0008) [2023-12-27 14:01:09,460][20381] Updated weights for policy 1, policy_version 1206990 (0.0008) [2023-12-27 14:01:09,520][20381] Updated weights for policy 1, policy_version 1207000 (0.0011) [2023-12-27 14:01:10,054][20347] Updated weights for policy 0, policy_version 1208023 (0.0008) [2023-12-27 14:01:10,117][20347] Updated weights for policy 0, policy_version 1208033 (0.0007) [2023-12-27 14:01:10,176][20347] Updated weights for policy 0, policy_version 1208043 (0.0007) [2023-12-27 14:01:10,224][20381] Updated weights for policy 1, policy_version 1207010 (0.0007) [2023-12-27 14:01:10,281][20381] Updated weights for policy 1, policy_version 1207020 (0.0008) [2023-12-27 14:01:10,341][20381] Updated weights for policy 1, policy_version 1207030 (0.0007) [2023-12-27 14:01:10,815][20347] Updated weights for policy 0, policy_version 1208053 (0.0006) [2023-12-27 14:01:10,873][20347] Updated weights for policy 0, policy_version 1208063 (0.0008) [2023-12-27 14:01:10,926][20347] Updated weights for policy 0, policy_version 1208073 (0.0007) [2023-12-27 14:01:11,055][20381] Updated weights for policy 1, policy_version 1207040 (0.0009) [2023-12-27 14:01:11,120][20381] Updated weights for policy 1, policy_version 1207050 (0.0011) [2023-12-27 14:01:11,193][20381] Updated weights for policy 1, policy_version 1207060 (0.0009) [2023-12-27 14:01:11,693][20347] Updated weights for policy 0, policy_version 1208083 (0.0008) [2023-12-27 14:01:11,761][20347] Updated weights for policy 0, policy_version 1208093 (0.0010) [2023-12-27 14:01:11,817][20347] Updated weights for policy 0, policy_version 1208103 (0.0008) [2023-12-27 14:01:11,903][20381] Updated weights for policy 1, policy_version 1207070 (0.0009) [2023-12-27 14:01:11,950][20381] Updated weights for policy 1, policy_version 1207080 (0.0008) [2023-12-27 14:01:12,004][20381] Updated weights for policy 1, policy_version 1207090 (0.0008) [2023-12-27 14:01:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 618373120. Throughput: 0: 10241.0, 1: 10112.2. Samples: 618364468. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:12,363][19170] Avg episode reward: [(0, '890.590'), (1, '747.010')] [2023-12-27 14:01:12,503][20347] Updated weights for policy 0, policy_version 1208113 (0.0008) [2023-12-27 14:01:12,576][20347] Updated weights for policy 0, policy_version 1208123 (0.0006) [2023-12-27 14:01:12,635][20347] Updated weights for policy 0, policy_version 1208133 (0.0008) [2023-12-27 14:01:12,695][20347] Updated weights for policy 0, policy_version 1208143 (0.0008) [2023-12-27 14:01:12,767][20381] Updated weights for policy 1, policy_version 1207100 (0.0009) [2023-12-27 14:01:12,824][20381] Updated weights for policy 1, policy_version 1207110 (0.0008) [2023-12-27 14:01:12,878][20381] Updated weights for policy 1, policy_version 1207120 (0.0007) [2023-12-27 14:01:13,313][20347] Updated weights for policy 0, policy_version 1208153 (0.0007) [2023-12-27 14:01:13,316][20291] KL-divergence is very high: 102.6351 [2023-12-27 14:01:13,354][20291] KL-divergence is very high: 112.4238 [2023-12-27 14:01:13,359][20347] Updated weights for policy 0, policy_version 1208163 (0.0006) [2023-12-27 14:01:13,410][20347] Updated weights for policy 0, policy_version 1208173 (0.0007) [2023-12-27 14:01:13,622][20381] Updated weights for policy 1, policy_version 1207130 (0.0007) [2023-12-27 14:01:13,668][20381] Updated weights for policy 1, policy_version 1207140 (0.0007) [2023-12-27 14:01:13,725][20381] Updated weights for policy 1, policy_version 1207150 (0.0007) [2023-12-27 14:01:13,782][20381] Updated weights for policy 1, policy_version 1207160 (0.0005) [2023-12-27 14:01:14,045][20347] Updated weights for policy 0, policy_version 1208183 (0.0005) [2023-12-27 14:01:14,103][20347] Updated weights for policy 0, policy_version 1208193 (0.0009) [2023-12-27 14:01:14,149][20347] Updated weights for policy 0, policy_version 1208203 (0.0010) [2023-12-27 14:01:14,455][20381] Updated weights for policy 1, policy_version 1207170 (0.0007) [2023-12-27 14:01:14,504][20381] Updated weights for policy 1, policy_version 1207180 (0.0010) [2023-12-27 14:01:14,562][20381] Updated weights for policy 1, policy_version 1207190 (0.0009) [2023-12-27 14:01:14,741][20347] Updated weights for policy 0, policy_version 1208213 (0.0009) [2023-12-27 14:01:14,808][20347] Updated weights for policy 0, policy_version 1208223 (0.0011) [2023-12-27 14:01:14,869][20347] Updated weights for policy 0, policy_version 1208233 (0.0011) [2023-12-27 14:01:15,368][20381] Updated weights for policy 1, policy_version 1207200 (0.0006) [2023-12-27 14:01:15,419][20381] Updated weights for policy 1, policy_version 1207210 (0.0007) [2023-12-27 14:01:15,474][20381] Updated weights for policy 1, policy_version 1207220 (0.0007) [2023-12-27 14:01:15,559][20347] Updated weights for policy 0, policy_version 1208243 (0.0010) [2023-12-27 14:01:15,613][20347] Updated weights for policy 0, policy_version 1208253 (0.0010) [2023-12-27 14:01:15,666][20347] Updated weights for policy 0, policy_version 1208263 (0.0010) [2023-12-27 14:01:16,137][20381] Updated weights for policy 1, policy_version 1207230 (0.0006) [2023-12-27 14:01:16,189][20381] Updated weights for policy 1, policy_version 1207240 (0.0005) [2023-12-27 14:01:16,251][20381] Updated weights for policy 1, policy_version 1207250 (0.0010) [2023-12-27 14:01:16,340][20347] Updated weights for policy 0, policy_version 1208273 (0.0010) [2023-12-27 14:01:16,400][20347] Updated weights for policy 0, policy_version 1208283 (0.0010) [2023-12-27 14:01:16,420][20291] KL-divergence is very high: 105.8269 [2023-12-27 14:01:16,442][20291] KL-divergence is very high: 169.8101 [2023-12-27 14:01:16,458][20347] Updated weights for policy 0, policy_version 1208293 (0.0010) [2023-12-27 14:01:16,504][20347] Updated weights for policy 0, policy_version 1208303 (0.0009) [2023-12-27 14:01:16,900][20381] Updated weights for policy 1, policy_version 1207260 (0.0009) [2023-12-27 14:01:16,950][20381] Updated weights for policy 1, policy_version 1207270 (0.0005) [2023-12-27 14:01:16,994][20381] Updated weights for policy 1, policy_version 1207280 (0.0005) [2023-12-27 14:01:17,153][20347] Updated weights for policy 0, policy_version 1208313 (0.0006) [2023-12-27 14:01:17,213][20347] Updated weights for policy 0, policy_version 1208323 (0.0006) [2023-12-27 14:01:17,267][20347] Updated weights for policy 0, policy_version 1208333 (0.0005) [2023-12-27 14:01:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 618479616. Throughput: 0: 10260.2, 1: 10063.1. Samples: 618486104. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:17,363][19170] Avg episode reward: [(0, '911.810'), (1, '766.790')] [2023-12-27 14:01:17,671][20381] Updated weights for policy 1, policy_version 1207290 (0.0006) [2023-12-27 14:01:17,723][20381] Updated weights for policy 1, policy_version 1207300 (0.0007) [2023-12-27 14:01:17,778][20381] Updated weights for policy 1, policy_version 1207310 (0.0005) [2023-12-27 14:01:17,837][20381] Updated weights for policy 1, policy_version 1207320 (0.0005) [2023-12-27 14:01:17,853][20347] Updated weights for policy 0, policy_version 1208343 (0.0007) [2023-12-27 14:01:17,912][20347] Updated weights for policy 0, policy_version 1208353 (0.0005) [2023-12-27 14:01:17,964][20347] Updated weights for policy 0, policy_version 1208363 (0.0005) [2023-12-27 14:01:18,529][20381] Updated weights for policy 1, policy_version 1207330 (0.0008) [2023-12-27 14:01:18,563][20347] Updated weights for policy 0, policy_version 1208373 (0.0006) [2023-12-27 14:01:18,586][20381] Updated weights for policy 1, policy_version 1207340 (0.0008) [2023-12-27 14:01:18,613][20347] Updated weights for policy 0, policy_version 1208383 (0.0007) [2023-12-27 14:01:18,642][20381] Updated weights for policy 1, policy_version 1207350 (0.0005) [2023-12-27 14:01:18,663][20347] Updated weights for policy 0, policy_version 1208393 (0.0006) [2023-12-27 14:01:19,319][20381] Updated weights for policy 1, policy_version 1207360 (0.0006) [2023-12-27 14:01:19,337][20347] Updated weights for policy 0, policy_version 1208403 (0.0007) [2023-12-27 14:01:19,381][20381] Updated weights for policy 1, policy_version 1207370 (0.0007) [2023-12-27 14:01:19,400][20347] Updated weights for policy 0, policy_version 1208413 (0.0009) [2023-12-27 14:01:19,433][20381] Updated weights for policy 1, policy_version 1207380 (0.0006) [2023-12-27 14:01:19,457][20347] Updated weights for policy 0, policy_version 1208423 (0.0007) [2023-12-27 14:01:20,099][20381] Updated weights for policy 1, policy_version 1207390 (0.0007) [2023-12-27 14:01:20,150][20381] Updated weights for policy 1, policy_version 1207400 (0.0008) [2023-12-27 14:01:20,174][20347] Updated weights for policy 0, policy_version 1208433 (0.0008) [2023-12-27 14:01:20,202][20381] Updated weights for policy 1, policy_version 1207410 (0.0007) [2023-12-27 14:01:20,227][20347] Updated weights for policy 0, policy_version 1208443 (0.0007) [2023-12-27 14:01:20,284][20347] Updated weights for policy 0, policy_version 1208453 (0.0007) [2023-12-27 14:01:20,343][20347] Updated weights for policy 0, policy_version 1208463 (0.0007) [2023-12-27 14:01:20,887][20381] Updated weights for policy 1, policy_version 1207420 (0.0008) [2023-12-27 14:01:20,947][20381] Updated weights for policy 1, policy_version 1207430 (0.0007) [2023-12-27 14:01:21,006][20381] Updated weights for policy 1, policy_version 1207440 (0.0011) [2023-12-27 14:01:21,074][20347] Updated weights for policy 0, policy_version 1208473 (0.0008) [2023-12-27 14:01:21,130][20347] Updated weights for policy 0, policy_version 1208483 (0.0007) [2023-12-27 14:01:21,195][20347] Updated weights for policy 0, policy_version 1208493 (0.0008) [2023-12-27 14:01:21,735][20381] Updated weights for policy 1, policy_version 1207450 (0.0010) [2023-12-27 14:01:21,802][20381] Updated weights for policy 1, policy_version 1207460 (0.0008) [2023-12-27 14:01:21,858][20381] Updated weights for policy 1, policy_version 1207470 (0.0007) [2023-12-27 14:01:21,917][20381] Updated weights for policy 1, policy_version 1207480 (0.0007) [2023-12-27 14:01:21,931][20347] Updated weights for policy 0, policy_version 1208503 (0.0008) [2023-12-27 14:01:21,991][20347] Updated weights for policy 0, policy_version 1208513 (0.0008) [2023-12-27 14:01:22,048][20347] Updated weights for policy 0, policy_version 1208523 (0.0008) [2023-12-27 14:01:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 618577920. Throughput: 0: 10318.3, 1: 10058.0. Samples: 618548512. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:22,363][19170] Avg episode reward: [(0, '951.000'), (1, '786.160')] [2023-12-27 14:01:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001207480_309141504.pth... [2023-12-27 14:01:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001208528_309436416.pth... [2023-12-27 14:01:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001206296_308838400.pth [2023-12-27 14:01:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001207312_309125120.pth [2023-12-27 14:01:22,623][20381] Updated weights for policy 1, policy_version 1207490 (0.0009) [2023-12-27 14:01:22,686][20381] Updated weights for policy 1, policy_version 1207500 (0.0006) [2023-12-27 14:01:22,736][20347] Updated weights for policy 0, policy_version 1208533 (0.0009) [2023-12-27 14:01:22,746][20381] Updated weights for policy 1, policy_version 1207510 (0.0009) [2023-12-27 14:01:22,796][20347] Updated weights for policy 0, policy_version 1208543 (0.0007) [2023-12-27 14:01:22,861][20347] Updated weights for policy 0, policy_version 1208553 (0.0008) [2023-12-27 14:01:23,395][20381] Updated weights for policy 1, policy_version 1207520 (0.0009) [2023-12-27 14:01:23,433][20347] Updated weights for policy 0, policy_version 1208563 (0.0005) [2023-12-27 14:01:23,452][20381] Updated weights for policy 1, policy_version 1207530 (0.0010) [2023-12-27 14:01:23,486][20347] Updated weights for policy 0, policy_version 1208573 (0.0010) [2023-12-27 14:01:23,511][20381] Updated weights for policy 1, policy_version 1207540 (0.0010) [2023-12-27 14:01:23,552][20347] Updated weights for policy 0, policy_version 1208583 (0.0010) [2023-12-27 14:01:24,056][20381] Updated weights for policy 1, policy_version 1207550 (0.0006) [2023-12-27 14:01:24,111][20381] Updated weights for policy 1, policy_version 1207560 (0.0006) [2023-12-27 14:01:24,145][20347] Updated weights for policy 0, policy_version 1208593 (0.0010) [2023-12-27 14:01:24,167][20381] Updated weights for policy 1, policy_version 1207570 (0.0006) [2023-12-27 14:01:24,200][20347] Updated weights for policy 0, policy_version 1208603 (0.0010) [2023-12-27 14:01:24,254][20347] Updated weights for policy 0, policy_version 1208613 (0.0010) [2023-12-27 14:01:24,312][20347] Updated weights for policy 0, policy_version 1208623 (0.0009) [2023-12-27 14:01:24,711][20381] Updated weights for policy 1, policy_version 1207580 (0.0006) [2023-12-27 14:01:24,772][20381] Updated weights for policy 1, policy_version 1207590 (0.0010) [2023-12-27 14:01:24,828][20381] Updated weights for policy 1, policy_version 1207600 (0.0010) [2023-12-27 14:01:24,885][20347] Updated weights for policy 0, policy_version 1208633 (0.0005) [2023-12-27 14:01:24,949][20347] Updated weights for policy 0, policy_version 1208643 (0.0005) [2023-12-27 14:01:25,016][20347] Updated weights for policy 0, policy_version 1208653 (0.0005) [2023-12-27 14:01:25,455][20381] Updated weights for policy 1, policy_version 1207610 (0.0010) [2023-12-27 14:01:25,505][20381] Updated weights for policy 1, policy_version 1207620 (0.0010) [2023-12-27 14:01:25,549][20381] Updated weights for policy 1, policy_version 1207630 (0.0010) [2023-12-27 14:01:25,608][20347] Updated weights for policy 0, policy_version 1208663 (0.0009) [2023-12-27 14:01:25,610][20381] Updated weights for policy 1, policy_version 1207640 (0.0006) [2023-12-27 14:01:25,656][20347] Updated weights for policy 0, policy_version 1208673 (0.0010) [2023-12-27 14:01:25,704][20347] Updated weights for policy 0, policy_version 1208683 (0.0010) [2023-12-27 14:01:26,199][20381] Updated weights for policy 1, policy_version 1207650 (0.0006) [2023-12-27 14:01:26,255][20381] Updated weights for policy 1, policy_version 1207660 (0.0006) [2023-12-27 14:01:26,312][20381] Updated weights for policy 1, policy_version 1207670 (0.0005) [2023-12-27 14:01:26,345][20347] Updated weights for policy 0, policy_version 1208693 (0.0010) [2023-12-27 14:01:26,398][20347] Updated weights for policy 0, policy_version 1208703 (0.0009) [2023-12-27 14:01:26,452][20347] Updated weights for policy 0, policy_version 1208713 (0.0010) [2023-12-27 14:01:26,867][20381] Updated weights for policy 1, policy_version 1207680 (0.0005) [2023-12-27 14:01:26,911][20381] Updated weights for policy 1, policy_version 1207690 (0.0005) [2023-12-27 14:01:26,964][20381] Updated weights for policy 1, policy_version 1207700 (0.0008) [2023-12-27 14:01:27,099][20347] Updated weights for policy 0, policy_version 1208723 (0.0009) [2023-12-27 14:01:27,170][20347] Updated weights for policy 0, policy_version 1208733 (0.0010) [2023-12-27 14:01:27,185][20291] KL-divergence is very high: 114.9757 [2023-12-27 14:01:27,224][20347] Updated weights for policy 0, policy_version 1208743 (0.0010) [2023-12-27 14:01:27,230][20291] KL-divergence is very high: 103.6519 [2023-12-27 14:01:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 618692608. Throughput: 0: 10408.8, 1: 10177.2. Samples: 618677488. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:27,363][19170] Avg episode reward: [(0, '960.890'), (1, '814.810')] [2023-12-27 14:01:27,608][20381] Updated weights for policy 1, policy_version 1207710 (0.0008) [2023-12-27 14:01:27,668][20381] Updated weights for policy 1, policy_version 1207720 (0.0007) [2023-12-27 14:01:27,731][20381] Updated weights for policy 1, policy_version 1207730 (0.0007) [2023-12-27 14:01:27,887][20347] Updated weights for policy 0, policy_version 1208753 (0.0010) [2023-12-27 14:01:27,935][20347] Updated weights for policy 0, policy_version 1208763 (0.0010) [2023-12-27 14:01:27,977][20347] Updated weights for policy 0, policy_version 1208773 (0.0006) [2023-12-27 14:01:28,027][20347] Updated weights for policy 0, policy_version 1208783 (0.0008) [2023-12-27 14:01:28,410][20381] Updated weights for policy 1, policy_version 1207740 (0.0007) [2023-12-27 14:01:28,462][20381] Updated weights for policy 1, policy_version 1207750 (0.0009) [2023-12-27 14:01:28,522][20381] Updated weights for policy 1, policy_version 1207760 (0.0008) [2023-12-27 14:01:28,663][20347] Updated weights for policy 0, policy_version 1208793 (0.0007) [2023-12-27 14:01:28,727][20347] Updated weights for policy 0, policy_version 1208803 (0.0006) [2023-12-27 14:01:28,787][20347] Updated weights for policy 0, policy_version 1208813 (0.0008) [2023-12-27 14:01:29,219][20381] Updated weights for policy 1, policy_version 1207770 (0.0008) [2023-12-27 14:01:29,283][20381] Updated weights for policy 1, policy_version 1207780 (0.0008) [2023-12-27 14:01:29,340][20381] Updated weights for policy 1, policy_version 1207790 (0.0007) [2023-12-27 14:01:29,403][20381] Updated weights for policy 1, policy_version 1207800 (0.0007) [2023-12-27 14:01:29,471][20347] Updated weights for policy 0, policy_version 1208823 (0.0010) [2023-12-27 14:01:29,520][20347] Updated weights for policy 0, policy_version 1208833 (0.0010) [2023-12-27 14:01:29,567][20347] Updated weights for policy 0, policy_version 1208843 (0.0010) [2023-12-27 14:01:30,089][20381] Updated weights for policy 1, policy_version 1207810 (0.0008) [2023-12-27 14:01:30,143][20381] Updated weights for policy 1, policy_version 1207820 (0.0008) [2023-12-27 14:01:30,198][20381] Updated weights for policy 1, policy_version 1207830 (0.0008) [2023-12-27 14:01:30,276][20347] Updated weights for policy 0, policy_version 1208853 (0.0009) [2023-12-27 14:01:30,334][20347] Updated weights for policy 0, policy_version 1208863 (0.0009) [2023-12-27 14:01:30,361][20291] KL-divergence is very high: 157.4483 [2023-12-27 14:01:30,392][20347] Updated weights for policy 0, policy_version 1208873 (0.0008) [2023-12-27 14:01:30,401][20291] KL-divergence is very high: 140.0750 [2023-12-27 14:01:30,855][20381] Updated weights for policy 1, policy_version 1207840 (0.0008) [2023-12-27 14:01:30,905][20381] Updated weights for policy 1, policy_version 1207850 (0.0007) [2023-12-27 14:01:30,961][20381] Updated weights for policy 1, policy_version 1207860 (0.0008) [2023-12-27 14:01:31,065][20347] Updated weights for policy 0, policy_version 1208883 (0.0008) [2023-12-27 14:01:31,128][20347] Updated weights for policy 0, policy_version 1208893 (0.0008) [2023-12-27 14:01:31,190][20347] Updated weights for policy 0, policy_version 1208903 (0.0008) [2023-12-27 14:01:31,639][20381] Updated weights for policy 1, policy_version 1207870 (0.0008) [2023-12-27 14:01:31,698][20381] Updated weights for policy 1, policy_version 1207880 (0.0007) [2023-12-27 14:01:31,765][20381] Updated weights for policy 1, policy_version 1207890 (0.0008) [2023-12-27 14:01:31,901][20347] Updated weights for policy 0, policy_version 1208913 (0.0009) [2023-12-27 14:01:31,954][20347] Updated weights for policy 0, policy_version 1208923 (0.0007) [2023-12-27 14:01:32,010][20347] Updated weights for policy 0, policy_version 1208933 (0.0008) [2023-12-27 14:01:32,066][20347] Updated weights for policy 0, policy_version 1208943 (0.0007) [2023-12-27 14:01:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 618790912. Throughput: 0: 10381.5, 1: 10188.7. Samples: 618802232. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:32,363][19170] Avg episode reward: [(0, '951.080'), (1, '872.690')] [2023-12-27 14:01:32,472][20381] Updated weights for policy 1, policy_version 1207900 (0.0007) [2023-12-27 14:01:32,528][20381] Updated weights for policy 1, policy_version 1207910 (0.0005) [2023-12-27 14:01:32,580][20381] Updated weights for policy 1, policy_version 1207920 (0.0005) [2023-12-27 14:01:32,741][20347] Updated weights for policy 0, policy_version 1208953 (0.0008) [2023-12-27 14:01:32,804][20347] Updated weights for policy 0, policy_version 1208963 (0.0008) [2023-12-27 14:01:32,858][20347] Updated weights for policy 0, policy_version 1208973 (0.0008) [2023-12-27 14:01:33,177][20381] Updated weights for policy 1, policy_version 1207930 (0.0006) [2023-12-27 14:01:33,243][20381] Updated weights for policy 1, policy_version 1207940 (0.0005) [2023-12-27 14:01:33,292][20381] Updated weights for policy 1, policy_version 1207950 (0.0007) [2023-12-27 14:01:33,347][20381] Updated weights for policy 1, policy_version 1207960 (0.0008) [2023-12-27 14:01:33,436][20347] Updated weights for policy 0, policy_version 1208983 (0.0006) [2023-12-27 14:01:33,488][20347] Updated weights for policy 0, policy_version 1208993 (0.0006) [2023-12-27 14:01:33,548][20347] Updated weights for policy 0, policy_version 1209003 (0.0010) [2023-12-27 14:01:34,025][20381] Updated weights for policy 1, policy_version 1207970 (0.0008) [2023-12-27 14:01:34,080][20381] Updated weights for policy 1, policy_version 1207980 (0.0007) [2023-12-27 14:01:34,114][20347] Updated weights for policy 0, policy_version 1209013 (0.0010) [2023-12-27 14:01:34,151][20381] Updated weights for policy 1, policy_version 1207990 (0.0006) [2023-12-27 14:01:34,175][20347] Updated weights for policy 0, policy_version 1209023 (0.0010) [2023-12-27 14:01:34,232][20347] Updated weights for policy 0, policy_version 1209033 (0.0010) [2023-12-27 14:01:34,820][20381] Updated weights for policy 1, policy_version 1208000 (0.0007) [2023-12-27 14:01:34,872][20381] Updated weights for policy 1, policy_version 1208010 (0.0005) [2023-12-27 14:01:34,891][20347] Updated weights for policy 0, policy_version 1209043 (0.0009) [2023-12-27 14:01:34,928][20381] Updated weights for policy 1, policy_version 1208020 (0.0005) [2023-12-27 14:01:34,943][20347] Updated weights for policy 0, policy_version 1209053 (0.0005) [2023-12-27 14:01:34,998][20347] Updated weights for policy 0, policy_version 1209063 (0.0005) [2023-12-27 14:01:35,454][20381] Updated weights for policy 1, policy_version 1208030 (0.0005) [2023-12-27 14:01:35,506][20381] Updated weights for policy 1, policy_version 1208040 (0.0006) [2023-12-27 14:01:35,551][20347] Updated weights for policy 0, policy_version 1209073 (0.0006) [2023-12-27 14:01:35,565][20381] Updated weights for policy 1, policy_version 1208050 (0.0007) [2023-12-27 14:01:35,596][20347] Updated weights for policy 0, policy_version 1209083 (0.0009) [2023-12-27 14:01:35,647][20347] Updated weights for policy 0, policy_version 1209093 (0.0005) [2023-12-27 14:01:35,699][20347] Updated weights for policy 0, policy_version 1209103 (0.0010) [2023-12-27 14:01:36,120][20381] Updated weights for policy 1, policy_version 1208060 (0.0009) [2023-12-27 14:01:36,177][20381] Updated weights for policy 1, policy_version 1208070 (0.0008) [2023-12-27 14:01:36,236][20381] Updated weights for policy 1, policy_version 1208080 (0.0010) [2023-12-27 14:01:36,335][20347] Updated weights for policy 0, policy_version 1209113 (0.0008) [2023-12-27 14:01:36,386][20347] Updated weights for policy 0, policy_version 1209123 (0.0007) [2023-12-27 14:01:36,454][20347] Updated weights for policy 0, policy_version 1209133 (0.0008) [2023-12-27 14:01:36,968][20381] Updated weights for policy 1, policy_version 1208090 (0.0011) [2023-12-27 14:01:37,028][20381] Updated weights for policy 1, policy_version 1208100 (0.0010) [2023-12-27 14:01:37,076][20381] Updated weights for policy 1, policy_version 1208110 (0.0010) [2023-12-27 14:01:37,128][20381] Updated weights for policy 1, policy_version 1208120 (0.0010) [2023-12-27 14:01:37,163][20347] Updated weights for policy 0, policy_version 1209143 (0.0007) [2023-12-27 14:01:37,228][20347] Updated weights for policy 0, policy_version 1209153 (0.0007) [2023-12-27 14:01:37,282][20347] Updated weights for policy 0, policy_version 1209163 (0.0007) [2023-12-27 14:01:37,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 618905600. Throughput: 0: 10432.4, 1: 10248.8. Samples: 618867588. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) [2023-12-27 14:01:37,364][19170] Avg episode reward: [(0, '921.750'), (1, '853.210')] [2023-12-27 14:01:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001208120_309305344.pth... [2023-12-27 14:01:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001209168_309600256.pth... [2023-12-27 14:01:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001206904_308994048.pth [2023-12-27 14:01:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001207920_309280768.pth [2023-12-27 14:01:37,826][20381] Updated weights for policy 1, policy_version 1208130 (0.0011) [2023-12-27 14:01:37,881][20381] Updated weights for policy 1, policy_version 1208140 (0.0011) [2023-12-27 14:01:37,934][20381] Updated weights for policy 1, policy_version 1208150 (0.0011) [2023-12-27 14:01:37,993][20347] Updated weights for policy 0, policy_version 1209173 (0.0007) [2023-12-27 14:01:38,045][20347] Updated weights for policy 0, policy_version 1209183 (0.0007) [2023-12-27 14:01:38,109][20347] Updated weights for policy 0, policy_version 1209193 (0.0007) [2023-12-27 14:01:38,644][20381] Updated weights for policy 1, policy_version 1208160 (0.0010) [2023-12-27 14:01:38,703][20381] Updated weights for policy 1, policy_version 1208170 (0.0011) [2023-12-27 14:01:38,759][20381] Updated weights for policy 1, policy_version 1208180 (0.0011) [2023-12-27 14:01:38,802][20347] Updated weights for policy 0, policy_version 1209203 (0.0008) [2023-12-27 14:01:38,858][20347] Updated weights for policy 0, policy_version 1209213 (0.0010) [2023-12-27 14:01:38,917][20347] Updated weights for policy 0, policy_version 1209223 (0.0010) [2023-12-27 14:01:39,495][20381] Updated weights for policy 1, policy_version 1208190 (0.0009) [2023-12-27 14:01:39,564][20381] Updated weights for policy 1, policy_version 1208200 (0.0008) [2023-12-27 14:01:39,627][20381] Updated weights for policy 1, policy_version 1208210 (0.0007) [2023-12-27 14:01:39,680][20347] Updated weights for policy 0, policy_version 1209233 (0.0010) [2023-12-27 14:01:39,745][20347] Updated weights for policy 0, policy_version 1209243 (0.0005) [2023-12-27 14:01:39,807][20347] Updated weights for policy 0, policy_version 1209253 (0.0006) [2023-12-27 14:01:39,884][20347] Updated weights for policy 0, policy_version 1209263 (0.0006) [2023-12-27 14:01:40,348][20381] Updated weights for policy 1, policy_version 1208220 (0.0007) [2023-12-27 14:01:40,413][20381] Updated weights for policy 1, policy_version 1208230 (0.0007) [2023-12-27 14:01:40,467][20381] Updated weights for policy 1, policy_version 1208240 (0.0005) [2023-12-27 14:01:40,515][20347] Updated weights for policy 0, policy_version 1209273 (0.0008) [2023-12-27 14:01:40,579][20347] Updated weights for policy 0, policy_version 1209283 (0.0010) [2023-12-27 14:01:40,637][20347] Updated weights for policy 0, policy_version 1209293 (0.0010) [2023-12-27 14:01:41,100][20381] Updated weights for policy 1, policy_version 1208251 (0.0007) [2023-12-27 14:01:41,167][20381] Updated weights for policy 1, policy_version 1208261 (0.0008) [2023-12-27 14:01:41,225][20381] Updated weights for policy 1, policy_version 1208271 (0.0009) [2023-12-27 14:01:41,359][20347] Updated weights for policy 0, policy_version 1209303 (0.0010) [2023-12-27 14:01:41,429][20347] Updated weights for policy 0, policy_version 1209313 (0.0010) [2023-12-27 14:01:41,484][20347] Updated weights for policy 0, policy_version 1209323 (0.0010) [2023-12-27 14:01:42,026][20381] Updated weights for policy 1, policy_version 1208281 (0.0008) [2023-12-27 14:01:42,087][20381] Updated weights for policy 1, policy_version 1208291 (0.0006) [2023-12-27 14:01:42,151][20381] Updated weights for policy 1, policy_version 1208301 (0.0006) [2023-12-27 14:01:42,163][20347] Updated weights for policy 0, policy_version 1209333 (0.0009) [2023-12-27 14:01:42,211][20381] Updated weights for policy 1, policy_version 1208311 (0.0006) [2023-12-27 14:01:42,230][20347] Updated weights for policy 0, policy_version 1209343 (0.0006) [2023-12-27 14:01:42,298][20347] Updated weights for policy 0, policy_version 1209353 (0.0007) [2023-12-27 14:01:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 619003904. Throughput: 0: 10409.5, 1: 10278.1. Samples: 618990192. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:01:42,363][19170] Avg episode reward: [(0, '912.010'), (1, '766.490')] [2023-12-27 14:01:42,839][20381] Updated weights for policy 1, policy_version 1208321 (0.0007) [2023-12-27 14:01:42,893][20381] Updated weights for policy 1, policy_version 1208331 (0.0007) [2023-12-27 14:01:42,939][20381] Updated weights for policy 1, policy_version 1208341 (0.0008) [2023-12-27 14:01:42,968][20347] Updated weights for policy 0, policy_version 1209363 (0.0006) [2023-12-27 14:01:43,025][20347] Updated weights for policy 0, policy_version 1209373 (0.0007) [2023-12-27 14:01:43,080][20347] Updated weights for policy 0, policy_version 1209383 (0.0007) [2023-12-27 14:01:43,583][20381] Updated weights for policy 1, policy_version 1208351 (0.0010) [2023-12-27 14:01:43,631][20381] Updated weights for policy 1, policy_version 1208361 (0.0010) [2023-12-27 14:01:43,686][20381] Updated weights for policy 1, policy_version 1208371 (0.0010) [2023-12-27 14:01:43,796][20347] Updated weights for policy 0, policy_version 1209393 (0.0007) [2023-12-27 14:01:43,851][20347] Updated weights for policy 0, policy_version 1209403 (0.0007) [2023-12-27 14:01:43,910][20347] Updated weights for policy 0, policy_version 1209413 (0.0008) [2023-12-27 14:01:43,964][20347] Updated weights for policy 0, policy_version 1209423 (0.0005) [2023-12-27 14:01:44,343][20381] Updated weights for policy 1, policy_version 1208381 (0.0011) [2023-12-27 14:01:44,396][20381] Updated weights for policy 1, policy_version 1208391 (0.0011) [2023-12-27 14:01:44,461][20381] Updated weights for policy 1, policy_version 1208401 (0.0011) [2023-12-27 14:01:44,605][20347] Updated weights for policy 0, policy_version 1209433 (0.0005) [2023-12-27 14:01:44,665][20347] Updated weights for policy 0, policy_version 1209443 (0.0007) [2023-12-27 14:01:44,728][20347] Updated weights for policy 0, policy_version 1209453 (0.0007) [2023-12-27 14:01:45,179][20381] Updated weights for policy 1, policy_version 1208411 (0.0010) [2023-12-27 14:01:45,239][20381] Updated weights for policy 1, policy_version 1208421 (0.0008) [2023-12-27 14:01:45,300][20381] Updated weights for policy 1, policy_version 1208431 (0.0009) [2023-12-27 14:01:45,411][20347] Updated weights for policy 0, policy_version 1209463 (0.0009) [2023-12-27 14:01:45,463][20347] Updated weights for policy 0, policy_version 1209473 (0.0010) [2023-12-27 14:01:45,517][20347] Updated weights for policy 0, policy_version 1209483 (0.0006) [2023-12-27 14:01:46,014][20381] Updated weights for policy 1, policy_version 1208441 (0.0007) [2023-12-27 14:01:46,063][20381] Updated weights for policy 1, policy_version 1208451 (0.0007) [2023-12-27 14:01:46,112][20381] Updated weights for policy 1, policy_version 1208461 (0.0007) [2023-12-27 14:01:46,136][20347] Updated weights for policy 0, policy_version 1209493 (0.0007) [2023-12-27 14:01:46,171][20381] Updated weights for policy 1, policy_version 1208471 (0.0008) [2023-12-27 14:01:46,187][20347] Updated weights for policy 0, policy_version 1209503 (0.0006) [2023-12-27 14:01:46,237][20347] Updated weights for policy 0, policy_version 1209513 (0.0005) [2023-12-27 14:01:46,812][20381] Updated weights for policy 1, policy_version 1208481 (0.0010) [2023-12-27 14:01:46,841][20347] Updated weights for policy 0, policy_version 1209523 (0.0005) [2023-12-27 14:01:46,871][20381] Updated weights for policy 1, policy_version 1208491 (0.0010) [2023-12-27 14:01:46,902][20347] Updated weights for policy 0, policy_version 1209533 (0.0005) [2023-12-27 14:01:46,933][20381] Updated weights for policy 1, policy_version 1208501 (0.0011) [2023-12-27 14:01:46,962][20347] Updated weights for policy 0, policy_version 1209543 (0.0005) [2023-12-27 14:01:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 619102208. Throughput: 0: 10444.4, 1: 10285.8. Samples: 619113640. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:01:47,363][19170] Avg episode reward: [(0, '931.560'), (1, '776.370')] [2023-12-27 14:01:47,555][20347] Updated weights for policy 0, policy_version 1209553 (0.0005) [2023-12-27 14:01:47,599][20381] Updated weights for policy 1, policy_version 1208511 (0.0007) [2023-12-27 14:01:47,611][20347] Updated weights for policy 0, policy_version 1209563 (0.0005) [2023-12-27 14:01:47,654][20381] Updated weights for policy 1, policy_version 1208521 (0.0005) [2023-12-27 14:01:47,675][20347] Updated weights for policy 0, policy_version 1209573 (0.0005) [2023-12-27 14:01:47,708][20381] Updated weights for policy 1, policy_version 1208531 (0.0005) [2023-12-27 14:01:47,732][20347] Updated weights for policy 0, policy_version 1209583 (0.0008) [2023-12-27 14:01:48,308][20347] Updated weights for policy 0, policy_version 1209593 (0.0007) [2023-12-27 14:01:48,347][20381] Updated weights for policy 1, policy_version 1208541 (0.0007) [2023-12-27 14:01:48,372][20347] Updated weights for policy 0, policy_version 1209603 (0.0008) [2023-12-27 14:01:48,408][20381] Updated weights for policy 1, policy_version 1208551 (0.0007) [2023-12-27 14:01:48,426][20347] Updated weights for policy 0, policy_version 1209613 (0.0006) [2023-12-27 14:01:48,466][20381] Updated weights for policy 1, policy_version 1208561 (0.0008) [2023-12-27 14:01:49,134][20347] Updated weights for policy 0, policy_version 1209623 (0.0008) [2023-12-27 14:01:49,184][20381] Updated weights for policy 1, policy_version 1208571 (0.0007) [2023-12-27 14:01:49,185][20347] Updated weights for policy 0, policy_version 1209633 (0.0007) [2023-12-27 14:01:49,244][20347] Updated weights for policy 0, policy_version 1209643 (0.0008) [2023-12-27 14:01:49,258][20381] Updated weights for policy 1, policy_version 1208581 (0.0008) [2023-12-27 14:01:49,343][20381] Updated weights for policy 1, policy_version 1208591 (0.0008) [2023-12-27 14:01:49,930][20347] Updated weights for policy 0, policy_version 1209653 (0.0009) [2023-12-27 14:01:49,994][20347] Updated weights for policy 0, policy_version 1209663 (0.0007) [2023-12-27 14:01:50,059][20347] Updated weights for policy 0, policy_version 1209673 (0.0008) [2023-12-27 14:01:50,091][20381] Updated weights for policy 1, policy_version 1208601 (0.0007) [2023-12-27 14:01:50,145][20381] Updated weights for policy 1, policy_version 1208611 (0.0008) [2023-12-27 14:01:50,201][20381] Updated weights for policy 1, policy_version 1208621 (0.0007) [2023-12-27 14:01:50,258][20381] Updated weights for policy 1, policy_version 1208631 (0.0007) [2023-12-27 14:01:50,734][20347] Updated weights for policy 0, policy_version 1209683 (0.0007) [2023-12-27 14:01:50,795][20347] Updated weights for policy 0, policy_version 1209693 (0.0006) [2023-12-27 14:01:50,858][20347] Updated weights for policy 0, policy_version 1209703 (0.0006) [2023-12-27 14:01:50,992][20381] Updated weights for policy 1, policy_version 1208641 (0.0010) [2023-12-27 14:01:51,050][20381] Updated weights for policy 1, policy_version 1208651 (0.0010) [2023-12-27 14:01:51,112][20381] Updated weights for policy 1, policy_version 1208661 (0.0007) [2023-12-27 14:01:51,538][20347] Updated weights for policy 0, policy_version 1209713 (0.0006) [2023-12-27 14:01:51,600][20347] Updated weights for policy 0, policy_version 1209723 (0.0007) [2023-12-27 14:01:51,671][20347] Updated weights for policy 0, policy_version 1209733 (0.0007) [2023-12-27 14:01:51,732][20347] Updated weights for policy 0, policy_version 1209743 (0.0005) [2023-12-27 14:01:51,884][20381] Updated weights for policy 1, policy_version 1208671 (0.0011) [2023-12-27 14:01:51,937][20381] Updated weights for policy 1, policy_version 1208681 (0.0011) [2023-12-27 14:01:51,993][20381] Updated weights for policy 1, policy_version 1208691 (0.0011) [2023-12-27 14:01:52,352][20347] Updated weights for policy 0, policy_version 1209753 (0.0007) [2023-12-27 14:01:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 619200512. Throughput: 0: 10461.4, 1: 10216.8. Samples: 619174244. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:01:52,363][19170] Avg episode reward: [(0, '980.400'), (1, '815.590')] [2023-12-27 14:01:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001208696_309452800.pth... [2023-12-27 14:01:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001207480_309141504.pth [2023-12-27 14:01:52,415][20347] Updated weights for policy 0, policy_version 1209763 (0.0008) [2023-12-27 14:01:52,473][20347] Updated weights for policy 0, policy_version 1209773 (0.0006) [2023-12-27 14:01:52,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001209776_309755904.pth... [2023-12-27 14:01:52,493][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001208528_309436416.pth [2023-12-27 14:01:52,692][20381] Updated weights for policy 1, policy_version 1208701 (0.0011) [2023-12-27 14:01:52,748][20381] Updated weights for policy 1, policy_version 1208711 (0.0011) [2023-12-27 14:01:52,807][20381] Updated weights for policy 1, policy_version 1208721 (0.0010) [2023-12-27 14:01:53,139][20347] Updated weights for policy 0, policy_version 1209783 (0.0008) [2023-12-27 14:01:53,189][20347] Updated weights for policy 0, policy_version 1209793 (0.0008) [2023-12-27 14:01:53,238][20347] Updated weights for policy 0, policy_version 1209803 (0.0005) [2023-12-27 14:01:53,435][20381] Updated weights for policy 1, policy_version 1208731 (0.0010) [2023-12-27 14:01:53,481][20381] Updated weights for policy 1, policy_version 1208741 (0.0010) [2023-12-27 14:01:53,529][20381] Updated weights for policy 1, policy_version 1208751 (0.0010) [2023-12-27 14:01:53,882][20347] Updated weights for policy 0, policy_version 1209813 (0.0005) [2023-12-27 14:01:53,937][20347] Updated weights for policy 0, policy_version 1209823 (0.0005) [2023-12-27 14:01:54,003][20347] Updated weights for policy 0, policy_version 1209833 (0.0006) [2023-12-27 14:01:54,242][20381] Updated weights for policy 1, policy_version 1208761 (0.0010) [2023-12-27 14:01:54,299][20381] Updated weights for policy 1, policy_version 1208771 (0.0006) [2023-12-27 14:01:54,357][20381] Updated weights for policy 1, policy_version 1208781 (0.0007) [2023-12-27 14:01:54,419][20381] Updated weights for policy 1, policy_version 1208791 (0.0005) [2023-12-27 14:01:54,636][20347] Updated weights for policy 0, policy_version 1209843 (0.0008) [2023-12-27 14:01:54,685][20347] Updated weights for policy 0, policy_version 1209853 (0.0007) [2023-12-27 14:01:54,736][20347] Updated weights for policy 0, policy_version 1209863 (0.0007) [2023-12-27 14:01:55,012][20381] Updated weights for policy 1, policy_version 1208801 (0.0006) [2023-12-27 14:01:55,073][20381] Updated weights for policy 1, policy_version 1208811 (0.0006) [2023-12-27 14:01:55,124][20381] Updated weights for policy 1, policy_version 1208821 (0.0007) [2023-12-27 14:01:55,463][20347] Updated weights for policy 0, policy_version 1209873 (0.0007) [2023-12-27 14:01:55,522][20347] Updated weights for policy 0, policy_version 1209883 (0.0008) [2023-12-27 14:01:55,581][20347] Updated weights for policy 0, policy_version 1209893 (0.0008) [2023-12-27 14:01:55,635][20347] Updated weights for policy 0, policy_version 1209903 (0.0007) [2023-12-27 14:01:55,783][20381] Updated weights for policy 1, policy_version 1208831 (0.0007) [2023-12-27 14:01:55,831][20381] Updated weights for policy 1, policy_version 1208841 (0.0005) [2023-12-27 14:01:55,881][20381] Updated weights for policy 1, policy_version 1208851 (0.0005) [2023-12-27 14:01:56,258][20347] Updated weights for policy 0, policy_version 1209913 (0.0007) [2023-12-27 14:01:56,309][20347] Updated weights for policy 0, policy_version 1209923 (0.0007) [2023-12-27 14:01:56,358][20347] Updated weights for policy 0, policy_version 1209933 (0.0008) [2023-12-27 14:01:56,554][20381] Updated weights for policy 1, policy_version 1208861 (0.0007) [2023-12-27 14:01:56,598][20381] Updated weights for policy 1, policy_version 1208871 (0.0007) [2023-12-27 14:01:56,641][20381] Updated weights for policy 1, policy_version 1208881 (0.0007) [2023-12-27 14:01:56,953][20347] Updated weights for policy 0, policy_version 1209943 (0.0006) [2023-12-27 14:01:57,010][20347] Updated weights for policy 0, policy_version 1209953 (0.0006) [2023-12-27 14:01:57,065][20347] Updated weights for policy 0, policy_version 1209963 (0.0006) [2023-12-27 14:01:57,243][20381] Updated weights for policy 1, policy_version 1208891 (0.0007) [2023-12-27 14:01:57,303][20381] Updated weights for policy 1, policy_version 1208901 (0.0005) [2023-12-27 14:01:57,357][20381] Updated weights for policy 1, policy_version 1208911 (0.0005) [2023-12-27 14:01:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 619307008. Throughput: 0: 10463.7, 1: 10273.2. Samples: 619297628. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:01:57,363][19170] Avg episode reward: [(0, '980.460'), (1, '804.900')] [2023-12-27 14:01:57,598][20347] Updated weights for policy 0, policy_version 1209973 (0.0006) [2023-12-27 14:01:57,651][20347] Updated weights for policy 0, policy_version 1209983 (0.0005) [2023-12-27 14:01:57,701][20347] Updated weights for policy 0, policy_version 1209993 (0.0008) [2023-12-27 14:01:57,977][20381] Updated weights for policy 1, policy_version 1208921 (0.0006) [2023-12-27 14:01:58,038][20381] Updated weights for policy 1, policy_version 1208931 (0.0006) [2023-12-27 14:01:58,098][20381] Updated weights for policy 1, policy_version 1208941 (0.0005) [2023-12-27 14:01:58,152][20381] Updated weights for policy 1, policy_version 1208951 (0.0007) [2023-12-27 14:01:58,394][20347] Updated weights for policy 0, policy_version 1210003 (0.0009) [2023-12-27 14:01:58,459][20347] Updated weights for policy 0, policy_version 1210013 (0.0008) [2023-12-27 14:01:58,522][20347] Updated weights for policy 0, policy_version 1210023 (0.0008) [2023-12-27 14:01:58,882][20381] Updated weights for policy 1, policy_version 1208961 (0.0009) [2023-12-27 14:01:58,944][20381] Updated weights for policy 1, policy_version 1208971 (0.0006) [2023-12-27 14:01:59,008][20381] Updated weights for policy 1, policy_version 1208981 (0.0006) [2023-12-27 14:01:59,223][20347] Updated weights for policy 0, policy_version 1210033 (0.0007) [2023-12-27 14:01:59,286][20347] Updated weights for policy 0, policy_version 1210043 (0.0008) [2023-12-27 14:01:59,346][20347] Updated weights for policy 0, policy_version 1210053 (0.0006) [2023-12-27 14:01:59,414][20347] Updated weights for policy 0, policy_version 1210063 (0.0007) [2023-12-27 14:01:59,724][20381] Updated weights for policy 1, policy_version 1208991 (0.0009) [2023-12-27 14:01:59,783][20381] Updated weights for policy 1, policy_version 1209001 (0.0011) [2023-12-27 14:01:59,845][20381] Updated weights for policy 1, policy_version 1209011 (0.0010) [2023-12-27 14:02:00,124][20347] Updated weights for policy 0, policy_version 1210073 (0.0007) [2023-12-27 14:02:00,172][20347] Updated weights for policy 0, policy_version 1210083 (0.0007) [2023-12-27 14:02:00,220][20347] Updated weights for policy 0, policy_version 1210093 (0.0007) [2023-12-27 14:02:00,528][20381] Updated weights for policy 1, policy_version 1209021 (0.0010) [2023-12-27 14:02:00,583][20381] Updated weights for policy 1, policy_version 1209031 (0.0010) [2023-12-27 14:02:00,637][20381] Updated weights for policy 1, policy_version 1209041 (0.0010) [2023-12-27 14:02:00,944][20347] Updated weights for policy 0, policy_version 1210103 (0.0008) [2023-12-27 14:02:00,998][20347] Updated weights for policy 0, policy_version 1210113 (0.0007) [2023-12-27 14:02:01,059][20347] Updated weights for policy 0, policy_version 1210123 (0.0007) [2023-12-27 14:02:01,300][20381] Updated weights for policy 1, policy_version 1209051 (0.0009) [2023-12-27 14:02:01,354][20381] Updated weights for policy 1, policy_version 1209061 (0.0008) [2023-12-27 14:02:01,423][20381] Updated weights for policy 1, policy_version 1209071 (0.0008) [2023-12-27 14:02:01,786][20347] Updated weights for policy 0, policy_version 1210133 (0.0007) [2023-12-27 14:02:01,840][20347] Updated weights for policy 0, policy_version 1210143 (0.0007) [2023-12-27 14:02:01,899][20347] Updated weights for policy 0, policy_version 1210153 (0.0005) [2023-12-27 14:02:02,119][20381] Updated weights for policy 1, policy_version 1209081 (0.0008) [2023-12-27 14:02:02,168][20381] Updated weights for policy 1, policy_version 1209091 (0.0007) [2023-12-27 14:02:02,220][20381] Updated weights for policy 1, policy_version 1209101 (0.0007) [2023-12-27 14:02:02,280][20381] Updated weights for policy 1, policy_version 1209111 (0.0007) [2023-12-27 14:02:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 619413504. Throughput: 0: 10472.9, 1: 10304.4. Samples: 619421080. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:02,363][19170] Avg episode reward: [(0, '970.660'), (1, '823.780')] [2023-12-27 14:02:02,474][20347] Updated weights for policy 0, policy_version 1210163 (0.0005) [2023-12-27 14:02:02,533][20347] Updated weights for policy 0, policy_version 1210173 (0.0005) [2023-12-27 14:02:02,592][20347] Updated weights for policy 0, policy_version 1210183 (0.0005) [2023-12-27 14:02:02,976][20381] Updated weights for policy 1, policy_version 1209121 (0.0007) [2023-12-27 14:02:03,032][20381] Updated weights for policy 1, policy_version 1209131 (0.0007) [2023-12-27 14:02:03,082][20381] Updated weights for policy 1, policy_version 1209141 (0.0009) [2023-12-27 14:02:03,148][20347] Updated weights for policy 0, policy_version 1210193 (0.0005) [2023-12-27 14:02:03,194][20347] Updated weights for policy 0, policy_version 1210203 (0.0006) [2023-12-27 14:02:03,251][20347] Updated weights for policy 0, policy_version 1210213 (0.0005) [2023-12-27 14:02:03,296][20347] Updated weights for policy 0, policy_version 1210223 (0.0005) [2023-12-27 14:02:03,673][20381] Updated weights for policy 1, policy_version 1209151 (0.0005) [2023-12-27 14:02:03,727][20381] Updated weights for policy 1, policy_version 1209161 (0.0005) [2023-12-27 14:02:03,771][20381] Updated weights for policy 1, policy_version 1209171 (0.0006) [2023-12-27 14:02:03,836][20347] Updated weights for policy 0, policy_version 1210233 (0.0006) [2023-12-27 14:02:03,893][20347] Updated weights for policy 0, policy_version 1210243 (0.0006) [2023-12-27 14:02:03,949][20347] Updated weights for policy 0, policy_version 1210253 (0.0007) [2023-12-27 14:02:04,459][20381] Updated weights for policy 1, policy_version 1209181 (0.0006) [2023-12-27 14:02:04,512][20381] Updated weights for policy 1, policy_version 1209191 (0.0008) [2023-12-27 14:02:04,559][20381] Updated weights for policy 1, policy_version 1209201 (0.0006) [2023-12-27 14:02:04,616][20347] Updated weights for policy 0, policy_version 1210263 (0.0009) [2023-12-27 14:02:04,664][20347] Updated weights for policy 0, policy_version 1210273 (0.0007) [2023-12-27 14:02:04,730][20347] Updated weights for policy 0, policy_version 1210283 (0.0005) [2023-12-27 14:02:05,114][20381] Updated weights for policy 1, policy_version 1209211 (0.0005) [2023-12-27 14:02:05,173][20381] Updated weights for policy 1, policy_version 1209221 (0.0008) [2023-12-27 14:02:05,230][20381] Updated weights for policy 1, policy_version 1209231 (0.0008) [2023-12-27 14:02:05,360][20347] Updated weights for policy 0, policy_version 1210293 (0.0006) [2023-12-27 14:02:05,404][20347] Updated weights for policy 0, policy_version 1210303 (0.0007) [2023-12-27 14:02:05,451][20347] Updated weights for policy 0, policy_version 1210313 (0.0007) [2023-12-27 14:02:05,845][20381] Updated weights for policy 1, policy_version 1209241 (0.0008) [2023-12-27 14:02:05,905][20381] Updated weights for policy 1, policy_version 1209251 (0.0006) [2023-12-27 14:02:05,963][20381] Updated weights for policy 1, policy_version 1209261 (0.0011) [2023-12-27 14:02:06,022][20381] Updated weights for policy 1, policy_version 1209271 (0.0007) [2023-12-27 14:02:06,072][20347] Updated weights for policy 0, policy_version 1210323 (0.0006) [2023-12-27 14:02:06,143][20347] Updated weights for policy 0, policy_version 1210333 (0.0007) [2023-12-27 14:02:06,203][20347] Updated weights for policy 0, policy_version 1210343 (0.0007) [2023-12-27 14:02:06,667][20381] Updated weights for policy 1, policy_version 1209281 (0.0008) [2023-12-27 14:02:06,736][20381] Updated weights for policy 1, policy_version 1209291 (0.0008) [2023-12-27 14:02:06,796][20381] Updated weights for policy 1, policy_version 1209301 (0.0008) [2023-12-27 14:02:06,824][20347] Updated weights for policy 0, policy_version 1210353 (0.0010) [2023-12-27 14:02:06,891][20347] Updated weights for policy 0, policy_version 1210363 (0.0006) [2023-12-27 14:02:06,955][20347] Updated weights for policy 0, policy_version 1210373 (0.0006) [2023-12-27 14:02:07,023][20347] Updated weights for policy 0, policy_version 1210383 (0.0006) [2023-12-27 14:02:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 619520000. Throughput: 0: 10526.9, 1: 10369.1. Samples: 619488828. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:07,363][19170] Avg episode reward: [(0, '970.740'), (1, '794.820')] [2023-12-27 14:02:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001209304_309608448.pth... [2023-12-27 14:02:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001210384_309911552.pth... [2023-12-27 14:02:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001208120_309305344.pth [2023-12-27 14:02:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001209168_309600256.pth [2023-12-27 14:02:07,500][20381] Updated weights for policy 1, policy_version 1209311 (0.0008) [2023-12-27 14:02:07,545][20381] Updated weights for policy 1, policy_version 1209321 (0.0008) [2023-12-27 14:02:07,573][20347] Updated weights for policy 0, policy_version 1210393 (0.0006) [2023-12-27 14:02:07,592][20381] Updated weights for policy 1, policy_version 1209331 (0.0007) [2023-12-27 14:02:07,624][20347] Updated weights for policy 0, policy_version 1210403 (0.0006) [2023-12-27 14:02:07,672][20347] Updated weights for policy 0, policy_version 1210413 (0.0005) [2023-12-27 14:02:08,229][20381] Updated weights for policy 1, policy_version 1209341 (0.0007) [2023-12-27 14:02:08,286][20381] Updated weights for policy 1, policy_version 1209351 (0.0008) [2023-12-27 14:02:08,341][20381] Updated weights for policy 1, policy_version 1209361 (0.0007) [2023-12-27 14:02:08,362][20347] Updated weights for policy 0, policy_version 1210423 (0.0007) [2023-12-27 14:02:08,417][20347] Updated weights for policy 0, policy_version 1210433 (0.0008) [2023-12-27 14:02:08,473][20347] Updated weights for policy 0, policy_version 1210443 (0.0008) [2023-12-27 14:02:08,938][20381] Updated weights for policy 1, policy_version 1209371 (0.0007) [2023-12-27 14:02:08,988][20381] Updated weights for policy 1, policy_version 1209381 (0.0008) [2023-12-27 14:02:09,042][20381] Updated weights for policy 1, policy_version 1209391 (0.0008) [2023-12-27 14:02:09,205][20347] Updated weights for policy 0, policy_version 1210453 (0.0008) [2023-12-27 14:02:09,269][20347] Updated weights for policy 0, policy_version 1210463 (0.0008) [2023-12-27 14:02:09,334][20347] Updated weights for policy 0, policy_version 1210473 (0.0006) [2023-12-27 14:02:09,769][20381] Updated weights for policy 1, policy_version 1209401 (0.0009) [2023-12-27 14:02:09,832][20381] Updated weights for policy 1, policy_version 1209411 (0.0008) [2023-12-27 14:02:09,902][20381] Updated weights for policy 1, policy_version 1209421 (0.0007) [2023-12-27 14:02:09,972][20381] Updated weights for policy 1, policy_version 1209431 (0.0007) [2023-12-27 14:02:10,115][20347] Updated weights for policy 0, policy_version 1210483 (0.0009) [2023-12-27 14:02:10,175][20347] Updated weights for policy 0, policy_version 1210493 (0.0008) [2023-12-27 14:02:10,238][20347] Updated weights for policy 0, policy_version 1210503 (0.0008) [2023-12-27 14:02:10,648][20381] Updated weights for policy 1, policy_version 1209441 (0.0008) [2023-12-27 14:02:10,706][20381] Updated weights for policy 1, policy_version 1209451 (0.0008) [2023-12-27 14:02:10,758][20381] Updated weights for policy 1, policy_version 1209461 (0.0008) [2023-12-27 14:02:10,945][20347] Updated weights for policy 0, policy_version 1210513 (0.0008) [2023-12-27 14:02:11,000][20347] Updated weights for policy 0, policy_version 1210523 (0.0005) [2023-12-27 14:02:11,068][20347] Updated weights for policy 0, policy_version 1210533 (0.0008) [2023-12-27 14:02:11,130][20347] Updated weights for policy 0, policy_version 1210543 (0.0008) [2023-12-27 14:02:11,530][20381] Updated weights for policy 1, policy_version 1209471 (0.0008) [2023-12-27 14:02:11,581][20381] Updated weights for policy 1, policy_version 1209481 (0.0010) [2023-12-27 14:02:11,653][20381] Updated weights for policy 1, policy_version 1209491 (0.0008) [2023-12-27 14:02:11,781][20347] Updated weights for policy 0, policy_version 1210553 (0.0007) [2023-12-27 14:02:11,821][20291] KL-divergence is very high: 132.6647 [2023-12-27 14:02:11,841][20347] Updated weights for policy 0, policy_version 1210563 (0.0007) [2023-12-27 14:02:11,861][20291] KL-divergence is very high: 104.7102 [2023-12-27 14:02:11,892][20347] Updated weights for policy 0, policy_version 1210573 (0.0008) [2023-12-27 14:02:12,363][19170] Fps is (10 sec: 20478.1, 60 sec: 20752.8, 300 sec: 20549.4). Total num frames: 619618304. Throughput: 0: 10468.8, 1: 10308.7. Samples: 619612492. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:12,364][19170] Avg episode reward: [(0, '941.510'), (1, '833.940')] [2023-12-27 14:02:12,451][20381] Updated weights for policy 1, policy_version 1209501 (0.0008) [2023-12-27 14:02:12,507][20381] Updated weights for policy 1, policy_version 1209511 (0.0009) [2023-12-27 14:02:12,517][20347] Updated weights for policy 0, policy_version 1210583 (0.0006) [2023-12-27 14:02:12,564][20381] Updated weights for policy 1, policy_version 1209521 (0.0008) [2023-12-27 14:02:12,574][20347] Updated weights for policy 0, policy_version 1210593 (0.0005) [2023-12-27 14:02:12,637][20347] Updated weights for policy 0, policy_version 1210603 (0.0006) [2023-12-27 14:02:13,210][20347] Updated weights for policy 0, policy_version 1210613 (0.0007) [2023-12-27 14:02:13,268][20347] Updated weights for policy 0, policy_version 1210623 (0.0007) [2023-12-27 14:02:13,318][20381] Updated weights for policy 1, policy_version 1209531 (0.0008) [2023-12-27 14:02:13,322][20347] Updated weights for policy 0, policy_version 1210633 (0.0005) [2023-12-27 14:02:13,374][20381] Updated weights for policy 1, policy_version 1209541 (0.0006) [2023-12-27 14:02:13,433][20381] Updated weights for policy 1, policy_version 1209551 (0.0007) [2023-12-27 14:02:13,957][20347] Updated weights for policy 0, policy_version 1210643 (0.0007) [2023-12-27 14:02:14,005][20347] Updated weights for policy 0, policy_version 1210653 (0.0007) [2023-12-27 14:02:14,054][20347] Updated weights for policy 0, policy_version 1210663 (0.0006) [2023-12-27 14:02:14,126][20381] Updated weights for policy 1, policy_version 1209561 (0.0009) [2023-12-27 14:02:14,187][20381] Updated weights for policy 1, policy_version 1209571 (0.0008) [2023-12-27 14:02:14,244][20381] Updated weights for policy 1, policy_version 1209581 (0.0007) [2023-12-27 14:02:14,300][20381] Updated weights for policy 1, policy_version 1209591 (0.0008) [2023-12-27 14:02:14,617][20347] Updated weights for policy 0, policy_version 1210673 (0.0005) [2023-12-27 14:02:14,667][20347] Updated weights for policy 0, policy_version 1210683 (0.0005) [2023-12-27 14:02:14,716][20347] Updated weights for policy 0, policy_version 1210693 (0.0005) [2023-12-27 14:02:14,779][20347] Updated weights for policy 0, policy_version 1210703 (0.0006) [2023-12-27 14:02:15,013][20381] Updated weights for policy 1, policy_version 1209601 (0.0008) [2023-12-27 14:02:15,063][20381] Updated weights for policy 1, policy_version 1209611 (0.0008) [2023-12-27 14:02:15,119][20381] Updated weights for policy 1, policy_version 1209621 (0.0008) [2023-12-27 14:02:15,451][20347] Updated weights for policy 0, policy_version 1210713 (0.0009) [2023-12-27 14:02:15,507][20347] Updated weights for policy 0, policy_version 1210723 (0.0009) [2023-12-27 14:02:15,566][20347] Updated weights for policy 0, policy_version 1210733 (0.0011) [2023-12-27 14:02:15,864][20381] Updated weights for policy 1, policy_version 1209631 (0.0007) [2023-12-27 14:02:15,912][20381] Updated weights for policy 1, policy_version 1209641 (0.0007) [2023-12-27 14:02:15,964][20381] Updated weights for policy 1, policy_version 1209651 (0.0007) [2023-12-27 14:02:16,219][20347] Updated weights for policy 0, policy_version 1210743 (0.0010) [2023-12-27 14:02:16,269][20347] Updated weights for policy 0, policy_version 1210753 (0.0009) [2023-12-27 14:02:16,325][20347] Updated weights for policy 0, policy_version 1210763 (0.0008) [2023-12-27 14:02:16,655][20381] Updated weights for policy 1, policy_version 1209661 (0.0006) [2023-12-27 14:02:16,707][20381] Updated weights for policy 1, policy_version 1209671 (0.0005) [2023-12-27 14:02:16,764][20381] Updated weights for policy 1, policy_version 1209681 (0.0008) [2023-12-27 14:02:16,934][20347] Updated weights for policy 0, policy_version 1210773 (0.0008) [2023-12-27 14:02:16,987][20347] Updated weights for policy 0, policy_version 1210783 (0.0010) [2023-12-27 14:02:17,040][20347] Updated weights for policy 0, policy_version 1210793 (0.0010) [2023-12-27 14:02:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 619724800. Throughput: 0: 10545.0, 1: 10201.1. Samples: 619735804. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:17,363][19170] Avg episode reward: [(0, '951.220'), (1, '862.720')] [2023-12-27 14:02:17,425][20381] Updated weights for policy 1, policy_version 1209691 (0.0011) [2023-12-27 14:02:17,473][20381] Updated weights for policy 1, policy_version 1209701 (0.0010) [2023-12-27 14:02:17,527][20381] Updated weights for policy 1, policy_version 1209711 (0.0010) [2023-12-27 14:02:17,713][20347] Updated weights for policy 0, policy_version 1210803 (0.0010) [2023-12-27 14:02:17,765][20347] Updated weights for policy 0, policy_version 1210813 (0.0010) [2023-12-27 14:02:17,820][20347] Updated weights for policy 0, policy_version 1210823 (0.0010) [2023-12-27 14:02:18,211][20381] Updated weights for policy 1, policy_version 1209721 (0.0011) [2023-12-27 14:02:18,259][20381] Updated weights for policy 1, policy_version 1209731 (0.0010) [2023-12-27 14:02:18,317][20381] Updated weights for policy 1, policy_version 1209741 (0.0011) [2023-12-27 14:02:18,381][20381] Updated weights for policy 1, policy_version 1209751 (0.0011) [2023-12-27 14:02:18,489][20347] Updated weights for policy 0, policy_version 1210833 (0.0010) [2023-12-27 14:02:18,550][20347] Updated weights for policy 0, policy_version 1210843 (0.0010) [2023-12-27 14:02:18,605][20347] Updated weights for policy 0, policy_version 1210853 (0.0010) [2023-12-27 14:02:18,667][20347] Updated weights for policy 0, policy_version 1210863 (0.0006) [2023-12-27 14:02:19,060][20381] Updated weights for policy 1, policy_version 1209761 (0.0011) [2023-12-27 14:02:19,108][20381] Updated weights for policy 1, policy_version 1209771 (0.0010) [2023-12-27 14:02:19,153][20381] Updated weights for policy 1, policy_version 1209781 (0.0010) [2023-12-27 14:02:19,256][20347] Updated weights for policy 0, policy_version 1210873 (0.0009) [2023-12-27 14:02:19,315][20347] Updated weights for policy 0, policy_version 1210883 (0.0008) [2023-12-27 14:02:19,379][20347] Updated weights for policy 0, policy_version 1210893 (0.0008) [2023-12-27 14:02:19,860][20381] Updated weights for policy 1, policy_version 1209791 (0.0009) [2023-12-27 14:02:19,932][20381] Updated weights for policy 1, policy_version 1209801 (0.0008) [2023-12-27 14:02:19,997][20381] Updated weights for policy 1, policy_version 1209811 (0.0006) [2023-12-27 14:02:20,079][20347] Updated weights for policy 0, policy_version 1210903 (0.0008) [2023-12-27 14:02:20,134][20347] Updated weights for policy 0, policy_version 1210913 (0.0006) [2023-12-27 14:02:20,186][20347] Updated weights for policy 0, policy_version 1210923 (0.0005) [2023-12-27 14:02:20,676][20381] Updated weights for policy 1, policy_version 1209821 (0.0008) [2023-12-27 14:02:20,745][20381] Updated weights for policy 1, policy_version 1209831 (0.0007) [2023-12-27 14:02:20,806][20381] Updated weights for policy 1, policy_version 1209841 (0.0006) [2023-12-27 14:02:20,836][20347] Updated weights for policy 0, policy_version 1210933 (0.0006) [2023-12-27 14:02:20,900][20347] Updated weights for policy 0, policy_version 1210943 (0.0006) [2023-12-27 14:02:20,962][20347] Updated weights for policy 0, policy_version 1210953 (0.0008) [2023-12-27 14:02:21,472][20381] Updated weights for policy 1, policy_version 1209851 (0.0008) [2023-12-27 14:02:21,532][20381] Updated weights for policy 1, policy_version 1209861 (0.0008) [2023-12-27 14:02:21,593][20381] Updated weights for policy 1, policy_version 1209871 (0.0008) [2023-12-27 14:02:21,736][20347] Updated weights for policy 0, policy_version 1210963 (0.0007) [2023-12-27 14:02:21,806][20347] Updated weights for policy 0, policy_version 1210973 (0.0008) [2023-12-27 14:02:21,863][20347] Updated weights for policy 0, policy_version 1210983 (0.0008) [2023-12-27 14:02:22,348][20381] Updated weights for policy 1, policy_version 1209881 (0.0009) [2023-12-27 14:02:22,362][19170] Fps is (10 sec: 20481.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 619823104. Throughput: 0: 10516.5, 1: 10146.9. Samples: 619797436. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:22,363][19170] Avg episode reward: [(0, '951.150'), (1, '870.010')] [2023-12-27 14:02:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001210992_310067200.pth... [2023-12-27 14:02:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001209776_309755904.pth [2023-12-27 14:02:22,421][20381] Updated weights for policy 1, policy_version 1209891 (0.0008) [2023-12-27 14:02:22,481][20381] Updated weights for policy 1, policy_version 1209901 (0.0008) [2023-12-27 14:02:22,543][20381] Updated weights for policy 1, policy_version 1209911 (0.0007) [2023-12-27 14:02:22,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001209912_309764096.pth... [2023-12-27 14:02:22,551][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001208696_309452800.pth [2023-12-27 14:02:22,588][20347] Updated weights for policy 0, policy_version 1210993 (0.0007) [2023-12-27 14:02:22,645][20347] Updated weights for policy 0, policy_version 1211003 (0.0009) [2023-12-27 14:02:22,705][20347] Updated weights for policy 0, policy_version 1211013 (0.0006) [2023-12-27 14:02:22,768][20347] Updated weights for policy 0, policy_version 1211023 (0.0006) [2023-12-27 14:02:23,252][20381] Updated weights for policy 1, policy_version 1209921 (0.0006) [2023-12-27 14:02:23,308][20381] Updated weights for policy 1, policy_version 1209931 (0.0005) [2023-12-27 14:02:23,355][20381] Updated weights for policy 1, policy_version 1209941 (0.0006) [2023-12-27 14:02:23,411][20347] Updated weights for policy 0, policy_version 1211033 (0.0010) [2023-12-27 14:02:23,464][20347] Updated weights for policy 0, policy_version 1211043 (0.0010) [2023-12-27 14:02:23,522][20347] Updated weights for policy 0, policy_version 1211053 (0.0010) [2023-12-27 14:02:23,989][20381] Updated weights for policy 1, policy_version 1209951 (0.0009) [2023-12-27 14:02:24,038][20381] Updated weights for policy 1, policy_version 1209961 (0.0009) [2023-12-27 14:02:24,094][20381] Updated weights for policy 1, policy_version 1209971 (0.0005) [2023-12-27 14:02:24,197][20347] Updated weights for policy 0, policy_version 1211063 (0.0010) [2023-12-27 14:02:24,259][20347] Updated weights for policy 0, policy_version 1211073 (0.0010) [2023-12-27 14:02:24,322][20347] Updated weights for policy 0, policy_version 1211083 (0.0007) [2023-12-27 14:02:24,752][20381] Updated weights for policy 1, policy_version 1209981 (0.0006) [2023-12-27 14:02:24,806][20381] Updated weights for policy 1, policy_version 1209991 (0.0007) [2023-12-27 14:02:24,860][20381] Updated weights for policy 1, policy_version 1210001 (0.0007) [2023-12-27 14:02:25,014][20347] Updated weights for policy 0, policy_version 1211093 (0.0009) [2023-12-27 14:02:25,061][20347] Updated weights for policy 0, policy_version 1211103 (0.0010) [2023-12-27 14:02:25,106][20347] Updated weights for policy 0, policy_version 1211113 (0.0010) [2023-12-27 14:02:25,555][20381] Updated weights for policy 1, policy_version 1210011 (0.0007) [2023-12-27 14:02:25,617][20381] Updated weights for policy 1, policy_version 1210021 (0.0007) [2023-12-27 14:02:25,670][20381] Updated weights for policy 1, policy_version 1210031 (0.0009) [2023-12-27 14:02:25,751][20347] Updated weights for policy 0, policy_version 1211123 (0.0009) [2023-12-27 14:02:25,800][20347] Updated weights for policy 0, policy_version 1211133 (0.0007) [2023-12-27 14:02:25,852][20347] Updated weights for policy 0, policy_version 1211143 (0.0005) [2023-12-27 14:02:26,348][20381] Updated weights for policy 1, policy_version 1210041 (0.0009) [2023-12-27 14:02:26,399][20381] Updated weights for policy 1, policy_version 1210051 (0.0008) [2023-12-27 14:02:26,454][20381] Updated weights for policy 1, policy_version 1210061 (0.0008) [2023-12-27 14:02:26,465][20347] Updated weights for policy 0, policy_version 1211153 (0.0009) [2023-12-27 14:02:26,503][20381] Updated weights for policy 1, policy_version 1210071 (0.0005) [2023-12-27 14:02:26,519][20347] Updated weights for policy 0, policy_version 1211163 (0.0010) [2023-12-27 14:02:26,575][20347] Updated weights for policy 0, policy_version 1211173 (0.0009) [2023-12-27 14:02:26,632][20347] Updated weights for policy 0, policy_version 1211183 (0.0010) [2023-12-27 14:02:27,222][20381] Updated weights for policy 1, policy_version 1210081 (0.0006) [2023-12-27 14:02:27,273][20381] Updated weights for policy 1, policy_version 1210091 (0.0006) [2023-12-27 14:02:27,283][20347] Updated weights for policy 0, policy_version 1211193 (0.0009) [2023-12-27 14:02:27,306][20291] KL-divergence is very high: 131.3849 [2023-12-27 14:02:27,327][20381] Updated weights for policy 1, policy_version 1210101 (0.0008) [2023-12-27 14:02:27,342][20347] Updated weights for policy 0, policy_version 1211203 (0.0007) [2023-12-27 14:02:27,351][20291] KL-divergence is very high: 125.3988 [2023-12-27 14:02:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 619929600. Throughput: 0: 10503.1, 1: 10129.7. Samples: 619918668. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:27,363][19170] Avg episode reward: [(0, '931.520'), (1, '887.690')] [2023-12-27 14:02:27,393][20347] Updated weights for policy 0, policy_version 1211213 (0.0010) [2023-12-27 14:02:27,880][20381] Updated weights for policy 1, policy_version 1210111 (0.0008) [2023-12-27 14:02:27,931][20381] Updated weights for policy 1, policy_version 1210121 (0.0010) [2023-12-27 14:02:27,975][20381] Updated weights for policy 1, policy_version 1210131 (0.0010) [2023-12-27 14:02:28,056][20347] Updated weights for policy 0, policy_version 1211223 (0.0010) [2023-12-27 14:02:28,107][20347] Updated weights for policy 0, policy_version 1211233 (0.0010) [2023-12-27 14:02:28,151][20347] Updated weights for policy 0, policy_version 1211243 (0.0010) [2023-12-27 14:02:28,622][20381] Updated weights for policy 1, policy_version 1210141 (0.0010) [2023-12-27 14:02:28,675][20381] Updated weights for policy 1, policy_version 1210151 (0.0010) [2023-12-27 14:02:28,735][20381] Updated weights for policy 1, policy_version 1210161 (0.0007) [2023-12-27 14:02:28,861][20347] Updated weights for policy 0, policy_version 1211253 (0.0010) [2023-12-27 14:02:28,917][20347] Updated weights for policy 0, policy_version 1211263 (0.0010) [2023-12-27 14:02:28,971][20347] Updated weights for policy 0, policy_version 1211273 (0.0010) [2023-12-27 14:02:29,425][20381] Updated weights for policy 1, policy_version 1210171 (0.0006) [2023-12-27 14:02:29,483][20381] Updated weights for policy 1, policy_version 1210181 (0.0006) [2023-12-27 14:02:29,544][20381] Updated weights for policy 1, policy_version 1210191 (0.0007) [2023-12-27 14:02:29,684][20347] Updated weights for policy 0, policy_version 1211283 (0.0010) [2023-12-27 14:02:29,745][20347] Updated weights for policy 0, policy_version 1211293 (0.0010) [2023-12-27 14:02:29,806][20347] Updated weights for policy 0, policy_version 1211303 (0.0008) [2023-12-27 14:02:30,131][20381] Updated weights for policy 1, policy_version 1210201 (0.0008) [2023-12-27 14:02:30,177][20292] KL-divergence is very high: 136.4422 [2023-12-27 14:02:30,180][20381] Updated weights for policy 1, policy_version 1210211 (0.0010) [2023-12-27 14:02:30,222][20292] KL-divergence is very high: 157.5592 [2023-12-27 14:02:30,240][20381] Updated weights for policy 1, policy_version 1210221 (0.0006) [2023-12-27 14:02:30,293][20381] Updated weights for policy 1, policy_version 1210231 (0.0011) [2023-12-27 14:02:30,477][20347] Updated weights for policy 0, policy_version 1211313 (0.0010) [2023-12-27 14:02:30,525][20347] Updated weights for policy 0, policy_version 1211323 (0.0010) [2023-12-27 14:02:30,573][20347] Updated weights for policy 0, policy_version 1211333 (0.0010) [2023-12-27 14:02:30,623][20347] Updated weights for policy 0, policy_version 1211343 (0.0010) [2023-12-27 14:02:30,968][20381] Updated weights for policy 1, policy_version 1210241 (0.0011) [2023-12-27 14:02:31,034][20381] Updated weights for policy 1, policy_version 1210251 (0.0010) [2023-12-27 14:02:31,094][20381] Updated weights for policy 1, policy_version 1210261 (0.0011) [2023-12-27 14:02:31,322][20347] Updated weights for policy 0, policy_version 1211353 (0.0007) [2023-12-27 14:02:31,385][20347] Updated weights for policy 0, policy_version 1211363 (0.0008) [2023-12-27 14:02:31,439][20347] Updated weights for policy 0, policy_version 1211373 (0.0007) [2023-12-27 14:02:31,852][20381] Updated weights for policy 1, policy_version 1210271 (0.0010) [2023-12-27 14:02:31,920][20381] Updated weights for policy 1, policy_version 1210281 (0.0011) [2023-12-27 14:02:31,964][20381] Updated weights for policy 1, policy_version 1210291 (0.0008) [2023-12-27 14:02:32,065][20347] Updated weights for policy 0, policy_version 1211383 (0.0007) [2023-12-27 14:02:32,119][20347] Updated weights for policy 0, policy_version 1211393 (0.0007) [2023-12-27 14:02:32,127][20291] KL-divergence is very high: 157.0461 [2023-12-27 14:02:32,131][20291] KL-divergence is very high: 140.2641 [2023-12-27 14:02:32,147][20291] KL-divergence is very high: 114.0310 [2023-12-27 14:02:32,171][20347] Updated weights for policy 0, policy_version 1211403 (0.0007) [2023-12-27 14:02:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 620036096. Throughput: 0: 10500.1, 1: 10164.1. Samples: 620043528. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:32,363][19170] Avg episode reward: [(0, '892.270'), (1, '805.150')] [2023-12-27 14:02:32,711][20381] Updated weights for policy 1, policy_version 1210301 (0.0011) [2023-12-27 14:02:32,769][20381] Updated weights for policy 1, policy_version 1210311 (0.0010) [2023-12-27 14:02:32,820][20381] Updated weights for policy 1, policy_version 1210321 (0.0010) [2023-12-27 14:02:32,901][20347] Updated weights for policy 0, policy_version 1211413 (0.0007) [2023-12-27 14:02:32,951][20347] Updated weights for policy 0, policy_version 1211423 (0.0007) [2023-12-27 14:02:33,000][20347] Updated weights for policy 0, policy_version 1211433 (0.0008) [2023-12-27 14:02:33,398][20381] Updated weights for policy 1, policy_version 1210331 (0.0009) [2023-12-27 14:02:33,452][20381] Updated weights for policy 1, policy_version 1210341 (0.0005) [2023-12-27 14:02:33,512][20381] Updated weights for policy 1, policy_version 1210351 (0.0006) [2023-12-27 14:02:33,640][20347] Updated weights for policy 0, policy_version 1211443 (0.0008) [2023-12-27 14:02:33,685][20347] Updated weights for policy 0, policy_version 1211453 (0.0006) [2023-12-27 14:02:33,738][20347] Updated weights for policy 0, policy_version 1211463 (0.0007) [2023-12-27 14:02:34,099][20381] Updated weights for policy 1, policy_version 1210361 (0.0007) [2023-12-27 14:02:34,160][20381] Updated weights for policy 1, policy_version 1210371 (0.0007) [2023-12-27 14:02:34,225][20381] Updated weights for policy 1, policy_version 1210381 (0.0006) [2023-12-27 14:02:34,282][20381] Updated weights for policy 1, policy_version 1210391 (0.0005) [2023-12-27 14:02:34,386][20347] Updated weights for policy 0, policy_version 1211473 (0.0005) [2023-12-27 14:02:34,452][20347] Updated weights for policy 0, policy_version 1211483 (0.0006) [2023-12-27 14:02:34,517][20347] Updated weights for policy 0, policy_version 1211493 (0.0008) [2023-12-27 14:02:34,584][20347] Updated weights for policy 0, policy_version 1211503 (0.0008) [2023-12-27 14:02:34,924][20381] Updated weights for policy 1, policy_version 1210401 (0.0007) [2023-12-27 14:02:34,978][20381] Updated weights for policy 1, policy_version 1210411 (0.0008) [2023-12-27 14:02:35,030][20381] Updated weights for policy 1, policy_version 1210421 (0.0009) [2023-12-27 14:02:35,167][20347] Updated weights for policy 0, policy_version 1211513 (0.0006) [2023-12-27 14:02:35,230][20347] Updated weights for policy 0, policy_version 1211523 (0.0005) [2023-12-27 14:02:35,285][20347] Updated weights for policy 0, policy_version 1211533 (0.0008) [2023-12-27 14:02:35,646][20381] Updated weights for policy 1, policy_version 1210431 (0.0006) [2023-12-27 14:02:35,705][20381] Updated weights for policy 1, policy_version 1210441 (0.0005) [2023-12-27 14:02:35,759][20381] Updated weights for policy 1, policy_version 1210451 (0.0007) [2023-12-27 14:02:35,878][20347] Updated weights for policy 0, policy_version 1211543 (0.0009) [2023-12-27 14:02:35,923][20347] Updated weights for policy 0, policy_version 1211553 (0.0010) [2023-12-27 14:02:35,975][20347] Updated weights for policy 0, policy_version 1211563 (0.0010) [2023-12-27 14:02:36,440][20381] Updated weights for policy 1, policy_version 1210461 (0.0007) [2023-12-27 14:02:36,499][20381] Updated weights for policy 1, policy_version 1210471 (0.0008) [2023-12-27 14:02:36,564][20381] Updated weights for policy 1, policy_version 1210481 (0.0007) [2023-12-27 14:02:36,676][20347] Updated weights for policy 0, policy_version 1211573 (0.0010) [2023-12-27 14:02:36,722][20347] Updated weights for policy 0, policy_version 1211583 (0.0010) [2023-12-27 14:02:36,777][20347] Updated weights for policy 0, policy_version 1211593 (0.0010) [2023-12-27 14:02:37,188][20381] Updated weights for policy 1, policy_version 1210491 (0.0007) [2023-12-27 14:02:37,243][20381] Updated weights for policy 1, policy_version 1210501 (0.0007) [2023-12-27 14:02:37,299][20381] Updated weights for policy 1, policy_version 1210511 (0.0007) [2023-12-27 14:02:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 620142592. Throughput: 0: 10513.6, 1: 10240.5. Samples: 620108180. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:37,363][19170] Avg episode reward: [(0, '882.790'), (1, '775.410')] [2023-12-27 14:02:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001211600_310222848.pth... [2023-12-27 14:02:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001210520_309919744.pth... [2023-12-27 14:02:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001210384_309911552.pth [2023-12-27 14:02:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001209304_309608448.pth [2023-12-27 14:02:37,500][20347] Updated weights for policy 0, policy_version 1211603 (0.0010) [2023-12-27 14:02:37,554][20291] KL-divergence is very high: 117.5374 [2023-12-27 14:02:37,560][20347] Updated weights for policy 0, policy_version 1211613 (0.0008) [2023-12-27 14:02:37,604][20291] KL-divergence is very high: 111.2850 [2023-12-27 14:02:37,621][20347] Updated weights for policy 0, policy_version 1211623 (0.0008) [2023-12-27 14:02:37,991][20381] Updated weights for policy 1, policy_version 1210521 (0.0007) [2023-12-27 14:02:38,051][20381] Updated weights for policy 1, policy_version 1210531 (0.0007) [2023-12-27 14:02:38,096][20381] Updated weights for policy 1, policy_version 1210541 (0.0008) [2023-12-27 14:02:38,151][20381] Updated weights for policy 1, policy_version 1210551 (0.0007) [2023-12-27 14:02:38,265][20347] Updated weights for policy 0, policy_version 1211633 (0.0006) [2023-12-27 14:02:38,318][20347] Updated weights for policy 0, policy_version 1211643 (0.0007) [2023-12-27 14:02:38,385][20347] Updated weights for policy 0, policy_version 1211653 (0.0007) [2023-12-27 14:02:38,441][20347] Updated weights for policy 0, policy_version 1211663 (0.0007) [2023-12-27 14:02:38,895][20381] Updated weights for policy 1, policy_version 1210561 (0.0007) [2023-12-27 14:02:38,964][20381] Updated weights for policy 1, policy_version 1210571 (0.0008) [2023-12-27 14:02:39,023][20381] Updated weights for policy 1, policy_version 1210581 (0.0007) [2023-12-27 14:02:39,082][20347] Updated weights for policy 0, policy_version 1211673 (0.0007) [2023-12-27 14:02:39,131][20347] Updated weights for policy 0, policy_version 1211683 (0.0008) [2023-12-27 14:02:39,176][20347] Updated weights for policy 0, policy_version 1211693 (0.0007) [2023-12-27 14:02:39,751][20381] Updated weights for policy 1, policy_version 1210591 (0.0006) [2023-12-27 14:02:39,806][20381] Updated weights for policy 1, policy_version 1210601 (0.0006) [2023-12-27 14:02:39,872][20381] Updated weights for policy 1, policy_version 1210611 (0.0008) [2023-12-27 14:02:39,908][20347] Updated weights for policy 0, policy_version 1211703 (0.0008) [2023-12-27 14:02:39,976][20347] Updated weights for policy 0, policy_version 1211713 (0.0006) [2023-12-27 14:02:40,040][20347] Updated weights for policy 0, policy_version 1211723 (0.0008) [2023-12-27 14:02:40,538][20381] Updated weights for policy 1, policy_version 1210621 (0.0007) [2023-12-27 14:02:40,598][20381] Updated weights for policy 1, policy_version 1210631 (0.0008) [2023-12-27 14:02:40,659][20381] Updated weights for policy 1, policy_version 1210641 (0.0005) [2023-12-27 14:02:40,726][20291] KL-divergence is very high: 190.0778 [2023-12-27 14:02:40,748][20347] Updated weights for policy 0, policy_version 1211733 (0.0008) [2023-12-27 14:02:40,773][20291] KL-divergence is very high: 137.9226 [2023-12-27 14:02:40,806][20347] Updated weights for policy 0, policy_version 1211743 (0.0007) [2023-12-27 14:02:40,867][20347] Updated weights for policy 0, policy_version 1211753 (0.0008) [2023-12-27 14:02:41,333][20381] Updated weights for policy 1, policy_version 1210651 (0.0008) [2023-12-27 14:02:41,399][20381] Updated weights for policy 1, policy_version 1210661 (0.0008) [2023-12-27 14:02:41,459][20381] Updated weights for policy 1, policy_version 1210671 (0.0006) [2023-12-27 14:02:41,573][20347] Updated weights for policy 0, policy_version 1211763 (0.0006) [2023-12-27 14:02:41,644][20347] Updated weights for policy 0, policy_version 1211773 (0.0009) [2023-12-27 14:02:41,704][20347] Updated weights for policy 0, policy_version 1211783 (0.0009) [2023-12-27 14:02:42,086][20381] Updated weights for policy 1, policy_version 1210681 (0.0006) [2023-12-27 14:02:42,152][20381] Updated weights for policy 1, policy_version 1210691 (0.0008) [2023-12-27 14:02:42,217][20381] Updated weights for policy 1, policy_version 1210701 (0.0008) [2023-12-27 14:02:42,297][20381] Updated weights for policy 1, policy_version 1210711 (0.0008) [2023-12-27 14:02:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 620240896. Throughput: 0: 10493.4, 1: 10245.4. Samples: 620230876. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:42,363][19170] Avg episode reward: [(0, '892.820'), (1, '716.710')] [2023-12-27 14:02:42,453][20347] Updated weights for policy 0, policy_version 1211793 (0.0009) [2023-12-27 14:02:42,519][20347] Updated weights for policy 0, policy_version 1211803 (0.0009) [2023-12-27 14:02:42,600][20347] Updated weights for policy 0, policy_version 1211813 (0.0009) [2023-12-27 14:02:42,654][20347] Updated weights for policy 0, policy_version 1211823 (0.0011) [2023-12-27 14:02:42,972][20381] Updated weights for policy 1, policy_version 1210721 (0.0007) [2023-12-27 14:02:43,028][20381] Updated weights for policy 1, policy_version 1210731 (0.0008) [2023-12-27 14:02:43,074][20381] Updated weights for policy 1, policy_version 1210741 (0.0006) [2023-12-27 14:02:43,304][20347] Updated weights for policy 0, policy_version 1211833 (0.0010) [2023-12-27 14:02:43,356][20347] Updated weights for policy 0, policy_version 1211844 (0.0009) [2023-12-27 14:02:43,413][20347] Updated weights for policy 0, policy_version 1211854 (0.0008) [2023-12-27 14:02:43,696][20381] Updated weights for policy 1, policy_version 1210751 (0.0006) [2023-12-27 14:02:43,755][20381] Updated weights for policy 1, policy_version 1210761 (0.0007) [2023-12-27 14:02:43,815][20381] Updated weights for policy 1, policy_version 1210771 (0.0008) [2023-12-27 14:02:44,138][20347] Updated weights for policy 0, policy_version 1211864 (0.0010) [2023-12-27 14:02:44,199][20347] Updated weights for policy 0, policy_version 1211874 (0.0010) [2023-12-27 14:02:44,263][20347] Updated weights for policy 0, policy_version 1211884 (0.0010) [2023-12-27 14:02:44,446][20381] Updated weights for policy 1, policy_version 1210781 (0.0008) [2023-12-27 14:02:44,491][20381] Updated weights for policy 1, policy_version 1210791 (0.0008) [2023-12-27 14:02:44,541][20381] Updated weights for policy 1, policy_version 1210801 (0.0007) [2023-12-27 14:02:44,823][20347] Updated weights for policy 0, policy_version 1211894 (0.0008) [2023-12-27 14:02:44,883][20347] Updated weights for policy 0, policy_version 1211904 (0.0008) [2023-12-27 14:02:44,942][20347] Updated weights for policy 0, policy_version 1211914 (0.0007) [2023-12-27 14:02:45,271][20381] Updated weights for policy 1, policy_version 1210811 (0.0007) [2023-12-27 14:02:45,332][20381] Updated weights for policy 1, policy_version 1210821 (0.0008) [2023-12-27 14:02:45,380][20381] Updated weights for policy 1, policy_version 1210831 (0.0008) [2023-12-27 14:02:45,582][20347] Updated weights for policy 0, policy_version 1211924 (0.0007) [2023-12-27 14:02:45,634][20347] Updated weights for policy 0, policy_version 1211934 (0.0005) [2023-12-27 14:02:45,687][20347] Updated weights for policy 0, policy_version 1211944 (0.0006) [2023-12-27 14:02:46,105][20381] Updated weights for policy 1, policy_version 1210841 (0.0007) [2023-12-27 14:02:46,163][20381] Updated weights for policy 1, policy_version 1210851 (0.0007) [2023-12-27 14:02:46,222][20381] Updated weights for policy 1, policy_version 1210861 (0.0008) [2023-12-27 14:02:46,281][20381] Updated weights for policy 1, policy_version 1210871 (0.0007) [2023-12-27 14:02:46,293][20347] Updated weights for policy 0, policy_version 1211954 (0.0008) [2023-12-27 14:02:46,349][20347] Updated weights for policy 0, policy_version 1211964 (0.0007) [2023-12-27 14:02:46,398][20347] Updated weights for policy 0, policy_version 1211974 (0.0010) [2023-12-27 14:02:46,447][20347] Updated weights for policy 0, policy_version 1211984 (0.0010) [2023-12-27 14:02:47,022][20381] Updated weights for policy 1, policy_version 1210881 (0.0007) [2023-12-27 14:02:47,070][20381] Updated weights for policy 1, policy_version 1210891 (0.0006) [2023-12-27 14:02:47,090][20347] Updated weights for policy 0, policy_version 1211994 (0.0010) [2023-12-27 14:02:47,119][20381] Updated weights for policy 1, policy_version 1210901 (0.0009) [2023-12-27 14:02:47,142][20347] Updated weights for policy 0, policy_version 1212004 (0.0010) [2023-12-27 14:02:47,199][20347] Updated weights for policy 0, policy_version 1212014 (0.0010) [2023-12-27 14:02:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 620347392. Throughput: 0: 10481.0, 1: 10230.9. Samples: 620353116. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:47,363][19170] Avg episode reward: [(0, '902.300'), (1, '677.710')] [2023-12-27 14:02:47,795][20347] Updated weights for policy 0, policy_version 1212024 (0.0010) [2023-12-27 14:02:47,854][20347] Updated weights for policy 0, policy_version 1212034 (0.0010) [2023-12-27 14:02:47,860][20381] Updated weights for policy 1, policy_version 1210911 (0.0006) [2023-12-27 14:02:47,909][20347] Updated weights for policy 0, policy_version 1212044 (0.0010) [2023-12-27 14:02:47,918][20381] Updated weights for policy 1, policy_version 1210921 (0.0005) [2023-12-27 14:02:47,978][20381] Updated weights for policy 1, policy_version 1210931 (0.0007) [2023-12-27 14:02:48,601][20347] Updated weights for policy 0, policy_version 1212054 (0.0010) [2023-12-27 14:02:48,649][20347] Updated weights for policy 0, policy_version 1212064 (0.0007) [2023-12-27 14:02:48,674][20381] Updated weights for policy 1, policy_version 1210941 (0.0007) [2023-12-27 14:02:48,711][20347] Updated weights for policy 0, policy_version 1212074 (0.0006) [2023-12-27 14:02:48,728][20381] Updated weights for policy 1, policy_version 1210951 (0.0007) [2023-12-27 14:02:48,790][20381] Updated weights for policy 1, policy_version 1210961 (0.0007) [2023-12-27 14:02:49,341][20347] Updated weights for policy 0, policy_version 1212084 (0.0006) [2023-12-27 14:02:49,407][20347] Updated weights for policy 0, policy_version 1212094 (0.0008) [2023-12-27 14:02:49,465][20347] Updated weights for policy 0, policy_version 1212104 (0.0007) [2023-12-27 14:02:49,498][20381] Updated weights for policy 1, policy_version 1210971 (0.0007) [2023-12-27 14:02:49,553][20381] Updated weights for policy 1, policy_version 1210981 (0.0007) [2023-12-27 14:02:49,609][20381] Updated weights for policy 1, policy_version 1210991 (0.0007) [2023-12-27 14:02:50,167][20347] Updated weights for policy 0, policy_version 1212114 (0.0006) [2023-12-27 14:02:50,226][20347] Updated weights for policy 0, policy_version 1212124 (0.0005) [2023-12-27 14:02:50,275][20347] Updated weights for policy 0, policy_version 1212134 (0.0006) [2023-12-27 14:02:50,331][20347] Updated weights for policy 0, policy_version 1212144 (0.0007) [2023-12-27 14:02:50,336][20381] Updated weights for policy 1, policy_version 1211001 (0.0008) [2023-12-27 14:02:50,391][20381] Updated weights for policy 1, policy_version 1211011 (0.0009) [2023-12-27 14:02:50,444][20381] Updated weights for policy 1, policy_version 1211021 (0.0007) [2023-12-27 14:02:50,501][20381] Updated weights for policy 1, policy_version 1211031 (0.0008) [2023-12-27 14:02:51,017][20347] Updated weights for policy 0, policy_version 1212154 (0.0006) [2023-12-27 14:02:51,052][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 14:02:51,171][20381] Updated weights for policy 1, policy_version 1211041 (0.0009) [2023-12-27 14:02:51,221][20381] Updated weights for policy 1, policy_version 1211051 (0.0010) [2023-12-27 14:02:51,276][20381] Updated weights for policy 1, policy_version 1211061 (0.0010) [2023-12-27 14:02:51,760][20347] Updated weights for policy 0, policy_version 1212164 (0.0008) [2023-12-27 14:02:51,824][20347] Updated weights for policy 0, policy_version 1212174 (0.0007) [2023-12-27 14:02:51,885][20347] Updated weights for policy 0, policy_version 1212184 (0.0007) [2023-12-27 14:02:51,979][20381] Updated weights for policy 1, policy_version 1211071 (0.0008) [2023-12-27 14:02:52,024][20381] Updated weights for policy 1, policy_version 1211081 (0.0007) [2023-12-27 14:02:52,086][20381] Updated weights for policy 1, policy_version 1211091 (0.0007) [2023-12-27 14:02:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 620445696. Throughput: 0: 10410.9, 1: 10143.4. Samples: 620413776. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:52,363][19170] Avg episode reward: [(0, '960.820'), (1, '794.730')] [2023-12-27 14:02:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001211096_310067200.pth... [2023-12-27 14:02:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001212192_310378496.pth... [2023-12-27 14:02:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001209912_309764096.pth [2023-12-27 14:02:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001210992_310067200.pth [2023-12-27 14:02:52,573][20347] Updated weights for policy 0, policy_version 1212194 (0.0007) [2023-12-27 14:02:52,628][20347] Updated weights for policy 0, policy_version 1212204 (0.0008) [2023-12-27 14:02:52,682][20347] Updated weights for policy 0, policy_version 1212214 (0.0007) [2023-12-27 14:02:52,739][20347] Updated weights for policy 0, policy_version 1212224 (0.0008) [2023-12-27 14:02:52,789][20381] Updated weights for policy 1, policy_version 1211101 (0.0007) [2023-12-27 14:02:52,849][20381] Updated weights for policy 1, policy_version 1211111 (0.0005) [2023-12-27 14:02:52,909][20381] Updated weights for policy 1, policy_version 1211121 (0.0008) [2023-12-27 14:02:53,401][20347] Updated weights for policy 0, policy_version 1212234 (0.0008) [2023-12-27 14:02:53,447][20347] Updated weights for policy 0, policy_version 1212244 (0.0008) [2023-12-27 14:02:53,495][20347] Updated weights for policy 0, policy_version 1212254 (0.0007) [2023-12-27 14:02:53,530][20381] Updated weights for policy 1, policy_version 1211131 (0.0010) [2023-12-27 14:02:53,574][20381] Updated weights for policy 1, policy_version 1211141 (0.0008) [2023-12-27 14:02:53,619][20381] Updated weights for policy 1, policy_version 1211151 (0.0007) [2023-12-27 14:02:54,227][20347] Updated weights for policy 0, policy_version 1212264 (0.0008) [2023-12-27 14:02:54,258][20381] Updated weights for policy 1, policy_version 1211161 (0.0008) [2023-12-27 14:02:54,280][20347] Updated weights for policy 0, policy_version 1212274 (0.0007) [2023-12-27 14:02:54,307][20381] Updated weights for policy 1, policy_version 1211171 (0.0007) [2023-12-27 14:02:54,333][20347] Updated weights for policy 0, policy_version 1212284 (0.0006) [2023-12-27 14:02:54,362][20381] Updated weights for policy 1, policy_version 1211181 (0.0007) [2023-12-27 14:02:54,419][20381] Updated weights for policy 1, policy_version 1211191 (0.0010) [2023-12-27 14:02:54,926][20347] Updated weights for policy 0, policy_version 1212294 (0.0006) [2023-12-27 14:02:54,979][20381] Updated weights for policy 1, policy_version 1211201 (0.0007) [2023-12-27 14:02:54,985][20347] Updated weights for policy 0, policy_version 1212304 (0.0008) [2023-12-27 14:02:55,038][20381] Updated weights for policy 1, policy_version 1211211 (0.0011) [2023-12-27 14:02:55,041][20347] Updated weights for policy 0, policy_version 1212314 (0.0005) [2023-12-27 14:02:55,096][20381] Updated weights for policy 1, policy_version 1211221 (0.0010) [2023-12-27 14:02:55,654][20381] Updated weights for policy 1, policy_version 1211231 (0.0006) [2023-12-27 14:02:55,712][20381] Updated weights for policy 1, policy_version 1211241 (0.0006) [2023-12-27 14:02:55,713][20347] Updated weights for policy 0, policy_version 1212324 (0.0006) [2023-12-27 14:02:55,769][20381] Updated weights for policy 1, policy_version 1211251 (0.0006) [2023-12-27 14:02:55,771][20347] Updated weights for policy 0, policy_version 1212334 (0.0006) [2023-12-27 14:02:55,772][20291] KL-divergence is very high: 265.9288 [2023-12-27 14:02:55,822][20291] KL-divergence is very high: 304.9715 [2023-12-27 14:02:55,834][20347] Updated weights for policy 0, policy_version 1212344 (0.0007) [2023-12-27 14:02:55,871][20291] KL-divergence is very high: 175.3697 [2023-12-27 14:02:56,325][20381] Updated weights for policy 1, policy_version 1211261 (0.0008) [2023-12-27 14:02:56,382][20381] Updated weights for policy 1, policy_version 1211271 (0.0007) [2023-12-27 14:02:56,406][20347] Updated weights for policy 0, policy_version 1212354 (0.0005) [2023-12-27 14:02:56,435][20381] Updated weights for policy 1, policy_version 1211281 (0.0006) [2023-12-27 14:02:56,456][20347] Updated weights for policy 0, policy_version 1212364 (0.0005) [2023-12-27 14:02:56,501][20347] Updated weights for policy 0, policy_version 1212374 (0.0005) [2023-12-27 14:02:56,544][20347] Updated weights for policy 0, policy_version 1212384 (0.0005) [2023-12-27 14:02:57,085][20347] Updated weights for policy 0, policy_version 1212394 (0.0007) [2023-12-27 14:02:57,086][20381] Updated weights for policy 1, policy_version 1211291 (0.0007) [2023-12-27 14:02:57,129][20347] Updated weights for policy 0, policy_version 1212404 (0.0007) [2023-12-27 14:02:57,138][20381] Updated weights for policy 1, policy_version 1211301 (0.0005) [2023-12-27 14:02:57,180][20347] Updated weights for policy 0, policy_version 1212414 (0.0008) [2023-12-27 14:02:57,182][20381] Updated weights for policy 1, policy_version 1211311 (0.0005) [2023-12-27 14:02:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 620560384. Throughput: 0: 10431.2, 1: 10213.3. Samples: 620541472. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:02:57,363][19170] Avg episode reward: [(0, '980.450'), (1, '745.880')] [2023-12-27 14:02:57,738][20381] Updated weights for policy 1, policy_version 1211321 (0.0005) [2023-12-27 14:02:57,792][20381] Updated weights for policy 1, policy_version 1211331 (0.0006) [2023-12-27 14:02:57,842][20381] Updated weights for policy 1, policy_version 1211341 (0.0005) [2023-12-27 14:02:57,893][20381] Updated weights for policy 1, policy_version 1211351 (0.0005) [2023-12-27 14:02:57,920][20347] Updated weights for policy 0, policy_version 1212424 (0.0007) [2023-12-27 14:02:57,974][20347] Updated weights for policy 0, policy_version 1212434 (0.0007) [2023-12-27 14:02:58,025][20347] Updated weights for policy 0, policy_version 1212444 (0.0007) [2023-12-27 14:02:58,544][20381] Updated weights for policy 1, policy_version 1211361 (0.0008) [2023-12-27 14:02:58,608][20381] Updated weights for policy 1, policy_version 1211371 (0.0009) [2023-12-27 14:02:58,674][20381] Updated weights for policy 1, policy_version 1211381 (0.0008) [2023-12-27 14:02:58,764][20347] Updated weights for policy 0, policy_version 1212454 (0.0008) [2023-12-27 14:02:58,828][20347] Updated weights for policy 0, policy_version 1212464 (0.0008) [2023-12-27 14:02:58,899][20347] Updated weights for policy 0, policy_version 1212474 (0.0009) [2023-12-27 14:02:59,482][20381] Updated weights for policy 1, policy_version 1211391 (0.0008) [2023-12-27 14:02:59,539][20381] Updated weights for policy 1, policy_version 1211401 (0.0008) [2023-12-27 14:02:59,592][20381] Updated weights for policy 1, policy_version 1211411 (0.0008) [2023-12-27 14:02:59,644][20347] Updated weights for policy 0, policy_version 1212484 (0.0008) [2023-12-27 14:02:59,701][20347] Updated weights for policy 0, policy_version 1212494 (0.0008) [2023-12-27 14:02:59,755][20347] Updated weights for policy 0, policy_version 1212504 (0.0007) [2023-12-27 14:03:00,264][20381] Updated weights for policy 1, policy_version 1211421 (0.0006) [2023-12-27 14:03:00,318][20381] Updated weights for policy 1, policy_version 1211431 (0.0007) [2023-12-27 14:03:00,383][20381] Updated weights for policy 1, policy_version 1211441 (0.0008) [2023-12-27 14:03:00,499][20347] Updated weights for policy 0, policy_version 1212514 (0.0007) [2023-12-27 14:03:00,553][20347] Updated weights for policy 0, policy_version 1212524 (0.0005) [2023-12-27 14:03:00,602][20347] Updated weights for policy 0, policy_version 1212534 (0.0006) [2023-12-27 14:03:00,661][20347] Updated weights for policy 0, policy_version 1212544 (0.0005) [2023-12-27 14:03:00,910][20381] Updated weights for policy 1, policy_version 1211451 (0.0009) [2023-12-27 14:03:00,973][20381] Updated weights for policy 1, policy_version 1211461 (0.0005) [2023-12-27 14:03:01,025][20381] Updated weights for policy 1, policy_version 1211471 (0.0006) [2023-12-27 14:03:01,298][20347] Updated weights for policy 0, policy_version 1212554 (0.0010) [2023-12-27 14:03:01,360][20347] Updated weights for policy 0, policy_version 1212564 (0.0010) [2023-12-27 14:03:01,420][20347] Updated weights for policy 0, policy_version 1212574 (0.0009) [2023-12-27 14:03:01,652][20381] Updated weights for policy 1, policy_version 1211481 (0.0010) [2023-12-27 14:03:01,713][20381] Updated weights for policy 1, policy_version 1211491 (0.0011) [2023-12-27 14:03:01,777][20381] Updated weights for policy 1, policy_version 1211501 (0.0010) [2023-12-27 14:03:01,838][20381] Updated weights for policy 1, policy_version 1211511 (0.0010) [2023-12-27 14:03:02,078][20347] Updated weights for policy 0, policy_version 1212584 (0.0007) [2023-12-27 14:03:02,109][20291] KL-divergence is very high: 113.9996 [2023-12-27 14:03:02,126][20347] Updated weights for policy 0, policy_version 1212594 (0.0007) [2023-12-27 14:03:02,177][20347] Updated weights for policy 0, policy_version 1212604 (0.0007) [2023-12-27 14:03:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 620658688. Throughput: 0: 10353.5, 1: 10343.1. Samples: 620667152. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:02,363][19170] Avg episode reward: [(0, '913.240'), (1, '804.540')] [2023-12-27 14:03:02,558][20381] Updated weights for policy 1, policy_version 1211521 (0.0011) [2023-12-27 14:03:02,620][20381] Updated weights for policy 1, policy_version 1211531 (0.0010) [2023-12-27 14:03:02,675][20381] Updated weights for policy 1, policy_version 1211541 (0.0011) [2023-12-27 14:03:02,855][20347] Updated weights for policy 0, policy_version 1212614 (0.0008) [2023-12-27 14:03:02,909][20347] Updated weights for policy 0, policy_version 1212624 (0.0007) [2023-12-27 14:03:02,963][20347] Updated weights for policy 0, policy_version 1212634 (0.0007) [2023-12-27 14:03:03,308][20381] Updated weights for policy 1, policy_version 1211551 (0.0011) [2023-12-27 14:03:03,366][20381] Updated weights for policy 1, policy_version 1211561 (0.0010) [2023-12-27 14:03:03,418][20381] Updated weights for policy 1, policy_version 1211571 (0.0011) [2023-12-27 14:03:03,635][20347] Updated weights for policy 0, policy_version 1212644 (0.0006) [2023-12-27 14:03:03,682][20347] Updated weights for policy 0, policy_version 1212654 (0.0007) [2023-12-27 14:03:03,728][20347] Updated weights for policy 0, policy_version 1212664 (0.0010) [2023-12-27 14:03:04,064][20381] Updated weights for policy 1, policy_version 1211581 (0.0011) [2023-12-27 14:03:04,119][20381] Updated weights for policy 1, policy_version 1211591 (0.0011) [2023-12-27 14:03:04,179][20381] Updated weights for policy 1, policy_version 1211601 (0.0011) [2023-12-27 14:03:04,415][20347] Updated weights for policy 0, policy_version 1212674 (0.0009) [2023-12-27 14:03:04,473][20347] Updated weights for policy 0, policy_version 1212684 (0.0007) [2023-12-27 14:03:04,536][20347] Updated weights for policy 0, policy_version 1212694 (0.0006) [2023-12-27 14:03:04,598][20347] Updated weights for policy 0, policy_version 1212704 (0.0007) [2023-12-27 14:03:04,840][20381] Updated weights for policy 1, policy_version 1211611 (0.0011) [2023-12-27 14:03:04,888][20381] Updated weights for policy 1, policy_version 1211621 (0.0010) [2023-12-27 14:03:04,937][20381] Updated weights for policy 1, policy_version 1211631 (0.0010) [2023-12-27 14:03:05,224][20347] Updated weights for policy 0, policy_version 1212714 (0.0006) [2023-12-27 14:03:05,279][20347] Updated weights for policy 0, policy_version 1212724 (0.0005) [2023-12-27 14:03:05,328][20347] Updated weights for policy 0, policy_version 1212734 (0.0005) [2023-12-27 14:03:05,545][20381] Updated weights for policy 1, policy_version 1211641 (0.0006) [2023-12-27 14:03:05,602][20381] Updated weights for policy 1, policy_version 1211651 (0.0011) [2023-12-27 14:03:05,653][20381] Updated weights for policy 1, policy_version 1211661 (0.0010) [2023-12-27 14:03:05,708][20381] Updated weights for policy 1, policy_version 1211671 (0.0010) [2023-12-27 14:03:05,888][20347] Updated weights for policy 0, policy_version 1212744 (0.0009) [2023-12-27 14:03:05,937][20347] Updated weights for policy 0, policy_version 1212754 (0.0009) [2023-12-27 14:03:05,990][20347] Updated weights for policy 0, policy_version 1212764 (0.0005) [2023-12-27 14:03:06,411][20381] Updated weights for policy 1, policy_version 1211681 (0.0008) [2023-12-27 14:03:06,461][20381] Updated weights for policy 1, policy_version 1211691 (0.0008) [2023-12-27 14:03:06,513][20381] Updated weights for policy 1, policy_version 1211701 (0.0008) [2023-12-27 14:03:06,674][20347] Updated weights for policy 0, policy_version 1212774 (0.0007) [2023-12-27 14:03:06,736][20347] Updated weights for policy 0, policy_version 1212784 (0.0008) [2023-12-27 14:03:06,799][20347] Updated weights for policy 0, policy_version 1212794 (0.0006) [2023-12-27 14:03:07,217][20381] Updated weights for policy 1, policy_version 1211711 (0.0010) [2023-12-27 14:03:07,285][20381] Updated weights for policy 1, policy_version 1211721 (0.0008) [2023-12-27 14:03:07,345][20381] Updated weights for policy 1, policy_version 1211731 (0.0006) [2023-12-27 14:03:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 620756992. Throughput: 0: 10339.2, 1: 10378.1. Samples: 620729716. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:07,363][19170] Avg episode reward: [(0, '893.390'), (1, '814.140')] [2023-12-27 14:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001212800_310534144.pth... [2023-12-27 14:03:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001211736_310231040.pth... [2023-12-27 14:03:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001211600_310222848.pth [2023-12-27 14:03:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001210520_309919744.pth [2023-12-27 14:03:07,437][20347] Updated weights for policy 0, policy_version 1212804 (0.0007) [2023-12-27 14:03:07,491][20347] Updated weights for policy 0, policy_version 1212814 (0.0009) [2023-12-27 14:03:07,547][20347] Updated weights for policy 0, policy_version 1212824 (0.0009) [2023-12-27 14:03:07,940][20381] Updated weights for policy 1, policy_version 1211741 (0.0006) [2023-12-27 14:03:08,003][20381] Updated weights for policy 1, policy_version 1211751 (0.0006) [2023-12-27 14:03:08,059][20381] Updated weights for policy 1, policy_version 1211761 (0.0008) [2023-12-27 14:03:08,189][20347] Updated weights for policy 0, policy_version 1212834 (0.0008) [2023-12-27 14:03:08,250][20347] Updated weights for policy 0, policy_version 1212844 (0.0005) [2023-12-27 14:03:08,308][20347] Updated weights for policy 0, policy_version 1212854 (0.0005) [2023-12-27 14:03:08,368][20347] Updated weights for policy 0, policy_version 1212864 (0.0007) [2023-12-27 14:03:08,645][20381] Updated weights for policy 1, policy_version 1211771 (0.0006) [2023-12-27 14:03:08,695][20381] Updated weights for policy 1, policy_version 1211781 (0.0009) [2023-12-27 14:03:08,745][20381] Updated weights for policy 1, policy_version 1211791 (0.0007) [2023-12-27 14:03:08,975][20347] Updated weights for policy 0, policy_version 1212874 (0.0007) [2023-12-27 14:03:09,040][20347] Updated weights for policy 0, policy_version 1212884 (0.0008) [2023-12-27 14:03:09,102][20347] Updated weights for policy 0, policy_version 1212894 (0.0007) [2023-12-27 14:03:09,446][20381] Updated weights for policy 1, policy_version 1211801 (0.0009) [2023-12-27 14:03:09,501][20381] Updated weights for policy 1, policy_version 1211811 (0.0007) [2023-12-27 14:03:09,557][20381] Updated weights for policy 1, policy_version 1211821 (0.0007) [2023-12-27 14:03:09,611][20381] Updated weights for policy 1, policy_version 1211831 (0.0008) [2023-12-27 14:03:09,838][20347] Updated weights for policy 0, policy_version 1212904 (0.0009) [2023-12-27 14:03:09,904][20347] Updated weights for policy 0, policy_version 1212914 (0.0011) [2023-12-27 14:03:09,967][20347] Updated weights for policy 0, policy_version 1212924 (0.0010) [2023-12-27 14:03:10,387][20381] Updated weights for policy 1, policy_version 1211841 (0.0011) [2023-12-27 14:03:10,447][20381] Updated weights for policy 1, policy_version 1211851 (0.0011) [2023-12-27 14:03:10,497][20381] Updated weights for policy 1, policy_version 1211861 (0.0011) [2023-12-27 14:03:10,659][20347] Updated weights for policy 0, policy_version 1212934 (0.0007) [2023-12-27 14:03:10,714][20347] Updated weights for policy 0, policy_version 1212944 (0.0007) [2023-12-27 14:03:10,779][20347] Updated weights for policy 0, policy_version 1212954 (0.0009) [2023-12-27 14:03:11,158][20381] Updated weights for policy 1, policy_version 1211871 (0.0010) [2023-12-27 14:03:11,222][20381] Updated weights for policy 1, policy_version 1211881 (0.0008) [2023-12-27 14:03:11,291][20381] Updated weights for policy 1, policy_version 1211891 (0.0008) [2023-12-27 14:03:11,443][20347] Updated weights for policy 0, policy_version 1212964 (0.0008) [2023-12-27 14:03:11,500][20347] Updated weights for policy 0, policy_version 1212974 (0.0008) [2023-12-27 14:03:11,513][20291] KL-divergence is very high: 113.6363 [2023-12-27 14:03:11,519][20291] KL-divergence is very high: 203.8770 [2023-12-27 14:03:11,526][20291] KL-divergence is very high: 205.8632 [2023-12-27 14:03:11,543][20291] KL-divergence is very high: 144.4462 [2023-12-27 14:03:11,560][20347] Updated weights for policy 0, policy_version 1212984 (0.0008) [2023-12-27 14:03:11,561][20291] KL-divergence is very high: 120.8080 [2023-12-27 14:03:11,597][20291] KL-divergence is very high: 101.2621 [2023-12-27 14:03:12,028][20381] Updated weights for policy 1, policy_version 1211901 (0.0008) [2023-12-27 14:03:12,090][20381] Updated weights for policy 1, policy_version 1211911 (0.0008) [2023-12-27 14:03:12,146][20381] Updated weights for policy 1, policy_version 1211921 (0.0008) [2023-12-27 14:03:12,254][20347] Updated weights for policy 0, policy_version 1212994 (0.0008) [2023-12-27 14:03:12,313][20347] Updated weights for policy 0, policy_version 1213004 (0.0009) [2023-12-27 14:03:12,360][20291] KL-divergence is very high: 225.7014 [2023-12-27 14:03:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.4, 300 sec: 20577.2). Total num frames: 620863488. Throughput: 0: 10399.5, 1: 10429.8. Samples: 620855992. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:12,363][19170] Avg episode reward: [(0, '814.530'), (1, '843.540')] [2023-12-27 14:03:12,373][20347] Updated weights for policy 0, policy_version 1213014 (0.0009) [2023-12-27 14:03:12,381][20291] KL-divergence is very high: 256.0049 [2023-12-27 14:03:12,418][20291] KL-divergence is very high: 179.0823 [2023-12-27 14:03:12,428][20291] KL-divergence is very high: 135.4017 [2023-12-27 14:03:12,435][20347] Updated weights for policy 0, policy_version 1213024 (0.0009) [2023-12-27 14:03:12,946][20381] Updated weights for policy 1, policy_version 1211931 (0.0008) [2023-12-27 14:03:12,997][20381] Updated weights for policy 1, policy_version 1211941 (0.0010) [2023-12-27 14:03:13,052][20381] Updated weights for policy 1, policy_version 1211951 (0.0010) [2023-12-27 14:03:13,187][20347] Updated weights for policy 0, policy_version 1213034 (0.0007) [2023-12-27 14:03:13,189][20291] KL-divergence is very high: 104.7639 [2023-12-27 14:03:13,231][20291] KL-divergence is very high: 124.5565 [2023-12-27 14:03:13,242][20347] Updated weights for policy 0, policy_version 1213044 (0.0006) [2023-12-27 14:03:13,264][20291] KL-divergence is very high: 526.2662 [2023-12-27 14:03:13,269][20291] KL-divergence is very high: 694.1128 [2023-12-27 14:03:13,293][20347] Updated weights for policy 0, policy_version 1213054 (0.0007) [2023-12-27 14:03:13,651][20381] Updated weights for policy 1, policy_version 1211961 (0.0011) [2023-12-27 14:03:13,711][20381] Updated weights for policy 1, policy_version 1211971 (0.0010) [2023-12-27 14:03:13,782][20381] Updated weights for policy 1, policy_version 1211981 (0.0005) [2023-12-27 14:03:13,845][20381] Updated weights for policy 1, policy_version 1211991 (0.0005) [2023-12-27 14:03:13,950][20347] Updated weights for policy 0, policy_version 1213064 (0.0006) [2023-12-27 14:03:14,004][20347] Updated weights for policy 0, policy_version 1213074 (0.0007) [2023-12-27 14:03:14,051][20347] Updated weights for policy 0, policy_version 1213084 (0.0007) [2023-12-27 14:03:14,406][20381] Updated weights for policy 1, policy_version 1212001 (0.0006) [2023-12-27 14:03:14,461][20381] Updated weights for policy 1, policy_version 1212011 (0.0011) [2023-12-27 14:03:14,521][20381] Updated weights for policy 1, policy_version 1212021 (0.0008) [2023-12-27 14:03:14,687][20347] Updated weights for policy 0, policy_version 1213094 (0.0009) [2023-12-27 14:03:14,739][20347] Updated weights for policy 0, policy_version 1213104 (0.0010) [2023-12-27 14:03:14,802][20347] Updated weights for policy 0, policy_version 1213114 (0.0010) [2023-12-27 14:03:15,203][20381] Updated weights for policy 1, policy_version 1212031 (0.0009) [2023-12-27 14:03:15,266][20381] Updated weights for policy 1, policy_version 1212041 (0.0008) [2023-12-27 14:03:15,326][20381] Updated weights for policy 1, policy_version 1212051 (0.0007) [2023-12-27 14:03:15,479][20347] Updated weights for policy 0, policy_version 1213124 (0.0008) [2023-12-27 14:03:15,536][20347] Updated weights for policy 0, policy_version 1213134 (0.0008) [2023-12-27 14:03:15,595][20347] Updated weights for policy 0, policy_version 1213144 (0.0010) [2023-12-27 14:03:15,945][20381] Updated weights for policy 1, policy_version 1212061 (0.0007) [2023-12-27 14:03:16,004][20381] Updated weights for policy 1, policy_version 1212071 (0.0007) [2023-12-27 14:03:16,062][20381] Updated weights for policy 1, policy_version 1212081 (0.0007) [2023-12-27 14:03:16,216][20347] Updated weights for policy 0, policy_version 1213154 (0.0007) [2023-12-27 14:03:16,264][20347] Updated weights for policy 0, policy_version 1213164 (0.0010) [2023-12-27 14:03:16,312][20347] Updated weights for policy 0, policy_version 1213174 (0.0010) [2023-12-27 14:03:16,371][20347] Updated weights for policy 0, policy_version 1213184 (0.0010) [2023-12-27 14:03:16,713][20381] Updated weights for policy 1, policy_version 1212091 (0.0006) [2023-12-27 14:03:16,765][20381] Updated weights for policy 1, policy_version 1212101 (0.0005) [2023-12-27 14:03:16,811][20381] Updated weights for policy 1, policy_version 1212111 (0.0005) [2023-12-27 14:03:17,035][20347] Updated weights for policy 0, policy_version 1213194 (0.0011) [2023-12-27 14:03:17,086][20347] Updated weights for policy 0, policy_version 1213204 (0.0010) [2023-12-27 14:03:17,148][20347] Updated weights for policy 0, policy_version 1213214 (0.0010) [2023-12-27 14:03:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 620969984. Throughput: 0: 10374.0, 1: 10416.9. Samples: 620979116. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:17,363][19170] Avg episode reward: [(0, '765.600'), (1, '814.340')] [2023-12-27 14:03:17,475][20381] Updated weights for policy 1, policy_version 1212121 (0.0006) [2023-12-27 14:03:17,523][20381] Updated weights for policy 1, policy_version 1212131 (0.0008) [2023-12-27 14:03:17,569][20381] Updated weights for policy 1, policy_version 1212141 (0.0007) [2023-12-27 14:03:17,620][20381] Updated weights for policy 1, policy_version 1212151 (0.0007) [2023-12-27 14:03:17,843][20347] Updated weights for policy 0, policy_version 1213224 (0.0006) [2023-12-27 14:03:17,904][20347] Updated weights for policy 0, policy_version 1213234 (0.0005) [2023-12-27 14:03:17,963][20347] Updated weights for policy 0, policy_version 1213244 (0.0005) [2023-12-27 14:03:18,248][20381] Updated weights for policy 1, policy_version 1212161 (0.0006) [2023-12-27 14:03:18,297][20381] Updated weights for policy 1, policy_version 1212171 (0.0009) [2023-12-27 14:03:18,360][20381] Updated weights for policy 1, policy_version 1212181 (0.0009) [2023-12-27 14:03:18,541][20347] Updated weights for policy 0, policy_version 1213254 (0.0006) [2023-12-27 14:03:18,604][20347] Updated weights for policy 0, policy_version 1213264 (0.0005) [2023-12-27 14:03:18,664][20347] Updated weights for policy 0, policy_version 1213274 (0.0007) [2023-12-27 14:03:19,100][20381] Updated weights for policy 1, policy_version 1212191 (0.0007) [2023-12-27 14:03:19,156][20381] Updated weights for policy 1, policy_version 1212201 (0.0008) [2023-12-27 14:03:19,206][20381] Updated weights for policy 1, policy_version 1212211 (0.0008) [2023-12-27 14:03:19,252][20347] Updated weights for policy 0, policy_version 1213284 (0.0007) [2023-12-27 14:03:19,315][20347] Updated weights for policy 0, policy_version 1213294 (0.0007) [2023-12-27 14:03:19,381][20347] Updated weights for policy 0, policy_version 1213304 (0.0007) [2023-12-27 14:03:19,917][20381] Updated weights for policy 1, policy_version 1212221 (0.0009) [2023-12-27 14:03:19,986][20381] Updated weights for policy 1, policy_version 1212231 (0.0011) [2023-12-27 14:03:20,055][20381] Updated weights for policy 1, policy_version 1212241 (0.0009) [2023-12-27 14:03:20,055][20347] Updated weights for policy 0, policy_version 1213314 (0.0006) [2023-12-27 14:03:20,114][20347] Updated weights for policy 0, policy_version 1213324 (0.0006) [2023-12-27 14:03:20,176][20347] Updated weights for policy 0, policy_version 1213334 (0.0007) [2023-12-27 14:03:20,235][20347] Updated weights for policy 0, policy_version 1213344 (0.0007) [2023-12-27 14:03:20,679][20381] Updated weights for policy 1, policy_version 1212251 (0.0008) [2023-12-27 14:03:20,747][20381] Updated weights for policy 1, policy_version 1212261 (0.0009) [2023-12-27 14:03:20,807][20381] Updated weights for policy 1, policy_version 1212271 (0.0011) [2023-12-27 14:03:20,969][20347] Updated weights for policy 0, policy_version 1213354 (0.0007) [2023-12-27 14:03:21,026][20347] Updated weights for policy 0, policy_version 1213364 (0.0007) [2023-12-27 14:03:21,087][20347] Updated weights for policy 0, policy_version 1213374 (0.0007) [2023-12-27 14:03:21,522][20381] Updated weights for policy 1, policy_version 1212281 (0.0011) [2023-12-27 14:03:21,585][20381] Updated weights for policy 1, policy_version 1212291 (0.0011) [2023-12-27 14:03:21,652][20381] Updated weights for policy 1, policy_version 1212301 (0.0012) [2023-12-27 14:03:21,723][20381] Updated weights for policy 1, policy_version 1212311 (0.0008) [2023-12-27 14:03:21,811][20347] Updated weights for policy 0, policy_version 1213384 (0.0006) [2023-12-27 14:03:21,874][20347] Updated weights for policy 0, policy_version 1213394 (0.0008) [2023-12-27 14:03:21,931][20347] Updated weights for policy 0, policy_version 1213404 (0.0007) [2023-12-27 14:03:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 621068288. Throughput: 0: 10384.2, 1: 10348.2. Samples: 621041136. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:22,363][19170] Avg episode reward: [(0, '872.810'), (1, '804.630')] [2023-12-27 14:03:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001212312_310378496.pth... [2023-12-27 14:03:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001213408_310689792.pth... [2023-12-27 14:03:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001211096_310067200.pth [2023-12-27 14:03:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001212192_310378496.pth [2023-12-27 14:03:22,491][20381] Updated weights for policy 1, policy_version 1212321 (0.0010) [2023-12-27 14:03:22,553][20381] Updated weights for policy 1, policy_version 1212331 (0.0007) [2023-12-27 14:03:22,564][20347] Updated weights for policy 0, policy_version 1213414 (0.0007) [2023-12-27 14:03:22,613][20381] Updated weights for policy 1, policy_version 1212341 (0.0005) [2023-12-27 14:03:22,622][20347] Updated weights for policy 0, policy_version 1213424 (0.0007) [2023-12-27 14:03:22,683][20347] Updated weights for policy 0, policy_version 1213434 (0.0007) [2023-12-27 14:03:23,313][20381] Updated weights for policy 1, policy_version 1212351 (0.0007) [2023-12-27 14:03:23,318][20347] Updated weights for policy 0, policy_version 1213444 (0.0007) [2023-12-27 14:03:23,364][20381] Updated weights for policy 1, policy_version 1212361 (0.0008) [2023-12-27 14:03:23,375][20347] Updated weights for policy 0, policy_version 1213454 (0.0005) [2023-12-27 14:03:23,419][20381] Updated weights for policy 1, policy_version 1212371 (0.0008) [2023-12-27 14:03:23,425][20347] Updated weights for policy 0, policy_version 1213464 (0.0006) [2023-12-27 14:03:24,049][20347] Updated weights for policy 0, policy_version 1213474 (0.0007) [2023-12-27 14:03:24,105][20347] Updated weights for policy 0, policy_version 1213484 (0.0005) [2023-12-27 14:03:24,124][20381] Updated weights for policy 1, policy_version 1212381 (0.0007) [2023-12-27 14:03:24,164][20347] Updated weights for policy 0, policy_version 1213494 (0.0007) [2023-12-27 14:03:24,182][20381] Updated weights for policy 1, policy_version 1212391 (0.0009) [2023-12-27 14:03:24,220][20347] Updated weights for policy 0, policy_version 1213504 (0.0009) [2023-12-27 14:03:24,239][20381] Updated weights for policy 1, policy_version 1212401 (0.0009) [2023-12-27 14:03:24,880][20347] Updated weights for policy 0, policy_version 1213514 (0.0006) [2023-12-27 14:03:24,939][20381] Updated weights for policy 1, policy_version 1212411 (0.0008) [2023-12-27 14:03:24,941][20347] Updated weights for policy 0, policy_version 1213524 (0.0005) [2023-12-27 14:03:24,988][20347] Updated weights for policy 0, policy_version 1213534 (0.0005) [2023-12-27 14:03:24,991][20381] Updated weights for policy 1, policy_version 1212421 (0.0007) [2023-12-27 14:03:25,044][20381] Updated weights for policy 1, policy_version 1212431 (0.0008) [2023-12-27 14:03:25,640][20347] Updated weights for policy 0, policy_version 1213544 (0.0006) [2023-12-27 14:03:25,692][20347] Updated weights for policy 0, policy_version 1213554 (0.0007) [2023-12-27 14:03:25,744][20347] Updated weights for policy 0, policy_version 1213564 (0.0007) [2023-12-27 14:03:25,746][20381] Updated weights for policy 1, policy_version 1212441 (0.0007) [2023-12-27 14:03:25,800][20381] Updated weights for policy 1, policy_version 1212451 (0.0007) [2023-12-27 14:03:25,849][20381] Updated weights for policy 1, policy_version 1212461 (0.0008) [2023-12-27 14:03:25,911][20381] Updated weights for policy 1, policy_version 1212471 (0.0008) [2023-12-27 14:03:26,417][20347] Updated weights for policy 0, policy_version 1213574 (0.0007) [2023-12-27 14:03:26,469][20347] Updated weights for policy 0, policy_version 1213584 (0.0008) [2023-12-27 14:03:26,528][20347] Updated weights for policy 0, policy_version 1213594 (0.0007) [2023-12-27 14:03:26,624][20381] Updated weights for policy 1, policy_version 1212481 (0.0007) [2023-12-27 14:03:26,678][20381] Updated weights for policy 1, policy_version 1212491 (0.0005) [2023-12-27 14:03:26,737][20381] Updated weights for policy 1, policy_version 1212501 (0.0005) [2023-12-27 14:03:27,201][20347] Updated weights for policy 0, policy_version 1213604 (0.0008) [2023-12-27 14:03:27,249][20347] Updated weights for policy 0, policy_version 1213614 (0.0005) [2023-12-27 14:03:27,293][20347] Updated weights for policy 0, policy_version 1213624 (0.0005) [2023-12-27 14:03:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 621174784. Throughput: 0: 10397.3, 1: 10297.9. Samples: 621162156. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:27,363][19170] Avg episode reward: [(0, '882.540'), (1, '785.050')] [2023-12-27 14:03:27,363][20381] Updated weights for policy 1, policy_version 1212511 (0.0006) [2023-12-27 14:03:27,417][20381] Updated weights for policy 1, policy_version 1212521 (0.0007) [2023-12-27 14:03:27,461][20381] Updated weights for policy 1, policy_version 1212531 (0.0007) [2023-12-27 14:03:27,873][20347] Updated weights for policy 0, policy_version 1213634 (0.0009) [2023-12-27 14:03:27,917][20347] Updated weights for policy 0, policy_version 1213644 (0.0010) [2023-12-27 14:03:27,962][20347] Updated weights for policy 0, policy_version 1213654 (0.0006) [2023-12-27 14:03:28,012][20347] Updated weights for policy 0, policy_version 1213664 (0.0007) [2023-12-27 14:03:28,189][20381] Updated weights for policy 1, policy_version 1212541 (0.0007) [2023-12-27 14:03:28,250][20381] Updated weights for policy 1, policy_version 1212551 (0.0007) [2023-12-27 14:03:28,303][20381] Updated weights for policy 1, policy_version 1212561 (0.0008) [2023-12-27 14:03:28,653][20347] Updated weights for policy 0, policy_version 1213674 (0.0008) [2023-12-27 14:03:28,713][20347] Updated weights for policy 0, policy_version 1213684 (0.0007) [2023-12-27 14:03:28,771][20347] Updated weights for policy 0, policy_version 1213694 (0.0008) [2023-12-27 14:03:28,961][20381] Updated weights for policy 1, policy_version 1212571 (0.0008) [2023-12-27 14:03:29,018][20381] Updated weights for policy 1, policy_version 1212581 (0.0007) [2023-12-27 14:03:29,070][20381] Updated weights for policy 1, policy_version 1212591 (0.0006) [2023-12-27 14:03:29,492][20347] Updated weights for policy 0, policy_version 1213704 (0.0008) [2023-12-27 14:03:29,544][20347] Updated weights for policy 0, policy_version 1213714 (0.0008) [2023-12-27 14:03:29,599][20347] Updated weights for policy 0, policy_version 1213724 (0.0007) [2023-12-27 14:03:29,763][20381] Updated weights for policy 1, policy_version 1212601 (0.0007) [2023-12-27 14:03:29,815][20381] Updated weights for policy 1, policy_version 1212611 (0.0008) [2023-12-27 14:03:29,876][20381] Updated weights for policy 1, policy_version 1212621 (0.0008) [2023-12-27 14:03:29,946][20381] Updated weights for policy 1, policy_version 1212631 (0.0008) [2023-12-27 14:03:30,273][20347] Updated weights for policy 0, policy_version 1213734 (0.0006) [2023-12-27 14:03:30,327][20347] Updated weights for policy 0, policy_version 1213744 (0.0006) [2023-12-27 14:03:30,393][20347] Updated weights for policy 0, policy_version 1213754 (0.0006) [2023-12-27 14:03:30,685][20381] Updated weights for policy 1, policy_version 1212641 (0.0008) [2023-12-27 14:03:30,729][20381] Updated weights for policy 1, policy_version 1212651 (0.0008) [2023-12-27 14:03:30,777][20381] Updated weights for policy 1, policy_version 1212661 (0.0009) [2023-12-27 14:03:30,954][20347] Updated weights for policy 0, policy_version 1213764 (0.0006) [2023-12-27 14:03:31,012][20347] Updated weights for policy 0, policy_version 1213774 (0.0007) [2023-12-27 14:03:31,071][20347] Updated weights for policy 0, policy_version 1213784 (0.0006) [2023-12-27 14:03:31,526][20381] Updated weights for policy 1, policy_version 1212671 (0.0006) [2023-12-27 14:03:31,589][20381] Updated weights for policy 1, policy_version 1212681 (0.0007) [2023-12-27 14:03:31,651][20381] Updated weights for policy 1, policy_version 1212691 (0.0008) [2023-12-27 14:03:31,719][20347] Updated weights for policy 0, policy_version 1213794 (0.0007) [2023-12-27 14:03:31,777][20347] Updated weights for policy 0, policy_version 1213804 (0.0007) [2023-12-27 14:03:31,841][20347] Updated weights for policy 0, policy_version 1213814 (0.0008) [2023-12-27 14:03:31,901][20347] Updated weights for policy 0, policy_version 1213824 (0.0006) [2023-12-27 14:03:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 621273088. Throughput: 0: 10453.6, 1: 10289.0. Samples: 621286536. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:32,363][19170] Avg episode reward: [(0, '892.380'), (1, '814.560')] [2023-12-27 14:03:32,400][20381] Updated weights for policy 1, policy_version 1212701 (0.0009) [2023-12-27 14:03:32,462][20381] Updated weights for policy 1, policy_version 1212711 (0.0011) [2023-12-27 14:03:32,523][20381] Updated weights for policy 1, policy_version 1212721 (0.0011) [2023-12-27 14:03:32,563][20347] Updated weights for policy 0, policy_version 1213834 (0.0008) [2023-12-27 14:03:32,620][20347] Updated weights for policy 0, policy_version 1213844 (0.0007) [2023-12-27 14:03:32,683][20347] Updated weights for policy 0, policy_version 1213854 (0.0007) [2023-12-27 14:03:33,210][20381] Updated weights for policy 1, policy_version 1212731 (0.0010) [2023-12-27 14:03:33,260][20381] Updated weights for policy 1, policy_version 1212741 (0.0010) [2023-12-27 14:03:33,291][20347] Updated weights for policy 0, policy_version 1213864 (0.0007) [2023-12-27 14:03:33,322][20381] Updated weights for policy 1, policy_version 1212751 (0.0008) [2023-12-27 14:03:33,345][20347] Updated weights for policy 0, policy_version 1213874 (0.0006) [2023-12-27 14:03:33,403][20347] Updated weights for policy 0, policy_version 1213884 (0.0005) [2023-12-27 14:03:33,971][20347] Updated weights for policy 0, policy_version 1213894 (0.0007) [2023-12-27 14:03:33,985][20381] Updated weights for policy 1, policy_version 1212761 (0.0011) [2023-12-27 14:03:34,019][20347] Updated weights for policy 0, policy_version 1213904 (0.0006) [2023-12-27 14:03:34,042][20381] Updated weights for policy 1, policy_version 1212771 (0.0010) [2023-12-27 14:03:34,076][20347] Updated weights for policy 0, policy_version 1213914 (0.0005) [2023-12-27 14:03:34,087][20381] Updated weights for policy 1, policy_version 1212781 (0.0010) [2023-12-27 14:03:34,148][20381] Updated weights for policy 1, policy_version 1212791 (0.0010) [2023-12-27 14:03:34,774][20347] Updated weights for policy 0, policy_version 1213924 (0.0006) [2023-12-27 14:03:34,826][20347] Updated weights for policy 0, policy_version 1213934 (0.0006) [2023-12-27 14:03:34,843][20381] Updated weights for policy 1, policy_version 1212801 (0.0007) [2023-12-27 14:03:34,882][20347] Updated weights for policy 0, policy_version 1213944 (0.0008) [2023-12-27 14:03:34,900][20381] Updated weights for policy 1, policy_version 1212811 (0.0005) [2023-12-27 14:03:34,956][20381] Updated weights for policy 1, policy_version 1212821 (0.0008) [2023-12-27 14:03:35,566][20381] Updated weights for policy 1, policy_version 1212831 (0.0006) [2023-12-27 14:03:35,611][20347] Updated weights for policy 0, policy_version 1213954 (0.0007) [2023-12-27 14:03:35,628][20381] Updated weights for policy 1, policy_version 1212841 (0.0007) [2023-12-27 14:03:35,664][20347] Updated weights for policy 0, policy_version 1213964 (0.0007) [2023-12-27 14:03:35,690][20381] Updated weights for policy 1, policy_version 1212851 (0.0008) [2023-12-27 14:03:35,722][20347] Updated weights for policy 0, policy_version 1213974 (0.0005) [2023-12-27 14:03:35,767][20347] Updated weights for policy 0, policy_version 1213984 (0.0006) [2023-12-27 14:03:36,346][20381] Updated weights for policy 1, policy_version 1212861 (0.0008) [2023-12-27 14:03:36,407][20381] Updated weights for policy 1, policy_version 1212871 (0.0010) [2023-12-27 14:03:36,442][20347] Updated weights for policy 0, policy_version 1213994 (0.0011) [2023-12-27 14:03:36,467][20381] Updated weights for policy 1, policy_version 1212881 (0.0011) [2023-12-27 14:03:36,498][20347] Updated weights for policy 0, policy_version 1214004 (0.0010) [2023-12-27 14:03:36,561][20347] Updated weights for policy 0, policy_version 1214014 (0.0010) [2023-12-27 14:03:37,135][20347] Updated weights for policy 0, policy_version 1214024 (0.0008) [2023-12-27 14:03:37,182][20381] Updated weights for policy 1, policy_version 1212891 (0.0010) [2023-12-27 14:03:37,187][20347] Updated weights for policy 0, policy_version 1214034 (0.0010) [2023-12-27 14:03:37,240][20347] Updated weights for policy 0, policy_version 1214044 (0.0006) [2023-12-27 14:03:37,244][20381] Updated weights for policy 1, policy_version 1212901 (0.0007) [2023-12-27 14:03:37,304][20381] Updated weights for policy 1, policy_version 1212911 (0.0006) [2023-12-27 14:03:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 621387776. Throughput: 0: 10469.3, 1: 10313.2. Samples: 621348992. Policy #0 lag: (min: 2.0, avg: 15.3, max: 34.0) [2023-12-27 14:03:37,364][19170] Avg episode reward: [(0, '951.090'), (1, '765.400')] [2023-12-27 14:03:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001212920_310534144.pth... [2023-12-27 14:03:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001214048_310853632.pth... [2023-12-27 14:03:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001212800_310534144.pth [2023-12-27 14:03:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001211736_310231040.pth [2023-12-27 14:03:37,899][20347] Updated weights for policy 0, policy_version 1214054 (0.0008) [2023-12-27 14:03:37,918][20381] Updated weights for policy 1, policy_version 1212921 (0.0008) [2023-12-27 14:03:37,968][20347] Updated weights for policy 0, policy_version 1214064 (0.0011) [2023-12-27 14:03:37,977][20381] Updated weights for policy 1, policy_version 1212931 (0.0008) [2023-12-27 14:03:38,034][20347] Updated weights for policy 0, policy_version 1214074 (0.0008) [2023-12-27 14:03:38,037][20381] Updated weights for policy 1, policy_version 1212941 (0.0008) [2023-12-27 14:03:38,085][20381] Updated weights for policy 1, policy_version 1212951 (0.0007) [2023-12-27 14:03:38,619][20347] Updated weights for policy 0, policy_version 1214084 (0.0008) [2023-12-27 14:03:38,677][20347] Updated weights for policy 0, policy_version 1214094 (0.0010) [2023-12-27 14:03:38,737][20347] Updated weights for policy 0, policy_version 1214104 (0.0010) [2023-12-27 14:03:38,793][20381] Updated weights for policy 1, policy_version 1212961 (0.0005) [2023-12-27 14:03:38,850][20381] Updated weights for policy 1, policy_version 1212971 (0.0007) [2023-12-27 14:03:38,903][20381] Updated weights for policy 1, policy_version 1212981 (0.0007) [2023-12-27 14:03:39,451][20347] Updated weights for policy 0, policy_version 1214114 (0.0010) [2023-12-27 14:03:39,518][20347] Updated weights for policy 0, policy_version 1214124 (0.0008) [2023-12-27 14:03:39,576][20347] Updated weights for policy 0, policy_version 1214134 (0.0008) [2023-12-27 14:03:39,626][20381] Updated weights for policy 1, policy_version 1212991 (0.0008) [2023-12-27 14:03:39,630][20347] Updated weights for policy 0, policy_version 1214144 (0.0006) [2023-12-27 14:03:39,692][20381] Updated weights for policy 1, policy_version 1213001 (0.0009) [2023-12-27 14:03:39,759][20381] Updated weights for policy 1, policy_version 1213011 (0.0008) [2023-12-27 14:03:40,342][20347] Updated weights for policy 0, policy_version 1214154 (0.0008) [2023-12-27 14:03:40,401][20347] Updated weights for policy 0, policy_version 1214164 (0.0008) [2023-12-27 14:03:40,434][20381] Updated weights for policy 1, policy_version 1213021 (0.0008) [2023-12-27 14:03:40,455][20347] Updated weights for policy 0, policy_version 1214174 (0.0007) [2023-12-27 14:03:40,492][20381] Updated weights for policy 1, policy_version 1213031 (0.0006) [2023-12-27 14:03:40,555][20381] Updated weights for policy 1, policy_version 1213041 (0.0008) [2023-12-27 14:03:41,153][20347] Updated weights for policy 0, policy_version 1214184 (0.0008) [2023-12-27 14:03:41,212][20347] Updated weights for policy 0, policy_version 1214194 (0.0008) [2023-12-27 14:03:41,216][20381] Updated weights for policy 1, policy_version 1213051 (0.0007) [2023-12-27 14:03:41,283][20381] Updated weights for policy 1, policy_version 1213061 (0.0008) [2023-12-27 14:03:41,298][20347] Updated weights for policy 0, policy_version 1214204 (0.0010) [2023-12-27 14:03:41,338][20381] Updated weights for policy 1, policy_version 1213071 (0.0008) [2023-12-27 14:03:42,020][20381] Updated weights for policy 1, policy_version 1213081 (0.0008) [2023-12-27 14:03:42,037][20347] Updated weights for policy 0, policy_version 1214214 (0.0010) [2023-12-27 14:03:42,083][20381] Updated weights for policy 1, policy_version 1213091 (0.0007) [2023-12-27 14:03:42,097][20347] Updated weights for policy 0, policy_version 1214224 (0.0008) [2023-12-27 14:03:42,144][20381] Updated weights for policy 1, policy_version 1213101 (0.0006) [2023-12-27 14:03:42,158][20347] Updated weights for policy 0, policy_version 1214234 (0.0010) [2023-12-27 14:03:42,213][20381] Updated weights for policy 1, policy_version 1213111 (0.0006) [2023-12-27 14:03:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 621486080. Throughput: 0: 10466.0, 1: 10216.6. Samples: 621472192. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:03:42,363][19170] Avg episode reward: [(0, '961.280'), (1, '784.740')] [2023-12-27 14:03:42,847][20347] Updated weights for policy 0, policy_version 1214244 (0.0007) [2023-12-27 14:03:42,903][20347] Updated weights for policy 0, policy_version 1214254 (0.0006) [2023-12-27 14:03:42,916][20381] Updated weights for policy 1, policy_version 1213121 (0.0006) [2023-12-27 14:03:42,960][20347] Updated weights for policy 0, policy_version 1214264 (0.0007) [2023-12-27 14:03:42,981][20381] Updated weights for policy 1, policy_version 1213131 (0.0006) [2023-12-27 14:03:43,047][20381] Updated weights for policy 1, policy_version 1213141 (0.0007) [2023-12-27 14:03:43,627][20347] Updated weights for policy 0, policy_version 1214274 (0.0007) [2023-12-27 14:03:43,664][20381] Updated weights for policy 1, policy_version 1213151 (0.0006) [2023-12-27 14:03:43,672][20347] Updated weights for policy 0, policy_version 1214284 (0.0008) [2023-12-27 14:03:43,724][20381] Updated weights for policy 1, policy_version 1213161 (0.0006) [2023-12-27 14:03:43,725][20347] Updated weights for policy 0, policy_version 1214294 (0.0007) [2023-12-27 14:03:43,780][20381] Updated weights for policy 1, policy_version 1213171 (0.0006) [2023-12-27 14:03:43,781][20347] Updated weights for policy 0, policy_version 1214304 (0.0007) [2023-12-27 14:03:44,446][20381] Updated weights for policy 1, policy_version 1213181 (0.0007) [2023-12-27 14:03:44,492][20347] Updated weights for policy 0, policy_version 1214314 (0.0006) [2023-12-27 14:03:44,511][20381] Updated weights for policy 1, policy_version 1213191 (0.0006) [2023-12-27 14:03:44,553][20347] Updated weights for policy 0, policy_version 1214324 (0.0007) [2023-12-27 14:03:44,574][20381] Updated weights for policy 1, policy_version 1213201 (0.0006) [2023-12-27 14:03:44,608][20347] Updated weights for policy 0, policy_version 1214334 (0.0010) [2023-12-27 14:03:45,350][20381] Updated weights for policy 1, policy_version 1213211 (0.0007) [2023-12-27 14:03:45,395][20347] Updated weights for policy 0, policy_version 1214344 (0.0010) [2023-12-27 14:03:45,400][20381] Updated weights for policy 1, policy_version 1213221 (0.0005) [2023-12-27 14:03:45,450][20347] Updated weights for policy 0, policy_version 1214354 (0.0011) [2023-12-27 14:03:45,451][20381] Updated weights for policy 1, policy_version 1213231 (0.0005) [2023-12-27 14:03:45,510][20347] Updated weights for policy 0, policy_version 1214364 (0.0010) [2023-12-27 14:03:46,039][20381] Updated weights for policy 1, policy_version 1213241 (0.0006) [2023-12-27 14:03:46,089][20381] Updated weights for policy 1, policy_version 1213251 (0.0007) [2023-12-27 14:03:46,145][20347] Updated weights for policy 0, policy_version 1214374 (0.0008) [2023-12-27 14:03:46,146][20381] Updated weights for policy 1, policy_version 1213261 (0.0008) [2023-12-27 14:03:46,200][20347] Updated weights for policy 0, policy_version 1214384 (0.0007) [2023-12-27 14:03:46,202][20381] Updated weights for policy 1, policy_version 1213271 (0.0006) [2023-12-27 14:03:46,248][20347] Updated weights for policy 0, policy_version 1214394 (0.0007) [2023-12-27 14:03:46,886][20381] Updated weights for policy 1, policy_version 1213281 (0.0005) [2023-12-27 14:03:46,906][20347] Updated weights for policy 0, policy_version 1214404 (0.0007) [2023-12-27 14:03:46,933][20381] Updated weights for policy 1, policy_version 1213291 (0.0005) [2023-12-27 14:03:46,952][20347] Updated weights for policy 0, policy_version 1214414 (0.0007) [2023-12-27 14:03:46,984][20381] Updated weights for policy 1, policy_version 1213301 (0.0005) [2023-12-27 14:03:47,002][20347] Updated weights for policy 0, policy_version 1214424 (0.0007) [2023-12-27 14:03:47,363][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.4, 300 sec: 20632.7). Total num frames: 621584384. Throughput: 0: 10407.7, 1: 10177.8. Samples: 621593504. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:03:47,364][19170] Avg episode reward: [(0, '961.230'), (1, '862.920')] [2023-12-27 14:03:47,501][20381] Updated weights for policy 1, policy_version 1213311 (0.0008) [2023-12-27 14:03:47,553][20381] Updated weights for policy 1, policy_version 1213321 (0.0005) [2023-12-27 14:03:47,615][20381] Updated weights for policy 1, policy_version 1213331 (0.0006) [2023-12-27 14:03:47,773][20347] Updated weights for policy 0, policy_version 1214434 (0.0009) [2023-12-27 14:03:47,830][20347] Updated weights for policy 0, policy_version 1214444 (0.0007) [2023-12-27 14:03:47,888][20347] Updated weights for policy 0, policy_version 1214454 (0.0007) [2023-12-27 14:03:47,942][20347] Updated weights for policy 0, policy_version 1214464 (0.0007) [2023-12-27 14:03:48,246][20381] Updated weights for policy 1, policy_version 1213341 (0.0006) [2023-12-27 14:03:48,305][20381] Updated weights for policy 1, policy_version 1213351 (0.0005) [2023-12-27 14:03:48,366][20381] Updated weights for policy 1, policy_version 1213361 (0.0007) [2023-12-27 14:03:48,631][20347] Updated weights for policy 0, policy_version 1214474 (0.0008) [2023-12-27 14:03:48,685][20291] KL-divergence is very high: 106.9106 [2023-12-27 14:03:48,691][20347] Updated weights for policy 0, policy_version 1214484 (0.0008) [2023-12-27 14:03:48,748][20347] Updated weights for policy 0, policy_version 1214494 (0.0008) [2023-12-27 14:03:49,038][20381] Updated weights for policy 1, policy_version 1213371 (0.0007) [2023-12-27 14:03:49,101][20381] Updated weights for policy 1, policy_version 1213381 (0.0005) [2023-12-27 14:03:49,156][20381] Updated weights for policy 1, policy_version 1213391 (0.0007) [2023-12-27 14:03:49,459][20347] Updated weights for policy 0, policy_version 1214504 (0.0008) [2023-12-27 14:03:49,512][20347] Updated weights for policy 0, policy_version 1214514 (0.0008) [2023-12-27 14:03:49,565][20347] Updated weights for policy 0, policy_version 1214524 (0.0007) [2023-12-27 14:03:49,777][20381] Updated weights for policy 1, policy_version 1213401 (0.0008) [2023-12-27 14:03:49,830][20381] Updated weights for policy 1, policy_version 1213411 (0.0008) [2023-12-27 14:03:49,890][20381] Updated weights for policy 1, policy_version 1213421 (0.0007) [2023-12-27 14:03:49,954][20381] Updated weights for policy 1, policy_version 1213431 (0.0009) [2023-12-27 14:03:50,350][20347] Updated weights for policy 0, policy_version 1214534 (0.0007) [2023-12-27 14:03:50,404][20347] Updated weights for policy 0, policy_version 1214544 (0.0008) [2023-12-27 14:03:50,457][20347] Updated weights for policy 0, policy_version 1214554 (0.0007) [2023-12-27 14:03:50,607][20381] Updated weights for policy 1, policy_version 1213441 (0.0008) [2023-12-27 14:03:50,661][20381] Updated weights for policy 1, policy_version 1213451 (0.0007) [2023-12-27 14:03:50,714][20381] Updated weights for policy 1, policy_version 1213461 (0.0007) [2023-12-27 14:03:51,201][20347] Updated weights for policy 0, policy_version 1214564 (0.0008) [2023-12-27 14:03:51,257][20347] Updated weights for policy 0, policy_version 1214574 (0.0009) [2023-12-27 14:03:51,314][20347] Updated weights for policy 0, policy_version 1214584 (0.0008) [2023-12-27 14:03:51,445][20381] Updated weights for policy 1, policy_version 1213471 (0.0007) [2023-12-27 14:03:51,509][20381] Updated weights for policy 1, policy_version 1213481 (0.0008) [2023-12-27 14:03:51,572][20381] Updated weights for policy 1, policy_version 1213491 (0.0008) [2023-12-27 14:03:51,595][20292] KL-divergence is very high: 627.2419 [2023-12-27 14:03:52,032][20347] Updated weights for policy 0, policy_version 1214594 (0.0007) [2023-12-27 14:03:52,091][20347] Updated weights for policy 0, policy_version 1214604 (0.0008) [2023-12-27 14:03:52,142][20347] Updated weights for policy 0, policy_version 1214614 (0.0007) [2023-12-27 14:03:52,194][20347] Updated weights for policy 0, policy_version 1214624 (0.0008) [2023-12-27 14:03:52,299][20381] Updated weights for policy 1, policy_version 1213501 (0.0008) [2023-12-27 14:03:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 621682688. Throughput: 0: 10353.6, 1: 10194.1. Samples: 621654364. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:03:52,363][19170] Avg episode reward: [(0, '970.720'), (1, '824.010')] [2023-12-27 14:03:52,364][20381] Updated weights for policy 1, policy_version 1213511 (0.0007) [2023-12-27 14:03:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001214624_311001088.pth... [2023-12-27 14:03:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001213408_310689792.pth [2023-12-27 14:03:52,415][20381] Updated weights for policy 1, policy_version 1213521 (0.0007) [2023-12-27 14:03:52,453][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001213528_310689792.pth... [2023-12-27 14:03:52,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001212312_310378496.pth [2023-12-27 14:03:52,946][20347] Updated weights for policy 0, policy_version 1214634 (0.0008) [2023-12-27 14:03:53,005][20347] Updated weights for policy 0, policy_version 1214644 (0.0008) [2023-12-27 14:03:53,063][20347] Updated weights for policy 0, policy_version 1214654 (0.0008) [2023-12-27 14:03:53,097][20381] Updated weights for policy 1, policy_version 1213531 (0.0008) [2023-12-27 14:03:53,156][20381] Updated weights for policy 1, policy_version 1213541 (0.0007) [2023-12-27 14:03:53,213][20381] Updated weights for policy 1, policy_version 1213551 (0.0007) [2023-12-27 14:03:53,781][20347] Updated weights for policy 0, policy_version 1214664 (0.0007) [2023-12-27 14:03:53,837][20347] Updated weights for policy 0, policy_version 1214674 (0.0007) [2023-12-27 14:03:53,879][20381] Updated weights for policy 1, policy_version 1213561 (0.0006) [2023-12-27 14:03:53,893][20347] Updated weights for policy 0, policy_version 1214684 (0.0008) [2023-12-27 14:03:53,925][20381] Updated weights for policy 1, policy_version 1213571 (0.0005) [2023-12-27 14:03:53,969][20381] Updated weights for policy 1, policy_version 1213581 (0.0005) [2023-12-27 14:03:54,018][20381] Updated weights for policy 1, policy_version 1213591 (0.0005) [2023-12-27 14:03:54,497][20347] Updated weights for policy 0, policy_version 1214694 (0.0007) [2023-12-27 14:03:54,563][20347] Updated weights for policy 0, policy_version 1214705 (0.0011) [2023-12-27 14:03:54,587][20381] Updated weights for policy 1, policy_version 1213601 (0.0005) [2023-12-27 14:03:54,624][20347] Updated weights for policy 0, policy_version 1214715 (0.0007) [2023-12-27 14:03:54,641][20381] Updated weights for policy 1, policy_version 1213611 (0.0005) [2023-12-27 14:03:54,693][20381] Updated weights for policy 1, policy_version 1213621 (0.0005) [2023-12-27 14:03:55,262][20381] Updated weights for policy 1, policy_version 1213631 (0.0009) [2023-12-27 14:03:55,307][20381] Updated weights for policy 1, policy_version 1213641 (0.0010) [2023-12-27 14:03:55,328][20347] Updated weights for policy 0, policy_version 1214725 (0.0007) [2023-12-27 14:03:55,338][20292] KL-divergence is very high: 106.2923 [2023-12-27 14:03:55,360][20381] Updated weights for policy 1, policy_version 1213651 (0.0010) [2023-12-27 14:03:55,361][20292] KL-divergence is very high: 221.3158 [2023-12-27 14:03:55,386][20292] KL-divergence is very high: 196.5287 [2023-12-27 14:03:55,389][20347] Updated weights for policy 0, policy_version 1214735 (0.0006) [2023-12-27 14:03:55,455][20347] Updated weights for policy 0, policy_version 1214745 (0.0008) [2023-12-27 14:03:55,936][20381] Updated weights for policy 1, policy_version 1213661 (0.0008) [2023-12-27 14:03:55,982][20381] Updated weights for policy 1, policy_version 1213671 (0.0010) [2023-12-27 14:03:56,037][20381] Updated weights for policy 1, policy_version 1213681 (0.0011) [2023-12-27 14:03:56,057][20347] Updated weights for policy 0, policy_version 1214755 (0.0007) [2023-12-27 14:03:56,107][20347] Updated weights for policy 0, policy_version 1214765 (0.0007) [2023-12-27 14:03:56,156][20347] Updated weights for policy 0, policy_version 1214775 (0.0007) [2023-12-27 14:03:56,633][20381] Updated weights for policy 1, policy_version 1213691 (0.0009) [2023-12-27 14:03:56,695][20381] Updated weights for policy 1, policy_version 1213701 (0.0005) [2023-12-27 14:03:56,752][20381] Updated weights for policy 1, policy_version 1213711 (0.0009) [2023-12-27 14:03:56,895][20347] Updated weights for policy 0, policy_version 1214785 (0.0009) [2023-12-27 14:03:56,953][20347] Updated weights for policy 0, policy_version 1214795 (0.0008) [2023-12-27 14:03:57,003][20347] Updated weights for policy 0, policy_version 1214805 (0.0007) [2023-12-27 14:03:57,046][20347] Updated weights for policy 0, policy_version 1214815 (0.0007) [2023-12-27 14:03:57,310][20381] Updated weights for policy 1, policy_version 1213721 (0.0010) [2023-12-27 14:03:57,357][20381] Updated weights for policy 1, policy_version 1213731 (0.0007) [2023-12-27 14:03:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 621789184. Throughput: 0: 10271.6, 1: 10237.5. Samples: 621778900. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:03:57,363][19170] Avg episode reward: [(0, '951.140'), (1, '872.860')] [2023-12-27 14:03:57,407][20381] Updated weights for policy 1, policy_version 1213741 (0.0007) [2023-12-27 14:03:57,459][20381] Updated weights for policy 1, policy_version 1213751 (0.0008) [2023-12-27 14:03:57,735][20347] Updated weights for policy 0, policy_version 1214825 (0.0008) [2023-12-27 14:03:57,789][20347] Updated weights for policy 0, policy_version 1214835 (0.0007) [2023-12-27 14:03:57,847][20347] Updated weights for policy 0, policy_version 1214845 (0.0007) [2023-12-27 14:03:58,087][20381] Updated weights for policy 1, policy_version 1213761 (0.0006) [2023-12-27 14:03:58,148][20381] Updated weights for policy 1, policy_version 1213771 (0.0007) [2023-12-27 14:03:58,209][20381] Updated weights for policy 1, policy_version 1213781 (0.0007) [2023-12-27 14:03:58,597][20347] Updated weights for policy 0, policy_version 1214855 (0.0008) [2023-12-27 14:03:58,657][20347] Updated weights for policy 0, policy_version 1214865 (0.0008) [2023-12-27 14:03:58,720][20347] Updated weights for policy 0, policy_version 1214875 (0.0008) [2023-12-27 14:03:58,949][20381] Updated weights for policy 1, policy_version 1213791 (0.0008) [2023-12-27 14:03:59,010][20381] Updated weights for policy 1, policy_version 1213801 (0.0007) [2023-12-27 14:03:59,071][20381] Updated weights for policy 1, policy_version 1213811 (0.0008) [2023-12-27 14:03:59,506][20347] Updated weights for policy 0, policy_version 1214885 (0.0008) [2023-12-27 14:03:59,559][20347] Updated weights for policy 0, policy_version 1214895 (0.0008) [2023-12-27 14:03:59,615][20347] Updated weights for policy 0, policy_version 1214905 (0.0008) [2023-12-27 14:03:59,836][20381] Updated weights for policy 1, policy_version 1213821 (0.0007) [2023-12-27 14:03:59,895][20381] Updated weights for policy 1, policy_version 1213831 (0.0006) [2023-12-27 14:03:59,955][20381] Updated weights for policy 1, policy_version 1213841 (0.0008) [2023-12-27 14:04:00,287][20347] Updated weights for policy 0, policy_version 1214915 (0.0007) [2023-12-27 14:04:00,346][20347] Updated weights for policy 0, policy_version 1214925 (0.0008) [2023-12-27 14:04:00,402][20347] Updated weights for policy 0, policy_version 1214935 (0.0006) [2023-12-27 14:04:00,604][20381] Updated weights for policy 1, policy_version 1213851 (0.0008) [2023-12-27 14:04:00,660][20381] Updated weights for policy 1, policy_version 1213861 (0.0006) [2023-12-27 14:04:00,716][20381] Updated weights for policy 1, policy_version 1213871 (0.0008) [2023-12-27 14:04:01,041][20347] Updated weights for policy 0, policy_version 1214945 (0.0008) [2023-12-27 14:04:01,105][20347] Updated weights for policy 0, policy_version 1214955 (0.0006) [2023-12-27 14:04:01,177][20347] Updated weights for policy 0, policy_version 1214965 (0.0008) [2023-12-27 14:04:01,237][20347] Updated weights for policy 0, policy_version 1214975 (0.0007) [2023-12-27 14:04:01,331][20381] Updated weights for policy 1, policy_version 1213881 (0.0006) [2023-12-27 14:04:01,393][20381] Updated weights for policy 1, policy_version 1213891 (0.0008) [2023-12-27 14:04:01,455][20381] Updated weights for policy 1, policy_version 1213901 (0.0006) [2023-12-27 14:04:01,517][20381] Updated weights for policy 1, policy_version 1213911 (0.0005) [2023-12-27 14:04:01,868][20347] Updated weights for policy 0, policy_version 1214985 (0.0006) [2023-12-27 14:04:01,916][20347] Updated weights for policy 0, policy_version 1214995 (0.0007) [2023-12-27 14:04:01,943][20291] KL-divergence is very high: 123.7497 [2023-12-27 14:04:01,972][20347] Updated weights for policy 0, policy_version 1215005 (0.0007) [2023-12-27 14:04:02,118][20381] Updated weights for policy 1, policy_version 1213921 (0.0008) [2023-12-27 14:04:02,172][20381] Updated weights for policy 1, policy_version 1213931 (0.0008) [2023-12-27 14:04:02,228][20381] Updated weights for policy 1, policy_version 1213941 (0.0009) [2023-12-27 14:04:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 621895680. Throughput: 0: 10229.2, 1: 10268.6. Samples: 621901516. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:02,363][19170] Avg episode reward: [(0, '892.430'), (1, '911.970')] [2023-12-27 14:04:02,671][20347] Updated weights for policy 0, policy_version 1215015 (0.0007) [2023-12-27 14:04:02,732][20347] Updated weights for policy 0, policy_version 1215025 (0.0008) [2023-12-27 14:04:02,781][20347] Updated weights for policy 0, policy_version 1215035 (0.0008) [2023-12-27 14:04:02,923][20381] Updated weights for policy 1, policy_version 1213951 (0.0008) [2023-12-27 14:04:02,968][20381] Updated weights for policy 1, policy_version 1213961 (0.0010) [2023-12-27 14:04:03,016][20381] Updated weights for policy 1, policy_version 1213971 (0.0010) [2023-12-27 14:04:03,426][20347] Updated weights for policy 0, policy_version 1215045 (0.0006) [2023-12-27 14:04:03,483][20347] Updated weights for policy 0, policy_version 1215055 (0.0005) [2023-12-27 14:04:03,528][20347] Updated weights for policy 0, policy_version 1215065 (0.0005) [2023-12-27 14:04:03,716][20381] Updated weights for policy 1, policy_version 1213981 (0.0010) [2023-12-27 14:04:03,768][20381] Updated weights for policy 1, policy_version 1213991 (0.0008) [2023-12-27 14:04:03,781][20292] KL-divergence is very high: 111.2704 [2023-12-27 14:04:03,829][20381] Updated weights for policy 1, policy_version 1214001 (0.0011) [2023-12-27 14:04:04,111][20347] Updated weights for policy 0, policy_version 1215075 (0.0006) [2023-12-27 14:04:04,173][20347] Updated weights for policy 0, policy_version 1215085 (0.0008) [2023-12-27 14:04:04,230][20347] Updated weights for policy 0, policy_version 1215095 (0.0008) [2023-12-27 14:04:04,488][20381] Updated weights for policy 1, policy_version 1214011 (0.0010) [2023-12-27 14:04:04,543][20381] Updated weights for policy 1, policy_version 1214021 (0.0008) [2023-12-27 14:04:04,598][20381] Updated weights for policy 1, policy_version 1214031 (0.0007) [2023-12-27 14:04:04,912][20347] Updated weights for policy 0, policy_version 1215105 (0.0008) [2023-12-27 14:04:04,961][20347] Updated weights for policy 0, policy_version 1215115 (0.0010) [2023-12-27 14:04:05,012][20347] Updated weights for policy 0, policy_version 1215125 (0.0008) [2023-12-27 14:04:05,063][20347] Updated weights for policy 0, policy_version 1215135 (0.0009) [2023-12-27 14:04:05,168][20381] Updated weights for policy 1, policy_version 1214041 (0.0007) [2023-12-27 14:04:05,231][20381] Updated weights for policy 1, policy_version 1214051 (0.0007) [2023-12-27 14:04:05,286][20381] Updated weights for policy 1, policy_version 1214061 (0.0007) [2023-12-27 14:04:05,338][20381] Updated weights for policy 1, policy_version 1214071 (0.0005) [2023-12-27 14:04:05,782][20347] Updated weights for policy 0, policy_version 1215145 (0.0006) [2023-12-27 14:04:05,839][20347] Updated weights for policy 0, policy_version 1215155 (0.0007) [2023-12-27 14:04:05,903][20347] Updated weights for policy 0, policy_version 1215165 (0.0005) [2023-12-27 14:04:05,958][20381] Updated weights for policy 1, policy_version 1214081 (0.0008) [2023-12-27 14:04:06,002][20381] Updated weights for policy 1, policy_version 1214091 (0.0009) [2023-12-27 14:04:06,047][20381] Updated weights for policy 1, policy_version 1214101 (0.0008) [2023-12-27 14:04:06,450][20347] Updated weights for policy 0, policy_version 1215175 (0.0007) [2023-12-27 14:04:06,517][20347] Updated weights for policy 0, policy_version 1215185 (0.0006) [2023-12-27 14:04:06,576][20347] Updated weights for policy 0, policy_version 1215195 (0.0007) [2023-12-27 14:04:06,582][20291] KL-divergence is very high: 121.0117 [2023-12-27 14:04:06,753][20381] Updated weights for policy 1, policy_version 1214111 (0.0008) [2023-12-27 14:04:06,806][20381] Updated weights for policy 1, policy_version 1214121 (0.0007) [2023-12-27 14:04:06,857][20381] Updated weights for policy 1, policy_version 1214131 (0.0008) [2023-12-27 14:04:07,265][20291] KL-divergence is very high: 102.5989 [2023-12-27 14:04:07,275][20347] Updated weights for policy 0, policy_version 1215205 (0.0007) [2023-12-27 14:04:07,331][20347] Updated weights for policy 0, policy_version 1215215 (0.0006) [2023-12-27 14:04:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 621993984. Throughput: 0: 10213.9, 1: 10328.6. Samples: 621965544. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:07,363][19170] Avg episode reward: [(0, '862.750'), (1, '931.600')] [2023-12-27 14:04:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001214136_310845440.pth... [2023-12-27 14:04:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001212920_310534144.pth [2023-12-27 14:04:07,391][20347] Updated weights for policy 0, policy_version 1215225 (0.0008) [2023-12-27 14:04:07,431][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001215232_311156736.pth... [2023-12-27 14:04:07,435][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001214048_310853632.pth [2023-12-27 14:04:07,550][20381] Updated weights for policy 1, policy_version 1214141 (0.0008) [2023-12-27 14:04:07,601][20381] Updated weights for policy 1, policy_version 1214151 (0.0010) [2023-12-27 14:04:07,652][20381] Updated weights for policy 1, policy_version 1214161 (0.0010) [2023-12-27 14:04:08,087][20347] Updated weights for policy 0, policy_version 1215235 (0.0008) [2023-12-27 14:04:08,151][20347] Updated weights for policy 0, policy_version 1215245 (0.0007) [2023-12-27 14:04:08,212][20347] Updated weights for policy 0, policy_version 1215255 (0.0007) [2023-12-27 14:04:08,362][20381] Updated weights for policy 1, policy_version 1214171 (0.0010) [2023-12-27 14:04:08,418][20381] Updated weights for policy 1, policy_version 1214181 (0.0011) [2023-12-27 14:04:08,480][20381] Updated weights for policy 1, policy_version 1214191 (0.0011) [2023-12-27 14:04:08,893][20347] Updated weights for policy 0, policy_version 1215265 (0.0006) [2023-12-27 14:04:08,950][20347] Updated weights for policy 0, policy_version 1215275 (0.0007) [2023-12-27 14:04:09,010][20347] Updated weights for policy 0, policy_version 1215285 (0.0010) [2023-12-27 14:04:09,064][20347] Updated weights for policy 0, policy_version 1215295 (0.0010) [2023-12-27 14:04:09,133][20381] Updated weights for policy 1, policy_version 1214201 (0.0010) [2023-12-27 14:04:09,191][20381] Updated weights for policy 1, policy_version 1214211 (0.0010) [2023-12-27 14:04:09,251][20381] Updated weights for policy 1, policy_version 1214221 (0.0009) [2023-12-27 14:04:09,302][20381] Updated weights for policy 1, policy_version 1214231 (0.0008) [2023-12-27 14:04:09,776][20347] Updated weights for policy 0, policy_version 1215305 (0.0007) [2023-12-27 14:04:09,840][20347] Updated weights for policy 0, policy_version 1215315 (0.0008) [2023-12-27 14:04:09,898][20347] Updated weights for policy 0, policy_version 1215325 (0.0007) [2023-12-27 14:04:10,045][20381] Updated weights for policy 1, policy_version 1214241 (0.0006) [2023-12-27 14:04:10,109][20381] Updated weights for policy 1, policy_version 1214251 (0.0005) [2023-12-27 14:04:10,169][20381] Updated weights for policy 1, policy_version 1214261 (0.0006) [2023-12-27 14:04:10,568][20347] Updated weights for policy 0, policy_version 1215335 (0.0011) [2023-12-27 14:04:10,622][20347] Updated weights for policy 0, policy_version 1215345 (0.0010) [2023-12-27 14:04:10,683][20347] Updated weights for policy 0, policy_version 1215355 (0.0010) [2023-12-27 14:04:10,802][20381] Updated weights for policy 1, policy_version 1214271 (0.0007) [2023-12-27 14:04:10,863][20381] Updated weights for policy 1, policy_version 1214281 (0.0007) [2023-12-27 14:04:10,924][20381] Updated weights for policy 1, policy_version 1214291 (0.0006) [2023-12-27 14:04:11,381][20347] Updated weights for policy 0, policy_version 1215365 (0.0008) [2023-12-27 14:04:11,445][20347] Updated weights for policy 0, policy_version 1215375 (0.0007) [2023-12-27 14:04:11,498][20347] Updated weights for policy 0, policy_version 1215385 (0.0010) [2023-12-27 14:04:11,631][20381] Updated weights for policy 1, policy_version 1214301 (0.0007) [2023-12-27 14:04:11,692][20381] Updated weights for policy 1, policy_version 1214311 (0.0008) [2023-12-27 14:04:11,760][20381] Updated weights for policy 1, policy_version 1214321 (0.0007) [2023-12-27 14:04:12,213][20347] Updated weights for policy 0, policy_version 1215395 (0.0010) [2023-12-27 14:04:12,295][20347] Updated weights for policy 0, policy_version 1215405 (0.0010) [2023-12-27 14:04:12,307][20291] KL-divergence is very high: 188.7649 [2023-12-27 14:04:12,313][20291] KL-divergence is very high: 210.1953 [2023-12-27 14:04:12,360][20347] Updated weights for policy 0, policy_version 1215415 (0.0010) [2023-12-27 14:04:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 622092288. Throughput: 0: 10214.5, 1: 10378.8. Samples: 622088852. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:12,363][19170] Avg episode reward: [(0, '670.770'), (1, '931.670')] [2023-12-27 14:04:12,503][20381] Updated weights for policy 1, policy_version 1214331 (0.0007) [2023-12-27 14:04:12,562][20381] Updated weights for policy 1, policy_version 1214341 (0.0008) [2023-12-27 14:04:12,622][20381] Updated weights for policy 1, policy_version 1214351 (0.0007) [2023-12-27 14:04:13,078][20347] Updated weights for policy 0, policy_version 1215425 (0.0011) [2023-12-27 14:04:13,078][20291] KL-divergence is very high: 127.2682 [2023-12-27 14:04:13,089][20291] KL-divergence is very high: 263.7591 [2023-12-27 14:04:13,107][20291] KL-divergence is very high: 153.3308 [2023-12-27 14:04:13,134][20347] Updated weights for policy 0, policy_version 1215435 (0.0007) [2023-12-27 14:04:13,174][20291] KL-divergence is very high: 106.0865 [2023-12-27 14:04:13,184][20347] Updated weights for policy 0, policy_version 1215445 (0.0007) [2023-12-27 14:04:13,190][20291] KL-divergence is very high: 137.9586 [2023-12-27 14:04:13,222][20291] KL-divergence is very high: 170.0734 [2023-12-27 14:04:13,241][20291] KL-divergence is very high: 171.7975 [2023-12-27 14:04:13,246][20347] Updated weights for policy 0, policy_version 1215455 (0.0008) [2023-12-27 14:04:13,273][20381] Updated weights for policy 1, policy_version 1214361 (0.0008) [2023-12-27 14:04:13,328][20381] Updated weights for policy 1, policy_version 1214371 (0.0005) [2023-12-27 14:04:13,386][20381] Updated weights for policy 1, policy_version 1214381 (0.0005) [2023-12-27 14:04:13,444][20381] Updated weights for policy 1, policy_version 1214391 (0.0005) [2023-12-27 14:04:13,887][20347] Updated weights for policy 0, policy_version 1215465 (0.0006) [2023-12-27 14:04:13,939][20347] Updated weights for policy 0, policy_version 1215475 (0.0006) [2023-12-27 14:04:13,990][20347] Updated weights for policy 0, policy_version 1215485 (0.0007) [2023-12-27 14:04:14,041][20381] Updated weights for policy 1, policy_version 1214401 (0.0005) [2023-12-27 14:04:14,099][20381] Updated weights for policy 1, policy_version 1214411 (0.0007) [2023-12-27 14:04:14,143][20381] Updated weights for policy 1, policy_version 1214421 (0.0007) [2023-12-27 14:04:14,624][20347] Updated weights for policy 0, policy_version 1215495 (0.0006) [2023-12-27 14:04:14,670][20347] Updated weights for policy 0, policy_version 1215505 (0.0005) [2023-12-27 14:04:14,725][20347] Updated weights for policy 0, policy_version 1215515 (0.0006) [2023-12-27 14:04:14,795][20381] Updated weights for policy 1, policy_version 1214431 (0.0008) [2023-12-27 14:04:14,866][20381] Updated weights for policy 1, policy_version 1214441 (0.0007) [2023-12-27 14:04:14,930][20381] Updated weights for policy 1, policy_version 1214451 (0.0008) [2023-12-27 14:04:15,415][20347] Updated weights for policy 0, policy_version 1215525 (0.0009) [2023-12-27 14:04:15,476][20347] Updated weights for policy 0, policy_version 1215535 (0.0010) [2023-12-27 14:04:15,537][20347] Updated weights for policy 0, policy_version 1215545 (0.0007) [2023-12-27 14:04:15,618][20381] Updated weights for policy 1, policy_version 1214461 (0.0007) [2023-12-27 14:04:15,675][20381] Updated weights for policy 1, policy_version 1214471 (0.0007) [2023-12-27 14:04:15,722][20381] Updated weights for policy 1, policy_version 1214481 (0.0007) [2023-12-27 14:04:16,182][20347] Updated weights for policy 0, policy_version 1215555 (0.0007) [2023-12-27 14:04:16,233][20347] Updated weights for policy 0, policy_version 1215565 (0.0010) [2023-12-27 14:04:16,274][20291] KL-divergence is very high: 130.2951 [2023-12-27 14:04:16,278][20347] Updated weights for policy 0, policy_version 1215575 (0.0010) [2023-12-27 14:04:16,279][20291] KL-divergence is very high: 134.1594 [2023-12-27 14:04:16,391][20381] Updated weights for policy 1, policy_version 1214491 (0.0007) [2023-12-27 14:04:16,439][20381] Updated weights for policy 1, policy_version 1214501 (0.0007) [2023-12-27 14:04:16,489][20381] Updated weights for policy 1, policy_version 1214511 (0.0007) [2023-12-27 14:04:16,920][20347] Updated weights for policy 0, policy_version 1215585 (0.0010) [2023-12-27 14:04:16,973][20347] Updated weights for policy 0, policy_version 1215595 (0.0005) [2023-12-27 14:04:17,000][20291] KL-divergence is very high: 128.2549 [2023-12-27 14:04:17,027][20347] Updated weights for policy 0, policy_version 1215605 (0.0009) [2023-12-27 14:04:17,083][20347] Updated weights for policy 0, policy_version 1215615 (0.0010) [2023-12-27 14:04:17,211][20381] Updated weights for policy 1, policy_version 1214521 (0.0008) [2023-12-27 14:04:17,267][20381] Updated weights for policy 1, policy_version 1214531 (0.0007) [2023-12-27 14:04:17,315][20381] Updated weights for policy 1, policy_version 1214541 (0.0007) [2023-12-27 14:04:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 622198784. Throughput: 0: 10161.4, 1: 10412.4. Samples: 622212360. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:17,364][19170] Avg episode reward: [(0, '471.920'), (1, '921.970')] [2023-12-27 14:04:17,392][20381] Updated weights for policy 1, policy_version 1214551 (0.0008) [2023-12-27 14:04:17,747][20347] Updated weights for policy 0, policy_version 1215625 (0.0010) [2023-12-27 14:04:17,814][20347] Updated weights for policy 0, policy_version 1215635 (0.0006) [2023-12-27 14:04:17,880][20347] Updated weights for policy 0, policy_version 1215645 (0.0008) [2023-12-27 14:04:18,068][20381] Updated weights for policy 1, policy_version 1214561 (0.0006) [2023-12-27 14:04:18,123][20381] Updated weights for policy 1, policy_version 1214571 (0.0006) [2023-12-27 14:04:18,179][20381] Updated weights for policy 1, policy_version 1214581 (0.0006) [2023-12-27 14:04:18,585][20347] Updated weights for policy 0, policy_version 1215655 (0.0007) [2023-12-27 14:04:18,636][20347] Updated weights for policy 0, policy_version 1215665 (0.0008) [2023-12-27 14:04:18,696][20347] Updated weights for policy 0, policy_version 1215675 (0.0008) [2023-12-27 14:04:18,779][20381] Updated weights for policy 1, policy_version 1214591 (0.0007) [2023-12-27 14:04:18,845][20381] Updated weights for policy 1, policy_version 1214601 (0.0007) [2023-12-27 14:04:18,910][20381] Updated weights for policy 1, policy_version 1214611 (0.0006) [2023-12-27 14:04:19,425][20347] Updated weights for policy 0, policy_version 1215685 (0.0007) [2023-12-27 14:04:19,491][20347] Updated weights for policy 0, policy_version 1215695 (0.0006) [2023-12-27 14:04:19,557][20347] Updated weights for policy 0, policy_version 1215705 (0.0007) [2023-12-27 14:04:19,639][20381] Updated weights for policy 1, policy_version 1214621 (0.0008) [2023-12-27 14:04:19,704][20381] Updated weights for policy 1, policy_version 1214631 (0.0008) [2023-12-27 14:04:19,766][20381] Updated weights for policy 1, policy_version 1214641 (0.0009) [2023-12-27 14:04:20,156][20347] Updated weights for policy 0, policy_version 1215715 (0.0008) [2023-12-27 14:04:20,224][20347] Updated weights for policy 0, policy_version 1215725 (0.0007) [2023-12-27 14:04:20,282][20347] Updated weights for policy 0, policy_version 1215735 (0.0006) [2023-12-27 14:04:20,511][20381] Updated weights for policy 1, policy_version 1214651 (0.0008) [2023-12-27 14:04:20,562][20381] Updated weights for policy 1, policy_version 1214661 (0.0007) [2023-12-27 14:04:20,628][20381] Updated weights for policy 1, policy_version 1214671 (0.0007) [2023-12-27 14:04:20,886][20347] Updated weights for policy 0, policy_version 1215745 (0.0008) [2023-12-27 14:04:20,949][20347] Updated weights for policy 0, policy_version 1215755 (0.0008) [2023-12-27 14:04:21,015][20347] Updated weights for policy 0, policy_version 1215765 (0.0007) [2023-12-27 14:04:21,082][20347] Updated weights for policy 0, policy_version 1215775 (0.0009) [2023-12-27 14:04:21,375][20381] Updated weights for policy 1, policy_version 1214681 (0.0007) [2023-12-27 14:04:21,436][20381] Updated weights for policy 1, policy_version 1214691 (0.0008) [2023-12-27 14:04:21,496][20381] Updated weights for policy 1, policy_version 1214701 (0.0007) [2023-12-27 14:04:21,558][20381] Updated weights for policy 1, policy_version 1214711 (0.0008) [2023-12-27 14:04:21,774][20347] Updated weights for policy 0, policy_version 1215785 (0.0008) [2023-12-27 14:04:21,824][20347] Updated weights for policy 0, policy_version 1215795 (0.0008) [2023-12-27 14:04:21,882][20347] Updated weights for policy 0, policy_version 1215805 (0.0009) [2023-12-27 14:04:22,249][20381] Updated weights for policy 1, policy_version 1214721 (0.0007) [2023-12-27 14:04:22,308][20381] Updated weights for policy 1, policy_version 1214731 (0.0007) [2023-12-27 14:04:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 622297088. Throughput: 0: 10111.7, 1: 10392.6. Samples: 622271680. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:22,363][19170] Avg episode reward: [(0, '88.040'), (1, '912.160')] [2023-12-27 14:04:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001215808_311304192.pth... [2023-12-27 14:04:22,373][20381] Updated weights for policy 1, policy_version 1214741 (0.0007) [2023-12-27 14:04:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001214624_311001088.pth [2023-12-27 14:04:22,390][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001214744_311001088.pth... [2023-12-27 14:04:22,395][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001213528_310689792.pth [2023-12-27 14:04:22,614][20347] Updated weights for policy 0, policy_version 1215815 (0.0008) [2023-12-27 14:04:22,671][20347] Updated weights for policy 0, policy_version 1215825 (0.0008) [2023-12-27 14:04:22,732][20347] Updated weights for policy 0, policy_version 1215835 (0.0008) [2023-12-27 14:04:23,087][20381] Updated weights for policy 1, policy_version 1214751 (0.0008) [2023-12-27 14:04:23,147][20381] Updated weights for policy 1, policy_version 1214761 (0.0008) [2023-12-27 14:04:23,194][20381] Updated weights for policy 1, policy_version 1214771 (0.0008) [2023-12-27 14:04:23,473][20347] Updated weights for policy 0, policy_version 1215845 (0.0007) [2023-12-27 14:04:23,526][20347] Updated weights for policy 0, policy_version 1215855 (0.0006) [2023-12-27 14:04:23,575][20347] Updated weights for policy 0, policy_version 1215865 (0.0007) [2023-12-27 14:04:23,818][20381] Updated weights for policy 1, policy_version 1214781 (0.0006) [2023-12-27 14:04:23,867][20381] Updated weights for policy 1, policy_version 1214791 (0.0007) [2023-12-27 14:04:23,915][20381] Updated weights for policy 1, policy_version 1214801 (0.0007) [2023-12-27 14:04:24,233][20347] Updated weights for policy 0, policy_version 1215875 (0.0009) [2023-12-27 14:04:24,294][20347] Updated weights for policy 0, policy_version 1215885 (0.0010) [2023-12-27 14:04:24,353][20347] Updated weights for policy 0, policy_version 1215895 (0.0007) [2023-12-27 14:04:24,581][20381] Updated weights for policy 1, policy_version 1214811 (0.0007) [2023-12-27 14:04:24,645][20381] Updated weights for policy 1, policy_version 1214821 (0.0008) [2023-12-27 14:04:24,708][20381] Updated weights for policy 1, policy_version 1214831 (0.0006) [2023-12-27 14:04:25,023][20347] Updated weights for policy 0, policy_version 1215905 (0.0009) [2023-12-27 14:04:25,086][20347] Updated weights for policy 0, policy_version 1215915 (0.0005) [2023-12-27 14:04:25,144][20347] Updated weights for policy 0, policy_version 1215925 (0.0005) [2023-12-27 14:04:25,198][20347] Updated weights for policy 0, policy_version 1215935 (0.0006) [2023-12-27 14:04:25,351][20381] Updated weights for policy 1, policy_version 1214841 (0.0006) [2023-12-27 14:04:25,400][20381] Updated weights for policy 1, policy_version 1214851 (0.0009) [2023-12-27 14:04:25,447][20381] Updated weights for policy 1, policy_version 1214861 (0.0010) [2023-12-27 14:04:25,506][20381] Updated weights for policy 1, policy_version 1214871 (0.0008) [2023-12-27 14:04:25,734][20347] Updated weights for policy 0, policy_version 1215945 (0.0006) [2023-12-27 14:04:25,787][20347] Updated weights for policy 0, policy_version 1215955 (0.0005) [2023-12-27 14:04:25,836][20347] Updated weights for policy 0, policy_version 1215965 (0.0006) [2023-12-27 14:04:26,246][20381] Updated weights for policy 1, policy_version 1214881 (0.0010) [2023-12-27 14:04:26,297][20381] Updated weights for policy 1, policy_version 1214891 (0.0009) [2023-12-27 14:04:26,348][20381] Updated weights for policy 1, policy_version 1214901 (0.0006) [2023-12-27 14:04:26,497][20347] Updated weights for policy 0, policy_version 1215975 (0.0009) [2023-12-27 14:04:26,543][20347] Updated weights for policy 0, policy_version 1215985 (0.0010) [2023-12-27 14:04:26,595][20347] Updated weights for policy 0, policy_version 1215995 (0.0007) [2023-12-27 14:04:26,885][20381] Updated weights for policy 1, policy_version 1214911 (0.0005) [2023-12-27 14:04:26,938][20381] Updated weights for policy 1, policy_version 1214921 (0.0005) [2023-12-27 14:04:26,993][20381] Updated weights for policy 1, policy_version 1214931 (0.0005) [2023-12-27 14:04:27,359][20347] Updated weights for policy 0, policy_version 1216005 (0.0008) [2023-12-27 14:04:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 622403584. Throughput: 0: 10104.3, 1: 10383.7. Samples: 622394156. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:27,363][19170] Avg episode reward: [(0, '151.630'), (1, '921.880')] [2023-12-27 14:04:27,406][20347] Updated weights for policy 0, policy_version 1216015 (0.0007) [2023-12-27 14:04:27,469][20347] Updated weights for policy 0, policy_version 1216025 (0.0007) [2023-12-27 14:04:27,595][20381] Updated weights for policy 1, policy_version 1214941 (0.0005) [2023-12-27 14:04:27,655][20381] Updated weights for policy 1, policy_version 1214951 (0.0007) [2023-12-27 14:04:27,713][20381] Updated weights for policy 1, policy_version 1214961 (0.0011) [2023-12-27 14:04:28,168][20347] Updated weights for policy 0, policy_version 1216035 (0.0008) [2023-12-27 14:04:28,224][20347] Updated weights for policy 0, policy_version 1216045 (0.0011) [2023-12-27 14:04:28,289][20347] Updated weights for policy 0, policy_version 1216055 (0.0006) [2023-12-27 14:04:28,326][20381] Updated weights for policy 1, policy_version 1214971 (0.0011) [2023-12-27 14:04:28,387][20381] Updated weights for policy 1, policy_version 1214981 (0.0009) [2023-12-27 14:04:28,444][20381] Updated weights for policy 1, policy_version 1214991 (0.0009) [2023-12-27 14:04:28,946][20347] Updated weights for policy 0, policy_version 1216065 (0.0007) [2023-12-27 14:04:28,997][20347] Updated weights for policy 0, policy_version 1216075 (0.0007) [2023-12-27 14:04:29,048][20347] Updated weights for policy 0, policy_version 1216085 (0.0007) [2023-12-27 14:04:29,053][20381] Updated weights for policy 1, policy_version 1215001 (0.0008) [2023-12-27 14:04:29,098][20347] Updated weights for policy 0, policy_version 1216095 (0.0005) [2023-12-27 14:04:29,104][20381] Updated weights for policy 1, policy_version 1215011 (0.0010) [2023-12-27 14:04:29,122][20292] KL-divergence is very high: 199.7752 [2023-12-27 14:04:29,136][20292] KL-divergence is very high: 200.3972 [2023-12-27 14:04:29,151][20381] Updated weights for policy 1, policy_version 1215021 (0.0010) [2023-12-27 14:04:29,158][20292] KL-divergence is very high: 110.2713 [2023-12-27 14:04:29,195][20381] Updated weights for policy 1, policy_version 1215031 (0.0010) [2023-12-27 14:04:29,813][20347] Updated weights for policy 0, policy_version 1216105 (0.0007) [2023-12-27 14:04:29,872][20381] Updated weights for policy 1, policy_version 1215041 (0.0010) [2023-12-27 14:04:29,874][20347] Updated weights for policy 0, policy_version 1216115 (0.0008) [2023-12-27 14:04:29,934][20347] Updated weights for policy 0, policy_version 1216125 (0.0009) [2023-12-27 14:04:29,935][20381] Updated weights for policy 1, policy_version 1215051 (0.0009) [2023-12-27 14:04:29,991][20381] Updated weights for policy 1, policy_version 1215061 (0.0009) [2023-12-27 14:04:30,535][20347] Updated weights for policy 0, policy_version 1216135 (0.0006) [2023-12-27 14:04:30,587][20347] Updated weights for policy 0, policy_version 1216145 (0.0005) [2023-12-27 14:04:30,641][20347] Updated weights for policy 0, policy_version 1216155 (0.0005) [2023-12-27 14:04:30,691][20381] Updated weights for policy 1, policy_version 1215071 (0.0006) [2023-12-27 14:04:30,750][20381] Updated weights for policy 1, policy_version 1215081 (0.0008) [2023-12-27 14:04:30,806][20381] Updated weights for policy 1, policy_version 1215091 (0.0008) [2023-12-27 14:04:31,229][20347] Updated weights for policy 0, policy_version 1216165 (0.0011) [2023-12-27 14:04:31,291][20347] Updated weights for policy 0, policy_version 1216175 (0.0010) [2023-12-27 14:04:31,350][20347] Updated weights for policy 0, policy_version 1216185 (0.0008) [2023-12-27 14:04:31,550][20381] Updated weights for policy 1, policy_version 1215101 (0.0009) [2023-12-27 14:04:31,605][20381] Updated weights for policy 1, policy_version 1215111 (0.0008) [2023-12-27 14:04:31,670][20381] Updated weights for policy 1, policy_version 1215121 (0.0008) [2023-12-27 14:04:32,076][20347] Updated weights for policy 0, policy_version 1216195 (0.0008) [2023-12-27 14:04:32,135][20347] Updated weights for policy 0, policy_version 1216205 (0.0007) [2023-12-27 14:04:32,200][20347] Updated weights for policy 0, policy_version 1216215 (0.0007) [2023-12-27 14:04:32,261][20381] Updated weights for policy 1, policy_version 1215131 (0.0008) [2023-12-27 14:04:32,324][20381] Updated weights for policy 1, policy_version 1215141 (0.0009) [2023-12-27 14:04:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 622510080. Throughput: 0: 10154.9, 1: 10445.8. Samples: 622520532. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:32,363][19170] Avg episode reward: [(0, '446.430'), (1, '931.650')] [2023-12-27 14:04:32,386][20381] Updated weights for policy 1, policy_version 1215151 (0.0006) [2023-12-27 14:04:32,848][20347] Updated weights for policy 0, policy_version 1216225 (0.0007) [2023-12-27 14:04:32,915][20347] Updated weights for policy 0, policy_version 1216235 (0.0005) [2023-12-27 14:04:32,974][20347] Updated weights for policy 0, policy_version 1216245 (0.0006) [2023-12-27 14:04:32,975][20381] Updated weights for policy 1, policy_version 1215161 (0.0006) [2023-12-27 14:04:33,027][20347] Updated weights for policy 0, policy_version 1216255 (0.0006) [2023-12-27 14:04:33,042][20381] Updated weights for policy 1, policy_version 1215171 (0.0006) [2023-12-27 14:04:33,104][20381] Updated weights for policy 1, policy_version 1215181 (0.0005) [2023-12-27 14:04:33,165][20381] Updated weights for policy 1, policy_version 1215191 (0.0008) [2023-12-27 14:04:33,670][20347] Updated weights for policy 0, policy_version 1216265 (0.0007) [2023-12-27 14:04:33,720][20347] Updated weights for policy 0, policy_version 1216275 (0.0007) [2023-12-27 14:04:33,775][20347] Updated weights for policy 0, policy_version 1216285 (0.0006) [2023-12-27 14:04:33,788][20381] Updated weights for policy 1, policy_version 1215201 (0.0010) [2023-12-27 14:04:33,843][20381] Updated weights for policy 1, policy_version 1215211 (0.0010) [2023-12-27 14:04:33,888][20381] Updated weights for policy 1, policy_version 1215221 (0.0010) [2023-12-27 14:04:34,479][20347] Updated weights for policy 0, policy_version 1216295 (0.0007) [2023-12-27 14:04:34,540][20347] Updated weights for policy 0, policy_version 1216305 (0.0006) [2023-12-27 14:04:34,600][20381] Updated weights for policy 1, policy_version 1215231 (0.0007) [2023-12-27 14:04:34,601][20347] Updated weights for policy 0, policy_version 1216315 (0.0006) [2023-12-27 14:04:34,663][20381] Updated weights for policy 1, policy_version 1215241 (0.0009) [2023-12-27 14:04:34,723][20381] Updated weights for policy 1, policy_version 1215251 (0.0011) [2023-12-27 14:04:35,261][20347] Updated weights for policy 0, policy_version 1216325 (0.0007) [2023-12-27 14:04:35,321][20347] Updated weights for policy 0, policy_version 1216335 (0.0008) [2023-12-27 14:04:35,333][20381] Updated weights for policy 1, policy_version 1215261 (0.0008) [2023-12-27 14:04:35,374][20347] Updated weights for policy 0, policy_version 1216345 (0.0005) [2023-12-27 14:04:35,380][20381] Updated weights for policy 1, policy_version 1215271 (0.0008) [2023-12-27 14:04:35,429][20381] Updated weights for policy 1, policy_version 1215281 (0.0011) [2023-12-27 14:04:36,044][20347] Updated weights for policy 0, policy_version 1216355 (0.0008) [2023-12-27 14:04:36,102][20347] Updated weights for policy 0, policy_version 1216365 (0.0010) [2023-12-27 14:04:36,109][20381] Updated weights for policy 1, policy_version 1215291 (0.0010) [2023-12-27 14:04:36,166][20381] Updated weights for policy 1, policy_version 1215301 (0.0008) [2023-12-27 14:04:36,168][20347] Updated weights for policy 0, policy_version 1216375 (0.0010) [2023-12-27 14:04:36,223][20381] Updated weights for policy 1, policy_version 1215311 (0.0006) [2023-12-27 14:04:36,861][20347] Updated weights for policy 0, policy_version 1216385 (0.0010) [2023-12-27 14:04:36,863][20381] Updated weights for policy 1, policy_version 1215321 (0.0010) [2023-12-27 14:04:36,915][20347] Updated weights for policy 0, policy_version 1216395 (0.0010) [2023-12-27 14:04:36,929][20381] Updated weights for policy 1, policy_version 1215331 (0.0011) [2023-12-27 14:04:36,980][20347] Updated weights for policy 0, policy_version 1216405 (0.0010) [2023-12-27 14:04:36,992][20381] Updated weights for policy 1, policy_version 1215341 (0.0010) [2023-12-27 14:04:37,034][20347] Updated weights for policy 0, policy_version 1216415 (0.0008) [2023-12-27 14:04:37,057][20381] Updated weights for policy 1, policy_version 1215351 (0.0011) [2023-12-27 14:04:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 622616576. Throughput: 0: 10209.1, 1: 10441.1. Samples: 622583624. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:37,363][19170] Avg episode reward: [(0, '542.620'), (1, '941.280')] [2023-12-27 14:04:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001215352_311156736.pth... [2023-12-27 14:04:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001216416_311459840.pth... [2023-12-27 14:04:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001214136_310845440.pth [2023-12-27 14:04:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001215232_311156736.pth [2023-12-27 14:04:37,602][20347] Updated weights for policy 0, policy_version 1216425 (0.0005) [2023-12-27 14:04:37,661][20347] Updated weights for policy 0, policy_version 1216435 (0.0005) [2023-12-27 14:04:37,718][20347] Updated weights for policy 0, policy_version 1216445 (0.0005) [2023-12-27 14:04:37,728][20381] Updated weights for policy 1, policy_version 1215361 (0.0006) [2023-12-27 14:04:37,789][20381] Updated weights for policy 1, policy_version 1215371 (0.0006) [2023-12-27 14:04:37,840][20381] Updated weights for policy 1, policy_version 1215381 (0.0007) [2023-12-27 14:04:38,328][20347] Updated weights for policy 0, policy_version 1216455 (0.0009) [2023-12-27 14:04:38,388][20347] Updated weights for policy 0, policy_version 1216465 (0.0007) [2023-12-27 14:04:38,441][20347] Updated weights for policy 0, policy_version 1216475 (0.0010) [2023-12-27 14:04:38,483][20381] Updated weights for policy 1, policy_version 1215391 (0.0009) [2023-12-27 14:04:38,538][20381] Updated weights for policy 1, policy_version 1215401 (0.0007) [2023-12-27 14:04:38,595][20381] Updated weights for policy 1, policy_version 1215411 (0.0008) [2023-12-27 14:04:39,157][20347] Updated weights for policy 0, policy_version 1216485 (0.0009) [2023-12-27 14:04:39,201][20381] Updated weights for policy 1, policy_version 1215421 (0.0007) [2023-12-27 14:04:39,216][20347] Updated weights for policy 0, policy_version 1216495 (0.0008) [2023-12-27 14:04:39,265][20381] Updated weights for policy 1, policy_version 1215431 (0.0008) [2023-12-27 14:04:39,278][20347] Updated weights for policy 0, policy_version 1216505 (0.0008) [2023-12-27 14:04:39,332][20381] Updated weights for policy 1, policy_version 1215441 (0.0007) [2023-12-27 14:04:39,974][20347] Updated weights for policy 0, policy_version 1216515 (0.0007) [2023-12-27 14:04:40,036][20347] Updated weights for policy 0, policy_version 1216525 (0.0007) [2023-12-27 14:04:40,062][20381] Updated weights for policy 1, policy_version 1215451 (0.0007) [2023-12-27 14:04:40,094][20347] Updated weights for policy 0, policy_version 1216535 (0.0007) [2023-12-27 14:04:40,116][20381] Updated weights for policy 1, policy_version 1215461 (0.0006) [2023-12-27 14:04:40,172][20381] Updated weights for policy 1, policy_version 1215471 (0.0008) [2023-12-27 14:04:40,735][20347] Updated weights for policy 0, policy_version 1216545 (0.0006) [2023-12-27 14:04:40,787][20347] Updated weights for policy 0, policy_version 1216555 (0.0007) [2023-12-27 14:04:40,835][20347] Updated weights for policy 0, policy_version 1216565 (0.0008) [2023-12-27 14:04:40,890][20347] Updated weights for policy 0, policy_version 1216575 (0.0008) [2023-12-27 14:04:40,891][20381] Updated weights for policy 1, policy_version 1215481 (0.0008) [2023-12-27 14:04:40,946][20381] Updated weights for policy 1, policy_version 1215491 (0.0008) [2023-12-27 14:04:41,001][20381] Updated weights for policy 1, policy_version 1215501 (0.0007) [2023-12-27 14:04:41,066][20381] Updated weights for policy 1, policy_version 1215511 (0.0009) [2023-12-27 14:04:41,622][20347] Updated weights for policy 0, policy_version 1216585 (0.0007) [2023-12-27 14:04:41,686][20347] Updated weights for policy 0, policy_version 1216595 (0.0008) [2023-12-27 14:04:41,754][20347] Updated weights for policy 0, policy_version 1216605 (0.0008) [2023-12-27 14:04:41,843][20381] Updated weights for policy 1, policy_version 1215521 (0.0008) [2023-12-27 14:04:41,892][20381] Updated weights for policy 1, policy_version 1215531 (0.0007) [2023-12-27 14:04:41,951][20381] Updated weights for policy 1, policy_version 1215541 (0.0008) [2023-12-27 14:04:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 622714880. Throughput: 0: 10247.0, 1: 10384.6. Samples: 622707316. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:42,363][19170] Avg episode reward: [(0, '496.600'), (1, '951.160')] [2023-12-27 14:04:42,459][20347] Updated weights for policy 0, policy_version 1216615 (0.0008) [2023-12-27 14:04:42,513][20347] Updated weights for policy 0, policy_version 1216625 (0.0006) [2023-12-27 14:04:42,582][20347] Updated weights for policy 0, policy_version 1216635 (0.0006) [2023-12-27 14:04:42,711][20381] Updated weights for policy 1, policy_version 1215551 (0.0010) [2023-12-27 14:04:42,776][20381] Updated weights for policy 1, policy_version 1215561 (0.0011) [2023-12-27 14:04:42,832][20381] Updated weights for policy 1, policy_version 1215571 (0.0011) [2023-12-27 14:04:43,144][20347] Updated weights for policy 0, policy_version 1216645 (0.0006) [2023-12-27 14:04:43,195][20347] Updated weights for policy 0, policy_version 1216655 (0.0006) [2023-12-27 14:04:43,249][20347] Updated weights for policy 0, policy_version 1216665 (0.0006) [2023-12-27 14:04:43,508][20381] Updated weights for policy 1, policy_version 1215581 (0.0009) [2023-12-27 14:04:43,562][20381] Updated weights for policy 1, policy_version 1215591 (0.0007) [2023-12-27 14:04:43,617][20381] Updated weights for policy 1, policy_version 1215601 (0.0005) [2023-12-27 14:04:43,861][20347] Updated weights for policy 0, policy_version 1216675 (0.0006) [2023-12-27 14:04:43,920][20347] Updated weights for policy 0, policy_version 1216685 (0.0008) [2023-12-27 14:04:43,978][20347] Updated weights for policy 0, policy_version 1216695 (0.0008) [2023-12-27 14:04:44,253][20381] Updated weights for policy 1, policy_version 1215611 (0.0006) [2023-12-27 14:04:44,312][20381] Updated weights for policy 1, policy_version 1215621 (0.0008) [2023-12-27 14:04:44,361][20381] Updated weights for policy 1, policy_version 1215631 (0.0008) [2023-12-27 14:04:44,654][20347] Updated weights for policy 0, policy_version 1216705 (0.0008) [2023-12-27 14:04:44,720][20347] Updated weights for policy 0, policy_version 1216715 (0.0005) [2023-12-27 14:04:44,782][20347] Updated weights for policy 0, policy_version 1216725 (0.0006) [2023-12-27 14:04:44,840][20347] Updated weights for policy 0, policy_version 1216735 (0.0007) [2023-12-27 14:04:45,081][20381] Updated weights for policy 1, policy_version 1215641 (0.0007) [2023-12-27 14:04:45,143][20381] Updated weights for policy 1, policy_version 1215651 (0.0006) [2023-12-27 14:04:45,203][20381] Updated weights for policy 1, policy_version 1215661 (0.0005) [2023-12-27 14:04:45,269][20381] Updated weights for policy 1, policy_version 1215671 (0.0008) [2023-12-27 14:04:45,504][20347] Updated weights for policy 0, policy_version 1216745 (0.0008) [2023-12-27 14:04:45,551][20347] Updated weights for policy 0, policy_version 1216755 (0.0008) [2023-12-27 14:04:45,592][20347] Updated weights for policy 0, policy_version 1216765 (0.0007) [2023-12-27 14:04:45,878][20381] Updated weights for policy 1, policy_version 1215681 (0.0006) [2023-12-27 14:04:45,937][20381] Updated weights for policy 1, policy_version 1215691 (0.0007) [2023-12-27 14:04:45,989][20381] Updated weights for policy 1, policy_version 1215701 (0.0008) [2023-12-27 14:04:46,302][20347] Updated weights for policy 0, policy_version 1216775 (0.0007) [2023-12-27 14:04:46,352][20347] Updated weights for policy 0, policy_version 1216785 (0.0007) [2023-12-27 14:04:46,401][20347] Updated weights for policy 0, policy_version 1216795 (0.0007) [2023-12-27 14:04:46,663][20381] Updated weights for policy 1, policy_version 1215711 (0.0008) [2023-12-27 14:04:46,725][20381] Updated weights for policy 1, policy_version 1215721 (0.0008) [2023-12-27 14:04:46,790][20381] Updated weights for policy 1, policy_version 1215731 (0.0008) [2023-12-27 14:04:47,081][20347] Updated weights for policy 0, policy_version 1216805 (0.0009) [2023-12-27 14:04:47,145][20347] Updated weights for policy 0, policy_version 1216815 (0.0010) [2023-12-27 14:04:47,203][20347] Updated weights for policy 0, policy_version 1216825 (0.0010) [2023-12-27 14:04:47,357][20381] Updated weights for policy 1, policy_version 1215741 (0.0008) [2023-12-27 14:04:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 622821376. Throughput: 0: 10318.8, 1: 10332.3. Samples: 622830816. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:47,363][19170] Avg episode reward: [(0, '624.290'), (1, '921.760')] [2023-12-27 14:04:47,419][20381] Updated weights for policy 1, policy_version 1215751 (0.0009) [2023-12-27 14:04:47,478][20381] Updated weights for policy 1, policy_version 1215761 (0.0008) [2023-12-27 14:04:47,791][20347] Updated weights for policy 0, policy_version 1216835 (0.0008) [2023-12-27 14:04:47,845][20347] Updated weights for policy 0, policy_version 1216845 (0.0005) [2023-12-27 14:04:47,897][20347] Updated weights for policy 0, policy_version 1216855 (0.0005) [2023-12-27 14:04:48,166][20381] Updated weights for policy 1, policy_version 1215771 (0.0007) [2023-12-27 14:04:48,226][20381] Updated weights for policy 1, policy_version 1215781 (0.0007) [2023-12-27 14:04:48,276][20381] Updated weights for policy 1, policy_version 1215791 (0.0010) [2023-12-27 14:04:48,468][20347] Updated weights for policy 0, policy_version 1216865 (0.0005) [2023-12-27 14:04:48,533][20347] Updated weights for policy 0, policy_version 1216875 (0.0010) [2023-12-27 14:04:48,598][20347] Updated weights for policy 0, policy_version 1216885 (0.0010) [2023-12-27 14:04:48,660][20347] Updated weights for policy 0, policy_version 1216895 (0.0010) [2023-12-27 14:04:48,950][20381] Updated weights for policy 1, policy_version 1215801 (0.0010) [2023-12-27 14:04:49,009][20381] Updated weights for policy 1, policy_version 1215811 (0.0011) [2023-12-27 14:04:49,070][20381] Updated weights for policy 1, policy_version 1215821 (0.0006) [2023-12-27 14:04:49,122][20381] Updated weights for policy 1, policy_version 1215831 (0.0005) [2023-12-27 14:04:49,323][20347] Updated weights for policy 0, policy_version 1216905 (0.0008) [2023-12-27 14:04:49,390][20347] Updated weights for policy 0, policy_version 1216915 (0.0008) [2023-12-27 14:04:49,453][20347] Updated weights for policy 0, policy_version 1216925 (0.0008) [2023-12-27 14:04:49,861][20381] Updated weights for policy 1, policy_version 1215841 (0.0008) [2023-12-27 14:04:49,912][20381] Updated weights for policy 1, policy_version 1215851 (0.0007) [2023-12-27 14:04:49,981][20381] Updated weights for policy 1, policy_version 1215861 (0.0008) [2023-12-27 14:04:50,202][20347] Updated weights for policy 0, policy_version 1216935 (0.0009) [2023-12-27 14:04:50,266][20347] Updated weights for policy 0, policy_version 1216945 (0.0007) [2023-12-27 14:04:50,321][20347] Updated weights for policy 0, policy_version 1216955 (0.0007) [2023-12-27 14:04:50,660][20381] Updated weights for policy 1, policy_version 1215871 (0.0010) [2023-12-27 14:04:50,715][20381] Updated weights for policy 1, policy_version 1215881 (0.0010) [2023-12-27 14:04:50,768][20381] Updated weights for policy 1, policy_version 1215891 (0.0011) [2023-12-27 14:04:51,039][20347] Updated weights for policy 0, policy_version 1216965 (0.0007) [2023-12-27 14:04:51,101][20347] Updated weights for policy 0, policy_version 1216975 (0.0008) [2023-12-27 14:04:51,157][20347] Updated weights for policy 0, policy_version 1216985 (0.0008) [2023-12-27 14:04:51,378][20381] Updated weights for policy 1, policy_version 1215901 (0.0009) [2023-12-27 14:04:51,441][20381] Updated weights for policy 1, policy_version 1215911 (0.0008) [2023-12-27 14:04:51,497][20381] Updated weights for policy 1, policy_version 1215921 (0.0011) [2023-12-27 14:04:51,939][20347] Updated weights for policy 0, policy_version 1216995 (0.0007) [2023-12-27 14:04:51,999][20347] Updated weights for policy 0, policy_version 1217005 (0.0007) [2023-12-27 14:04:52,056][20347] Updated weights for policy 0, policy_version 1217015 (0.0006) [2023-12-27 14:04:52,216][20381] Updated weights for policy 1, policy_version 1215931 (0.0010) [2023-12-27 14:04:52,274][20381] Updated weights for policy 1, policy_version 1215941 (0.0008) [2023-12-27 14:04:52,338][20381] Updated weights for policy 1, policy_version 1215951 (0.0007) [2023-12-27 14:04:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 622919680. Throughput: 0: 10326.0, 1: 10263.7. Samples: 622892080. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:52,363][19170] Avg episode reward: [(0, '752.650'), (1, '872.830')] [2023-12-27 14:04:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001217024_311615488.pth... [2023-12-27 14:04:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001215808_311304192.pth [2023-12-27 14:04:52,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001215960_311312384.pth... [2023-12-27 14:04:52,401][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001214744_311001088.pth [2023-12-27 14:04:52,660][20347] Updated weights for policy 0, policy_version 1217025 (0.0006) [2023-12-27 14:04:52,722][20347] Updated weights for policy 0, policy_version 1217035 (0.0006) [2023-12-27 14:04:52,784][20347] Updated weights for policy 0, policy_version 1217045 (0.0006) [2023-12-27 14:04:52,840][20347] Updated weights for policy 0, policy_version 1217055 (0.0006) [2023-12-27 14:04:53,061][20381] Updated weights for policy 1, policy_version 1215961 (0.0008) [2023-12-27 14:04:53,111][20381] Updated weights for policy 1, policy_version 1215971 (0.0009) [2023-12-27 14:04:53,175][20381] Updated weights for policy 1, policy_version 1215981 (0.0009) [2023-12-27 14:04:53,233][20381] Updated weights for policy 1, policy_version 1215991 (0.0008) [2023-12-27 14:04:53,411][20347] Updated weights for policy 0, policy_version 1217065 (0.0007) [2023-12-27 14:04:53,466][20347] Updated weights for policy 0, policy_version 1217075 (0.0006) [2023-12-27 14:04:53,523][20347] Updated weights for policy 0, policy_version 1217085 (0.0007) [2023-12-27 14:04:53,922][20381] Updated weights for policy 1, policy_version 1216001 (0.0008) [2023-12-27 14:04:53,976][20381] Updated weights for policy 1, policy_version 1216011 (0.0007) [2023-12-27 14:04:54,027][20381] Updated weights for policy 1, policy_version 1216021 (0.0007) [2023-12-27 14:04:54,187][20347] Updated weights for policy 0, policy_version 1217095 (0.0007) [2023-12-27 14:04:54,238][20347] Updated weights for policy 0, policy_version 1217105 (0.0010) [2023-12-27 14:04:54,306][20347] Updated weights for policy 0, policy_version 1217115 (0.0011) [2023-12-27 14:04:54,616][20381] Updated weights for policy 1, policy_version 1216031 (0.0007) [2023-12-27 14:04:54,675][20381] Updated weights for policy 1, policy_version 1216041 (0.0007) [2023-12-27 14:04:54,730][20381] Updated weights for policy 1, policy_version 1216051 (0.0007) [2023-12-27 14:04:54,963][20347] Updated weights for policy 0, policy_version 1217125 (0.0010) [2023-12-27 14:04:55,023][20347] Updated weights for policy 0, policy_version 1217135 (0.0007) [2023-12-27 14:04:55,078][20347] Updated weights for policy 0, policy_version 1217145 (0.0010) [2023-12-27 14:04:55,407][20381] Updated weights for policy 1, policy_version 1216061 (0.0007) [2023-12-27 14:04:55,464][20381] Updated weights for policy 1, policy_version 1216071 (0.0007) [2023-12-27 14:04:55,524][20381] Updated weights for policy 1, policy_version 1216081 (0.0007) [2023-12-27 14:04:55,712][20347] Updated weights for policy 0, policy_version 1217155 (0.0009) [2023-12-27 14:04:55,769][20347] Updated weights for policy 0, policy_version 1217165 (0.0010) [2023-12-27 14:04:55,822][20347] Updated weights for policy 0, policy_version 1217175 (0.0010) [2023-12-27 14:04:56,218][20381] Updated weights for policy 1, policy_version 1216091 (0.0008) [2023-12-27 14:04:56,282][20381] Updated weights for policy 1, policy_version 1216101 (0.0008) [2023-12-27 14:04:56,353][20381] Updated weights for policy 1, policy_version 1216111 (0.0009) [2023-12-27 14:04:56,442][20347] Updated weights for policy 0, policy_version 1217185 (0.0010) [2023-12-27 14:04:56,497][20347] Updated weights for policy 0, policy_version 1217195 (0.0005) [2023-12-27 14:04:56,550][20347] Updated weights for policy 0, policy_version 1217205 (0.0005) [2023-12-27 14:04:56,604][20347] Updated weights for policy 0, policy_version 1217215 (0.0006) [2023-12-27 14:04:57,046][20381] Updated weights for policy 1, policy_version 1216121 (0.0008) [2023-12-27 14:04:57,097][20381] Updated weights for policy 1, policy_version 1216131 (0.0008) [2023-12-27 14:04:57,142][20381] Updated weights for policy 1, policy_version 1216141 (0.0008) [2023-12-27 14:04:57,195][20381] Updated weights for policy 1, policy_version 1216151 (0.0007) [2023-12-27 14:04:57,233][20347] Updated weights for policy 0, policy_version 1217225 (0.0008) [2023-12-27 14:04:57,281][20347] Updated weights for policy 0, policy_version 1217235 (0.0008) [2023-12-27 14:04:57,337][20347] Updated weights for policy 0, policy_version 1217245 (0.0007) [2023-12-27 14:04:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 623034368. Throughput: 0: 10322.6, 1: 10281.0. Samples: 623016020. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:04:57,363][19170] Avg episode reward: [(0, '541.730'), (1, '872.840')] [2023-12-27 14:04:57,876][20381] Updated weights for policy 1, policy_version 1216161 (0.0008) [2023-12-27 14:04:57,932][20381] Updated weights for policy 1, policy_version 1216171 (0.0007) [2023-12-27 14:04:57,976][20381] Updated weights for policy 1, policy_version 1216181 (0.0007) [2023-12-27 14:04:58,027][20347] Updated weights for policy 0, policy_version 1217255 (0.0008) [2023-12-27 14:04:58,072][20347] Updated weights for policy 0, policy_version 1217265 (0.0007) [2023-12-27 14:04:58,116][20347] Updated weights for policy 0, policy_version 1217275 (0.0007) [2023-12-27 14:04:58,744][20381] Updated weights for policy 1, policy_version 1216191 (0.0008) [2023-12-27 14:04:58,808][20381] Updated weights for policy 1, policy_version 1216201 (0.0008) [2023-12-27 14:04:58,876][20381] Updated weights for policy 1, policy_version 1216211 (0.0008) [2023-12-27 14:04:58,938][20347] Updated weights for policy 0, policy_version 1217285 (0.0007) [2023-12-27 14:04:58,990][20347] Updated weights for policy 0, policy_version 1217295 (0.0007) [2023-12-27 14:04:59,052][20347] Updated weights for policy 0, policy_version 1217305 (0.0007) [2023-12-27 14:04:59,700][20381] Updated weights for policy 1, policy_version 1216221 (0.0007) [2023-12-27 14:04:59,762][20381] Updated weights for policy 1, policy_version 1216231 (0.0005) [2023-12-27 14:04:59,797][20347] Updated weights for policy 0, policy_version 1217315 (0.0008) [2023-12-27 14:04:59,822][20381] Updated weights for policy 1, policy_version 1216241 (0.0007) [2023-12-27 14:04:59,859][20347] Updated weights for policy 0, policy_version 1217325 (0.0011) [2023-12-27 14:04:59,913][20347] Updated weights for policy 0, policy_version 1217335 (0.0010) [2023-12-27 14:05:00,417][20381] Updated weights for policy 1, policy_version 1216251 (0.0008) [2023-12-27 14:05:00,483][20381] Updated weights for policy 1, policy_version 1216261 (0.0007) [2023-12-27 14:05:00,535][20381] Updated weights for policy 1, policy_version 1216271 (0.0005) [2023-12-27 14:05:00,611][20347] Updated weights for policy 0, policy_version 1217345 (0.0010) [2023-12-27 14:05:00,658][20347] Updated weights for policy 0, policy_version 1217355 (0.0006) [2023-12-27 14:05:00,714][20347] Updated weights for policy 0, policy_version 1217365 (0.0007) [2023-12-27 14:05:00,762][20347] Updated weights for policy 0, policy_version 1217375 (0.0008) [2023-12-27 14:05:01,083][20381] Updated weights for policy 1, policy_version 1216281 (0.0006) [2023-12-27 14:05:01,145][20381] Updated weights for policy 1, policy_version 1216291 (0.0007) [2023-12-27 14:05:01,206][20381] Updated weights for policy 1, policy_version 1216301 (0.0008) [2023-12-27 14:05:01,262][20381] Updated weights for policy 1, policy_version 1216311 (0.0008) [2023-12-27 14:05:01,491][20347] Updated weights for policy 0, policy_version 1217385 (0.0008) [2023-12-27 14:05:01,542][20347] Updated weights for policy 0, policy_version 1217395 (0.0008) [2023-12-27 14:05:01,591][20347] Updated weights for policy 0, policy_version 1217405 (0.0007) [2023-12-27 14:05:01,870][20381] Updated weights for policy 1, policy_version 1216321 (0.0006) [2023-12-27 14:05:01,925][20381] Updated weights for policy 1, policy_version 1216331 (0.0005) [2023-12-27 14:05:01,980][20381] Updated weights for policy 1, policy_version 1216341 (0.0006) [2023-12-27 14:05:02,301][20347] Updated weights for policy 0, policy_version 1217415 (0.0008) [2023-12-27 14:05:02,354][20347] Updated weights for policy 0, policy_version 1217425 (0.0008) [2023-12-27 14:05:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 623124480. Throughput: 0: 10277.5, 1: 10285.6. Samples: 623137696. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:02,363][19170] Avg episode reward: [(0, '567.090'), (1, '872.990')] [2023-12-27 14:05:02,413][20347] Updated weights for policy 0, policy_version 1217435 (0.0008) [2023-12-27 14:05:02,650][20381] Updated weights for policy 1, policy_version 1216351 (0.0010) [2023-12-27 14:05:02,710][20381] Updated weights for policy 1, policy_version 1216361 (0.0011) [2023-12-27 14:05:02,773][20381] Updated weights for policy 1, policy_version 1216371 (0.0011) [2023-12-27 14:05:03,094][20347] Updated weights for policy 0, policy_version 1217445 (0.0009) [2023-12-27 14:05:03,151][20347] Updated weights for policy 0, policy_version 1217455 (0.0010) [2023-12-27 14:05:03,205][20347] Updated weights for policy 0, policy_version 1217465 (0.0010) [2023-12-27 14:05:03,452][20381] Updated weights for policy 1, policy_version 1216381 (0.0011) [2023-12-27 14:05:03,500][20381] Updated weights for policy 1, policy_version 1216391 (0.0010) [2023-12-27 14:05:03,544][20381] Updated weights for policy 1, policy_version 1216401 (0.0010) [2023-12-27 14:05:03,875][20347] Updated weights for policy 0, policy_version 1217475 (0.0010) [2023-12-27 14:05:03,929][20347] Updated weights for policy 0, policy_version 1217485 (0.0010) [2023-12-27 14:05:03,988][20347] Updated weights for policy 0, policy_version 1217495 (0.0011) [2023-12-27 14:05:04,254][20381] Updated weights for policy 1, policy_version 1216411 (0.0010) [2023-12-27 14:05:04,322][20381] Updated weights for policy 1, policy_version 1216421 (0.0009) [2023-12-27 14:05:04,384][20381] Updated weights for policy 1, policy_version 1216431 (0.0010) [2023-12-27 14:05:04,619][20347] Updated weights for policy 0, policy_version 1217505 (0.0010) [2023-12-27 14:05:04,667][20347] Updated weights for policy 0, policy_version 1217515 (0.0010) [2023-12-27 14:05:04,712][20347] Updated weights for policy 0, policy_version 1217525 (0.0010) [2023-12-27 14:05:04,760][20347] Updated weights for policy 0, policy_version 1217535 (0.0010) [2023-12-27 14:05:05,069][20381] Updated weights for policy 1, policy_version 1216441 (0.0010) [2023-12-27 14:05:05,122][20381] Updated weights for policy 1, policy_version 1216451 (0.0009) [2023-12-27 14:05:05,183][20381] Updated weights for policy 1, policy_version 1216461 (0.0010) [2023-12-27 14:05:05,238][20381] Updated weights for policy 1, policy_version 1216471 (0.0011) [2023-12-27 14:05:05,399][20347] Updated weights for policy 0, policy_version 1217545 (0.0006) [2023-12-27 14:05:05,448][20347] Updated weights for policy 0, policy_version 1217555 (0.0007) [2023-12-27 14:05:05,503][20347] Updated weights for policy 0, policy_version 1217565 (0.0010) [2023-12-27 14:05:05,893][20381] Updated weights for policy 1, policy_version 1216481 (0.0008) [2023-12-27 14:05:05,950][20381] Updated weights for policy 1, policy_version 1216491 (0.0010) [2023-12-27 14:05:06,011][20381] Updated weights for policy 1, policy_version 1216501 (0.0010) [2023-12-27 14:05:06,097][20347] Updated weights for policy 0, policy_version 1217575 (0.0010) [2023-12-27 14:05:06,159][20347] Updated weights for policy 0, policy_version 1217585 (0.0010) [2023-12-27 14:05:06,221][20347] Updated weights for policy 0, policy_version 1217595 (0.0010) [2023-12-27 14:05:06,690][20381] Updated weights for policy 1, policy_version 1216511 (0.0008) [2023-12-27 14:05:06,744][20381] Updated weights for policy 1, policy_version 1216521 (0.0006) [2023-12-27 14:05:06,804][20381] Updated weights for policy 1, policy_version 1216531 (0.0009) [2023-12-27 14:05:06,833][20347] Updated weights for policy 0, policy_version 1217605 (0.0008) [2023-12-27 14:05:06,884][20347] Updated weights for policy 0, policy_version 1217615 (0.0007) [2023-12-27 14:05:06,934][20347] Updated weights for policy 0, policy_version 1217625 (0.0006) [2023-12-27 14:05:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 623230976. Throughput: 0: 10317.2, 1: 10298.9. Samples: 623199408. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:07,364][19170] Avg episode reward: [(0, '658.900'), (1, '902.430')] [2023-12-27 14:05:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001216536_311459840.pth... [2023-12-27 14:05:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001217632_311771136.pth... [2023-12-27 14:05:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001215352_311156736.pth [2023-12-27 14:05:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001216416_311459840.pth [2023-12-27 14:05:07,517][20381] Updated weights for policy 1, policy_version 1216541 (0.0008) [2023-12-27 14:05:07,541][20347] Updated weights for policy 0, policy_version 1217635 (0.0010) [2023-12-27 14:05:07,572][20381] Updated weights for policy 1, policy_version 1216551 (0.0006) [2023-12-27 14:05:07,602][20347] Updated weights for policy 0, policy_version 1217645 (0.0010) [2023-12-27 14:05:07,631][20381] Updated weights for policy 1, policy_version 1216561 (0.0008) [2023-12-27 14:05:07,664][20347] Updated weights for policy 0, policy_version 1217655 (0.0010) [2023-12-27 14:05:08,337][20347] Updated weights for policy 0, policy_version 1217665 (0.0010) [2023-12-27 14:05:08,338][20381] Updated weights for policy 1, policy_version 1216571 (0.0006) [2023-12-27 14:05:08,402][20381] Updated weights for policy 1, policy_version 1216581 (0.0008) [2023-12-27 14:05:08,405][20347] Updated weights for policy 0, policy_version 1217675 (0.0009) [2023-12-27 14:05:08,463][20381] Updated weights for policy 1, policy_version 1216591 (0.0005) [2023-12-27 14:05:08,464][20347] Updated weights for policy 0, policy_version 1217685 (0.0007) [2023-12-27 14:05:08,519][20347] Updated weights for policy 0, policy_version 1217695 (0.0008) [2023-12-27 14:05:09,108][20347] Updated weights for policy 0, policy_version 1217705 (0.0010) [2023-12-27 14:05:09,139][20381] Updated weights for policy 1, policy_version 1216601 (0.0006) [2023-12-27 14:05:09,163][20347] Updated weights for policy 0, policy_version 1217715 (0.0010) [2023-12-27 14:05:09,190][20381] Updated weights for policy 1, policy_version 1216611 (0.0010) [2023-12-27 14:05:09,228][20347] Updated weights for policy 0, policy_version 1217725 (0.0009) [2023-12-27 14:05:09,252][20381] Updated weights for policy 1, policy_version 1216621 (0.0010) [2023-12-27 14:05:09,312][20381] Updated weights for policy 1, policy_version 1216631 (0.0007) [2023-12-27 14:05:09,992][20347] Updated weights for policy 0, policy_version 1217735 (0.0006) [2023-12-27 14:05:10,057][20381] Updated weights for policy 1, policy_version 1216641 (0.0008) [2023-12-27 14:05:10,060][20347] Updated weights for policy 0, policy_version 1217745 (0.0007) [2023-12-27 14:05:10,114][20381] Updated weights for policy 1, policy_version 1216651 (0.0007) [2023-12-27 14:05:10,116][20347] Updated weights for policy 0, policy_version 1217755 (0.0005) [2023-12-27 14:05:10,176][20381] Updated weights for policy 1, policy_version 1216661 (0.0009) [2023-12-27 14:05:10,804][20347] Updated weights for policy 0, policy_version 1217765 (0.0009) [2023-12-27 14:05:10,876][20381] Updated weights for policy 1, policy_version 1216671 (0.0006) [2023-12-27 14:05:10,890][20347] Updated weights for policy 0, policy_version 1217775 (0.0007) [2023-12-27 14:05:10,932][20381] Updated weights for policy 1, policy_version 1216681 (0.0005) [2023-12-27 14:05:10,943][20347] Updated weights for policy 0, policy_version 1217785 (0.0010) [2023-12-27 14:05:10,986][20381] Updated weights for policy 1, policy_version 1216691 (0.0006) [2023-12-27 14:05:11,623][20347] Updated weights for policy 0, policy_version 1217795 (0.0010) [2023-12-27 14:05:11,693][20347] Updated weights for policy 0, policy_version 1217805 (0.0008) [2023-12-27 14:05:11,726][20381] Updated weights for policy 1, policy_version 1216701 (0.0007) [2023-12-27 14:05:11,757][20347] Updated weights for policy 0, policy_version 1217815 (0.0011) [2023-12-27 14:05:11,786][20381] Updated weights for policy 1, policy_version 1216711 (0.0006) [2023-12-27 14:05:11,848][20381] Updated weights for policy 1, policy_version 1216721 (0.0006) [2023-12-27 14:05:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 623329280. Throughput: 0: 10361.3, 1: 10269.1. Samples: 623322524. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:12,363][19170] Avg episode reward: [(0, '462.990'), (1, '902.290')] [2023-12-27 14:05:12,490][20347] Updated weights for policy 0, policy_version 1217825 (0.0008) [2023-12-27 14:05:12,532][20381] Updated weights for policy 1, policy_version 1216731 (0.0006) [2023-12-27 14:05:12,546][20347] Updated weights for policy 0, policy_version 1217835 (0.0010) [2023-12-27 14:05:12,589][20381] Updated weights for policy 1, policy_version 1216741 (0.0005) [2023-12-27 14:05:12,602][20347] Updated weights for policy 0, policy_version 1217845 (0.0010) [2023-12-27 14:05:12,645][20381] Updated weights for policy 1, policy_version 1216751 (0.0006) [2023-12-27 14:05:12,663][20347] Updated weights for policy 0, policy_version 1217855 (0.0008) [2023-12-27 14:05:13,331][20347] Updated weights for policy 0, policy_version 1217865 (0.0006) [2023-12-27 14:05:13,333][20381] Updated weights for policy 1, policy_version 1216761 (0.0007) [2023-12-27 14:05:13,383][20347] Updated weights for policy 0, policy_version 1217875 (0.0005) [2023-12-27 14:05:13,389][20381] Updated weights for policy 1, policy_version 1216771 (0.0005) [2023-12-27 14:05:13,436][20347] Updated weights for policy 0, policy_version 1217885 (0.0006) [2023-12-27 14:05:13,442][20381] Updated weights for policy 1, policy_version 1216781 (0.0006) [2023-12-27 14:05:13,501][20381] Updated weights for policy 1, policy_version 1216791 (0.0006) [2023-12-27 14:05:14,086][20381] Updated weights for policy 1, policy_version 1216801 (0.0005) [2023-12-27 14:05:14,109][20347] Updated weights for policy 0, policy_version 1217895 (0.0009) [2023-12-27 14:05:14,138][20381] Updated weights for policy 1, policy_version 1216811 (0.0008) [2023-12-27 14:05:14,159][20347] Updated weights for policy 0, policy_version 1217905 (0.0010) [2023-12-27 14:05:14,193][20381] Updated weights for policy 1, policy_version 1216821 (0.0010) [2023-12-27 14:05:14,204][20347] Updated weights for policy 0, policy_version 1217915 (0.0010) [2023-12-27 14:05:14,770][20381] Updated weights for policy 1, policy_version 1216831 (0.0010) [2023-12-27 14:05:14,829][20347] Updated weights for policy 0, policy_version 1217925 (0.0008) [2023-12-27 14:05:14,835][20381] Updated weights for policy 1, policy_version 1216841 (0.0009) [2023-12-27 14:05:14,885][20347] Updated weights for policy 0, policy_version 1217935 (0.0007) [2023-12-27 14:05:14,894][20381] Updated weights for policy 1, policy_version 1216851 (0.0008) [2023-12-27 14:05:14,952][20347] Updated weights for policy 0, policy_version 1217945 (0.0009) [2023-12-27 14:05:15,586][20381] Updated weights for policy 1, policy_version 1216861 (0.0009) [2023-12-27 14:05:15,592][20347] Updated weights for policy 0, policy_version 1217955 (0.0007) [2023-12-27 14:05:15,637][20381] Updated weights for policy 1, policy_version 1216871 (0.0010) [2023-12-27 14:05:15,653][20347] Updated weights for policy 0, policy_version 1217965 (0.0006) [2023-12-27 14:05:15,689][20381] Updated weights for policy 1, policy_version 1216881 (0.0010) [2023-12-27 14:05:15,713][20347] Updated weights for policy 0, policy_version 1217975 (0.0007) [2023-12-27 14:05:16,314][20347] Updated weights for policy 0, policy_version 1217985 (0.0006) [2023-12-27 14:05:16,370][20347] Updated weights for policy 0, policy_version 1217995 (0.0005) [2023-12-27 14:05:16,374][20381] Updated weights for policy 1, policy_version 1216891 (0.0010) [2023-12-27 14:05:16,420][20347] Updated weights for policy 0, policy_version 1218005 (0.0009) [2023-12-27 14:05:16,426][20381] Updated weights for policy 1, policy_version 1216901 (0.0006) [2023-12-27 14:05:16,475][20347] Updated weights for policy 0, policy_version 1218015 (0.0010) [2023-12-27 14:05:16,475][20381] Updated weights for policy 1, policy_version 1216911 (0.0006) [2023-12-27 14:05:17,073][20347] Updated weights for policy 0, policy_version 1218025 (0.0010) [2023-12-27 14:05:17,129][20347] Updated weights for policy 0, policy_version 1218035 (0.0006) [2023-12-27 14:05:17,154][20381] Updated weights for policy 1, policy_version 1216921 (0.0006) [2023-12-27 14:05:17,181][20347] Updated weights for policy 0, policy_version 1218045 (0.0006) [2023-12-27 14:05:17,200][20381] Updated weights for policy 1, policy_version 1216931 (0.0007) [2023-12-27 14:05:17,247][20381] Updated weights for policy 1, policy_version 1216941 (0.0006) [2023-12-27 14:05:17,291][20381] Updated weights for policy 1, policy_version 1216951 (0.0005) [2023-12-27 14:05:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 623443968. Throughput: 0: 10394.6, 1: 10229.8. Samples: 623448628. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:17,363][19170] Avg episode reward: [(0, '413.000'), (1, '931.650')] [2023-12-27 14:05:17,886][20381] Updated weights for policy 1, policy_version 1216961 (0.0007) [2023-12-27 14:05:17,887][20347] Updated weights for policy 0, policy_version 1218055 (0.0006) [2023-12-27 14:05:17,939][20381] Updated weights for policy 1, policy_version 1216971 (0.0008) [2023-12-27 14:05:17,946][20347] Updated weights for policy 0, policy_version 1218065 (0.0005) [2023-12-27 14:05:17,998][20381] Updated weights for policy 1, policy_version 1216981 (0.0007) [2023-12-27 14:05:18,008][20347] Updated weights for policy 0, policy_version 1218075 (0.0006) [2023-12-27 14:05:18,590][20347] Updated weights for policy 0, policy_version 1218085 (0.0006) [2023-12-27 14:05:18,654][20347] Updated weights for policy 0, policy_version 1218095 (0.0007) [2023-12-27 14:05:18,716][20347] Updated weights for policy 0, policy_version 1218105 (0.0007) [2023-12-27 14:05:18,746][20381] Updated weights for policy 1, policy_version 1216991 (0.0008) [2023-12-27 14:05:18,798][20381] Updated weights for policy 1, policy_version 1217001 (0.0008) [2023-12-27 14:05:18,853][20381] Updated weights for policy 1, policy_version 1217011 (0.0007) [2023-12-27 14:05:19,414][20347] Updated weights for policy 0, policy_version 1218115 (0.0007) [2023-12-27 14:05:19,483][20347] Updated weights for policy 0, policy_version 1218125 (0.0007) [2023-12-27 14:05:19,546][20347] Updated weights for policy 0, policy_version 1218135 (0.0008) [2023-12-27 14:05:19,589][20381] Updated weights for policy 1, policy_version 1217021 (0.0007) [2023-12-27 14:05:19,651][20381] Updated weights for policy 1, policy_version 1217031 (0.0007) [2023-12-27 14:05:19,713][20381] Updated weights for policy 1, policy_version 1217041 (0.0008) [2023-12-27 14:05:20,211][20347] Updated weights for policy 0, policy_version 1218145 (0.0006) [2023-12-27 14:05:20,276][20347] Updated weights for policy 0, policy_version 1218156 (0.0008) [2023-12-27 14:05:20,335][20347] Updated weights for policy 0, policy_version 1218166 (0.0010) [2023-12-27 14:05:20,396][20347] Updated weights for policy 0, policy_version 1218176 (0.0006) [2023-12-27 14:05:20,492][20381] Updated weights for policy 1, policy_version 1217051 (0.0006) [2023-12-27 14:05:20,544][20381] Updated weights for policy 1, policy_version 1217061 (0.0005) [2023-12-27 14:05:20,608][20381] Updated weights for policy 1, policy_version 1217071 (0.0007) [2023-12-27 14:05:21,155][20347] Updated weights for policy 0, policy_version 1218186 (0.0008) [2023-12-27 14:05:21,211][20347] Updated weights for policy 0, policy_version 1218196 (0.0008) [2023-12-27 14:05:21,270][20347] Updated weights for policy 0, policy_version 1218206 (0.0008) [2023-12-27 14:05:21,323][20381] Updated weights for policy 1, policy_version 1217081 (0.0007) [2023-12-27 14:05:21,407][20381] Updated weights for policy 1, policy_version 1217091 (0.0008) [2023-12-27 14:05:21,463][20381] Updated weights for policy 1, policy_version 1217101 (0.0005) [2023-12-27 14:05:21,523][20381] Updated weights for policy 1, policy_version 1217111 (0.0006) [2023-12-27 14:05:22,059][20347] Updated weights for policy 0, policy_version 1218216 (0.0007) [2023-12-27 14:05:22,119][20347] Updated weights for policy 0, policy_version 1218226 (0.0007) [2023-12-27 14:05:22,151][20381] Updated weights for policy 1, policy_version 1217121 (0.0006) [2023-12-27 14:05:22,168][20347] Updated weights for policy 0, policy_version 1218236 (0.0005) [2023-12-27 14:05:22,202][20381] Updated weights for policy 1, policy_version 1217131 (0.0006) [2023-12-27 14:05:22,259][20381] Updated weights for policy 1, policy_version 1217141 (0.0007) [2023-12-27 14:05:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 623542272. Throughput: 0: 10391.5, 1: 10183.1. Samples: 623509484. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:22,363][19170] Avg episode reward: [(0, '456.860'), (1, '970.750')] [2023-12-27 14:05:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001217144_311615488.pth... [2023-12-27 14:05:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001218240_311926784.pth... [2023-12-27 14:05:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001217024_311615488.pth [2023-12-27 14:05:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001215960_311312384.pth [2023-12-27 14:05:22,887][20347] Updated weights for policy 0, policy_version 1218246 (0.0006) [2023-12-27 14:05:22,936][20347] Updated weights for policy 0, policy_version 1218256 (0.0007) [2023-12-27 14:05:22,994][20347] Updated weights for policy 0, policy_version 1218266 (0.0008) [2023-12-27 14:05:23,041][20381] Updated weights for policy 1, policy_version 1217151 (0.0010) [2023-12-27 14:05:23,103][20381] Updated weights for policy 1, policy_version 1217161 (0.0011) [2023-12-27 14:05:23,157][20381] Updated weights for policy 1, policy_version 1217171 (0.0011) [2023-12-27 14:05:23,713][20347] Updated weights for policy 0, policy_version 1218276 (0.0009) [2023-12-27 14:05:23,775][20347] Updated weights for policy 0, policy_version 1218286 (0.0008) [2023-12-27 14:05:23,793][20381] Updated weights for policy 1, policy_version 1217181 (0.0009) [2023-12-27 14:05:23,832][20347] Updated weights for policy 0, policy_version 1218296 (0.0007) [2023-12-27 14:05:23,851][20381] Updated weights for policy 1, policy_version 1217191 (0.0007) [2023-12-27 14:05:23,911][20381] Updated weights for policy 1, policy_version 1217201 (0.0010) [2023-12-27 14:05:24,534][20381] Updated weights for policy 1, policy_version 1217211 (0.0009) [2023-12-27 14:05:24,548][20347] Updated weights for policy 0, policy_version 1218306 (0.0008) [2023-12-27 14:05:24,585][20381] Updated weights for policy 1, policy_version 1217221 (0.0005) [2023-12-27 14:05:24,599][20347] Updated weights for policy 0, policy_version 1218316 (0.0006) [2023-12-27 14:05:24,644][20381] Updated weights for policy 1, policy_version 1217231 (0.0010) [2023-12-27 14:05:24,657][20347] Updated weights for policy 0, policy_version 1218326 (0.0007) [2023-12-27 14:05:24,714][20347] Updated weights for policy 0, policy_version 1218336 (0.0005) [2023-12-27 14:05:25,307][20381] Updated weights for policy 1, policy_version 1217241 (0.0010) [2023-12-27 14:05:25,316][20347] Updated weights for policy 0, policy_version 1218346 (0.0007) [2023-12-27 14:05:25,355][20381] Updated weights for policy 1, policy_version 1217251 (0.0005) [2023-12-27 14:05:25,365][20347] Updated weights for policy 0, policy_version 1218356 (0.0008) [2023-12-27 14:05:25,404][20381] Updated weights for policy 1, policy_version 1217261 (0.0008) [2023-12-27 14:05:25,419][20347] Updated weights for policy 0, policy_version 1218366 (0.0006) [2023-12-27 14:05:25,463][20381] Updated weights for policy 1, policy_version 1217271 (0.0010) [2023-12-27 14:05:26,109][20347] Updated weights for policy 0, policy_version 1218376 (0.0007) [2023-12-27 14:05:26,147][20381] Updated weights for policy 1, policy_version 1217281 (0.0011) [2023-12-27 14:05:26,162][20347] Updated weights for policy 0, policy_version 1218386 (0.0005) [2023-12-27 14:05:26,206][20381] Updated weights for policy 1, policy_version 1217291 (0.0011) [2023-12-27 14:05:26,208][20347] Updated weights for policy 0, policy_version 1218396 (0.0005) [2023-12-27 14:05:26,265][20381] Updated weights for policy 1, policy_version 1217301 (0.0010) [2023-12-27 14:05:26,874][20381] Updated weights for policy 1, policy_version 1217311 (0.0008) [2023-12-27 14:05:26,906][20347] Updated weights for policy 0, policy_version 1218406 (0.0006) [2023-12-27 14:05:26,922][20381] Updated weights for policy 1, policy_version 1217321 (0.0006) [2023-12-27 14:05:26,960][20347] Updated weights for policy 0, policy_version 1218416 (0.0006) [2023-12-27 14:05:26,970][20381] Updated weights for policy 1, policy_version 1217331 (0.0005) [2023-12-27 14:05:27,019][20347] Updated weights for policy 0, policy_version 1218426 (0.0007) [2023-12-27 14:05:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 623640576. Throughput: 0: 10326.3, 1: 10158.7. Samples: 623629140. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:27,363][19170] Avg episode reward: [(0, '401.320'), (1, '902.240')] [2023-12-27 14:05:27,644][20381] Updated weights for policy 1, policy_version 1217341 (0.0007) [2023-12-27 14:05:27,658][20347] Updated weights for policy 0, policy_version 1218436 (0.0007) [2023-12-27 14:05:27,695][20381] Updated weights for policy 1, policy_version 1217351 (0.0007) [2023-12-27 14:05:27,714][20347] Updated weights for policy 0, policy_version 1218446 (0.0005) [2023-12-27 14:05:27,750][20381] Updated weights for policy 1, policy_version 1217361 (0.0007) [2023-12-27 14:05:27,764][20347] Updated weights for policy 0, policy_version 1218456 (0.0005) [2023-12-27 14:05:28,415][20347] Updated weights for policy 0, policy_version 1218466 (0.0006) [2023-12-27 14:05:28,439][20381] Updated weights for policy 1, policy_version 1217371 (0.0007) [2023-12-27 14:05:28,473][20347] Updated weights for policy 0, policy_version 1218476 (0.0007) [2023-12-27 14:05:28,494][20381] Updated weights for policy 1, policy_version 1217381 (0.0007) [2023-12-27 14:05:28,540][20347] Updated weights for policy 0, policy_version 1218486 (0.0007) [2023-12-27 14:05:28,550][20381] Updated weights for policy 1, policy_version 1217391 (0.0008) [2023-12-27 14:05:28,604][20347] Updated weights for policy 0, policy_version 1218496 (0.0007) [2023-12-27 14:05:29,174][20347] Updated weights for policy 0, policy_version 1218506 (0.0006) [2023-12-27 14:05:29,177][20381] Updated weights for policy 1, policy_version 1217401 (0.0007) [2023-12-27 14:05:29,227][20347] Updated weights for policy 0, policy_version 1218516 (0.0007) [2023-12-27 14:05:29,245][20381] Updated weights for policy 1, policy_version 1217411 (0.0009) [2023-12-27 14:05:29,284][20347] Updated weights for policy 0, policy_version 1218526 (0.0007) [2023-12-27 14:05:29,299][20381] Updated weights for policy 1, policy_version 1217421 (0.0007) [2023-12-27 14:05:29,377][20381] Updated weights for policy 1, policy_version 1217431 (0.0007) [2023-12-27 14:05:30,021][20347] Updated weights for policy 0, policy_version 1218536 (0.0006) [2023-12-27 14:05:30,057][20381] Updated weights for policy 1, policy_version 1217441 (0.0007) [2023-12-27 14:05:30,080][20347] Updated weights for policy 0, policy_version 1218546 (0.0007) [2023-12-27 14:05:30,123][20381] Updated weights for policy 1, policy_version 1217451 (0.0008) [2023-12-27 14:05:30,140][20347] Updated weights for policy 0, policy_version 1218556 (0.0007) [2023-12-27 14:05:30,188][20381] Updated weights for policy 1, policy_version 1217461 (0.0006) [2023-12-27 14:05:30,703][20347] Updated weights for policy 0, policy_version 1218566 (0.0006) [2023-12-27 14:05:30,759][20347] Updated weights for policy 0, policy_version 1218576 (0.0005) [2023-12-27 14:05:30,816][20381] Updated weights for policy 1, policy_version 1217471 (0.0007) [2023-12-27 14:05:30,824][20347] Updated weights for policy 0, policy_version 1218586 (0.0005) [2023-12-27 14:05:30,873][20381] Updated weights for policy 1, policy_version 1217481 (0.0005) [2023-12-27 14:05:30,924][20381] Updated weights for policy 1, policy_version 1217491 (0.0005) [2023-12-27 14:05:31,468][20347] Updated weights for policy 0, policy_version 1218596 (0.0006) [2023-12-27 14:05:31,521][20347] Updated weights for policy 0, policy_version 1218606 (0.0007) [2023-12-27 14:05:31,571][20347] Updated weights for policy 0, policy_version 1218616 (0.0007) [2023-12-27 14:05:31,628][20381] Updated weights for policy 1, policy_version 1217501 (0.0007) [2023-12-27 14:05:31,692][20381] Updated weights for policy 1, policy_version 1217511 (0.0009) [2023-12-27 14:05:31,760][20381] Updated weights for policy 1, policy_version 1217521 (0.0008) [2023-12-27 14:05:32,271][20347] Updated weights for policy 0, policy_version 1218626 (0.0008) [2023-12-27 14:05:32,330][20347] Updated weights for policy 0, policy_version 1218636 (0.0007) [2023-12-27 14:05:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 623738880. Throughput: 0: 10348.1, 1: 10186.3. Samples: 623754864. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:32,363][19170] Avg episode reward: [(0, '651.380'), (1, '912.090')] [2023-12-27 14:05:32,394][20347] Updated weights for policy 0, policy_version 1218646 (0.0008) [2023-12-27 14:05:32,448][20381] Updated weights for policy 1, policy_version 1217531 (0.0007) [2023-12-27 14:05:32,453][20347] Updated weights for policy 0, policy_version 1218656 (0.0006) [2023-12-27 14:05:32,511][20381] Updated weights for policy 1, policy_version 1217541 (0.0005) [2023-12-27 14:05:32,572][20381] Updated weights for policy 1, policy_version 1217551 (0.0005) [2023-12-27 14:05:33,075][20381] Updated weights for policy 1, policy_version 1217561 (0.0005) [2023-12-27 14:05:33,136][20381] Updated weights for policy 1, policy_version 1217571 (0.0006) [2023-12-27 14:05:33,183][20347] Updated weights for policy 0, policy_version 1218666 (0.0006) [2023-12-27 14:05:33,191][20381] Updated weights for policy 1, policy_version 1217581 (0.0007) [2023-12-27 14:05:33,226][20347] Updated weights for policy 0, policy_version 1218676 (0.0006) [2023-12-27 14:05:33,241][20381] Updated weights for policy 1, policy_version 1217591 (0.0007) [2023-12-27 14:05:33,283][20347] Updated weights for policy 0, policy_version 1218686 (0.0007) [2023-12-27 14:05:33,860][20347] Updated weights for policy 0, policy_version 1218696 (0.0006) [2023-12-27 14:05:33,909][20347] Updated weights for policy 0, policy_version 1218706 (0.0005) [2023-12-27 14:05:33,919][20381] Updated weights for policy 1, policy_version 1217601 (0.0008) [2023-12-27 14:05:33,958][20347] Updated weights for policy 0, policy_version 1218716 (0.0005) [2023-12-27 14:05:33,971][20381] Updated weights for policy 1, policy_version 1217611 (0.0008) [2023-12-27 14:05:34,029][20381] Updated weights for policy 1, policy_version 1217621 (0.0006) [2023-12-27 14:05:34,573][20347] Updated weights for policy 0, policy_version 1218726 (0.0005) [2023-12-27 14:05:34,635][20347] Updated weights for policy 0, policy_version 1218736 (0.0006) [2023-12-27 14:05:34,693][20347] Updated weights for policy 0, policy_version 1218746 (0.0006) [2023-12-27 14:05:34,738][20381] Updated weights for policy 1, policy_version 1217631 (0.0007) [2023-12-27 14:05:34,804][20381] Updated weights for policy 1, policy_version 1217641 (0.0008) [2023-12-27 14:05:34,867][20381] Updated weights for policy 1, policy_version 1217651 (0.0008) [2023-12-27 14:05:35,300][20347] Updated weights for policy 0, policy_version 1218756 (0.0006) [2023-12-27 14:05:35,355][20347] Updated weights for policy 0, policy_version 1218766 (0.0005) [2023-12-27 14:05:35,405][20347] Updated weights for policy 0, policy_version 1218776 (0.0005) [2023-12-27 14:05:35,537][20381] Updated weights for policy 1, policy_version 1217661 (0.0006) [2023-12-27 14:05:35,586][20381] Updated weights for policy 1, policy_version 1217671 (0.0009) [2023-12-27 14:05:35,645][20381] Updated weights for policy 1, policy_version 1217681 (0.0007) [2023-12-27 14:05:35,987][20347] Updated weights for policy 0, policy_version 1218786 (0.0006) [2023-12-27 14:05:36,039][20347] Updated weights for policy 0, policy_version 1218796 (0.0008) [2023-12-27 14:05:36,088][20347] Updated weights for policy 0, policy_version 1218806 (0.0008) [2023-12-27 14:05:36,154][20347] Updated weights for policy 0, policy_version 1218816 (0.0009) [2023-12-27 14:05:36,329][20381] Updated weights for policy 1, policy_version 1217691 (0.0007) [2023-12-27 14:05:36,391][20381] Updated weights for policy 1, policy_version 1217701 (0.0007) [2023-12-27 14:05:36,441][20381] Updated weights for policy 1, policy_version 1217711 (0.0008) [2023-12-27 14:05:36,934][20347] Updated weights for policy 0, policy_version 1218826 (0.0005) [2023-12-27 14:05:36,995][20347] Updated weights for policy 0, policy_version 1218836 (0.0006) [2023-12-27 14:05:37,050][20347] Updated weights for policy 0, policy_version 1218846 (0.0005) [2023-12-27 14:05:37,137][20381] Updated weights for policy 1, policy_version 1217721 (0.0008) [2023-12-27 14:05:37,183][20381] Updated weights for policy 1, policy_version 1217731 (0.0009) [2023-12-27 14:05:37,235][20381] Updated weights for policy 1, policy_version 1217741 (0.0007) [2023-12-27 14:05:37,297][20381] Updated weights for policy 1, policy_version 1217751 (0.0005) [2023-12-27 14:05:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 623853568. Throughput: 0: 10358.0, 1: 10248.4. Samples: 623819372. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:37,363][19170] Avg episode reward: [(0, '394.540'), (1, '931.550')] [2023-12-27 14:05:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001217752_311771136.pth... [2023-12-27 14:05:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001218848_312082432.pth... [2023-12-27 14:05:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001216536_311459840.pth [2023-12-27 14:05:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001217632_311771136.pth [2023-12-27 14:05:37,675][20347] Updated weights for policy 0, policy_version 1218856 (0.0006) [2023-12-27 14:05:37,728][20347] Updated weights for policy 0, policy_version 1218866 (0.0009) [2023-12-27 14:05:37,781][20347] Updated weights for policy 0, policy_version 1218876 (0.0008) [2023-12-27 14:05:37,913][20381] Updated weights for policy 1, policy_version 1217761 (0.0007) [2023-12-27 14:05:37,966][20381] Updated weights for policy 1, policy_version 1217771 (0.0006) [2023-12-27 14:05:38,023][20381] Updated weights for policy 1, policy_version 1217781 (0.0007) [2023-12-27 14:05:38,485][20347] Updated weights for policy 0, policy_version 1218886 (0.0006) [2023-12-27 14:05:38,541][20347] Updated weights for policy 0, policy_version 1218896 (0.0006) [2023-12-27 14:05:38,601][20347] Updated weights for policy 0, policy_version 1218906 (0.0008) [2023-12-27 14:05:38,777][20381] Updated weights for policy 1, policy_version 1217791 (0.0008) [2023-12-27 14:05:38,835][20381] Updated weights for policy 1, policy_version 1217801 (0.0007) [2023-12-27 14:05:38,891][20381] Updated weights for policy 1, policy_version 1217811 (0.0007) [2023-12-27 14:05:39,286][20347] Updated weights for policy 0, policy_version 1218916 (0.0008) [2023-12-27 14:05:39,351][20347] Updated weights for policy 0, policy_version 1218926 (0.0008) [2023-12-27 14:05:39,416][20347] Updated weights for policy 0, policy_version 1218936 (0.0007) [2023-12-27 14:05:39,619][20381] Updated weights for policy 1, policy_version 1217821 (0.0007) [2023-12-27 14:05:39,679][20381] Updated weights for policy 1, policy_version 1217831 (0.0008) [2023-12-27 14:05:39,737][20381] Updated weights for policy 1, policy_version 1217841 (0.0008) [2023-12-27 14:05:40,184][20347] Updated weights for policy 0, policy_version 1218946 (0.0008) [2023-12-27 14:05:40,243][20347] Updated weights for policy 0, policy_version 1218956 (0.0007) [2023-12-27 14:05:40,311][20347] Updated weights for policy 0, policy_version 1218966 (0.0008) [2023-12-27 14:05:40,342][20381] Updated weights for policy 1, policy_version 1217851 (0.0008) [2023-12-27 14:05:40,369][20347] Updated weights for policy 0, policy_version 1218976 (0.0008) [2023-12-27 14:05:40,403][20381] Updated weights for policy 1, policy_version 1217861 (0.0008) [2023-12-27 14:05:40,466][20381] Updated weights for policy 1, policy_version 1217871 (0.0008) [2023-12-27 14:05:41,101][20347] Updated weights for policy 0, policy_version 1218986 (0.0008) [2023-12-27 14:05:41,166][20347] Updated weights for policy 0, policy_version 1218996 (0.0006) [2023-12-27 14:05:41,176][20381] Updated weights for policy 1, policy_version 1217881 (0.0008) [2023-12-27 14:05:41,220][20347] Updated weights for policy 0, policy_version 1219006 (0.0006) [2023-12-27 14:05:41,241][20381] Updated weights for policy 1, policy_version 1217891 (0.0008) [2023-12-27 14:05:41,303][20381] Updated weights for policy 1, policy_version 1217901 (0.0007) [2023-12-27 14:05:41,367][20381] Updated weights for policy 1, policy_version 1217911 (0.0007) [2023-12-27 14:05:41,988][20347] Updated weights for policy 0, policy_version 1219016 (0.0007) [2023-12-27 14:05:41,993][20381] Updated weights for policy 1, policy_version 1217921 (0.0005) [2023-12-27 14:05:42,044][20347] Updated weights for policy 0, policy_version 1219026 (0.0007) [2023-12-27 14:05:42,045][20381] Updated weights for policy 1, policy_version 1217931 (0.0005) [2023-12-27 14:05:42,106][20347] Updated weights for policy 0, policy_version 1219036 (0.0008) [2023-12-27 14:05:42,108][20381] Updated weights for policy 1, policy_version 1217941 (0.0006) [2023-12-27 14:05:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 623951872. Throughput: 0: 10321.6, 1: 10215.9. Samples: 623940208. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 14:05:42,363][19170] Avg episode reward: [(0, '577.570'), (1, '951.100')] [2023-12-27 14:05:42,786][20347] Updated weights for policy 0, policy_version 1219046 (0.0009) [2023-12-27 14:05:42,822][20381] Updated weights for policy 1, policy_version 1217951 (0.0006) [2023-12-27 14:05:42,846][20347] Updated weights for policy 0, policy_version 1219056 (0.0011) [2023-12-27 14:05:42,879][20381] Updated weights for policy 1, policy_version 1217961 (0.0008) [2023-12-27 14:05:42,905][20347] Updated weights for policy 0, policy_version 1219066 (0.0010) [2023-12-27 14:05:42,932][20381] Updated weights for policy 1, policy_version 1217971 (0.0006) [2023-12-27 14:05:43,554][20347] Updated weights for policy 0, policy_version 1219076 (0.0010) [2023-12-27 14:05:43,608][20347] Updated weights for policy 0, policy_version 1219086 (0.0010) [2023-12-27 14:05:43,647][20381] Updated weights for policy 1, policy_version 1217981 (0.0007) [2023-12-27 14:05:43,666][20347] Updated weights for policy 0, policy_version 1219096 (0.0010) [2023-12-27 14:05:43,700][20381] Updated weights for policy 1, policy_version 1217991 (0.0005) [2023-12-27 14:05:43,756][20381] Updated weights for policy 1, policy_version 1218001 (0.0005) [2023-12-27 14:05:44,321][20381] Updated weights for policy 1, policy_version 1218011 (0.0005) [2023-12-27 14:05:44,345][20347] Updated weights for policy 0, policy_version 1219106 (0.0009) [2023-12-27 14:05:44,389][20381] Updated weights for policy 1, policy_version 1218021 (0.0010) [2023-12-27 14:05:44,399][20347] Updated weights for policy 0, policy_version 1219116 (0.0008) [2023-12-27 14:05:44,449][20381] Updated weights for policy 1, policy_version 1218031 (0.0006) [2023-12-27 14:05:44,451][20347] Updated weights for policy 0, policy_version 1219126 (0.0006) [2023-12-27 14:05:44,507][20347] Updated weights for policy 0, policy_version 1219136 (0.0005) [2023-12-27 14:05:45,064][20381] Updated weights for policy 1, policy_version 1218041 (0.0006) [2023-12-27 14:05:45,133][20381] Updated weights for policy 1, policy_version 1218051 (0.0007) [2023-12-27 14:05:45,157][20347] Updated weights for policy 0, policy_version 1219146 (0.0007) [2023-12-27 14:05:45,193][20381] Updated weights for policy 1, policy_version 1218061 (0.0006) [2023-12-27 14:05:45,218][20347] Updated weights for policy 0, policy_version 1219156 (0.0007) [2023-12-27 14:05:45,252][20381] Updated weights for policy 1, policy_version 1218071 (0.0009) [2023-12-27 14:05:45,275][20347] Updated weights for policy 0, policy_version 1219166 (0.0006) [2023-12-27 14:05:45,874][20347] Updated weights for policy 0, policy_version 1219176 (0.0006) [2023-12-27 14:05:45,884][20381] Updated weights for policy 1, policy_version 1218081 (0.0010) [2023-12-27 14:05:45,940][20347] Updated weights for policy 0, policy_version 1219186 (0.0005) [2023-12-27 14:05:45,943][20381] Updated weights for policy 1, policy_version 1218091 (0.0010) [2023-12-27 14:05:45,991][20347] Updated weights for policy 0, policy_version 1219196 (0.0005) [2023-12-27 14:05:46,003][20381] Updated weights for policy 1, policy_version 1218101 (0.0008) [2023-12-27 14:05:46,561][20347] Updated weights for policy 0, policy_version 1219206 (0.0005) [2023-12-27 14:05:46,616][20347] Updated weights for policy 0, policy_version 1219216 (0.0005) [2023-12-27 14:05:46,673][20381] Updated weights for policy 1, policy_version 1218111 (0.0007) [2023-12-27 14:05:46,674][20347] Updated weights for policy 0, policy_version 1219226 (0.0005) [2023-12-27 14:05:46,729][20381] Updated weights for policy 1, policy_version 1218121 (0.0005) [2023-12-27 14:05:46,789][20381] Updated weights for policy 1, policy_version 1218131 (0.0005) [2023-12-27 14:05:47,302][20347] Updated weights for policy 0, policy_version 1219236 (0.0007) [2023-12-27 14:05:47,351][20347] Updated weights for policy 0, policy_version 1219246 (0.0010) [2023-12-27 14:05:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 624050176. Throughput: 0: 10374.4, 1: 10259.7. Samples: 624066232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:05:47,363][19170] Avg episode reward: [(0, '367.510'), (1, '941.470')] [2023-12-27 14:05:47,371][20381] Updated weights for policy 1, policy_version 1218141 (0.0010) [2023-12-27 14:05:47,400][20347] Updated weights for policy 0, policy_version 1219256 (0.0010) [2023-12-27 14:05:47,419][20381] Updated weights for policy 1, policy_version 1218151 (0.0010) [2023-12-27 14:05:47,470][20381] Updated weights for policy 1, policy_version 1218161 (0.0010) [2023-12-27 14:05:48,013][20347] Updated weights for policy 0, policy_version 1219266 (0.0008) [2023-12-27 14:05:48,061][20347] Updated weights for policy 0, policy_version 1219276 (0.0010) [2023-12-27 14:05:48,081][20381] Updated weights for policy 1, policy_version 1218171 (0.0009) [2023-12-27 14:05:48,104][20347] Updated weights for policy 0, policy_version 1219286 (0.0009) [2023-12-27 14:05:48,137][20381] Updated weights for policy 1, policy_version 1218181 (0.0005) [2023-12-27 14:05:48,162][20347] Updated weights for policy 0, policy_version 1219296 (0.0009) [2023-12-27 14:05:48,191][20381] Updated weights for policy 1, policy_version 1218191 (0.0005) [2023-12-27 14:05:48,845][20347] Updated weights for policy 0, policy_version 1219306 (0.0009) [2023-12-27 14:05:48,859][20381] Updated weights for policy 1, policy_version 1218201 (0.0010) [2023-12-27 14:05:48,901][20347] Updated weights for policy 0, policy_version 1219316 (0.0007) [2023-12-27 14:05:48,916][20381] Updated weights for policy 1, policy_version 1218211 (0.0006) [2023-12-27 14:05:48,966][20347] Updated weights for policy 0, policy_version 1219326 (0.0006) [2023-12-27 14:05:48,978][20381] Updated weights for policy 1, policy_version 1218221 (0.0007) [2023-12-27 14:05:49,040][20381] Updated weights for policy 1, policy_version 1218231 (0.0011) [2023-12-27 14:05:49,647][20347] Updated weights for policy 0, policy_version 1219336 (0.0007) [2023-12-27 14:05:49,700][20347] Updated weights for policy 0, policy_version 1219346 (0.0006) [2023-12-27 14:05:49,709][20292] KL-divergence is very high: 113.7036 [2023-12-27 14:05:49,709][20381] Updated weights for policy 1, policy_version 1218241 (0.0010) [2023-12-27 14:05:49,721][20292] KL-divergence is very high: 129.3654 [2023-12-27 14:05:49,753][20347] Updated weights for policy 0, policy_version 1219356 (0.0006) [2023-12-27 14:05:49,758][20292] KL-divergence is very high: 100.2660 [2023-12-27 14:05:49,772][20381] Updated weights for policy 1, policy_version 1218251 (0.0009) [2023-12-27 14:05:49,835][20381] Updated weights for policy 1, policy_version 1218261 (0.0008) [2023-12-27 14:05:50,432][20347] Updated weights for policy 0, policy_version 1219366 (0.0006) [2023-12-27 14:05:50,486][20347] Updated weights for policy 0, policy_version 1219376 (0.0005) [2023-12-27 14:05:50,537][20347] Updated weights for policy 0, policy_version 1219386 (0.0005) [2023-12-27 14:05:50,539][20381] Updated weights for policy 1, policy_version 1218271 (0.0010) [2023-12-27 14:05:50,599][20381] Updated weights for policy 1, policy_version 1218281 (0.0009) [2023-12-27 14:05:50,657][20381] Updated weights for policy 1, policy_version 1218291 (0.0010) [2023-12-27 14:05:51,159][20347] Updated weights for policy 0, policy_version 1219396 (0.0007) [2023-12-27 14:05:51,221][20347] Updated weights for policy 0, policy_version 1219406 (0.0006) [2023-12-27 14:05:51,286][20347] Updated weights for policy 0, policy_version 1219416 (0.0008) [2023-12-27 14:05:51,381][20381] Updated weights for policy 1, policy_version 1218301 (0.0009) [2023-12-27 14:05:51,443][20381] Updated weights for policy 1, policy_version 1218311 (0.0008) [2023-12-27 14:05:51,505][20381] Updated weights for policy 1, policy_version 1218321 (0.0008) [2023-12-27 14:05:51,998][20347] Updated weights for policy 0, policy_version 1219426 (0.0008) [2023-12-27 14:05:52,056][20347] Updated weights for policy 0, policy_version 1219436 (0.0008) [2023-12-27 14:05:52,112][20347] Updated weights for policy 0, policy_version 1219446 (0.0007) [2023-12-27 14:05:52,172][20347] Updated weights for policy 0, policy_version 1219456 (0.0007) [2023-12-27 14:05:52,252][20381] Updated weights for policy 1, policy_version 1218331 (0.0007) [2023-12-27 14:05:52,307][20381] Updated weights for policy 1, policy_version 1218341 (0.0008) [2023-12-27 14:05:52,361][20381] Updated weights for policy 1, policy_version 1218351 (0.0007) [2023-12-27 14:05:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 624156672. Throughput: 0: 10372.9, 1: 10283.0. Samples: 624128924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:05:52,364][19170] Avg episode reward: [(0, '420.140'), (1, '912.280')] [2023-12-27 14:05:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001219456_312238080.pth... [2023-12-27 14:05:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001218240_311926784.pth [2023-12-27 14:05:52,420][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001218360_311926784.pth... [2023-12-27 14:05:52,424][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001217144_311615488.pth [2023-12-27 14:05:52,809][20347] Updated weights for policy 0, policy_version 1219466 (0.0010) [2023-12-27 14:05:52,867][20347] Updated weights for policy 0, policy_version 1219476 (0.0011) [2023-12-27 14:05:52,918][20347] Updated weights for policy 0, policy_version 1219486 (0.0010) [2023-12-27 14:05:53,010][20381] Updated weights for policy 1, policy_version 1218361 (0.0008) [2023-12-27 14:05:53,058][20381] Updated weights for policy 1, policy_version 1218371 (0.0007) [2023-12-27 14:05:53,105][20381] Updated weights for policy 1, policy_version 1218381 (0.0007) [2023-12-27 14:05:53,105][20292] KL-divergence is very high: 225.3442 [2023-12-27 14:05:53,144][20292] KL-divergence is very high: 126.2111 [2023-12-27 14:05:53,153][20381] Updated weights for policy 1, policy_version 1218391 (0.0006) [2023-12-27 14:05:53,533][20347] Updated weights for policy 0, policy_version 1219496 (0.0010) [2023-12-27 14:05:53,581][20347] Updated weights for policy 0, policy_version 1219506 (0.0010) [2023-12-27 14:05:53,630][20347] Updated weights for policy 0, policy_version 1219516 (0.0010) [2023-12-27 14:05:53,863][20381] Updated weights for policy 1, policy_version 1218401 (0.0008) [2023-12-27 14:05:53,932][20381] Updated weights for policy 1, policy_version 1218411 (0.0009) [2023-12-27 14:05:53,996][20381] Updated weights for policy 1, policy_version 1218421 (0.0008) [2023-12-27 14:05:54,307][20347] Updated weights for policy 0, policy_version 1219526 (0.0010) [2023-12-27 14:05:54,358][20347] Updated weights for policy 0, policy_version 1219536 (0.0009) [2023-12-27 14:05:54,408][20347] Updated weights for policy 0, policy_version 1219546 (0.0006) [2023-12-27 14:05:54,630][20381] Updated weights for policy 1, policy_version 1218431 (0.0006) [2023-12-27 14:05:54,689][20381] Updated weights for policy 1, policy_version 1218441 (0.0005) [2023-12-27 14:05:54,750][20381] Updated weights for policy 1, policy_version 1218451 (0.0005) [2023-12-27 14:05:55,040][20347] Updated weights for policy 0, policy_version 1219556 (0.0008) [2023-12-27 14:05:55,096][20347] Updated weights for policy 0, policy_version 1219566 (0.0005) [2023-12-27 14:05:55,152][20347] Updated weights for policy 0, policy_version 1219576 (0.0007) [2023-12-27 14:05:55,355][20381] Updated weights for policy 1, policy_version 1218461 (0.0007) [2023-12-27 14:05:55,406][20381] Updated weights for policy 1, policy_version 1218471 (0.0007) [2023-12-27 14:05:55,449][20381] Updated weights for policy 1, policy_version 1218481 (0.0008) [2023-12-27 14:05:55,793][20347] Updated weights for policy 0, policy_version 1219586 (0.0008) [2023-12-27 14:05:55,847][20347] Updated weights for policy 0, policy_version 1219596 (0.0007) [2023-12-27 14:05:55,905][20347] Updated weights for policy 0, policy_version 1219606 (0.0008) [2023-12-27 14:05:55,963][20347] Updated weights for policy 0, policy_version 1219616 (0.0008) [2023-12-27 14:05:56,149][20381] Updated weights for policy 1, policy_version 1218491 (0.0007) [2023-12-27 14:05:56,200][20381] Updated weights for policy 1, policy_version 1218501 (0.0005) [2023-12-27 14:05:56,250][20381] Updated weights for policy 1, policy_version 1218511 (0.0005) [2023-12-27 14:05:56,681][20347] Updated weights for policy 0, policy_version 1219626 (0.0008) [2023-12-27 14:05:56,728][20347] Updated weights for policy 0, policy_version 1219637 (0.0008) [2023-12-27 14:05:56,774][20381] Updated weights for policy 1, policy_version 1218521 (0.0005) [2023-12-27 14:05:56,782][20347] Updated weights for policy 0, policy_version 1219647 (0.0007) [2023-12-27 14:05:56,829][20381] Updated weights for policy 1, policy_version 1218531 (0.0007) [2023-12-27 14:05:56,879][20381] Updated weights for policy 1, policy_version 1218541 (0.0007) [2023-12-27 14:05:56,923][20381] Updated weights for policy 1, policy_version 1218551 (0.0007) [2023-12-27 14:05:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 624263168. Throughput: 0: 10358.3, 1: 10320.1. Samples: 624253052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:05:57,363][19170] Avg episode reward: [(0, '427.020'), (1, '872.950')] [2023-12-27 14:05:57,444][20347] Updated weights for policy 0, policy_version 1219657 (0.0007) [2023-12-27 14:05:57,495][20347] Updated weights for policy 0, policy_version 1219667 (0.0010) [2023-12-27 14:05:57,556][20347] Updated weights for policy 0, policy_version 1219677 (0.0010) [2023-12-27 14:05:57,620][20381] Updated weights for policy 1, policy_version 1218561 (0.0010) [2023-12-27 14:05:57,674][20381] Updated weights for policy 1, policy_version 1218571 (0.0010) [2023-12-27 14:05:57,726][20381] Updated weights for policy 1, policy_version 1218581 (0.0008) [2023-12-27 14:05:58,206][20347] Updated weights for policy 0, policy_version 1219687 (0.0009) [2023-12-27 14:05:58,270][20347] Updated weights for policy 0, policy_version 1219697 (0.0011) [2023-12-27 14:05:58,327][20347] Updated weights for policy 0, policy_version 1219707 (0.0010) [2023-12-27 14:05:58,375][20381] Updated weights for policy 1, policy_version 1218591 (0.0008) [2023-12-27 14:05:58,430][20381] Updated weights for policy 1, policy_version 1218601 (0.0008) [2023-12-27 14:05:58,489][20381] Updated weights for policy 1, policy_version 1218611 (0.0008) [2023-12-27 14:05:59,156][20347] Updated weights for policy 0, policy_version 1219717 (0.0008) [2023-12-27 14:05:59,214][20347] Updated weights for policy 0, policy_version 1219727 (0.0008) [2023-12-27 14:05:59,276][20347] Updated weights for policy 0, policy_version 1219737 (0.0009) [2023-12-27 14:05:59,295][20381] Updated weights for policy 1, policy_version 1218621 (0.0009) [2023-12-27 14:05:59,353][20381] Updated weights for policy 1, policy_version 1218631 (0.0010) [2023-12-27 14:05:59,419][20381] Updated weights for policy 1, policy_version 1218641 (0.0008) [2023-12-27 14:05:59,938][20347] Updated weights for policy 0, policy_version 1219747 (0.0008) [2023-12-27 14:05:59,992][20347] Updated weights for policy 0, policy_version 1219757 (0.0008) [2023-12-27 14:06:00,055][20347] Updated weights for policy 0, policy_version 1219767 (0.0007) [2023-12-27 14:06:00,138][20381] Updated weights for policy 1, policy_version 1218651 (0.0008) [2023-12-27 14:06:00,186][20381] Updated weights for policy 1, policy_version 1218661 (0.0010) [2023-12-27 14:06:00,245][20381] Updated weights for policy 1, policy_version 1218671 (0.0010) [2023-12-27 14:06:00,730][20347] Updated weights for policy 0, policy_version 1219777 (0.0007) [2023-12-27 14:06:00,783][20347] Updated weights for policy 0, policy_version 1219787 (0.0005) [2023-12-27 14:06:00,842][20347] Updated weights for policy 0, policy_version 1219797 (0.0005) [2023-12-27 14:06:00,900][20347] Updated weights for policy 0, policy_version 1219807 (0.0005) [2023-12-27 14:06:00,922][20381] Updated weights for policy 1, policy_version 1218681 (0.0009) [2023-12-27 14:06:00,977][20381] Updated weights for policy 1, policy_version 1218691 (0.0007) [2023-12-27 14:06:01,022][20381] Updated weights for policy 1, policy_version 1218701 (0.0008) [2023-12-27 14:06:01,084][20381] Updated weights for policy 1, policy_version 1218711 (0.0008) [2023-12-27 14:06:01,537][20347] Updated weights for policy 0, policy_version 1219817 (0.0006) [2023-12-27 14:06:01,593][20347] Updated weights for policy 0, policy_version 1219827 (0.0006) [2023-12-27 14:06:01,658][20347] Updated weights for policy 0, policy_version 1219837 (0.0009) [2023-12-27 14:06:01,849][20381] Updated weights for policy 1, policy_version 1218721 (0.0008) [2023-12-27 14:06:01,902][20381] Updated weights for policy 1, policy_version 1218731 (0.0006) [2023-12-27 14:06:01,959][20381] Updated weights for policy 1, policy_version 1218741 (0.0007) [2023-12-27 14:06:02,292][20347] Updated weights for policy 0, policy_version 1219847 (0.0008) [2023-12-27 14:06:02,357][20347] Updated weights for policy 0, policy_version 1219857 (0.0008) [2023-12-27 14:06:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 624361472. Throughput: 0: 10325.0, 1: 10280.2. Samples: 624375864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:02,363][19170] Avg episode reward: [(0, '357.820'), (1, '892.430')] [2023-12-27 14:06:02,413][20347] Updated weights for policy 0, policy_version 1219867 (0.0006) [2023-12-27 14:06:02,548][20381] Updated weights for policy 1, policy_version 1218751 (0.0007) [2023-12-27 14:06:02,609][20381] Updated weights for policy 1, policy_version 1218761 (0.0008) [2023-12-27 14:06:02,665][20381] Updated weights for policy 1, policy_version 1218771 (0.0008) [2023-12-27 14:06:03,064][20347] Updated weights for policy 0, policy_version 1219877 (0.0008) [2023-12-27 14:06:03,109][20347] Updated weights for policy 0, policy_version 1219887 (0.0007) [2023-12-27 14:06:03,156][20347] Updated weights for policy 0, policy_version 1219897 (0.0005) [2023-12-27 14:06:03,374][20381] Updated weights for policy 1, policy_version 1218781 (0.0007) [2023-12-27 14:06:03,427][20381] Updated weights for policy 1, policy_version 1218791 (0.0007) [2023-12-27 14:06:03,480][20381] Updated weights for policy 1, policy_version 1218801 (0.0007) [2023-12-27 14:06:03,777][20347] Updated weights for policy 0, policy_version 1219907 (0.0005) [2023-12-27 14:06:03,828][20347] Updated weights for policy 0, policy_version 1219917 (0.0005) [2023-12-27 14:06:03,890][20347] Updated weights for policy 0, policy_version 1219927 (0.0009) [2023-12-27 14:06:04,200][20381] Updated weights for policy 1, policy_version 1218811 (0.0009) [2023-12-27 14:06:04,260][20381] Updated weights for policy 1, policy_version 1218821 (0.0011) [2023-12-27 14:06:04,322][20381] Updated weights for policy 1, policy_version 1218831 (0.0011) [2023-12-27 14:06:04,588][20347] Updated weights for policy 0, policy_version 1219937 (0.0009) [2023-12-27 14:06:04,646][20347] Updated weights for policy 0, policy_version 1219947 (0.0008) [2023-12-27 14:06:04,690][20347] Updated weights for policy 0, policy_version 1219957 (0.0009) [2023-12-27 14:06:04,739][20347] Updated weights for policy 0, policy_version 1219967 (0.0007) [2023-12-27 14:06:05,007][20381] Updated weights for policy 1, policy_version 1218841 (0.0011) [2023-12-27 14:06:05,059][20381] Updated weights for policy 1, policy_version 1218851 (0.0010) [2023-12-27 14:06:05,108][20381] Updated weights for policy 1, policy_version 1218861 (0.0011) [2023-12-27 14:06:05,160][20381] Updated weights for policy 1, policy_version 1218871 (0.0010) [2023-12-27 14:06:05,355][20347] Updated weights for policy 0, policy_version 1219977 (0.0006) [2023-12-27 14:06:05,408][20347] Updated weights for policy 0, policy_version 1219987 (0.0005) [2023-12-27 14:06:05,467][20347] Updated weights for policy 0, policy_version 1219997 (0.0008) [2023-12-27 14:06:05,853][20381] Updated weights for policy 1, policy_version 1218881 (0.0011) [2023-12-27 14:06:05,914][20381] Updated weights for policy 1, policy_version 1218891 (0.0011) [2023-12-27 14:06:05,970][20381] Updated weights for policy 1, policy_version 1218901 (0.0011) [2023-12-27 14:06:06,038][20347] Updated weights for policy 0, policy_version 1220007 (0.0006) [2023-12-27 14:06:06,093][20347] Updated weights for policy 0, policy_version 1220017 (0.0008) [2023-12-27 14:06:06,154][20347] Updated weights for policy 0, policy_version 1220027 (0.0007) [2023-12-27 14:06:06,648][20381] Updated weights for policy 1, policy_version 1218911 (0.0011) [2023-12-27 14:06:06,712][20381] Updated weights for policy 1, policy_version 1218921 (0.0011) [2023-12-27 14:06:06,765][20381] Updated weights for policy 1, policy_version 1218931 (0.0011) [2023-12-27 14:06:06,791][20347] Updated weights for policy 0, policy_version 1220037 (0.0008) [2023-12-27 14:06:06,840][20347] Updated weights for policy 0, policy_version 1220047 (0.0009) [2023-12-27 14:06:06,889][20347] Updated weights for policy 0, policy_version 1220057 (0.0005) [2023-12-27 14:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 624467968. Throughput: 0: 10338.0, 1: 10283.2. Samples: 624437432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:07,363][19170] Avg episode reward: [(0, '320.750'), (1, '921.750')] [2023-12-27 14:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001218936_312074240.pth... [2023-12-27 14:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001220064_312393728.pth... [2023-12-27 14:06:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001217752_311771136.pth [2023-12-27 14:06:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001218848_312082432.pth [2023-12-27 14:06:07,425][20381] Updated weights for policy 1, policy_version 1218941 (0.0010) [2023-12-27 14:06:07,476][20347] Updated weights for policy 0, policy_version 1220067 (0.0007) [2023-12-27 14:06:07,478][20381] Updated weights for policy 1, policy_version 1218951 (0.0011) [2023-12-27 14:06:07,531][20381] Updated weights for policy 1, policy_version 1218961 (0.0011) [2023-12-27 14:06:07,534][20347] Updated weights for policy 0, policy_version 1220077 (0.0005) [2023-12-27 14:06:07,597][20347] Updated weights for policy 0, policy_version 1220087 (0.0008) [2023-12-27 14:06:08,181][20347] Updated weights for policy 0, policy_version 1220097 (0.0010) [2023-12-27 14:06:08,220][20381] Updated weights for policy 1, policy_version 1218971 (0.0009) [2023-12-27 14:06:08,240][20347] Updated weights for policy 0, policy_version 1220107 (0.0010) [2023-12-27 14:06:08,273][20381] Updated weights for policy 1, policy_version 1218981 (0.0005) [2023-12-27 14:06:08,297][20347] Updated weights for policy 0, policy_version 1220117 (0.0010) [2023-12-27 14:06:08,327][20381] Updated weights for policy 1, policy_version 1218991 (0.0007) [2023-12-27 14:06:08,359][20347] Updated weights for policy 0, policy_version 1220127 (0.0010) [2023-12-27 14:06:09,039][20381] Updated weights for policy 1, policy_version 1219001 (0.0009) [2023-12-27 14:06:09,077][20347] Updated weights for policy 0, policy_version 1220137 (0.0010) [2023-12-27 14:06:09,095][20381] Updated weights for policy 1, policy_version 1219011 (0.0009) [2023-12-27 14:06:09,136][20347] Updated weights for policy 0, policy_version 1220147 (0.0010) [2023-12-27 14:06:09,152][20381] Updated weights for policy 1, policy_version 1219021 (0.0009) [2023-12-27 14:06:09,188][20347] Updated weights for policy 0, policy_version 1220157 (0.0010) [2023-12-27 14:06:09,207][20381] Updated weights for policy 1, policy_version 1219031 (0.0010) [2023-12-27 14:06:09,929][20347] Updated weights for policy 0, policy_version 1220167 (0.0009) [2023-12-27 14:06:09,995][20347] Updated weights for policy 0, policy_version 1220177 (0.0007) [2023-12-27 14:06:10,004][20381] Updated weights for policy 1, policy_version 1219041 (0.0008) [2023-12-27 14:06:10,056][20347] Updated weights for policy 0, policy_version 1220187 (0.0006) [2023-12-27 14:06:10,064][20381] Updated weights for policy 1, policy_version 1219051 (0.0007) [2023-12-27 14:06:10,121][20381] Updated weights for policy 1, policy_version 1219061 (0.0007) [2023-12-27 14:06:10,721][20347] Updated weights for policy 0, policy_version 1220197 (0.0006) [2023-12-27 14:06:10,775][20347] Updated weights for policy 0, policy_version 1220207 (0.0006) [2023-12-27 14:06:10,799][20381] Updated weights for policy 1, policy_version 1219071 (0.0006) [2023-12-27 14:06:10,824][20347] Updated weights for policy 0, policy_version 1220217 (0.0006) [2023-12-27 14:06:10,849][20381] Updated weights for policy 1, policy_version 1219081 (0.0006) [2023-12-27 14:06:10,901][20381] Updated weights for policy 1, policy_version 1219091 (0.0007) [2023-12-27 14:06:11,529][20347] Updated weights for policy 0, policy_version 1220227 (0.0006) [2023-12-27 14:06:11,588][20347] Updated weights for policy 0, policy_version 1220237 (0.0007) [2023-12-27 14:06:11,655][20347] Updated weights for policy 0, policy_version 1220247 (0.0007) [2023-12-27 14:06:11,668][20381] Updated weights for policy 1, policy_version 1219101 (0.0008) [2023-12-27 14:06:11,729][20381] Updated weights for policy 1, policy_version 1219111 (0.0007) [2023-12-27 14:06:11,790][20381] Updated weights for policy 1, policy_version 1219121 (0.0009) [2023-12-27 14:06:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 624566272. Throughput: 0: 10459.5, 1: 10264.8. Samples: 624561732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:12,363][19170] Avg episode reward: [(0, '475.170'), (1, '921.820')] [2023-12-27 14:06:12,421][20347] Updated weights for policy 0, policy_version 1220257 (0.0007) [2023-12-27 14:06:12,461][20381] Updated weights for policy 1, policy_version 1219131 (0.0009) [2023-12-27 14:06:12,481][20347] Updated weights for policy 0, policy_version 1220267 (0.0007) [2023-12-27 14:06:12,520][20381] Updated weights for policy 1, policy_version 1219141 (0.0009) [2023-12-27 14:06:12,541][20347] Updated weights for policy 0, policy_version 1220277 (0.0007) [2023-12-27 14:06:12,579][20381] Updated weights for policy 1, policy_version 1219151 (0.0010) [2023-12-27 14:06:12,598][20347] Updated weights for policy 0, policy_version 1220287 (0.0006) [2023-12-27 14:06:13,246][20381] Updated weights for policy 1, policy_version 1219161 (0.0010) [2023-12-27 14:06:13,277][20347] Updated weights for policy 0, policy_version 1220297 (0.0006) [2023-12-27 14:06:13,301][20381] Updated weights for policy 1, policy_version 1219171 (0.0010) [2023-12-27 14:06:13,325][20347] Updated weights for policy 0, policy_version 1220307 (0.0005) [2023-12-27 14:06:13,356][20381] Updated weights for policy 1, policy_version 1219181 (0.0010) [2023-12-27 14:06:13,376][20347] Updated weights for policy 0, policy_version 1220317 (0.0005) [2023-12-27 14:06:13,407][20381] Updated weights for policy 1, policy_version 1219191 (0.0010) [2023-12-27 14:06:14,069][20381] Updated weights for policy 1, policy_version 1219201 (0.0010) [2023-12-27 14:06:14,079][20347] Updated weights for policy 0, policy_version 1220327 (0.0006) [2023-12-27 14:06:14,125][20381] Updated weights for policy 1, policy_version 1219211 (0.0011) [2023-12-27 14:06:14,125][20347] Updated weights for policy 0, policy_version 1220337 (0.0009) [2023-12-27 14:06:14,171][20347] Updated weights for policy 0, policy_version 1220347 (0.0006) [2023-12-27 14:06:14,185][20381] Updated weights for policy 1, policy_version 1219221 (0.0011) [2023-12-27 14:06:14,797][20381] Updated weights for policy 1, policy_version 1219231 (0.0007) [2023-12-27 14:06:14,862][20381] Updated weights for policy 1, policy_version 1219241 (0.0007) [2023-12-27 14:06:14,912][20347] Updated weights for policy 0, policy_version 1220357 (0.0005) [2023-12-27 14:06:14,925][20381] Updated weights for policy 1, policy_version 1219251 (0.0008) [2023-12-27 14:06:14,972][20347] Updated weights for policy 0, policy_version 1220367 (0.0007) [2023-12-27 14:06:15,040][20347] Updated weights for policy 0, policy_version 1220377 (0.0008) [2023-12-27 14:06:15,503][20381] Updated weights for policy 1, policy_version 1219261 (0.0006) [2023-12-27 14:06:15,549][20381] Updated weights for policy 1, policy_version 1219271 (0.0005) [2023-12-27 14:06:15,598][20381] Updated weights for policy 1, policy_version 1219281 (0.0005) [2023-12-27 14:06:15,763][20347] Updated weights for policy 0, policy_version 1220387 (0.0009) [2023-12-27 14:06:15,820][20347] Updated weights for policy 0, policy_version 1220397 (0.0009) [2023-12-27 14:06:15,866][20347] Updated weights for policy 0, policy_version 1220407 (0.0008) [2023-12-27 14:06:16,147][20381] Updated weights for policy 1, policy_version 1219291 (0.0007) [2023-12-27 14:06:16,193][20381] Updated weights for policy 1, policy_version 1219301 (0.0005) [2023-12-27 14:06:16,252][20381] Updated weights for policy 1, policy_version 1219311 (0.0006) [2023-12-27 14:06:16,625][20347] Updated weights for policy 0, policy_version 1220417 (0.0008) [2023-12-27 14:06:16,672][20347] Updated weights for policy 0, policy_version 1220427 (0.0007) [2023-12-27 14:06:16,723][20347] Updated weights for policy 0, policy_version 1220437 (0.0007) [2023-12-27 14:06:16,782][20347] Updated weights for policy 0, policy_version 1220447 (0.0007) [2023-12-27 14:06:16,865][20381] Updated weights for policy 1, policy_version 1219321 (0.0006) [2023-12-27 14:06:16,932][20381] Updated weights for policy 1, policy_version 1219331 (0.0011) [2023-12-27 14:06:16,989][20381] Updated weights for policy 1, policy_version 1219341 (0.0006) [2023-12-27 14:06:17,051][20381] Updated weights for policy 1, policy_version 1219351 (0.0005) [2023-12-27 14:06:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 624672768. Throughput: 0: 10332.2, 1: 10332.7. Samples: 624684784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:17,363][19170] Avg episode reward: [(0, '338.860'), (1, '921.920')] [2023-12-27 14:06:17,511][20347] Updated weights for policy 0, policy_version 1220457 (0.0007) [2023-12-27 14:06:17,569][20347] Updated weights for policy 0, policy_version 1220467 (0.0008) [2023-12-27 14:06:17,623][20347] Updated weights for policy 0, policy_version 1220477 (0.0008) [2023-12-27 14:06:17,656][20381] Updated weights for policy 1, policy_version 1219361 (0.0008) [2023-12-27 14:06:17,716][20381] Updated weights for policy 1, policy_version 1219371 (0.0008) [2023-12-27 14:06:17,766][20381] Updated weights for policy 1, policy_version 1219381 (0.0006) [2023-12-27 14:06:18,327][20347] Updated weights for policy 0, policy_version 1220487 (0.0007) [2023-12-27 14:06:18,393][20347] Updated weights for policy 0, policy_version 1220497 (0.0007) [2023-12-27 14:06:18,453][20347] Updated weights for policy 0, policy_version 1220507 (0.0007) [2023-12-27 14:06:18,467][20381] Updated weights for policy 1, policy_version 1219391 (0.0008) [2023-12-27 14:06:18,523][20381] Updated weights for policy 1, policy_version 1219401 (0.0005) [2023-12-27 14:06:18,580][20381] Updated weights for policy 1, policy_version 1219411 (0.0010) [2023-12-27 14:06:19,124][20347] Updated weights for policy 0, policy_version 1220517 (0.0008) [2023-12-27 14:06:19,171][20347] Updated weights for policy 0, policy_version 1220527 (0.0007) [2023-12-27 14:06:19,228][20347] Updated weights for policy 0, policy_version 1220537 (0.0008) [2023-12-27 14:06:19,273][20381] Updated weights for policy 1, policy_version 1219421 (0.0009) [2023-12-27 14:06:19,328][20381] Updated weights for policy 1, policy_version 1219431 (0.0007) [2023-12-27 14:06:19,398][20381] Updated weights for policy 1, policy_version 1219441 (0.0009) [2023-12-27 14:06:19,995][20347] Updated weights for policy 0, policy_version 1220547 (0.0008) [2023-12-27 14:06:20,044][20347] Updated weights for policy 0, policy_version 1220557 (0.0010) [2023-12-27 14:06:20,103][20347] Updated weights for policy 0, policy_version 1220567 (0.0010) [2023-12-27 14:06:20,140][20381] Updated weights for policy 1, policy_version 1219451 (0.0007) [2023-12-27 14:06:20,198][20381] Updated weights for policy 1, policy_version 1219461 (0.0008) [2023-12-27 14:06:20,255][20381] Updated weights for policy 1, policy_version 1219471 (0.0007) [2023-12-27 14:06:20,772][20347] Updated weights for policy 0, policy_version 1220577 (0.0007) [2023-12-27 14:06:20,827][20347] Updated weights for policy 0, policy_version 1220587 (0.0008) [2023-12-27 14:06:20,889][20347] Updated weights for policy 0, policy_version 1220597 (0.0007) [2023-12-27 14:06:20,950][20347] Updated weights for policy 0, policy_version 1220607 (0.0007) [2023-12-27 14:06:20,977][20381] Updated weights for policy 1, policy_version 1219481 (0.0007) [2023-12-27 14:06:21,036][20381] Updated weights for policy 1, policy_version 1219491 (0.0009) [2023-12-27 14:06:21,108][20381] Updated weights for policy 1, policy_version 1219501 (0.0007) [2023-12-27 14:06:21,178][20381] Updated weights for policy 1, policy_version 1219511 (0.0007) [2023-12-27 14:06:21,717][20347] Updated weights for policy 0, policy_version 1220617 (0.0008) [2023-12-27 14:06:21,787][20347] Updated weights for policy 0, policy_version 1220627 (0.0008) [2023-12-27 14:06:21,854][20347] Updated weights for policy 0, policy_version 1220637 (0.0008) [2023-12-27 14:06:22,010][20381] Updated weights for policy 1, policy_version 1219521 (0.0007) [2023-12-27 14:06:22,072][20381] Updated weights for policy 1, policy_version 1219531 (0.0007) [2023-12-27 14:06:22,135][20381] Updated weights for policy 1, policy_version 1219541 (0.0008) [2023-12-27 14:06:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 624771072. Throughput: 0: 10263.6, 1: 10273.2. Samples: 624743532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:22,363][19170] Avg episode reward: [(0, '364.220'), (1, '960.920')] [2023-12-27 14:06:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001220640_312541184.pth... [2023-12-27 14:06:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001219544_312229888.pth... [2023-12-27 14:06:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001219456_312238080.pth [2023-12-27 14:06:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001218360_311926784.pth [2023-12-27 14:06:22,561][20347] Updated weights for policy 0, policy_version 1220647 (0.0008) [2023-12-27 14:06:22,626][20347] Updated weights for policy 0, policy_version 1220657 (0.0008) [2023-12-27 14:06:22,686][20347] Updated weights for policy 0, policy_version 1220667 (0.0005) [2023-12-27 14:06:22,880][20381] Updated weights for policy 1, policy_version 1219551 (0.0008) [2023-12-27 14:06:22,938][20381] Updated weights for policy 1, policy_version 1219561 (0.0006) [2023-12-27 14:06:22,998][20381] Updated weights for policy 1, policy_version 1219571 (0.0005) [2023-12-27 14:06:23,309][20347] Updated weights for policy 0, policy_version 1220677 (0.0008) [2023-12-27 14:06:23,357][20347] Updated weights for policy 0, policy_version 1220687 (0.0010) [2023-12-27 14:06:23,405][20347] Updated weights for policy 0, policy_version 1220697 (0.0010) [2023-12-27 14:06:23,582][20381] Updated weights for policy 1, policy_version 1219581 (0.0008) [2023-12-27 14:06:23,636][20381] Updated weights for policy 1, policy_version 1219591 (0.0010) [2023-12-27 14:06:23,690][20381] Updated weights for policy 1, policy_version 1219601 (0.0010) [2023-12-27 14:06:24,088][20347] Updated weights for policy 0, policy_version 1220707 (0.0009) [2023-12-27 14:06:24,154][20347] Updated weights for policy 0, policy_version 1220717 (0.0006) [2023-12-27 14:06:24,212][20347] Updated weights for policy 0, policy_version 1220727 (0.0006) [2023-12-27 14:06:24,381][20381] Updated weights for policy 1, policy_version 1219611 (0.0011) [2023-12-27 14:06:24,438][20381] Updated weights for policy 1, policy_version 1219621 (0.0010) [2023-12-27 14:06:24,502][20381] Updated weights for policy 1, policy_version 1219631 (0.0006) [2023-12-27 14:06:24,875][20347] Updated weights for policy 0, policy_version 1220737 (0.0010) [2023-12-27 14:06:24,937][20347] Updated weights for policy 0, policy_version 1220747 (0.0011) [2023-12-27 14:06:24,986][20347] Updated weights for policy 0, policy_version 1220757 (0.0010) [2023-12-27 14:06:25,051][20347] Updated weights for policy 0, policy_version 1220767 (0.0010) [2023-12-27 14:06:25,137][20381] Updated weights for policy 1, policy_version 1219641 (0.0008) [2023-12-27 14:06:25,193][20381] Updated weights for policy 1, policy_version 1219651 (0.0008) [2023-12-27 14:06:25,248][20381] Updated weights for policy 1, policy_version 1219661 (0.0008) [2023-12-27 14:06:25,308][20381] Updated weights for policy 1, policy_version 1219671 (0.0007) [2023-12-27 14:06:25,721][20347] Updated weights for policy 0, policy_version 1220777 (0.0010) [2023-12-27 14:06:25,768][20347] Updated weights for policy 0, policy_version 1220787 (0.0010) [2023-12-27 14:06:25,819][20347] Updated weights for policy 0, policy_version 1220797 (0.0010) [2023-12-27 14:06:25,941][20381] Updated weights for policy 1, policy_version 1219681 (0.0007) [2023-12-27 14:06:25,995][20381] Updated weights for policy 1, policy_version 1219691 (0.0007) [2023-12-27 14:06:26,050][20381] Updated weights for policy 1, policy_version 1219701 (0.0008) [2023-12-27 14:06:26,520][20347] Updated weights for policy 0, policy_version 1220807 (0.0010) [2023-12-27 14:06:26,581][20347] Updated weights for policy 0, policy_version 1220817 (0.0010) [2023-12-27 14:06:26,638][20347] Updated weights for policy 0, policy_version 1220827 (0.0010) [2023-12-27 14:06:26,672][20381] Updated weights for policy 1, policy_version 1219711 (0.0010) [2023-12-27 14:06:26,728][20381] Updated weights for policy 1, policy_version 1219721 (0.0010) [2023-12-27 14:06:26,784][20381] Updated weights for policy 1, policy_version 1219731 (0.0010) [2023-12-27 14:06:27,242][20347] Updated weights for policy 0, policy_version 1220837 (0.0008) [2023-12-27 14:06:27,303][20347] Updated weights for policy 0, policy_version 1220847 (0.0006) [2023-12-27 14:06:27,353][20347] Updated weights for policy 0, policy_version 1220857 (0.0006) [2023-12-27 14:06:27,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20479.9, 300 sec: 20604.9). Total num frames: 624869376. Throughput: 0: 10261.8, 1: 10251.1. Samples: 624863296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:27,364][19170] Avg episode reward: [(0, '315.640'), (1, '912.030')] [2023-12-27 14:06:27,426][20381] Updated weights for policy 1, policy_version 1219741 (0.0009) [2023-12-27 14:06:27,479][20381] Updated weights for policy 1, policy_version 1219751 (0.0006) [2023-12-27 14:06:27,534][20381] Updated weights for policy 1, policy_version 1219761 (0.0009) [2023-12-27 14:06:27,927][20347] Updated weights for policy 0, policy_version 1220867 (0.0009) [2023-12-27 14:06:27,973][20347] Updated weights for policy 0, policy_version 1220877 (0.0008) [2023-12-27 14:06:28,021][20347] Updated weights for policy 0, policy_version 1220887 (0.0010) [2023-12-27 14:06:28,136][20381] Updated weights for policy 1, policy_version 1219771 (0.0005) [2023-12-27 14:06:28,191][20381] Updated weights for policy 1, policy_version 1219781 (0.0005) [2023-12-27 14:06:28,234][20381] Updated weights for policy 1, policy_version 1219791 (0.0005) [2023-12-27 14:06:28,703][20347] Updated weights for policy 0, policy_version 1220897 (0.0010) [2023-12-27 14:06:28,760][20347] Updated weights for policy 0, policy_version 1220907 (0.0011) [2023-12-27 14:06:28,821][20347] Updated weights for policy 0, policy_version 1220917 (0.0010) [2023-12-27 14:06:28,886][20347] Updated weights for policy 0, policy_version 1220927 (0.0011) [2023-12-27 14:06:28,892][20381] Updated weights for policy 1, policy_version 1219801 (0.0005) [2023-12-27 14:06:28,954][20381] Updated weights for policy 1, policy_version 1219811 (0.0005) [2023-12-27 14:06:29,010][20381] Updated weights for policy 1, policy_version 1219821 (0.0005) [2023-12-27 14:06:29,061][20381] Updated weights for policy 1, policy_version 1219831 (0.0009) [2023-12-27 14:06:29,554][20347] Updated weights for policy 0, policy_version 1220937 (0.0006) [2023-12-27 14:06:29,607][20347] Updated weights for policy 0, policy_version 1220947 (0.0005) [2023-12-27 14:06:29,658][20347] Updated weights for policy 0, policy_version 1220957 (0.0005) [2023-12-27 14:06:29,750][20381] Updated weights for policy 1, policy_version 1219841 (0.0011) [2023-12-27 14:06:29,812][20381] Updated weights for policy 1, policy_version 1219851 (0.0011) [2023-12-27 14:06:29,878][20381] Updated weights for policy 1, policy_version 1219861 (0.0009) [2023-12-27 14:06:30,272][20347] Updated weights for policy 0, policy_version 1220967 (0.0008) [2023-12-27 14:06:30,330][20347] Updated weights for policy 0, policy_version 1220977 (0.0010) [2023-12-27 14:06:30,389][20347] Updated weights for policy 0, policy_version 1220987 (0.0010) [2023-12-27 14:06:30,568][20381] Updated weights for policy 1, policy_version 1219871 (0.0006) [2023-12-27 14:06:30,623][20381] Updated weights for policy 1, policy_version 1219881 (0.0005) [2023-12-27 14:06:30,673][20381] Updated weights for policy 1, policy_version 1219891 (0.0005) [2023-12-27 14:06:31,066][20347] Updated weights for policy 0, policy_version 1220997 (0.0010) [2023-12-27 14:06:31,130][20347] Updated weights for policy 0, policy_version 1221007 (0.0011) [2023-12-27 14:06:31,189][20347] Updated weights for policy 0, policy_version 1221017 (0.0007) [2023-12-27 14:06:31,228][20381] Updated weights for policy 1, policy_version 1219901 (0.0008) [2023-12-27 14:06:31,290][20381] Updated weights for policy 1, policy_version 1219911 (0.0011) [2023-12-27 14:06:31,357][20381] Updated weights for policy 1, policy_version 1219921 (0.0010) [2023-12-27 14:06:31,892][20347] Updated weights for policy 0, policy_version 1221027 (0.0007) [2023-12-27 14:06:31,945][20347] Updated weights for policy 0, policy_version 1221037 (0.0007) [2023-12-27 14:06:32,002][20381] Updated weights for policy 1, policy_version 1219931 (0.0010) [2023-12-27 14:06:32,002][20347] Updated weights for policy 0, policy_version 1221047 (0.0008) [2023-12-27 14:06:32,067][20381] Updated weights for policy 1, policy_version 1219941 (0.0006) [2023-12-27 14:06:32,136][20381] Updated weights for policy 1, policy_version 1219951 (0.0006) [2023-12-27 14:06:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 624984064. Throughput: 0: 10298.7, 1: 10284.7. Samples: 624992488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:32,363][19170] Avg episode reward: [(0, '609.620'), (1, '843.680')] [2023-12-27 14:06:32,684][20381] Updated weights for policy 1, policy_version 1219961 (0.0005) [2023-12-27 14:06:32,699][20347] Updated weights for policy 0, policy_version 1221057 (0.0006) [2023-12-27 14:06:32,746][20381] Updated weights for policy 1, policy_version 1219971 (0.0007) [2023-12-27 14:06:32,759][20347] Updated weights for policy 0, policy_version 1221067 (0.0006) [2023-12-27 14:06:32,804][20381] Updated weights for policy 1, policy_version 1219981 (0.0005) [2023-12-27 14:06:32,812][20347] Updated weights for policy 0, policy_version 1221077 (0.0008) [2023-12-27 14:06:32,863][20381] Updated weights for policy 1, policy_version 1219991 (0.0006) [2023-12-27 14:06:32,864][20347] Updated weights for policy 0, policy_version 1221087 (0.0008) [2023-12-27 14:06:33,463][20381] Updated weights for policy 1, policy_version 1220001 (0.0007) [2023-12-27 14:06:33,498][20347] Updated weights for policy 0, policy_version 1221097 (0.0010) [2023-12-27 14:06:33,525][20381] Updated weights for policy 1, policy_version 1220011 (0.0006) [2023-12-27 14:06:33,549][20347] Updated weights for policy 0, policy_version 1221107 (0.0010) [2023-12-27 14:06:33,570][20381] Updated weights for policy 1, policy_version 1220021 (0.0006) [2023-12-27 14:06:33,602][20347] Updated weights for policy 0, policy_version 1221117 (0.0008) [2023-12-27 14:06:34,232][20347] Updated weights for policy 0, policy_version 1221127 (0.0008) [2023-12-27 14:06:34,288][20347] Updated weights for policy 0, policy_version 1221137 (0.0006) [2023-12-27 14:06:34,297][20381] Updated weights for policy 1, policy_version 1220031 (0.0007) [2023-12-27 14:06:34,345][20347] Updated weights for policy 0, policy_version 1221147 (0.0007) [2023-12-27 14:06:34,363][20381] Updated weights for policy 1, policy_version 1220041 (0.0008) [2023-12-27 14:06:34,426][20381] Updated weights for policy 1, policy_version 1220051 (0.0011) [2023-12-27 14:06:35,032][20347] Updated weights for policy 0, policy_version 1221157 (0.0010) [2023-12-27 14:06:35,079][20381] Updated weights for policy 1, policy_version 1220061 (0.0011) [2023-12-27 14:06:35,084][20347] Updated weights for policy 0, policy_version 1221167 (0.0010) [2023-12-27 14:06:35,132][20381] Updated weights for policy 1, policy_version 1220071 (0.0007) [2023-12-27 14:06:35,142][20347] Updated weights for policy 0, policy_version 1221177 (0.0010) [2023-12-27 14:06:35,191][20381] Updated weights for policy 1, policy_version 1220081 (0.0005) [2023-12-27 14:06:35,794][20347] Updated weights for policy 0, policy_version 1221187 (0.0009) [2023-12-27 14:06:35,850][20347] Updated weights for policy 0, policy_version 1221197 (0.0005) [2023-12-27 14:06:35,861][20381] Updated weights for policy 1, policy_version 1220091 (0.0009) [2023-12-27 14:06:35,900][20347] Updated weights for policy 0, policy_version 1221207 (0.0008) [2023-12-27 14:06:35,917][20381] Updated weights for policy 1, policy_version 1220101 (0.0010) [2023-12-27 14:06:35,975][20381] Updated weights for policy 1, policy_version 1220111 (0.0010) [2023-12-27 14:06:36,540][20347] Updated weights for policy 0, policy_version 1221217 (0.0006) [2023-12-27 14:06:36,596][20347] Updated weights for policy 0, policy_version 1221227 (0.0006) [2023-12-27 14:06:36,628][20381] Updated weights for policy 1, policy_version 1220121 (0.0011) [2023-12-27 14:06:36,660][20347] Updated weights for policy 0, policy_version 1221237 (0.0006) [2023-12-27 14:06:36,682][20381] Updated weights for policy 1, policy_version 1220131 (0.0011) [2023-12-27 14:06:36,714][20347] Updated weights for policy 0, policy_version 1221247 (0.0006) [2023-12-27 14:06:36,738][20381] Updated weights for policy 1, policy_version 1220141 (0.0011) [2023-12-27 14:06:36,797][20381] Updated weights for policy 1, policy_version 1220151 (0.0011) [2023-12-27 14:06:37,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 625082368. Throughput: 0: 10297.7, 1: 10299.4. Samples: 625055792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:37,363][19170] Avg episode reward: [(0, '633.780'), (1, '882.830')] [2023-12-27 14:06:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001220152_312385536.pth... [2023-12-27 14:06:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001218936_312074240.pth [2023-12-27 14:06:37,389][20347] Updated weights for policy 0, policy_version 1221257 (0.0005) [2023-12-27 14:06:37,447][20347] Updated weights for policy 0, policy_version 1221267 (0.0006) [2023-12-27 14:06:37,487][20381] Updated weights for policy 1, policy_version 1220161 (0.0011) [2023-12-27 14:06:37,497][20347] Updated weights for policy 0, policy_version 1221277 (0.0006) [2023-12-27 14:06:37,509][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001221280_312705024.pth... [2023-12-27 14:06:37,512][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001220064_312393728.pth [2023-12-27 14:06:37,543][20381] Updated weights for policy 1, policy_version 1220171 (0.0010) [2023-12-27 14:06:37,603][20381] Updated weights for policy 1, policy_version 1220181 (0.0011) [2023-12-27 14:06:38,111][20347] Updated weights for policy 0, policy_version 1221287 (0.0006) [2023-12-27 14:06:38,165][20347] Updated weights for policy 0, policy_version 1221297 (0.0005) [2023-12-27 14:06:38,224][20347] Updated weights for policy 0, policy_version 1221307 (0.0007) [2023-12-27 14:06:38,288][20381] Updated weights for policy 1, policy_version 1220191 (0.0011) [2023-12-27 14:06:38,340][20381] Updated weights for policy 1, policy_version 1220201 (0.0011) [2023-12-27 14:06:38,405][20381] Updated weights for policy 1, policy_version 1220211 (0.0008) [2023-12-27 14:06:38,897][20347] Updated weights for policy 0, policy_version 1221317 (0.0007) [2023-12-27 14:06:38,951][20347] Updated weights for policy 0, policy_version 1221327 (0.0007) [2023-12-27 14:06:39,008][20347] Updated weights for policy 0, policy_version 1221337 (0.0007) [2023-12-27 14:06:39,103][20381] Updated weights for policy 1, policy_version 1220221 (0.0010) [2023-12-27 14:06:39,171][20381] Updated weights for policy 1, policy_version 1220231 (0.0009) [2023-12-27 14:06:39,228][20381] Updated weights for policy 1, policy_version 1220241 (0.0008) [2023-12-27 14:06:39,751][20347] Updated weights for policy 0, policy_version 1221347 (0.0007) [2023-12-27 14:06:39,800][20347] Updated weights for policy 0, policy_version 1221357 (0.0007) [2023-12-27 14:06:39,874][20347] Updated weights for policy 0, policy_version 1221367 (0.0007) [2023-12-27 14:06:39,919][20381] Updated weights for policy 1, policy_version 1220251 (0.0011) [2023-12-27 14:06:39,981][20381] Updated weights for policy 1, policy_version 1220261 (0.0007) [2023-12-27 14:06:40,038][20381] Updated weights for policy 1, policy_version 1220271 (0.0009) [2023-12-27 14:06:40,526][20347] Updated weights for policy 0, policy_version 1221377 (0.0007) [2023-12-27 14:06:40,586][20347] Updated weights for policy 0, policy_version 1221387 (0.0007) [2023-12-27 14:06:40,648][20347] Updated weights for policy 0, policy_version 1221397 (0.0007) [2023-12-27 14:06:40,709][20347] Updated weights for policy 0, policy_version 1221407 (0.0007) [2023-12-27 14:06:40,748][20381] Updated weights for policy 1, policy_version 1220281 (0.0010) [2023-12-27 14:06:40,812][20381] Updated weights for policy 1, policy_version 1220291 (0.0006) [2023-12-27 14:06:40,878][20381] Updated weights for policy 1, policy_version 1220301 (0.0006) [2023-12-27 14:06:40,937][20381] Updated weights for policy 1, policy_version 1220311 (0.0009) [2023-12-27 14:06:41,490][20347] Updated weights for policy 0, policy_version 1221417 (0.0007) [2023-12-27 14:06:41,546][20347] Updated weights for policy 0, policy_version 1221427 (0.0008) [2023-12-27 14:06:41,608][20347] Updated weights for policy 0, policy_version 1221437 (0.0007) [2023-12-27 14:06:41,704][20381] Updated weights for policy 1, policy_version 1220321 (0.0008) [2023-12-27 14:06:41,772][20381] Updated weights for policy 1, policy_version 1220331 (0.0009) [2023-12-27 14:06:41,834][20381] Updated weights for policy 1, policy_version 1220341 (0.0008) [2023-12-27 14:06:42,345][20347] Updated weights for policy 0, policy_version 1221447 (0.0007) [2023-12-27 14:06:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 625180672. Throughput: 0: 10257.1, 1: 10269.0. Samples: 625176728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:42,364][19170] Avg episode reward: [(0, '510.380'), (1, '853.520')] [2023-12-27 14:06:42,410][20347] Updated weights for policy 0, policy_version 1221457 (0.0007) [2023-12-27 14:06:42,459][20347] Updated weights for policy 0, policy_version 1221467 (0.0007) [2023-12-27 14:06:42,530][20381] Updated weights for policy 1, policy_version 1220351 (0.0008) [2023-12-27 14:06:42,580][20381] Updated weights for policy 1, policy_version 1220361 (0.0008) [2023-12-27 14:06:42,639][20381] Updated weights for policy 1, policy_version 1220371 (0.0005) [2023-12-27 14:06:43,189][20347] Updated weights for policy 0, policy_version 1221477 (0.0007) [2023-12-27 14:06:43,234][20381] Updated weights for policy 1, policy_version 1220381 (0.0006) [2023-12-27 14:06:43,239][20347] Updated weights for policy 0, policy_version 1221487 (0.0008) [2023-12-27 14:06:43,285][20381] Updated weights for policy 1, policy_version 1220391 (0.0006) [2023-12-27 14:06:43,294][20347] Updated weights for policy 0, policy_version 1221497 (0.0008) [2023-12-27 14:06:43,338][20381] Updated weights for policy 1, policy_version 1220401 (0.0006) [2023-12-27 14:06:43,981][20347] Updated weights for policy 0, policy_version 1221507 (0.0006) [2023-12-27 14:06:44,029][20347] Updated weights for policy 0, policy_version 1221517 (0.0008) [2023-12-27 14:06:44,047][20381] Updated weights for policy 1, policy_version 1220411 (0.0007) [2023-12-27 14:06:44,084][20347] Updated weights for policy 0, policy_version 1221527 (0.0009) [2023-12-27 14:06:44,098][20381] Updated weights for policy 1, policy_version 1220421 (0.0010) [2023-12-27 14:06:44,149][20381] Updated weights for policy 1, policy_version 1220431 (0.0006) [2023-12-27 14:06:44,769][20347] Updated weights for policy 0, policy_version 1221537 (0.0007) [2023-12-27 14:06:44,839][20347] Updated weights for policy 0, policy_version 1221547 (0.0008) [2023-12-27 14:06:44,842][20381] Updated weights for policy 1, policy_version 1220441 (0.0008) [2023-12-27 14:06:44,902][20381] Updated weights for policy 1, policy_version 1220451 (0.0007) [2023-12-27 14:06:44,905][20347] Updated weights for policy 0, policy_version 1221557 (0.0007) [2023-12-27 14:06:44,959][20381] Updated weights for policy 1, policy_version 1220461 (0.0007) [2023-12-27 14:06:44,970][20347] Updated weights for policy 0, policy_version 1221567 (0.0008) [2023-12-27 14:06:45,024][20381] Updated weights for policy 1, policy_version 1220471 (0.0006) [2023-12-27 14:06:45,624][20347] Updated weights for policy 0, policy_version 1221577 (0.0008) [2023-12-27 14:06:45,675][20347] Updated weights for policy 0, policy_version 1221587 (0.0007) [2023-12-27 14:06:45,711][20381] Updated weights for policy 1, policy_version 1220481 (0.0006) [2023-12-27 14:06:45,733][20347] Updated weights for policy 0, policy_version 1221597 (0.0008) [2023-12-27 14:06:45,768][20381] Updated weights for policy 1, policy_version 1220491 (0.0008) [2023-12-27 14:06:45,826][20381] Updated weights for policy 1, policy_version 1220501 (0.0007) [2023-12-27 14:06:46,378][20347] Updated weights for policy 0, policy_version 1221607 (0.0006) [2023-12-27 14:06:46,418][20381] Updated weights for policy 1, policy_version 1220511 (0.0009) [2023-12-27 14:06:46,429][20347] Updated weights for policy 0, policy_version 1221617 (0.0007) [2023-12-27 14:06:46,465][20381] Updated weights for policy 1, policy_version 1220521 (0.0009) [2023-12-27 14:06:46,483][20347] Updated weights for policy 0, policy_version 1221627 (0.0005) [2023-12-27 14:06:46,510][20381] Updated weights for policy 1, policy_version 1220531 (0.0010) [2023-12-27 14:06:47,083][20347] Updated weights for policy 0, policy_version 1221637 (0.0006) [2023-12-27 14:06:47,087][20381] Updated weights for policy 1, policy_version 1220541 (0.0008) [2023-12-27 14:06:47,139][20381] Updated weights for policy 1, policy_version 1220551 (0.0005) [2023-12-27 14:06:47,140][20347] Updated weights for policy 0, policy_version 1221647 (0.0011) [2023-12-27 14:06:47,185][20347] Updated weights for policy 0, policy_version 1221657 (0.0010) [2023-12-27 14:06:47,194][20381] Updated weights for policy 1, policy_version 1220561 (0.0005) [2023-12-27 14:06:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 625295360. Throughput: 0: 10230.1, 1: 10307.7. Samples: 625300068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:47,364][19170] Avg episode reward: [(0, '543.500'), (1, '843.820')] [2023-12-27 14:06:47,769][20381] Updated weights for policy 1, policy_version 1220571 (0.0007) [2023-12-27 14:06:47,827][20381] Updated weights for policy 1, policy_version 1220581 (0.0010) [2023-12-27 14:06:47,850][20347] Updated weights for policy 0, policy_version 1221667 (0.0010) [2023-12-27 14:06:47,883][20381] Updated weights for policy 1, policy_version 1220591 (0.0011) [2023-12-27 14:06:47,919][20347] Updated weights for policy 0, policy_version 1221677 (0.0008) [2023-12-27 14:06:47,977][20347] Updated weights for policy 0, policy_version 1221687 (0.0007) [2023-12-27 14:06:48,590][20381] Updated weights for policy 1, policy_version 1220601 (0.0010) [2023-12-27 14:06:48,646][20381] Updated weights for policy 1, policy_version 1220611 (0.0011) [2023-12-27 14:06:48,648][20347] Updated weights for policy 0, policy_version 1221697 (0.0007) [2023-12-27 14:06:48,700][20347] Updated weights for policy 0, policy_version 1221707 (0.0005) [2023-12-27 14:06:48,702][20381] Updated weights for policy 1, policy_version 1220621 (0.0011) [2023-12-27 14:06:48,751][20347] Updated weights for policy 0, policy_version 1221717 (0.0005) [2023-12-27 14:06:48,761][20381] Updated weights for policy 1, policy_version 1220631 (0.0009) [2023-12-27 14:06:48,799][20347] Updated weights for policy 0, policy_version 1221727 (0.0005) [2023-12-27 14:06:49,435][20347] Updated weights for policy 0, policy_version 1221737 (0.0008) [2023-12-27 14:06:49,457][20381] Updated weights for policy 1, policy_version 1220641 (0.0007) [2023-12-27 14:06:49,493][20347] Updated weights for policy 0, policy_version 1221747 (0.0008) [2023-12-27 14:06:49,506][20381] Updated weights for policy 1, policy_version 1220651 (0.0006) [2023-12-27 14:06:49,548][20347] Updated weights for policy 0, policy_version 1221757 (0.0007) [2023-12-27 14:06:49,558][20381] Updated weights for policy 1, policy_version 1220661 (0.0005) [2023-12-27 14:06:50,226][20347] Updated weights for policy 0, policy_version 1221767 (0.0006) [2023-12-27 14:06:50,246][20381] Updated weights for policy 1, policy_version 1220671 (0.0006) [2023-12-27 14:06:50,284][20347] Updated weights for policy 0, policy_version 1221777 (0.0006) [2023-12-27 14:06:50,307][20381] Updated weights for policy 1, policy_version 1220681 (0.0006) [2023-12-27 14:06:50,342][20347] Updated weights for policy 0, policy_version 1221787 (0.0007) [2023-12-27 14:06:50,371][20381] Updated weights for policy 1, policy_version 1220691 (0.0006) [2023-12-27 14:06:50,962][20347] Updated weights for policy 0, policy_version 1221797 (0.0006) [2023-12-27 14:06:50,964][20381] Updated weights for policy 1, policy_version 1220701 (0.0006) [2023-12-27 14:06:51,024][20347] Updated weights for policy 0, policy_version 1221807 (0.0007) [2023-12-27 14:06:51,027][20381] Updated weights for policy 1, policy_version 1220711 (0.0008) [2023-12-27 14:06:51,088][20347] Updated weights for policy 0, policy_version 1221817 (0.0009) [2023-12-27 14:06:51,090][20381] Updated weights for policy 1, policy_version 1220721 (0.0007) [2023-12-27 14:06:51,754][20347] Updated weights for policy 0, policy_version 1221827 (0.0008) [2023-12-27 14:06:51,815][20347] Updated weights for policy 0, policy_version 1221837 (0.0006) [2023-12-27 14:06:51,830][20381] Updated weights for policy 1, policy_version 1220731 (0.0008) [2023-12-27 14:06:51,876][20347] Updated weights for policy 0, policy_version 1221847 (0.0007) [2023-12-27 14:06:51,891][20381] Updated weights for policy 1, policy_version 1220741 (0.0006) [2023-12-27 14:06:51,943][20381] Updated weights for policy 1, policy_version 1220751 (0.0008) [2023-12-27 14:06:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 625393664. Throughput: 0: 10225.2, 1: 10336.5. Samples: 625362712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:52,363][19170] Avg episode reward: [(0, '597.790'), (1, '824.420')] [2023-12-27 14:06:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001220760_312541184.pth... [2023-12-27 14:06:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001221856_312852480.pth... [2023-12-27 14:06:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001219544_312229888.pth [2023-12-27 14:06:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001220640_312541184.pth [2023-12-27 14:06:52,542][20347] Updated weights for policy 0, policy_version 1221857 (0.0006) [2023-12-27 14:06:52,599][20347] Updated weights for policy 0, policy_version 1221867 (0.0009) [2023-12-27 14:06:52,655][20381] Updated weights for policy 1, policy_version 1220761 (0.0008) [2023-12-27 14:06:52,665][20347] Updated weights for policy 0, policy_version 1221877 (0.0010) [2023-12-27 14:06:52,709][20381] Updated weights for policy 1, policy_version 1220771 (0.0006) [2023-12-27 14:06:52,724][20347] Updated weights for policy 0, policy_version 1221887 (0.0010) [2023-12-27 14:06:52,768][20381] Updated weights for policy 1, policy_version 1220781 (0.0008) [2023-12-27 14:06:52,828][20381] Updated weights for policy 1, policy_version 1220791 (0.0007) [2023-12-27 14:06:53,388][20347] Updated weights for policy 0, policy_version 1221897 (0.0007) [2023-12-27 14:06:53,445][20347] Updated weights for policy 0, policy_version 1221907 (0.0005) [2023-12-27 14:06:53,503][20347] Updated weights for policy 0, policy_version 1221917 (0.0006) [2023-12-27 14:06:53,537][20381] Updated weights for policy 1, policy_version 1220801 (0.0009) [2023-12-27 14:06:53,589][20381] Updated weights for policy 1, policy_version 1220811 (0.0007) [2023-12-27 14:06:53,636][20381] Updated weights for policy 1, policy_version 1220821 (0.0007) [2023-12-27 14:06:54,044][20347] Updated weights for policy 0, policy_version 1221927 (0.0007) [2023-12-27 14:06:54,095][20347] Updated weights for policy 0, policy_version 1221937 (0.0005) [2023-12-27 14:06:54,157][20347] Updated weights for policy 0, policy_version 1221947 (0.0009) [2023-12-27 14:06:54,396][20381] Updated weights for policy 1, policy_version 1220831 (0.0006) [2023-12-27 14:06:54,452][20381] Updated weights for policy 1, policy_version 1220841 (0.0005) [2023-12-27 14:06:54,509][20381] Updated weights for policy 1, policy_version 1220851 (0.0005) [2023-12-27 14:06:54,860][20347] Updated weights for policy 0, policy_version 1221957 (0.0010) [2023-12-27 14:06:54,915][20347] Updated weights for policy 0, policy_version 1221967 (0.0010) [2023-12-27 14:06:54,978][20347] Updated weights for policy 0, policy_version 1221977 (0.0010) [2023-12-27 14:06:55,151][20381] Updated weights for policy 1, policy_version 1220861 (0.0008) [2023-12-27 14:06:55,207][20381] Updated weights for policy 1, policy_version 1220871 (0.0011) [2023-12-27 14:06:55,263][20381] Updated weights for policy 1, policy_version 1220881 (0.0011) [2023-12-27 14:06:55,690][20347] Updated weights for policy 0, policy_version 1221987 (0.0011) [2023-12-27 14:06:55,746][20347] Updated weights for policy 0, policy_version 1221997 (0.0010) [2023-12-27 14:06:55,797][20347] Updated weights for policy 0, policy_version 1222007 (0.0010) [2023-12-27 14:06:55,885][20381] Updated weights for policy 1, policy_version 1220891 (0.0011) [2023-12-27 14:06:55,936][20381] Updated weights for policy 1, policy_version 1220901 (0.0009) [2023-12-27 14:06:55,986][20381] Updated weights for policy 1, policy_version 1220911 (0.0005) [2023-12-27 14:06:56,458][20347] Updated weights for policy 0, policy_version 1222017 (0.0010) [2023-12-27 14:06:56,510][20347] Updated weights for policy 0, policy_version 1222027 (0.0010) [2023-12-27 14:06:56,569][20347] Updated weights for policy 0, policy_version 1222037 (0.0010) [2023-12-27 14:06:56,629][20347] Updated weights for policy 0, policy_version 1222047 (0.0010) [2023-12-27 14:06:56,670][20381] Updated weights for policy 1, policy_version 1220921 (0.0008) [2023-12-27 14:06:56,716][20381] Updated weights for policy 1, policy_version 1220931 (0.0010) [2023-12-27 14:06:56,764][20381] Updated weights for policy 1, policy_version 1220941 (0.0010) [2023-12-27 14:06:56,811][20381] Updated weights for policy 1, policy_version 1220951 (0.0010) [2023-12-27 14:06:57,199][20347] Updated weights for policy 0, policy_version 1222057 (0.0010) [2023-12-27 14:06:57,248][20347] Updated weights for policy 0, policy_version 1222067 (0.0010) [2023-12-27 14:06:57,296][20347] Updated weights for policy 0, policy_version 1222077 (0.0010) [2023-12-27 14:06:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 625500160. Throughput: 0: 10190.6, 1: 10345.3. Samples: 625485844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:06:57,363][19170] Avg episode reward: [(0, '651.450'), (1, '843.770')] [2023-12-27 14:06:57,482][20381] Updated weights for policy 1, policy_version 1220961 (0.0007) [2023-12-27 14:06:57,533][20381] Updated weights for policy 1, policy_version 1220971 (0.0007) [2023-12-27 14:06:57,583][20381] Updated weights for policy 1, policy_version 1220981 (0.0007) [2023-12-27 14:06:57,970][20347] Updated weights for policy 0, policy_version 1222087 (0.0007) [2023-12-27 14:06:58,019][20347] Updated weights for policy 0, policy_version 1222097 (0.0008) [2023-12-27 14:06:58,069][20347] Updated weights for policy 0, policy_version 1222107 (0.0010) [2023-12-27 14:06:58,291][20381] Updated weights for policy 1, policy_version 1220991 (0.0006) [2023-12-27 14:06:58,355][20381] Updated weights for policy 1, policy_version 1221001 (0.0009) [2023-12-27 14:06:58,423][20381] Updated weights for policy 1, policy_version 1221011 (0.0008) [2023-12-27 14:06:58,767][20347] Updated weights for policy 0, policy_version 1222117 (0.0009) [2023-12-27 14:06:58,835][20347] Updated weights for policy 0, policy_version 1222127 (0.0007) [2023-12-27 14:06:58,900][20347] Updated weights for policy 0, policy_version 1222137 (0.0009) [2023-12-27 14:06:59,168][20381] Updated weights for policy 1, policy_version 1221021 (0.0009) [2023-12-27 14:06:59,234][20381] Updated weights for policy 1, policy_version 1221031 (0.0009) [2023-12-27 14:06:59,296][20381] Updated weights for policy 1, policy_version 1221041 (0.0008) [2023-12-27 14:06:59,638][20347] Updated weights for policy 0, policy_version 1222147 (0.0008) [2023-12-27 14:06:59,699][20347] Updated weights for policy 0, policy_version 1222157 (0.0007) [2023-12-27 14:06:59,754][20347] Updated weights for policy 0, policy_version 1222167 (0.0007) [2023-12-27 14:07:00,011][20381] Updated weights for policy 1, policy_version 1221051 (0.0010) [2023-12-27 14:07:00,075][20381] Updated weights for policy 1, policy_version 1221061 (0.0010) [2023-12-27 14:07:00,136][20381] Updated weights for policy 1, policy_version 1221071 (0.0010) [2023-12-27 14:07:00,428][20347] Updated weights for policy 0, policy_version 1222177 (0.0008) [2023-12-27 14:07:00,477][20347] Updated weights for policy 0, policy_version 1222187 (0.0007) [2023-12-27 14:07:00,525][20347] Updated weights for policy 0, policy_version 1222197 (0.0007) [2023-12-27 14:07:00,578][20347] Updated weights for policy 0, policy_version 1222207 (0.0007) [2023-12-27 14:07:00,771][20381] Updated weights for policy 1, policy_version 1221081 (0.0010) [2023-12-27 14:07:00,819][20381] Updated weights for policy 1, policy_version 1221091 (0.0005) [2023-12-27 14:07:00,865][20381] Updated weights for policy 1, policy_version 1221101 (0.0005) [2023-12-27 14:07:00,914][20381] Updated weights for policy 1, policy_version 1221111 (0.0010) [2023-12-27 14:07:01,176][20347] Updated weights for policy 0, policy_version 1222217 (0.0008) [2023-12-27 14:07:01,235][20347] Updated weights for policy 0, policy_version 1222227 (0.0007) [2023-12-27 14:07:01,289][20347] Updated weights for policy 0, policy_version 1222237 (0.0008) [2023-12-27 14:07:01,582][20381] Updated weights for policy 1, policy_version 1221121 (0.0010) [2023-12-27 14:07:01,641][20381] Updated weights for policy 1, policy_version 1221131 (0.0009) [2023-12-27 14:07:01,693][20381] Updated weights for policy 1, policy_version 1221141 (0.0006) [2023-12-27 14:07:01,977][20347] Updated weights for policy 0, policy_version 1222247 (0.0006) [2023-12-27 14:07:02,034][20347] Updated weights for policy 0, policy_version 1222257 (0.0005) [2023-12-27 14:07:02,091][20347] Updated weights for policy 0, policy_version 1222267 (0.0005) [2023-12-27 14:07:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 625598464. Throughput: 0: 10297.0, 1: 10236.4. Samples: 625608784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:02,363][19170] Avg episode reward: [(0, '728.710'), (1, '814.360')] [2023-12-27 14:07:02,423][20381] Updated weights for policy 1, policy_version 1221151 (0.0008) [2023-12-27 14:07:02,480][20381] Updated weights for policy 1, policy_version 1221161 (0.0006) [2023-12-27 14:07:02,539][20381] Updated weights for policy 1, policy_version 1221171 (0.0010) [2023-12-27 14:07:02,780][20347] Updated weights for policy 0, policy_version 1222277 (0.0007) [2023-12-27 14:07:02,828][20347] Updated weights for policy 0, policy_version 1222287 (0.0007) [2023-12-27 14:07:02,879][20347] Updated weights for policy 0, policy_version 1222297 (0.0007) [2023-12-27 14:07:03,135][20381] Updated weights for policy 1, policy_version 1221181 (0.0007) [2023-12-27 14:07:03,189][20381] Updated weights for policy 1, policy_version 1221191 (0.0005) [2023-12-27 14:07:03,243][20381] Updated weights for policy 1, policy_version 1221201 (0.0005) [2023-12-27 14:07:03,507][20347] Updated weights for policy 0, policy_version 1222307 (0.0007) [2023-12-27 14:07:03,556][20347] Updated weights for policy 0, policy_version 1222317 (0.0009) [2023-12-27 14:07:03,615][20347] Updated weights for policy 0, policy_version 1222327 (0.0009) [2023-12-27 14:07:03,770][20381] Updated weights for policy 1, policy_version 1221211 (0.0007) [2023-12-27 14:07:03,823][20381] Updated weights for policy 1, policy_version 1221221 (0.0010) [2023-12-27 14:07:03,882][20381] Updated weights for policy 1, policy_version 1221231 (0.0011) [2023-12-27 14:07:04,361][20347] Updated weights for policy 0, policy_version 1222337 (0.0009) [2023-12-27 14:07:04,413][20347] Updated weights for policy 0, policy_version 1222347 (0.0007) [2023-12-27 14:07:04,462][20347] Updated weights for policy 0, policy_version 1222357 (0.0007) [2023-12-27 14:07:04,517][20347] Updated weights for policy 0, policy_version 1222367 (0.0007) [2023-12-27 14:07:04,600][20381] Updated weights for policy 1, policy_version 1221241 (0.0010) [2023-12-27 14:07:04,652][20381] Updated weights for policy 1, policy_version 1221251 (0.0008) [2023-12-27 14:07:04,707][20381] Updated weights for policy 1, policy_version 1221261 (0.0007) [2023-12-27 14:07:04,757][20381] Updated weights for policy 1, policy_version 1221271 (0.0007) [2023-12-27 14:07:05,264][20347] Updated weights for policy 0, policy_version 1222377 (0.0008) [2023-12-27 14:07:05,308][20347] Updated weights for policy 0, policy_version 1222387 (0.0008) [2023-12-27 14:07:05,342][20381] Updated weights for policy 1, policy_version 1221281 (0.0006) [2023-12-27 14:07:05,365][20347] Updated weights for policy 0, policy_version 1222397 (0.0006) [2023-12-27 14:07:05,401][20381] Updated weights for policy 1, policy_version 1221291 (0.0011) [2023-12-27 14:07:05,460][20381] Updated weights for policy 1, policy_version 1221301 (0.0011) [2023-12-27 14:07:06,055][20347] Updated weights for policy 0, policy_version 1222407 (0.0006) [2023-12-27 14:07:06,120][20347] Updated weights for policy 0, policy_version 1222417 (0.0007) [2023-12-27 14:07:06,142][20381] Updated weights for policy 1, policy_version 1221311 (0.0009) [2023-12-27 14:07:06,180][20347] Updated weights for policy 0, policy_version 1222427 (0.0007) [2023-12-27 14:07:06,189][20381] Updated weights for policy 1, policy_version 1221321 (0.0008) [2023-12-27 14:07:06,248][20381] Updated weights for policy 1, policy_version 1221331 (0.0006) [2023-12-27 14:07:06,816][20381] Updated weights for policy 1, policy_version 1221341 (0.0008) [2023-12-27 14:07:06,865][20381] Updated weights for policy 1, policy_version 1221351 (0.0010) [2023-12-27 14:07:06,916][20381] Updated weights for policy 1, policy_version 1221361 (0.0008) [2023-12-27 14:07:06,922][20347] Updated weights for policy 0, policy_version 1222437 (0.0007) [2023-12-27 14:07:06,971][20347] Updated weights for policy 0, policy_version 1222447 (0.0006) [2023-12-27 14:07:07,032][20347] Updated weights for policy 0, policy_version 1222457 (0.0008) [2023-12-27 14:07:07,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.4, 300 sec: 20632.8). Total num frames: 625704960. Throughput: 0: 10317.5, 1: 10324.3. Samples: 625672416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:07,364][19170] Avg episode reward: [(0, '794.780'), (1, '863.090')] [2023-12-27 14:07:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001222464_313008128.pth... [2023-12-27 14:07:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001221368_312696832.pth... [2023-12-27 14:07:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001220152_312385536.pth [2023-12-27 14:07:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001221280_312705024.pth [2023-12-27 14:07:07,534][20381] Updated weights for policy 1, policy_version 1221371 (0.0011) [2023-12-27 14:07:07,592][20381] Updated weights for policy 1, policy_version 1221381 (0.0010) [2023-12-27 14:07:07,647][20381] Updated weights for policy 1, policy_version 1221391 (0.0011) [2023-12-27 14:07:07,709][20347] Updated weights for policy 0, policy_version 1222467 (0.0006) [2023-12-27 14:07:07,771][20347] Updated weights for policy 0, policy_version 1222477 (0.0005) [2023-12-27 14:07:07,827][20347] Updated weights for policy 0, policy_version 1222487 (0.0009) [2023-12-27 14:07:08,338][20381] Updated weights for policy 1, policy_version 1221401 (0.0009) [2023-12-27 14:07:08,392][20381] Updated weights for policy 1, policy_version 1221411 (0.0009) [2023-12-27 14:07:08,410][20347] Updated weights for policy 0, policy_version 1222497 (0.0010) [2023-12-27 14:07:08,441][20381] Updated weights for policy 1, policy_version 1221421 (0.0008) [2023-12-27 14:07:08,470][20347] Updated weights for policy 0, policy_version 1222507 (0.0008) [2023-12-27 14:07:08,493][20381] Updated weights for policy 1, policy_version 1221431 (0.0008) [2023-12-27 14:07:08,521][20347] Updated weights for policy 0, policy_version 1222517 (0.0008) [2023-12-27 14:07:08,580][20347] Updated weights for policy 0, policy_version 1222527 (0.0007) [2023-12-27 14:07:09,205][20381] Updated weights for policy 1, policy_version 1221441 (0.0008) [2023-12-27 14:07:09,267][20381] Updated weights for policy 1, policy_version 1221451 (0.0007) [2023-12-27 14:07:09,317][20347] Updated weights for policy 0, policy_version 1222537 (0.0010) [2023-12-27 14:07:09,327][20381] Updated weights for policy 1, policy_version 1221461 (0.0005) [2023-12-27 14:07:09,396][20347] Updated weights for policy 0, policy_version 1222548 (0.0010) [2023-12-27 14:07:09,457][20347] Updated weights for policy 0, policy_version 1222558 (0.0007) [2023-12-27 14:07:10,093][20381] Updated weights for policy 1, policy_version 1221471 (0.0010) [2023-12-27 14:07:10,138][20347] Updated weights for policy 0, policy_version 1222568 (0.0006) [2023-12-27 14:07:10,149][20381] Updated weights for policy 1, policy_version 1221481 (0.0010) [2023-12-27 14:07:10,203][20347] Updated weights for policy 0, policy_version 1222578 (0.0007) [2023-12-27 14:07:10,210][20381] Updated weights for policy 1, policy_version 1221491 (0.0011) [2023-12-27 14:07:10,259][20347] Updated weights for policy 0, policy_version 1222588 (0.0006) [2023-12-27 14:07:10,853][20381] Updated weights for policy 1, policy_version 1221501 (0.0010) [2023-12-27 14:07:10,902][20381] Updated weights for policy 1, policy_version 1221511 (0.0010) [2023-12-27 14:07:10,913][20347] Updated weights for policy 0, policy_version 1222598 (0.0006) [2023-12-27 14:07:10,952][20381] Updated weights for policy 1, policy_version 1221521 (0.0010) [2023-12-27 14:07:10,973][20347] Updated weights for policy 0, policy_version 1222608 (0.0007) [2023-12-27 14:07:11,026][20347] Updated weights for policy 0, policy_version 1222618 (0.0008) [2023-12-27 14:07:11,725][20347] Updated weights for policy 0, policy_version 1222628 (0.0008) [2023-12-27 14:07:11,739][20381] Updated weights for policy 1, policy_version 1221531 (0.0009) [2023-12-27 14:07:11,790][20347] Updated weights for policy 0, policy_version 1222638 (0.0007) [2023-12-27 14:07:11,802][20381] Updated weights for policy 1, policy_version 1221541 (0.0008) [2023-12-27 14:07:11,848][20347] Updated weights for policy 0, policy_version 1222648 (0.0007) [2023-12-27 14:07:11,867][20381] Updated weights for policy 1, policy_version 1221551 (0.0006) [2023-12-27 14:07:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 625803264. Throughput: 0: 10339.8, 1: 10369.3. Samples: 625795200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:12,363][19170] Avg episode reward: [(0, '735.780'), (1, '843.560')] [2023-12-27 14:07:12,514][20381] Updated weights for policy 1, policy_version 1221561 (0.0006) [2023-12-27 14:07:12,564][20347] Updated weights for policy 0, policy_version 1222658 (0.0007) [2023-12-27 14:07:12,576][20381] Updated weights for policy 1, policy_version 1221571 (0.0011) [2023-12-27 14:07:12,622][20347] Updated weights for policy 0, policy_version 1222668 (0.0006) [2023-12-27 14:07:12,636][20381] Updated weights for policy 1, policy_version 1221581 (0.0009) [2023-12-27 14:07:12,685][20347] Updated weights for policy 0, policy_version 1222678 (0.0008) [2023-12-27 14:07:12,693][20381] Updated weights for policy 1, policy_version 1221591 (0.0006) [2023-12-27 14:07:12,743][20347] Updated weights for policy 0, policy_version 1222688 (0.0008) [2023-12-27 14:07:13,353][20381] Updated weights for policy 1, policy_version 1221601 (0.0011) [2023-12-27 14:07:13,385][20347] Updated weights for policy 0, policy_version 1222698 (0.0005) [2023-12-27 14:07:13,408][20381] Updated weights for policy 1, policy_version 1221611 (0.0010) [2023-12-27 14:07:13,448][20347] Updated weights for policy 0, policy_version 1222708 (0.0005) [2023-12-27 14:07:13,456][20381] Updated weights for policy 1, policy_version 1221621 (0.0010) [2023-12-27 14:07:13,505][20347] Updated weights for policy 0, policy_version 1222718 (0.0005) [2023-12-27 14:07:14,058][20347] Updated weights for policy 0, policy_version 1222728 (0.0006) [2023-12-27 14:07:14,072][20381] Updated weights for policy 1, policy_version 1221631 (0.0010) [2023-12-27 14:07:14,110][20347] Updated weights for policy 0, policy_version 1222738 (0.0005) [2023-12-27 14:07:14,117][20381] Updated weights for policy 1, policy_version 1221641 (0.0010) [2023-12-27 14:07:14,168][20381] Updated weights for policy 1, policy_version 1221651 (0.0007) [2023-12-27 14:07:14,170][20347] Updated weights for policy 0, policy_version 1222748 (0.0007) [2023-12-27 14:07:14,815][20381] Updated weights for policy 1, policy_version 1221661 (0.0009) [2023-12-27 14:07:14,879][20347] Updated weights for policy 0, policy_version 1222758 (0.0006) [2023-12-27 14:07:14,881][20381] Updated weights for policy 1, policy_version 1221671 (0.0011) [2023-12-27 14:07:14,936][20347] Updated weights for policy 0, policy_version 1222768 (0.0008) [2023-12-27 14:07:14,943][20381] Updated weights for policy 1, policy_version 1221681 (0.0011) [2023-12-27 14:07:14,997][20347] Updated weights for policy 0, policy_version 1222778 (0.0008) [2023-12-27 14:07:15,626][20381] Updated weights for policy 1, policy_version 1221691 (0.0010) [2023-12-27 14:07:15,652][20347] Updated weights for policy 0, policy_version 1222788 (0.0006) [2023-12-27 14:07:15,677][20381] Updated weights for policy 1, policy_version 1221701 (0.0007) [2023-12-27 14:07:15,711][20347] Updated weights for policy 0, policy_version 1222798 (0.0005) [2023-12-27 14:07:15,734][20381] Updated weights for policy 1, policy_version 1221711 (0.0007) [2023-12-27 14:07:15,757][20347] Updated weights for policy 0, policy_version 1222808 (0.0005) [2023-12-27 14:07:16,313][20347] Updated weights for policy 0, policy_version 1222818 (0.0005) [2023-12-27 14:07:16,363][20347] Updated weights for policy 0, policy_version 1222828 (0.0007) [2023-12-27 14:07:16,390][20381] Updated weights for policy 1, policy_version 1221721 (0.0011) [2023-12-27 14:07:16,414][20347] Updated weights for policy 0, policy_version 1222838 (0.0007) [2023-12-27 14:07:16,447][20381] Updated weights for policy 1, policy_version 1221731 (0.0010) [2023-12-27 14:07:16,463][20347] Updated weights for policy 0, policy_version 1222848 (0.0006) [2023-12-27 14:07:16,501][20381] Updated weights for policy 1, policy_version 1221741 (0.0010) [2023-12-27 14:07:16,555][20381] Updated weights for policy 1, policy_version 1221751 (0.0010) [2023-12-27 14:07:17,188][20347] Updated weights for policy 0, policy_version 1222858 (0.0007) [2023-12-27 14:07:17,206][20381] Updated weights for policy 1, policy_version 1221761 (0.0010) [2023-12-27 14:07:17,238][20347] Updated weights for policy 0, policy_version 1222868 (0.0010) [2023-12-27 14:07:17,256][20381] Updated weights for policy 1, policy_version 1221771 (0.0010) [2023-12-27 14:07:17,291][20347] Updated weights for policy 0, policy_version 1222878 (0.0006) [2023-12-27 14:07:17,305][20381] Updated weights for policy 1, policy_version 1221781 (0.0010) [2023-12-27 14:07:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 625917952. Throughput: 0: 10303.2, 1: 10303.5. Samples: 625919788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:17,363][19170] Avg episode reward: [(0, '672.650'), (1, '804.530')] [2023-12-27 14:07:17,929][20381] Updated weights for policy 1, policy_version 1221791 (0.0007) [2023-12-27 14:07:17,954][20347] Updated weights for policy 0, policy_version 1222888 (0.0007) [2023-12-27 14:07:17,980][20381] Updated weights for policy 1, policy_version 1221801 (0.0005) [2023-12-27 14:07:18,006][20347] Updated weights for policy 0, policy_version 1222898 (0.0008) [2023-12-27 14:07:18,040][20381] Updated weights for policy 1, policy_version 1221811 (0.0006) [2023-12-27 14:07:18,070][20347] Updated weights for policy 0, policy_version 1222908 (0.0007) [2023-12-27 14:07:18,732][20381] Updated weights for policy 1, policy_version 1221821 (0.0007) [2023-12-27 14:07:18,794][20381] Updated weights for policy 1, policy_version 1221831 (0.0007) [2023-12-27 14:07:18,795][20347] Updated weights for policy 0, policy_version 1222918 (0.0007) [2023-12-27 14:07:18,846][20347] Updated weights for policy 0, policy_version 1222928 (0.0006) [2023-12-27 14:07:18,848][20381] Updated weights for policy 1, policy_version 1221841 (0.0008) [2023-12-27 14:07:18,903][20347] Updated weights for policy 0, policy_version 1222938 (0.0006) [2023-12-27 14:07:19,581][20381] Updated weights for policy 1, policy_version 1221851 (0.0007) [2023-12-27 14:07:19,605][20347] Updated weights for policy 0, policy_version 1222948 (0.0008) [2023-12-27 14:07:19,639][20381] Updated weights for policy 1, policy_version 1221861 (0.0005) [2023-12-27 14:07:19,670][20347] Updated weights for policy 0, policy_version 1222958 (0.0007) [2023-12-27 14:07:19,696][20381] Updated weights for policy 1, policy_version 1221871 (0.0006) [2023-12-27 14:07:19,727][20347] Updated weights for policy 0, policy_version 1222968 (0.0007) [2023-12-27 14:07:20,284][20381] Updated weights for policy 1, policy_version 1221881 (0.0006) [2023-12-27 14:07:20,341][20381] Updated weights for policy 1, policy_version 1221891 (0.0008) [2023-12-27 14:07:20,396][20381] Updated weights for policy 1, policy_version 1221901 (0.0008) [2023-12-27 14:07:20,442][20381] Updated weights for policy 1, policy_version 1221911 (0.0008) [2023-12-27 14:07:20,456][20347] Updated weights for policy 0, policy_version 1222978 (0.0009) [2023-12-27 14:07:20,507][20347] Updated weights for policy 0, policy_version 1222988 (0.0010) [2023-12-27 14:07:20,556][20347] Updated weights for policy 0, policy_version 1222998 (0.0010) [2023-12-27 14:07:20,623][20347] Updated weights for policy 0, policy_version 1223008 (0.0009) [2023-12-27 14:07:21,261][20381] Updated weights for policy 1, policy_version 1221921 (0.0007) [2023-12-27 14:07:21,328][20381] Updated weights for policy 1, policy_version 1221931 (0.0008) [2023-12-27 14:07:21,345][20347] Updated weights for policy 0, policy_version 1223018 (0.0009) [2023-12-27 14:07:21,401][20381] Updated weights for policy 1, policy_version 1221941 (0.0008) [2023-12-27 14:07:21,413][20347] Updated weights for policy 0, policy_version 1223028 (0.0008) [2023-12-27 14:07:21,471][20347] Updated weights for policy 0, policy_version 1223038 (0.0008) [2023-12-27 14:07:22,098][20381] Updated weights for policy 1, policy_version 1221951 (0.0010) [2023-12-27 14:07:22,150][20381] Updated weights for policy 1, policy_version 1221961 (0.0010) [2023-12-27 14:07:22,213][20381] Updated weights for policy 1, policy_version 1221971 (0.0010) [2023-12-27 14:07:22,217][20347] Updated weights for policy 0, policy_version 1223048 (0.0009) [2023-12-27 14:07:22,274][20347] Updated weights for policy 0, policy_version 1223058 (0.0007) [2023-12-27 14:07:22,337][20347] Updated weights for policy 0, policy_version 1223068 (0.0008) [2023-12-27 14:07:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 626016256. Throughput: 0: 10260.7, 1: 10298.0. Samples: 625980932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:22,363][19170] Avg episode reward: [(0, '758.700'), (1, '882.900')] [2023-12-27 14:07:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001221976_312852480.pth... [2023-12-27 14:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001223072_313163776.pth... [2023-12-27 14:07:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001220760_312541184.pth [2023-12-27 14:07:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001221856_312852480.pth [2023-12-27 14:07:22,941][20381] Updated weights for policy 1, policy_version 1221981 (0.0011) [2023-12-27 14:07:23,002][20381] Updated weights for policy 1, policy_version 1221991 (0.0011) [2023-12-27 14:07:23,058][20381] Updated weights for policy 1, policy_version 1222001 (0.0011) [2023-12-27 14:07:23,075][20347] Updated weights for policy 0, policy_version 1223078 (0.0007) [2023-12-27 14:07:23,130][20347] Updated weights for policy 0, policy_version 1223088 (0.0007) [2023-12-27 14:07:23,183][20347] Updated weights for policy 0, policy_version 1223098 (0.0006) [2023-12-27 14:07:23,673][20381] Updated weights for policy 1, policy_version 1222011 (0.0011) [2023-12-27 14:07:23,733][20381] Updated weights for policy 1, policy_version 1222021 (0.0011) [2023-12-27 14:07:23,783][20381] Updated weights for policy 1, policy_version 1222031 (0.0011) [2023-12-27 14:07:23,827][20347] Updated weights for policy 0, policy_version 1223108 (0.0007) [2023-12-27 14:07:23,884][20347] Updated weights for policy 0, policy_version 1223118 (0.0005) [2023-12-27 14:07:23,941][20347] Updated weights for policy 0, policy_version 1223128 (0.0005) [2023-12-27 14:07:24,469][20347] Updated weights for policy 0, policy_version 1223138 (0.0006) [2023-12-27 14:07:24,509][20381] Updated weights for policy 1, policy_version 1222041 (0.0010) [2023-12-27 14:07:24,521][20347] Updated weights for policy 0, policy_version 1223148 (0.0008) [2023-12-27 14:07:24,573][20381] Updated weights for policy 1, policy_version 1222051 (0.0011) [2023-12-27 14:07:24,574][20347] Updated weights for policy 0, policy_version 1223158 (0.0008) [2023-12-27 14:07:24,626][20347] Updated weights for policy 0, policy_version 1223168 (0.0008) [2023-12-27 14:07:24,636][20381] Updated weights for policy 1, policy_version 1222061 (0.0011) [2023-12-27 14:07:24,699][20381] Updated weights for policy 1, policy_version 1222071 (0.0011) [2023-12-27 14:07:25,361][20381] Updated weights for policy 1, policy_version 1222081 (0.0011) [2023-12-27 14:07:25,370][20347] Updated weights for policy 0, policy_version 1223178 (0.0006) [2023-12-27 14:07:25,413][20381] Updated weights for policy 1, policy_version 1222091 (0.0010) [2023-12-27 14:07:25,422][20347] Updated weights for policy 0, policy_version 1223188 (0.0005) [2023-12-27 14:07:25,468][20381] Updated weights for policy 1, policy_version 1222101 (0.0010) [2023-12-27 14:07:25,470][20347] Updated weights for policy 0, policy_version 1223198 (0.0005) [2023-12-27 14:07:26,129][20381] Updated weights for policy 1, policy_version 1222111 (0.0007) [2023-12-27 14:07:26,153][20347] Updated weights for policy 0, policy_version 1223208 (0.0006) [2023-12-27 14:07:26,188][20381] Updated weights for policy 1, policy_version 1222121 (0.0009) [2023-12-27 14:07:26,204][20347] Updated weights for policy 0, policy_version 1223218 (0.0005) [2023-12-27 14:07:26,246][20381] Updated weights for policy 1, policy_version 1222131 (0.0006) [2023-12-27 14:07:26,259][20347] Updated weights for policy 0, policy_version 1223228 (0.0005) [2023-12-27 14:07:26,795][20381] Updated weights for policy 1, policy_version 1222141 (0.0005) [2023-12-27 14:07:26,845][20347] Updated weights for policy 0, policy_version 1223238 (0.0006) [2023-12-27 14:07:26,859][20381] Updated weights for policy 1, policy_version 1222151 (0.0005) [2023-12-27 14:07:26,900][20347] Updated weights for policy 0, policy_version 1223248 (0.0007) [2023-12-27 14:07:26,922][20381] Updated weights for policy 1, policy_version 1222161 (0.0005) [2023-12-27 14:07:26,951][20347] Updated weights for policy 0, policy_version 1223258 (0.0010) [2023-12-27 14:07:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 626114560. Throughput: 0: 10251.2, 1: 10310.6. Samples: 626102008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:27,363][19170] Avg episode reward: [(0, '767.240'), (1, '902.270')] [2023-12-27 14:07:27,508][20381] Updated weights for policy 1, policy_version 1222171 (0.0007) [2023-12-27 14:07:27,559][20381] Updated weights for policy 1, policy_version 1222181 (0.0010) [2023-12-27 14:07:27,562][20347] Updated weights for policy 0, policy_version 1223268 (0.0008) [2023-12-27 14:07:27,613][20381] Updated weights for policy 1, policy_version 1222191 (0.0010) [2023-12-27 14:07:27,616][20347] Updated weights for policy 0, policy_version 1223278 (0.0005) [2023-12-27 14:07:27,672][20347] Updated weights for policy 0, policy_version 1223288 (0.0006) [2023-12-27 14:07:28,272][20381] Updated weights for policy 1, policy_version 1222201 (0.0009) [2023-12-27 14:07:28,300][20347] Updated weights for policy 0, policy_version 1223298 (0.0007) [2023-12-27 14:07:28,324][20381] Updated weights for policy 1, policy_version 1222211 (0.0011) [2023-12-27 14:07:28,357][20347] Updated weights for policy 0, policy_version 1223308 (0.0010) [2023-12-27 14:07:28,383][20381] Updated weights for policy 1, policy_version 1222221 (0.0010) [2023-12-27 14:07:28,412][20347] Updated weights for policy 0, policy_version 1223318 (0.0007) [2023-12-27 14:07:28,442][20381] Updated weights for policy 1, policy_version 1222231 (0.0010) [2023-12-27 14:07:28,471][20347] Updated weights for policy 0, policy_version 1223328 (0.0010) [2023-12-27 14:07:29,092][20347] Updated weights for policy 0, policy_version 1223338 (0.0005) [2023-12-27 14:07:29,146][20381] Updated weights for policy 1, policy_version 1222241 (0.0010) [2023-12-27 14:07:29,148][20347] Updated weights for policy 0, policy_version 1223348 (0.0005) [2023-12-27 14:07:29,200][20381] Updated weights for policy 1, policy_version 1222251 (0.0010) [2023-12-27 14:07:29,206][20347] Updated weights for policy 0, policy_version 1223358 (0.0007) [2023-12-27 14:07:29,263][20381] Updated weights for policy 1, policy_version 1222261 (0.0009) [2023-12-27 14:07:29,808][20347] Updated weights for policy 0, policy_version 1223368 (0.0010) [2023-12-27 14:07:29,875][20347] Updated weights for policy 0, policy_version 1223378 (0.0010) [2023-12-27 14:07:29,924][20381] Updated weights for policy 1, policy_version 1222271 (0.0006) [2023-12-27 14:07:29,937][20347] Updated weights for policy 0, policy_version 1223388 (0.0006) [2023-12-27 14:07:29,982][20381] Updated weights for policy 1, policy_version 1222281 (0.0007) [2023-12-27 14:07:30,037][20381] Updated weights for policy 1, policy_version 1222291 (0.0007) [2023-12-27 14:07:30,591][20347] Updated weights for policy 0, policy_version 1223398 (0.0008) [2023-12-27 14:07:30,646][20347] Updated weights for policy 0, policy_version 1223408 (0.0006) [2023-12-27 14:07:30,697][20347] Updated weights for policy 0, policy_version 1223418 (0.0005) [2023-12-27 14:07:30,727][20381] Updated weights for policy 1, policy_version 1222301 (0.0008) [2023-12-27 14:07:30,781][20381] Updated weights for policy 1, policy_version 1222311 (0.0005) [2023-12-27 14:07:30,839][20381] Updated weights for policy 1, policy_version 1222321 (0.0005) [2023-12-27 14:07:31,356][20347] Updated weights for policy 0, policy_version 1223428 (0.0006) [2023-12-27 14:07:31,418][20347] Updated weights for policy 0, policy_version 1223438 (0.0008) [2023-12-27 14:07:31,473][20347] Updated weights for policy 0, policy_version 1223448 (0.0008) [2023-12-27 14:07:31,490][20381] Updated weights for policy 1, policy_version 1222331 (0.0006) [2023-12-27 14:07:31,544][20381] Updated weights for policy 1, policy_version 1222341 (0.0006) [2023-12-27 14:07:31,604][20381] Updated weights for policy 1, policy_version 1222351 (0.0008) [2023-12-27 14:07:32,156][20347] Updated weights for policy 0, policy_version 1223458 (0.0008) [2023-12-27 14:07:32,223][20347] Updated weights for policy 0, policy_version 1223468 (0.0008) [2023-12-27 14:07:32,258][20381] Updated weights for policy 1, policy_version 1222361 (0.0008) [2023-12-27 14:07:32,290][20347] Updated weights for policy 0, policy_version 1223478 (0.0008) [2023-12-27 14:07:32,311][20381] Updated weights for policy 1, policy_version 1222371 (0.0007) [2023-12-27 14:07:32,352][20347] Updated weights for policy 0, policy_version 1223488 (0.0007) [2023-12-27 14:07:32,359][20381] Updated weights for policy 1, policy_version 1222381 (0.0006) [2023-12-27 14:07:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 626221056. Throughput: 0: 10362.5, 1: 10337.3. Samples: 626231556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:32,363][19170] Avg episode reward: [(0, '737.990'), (1, '912.050')] [2023-12-27 14:07:32,418][20381] Updated weights for policy 1, policy_version 1222391 (0.0008) [2023-12-27 14:07:32,940][20347] Updated weights for policy 0, policy_version 1223498 (0.0006) [2023-12-27 14:07:32,995][20347] Updated weights for policy 0, policy_version 1223508 (0.0006) [2023-12-27 14:07:33,020][20381] Updated weights for policy 1, policy_version 1222401 (0.0006) [2023-12-27 14:07:33,047][20347] Updated weights for policy 0, policy_version 1223518 (0.0007) [2023-12-27 14:07:33,072][20381] Updated weights for policy 1, policy_version 1222411 (0.0005) [2023-12-27 14:07:33,127][20381] Updated weights for policy 1, policy_version 1222421 (0.0008) [2023-12-27 14:07:33,620][20347] Updated weights for policy 0, policy_version 1223528 (0.0006) [2023-12-27 14:07:33,675][20347] Updated weights for policy 0, policy_version 1223538 (0.0006) [2023-12-27 14:07:33,734][20347] Updated weights for policy 0, policy_version 1223548 (0.0008) [2023-12-27 14:07:33,808][20381] Updated weights for policy 1, policy_version 1222431 (0.0005) [2023-12-27 14:07:33,858][20381] Updated weights for policy 1, policy_version 1222441 (0.0005) [2023-12-27 14:07:33,903][20381] Updated weights for policy 1, policy_version 1222451 (0.0005) [2023-12-27 14:07:34,446][20347] Updated weights for policy 0, policy_version 1223558 (0.0009) [2023-12-27 14:07:34,505][20347] Updated weights for policy 0, policy_version 1223568 (0.0009) [2023-12-27 14:07:34,546][20381] Updated weights for policy 1, policy_version 1222461 (0.0006) [2023-12-27 14:07:34,570][20347] Updated weights for policy 0, policy_version 1223578 (0.0008) [2023-12-27 14:07:34,604][20381] Updated weights for policy 1, policy_version 1222471 (0.0006) [2023-12-27 14:07:34,668][20381] Updated weights for policy 1, policy_version 1222481 (0.0009) [2023-12-27 14:07:35,228][20347] Updated weights for policy 0, policy_version 1223588 (0.0007) [2023-12-27 14:07:35,280][20347] Updated weights for policy 0, policy_version 1223598 (0.0005) [2023-12-27 14:07:35,334][20347] Updated weights for policy 0, policy_version 1223608 (0.0007) [2023-12-27 14:07:35,346][20381] Updated weights for policy 1, policy_version 1222491 (0.0007) [2023-12-27 14:07:35,397][20381] Updated weights for policy 1, policy_version 1222501 (0.0007) [2023-12-27 14:07:35,443][20381] Updated weights for policy 1, policy_version 1222511 (0.0010) [2023-12-27 14:07:35,925][20347] Updated weights for policy 0, policy_version 1223618 (0.0006) [2023-12-27 14:07:35,983][20347] Updated weights for policy 0, policy_version 1223628 (0.0007) [2023-12-27 14:07:36,035][20347] Updated weights for policy 0, policy_version 1223638 (0.0007) [2023-12-27 14:07:36,087][20347] Updated weights for policy 0, policy_version 1223648 (0.0008) [2023-12-27 14:07:36,154][20381] Updated weights for policy 1, policy_version 1222521 (0.0006) [2023-12-27 14:07:36,213][20381] Updated weights for policy 1, policy_version 1222531 (0.0008) [2023-12-27 14:07:36,273][20381] Updated weights for policy 1, policy_version 1222541 (0.0007) [2023-12-27 14:07:36,332][20381] Updated weights for policy 1, policy_version 1222551 (0.0007) [2023-12-27 14:07:36,768][20347] Updated weights for policy 0, policy_version 1223658 (0.0006) [2023-12-27 14:07:36,818][20347] Updated weights for policy 0, policy_version 1223668 (0.0007) [2023-12-27 14:07:36,874][20347] Updated weights for policy 0, policy_version 1223678 (0.0006) [2023-12-27 14:07:37,050][20381] Updated weights for policy 1, policy_version 1222561 (0.0008) [2023-12-27 14:07:37,104][20381] Updated weights for policy 1, policy_version 1222571 (0.0007) [2023-12-27 14:07:37,162][20381] Updated weights for policy 1, policy_version 1222581 (0.0007) [2023-12-27 14:07:37,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 626327552. Throughput: 0: 10377.7, 1: 10352.1. Samples: 626295552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:37,363][19170] Avg episode reward: [(0, '739.380'), (1, '892.650')] [2023-12-27 14:07:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001222584_313008128.pth... [2023-12-27 14:07:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001223680_313319424.pth... [2023-12-27 14:07:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001222464_313008128.pth [2023-12-27 14:07:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001221368_312696832.pth [2023-12-27 14:07:37,531][20347] Updated weights for policy 0, policy_version 1223688 (0.0008) [2023-12-27 14:07:37,599][20347] Updated weights for policy 0, policy_version 1223698 (0.0008) [2023-12-27 14:07:37,665][20347] Updated weights for policy 0, policy_version 1223708 (0.0007) [2023-12-27 14:07:37,874][20381] Updated weights for policy 1, policy_version 1222591 (0.0007) [2023-12-27 14:07:37,924][20381] Updated weights for policy 1, policy_version 1222601 (0.0007) [2023-12-27 14:07:37,983][20381] Updated weights for policy 1, policy_version 1222611 (0.0008) [2023-12-27 14:07:38,312][20347] Updated weights for policy 0, policy_version 1223718 (0.0008) [2023-12-27 14:07:38,378][20347] Updated weights for policy 0, policy_version 1223728 (0.0011) [2023-12-27 14:07:38,467][20347] Updated weights for policy 0, policy_version 1223738 (0.0011) [2023-12-27 14:07:38,502][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 14:07:38,682][20381] Updated weights for policy 1, policy_version 1222621 (0.0007) [2023-12-27 14:07:38,741][20381] Updated weights for policy 1, policy_version 1222631 (0.0007) [2023-12-27 14:07:38,800][20381] Updated weights for policy 1, policy_version 1222641 (0.0008) [2023-12-27 14:07:39,127][20347] Updated weights for policy 0, policy_version 1223748 (0.0010) [2023-12-27 14:07:39,181][20347] Updated weights for policy 0, policy_version 1223758 (0.0010) [2023-12-27 14:07:39,242][20347] Updated weights for policy 0, policy_version 1223768 (0.0009) [2023-12-27 14:07:39,539][20381] Updated weights for policy 1, policy_version 1222651 (0.0008) [2023-12-27 14:07:39,605][20381] Updated weights for policy 1, policy_version 1222661 (0.0009) [2023-12-27 14:07:39,669][20381] Updated weights for policy 1, policy_version 1222671 (0.0008) [2023-12-27 14:07:39,986][20347] Updated weights for policy 0, policy_version 1223778 (0.0007) [2023-12-27 14:07:40,054][20347] Updated weights for policy 0, policy_version 1223788 (0.0006) [2023-12-27 14:07:40,117][20347] Updated weights for policy 0, policy_version 1223798 (0.0005) [2023-12-27 14:07:40,181][20347] Updated weights for policy 0, policy_version 1223808 (0.0005) [2023-12-27 14:07:40,334][20381] Updated weights for policy 1, policy_version 1222681 (0.0007) [2023-12-27 14:07:40,390][20381] Updated weights for policy 1, policy_version 1222691 (0.0011) [2023-12-27 14:07:40,454][20381] Updated weights for policy 1, policy_version 1222701 (0.0005) [2023-12-27 14:07:40,517][20381] Updated weights for policy 1, policy_version 1222711 (0.0010) [2023-12-27 14:07:40,814][20347] Updated weights for policy 0, policy_version 1223818 (0.0008) [2023-12-27 14:07:40,870][20347] Updated weights for policy 0, policy_version 1223828 (0.0008) [2023-12-27 14:07:40,916][20347] Updated weights for policy 0, policy_version 1223838 (0.0009) [2023-12-27 14:07:41,086][20381] Updated weights for policy 1, policy_version 1222721 (0.0008) [2023-12-27 14:07:41,154][20381] Updated weights for policy 1, policy_version 1222731 (0.0007) [2023-12-27 14:07:41,218][20381] Updated weights for policy 1, policy_version 1222741 (0.0007) [2023-12-27 14:07:41,633][20347] Updated weights for policy 0, policy_version 1223848 (0.0007) [2023-12-27 14:07:41,698][20347] Updated weights for policy 0, policy_version 1223858 (0.0008) [2023-12-27 14:07:41,764][20347] Updated weights for policy 0, policy_version 1223868 (0.0006) [2023-12-27 14:07:41,951][20381] Updated weights for policy 1, policy_version 1222751 (0.0008) [2023-12-27 14:07:41,972][20292] KL-divergence is very high: 130.2044 [2023-12-27 14:07:42,019][20381] Updated weights for policy 1, policy_version 1222761 (0.0008) [2023-12-27 14:07:42,027][20292] KL-divergence is very high: 118.3907 [2023-12-27 14:07:42,083][20381] Updated weights for policy 1, policy_version 1222771 (0.0008) [2023-12-27 14:07:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 626425856. Throughput: 0: 10347.9, 1: 10351.4. Samples: 626417316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 14:07:42,363][19170] Avg episode reward: [(0, '719.690'), (1, '833.830')] [2023-12-27 14:07:42,412][20347] Updated weights for policy 0, policy_version 1223878 (0.0007) [2023-12-27 14:07:42,467][20347] Updated weights for policy 0, policy_version 1223888 (0.0005) [2023-12-27 14:07:42,524][20347] Updated weights for policy 0, policy_version 1223898 (0.0005) [2023-12-27 14:07:42,783][20381] Updated weights for policy 1, policy_version 1222781 (0.0009) [2023-12-27 14:07:42,845][20381] Updated weights for policy 1, policy_version 1222791 (0.0011) [2023-12-27 14:07:42,897][20381] Updated weights for policy 1, policy_version 1222801 (0.0011) [2023-12-27 14:07:43,204][20347] Updated weights for policy 0, policy_version 1223908 (0.0010) [2023-12-27 14:07:43,263][20347] Updated weights for policy 0, policy_version 1223918 (0.0010) [2023-12-27 14:07:43,324][20347] Updated weights for policy 0, policy_version 1223928 (0.0010) [2023-12-27 14:07:43,581][20381] Updated weights for policy 1, policy_version 1222811 (0.0011) [2023-12-27 14:07:43,630][20381] Updated weights for policy 1, policy_version 1222821 (0.0009) [2023-12-27 14:07:43,676][20381] Updated weights for policy 1, policy_version 1222831 (0.0005) [2023-12-27 14:07:43,938][20347] Updated weights for policy 0, policy_version 1223938 (0.0008) [2023-12-27 14:07:43,999][20347] Updated weights for policy 0, policy_version 1223948 (0.0010) [2023-12-27 14:07:44,059][20347] Updated weights for policy 0, policy_version 1223958 (0.0010) [2023-12-27 14:07:44,119][20347] Updated weights for policy 0, policy_version 1223968 (0.0010) [2023-12-27 14:07:44,259][20381] Updated weights for policy 1, policy_version 1222841 (0.0006) [2023-12-27 14:07:44,305][20381] Updated weights for policy 1, policy_version 1222851 (0.0010) [2023-12-27 14:07:44,353][20381] Updated weights for policy 1, policy_version 1222861 (0.0010) [2023-12-27 14:07:44,406][20381] Updated weights for policy 1, policy_version 1222871 (0.0011) [2023-12-27 14:07:44,737][20347] Updated weights for policy 0, policy_version 1223978 (0.0008) [2023-12-27 14:07:44,802][20347] Updated weights for policy 0, policy_version 1223988 (0.0009) [2023-12-27 14:07:44,867][20347] Updated weights for policy 0, policy_version 1223998 (0.0010) [2023-12-27 14:07:45,101][20381] Updated weights for policy 1, policy_version 1222881 (0.0011) [2023-12-27 14:07:45,163][20381] Updated weights for policy 1, policy_version 1222891 (0.0011) [2023-12-27 14:07:45,226][20381] Updated weights for policy 1, policy_version 1222901 (0.0011) [2023-12-27 14:07:45,598][20347] Updated weights for policy 0, policy_version 1224008 (0.0008) [2023-12-27 14:07:45,659][20347] Updated weights for policy 0, policy_version 1224018 (0.0008) [2023-12-27 14:07:45,727][20347] Updated weights for policy 0, policy_version 1224028 (0.0008) [2023-12-27 14:07:45,861][20381] Updated weights for policy 1, policy_version 1222911 (0.0011) [2023-12-27 14:07:45,913][20381] Updated weights for policy 1, policy_version 1222921 (0.0011) [2023-12-27 14:07:45,966][20381] Updated weights for policy 1, policy_version 1222931 (0.0010) [2023-12-27 14:07:46,370][20347] Updated weights for policy 0, policy_version 1224038 (0.0010) [2023-12-27 14:07:46,428][20347] Updated weights for policy 0, policy_version 1224048 (0.0010) [2023-12-27 14:07:46,479][20347] Updated weights for policy 0, policy_version 1224058 (0.0010) [2023-12-27 14:07:46,642][20381] Updated weights for policy 1, policy_version 1222941 (0.0011) [2023-12-27 14:07:46,699][20381] Updated weights for policy 1, policy_version 1222951 (0.0010) [2023-12-27 14:07:46,761][20381] Updated weights for policy 1, policy_version 1222961 (0.0011) [2023-12-27 14:07:47,087][20347] Updated weights for policy 0, policy_version 1224068 (0.0010) [2023-12-27 14:07:47,148][20347] Updated weights for policy 0, policy_version 1224078 (0.0010) [2023-12-27 14:07:47,198][20347] Updated weights for policy 0, policy_version 1224088 (0.0010) [2023-12-27 14:07:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 626532352. Throughput: 0: 10354.5, 1: 10385.1. Samples: 626542072. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:07:47,364][19170] Avg episode reward: [(0, '815.710'), (1, '814.070')] [2023-12-27 14:07:47,432][20381] Updated weights for policy 1, policy_version 1222971 (0.0010) [2023-12-27 14:07:47,496][20381] Updated weights for policy 1, policy_version 1222981 (0.0011) [2023-12-27 14:07:47,557][20381] Updated weights for policy 1, policy_version 1222991 (0.0005) [2023-12-27 14:07:47,823][20347] Updated weights for policy 0, policy_version 1224098 (0.0010) [2023-12-27 14:07:47,884][20347] Updated weights for policy 0, policy_version 1224108 (0.0010) [2023-12-27 14:07:47,932][20347] Updated weights for policy 0, policy_version 1224118 (0.0010) [2023-12-27 14:07:47,988][20347] Updated weights for policy 0, policy_version 1224128 (0.0010) [2023-12-27 14:07:48,078][20381] Updated weights for policy 1, policy_version 1223001 (0.0006) [2023-12-27 14:07:48,126][20381] Updated weights for policy 1, policy_version 1223011 (0.0010) [2023-12-27 14:07:48,178][20381] Updated weights for policy 1, policy_version 1223021 (0.0010) [2023-12-27 14:07:48,223][20381] Updated weights for policy 1, policy_version 1223031 (0.0010) [2023-12-27 14:07:48,716][20347] Updated weights for policy 0, policy_version 1224138 (0.0010) [2023-12-27 14:07:48,769][20347] Updated weights for policy 0, policy_version 1224148 (0.0007) [2023-12-27 14:07:48,833][20347] Updated weights for policy 0, policy_version 1224158 (0.0008) [2023-12-27 14:07:48,946][20381] Updated weights for policy 1, policy_version 1223041 (0.0007) [2023-12-27 14:07:48,994][20381] Updated weights for policy 1, policy_version 1223051 (0.0007) [2023-12-27 14:07:49,048][20381] Updated weights for policy 1, policy_version 1223061 (0.0007) [2023-12-27 14:07:49,531][20347] Updated weights for policy 0, policy_version 1224168 (0.0008) [2023-12-27 14:07:49,589][20347] Updated weights for policy 0, policy_version 1224178 (0.0007) [2023-12-27 14:07:49,644][20347] Updated weights for policy 0, policy_version 1224188 (0.0008) [2023-12-27 14:07:49,763][20381] Updated weights for policy 1, policy_version 1223071 (0.0009) [2023-12-27 14:07:49,823][20381] Updated weights for policy 1, policy_version 1223081 (0.0010) [2023-12-27 14:07:49,884][20381] Updated weights for policy 1, policy_version 1223091 (0.0008) [2023-12-27 14:07:50,296][20347] Updated weights for policy 0, policy_version 1224198 (0.0006) [2023-12-27 14:07:50,352][20347] Updated weights for policy 0, policy_version 1224208 (0.0008) [2023-12-27 14:07:50,403][20347] Updated weights for policy 0, policy_version 1224218 (0.0008) [2023-12-27 14:07:50,639][20381] Updated weights for policy 1, policy_version 1223101 (0.0009) [2023-12-27 14:07:50,691][20381] Updated weights for policy 1, policy_version 1223111 (0.0007) [2023-12-27 14:07:50,749][20381] Updated weights for policy 1, policy_version 1223121 (0.0007) [2023-12-27 14:07:51,101][20347] Updated weights for policy 0, policy_version 1224228 (0.0008) [2023-12-27 14:07:51,169][20347] Updated weights for policy 0, policy_version 1224238 (0.0011) [2023-12-27 14:07:51,225][20347] Updated weights for policy 0, policy_version 1224248 (0.0010) [2023-12-27 14:07:51,471][20381] Updated weights for policy 1, policy_version 1223131 (0.0006) [2023-12-27 14:07:51,542][20381] Updated weights for policy 1, policy_version 1223141 (0.0006) [2023-12-27 14:07:51,595][20381] Updated weights for policy 1, policy_version 1223151 (0.0008) [2023-12-27 14:07:51,967][20347] Updated weights for policy 0, policy_version 1224258 (0.0011) [2023-12-27 14:07:52,028][20347] Updated weights for policy 0, policy_version 1224268 (0.0006) [2023-12-27 14:07:52,085][20347] Updated weights for policy 0, policy_version 1224278 (0.0008) [2023-12-27 14:07:52,141][20347] Updated weights for policy 0, policy_version 1224288 (0.0006) [2023-12-27 14:07:52,326][20381] Updated weights for policy 1, policy_version 1223161 (0.0008) [2023-12-27 14:07:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 626630656. Throughput: 0: 10365.1, 1: 10319.5. Samples: 626603220. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:07:52,363][19170] Avg episode reward: [(0, '796.310'), (1, '843.450')] [2023-12-27 14:07:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001224288_313475072.pth... [2023-12-27 14:07:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001223072_313163776.pth [2023-12-27 14:07:52,389][20381] Updated weights for policy 1, policy_version 1223171 (0.0008) [2023-12-27 14:07:52,443][20381] Updated weights for policy 1, policy_version 1223181 (0.0005) [2023-12-27 14:07:52,499][20381] Updated weights for policy 1, policy_version 1223191 (0.0005) [2023-12-27 14:07:52,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001223192_313163776.pth... [2023-12-27 14:07:52,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001221976_312852480.pth [2023-12-27 14:07:52,885][20347] Updated weights for policy 0, policy_version 1224298 (0.0008) [2023-12-27 14:07:52,936][20347] Updated weights for policy 0, policy_version 1224308 (0.0008) [2023-12-27 14:07:52,988][20347] Updated weights for policy 0, policy_version 1224318 (0.0009) [2023-12-27 14:07:53,099][20381] Updated weights for policy 1, policy_version 1223201 (0.0005) [2023-12-27 14:07:53,143][20381] Updated weights for policy 1, policy_version 1223211 (0.0005) [2023-12-27 14:07:53,198][20381] Updated weights for policy 1, policy_version 1223221 (0.0005) [2023-12-27 14:07:53,622][20347] Updated weights for policy 0, policy_version 1224328 (0.0006) [2023-12-27 14:07:53,677][20347] Updated weights for policy 0, policy_version 1224338 (0.0005) [2023-12-27 14:07:53,730][20347] Updated weights for policy 0, policy_version 1224348 (0.0007) [2023-12-27 14:07:53,832][20381] Updated weights for policy 1, policy_version 1223231 (0.0008) [2023-12-27 14:07:53,901][20381] Updated weights for policy 1, policy_version 1223241 (0.0011) [2023-12-27 14:07:53,956][20381] Updated weights for policy 1, policy_version 1223251 (0.0011) [2023-12-27 14:07:54,349][20347] Updated weights for policy 0, policy_version 1224358 (0.0007) [2023-12-27 14:07:54,407][20347] Updated weights for policy 0, policy_version 1224368 (0.0008) [2023-12-27 14:07:54,463][20347] Updated weights for policy 0, policy_version 1224378 (0.0007) [2023-12-27 14:07:54,639][20381] Updated weights for policy 1, policy_version 1223261 (0.0011) [2023-12-27 14:07:54,707][20381] Updated weights for policy 1, policy_version 1223271 (0.0009) [2023-12-27 14:07:54,772][20381] Updated weights for policy 1, policy_version 1223281 (0.0008) [2023-12-27 14:07:55,191][20347] Updated weights for policy 0, policy_version 1224388 (0.0007) [2023-12-27 14:07:55,251][20347] Updated weights for policy 0, policy_version 1224398 (0.0009) [2023-12-27 14:07:55,310][20347] Updated weights for policy 0, policy_version 1224408 (0.0008) [2023-12-27 14:07:55,400][20381] Updated weights for policy 1, policy_version 1223291 (0.0008) [2023-12-27 14:07:55,448][20381] Updated weights for policy 1, policy_version 1223301 (0.0008) [2023-12-27 14:07:55,493][20381] Updated weights for policy 1, policy_version 1223311 (0.0006) [2023-12-27 14:07:56,048][20347] Updated weights for policy 0, policy_version 1224418 (0.0008) [2023-12-27 14:07:56,099][20347] Updated weights for policy 0, policy_version 1224428 (0.0008) [2023-12-27 14:07:56,106][20381] Updated weights for policy 1, policy_version 1223321 (0.0005) [2023-12-27 14:07:56,155][20347] Updated weights for policy 0, policy_version 1224438 (0.0008) [2023-12-27 14:07:56,158][20381] Updated weights for policy 1, policy_version 1223331 (0.0006) [2023-12-27 14:07:56,209][20347] Updated weights for policy 0, policy_version 1224448 (0.0008) [2023-12-27 14:07:56,218][20381] Updated weights for policy 1, policy_version 1223341 (0.0005) [2023-12-27 14:07:56,277][20381] Updated weights for policy 1, policy_version 1223351 (0.0006) [2023-12-27 14:07:56,859][20381] Updated weights for policy 1, policy_version 1223361 (0.0005) [2023-12-27 14:07:56,872][20347] Updated weights for policy 0, policy_version 1224458 (0.0010) [2023-12-27 14:07:56,914][20381] Updated weights for policy 1, policy_version 1223371 (0.0005) [2023-12-27 14:07:56,920][20347] Updated weights for policy 0, policy_version 1224468 (0.0010) [2023-12-27 14:07:56,965][20347] Updated weights for policy 0, policy_version 1224478 (0.0009) [2023-12-27 14:07:56,968][20381] Updated weights for policy 1, policy_version 1223381 (0.0006) [2023-12-27 14:07:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 626737152. Throughput: 0: 10342.0, 1: 10338.1. Samples: 626725808. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:07:57,363][19170] Avg episode reward: [(0, '814.480'), (1, '833.680')] [2023-12-27 14:07:57,507][20381] Updated weights for policy 1, policy_version 1223391 (0.0007) [2023-12-27 14:07:57,527][20347] Updated weights for policy 0, policy_version 1224488 (0.0005) [2023-12-27 14:07:57,552][20381] Updated weights for policy 1, policy_version 1223401 (0.0007) [2023-12-27 14:07:57,572][20347] Updated weights for policy 0, policy_version 1224498 (0.0005) [2023-12-27 14:07:57,598][20381] Updated weights for policy 1, policy_version 1223411 (0.0006) [2023-12-27 14:07:57,618][20347] Updated weights for policy 0, policy_version 1224508 (0.0005) [2023-12-27 14:07:58,178][20381] Updated weights for policy 1, policy_version 1223421 (0.0006) [2023-12-27 14:07:58,191][20347] Updated weights for policy 0, policy_version 1224518 (0.0008) [2023-12-27 14:07:58,241][20381] Updated weights for policy 1, policy_version 1223431 (0.0008) [2023-12-27 14:07:58,256][20347] Updated weights for policy 0, policy_version 1224528 (0.0011) [2023-12-27 14:07:58,301][20381] Updated weights for policy 1, policy_version 1223441 (0.0006) [2023-12-27 14:07:58,325][20347] Updated weights for policy 0, policy_version 1224538 (0.0010) [2023-12-27 14:07:59,060][20347] Updated weights for policy 0, policy_version 1224548 (0.0009) [2023-12-27 14:07:59,085][20381] Updated weights for policy 1, policy_version 1223451 (0.0008) [2023-12-27 14:07:59,118][20347] Updated weights for policy 0, policy_version 1224558 (0.0006) [2023-12-27 14:07:59,141][20381] Updated weights for policy 1, policy_version 1223461 (0.0005) [2023-12-27 14:07:59,178][20347] Updated weights for policy 0, policy_version 1224568 (0.0007) [2023-12-27 14:07:59,203][20381] Updated weights for policy 1, policy_version 1223471 (0.0005) [2023-12-27 14:07:59,929][20347] Updated weights for policy 0, policy_version 1224579 (0.0008) [2023-12-27 14:07:59,949][20381] Updated weights for policy 1, policy_version 1223481 (0.0008) [2023-12-27 14:07:59,992][20347] Updated weights for policy 0, policy_version 1224589 (0.0007) [2023-12-27 14:08:00,013][20381] Updated weights for policy 1, policy_version 1223491 (0.0006) [2023-12-27 14:08:00,058][20347] Updated weights for policy 0, policy_version 1224599 (0.0006) [2023-12-27 14:08:00,074][20381] Updated weights for policy 1, policy_version 1223501 (0.0007) [2023-12-27 14:08:00,132][20381] Updated weights for policy 1, policy_version 1223511 (0.0007) [2023-12-27 14:08:00,637][20347] Updated weights for policy 0, policy_version 1224609 (0.0005) [2023-12-27 14:08:00,694][20347] Updated weights for policy 0, policy_version 1224619 (0.0007) [2023-12-27 14:08:00,758][20347] Updated weights for policy 0, policy_version 1224629 (0.0008) [2023-12-27 14:08:00,786][20381] Updated weights for policy 1, policy_version 1223521 (0.0007) [2023-12-27 14:08:00,809][20347] Updated weights for policy 0, policy_version 1224639 (0.0010) [2023-12-27 14:08:00,836][20381] Updated weights for policy 1, policy_version 1223531 (0.0005) [2023-12-27 14:08:00,889][20381] Updated weights for policy 1, policy_version 1223541 (0.0006) [2023-12-27 14:08:01,479][20347] Updated weights for policy 0, policy_version 1224649 (0.0006) [2023-12-27 14:08:01,537][20347] Updated weights for policy 0, policy_version 1224659 (0.0007) [2023-12-27 14:08:01,593][20347] Updated weights for policy 0, policy_version 1224669 (0.0010) [2023-12-27 14:08:01,595][20381] Updated weights for policy 1, policy_version 1223551 (0.0006) [2023-12-27 14:08:01,659][20381] Updated weights for policy 1, policy_version 1223561 (0.0008) [2023-12-27 14:08:01,710][20381] Updated weights for policy 1, policy_version 1223571 (0.0008) [2023-12-27 14:08:02,293][20347] Updated weights for policy 0, policy_version 1224679 (0.0009) [2023-12-27 14:08:02,346][20347] Updated weights for policy 0, policy_version 1224689 (0.0010) [2023-12-27 14:08:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 626835456. Throughput: 0: 10361.8, 1: 10371.2. Samples: 626852772. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:02,363][19170] Avg episode reward: [(0, '746.210'), (1, '814.110')] [2023-12-27 14:08:02,377][20381] Updated weights for policy 1, policy_version 1223581 (0.0007) [2023-12-27 14:08:02,412][20347] Updated weights for policy 0, policy_version 1224699 (0.0011) [2023-12-27 14:08:02,435][20381] Updated weights for policy 1, policy_version 1223591 (0.0006) [2023-12-27 14:08:02,482][20381] Updated weights for policy 1, policy_version 1223601 (0.0008) [2023-12-27 14:08:03,079][20347] Updated weights for policy 0, policy_version 1224709 (0.0010) [2023-12-27 14:08:03,106][20381] Updated weights for policy 1, policy_version 1223611 (0.0006) [2023-12-27 14:08:03,133][20347] Updated weights for policy 0, policy_version 1224719 (0.0010) [2023-12-27 14:08:03,156][20381] Updated weights for policy 1, policy_version 1223621 (0.0005) [2023-12-27 14:08:03,190][20347] Updated weights for policy 0, policy_version 1224729 (0.0010) [2023-12-27 14:08:03,210][20381] Updated weights for policy 1, policy_version 1223631 (0.0006) [2023-12-27 14:08:03,807][20347] Updated weights for policy 0, policy_version 1224739 (0.0009) [2023-12-27 14:08:03,862][20347] Updated weights for policy 0, policy_version 1224749 (0.0007) [2023-12-27 14:08:03,907][20381] Updated weights for policy 1, policy_version 1223641 (0.0006) [2023-12-27 14:08:03,920][20347] Updated weights for policy 0, policy_version 1224759 (0.0006) [2023-12-27 14:08:03,968][20381] Updated weights for policy 1, policy_version 1223651 (0.0009) [2023-12-27 14:08:04,023][20381] Updated weights for policy 1, policy_version 1223661 (0.0008) [2023-12-27 14:08:04,084][20381] Updated weights for policy 1, policy_version 1223671 (0.0007) [2023-12-27 14:08:04,613][20347] Updated weights for policy 0, policy_version 1224769 (0.0010) [2023-12-27 14:08:04,671][20347] Updated weights for policy 0, policy_version 1224779 (0.0010) [2023-12-27 14:08:04,722][20347] Updated weights for policy 0, policy_version 1224789 (0.0010) [2023-12-27 14:08:04,735][20381] Updated weights for policy 1, policy_version 1223681 (0.0006) [2023-12-27 14:08:04,769][20347] Updated weights for policy 0, policy_version 1224799 (0.0010) [2023-12-27 14:08:04,784][20381] Updated weights for policy 1, policy_version 1223691 (0.0007) [2023-12-27 14:08:04,841][20381] Updated weights for policy 1, policy_version 1223701 (0.0007) [2023-12-27 14:08:05,429][20347] Updated weights for policy 0, policy_version 1224809 (0.0009) [2023-12-27 14:08:05,483][20347] Updated weights for policy 0, policy_version 1224819 (0.0008) [2023-12-27 14:08:05,524][20381] Updated weights for policy 1, policy_version 1223711 (0.0006) [2023-12-27 14:08:05,537][20347] Updated weights for policy 0, policy_version 1224829 (0.0009) [2023-12-27 14:08:05,579][20381] Updated weights for policy 1, policy_version 1223721 (0.0006) [2023-12-27 14:08:05,641][20381] Updated weights for policy 1, policy_version 1223731 (0.0008) [2023-12-27 14:08:06,211][20347] Updated weights for policy 0, policy_version 1224839 (0.0009) [2023-12-27 14:08:06,268][20347] Updated weights for policy 0, policy_version 1224849 (0.0008) [2023-12-27 14:08:06,330][20347] Updated weights for policy 0, policy_version 1224859 (0.0008) [2023-12-27 14:08:06,362][20381] Updated weights for policy 1, policy_version 1223741 (0.0006) [2023-12-27 14:08:06,422][20381] Updated weights for policy 1, policy_version 1223751 (0.0008) [2023-12-27 14:08:06,485][20381] Updated weights for policy 1, policy_version 1223761 (0.0007) [2023-12-27 14:08:07,031][20347] Updated weights for policy 0, policy_version 1224869 (0.0007) [2023-12-27 14:08:07,078][20347] Updated weights for policy 0, policy_version 1224879 (0.0008) [2023-12-27 14:08:07,127][20347] Updated weights for policy 0, policy_version 1224889 (0.0008) [2023-12-27 14:08:07,187][20381] Updated weights for policy 1, policy_version 1223771 (0.0007) [2023-12-27 14:08:07,243][20381] Updated weights for policy 1, policy_version 1223781 (0.0008) [2023-12-27 14:08:07,299][20381] Updated weights for policy 1, policy_version 1223791 (0.0008) [2023-12-27 14:08:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 626950144. Throughput: 0: 10394.5, 1: 10357.4. Samples: 626914776. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:07,364][19170] Avg episode reward: [(0, '707.850'), (1, '843.380')] [2023-12-27 14:08:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001223800_313319424.pth... [2023-12-27 14:08:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001224896_313630720.pth... [2023-12-27 14:08:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001223680_313319424.pth [2023-12-27 14:08:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001222584_313008128.pth [2023-12-27 14:08:07,810][20347] Updated weights for policy 0, policy_version 1224899 (0.0008) [2023-12-27 14:08:07,870][20347] Updated weights for policy 0, policy_version 1224909 (0.0008) [2023-12-27 14:08:07,915][20347] Updated weights for policy 0, policy_version 1224919 (0.0008) [2023-12-27 14:08:07,968][20381] Updated weights for policy 1, policy_version 1223801 (0.0005) [2023-12-27 14:08:08,035][20381] Updated weights for policy 1, policy_version 1223811 (0.0007) [2023-12-27 14:08:08,080][20381] Updated weights for policy 1, policy_version 1223821 (0.0008) [2023-12-27 14:08:08,135][20381] Updated weights for policy 1, policy_version 1223831 (0.0007) [2023-12-27 14:08:08,628][20347] Updated weights for policy 0, policy_version 1224929 (0.0007) [2023-12-27 14:08:08,681][20347] Updated weights for policy 0, policy_version 1224939 (0.0007) [2023-12-27 14:08:08,740][20347] Updated weights for policy 0, policy_version 1224949 (0.0008) [2023-12-27 14:08:08,792][20347] Updated weights for policy 0, policy_version 1224959 (0.0007) [2023-12-27 14:08:08,851][20381] Updated weights for policy 1, policy_version 1223841 (0.0009) [2023-12-27 14:08:08,911][20381] Updated weights for policy 1, policy_version 1223851 (0.0009) [2023-12-27 14:08:08,970][20381] Updated weights for policy 1, policy_version 1223861 (0.0008) [2023-12-27 14:08:09,510][20347] Updated weights for policy 0, policy_version 1224969 (0.0006) [2023-12-27 14:08:09,576][20347] Updated weights for policy 0, policy_version 1224979 (0.0007) [2023-12-27 14:08:09,643][20347] Updated weights for policy 0, policy_version 1224989 (0.0008) [2023-12-27 14:08:09,664][20381] Updated weights for policy 1, policy_version 1223871 (0.0007) [2023-12-27 14:08:09,723][20381] Updated weights for policy 1, policy_version 1223881 (0.0008) [2023-12-27 14:08:09,788][20381] Updated weights for policy 1, policy_version 1223891 (0.0008) [2023-12-27 14:08:10,356][20347] Updated weights for policy 0, policy_version 1224999 (0.0006) [2023-12-27 14:08:10,419][20347] Updated weights for policy 0, policy_version 1225009 (0.0008) [2023-12-27 14:08:10,476][20347] Updated weights for policy 0, policy_version 1225019 (0.0008) [2023-12-27 14:08:10,495][20381] Updated weights for policy 1, policy_version 1223901 (0.0009) [2023-12-27 14:08:10,544][20381] Updated weights for policy 1, policy_version 1223911 (0.0008) [2023-12-27 14:08:10,606][20381] Updated weights for policy 1, policy_version 1223921 (0.0008) [2023-12-27 14:08:11,191][20347] Updated weights for policy 0, policy_version 1225029 (0.0010) [2023-12-27 14:08:11,248][20347] Updated weights for policy 0, policy_version 1225039 (0.0010) [2023-12-27 14:08:11,307][20381] Updated weights for policy 1, policy_version 1223931 (0.0008) [2023-12-27 14:08:11,307][20347] Updated weights for policy 0, policy_version 1225049 (0.0008) [2023-12-27 14:08:11,365][20381] Updated weights for policy 1, policy_version 1223941 (0.0008) [2023-12-27 14:08:11,430][20381] Updated weights for policy 1, policy_version 1223951 (0.0009) [2023-12-27 14:08:12,010][20347] Updated weights for policy 0, policy_version 1225059 (0.0008) [2023-12-27 14:08:12,083][20347] Updated weights for policy 0, policy_version 1225069 (0.0006) [2023-12-27 14:08:12,143][20381] Updated weights for policy 1, policy_version 1223961 (0.0010) [2023-12-27 14:08:12,148][20347] Updated weights for policy 0, policy_version 1225079 (0.0007) [2023-12-27 14:08:12,203][20381] Updated weights for policy 1, policy_version 1223971 (0.0007) [2023-12-27 14:08:12,265][20381] Updated weights for policy 1, policy_version 1223981 (0.0007) [2023-12-27 14:08:12,326][20381] Updated weights for policy 1, policy_version 1223991 (0.0009) [2023-12-27 14:08:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 627048448. Throughput: 0: 10377.5, 1: 10340.1. Samples: 627034300. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:12,363][19170] Avg episode reward: [(0, '717.880'), (1, '892.400')] [2023-12-27 14:08:12,762][20347] Updated weights for policy 0, policy_version 1225089 (0.0008) [2023-12-27 14:08:12,829][20347] Updated weights for policy 0, policy_version 1225099 (0.0008) [2023-12-27 14:08:12,831][20291] KL-divergence is very high: 113.4225 [2023-12-27 14:08:12,899][20347] Updated weights for policy 0, policy_version 1225109 (0.0008) [2023-12-27 14:08:12,947][20381] Updated weights for policy 1, policy_version 1224001 (0.0007) [2023-12-27 14:08:12,963][20347] Updated weights for policy 0, policy_version 1225119 (0.0007) [2023-12-27 14:08:12,963][20291] KL-divergence is very high: 119.8165 [2023-12-27 14:08:13,009][20381] Updated weights for policy 1, policy_version 1224011 (0.0006) [2023-12-27 14:08:13,072][20381] Updated weights for policy 1, policy_version 1224021 (0.0006) [2023-12-27 14:08:13,602][20291] KL-divergence is very high: 421.8682 [2023-12-27 14:08:13,613][20291] KL-divergence is very high: 176.3735 [2023-12-27 14:08:13,619][20291] KL-divergence is very high: 443.0884 [2023-12-27 14:08:13,648][20291] KL-divergence is very high: 137.9647 [2023-12-27 14:08:13,652][20347] Updated weights for policy 0, policy_version 1225129 (0.0007) [2023-12-27 14:08:13,665][20291] KL-divergence is very high: 142.2885 [2023-12-27 14:08:13,669][20381] Updated weights for policy 1, policy_version 1224031 (0.0010) [2023-12-27 14:08:13,671][20291] KL-divergence is very high: 133.3432 [2023-12-27 14:08:13,677][20291] KL-divergence is very high: 133.6793 [2023-12-27 14:08:13,694][20291] KL-divergence is very high: 132.3575 [2023-12-27 14:08:13,711][20291] KL-divergence is very high: 219.6055 [2023-12-27 14:08:13,711][20347] Updated weights for policy 0, policy_version 1225139 (0.0005) [2023-12-27 14:08:13,714][20381] Updated weights for policy 1, policy_version 1224041 (0.0010) [2023-12-27 14:08:13,738][20291] KL-divergence is very high: 202.5042 [2023-12-27 14:08:13,755][20291] KL-divergence is very high: 278.2550 [2023-12-27 14:08:13,763][20381] Updated weights for policy 1, policy_version 1224051 (0.0010) [2023-12-27 14:08:13,768][20347] Updated weights for policy 0, policy_version 1225149 (0.0006) [2023-12-27 14:08:14,458][20381] Updated weights for policy 1, policy_version 1224061 (0.0009) [2023-12-27 14:08:14,465][20347] Updated weights for policy 0, policy_version 1225159 (0.0007) [2023-12-27 14:08:14,476][20291] KL-divergence is very high: 103.5815 [2023-12-27 14:08:14,508][20381] Updated weights for policy 1, policy_version 1224071 (0.0011) [2023-12-27 14:08:14,521][20347] Updated weights for policy 0, policy_version 1225169 (0.0007) [2023-12-27 14:08:14,563][20381] Updated weights for policy 1, policy_version 1224081 (0.0010) [2023-12-27 14:08:14,573][20347] Updated weights for policy 0, policy_version 1225179 (0.0005) [2023-12-27 14:08:15,277][20347] Updated weights for policy 0, policy_version 1225189 (0.0006) [2023-12-27 14:08:15,280][20381] Updated weights for policy 1, policy_version 1224091 (0.0010) [2023-12-27 14:08:15,328][20347] Updated weights for policy 0, policy_version 1225199 (0.0006) [2023-12-27 14:08:15,336][20381] Updated weights for policy 1, policy_version 1224101 (0.0010) [2023-12-27 14:08:15,376][20347] Updated weights for policy 0, policy_version 1225209 (0.0007) [2023-12-27 14:08:15,386][20381] Updated weights for policy 1, policy_version 1224111 (0.0006) [2023-12-27 14:08:16,066][20381] Updated weights for policy 1, policy_version 1224121 (0.0007) [2023-12-27 14:08:16,091][20347] Updated weights for policy 0, policy_version 1225219 (0.0007) [2023-12-27 14:08:16,111][20381] Updated weights for policy 1, policy_version 1224131 (0.0010) [2023-12-27 14:08:16,135][20347] Updated weights for policy 0, policy_version 1225229 (0.0005) [2023-12-27 14:08:16,156][20381] Updated weights for policy 1, policy_version 1224141 (0.0006) [2023-12-27 14:08:16,181][20347] Updated weights for policy 0, policy_version 1225239 (0.0006) [2023-12-27 14:08:16,205][20381] Updated weights for policy 1, policy_version 1224151 (0.0005) [2023-12-27 14:08:16,813][20347] Updated weights for policy 0, policy_version 1225249 (0.0007) [2023-12-27 14:08:16,864][20347] Updated weights for policy 0, policy_version 1225259 (0.0009) [2023-12-27 14:08:16,926][20381] Updated weights for policy 1, policy_version 1224161 (0.0008) [2023-12-27 14:08:16,932][20347] Updated weights for policy 0, policy_version 1225269 (0.0011) [2023-12-27 14:08:16,978][20381] Updated weights for policy 1, policy_version 1224171 (0.0007) [2023-12-27 14:08:16,990][20347] Updated weights for policy 0, policy_version 1225279 (0.0008) [2023-12-27 14:08:17,027][20381] Updated weights for policy 1, policy_version 1224181 (0.0008) [2023-12-27 14:08:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 627146752. Throughput: 0: 10275.9, 1: 10275.3. Samples: 627156360. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:17,363][19170] Avg episode reward: [(0, '756.960'), (1, '931.550')] [2023-12-27 14:08:17,642][20347] Updated weights for policy 0, policy_version 1225289 (0.0010) [2023-12-27 14:08:17,693][20347] Updated weights for policy 0, policy_version 1225299 (0.0010) [2023-12-27 14:08:17,735][20381] Updated weights for policy 1, policy_version 1224191 (0.0006) [2023-12-27 14:08:17,749][20347] Updated weights for policy 0, policy_version 1225309 (0.0010) [2023-12-27 14:08:17,788][20381] Updated weights for policy 1, policy_version 1224201 (0.0006) [2023-12-27 14:08:17,841][20381] Updated weights for policy 1, policy_version 1224211 (0.0007) [2023-12-27 14:08:18,378][20347] Updated weights for policy 0, policy_version 1225319 (0.0008) [2023-12-27 14:08:18,442][20347] Updated weights for policy 0, policy_version 1225329 (0.0005) [2023-12-27 14:08:18,503][20347] Updated weights for policy 0, policy_version 1225339 (0.0005) [2023-12-27 14:08:18,574][20381] Updated weights for policy 1, policy_version 1224221 (0.0008) [2023-12-27 14:08:18,631][20381] Updated weights for policy 1, policy_version 1224231 (0.0009) [2023-12-27 14:08:18,685][20381] Updated weights for policy 1, policy_version 1224241 (0.0007) [2023-12-27 14:08:19,108][20347] Updated weights for policy 0, policy_version 1225349 (0.0008) [2023-12-27 14:08:19,157][20347] Updated weights for policy 0, policy_version 1225359 (0.0010) [2023-12-27 14:08:19,220][20347] Updated weights for policy 0, policy_version 1225369 (0.0009) [2023-12-27 14:08:19,421][20381] Updated weights for policy 1, policy_version 1224251 (0.0008) [2023-12-27 14:08:19,467][20292] KL-divergence is very high: 346.5456 [2023-12-27 14:08:19,487][20381] Updated weights for policy 1, policy_version 1224261 (0.0008) [2023-12-27 14:08:19,521][20292] KL-divergence is very high: 171.6353 [2023-12-27 14:08:19,551][20381] Updated weights for policy 1, policy_version 1224271 (0.0007) [2023-12-27 14:08:19,944][20347] Updated weights for policy 0, policy_version 1225379 (0.0010) [2023-12-27 14:08:20,000][20347] Updated weights for policy 0, policy_version 1225389 (0.0007) [2023-12-27 14:08:20,059][20347] Updated weights for policy 0, policy_version 1225399 (0.0006) [2023-12-27 14:08:20,325][20381] Updated weights for policy 1, policy_version 1224281 (0.0008) [2023-12-27 14:08:20,385][20381] Updated weights for policy 1, policy_version 1224291 (0.0008) [2023-12-27 14:08:20,434][20381] Updated weights for policy 1, policy_version 1224301 (0.0008) [2023-12-27 14:08:20,439][20292] KL-divergence is very high: 118.8854 [2023-12-27 14:08:20,456][20292] KL-divergence is very high: 115.3597 [2023-12-27 14:08:20,462][20292] KL-divergence is very high: 151.7210 [2023-12-27 14:08:20,489][20381] Updated weights for policy 1, policy_version 1224311 (0.0008) [2023-12-27 14:08:20,742][20347] Updated weights for policy 0, policy_version 1225409 (0.0008) [2023-12-27 14:08:20,803][20347] Updated weights for policy 0, policy_version 1225419 (0.0008) [2023-12-27 14:08:20,861][20347] Updated weights for policy 0, policy_version 1225429 (0.0008) [2023-12-27 14:08:20,920][20347] Updated weights for policy 0, policy_version 1225439 (0.0008) [2023-12-27 14:08:21,254][20381] Updated weights for policy 1, policy_version 1224321 (0.0008) [2023-12-27 14:08:21,320][20381] Updated weights for policy 1, policy_version 1224331 (0.0008) [2023-12-27 14:08:21,388][20381] Updated weights for policy 1, policy_version 1224341 (0.0007) [2023-12-27 14:08:21,688][20347] Updated weights for policy 0, policy_version 1225449 (0.0008) [2023-12-27 14:08:21,756][20347] Updated weights for policy 0, policy_version 1225459 (0.0008) [2023-12-27 14:08:21,817][20347] Updated weights for policy 0, policy_version 1225469 (0.0008) [2023-12-27 14:08:22,093][20381] Updated weights for policy 1, policy_version 1224351 (0.0008) [2023-12-27 14:08:22,155][20381] Updated weights for policy 1, policy_version 1224361 (0.0008) [2023-12-27 14:08:22,218][20381] Updated weights for policy 1, policy_version 1224371 (0.0008) [2023-12-27 14:08:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 627245056. Throughput: 0: 10261.6, 1: 10209.5. Samples: 627216748. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:22,363][19170] Avg episode reward: [(0, '679.560'), (1, '892.550')] [2023-12-27 14:08:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001224376_313466880.pth... [2023-12-27 14:08:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001225472_313778176.pth... [2023-12-27 14:08:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001224288_313475072.pth [2023-12-27 14:08:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001223192_313163776.pth [2023-12-27 14:08:22,463][20347] Updated weights for policy 0, policy_version 1225479 (0.0007) [2023-12-27 14:08:22,513][20347] Updated weights for policy 0, policy_version 1225489 (0.0008) [2023-12-27 14:08:22,558][20347] Updated weights for policy 0, policy_version 1225499 (0.0008) [2023-12-27 14:08:22,864][20381] Updated weights for policy 1, policy_version 1224381 (0.0007) [2023-12-27 14:08:22,922][20381] Updated weights for policy 1, policy_version 1224391 (0.0006) [2023-12-27 14:08:22,980][20381] Updated weights for policy 1, policy_version 1224401 (0.0008) [2023-12-27 14:08:23,319][20347] Updated weights for policy 0, policy_version 1225509 (0.0005) [2023-12-27 14:08:23,368][20347] Updated weights for policy 0, policy_version 1225519 (0.0009) [2023-12-27 14:08:23,434][20347] Updated weights for policy 0, policy_version 1225529 (0.0008) [2023-12-27 14:08:23,578][20381] Updated weights for policy 1, policy_version 1224411 (0.0007) [2023-12-27 14:08:23,633][20381] Updated weights for policy 1, policy_version 1224421 (0.0005) [2023-12-27 14:08:23,695][20381] Updated weights for policy 1, policy_version 1224431 (0.0005) [2023-12-27 14:08:24,037][20347] Updated weights for policy 0, policy_version 1225539 (0.0007) [2023-12-27 14:08:24,105][20347] Updated weights for policy 0, policy_version 1225549 (0.0006) [2023-12-27 14:08:24,162][20347] Updated weights for policy 0, policy_version 1225559 (0.0007) [2023-12-27 14:08:24,259][20381] Updated weights for policy 1, policy_version 1224441 (0.0005) [2023-12-27 14:08:24,314][20381] Updated weights for policy 1, policy_version 1224451 (0.0008) [2023-12-27 14:08:24,373][20381] Updated weights for policy 1, policy_version 1224461 (0.0008) [2023-12-27 14:08:24,428][20381] Updated weights for policy 1, policy_version 1224471 (0.0008) [2023-12-27 14:08:24,815][20347] Updated weights for policy 0, policy_version 1225569 (0.0008) [2023-12-27 14:08:24,877][20347] Updated weights for policy 0, policy_version 1225579 (0.0010) [2023-12-27 14:08:24,937][20347] Updated weights for policy 0, policy_version 1225589 (0.0007) [2023-12-27 14:08:25,003][20347] Updated weights for policy 0, policy_version 1225599 (0.0007) [2023-12-27 14:08:25,124][20381] Updated weights for policy 1, policy_version 1224481 (0.0006) [2023-12-27 14:08:25,181][20381] Updated weights for policy 1, policy_version 1224491 (0.0008) [2023-12-27 14:08:25,239][20381] Updated weights for policy 1, policy_version 1224501 (0.0007) [2023-12-27 14:08:25,716][20347] Updated weights for policy 0, policy_version 1225609 (0.0006) [2023-12-27 14:08:25,744][20291] KL-divergence is very high: 104.6519 [2023-12-27 14:08:25,755][20291] KL-divergence is very high: 116.5723 [2023-12-27 14:08:25,776][20291] KL-divergence is very high: 105.6039 [2023-12-27 14:08:25,782][20347] Updated weights for policy 0, policy_version 1225619 (0.0006) [2023-12-27 14:08:25,835][20381] Updated weights for policy 1, policy_version 1224511 (0.0006) [2023-12-27 14:08:25,844][20347] Updated weights for policy 0, policy_version 1225629 (0.0008) [2023-12-27 14:08:25,899][20381] Updated weights for policy 1, policy_version 1224521 (0.0005) [2023-12-27 14:08:25,959][20381] Updated weights for policy 1, policy_version 1224531 (0.0006) [2023-12-27 14:08:26,421][20347] Updated weights for policy 0, policy_version 1225639 (0.0008) [2023-12-27 14:08:26,468][20347] Updated weights for policy 0, policy_version 1225649 (0.0008) [2023-12-27 14:08:26,499][20381] Updated weights for policy 1, policy_version 1224541 (0.0005) [2023-12-27 14:08:26,513][20347] Updated weights for policy 0, policy_version 1225659 (0.0007) [2023-12-27 14:08:26,542][20381] Updated weights for policy 1, policy_version 1224551 (0.0006) [2023-12-27 14:08:26,596][20381] Updated weights for policy 1, policy_version 1224561 (0.0008) [2023-12-27 14:08:27,213][20347] Updated weights for policy 0, policy_version 1225669 (0.0008) [2023-12-27 14:08:27,235][20381] Updated weights for policy 1, policy_version 1224571 (0.0007) [2023-12-27 14:08:27,262][20347] Updated weights for policy 0, policy_version 1225679 (0.0007) [2023-12-27 14:08:27,287][20381] Updated weights for policy 1, policy_version 1224581 (0.0005) [2023-12-27 14:08:27,317][20347] Updated weights for policy 0, policy_version 1225689 (0.0007) [2023-12-27 14:08:27,343][20381] Updated weights for policy 1, policy_version 1224591 (0.0006) [2023-12-27 14:08:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 627351552. Throughput: 0: 10250.8, 1: 10254.2. Samples: 627340044. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:27,363][19170] Avg episode reward: [(0, '755.940'), (1, '853.400')] [2023-12-27 14:08:27,879][20347] Updated weights for policy 0, policy_version 1225699 (0.0006) [2023-12-27 14:08:27,928][20347] Updated weights for policy 0, policy_version 1225709 (0.0008) [2023-12-27 14:08:27,968][20381] Updated weights for policy 1, policy_version 1224601 (0.0009) [2023-12-27 14:08:27,976][20347] Updated weights for policy 0, policy_version 1225719 (0.0006) [2023-12-27 14:08:28,022][20381] Updated weights for policy 1, policy_version 1224611 (0.0005) [2023-12-27 14:08:28,070][20381] Updated weights for policy 1, policy_version 1224621 (0.0008) [2023-12-27 14:08:28,118][20381] Updated weights for policy 1, policy_version 1224631 (0.0007) [2023-12-27 14:08:28,572][20347] Updated weights for policy 0, policy_version 1225729 (0.0006) [2023-12-27 14:08:28,633][20347] Updated weights for policy 0, policy_version 1225739 (0.0005) [2023-12-27 14:08:28,691][20347] Updated weights for policy 0, policy_version 1225749 (0.0005) [2023-12-27 14:08:28,748][20347] Updated weights for policy 0, policy_version 1225759 (0.0005) [2023-12-27 14:08:28,860][20381] Updated weights for policy 1, policy_version 1224641 (0.0009) [2023-12-27 14:08:28,913][20381] Updated weights for policy 1, policy_version 1224651 (0.0010) [2023-12-27 14:08:28,973][20381] Updated weights for policy 1, policy_version 1224661 (0.0008) [2023-12-27 14:08:29,352][20347] Updated weights for policy 0, policy_version 1225769 (0.0008) [2023-12-27 14:08:29,419][20347] Updated weights for policy 0, policy_version 1225779 (0.0009) [2023-12-27 14:08:29,482][20347] Updated weights for policy 0, policy_version 1225789 (0.0007) [2023-12-27 14:08:29,679][20381] Updated weights for policy 1, policy_version 1224671 (0.0008) [2023-12-27 14:08:29,739][20381] Updated weights for policy 1, policy_version 1224681 (0.0008) [2023-12-27 14:08:29,796][20381] Updated weights for policy 1, policy_version 1224691 (0.0008) [2023-12-27 14:08:30,141][20347] Updated weights for policy 0, policy_version 1225799 (0.0006) [2023-12-27 14:08:30,198][20347] Updated weights for policy 0, policy_version 1225809 (0.0006) [2023-12-27 14:08:30,256][20347] Updated weights for policy 0, policy_version 1225819 (0.0006) [2023-12-27 14:08:30,526][20381] Updated weights for policy 1, policy_version 1224701 (0.0008) [2023-12-27 14:08:30,581][20381] Updated weights for policy 1, policy_version 1224711 (0.0008) [2023-12-27 14:08:30,627][20381] Updated weights for policy 1, policy_version 1224721 (0.0007) [2023-12-27 14:08:30,801][20347] Updated weights for policy 0, policy_version 1225829 (0.0005) [2023-12-27 14:08:30,858][20347] Updated weights for policy 0, policy_version 1225839 (0.0005) [2023-12-27 14:08:30,906][20347] Updated weights for policy 0, policy_version 1225849 (0.0007) [2023-12-27 14:08:31,392][20381] Updated weights for policy 1, policy_version 1224731 (0.0008) [2023-12-27 14:08:31,433][20292] KL-divergence is very high: 117.7207 [2023-12-27 14:08:31,451][20381] Updated weights for policy 1, policy_version 1224741 (0.0008) [2023-12-27 14:08:31,509][20381] Updated weights for policy 1, policy_version 1224751 (0.0007) [2023-12-27 14:08:31,582][20347] Updated weights for policy 0, policy_version 1225859 (0.0007) [2023-12-27 14:08:31,645][20347] Updated weights for policy 0, policy_version 1225869 (0.0008) [2023-12-27 14:08:31,707][20347] Updated weights for policy 0, policy_version 1225879 (0.0008) [2023-12-27 14:08:32,265][20381] Updated weights for policy 1, policy_version 1224761 (0.0007) [2023-12-27 14:08:32,318][20381] Updated weights for policy 1, policy_version 1224771 (0.0009) [2023-12-27 14:08:32,322][20292] KL-divergence is very high: 117.4901 [2023-12-27 14:08:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 627449856. Throughput: 0: 10306.6, 1: 10225.3. Samples: 627466004. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:32,363][19170] Avg episode reward: [(0, '814.330'), (1, '882.850')] [2023-12-27 14:08:32,379][20381] Updated weights for policy 1, policy_version 1224781 (0.0007) [2023-12-27 14:08:32,419][20347] Updated weights for policy 0, policy_version 1225889 (0.0008) [2023-12-27 14:08:32,433][20381] Updated weights for policy 1, policy_version 1224791 (0.0007) [2023-12-27 14:08:32,478][20347] Updated weights for policy 0, policy_version 1225899 (0.0008) [2023-12-27 14:08:32,536][20347] Updated weights for policy 0, policy_version 1225909 (0.0007) [2023-12-27 14:08:32,591][20347] Updated weights for policy 0, policy_version 1225919 (0.0007) [2023-12-27 14:08:33,055][20381] Updated weights for policy 1, policy_version 1224801 (0.0006) [2023-12-27 14:08:33,106][20381] Updated weights for policy 1, policy_version 1224811 (0.0006) [2023-12-27 14:08:33,154][20381] Updated weights for policy 1, policy_version 1224821 (0.0005) [2023-12-27 14:08:33,280][20347] Updated weights for policy 0, policy_version 1225929 (0.0009) [2023-12-27 14:08:33,340][20347] Updated weights for policy 0, policy_version 1225939 (0.0010) [2023-12-27 14:08:33,390][20347] Updated weights for policy 0, policy_version 1225949 (0.0010) [2023-12-27 14:08:33,738][20381] Updated weights for policy 1, policy_version 1224831 (0.0008) [2023-12-27 14:08:33,793][20381] Updated weights for policy 1, policy_version 1224841 (0.0005) [2023-12-27 14:08:33,847][20381] Updated weights for policy 1, policy_version 1224851 (0.0009) [2023-12-27 14:08:33,941][20347] Updated weights for policy 0, policy_version 1225959 (0.0007) [2023-12-27 14:08:33,994][20347] Updated weights for policy 0, policy_version 1225969 (0.0005) [2023-12-27 14:08:34,045][20347] Updated weights for policy 0, policy_version 1225979 (0.0005) [2023-12-27 14:08:34,529][20381] Updated weights for policy 1, policy_version 1224861 (0.0009) [2023-12-27 14:08:34,592][20381] Updated weights for policy 1, policy_version 1224871 (0.0007) [2023-12-27 14:08:34,636][20347] Updated weights for policy 0, policy_version 1225989 (0.0008) [2023-12-27 14:08:34,653][20381] Updated weights for policy 1, policy_version 1224881 (0.0008) [2023-12-27 14:08:34,692][20347] Updated weights for policy 0, policy_version 1225999 (0.0010) [2023-12-27 14:08:34,744][20347] Updated weights for policy 0, policy_version 1226009 (0.0010) [2023-12-27 14:08:35,268][20381] Updated weights for policy 1, policy_version 1224891 (0.0006) [2023-12-27 14:08:35,328][20381] Updated weights for policy 1, policy_version 1224901 (0.0006) [2023-12-27 14:08:35,374][20381] Updated weights for policy 1, policy_version 1224911 (0.0005) [2023-12-27 14:08:35,406][20347] Updated weights for policy 0, policy_version 1226019 (0.0010) [2023-12-27 14:08:35,463][20347] Updated weights for policy 0, policy_version 1226029 (0.0009) [2023-12-27 14:08:35,520][20347] Updated weights for policy 0, policy_version 1226039 (0.0010) [2023-12-27 14:08:36,071][20381] Updated weights for policy 1, policy_version 1224921 (0.0006) [2023-12-27 14:08:36,118][20347] Updated weights for policy 0, policy_version 1226049 (0.0009) [2023-12-27 14:08:36,135][20381] Updated weights for policy 1, policy_version 1224931 (0.0008) [2023-12-27 14:08:36,182][20347] Updated weights for policy 0, policy_version 1226059 (0.0006) [2023-12-27 14:08:36,195][20381] Updated weights for policy 1, policy_version 1224941 (0.0011) [2023-12-27 14:08:36,242][20347] Updated weights for policy 0, policy_version 1226069 (0.0008) [2023-12-27 14:08:36,253][20381] Updated weights for policy 1, policy_version 1224951 (0.0011) [2023-12-27 14:08:36,302][20347] Updated weights for policy 0, policy_version 1226079 (0.0011) [2023-12-27 14:08:36,960][20381] Updated weights for policy 1, policy_version 1224961 (0.0007) [2023-12-27 14:08:36,987][20347] Updated weights for policy 0, policy_version 1226089 (0.0006) [2023-12-27 14:08:37,013][20381] Updated weights for policy 1, policy_version 1224971 (0.0011) [2023-12-27 14:08:37,036][20347] Updated weights for policy 0, policy_version 1226099 (0.0005) [2023-12-27 14:08:37,067][20381] Updated weights for policy 1, policy_version 1224981 (0.0011) [2023-12-27 14:08:37,095][20347] Updated weights for policy 0, policy_version 1226109 (0.0005) [2023-12-27 14:08:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 627564544. Throughput: 0: 10360.7, 1: 10273.9. Samples: 627531780. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:37,364][19170] Avg episode reward: [(0, '824.200'), (1, '912.180')] [2023-12-27 14:08:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001224984_313622528.pth... [2023-12-27 14:08:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001226112_313942016.pth... [2023-12-27 14:08:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001223800_313319424.pth [2023-12-27 14:08:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001224896_313630720.pth [2023-12-27 14:08:37,714][20381] Updated weights for policy 1, policy_version 1224991 (0.0007) [2023-12-27 14:08:37,740][20347] Updated weights for policy 0, policy_version 1226119 (0.0008) [2023-12-27 14:08:37,763][20381] Updated weights for policy 1, policy_version 1225001 (0.0005) [2023-12-27 14:08:37,794][20347] Updated weights for policy 0, policy_version 1226129 (0.0006) [2023-12-27 14:08:37,828][20381] Updated weights for policy 1, policy_version 1225011 (0.0009) [2023-12-27 14:08:37,851][20347] Updated weights for policy 0, policy_version 1226139 (0.0005) [2023-12-27 14:08:38,432][20347] Updated weights for policy 0, policy_version 1226149 (0.0007) [2023-12-27 14:08:38,482][20381] Updated weights for policy 1, policy_version 1225021 (0.0008) [2023-12-27 14:08:38,494][20347] Updated weights for policy 0, policy_version 1226159 (0.0008) [2023-12-27 14:08:38,544][20381] Updated weights for policy 1, policy_version 1225031 (0.0009) [2023-12-27 14:08:38,551][20347] Updated weights for policy 0, policy_version 1226169 (0.0005) [2023-12-27 14:08:38,604][20381] Updated weights for policy 1, policy_version 1225041 (0.0011) [2023-12-27 14:08:39,206][20347] Updated weights for policy 0, policy_version 1226179 (0.0008) [2023-12-27 14:08:39,275][20347] Updated weights for policy 0, policy_version 1226189 (0.0008) [2023-12-27 14:08:39,278][20381] Updated weights for policy 1, policy_version 1225051 (0.0009) [2023-12-27 14:08:39,330][20347] Updated weights for policy 0, policy_version 1226199 (0.0010) [2023-12-27 14:08:39,343][20381] Updated weights for policy 1, policy_version 1225061 (0.0007) [2023-12-27 14:08:39,415][20381] Updated weights for policy 1, policy_version 1225071 (0.0007) [2023-12-27 14:08:39,988][20347] Updated weights for policy 0, policy_version 1226209 (0.0008) [2023-12-27 14:08:40,049][20347] Updated weights for policy 0, policy_version 1226219 (0.0008) [2023-12-27 14:08:40,099][20347] Updated weights for policy 0, policy_version 1226229 (0.0008) [2023-12-27 14:08:40,124][20381] Updated weights for policy 1, policy_version 1225081 (0.0008) [2023-12-27 14:08:40,160][20347] Updated weights for policy 0, policy_version 1226239 (0.0009) [2023-12-27 14:08:40,188][20381] Updated weights for policy 1, policy_version 1225091 (0.0008) [2023-12-27 14:08:40,245][20381] Updated weights for policy 1, policy_version 1225101 (0.0007) [2023-12-27 14:08:40,305][20381] Updated weights for policy 1, policy_version 1225111 (0.0008) [2023-12-27 14:08:40,818][20347] Updated weights for policy 0, policy_version 1226249 (0.0006) [2023-12-27 14:08:40,873][20347] Updated weights for policy 0, policy_version 1226259 (0.0007) [2023-12-27 14:08:40,927][20347] Updated weights for policy 0, policy_version 1226269 (0.0008) [2023-12-27 14:08:41,050][20381] Updated weights for policy 1, policy_version 1225121 (0.0008) [2023-12-27 14:08:41,111][20381] Updated weights for policy 1, policy_version 1225131 (0.0008) [2023-12-27 14:08:41,173][20381] Updated weights for policy 1, policy_version 1225141 (0.0007) [2023-12-27 14:08:41,653][20347] Updated weights for policy 0, policy_version 1226279 (0.0008) [2023-12-27 14:08:41,716][20347] Updated weights for policy 0, policy_version 1226289 (0.0007) [2023-12-27 14:08:41,785][20347] Updated weights for policy 0, policy_version 1226299 (0.0008) [2023-12-27 14:08:41,933][20381] Updated weights for policy 1, policy_version 1225151 (0.0008) [2023-12-27 14:08:41,947][20292] KL-divergence is very high: 276.2312 [2023-12-27 14:08:41,996][20381] Updated weights for policy 1, policy_version 1225161 (0.0008) [2023-12-27 14:08:41,997][20292] KL-divergence is very high: 292.1806 [2023-12-27 14:08:42,046][20292] KL-divergence is very high: 133.3226 [2023-12-27 14:08:42,056][20381] Updated weights for policy 1, policy_version 1225171 (0.0007) [2023-12-27 14:08:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 627662848. Throughput: 0: 10444.5, 1: 10192.5. Samples: 627654468. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:42,363][19170] Avg episode reward: [(0, '853.490'), (1, '951.250')] [2023-12-27 14:08:42,454][20347] Updated weights for policy 0, policy_version 1226309 (0.0006) [2023-12-27 14:08:42,505][20347] Updated weights for policy 0, policy_version 1226319 (0.0007) [2023-12-27 14:08:42,559][20347] Updated weights for policy 0, policy_version 1226329 (0.0008) [2023-12-27 14:08:42,762][20381] Updated weights for policy 1, policy_version 1225181 (0.0007) [2023-12-27 14:08:42,815][20381] Updated weights for policy 1, policy_version 1225191 (0.0007) [2023-12-27 14:08:42,864][20381] Updated weights for policy 1, policy_version 1225201 (0.0009) [2023-12-27 14:08:43,207][20347] Updated weights for policy 0, policy_version 1226339 (0.0007) [2023-12-27 14:08:43,255][20347] Updated weights for policy 0, policy_version 1226349 (0.0008) [2023-12-27 14:08:43,304][20347] Updated weights for policy 0, policy_version 1226359 (0.0009) [2023-12-27 14:08:43,466][20381] Updated weights for policy 1, policy_version 1225211 (0.0006) [2023-12-27 14:08:43,514][20381] Updated weights for policy 1, policy_version 1225221 (0.0005) [2023-12-27 14:08:43,564][20381] Updated weights for policy 1, policy_version 1225231 (0.0005) [2023-12-27 14:08:44,076][20347] Updated weights for policy 0, policy_version 1226369 (0.0009) [2023-12-27 14:08:44,129][20381] Updated weights for policy 1, policy_version 1225241 (0.0005) [2023-12-27 14:08:44,134][20347] Updated weights for policy 0, policy_version 1226379 (0.0006) [2023-12-27 14:08:44,181][20347] Updated weights for policy 0, policy_version 1226389 (0.0005) [2023-12-27 14:08:44,184][20381] Updated weights for policy 1, policy_version 1225251 (0.0007) [2023-12-27 14:08:44,196][20292] KL-divergence is very high: 166.8701 [2023-12-27 14:08:44,213][20292] KL-divergence is very high: 332.2947 [2023-12-27 14:08:44,225][20292] KL-divergence is very high: 332.0626 [2023-12-27 14:08:44,236][20347] Updated weights for policy 0, policy_version 1226399 (0.0005) [2023-12-27 14:08:44,240][20381] Updated weights for policy 1, policy_version 1225261 (0.0007) [2023-12-27 14:08:44,241][20292] KL-divergence is very high: 206.6395 [2023-12-27 14:08:44,299][20381] Updated weights for policy 1, policy_version 1225271 (0.0009) [2023-12-27 14:08:44,795][20347] Updated weights for policy 0, policy_version 1226409 (0.0009) [2023-12-27 14:08:44,864][20347] Updated weights for policy 0, policy_version 1226419 (0.0006) [2023-12-27 14:08:44,926][20347] Updated weights for policy 0, policy_version 1226429 (0.0010) [2023-12-27 14:08:45,062][20381] Updated weights for policy 1, policy_version 1225281 (0.0008) [2023-12-27 14:08:45,119][20381] Updated weights for policy 1, policy_version 1225291 (0.0008) [2023-12-27 14:08:45,178][20381] Updated weights for policy 1, policy_version 1225301 (0.0007) [2023-12-27 14:08:45,607][20347] Updated weights for policy 0, policy_version 1226439 (0.0010) [2023-12-27 14:08:45,654][20291] KL-divergence is very high: 215.2144 [2023-12-27 14:08:45,660][20347] Updated weights for policy 0, policy_version 1226449 (0.0006) [2023-12-27 14:08:45,696][20291] KL-divergence is very high: 238.8667 [2023-12-27 14:08:45,712][20347] Updated weights for policy 0, policy_version 1226459 (0.0007) [2023-12-27 14:08:45,893][20381] Updated weights for policy 1, policy_version 1225311 (0.0008) [2023-12-27 14:08:45,950][20381] Updated weights for policy 1, policy_version 1225321 (0.0008) [2023-12-27 14:08:46,006][20381] Updated weights for policy 1, policy_version 1225331 (0.0009) [2023-12-27 14:08:46,310][20347] Updated weights for policy 0, policy_version 1226469 (0.0007) [2023-12-27 14:08:46,370][20347] Updated weights for policy 0, policy_version 1226479 (0.0008) [2023-12-27 14:08:46,428][20347] Updated weights for policy 0, policy_version 1226489 (0.0007) [2023-12-27 14:08:46,728][20381] Updated weights for policy 1, policy_version 1225341 (0.0009) [2023-12-27 14:08:46,777][20381] Updated weights for policy 1, policy_version 1225351 (0.0005) [2023-12-27 14:08:46,836][20381] Updated weights for policy 1, policy_version 1225361 (0.0005) [2023-12-27 14:08:47,023][20347] Updated weights for policy 0, policy_version 1226499 (0.0007) [2023-12-27 14:08:47,077][20347] Updated weights for policy 0, policy_version 1226509 (0.0008) [2023-12-27 14:08:47,132][20347] Updated weights for policy 0, policy_version 1226519 (0.0007) [2023-12-27 14:08:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 627769344. Throughput: 0: 10435.6, 1: 10153.8. Samples: 627779296. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:47,363][19170] Avg episode reward: [(0, '804.530'), (1, '921.930')] [2023-12-27 14:08:47,487][20381] Updated weights for policy 1, policy_version 1225371 (0.0005) [2023-12-27 14:08:47,545][20381] Updated weights for policy 1, policy_version 1225381 (0.0005) [2023-12-27 14:08:47,605][20381] Updated weights for policy 1, policy_version 1225391 (0.0006) [2023-12-27 14:08:47,772][20347] Updated weights for policy 0, policy_version 1226529 (0.0008) [2023-12-27 14:08:47,827][20347] Updated weights for policy 0, policy_version 1226539 (0.0008) [2023-12-27 14:08:47,881][20347] Updated weights for policy 0, policy_version 1226549 (0.0007) [2023-12-27 14:08:47,931][20347] Updated weights for policy 0, policy_version 1226559 (0.0006) [2023-12-27 14:08:48,177][20381] Updated weights for policy 1, policy_version 1225401 (0.0008) [2023-12-27 14:08:48,237][20381] Updated weights for policy 1, policy_version 1225411 (0.0006) [2023-12-27 14:08:48,301][20381] Updated weights for policy 1, policy_version 1225421 (0.0007) [2023-12-27 14:08:48,360][20381] Updated weights for policy 1, policy_version 1225431 (0.0007) [2023-12-27 14:08:48,630][20347] Updated weights for policy 0, policy_version 1226569 (0.0008) [2023-12-27 14:08:48,684][20347] Updated weights for policy 0, policy_version 1226579 (0.0008) [2023-12-27 14:08:48,737][20347] Updated weights for policy 0, policy_version 1226589 (0.0008) [2023-12-27 14:08:49,002][20381] Updated weights for policy 1, policy_version 1225441 (0.0008) [2023-12-27 14:08:49,058][20381] Updated weights for policy 1, policy_version 1225451 (0.0007) [2023-12-27 14:08:49,113][20381] Updated weights for policy 1, policy_version 1225461 (0.0008) [2023-12-27 14:08:49,452][20347] Updated weights for policy 0, policy_version 1226599 (0.0008) [2023-12-27 14:08:49,516][20347] Updated weights for policy 0, policy_version 1226609 (0.0007) [2023-12-27 14:08:49,579][20347] Updated weights for policy 0, policy_version 1226619 (0.0008) [2023-12-27 14:08:49,803][20381] Updated weights for policy 1, policy_version 1225471 (0.0008) [2023-12-27 14:08:49,871][20381] Updated weights for policy 1, policy_version 1225481 (0.0009) [2023-12-27 14:08:49,927][20381] Updated weights for policy 1, policy_version 1225491 (0.0008) [2023-12-27 14:08:50,271][20347] Updated weights for policy 0, policy_version 1226629 (0.0007) [2023-12-27 14:08:50,332][20347] Updated weights for policy 0, policy_version 1226639 (0.0008) [2023-12-27 14:08:50,384][20347] Updated weights for policy 0, policy_version 1226649 (0.0007) [2023-12-27 14:08:50,601][20381] Updated weights for policy 1, policy_version 1225501 (0.0009) [2023-12-27 14:08:50,661][20381] Updated weights for policy 1, policy_version 1225511 (0.0007) [2023-12-27 14:08:50,717][20381] Updated weights for policy 1, policy_version 1225521 (0.0008) [2023-12-27 14:08:51,083][20347] Updated weights for policy 0, policy_version 1226659 (0.0008) [2023-12-27 14:08:51,148][20347] Updated weights for policy 0, policy_version 1226669 (0.0007) [2023-12-27 14:08:51,204][20347] Updated weights for policy 0, policy_version 1226679 (0.0007) [2023-12-27 14:08:51,406][20381] Updated weights for policy 1, policy_version 1225531 (0.0008) [2023-12-27 14:08:51,465][20381] Updated weights for policy 1, policy_version 1225541 (0.0007) [2023-12-27 14:08:51,521][20381] Updated weights for policy 1, policy_version 1225551 (0.0007) [2023-12-27 14:08:51,922][20347] Updated weights for policy 0, policy_version 1226689 (0.0007) [2023-12-27 14:08:51,981][20347] Updated weights for policy 0, policy_version 1226699 (0.0008) [2023-12-27 14:08:52,042][20347] Updated weights for policy 0, policy_version 1226709 (0.0008) [2023-12-27 14:08:52,100][20347] Updated weights for policy 0, policy_version 1226719 (0.0007) [2023-12-27 14:08:52,252][20381] Updated weights for policy 1, policy_version 1225561 (0.0008) [2023-12-27 14:08:52,317][20381] Updated weights for policy 1, policy_version 1225571 (0.0008) [2023-12-27 14:08:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 627867648. Throughput: 0: 10410.0, 1: 10165.8. Samples: 627840684. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:52,363][19170] Avg episode reward: [(0, '882.790'), (1, '921.860')] [2023-12-27 14:08:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001226720_314097664.pth... [2023-12-27 14:08:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001225472_313778176.pth [2023-12-27 14:08:52,385][20381] Updated weights for policy 1, policy_version 1225581 (0.0007) [2023-12-27 14:08:52,441][20381] Updated weights for policy 1, policy_version 1225591 (0.0007) [2023-12-27 14:08:52,446][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001225592_313778176.pth... [2023-12-27 14:08:52,449][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001224376_313466880.pth [2023-12-27 14:08:52,819][20347] Updated weights for policy 0, policy_version 1226729 (0.0008) [2023-12-27 14:08:52,874][20347] Updated weights for policy 0, policy_version 1226739 (0.0007) [2023-12-27 14:08:52,919][20347] Updated weights for policy 0, policy_version 1226749 (0.0007) [2023-12-27 14:08:53,102][20381] Updated weights for policy 1, policy_version 1225601 (0.0007) [2023-12-27 14:08:53,150][20381] Updated weights for policy 1, policy_version 1225611 (0.0008) [2023-12-27 14:08:53,204][20381] Updated weights for policy 1, policy_version 1225621 (0.0008) [2023-12-27 14:08:53,588][20347] Updated weights for policy 0, policy_version 1226759 (0.0006) [2023-12-27 14:08:53,645][20347] Updated weights for policy 0, policy_version 1226769 (0.0005) [2023-12-27 14:08:53,700][20347] Updated weights for policy 0, policy_version 1226779 (0.0005) [2023-12-27 14:08:53,944][20381] Updated weights for policy 1, policy_version 1225631 (0.0008) [2023-12-27 14:08:54,006][20381] Updated weights for policy 1, policy_version 1225641 (0.0008) [2023-12-27 14:08:54,061][20381] Updated weights for policy 1, policy_version 1225651 (0.0008) [2023-12-27 14:08:54,246][20347] Updated weights for policy 0, policy_version 1226789 (0.0006) [2023-12-27 14:08:54,300][20347] Updated weights for policy 0, policy_version 1226799 (0.0007) [2023-12-27 14:08:54,357][20347] Updated weights for policy 0, policy_version 1226809 (0.0007) [2023-12-27 14:08:54,684][20381] Updated weights for policy 1, policy_version 1225661 (0.0006) [2023-12-27 14:08:54,742][20381] Updated weights for policy 1, policy_version 1225671 (0.0005) [2023-12-27 14:08:54,803][20381] Updated weights for policy 1, policy_version 1225681 (0.0006) [2023-12-27 14:08:55,051][20347] Updated weights for policy 0, policy_version 1226819 (0.0009) [2023-12-27 14:08:55,106][20347] Updated weights for policy 0, policy_version 1226829 (0.0010) [2023-12-27 14:08:55,173][20347] Updated weights for policy 0, policy_version 1226839 (0.0011) [2023-12-27 14:08:55,504][20381] Updated weights for policy 1, policy_version 1225691 (0.0009) [2023-12-27 14:08:55,565][20381] Updated weights for policy 1, policy_version 1225701 (0.0008) [2023-12-27 14:08:55,620][20381] Updated weights for policy 1, policy_version 1225711 (0.0008) [2023-12-27 14:08:55,868][20347] Updated weights for policy 0, policy_version 1226849 (0.0010) [2023-12-27 14:08:55,929][20347] Updated weights for policy 0, policy_version 1226859 (0.0006) [2023-12-27 14:08:55,990][20347] Updated weights for policy 0, policy_version 1226869 (0.0010) [2023-12-27 14:08:56,045][20347] Updated weights for policy 0, policy_version 1226879 (0.0010) [2023-12-27 14:08:56,353][20381] Updated weights for policy 1, policy_version 1225721 (0.0007) [2023-12-27 14:08:56,404][20381] Updated weights for policy 1, policy_version 1225731 (0.0007) [2023-12-27 14:08:56,449][20381] Updated weights for policy 1, policy_version 1225741 (0.0007) [2023-12-27 14:08:56,493][20381] Updated weights for policy 1, policy_version 1225751 (0.0007) [2023-12-27 14:08:56,698][20347] Updated weights for policy 0, policy_version 1226889 (0.0010) [2023-12-27 14:08:56,755][20347] Updated weights for policy 0, policy_version 1226899 (0.0010) [2023-12-27 14:08:56,802][20347] Updated weights for policy 0, policy_version 1226909 (0.0010) [2023-12-27 14:08:57,171][20381] Updated weights for policy 1, policy_version 1225761 (0.0009) [2023-12-27 14:08:57,217][20381] Updated weights for policy 1, policy_version 1225771 (0.0007) [2023-12-27 14:08:57,275][20381] Updated weights for policy 1, policy_version 1225781 (0.0008) [2023-12-27 14:08:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 627974144. Throughput: 0: 10440.7, 1: 10186.6. Samples: 627962528. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:08:57,363][19170] Avg episode reward: [(0, '843.340'), (1, '951.140')] [2023-12-27 14:08:57,470][20347] Updated weights for policy 0, policy_version 1226919 (0.0010) [2023-12-27 14:08:57,521][20347] Updated weights for policy 0, policy_version 1226929 (0.0010) [2023-12-27 14:08:57,566][20347] Updated weights for policy 0, policy_version 1226939 (0.0010) [2023-12-27 14:08:57,981][20381] Updated weights for policy 1, policy_version 1225791 (0.0007) [2023-12-27 14:08:58,030][20381] Updated weights for policy 1, policy_version 1225801 (0.0007) [2023-12-27 14:08:58,078][20381] Updated weights for policy 1, policy_version 1225811 (0.0007) [2023-12-27 14:08:58,251][20347] Updated weights for policy 0, policy_version 1226949 (0.0010) [2023-12-27 14:08:58,316][20347] Updated weights for policy 0, policy_version 1226959 (0.0010) [2023-12-27 14:08:58,381][20347] Updated weights for policy 0, policy_version 1226969 (0.0009) [2023-12-27 14:08:58,834][20381] Updated weights for policy 1, policy_version 1225821 (0.0008) [2023-12-27 14:08:58,897][20381] Updated weights for policy 1, policy_version 1225831 (0.0009) [2023-12-27 14:08:58,950][20381] Updated weights for policy 1, policy_version 1225841 (0.0009) [2023-12-27 14:08:59,201][20347] Updated weights for policy 0, policy_version 1226979 (0.0010) [2023-12-27 14:08:59,277][20347] Updated weights for policy 0, policy_version 1226989 (0.0009) [2023-12-27 14:08:59,335][20347] Updated weights for policy 0, policy_version 1226999 (0.0009) [2023-12-27 14:08:59,749][20381] Updated weights for policy 1, policy_version 1225851 (0.0009) [2023-12-27 14:08:59,797][20381] Updated weights for policy 1, policy_version 1225861 (0.0008) [2023-12-27 14:08:59,860][20381] Updated weights for policy 1, policy_version 1225871 (0.0008) [2023-12-27 14:09:00,051][20347] Updated weights for policy 0, policy_version 1227009 (0.0007) [2023-12-27 14:09:00,111][20347] Updated weights for policy 0, policy_version 1227019 (0.0006) [2023-12-27 14:09:00,172][20347] Updated weights for policy 0, policy_version 1227029 (0.0006) [2023-12-27 14:09:00,231][20347] Updated weights for policy 0, policy_version 1227039 (0.0006) [2023-12-27 14:09:00,567][20381] Updated weights for policy 1, policy_version 1225881 (0.0008) [2023-12-27 14:09:00,619][20381] Updated weights for policy 1, policy_version 1225891 (0.0005) [2023-12-27 14:09:00,666][20381] Updated weights for policy 1, policy_version 1225901 (0.0005) [2023-12-27 14:09:00,724][20381] Updated weights for policy 1, policy_version 1225911 (0.0008) [2023-12-27 14:09:00,894][20347] Updated weights for policy 0, policy_version 1227049 (0.0010) [2023-12-27 14:09:00,943][20347] Updated weights for policy 0, policy_version 1227059 (0.0005) [2023-12-27 14:09:00,994][20347] Updated weights for policy 0, policy_version 1227069 (0.0005) [2023-12-27 14:09:01,353][20381] Updated weights for policy 1, policy_version 1225921 (0.0006) [2023-12-27 14:09:01,419][20381] Updated weights for policy 1, policy_version 1225931 (0.0007) [2023-12-27 14:09:01,472][20381] Updated weights for policy 1, policy_version 1225941 (0.0008) [2023-12-27 14:09:01,674][20347] Updated weights for policy 0, policy_version 1227079 (0.0007) [2023-12-27 14:09:01,739][20347] Updated weights for policy 0, policy_version 1227089 (0.0007) [2023-12-27 14:09:01,796][20347] Updated weights for policy 0, policy_version 1227099 (0.0008) [2023-12-27 14:09:02,143][20381] Updated weights for policy 1, policy_version 1225951 (0.0010) [2023-12-27 14:09:02,198][20381] Updated weights for policy 1, policy_version 1225961 (0.0010) [2023-12-27 14:09:02,249][20381] Updated weights for policy 1, policy_version 1225971 (0.0010) [2023-12-27 14:09:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 628072448. Throughput: 0: 10394.8, 1: 10149.4. Samples: 628080848. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:02,363][19170] Avg episode reward: [(0, '833.650'), (1, '941.420')] [2023-12-27 14:09:02,516][20347] Updated weights for policy 0, policy_version 1227109 (0.0008) [2023-12-27 14:09:02,575][20347] Updated weights for policy 0, policy_version 1227119 (0.0008) [2023-12-27 14:09:02,631][20347] Updated weights for policy 0, policy_version 1227129 (0.0007) [2023-12-27 14:09:02,942][20381] Updated weights for policy 1, policy_version 1225981 (0.0007) [2023-12-27 14:09:03,006][20381] Updated weights for policy 1, policy_version 1225991 (0.0007) [2023-12-27 14:09:03,070][20381] Updated weights for policy 1, policy_version 1226001 (0.0007) [2023-12-27 14:09:03,260][20347] Updated weights for policy 0, policy_version 1227139 (0.0007) [2023-12-27 14:09:03,319][20347] Updated weights for policy 0, policy_version 1227149 (0.0008) [2023-12-27 14:09:03,376][20347] Updated weights for policy 0, policy_version 1227159 (0.0006) [2023-12-27 14:09:03,664][20381] Updated weights for policy 1, policy_version 1226011 (0.0007) [2023-12-27 14:09:03,712][20381] Updated weights for policy 1, policy_version 1226021 (0.0005) [2023-12-27 14:09:03,765][20381] Updated weights for policy 1, policy_version 1226031 (0.0005) [2023-12-27 14:09:04,006][20347] Updated weights for policy 0, policy_version 1227169 (0.0006) [2023-12-27 14:09:04,067][20347] Updated weights for policy 0, policy_version 1227179 (0.0007) [2023-12-27 14:09:04,133][20347] Updated weights for policy 0, policy_version 1227189 (0.0007) [2023-12-27 14:09:04,190][20347] Updated weights for policy 0, policy_version 1227199 (0.0006) [2023-12-27 14:09:04,402][20381] Updated weights for policy 1, policy_version 1226041 (0.0005) [2023-12-27 14:09:04,456][20381] Updated weights for policy 1, policy_version 1226051 (0.0007) [2023-12-27 14:09:04,512][20381] Updated weights for policy 1, policy_version 1226061 (0.0007) [2023-12-27 14:09:04,574][20381] Updated weights for policy 1, policy_version 1226071 (0.0006) [2023-12-27 14:09:04,861][20347] Updated weights for policy 0, policy_version 1227209 (0.0007) [2023-12-27 14:09:04,924][20347] Updated weights for policy 0, policy_version 1227219 (0.0007) [2023-12-27 14:09:04,974][20347] Updated weights for policy 0, policy_version 1227229 (0.0007) [2023-12-27 14:09:05,243][20381] Updated weights for policy 1, policy_version 1226081 (0.0010) [2023-12-27 14:09:05,298][20381] Updated weights for policy 1, policy_version 1226091 (0.0010) [2023-12-27 14:09:05,352][20381] Updated weights for policy 1, policy_version 1226101 (0.0009) [2023-12-27 14:09:05,604][20347] Updated weights for policy 0, policy_version 1227239 (0.0009) [2023-12-27 14:09:05,670][20347] Updated weights for policy 0, policy_version 1227249 (0.0010) [2023-12-27 14:09:05,721][20347] Updated weights for policy 0, policy_version 1227259 (0.0010) [2023-12-27 14:09:05,887][20381] Updated weights for policy 1, policy_version 1226111 (0.0005) [2023-12-27 14:09:05,891][20292] KL-divergence is very high: 241.0701 [2023-12-27 14:09:05,928][20292] KL-divergence is very high: 187.5217 [2023-12-27 14:09:05,933][20381] Updated weights for policy 1, policy_version 1226121 (0.0005) [2023-12-27 14:09:05,988][20381] Updated weights for policy 1, policy_version 1226131 (0.0005) [2023-12-27 14:09:06,347][20347] Updated weights for policy 0, policy_version 1227269 (0.0010) [2023-12-27 14:09:06,408][20347] Updated weights for policy 0, policy_version 1227279 (0.0010) [2023-12-27 14:09:06,468][20347] Updated weights for policy 0, policy_version 1227289 (0.0007) [2023-12-27 14:09:06,660][20381] Updated weights for policy 1, policy_version 1226141 (0.0008) [2023-12-27 14:09:06,729][20381] Updated weights for policy 1, policy_version 1226151 (0.0011) [2023-12-27 14:09:06,793][20381] Updated weights for policy 1, policy_version 1226161 (0.0008) [2023-12-27 14:09:07,164][20347] Updated weights for policy 0, policy_version 1227299 (0.0008) [2023-12-27 14:09:07,220][20347] Updated weights for policy 0, policy_version 1227309 (0.0007) [2023-12-27 14:09:07,269][20347] Updated weights for policy 0, policy_version 1227319 (0.0008) [2023-12-27 14:09:07,359][20381] Updated weights for policy 1, policy_version 1226171 (0.0011) [2023-12-27 14:09:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 628178944. Throughput: 0: 10390.4, 1: 10216.4. Samples: 628144056. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:07,363][19170] Avg episode reward: [(0, '814.730'), (1, '912.140')] [2023-12-27 14:09:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001227328_314253312.pth... [2023-12-27 14:09:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001226112_313942016.pth [2023-12-27 14:09:07,407][20381] Updated weights for policy 1, policy_version 1226181 (0.0010) [2023-12-27 14:09:07,413][20292] KL-divergence is very high: 171.1776 [2023-12-27 14:09:07,461][20381] Updated weights for policy 1, policy_version 1226191 (0.0005) [2023-12-27 14:09:07,511][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001226200_313933824.pth... [2023-12-27 14:09:07,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001224984_313622528.pth [2023-12-27 14:09:07,938][20347] Updated weights for policy 0, policy_version 1227329 (0.0008) [2023-12-27 14:09:07,992][20347] Updated weights for policy 0, policy_version 1227339 (0.0006) [2023-12-27 14:09:08,045][20347] Updated weights for policy 0, policy_version 1227349 (0.0007) [2023-12-27 14:09:08,100][20347] Updated weights for policy 0, policy_version 1227359 (0.0007) [2023-12-27 14:09:08,121][20381] Updated weights for policy 1, policy_version 1226201 (0.0005) [2023-12-27 14:09:08,181][20381] Updated weights for policy 1, policy_version 1226211 (0.0006) [2023-12-27 14:09:08,238][20381] Updated weights for policy 1, policy_version 1226221 (0.0008) [2023-12-27 14:09:08,292][20381] Updated weights for policy 1, policy_version 1226231 (0.0008) [2023-12-27 14:09:08,768][20347] Updated weights for policy 0, policy_version 1227369 (0.0010) [2023-12-27 14:09:08,823][20347] Updated weights for policy 0, policy_version 1227379 (0.0010) [2023-12-27 14:09:08,881][20347] Updated weights for policy 0, policy_version 1227389 (0.0010) [2023-12-27 14:09:08,997][20381] Updated weights for policy 1, policy_version 1226241 (0.0007) [2023-12-27 14:09:09,045][20381] Updated weights for policy 1, policy_version 1226251 (0.0007) [2023-12-27 14:09:09,093][20381] Updated weights for policy 1, policy_version 1226261 (0.0008) [2023-12-27 14:09:09,619][20347] Updated weights for policy 0, policy_version 1227399 (0.0009) [2023-12-27 14:09:09,671][20347] Updated weights for policy 0, policy_version 1227409 (0.0007) [2023-12-27 14:09:09,734][20347] Updated weights for policy 0, policy_version 1227419 (0.0008) [2023-12-27 14:09:09,803][20381] Updated weights for policy 1, policy_version 1226271 (0.0008) [2023-12-27 14:09:09,870][20381] Updated weights for policy 1, policy_version 1226281 (0.0007) [2023-12-27 14:09:09,932][20381] Updated weights for policy 1, policy_version 1226291 (0.0008) [2023-12-27 14:09:10,510][20347] Updated weights for policy 0, policy_version 1227429 (0.0008) [2023-12-27 14:09:10,568][20347] Updated weights for policy 0, policy_version 1227439 (0.0007) [2023-12-27 14:09:10,591][20381] Updated weights for policy 1, policy_version 1226301 (0.0009) [2023-12-27 14:09:10,627][20347] Updated weights for policy 0, policy_version 1227449 (0.0008) [2023-12-27 14:09:10,658][20381] Updated weights for policy 1, policy_version 1226311 (0.0006) [2023-12-27 14:09:10,722][20381] Updated weights for policy 1, policy_version 1226321 (0.0007) [2023-12-27 14:09:11,309][20347] Updated weights for policy 0, policy_version 1227459 (0.0007) [2023-12-27 14:09:11,376][20347] Updated weights for policy 0, policy_version 1227469 (0.0008) [2023-12-27 14:09:11,390][20381] Updated weights for policy 1, policy_version 1226331 (0.0007) [2023-12-27 14:09:11,440][20347] Updated weights for policy 0, policy_version 1227479 (0.0008) [2023-12-27 14:09:11,455][20381] Updated weights for policy 1, policy_version 1226341 (0.0007) [2023-12-27 14:09:11,522][20381] Updated weights for policy 1, policy_version 1226351 (0.0006) [2023-12-27 14:09:12,165][20347] Updated weights for policy 0, policy_version 1227489 (0.0007) [2023-12-27 14:09:12,222][20347] Updated weights for policy 0, policy_version 1227499 (0.0008) [2023-12-27 14:09:12,249][20381] Updated weights for policy 1, policy_version 1226361 (0.0006) [2023-12-27 14:09:12,285][20347] Updated weights for policy 0, policy_version 1227509 (0.0007) [2023-12-27 14:09:12,314][20381] Updated weights for policy 1, policy_version 1226371 (0.0006) [2023-12-27 14:09:12,341][20347] Updated weights for policy 0, policy_version 1227519 (0.0008) [2023-12-27 14:09:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 628277248. Throughput: 0: 10390.8, 1: 10238.8. Samples: 628268372. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:12,365][19170] Avg episode reward: [(0, '843.930'), (1, '912.360')] [2023-12-27 14:09:12,382][20381] Updated weights for policy 1, policy_version 1226381 (0.0007) [2023-12-27 14:09:12,443][20381] Updated weights for policy 1, policy_version 1226391 (0.0008) [2023-12-27 14:09:13,049][20347] Updated weights for policy 0, policy_version 1227529 (0.0006) [2023-12-27 14:09:13,092][20381] Updated weights for policy 1, policy_version 1226401 (0.0006) [2023-12-27 14:09:13,107][20347] Updated weights for policy 0, policy_version 1227539 (0.0007) [2023-12-27 14:09:13,151][20381] Updated weights for policy 1, policy_version 1226411 (0.0005) [2023-12-27 14:09:13,164][20347] Updated weights for policy 0, policy_version 1227549 (0.0005) [2023-12-27 14:09:13,206][20381] Updated weights for policy 1, policy_version 1226421 (0.0005) [2023-12-27 14:09:13,761][20347] Updated weights for policy 0, policy_version 1227559 (0.0006) [2023-12-27 14:09:13,814][20347] Updated weights for policy 0, policy_version 1227569 (0.0006) [2023-12-27 14:09:13,827][20381] Updated weights for policy 1, policy_version 1226431 (0.0006) [2023-12-27 14:09:13,859][20347] Updated weights for policy 0, policy_version 1227579 (0.0005) [2023-12-27 14:09:13,886][20381] Updated weights for policy 1, policy_version 1226441 (0.0007) [2023-12-27 14:09:13,944][20381] Updated weights for policy 1, policy_version 1226451 (0.0006) [2023-12-27 14:09:14,538][20347] Updated weights for policy 0, policy_version 1227589 (0.0006) [2023-12-27 14:09:14,597][20347] Updated weights for policy 0, policy_version 1227599 (0.0007) [2023-12-27 14:09:14,636][20381] Updated weights for policy 1, policy_version 1226461 (0.0006) [2023-12-27 14:09:14,653][20347] Updated weights for policy 0, policy_version 1227609 (0.0006) [2023-12-27 14:09:14,713][20381] Updated weights for policy 1, policy_version 1226471 (0.0007) [2023-12-27 14:09:14,766][20381] Updated weights for policy 1, policy_version 1226481 (0.0008) [2023-12-27 14:09:15,344][20347] Updated weights for policy 0, policy_version 1227619 (0.0006) [2023-12-27 14:09:15,399][20347] Updated weights for policy 0, policy_version 1227629 (0.0007) [2023-12-27 14:09:15,430][20381] Updated weights for policy 1, policy_version 1226491 (0.0007) [2023-12-27 14:09:15,454][20347] Updated weights for policy 0, policy_version 1227639 (0.0007) [2023-12-27 14:09:15,486][20381] Updated weights for policy 1, policy_version 1226501 (0.0006) [2023-12-27 14:09:15,493][20292] KL-divergence is very high: 234.6236 [2023-12-27 14:09:15,536][20292] KL-divergence is very high: 271.1904 [2023-12-27 14:09:15,541][20381] Updated weights for policy 1, policy_version 1226511 (0.0007) [2023-12-27 14:09:15,582][20292] KL-divergence is very high: 129.0984 [2023-12-27 14:09:16,171][20381] Updated weights for policy 1, policy_version 1226521 (0.0008) [2023-12-27 14:09:16,192][20347] Updated weights for policy 0, policy_version 1227649 (0.0006) [2023-12-27 14:09:16,225][20381] Updated weights for policy 1, policy_version 1226531 (0.0006) [2023-12-27 14:09:16,246][20347] Updated weights for policy 0, policy_version 1227659 (0.0006) [2023-12-27 14:09:16,280][20381] Updated weights for policy 1, policy_version 1226541 (0.0005) [2023-12-27 14:09:16,299][20347] Updated weights for policy 0, policy_version 1227669 (0.0007) [2023-12-27 14:09:16,331][20381] Updated weights for policy 1, policy_version 1226551 (0.0005) [2023-12-27 14:09:16,360][20347] Updated weights for policy 0, policy_version 1227679 (0.0008) [2023-12-27 14:09:16,932][20381] Updated weights for policy 1, policy_version 1226561 (0.0006) [2023-12-27 14:09:16,993][20381] Updated weights for policy 1, policy_version 1226571 (0.0005) [2023-12-27 14:09:17,054][20381] Updated weights for policy 1, policy_version 1226581 (0.0007) [2023-12-27 14:09:17,056][20347] Updated weights for policy 0, policy_version 1227689 (0.0007) [2023-12-27 14:09:17,115][20347] Updated weights for policy 0, policy_version 1227699 (0.0007) [2023-12-27 14:09:17,175][20347] Updated weights for policy 0, policy_version 1227709 (0.0008) [2023-12-27 14:09:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 628383744. Throughput: 0: 10292.2, 1: 10282.5. Samples: 628391872. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:17,364][19170] Avg episode reward: [(0, '921.740'), (1, '931.670')] [2023-12-27 14:09:17,691][20381] Updated weights for policy 1, policy_version 1226591 (0.0005) [2023-12-27 14:09:17,759][20381] Updated weights for policy 1, policy_version 1226601 (0.0006) [2023-12-27 14:09:17,820][20381] Updated weights for policy 1, policy_version 1226611 (0.0005) [2023-12-27 14:09:17,841][20347] Updated weights for policy 0, policy_version 1227719 (0.0008) [2023-12-27 14:09:17,890][20347] Updated weights for policy 0, policy_version 1227729 (0.0007) [2023-12-27 14:09:17,950][20347] Updated weights for policy 0, policy_version 1227739 (0.0008) [2023-12-27 14:09:18,420][20381] Updated weights for policy 1, policy_version 1226621 (0.0008) [2023-12-27 14:09:18,486][20381] Updated weights for policy 1, policy_version 1226631 (0.0011) [2023-12-27 14:09:18,544][20381] Updated weights for policy 1, policy_version 1226641 (0.0010) [2023-12-27 14:09:18,673][20347] Updated weights for policy 0, policy_version 1227749 (0.0007) [2023-12-27 14:09:18,723][20347] Updated weights for policy 0, policy_version 1227759 (0.0006) [2023-12-27 14:09:18,775][20347] Updated weights for policy 0, policy_version 1227769 (0.0005) [2023-12-27 14:09:19,236][20381] Updated weights for policy 1, policy_version 1226651 (0.0010) [2023-12-27 14:09:19,298][20381] Updated weights for policy 1, policy_version 1226661 (0.0007) [2023-12-27 14:09:19,365][20381] Updated weights for policy 1, policy_version 1226671 (0.0007) [2023-12-27 14:09:19,465][20347] Updated weights for policy 0, policy_version 1227779 (0.0007) [2023-12-27 14:09:19,527][20347] Updated weights for policy 0, policy_version 1227789 (0.0008) [2023-12-27 14:09:19,585][20347] Updated weights for policy 0, policy_version 1227799 (0.0007) [2023-12-27 14:09:20,049][20381] Updated weights for policy 1, policy_version 1226681 (0.0009) [2023-12-27 14:09:20,098][20381] Updated weights for policy 1, policy_version 1226691 (0.0007) [2023-12-27 14:09:20,107][20292] KL-divergence is very high: 209.9542 [2023-12-27 14:09:20,150][20381] Updated weights for policy 1, policy_version 1226701 (0.0008) [2023-12-27 14:09:20,212][20381] Updated weights for policy 1, policy_version 1226711 (0.0008) [2023-12-27 14:09:20,286][20347] Updated weights for policy 0, policy_version 1227809 (0.0007) [2023-12-27 14:09:20,348][20347] Updated weights for policy 0, policy_version 1227819 (0.0006) [2023-12-27 14:09:20,405][20347] Updated weights for policy 0, policy_version 1227829 (0.0007) [2023-12-27 14:09:20,464][20347] Updated weights for policy 0, policy_version 1227839 (0.0005) [2023-12-27 14:09:20,897][20381] Updated weights for policy 1, policy_version 1226721 (0.0006) [2023-12-27 14:09:20,962][20381] Updated weights for policy 1, policy_version 1226731 (0.0007) [2023-12-27 14:09:21,020][20381] Updated weights for policy 1, policy_version 1226741 (0.0006) [2023-12-27 14:09:21,183][20347] Updated weights for policy 0, policy_version 1227849 (0.0008) [2023-12-27 14:09:21,246][20347] Updated weights for policy 0, policy_version 1227859 (0.0007) [2023-12-27 14:09:21,314][20347] Updated weights for policy 0, policy_version 1227869 (0.0006) [2023-12-27 14:09:21,720][20381] Updated weights for policy 1, policy_version 1226751 (0.0008) [2023-12-27 14:09:21,779][20381] Updated weights for policy 1, policy_version 1226761 (0.0008) [2023-12-27 14:09:21,837][20381] Updated weights for policy 1, policy_version 1226771 (0.0006) [2023-12-27 14:09:22,065][20347] Updated weights for policy 0, policy_version 1227879 (0.0008) [2023-12-27 14:09:22,118][20347] Updated weights for policy 0, policy_version 1227889 (0.0007) [2023-12-27 14:09:22,178][20347] Updated weights for policy 0, policy_version 1227899 (0.0008) [2023-12-27 14:09:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 628482048. Throughput: 0: 10219.6, 1: 10236.3. Samples: 628452292. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:22,363][19170] Avg episode reward: [(0, '882.970'), (1, '902.170')] [2023-12-27 14:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001226776_314081280.pth... [2023-12-27 14:09:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001227904_314400768.pth... [2023-12-27 14:09:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001225592_313778176.pth [2023-12-27 14:09:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001226720_314097664.pth [2023-12-27 14:09:22,552][20381] Updated weights for policy 1, policy_version 1226781 (0.0007) [2023-12-27 14:09:22,608][20381] Updated weights for policy 1, policy_version 1226791 (0.0008) [2023-12-27 14:09:22,666][20381] Updated weights for policy 1, policy_version 1226801 (0.0008) [2023-12-27 14:09:22,914][20347] Updated weights for policy 0, policy_version 1227909 (0.0007) [2023-12-27 14:09:22,967][20347] Updated weights for policy 0, policy_version 1227919 (0.0008) [2023-12-27 14:09:23,020][20347] Updated weights for policy 0, policy_version 1227929 (0.0008) [2023-12-27 14:09:23,343][20381] Updated weights for policy 1, policy_version 1226811 (0.0007) [2023-12-27 14:09:23,400][20381] Updated weights for policy 1, policy_version 1226821 (0.0006) [2023-12-27 14:09:23,460][20381] Updated weights for policy 1, policy_version 1226831 (0.0007) [2023-12-27 14:09:23,734][20347] Updated weights for policy 0, policy_version 1227939 (0.0008) [2023-12-27 14:09:23,788][20347] Updated weights for policy 0, policy_version 1227949 (0.0007) [2023-12-27 14:09:23,844][20347] Updated weights for policy 0, policy_version 1227959 (0.0007) [2023-12-27 14:09:24,142][20381] Updated weights for policy 1, policy_version 1226841 (0.0010) [2023-12-27 14:09:24,202][20381] Updated weights for policy 1, policy_version 1226851 (0.0008) [2023-12-27 14:09:24,261][20381] Updated weights for policy 1, policy_version 1226861 (0.0011) [2023-12-27 14:09:24,317][20381] Updated weights for policy 1, policy_version 1226871 (0.0010) [2023-12-27 14:09:24,461][20347] Updated weights for policy 0, policy_version 1227969 (0.0006) [2023-12-27 14:09:24,515][20347] Updated weights for policy 0, policy_version 1227979 (0.0007) [2023-12-27 14:09:24,533][20291] KL-divergence is very high: 149.3145 [2023-12-27 14:09:24,563][20347] Updated weights for policy 0, policy_version 1227989 (0.0007) [2023-12-27 14:09:24,573][20291] KL-divergence is very high: 123.3437 [2023-12-27 14:09:24,619][20347] Updated weights for policy 0, policy_version 1227999 (0.0007) [2023-12-27 14:09:25,004][20381] Updated weights for policy 1, policy_version 1226881 (0.0011) [2023-12-27 14:09:25,063][20381] Updated weights for policy 1, policy_version 1226891 (0.0005) [2023-12-27 14:09:25,117][20381] Updated weights for policy 1, policy_version 1226901 (0.0005) [2023-12-27 14:09:25,289][20347] Updated weights for policy 0, policy_version 1228009 (0.0008) [2023-12-27 14:09:25,338][20347] Updated weights for policy 0, policy_version 1228019 (0.0007) [2023-12-27 14:09:25,394][20347] Updated weights for policy 0, policy_version 1228029 (0.0007) [2023-12-27 14:09:25,754][20381] Updated weights for policy 1, policy_version 1226911 (0.0011) [2023-12-27 14:09:25,809][20381] Updated weights for policy 1, policy_version 1226921 (0.0011) [2023-12-27 14:09:25,869][20381] Updated weights for policy 1, policy_version 1226931 (0.0011) [2023-12-27 14:09:26,111][20347] Updated weights for policy 0, policy_version 1228039 (0.0008) [2023-12-27 14:09:26,169][20347] Updated weights for policy 0, policy_version 1228049 (0.0007) [2023-12-27 14:09:26,231][20347] Updated weights for policy 0, policy_version 1228059 (0.0007) [2023-12-27 14:09:26,560][20381] Updated weights for policy 1, policy_version 1226941 (0.0010) [2023-12-27 14:09:26,611][20381] Updated weights for policy 1, policy_version 1226951 (0.0010) [2023-12-27 14:09:26,663][20381] Updated weights for policy 1, policy_version 1226961 (0.0010) [2023-12-27 14:09:26,917][20347] Updated weights for policy 0, policy_version 1228069 (0.0007) [2023-12-27 14:09:26,968][20347] Updated weights for policy 0, policy_version 1228079 (0.0007) [2023-12-27 14:09:27,014][20347] Updated weights for policy 0, policy_version 1228089 (0.0007) [2023-12-27 14:09:27,359][20381] Updated weights for policy 1, policy_version 1226971 (0.0011) [2023-12-27 14:09:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 628580352. Throughput: 0: 10136.7, 1: 10287.3. Samples: 628573548. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:27,363][19170] Avg episode reward: [(0, '853.670'), (1, '921.770')] [2023-12-27 14:09:27,413][20381] Updated weights for policy 1, policy_version 1226981 (0.0010) [2023-12-27 14:09:27,430][20292] KL-divergence is very high: 124.8475 [2023-12-27 14:09:27,461][20381] Updated weights for policy 1, policy_version 1226991 (0.0010) [2023-12-27 14:09:27,470][20292] KL-divergence is very high: 168.4374 [2023-12-27 14:09:27,733][20347] Updated weights for policy 0, policy_version 1228099 (0.0006) [2023-12-27 14:09:27,779][20347] Updated weights for policy 0, policy_version 1228109 (0.0005) [2023-12-27 14:09:27,823][20347] Updated weights for policy 0, policy_version 1228119 (0.0005) [2023-12-27 14:09:28,030][20381] Updated weights for policy 1, policy_version 1227001 (0.0010) [2023-12-27 14:09:28,081][20381] Updated weights for policy 1, policy_version 1227011 (0.0008) [2023-12-27 14:09:28,129][20381] Updated weights for policy 1, policy_version 1227021 (0.0007) [2023-12-27 14:09:28,181][20381] Updated weights for policy 1, policy_version 1227031 (0.0006) [2023-12-27 14:09:28,486][20347] Updated weights for policy 0, policy_version 1228129 (0.0007) [2023-12-27 14:09:28,550][20347] Updated weights for policy 0, policy_version 1228139 (0.0010) [2023-12-27 14:09:28,609][20347] Updated weights for policy 0, policy_version 1228149 (0.0010) [2023-12-27 14:09:28,663][20347] Updated weights for policy 0, policy_version 1228159 (0.0007) [2023-12-27 14:09:28,864][20381] Updated weights for policy 1, policy_version 1227041 (0.0010) [2023-12-27 14:09:28,923][20381] Updated weights for policy 1, policy_version 1227051 (0.0011) [2023-12-27 14:09:28,983][20381] Updated weights for policy 1, policy_version 1227061 (0.0010) [2023-12-27 14:09:29,264][20347] Updated weights for policy 0, policy_version 1228169 (0.0007) [2023-12-27 14:09:29,328][20347] Updated weights for policy 0, policy_version 1228179 (0.0007) [2023-12-27 14:09:29,392][20347] Updated weights for policy 0, policy_version 1228189 (0.0007) [2023-12-27 14:09:29,640][20381] Updated weights for policy 1, policy_version 1227071 (0.0007) [2023-12-27 14:09:29,693][20381] Updated weights for policy 1, policy_version 1227081 (0.0005) [2023-12-27 14:09:29,757][20381] Updated weights for policy 1, policy_version 1227091 (0.0005) [2023-12-27 14:09:30,038][20347] Updated weights for policy 0, policy_version 1228199 (0.0007) [2023-12-27 14:09:30,099][20347] Updated weights for policy 0, policy_version 1228209 (0.0006) [2023-12-27 14:09:30,158][20347] Updated weights for policy 0, policy_version 1228219 (0.0006) [2023-12-27 14:09:30,397][20381] Updated weights for policy 1, policy_version 1227101 (0.0008) [2023-12-27 14:09:30,447][20381] Updated weights for policy 1, policy_version 1227111 (0.0010) [2023-12-27 14:09:30,500][20381] Updated weights for policy 1, policy_version 1227121 (0.0010) [2023-12-27 14:09:30,853][20347] Updated weights for policy 0, policy_version 1228229 (0.0008) [2023-12-27 14:09:30,905][20347] Updated weights for policy 0, policy_version 1228239 (0.0009) [2023-12-27 14:09:30,960][20347] Updated weights for policy 0, policy_version 1228249 (0.0007) [2023-12-27 14:09:31,105][20381] Updated weights for policy 1, policy_version 1227131 (0.0009) [2023-12-27 14:09:31,185][20381] Updated weights for policy 1, policy_version 1227141 (0.0007) [2023-12-27 14:09:31,251][20381] Updated weights for policy 1, policy_version 1227151 (0.0006) [2023-12-27 14:09:31,778][20347] Updated weights for policy 0, policy_version 1228259 (0.0008) [2023-12-27 14:09:31,829][20347] Updated weights for policy 0, policy_version 1228269 (0.0008) [2023-12-27 14:09:31,878][20381] Updated weights for policy 1, policy_version 1227161 (0.0006) [2023-12-27 14:09:31,884][20347] Updated weights for policy 0, policy_version 1228279 (0.0008) [2023-12-27 14:09:31,926][20381] Updated weights for policy 1, policy_version 1227171 (0.0010) [2023-12-27 14:09:31,986][20381] Updated weights for policy 1, policy_version 1227181 (0.0005) [2023-12-27 14:09:32,048][20381] Updated weights for policy 1, policy_version 1227191 (0.0007) [2023-12-27 14:09:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 628686848. Throughput: 0: 10075.4, 1: 10332.6. Samples: 628697656. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:32,363][19170] Avg episode reward: [(0, '873.480'), (1, '912.140')] [2023-12-27 14:09:32,546][20347] Updated weights for policy 0, policy_version 1228289 (0.0008) [2023-12-27 14:09:32,604][20347] Updated weights for policy 0, policy_version 1228299 (0.0007) [2023-12-27 14:09:32,661][20347] Updated weights for policy 0, policy_version 1228309 (0.0007) [2023-12-27 14:09:32,693][20381] Updated weights for policy 1, policy_version 1227201 (0.0011) [2023-12-27 14:09:32,720][20347] Updated weights for policy 0, policy_version 1228319 (0.0007) [2023-12-27 14:09:32,739][20381] Updated weights for policy 1, policy_version 1227211 (0.0010) [2023-12-27 14:09:32,785][20381] Updated weights for policy 1, policy_version 1227221 (0.0010) [2023-12-27 14:09:33,354][20347] Updated weights for policy 0, policy_version 1228329 (0.0007) [2023-12-27 14:09:33,411][20347] Updated weights for policy 0, policy_version 1228339 (0.0007) [2023-12-27 14:09:33,450][20381] Updated weights for policy 1, policy_version 1227231 (0.0011) [2023-12-27 14:09:33,468][20347] Updated weights for policy 0, policy_version 1228349 (0.0005) [2023-12-27 14:09:33,511][20381] Updated weights for policy 1, policy_version 1227241 (0.0010) [2023-12-27 14:09:33,564][20381] Updated weights for policy 1, policy_version 1227251 (0.0006) [2023-12-27 14:09:34,048][20347] Updated weights for policy 0, policy_version 1228359 (0.0005) [2023-12-27 14:09:34,098][20347] Updated weights for policy 0, policy_version 1228369 (0.0005) [2023-12-27 14:09:34,153][20347] Updated weights for policy 0, policy_version 1228379 (0.0008) [2023-12-27 14:09:34,259][20381] Updated weights for policy 1, policy_version 1227261 (0.0007) [2023-12-27 14:09:34,322][20381] Updated weights for policy 1, policy_version 1227271 (0.0006) [2023-12-27 14:09:34,389][20381] Updated weights for policy 1, policy_version 1227281 (0.0009) [2023-12-27 14:09:34,830][20347] Updated weights for policy 0, policy_version 1228389 (0.0007) [2023-12-27 14:09:34,878][20347] Updated weights for policy 0, policy_version 1228399 (0.0005) [2023-12-27 14:09:34,928][20347] Updated weights for policy 0, policy_version 1228409 (0.0009) [2023-12-27 14:09:35,017][20381] Updated weights for policy 1, policy_version 1227291 (0.0010) [2023-12-27 14:09:35,071][20381] Updated weights for policy 1, policy_version 1227301 (0.0007) [2023-12-27 14:09:35,131][20381] Updated weights for policy 1, policy_version 1227311 (0.0007) [2023-12-27 14:09:35,183][20292] KL-divergence is very high: 125.9918 [2023-12-27 14:09:35,543][20347] Updated weights for policy 0, policy_version 1228419 (0.0010) [2023-12-27 14:09:35,598][20347] Updated weights for policy 0, policy_version 1228429 (0.0010) [2023-12-27 14:09:35,664][20347] Updated weights for policy 0, policy_version 1228439 (0.0010) [2023-12-27 14:09:35,786][20381] Updated weights for policy 1, policy_version 1227321 (0.0006) [2023-12-27 14:09:35,844][20381] Updated weights for policy 1, policy_version 1227331 (0.0010) [2023-12-27 14:09:35,899][20381] Updated weights for policy 1, policy_version 1227341 (0.0010) [2023-12-27 14:09:35,956][20381] Updated weights for policy 1, policy_version 1227351 (0.0010) [2023-12-27 14:09:36,309][20347] Updated weights for policy 0, policy_version 1228449 (0.0011) [2023-12-27 14:09:36,359][20347] Updated weights for policy 0, policy_version 1228459 (0.0009) [2023-12-27 14:09:36,428][20347] Updated weights for policy 0, policy_version 1228469 (0.0008) [2023-12-27 14:09:36,491][20347] Updated weights for policy 0, policy_version 1228479 (0.0007) [2023-12-27 14:09:36,590][20381] Updated weights for policy 1, policy_version 1227361 (0.0007) [2023-12-27 14:09:36,647][20381] Updated weights for policy 1, policy_version 1227371 (0.0006) [2023-12-27 14:09:36,704][20381] Updated weights for policy 1, policy_version 1227381 (0.0006) [2023-12-27 14:09:37,132][20347] Updated weights for policy 0, policy_version 1228489 (0.0007) [2023-12-27 14:09:37,190][20347] Updated weights for policy 0, policy_version 1228499 (0.0010) [2023-12-27 14:09:37,242][20347] Updated weights for policy 0, policy_version 1228509 (0.0006) [2023-12-27 14:09:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 628793344. Throughput: 0: 10136.4, 1: 10326.3. Samples: 628761504. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:37,363][19170] Avg episode reward: [(0, '931.720'), (1, '882.840')] [2023-12-27 14:09:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001228512_314556416.pth... [2023-12-27 14:09:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001227328_314253312.pth [2023-12-27 14:09:37,386][20381] Updated weights for policy 1, policy_version 1227391 (0.0008) [2023-12-27 14:09:37,450][20381] Updated weights for policy 1, policy_version 1227401 (0.0007) [2023-12-27 14:09:37,512][20381] Updated weights for policy 1, policy_version 1227411 (0.0007) [2023-12-27 14:09:37,538][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001227416_314245120.pth... [2023-12-27 14:09:37,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001226200_313933824.pth [2023-12-27 14:09:37,928][20347] Updated weights for policy 0, policy_version 1228519 (0.0009) [2023-12-27 14:09:37,977][20347] Updated weights for policy 0, policy_version 1228529 (0.0010) [2023-12-27 14:09:38,026][20347] Updated weights for policy 0, policy_version 1228539 (0.0010) [2023-12-27 14:09:38,095][20381] Updated weights for policy 1, policy_version 1227421 (0.0007) [2023-12-27 14:09:38,146][20381] Updated weights for policy 1, policy_version 1227431 (0.0007) [2023-12-27 14:09:38,199][20381] Updated weights for policy 1, policy_version 1227441 (0.0007) [2023-12-27 14:09:38,743][20347] Updated weights for policy 0, policy_version 1228549 (0.0007) [2023-12-27 14:09:38,804][20347] Updated weights for policy 0, policy_version 1228559 (0.0008) [2023-12-27 14:09:38,860][20347] Updated weights for policy 0, policy_version 1228569 (0.0008) [2023-12-27 14:09:38,914][20381] Updated weights for policy 1, policy_version 1227451 (0.0007) [2023-12-27 14:09:38,970][20381] Updated weights for policy 1, policy_version 1227461 (0.0007) [2023-12-27 14:09:39,028][20381] Updated weights for policy 1, policy_version 1227471 (0.0007) [2023-12-27 14:09:39,610][20347] Updated weights for policy 0, policy_version 1228579 (0.0007) [2023-12-27 14:09:39,663][20347] Updated weights for policy 0, policy_version 1228589 (0.0007) [2023-12-27 14:09:39,715][20347] Updated weights for policy 0, policy_version 1228599 (0.0006) [2023-12-27 14:09:39,748][20381] Updated weights for policy 1, policy_version 1227481 (0.0006) [2023-12-27 14:09:39,806][20381] Updated weights for policy 1, policy_version 1227491 (0.0007) [2023-12-27 14:09:39,874][20381] Updated weights for policy 1, policy_version 1227501 (0.0008) [2023-12-27 14:09:39,936][20381] Updated weights for policy 1, policy_version 1227511 (0.0008) [2023-12-27 14:09:40,387][20347] Updated weights for policy 0, policy_version 1228609 (0.0006) [2023-12-27 14:09:40,440][20347] Updated weights for policy 0, policy_version 1228619 (0.0007) [2023-12-27 14:09:40,495][20347] Updated weights for policy 0, policy_version 1228629 (0.0006) [2023-12-27 14:09:40,556][20347] Updated weights for policy 0, policy_version 1228639 (0.0010) [2023-12-27 14:09:40,617][20381] Updated weights for policy 1, policy_version 1227521 (0.0007) [2023-12-27 14:09:40,648][20292] KL-divergence is very high: 166.2358 [2023-12-27 14:09:40,662][20292] KL-divergence is very high: 192.0873 [2023-12-27 14:09:40,686][20381] Updated weights for policy 1, policy_version 1227531 (0.0008) [2023-12-27 14:09:40,701][20292] KL-divergence is very high: 137.4250 [2023-12-27 14:09:40,754][20381] Updated weights for policy 1, policy_version 1227541 (0.0007) [2023-12-27 14:09:41,225][20347] Updated weights for policy 0, policy_version 1228649 (0.0008) [2023-12-27 14:09:41,285][20347] Updated weights for policy 0, policy_version 1228659 (0.0008) [2023-12-27 14:09:41,349][20347] Updated weights for policy 0, policy_version 1228669 (0.0008) [2023-12-27 14:09:41,517][20381] Updated weights for policy 1, policy_version 1227551 (0.0008) [2023-12-27 14:09:41,576][20381] Updated weights for policy 1, policy_version 1227561 (0.0007) [2023-12-27 14:09:41,641][20381] Updated weights for policy 1, policy_version 1227571 (0.0008) [2023-12-27 14:09:41,652][20292] KL-divergence is very high: 132.6329 [2023-12-27 14:09:41,667][20292] KL-divergence is very high: 144.5520 [2023-12-27 14:09:42,066][20347] Updated weights for policy 0, policy_version 1228679 (0.0007) [2023-12-27 14:09:42,132][20347] Updated weights for policy 0, policy_version 1228689 (0.0006) [2023-12-27 14:09:42,198][20347] Updated weights for policy 0, policy_version 1228699 (0.0006) [2023-12-27 14:09:42,357][20381] Updated weights for policy 1, policy_version 1227581 (0.0008) [2023-12-27 14:09:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 628891648. Throughput: 0: 10146.8, 1: 10341.8. Samples: 628884516. Policy #0 lag: (min: 31.0, avg: 32.6, max: 60.0) [2023-12-27 14:09:42,363][19170] Avg episode reward: [(0, '922.000'), (1, '882.910')] [2023-12-27 14:09:42,421][20381] Updated weights for policy 1, policy_version 1227591 (0.0008) [2023-12-27 14:09:42,488][20381] Updated weights for policy 1, policy_version 1227601 (0.0007) [2023-12-27 14:09:42,838][20347] Updated weights for policy 0, policy_version 1228709 (0.0006) [2023-12-27 14:09:42,896][20347] Updated weights for policy 0, policy_version 1228719 (0.0008) [2023-12-27 14:09:42,956][20347] Updated weights for policy 0, policy_version 1228729 (0.0008) [2023-12-27 14:09:43,136][20381] Updated weights for policy 1, policy_version 1227611 (0.0007) [2023-12-27 14:09:43,183][20381] Updated weights for policy 1, policy_version 1227621 (0.0005) [2023-12-27 14:09:43,229][20381] Updated weights for policy 1, policy_version 1227631 (0.0005) [2023-12-27 14:09:43,229][20292] KL-divergence is very high: 231.6759 [2023-12-27 14:09:43,265][20292] KL-divergence is very high: 216.2414 [2023-12-27 14:09:43,614][20347] Updated weights for policy 0, policy_version 1228739 (0.0006) [2023-12-27 14:09:43,676][20347] Updated weights for policy 0, policy_version 1228749 (0.0005) [2023-12-27 14:09:43,732][20347] Updated weights for policy 0, policy_version 1228759 (0.0006) [2023-12-27 14:09:43,808][20381] Updated weights for policy 1, policy_version 1227641 (0.0008) [2023-12-27 14:09:43,866][20381] Updated weights for policy 1, policy_version 1227651 (0.0005) [2023-12-27 14:09:43,916][20381] Updated weights for policy 1, policy_version 1227661 (0.0005) [2023-12-27 14:09:43,966][20381] Updated weights for policy 1, policy_version 1227671 (0.0005) [2023-12-27 14:09:44,304][20347] Updated weights for policy 0, policy_version 1228769 (0.0006) [2023-12-27 14:09:44,361][20347] Updated weights for policy 0, policy_version 1228779 (0.0010) [2023-12-27 14:09:44,421][20347] Updated weights for policy 0, policy_version 1228789 (0.0010) [2023-12-27 14:09:44,479][20347] Updated weights for policy 0, policy_version 1228799 (0.0009) [2023-12-27 14:09:44,557][20381] Updated weights for policy 1, policy_version 1227681 (0.0009) [2023-12-27 14:09:44,615][20381] Updated weights for policy 1, policy_version 1227691 (0.0011) [2023-12-27 14:09:44,666][20381] Updated weights for policy 1, policy_version 1227701 (0.0011) [2023-12-27 14:09:45,137][20347] Updated weights for policy 0, policy_version 1228809 (0.0010) [2023-12-27 14:09:45,198][20347] Updated weights for policy 0, policy_version 1228819 (0.0011) [2023-12-27 14:09:45,262][20347] Updated weights for policy 0, policy_version 1228829 (0.0010) [2023-12-27 14:09:45,365][20381] Updated weights for policy 1, policy_version 1227711 (0.0009) [2023-12-27 14:09:45,415][20381] Updated weights for policy 1, policy_version 1227721 (0.0009) [2023-12-27 14:09:45,468][20381] Updated weights for policy 1, policy_version 1227731 (0.0010) [2023-12-27 14:09:45,875][20347] Updated weights for policy 0, policy_version 1228839 (0.0008) [2023-12-27 14:09:45,934][20347] Updated weights for policy 0, policy_version 1228849 (0.0007) [2023-12-27 14:09:45,985][20347] Updated weights for policy 0, policy_version 1228859 (0.0005) [2023-12-27 14:09:46,191][20381] Updated weights for policy 1, policy_version 1227741 (0.0010) [2023-12-27 14:09:46,252][20381] Updated weights for policy 1, policy_version 1227751 (0.0011) [2023-12-27 14:09:46,307][20381] Updated weights for policy 1, policy_version 1227761 (0.0010) [2023-12-27 14:09:46,599][20347] Updated weights for policy 0, policy_version 1228869 (0.0008) [2023-12-27 14:09:46,646][20347] Updated weights for policy 0, policy_version 1228879 (0.0009) [2023-12-27 14:09:46,699][20347] Updated weights for policy 0, policy_version 1228889 (0.0009) [2023-12-27 14:09:46,920][20381] Updated weights for policy 1, policy_version 1227771 (0.0011) [2023-12-27 14:09:46,973][20381] Updated weights for policy 1, policy_version 1227781 (0.0010) [2023-12-27 14:09:47,024][20381] Updated weights for policy 1, policy_version 1227791 (0.0010) [2023-12-27 14:09:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 628998144. Throughput: 0: 10236.6, 1: 10419.6. Samples: 629010380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:09:47,363][19170] Avg episode reward: [(0, '921.870'), (1, '912.880')] [2023-12-27 14:09:47,392][20347] Updated weights for policy 0, policy_version 1228899 (0.0008) [2023-12-27 14:09:47,455][20347] Updated weights for policy 0, policy_version 1228909 (0.0008) [2023-12-27 14:09:47,519][20347] Updated weights for policy 0, policy_version 1228919 (0.0008) [2023-12-27 14:09:47,618][20381] Updated weights for policy 1, policy_version 1227801 (0.0010) [2023-12-27 14:09:47,675][20381] Updated weights for policy 1, policy_version 1227811 (0.0005) [2023-12-27 14:09:47,731][20381] Updated weights for policy 1, policy_version 1227821 (0.0006) [2023-12-27 14:09:47,784][20381] Updated weights for policy 1, policy_version 1227831 (0.0008) [2023-12-27 14:09:48,156][20347] Updated weights for policy 0, policy_version 1228929 (0.0007) [2023-12-27 14:09:48,207][20347] Updated weights for policy 0, policy_version 1228939 (0.0007) [2023-12-27 14:09:48,260][20347] Updated weights for policy 0, policy_version 1228949 (0.0009) [2023-12-27 14:09:48,306][20347] Updated weights for policy 0, policy_version 1228959 (0.0007) [2023-12-27 14:09:48,410][20381] Updated weights for policy 1, policy_version 1227841 (0.0010) [2023-12-27 14:09:48,470][20381] Updated weights for policy 1, policy_version 1227851 (0.0011) [2023-12-27 14:09:48,530][20381] Updated weights for policy 1, policy_version 1227861 (0.0010) [2023-12-27 14:09:49,024][20347] Updated weights for policy 0, policy_version 1228969 (0.0006) [2023-12-27 14:09:49,081][20347] Updated weights for policy 0, policy_version 1228979 (0.0005) [2023-12-27 14:09:49,145][20347] Updated weights for policy 0, policy_version 1228989 (0.0005) [2023-12-27 14:09:49,218][20381] Updated weights for policy 1, policy_version 1227871 (0.0010) [2023-12-27 14:09:49,285][20381] Updated weights for policy 1, policy_version 1227881 (0.0009) [2023-12-27 14:09:49,353][20381] Updated weights for policy 1, policy_version 1227891 (0.0010) [2023-12-27 14:09:49,827][20347] Updated weights for policy 0, policy_version 1228999 (0.0008) [2023-12-27 14:09:49,890][20347] Updated weights for policy 0, policy_version 1229009 (0.0007) [2023-12-27 14:09:49,961][20347] Updated weights for policy 0, policy_version 1229019 (0.0008) [2023-12-27 14:09:50,055][20381] Updated weights for policy 1, policy_version 1227901 (0.0009) [2023-12-27 14:09:50,114][20381] Updated weights for policy 1, policy_version 1227911 (0.0011) [2023-12-27 14:09:50,172][20381] Updated weights for policy 1, policy_version 1227921 (0.0010) [2023-12-27 14:09:50,679][20347] Updated weights for policy 0, policy_version 1229029 (0.0008) [2023-12-27 14:09:50,735][20347] Updated weights for policy 0, policy_version 1229039 (0.0007) [2023-12-27 14:09:50,801][20347] Updated weights for policy 0, policy_version 1229049 (0.0006) [2023-12-27 14:09:50,815][20381] Updated weights for policy 1, policy_version 1227931 (0.0009) [2023-12-27 14:09:50,869][20381] Updated weights for policy 1, policy_version 1227941 (0.0006) [2023-12-27 14:09:50,925][20381] Updated weights for policy 1, policy_version 1227951 (0.0008) [2023-12-27 14:09:51,475][20347] Updated weights for policy 0, policy_version 1229059 (0.0007) [2023-12-27 14:09:51,532][20347] Updated weights for policy 0, policy_version 1229069 (0.0008) [2023-12-27 14:09:51,586][20347] Updated weights for policy 0, policy_version 1229079 (0.0007) [2023-12-27 14:09:51,683][20381] Updated weights for policy 1, policy_version 1227961 (0.0008) [2023-12-27 14:09:51,747][20381] Updated weights for policy 1, policy_version 1227971 (0.0009) [2023-12-27 14:09:51,807][20381] Updated weights for policy 1, policy_version 1227981 (0.0009) [2023-12-27 14:09:51,863][20381] Updated weights for policy 1, policy_version 1227991 (0.0009) [2023-12-27 14:09:52,277][20347] Updated weights for policy 0, policy_version 1229089 (0.0007) [2023-12-27 14:09:52,324][20347] Updated weights for policy 0, policy_version 1229099 (0.0007) [2023-12-27 14:09:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 629096448. Throughput: 0: 10216.6, 1: 10401.3. Samples: 629071864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:09:52,363][19170] Avg episode reward: [(0, '921.630'), (1, '902.320')] [2023-12-27 14:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001227992_314392576.pth... [2023-12-27 14:09:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001226776_314081280.pth [2023-12-27 14:09:52,391][20347] Updated weights for policy 0, policy_version 1229109 (0.0007) [2023-12-27 14:09:52,446][20347] Updated weights for policy 0, policy_version 1229119 (0.0008) [2023-12-27 14:09:52,448][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001229120_314712064.pth... [2023-12-27 14:09:52,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001227904_314400768.pth [2023-12-27 14:09:52,629][20381] Updated weights for policy 1, policy_version 1228001 (0.0010) [2023-12-27 14:09:52,682][20381] Updated weights for policy 1, policy_version 1228011 (0.0011) [2023-12-27 14:09:52,734][20381] Updated weights for policy 1, policy_version 1228021 (0.0011) [2023-12-27 14:09:53,164][20347] Updated weights for policy 0, policy_version 1229129 (0.0007) [2023-12-27 14:09:53,212][20347] Updated weights for policy 0, policy_version 1229139 (0.0005) [2023-12-27 14:09:53,261][20347] Updated weights for policy 0, policy_version 1229149 (0.0007) [2023-12-27 14:09:53,364][20381] Updated weights for policy 1, policy_version 1228031 (0.0009) [2023-12-27 14:09:53,410][20381] Updated weights for policy 1, policy_version 1228041 (0.0008) [2023-12-27 14:09:53,462][20381] Updated weights for policy 1, policy_version 1228051 (0.0007) [2023-12-27 14:09:53,958][20347] Updated weights for policy 0, policy_version 1229159 (0.0008) [2023-12-27 14:09:54,022][20347] Updated weights for policy 0, policy_version 1229169 (0.0007) [2023-12-27 14:09:54,081][20347] Updated weights for policy 0, policy_version 1229179 (0.0007) [2023-12-27 14:09:54,164][20381] Updated weights for policy 1, policy_version 1228061 (0.0008) [2023-12-27 14:09:54,216][20381] Updated weights for policy 1, policy_version 1228071 (0.0008) [2023-12-27 14:09:54,267][20381] Updated weights for policy 1, policy_version 1228081 (0.0008) [2023-12-27 14:09:54,690][20347] Updated weights for policy 0, policy_version 1229189 (0.0006) [2023-12-27 14:09:54,746][20347] Updated weights for policy 0, policy_version 1229199 (0.0005) [2023-12-27 14:09:54,795][20347] Updated weights for policy 0, policy_version 1229209 (0.0005) [2023-12-27 14:09:54,964][20381] Updated weights for policy 1, policy_version 1228091 (0.0008) [2023-12-27 14:09:55,028][20381] Updated weights for policy 1, policy_version 1228101 (0.0007) [2023-12-27 14:09:55,081][20381] Updated weights for policy 1, policy_version 1228111 (0.0005) [2023-12-27 14:09:55,449][20347] Updated weights for policy 0, policy_version 1229219 (0.0006) [2023-12-27 14:09:55,507][20347] Updated weights for policy 0, policy_version 1229229 (0.0008) [2023-12-27 14:09:55,564][20347] Updated weights for policy 0, policy_version 1229239 (0.0008) [2023-12-27 14:09:55,642][20381] Updated weights for policy 1, policy_version 1228121 (0.0005) [2023-12-27 14:09:55,703][20381] Updated weights for policy 1, policy_version 1228131 (0.0008) [2023-12-27 14:09:55,763][20381] Updated weights for policy 1, policy_version 1228141 (0.0008) [2023-12-27 14:09:55,828][20381] Updated weights for policy 1, policy_version 1228151 (0.0007) [2023-12-27 14:09:56,266][20347] Updated weights for policy 0, policy_version 1229249 (0.0007) [2023-12-27 14:09:56,321][20347] Updated weights for policy 0, policy_version 1229259 (0.0008) [2023-12-27 14:09:56,385][20347] Updated weights for policy 0, policy_version 1229269 (0.0008) [2023-12-27 14:09:56,440][20347] Updated weights for policy 0, policy_version 1229279 (0.0006) [2023-12-27 14:09:56,519][20381] Updated weights for policy 1, policy_version 1228161 (0.0007) [2023-12-27 14:09:56,564][20381] Updated weights for policy 1, policy_version 1228171 (0.0008) [2023-12-27 14:09:56,615][20381] Updated weights for policy 1, policy_version 1228181 (0.0007) [2023-12-27 14:09:57,128][20347] Updated weights for policy 0, policy_version 1229289 (0.0007) [2023-12-27 14:09:57,173][20347] Updated weights for policy 0, policy_version 1229299 (0.0008) [2023-12-27 14:09:57,217][20347] Updated weights for policy 0, policy_version 1229309 (0.0007) [2023-12-27 14:09:57,295][20381] Updated weights for policy 1, policy_version 1228191 (0.0007) [2023-12-27 14:09:57,348][20381] Updated weights for policy 1, policy_version 1228201 (0.0008) [2023-12-27 14:09:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 629202944. Throughput: 0: 10216.3, 1: 10360.3. Samples: 629194320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:09:57,363][19170] Avg episode reward: [(0, '882.740'), (1, '951.180')] [2023-12-27 14:09:57,393][20381] Updated weights for policy 1, policy_version 1228211 (0.0007) [2023-12-27 14:09:57,959][20347] Updated weights for policy 0, policy_version 1229319 (0.0005) [2023-12-27 14:09:58,005][20381] Updated weights for policy 1, policy_version 1228221 (0.0006) [2023-12-27 14:09:58,018][20347] Updated weights for policy 0, policy_version 1229329 (0.0006) [2023-12-27 14:09:58,059][20381] Updated weights for policy 1, policy_version 1228231 (0.0005) [2023-12-27 14:09:58,064][20347] Updated weights for policy 0, policy_version 1229339 (0.0007) [2023-12-27 14:09:58,112][20381] Updated weights for policy 1, policy_version 1228241 (0.0005) [2023-12-27 14:09:58,797][20381] Updated weights for policy 1, policy_version 1228251 (0.0006) [2023-12-27 14:09:58,828][20347] Updated weights for policy 0, policy_version 1229349 (0.0007) [2023-12-27 14:09:58,867][20381] Updated weights for policy 1, policy_version 1228261 (0.0009) [2023-12-27 14:09:58,898][20347] Updated weights for policy 0, policy_version 1229359 (0.0007) [2023-12-27 14:09:58,934][20381] Updated weights for policy 1, policy_version 1228271 (0.0007) [2023-12-27 14:09:58,960][20347] Updated weights for policy 0, policy_version 1229369 (0.0006) [2023-12-27 14:09:59,612][20381] Updated weights for policy 1, policy_version 1228281 (0.0006) [2023-12-27 14:09:59,671][20381] Updated weights for policy 1, policy_version 1228291 (0.0008) [2023-12-27 14:09:59,682][20347] Updated weights for policy 0, policy_version 1229379 (0.0007) [2023-12-27 14:09:59,726][20381] Updated weights for policy 1, policy_version 1228301 (0.0008) [2023-12-27 14:09:59,734][20347] Updated weights for policy 0, policy_version 1229389 (0.0009) [2023-12-27 14:09:59,779][20381] Updated weights for policy 1, policy_version 1228311 (0.0006) [2023-12-27 14:09:59,781][20347] Updated weights for policy 0, policy_version 1229399 (0.0006) [2023-12-27 14:10:00,448][20381] Updated weights for policy 1, policy_version 1228321 (0.0005) [2023-12-27 14:10:00,506][20381] Updated weights for policy 1, policy_version 1228331 (0.0005) [2023-12-27 14:10:00,521][20347] Updated weights for policy 0, policy_version 1229409 (0.0008) [2023-12-27 14:10:00,561][20381] Updated weights for policy 1, policy_version 1228341 (0.0007) [2023-12-27 14:10:00,583][20347] Updated weights for policy 0, policy_version 1229419 (0.0006) [2023-12-27 14:10:00,631][20347] Updated weights for policy 0, policy_version 1229429 (0.0007) [2023-12-27 14:10:00,676][20347] Updated weights for policy 0, policy_version 1229439 (0.0008) [2023-12-27 14:10:01,117][20381] Updated weights for policy 1, policy_version 1228351 (0.0008) [2023-12-27 14:10:01,184][20381] Updated weights for policy 1, policy_version 1228361 (0.0008) [2023-12-27 14:10:01,233][20381] Updated weights for policy 1, policy_version 1228371 (0.0008) [2023-12-27 14:10:01,349][20347] Updated weights for policy 0, policy_version 1229449 (0.0008) [2023-12-27 14:10:01,396][20291] KL-divergence is very high: 128.3782 [2023-12-27 14:10:01,415][20347] Updated weights for policy 0, policy_version 1229459 (0.0008) [2023-12-27 14:10:01,471][20347] Updated weights for policy 0, policy_version 1229469 (0.0008) [2023-12-27 14:10:02,021][20381] Updated weights for policy 1, policy_version 1228381 (0.0009) [2023-12-27 14:10:02,082][20381] Updated weights for policy 1, policy_version 1228391 (0.0009) [2023-12-27 14:10:02,136][20381] Updated weights for policy 1, policy_version 1228401 (0.0011) [2023-12-27 14:10:02,222][20347] Updated weights for policy 0, policy_version 1229479 (0.0007) [2023-12-27 14:10:02,290][20347] Updated weights for policy 0, policy_version 1229489 (0.0007) [2023-12-27 14:10:02,356][20347] Updated weights for policy 0, policy_version 1229499 (0.0006) [2023-12-27 14:10:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 629301248. Throughput: 0: 10153.6, 1: 10342.0. Samples: 629314168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:02,363][19170] Avg episode reward: [(0, '863.060'), (1, '1000.000')] [2023-12-27 14:10:02,834][20381] Updated weights for policy 1, policy_version 1228411 (0.0010) [2023-12-27 14:10:02,880][20381] Updated weights for policy 1, policy_version 1228421 (0.0007) [2023-12-27 14:10:02,928][20381] Updated weights for policy 1, policy_version 1228431 (0.0005) [2023-12-27 14:10:03,037][20347] Updated weights for policy 0, policy_version 1229509 (0.0007) [2023-12-27 14:10:03,074][20291] KL-divergence is very high: 147.0614 [2023-12-27 14:10:03,087][20347] Updated weights for policy 0, policy_version 1229519 (0.0005) [2023-12-27 14:10:03,098][20291] KL-divergence is very high: 180.7692 [2023-12-27 14:10:03,112][20291] KL-divergence is very high: 176.6916 [2023-12-27 14:10:03,138][20291] KL-divergence is very high: 113.7411 [2023-12-27 14:10:03,139][20347] Updated weights for policy 0, policy_version 1229529 (0.0006) [2023-12-27 14:10:03,528][20381] Updated weights for policy 1, policy_version 1228441 (0.0006) [2023-12-27 14:10:03,585][20381] Updated weights for policy 1, policy_version 1228451 (0.0005) [2023-12-27 14:10:03,643][20381] Updated weights for policy 1, policy_version 1228461 (0.0010) [2023-12-27 14:10:03,674][20347] Updated weights for policy 0, policy_version 1229539 (0.0006) [2023-12-27 14:10:03,695][20381] Updated weights for policy 1, policy_version 1228471 (0.0010) [2023-12-27 14:10:03,719][20347] Updated weights for policy 0, policy_version 1229549 (0.0006) [2023-12-27 14:10:03,765][20347] Updated weights for policy 0, policy_version 1229559 (0.0007) [2023-12-27 14:10:04,354][20381] Updated weights for policy 1, policy_version 1228481 (0.0009) [2023-12-27 14:10:04,412][20381] Updated weights for policy 1, policy_version 1228491 (0.0007) [2023-12-27 14:10:04,467][20381] Updated weights for policy 1, policy_version 1228501 (0.0007) [2023-12-27 14:10:04,478][20347] Updated weights for policy 0, policy_version 1229570 (0.0010) [2023-12-27 14:10:04,538][20347] Updated weights for policy 0, policy_version 1229580 (0.0009) [2023-12-27 14:10:04,586][20347] Updated weights for policy 0, policy_version 1229590 (0.0010) [2023-12-27 14:10:04,641][20347] Updated weights for policy 0, policy_version 1229600 (0.0010) [2023-12-27 14:10:05,079][20381] Updated weights for policy 1, policy_version 1228511 (0.0005) [2023-12-27 14:10:05,125][20381] Updated weights for policy 1, policy_version 1228521 (0.0005) [2023-12-27 14:10:05,185][20381] Updated weights for policy 1, policy_version 1228531 (0.0006) [2023-12-27 14:10:05,261][20347] Updated weights for policy 0, policy_version 1229610 (0.0010) [2023-12-27 14:10:05,318][20347] Updated weights for policy 0, policy_version 1229620 (0.0010) [2023-12-27 14:10:05,381][20347] Updated weights for policy 0, policy_version 1229630 (0.0008) [2023-12-27 14:10:05,806][20381] Updated weights for policy 1, policy_version 1228541 (0.0006) [2023-12-27 14:10:05,859][20381] Updated weights for policy 1, policy_version 1228551 (0.0005) [2023-12-27 14:10:05,913][20381] Updated weights for policy 1, policy_version 1228561 (0.0007) [2023-12-27 14:10:05,980][20347] Updated weights for policy 0, policy_version 1229640 (0.0006) [2023-12-27 14:10:06,028][20347] Updated weights for policy 0, policy_version 1229650 (0.0005) [2023-12-27 14:10:06,079][20347] Updated weights for policy 0, policy_version 1229660 (0.0005) [2023-12-27 14:10:06,599][20381] Updated weights for policy 1, policy_version 1228571 (0.0007) [2023-12-27 14:10:06,661][20381] Updated weights for policy 1, policy_version 1228581 (0.0010) [2023-12-27 14:10:06,719][20381] Updated weights for policy 1, policy_version 1228591 (0.0010) [2023-12-27 14:10:06,746][20347] Updated weights for policy 0, policy_version 1229670 (0.0010) [2023-12-27 14:10:06,794][20347] Updated weights for policy 0, policy_version 1229680 (0.0010) [2023-12-27 14:10:06,852][20347] Updated weights for policy 0, policy_version 1229690 (0.0010) [2023-12-27 14:10:07,350][20381] Updated weights for policy 1, policy_version 1228601 (0.0010) [2023-12-27 14:10:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 629407744. Throughput: 0: 10200.0, 1: 10386.6. Samples: 629378688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:07,363][19170] Avg episode reward: [(0, '853.150'), (1, '951.110')] [2023-12-27 14:10:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001229696_314859520.pth... [2023-12-27 14:10:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001228512_314556416.pth [2023-12-27 14:10:07,400][20381] Updated weights for policy 1, policy_version 1228611 (0.0007) [2023-12-27 14:10:07,449][20381] Updated weights for policy 1, policy_version 1228621 (0.0008) [2023-12-27 14:10:07,504][20381] Updated weights for policy 1, policy_version 1228631 (0.0007) [2023-12-27 14:10:07,507][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001228632_314556416.pth... [2023-12-27 14:10:07,510][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001227416_314245120.pth [2023-12-27 14:10:07,567][20347] Updated weights for policy 0, policy_version 1229700 (0.0010) [2023-12-27 14:10:07,624][20347] Updated weights for policy 0, policy_version 1229710 (0.0010) [2023-12-27 14:10:07,676][20347] Updated weights for policy 0, policy_version 1229720 (0.0010) [2023-12-27 14:10:08,098][20381] Updated weights for policy 1, policy_version 1228641 (0.0005) [2023-12-27 14:10:08,159][20381] Updated weights for policy 1, policy_version 1228651 (0.0005) [2023-12-27 14:10:08,222][20381] Updated weights for policy 1, policy_version 1228661 (0.0006) [2023-12-27 14:10:08,331][20347] Updated weights for policy 0, policy_version 1229730 (0.0010) [2023-12-27 14:10:08,395][20347] Updated weights for policy 0, policy_version 1229740 (0.0010) [2023-12-27 14:10:08,454][20347] Updated weights for policy 0, policy_version 1229750 (0.0010) [2023-12-27 14:10:08,516][20347] Updated weights for policy 0, policy_version 1229760 (0.0010) [2023-12-27 14:10:08,829][20381] Updated weights for policy 1, policy_version 1228671 (0.0007) [2023-12-27 14:10:08,894][20381] Updated weights for policy 1, policy_version 1228681 (0.0007) [2023-12-27 14:10:08,947][20381] Updated weights for policy 1, policy_version 1228691 (0.0009) [2023-12-27 14:10:09,147][20347] Updated weights for policy 0, policy_version 1229770 (0.0010) [2023-12-27 14:10:09,205][20347] Updated weights for policy 0, policy_version 1229780 (0.0010) [2023-12-27 14:10:09,273][20347] Updated weights for policy 0, policy_version 1229790 (0.0011) [2023-12-27 14:10:09,707][20381] Updated weights for policy 1, policy_version 1228701 (0.0008) [2023-12-27 14:10:09,771][20381] Updated weights for policy 1, policy_version 1228711 (0.0009) [2023-12-27 14:10:09,838][20381] Updated weights for policy 1, policy_version 1228721 (0.0009) [2023-12-27 14:10:10,080][20347] Updated weights for policy 0, policy_version 1229800 (0.0007) [2023-12-27 14:10:10,138][20347] Updated weights for policy 0, policy_version 1229810 (0.0007) [2023-12-27 14:10:10,198][20347] Updated weights for policy 0, policy_version 1229820 (0.0008) [2023-12-27 14:10:10,509][20381] Updated weights for policy 1, policy_version 1228731 (0.0007) [2023-12-27 14:10:10,570][20381] Updated weights for policy 1, policy_version 1228741 (0.0006) [2023-12-27 14:10:10,626][20381] Updated weights for policy 1, policy_version 1228751 (0.0005) [2023-12-27 14:10:10,876][20347] Updated weights for policy 0, policy_version 1229830 (0.0009) [2023-12-27 14:10:10,942][20347] Updated weights for policy 0, policy_version 1229840 (0.0009) [2023-12-27 14:10:10,994][20347] Updated weights for policy 0, policy_version 1229850 (0.0007) [2023-12-27 14:10:11,306][20381] Updated weights for policy 1, policy_version 1228761 (0.0007) [2023-12-27 14:10:11,368][20381] Updated weights for policy 1, policy_version 1228771 (0.0008) [2023-12-27 14:10:11,436][20381] Updated weights for policy 1, policy_version 1228781 (0.0007) [2023-12-27 14:10:11,495][20381] Updated weights for policy 1, policy_version 1228791 (0.0006) [2023-12-27 14:10:11,763][20347] Updated weights for policy 0, policy_version 1229860 (0.0007) [2023-12-27 14:10:11,821][20347] Updated weights for policy 0, policy_version 1229870 (0.0008) [2023-12-27 14:10:11,884][20347] Updated weights for policy 0, policy_version 1229880 (0.0009) [2023-12-27 14:10:12,177][20381] Updated weights for policy 1, policy_version 1228801 (0.0005) [2023-12-27 14:10:12,232][20381] Updated weights for policy 1, policy_version 1228811 (0.0007) [2023-12-27 14:10:12,292][20381] Updated weights for policy 1, policy_version 1228821 (0.0007) [2023-12-27 14:10:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 629514240. Throughput: 0: 10230.0, 1: 10426.4. Samples: 629503084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:12,363][19170] Avg episode reward: [(0, '921.630'), (1, '912.040')] [2023-12-27 14:10:12,549][20347] Updated weights for policy 0, policy_version 1229890 (0.0008) [2023-12-27 14:10:12,606][20347] Updated weights for policy 0, policy_version 1229900 (0.0006) [2023-12-27 14:10:12,665][20347] Updated weights for policy 0, policy_version 1229910 (0.0006) [2023-12-27 14:10:12,714][20347] Updated weights for policy 0, policy_version 1229920 (0.0006) [2023-12-27 14:10:12,922][20381] Updated weights for policy 1, policy_version 1228831 (0.0006) [2023-12-27 14:10:12,979][20381] Updated weights for policy 1, policy_version 1228841 (0.0006) [2023-12-27 14:10:13,028][20381] Updated weights for policy 1, policy_version 1228851 (0.0006) [2023-12-27 14:10:13,340][20347] Updated weights for policy 0, policy_version 1229930 (0.0007) [2023-12-27 14:10:13,385][20347] Updated weights for policy 0, policy_version 1229940 (0.0008) [2023-12-27 14:10:13,431][20347] Updated weights for policy 0, policy_version 1229950 (0.0008) [2023-12-27 14:10:13,712][20381] Updated weights for policy 1, policy_version 1228861 (0.0008) [2023-12-27 14:10:13,763][20381] Updated weights for policy 1, policy_version 1228871 (0.0007) [2023-12-27 14:10:13,810][20381] Updated weights for policy 1, policy_version 1228881 (0.0007) [2023-12-27 14:10:14,156][20347] Updated weights for policy 0, policy_version 1229960 (0.0007) [2023-12-27 14:10:14,215][20347] Updated weights for policy 0, policy_version 1229970 (0.0007) [2023-12-27 14:10:14,275][20347] Updated weights for policy 0, policy_version 1229980 (0.0008) [2023-12-27 14:10:14,466][20381] Updated weights for policy 1, policy_version 1228891 (0.0007) [2023-12-27 14:10:14,524][20381] Updated weights for policy 1, policy_version 1228901 (0.0007) [2023-12-27 14:10:14,578][20381] Updated weights for policy 1, policy_version 1228911 (0.0008) [2023-12-27 14:10:14,963][20347] Updated weights for policy 0, policy_version 1229990 (0.0008) [2023-12-27 14:10:15,021][20347] Updated weights for policy 0, policy_version 1230000 (0.0008) [2023-12-27 14:10:15,078][20347] Updated weights for policy 0, policy_version 1230010 (0.0008) [2023-12-27 14:10:15,256][20381] Updated weights for policy 1, policy_version 1228921 (0.0008) [2023-12-27 14:10:15,318][20381] Updated weights for policy 1, policy_version 1228931 (0.0007) [2023-12-27 14:10:15,383][20381] Updated weights for policy 1, policy_version 1228941 (0.0008) [2023-12-27 14:10:15,438][20381] Updated weights for policy 1, policy_version 1228951 (0.0007) [2023-12-27 14:10:15,799][20347] Updated weights for policy 0, policy_version 1230020 (0.0007) [2023-12-27 14:10:15,843][20347] Updated weights for policy 0, policy_version 1230030 (0.0007) [2023-12-27 14:10:15,904][20347] Updated weights for policy 0, policy_version 1230040 (0.0007) [2023-12-27 14:10:16,076][20381] Updated weights for policy 1, policy_version 1228961 (0.0010) [2023-12-27 14:10:16,131][20381] Updated weights for policy 1, policy_version 1228971 (0.0010) [2023-12-27 14:10:16,195][20381] Updated weights for policy 1, policy_version 1228981 (0.0006) [2023-12-27 14:10:16,634][20347] Updated weights for policy 0, policy_version 1230050 (0.0009) [2023-12-27 14:10:16,678][20347] Updated weights for policy 0, policy_version 1230060 (0.0010) [2023-12-27 14:10:16,726][20347] Updated weights for policy 0, policy_version 1230070 (0.0010) [2023-12-27 14:10:16,774][20347] Updated weights for policy 0, policy_version 1230080 (0.0010) [2023-12-27 14:10:16,809][20381] Updated weights for policy 1, policy_version 1228991 (0.0007) [2023-12-27 14:10:16,857][20381] Updated weights for policy 1, policy_version 1229001 (0.0007) [2023-12-27 14:10:16,902][20381] Updated weights for policy 1, policy_version 1229011 (0.0007) [2023-12-27 14:10:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 629612544. Throughput: 0: 10215.5, 1: 10387.3. Samples: 629624780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:17,363][19170] Avg episode reward: [(0, '931.540'), (1, '931.660')] [2023-12-27 14:10:17,483][20347] Updated weights for policy 0, policy_version 1230090 (0.0010) [2023-12-27 14:10:17,534][20347] Updated weights for policy 0, policy_version 1230100 (0.0010) [2023-12-27 14:10:17,579][20347] Updated weights for policy 0, policy_version 1230110 (0.0010) [2023-12-27 14:10:17,652][20381] Updated weights for policy 1, policy_version 1229021 (0.0006) [2023-12-27 14:10:17,706][20381] Updated weights for policy 1, policy_version 1229031 (0.0005) [2023-12-27 14:10:17,755][20381] Updated weights for policy 1, policy_version 1229041 (0.0005) [2023-12-27 14:10:18,260][20347] Updated weights for policy 0, policy_version 1230120 (0.0008) [2023-12-27 14:10:18,309][20347] Updated weights for policy 0, policy_version 1230130 (0.0005) [2023-12-27 14:10:18,345][20381] Updated weights for policy 1, policy_version 1229051 (0.0006) [2023-12-27 14:10:18,376][20347] Updated weights for policy 0, policy_version 1230140 (0.0007) [2023-12-27 14:10:18,403][20381] Updated weights for policy 1, policy_version 1229061 (0.0007) [2023-12-27 14:10:18,464][20381] Updated weights for policy 1, policy_version 1229071 (0.0006) [2023-12-27 14:10:19,015][20347] Updated weights for policy 0, policy_version 1230150 (0.0006) [2023-12-27 14:10:19,037][20381] Updated weights for policy 1, policy_version 1229081 (0.0008) [2023-12-27 14:10:19,065][20347] Updated weights for policy 0, policy_version 1230160 (0.0005) [2023-12-27 14:10:19,094][20381] Updated weights for policy 1, policy_version 1229091 (0.0008) [2023-12-27 14:10:19,119][20347] Updated weights for policy 0, policy_version 1230170 (0.0005) [2023-12-27 14:10:19,154][20381] Updated weights for policy 1, policy_version 1229101 (0.0007) [2023-12-27 14:10:19,220][20381] Updated weights for policy 1, policy_version 1229111 (0.0006) [2023-12-27 14:10:19,814][20347] Updated weights for policy 0, policy_version 1230180 (0.0006) [2023-12-27 14:10:19,821][20381] Updated weights for policy 1, policy_version 1229121 (0.0006) [2023-12-27 14:10:19,878][20347] Updated weights for policy 0, policy_version 1230190 (0.0007) [2023-12-27 14:10:19,886][20381] Updated weights for policy 1, policy_version 1229131 (0.0008) [2023-12-27 14:10:19,938][20347] Updated weights for policy 0, policy_version 1230200 (0.0007) [2023-12-27 14:10:19,952][20381] Updated weights for policy 1, policy_version 1229141 (0.0009) [2023-12-27 14:10:20,592][20347] Updated weights for policy 0, policy_version 1230210 (0.0010) [2023-12-27 14:10:20,653][20347] Updated weights for policy 0, policy_version 1230220 (0.0009) [2023-12-27 14:10:20,678][20381] Updated weights for policy 1, policy_version 1229151 (0.0008) [2023-12-27 14:10:20,712][20347] Updated weights for policy 0, policy_version 1230230 (0.0009) [2023-12-27 14:10:20,734][20381] Updated weights for policy 1, policy_version 1229161 (0.0008) [2023-12-27 14:10:20,771][20347] Updated weights for policy 0, policy_version 1230240 (0.0010) [2023-12-27 14:10:20,793][20381] Updated weights for policy 1, policy_version 1229171 (0.0008) [2023-12-27 14:10:21,500][20381] Updated weights for policy 1, policy_version 1229181 (0.0009) [2023-12-27 14:10:21,512][20347] Updated weights for policy 0, policy_version 1230250 (0.0008) [2023-12-27 14:10:21,553][20381] Updated weights for policy 1, policy_version 1229191 (0.0011) [2023-12-27 14:10:21,566][20347] Updated weights for policy 0, policy_version 1230260 (0.0010) [2023-12-27 14:10:21,619][20381] Updated weights for policy 1, policy_version 1229201 (0.0012) [2023-12-27 14:10:21,630][20347] Updated weights for policy 0, policy_version 1230270 (0.0007) [2023-12-27 14:10:22,309][20347] Updated weights for policy 0, policy_version 1230280 (0.0009) [2023-12-27 14:10:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 629710848. Throughput: 0: 10165.5, 1: 10418.5. Samples: 629687784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:22,363][19170] Avg episode reward: [(0, '902.320'), (1, '912.210')] [2023-12-27 14:10:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001229208_314703872.pth... [2023-12-27 14:10:22,375][20347] Updated weights for policy 0, policy_version 1230290 (0.0008) [2023-12-27 14:10:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001227992_314392576.pth [2023-12-27 14:10:22,427][20381] Updated weights for policy 1, policy_version 1229211 (0.0008) [2023-12-27 14:10:22,443][20347] Updated weights for policy 0, policy_version 1230300 (0.0006) [2023-12-27 14:10:22,468][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001230304_315015168.pth... [2023-12-27 14:10:22,472][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001229120_314712064.pth [2023-12-27 14:10:22,479][20381] Updated weights for policy 1, policy_version 1229221 (0.0007) [2023-12-27 14:10:22,544][20381] Updated weights for policy 1, policy_version 1229231 (0.0007) [2023-12-27 14:10:23,134][20347] Updated weights for policy 0, policy_version 1230310 (0.0009) [2023-12-27 14:10:23,190][20347] Updated weights for policy 0, policy_version 1230320 (0.0010) [2023-12-27 14:10:23,214][20381] Updated weights for policy 1, policy_version 1229241 (0.0006) [2023-12-27 14:10:23,244][20347] Updated weights for policy 0, policy_version 1230330 (0.0008) [2023-12-27 14:10:23,272][20381] Updated weights for policy 1, policy_version 1229251 (0.0011) [2023-12-27 14:10:23,337][20381] Updated weights for policy 1, policy_version 1229261 (0.0011) [2023-12-27 14:10:23,402][20381] Updated weights for policy 1, policy_version 1229271 (0.0011) [2023-12-27 14:10:23,846][20347] Updated weights for policy 0, policy_version 1230340 (0.0007) [2023-12-27 14:10:23,904][20347] Updated weights for policy 0, policy_version 1230350 (0.0010) [2023-12-27 14:10:23,965][20347] Updated weights for policy 0, policy_version 1230360 (0.0008) [2023-12-27 14:10:24,074][20381] Updated weights for policy 1, policy_version 1229281 (0.0007) [2023-12-27 14:10:24,128][20381] Updated weights for policy 1, policy_version 1229291 (0.0011) [2023-12-27 14:10:24,184][20381] Updated weights for policy 1, policy_version 1229301 (0.0010) [2023-12-27 14:10:24,583][20347] Updated weights for policy 0, policy_version 1230370 (0.0006) [2023-12-27 14:10:24,641][20347] Updated weights for policy 0, policy_version 1230380 (0.0010) [2023-12-27 14:10:24,699][20347] Updated weights for policy 0, policy_version 1230390 (0.0007) [2023-12-27 14:10:24,749][20347] Updated weights for policy 0, policy_version 1230400 (0.0007) [2023-12-27 14:10:24,872][20381] Updated weights for policy 1, policy_version 1229311 (0.0010) [2023-12-27 14:10:24,921][20381] Updated weights for policy 1, policy_version 1229321 (0.0010) [2023-12-27 14:10:24,970][20381] Updated weights for policy 1, policy_version 1229331 (0.0010) [2023-12-27 14:10:25,411][20347] Updated weights for policy 0, policy_version 1230410 (0.0005) [2023-12-27 14:10:25,464][20347] Updated weights for policy 0, policy_version 1230420 (0.0007) [2023-12-27 14:10:25,513][20347] Updated weights for policy 0, policy_version 1230430 (0.0005) [2023-12-27 14:10:25,671][20381] Updated weights for policy 1, policy_version 1229341 (0.0009) [2023-12-27 14:10:25,721][20381] Updated weights for policy 1, policy_version 1229351 (0.0005) [2023-12-27 14:10:25,784][20381] Updated weights for policy 1, policy_version 1229361 (0.0008) [2023-12-27 14:10:26,067][20347] Updated weights for policy 0, policy_version 1230440 (0.0009) [2023-12-27 14:10:26,126][20347] Updated weights for policy 0, policy_version 1230450 (0.0010) [2023-12-27 14:10:26,188][20347] Updated weights for policy 0, policy_version 1230460 (0.0010) [2023-12-27 14:10:26,430][20381] Updated weights for policy 1, policy_version 1229371 (0.0009) [2023-12-27 14:10:26,492][20381] Updated weights for policy 1, policy_version 1229381 (0.0006) [2023-12-27 14:10:26,544][20381] Updated weights for policy 1, policy_version 1229391 (0.0005) [2023-12-27 14:10:26,856][20347] Updated weights for policy 0, policy_version 1230470 (0.0010) [2023-12-27 14:10:26,904][20347] Updated weights for policy 0, policy_version 1230480 (0.0010) [2023-12-27 14:10:26,960][20347] Updated weights for policy 0, policy_version 1230490 (0.0010) [2023-12-27 14:10:27,113][20381] Updated weights for policy 1, policy_version 1229401 (0.0005) [2023-12-27 14:10:27,164][20381] Updated weights for policy 1, policy_version 1229411 (0.0008) [2023-12-27 14:10:27,207][20381] Updated weights for policy 1, policy_version 1229421 (0.0007) [2023-12-27 14:10:27,251][20381] Updated weights for policy 1, policy_version 1229431 (0.0007) [2023-12-27 14:10:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 629825536. Throughput: 0: 10194.2, 1: 10382.8. Samples: 629810480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:27,363][19170] Avg episode reward: [(0, '931.750'), (1, '912.210')] [2023-12-27 14:10:27,598][20347] Updated weights for policy 0, policy_version 1230500 (0.0009) [2023-12-27 14:10:27,652][20347] Updated weights for policy 0, policy_version 1230510 (0.0007) [2023-12-27 14:10:27,710][20347] Updated weights for policy 0, policy_version 1230520 (0.0010) [2023-12-27 14:10:27,954][20381] Updated weights for policy 1, policy_version 1229441 (0.0007) [2023-12-27 14:10:28,006][20381] Updated weights for policy 1, policy_version 1229451 (0.0007) [2023-12-27 14:10:28,055][20381] Updated weights for policy 1, policy_version 1229461 (0.0008) [2023-12-27 14:10:28,339][20347] Updated weights for policy 0, policy_version 1230530 (0.0010) [2023-12-27 14:10:28,404][20347] Updated weights for policy 0, policy_version 1230540 (0.0008) [2023-12-27 14:10:28,461][20347] Updated weights for policy 0, policy_version 1230550 (0.0008) [2023-12-27 14:10:28,512][20347] Updated weights for policy 0, policy_version 1230560 (0.0008) [2023-12-27 14:10:28,778][20381] Updated weights for policy 1, policy_version 1229471 (0.0008) [2023-12-27 14:10:28,835][20381] Updated weights for policy 1, policy_version 1229481 (0.0008) [2023-12-27 14:10:28,888][20381] Updated weights for policy 1, policy_version 1229491 (0.0008) [2023-12-27 14:10:29,150][20347] Updated weights for policy 0, policy_version 1230570 (0.0006) [2023-12-27 14:10:29,200][20347] Updated weights for policy 0, policy_version 1230580 (0.0006) [2023-12-27 14:10:29,264][20347] Updated weights for policy 0, policy_version 1230590 (0.0007) [2023-12-27 14:10:29,636][20381] Updated weights for policy 1, policy_version 1229501 (0.0007) [2023-12-27 14:10:29,703][20381] Updated weights for policy 1, policy_version 1229511 (0.0008) [2023-12-27 14:10:29,758][20381] Updated weights for policy 1, policy_version 1229521 (0.0008) [2023-12-27 14:10:29,883][20347] Updated weights for policy 0, policy_version 1230600 (0.0007) [2023-12-27 14:10:29,942][20347] Updated weights for policy 0, policy_version 1230610 (0.0009) [2023-12-27 14:10:29,999][20347] Updated weights for policy 0, policy_version 1230620 (0.0009) [2023-12-27 14:10:30,477][20381] Updated weights for policy 1, policy_version 1229531 (0.0007) [2023-12-27 14:10:30,529][20381] Updated weights for policy 1, policy_version 1229541 (0.0007) [2023-12-27 14:10:30,582][20381] Updated weights for policy 1, policy_version 1229551 (0.0007) [2023-12-27 14:10:30,650][20347] Updated weights for policy 0, policy_version 1230630 (0.0006) [2023-12-27 14:10:30,702][20347] Updated weights for policy 0, policy_version 1230640 (0.0008) [2023-12-27 14:10:30,734][20291] KL-divergence is very high: 190.3606 [2023-12-27 14:10:30,757][20347] Updated weights for policy 0, policy_version 1230650 (0.0005) [2023-12-27 14:10:30,773][20291] KL-divergence is very high: 636.1457 [2023-12-27 14:10:31,235][20381] Updated weights for policy 1, policy_version 1229561 (0.0007) [2023-12-27 14:10:31,288][20381] Updated weights for policy 1, policy_version 1229571 (0.0008) [2023-12-27 14:10:31,341][20381] Updated weights for policy 1, policy_version 1229581 (0.0008) [2023-12-27 14:10:31,408][20381] Updated weights for policy 1, policy_version 1229591 (0.0008) [2023-12-27 14:10:31,446][20347] Updated weights for policy 0, policy_version 1230660 (0.0007) [2023-12-27 14:10:31,496][20347] Updated weights for policy 0, policy_version 1230670 (0.0007) [2023-12-27 14:10:31,552][20347] Updated weights for policy 0, policy_version 1230680 (0.0007) [2023-12-27 14:10:31,560][20291] KL-divergence is very high: 270.7071 [2023-12-27 14:10:32,172][20381] Updated weights for policy 1, policy_version 1229601 (0.0007) [2023-12-27 14:10:32,228][20381] Updated weights for policy 1, policy_version 1229611 (0.0006) [2023-12-27 14:10:32,264][20347] Updated weights for policy 0, policy_version 1230690 (0.0009) [2023-12-27 14:10:32,288][20381] Updated weights for policy 1, policy_version 1229621 (0.0008) [2023-12-27 14:10:32,325][20347] Updated weights for policy 0, policy_version 1230700 (0.0010) [2023-12-27 14:10:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 629923840. Throughput: 0: 10198.7, 1: 10317.1. Samples: 629933596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:32,363][19170] Avg episode reward: [(0, '941.500'), (1, '951.320')] [2023-12-27 14:10:32,385][20347] Updated weights for policy 0, policy_version 1230710 (0.0011) [2023-12-27 14:10:32,450][20347] Updated weights for policy 0, policy_version 1230720 (0.0006) [2023-12-27 14:10:33,012][20381] Updated weights for policy 1, policy_version 1229631 (0.0011) [2023-12-27 14:10:33,073][20381] Updated weights for policy 1, policy_version 1229641 (0.0011) [2023-12-27 14:10:33,097][20347] Updated weights for policy 0, policy_version 1230730 (0.0006) [2023-12-27 14:10:33,131][20381] Updated weights for policy 1, policy_version 1229651 (0.0011) [2023-12-27 14:10:33,154][20347] Updated weights for policy 0, policy_version 1230740 (0.0005) [2023-12-27 14:10:33,208][20347] Updated weights for policy 0, policy_version 1230750 (0.0005) [2023-12-27 14:10:33,803][20347] Updated weights for policy 0, policy_version 1230760 (0.0007) [2023-12-27 14:10:33,825][20381] Updated weights for policy 1, policy_version 1229661 (0.0008) [2023-12-27 14:10:33,858][20347] Updated weights for policy 0, policy_version 1230770 (0.0007) [2023-12-27 14:10:33,882][20381] Updated weights for policy 1, policy_version 1229671 (0.0005) [2023-12-27 14:10:33,907][20347] Updated weights for policy 0, policy_version 1230780 (0.0008) [2023-12-27 14:10:33,929][20381] Updated weights for policy 1, policy_version 1229681 (0.0006) [2023-12-27 14:10:34,565][20347] Updated weights for policy 0, policy_version 1230790 (0.0009) [2023-12-27 14:10:34,581][20381] Updated weights for policy 1, policy_version 1229691 (0.0011) [2023-12-27 14:10:34,625][20347] Updated weights for policy 0, policy_version 1230800 (0.0010) [2023-12-27 14:10:34,643][20381] Updated weights for policy 1, policy_version 1229701 (0.0009) [2023-12-27 14:10:34,677][20347] Updated weights for policy 0, policy_version 1230810 (0.0005) [2023-12-27 14:10:34,702][20381] Updated weights for policy 1, policy_version 1229711 (0.0008) [2023-12-27 14:10:35,270][20347] Updated weights for policy 0, policy_version 1230820 (0.0007) [2023-12-27 14:10:35,320][20347] Updated weights for policy 0, policy_version 1230830 (0.0010) [2023-12-27 14:10:35,371][20347] Updated weights for policy 0, policy_version 1230840 (0.0010) [2023-12-27 14:10:35,394][20381] Updated weights for policy 1, policy_version 1229721 (0.0007) [2023-12-27 14:10:35,446][20381] Updated weights for policy 1, policy_version 1229731 (0.0007) [2023-12-27 14:10:35,502][20381] Updated weights for policy 1, policy_version 1229741 (0.0005) [2023-12-27 14:10:35,555][20381] Updated weights for policy 1, policy_version 1229751 (0.0008) [2023-12-27 14:10:35,976][20347] Updated weights for policy 0, policy_version 1230850 (0.0010) [2023-12-27 14:10:36,024][20347] Updated weights for policy 0, policy_version 1230860 (0.0010) [2023-12-27 14:10:36,076][20347] Updated weights for policy 0, policy_version 1230870 (0.0010) [2023-12-27 14:10:36,140][20347] Updated weights for policy 0, policy_version 1230880 (0.0010) [2023-12-27 14:10:36,270][20381] Updated weights for policy 1, policy_version 1229761 (0.0010) [2023-12-27 14:10:36,333][20381] Updated weights for policy 1, policy_version 1229771 (0.0008) [2023-12-27 14:10:36,396][20381] Updated weights for policy 1, policy_version 1229781 (0.0011) [2023-12-27 14:10:36,843][20347] Updated weights for policy 0, policy_version 1230890 (0.0011) [2023-12-27 14:10:36,902][20347] Updated weights for policy 0, policy_version 1230900 (0.0011) [2023-12-27 14:10:36,961][20347] Updated weights for policy 0, policy_version 1230910 (0.0009) [2023-12-27 14:10:37,012][20381] Updated weights for policy 1, policy_version 1229791 (0.0011) [2023-12-27 14:10:37,072][20381] Updated weights for policy 1, policy_version 1229801 (0.0010) [2023-12-27 14:10:37,137][20381] Updated weights for policy 1, policy_version 1229811 (0.0011) [2023-12-27 14:10:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 630030336. Throughput: 0: 10252.1, 1: 10290.0. Samples: 629996264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:37,364][19170] Avg episode reward: [(0, '932.160'), (1, '951.240')] [2023-12-27 14:10:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001230912_315170816.pth... [2023-12-27 14:10:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001229816_314859520.pth... [2023-12-27 14:10:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001229696_314859520.pth [2023-12-27 14:10:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001228632_314556416.pth [2023-12-27 14:10:37,557][20347] Updated weights for policy 0, policy_version 1230920 (0.0009) [2023-12-27 14:10:37,603][20347] Updated weights for policy 0, policy_version 1230930 (0.0010) [2023-12-27 14:10:37,659][20347] Updated weights for policy 0, policy_version 1230940 (0.0010) [2023-12-27 14:10:37,787][20381] Updated weights for policy 1, policy_version 1229821 (0.0008) [2023-12-27 14:10:37,833][20381] Updated weights for policy 1, policy_version 1229831 (0.0005) [2023-12-27 14:10:37,886][20381] Updated weights for policy 1, policy_version 1229841 (0.0010) [2023-12-27 14:10:38,328][20347] Updated weights for policy 0, policy_version 1230950 (0.0009) [2023-12-27 14:10:38,390][20347] Updated weights for policy 0, policy_version 1230960 (0.0007) [2023-12-27 14:10:38,451][20347] Updated weights for policy 0, policy_version 1230970 (0.0008) [2023-12-27 14:10:38,563][20381] Updated weights for policy 1, policy_version 1229851 (0.0009) [2023-12-27 14:10:38,622][20381] Updated weights for policy 1, policy_version 1229861 (0.0006) [2023-12-27 14:10:38,674][20381] Updated weights for policy 1, policy_version 1229871 (0.0006) [2023-12-27 14:10:39,135][20347] Updated weights for policy 0, policy_version 1230980 (0.0007) [2023-12-27 14:10:39,187][20347] Updated weights for policy 0, policy_version 1230990 (0.0009) [2023-12-27 14:10:39,243][20347] Updated weights for policy 0, policy_version 1231000 (0.0009) [2023-12-27 14:10:39,329][20381] Updated weights for policy 1, policy_version 1229881 (0.0006) [2023-12-27 14:10:39,391][20381] Updated weights for policy 1, policy_version 1229891 (0.0008) [2023-12-27 14:10:39,456][20381] Updated weights for policy 1, policy_version 1229901 (0.0008) [2023-12-27 14:10:39,515][20381] Updated weights for policy 1, policy_version 1229911 (0.0008) [2023-12-27 14:10:39,992][20347] Updated weights for policy 0, policy_version 1231010 (0.0008) [2023-12-27 14:10:40,059][20347] Updated weights for policy 0, policy_version 1231020 (0.0007) [2023-12-27 14:10:40,120][20347] Updated weights for policy 0, policy_version 1231030 (0.0008) [2023-12-27 14:10:40,176][20347] Updated weights for policy 0, policy_version 1231040 (0.0006) [2023-12-27 14:10:40,189][20381] Updated weights for policy 1, policy_version 1229921 (0.0007) [2023-12-27 14:10:40,240][20381] Updated weights for policy 1, policy_version 1229931 (0.0008) [2023-12-27 14:10:40,300][20381] Updated weights for policy 1, policy_version 1229941 (0.0008) [2023-12-27 14:10:40,869][20347] Updated weights for policy 0, policy_version 1231050 (0.0007) [2023-12-27 14:10:40,924][20347] Updated weights for policy 0, policy_version 1231060 (0.0010) [2023-12-27 14:10:40,973][20347] Updated weights for policy 0, policy_version 1231070 (0.0010) [2023-12-27 14:10:41,015][20381] Updated weights for policy 1, policy_version 1229951 (0.0007) [2023-12-27 14:10:41,077][20381] Updated weights for policy 1, policy_version 1229961 (0.0006) [2023-12-27 14:10:41,140][20381] Updated weights for policy 1, policy_version 1229971 (0.0008) [2023-12-27 14:10:41,753][20347] Updated weights for policy 0, policy_version 1231080 (0.0007) [2023-12-27 14:10:41,820][20347] Updated weights for policy 0, policy_version 1231090 (0.0007) [2023-12-27 14:10:41,837][20381] Updated weights for policy 1, policy_version 1229981 (0.0007) [2023-12-27 14:10:41,879][20347] Updated weights for policy 0, policy_version 1231100 (0.0007) [2023-12-27 14:10:41,898][20381] Updated weights for policy 1, policy_version 1229991 (0.0006) [2023-12-27 14:10:41,962][20381] Updated weights for policy 1, policy_version 1230001 (0.0006) [2023-12-27 14:10:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 630128640. Throughput: 0: 10271.9, 1: 10291.0. Samples: 630119644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:42,363][19170] Avg episode reward: [(0, '873.390'), (1, '922.020')] [2023-12-27 14:10:42,586][20381] Updated weights for policy 1, policy_version 1230011 (0.0006) [2023-12-27 14:10:42,597][20347] Updated weights for policy 0, policy_version 1231110 (0.0007) [2023-12-27 14:10:42,652][20381] Updated weights for policy 1, policy_version 1230021 (0.0009) [2023-12-27 14:10:42,658][20347] Updated weights for policy 0, policy_version 1231120 (0.0006) [2023-12-27 14:10:42,719][20347] Updated weights for policy 0, policy_version 1231130 (0.0006) [2023-12-27 14:10:42,719][20381] Updated weights for policy 1, policy_version 1230031 (0.0009) [2023-12-27 14:10:43,298][20381] Updated weights for policy 1, policy_version 1230041 (0.0008) [2023-12-27 14:10:43,310][20347] Updated weights for policy 0, policy_version 1231140 (0.0007) [2023-12-27 14:10:43,343][20381] Updated weights for policy 1, policy_version 1230051 (0.0005) [2023-12-27 14:10:43,360][20347] Updated weights for policy 0, policy_version 1231150 (0.0006) [2023-12-27 14:10:43,390][20381] Updated weights for policy 1, policy_version 1230061 (0.0006) [2023-12-27 14:10:43,418][20347] Updated weights for policy 0, policy_version 1231160 (0.0008) [2023-12-27 14:10:43,436][20381] Updated weights for policy 1, policy_version 1230071 (0.0006) [2023-12-27 14:10:44,056][20347] Updated weights for policy 0, policy_version 1231170 (0.0008) [2023-12-27 14:10:44,110][20347] Updated weights for policy 0, policy_version 1231180 (0.0007) [2023-12-27 14:10:44,138][20381] Updated weights for policy 1, policy_version 1230081 (0.0007) [2023-12-27 14:10:44,163][20347] Updated weights for policy 0, policy_version 1231190 (0.0005) [2023-12-27 14:10:44,194][20381] Updated weights for policy 1, policy_version 1230091 (0.0007) [2023-12-27 14:10:44,224][20347] Updated weights for policy 0, policy_version 1231200 (0.0005) [2023-12-27 14:10:44,254][20381] Updated weights for policy 1, policy_version 1230101 (0.0008) [2023-12-27 14:10:44,747][20347] Updated weights for policy 0, policy_version 1231210 (0.0008) [2023-12-27 14:10:44,812][20347] Updated weights for policy 0, policy_version 1231220 (0.0007) [2023-12-27 14:10:44,877][20347] Updated weights for policy 0, policy_version 1231230 (0.0007) [2023-12-27 14:10:45,010][20381] Updated weights for policy 1, policy_version 1230111 (0.0008) [2023-12-27 14:10:45,070][20381] Updated weights for policy 1, policy_version 1230121 (0.0008) [2023-12-27 14:10:45,130][20381] Updated weights for policy 1, policy_version 1230131 (0.0008) [2023-12-27 14:10:45,593][20347] Updated weights for policy 0, policy_version 1231240 (0.0007) [2023-12-27 14:10:45,638][20347] Updated weights for policy 0, policy_version 1231250 (0.0005) [2023-12-27 14:10:45,694][20347] Updated weights for policy 0, policy_version 1231260 (0.0007) [2023-12-27 14:10:45,841][20381] Updated weights for policy 1, policy_version 1230141 (0.0007) [2023-12-27 14:10:45,899][20381] Updated weights for policy 1, policy_version 1230151 (0.0007) [2023-12-27 14:10:45,948][20381] Updated weights for policy 1, policy_version 1230161 (0.0008) [2023-12-27 14:10:46,320][20347] Updated weights for policy 0, policy_version 1231270 (0.0007) [2023-12-27 14:10:46,364][20347] Updated weights for policy 0, policy_version 1231280 (0.0007) [2023-12-27 14:10:46,419][20347] Updated weights for policy 0, policy_version 1231290 (0.0005) [2023-12-27 14:10:46,593][20381] Updated weights for policy 1, policy_version 1230171 (0.0007) [2023-12-27 14:10:46,646][20381] Updated weights for policy 1, policy_version 1230181 (0.0005) [2023-12-27 14:10:46,707][20381] Updated weights for policy 1, policy_version 1230191 (0.0005) [2023-12-27 14:10:47,132][20347] Updated weights for policy 0, policy_version 1231300 (0.0007) [2023-12-27 14:10:47,183][20347] Updated weights for policy 0, policy_version 1231310 (0.0008) [2023-12-27 14:10:47,237][20347] Updated weights for policy 0, policy_version 1231320 (0.0007) [2023-12-27 14:10:47,300][20381] Updated weights for policy 1, policy_version 1230201 (0.0005) [2023-12-27 14:10:47,353][20381] Updated weights for policy 1, policy_version 1230211 (0.0007) [2023-12-27 14:10:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 630235136. Throughput: 0: 10374.8, 1: 10305.8. Samples: 630244792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:47,363][19170] Avg episode reward: [(0, '834.150'), (1, '931.790')] [2023-12-27 14:10:47,405][20381] Updated weights for policy 1, policy_version 1230221 (0.0007) [2023-12-27 14:10:47,457][20381] Updated weights for policy 1, policy_version 1230231 (0.0008) [2023-12-27 14:10:47,877][20347] Updated weights for policy 0, policy_version 1231330 (0.0007) [2023-12-27 14:10:47,936][20347] Updated weights for policy 0, policy_version 1231340 (0.0005) [2023-12-27 14:10:47,997][20347] Updated weights for policy 0, policy_version 1231350 (0.0006) [2023-12-27 14:10:48,064][20347] Updated weights for policy 0, policy_version 1231360 (0.0008) [2023-12-27 14:10:48,107][20381] Updated weights for policy 1, policy_version 1230241 (0.0006) [2023-12-27 14:10:48,156][20381] Updated weights for policy 1, policy_version 1230251 (0.0005) [2023-12-27 14:10:48,211][20381] Updated weights for policy 1, policy_version 1230261 (0.0005) [2023-12-27 14:10:48,688][20347] Updated weights for policy 0, policy_version 1231370 (0.0008) [2023-12-27 14:10:48,749][20347] Updated weights for policy 0, policy_version 1231380 (0.0008) [2023-12-27 14:10:48,806][20347] Updated weights for policy 0, policy_version 1231390 (0.0008) [2023-12-27 14:10:48,862][20381] Updated weights for policy 1, policy_version 1230271 (0.0007) [2023-12-27 14:10:48,916][20381] Updated weights for policy 1, policy_version 1230281 (0.0006) [2023-12-27 14:10:48,976][20381] Updated weights for policy 1, policy_version 1230291 (0.0007) [2023-12-27 14:10:49,554][20347] Updated weights for policy 0, policy_version 1231400 (0.0008) [2023-12-27 14:10:49,615][20347] Updated weights for policy 0, policy_version 1231410 (0.0007) [2023-12-27 14:10:49,670][20347] Updated weights for policy 0, policy_version 1231420 (0.0007) [2023-12-27 14:10:49,696][20381] Updated weights for policy 1, policy_version 1230301 (0.0007) [2023-12-27 14:10:49,756][20381] Updated weights for policy 1, policy_version 1230311 (0.0006) [2023-12-27 14:10:49,818][20381] Updated weights for policy 1, policy_version 1230321 (0.0007) [2023-12-27 14:10:50,377][20347] Updated weights for policy 0, policy_version 1231430 (0.0008) [2023-12-27 14:10:50,428][20347] Updated weights for policy 0, policy_version 1231440 (0.0005) [2023-12-27 14:10:50,478][20347] Updated weights for policy 0, policy_version 1231450 (0.0005) [2023-12-27 14:10:50,534][20381] Updated weights for policy 1, policy_version 1230331 (0.0008) [2023-12-27 14:10:50,593][20381] Updated weights for policy 1, policy_version 1230341 (0.0007) [2023-12-27 14:10:50,654][20381] Updated weights for policy 1, policy_version 1230351 (0.0008) [2023-12-27 14:10:51,082][20347] Updated weights for policy 0, policy_version 1231460 (0.0006) [2023-12-27 14:10:51,149][20347] Updated weights for policy 0, policy_version 1231470 (0.0006) [2023-12-27 14:10:51,199][20347] Updated weights for policy 0, policy_version 1231480 (0.0006) [2023-12-27 14:10:51,346][20381] Updated weights for policy 1, policy_version 1230361 (0.0008) [2023-12-27 14:10:51,413][20381] Updated weights for policy 1, policy_version 1230371 (0.0007) [2023-12-27 14:10:51,479][20381] Updated weights for policy 1, policy_version 1230381 (0.0009) [2023-12-27 14:10:51,540][20381] Updated weights for policy 1, policy_version 1230391 (0.0007) [2023-12-27 14:10:51,942][20347] Updated weights for policy 0, policy_version 1231490 (0.0008) [2023-12-27 14:10:51,998][20347] Updated weights for policy 0, policy_version 1231500 (0.0008) [2023-12-27 14:10:52,056][20347] Updated weights for policy 0, policy_version 1231510 (0.0008) [2023-12-27 14:10:52,119][20347] Updated weights for policy 0, policy_version 1231520 (0.0008) [2023-12-27 14:10:52,177][20381] Updated weights for policy 1, policy_version 1230401 (0.0008) [2023-12-27 14:10:52,214][20292] KL-divergence is very high: 264.4767 [2023-12-27 14:10:52,230][20381] Updated weights for policy 1, policy_version 1230411 (0.0007) [2023-12-27 14:10:52,260][20292] KL-divergence is very high: 267.4831 [2023-12-27 14:10:52,291][20381] Updated weights for policy 1, policy_version 1230421 (0.0008) [2023-12-27 14:10:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 630341632. Throughput: 0: 10340.9, 1: 10263.6. Samples: 630305892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:52,363][19170] Avg episode reward: [(0, '951.140'), (1, '970.730')] [2023-12-27 14:10:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001230424_315015168.pth... [2023-12-27 14:10:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001231520_315326464.pth... [2023-12-27 14:10:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001229208_314703872.pth [2023-12-27 14:10:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001230304_315015168.pth [2023-12-27 14:10:52,857][20347] Updated weights for policy 0, policy_version 1231530 (0.0008) [2023-12-27 14:10:52,916][20347] Updated weights for policy 0, policy_version 1231542 (0.0010) [2023-12-27 14:10:52,964][20347] Updated weights for policy 0, policy_version 1231552 (0.0007) [2023-12-27 14:10:52,993][20381] Updated weights for policy 1, policy_version 1230431 (0.0007) [2023-12-27 14:10:53,048][20381] Updated weights for policy 1, policy_version 1230441 (0.0008) [2023-12-27 14:10:53,105][20381] Updated weights for policy 1, policy_version 1230451 (0.0008) [2023-12-27 14:10:53,611][20347] Updated weights for policy 0, policy_version 1231562 (0.0007) [2023-12-27 14:10:53,656][20347] Updated weights for policy 0, policy_version 1231572 (0.0007) [2023-12-27 14:10:53,703][20347] Updated weights for policy 0, policy_version 1231582 (0.0008) [2023-12-27 14:10:53,778][20381] Updated weights for policy 1, policy_version 1230461 (0.0009) [2023-12-27 14:10:53,824][20381] Updated weights for policy 1, policy_version 1230471 (0.0007) [2023-12-27 14:10:53,873][20381] Updated weights for policy 1, policy_version 1230481 (0.0005) [2023-12-27 14:10:54,311][20347] Updated weights for policy 0, policy_version 1231592 (0.0007) [2023-12-27 14:10:54,369][20347] Updated weights for policy 0, policy_version 1231602 (0.0007) [2023-12-27 14:10:54,421][20347] Updated weights for policy 0, policy_version 1231612 (0.0008) [2023-12-27 14:10:54,544][20381] Updated weights for policy 1, policy_version 1230491 (0.0009) [2023-12-27 14:10:54,592][20381] Updated weights for policy 1, policy_version 1230501 (0.0010) [2023-12-27 14:10:54,640][20381] Updated weights for policy 1, policy_version 1230511 (0.0010) [2023-12-27 14:10:55,137][20347] Updated weights for policy 0, policy_version 1231622 (0.0008) [2023-12-27 14:10:55,188][20347] Updated weights for policy 0, policy_version 1231632 (0.0008) [2023-12-27 14:10:55,236][20347] Updated weights for policy 0, policy_version 1231642 (0.0007) [2023-12-27 14:10:55,332][20381] Updated weights for policy 1, policy_version 1230521 (0.0010) [2023-12-27 14:10:55,392][20381] Updated weights for policy 1, policy_version 1230531 (0.0010) [2023-12-27 14:10:55,449][20381] Updated weights for policy 1, policy_version 1230541 (0.0010) [2023-12-27 14:10:55,502][20381] Updated weights for policy 1, policy_version 1230551 (0.0010) [2023-12-27 14:10:55,944][20347] Updated weights for policy 0, policy_version 1231652 (0.0007) [2023-12-27 14:10:55,999][20347] Updated weights for policy 0, policy_version 1231662 (0.0007) [2023-12-27 14:10:56,053][20347] Updated weights for policy 0, policy_version 1231672 (0.0008) [2023-12-27 14:10:56,158][20381] Updated weights for policy 1, policy_version 1230561 (0.0010) [2023-12-27 14:10:56,211][20381] Updated weights for policy 1, policy_version 1230571 (0.0010) [2023-12-27 14:10:56,265][20381] Updated weights for policy 1, policy_version 1230581 (0.0010) [2023-12-27 14:10:56,796][20347] Updated weights for policy 0, policy_version 1231682 (0.0007) [2023-12-27 14:10:56,854][20347] Updated weights for policy 0, policy_version 1231692 (0.0007) [2023-12-27 14:10:56,910][20347] Updated weights for policy 0, policy_version 1231702 (0.0007) [2023-12-27 14:10:56,964][20347] Updated weights for policy 0, policy_version 1231712 (0.0007) [2023-12-27 14:10:56,995][20381] Updated weights for policy 1, policy_version 1230591 (0.0008) [2023-12-27 14:10:57,053][20381] Updated weights for policy 1, policy_version 1230601 (0.0006) [2023-12-27 14:10:57,108][20381] Updated weights for policy 1, policy_version 1230611 (0.0005) [2023-12-27 14:10:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 630439936. Throughput: 0: 10349.6, 1: 10219.7. Samples: 630428700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:10:57,363][19170] Avg episode reward: [(0, '931.940'), (1, '980.480')] [2023-12-27 14:10:57,662][20381] Updated weights for policy 1, policy_version 1230621 (0.0006) [2023-12-27 14:10:57,683][20347] Updated weights for policy 0, policy_version 1231722 (0.0008) [2023-12-27 14:10:57,720][20381] Updated weights for policy 1, policy_version 1230631 (0.0007) [2023-12-27 14:10:57,741][20347] Updated weights for policy 0, policy_version 1231732 (0.0006) [2023-12-27 14:10:57,772][20381] Updated weights for policy 1, policy_version 1230642 (0.0007) [2023-12-27 14:10:57,787][20347] Updated weights for policy 0, policy_version 1231742 (0.0008) [2023-12-27 14:10:58,508][20347] Updated weights for policy 0, policy_version 1231752 (0.0008) [2023-12-27 14:10:58,515][20381] Updated weights for policy 1, policy_version 1230653 (0.0011) [2023-12-27 14:10:58,575][20347] Updated weights for policy 0, policy_version 1231762 (0.0007) [2023-12-27 14:10:58,578][20381] Updated weights for policy 1, policy_version 1230663 (0.0008) [2023-12-27 14:10:58,635][20381] Updated weights for policy 1, policy_version 1230673 (0.0009) [2023-12-27 14:10:58,641][20347] Updated weights for policy 0, policy_version 1231772 (0.0007) [2023-12-27 14:10:59,390][20347] Updated weights for policy 0, policy_version 1231782 (0.0010) [2023-12-27 14:10:59,443][20347] Updated weights for policy 0, policy_version 1231792 (0.0010) [2023-12-27 14:10:59,484][20381] Updated weights for policy 1, policy_version 1230683 (0.0013) [2023-12-27 14:10:59,490][20347] Updated weights for policy 0, policy_version 1231802 (0.0008) [2023-12-27 14:10:59,543][20381] Updated weights for policy 1, policy_version 1230693 (0.0011) [2023-12-27 14:10:59,601][20381] Updated weights for policy 1, policy_version 1230703 (0.0010) [2023-12-27 14:11:00,201][20347] Updated weights for policy 0, policy_version 1231812 (0.0008) [2023-12-27 14:11:00,249][20347] Updated weights for policy 0, policy_version 1231822 (0.0007) [2023-12-27 14:11:00,296][20347] Updated weights for policy 0, policy_version 1231832 (0.0007) [2023-12-27 14:11:00,299][20381] Updated weights for policy 1, policy_version 1230713 (0.0010) [2023-12-27 14:11:00,355][20381] Updated weights for policy 1, policy_version 1230723 (0.0010) [2023-12-27 14:11:00,410][20381] Updated weights for policy 1, policy_version 1230733 (0.0010) [2023-12-27 14:11:00,465][20381] Updated weights for policy 1, policy_version 1230743 (0.0010) [2023-12-27 14:11:00,996][20347] Updated weights for policy 0, policy_version 1231842 (0.0007) [2023-12-27 14:11:01,054][20347] Updated weights for policy 0, policy_version 1231852 (0.0008) [2023-12-27 14:11:01,115][20347] Updated weights for policy 0, policy_version 1231862 (0.0007) [2023-12-27 14:11:01,176][20347] Updated weights for policy 0, policy_version 1231872 (0.0007) [2023-12-27 14:11:01,178][20381] Updated weights for policy 1, policy_version 1230753 (0.0008) [2023-12-27 14:11:01,233][20381] Updated weights for policy 1, policy_version 1230763 (0.0007) [2023-12-27 14:11:01,294][20381] Updated weights for policy 1, policy_version 1230773 (0.0008) [2023-12-27 14:11:01,808][20347] Updated weights for policy 0, policy_version 1231882 (0.0008) [2023-12-27 14:11:01,869][20347] Updated weights for policy 0, policy_version 1231892 (0.0008) [2023-12-27 14:11:01,927][20347] Updated weights for policy 0, policy_version 1231902 (0.0008) [2023-12-27 14:11:01,930][20381] Updated weights for policy 1, policy_version 1230783 (0.0006) [2023-12-27 14:11:01,987][20381] Updated weights for policy 1, policy_version 1230793 (0.0005) [2023-12-27 14:11:02,042][20381] Updated weights for policy 1, policy_version 1230803 (0.0007) [2023-12-27 14:11:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 630538240. Throughput: 0: 10331.7, 1: 10190.0. Samples: 630548256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:02,363][19170] Avg episode reward: [(0, '931.900'), (1, '990.220')] [2023-12-27 14:11:02,614][20381] Updated weights for policy 1, policy_version 1230813 (0.0006) [2023-12-27 14:11:02,669][20381] Updated weights for policy 1, policy_version 1230823 (0.0006) [2023-12-27 14:11:02,698][20347] Updated weights for policy 0, policy_version 1231912 (0.0008) [2023-12-27 14:11:02,723][20381] Updated weights for policy 1, policy_version 1230833 (0.0005) [2023-12-27 14:11:02,758][20347] Updated weights for policy 0, policy_version 1231922 (0.0007) [2023-12-27 14:11:02,813][20347] Updated weights for policy 0, policy_version 1231932 (0.0008) [2023-12-27 14:11:03,381][20381] Updated weights for policy 1, policy_version 1230843 (0.0006) [2023-12-27 14:11:03,413][20347] Updated weights for policy 0, policy_version 1231942 (0.0007) [2023-12-27 14:11:03,433][20381] Updated weights for policy 1, policy_version 1230853 (0.0005) [2023-12-27 14:11:03,475][20347] Updated weights for policy 0, policy_version 1231952 (0.0006) [2023-12-27 14:11:03,485][20381] Updated weights for policy 1, policy_version 1230863 (0.0006) [2023-12-27 14:11:03,523][20347] Updated weights for policy 0, policy_version 1231962 (0.0005) [2023-12-27 14:11:04,085][20381] Updated weights for policy 1, policy_version 1230873 (0.0006) [2023-12-27 14:11:04,149][20381] Updated weights for policy 1, policy_version 1230883 (0.0010) [2023-12-27 14:11:04,207][20381] Updated weights for policy 1, policy_version 1230893 (0.0011) [2023-12-27 14:11:04,267][20381] Updated weights for policy 1, policy_version 1230903 (0.0011) [2023-12-27 14:11:04,269][20347] Updated weights for policy 0, policy_version 1231972 (0.0006) [2023-12-27 14:11:04,326][20347] Updated weights for policy 0, policy_version 1231982 (0.0007) [2023-12-27 14:11:04,386][20347] Updated weights for policy 0, policy_version 1231992 (0.0008) [2023-12-27 14:11:04,945][20381] Updated weights for policy 1, policy_version 1230913 (0.0006) [2023-12-27 14:11:05,007][20381] Updated weights for policy 1, policy_version 1230923 (0.0005) [2023-12-27 14:11:05,068][20381] Updated weights for policy 1, policy_version 1230933 (0.0006) [2023-12-27 14:11:05,106][20347] Updated weights for policy 0, policy_version 1232002 (0.0007) [2023-12-27 14:11:05,168][20347] Updated weights for policy 0, policy_version 1232012 (0.0008) [2023-12-27 14:11:05,226][20347] Updated weights for policy 0, policy_version 1232022 (0.0008) [2023-12-27 14:11:05,279][20347] Updated weights for policy 0, policy_version 1232032 (0.0007) [2023-12-27 14:11:05,679][20381] Updated weights for policy 1, policy_version 1230943 (0.0008) [2023-12-27 14:11:05,727][20381] Updated weights for policy 1, policy_version 1230953 (0.0007) [2023-12-27 14:11:05,777][20381] Updated weights for policy 1, policy_version 1230963 (0.0007) [2023-12-27 14:11:05,912][20347] Updated weights for policy 0, policy_version 1232042 (0.0007) [2023-12-27 14:11:05,960][20347] Updated weights for policy 0, policy_version 1232052 (0.0009) [2023-12-27 14:11:06,009][20347] Updated weights for policy 0, policy_version 1232062 (0.0005) [2023-12-27 14:11:06,459][20381] Updated weights for policy 1, policy_version 1230973 (0.0009) [2023-12-27 14:11:06,518][20381] Updated weights for policy 1, policy_version 1230983 (0.0009) [2023-12-27 14:11:06,579][20381] Updated weights for policy 1, policy_version 1230993 (0.0010) [2023-12-27 14:11:06,719][20347] Updated weights for policy 0, policy_version 1232072 (0.0009) [2023-12-27 14:11:06,777][20347] Updated weights for policy 0, policy_version 1232082 (0.0010) [2023-12-27 14:11:06,834][20347] Updated weights for policy 0, policy_version 1232092 (0.0010) [2023-12-27 14:11:07,251][20381] Updated weights for policy 1, policy_version 1231003 (0.0010) [2023-12-27 14:11:07,313][20381] Updated weights for policy 1, policy_version 1231013 (0.0011) [2023-12-27 14:11:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 630636544. Throughput: 0: 10316.6, 1: 10184.4. Samples: 630610328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:07,363][19170] Avg episode reward: [(0, '922.020'), (1, '961.000')] [2023-12-27 14:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001232096_315473920.pth... [2023-12-27 14:11:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001230912_315170816.pth [2023-12-27 14:11:07,374][20381] Updated weights for policy 1, policy_version 1231023 (0.0011) [2023-12-27 14:11:07,426][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001231032_315170816.pth... [2023-12-27 14:11:07,430][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001229816_314859520.pth [2023-12-27 14:11:07,517][20347] Updated weights for policy 0, policy_version 1232102 (0.0010) [2023-12-27 14:11:07,568][20347] Updated weights for policy 0, policy_version 1232112 (0.0010) [2023-12-27 14:11:07,619][20347] Updated weights for policy 0, policy_version 1232122 (0.0010) [2023-12-27 14:11:08,024][20381] Updated weights for policy 1, policy_version 1231033 (0.0010) [2023-12-27 14:11:08,082][20381] Updated weights for policy 1, policy_version 1231043 (0.0010) [2023-12-27 14:11:08,131][20381] Updated weights for policy 1, policy_version 1231053 (0.0010) [2023-12-27 14:11:08,174][20381] Updated weights for policy 1, policy_version 1231063 (0.0007) [2023-12-27 14:11:08,295][20347] Updated weights for policy 0, policy_version 1232132 (0.0010) [2023-12-27 14:11:08,349][20347] Updated weights for policy 0, policy_version 1232142 (0.0010) [2023-12-27 14:11:08,412][20347] Updated weights for policy 0, policy_version 1232152 (0.0010) [2023-12-27 14:11:08,842][20381] Updated weights for policy 1, policy_version 1231073 (0.0010) [2023-12-27 14:11:08,895][20381] Updated weights for policy 1, policy_version 1231083 (0.0011) [2023-12-27 14:11:08,960][20381] Updated weights for policy 1, policy_version 1231093 (0.0011) [2023-12-27 14:11:09,002][20347] Updated weights for policy 0, policy_version 1232162 (0.0007) [2023-12-27 14:11:09,061][20347] Updated weights for policy 0, policy_version 1232172 (0.0005) [2023-12-27 14:11:09,115][20347] Updated weights for policy 0, policy_version 1232182 (0.0007) [2023-12-27 14:11:09,161][20347] Updated weights for policy 0, policy_version 1232192 (0.0007) [2023-12-27 14:11:09,670][20381] Updated weights for policy 1, policy_version 1231103 (0.0010) [2023-12-27 14:11:09,731][20381] Updated weights for policy 1, policy_version 1231113 (0.0006) [2023-12-27 14:11:09,794][20381] Updated weights for policy 1, policy_version 1231123 (0.0008) [2023-12-27 14:11:09,897][20347] Updated weights for policy 0, policy_version 1232202 (0.0010) [2023-12-27 14:11:09,960][20347] Updated weights for policy 0, policy_version 1232212 (0.0011) [2023-12-27 14:11:10,011][20347] Updated weights for policy 0, policy_version 1232222 (0.0011) [2023-12-27 14:11:10,497][20381] Updated weights for policy 1, policy_version 1231133 (0.0007) [2023-12-27 14:11:10,552][20381] Updated weights for policy 1, policy_version 1231143 (0.0007) [2023-12-27 14:11:10,608][20381] Updated weights for policy 1, policy_version 1231153 (0.0007) [2023-12-27 14:11:10,775][20347] Updated weights for policy 0, policy_version 1232232 (0.0011) [2023-12-27 14:11:10,833][20347] Updated weights for policy 0, policy_version 1232242 (0.0010) [2023-12-27 14:11:10,892][20347] Updated weights for policy 0, policy_version 1232252 (0.0010) [2023-12-27 14:11:11,292][20381] Updated weights for policy 1, policy_version 1231163 (0.0007) [2023-12-27 14:11:11,360][20381] Updated weights for policy 1, policy_version 1231173 (0.0008) [2023-12-27 14:11:11,433][20381] Updated weights for policy 1, policy_version 1231183 (0.0007) [2023-12-27 14:11:11,620][20347] Updated weights for policy 0, policy_version 1232262 (0.0009) [2023-12-27 14:11:11,683][20347] Updated weights for policy 0, policy_version 1232272 (0.0009) [2023-12-27 14:11:11,747][20347] Updated weights for policy 0, policy_version 1232282 (0.0009) [2023-12-27 14:11:12,193][20381] Updated weights for policy 1, policy_version 1231193 (0.0009) [2023-12-27 14:11:12,252][20381] Updated weights for policy 1, policy_version 1231203 (0.0008) [2023-12-27 14:11:12,280][20292] KL-divergence is very high: 116.1428 [2023-12-27 14:11:12,308][20381] Updated weights for policy 1, policy_version 1231213 (0.0010) [2023-12-27 14:11:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 630734848. Throughput: 0: 10272.4, 1: 10227.5. Samples: 630732980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:12,363][19170] Avg episode reward: [(0, '892.770'), (1, '912.230')] [2023-12-27 14:11:12,374][20381] Updated weights for policy 1, policy_version 1231223 (0.0013) [2023-12-27 14:11:12,538][20347] Updated weights for policy 0, policy_version 1232292 (0.0008) [2023-12-27 14:11:12,602][20347] Updated weights for policy 0, policy_version 1232302 (0.0008) [2023-12-27 14:11:12,654][20347] Updated weights for policy 0, policy_version 1232312 (0.0008) [2023-12-27 14:11:12,946][20381] Updated weights for policy 1, policy_version 1231233 (0.0006) [2023-12-27 14:11:12,996][20381] Updated weights for policy 1, policy_version 1231243 (0.0007) [2023-12-27 14:11:13,044][20381] Updated weights for policy 1, policy_version 1231253 (0.0010) [2023-12-27 14:11:13,358][20347] Updated weights for policy 0, policy_version 1232322 (0.0007) [2023-12-27 14:11:13,411][20347] Updated weights for policy 0, policy_version 1232332 (0.0007) [2023-12-27 14:11:13,465][20347] Updated weights for policy 0, policy_version 1232342 (0.0007) [2023-12-27 14:11:13,522][20347] Updated weights for policy 0, policy_version 1232352 (0.0007) [2023-12-27 14:11:13,731][20381] Updated weights for policy 1, policy_version 1231263 (0.0011) [2023-12-27 14:11:13,789][20381] Updated weights for policy 1, policy_version 1231273 (0.0010) [2023-12-27 14:11:13,847][20381] Updated weights for policy 1, policy_version 1231283 (0.0007) [2023-12-27 14:11:14,143][20347] Updated weights for policy 0, policy_version 1232362 (0.0008) [2023-12-27 14:11:14,204][20347] Updated weights for policy 0, policy_version 1232372 (0.0008) [2023-12-27 14:11:14,258][20347] Updated weights for policy 0, policy_version 1232382 (0.0006) [2023-12-27 14:11:14,479][20381] Updated weights for policy 1, policy_version 1231293 (0.0011) [2023-12-27 14:11:14,532][20381] Updated weights for policy 1, policy_version 1231303 (0.0007) [2023-12-27 14:11:14,590][20381] Updated weights for policy 1, policy_version 1231313 (0.0010) [2023-12-27 14:11:14,922][20347] Updated weights for policy 0, policy_version 1232392 (0.0007) [2023-12-27 14:11:14,974][20347] Updated weights for policy 0, policy_version 1232402 (0.0007) [2023-12-27 14:11:15,028][20347] Updated weights for policy 0, policy_version 1232412 (0.0007) [2023-12-27 14:11:15,231][20381] Updated weights for policy 1, policy_version 1231323 (0.0009) [2023-12-27 14:11:15,281][20381] Updated weights for policy 1, policy_version 1231333 (0.0007) [2023-12-27 14:11:15,331][20381] Updated weights for policy 1, policy_version 1231343 (0.0008) [2023-12-27 14:11:15,725][20347] Updated weights for policy 0, policy_version 1232422 (0.0007) [2023-12-27 14:11:15,784][20347] Updated weights for policy 0, policy_version 1232432 (0.0006) [2023-12-27 14:11:15,836][20347] Updated weights for policy 0, policy_version 1232442 (0.0005) [2023-12-27 14:11:16,015][20381] Updated weights for policy 1, policy_version 1231353 (0.0008) [2023-12-27 14:11:16,068][20381] Updated weights for policy 1, policy_version 1231363 (0.0009) [2023-12-27 14:11:16,126][20381] Updated weights for policy 1, policy_version 1231373 (0.0007) [2023-12-27 14:11:16,187][20381] Updated weights for policy 1, policy_version 1231383 (0.0007) [2023-12-27 14:11:16,404][20347] Updated weights for policy 0, policy_version 1232452 (0.0005) [2023-12-27 14:11:16,462][20347] Updated weights for policy 0, policy_version 1232462 (0.0007) [2023-12-27 14:11:16,522][20347] Updated weights for policy 0, policy_version 1232472 (0.0005) [2023-12-27 14:11:16,800][20381] Updated weights for policy 1, policy_version 1231393 (0.0008) [2023-12-27 14:11:16,845][20381] Updated weights for policy 1, policy_version 1231403 (0.0008) [2023-12-27 14:11:16,903][20381] Updated weights for policy 1, policy_version 1231413 (0.0008) [2023-12-27 14:11:17,071][20347] Updated weights for policy 0, policy_version 1232482 (0.0006) [2023-12-27 14:11:17,123][20347] Updated weights for policy 0, policy_version 1232492 (0.0007) [2023-12-27 14:11:17,171][20347] Updated weights for policy 0, policy_version 1232502 (0.0009) [2023-12-27 14:11:17,223][20347] Updated weights for policy 0, policy_version 1232512 (0.0008) [2023-12-27 14:11:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 630849536. Throughput: 0: 10254.8, 1: 10282.3. Samples: 630857764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:17,363][19170] Avg episode reward: [(0, '882.780'), (1, '941.460')] [2023-12-27 14:11:17,535][20381] Updated weights for policy 1, policy_version 1231423 (0.0008) [2023-12-27 14:11:17,580][20381] Updated weights for policy 1, policy_version 1231433 (0.0010) [2023-12-27 14:11:17,629][20381] Updated weights for policy 1, policy_version 1231443 (0.0008) [2023-12-27 14:11:17,873][20347] Updated weights for policy 0, policy_version 1232522 (0.0005) [2023-12-27 14:11:17,933][20347] Updated weights for policy 0, policy_version 1232532 (0.0007) [2023-12-27 14:11:17,985][20347] Updated weights for policy 0, policy_version 1232542 (0.0008) [2023-12-27 14:11:18,377][20381] Updated weights for policy 1, policy_version 1231453 (0.0007) [2023-12-27 14:11:18,442][20381] Updated weights for policy 1, policy_version 1231463 (0.0006) [2023-12-27 14:11:18,510][20381] Updated weights for policy 1, policy_version 1231473 (0.0008) [2023-12-27 14:11:18,568][20347] Updated weights for policy 0, policy_version 1232552 (0.0006) [2023-12-27 14:11:18,623][20347] Updated weights for policy 0, policy_version 1232562 (0.0007) [2023-12-27 14:11:18,678][20347] Updated weights for policy 0, policy_version 1232572 (0.0008) [2023-12-27 14:11:19,163][20381] Updated weights for policy 1, policy_version 1231483 (0.0006) [2023-12-27 14:11:19,219][20381] Updated weights for policy 1, policy_version 1231493 (0.0006) [2023-12-27 14:11:19,281][20381] Updated weights for policy 1, policy_version 1231503 (0.0008) [2023-12-27 14:11:19,419][20347] Updated weights for policy 0, policy_version 1232582 (0.0007) [2023-12-27 14:11:19,469][20347] Updated weights for policy 0, policy_version 1232592 (0.0007) [2023-12-27 14:11:19,530][20347] Updated weights for policy 0, policy_version 1232602 (0.0008) [2023-12-27 14:11:20,037][20381] Updated weights for policy 1, policy_version 1231513 (0.0008) [2023-12-27 14:11:20,099][20381] Updated weights for policy 1, policy_version 1231523 (0.0008) [2023-12-27 14:11:20,161][20381] Updated weights for policy 1, policy_version 1231533 (0.0008) [2023-12-27 14:11:20,219][20381] Updated weights for policy 1, policy_version 1231543 (0.0008) [2023-12-27 14:11:20,249][20347] Updated weights for policy 0, policy_version 1232612 (0.0008) [2023-12-27 14:11:20,310][20347] Updated weights for policy 0, policy_version 1232622 (0.0009) [2023-12-27 14:11:20,370][20347] Updated weights for policy 0, policy_version 1232632 (0.0007) [2023-12-27 14:11:20,971][20381] Updated weights for policy 1, policy_version 1231553 (0.0008) [2023-12-27 14:11:20,978][20347] Updated weights for policy 0, policy_version 1232642 (0.0007) [2023-12-27 14:11:21,033][20381] Updated weights for policy 1, policy_version 1231563 (0.0007) [2023-12-27 14:11:21,037][20347] Updated weights for policy 0, policy_version 1232652 (0.0007) [2023-12-27 14:11:21,098][20347] Updated weights for policy 0, policy_version 1232662 (0.0008) [2023-12-27 14:11:21,098][20381] Updated weights for policy 1, policy_version 1231573 (0.0007) [2023-12-27 14:11:21,162][20347] Updated weights for policy 0, policy_version 1232672 (0.0008) [2023-12-27 14:11:21,852][20381] Updated weights for policy 1, policy_version 1231583 (0.0007) [2023-12-27 14:11:21,884][20347] Updated weights for policy 0, policy_version 1232682 (0.0008) [2023-12-27 14:11:21,909][20381] Updated weights for policy 1, policy_version 1231593 (0.0006) [2023-12-27 14:11:21,936][20347] Updated weights for policy 0, policy_version 1232692 (0.0008) [2023-12-27 14:11:21,964][20381] Updated weights for policy 1, policy_version 1231603 (0.0007) [2023-12-27 14:11:21,992][20347] Updated weights for policy 0, policy_version 1232702 (0.0008) [2023-12-27 14:11:22,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 630947840. Throughput: 0: 10218.5, 1: 10267.7. Samples: 630918136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:22,363][19170] Avg episode reward: [(0, '970.960'), (1, '980.460')] [2023-12-27 14:11:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001231608_315318272.pth... [2023-12-27 14:11:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001232704_315629568.pth... [2023-12-27 14:11:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001230424_315015168.pth [2023-12-27 14:11:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001231520_315326464.pth [2023-12-27 14:11:22,694][20381] Updated weights for policy 1, policy_version 1231613 (0.0007) [2023-12-27 14:11:22,710][20347] Updated weights for policy 0, policy_version 1232712 (0.0010) [2023-12-27 14:11:22,748][20381] Updated weights for policy 1, policy_version 1231623 (0.0009) [2023-12-27 14:11:22,756][20347] Updated weights for policy 0, policy_version 1232722 (0.0010) [2023-12-27 14:11:22,808][20381] Updated weights for policy 1, policy_version 1231633 (0.0006) [2023-12-27 14:11:22,810][20347] Updated weights for policy 0, policy_version 1232732 (0.0010) [2023-12-27 14:11:23,410][20347] Updated weights for policy 0, policy_version 1232742 (0.0010) [2023-12-27 14:11:23,462][20347] Updated weights for policy 0, policy_version 1232752 (0.0010) [2023-12-27 14:11:23,474][20381] Updated weights for policy 1, policy_version 1231643 (0.0007) [2023-12-27 14:11:23,516][20347] Updated weights for policy 0, policy_version 1232762 (0.0006) [2023-12-27 14:11:23,534][20381] Updated weights for policy 1, policy_version 1231653 (0.0008) [2023-12-27 14:11:23,596][20381] Updated weights for policy 1, policy_version 1231663 (0.0011) [2023-12-27 14:11:24,099][20347] Updated weights for policy 0, policy_version 1232772 (0.0009) [2023-12-27 14:11:24,164][20347] Updated weights for policy 0, policy_version 1232782 (0.0010) [2023-12-27 14:11:24,225][20347] Updated weights for policy 0, policy_version 1232792 (0.0010) [2023-12-27 14:11:24,252][20381] Updated weights for policy 1, policy_version 1231673 (0.0011) [2023-12-27 14:11:24,304][20381] Updated weights for policy 1, policy_version 1231683 (0.0010) [2023-12-27 14:11:24,353][20381] Updated weights for policy 1, policy_version 1231693 (0.0010) [2023-12-27 14:11:24,402][20381] Updated weights for policy 1, policy_version 1231703 (0.0010) [2023-12-27 14:11:24,892][20347] Updated weights for policy 0, policy_version 1232802 (0.0010) [2023-12-27 14:11:24,942][20347] Updated weights for policy 0, policy_version 1232812 (0.0010) [2023-12-27 14:11:24,987][20347] Updated weights for policy 0, policy_version 1232822 (0.0010) [2023-12-27 14:11:25,035][20347] Updated weights for policy 0, policy_version 1232832 (0.0010) [2023-12-27 14:11:25,111][20381] Updated weights for policy 1, policy_version 1231713 (0.0006) [2023-12-27 14:11:25,166][20381] Updated weights for policy 1, policy_version 1231723 (0.0006) [2023-12-27 14:11:25,226][20381] Updated weights for policy 1, policy_version 1231733 (0.0005) [2023-12-27 14:11:25,722][20347] Updated weights for policy 0, policy_version 1232842 (0.0010) [2023-12-27 14:11:25,773][20347] Updated weights for policy 0, policy_version 1232852 (0.0010) [2023-12-27 14:11:25,828][20347] Updated weights for policy 0, policy_version 1232862 (0.0010) [2023-12-27 14:11:25,882][20381] Updated weights for policy 1, policy_version 1231743 (0.0005) [2023-12-27 14:11:25,934][20381] Updated weights for policy 1, policy_version 1231753 (0.0007) [2023-12-27 14:11:25,986][20381] Updated weights for policy 1, policy_version 1231763 (0.0007) [2023-12-27 14:11:26,506][20347] Updated weights for policy 0, policy_version 1232872 (0.0008) [2023-12-27 14:11:26,556][20347] Updated weights for policy 0, policy_version 1232882 (0.0008) [2023-12-27 14:11:26,608][20347] Updated weights for policy 0, policy_version 1232892 (0.0007) [2023-12-27 14:11:26,695][20381] Updated weights for policy 1, policy_version 1231773 (0.0008) [2023-12-27 14:11:26,742][20381] Updated weights for policy 1, policy_version 1231783 (0.0008) [2023-12-27 14:11:26,794][20381] Updated weights for policy 1, policy_version 1231793 (0.0007) [2023-12-27 14:11:27,299][20347] Updated weights for policy 0, policy_version 1232902 (0.0008) [2023-12-27 14:11:27,362][20347] Updated weights for policy 0, policy_version 1232912 (0.0008) [2023-12-27 14:11:27,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 631046144. Throughput: 0: 10243.9, 1: 10219.4. Samples: 631040496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:27,363][19170] Avg episode reward: [(0, '970.690'), (1, '970.790')] [2023-12-27 14:11:27,412][20347] Updated weights for policy 0, policy_version 1232922 (0.0009) [2023-12-27 14:11:27,505][20381] Updated weights for policy 1, policy_version 1231803 (0.0007) [2023-12-27 14:11:27,563][20381] Updated weights for policy 1, policy_version 1231813 (0.0006) [2023-12-27 14:11:27,623][20381] Updated weights for policy 1, policy_version 1231823 (0.0007) [2023-12-27 14:11:28,016][20347] Updated weights for policy 0, policy_version 1232932 (0.0007) [2023-12-27 14:11:28,074][20347] Updated weights for policy 0, policy_version 1232942 (0.0008) [2023-12-27 14:11:28,132][20347] Updated weights for policy 0, policy_version 1232952 (0.0010) [2023-12-27 14:11:28,156][20381] Updated weights for policy 1, policy_version 1231833 (0.0005) [2023-12-27 14:11:28,213][20381] Updated weights for policy 1, policy_version 1231843 (0.0007) [2023-12-27 14:11:28,273][20381] Updated weights for policy 1, policy_version 1231853 (0.0008) [2023-12-27 14:11:28,319][20381] Updated weights for policy 1, policy_version 1231863 (0.0010) [2023-12-27 14:11:28,727][20347] Updated weights for policy 0, policy_version 1232962 (0.0008) [2023-12-27 14:11:28,787][20347] Updated weights for policy 0, policy_version 1232972 (0.0009) [2023-12-27 14:11:28,853][20347] Updated weights for policy 0, policy_version 1232982 (0.0007) [2023-12-27 14:11:28,909][20347] Updated weights for policy 0, policy_version 1232992 (0.0007) [2023-12-27 14:11:28,935][20381] Updated weights for policy 1, policy_version 1231873 (0.0007) [2023-12-27 14:11:28,989][20381] Updated weights for policy 1, policy_version 1231883 (0.0006) [2023-12-27 14:11:29,044][20381] Updated weights for policy 1, policy_version 1231893 (0.0007) [2023-12-27 14:11:29,641][20347] Updated weights for policy 0, policy_version 1233002 (0.0008) [2023-12-27 14:11:29,704][20347] Updated weights for policy 0, policy_version 1233012 (0.0007) [2023-12-27 14:11:29,720][20381] Updated weights for policy 1, policy_version 1231903 (0.0006) [2023-12-27 14:11:29,766][20347] Updated weights for policy 0, policy_version 1233022 (0.0008) [2023-12-27 14:11:29,783][20381] Updated weights for policy 1, policy_version 1231913 (0.0006) [2023-12-27 14:11:29,850][20381] Updated weights for policy 1, policy_version 1231923 (0.0008) [2023-12-27 14:11:30,474][20347] Updated weights for policy 0, policy_version 1233032 (0.0007) [2023-12-27 14:11:30,528][20347] Updated weights for policy 0, policy_version 1233042 (0.0007) [2023-12-27 14:11:30,547][20381] Updated weights for policy 1, policy_version 1231933 (0.0006) [2023-12-27 14:11:30,583][20347] Updated weights for policy 0, policy_version 1233052 (0.0005) [2023-12-27 14:11:30,596][20381] Updated weights for policy 1, policy_version 1231943 (0.0005) [2023-12-27 14:11:30,651][20381] Updated weights for policy 1, policy_version 1231953 (0.0007) [2023-12-27 14:11:31,172][20347] Updated weights for policy 0, policy_version 1233062 (0.0008) [2023-12-27 14:11:31,229][20347] Updated weights for policy 0, policy_version 1233072 (0.0009) [2023-12-27 14:11:31,248][20381] Updated weights for policy 1, policy_version 1231963 (0.0006) [2023-12-27 14:11:31,293][20347] Updated weights for policy 0, policy_version 1233082 (0.0008) [2023-12-27 14:11:31,306][20381] Updated weights for policy 1, policy_version 1231973 (0.0006) [2023-12-27 14:11:31,363][20381] Updated weights for policy 1, policy_version 1231983 (0.0008) [2023-12-27 14:11:32,013][20347] Updated weights for policy 0, policy_version 1233092 (0.0008) [2023-12-27 14:11:32,063][20347] Updated weights for policy 0, policy_version 1233102 (0.0010) [2023-12-27 14:11:32,090][20381] Updated weights for policy 1, policy_version 1231993 (0.0008) [2023-12-27 14:11:32,117][20347] Updated weights for policy 0, policy_version 1233112 (0.0010) [2023-12-27 14:11:32,146][20381] Updated weights for policy 1, policy_version 1232003 (0.0006) [2023-12-27 14:11:32,201][20381] Updated weights for policy 1, policy_version 1232013 (0.0007) [2023-12-27 14:11:32,257][20381] Updated weights for policy 1, policy_version 1232023 (0.0007) [2023-12-27 14:11:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 631160832. Throughput: 0: 10240.8, 1: 10231.5. Samples: 631166044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:32,363][19170] Avg episode reward: [(0, '970.680'), (1, '951.180')] [2023-12-27 14:11:32,758][20347] Updated weights for policy 0, policy_version 1233122 (0.0010) [2023-12-27 14:11:32,825][20347] Updated weights for policy 0, policy_version 1233132 (0.0010) [2023-12-27 14:11:32,885][20347] Updated weights for policy 0, policy_version 1233142 (0.0009) [2023-12-27 14:11:32,943][20347] Updated weights for policy 0, policy_version 1233152 (0.0010) [2023-12-27 14:11:32,963][20381] Updated weights for policy 1, policy_version 1232033 (0.0006) [2023-12-27 14:11:33,018][20381] Updated weights for policy 1, policy_version 1232043 (0.0005) [2023-12-27 14:11:33,073][20381] Updated weights for policy 1, policy_version 1232053 (0.0005) [2023-12-27 14:11:33,526][20347] Updated weights for policy 0, policy_version 1233162 (0.0009) [2023-12-27 14:11:33,572][20347] Updated weights for policy 0, policy_version 1233172 (0.0008) [2023-12-27 14:11:33,622][20347] Updated weights for policy 0, policy_version 1233182 (0.0007) [2023-12-27 14:11:33,686][20381] Updated weights for policy 1, policy_version 1232063 (0.0006) [2023-12-27 14:11:33,745][20381] Updated weights for policy 1, policy_version 1232073 (0.0008) [2023-12-27 14:11:33,790][20381] Updated weights for policy 1, policy_version 1232083 (0.0008) [2023-12-27 14:11:34,262][20347] Updated weights for policy 0, policy_version 1233192 (0.0007) [2023-12-27 14:11:34,317][20347] Updated weights for policy 0, policy_version 1233202 (0.0007) [2023-12-27 14:11:34,375][20347] Updated weights for policy 0, policy_version 1233212 (0.0007) [2023-12-27 14:11:34,492][20381] Updated weights for policy 1, policy_version 1232093 (0.0007) [2023-12-27 14:11:34,547][20381] Updated weights for policy 1, policy_version 1232103 (0.0008) [2023-12-27 14:11:34,607][20381] Updated weights for policy 1, policy_version 1232113 (0.0008) [2023-12-27 14:11:35,020][20347] Updated weights for policy 0, policy_version 1233222 (0.0007) [2023-12-27 14:11:35,075][20347] Updated weights for policy 0, policy_version 1233232 (0.0007) [2023-12-27 14:11:35,131][20347] Updated weights for policy 0, policy_version 1233242 (0.0007) [2023-12-27 14:11:35,274][20381] Updated weights for policy 1, policy_version 1232123 (0.0007) [2023-12-27 14:11:35,326][20381] Updated weights for policy 1, policy_version 1232133 (0.0005) [2023-12-27 14:11:35,385][20381] Updated weights for policy 1, policy_version 1232143 (0.0005) [2023-12-27 14:11:35,697][20347] Updated weights for policy 0, policy_version 1233252 (0.0006) [2023-12-27 14:11:35,759][20347] Updated weights for policy 0, policy_version 1233262 (0.0005) [2023-12-27 14:11:35,814][20347] Updated weights for policy 0, policy_version 1233272 (0.0008) [2023-12-27 14:11:35,997][20381] Updated weights for policy 1, policy_version 1232153 (0.0005) [2023-12-27 14:11:36,050][20381] Updated weights for policy 1, policy_version 1232163 (0.0007) [2023-12-27 14:11:36,116][20381] Updated weights for policy 1, policy_version 1232173 (0.0007) [2023-12-27 14:11:36,168][20381] Updated weights for policy 1, policy_version 1232183 (0.0008) [2023-12-27 14:11:36,518][20347] Updated weights for policy 0, policy_version 1233282 (0.0007) [2023-12-27 14:11:36,574][20347] Updated weights for policy 0, policy_version 1233292 (0.0008) [2023-12-27 14:11:36,630][20347] Updated weights for policy 0, policy_version 1233302 (0.0007) [2023-12-27 14:11:36,689][20347] Updated weights for policy 0, policy_version 1233312 (0.0008) [2023-12-27 14:11:36,852][20381] Updated weights for policy 1, policy_version 1232193 (0.0009) [2023-12-27 14:11:36,914][20381] Updated weights for policy 1, policy_version 1232203 (0.0007) [2023-12-27 14:11:36,960][20381] Updated weights for policy 1, policy_version 1232213 (0.0008) [2023-12-27 14:11:37,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 631259136. Throughput: 0: 10295.5, 1: 10241.6. Samples: 631230056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:37,363][19170] Avg episode reward: [(0, '941.320'), (1, '941.320')] [2023-12-27 14:11:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001232216_315473920.pth... [2023-12-27 14:11:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001231032_315170816.pth [2023-12-27 14:11:37,394][20347] Updated weights for policy 0, policy_version 1233322 (0.0007) [2023-12-27 14:11:37,447][20347] Updated weights for policy 0, policy_version 1233332 (0.0007) [2023-12-27 14:11:37,496][20347] Updated weights for policy 0, policy_version 1233342 (0.0006) [2023-12-27 14:11:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001233344_315793408.pth... [2023-12-27 14:11:37,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001232096_315473920.pth [2023-12-27 14:11:37,666][20381] Updated weights for policy 1, policy_version 1232223 (0.0010) [2023-12-27 14:11:37,724][20381] Updated weights for policy 1, policy_version 1232233 (0.0011) [2023-12-27 14:11:37,769][20381] Updated weights for policy 1, policy_version 1232243 (0.0010) [2023-12-27 14:11:38,169][20347] Updated weights for policy 0, policy_version 1233352 (0.0007) [2023-12-27 14:11:38,225][20347] Updated weights for policy 0, policy_version 1233362 (0.0008) [2023-12-27 14:11:38,282][20347] Updated weights for policy 0, policy_version 1233372 (0.0009) [2023-12-27 14:11:38,466][20381] Updated weights for policy 1, policy_version 1232253 (0.0009) [2023-12-27 14:11:38,518][20381] Updated weights for policy 1, policy_version 1232263 (0.0008) [2023-12-27 14:11:38,573][20381] Updated weights for policy 1, policy_version 1232273 (0.0007) [2023-12-27 14:11:39,026][20347] Updated weights for policy 0, policy_version 1233382 (0.0008) [2023-12-27 14:11:39,078][20347] Updated weights for policy 0, policy_version 1233392 (0.0007) [2023-12-27 14:11:39,125][20347] Updated weights for policy 0, policy_version 1233402 (0.0007) [2023-12-27 14:11:39,172][20381] Updated weights for policy 1, policy_version 1232283 (0.0008) [2023-12-27 14:11:39,223][20381] Updated weights for policy 1, policy_version 1232293 (0.0009) [2023-12-27 14:11:39,277][20381] Updated weights for policy 1, policy_version 1232303 (0.0011) [2023-12-27 14:11:39,830][20347] Updated weights for policy 0, policy_version 1233412 (0.0007) [2023-12-27 14:11:39,889][20347] Updated weights for policy 0, policy_version 1233422 (0.0006) [2023-12-27 14:11:39,948][20347] Updated weights for policy 0, policy_version 1233432 (0.0006) [2023-12-27 14:11:40,058][20381] Updated weights for policy 1, policy_version 1232313 (0.0009) [2023-12-27 14:11:40,124][20381] Updated weights for policy 1, policy_version 1232323 (0.0011) [2023-12-27 14:11:40,188][20381] Updated weights for policy 1, policy_version 1232333 (0.0011) [2023-12-27 14:11:40,248][20381] Updated weights for policy 1, policy_version 1232343 (0.0011) [2023-12-27 14:11:40,701][20347] Updated weights for policy 0, policy_version 1233442 (0.0008) [2023-12-27 14:11:40,760][20347] Updated weights for policy 0, policy_version 1233452 (0.0008) [2023-12-27 14:11:40,817][20347] Updated weights for policy 0, policy_version 1233462 (0.0007) [2023-12-27 14:11:40,868][20347] Updated weights for policy 0, policy_version 1233472 (0.0007) [2023-12-27 14:11:40,905][20381] Updated weights for policy 1, policy_version 1232353 (0.0010) [2023-12-27 14:11:40,955][20381] Updated weights for policy 1, policy_version 1232363 (0.0010) [2023-12-27 14:11:41,003][20381] Updated weights for policy 1, policy_version 1232373 (0.0010) [2023-12-27 14:11:41,611][20347] Updated weights for policy 0, policy_version 1233482 (0.0008) [2023-12-27 14:11:41,683][20347] Updated weights for policy 0, policy_version 1233492 (0.0008) [2023-12-27 14:11:41,754][20347] Updated weights for policy 0, policy_version 1233502 (0.0008) [2023-12-27 14:11:41,780][20381] Updated weights for policy 1, policy_version 1232383 (0.0006) [2023-12-27 14:11:41,834][20381] Updated weights for policy 1, policy_version 1232393 (0.0007) [2023-12-27 14:11:41,883][20381] Updated weights for policy 1, policy_version 1232403 (0.0007) [2023-12-27 14:11:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 631357440. Throughput: 0: 10260.9, 1: 10238.5. Samples: 631351172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:11:42,363][19170] Avg episode reward: [(0, '941.300'), (1, '970.680')] [2023-12-27 14:11:42,537][20347] Updated weights for policy 0, policy_version 1233512 (0.0008) [2023-12-27 14:11:42,564][20381] Updated weights for policy 1, policy_version 1232413 (0.0007) [2023-12-27 14:11:42,597][20347] Updated weights for policy 0, policy_version 1233522 (0.0006) [2023-12-27 14:11:42,625][20381] Updated weights for policy 1, policy_version 1232423 (0.0007) [2023-12-27 14:11:42,657][20347] Updated weights for policy 0, policy_version 1233532 (0.0006) [2023-12-27 14:11:42,686][20381] Updated weights for policy 1, policy_version 1232433 (0.0006) [2023-12-27 14:11:43,321][20381] Updated weights for policy 1, policy_version 1232443 (0.0006) [2023-12-27 14:11:43,355][20347] Updated weights for policy 0, policy_version 1233542 (0.0007) [2023-12-27 14:11:43,374][20381] Updated weights for policy 1, policy_version 1232453 (0.0005) [2023-12-27 14:11:43,402][20347] Updated weights for policy 0, policy_version 1233552 (0.0007) [2023-12-27 14:11:43,426][20381] Updated weights for policy 1, policy_version 1232463 (0.0006) [2023-12-27 14:11:43,455][20347] Updated weights for policy 0, policy_version 1233562 (0.0005) [2023-12-27 14:11:44,004][20381] Updated weights for policy 1, policy_version 1232473 (0.0006) [2023-12-27 14:11:44,052][20381] Updated weights for policy 1, policy_version 1232483 (0.0006) [2023-12-27 14:11:44,103][20381] Updated weights for policy 1, policy_version 1232493 (0.0005) [2023-12-27 14:11:44,151][20347] Updated weights for policy 0, policy_version 1233572 (0.0006) [2023-12-27 14:11:44,168][20381] Updated weights for policy 1, policy_version 1232503 (0.0008) [2023-12-27 14:11:44,205][20347] Updated weights for policy 0, policy_version 1233582 (0.0007) [2023-12-27 14:11:44,265][20347] Updated weights for policy 0, policy_version 1233592 (0.0006) [2023-12-27 14:11:44,751][20381] Updated weights for policy 1, policy_version 1232513 (0.0006) [2023-12-27 14:11:44,811][20381] Updated weights for policy 1, policy_version 1232523 (0.0010) [2023-12-27 14:11:44,868][20381] Updated weights for policy 1, policy_version 1232533 (0.0009) [2023-12-27 14:11:44,965][20347] Updated weights for policy 0, policy_version 1233602 (0.0007) [2023-12-27 14:11:45,027][20347] Updated weights for policy 0, policy_version 1233612 (0.0009) [2023-12-27 14:11:45,088][20347] Updated weights for policy 0, policy_version 1233622 (0.0008) [2023-12-27 14:11:45,141][20347] Updated weights for policy 0, policy_version 1233632 (0.0007) [2023-12-27 14:11:45,471][20381] Updated weights for policy 1, policy_version 1232543 (0.0009) [2023-12-27 14:11:45,522][20381] Updated weights for policy 1, policy_version 1232553 (0.0010) [2023-12-27 14:11:45,568][20381] Updated weights for policy 1, policy_version 1232563 (0.0010) [2023-12-27 14:11:45,864][20347] Updated weights for policy 0, policy_version 1233642 (0.0007) [2023-12-27 14:11:45,924][20347] Updated weights for policy 0, policy_version 1233652 (0.0007) [2023-12-27 14:11:45,984][20347] Updated weights for policy 0, policy_version 1233662 (0.0007) [2023-12-27 14:11:46,234][20381] Updated weights for policy 1, policy_version 1232573 (0.0008) [2023-12-27 14:11:46,278][20381] Updated weights for policy 1, policy_version 1232583 (0.0005) [2023-12-27 14:11:46,327][20381] Updated weights for policy 1, policy_version 1232593 (0.0007) [2023-12-27 14:11:46,679][20347] Updated weights for policy 0, policy_version 1233672 (0.0009) [2023-12-27 14:11:46,735][20347] Updated weights for policy 0, policy_version 1233682 (0.0010) [2023-12-27 14:11:46,783][20347] Updated weights for policy 0, policy_version 1233692 (0.0010) [2023-12-27 14:11:46,946][20381] Updated weights for policy 1, policy_version 1232603 (0.0005) [2023-12-27 14:11:47,002][20381] Updated weights for policy 1, policy_version 1232613 (0.0005) [2023-12-27 14:11:47,060][20381] Updated weights for policy 1, policy_version 1232623 (0.0005) [2023-12-27 14:11:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 631463936. Throughput: 0: 10250.7, 1: 10344.6. Samples: 631475044. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:11:47,363][19170] Avg episode reward: [(0, '990.210'), (1, '960.960')] [2023-12-27 14:11:47,459][20347] Updated weights for policy 0, policy_version 1233702 (0.0010) [2023-12-27 14:11:47,505][20347] Updated weights for policy 0, policy_version 1233712 (0.0010) [2023-12-27 14:11:47,565][20347] Updated weights for policy 0, policy_version 1233722 (0.0007) [2023-12-27 14:11:47,633][20381] Updated weights for policy 1, policy_version 1232633 (0.0009) [2023-12-27 14:11:47,693][20381] Updated weights for policy 1, policy_version 1232643 (0.0007) [2023-12-27 14:11:47,754][20381] Updated weights for policy 1, policy_version 1232653 (0.0006) [2023-12-27 14:11:47,812][20381] Updated weights for policy 1, policy_version 1232663 (0.0007) [2023-12-27 14:11:48,173][20347] Updated weights for policy 0, policy_version 1233732 (0.0006) [2023-12-27 14:11:48,226][20347] Updated weights for policy 0, policy_version 1233742 (0.0005) [2023-12-27 14:11:48,290][20347] Updated weights for policy 0, policy_version 1233752 (0.0005) [2023-12-27 14:11:48,430][20381] Updated weights for policy 1, policy_version 1232673 (0.0008) [2023-12-27 14:11:48,484][20381] Updated weights for policy 1, policy_version 1232683 (0.0008) [2023-12-27 14:11:48,535][20381] Updated weights for policy 1, policy_version 1232693 (0.0008) [2023-12-27 14:11:48,890][20347] Updated weights for policy 0, policy_version 1233762 (0.0007) [2023-12-27 14:11:48,954][20347] Updated weights for policy 0, policy_version 1233772 (0.0007) [2023-12-27 14:11:49,010][20347] Updated weights for policy 0, policy_version 1233782 (0.0008) [2023-12-27 14:11:49,066][20347] Updated weights for policy 0, policy_version 1233792 (0.0006) [2023-12-27 14:11:49,195][20381] Updated weights for policy 1, policy_version 1232703 (0.0010) [2023-12-27 14:11:49,251][20381] Updated weights for policy 1, policy_version 1232713 (0.0009) [2023-12-27 14:11:49,313][20381] Updated weights for policy 1, policy_version 1232723 (0.0008) [2023-12-27 14:11:49,674][20347] Updated weights for policy 0, policy_version 1233802 (0.0006) [2023-12-27 14:11:49,740][20347] Updated weights for policy 0, policy_version 1233812 (0.0008) [2023-12-27 14:11:49,805][20347] Updated weights for policy 0, policy_version 1233822 (0.0007) [2023-12-27 14:11:50,044][20381] Updated weights for policy 1, policy_version 1232733 (0.0010) [2023-12-27 14:11:50,102][20381] Updated weights for policy 1, policy_version 1232743 (0.0011) [2023-12-27 14:11:50,161][20381] Updated weights for policy 1, policy_version 1232753 (0.0010) [2023-12-27 14:11:50,498][20347] Updated weights for policy 0, policy_version 1233832 (0.0006) [2023-12-27 14:11:50,556][20347] Updated weights for policy 0, policy_version 1233842 (0.0005) [2023-12-27 14:11:50,622][20347] Updated weights for policy 0, policy_version 1233852 (0.0009) [2023-12-27 14:11:50,846][20381] Updated weights for policy 1, policy_version 1232763 (0.0009) [2023-12-27 14:11:50,906][20381] Updated weights for policy 1, policy_version 1232773 (0.0005) [2023-12-27 14:11:50,966][20381] Updated weights for policy 1, policy_version 1232783 (0.0008) [2023-12-27 14:11:51,291][20347] Updated weights for policy 0, policy_version 1233862 (0.0007) [2023-12-27 14:11:51,355][20347] Updated weights for policy 0, policy_version 1233872 (0.0007) [2023-12-27 14:11:51,418][20347] Updated weights for policy 0, policy_version 1233882 (0.0009) [2023-12-27 14:11:51,671][20381] Updated weights for policy 1, policy_version 1232793 (0.0006) [2023-12-27 14:11:51,736][20381] Updated weights for policy 1, policy_version 1232803 (0.0007) [2023-12-27 14:11:51,801][20381] Updated weights for policy 1, policy_version 1232813 (0.0007) [2023-12-27 14:11:51,872][20381] Updated weights for policy 1, policy_version 1232823 (0.0008) [2023-12-27 14:11:52,110][20347] Updated weights for policy 0, policy_version 1233892 (0.0009) [2023-12-27 14:11:52,165][20347] Updated weights for policy 0, policy_version 1233902 (0.0007) [2023-12-27 14:11:52,220][20347] Updated weights for policy 0, policy_version 1233912 (0.0008) [2023-12-27 14:11:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 631570432. Throughput: 0: 10307.1, 1: 10324.9. Samples: 631538768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:11:52,363][19170] Avg episode reward: [(0, '990.210'), (1, '951.240')] [2023-12-27 14:11:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001232824_315629568.pth... [2023-12-27 14:11:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001233920_315940864.pth... [2023-12-27 14:11:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001232704_315629568.pth [2023-12-27 14:11:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001231608_315318272.pth [2023-12-27 14:11:52,540][20381] Updated weights for policy 1, policy_version 1232833 (0.0010) [2023-12-27 14:11:52,596][20381] Updated weights for policy 1, policy_version 1232843 (0.0011) [2023-12-27 14:11:52,656][20381] Updated weights for policy 1, policy_version 1232853 (0.0011) [2023-12-27 14:11:52,920][20347] Updated weights for policy 0, policy_version 1233922 (0.0007) [2023-12-27 14:11:52,977][20347] Updated weights for policy 0, policy_version 1233932 (0.0007) [2023-12-27 14:11:53,027][20347] Updated weights for policy 0, policy_version 1233942 (0.0007) [2023-12-27 14:11:53,078][20347] Updated weights for policy 0, policy_version 1233952 (0.0007) [2023-12-27 14:11:53,319][20381] Updated weights for policy 1, policy_version 1232863 (0.0007) [2023-12-27 14:11:53,367][20381] Updated weights for policy 1, policy_version 1232873 (0.0010) [2023-12-27 14:11:53,421][20381] Updated weights for policy 1, policy_version 1232883 (0.0007) [2023-12-27 14:11:53,658][20347] Updated weights for policy 0, policy_version 1233962 (0.0005) [2023-12-27 14:11:53,713][20347] Updated weights for policy 0, policy_version 1233972 (0.0005) [2023-12-27 14:11:53,765][20347] Updated weights for policy 0, policy_version 1233982 (0.0007) [2023-12-27 14:11:53,999][20381] Updated weights for policy 1, policy_version 1232893 (0.0010) [2023-12-27 14:11:54,048][20381] Updated weights for policy 1, policy_version 1232903 (0.0010) [2023-12-27 14:11:54,109][20381] Updated weights for policy 1, policy_version 1232913 (0.0011) [2023-12-27 14:11:54,368][20347] Updated weights for policy 0, policy_version 1233992 (0.0010) [2023-12-27 14:11:54,415][20347] Updated weights for policy 0, policy_version 1234002 (0.0010) [2023-12-27 14:11:54,470][20347] Updated weights for policy 0, policy_version 1234012 (0.0010) [2023-12-27 14:11:54,731][20381] Updated weights for policy 1, policy_version 1232923 (0.0006) [2023-12-27 14:11:54,779][20381] Updated weights for policy 1, policy_version 1232933 (0.0005) [2023-12-27 14:11:54,830][20381] Updated weights for policy 1, policy_version 1232943 (0.0005) [2023-12-27 14:11:55,142][20347] Updated weights for policy 0, policy_version 1234022 (0.0009) [2023-12-27 14:11:55,192][20347] Updated weights for policy 0, policy_version 1234032 (0.0007) [2023-12-27 14:11:55,246][20347] Updated weights for policy 0, policy_version 1234042 (0.0007) [2023-12-27 14:11:55,462][20381] Updated weights for policy 1, policy_version 1232953 (0.0009) [2023-12-27 14:11:55,523][20381] Updated weights for policy 1, policy_version 1232963 (0.0010) [2023-12-27 14:11:55,572][20381] Updated weights for policy 1, policy_version 1232973 (0.0009) [2023-12-27 14:11:55,618][20381] Updated weights for policy 1, policy_version 1232983 (0.0005) [2023-12-27 14:11:55,888][20347] Updated weights for policy 0, policy_version 1234052 (0.0007) [2023-12-27 14:11:55,937][20347] Updated weights for policy 0, policy_version 1234062 (0.0009) [2023-12-27 14:11:55,995][20347] Updated weights for policy 0, policy_version 1234072 (0.0010) [2023-12-27 14:11:56,244][20381] Updated weights for policy 1, policy_version 1232993 (0.0007) [2023-12-27 14:11:56,303][20381] Updated weights for policy 1, policy_version 1233003 (0.0006) [2023-12-27 14:11:56,353][20381] Updated weights for policy 1, policy_version 1233013 (0.0005) [2023-12-27 14:11:56,602][20347] Updated weights for policy 0, policy_version 1234082 (0.0009) [2023-12-27 14:11:56,652][20347] Updated weights for policy 0, policy_version 1234092 (0.0005) [2023-12-27 14:11:56,704][20347] Updated weights for policy 0, policy_version 1234102 (0.0008) [2023-12-27 14:11:56,751][20347] Updated weights for policy 0, policy_version 1234112 (0.0010) [2023-12-27 14:11:57,005][20381] Updated weights for policy 1, policy_version 1233023 (0.0009) [2023-12-27 14:11:57,066][20381] Updated weights for policy 1, policy_version 1233033 (0.0008) [2023-12-27 14:11:57,131][20381] Updated weights for policy 1, policy_version 1233043 (0.0008) [2023-12-27 14:11:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 631676928. Throughput: 0: 10347.3, 1: 10397.9. Samples: 631666512. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:11:57,363][19170] Avg episode reward: [(0, '970.730'), (1, '990.280')] [2023-12-27 14:11:57,435][20347] Updated weights for policy 0, policy_version 1234122 (0.0010) [2023-12-27 14:11:57,486][20347] Updated weights for policy 0, policy_version 1234132 (0.0010) [2023-12-27 14:11:57,532][20347] Updated weights for policy 0, policy_version 1234142 (0.0006) [2023-12-27 14:11:57,807][20381] Updated weights for policy 1, policy_version 1233053 (0.0007) [2023-12-27 14:11:57,878][20381] Updated weights for policy 1, policy_version 1233063 (0.0005) [2023-12-27 14:11:57,936][20381] Updated weights for policy 1, policy_version 1233073 (0.0006) [2023-12-27 14:11:58,103][20347] Updated weights for policy 0, policy_version 1234152 (0.0006) [2023-12-27 14:11:58,165][20347] Updated weights for policy 0, policy_version 1234162 (0.0006) [2023-12-27 14:11:58,226][20347] Updated weights for policy 0, policy_version 1234172 (0.0007) [2023-12-27 14:11:58,666][20381] Updated weights for policy 1, policy_version 1233083 (0.0007) [2023-12-27 14:11:58,729][20381] Updated weights for policy 1, policy_version 1233093 (0.0008) [2023-12-27 14:11:58,793][20381] Updated weights for policy 1, policy_version 1233103 (0.0008) [2023-12-27 14:11:58,954][20347] Updated weights for policy 0, policy_version 1234182 (0.0008) [2023-12-27 14:11:59,015][20347] Updated weights for policy 0, policy_version 1234192 (0.0008) [2023-12-27 14:11:59,073][20347] Updated weights for policy 0, policy_version 1234202 (0.0007) [2023-12-27 14:11:59,535][20381] Updated weights for policy 1, policy_version 1233113 (0.0008) [2023-12-27 14:11:59,592][20381] Updated weights for policy 1, policy_version 1233123 (0.0009) [2023-12-27 14:11:59,657][20381] Updated weights for policy 1, policy_version 1233133 (0.0011) [2023-12-27 14:11:59,721][20381] Updated weights for policy 1, policy_version 1233143 (0.0010) [2023-12-27 14:11:59,817][20347] Updated weights for policy 0, policy_version 1234212 (0.0010) [2023-12-27 14:11:59,882][20347] Updated weights for policy 0, policy_version 1234222 (0.0009) [2023-12-27 14:11:59,942][20347] Updated weights for policy 0, policy_version 1234232 (0.0009) [2023-12-27 14:12:00,440][20381] Updated weights for policy 1, policy_version 1233153 (0.0010) [2023-12-27 14:12:00,485][20381] Updated weights for policy 1, policy_version 1233163 (0.0010) [2023-12-27 14:12:00,529][20381] Updated weights for policy 1, policy_version 1233173 (0.0010) [2023-12-27 14:12:00,610][20347] Updated weights for policy 0, policy_version 1234242 (0.0009) [2023-12-27 14:12:00,658][20347] Updated weights for policy 0, policy_version 1234252 (0.0010) [2023-12-27 14:12:00,702][20347] Updated weights for policy 0, policy_version 1234262 (0.0010) [2023-12-27 14:12:00,748][20347] Updated weights for policy 0, policy_version 1234272 (0.0010) [2023-12-27 14:12:01,225][20381] Updated weights for policy 1, policy_version 1233183 (0.0010) [2023-12-27 14:12:01,284][20381] Updated weights for policy 1, policy_version 1233193 (0.0008) [2023-12-27 14:12:01,338][20381] Updated weights for policy 1, policy_version 1233203 (0.0011) [2023-12-27 14:12:01,490][20347] Updated weights for policy 0, policy_version 1234282 (0.0010) [2023-12-27 14:12:01,535][20347] Updated weights for policy 0, policy_version 1234292 (0.0010) [2023-12-27 14:12:01,590][20347] Updated weights for policy 0, policy_version 1234302 (0.0010) [2023-12-27 14:12:02,066][20381] Updated weights for policy 1, policy_version 1233213 (0.0010) [2023-12-27 14:12:02,123][20381] Updated weights for policy 1, policy_version 1233223 (0.0008) [2023-12-27 14:12:02,179][20381] Updated weights for policy 1, policy_version 1233233 (0.0007) [2023-12-27 14:12:02,241][20347] Updated weights for policy 0, policy_version 1234312 (0.0009) [2023-12-27 14:12:02,307][20347] Updated weights for policy 0, policy_version 1234322 (0.0011) [2023-12-27 14:12:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 631775232. Throughput: 0: 10348.7, 1: 10306.9. Samples: 631787268. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:02,363][19170] Avg episode reward: [(0, '980.520'), (1, '1000.000')] [2023-12-27 14:12:02,371][20347] Updated weights for policy 0, policy_version 1234332 (0.0010) [2023-12-27 14:12:02,880][20381] Updated weights for policy 1, policy_version 1233243 (0.0008) [2023-12-27 14:12:02,942][20381] Updated weights for policy 1, policy_version 1233253 (0.0005) [2023-12-27 14:12:03,002][20381] Updated weights for policy 1, policy_version 1233263 (0.0005) [2023-12-27 14:12:03,016][20347] Updated weights for policy 0, policy_version 1234342 (0.0008) [2023-12-27 14:12:03,066][20347] Updated weights for policy 0, policy_version 1234352 (0.0007) [2023-12-27 14:12:03,116][20347] Updated weights for policy 0, policy_version 1234362 (0.0005) [2023-12-27 14:12:03,501][20381] Updated weights for policy 1, policy_version 1233273 (0.0005) [2023-12-27 14:12:03,549][20381] Updated weights for policy 1, policy_version 1233283 (0.0005) [2023-12-27 14:12:03,600][20381] Updated weights for policy 1, policy_version 1233293 (0.0006) [2023-12-27 14:12:03,647][20381] Updated weights for policy 1, policy_version 1233303 (0.0010) [2023-12-27 14:12:03,802][20347] Updated weights for policy 0, policy_version 1234372 (0.0006) [2023-12-27 14:12:03,856][20347] Updated weights for policy 0, policy_version 1234382 (0.0008) [2023-12-27 14:12:03,929][20347] Updated weights for policy 0, policy_version 1234392 (0.0006) [2023-12-27 14:12:04,344][20381] Updated weights for policy 1, policy_version 1233313 (0.0008) [2023-12-27 14:12:04,404][20381] Updated weights for policy 1, policy_version 1233323 (0.0007) [2023-12-27 14:12:04,464][20381] Updated weights for policy 1, policy_version 1233333 (0.0007) [2023-12-27 14:12:04,566][20347] Updated weights for policy 0, policy_version 1234402 (0.0008) [2023-12-27 14:12:04,620][20347] Updated weights for policy 0, policy_version 1234412 (0.0007) [2023-12-27 14:12:04,677][20347] Updated weights for policy 0, policy_version 1234422 (0.0007) [2023-12-27 14:12:04,732][20347] Updated weights for policy 0, policy_version 1234432 (0.0007) [2023-12-27 14:12:05,145][20381] Updated weights for policy 1, policy_version 1233343 (0.0006) [2023-12-27 14:12:05,192][20381] Updated weights for policy 1, policy_version 1233353 (0.0005) [2023-12-27 14:12:05,246][20381] Updated weights for policy 1, policy_version 1233363 (0.0007) [2023-12-27 14:12:05,319][20347] Updated weights for policy 0, policy_version 1234442 (0.0009) [2023-12-27 14:12:05,364][20347] Updated weights for policy 0, policy_version 1234452 (0.0010) [2023-12-27 14:12:05,416][20347] Updated weights for policy 0, policy_version 1234462 (0.0010) [2023-12-27 14:12:05,916][20381] Updated weights for policy 1, policy_version 1233373 (0.0009) [2023-12-27 14:12:05,974][20381] Updated weights for policy 1, policy_version 1233383 (0.0006) [2023-12-27 14:12:06,041][20381] Updated weights for policy 1, policy_version 1233393 (0.0005) [2023-12-27 14:12:06,045][20347] Updated weights for policy 0, policy_version 1234472 (0.0008) [2023-12-27 14:12:06,097][20347] Updated weights for policy 0, policy_version 1234482 (0.0010) [2023-12-27 14:12:06,154][20347] Updated weights for policy 0, policy_version 1234492 (0.0010) [2023-12-27 14:12:06,696][20381] Updated weights for policy 1, policy_version 1233403 (0.0006) [2023-12-27 14:12:06,751][20381] Updated weights for policy 1, policy_version 1233413 (0.0007) [2023-12-27 14:12:06,808][20381] Updated weights for policy 1, policy_version 1233423 (0.0007) [2023-12-27 14:12:06,838][20347] Updated weights for policy 0, policy_version 1234502 (0.0011) [2023-12-27 14:12:06,897][20347] Updated weights for policy 0, policy_version 1234512 (0.0011) [2023-12-27 14:12:06,953][20347] Updated weights for policy 0, policy_version 1234522 (0.0010) [2023-12-27 14:12:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 631881728. Throughput: 0: 10357.7, 1: 10389.0. Samples: 631851744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 14:12:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001233432_315785216.pth... [2023-12-27 14:12:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001234528_316096512.pth... [2023-12-27 14:12:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001232216_315473920.pth [2023-12-27 14:12:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001233344_315793408.pth [2023-12-27 14:12:07,436][20381] Updated weights for policy 1, policy_version 1233433 (0.0008) [2023-12-27 14:12:07,491][20381] Updated weights for policy 1, policy_version 1233443 (0.0005) [2023-12-27 14:12:07,554][20381] Updated weights for policy 1, policy_version 1233453 (0.0006) [2023-12-27 14:12:07,564][20347] Updated weights for policy 0, policy_version 1234532 (0.0006) [2023-12-27 14:12:07,612][20381] Updated weights for policy 1, policy_version 1233463 (0.0010) [2023-12-27 14:12:07,621][20347] Updated weights for policy 0, policy_version 1234542 (0.0009) [2023-12-27 14:12:07,670][20347] Updated weights for policy 0, policy_version 1234552 (0.0010) [2023-12-27 14:12:08,260][20381] Updated weights for policy 1, policy_version 1233473 (0.0007) [2023-12-27 14:12:08,317][20381] Updated weights for policy 1, policy_version 1233483 (0.0008) [2023-12-27 14:12:08,373][20347] Updated weights for policy 0, policy_version 1234562 (0.0010) [2023-12-27 14:12:08,381][20381] Updated weights for policy 1, policy_version 1233493 (0.0009) [2023-12-27 14:12:08,436][20347] Updated weights for policy 0, policy_version 1234572 (0.0010) [2023-12-27 14:12:08,500][20347] Updated weights for policy 0, policy_version 1234582 (0.0010) [2023-12-27 14:12:08,563][20347] Updated weights for policy 0, policy_version 1234592 (0.0010) [2023-12-27 14:12:09,073][20381] Updated weights for policy 1, policy_version 1233503 (0.0006) [2023-12-27 14:12:09,129][20381] Updated weights for policy 1, policy_version 1233513 (0.0007) [2023-12-27 14:12:09,183][20381] Updated weights for policy 1, policy_version 1233523 (0.0007) [2023-12-27 14:12:09,249][20347] Updated weights for policy 0, policy_version 1234602 (0.0010) [2023-12-27 14:12:09,312][20347] Updated weights for policy 0, policy_version 1234612 (0.0011) [2023-12-27 14:12:09,377][20347] Updated weights for policy 0, policy_version 1234622 (0.0009) [2023-12-27 14:12:09,869][20381] Updated weights for policy 1, policy_version 1233533 (0.0006) [2023-12-27 14:12:09,924][20381] Updated weights for policy 1, policy_version 1233543 (0.0008) [2023-12-27 14:12:09,986][20381] Updated weights for policy 1, policy_version 1233553 (0.0008) [2023-12-27 14:12:10,137][20347] Updated weights for policy 0, policy_version 1234632 (0.0008) [2023-12-27 14:12:10,193][20347] Updated weights for policy 0, policy_version 1234642 (0.0008) [2023-12-27 14:12:10,241][20347] Updated weights for policy 0, policy_version 1234652 (0.0008) [2023-12-27 14:12:10,675][20381] Updated weights for policy 1, policy_version 1233563 (0.0006) [2023-12-27 14:12:10,733][20381] Updated weights for policy 1, policy_version 1233573 (0.0005) [2023-12-27 14:12:10,790][20381] Updated weights for policy 1, policy_version 1233583 (0.0005) [2023-12-27 14:12:10,917][20347] Updated weights for policy 0, policy_version 1234662 (0.0009) [2023-12-27 14:12:10,971][20347] Updated weights for policy 0, policy_version 1234672 (0.0008) [2023-12-27 14:12:11,029][20347] Updated weights for policy 0, policy_version 1234682 (0.0008) [2023-12-27 14:12:11,421][20381] Updated weights for policy 1, policy_version 1233593 (0.0006) [2023-12-27 14:12:11,480][20381] Updated weights for policy 1, policy_version 1233603 (0.0008) [2023-12-27 14:12:11,534][20381] Updated weights for policy 1, policy_version 1233613 (0.0007) [2023-12-27 14:12:11,586][20381] Updated weights for policy 1, policy_version 1233623 (0.0008) [2023-12-27 14:12:11,763][20347] Updated weights for policy 0, policy_version 1234692 (0.0008) [2023-12-27 14:12:11,819][20347] Updated weights for policy 0, policy_version 1234702 (0.0007) [2023-12-27 14:12:11,871][20347] Updated weights for policy 0, policy_version 1234712 (0.0008) [2023-12-27 14:12:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 631980032. Throughput: 0: 10320.2, 1: 10442.4. Samples: 631974808. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:12,363][19170] Avg episode reward: [(0, '980.420'), (1, '1000.000')] [2023-12-27 14:12:12,370][20381] Updated weights for policy 1, policy_version 1233633 (0.0008) [2023-12-27 14:12:12,440][20381] Updated weights for policy 1, policy_version 1233643 (0.0007) [2023-12-27 14:12:12,502][20381] Updated weights for policy 1, policy_version 1233653 (0.0008) [2023-12-27 14:12:12,570][20347] Updated weights for policy 0, policy_version 1234722 (0.0007) [2023-12-27 14:12:12,630][20347] Updated weights for policy 0, policy_version 1234732 (0.0007) [2023-12-27 14:12:12,689][20347] Updated weights for policy 0, policy_version 1234742 (0.0011) [2023-12-27 14:12:12,748][20347] Updated weights for policy 0, policy_version 1234752 (0.0011) [2023-12-27 14:12:13,139][20381] Updated weights for policy 1, policy_version 1233663 (0.0006) [2023-12-27 14:12:13,196][20381] Updated weights for policy 1, policy_version 1233673 (0.0006) [2023-12-27 14:12:13,253][20381] Updated weights for policy 1, policy_version 1233683 (0.0008) [2023-12-27 14:12:13,398][20347] Updated weights for policy 0, policy_version 1234762 (0.0007) [2023-12-27 14:12:13,453][20347] Updated weights for policy 0, policy_version 1234772 (0.0005) [2023-12-27 14:12:13,509][20347] Updated weights for policy 0, policy_version 1234782 (0.0005) [2023-12-27 14:12:13,924][20381] Updated weights for policy 1, policy_version 1233693 (0.0007) [2023-12-27 14:12:13,976][20381] Updated weights for policy 1, policy_version 1233703 (0.0007) [2023-12-27 14:12:14,037][20381] Updated weights for policy 1, policy_version 1233713 (0.0005) [2023-12-27 14:12:14,123][20347] Updated weights for policy 0, policy_version 1234792 (0.0006) [2023-12-27 14:12:14,183][20347] Updated weights for policy 0, policy_version 1234802 (0.0009) [2023-12-27 14:12:14,232][20347] Updated weights for policy 0, policy_version 1234812 (0.0010) [2023-12-27 14:12:14,600][20381] Updated weights for policy 1, policy_version 1233723 (0.0007) [2023-12-27 14:12:14,647][20381] Updated weights for policy 1, policy_version 1233733 (0.0006) [2023-12-27 14:12:14,692][20381] Updated weights for policy 1, policy_version 1233743 (0.0006) [2023-12-27 14:12:14,794][20347] Updated weights for policy 0, policy_version 1234822 (0.0008) [2023-12-27 14:12:14,862][20347] Updated weights for policy 0, policy_version 1234832 (0.0008) [2023-12-27 14:12:14,927][20347] Updated weights for policy 0, policy_version 1234842 (0.0008) [2023-12-27 14:12:15,343][20381] Updated weights for policy 1, policy_version 1233753 (0.0006) [2023-12-27 14:12:15,402][20381] Updated weights for policy 1, policy_version 1233763 (0.0007) [2023-12-27 14:12:15,458][20381] Updated weights for policy 1, policy_version 1233773 (0.0007) [2023-12-27 14:12:15,522][20381] Updated weights for policy 1, policy_version 1233783 (0.0008) [2023-12-27 14:12:15,595][20347] Updated weights for policy 0, policy_version 1234852 (0.0008) [2023-12-27 14:12:15,653][20347] Updated weights for policy 0, policy_version 1234862 (0.0007) [2023-12-27 14:12:15,708][20347] Updated weights for policy 0, policy_version 1234872 (0.0008) [2023-12-27 14:12:16,178][20381] Updated weights for policy 1, policy_version 1233793 (0.0007) [2023-12-27 14:12:16,232][20381] Updated weights for policy 1, policy_version 1233803 (0.0006) [2023-12-27 14:12:16,280][20381] Updated weights for policy 1, policy_version 1233813 (0.0005) [2023-12-27 14:12:16,408][20347] Updated weights for policy 0, policy_version 1234882 (0.0007) [2023-12-27 14:12:16,461][20347] Updated weights for policy 0, policy_version 1234892 (0.0006) [2023-12-27 14:12:16,508][20347] Updated weights for policy 0, policy_version 1234902 (0.0007) [2023-12-27 14:12:16,552][20347] Updated weights for policy 0, policy_version 1234912 (0.0007) [2023-12-27 14:12:16,934][20381] Updated weights for policy 1, policy_version 1233823 (0.0005) [2023-12-27 14:12:16,987][20381] Updated weights for policy 1, policy_version 1233833 (0.0006) [2023-12-27 14:12:17,036][20381] Updated weights for policy 1, policy_version 1233843 (0.0005) [2023-12-27 14:12:17,265][20347] Updated weights for policy 0, policy_version 1234922 (0.0007) [2023-12-27 14:12:17,314][20347] Updated weights for policy 0, policy_version 1234932 (0.0008) [2023-12-27 14:12:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 632086528. Throughput: 0: 10322.0, 1: 10439.5. Samples: 632100308. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:17,363][19170] Avg episode reward: [(0, '980.510'), (1, '1000.000')] [2023-12-27 14:12:17,365][20347] Updated weights for policy 0, policy_version 1234942 (0.0007) [2023-12-27 14:12:17,675][20381] Updated weights for policy 1, policy_version 1233853 (0.0006) [2023-12-27 14:12:17,740][20381] Updated weights for policy 1, policy_version 1233863 (0.0007) [2023-12-27 14:12:17,788][20381] Updated weights for policy 1, policy_version 1233873 (0.0008) [2023-12-27 14:12:18,073][20347] Updated weights for policy 0, policy_version 1234952 (0.0007) [2023-12-27 14:12:18,128][20347] Updated weights for policy 0, policy_version 1234962 (0.0008) [2023-12-27 14:12:18,183][20347] Updated weights for policy 0, policy_version 1234972 (0.0007) [2023-12-27 14:12:18,454][20381] Updated weights for policy 1, policy_version 1233883 (0.0007) [2023-12-27 14:12:18,515][20381] Updated weights for policy 1, policy_version 1233893 (0.0008) [2023-12-27 14:12:18,584][20381] Updated weights for policy 1, policy_version 1233903 (0.0008) [2023-12-27 14:12:18,894][20347] Updated weights for policy 0, policy_version 1234982 (0.0008) [2023-12-27 14:12:18,951][20347] Updated weights for policy 0, policy_version 1234992 (0.0006) [2023-12-27 14:12:19,011][20347] Updated weights for policy 0, policy_version 1235002 (0.0005) [2023-12-27 14:12:19,294][20381] Updated weights for policy 1, policy_version 1233913 (0.0007) [2023-12-27 14:12:19,357][20381] Updated weights for policy 1, policy_version 1233923 (0.0008) [2023-12-27 14:12:19,415][20381] Updated weights for policy 1, policy_version 1233933 (0.0011) [2023-12-27 14:12:19,471][20381] Updated weights for policy 1, policy_version 1233943 (0.0011) [2023-12-27 14:12:19,667][20347] Updated weights for policy 0, policy_version 1235012 (0.0006) [2023-12-27 14:12:19,736][20347] Updated weights for policy 0, policy_version 1235022 (0.0008) [2023-12-27 14:12:19,799][20347] Updated weights for policy 0, policy_version 1235032 (0.0009) [2023-12-27 14:12:20,192][20381] Updated weights for policy 1, policy_version 1233953 (0.0011) [2023-12-27 14:12:20,253][20381] Updated weights for policy 1, policy_version 1233963 (0.0010) [2023-12-27 14:12:20,315][20381] Updated weights for policy 1, policy_version 1233973 (0.0011) [2023-12-27 14:12:20,519][20347] Updated weights for policy 0, policy_version 1235042 (0.0009) [2023-12-27 14:12:20,578][20347] Updated weights for policy 0, policy_version 1235052 (0.0008) [2023-12-27 14:12:20,642][20347] Updated weights for policy 0, policy_version 1235062 (0.0008) [2023-12-27 14:12:20,703][20347] Updated weights for policy 0, policy_version 1235072 (0.0007) [2023-12-27 14:12:21,008][20381] Updated weights for policy 1, policy_version 1233983 (0.0011) [2023-12-27 14:12:21,078][20381] Updated weights for policy 1, policy_version 1233993 (0.0011) [2023-12-27 14:12:21,147][20381] Updated weights for policy 1, policy_version 1234003 (0.0012) [2023-12-27 14:12:21,453][20347] Updated weights for policy 0, policy_version 1235082 (0.0008) [2023-12-27 14:12:21,512][20347] Updated weights for policy 0, policy_version 1235092 (0.0008) [2023-12-27 14:12:21,570][20347] Updated weights for policy 0, policy_version 1235102 (0.0008) [2023-12-27 14:12:21,858][20381] Updated weights for policy 1, policy_version 1234013 (0.0010) [2023-12-27 14:12:21,925][20381] Updated weights for policy 1, policy_version 1234023 (0.0008) [2023-12-27 14:12:21,989][20381] Updated weights for policy 1, policy_version 1234033 (0.0008) [2023-12-27 14:12:22,283][20347] Updated weights for policy 0, policy_version 1235112 (0.0010) [2023-12-27 14:12:22,341][20347] Updated weights for policy 0, policy_version 1235122 (0.0010) [2023-12-27 14:12:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 632184832. Throughput: 0: 10260.3, 1: 10416.5. Samples: 632160512. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:22,363][19170] Avg episode reward: [(0, '980.530'), (1, '970.750')] [2023-12-27 14:12:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001234040_315940864.pth... [2023-12-27 14:12:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001232824_315629568.pth [2023-12-27 14:12:22,412][20347] Updated weights for policy 0, policy_version 1235132 (0.0009) [2023-12-27 14:12:22,436][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001235136_316252160.pth... [2023-12-27 14:12:22,441][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001233920_315940864.pth [2023-12-27 14:12:22,641][20381] Updated weights for policy 1, policy_version 1234043 (0.0007) [2023-12-27 14:12:22,701][20381] Updated weights for policy 1, policy_version 1234053 (0.0011) [2023-12-27 14:12:22,761][20381] Updated weights for policy 1, policy_version 1234063 (0.0011) [2023-12-27 14:12:23,082][20347] Updated weights for policy 0, policy_version 1235142 (0.0008) [2023-12-27 14:12:23,133][20347] Updated weights for policy 0, policy_version 1235152 (0.0005) [2023-12-27 14:12:23,184][20347] Updated weights for policy 0, policy_version 1235162 (0.0006) [2023-12-27 14:12:23,438][20381] Updated weights for policy 1, policy_version 1234073 (0.0011) [2023-12-27 14:12:23,498][20381] Updated weights for policy 1, policy_version 1234083 (0.0011) [2023-12-27 14:12:23,553][20381] Updated weights for policy 1, policy_version 1234093 (0.0010) [2023-12-27 14:12:23,602][20381] Updated weights for policy 1, policy_version 1234103 (0.0010) [2023-12-27 14:12:23,865][20347] Updated weights for policy 0, policy_version 1235172 (0.0006) [2023-12-27 14:12:23,915][20347] Updated weights for policy 0, policy_version 1235182 (0.0009) [2023-12-27 14:12:23,966][20347] Updated weights for policy 0, policy_version 1235192 (0.0009) [2023-12-27 14:12:24,270][20381] Updated weights for policy 1, policy_version 1234113 (0.0007) [2023-12-27 14:12:24,326][20381] Updated weights for policy 1, policy_version 1234123 (0.0006) [2023-12-27 14:12:24,381][20381] Updated weights for policy 1, policy_version 1234133 (0.0006) [2023-12-27 14:12:24,611][20347] Updated weights for policy 0, policy_version 1235202 (0.0007) [2023-12-27 14:12:24,667][20347] Updated weights for policy 0, policy_version 1235212 (0.0006) [2023-12-27 14:12:24,717][20347] Updated weights for policy 0, policy_version 1235222 (0.0007) [2023-12-27 14:12:24,767][20347] Updated weights for policy 0, policy_version 1235232 (0.0006) [2023-12-27 14:12:24,977][20381] Updated weights for policy 1, policy_version 1234143 (0.0005) [2023-12-27 14:12:25,028][20381] Updated weights for policy 1, policy_version 1234153 (0.0005) [2023-12-27 14:12:25,079][20381] Updated weights for policy 1, policy_version 1234163 (0.0005) [2023-12-27 14:12:25,445][20347] Updated weights for policy 0, policy_version 1235242 (0.0008) [2023-12-27 14:12:25,490][20347] Updated weights for policy 0, policy_version 1235252 (0.0008) [2023-12-27 14:12:25,538][20347] Updated weights for policy 0, policy_version 1235262 (0.0006) [2023-12-27 14:12:25,663][20381] Updated weights for policy 1, policy_version 1234173 (0.0005) [2023-12-27 14:12:25,716][20381] Updated weights for policy 1, policy_version 1234183 (0.0005) [2023-12-27 14:12:25,763][20381] Updated weights for policy 1, policy_version 1234193 (0.0005) [2023-12-27 14:12:26,246][20347] Updated weights for policy 0, policy_version 1235272 (0.0007) [2023-12-27 14:12:26,309][20347] Updated weights for policy 0, policy_version 1235282 (0.0008) [2023-12-27 14:12:26,331][20381] Updated weights for policy 1, policy_version 1234203 (0.0007) [2023-12-27 14:12:26,367][20347] Updated weights for policy 0, policy_version 1235292 (0.0006) [2023-12-27 14:12:26,386][20381] Updated weights for policy 1, policy_version 1234213 (0.0007) [2023-12-27 14:12:26,442][20381] Updated weights for policy 1, policy_version 1234223 (0.0008) [2023-12-27 14:12:26,930][20347] Updated weights for policy 0, policy_version 1235302 (0.0006) [2023-12-27 14:12:26,979][20347] Updated weights for policy 0, policy_version 1235312 (0.0007) [2023-12-27 14:12:27,024][20347] Updated weights for policy 0, policy_version 1235322 (0.0007) [2023-12-27 14:12:27,097][20381] Updated weights for policy 1, policy_version 1234233 (0.0008) [2023-12-27 14:12:27,170][20381] Updated weights for policy 1, policy_version 1234243 (0.0008) [2023-12-27 14:12:27,219][20381] Updated weights for policy 1, policy_version 1234253 (0.0008) [2023-12-27 14:12:27,274][20381] Updated weights for policy 1, policy_version 1234263 (0.0008) [2023-12-27 14:12:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.7, 300 sec: 20605.0). Total num frames: 632299520. Throughput: 0: 10270.6, 1: 10466.4. Samples: 632284336. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:27,363][19170] Avg episode reward: [(0, '951.250'), (1, '951.260')] [2023-12-27 14:12:27,740][20347] Updated weights for policy 0, policy_version 1235332 (0.0007) [2023-12-27 14:12:27,796][20347] Updated weights for policy 0, policy_version 1235342 (0.0005) [2023-12-27 14:12:27,854][20347] Updated weights for policy 0, policy_version 1235352 (0.0005) [2023-12-27 14:12:27,943][20381] Updated weights for policy 1, policy_version 1234273 (0.0007) [2023-12-27 14:12:27,988][20381] Updated weights for policy 1, policy_version 1234283 (0.0007) [2023-12-27 14:12:28,041][20381] Updated weights for policy 1, policy_version 1234293 (0.0007) [2023-12-27 14:12:28,472][20347] Updated weights for policy 0, policy_version 1235362 (0.0007) [2023-12-27 14:12:28,530][20347] Updated weights for policy 0, policy_version 1235372 (0.0008) [2023-12-27 14:12:28,589][20347] Updated weights for policy 0, policy_version 1235382 (0.0007) [2023-12-27 14:12:28,647][20347] Updated weights for policy 0, policy_version 1235392 (0.0007) [2023-12-27 14:12:28,770][20381] Updated weights for policy 1, policy_version 1234303 (0.0009) [2023-12-27 14:12:28,832][20381] Updated weights for policy 1, policy_version 1234313 (0.0008) [2023-12-27 14:12:28,887][20381] Updated weights for policy 1, policy_version 1234323 (0.0008) [2023-12-27 14:12:29,321][20347] Updated weights for policy 0, policy_version 1235402 (0.0008) [2023-12-27 14:12:29,390][20347] Updated weights for policy 0, policy_version 1235412 (0.0008) [2023-12-27 14:12:29,451][20347] Updated weights for policy 0, policy_version 1235422 (0.0007) [2023-12-27 14:12:29,590][20381] Updated weights for policy 1, policy_version 1234333 (0.0008) [2023-12-27 14:12:29,648][20381] Updated weights for policy 1, policy_version 1234343 (0.0008) [2023-12-27 14:12:29,713][20381] Updated weights for policy 1, policy_version 1234353 (0.0008) [2023-12-27 14:12:30,130][20347] Updated weights for policy 0, policy_version 1235432 (0.0007) [2023-12-27 14:12:30,194][20347] Updated weights for policy 0, policy_version 1235442 (0.0006) [2023-12-27 14:12:30,256][20347] Updated weights for policy 0, policy_version 1235452 (0.0008) [2023-12-27 14:12:30,443][20381] Updated weights for policy 1, policy_version 1234363 (0.0008) [2023-12-27 14:12:30,507][20381] Updated weights for policy 1, policy_version 1234373 (0.0008) [2023-12-27 14:12:30,574][20381] Updated weights for policy 1, policy_version 1234383 (0.0008) [2023-12-27 14:12:30,820][20347] Updated weights for policy 0, policy_version 1235462 (0.0007) [2023-12-27 14:12:30,871][20347] Updated weights for policy 0, policy_version 1235472 (0.0005) [2023-12-27 14:12:30,914][20347] Updated weights for policy 0, policy_version 1235482 (0.0005) [2023-12-27 14:12:31,306][20381] Updated weights for policy 1, policy_version 1234393 (0.0007) [2023-12-27 14:12:31,370][20381] Updated weights for policy 1, policy_version 1234403 (0.0008) [2023-12-27 14:12:31,430][20381] Updated weights for policy 1, policy_version 1234413 (0.0011) [2023-12-27 14:12:31,496][20381] Updated weights for policy 1, policy_version 1234423 (0.0011) [2023-12-27 14:12:31,558][20347] Updated weights for policy 0, policy_version 1235492 (0.0005) [2023-12-27 14:12:31,617][20347] Updated weights for policy 0, policy_version 1235502 (0.0008) [2023-12-27 14:12:31,686][20347] Updated weights for policy 0, policy_version 1235512 (0.0008) [2023-12-27 14:12:32,189][20381] Updated weights for policy 1, policy_version 1234433 (0.0008) [2023-12-27 14:12:32,237][20381] Updated weights for policy 1, policy_version 1234443 (0.0009) [2023-12-27 14:12:32,291][20381] Updated weights for policy 1, policy_version 1234453 (0.0007) [2023-12-27 14:12:32,319][20347] Updated weights for policy 0, policy_version 1235522 (0.0009) [2023-12-27 14:12:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 632397824. Throughput: 0: 10385.8, 1: 10317.8. Samples: 632406712. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:32,363][19170] Avg episode reward: [(0, '951.210'), (1, '970.830')] [2023-12-27 14:12:32,383][20347] Updated weights for policy 0, policy_version 1235532 (0.0008) [2023-12-27 14:12:32,439][20347] Updated weights for policy 0, policy_version 1235542 (0.0007) [2023-12-27 14:12:32,488][20347] Updated weights for policy 0, policy_version 1235552 (0.0008) [2023-12-27 14:12:32,956][20381] Updated weights for policy 1, policy_version 1234463 (0.0005) [2023-12-27 14:12:33,006][20381] Updated weights for policy 1, policy_version 1234473 (0.0005) [2023-12-27 14:12:33,051][20381] Updated weights for policy 1, policy_version 1234483 (0.0005) [2023-12-27 14:12:33,179][20347] Updated weights for policy 0, policy_version 1235562 (0.0008) [2023-12-27 14:12:33,234][20347] Updated weights for policy 0, policy_version 1235572 (0.0007) [2023-12-27 14:12:33,290][20347] Updated weights for policy 0, policy_version 1235582 (0.0008) [2023-12-27 14:12:33,600][20381] Updated weights for policy 1, policy_version 1234493 (0.0008) [2023-12-27 14:12:33,650][20381] Updated weights for policy 1, policy_version 1234503 (0.0010) [2023-12-27 14:12:33,698][20381] Updated weights for policy 1, policy_version 1234513 (0.0010) [2023-12-27 14:12:34,022][20347] Updated weights for policy 0, policy_version 1235592 (0.0008) [2023-12-27 14:12:34,078][20347] Updated weights for policy 0, policy_version 1235602 (0.0008) [2023-12-27 14:12:34,141][20347] Updated weights for policy 0, policy_version 1235612 (0.0008) [2023-12-27 14:12:34,337][20381] Updated weights for policy 1, policy_version 1234523 (0.0009) [2023-12-27 14:12:34,391][20381] Updated weights for policy 1, policy_version 1234533 (0.0008) [2023-12-27 14:12:34,444][20381] Updated weights for policy 1, policy_version 1234543 (0.0008) [2023-12-27 14:12:34,773][20347] Updated weights for policy 0, policy_version 1235622 (0.0007) [2023-12-27 14:12:34,825][20347] Updated weights for policy 0, policy_version 1235632 (0.0005) [2023-12-27 14:12:34,875][20347] Updated weights for policy 0, policy_version 1235642 (0.0006) [2023-12-27 14:12:35,121][20381] Updated weights for policy 1, policy_version 1234553 (0.0009) [2023-12-27 14:12:35,170][20381] Updated weights for policy 1, policy_version 1234563 (0.0010) [2023-12-27 14:12:35,218][20381] Updated weights for policy 1, policy_version 1234573 (0.0010) [2023-12-27 14:12:35,263][20381] Updated weights for policy 1, policy_version 1234583 (0.0010) [2023-12-27 14:12:35,523][20347] Updated weights for policy 0, policy_version 1235652 (0.0007) [2023-12-27 14:12:35,615][20347] Updated weights for policy 0, policy_version 1235662 (0.0008) [2023-12-27 14:12:35,682][20347] Updated weights for policy 0, policy_version 1235672 (0.0008) [2023-12-27 14:12:35,864][20381] Updated weights for policy 1, policy_version 1234593 (0.0006) [2023-12-27 14:12:35,917][20381] Updated weights for policy 1, policy_version 1234603 (0.0007) [2023-12-27 14:12:35,972][20381] Updated weights for policy 1, policy_version 1234613 (0.0008) [2023-12-27 14:12:36,396][20347] Updated weights for policy 0, policy_version 1235682 (0.0008) [2023-12-27 14:12:36,455][20347] Updated weights for policy 0, policy_version 1235692 (0.0008) [2023-12-27 14:12:36,519][20347] Updated weights for policy 0, policy_version 1235702 (0.0008) [2023-12-27 14:12:36,575][20347] Updated weights for policy 0, policy_version 1235712 (0.0007) [2023-12-27 14:12:36,631][20381] Updated weights for policy 1, policy_version 1234623 (0.0008) [2023-12-27 14:12:36,687][20381] Updated weights for policy 1, policy_version 1234633 (0.0007) [2023-12-27 14:12:36,742][20381] Updated weights for policy 1, policy_version 1234643 (0.0007) [2023-12-27 14:12:37,273][20347] Updated weights for policy 0, policy_version 1235722 (0.0008) [2023-12-27 14:12:37,327][20347] Updated weights for policy 0, policy_version 1235732 (0.0007) [2023-12-27 14:12:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 632496128. Throughput: 0: 10359.0, 1: 10352.7. Samples: 632470796. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:37,363][19170] Avg episode reward: [(0, '961.010'), (1, '980.560')] [2023-12-27 14:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001234648_316096512.pth... [2023-12-27 14:12:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001233432_315785216.pth [2023-12-27 14:12:37,380][20347] Updated weights for policy 0, policy_version 1235742 (0.0008) [2023-12-27 14:12:37,389][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001235744_316407808.pth... [2023-12-27 14:12:37,392][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001234528_316096512.pth [2023-12-27 14:12:37,437][20381] Updated weights for policy 1, policy_version 1234653 (0.0007) [2023-12-27 14:12:37,495][20381] Updated weights for policy 1, policy_version 1234663 (0.0007) [2023-12-27 14:12:37,545][20381] Updated weights for policy 1, policy_version 1234673 (0.0007) [2023-12-27 14:12:38,127][20347] Updated weights for policy 0, policy_version 1235752 (0.0007) [2023-12-27 14:12:38,157][20381] Updated weights for policy 1, policy_version 1234683 (0.0007) [2023-12-27 14:12:38,180][20347] Updated weights for policy 0, policy_version 1235762 (0.0007) [2023-12-27 14:12:38,216][20381] Updated weights for policy 1, policy_version 1234693 (0.0006) [2023-12-27 14:12:38,229][20347] Updated weights for policy 0, policy_version 1235772 (0.0008) [2023-12-27 14:12:38,273][20381] Updated weights for policy 1, policy_version 1234703 (0.0005) [2023-12-27 14:12:38,911][20381] Updated weights for policy 1, policy_version 1234713 (0.0007) [2023-12-27 14:12:38,945][20347] Updated weights for policy 0, policy_version 1235782 (0.0008) [2023-12-27 14:12:38,977][20381] Updated weights for policy 1, policy_version 1234723 (0.0006) [2023-12-27 14:12:39,007][20347] Updated weights for policy 0, policy_version 1235792 (0.0007) [2023-12-27 14:12:39,031][20381] Updated weights for policy 1, policy_version 1234733 (0.0005) [2023-12-27 14:12:39,071][20347] Updated weights for policy 0, policy_version 1235802 (0.0006) [2023-12-27 14:12:39,092][20381] Updated weights for policy 1, policy_version 1234743 (0.0005) [2023-12-27 14:12:39,795][20381] Updated weights for policy 1, policy_version 1234753 (0.0009) [2023-12-27 14:12:39,808][20347] Updated weights for policy 0, policy_version 1235812 (0.0008) [2023-12-27 14:12:39,866][20381] Updated weights for policy 1, policy_version 1234763 (0.0008) [2023-12-27 14:12:39,874][20347] Updated weights for policy 0, policy_version 1235822 (0.0009) [2023-12-27 14:12:39,929][20381] Updated weights for policy 1, policy_version 1234773 (0.0006) [2023-12-27 14:12:39,937][20347] Updated weights for policy 0, policy_version 1235832 (0.0008) [2023-12-27 14:12:40,553][20381] Updated weights for policy 1, policy_version 1234783 (0.0006) [2023-12-27 14:12:40,616][20381] Updated weights for policy 1, policy_version 1234793 (0.0005) [2023-12-27 14:12:40,622][20347] Updated weights for policy 0, policy_version 1235842 (0.0007) [2023-12-27 14:12:40,677][20347] Updated weights for policy 0, policy_version 1235852 (0.0005) [2023-12-27 14:12:40,677][20381] Updated weights for policy 1, policy_version 1234803 (0.0006) [2023-12-27 14:12:40,732][20347] Updated weights for policy 0, policy_version 1235862 (0.0005) [2023-12-27 14:12:40,795][20347] Updated weights for policy 0, policy_version 1235872 (0.0008) [2023-12-27 14:12:41,316][20381] Updated weights for policy 1, policy_version 1234813 (0.0007) [2023-12-27 14:12:41,381][20381] Updated weights for policy 1, policy_version 1234823 (0.0008) [2023-12-27 14:12:41,440][20381] Updated weights for policy 1, policy_version 1234833 (0.0008) [2023-12-27 14:12:41,506][20347] Updated weights for policy 0, policy_version 1235882 (0.0005) [2023-12-27 14:12:41,559][20347] Updated weights for policy 0, policy_version 1235892 (0.0005) [2023-12-27 14:12:41,609][20347] Updated weights for policy 0, policy_version 1235902 (0.0009) [2023-12-27 14:12:42,215][20381] Updated weights for policy 1, policy_version 1234843 (0.0008) [2023-12-27 14:12:42,276][20381] Updated weights for policy 1, policy_version 1234853 (0.0008) [2023-12-27 14:12:42,277][20347] Updated weights for policy 0, policy_version 1235912 (0.0007) [2023-12-27 14:12:42,331][20347] Updated weights for policy 0, policy_version 1235922 (0.0007) [2023-12-27 14:12:42,337][20381] Updated weights for policy 1, policy_version 1234863 (0.0008) [2023-12-27 14:12:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 632594432. Throughput: 0: 10260.0, 1: 10339.5. Samples: 632593492. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:42,363][19170] Avg episode reward: [(0, '990.260'), (1, '970.720')] [2023-12-27 14:12:42,396][20347] Updated weights for policy 0, policy_version 1235932 (0.0007) [2023-12-27 14:12:43,035][20381] Updated weights for policy 1, policy_version 1234873 (0.0008) [2023-12-27 14:12:43,068][20347] Updated weights for policy 0, policy_version 1235942 (0.0009) [2023-12-27 14:12:43,081][20381] Updated weights for policy 1, policy_version 1234883 (0.0006) [2023-12-27 14:12:43,118][20347] Updated weights for policy 0, policy_version 1235952 (0.0009) [2023-12-27 14:12:43,134][20381] Updated weights for policy 1, policy_version 1234893 (0.0008) [2023-12-27 14:12:43,173][20347] Updated weights for policy 0, policy_version 1235962 (0.0010) [2023-12-27 14:12:43,186][20381] Updated weights for policy 1, policy_version 1234903 (0.0005) [2023-12-27 14:12:43,826][20347] Updated weights for policy 0, policy_version 1235972 (0.0008) [2023-12-27 14:12:43,885][20347] Updated weights for policy 0, policy_version 1235982 (0.0005) [2023-12-27 14:12:43,901][20381] Updated weights for policy 1, policy_version 1234913 (0.0006) [2023-12-27 14:12:43,938][20347] Updated weights for policy 0, policy_version 1235992 (0.0006) [2023-12-27 14:12:43,957][20381] Updated weights for policy 1, policy_version 1234923 (0.0006) [2023-12-27 14:12:44,009][20381] Updated weights for policy 1, policy_version 1234933 (0.0008) [2023-12-27 14:12:44,558][20347] Updated weights for policy 0, policy_version 1236002 (0.0006) [2023-12-27 14:12:44,610][20347] Updated weights for policy 0, policy_version 1236012 (0.0008) [2023-12-27 14:12:44,670][20347] Updated weights for policy 0, policy_version 1236022 (0.0008) [2023-12-27 14:12:44,690][20381] Updated weights for policy 1, policy_version 1234943 (0.0008) [2023-12-27 14:12:44,725][20347] Updated weights for policy 0, policy_version 1236032 (0.0006) [2023-12-27 14:12:44,742][20381] Updated weights for policy 1, policy_version 1234953 (0.0010) [2023-12-27 14:12:44,804][20381] Updated weights for policy 1, policy_version 1234963 (0.0010) [2023-12-27 14:12:45,427][20347] Updated weights for policy 0, policy_version 1236042 (0.0006) [2023-12-27 14:12:45,479][20381] Updated weights for policy 1, policy_version 1234973 (0.0008) [2023-12-27 14:12:45,484][20347] Updated weights for policy 0, policy_version 1236052 (0.0007) [2023-12-27 14:12:45,527][20381] Updated weights for policy 1, policy_version 1234983 (0.0006) [2023-12-27 14:12:45,533][20347] Updated weights for policy 0, policy_version 1236062 (0.0007) [2023-12-27 14:12:45,575][20381] Updated weights for policy 1, policy_version 1234993 (0.0007) [2023-12-27 14:12:46,208][20381] Updated weights for policy 1, policy_version 1235003 (0.0006) [2023-12-27 14:12:46,234][20347] Updated weights for policy 0, policy_version 1236072 (0.0007) [2023-12-27 14:12:46,264][20381] Updated weights for policy 1, policy_version 1235013 (0.0006) [2023-12-27 14:12:46,290][20347] Updated weights for policy 0, policy_version 1236082 (0.0008) [2023-12-27 14:12:46,313][20381] Updated weights for policy 1, policy_version 1235023 (0.0005) [2023-12-27 14:12:46,344][20347] Updated weights for policy 0, policy_version 1236092 (0.0007) [2023-12-27 14:12:46,913][20381] Updated weights for policy 1, policy_version 1235033 (0.0008) [2023-12-27 14:12:46,967][20381] Updated weights for policy 1, policy_version 1235043 (0.0006) [2023-12-27 14:12:47,014][20381] Updated weights for policy 1, policy_version 1235053 (0.0007) [2023-12-27 14:12:47,067][20347] Updated weights for policy 0, policy_version 1236102 (0.0007) [2023-12-27 14:12:47,069][20381] Updated weights for policy 1, policy_version 1235063 (0.0007) [2023-12-27 14:12:47,116][20347] Updated weights for policy 0, policy_version 1236112 (0.0007) [2023-12-27 14:12:47,164][20347] Updated weights for policy 0, policy_version 1236122 (0.0007) [2023-12-27 14:12:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 632709120. Throughput: 0: 10234.8, 1: 10399.1. Samples: 632715792. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:47,363][19170] Avg episode reward: [(0, '970.680'), (1, '960.970')] [2023-12-27 14:12:47,715][20381] Updated weights for policy 1, policy_version 1235073 (0.0005) [2023-12-27 14:12:47,763][20381] Updated weights for policy 1, policy_version 1235083 (0.0010) [2023-12-27 14:12:47,805][20347] Updated weights for policy 0, policy_version 1236132 (0.0006) [2023-12-27 14:12:47,816][20381] Updated weights for policy 1, policy_version 1235093 (0.0010) [2023-12-27 14:12:47,860][20347] Updated weights for policy 0, policy_version 1236142 (0.0007) [2023-12-27 14:12:47,909][20347] Updated weights for policy 0, policy_version 1236152 (0.0007) [2023-12-27 14:12:48,489][20381] Updated weights for policy 1, policy_version 1235103 (0.0011) [2023-12-27 14:12:48,548][20381] Updated weights for policy 1, policy_version 1235113 (0.0011) [2023-12-27 14:12:48,576][20347] Updated weights for policy 0, policy_version 1236162 (0.0007) [2023-12-27 14:12:48,610][20381] Updated weights for policy 1, policy_version 1235123 (0.0011) [2023-12-27 14:12:48,635][20347] Updated weights for policy 0, policy_version 1236172 (0.0008) [2023-12-27 14:12:48,694][20347] Updated weights for policy 0, policy_version 1236182 (0.0008) [2023-12-27 14:12:48,743][20347] Updated weights for policy 0, policy_version 1236192 (0.0007) [2023-12-27 14:12:49,219][20381] Updated weights for policy 1, policy_version 1235133 (0.0008) [2023-12-27 14:12:49,280][20381] Updated weights for policy 1, policy_version 1235143 (0.0011) [2023-12-27 14:12:49,342][20381] Updated weights for policy 1, policy_version 1235153 (0.0010) [2023-12-27 14:12:49,491][20347] Updated weights for policy 0, policy_version 1236202 (0.0007) [2023-12-27 14:12:49,545][20347] Updated weights for policy 0, policy_version 1236212 (0.0008) [2023-12-27 14:12:49,597][20347] Updated weights for policy 0, policy_version 1236222 (0.0007) [2023-12-27 14:12:50,035][20381] Updated weights for policy 1, policy_version 1235163 (0.0009) [2023-12-27 14:12:50,097][20381] Updated weights for policy 1, policy_version 1235173 (0.0011) [2023-12-27 14:12:50,156][20381] Updated weights for policy 1, policy_version 1235183 (0.0010) [2023-12-27 14:12:50,345][20347] Updated weights for policy 0, policy_version 1236232 (0.0007) [2023-12-27 14:12:50,406][20347] Updated weights for policy 0, policy_version 1236242 (0.0007) [2023-12-27 14:12:50,465][20347] Updated weights for policy 0, policy_version 1236252 (0.0007) [2023-12-27 14:12:50,845][20381] Updated weights for policy 1, policy_version 1235193 (0.0010) [2023-12-27 14:12:50,907][20381] Updated weights for policy 1, policy_version 1235203 (0.0010) [2023-12-27 14:12:50,966][20381] Updated weights for policy 1, policy_version 1235213 (0.0010) [2023-12-27 14:12:51,028][20381] Updated weights for policy 1, policy_version 1235223 (0.0010) [2023-12-27 14:12:51,178][20347] Updated weights for policy 0, policy_version 1236262 (0.0008) [2023-12-27 14:12:51,228][20347] Updated weights for policy 0, policy_version 1236272 (0.0007) [2023-12-27 14:12:51,290][20347] Updated weights for policy 0, policy_version 1236282 (0.0007) [2023-12-27 14:12:51,768][20381] Updated weights for policy 1, policy_version 1235233 (0.0008) [2023-12-27 14:12:51,820][20381] Updated weights for policy 1, policy_version 1235243 (0.0009) [2023-12-27 14:12:51,885][20381] Updated weights for policy 1, policy_version 1235253 (0.0006) [2023-12-27 14:12:52,066][20347] Updated weights for policy 0, policy_version 1236292 (0.0007) [2023-12-27 14:12:52,121][20347] Updated weights for policy 0, policy_version 1236302 (0.0008) [2023-12-27 14:12:52,186][20347] Updated weights for policy 0, policy_version 1236312 (0.0008) [2023-12-27 14:12:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 632807424. Throughput: 0: 10204.6, 1: 10359.9. Samples: 632777148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:52,363][19170] Avg episode reward: [(0, '970.680'), (1, '961.010')] [2023-12-27 14:12:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001235256_316252160.pth... [2023-12-27 14:12:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001236320_316555264.pth... [2023-12-27 14:12:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001235136_316252160.pth [2023-12-27 14:12:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001234040_315940864.pth [2023-12-27 14:12:52,616][20381] Updated weights for policy 1, policy_version 1235263 (0.0007) [2023-12-27 14:12:52,674][20381] Updated weights for policy 1, policy_version 1235273 (0.0005) [2023-12-27 14:12:52,734][20381] Updated weights for policy 1, policy_version 1235283 (0.0005) [2023-12-27 14:12:52,931][20347] Updated weights for policy 0, policy_version 1236322 (0.0008) [2023-12-27 14:12:52,980][20347] Updated weights for policy 0, policy_version 1236332 (0.0007) [2023-12-27 14:12:53,033][20347] Updated weights for policy 0, policy_version 1236342 (0.0007) [2023-12-27 14:12:53,086][20347] Updated weights for policy 0, policy_version 1236352 (0.0007) [2023-12-27 14:12:53,316][20381] Updated weights for policy 1, policy_version 1235293 (0.0006) [2023-12-27 14:12:53,362][20381] Updated weights for policy 1, policy_version 1235303 (0.0010) [2023-12-27 14:12:53,420][20381] Updated weights for policy 1, policy_version 1235313 (0.0010) [2023-12-27 14:12:53,789][20347] Updated weights for policy 0, policy_version 1236362 (0.0008) [2023-12-27 14:12:53,835][20347] Updated weights for policy 0, policy_version 1236372 (0.0005) [2023-12-27 14:12:53,889][20347] Updated weights for policy 0, policy_version 1236382 (0.0005) [2023-12-27 14:12:54,063][20381] Updated weights for policy 1, policy_version 1235323 (0.0007) [2023-12-27 14:12:54,134][20381] Updated weights for policy 1, policy_version 1235333 (0.0007) [2023-12-27 14:12:54,199][20381] Updated weights for policy 1, policy_version 1235343 (0.0008) [2023-12-27 14:12:54,538][20347] Updated weights for policy 0, policy_version 1236392 (0.0009) [2023-12-27 14:12:54,606][20347] Updated weights for policy 0, policy_version 1236402 (0.0010) [2023-12-27 14:12:54,667][20347] Updated weights for policy 0, policy_version 1236412 (0.0010) [2023-12-27 14:12:54,817][20381] Updated weights for policy 1, policy_version 1235353 (0.0007) [2023-12-27 14:12:54,871][20381] Updated weights for policy 1, policy_version 1235363 (0.0007) [2023-12-27 14:12:54,918][20381] Updated weights for policy 1, policy_version 1235373 (0.0005) [2023-12-27 14:12:54,977][20381] Updated weights for policy 1, policy_version 1235383 (0.0006) [2023-12-27 14:12:55,320][20347] Updated weights for policy 0, policy_version 1236422 (0.0008) [2023-12-27 14:12:55,376][20347] Updated weights for policy 0, policy_version 1236432 (0.0008) [2023-12-27 14:12:55,435][20347] Updated weights for policy 0, policy_version 1236442 (0.0008) [2023-12-27 14:12:55,644][20381] Updated weights for policy 1, policy_version 1235393 (0.0007) [2023-12-27 14:12:55,695][20381] Updated weights for policy 1, policy_version 1235403 (0.0007) [2023-12-27 14:12:55,744][20381] Updated weights for policy 1, policy_version 1235413 (0.0006) [2023-12-27 14:12:56,113][20347] Updated weights for policy 0, policy_version 1236452 (0.0008) [2023-12-27 14:12:56,172][20347] Updated weights for policy 0, policy_version 1236462 (0.0007) [2023-12-27 14:12:56,231][20347] Updated weights for policy 0, policy_version 1236472 (0.0008) [2023-12-27 14:12:56,336][20381] Updated weights for policy 1, policy_version 1235423 (0.0008) [2023-12-27 14:12:56,394][20381] Updated weights for policy 1, policy_version 1235433 (0.0006) [2023-12-27 14:12:56,446][20381] Updated weights for policy 1, policy_version 1235443 (0.0007) [2023-12-27 14:12:56,763][20347] Updated weights for policy 0, policy_version 1236482 (0.0005) [2023-12-27 14:12:56,816][20347] Updated weights for policy 0, policy_version 1236492 (0.0005) [2023-12-27 14:12:56,876][20347] Updated weights for policy 0, policy_version 1236502 (0.0005) [2023-12-27 14:12:56,930][20347] Updated weights for policy 0, policy_version 1236512 (0.0009) [2023-12-27 14:12:57,089][20381] Updated weights for policy 1, policy_version 1235453 (0.0009) [2023-12-27 14:12:57,139][20381] Updated weights for policy 1, policy_version 1235463 (0.0007) [2023-12-27 14:12:57,187][20381] Updated weights for policy 1, policy_version 1235473 (0.0006) [2023-12-27 14:12:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 632913920. Throughput: 0: 10162.8, 1: 10379.6. Samples: 632899216. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:12:57,363][19170] Avg episode reward: [(0, '970.860'), (1, '980.510')] [2023-12-27 14:12:57,526][20347] Updated weights for policy 0, policy_version 1236522 (0.0007) [2023-12-27 14:12:57,588][20347] Updated weights for policy 0, policy_version 1236532 (0.0006) [2023-12-27 14:12:57,645][20347] Updated weights for policy 0, policy_version 1236542 (0.0005) [2023-12-27 14:12:57,844][20381] Updated weights for policy 1, policy_version 1235483 (0.0008) [2023-12-27 14:12:57,908][20381] Updated weights for policy 1, policy_version 1235493 (0.0005) [2023-12-27 14:12:57,969][20381] Updated weights for policy 1, policy_version 1235503 (0.0005) [2023-12-27 14:12:58,269][20347] Updated weights for policy 0, policy_version 1236552 (0.0007) [2023-12-27 14:12:58,337][20347] Updated weights for policy 0, policy_version 1236562 (0.0009) [2023-12-27 14:12:58,403][20347] Updated weights for policy 0, policy_version 1236572 (0.0009) [2023-12-27 14:12:58,698][20381] Updated weights for policy 1, policy_version 1235513 (0.0007) [2023-12-27 14:12:58,760][20381] Updated weights for policy 1, policy_version 1235523 (0.0008) [2023-12-27 14:12:58,828][20381] Updated weights for policy 1, policy_version 1235533 (0.0008) [2023-12-27 14:12:58,886][20381] Updated weights for policy 1, policy_version 1235543 (0.0007) [2023-12-27 14:12:59,181][20347] Updated weights for policy 0, policy_version 1236582 (0.0007) [2023-12-27 14:12:59,256][20347] Updated weights for policy 0, policy_version 1236593 (0.0008) [2023-12-27 14:12:59,311][20347] Updated weights for policy 0, policy_version 1236603 (0.0006) [2023-12-27 14:12:59,673][20381] Updated weights for policy 1, policy_version 1235553 (0.0007) [2023-12-27 14:12:59,731][20381] Updated weights for policy 1, policy_version 1235563 (0.0007) [2023-12-27 14:12:59,789][20381] Updated weights for policy 1, policy_version 1235573 (0.0005) [2023-12-27 14:12:59,931][20347] Updated weights for policy 0, policy_version 1236613 (0.0008) [2023-12-27 14:12:59,992][20347] Updated weights for policy 0, policy_version 1236623 (0.0010) [2023-12-27 14:13:00,048][20347] Updated weights for policy 0, policy_version 1236633 (0.0010) [2023-12-27 14:13:00,474][20381] Updated weights for policy 1, policy_version 1235583 (0.0009) [2023-12-27 14:13:00,528][20381] Updated weights for policy 1, policy_version 1235593 (0.0010) [2023-12-27 14:13:00,588][20381] Updated weights for policy 1, policy_version 1235603 (0.0011) [2023-12-27 14:13:00,714][20347] Updated weights for policy 0, policy_version 1236643 (0.0009) [2023-12-27 14:13:00,771][20347] Updated weights for policy 0, policy_version 1236653 (0.0010) [2023-12-27 14:13:00,824][20347] Updated weights for policy 0, policy_version 1236663 (0.0010) [2023-12-27 14:13:01,210][20381] Updated weights for policy 1, policy_version 1235613 (0.0010) [2023-12-27 14:13:01,263][20381] Updated weights for policy 1, policy_version 1235623 (0.0007) [2023-12-27 14:13:01,320][20381] Updated weights for policy 1, policy_version 1235633 (0.0007) [2023-12-27 14:13:01,495][20347] Updated weights for policy 0, policy_version 1236673 (0.0010) [2023-12-27 14:13:01,544][20347] Updated weights for policy 0, policy_version 1236683 (0.0007) [2023-12-27 14:13:01,592][20347] Updated weights for policy 0, policy_version 1236693 (0.0008) [2023-12-27 14:13:01,655][20347] Updated weights for policy 0, policy_version 1236703 (0.0008) [2023-12-27 14:13:01,994][20381] Updated weights for policy 1, policy_version 1235643 (0.0007) [2023-12-27 14:13:02,061][20381] Updated weights for policy 1, policy_version 1235653 (0.0005) [2023-12-27 14:13:02,119][20381] Updated weights for policy 1, policy_version 1235663 (0.0006) [2023-12-27 14:13:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 633012224. Throughput: 0: 10152.2, 1: 10344.1. Samples: 633022640. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:02,363][19170] Avg episode reward: [(0, '961.140'), (1, '1000.000')] [2023-12-27 14:13:02,447][20347] Updated weights for policy 0, policy_version 1236713 (0.0007) [2023-12-27 14:13:02,504][20347] Updated weights for policy 0, policy_version 1236723 (0.0007) [2023-12-27 14:13:02,560][20347] Updated weights for policy 0, policy_version 1236733 (0.0005) [2023-12-27 14:13:02,749][20381] Updated weights for policy 1, policy_version 1235673 (0.0007) [2023-12-27 14:13:02,803][20381] Updated weights for policy 1, policy_version 1235683 (0.0010) [2023-12-27 14:13:02,861][20381] Updated weights for policy 1, policy_version 1235693 (0.0009) [2023-12-27 14:13:02,921][20381] Updated weights for policy 1, policy_version 1235703 (0.0005) [2023-12-27 14:13:03,179][20347] Updated weights for policy 0, policy_version 1236743 (0.0009) [2023-12-27 14:13:03,230][20347] Updated weights for policy 0, policy_version 1236753 (0.0010) [2023-12-27 14:13:03,285][20347] Updated weights for policy 0, policy_version 1236763 (0.0008) [2023-12-27 14:13:03,550][20381] Updated weights for policy 1, policy_version 1235713 (0.0010) [2023-12-27 14:13:03,604][20381] Updated weights for policy 1, policy_version 1235723 (0.0010) [2023-12-27 14:13:03,654][20381] Updated weights for policy 1, policy_version 1235733 (0.0010) [2023-12-27 14:13:03,969][20347] Updated weights for policy 0, policy_version 1236773 (0.0008) [2023-12-27 14:13:04,019][20347] Updated weights for policy 0, policy_version 1236783 (0.0010) [2023-12-27 14:13:04,072][20347] Updated weights for policy 0, policy_version 1236793 (0.0010) [2023-12-27 14:13:04,344][20381] Updated weights for policy 1, policy_version 1235743 (0.0010) [2023-12-27 14:13:04,401][20381] Updated weights for policy 1, policy_version 1235753 (0.0010) [2023-12-27 14:13:04,462][20381] Updated weights for policy 1, policy_version 1235763 (0.0011) [2023-12-27 14:13:04,726][20347] Updated weights for policy 0, policy_version 1236803 (0.0010) [2023-12-27 14:13:04,787][20347] Updated weights for policy 0, policy_version 1236813 (0.0010) [2023-12-27 14:13:04,848][20347] Updated weights for policy 0, policy_version 1236823 (0.0010) [2023-12-27 14:13:05,095][20381] Updated weights for policy 1, policy_version 1235773 (0.0010) [2023-12-27 14:13:05,148][20381] Updated weights for policy 1, policy_version 1235783 (0.0010) [2023-12-27 14:13:05,211][20381] Updated weights for policy 1, policy_version 1235793 (0.0010) [2023-12-27 14:13:05,433][20347] Updated weights for policy 0, policy_version 1236833 (0.0009) [2023-12-27 14:13:05,484][20347] Updated weights for policy 0, policy_version 1236843 (0.0009) [2023-12-27 14:13:05,541][20347] Updated weights for policy 0, policy_version 1236853 (0.0010) [2023-12-27 14:13:05,602][20347] Updated weights for policy 0, policy_version 1236863 (0.0006) [2023-12-27 14:13:05,791][20381] Updated weights for policy 1, policy_version 1235803 (0.0010) [2023-12-27 14:13:05,840][20381] Updated weights for policy 1, policy_version 1235813 (0.0008) [2023-12-27 14:13:05,893][20381] Updated weights for policy 1, policy_version 1235823 (0.0008) [2023-12-27 14:13:06,246][20347] Updated weights for policy 0, policy_version 1236873 (0.0006) [2023-12-27 14:13:06,300][20347] Updated weights for policy 0, policy_version 1236883 (0.0006) [2023-12-27 14:13:06,349][20347] Updated weights for policy 0, policy_version 1236893 (0.0006) [2023-12-27 14:13:06,616][20381] Updated weights for policy 1, policy_version 1235833 (0.0008) [2023-12-27 14:13:06,672][20381] Updated weights for policy 1, policy_version 1235843 (0.0011) [2023-12-27 14:13:06,731][20381] Updated weights for policy 1, policy_version 1235853 (0.0010) [2023-12-27 14:13:06,793][20381] Updated weights for policy 1, policy_version 1235863 (0.0010) [2023-12-27 14:13:06,986][20347] Updated weights for policy 0, policy_version 1236903 (0.0010) [2023-12-27 14:13:07,044][20347] Updated weights for policy 0, policy_version 1236913 (0.0010) [2023-12-27 14:13:07,106][20347] Updated weights for policy 0, policy_version 1236923 (0.0009) [2023-12-27 14:13:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 633118720. Throughput: 0: 10189.7, 1: 10379.8. Samples: 633086140. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:07,363][19170] Avg episode reward: [(0, '970.800'), (1, '1000.000')] [2023-12-27 14:13:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001235864_316407808.pth... [2023-12-27 14:13:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001236928_316710912.pth... [2023-12-27 14:13:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001234648_316096512.pth [2023-12-27 14:13:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001235744_316407808.pth [2023-12-27 14:13:07,437][20381] Updated weights for policy 1, policy_version 1235873 (0.0006) [2023-12-27 14:13:07,493][20381] Updated weights for policy 1, policy_version 1235883 (0.0010) [2023-12-27 14:13:07,547][20381] Updated weights for policy 1, policy_version 1235893 (0.0010) [2023-12-27 14:13:07,719][20347] Updated weights for policy 0, policy_version 1236933 (0.0006) [2023-12-27 14:13:07,772][20347] Updated weights for policy 0, policy_version 1236943 (0.0005) [2023-12-27 14:13:07,828][20347] Updated weights for policy 0, policy_version 1236953 (0.0010) [2023-12-27 14:13:08,160][20381] Updated weights for policy 1, policy_version 1235903 (0.0011) [2023-12-27 14:13:08,217][20381] Updated weights for policy 1, policy_version 1235913 (0.0010) [2023-12-27 14:13:08,265][20381] Updated weights for policy 1, policy_version 1235923 (0.0010) [2023-12-27 14:13:08,482][20347] Updated weights for policy 0, policy_version 1236963 (0.0010) [2023-12-27 14:13:08,544][20347] Updated weights for policy 0, policy_version 1236973 (0.0010) [2023-12-27 14:13:08,597][20347] Updated weights for policy 0, policy_version 1236983 (0.0010) [2023-12-27 14:13:08,900][20381] Updated weights for policy 1, policy_version 1235933 (0.0008) [2023-12-27 14:13:08,948][20381] Updated weights for policy 1, policy_version 1235943 (0.0010) [2023-12-27 14:13:08,996][20381] Updated weights for policy 1, policy_version 1235953 (0.0010) [2023-12-27 14:13:09,292][20347] Updated weights for policy 0, policy_version 1236993 (0.0011) [2023-12-27 14:13:09,356][20347] Updated weights for policy 0, policy_version 1237003 (0.0010) [2023-12-27 14:13:09,418][20347] Updated weights for policy 0, policy_version 1237013 (0.0008) [2023-12-27 14:13:09,490][20347] Updated weights for policy 0, policy_version 1237023 (0.0010) [2023-12-27 14:13:09,708][20381] Updated weights for policy 1, policy_version 1235963 (0.0009) [2023-12-27 14:13:09,771][20381] Updated weights for policy 1, policy_version 1235973 (0.0005) [2023-12-27 14:13:09,831][20381] Updated weights for policy 1, policy_version 1235983 (0.0007) [2023-12-27 14:13:10,217][20347] Updated weights for policy 0, policy_version 1237033 (0.0008) [2023-12-27 14:13:10,275][20347] Updated weights for policy 0, policy_version 1237043 (0.0009) [2023-12-27 14:13:10,345][20347] Updated weights for policy 0, policy_version 1237053 (0.0008) [2023-12-27 14:13:10,489][20381] Updated weights for policy 1, policy_version 1235993 (0.0009) [2023-12-27 14:13:10,546][20381] Updated weights for policy 1, policy_version 1236003 (0.0008) [2023-12-27 14:13:10,609][20381] Updated weights for policy 1, policy_version 1236013 (0.0007) [2023-12-27 14:13:10,669][20381] Updated weights for policy 1, policy_version 1236023 (0.0008) [2023-12-27 14:13:11,045][20347] Updated weights for policy 0, policy_version 1237063 (0.0008) [2023-12-27 14:13:11,105][20347] Updated weights for policy 0, policy_version 1237073 (0.0008) [2023-12-27 14:13:11,176][20347] Updated weights for policy 0, policy_version 1237083 (0.0009) [2023-12-27 14:13:11,495][20381] Updated weights for policy 1, policy_version 1236033 (0.0007) [2023-12-27 14:13:11,560][20381] Updated weights for policy 1, policy_version 1236043 (0.0008) [2023-12-27 14:13:11,630][20381] Updated weights for policy 1, policy_version 1236053 (0.0007) [2023-12-27 14:13:11,955][20347] Updated weights for policy 0, policy_version 1237093 (0.0008) [2023-12-27 14:13:12,011][20347] Updated weights for policy 0, policy_version 1237103 (0.0008) [2023-12-27 14:13:12,068][20347] Updated weights for policy 0, policy_version 1237113 (0.0008) [2023-12-27 14:13:12,305][20381] Updated weights for policy 1, policy_version 1236063 (0.0010) [2023-12-27 14:13:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 633217024. Throughput: 0: 10232.5, 1: 10370.8. Samples: 633211484. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:12,363][19170] Avg episode reward: [(0, '980.490'), (1, '1000.000')] [2023-12-27 14:13:12,369][20381] Updated weights for policy 1, policy_version 1236073 (0.0009) [2023-12-27 14:13:12,421][20381] Updated weights for policy 1, policy_version 1236083 (0.0008) [2023-12-27 14:13:12,754][20347] Updated weights for policy 0, policy_version 1237123 (0.0009) [2023-12-27 14:13:12,810][20347] Updated weights for policy 0, policy_version 1237133 (0.0010) [2023-12-27 14:13:12,865][20347] Updated weights for policy 0, policy_version 1237143 (0.0010) [2023-12-27 14:13:13,152][20381] Updated weights for policy 1, policy_version 1236093 (0.0009) [2023-12-27 14:13:13,197][20381] Updated weights for policy 1, policy_version 1236103 (0.0010) [2023-12-27 14:13:13,245][20381] Updated weights for policy 1, policy_version 1236113 (0.0010) [2023-12-27 14:13:13,491][20347] Updated weights for policy 0, policy_version 1237153 (0.0010) [2023-12-27 14:13:13,550][20347] Updated weights for policy 0, policy_version 1237163 (0.0010) [2023-12-27 14:13:13,606][20347] Updated weights for policy 0, policy_version 1237173 (0.0010) [2023-12-27 14:13:13,661][20347] Updated weights for policy 0, policy_version 1237183 (0.0009) [2023-12-27 14:13:13,942][20381] Updated weights for policy 1, policy_version 1236123 (0.0010) [2023-12-27 14:13:13,995][20381] Updated weights for policy 1, policy_version 1236133 (0.0007) [2023-12-27 14:13:14,051][20381] Updated weights for policy 1, policy_version 1236143 (0.0010) [2023-12-27 14:13:14,325][20347] Updated weights for policy 0, policy_version 1237193 (0.0010) [2023-12-27 14:13:14,382][20347] Updated weights for policy 0, policy_version 1237203 (0.0010) [2023-12-27 14:13:14,436][20347] Updated weights for policy 0, policy_version 1237213 (0.0010) [2023-12-27 14:13:14,702][20381] Updated weights for policy 1, policy_version 1236153 (0.0010) [2023-12-27 14:13:14,756][20381] Updated weights for policy 1, policy_version 1236163 (0.0010) [2023-12-27 14:13:14,820][20381] Updated weights for policy 1, policy_version 1236173 (0.0011) [2023-12-27 14:13:14,879][20381] Updated weights for policy 1, policy_version 1236183 (0.0011) [2023-12-27 14:13:15,159][20347] Updated weights for policy 0, policy_version 1237223 (0.0010) [2023-12-27 14:13:15,214][20347] Updated weights for policy 0, policy_version 1237233 (0.0010) [2023-12-27 14:13:15,267][20347] Updated weights for policy 0, policy_version 1237243 (0.0010) [2023-12-27 14:13:15,477][20381] Updated weights for policy 1, policy_version 1236193 (0.0008) [2023-12-27 14:13:15,533][20381] Updated weights for policy 1, policy_version 1236203 (0.0008) [2023-12-27 14:13:15,592][20381] Updated weights for policy 1, policy_version 1236213 (0.0007) [2023-12-27 14:13:15,873][20347] Updated weights for policy 0, policy_version 1237253 (0.0010) [2023-12-27 14:13:15,925][20347] Updated weights for policy 0, policy_version 1237263 (0.0010) [2023-12-27 14:13:15,983][20347] Updated weights for policy 0, policy_version 1237273 (0.0010) [2023-12-27 14:13:16,252][20381] Updated weights for policy 1, policy_version 1236223 (0.0006) [2023-12-27 14:13:16,310][20381] Updated weights for policy 1, policy_version 1236233 (0.0008) [2023-12-27 14:13:16,365][20381] Updated weights for policy 1, policy_version 1236243 (0.0007) [2023-12-27 14:13:16,568][20347] Updated weights for policy 0, policy_version 1237283 (0.0008) [2023-12-27 14:13:16,625][20347] Updated weights for policy 0, policy_version 1237293 (0.0007) [2023-12-27 14:13:16,676][20347] Updated weights for policy 0, policy_version 1237303 (0.0010) [2023-12-27 14:13:16,972][20381] Updated weights for policy 1, policy_version 1236253 (0.0005) [2023-12-27 14:13:17,036][20381] Updated weights for policy 1, policy_version 1236263 (0.0005) [2023-12-27 14:13:17,099][20381] Updated weights for policy 1, policy_version 1236273 (0.0005) [2023-12-27 14:13:17,301][20347] Updated weights for policy 0, policy_version 1237313 (0.0008) [2023-12-27 14:13:17,361][20347] Updated weights for policy 0, policy_version 1237323 (0.0010) [2023-12-27 14:13:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 633323520. Throughput: 0: 10211.1, 1: 10442.1. Samples: 633336104. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:17,363][19170] Avg episode reward: [(0, '980.620'), (1, '1000.000')] [2023-12-27 14:13:17,418][20347] Updated weights for policy 0, policy_version 1237333 (0.0010) [2023-12-27 14:13:17,469][20347] Updated weights for policy 0, policy_version 1237343 (0.0010) [2023-12-27 14:13:17,706][20381] Updated weights for policy 1, policy_version 1236283 (0.0007) [2023-12-27 14:13:17,765][20381] Updated weights for policy 1, policy_version 1236293 (0.0007) [2023-12-27 14:13:17,822][20381] Updated weights for policy 1, policy_version 1236303 (0.0008) [2023-12-27 14:13:18,132][20347] Updated weights for policy 0, policy_version 1237353 (0.0007) [2023-12-27 14:13:18,185][20347] Updated weights for policy 0, policy_version 1237363 (0.0007) [2023-12-27 14:13:18,236][20347] Updated weights for policy 0, policy_version 1237373 (0.0005) [2023-12-27 14:13:18,473][20381] Updated weights for policy 1, policy_version 1236313 (0.0007) [2023-12-27 14:13:18,529][20381] Updated weights for policy 1, policy_version 1236323 (0.0007) [2023-12-27 14:13:18,590][20381] Updated weights for policy 1, policy_version 1236333 (0.0008) [2023-12-27 14:13:18,649][20381] Updated weights for policy 1, policy_version 1236343 (0.0008) [2023-12-27 14:13:18,880][20347] Updated weights for policy 0, policy_version 1237383 (0.0005) [2023-12-27 14:13:18,929][20347] Updated weights for policy 0, policy_version 1237393 (0.0005) [2023-12-27 14:13:18,983][20347] Updated weights for policy 0, policy_version 1237403 (0.0007) [2023-12-27 14:13:19,399][20381] Updated weights for policy 1, policy_version 1236355 (0.0008) [2023-12-27 14:13:19,459][20381] Updated weights for policy 1, policy_version 1236365 (0.0007) [2023-12-27 14:13:19,523][20381] Updated weights for policy 1, policy_version 1236375 (0.0007) [2023-12-27 14:13:19,664][20347] Updated weights for policy 0, policy_version 1237413 (0.0006) [2023-12-27 14:13:19,720][20347] Updated weights for policy 0, policy_version 1237423 (0.0005) [2023-12-27 14:13:19,779][20347] Updated weights for policy 0, policy_version 1237433 (0.0006) [2023-12-27 14:13:20,293][20381] Updated weights for policy 1, policy_version 1236385 (0.0008) [2023-12-27 14:13:20,357][20381] Updated weights for policy 1, policy_version 1236395 (0.0008) [2023-12-27 14:13:20,366][20347] Updated weights for policy 0, policy_version 1237443 (0.0006) [2023-12-27 14:13:20,418][20381] Updated weights for policy 1, policy_version 1236405 (0.0007) [2023-12-27 14:13:20,424][20347] Updated weights for policy 0, policy_version 1237453 (0.0006) [2023-12-27 14:13:20,472][20347] Updated weights for policy 0, policy_version 1237463 (0.0007) [2023-12-27 14:13:21,178][20381] Updated weights for policy 1, policy_version 1236415 (0.0008) [2023-12-27 14:13:21,196][20347] Updated weights for policy 0, policy_version 1237473 (0.0006) [2023-12-27 14:13:21,233][20381] Updated weights for policy 1, policy_version 1236425 (0.0007) [2023-12-27 14:13:21,256][20347] Updated weights for policy 0, policy_version 1237483 (0.0006) [2023-12-27 14:13:21,294][20381] Updated weights for policy 1, policy_version 1236435 (0.0006) [2023-12-27 14:13:21,315][20347] Updated weights for policy 0, policy_version 1237493 (0.0007) [2023-12-27 14:13:21,386][20347] Updated weights for policy 0, policy_version 1237503 (0.0008) [2023-12-27 14:13:22,046][20381] Updated weights for policy 1, policy_version 1236445 (0.0005) [2023-12-27 14:13:22,063][20347] Updated weights for policy 0, policy_version 1237513 (0.0007) [2023-12-27 14:13:22,112][20381] Updated weights for policy 1, policy_version 1236455 (0.0007) [2023-12-27 14:13:22,124][20347] Updated weights for policy 0, policy_version 1237523 (0.0007) [2023-12-27 14:13:22,174][20381] Updated weights for policy 1, policy_version 1236465 (0.0008) [2023-12-27 14:13:22,176][20347] Updated weights for policy 0, policy_version 1237533 (0.0005) [2023-12-27 14:13:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 633430016. Throughput: 0: 10222.6, 1: 10365.3. Samples: 633397248. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:22,363][19170] Avg episode reward: [(0, '970.890'), (1, '1000.000')] [2023-12-27 14:13:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001237536_316866560.pth... [2023-12-27 14:13:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001236472_316563456.pth... [2023-12-27 14:13:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001236320_316555264.pth [2023-12-27 14:13:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001235256_316252160.pth [2023-12-27 14:13:22,856][20347] Updated weights for policy 0, policy_version 1237543 (0.0007) [2023-12-27 14:13:22,913][20347] Updated weights for policy 0, policy_version 1237553 (0.0008) [2023-12-27 14:13:22,915][20381] Updated weights for policy 1, policy_version 1236475 (0.0008) [2023-12-27 14:13:22,967][20347] Updated weights for policy 0, policy_version 1237563 (0.0006) [2023-12-27 14:13:22,972][20381] Updated weights for policy 1, policy_version 1236485 (0.0007) [2023-12-27 14:13:23,030][20381] Updated weights for policy 1, policy_version 1236495 (0.0007) [2023-12-27 14:13:23,661][20347] Updated weights for policy 0, policy_version 1237573 (0.0007) [2023-12-27 14:13:23,711][20347] Updated weights for policy 0, policy_version 1237583 (0.0007) [2023-12-27 14:13:23,713][20381] Updated weights for policy 1, policy_version 1236505 (0.0008) [2023-12-27 14:13:23,758][20347] Updated weights for policy 0, policy_version 1237593 (0.0007) [2023-12-27 14:13:23,787][20381] Updated weights for policy 1, policy_version 1236515 (0.0007) [2023-12-27 14:13:23,841][20381] Updated weights for policy 1, policy_version 1236525 (0.0008) [2023-12-27 14:13:23,892][20381] Updated weights for policy 1, policy_version 1236535 (0.0007) [2023-12-27 14:13:24,425][20347] Updated weights for policy 0, policy_version 1237603 (0.0006) [2023-12-27 14:13:24,482][20347] Updated weights for policy 0, policy_version 1237613 (0.0007) [2023-12-27 14:13:24,535][20347] Updated weights for policy 0, policy_version 1237623 (0.0008) [2023-12-27 14:13:24,538][20381] Updated weights for policy 1, policy_version 1236545 (0.0005) [2023-12-27 14:13:24,601][20381] Updated weights for policy 1, policy_version 1236555 (0.0007) [2023-12-27 14:13:24,656][20381] Updated weights for policy 1, policy_version 1236565 (0.0008) [2023-12-27 14:13:25,092][20347] Updated weights for policy 0, policy_version 1237633 (0.0006) [2023-12-27 14:13:25,148][20347] Updated weights for policy 0, policy_version 1237643 (0.0005) [2023-12-27 14:13:25,203][20347] Updated weights for policy 0, policy_version 1237653 (0.0005) [2023-12-27 14:13:25,262][20347] Updated weights for policy 0, policy_version 1237663 (0.0010) [2023-12-27 14:13:25,298][20381] Updated weights for policy 1, policy_version 1236575 (0.0009) [2023-12-27 14:13:25,343][20381] Updated weights for policy 1, policy_version 1236585 (0.0010) [2023-12-27 14:13:25,394][20381] Updated weights for policy 1, policy_version 1236595 (0.0011) [2023-12-27 14:13:25,901][20347] Updated weights for policy 0, policy_version 1237673 (0.0007) [2023-12-27 14:13:25,957][20347] Updated weights for policy 0, policy_version 1237683 (0.0008) [2023-12-27 14:13:26,017][20347] Updated weights for policy 0, policy_version 1237693 (0.0008) [2023-12-27 14:13:26,073][20381] Updated weights for policy 1, policy_version 1236605 (0.0008) [2023-12-27 14:13:26,122][20381] Updated weights for policy 1, policy_version 1236615 (0.0007) [2023-12-27 14:13:26,168][20381] Updated weights for policy 1, policy_version 1236625 (0.0005) [2023-12-27 14:13:26,666][20347] Updated weights for policy 0, policy_version 1237703 (0.0009) [2023-12-27 14:13:26,715][20347] Updated weights for policy 0, policy_version 1237713 (0.0007) [2023-12-27 14:13:26,764][20347] Updated weights for policy 0, policy_version 1237723 (0.0008) [2023-12-27 14:13:26,781][20381] Updated weights for policy 1, policy_version 1236635 (0.0010) [2023-12-27 14:13:26,850][20381] Updated weights for policy 1, policy_version 1236645 (0.0008) [2023-12-27 14:13:26,914][20381] Updated weights for policy 1, policy_version 1236655 (0.0007) [2023-12-27 14:13:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 633528320. Throughput: 0: 10314.3, 1: 10262.1. Samples: 633519432. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:27,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 14:13:27,423][20347] Updated weights for policy 0, policy_version 1237733 (0.0006) [2023-12-27 14:13:27,479][20347] Updated weights for policy 0, policy_version 1237743 (0.0009) [2023-12-27 14:13:27,526][20347] Updated weights for policy 0, policy_version 1237753 (0.0007) [2023-12-27 14:13:27,544][20381] Updated weights for policy 1, policy_version 1236665 (0.0011) [2023-12-27 14:13:27,588][20381] Updated weights for policy 1, policy_version 1236675 (0.0010) [2023-12-27 14:13:27,642][20381] Updated weights for policy 1, policy_version 1236685 (0.0009) [2023-12-27 14:13:27,700][20381] Updated weights for policy 1, policy_version 1236695 (0.0007) [2023-12-27 14:13:28,184][20347] Updated weights for policy 0, policy_version 1237763 (0.0006) [2023-12-27 14:13:28,235][20347] Updated weights for policy 0, policy_version 1237773 (0.0005) [2023-12-27 14:13:28,285][20347] Updated weights for policy 0, policy_version 1237783 (0.0005) [2023-12-27 14:13:28,356][20381] Updated weights for policy 1, policy_version 1236705 (0.0007) [2023-12-27 14:13:28,422][20381] Updated weights for policy 1, policy_version 1236715 (0.0007) [2023-12-27 14:13:28,478][20381] Updated weights for policy 1, policy_version 1236725 (0.0007) [2023-12-27 14:13:28,890][20347] Updated weights for policy 0, policy_version 1237793 (0.0006) [2023-12-27 14:13:28,941][20347] Updated weights for policy 0, policy_version 1237803 (0.0005) [2023-12-27 14:13:28,989][20347] Updated weights for policy 0, policy_version 1237813 (0.0005) [2023-12-27 14:13:29,038][20347] Updated weights for policy 0, policy_version 1237823 (0.0005) [2023-12-27 14:13:29,208][20381] Updated weights for policy 1, policy_version 1236735 (0.0009) [2023-12-27 14:13:29,282][20381] Updated weights for policy 1, policy_version 1236745 (0.0007) [2023-12-27 14:13:29,338][20381] Updated weights for policy 1, policy_version 1236755 (0.0007) [2023-12-27 14:13:29,680][20347] Updated weights for policy 0, policy_version 1237833 (0.0006) [2023-12-27 14:13:29,731][20347] Updated weights for policy 0, policy_version 1237843 (0.0005) [2023-12-27 14:13:29,782][20347] Updated weights for policy 0, policy_version 1237853 (0.0005) [2023-12-27 14:13:29,952][20381] Updated weights for policy 1, policy_version 1236765 (0.0008) [2023-12-27 14:13:30,003][20381] Updated weights for policy 1, policy_version 1236775 (0.0007) [2023-12-27 14:13:30,057][20381] Updated weights for policy 1, policy_version 1236785 (0.0009) [2023-12-27 14:13:30,375][20347] Updated weights for policy 0, policy_version 1237863 (0.0006) [2023-12-27 14:13:30,436][20347] Updated weights for policy 0, policy_version 1237873 (0.0008) [2023-12-27 14:13:30,483][20347] Updated weights for policy 0, policy_version 1237883 (0.0007) [2023-12-27 14:13:30,785][20381] Updated weights for policy 1, policy_version 1236795 (0.0008) [2023-12-27 14:13:30,834][20381] Updated weights for policy 1, policy_version 1236805 (0.0008) [2023-12-27 14:13:30,877][20381] Updated weights for policy 1, policy_version 1236815 (0.0005) [2023-12-27 14:13:31,126][20347] Updated weights for policy 0, policy_version 1237893 (0.0007) [2023-12-27 14:13:31,192][20347] Updated weights for policy 0, policy_version 1237903 (0.0007) [2023-12-27 14:13:31,265][20347] Updated weights for policy 0, policy_version 1237913 (0.0007) [2023-12-27 14:13:31,611][20381] Updated weights for policy 1, policy_version 1236825 (0.0005) [2023-12-27 14:13:31,680][20381] Updated weights for policy 1, policy_version 1236835 (0.0009) [2023-12-27 14:13:31,745][20381] Updated weights for policy 1, policy_version 1236845 (0.0011) [2023-12-27 14:13:31,802][20381] Updated weights for policy 1, policy_version 1236855 (0.0006) [2023-12-27 14:13:32,016][20347] Updated weights for policy 0, policy_version 1237923 (0.0007) [2023-12-27 14:13:32,076][20347] Updated weights for policy 0, policy_version 1237933 (0.0006) [2023-12-27 14:13:32,137][20347] Updated weights for policy 0, policy_version 1237943 (0.0007) [2023-12-27 14:13:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 633634816. Throughput: 0: 10379.7, 1: 10279.2. Samples: 633645440. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:32,363][19170] Avg episode reward: [(0, '970.690'), (1, '1000.000')] [2023-12-27 14:13:32,530][20381] Updated weights for policy 1, policy_version 1236865 (0.0007) [2023-12-27 14:13:32,595][20381] Updated weights for policy 1, policy_version 1236875 (0.0006) [2023-12-27 14:13:32,660][20381] Updated weights for policy 1, policy_version 1236885 (0.0007) [2023-12-27 14:13:32,822][20347] Updated weights for policy 0, policy_version 1237953 (0.0008) [2023-12-27 14:13:32,875][20347] Updated weights for policy 0, policy_version 1237963 (0.0006) [2023-12-27 14:13:32,923][20347] Updated weights for policy 0, policy_version 1237973 (0.0005) [2023-12-27 14:13:32,968][20347] Updated weights for policy 0, policy_version 1237983 (0.0006) [2023-12-27 14:13:33,256][20381] Updated weights for policy 1, policy_version 1236895 (0.0006) [2023-12-27 14:13:33,305][20381] Updated weights for policy 1, policy_version 1236905 (0.0007) [2023-12-27 14:13:33,361][20381] Updated weights for policy 1, policy_version 1236915 (0.0005) [2023-12-27 14:13:33,651][20347] Updated weights for policy 0, policy_version 1237993 (0.0006) [2023-12-27 14:13:33,695][20347] Updated weights for policy 0, policy_version 1238003 (0.0005) [2023-12-27 14:13:33,738][20347] Updated weights for policy 0, policy_version 1238013 (0.0005) [2023-12-27 14:13:33,904][20381] Updated weights for policy 1, policy_version 1236925 (0.0005) [2023-12-27 14:13:33,954][20381] Updated weights for policy 1, policy_version 1236935 (0.0005) [2023-12-27 14:13:34,003][20381] Updated weights for policy 1, policy_version 1236945 (0.0005) [2023-12-27 14:13:34,432][20347] Updated weights for policy 0, policy_version 1238023 (0.0007) [2023-12-27 14:13:34,480][20347] Updated weights for policy 0, policy_version 1238033 (0.0008) [2023-12-27 14:13:34,539][20347] Updated weights for policy 0, policy_version 1238043 (0.0010) [2023-12-27 14:13:34,645][20381] Updated weights for policy 1, policy_version 1236955 (0.0007) [2023-12-27 14:13:34,707][20381] Updated weights for policy 1, policy_version 1236965 (0.0009) [2023-12-27 14:13:34,759][20381] Updated weights for policy 1, policy_version 1236975 (0.0010) [2023-12-27 14:13:35,145][20347] Updated weights for policy 0, policy_version 1238053 (0.0008) [2023-12-27 14:13:35,196][20347] Updated weights for policy 0, policy_version 1238063 (0.0010) [2023-12-27 14:13:35,251][20347] Updated weights for policy 0, policy_version 1238073 (0.0010) [2023-12-27 14:13:35,446][20381] Updated weights for policy 1, policy_version 1236985 (0.0010) [2023-12-27 14:13:35,501][20381] Updated weights for policy 1, policy_version 1236995 (0.0011) [2023-12-27 14:13:35,558][20381] Updated weights for policy 1, policy_version 1237005 (0.0010) [2023-12-27 14:13:35,613][20381] Updated weights for policy 1, policy_version 1237015 (0.0010) [2023-12-27 14:13:35,860][20347] Updated weights for policy 0, policy_version 1238083 (0.0007) [2023-12-27 14:13:35,916][20347] Updated weights for policy 0, policy_version 1238093 (0.0005) [2023-12-27 14:13:35,970][20347] Updated weights for policy 0, policy_version 1238103 (0.0005) [2023-12-27 14:13:36,271][20381] Updated weights for policy 1, policy_version 1237025 (0.0010) [2023-12-27 14:13:36,326][20381] Updated weights for policy 1, policy_version 1237035 (0.0010) [2023-12-27 14:13:36,384][20381] Updated weights for policy 1, policy_version 1237045 (0.0010) [2023-12-27 14:13:36,600][20347] Updated weights for policy 0, policy_version 1238113 (0.0006) [2023-12-27 14:13:36,652][20347] Updated weights for policy 0, policy_version 1238123 (0.0007) [2023-12-27 14:13:36,704][20347] Updated weights for policy 0, policy_version 1238133 (0.0007) [2023-12-27 14:13:36,759][20347] Updated weights for policy 0, policy_version 1238143 (0.0008) [2023-12-27 14:13:37,071][20381] Updated weights for policy 1, policy_version 1237055 (0.0010) [2023-12-27 14:13:37,123][20381] Updated weights for policy 1, policy_version 1237065 (0.0010) [2023-12-27 14:13:37,181][20381] Updated weights for policy 1, policy_version 1237075 (0.0010) [2023-12-27 14:13:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 633741312. Throughput: 0: 10412.5, 1: 10315.8. Samples: 633709916. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:37,363][19170] Avg episode reward: [(0, '951.200'), (1, '1000.000')] [2023-12-27 14:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001237080_316719104.pth... [2023-12-27 14:13:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001238144_317022208.pth... [2023-12-27 14:13:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001235864_316407808.pth [2023-12-27 14:13:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001236928_316710912.pth [2023-12-27 14:13:37,481][20347] Updated weights for policy 0, policy_version 1238153 (0.0007) [2023-12-27 14:13:37,537][20347] Updated weights for policy 0, policy_version 1238163 (0.0007) [2023-12-27 14:13:37,592][20347] Updated weights for policy 0, policy_version 1238173 (0.0007) [2023-12-27 14:13:37,880][20381] Updated weights for policy 1, policy_version 1237085 (0.0010) [2023-12-27 14:13:37,931][20381] Updated weights for policy 1, policy_version 1237095 (0.0010) [2023-12-27 14:13:37,989][20381] Updated weights for policy 1, policy_version 1237105 (0.0007) [2023-12-27 14:13:38,190][20347] Updated weights for policy 0, policy_version 1238183 (0.0006) [2023-12-27 14:13:38,253][20347] Updated weights for policy 0, policy_version 1238193 (0.0005) [2023-12-27 14:13:38,318][20347] Updated weights for policy 0, policy_version 1238203 (0.0006) [2023-12-27 14:13:38,695][20381] Updated weights for policy 1, policy_version 1237115 (0.0008) [2023-12-27 14:13:38,744][20381] Updated weights for policy 1, policy_version 1237125 (0.0009) [2023-12-27 14:13:38,794][20381] Updated weights for policy 1, policy_version 1237135 (0.0010) [2023-12-27 14:13:38,955][20347] Updated weights for policy 0, policy_version 1238213 (0.0007) [2023-12-27 14:13:39,010][20347] Updated weights for policy 0, policy_version 1238223 (0.0007) [2023-12-27 14:13:39,079][20347] Updated weights for policy 0, policy_version 1238233 (0.0007) [2023-12-27 14:13:39,483][20381] Updated weights for policy 1, policy_version 1237145 (0.0010) [2023-12-27 14:13:39,536][20381] Updated weights for policy 1, policy_version 1237155 (0.0008) [2023-12-27 14:13:39,584][20381] Updated weights for policy 1, policy_version 1237165 (0.0008) [2023-12-27 14:13:39,632][20381] Updated weights for policy 1, policy_version 1237175 (0.0007) [2023-12-27 14:13:39,754][20347] Updated weights for policy 0, policy_version 1238243 (0.0007) [2023-12-27 14:13:39,813][20347] Updated weights for policy 0, policy_version 1238253 (0.0010) [2023-12-27 14:13:39,883][20347] Updated weights for policy 0, policy_version 1238263 (0.0009) [2023-12-27 14:13:40,335][20381] Updated weights for policy 1, policy_version 1237185 (0.0006) [2023-12-27 14:13:40,394][20381] Updated weights for policy 1, policy_version 1237195 (0.0007) [2023-12-27 14:13:40,454][20381] Updated weights for policy 1, policy_version 1237205 (0.0009) [2023-12-27 14:13:40,552][20347] Updated weights for policy 0, policy_version 1238273 (0.0008) [2023-12-27 14:13:40,609][20347] Updated weights for policy 0, policy_version 1238283 (0.0010) [2023-12-27 14:13:40,669][20347] Updated weights for policy 0, policy_version 1238293 (0.0009) [2023-12-27 14:13:40,724][20347] Updated weights for policy 0, policy_version 1238303 (0.0008) [2023-12-27 14:13:41,060][20381] Updated weights for policy 1, policy_version 1237215 (0.0008) [2023-12-27 14:13:41,120][20381] Updated weights for policy 1, policy_version 1237225 (0.0008) [2023-12-27 14:13:41,182][20381] Updated weights for policy 1, policy_version 1237235 (0.0008) [2023-12-27 14:13:41,463][20347] Updated weights for policy 0, policy_version 1238313 (0.0010) [2023-12-27 14:13:41,525][20347] Updated weights for policy 0, policy_version 1238323 (0.0010) [2023-12-27 14:13:41,587][20347] Updated weights for policy 0, policy_version 1238333 (0.0010) [2023-12-27 14:13:41,867][20381] Updated weights for policy 1, policy_version 1237245 (0.0006) [2023-12-27 14:13:41,925][20381] Updated weights for policy 1, policy_version 1237255 (0.0007) [2023-12-27 14:13:41,985][20381] Updated weights for policy 1, policy_version 1237265 (0.0008) [2023-12-27 14:13:42,283][20347] Updated weights for policy 0, policy_version 1238343 (0.0009) [2023-12-27 14:13:42,349][20347] Updated weights for policy 0, policy_version 1238353 (0.0008) [2023-12-27 14:13:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 633839616. Throughput: 0: 10488.4, 1: 10279.1. Samples: 633833756. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:13:42,363][19170] Avg episode reward: [(0, '970.740'), (1, '990.230')] [2023-12-27 14:13:42,414][20347] Updated weights for policy 0, policy_version 1238363 (0.0009) [2023-12-27 14:13:42,659][20381] Updated weights for policy 1, policy_version 1237275 (0.0007) [2023-12-27 14:13:42,720][20381] Updated weights for policy 1, policy_version 1237285 (0.0008) [2023-12-27 14:13:42,786][20381] Updated weights for policy 1, policy_version 1237295 (0.0008) [2023-12-27 14:13:43,094][20347] Updated weights for policy 0, policy_version 1238373 (0.0007) [2023-12-27 14:13:43,152][20347] Updated weights for policy 0, policy_version 1238383 (0.0008) [2023-12-27 14:13:43,207][20347] Updated weights for policy 0, policy_version 1238393 (0.0007) [2023-12-27 14:13:43,519][20381] Updated weights for policy 1, policy_version 1237305 (0.0008) [2023-12-27 14:13:43,572][20381] Updated weights for policy 1, policy_version 1237315 (0.0008) [2023-12-27 14:13:43,625][20381] Updated weights for policy 1, policy_version 1237325 (0.0007) [2023-12-27 14:13:43,676][20381] Updated weights for policy 1, policy_version 1237335 (0.0008) [2023-12-27 14:13:43,885][20347] Updated weights for policy 0, policy_version 1238403 (0.0009) [2023-12-27 14:13:43,952][20347] Updated weights for policy 0, policy_version 1238413 (0.0010) [2023-12-27 14:13:43,954][20291] KL-divergence is very high: 101.2681 [2023-12-27 14:13:44,007][20291] KL-divergence is very high: 112.1356 [2023-12-27 14:13:44,019][20347] Updated weights for policy 0, policy_version 1238423 (0.0007) [2023-12-27 14:13:44,263][20381] Updated weights for policy 1, policy_version 1237345 (0.0006) [2023-12-27 14:13:44,315][20381] Updated weights for policy 1, policy_version 1237355 (0.0007) [2023-12-27 14:13:44,366][20381] Updated weights for policy 1, policy_version 1237365 (0.0007) [2023-12-27 14:13:44,663][20347] Updated weights for policy 0, policy_version 1238433 (0.0006) [2023-12-27 14:13:44,707][20347] Updated weights for policy 0, policy_version 1238443 (0.0010) [2023-12-27 14:13:44,759][20347] Updated weights for policy 0, policy_version 1238453 (0.0010) [2023-12-27 14:13:44,824][20347] Updated weights for policy 0, policy_version 1238463 (0.0010) [2023-12-27 14:13:45,061][20381] Updated weights for policy 1, policy_version 1237375 (0.0008) [2023-12-27 14:13:45,117][20381] Updated weights for policy 1, policy_version 1237385 (0.0008) [2023-12-27 14:13:45,173][20381] Updated weights for policy 1, policy_version 1237395 (0.0007) [2023-12-27 14:13:45,513][20347] Updated weights for policy 0, policy_version 1238473 (0.0010) [2023-12-27 14:13:45,570][20347] Updated weights for policy 0, policy_version 1238483 (0.0010) [2023-12-27 14:13:45,636][20347] Updated weights for policy 0, policy_version 1238493 (0.0010) [2023-12-27 14:13:45,793][20381] Updated weights for policy 1, policy_version 1237405 (0.0006) [2023-12-27 14:13:45,846][20381] Updated weights for policy 1, policy_version 1237415 (0.0005) [2023-12-27 14:13:45,894][20381] Updated weights for policy 1, policy_version 1237425 (0.0005) [2023-12-27 14:13:46,291][20347] Updated weights for policy 0, policy_version 1238503 (0.0010) [2023-12-27 14:13:46,340][20347] Updated weights for policy 0, policy_version 1238513 (0.0010) [2023-12-27 14:13:46,395][20347] Updated weights for policy 0, policy_version 1238523 (0.0010) [2023-12-27 14:13:46,474][20381] Updated weights for policy 1, policy_version 1237435 (0.0005) [2023-12-27 14:13:46,520][20381] Updated weights for policy 1, policy_version 1237445 (0.0005) [2023-12-27 14:13:46,577][20381] Updated weights for policy 1, policy_version 1237455 (0.0005) [2023-12-27 14:13:47,050][20347] Updated weights for policy 0, policy_version 1238533 (0.0009) [2023-12-27 14:13:47,107][20347] Updated weights for policy 0, policy_version 1238543 (0.0007) [2023-12-27 14:13:47,163][20347] Updated weights for policy 0, policy_version 1238553 (0.0007) [2023-12-27 14:13:47,228][20381] Updated weights for policy 1, policy_version 1237465 (0.0006) [2023-12-27 14:13:47,282][20381] Updated weights for policy 1, policy_version 1237475 (0.0006) [2023-12-27 14:13:47,337][20381] Updated weights for policy 1, policy_version 1237485 (0.0005) [2023-12-27 14:13:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 633946112. Throughput: 0: 10465.2, 1: 10339.4. Samples: 633958844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:13:47,363][19170] Avg episode reward: [(0, '961.200'), (1, '970.750')] [2023-12-27 14:13:47,390][20381] Updated weights for policy 1, policy_version 1237495 (0.0007) [2023-12-27 14:13:47,873][20347] Updated weights for policy 0, policy_version 1238563 (0.0007) [2023-12-27 14:13:47,921][20347] Updated weights for policy 0, policy_version 1238573 (0.0007) [2023-12-27 14:13:47,979][20347] Updated weights for policy 0, policy_version 1238583 (0.0006) [2023-12-27 14:13:48,051][20381] Updated weights for policy 1, policy_version 1237505 (0.0009) [2023-12-27 14:13:48,109][20381] Updated weights for policy 1, policy_version 1237515 (0.0008) [2023-12-27 14:13:48,164][20381] Updated weights for policy 1, policy_version 1237525 (0.0009) [2023-12-27 14:13:48,594][20347] Updated weights for policy 0, policy_version 1238593 (0.0006) [2023-12-27 14:13:48,641][20347] Updated weights for policy 0, policy_version 1238603 (0.0005) [2023-12-27 14:13:48,705][20347] Updated weights for policy 0, policy_version 1238613 (0.0005) [2023-12-27 14:13:48,772][20347] Updated weights for policy 0, policy_version 1238623 (0.0007) [2023-12-27 14:13:48,866][20381] Updated weights for policy 1, policy_version 1237535 (0.0008) [2023-12-27 14:13:48,919][20381] Updated weights for policy 1, policy_version 1237545 (0.0007) [2023-12-27 14:13:48,965][20381] Updated weights for policy 1, policy_version 1237555 (0.0007) [2023-12-27 14:13:49,377][20347] Updated weights for policy 0, policy_version 1238633 (0.0010) [2023-12-27 14:13:49,437][20347] Updated weights for policy 0, policy_version 1238643 (0.0008) [2023-12-27 14:13:49,490][20347] Updated weights for policy 0, policy_version 1238653 (0.0010) [2023-12-27 14:13:49,746][20381] Updated weights for policy 1, policy_version 1237565 (0.0007) [2023-12-27 14:13:49,807][20381] Updated weights for policy 1, policy_version 1237575 (0.0007) [2023-12-27 14:13:49,876][20381] Updated weights for policy 1, policy_version 1237585 (0.0008) [2023-12-27 14:13:50,158][20347] Updated weights for policy 0, policy_version 1238663 (0.0009) [2023-12-27 14:13:50,212][20347] Updated weights for policy 0, policy_version 1238673 (0.0009) [2023-12-27 14:13:50,269][20347] Updated weights for policy 0, policy_version 1238683 (0.0009) [2023-12-27 14:13:50,660][20381] Updated weights for policy 1, policy_version 1237595 (0.0007) [2023-12-27 14:13:50,723][20381] Updated weights for policy 1, policy_version 1237605 (0.0008) [2023-12-27 14:13:50,783][20381] Updated weights for policy 1, policy_version 1237615 (0.0008) [2023-12-27 14:13:50,942][20347] Updated weights for policy 0, policy_version 1238693 (0.0008) [2023-12-27 14:13:50,970][20291] KL-divergence is very high: 351.6855 [2023-12-27 14:13:51,000][20347] Updated weights for policy 0, policy_version 1238703 (0.0007) [2023-12-27 14:13:51,012][20291] KL-divergence is very high: 160.3860 [2023-12-27 14:13:51,018][20291] KL-divergence is very high: 170.4750 [2023-12-27 14:13:51,063][20347] Updated weights for policy 0, policy_version 1238713 (0.0008) [2023-12-27 14:13:51,070][20291] KL-divergence is very high: 143.6905 [2023-12-27 14:13:51,494][20381] Updated weights for policy 1, policy_version 1237625 (0.0008) [2023-12-27 14:13:51,559][20381] Updated weights for policy 1, policy_version 1237635 (0.0008) [2023-12-27 14:13:51,616][20381] Updated weights for policy 1, policy_version 1237645 (0.0007) [2023-12-27 14:13:51,677][20381] Updated weights for policy 1, policy_version 1237655 (0.0007) [2023-12-27 14:13:51,793][20347] Updated weights for policy 0, policy_version 1238723 (0.0007) [2023-12-27 14:13:51,842][20347] Updated weights for policy 0, policy_version 1238733 (0.0007) [2023-12-27 14:13:51,898][20347] Updated weights for policy 0, policy_version 1238743 (0.0008) [2023-12-27 14:13:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 634044416. Throughput: 0: 10466.8, 1: 10269.6. Samples: 634019280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:13:52,363][19170] Avg episode reward: [(0, '941.830'), (1, '980.520')] [2023-12-27 14:13:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001238752_317177856.pth... [2023-12-27 14:13:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001237536_316866560.pth [2023-12-27 14:13:52,416][20381] Updated weights for policy 1, policy_version 1237665 (0.0007) [2023-12-27 14:13:52,476][20381] Updated weights for policy 1, policy_version 1237675 (0.0008) [2023-12-27 14:13:52,541][20381] Updated weights for policy 1, policy_version 1237685 (0.0009) [2023-12-27 14:13:52,557][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001237688_316874752.pth... [2023-12-27 14:13:52,560][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001236472_316563456.pth [2023-12-27 14:13:52,575][20347] Updated weights for policy 0, policy_version 1238753 (0.0008) [2023-12-27 14:13:52,639][20347] Updated weights for policy 0, policy_version 1238763 (0.0006) [2023-12-27 14:13:52,704][20347] Updated weights for policy 0, policy_version 1238773 (0.0007) [2023-12-27 14:13:52,760][20347] Updated weights for policy 0, policy_version 1238783 (0.0008) [2023-12-27 14:13:53,150][20381] Updated weights for policy 1, policy_version 1237695 (0.0006) [2023-12-27 14:13:53,203][20381] Updated weights for policy 1, policy_version 1237705 (0.0007) [2023-12-27 14:13:53,258][20381] Updated weights for policy 1, policy_version 1237715 (0.0008) [2023-12-27 14:13:53,406][20347] Updated weights for policy 0, policy_version 1238793 (0.0006) [2023-12-27 14:13:53,473][20347] Updated weights for policy 0, policy_version 1238803 (0.0008) [2023-12-27 14:13:53,525][20347] Updated weights for policy 0, policy_version 1238813 (0.0006) [2023-12-27 14:13:53,854][20381] Updated weights for policy 1, policy_version 1237725 (0.0007) [2023-12-27 14:13:53,903][20381] Updated weights for policy 1, policy_version 1237735 (0.0008) [2023-12-27 14:13:53,955][20381] Updated weights for policy 1, policy_version 1237745 (0.0007) [2023-12-27 14:13:54,193][20347] Updated weights for policy 0, policy_version 1238823 (0.0005) [2023-12-27 14:13:54,239][20347] Updated weights for policy 0, policy_version 1238833 (0.0005) [2023-12-27 14:13:54,290][20347] Updated weights for policy 0, policy_version 1238843 (0.0005) [2023-12-27 14:13:54,618][20381] Updated weights for policy 1, policy_version 1237755 (0.0008) [2023-12-27 14:13:54,667][20381] Updated weights for policy 1, policy_version 1237765 (0.0005) [2023-12-27 14:13:54,727][20381] Updated weights for policy 1, policy_version 1237775 (0.0010) [2023-12-27 14:13:54,890][20347] Updated weights for policy 0, policy_version 1238853 (0.0006) [2023-12-27 14:13:54,941][20347] Updated weights for policy 0, policy_version 1238863 (0.0007) [2023-12-27 14:13:54,998][20347] Updated weights for policy 0, policy_version 1238873 (0.0007) [2023-12-27 14:13:55,397][20381] Updated weights for policy 1, policy_version 1237785 (0.0011) [2023-12-27 14:13:55,443][20381] Updated weights for policy 1, policy_version 1237795 (0.0010) [2023-12-27 14:13:55,488][20381] Updated weights for policy 1, policy_version 1237805 (0.0010) [2023-12-27 14:13:55,533][20381] Updated weights for policy 1, policy_version 1237815 (0.0010) [2023-12-27 14:13:55,710][20347] Updated weights for policy 0, policy_version 1238883 (0.0007) [2023-12-27 14:13:55,760][20347] Updated weights for policy 0, policy_version 1238893 (0.0007) [2023-12-27 14:13:55,814][20347] Updated weights for policy 0, policy_version 1238903 (0.0007) [2023-12-27 14:13:56,247][20381] Updated weights for policy 1, policy_version 1237825 (0.0008) [2023-12-27 14:13:56,296][20381] Updated weights for policy 1, policy_version 1237835 (0.0007) [2023-12-27 14:13:56,354][20381] Updated weights for policy 1, policy_version 1237845 (0.0008) [2023-12-27 14:13:56,516][20347] Updated weights for policy 0, policy_version 1238913 (0.0007) [2023-12-27 14:13:56,569][20347] Updated weights for policy 0, policy_version 1238923 (0.0008) [2023-12-27 14:13:56,627][20347] Updated weights for policy 0, policy_version 1238933 (0.0008) [2023-12-27 14:13:56,676][20347] Updated weights for policy 0, policy_version 1238943 (0.0007) [2023-12-27 14:13:56,977][20381] Updated weights for policy 1, policy_version 1237855 (0.0007) [2023-12-27 14:13:57,017][20381] Updated weights for policy 1, policy_version 1237865 (0.0005) [2023-12-27 14:13:57,070][20381] Updated weights for policy 1, policy_version 1237875 (0.0005) [2023-12-27 14:13:57,334][20347] Updated weights for policy 0, policy_version 1238953 (0.0006) [2023-12-27 14:13:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 634150912. Throughput: 0: 10440.4, 1: 10243.4. Samples: 634142256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:13:57,363][19170] Avg episode reward: [(0, '951.500'), (1, '1000.000')] [2023-12-27 14:13:57,382][20347] Updated weights for policy 0, policy_version 1238963 (0.0008) [2023-12-27 14:13:57,437][20347] Updated weights for policy 0, policy_version 1238973 (0.0007) [2023-12-27 14:13:57,642][20381] Updated weights for policy 1, policy_version 1237885 (0.0007) [2023-12-27 14:13:57,695][20381] Updated weights for policy 1, policy_version 1237895 (0.0008) [2023-12-27 14:13:57,750][20381] Updated weights for policy 1, policy_version 1237905 (0.0005) [2023-12-27 14:13:58,053][20347] Updated weights for policy 0, policy_version 1238983 (0.0006) [2023-12-27 14:13:58,103][20347] Updated weights for policy 0, policy_version 1238993 (0.0007) [2023-12-27 14:13:58,163][20347] Updated weights for policy 0, policy_version 1239003 (0.0007) [2023-12-27 14:13:58,495][20381] Updated weights for policy 1, policy_version 1237915 (0.0008) [2023-12-27 14:13:58,560][20381] Updated weights for policy 1, policy_version 1237925 (0.0010) [2023-12-27 14:13:58,628][20381] Updated weights for policy 1, policy_version 1237935 (0.0007) [2023-12-27 14:13:58,946][20347] Updated weights for policy 0, policy_version 1239013 (0.0007) [2023-12-27 14:13:59,010][20347] Updated weights for policy 0, policy_version 1239023 (0.0007) [2023-12-27 14:13:59,077][20347] Updated weights for policy 0, policy_version 1239033 (0.0006) [2023-12-27 14:13:59,335][20381] Updated weights for policy 1, policy_version 1237945 (0.0007) [2023-12-27 14:13:59,402][20381] Updated weights for policy 1, policy_version 1237955 (0.0009) [2023-12-27 14:13:59,461][20381] Updated weights for policy 1, policy_version 1237965 (0.0009) [2023-12-27 14:13:59,515][20381] Updated weights for policy 1, policy_version 1237975 (0.0010) [2023-12-27 14:13:59,826][20347] Updated weights for policy 0, policy_version 1239043 (0.0007) [2023-12-27 14:13:59,882][20347] Updated weights for policy 0, policy_version 1239053 (0.0008) [2023-12-27 14:13:59,939][20347] Updated weights for policy 0, policy_version 1239063 (0.0008) [2023-12-27 14:14:00,213][20381] Updated weights for policy 1, policy_version 1237985 (0.0009) [2023-12-27 14:14:00,277][20381] Updated weights for policy 1, policy_version 1237995 (0.0008) [2023-12-27 14:14:00,334][20381] Updated weights for policy 1, policy_version 1238005 (0.0008) [2023-12-27 14:14:00,588][20347] Updated weights for policy 0, policy_version 1239073 (0.0008) [2023-12-27 14:14:00,645][20347] Updated weights for policy 0, policy_version 1239083 (0.0010) [2023-12-27 14:14:00,702][20347] Updated weights for policy 0, policy_version 1239093 (0.0008) [2023-12-27 14:14:00,750][20347] Updated weights for policy 0, policy_version 1239103 (0.0005) [2023-12-27 14:14:00,974][20381] Updated weights for policy 1, policy_version 1238015 (0.0006) [2023-12-27 14:14:01,061][20381] Updated weights for policy 1, policy_version 1238025 (0.0006) [2023-12-27 14:14:01,121][20381] Updated weights for policy 1, policy_version 1238035 (0.0008) [2023-12-27 14:14:01,412][20347] Updated weights for policy 0, policy_version 1239113 (0.0010) [2023-12-27 14:14:01,460][20347] Updated weights for policy 0, policy_version 1239123 (0.0010) [2023-12-27 14:14:01,520][20347] Updated weights for policy 0, policy_version 1239133 (0.0009) [2023-12-27 14:14:01,751][20381] Updated weights for policy 1, policy_version 1238045 (0.0009) [2023-12-27 14:14:01,807][20381] Updated weights for policy 1, policy_version 1238055 (0.0008) [2023-12-27 14:14:01,867][20381] Updated weights for policy 1, policy_version 1238065 (0.0008) [2023-12-27 14:14:02,203][20347] Updated weights for policy 0, policy_version 1239143 (0.0007) [2023-12-27 14:14:02,263][20347] Updated weights for policy 0, policy_version 1239153 (0.0008) [2023-12-27 14:14:02,327][20347] Updated weights for policy 0, policy_version 1239163 (0.0007) [2023-12-27 14:14:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 634257408. Throughput: 0: 10405.6, 1: 10223.4. Samples: 634264412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:02,363][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 14:14:02,623][20381] Updated weights for policy 1, policy_version 1238075 (0.0008) [2023-12-27 14:14:02,672][20381] Updated weights for policy 1, policy_version 1238085 (0.0010) [2023-12-27 14:14:02,721][20381] Updated weights for policy 1, policy_version 1238095 (0.0007) [2023-12-27 14:14:02,951][20347] Updated weights for policy 0, policy_version 1239173 (0.0007) [2023-12-27 14:14:03,001][20347] Updated weights for policy 0, policy_version 1239183 (0.0007) [2023-12-27 14:14:03,057][20347] Updated weights for policy 0, policy_version 1239193 (0.0007) [2023-12-27 14:14:03,437][20381] Updated weights for policy 1, policy_version 1238105 (0.0008) [2023-12-27 14:14:03,491][20381] Updated weights for policy 1, policy_version 1238115 (0.0010) [2023-12-27 14:14:03,548][20381] Updated weights for policy 1, policy_version 1238125 (0.0010) [2023-12-27 14:14:03,603][20381] Updated weights for policy 1, policy_version 1238135 (0.0010) [2023-12-27 14:14:03,769][20347] Updated weights for policy 0, policy_version 1239203 (0.0007) [2023-12-27 14:14:03,826][20347] Updated weights for policy 0, policy_version 1239213 (0.0007) [2023-12-27 14:14:03,886][20347] Updated weights for policy 0, policy_version 1239223 (0.0007) [2023-12-27 14:14:04,350][20381] Updated weights for policy 1, policy_version 1238145 (0.0007) [2023-12-27 14:14:04,405][20381] Updated weights for policy 1, policy_version 1238155 (0.0008) [2023-12-27 14:14:04,460][20381] Updated weights for policy 1, policy_version 1238165 (0.0008) [2023-12-27 14:14:04,520][20347] Updated weights for policy 0, policy_version 1239233 (0.0006) [2023-12-27 14:14:04,575][20347] Updated weights for policy 0, policy_version 1239243 (0.0006) [2023-12-27 14:14:04,624][20347] Updated weights for policy 0, policy_version 1239253 (0.0008) [2023-12-27 14:14:04,680][20347] Updated weights for policy 0, policy_version 1239263 (0.0008) [2023-12-27 14:14:05,063][20381] Updated weights for policy 1, policy_version 1238175 (0.0006) [2023-12-27 14:14:05,110][20381] Updated weights for policy 1, policy_version 1238185 (0.0005) [2023-12-27 14:14:05,167][20381] Updated weights for policy 1, policy_version 1238195 (0.0006) [2023-12-27 14:14:05,380][20347] Updated weights for policy 0, policy_version 1239273 (0.0005) [2023-12-27 14:14:05,443][20347] Updated weights for policy 0, policy_version 1239283 (0.0005) [2023-12-27 14:14:05,502][20347] Updated weights for policy 0, policy_version 1239293 (0.0005) [2023-12-27 14:14:05,748][20381] Updated weights for policy 1, policy_version 1238205 (0.0009) [2023-12-27 14:14:05,797][20381] Updated weights for policy 1, policy_version 1238215 (0.0010) [2023-12-27 14:14:05,851][20381] Updated weights for policy 1, policy_version 1238225 (0.0010) [2023-12-27 14:14:06,114][20347] Updated weights for policy 0, policy_version 1239303 (0.0006) [2023-12-27 14:14:06,172][20347] Updated weights for policy 0, policy_version 1239313 (0.0008) [2023-12-27 14:14:06,230][20347] Updated weights for policy 0, policy_version 1239323 (0.0007) [2023-12-27 14:14:06,567][20381] Updated weights for policy 1, policy_version 1238235 (0.0009) [2023-12-27 14:14:06,629][20381] Updated weights for policy 1, policy_version 1238245 (0.0008) [2023-12-27 14:14:06,679][20381] Updated weights for policy 1, policy_version 1238255 (0.0008) [2023-12-27 14:14:06,900][20347] Updated weights for policy 0, policy_version 1239333 (0.0008) [2023-12-27 14:14:06,956][20347] Updated weights for policy 0, policy_version 1239343 (0.0008) [2023-12-27 14:14:07,005][20347] Updated weights for policy 0, policy_version 1239353 (0.0008) [2023-12-27 14:14:07,343][20381] Updated weights for policy 1, policy_version 1238265 (0.0008) [2023-12-27 14:14:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 634355712. Throughput: 0: 10385.0, 1: 10247.8. Samples: 634325728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 14:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001239360_317333504.pth... [2023-12-27 14:14:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001238144_317022208.pth [2023-12-27 14:14:07,398][20381] Updated weights for policy 1, policy_version 1238275 (0.0010) [2023-12-27 14:14:07,443][20381] Updated weights for policy 1, policy_version 1238285 (0.0010) [2023-12-27 14:14:07,494][20381] Updated weights for policy 1, policy_version 1238295 (0.0010) [2023-12-27 14:14:07,499][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001238296_317030400.pth... [2023-12-27 14:14:07,502][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001237080_316719104.pth [2023-12-27 14:14:07,723][20347] Updated weights for policy 0, policy_version 1239363 (0.0007) [2023-12-27 14:14:07,773][20347] Updated weights for policy 0, policy_version 1239373 (0.0005) [2023-12-27 14:14:07,823][20347] Updated weights for policy 0, policy_version 1239383 (0.0006) [2023-12-27 14:14:08,075][20381] Updated weights for policy 1, policy_version 1238305 (0.0007) [2023-12-27 14:14:08,130][20381] Updated weights for policy 1, policy_version 1238315 (0.0010) [2023-12-27 14:14:08,183][20381] Updated weights for policy 1, policy_version 1238325 (0.0008) [2023-12-27 14:14:08,422][20347] Updated weights for policy 0, policy_version 1239393 (0.0006) [2023-12-27 14:14:08,474][20347] Updated weights for policy 0, policy_version 1239403 (0.0005) [2023-12-27 14:14:08,526][20347] Updated weights for policy 0, policy_version 1239413 (0.0005) [2023-12-27 14:14:08,583][20347] Updated weights for policy 0, policy_version 1239423 (0.0007) [2023-12-27 14:14:08,866][20381] Updated weights for policy 1, policy_version 1238335 (0.0007) [2023-12-27 14:14:08,916][20381] Updated weights for policy 1, policy_version 1238345 (0.0008) [2023-12-27 14:14:08,968][20381] Updated weights for policy 1, policy_version 1238355 (0.0005) [2023-12-27 14:14:09,247][20347] Updated weights for policy 0, policy_version 1239433 (0.0008) [2023-12-27 14:14:09,311][20347] Updated weights for policy 0, policy_version 1239443 (0.0008) [2023-12-27 14:14:09,382][20347] Updated weights for policy 0, policy_version 1239453 (0.0008) [2023-12-27 14:14:09,681][20381] Updated weights for policy 1, policy_version 1238365 (0.0007) [2023-12-27 14:14:09,743][20381] Updated weights for policy 1, policy_version 1238375 (0.0008) [2023-12-27 14:14:09,801][20381] Updated weights for policy 1, policy_version 1238385 (0.0008) [2023-12-27 14:14:10,176][20347] Updated weights for policy 0, policy_version 1239463 (0.0009) [2023-12-27 14:14:10,239][20347] Updated weights for policy 0, policy_version 1239473 (0.0008) [2023-12-27 14:14:10,290][20347] Updated weights for policy 0, policy_version 1239483 (0.0008) [2023-12-27 14:14:10,461][20381] Updated weights for policy 1, policy_version 1238395 (0.0007) [2023-12-27 14:14:10,511][20381] Updated weights for policy 1, policy_version 1238405 (0.0006) [2023-12-27 14:14:10,574][20381] Updated weights for policy 1, policy_version 1238415 (0.0005) [2023-12-27 14:14:10,993][20347] Updated weights for policy 0, policy_version 1239493 (0.0008) [2023-12-27 14:14:11,057][20347] Updated weights for policy 0, policy_version 1239503 (0.0009) [2023-12-27 14:14:11,113][20347] Updated weights for policy 0, policy_version 1239513 (0.0008) [2023-12-27 14:14:11,221][20381] Updated weights for policy 1, policy_version 1238425 (0.0005) [2023-12-27 14:14:11,281][20381] Updated weights for policy 1, policy_version 1238435 (0.0006) [2023-12-27 14:14:11,341][20381] Updated weights for policy 1, policy_version 1238445 (0.0008) [2023-12-27 14:14:11,407][20381] Updated weights for policy 1, policy_version 1238455 (0.0007) [2023-12-27 14:14:11,885][20347] Updated weights for policy 0, policy_version 1239523 (0.0007) [2023-12-27 14:14:11,931][20347] Updated weights for policy 0, policy_version 1239533 (0.0007) [2023-12-27 14:14:11,992][20347] Updated weights for policy 0, policy_version 1239543 (0.0007) [2023-12-27 14:14:12,090][20381] Updated weights for policy 1, policy_version 1238465 (0.0005) [2023-12-27 14:14:12,152][20381] Updated weights for policy 1, policy_version 1238475 (0.0008) [2023-12-27 14:14:12,201][20381] Updated weights for policy 1, policy_version 1238485 (0.0008) [2023-12-27 14:14:12,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 634462208. Throughput: 0: 10359.3, 1: 10343.8. Samples: 634451072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:12,363][19170] Avg episode reward: [(0, '980.560'), (1, '1000.000')] [2023-12-27 14:14:12,705][20347] Updated weights for policy 0, policy_version 1239553 (0.0008) [2023-12-27 14:14:12,761][20347] Updated weights for policy 0, policy_version 1239563 (0.0007) [2023-12-27 14:14:12,814][20347] Updated weights for policy 0, policy_version 1239573 (0.0007) [2023-12-27 14:14:12,873][20347] Updated weights for policy 0, policy_version 1239583 (0.0008) [2023-12-27 14:14:12,897][20381] Updated weights for policy 1, policy_version 1238495 (0.0009) [2023-12-27 14:14:12,956][20381] Updated weights for policy 1, policy_version 1238505 (0.0011) [2023-12-27 14:14:13,016][20381] Updated weights for policy 1, policy_version 1238515 (0.0010) [2023-12-27 14:14:13,578][20347] Updated weights for policy 0, policy_version 1239593 (0.0007) [2023-12-27 14:14:13,636][20347] Updated weights for policy 0, policy_version 1239603 (0.0007) [2023-12-27 14:14:13,696][20347] Updated weights for policy 0, policy_version 1239613 (0.0007) [2023-12-27 14:14:13,736][20381] Updated weights for policy 1, policy_version 1238525 (0.0008) [2023-12-27 14:14:13,791][20381] Updated weights for policy 1, policy_version 1238535 (0.0008) [2023-12-27 14:14:13,849][20381] Updated weights for policy 1, policy_version 1238545 (0.0006) [2023-12-27 14:14:14,326][20347] Updated weights for policy 0, policy_version 1239623 (0.0008) [2023-12-27 14:14:14,339][20291] KL-divergence is very high: 241.3853 [2023-12-27 14:14:14,389][20291] KL-divergence is very high: 333.4487 [2023-12-27 14:14:14,390][20347] Updated weights for policy 0, policy_version 1239633 (0.0008) [2023-12-27 14:14:14,433][20291] KL-divergence is very high: 200.7927 [2023-12-27 14:14:14,446][20347] Updated weights for policy 0, policy_version 1239643 (0.0007) [2023-12-27 14:14:14,494][20381] Updated weights for policy 1, policy_version 1238555 (0.0005) [2023-12-27 14:14:14,555][20381] Updated weights for policy 1, policy_version 1238565 (0.0006) [2023-12-27 14:14:14,618][20381] Updated weights for policy 1, policy_version 1238575 (0.0007) [2023-12-27 14:14:15,159][20347] Updated weights for policy 0, policy_version 1239653 (0.0008) [2023-12-27 14:14:15,220][20347] Updated weights for policy 0, policy_version 1239663 (0.0007) [2023-12-27 14:14:15,281][20347] Updated weights for policy 0, policy_version 1239673 (0.0007) [2023-12-27 14:14:15,332][20381] Updated weights for policy 1, policy_version 1238585 (0.0006) [2023-12-27 14:14:15,390][20381] Updated weights for policy 1, policy_version 1238595 (0.0007) [2023-12-27 14:14:15,453][20381] Updated weights for policy 1, policy_version 1238605 (0.0007) [2023-12-27 14:14:15,517][20381] Updated weights for policy 1, policy_version 1238615 (0.0008) [2023-12-27 14:14:15,987][20347] Updated weights for policy 0, policy_version 1239683 (0.0008) [2023-12-27 14:14:16,025][20291] KL-divergence is very high: 178.7398 [2023-12-27 14:14:16,041][20347] Updated weights for policy 0, policy_version 1239693 (0.0007) [2023-12-27 14:14:16,068][20291] KL-divergence is very high: 134.5750 [2023-12-27 14:14:16,097][20347] Updated weights for policy 0, policy_version 1239703 (0.0007) [2023-12-27 14:14:16,114][20291] KL-divergence is very high: 103.5628 [2023-12-27 14:14:16,146][20381] Updated weights for policy 1, policy_version 1238625 (0.0010) [2023-12-27 14:14:16,204][20381] Updated weights for policy 1, policy_version 1238635 (0.0010) [2023-12-27 14:14:16,261][20381] Updated weights for policy 1, policy_version 1238645 (0.0010) [2023-12-27 14:14:16,776][20347] Updated weights for policy 0, policy_version 1239713 (0.0006) [2023-12-27 14:14:16,832][20347] Updated weights for policy 0, policy_version 1239723 (0.0007) [2023-12-27 14:14:16,889][20347] Updated weights for policy 0, policy_version 1239733 (0.0006) [2023-12-27 14:14:16,917][20381] Updated weights for policy 1, policy_version 1238655 (0.0007) [2023-12-27 14:14:16,949][20347] Updated weights for policy 0, policy_version 1239743 (0.0006) [2023-12-27 14:14:16,978][20381] Updated weights for policy 1, policy_version 1238665 (0.0007) [2023-12-27 14:14:17,031][20381] Updated weights for policy 1, policy_version 1238675 (0.0010) [2023-12-27 14:14:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 634560512. Throughput: 0: 10237.8, 1: 10321.3. Samples: 634570600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:17,363][19170] Avg episode reward: [(0, '951.170'), (1, '990.270')] [2023-12-27 14:14:17,646][20347] Updated weights for policy 0, policy_version 1239753 (0.0007) [2023-12-27 14:14:17,699][20347] Updated weights for policy 0, policy_version 1239763 (0.0006) [2023-12-27 14:14:17,704][20381] Updated weights for policy 1, policy_version 1238685 (0.0011) [2023-12-27 14:14:17,757][20381] Updated weights for policy 1, policy_version 1238695 (0.0011) [2023-12-27 14:14:17,758][20347] Updated weights for policy 0, policy_version 1239773 (0.0005) [2023-12-27 14:14:17,803][20381] Updated weights for policy 1, policy_version 1238705 (0.0010) [2023-12-27 14:14:18,458][20381] Updated weights for policy 1, policy_version 1238715 (0.0011) [2023-12-27 14:14:18,466][20347] Updated weights for policy 0, policy_version 1239783 (0.0007) [2023-12-27 14:14:18,524][20381] Updated weights for policy 1, policy_version 1238725 (0.0011) [2023-12-27 14:14:18,529][20347] Updated weights for policy 0, policy_version 1239793 (0.0007) [2023-12-27 14:14:18,579][20347] Updated weights for policy 0, policy_version 1239803 (0.0005) [2023-12-27 14:14:18,581][20381] Updated weights for policy 1, policy_version 1238735 (0.0011) [2023-12-27 14:14:19,235][20381] Updated weights for policy 1, policy_version 1238745 (0.0009) [2023-12-27 14:14:19,256][20347] Updated weights for policy 0, policy_version 1239813 (0.0006) [2023-12-27 14:14:19,294][20381] Updated weights for policy 1, policy_version 1238755 (0.0008) [2023-12-27 14:14:19,314][20347] Updated weights for policy 0, policy_version 1239823 (0.0007) [2023-12-27 14:14:19,362][20381] Updated weights for policy 1, policy_version 1238765 (0.0008) [2023-12-27 14:14:19,376][20347] Updated weights for policy 0, policy_version 1239833 (0.0007) [2023-12-27 14:14:19,414][20381] Updated weights for policy 1, policy_version 1238775 (0.0008) [2023-12-27 14:14:20,133][20347] Updated weights for policy 0, policy_version 1239843 (0.0008) [2023-12-27 14:14:20,183][20381] Updated weights for policy 1, policy_version 1238785 (0.0008) [2023-12-27 14:14:20,187][20347] Updated weights for policy 0, policy_version 1239853 (0.0006) [2023-12-27 14:14:20,237][20291] KL-divergence is very high: 616.6479 [2023-12-27 14:14:20,242][20347] Updated weights for policy 0, policy_version 1239863 (0.0006) [2023-12-27 14:14:20,248][20381] Updated weights for policy 1, policy_version 1238795 (0.0008) [2023-12-27 14:14:20,282][20291] KL-divergence is very high: 256.1531 [2023-12-27 14:14:20,308][20381] Updated weights for policy 1, policy_version 1238805 (0.0008) [2023-12-27 14:14:20,932][20347] Updated weights for policy 0, policy_version 1239873 (0.0006) [2023-12-27 14:14:20,988][20347] Updated weights for policy 0, policy_version 1239883 (0.0005) [2023-12-27 14:14:21,004][20291] KL-divergence is very high: 280.1350 [2023-12-27 14:14:21,028][20381] Updated weights for policy 1, policy_version 1238815 (0.0008) [2023-12-27 14:14:21,047][20347] Updated weights for policy 0, policy_version 1239893 (0.0006) [2023-12-27 14:14:21,052][20291] KL-divergence is very high: 147.4240 [2023-12-27 14:14:21,097][20381] Updated weights for policy 1, policy_version 1238825 (0.0006) [2023-12-27 14:14:21,108][20347] Updated weights for policy 0, policy_version 1239903 (0.0007) [2023-12-27 14:14:21,164][20381] Updated weights for policy 1, policy_version 1238835 (0.0007) [2023-12-27 14:14:21,856][20347] Updated weights for policy 0, policy_version 1239913 (0.0008) [2023-12-27 14:14:21,866][20381] Updated weights for policy 1, policy_version 1238845 (0.0007) [2023-12-27 14:14:21,918][20291] KL-divergence is very high: 463.8380 [2023-12-27 14:14:21,918][20347] Updated weights for policy 0, policy_version 1239923 (0.0007) [2023-12-27 14:14:21,921][20381] Updated weights for policy 1, policy_version 1238855 (0.0007) [2023-12-27 14:14:21,972][20347] Updated weights for policy 0, policy_version 1239933 (0.0007) [2023-12-27 14:14:21,981][20381] Updated weights for policy 1, policy_version 1238865 (0.0006) [2023-12-27 14:14:22,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 634658816. Throughput: 0: 10197.4, 1: 10259.5. Samples: 634630476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:22,363][19170] Avg episode reward: [(0, '941.300'), (1, '990.270')] [2023-12-27 14:14:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001239936_317480960.pth... [2023-12-27 14:14:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001238872_317177856.pth... [2023-12-27 14:14:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001237688_316874752.pth [2023-12-27 14:14:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001238752_317177856.pth [2023-12-27 14:14:22,688][20347] Updated weights for policy 0, policy_version 1239943 (0.0007) [2023-12-27 14:14:22,698][20381] Updated weights for policy 1, policy_version 1238875 (0.0008) [2023-12-27 14:14:22,749][20347] Updated weights for policy 0, policy_version 1239953 (0.0007) [2023-12-27 14:14:22,755][20381] Updated weights for policy 1, policy_version 1238885 (0.0006) [2023-12-27 14:14:22,809][20347] Updated weights for policy 0, policy_version 1239963 (0.0007) [2023-12-27 14:14:22,814][20381] Updated weights for policy 1, policy_version 1238895 (0.0008) [2023-12-27 14:14:23,505][20381] Updated weights for policy 1, policy_version 1238905 (0.0007) [2023-12-27 14:14:23,507][20347] Updated weights for policy 0, policy_version 1239973 (0.0008) [2023-12-27 14:14:23,561][20381] Updated weights for policy 1, policy_version 1238915 (0.0006) [2023-12-27 14:14:23,563][20347] Updated weights for policy 0, policy_version 1239983 (0.0005) [2023-12-27 14:14:23,613][20381] Updated weights for policy 1, policy_version 1238925 (0.0005) [2023-12-27 14:14:23,614][20347] Updated weights for policy 0, policy_version 1239993 (0.0007) [2023-12-27 14:14:23,665][20381] Updated weights for policy 1, policy_version 1238935 (0.0005) [2023-12-27 14:14:24,250][20381] Updated weights for policy 1, policy_version 1238945 (0.0007) [2023-12-27 14:14:24,297][20381] Updated weights for policy 1, policy_version 1238955 (0.0007) [2023-12-27 14:14:24,328][20347] Updated weights for policy 0, policy_version 1240003 (0.0008) [2023-12-27 14:14:24,351][20381] Updated weights for policy 1, policy_version 1238965 (0.0009) [2023-12-27 14:14:24,379][20347] Updated weights for policy 0, policy_version 1240013 (0.0007) [2023-12-27 14:14:24,435][20347] Updated weights for policy 0, policy_version 1240023 (0.0006) [2023-12-27 14:14:25,051][20381] Updated weights for policy 1, policy_version 1238975 (0.0008) [2023-12-27 14:14:25,106][20381] Updated weights for policy 1, policy_version 1238985 (0.0007) [2023-12-27 14:14:25,116][20347] Updated weights for policy 0, policy_version 1240033 (0.0007) [2023-12-27 14:14:25,162][20381] Updated weights for policy 1, policy_version 1238995 (0.0006) [2023-12-27 14:14:25,180][20347] Updated weights for policy 0, policy_version 1240043 (0.0006) [2023-12-27 14:14:25,240][20347] Updated weights for policy 0, policy_version 1240053 (0.0007) [2023-12-27 14:14:25,283][20347] Updated weights for policy 0, policy_version 1240063 (0.0009) [2023-12-27 14:14:25,873][20381] Updated weights for policy 1, policy_version 1239005 (0.0008) [2023-12-27 14:14:25,890][20347] Updated weights for policy 0, policy_version 1240073 (0.0007) [2023-12-27 14:14:25,929][20381] Updated weights for policy 1, policy_version 1239015 (0.0005) [2023-12-27 14:14:25,942][20347] Updated weights for policy 0, policy_version 1240083 (0.0005) [2023-12-27 14:14:25,988][20381] Updated weights for policy 1, policy_version 1239025 (0.0008) [2023-12-27 14:14:25,998][20347] Updated weights for policy 0, policy_version 1240093 (0.0006) [2023-12-27 14:14:26,655][20381] Updated weights for policy 1, policy_version 1239035 (0.0007) [2023-12-27 14:14:26,698][20347] Updated weights for policy 0, policy_version 1240103 (0.0007) [2023-12-27 14:14:26,714][20381] Updated weights for policy 1, policy_version 1239045 (0.0005) [2023-12-27 14:14:26,757][20347] Updated weights for policy 0, policy_version 1240113 (0.0008) [2023-12-27 14:14:26,775][20381] Updated weights for policy 1, policy_version 1239055 (0.0009) [2023-12-27 14:14:26,812][20347] Updated weights for policy 0, policy_version 1240123 (0.0008) [2023-12-27 14:14:27,355][20381] Updated weights for policy 1, policy_version 1239065 (0.0011) [2023-12-27 14:14:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 634757120. Throughput: 0: 10132.5, 1: 10244.6. Samples: 634750720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:27,363][19170] Avg episode reward: [(0, '941.410'), (1, '1000.000')] [2023-12-27 14:14:27,412][20381] Updated weights for policy 1, policy_version 1239075 (0.0010) [2023-12-27 14:14:27,466][20381] Updated weights for policy 1, policy_version 1239085 (0.0010) [2023-12-27 14:14:27,475][20347] Updated weights for policy 0, policy_version 1240133 (0.0006) [2023-12-27 14:14:27,523][20381] Updated weights for policy 1, policy_version 1239095 (0.0010) [2023-12-27 14:14:27,531][20347] Updated weights for policy 0, policy_version 1240143 (0.0005) [2023-12-27 14:14:27,585][20347] Updated weights for policy 0, policy_version 1240153 (0.0005) [2023-12-27 14:14:28,126][20381] Updated weights for policy 1, policy_version 1239105 (0.0006) [2023-12-27 14:14:28,191][20381] Updated weights for policy 1, policy_version 1239115 (0.0006) [2023-12-27 14:14:28,211][20347] Updated weights for policy 0, policy_version 1240163 (0.0006) [2023-12-27 14:14:28,247][20381] Updated weights for policy 1, policy_version 1239125 (0.0007) [2023-12-27 14:14:28,268][20347] Updated weights for policy 0, policy_version 1240173 (0.0008) [2023-12-27 14:14:28,320][20347] Updated weights for policy 0, policy_version 1240183 (0.0010) [2023-12-27 14:14:28,851][20381] Updated weights for policy 1, policy_version 1239135 (0.0007) [2023-12-27 14:14:28,904][20381] Updated weights for policy 1, policy_version 1239145 (0.0007) [2023-12-27 14:14:28,954][20381] Updated weights for policy 1, policy_version 1239155 (0.0007) [2023-12-27 14:14:28,990][20347] Updated weights for policy 0, policy_version 1240193 (0.0010) [2023-12-27 14:14:29,035][20347] Updated weights for policy 0, policy_version 1240203 (0.0009) [2023-12-27 14:14:29,077][20347] Updated weights for policy 0, policy_version 1240213 (0.0010) [2023-12-27 14:14:29,122][20347] Updated weights for policy 0, policy_version 1240223 (0.0010) [2023-12-27 14:14:29,645][20381] Updated weights for policy 1, policy_version 1239165 (0.0007) [2023-12-27 14:14:29,708][20381] Updated weights for policy 1, policy_version 1239175 (0.0005) [2023-12-27 14:14:29,762][20381] Updated weights for policy 1, policy_version 1239185 (0.0009) [2023-12-27 14:14:29,875][20347] Updated weights for policy 0, policy_version 1240233 (0.0011) [2023-12-27 14:14:29,880][20291] KL-divergence is very high: 228.6426 [2023-12-27 14:14:29,888][20291] KL-divergence is very high: 512.6409 [2023-12-27 14:14:29,932][20291] KL-divergence is very high: 298.9854 [2023-12-27 14:14:29,940][20291] KL-divergence is very high: 592.8341 [2023-12-27 14:14:29,940][20347] Updated weights for policy 0, policy_version 1240243 (0.0011) [2023-12-27 14:14:29,983][20291] KL-divergence is very high: 110.0791 [2023-12-27 14:14:29,989][20291] KL-divergence is very high: 256.3078 [2023-12-27 14:14:30,000][20347] Updated weights for policy 0, policy_version 1240253 (0.0010) [2023-12-27 14:14:30,407][20381] Updated weights for policy 1, policy_version 1239195 (0.0006) [2023-12-27 14:14:30,463][20381] Updated weights for policy 1, policy_version 1239205 (0.0008) [2023-12-27 14:14:30,520][20381] Updated weights for policy 1, policy_version 1239215 (0.0007) [2023-12-27 14:14:30,698][20347] Updated weights for policy 0, policy_version 1240263 (0.0008) [2023-12-27 14:14:30,758][20347] Updated weights for policy 0, policy_version 1240273 (0.0006) [2023-12-27 14:14:30,807][20347] Updated weights for policy 0, policy_version 1240283 (0.0007) [2023-12-27 14:14:31,201][20381] Updated weights for policy 1, policy_version 1239225 (0.0009) [2023-12-27 14:14:31,263][20381] Updated weights for policy 1, policy_version 1239235 (0.0008) [2023-12-27 14:14:31,324][20381] Updated weights for policy 1, policy_version 1239245 (0.0008) [2023-12-27 14:14:31,388][20381] Updated weights for policy 1, policy_version 1239255 (0.0007) [2023-12-27 14:14:31,464][20347] Updated weights for policy 0, policy_version 1240293 (0.0007) [2023-12-27 14:14:31,516][20347] Updated weights for policy 0, policy_version 1240303 (0.0007) [2023-12-27 14:14:31,572][20347] Updated weights for policy 0, policy_version 1240313 (0.0005) [2023-12-27 14:14:32,035][20381] Updated weights for policy 1, policy_version 1239265 (0.0007) [2023-12-27 14:14:32,089][20381] Updated weights for policy 1, policy_version 1239275 (0.0007) [2023-12-27 14:14:32,148][20381] Updated weights for policy 1, policy_version 1239285 (0.0008) [2023-12-27 14:14:32,308][20347] Updated weights for policy 0, policy_version 1240323 (0.0007) [2023-12-27 14:14:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 634863616. Throughput: 0: 10136.3, 1: 10239.1. Samples: 634875736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:32,363][19170] Avg episode reward: [(0, '951.170'), (1, '1000.000')] [2023-12-27 14:14:32,383][20347] Updated weights for policy 0, policy_version 1240333 (0.0008) [2023-12-27 14:14:32,436][20347] Updated weights for policy 0, policy_version 1240343 (0.0008) [2023-12-27 14:14:32,820][20381] Updated weights for policy 1, policy_version 1239295 (0.0008) [2023-12-27 14:14:32,876][20381] Updated weights for policy 1, policy_version 1239305 (0.0010) [2023-12-27 14:14:32,929][20381] Updated weights for policy 1, policy_version 1239315 (0.0010) [2023-12-27 14:14:33,107][20347] Updated weights for policy 0, policy_version 1240353 (0.0007) [2023-12-27 14:14:33,161][20347] Updated weights for policy 0, policy_version 1240363 (0.0008) [2023-12-27 14:14:33,209][20347] Updated weights for policy 0, policy_version 1240373 (0.0008) [2023-12-27 14:14:33,261][20347] Updated weights for policy 0, policy_version 1240383 (0.0008) [2023-12-27 14:14:33,585][20381] Updated weights for policy 1, policy_version 1239325 (0.0010) [2023-12-27 14:14:33,631][20381] Updated weights for policy 1, policy_version 1239335 (0.0008) [2023-12-27 14:14:33,682][20381] Updated weights for policy 1, policy_version 1239345 (0.0005) [2023-12-27 14:14:33,982][20347] Updated weights for policy 0, policy_version 1240393 (0.0008) [2023-12-27 14:14:34,037][20347] Updated weights for policy 0, policy_version 1240403 (0.0010) [2023-12-27 14:14:34,091][20347] Updated weights for policy 0, policy_version 1240413 (0.0005) [2023-12-27 14:14:34,309][20381] Updated weights for policy 1, policy_version 1239355 (0.0007) [2023-12-27 14:14:34,367][20381] Updated weights for policy 1, policy_version 1239365 (0.0011) [2023-12-27 14:14:34,420][20381] Updated weights for policy 1, policy_version 1239375 (0.0011) [2023-12-27 14:14:34,689][20347] Updated weights for policy 0, policy_version 1240423 (0.0006) [2023-12-27 14:14:34,742][20347] Updated weights for policy 0, policy_version 1240433 (0.0005) [2023-12-27 14:14:34,794][20347] Updated weights for policy 0, policy_version 1240443 (0.0006) [2023-12-27 14:14:35,109][20381] Updated weights for policy 1, policy_version 1239385 (0.0011) [2023-12-27 14:14:35,158][20381] Updated weights for policy 1, policy_version 1239395 (0.0010) [2023-12-27 14:14:35,200][20381] Updated weights for policy 1, policy_version 1239405 (0.0010) [2023-12-27 14:14:35,250][20381] Updated weights for policy 1, policy_version 1239415 (0.0010) [2023-12-27 14:14:35,379][20347] Updated weights for policy 0, policy_version 1240453 (0.0006) [2023-12-27 14:14:35,436][20347] Updated weights for policy 0, policy_version 1240463 (0.0007) [2023-12-27 14:14:35,493][20347] Updated weights for policy 0, policy_version 1240473 (0.0007) [2023-12-27 14:14:35,956][20381] Updated weights for policy 1, policy_version 1239425 (0.0011) [2023-12-27 14:14:36,014][20381] Updated weights for policy 1, policy_version 1239435 (0.0010) [2023-12-27 14:14:36,067][20347] Updated weights for policy 0, policy_version 1240483 (0.0007) [2023-12-27 14:14:36,070][20381] Updated weights for policy 1, policy_version 1239445 (0.0011) [2023-12-27 14:14:36,138][20347] Updated weights for policy 0, policy_version 1240493 (0.0008) [2023-12-27 14:14:36,198][20347] Updated weights for policy 0, policy_version 1240503 (0.0006) [2023-12-27 14:14:36,711][20381] Updated weights for policy 1, policy_version 1239455 (0.0009) [2023-12-27 14:14:36,760][20381] Updated weights for policy 1, policy_version 1239465 (0.0009) [2023-12-27 14:14:36,812][20381] Updated weights for policy 1, policy_version 1239475 (0.0008) [2023-12-27 14:14:36,853][20347] Updated weights for policy 0, policy_version 1240513 (0.0008) [2023-12-27 14:14:36,917][20347] Updated weights for policy 0, policy_version 1240523 (0.0007) [2023-12-27 14:14:36,970][20347] Updated weights for policy 0, policy_version 1240533 (0.0008) [2023-12-27 14:14:37,030][20347] Updated weights for policy 0, policy_version 1240543 (0.0008) [2023-12-27 14:14:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20479.9, 300 sec: 20604.9). Total num frames: 634970112. Throughput: 0: 10127.7, 1: 10306.5. Samples: 634938820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:37,364][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 14:14:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001239480_317333504.pth... [2023-12-27 14:14:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001240544_317636608.pth... [2023-12-27 14:14:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001238296_317030400.pth [2023-12-27 14:14:37,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001239360_317333504.pth [2023-12-27 14:14:37,538][20381] Updated weights for policy 1, policy_version 1239485 (0.0008) [2023-12-27 14:14:37,593][20381] Updated weights for policy 1, policy_version 1239495 (0.0007) [2023-12-27 14:14:37,649][20381] Updated weights for policy 1, policy_version 1239505 (0.0007) [2023-12-27 14:14:37,722][20347] Updated weights for policy 0, policy_version 1240553 (0.0007) [2023-12-27 14:14:37,771][20347] Updated weights for policy 0, policy_version 1240563 (0.0007) [2023-12-27 14:14:37,816][20347] Updated weights for policy 0, policy_version 1240573 (0.0008) [2023-12-27 14:14:38,304][20381] Updated weights for policy 1, policy_version 1239515 (0.0007) [2023-12-27 14:14:38,371][20381] Updated weights for policy 1, policy_version 1239525 (0.0008) [2023-12-27 14:14:38,431][20381] Updated weights for policy 1, policy_version 1239535 (0.0007) [2023-12-27 14:14:38,497][20347] Updated weights for policy 0, policy_version 1240583 (0.0007) [2023-12-27 14:14:38,560][20347] Updated weights for policy 0, policy_version 1240593 (0.0008) [2023-12-27 14:14:38,611][20347] Updated weights for policy 0, policy_version 1240603 (0.0009) [2023-12-27 14:14:38,993][20381] Updated weights for policy 1, policy_version 1239545 (0.0006) [2023-12-27 14:14:39,045][20381] Updated weights for policy 1, policy_version 1239555 (0.0005) [2023-12-27 14:14:39,098][20381] Updated weights for policy 1, policy_version 1239565 (0.0005) [2023-12-27 14:14:39,144][20381] Updated weights for policy 1, policy_version 1239575 (0.0005) [2023-12-27 14:14:39,378][20347] Updated weights for policy 0, policy_version 1240613 (0.0009) [2023-12-27 14:14:39,460][20347] Updated weights for policy 0, policy_version 1240623 (0.0009) [2023-12-27 14:14:39,522][20347] Updated weights for policy 0, policy_version 1240633 (0.0008) [2023-12-27 14:14:39,803][20381] Updated weights for policy 1, policy_version 1239585 (0.0008) [2023-12-27 14:14:39,870][20381] Updated weights for policy 1, policy_version 1239595 (0.0007) [2023-12-27 14:14:39,929][20381] Updated weights for policy 1, policy_version 1239605 (0.0008) [2023-12-27 14:14:40,246][20347] Updated weights for policy 0, policy_version 1240643 (0.0007) [2023-12-27 14:14:40,312][20347] Updated weights for policy 0, policy_version 1240653 (0.0008) [2023-12-27 14:14:40,373][20347] Updated weights for policy 0, policy_version 1240663 (0.0008) [2023-12-27 14:14:40,627][20381] Updated weights for policy 1, policy_version 1239615 (0.0007) [2023-12-27 14:14:40,677][20381] Updated weights for policy 1, policy_version 1239625 (0.0005) [2023-12-27 14:14:40,730][20381] Updated weights for policy 1, policy_version 1239635 (0.0005) [2023-12-27 14:14:41,096][20347] Updated weights for policy 0, policy_version 1240673 (0.0007) [2023-12-27 14:14:41,160][20347] Updated weights for policy 0, policy_version 1240683 (0.0008) [2023-12-27 14:14:41,213][20347] Updated weights for policy 0, policy_version 1240693 (0.0008) [2023-12-27 14:14:41,279][20347] Updated weights for policy 0, policy_version 1240703 (0.0008) [2023-12-27 14:14:41,420][20381] Updated weights for policy 1, policy_version 1239645 (0.0007) [2023-12-27 14:14:41,482][20381] Updated weights for policy 1, policy_version 1239655 (0.0008) [2023-12-27 14:14:41,531][20381] Updated weights for policy 1, policy_version 1239665 (0.0006) [2023-12-27 14:14:42,067][20347] Updated weights for policy 0, policy_version 1240713 (0.0008) [2023-12-27 14:14:42,133][20347] Updated weights for policy 0, policy_version 1240723 (0.0007) [2023-12-27 14:14:42,192][20347] Updated weights for policy 0, policy_version 1240733 (0.0008) [2023-12-27 14:14:42,230][20381] Updated weights for policy 1, policy_version 1239675 (0.0006) [2023-12-27 14:14:42,294][20381] Updated weights for policy 1, policy_version 1239685 (0.0007) [2023-12-27 14:14:42,360][20381] Updated weights for policy 1, policy_version 1239695 (0.0007) [2023-12-27 14:14:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 635068416. Throughput: 0: 10108.1, 1: 10335.8. Samples: 635062232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.250')] [2023-12-27 14:14:42,913][20347] Updated weights for policy 0, policy_version 1240743 (0.0008) [2023-12-27 14:14:42,972][20347] Updated weights for policy 0, policy_version 1240753 (0.0007) [2023-12-27 14:14:43,021][20381] Updated weights for policy 1, policy_version 1239705 (0.0007) [2023-12-27 14:14:43,028][20347] Updated weights for policy 0, policy_version 1240763 (0.0007) [2023-12-27 14:14:43,084][20381] Updated weights for policy 1, policy_version 1239715 (0.0008) [2023-12-27 14:14:43,137][20381] Updated weights for policy 1, policy_version 1239725 (0.0008) [2023-12-27 14:14:43,184][20381] Updated weights for policy 1, policy_version 1239735 (0.0007) [2023-12-27 14:14:43,696][20347] Updated weights for policy 0, policy_version 1240773 (0.0005) [2023-12-27 14:14:43,748][20347] Updated weights for policy 0, policy_version 1240783 (0.0009) [2023-12-27 14:14:43,799][20347] Updated weights for policy 0, policy_version 1240793 (0.0008) [2023-12-27 14:14:43,840][20381] Updated weights for policy 1, policy_version 1239745 (0.0008) [2023-12-27 14:14:43,895][20381] Updated weights for policy 1, policy_version 1239755 (0.0009) [2023-12-27 14:14:43,942][20381] Updated weights for policy 1, policy_version 1239765 (0.0010) [2023-12-27 14:14:44,384][20347] Updated weights for policy 0, policy_version 1240803 (0.0006) [2023-12-27 14:14:44,442][20347] Updated weights for policy 0, policy_version 1240813 (0.0005) [2023-12-27 14:14:44,500][20347] Updated weights for policy 0, policy_version 1240823 (0.0006) [2023-12-27 14:14:44,594][20381] Updated weights for policy 1, policy_version 1239775 (0.0009) [2023-12-27 14:14:44,640][20381] Updated weights for policy 1, policy_version 1239785 (0.0008) [2023-12-27 14:14:44,691][20381] Updated weights for policy 1, policy_version 1239795 (0.0006) [2023-12-27 14:14:45,102][20347] Updated weights for policy 0, policy_version 1240833 (0.0007) [2023-12-27 14:14:45,167][20347] Updated weights for policy 0, policy_version 1240843 (0.0009) [2023-12-27 14:14:45,231][20347] Updated weights for policy 0, policy_version 1240853 (0.0008) [2023-12-27 14:14:45,302][20347] Updated weights for policy 0, policy_version 1240863 (0.0009) [2023-12-27 14:14:45,351][20381] Updated weights for policy 1, policy_version 1239805 (0.0007) [2023-12-27 14:14:45,409][20381] Updated weights for policy 1, policy_version 1239815 (0.0008) [2023-12-27 14:14:45,467][20381] Updated weights for policy 1, policy_version 1239825 (0.0008) [2023-12-27 14:14:45,978][20347] Updated weights for policy 0, policy_version 1240873 (0.0008) [2023-12-27 14:14:46,040][20347] Updated weights for policy 0, policy_version 1240883 (0.0008) [2023-12-27 14:14:46,098][20347] Updated weights for policy 0, policy_version 1240893 (0.0008) [2023-12-27 14:14:46,166][20381] Updated weights for policy 1, policy_version 1239835 (0.0008) [2023-12-27 14:14:46,225][20381] Updated weights for policy 1, policy_version 1239845 (0.0007) [2023-12-27 14:14:46,292][20381] Updated weights for policy 1, policy_version 1239855 (0.0006) [2023-12-27 14:14:46,789][20347] Updated weights for policy 0, policy_version 1240903 (0.0009) [2023-12-27 14:14:46,847][20347] Updated weights for policy 0, policy_version 1240913 (0.0010) [2023-12-27 14:14:46,899][20381] Updated weights for policy 1, policy_version 1239865 (0.0006) [2023-12-27 14:14:46,901][20347] Updated weights for policy 0, policy_version 1240923 (0.0010) [2023-12-27 14:14:46,955][20381] Updated weights for policy 1, policy_version 1239875 (0.0010) [2023-12-27 14:14:47,003][20381] Updated weights for policy 1, policy_version 1239885 (0.0010) [2023-12-27 14:14:47,064][20381] Updated weights for policy 1, policy_version 1239895 (0.0010) [2023-12-27 14:14:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 635174912. Throughput: 0: 10099.8, 1: 10352.1. Samples: 635184748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.250')] [2023-12-27 14:14:47,546][20347] Updated weights for policy 0, policy_version 1240933 (0.0009) [2023-12-27 14:14:47,599][20347] Updated weights for policy 0, policy_version 1240943 (0.0007) [2023-12-27 14:14:47,661][20347] Updated weights for policy 0, policy_version 1240953 (0.0006) [2023-12-27 14:14:47,742][20381] Updated weights for policy 1, policy_version 1239905 (0.0010) [2023-12-27 14:14:47,796][20381] Updated weights for policy 1, policy_version 1239915 (0.0010) [2023-12-27 14:14:47,845][20381] Updated weights for policy 1, policy_version 1239925 (0.0010) [2023-12-27 14:14:48,294][20347] Updated weights for policy 0, policy_version 1240963 (0.0006) [2023-12-27 14:14:48,359][20347] Updated weights for policy 0, policy_version 1240973 (0.0007) [2023-12-27 14:14:48,419][20347] Updated weights for policy 0, policy_version 1240983 (0.0008) [2023-12-27 14:14:48,467][20381] Updated weights for policy 1, policy_version 1239935 (0.0008) [2023-12-27 14:14:48,520][20381] Updated weights for policy 1, policy_version 1239945 (0.0011) [2023-12-27 14:14:48,578][20381] Updated weights for policy 1, policy_version 1239955 (0.0011) [2023-12-27 14:14:49,051][20347] Updated weights for policy 0, policy_version 1240993 (0.0008) [2023-12-27 14:14:49,102][20347] Updated weights for policy 0, policy_version 1241003 (0.0008) [2023-12-27 14:14:49,158][20347] Updated weights for policy 0, policy_version 1241013 (0.0007) [2023-12-27 14:14:49,217][20347] Updated weights for policy 0, policy_version 1241023 (0.0007) [2023-12-27 14:14:49,271][20381] Updated weights for policy 1, policy_version 1239965 (0.0010) [2023-12-27 14:14:49,331][20381] Updated weights for policy 1, policy_version 1239975 (0.0008) [2023-12-27 14:14:49,394][20381] Updated weights for policy 1, policy_version 1239985 (0.0008) [2023-12-27 14:14:49,988][20347] Updated weights for policy 0, policy_version 1241033 (0.0007) [2023-12-27 14:14:50,039][20347] Updated weights for policy 0, policy_version 1241043 (0.0008) [2023-12-27 14:14:50,072][20381] Updated weights for policy 1, policy_version 1239995 (0.0006) [2023-12-27 14:14:50,097][20347] Updated weights for policy 0, policy_version 1241053 (0.0007) [2023-12-27 14:14:50,131][20381] Updated weights for policy 1, policy_version 1240005 (0.0009) [2023-12-27 14:14:50,180][20381] Updated weights for policy 1, policy_version 1240015 (0.0010) [2023-12-27 14:14:50,801][20347] Updated weights for policy 0, policy_version 1241063 (0.0006) [2023-12-27 14:14:50,857][20347] Updated weights for policy 0, policy_version 1241073 (0.0005) [2023-12-27 14:14:50,871][20381] Updated weights for policy 1, policy_version 1240025 (0.0010) [2023-12-27 14:14:50,918][20347] Updated weights for policy 0, policy_version 1241083 (0.0006) [2023-12-27 14:14:50,930][20381] Updated weights for policy 1, policy_version 1240035 (0.0011) [2023-12-27 14:14:50,987][20381] Updated weights for policy 1, policy_version 1240045 (0.0008) [2023-12-27 14:14:51,045][20381] Updated weights for policy 1, policy_version 1240055 (0.0010) [2023-12-27 14:14:51,590][20347] Updated weights for policy 0, policy_version 1241093 (0.0007) [2023-12-27 14:14:51,657][20347] Updated weights for policy 0, policy_version 1241103 (0.0007) [2023-12-27 14:14:51,716][20347] Updated weights for policy 0, policy_version 1241113 (0.0009) [2023-12-27 14:14:51,749][20381] Updated weights for policy 1, policy_version 1240065 (0.0008) [2023-12-27 14:14:51,818][20381] Updated weights for policy 1, policy_version 1240075 (0.0006) [2023-12-27 14:14:51,881][20381] Updated weights for policy 1, policy_version 1240085 (0.0006) [2023-12-27 14:14:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 635273216. Throughput: 0: 10101.8, 1: 10364.4. Samples: 635246712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 14:14:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001240088_317489152.pth... [2023-12-27 14:14:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001238872_317177856.pth [2023-12-27 14:14:52,397][20347] Updated weights for policy 0, policy_version 1241123 (0.0007) [2023-12-27 14:14:52,457][20347] Updated weights for policy 0, policy_version 1241133 (0.0007) [2023-12-27 14:14:52,518][20347] Updated weights for policy 0, policy_version 1241143 (0.0006) [2023-12-27 14:14:52,531][20381] Updated weights for policy 1, policy_version 1240095 (0.0005) [2023-12-27 14:14:52,561][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001241152_317792256.pth... [2023-12-27 14:14:52,564][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001239936_317480960.pth [2023-12-27 14:14:52,591][20381] Updated weights for policy 1, policy_version 1240105 (0.0008) [2023-12-27 14:14:52,650][20381] Updated weights for policy 1, policy_version 1240115 (0.0007) [2023-12-27 14:14:53,141][20347] Updated weights for policy 0, policy_version 1241153 (0.0006) [2023-12-27 14:14:53,189][20347] Updated weights for policy 0, policy_version 1241163 (0.0005) [2023-12-27 14:14:53,245][20347] Updated weights for policy 0, policy_version 1241173 (0.0005) [2023-12-27 14:14:53,255][20381] Updated weights for policy 1, policy_version 1240125 (0.0008) [2023-12-27 14:14:53,303][20347] Updated weights for policy 0, policy_version 1241183 (0.0006) [2023-12-27 14:14:53,305][20381] Updated weights for policy 1, policy_version 1240135 (0.0006) [2023-12-27 14:14:53,357][20381] Updated weights for policy 1, policy_version 1240145 (0.0006) [2023-12-27 14:14:53,905][20347] Updated weights for policy 0, policy_version 1241193 (0.0007) [2023-12-27 14:14:53,967][20347] Updated weights for policy 0, policy_version 1241203 (0.0008) [2023-12-27 14:14:54,027][20347] Updated weights for policy 0, policy_version 1241213 (0.0007) [2023-12-27 14:14:54,061][20381] Updated weights for policy 1, policy_version 1240155 (0.0006) [2023-12-27 14:14:54,108][20381] Updated weights for policy 1, policy_version 1240165 (0.0007) [2023-12-27 14:14:54,165][20381] Updated weights for policy 1, policy_version 1240175 (0.0008) [2023-12-27 14:14:54,694][20347] Updated weights for policy 0, policy_version 1241223 (0.0008) [2023-12-27 14:14:54,748][20347] Updated weights for policy 0, policy_version 1241233 (0.0007) [2023-12-27 14:14:54,812][20347] Updated weights for policy 0, policy_version 1241243 (0.0007) [2023-12-27 14:14:54,814][20381] Updated weights for policy 1, policy_version 1240185 (0.0008) [2023-12-27 14:14:54,872][20381] Updated weights for policy 1, policy_version 1240195 (0.0006) [2023-12-27 14:14:54,929][20381] Updated weights for policy 1, policy_version 1240205 (0.0005) [2023-12-27 14:14:54,989][20381] Updated weights for policy 1, policy_version 1240215 (0.0007) [2023-12-27 14:14:55,488][20347] Updated weights for policy 0, policy_version 1241253 (0.0007) [2023-12-27 14:14:55,538][20347] Updated weights for policy 0, policy_version 1241263 (0.0005) [2023-12-27 14:14:55,580][20381] Updated weights for policy 1, policy_version 1240225 (0.0007) [2023-12-27 14:14:55,589][20347] Updated weights for policy 0, policy_version 1241273 (0.0005) [2023-12-27 14:14:55,628][20381] Updated weights for policy 1, policy_version 1240235 (0.0005) [2023-12-27 14:14:55,678][20381] Updated weights for policy 1, policy_version 1240245 (0.0008) [2023-12-27 14:14:56,232][20347] Updated weights for policy 0, policy_version 1241283 (0.0006) [2023-12-27 14:14:56,286][20347] Updated weights for policy 0, policy_version 1241293 (0.0007) [2023-12-27 14:14:56,333][20347] Updated weights for policy 0, policy_version 1241303 (0.0007) [2023-12-27 14:14:56,390][20381] Updated weights for policy 1, policy_version 1240255 (0.0010) [2023-12-27 14:14:56,445][20381] Updated weights for policy 1, policy_version 1240265 (0.0010) [2023-12-27 14:14:56,500][20381] Updated weights for policy 1, policy_version 1240275 (0.0010) [2023-12-27 14:14:57,019][20347] Updated weights for policy 0, policy_version 1241313 (0.0006) [2023-12-27 14:14:57,070][20347] Updated weights for policy 0, policy_version 1241323 (0.0010) [2023-12-27 14:14:57,125][20347] Updated weights for policy 0, policy_version 1241333 (0.0010) [2023-12-27 14:14:57,183][20381] Updated weights for policy 1, policy_version 1240285 (0.0010) [2023-12-27 14:14:57,183][20347] Updated weights for policy 0, policy_version 1241343 (0.0010) [2023-12-27 14:14:57,232][20381] Updated weights for policy 1, policy_version 1240295 (0.0010) [2023-12-27 14:14:57,282][20381] Updated weights for policy 1, policy_version 1240305 (0.0008) [2023-12-27 14:14:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 635387904. Throughput: 0: 10128.9, 1: 10345.5. Samples: 635372416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:14:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 14:14:57,853][20347] Updated weights for policy 0, policy_version 1241353 (0.0010) [2023-12-27 14:14:57,886][20381] Updated weights for policy 1, policy_version 1240315 (0.0008) [2023-12-27 14:14:57,907][20347] Updated weights for policy 0, policy_version 1241363 (0.0010) [2023-12-27 14:14:57,936][20381] Updated weights for policy 1, policy_version 1240325 (0.0005) [2023-12-27 14:14:57,958][20347] Updated weights for policy 0, policy_version 1241373 (0.0010) [2023-12-27 14:14:57,987][20381] Updated weights for policy 1, policy_version 1240335 (0.0005) [2023-12-27 14:14:58,668][20381] Updated weights for policy 1, policy_version 1240345 (0.0005) [2023-12-27 14:14:58,683][20347] Updated weights for policy 0, policy_version 1241383 (0.0010) [2023-12-27 14:14:58,727][20381] Updated weights for policy 1, policy_version 1240355 (0.0008) [2023-12-27 14:14:58,749][20347] Updated weights for policy 0, policy_version 1241393 (0.0010) [2023-12-27 14:14:58,791][20381] Updated weights for policy 1, policy_version 1240365 (0.0009) [2023-12-27 14:14:58,815][20347] Updated weights for policy 0, policy_version 1241403 (0.0009) [2023-12-27 14:14:58,850][20381] Updated weights for policy 1, policy_version 1240375 (0.0008) [2023-12-27 14:14:59,560][20347] Updated weights for policy 0, policy_version 1241413 (0.0007) [2023-12-27 14:14:59,623][20347] Updated weights for policy 0, policy_version 1241423 (0.0005) [2023-12-27 14:14:59,682][20347] Updated weights for policy 0, policy_version 1241433 (0.0007) [2023-12-27 14:14:59,719][20381] Updated weights for policy 1, policy_version 1240385 (0.0009) [2023-12-27 14:14:59,776][20381] Updated weights for policy 1, policy_version 1240395 (0.0010) [2023-12-27 14:14:59,836][20381] Updated weights for policy 1, policy_version 1240405 (0.0010) [2023-12-27 14:15:00,347][20347] Updated weights for policy 0, policy_version 1241443 (0.0006) [2023-12-27 14:15:00,403][20347] Updated weights for policy 0, policy_version 1241453 (0.0008) [2023-12-27 14:15:00,444][20381] Updated weights for policy 1, policy_version 1240415 (0.0007) [2023-12-27 14:15:00,465][20347] Updated weights for policy 0, policy_version 1241463 (0.0007) [2023-12-27 14:15:00,501][20381] Updated weights for policy 1, policy_version 1240425 (0.0006) [2023-12-27 14:15:00,557][20381] Updated weights for policy 1, policy_version 1240435 (0.0006) [2023-12-27 14:15:01,109][20347] Updated weights for policy 0, policy_version 1241473 (0.0006) [2023-12-27 14:15:01,168][20347] Updated weights for policy 0, policy_version 1241483 (0.0008) [2023-12-27 14:15:01,224][20347] Updated weights for policy 0, policy_version 1241493 (0.0007) [2023-12-27 14:15:01,243][20381] Updated weights for policy 1, policy_version 1240445 (0.0007) [2023-12-27 14:15:01,288][20347] Updated weights for policy 0, policy_version 1241503 (0.0007) [2023-12-27 14:15:01,296][20381] Updated weights for policy 1, policy_version 1240455 (0.0009) [2023-12-27 14:15:01,359][20381] Updated weights for policy 1, policy_version 1240465 (0.0010) [2023-12-27 14:15:01,995][20347] Updated weights for policy 0, policy_version 1241513 (0.0007) [2023-12-27 14:15:02,054][20347] Updated weights for policy 0, policy_version 1241523 (0.0007) [2023-12-27 14:15:02,079][20381] Updated weights for policy 1, policy_version 1240475 (0.0006) [2023-12-27 14:15:02,113][20347] Updated weights for policy 0, policy_version 1241533 (0.0006) [2023-12-27 14:15:02,137][20381] Updated weights for policy 1, policy_version 1240485 (0.0005) [2023-12-27 14:15:02,138][20292] KL-divergence is very high: 251.2618 [2023-12-27 14:15:02,194][20381] Updated weights for policy 1, policy_version 1240495 (0.0007) [2023-12-27 14:15:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 635486208. Throughput: 0: 10157.0, 1: 10345.2. Samples: 635493196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.230')] [2023-12-27 14:15:02,804][20347] Updated weights for policy 0, policy_version 1241543 (0.0006) [2023-12-27 14:15:02,808][20381] Updated weights for policy 1, policy_version 1240505 (0.0005) [2023-12-27 14:15:02,858][20381] Updated weights for policy 1, policy_version 1240515 (0.0005) [2023-12-27 14:15:02,866][20347] Updated weights for policy 0, policy_version 1241553 (0.0005) [2023-12-27 14:15:02,908][20381] Updated weights for policy 1, policy_version 1240525 (0.0007) [2023-12-27 14:15:02,924][20347] Updated weights for policy 0, policy_version 1241563 (0.0005) [2023-12-27 14:15:02,957][20381] Updated weights for policy 1, policy_version 1240535 (0.0008) [2023-12-27 14:15:03,545][20347] Updated weights for policy 0, policy_version 1241573 (0.0007) [2023-12-27 14:15:03,557][20381] Updated weights for policy 1, policy_version 1240545 (0.0006) [2023-12-27 14:15:03,602][20347] Updated weights for policy 0, policy_version 1241583 (0.0009) [2023-12-27 14:15:03,611][20381] Updated weights for policy 1, policy_version 1240555 (0.0005) [2023-12-27 14:15:03,657][20347] Updated weights for policy 0, policy_version 1241593 (0.0010) [2023-12-27 14:15:03,671][20381] Updated weights for policy 1, policy_version 1240565 (0.0005) [2023-12-27 14:15:04,268][20381] Updated weights for policy 1, policy_version 1240575 (0.0007) [2023-12-27 14:15:04,330][20381] Updated weights for policy 1, policy_version 1240585 (0.0007) [2023-12-27 14:15:04,375][20347] Updated weights for policy 0, policy_version 1241603 (0.0010) [2023-12-27 14:15:04,389][20381] Updated weights for policy 1, policy_version 1240595 (0.0007) [2023-12-27 14:15:04,433][20347] Updated weights for policy 0, policy_version 1241613 (0.0007) [2023-12-27 14:15:04,515][20347] Updated weights for policy 0, policy_version 1241623 (0.0007) [2023-12-27 14:15:05,060][20381] Updated weights for policy 1, policy_version 1240605 (0.0008) [2023-12-27 14:15:05,118][20381] Updated weights for policy 1, policy_version 1240615 (0.0006) [2023-12-27 14:15:05,142][20347] Updated weights for policy 0, policy_version 1241633 (0.0008) [2023-12-27 14:15:05,169][20381] Updated weights for policy 1, policy_version 1240625 (0.0005) [2023-12-27 14:15:05,189][20347] Updated weights for policy 0, policy_version 1241643 (0.0007) [2023-12-27 14:15:05,239][20347] Updated weights for policy 0, policy_version 1241653 (0.0008) [2023-12-27 14:15:05,286][20347] Updated weights for policy 0, policy_version 1241663 (0.0007) [2023-12-27 14:15:05,818][20381] Updated weights for policy 1, policy_version 1240635 (0.0007) [2023-12-27 14:15:05,885][20381] Updated weights for policy 1, policy_version 1240645 (0.0011) [2023-12-27 14:15:05,938][20347] Updated weights for policy 0, policy_version 1241673 (0.0010) [2023-12-27 14:15:05,940][20381] Updated weights for policy 1, policy_version 1240655 (0.0010) [2023-12-27 14:15:05,994][20347] Updated weights for policy 0, policy_version 1241683 (0.0010) [2023-12-27 14:15:06,048][20347] Updated weights for policy 0, policy_version 1241693 (0.0010) [2023-12-27 14:15:06,651][20381] Updated weights for policy 1, policy_version 1240665 (0.0010) [2023-12-27 14:15:06,701][20347] Updated weights for policy 0, policy_version 1241703 (0.0010) [2023-12-27 14:15:06,710][20381] Updated weights for policy 1, policy_version 1240675 (0.0006) [2023-12-27 14:15:06,753][20347] Updated weights for policy 0, policy_version 1241713 (0.0010) [2023-12-27 14:15:06,766][20381] Updated weights for policy 1, policy_version 1240685 (0.0006) [2023-12-27 14:15:06,799][20347] Updated weights for policy 0, policy_version 1241723 (0.0010) [2023-12-27 14:15:06,822][20381] Updated weights for policy 1, policy_version 1240695 (0.0006) [2023-12-27 14:15:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 635584512. Throughput: 0: 10180.8, 1: 10412.1. Samples: 635557156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.230')] [2023-12-27 14:15:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001240696_317644800.pth... [2023-12-27 14:15:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001241728_317939712.pth... [2023-12-27 14:15:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001240544_317636608.pth [2023-12-27 14:15:07,388][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001239480_317333504.pth [2023-12-27 14:15:07,464][20381] Updated weights for policy 1, policy_version 1240705 (0.0009) [2023-12-27 14:15:07,477][20347] Updated weights for policy 0, policy_version 1241733 (0.0008) [2023-12-27 14:15:07,526][20381] Updated weights for policy 1, policy_version 1240715 (0.0008) [2023-12-27 14:15:07,529][20347] Updated weights for policy 0, policy_version 1241743 (0.0005) [2023-12-27 14:15:07,586][20347] Updated weights for policy 0, policy_version 1241753 (0.0007) [2023-12-27 14:15:07,588][20381] Updated weights for policy 1, policy_version 1240725 (0.0007) [2023-12-27 14:15:08,204][20347] Updated weights for policy 0, policy_version 1241763 (0.0007) [2023-12-27 14:15:08,259][20347] Updated weights for policy 0, policy_version 1241773 (0.0008) [2023-12-27 14:15:08,287][20381] Updated weights for policy 1, policy_version 1240735 (0.0006) [2023-12-27 14:15:08,314][20347] Updated weights for policy 0, policy_version 1241783 (0.0007) [2023-12-27 14:15:08,366][20381] Updated weights for policy 1, policy_version 1240749 (0.0006) [2023-12-27 14:15:08,419][20381] Updated weights for policy 1, policy_version 1240759 (0.0008) [2023-12-27 14:15:08,919][20347] Updated weights for policy 0, policy_version 1241793 (0.0007) [2023-12-27 14:15:08,979][20347] Updated weights for policy 0, policy_version 1241803 (0.0005) [2023-12-27 14:15:09,035][20347] Updated weights for policy 0, policy_version 1241813 (0.0005) [2023-12-27 14:15:09,088][20347] Updated weights for policy 0, policy_version 1241823 (0.0006) [2023-12-27 14:15:09,130][20381] Updated weights for policy 1, policy_version 1240769 (0.0006) [2023-12-27 14:15:09,181][20381] Updated weights for policy 1, policy_version 1240779 (0.0005) [2023-12-27 14:15:09,257][20381] Updated weights for policy 1, policy_version 1240789 (0.0007) [2023-12-27 14:15:09,750][20347] Updated weights for policy 0, policy_version 1241833 (0.0008) [2023-12-27 14:15:09,812][20347] Updated weights for policy 0, policy_version 1241843 (0.0008) [2023-12-27 14:15:09,881][20347] Updated weights for policy 0, policy_version 1241853 (0.0008) [2023-12-27 14:15:09,956][20381] Updated weights for policy 1, policy_version 1240799 (0.0007) [2023-12-27 14:15:10,018][20381] Updated weights for policy 1, policy_version 1240809 (0.0008) [2023-12-27 14:15:10,074][20381] Updated weights for policy 1, policy_version 1240819 (0.0009) [2023-12-27 14:15:10,605][20347] Updated weights for policy 0, policy_version 1241863 (0.0009) [2023-12-27 14:15:10,663][20347] Updated weights for policy 0, policy_version 1241873 (0.0010) [2023-12-27 14:15:10,699][20381] Updated weights for policy 1, policy_version 1240829 (0.0006) [2023-12-27 14:15:10,715][20347] Updated weights for policy 0, policy_version 1241883 (0.0010) [2023-12-27 14:15:10,749][20381] Updated weights for policy 1, policy_version 1240839 (0.0010) [2023-12-27 14:15:10,804][20381] Updated weights for policy 1, policy_version 1240849 (0.0010) [2023-12-27 14:15:11,432][20347] Updated weights for policy 0, policy_version 1241893 (0.0007) [2023-12-27 14:15:11,483][20381] Updated weights for policy 1, policy_version 1240859 (0.0007) [2023-12-27 14:15:11,497][20347] Updated weights for policy 0, policy_version 1241903 (0.0010) [2023-12-27 14:15:11,540][20381] Updated weights for policy 1, policy_version 1240869 (0.0008) [2023-12-27 14:15:11,553][20347] Updated weights for policy 0, policy_version 1241913 (0.0010) [2023-12-27 14:15:11,603][20381] Updated weights for policy 1, policy_version 1240879 (0.0008) [2023-12-27 14:15:12,238][20347] Updated weights for policy 0, policy_version 1241923 (0.0010) [2023-12-27 14:15:12,306][20347] Updated weights for policy 0, policy_version 1241933 (0.0008) [2023-12-27 14:15:12,345][20381] Updated weights for policy 1, policy_version 1240889 (0.0008) [2023-12-27 14:15:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 635682816. Throughput: 0: 10252.7, 1: 10432.1. Samples: 635681536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.450')] [2023-12-27 14:15:12,369][20347] Updated weights for policy 0, policy_version 1241943 (0.0008) [2023-12-27 14:15:12,416][20381] Updated weights for policy 1, policy_version 1240899 (0.0009) [2023-12-27 14:15:12,479][20381] Updated weights for policy 1, policy_version 1240909 (0.0010) [2023-12-27 14:15:12,538][20381] Updated weights for policy 1, policy_version 1240919 (0.0008) [2023-12-27 14:15:13,053][20347] Updated weights for policy 0, policy_version 1241953 (0.0007) [2023-12-27 14:15:13,100][20347] Updated weights for policy 0, policy_version 1241963 (0.0007) [2023-12-27 14:15:13,148][20347] Updated weights for policy 0, policy_version 1241973 (0.0005) [2023-12-27 14:15:13,198][20347] Updated weights for policy 0, policy_version 1241983 (0.0006) [2023-12-27 14:15:13,203][20381] Updated weights for policy 1, policy_version 1240929 (0.0007) [2023-12-27 14:15:13,258][20381] Updated weights for policy 1, policy_version 1240939 (0.0008) [2023-12-27 14:15:13,325][20381] Updated weights for policy 1, policy_version 1240949 (0.0008) [2023-12-27 14:15:13,860][20347] Updated weights for policy 0, policy_version 1241993 (0.0008) [2023-12-27 14:15:13,919][20347] Updated weights for policy 0, policy_version 1242003 (0.0005) [2023-12-27 14:15:13,973][20347] Updated weights for policy 0, policy_version 1242013 (0.0005) [2023-12-27 14:15:14,024][20381] Updated weights for policy 1, policy_version 1240959 (0.0008) [2023-12-27 14:15:14,079][20381] Updated weights for policy 1, policy_version 1240969 (0.0007) [2023-12-27 14:15:14,125][20292] KL-divergence is very high: 152.0167 [2023-12-27 14:15:14,144][20381] Updated weights for policy 1, policy_version 1240979 (0.0007) [2023-12-27 14:15:14,542][20347] Updated weights for policy 0, policy_version 1242023 (0.0007) [2023-12-27 14:15:14,591][20347] Updated weights for policy 0, policy_version 1242033 (0.0008) [2023-12-27 14:15:14,642][20347] Updated weights for policy 0, policy_version 1242043 (0.0007) [2023-12-27 14:15:14,861][20381] Updated weights for policy 1, policy_version 1240989 (0.0007) [2023-12-27 14:15:14,929][20381] Updated weights for policy 1, policy_version 1240999 (0.0006) [2023-12-27 14:15:14,995][20381] Updated weights for policy 1, policy_version 1241009 (0.0011) [2023-12-27 14:15:15,389][20347] Updated weights for policy 0, policy_version 1242053 (0.0008) [2023-12-27 14:15:15,442][20347] Updated weights for policy 0, policy_version 1242063 (0.0005) [2023-12-27 14:15:15,498][20347] Updated weights for policy 0, policy_version 1242073 (0.0005) [2023-12-27 14:15:15,562][20381] Updated weights for policy 1, policy_version 1241019 (0.0009) [2023-12-27 14:15:15,627][20381] Updated weights for policy 1, policy_version 1241029 (0.0007) [2023-12-27 14:15:15,683][20381] Updated weights for policy 1, policy_version 1241039 (0.0011) [2023-12-27 14:15:16,054][20347] Updated weights for policy 0, policy_version 1242083 (0.0005) [2023-12-27 14:15:16,108][20347] Updated weights for policy 0, policy_version 1242093 (0.0005) [2023-12-27 14:15:16,166][20347] Updated weights for policy 0, policy_version 1242103 (0.0005) [2023-12-27 14:15:16,252][20381] Updated weights for policy 1, policy_version 1241049 (0.0006) [2023-12-27 14:15:16,313][20381] Updated weights for policy 1, policy_version 1241059 (0.0005) [2023-12-27 14:15:16,371][20381] Updated weights for policy 1, policy_version 1241069 (0.0005) [2023-12-27 14:15:16,425][20381] Updated weights for policy 1, policy_version 1241079 (0.0005) [2023-12-27 14:15:16,697][20347] Updated weights for policy 0, policy_version 1242113 (0.0006) [2023-12-27 14:15:16,751][20347] Updated weights for policy 0, policy_version 1242123 (0.0007) [2023-12-27 14:15:16,802][20347] Updated weights for policy 0, policy_version 1242133 (0.0007) [2023-12-27 14:15:16,857][20347] Updated weights for policy 0, policy_version 1242143 (0.0010) [2023-12-27 14:15:17,004][20381] Updated weights for policy 1, policy_version 1241089 (0.0010) [2023-12-27 14:15:17,063][20381] Updated weights for policy 1, policy_version 1241099 (0.0010) [2023-12-27 14:15:17,115][20381] Updated weights for policy 1, policy_version 1241109 (0.0010) [2023-12-27 14:15:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 635797504. Throughput: 0: 10307.2, 1: 10393.1. Samples: 635807248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:17,363][19170] Avg episode reward: [(0, '980.490'), (1, '951.140')] [2023-12-27 14:15:17,463][20347] Updated weights for policy 0, policy_version 1242153 (0.0007) [2023-12-27 14:15:17,519][20347] Updated weights for policy 0, policy_version 1242163 (0.0008) [2023-12-27 14:15:17,584][20347] Updated weights for policy 0, policy_version 1242173 (0.0007) [2023-12-27 14:15:17,807][20381] Updated weights for policy 1, policy_version 1241119 (0.0008) [2023-12-27 14:15:17,856][20381] Updated weights for policy 1, policy_version 1241129 (0.0007) [2023-12-27 14:15:17,905][20381] Updated weights for policy 1, policy_version 1241139 (0.0007) [2023-12-27 14:15:18,269][20347] Updated weights for policy 0, policy_version 1242183 (0.0008) [2023-12-27 14:15:18,321][20347] Updated weights for policy 0, policy_version 1242193 (0.0007) [2023-12-27 14:15:18,385][20347] Updated weights for policy 0, policy_version 1242203 (0.0006) [2023-12-27 14:15:18,616][20381] Updated weights for policy 1, policy_version 1241149 (0.0009) [2023-12-27 14:15:18,669][20381] Updated weights for policy 1, policy_version 1241159 (0.0009) [2023-12-27 14:15:18,720][20381] Updated weights for policy 1, policy_version 1241169 (0.0007) [2023-12-27 14:15:19,015][20347] Updated weights for policy 0, policy_version 1242213 (0.0007) [2023-12-27 14:15:19,074][20347] Updated weights for policy 0, policy_version 1242223 (0.0008) [2023-12-27 14:15:19,136][20347] Updated weights for policy 0, policy_version 1242233 (0.0008) [2023-12-27 14:15:19,431][20381] Updated weights for policy 1, policy_version 1241179 (0.0008) [2023-12-27 14:15:19,480][20381] Updated weights for policy 1, policy_version 1241189 (0.0008) [2023-12-27 14:15:19,542][20381] Updated weights for policy 1, policy_version 1241199 (0.0007) [2023-12-27 14:15:19,891][20347] Updated weights for policy 0, policy_version 1242243 (0.0008) [2023-12-27 14:15:19,949][20347] Updated weights for policy 0, policy_version 1242253 (0.0009) [2023-12-27 14:15:20,003][20347] Updated weights for policy 0, policy_version 1242263 (0.0009) [2023-12-27 14:15:20,302][20381] Updated weights for policy 1, policy_version 1241209 (0.0008) [2023-12-27 14:15:20,359][20381] Updated weights for policy 1, policy_version 1241219 (0.0006) [2023-12-27 14:15:20,418][20381] Updated weights for policy 1, policy_version 1241229 (0.0008) [2023-12-27 14:15:20,475][20381] Updated weights for policy 1, policy_version 1241239 (0.0008) [2023-12-27 14:15:20,710][20347] Updated weights for policy 0, policy_version 1242273 (0.0008) [2023-12-27 14:15:20,769][20347] Updated weights for policy 0, policy_version 1242283 (0.0006) [2023-12-27 14:15:20,832][20347] Updated weights for policy 0, policy_version 1242293 (0.0006) [2023-12-27 14:15:20,889][20347] Updated weights for policy 0, policy_version 1242303 (0.0006) [2023-12-27 14:15:21,203][20381] Updated weights for policy 1, policy_version 1241249 (0.0006) [2023-12-27 14:15:21,266][20381] Updated weights for policy 1, policy_version 1241259 (0.0007) [2023-12-27 14:15:21,326][20381] Updated weights for policy 1, policy_version 1241269 (0.0008) [2023-12-27 14:15:21,515][20347] Updated weights for policy 0, policy_version 1242313 (0.0007) [2023-12-27 14:15:21,573][20347] Updated weights for policy 0, policy_version 1242323 (0.0007) [2023-12-27 14:15:21,633][20347] Updated weights for policy 0, policy_version 1242333 (0.0007) [2023-12-27 14:15:22,065][20381] Updated weights for policy 1, policy_version 1241279 (0.0008) [2023-12-27 14:15:22,129][20381] Updated weights for policy 1, policy_version 1241289 (0.0008) [2023-12-27 14:15:22,186][20381] Updated weights for policy 1, policy_version 1241299 (0.0008) [2023-12-27 14:15:22,312][20347] Updated weights for policy 0, policy_version 1242343 (0.0007) [2023-12-27 14:15:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 635895808. Throughput: 0: 10290.5, 1: 10342.2. Samples: 635867288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:22,363][19170] Avg episode reward: [(0, '980.490'), (1, '951.070')] [2023-12-27 14:15:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001241304_317800448.pth... [2023-12-27 14:15:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001240088_317489152.pth [2023-12-27 14:15:22,373][20347] Updated weights for policy 0, policy_version 1242353 (0.0009) [2023-12-27 14:15:22,431][20347] Updated weights for policy 0, policy_version 1242363 (0.0005) [2023-12-27 14:15:22,459][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001242368_318103552.pth... [2023-12-27 14:15:22,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001241152_317792256.pth [2023-12-27 14:15:22,908][20381] Updated weights for policy 1, policy_version 1241309 (0.0007) [2023-12-27 14:15:22,970][20381] Updated weights for policy 1, policy_version 1241319 (0.0008) [2023-12-27 14:15:23,034][20381] Updated weights for policy 1, policy_version 1241329 (0.0007) [2023-12-27 14:15:23,119][20347] Updated weights for policy 0, policy_version 1242373 (0.0007) [2023-12-27 14:15:23,178][20347] Updated weights for policy 0, policy_version 1242383 (0.0008) [2023-12-27 14:15:23,227][20347] Updated weights for policy 0, policy_version 1242393 (0.0008) [2023-12-27 14:15:23,610][20381] Updated weights for policy 1, policy_version 1241339 (0.0009) [2023-12-27 14:15:23,659][20381] Updated weights for policy 1, policy_version 1241349 (0.0010) [2023-12-27 14:15:23,718][20381] Updated weights for policy 1, policy_version 1241359 (0.0008) [2023-12-27 14:15:23,806][20347] Updated weights for policy 0, policy_version 1242403 (0.0008) [2023-12-27 14:15:23,854][20347] Updated weights for policy 0, policy_version 1242413 (0.0007) [2023-12-27 14:15:23,898][20347] Updated weights for policy 0, policy_version 1242423 (0.0009) [2023-12-27 14:15:24,360][20381] Updated weights for policy 1, policy_version 1241369 (0.0010) [2023-12-27 14:15:24,424][20381] Updated weights for policy 1, policy_version 1241379 (0.0010) [2023-12-27 14:15:24,486][20381] Updated weights for policy 1, policy_version 1241389 (0.0010) [2023-12-27 14:15:24,545][20381] Updated weights for policy 1, policy_version 1241399 (0.0010) [2023-12-27 14:15:24,622][20347] Updated weights for policy 0, policy_version 1242433 (0.0007) [2023-12-27 14:15:24,683][20347] Updated weights for policy 0, policy_version 1242443 (0.0006) [2023-12-27 14:15:24,739][20347] Updated weights for policy 0, policy_version 1242453 (0.0007) [2023-12-27 14:15:24,785][20347] Updated weights for policy 0, policy_version 1242463 (0.0007) [2023-12-27 14:15:25,219][20381] Updated weights for policy 1, policy_version 1241409 (0.0011) [2023-12-27 14:15:25,284][20381] Updated weights for policy 1, policy_version 1241419 (0.0011) [2023-12-27 14:15:25,342][20381] Updated weights for policy 1, policy_version 1241429 (0.0006) [2023-12-27 14:15:25,444][20347] Updated weights for policy 0, policy_version 1242473 (0.0006) [2023-12-27 14:15:25,501][20347] Updated weights for policy 0, policy_version 1242483 (0.0005) [2023-12-27 14:15:25,550][20347] Updated weights for policy 0, policy_version 1242493 (0.0006) [2023-12-27 14:15:26,005][20381] Updated weights for policy 1, policy_version 1241439 (0.0005) [2023-12-27 14:15:26,067][20381] Updated weights for policy 1, policy_version 1241449 (0.0008) [2023-12-27 14:15:26,097][20347] Updated weights for policy 0, policy_version 1242503 (0.0006) [2023-12-27 14:15:26,129][20381] Updated weights for policy 1, policy_version 1241459 (0.0005) [2023-12-27 14:15:26,144][20347] Updated weights for policy 0, policy_version 1242513 (0.0007) [2023-12-27 14:15:26,187][20347] Updated weights for policy 0, policy_version 1242523 (0.0007) [2023-12-27 14:15:26,680][20381] Updated weights for policy 1, policy_version 1241469 (0.0007) [2023-12-27 14:15:26,727][20381] Updated weights for policy 1, policy_version 1241479 (0.0007) [2023-12-27 14:15:26,782][20381] Updated weights for policy 1, policy_version 1241489 (0.0010) [2023-12-27 14:15:26,796][20347] Updated weights for policy 0, policy_version 1242533 (0.0005) [2023-12-27 14:15:26,850][20347] Updated weights for policy 0, policy_version 1242543 (0.0005) [2023-12-27 14:15:26,892][20347] Updated weights for policy 0, policy_version 1242553 (0.0005) [2023-12-27 14:15:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 636002304. Throughput: 0: 10366.0, 1: 10287.4. Samples: 635991632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:27,363][19170] Avg episode reward: [(0, '990.260'), (1, '941.300')] [2023-12-27 14:15:27,477][20381] Updated weights for policy 1, policy_version 1241499 (0.0011) [2023-12-27 14:15:27,498][20347] Updated weights for policy 0, policy_version 1242563 (0.0007) [2023-12-27 14:15:27,533][20381] Updated weights for policy 1, policy_version 1241509 (0.0011) [2023-12-27 14:15:27,552][20347] Updated weights for policy 0, policy_version 1242573 (0.0010) [2023-12-27 14:15:27,594][20381] Updated weights for policy 1, policy_version 1241519 (0.0011) [2023-12-27 14:15:27,605][20347] Updated weights for policy 0, policy_version 1242583 (0.0009) [2023-12-27 14:15:28,162][20347] Updated weights for policy 0, policy_version 1242593 (0.0006) [2023-12-27 14:15:28,225][20347] Updated weights for policy 0, policy_version 1242603 (0.0009) [2023-12-27 14:15:28,286][20381] Updated weights for policy 1, policy_version 1241529 (0.0011) [2023-12-27 14:15:28,287][20347] Updated weights for policy 0, policy_version 1242613 (0.0010) [2023-12-27 14:15:28,345][20381] Updated weights for policy 1, policy_version 1241539 (0.0009) [2023-12-27 14:15:28,357][20347] Updated weights for policy 0, policy_version 1242623 (0.0011) [2023-12-27 14:15:28,399][20381] Updated weights for policy 1, policy_version 1241549 (0.0007) [2023-12-27 14:15:28,455][20381] Updated weights for policy 1, policy_version 1241559 (0.0009) [2023-12-27 14:15:29,011][20347] Updated weights for policy 0, policy_version 1242633 (0.0007) [2023-12-27 14:15:29,067][20347] Updated weights for policy 0, policy_version 1242643 (0.0010) [2023-12-27 14:15:29,110][20381] Updated weights for policy 1, policy_version 1241569 (0.0011) [2023-12-27 14:15:29,130][20347] Updated weights for policy 0, policy_version 1242653 (0.0010) [2023-12-27 14:15:29,163][20381] Updated weights for policy 1, policy_version 1241579 (0.0011) [2023-12-27 14:15:29,220][20381] Updated weights for policy 1, policy_version 1241589 (0.0008) [2023-12-27 14:15:29,795][20347] Updated weights for policy 0, policy_version 1242663 (0.0007) [2023-12-27 14:15:29,856][20347] Updated weights for policy 0, policy_version 1242673 (0.0008) [2023-12-27 14:15:29,899][20381] Updated weights for policy 1, policy_version 1241599 (0.0008) [2023-12-27 14:15:29,918][20347] Updated weights for policy 0, policy_version 1242683 (0.0008) [2023-12-27 14:15:29,961][20381] Updated weights for policy 1, policy_version 1241609 (0.0009) [2023-12-27 14:15:30,019][20381] Updated weights for policy 1, policy_version 1241619 (0.0008) [2023-12-27 14:15:30,580][20347] Updated weights for policy 0, policy_version 1242693 (0.0010) [2023-12-27 14:15:30,641][20347] Updated weights for policy 0, policy_version 1242703 (0.0010) [2023-12-27 14:15:30,695][20347] Updated weights for policy 0, policy_version 1242713 (0.0010) [2023-12-27 14:15:30,701][20381] Updated weights for policy 1, policy_version 1241629 (0.0007) [2023-12-27 14:15:30,756][20381] Updated weights for policy 1, policy_version 1241639 (0.0009) [2023-12-27 14:15:30,804][20381] Updated weights for policy 1, policy_version 1241649 (0.0010) [2023-12-27 14:15:31,420][20347] Updated weights for policy 0, policy_version 1242723 (0.0009) [2023-12-27 14:15:31,476][20347] Updated weights for policy 0, policy_version 1242733 (0.0006) [2023-12-27 14:15:31,539][20347] Updated weights for policy 0, policy_version 1242743 (0.0007) [2023-12-27 14:15:31,554][20381] Updated weights for policy 1, policy_version 1241659 (0.0009) [2023-12-27 14:15:31,617][20381] Updated weights for policy 1, policy_version 1241669 (0.0008) [2023-12-27 14:15:31,674][20381] Updated weights for policy 1, policy_version 1241679 (0.0009) [2023-12-27 14:15:32,306][20347] Updated weights for policy 0, policy_version 1242753 (0.0007) [2023-12-27 14:15:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 636100608. Throughput: 0: 10436.6, 1: 10239.2. Samples: 636115156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.300')] [2023-12-27 14:15:32,370][20347] Updated weights for policy 0, policy_version 1242763 (0.0009) [2023-12-27 14:15:32,430][20347] Updated weights for policy 0, policy_version 1242773 (0.0007) [2023-12-27 14:15:32,491][20381] Updated weights for policy 1, policy_version 1241689 (0.0010) [2023-12-27 14:15:32,492][20347] Updated weights for policy 0, policy_version 1242783 (0.0006) [2023-12-27 14:15:32,550][20381] Updated weights for policy 1, policy_version 1241699 (0.0006) [2023-12-27 14:15:32,623][20381] Updated weights for policy 1, policy_version 1241709 (0.0007) [2023-12-27 14:15:32,686][20381] Updated weights for policy 1, policy_version 1241719 (0.0009) [2023-12-27 14:15:33,200][20347] Updated weights for policy 0, policy_version 1242793 (0.0006) [2023-12-27 14:15:33,258][20347] Updated weights for policy 0, policy_version 1242803 (0.0006) [2023-12-27 14:15:33,280][20381] Updated weights for policy 1, policy_version 1241729 (0.0007) [2023-12-27 14:15:33,316][20347] Updated weights for policy 0, policy_version 1242813 (0.0007) [2023-12-27 14:15:33,337][20381] Updated weights for policy 1, policy_version 1241739 (0.0005) [2023-12-27 14:15:33,396][20381] Updated weights for policy 1, policy_version 1241749 (0.0007) [2023-12-27 14:15:33,941][20381] Updated weights for policy 1, policy_version 1241759 (0.0008) [2023-12-27 14:15:34,005][20381] Updated weights for policy 1, policy_version 1241769 (0.0006) [2023-12-27 14:15:34,018][20347] Updated weights for policy 0, policy_version 1242823 (0.0006) [2023-12-27 14:15:34,059][20381] Updated weights for policy 1, policy_version 1241779 (0.0010) [2023-12-27 14:15:34,068][20347] Updated weights for policy 0, policy_version 1242833 (0.0007) [2023-12-27 14:15:34,121][20347] Updated weights for policy 0, policy_version 1242843 (0.0007) [2023-12-27 14:15:34,772][20381] Updated weights for policy 1, policy_version 1241789 (0.0011) [2023-12-27 14:15:34,833][20381] Updated weights for policy 1, policy_version 1241799 (0.0011) [2023-12-27 14:15:34,855][20347] Updated weights for policy 0, policy_version 1242853 (0.0008) [2023-12-27 14:15:34,893][20381] Updated weights for policy 1, policy_version 1241809 (0.0009) [2023-12-27 14:15:34,915][20347] Updated weights for policy 0, policy_version 1242863 (0.0007) [2023-12-27 14:15:34,973][20347] Updated weights for policy 0, policy_version 1242873 (0.0008) [2023-12-27 14:15:35,589][20381] Updated weights for policy 1, policy_version 1241819 (0.0008) [2023-12-27 14:15:35,648][20381] Updated weights for policy 1, policy_version 1241829 (0.0008) [2023-12-27 14:15:35,678][20347] Updated weights for policy 0, policy_version 1242883 (0.0008) [2023-12-27 14:15:35,699][20381] Updated weights for policy 1, policy_version 1241839 (0.0006) [2023-12-27 14:15:35,731][20347] Updated weights for policy 0, policy_version 1242893 (0.0007) [2023-12-27 14:15:35,787][20347] Updated weights for policy 0, policy_version 1242903 (0.0007) [2023-12-27 14:15:36,486][20381] Updated weights for policy 1, policy_version 1241849 (0.0006) [2023-12-27 14:15:36,542][20381] Updated weights for policy 1, policy_version 1241859 (0.0007) [2023-12-27 14:15:36,544][20347] Updated weights for policy 0, policy_version 1242913 (0.0008) [2023-12-27 14:15:36,598][20347] Updated weights for policy 0, policy_version 1242923 (0.0008) [2023-12-27 14:15:36,603][20381] Updated weights for policy 1, policy_version 1241869 (0.0008) [2023-12-27 14:15:36,655][20347] Updated weights for policy 0, policy_version 1242933 (0.0011) [2023-12-27 14:15:36,664][20381] Updated weights for policy 1, policy_version 1241879 (0.0006) [2023-12-27 14:15:36,704][20347] Updated weights for policy 0, policy_version 1242943 (0.0010) [2023-12-27 14:15:37,302][20347] Updated weights for policy 0, policy_version 1242953 (0.0011) [2023-12-27 14:15:37,337][20381] Updated weights for policy 1, policy_version 1241889 (0.0010) [2023-12-27 14:15:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 636198912. Throughput: 0: 10381.0, 1: 10258.4. Samples: 636175484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:37,363][20347] Updated weights for policy 0, policy_version 1242963 (0.0010) [2023-12-27 14:15:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.320')] [2023-12-27 14:15:37,396][20381] Updated weights for policy 1, policy_version 1241899 (0.0010) [2023-12-27 14:15:37,419][20347] Updated weights for policy 0, policy_version 1242973 (0.0011) [2023-12-27 14:15:37,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001242976_318259200.pth... [2023-12-27 14:15:37,438][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001241728_317939712.pth [2023-12-27 14:15:37,451][20381] Updated weights for policy 1, policy_version 1241909 (0.0008) [2023-12-27 14:15:37,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001241912_317956096.pth... [2023-12-27 14:15:37,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001240696_317644800.pth [2023-12-27 14:15:38,041][20347] Updated weights for policy 0, policy_version 1242983 (0.0010) [2023-12-27 14:15:38,090][20347] Updated weights for policy 0, policy_version 1242993 (0.0010) [2023-12-27 14:15:38,142][20347] Updated weights for policy 0, policy_version 1243003 (0.0010) [2023-12-27 14:15:38,152][20381] Updated weights for policy 1, policy_version 1241919 (0.0010) [2023-12-27 14:15:38,205][20381] Updated weights for policy 1, policy_version 1241929 (0.0007) [2023-12-27 14:15:38,254][20381] Updated weights for policy 1, policy_version 1241939 (0.0005) [2023-12-27 14:15:38,843][20347] Updated weights for policy 0, policy_version 1243013 (0.0010) [2023-12-27 14:15:38,903][20347] Updated weights for policy 0, policy_version 1243023 (0.0011) [2023-12-27 14:15:38,936][20381] Updated weights for policy 1, policy_version 1241949 (0.0006) [2023-12-27 14:15:38,965][20347] Updated weights for policy 0, policy_version 1243033 (0.0010) [2023-12-27 14:15:38,992][20381] Updated weights for policy 1, policy_version 1241959 (0.0008) [2023-12-27 14:15:39,053][20381] Updated weights for policy 1, policy_version 1241969 (0.0009) [2023-12-27 14:15:39,634][20347] Updated weights for policy 0, policy_version 1243043 (0.0009) [2023-12-27 14:15:39,695][20347] Updated weights for policy 0, policy_version 1243053 (0.0006) [2023-12-27 14:15:39,709][20381] Updated weights for policy 1, policy_version 1241979 (0.0010) [2023-12-27 14:15:39,749][20347] Updated weights for policy 0, policy_version 1243063 (0.0006) [2023-12-27 14:15:39,769][20381] Updated weights for policy 1, policy_version 1241989 (0.0011) [2023-12-27 14:15:39,833][20381] Updated weights for policy 1, policy_version 1241999 (0.0010) [2023-12-27 14:15:40,420][20347] Updated weights for policy 0, policy_version 1243073 (0.0008) [2023-12-27 14:15:40,465][20381] Updated weights for policy 1, policy_version 1242009 (0.0007) [2023-12-27 14:15:40,482][20347] Updated weights for policy 0, policy_version 1243083 (0.0007) [2023-12-27 14:15:40,516][20381] Updated weights for policy 1, policy_version 1242019 (0.0007) [2023-12-27 14:15:40,545][20347] Updated weights for policy 0, policy_version 1243093 (0.0007) [2023-12-27 14:15:40,571][20381] Updated weights for policy 1, policy_version 1242029 (0.0006) [2023-12-27 14:15:40,604][20347] Updated weights for policy 0, policy_version 1243103 (0.0007) [2023-12-27 14:15:40,632][20381] Updated weights for policy 1, policy_version 1242039 (0.0007) [2023-12-27 14:15:41,298][20347] Updated weights for policy 0, policy_version 1243113 (0.0010) [2023-12-27 14:15:41,338][20381] Updated weights for policy 1, policy_version 1242049 (0.0006) [2023-12-27 14:15:41,359][20347] Updated weights for policy 0, policy_version 1243123 (0.0009) [2023-12-27 14:15:41,406][20381] Updated weights for policy 1, policy_version 1242059 (0.0007) [2023-12-27 14:15:41,421][20347] Updated weights for policy 0, policy_version 1243133 (0.0007) [2023-12-27 14:15:41,465][20381] Updated weights for policy 1, policy_version 1242069 (0.0008) [2023-12-27 14:15:42,158][20347] Updated weights for policy 0, policy_version 1243143 (0.0007) [2023-12-27 14:15:42,166][20291] KL-divergence is very high: 151.5448 [2023-12-27 14:15:42,186][20381] Updated weights for policy 1, policy_version 1242079 (0.0008) [2023-12-27 14:15:42,209][20291] KL-divergence is very high: 109.6248 [2023-12-27 14:15:42,215][20347] Updated weights for policy 0, policy_version 1243153 (0.0006) [2023-12-27 14:15:42,244][20381] Updated weights for policy 1, policy_version 1242089 (0.0007) [2023-12-27 14:15:42,272][20347] Updated weights for policy 0, policy_version 1243163 (0.0007) [2023-12-27 14:15:42,306][20381] Updated weights for policy 1, policy_version 1242099 (0.0007) [2023-12-27 14:15:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 636313600. Throughput: 0: 10359.7, 1: 10200.7. Samples: 636297636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:15:42,363][19170] Avg episode reward: [(0, '990.240'), (1, '931.490')] [2023-12-27 14:15:42,977][20347] Updated weights for policy 0, policy_version 1243173 (0.0007) [2023-12-27 14:15:42,993][20381] Updated weights for policy 1, policy_version 1242109 (0.0008) [2023-12-27 14:15:43,031][20347] Updated weights for policy 0, policy_version 1243183 (0.0005) [2023-12-27 14:15:43,049][20381] Updated weights for policy 1, policy_version 1242119 (0.0008) [2023-12-27 14:15:43,089][20347] Updated weights for policy 0, policy_version 1243193 (0.0006) [2023-12-27 14:15:43,108][20381] Updated weights for policy 1, policy_version 1242129 (0.0007) [2023-12-27 14:15:43,640][20347] Updated weights for policy 0, policy_version 1243203 (0.0006) [2023-12-27 14:15:43,689][20381] Updated weights for policy 1, policy_version 1242139 (0.0008) [2023-12-27 14:15:43,697][20347] Updated weights for policy 0, policy_version 1243213 (0.0007) [2023-12-27 14:15:43,738][20381] Updated weights for policy 1, policy_version 1242149 (0.0005) [2023-12-27 14:15:43,750][20347] Updated weights for policy 0, policy_version 1243223 (0.0007) [2023-12-27 14:15:43,790][20381] Updated weights for policy 1, policy_version 1242159 (0.0005) [2023-12-27 14:15:44,429][20347] Updated weights for policy 0, policy_version 1243233 (0.0006) [2023-12-27 14:15:44,471][20381] Updated weights for policy 1, policy_version 1242169 (0.0006) [2023-12-27 14:15:44,488][20347] Updated weights for policy 0, policy_version 1243243 (0.0006) [2023-12-27 14:15:44,527][20381] Updated weights for policy 1, policy_version 1242179 (0.0011) [2023-12-27 14:15:44,543][20347] Updated weights for policy 0, policy_version 1243253 (0.0006) [2023-12-27 14:15:44,586][20381] Updated weights for policy 1, policy_version 1242189 (0.0011) [2023-12-27 14:15:44,601][20347] Updated weights for policy 0, policy_version 1243263 (0.0006) [2023-12-27 14:15:44,642][20381] Updated weights for policy 1, policy_version 1242199 (0.0006) [2023-12-27 14:15:45,299][20347] Updated weights for policy 0, policy_version 1243273 (0.0008) [2023-12-27 14:15:45,361][20381] Updated weights for policy 1, policy_version 1242209 (0.0007) [2023-12-27 14:15:45,364][20347] Updated weights for policy 0, policy_version 1243283 (0.0008) [2023-12-27 14:15:45,418][20381] Updated weights for policy 1, policy_version 1242219 (0.0008) [2023-12-27 14:15:45,420][20347] Updated weights for policy 0, policy_version 1243293 (0.0006) [2023-12-27 14:15:45,476][20381] Updated weights for policy 1, policy_version 1242229 (0.0007) [2023-12-27 14:15:46,057][20347] Updated weights for policy 0, policy_version 1243303 (0.0006) [2023-12-27 14:15:46,116][20347] Updated weights for policy 0, policy_version 1243313 (0.0005) [2023-12-27 14:15:46,174][20347] Updated weights for policy 0, policy_version 1243323 (0.0005) [2023-12-27 14:15:46,200][20381] Updated weights for policy 1, policy_version 1242239 (0.0008) [2023-12-27 14:15:46,251][20381] Updated weights for policy 1, policy_version 1242249 (0.0007) [2023-12-27 14:15:46,309][20381] Updated weights for policy 1, policy_version 1242259 (0.0008) [2023-12-27 14:15:46,676][20347] Updated weights for policy 0, policy_version 1243333 (0.0007) [2023-12-27 14:15:46,726][20347] Updated weights for policy 0, policy_version 1243343 (0.0005) [2023-12-27 14:15:46,780][20347] Updated weights for policy 0, policy_version 1243353 (0.0006) [2023-12-27 14:15:47,074][20381] Updated weights for policy 1, policy_version 1242269 (0.0009) [2023-12-27 14:15:47,144][20381] Updated weights for policy 1, policy_version 1242279 (0.0008) [2023-12-27 14:15:47,211][20381] Updated weights for policy 1, policy_version 1242289 (0.0008) [2023-12-27 14:15:47,340][20347] Updated weights for policy 0, policy_version 1243363 (0.0007) [2023-12-27 14:15:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 636411904. Throughput: 0: 10449.2, 1: 10172.0. Samples: 636421152. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:15:47,363][19170] Avg episode reward: [(0, '990.240'), (1, '892.330')] [2023-12-27 14:15:47,403][20347] Updated weights for policy 0, policy_version 1243373 (0.0005) [2023-12-27 14:15:47,467][20347] Updated weights for policy 0, policy_version 1243383 (0.0005) [2023-12-27 14:15:47,916][20381] Updated weights for policy 1, policy_version 1242299 (0.0008) [2023-12-27 14:15:47,970][20381] Updated weights for policy 1, policy_version 1242309 (0.0009) [2023-12-27 14:15:48,028][20381] Updated weights for policy 1, policy_version 1242319 (0.0008) [2023-12-27 14:15:48,069][20347] Updated weights for policy 0, policy_version 1243393 (0.0008) [2023-12-27 14:15:48,127][20347] Updated weights for policy 0, policy_version 1243403 (0.0007) [2023-12-27 14:15:48,182][20347] Updated weights for policy 0, policy_version 1243413 (0.0008) [2023-12-27 14:15:48,232][20347] Updated weights for policy 0, policy_version 1243423 (0.0007) [2023-12-27 14:15:48,758][20381] Updated weights for policy 1, policy_version 1242329 (0.0008) [2023-12-27 14:15:48,820][20381] Updated weights for policy 1, policy_version 1242339 (0.0011) [2023-12-27 14:15:48,877][20381] Updated weights for policy 1, policy_version 1242349 (0.0011) [2023-12-27 14:15:48,935][20381] Updated weights for policy 1, policy_version 1242359 (0.0011) [2023-12-27 14:15:48,950][20347] Updated weights for policy 0, policy_version 1243433 (0.0007) [2023-12-27 14:15:49,005][20347] Updated weights for policy 0, policy_version 1243443 (0.0007) [2023-12-27 14:15:49,065][20347] Updated weights for policy 0, policy_version 1243453 (0.0007) [2023-12-27 14:15:49,656][20381] Updated weights for policy 1, policy_version 1242369 (0.0010) [2023-12-27 14:15:49,721][20381] Updated weights for policy 1, policy_version 1242379 (0.0008) [2023-12-27 14:15:49,778][20347] Updated weights for policy 0, policy_version 1243463 (0.0008) [2023-12-27 14:15:49,779][20381] Updated weights for policy 1, policy_version 1242389 (0.0008) [2023-12-27 14:15:49,841][20347] Updated weights for policy 0, policy_version 1243473 (0.0007) [2023-12-27 14:15:49,900][20347] Updated weights for policy 0, policy_version 1243483 (0.0007) [2023-12-27 14:15:50,483][20381] Updated weights for policy 1, policy_version 1242399 (0.0010) [2023-12-27 14:15:50,531][20347] Updated weights for policy 0, policy_version 1243493 (0.0008) [2023-12-27 14:15:50,544][20381] Updated weights for policy 1, policy_version 1242409 (0.0011) [2023-12-27 14:15:50,591][20347] Updated weights for policy 0, policy_version 1243503 (0.0007) [2023-12-27 14:15:50,602][20381] Updated weights for policy 1, policy_version 1242419 (0.0011) [2023-12-27 14:15:50,644][20347] Updated weights for policy 0, policy_version 1243513 (0.0006) [2023-12-27 14:15:51,281][20381] Updated weights for policy 1, policy_version 1242429 (0.0010) [2023-12-27 14:15:51,302][20292] KL-divergence is very high: 105.9116 [2023-12-27 14:15:51,349][20381] Updated weights for policy 1, policy_version 1242439 (0.0009) [2023-12-27 14:15:51,409][20381] Updated weights for policy 1, policy_version 1242449 (0.0008) [2023-12-27 14:15:51,434][20347] Updated weights for policy 0, policy_version 1243523 (0.0007) [2023-12-27 14:15:51,495][20347] Updated weights for policy 0, policy_version 1243533 (0.0007) [2023-12-27 14:15:51,555][20347] Updated weights for policy 0, policy_version 1243543 (0.0007) [2023-12-27 14:15:52,141][20381] Updated weights for policy 1, policy_version 1242459 (0.0008) [2023-12-27 14:15:52,191][20381] Updated weights for policy 1, policy_version 1242469 (0.0007) [2023-12-27 14:15:52,238][20381] Updated weights for policy 1, policy_version 1242479 (0.0008) [2023-12-27 14:15:52,270][20347] Updated weights for policy 0, policy_version 1243553 (0.0007) [2023-12-27 14:15:52,333][20347] Updated weights for policy 0, policy_version 1243563 (0.0007) [2023-12-27 14:15:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 636510208. Throughput: 0: 10455.5, 1: 10081.3. Samples: 636481316. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:15:52,363][19170] Avg episode reward: [(0, '990.270'), (1, '921.630')] [2023-12-27 14:15:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001242488_318103552.pth... [2023-12-27 14:15:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001241304_317800448.pth [2023-12-27 14:15:52,398][20347] Updated weights for policy 0, policy_version 1243573 (0.0008) [2023-12-27 14:15:52,450][20347] Updated weights for policy 0, policy_version 1243583 (0.0008) [2023-12-27 14:15:52,456][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001243584_318414848.pth... [2023-12-27 14:15:52,461][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001242368_318103552.pth [2023-12-27 14:15:52,935][20381] Updated weights for policy 1, policy_version 1242489 (0.0007) [2023-12-27 14:15:53,000][20381] Updated weights for policy 1, policy_version 1242499 (0.0008) [2023-12-27 14:15:53,064][20381] Updated weights for policy 1, policy_version 1242509 (0.0007) [2023-12-27 14:15:53,124][20381] Updated weights for policy 1, policy_version 1242519 (0.0006) [2023-12-27 14:15:53,141][20347] Updated weights for policy 0, policy_version 1243593 (0.0006) [2023-12-27 14:15:53,199][20347] Updated weights for policy 0, policy_version 1243603 (0.0007) [2023-12-27 14:15:53,259][20347] Updated weights for policy 0, policy_version 1243613 (0.0008) [2023-12-27 14:15:53,810][20381] Updated weights for policy 1, policy_version 1242529 (0.0005) [2023-12-27 14:15:53,862][20381] Updated weights for policy 1, policy_version 1242539 (0.0005) [2023-12-27 14:15:53,914][20381] Updated weights for policy 1, policy_version 1242549 (0.0005) [2023-12-27 14:15:53,918][20347] Updated weights for policy 0, policy_version 1243623 (0.0007) [2023-12-27 14:15:53,967][20347] Updated weights for policy 0, policy_version 1243633 (0.0009) [2023-12-27 14:15:54,021][20347] Updated weights for policy 0, policy_version 1243643 (0.0008) [2023-12-27 14:15:54,500][20381] Updated weights for policy 1, policy_version 1242559 (0.0007) [2023-12-27 14:15:54,549][20381] Updated weights for policy 1, policy_version 1242569 (0.0008) [2023-12-27 14:15:54,607][20381] Updated weights for policy 1, policy_version 1242579 (0.0008) [2023-12-27 14:15:54,725][20347] Updated weights for policy 0, policy_version 1243653 (0.0008) [2023-12-27 14:15:54,774][20347] Updated weights for policy 0, policy_version 1243663 (0.0007) [2023-12-27 14:15:54,829][20347] Updated weights for policy 0, policy_version 1243673 (0.0008) [2023-12-27 14:15:55,325][20381] Updated weights for policy 1, policy_version 1242589 (0.0007) [2023-12-27 14:15:55,370][20381] Updated weights for policy 1, policy_version 1242599 (0.0009) [2023-12-27 14:15:55,423][20381] Updated weights for policy 1, policy_version 1242609 (0.0009) [2023-12-27 14:15:55,437][20347] Updated weights for policy 0, policy_version 1243683 (0.0006) [2023-12-27 14:15:55,483][20347] Updated weights for policy 0, policy_version 1243693 (0.0005) [2023-12-27 14:15:55,544][20347] Updated weights for policy 0, policy_version 1243703 (0.0005) [2023-12-27 14:15:56,071][20347] Updated weights for policy 0, policy_version 1243713 (0.0005) [2023-12-27 14:15:56,130][20347] Updated weights for policy 0, policy_version 1243723 (0.0005) [2023-12-27 14:15:56,183][20347] Updated weights for policy 0, policy_version 1243733 (0.0005) [2023-12-27 14:15:56,190][20381] Updated weights for policy 1, policy_version 1242619 (0.0007) [2023-12-27 14:15:56,233][20347] Updated weights for policy 0, policy_version 1243743 (0.0005) [2023-12-27 14:15:56,249][20381] Updated weights for policy 1, policy_version 1242629 (0.0007) [2023-12-27 14:15:56,299][20381] Updated weights for policy 1, policy_version 1242639 (0.0005) [2023-12-27 14:15:56,750][20347] Updated weights for policy 0, policy_version 1243753 (0.0005) [2023-12-27 14:15:56,807][20347] Updated weights for policy 0, policy_version 1243763 (0.0005) [2023-12-27 14:15:56,855][20347] Updated weights for policy 0, policy_version 1243773 (0.0007) [2023-12-27 14:15:56,880][20381] Updated weights for policy 1, policy_version 1242649 (0.0006) [2023-12-27 14:15:56,930][20381] Updated weights for policy 1, policy_version 1242659 (0.0007) [2023-12-27 14:15:56,982][20381] Updated weights for policy 1, policy_version 1242669 (0.0008) [2023-12-27 14:15:57,031][20381] Updated weights for policy 1, policy_version 1242679 (0.0008) [2023-12-27 14:15:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 636616704. Throughput: 0: 10447.5, 1: 10061.6. Samples: 636604444. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:15:57,363][19170] Avg episode reward: [(0, '970.800'), (1, '921.580')] [2023-12-27 14:15:57,397][20347] Updated weights for policy 0, policy_version 1243783 (0.0010) [2023-12-27 14:15:57,447][20347] Updated weights for policy 0, policy_version 1243793 (0.0010) [2023-12-27 14:15:57,505][20347] Updated weights for policy 0, policy_version 1243803 (0.0010) [2023-12-27 14:15:57,651][20381] Updated weights for policy 1, policy_version 1242689 (0.0005) [2023-12-27 14:15:57,704][20381] Updated weights for policy 1, policy_version 1242699 (0.0005) [2023-12-27 14:15:57,748][20381] Updated weights for policy 1, policy_version 1242709 (0.0005) [2023-12-27 14:15:58,157][20347] Updated weights for policy 0, policy_version 1243813 (0.0008) [2023-12-27 14:15:58,215][20347] Updated weights for policy 0, policy_version 1243823 (0.0007) [2023-12-27 14:15:58,268][20347] Updated weights for policy 0, policy_version 1243833 (0.0008) [2023-12-27 14:15:58,328][20381] Updated weights for policy 1, policy_version 1242719 (0.0007) [2023-12-27 14:15:58,395][20381] Updated weights for policy 1, policy_version 1242729 (0.0010) [2023-12-27 14:15:58,455][20381] Updated weights for policy 1, policy_version 1242739 (0.0008) [2023-12-27 14:15:59,082][20347] Updated weights for policy 0, policy_version 1243843 (0.0009) [2023-12-27 14:15:59,155][20347] Updated weights for policy 0, policy_version 1243853 (0.0008) [2023-12-27 14:15:59,222][20347] Updated weights for policy 0, policy_version 1243863 (0.0008) [2023-12-27 14:15:59,312][20381] Updated weights for policy 1, policy_version 1242749 (0.0008) [2023-12-27 14:15:59,376][20381] Updated weights for policy 1, policy_version 1242759 (0.0008) [2023-12-27 14:15:59,444][20381] Updated weights for policy 1, policy_version 1242769 (0.0008) [2023-12-27 14:15:59,917][20347] Updated weights for policy 0, policy_version 1243873 (0.0008) [2023-12-27 14:15:59,982][20347] Updated weights for policy 0, policy_version 1243883 (0.0008) [2023-12-27 14:16:00,039][20347] Updated weights for policy 0, policy_version 1243893 (0.0008) [2023-12-27 14:16:00,095][20347] Updated weights for policy 0, policy_version 1243903 (0.0008) [2023-12-27 14:16:00,116][20381] Updated weights for policy 1, policy_version 1242779 (0.0006) [2023-12-27 14:16:00,180][20381] Updated weights for policy 1, policy_version 1242789 (0.0008) [2023-12-27 14:16:00,237][20381] Updated weights for policy 1, policy_version 1242799 (0.0009) [2023-12-27 14:16:00,766][20347] Updated weights for policy 0, policy_version 1243913 (0.0008) [2023-12-27 14:16:00,818][20347] Updated weights for policy 0, policy_version 1243923 (0.0008) [2023-12-27 14:16:00,868][20347] Updated weights for policy 0, policy_version 1243933 (0.0007) [2023-12-27 14:16:00,881][20381] Updated weights for policy 1, policy_version 1242809 (0.0009) [2023-12-27 14:16:00,935][20381] Updated weights for policy 1, policy_version 1242819 (0.0005) [2023-12-27 14:16:00,989][20381] Updated weights for policy 1, policy_version 1242829 (0.0007) [2023-12-27 14:16:01,055][20381] Updated weights for policy 1, policy_version 1242840 (0.0008) [2023-12-27 14:16:01,542][20347] Updated weights for policy 0, policy_version 1243943 (0.0008) [2023-12-27 14:16:01,595][20347] Updated weights for policy 0, policy_version 1243953 (0.0007) [2023-12-27 14:16:01,654][20347] Updated weights for policy 0, policy_version 1243963 (0.0008) [2023-12-27 14:16:01,726][20381] Updated weights for policy 1, policy_version 1242850 (0.0008) [2023-12-27 14:16:01,782][20381] Updated weights for policy 1, policy_version 1242860 (0.0007) [2023-12-27 14:16:01,831][20381] Updated weights for policy 1, policy_version 1242870 (0.0007) [2023-12-27 14:16:02,357][20347] Updated weights for policy 0, policy_version 1243973 (0.0008) [2023-12-27 14:16:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 636715008. Throughput: 0: 10419.8, 1: 10060.2. Samples: 636728848. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:02,363][19170] Avg episode reward: [(0, '941.390'), (1, '941.360')] [2023-12-27 14:16:02,412][20347] Updated weights for policy 0, policy_version 1243983 (0.0008) [2023-12-27 14:16:02,460][20347] Updated weights for policy 0, policy_version 1243993 (0.0008) [2023-12-27 14:16:02,520][20381] Updated weights for policy 1, policy_version 1242880 (0.0007) [2023-12-27 14:16:02,561][20292] KL-divergence is very high: 356.7831 [2023-12-27 14:16:02,566][20381] Updated weights for policy 1, policy_version 1242890 (0.0007) [2023-12-27 14:16:02,607][20292] KL-divergence is very high: 225.6462 [2023-12-27 14:16:02,624][20381] Updated weights for policy 1, policy_version 1242900 (0.0008) [2023-12-27 14:16:03,138][20347] Updated weights for policy 0, policy_version 1244003 (0.0008) [2023-12-27 14:16:03,183][20347] Updated weights for policy 0, policy_version 1244013 (0.0007) [2023-12-27 14:16:03,233][20347] Updated weights for policy 0, policy_version 1244023 (0.0006) [2023-12-27 14:16:03,267][20381] Updated weights for policy 1, policy_version 1242910 (0.0009) [2023-12-27 14:16:03,314][20381] Updated weights for policy 1, policy_version 1242920 (0.0008) [2023-12-27 14:16:03,366][20381] Updated weights for policy 1, policy_version 1242930 (0.0008) [2023-12-27 14:16:03,899][20347] Updated weights for policy 0, policy_version 1244033 (0.0006) [2023-12-27 14:16:03,951][20347] Updated weights for policy 0, policy_version 1244043 (0.0008) [2023-12-27 14:16:03,999][20347] Updated weights for policy 0, policy_version 1244053 (0.0008) [2023-12-27 14:16:04,043][20381] Updated weights for policy 1, policy_version 1242940 (0.0007) [2023-12-27 14:16:04,055][20347] Updated weights for policy 0, policy_version 1244063 (0.0008) [2023-12-27 14:16:04,105][20381] Updated weights for policy 1, policy_version 1242950 (0.0006) [2023-12-27 14:16:04,165][20381] Updated weights for policy 1, policy_version 1242960 (0.0007) [2023-12-27 14:16:04,798][20347] Updated weights for policy 0, policy_version 1244073 (0.0007) [2023-12-27 14:16:04,815][20381] Updated weights for policy 1, policy_version 1242970 (0.0006) [2023-12-27 14:16:04,850][20347] Updated weights for policy 0, policy_version 1244083 (0.0010) [2023-12-27 14:16:04,866][20381] Updated weights for policy 1, policy_version 1242980 (0.0005) [2023-12-27 14:16:04,901][20347] Updated weights for policy 0, policy_version 1244093 (0.0009) [2023-12-27 14:16:04,917][20381] Updated weights for policy 1, policy_version 1242990 (0.0006) [2023-12-27 14:16:04,967][20381] Updated weights for policy 1, policy_version 1243000 (0.0007) [2023-12-27 14:16:05,562][20347] Updated weights for policy 0, policy_version 1244103 (0.0007) [2023-12-27 14:16:05,621][20347] Updated weights for policy 0, policy_version 1244113 (0.0007) [2023-12-27 14:16:05,681][20347] Updated weights for policy 0, policy_version 1244123 (0.0009) [2023-12-27 14:16:05,705][20381] Updated weights for policy 1, policy_version 1243010 (0.0007) [2023-12-27 14:16:05,761][20381] Updated weights for policy 1, policy_version 1243020 (0.0008) [2023-12-27 14:16:05,814][20381] Updated weights for policy 1, policy_version 1243030 (0.0008) [2023-12-27 14:16:06,388][20347] Updated weights for policy 0, policy_version 1244133 (0.0005) [2023-12-27 14:16:06,448][20347] Updated weights for policy 0, policy_version 1244143 (0.0007) [2023-12-27 14:16:06,510][20347] Updated weights for policy 0, policy_version 1244153 (0.0007) [2023-12-27 14:16:06,542][20381] Updated weights for policy 1, policy_version 1243040 (0.0006) [2023-12-27 14:16:06,599][20381] Updated weights for policy 1, policy_version 1243050 (0.0006) [2023-12-27 14:16:06,662][20381] Updated weights for policy 1, policy_version 1243060 (0.0008) [2023-12-27 14:16:07,151][20347] Updated weights for policy 0, policy_version 1244163 (0.0006) [2023-12-27 14:16:07,203][20347] Updated weights for policy 0, policy_version 1244173 (0.0005) [2023-12-27 14:16:07,255][20347] Updated weights for policy 0, policy_version 1244183 (0.0006) [2023-12-27 14:16:07,296][20381] Updated weights for policy 1, policy_version 1243070 (0.0006) [2023-12-27 14:16:07,347][20381] Updated weights for policy 1, policy_version 1243080 (0.0005) [2023-12-27 14:16:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 636821504. Throughput: 0: 10413.4, 1: 10097.7. Samples: 636790288. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:07,363][19170] Avg episode reward: [(0, '931.610'), (1, '951.200')] [2023-12-27 14:16:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001244192_318570496.pth... [2023-12-27 14:16:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001242976_318259200.pth [2023-12-27 14:16:07,409][20381] Updated weights for policy 1, policy_version 1243090 (0.0007) [2023-12-27 14:16:07,448][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001243096_318259200.pth... [2023-12-27 14:16:07,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001241912_317956096.pth [2023-12-27 14:16:07,852][20347] Updated weights for policy 0, policy_version 1244193 (0.0008) [2023-12-27 14:16:07,908][20347] Updated weights for policy 0, policy_version 1244203 (0.0006) [2023-12-27 14:16:07,955][20347] Updated weights for policy 0, policy_version 1244213 (0.0005) [2023-12-27 14:16:08,004][20347] Updated weights for policy 0, policy_version 1244223 (0.0005) [2023-12-27 14:16:08,085][20381] Updated weights for policy 1, policy_version 1243100 (0.0009) [2023-12-27 14:16:08,138][20381] Updated weights for policy 1, policy_version 1243110 (0.0011) [2023-12-27 14:16:08,197][20381] Updated weights for policy 1, policy_version 1243120 (0.0011) [2023-12-27 14:16:08,669][20347] Updated weights for policy 0, policy_version 1244233 (0.0007) [2023-12-27 14:16:08,728][20347] Updated weights for policy 0, policy_version 1244243 (0.0007) [2023-12-27 14:16:08,781][20347] Updated weights for policy 0, policy_version 1244253 (0.0007) [2023-12-27 14:16:08,811][20381] Updated weights for policy 1, policy_version 1243130 (0.0010) [2023-12-27 14:16:08,861][20381] Updated weights for policy 1, policy_version 1243140 (0.0010) [2023-12-27 14:16:08,913][20381] Updated weights for policy 1, policy_version 1243150 (0.0010) [2023-12-27 14:16:08,968][20381] Updated weights for policy 1, policy_version 1243160 (0.0010) [2023-12-27 14:16:09,464][20347] Updated weights for policy 0, policy_version 1244263 (0.0007) [2023-12-27 14:16:09,525][20347] Updated weights for policy 0, policy_version 1244273 (0.0006) [2023-12-27 14:16:09,586][20347] Updated weights for policy 0, policy_version 1244283 (0.0008) [2023-12-27 14:16:09,675][20381] Updated weights for policy 1, policy_version 1243170 (0.0007) [2023-12-27 14:16:09,735][20381] Updated weights for policy 1, policy_version 1243180 (0.0006) [2023-12-27 14:16:09,784][20381] Updated weights for policy 1, policy_version 1243190 (0.0008) [2023-12-27 14:16:10,265][20347] Updated weights for policy 0, policy_version 1244293 (0.0006) [2023-12-27 14:16:10,329][20347] Updated weights for policy 0, policy_version 1244303 (0.0007) [2023-12-27 14:16:10,388][20347] Updated weights for policy 0, policy_version 1244313 (0.0009) [2023-12-27 14:16:10,536][20381] Updated weights for policy 1, policy_version 1243200 (0.0007) [2023-12-27 14:16:10,592][20381] Updated weights for policy 1, policy_version 1243210 (0.0008) [2023-12-27 14:16:10,649][20381] Updated weights for policy 1, policy_version 1243220 (0.0008) [2023-12-27 14:16:11,024][20347] Updated weights for policy 0, policy_version 1244323 (0.0007) [2023-12-27 14:16:11,088][20347] Updated weights for policy 0, policy_version 1244333 (0.0008) [2023-12-27 14:16:11,151][20347] Updated weights for policy 0, policy_version 1244343 (0.0008) [2023-12-27 14:16:11,339][20381] Updated weights for policy 1, policy_version 1243230 (0.0008) [2023-12-27 14:16:11,407][20381] Updated weights for policy 1, policy_version 1243240 (0.0008) [2023-12-27 14:16:11,468][20381] Updated weights for policy 1, policy_version 1243250 (0.0007) [2023-12-27 14:16:11,960][20347] Updated weights for policy 0, policy_version 1244353 (0.0008) [2023-12-27 14:16:12,011][20347] Updated weights for policy 0, policy_version 1244363 (0.0008) [2023-12-27 14:16:12,072][20347] Updated weights for policy 0, policy_version 1244373 (0.0007) [2023-12-27 14:16:12,132][20347] Updated weights for policy 0, policy_version 1244383 (0.0007) [2023-12-27 14:16:12,153][20381] Updated weights for policy 1, policy_version 1243260 (0.0007) [2023-12-27 14:16:12,210][20381] Updated weights for policy 1, policy_version 1243270 (0.0008) [2023-12-27 14:16:12,271][20381] Updated weights for policy 1, policy_version 1243280 (0.0008) [2023-12-27 14:16:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 636928000. Throughput: 0: 10385.7, 1: 10107.4. Samples: 636913824. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:12,363][19170] Avg episode reward: [(0, '921.930'), (1, '960.870')] [2023-12-27 14:16:12,836][20347] Updated weights for policy 0, policy_version 1244393 (0.0005) [2023-12-27 14:16:12,890][20347] Updated weights for policy 0, policy_version 1244403 (0.0006) [2023-12-27 14:16:12,939][20347] Updated weights for policy 0, policy_version 1244413 (0.0005) [2023-12-27 14:16:13,007][20381] Updated weights for policy 1, policy_version 1243290 (0.0008) [2023-12-27 14:16:13,058][20381] Updated weights for policy 1, policy_version 1243300 (0.0007) [2023-12-27 14:16:13,125][20381] Updated weights for policy 1, policy_version 1243310 (0.0005) [2023-12-27 14:16:13,184][20381] Updated weights for policy 1, policy_version 1243320 (0.0008) [2023-12-27 14:16:13,546][20347] Updated weights for policy 0, policy_version 1244423 (0.0007) [2023-12-27 14:16:13,551][20291] KL-divergence is very high: 226.0490 [2023-12-27 14:16:13,594][20291] KL-divergence is very high: 214.0849 [2023-12-27 14:16:13,599][20347] Updated weights for policy 0, policy_version 1244433 (0.0006) [2023-12-27 14:16:13,638][20291] KL-divergence is very high: 142.8332 [2023-12-27 14:16:13,654][20347] Updated weights for policy 0, policy_version 1244443 (0.0005) [2023-12-27 14:16:13,833][20381] Updated weights for policy 1, policy_version 1243330 (0.0007) [2023-12-27 14:16:13,884][20381] Updated weights for policy 1, policy_version 1243340 (0.0007) [2023-12-27 14:16:13,938][20381] Updated weights for policy 1, policy_version 1243350 (0.0007) [2023-12-27 14:16:14,265][20347] Updated weights for policy 0, policy_version 1244453 (0.0007) [2023-12-27 14:16:14,327][20347] Updated weights for policy 0, policy_version 1244463 (0.0008) [2023-12-27 14:16:14,394][20347] Updated weights for policy 0, policy_version 1244473 (0.0007) [2023-12-27 14:16:14,639][20381] Updated weights for policy 1, policy_version 1243360 (0.0009) [2023-12-27 14:16:14,693][20381] Updated weights for policy 1, policy_version 1243370 (0.0009) [2023-12-27 14:16:14,742][20381] Updated weights for policy 1, policy_version 1243380 (0.0008) [2023-12-27 14:16:14,984][20347] Updated weights for policy 0, policy_version 1244483 (0.0007) [2023-12-27 14:16:15,044][20347] Updated weights for policy 0, policy_version 1244493 (0.0010) [2023-12-27 14:16:15,107][20347] Updated weights for policy 0, policy_version 1244503 (0.0010) [2023-12-27 14:16:15,496][20381] Updated weights for policy 1, policy_version 1243390 (0.0008) [2023-12-27 14:16:15,557][20381] Updated weights for policy 1, policy_version 1243400 (0.0007) [2023-12-27 14:16:15,620][20381] Updated weights for policy 1, policy_version 1243410 (0.0007) [2023-12-27 14:16:15,776][20347] Updated weights for policy 0, policy_version 1244513 (0.0005) [2023-12-27 14:16:15,834][20347] Updated weights for policy 0, policy_version 1244523 (0.0010) [2023-12-27 14:16:15,886][20347] Updated weights for policy 0, policy_version 1244533 (0.0010) [2023-12-27 14:16:15,941][20347] Updated weights for policy 0, policy_version 1244543 (0.0010) [2023-12-27 14:16:16,270][20381] Updated weights for policy 1, policy_version 1243420 (0.0007) [2023-12-27 14:16:16,332][20381] Updated weights for policy 1, policy_version 1243430 (0.0005) [2023-12-27 14:16:16,391][20381] Updated weights for policy 1, policy_version 1243440 (0.0006) [2023-12-27 14:16:16,609][20347] Updated weights for policy 0, policy_version 1244553 (0.0007) [2023-12-27 14:16:16,671][20347] Updated weights for policy 0, policy_version 1244563 (0.0005) [2023-12-27 14:16:16,727][20347] Updated weights for policy 0, policy_version 1244573 (0.0009) [2023-12-27 14:16:16,956][20381] Updated weights for policy 1, policy_version 1243450 (0.0006) [2023-12-27 14:16:17,014][20381] Updated weights for policy 1, policy_version 1243460 (0.0010) [2023-12-27 14:16:17,072][20381] Updated weights for policy 1, policy_version 1243470 (0.0010) [2023-12-27 14:16:17,123][20381] Updated weights for policy 1, policy_version 1243480 (0.0010) [2023-12-27 14:16:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 637026304. Throughput: 0: 10354.8, 1: 10132.8. Samples: 637037100. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:17,363][19170] Avg episode reward: [(0, '921.930'), (1, '951.100')] [2023-12-27 14:16:17,394][20347] Updated weights for policy 0, policy_version 1244583 (0.0009) [2023-12-27 14:16:17,439][20347] Updated weights for policy 0, policy_version 1244593 (0.0007) [2023-12-27 14:16:17,488][20347] Updated weights for policy 0, policy_version 1244603 (0.0008) [2023-12-27 14:16:17,800][20381] Updated weights for policy 1, policy_version 1243490 (0.0007) [2023-12-27 14:16:17,863][20381] Updated weights for policy 1, policy_version 1243500 (0.0005) [2023-12-27 14:16:17,911][20381] Updated weights for policy 1, policy_version 1243510 (0.0005) [2023-12-27 14:16:18,179][20347] Updated weights for policy 0, policy_version 1244613 (0.0008) [2023-12-27 14:16:18,225][20347] Updated weights for policy 0, policy_version 1244623 (0.0008) [2023-12-27 14:16:18,276][20347] Updated weights for policy 0, policy_version 1244633 (0.0007) [2023-12-27 14:16:18,508][20381] Updated weights for policy 1, policy_version 1243520 (0.0007) [2023-12-27 14:16:18,568][20381] Updated weights for policy 1, policy_version 1243530 (0.0008) [2023-12-27 14:16:18,628][20381] Updated weights for policy 1, policy_version 1243540 (0.0008) [2023-12-27 14:16:18,927][20347] Updated weights for policy 0, policy_version 1244643 (0.0007) [2023-12-27 14:16:18,981][20347] Updated weights for policy 0, policy_version 1244653 (0.0007) [2023-12-27 14:16:19,038][20347] Updated weights for policy 0, policy_version 1244663 (0.0006) [2023-12-27 14:16:19,244][20381] Updated weights for policy 1, policy_version 1243550 (0.0008) [2023-12-27 14:16:19,301][20381] Updated weights for policy 1, policy_version 1243560 (0.0009) [2023-12-27 14:16:19,366][20381] Updated weights for policy 1, policy_version 1243570 (0.0010) [2023-12-27 14:16:19,720][20347] Updated weights for policy 0, policy_version 1244673 (0.0006) [2023-12-27 14:16:19,783][20347] Updated weights for policy 0, policy_version 1244683 (0.0007) [2023-12-27 14:16:19,850][20347] Updated weights for policy 0, policy_version 1244693 (0.0008) [2023-12-27 14:16:19,903][20347] Updated weights for policy 0, policy_version 1244703 (0.0007) [2023-12-27 14:16:20,118][20381] Updated weights for policy 1, policy_version 1243580 (0.0008) [2023-12-27 14:16:20,173][20381] Updated weights for policy 1, policy_version 1243590 (0.0011) [2023-12-27 14:16:20,224][20381] Updated weights for policy 1, policy_version 1243600 (0.0010) [2023-12-27 14:16:20,624][20347] Updated weights for policy 0, policy_version 1244713 (0.0007) [2023-12-27 14:16:20,688][20347] Updated weights for policy 0, policy_version 1244723 (0.0006) [2023-12-27 14:16:20,746][20347] Updated weights for policy 0, policy_version 1244733 (0.0007) [2023-12-27 14:16:20,976][20381] Updated weights for policy 1, policy_version 1243610 (0.0011) [2023-12-27 14:16:21,038][20381] Updated weights for policy 1, policy_version 1243620 (0.0008) [2023-12-27 14:16:21,105][20381] Updated weights for policy 1, policy_version 1243630 (0.0008) [2023-12-27 14:16:21,167][20381] Updated weights for policy 1, policy_version 1243640 (0.0009) [2023-12-27 14:16:21,498][20347] Updated weights for policy 0, policy_version 1244743 (0.0008) [2023-12-27 14:16:21,552][20347] Updated weights for policy 0, policy_version 1244753 (0.0006) [2023-12-27 14:16:21,616][20347] Updated weights for policy 0, policy_version 1244763 (0.0006) [2023-12-27 14:16:21,931][20381] Updated weights for policy 1, policy_version 1243650 (0.0007) [2023-12-27 14:16:21,992][20381] Updated weights for policy 1, policy_version 1243660 (0.0007) [2023-12-27 14:16:22,058][20381] Updated weights for policy 1, policy_version 1243670 (0.0008) [2023-12-27 14:16:22,311][20347] Updated weights for policy 0, policy_version 1244773 (0.0009) [2023-12-27 14:16:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 637124608. Throughput: 0: 10404.3, 1: 10118.8. Samples: 637099024. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:22,363][19170] Avg episode reward: [(0, '951.110'), (1, '980.470')] [2023-12-27 14:16:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001243672_318406656.pth... [2023-12-27 14:16:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001242488_318103552.pth [2023-12-27 14:16:22,380][20347] Updated weights for policy 0, policy_version 1244783 (0.0010) [2023-12-27 14:16:22,437][20347] Updated weights for policy 0, policy_version 1244793 (0.0009) [2023-12-27 14:16:22,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001244800_318726144.pth... [2023-12-27 14:16:22,476][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001243584_318414848.pth [2023-12-27 14:16:22,824][20381] Updated weights for policy 1, policy_version 1243680 (0.0007) [2023-12-27 14:16:22,893][20381] Updated weights for policy 1, policy_version 1243690 (0.0008) [2023-12-27 14:16:22,956][20381] Updated weights for policy 1, policy_version 1243700 (0.0008) [2023-12-27 14:16:23,122][20347] Updated weights for policy 0, policy_version 1244803 (0.0009) [2023-12-27 14:16:23,180][20347] Updated weights for policy 0, policy_version 1244813 (0.0010) [2023-12-27 14:16:23,234][20347] Updated weights for policy 0, policy_version 1244823 (0.0010) [2023-12-27 14:16:23,526][20381] Updated weights for policy 1, policy_version 1243710 (0.0008) [2023-12-27 14:16:23,580][20381] Updated weights for policy 1, policy_version 1243720 (0.0005) [2023-12-27 14:16:23,639][20381] Updated weights for policy 1, policy_version 1243730 (0.0005) [2023-12-27 14:16:23,827][20347] Updated weights for policy 0, policy_version 1244833 (0.0010) [2023-12-27 14:16:23,876][20347] Updated weights for policy 0, policy_version 1244843 (0.0008) [2023-12-27 14:16:23,926][20347] Updated weights for policy 0, policy_version 1244853 (0.0007) [2023-12-27 14:16:23,980][20347] Updated weights for policy 0, policy_version 1244863 (0.0007) [2023-12-27 14:16:24,271][20381] Updated weights for policy 1, policy_version 1243740 (0.0008) [2023-12-27 14:16:24,320][20381] Updated weights for policy 1, policy_version 1243750 (0.0010) [2023-12-27 14:16:24,382][20381] Updated weights for policy 1, policy_version 1243760 (0.0010) [2023-12-27 14:16:24,684][20347] Updated weights for policy 0, policy_version 1244873 (0.0007) [2023-12-27 14:16:24,740][20347] Updated weights for policy 0, policy_version 1244883 (0.0009) [2023-12-27 14:16:24,785][20347] Updated weights for policy 0, policy_version 1244893 (0.0010) [2023-12-27 14:16:24,996][20381] Updated weights for policy 1, policy_version 1243770 (0.0010) [2023-12-27 14:16:25,052][20381] Updated weights for policy 1, policy_version 1243780 (0.0008) [2023-12-27 14:16:25,098][20381] Updated weights for policy 1, policy_version 1243790 (0.0009) [2023-12-27 14:16:25,148][20381] Updated weights for policy 1, policy_version 1243800 (0.0007) [2023-12-27 14:16:25,387][20291] KL-divergence is very high: 115.2245 [2023-12-27 14:16:25,428][20347] Updated weights for policy 0, policy_version 1244903 (0.0006) [2023-12-27 14:16:25,435][20291] KL-divergence is very high: 259.1400 [2023-12-27 14:16:25,482][20291] KL-divergence is very high: 122.8759 [2023-12-27 14:16:25,488][20347] Updated weights for policy 0, policy_version 1244913 (0.0006) [2023-12-27 14:16:25,538][20347] Updated weights for policy 0, policy_version 1244923 (0.0007) [2023-12-27 14:16:25,538][20291] KL-divergence is very high: 153.5679 [2023-12-27 14:16:25,828][20381] Updated weights for policy 1, policy_version 1243810 (0.0007) [2023-12-27 14:16:25,885][20381] Updated weights for policy 1, policy_version 1243820 (0.0006) [2023-12-27 14:16:25,943][20381] Updated weights for policy 1, policy_version 1243830 (0.0005) [2023-12-27 14:16:26,155][20291] KL-divergence is very high: 121.7895 [2023-12-27 14:16:26,179][20347] Updated weights for policy 0, policy_version 1244933 (0.0006) [2023-12-27 14:16:26,238][20347] Updated weights for policy 0, policy_version 1244943 (0.0007) [2023-12-27 14:16:26,302][20347] Updated weights for policy 0, policy_version 1244953 (0.0005) [2023-12-27 14:16:26,631][20381] Updated weights for policy 1, policy_version 1243840 (0.0007) [2023-12-27 14:16:26,690][20381] Updated weights for policy 1, policy_version 1243850 (0.0009) [2023-12-27 14:16:26,745][20381] Updated weights for policy 1, policy_version 1243860 (0.0006) [2023-12-27 14:16:26,820][20347] Updated weights for policy 0, policy_version 1244963 (0.0006) [2023-12-27 14:16:26,880][20347] Updated weights for policy 0, policy_version 1244973 (0.0010) [2023-12-27 14:16:26,946][20347] Updated weights for policy 0, policy_version 1244983 (0.0010) [2023-12-27 14:16:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 637231104. Throughput: 0: 10389.9, 1: 10138.9. Samples: 637221432. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:27,363][19170] Avg episode reward: [(0, '912.040'), (1, '970.670')] [2023-12-27 14:16:27,364][20381] Updated weights for policy 1, policy_version 1243870 (0.0008) [2023-12-27 14:16:27,412][20381] Updated weights for policy 1, policy_version 1243880 (0.0005) [2023-12-27 14:16:27,456][20381] Updated weights for policy 1, policy_version 1243890 (0.0005) [2023-12-27 14:16:27,598][20347] Updated weights for policy 0, policy_version 1244993 (0.0010) [2023-12-27 14:16:27,649][20347] Updated weights for policy 0, policy_version 1245003 (0.0010) [2023-12-27 14:16:27,694][20347] Updated weights for policy 0, policy_version 1245013 (0.0010) [2023-12-27 14:16:27,749][20347] Updated weights for policy 0, policy_version 1245023 (0.0008) [2023-12-27 14:16:28,030][20381] Updated weights for policy 1, policy_version 1243900 (0.0005) [2023-12-27 14:16:28,088][20381] Updated weights for policy 1, policy_version 1243910 (0.0005) [2023-12-27 14:16:28,140][20381] Updated weights for policy 1, policy_version 1243920 (0.0005) [2023-12-27 14:16:28,425][20347] Updated weights for policy 0, policy_version 1245033 (0.0007) [2023-12-27 14:16:28,485][20347] Updated weights for policy 0, policy_version 1245043 (0.0007) [2023-12-27 14:16:28,539][20347] Updated weights for policy 0, policy_version 1245053 (0.0007) [2023-12-27 14:16:28,732][20381] Updated weights for policy 1, policy_version 1243930 (0.0005) [2023-12-27 14:16:28,794][20381] Updated weights for policy 1, policy_version 1243940 (0.0006) [2023-12-27 14:16:28,855][20381] Updated weights for policy 1, policy_version 1243950 (0.0005) [2023-12-27 14:16:28,918][20381] Updated weights for policy 1, policy_version 1243960 (0.0005) [2023-12-27 14:16:29,237][20347] Updated weights for policy 0, policy_version 1245064 (0.0008) [2023-12-27 14:16:29,292][20347] Updated weights for policy 0, policy_version 1245074 (0.0008) [2023-12-27 14:16:29,356][20347] Updated weights for policy 0, policy_version 1245084 (0.0008) [2023-12-27 14:16:29,440][20381] Updated weights for policy 1, policy_version 1243970 (0.0008) [2023-12-27 14:16:29,496][20381] Updated weights for policy 1, policy_version 1243980 (0.0011) [2023-12-27 14:16:29,552][20381] Updated weights for policy 1, policy_version 1243990 (0.0007) [2023-12-27 14:16:30,099][20347] Updated weights for policy 0, policy_version 1245094 (0.0008) [2023-12-27 14:16:30,153][20347] Updated weights for policy 0, policy_version 1245104 (0.0007) [2023-12-27 14:16:30,204][20347] Updated weights for policy 0, policy_version 1245114 (0.0007) [2023-12-27 14:16:30,269][20381] Updated weights for policy 1, policy_version 1244000 (0.0009) [2023-12-27 14:16:30,327][20381] Updated weights for policy 1, policy_version 1244010 (0.0010) [2023-12-27 14:16:30,386][20381] Updated weights for policy 1, policy_version 1244020 (0.0010) [2023-12-27 14:16:30,898][20347] Updated weights for policy 0, policy_version 1245124 (0.0007) [2023-12-27 14:16:30,954][20291] KL-divergence is very high: 863.9658 [2023-12-27 14:16:30,955][20347] Updated weights for policy 0, policy_version 1245134 (0.0007) [2023-12-27 14:16:31,002][20291] KL-divergence is very high: 568.9260 [2023-12-27 14:16:31,012][20347] Updated weights for policy 0, policy_version 1245144 (0.0006) [2023-12-27 14:16:31,033][20291] KL-divergence is very high: 144.5429 [2023-12-27 14:16:31,053][20291] KL-divergence is very high: 192.6303 [2023-12-27 14:16:31,065][20381] Updated weights for policy 1, policy_version 1244030 (0.0010) [2023-12-27 14:16:31,130][20381] Updated weights for policy 1, policy_version 1244040 (0.0011) [2023-12-27 14:16:31,187][20381] Updated weights for policy 1, policy_version 1244050 (0.0007) [2023-12-27 14:16:31,749][20347] Updated weights for policy 0, policy_version 1245154 (0.0008) [2023-12-27 14:16:31,808][20347] Updated weights for policy 0, policy_version 1245164 (0.0008) [2023-12-27 14:16:31,853][20347] Updated weights for policy 0, policy_version 1245174 (0.0008) [2023-12-27 14:16:31,857][20381] Updated weights for policy 1, policy_version 1244060 (0.0005) [2023-12-27 14:16:31,872][20291] KL-divergence is very high: 431.9062 [2023-12-27 14:16:31,877][20291] KL-divergence is very high: 145.7564 [2023-12-27 14:16:31,900][20347] Updated weights for policy 0, policy_version 1245184 (0.0006) [2023-12-27 14:16:31,910][20381] Updated weights for policy 1, policy_version 1244070 (0.0007) [2023-12-27 14:16:31,963][20381] Updated weights for policy 1, policy_version 1244080 (0.0007) [2023-12-27 14:16:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 637337600. Throughput: 0: 10341.2, 1: 10238.2. Samples: 637347220. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:32,363][19170] Avg episode reward: [(0, '912.080'), (1, '941.350')] [2023-12-27 14:16:32,611][20347] Updated weights for policy 0, policy_version 1245194 (0.0007) [2023-12-27 14:16:32,668][20381] Updated weights for policy 1, policy_version 1244090 (0.0007) [2023-12-27 14:16:32,673][20347] Updated weights for policy 0, policy_version 1245204 (0.0007) [2023-12-27 14:16:32,724][20381] Updated weights for policy 1, policy_version 1244100 (0.0005) [2023-12-27 14:16:32,729][20347] Updated weights for policy 0, policy_version 1245214 (0.0006) [2023-12-27 14:16:32,776][20381] Updated weights for policy 1, policy_version 1244110 (0.0007) [2023-12-27 14:16:32,827][20381] Updated weights for policy 1, policy_version 1244120 (0.0007) [2023-12-27 14:16:33,400][20347] Updated weights for policy 0, policy_version 1245224 (0.0006) [2023-12-27 14:16:33,444][20347] Updated weights for policy 0, policy_version 1245234 (0.0007) [2023-12-27 14:16:33,494][20347] Updated weights for policy 0, policy_version 1245244 (0.0006) [2023-12-27 14:16:33,527][20381] Updated weights for policy 1, policy_version 1244130 (0.0010) [2023-12-27 14:16:33,581][20381] Updated weights for policy 1, policy_version 1244140 (0.0010) [2023-12-27 14:16:33,637][20381] Updated weights for policy 1, policy_version 1244150 (0.0008) [2023-12-27 14:16:34,208][20347] Updated weights for policy 0, policy_version 1245254 (0.0007) [2023-12-27 14:16:34,269][20347] Updated weights for policy 0, policy_version 1245264 (0.0008) [2023-12-27 14:16:34,327][20347] Updated weights for policy 0, policy_version 1245274 (0.0007) [2023-12-27 14:16:34,347][20381] Updated weights for policy 1, policy_version 1244160 (0.0008) [2023-12-27 14:16:34,402][20381] Updated weights for policy 1, policy_version 1244170 (0.0008) [2023-12-27 14:16:34,462][20381] Updated weights for policy 1, policy_version 1244180 (0.0008) [2023-12-27 14:16:35,010][20347] Updated weights for policy 0, policy_version 1245284 (0.0006) [2023-12-27 14:16:35,065][20347] Updated weights for policy 0, policy_version 1245294 (0.0006) [2023-12-27 14:16:35,123][20347] Updated weights for policy 0, policy_version 1245304 (0.0005) [2023-12-27 14:16:35,178][20381] Updated weights for policy 1, policy_version 1244190 (0.0007) [2023-12-27 14:16:35,237][20381] Updated weights for policy 1, policy_version 1244200 (0.0008) [2023-12-27 14:16:35,286][20381] Updated weights for policy 1, policy_version 1244210 (0.0005) [2023-12-27 14:16:35,731][20347] Updated weights for policy 0, policy_version 1245314 (0.0006) [2023-12-27 14:16:35,781][20347] Updated weights for policy 0, policy_version 1245324 (0.0006) [2023-12-27 14:16:35,838][20347] Updated weights for policy 0, policy_version 1245334 (0.0005) [2023-12-27 14:16:35,897][20347] Updated weights for policy 0, policy_version 1245344 (0.0007) [2023-12-27 14:16:35,954][20381] Updated weights for policy 1, policy_version 1244220 (0.0008) [2023-12-27 14:16:35,999][20381] Updated weights for policy 1, policy_version 1244230 (0.0010) [2023-12-27 14:16:36,050][20381] Updated weights for policy 1, policy_version 1244240 (0.0010) [2023-12-27 14:16:36,513][20347] Updated weights for policy 0, policy_version 1245354 (0.0007) [2023-12-27 14:16:36,577][20347] Updated weights for policy 0, policy_version 1245364 (0.0007) [2023-12-27 14:16:36,635][20347] Updated weights for policy 0, policy_version 1245374 (0.0008) [2023-12-27 14:16:36,813][20381] Updated weights for policy 1, policy_version 1244250 (0.0007) [2023-12-27 14:16:36,879][20381] Updated weights for policy 1, policy_version 1244260 (0.0007) [2023-12-27 14:16:36,940][20381] Updated weights for policy 1, policy_version 1244270 (0.0007) [2023-12-27 14:16:37,002][20381] Updated weights for policy 1, policy_version 1244280 (0.0008) [2023-12-27 14:16:37,253][20347] Updated weights for policy 0, policy_version 1245384 (0.0006) [2023-12-27 14:16:37,310][20347] Updated weights for policy 0, policy_version 1245394 (0.0005) [2023-12-27 14:16:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 637435904. Throughput: 0: 10319.7, 1: 10260.8. Samples: 637407440. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:37,364][19170] Avg episode reward: [(0, '864.810'), (1, '970.680')] [2023-12-27 14:16:37,365][20347] Updated weights for policy 0, policy_version 1245404 (0.0007) [2023-12-27 14:16:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001244280_318562304.pth... [2023-12-27 14:16:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001243096_318259200.pth [2023-12-27 14:16:37,387][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001245408_318881792.pth... [2023-12-27 14:16:37,391][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001244192_318570496.pth [2023-12-27 14:16:37,695][20381] Updated weights for policy 1, policy_version 1244290 (0.0006) [2023-12-27 14:16:37,756][20381] Updated weights for policy 1, policy_version 1244300 (0.0008) [2023-12-27 14:16:37,811][20381] Updated weights for policy 1, policy_version 1244310 (0.0006) [2023-12-27 14:16:38,026][20347] Updated weights for policy 0, policy_version 1245414 (0.0006) [2023-12-27 14:16:38,072][20347] Updated weights for policy 0, policy_version 1245424 (0.0006) [2023-12-27 14:16:38,119][20347] Updated weights for policy 0, policy_version 1245434 (0.0006) [2023-12-27 14:16:38,390][20381] Updated weights for policy 1, policy_version 1244320 (0.0007) [2023-12-27 14:16:38,446][20381] Updated weights for policy 1, policy_version 1244330 (0.0006) [2023-12-27 14:16:38,504][20381] Updated weights for policy 1, policy_version 1244340 (0.0005) [2023-12-27 14:16:38,764][20291] KL-divergence is very high: 111.2165 [2023-12-27 14:16:38,781][20347] Updated weights for policy 0, policy_version 1245444 (0.0006) [2023-12-27 14:16:38,808][20291] KL-divergence is very high: 202.9900 [2023-12-27 14:16:38,814][20291] KL-divergence is very high: 107.1425 [2023-12-27 14:16:38,838][20347] Updated weights for policy 0, policy_version 1245454 (0.0008) [2023-12-27 14:16:38,856][20291] KL-divergence is very high: 114.6727 [2023-12-27 14:16:38,893][20347] Updated weights for policy 0, policy_version 1245464 (0.0008) [2023-12-27 14:16:39,166][20381] Updated weights for policy 1, policy_version 1244350 (0.0006) [2023-12-27 14:16:39,232][20381] Updated weights for policy 1, policy_version 1244360 (0.0007) [2023-12-27 14:16:39,288][20381] Updated weights for policy 1, policy_version 1244370 (0.0008) [2023-12-27 14:16:39,582][20347] Updated weights for policy 0, policy_version 1245474 (0.0007) [2023-12-27 14:16:39,640][20347] Updated weights for policy 0, policy_version 1245484 (0.0008) [2023-12-27 14:16:39,697][20347] Updated weights for policy 0, policy_version 1245494 (0.0009) [2023-12-27 14:16:39,752][20347] Updated weights for policy 0, policy_version 1245504 (0.0008) [2023-12-27 14:16:39,926][20381] Updated weights for policy 1, policy_version 1244380 (0.0007) [2023-12-27 14:16:39,985][20381] Updated weights for policy 1, policy_version 1244390 (0.0008) [2023-12-27 14:16:40,048][20381] Updated weights for policy 1, policy_version 1244400 (0.0008) [2023-12-27 14:16:40,399][20347] Updated weights for policy 0, policy_version 1245514 (0.0007) [2023-12-27 14:16:40,463][20347] Updated weights for policy 0, policy_version 1245524 (0.0006) [2023-12-27 14:16:40,518][20347] Updated weights for policy 0, policy_version 1245534 (0.0006) [2023-12-27 14:16:40,783][20381] Updated weights for policy 1, policy_version 1244410 (0.0009) [2023-12-27 14:16:40,838][20381] Updated weights for policy 1, policy_version 1244420 (0.0010) [2023-12-27 14:16:40,899][20381] Updated weights for policy 1, policy_version 1244430 (0.0008) [2023-12-27 14:16:40,955][20381] Updated weights for policy 1, policy_version 1244440 (0.0009) [2023-12-27 14:16:41,105][20347] Updated weights for policy 0, policy_version 1245544 (0.0007) [2023-12-27 14:16:41,173][20347] Updated weights for policy 0, policy_version 1245554 (0.0010) [2023-12-27 14:16:41,241][20347] Updated weights for policy 0, policy_version 1245564 (0.0006) [2023-12-27 14:16:41,695][20381] Updated weights for policy 1, policy_version 1244450 (0.0011) [2023-12-27 14:16:41,756][20381] Updated weights for policy 1, policy_version 1244460 (0.0012) [2023-12-27 14:16:41,819][20381] Updated weights for policy 1, policy_version 1244470 (0.0011) [2023-12-27 14:16:41,933][20347] Updated weights for policy 0, policy_version 1245574 (0.0007) [2023-12-27 14:16:41,988][20347] Updated weights for policy 0, policy_version 1245584 (0.0006) [2023-12-27 14:16:42,044][20347] Updated weights for policy 0, policy_version 1245594 (0.0008) [2023-12-27 14:16:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 637542400. Throughput: 0: 10370.3, 1: 10257.2. Samples: 637532684. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:42,363][19170] Avg episode reward: [(0, '854.840'), (1, '980.470')] [2023-12-27 14:16:42,596][20381] Updated weights for policy 1, policy_version 1244480 (0.0009) [2023-12-27 14:16:42,644][20381] Updated weights for policy 1, policy_version 1244490 (0.0008) [2023-12-27 14:16:42,703][20381] Updated weights for policy 1, policy_version 1244500 (0.0009) [2023-12-27 14:16:42,765][20347] Updated weights for policy 0, policy_version 1245604 (0.0008) [2023-12-27 14:16:42,820][20347] Updated weights for policy 0, policy_version 1245614 (0.0005) [2023-12-27 14:16:42,883][20347] Updated weights for policy 0, policy_version 1245624 (0.0007) [2023-12-27 14:16:43,367][20381] Updated weights for policy 1, policy_version 1244510 (0.0009) [2023-12-27 14:16:43,425][20381] Updated weights for policy 1, policy_version 1244520 (0.0011) [2023-12-27 14:16:43,487][20381] Updated weights for policy 1, policy_version 1244530 (0.0010) [2023-12-27 14:16:43,552][20347] Updated weights for policy 0, policy_version 1245634 (0.0010) [2023-12-27 14:16:43,604][20347] Updated weights for policy 0, policy_version 1245644 (0.0010) [2023-12-27 14:16:43,653][20347] Updated weights for policy 0, policy_version 1245654 (0.0008) [2023-12-27 14:16:43,709][20347] Updated weights for policy 0, policy_version 1245664 (0.0006) [2023-12-27 14:16:44,055][20381] Updated weights for policy 1, policy_version 1244540 (0.0009) [2023-12-27 14:16:44,104][20381] Updated weights for policy 1, policy_version 1244550 (0.0005) [2023-12-27 14:16:44,160][20381] Updated weights for policy 1, policy_version 1244560 (0.0006) [2023-12-27 14:16:44,246][20347] Updated weights for policy 0, policy_version 1245674 (0.0010) [2023-12-27 14:16:44,291][20347] Updated weights for policy 0, policy_version 1245684 (0.0010) [2023-12-27 14:16:44,345][20347] Updated weights for policy 0, policy_version 1245694 (0.0010) [2023-12-27 14:16:44,805][20381] Updated weights for policy 1, policy_version 1244570 (0.0006) [2023-12-27 14:16:44,868][20381] Updated weights for policy 1, policy_version 1244580 (0.0008) [2023-12-27 14:16:44,926][20381] Updated weights for policy 1, policy_version 1244590 (0.0008) [2023-12-27 14:16:44,990][20381] Updated weights for policy 1, policy_version 1244600 (0.0008) [2023-12-27 14:16:45,076][20347] Updated weights for policy 0, policy_version 1245704 (0.0011) [2023-12-27 14:16:45,132][20347] Updated weights for policy 0, policy_version 1245714 (0.0008) [2023-12-27 14:16:45,192][20347] Updated weights for policy 0, policy_version 1245724 (0.0005) [2023-12-27 14:16:45,672][20381] Updated weights for policy 1, policy_version 1244610 (0.0006) [2023-12-27 14:16:45,734][20381] Updated weights for policy 1, policy_version 1244620 (0.0005) [2023-12-27 14:16:45,748][20347] Updated weights for policy 0, policy_version 1245734 (0.0009) [2023-12-27 14:16:45,797][20381] Updated weights for policy 1, policy_version 1244630 (0.0008) [2023-12-27 14:16:45,808][20347] Updated weights for policy 0, policy_version 1245744 (0.0010) [2023-12-27 14:16:45,859][20347] Updated weights for policy 0, policy_version 1245754 (0.0010) [2023-12-27 14:16:46,388][20381] Updated weights for policy 1, policy_version 1244640 (0.0005) [2023-12-27 14:16:46,438][20381] Updated weights for policy 1, policy_version 1244650 (0.0005) [2023-12-27 14:16:46,469][20347] Updated weights for policy 0, policy_version 1245764 (0.0011) [2023-12-27 14:16:46,489][20381] Updated weights for policy 1, policy_version 1244660 (0.0005) [2023-12-27 14:16:46,521][20347] Updated weights for policy 0, policy_version 1245774 (0.0010) [2023-12-27 14:16:46,572][20347] Updated weights for policy 0, policy_version 1245784 (0.0010) [2023-12-27 14:16:47,137][20381] Updated weights for policy 1, policy_version 1244670 (0.0007) [2023-12-27 14:16:47,192][20381] Updated weights for policy 1, policy_version 1244680 (0.0007) [2023-12-27 14:16:47,238][20381] Updated weights for policy 1, policy_version 1244690 (0.0006) [2023-12-27 14:16:47,244][20347] Updated weights for policy 0, policy_version 1245794 (0.0009) [2023-12-27 14:16:47,301][20347] Updated weights for policy 0, policy_version 1245804 (0.0008) [2023-12-27 14:16:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 637648896. Throughput: 0: 10401.5, 1: 10283.4. Samples: 637659668. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:47,363][19170] Avg episode reward: [(0, '843.890'), (1, '960.920')] [2023-12-27 14:16:47,364][20347] Updated weights for policy 0, policy_version 1245814 (0.0007) [2023-12-27 14:16:47,426][20347] Updated weights for policy 0, policy_version 1245824 (0.0006) [2023-12-27 14:16:47,821][20381] Updated weights for policy 1, policy_version 1244700 (0.0005) [2023-12-27 14:16:47,882][20381] Updated weights for policy 1, policy_version 1244710 (0.0005) [2023-12-27 14:16:47,937][20381] Updated weights for policy 1, policy_version 1244720 (0.0005) [2023-12-27 14:16:48,125][20347] Updated weights for policy 0, policy_version 1245834 (0.0007) [2023-12-27 14:16:48,179][20347] Updated weights for policy 0, policy_version 1245844 (0.0007) [2023-12-27 14:16:48,239][20347] Updated weights for policy 0, policy_version 1245854 (0.0007) [2023-12-27 14:16:48,550][20381] Updated weights for policy 1, policy_version 1244730 (0.0006) [2023-12-27 14:16:48,607][20381] Updated weights for policy 1, policy_version 1244740 (0.0011) [2023-12-27 14:16:48,660][20381] Updated weights for policy 1, policy_version 1244750 (0.0011) [2023-12-27 14:16:48,713][20381] Updated weights for policy 1, policy_version 1244760 (0.0011) [2023-12-27 14:16:48,937][20347] Updated weights for policy 0, policy_version 1245864 (0.0007) [2023-12-27 14:16:48,988][20347] Updated weights for policy 0, policy_version 1245874 (0.0007) [2023-12-27 14:16:49,047][20347] Updated weights for policy 0, policy_version 1245884 (0.0007) [2023-12-27 14:16:49,411][20381] Updated weights for policy 1, policy_version 1244770 (0.0007) [2023-12-27 14:16:49,468][20381] Updated weights for policy 1, policy_version 1244780 (0.0005) [2023-12-27 14:16:49,524][20381] Updated weights for policy 1, policy_version 1244790 (0.0006) [2023-12-27 14:16:49,783][20347] Updated weights for policy 0, policy_version 1245894 (0.0008) [2023-12-27 14:16:49,843][20347] Updated weights for policy 0, policy_version 1245904 (0.0007) [2023-12-27 14:16:49,898][20347] Updated weights for policy 0, policy_version 1245914 (0.0008) [2023-12-27 14:16:50,120][20381] Updated weights for policy 1, policy_version 1244800 (0.0005) [2023-12-27 14:16:50,171][20381] Updated weights for policy 1, policy_version 1244810 (0.0005) [2023-12-27 14:16:50,223][20381] Updated weights for policy 1, policy_version 1244820 (0.0005) [2023-12-27 14:16:50,640][20347] Updated weights for policy 0, policy_version 1245924 (0.0007) [2023-12-27 14:16:50,695][20347] Updated weights for policy 0, policy_version 1245934 (0.0007) [2023-12-27 14:16:50,750][20347] Updated weights for policy 0, policy_version 1245944 (0.0008) [2023-12-27 14:16:50,873][20381] Updated weights for policy 1, policy_version 1244830 (0.0007) [2023-12-27 14:16:50,931][20381] Updated weights for policy 1, policy_version 1244840 (0.0007) [2023-12-27 14:16:50,998][20381] Updated weights for policy 1, policy_version 1244850 (0.0005) [2023-12-27 14:16:51,455][20347] Updated weights for policy 0, policy_version 1245954 (0.0008) [2023-12-27 14:16:51,515][20347] Updated weights for policy 0, policy_version 1245964 (0.0008) [2023-12-27 14:16:51,574][20347] Updated weights for policy 0, policy_version 1245974 (0.0008) [2023-12-27 14:16:51,641][20347] Updated weights for policy 0, policy_version 1245984 (0.0007) [2023-12-27 14:16:51,697][20381] Updated weights for policy 1, policy_version 1244860 (0.0007) [2023-12-27 14:16:51,758][20381] Updated weights for policy 1, policy_version 1244870 (0.0009) [2023-12-27 14:16:51,816][20381] Updated weights for policy 1, policy_version 1244880 (0.0007) [2023-12-27 14:16:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 637747200. Throughput: 0: 10369.8, 1: 10325.1. Samples: 637721560. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:52,363][19170] Avg episode reward: [(0, '863.700'), (1, '960.950')] [2023-12-27 14:16:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001244888_318717952.pth... [2023-12-27 14:16:52,390][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001243672_318406656.pth [2023-12-27 14:16:52,415][20347] Updated weights for policy 0, policy_version 1245994 (0.0007) [2023-12-27 14:16:52,467][20347] Updated weights for policy 0, policy_version 1246004 (0.0008) [2023-12-27 14:16:52,485][20381] Updated weights for policy 1, policy_version 1244890 (0.0007) [2023-12-27 14:16:52,490][20291] KL-divergence is very high: 102.5502 [2023-12-27 14:16:52,525][20347] Updated weights for policy 0, policy_version 1246014 (0.0006) [2023-12-27 14:16:52,533][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001246016_319037440.pth... [2023-12-27 14:16:52,537][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001244800_318726144.pth [2023-12-27 14:16:52,542][20381] Updated weights for policy 1, policy_version 1244900 (0.0006) [2023-12-27 14:16:52,598][20381] Updated weights for policy 1, policy_version 1244910 (0.0008) [2023-12-27 14:16:52,657][20381] Updated weights for policy 1, policy_version 1244920 (0.0007) [2023-12-27 14:16:53,207][20347] Updated weights for policy 0, policy_version 1246024 (0.0006) [2023-12-27 14:16:53,252][20347] Updated weights for policy 0, policy_version 1246034 (0.0007) [2023-12-27 14:16:53,308][20347] Updated weights for policy 0, policy_version 1246044 (0.0008) [2023-12-27 14:16:53,363][20381] Updated weights for policy 1, policy_version 1244930 (0.0005) [2023-12-27 14:16:53,424][20381] Updated weights for policy 1, policy_version 1244940 (0.0006) [2023-12-27 14:16:53,483][20381] Updated weights for policy 1, policy_version 1244950 (0.0009) [2023-12-27 14:16:54,009][20347] Updated weights for policy 0, policy_version 1246054 (0.0007) [2023-12-27 14:16:54,062][20347] Updated weights for policy 0, policy_version 1246064 (0.0008) [2023-12-27 14:16:54,108][20347] Updated weights for policy 0, policy_version 1246074 (0.0008) [2023-12-27 14:16:54,136][20381] Updated weights for policy 1, policy_version 1244960 (0.0008) [2023-12-27 14:16:54,140][20292] KL-divergence is very high: 356.0228 [2023-12-27 14:16:54,181][20292] KL-divergence is very high: 214.9659 [2023-12-27 14:16:54,185][20381] Updated weights for policy 1, policy_version 1244970 (0.0008) [2023-12-27 14:16:54,236][20381] Updated weights for policy 1, policy_version 1244980 (0.0009) [2023-12-27 14:16:54,762][20347] Updated weights for policy 0, policy_version 1246084 (0.0008) [2023-12-27 14:16:54,822][20347] Updated weights for policy 0, policy_version 1246094 (0.0010) [2023-12-27 14:16:54,877][20347] Updated weights for policy 0, policy_version 1246104 (0.0010) [2023-12-27 14:16:54,947][20381] Updated weights for policy 1, policy_version 1244990 (0.0010) [2023-12-27 14:16:55,009][20381] Updated weights for policy 1, policy_version 1245000 (0.0010) [2023-12-27 14:16:55,072][20381] Updated weights for policy 1, policy_version 1245010 (0.0006) [2023-12-27 14:16:55,552][20347] Updated weights for policy 0, policy_version 1246114 (0.0010) [2023-12-27 14:16:55,609][20347] Updated weights for policy 0, policy_version 1246124 (0.0010) [2023-12-27 14:16:55,664][20347] Updated weights for policy 0, policy_version 1246134 (0.0010) [2023-12-27 14:16:55,720][20347] Updated weights for policy 0, policy_version 1246144 (0.0010) [2023-12-27 14:16:55,721][20381] Updated weights for policy 1, policy_version 1245020 (0.0006) [2023-12-27 14:16:55,766][20381] Updated weights for policy 1, policy_version 1245030 (0.0005) [2023-12-27 14:16:55,817][20381] Updated weights for policy 1, policy_version 1245040 (0.0005) [2023-12-27 14:16:56,327][20347] Updated weights for policy 0, policy_version 1246154 (0.0010) [2023-12-27 14:16:56,373][20347] Updated weights for policy 0, policy_version 1246164 (0.0009) [2023-12-27 14:16:56,396][20291] KL-divergence is very high: 283.8735 [2023-12-27 14:16:56,412][20291] KL-divergence is very high: 240.0021 [2023-12-27 14:16:56,430][20347] Updated weights for policy 0, policy_version 1246174 (0.0005) [2023-12-27 14:16:56,450][20381] Updated weights for policy 1, policy_version 1245050 (0.0006) [2023-12-27 14:16:56,497][20381] Updated weights for policy 1, policy_version 1245060 (0.0010) [2023-12-27 14:16:56,551][20381] Updated weights for policy 1, policy_version 1245070 (0.0010) [2023-12-27 14:16:56,601][20381] Updated weights for policy 1, policy_version 1245080 (0.0010) [2023-12-27 14:16:56,955][20291] KL-divergence is very high: 174.6667 [2023-12-27 14:16:56,997][20347] Updated weights for policy 0, policy_version 1246184 (0.0009) [2023-12-27 14:16:57,025][20291] KL-divergence is very high: 176.0852 [2023-12-27 14:16:57,047][20347] Updated weights for policy 0, policy_version 1246194 (0.0009) [2023-12-27 14:16:57,065][20291] KL-divergence is very high: 110.4942 [2023-12-27 14:16:57,094][20347] Updated weights for policy 0, policy_version 1246204 (0.0010) [2023-12-27 14:16:57,277][20381] Updated weights for policy 1, policy_version 1245090 (0.0010) [2023-12-27 14:16:57,329][20381] Updated weights for policy 1, policy_version 1245100 (0.0010) [2023-12-27 14:16:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 637853696. Throughput: 0: 10313.3, 1: 10344.1. Samples: 637843408. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:16:57,363][19170] Avg episode reward: [(0, '824.760'), (1, '970.730')] [2023-12-27 14:16:57,389][20381] Updated weights for policy 1, policy_version 1245110 (0.0010) [2023-12-27 14:16:57,741][20347] Updated weights for policy 0, policy_version 1246214 (0.0009) [2023-12-27 14:16:57,785][20347] Updated weights for policy 0, policy_version 1246224 (0.0010) [2023-12-27 14:16:57,833][20347] Updated weights for policy 0, policy_version 1246234 (0.0010) [2023-12-27 14:16:58,035][20381] Updated weights for policy 1, policy_version 1245120 (0.0006) [2023-12-27 14:16:58,086][20381] Updated weights for policy 1, policy_version 1245130 (0.0005) [2023-12-27 14:16:58,140][20381] Updated weights for policy 1, policy_version 1245140 (0.0005) [2023-12-27 14:16:58,533][20347] Updated weights for policy 0, policy_version 1246244 (0.0010) [2023-12-27 14:16:58,594][20347] Updated weights for policy 0, policy_version 1246254 (0.0007) [2023-12-27 14:16:58,621][20291] KL-divergence is very high: 128.3201 [2023-12-27 14:16:58,660][20347] Updated weights for policy 0, policy_version 1246264 (0.0008) [2023-12-27 14:16:58,887][20381] Updated weights for policy 1, policy_version 1245150 (0.0008) [2023-12-27 14:16:58,944][20381] Updated weights for policy 1, policy_version 1245160 (0.0011) [2023-12-27 14:16:59,009][20381] Updated weights for policy 1, policy_version 1245170 (0.0010) [2023-12-27 14:16:59,421][20347] Updated weights for policy 0, policy_version 1246274 (0.0007) [2023-12-27 14:16:59,485][20347] Updated weights for policy 0, policy_version 1246284 (0.0005) [2023-12-27 14:16:59,536][20347] Updated weights for policy 0, policy_version 1246294 (0.0005) [2023-12-27 14:16:59,599][20347] Updated weights for policy 0, policy_version 1246304 (0.0010) [2023-12-27 14:16:59,764][20381] Updated weights for policy 1, policy_version 1245180 (0.0010) [2023-12-27 14:16:59,809][20381] Updated weights for policy 1, policy_version 1245190 (0.0008) [2023-12-27 14:16:59,872][20381] Updated weights for policy 1, policy_version 1245200 (0.0006) [2023-12-27 14:17:00,272][20347] Updated weights for policy 0, policy_version 1246314 (0.0008) [2023-12-27 14:17:00,321][20347] Updated weights for policy 0, policy_version 1246324 (0.0007) [2023-12-27 14:17:00,379][20347] Updated weights for policy 0, policy_version 1246334 (0.0008) [2023-12-27 14:17:00,571][20381] Updated weights for policy 1, policy_version 1245210 (0.0008) [2023-12-27 14:17:00,630][20292] KL-divergence is very high: 281.9775 [2023-12-27 14:17:00,631][20381] Updated weights for policy 1, policy_version 1245220 (0.0008) [2023-12-27 14:17:00,673][20292] KL-divergence is very high: 121.3821 [2023-12-27 14:17:00,685][20381] Updated weights for policy 1, policy_version 1245230 (0.0010) [2023-12-27 14:17:00,736][20381] Updated weights for policy 1, policy_version 1245240 (0.0010) [2023-12-27 14:17:01,132][20347] Updated weights for policy 0, policy_version 1246344 (0.0008) [2023-12-27 14:17:01,197][20347] Updated weights for policy 0, policy_version 1246354 (0.0008) [2023-12-27 14:17:01,253][20347] Updated weights for policy 0, policy_version 1246364 (0.0007) [2023-12-27 14:17:01,444][20381] Updated weights for policy 1, policy_version 1245250 (0.0011) [2023-12-27 14:17:01,499][20381] Updated weights for policy 1, policy_version 1245260 (0.0009) [2023-12-27 14:17:01,558][20381] Updated weights for policy 1, policy_version 1245270 (0.0005) [2023-12-27 14:17:01,923][20347] Updated weights for policy 0, policy_version 1246374 (0.0007) [2023-12-27 14:17:01,980][20347] Updated weights for policy 0, policy_version 1246384 (0.0007) [2023-12-27 14:17:02,036][20347] Updated weights for policy 0, policy_version 1246394 (0.0007) [2023-12-27 14:17:02,221][20381] Updated weights for policy 1, policy_version 1245280 (0.0007) [2023-12-27 14:17:02,280][20381] Updated weights for policy 1, policy_version 1245290 (0.0006) [2023-12-27 14:17:02,336][20381] Updated weights for policy 1, policy_version 1245300 (0.0007) [2023-12-27 14:17:02,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 637960192. Throughput: 0: 10294.8, 1: 10330.1. Samples: 637965224. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:02,364][19170] Avg episode reward: [(0, '746.710'), (1, '980.470')] [2023-12-27 14:17:02,751][20347] Updated weights for policy 0, policy_version 1246404 (0.0008) [2023-12-27 14:17:02,811][20347] Updated weights for policy 0, policy_version 1246414 (0.0007) [2023-12-27 14:17:02,829][20291] KL-divergence is very high: 178.6854 [2023-12-27 14:17:02,867][20347] Updated weights for policy 0, policy_version 1246424 (0.0007) [2023-12-27 14:17:03,023][20381] Updated weights for policy 1, policy_version 1245310 (0.0010) [2023-12-27 14:17:03,071][20381] Updated weights for policy 1, policy_version 1245320 (0.0010) [2023-12-27 14:17:03,115][20381] Updated weights for policy 1, policy_version 1245330 (0.0010) [2023-12-27 14:17:03,527][20347] Updated weights for policy 0, policy_version 1246434 (0.0008) [2023-12-27 14:17:03,570][20347] Updated weights for policy 0, policy_version 1246444 (0.0008) [2023-12-27 14:17:03,625][20347] Updated weights for policy 0, policy_version 1246454 (0.0005) [2023-12-27 14:17:03,674][20347] Updated weights for policy 0, policy_version 1246464 (0.0007) [2023-12-27 14:17:03,813][20381] Updated weights for policy 1, policy_version 1245340 (0.0011) [2023-12-27 14:17:03,879][20381] Updated weights for policy 1, policy_version 1245350 (0.0011) [2023-12-27 14:17:03,935][20381] Updated weights for policy 1, policy_version 1245360 (0.0010) [2023-12-27 14:17:04,318][20347] Updated weights for policy 0, policy_version 1246474 (0.0011) [2023-12-27 14:17:04,376][20347] Updated weights for policy 0, policy_version 1246484 (0.0011) [2023-12-27 14:17:04,428][20347] Updated weights for policy 0, policy_version 1246494 (0.0011) [2023-12-27 14:17:04,621][20381] Updated weights for policy 1, policy_version 1245370 (0.0010) [2023-12-27 14:17:04,676][20381] Updated weights for policy 1, policy_version 1245380 (0.0010) [2023-12-27 14:17:04,727][20381] Updated weights for policy 1, policy_version 1245390 (0.0010) [2023-12-27 14:17:04,772][20381] Updated weights for policy 1, policy_version 1245400 (0.0010) [2023-12-27 14:17:05,038][20347] Updated weights for policy 0, policy_version 1246504 (0.0010) [2023-12-27 14:17:05,095][20347] Updated weights for policy 0, policy_version 1246514 (0.0009) [2023-12-27 14:17:05,150][20347] Updated weights for policy 0, policy_version 1246524 (0.0006) [2023-12-27 14:17:05,421][20381] Updated weights for policy 1, policy_version 1245410 (0.0005) [2023-12-27 14:17:05,476][20381] Updated weights for policy 1, policy_version 1245420 (0.0006) [2023-12-27 14:17:05,542][20381] Updated weights for policy 1, policy_version 1245430 (0.0007) [2023-12-27 14:17:05,765][20347] Updated weights for policy 0, policy_version 1246534 (0.0006) [2023-12-27 14:17:05,824][20347] Updated weights for policy 0, policy_version 1246544 (0.0005) [2023-12-27 14:17:05,870][20347] Updated weights for policy 0, policy_version 1246554 (0.0005) [2023-12-27 14:17:06,185][20381] Updated weights for policy 1, policy_version 1245440 (0.0006) [2023-12-27 14:17:06,245][20381] Updated weights for policy 1, policy_version 1245450 (0.0007) [2023-12-27 14:17:06,299][20381] Updated weights for policy 1, policy_version 1245460 (0.0009) [2023-12-27 14:17:06,528][20347] Updated weights for policy 0, policy_version 1246564 (0.0007) [2023-12-27 14:17:06,576][20347] Updated weights for policy 0, policy_version 1246574 (0.0008) [2023-12-27 14:17:06,626][20347] Updated weights for policy 0, policy_version 1246584 (0.0008) [2023-12-27 14:17:06,928][20381] Updated weights for policy 1, policy_version 1245470 (0.0011) [2023-12-27 14:17:06,978][20381] Updated weights for policy 1, policy_version 1245480 (0.0011) [2023-12-27 14:17:07,034][20381] Updated weights for policy 1, policy_version 1245490 (0.0006) [2023-12-27 14:17:07,303][20347] Updated weights for policy 0, policy_version 1246594 (0.0008) [2023-12-27 14:17:07,352][20347] Updated weights for policy 0, policy_version 1246604 (0.0007) [2023-12-27 14:17:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 638058496. Throughput: 0: 10328.6, 1: 10309.1. Samples: 638027724. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:07,363][19170] Avg episode reward: [(0, '766.110'), (1, '980.500')] [2023-12-27 14:17:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001245496_318873600.pth... [2023-12-27 14:17:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001244280_318562304.pth [2023-12-27 14:17:07,404][20347] Updated weights for policy 0, policy_version 1246614 (0.0005) [2023-12-27 14:17:07,457][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001246624_319193088.pth... [2023-12-27 14:17:07,457][20347] Updated weights for policy 0, policy_version 1246624 (0.0005) [2023-12-27 14:17:07,460][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001245408_318881792.pth [2023-12-27 14:17:07,649][20381] Updated weights for policy 1, policy_version 1245500 (0.0006) [2023-12-27 14:17:07,716][20381] Updated weights for policy 1, policy_version 1245510 (0.0006) [2023-12-27 14:17:07,777][20381] Updated weights for policy 1, policy_version 1245520 (0.0006) [2023-12-27 14:17:08,099][20347] Updated weights for policy 0, policy_version 1246634 (0.0005) [2023-12-27 14:17:08,149][20347] Updated weights for policy 0, policy_version 1246644 (0.0005) [2023-12-27 14:17:08,197][20347] Updated weights for policy 0, policy_version 1246654 (0.0007) [2023-12-27 14:17:08,387][20381] Updated weights for policy 1, policy_version 1245530 (0.0008) [2023-12-27 14:17:08,450][20381] Updated weights for policy 1, policy_version 1245540 (0.0008) [2023-12-27 14:17:08,507][20381] Updated weights for policy 1, policy_version 1245550 (0.0008) [2023-12-27 14:17:08,566][20381] Updated weights for policy 1, policy_version 1245560 (0.0008) [2023-12-27 14:17:08,863][20347] Updated weights for policy 0, policy_version 1246664 (0.0007) [2023-12-27 14:17:08,929][20347] Updated weights for policy 0, policy_version 1246674 (0.0007) [2023-12-27 14:17:08,983][20347] Updated weights for policy 0, policy_version 1246684 (0.0007) [2023-12-27 14:17:09,239][20381] Updated weights for policy 1, policy_version 1245571 (0.0007) [2023-12-27 14:17:09,297][20381] Updated weights for policy 1, policy_version 1245581 (0.0008) [2023-12-27 14:17:09,363][20381] Updated weights for policy 1, policy_version 1245591 (0.0008) [2023-12-27 14:17:09,716][20347] Updated weights for policy 0, policy_version 1246694 (0.0006) [2023-12-27 14:17:09,767][20347] Updated weights for policy 0, policy_version 1246704 (0.0006) [2023-12-27 14:17:09,829][20347] Updated weights for policy 0, policy_version 1246714 (0.0008) [2023-12-27 14:17:10,114][20381] Updated weights for policy 1, policy_version 1245601 (0.0010) [2023-12-27 14:17:10,175][20381] Updated weights for policy 1, policy_version 1245611 (0.0011) [2023-12-27 14:17:10,238][20381] Updated weights for policy 1, policy_version 1245621 (0.0011) [2023-12-27 14:17:10,547][20347] Updated weights for policy 0, policy_version 1246724 (0.0007) [2023-12-27 14:17:10,596][20347] Updated weights for policy 0, policy_version 1246734 (0.0008) [2023-12-27 14:17:10,651][20347] Updated weights for policy 0, policy_version 1246744 (0.0008) [2023-12-27 14:17:10,875][20381] Updated weights for policy 1, policy_version 1245631 (0.0009) [2023-12-27 14:17:10,927][20292] KL-divergence is very high: 115.3978 [2023-12-27 14:17:10,931][20381] Updated weights for policy 1, policy_version 1245641 (0.0008) [2023-12-27 14:17:10,958][20292] KL-divergence is very high: 144.7850 [2023-12-27 14:17:10,993][20381] Updated weights for policy 1, policy_version 1245651 (0.0007) [2023-12-27 14:17:11,425][20347] Updated weights for policy 0, policy_version 1246754 (0.0008) [2023-12-27 14:17:11,484][20347] Updated weights for policy 0, policy_version 1246764 (0.0008) [2023-12-27 14:17:11,537][20347] Updated weights for policy 0, policy_version 1246774 (0.0008) [2023-12-27 14:17:11,605][20347] Updated weights for policy 0, policy_version 1246784 (0.0008) [2023-12-27 14:17:11,758][20381] Updated weights for policy 1, policy_version 1245661 (0.0007) [2023-12-27 14:17:11,815][20381] Updated weights for policy 1, policy_version 1245671 (0.0007) [2023-12-27 14:17:11,878][20381] Updated weights for policy 1, policy_version 1245681 (0.0005) [2023-12-27 14:17:12,322][20347] Updated weights for policy 0, policy_version 1246794 (0.0008) [2023-12-27 14:17:12,362][19170] Fps is (10 sec: 19661.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 638156800. Throughput: 0: 10337.9, 1: 10340.8. Samples: 638151972. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:12,363][19170] Avg episode reward: [(0, '844.500'), (1, '961.020')] [2023-12-27 14:17:12,383][20347] Updated weights for policy 0, policy_version 1246804 (0.0009) [2023-12-27 14:17:12,438][20347] Updated weights for policy 0, policy_version 1246814 (0.0007) [2023-12-27 14:17:12,522][20381] Updated weights for policy 1, policy_version 1245691 (0.0007) [2023-12-27 14:17:12,583][20381] Updated weights for policy 1, policy_version 1245701 (0.0008) [2023-12-27 14:17:12,636][20381] Updated weights for policy 1, policy_version 1245711 (0.0008) [2023-12-27 14:17:13,136][20347] Updated weights for policy 0, policy_version 1246824 (0.0008) [2023-12-27 14:17:13,193][20347] Updated weights for policy 0, policy_version 1246834 (0.0008) [2023-12-27 14:17:13,242][20347] Updated weights for policy 0, policy_version 1246844 (0.0007) [2023-12-27 14:17:13,298][20381] Updated weights for policy 1, policy_version 1245721 (0.0008) [2023-12-27 14:17:13,351][20381] Updated weights for policy 1, policy_version 1245731 (0.0007) [2023-12-27 14:17:13,403][20381] Updated weights for policy 1, policy_version 1245741 (0.0006) [2023-12-27 14:17:13,449][20381] Updated weights for policy 1, policy_version 1245751 (0.0005) [2023-12-27 14:17:13,955][20347] Updated weights for policy 0, policy_version 1246854 (0.0007) [2023-12-27 14:17:14,008][20347] Updated weights for policy 0, policy_version 1246864 (0.0007) [2023-12-27 14:17:14,043][20381] Updated weights for policy 1, policy_version 1245761 (0.0006) [2023-12-27 14:17:14,063][20347] Updated weights for policy 0, policy_version 1246874 (0.0007) [2023-12-27 14:17:14,098][20381] Updated weights for policy 1, policy_version 1245771 (0.0008) [2023-12-27 14:17:14,150][20381] Updated weights for policy 1, policy_version 1245781 (0.0005) [2023-12-27 14:17:14,784][20347] Updated weights for policy 0, policy_version 1246884 (0.0006) [2023-12-27 14:17:14,806][20381] Updated weights for policy 1, policy_version 1245791 (0.0008) [2023-12-27 14:17:14,859][20347] Updated weights for policy 0, policy_version 1246894 (0.0007) [2023-12-27 14:17:14,871][20381] Updated weights for policy 1, policy_version 1245801 (0.0007) [2023-12-27 14:17:14,918][20347] Updated weights for policy 0, policy_version 1246904 (0.0007) [2023-12-27 14:17:14,928][20381] Updated weights for policy 1, policy_version 1245811 (0.0006) [2023-12-27 14:17:15,576][20347] Updated weights for policy 0, policy_version 1246914 (0.0006) [2023-12-27 14:17:15,635][20347] Updated weights for policy 0, policy_version 1246924 (0.0008) [2023-12-27 14:17:15,666][20381] Updated weights for policy 1, policy_version 1245821 (0.0006) [2023-12-27 14:17:15,699][20347] Updated weights for policy 0, policy_version 1246934 (0.0006) [2023-12-27 14:17:15,724][20381] Updated weights for policy 1, policy_version 1245831 (0.0006) [2023-12-27 14:17:15,748][20347] Updated weights for policy 0, policy_version 1246944 (0.0005) [2023-12-27 14:17:15,780][20381] Updated weights for policy 1, policy_version 1245841 (0.0006) [2023-12-27 14:17:16,434][20381] Updated weights for policy 1, policy_version 1245851 (0.0008) [2023-12-27 14:17:16,442][20347] Updated weights for policy 0, policy_version 1246954 (0.0008) [2023-12-27 14:17:16,482][20381] Updated weights for policy 1, policy_version 1245861 (0.0006) [2023-12-27 14:17:16,491][20347] Updated weights for policy 0, policy_version 1246964 (0.0008) [2023-12-27 14:17:16,538][20381] Updated weights for policy 1, policy_version 1245871 (0.0006) [2023-12-27 14:17:16,540][20347] Updated weights for policy 0, policy_version 1246974 (0.0005) [2023-12-27 14:17:17,238][20347] Updated weights for policy 0, policy_version 1246984 (0.0007) [2023-12-27 14:17:17,251][20381] Updated weights for policy 1, policy_version 1245881 (0.0007) [2023-12-27 14:17:17,288][20347] Updated weights for policy 0, policy_version 1246994 (0.0006) [2023-12-27 14:17:17,308][20381] Updated weights for policy 1, policy_version 1245891 (0.0007) [2023-12-27 14:17:17,340][20347] Updated weights for policy 0, policy_version 1247004 (0.0005) [2023-12-27 14:17:17,361][20381] Updated weights for policy 1, policy_version 1245901 (0.0006) [2023-12-27 14:17:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 638255104. Throughput: 0: 10272.9, 1: 10294.9. Samples: 638272772. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:17,363][19170] Avg episode reward: [(0, '834.390'), (1, '941.400')] [2023-12-27 14:17:17,413][20381] Updated weights for policy 1, policy_version 1245911 (0.0008) [2023-12-27 14:17:18,019][20347] Updated weights for policy 0, policy_version 1247014 (0.0007) [2023-12-27 14:17:18,083][20347] Updated weights for policy 0, policy_version 1247024 (0.0008) [2023-12-27 14:17:18,122][20381] Updated weights for policy 1, policy_version 1245921 (0.0006) [2023-12-27 14:17:18,140][20347] Updated weights for policy 0, policy_version 1247034 (0.0007) [2023-12-27 14:17:18,164][20381] Updated weights for policy 1, policy_version 1245931 (0.0006) [2023-12-27 14:17:18,221][20381] Updated weights for policy 1, policy_version 1245941 (0.0007) [2023-12-27 14:17:18,788][20347] Updated weights for policy 0, policy_version 1247044 (0.0006) [2023-12-27 14:17:18,844][20347] Updated weights for policy 0, policy_version 1247054 (0.0007) [2023-12-27 14:17:18,875][20381] Updated weights for policy 1, policy_version 1245951 (0.0010) [2023-12-27 14:17:18,897][20347] Updated weights for policy 0, policy_version 1247064 (0.0005) [2023-12-27 14:17:18,932][20381] Updated weights for policy 1, policy_version 1245961 (0.0011) [2023-12-27 14:17:18,980][20381] Updated weights for policy 1, policy_version 1245971 (0.0011) [2023-12-27 14:17:19,626][20347] Updated weights for policy 0, policy_version 1247074 (0.0007) [2023-12-27 14:17:19,689][20347] Updated weights for policy 0, policy_version 1247084 (0.0008) [2023-12-27 14:17:19,730][20381] Updated weights for policy 1, policy_version 1245981 (0.0011) [2023-12-27 14:17:19,731][20291] KL-divergence is very high: 107.7449 [2023-12-27 14:17:19,742][20347] Updated weights for policy 0, policy_version 1247094 (0.0008) [2023-12-27 14:17:19,788][20381] Updated weights for policy 1, policy_version 1245991 (0.0011) [2023-12-27 14:17:19,798][20347] Updated weights for policy 0, policy_version 1247104 (0.0009) [2023-12-27 14:17:19,854][20381] Updated weights for policy 1, policy_version 1246001 (0.0010) [2023-12-27 14:17:20,473][20381] Updated weights for policy 1, policy_version 1246011 (0.0008) [2023-12-27 14:17:20,532][20347] Updated weights for policy 0, policy_version 1247114 (0.0006) [2023-12-27 14:17:20,534][20381] Updated weights for policy 1, policy_version 1246021 (0.0008) [2023-12-27 14:17:20,595][20347] Updated weights for policy 0, policy_version 1247124 (0.0008) [2023-12-27 14:17:20,597][20381] Updated weights for policy 1, policy_version 1246031 (0.0006) [2023-12-27 14:17:20,656][20347] Updated weights for policy 0, policy_version 1247134 (0.0007) [2023-12-27 14:17:21,308][20381] Updated weights for policy 1, policy_version 1246041 (0.0006) [2023-12-27 14:17:21,392][20381] Updated weights for policy 1, policy_version 1246051 (0.0009) [2023-12-27 14:17:21,416][20347] Updated weights for policy 0, policy_version 1247144 (0.0008) [2023-12-27 14:17:21,450][20381] Updated weights for policy 1, policy_version 1246061 (0.0009) [2023-12-27 14:17:21,481][20347] Updated weights for policy 0, policy_version 1247154 (0.0008) [2023-12-27 14:17:21,506][20381] Updated weights for policy 1, policy_version 1246071 (0.0005) [2023-12-27 14:17:21,546][20347] Updated weights for policy 0, policy_version 1247164 (0.0008) [2023-12-27 14:17:22,220][20381] Updated weights for policy 1, policy_version 1246081 (0.0005) [2023-12-27 14:17:22,255][20347] Updated weights for policy 0, policy_version 1247174 (0.0007) [2023-12-27 14:17:22,284][20381] Updated weights for policy 1, policy_version 1246091 (0.0006) [2023-12-27 14:17:22,314][20347] Updated weights for policy 0, policy_version 1247184 (0.0010) [2023-12-27 14:17:22,355][20381] Updated weights for policy 1, policy_version 1246101 (0.0006) [2023-12-27 14:17:22,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 638353408. Throughput: 0: 10276.9, 1: 10294.5. Samples: 638333152. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:22,363][19170] Avg episode reward: [(0, '824.780'), (1, '854.400')] [2023-12-27 14:17:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001246104_319029248.pth... [2023-12-27 14:17:22,375][20347] Updated weights for policy 0, policy_version 1247194 (0.0010) [2023-12-27 14:17:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001244888_318717952.pth [2023-12-27 14:17:22,406][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001247200_319340544.pth... [2023-12-27 14:17:22,409][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001246016_319037440.pth [2023-12-27 14:17:22,994][20381] Updated weights for policy 1, policy_version 1246111 (0.0008) [2023-12-27 14:17:23,045][20381] Updated weights for policy 1, policy_version 1246121 (0.0008) [2023-12-27 14:17:23,094][20381] Updated weights for policy 1, policy_version 1246131 (0.0008) [2023-12-27 14:17:23,164][20347] Updated weights for policy 0, policy_version 1247204 (0.0008) [2023-12-27 14:17:23,186][20291] KL-divergence is very high: 148.6100 [2023-12-27 14:17:23,224][20347] Updated weights for policy 0, policy_version 1247214 (0.0007) [2023-12-27 14:17:23,281][20347] Updated weights for policy 0, policy_version 1247224 (0.0008) [2023-12-27 14:17:23,785][20381] Updated weights for policy 1, policy_version 1246141 (0.0008) [2023-12-27 14:17:23,840][20381] Updated weights for policy 1, policy_version 1246151 (0.0008) [2023-12-27 14:17:23,889][20381] Updated weights for policy 1, policy_version 1246161 (0.0008) [2023-12-27 14:17:23,932][20347] Updated weights for policy 0, policy_version 1247234 (0.0008) [2023-12-27 14:17:23,986][20347] Updated weights for policy 0, policy_version 1247244 (0.0007) [2023-12-27 14:17:24,041][20347] Updated weights for policy 0, policy_version 1247254 (0.0007) [2023-12-27 14:17:24,096][20347] Updated weights for policy 0, policy_version 1247264 (0.0008) [2023-12-27 14:17:24,561][20381] Updated weights for policy 1, policy_version 1246171 (0.0008) [2023-12-27 14:17:24,622][20381] Updated weights for policy 1, policy_version 1246181 (0.0008) [2023-12-27 14:17:24,675][20381] Updated weights for policy 1, policy_version 1246191 (0.0010) [2023-12-27 14:17:24,810][20347] Updated weights for policy 0, policy_version 1247274 (0.0006) [2023-12-27 14:17:24,859][20347] Updated weights for policy 0, policy_version 1247284 (0.0007) [2023-12-27 14:17:24,915][20347] Updated weights for policy 0, policy_version 1247294 (0.0007) [2023-12-27 14:17:25,290][20381] Updated weights for policy 1, policy_version 1246201 (0.0010) [2023-12-27 14:17:25,338][20381] Updated weights for policy 1, policy_version 1246211 (0.0005) [2023-12-27 14:17:25,388][20381] Updated weights for policy 1, policy_version 1246221 (0.0005) [2023-12-27 14:17:25,440][20381] Updated weights for policy 1, policy_version 1246231 (0.0010) [2023-12-27 14:17:25,608][20347] Updated weights for policy 0, policy_version 1247304 (0.0007) [2023-12-27 14:17:25,660][20291] KL-divergence is very high: 163.0295 [2023-12-27 14:17:25,664][20347] Updated weights for policy 0, policy_version 1247314 (0.0007) [2023-12-27 14:17:25,666][20291] KL-divergence is very high: 166.2413 [2023-12-27 14:17:25,702][20291] KL-divergence is very high: 110.7180 [2023-12-27 14:17:25,716][20347] Updated weights for policy 0, policy_version 1247324 (0.0007) [2023-12-27 14:17:26,101][20381] Updated weights for policy 1, policy_version 1246241 (0.0007) [2023-12-27 14:17:26,165][20381] Updated weights for policy 1, policy_version 1246251 (0.0005) [2023-12-27 14:17:26,228][20381] Updated weights for policy 1, policy_version 1246261 (0.0006) [2023-12-27 14:17:26,308][20347] Updated weights for policy 0, policy_version 1247334 (0.0009) [2023-12-27 14:17:26,363][20347] Updated weights for policy 0, policy_version 1247344 (0.0008) [2023-12-27 14:17:26,418][20347] Updated weights for policy 0, policy_version 1247354 (0.0005) [2023-12-27 14:17:26,849][20381] Updated weights for policy 1, policy_version 1246271 (0.0007) [2023-12-27 14:17:26,906][20381] Updated weights for policy 1, policy_version 1246281 (0.0011) [2023-12-27 14:17:26,957][20381] Updated weights for policy 1, policy_version 1246291 (0.0010) [2023-12-27 14:17:27,060][20347] Updated weights for policy 0, policy_version 1247364 (0.0007) [2023-12-27 14:17:27,105][20347] Updated weights for policy 0, policy_version 1247374 (0.0010) [2023-12-27 14:17:27,149][20347] Updated weights for policy 0, policy_version 1247384 (0.0010) [2023-12-27 14:17:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 638468096. Throughput: 0: 10139.3, 1: 10328.9. Samples: 638453756. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:27,363][19170] Avg episode reward: [(0, '844.550'), (1, '845.100')] [2023-12-27 14:17:27,609][20381] Updated weights for policy 1, policy_version 1246301 (0.0011) [2023-12-27 14:17:27,666][20381] Updated weights for policy 1, policy_version 1246311 (0.0010) [2023-12-27 14:17:27,710][20381] Updated weights for policy 1, policy_version 1246321 (0.0010) [2023-12-27 14:17:27,736][20347] Updated weights for policy 0, policy_version 1247394 (0.0010) [2023-12-27 14:17:27,787][20347] Updated weights for policy 0, policy_version 1247404 (0.0010) [2023-12-27 14:17:27,832][20347] Updated weights for policy 0, policy_version 1247414 (0.0005) [2023-12-27 14:17:27,882][20347] Updated weights for policy 0, policy_version 1247424 (0.0006) [2023-12-27 14:17:28,317][20381] Updated weights for policy 1, policy_version 1246331 (0.0008) [2023-12-27 14:17:28,382][20381] Updated weights for policy 1, policy_version 1246341 (0.0009) [2023-12-27 14:17:28,434][20381] Updated weights for policy 1, policy_version 1246351 (0.0007) [2023-12-27 14:17:28,582][20347] Updated weights for policy 0, policy_version 1247434 (0.0010) [2023-12-27 14:17:28,641][20347] Updated weights for policy 0, policy_version 1247444 (0.0010) [2023-12-27 14:17:28,702][20347] Updated weights for policy 0, policy_version 1247454 (0.0007) [2023-12-27 14:17:29,047][20381] Updated weights for policy 1, policy_version 1246361 (0.0007) [2023-12-27 14:17:29,104][20381] Updated weights for policy 1, policy_version 1246371 (0.0010) [2023-12-27 14:17:29,157][20381] Updated weights for policy 1, policy_version 1246381 (0.0009) [2023-12-27 14:17:29,209][20381] Updated weights for policy 1, policy_version 1246391 (0.0007) [2023-12-27 14:17:29,414][20347] Updated weights for policy 0, policy_version 1247464 (0.0010) [2023-12-27 14:17:29,469][20347] Updated weights for policy 0, policy_version 1247474 (0.0010) [2023-12-27 14:17:29,526][20347] Updated weights for policy 0, policy_version 1247484 (0.0010) [2023-12-27 14:17:29,930][20381] Updated weights for policy 1, policy_version 1246401 (0.0008) [2023-12-27 14:17:30,000][20381] Updated weights for policy 1, policy_version 1246411 (0.0009) [2023-12-27 14:17:30,055][20381] Updated weights for policy 1, policy_version 1246421 (0.0007) [2023-12-27 14:17:30,281][20347] Updated weights for policy 0, policy_version 1247494 (0.0008) [2023-12-27 14:17:30,328][20347] Updated weights for policy 0, policy_version 1247504 (0.0008) [2023-12-27 14:17:30,383][20347] Updated weights for policy 0, policy_version 1247514 (0.0008) [2023-12-27 14:17:30,723][20381] Updated weights for policy 1, policy_version 1246431 (0.0007) [2023-12-27 14:17:30,779][20381] Updated weights for policy 1, policy_version 1246441 (0.0007) [2023-12-27 14:17:30,829][20381] Updated weights for policy 1, policy_version 1246451 (0.0005) [2023-12-27 14:17:31,077][20347] Updated weights for policy 0, policy_version 1247524 (0.0009) [2023-12-27 14:17:31,140][20347] Updated weights for policy 0, policy_version 1247534 (0.0010) [2023-12-27 14:17:31,197][20347] Updated weights for policy 0, policy_version 1247544 (0.0007) [2023-12-27 14:17:31,508][20381] Updated weights for policy 1, policy_version 1246461 (0.0009) [2023-12-27 14:17:31,572][20381] Updated weights for policy 1, policy_version 1246471 (0.0007) [2023-12-27 14:17:31,635][20381] Updated weights for policy 1, policy_version 1246481 (0.0007) [2023-12-27 14:17:31,871][20347] Updated weights for policy 0, policy_version 1247554 (0.0006) [2023-12-27 14:17:31,934][20347] Updated weights for policy 0, policy_version 1247564 (0.0006) [2023-12-27 14:17:31,991][20347] Updated weights for policy 0, policy_version 1247574 (0.0008) [2023-12-27 14:17:32,049][20347] Updated weights for policy 0, policy_version 1247584 (0.0008) [2023-12-27 14:17:32,293][20381] Updated weights for policy 1, policy_version 1246491 (0.0007) [2023-12-27 14:17:32,345][20381] Updated weights for policy 1, policy_version 1246501 (0.0007) [2023-12-27 14:17:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 638566400. Throughput: 0: 10101.5, 1: 10327.7. Samples: 638578980. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:32,363][19170] Avg episode reward: [(0, '729.030'), (1, '854.500')] [2023-12-27 14:17:32,414][20381] Updated weights for policy 1, policy_version 1246511 (0.0008) [2023-12-27 14:17:32,736][20347] Updated weights for policy 0, policy_version 1247594 (0.0007) [2023-12-27 14:17:32,787][20347] Updated weights for policy 0, policy_version 1247604 (0.0007) [2023-12-27 14:17:32,842][20347] Updated weights for policy 0, policy_version 1247614 (0.0007) [2023-12-27 14:17:33,090][20381] Updated weights for policy 1, policy_version 1246521 (0.0008) [2023-12-27 14:17:33,137][20381] Updated weights for policy 1, policy_version 1246531 (0.0007) [2023-12-27 14:17:33,182][20381] Updated weights for policy 1, policy_version 1246541 (0.0007) [2023-12-27 14:17:33,227][20381] Updated weights for policy 1, policy_version 1246551 (0.0008) [2023-12-27 14:17:33,514][20347] Updated weights for policy 0, policy_version 1247624 (0.0006) [2023-12-27 14:17:33,557][20347] Updated weights for policy 0, policy_version 1247634 (0.0005) [2023-12-27 14:17:33,610][20347] Updated weights for policy 0, policy_version 1247644 (0.0005) [2023-12-27 14:17:33,887][20381] Updated weights for policy 1, policy_version 1246561 (0.0007) [2023-12-27 14:17:33,928][20381] Updated weights for policy 1, policy_version 1246571 (0.0007) [2023-12-27 14:17:33,973][20381] Updated weights for policy 1, policy_version 1246581 (0.0007) [2023-12-27 14:17:34,215][20347] Updated weights for policy 0, policy_version 1247654 (0.0006) [2023-12-27 14:17:34,279][20347] Updated weights for policy 0, policy_version 1247664 (0.0007) [2023-12-27 14:17:34,343][20347] Updated weights for policy 0, policy_version 1247674 (0.0006) [2023-12-27 14:17:34,734][20381] Updated weights for policy 1, policy_version 1246591 (0.0007) [2023-12-27 14:17:34,793][20381] Updated weights for policy 1, policy_version 1246601 (0.0008) [2023-12-27 14:17:34,849][20381] Updated weights for policy 1, policy_version 1246611 (0.0008) [2023-12-27 14:17:34,997][20347] Updated weights for policy 0, policy_version 1247684 (0.0007) [2023-12-27 14:17:35,050][20347] Updated weights for policy 0, policy_version 1247694 (0.0007) [2023-12-27 14:17:35,095][20347] Updated weights for policy 0, policy_version 1247704 (0.0008) [2023-12-27 14:17:35,552][20381] Updated weights for policy 1, policy_version 1246621 (0.0008) [2023-12-27 14:17:35,612][20381] Updated weights for policy 1, policy_version 1246631 (0.0007) [2023-12-27 14:17:35,675][20381] Updated weights for policy 1, policy_version 1246641 (0.0008) [2023-12-27 14:17:35,761][20347] Updated weights for policy 0, policy_version 1247714 (0.0007) [2023-12-27 14:17:35,812][20347] Updated weights for policy 0, policy_version 1247724 (0.0005) [2023-12-27 14:17:35,863][20347] Updated weights for policy 0, policy_version 1247734 (0.0007) [2023-12-27 14:17:35,914][20347] Updated weights for policy 0, policy_version 1247744 (0.0008) [2023-12-27 14:17:36,333][20381] Updated weights for policy 1, policy_version 1246651 (0.0008) [2023-12-27 14:17:36,395][20381] Updated weights for policy 1, policy_version 1246661 (0.0007) [2023-12-27 14:17:36,460][20381] Updated weights for policy 1, policy_version 1246671 (0.0008) [2023-12-27 14:17:36,561][20347] Updated weights for policy 0, policy_version 1247754 (0.0005) [2023-12-27 14:17:36,575][20291] KL-divergence is very high: 156.1195 [2023-12-27 14:17:36,622][20347] Updated weights for policy 0, policy_version 1247764 (0.0008) [2023-12-27 14:17:36,624][20291] KL-divergence is very high: 202.3409 [2023-12-27 14:17:36,670][20291] KL-divergence is very high: 125.0730 [2023-12-27 14:17:36,682][20347] Updated weights for policy 0, policy_version 1247774 (0.0008) [2023-12-27 14:17:37,166][20381] Updated weights for policy 1, policy_version 1246681 (0.0007) [2023-12-27 14:17:37,211][20292] KL-divergence is very high: 316.8234 [2023-12-27 14:17:37,216][20381] Updated weights for policy 1, policy_version 1246691 (0.0008) [2023-12-27 14:17:37,249][20292] KL-divergence is very high: 388.0711 [2023-12-27 14:17:37,265][20381] Updated weights for policy 1, policy_version 1246701 (0.0008) [2023-12-27 14:17:37,297][20292] KL-divergence is very high: 163.0316 [2023-12-27 14:17:37,324][20381] Updated weights for policy 1, policy_version 1246711 (0.0007) [2023-12-27 14:17:37,345][20347] Updated weights for policy 0, policy_version 1247784 (0.0007) [2023-12-27 14:17:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 638672896. Throughput: 0: 10158.1, 1: 10277.6. Samples: 638641168. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:37,363][19170] Avg episode reward: [(0, '814.730'), (1, '874.670')] [2023-12-27 14:17:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001246712_319184896.pth... [2023-12-27 14:17:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001245496_318873600.pth [2023-12-27 14:17:37,402][20347] Updated weights for policy 0, policy_version 1247794 (0.0008) [2023-12-27 14:17:37,458][20347] Updated weights for policy 0, policy_version 1247804 (0.0007) [2023-12-27 14:17:37,478][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001247808_319496192.pth... [2023-12-27 14:17:37,482][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001246624_319193088.pth [2023-12-27 14:17:38,017][20381] Updated weights for policy 1, policy_version 1246721 (0.0008) [2023-12-27 14:17:38,077][20381] Updated weights for policy 1, policy_version 1246731 (0.0008) [2023-12-27 14:17:38,139][20381] Updated weights for policy 1, policy_version 1246741 (0.0009) [2023-12-27 14:17:38,164][20347] Updated weights for policy 0, policy_version 1247814 (0.0006) [2023-12-27 14:17:38,215][20347] Updated weights for policy 0, policy_version 1247824 (0.0005) [2023-12-27 14:17:38,266][20347] Updated weights for policy 0, policy_version 1247834 (0.0005) [2023-12-27 14:17:38,841][20347] Updated weights for policy 0, policy_version 1247844 (0.0006) [2023-12-27 14:17:38,843][20381] Updated weights for policy 1, policy_version 1246751 (0.0008) [2023-12-27 14:17:38,896][20347] Updated weights for policy 0, policy_version 1247854 (0.0006) [2023-12-27 14:17:38,901][20381] Updated weights for policy 1, policy_version 1246761 (0.0007) [2023-12-27 14:17:38,949][20347] Updated weights for policy 0, policy_version 1247864 (0.0006) [2023-12-27 14:17:38,951][20381] Updated weights for policy 1, policy_version 1246771 (0.0005) [2023-12-27 14:17:39,671][20347] Updated weights for policy 0, policy_version 1247874 (0.0007) [2023-12-27 14:17:39,709][20381] Updated weights for policy 1, policy_version 1246781 (0.0007) [2023-12-27 14:17:39,730][20347] Updated weights for policy 0, policy_version 1247884 (0.0009) [2023-12-27 14:17:39,737][20291] KL-divergence is very high: 187.2815 [2023-12-27 14:17:39,743][20291] KL-divergence is very high: 172.2068 [2023-12-27 14:17:39,771][20381] Updated weights for policy 1, policy_version 1246791 (0.0007) [2023-12-27 14:17:39,792][20347] Updated weights for policy 0, policy_version 1247894 (0.0010) [2023-12-27 14:17:39,833][20381] Updated weights for policy 1, policy_version 1246801 (0.0006) [2023-12-27 14:17:39,865][20347] Updated weights for policy 0, policy_version 1247904 (0.0009) [2023-12-27 14:17:40,470][20381] Updated weights for policy 1, policy_version 1246811 (0.0007) [2023-12-27 14:17:40,524][20381] Updated weights for policy 1, policy_version 1246821 (0.0006) [2023-12-27 14:17:40,543][20347] Updated weights for policy 0, policy_version 1247914 (0.0008) [2023-12-27 14:17:40,579][20381] Updated weights for policy 1, policy_version 1246831 (0.0005) [2023-12-27 14:17:40,602][20347] Updated weights for policy 0, policy_version 1247924 (0.0008) [2023-12-27 14:17:40,652][20347] Updated weights for policy 0, policy_version 1247934 (0.0007) [2023-12-27 14:17:41,215][20381] Updated weights for policy 1, policy_version 1246841 (0.0006) [2023-12-27 14:17:41,272][20381] Updated weights for policy 1, policy_version 1246851 (0.0009) [2023-12-27 14:17:41,321][20381] Updated weights for policy 1, policy_version 1246861 (0.0009) [2023-12-27 14:17:41,389][20381] Updated weights for policy 1, policy_version 1246872 (0.0009) [2023-12-27 14:17:41,432][20347] Updated weights for policy 0, policy_version 1247944 (0.0009) [2023-12-27 14:17:41,486][20347] Updated weights for policy 0, policy_version 1247954 (0.0009) [2023-12-27 14:17:41,534][20347] Updated weights for policy 0, policy_version 1247964 (0.0009) [2023-12-27 14:17:42,199][20381] Updated weights for policy 1, policy_version 1246882 (0.0006) [2023-12-27 14:17:42,259][20381] Updated weights for policy 1, policy_version 1246892 (0.0008) [2023-12-27 14:17:42,318][20381] Updated weights for policy 1, policy_version 1246902 (0.0008) [2023-12-27 14:17:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 638771200. Throughput: 0: 10208.1, 1: 10257.5. Samples: 638764360. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 14:17:42,363][19170] Avg episode reward: [(0, '696.950'), (1, '893.230')] [2023-12-27 14:17:42,384][20347] Updated weights for policy 0, policy_version 1247974 (0.0008) [2023-12-27 14:17:42,445][20347] Updated weights for policy 0, policy_version 1247984 (0.0008) [2023-12-27 14:17:42,498][20347] Updated weights for policy 0, policy_version 1247994 (0.0008) [2023-12-27 14:17:42,940][20381] Updated weights for policy 1, policy_version 1246912 (0.0008) [2023-12-27 14:17:42,992][20381] Updated weights for policy 1, policy_version 1246922 (0.0007) [2023-12-27 14:17:43,037][20381] Updated weights for policy 1, policy_version 1246932 (0.0007) [2023-12-27 14:17:43,213][20347] Updated weights for policy 0, policy_version 1248004 (0.0009) [2023-12-27 14:17:43,277][20347] Updated weights for policy 0, policy_version 1248014 (0.0007) [2023-12-27 14:17:43,325][20347] Updated weights for policy 0, policy_version 1248024 (0.0007) [2023-12-27 14:17:43,746][20381] Updated weights for policy 1, policy_version 1246942 (0.0008) [2023-12-27 14:17:43,805][20381] Updated weights for policy 1, policy_version 1246952 (0.0008) [2023-12-27 14:17:43,864][20381] Updated weights for policy 1, policy_version 1246962 (0.0009) [2023-12-27 14:17:43,958][20347] Updated weights for policy 0, policy_version 1248034 (0.0006) [2023-12-27 14:17:44,024][20347] Updated weights for policy 0, policy_version 1248044 (0.0006) [2023-12-27 14:17:44,078][20347] Updated weights for policy 0, policy_version 1248054 (0.0006) [2023-12-27 14:17:44,135][20347] Updated weights for policy 0, policy_version 1248064 (0.0008) [2023-12-27 14:17:44,487][20381] Updated weights for policy 1, policy_version 1246972 (0.0006) [2023-12-27 14:17:44,542][20381] Updated weights for policy 1, policy_version 1246982 (0.0008) [2023-12-27 14:17:44,601][20381] Updated weights for policy 1, policy_version 1246992 (0.0008) [2023-12-27 14:17:44,785][20347] Updated weights for policy 0, policy_version 1248074 (0.0007) [2023-12-27 14:17:44,851][20347] Updated weights for policy 0, policy_version 1248084 (0.0007) [2023-12-27 14:17:44,911][20347] Updated weights for policy 0, policy_version 1248094 (0.0008) [2023-12-27 14:17:45,316][20381] Updated weights for policy 1, policy_version 1247002 (0.0008) [2023-12-27 14:17:45,378][20381] Updated weights for policy 1, policy_version 1247012 (0.0008) [2023-12-27 14:17:45,435][20381] Updated weights for policy 1, policy_version 1247022 (0.0008) [2023-12-27 14:17:45,492][20381] Updated weights for policy 1, policy_version 1247032 (0.0008) [2023-12-27 14:17:45,562][20347] Updated weights for policy 0, policy_version 1248104 (0.0008) [2023-12-27 14:17:45,621][20347] Updated weights for policy 0, policy_version 1248114 (0.0008) [2023-12-27 14:17:45,686][20347] Updated weights for policy 0, policy_version 1248124 (0.0008) [2023-12-27 14:17:46,146][20381] Updated weights for policy 1, policy_version 1247042 (0.0005) [2023-12-27 14:17:46,203][20381] Updated weights for policy 1, policy_version 1247052 (0.0006) [2023-12-27 14:17:46,262][20381] Updated weights for policy 1, policy_version 1247062 (0.0006) [2023-12-27 14:17:46,387][20347] Updated weights for policy 0, policy_version 1248134 (0.0008) [2023-12-27 14:17:46,439][20347] Updated weights for policy 0, policy_version 1248144 (0.0008) [2023-12-27 14:17:46,503][20347] Updated weights for policy 0, policy_version 1248154 (0.0008) [2023-12-27 14:17:46,831][20381] Updated weights for policy 1, policy_version 1247072 (0.0006) [2023-12-27 14:17:46,878][20381] Updated weights for policy 1, policy_version 1247082 (0.0005) [2023-12-27 14:17:46,930][20381] Updated weights for policy 1, policy_version 1247092 (0.0005) [2023-12-27 14:17:47,209][20347] Updated weights for policy 0, policy_version 1248164 (0.0008) [2023-12-27 14:17:47,267][20347] Updated weights for policy 0, policy_version 1248174 (0.0008) [2023-12-27 14:17:47,325][20347] Updated weights for policy 0, policy_version 1248184 (0.0007) [2023-12-27 14:17:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 638869504. Throughput: 0: 10139.5, 1: 10299.8. Samples: 638884988. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:17:47,363][19170] Avg episode reward: [(0, '717.960'), (1, '922.140')] [2023-12-27 14:17:47,524][20381] Updated weights for policy 1, policy_version 1247102 (0.0005) [2023-12-27 14:17:47,577][20381] Updated weights for policy 1, policy_version 1247112 (0.0010) [2023-12-27 14:17:47,627][20381] Updated weights for policy 1, policy_version 1247122 (0.0011) [2023-12-27 14:17:48,017][20347] Updated weights for policy 0, policy_version 1248194 (0.0009) [2023-12-27 14:17:48,084][20347] Updated weights for policy 0, policy_version 1248204 (0.0008) [2023-12-27 14:17:48,144][20347] Updated weights for policy 0, policy_version 1248214 (0.0008) [2023-12-27 14:17:48,187][20381] Updated weights for policy 1, policy_version 1247132 (0.0011) [2023-12-27 14:17:48,197][20347] Updated weights for policy 0, policy_version 1248224 (0.0006) [2023-12-27 14:17:48,241][20381] Updated weights for policy 1, policy_version 1247142 (0.0011) [2023-12-27 14:17:48,294][20381] Updated weights for policy 1, policy_version 1247152 (0.0011) [2023-12-27 14:17:48,908][20347] Updated weights for policy 0, policy_version 1248234 (0.0007) [2023-12-27 14:17:48,958][20347] Updated weights for policy 0, policy_version 1248244 (0.0007) [2023-12-27 14:17:48,960][20381] Updated weights for policy 1, policy_version 1247162 (0.0010) [2023-12-27 14:17:49,006][20381] Updated weights for policy 1, policy_version 1247172 (0.0010) [2023-12-27 14:17:49,007][20347] Updated weights for policy 0, policy_version 1248254 (0.0006) [2023-12-27 14:17:49,053][20381] Updated weights for policy 1, policy_version 1247182 (0.0010) [2023-12-27 14:17:49,109][20381] Updated weights for policy 1, policy_version 1247192 (0.0011) [2023-12-27 14:17:49,729][20347] Updated weights for policy 0, policy_version 1248264 (0.0008) [2023-12-27 14:17:49,789][20347] Updated weights for policy 0, policy_version 1248274 (0.0006) [2023-12-27 14:17:49,836][20381] Updated weights for policy 1, policy_version 1247202 (0.0007) [2023-12-27 14:17:49,857][20347] Updated weights for policy 0, policy_version 1248284 (0.0008) [2023-12-27 14:17:49,892][20381] Updated weights for policy 1, policy_version 1247212 (0.0007) [2023-12-27 14:17:49,961][20381] Updated weights for policy 1, policy_version 1247222 (0.0009) [2023-12-27 14:17:50,510][20347] Updated weights for policy 0, policy_version 1248294 (0.0007) [2023-12-27 14:17:50,570][20347] Updated weights for policy 0, policy_version 1248304 (0.0007) [2023-12-27 14:17:50,630][20347] Updated weights for policy 0, policy_version 1248314 (0.0007) [2023-12-27 14:17:50,745][20381] Updated weights for policy 1, policy_version 1247232 (0.0007) [2023-12-27 14:17:50,805][20381] Updated weights for policy 1, policy_version 1247242 (0.0006) [2023-12-27 14:17:50,863][20381] Updated weights for policy 1, policy_version 1247252 (0.0006) [2023-12-27 14:17:51,367][20347] Updated weights for policy 0, policy_version 1248324 (0.0008) [2023-12-27 14:17:51,436][20347] Updated weights for policy 0, policy_version 1248334 (0.0008) [2023-12-27 14:17:51,498][20347] Updated weights for policy 0, policy_version 1248344 (0.0008) [2023-12-27 14:17:51,540][20381] Updated weights for policy 1, policy_version 1247262 (0.0006) [2023-12-27 14:17:51,603][20381] Updated weights for policy 1, policy_version 1247272 (0.0008) [2023-12-27 14:17:51,679][20381] Updated weights for policy 1, policy_version 1247282 (0.0008) [2023-12-27 14:17:52,204][20347] Updated weights for policy 0, policy_version 1248354 (0.0007) [2023-12-27 14:17:52,269][20347] Updated weights for policy 0, policy_version 1248364 (0.0006) [2023-12-27 14:17:52,331][20347] Updated weights for policy 0, policy_version 1248374 (0.0006) [2023-12-27 14:17:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 638967808. Throughput: 0: 10076.6, 1: 10321.2. Samples: 638945624. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:17:52,363][19170] Avg episode reward: [(0, '697.640'), (1, '932.070')] [2023-12-27 14:17:52,379][20381] Updated weights for policy 1, policy_version 1247292 (0.0009) [2023-12-27 14:17:52,405][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001248384_319643648.pth... [2023-12-27 14:17:52,406][20347] Updated weights for policy 0, policy_version 1248384 (0.0008) [2023-12-27 14:17:52,408][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001247200_319340544.pth [2023-12-27 14:17:52,443][20381] Updated weights for policy 1, policy_version 1247302 (0.0007) [2023-12-27 14:17:52,493][20381] Updated weights for policy 1, policy_version 1247312 (0.0009) [2023-12-27 14:17:52,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001247320_319340544.pth... [2023-12-27 14:17:52,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001246104_319029248.pth [2023-12-27 14:17:53,061][20291] KL-divergence is very high: 139.8652 [2023-12-27 14:17:53,071][20291] KL-divergence is very high: 277.0792 [2023-12-27 14:17:53,075][20347] Updated weights for policy 0, policy_version 1248394 (0.0006) [2023-12-27 14:17:53,080][20291] KL-divergence is very high: 113.9061 [2023-12-27 14:17:53,089][20381] Updated weights for policy 1, policy_version 1247322 (0.0007) [2023-12-27 14:17:53,102][20291] KL-divergence is very high: 123.0123 [2023-12-27 14:17:53,113][20291] KL-divergence is very high: 149.0110 [2023-12-27 14:17:53,127][20347] Updated weights for policy 0, policy_version 1248404 (0.0008) [2023-12-27 14:17:53,140][20381] Updated weights for policy 1, policy_version 1247332 (0.0005) [2023-12-27 14:17:53,178][20347] Updated weights for policy 0, policy_version 1248414 (0.0010) [2023-12-27 14:17:53,188][20381] Updated weights for policy 1, policy_version 1247342 (0.0005) [2023-12-27 14:17:53,239][20381] Updated weights for policy 1, policy_version 1247352 (0.0007) [2023-12-27 14:17:53,746][20347] Updated weights for policy 0, policy_version 1248424 (0.0010) [2023-12-27 14:17:53,802][20347] Updated weights for policy 0, policy_version 1248434 (0.0010) [2023-12-27 14:17:53,857][20347] Updated weights for policy 0, policy_version 1248444 (0.0010) [2023-12-27 14:17:53,885][20381] Updated weights for policy 1, policy_version 1247362 (0.0006) [2023-12-27 14:17:53,948][20381] Updated weights for policy 1, policy_version 1247372 (0.0005) [2023-12-27 14:17:54,010][20381] Updated weights for policy 1, policy_version 1247382 (0.0005) [2023-12-27 14:17:54,572][20347] Updated weights for policy 0, policy_version 1248454 (0.0009) [2023-12-27 14:17:54,623][20347] Updated weights for policy 0, policy_version 1248464 (0.0007) [2023-12-27 14:17:54,636][20381] Updated weights for policy 1, policy_version 1247392 (0.0007) [2023-12-27 14:17:54,679][20347] Updated weights for policy 0, policy_version 1248474 (0.0006) [2023-12-27 14:17:54,691][20381] Updated weights for policy 1, policy_version 1247402 (0.0007) [2023-12-27 14:17:54,747][20381] Updated weights for policy 1, policy_version 1247412 (0.0008) [2023-12-27 14:17:55,376][20347] Updated weights for policy 0, policy_version 1248484 (0.0005) [2023-12-27 14:17:55,431][20347] Updated weights for policy 0, policy_version 1248494 (0.0005) [2023-12-27 14:17:55,444][20381] Updated weights for policy 1, policy_version 1247422 (0.0007) [2023-12-27 14:17:55,477][20347] Updated weights for policy 0, policy_version 1248504 (0.0005) [2023-12-27 14:17:55,502][20381] Updated weights for policy 1, policy_version 1247432 (0.0006) [2023-12-27 14:17:55,557][20381] Updated weights for policy 1, policy_version 1247442 (0.0007) [2023-12-27 14:17:56,090][20347] Updated weights for policy 0, policy_version 1248514 (0.0006) [2023-12-27 14:17:56,152][20347] Updated weights for policy 0, policy_version 1248524 (0.0005) [2023-12-27 14:17:56,194][20381] Updated weights for policy 1, policy_version 1247452 (0.0011) [2023-12-27 14:17:56,199][20347] Updated weights for policy 0, policy_version 1248534 (0.0006) [2023-12-27 14:17:56,243][20347] Updated weights for policy 0, policy_version 1248544 (0.0005) [2023-12-27 14:17:56,251][20381] Updated weights for policy 1, policy_version 1247462 (0.0010) [2023-12-27 14:17:56,306][20381] Updated weights for policy 1, policy_version 1247472 (0.0010) [2023-12-27 14:17:56,812][20347] Updated weights for policy 0, policy_version 1248554 (0.0005) [2023-12-27 14:17:56,857][20347] Updated weights for policy 0, policy_version 1248564 (0.0005) [2023-12-27 14:17:56,900][20381] Updated weights for policy 1, policy_version 1247482 (0.0011) [2023-12-27 14:17:56,905][20347] Updated weights for policy 0, policy_version 1248574 (0.0005) [2023-12-27 14:17:56,951][20381] Updated weights for policy 1, policy_version 1247492 (0.0010) [2023-12-27 14:17:57,007][20381] Updated weights for policy 1, policy_version 1247502 (0.0007) [2023-12-27 14:17:57,061][20381] Updated weights for policy 1, policy_version 1247512 (0.0005) [2023-12-27 14:17:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 639082496. Throughput: 0: 10088.9, 1: 10323.9. Samples: 639070548. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:17:57,363][19170] Avg episode reward: [(0, '679.060'), (1, '932.000')] [2023-12-27 14:17:57,573][20347] Updated weights for policy 0, policy_version 1248584 (0.0009) [2023-12-27 14:17:57,596][20381] Updated weights for policy 1, policy_version 1247522 (0.0005) [2023-12-27 14:17:57,631][20347] Updated weights for policy 0, policy_version 1248594 (0.0010) [2023-12-27 14:17:57,647][20381] Updated weights for policy 1, policy_version 1247532 (0.0005) [2023-12-27 14:17:57,685][20347] Updated weights for policy 0, policy_version 1248604 (0.0010) [2023-12-27 14:17:57,701][20381] Updated weights for policy 1, policy_version 1247542 (0.0005) [2023-12-27 14:17:58,293][20347] Updated weights for policy 0, policy_version 1248614 (0.0009) [2023-12-27 14:17:58,343][20347] Updated weights for policy 0, policy_version 1248624 (0.0008) [2023-12-27 14:17:58,371][20381] Updated weights for policy 1, policy_version 1247552 (0.0007) [2023-12-27 14:17:58,414][20347] Updated weights for policy 0, policy_version 1248634 (0.0007) [2023-12-27 14:17:58,436][20381] Updated weights for policy 1, policy_version 1247562 (0.0007) [2023-12-27 14:17:58,495][20381] Updated weights for policy 1, policy_version 1247572 (0.0008) [2023-12-27 14:17:59,216][20347] Updated weights for policy 0, policy_version 1248644 (0.0008) [2023-12-27 14:17:59,252][20381] Updated weights for policy 1, policy_version 1247582 (0.0008) [2023-12-27 14:17:59,282][20347] Updated weights for policy 0, policy_version 1248654 (0.0008) [2023-12-27 14:17:59,314][20381] Updated weights for policy 1, policy_version 1247592 (0.0007) [2023-12-27 14:17:59,343][20347] Updated weights for policy 0, policy_version 1248664 (0.0008) [2023-12-27 14:17:59,377][20381] Updated weights for policy 1, policy_version 1247602 (0.0006) [2023-12-27 14:17:59,927][20347] Updated weights for policy 0, policy_version 1248674 (0.0007) [2023-12-27 14:17:59,981][20347] Updated weights for policy 0, policy_version 1248684 (0.0008) [2023-12-27 14:18:00,035][20347] Updated weights for policy 0, policy_version 1248694 (0.0008) [2023-12-27 14:18:00,098][20381] Updated weights for policy 1, policy_version 1247612 (0.0008) [2023-12-27 14:18:00,101][20347] Updated weights for policy 0, policy_version 1248704 (0.0008) [2023-12-27 14:18:00,165][20381] Updated weights for policy 1, policy_version 1247622 (0.0008) [2023-12-27 14:18:00,228][20381] Updated weights for policy 1, policy_version 1247632 (0.0008) [2023-12-27 14:18:00,832][20347] Updated weights for policy 0, policy_version 1248714 (0.0007) [2023-12-27 14:18:00,837][20381] Updated weights for policy 1, policy_version 1247642 (0.0007) [2023-12-27 14:18:00,874][20347] Updated weights for policy 0, policy_version 1248724 (0.0006) [2023-12-27 14:18:00,884][20381] Updated weights for policy 1, policy_version 1247652 (0.0008) [2023-12-27 14:18:00,920][20347] Updated weights for policy 0, policy_version 1248734 (0.0005) [2023-12-27 14:18:00,928][20381] Updated weights for policy 1, policy_version 1247662 (0.0007) [2023-12-27 14:18:00,975][20381] Updated weights for policy 1, policy_version 1247672 (0.0008) [2023-12-27 14:18:01,635][20347] Updated weights for policy 0, policy_version 1248744 (0.0008) [2023-12-27 14:18:01,671][20381] Updated weights for policy 1, policy_version 1247682 (0.0007) [2023-12-27 14:18:01,695][20347] Updated weights for policy 0, policy_version 1248754 (0.0007) [2023-12-27 14:18:01,732][20381] Updated weights for policy 1, policy_version 1247692 (0.0009) [2023-12-27 14:18:01,766][20347] Updated weights for policy 0, policy_version 1248764 (0.0007) [2023-12-27 14:18:01,792][20381] Updated weights for policy 1, policy_version 1247702 (0.0008) [2023-12-27 14:18:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 639180800. Throughput: 0: 10151.4, 1: 10352.8. Samples: 639195464. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:02,363][19170] Avg episode reward: [(0, '635.670'), (1, '893.280')] [2023-12-27 14:18:02,456][20347] Updated weights for policy 0, policy_version 1248774 (0.0007) [2023-12-27 14:18:02,483][20381] Updated weights for policy 1, policy_version 1247712 (0.0006) [2023-12-27 14:18:02,504][20347] Updated weights for policy 0, policy_version 1248784 (0.0006) [2023-12-27 14:18:02,533][20381] Updated weights for policy 1, policy_version 1247722 (0.0006) [2023-12-27 14:18:02,557][20347] Updated weights for policy 0, policy_version 1248794 (0.0006) [2023-12-27 14:18:02,588][20381] Updated weights for policy 1, policy_version 1247732 (0.0006) [2023-12-27 14:18:03,223][20347] Updated weights for policy 0, policy_version 1248804 (0.0006) [2023-12-27 14:18:03,269][20381] Updated weights for policy 1, policy_version 1247742 (0.0006) [2023-12-27 14:18:03,279][20347] Updated weights for policy 0, policy_version 1248814 (0.0006) [2023-12-27 14:18:03,314][20381] Updated weights for policy 1, policy_version 1247752 (0.0005) [2023-12-27 14:18:03,330][20347] Updated weights for policy 0, policy_version 1248824 (0.0007) [2023-12-27 14:18:03,362][20381] Updated weights for policy 1, policy_version 1247762 (0.0006) [2023-12-27 14:18:03,996][20381] Updated weights for policy 1, policy_version 1247772 (0.0008) [2023-12-27 14:18:04,042][20381] Updated weights for policy 1, policy_version 1247782 (0.0011) [2023-12-27 14:18:04,048][20347] Updated weights for policy 0, policy_version 1248834 (0.0005) [2023-12-27 14:18:04,099][20381] Updated weights for policy 1, policy_version 1247792 (0.0009) [2023-12-27 14:18:04,108][20347] Updated weights for policy 0, policy_version 1248844 (0.0006) [2023-12-27 14:18:04,159][20347] Updated weights for policy 0, policy_version 1248854 (0.0007) [2023-12-27 14:18:04,212][20347] Updated weights for policy 0, policy_version 1248864 (0.0007) [2023-12-27 14:18:04,800][20381] Updated weights for policy 1, policy_version 1247802 (0.0007) [2023-12-27 14:18:04,859][20381] Updated weights for policy 1, policy_version 1247812 (0.0007) [2023-12-27 14:18:04,903][20347] Updated weights for policy 0, policy_version 1248874 (0.0007) [2023-12-27 14:18:04,921][20381] Updated weights for policy 1, policy_version 1247822 (0.0007) [2023-12-27 14:18:04,954][20347] Updated weights for policy 0, policy_version 1248884 (0.0007) [2023-12-27 14:18:04,974][20381] Updated weights for policy 1, policy_version 1247832 (0.0005) [2023-12-27 14:18:05,001][20347] Updated weights for policy 0, policy_version 1248894 (0.0007) [2023-12-27 14:18:05,595][20381] Updated weights for policy 1, policy_version 1247842 (0.0007) [2023-12-27 14:18:05,650][20381] Updated weights for policy 1, policy_version 1247852 (0.0008) [2023-12-27 14:18:05,704][20347] Updated weights for policy 0, policy_version 1248904 (0.0007) [2023-12-27 14:18:05,706][20381] Updated weights for policy 1, policy_version 1247862 (0.0006) [2023-12-27 14:18:05,753][20347] Updated weights for policy 0, policy_version 1248914 (0.0007) [2023-12-27 14:18:05,810][20347] Updated weights for policy 0, policy_version 1248924 (0.0007) [2023-12-27 14:18:06,368][20381] Updated weights for policy 1, policy_version 1247872 (0.0009) [2023-12-27 14:18:06,431][20381] Updated weights for policy 1, policy_version 1247882 (0.0008) [2023-12-27 14:18:06,489][20381] Updated weights for policy 1, policy_version 1247892 (0.0008) [2023-12-27 14:18:06,526][20347] Updated weights for policy 0, policy_version 1248934 (0.0008) [2023-12-27 14:18:06,583][20347] Updated weights for policy 0, policy_version 1248944 (0.0008) [2023-12-27 14:18:06,647][20347] Updated weights for policy 0, policy_version 1248954 (0.0008) [2023-12-27 14:18:07,189][20381] Updated weights for policy 1, policy_version 1247902 (0.0006) [2023-12-27 14:18:07,215][20292] KL-divergence is very high: 102.3632 [2023-12-27 14:18:07,246][20381] Updated weights for policy 1, policy_version 1247912 (0.0008) [2023-12-27 14:18:07,282][20347] Updated weights for policy 0, policy_version 1248964 (0.0007) [2023-12-27 14:18:07,303][20381] Updated weights for policy 1, policy_version 1247922 (0.0006) [2023-12-27 14:18:07,327][20347] Updated weights for policy 0, policy_version 1248974 (0.0005) [2023-12-27 14:18:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 639287296. Throughput: 0: 10138.6, 1: 10375.9. Samples: 639256300. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:07,363][19170] Avg episode reward: [(0, '465.510'), (1, '922.800')] [2023-12-27 14:18:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001247928_319496192.pth... [2023-12-27 14:18:07,372][20347] Updated weights for policy 0, policy_version 1248984 (0.0007) [2023-12-27 14:18:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001246712_319184896.pth [2023-12-27 14:18:07,411][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001248992_319799296.pth... [2023-12-27 14:18:07,414][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001247808_319496192.pth [2023-12-27 14:18:08,002][20381] Updated weights for policy 1, policy_version 1247932 (0.0007) [2023-12-27 14:18:08,060][20381] Updated weights for policy 1, policy_version 1247942 (0.0008) [2023-12-27 14:18:08,082][20347] Updated weights for policy 0, policy_version 1248994 (0.0007) [2023-12-27 14:18:08,115][20381] Updated weights for policy 1, policy_version 1247952 (0.0006) [2023-12-27 14:18:08,144][20347] Updated weights for policy 0, policy_version 1249004 (0.0006) [2023-12-27 14:18:08,203][20347] Updated weights for policy 0, policy_version 1249014 (0.0007) [2023-12-27 14:18:08,266][20347] Updated weights for policy 0, policy_version 1249024 (0.0005) [2023-12-27 14:18:08,820][20381] Updated weights for policy 1, policy_version 1247962 (0.0007) [2023-12-27 14:18:08,883][20381] Updated weights for policy 1, policy_version 1247972 (0.0008) [2023-12-27 14:18:08,936][20381] Updated weights for policy 1, policy_version 1247982 (0.0007) [2023-12-27 14:18:08,943][20347] Updated weights for policy 0, policy_version 1249034 (0.0010) [2023-12-27 14:18:08,990][20381] Updated weights for policy 1, policy_version 1247992 (0.0007) [2023-12-27 14:18:08,999][20347] Updated weights for policy 0, policy_version 1249044 (0.0010) [2023-12-27 14:18:09,055][20347] Updated weights for policy 0, policy_version 1249054 (0.0010) [2023-12-27 14:18:09,727][20381] Updated weights for policy 1, policy_version 1248002 (0.0006) [2023-12-27 14:18:09,781][20381] Updated weights for policy 1, policy_version 1248012 (0.0005) [2023-12-27 14:18:09,848][20347] Updated weights for policy 0, policy_version 1249064 (0.0007) [2023-12-27 14:18:09,853][20381] Updated weights for policy 1, policy_version 1248022 (0.0009) [2023-12-27 14:18:09,914][20347] Updated weights for policy 0, policy_version 1249074 (0.0007) [2023-12-27 14:18:09,977][20347] Updated weights for policy 0, policy_version 1249084 (0.0007) [2023-12-27 14:18:10,522][20381] Updated weights for policy 1, policy_version 1248032 (0.0008) [2023-12-27 14:18:10,570][20381] Updated weights for policy 1, policy_version 1248042 (0.0008) [2023-12-27 14:18:10,628][20381] Updated weights for policy 1, policy_version 1248052 (0.0009) [2023-12-27 14:18:10,642][20347] Updated weights for policy 0, policy_version 1249094 (0.0009) [2023-12-27 14:18:10,697][20347] Updated weights for policy 0, policy_version 1249104 (0.0010) [2023-12-27 14:18:10,749][20347] Updated weights for policy 0, policy_version 1249114 (0.0010) [2023-12-27 14:18:11,415][20381] Updated weights for policy 1, policy_version 1248062 (0.0010) [2023-12-27 14:18:11,445][20347] Updated weights for policy 0, policy_version 1249124 (0.0010) [2023-12-27 14:18:11,469][20381] Updated weights for policy 1, policy_version 1248072 (0.0008) [2023-12-27 14:18:11,494][20347] Updated weights for policy 0, policy_version 1249134 (0.0006) [2023-12-27 14:18:11,520][20381] Updated weights for policy 1, policy_version 1248082 (0.0005) [2023-12-27 14:18:11,548][20347] Updated weights for policy 0, policy_version 1249144 (0.0006) [2023-12-27 14:18:12,215][20381] Updated weights for policy 1, policy_version 1248092 (0.0006) [2023-12-27 14:18:12,285][20381] Updated weights for policy 1, policy_version 1248102 (0.0007) [2023-12-27 14:18:12,331][20347] Updated weights for policy 0, policy_version 1249154 (0.0008) [2023-12-27 14:18:12,355][20381] Updated weights for policy 1, policy_version 1248112 (0.0007) [2023-12-27 14:18:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 639377408. Throughput: 0: 10171.8, 1: 10319.8. Samples: 639375876. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:12,363][19170] Avg episode reward: [(0, '736.180'), (1, '932.500')] [2023-12-27 14:18:12,393][20347] Updated weights for policy 0, policy_version 1249164 (0.0007) [2023-12-27 14:18:12,451][20347] Updated weights for policy 0, policy_version 1249174 (0.0008) [2023-12-27 14:18:12,500][20347] Updated weights for policy 0, policy_version 1249184 (0.0009) [2023-12-27 14:18:13,026][20381] Updated weights for policy 1, policy_version 1248122 (0.0008) [2023-12-27 14:18:13,081][20381] Updated weights for policy 1, policy_version 1248132 (0.0011) [2023-12-27 14:18:13,132][20381] Updated weights for policy 1, policy_version 1248142 (0.0010) [2023-12-27 14:18:13,184][20381] Updated weights for policy 1, policy_version 1248152 (0.0010) [2023-12-27 14:18:13,217][20347] Updated weights for policy 0, policy_version 1249194 (0.0009) [2023-12-27 14:18:13,277][20347] Updated weights for policy 0, policy_version 1249204 (0.0008) [2023-12-27 14:18:13,331][20347] Updated weights for policy 0, policy_version 1249214 (0.0008) [2023-12-27 14:18:13,842][20381] Updated weights for policy 1, policy_version 1248162 (0.0010) [2023-12-27 14:18:13,900][20381] Updated weights for policy 1, policy_version 1248172 (0.0011) [2023-12-27 14:18:13,952][20381] Updated weights for policy 1, policy_version 1248182 (0.0010) [2023-12-27 14:18:14,030][20347] Updated weights for policy 0, policy_version 1249224 (0.0008) [2023-12-27 14:18:14,085][20347] Updated weights for policy 0, policy_version 1249234 (0.0008) [2023-12-27 14:18:14,136][20347] Updated weights for policy 0, policy_version 1249244 (0.0007) [2023-12-27 14:18:14,617][20381] Updated weights for policy 1, policy_version 1248192 (0.0010) [2023-12-27 14:18:14,669][20381] Updated weights for policy 1, policy_version 1248202 (0.0010) [2023-12-27 14:18:14,720][20381] Updated weights for policy 1, policy_version 1248212 (0.0009) [2023-12-27 14:18:14,749][20347] Updated weights for policy 0, policy_version 1249254 (0.0009) [2023-12-27 14:18:14,814][20347] Updated weights for policy 0, policy_version 1249264 (0.0011) [2023-12-27 14:18:14,874][20347] Updated weights for policy 0, policy_version 1249274 (0.0011) [2023-12-27 14:18:15,432][20381] Updated weights for policy 1, policy_version 1248222 (0.0008) [2023-12-27 14:18:15,490][20381] Updated weights for policy 1, policy_version 1248232 (0.0007) [2023-12-27 14:18:15,543][20381] Updated weights for policy 1, policy_version 1248242 (0.0010) [2023-12-27 14:18:15,600][20347] Updated weights for policy 0, policy_version 1249284 (0.0010) [2023-12-27 14:18:15,656][20347] Updated weights for policy 0, policy_version 1249294 (0.0008) [2023-12-27 14:18:15,707][20347] Updated weights for policy 0, policy_version 1249304 (0.0006) [2023-12-27 14:18:16,170][20381] Updated weights for policy 1, policy_version 1248252 (0.0008) [2023-12-27 14:18:16,219][20381] Updated weights for policy 1, policy_version 1248262 (0.0011) [2023-12-27 14:18:16,275][20381] Updated weights for policy 1, policy_version 1248272 (0.0011) [2023-12-27 14:18:16,434][20347] Updated weights for policy 0, policy_version 1249314 (0.0007) [2023-12-27 14:18:16,484][20347] Updated weights for policy 0, policy_version 1249324 (0.0007) [2023-12-27 14:18:16,532][20347] Updated weights for policy 0, policy_version 1249334 (0.0007) [2023-12-27 14:18:16,584][20347] Updated weights for policy 0, policy_version 1249344 (0.0008) [2023-12-27 14:18:16,877][20381] Updated weights for policy 1, policy_version 1248282 (0.0010) [2023-12-27 14:18:16,935][20381] Updated weights for policy 1, policy_version 1248292 (0.0011) [2023-12-27 14:18:16,984][20381] Updated weights for policy 1, policy_version 1248302 (0.0009) [2023-12-27 14:18:17,036][20381] Updated weights for policy 1, policy_version 1248312 (0.0005) [2023-12-27 14:18:17,253][20347] Updated weights for policy 0, policy_version 1249354 (0.0007) [2023-12-27 14:18:17,303][20347] Updated weights for policy 0, policy_version 1249364 (0.0007) [2023-12-27 14:18:17,357][20347] Updated weights for policy 0, policy_version 1249374 (0.0007) [2023-12-27 14:18:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 639483904. Throughput: 0: 10107.1, 1: 10309.1. Samples: 639497708. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:17,363][19170] Avg episode reward: [(0, '807.090'), (1, '912.430')] [2023-12-27 14:18:17,720][20381] Updated weights for policy 1, policy_version 1248322 (0.0005) [2023-12-27 14:18:17,777][20381] Updated weights for policy 1, policy_version 1248332 (0.0005) [2023-12-27 14:18:17,822][20381] Updated weights for policy 1, policy_version 1248342 (0.0005) [2023-12-27 14:18:18,003][20347] Updated weights for policy 0, policy_version 1249384 (0.0005) [2023-12-27 14:18:18,048][20347] Updated weights for policy 0, policy_version 1249394 (0.0005) [2023-12-27 14:18:18,093][20347] Updated weights for policy 0, policy_version 1249404 (0.0005) [2023-12-27 14:18:18,405][20381] Updated weights for policy 1, policy_version 1248352 (0.0007) [2023-12-27 14:18:18,464][20381] Updated weights for policy 1, policy_version 1248362 (0.0006) [2023-12-27 14:18:18,523][20381] Updated weights for policy 1, policy_version 1248372 (0.0011) [2023-12-27 14:18:18,741][20347] Updated weights for policy 0, policy_version 1249414 (0.0007) [2023-12-27 14:18:18,796][20347] Updated weights for policy 0, policy_version 1249424 (0.0008) [2023-12-27 14:18:18,854][20347] Updated weights for policy 0, policy_version 1249434 (0.0007) [2023-12-27 14:18:19,192][20381] Updated weights for policy 1, policy_version 1248382 (0.0009) [2023-12-27 14:18:19,249][20381] Updated weights for policy 1, policy_version 1248392 (0.0010) [2023-12-27 14:18:19,310][20381] Updated weights for policy 1, policy_version 1248402 (0.0008) [2023-12-27 14:18:19,537][20347] Updated weights for policy 0, policy_version 1249444 (0.0007) [2023-12-27 14:18:19,607][20347] Updated weights for policy 0, policy_version 1249454 (0.0007) [2023-12-27 14:18:19,678][20347] Updated weights for policy 0, policy_version 1249464 (0.0007) [2023-12-27 14:18:20,029][20381] Updated weights for policy 1, policy_version 1248412 (0.0008) [2023-12-27 14:18:20,083][20381] Updated weights for policy 1, policy_version 1248422 (0.0008) [2023-12-27 14:18:20,141][20381] Updated weights for policy 1, policy_version 1248432 (0.0009) [2023-12-27 14:18:20,269][20347] Updated weights for policy 0, policy_version 1249474 (0.0006) [2023-12-27 14:18:20,326][20347] Updated weights for policy 0, policy_version 1249484 (0.0008) [2023-12-27 14:18:20,386][20347] Updated weights for policy 0, policy_version 1249494 (0.0008) [2023-12-27 14:18:20,445][20347] Updated weights for policy 0, policy_version 1249504 (0.0007) [2023-12-27 14:18:20,884][20381] Updated weights for policy 1, policy_version 1248442 (0.0008) [2023-12-27 14:18:20,944][20381] Updated weights for policy 1, policy_version 1248452 (0.0008) [2023-12-27 14:18:21,000][20381] Updated weights for policy 1, policy_version 1248462 (0.0008) [2023-12-27 14:18:21,066][20381] Updated weights for policy 1, policy_version 1248472 (0.0008) [2023-12-27 14:18:21,176][20347] Updated weights for policy 0, policy_version 1249514 (0.0007) [2023-12-27 14:18:21,233][20347] Updated weights for policy 0, policy_version 1249524 (0.0006) [2023-12-27 14:18:21,292][20347] Updated weights for policy 0, policy_version 1249534 (0.0006) [2023-12-27 14:18:21,854][20381] Updated weights for policy 1, policy_version 1248482 (0.0008) [2023-12-27 14:18:21,905][20381] Updated weights for policy 1, policy_version 1248492 (0.0008) [2023-12-27 14:18:21,966][20381] Updated weights for policy 1, policy_version 1248502 (0.0008) [2023-12-27 14:18:22,003][20347] Updated weights for policy 0, policy_version 1249544 (0.0010) [2023-12-27 14:18:22,063][20347] Updated weights for policy 0, policy_version 1249554 (0.0011) [2023-12-27 14:18:22,122][20347] Updated weights for policy 0, policy_version 1249564 (0.0010) [2023-12-27 14:18:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 639590400. Throughput: 0: 10112.0, 1: 10314.3. Samples: 639560352. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:22,363][19170] Avg episode reward: [(0, '795.870'), (1, '902.290')] [2023-12-27 14:18:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001248504_319643648.pth... [2023-12-27 14:18:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001249568_319946752.pth... [2023-12-27 14:18:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001247320_319340544.pth [2023-12-27 14:18:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001248384_319643648.pth [2023-12-27 14:18:22,696][20381] Updated weights for policy 1, policy_version 1248512 (0.0007) [2023-12-27 14:18:22,755][20381] Updated weights for policy 1, policy_version 1248522 (0.0008) [2023-12-27 14:18:22,811][20381] Updated weights for policy 1, policy_version 1248532 (0.0007) [2023-12-27 14:18:22,816][20292] KL-divergence is very high: 172.3320 [2023-12-27 14:18:22,824][20347] Updated weights for policy 0, policy_version 1249574 (0.0010) [2023-12-27 14:18:22,877][20347] Updated weights for policy 0, policy_version 1249584 (0.0011) [2023-12-27 14:18:22,940][20347] Updated weights for policy 0, policy_version 1249594 (0.0009) [2023-12-27 14:18:23,518][20381] Updated weights for policy 1, policy_version 1248542 (0.0007) [2023-12-27 14:18:23,551][20347] Updated weights for policy 0, policy_version 1249604 (0.0007) [2023-12-27 14:18:23,566][20381] Updated weights for policy 1, policy_version 1248552 (0.0006) [2023-12-27 14:18:23,608][20347] Updated weights for policy 0, policy_version 1249614 (0.0007) [2023-12-27 14:18:23,614][20381] Updated weights for policy 1, policy_version 1248562 (0.0005) [2023-12-27 14:18:23,662][20347] Updated weights for policy 0, policy_version 1249624 (0.0007) [2023-12-27 14:18:24,260][20381] Updated weights for policy 1, policy_version 1248572 (0.0006) [2023-12-27 14:18:24,294][20347] Updated weights for policy 0, policy_version 1249634 (0.0009) [2023-12-27 14:18:24,324][20381] Updated weights for policy 1, policy_version 1248582 (0.0008) [2023-12-27 14:18:24,348][20347] Updated weights for policy 0, policy_version 1249644 (0.0006) [2023-12-27 14:18:24,377][20381] Updated weights for policy 1, policy_version 1248592 (0.0007) [2023-12-27 14:18:24,403][20347] Updated weights for policy 0, policy_version 1249654 (0.0005) [2023-12-27 14:18:24,461][20347] Updated weights for policy 0, policy_version 1249664 (0.0006) [2023-12-27 14:18:24,996][20381] Updated weights for policy 1, policy_version 1248602 (0.0007) [2023-12-27 14:18:25,041][20347] Updated weights for policy 0, policy_version 1249674 (0.0005) [2023-12-27 14:18:25,059][20381] Updated weights for policy 1, policy_version 1248612 (0.0005) [2023-12-27 14:18:25,105][20347] Updated weights for policy 0, policy_version 1249684 (0.0009) [2023-12-27 14:18:25,118][20381] Updated weights for policy 1, policy_version 1248622 (0.0005) [2023-12-27 14:18:25,164][20347] Updated weights for policy 0, policy_version 1249694 (0.0010) [2023-12-27 14:18:25,171][20381] Updated weights for policy 1, policy_version 1248632 (0.0007) [2023-12-27 14:18:25,735][20381] Updated weights for policy 1, policy_version 1248642 (0.0005) [2023-12-27 14:18:25,798][20381] Updated weights for policy 1, policy_version 1248652 (0.0006) [2023-12-27 14:18:25,812][20347] Updated weights for policy 0, policy_version 1249704 (0.0009) [2023-12-27 14:18:25,856][20381] Updated weights for policy 1, policy_version 1248662 (0.0006) [2023-12-27 14:18:25,867][20347] Updated weights for policy 0, policy_version 1249714 (0.0009) [2023-12-27 14:18:25,924][20347] Updated weights for policy 0, policy_version 1249724 (0.0010) [2023-12-27 14:18:26,420][20381] Updated weights for policy 1, policy_version 1248672 (0.0006) [2023-12-27 14:18:26,482][20381] Updated weights for policy 1, policy_version 1248682 (0.0006) [2023-12-27 14:18:26,542][20381] Updated weights for policy 1, policy_version 1248692 (0.0007) [2023-12-27 14:18:26,617][20347] Updated weights for policy 0, policy_version 1249734 (0.0010) [2023-12-27 14:18:26,668][20347] Updated weights for policy 0, policy_version 1249744 (0.0010) [2023-12-27 14:18:26,715][20347] Updated weights for policy 0, policy_version 1249754 (0.0010) [2023-12-27 14:18:27,191][20381] Updated weights for policy 1, policy_version 1248702 (0.0007) [2023-12-27 14:18:27,250][20381] Updated weights for policy 1, policy_version 1248712 (0.0007) [2023-12-27 14:18:27,308][20381] Updated weights for policy 1, policy_version 1248722 (0.0007) [2023-12-27 14:18:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 639696896. Throughput: 0: 10116.2, 1: 10342.7. Samples: 639685008. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:27,363][19170] Avg episode reward: [(0, '766.590'), (1, '863.190')] [2023-12-27 14:18:27,369][20347] Updated weights for policy 0, policy_version 1249764 (0.0008) [2023-12-27 14:18:27,414][20347] Updated weights for policy 0, policy_version 1249774 (0.0005) [2023-12-27 14:18:27,463][20347] Updated weights for policy 0, policy_version 1249784 (0.0005) [2023-12-27 14:18:27,904][20381] Updated weights for policy 1, policy_version 1248732 (0.0006) [2023-12-27 14:18:27,955][20381] Updated weights for policy 1, policy_version 1248742 (0.0005) [2023-12-27 14:18:28,010][20381] Updated weights for policy 1, policy_version 1248752 (0.0005) [2023-12-27 14:18:28,031][20347] Updated weights for policy 0, policy_version 1249794 (0.0006) [2023-12-27 14:18:28,087][20347] Updated weights for policy 0, policy_version 1249804 (0.0008) [2023-12-27 14:18:28,147][20347] Updated weights for policy 0, policy_version 1249814 (0.0010) [2023-12-27 14:18:28,205][20347] Updated weights for policy 0, policy_version 1249824 (0.0006) [2023-12-27 14:18:28,612][20381] Updated weights for policy 1, policy_version 1248762 (0.0006) [2023-12-27 14:18:28,664][20381] Updated weights for policy 1, policy_version 1248772 (0.0008) [2023-12-27 14:18:28,709][20381] Updated weights for policy 1, policy_version 1248782 (0.0008) [2023-12-27 14:18:28,755][20381] Updated weights for policy 1, policy_version 1248792 (0.0008) [2023-12-27 14:18:28,835][20347] Updated weights for policy 0, policy_version 1249834 (0.0010) [2023-12-27 14:18:28,893][20347] Updated weights for policy 0, policy_version 1249844 (0.0010) [2023-12-27 14:18:28,956][20347] Updated weights for policy 0, policy_version 1249854 (0.0010) [2023-12-27 14:18:29,499][20381] Updated weights for policy 1, policy_version 1248802 (0.0008) [2023-12-27 14:18:29,558][20381] Updated weights for policy 1, policy_version 1248812 (0.0007) [2023-12-27 14:18:29,617][20347] Updated weights for policy 0, policy_version 1249864 (0.0008) [2023-12-27 14:18:29,618][20381] Updated weights for policy 1, policy_version 1248822 (0.0008) [2023-12-27 14:18:29,671][20347] Updated weights for policy 0, policy_version 1249874 (0.0005) [2023-12-27 14:18:29,731][20347] Updated weights for policy 0, policy_version 1249884 (0.0006) [2023-12-27 14:18:30,340][20381] Updated weights for policy 1, policy_version 1248832 (0.0008) [2023-12-27 14:18:30,399][20347] Updated weights for policy 0, policy_version 1249894 (0.0007) [2023-12-27 14:18:30,402][20381] Updated weights for policy 1, policy_version 1248842 (0.0007) [2023-12-27 14:18:30,463][20347] Updated weights for policy 0, policy_version 1249904 (0.0007) [2023-12-27 14:18:30,466][20381] Updated weights for policy 1, policy_version 1248852 (0.0007) [2023-12-27 14:18:30,528][20347] Updated weights for policy 0, policy_version 1249914 (0.0009) [2023-12-27 14:18:31,165][20347] Updated weights for policy 0, policy_version 1249924 (0.0007) [2023-12-27 14:18:31,188][20381] Updated weights for policy 1, policy_version 1248862 (0.0007) [2023-12-27 14:18:31,223][20347] Updated weights for policy 0, policy_version 1249934 (0.0006) [2023-12-27 14:18:31,252][20381] Updated weights for policy 1, policy_version 1248872 (0.0007) [2023-12-27 14:18:31,284][20347] Updated weights for policy 0, policy_version 1249944 (0.0006) [2023-12-27 14:18:31,308][20381] Updated weights for policy 1, policy_version 1248882 (0.0006) [2023-12-27 14:18:31,965][20381] Updated weights for policy 1, policy_version 1248892 (0.0007) [2023-12-27 14:18:32,023][20381] Updated weights for policy 1, policy_version 1248902 (0.0007) [2023-12-27 14:18:32,030][20347] Updated weights for policy 0, policy_version 1249954 (0.0007) [2023-12-27 14:18:32,078][20381] Updated weights for policy 1, policy_version 1248912 (0.0007) [2023-12-27 14:18:32,092][20347] Updated weights for policy 0, policy_version 1249964 (0.0009) [2023-12-27 14:18:32,152][20347] Updated weights for policy 0, policy_version 1249974 (0.0008) [2023-12-27 14:18:32,207][20347] Updated weights for policy 0, policy_version 1249984 (0.0007) [2023-12-27 14:18:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 639803392. Throughput: 0: 10223.2, 1: 10339.8. Samples: 639810324. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:32,363][19170] Avg episode reward: [(0, '717.600'), (1, '892.530')] [2023-12-27 14:18:32,714][20381] Updated weights for policy 1, policy_version 1248922 (0.0009) [2023-12-27 14:18:32,777][20381] Updated weights for policy 1, policy_version 1248932 (0.0008) [2023-12-27 14:18:32,832][20381] Updated weights for policy 1, policy_version 1248942 (0.0008) [2023-12-27 14:18:32,885][20381] Updated weights for policy 1, policy_version 1248952 (0.0008) [2023-12-27 14:18:32,929][20347] Updated weights for policy 0, policy_version 1249994 (0.0008) [2023-12-27 14:18:32,993][20347] Updated weights for policy 0, policy_version 1250004 (0.0008) [2023-12-27 14:18:33,044][20347] Updated weights for policy 0, policy_version 1250014 (0.0007) [2023-12-27 14:18:33,528][20381] Updated weights for policy 1, policy_version 1248962 (0.0009) [2023-12-27 14:18:33,582][20381] Updated weights for policy 1, policy_version 1248972 (0.0005) [2023-12-27 14:18:33,587][20347] Updated weights for policy 0, policy_version 1250024 (0.0005) [2023-12-27 14:18:33,630][20381] Updated weights for policy 1, policy_version 1248982 (0.0005) [2023-12-27 14:18:33,643][20347] Updated weights for policy 0, policy_version 1250034 (0.0005) [2023-12-27 14:18:33,696][20347] Updated weights for policy 0, policy_version 1250044 (0.0005) [2023-12-27 14:18:34,216][20347] Updated weights for policy 0, policy_version 1250054 (0.0008) [2023-12-27 14:18:34,272][20347] Updated weights for policy 0, policy_version 1250064 (0.0010) [2023-12-27 14:18:34,301][20381] Updated weights for policy 1, policy_version 1248992 (0.0008) [2023-12-27 14:18:34,331][20347] Updated weights for policy 0, policy_version 1250074 (0.0010) [2023-12-27 14:18:34,361][20381] Updated weights for policy 1, policy_version 1249002 (0.0009) [2023-12-27 14:18:34,421][20381] Updated weights for policy 1, policy_version 1249012 (0.0011) [2023-12-27 14:18:35,036][20347] Updated weights for policy 0, policy_version 1250084 (0.0008) [2023-12-27 14:18:35,077][20381] Updated weights for policy 1, policy_version 1249022 (0.0007) [2023-12-27 14:18:35,079][20347] Updated weights for policy 0, policy_version 1250094 (0.0005) [2023-12-27 14:18:35,123][20381] Updated weights for policy 1, policy_version 1249032 (0.0005) [2023-12-27 14:18:35,128][20347] Updated weights for policy 0, policy_version 1250104 (0.0007) [2023-12-27 14:18:35,175][20381] Updated weights for policy 1, policy_version 1249042 (0.0005) [2023-12-27 14:18:35,682][20347] Updated weights for policy 0, policy_version 1250114 (0.0006) [2023-12-27 14:18:35,730][20347] Updated weights for policy 0, policy_version 1250124 (0.0007) [2023-12-27 14:18:35,764][20381] Updated weights for policy 1, policy_version 1249052 (0.0006) [2023-12-27 14:18:35,795][20347] Updated weights for policy 0, policy_version 1250134 (0.0007) [2023-12-27 14:18:35,813][20381] Updated weights for policy 1, policy_version 1249062 (0.0005) [2023-12-27 14:18:35,849][20347] Updated weights for policy 0, policy_version 1250144 (0.0005) [2023-12-27 14:18:35,874][20381] Updated weights for policy 1, policy_version 1249072 (0.0007) [2023-12-27 14:18:36,471][20381] Updated weights for policy 1, policy_version 1249082 (0.0007) [2023-12-27 14:18:36,531][20381] Updated weights for policy 1, policy_version 1249092 (0.0007) [2023-12-27 14:18:36,589][20381] Updated weights for policy 1, policy_version 1249102 (0.0008) [2023-12-27 14:18:36,605][20347] Updated weights for policy 0, policy_version 1250154 (0.0006) [2023-12-27 14:18:36,651][20381] Updated weights for policy 1, policy_version 1249112 (0.0007) [2023-12-27 14:18:36,662][20347] Updated weights for policy 0, policy_version 1250164 (0.0007) [2023-12-27 14:18:36,716][20347] Updated weights for policy 0, policy_version 1250174 (0.0008) [2023-12-27 14:18:37,341][20381] Updated weights for policy 1, policy_version 1249122 (0.0005) [2023-12-27 14:18:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 639901696. Throughput: 0: 10304.5, 1: 10344.5. Samples: 639874832. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:37,363][19170] Avg episode reward: [(0, '756.400'), (1, '873.480')] [2023-12-27 14:18:37,398][20347] Updated weights for policy 0, policy_version 1250184 (0.0008) [2023-12-27 14:18:37,398][20381] Updated weights for policy 1, policy_version 1249132 (0.0006) [2023-12-27 14:18:37,448][20347] Updated weights for policy 0, policy_version 1250194 (0.0007) [2023-12-27 14:18:37,465][20381] Updated weights for policy 1, policy_version 1249142 (0.0006) [2023-12-27 14:18:37,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001249144_319807488.pth... [2023-12-27 14:18:37,476][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001247928_319496192.pth [2023-12-27 14:18:37,503][20347] Updated weights for policy 0, policy_version 1250204 (0.0007) [2023-12-27 14:18:37,520][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001250208_320110592.pth... [2023-12-27 14:18:37,522][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001248992_319799296.pth [2023-12-27 14:18:38,082][20381] Updated weights for policy 1, policy_version 1249152 (0.0007) [2023-12-27 14:18:38,136][20347] Updated weights for policy 0, policy_version 1250214 (0.0007) [2023-12-27 14:18:38,139][20381] Updated weights for policy 1, policy_version 1249162 (0.0008) [2023-12-27 14:18:38,192][20381] Updated weights for policy 1, policy_version 1249172 (0.0010) [2023-12-27 14:18:38,194][20347] Updated weights for policy 0, policy_version 1250224 (0.0009) [2023-12-27 14:18:38,241][20347] Updated weights for policy 0, policy_version 1250234 (0.0006) [2023-12-27 14:18:38,812][20381] Updated weights for policy 1, policy_version 1249182 (0.0007) [2023-12-27 14:18:38,872][20381] Updated weights for policy 1, policy_version 1249192 (0.0008) [2023-12-27 14:18:38,929][20347] Updated weights for policy 0, policy_version 1250244 (0.0007) [2023-12-27 14:18:38,934][20381] Updated weights for policy 1, policy_version 1249202 (0.0007) [2023-12-27 14:18:38,986][20347] Updated weights for policy 0, policy_version 1250254 (0.0006) [2023-12-27 14:18:39,045][20347] Updated weights for policy 0, policy_version 1250264 (0.0009) [2023-12-27 14:18:39,702][20381] Updated weights for policy 1, policy_version 1249212 (0.0008) [2023-12-27 14:18:39,727][20347] Updated weights for policy 0, policy_version 1250274 (0.0008) [2023-12-27 14:18:39,759][20381] Updated weights for policy 1, policy_version 1249222 (0.0008) [2023-12-27 14:18:39,783][20347] Updated weights for policy 0, policy_version 1250284 (0.0007) [2023-12-27 14:18:39,816][20381] Updated weights for policy 1, policy_version 1249232 (0.0010) [2023-12-27 14:18:39,846][20347] Updated weights for policy 0, policy_version 1250294 (0.0009) [2023-12-27 14:18:39,902][20347] Updated weights for policy 0, policy_version 1250304 (0.0010) [2023-12-27 14:18:40,504][20381] Updated weights for policy 1, policy_version 1249242 (0.0007) [2023-12-27 14:18:40,565][20381] Updated weights for policy 1, policy_version 1249252 (0.0010) [2023-12-27 14:18:40,611][20347] Updated weights for policy 0, policy_version 1250314 (0.0006) [2023-12-27 14:18:40,617][20381] Updated weights for policy 1, policy_version 1249262 (0.0010) [2023-12-27 14:18:40,672][20347] Updated weights for policy 0, policy_version 1250324 (0.0007) [2023-12-27 14:18:40,678][20381] Updated weights for policy 1, policy_version 1249272 (0.0008) [2023-12-27 14:18:40,731][20347] Updated weights for policy 0, policy_version 1250334 (0.0007) [2023-12-27 14:18:41,429][20381] Updated weights for policy 1, policy_version 1249282 (0.0008) [2023-12-27 14:18:41,451][20347] Updated weights for policy 0, policy_version 1250344 (0.0006) [2023-12-27 14:18:41,456][20291] KL-divergence is very high: 227.7349 [2023-12-27 14:18:41,486][20381] Updated weights for policy 1, policy_version 1249292 (0.0007) [2023-12-27 14:18:41,501][20291] KL-divergence is very high: 199.4535 [2023-12-27 14:18:41,506][20347] Updated weights for policy 0, policy_version 1250354 (0.0006) [2023-12-27 14:18:41,542][20381] Updated weights for policy 1, policy_version 1249302 (0.0006) [2023-12-27 14:18:41,559][20347] Updated weights for policy 0, policy_version 1250364 (0.0006) [2023-12-27 14:18:42,251][20381] Updated weights for policy 1, policy_version 1249312 (0.0006) [2023-12-27 14:18:42,291][20347] Updated weights for policy 0, policy_version 1250374 (0.0006) [2023-12-27 14:18:42,313][20381] Updated weights for policy 1, policy_version 1249322 (0.0007) [2023-12-27 14:18:42,358][20347] Updated weights for policy 0, policy_version 1250384 (0.0008) [2023-12-27 14:18:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 640000000. Throughput: 0: 10291.5, 1: 10335.4. Samples: 639998764. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:42,363][19170] Avg episode reward: [(0, '785.320'), (1, '912.060')] [2023-12-27 14:18:42,377][20381] Updated weights for policy 1, policy_version 1249332 (0.0007) [2023-12-27 14:18:42,429][20347] Updated weights for policy 0, policy_version 1250394 (0.0008) [2023-12-27 14:18:42,995][20347] Updated weights for policy 0, policy_version 1250404 (0.0007) [2023-12-27 14:18:43,040][20347] Updated weights for policy 0, policy_version 1250414 (0.0005) [2023-12-27 14:18:43,089][20347] Updated weights for policy 0, policy_version 1250424 (0.0005) [2023-12-27 14:18:43,141][20381] Updated weights for policy 1, policy_version 1249342 (0.0006) [2023-12-27 14:18:43,191][20381] Updated weights for policy 1, policy_version 1249352 (0.0008) [2023-12-27 14:18:43,248][20381] Updated weights for policy 1, policy_version 1249362 (0.0009) [2023-12-27 14:18:43,694][20347] Updated weights for policy 0, policy_version 1250434 (0.0006) [2023-12-27 14:18:43,748][20347] Updated weights for policy 0, policy_version 1250444 (0.0007) [2023-12-27 14:18:43,801][20347] Updated weights for policy 0, policy_version 1250454 (0.0007) [2023-12-27 14:18:43,851][20347] Updated weights for policy 0, policy_version 1250464 (0.0007) [2023-12-27 14:18:43,971][20381] Updated weights for policy 1, policy_version 1249372 (0.0008) [2023-12-27 14:18:44,018][20381] Updated weights for policy 1, policy_version 1249382 (0.0008) [2023-12-27 14:18:44,068][20381] Updated weights for policy 1, policy_version 1249392 (0.0007) [2023-12-27 14:18:44,563][20347] Updated weights for policy 0, policy_version 1250474 (0.0007) [2023-12-27 14:18:44,626][20347] Updated weights for policy 0, policy_version 1250484 (0.0005) [2023-12-27 14:18:44,687][20347] Updated weights for policy 0, policy_version 1250494 (0.0005) [2023-12-27 14:18:44,703][20381] Updated weights for policy 1, policy_version 1249402 (0.0007) [2023-12-27 14:18:44,764][20381] Updated weights for policy 1, policy_version 1249412 (0.0007) [2023-12-27 14:18:44,827][20381] Updated weights for policy 1, policy_version 1249422 (0.0007) [2023-12-27 14:18:44,886][20381] Updated weights for policy 1, policy_version 1249432 (0.0007) [2023-12-27 14:18:45,258][20347] Updated weights for policy 0, policy_version 1250504 (0.0005) [2023-12-27 14:18:45,308][20347] Updated weights for policy 0, policy_version 1250514 (0.0009) [2023-12-27 14:18:45,353][20347] Updated weights for policy 0, policy_version 1250524 (0.0008) [2023-12-27 14:18:45,491][20381] Updated weights for policy 1, policy_version 1249442 (0.0005) [2023-12-27 14:18:45,552][20381] Updated weights for policy 1, policy_version 1249452 (0.0007) [2023-12-27 14:18:45,612][20381] Updated weights for policy 1, policy_version 1249462 (0.0008) [2023-12-27 14:18:46,056][20347] Updated weights for policy 0, policy_version 1250534 (0.0008) [2023-12-27 14:18:46,106][20347] Updated weights for policy 0, policy_version 1250544 (0.0008) [2023-12-27 14:18:46,165][20347] Updated weights for policy 0, policy_version 1250554 (0.0008) [2023-12-27 14:18:46,256][20381] Updated weights for policy 1, policy_version 1249472 (0.0008) [2023-12-27 14:18:46,308][20381] Updated weights for policy 1, policy_version 1249482 (0.0008) [2023-12-27 14:18:46,358][20381] Updated weights for policy 1, policy_version 1249492 (0.0007) [2023-12-27 14:18:46,796][20347] Updated weights for policy 0, policy_version 1250564 (0.0008) [2023-12-27 14:18:46,842][20347] Updated weights for policy 0, policy_version 1250574 (0.0008) [2023-12-27 14:18:46,894][20347] Updated weights for policy 0, policy_version 1250584 (0.0007) [2023-12-27 14:18:47,076][20381] Updated weights for policy 1, policy_version 1249502 (0.0009) [2023-12-27 14:18:47,131][20381] Updated weights for policy 1, policy_version 1249512 (0.0006) [2023-12-27 14:18:47,186][20381] Updated weights for policy 1, policy_version 1249522 (0.0005) [2023-12-27 14:18:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 640114688. Throughput: 0: 10330.3, 1: 10280.0. Samples: 640122928. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:47,364][19170] Avg episode reward: [(0, '873.240'), (1, '922.340')] [2023-12-27 14:18:47,576][20347] Updated weights for policy 0, policy_version 1250594 (0.0007) [2023-12-27 14:18:47,636][20347] Updated weights for policy 0, policy_version 1250604 (0.0007) [2023-12-27 14:18:47,698][20347] Updated weights for policy 0, policy_version 1250614 (0.0005) [2023-12-27 14:18:47,758][20381] Updated weights for policy 1, policy_version 1249532 (0.0008) [2023-12-27 14:18:47,767][20347] Updated weights for policy 0, policy_version 1250624 (0.0007) [2023-12-27 14:18:47,813][20381] Updated weights for policy 1, policy_version 1249542 (0.0010) [2023-12-27 14:18:47,871][20381] Updated weights for policy 1, policy_version 1249552 (0.0011) [2023-12-27 14:18:48,363][20347] Updated weights for policy 0, policy_version 1250634 (0.0008) [2023-12-27 14:18:48,423][20347] Updated weights for policy 0, policy_version 1250644 (0.0006) [2023-12-27 14:18:48,487][20347] Updated weights for policy 0, policy_version 1250654 (0.0007) [2023-12-27 14:18:48,544][20381] Updated weights for policy 1, policy_version 1249562 (0.0011) [2023-12-27 14:18:48,598][20381] Updated weights for policy 1, policy_version 1249572 (0.0011) [2023-12-27 14:18:48,658][20381] Updated weights for policy 1, policy_version 1249582 (0.0011) [2023-12-27 14:18:48,722][20381] Updated weights for policy 1, policy_version 1249592 (0.0011) [2023-12-27 14:18:49,134][20347] Updated weights for policy 0, policy_version 1250664 (0.0007) [2023-12-27 14:18:49,188][20347] Updated weights for policy 0, policy_version 1250674 (0.0007) [2023-12-27 14:18:49,250][20347] Updated weights for policy 0, policy_version 1250684 (0.0008) [2023-12-27 14:18:49,443][20381] Updated weights for policy 1, policy_version 1249602 (0.0011) [2023-12-27 14:18:49,496][20381] Updated weights for policy 1, policy_version 1249612 (0.0010) [2023-12-27 14:18:49,548][20381] Updated weights for policy 1, policy_version 1249622 (0.0005) [2023-12-27 14:18:49,959][20347] Updated weights for policy 0, policy_version 1250694 (0.0008) [2023-12-27 14:18:50,016][20347] Updated weights for policy 0, policy_version 1250704 (0.0008) [2023-12-27 14:18:50,077][20347] Updated weights for policy 0, policy_version 1250714 (0.0006) [2023-12-27 14:18:50,241][20381] Updated weights for policy 1, policy_version 1249632 (0.0008) [2023-12-27 14:18:50,296][20381] Updated weights for policy 1, policy_version 1249642 (0.0007) [2023-12-27 14:18:50,355][20381] Updated weights for policy 1, policy_version 1249652 (0.0007) [2023-12-27 14:18:50,755][20347] Updated weights for policy 0, policy_version 1250724 (0.0006) [2023-12-27 14:18:50,818][20347] Updated weights for policy 0, policy_version 1250734 (0.0006) [2023-12-27 14:18:50,882][20347] Updated weights for policy 0, policy_version 1250744 (0.0006) [2023-12-27 14:18:51,017][20381] Updated weights for policy 1, policy_version 1249662 (0.0007) [2023-12-27 14:18:51,081][20381] Updated weights for policy 1, policy_version 1249672 (0.0007) [2023-12-27 14:18:51,144][20381] Updated weights for policy 1, policy_version 1249682 (0.0008) [2023-12-27 14:18:51,548][20347] Updated weights for policy 0, policy_version 1250754 (0.0010) [2023-12-27 14:18:51,615][20347] Updated weights for policy 0, policy_version 1250764 (0.0010) [2023-12-27 14:18:51,686][20347] Updated weights for policy 0, policy_version 1250774 (0.0008) [2023-12-27 14:18:51,752][20347] Updated weights for policy 0, policy_version 1250784 (0.0009) [2023-12-27 14:18:51,908][20381] Updated weights for policy 1, policy_version 1249692 (0.0006) [2023-12-27 14:18:51,965][20381] Updated weights for policy 1, policy_version 1249702 (0.0005) [2023-12-27 14:18:52,022][20381] Updated weights for policy 1, policy_version 1249712 (0.0005) [2023-12-27 14:18:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 640212992. Throughput: 0: 10364.5, 1: 10285.5. Samples: 640185548. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:52,363][19170] Avg episode reward: [(0, '843.800'), (1, '874.560')] [2023-12-27 14:18:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001249720_319954944.pth... [2023-12-27 14:18:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001250784_320258048.pth... [2023-12-27 14:18:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001248504_319643648.pth [2023-12-27 14:18:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001249568_319946752.pth [2023-12-27 14:18:52,462][20347] Updated weights for policy 0, policy_version 1250794 (0.0007) [2023-12-27 14:18:52,523][20347] Updated weights for policy 0, policy_version 1250804 (0.0008) [2023-12-27 14:18:52,587][20347] Updated weights for policy 0, policy_version 1250814 (0.0008) [2023-12-27 14:18:52,678][20381] Updated weights for policy 1, policy_version 1249722 (0.0007) [2023-12-27 14:18:52,734][20381] Updated weights for policy 1, policy_version 1249732 (0.0008) [2023-12-27 14:18:52,793][20381] Updated weights for policy 1, policy_version 1249742 (0.0008) [2023-12-27 14:18:52,851][20381] Updated weights for policy 1, policy_version 1249752 (0.0007) [2023-12-27 14:18:53,228][20347] Updated weights for policy 0, policy_version 1250824 (0.0008) [2023-12-27 14:18:53,276][20347] Updated weights for policy 0, policy_version 1250834 (0.0007) [2023-12-27 14:18:53,328][20347] Updated weights for policy 0, policy_version 1250844 (0.0007) [2023-12-27 14:18:53,554][20381] Updated weights for policy 1, policy_version 1249762 (0.0006) [2023-12-27 14:18:53,613][20381] Updated weights for policy 1, policy_version 1249772 (0.0005) [2023-12-27 14:18:53,668][20381] Updated weights for policy 1, policy_version 1249782 (0.0007) [2023-12-27 14:18:54,017][20347] Updated weights for policy 0, policy_version 1250854 (0.0007) [2023-12-27 14:18:54,066][20347] Updated weights for policy 0, policy_version 1250864 (0.0007) [2023-12-27 14:18:54,120][20347] Updated weights for policy 0, policy_version 1250874 (0.0008) [2023-12-27 14:18:54,327][20381] Updated weights for policy 1, policy_version 1249792 (0.0007) [2023-12-27 14:18:54,350][20292] KL-divergence is very high: 466.1221 [2023-12-27 14:18:54,360][20292] KL-divergence is very high: 351.0484 [2023-12-27 14:18:54,378][20381] Updated weights for policy 1, policy_version 1249802 (0.0007) [2023-12-27 14:18:54,388][20292] KL-divergence is very high: 508.0121 [2023-12-27 14:18:54,398][20292] KL-divergence is very high: 311.1063 [2023-12-27 14:18:54,434][20381] Updated weights for policy 1, policy_version 1249812 (0.0008) [2023-12-27 14:18:54,434][20292] KL-divergence is very high: 290.7567 [2023-12-27 14:18:54,445][20292] KL-divergence is very high: 178.6609 [2023-12-27 14:18:54,818][20347] Updated weights for policy 0, policy_version 1250884 (0.0007) [2023-12-27 14:18:54,879][20347] Updated weights for policy 0, policy_version 1250894 (0.0007) [2023-12-27 14:18:54,927][20347] Updated weights for policy 0, policy_version 1250904 (0.0007) [2023-12-27 14:18:55,081][20381] Updated weights for policy 1, policy_version 1249822 (0.0008) [2023-12-27 14:18:55,135][20381] Updated weights for policy 1, policy_version 1249832 (0.0008) [2023-12-27 14:18:55,183][20381] Updated weights for policy 1, policy_version 1249842 (0.0006) [2023-12-27 14:18:55,629][20347] Updated weights for policy 0, policy_version 1250914 (0.0007) [2023-12-27 14:18:55,692][20347] Updated weights for policy 0, policy_version 1250924 (0.0008) [2023-12-27 14:18:55,739][20347] Updated weights for policy 0, policy_version 1250934 (0.0007) [2023-12-27 14:18:55,775][20381] Updated weights for policy 1, policy_version 1249852 (0.0006) [2023-12-27 14:18:55,788][20347] Updated weights for policy 0, policy_version 1250944 (0.0005) [2023-12-27 14:18:55,826][20381] Updated weights for policy 1, policy_version 1249862 (0.0007) [2023-12-27 14:18:55,875][20381] Updated weights for policy 1, policy_version 1249872 (0.0007) [2023-12-27 14:18:56,382][20347] Updated weights for policy 0, policy_version 1250954 (0.0005) [2023-12-27 14:18:56,427][20347] Updated weights for policy 0, policy_version 1250964 (0.0005) [2023-12-27 14:18:56,474][20347] Updated weights for policy 0, policy_version 1250974 (0.0005) [2023-12-27 14:18:56,564][20381] Updated weights for policy 1, policy_version 1249882 (0.0007) [2023-12-27 14:18:56,623][20381] Updated weights for policy 1, policy_version 1249892 (0.0007) [2023-12-27 14:18:56,687][20381] Updated weights for policy 1, policy_version 1249902 (0.0007) [2023-12-27 14:18:56,751][20381] Updated weights for policy 1, policy_version 1249912 (0.0008) [2023-12-27 14:18:57,132][20347] Updated weights for policy 0, policy_version 1250984 (0.0007) [2023-12-27 14:18:57,188][20347] Updated weights for policy 0, policy_version 1250994 (0.0007) [2023-12-27 14:18:57,242][20347] Updated weights for policy 0, policy_version 1251004 (0.0008) [2023-12-27 14:18:57,358][20381] Updated weights for policy 1, policy_version 1249922 (0.0008) [2023-12-27 14:18:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 640319488. Throughput: 0: 10378.8, 1: 10324.4. Samples: 640307516. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:18:57,363][19170] Avg episode reward: [(0, '804.960'), (1, '885.440')] [2023-12-27 14:18:57,408][20381] Updated weights for policy 1, policy_version 1249932 (0.0008) [2023-12-27 14:18:57,454][20381] Updated weights for policy 1, policy_version 1249942 (0.0008) [2023-12-27 14:18:57,967][20347] Updated weights for policy 0, policy_version 1251014 (0.0009) [2023-12-27 14:18:58,010][20347] Updated weights for policy 0, policy_version 1251024 (0.0007) [2023-12-27 14:18:58,058][20347] Updated weights for policy 0, policy_version 1251034 (0.0007) [2023-12-27 14:18:58,094][20381] Updated weights for policy 1, policy_version 1249952 (0.0008) [2023-12-27 14:18:58,147][20381] Updated weights for policy 1, policy_version 1249962 (0.0006) [2023-12-27 14:18:58,210][20381] Updated weights for policy 1, policy_version 1249972 (0.0007) [2023-12-27 14:18:58,824][20347] Updated weights for policy 0, policy_version 1251044 (0.0008) [2023-12-27 14:18:58,883][20347] Updated weights for policy 0, policy_version 1251054 (0.0008) [2023-12-27 14:18:58,952][20347] Updated weights for policy 0, policy_version 1251064 (0.0008) [2023-12-27 14:18:58,961][20381] Updated weights for policy 1, policy_version 1249982 (0.0009) [2023-12-27 14:18:59,018][20381] Updated weights for policy 1, policy_version 1249992 (0.0011) [2023-12-27 14:18:59,076][20381] Updated weights for policy 1, policy_version 1250002 (0.0009) [2023-12-27 14:18:59,722][20347] Updated weights for policy 0, policy_version 1251074 (0.0006) [2023-12-27 14:18:59,774][20347] Updated weights for policy 0, policy_version 1251084 (0.0008) [2023-12-27 14:18:59,826][20347] Updated weights for policy 0, policy_version 1251094 (0.0008) [2023-12-27 14:18:59,831][20381] Updated weights for policy 1, policy_version 1250012 (0.0010) [2023-12-27 14:18:59,886][20347] Updated weights for policy 0, policy_version 1251104 (0.0007) [2023-12-27 14:18:59,892][20381] Updated weights for policy 1, policy_version 1250022 (0.0007) [2023-12-27 14:18:59,954][20381] Updated weights for policy 1, policy_version 1250032 (0.0008) [2023-12-27 14:18:59,984][20292] KL-divergence is very high: 146.8825 [2023-12-27 14:19:00,602][20381] Updated weights for policy 1, policy_version 1250042 (0.0007) [2023-12-27 14:19:00,619][20347] Updated weights for policy 0, policy_version 1251114 (0.0006) [2023-12-27 14:19:00,652][20381] Updated weights for policy 1, policy_version 1250052 (0.0005) [2023-12-27 14:19:00,677][20347] Updated weights for policy 0, policy_version 1251124 (0.0008) [2023-12-27 14:19:00,725][20381] Updated weights for policy 1, policy_version 1250062 (0.0007) [2023-12-27 14:19:00,737][20347] Updated weights for policy 0, policy_version 1251134 (0.0008) [2023-12-27 14:19:00,782][20381] Updated weights for policy 1, policy_version 1250072 (0.0006) [2023-12-27 14:19:01,432][20381] Updated weights for policy 1, policy_version 1250082 (0.0008) [2023-12-27 14:19:01,456][20347] Updated weights for policy 0, policy_version 1251144 (0.0007) [2023-12-27 14:19:01,481][20381] Updated weights for policy 1, policy_version 1250092 (0.0008) [2023-12-27 14:19:01,514][20347] Updated weights for policy 0, policy_version 1251154 (0.0007) [2023-12-27 14:19:01,532][20381] Updated weights for policy 1, policy_version 1250102 (0.0007) [2023-12-27 14:19:01,565][20347] Updated weights for policy 0, policy_version 1251164 (0.0008) [2023-12-27 14:19:02,197][20381] Updated weights for policy 1, policy_version 1250112 (0.0006) [2023-12-27 14:19:02,247][20347] Updated weights for policy 0, policy_version 1251174 (0.0007) [2023-12-27 14:19:02,258][20381] Updated weights for policy 1, policy_version 1250122 (0.0006) [2023-12-27 14:19:02,301][20347] Updated weights for policy 0, policy_version 1251184 (0.0006) [2023-12-27 14:19:02,316][20381] Updated weights for policy 1, policy_version 1250132 (0.0008) [2023-12-27 14:19:02,360][20347] Updated weights for policy 0, policy_version 1251194 (0.0006) [2023-12-27 14:19:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 640417792. Throughput: 0: 10382.0, 1: 10316.6. Samples: 640429140. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:02,363][19170] Avg episode reward: [(0, '746.080'), (1, '847.510')] [2023-12-27 14:19:02,983][20381] Updated weights for policy 1, policy_version 1250142 (0.0008) [2023-12-27 14:19:03,037][20381] Updated weights for policy 1, policy_version 1250152 (0.0008) [2023-12-27 14:19:03,071][20347] Updated weights for policy 0, policy_version 1251204 (0.0006) [2023-12-27 14:19:03,095][20381] Updated weights for policy 1, policy_version 1250162 (0.0006) [2023-12-27 14:19:03,119][20347] Updated weights for policy 0, policy_version 1251214 (0.0007) [2023-12-27 14:19:03,169][20347] Updated weights for policy 0, policy_version 1251224 (0.0007) [2023-12-27 14:19:03,747][20381] Updated weights for policy 1, policy_version 1250172 (0.0006) [2023-12-27 14:19:03,799][20292] KL-divergence is very high: 164.2165 [2023-12-27 14:19:03,804][20381] Updated weights for policy 1, policy_version 1250182 (0.0008) [2023-12-27 14:19:03,843][20292] KL-divergence is very high: 184.8871 [2023-12-27 14:19:03,862][20381] Updated weights for policy 1, policy_version 1250192 (0.0008) [2023-12-27 14:19:03,876][20347] Updated weights for policy 0, policy_version 1251234 (0.0008) [2023-12-27 14:19:03,934][20347] Updated weights for policy 0, policy_version 1251244 (0.0007) [2023-12-27 14:19:03,989][20347] Updated weights for policy 0, policy_version 1251254 (0.0008) [2023-12-27 14:19:04,047][20347] Updated weights for policy 0, policy_version 1251264 (0.0008) [2023-12-27 14:19:04,531][20381] Updated weights for policy 1, policy_version 1250202 (0.0007) [2023-12-27 14:19:04,590][20381] Updated weights for policy 1, policy_version 1250212 (0.0008) [2023-12-27 14:19:04,652][20381] Updated weights for policy 1, policy_version 1250222 (0.0011) [2023-12-27 14:19:04,710][20381] Updated weights for policy 1, policy_version 1250232 (0.0011) [2023-12-27 14:19:04,795][20347] Updated weights for policy 0, policy_version 1251274 (0.0008) [2023-12-27 14:19:04,837][20347] Updated weights for policy 0, policy_version 1251284 (0.0008) [2023-12-27 14:19:04,883][20347] Updated weights for policy 0, policy_version 1251294 (0.0007) [2023-12-27 14:19:05,279][20381] Updated weights for policy 1, policy_version 1250242 (0.0009) [2023-12-27 14:19:05,331][20381] Updated weights for policy 1, policy_version 1250252 (0.0010) [2023-12-27 14:19:05,359][20292] KL-divergence is very high: 367.7566 [2023-12-27 14:19:05,380][20292] KL-divergence is very high: 686.6934 [2023-12-27 14:19:05,386][20381] Updated weights for policy 1, policy_version 1250262 (0.0010) [2023-12-27 14:19:05,605][20347] Updated weights for policy 0, policy_version 1251304 (0.0007) [2023-12-27 14:19:05,667][20347] Updated weights for policy 0, policy_version 1251314 (0.0007) [2023-12-27 14:19:05,720][20347] Updated weights for policy 0, policy_version 1251324 (0.0007) [2023-12-27 14:19:06,021][20292] KL-divergence is very high: 610.8773 [2023-12-27 14:19:06,027][20381] Updated weights for policy 1, policy_version 1250272 (0.0006) [2023-12-27 14:19:06,066][20292] KL-divergence is very high: 578.7110 [2023-12-27 14:19:06,083][20381] Updated weights for policy 1, policy_version 1250282 (0.0009) [2023-12-27 14:19:06,115][20292] KL-divergence is very high: 412.4816 [2023-12-27 14:19:06,145][20292] KL-divergence is very high: 104.3288 [2023-12-27 14:19:06,146][20381] Updated weights for policy 1, policy_version 1250292 (0.0011) [2023-12-27 14:19:06,160][20292] KL-divergence is very high: 295.5627 [2023-12-27 14:19:06,394][20347] Updated weights for policy 0, policy_version 1251334 (0.0007) [2023-12-27 14:19:06,444][20347] Updated weights for policy 0, policy_version 1251344 (0.0007) [2023-12-27 14:19:06,505][20347] Updated weights for policy 0, policy_version 1251354 (0.0008) [2023-12-27 14:19:06,785][20381] Updated weights for policy 1, policy_version 1250302 (0.0008) [2023-12-27 14:19:06,842][20381] Updated weights for policy 1, policy_version 1250312 (0.0008) [2023-12-27 14:19:06,897][20381] Updated weights for policy 1, policy_version 1250322 (0.0008) [2023-12-27 14:19:07,232][20347] Updated weights for policy 0, policy_version 1251364 (0.0007) [2023-12-27 14:19:07,297][20347] Updated weights for policy 0, policy_version 1251374 (0.0009) [2023-12-27 14:19:07,346][20347] Updated weights for policy 0, policy_version 1251384 (0.0010) [2023-12-27 14:19:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 640516096. Throughput: 0: 10327.7, 1: 10333.1. Samples: 640490088. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:07,363][19170] Avg episode reward: [(0, '726.880'), (1, '778.490')] [2023-12-27 14:19:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001250328_320110592.pth... [2023-12-27 14:19:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001249144_319807488.pth [2023-12-27 14:19:07,394][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001251392_320413696.pth... [2023-12-27 14:19:07,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001250208_320110592.pth [2023-12-27 14:19:07,587][20381] Updated weights for policy 1, policy_version 1250332 (0.0007) [2023-12-27 14:19:07,649][20381] Updated weights for policy 1, policy_version 1250342 (0.0006) [2023-12-27 14:19:07,703][20381] Updated weights for policy 1, policy_version 1250352 (0.0006) [2023-12-27 14:19:07,974][20347] Updated weights for policy 0, policy_version 1251394 (0.0009) [2023-12-27 14:19:08,030][20347] Updated weights for policy 0, policy_version 1251404 (0.0007) [2023-12-27 14:19:08,078][20347] Updated weights for policy 0, policy_version 1251414 (0.0005) [2023-12-27 14:19:08,129][20347] Updated weights for policy 0, policy_version 1251424 (0.0005) [2023-12-27 14:19:08,374][20381] Updated weights for policy 1, policy_version 1250362 (0.0006) [2023-12-27 14:19:08,429][20381] Updated weights for policy 1, policy_version 1250372 (0.0007) [2023-12-27 14:19:08,491][20381] Updated weights for policy 1, policy_version 1250382 (0.0008) [2023-12-27 14:19:08,546][20381] Updated weights for policy 1, policy_version 1250392 (0.0008) [2023-12-27 14:19:08,787][20347] Updated weights for policy 0, policy_version 1251434 (0.0007) [2023-12-27 14:19:08,856][20347] Updated weights for policy 0, policy_version 1251444 (0.0009) [2023-12-27 14:19:08,923][20347] Updated weights for policy 0, policy_version 1251454 (0.0008) [2023-12-27 14:19:09,242][20381] Updated weights for policy 1, policy_version 1250402 (0.0007) [2023-12-27 14:19:09,300][20381] Updated weights for policy 1, policy_version 1250412 (0.0007) [2023-12-27 14:19:09,365][20381] Updated weights for policy 1, policy_version 1250422 (0.0009) [2023-12-27 14:19:09,604][20347] Updated weights for policy 0, policy_version 1251464 (0.0007) [2023-12-27 14:19:09,665][20347] Updated weights for policy 0, policy_version 1251474 (0.0008) [2023-12-27 14:19:09,728][20347] Updated weights for policy 0, policy_version 1251484 (0.0008) [2023-12-27 14:19:10,128][20381] Updated weights for policy 1, policy_version 1250432 (0.0006) [2023-12-27 14:19:10,190][20381] Updated weights for policy 1, policy_version 1250442 (0.0007) [2023-12-27 14:19:10,253][20381] Updated weights for policy 1, policy_version 1250452 (0.0006) [2023-12-27 14:19:10,509][20347] Updated weights for policy 0, policy_version 1251494 (0.0008) [2023-12-27 14:19:10,569][20347] Updated weights for policy 0, policy_version 1251504 (0.0007) [2023-12-27 14:19:10,628][20347] Updated weights for policy 0, policy_version 1251514 (0.0008) [2023-12-27 14:19:10,907][20381] Updated weights for policy 1, policy_version 1250462 (0.0007) [2023-12-27 14:19:10,959][20381] Updated weights for policy 1, policy_version 1250472 (0.0008) [2023-12-27 14:19:11,040][20381] Updated weights for policy 1, policy_version 1250482 (0.0008) [2023-12-27 14:19:11,359][20347] Updated weights for policy 0, policy_version 1251524 (0.0008) [2023-12-27 14:19:11,432][20347] Updated weights for policy 0, policy_version 1251534 (0.0008) [2023-12-27 14:19:11,495][20347] Updated weights for policy 0, policy_version 1251544 (0.0008) [2023-12-27 14:19:11,751][20381] Updated weights for policy 1, policy_version 1250492 (0.0008) [2023-12-27 14:19:11,808][20381] Updated weights for policy 1, policy_version 1250502 (0.0007) [2023-12-27 14:19:11,872][20381] Updated weights for policy 1, policy_version 1250512 (0.0006) [2023-12-27 14:19:12,264][20347] Updated weights for policy 0, policy_version 1251554 (0.0008) [2023-12-27 14:19:12,323][20347] Updated weights for policy 0, policy_version 1251564 (0.0008) [2023-12-27 14:19:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 640614400. Throughput: 0: 10276.4, 1: 10297.9. Samples: 640610848. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:12,363][19170] Avg episode reward: [(0, '786.190'), (1, '585.660')] [2023-12-27 14:19:12,388][20347] Updated weights for policy 0, policy_version 1251574 (0.0009) [2023-12-27 14:19:12,449][20347] Updated weights for policy 0, policy_version 1251584 (0.0008) [2023-12-27 14:19:12,592][20381] Updated weights for policy 1, policy_version 1250522 (0.0007) [2023-12-27 14:19:12,662][20381] Updated weights for policy 1, policy_version 1250532 (0.0008) [2023-12-27 14:19:12,722][20381] Updated weights for policy 1, policy_version 1250542 (0.0006) [2023-12-27 14:19:12,777][20381] Updated weights for policy 1, policy_version 1250552 (0.0006) [2023-12-27 14:19:13,093][20347] Updated weights for policy 0, policy_version 1251594 (0.0010) [2023-12-27 14:19:13,152][20347] Updated weights for policy 0, policy_version 1251604 (0.0010) [2023-12-27 14:19:13,210][20347] Updated weights for policy 0, policy_version 1251614 (0.0010) [2023-12-27 14:19:13,447][20381] Updated weights for policy 1, policy_version 1250562 (0.0007) [2023-12-27 14:19:13,498][20381] Updated weights for policy 1, policy_version 1250572 (0.0007) [2023-12-27 14:19:13,559][20381] Updated weights for policy 1, policy_version 1250582 (0.0008) [2023-12-27 14:19:13,834][20347] Updated weights for policy 0, policy_version 1251624 (0.0006) [2023-12-27 14:19:13,878][20347] Updated weights for policy 0, policy_version 1251634 (0.0005) [2023-12-27 14:19:13,928][20347] Updated weights for policy 0, policy_version 1251644 (0.0005) [2023-12-27 14:19:14,219][20381] Updated weights for policy 1, policy_version 1250592 (0.0009) [2023-12-27 14:19:14,282][20381] Updated weights for policy 1, policy_version 1250602 (0.0006) [2023-12-27 14:19:14,338][20381] Updated weights for policy 1, policy_version 1250612 (0.0007) [2023-12-27 14:19:14,516][20347] Updated weights for policy 0, policy_version 1251654 (0.0005) [2023-12-27 14:19:14,577][20347] Updated weights for policy 0, policy_version 1251664 (0.0005) [2023-12-27 14:19:14,636][20347] Updated weights for policy 0, policy_version 1251674 (0.0005) [2023-12-27 14:19:14,942][20381] Updated weights for policy 1, policy_version 1250622 (0.0007) [2023-12-27 14:19:15,003][20381] Updated weights for policy 1, policy_version 1250632 (0.0008) [2023-12-27 14:19:15,065][20381] Updated weights for policy 1, policy_version 1250642 (0.0007) [2023-12-27 14:19:15,295][20347] Updated weights for policy 0, policy_version 1251684 (0.0007) [2023-12-27 14:19:15,351][20347] Updated weights for policy 0, policy_version 1251694 (0.0008) [2023-12-27 14:19:15,409][20347] Updated weights for policy 0, policy_version 1251704 (0.0007) [2023-12-27 14:19:15,791][20381] Updated weights for policy 1, policy_version 1250652 (0.0007) [2023-12-27 14:19:15,842][20381] Updated weights for policy 1, policy_version 1250662 (0.0009) [2023-12-27 14:19:15,897][20381] Updated weights for policy 1, policy_version 1250672 (0.0010) [2023-12-27 14:19:16,040][20347] Updated weights for policy 0, policy_version 1251714 (0.0008) [2023-12-27 14:19:16,088][20347] Updated weights for policy 0, policy_version 1251724 (0.0007) [2023-12-27 14:19:16,138][20347] Updated weights for policy 0, policy_version 1251734 (0.0008) [2023-12-27 14:19:16,188][20347] Updated weights for policy 0, policy_version 1251744 (0.0007) [2023-12-27 14:19:16,616][20381] Updated weights for policy 1, policy_version 1250682 (0.0010) [2023-12-27 14:19:16,671][20381] Updated weights for policy 1, policy_version 1250692 (0.0007) [2023-12-27 14:19:16,719][20381] Updated weights for policy 1, policy_version 1250702 (0.0008) [2023-12-27 14:19:16,768][20381] Updated weights for policy 1, policy_version 1250712 (0.0007) [2023-12-27 14:19:16,885][20347] Updated weights for policy 0, policy_version 1251754 (0.0008) [2023-12-27 14:19:16,934][20347] Updated weights for policy 0, policy_version 1251764 (0.0007) [2023-12-27 14:19:16,988][20347] Updated weights for policy 0, policy_version 1251774 (0.0006) [2023-12-27 14:19:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 640720896. Throughput: 0: 10257.3, 1: 10263.5. Samples: 640733764. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:17,364][19170] Avg episode reward: [(0, '748.440'), (1, '662.800')] [2023-12-27 14:19:17,464][20381] Updated weights for policy 1, policy_version 1250722 (0.0007) [2023-12-27 14:19:17,527][20381] Updated weights for policy 1, policy_version 1250732 (0.0007) [2023-12-27 14:19:17,580][20347] Updated weights for policy 0, policy_version 1251784 (0.0007) [2023-12-27 14:19:17,590][20381] Updated weights for policy 1, policy_version 1250742 (0.0006) [2023-12-27 14:19:17,628][20347] Updated weights for policy 0, policy_version 1251794 (0.0007) [2023-12-27 14:19:17,680][20347] Updated weights for policy 0, policy_version 1251804 (0.0006) [2023-12-27 14:19:18,214][20381] Updated weights for policy 1, policy_version 1250752 (0.0010) [2023-12-27 14:19:18,247][20347] Updated weights for policy 0, policy_version 1251814 (0.0009) [2023-12-27 14:19:18,273][20381] Updated weights for policy 1, policy_version 1250762 (0.0011) [2023-12-27 14:19:18,312][20347] Updated weights for policy 0, policy_version 1251824 (0.0009) [2023-12-27 14:19:18,333][20381] Updated weights for policy 1, policy_version 1250772 (0.0008) [2023-12-27 14:19:18,379][20347] Updated weights for policy 0, policy_version 1251834 (0.0008) [2023-12-27 14:19:18,988][20381] Updated weights for policy 1, policy_version 1250782 (0.0009) [2023-12-27 14:19:19,034][20381] Updated weights for policy 1, policy_version 1250792 (0.0006) [2023-12-27 14:19:19,052][20347] Updated weights for policy 0, policy_version 1251844 (0.0007) [2023-12-27 14:19:19,088][20381] Updated weights for policy 1, policy_version 1250802 (0.0008) [2023-12-27 14:19:19,103][20347] Updated weights for policy 0, policy_version 1251854 (0.0006) [2023-12-27 14:19:19,160][20347] Updated weights for policy 0, policy_version 1251864 (0.0006) [2023-12-27 14:19:19,778][20381] Updated weights for policy 1, policy_version 1250812 (0.0010) [2023-12-27 14:19:19,845][20381] Updated weights for policy 1, policy_version 1250822 (0.0009) [2023-12-27 14:19:19,865][20347] Updated weights for policy 0, policy_version 1251874 (0.0005) [2023-12-27 14:19:19,898][20381] Updated weights for policy 1, policy_version 1250832 (0.0007) [2023-12-27 14:19:19,927][20347] Updated weights for policy 0, policy_version 1251884 (0.0008) [2023-12-27 14:19:19,988][20347] Updated weights for policy 0, policy_version 1251894 (0.0006) [2023-12-27 14:19:20,043][20347] Updated weights for policy 0, policy_version 1251904 (0.0007) [2023-12-27 14:19:20,629][20381] Updated weights for policy 1, policy_version 1250842 (0.0008) [2023-12-27 14:19:20,672][20347] Updated weights for policy 0, policy_version 1251914 (0.0007) [2023-12-27 14:19:20,718][20381] Updated weights for policy 1, policy_version 1250852 (0.0008) [2023-12-27 14:19:20,735][20347] Updated weights for policy 0, policy_version 1251924 (0.0006) [2023-12-27 14:19:20,777][20381] Updated weights for policy 1, policy_version 1250862 (0.0007) [2023-12-27 14:19:20,795][20347] Updated weights for policy 0, policy_version 1251934 (0.0008) [2023-12-27 14:19:20,831][20381] Updated weights for policy 1, policy_version 1250872 (0.0007) [2023-12-27 14:19:21,530][20347] Updated weights for policy 0, policy_version 1251944 (0.0007) [2023-12-27 14:19:21,554][20381] Updated weights for policy 1, policy_version 1250882 (0.0006) [2023-12-27 14:19:21,587][20347] Updated weights for policy 0, policy_version 1251954 (0.0008) [2023-12-27 14:19:21,616][20381] Updated weights for policy 1, policy_version 1250892 (0.0006) [2023-12-27 14:19:21,651][20347] Updated weights for policy 0, policy_version 1251964 (0.0008) [2023-12-27 14:19:21,681][20381] Updated weights for policy 1, policy_version 1250902 (0.0007) [2023-12-27 14:19:22,286][20381] Updated weights for policy 1, policy_version 1250912 (0.0009) [2023-12-27 14:19:22,355][20381] Updated weights for policy 1, policy_version 1250922 (0.0008) [2023-12-27 14:19:22,356][20347] Updated weights for policy 0, policy_version 1251974 (0.0006) [2023-12-27 14:19:22,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 640819200. Throughput: 0: 10249.9, 1: 10236.3. Samples: 640796712. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:22,363][19170] Avg episode reward: [(0, '728.220'), (1, '572.020')] [2023-12-27 14:19:22,417][20347] Updated weights for policy 0, policy_version 1251984 (0.0007) [2023-12-27 14:19:22,419][20381] Updated weights for policy 1, policy_version 1250932 (0.0008) [2023-12-27 14:19:22,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001250936_320266240.pth... [2023-12-27 14:19:22,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001249720_319954944.pth [2023-12-27 14:19:22,477][20347] Updated weights for policy 0, policy_version 1251994 (0.0011) [2023-12-27 14:19:22,511][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001252000_320569344.pth... [2023-12-27 14:19:22,514][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001250784_320258048.pth [2023-12-27 14:19:23,080][20381] Updated weights for policy 1, policy_version 1250942 (0.0006) [2023-12-27 14:19:23,129][20347] Updated weights for policy 0, policy_version 1252004 (0.0010) [2023-12-27 14:19:23,142][20381] Updated weights for policy 1, policy_version 1250952 (0.0005) [2023-12-27 14:19:23,184][20347] Updated weights for policy 0, policy_version 1252014 (0.0010) [2023-12-27 14:19:23,189][20381] Updated weights for policy 1, policy_version 1250962 (0.0005) [2023-12-27 14:19:23,242][20347] Updated weights for policy 0, policy_version 1252024 (0.0010) [2023-12-27 14:19:23,794][20381] Updated weights for policy 1, policy_version 1250972 (0.0007) [2023-12-27 14:19:23,853][20381] Updated weights for policy 1, policy_version 1250982 (0.0006) [2023-12-27 14:19:23,904][20347] Updated weights for policy 0, policy_version 1252034 (0.0010) [2023-12-27 14:19:23,906][20381] Updated weights for policy 1, policy_version 1250992 (0.0010) [2023-12-27 14:19:23,966][20347] Updated weights for policy 0, policy_version 1252044 (0.0010) [2023-12-27 14:19:24,020][20347] Updated weights for policy 0, policy_version 1252054 (0.0009) [2023-12-27 14:19:24,070][20347] Updated weights for policy 0, policy_version 1252064 (0.0008) [2023-12-27 14:19:24,522][20381] Updated weights for policy 1, policy_version 1251002 (0.0010) [2023-12-27 14:19:24,570][20381] Updated weights for policy 1, policy_version 1251012 (0.0007) [2023-12-27 14:19:24,617][20381] Updated weights for policy 1, policy_version 1251022 (0.0007) [2023-12-27 14:19:24,665][20381] Updated weights for policy 1, policy_version 1251032 (0.0007) [2023-12-27 14:19:24,735][20347] Updated weights for policy 0, policy_version 1252074 (0.0010) [2023-12-27 14:19:24,790][20347] Updated weights for policy 0, policy_version 1252084 (0.0010) [2023-12-27 14:19:24,841][20347] Updated weights for policy 0, policy_version 1252094 (0.0010) [2023-12-27 14:19:25,341][20381] Updated weights for policy 1, policy_version 1251042 (0.0008) [2023-12-27 14:19:25,400][20381] Updated weights for policy 1, policy_version 1251052 (0.0008) [2023-12-27 14:19:25,457][20381] Updated weights for policy 1, policy_version 1251062 (0.0008) [2023-12-27 14:19:25,502][20347] Updated weights for policy 0, policy_version 1252104 (0.0007) [2023-12-27 14:19:25,515][20291] KL-divergence is very high: 115.2677 [2023-12-27 14:19:25,556][20347] Updated weights for policy 0, policy_version 1252114 (0.0005) [2023-12-27 14:19:25,610][20347] Updated weights for policy 0, policy_version 1252124 (0.0006) [2023-12-27 14:19:26,159][20347] Updated weights for policy 0, policy_version 1252134 (0.0006) [2023-12-27 14:19:26,204][20381] Updated weights for policy 1, policy_version 1251072 (0.0008) [2023-12-27 14:19:26,225][20347] Updated weights for policy 0, policy_version 1252144 (0.0006) [2023-12-27 14:19:26,261][20381] Updated weights for policy 1, policy_version 1251082 (0.0006) [2023-12-27 14:19:26,283][20347] Updated weights for policy 0, policy_version 1252154 (0.0008) [2023-12-27 14:19:26,318][20381] Updated weights for policy 1, policy_version 1251092 (0.0005) [2023-12-27 14:19:26,869][20347] Updated weights for policy 0, policy_version 1252164 (0.0008) [2023-12-27 14:19:26,904][20381] Updated weights for policy 1, policy_version 1251102 (0.0005) [2023-12-27 14:19:26,917][20347] Updated weights for policy 0, policy_version 1252174 (0.0007) [2023-12-27 14:19:26,956][20381] Updated weights for policy 1, policy_version 1251112 (0.0005) [2023-12-27 14:19:26,969][20347] Updated weights for policy 0, policy_version 1252184 (0.0007) [2023-12-27 14:19:27,009][20381] Updated weights for policy 1, policy_version 1251122 (0.0005) [2023-12-27 14:19:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 640933888. Throughput: 0: 10261.7, 1: 10227.0. Samples: 640920756. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:27,363][19170] Avg episode reward: [(0, '639.210'), (1, '372.470')] [2023-12-27 14:19:27,612][20347] Updated weights for policy 0, policy_version 1252194 (0.0007) [2023-12-27 14:19:27,626][20381] Updated weights for policy 1, policy_version 1251132 (0.0006) [2023-12-27 14:19:27,666][20347] Updated weights for policy 0, policy_version 1252204 (0.0006) [2023-12-27 14:19:27,679][20381] Updated weights for policy 1, policy_version 1251142 (0.0006) [2023-12-27 14:19:27,719][20347] Updated weights for policy 0, policy_version 1252214 (0.0006) [2023-12-27 14:19:27,730][20381] Updated weights for policy 1, policy_version 1251152 (0.0007) [2023-12-27 14:19:27,766][20347] Updated weights for policy 0, policy_version 1252224 (0.0005) [2023-12-27 14:19:28,325][20381] Updated weights for policy 1, policy_version 1251162 (0.0007) [2023-12-27 14:19:28,399][20381] Updated weights for policy 1, policy_version 1251172 (0.0010) [2023-12-27 14:19:28,458][20381] Updated weights for policy 1, policy_version 1251182 (0.0011) [2023-12-27 14:19:28,460][20347] Updated weights for policy 0, policy_version 1252234 (0.0007) [2023-12-27 14:19:28,516][20381] Updated weights for policy 1, policy_version 1251192 (0.0011) [2023-12-27 14:19:28,521][20347] Updated weights for policy 0, policy_version 1252244 (0.0006) [2023-12-27 14:19:28,579][20347] Updated weights for policy 0, policy_version 1252254 (0.0007) [2023-12-27 14:19:29,156][20381] Updated weights for policy 1, policy_version 1251202 (0.0010) [2023-12-27 14:19:29,202][20381] Updated weights for policy 1, policy_version 1251212 (0.0010) [2023-12-27 14:19:29,204][20347] Updated weights for policy 0, policy_version 1252264 (0.0006) [2023-12-27 14:19:29,263][20381] Updated weights for policy 1, policy_version 1251222 (0.0009) [2023-12-27 14:19:29,275][20347] Updated weights for policy 0, policy_version 1252274 (0.0008) [2023-12-27 14:19:29,334][20347] Updated weights for policy 0, policy_version 1252284 (0.0006) [2023-12-27 14:19:29,965][20381] Updated weights for policy 1, policy_version 1251232 (0.0008) [2023-12-27 14:19:30,025][20381] Updated weights for policy 1, policy_version 1251242 (0.0007) [2023-12-27 14:19:30,035][20347] Updated weights for policy 0, policy_version 1252294 (0.0008) [2023-12-27 14:19:30,080][20381] Updated weights for policy 1, policy_version 1251252 (0.0008) [2023-12-27 14:19:30,092][20347] Updated weights for policy 0, policy_version 1252304 (0.0006) [2023-12-27 14:19:30,145][20347] Updated weights for policy 0, policy_version 1252314 (0.0007) [2023-12-27 14:19:30,779][20381] Updated weights for policy 1, policy_version 1251262 (0.0007) [2023-12-27 14:19:30,799][20347] Updated weights for policy 0, policy_version 1252324 (0.0010) [2023-12-27 14:19:30,834][20381] Updated weights for policy 1, policy_version 1251272 (0.0006) [2023-12-27 14:19:30,857][20347] Updated weights for policy 0, policy_version 1252334 (0.0010) [2023-12-27 14:19:30,891][20381] Updated weights for policy 1, policy_version 1251282 (0.0008) [2023-12-27 14:19:30,914][20347] Updated weights for policy 0, policy_version 1252344 (0.0010) [2023-12-27 14:19:31,625][20381] Updated weights for policy 1, policy_version 1251292 (0.0007) [2023-12-27 14:19:31,639][20347] Updated weights for policy 0, policy_version 1252354 (0.0009) [2023-12-27 14:19:31,688][20381] Updated weights for policy 1, policy_version 1251302 (0.0007) [2023-12-27 14:19:31,698][20347] Updated weights for policy 0, policy_version 1252364 (0.0008) [2023-12-27 14:19:31,750][20381] Updated weights for policy 1, policy_version 1251312 (0.0009) [2023-12-27 14:19:31,765][20347] Updated weights for policy 0, policy_version 1252374 (0.0010) [2023-12-27 14:19:31,823][20347] Updated weights for policy 0, policy_version 1252384 (0.0010) [2023-12-27 14:19:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 641032192. Throughput: 0: 10242.2, 1: 10270.8. Samples: 641046008. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:32,363][19170] Avg episode reward: [(0, '510.370'), (1, '520.370')] [2023-12-27 14:19:32,453][20381] Updated weights for policy 1, policy_version 1251322 (0.0008) [2023-12-27 14:19:32,499][20381] Updated weights for policy 1, policy_version 1251332 (0.0007) [2023-12-27 14:19:32,515][20347] Updated weights for policy 0, policy_version 1252394 (0.0008) [2023-12-27 14:19:32,551][20381] Updated weights for policy 1, policy_version 1251342 (0.0005) [2023-12-27 14:19:32,566][20347] Updated weights for policy 0, policy_version 1252404 (0.0007) [2023-12-27 14:19:32,609][20381] Updated weights for policy 1, policy_version 1251352 (0.0007) [2023-12-27 14:19:32,620][20347] Updated weights for policy 0, policy_version 1252414 (0.0006) [2023-12-27 14:19:33,214][20347] Updated weights for policy 0, policy_version 1252424 (0.0005) [2023-12-27 14:19:33,267][20347] Updated weights for policy 0, policy_version 1252434 (0.0005) [2023-12-27 14:19:33,321][20347] Updated weights for policy 0, policy_version 1252444 (0.0005) [2023-12-27 14:19:33,350][20381] Updated weights for policy 1, policy_version 1251362 (0.0007) [2023-12-27 14:19:33,409][20381] Updated weights for policy 1, policy_version 1251372 (0.0007) [2023-12-27 14:19:33,472][20381] Updated weights for policy 1, policy_version 1251382 (0.0008) [2023-12-27 14:19:33,949][20347] Updated weights for policy 0, policy_version 1252454 (0.0010) [2023-12-27 14:19:33,993][20347] Updated weights for policy 0, policy_version 1252464 (0.0010) [2023-12-27 14:19:34,041][20347] Updated weights for policy 0, policy_version 1252474 (0.0010) [2023-12-27 14:19:34,114][20381] Updated weights for policy 1, policy_version 1251392 (0.0006) [2023-12-27 14:19:34,179][20381] Updated weights for policy 1, policy_version 1251402 (0.0006) [2023-12-27 14:19:34,239][20381] Updated weights for policy 1, policy_version 1251412 (0.0009) [2023-12-27 14:19:34,675][20347] Updated weights for policy 0, policy_version 1252484 (0.0010) [2023-12-27 14:19:34,730][20347] Updated weights for policy 0, policy_version 1252494 (0.0010) [2023-12-27 14:19:34,784][20347] Updated weights for policy 0, policy_version 1252504 (0.0010) [2023-12-27 14:19:34,954][20381] Updated weights for policy 1, policy_version 1251422 (0.0008) [2023-12-27 14:19:35,015][20381] Updated weights for policy 1, policy_version 1251432 (0.0008) [2023-12-27 14:19:35,072][20381] Updated weights for policy 1, policy_version 1251442 (0.0008) [2023-12-27 14:19:35,371][20347] Updated weights for policy 0, policy_version 1252514 (0.0010) [2023-12-27 14:19:35,424][20347] Updated weights for policy 0, policy_version 1252524 (0.0007) [2023-12-27 14:19:35,475][20347] Updated weights for policy 0, policy_version 1252534 (0.0007) [2023-12-27 14:19:35,520][20347] Updated weights for policy 0, policy_version 1252544 (0.0009) [2023-12-27 14:19:35,671][20381] Updated weights for policy 1, policy_version 1251452 (0.0007) [2023-12-27 14:19:35,726][20381] Updated weights for policy 1, policy_version 1251462 (0.0006) [2023-12-27 14:19:35,782][20381] Updated weights for policy 1, policy_version 1251472 (0.0005) [2023-12-27 14:19:36,253][20347] Updated weights for policy 0, policy_version 1252554 (0.0008) [2023-12-27 14:19:36,321][20347] Updated weights for policy 0, policy_version 1252564 (0.0008) [2023-12-27 14:19:36,391][20347] Updated weights for policy 0, policy_version 1252574 (0.0007) [2023-12-27 14:19:36,452][20381] Updated weights for policy 1, policy_version 1251482 (0.0006) [2023-12-27 14:19:36,514][20381] Updated weights for policy 1, policy_version 1251492 (0.0006) [2023-12-27 14:19:36,574][20381] Updated weights for policy 1, policy_version 1251502 (0.0008) [2023-12-27 14:19:36,633][20381] Updated weights for policy 1, policy_version 1251512 (0.0008) [2023-12-27 14:19:36,988][20291] KL-divergence is very high: 111.0231 [2023-12-27 14:19:37,006][20347] Updated weights for policy 0, policy_version 1252584 (0.0009) [2023-12-27 14:19:37,070][20347] Updated weights for policy 0, policy_version 1252594 (0.0007) [2023-12-27 14:19:37,128][20347] Updated weights for policy 0, policy_version 1252604 (0.0008) [2023-12-27 14:19:37,249][20381] Updated weights for policy 1, policy_version 1251522 (0.0007) [2023-12-27 14:19:37,299][20381] Updated weights for policy 1, policy_version 1251532 (0.0008) [2023-12-27 14:19:37,360][20381] Updated weights for policy 1, policy_version 1251542 (0.0008) [2023-12-27 14:19:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 641138688. Throughput: 0: 10279.7, 1: 10231.6. Samples: 641108560. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:37,363][19170] Avg episode reward: [(0, '591.020'), (1, '586.820')] [2023-12-27 14:19:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001252608_320724992.pth... [2023-12-27 14:19:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001251544_320421888.pth... [2023-12-27 14:19:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001251392_320413696.pth [2023-12-27 14:19:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001250328_320110592.pth [2023-12-27 14:19:37,745][20347] Updated weights for policy 0, policy_version 1252614 (0.0006) [2023-12-27 14:19:37,801][20347] Updated weights for policy 0, policy_version 1252624 (0.0006) [2023-12-27 14:19:37,851][20347] Updated weights for policy 0, policy_version 1252634 (0.0007) [2023-12-27 14:19:38,037][20381] Updated weights for policy 1, policy_version 1251552 (0.0008) [2023-12-27 14:19:38,099][20381] Updated weights for policy 1, policy_version 1251562 (0.0007) [2023-12-27 14:19:38,155][20381] Updated weights for policy 1, policy_version 1251572 (0.0008) [2023-12-27 14:19:38,525][20347] Updated weights for policy 0, policy_version 1252644 (0.0006) [2023-12-27 14:19:38,587][20347] Updated weights for policy 0, policy_version 1252654 (0.0008) [2023-12-27 14:19:38,649][20347] Updated weights for policy 0, policy_version 1252664 (0.0008) [2023-12-27 14:19:38,852][20381] Updated weights for policy 1, policy_version 1251582 (0.0007) [2023-12-27 14:19:38,917][20381] Updated weights for policy 1, policy_version 1251592 (0.0006) [2023-12-27 14:19:38,967][20381] Updated weights for policy 1, policy_version 1251602 (0.0005) [2023-12-27 14:19:39,342][20347] Updated weights for policy 0, policy_version 1252674 (0.0009) [2023-12-27 14:19:39,410][20347] Updated weights for policy 0, policy_version 1252684 (0.0007) [2023-12-27 14:19:39,472][20347] Updated weights for policy 0, policy_version 1252694 (0.0009) [2023-12-27 14:19:39,537][20347] Updated weights for policy 0, policy_version 1252704 (0.0008) [2023-12-27 14:19:39,570][20381] Updated weights for policy 1, policy_version 1251612 (0.0006) [2023-12-27 14:19:39,624][20381] Updated weights for policy 1, policy_version 1251622 (0.0008) [2023-12-27 14:19:39,685][20381] Updated weights for policy 1, policy_version 1251632 (0.0006) [2023-12-27 14:19:40,263][20347] Updated weights for policy 0, policy_version 1252714 (0.0008) [2023-12-27 14:19:40,321][20347] Updated weights for policy 0, policy_version 1252724 (0.0008) [2023-12-27 14:19:40,351][20381] Updated weights for policy 1, policy_version 1251642 (0.0006) [2023-12-27 14:19:40,383][20347] Updated weights for policy 0, policy_version 1252734 (0.0008) [2023-12-27 14:19:40,409][20381] Updated weights for policy 1, policy_version 1251652 (0.0006) [2023-12-27 14:19:40,463][20381] Updated weights for policy 1, policy_version 1251662 (0.0008) [2023-12-27 14:19:40,519][20381] Updated weights for policy 1, policy_version 1251672 (0.0008) [2023-12-27 14:19:41,108][20347] Updated weights for policy 0, policy_version 1252744 (0.0008) [2023-12-27 14:19:41,177][20347] Updated weights for policy 0, policy_version 1252754 (0.0007) [2023-12-27 14:19:41,243][20347] Updated weights for policy 0, policy_version 1252764 (0.0007) [2023-12-27 14:19:41,303][20381] Updated weights for policy 1, policy_version 1251682 (0.0008) [2023-12-27 14:19:41,370][20381] Updated weights for policy 1, policy_version 1251692 (0.0007) [2023-12-27 14:19:41,436][20381] Updated weights for policy 1, policy_version 1251702 (0.0008) [2023-12-27 14:19:42,014][20347] Updated weights for policy 0, policy_version 1252774 (0.0008) [2023-12-27 14:19:42,070][20347] Updated weights for policy 0, policy_version 1252784 (0.0008) [2023-12-27 14:19:42,122][20347] Updated weights for policy 0, policy_version 1252794 (0.0009) [2023-12-27 14:19:42,155][20381] Updated weights for policy 1, policy_version 1251712 (0.0009) [2023-12-27 14:19:42,211][20381] Updated weights for policy 1, policy_version 1251722 (0.0006) [2023-12-27 14:19:42,276][20381] Updated weights for policy 1, policy_version 1251732 (0.0007) [2023-12-27 14:19:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 641245184. Throughput: 0: 10288.6, 1: 10273.5. Samples: 641232812. Policy #0 lag: (min: 28.0, avg: 43.5, max: 60.0) [2023-12-27 14:19:42,363][19170] Avg episode reward: [(0, '503.420'), (1, '573.020')] [2023-12-27 14:19:42,808][20347] Updated weights for policy 0, policy_version 1252804 (0.0008) [2023-12-27 14:19:42,857][20347] Updated weights for policy 0, policy_version 1252814 (0.0007) [2023-12-27 14:19:42,908][20347] Updated weights for policy 0, policy_version 1252824 (0.0007) [2023-12-27 14:19:42,949][20381] Updated weights for policy 1, policy_version 1251742 (0.0007) [2023-12-27 14:19:43,002][20381] Updated weights for policy 1, policy_version 1251752 (0.0005) [2023-12-27 14:19:43,049][20381] Updated weights for policy 1, policy_version 1251762 (0.0006) [2023-12-27 14:19:43,580][20347] Updated weights for policy 0, policy_version 1252834 (0.0006) [2023-12-27 14:19:43,637][20347] Updated weights for policy 0, policy_version 1252844 (0.0005) [2023-12-27 14:19:43,694][20347] Updated weights for policy 0, policy_version 1252854 (0.0007) [2023-12-27 14:19:43,750][20347] Updated weights for policy 0, policy_version 1252864 (0.0007) [2023-12-27 14:19:43,751][20381] Updated weights for policy 1, policy_version 1251772 (0.0007) [2023-12-27 14:19:43,802][20381] Updated weights for policy 1, policy_version 1251782 (0.0005) [2023-12-27 14:19:43,855][20381] Updated weights for policy 1, policy_version 1251792 (0.0008) [2023-12-27 14:19:44,354][20347] Updated weights for policy 0, policy_version 1252874 (0.0007) [2023-12-27 14:19:44,403][20347] Updated weights for policy 0, policy_version 1252884 (0.0007) [2023-12-27 14:19:44,449][20347] Updated weights for policy 0, policy_version 1252894 (0.0007) [2023-12-27 14:19:44,558][20381] Updated weights for policy 1, policy_version 1251802 (0.0008) [2023-12-27 14:19:44,611][20381] Updated weights for policy 1, policy_version 1251812 (0.0008) [2023-12-27 14:19:44,669][20381] Updated weights for policy 1, policy_version 1251822 (0.0009) [2023-12-27 14:19:44,720][20381] Updated weights for policy 1, policy_version 1251832 (0.0007) [2023-12-27 14:19:45,173][20347] Updated weights for policy 0, policy_version 1252904 (0.0007) [2023-12-27 14:19:45,225][20347] Updated weights for policy 0, policy_version 1252914 (0.0007) [2023-12-27 14:19:45,281][20347] Updated weights for policy 0, policy_version 1252924 (0.0008) [2023-12-27 14:19:45,447][20381] Updated weights for policy 1, policy_version 1251842 (0.0010) [2023-12-27 14:19:45,508][20381] Updated weights for policy 1, policy_version 1251852 (0.0010) [2023-12-27 14:19:45,552][20381] Updated weights for policy 1, policy_version 1251862 (0.0010) [2023-12-27 14:19:45,915][20347] Updated weights for policy 0, policy_version 1252934 (0.0006) [2023-12-27 14:19:45,965][20347] Updated weights for policy 0, policy_version 1252944 (0.0009) [2023-12-27 14:19:46,016][20347] Updated weights for policy 0, policy_version 1252954 (0.0010) [2023-12-27 14:19:46,161][20381] Updated weights for policy 1, policy_version 1251872 (0.0009) [2023-12-27 14:19:46,220][20381] Updated weights for policy 1, policy_version 1251882 (0.0007) [2023-12-27 14:19:46,270][20381] Updated weights for policy 1, policy_version 1251892 (0.0007) [2023-12-27 14:19:46,712][20347] Updated weights for policy 0, policy_version 1252964 (0.0010) [2023-12-27 14:19:46,765][20347] Updated weights for policy 0, policy_version 1252974 (0.0011) [2023-12-27 14:19:46,818][20347] Updated weights for policy 0, policy_version 1252984 (0.0011) [2023-12-27 14:19:46,931][20381] Updated weights for policy 1, policy_version 1251902 (0.0006) [2023-12-27 14:19:46,989][20381] Updated weights for policy 1, policy_version 1251912 (0.0005) [2023-12-27 14:19:47,043][20381] Updated weights for policy 1, policy_version 1251922 (0.0005) [2023-12-27 14:19:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 641343488. Throughput: 0: 10310.4, 1: 10257.9. Samples: 641354716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:19:47,364][19170] Avg episode reward: [(0, '472.490'), (1, '560.060')] [2023-12-27 14:19:47,510][20347] Updated weights for policy 0, policy_version 1252994 (0.0011) [2023-12-27 14:19:47,563][20347] Updated weights for policy 0, policy_version 1253004 (0.0010) [2023-12-27 14:19:47,601][20381] Updated weights for policy 1, policy_version 1251932 (0.0006) [2023-12-27 14:19:47,618][20347] Updated weights for policy 0, policy_version 1253014 (0.0010) [2023-12-27 14:19:47,656][20381] Updated weights for policy 1, policy_version 1251942 (0.0005) [2023-12-27 14:19:47,670][20347] Updated weights for policy 0, policy_version 1253024 (0.0010) [2023-12-27 14:19:47,716][20381] Updated weights for policy 1, policy_version 1251952 (0.0006) [2023-12-27 14:19:48,320][20381] Updated weights for policy 1, policy_version 1251962 (0.0006) [2023-12-27 14:19:48,370][20347] Updated weights for policy 0, policy_version 1253034 (0.0010) [2023-12-27 14:19:48,386][20381] Updated weights for policy 1, policy_version 1251972 (0.0007) [2023-12-27 14:19:48,422][20347] Updated weights for policy 0, policy_version 1253044 (0.0008) [2023-12-27 14:19:48,448][20381] Updated weights for policy 1, policy_version 1251982 (0.0007) [2023-12-27 14:19:48,478][20347] Updated weights for policy 0, policy_version 1253054 (0.0007) [2023-12-27 14:19:48,506][20381] Updated weights for policy 1, policy_version 1251992 (0.0006) [2023-12-27 14:19:49,136][20347] Updated weights for policy 0, policy_version 1253064 (0.0010) [2023-12-27 14:19:49,178][20381] Updated weights for policy 1, policy_version 1252002 (0.0006) [2023-12-27 14:19:49,188][20347] Updated weights for policy 0, policy_version 1253074 (0.0010) [2023-12-27 14:19:49,240][20381] Updated weights for policy 1, policy_version 1252012 (0.0007) [2023-12-27 14:19:49,245][20347] Updated weights for policy 0, policy_version 1253084 (0.0009) [2023-12-27 14:19:49,298][20381] Updated weights for policy 1, policy_version 1252022 (0.0007) [2023-12-27 14:19:50,049][20347] Updated weights for policy 0, policy_version 1253094 (0.0010) [2023-12-27 14:19:50,070][20381] Updated weights for policy 1, policy_version 1252033 (0.0009) [2023-12-27 14:19:50,103][20347] Updated weights for policy 0, policy_version 1253104 (0.0009) [2023-12-27 14:19:50,124][20381] Updated weights for policy 1, policy_version 1252043 (0.0009) [2023-12-27 14:19:50,152][20347] Updated weights for policy 0, policy_version 1253114 (0.0009) [2023-12-27 14:19:50,177][20381] Updated weights for policy 1, policy_version 1252053 (0.0009) [2023-12-27 14:19:50,881][20347] Updated weights for policy 0, policy_version 1253124 (0.0009) [2023-12-27 14:19:50,940][20347] Updated weights for policy 0, policy_version 1253134 (0.0005) [2023-12-27 14:19:50,957][20381] Updated weights for policy 1, policy_version 1252063 (0.0009) [2023-12-27 14:19:51,005][20347] Updated weights for policy 0, policy_version 1253144 (0.0005) [2023-12-27 14:19:51,016][20381] Updated weights for policy 1, policy_version 1252073 (0.0008) [2023-12-27 14:19:51,085][20381] Updated weights for policy 1, policy_version 1252083 (0.0008) [2023-12-27 14:19:51,657][20347] Updated weights for policy 0, policy_version 1253154 (0.0008) [2023-12-27 14:19:51,712][20347] Updated weights for policy 0, policy_version 1253164 (0.0007) [2023-12-27 14:19:51,783][20347] Updated weights for policy 0, policy_version 1253174 (0.0009) [2023-12-27 14:19:51,825][20381] Updated weights for policy 1, policy_version 1252093 (0.0008) [2023-12-27 14:19:51,845][20347] Updated weights for policy 0, policy_version 1253184 (0.0008) [2023-12-27 14:19:51,889][20381] Updated weights for policy 1, policy_version 1252103 (0.0006) [2023-12-27 14:19:51,957][20381] Updated weights for policy 1, policy_version 1252113 (0.0007) [2023-12-27 14:19:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 641441792. Throughput: 0: 10322.1, 1: 10229.3. Samples: 641414904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:19:52,363][19170] Avg episode reward: [(0, '601.440'), (1, '479.790')] [2023-12-27 14:19:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001253184_320872448.pth... [2023-12-27 14:19:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001252120_320569344.pth... [2023-12-27 14:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001250936_320266240.pth [2023-12-27 14:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001252120_320569344.pth [2023-12-27 14:19:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001252000_320569344.pth [2023-12-27 14:19:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001253184_320872448.pth [2023-12-27 14:19:52,603][20347] Updated weights for policy 0, policy_version 1253194 (0.0007) [2023-12-27 14:19:52,656][20381] Updated weights for policy 1, policy_version 1252123 (0.0009) [2023-12-27 14:19:52,670][20347] Updated weights for policy 0, policy_version 1253204 (0.0007) [2023-12-27 14:19:52,717][20381] Updated weights for policy 1, policy_version 1252133 (0.0008) [2023-12-27 14:19:52,730][20347] Updated weights for policy 0, policy_version 1253214 (0.0006) [2023-12-27 14:19:52,777][20381] Updated weights for policy 1, policy_version 1252143 (0.0007) [2023-12-27 14:19:53,370][20347] Updated weights for policy 0, policy_version 1253224 (0.0008) [2023-12-27 14:19:53,376][20381] Updated weights for policy 1, policy_version 1252153 (0.0006) [2023-12-27 14:19:53,426][20347] Updated weights for policy 0, policy_version 1253234 (0.0008) [2023-12-27 14:19:53,433][20381] Updated weights for policy 1, policy_version 1252163 (0.0007) [2023-12-27 14:19:53,481][20347] Updated weights for policy 0, policy_version 1253244 (0.0008) [2023-12-27 14:19:53,488][20381] Updated weights for policy 1, policy_version 1252173 (0.0008) [2023-12-27 14:19:53,546][20381] Updated weights for policy 1, policy_version 1252183 (0.0008) [2023-12-27 14:19:54,177][20347] Updated weights for policy 0, policy_version 1253254 (0.0007) [2023-12-27 14:19:54,194][20381] Updated weights for policy 1, policy_version 1252193 (0.0008) [2023-12-27 14:19:54,234][20347] Updated weights for policy 0, policy_version 1253264 (0.0006) [2023-12-27 14:19:54,258][20381] Updated weights for policy 1, policy_version 1252203 (0.0005) [2023-12-27 14:19:54,289][20347] Updated weights for policy 0, policy_version 1253274 (0.0007) [2023-12-27 14:19:54,322][20381] Updated weights for policy 1, policy_version 1252213 (0.0006) [2023-12-27 14:19:54,982][20381] Updated weights for policy 1, policy_version 1252223 (0.0007) [2023-12-27 14:19:54,988][20347] Updated weights for policy 0, policy_version 1253284 (0.0005) [2023-12-27 14:19:55,036][20381] Updated weights for policy 1, policy_version 1252233 (0.0006) [2023-12-27 14:19:55,041][20347] Updated weights for policy 0, policy_version 1253294 (0.0005) [2023-12-27 14:19:55,082][20381] Updated weights for policy 1, policy_version 1252243 (0.0006) [2023-12-27 14:19:55,087][20347] Updated weights for policy 0, policy_version 1253304 (0.0006) [2023-12-27 14:19:55,734][20347] Updated weights for policy 0, policy_version 1253314 (0.0008) [2023-12-27 14:19:55,779][20381] Updated weights for policy 1, policy_version 1252253 (0.0006) [2023-12-27 14:19:55,781][20347] Updated weights for policy 0, policy_version 1253324 (0.0007) [2023-12-27 14:19:55,828][20381] Updated weights for policy 1, policy_version 1252263 (0.0005) [2023-12-27 14:19:55,833][20347] Updated weights for policy 0, policy_version 1253334 (0.0006) [2023-12-27 14:19:55,884][20347] Updated weights for policy 0, policy_version 1253344 (0.0006) [2023-12-27 14:19:55,885][20381] Updated weights for policy 1, policy_version 1252273 (0.0006) [2023-12-27 14:19:56,488][20347] Updated weights for policy 0, policy_version 1253354 (0.0005) [2023-12-27 14:19:56,515][20381] Updated weights for policy 1, policy_version 1252283 (0.0007) [2023-12-27 14:19:56,543][20347] Updated weights for policy 0, policy_version 1253364 (0.0005) [2023-12-27 14:19:56,565][20381] Updated weights for policy 1, policy_version 1252293 (0.0005) [2023-12-27 14:19:56,590][20347] Updated weights for policy 0, policy_version 1253374 (0.0005) [2023-12-27 14:19:56,619][20381] Updated weights for policy 1, policy_version 1252303 (0.0005) [2023-12-27 14:19:57,181][20347] Updated weights for policy 0, policy_version 1253384 (0.0006) [2023-12-27 14:19:57,205][20381] Updated weights for policy 1, policy_version 1252313 (0.0006) [2023-12-27 14:19:57,230][20347] Updated weights for policy 0, policy_version 1253394 (0.0006) [2023-12-27 14:19:57,254][20381] Updated weights for policy 1, policy_version 1252323 (0.0006) [2023-12-27 14:19:57,285][20347] Updated weights for policy 0, policy_version 1253404 (0.0007) [2023-12-27 14:19:57,314][20381] Updated weights for policy 1, policy_version 1252333 (0.0007) [2023-12-27 14:19:57,362][20381] Updated weights for policy 1, policy_version 1252343 (0.0007) [2023-12-27 14:19:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 641548288. Throughput: 0: 10312.0, 1: 10236.0. Samples: 641535512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:19:57,363][19170] Avg episode reward: [(0, '639.830'), (1, '382.140')] [2023-12-27 14:19:57,904][20347] Updated weights for policy 0, policy_version 1253414 (0.0007) [2023-12-27 14:19:57,959][20347] Updated weights for policy 0, policy_version 1253424 (0.0005) [2023-12-27 14:19:58,020][20381] Updated weights for policy 1, policy_version 1252353 (0.0006) [2023-12-27 14:19:58,021][20347] Updated weights for policy 0, policy_version 1253434 (0.0009) [2023-12-27 14:19:58,068][20381] Updated weights for policy 1, policy_version 1252363 (0.0006) [2023-12-27 14:19:58,118][20381] Updated weights for policy 1, policy_version 1252373 (0.0006) [2023-12-27 14:19:58,713][20347] Updated weights for policy 0, policy_version 1253444 (0.0008) [2023-12-27 14:19:58,775][20347] Updated weights for policy 0, policy_version 1253454 (0.0007) [2023-12-27 14:19:58,845][20347] Updated weights for policy 0, policy_version 1253464 (0.0007) [2023-12-27 14:19:58,877][20381] Updated weights for policy 1, policy_version 1252383 (0.0007) [2023-12-27 14:19:58,947][20381] Updated weights for policy 1, policy_version 1252393 (0.0007) [2023-12-27 14:19:59,009][20381] Updated weights for policy 1, policy_version 1252403 (0.0009) [2023-12-27 14:19:59,605][20347] Updated weights for policy 0, policy_version 1253474 (0.0008) [2023-12-27 14:19:59,664][20347] Updated weights for policy 0, policy_version 1253484 (0.0008) [2023-12-27 14:19:59,719][20381] Updated weights for policy 1, policy_version 1252413 (0.0007) [2023-12-27 14:19:59,727][20347] Updated weights for policy 0, policy_version 1253494 (0.0008) [2023-12-27 14:19:59,782][20381] Updated weights for policy 1, policy_version 1252423 (0.0009) [2023-12-27 14:19:59,791][20347] Updated weights for policy 0, policy_version 1253504 (0.0007) [2023-12-27 14:19:59,846][20381] Updated weights for policy 1, policy_version 1252433 (0.0007) [2023-12-27 14:20:00,501][20381] Updated weights for policy 1, policy_version 1252443 (0.0009) [2023-12-27 14:20:00,515][20347] Updated weights for policy 0, policy_version 1253514 (0.0007) [2023-12-27 14:20:00,562][20381] Updated weights for policy 1, policy_version 1252453 (0.0010) [2023-12-27 14:20:00,562][20347] Updated weights for policy 0, policy_version 1253524 (0.0007) [2023-12-27 14:20:00,618][20347] Updated weights for policy 0, policy_version 1253534 (0.0005) [2023-12-27 14:20:00,622][20381] Updated weights for policy 1, policy_version 1252463 (0.0010) [2023-12-27 14:20:01,286][20347] Updated weights for policy 0, policy_version 1253544 (0.0006) [2023-12-27 14:20:01,317][20381] Updated weights for policy 1, policy_version 1252473 (0.0010) [2023-12-27 14:20:01,343][20347] Updated weights for policy 0, policy_version 1253554 (0.0007) [2023-12-27 14:20:01,384][20381] Updated weights for policy 1, policy_version 1252483 (0.0008) [2023-12-27 14:20:01,404][20347] Updated weights for policy 0, policy_version 1253564 (0.0006) [2023-12-27 14:20:01,440][20381] Updated weights for policy 1, policy_version 1252493 (0.0008) [2023-12-27 14:20:01,491][20381] Updated weights for policy 1, policy_version 1252503 (0.0008) [2023-12-27 14:20:02,075][20347] Updated weights for policy 0, policy_version 1253574 (0.0006) [2023-12-27 14:20:02,131][20347] Updated weights for policy 0, policy_version 1253584 (0.0005) [2023-12-27 14:20:02,188][20347] Updated weights for policy 0, policy_version 1253594 (0.0006) [2023-12-27 14:20:02,235][20381] Updated weights for policy 1, policy_version 1252513 (0.0008) [2023-12-27 14:20:02,292][20381] Updated weights for policy 1, policy_version 1252523 (0.0008) [2023-12-27 14:20:02,355][20381] Updated weights for policy 1, policy_version 1252533 (0.0008) [2023-12-27 14:20:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 641646592. Throughput: 0: 10304.2, 1: 10241.8. Samples: 641658332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:02,363][19170] Avg episode reward: [(0, '698.130'), (1, '509.280')] [2023-12-27 14:20:02,866][20347] Updated weights for policy 0, policy_version 1253604 (0.0007) [2023-12-27 14:20:02,917][20347] Updated weights for policy 0, policy_version 1253614 (0.0007) [2023-12-27 14:20:02,972][20347] Updated weights for policy 0, policy_version 1253624 (0.0008) [2023-12-27 14:20:03,019][20381] Updated weights for policy 1, policy_version 1252543 (0.0006) [2023-12-27 14:20:03,072][20381] Updated weights for policy 1, policy_version 1252553 (0.0007) [2023-12-27 14:20:03,129][20381] Updated weights for policy 1, policy_version 1252563 (0.0007) [2023-12-27 14:20:03,655][20347] Updated weights for policy 0, policy_version 1253634 (0.0007) [2023-12-27 14:20:03,708][20347] Updated weights for policy 0, policy_version 1253644 (0.0007) [2023-12-27 14:20:03,760][20347] Updated weights for policy 0, policy_version 1253654 (0.0007) [2023-12-27 14:20:03,764][20381] Updated weights for policy 1, policy_version 1252573 (0.0006) [2023-12-27 14:20:03,817][20347] Updated weights for policy 0, policy_version 1253664 (0.0006) [2023-12-27 14:20:03,822][20381] Updated weights for policy 1, policy_version 1252583 (0.0007) [2023-12-27 14:20:03,888][20381] Updated weights for policy 1, policy_version 1252593 (0.0010) [2023-12-27 14:20:04,541][20347] Updated weights for policy 0, policy_version 1253674 (0.0007) [2023-12-27 14:20:04,570][20381] Updated weights for policy 1, policy_version 1252603 (0.0011) [2023-12-27 14:20:04,600][20347] Updated weights for policy 0, policy_version 1253684 (0.0007) [2023-12-27 14:20:04,629][20381] Updated weights for policy 1, policy_version 1252613 (0.0009) [2023-12-27 14:20:04,657][20347] Updated weights for policy 0, policy_version 1253694 (0.0006) [2023-12-27 14:20:04,678][20381] Updated weights for policy 1, policy_version 1252623 (0.0009) [2023-12-27 14:20:05,231][20381] Updated weights for policy 1, policy_version 1252633 (0.0006) [2023-12-27 14:20:05,289][20381] Updated weights for policy 1, policy_version 1252643 (0.0009) [2023-12-27 14:20:05,346][20381] Updated weights for policy 1, policy_version 1252653 (0.0010) [2023-12-27 14:20:05,373][20347] Updated weights for policy 0, policy_version 1253704 (0.0005) [2023-12-27 14:20:05,401][20381] Updated weights for policy 1, policy_version 1252663 (0.0009) [2023-12-27 14:20:05,423][20347] Updated weights for policy 0, policy_version 1253714 (0.0006) [2023-12-27 14:20:05,467][20347] Updated weights for policy 0, policy_version 1253724 (0.0008) [2023-12-27 14:20:06,016][20381] Updated weights for policy 1, policy_version 1252673 (0.0008) [2023-12-27 14:20:06,077][20381] Updated weights for policy 1, policy_version 1252683 (0.0008) [2023-12-27 14:20:06,151][20381] Updated weights for policy 1, policy_version 1252693 (0.0009) [2023-12-27 14:20:06,215][20347] Updated weights for policy 0, policy_version 1253734 (0.0008) [2023-12-27 14:20:06,267][20347] Updated weights for policy 0, policy_version 1253744 (0.0007) [2023-12-27 14:20:06,322][20347] Updated weights for policy 0, policy_version 1253754 (0.0008) [2023-12-27 14:20:06,805][20381] Updated weights for policy 1, policy_version 1252703 (0.0010) [2023-12-27 14:20:06,857][20381] Updated weights for policy 1, policy_version 1252713 (0.0010) [2023-12-27 14:20:06,910][20381] Updated weights for policy 1, policy_version 1252723 (0.0006) [2023-12-27 14:20:06,991][20347] Updated weights for policy 0, policy_version 1253764 (0.0007) [2023-12-27 14:20:07,034][20291] KL-divergence is very high: 107.9535 [2023-12-27 14:20:07,049][20347] Updated weights for policy 0, policy_version 1253774 (0.0007) [2023-12-27 14:20:07,108][20347] Updated weights for policy 0, policy_version 1253784 (0.0006) [2023-12-27 14:20:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 641753088. Throughput: 0: 10246.1, 1: 10287.1. Samples: 641720708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:07,363][19170] Avg episode reward: [(0, '707.490'), (1, '453.830')] [2023-12-27 14:20:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001252728_320724992.pth... [2023-12-27 14:20:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001253792_321028096.pth... [2023-12-27 14:20:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001251544_320421888.pth [2023-12-27 14:20:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001252608_320724992.pth [2023-12-27 14:20:07,554][20381] Updated weights for policy 1, policy_version 1252733 (0.0008) [2023-12-27 14:20:07,610][20381] Updated weights for policy 1, policy_version 1252743 (0.0011) [2023-12-27 14:20:07,670][20381] Updated weights for policy 1, policy_version 1252753 (0.0011) [2023-12-27 14:20:07,798][20347] Updated weights for policy 0, policy_version 1253794 (0.0007) [2023-12-27 14:20:07,855][20347] Updated weights for policy 0, policy_version 1253804 (0.0006) [2023-12-27 14:20:07,913][20347] Updated weights for policy 0, policy_version 1253814 (0.0010) [2023-12-27 14:20:07,975][20347] Updated weights for policy 0, policy_version 1253824 (0.0010) [2023-12-27 14:20:08,301][20381] Updated weights for policy 1, policy_version 1252763 (0.0010) [2023-12-27 14:20:08,361][20381] Updated weights for policy 1, policy_version 1252773 (0.0008) [2023-12-27 14:20:08,421][20381] Updated weights for policy 1, policy_version 1252783 (0.0007) [2023-12-27 14:20:08,641][20347] Updated weights for policy 0, policy_version 1253834 (0.0007) [2023-12-27 14:20:08,693][20347] Updated weights for policy 0, policy_version 1253844 (0.0008) [2023-12-27 14:20:08,748][20347] Updated weights for policy 0, policy_version 1253854 (0.0008) [2023-12-27 14:20:09,095][20381] Updated weights for policy 1, policy_version 1252793 (0.0008) [2023-12-27 14:20:09,154][20381] Updated weights for policy 1, policy_version 1252803 (0.0007) [2023-12-27 14:20:09,209][20381] Updated weights for policy 1, policy_version 1252813 (0.0007) [2023-12-27 14:20:09,268][20381] Updated weights for policy 1, policy_version 1252823 (0.0008) [2023-12-27 14:20:09,477][20347] Updated weights for policy 0, policy_version 1253864 (0.0006) [2023-12-27 14:20:09,542][20347] Updated weights for policy 0, policy_version 1253874 (0.0005) [2023-12-27 14:20:09,599][20347] Updated weights for policy 0, policy_version 1253884 (0.0007) [2023-12-27 14:20:10,015][20381] Updated weights for policy 1, policy_version 1252833 (0.0008) [2023-12-27 14:20:10,071][20381] Updated weights for policy 1, policy_version 1252843 (0.0008) [2023-12-27 14:20:10,127][20381] Updated weights for policy 1, policy_version 1252853 (0.0008) [2023-12-27 14:20:10,320][20347] Updated weights for policy 0, policy_version 1253894 (0.0007) [2023-12-27 14:20:10,382][20347] Updated weights for policy 0, policy_version 1253904 (0.0008) [2023-12-27 14:20:10,435][20347] Updated weights for policy 0, policy_version 1253914 (0.0006) [2023-12-27 14:20:10,864][20381] Updated weights for policy 1, policy_version 1252863 (0.0006) [2023-12-27 14:20:10,929][20381] Updated weights for policy 1, policy_version 1252873 (0.0008) [2023-12-27 14:20:10,986][20381] Updated weights for policy 1, policy_version 1252883 (0.0007) [2023-12-27 14:20:11,028][20347] Updated weights for policy 0, policy_version 1253924 (0.0006) [2023-12-27 14:20:11,092][20347] Updated weights for policy 0, policy_version 1253934 (0.0007) [2023-12-27 14:20:11,158][20347] Updated weights for policy 0, policy_version 1253944 (0.0008) [2023-12-27 14:20:11,814][20381] Updated weights for policy 1, policy_version 1252893 (0.0008) [2023-12-27 14:20:11,874][20381] Updated weights for policy 1, policy_version 1252903 (0.0008) [2023-12-27 14:20:11,888][20347] Updated weights for policy 0, policy_version 1253954 (0.0007) [2023-12-27 14:20:11,939][20381] Updated weights for policy 1, policy_version 1252913 (0.0008) [2023-12-27 14:20:11,945][20347] Updated weights for policy 0, policy_version 1253964 (0.0006) [2023-12-27 14:20:12,003][20347] Updated weights for policy 0, policy_version 1253974 (0.0006) [2023-12-27 14:20:12,065][20347] Updated weights for policy 0, policy_version 1253984 (0.0006) [2023-12-27 14:20:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 641851392. Throughput: 0: 10222.5, 1: 10264.5. Samples: 641842668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:12,363][19170] Avg episode reward: [(0, '668.220'), (1, '317.200')] [2023-12-27 14:20:12,635][20381] Updated weights for policy 1, policy_version 1252923 (0.0008) [2023-12-27 14:20:12,696][20381] Updated weights for policy 1, policy_version 1252933 (0.0007) [2023-12-27 14:20:12,745][20347] Updated weights for policy 0, policy_version 1253994 (0.0008) [2023-12-27 14:20:12,759][20381] Updated weights for policy 1, policy_version 1252943 (0.0007) [2023-12-27 14:20:12,798][20347] Updated weights for policy 0, policy_version 1254004 (0.0008) [2023-12-27 14:20:12,854][20347] Updated weights for policy 0, policy_version 1254014 (0.0008) [2023-12-27 14:20:13,423][20381] Updated weights for policy 1, policy_version 1252953 (0.0005) [2023-12-27 14:20:13,475][20381] Updated weights for policy 1, policy_version 1252963 (0.0005) [2023-12-27 14:20:13,528][20381] Updated weights for policy 1, policy_version 1252973 (0.0007) [2023-12-27 14:20:13,567][20347] Updated weights for policy 0, policy_version 1254024 (0.0006) [2023-12-27 14:20:13,580][20381] Updated weights for policy 1, policy_version 1252983 (0.0005) [2023-12-27 14:20:13,610][20347] Updated weights for policy 0, policy_version 1254034 (0.0006) [2023-12-27 14:20:13,662][20347] Updated weights for policy 0, policy_version 1254044 (0.0007) [2023-12-27 14:20:14,238][20381] Updated weights for policy 1, policy_version 1252993 (0.0007) [2023-12-27 14:20:14,290][20381] Updated weights for policy 1, policy_version 1253003 (0.0008) [2023-12-27 14:20:14,347][20381] Updated weights for policy 1, policy_version 1253013 (0.0007) [2023-12-27 14:20:14,375][20347] Updated weights for policy 0, policy_version 1254054 (0.0007) [2023-12-27 14:20:14,438][20347] Updated weights for policy 0, policy_version 1254064 (0.0008) [2023-12-27 14:20:14,504][20347] Updated weights for policy 0, policy_version 1254074 (0.0008) [2023-12-27 14:20:14,998][20381] Updated weights for policy 1, policy_version 1253023 (0.0007) [2023-12-27 14:20:15,070][20381] Updated weights for policy 1, policy_version 1253033 (0.0007) [2023-12-27 14:20:15,141][20381] Updated weights for policy 1, policy_version 1253043 (0.0009) [2023-12-27 14:20:15,208][20347] Updated weights for policy 0, policy_version 1254084 (0.0007) [2023-12-27 14:20:15,271][20291] KL-divergence is very high: 107.8480 [2023-12-27 14:20:15,294][20347] Updated weights for policy 0, policy_version 1254094 (0.0007) [2023-12-27 14:20:15,353][20347] Updated weights for policy 0, policy_version 1254104 (0.0008) [2023-12-27 14:20:15,787][20381] Updated weights for policy 1, policy_version 1253053 (0.0011) [2023-12-27 14:20:15,840][20381] Updated weights for policy 1, policy_version 1253063 (0.0011) [2023-12-27 14:20:15,898][20381] Updated weights for policy 1, policy_version 1253073 (0.0010) [2023-12-27 14:20:15,989][20347] Updated weights for policy 0, policy_version 1254114 (0.0008) [2023-12-27 14:20:16,043][20347] Updated weights for policy 0, policy_version 1254124 (0.0007) [2023-12-27 14:20:16,095][20347] Updated weights for policy 0, policy_version 1254134 (0.0007) [2023-12-27 14:20:16,144][20347] Updated weights for policy 0, policy_version 1254144 (0.0005) [2023-12-27 14:20:16,603][20381] Updated weights for policy 1, policy_version 1253083 (0.0010) [2023-12-27 14:20:16,662][20381] Updated weights for policy 1, policy_version 1253093 (0.0010) [2023-12-27 14:20:16,725][20381] Updated weights for policy 1, policy_version 1253103 (0.0007) [2023-12-27 14:20:16,788][20347] Updated weights for policy 0, policy_version 1254154 (0.0008) [2023-12-27 14:20:16,847][20347] Updated weights for policy 0, policy_version 1254164 (0.0007) [2023-12-27 14:20:16,908][20347] Updated weights for policy 0, policy_version 1254174 (0.0007) [2023-12-27 14:20:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 641949696. Throughput: 0: 10162.9, 1: 10214.3. Samples: 641962984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:17,364][19170] Avg episode reward: [(0, '708.870'), (1, '437.010')] [2023-12-27 14:20:17,374][20381] Updated weights for policy 1, policy_version 1253113 (0.0008) [2023-12-27 14:20:17,441][20381] Updated weights for policy 1, policy_version 1253123 (0.0008) [2023-12-27 14:20:17,491][20381] Updated weights for policy 1, policy_version 1253133 (0.0007) [2023-12-27 14:20:17,493][20347] Updated weights for policy 0, policy_version 1254184 (0.0007) [2023-12-27 14:20:17,540][20381] Updated weights for policy 1, policy_version 1253143 (0.0007) [2023-12-27 14:20:17,547][20347] Updated weights for policy 0, policy_version 1254194 (0.0008) [2023-12-27 14:20:17,606][20347] Updated weights for policy 0, policy_version 1254204 (0.0008) [2023-12-27 14:20:18,115][20381] Updated weights for policy 1, policy_version 1253153 (0.0007) [2023-12-27 14:20:18,167][20381] Updated weights for policy 1, policy_version 1253163 (0.0007) [2023-12-27 14:20:18,221][20347] Updated weights for policy 0, policy_version 1254214 (0.0007) [2023-12-27 14:20:18,222][20381] Updated weights for policy 1, policy_version 1253173 (0.0006) [2023-12-27 14:20:18,272][20347] Updated weights for policy 0, policy_version 1254224 (0.0006) [2023-12-27 14:20:18,321][20347] Updated weights for policy 0, policy_version 1254234 (0.0007) [2023-12-27 14:20:18,950][20381] Updated weights for policy 1, policy_version 1253183 (0.0007) [2023-12-27 14:20:19,003][20381] Updated weights for policy 1, policy_version 1253193 (0.0008) [2023-12-27 14:20:19,041][20347] Updated weights for policy 0, policy_version 1254244 (0.0009) [2023-12-27 14:20:19,054][20381] Updated weights for policy 1, policy_version 1253203 (0.0008) [2023-12-27 14:20:19,098][20347] Updated weights for policy 0, policy_version 1254254 (0.0006) [2023-12-27 14:20:19,146][20347] Updated weights for policy 0, policy_version 1254264 (0.0007) [2023-12-27 14:20:19,774][20381] Updated weights for policy 1, policy_version 1253213 (0.0007) [2023-12-27 14:20:19,825][20381] Updated weights for policy 1, policy_version 1253223 (0.0008) [2023-12-27 14:20:19,884][20381] Updated weights for policy 1, policy_version 1253233 (0.0009) [2023-12-27 14:20:19,916][20347] Updated weights for policy 0, policy_version 1254274 (0.0008) [2023-12-27 14:20:19,987][20347] Updated weights for policy 0, policy_version 1254284 (0.0008) [2023-12-27 14:20:20,053][20347] Updated weights for policy 0, policy_version 1254294 (0.0008) [2023-12-27 14:20:20,116][20347] Updated weights for policy 0, policy_version 1254304 (0.0008) [2023-12-27 14:20:20,636][20381] Updated weights for policy 1, policy_version 1253243 (0.0009) [2023-12-27 14:20:20,702][20381] Updated weights for policy 1, policy_version 1253253 (0.0011) [2023-12-27 14:20:20,760][20381] Updated weights for policy 1, policy_version 1253263 (0.0011) [2023-12-27 14:20:20,766][20347] Updated weights for policy 0, policy_version 1254314 (0.0008) [2023-12-27 14:20:20,824][20347] Updated weights for policy 0, policy_version 1254324 (0.0005) [2023-12-27 14:20:20,878][20347] Updated weights for policy 0, policy_version 1254334 (0.0006) [2023-12-27 14:20:21,518][20381] Updated weights for policy 1, policy_version 1253273 (0.0009) [2023-12-27 14:20:21,578][20381] Updated weights for policy 1, policy_version 1253283 (0.0007) [2023-12-27 14:20:21,589][20347] Updated weights for policy 0, policy_version 1254344 (0.0006) [2023-12-27 14:20:21,643][20381] Updated weights for policy 1, policy_version 1253293 (0.0007) [2023-12-27 14:20:21,658][20347] Updated weights for policy 0, policy_version 1254354 (0.0007) [2023-12-27 14:20:21,712][20381] Updated weights for policy 1, policy_version 1253303 (0.0007) [2023-12-27 14:20:21,726][20347] Updated weights for policy 0, policy_version 1254364 (0.0008) [2023-12-27 14:20:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 642048000. Throughput: 0: 10123.5, 1: 10236.5. Samples: 642024764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:22,363][19170] Avg episode reward: [(0, '786.570'), (1, '561.150')] [2023-12-27 14:20:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001254368_321175552.pth... [2023-12-27 14:20:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001253184_320872448.pth [2023-12-27 14:20:22,393][20381] Updated weights for policy 1, policy_version 1253313 (0.0009) [2023-12-27 14:20:22,442][20347] Updated weights for policy 0, policy_version 1254374 (0.0007) [2023-12-27 14:20:22,452][20381] Updated weights for policy 1, policy_version 1253323 (0.0008) [2023-12-27 14:20:22,509][20347] Updated weights for policy 0, policy_version 1254384 (0.0007) [2023-12-27 14:20:22,514][20381] Updated weights for policy 1, policy_version 1253333 (0.0008) [2023-12-27 14:20:22,529][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001253336_320880640.pth... [2023-12-27 14:20:22,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001252120_320569344.pth [2023-12-27 14:20:22,575][20347] Updated weights for policy 0, policy_version 1254394 (0.0006) [2023-12-27 14:20:23,145][20347] Updated weights for policy 0, policy_version 1254404 (0.0006) [2023-12-27 14:20:23,201][20347] Updated weights for policy 0, policy_version 1254414 (0.0007) [2023-12-27 14:20:23,211][20381] Updated weights for policy 1, policy_version 1253343 (0.0009) [2023-12-27 14:20:23,254][20347] Updated weights for policy 0, policy_version 1254424 (0.0005) [2023-12-27 14:20:23,267][20381] Updated weights for policy 1, policy_version 1253353 (0.0010) [2023-12-27 14:20:23,320][20381] Updated weights for policy 1, policy_version 1253363 (0.0010) [2023-12-27 14:20:23,914][20347] Updated weights for policy 0, policy_version 1254434 (0.0007) [2023-12-27 14:20:23,957][20381] Updated weights for policy 1, policy_version 1253373 (0.0010) [2023-12-27 14:20:23,972][20347] Updated weights for policy 0, policy_version 1254444 (0.0010) [2023-12-27 14:20:24,015][20381] Updated weights for policy 1, policy_version 1253383 (0.0010) [2023-12-27 14:20:24,027][20347] Updated weights for policy 0, policy_version 1254454 (0.0010) [2023-12-27 14:20:24,068][20381] Updated weights for policy 1, policy_version 1253393 (0.0010) [2023-12-27 14:20:24,086][20347] Updated weights for policy 0, policy_version 1254464 (0.0010) [2023-12-27 14:20:24,701][20347] Updated weights for policy 0, policy_version 1254474 (0.0010) [2023-12-27 14:20:24,750][20347] Updated weights for policy 0, policy_version 1254484 (0.0010) [2023-12-27 14:20:24,752][20381] Updated weights for policy 1, policy_version 1253403 (0.0009) [2023-12-27 14:20:24,798][20347] Updated weights for policy 0, policy_version 1254494 (0.0008) [2023-12-27 14:20:24,804][20381] Updated weights for policy 1, policy_version 1253413 (0.0006) [2023-12-27 14:20:24,856][20381] Updated weights for policy 1, policy_version 1253423 (0.0008) [2023-12-27 14:20:25,463][20347] Updated weights for policy 0, policy_version 1254504 (0.0007) [2023-12-27 14:20:25,491][20381] Updated weights for policy 1, policy_version 1253433 (0.0007) [2023-12-27 14:20:25,520][20347] Updated weights for policy 0, policy_version 1254514 (0.0005) [2023-12-27 14:20:25,544][20381] Updated weights for policy 1, policy_version 1253443 (0.0007) [2023-12-27 14:20:25,574][20347] Updated weights for policy 0, policy_version 1254524 (0.0008) [2023-12-27 14:20:25,597][20381] Updated weights for policy 1, policy_version 1253453 (0.0008) [2023-12-27 14:20:25,641][20381] Updated weights for policy 1, policy_version 1253463 (0.0007) [2023-12-27 14:20:26,203][20347] Updated weights for policy 0, policy_version 1254534 (0.0008) [2023-12-27 14:20:26,260][20347] Updated weights for policy 0, policy_version 1254544 (0.0007) [2023-12-27 14:20:26,310][20347] Updated weights for policy 0, policy_version 1254554 (0.0008) [2023-12-27 14:20:26,369][20381] Updated weights for policy 1, policy_version 1253473 (0.0008) [2023-12-27 14:20:26,421][20381] Updated weights for policy 1, policy_version 1253483 (0.0009) [2023-12-27 14:20:26,475][20381] Updated weights for policy 1, policy_version 1253493 (0.0008) [2023-12-27 14:20:26,885][20347] Updated weights for policy 0, policy_version 1254564 (0.0007) [2023-12-27 14:20:26,945][20347] Updated weights for policy 0, policy_version 1254574 (0.0008) [2023-12-27 14:20:27,002][20347] Updated weights for policy 0, policy_version 1254584 (0.0005) [2023-12-27 14:20:27,076][20381] Updated weights for policy 1, policy_version 1253503 (0.0007) [2023-12-27 14:20:27,122][20381] Updated weights for policy 1, policy_version 1253513 (0.0008) [2023-12-27 14:20:27,173][20381] Updated weights for policy 1, policy_version 1253523 (0.0008) [2023-12-27 14:20:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 642162688. Throughput: 0: 10161.8, 1: 10171.8. Samples: 642147824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:27,363][19170] Avg episode reward: [(0, '786.360'), (1, '603.570')] [2023-12-27 14:20:27,546][20347] Updated weights for policy 0, policy_version 1254594 (0.0006) [2023-12-27 14:20:27,614][20347] Updated weights for policy 0, policy_version 1254604 (0.0007) [2023-12-27 14:20:27,675][20347] Updated weights for policy 0, policy_version 1254614 (0.0008) [2023-12-27 14:20:27,727][20347] Updated weights for policy 0, policy_version 1254624 (0.0005) [2023-12-27 14:20:27,913][20381] Updated weights for policy 1, policy_version 1253533 (0.0008) [2023-12-27 14:20:27,966][20381] Updated weights for policy 1, policy_version 1253543 (0.0007) [2023-12-27 14:20:28,023][20381] Updated weights for policy 1, policy_version 1253553 (0.0007) [2023-12-27 14:20:28,297][20347] Updated weights for policy 0, policy_version 1254634 (0.0010) [2023-12-27 14:20:28,356][20347] Updated weights for policy 0, policy_version 1254644 (0.0010) [2023-12-27 14:20:28,414][20347] Updated weights for policy 0, policy_version 1254654 (0.0010) [2023-12-27 14:20:28,702][20381] Updated weights for policy 1, policy_version 1253563 (0.0007) [2023-12-27 14:20:28,755][20381] Updated weights for policy 1, policy_version 1253573 (0.0008) [2023-12-27 14:20:28,807][20381] Updated weights for policy 1, policy_version 1253583 (0.0008) [2023-12-27 14:20:29,102][20347] Updated weights for policy 0, policy_version 1254664 (0.0006) [2023-12-27 14:20:29,168][20347] Updated weights for policy 0, policy_version 1254674 (0.0005) [2023-12-27 14:20:29,230][20347] Updated weights for policy 0, policy_version 1254684 (0.0006) [2023-12-27 14:20:29,529][20381] Updated weights for policy 1, policy_version 1253593 (0.0007) [2023-12-27 14:20:29,593][20381] Updated weights for policy 1, policy_version 1253603 (0.0007) [2023-12-27 14:20:29,651][20381] Updated weights for policy 1, policy_version 1253613 (0.0008) [2023-12-27 14:20:29,711][20381] Updated weights for policy 1, policy_version 1253623 (0.0008) [2023-12-27 14:20:29,842][20347] Updated weights for policy 0, policy_version 1254694 (0.0009) [2023-12-27 14:20:29,905][20347] Updated weights for policy 0, policy_version 1254704 (0.0006) [2023-12-27 14:20:29,964][20347] Updated weights for policy 0, policy_version 1254714 (0.0011) [2023-12-27 14:20:30,436][20381] Updated weights for policy 1, policy_version 1253633 (0.0008) [2023-12-27 14:20:30,490][20381] Updated weights for policy 1, policy_version 1253643 (0.0007) [2023-12-27 14:20:30,543][20381] Updated weights for policy 1, policy_version 1253653 (0.0008) [2023-12-27 14:20:30,597][20347] Updated weights for policy 0, policy_version 1254724 (0.0008) [2023-12-27 14:20:30,652][20347] Updated weights for policy 0, policy_version 1254734 (0.0010) [2023-12-27 14:20:30,697][20347] Updated weights for policy 0, policy_version 1254744 (0.0010) [2023-12-27 14:20:31,283][20381] Updated weights for policy 1, policy_version 1253663 (0.0008) [2023-12-27 14:20:31,339][20381] Updated weights for policy 1, policy_version 1253673 (0.0008) [2023-12-27 14:20:31,401][20381] Updated weights for policy 1, policy_version 1253683 (0.0008) [2023-12-27 14:20:31,411][20347] Updated weights for policy 0, policy_version 1254754 (0.0010) [2023-12-27 14:20:31,468][20347] Updated weights for policy 0, policy_version 1254764 (0.0006) [2023-12-27 14:20:31,531][20347] Updated weights for policy 0, policy_version 1254774 (0.0008) [2023-12-27 14:20:31,590][20347] Updated weights for policy 0, policy_version 1254784 (0.0008) [2023-12-27 14:20:32,090][20381] Updated weights for policy 1, policy_version 1253693 (0.0005) [2023-12-27 14:20:32,150][20381] Updated weights for policy 1, policy_version 1253703 (0.0006) [2023-12-27 14:20:32,217][20381] Updated weights for policy 1, policy_version 1253713 (0.0005) [2023-12-27 14:20:32,340][20347] Updated weights for policy 0, policy_version 1254794 (0.0006) [2023-12-27 14:20:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 642260992. Throughput: 0: 10244.3, 1: 10140.9. Samples: 642272048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:32,363][19170] Avg episode reward: [(0, '767.140'), (1, '542.630')] [2023-12-27 14:20:32,404][20347] Updated weights for policy 0, policy_version 1254804 (0.0007) [2023-12-27 14:20:32,463][20347] Updated weights for policy 0, policy_version 1254814 (0.0007) [2023-12-27 14:20:32,912][20381] Updated weights for policy 1, policy_version 1253723 (0.0007) [2023-12-27 14:20:32,967][20381] Updated weights for policy 1, policy_version 1253733 (0.0008) [2023-12-27 14:20:33,015][20381] Updated weights for policy 1, policy_version 1253743 (0.0007) [2023-12-27 14:20:33,080][20347] Updated weights for policy 0, policy_version 1254824 (0.0007) [2023-12-27 14:20:33,123][20347] Updated weights for policy 0, policy_version 1254834 (0.0008) [2023-12-27 14:20:33,173][20347] Updated weights for policy 0, policy_version 1254844 (0.0008) [2023-12-27 14:20:33,692][20381] Updated weights for policy 1, policy_version 1253753 (0.0007) [2023-12-27 14:20:33,745][20292] KL-divergence is very high: 157.5583 [2023-12-27 14:20:33,747][20381] Updated weights for policy 1, policy_version 1253763 (0.0009) [2023-12-27 14:20:33,750][20292] KL-divergence is very high: 182.2915 [2023-12-27 14:20:33,788][20292] KL-divergence is very high: 105.0830 [2023-12-27 14:20:33,793][20292] KL-divergence is very high: 116.2378 [2023-12-27 14:20:33,799][20381] Updated weights for policy 1, policy_version 1253773 (0.0009) [2023-12-27 14:20:33,844][20347] Updated weights for policy 0, policy_version 1254854 (0.0006) [2023-12-27 14:20:33,856][20381] Updated weights for policy 1, policy_version 1253783 (0.0008) [2023-12-27 14:20:33,893][20347] Updated weights for policy 0, policy_version 1254864 (0.0006) [2023-12-27 14:20:33,952][20347] Updated weights for policy 0, policy_version 1254874 (0.0010) [2023-12-27 14:20:34,581][20381] Updated weights for policy 1, policy_version 1253793 (0.0008) [2023-12-27 14:20:34,644][20381] Updated weights for policy 1, policy_version 1253803 (0.0008) [2023-12-27 14:20:34,654][20347] Updated weights for policy 0, policy_version 1254884 (0.0009) [2023-12-27 14:20:34,702][20381] Updated weights for policy 1, policy_version 1253813 (0.0007) [2023-12-27 14:20:34,711][20347] Updated weights for policy 0, policy_version 1254894 (0.0007) [2023-12-27 14:20:34,767][20347] Updated weights for policy 0, policy_version 1254904 (0.0007) [2023-12-27 14:20:35,304][20381] Updated weights for policy 1, policy_version 1253823 (0.0007) [2023-12-27 14:20:35,352][20381] Updated weights for policy 1, policy_version 1253833 (0.0007) [2023-12-27 14:20:35,400][20381] Updated weights for policy 1, policy_version 1253843 (0.0008) [2023-12-27 14:20:35,496][20347] Updated weights for policy 0, policy_version 1254914 (0.0007) [2023-12-27 14:20:35,548][20347] Updated weights for policy 0, policy_version 1254924 (0.0007) [2023-12-27 14:20:35,593][20347] Updated weights for policy 0, policy_version 1254934 (0.0007) [2023-12-27 14:20:35,642][20347] Updated weights for policy 0, policy_version 1254944 (0.0008) [2023-12-27 14:20:36,089][20381] Updated weights for policy 1, policy_version 1253853 (0.0008) [2023-12-27 14:20:36,154][20381] Updated weights for policy 1, policy_version 1253863 (0.0008) [2023-12-27 14:20:36,214][20381] Updated weights for policy 1, policy_version 1253873 (0.0008) [2023-12-27 14:20:36,378][20347] Updated weights for policy 0, policy_version 1254954 (0.0008) [2023-12-27 14:20:36,428][20347] Updated weights for policy 0, policy_version 1254964 (0.0007) [2023-12-27 14:20:36,484][20347] Updated weights for policy 0, policy_version 1254974 (0.0008) [2023-12-27 14:20:36,918][20381] Updated weights for policy 1, policy_version 1253883 (0.0008) [2023-12-27 14:20:36,978][20381] Updated weights for policy 1, policy_version 1253893 (0.0007) [2023-12-27 14:20:37,034][20381] Updated weights for policy 1, policy_version 1253903 (0.0008) [2023-12-27 14:20:37,193][20347] Updated weights for policy 0, policy_version 1254984 (0.0008) [2023-12-27 14:20:37,246][20347] Updated weights for policy 0, policy_version 1254994 (0.0008) [2023-12-27 14:20:37,305][20347] Updated weights for policy 0, policy_version 1255004 (0.0006) [2023-12-27 14:20:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 642367488. Throughput: 0: 10270.6, 1: 10149.5. Samples: 642333808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:37,363][19170] Avg episode reward: [(0, '735.960'), (1, '594.270')] [2023-12-27 14:20:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001253912_321028096.pth... [2023-12-27 14:20:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001255008_321339392.pth... [2023-12-27 14:20:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001252728_320724992.pth [2023-12-27 14:20:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001253792_321028096.pth [2023-12-27 14:20:37,732][20381] Updated weights for policy 1, policy_version 1253913 (0.0008) [2023-12-27 14:20:37,776][20381] Updated weights for policy 1, policy_version 1253923 (0.0007) [2023-12-27 14:20:37,828][20381] Updated weights for policy 1, policy_version 1253933 (0.0008) [2023-12-27 14:20:37,880][20381] Updated weights for policy 1, policy_version 1253943 (0.0007) [2023-12-27 14:20:37,991][20347] Updated weights for policy 0, policy_version 1255014 (0.0007) [2023-12-27 14:20:38,047][20347] Updated weights for policy 0, policy_version 1255024 (0.0007) [2023-12-27 14:20:38,104][20347] Updated weights for policy 0, policy_version 1255034 (0.0008) [2023-12-27 14:20:38,594][20381] Updated weights for policy 1, policy_version 1253953 (0.0008) [2023-12-27 14:20:38,653][20381] Updated weights for policy 1, policy_version 1253963 (0.0009) [2023-12-27 14:20:38,709][20381] Updated weights for policy 1, policy_version 1253973 (0.0008) [2023-12-27 14:20:38,796][20347] Updated weights for policy 0, policy_version 1255044 (0.0008) [2023-12-27 14:20:38,850][20347] Updated weights for policy 0, policy_version 1255054 (0.0007) [2023-12-27 14:20:38,900][20347] Updated weights for policy 0, policy_version 1255064 (0.0007) [2023-12-27 14:20:39,498][20381] Updated weights for policy 1, policy_version 1253983 (0.0008) [2023-12-27 14:20:39,559][20381] Updated weights for policy 1, policy_version 1253993 (0.0009) [2023-12-27 14:20:39,572][20347] Updated weights for policy 0, policy_version 1255074 (0.0008) [2023-12-27 14:20:39,621][20291] KL-divergence is very high: 157.0153 [2023-12-27 14:20:39,624][20381] Updated weights for policy 1, policy_version 1254003 (0.0009) [2023-12-27 14:20:39,627][20291] KL-divergence is very high: 125.5921 [2023-12-27 14:20:39,634][20291] KL-divergence is very high: 109.1998 [2023-12-27 14:20:39,641][20347] Updated weights for policy 0, policy_version 1255084 (0.0007) [2023-12-27 14:20:39,701][20347] Updated weights for policy 0, policy_version 1255094 (0.0008) [2023-12-27 14:20:39,763][20347] Updated weights for policy 0, policy_version 1255104 (0.0008) [2023-12-27 14:20:40,343][20381] Updated weights for policy 1, policy_version 1254013 (0.0008) [2023-12-27 14:20:40,405][20381] Updated weights for policy 1, policy_version 1254023 (0.0007) [2023-12-27 14:20:40,458][20381] Updated weights for policy 1, policy_version 1254033 (0.0007) [2023-12-27 14:20:40,468][20347] Updated weights for policy 0, policy_version 1255114 (0.0006) [2023-12-27 14:20:40,528][20347] Updated weights for policy 0, policy_version 1255124 (0.0007) [2023-12-27 14:20:40,585][20347] Updated weights for policy 0, policy_version 1255134 (0.0008) [2023-12-27 14:20:41,055][20381] Updated weights for policy 1, policy_version 1254043 (0.0008) [2023-12-27 14:20:41,118][20381] Updated weights for policy 1, policy_version 1254053 (0.0011) [2023-12-27 14:20:41,193][20381] Updated weights for policy 1, policy_version 1254063 (0.0011) [2023-12-27 14:20:41,322][20347] Updated weights for policy 0, policy_version 1255144 (0.0008) [2023-12-27 14:20:41,399][20347] Updated weights for policy 0, policy_version 1255154 (0.0008) [2023-12-27 14:20:41,462][20347] Updated weights for policy 0, policy_version 1255164 (0.0007) [2023-12-27 14:20:41,975][20381] Updated weights for policy 1, policy_version 1254073 (0.0009) [2023-12-27 14:20:42,041][20381] Updated weights for policy 1, policy_version 1254083 (0.0006) [2023-12-27 14:20:42,107][20381] Updated weights for policy 1, policy_version 1254093 (0.0006) [2023-12-27 14:20:42,161][20381] Updated weights for policy 1, policy_version 1254103 (0.0007) [2023-12-27 14:20:42,166][20347] Updated weights for policy 0, policy_version 1255174 (0.0007) [2023-12-27 14:20:42,232][20347] Updated weights for policy 0, policy_version 1255184 (0.0007) [2023-12-27 14:20:42,296][20347] Updated weights for policy 0, policy_version 1255194 (0.0007) [2023-12-27 14:20:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 642465792. Throughput: 0: 10275.0, 1: 10112.6. Samples: 642452956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:42,363][19170] Avg episode reward: [(0, '728.150'), (1, '452.420')] [2023-12-27 14:20:42,825][20381] Updated weights for policy 1, policy_version 1254113 (0.0010) [2023-12-27 14:20:42,887][20381] Updated weights for policy 1, policy_version 1254123 (0.0010) [2023-12-27 14:20:42,934][20347] Updated weights for policy 0, policy_version 1255204 (0.0007) [2023-12-27 14:20:42,943][20381] Updated weights for policy 1, policy_version 1254133 (0.0011) [2023-12-27 14:20:42,985][20347] Updated weights for policy 0, policy_version 1255214 (0.0006) [2023-12-27 14:20:43,037][20347] Updated weights for policy 0, policy_version 1255224 (0.0008) [2023-12-27 14:20:43,577][20381] Updated weights for policy 1, policy_version 1254143 (0.0007) [2023-12-27 14:20:43,639][20381] Updated weights for policy 1, policy_version 1254153 (0.0005) [2023-12-27 14:20:43,697][20381] Updated weights for policy 1, policy_version 1254163 (0.0006) [2023-12-27 14:20:43,767][20347] Updated weights for policy 0, policy_version 1255234 (0.0009) [2023-12-27 14:20:43,830][20347] Updated weights for policy 0, policy_version 1255244 (0.0007) [2023-12-27 14:20:43,882][20347] Updated weights for policy 0, policy_version 1255254 (0.0008) [2023-12-27 14:20:43,936][20347] Updated weights for policy 0, policy_version 1255264 (0.0007) [2023-12-27 14:20:44,331][20381] Updated weights for policy 1, policy_version 1254173 (0.0008) [2023-12-27 14:20:44,391][20381] Updated weights for policy 1, policy_version 1254183 (0.0008) [2023-12-27 14:20:44,456][20381] Updated weights for policy 1, policy_version 1254193 (0.0008) [2023-12-27 14:20:44,630][20347] Updated weights for policy 0, policy_version 1255274 (0.0008) [2023-12-27 14:20:44,689][20347] Updated weights for policy 0, policy_version 1255284 (0.0008) [2023-12-27 14:20:44,754][20347] Updated weights for policy 0, policy_version 1255294 (0.0007) [2023-12-27 14:20:45,140][20381] Updated weights for policy 1, policy_version 1254203 (0.0008) [2023-12-27 14:20:45,203][20381] Updated weights for policy 1, policy_version 1254213 (0.0008) [2023-12-27 14:20:45,262][20381] Updated weights for policy 1, policy_version 1254223 (0.0007) [2023-12-27 14:20:45,462][20347] Updated weights for policy 0, policy_version 1255304 (0.0007) [2023-12-27 14:20:45,512][20347] Updated weights for policy 0, policy_version 1255314 (0.0009) [2023-12-27 14:20:45,567][20347] Updated weights for policy 0, policy_version 1255324 (0.0008) [2023-12-27 14:20:45,828][20381] Updated weights for policy 1, policy_version 1254233 (0.0007) [2023-12-27 14:20:45,884][20381] Updated weights for policy 1, policy_version 1254243 (0.0006) [2023-12-27 14:20:45,950][20381] Updated weights for policy 1, policy_version 1254253 (0.0005) [2023-12-27 14:20:46,004][20381] Updated weights for policy 1, policy_version 1254263 (0.0005) [2023-12-27 14:20:46,205][20347] Updated weights for policy 0, policy_version 1255334 (0.0006) [2023-12-27 14:20:46,259][20347] Updated weights for policy 0, policy_version 1255344 (0.0005) [2023-12-27 14:20:46,315][20347] Updated weights for policy 0, policy_version 1255354 (0.0005) [2023-12-27 14:20:46,555][20381] Updated weights for policy 1, policy_version 1254273 (0.0007) [2023-12-27 14:20:46,611][20381] Updated weights for policy 1, policy_version 1254283 (0.0006) [2023-12-27 14:20:46,670][20381] Updated weights for policy 1, policy_version 1254293 (0.0007) [2023-12-27 14:20:46,932][20347] Updated weights for policy 0, policy_version 1255364 (0.0006) [2023-12-27 14:20:46,981][20347] Updated weights for policy 0, policy_version 1255374 (0.0007) [2023-12-27 14:20:47,029][20347] Updated weights for policy 0, policy_version 1255384 (0.0007) [2023-12-27 14:20:47,344][20381] Updated weights for policy 1, policy_version 1254303 (0.0006) [2023-12-27 14:20:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 642564096. Throughput: 0: 10240.5, 1: 10168.4. Samples: 642576728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:47,363][19170] Avg episode reward: [(0, '664.540'), (1, '232.840')] [2023-12-27 14:20:47,401][20381] Updated weights for policy 1, policy_version 1254313 (0.0007) [2023-12-27 14:20:47,462][20381] Updated weights for policy 1, policy_version 1254323 (0.0007) [2023-12-27 14:20:47,750][20347] Updated weights for policy 0, policy_version 1255394 (0.0008) [2023-12-27 14:20:47,805][20347] Updated weights for policy 0, policy_version 1255404 (0.0007) [2023-12-27 14:20:47,860][20347] Updated weights for policy 0, policy_version 1255414 (0.0007) [2023-12-27 14:20:47,913][20347] Updated weights for policy 0, policy_version 1255424 (0.0007) [2023-12-27 14:20:48,129][20381] Updated weights for policy 1, policy_version 1254333 (0.0007) [2023-12-27 14:20:48,183][20381] Updated weights for policy 1, policy_version 1254343 (0.0007) [2023-12-27 14:20:48,238][20381] Updated weights for policy 1, policy_version 1254353 (0.0008) [2023-12-27 14:20:48,590][20347] Updated weights for policy 0, policy_version 1255434 (0.0009) [2023-12-27 14:20:48,649][20347] Updated weights for policy 0, policy_version 1255444 (0.0007) [2023-12-27 14:20:48,712][20347] Updated weights for policy 0, policy_version 1255454 (0.0007) [2023-12-27 14:20:48,964][20381] Updated weights for policy 1, policy_version 1254363 (0.0007) [2023-12-27 14:20:49,022][20381] Updated weights for policy 1, policy_version 1254373 (0.0008) [2023-12-27 14:20:49,077][20381] Updated weights for policy 1, policy_version 1254383 (0.0008) [2023-12-27 14:20:49,428][20347] Updated weights for policy 0, policy_version 1255464 (0.0008) [2023-12-27 14:20:49,490][20347] Updated weights for policy 0, policy_version 1255474 (0.0008) [2023-12-27 14:20:49,543][20291] KL-divergence is very high: 127.3500 [2023-12-27 14:20:49,549][20347] Updated weights for policy 0, policy_version 1255484 (0.0008) [2023-12-27 14:20:49,821][20381] Updated weights for policy 1, policy_version 1254393 (0.0007) [2023-12-27 14:20:49,882][20381] Updated weights for policy 1, policy_version 1254403 (0.0008) [2023-12-27 14:20:49,943][20381] Updated weights for policy 1, policy_version 1254413 (0.0008) [2023-12-27 14:20:50,013][20381] Updated weights for policy 1, policy_version 1254423 (0.0007) [2023-12-27 14:20:50,289][20347] Updated weights for policy 0, policy_version 1255494 (0.0009) [2023-12-27 14:20:50,344][20347] Updated weights for policy 0, policy_version 1255504 (0.0010) [2023-12-27 14:20:50,402][20347] Updated weights for policy 0, policy_version 1255514 (0.0008) [2023-12-27 14:20:50,704][20381] Updated weights for policy 1, policy_version 1254433 (0.0007) [2023-12-27 14:20:50,765][20381] Updated weights for policy 1, policy_version 1254443 (0.0006) [2023-12-27 14:20:50,821][20381] Updated weights for policy 1, policy_version 1254453 (0.0007) [2023-12-27 14:20:51,076][20347] Updated weights for policy 0, policy_version 1255524 (0.0010) [2023-12-27 14:20:51,132][20347] Updated weights for policy 0, policy_version 1255534 (0.0010) [2023-12-27 14:20:51,195][20347] Updated weights for policy 0, policy_version 1255544 (0.0011) [2023-12-27 14:20:51,507][20381] Updated weights for policy 1, policy_version 1254463 (0.0008) [2023-12-27 14:20:51,572][20381] Updated weights for policy 1, policy_version 1254473 (0.0009) [2023-12-27 14:20:51,641][20381] Updated weights for policy 1, policy_version 1254483 (0.0007) [2023-12-27 14:20:51,945][20347] Updated weights for policy 0, policy_version 1255554 (0.0010) [2023-12-27 14:20:51,993][20347] Updated weights for policy 0, policy_version 1255564 (0.0007) [2023-12-27 14:20:52,051][20347] Updated weights for policy 0, policy_version 1255574 (0.0007) [2023-12-27 14:20:52,113][20347] Updated weights for policy 0, policy_version 1255584 (0.0008) [2023-12-27 14:20:52,282][20381] Updated weights for policy 1, policy_version 1254493 (0.0010) [2023-12-27 14:20:52,342][20381] Updated weights for policy 1, policy_version 1254503 (0.0007) [2023-12-27 14:20:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 642662400. Throughput: 0: 10233.2, 1: 10106.2. Samples: 642635984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:52,364][19170] Avg episode reward: [(0, '648.010'), (1, '68.540')] [2023-12-27 14:20:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001255584_321486848.pth... [2023-12-27 14:20:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001254368_321175552.pth [2023-12-27 14:20:52,413][20381] Updated weights for policy 1, policy_version 1254513 (0.0009) [2023-12-27 14:20:52,448][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001254520_321183744.pth... [2023-12-27 14:20:52,451][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001253336_320880640.pth [2023-12-27 14:20:52,819][20347] Updated weights for policy 0, policy_version 1255594 (0.0006) [2023-12-27 14:20:52,881][20347] Updated weights for policy 0, policy_version 1255604 (0.0008) [2023-12-27 14:20:52,944][20347] Updated weights for policy 0, policy_version 1255614 (0.0008) [2023-12-27 14:20:53,081][20381] Updated weights for policy 1, policy_version 1254523 (0.0008) [2023-12-27 14:20:53,135][20381] Updated weights for policy 1, policy_version 1254533 (0.0007) [2023-12-27 14:20:53,193][20381] Updated weights for policy 1, policy_version 1254543 (0.0005) [2023-12-27 14:20:53,647][20347] Updated weights for policy 0, policy_version 1255624 (0.0007) [2023-12-27 14:20:53,702][20347] Updated weights for policy 0, policy_version 1255634 (0.0008) [2023-12-27 14:20:53,754][20347] Updated weights for policy 0, policy_version 1255644 (0.0007) [2023-12-27 14:20:53,811][20381] Updated weights for policy 1, policy_version 1254553 (0.0006) [2023-12-27 14:20:53,868][20381] Updated weights for policy 1, policy_version 1254563 (0.0005) [2023-12-27 14:20:53,916][20381] Updated weights for policy 1, policy_version 1254573 (0.0005) [2023-12-27 14:20:53,972][20381] Updated weights for policy 1, policy_version 1254583 (0.0005) [2023-12-27 14:20:54,478][20347] Updated weights for policy 0, policy_version 1255654 (0.0009) [2023-12-27 14:20:54,531][20347] Updated weights for policy 0, policy_version 1255664 (0.0010) [2023-12-27 14:20:54,565][20381] Updated weights for policy 1, policy_version 1254593 (0.0010) [2023-12-27 14:20:54,578][20347] Updated weights for policy 0, policy_version 1255674 (0.0010) [2023-12-27 14:20:54,621][20381] Updated weights for policy 1, policy_version 1254603 (0.0011) [2023-12-27 14:20:54,676][20381] Updated weights for policy 1, policy_version 1254613 (0.0011) [2023-12-27 14:20:55,241][20347] Updated weights for policy 0, policy_version 1255684 (0.0009) [2023-12-27 14:20:55,299][20347] Updated weights for policy 0, policy_version 1255694 (0.0007) [2023-12-27 14:20:55,318][20381] Updated weights for policy 1, policy_version 1254623 (0.0011) [2023-12-27 14:20:55,356][20347] Updated weights for policy 0, policy_version 1255704 (0.0008) [2023-12-27 14:20:55,378][20381] Updated weights for policy 1, policy_version 1254633 (0.0011) [2023-12-27 14:20:55,433][20381] Updated weights for policy 1, policy_version 1254643 (0.0010) [2023-12-27 14:20:55,989][20347] Updated weights for policy 0, policy_version 1255714 (0.0006) [2023-12-27 14:20:56,047][20347] Updated weights for policy 0, policy_version 1255724 (0.0007) [2023-12-27 14:20:56,094][20347] Updated weights for policy 0, policy_version 1255734 (0.0007) [2023-12-27 14:20:56,128][20381] Updated weights for policy 1, policy_version 1254653 (0.0010) [2023-12-27 14:20:56,149][20347] Updated weights for policy 0, policy_version 1255744 (0.0007) [2023-12-27 14:20:56,184][20381] Updated weights for policy 1, policy_version 1254663 (0.0008) [2023-12-27 14:20:56,244][20381] Updated weights for policy 1, policy_version 1254673 (0.0007) [2023-12-27 14:20:56,822][20381] Updated weights for policy 1, policy_version 1254683 (0.0008) [2023-12-27 14:20:56,834][20347] Updated weights for policy 0, policy_version 1255754 (0.0005) [2023-12-27 14:20:56,880][20381] Updated weights for policy 1, policy_version 1254693 (0.0010) [2023-12-27 14:20:56,899][20347] Updated weights for policy 0, policy_version 1255764 (0.0005) [2023-12-27 14:20:56,933][20381] Updated weights for policy 1, policy_version 1254703 (0.0010) [2023-12-27 14:20:56,954][20347] Updated weights for policy 0, policy_version 1255774 (0.0005) [2023-12-27 14:20:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 642768896. Throughput: 0: 10210.9, 1: 10150.0. Samples: 642758912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:20:57,364][19170] Avg episode reward: [(0, '721.520'), (1, '40.340')] [2023-12-27 14:20:57,555][20381] Updated weights for policy 1, policy_version 1254713 (0.0010) [2023-12-27 14:20:57,615][20381] Updated weights for policy 1, policy_version 1254723 (0.0005) [2023-12-27 14:20:57,629][20347] Updated weights for policy 0, policy_version 1255784 (0.0007) [2023-12-27 14:20:57,674][20381] Updated weights for policy 1, policy_version 1254733 (0.0007) [2023-12-27 14:20:57,688][20347] Updated weights for policy 0, policy_version 1255794 (0.0007) [2023-12-27 14:20:57,724][20381] Updated weights for policy 1, policy_version 1254743 (0.0010) [2023-12-27 14:20:57,747][20347] Updated weights for policy 0, policy_version 1255804 (0.0008) [2023-12-27 14:20:58,354][20381] Updated weights for policy 1, policy_version 1254753 (0.0011) [2023-12-27 14:20:58,413][20381] Updated weights for policy 1, policy_version 1254763 (0.0010) [2023-12-27 14:20:58,453][20347] Updated weights for policy 0, policy_version 1255814 (0.0008) [2023-12-27 14:20:58,477][20381] Updated weights for policy 1, policy_version 1254773 (0.0010) [2023-12-27 14:20:58,512][20347] Updated weights for policy 0, policy_version 1255824 (0.0008) [2023-12-27 14:20:58,577][20347] Updated weights for policy 0, policy_version 1255834 (0.0008) [2023-12-27 14:20:59,269][20381] Updated weights for policy 1, policy_version 1254783 (0.0010) [2023-12-27 14:20:59,333][20381] Updated weights for policy 1, policy_version 1254793 (0.0007) [2023-12-27 14:20:59,344][20347] Updated weights for policy 0, policy_version 1255844 (0.0008) [2023-12-27 14:20:59,400][20381] Updated weights for policy 1, policy_version 1254803 (0.0007) [2023-12-27 14:20:59,402][20347] Updated weights for policy 0, policy_version 1255854 (0.0009) [2023-12-27 14:20:59,461][20347] Updated weights for policy 0, policy_version 1255864 (0.0007) [2023-12-27 14:21:00,084][20381] Updated weights for policy 1, policy_version 1254813 (0.0007) [2023-12-27 14:21:00,101][20347] Updated weights for policy 0, policy_version 1255874 (0.0006) [2023-12-27 14:21:00,141][20381] Updated weights for policy 1, policy_version 1254823 (0.0006) [2023-12-27 14:21:00,154][20347] Updated weights for policy 0, policy_version 1255884 (0.0006) [2023-12-27 14:21:00,197][20381] Updated weights for policy 1, policy_version 1254833 (0.0008) [2023-12-27 14:21:00,207][20347] Updated weights for policy 0, policy_version 1255894 (0.0006) [2023-12-27 14:21:00,262][20347] Updated weights for policy 0, policy_version 1255904 (0.0006) [2023-12-27 14:21:00,811][20381] Updated weights for policy 1, policy_version 1254843 (0.0007) [2023-12-27 14:21:00,868][20381] Updated weights for policy 1, policy_version 1254853 (0.0010) [2023-12-27 14:21:00,925][20381] Updated weights for policy 1, policy_version 1254863 (0.0010) [2023-12-27 14:21:00,998][20347] Updated weights for policy 0, policy_version 1255914 (0.0005) [2023-12-27 14:21:01,056][20347] Updated weights for policy 0, policy_version 1255924 (0.0007) [2023-12-27 14:21:01,118][20347] Updated weights for policy 0, policy_version 1255934 (0.0010) [2023-12-27 14:21:01,610][20381] Updated weights for policy 1, policy_version 1254873 (0.0010) [2023-12-27 14:21:01,670][20381] Updated weights for policy 1, policy_version 1254883 (0.0007) [2023-12-27 14:21:01,727][20381] Updated weights for policy 1, policy_version 1254893 (0.0008) [2023-12-27 14:21:01,795][20381] Updated weights for policy 1, policy_version 1254903 (0.0007) [2023-12-27 14:21:01,884][20347] Updated weights for policy 0, policy_version 1255944 (0.0010) [2023-12-27 14:21:01,943][20347] Updated weights for policy 0, policy_version 1255954 (0.0010) [2023-12-27 14:21:02,004][20347] Updated weights for policy 0, policy_version 1255964 (0.0007) [2023-12-27 14:21:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 642867200. Throughput: 0: 10194.7, 1: 10188.8. Samples: 642880236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:02,363][19170] Avg episode reward: [(0, '682.850'), (1, '40.190')] [2023-12-27 14:21:02,489][20381] Updated weights for policy 1, policy_version 1254913 (0.0007) [2023-12-27 14:21:02,541][20381] Updated weights for policy 1, policy_version 1254923 (0.0007) [2023-12-27 14:21:02,590][20381] Updated weights for policy 1, policy_version 1254933 (0.0009) [2023-12-27 14:21:02,609][20347] Updated weights for policy 0, policy_version 1255974 (0.0008) [2023-12-27 14:21:02,675][20347] Updated weights for policy 0, policy_version 1255984 (0.0008) [2023-12-27 14:21:02,745][20347] Updated weights for policy 0, policy_version 1255994 (0.0008) [2023-12-27 14:21:03,309][20381] Updated weights for policy 1, policy_version 1254943 (0.0006) [2023-12-27 14:21:03,349][20347] Updated weights for policy 0, policy_version 1256004 (0.0007) [2023-12-27 14:21:03,372][20381] Updated weights for policy 1, policy_version 1254953 (0.0005) [2023-12-27 14:21:03,405][20347] Updated weights for policy 0, policy_version 1256014 (0.0010) [2023-12-27 14:21:03,430][20381] Updated weights for policy 1, policy_version 1254963 (0.0006) [2023-12-27 14:21:03,456][20347] Updated weights for policy 0, policy_version 1256024 (0.0010) [2023-12-27 14:21:04,029][20381] Updated weights for policy 1, policy_version 1254973 (0.0007) [2023-12-27 14:21:04,090][20381] Updated weights for policy 1, policy_version 1254983 (0.0008) [2023-12-27 14:21:04,119][20347] Updated weights for policy 0, policy_version 1256034 (0.0009) [2023-12-27 14:21:04,149][20381] Updated weights for policy 1, policy_version 1254993 (0.0008) [2023-12-27 14:21:04,181][20347] Updated weights for policy 0, policy_version 1256044 (0.0007) [2023-12-27 14:21:04,239][20347] Updated weights for policy 0, policy_version 1256054 (0.0010) [2023-12-27 14:21:04,296][20347] Updated weights for policy 0, policy_version 1256064 (0.0010) [2023-12-27 14:21:04,874][20381] Updated weights for policy 1, policy_version 1255003 (0.0006) [2023-12-27 14:21:04,932][20381] Updated weights for policy 1, policy_version 1255013 (0.0007) [2023-12-27 14:21:04,949][20347] Updated weights for policy 0, policy_version 1256074 (0.0010) [2023-12-27 14:21:04,991][20381] Updated weights for policy 1, policy_version 1255023 (0.0007) [2023-12-27 14:21:05,001][20347] Updated weights for policy 0, policy_version 1256084 (0.0010) [2023-12-27 14:21:05,050][20347] Updated weights for policy 0, policy_version 1256094 (0.0009) [2023-12-27 14:21:05,567][20381] Updated weights for policy 1, policy_version 1255033 (0.0006) [2023-12-27 14:21:05,623][20381] Updated weights for policy 1, policy_version 1255043 (0.0007) [2023-12-27 14:21:05,643][20347] Updated weights for policy 0, policy_version 1256104 (0.0009) [2023-12-27 14:21:05,682][20381] Updated weights for policy 1, policy_version 1255053 (0.0005) [2023-12-27 14:21:05,695][20347] Updated weights for policy 0, policy_version 1256114 (0.0010) [2023-12-27 14:21:05,737][20381] Updated weights for policy 1, policy_version 1255063 (0.0005) [2023-12-27 14:21:05,754][20347] Updated weights for policy 0, policy_version 1256124 (0.0011) [2023-12-27 14:21:06,405][20347] Updated weights for policy 0, policy_version 1256134 (0.0011) [2023-12-27 14:21:06,409][20381] Updated weights for policy 1, policy_version 1255073 (0.0009) [2023-12-27 14:21:06,461][20347] Updated weights for policy 0, policy_version 1256144 (0.0010) [2023-12-27 14:21:06,470][20381] Updated weights for policy 1, policy_version 1255083 (0.0010) [2023-12-27 14:21:06,519][20347] Updated weights for policy 0, policy_version 1256154 (0.0009) [2023-12-27 14:21:06,531][20381] Updated weights for policy 1, policy_version 1255093 (0.0011) [2023-12-27 14:21:07,173][20347] Updated weights for policy 0, policy_version 1256164 (0.0010) [2023-12-27 14:21:07,206][20381] Updated weights for policy 1, policy_version 1255103 (0.0011) [2023-12-27 14:21:07,222][20347] Updated weights for policy 0, policy_version 1256174 (0.0010) [2023-12-27 14:21:07,259][20381] Updated weights for policy 1, policy_version 1255113 (0.0011) [2023-12-27 14:21:07,264][20291] KL-divergence is very high: 385.2014 [2023-12-27 14:21:07,270][20347] Updated weights for policy 0, policy_version 1256184 (0.0010) [2023-12-27 14:21:07,277][20291] KL-divergence is very high: 378.0436 [2023-12-27 14:21:07,310][20291] KL-divergence is very high: 178.0775 [2023-12-27 14:21:07,315][20381] Updated weights for policy 1, policy_version 1255123 (0.0011) [2023-12-27 14:21:07,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 642981888. Throughput: 0: 10217.2, 1: 10185.3. Samples: 642942872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:07,363][19170] Avg episode reward: [(0, '780.490'), (1, '83.450')] [2023-12-27 14:21:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001255128_321339392.pth... [2023-12-27 14:21:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001256192_321642496.pth... [2023-12-27 14:21:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001255008_321339392.pth [2023-12-27 14:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001253912_321028096.pth [2023-12-27 14:21:07,841][20347] Updated weights for policy 0, policy_version 1256194 (0.0009) [2023-12-27 14:21:07,900][20347] Updated weights for policy 0, policy_version 1256204 (0.0008) [2023-12-27 14:21:07,958][20347] Updated weights for policy 0, policy_version 1256214 (0.0010) [2023-12-27 14:21:08,012][20347] Updated weights for policy 0, policy_version 1256224 (0.0010) [2023-12-27 14:21:08,024][20381] Updated weights for policy 1, policy_version 1255133 (0.0011) [2023-12-27 14:21:08,085][20381] Updated weights for policy 1, policy_version 1255143 (0.0011) [2023-12-27 14:21:08,136][20381] Updated weights for policy 1, policy_version 1255153 (0.0010) [2023-12-27 14:21:08,622][20347] Updated weights for policy 0, policy_version 1256234 (0.0010) [2023-12-27 14:21:08,684][20347] Updated weights for policy 0, policy_version 1256244 (0.0010) [2023-12-27 14:21:08,745][20347] Updated weights for policy 0, policy_version 1256254 (0.0011) [2023-12-27 14:21:08,850][20381] Updated weights for policy 1, policy_version 1255163 (0.0009) [2023-12-27 14:21:08,898][20381] Updated weights for policy 1, policy_version 1255173 (0.0007) [2023-12-27 14:21:08,955][20381] Updated weights for policy 1, policy_version 1255183 (0.0007) [2023-12-27 14:21:09,439][20347] Updated weights for policy 0, policy_version 1256264 (0.0009) [2023-12-27 14:21:09,504][20347] Updated weights for policy 0, policy_version 1256274 (0.0011) [2023-12-27 14:21:09,561][20347] Updated weights for policy 0, policy_version 1256284 (0.0011) [2023-12-27 14:21:09,735][20381] Updated weights for policy 1, policy_version 1255193 (0.0007) [2023-12-27 14:21:09,796][20381] Updated weights for policy 1, policy_version 1255203 (0.0007) [2023-12-27 14:21:09,865][20381] Updated weights for policy 1, policy_version 1255213 (0.0008) [2023-12-27 14:21:09,930][20381] Updated weights for policy 1, policy_version 1255223 (0.0008) [2023-12-27 14:21:10,294][20347] Updated weights for policy 0, policy_version 1256294 (0.0009) [2023-12-27 14:21:10,359][20347] Updated weights for policy 0, policy_version 1256304 (0.0007) [2023-12-27 14:21:10,420][20347] Updated weights for policy 0, policy_version 1256314 (0.0007) [2023-12-27 14:21:10,564][20381] Updated weights for policy 1, policy_version 1255233 (0.0007) [2023-12-27 14:21:10,623][20381] Updated weights for policy 1, policy_version 1255243 (0.0008) [2023-12-27 14:21:10,683][20381] Updated weights for policy 1, policy_version 1255253 (0.0009) [2023-12-27 14:21:11,102][20347] Updated weights for policy 0, policy_version 1256324 (0.0010) [2023-12-27 14:21:11,175][20347] Updated weights for policy 0, policy_version 1256334 (0.0008) [2023-12-27 14:21:11,240][20347] Updated weights for policy 0, policy_version 1256344 (0.0006) [2023-12-27 14:21:11,456][20381] Updated weights for policy 1, policy_version 1255263 (0.0008) [2023-12-27 14:21:11,515][20381] Updated weights for policy 1, policy_version 1255273 (0.0006) [2023-12-27 14:21:11,574][20381] Updated weights for policy 1, policy_version 1255283 (0.0006) [2023-12-27 14:21:11,995][20347] Updated weights for policy 0, policy_version 1256354 (0.0008) [2023-12-27 14:21:12,050][20347] Updated weights for policy 0, policy_version 1256364 (0.0008) [2023-12-27 14:21:12,105][20347] Updated weights for policy 0, policy_version 1256374 (0.0008) [2023-12-27 14:21:12,161][20347] Updated weights for policy 0, policy_version 1256384 (0.0006) [2023-12-27 14:21:12,292][20381] Updated weights for policy 1, policy_version 1255293 (0.0009) [2023-12-27 14:21:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 643072000. Throughput: 0: 10233.0, 1: 10180.9. Samples: 643066452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:12,363][19170] Avg episode reward: [(0, '713.090'), (1, '36.270')] [2023-12-27 14:21:12,365][20381] Updated weights for policy 1, policy_version 1255303 (0.0010) [2023-12-27 14:21:12,423][20381] Updated weights for policy 1, policy_version 1255313 (0.0010) [2023-12-27 14:21:12,880][20347] Updated weights for policy 0, policy_version 1256394 (0.0008) [2023-12-27 14:21:12,932][20347] Updated weights for policy 0, policy_version 1256404 (0.0007) [2023-12-27 14:21:12,983][20347] Updated weights for policy 0, policy_version 1256414 (0.0010) [2023-12-27 14:21:13,118][20381] Updated weights for policy 1, policy_version 1255323 (0.0010) [2023-12-27 14:21:13,166][20381] Updated weights for policy 1, policy_version 1255333 (0.0005) [2023-12-27 14:21:13,232][20381] Updated weights for policy 1, policy_version 1255343 (0.0005) [2023-12-27 14:21:13,646][20347] Updated weights for policy 0, policy_version 1256424 (0.0008) [2023-12-27 14:21:13,697][20347] Updated weights for policy 0, policy_version 1256434 (0.0007) [2023-12-27 14:21:13,755][20347] Updated weights for policy 0, policy_version 1256444 (0.0008) [2023-12-27 14:21:13,812][20381] Updated weights for policy 1, policy_version 1255353 (0.0005) [2023-12-27 14:21:13,872][20381] Updated weights for policy 1, policy_version 1255363 (0.0008) [2023-12-27 14:21:13,919][20381] Updated weights for policy 1, policy_version 1255373 (0.0006) [2023-12-27 14:21:13,966][20381] Updated weights for policy 1, policy_version 1255383 (0.0005) [2023-12-27 14:21:14,447][20347] Updated weights for policy 0, policy_version 1256454 (0.0008) [2023-12-27 14:21:14,501][20347] Updated weights for policy 0, policy_version 1256464 (0.0008) [2023-12-27 14:21:14,560][20347] Updated weights for policy 0, policy_version 1256474 (0.0007) [2023-12-27 14:21:14,580][20381] Updated weights for policy 1, policy_version 1255393 (0.0007) [2023-12-27 14:21:14,630][20381] Updated weights for policy 1, policy_version 1255403 (0.0007) [2023-12-27 14:21:14,687][20381] Updated weights for policy 1, policy_version 1255413 (0.0007) [2023-12-27 14:21:15,255][20347] Updated weights for policy 0, policy_version 1256484 (0.0005) [2023-12-27 14:21:15,318][20347] Updated weights for policy 0, policy_version 1256494 (0.0005) [2023-12-27 14:21:15,376][20347] Updated weights for policy 0, policy_version 1256504 (0.0007) [2023-12-27 14:21:15,382][20381] Updated weights for policy 1, policy_version 1255423 (0.0006) [2023-12-27 14:21:15,438][20381] Updated weights for policy 1, policy_version 1255433 (0.0007) [2023-12-27 14:21:15,490][20381] Updated weights for policy 1, policy_version 1255443 (0.0007) [2023-12-27 14:21:16,028][20347] Updated weights for policy 0, policy_version 1256514 (0.0006) [2023-12-27 14:21:16,079][20347] Updated weights for policy 0, policy_version 1256524 (0.0007) [2023-12-27 14:21:16,127][20347] Updated weights for policy 0, policy_version 1256534 (0.0007) [2023-12-27 14:21:16,144][20381] Updated weights for policy 1, policy_version 1255453 (0.0007) [2023-12-27 14:21:16,179][20347] Updated weights for policy 0, policy_version 1256544 (0.0006) [2023-12-27 14:21:16,200][20381] Updated weights for policy 1, policy_version 1255463 (0.0011) [2023-12-27 14:21:16,243][20381] Updated weights for policy 1, policy_version 1255473 (0.0010) [2023-12-27 14:21:16,875][20347] Updated weights for policy 0, policy_version 1256554 (0.0007) [2023-12-27 14:21:16,933][20381] Updated weights for policy 1, policy_version 1255483 (0.0010) [2023-12-27 14:21:16,936][20347] Updated weights for policy 0, policy_version 1256564 (0.0007) [2023-12-27 14:21:16,990][20381] Updated weights for policy 1, policy_version 1255493 (0.0008) [2023-12-27 14:21:16,999][20347] Updated weights for policy 0, policy_version 1256574 (0.0009) [2023-12-27 14:21:17,050][20381] Updated weights for policy 1, policy_version 1255503 (0.0008) [2023-12-27 14:21:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 643178496. Throughput: 0: 10128.6, 1: 10208.2. Samples: 643187200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:17,363][19170] Avg episode reward: [(0, '759.300'), (1, '198.190')] [2023-12-27 14:21:17,694][20347] Updated weights for policy 0, policy_version 1256584 (0.0008) [2023-12-27 14:21:17,756][20347] Updated weights for policy 0, policy_version 1256594 (0.0008) [2023-12-27 14:21:17,805][20347] Updated weights for policy 0, policy_version 1256604 (0.0008) [2023-12-27 14:21:17,806][20381] Updated weights for policy 1, policy_version 1255513 (0.0008) [2023-12-27 14:21:17,865][20381] Updated weights for policy 1, policy_version 1255523 (0.0006) [2023-12-27 14:21:17,914][20381] Updated weights for policy 1, policy_version 1255533 (0.0008) [2023-12-27 14:21:17,976][20381] Updated weights for policy 1, policy_version 1255543 (0.0009) [2023-12-27 14:21:18,504][20347] Updated weights for policy 0, policy_version 1256614 (0.0007) [2023-12-27 14:21:18,557][20347] Updated weights for policy 0, policy_version 1256624 (0.0007) [2023-12-27 14:21:18,610][20347] Updated weights for policy 0, policy_version 1256634 (0.0006) [2023-12-27 14:21:18,631][20381] Updated weights for policy 1, policy_version 1255553 (0.0010) [2023-12-27 14:21:18,683][20381] Updated weights for policy 1, policy_version 1255563 (0.0010) [2023-12-27 14:21:18,736][20381] Updated weights for policy 1, policy_version 1255573 (0.0011) [2023-12-27 14:21:19,319][20347] Updated weights for policy 0, policy_version 1256644 (0.0006) [2023-12-27 14:21:19,383][20347] Updated weights for policy 0, policy_version 1256654 (0.0008) [2023-12-27 14:21:19,442][20347] Updated weights for policy 0, policy_version 1256664 (0.0006) [2023-12-27 14:21:19,472][20381] Updated weights for policy 1, policy_version 1255583 (0.0008) [2023-12-27 14:21:19,540][20381] Updated weights for policy 1, policy_version 1255593 (0.0007) [2023-12-27 14:21:19,610][20381] Updated weights for policy 1, policy_version 1255603 (0.0006) [2023-12-27 14:21:20,181][20347] Updated weights for policy 0, policy_version 1256674 (0.0007) [2023-12-27 14:21:20,234][20347] Updated weights for policy 0, policy_version 1256684 (0.0009) [2023-12-27 14:21:20,293][20347] Updated weights for policy 0, policy_version 1256694 (0.0009) [2023-12-27 14:21:20,342][20381] Updated weights for policy 1, policy_version 1255613 (0.0007) [2023-12-27 14:21:20,347][20347] Updated weights for policy 0, policy_version 1256704 (0.0009) [2023-12-27 14:21:20,404][20381] Updated weights for policy 1, policy_version 1255623 (0.0007) [2023-12-27 14:21:20,456][20381] Updated weights for policy 1, policy_version 1255633 (0.0010) [2023-12-27 14:21:21,098][20347] Updated weights for policy 0, policy_version 1256714 (0.0009) [2023-12-27 14:21:21,159][20347] Updated weights for policy 0, policy_version 1256724 (0.0010) [2023-12-27 14:21:21,219][20347] Updated weights for policy 0, policy_version 1256734 (0.0007) [2023-12-27 14:21:21,274][20381] Updated weights for policy 1, policy_version 1255643 (0.0010) [2023-12-27 14:21:21,337][20381] Updated weights for policy 1, policy_version 1255653 (0.0009) [2023-12-27 14:21:21,400][20381] Updated weights for policy 1, policy_version 1255663 (0.0012) [2023-12-27 14:21:21,918][20347] Updated weights for policy 0, policy_version 1256744 (0.0007) [2023-12-27 14:21:21,980][20347] Updated weights for policy 0, policy_version 1256754 (0.0007) [2023-12-27 14:21:22,040][20347] Updated weights for policy 0, policy_version 1256764 (0.0008) [2023-12-27 14:21:22,128][20381] Updated weights for policy 1, policy_version 1255673 (0.0009) [2023-12-27 14:21:22,178][20381] Updated weights for policy 1, policy_version 1255683 (0.0007) [2023-12-27 14:21:22,229][20381] Updated weights for policy 1, policy_version 1255693 (0.0008) [2023-12-27 14:21:22,287][20381] Updated weights for policy 1, policy_version 1255703 (0.0007) [2023-12-27 14:21:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 643276800. Throughput: 0: 10091.7, 1: 10181.6. Samples: 643246104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:22,363][19170] Avg episode reward: [(0, '760.330'), (1, '231.600')] [2023-12-27 14:21:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001255704_321486848.pth... [2023-12-27 14:21:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001256768_321789952.pth... [2023-12-27 14:21:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001254520_321183744.pth [2023-12-27 14:21:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001255584_321486848.pth [2023-12-27 14:21:22,712][20347] Updated weights for policy 0, policy_version 1256774 (0.0008) [2023-12-27 14:21:22,777][20347] Updated weights for policy 0, policy_version 1256784 (0.0008) [2023-12-27 14:21:22,837][20347] Updated weights for policy 0, policy_version 1256794 (0.0006) [2023-12-27 14:21:23,021][20381] Updated weights for policy 1, policy_version 1255713 (0.0007) [2023-12-27 14:21:23,080][20381] Updated weights for policy 1, policy_version 1255723 (0.0007) [2023-12-27 14:21:23,135][20381] Updated weights for policy 1, policy_version 1255733 (0.0008) [2023-12-27 14:21:23,528][20347] Updated weights for policy 0, policy_version 1256804 (0.0006) [2023-12-27 14:21:23,587][20347] Updated weights for policy 0, policy_version 1256814 (0.0005) [2023-12-27 14:21:23,651][20347] Updated weights for policy 0, policy_version 1256826 (0.0010) [2023-12-27 14:21:23,678][20381] Updated weights for policy 1, policy_version 1255743 (0.0006) [2023-12-27 14:21:23,727][20381] Updated weights for policy 1, policy_version 1255753 (0.0007) [2023-12-27 14:21:23,777][20381] Updated weights for policy 1, policy_version 1255763 (0.0006) [2023-12-27 14:21:24,342][20347] Updated weights for policy 0, policy_version 1256836 (0.0008) [2023-12-27 14:21:24,398][20347] Updated weights for policy 0, policy_version 1256846 (0.0007) [2023-12-27 14:21:24,423][20381] Updated weights for policy 1, policy_version 1255773 (0.0006) [2023-12-27 14:21:24,447][20347] Updated weights for policy 0, policy_version 1256856 (0.0010) [2023-12-27 14:21:24,481][20381] Updated weights for policy 1, policy_version 1255783 (0.0006) [2023-12-27 14:21:24,538][20381] Updated weights for policy 1, policy_version 1255793 (0.0008) [2023-12-27 14:21:25,043][20347] Updated weights for policy 0, policy_version 1256866 (0.0010) [2023-12-27 14:21:25,095][20347] Updated weights for policy 0, policy_version 1256876 (0.0010) [2023-12-27 14:21:25,151][20347] Updated weights for policy 0, policy_version 1256886 (0.0010) [2023-12-27 14:21:25,177][20381] Updated weights for policy 1, policy_version 1255803 (0.0007) [2023-12-27 14:21:25,203][20347] Updated weights for policy 0, policy_version 1256896 (0.0010) [2023-12-27 14:21:25,227][20381] Updated weights for policy 1, policy_version 1255813 (0.0008) [2023-12-27 14:21:25,287][20381] Updated weights for policy 1, policy_version 1255823 (0.0007) [2023-12-27 14:21:25,867][20347] Updated weights for policy 0, policy_version 1256906 (0.0010) [2023-12-27 14:21:25,919][20381] Updated weights for policy 1, policy_version 1255833 (0.0008) [2023-12-27 14:21:25,932][20347] Updated weights for policy 0, policy_version 1256916 (0.0010) [2023-12-27 14:21:25,971][20381] Updated weights for policy 1, policy_version 1255843 (0.0005) [2023-12-27 14:21:25,986][20347] Updated weights for policy 0, policy_version 1256926 (0.0010) [2023-12-27 14:21:26,019][20381] Updated weights for policy 1, policy_version 1255853 (0.0005) [2023-12-27 14:21:26,066][20381] Updated weights for policy 1, policy_version 1255863 (0.0005) [2023-12-27 14:21:26,660][20347] Updated weights for policy 0, policy_version 1256936 (0.0010) [2023-12-27 14:21:26,669][20381] Updated weights for policy 1, policy_version 1255873 (0.0010) [2023-12-27 14:21:26,714][20381] Updated weights for policy 1, policy_version 1255883 (0.0010) [2023-12-27 14:21:26,715][20347] Updated weights for policy 0, policy_version 1256946 (0.0010) [2023-12-27 14:21:26,758][20381] Updated weights for policy 1, policy_version 1255893 (0.0010) [2023-12-27 14:21:26,768][20347] Updated weights for policy 0, policy_version 1256956 (0.0010) [2023-12-27 14:21:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 643375104. Throughput: 0: 10101.7, 1: 10247.4. Samples: 643368664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:27,363][19170] Avg episode reward: [(0, '712.480'), (1, '238.910')] [2023-12-27 14:21:27,385][20381] Updated weights for policy 1, policy_version 1255903 (0.0007) [2023-12-27 14:21:27,433][20381] Updated weights for policy 1, policy_version 1255913 (0.0005) [2023-12-27 14:21:27,444][20347] Updated weights for policy 0, policy_version 1256966 (0.0010) [2023-12-27 14:21:27,488][20381] Updated weights for policy 1, policy_version 1255923 (0.0005) [2023-12-27 14:21:27,500][20347] Updated weights for policy 0, policy_version 1256976 (0.0010) [2023-12-27 14:21:27,553][20347] Updated weights for policy 0, policy_version 1256986 (0.0010) [2023-12-27 14:21:28,106][20381] Updated weights for policy 1, policy_version 1255933 (0.0009) [2023-12-27 14:21:28,154][20381] Updated weights for policy 1, policy_version 1255943 (0.0010) [2023-12-27 14:21:28,201][20381] Updated weights for policy 1, policy_version 1255953 (0.0010) [2023-12-27 14:21:28,235][20347] Updated weights for policy 0, policy_version 1256996 (0.0009) [2023-12-27 14:21:28,282][20347] Updated weights for policy 0, policy_version 1257006 (0.0005) [2023-12-27 14:21:28,332][20347] Updated weights for policy 0, policy_version 1257016 (0.0006) [2023-12-27 14:21:28,895][20381] Updated weights for policy 1, policy_version 1255963 (0.0009) [2023-12-27 14:21:28,951][20381] Updated weights for policy 1, policy_version 1255973 (0.0005) [2023-12-27 14:21:29,001][20381] Updated weights for policy 1, policy_version 1255983 (0.0005) [2023-12-27 14:21:29,020][20347] Updated weights for policy 0, policy_version 1257026 (0.0008) [2023-12-27 14:21:29,072][20347] Updated weights for policy 0, policy_version 1257036 (0.0010) [2023-12-27 14:21:29,131][20347] Updated weights for policy 0, policy_version 1257046 (0.0010) [2023-12-27 14:21:29,191][20347] Updated weights for policy 0, policy_version 1257056 (0.0010) [2023-12-27 14:21:29,628][20381] Updated weights for policy 1, policy_version 1255993 (0.0006) [2023-12-27 14:21:29,693][20381] Updated weights for policy 1, policy_version 1256003 (0.0008) [2023-12-27 14:21:29,753][20381] Updated weights for policy 1, policy_version 1256013 (0.0010) [2023-12-27 14:21:29,808][20381] Updated weights for policy 1, policy_version 1256023 (0.0009) [2023-12-27 14:21:29,856][20347] Updated weights for policy 0, policy_version 1257066 (0.0007) [2023-12-27 14:21:29,914][20347] Updated weights for policy 0, policy_version 1257076 (0.0007) [2023-12-27 14:21:29,976][20347] Updated weights for policy 0, policy_version 1257086 (0.0006) [2023-12-27 14:21:30,511][20381] Updated weights for policy 1, policy_version 1256033 (0.0010) [2023-12-27 14:21:30,558][20381] Updated weights for policy 1, policy_version 1256043 (0.0010) [2023-12-27 14:21:30,589][20347] Updated weights for policy 0, policy_version 1257096 (0.0006) [2023-12-27 14:21:30,606][20381] Updated weights for policy 1, policy_version 1256053 (0.0010) [2023-12-27 14:21:30,648][20347] Updated weights for policy 0, policy_version 1257106 (0.0006) [2023-12-27 14:21:30,708][20347] Updated weights for policy 0, policy_version 1257116 (0.0007) [2023-12-27 14:21:31,298][20381] Updated weights for policy 1, policy_version 1256063 (0.0010) [2023-12-27 14:21:31,359][20381] Updated weights for policy 1, policy_version 1256073 (0.0010) [2023-12-27 14:21:31,381][20347] Updated weights for policy 0, policy_version 1257126 (0.0007) [2023-12-27 14:21:31,424][20381] Updated weights for policy 1, policy_version 1256083 (0.0007) [2023-12-27 14:21:31,441][20347] Updated weights for policy 0, policy_version 1257136 (0.0008) [2023-12-27 14:21:31,501][20347] Updated weights for policy 0, policy_version 1257146 (0.0007) [2023-12-27 14:21:32,147][20381] Updated weights for policy 1, policy_version 1256093 (0.0007) [2023-12-27 14:21:32,208][20381] Updated weights for policy 1, policy_version 1256103 (0.0008) [2023-12-27 14:21:32,222][20347] Updated weights for policy 0, policy_version 1257156 (0.0007) [2023-12-27 14:21:32,266][20381] Updated weights for policy 1, policy_version 1256113 (0.0006) [2023-12-27 14:21:32,283][20347] Updated weights for policy 0, policy_version 1257166 (0.0007) [2023-12-27 14:21:32,338][20347] Updated weights for policy 0, policy_version 1257176 (0.0006) [2023-12-27 14:21:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 643481600. Throughput: 0: 10131.4, 1: 10225.1. Samples: 643492772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:32,363][19170] Avg episode reward: [(0, '738.600'), (1, '453.680')] [2023-12-27 14:21:32,931][20381] Updated weights for policy 1, policy_version 1256123 (0.0007) [2023-12-27 14:21:32,989][20381] Updated weights for policy 1, policy_version 1256133 (0.0008) [2023-12-27 14:21:33,042][20381] Updated weights for policy 1, policy_version 1256143 (0.0007) [2023-12-27 14:21:33,044][20347] Updated weights for policy 0, policy_version 1257186 (0.0007) [2023-12-27 14:21:33,098][20347] Updated weights for policy 0, policy_version 1257196 (0.0007) [2023-12-27 14:21:33,149][20347] Updated weights for policy 0, policy_version 1257206 (0.0007) [2023-12-27 14:21:33,200][20347] Updated weights for policy 0, policy_version 1257216 (0.0008) [2023-12-27 14:21:33,650][20381] Updated weights for policy 1, policy_version 1256153 (0.0005) [2023-12-27 14:21:33,697][20381] Updated weights for policy 1, policy_version 1256163 (0.0005) [2023-12-27 14:21:33,742][20381] Updated weights for policy 1, policy_version 1256173 (0.0005) [2023-12-27 14:21:33,793][20381] Updated weights for policy 1, policy_version 1256183 (0.0005) [2023-12-27 14:21:33,803][20347] Updated weights for policy 0, policy_version 1257226 (0.0005) [2023-12-27 14:21:33,848][20347] Updated weights for policy 0, policy_version 1257236 (0.0005) [2023-12-27 14:21:33,891][20347] Updated weights for policy 0, policy_version 1257246 (0.0005) [2023-12-27 14:21:34,453][20381] Updated weights for policy 1, policy_version 1256193 (0.0008) [2023-12-27 14:21:34,472][20347] Updated weights for policy 0, policy_version 1257256 (0.0007) [2023-12-27 14:21:34,513][20381] Updated weights for policy 1, policy_version 1256203 (0.0007) [2023-12-27 14:21:34,528][20347] Updated weights for policy 0, policy_version 1257266 (0.0009) [2023-12-27 14:21:34,572][20381] Updated weights for policy 1, policy_version 1256213 (0.0006) [2023-12-27 14:21:34,589][20347] Updated weights for policy 0, policy_version 1257276 (0.0006) [2023-12-27 14:21:35,183][20381] Updated weights for policy 1, policy_version 1256223 (0.0007) [2023-12-27 14:21:35,231][20381] Updated weights for policy 1, policy_version 1256233 (0.0006) [2023-12-27 14:21:35,280][20381] Updated weights for policy 1, policy_version 1256243 (0.0006) [2023-12-27 14:21:35,304][20347] Updated weights for policy 0, policy_version 1257286 (0.0007) [2023-12-27 14:21:35,333][20291] KL-divergence is very high: 177.4073 [2023-12-27 14:21:35,360][20347] Updated weights for policy 0, policy_version 1257296 (0.0008) [2023-12-27 14:21:35,376][20291] KL-divergence is very high: 220.6356 [2023-12-27 14:21:35,411][20347] Updated weights for policy 0, policy_version 1257306 (0.0007) [2023-12-27 14:21:35,418][20291] KL-divergence is very high: 127.2642 [2023-12-27 14:21:35,869][20381] Updated weights for policy 1, policy_version 1256253 (0.0006) [2023-12-27 14:21:35,913][20381] Updated weights for policy 1, policy_version 1256263 (0.0007) [2023-12-27 14:21:35,961][20381] Updated weights for policy 1, policy_version 1256273 (0.0008) [2023-12-27 14:21:36,126][20347] Updated weights for policy 0, policy_version 1257316 (0.0007) [2023-12-27 14:21:36,187][20347] Updated weights for policy 0, policy_version 1257326 (0.0008) [2023-12-27 14:21:36,250][20347] Updated weights for policy 0, policy_version 1257336 (0.0008) [2023-12-27 14:21:36,618][20381] Updated weights for policy 1, policy_version 1256283 (0.0008) [2023-12-27 14:21:36,667][20381] Updated weights for policy 1, policy_version 1256293 (0.0008) [2023-12-27 14:21:36,730][20381] Updated weights for policy 1, policy_version 1256303 (0.0008) [2023-12-27 14:21:36,999][20347] Updated weights for policy 0, policy_version 1257346 (0.0008) [2023-12-27 14:21:37,053][20347] Updated weights for policy 0, policy_version 1257356 (0.0007) [2023-12-27 14:21:37,105][20347] Updated weights for policy 0, policy_version 1257366 (0.0005) [2023-12-27 14:21:37,151][20347] Updated weights for policy 0, policy_version 1257376 (0.0005) [2023-12-27 14:21:37,340][20381] Updated weights for policy 1, policy_version 1256313 (0.0006) [2023-12-27 14:21:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 643588096. Throughput: 0: 10178.0, 1: 10293.4. Samples: 643557196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:37,363][19170] Avg episode reward: [(0, '824.420'), (1, '370.070')] [2023-12-27 14:21:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001257376_321945600.pth... [2023-12-27 14:21:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001256192_321642496.pth [2023-12-27 14:21:37,392][20381] Updated weights for policy 1, policy_version 1256323 (0.0007) [2023-12-27 14:21:37,447][20381] Updated weights for policy 1, policy_version 1256333 (0.0007) [2023-12-27 14:21:37,512][20381] Updated weights for policy 1, policy_version 1256343 (0.0007) [2023-12-27 14:21:37,516][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001256344_321650688.pth... [2023-12-27 14:21:37,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001255128_321339392.pth [2023-12-27 14:21:37,833][20347] Updated weights for policy 0, policy_version 1257386 (0.0005) [2023-12-27 14:21:37,855][20291] KL-divergence is very high: 330.5535 [2023-12-27 14:21:37,890][20347] Updated weights for policy 0, policy_version 1257396 (0.0006) [2023-12-27 14:21:37,947][20347] Updated weights for policy 0, policy_version 1257406 (0.0008) [2023-12-27 14:21:38,210][20381] Updated weights for policy 1, policy_version 1256353 (0.0010) [2023-12-27 14:21:38,272][20381] Updated weights for policy 1, policy_version 1256363 (0.0010) [2023-12-27 14:21:38,328][20381] Updated weights for policy 1, policy_version 1256373 (0.0010) [2023-12-27 14:21:38,630][20347] Updated weights for policy 0, policy_version 1257416 (0.0006) [2023-12-27 14:21:38,688][20347] Updated weights for policy 0, policy_version 1257426 (0.0009) [2023-12-27 14:21:38,743][20347] Updated weights for policy 0, policy_version 1257436 (0.0010) [2023-12-27 14:21:39,028][20381] Updated weights for policy 1, policy_version 1256383 (0.0006) [2023-12-27 14:21:39,086][20381] Updated weights for policy 1, policy_version 1256393 (0.0005) [2023-12-27 14:21:39,143][20381] Updated weights for policy 1, policy_version 1256403 (0.0005) [2023-12-27 14:21:39,439][20347] Updated weights for policy 0, policy_version 1257446 (0.0008) [2023-12-27 14:21:39,502][20347] Updated weights for policy 0, policy_version 1257456 (0.0006) [2023-12-27 14:21:39,570][20347] Updated weights for policy 0, policy_version 1257466 (0.0006) [2023-12-27 14:21:39,794][20381] Updated weights for policy 1, policy_version 1256413 (0.0006) [2023-12-27 14:21:39,860][20381] Updated weights for policy 1, policy_version 1256423 (0.0008) [2023-12-27 14:21:39,918][20381] Updated weights for policy 1, policy_version 1256433 (0.0007) [2023-12-27 14:21:40,278][20347] Updated weights for policy 0, policy_version 1257476 (0.0007) [2023-12-27 14:21:40,349][20347] Updated weights for policy 0, policy_version 1257486 (0.0006) [2023-12-27 14:21:40,409][20347] Updated weights for policy 0, policy_version 1257496 (0.0006) [2023-12-27 14:21:40,628][20381] Updated weights for policy 1, policy_version 1256443 (0.0008) [2023-12-27 14:21:40,686][20381] Updated weights for policy 1, policy_version 1256453 (0.0008) [2023-12-27 14:21:40,740][20381] Updated weights for policy 1, policy_version 1256463 (0.0008) [2023-12-27 14:21:41,019][20347] Updated weights for policy 0, policy_version 1257506 (0.0008) [2023-12-27 14:21:41,084][20347] Updated weights for policy 0, policy_version 1257516 (0.0008) [2023-12-27 14:21:41,145][20347] Updated weights for policy 0, policy_version 1257526 (0.0008) [2023-12-27 14:21:41,205][20347] Updated weights for policy 0, policy_version 1257536 (0.0006) [2023-12-27 14:21:41,514][20381] Updated weights for policy 1, policy_version 1256473 (0.0008) [2023-12-27 14:21:41,574][20381] Updated weights for policy 1, policy_version 1256483 (0.0007) [2023-12-27 14:21:41,640][20381] Updated weights for policy 1, policy_version 1256493 (0.0008) [2023-12-27 14:21:41,707][20381] Updated weights for policy 1, policy_version 1256503 (0.0007) [2023-12-27 14:21:41,977][20347] Updated weights for policy 0, policy_version 1257546 (0.0007) [2023-12-27 14:21:42,039][20347] Updated weights for policy 0, policy_version 1257556 (0.0007) [2023-12-27 14:21:42,103][20347] Updated weights for policy 0, policy_version 1257566 (0.0008) [2023-12-27 14:21:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 643686400. Throughput: 0: 10179.0, 1: 10283.2. Samples: 643679712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:42,363][19170] Avg episode reward: [(0, '863.740'), (1, '378.270')] [2023-12-27 14:21:42,390][20381] Updated weights for policy 1, policy_version 1256513 (0.0008) [2023-12-27 14:21:42,455][20381] Updated weights for policy 1, policy_version 1256523 (0.0008) [2023-12-27 14:21:42,518][20381] Updated weights for policy 1, policy_version 1256533 (0.0008) [2023-12-27 14:21:42,764][20347] Updated weights for policy 0, policy_version 1257576 (0.0007) [2023-12-27 14:21:42,796][20291] KL-divergence is very high: 176.4586 [2023-12-27 14:21:42,803][20291] KL-divergence is very high: 169.2366 [2023-12-27 14:21:42,826][20347] Updated weights for policy 0, policy_version 1257586 (0.0008) [2023-12-27 14:21:42,846][20291] KL-divergence is very high: 110.8966 [2023-12-27 14:21:42,885][20347] Updated weights for policy 0, policy_version 1257596 (0.0007) [2023-12-27 14:21:43,192][20381] Updated weights for policy 1, policy_version 1256543 (0.0008) [2023-12-27 14:21:43,247][20381] Updated weights for policy 1, policy_version 1256553 (0.0008) [2023-12-27 14:21:43,306][20381] Updated weights for policy 1, policy_version 1256563 (0.0008) [2023-12-27 14:21:43,535][20347] Updated weights for policy 0, policy_version 1257606 (0.0007) [2023-12-27 14:21:43,557][20291] KL-divergence is very high: 281.1956 [2023-12-27 14:21:43,564][20291] KL-divergence is very high: 282.1726 [2023-12-27 14:21:43,572][20291] KL-divergence is very high: 273.7384 [2023-12-27 14:21:43,607][20347] Updated weights for policy 0, policy_version 1257616 (0.0008) [2023-12-27 14:21:43,614][20291] KL-divergence is very high: 132.8016 [2023-12-27 14:21:43,670][20347] Updated weights for policy 0, policy_version 1257626 (0.0006) [2023-12-27 14:21:43,935][20381] Updated weights for policy 1, policy_version 1256573 (0.0007) [2023-12-27 14:21:43,990][20381] Updated weights for policy 1, policy_version 1256583 (0.0008) [2023-12-27 14:21:44,045][20381] Updated weights for policy 1, policy_version 1256593 (0.0007) [2023-12-27 14:21:44,248][20347] Updated weights for policy 0, policy_version 1257636 (0.0006) [2023-12-27 14:21:44,299][20347] Updated weights for policy 0, policy_version 1257646 (0.0007) [2023-12-27 14:21:44,355][20347] Updated weights for policy 0, policy_version 1257656 (0.0007) [2023-12-27 14:21:44,664][20381] Updated weights for policy 1, policy_version 1256603 (0.0008) [2023-12-27 14:21:44,720][20381] Updated weights for policy 1, policy_version 1256613 (0.0005) [2023-12-27 14:21:44,776][20381] Updated weights for policy 1, policy_version 1256623 (0.0006) [2023-12-27 14:21:45,073][20347] Updated weights for policy 0, policy_version 1257666 (0.0005) [2023-12-27 14:21:45,136][20347] Updated weights for policy 0, policy_version 1257676 (0.0007) [2023-12-27 14:21:45,196][20347] Updated weights for policy 0, policy_version 1257686 (0.0008) [2023-12-27 14:21:45,257][20347] Updated weights for policy 0, policy_version 1257696 (0.0008) [2023-12-27 14:21:45,464][20381] Updated weights for policy 1, policy_version 1256633 (0.0008) [2023-12-27 14:21:45,510][20381] Updated weights for policy 1, policy_version 1256643 (0.0010) [2023-12-27 14:21:45,561][20381] Updated weights for policy 1, policy_version 1256653 (0.0006) [2023-12-27 14:21:45,606][20381] Updated weights for policy 1, policy_version 1256663 (0.0008) [2023-12-27 14:21:45,833][20347] Updated weights for policy 0, policy_version 1257706 (0.0005) [2023-12-27 14:21:45,878][20347] Updated weights for policy 0, policy_version 1257716 (0.0005) [2023-12-27 14:21:45,924][20347] Updated weights for policy 0, policy_version 1257726 (0.0005) [2023-12-27 14:21:46,308][20381] Updated weights for policy 1, policy_version 1256673 (0.0008) [2023-12-27 14:21:46,355][20381] Updated weights for policy 1, policy_version 1256683 (0.0006) [2023-12-27 14:21:46,403][20381] Updated weights for policy 1, policy_version 1256693 (0.0006) [2023-12-27 14:21:46,486][20347] Updated weights for policy 0, policy_version 1257736 (0.0006) [2023-12-27 14:21:46,548][20347] Updated weights for policy 0, policy_version 1257746 (0.0007) [2023-12-27 14:21:46,599][20347] Updated weights for policy 0, policy_version 1257756 (0.0007) [2023-12-27 14:21:47,109][20381] Updated weights for policy 1, policy_version 1256703 (0.0007) [2023-12-27 14:21:47,162][20381] Updated weights for policy 1, policy_version 1256713 (0.0008) [2023-12-27 14:21:47,221][20347] Updated weights for policy 0, policy_version 1257766 (0.0006) [2023-12-27 14:21:47,221][20381] Updated weights for policy 1, policy_version 1256723 (0.0008) [2023-12-27 14:21:47,282][20347] Updated weights for policy 0, policy_version 1257776 (0.0005) [2023-12-27 14:21:47,342][20347] Updated weights for policy 0, policy_version 1257786 (0.0005) [2023-12-27 14:21:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 643792896. Throughput: 0: 10271.6, 1: 10255.9. Samples: 643803976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:47,363][19170] Avg episode reward: [(0, '814.950'), (1, '292.070')] [2023-12-27 14:21:47,917][20381] Updated weights for policy 1, policy_version 1256733 (0.0007) [2023-12-27 14:21:47,970][20381] Updated weights for policy 1, policy_version 1256743 (0.0007) [2023-12-27 14:21:48,015][20347] Updated weights for policy 0, policy_version 1257796 (0.0007) [2023-12-27 14:21:48,032][20381] Updated weights for policy 1, policy_version 1256753 (0.0007) [2023-12-27 14:21:48,083][20347] Updated weights for policy 0, policy_version 1257806 (0.0008) [2023-12-27 14:21:48,149][20347] Updated weights for policy 0, policy_version 1257816 (0.0009) [2023-12-27 14:21:48,734][20381] Updated weights for policy 1, policy_version 1256763 (0.0007) [2023-12-27 14:21:48,797][20381] Updated weights for policy 1, policy_version 1256773 (0.0011) [2023-12-27 14:21:48,809][20347] Updated weights for policy 0, policy_version 1257826 (0.0008) [2023-12-27 14:21:48,857][20381] Updated weights for policy 1, policy_version 1256783 (0.0011) [2023-12-27 14:21:48,869][20347] Updated weights for policy 0, policy_version 1257836 (0.0006) [2023-12-27 14:21:48,915][20347] Updated weights for policy 0, policy_version 1257846 (0.0006) [2023-12-27 14:21:48,969][20347] Updated weights for policy 0, policy_version 1257856 (0.0007) [2023-12-27 14:21:49,517][20381] Updated weights for policy 1, policy_version 1256793 (0.0010) [2023-12-27 14:21:49,571][20381] Updated weights for policy 1, policy_version 1256803 (0.0008) [2023-12-27 14:21:49,646][20381] Updated weights for policy 1, policy_version 1256813 (0.0007) [2023-12-27 14:21:49,650][20347] Updated weights for policy 0, policy_version 1257866 (0.0008) [2023-12-27 14:21:49,696][20381] Updated weights for policy 1, policy_version 1256823 (0.0008) [2023-12-27 14:21:49,708][20347] Updated weights for policy 0, policy_version 1257876 (0.0007) [2023-12-27 14:21:49,763][20347] Updated weights for policy 0, policy_version 1257886 (0.0008) [2023-12-27 14:21:50,403][20381] Updated weights for policy 1, policy_version 1256833 (0.0008) [2023-12-27 14:21:50,458][20381] Updated weights for policy 1, policy_version 1256843 (0.0006) [2023-12-27 14:21:50,472][20347] Updated weights for policy 0, policy_version 1257896 (0.0007) [2023-12-27 14:21:50,518][20381] Updated weights for policy 1, policy_version 1256853 (0.0008) [2023-12-27 14:21:50,531][20347] Updated weights for policy 0, policy_version 1257906 (0.0006) [2023-12-27 14:21:50,594][20347] Updated weights for policy 0, policy_version 1257916 (0.0008) [2023-12-27 14:21:51,183][20381] Updated weights for policy 1, policy_version 1256863 (0.0007) [2023-12-27 14:21:51,252][20381] Updated weights for policy 1, policy_version 1256873 (0.0007) [2023-12-27 14:21:51,254][20347] Updated weights for policy 0, policy_version 1257926 (0.0006) [2023-12-27 14:21:51,315][20347] Updated weights for policy 0, policy_version 1257936 (0.0006) [2023-12-27 14:21:51,315][20381] Updated weights for policy 1, policy_version 1256883 (0.0007) [2023-12-27 14:21:51,384][20347] Updated weights for policy 0, policy_version 1257946 (0.0007) [2023-12-27 14:21:52,025][20381] Updated weights for policy 1, policy_version 1256893 (0.0009) [2023-12-27 14:21:52,087][20381] Updated weights for policy 1, policy_version 1256903 (0.0010) [2023-12-27 14:21:52,143][20381] Updated weights for policy 1, policy_version 1256913 (0.0010) [2023-12-27 14:21:52,148][20347] Updated weights for policy 0, policy_version 1257956 (0.0006) [2023-12-27 14:21:52,207][20347] Updated weights for policy 0, policy_version 1257966 (0.0006) [2023-12-27 14:21:52,269][20347] Updated weights for policy 0, policy_version 1257976 (0.0007) [2023-12-27 14:21:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 643899392. Throughput: 0: 10235.3, 1: 10254.3. Samples: 643864912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:52,363][19170] Avg episode reward: [(0, '814.630'), (1, '307.820')] [2023-12-27 14:21:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001256920_321798144.pth... [2023-12-27 14:21:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001257984_322101248.pth... [2023-12-27 14:21:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001255704_321486848.pth [2023-12-27 14:21:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001256768_321789952.pth [2023-12-27 14:21:52,809][20381] Updated weights for policy 1, policy_version 1256923 (0.0009) [2023-12-27 14:21:52,870][20381] Updated weights for policy 1, policy_version 1256933 (0.0008) [2023-12-27 14:21:52,932][20381] Updated weights for policy 1, policy_version 1256943 (0.0008) [2023-12-27 14:21:52,948][20347] Updated weights for policy 0, policy_version 1257986 (0.0007) [2023-12-27 14:21:53,000][20347] Updated weights for policy 0, policy_version 1257996 (0.0008) [2023-12-27 14:21:53,050][20347] Updated weights for policy 0, policy_version 1258006 (0.0008) [2023-12-27 14:21:53,104][20347] Updated weights for policy 0, policy_version 1258016 (0.0008) [2023-12-27 14:21:53,603][20381] Updated weights for policy 1, policy_version 1256953 (0.0006) [2023-12-27 14:21:53,660][20381] Updated weights for policy 1, policy_version 1256963 (0.0008) [2023-12-27 14:21:53,713][20381] Updated weights for policy 1, policy_version 1256973 (0.0005) [2023-12-27 14:21:53,762][20381] Updated weights for policy 1, policy_version 1256983 (0.0005) [2023-12-27 14:21:53,774][20347] Updated weights for policy 0, policy_version 1258026 (0.0007) [2023-12-27 14:21:53,818][20347] Updated weights for policy 0, policy_version 1258036 (0.0005) [2023-12-27 14:21:53,875][20347] Updated weights for policy 0, policy_version 1258046 (0.0005) [2023-12-27 14:21:54,359][20381] Updated weights for policy 1, policy_version 1256993 (0.0008) [2023-12-27 14:21:54,405][20381] Updated weights for policy 1, policy_version 1257003 (0.0010) [2023-12-27 14:21:54,458][20381] Updated weights for policy 1, policy_version 1257013 (0.0011) [2023-12-27 14:21:54,499][20347] Updated weights for policy 0, policy_version 1258056 (0.0009) [2023-12-27 14:21:54,557][20347] Updated weights for policy 0, policy_version 1258066 (0.0008) [2023-12-27 14:21:54,617][20347] Updated weights for policy 0, policy_version 1258076 (0.0007) [2023-12-27 14:21:55,135][20381] Updated weights for policy 1, policy_version 1257023 (0.0008) [2023-12-27 14:21:55,188][20381] Updated weights for policy 1, policy_version 1257033 (0.0007) [2023-12-27 14:21:55,220][20347] Updated weights for policy 0, policy_version 1258086 (0.0010) [2023-12-27 14:21:55,245][20381] Updated weights for policy 1, policy_version 1257043 (0.0010) [2023-12-27 14:21:55,275][20347] Updated weights for policy 0, policy_version 1258096 (0.0010) [2023-12-27 14:21:55,325][20347] Updated weights for policy 0, policy_version 1258106 (0.0006) [2023-12-27 14:21:55,917][20381] Updated weights for policy 1, policy_version 1257053 (0.0011) [2023-12-27 14:21:55,963][20347] Updated weights for policy 0, policy_version 1258116 (0.0009) [2023-12-27 14:21:55,970][20381] Updated weights for policy 1, policy_version 1257063 (0.0011) [2023-12-27 14:21:56,019][20347] Updated weights for policy 0, policy_version 1258126 (0.0007) [2023-12-27 14:21:56,023][20381] Updated weights for policy 1, policy_version 1257073 (0.0009) [2023-12-27 14:21:56,071][20347] Updated weights for policy 0, policy_version 1258136 (0.0007) [2023-12-27 14:21:56,586][20381] Updated weights for policy 1, policy_version 1257083 (0.0007) [2023-12-27 14:21:56,641][20381] Updated weights for policy 1, policy_version 1257093 (0.0005) [2023-12-27 14:21:56,693][20381] Updated weights for policy 1, policy_version 1257103 (0.0005) [2023-12-27 14:21:56,706][20347] Updated weights for policy 0, policy_version 1258146 (0.0007) [2023-12-27 14:21:56,752][20347] Updated weights for policy 0, policy_version 1258156 (0.0005) [2023-12-27 14:21:56,800][20347] Updated weights for policy 0, policy_version 1258166 (0.0005) [2023-12-27 14:21:56,847][20347] Updated weights for policy 0, policy_version 1258176 (0.0005) [2023-12-27 14:21:57,261][20381] Updated weights for policy 1, policy_version 1257113 (0.0005) [2023-12-27 14:21:57,322][20381] Updated weights for policy 1, policy_version 1257123 (0.0006) [2023-12-27 14:21:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 643997696. Throughput: 0: 10206.8, 1: 10300.8. Samples: 643989292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:21:57,363][19170] Avg episode reward: [(0, '834.400'), (1, '516.230')] [2023-12-27 14:21:57,382][20381] Updated weights for policy 1, policy_version 1257133 (0.0005) [2023-12-27 14:21:57,433][20381] Updated weights for policy 1, policy_version 1257143 (0.0005) [2023-12-27 14:21:57,496][20347] Updated weights for policy 0, policy_version 1258186 (0.0005) [2023-12-27 14:21:57,515][20291] KL-divergence is very high: 202.3242 [2023-12-27 14:21:57,552][20347] Updated weights for policy 0, policy_version 1258196 (0.0009) [2023-12-27 14:21:57,558][20291] KL-divergence is very high: 236.3190 [2023-12-27 14:21:57,604][20291] KL-divergence is very high: 115.9103 [2023-12-27 14:21:57,608][20347] Updated weights for policy 0, policy_version 1258206 (0.0008) [2023-12-27 14:21:57,955][20381] Updated weights for policy 1, policy_version 1257153 (0.0010) [2023-12-27 14:21:58,000][20381] Updated weights for policy 1, policy_version 1257163 (0.0010) [2023-12-27 14:21:58,048][20381] Updated weights for policy 1, policy_version 1257173 (0.0010) [2023-12-27 14:21:58,242][20347] Updated weights for policy 0, policy_version 1258216 (0.0008) [2023-12-27 14:21:58,297][20347] Updated weights for policy 0, policy_version 1258226 (0.0007) [2023-12-27 14:21:58,364][20347] Updated weights for policy 0, policy_version 1258236 (0.0008) [2023-12-27 14:21:58,783][20381] Updated weights for policy 1, policy_version 1257183 (0.0008) [2023-12-27 14:21:58,839][20381] Updated weights for policy 1, policy_version 1257193 (0.0008) [2023-12-27 14:21:58,894][20381] Updated weights for policy 1, policy_version 1257203 (0.0009) [2023-12-27 14:21:59,090][20347] Updated weights for policy 0, policy_version 1258246 (0.0008) [2023-12-27 14:21:59,148][20347] Updated weights for policy 0, policy_version 1258256 (0.0007) [2023-12-27 14:21:59,208][20347] Updated weights for policy 0, policy_version 1258266 (0.0009) [2023-12-27 14:21:59,548][20381] Updated weights for policy 1, policy_version 1257213 (0.0007) [2023-12-27 14:21:59,595][20381] Updated weights for policy 1, policy_version 1257223 (0.0010) [2023-12-27 14:21:59,652][20381] Updated weights for policy 1, policy_version 1257233 (0.0008) [2023-12-27 14:21:59,892][20347] Updated weights for policy 0, policy_version 1258276 (0.0007) [2023-12-27 14:21:59,958][20347] Updated weights for policy 0, policy_version 1258286 (0.0007) [2023-12-27 14:22:00,016][20347] Updated weights for policy 0, policy_version 1258296 (0.0006) [2023-12-27 14:22:00,351][20381] Updated weights for policy 1, policy_version 1257243 (0.0009) [2023-12-27 14:22:00,403][20381] Updated weights for policy 1, policy_version 1257253 (0.0010) [2023-12-27 14:22:00,457][20381] Updated weights for policy 1, policy_version 1257263 (0.0005) [2023-12-27 14:22:00,698][20347] Updated weights for policy 0, policy_version 1258306 (0.0006) [2023-12-27 14:22:00,748][20347] Updated weights for policy 0, policy_version 1258316 (0.0007) [2023-12-27 14:22:00,797][20347] Updated weights for policy 0, policy_version 1258326 (0.0009) [2023-12-27 14:22:00,844][20347] Updated weights for policy 0, policy_version 1258336 (0.0007) [2023-12-27 14:22:01,096][20381] Updated weights for policy 1, policy_version 1257273 (0.0006) [2023-12-27 14:22:01,155][20381] Updated weights for policy 1, policy_version 1257283 (0.0010) [2023-12-27 14:22:01,214][20381] Updated weights for policy 1, policy_version 1257293 (0.0010) [2023-12-27 14:22:01,278][20381] Updated weights for policy 1, policy_version 1257303 (0.0008) [2023-12-27 14:22:01,637][20347] Updated weights for policy 0, policy_version 1258346 (0.0007) [2023-12-27 14:22:01,696][20347] Updated weights for policy 0, policy_version 1258356 (0.0007) [2023-12-27 14:22:01,763][20347] Updated weights for policy 0, policy_version 1258366 (0.0008) [2023-12-27 14:22:01,918][20381] Updated weights for policy 1, policy_version 1257313 (0.0006) [2023-12-27 14:22:01,976][20381] Updated weights for policy 1, policy_version 1257323 (0.0005) [2023-12-27 14:22:02,038][20381] Updated weights for policy 1, policy_version 1257333 (0.0005) [2023-12-27 14:22:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 644104192. Throughput: 0: 10250.6, 1: 10414.6. Samples: 644117140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:02,363][19170] Avg episode reward: [(0, '834.190'), (1, '522.350')] [2023-12-27 14:22:02,371][20347] Updated weights for policy 0, policy_version 1258376 (0.0008) [2023-12-27 14:22:02,430][20347] Updated weights for policy 0, policy_version 1258386 (0.0007) [2023-12-27 14:22:02,491][20347] Updated weights for policy 0, policy_version 1258396 (0.0008) [2023-12-27 14:22:02,619][20381] Updated weights for policy 1, policy_version 1257343 (0.0007) [2023-12-27 14:22:02,676][20381] Updated weights for policy 1, policy_version 1257353 (0.0008) [2023-12-27 14:22:02,729][20381] Updated weights for policy 1, policy_version 1257363 (0.0009) [2023-12-27 14:22:03,117][20347] Updated weights for policy 0, policy_version 1258406 (0.0006) [2023-12-27 14:22:03,176][20347] Updated weights for policy 0, policy_version 1258416 (0.0007) [2023-12-27 14:22:03,231][20347] Updated weights for policy 0, policy_version 1258426 (0.0008) [2023-12-27 14:22:03,355][20381] Updated weights for policy 1, policy_version 1257373 (0.0006) [2023-12-27 14:22:03,405][20381] Updated weights for policy 1, policy_version 1257383 (0.0005) [2023-12-27 14:22:03,456][20381] Updated weights for policy 1, policy_version 1257393 (0.0005) [2023-12-27 14:22:03,880][20347] Updated weights for policy 0, policy_version 1258436 (0.0006) [2023-12-27 14:22:03,939][20347] Updated weights for policy 0, policy_version 1258446 (0.0007) [2023-12-27 14:22:03,993][20347] Updated weights for policy 0, policy_version 1258456 (0.0008) [2023-12-27 14:22:04,048][20381] Updated weights for policy 1, policy_version 1257403 (0.0006) [2023-12-27 14:22:04,106][20381] Updated weights for policy 1, policy_version 1257413 (0.0007) [2023-12-27 14:22:04,165][20381] Updated weights for policy 1, policy_version 1257423 (0.0008) [2023-12-27 14:22:04,709][20347] Updated weights for policy 0, policy_version 1258466 (0.0009) [2023-12-27 14:22:04,774][20347] Updated weights for policy 0, policy_version 1258476 (0.0008) [2023-12-27 14:22:04,827][20381] Updated weights for policy 1, policy_version 1257433 (0.0007) [2023-12-27 14:22:04,835][20347] Updated weights for policy 0, policy_version 1258486 (0.0007) [2023-12-27 14:22:04,881][20347] Updated weights for policy 0, policy_version 1258496 (0.0009) [2023-12-27 14:22:04,883][20381] Updated weights for policy 1, policy_version 1257443 (0.0005) [2023-12-27 14:22:04,935][20381] Updated weights for policy 1, policy_version 1257453 (0.0006) [2023-12-27 14:22:04,982][20381] Updated weights for policy 1, policy_version 1257463 (0.0005) [2023-12-27 14:22:05,522][20347] Updated weights for policy 0, policy_version 1258506 (0.0006) [2023-12-27 14:22:05,580][20347] Updated weights for policy 0, policy_version 1258516 (0.0005) [2023-12-27 14:22:05,634][20347] Updated weights for policy 0, policy_version 1258526 (0.0006) [2023-12-27 14:22:05,636][20381] Updated weights for policy 1, policy_version 1257473 (0.0005) [2023-12-27 14:22:05,697][20381] Updated weights for policy 1, policy_version 1257483 (0.0007) [2023-12-27 14:22:05,753][20381] Updated weights for policy 1, policy_version 1257493 (0.0005) [2023-12-27 14:22:06,241][20347] Updated weights for policy 0, policy_version 1258536 (0.0007) [2023-12-27 14:22:06,305][20347] Updated weights for policy 0, policy_version 1258546 (0.0007) [2023-12-27 14:22:06,356][20381] Updated weights for policy 1, policy_version 1257503 (0.0006) [2023-12-27 14:22:06,371][20347] Updated weights for policy 0, policy_version 1258556 (0.0008) [2023-12-27 14:22:06,401][20381] Updated weights for policy 1, policy_version 1257513 (0.0007) [2023-12-27 14:22:06,455][20381] Updated weights for policy 1, policy_version 1257523 (0.0008) [2023-12-27 14:22:07,038][20347] Updated weights for policy 0, policy_version 1258566 (0.0006) [2023-12-27 14:22:07,098][20347] Updated weights for policy 0, policy_version 1258576 (0.0005) [2023-12-27 14:22:07,131][20381] Updated weights for policy 1, policy_version 1257533 (0.0008) [2023-12-27 14:22:07,152][20347] Updated weights for policy 0, policy_version 1258586 (0.0010) [2023-12-27 14:22:07,195][20381] Updated weights for policy 1, policy_version 1257543 (0.0006) [2023-12-27 14:22:07,256][20381] Updated weights for policy 1, policy_version 1257553 (0.0006) [2023-12-27 14:22:07,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 644218880. Throughput: 0: 10297.1, 1: 10485.2. Samples: 644181308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:07,363][19170] Avg episode reward: [(0, '863.650'), (1, '660.500')] [2023-12-27 14:22:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001257560_321961984.pth... [2023-12-27 14:22:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001258592_322256896.pth... [2023-12-27 14:22:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001257376_321945600.pth [2023-12-27 14:22:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001256344_321650688.pth [2023-12-27 14:22:07,737][20347] Updated weights for policy 0, policy_version 1258596 (0.0009) [2023-12-27 14:22:07,786][20347] Updated weights for policy 0, policy_version 1258606 (0.0007) [2023-12-27 14:22:07,829][20347] Updated weights for policy 0, policy_version 1258616 (0.0008) [2023-12-27 14:22:07,860][20381] Updated weights for policy 1, policy_version 1257563 (0.0006) [2023-12-27 14:22:07,910][20381] Updated weights for policy 1, policy_version 1257573 (0.0007) [2023-12-27 14:22:07,974][20381] Updated weights for policy 1, policy_version 1257583 (0.0010) [2023-12-27 14:22:08,534][20347] Updated weights for policy 0, policy_version 1258626 (0.0007) [2023-12-27 14:22:08,593][20347] Updated weights for policy 0, policy_version 1258636 (0.0007) [2023-12-27 14:22:08,629][20381] Updated weights for policy 1, policy_version 1257593 (0.0008) [2023-12-27 14:22:08,651][20347] Updated weights for policy 0, policy_version 1258646 (0.0007) [2023-12-27 14:22:08,687][20381] Updated weights for policy 1, policy_version 1257603 (0.0007) [2023-12-27 14:22:08,706][20347] Updated weights for policy 0, policy_version 1258656 (0.0005) [2023-12-27 14:22:08,735][20381] Updated weights for policy 1, policy_version 1257613 (0.0008) [2023-12-27 14:22:08,786][20381] Updated weights for policy 1, policy_version 1257623 (0.0008) [2023-12-27 14:22:09,386][20347] Updated weights for policy 0, policy_version 1258666 (0.0009) [2023-12-27 14:22:09,451][20347] Updated weights for policy 0, policy_version 1258676 (0.0009) [2023-12-27 14:22:09,513][20347] Updated weights for policy 0, policy_version 1258686 (0.0007) [2023-12-27 14:22:09,551][20381] Updated weights for policy 1, policy_version 1257633 (0.0007) [2023-12-27 14:22:09,611][20381] Updated weights for policy 1, policy_version 1257643 (0.0008) [2023-12-27 14:22:09,662][20381] Updated weights for policy 1, policy_version 1257653 (0.0007) [2023-12-27 14:22:10,270][20347] Updated weights for policy 0, policy_version 1258696 (0.0007) [2023-12-27 14:22:10,328][20347] Updated weights for policy 0, policy_version 1258706 (0.0007) [2023-12-27 14:22:10,332][20381] Updated weights for policy 1, policy_version 1257663 (0.0006) [2023-12-27 14:22:10,390][20381] Updated weights for policy 1, policy_version 1257673 (0.0008) [2023-12-27 14:22:10,395][20347] Updated weights for policy 0, policy_version 1258716 (0.0007) [2023-12-27 14:22:10,444][20381] Updated weights for policy 1, policy_version 1257683 (0.0008) [2023-12-27 14:22:11,064][20347] Updated weights for policy 0, policy_version 1258726 (0.0008) [2023-12-27 14:22:11,101][20381] Updated weights for policy 1, policy_version 1257693 (0.0007) [2023-12-27 14:22:11,128][20347] Updated weights for policy 0, policy_version 1258736 (0.0007) [2023-12-27 14:22:11,166][20381] Updated weights for policy 1, policy_version 1257703 (0.0008) [2023-12-27 14:22:11,194][20347] Updated weights for policy 0, policy_version 1258746 (0.0008) [2023-12-27 14:22:11,233][20381] Updated weights for policy 1, policy_version 1257713 (0.0007) [2023-12-27 14:22:11,939][20381] Updated weights for policy 1, policy_version 1257723 (0.0007) [2023-12-27 14:22:11,941][20347] Updated weights for policy 0, policy_version 1258756 (0.0007) [2023-12-27 14:22:11,998][20381] Updated weights for policy 1, policy_version 1257733 (0.0010) [2023-12-27 14:22:12,004][20347] Updated weights for policy 0, policy_version 1258766 (0.0006) [2023-12-27 14:22:12,056][20381] Updated weights for policy 1, policy_version 1257743 (0.0010) [2023-12-27 14:22:12,062][20347] Updated weights for policy 0, policy_version 1258776 (0.0005) [2023-12-27 14:22:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 644317184. Throughput: 0: 10339.5, 1: 10502.0. Samples: 644306528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:12,363][19170] Avg episode reward: [(0, '785.360'), (1, '670.590')] [2023-12-27 14:22:12,756][20381] Updated weights for policy 1, policy_version 1257753 (0.0010) [2023-12-27 14:22:12,762][20347] Updated weights for policy 0, policy_version 1258786 (0.0007) [2023-12-27 14:22:12,816][20381] Updated weights for policy 1, policy_version 1257763 (0.0008) [2023-12-27 14:22:12,822][20347] Updated weights for policy 0, policy_version 1258796 (0.0011) [2023-12-27 14:22:12,872][20381] Updated weights for policy 1, policy_version 1257773 (0.0008) [2023-12-27 14:22:12,878][20347] Updated weights for policy 0, policy_version 1258806 (0.0010) [2023-12-27 14:22:12,923][20381] Updated weights for policy 1, policy_version 1257783 (0.0007) [2023-12-27 14:22:12,936][20347] Updated weights for policy 0, policy_version 1258816 (0.0010) [2023-12-27 14:22:13,538][20347] Updated weights for policy 0, policy_version 1258826 (0.0011) [2023-12-27 14:22:13,581][20381] Updated weights for policy 1, policy_version 1257793 (0.0006) [2023-12-27 14:22:13,590][20347] Updated weights for policy 0, policy_version 1258836 (0.0010) [2023-12-27 14:22:13,625][20381] Updated weights for policy 1, policy_version 1257803 (0.0006) [2023-12-27 14:22:13,643][20347] Updated weights for policy 0, policy_version 1258846 (0.0009) [2023-12-27 14:22:13,673][20381] Updated weights for policy 1, policy_version 1257813 (0.0007) [2023-12-27 14:22:14,328][20347] Updated weights for policy 0, policy_version 1258856 (0.0009) [2023-12-27 14:22:14,387][20381] Updated weights for policy 1, policy_version 1257823 (0.0007) [2023-12-27 14:22:14,390][20347] Updated weights for policy 0, policy_version 1258866 (0.0009) [2023-12-27 14:22:14,443][20347] Updated weights for policy 0, policy_version 1258876 (0.0009) [2023-12-27 14:22:14,443][20381] Updated weights for policy 1, policy_version 1257833 (0.0007) [2023-12-27 14:22:14,495][20381] Updated weights for policy 1, policy_version 1257843 (0.0006) [2023-12-27 14:22:15,133][20347] Updated weights for policy 0, policy_version 1258886 (0.0007) [2023-12-27 14:22:15,185][20347] Updated weights for policy 0, policy_version 1258896 (0.0008) [2023-12-27 14:22:15,240][20347] Updated weights for policy 0, policy_version 1258906 (0.0008) [2023-12-27 14:22:15,249][20381] Updated weights for policy 1, policy_version 1257853 (0.0007) [2023-12-27 14:22:15,307][20381] Updated weights for policy 1, policy_version 1257863 (0.0008) [2023-12-27 14:22:15,361][20381] Updated weights for policy 1, policy_version 1257873 (0.0008) [2023-12-27 14:22:15,835][20347] Updated weights for policy 0, policy_version 1258916 (0.0008) [2023-12-27 14:22:15,890][20347] Updated weights for policy 0, policy_version 1258926 (0.0008) [2023-12-27 14:22:15,941][20347] Updated weights for policy 0, policy_version 1258936 (0.0006) [2023-12-27 14:22:15,956][20381] Updated weights for policy 1, policy_version 1257883 (0.0007) [2023-12-27 14:22:16,011][20381] Updated weights for policy 1, policy_version 1257893 (0.0011) [2023-12-27 14:22:16,063][20381] Updated weights for policy 1, policy_version 1257903 (0.0007) [2023-12-27 14:22:16,597][20347] Updated weights for policy 0, policy_version 1258946 (0.0009) [2023-12-27 14:22:16,622][20381] Updated weights for policy 1, policy_version 1257913 (0.0006) [2023-12-27 14:22:16,647][20347] Updated weights for policy 0, policy_version 1258956 (0.0005) [2023-12-27 14:22:16,678][20381] Updated weights for policy 1, policy_version 1257923 (0.0005) [2023-12-27 14:22:16,702][20347] Updated weights for policy 0, policy_version 1258966 (0.0005) [2023-12-27 14:22:16,732][20381] Updated weights for policy 1, policy_version 1257933 (0.0010) [2023-12-27 14:22:16,754][20347] Updated weights for policy 0, policy_version 1258976 (0.0005) [2023-12-27 14:22:16,787][20381] Updated weights for policy 1, policy_version 1257943 (0.0010) [2023-12-27 14:22:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 644415488. Throughput: 0: 10350.1, 1: 10500.9. Samples: 644431068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:17,363][19170] Avg episode reward: [(0, '814.370'), (1, '202.280')] [2023-12-27 14:22:17,374][20347] Updated weights for policy 0, policy_version 1258986 (0.0007) [2023-12-27 14:22:17,435][20347] Updated weights for policy 0, policy_version 1258996 (0.0007) [2023-12-27 14:22:17,469][20381] Updated weights for policy 1, policy_version 1257953 (0.0011) [2023-12-27 14:22:17,499][20347] Updated weights for policy 0, policy_version 1259006 (0.0009) [2023-12-27 14:22:17,521][20381] Updated weights for policy 1, policy_version 1257963 (0.0009) [2023-12-27 14:22:17,571][20381] Updated weights for policy 1, policy_version 1257973 (0.0011) [2023-12-27 14:22:18,194][20347] Updated weights for policy 0, policy_version 1259016 (0.0007) [2023-12-27 14:22:18,250][20347] Updated weights for policy 0, policy_version 1259026 (0.0006) [2023-12-27 14:22:18,260][20381] Updated weights for policy 1, policy_version 1257983 (0.0011) [2023-12-27 14:22:18,308][20347] Updated weights for policy 0, policy_version 1259036 (0.0007) [2023-12-27 14:22:18,312][20381] Updated weights for policy 1, policy_version 1257993 (0.0010) [2023-12-27 14:22:18,377][20381] Updated weights for policy 1, policy_version 1258003 (0.0011) [2023-12-27 14:22:18,994][20347] Updated weights for policy 0, policy_version 1259046 (0.0007) [2023-12-27 14:22:19,047][20347] Updated weights for policy 0, policy_version 1259056 (0.0008) [2023-12-27 14:22:19,058][20381] Updated weights for policy 1, policy_version 1258013 (0.0011) [2023-12-27 14:22:19,102][20347] Updated weights for policy 0, policy_version 1259066 (0.0010) [2023-12-27 14:22:19,114][20381] Updated weights for policy 1, policy_version 1258023 (0.0011) [2023-12-27 14:22:19,178][20381] Updated weights for policy 1, policy_version 1258033 (0.0010) [2023-12-27 14:22:19,765][20347] Updated weights for policy 0, policy_version 1259076 (0.0007) [2023-12-27 14:22:19,829][20347] Updated weights for policy 0, policy_version 1259086 (0.0006) [2023-12-27 14:22:19,887][20347] Updated weights for policy 0, policy_version 1259096 (0.0008) [2023-12-27 14:22:19,904][20381] Updated weights for policy 1, policy_version 1258043 (0.0010) [2023-12-27 14:22:19,965][20381] Updated weights for policy 1, policy_version 1258053 (0.0008) [2023-12-27 14:22:20,032][20381] Updated weights for policy 1, policy_version 1258063 (0.0008) [2023-12-27 14:22:20,601][20381] Updated weights for policy 1, policy_version 1258073 (0.0011) [2023-12-27 14:22:20,663][20381] Updated weights for policy 1, policy_version 1258083 (0.0011) [2023-12-27 14:22:20,665][20347] Updated weights for policy 0, policy_version 1259106 (0.0007) [2023-12-27 14:22:20,723][20381] Updated weights for policy 1, policy_version 1258093 (0.0011) [2023-12-27 14:22:20,728][20347] Updated weights for policy 0, policy_version 1259116 (0.0005) [2023-12-27 14:22:20,779][20381] Updated weights for policy 1, policy_version 1258103 (0.0011) [2023-12-27 14:22:20,791][20347] Updated weights for policy 0, policy_version 1259126 (0.0006) [2023-12-27 14:22:20,857][20347] Updated weights for policy 0, policy_version 1259136 (0.0008) [2023-12-27 14:22:21,463][20381] Updated weights for policy 1, policy_version 1258113 (0.0009) [2023-12-27 14:22:21,523][20381] Updated weights for policy 1, policy_version 1258123 (0.0008) [2023-12-27 14:22:21,576][20381] Updated weights for policy 1, policy_version 1258133 (0.0008) [2023-12-27 14:22:21,583][20347] Updated weights for policy 0, policy_version 1259146 (0.0006) [2023-12-27 14:22:21,642][20347] Updated weights for policy 0, policy_version 1259156 (0.0008) [2023-12-27 14:22:21,697][20347] Updated weights for policy 0, policy_version 1259166 (0.0008) [2023-12-27 14:22:22,275][20381] Updated weights for policy 1, policy_version 1258143 (0.0008) [2023-12-27 14:22:22,336][20381] Updated weights for policy 1, policy_version 1258153 (0.0008) [2023-12-27 14:22:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 644513792. Throughput: 0: 10308.5, 1: 10429.1. Samples: 644490392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:22,363][19170] Avg episode reward: [(0, '863.300'), (1, '398.250')] [2023-12-27 14:22:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001259168_322404352.pth... [2023-12-27 14:22:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001257984_322101248.pth [2023-12-27 14:22:22,407][20381] Updated weights for policy 1, policy_version 1258163 (0.0007) [2023-12-27 14:22:22,435][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001258168_322117632.pth... [2023-12-27 14:22:22,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001256920_321798144.pth [2023-12-27 14:22:22,451][20347] Updated weights for policy 0, policy_version 1259176 (0.0008) [2023-12-27 14:22:22,509][20347] Updated weights for policy 0, policy_version 1259186 (0.0008) [2023-12-27 14:22:22,582][20347] Updated weights for policy 0, policy_version 1259196 (0.0009) [2023-12-27 14:22:23,123][20381] Updated weights for policy 1, policy_version 1258173 (0.0008) [2023-12-27 14:22:23,179][20381] Updated weights for policy 1, policy_version 1258183 (0.0010) [2023-12-27 14:22:23,244][20381] Updated weights for policy 1, policy_version 1258193 (0.0010) [2023-12-27 14:22:23,259][20347] Updated weights for policy 0, policy_version 1259206 (0.0005) [2023-12-27 14:22:23,319][20347] Updated weights for policy 0, policy_version 1259216 (0.0005) [2023-12-27 14:22:23,380][20347] Updated weights for policy 0, policy_version 1259226 (0.0005) [2023-12-27 14:22:23,922][20381] Updated weights for policy 1, policy_version 1258203 (0.0008) [2023-12-27 14:22:23,942][20347] Updated weights for policy 0, policy_version 1259236 (0.0006) [2023-12-27 14:22:23,989][20381] Updated weights for policy 1, policy_version 1258213 (0.0008) [2023-12-27 14:22:23,993][20347] Updated weights for policy 0, policy_version 1259246 (0.0009) [2023-12-27 14:22:24,046][20381] Updated weights for policy 1, policy_version 1258223 (0.0006) [2023-12-27 14:22:24,057][20347] Updated weights for policy 0, policy_version 1259256 (0.0005) [2023-12-27 14:22:24,634][20381] Updated weights for policy 1, policy_version 1258233 (0.0006) [2023-12-27 14:22:24,676][20347] Updated weights for policy 0, policy_version 1259266 (0.0005) [2023-12-27 14:22:24,692][20381] Updated weights for policy 1, policy_version 1258243 (0.0008) [2023-12-27 14:22:24,728][20347] Updated weights for policy 0, policy_version 1259276 (0.0005) [2023-12-27 14:22:24,746][20381] Updated weights for policy 1, policy_version 1258253 (0.0007) [2023-12-27 14:22:24,776][20347] Updated weights for policy 0, policy_version 1259286 (0.0005) [2023-12-27 14:22:24,803][20381] Updated weights for policy 1, policy_version 1258263 (0.0008) [2023-12-27 14:22:24,828][20347] Updated weights for policy 0, policy_version 1259296 (0.0005) [2023-12-27 14:22:25,424][20347] Updated weights for policy 0, policy_version 1259306 (0.0006) [2023-12-27 14:22:25,429][20381] Updated weights for policy 1, policy_version 1258273 (0.0009) [2023-12-27 14:22:25,475][20347] Updated weights for policy 0, policy_version 1259316 (0.0007) [2023-12-27 14:22:25,477][20381] Updated weights for policy 1, policy_version 1258283 (0.0006) [2023-12-27 14:22:25,522][20347] Updated weights for policy 0, policy_version 1259326 (0.0008) [2023-12-27 14:22:25,522][20381] Updated weights for policy 1, policy_version 1258293 (0.0008) [2023-12-27 14:22:26,195][20381] Updated weights for policy 1, policy_version 1258303 (0.0007) [2023-12-27 14:22:26,202][20347] Updated weights for policy 0, policy_version 1259336 (0.0007) [2023-12-27 14:22:26,241][20381] Updated weights for policy 1, policy_version 1258313 (0.0006) [2023-12-27 14:22:26,258][20347] Updated weights for policy 0, policy_version 1259346 (0.0007) [2023-12-27 14:22:26,291][20381] Updated weights for policy 1, policy_version 1258323 (0.0008) [2023-12-27 14:22:26,317][20347] Updated weights for policy 0, policy_version 1259356 (0.0006) [2023-12-27 14:22:26,890][20347] Updated weights for policy 0, policy_version 1259366 (0.0007) [2023-12-27 14:22:26,938][20347] Updated weights for policy 0, policy_version 1259376 (0.0007) [2023-12-27 14:22:26,996][20347] Updated weights for policy 0, policy_version 1259386 (0.0006) [2023-12-27 14:22:27,040][20381] Updated weights for policy 1, policy_version 1258333 (0.0008) [2023-12-27 14:22:27,085][20381] Updated weights for policy 1, policy_version 1258343 (0.0010) [2023-12-27 14:22:27,130][20381] Updated weights for policy 1, policy_version 1258353 (0.0010) [2023-12-27 14:22:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 644628480. Throughput: 0: 10336.4, 1: 10430.7. Samples: 644614236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:27,363][19170] Avg episode reward: [(0, '902.480'), (1, '682.400')] [2023-12-27 14:22:27,613][20347] Updated weights for policy 0, policy_version 1259396 (0.0006) [2023-12-27 14:22:27,660][20347] Updated weights for policy 0, policy_version 1259406 (0.0009) [2023-12-27 14:22:27,715][20381] Updated weights for policy 1, policy_version 1258363 (0.0009) [2023-12-27 14:22:27,721][20347] Updated weights for policy 0, policy_version 1259416 (0.0005) [2023-12-27 14:22:27,777][20381] Updated weights for policy 1, policy_version 1258373 (0.0005) [2023-12-27 14:22:27,834][20381] Updated weights for policy 1, policy_version 1258383 (0.0010) [2023-12-27 14:22:28,344][20347] Updated weights for policy 0, policy_version 1259426 (0.0007) [2023-12-27 14:22:28,402][20347] Updated weights for policy 0, policy_version 1259436 (0.0010) [2023-12-27 14:22:28,465][20347] Updated weights for policy 0, policy_version 1259446 (0.0010) [2023-12-27 14:22:28,485][20381] Updated weights for policy 1, policy_version 1258393 (0.0010) [2023-12-27 14:22:28,526][20347] Updated weights for policy 0, policy_version 1259456 (0.0009) [2023-12-27 14:22:28,542][20381] Updated weights for policy 1, policy_version 1258403 (0.0008) [2023-12-27 14:22:28,587][20381] Updated weights for policy 1, policy_version 1258413 (0.0010) [2023-12-27 14:22:28,639][20381] Updated weights for policy 1, policy_version 1258423 (0.0010) [2023-12-27 14:22:29,091][20347] Updated weights for policy 0, policy_version 1259466 (0.0005) [2023-12-27 14:22:29,147][20347] Updated weights for policy 0, policy_version 1259476 (0.0005) [2023-12-27 14:22:29,201][20347] Updated weights for policy 0, policy_version 1259486 (0.0006) [2023-12-27 14:22:29,332][20381] Updated weights for policy 1, policy_version 1258433 (0.0009) [2023-12-27 14:22:29,393][20381] Updated weights for policy 1, policy_version 1258443 (0.0009) [2023-12-27 14:22:29,446][20381] Updated weights for policy 1, policy_version 1258453 (0.0008) [2023-12-27 14:22:29,914][20347] Updated weights for policy 0, policy_version 1259496 (0.0009) [2023-12-27 14:22:29,982][20347] Updated weights for policy 0, policy_version 1259506 (0.0010) [2023-12-27 14:22:30,041][20347] Updated weights for policy 0, policy_version 1259516 (0.0010) [2023-12-27 14:22:30,143][20381] Updated weights for policy 1, policy_version 1258463 (0.0010) [2023-12-27 14:22:30,200][20381] Updated weights for policy 1, policy_version 1258473 (0.0011) [2023-12-27 14:22:30,261][20381] Updated weights for policy 1, policy_version 1258483 (0.0011) [2023-12-27 14:22:30,701][20347] Updated weights for policy 0, policy_version 1259526 (0.0010) [2023-12-27 14:22:30,749][20347] Updated weights for policy 0, policy_version 1259536 (0.0010) [2023-12-27 14:22:30,794][20347] Updated weights for policy 0, policy_version 1259546 (0.0010) [2023-12-27 14:22:30,925][20381] Updated weights for policy 1, policy_version 1258493 (0.0011) [2023-12-27 14:22:30,979][20381] Updated weights for policy 1, policy_version 1258503 (0.0010) [2023-12-27 14:22:31,039][20381] Updated weights for policy 1, policy_version 1258513 (0.0010) [2023-12-27 14:22:31,452][20347] Updated weights for policy 0, policy_version 1259556 (0.0009) [2023-12-27 14:22:31,510][20347] Updated weights for policy 0, policy_version 1259566 (0.0006) [2023-12-27 14:22:31,572][20347] Updated weights for policy 0, policy_version 1259576 (0.0006) [2023-12-27 14:22:31,769][20381] Updated weights for policy 1, policy_version 1258523 (0.0009) [2023-12-27 14:22:31,825][20381] Updated weights for policy 1, policy_version 1258533 (0.0009) [2023-12-27 14:22:31,888][20381] Updated weights for policy 1, policy_version 1258543 (0.0006) [2023-12-27 14:22:32,257][20347] Updated weights for policy 0, policy_version 1259586 (0.0008) [2023-12-27 14:22:32,304][20347] Updated weights for policy 0, policy_version 1259596 (0.0008) [2023-12-27 14:22:32,360][20347] Updated weights for policy 0, policy_version 1259606 (0.0008) [2023-12-27 14:22:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 644726784. Throughput: 0: 10358.7, 1: 10434.9. Samples: 644739688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:32,363][19170] Avg episode reward: [(0, '892.640'), (1, '660.920')] [2023-12-27 14:22:32,420][20347] Updated weights for policy 0, policy_version 1259616 (0.0006) [2023-12-27 14:22:32,576][20381] Updated weights for policy 1, policy_version 1258553 (0.0008) [2023-12-27 14:22:32,636][20381] Updated weights for policy 1, policy_version 1258563 (0.0007) [2023-12-27 14:22:32,688][20381] Updated weights for policy 1, policy_version 1258573 (0.0008) [2023-12-27 14:22:32,744][20381] Updated weights for policy 1, policy_version 1258583 (0.0007) [2023-12-27 14:22:33,158][20347] Updated weights for policy 0, policy_version 1259626 (0.0007) [2023-12-27 14:22:33,205][20347] Updated weights for policy 0, policy_version 1259636 (0.0007) [2023-12-27 14:22:33,249][20347] Updated weights for policy 0, policy_version 1259646 (0.0007) [2023-12-27 14:22:33,442][20381] Updated weights for policy 1, policy_version 1258593 (0.0010) [2023-12-27 14:22:33,499][20381] Updated weights for policy 1, policy_version 1258603 (0.0011) [2023-12-27 14:22:33,554][20381] Updated weights for policy 1, policy_version 1258613 (0.0011) [2023-12-27 14:22:33,886][20347] Updated weights for policy 0, policy_version 1259656 (0.0008) [2023-12-27 14:22:33,942][20347] Updated weights for policy 0, policy_version 1259666 (0.0007) [2023-12-27 14:22:33,995][20347] Updated weights for policy 0, policy_version 1259676 (0.0007) [2023-12-27 14:22:34,262][20381] Updated weights for policy 1, policy_version 1258623 (0.0011) [2023-12-27 14:22:34,322][20381] Updated weights for policy 1, policy_version 1258633 (0.0009) [2023-12-27 14:22:34,386][20381] Updated weights for policy 1, policy_version 1258643 (0.0005) [2023-12-27 14:22:34,658][20347] Updated weights for policy 0, policy_version 1259686 (0.0007) [2023-12-27 14:22:34,720][20347] Updated weights for policy 0, policy_version 1259696 (0.0006) [2023-12-27 14:22:34,777][20347] Updated weights for policy 0, policy_version 1259706 (0.0007) [2023-12-27 14:22:35,064][20381] Updated weights for policy 1, policy_version 1258653 (0.0011) [2023-12-27 14:22:35,116][20381] Updated weights for policy 1, policy_version 1258663 (0.0011) [2023-12-27 14:22:35,173][20381] Updated weights for policy 1, policy_version 1258674 (0.0010) [2023-12-27 14:22:35,466][20347] Updated weights for policy 0, policy_version 1259716 (0.0008) [2023-12-27 14:22:35,522][20347] Updated weights for policy 0, policy_version 1259726 (0.0007) [2023-12-27 14:22:35,577][20347] Updated weights for policy 0, policy_version 1259736 (0.0008) [2023-12-27 14:22:35,784][20381] Updated weights for policy 1, policy_version 1258684 (0.0006) [2023-12-27 14:22:35,834][20381] Updated weights for policy 1, policy_version 1258694 (0.0007) [2023-12-27 14:22:35,880][20381] Updated weights for policy 1, policy_version 1258704 (0.0008) [2023-12-27 14:22:36,280][20347] Updated weights for policy 0, policy_version 1259746 (0.0007) [2023-12-27 14:22:36,343][20347] Updated weights for policy 0, policy_version 1259756 (0.0008) [2023-12-27 14:22:36,402][20347] Updated weights for policy 0, policy_version 1259766 (0.0008) [2023-12-27 14:22:36,463][20347] Updated weights for policy 0, policy_version 1259776 (0.0008) [2023-12-27 14:22:36,619][20381] Updated weights for policy 1, policy_version 1258714 (0.0008) [2023-12-27 14:22:36,677][20381] Updated weights for policy 1, policy_version 1258724 (0.0010) [2023-12-27 14:22:36,736][20381] Updated weights for policy 1, policy_version 1258734 (0.0011) [2023-12-27 14:22:36,799][20381] Updated weights for policy 1, policy_version 1258744 (0.0011) [2023-12-27 14:22:37,170][20347] Updated weights for policy 0, policy_version 1259786 (0.0007) [2023-12-27 14:22:37,234][20347] Updated weights for policy 0, policy_version 1259796 (0.0006) [2023-12-27 14:22:37,290][20347] Updated weights for policy 0, policy_version 1259806 (0.0008) [2023-12-27 14:22:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 644833280. Throughput: 0: 10370.4, 1: 10422.2. Samples: 644800576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:37,363][19170] Avg episode reward: [(0, '843.820'), (1, '699.150')] [2023-12-27 14:22:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001258744_322265088.pth... [2023-12-27 14:22:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001259808_322568192.pth... [2023-12-27 14:22:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001257560_321961984.pth [2023-12-27 14:22:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001258592_322256896.pth [2023-12-27 14:22:37,504][20381] Updated weights for policy 1, policy_version 1258754 (0.0008) [2023-12-27 14:22:37,564][20381] Updated weights for policy 1, policy_version 1258764 (0.0008) [2023-12-27 14:22:37,624][20381] Updated weights for policy 1, policy_version 1258774 (0.0008) [2023-12-27 14:22:37,970][20347] Updated weights for policy 0, policy_version 1259816 (0.0007) [2023-12-27 14:22:38,023][20347] Updated weights for policy 0, policy_version 1259826 (0.0006) [2023-12-27 14:22:38,087][20347] Updated weights for policy 0, policy_version 1259836 (0.0008) [2023-12-27 14:22:38,337][20381] Updated weights for policy 1, policy_version 1258784 (0.0010) [2023-12-27 14:22:38,395][20381] Updated weights for policy 1, policy_version 1258794 (0.0006) [2023-12-27 14:22:38,458][20381] Updated weights for policy 1, policy_version 1258804 (0.0008) [2023-12-27 14:22:38,790][20347] Updated weights for policy 0, policy_version 1259846 (0.0008) [2023-12-27 14:22:38,844][20347] Updated weights for policy 0, policy_version 1259856 (0.0007) [2023-12-27 14:22:38,894][20347] Updated weights for policy 0, policy_version 1259866 (0.0007) [2023-12-27 14:22:39,094][20381] Updated weights for policy 1, policy_version 1258814 (0.0006) [2023-12-27 14:22:39,149][20381] Updated weights for policy 1, policy_version 1258824 (0.0010) [2023-12-27 14:22:39,199][20381] Updated weights for policy 1, policy_version 1258834 (0.0010) [2023-12-27 14:22:39,628][20347] Updated weights for policy 0, policy_version 1259876 (0.0007) [2023-12-27 14:22:39,684][20347] Updated weights for policy 0, policy_version 1259886 (0.0008) [2023-12-27 14:22:39,742][20347] Updated weights for policy 0, policy_version 1259896 (0.0007) [2023-12-27 14:22:39,918][20381] Updated weights for policy 1, policy_version 1258844 (0.0009) [2023-12-27 14:22:39,975][20381] Updated weights for policy 1, policy_version 1258854 (0.0006) [2023-12-27 14:22:40,034][20381] Updated weights for policy 1, policy_version 1258864 (0.0007) [2023-12-27 14:22:40,500][20347] Updated weights for policy 0, policy_version 1259906 (0.0007) [2023-12-27 14:22:40,553][20347] Updated weights for policy 0, policy_version 1259916 (0.0008) [2023-12-27 14:22:40,586][20291] KL-divergence is very high: 166.7981 [2023-12-27 14:22:40,601][20347] Updated weights for policy 0, policy_version 1259926 (0.0009) [2023-12-27 14:22:40,627][20291] KL-divergence is very high: 188.1128 [2023-12-27 14:22:40,657][20347] Updated weights for policy 0, policy_version 1259936 (0.0007) [2023-12-27 14:22:40,673][20381] Updated weights for policy 1, policy_version 1258874 (0.0008) [2023-12-27 14:22:40,730][20381] Updated weights for policy 1, policy_version 1258884 (0.0008) [2023-12-27 14:22:40,785][20381] Updated weights for policy 1, policy_version 1258894 (0.0008) [2023-12-27 14:22:40,843][20381] Updated weights for policy 1, policy_version 1258904 (0.0008) [2023-12-27 14:22:41,341][20347] Updated weights for policy 0, policy_version 1259946 (0.0007) [2023-12-27 14:22:41,414][20347] Updated weights for policy 0, policy_version 1259956 (0.0007) [2023-12-27 14:22:41,474][20347] Updated weights for policy 0, policy_version 1259966 (0.0007) [2023-12-27 14:22:41,686][20381] Updated weights for policy 1, policy_version 1258914 (0.0008) [2023-12-27 14:22:41,763][20381] Updated weights for policy 1, policy_version 1258924 (0.0008) [2023-12-27 14:22:41,821][20381] Updated weights for policy 1, policy_version 1258934 (0.0008) [2023-12-27 14:22:42,185][20347] Updated weights for policy 0, policy_version 1259976 (0.0008) [2023-12-27 14:22:42,244][20347] Updated weights for policy 0, policy_version 1259986 (0.0008) [2023-12-27 14:22:42,312][20347] Updated weights for policy 0, policy_version 1259996 (0.0008) [2023-12-27 14:22:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 644931584. Throughput: 0: 10297.5, 1: 10398.3. Samples: 644920604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:42,363][19170] Avg episode reward: [(0, '795.200'), (1, '549.560')] [2023-12-27 14:22:42,436][20381] Updated weights for policy 1, policy_version 1258944 (0.0008) [2023-12-27 14:22:42,496][20381] Updated weights for policy 1, policy_version 1258954 (0.0008) [2023-12-27 14:22:42,560][20381] Updated weights for policy 1, policy_version 1258964 (0.0008) [2023-12-27 14:22:43,037][20347] Updated weights for policy 0, policy_version 1260006 (0.0007) [2023-12-27 14:22:43,096][20347] Updated weights for policy 0, policy_version 1260016 (0.0008) [2023-12-27 14:22:43,147][20381] Updated weights for policy 1, policy_version 1258974 (0.0006) [2023-12-27 14:22:43,160][20347] Updated weights for policy 0, policy_version 1260026 (0.0007) [2023-12-27 14:22:43,197][20381] Updated weights for policy 1, policy_version 1258984 (0.0007) [2023-12-27 14:22:43,256][20381] Updated weights for policy 1, policy_version 1258994 (0.0006) [2023-12-27 14:22:43,872][20347] Updated weights for policy 0, policy_version 1260036 (0.0006) [2023-12-27 14:22:43,875][20381] Updated weights for policy 1, policy_version 1259004 (0.0008) [2023-12-27 14:22:43,916][20347] Updated weights for policy 0, policy_version 1260046 (0.0005) [2023-12-27 14:22:43,932][20381] Updated weights for policy 1, policy_version 1259014 (0.0010) [2023-12-27 14:22:43,990][20381] Updated weights for policy 1, policy_version 1259024 (0.0011) [2023-12-27 14:22:43,992][20347] Updated weights for policy 0, policy_version 1260056 (0.0006) [2023-12-27 14:22:44,675][20347] Updated weights for policy 0, policy_version 1260066 (0.0006) [2023-12-27 14:22:44,700][20381] Updated weights for policy 1, policy_version 1259034 (0.0010) [2023-12-27 14:22:44,728][20347] Updated weights for policy 0, policy_version 1260076 (0.0006) [2023-12-27 14:22:44,748][20381] Updated weights for policy 1, policy_version 1259044 (0.0010) [2023-12-27 14:22:44,787][20347] Updated weights for policy 0, policy_version 1260086 (0.0006) [2023-12-27 14:22:44,808][20381] Updated weights for policy 1, policy_version 1259054 (0.0011) [2023-12-27 14:22:44,846][20347] Updated weights for policy 0, policy_version 1260096 (0.0006) [2023-12-27 14:22:44,870][20381] Updated weights for policy 1, policy_version 1259064 (0.0009) [2023-12-27 14:22:45,584][20381] Updated weights for policy 1, policy_version 1259074 (0.0010) [2023-12-27 14:22:45,604][20347] Updated weights for policy 0, policy_version 1260106 (0.0005) [2023-12-27 14:22:45,641][20381] Updated weights for policy 1, policy_version 1259084 (0.0007) [2023-12-27 14:22:45,661][20347] Updated weights for policy 0, policy_version 1260116 (0.0006) [2023-12-27 14:22:45,687][20381] Updated weights for policy 1, policy_version 1259094 (0.0006) [2023-12-27 14:22:45,718][20347] Updated weights for policy 0, policy_version 1260126 (0.0008) [2023-12-27 14:22:46,309][20381] Updated weights for policy 1, policy_version 1259104 (0.0009) [2023-12-27 14:22:46,360][20381] Updated weights for policy 1, policy_version 1259114 (0.0009) [2023-12-27 14:22:46,362][20347] Updated weights for policy 0, policy_version 1260136 (0.0006) [2023-12-27 14:22:46,408][20381] Updated weights for policy 1, policy_version 1259124 (0.0010) [2023-12-27 14:22:46,413][20347] Updated weights for policy 0, policy_version 1260146 (0.0005) [2023-12-27 14:22:46,471][20347] Updated weights for policy 0, policy_version 1260156 (0.0008) [2023-12-27 14:22:46,994][20381] Updated weights for policy 1, policy_version 1259134 (0.0006) [2023-12-27 14:22:47,053][20381] Updated weights for policy 1, policy_version 1259144 (0.0006) [2023-12-27 14:22:47,103][20381] Updated weights for policy 1, policy_version 1259154 (0.0008) [2023-12-27 14:22:47,207][20347] Updated weights for policy 0, policy_version 1260166 (0.0007) [2023-12-27 14:22:47,270][20347] Updated weights for policy 0, policy_version 1260176 (0.0007) [2023-12-27 14:22:47,323][20347] Updated weights for policy 0, policy_version 1260186 (0.0008) [2023-12-27 14:22:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 645038080. Throughput: 0: 10228.5, 1: 10343.6. Samples: 645042880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:47,363][19170] Avg episode reward: [(0, '863.470'), (1, '617.570')] [2023-12-27 14:22:47,711][20381] Updated weights for policy 1, policy_version 1259164 (0.0010) [2023-12-27 14:22:47,772][20381] Updated weights for policy 1, policy_version 1259174 (0.0007) [2023-12-27 14:22:47,823][20381] Updated weights for policy 1, policy_version 1259184 (0.0006) [2023-12-27 14:22:48,021][20347] Updated weights for policy 0, policy_version 1260196 (0.0007) [2023-12-27 14:22:48,084][20347] Updated weights for policy 0, policy_version 1260206 (0.0011) [2023-12-27 14:22:48,144][20347] Updated weights for policy 0, policy_version 1260216 (0.0010) [2023-12-27 14:22:48,535][20381] Updated weights for policy 1, policy_version 1259194 (0.0007) [2023-12-27 14:22:48,585][20381] Updated weights for policy 1, policy_version 1259204 (0.0007) [2023-12-27 14:22:48,633][20381] Updated weights for policy 1, policy_version 1259214 (0.0007) [2023-12-27 14:22:48,685][20381] Updated weights for policy 1, policy_version 1259224 (0.0007) [2023-12-27 14:22:48,777][20347] Updated weights for policy 0, policy_version 1260226 (0.0009) [2023-12-27 14:22:48,836][20347] Updated weights for policy 0, policy_version 1260236 (0.0011) [2023-12-27 14:22:48,895][20347] Updated weights for policy 0, policy_version 1260246 (0.0009) [2023-12-27 14:22:48,949][20347] Updated weights for policy 0, policy_version 1260256 (0.0008) [2023-12-27 14:22:49,426][20381] Updated weights for policy 1, policy_version 1259234 (0.0006) [2023-12-27 14:22:49,491][20381] Updated weights for policy 1, policy_version 1259244 (0.0006) [2023-12-27 14:22:49,556][20381] Updated weights for policy 1, policy_version 1259254 (0.0007) [2023-12-27 14:22:49,631][20347] Updated weights for policy 0, policy_version 1260266 (0.0008) [2023-12-27 14:22:49,689][20347] Updated weights for policy 0, policy_version 1260276 (0.0007) [2023-12-27 14:22:49,749][20347] Updated weights for policy 0, policy_version 1260286 (0.0008) [2023-12-27 14:22:50,219][20381] Updated weights for policy 1, policy_version 1259264 (0.0008) [2023-12-27 14:22:50,283][20381] Updated weights for policy 1, policy_version 1259274 (0.0007) [2023-12-27 14:22:50,338][20381] Updated weights for policy 1, policy_version 1259284 (0.0007) [2023-12-27 14:22:50,459][20347] Updated weights for policy 0, policy_version 1260296 (0.0010) [2023-12-27 14:22:50,515][20347] Updated weights for policy 0, policy_version 1260306 (0.0010) [2023-12-27 14:22:50,586][20347] Updated weights for policy 0, policy_version 1260316 (0.0010) [2023-12-27 14:22:51,057][20381] Updated weights for policy 1, policy_version 1259294 (0.0008) [2023-12-27 14:22:51,114][20381] Updated weights for policy 1, policy_version 1259304 (0.0008) [2023-12-27 14:22:51,177][20381] Updated weights for policy 1, policy_version 1259314 (0.0008) [2023-12-27 14:22:51,302][20347] Updated weights for policy 0, policy_version 1260326 (0.0008) [2023-12-27 14:22:51,376][20347] Updated weights for policy 0, policy_version 1260336 (0.0007) [2023-12-27 14:22:51,431][20347] Updated weights for policy 0, policy_version 1260346 (0.0008) [2023-12-27 14:22:51,856][20381] Updated weights for policy 1, policy_version 1259324 (0.0007) [2023-12-27 14:22:51,911][20381] Updated weights for policy 1, policy_version 1259334 (0.0008) [2023-12-27 14:22:51,969][20381] Updated weights for policy 1, policy_version 1259344 (0.0007) [2023-12-27 14:22:52,146][20347] Updated weights for policy 0, policy_version 1260356 (0.0008) [2023-12-27 14:22:52,198][20347] Updated weights for policy 0, policy_version 1260366 (0.0008) [2023-12-27 14:22:52,258][20347] Updated weights for policy 0, policy_version 1260376 (0.0007) [2023-12-27 14:22:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 645136384. Throughput: 0: 10208.0, 1: 10290.4. Samples: 645103736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:52,364][19170] Avg episode reward: [(0, '892.540'), (1, '652.740')] [2023-12-27 14:22:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001259352_322420736.pth... [2023-12-27 14:22:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001260384_322715648.pth... [2023-12-27 14:22:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001259168_322404352.pth [2023-12-27 14:22:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001258168_322117632.pth [2023-12-27 14:22:52,651][20381] Updated weights for policy 1, policy_version 1259354 (0.0005) [2023-12-27 14:22:52,705][20381] Updated weights for policy 1, policy_version 1259364 (0.0006) [2023-12-27 14:22:52,758][20381] Updated weights for policy 1, policy_version 1259374 (0.0009) [2023-12-27 14:22:52,806][20381] Updated weights for policy 1, policy_version 1259384 (0.0006) [2023-12-27 14:22:52,889][20347] Updated weights for policy 0, policy_version 1260386 (0.0008) [2023-12-27 14:22:52,946][20347] Updated weights for policy 0, policy_version 1260396 (0.0005) [2023-12-27 14:22:52,995][20347] Updated weights for policy 0, policy_version 1260406 (0.0005) [2023-12-27 14:22:53,044][20347] Updated weights for policy 0, policy_version 1260416 (0.0005) [2023-12-27 14:22:53,501][20381] Updated weights for policy 1, policy_version 1259394 (0.0007) [2023-12-27 14:22:53,557][20381] Updated weights for policy 1, policy_version 1259404 (0.0008) [2023-12-27 14:22:53,558][20347] Updated weights for policy 0, policy_version 1260426 (0.0005) [2023-12-27 14:22:53,602][20347] Updated weights for policy 0, policy_version 1260436 (0.0005) [2023-12-27 14:22:53,611][20381] Updated weights for policy 1, policy_version 1259414 (0.0007) [2023-12-27 14:22:53,645][20347] Updated weights for policy 0, policy_version 1260446 (0.0005) [2023-12-27 14:22:54,218][20381] Updated weights for policy 1, policy_version 1259424 (0.0007) [2023-12-27 14:22:54,268][20381] Updated weights for policy 1, policy_version 1259434 (0.0007) [2023-12-27 14:22:54,268][20347] Updated weights for policy 0, policy_version 1260456 (0.0005) [2023-12-27 14:22:54,312][20347] Updated weights for policy 0, policy_version 1260466 (0.0005) [2023-12-27 14:22:54,319][20381] Updated weights for policy 1, policy_version 1259444 (0.0007) [2023-12-27 14:22:54,369][20347] Updated weights for policy 0, policy_version 1260476 (0.0005) [2023-12-27 14:22:54,969][20381] Updated weights for policy 1, policy_version 1259454 (0.0009) [2023-12-27 14:22:55,019][20381] Updated weights for policy 1, policy_version 1259464 (0.0009) [2023-12-27 14:22:55,020][20347] Updated weights for policy 0, policy_version 1260486 (0.0006) [2023-12-27 14:22:55,065][20347] Updated weights for policy 0, policy_version 1260496 (0.0010) [2023-12-27 14:22:55,065][20381] Updated weights for policy 1, policy_version 1259474 (0.0010) [2023-12-27 14:22:55,118][20347] Updated weights for policy 0, policy_version 1260506 (0.0010) [2023-12-27 14:22:55,720][20381] Updated weights for policy 1, policy_version 1259484 (0.0009) [2023-12-27 14:22:55,767][20381] Updated weights for policy 1, policy_version 1259494 (0.0006) [2023-12-27 14:22:55,804][20347] Updated weights for policy 0, policy_version 1260516 (0.0010) [2023-12-27 14:22:55,813][20381] Updated weights for policy 1, policy_version 1259504 (0.0010) [2023-12-27 14:22:55,848][20347] Updated weights for policy 0, policy_version 1260526 (0.0010) [2023-12-27 14:22:55,897][20347] Updated weights for policy 0, policy_version 1260536 (0.0005) [2023-12-27 14:22:56,507][20381] Updated weights for policy 1, policy_version 1259514 (0.0011) [2023-12-27 14:22:56,511][20347] Updated weights for policy 0, policy_version 1260546 (0.0006) [2023-12-27 14:22:56,562][20381] Updated weights for policy 1, policy_version 1259524 (0.0010) [2023-12-27 14:22:56,568][20347] Updated weights for policy 0, policy_version 1260556 (0.0009) [2023-12-27 14:22:56,615][20381] Updated weights for policy 1, policy_version 1259534 (0.0008) [2023-12-27 14:22:56,632][20347] Updated weights for policy 0, policy_version 1260566 (0.0005) [2023-12-27 14:22:56,669][20381] Updated weights for policy 1, policy_version 1259544 (0.0010) [2023-12-27 14:22:56,683][20347] Updated weights for policy 0, policy_version 1260576 (0.0005) [2023-12-27 14:22:57,305][20347] Updated weights for policy 0, policy_version 1260586 (0.0011) [2023-12-27 14:22:57,338][20381] Updated weights for policy 1, policy_version 1259554 (0.0009) [2023-12-27 14:22:57,361][20347] Updated weights for policy 0, policy_version 1260596 (0.0010) [2023-12-27 14:22:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 645234688. Throughput: 0: 10243.0, 1: 10282.0. Samples: 645230148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:22:57,363][19170] Avg episode reward: [(0, '873.300'), (1, '680.350')] [2023-12-27 14:22:57,395][20381] Updated weights for policy 1, policy_version 1259564 (0.0005) [2023-12-27 14:22:57,418][20347] Updated weights for policy 0, policy_version 1260606 (0.0010) [2023-12-27 14:22:57,446][20381] Updated weights for policy 1, policy_version 1259574 (0.0006) [2023-12-27 14:22:58,003][20381] Updated weights for policy 1, policy_version 1259584 (0.0007) [2023-12-27 14:22:58,068][20381] Updated weights for policy 1, policy_version 1259594 (0.0007) [2023-12-27 14:22:58,075][20291] KL-divergence is very high: 187.6462 [2023-12-27 14:22:58,080][20291] KL-divergence is very high: 255.7811 [2023-12-27 14:22:58,085][20291] KL-divergence is very high: 298.2350 [2023-12-27 14:22:58,087][20347] Updated weights for policy 0, policy_version 1260616 (0.0008) [2023-12-27 14:22:58,096][20291] KL-divergence is very high: 423.4297 [2023-12-27 14:22:58,102][20291] KL-divergence is very high: 326.1053 [2023-12-27 14:22:58,121][20291] KL-divergence is very high: 104.6998 [2023-12-27 14:22:58,132][20381] Updated weights for policy 1, policy_version 1259604 (0.0005) [2023-12-27 14:22:58,143][20347] Updated weights for policy 0, policy_version 1260626 (0.0008) [2023-12-27 14:22:58,206][20347] Updated weights for policy 0, policy_version 1260636 (0.0007) [2023-12-27 14:22:58,874][20381] Updated weights for policy 1, policy_version 1259614 (0.0007) [2023-12-27 14:22:58,930][20291] KL-divergence is very high: 278.0178 [2023-12-27 14:22:58,941][20381] Updated weights for policy 1, policy_version 1259624 (0.0008) [2023-12-27 14:22:58,941][20291] KL-divergence is very high: 222.2910 [2023-12-27 14:22:58,958][20347] Updated weights for policy 0, policy_version 1260646 (0.0007) [2023-12-27 14:22:58,973][20291] KL-divergence is very high: 264.0168 [2023-12-27 14:22:58,986][20291] KL-divergence is very high: 199.6561 [2023-12-27 14:22:59,001][20381] Updated weights for policy 1, policy_version 1259634 (0.0007) [2023-12-27 14:22:59,016][20347] Updated weights for policy 0, policy_version 1260656 (0.0007) [2023-12-27 14:22:59,024][20291] KL-divergence is very high: 187.4491 [2023-12-27 14:22:59,036][20291] KL-divergence is very high: 160.8623 [2023-12-27 14:22:59,068][20291] KL-divergence is very high: 260.9898 [2023-12-27 14:22:59,072][20347] Updated weights for policy 0, policy_version 1260666 (0.0006) [2023-12-27 14:22:59,079][20291] KL-divergence is very high: 212.4556 [2023-12-27 14:22:59,695][20381] Updated weights for policy 1, policy_version 1259644 (0.0006) [2023-12-27 14:22:59,756][20381] Updated weights for policy 1, policy_version 1259654 (0.0008) [2023-12-27 14:22:59,790][20347] Updated weights for policy 0, policy_version 1260676 (0.0007) [2023-12-27 14:22:59,819][20381] Updated weights for policy 1, policy_version 1259664 (0.0007) [2023-12-27 14:22:59,850][20347] Updated weights for policy 0, policy_version 1260686 (0.0006) [2023-12-27 14:22:59,905][20347] Updated weights for policy 0, policy_version 1260696 (0.0007) [2023-12-27 14:23:00,426][20381] Updated weights for policy 1, policy_version 1259674 (0.0007) [2023-12-27 14:23:00,479][20381] Updated weights for policy 1, policy_version 1259684 (0.0008) [2023-12-27 14:23:00,538][20381] Updated weights for policy 1, policy_version 1259694 (0.0008) [2023-12-27 14:23:00,585][20381] Updated weights for policy 1, policy_version 1259704 (0.0008) [2023-12-27 14:23:00,640][20347] Updated weights for policy 0, policy_version 1260706 (0.0007) [2023-12-27 14:23:00,687][20347] Updated weights for policy 0, policy_version 1260716 (0.0007) [2023-12-27 14:23:00,731][20347] Updated weights for policy 0, policy_version 1260726 (0.0007) [2023-12-27 14:23:00,774][20347] Updated weights for policy 0, policy_version 1260736 (0.0008) [2023-12-27 14:23:01,199][20381] Updated weights for policy 1, policy_version 1259714 (0.0008) [2023-12-27 14:23:01,251][20381] Updated weights for policy 1, policy_version 1259724 (0.0008) [2023-12-27 14:23:01,305][20381] Updated weights for policy 1, policy_version 1259734 (0.0008) [2023-12-27 14:23:01,481][20347] Updated weights for policy 0, policy_version 1260746 (0.0007) [2023-12-27 14:23:01,531][20347] Updated weights for policy 0, policy_version 1260756 (0.0006) [2023-12-27 14:23:01,579][20347] Updated weights for policy 0, policy_version 1260766 (0.0010) [2023-12-27 14:23:02,070][20381] Updated weights for policy 1, policy_version 1259744 (0.0007) [2023-12-27 14:23:02,125][20381] Updated weights for policy 1, policy_version 1259754 (0.0007) [2023-12-27 14:23:02,178][20381] Updated weights for policy 1, policy_version 1259764 (0.0008) [2023-12-27 14:23:02,249][20347] Updated weights for policy 0, policy_version 1260776 (0.0011) [2023-12-27 14:23:02,313][20347] Updated weights for policy 0, policy_version 1260786 (0.0007) [2023-12-27 14:23:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 645341184. Throughput: 0: 10209.9, 1: 10271.9. Samples: 645352748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:02,363][19170] Avg episode reward: [(0, '804.810'), (1, '740.040')] [2023-12-27 14:23:02,383][20347] Updated weights for policy 0, policy_version 1260796 (0.0007) [2023-12-27 14:23:02,929][20381] Updated weights for policy 1, policy_version 1259774 (0.0008) [2023-12-27 14:23:02,962][20347] Updated weights for policy 0, policy_version 1260806 (0.0009) [2023-12-27 14:23:02,988][20381] Updated weights for policy 1, policy_version 1259784 (0.0008) [2023-12-27 14:23:03,021][20347] Updated weights for policy 0, policy_version 1260816 (0.0006) [2023-12-27 14:23:03,043][20381] Updated weights for policy 1, policy_version 1259794 (0.0008) [2023-12-27 14:23:03,071][20347] Updated weights for policy 0, policy_version 1260826 (0.0006) [2023-12-27 14:23:03,673][20347] Updated weights for policy 0, policy_version 1260836 (0.0006) [2023-12-27 14:23:03,724][20347] Updated weights for policy 0, policy_version 1260846 (0.0006) [2023-12-27 14:23:03,743][20381] Updated weights for policy 1, policy_version 1259804 (0.0007) [2023-12-27 14:23:03,773][20347] Updated weights for policy 0, policy_version 1260856 (0.0007) [2023-12-27 14:23:03,798][20381] Updated weights for policy 1, policy_version 1259814 (0.0007) [2023-12-27 14:23:03,852][20381] Updated weights for policy 1, policy_version 1259824 (0.0007) [2023-12-27 14:23:04,410][20347] Updated weights for policy 0, policy_version 1260866 (0.0007) [2023-12-27 14:23:04,461][20347] Updated weights for policy 0, policy_version 1260876 (0.0008) [2023-12-27 14:23:04,474][20381] Updated weights for policy 1, policy_version 1259834 (0.0007) [2023-12-27 14:23:04,517][20347] Updated weights for policy 0, policy_version 1260886 (0.0007) [2023-12-27 14:23:04,530][20381] Updated weights for policy 1, policy_version 1259844 (0.0011) [2023-12-27 14:23:04,570][20347] Updated weights for policy 0, policy_version 1260896 (0.0007) [2023-12-27 14:23:04,580][20381] Updated weights for policy 1, policy_version 1259854 (0.0008) [2023-12-27 14:23:04,636][20381] Updated weights for policy 1, policy_version 1259864 (0.0007) [2023-12-27 14:23:05,207][20347] Updated weights for policy 0, policy_version 1260906 (0.0006) [2023-12-27 14:23:05,253][20347] Updated weights for policy 0, policy_version 1260916 (0.0006) [2023-12-27 14:23:05,282][20381] Updated weights for policy 1, policy_version 1259874 (0.0007) [2023-12-27 14:23:05,301][20347] Updated weights for policy 0, policy_version 1260926 (0.0007) [2023-12-27 14:23:05,333][20381] Updated weights for policy 1, policy_version 1259884 (0.0005) [2023-12-27 14:23:05,381][20381] Updated weights for policy 1, policy_version 1259894 (0.0005) [2023-12-27 14:23:05,941][20381] Updated weights for policy 1, policy_version 1259904 (0.0006) [2023-12-27 14:23:05,994][20347] Updated weights for policy 0, policy_version 1260936 (0.0009) [2023-12-27 14:23:05,996][20381] Updated weights for policy 1, policy_version 1259914 (0.0006) [2023-12-27 14:23:06,049][20347] Updated weights for policy 0, policy_version 1260946 (0.0007) [2023-12-27 14:23:06,059][20381] Updated weights for policy 1, policy_version 1259924 (0.0006) [2023-12-27 14:23:06,106][20347] Updated weights for policy 0, policy_version 1260956 (0.0008) [2023-12-27 14:23:06,757][20381] Updated weights for policy 1, policy_version 1259934 (0.0010) [2023-12-27 14:23:06,806][20347] Updated weights for policy 0, policy_version 1260966 (0.0006) [2023-12-27 14:23:06,815][20381] Updated weights for policy 1, policy_version 1259944 (0.0010) [2023-12-27 14:23:06,854][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 14:23:06,854][20347] Updated weights for policy 0, policy_version 1260976 (0.0006) [2023-12-27 14:23:06,871][20381] Updated weights for policy 1, policy_version 1259954 (0.0010) [2023-12-27 14:23:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 645447680. Throughput: 0: 10285.4, 1: 10307.4. Samples: 645417064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:07,363][19170] Avg episode reward: [(0, '814.650'), (1, '645.180')] [2023-12-27 14:23:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001259960_322576384.pth... [2023-12-27 14:23:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001260976_322871296.pth... [2023-12-27 14:23:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001258744_322265088.pth [2023-12-27 14:23:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001259808_322568192.pth [2023-12-27 14:23:07,509][20381] Updated weights for policy 1, policy_version 1259964 (0.0007) [2023-12-27 14:23:07,568][20381] Updated weights for policy 1, policy_version 1259974 (0.0011) [2023-12-27 14:23:07,568][20347] Updated weights for policy 0, policy_version 1260986 (0.0005) [2023-12-27 14:23:07,625][20347] Updated weights for policy 0, policy_version 1260996 (0.0006) [2023-12-27 14:23:07,626][20381] Updated weights for policy 1, policy_version 1259984 (0.0007) [2023-12-27 14:23:07,680][20347] Updated weights for policy 0, policy_version 1261006 (0.0006) [2023-12-27 14:23:08,228][20381] Updated weights for policy 1, policy_version 1259994 (0.0007) [2023-12-27 14:23:08,252][20347] Updated weights for policy 0, policy_version 1261016 (0.0005) [2023-12-27 14:23:08,270][20381] Updated weights for policy 1, policy_version 1260004 (0.0010) [2023-12-27 14:23:08,305][20347] Updated weights for policy 0, policy_version 1261026 (0.0007) [2023-12-27 14:23:08,328][20381] Updated weights for policy 1, policy_version 1260014 (0.0010) [2023-12-27 14:23:08,368][20347] Updated weights for policy 0, policy_version 1261036 (0.0007) [2023-12-27 14:23:08,394][20381] Updated weights for policy 1, policy_version 1260024 (0.0009) [2023-12-27 14:23:09,056][20347] Updated weights for policy 0, policy_version 1261046 (0.0007) [2023-12-27 14:23:09,112][20347] Updated weights for policy 0, policy_version 1261056 (0.0007) [2023-12-27 14:23:09,118][20381] Updated weights for policy 1, policy_version 1260034 (0.0011) [2023-12-27 14:23:09,169][20347] Updated weights for policy 0, policy_version 1261066 (0.0007) [2023-12-27 14:23:09,174][20381] Updated weights for policy 1, policy_version 1260044 (0.0011) [2023-12-27 14:23:09,240][20381] Updated weights for policy 1, policy_version 1260054 (0.0008) [2023-12-27 14:23:09,896][20347] Updated weights for policy 0, policy_version 1261076 (0.0007) [2023-12-27 14:23:09,960][20347] Updated weights for policy 0, policy_version 1261086 (0.0008) [2023-12-27 14:23:10,012][20347] Updated weights for policy 0, policy_version 1261096 (0.0007) [2023-12-27 14:23:10,029][20381] Updated weights for policy 1, policy_version 1260064 (0.0007) [2023-12-27 14:23:10,085][20381] Updated weights for policy 1, policy_version 1260074 (0.0006) [2023-12-27 14:23:10,152][20381] Updated weights for policy 1, policy_version 1260084 (0.0006) [2023-12-27 14:23:10,690][20347] Updated weights for policy 0, policy_version 1261106 (0.0006) [2023-12-27 14:23:10,748][20347] Updated weights for policy 0, policy_version 1261116 (0.0007) [2023-12-27 14:23:10,761][20381] Updated weights for policy 1, policy_version 1260094 (0.0005) [2023-12-27 14:23:10,800][20347] Updated weights for policy 0, policy_version 1261126 (0.0007) [2023-12-27 14:23:10,821][20381] Updated weights for policy 1, policy_version 1260104 (0.0006) [2023-12-27 14:23:10,850][20347] Updated weights for policy 0, policy_version 1261136 (0.0007) [2023-12-27 14:23:10,886][20381] Updated weights for policy 1, policy_version 1260114 (0.0007) [2023-12-27 14:23:11,557][20347] Updated weights for policy 0, policy_version 1261146 (0.0006) [2023-12-27 14:23:11,589][20381] Updated weights for policy 1, policy_version 1260124 (0.0008) [2023-12-27 14:23:11,620][20347] Updated weights for policy 0, policy_version 1261156 (0.0010) [2023-12-27 14:23:11,652][20381] Updated weights for policy 1, policy_version 1260134 (0.0006) [2023-12-27 14:23:11,687][20347] Updated weights for policy 0, policy_version 1261166 (0.0008) [2023-12-27 14:23:11,715][20381] Updated weights for policy 1, policy_version 1260144 (0.0008) [2023-12-27 14:23:12,279][20347] Updated weights for policy 0, policy_version 1261176 (0.0010) [2023-12-27 14:23:12,332][20347] Updated weights for policy 0, policy_version 1261186 (0.0010) [2023-12-27 14:23:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 645545984. Throughput: 0: 10317.4, 1: 10310.7. Samples: 645542500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:12,363][19170] Avg episode reward: [(0, '833.960'), (1, '699.150')] [2023-12-27 14:23:12,398][20347] Updated weights for policy 0, policy_version 1261196 (0.0011) [2023-12-27 14:23:12,480][20381] Updated weights for policy 1, policy_version 1260154 (0.0008) [2023-12-27 14:23:12,543][20381] Updated weights for policy 1, policy_version 1260164 (0.0007) [2023-12-27 14:23:12,605][20381] Updated weights for policy 1, policy_version 1260174 (0.0007) [2023-12-27 14:23:12,670][20381] Updated weights for policy 1, policy_version 1260184 (0.0008) [2023-12-27 14:23:13,049][20347] Updated weights for policy 0, policy_version 1261206 (0.0010) [2023-12-27 14:23:13,108][20347] Updated weights for policy 0, policy_version 1261216 (0.0009) [2023-12-27 14:23:13,167][20347] Updated weights for policy 0, policy_version 1261226 (0.0010) [2023-12-27 14:23:13,298][20381] Updated weights for policy 1, policy_version 1260194 (0.0007) [2023-12-27 14:23:13,346][20381] Updated weights for policy 1, policy_version 1260204 (0.0007) [2023-12-27 14:23:13,402][20381] Updated weights for policy 1, policy_version 1260214 (0.0008) [2023-12-27 14:23:13,806][20347] Updated weights for policy 0, policy_version 1261236 (0.0010) [2023-12-27 14:23:13,865][20347] Updated weights for policy 0, policy_version 1261246 (0.0010) [2023-12-27 14:23:13,920][20347] Updated weights for policy 0, policy_version 1261256 (0.0010) [2023-12-27 14:23:14,016][20381] Updated weights for policy 1, policy_version 1260224 (0.0007) [2023-12-27 14:23:14,064][20381] Updated weights for policy 1, policy_version 1260234 (0.0008) [2023-12-27 14:23:14,119][20381] Updated weights for policy 1, policy_version 1260244 (0.0008) [2023-12-27 14:23:14,602][20347] Updated weights for policy 0, policy_version 1261266 (0.0010) [2023-12-27 14:23:14,657][20347] Updated weights for policy 0, policy_version 1261276 (0.0010) [2023-12-27 14:23:14,711][20347] Updated weights for policy 0, policy_version 1261286 (0.0010) [2023-12-27 14:23:14,761][20347] Updated weights for policy 0, policy_version 1261296 (0.0008) [2023-12-27 14:23:14,787][20381] Updated weights for policy 1, policy_version 1260254 (0.0008) [2023-12-27 14:23:14,840][20381] Updated weights for policy 1, policy_version 1260264 (0.0007) [2023-12-27 14:23:14,900][20381] Updated weights for policy 1, policy_version 1260274 (0.0008) [2023-12-27 14:23:15,399][20347] Updated weights for policy 0, policy_version 1261306 (0.0011) [2023-12-27 14:23:15,459][20347] Updated weights for policy 0, policy_version 1261316 (0.0011) [2023-12-27 14:23:15,524][20347] Updated weights for policy 0, policy_version 1261326 (0.0009) [2023-12-27 14:23:15,640][20381] Updated weights for policy 1, policy_version 1260284 (0.0007) [2023-12-27 14:23:15,696][20381] Updated weights for policy 1, policy_version 1260294 (0.0006) [2023-12-27 14:23:15,762][20381] Updated weights for policy 1, policy_version 1260304 (0.0006) [2023-12-27 14:23:16,130][20347] Updated weights for policy 0, policy_version 1261336 (0.0007) [2023-12-27 14:23:16,186][20347] Updated weights for policy 0, policy_version 1261346 (0.0005) [2023-12-27 14:23:16,242][20347] Updated weights for policy 0, policy_version 1261356 (0.0005) [2023-12-27 14:23:16,392][20381] Updated weights for policy 1, policy_version 1260314 (0.0005) [2023-12-27 14:23:16,440][20381] Updated weights for policy 1, policy_version 1260324 (0.0005) [2023-12-27 14:23:16,493][20381] Updated weights for policy 1, policy_version 1260334 (0.0005) [2023-12-27 14:23:16,539][20381] Updated weights for policy 1, policy_version 1260344 (0.0005) [2023-12-27 14:23:16,855][20347] Updated weights for policy 0, policy_version 1261366 (0.0005) [2023-12-27 14:23:16,914][20347] Updated weights for policy 0, policy_version 1261376 (0.0005) [2023-12-27 14:23:16,973][20347] Updated weights for policy 0, policy_version 1261386 (0.0005) [2023-12-27 14:23:17,104][20381] Updated weights for policy 1, policy_version 1260354 (0.0008) [2023-12-27 14:23:17,150][20381] Updated weights for policy 1, policy_version 1260364 (0.0007) [2023-12-27 14:23:17,197][20381] Updated weights for policy 1, policy_version 1260374 (0.0007) [2023-12-27 14:23:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 645660672. Throughput: 0: 10317.1, 1: 10336.4. Samples: 645669096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:17,363][19170] Avg episode reward: [(0, '882.800'), (1, '689.970')] [2023-12-27 14:23:17,555][20347] Updated weights for policy 0, policy_version 1261396 (0.0007) [2023-12-27 14:23:17,610][20347] Updated weights for policy 0, policy_version 1261406 (0.0009) [2023-12-27 14:23:17,666][20347] Updated weights for policy 0, policy_version 1261416 (0.0009) [2023-12-27 14:23:17,862][20381] Updated weights for policy 1, policy_version 1260384 (0.0006) [2023-12-27 14:23:17,910][20381] Updated weights for policy 1, policy_version 1260394 (0.0009) [2023-12-27 14:23:17,966][20381] Updated weights for policy 1, policy_version 1260404 (0.0011) [2023-12-27 14:23:18,347][20347] Updated weights for policy 0, policy_version 1261426 (0.0010) [2023-12-27 14:23:18,407][20347] Updated weights for policy 0, policy_version 1261436 (0.0007) [2023-12-27 14:23:18,467][20347] Updated weights for policy 0, policy_version 1261446 (0.0011) [2023-12-27 14:23:18,527][20347] Updated weights for policy 0, policy_version 1261456 (0.0011) [2023-12-27 14:23:18,712][20381] Updated weights for policy 1, policy_version 1260414 (0.0007) [2023-12-27 14:23:18,765][20381] Updated weights for policy 1, policy_version 1260424 (0.0008) [2023-12-27 14:23:18,818][20381] Updated weights for policy 1, policy_version 1260434 (0.0008) [2023-12-27 14:23:19,166][20347] Updated weights for policy 0, policy_version 1261466 (0.0007) [2023-12-27 14:23:19,225][20347] Updated weights for policy 0, policy_version 1261476 (0.0009) [2023-12-27 14:23:19,291][20347] Updated weights for policy 0, policy_version 1261486 (0.0007) [2023-12-27 14:23:19,430][20381] Updated weights for policy 1, policy_version 1260444 (0.0007) [2023-12-27 14:23:19,496][20381] Updated weights for policy 1, policy_version 1260454 (0.0008) [2023-12-27 14:23:19,559][20381] Updated weights for policy 1, policy_version 1260464 (0.0005) [2023-12-27 14:23:19,959][20347] Updated weights for policy 0, policy_version 1261496 (0.0007) [2023-12-27 14:23:20,017][20347] Updated weights for policy 0, policy_version 1261506 (0.0006) [2023-12-27 14:23:20,074][20347] Updated weights for policy 0, policy_version 1261516 (0.0006) [2023-12-27 14:23:20,255][20381] Updated weights for policy 1, policy_version 1260474 (0.0008) [2023-12-27 14:23:20,317][20381] Updated weights for policy 1, policy_version 1260484 (0.0007) [2023-12-27 14:23:20,378][20381] Updated weights for policy 1, policy_version 1260494 (0.0011) [2023-12-27 14:23:20,431][20381] Updated weights for policy 1, policy_version 1260504 (0.0011) [2023-12-27 14:23:20,723][20347] Updated weights for policy 0, policy_version 1261526 (0.0007) [2023-12-27 14:23:20,780][20347] Updated weights for policy 0, policy_version 1261536 (0.0007) [2023-12-27 14:23:20,846][20347] Updated weights for policy 0, policy_version 1261546 (0.0007) [2023-12-27 14:23:21,122][20381] Updated weights for policy 1, policy_version 1260514 (0.0011) [2023-12-27 14:23:21,186][20381] Updated weights for policy 1, policy_version 1260524 (0.0010) [2023-12-27 14:23:21,254][20381] Updated weights for policy 1, policy_version 1260534 (0.0009) [2023-12-27 14:23:21,585][20347] Updated weights for policy 0, policy_version 1261556 (0.0006) [2023-12-27 14:23:21,648][20347] Updated weights for policy 0, policy_version 1261566 (0.0008) [2023-12-27 14:23:21,720][20347] Updated weights for policy 0, policy_version 1261576 (0.0008) [2023-12-27 14:23:22,017][20381] Updated weights for policy 1, policy_version 1260544 (0.0009) [2023-12-27 14:23:22,080][20381] Updated weights for policy 1, policy_version 1260554 (0.0007) [2023-12-27 14:23:22,136][20381] Updated weights for policy 1, policy_version 1260564 (0.0006) [2023-12-27 14:23:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 645758976. Throughput: 0: 10321.4, 1: 10370.0. Samples: 645731692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:22,363][19170] Avg episode reward: [(0, '902.330'), (1, '709.340')] [2023-12-27 14:23:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001260568_322732032.pth... [2023-12-27 14:23:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001261584_323026944.pth... [2023-12-27 14:23:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001259352_322420736.pth [2023-12-27 14:23:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001260384_322715648.pth [2023-12-27 14:23:22,478][20347] Updated weights for policy 0, policy_version 1261586 (0.0008) [2023-12-27 14:23:22,539][20347] Updated weights for policy 0, policy_version 1261596 (0.0009) [2023-12-27 14:23:22,595][20347] Updated weights for policy 0, policy_version 1261606 (0.0007) [2023-12-27 14:23:22,647][20347] Updated weights for policy 0, policy_version 1261616 (0.0008) [2023-12-27 14:23:22,746][20381] Updated weights for policy 1, policy_version 1260574 (0.0006) [2023-12-27 14:23:22,799][20381] Updated weights for policy 1, policy_version 1260584 (0.0005) [2023-12-27 14:23:22,852][20381] Updated weights for policy 1, policy_version 1260594 (0.0005) [2023-12-27 14:23:23,376][20347] Updated weights for policy 0, policy_version 1261626 (0.0010) [2023-12-27 14:23:23,433][20347] Updated weights for policy 0, policy_version 1261636 (0.0010) [2023-12-27 14:23:23,485][20381] Updated weights for policy 1, policy_version 1260604 (0.0007) [2023-12-27 14:23:23,498][20347] Updated weights for policy 0, policy_version 1261646 (0.0005) [2023-12-27 14:23:23,544][20381] Updated weights for policy 1, policy_version 1260614 (0.0011) [2023-12-27 14:23:23,598][20381] Updated weights for policy 1, policy_version 1260624 (0.0010) [2023-12-27 14:23:24,082][20347] Updated weights for policy 0, policy_version 1261656 (0.0005) [2023-12-27 14:23:24,132][20347] Updated weights for policy 0, policy_version 1261666 (0.0005) [2023-12-27 14:23:24,189][20347] Updated weights for policy 0, policy_version 1261676 (0.0005) [2023-12-27 14:23:24,284][20381] Updated weights for policy 1, policy_version 1260634 (0.0008) [2023-12-27 14:23:24,337][20381] Updated weights for policy 1, policy_version 1260644 (0.0008) [2023-12-27 14:23:24,397][20381] Updated weights for policy 1, policy_version 1260654 (0.0007) [2023-12-27 14:23:24,458][20381] Updated weights for policy 1, policy_version 1260664 (0.0008) [2023-12-27 14:23:24,723][20347] Updated weights for policy 0, policy_version 1261686 (0.0006) [2023-12-27 14:23:24,784][20347] Updated weights for policy 0, policy_version 1261696 (0.0007) [2023-12-27 14:23:24,850][20347] Updated weights for policy 0, policy_version 1261706 (0.0005) [2023-12-27 14:23:25,038][20381] Updated weights for policy 1, policy_version 1260674 (0.0008) [2023-12-27 14:23:25,091][20381] Updated weights for policy 1, policy_version 1260684 (0.0008) [2023-12-27 14:23:25,135][20381] Updated weights for policy 1, policy_version 1260694 (0.0007) [2023-12-27 14:23:25,388][20347] Updated weights for policy 0, policy_version 1261716 (0.0008) [2023-12-27 14:23:25,432][20347] Updated weights for policy 0, policy_version 1261726 (0.0010) [2023-12-27 14:23:25,487][20347] Updated weights for policy 0, policy_version 1261736 (0.0008) [2023-12-27 14:23:25,718][20381] Updated weights for policy 1, policy_version 1260704 (0.0010) [2023-12-27 14:23:25,776][20381] Updated weights for policy 1, policy_version 1260714 (0.0010) [2023-12-27 14:23:25,842][20381] Updated weights for policy 1, policy_version 1260724 (0.0009) [2023-12-27 14:23:26,159][20347] Updated weights for policy 0, policy_version 1261746 (0.0010) [2023-12-27 14:23:26,210][20347] Updated weights for policy 0, policy_version 1261756 (0.0010) [2023-12-27 14:23:26,256][20347] Updated weights for policy 0, policy_version 1261766 (0.0010) [2023-12-27 14:23:26,314][20347] Updated weights for policy 0, policy_version 1261776 (0.0010) [2023-12-27 14:23:26,440][20381] Updated weights for policy 1, policy_version 1260734 (0.0008) [2023-12-27 14:23:26,498][20381] Updated weights for policy 1, policy_version 1260744 (0.0010) [2023-12-27 14:23:26,555][20381] Updated weights for policy 1, policy_version 1260754 (0.0009) [2023-12-27 14:23:27,009][20347] Updated weights for policy 0, policy_version 1261786 (0.0008) [2023-12-27 14:23:27,057][20347] Updated weights for policy 0, policy_version 1261796 (0.0008) [2023-12-27 14:23:27,109][20347] Updated weights for policy 0, policy_version 1261806 (0.0008) [2023-12-27 14:23:27,161][20381] Updated weights for policy 1, policy_version 1260764 (0.0007) [2023-12-27 14:23:27,204][20381] Updated weights for policy 1, policy_version 1260774 (0.0007) [2023-12-27 14:23:27,256][20381] Updated weights for policy 1, policy_version 1260784 (0.0007) [2023-12-27 14:23:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 645873664. Throughput: 0: 10413.1, 1: 10435.4. Samples: 645858784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:27,363][19170] Avg episode reward: [(0, '863.290'), (1, '719.370')] [2023-12-27 14:23:27,807][20347] Updated weights for policy 0, policy_version 1261816 (0.0010) [2023-12-27 14:23:27,868][20347] Updated weights for policy 0, policy_version 1261826 (0.0010) [2023-12-27 14:23:27,907][20381] Updated weights for policy 1, policy_version 1260794 (0.0008) [2023-12-27 14:23:27,932][20347] Updated weights for policy 0, policy_version 1261836 (0.0010) [2023-12-27 14:23:27,959][20381] Updated weights for policy 1, policy_version 1260804 (0.0010) [2023-12-27 14:23:28,014][20381] Updated weights for policy 1, policy_version 1260814 (0.0010) [2023-12-27 14:23:28,075][20381] Updated weights for policy 1, policy_version 1260824 (0.0011) [2023-12-27 14:23:28,582][20347] Updated weights for policy 0, policy_version 1261846 (0.0010) [2023-12-27 14:23:28,638][20347] Updated weights for policy 0, policy_version 1261856 (0.0010) [2023-12-27 14:23:28,687][20347] Updated weights for policy 0, policy_version 1261866 (0.0010) [2023-12-27 14:23:28,745][20381] Updated weights for policy 1, policy_version 1260834 (0.0010) [2023-12-27 14:23:28,800][20381] Updated weights for policy 1, policy_version 1260844 (0.0010) [2023-12-27 14:23:28,857][20381] Updated weights for policy 1, policy_version 1260854 (0.0011) [2023-12-27 14:23:29,393][20347] Updated weights for policy 0, policy_version 1261876 (0.0010) [2023-12-27 14:23:29,452][20347] Updated weights for policy 0, policy_version 1261886 (0.0007) [2023-12-27 14:23:29,509][20347] Updated weights for policy 0, policy_version 1261896 (0.0007) [2023-12-27 14:23:29,526][20381] Updated weights for policy 1, policy_version 1260864 (0.0008) [2023-12-27 14:23:29,573][20381] Updated weights for policy 1, policy_version 1260874 (0.0007) [2023-12-27 14:23:29,623][20381] Updated weights for policy 1, policy_version 1260884 (0.0007) [2023-12-27 14:23:30,194][20347] Updated weights for policy 0, policy_version 1261906 (0.0006) [2023-12-27 14:23:30,246][20347] Updated weights for policy 0, policy_version 1261916 (0.0008) [2023-12-27 14:23:30,304][20347] Updated weights for policy 0, policy_version 1261926 (0.0008) [2023-12-27 14:23:30,358][20347] Updated weights for policy 0, policy_version 1261936 (0.0007) [2023-12-27 14:23:30,376][20381] Updated weights for policy 1, policy_version 1260894 (0.0007) [2023-12-27 14:23:30,436][20381] Updated weights for policy 1, policy_version 1260904 (0.0007) [2023-12-27 14:23:30,482][20381] Updated weights for policy 1, policy_version 1260914 (0.0008) [2023-12-27 14:23:31,059][20347] Updated weights for policy 0, policy_version 1261946 (0.0007) [2023-12-27 14:23:31,116][20347] Updated weights for policy 0, policy_version 1261956 (0.0008) [2023-12-27 14:23:31,152][20381] Updated weights for policy 1, policy_version 1260924 (0.0006) [2023-12-27 14:23:31,174][20347] Updated weights for policy 0, policy_version 1261966 (0.0009) [2023-12-27 14:23:31,210][20381] Updated weights for policy 1, policy_version 1260934 (0.0008) [2023-12-27 14:23:31,267][20381] Updated weights for policy 1, policy_version 1260944 (0.0007) [2023-12-27 14:23:31,951][20381] Updated weights for policy 1, policy_version 1260954 (0.0007) [2023-12-27 14:23:31,977][20347] Updated weights for policy 0, policy_version 1261976 (0.0009) [2023-12-27 14:23:32,001][20381] Updated weights for policy 1, policy_version 1260964 (0.0009) [2023-12-27 14:23:32,034][20347] Updated weights for policy 0, policy_version 1261986 (0.0006) [2023-12-27 14:23:32,057][20381] Updated weights for policy 1, policy_version 1260974 (0.0011) [2023-12-27 14:23:32,093][20347] Updated weights for policy 0, policy_version 1261996 (0.0005) [2023-12-27 14:23:32,111][20381] Updated weights for policy 1, policy_version 1260984 (0.0011) [2023-12-27 14:23:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 645971968. Throughput: 0: 10436.3, 1: 10393.3. Samples: 645980216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:32,363][19170] Avg episode reward: [(0, '765.880'), (1, '665.580')] [2023-12-27 14:23:32,793][20347] Updated weights for policy 0, policy_version 1262006 (0.0008) [2023-12-27 14:23:32,801][20381] Updated weights for policy 1, policy_version 1260994 (0.0006) [2023-12-27 14:23:32,844][20347] Updated weights for policy 0, policy_version 1262016 (0.0007) [2023-12-27 14:23:32,853][20381] Updated weights for policy 1, policy_version 1261004 (0.0007) [2023-12-27 14:23:32,894][20347] Updated weights for policy 0, policy_version 1262026 (0.0006) [2023-12-27 14:23:32,905][20381] Updated weights for policy 1, policy_version 1261014 (0.0006) [2023-12-27 14:23:33,516][20347] Updated weights for policy 0, policy_version 1262036 (0.0009) [2023-12-27 14:23:33,566][20381] Updated weights for policy 1, policy_version 1261024 (0.0005) [2023-12-27 14:23:33,579][20347] Updated weights for policy 0, policy_version 1262046 (0.0011) [2023-12-27 14:23:33,626][20381] Updated weights for policy 1, policy_version 1261034 (0.0007) [2023-12-27 14:23:33,641][20347] Updated weights for policy 0, policy_version 1262056 (0.0007) [2023-12-27 14:23:33,691][20381] Updated weights for policy 1, policy_version 1261044 (0.0008) [2023-12-27 14:23:34,300][20347] Updated weights for policy 0, policy_version 1262066 (0.0008) [2023-12-27 14:23:34,344][20381] Updated weights for policy 1, policy_version 1261054 (0.0007) [2023-12-27 14:23:34,354][20347] Updated weights for policy 0, policy_version 1262076 (0.0006) [2023-12-27 14:23:34,401][20381] Updated weights for policy 1, policy_version 1261064 (0.0006) [2023-12-27 14:23:34,413][20291] KL-divergence is very high: 150.2128 [2023-12-27 14:23:34,419][20347] Updated weights for policy 0, policy_version 1262086 (0.0006) [2023-12-27 14:23:34,421][20291] KL-divergence is very high: 149.8737 [2023-12-27 14:23:34,433][20291] KL-divergence is very high: 138.7718 [2023-12-27 14:23:34,460][20381] Updated weights for policy 1, policy_version 1261074 (0.0005) [2023-12-27 14:23:34,480][20347] Updated weights for policy 0, policy_version 1262096 (0.0008) [2023-12-27 14:23:35,053][20381] Updated weights for policy 1, policy_version 1261084 (0.0008) [2023-12-27 14:23:35,086][20291] KL-divergence is very high: 137.6756 [2023-12-27 14:23:35,099][20291] KL-divergence is very high: 128.0764 [2023-12-27 14:23:35,104][20381] Updated weights for policy 1, policy_version 1261094 (0.0010) [2023-12-27 14:23:35,133][20347] Updated weights for policy 0, policy_version 1262106 (0.0007) [2023-12-27 14:23:35,134][20291] KL-divergence is very high: 104.1002 [2023-12-27 14:23:35,159][20381] Updated weights for policy 1, policy_version 1261104 (0.0010) [2023-12-27 14:23:35,174][20291] KL-divergence is very high: 160.7912 [2023-12-27 14:23:35,183][20347] Updated weights for policy 0, policy_version 1262116 (0.0005) [2023-12-27 14:23:35,185][20291] KL-divergence is very high: 133.0582 [2023-12-27 14:23:35,218][20291] KL-divergence is very high: 107.8821 [2023-12-27 14:23:35,242][20347] Updated weights for policy 0, policy_version 1262126 (0.0006) [2023-12-27 14:23:35,782][20381] Updated weights for policy 1, policy_version 1261114 (0.0009) [2023-12-27 14:23:35,842][20381] Updated weights for policy 1, policy_version 1261124 (0.0005) [2023-12-27 14:23:35,897][20381] Updated weights for policy 1, policy_version 1261134 (0.0009) [2023-12-27 14:23:35,909][20347] Updated weights for policy 0, policy_version 1262136 (0.0006) [2023-12-27 14:23:35,952][20381] Updated weights for policy 1, policy_version 1261144 (0.0010) [2023-12-27 14:23:35,963][20291] KL-divergence is very high: 166.0974 [2023-12-27 14:23:35,965][20347] Updated weights for policy 0, policy_version 1262146 (0.0006) [2023-12-27 14:23:36,002][20291] KL-divergence is very high: 164.8344 [2023-12-27 14:23:36,011][20347] Updated weights for policy 0, policy_version 1262156 (0.0007) [2023-12-27 14:23:36,624][20381] Updated weights for policy 1, policy_version 1261154 (0.0011) [2023-12-27 14:23:36,678][20381] Updated weights for policy 1, policy_version 1261164 (0.0011) [2023-12-27 14:23:36,729][20381] Updated weights for policy 1, policy_version 1261174 (0.0009) [2023-12-27 14:23:36,737][20347] Updated weights for policy 0, policy_version 1262166 (0.0008) [2023-12-27 14:23:36,789][20347] Updated weights for policy 0, policy_version 1262176 (0.0007) [2023-12-27 14:23:36,835][20347] Updated weights for policy 0, policy_version 1262186 (0.0008) [2023-12-27 14:23:37,323][20381] Updated weights for policy 1, policy_version 1261184 (0.0011) [2023-12-27 14:23:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 646070272. Throughput: 0: 10457.3, 1: 10421.7. Samples: 646043288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:37,363][19170] Avg episode reward: [(0, '775.850'), (1, '640.810')] [2023-12-27 14:23:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001262192_323182592.pth... [2023-12-27 14:23:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001260976_322871296.pth [2023-12-27 14:23:37,378][20381] Updated weights for policy 1, policy_version 1261194 (0.0011) [2023-12-27 14:23:37,430][20381] Updated weights for policy 1, policy_version 1261204 (0.0010) [2023-12-27 14:23:37,449][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001261208_322895872.pth... [2023-12-27 14:23:37,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001259960_322576384.pth [2023-12-27 14:23:37,587][20347] Updated weights for policy 0, policy_version 1262196 (0.0009) [2023-12-27 14:23:37,644][20347] Updated weights for policy 0, policy_version 1262206 (0.0007) [2023-12-27 14:23:37,705][20347] Updated weights for policy 0, policy_version 1262216 (0.0008) [2023-12-27 14:23:38,089][20381] Updated weights for policy 1, policy_version 1261214 (0.0007) [2023-12-27 14:23:38,141][20381] Updated weights for policy 1, policy_version 1261224 (0.0005) [2023-12-27 14:23:38,186][20381] Updated weights for policy 1, policy_version 1261234 (0.0005) [2023-12-27 14:23:38,407][20347] Updated weights for policy 0, policy_version 1262226 (0.0008) [2023-12-27 14:23:38,468][20347] Updated weights for policy 0, policy_version 1262236 (0.0006) [2023-12-27 14:23:38,528][20347] Updated weights for policy 0, policy_version 1262246 (0.0005) [2023-12-27 14:23:38,589][20347] Updated weights for policy 0, policy_version 1262256 (0.0006) [2023-12-27 14:23:38,844][20381] Updated weights for policy 1, policy_version 1261244 (0.0008) [2023-12-27 14:23:38,908][20381] Updated weights for policy 1, policy_version 1261254 (0.0011) [2023-12-27 14:23:38,967][20381] Updated weights for policy 1, policy_version 1261264 (0.0011) [2023-12-27 14:23:39,196][20347] Updated weights for policy 0, policy_version 1262266 (0.0006) [2023-12-27 14:23:39,260][20347] Updated weights for policy 0, policy_version 1262276 (0.0008) [2023-12-27 14:23:39,330][20347] Updated weights for policy 0, policy_version 1262286 (0.0010) [2023-12-27 14:23:39,721][20381] Updated weights for policy 1, policy_version 1261274 (0.0010) [2023-12-27 14:23:39,778][20381] Updated weights for policy 1, policy_version 1261284 (0.0008) [2023-12-27 14:23:39,841][20381] Updated weights for policy 1, policy_version 1261294 (0.0007) [2023-12-27 14:23:39,906][20381] Updated weights for policy 1, policy_version 1261304 (0.0007) [2023-12-27 14:23:40,065][20347] Updated weights for policy 0, policy_version 1262296 (0.0008) [2023-12-27 14:23:40,123][20347] Updated weights for policy 0, policy_version 1262306 (0.0009) [2023-12-27 14:23:40,180][20347] Updated weights for policy 0, policy_version 1262316 (0.0009) [2023-12-27 14:23:40,584][20381] Updated weights for policy 1, policy_version 1261314 (0.0006) [2023-12-27 14:23:40,638][20381] Updated weights for policy 1, policy_version 1261324 (0.0008) [2023-12-27 14:23:40,693][20381] Updated weights for policy 1, policy_version 1261334 (0.0008) [2023-12-27 14:23:40,981][20347] Updated weights for policy 0, policy_version 1262326 (0.0008) [2023-12-27 14:23:41,045][20347] Updated weights for policy 0, policy_version 1262336 (0.0007) [2023-12-27 14:23:41,108][20347] Updated weights for policy 0, policy_version 1262346 (0.0006) [2023-12-27 14:23:41,424][20381] Updated weights for policy 1, policy_version 1261344 (0.0007) [2023-12-27 14:23:41,485][20381] Updated weights for policy 1, policy_version 1261354 (0.0006) [2023-12-27 14:23:41,554][20381] Updated weights for policy 1, policy_version 1261364 (0.0008) [2023-12-27 14:23:41,877][20347] Updated weights for policy 0, policy_version 1262356 (0.0008) [2023-12-27 14:23:41,939][20347] Updated weights for policy 0, policy_version 1262366 (0.0008) [2023-12-27 14:23:42,002][20347] Updated weights for policy 0, policy_version 1262376 (0.0008) [2023-12-27 14:23:42,174][20381] Updated weights for policy 1, policy_version 1261374 (0.0008) [2023-12-27 14:23:42,237][20381] Updated weights for policy 1, policy_version 1261384 (0.0008) [2023-12-27 14:23:42,298][20381] Updated weights for policy 1, policy_version 1261394 (0.0008) [2023-12-27 14:23:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 646176768. Throughput: 0: 10338.5, 1: 10402.1. Samples: 646163476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:42,363][19170] Avg episode reward: [(0, '765.450'), (1, '661.110')] [2023-12-27 14:23:42,656][20347] Updated weights for policy 0, policy_version 1262386 (0.0009) [2023-12-27 14:23:42,702][20347] Updated weights for policy 0, policy_version 1262396 (0.0007) [2023-12-27 14:23:42,750][20347] Updated weights for policy 0, policy_version 1262406 (0.0007) [2023-12-27 14:23:42,811][20347] Updated weights for policy 0, policy_version 1262416 (0.0008) [2023-12-27 14:23:43,005][20381] Updated weights for policy 1, policy_version 1261404 (0.0008) [2023-12-27 14:23:43,054][20381] Updated weights for policy 1, policy_version 1261414 (0.0007) [2023-12-27 14:23:43,111][20381] Updated weights for policy 1, policy_version 1261424 (0.0008) [2023-12-27 14:23:43,526][20347] Updated weights for policy 0, policy_version 1262426 (0.0006) [2023-12-27 14:23:43,577][20347] Updated weights for policy 0, policy_version 1262436 (0.0007) [2023-12-27 14:23:43,626][20347] Updated weights for policy 0, policy_version 1262446 (0.0005) [2023-12-27 14:23:43,787][20381] Updated weights for policy 1, policy_version 1261434 (0.0009) [2023-12-27 14:23:43,838][20381] Updated weights for policy 1, policy_version 1261444 (0.0010) [2023-12-27 14:23:43,884][20381] Updated weights for policy 1, policy_version 1261454 (0.0010) [2023-12-27 14:23:43,938][20381] Updated weights for policy 1, policy_version 1261464 (0.0010) [2023-12-27 14:23:44,272][20347] Updated weights for policy 0, policy_version 1262456 (0.0007) [2023-12-27 14:23:44,333][20347] Updated weights for policy 0, policy_version 1262466 (0.0007) [2023-12-27 14:23:44,386][20347] Updated weights for policy 0, policy_version 1262476 (0.0007) [2023-12-27 14:23:44,634][20381] Updated weights for policy 1, policy_version 1261474 (0.0010) [2023-12-27 14:23:44,695][20381] Updated weights for policy 1, policy_version 1261484 (0.0010) [2023-12-27 14:23:44,752][20381] Updated weights for policy 1, policy_version 1261494 (0.0010) [2023-12-27 14:23:45,058][20347] Updated weights for policy 0, policy_version 1262486 (0.0006) [2023-12-27 14:23:45,113][20347] Updated weights for policy 0, policy_version 1262496 (0.0007) [2023-12-27 14:23:45,158][20347] Updated weights for policy 0, policy_version 1262506 (0.0007) [2023-12-27 14:23:45,445][20381] Updated weights for policy 1, policy_version 1261504 (0.0010) [2023-12-27 14:23:45,498][20381] Updated weights for policy 1, policy_version 1261514 (0.0006) [2023-12-27 14:23:45,555][20381] Updated weights for policy 1, policy_version 1261524 (0.0006) [2023-12-27 14:23:45,850][20347] Updated weights for policy 0, policy_version 1262516 (0.0008) [2023-12-27 14:23:45,905][20347] Updated weights for policy 0, policy_version 1262526 (0.0010) [2023-12-27 14:23:45,961][20347] Updated weights for policy 0, policy_version 1262536 (0.0010) [2023-12-27 14:23:46,167][20381] Updated weights for policy 1, policy_version 1261534 (0.0005) [2023-12-27 14:23:46,231][20381] Updated weights for policy 1, policy_version 1261544 (0.0006) [2023-12-27 14:23:46,291][20381] Updated weights for policy 1, policy_version 1261554 (0.0005) [2023-12-27 14:23:46,605][20347] Updated weights for policy 0, policy_version 1262546 (0.0009) [2023-12-27 14:23:46,654][20347] Updated weights for policy 0, policy_version 1262556 (0.0006) [2023-12-27 14:23:46,705][20347] Updated weights for policy 0, policy_version 1262566 (0.0005) [2023-12-27 14:23:46,759][20347] Updated weights for policy 0, policy_version 1262576 (0.0006) [2023-12-27 14:23:46,893][20381] Updated weights for policy 1, policy_version 1261564 (0.0007) [2023-12-27 14:23:46,946][20381] Updated weights for policy 1, policy_version 1261574 (0.0008) [2023-12-27 14:23:47,000][20381] Updated weights for policy 1, policy_version 1261584 (0.0007) [2023-12-27 14:23:47,357][20347] Updated weights for policy 0, policy_version 1262586 (0.0005) [2023-12-27 14:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 646275072. Throughput: 0: 10354.8, 1: 10407.5. Samples: 646287056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:23:47,363][19170] Avg episode reward: [(0, '784.800'), (1, '653.490')] [2023-12-27 14:23:47,417][20347] Updated weights for policy 0, policy_version 1262596 (0.0005) [2023-12-27 14:23:47,488][20347] Updated weights for policy 0, policy_version 1262606 (0.0005) [2023-12-27 14:23:47,689][20381] Updated weights for policy 1, policy_version 1261594 (0.0008) [2023-12-27 14:23:47,748][20381] Updated weights for policy 1, policy_version 1261604 (0.0007) [2023-12-27 14:23:47,804][20381] Updated weights for policy 1, policy_version 1261614 (0.0007) [2023-12-27 14:23:47,863][20381] Updated weights for policy 1, policy_version 1261624 (0.0008) [2023-12-27 14:23:48,033][20347] Updated weights for policy 0, policy_version 1262616 (0.0006) [2023-12-27 14:23:48,081][20347] Updated weights for policy 0, policy_version 1262626 (0.0005) [2023-12-27 14:23:48,135][20347] Updated weights for policy 0, policy_version 1262636 (0.0006) [2023-12-27 14:23:48,554][20381] Updated weights for policy 1, policy_version 1261634 (0.0008) [2023-12-27 14:23:48,604][20381] Updated weights for policy 1, policy_version 1261644 (0.0008) [2023-12-27 14:23:48,661][20381] Updated weights for policy 1, policy_version 1261654 (0.0007) [2023-12-27 14:23:48,850][20347] Updated weights for policy 0, policy_version 1262646 (0.0008) [2023-12-27 14:23:48,905][20347] Updated weights for policy 0, policy_version 1262656 (0.0010) [2023-12-27 14:23:48,964][20347] Updated weights for policy 0, policy_version 1262666 (0.0008) [2023-12-27 14:23:49,446][20381] Updated weights for policy 1, policy_version 1261664 (0.0008) [2023-12-27 14:23:49,506][20381] Updated weights for policy 1, policy_version 1261674 (0.0008) [2023-12-27 14:23:49,566][20381] Updated weights for policy 1, policy_version 1261684 (0.0008) [2023-12-27 14:23:49,626][20347] Updated weights for policy 0, policy_version 1262676 (0.0006) [2023-12-27 14:23:49,685][20347] Updated weights for policy 0, policy_version 1262686 (0.0008) [2023-12-27 14:23:49,745][20347] Updated weights for policy 0, policy_version 1262696 (0.0009) [2023-12-27 14:23:50,252][20381] Updated weights for policy 1, policy_version 1261694 (0.0007) [2023-12-27 14:23:50,308][20381] Updated weights for policy 1, policy_version 1261704 (0.0007) [2023-12-27 14:23:50,370][20381] Updated weights for policy 1, policy_version 1261714 (0.0008) [2023-12-27 14:23:50,458][20347] Updated weights for policy 0, policy_version 1262706 (0.0006) [2023-12-27 14:23:50,513][20347] Updated weights for policy 0, policy_version 1262716 (0.0007) [2023-12-27 14:23:50,577][20347] Updated weights for policy 0, policy_version 1262726 (0.0010) [2023-12-27 14:23:50,633][20347] Updated weights for policy 0, policy_version 1262736 (0.0008) [2023-12-27 14:23:51,036][20381] Updated weights for policy 1, policy_version 1261724 (0.0008) [2023-12-27 14:23:51,098][20381] Updated weights for policy 1, policy_version 1261734 (0.0009) [2023-12-27 14:23:51,158][20381] Updated weights for policy 1, policy_version 1261744 (0.0008) [2023-12-27 14:23:51,284][20347] Updated weights for policy 0, policy_version 1262746 (0.0009) [2023-12-27 14:23:51,341][20347] Updated weights for policy 0, policy_version 1262756 (0.0009) [2023-12-27 14:23:51,407][20347] Updated weights for policy 0, policy_version 1262766 (0.0007) [2023-12-27 14:23:51,883][20381] Updated weights for policy 1, policy_version 1261754 (0.0011) [2023-12-27 14:23:51,929][20381] Updated weights for policy 1, policy_version 1261764 (0.0011) [2023-12-27 14:23:51,974][20381] Updated weights for policy 1, policy_version 1261774 (0.0009) [2023-12-27 14:23:52,039][20381] Updated weights for policy 1, policy_version 1261784 (0.0011) [2023-12-27 14:23:52,116][20347] Updated weights for policy 0, policy_version 1262776 (0.0006) [2023-12-27 14:23:52,163][20347] Updated weights for policy 0, policy_version 1262786 (0.0006) [2023-12-27 14:23:52,226][20347] Updated weights for policy 0, policy_version 1262796 (0.0005) [2023-12-27 14:23:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 646381568. Throughput: 0: 10328.0, 1: 10373.8. Samples: 646348648. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:23:52,363][19170] Avg episode reward: [(0, '765.400'), (1, '683.250')] [2023-12-27 14:23:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001261784_323043328.pth... [2023-12-27 14:23:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001262800_323338240.pth... [2023-12-27 14:23:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001260568_322732032.pth [2023-12-27 14:23:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001261584_323026944.pth [2023-12-27 14:23:52,788][20381] Updated weights for policy 1, policy_version 1261794 (0.0007) [2023-12-27 14:23:52,837][20347] Updated weights for policy 0, policy_version 1262806 (0.0006) [2023-12-27 14:23:52,846][20381] Updated weights for policy 1, policy_version 1261804 (0.0007) [2023-12-27 14:23:52,892][20347] Updated weights for policy 0, policy_version 1262816 (0.0006) [2023-12-27 14:23:52,896][20381] Updated weights for policy 1, policy_version 1261814 (0.0006) [2023-12-27 14:23:52,945][20347] Updated weights for policy 0, policy_version 1262826 (0.0005) [2023-12-27 14:23:53,474][20381] Updated weights for policy 1, policy_version 1261824 (0.0006) [2023-12-27 14:23:53,498][20347] Updated weights for policy 0, policy_version 1262836 (0.0005) [2023-12-27 14:23:53,527][20381] Updated weights for policy 1, policy_version 1261834 (0.0006) [2023-12-27 14:23:53,547][20347] Updated weights for policy 0, policy_version 1262846 (0.0006) [2023-12-27 14:23:53,577][20381] Updated weights for policy 1, policy_version 1261844 (0.0009) [2023-12-27 14:23:53,595][20347] Updated weights for policy 0, policy_version 1262856 (0.0008) [2023-12-27 14:23:54,256][20381] Updated weights for policy 1, policy_version 1261854 (0.0010) [2023-12-27 14:23:54,292][20347] Updated weights for policy 0, policy_version 1262866 (0.0007) [2023-12-27 14:23:54,318][20381] Updated weights for policy 1, policy_version 1261864 (0.0011) [2023-12-27 14:23:54,341][20347] Updated weights for policy 0, policy_version 1262876 (0.0006) [2023-12-27 14:23:54,372][20381] Updated weights for policy 1, policy_version 1261874 (0.0010) [2023-12-27 14:23:54,390][20347] Updated weights for policy 0, policy_version 1262886 (0.0008) [2023-12-27 14:23:54,437][20347] Updated weights for policy 0, policy_version 1262896 (0.0007) [2023-12-27 14:23:55,057][20381] Updated weights for policy 1, policy_version 1261884 (0.0008) [2023-12-27 14:23:55,111][20381] Updated weights for policy 1, policy_version 1261894 (0.0008) [2023-12-27 14:23:55,162][20381] Updated weights for policy 1, policy_version 1261904 (0.0007) [2023-12-27 14:23:55,178][20347] Updated weights for policy 0, policy_version 1262906 (0.0006) [2023-12-27 14:23:55,237][20347] Updated weights for policy 0, policy_version 1262916 (0.0005) [2023-12-27 14:23:55,287][20347] Updated weights for policy 0, policy_version 1262926 (0.0005) [2023-12-27 14:23:55,750][20381] Updated weights for policy 1, policy_version 1261914 (0.0007) [2023-12-27 14:23:55,813][20381] Updated weights for policy 1, policy_version 1261924 (0.0005) [2023-12-27 14:23:55,875][20381] Updated weights for policy 1, policy_version 1261934 (0.0005) [2023-12-27 14:23:55,911][20291] KL-divergence is very high: 113.8818 [2023-12-27 14:23:55,933][20381] Updated weights for policy 1, policy_version 1261944 (0.0005) [2023-12-27 14:23:55,935][20347] Updated weights for policy 0, policy_version 1262936 (0.0009) [2023-12-27 14:23:55,947][20291] KL-divergence is very high: 133.4332 [2023-12-27 14:23:55,983][20347] Updated weights for policy 0, policy_version 1262946 (0.0007) [2023-12-27 14:23:55,990][20291] KL-divergence is very high: 121.6806 [2023-12-27 14:23:56,030][20291] KL-divergence is very high: 102.8569 [2023-12-27 14:23:56,035][20347] Updated weights for policy 0, policy_version 1262956 (0.0010) [2023-12-27 14:23:56,506][20381] Updated weights for policy 1, policy_version 1261954 (0.0007) [2023-12-27 14:23:56,567][20381] Updated weights for policy 1, policy_version 1261964 (0.0006) [2023-12-27 14:23:56,626][20381] Updated weights for policy 1, policy_version 1261974 (0.0007) [2023-12-27 14:23:56,677][20347] Updated weights for policy 0, policy_version 1262966 (0.0007) [2023-12-27 14:23:56,732][20347] Updated weights for policy 0, policy_version 1262976 (0.0005) [2023-12-27 14:23:56,793][20347] Updated weights for policy 0, policy_version 1262986 (0.0005) [2023-12-27 14:23:57,292][20381] Updated weights for policy 1, policy_version 1261984 (0.0008) [2023-12-27 14:23:57,309][20347] Updated weights for policy 0, policy_version 1262996 (0.0006) [2023-12-27 14:23:57,350][20381] Updated weights for policy 1, policy_version 1261994 (0.0008) [2023-12-27 14:23:57,360][20347] Updated weights for policy 0, policy_version 1263006 (0.0006) [2023-12-27 14:23:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 646479872. Throughput: 0: 10331.7, 1: 10366.5. Samples: 646473920. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:23:57,363][19170] Avg episode reward: [(0, '774.890'), (1, '645.610')] [2023-12-27 14:23:57,404][20381] Updated weights for policy 1, policy_version 1262004 (0.0007) [2023-12-27 14:23:57,412][20347] Updated weights for policy 0, policy_version 1263016 (0.0005) [2023-12-27 14:23:58,008][20381] Updated weights for policy 1, policy_version 1262014 (0.0006) [2023-12-27 14:23:58,072][20381] Updated weights for policy 1, policy_version 1262024 (0.0005) [2023-12-27 14:23:58,077][20347] Updated weights for policy 0, policy_version 1263026 (0.0006) [2023-12-27 14:23:58,127][20347] Updated weights for policy 0, policy_version 1263036 (0.0009) [2023-12-27 14:23:58,129][20381] Updated weights for policy 1, policy_version 1262034 (0.0005) [2023-12-27 14:23:58,187][20347] Updated weights for policy 0, policy_version 1263046 (0.0010) [2023-12-27 14:23:58,251][20347] Updated weights for policy 0, policy_version 1263056 (0.0008) [2023-12-27 14:23:58,872][20381] Updated weights for policy 1, policy_version 1262044 (0.0006) [2023-12-27 14:23:58,937][20381] Updated weights for policy 1, policy_version 1262054 (0.0008) [2023-12-27 14:23:58,995][20381] Updated weights for policy 1, policy_version 1262064 (0.0006) [2023-12-27 14:23:59,091][20347] Updated weights for policy 0, policy_version 1263066 (0.0008) [2023-12-27 14:23:59,158][20347] Updated weights for policy 0, policy_version 1263076 (0.0007) [2023-12-27 14:23:59,220][20347] Updated weights for policy 0, policy_version 1263086 (0.0008) [2023-12-27 14:23:59,783][20381] Updated weights for policy 1, policy_version 1262074 (0.0007) [2023-12-27 14:23:59,847][20381] Updated weights for policy 1, policy_version 1262084 (0.0008) [2023-12-27 14:23:59,857][20347] Updated weights for policy 0, policy_version 1263096 (0.0009) [2023-12-27 14:23:59,913][20381] Updated weights for policy 1, policy_version 1262094 (0.0007) [2023-12-27 14:23:59,921][20347] Updated weights for policy 0, policy_version 1263106 (0.0008) [2023-12-27 14:23:59,979][20381] Updated weights for policy 1, policy_version 1262104 (0.0008) [2023-12-27 14:23:59,992][20347] Updated weights for policy 0, policy_version 1263116 (0.0008) [2023-12-27 14:24:00,689][20381] Updated weights for policy 1, policy_version 1262114 (0.0007) [2023-12-27 14:24:00,693][20347] Updated weights for policy 0, policy_version 1263126 (0.0007) [2023-12-27 14:24:00,694][20291] KL-divergence is very high: 110.7946 [2023-12-27 14:24:00,751][20381] Updated weights for policy 1, policy_version 1262124 (0.0006) [2023-12-27 14:24:00,753][20347] Updated weights for policy 0, policy_version 1263136 (0.0006) [2023-12-27 14:24:00,785][20291] KL-divergence is very high: 101.8915 [2023-12-27 14:24:00,804][20347] Updated weights for policy 0, policy_version 1263146 (0.0005) [2023-12-27 14:24:00,810][20381] Updated weights for policy 1, policy_version 1262134 (0.0007) [2023-12-27 14:24:00,823][20291] KL-divergence is very high: 140.7523 [2023-12-27 14:24:01,491][20347] Updated weights for policy 0, policy_version 1263156 (0.0007) [2023-12-27 14:24:01,539][20347] Updated weights for policy 0, policy_version 1263166 (0.0008) [2023-12-27 14:24:01,582][20381] Updated weights for policy 1, policy_version 1262145 (0.0008) [2023-12-27 14:24:01,604][20347] Updated weights for policy 0, policy_version 1263176 (0.0006) [2023-12-27 14:24:01,644][20381] Updated weights for policy 1, policy_version 1262155 (0.0008) [2023-12-27 14:24:01,695][20381] Updated weights for policy 1, policy_version 1262165 (0.0008) [2023-12-27 14:24:02,307][20347] Updated weights for policy 0, policy_version 1263186 (0.0008) [2023-12-27 14:24:02,320][20381] Updated weights for policy 1, policy_version 1262175 (0.0006) [2023-12-27 14:24:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 646578176. Throughput: 0: 10256.5, 1: 10310.7. Samples: 646594620. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:02,363][19170] Avg episode reward: [(0, '716.410'), (1, '815.960')] [2023-12-27 14:24:02,374][20347] Updated weights for policy 0, policy_version 1263196 (0.0008) [2023-12-27 14:24:02,378][20381] Updated weights for policy 1, policy_version 1262185 (0.0009) [2023-12-27 14:24:02,430][20347] Updated weights for policy 0, policy_version 1263206 (0.0006) [2023-12-27 14:24:02,436][20381] Updated weights for policy 1, policy_version 1262195 (0.0006) [2023-12-27 14:24:02,487][20347] Updated weights for policy 0, policy_version 1263216 (0.0005) [2023-12-27 14:24:03,041][20347] Updated weights for policy 0, policy_version 1263226 (0.0008) [2023-12-27 14:24:03,102][20347] Updated weights for policy 0, policy_version 1263236 (0.0008) [2023-12-27 14:24:03,110][20381] Updated weights for policy 1, policy_version 1262205 (0.0006) [2023-12-27 14:24:03,161][20381] Updated weights for policy 1, policy_version 1262215 (0.0005) [2023-12-27 14:24:03,163][20347] Updated weights for policy 0, policy_version 1263246 (0.0007) [2023-12-27 14:24:03,213][20381] Updated weights for policy 1, policy_version 1262225 (0.0005) [2023-12-27 14:24:03,802][20381] Updated weights for policy 1, policy_version 1262235 (0.0005) [2023-12-27 14:24:03,843][20347] Updated weights for policy 0, policy_version 1263256 (0.0007) [2023-12-27 14:24:03,862][20381] Updated weights for policy 1, policy_version 1262245 (0.0007) [2023-12-27 14:24:03,904][20347] Updated weights for policy 0, policy_version 1263266 (0.0007) [2023-12-27 14:24:03,921][20381] Updated weights for policy 1, policy_version 1262255 (0.0006) [2023-12-27 14:24:03,961][20347] Updated weights for policy 0, policy_version 1263276 (0.0009) [2023-12-27 14:24:04,609][20381] Updated weights for policy 1, policy_version 1262265 (0.0008) [2023-12-27 14:24:04,638][20347] Updated weights for policy 0, policy_version 1263286 (0.0007) [2023-12-27 14:24:04,662][20381] Updated weights for policy 1, policy_version 1262275 (0.0005) [2023-12-27 14:24:04,695][20347] Updated weights for policy 0, policy_version 1263296 (0.0006) [2023-12-27 14:24:04,708][20381] Updated weights for policy 1, policy_version 1262285 (0.0007) [2023-12-27 14:24:04,743][20347] Updated weights for policy 0, policy_version 1263306 (0.0006) [2023-12-27 14:24:04,753][20381] Updated weights for policy 1, policy_version 1262295 (0.0005) [2023-12-27 14:24:05,356][20347] Updated weights for policy 0, policy_version 1263316 (0.0007) [2023-12-27 14:24:05,408][20347] Updated weights for policy 0, policy_version 1263326 (0.0006) [2023-12-27 14:24:05,457][20347] Updated weights for policy 0, policy_version 1263336 (0.0006) [2023-12-27 14:24:05,480][20381] Updated weights for policy 1, policy_version 1262305 (0.0006) [2023-12-27 14:24:05,542][20381] Updated weights for policy 1, policy_version 1262315 (0.0005) [2023-12-27 14:24:05,600][20381] Updated weights for policy 1, policy_version 1262325 (0.0007) [2023-12-27 14:24:06,093][20347] Updated weights for policy 0, policy_version 1263346 (0.0005) [2023-12-27 14:24:06,154][20347] Updated weights for policy 0, policy_version 1263356 (0.0008) [2023-12-27 14:24:06,190][20381] Updated weights for policy 1, policy_version 1262335 (0.0006) [2023-12-27 14:24:06,212][20347] Updated weights for policy 0, policy_version 1263366 (0.0007) [2023-12-27 14:24:06,244][20381] Updated weights for policy 1, policy_version 1262345 (0.0008) [2023-12-27 14:24:06,273][20347] Updated weights for policy 0, policy_version 1263376 (0.0007) [2023-12-27 14:24:06,292][20381] Updated weights for policy 1, policy_version 1262355 (0.0007) [2023-12-27 14:24:06,941][20347] Updated weights for policy 0, policy_version 1263386 (0.0007) [2023-12-27 14:24:06,995][20347] Updated weights for policy 0, policy_version 1263396 (0.0007) [2023-12-27 14:24:07,022][20381] Updated weights for policy 1, policy_version 1262365 (0.0008) [2023-12-27 14:24:07,047][20347] Updated weights for policy 0, policy_version 1263406 (0.0006) [2023-12-27 14:24:07,073][20381] Updated weights for policy 1, policy_version 1262375 (0.0006) [2023-12-27 14:24:07,130][20381] Updated weights for policy 1, policy_version 1262385 (0.0008) [2023-12-27 14:24:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 646692864. Throughput: 0: 10268.0, 1: 10309.0. Samples: 646657656. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:07,363][19170] Avg episode reward: [(0, '765.340'), (1, '823.650')] [2023-12-27 14:24:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001262392_323198976.pth... [2023-12-27 14:24:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001263408_323493888.pth... [2023-12-27 14:24:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001262192_323182592.pth [2023-12-27 14:24:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001261208_322895872.pth [2023-12-27 14:24:07,736][20347] Updated weights for policy 0, policy_version 1263416 (0.0007) [2023-12-27 14:24:07,785][20347] Updated weights for policy 0, policy_version 1263426 (0.0008) [2023-12-27 14:24:07,835][20347] Updated weights for policy 0, policy_version 1263436 (0.0007) [2023-12-27 14:24:07,841][20381] Updated weights for policy 1, policy_version 1262395 (0.0008) [2023-12-27 14:24:07,892][20381] Updated weights for policy 1, policy_version 1262405 (0.0007) [2023-12-27 14:24:07,946][20381] Updated weights for policy 1, policy_version 1262415 (0.0008) [2023-12-27 14:24:08,481][20347] Updated weights for policy 0, policy_version 1263446 (0.0007) [2023-12-27 14:24:08,548][20347] Updated weights for policy 0, policy_version 1263456 (0.0007) [2023-12-27 14:24:08,603][20347] Updated weights for policy 0, policy_version 1263466 (0.0008) [2023-12-27 14:24:08,664][20381] Updated weights for policy 1, policy_version 1262425 (0.0008) [2023-12-27 14:24:08,717][20381] Updated weights for policy 1, policy_version 1262435 (0.0008) [2023-12-27 14:24:08,774][20381] Updated weights for policy 1, policy_version 1262445 (0.0008) [2023-12-27 14:24:08,841][20381] Updated weights for policy 1, policy_version 1262455 (0.0008) [2023-12-27 14:24:09,318][20347] Updated weights for policy 0, policy_version 1263476 (0.0007) [2023-12-27 14:24:09,381][20347] Updated weights for policy 0, policy_version 1263486 (0.0007) [2023-12-27 14:24:09,445][20347] Updated weights for policy 0, policy_version 1263496 (0.0008) [2023-12-27 14:24:09,514][20381] Updated weights for policy 1, policy_version 1262465 (0.0007) [2023-12-27 14:24:09,578][20381] Updated weights for policy 1, policy_version 1262475 (0.0008) [2023-12-27 14:24:09,645][20381] Updated weights for policy 1, policy_version 1262485 (0.0008) [2023-12-27 14:24:10,137][20347] Updated weights for policy 0, policy_version 1263506 (0.0007) [2023-12-27 14:24:10,186][20347] Updated weights for policy 0, policy_version 1263516 (0.0008) [2023-12-27 14:24:10,233][20347] Updated weights for policy 0, policy_version 1263526 (0.0008) [2023-12-27 14:24:10,295][20347] Updated weights for policy 0, policy_version 1263536 (0.0007) [2023-12-27 14:24:10,366][20381] Updated weights for policy 1, policy_version 1262495 (0.0008) [2023-12-27 14:24:10,425][20381] Updated weights for policy 1, policy_version 1262505 (0.0008) [2023-12-27 14:24:10,483][20381] Updated weights for policy 1, policy_version 1262515 (0.0008) [2023-12-27 14:24:11,006][20347] Updated weights for policy 0, policy_version 1263546 (0.0005) [2023-12-27 14:24:11,068][20347] Updated weights for policy 0, policy_version 1263556 (0.0007) [2023-12-27 14:24:11,134][20347] Updated weights for policy 0, policy_version 1263566 (0.0006) [2023-12-27 14:24:11,219][20381] Updated weights for policy 1, policy_version 1262525 (0.0008) [2023-12-27 14:24:11,279][20381] Updated weights for policy 1, policy_version 1262535 (0.0007) [2023-12-27 14:24:11,329][20381] Updated weights for policy 1, policy_version 1262545 (0.0006) [2023-12-27 14:24:11,841][20347] Updated weights for policy 0, policy_version 1263576 (0.0008) [2023-12-27 14:24:11,892][20347] Updated weights for policy 0, policy_version 1263586 (0.0008) [2023-12-27 14:24:11,953][20347] Updated weights for policy 0, policy_version 1263596 (0.0008) [2023-12-27 14:24:12,065][20381] Updated weights for policy 1, policy_version 1262555 (0.0008) [2023-12-27 14:24:12,126][20381] Updated weights for policy 1, policy_version 1262565 (0.0007) [2023-12-27 14:24:12,191][20381] Updated weights for policy 1, policy_version 1262575 (0.0008) [2023-12-27 14:24:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 646791168. Throughput: 0: 10236.7, 1: 10223.5. Samples: 646779496. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:12,363][19170] Avg episode reward: [(0, '814.330'), (1, '768.300')] [2023-12-27 14:24:12,709][20347] Updated weights for policy 0, policy_version 1263606 (0.0007) [2023-12-27 14:24:12,768][20347] Updated weights for policy 0, policy_version 1263616 (0.0006) [2023-12-27 14:24:12,821][20347] Updated weights for policy 0, policy_version 1263626 (0.0007) [2023-12-27 14:24:12,865][20381] Updated weights for policy 1, policy_version 1262585 (0.0006) [2023-12-27 14:24:12,919][20381] Updated weights for policy 1, policy_version 1262595 (0.0007) [2023-12-27 14:24:12,978][20381] Updated weights for policy 1, policy_version 1262605 (0.0007) [2023-12-27 14:24:13,034][20381] Updated weights for policy 1, policy_version 1262615 (0.0007) [2023-12-27 14:24:13,434][20347] Updated weights for policy 0, policy_version 1263636 (0.0006) [2023-12-27 14:24:13,489][20347] Updated weights for policy 0, policy_version 1263646 (0.0007) [2023-12-27 14:24:13,543][20347] Updated weights for policy 0, policy_version 1263656 (0.0008) [2023-12-27 14:24:13,695][20381] Updated weights for policy 1, policy_version 1262625 (0.0008) [2023-12-27 14:24:13,746][20381] Updated weights for policy 1, policy_version 1262635 (0.0007) [2023-12-27 14:24:13,793][20381] Updated weights for policy 1, policy_version 1262645 (0.0007) [2023-12-27 14:24:14,235][20347] Updated weights for policy 0, policy_version 1263666 (0.0007) [2023-12-27 14:24:14,287][20347] Updated weights for policy 0, policy_version 1263676 (0.0008) [2023-12-27 14:24:14,346][20347] Updated weights for policy 0, policy_version 1263686 (0.0008) [2023-12-27 14:24:14,402][20347] Updated weights for policy 0, policy_version 1263696 (0.0007) [2023-12-27 14:24:14,482][20381] Updated weights for policy 1, policy_version 1262655 (0.0008) [2023-12-27 14:24:14,534][20381] Updated weights for policy 1, policy_version 1262665 (0.0008) [2023-12-27 14:24:14,587][20381] Updated weights for policy 1, policy_version 1262675 (0.0008) [2023-12-27 14:24:15,083][20347] Updated weights for policy 0, policy_version 1263706 (0.0008) [2023-12-27 14:24:15,139][20347] Updated weights for policy 0, policy_version 1263716 (0.0009) [2023-12-27 14:24:15,199][20347] Updated weights for policy 0, policy_version 1263726 (0.0008) [2023-12-27 14:24:15,293][20381] Updated weights for policy 1, policy_version 1262685 (0.0008) [2023-12-27 14:24:15,358][20381] Updated weights for policy 1, policy_version 1262695 (0.0008) [2023-12-27 14:24:15,419][20381] Updated weights for policy 1, policy_version 1262705 (0.0005) [2023-12-27 14:24:15,924][20347] Updated weights for policy 0, policy_version 1263736 (0.0007) [2023-12-27 14:24:15,981][20347] Updated weights for policy 0, policy_version 1263746 (0.0007) [2023-12-27 14:24:16,038][20347] Updated weights for policy 0, policy_version 1263756 (0.0007) [2023-12-27 14:24:16,082][20381] Updated weights for policy 1, policy_version 1262715 (0.0006) [2023-12-27 14:24:16,130][20381] Updated weights for policy 1, policy_version 1262725 (0.0008) [2023-12-27 14:24:16,184][20381] Updated weights for policy 1, policy_version 1262735 (0.0008) [2023-12-27 14:24:16,645][20347] Updated weights for policy 0, policy_version 1263766 (0.0009) [2023-12-27 14:24:16,694][20347] Updated weights for policy 0, policy_version 1263776 (0.0010) [2023-12-27 14:24:16,743][20347] Updated weights for policy 0, policy_version 1263786 (0.0010) [2023-12-27 14:24:16,877][20381] Updated weights for policy 1, policy_version 1262745 (0.0009) [2023-12-27 14:24:16,928][20381] Updated weights for policy 1, policy_version 1262755 (0.0008) [2023-12-27 14:24:16,980][20381] Updated weights for policy 1, policy_version 1262765 (0.0008) [2023-12-27 14:24:17,062][20381] Updated weights for policy 1, policy_version 1262775 (0.0009) [2023-12-27 14:24:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 646889472. Throughput: 0: 10261.1, 1: 10188.0. Samples: 646900424. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:17,363][19170] Avg episode reward: [(0, '774.990'), (1, '797.640')] [2023-12-27 14:24:17,387][20347] Updated weights for policy 0, policy_version 1263796 (0.0010) [2023-12-27 14:24:17,437][20347] Updated weights for policy 0, policy_version 1263806 (0.0010) [2023-12-27 14:24:17,494][20347] Updated weights for policy 0, policy_version 1263816 (0.0010) [2023-12-27 14:24:17,657][20381] Updated weights for policy 1, policy_version 1262785 (0.0007) [2023-12-27 14:24:17,715][20381] Updated weights for policy 1, policy_version 1262795 (0.0007) [2023-12-27 14:24:17,766][20381] Updated weights for policy 1, policy_version 1262805 (0.0008) [2023-12-27 14:24:18,113][20347] Updated weights for policy 0, policy_version 1263826 (0.0008) [2023-12-27 14:24:18,167][20347] Updated weights for policy 0, policy_version 1263836 (0.0006) [2023-12-27 14:24:18,222][20347] Updated weights for policy 0, policy_version 1263846 (0.0010) [2023-12-27 14:24:18,276][20347] Updated weights for policy 0, policy_version 1263856 (0.0010) [2023-12-27 14:24:18,448][20381] Updated weights for policy 1, policy_version 1262815 (0.0006) [2023-12-27 14:24:18,509][20381] Updated weights for policy 1, policy_version 1262825 (0.0005) [2023-12-27 14:24:18,574][20381] Updated weights for policy 1, policy_version 1262835 (0.0007) [2023-12-27 14:24:18,938][20347] Updated weights for policy 0, policy_version 1263866 (0.0009) [2023-12-27 14:24:18,987][20347] Updated weights for policy 0, policy_version 1263876 (0.0008) [2023-12-27 14:24:19,038][20347] Updated weights for policy 0, policy_version 1263886 (0.0005) [2023-12-27 14:24:19,179][20381] Updated weights for policy 1, policy_version 1262845 (0.0009) [2023-12-27 14:24:19,229][20381] Updated weights for policy 1, policy_version 1262855 (0.0009) [2023-12-27 14:24:19,291][20381] Updated weights for policy 1, policy_version 1262865 (0.0008) [2023-12-27 14:24:19,716][20347] Updated weights for policy 0, policy_version 1263896 (0.0007) [2023-12-27 14:24:19,772][20347] Updated weights for policy 0, policy_version 1263906 (0.0008) [2023-12-27 14:24:19,831][20347] Updated weights for policy 0, policy_version 1263916 (0.0008) [2023-12-27 14:24:20,014][20381] Updated weights for policy 1, policy_version 1262875 (0.0008) [2023-12-27 14:24:20,074][20381] Updated weights for policy 1, policy_version 1262885 (0.0011) [2023-12-27 14:24:20,134][20381] Updated weights for policy 1, policy_version 1262895 (0.0011) [2023-12-27 14:24:20,584][20347] Updated weights for policy 0, policy_version 1263926 (0.0008) [2023-12-27 14:24:20,646][20347] Updated weights for policy 0, policy_version 1263936 (0.0008) [2023-12-27 14:24:20,702][20347] Updated weights for policy 0, policy_version 1263946 (0.0008) [2023-12-27 14:24:20,828][20381] Updated weights for policy 1, policy_version 1262905 (0.0010) [2023-12-27 14:24:20,877][20381] Updated weights for policy 1, policy_version 1262915 (0.0008) [2023-12-27 14:24:20,932][20381] Updated weights for policy 1, policy_version 1262925 (0.0008) [2023-12-27 14:24:20,988][20381] Updated weights for policy 1, policy_version 1262935 (0.0007) [2023-12-27 14:24:21,462][20347] Updated weights for policy 0, policy_version 1263956 (0.0008) [2023-12-27 14:24:21,518][20347] Updated weights for policy 0, policy_version 1263966 (0.0007) [2023-12-27 14:24:21,578][20347] Updated weights for policy 0, policy_version 1263976 (0.0006) [2023-12-27 14:24:21,787][20381] Updated weights for policy 1, policy_version 1262945 (0.0007) [2023-12-27 14:24:21,843][20381] Updated weights for policy 1, policy_version 1262955 (0.0008) [2023-12-27 14:24:21,903][20381] Updated weights for policy 1, policy_version 1262965 (0.0008) [2023-12-27 14:24:22,237][20347] Updated weights for policy 0, policy_version 1263986 (0.0008) [2023-12-27 14:24:22,296][20347] Updated weights for policy 0, policy_version 1263996 (0.0007) [2023-12-27 14:24:22,361][20347] Updated weights for policy 0, policy_version 1264006 (0.0008) [2023-12-27 14:24:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 646987776. Throughput: 0: 10263.3, 1: 10185.0. Samples: 646963464. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:22,363][19170] Avg episode reward: [(0, '726.200'), (1, '786.550')] [2023-12-27 14:24:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001262968_323346432.pth... [2023-12-27 14:24:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001261784_323043328.pth [2023-12-27 14:24:22,423][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001264016_323649536.pth... [2023-12-27 14:24:22,424][20347] Updated weights for policy 0, policy_version 1264016 (0.0007) [2023-12-27 14:24:22,426][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001262800_323338240.pth [2023-12-27 14:24:22,628][20381] Updated weights for policy 1, policy_version 1262975 (0.0010) [2023-12-27 14:24:22,689][20381] Updated weights for policy 1, policy_version 1262985 (0.0011) [2023-12-27 14:24:22,752][20381] Updated weights for policy 1, policy_version 1262995 (0.0011) [2023-12-27 14:24:23,173][20347] Updated weights for policy 0, policy_version 1264026 (0.0006) [2023-12-27 14:24:23,222][20347] Updated weights for policy 0, policy_version 1264036 (0.0005) [2023-12-27 14:24:23,271][20347] Updated weights for policy 0, policy_version 1264046 (0.0005) [2023-12-27 14:24:23,428][20381] Updated weights for policy 1, policy_version 1263005 (0.0009) [2023-12-27 14:24:23,477][20381] Updated weights for policy 1, policy_version 1263015 (0.0008) [2023-12-27 14:24:23,526][20381] Updated weights for policy 1, policy_version 1263025 (0.0008) [2023-12-27 14:24:23,968][20347] Updated weights for policy 0, policy_version 1264056 (0.0008) [2023-12-27 14:24:24,023][20347] Updated weights for policy 0, policy_version 1264066 (0.0007) [2023-12-27 14:24:24,090][20347] Updated weights for policy 0, policy_version 1264076 (0.0008) [2023-12-27 14:24:24,110][20381] Updated weights for policy 1, policy_version 1263035 (0.0007) [2023-12-27 14:24:24,160][20381] Updated weights for policy 1, policy_version 1263045 (0.0006) [2023-12-27 14:24:24,208][20381] Updated weights for policy 1, policy_version 1263055 (0.0008) [2023-12-27 14:24:24,689][20347] Updated weights for policy 0, policy_version 1264086 (0.0006) [2023-12-27 14:24:24,752][20347] Updated weights for policy 0, policy_version 1264096 (0.0006) [2023-12-27 14:24:24,806][20347] Updated weights for policy 0, policy_version 1264106 (0.0005) [2023-12-27 14:24:24,879][20381] Updated weights for policy 1, policy_version 1263065 (0.0007) [2023-12-27 14:24:24,936][20381] Updated weights for policy 1, policy_version 1263075 (0.0007) [2023-12-27 14:24:24,990][20381] Updated weights for policy 1, policy_version 1263085 (0.0008) [2023-12-27 14:24:25,044][20381] Updated weights for policy 1, policy_version 1263095 (0.0007) [2023-12-27 14:24:25,437][20347] Updated weights for policy 0, policy_version 1264116 (0.0007) [2023-12-27 14:24:25,495][20347] Updated weights for policy 0, policy_version 1264126 (0.0007) [2023-12-27 14:24:25,542][20347] Updated weights for policy 0, policy_version 1264136 (0.0008) [2023-12-27 14:24:25,724][20381] Updated weights for policy 1, policy_version 1263105 (0.0006) [2023-12-27 14:24:25,781][20381] Updated weights for policy 1, policy_version 1263115 (0.0005) [2023-12-27 14:24:25,832][20381] Updated weights for policy 1, policy_version 1263125 (0.0005) [2023-12-27 14:24:26,281][20347] Updated weights for policy 0, policy_version 1264146 (0.0008) [2023-12-27 14:24:26,340][20347] Updated weights for policy 0, policy_version 1264156 (0.0007) [2023-12-27 14:24:26,388][20381] Updated weights for policy 1, policy_version 1263135 (0.0008) [2023-12-27 14:24:26,396][20347] Updated weights for policy 0, policy_version 1264166 (0.0009) [2023-12-27 14:24:26,444][20381] Updated weights for policy 1, policy_version 1263145 (0.0006) [2023-12-27 14:24:26,453][20347] Updated weights for policy 0, policy_version 1264176 (0.0006) [2023-12-27 14:24:26,495][20381] Updated weights for policy 1, policy_version 1263155 (0.0006) [2023-12-27 14:24:27,048][20291] KL-divergence is very high: 164.0318 [2023-12-27 14:24:27,052][20347] Updated weights for policy 0, policy_version 1264186 (0.0007) [2023-12-27 14:24:27,091][20291] KL-divergence is very high: 106.4314 [2023-12-27 14:24:27,107][20347] Updated weights for policy 0, policy_version 1264196 (0.0008) [2023-12-27 14:24:27,164][20347] Updated weights for policy 0, policy_version 1264206 (0.0007) [2023-12-27 14:24:27,181][20381] Updated weights for policy 1, policy_version 1263165 (0.0006) [2023-12-27 14:24:27,226][20381] Updated weights for policy 1, policy_version 1263175 (0.0008) [2023-12-27 14:24:27,269][20381] Updated weights for policy 1, policy_version 1263185 (0.0007) [2023-12-27 14:24:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 647102464. Throughput: 0: 10310.2, 1: 10188.5. Samples: 647085916. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:27,363][19170] Avg episode reward: [(0, '687.780'), (1, '757.690')] [2023-12-27 14:24:27,861][20347] Updated weights for policy 0, policy_version 1264216 (0.0007) [2023-12-27 14:24:27,910][20347] Updated weights for policy 0, policy_version 1264226 (0.0006) [2023-12-27 14:24:27,956][20381] Updated weights for policy 1, policy_version 1263195 (0.0007) [2023-12-27 14:24:27,967][20347] Updated weights for policy 0, policy_version 1264236 (0.0007) [2023-12-27 14:24:28,012][20381] Updated weights for policy 1, policy_version 1263205 (0.0005) [2023-12-27 14:24:28,058][20381] Updated weights for policy 1, policy_version 1263215 (0.0005) [2023-12-27 14:24:28,674][20347] Updated weights for policy 0, policy_version 1264246 (0.0009) [2023-12-27 14:24:28,689][20381] Updated weights for policy 1, policy_version 1263225 (0.0006) [2023-12-27 14:24:28,742][20347] Updated weights for policy 0, policy_version 1264256 (0.0010) [2023-12-27 14:24:28,757][20381] Updated weights for policy 1, policy_version 1263235 (0.0008) [2023-12-27 14:24:28,796][20347] Updated weights for policy 0, policy_version 1264266 (0.0009) [2023-12-27 14:24:28,806][20381] Updated weights for policy 1, policy_version 1263245 (0.0006) [2023-12-27 14:24:28,862][20381] Updated weights for policy 1, policy_version 1263255 (0.0007) [2023-12-27 14:24:29,480][20347] Updated weights for policy 0, policy_version 1264276 (0.0010) [2023-12-27 14:24:29,543][20347] Updated weights for policy 0, policy_version 1264286 (0.0008) [2023-12-27 14:24:29,598][20381] Updated weights for policy 1, policy_version 1263265 (0.0007) [2023-12-27 14:24:29,606][20347] Updated weights for policy 0, policy_version 1264296 (0.0008) [2023-12-27 14:24:29,650][20381] Updated weights for policy 1, policy_version 1263275 (0.0007) [2023-12-27 14:24:29,709][20381] Updated weights for policy 1, policy_version 1263285 (0.0006) [2023-12-27 14:24:30,251][20347] Updated weights for policy 0, policy_version 1264306 (0.0007) [2023-12-27 14:24:30,307][20347] Updated weights for policy 0, policy_version 1264316 (0.0006) [2023-12-27 14:24:30,358][20381] Updated weights for policy 1, policy_version 1263295 (0.0007) [2023-12-27 14:24:30,366][20347] Updated weights for policy 0, policy_version 1264326 (0.0006) [2023-12-27 14:24:30,415][20381] Updated weights for policy 1, policy_version 1263305 (0.0008) [2023-12-27 14:24:30,430][20347] Updated weights for policy 0, policy_version 1264336 (0.0006) [2023-12-27 14:24:30,475][20381] Updated weights for policy 1, policy_version 1263315 (0.0008) [2023-12-27 14:24:31,103][20347] Updated weights for policy 0, policy_version 1264346 (0.0007) [2023-12-27 14:24:31,168][20347] Updated weights for policy 0, policy_version 1264356 (0.0008) [2023-12-27 14:24:31,197][20381] Updated weights for policy 1, policy_version 1263325 (0.0008) [2023-12-27 14:24:31,231][20347] Updated weights for policy 0, policy_version 1264366 (0.0006) [2023-12-27 14:24:31,250][20381] Updated weights for policy 1, policy_version 1263335 (0.0008) [2023-12-27 14:24:31,314][20381] Updated weights for policy 1, policy_version 1263345 (0.0008) [2023-12-27 14:24:31,917][20347] Updated weights for policy 0, policy_version 1264376 (0.0008) [2023-12-27 14:24:31,972][20347] Updated weights for policy 0, policy_version 1264386 (0.0007) [2023-12-27 14:24:32,029][20347] Updated weights for policy 0, policy_version 1264396 (0.0007) [2023-12-27 14:24:32,032][20381] Updated weights for policy 1, policy_version 1263355 (0.0008) [2023-12-27 14:24:32,091][20381] Updated weights for policy 1, policy_version 1263365 (0.0006) [2023-12-27 14:24:32,146][20381] Updated weights for policy 1, policy_version 1263375 (0.0006) [2023-12-27 14:24:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 647200768. Throughput: 0: 10297.0, 1: 10164.9. Samples: 647207840. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:32,363][19170] Avg episode reward: [(0, '696.520'), (1, '795.400')] [2023-12-27 14:24:32,744][20347] Updated weights for policy 0, policy_version 1264406 (0.0007) [2023-12-27 14:24:32,773][20381] Updated weights for policy 1, policy_version 1263385 (0.0006) [2023-12-27 14:24:32,803][20347] Updated weights for policy 0, policy_version 1264416 (0.0008) [2023-12-27 14:24:32,830][20381] Updated weights for policy 1, policy_version 1263395 (0.0005) [2023-12-27 14:24:32,863][20347] Updated weights for policy 0, policy_version 1264426 (0.0008) [2023-12-27 14:24:32,882][20381] Updated weights for policy 1, policy_version 1263405 (0.0005) [2023-12-27 14:24:32,933][20381] Updated weights for policy 1, policy_version 1263415 (0.0005) [2023-12-27 14:24:33,493][20347] Updated weights for policy 0, policy_version 1264436 (0.0007) [2023-12-27 14:24:33,498][20381] Updated weights for policy 1, policy_version 1263425 (0.0005) [2023-12-27 14:24:33,543][20381] Updated weights for policy 1, policy_version 1263435 (0.0005) [2023-12-27 14:24:33,545][20347] Updated weights for policy 0, policy_version 1264446 (0.0007) [2023-12-27 14:24:33,594][20381] Updated weights for policy 1, policy_version 1263445 (0.0005) [2023-12-27 14:24:33,596][20347] Updated weights for policy 0, policy_version 1264456 (0.0007) [2023-12-27 14:24:34,186][20381] Updated weights for policy 1, policy_version 1263455 (0.0007) [2023-12-27 14:24:34,209][20347] Updated weights for policy 0, policy_version 1264466 (0.0007) [2023-12-27 14:24:34,245][20381] Updated weights for policy 1, policy_version 1263465 (0.0005) [2023-12-27 14:24:34,258][20347] Updated weights for policy 0, policy_version 1264476 (0.0006) [2023-12-27 14:24:34,260][20292] KL-divergence is very high: 141.7858 [2023-12-27 14:24:34,310][20381] Updated weights for policy 1, policy_version 1263475 (0.0005) [2023-12-27 14:24:34,317][20347] Updated weights for policy 0, policy_version 1264486 (0.0007) [2023-12-27 14:24:34,375][20347] Updated weights for policy 0, policy_version 1264496 (0.0007) [2023-12-27 14:24:34,971][20381] Updated weights for policy 1, policy_version 1263485 (0.0008) [2023-12-27 14:24:35,033][20381] Updated weights for policy 1, policy_version 1263495 (0.0005) [2023-12-27 14:24:35,057][20347] Updated weights for policy 0, policy_version 1264506 (0.0007) [2023-12-27 14:24:35,081][20381] Updated weights for policy 1, policy_version 1263505 (0.0006) [2023-12-27 14:24:35,109][20347] Updated weights for policy 0, policy_version 1264516 (0.0007) [2023-12-27 14:24:35,164][20347] Updated weights for policy 0, policy_version 1264526 (0.0008) [2023-12-27 14:24:35,722][20381] Updated weights for policy 1, policy_version 1263515 (0.0006) [2023-12-27 14:24:35,783][20381] Updated weights for policy 1, policy_version 1263525 (0.0007) [2023-12-27 14:24:35,838][20381] Updated weights for policy 1, policy_version 1263535 (0.0008) [2023-12-27 14:24:35,862][20347] Updated weights for policy 0, policy_version 1264536 (0.0006) [2023-12-27 14:24:35,916][20347] Updated weights for policy 0, policy_version 1264546 (0.0007) [2023-12-27 14:24:35,976][20347] Updated weights for policy 0, policy_version 1264556 (0.0008) [2023-12-27 14:24:36,416][20381] Updated weights for policy 1, policy_version 1263545 (0.0006) [2023-12-27 14:24:36,478][20381] Updated weights for policy 1, policy_version 1263555 (0.0008) [2023-12-27 14:24:36,541][20381] Updated weights for policy 1, policy_version 1263565 (0.0008) [2023-12-27 14:24:36,609][20381] Updated weights for policy 1, policy_version 1263575 (0.0008) [2023-12-27 14:24:36,660][20347] Updated weights for policy 0, policy_version 1264566 (0.0006) [2023-12-27 14:24:36,714][20347] Updated weights for policy 0, policy_version 1264576 (0.0007) [2023-12-27 14:24:36,769][20347] Updated weights for policy 0, policy_version 1264586 (0.0007) [2023-12-27 14:24:37,326][20381] Updated weights for policy 1, policy_version 1263585 (0.0008) [2023-12-27 14:24:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 647299072. Throughput: 0: 10291.7, 1: 10264.4. Samples: 647273668. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:37,363][19170] Avg episode reward: [(0, '651.110'), (1, '720.170')] [2023-12-27 14:24:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001264592_323796992.pth... [2023-12-27 14:24:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001263408_323493888.pth [2023-12-27 14:24:37,385][20381] Updated weights for policy 1, policy_version 1263595 (0.0008) [2023-12-27 14:24:37,418][20347] Updated weights for policy 0, policy_version 1264596 (0.0006) [2023-12-27 14:24:37,436][20381] Updated weights for policy 1, policy_version 1263605 (0.0006) [2023-12-27 14:24:37,455][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001263608_323510272.pth... [2023-12-27 14:24:37,459][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001262392_323198976.pth [2023-12-27 14:24:37,472][20347] Updated weights for policy 0, policy_version 1264606 (0.0006) [2023-12-27 14:24:37,528][20347] Updated weights for policy 0, policy_version 1264616 (0.0008) [2023-12-27 14:24:38,132][20381] Updated weights for policy 1, policy_version 1263615 (0.0007) [2023-12-27 14:24:38,189][20381] Updated weights for policy 1, policy_version 1263625 (0.0007) [2023-12-27 14:24:38,224][20347] Updated weights for policy 0, policy_version 1264626 (0.0007) [2023-12-27 14:24:38,248][20381] Updated weights for policy 1, policy_version 1263635 (0.0008) [2023-12-27 14:24:38,283][20347] Updated weights for policy 0, policy_version 1264636 (0.0006) [2023-12-27 14:24:38,361][20347] Updated weights for policy 0, policy_version 1264647 (0.0024) [2023-12-27 14:24:38,961][20381] Updated weights for policy 1, policy_version 1263645 (0.0008) [2023-12-27 14:24:39,010][20381] Updated weights for policy 1, policy_version 1263655 (0.0011) [2023-12-27 14:24:39,028][20347] Updated weights for policy 0, policy_version 1264657 (0.0008) [2023-12-27 14:24:39,064][20381] Updated weights for policy 1, policy_version 1263665 (0.0011) [2023-12-27 14:24:39,077][20291] KL-divergence is very high: 214.6938 [2023-12-27 14:24:39,090][20347] Updated weights for policy 0, policy_version 1264667 (0.0011) [2023-12-27 14:24:39,122][20291] KL-divergence is very high: 103.2380 [2023-12-27 14:24:39,145][20347] Updated weights for policy 0, policy_version 1264677 (0.0010) [2023-12-27 14:24:39,203][20347] Updated weights for policy 0, policy_version 1264687 (0.0010) [2023-12-27 14:24:39,762][20381] Updated weights for policy 1, policy_version 1263675 (0.0010) [2023-12-27 14:24:39,824][20381] Updated weights for policy 1, policy_version 1263685 (0.0009) [2023-12-27 14:24:39,887][20381] Updated weights for policy 1, policy_version 1263695 (0.0010) [2023-12-27 14:24:39,962][20347] Updated weights for policy 0, policy_version 1264697 (0.0011) [2023-12-27 14:24:40,019][20347] Updated weights for policy 0, policy_version 1264707 (0.0011) [2023-12-27 14:24:40,078][20347] Updated weights for policy 0, policy_version 1264717 (0.0010) [2023-12-27 14:24:40,525][20381] Updated weights for policy 1, policy_version 1263705 (0.0010) [2023-12-27 14:24:40,585][20381] Updated weights for policy 1, policy_version 1263715 (0.0005) [2023-12-27 14:24:40,644][20381] Updated weights for policy 1, policy_version 1263725 (0.0011) [2023-12-27 14:24:40,704][20381] Updated weights for policy 1, policy_version 1263735 (0.0011) [2023-12-27 14:24:40,753][20347] Updated weights for policy 0, policy_version 1264727 (0.0008) [2023-12-27 14:24:40,816][20347] Updated weights for policy 0, policy_version 1264737 (0.0008) [2023-12-27 14:24:40,873][20347] Updated weights for policy 0, policy_version 1264747 (0.0007) [2023-12-27 14:24:41,402][20381] Updated weights for policy 1, policy_version 1263745 (0.0010) [2023-12-27 14:24:41,460][20381] Updated weights for policy 1, policy_version 1263755 (0.0009) [2023-12-27 14:24:41,511][20381] Updated weights for policy 1, policy_version 1263765 (0.0008) [2023-12-27 14:24:41,584][20347] Updated weights for policy 0, policy_version 1264757 (0.0008) [2023-12-27 14:24:41,649][20347] Updated weights for policy 0, policy_version 1264767 (0.0007) [2023-12-27 14:24:41,708][20347] Updated weights for policy 0, policy_version 1264777 (0.0010) [2023-12-27 14:24:42,286][20381] Updated weights for policy 1, policy_version 1263775 (0.0008) [2023-12-27 14:24:42,345][20347] Updated weights for policy 0, policy_version 1264787 (0.0011) [2023-12-27 14:24:42,351][20381] Updated weights for policy 1, policy_version 1263785 (0.0008) [2023-12-27 14:24:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 647397376. Throughput: 0: 10224.9, 1: 10240.1. Samples: 647394844. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:42,363][19170] Avg episode reward: [(0, '659.440'), (1, '728.600')] [2023-12-27 14:24:42,406][20347] Updated weights for policy 0, policy_version 1264797 (0.0010) [2023-12-27 14:24:42,416][20381] Updated weights for policy 1, policy_version 1263795 (0.0006) [2023-12-27 14:24:42,460][20347] Updated weights for policy 0, policy_version 1264807 (0.0009) [2023-12-27 14:24:43,093][20381] Updated weights for policy 1, policy_version 1263805 (0.0007) [2023-12-27 14:24:43,112][20347] Updated weights for policy 0, policy_version 1264817 (0.0006) [2023-12-27 14:24:43,139][20381] Updated weights for policy 1, policy_version 1263815 (0.0007) [2023-12-27 14:24:43,166][20347] Updated weights for policy 0, policy_version 1264827 (0.0010) [2023-12-27 14:24:43,195][20381] Updated weights for policy 1, policy_version 1263825 (0.0006) [2023-12-27 14:24:43,218][20347] Updated weights for policy 0, policy_version 1264837 (0.0010) [2023-12-27 14:24:43,270][20347] Updated weights for policy 0, policy_version 1264847 (0.0010) [2023-12-27 14:24:43,876][20381] Updated weights for policy 1, policy_version 1263835 (0.0007) [2023-12-27 14:24:43,928][20381] Updated weights for policy 1, policy_version 1263845 (0.0011) [2023-12-27 14:24:43,930][20347] Updated weights for policy 0, policy_version 1264857 (0.0009) [2023-12-27 14:24:43,982][20347] Updated weights for policy 0, policy_version 1264867 (0.0005) [2023-12-27 14:24:43,991][20381] Updated weights for policy 1, policy_version 1263855 (0.0011) [2023-12-27 14:24:44,039][20347] Updated weights for policy 0, policy_version 1264877 (0.0006) [2023-12-27 14:24:44,602][20347] Updated weights for policy 0, policy_version 1264887 (0.0005) [2023-12-27 14:24:44,641][20381] Updated weights for policy 1, policy_version 1263865 (0.0010) [2023-12-27 14:24:44,661][20347] Updated weights for policy 0, policy_version 1264897 (0.0006) [2023-12-27 14:24:44,700][20381] Updated weights for policy 1, policy_version 1263875 (0.0008) [2023-12-27 14:24:44,719][20347] Updated weights for policy 0, policy_version 1264907 (0.0010) [2023-12-27 14:24:44,754][20381] Updated weights for policy 1, policy_version 1263885 (0.0008) [2023-12-27 14:24:44,819][20381] Updated weights for policy 1, policy_version 1263895 (0.0011) [2023-12-27 14:24:45,340][20347] Updated weights for policy 0, policy_version 1264917 (0.0009) [2023-12-27 14:24:45,398][20347] Updated weights for policy 0, policy_version 1264927 (0.0006) [2023-12-27 14:24:45,466][20347] Updated weights for policy 0, policy_version 1264937 (0.0006) [2023-12-27 14:24:45,500][20381] Updated weights for policy 1, policy_version 1263905 (0.0010) [2023-12-27 14:24:45,561][20381] Updated weights for policy 1, policy_version 1263915 (0.0005) [2023-12-27 14:24:45,621][20381] Updated weights for policy 1, policy_version 1263925 (0.0005) [2023-12-27 14:24:46,023][20347] Updated weights for policy 0, policy_version 1264947 (0.0007) [2023-12-27 14:24:46,080][20347] Updated weights for policy 0, policy_version 1264957 (0.0010) [2023-12-27 14:24:46,135][20347] Updated weights for policy 0, policy_version 1264967 (0.0010) [2023-12-27 14:24:46,195][20381] Updated weights for policy 1, policy_version 1263935 (0.0010) [2023-12-27 14:24:46,241][20381] Updated weights for policy 1, policy_version 1263945 (0.0010) [2023-12-27 14:24:46,287][20381] Updated weights for policy 1, policy_version 1263955 (0.0010) [2023-12-27 14:24:46,682][20347] Updated weights for policy 0, policy_version 1264977 (0.0007) [2023-12-27 14:24:46,734][20347] Updated weights for policy 0, policy_version 1264987 (0.0008) [2023-12-27 14:24:46,803][20347] Updated weights for policy 0, policy_version 1264997 (0.0010) [2023-12-27 14:24:46,864][20347] Updated weights for policy 0, policy_version 1265007 (0.0010) [2023-12-27 14:24:47,022][20381] Updated weights for policy 1, policy_version 1263965 (0.0009) [2023-12-27 14:24:47,072][20381] Updated weights for policy 1, policy_version 1263975 (0.0008) [2023-12-27 14:24:47,125][20381] Updated weights for policy 1, policy_version 1263985 (0.0008) [2023-12-27 14:24:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 647512064. Throughput: 0: 10342.9, 1: 10245.8. Samples: 647521112. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:47,363][19170] Avg episode reward: [(0, '659.320'), (1, '659.160')] [2023-12-27 14:24:47,534][20347] Updated weights for policy 0, policy_version 1265017 (0.0008) [2023-12-27 14:24:47,586][20347] Updated weights for policy 0, policy_version 1265027 (0.0008) [2023-12-27 14:24:47,638][20347] Updated weights for policy 0, policy_version 1265037 (0.0007) [2023-12-27 14:24:47,780][20381] Updated weights for policy 1, policy_version 1263995 (0.0008) [2023-12-27 14:24:47,838][20381] Updated weights for policy 1, policy_version 1264005 (0.0010) [2023-12-27 14:24:47,887][20381] Updated weights for policy 1, policy_version 1264015 (0.0010) [2023-12-27 14:24:48,233][20347] Updated weights for policy 0, policy_version 1265047 (0.0007) [2023-12-27 14:24:48,279][20347] Updated weights for policy 0, policy_version 1265057 (0.0007) [2023-12-27 14:24:48,332][20347] Updated weights for policy 0, policy_version 1265067 (0.0009) [2023-12-27 14:24:48,577][20381] Updated weights for policy 1, policy_version 1264025 (0.0010) [2023-12-27 14:24:48,637][20381] Updated weights for policy 1, policy_version 1264035 (0.0008) [2023-12-27 14:24:48,696][20381] Updated weights for policy 1, policy_version 1264045 (0.0007) [2023-12-27 14:24:48,758][20381] Updated weights for policy 1, policy_version 1264055 (0.0007) [2023-12-27 14:24:49,040][20347] Updated weights for policy 0, policy_version 1265077 (0.0007) [2023-12-27 14:24:49,094][20347] Updated weights for policy 0, policy_version 1265087 (0.0005) [2023-12-27 14:24:49,147][20347] Updated weights for policy 0, policy_version 1265097 (0.0006) [2023-12-27 14:24:49,492][20381] Updated weights for policy 1, policy_version 1264065 (0.0009) [2023-12-27 14:24:49,562][20381] Updated weights for policy 1, policy_version 1264075 (0.0007) [2023-12-27 14:24:49,627][20381] Updated weights for policy 1, policy_version 1264085 (0.0009) [2023-12-27 14:24:49,777][20347] Updated weights for policy 0, policy_version 1265107 (0.0008) [2023-12-27 14:24:49,844][20347] Updated weights for policy 0, policy_version 1265117 (0.0007) [2023-12-27 14:24:49,898][20347] Updated weights for policy 0, policy_version 1265127 (0.0007) [2023-12-27 14:24:50,309][20381] Updated weights for policy 1, policy_version 1264095 (0.0007) [2023-12-27 14:24:50,374][20381] Updated weights for policy 1, policy_version 1264105 (0.0009) [2023-12-27 14:24:50,430][20381] Updated weights for policy 1, policy_version 1264115 (0.0011) [2023-12-27 14:24:50,594][20347] Updated weights for policy 0, policy_version 1265137 (0.0007) [2023-12-27 14:24:50,652][20347] Updated weights for policy 0, policy_version 1265147 (0.0006) [2023-12-27 14:24:50,700][20347] Updated weights for policy 0, policy_version 1265157 (0.0005) [2023-12-27 14:24:50,752][20347] Updated weights for policy 0, policy_version 1265167 (0.0005) [2023-12-27 14:24:51,029][20381] Updated weights for policy 1, policy_version 1264125 (0.0008) [2023-12-27 14:24:51,091][20381] Updated weights for policy 1, policy_version 1264135 (0.0013) [2023-12-27 14:24:51,150][20381] Updated weights for policy 1, policy_version 1264145 (0.0011) [2023-12-27 14:24:51,395][20347] Updated weights for policy 0, policy_version 1265177 (0.0008) [2023-12-27 14:24:51,455][20347] Updated weights for policy 0, policy_version 1265187 (0.0008) [2023-12-27 14:24:51,513][20347] Updated weights for policy 0, policy_version 1265197 (0.0007) [2023-12-27 14:24:51,874][20381] Updated weights for policy 1, policy_version 1264155 (0.0009) [2023-12-27 14:24:51,935][20381] Updated weights for policy 1, policy_version 1264165 (0.0007) [2023-12-27 14:24:51,996][20381] Updated weights for policy 1, policy_version 1264175 (0.0007) [2023-12-27 14:24:52,190][20347] Updated weights for policy 0, policy_version 1265207 (0.0006) [2023-12-27 14:24:52,200][20291] KL-divergence is very high: 103.7752 [2023-12-27 14:24:52,242][20347] Updated weights for policy 0, policy_version 1265217 (0.0005) [2023-12-27 14:24:52,243][20291] KL-divergence is very high: 281.1476 [2023-12-27 14:24:52,255][20291] KL-divergence is very high: 436.7643 [2023-12-27 14:24:52,308][20347] Updated weights for policy 0, policy_version 1265227 (0.0006) [2023-12-27 14:24:52,308][20291] KL-divergence is very high: 363.2045 [2023-12-27 14:24:52,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 647618560. Throughput: 0: 10325.6, 1: 10231.6. Samples: 647582732. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:52,363][19170] Avg episode reward: [(0, '650.540'), (1, '556.290')] [2023-12-27 14:24:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001264184_323657728.pth... [2023-12-27 14:24:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001265232_323960832.pth... [2023-12-27 14:24:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001262968_323346432.pth [2023-12-27 14:24:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001264016_323649536.pth [2023-12-27 14:24:52,709][20381] Updated weights for policy 1, policy_version 1264185 (0.0010) [2023-12-27 14:24:52,778][20381] Updated weights for policy 1, policy_version 1264195 (0.0007) [2023-12-27 14:24:52,843][20381] Updated weights for policy 1, policy_version 1264205 (0.0007) [2023-12-27 14:24:52,907][20381] Updated weights for policy 1, policy_version 1264215 (0.0006) [2023-12-27 14:24:53,006][20347] Updated weights for policy 0, policy_version 1265237 (0.0006) [2023-12-27 14:24:53,061][20347] Updated weights for policy 0, policy_version 1265247 (0.0005) [2023-12-27 14:24:53,122][20347] Updated weights for policy 0, policy_version 1265257 (0.0006) [2023-12-27 14:24:53,530][20381] Updated weights for policy 1, policy_version 1264225 (0.0010) [2023-12-27 14:24:53,587][20381] Updated weights for policy 1, policy_version 1264235 (0.0010) [2023-12-27 14:24:53,642][20381] Updated weights for policy 1, policy_version 1264245 (0.0011) [2023-12-27 14:24:53,680][20347] Updated weights for policy 0, policy_version 1265267 (0.0007) [2023-12-27 14:24:53,745][20347] Updated weights for policy 0, policy_version 1265277 (0.0010) [2023-12-27 14:24:53,810][20347] Updated weights for policy 0, policy_version 1265287 (0.0010) [2023-12-27 14:24:54,275][20381] Updated weights for policy 1, policy_version 1264255 (0.0007) [2023-12-27 14:24:54,334][20381] Updated weights for policy 1, policy_version 1264265 (0.0006) [2023-12-27 14:24:54,396][20381] Updated weights for policy 1, policy_version 1264275 (0.0007) [2023-12-27 14:24:54,425][20347] Updated weights for policy 0, policy_version 1265297 (0.0007) [2023-12-27 14:24:54,479][20347] Updated weights for policy 0, policy_version 1265307 (0.0008) [2023-12-27 14:24:54,542][20347] Updated weights for policy 0, policy_version 1265317 (0.0008) [2023-12-27 14:24:54,605][20347] Updated weights for policy 0, policy_version 1265327 (0.0008) [2023-12-27 14:24:55,042][20381] Updated weights for policy 1, policy_version 1264285 (0.0007) [2023-12-27 14:24:55,101][20381] Updated weights for policy 1, policy_version 1264295 (0.0008) [2023-12-27 14:24:55,155][20381] Updated weights for policy 1, policy_version 1264305 (0.0008) [2023-12-27 14:24:55,224][20347] Updated weights for policy 0, policy_version 1265337 (0.0007) [2023-12-27 14:24:55,275][20347] Updated weights for policy 0, policy_version 1265347 (0.0008) [2023-12-27 14:24:55,332][20347] Updated weights for policy 0, policy_version 1265357 (0.0007) [2023-12-27 14:24:55,752][20381] Updated weights for policy 1, policy_version 1264315 (0.0008) [2023-12-27 14:24:55,800][20381] Updated weights for policy 1, policy_version 1264325 (0.0008) [2023-12-27 14:24:55,850][20381] Updated weights for policy 1, policy_version 1264335 (0.0007) [2023-12-27 14:24:56,020][20347] Updated weights for policy 0, policy_version 1265367 (0.0008) [2023-12-27 14:24:56,026][20291] KL-divergence is very high: 100.7546 [2023-12-27 14:24:56,072][20347] Updated weights for policy 0, policy_version 1265377 (0.0007) [2023-12-27 14:24:56,130][20347] Updated weights for policy 0, policy_version 1265387 (0.0007) [2023-12-27 14:24:56,504][20381] Updated weights for policy 1, policy_version 1264345 (0.0006) [2023-12-27 14:24:56,561][20381] Updated weights for policy 1, policy_version 1264355 (0.0006) [2023-12-27 14:24:56,621][20381] Updated weights for policy 1, policy_version 1264365 (0.0006) [2023-12-27 14:24:56,682][20381] Updated weights for policy 1, policy_version 1264375 (0.0005) [2023-12-27 14:24:56,814][20347] Updated weights for policy 0, policy_version 1265397 (0.0009) [2023-12-27 14:24:56,870][20347] Updated weights for policy 0, policy_version 1265407 (0.0010) [2023-12-27 14:24:56,928][20347] Updated weights for policy 0, policy_version 1265417 (0.0009) [2023-12-27 14:24:57,276][20381] Updated weights for policy 1, policy_version 1264385 (0.0005) [2023-12-27 14:24:57,341][20381] Updated weights for policy 1, policy_version 1264395 (0.0005) [2023-12-27 14:24:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 647716864. Throughput: 0: 10370.8, 1: 10307.1. Samples: 647710000. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:24:57,363][19170] Avg episode reward: [(0, '641.340'), (1, '414.990')] [2023-12-27 14:24:57,401][20381] Updated weights for policy 1, policy_version 1264405 (0.0009) [2023-12-27 14:24:57,545][20347] Updated weights for policy 0, policy_version 1265427 (0.0007) [2023-12-27 14:24:57,593][20347] Updated weights for policy 0, policy_version 1265437 (0.0010) [2023-12-27 14:24:57,644][20347] Updated weights for policy 0, policy_version 1265447 (0.0010) [2023-12-27 14:24:58,033][20381] Updated weights for policy 1, policy_version 1264415 (0.0010) [2023-12-27 14:24:58,083][20381] Updated weights for policy 1, policy_version 1264425 (0.0009) [2023-12-27 14:24:58,134][20381] Updated weights for policy 1, policy_version 1264435 (0.0007) [2023-12-27 14:24:58,310][20347] Updated weights for policy 0, policy_version 1265457 (0.0010) [2023-12-27 14:24:58,379][20347] Updated weights for policy 0, policy_version 1265467 (0.0008) [2023-12-27 14:24:58,450][20347] Updated weights for policy 0, policy_version 1265477 (0.0008) [2023-12-27 14:24:58,509][20347] Updated weights for policy 0, policy_version 1265487 (0.0008) [2023-12-27 14:24:58,897][20381] Updated weights for policy 1, policy_version 1264445 (0.0008) [2023-12-27 14:24:58,956][20381] Updated weights for policy 1, policy_version 1264455 (0.0007) [2023-12-27 14:24:59,011][20381] Updated weights for policy 1, policy_version 1264465 (0.0009) [2023-12-27 14:24:59,285][20347] Updated weights for policy 0, policy_version 1265497 (0.0008) [2023-12-27 14:24:59,343][20347] Updated weights for policy 0, policy_version 1265507 (0.0008) [2023-12-27 14:24:59,404][20347] Updated weights for policy 0, policy_version 1265517 (0.0008) [2023-12-27 14:24:59,699][20381] Updated weights for policy 1, policy_version 1264475 (0.0009) [2023-12-27 14:24:59,764][20381] Updated weights for policy 1, policy_version 1264485 (0.0009) [2023-12-27 14:24:59,829][20381] Updated weights for policy 1, policy_version 1264495 (0.0010) [2023-12-27 14:25:00,107][20347] Updated weights for policy 0, policy_version 1265527 (0.0006) [2023-12-27 14:25:00,167][20347] Updated weights for policy 0, policy_version 1265537 (0.0006) [2023-12-27 14:25:00,226][20347] Updated weights for policy 0, policy_version 1265547 (0.0008) [2023-12-27 14:25:00,546][20381] Updated weights for policy 1, policy_version 1264505 (0.0009) [2023-12-27 14:25:00,597][20381] Updated weights for policy 1, policy_version 1264515 (0.0010) [2023-12-27 14:25:00,651][20381] Updated weights for policy 1, policy_version 1264525 (0.0010) [2023-12-27 14:25:00,692][20381] Updated weights for policy 1, policy_version 1264535 (0.0010) [2023-12-27 14:25:00,772][20347] Updated weights for policy 0, policy_version 1265557 (0.0011) [2023-12-27 14:25:00,823][20347] Updated weights for policy 0, policy_version 1265567 (0.0010) [2023-12-27 14:25:00,879][20347] Updated weights for policy 0, policy_version 1265577 (0.0010) [2023-12-27 14:25:01,412][20381] Updated weights for policy 1, policy_version 1264545 (0.0007) [2023-12-27 14:25:01,468][20381] Updated weights for policy 1, policy_version 1264555 (0.0005) [2023-12-27 14:25:01,525][20381] Updated weights for policy 1, policy_version 1264565 (0.0006) [2023-12-27 14:25:01,639][20347] Updated weights for policy 0, policy_version 1265587 (0.0008) [2023-12-27 14:25:01,701][20347] Updated weights for policy 0, policy_version 1265597 (0.0008) [2023-12-27 14:25:01,764][20347] Updated weights for policy 0, policy_version 1265607 (0.0007) [2023-12-27 14:25:02,152][20381] Updated weights for policy 1, policy_version 1264575 (0.0007) [2023-12-27 14:25:02,206][20381] Updated weights for policy 1, policy_version 1264585 (0.0007) [2023-12-27 14:25:02,259][20381] Updated weights for policy 1, policy_version 1264595 (0.0007) [2023-12-27 14:25:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 647823360. Throughput: 0: 10378.2, 1: 10323.1. Samples: 647831984. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:02,363][19170] Avg episode reward: [(0, '603.470'), (1, '370.740')] [2023-12-27 14:25:02,429][20347] Updated weights for policy 0, policy_version 1265617 (0.0009) [2023-12-27 14:25:02,478][20347] Updated weights for policy 0, policy_version 1265627 (0.0005) [2023-12-27 14:25:02,530][20347] Updated weights for policy 0, policy_version 1265637 (0.0006) [2023-12-27 14:25:02,576][20347] Updated weights for policy 0, policy_version 1265647 (0.0008) [2023-12-27 14:25:02,924][20381] Updated weights for policy 1, policy_version 1264605 (0.0007) [2023-12-27 14:25:02,981][20381] Updated weights for policy 1, policy_version 1264615 (0.0007) [2023-12-27 14:25:03,036][20381] Updated weights for policy 1, policy_version 1264625 (0.0007) [2023-12-27 14:25:03,267][20347] Updated weights for policy 0, policy_version 1265657 (0.0008) [2023-12-27 14:25:03,313][20347] Updated weights for policy 0, policy_version 1265667 (0.0007) [2023-12-27 14:25:03,360][20347] Updated weights for policy 0, policy_version 1265677 (0.0008) [2023-12-27 14:25:03,709][20381] Updated weights for policy 1, policy_version 1264635 (0.0008) [2023-12-27 14:25:03,763][20381] Updated weights for policy 1, policy_version 1264645 (0.0007) [2023-12-27 14:25:03,816][20381] Updated weights for policy 1, policy_version 1264655 (0.0007) [2023-12-27 14:25:04,087][20347] Updated weights for policy 0, policy_version 1265687 (0.0008) [2023-12-27 14:25:04,147][20347] Updated weights for policy 0, policy_version 1265697 (0.0008) [2023-12-27 14:25:04,209][20347] Updated weights for policy 0, policy_version 1265707 (0.0011) [2023-12-27 14:25:04,541][20381] Updated weights for policy 1, policy_version 1264665 (0.0007) [2023-12-27 14:25:04,605][20381] Updated weights for policy 1, policy_version 1264675 (0.0008) [2023-12-27 14:25:04,661][20381] Updated weights for policy 1, policy_version 1264685 (0.0005) [2023-12-27 14:25:04,718][20381] Updated weights for policy 1, policy_version 1264695 (0.0006) [2023-12-27 14:25:04,828][20347] Updated weights for policy 0, policy_version 1265717 (0.0009) [2023-12-27 14:25:04,884][20347] Updated weights for policy 0, policy_version 1265727 (0.0005) [2023-12-27 14:25:04,939][20347] Updated weights for policy 0, policy_version 1265737 (0.0005) [2023-12-27 14:25:05,378][20381] Updated weights for policy 1, policy_version 1264705 (0.0010) [2023-12-27 14:25:05,430][20381] Updated weights for policy 1, policy_version 1264715 (0.0007) [2023-12-27 14:25:05,469][20347] Updated weights for policy 0, policy_version 1265747 (0.0005) [2023-12-27 14:25:05,484][20381] Updated weights for policy 1, policy_version 1264725 (0.0007) [2023-12-27 14:25:05,521][20347] Updated weights for policy 0, policy_version 1265757 (0.0005) [2023-12-27 14:25:05,577][20347] Updated weights for policy 0, policy_version 1265767 (0.0005) [2023-12-27 14:25:06,093][20381] Updated weights for policy 1, policy_version 1264735 (0.0007) [2023-12-27 14:25:06,156][20381] Updated weights for policy 1, policy_version 1264745 (0.0009) [2023-12-27 14:25:06,216][20381] Updated weights for policy 1, policy_version 1264755 (0.0010) [2023-12-27 14:25:06,232][20347] Updated weights for policy 0, policy_version 1265777 (0.0006) [2023-12-27 14:25:06,292][20347] Updated weights for policy 0, policy_version 1265787 (0.0007) [2023-12-27 14:25:06,350][20347] Updated weights for policy 0, policy_version 1265797 (0.0007) [2023-12-27 14:25:06,402][20347] Updated weights for policy 0, policy_version 1265807 (0.0007) [2023-12-27 14:25:06,936][20381] Updated weights for policy 1, policy_version 1264765 (0.0011) [2023-12-27 14:25:06,990][20381] Updated weights for policy 1, policy_version 1264775 (0.0008) [2023-12-27 14:25:07,041][20347] Updated weights for policy 0, policy_version 1265817 (0.0010) [2023-12-27 14:25:07,045][20381] Updated weights for policy 1, policy_version 1264785 (0.0010) [2023-12-27 14:25:07,091][20347] Updated weights for policy 0, policy_version 1265827 (0.0010) [2023-12-27 14:25:07,156][20347] Updated weights for policy 0, policy_version 1265837 (0.0010) [2023-12-27 14:25:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 647929856. Throughput: 0: 10373.3, 1: 10302.3. Samples: 647893868. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:07,363][19170] Avg episode reward: [(0, '582.170'), (1, '660.910')] [2023-12-27 14:25:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001264792_323813376.pth... [2023-12-27 14:25:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001265840_324116480.pth... [2023-12-27 14:25:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001263608_323510272.pth [2023-12-27 14:25:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001264592_323796992.pth [2023-12-27 14:25:07,716][20381] Updated weights for policy 1, policy_version 1264795 (0.0011) [2023-12-27 14:25:07,761][20381] Updated weights for policy 1, policy_version 1264805 (0.0010) [2023-12-27 14:25:07,813][20381] Updated weights for policy 1, policy_version 1264815 (0.0008) [2023-12-27 14:25:07,831][20347] Updated weights for policy 0, policy_version 1265847 (0.0010) [2023-12-27 14:25:07,895][20347] Updated weights for policy 0, policy_version 1265857 (0.0010) [2023-12-27 14:25:07,960][20347] Updated weights for policy 0, policy_version 1265867 (0.0010) [2023-12-27 14:25:08,502][20381] Updated weights for policy 1, policy_version 1264825 (0.0007) [2023-12-27 14:25:08,558][20381] Updated weights for policy 1, policy_version 1264835 (0.0007) [2023-12-27 14:25:08,617][20381] Updated weights for policy 1, policy_version 1264845 (0.0007) [2023-12-27 14:25:08,661][20347] Updated weights for policy 0, policy_version 1265877 (0.0009) [2023-12-27 14:25:08,678][20381] Updated weights for policy 1, policy_version 1264855 (0.0007) [2023-12-27 14:25:08,721][20347] Updated weights for policy 0, policy_version 1265887 (0.0008) [2023-12-27 14:25:08,776][20347] Updated weights for policy 0, policy_version 1265897 (0.0007) [2023-12-27 14:25:09,311][20381] Updated weights for policy 1, policy_version 1264865 (0.0007) [2023-12-27 14:25:09,382][20381] Updated weights for policy 1, policy_version 1264875 (0.0006) [2023-12-27 14:25:09,442][20381] Updated weights for policy 1, policy_version 1264885 (0.0008) [2023-12-27 14:25:09,483][20347] Updated weights for policy 0, policy_version 1265907 (0.0007) [2023-12-27 14:25:09,537][20347] Updated weights for policy 0, policy_version 1265917 (0.0006) [2023-12-27 14:25:09,593][20347] Updated weights for policy 0, policy_version 1265927 (0.0008) [2023-12-27 14:25:10,142][20381] Updated weights for policy 1, policy_version 1264895 (0.0007) [2023-12-27 14:25:10,194][20381] Updated weights for policy 1, policy_version 1264905 (0.0007) [2023-12-27 14:25:10,248][20381] Updated weights for policy 1, policy_version 1264915 (0.0007) [2023-12-27 14:25:10,300][20347] Updated weights for policy 0, policy_version 1265937 (0.0008) [2023-12-27 14:25:10,360][20347] Updated weights for policy 0, policy_version 1265947 (0.0007) [2023-12-27 14:25:10,424][20347] Updated weights for policy 0, policy_version 1265957 (0.0006) [2023-12-27 14:25:10,479][20347] Updated weights for policy 0, policy_version 1265967 (0.0008) [2023-12-27 14:25:10,971][20381] Updated weights for policy 1, policy_version 1264925 (0.0008) [2023-12-27 14:25:11,026][20381] Updated weights for policy 1, policy_version 1264935 (0.0007) [2023-12-27 14:25:11,090][20381] Updated weights for policy 1, policy_version 1264945 (0.0008) [2023-12-27 14:25:11,176][20347] Updated weights for policy 0, policy_version 1265977 (0.0008) [2023-12-27 14:25:11,238][20347] Updated weights for policy 0, policy_version 1265987 (0.0008) [2023-12-27 14:25:11,307][20347] Updated weights for policy 0, policy_version 1265997 (0.0008) [2023-12-27 14:25:11,861][20381] Updated weights for policy 1, policy_version 1264955 (0.0007) [2023-12-27 14:25:11,914][20381] Updated weights for policy 1, policy_version 1264965 (0.0005) [2023-12-27 14:25:11,968][20381] Updated weights for policy 1, policy_version 1264975 (0.0005) [2023-12-27 14:25:12,098][20347] Updated weights for policy 0, policy_version 1266007 (0.0010) [2023-12-27 14:25:12,183][20347] Updated weights for policy 0, policy_version 1266017 (0.0008) [2023-12-27 14:25:12,243][20347] Updated weights for policy 0, policy_version 1266027 (0.0005) [2023-12-27 14:25:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 648028160. Throughput: 0: 10396.0, 1: 10279.8. Samples: 648016328. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:12,363][19170] Avg episode reward: [(0, '388.420'), (1, '630.800')] [2023-12-27 14:25:12,617][20381] Updated weights for policy 1, policy_version 1264985 (0.0007) [2023-12-27 14:25:12,668][20381] Updated weights for policy 1, policy_version 1264995 (0.0007) [2023-12-27 14:25:12,719][20381] Updated weights for policy 1, policy_version 1265005 (0.0007) [2023-12-27 14:25:12,778][20381] Updated weights for policy 1, policy_version 1265015 (0.0006) [2023-12-27 14:25:12,879][20347] Updated weights for policy 0, policy_version 1266037 (0.0011) [2023-12-27 14:25:12,938][20347] Updated weights for policy 0, policy_version 1266047 (0.0010) [2023-12-27 14:25:12,995][20347] Updated weights for policy 0, policy_version 1266057 (0.0007) [2023-12-27 14:25:13,396][20381] Updated weights for policy 1, policy_version 1265025 (0.0010) [2023-12-27 14:25:13,450][20381] Updated weights for policy 1, policy_version 1265035 (0.0010) [2023-12-27 14:25:13,495][20381] Updated weights for policy 1, policy_version 1265045 (0.0010) [2023-12-27 14:25:13,671][20347] Updated weights for policy 0, policy_version 1266067 (0.0009) [2023-12-27 14:25:13,727][20347] Updated weights for policy 0, policy_version 1266077 (0.0010) [2023-12-27 14:25:13,783][20347] Updated weights for policy 0, policy_version 1266087 (0.0010) [2023-12-27 14:25:14,189][20381] Updated weights for policy 1, policy_version 1265055 (0.0008) [2023-12-27 14:25:14,245][20381] Updated weights for policy 1, policy_version 1265065 (0.0007) [2023-12-27 14:25:14,292][20381] Updated weights for policy 1, policy_version 1265075 (0.0007) [2023-12-27 14:25:14,459][20347] Updated weights for policy 0, policy_version 1266097 (0.0010) [2023-12-27 14:25:14,516][20347] Updated weights for policy 0, policy_version 1266107 (0.0005) [2023-12-27 14:25:14,573][20347] Updated weights for policy 0, policy_version 1266117 (0.0005) [2023-12-27 14:25:14,629][20347] Updated weights for policy 0, policy_version 1266127 (0.0005) [2023-12-27 14:25:15,039][20381] Updated weights for policy 1, policy_version 1265085 (0.0006) [2023-12-27 14:25:15,097][20381] Updated weights for policy 1, policy_version 1265095 (0.0007) [2023-12-27 14:25:15,151][20381] Updated weights for policy 1, policy_version 1265105 (0.0007) [2023-12-27 14:25:15,288][20347] Updated weights for policy 0, policy_version 1266137 (0.0008) [2023-12-27 14:25:15,344][20347] Updated weights for policy 0, policy_version 1266147 (0.0008) [2023-12-27 14:25:15,400][20347] Updated weights for policy 0, policy_version 1266157 (0.0008) [2023-12-27 14:25:15,804][20381] Updated weights for policy 1, policy_version 1265115 (0.0006) [2023-12-27 14:25:15,864][20381] Updated weights for policy 1, policy_version 1265125 (0.0006) [2023-12-27 14:25:15,916][20381] Updated weights for policy 1, policy_version 1265135 (0.0008) [2023-12-27 14:25:16,051][20347] Updated weights for policy 0, policy_version 1266167 (0.0010) [2023-12-27 14:25:16,105][20347] Updated weights for policy 0, policy_version 1266177 (0.0010) [2023-12-27 14:25:16,163][20347] Updated weights for policy 0, policy_version 1266187 (0.0010) [2023-12-27 14:25:16,572][20381] Updated weights for policy 1, policy_version 1265145 (0.0007) [2023-12-27 14:25:16,629][20381] Updated weights for policy 1, policy_version 1265155 (0.0007) [2023-12-27 14:25:16,686][20381] Updated weights for policy 1, policy_version 1265165 (0.0005) [2023-12-27 14:25:16,744][20381] Updated weights for policy 1, policy_version 1265175 (0.0005) [2023-12-27 14:25:16,854][20347] Updated weights for policy 0, policy_version 1266197 (0.0010) [2023-12-27 14:25:16,902][20347] Updated weights for policy 0, policy_version 1266207 (0.0008) [2023-12-27 14:25:16,950][20347] Updated weights for policy 0, policy_version 1266217 (0.0007) [2023-12-27 14:25:17,313][20381] Updated weights for policy 1, policy_version 1265185 (0.0010) [2023-12-27 14:25:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 648126464. Throughput: 0: 10383.4, 1: 10327.0. Samples: 648139808. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:17,364][19170] Avg episode reward: [(0, '573.850'), (1, '690.860')] [2023-12-27 14:25:17,377][20381] Updated weights for policy 1, policy_version 1265195 (0.0006) [2023-12-27 14:25:17,445][20381] Updated weights for policy 1, policy_version 1265205 (0.0005) [2023-12-27 14:25:17,567][20347] Updated weights for policy 0, policy_version 1266227 (0.0007) [2023-12-27 14:25:17,628][20347] Updated weights for policy 0, policy_version 1266237 (0.0005) [2023-12-27 14:25:17,686][20347] Updated weights for policy 0, policy_version 1266247 (0.0005) [2023-12-27 14:25:18,043][20381] Updated weights for policy 1, policy_version 1265215 (0.0009) [2023-12-27 14:25:18,104][20381] Updated weights for policy 1, policy_version 1265225 (0.0011) [2023-12-27 14:25:18,164][20381] Updated weights for policy 1, policy_version 1265235 (0.0011) [2023-12-27 14:25:18,295][20347] Updated weights for policy 0, policy_version 1266257 (0.0006) [2023-12-27 14:25:18,353][20347] Updated weights for policy 0, policy_version 1266267 (0.0009) [2023-12-27 14:25:18,408][20347] Updated weights for policy 0, policy_version 1266277 (0.0006) [2023-12-27 14:25:18,464][20347] Updated weights for policy 0, policy_version 1266287 (0.0007) [2023-12-27 14:25:18,801][20381] Updated weights for policy 1, policy_version 1265245 (0.0011) [2023-12-27 14:25:18,860][20381] Updated weights for policy 1, policy_version 1265255 (0.0011) [2023-12-27 14:25:18,916][20381] Updated weights for policy 1, policy_version 1265265 (0.0011) [2023-12-27 14:25:19,137][20347] Updated weights for policy 0, policy_version 1266297 (0.0008) [2023-12-27 14:25:19,195][20347] Updated weights for policy 0, policy_version 1266307 (0.0006) [2023-12-27 14:25:19,266][20347] Updated weights for policy 0, policy_version 1266317 (0.0007) [2023-12-27 14:25:19,641][20381] Updated weights for policy 1, policy_version 1265275 (0.0010) [2023-12-27 14:25:19,702][20381] Updated weights for policy 1, policy_version 1265285 (0.0009) [2023-12-27 14:25:19,757][20381] Updated weights for policy 1, policy_version 1265295 (0.0007) [2023-12-27 14:25:19,980][20347] Updated weights for policy 0, policy_version 1266327 (0.0010) [2023-12-27 14:25:20,036][20347] Updated weights for policy 0, policy_version 1266337 (0.0010) [2023-12-27 14:25:20,091][20347] Updated weights for policy 0, policy_version 1266347 (0.0011) [2023-12-27 14:25:20,465][20381] Updated weights for policy 1, policy_version 1265305 (0.0008) [2023-12-27 14:25:20,518][20381] Updated weights for policy 1, policy_version 1265315 (0.0011) [2023-12-27 14:25:20,564][20381] Updated weights for policy 1, policy_version 1265325 (0.0010) [2023-12-27 14:25:20,625][20381] Updated weights for policy 1, policy_version 1265335 (0.0008) [2023-12-27 14:25:20,787][20347] Updated weights for policy 0, policy_version 1266357 (0.0010) [2023-12-27 14:25:20,846][20347] Updated weights for policy 0, policy_version 1266367 (0.0010) [2023-12-27 14:25:20,911][20347] Updated weights for policy 0, policy_version 1266377 (0.0010) [2023-12-27 14:25:21,360][20381] Updated weights for policy 1, policy_version 1265345 (0.0007) [2023-12-27 14:25:21,429][20381] Updated weights for policy 1, policy_version 1265355 (0.0007) [2023-12-27 14:25:21,492][20381] Updated weights for policy 1, policy_version 1265365 (0.0006) [2023-12-27 14:25:21,685][20347] Updated weights for policy 0, policy_version 1266387 (0.0009) [2023-12-27 14:25:21,754][20347] Updated weights for policy 0, policy_version 1266397 (0.0007) [2023-12-27 14:25:21,809][20347] Updated weights for policy 0, policy_version 1266407 (0.0008) [2023-12-27 14:25:22,165][20381] Updated weights for policy 1, policy_version 1265375 (0.0006) [2023-12-27 14:25:22,213][20381] Updated weights for policy 1, policy_version 1265385 (0.0009) [2023-12-27 14:25:22,264][20381] Updated weights for policy 1, policy_version 1265395 (0.0009) [2023-12-27 14:25:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 648232960. Throughput: 0: 10373.2, 1: 10246.5. Samples: 648201556. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:22,363][19170] Avg episode reward: [(0, '600.060'), (1, '777.480')] [2023-12-27 14:25:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001265400_323969024.pth... [2023-12-27 14:25:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001266416_324263936.pth... [2023-12-27 14:25:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001264184_323657728.pth [2023-12-27 14:25:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001265232_323960832.pth [2023-12-27 14:25:22,440][20347] Updated weights for policy 0, policy_version 1266417 (0.0007) [2023-12-27 14:25:22,495][20347] Updated weights for policy 0, policy_version 1266427 (0.0008) [2023-12-27 14:25:22,552][20347] Updated weights for policy 0, policy_version 1266437 (0.0007) [2023-12-27 14:25:22,610][20347] Updated weights for policy 0, policy_version 1266447 (0.0007) [2023-12-27 14:25:22,988][20381] Updated weights for policy 1, policy_version 1265405 (0.0009) [2023-12-27 14:25:23,035][20381] Updated weights for policy 1, policy_version 1265415 (0.0011) [2023-12-27 14:25:23,085][20381] Updated weights for policy 1, policy_version 1265425 (0.0011) [2023-12-27 14:25:23,235][20347] Updated weights for policy 0, policy_version 1266457 (0.0006) [2023-12-27 14:25:23,297][20347] Updated weights for policy 0, policy_version 1266467 (0.0005) [2023-12-27 14:25:23,352][20347] Updated weights for policy 0, policy_version 1266477 (0.0007) [2023-12-27 14:25:23,809][20381] Updated weights for policy 1, policy_version 1265435 (0.0008) [2023-12-27 14:25:23,860][20381] Updated weights for policy 1, policy_version 1265445 (0.0007) [2023-12-27 14:25:23,911][20381] Updated weights for policy 1, policy_version 1265455 (0.0007) [2023-12-27 14:25:23,976][20347] Updated weights for policy 0, policy_version 1266487 (0.0007) [2023-12-27 14:25:24,032][20347] Updated weights for policy 0, policy_version 1266497 (0.0008) [2023-12-27 14:25:24,079][20347] Updated weights for policy 0, policy_version 1266507 (0.0007) [2023-12-27 14:25:24,613][20381] Updated weights for policy 1, policy_version 1265465 (0.0007) [2023-12-27 14:25:24,666][20381] Updated weights for policy 1, policy_version 1265475 (0.0007) [2023-12-27 14:25:24,715][20381] Updated weights for policy 1, policy_version 1265485 (0.0007) [2023-12-27 14:25:24,765][20381] Updated weights for policy 1, policy_version 1265495 (0.0007) [2023-12-27 14:25:24,770][20347] Updated weights for policy 0, policy_version 1266517 (0.0006) [2023-12-27 14:25:24,831][20347] Updated weights for policy 0, policy_version 1266527 (0.0008) [2023-12-27 14:25:24,892][20347] Updated weights for policy 0, policy_version 1266537 (0.0007) [2023-12-27 14:25:25,393][20381] Updated weights for policy 1, policy_version 1265505 (0.0010) [2023-12-27 14:25:25,450][20381] Updated weights for policy 1, policy_version 1265515 (0.0010) [2023-12-27 14:25:25,508][20381] Updated weights for policy 1, policy_version 1265525 (0.0011) [2023-12-27 14:25:25,592][20347] Updated weights for policy 0, policy_version 1266547 (0.0008) [2023-12-27 14:25:25,652][20347] Updated weights for policy 0, policy_version 1266557 (0.0007) [2023-12-27 14:25:25,706][20347] Updated weights for policy 0, policy_version 1266567 (0.0007) [2023-12-27 14:25:26,177][20381] Updated weights for policy 1, policy_version 1265535 (0.0010) [2023-12-27 14:25:26,233][20381] Updated weights for policy 1, policy_version 1265545 (0.0007) [2023-12-27 14:25:26,288][20381] Updated weights for policy 1, policy_version 1265555 (0.0010) [2023-12-27 14:25:26,402][20347] Updated weights for policy 0, policy_version 1266577 (0.0007) [2023-12-27 14:25:26,458][20347] Updated weights for policy 0, policy_version 1266587 (0.0007) [2023-12-27 14:25:26,511][20347] Updated weights for policy 0, policy_version 1266597 (0.0007) [2023-12-27 14:25:26,568][20347] Updated weights for policy 0, policy_version 1266607 (0.0007) [2023-12-27 14:25:26,971][20381] Updated weights for policy 1, policy_version 1265565 (0.0008) [2023-12-27 14:25:27,020][20381] Updated weights for policy 1, policy_version 1265575 (0.0005) [2023-12-27 14:25:27,084][20381] Updated weights for policy 1, policy_version 1265585 (0.0005) [2023-12-27 14:25:27,191][20347] Updated weights for policy 0, policy_version 1266617 (0.0007) [2023-12-27 14:25:27,241][20347] Updated weights for policy 0, policy_version 1266627 (0.0007) [2023-12-27 14:25:27,291][20347] Updated weights for policy 0, policy_version 1266637 (0.0007) [2023-12-27 14:25:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 648339456. Throughput: 0: 10396.5, 1: 10239.1. Samples: 648323448. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:27,363][19170] Avg episode reward: [(0, '591.930'), (1, '767.190')] [2023-12-27 14:25:27,629][20381] Updated weights for policy 1, policy_version 1265595 (0.0007) [2023-12-27 14:25:27,679][20381] Updated weights for policy 1, policy_version 1265605 (0.0009) [2023-12-27 14:25:27,727][20381] Updated weights for policy 1, policy_version 1265615 (0.0010) [2023-12-27 14:25:27,986][20347] Updated weights for policy 0, policy_version 1266647 (0.0006) [2023-12-27 14:25:28,045][20347] Updated weights for policy 0, policy_version 1266657 (0.0009) [2023-12-27 14:25:28,106][20347] Updated weights for policy 0, policy_version 1266667 (0.0005) [2023-12-27 14:25:28,370][20381] Updated weights for policy 1, policy_version 1265625 (0.0010) [2023-12-27 14:25:28,422][20381] Updated weights for policy 1, policy_version 1265635 (0.0008) [2023-12-27 14:25:28,469][20381] Updated weights for policy 1, policy_version 1265645 (0.0008) [2023-12-27 14:25:28,524][20381] Updated weights for policy 1, policy_version 1265656 (0.0007) [2023-12-27 14:25:28,706][20347] Updated weights for policy 0, policy_version 1266677 (0.0007) [2023-12-27 14:25:28,762][20347] Updated weights for policy 0, policy_version 1266687 (0.0008) [2023-12-27 14:25:28,815][20347] Updated weights for policy 0, policy_version 1266697 (0.0007) [2023-12-27 14:25:29,189][20381] Updated weights for policy 1, policy_version 1265666 (0.0005) [2023-12-27 14:25:29,240][20381] Updated weights for policy 1, policy_version 1265676 (0.0007) [2023-12-27 14:25:29,296][20381] Updated weights for policy 1, policy_version 1265686 (0.0007) [2023-12-27 14:25:29,487][20347] Updated weights for policy 0, policy_version 1266707 (0.0006) [2023-12-27 14:25:29,545][20347] Updated weights for policy 0, policy_version 1266717 (0.0008) [2023-12-27 14:25:29,604][20347] Updated weights for policy 0, policy_version 1266727 (0.0008) [2023-12-27 14:25:30,016][20381] Updated weights for policy 1, policy_version 1265696 (0.0010) [2023-12-27 14:25:30,068][20381] Updated weights for policy 1, policy_version 1265706 (0.0007) [2023-12-27 14:25:30,114][20381] Updated weights for policy 1, policy_version 1265716 (0.0008) [2023-12-27 14:25:30,298][20347] Updated weights for policy 0, policy_version 1266737 (0.0007) [2023-12-27 14:25:30,361][20347] Updated weights for policy 0, policy_version 1266747 (0.0006) [2023-12-27 14:25:30,413][20347] Updated weights for policy 0, policy_version 1266757 (0.0006) [2023-12-27 14:25:30,464][20347] Updated weights for policy 0, policy_version 1266767 (0.0006) [2023-12-27 14:25:30,832][20381] Updated weights for policy 1, policy_version 1265726 (0.0007) [2023-12-27 14:25:30,878][20381] Updated weights for policy 1, policy_version 1265736 (0.0008) [2023-12-27 14:25:30,923][20381] Updated weights for policy 1, policy_version 1265746 (0.0009) [2023-12-27 14:25:31,042][20347] Updated weights for policy 0, policy_version 1266777 (0.0006) [2023-12-27 14:25:31,101][20347] Updated weights for policy 0, policy_version 1266787 (0.0006) [2023-12-27 14:25:31,166][20347] Updated weights for policy 0, policy_version 1266797 (0.0007) [2023-12-27 14:25:31,690][20381] Updated weights for policy 1, policy_version 1265756 (0.0007) [2023-12-27 14:25:31,752][20381] Updated weights for policy 1, policy_version 1265766 (0.0008) [2023-12-27 14:25:31,813][20381] Updated weights for policy 1, policy_version 1265776 (0.0007) [2023-12-27 14:25:31,822][20347] Updated weights for policy 0, policy_version 1266807 (0.0007) [2023-12-27 14:25:31,882][20347] Updated weights for policy 0, policy_version 1266817 (0.0008) [2023-12-27 14:25:31,941][20347] Updated weights for policy 0, policy_version 1266827 (0.0009) [2023-12-27 14:25:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 648437760. Throughput: 0: 10339.1, 1: 10273.3. Samples: 648448672. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:32,363][19170] Avg episode reward: [(0, '613.140'), (1, '826.010')] [2023-12-27 14:25:32,558][20381] Updated weights for policy 1, policy_version 1265786 (0.0005) [2023-12-27 14:25:32,598][20347] Updated weights for policy 0, policy_version 1266837 (0.0009) [2023-12-27 14:25:32,615][20381] Updated weights for policy 1, policy_version 1265796 (0.0005) [2023-12-27 14:25:32,655][20347] Updated weights for policy 0, policy_version 1266847 (0.0007) [2023-12-27 14:25:32,676][20381] Updated weights for policy 1, policy_version 1265806 (0.0006) [2023-12-27 14:25:32,709][20347] Updated weights for policy 0, policy_version 1266857 (0.0006) [2023-12-27 14:25:32,734][20381] Updated weights for policy 1, policy_version 1265816 (0.0005) [2023-12-27 14:25:33,346][20347] Updated weights for policy 0, policy_version 1266867 (0.0006) [2023-12-27 14:25:33,408][20347] Updated weights for policy 0, policy_version 1266877 (0.0005) [2023-12-27 14:25:33,428][20381] Updated weights for policy 1, policy_version 1265826 (0.0005) [2023-12-27 14:25:33,463][20347] Updated weights for policy 0, policy_version 1266887 (0.0009) [2023-12-27 14:25:33,473][20381] Updated weights for policy 1, policy_version 1265836 (0.0005) [2023-12-27 14:25:33,523][20381] Updated weights for policy 1, policy_version 1265846 (0.0007) [2023-12-27 14:25:34,079][20347] Updated weights for policy 0, policy_version 1266897 (0.0008) [2023-12-27 14:25:34,137][20347] Updated weights for policy 0, policy_version 1266907 (0.0007) [2023-12-27 14:25:34,204][20347] Updated weights for policy 0, policy_version 1266919 (0.0008) [2023-12-27 14:25:34,206][20381] Updated weights for policy 1, policy_version 1265856 (0.0006) [2023-12-27 14:25:34,268][20381] Updated weights for policy 1, policy_version 1265866 (0.0007) [2023-12-27 14:25:34,327][20381] Updated weights for policy 1, policy_version 1265876 (0.0007) [2023-12-27 14:25:34,974][20347] Updated weights for policy 0, policy_version 1266929 (0.0008) [2023-12-27 14:25:35,010][20381] Updated weights for policy 1, policy_version 1265886 (0.0008) [2023-12-27 14:25:35,025][20347] Updated weights for policy 0, policy_version 1266939 (0.0007) [2023-12-27 14:25:35,066][20381] Updated weights for policy 1, policy_version 1265896 (0.0006) [2023-12-27 14:25:35,076][20347] Updated weights for policy 0, policy_version 1266949 (0.0006) [2023-12-27 14:25:35,122][20381] Updated weights for policy 1, policy_version 1265906 (0.0007) [2023-12-27 14:25:35,127][20347] Updated weights for policy 0, policy_version 1266959 (0.0005) [2023-12-27 14:25:35,755][20347] Updated weights for policy 0, policy_version 1266969 (0.0006) [2023-12-27 14:25:35,780][20381] Updated weights for policy 1, policy_version 1265916 (0.0006) [2023-12-27 14:25:35,805][20347] Updated weights for policy 0, policy_version 1266979 (0.0007) [2023-12-27 14:25:35,846][20381] Updated weights for policy 1, policy_version 1265926 (0.0006) [2023-12-27 14:25:35,857][20347] Updated weights for policy 0, policy_version 1266989 (0.0006) [2023-12-27 14:25:35,901][20381] Updated weights for policy 1, policy_version 1265936 (0.0008) [2023-12-27 14:25:36,517][20381] Updated weights for policy 1, policy_version 1265946 (0.0007) [2023-12-27 14:25:36,560][20347] Updated weights for policy 0, policy_version 1266999 (0.0006) [2023-12-27 14:25:36,570][20381] Updated weights for policy 1, policy_version 1265956 (0.0010) [2023-12-27 14:25:36,613][20347] Updated weights for policy 0, policy_version 1267009 (0.0006) [2023-12-27 14:25:36,630][20381] Updated weights for policy 1, policy_version 1265966 (0.0010) [2023-12-27 14:25:36,676][20347] Updated weights for policy 0, policy_version 1267019 (0.0006) [2023-12-27 14:25:36,686][20381] Updated weights for policy 1, policy_version 1265976 (0.0010) [2023-12-27 14:25:37,318][20347] Updated weights for policy 0, policy_version 1267029 (0.0007) [2023-12-27 14:25:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 648536064. Throughput: 0: 10332.7, 1: 10259.3. Samples: 648509372. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:37,363][19170] Avg episode reward: [(0, '668.770'), (1, '816.990')] [2023-12-27 14:25:37,364][20347] Updated weights for policy 0, policy_version 1267039 (0.0006) [2023-12-27 14:25:37,393][20381] Updated weights for policy 1, policy_version 1265986 (0.0010) [2023-12-27 14:25:37,418][20347] Updated weights for policy 0, policy_version 1267049 (0.0006) [2023-12-27 14:25:37,452][20381] Updated weights for policy 1, policy_version 1265996 (0.0005) [2023-12-27 14:25:37,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001267056_324427776.pth... [2023-12-27 14:25:37,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001265840_324116480.pth [2023-12-27 14:25:37,506][20381] Updated weights for policy 1, policy_version 1266006 (0.0007) [2023-12-27 14:25:37,515][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001266008_324124672.pth... [2023-12-27 14:25:37,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001264792_323813376.pth [2023-12-27 14:25:38,046][20381] Updated weights for policy 1, policy_version 1266016 (0.0006) [2023-12-27 14:25:38,097][20381] Updated weights for policy 1, policy_version 1266026 (0.0005) [2023-12-27 14:25:38,146][20381] Updated weights for policy 1, policy_version 1266036 (0.0009) [2023-12-27 14:25:38,151][20347] Updated weights for policy 0, policy_version 1267059 (0.0005) [2023-12-27 14:25:38,204][20347] Updated weights for policy 0, policy_version 1267069 (0.0007) [2023-12-27 14:25:38,260][20347] Updated weights for policy 0, policy_version 1267079 (0.0007) [2023-12-27 14:25:38,831][20381] Updated weights for policy 1, policy_version 1266046 (0.0009) [2023-12-27 14:25:38,887][20381] Updated weights for policy 1, policy_version 1266056 (0.0010) [2023-12-27 14:25:38,940][20347] Updated weights for policy 0, policy_version 1267089 (0.0007) [2023-12-27 14:25:38,943][20381] Updated weights for policy 1, policy_version 1266066 (0.0010) [2023-12-27 14:25:39,000][20347] Updated weights for policy 0, policy_version 1267099 (0.0010) [2023-12-27 14:25:39,045][20347] Updated weights for policy 0, policy_version 1267109 (0.0010) [2023-12-27 14:25:39,097][20347] Updated weights for policy 0, policy_version 1267119 (0.0010) [2023-12-27 14:25:39,616][20381] Updated weights for policy 1, policy_version 1266076 (0.0012) [2023-12-27 14:25:39,674][20381] Updated weights for policy 1, policy_version 1266086 (0.0010) [2023-12-27 14:25:39,733][20381] Updated weights for policy 1, policy_version 1266096 (0.0011) [2023-12-27 14:25:39,826][20347] Updated weights for policy 0, policy_version 1267129 (0.0007) [2023-12-27 14:25:39,888][20347] Updated weights for policy 0, policy_version 1267139 (0.0007) [2023-12-27 14:25:39,954][20347] Updated weights for policy 0, policy_version 1267149 (0.0007) [2023-12-27 14:25:40,393][20381] Updated weights for policy 1, policy_version 1266106 (0.0010) [2023-12-27 14:25:40,449][20381] Updated weights for policy 1, policy_version 1266116 (0.0005) [2023-12-27 14:25:40,496][20381] Updated weights for policy 1, policy_version 1266126 (0.0005) [2023-12-27 14:25:40,550][20381] Updated weights for policy 1, policy_version 1266136 (0.0006) [2023-12-27 14:25:40,601][20347] Updated weights for policy 0, policy_version 1267159 (0.0007) [2023-12-27 14:25:40,663][20347] Updated weights for policy 0, policy_version 1267169 (0.0008) [2023-12-27 14:25:40,732][20347] Updated weights for policy 0, policy_version 1267179 (0.0008) [2023-12-27 14:25:41,217][20381] Updated weights for policy 1, policy_version 1266146 (0.0007) [2023-12-27 14:25:41,280][20381] Updated weights for policy 1, policy_version 1266156 (0.0007) [2023-12-27 14:25:41,342][20381] Updated weights for policy 1, policy_version 1266166 (0.0008) [2023-12-27 14:25:41,421][20347] Updated weights for policy 0, policy_version 1267189 (0.0009) [2023-12-27 14:25:41,474][20347] Updated weights for policy 0, policy_version 1267199 (0.0007) [2023-12-27 14:25:41,530][20347] Updated weights for policy 0, policy_version 1267209 (0.0008) [2023-12-27 14:25:42,079][20381] Updated weights for policy 1, policy_version 1266176 (0.0008) [2023-12-27 14:25:42,136][20381] Updated weights for policy 1, policy_version 1266186 (0.0007) [2023-12-27 14:25:42,192][20381] Updated weights for policy 1, policy_version 1266196 (0.0008) [2023-12-27 14:25:42,302][20347] Updated weights for policy 0, policy_version 1267219 (0.0008) [2023-12-27 14:25:42,360][20347] Updated weights for policy 0, policy_version 1267229 (0.0007) [2023-12-27 14:25:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 648642560. Throughput: 0: 10284.2, 1: 10273.7. Samples: 648635104. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:42,363][19170] Avg episode reward: [(0, '628.960'), (1, '776.970')] [2023-12-27 14:25:42,419][20347] Updated weights for policy 0, policy_version 1267239 (0.0007) [2023-12-27 14:25:42,833][20381] Updated weights for policy 1, policy_version 1266206 (0.0007) [2023-12-27 14:25:42,887][20381] Updated weights for policy 1, policy_version 1266216 (0.0007) [2023-12-27 14:25:42,950][20381] Updated weights for policy 1, policy_version 1266226 (0.0007) [2023-12-27 14:25:43,045][20347] Updated weights for policy 0, policy_version 1267249 (0.0006) [2023-12-27 14:25:43,100][20347] Updated weights for policy 0, policy_version 1267259 (0.0007) [2023-12-27 14:25:43,159][20347] Updated weights for policy 0, policy_version 1267269 (0.0008) [2023-12-27 14:25:43,212][20347] Updated weights for policy 0, policy_version 1267279 (0.0006) [2023-12-27 14:25:43,575][20381] Updated weights for policy 1, policy_version 1266236 (0.0007) [2023-12-27 14:25:43,639][20381] Updated weights for policy 1, policy_version 1266246 (0.0006) [2023-12-27 14:25:43,690][20381] Updated weights for policy 1, policy_version 1266256 (0.0005) [2023-12-27 14:25:43,760][20347] Updated weights for policy 0, policy_version 1267289 (0.0005) [2023-12-27 14:25:43,818][20347] Updated weights for policy 0, policy_version 1267299 (0.0007) [2023-12-27 14:25:43,873][20347] Updated weights for policy 0, policy_version 1267309 (0.0007) [2023-12-27 14:25:44,254][20381] Updated weights for policy 1, policy_version 1266266 (0.0010) [2023-12-27 14:25:44,309][20381] Updated weights for policy 1, policy_version 1266276 (0.0010) [2023-12-27 14:25:44,354][20381] Updated weights for policy 1, policy_version 1266286 (0.0010) [2023-12-27 14:25:44,403][20381] Updated weights for policy 1, policy_version 1266296 (0.0010) [2023-12-27 14:25:44,414][20347] Updated weights for policy 0, policy_version 1267319 (0.0005) [2023-12-27 14:25:44,471][20347] Updated weights for policy 0, policy_version 1267329 (0.0005) [2023-12-27 14:25:44,524][20347] Updated weights for policy 0, policy_version 1267339 (0.0005) [2023-12-27 14:25:45,129][20381] Updated weights for policy 1, policy_version 1266306 (0.0008) [2023-12-27 14:25:45,131][20347] Updated weights for policy 0, policy_version 1267349 (0.0006) [2023-12-27 14:25:45,186][20381] Updated weights for policy 1, policy_version 1266316 (0.0008) [2023-12-27 14:25:45,192][20347] Updated weights for policy 0, policy_version 1267359 (0.0007) [2023-12-27 14:25:45,245][20381] Updated weights for policy 1, policy_version 1266326 (0.0008) [2023-12-27 14:25:45,249][20347] Updated weights for policy 0, policy_version 1267369 (0.0007) [2023-12-27 14:25:45,958][20347] Updated weights for policy 0, policy_version 1267379 (0.0007) [2023-12-27 14:25:45,989][20381] Updated weights for policy 1, policy_version 1266336 (0.0010) [2023-12-27 14:25:46,013][20347] Updated weights for policy 0, policy_version 1267389 (0.0005) [2023-12-27 14:25:46,044][20381] Updated weights for policy 1, policy_version 1266346 (0.0008) [2023-12-27 14:25:46,079][20347] Updated weights for policy 0, policy_version 1267399 (0.0006) [2023-12-27 14:25:46,108][20381] Updated weights for policy 1, policy_version 1266356 (0.0007) [2023-12-27 14:25:46,708][20381] Updated weights for policy 1, policy_version 1266366 (0.0009) [2023-12-27 14:25:46,763][20381] Updated weights for policy 1, policy_version 1266376 (0.0006) [2023-12-27 14:25:46,765][20347] Updated weights for policy 0, policy_version 1267409 (0.0008) [2023-12-27 14:25:46,819][20347] Updated weights for policy 0, policy_version 1267419 (0.0006) [2023-12-27 14:25:46,824][20381] Updated weights for policy 1, policy_version 1266386 (0.0006) [2023-12-27 14:25:46,875][20347] Updated weights for policy 0, policy_version 1267429 (0.0006) [2023-12-27 14:25:46,936][20347] Updated weights for policy 0, policy_version 1267439 (0.0007) [2023-12-27 14:25:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 648749056. Throughput: 0: 10342.8, 1: 10296.0. Samples: 648760728. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 14:25:47,363][19170] Avg episode reward: [(0, '648.920'), (1, '798.280')] [2023-12-27 14:25:47,470][20381] Updated weights for policy 1, policy_version 1266396 (0.0005) [2023-12-27 14:25:47,520][20381] Updated weights for policy 1, policy_version 1266406 (0.0007) [2023-12-27 14:25:47,574][20381] Updated weights for policy 1, policy_version 1266416 (0.0006) [2023-12-27 14:25:47,613][20347] Updated weights for policy 0, policy_version 1267449 (0.0010) [2023-12-27 14:25:47,673][20347] Updated weights for policy 0, policy_version 1267459 (0.0010) [2023-12-27 14:25:47,740][20347] Updated weights for policy 0, policy_version 1267469 (0.0009) [2023-12-27 14:25:48,235][20381] Updated weights for policy 1, policy_version 1266426 (0.0007) [2023-12-27 14:25:48,294][20381] Updated weights for policy 1, policy_version 1266436 (0.0009) [2023-12-27 14:25:48,359][20381] Updated weights for policy 1, policy_version 1266446 (0.0009) [2023-12-27 14:25:48,378][20347] Updated weights for policy 0, policy_version 1267479 (0.0008) [2023-12-27 14:25:48,419][20381] Updated weights for policy 1, policy_version 1266456 (0.0009) [2023-12-27 14:25:48,442][20347] Updated weights for policy 0, policy_version 1267489 (0.0008) [2023-12-27 14:25:48,491][20347] Updated weights for policy 0, policy_version 1267499 (0.0008) [2023-12-27 14:25:49,146][20381] Updated weights for policy 1, policy_version 1266466 (0.0006) [2023-12-27 14:25:49,191][20381] Updated weights for policy 1, policy_version 1266476 (0.0006) [2023-12-27 14:25:49,216][20347] Updated weights for policy 0, policy_version 1267509 (0.0007) [2023-12-27 14:25:49,255][20381] Updated weights for policy 1, policy_version 1266486 (0.0007) [2023-12-27 14:25:49,271][20347] Updated weights for policy 0, policy_version 1267519 (0.0008) [2023-12-27 14:25:49,332][20347] Updated weights for policy 0, policy_version 1267529 (0.0008) [2023-12-27 14:25:49,986][20381] Updated weights for policy 1, policy_version 1266496 (0.0008) [2023-12-27 14:25:50,040][20381] Updated weights for policy 1, policy_version 1266506 (0.0005) [2023-12-27 14:25:50,092][20381] Updated weights for policy 1, policy_version 1266516 (0.0005) [2023-12-27 14:25:50,114][20347] Updated weights for policy 0, policy_version 1267539 (0.0010) [2023-12-27 14:25:50,164][20347] Updated weights for policy 0, policy_version 1267549 (0.0008) [2023-12-27 14:25:50,220][20347] Updated weights for policy 0, policy_version 1267559 (0.0008) [2023-12-27 14:25:50,782][20381] Updated weights for policy 1, policy_version 1266526 (0.0007) [2023-12-27 14:25:50,843][20381] Updated weights for policy 1, policy_version 1266536 (0.0006) [2023-12-27 14:25:50,902][20347] Updated weights for policy 0, policy_version 1267569 (0.0007) [2023-12-27 14:25:50,903][20381] Updated weights for policy 1, policy_version 1266546 (0.0006) [2023-12-27 14:25:50,972][20347] Updated weights for policy 0, policy_version 1267579 (0.0006) [2023-12-27 14:25:51,038][20347] Updated weights for policy 0, policy_version 1267589 (0.0007) [2023-12-27 14:25:51,103][20347] Updated weights for policy 0, policy_version 1267599 (0.0007) [2023-12-27 14:25:51,590][20381] Updated weights for policy 1, policy_version 1266556 (0.0008) [2023-12-27 14:25:51,648][20381] Updated weights for policy 1, policy_version 1266566 (0.0009) [2023-12-27 14:25:51,701][20381] Updated weights for policy 1, policy_version 1266576 (0.0011) [2023-12-27 14:25:51,833][20347] Updated weights for policy 0, policy_version 1267609 (0.0007) [2023-12-27 14:25:51,894][20347] Updated weights for policy 0, policy_version 1267619 (0.0007) [2023-12-27 14:25:51,955][20347] Updated weights for policy 0, policy_version 1267629 (0.0007) [2023-12-27 14:25:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 648847360. Throughput: 0: 10306.2, 1: 10298.9. Samples: 648821100. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:25:52,363][19170] Avg episode reward: [(0, '619.420'), (1, '799.240')] [2023-12-27 14:25:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001266584_324272128.pth... [2023-12-27 14:25:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001267632_324575232.pth... [2023-12-27 14:25:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001266416_324263936.pth [2023-12-27 14:25:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001265400_323969024.pth [2023-12-27 14:25:52,466][20381] Updated weights for policy 1, policy_version 1266586 (0.0010) [2023-12-27 14:25:52,525][20381] Updated weights for policy 1, policy_version 1266596 (0.0008) [2023-12-27 14:25:52,582][20381] Updated weights for policy 1, policy_version 1266606 (0.0008) [2023-12-27 14:25:52,633][20381] Updated weights for policy 1, policy_version 1266616 (0.0007) [2023-12-27 14:25:52,635][20347] Updated weights for policy 0, policy_version 1267639 (0.0006) [2023-12-27 14:25:52,682][20347] Updated weights for policy 0, policy_version 1267649 (0.0007) [2023-12-27 14:25:52,729][20347] Updated weights for policy 0, policy_version 1267659 (0.0005) [2023-12-27 14:25:53,327][20347] Updated weights for policy 0, policy_version 1267669 (0.0006) [2023-12-27 14:25:53,358][20381] Updated weights for policy 1, policy_version 1266626 (0.0006) [2023-12-27 14:25:53,388][20347] Updated weights for policy 0, policy_version 1267679 (0.0008) [2023-12-27 14:25:53,409][20381] Updated weights for policy 1, policy_version 1266636 (0.0005) [2023-12-27 14:25:53,438][20347] Updated weights for policy 0, policy_version 1267689 (0.0010) [2023-12-27 14:25:53,459][20381] Updated weights for policy 1, policy_version 1266646 (0.0005) [2023-12-27 14:25:53,980][20381] Updated weights for policy 1, policy_version 1266656 (0.0005) [2023-12-27 14:25:54,026][20381] Updated weights for policy 1, policy_version 1266666 (0.0005) [2023-12-27 14:25:54,075][20381] Updated weights for policy 1, policy_version 1266676 (0.0005) [2023-12-27 14:25:54,123][20347] Updated weights for policy 0, policy_version 1267699 (0.0007) [2023-12-27 14:25:54,183][20347] Updated weights for policy 0, policy_version 1267709 (0.0008) [2023-12-27 14:25:54,238][20347] Updated weights for policy 0, policy_version 1267719 (0.0007) [2023-12-27 14:25:54,656][20381] Updated weights for policy 1, policy_version 1266686 (0.0005) [2023-12-27 14:25:54,714][20381] Updated weights for policy 1, policy_version 1266696 (0.0005) [2023-12-27 14:25:54,771][20381] Updated weights for policy 1, policy_version 1266706 (0.0007) [2023-12-27 14:25:54,915][20347] Updated weights for policy 0, policy_version 1267729 (0.0008) [2023-12-27 14:25:54,984][20347] Updated weights for policy 0, policy_version 1267739 (0.0011) [2023-12-27 14:25:55,037][20347] Updated weights for policy 0, policy_version 1267749 (0.0010) [2023-12-27 14:25:55,089][20347] Updated weights for policy 0, policy_version 1267759 (0.0010) [2023-12-27 14:25:55,401][20381] Updated weights for policy 1, policy_version 1266716 (0.0009) [2023-12-27 14:25:55,461][20381] Updated weights for policy 1, policy_version 1266726 (0.0006) [2023-12-27 14:25:55,521][20381] Updated weights for policy 1, policy_version 1266736 (0.0009) [2023-12-27 14:25:55,718][20347] Updated weights for policy 0, policy_version 1267769 (0.0006) [2023-12-27 14:25:55,776][20347] Updated weights for policy 0, policy_version 1267779 (0.0005) [2023-12-27 14:25:55,837][20347] Updated weights for policy 0, policy_version 1267789 (0.0007) [2023-12-27 14:25:56,094][20381] Updated weights for policy 1, policy_version 1266746 (0.0006) [2023-12-27 14:25:56,143][20381] Updated weights for policy 1, policy_version 1266756 (0.0007) [2023-12-27 14:25:56,195][20381] Updated weights for policy 1, policy_version 1266766 (0.0009) [2023-12-27 14:25:56,248][20381] Updated weights for policy 1, policy_version 1266776 (0.0010) [2023-12-27 14:25:56,448][20347] Updated weights for policy 0, policy_version 1267799 (0.0010) [2023-12-27 14:25:56,499][20347] Updated weights for policy 0, policy_version 1267809 (0.0010) [2023-12-27 14:25:56,550][20347] Updated weights for policy 0, policy_version 1267819 (0.0010) [2023-12-27 14:25:56,930][20381] Updated weights for policy 1, policy_version 1266786 (0.0010) [2023-12-27 14:25:56,978][20381] Updated weights for policy 1, policy_version 1266796 (0.0010) [2023-12-27 14:25:57,026][20381] Updated weights for policy 1, policy_version 1266806 (0.0010) [2023-12-27 14:25:57,203][20347] Updated weights for policy 0, policy_version 1267829 (0.0009) [2023-12-27 14:25:57,262][20347] Updated weights for policy 0, policy_version 1267839 (0.0005) [2023-12-27 14:25:57,321][20347] Updated weights for policy 0, policy_version 1267849 (0.0006) [2023-12-27 14:25:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 648953856. Throughput: 0: 10314.7, 1: 10368.2. Samples: 648947064. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:25:57,364][19170] Avg episode reward: [(0, '649.140'), (1, '732.690')] [2023-12-27 14:25:57,711][20381] Updated weights for policy 1, policy_version 1266816 (0.0007) [2023-12-27 14:25:57,761][20381] Updated weights for policy 1, policy_version 1266826 (0.0007) [2023-12-27 14:25:57,807][20381] Updated weights for policy 1, policy_version 1266836 (0.0006) [2023-12-27 14:25:57,950][20347] Updated weights for policy 0, policy_version 1267859 (0.0007) [2023-12-27 14:25:58,000][20347] Updated weights for policy 0, policy_version 1267869 (0.0006) [2023-12-27 14:25:58,052][20347] Updated weights for policy 0, policy_version 1267879 (0.0010) [2023-12-27 14:25:58,513][20381] Updated weights for policy 1, policy_version 1266846 (0.0009) [2023-12-27 14:25:58,575][20381] Updated weights for policy 1, policy_version 1266856 (0.0011) [2023-12-27 14:25:58,632][20381] Updated weights for policy 1, policy_version 1266866 (0.0009) [2023-12-27 14:25:58,796][20347] Updated weights for policy 0, policy_version 1267889 (0.0007) [2023-12-27 14:25:58,862][20347] Updated weights for policy 0, policy_version 1267899 (0.0008) [2023-12-27 14:25:58,912][20347] Updated weights for policy 0, policy_version 1267909 (0.0008) [2023-12-27 14:25:58,966][20347] Updated weights for policy 0, policy_version 1267919 (0.0008) [2023-12-27 14:25:59,407][20381] Updated weights for policy 1, policy_version 1266876 (0.0006) [2023-12-27 14:25:59,463][20381] Updated weights for policy 1, policy_version 1266886 (0.0006) [2023-12-27 14:25:59,526][20381] Updated weights for policy 1, policy_version 1266896 (0.0005) [2023-12-27 14:25:59,692][20347] Updated weights for policy 0, policy_version 1267929 (0.0010) [2023-12-27 14:25:59,751][20347] Updated weights for policy 0, policy_version 1267939 (0.0005) [2023-12-27 14:25:59,806][20347] Updated weights for policy 0, policy_version 1267949 (0.0005) [2023-12-27 14:26:00,135][20381] Updated weights for policy 1, policy_version 1266906 (0.0006) [2023-12-27 14:26:00,193][20381] Updated weights for policy 1, policy_version 1266916 (0.0008) [2023-12-27 14:26:00,244][20381] Updated weights for policy 1, policy_version 1266926 (0.0006) [2023-12-27 14:26:00,303][20381] Updated weights for policy 1, policy_version 1266936 (0.0005) [2023-12-27 14:26:00,477][20347] Updated weights for policy 0, policy_version 1267959 (0.0006) [2023-12-27 14:26:00,507][20291] KL-divergence is very high: 104.3263 [2023-12-27 14:26:00,527][20291] KL-divergence is very high: 142.2010 [2023-12-27 14:26:00,527][20347] Updated weights for policy 0, policy_version 1267969 (0.0010) [2023-12-27 14:26:00,578][20347] Updated weights for policy 0, policy_version 1267979 (0.0010) [2023-12-27 14:26:00,838][20381] Updated weights for policy 1, policy_version 1266946 (0.0005) [2023-12-27 14:26:00,885][20381] Updated weights for policy 1, policy_version 1266956 (0.0007) [2023-12-27 14:26:00,932][20381] Updated weights for policy 1, policy_version 1266966 (0.0007) [2023-12-27 14:26:01,320][20347] Updated weights for policy 0, policy_version 1267989 (0.0008) [2023-12-27 14:26:01,388][20347] Updated weights for policy 0, policy_version 1267999 (0.0009) [2023-12-27 14:26:01,440][20347] Updated weights for policy 0, policy_version 1268009 (0.0008) [2023-12-27 14:26:01,687][20381] Updated weights for policy 1, policy_version 1266976 (0.0008) [2023-12-27 14:26:01,752][20381] Updated weights for policy 1, policy_version 1266986 (0.0007) [2023-12-27 14:26:01,807][20381] Updated weights for policy 1, policy_version 1266996 (0.0007) [2023-12-27 14:26:02,168][20347] Updated weights for policy 0, policy_version 1268019 (0.0007) [2023-12-27 14:26:02,225][20347] Updated weights for policy 0, policy_version 1268029 (0.0008) [2023-12-27 14:26:02,288][20347] Updated weights for policy 0, policy_version 1268039 (0.0008) [2023-12-27 14:26:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 649060352. Throughput: 0: 10310.7, 1: 10345.6. Samples: 649069340. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:02,363][19170] Avg episode reward: [(0, '687.530'), (1, '777.800')] [2023-12-27 14:26:02,452][20381] Updated weights for policy 1, policy_version 1267006 (0.0008) [2023-12-27 14:26:02,514][20381] Updated weights for policy 1, policy_version 1267016 (0.0007) [2023-12-27 14:26:02,575][20381] Updated weights for policy 1, policy_version 1267026 (0.0008) [2023-12-27 14:26:02,928][20347] Updated weights for policy 0, policy_version 1268049 (0.0008) [2023-12-27 14:26:02,987][20347] Updated weights for policy 0, policy_version 1268059 (0.0007) [2023-12-27 14:26:03,047][20347] Updated weights for policy 0, policy_version 1268069 (0.0008) [2023-12-27 14:26:03,106][20347] Updated weights for policy 0, policy_version 1268079 (0.0008) [2023-12-27 14:26:03,271][20381] Updated weights for policy 1, policy_version 1267036 (0.0008) [2023-12-27 14:26:03,320][20381] Updated weights for policy 1, policy_version 1267046 (0.0008) [2023-12-27 14:26:03,371][20381] Updated weights for policy 1, policy_version 1267056 (0.0007) [2023-12-27 14:26:03,751][20291] KL-divergence is very high: 110.8416 [2023-12-27 14:26:03,798][20347] Updated weights for policy 0, policy_version 1268089 (0.0008) [2023-12-27 14:26:03,866][20347] Updated weights for policy 0, policy_version 1268099 (0.0008) [2023-12-27 14:26:03,922][20347] Updated weights for policy 0, policy_version 1268109 (0.0008) [2023-12-27 14:26:04,018][20381] Updated weights for policy 1, policy_version 1267066 (0.0007) [2023-12-27 14:26:04,074][20381] Updated weights for policy 1, policy_version 1267076 (0.0009) [2023-12-27 14:26:04,137][20381] Updated weights for policy 1, policy_version 1267086 (0.0009) [2023-12-27 14:26:04,196][20381] Updated weights for policy 1, policy_version 1267096 (0.0009) [2023-12-27 14:26:04,628][20347] Updated weights for policy 0, policy_version 1268119 (0.0008) [2023-12-27 14:26:04,688][20347] Updated weights for policy 0, policy_version 1268129 (0.0008) [2023-12-27 14:26:04,748][20347] Updated weights for policy 0, policy_version 1268139 (0.0008) [2023-12-27 14:26:04,958][20381] Updated weights for policy 1, policy_version 1267106 (0.0008) [2023-12-27 14:26:05,006][20381] Updated weights for policy 1, policy_version 1267116 (0.0010) [2023-12-27 14:26:05,069][20381] Updated weights for policy 1, policy_version 1267126 (0.0009) [2023-12-27 14:26:05,398][20347] Updated weights for policy 0, policy_version 1268149 (0.0007) [2023-12-27 14:26:05,452][20347] Updated weights for policy 0, policy_version 1268159 (0.0005) [2023-12-27 14:26:05,508][20347] Updated weights for policy 0, policy_version 1268169 (0.0005) [2023-12-27 14:26:05,675][20381] Updated weights for policy 1, policy_version 1267136 (0.0005) [2023-12-27 14:26:05,729][20381] Updated weights for policy 1, policy_version 1267146 (0.0005) [2023-12-27 14:26:05,780][20381] Updated weights for policy 1, policy_version 1267156 (0.0006) [2023-12-27 14:26:06,118][20347] Updated weights for policy 0, policy_version 1268179 (0.0006) [2023-12-27 14:26:06,184][20347] Updated weights for policy 0, policy_version 1268189 (0.0008) [2023-12-27 14:26:06,246][20347] Updated weights for policy 0, policy_version 1268199 (0.0008) [2023-12-27 14:26:06,388][20381] Updated weights for policy 1, policy_version 1267166 (0.0008) [2023-12-27 14:26:06,452][20381] Updated weights for policy 1, policy_version 1267176 (0.0008) [2023-12-27 14:26:06,512][20381] Updated weights for policy 1, policy_version 1267186 (0.0008) [2023-12-27 14:26:06,949][20347] Updated weights for policy 0, policy_version 1268209 (0.0007) [2023-12-27 14:26:07,009][20347] Updated weights for policy 0, policy_version 1268219 (0.0006) [2023-12-27 14:26:07,021][20291] KL-divergence is very high: 106.6850 [2023-12-27 14:26:07,069][20347] Updated weights for policy 0, policy_version 1268229 (0.0007) [2023-12-27 14:26:07,127][20347] Updated weights for policy 0, policy_version 1268239 (0.0007) [2023-12-27 14:26:07,185][20381] Updated weights for policy 1, policy_version 1267196 (0.0009) [2023-12-27 14:26:07,241][20381] Updated weights for policy 1, policy_version 1267206 (0.0011) [2023-12-27 14:26:07,291][20381] Updated weights for policy 1, policy_version 1267216 (0.0010) [2023-12-27 14:26:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 649166848. Throughput: 0: 10287.3, 1: 10319.2. Samples: 649128856. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:07,364][19170] Avg episode reward: [(0, '647.840'), (1, '691.650')] [2023-12-27 14:26:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001268240_324730880.pth... [2023-12-27 14:26:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001267224_324435968.pth... [2023-12-27 14:26:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001266008_324124672.pth [2023-12-27 14:26:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001267056_324427776.pth [2023-12-27 14:26:07,756][20291] KL-divergence is very high: 218.0835 [2023-12-27 14:26:07,769][20291] KL-divergence is very high: 124.3366 [2023-12-27 14:26:07,799][20347] Updated weights for policy 0, policy_version 1268249 (0.0006) [2023-12-27 14:26:07,800][20291] KL-divergence is very high: 171.3226 [2023-12-27 14:26:07,850][20347] Updated weights for policy 0, policy_version 1268259 (0.0007) [2023-12-27 14:26:07,905][20347] Updated weights for policy 0, policy_version 1268269 (0.0006) [2023-12-27 14:26:07,985][20381] Updated weights for policy 1, policy_version 1267226 (0.0010) [2023-12-27 14:26:08,034][20381] Updated weights for policy 1, policy_version 1267236 (0.0010) [2023-12-27 14:26:08,088][20381] Updated weights for policy 1, policy_version 1267246 (0.0010) [2023-12-27 14:26:08,150][20381] Updated weights for policy 1, policy_version 1267256 (0.0010) [2023-12-27 14:26:08,555][20347] Updated weights for policy 0, policy_version 1268279 (0.0007) [2023-12-27 14:26:08,610][20347] Updated weights for policy 0, policy_version 1268289 (0.0007) [2023-12-27 14:26:08,664][20347] Updated weights for policy 0, policy_version 1268299 (0.0006) [2023-12-27 14:26:08,878][20381] Updated weights for policy 1, policy_version 1267266 (0.0007) [2023-12-27 14:26:08,938][20381] Updated weights for policy 1, policy_version 1267276 (0.0008) [2023-12-27 14:26:08,997][20381] Updated weights for policy 1, policy_version 1267286 (0.0008) [2023-12-27 14:26:09,301][20347] Updated weights for policy 0, policy_version 1268309 (0.0006) [2023-12-27 14:26:09,356][20347] Updated weights for policy 0, policy_version 1268319 (0.0007) [2023-12-27 14:26:09,419][20347] Updated weights for policy 0, policy_version 1268329 (0.0007) [2023-12-27 14:26:09,665][20381] Updated weights for policy 1, policy_version 1267296 (0.0006) [2023-12-27 14:26:09,726][20381] Updated weights for policy 1, policy_version 1267306 (0.0006) [2023-12-27 14:26:09,789][20381] Updated weights for policy 1, policy_version 1267316 (0.0010) [2023-12-27 14:26:10,166][20347] Updated weights for policy 0, policy_version 1268339 (0.0007) [2023-12-27 14:26:10,227][20347] Updated weights for policy 0, policy_version 1268349 (0.0006) [2023-12-27 14:26:10,286][20347] Updated weights for policy 0, policy_version 1268359 (0.0005) [2023-12-27 14:26:10,402][20381] Updated weights for policy 1, policy_version 1267326 (0.0010) [2023-12-27 14:26:10,462][20381] Updated weights for policy 1, policy_version 1267336 (0.0011) [2023-12-27 14:26:10,522][20381] Updated weights for policy 1, policy_version 1267346 (0.0011) [2023-12-27 14:26:10,855][20347] Updated weights for policy 0, policy_version 1268369 (0.0005) [2023-12-27 14:26:10,915][20347] Updated weights for policy 0, policy_version 1268379 (0.0010) [2023-12-27 14:26:10,966][20347] Updated weights for policy 0, policy_version 1268389 (0.0010) [2023-12-27 14:26:11,012][20347] Updated weights for policy 0, policy_version 1268399 (0.0010) [2023-12-27 14:26:11,252][20381] Updated weights for policy 1, policy_version 1267356 (0.0010) [2023-12-27 14:26:11,313][20381] Updated weights for policy 1, policy_version 1267366 (0.0006) [2023-12-27 14:26:11,381][20381] Updated weights for policy 1, policy_version 1267376 (0.0008) [2023-12-27 14:26:11,815][20347] Updated weights for policy 0, policy_version 1268409 (0.0008) [2023-12-27 14:26:11,877][20347] Updated weights for policy 0, policy_version 1268419 (0.0009) [2023-12-27 14:26:11,938][20347] Updated weights for policy 0, policy_version 1268429 (0.0008) [2023-12-27 14:26:12,041][20381] Updated weights for policy 1, policy_version 1267386 (0.0007) [2023-12-27 14:26:12,110][20381] Updated weights for policy 1, policy_version 1267396 (0.0006) [2023-12-27 14:26:12,174][20381] Updated weights for policy 1, policy_version 1267406 (0.0008) [2023-12-27 14:26:12,236][20381] Updated weights for policy 1, policy_version 1267416 (0.0008) [2023-12-27 14:26:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 649265152. Throughput: 0: 10325.6, 1: 10366.0. Samples: 649254572. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:12,363][19170] Avg episode reward: [(0, '716.890'), (1, '729.440')] [2023-12-27 14:26:12,644][20347] Updated weights for policy 0, policy_version 1268439 (0.0008) [2023-12-27 14:26:12,694][20347] Updated weights for policy 0, policy_version 1268449 (0.0008) [2023-12-27 14:26:12,761][20347] Updated weights for policy 0, policy_version 1268459 (0.0008) [2023-12-27 14:26:12,856][20381] Updated weights for policy 1, policy_version 1267426 (0.0008) [2023-12-27 14:26:12,910][20381] Updated weights for policy 1, policy_version 1267436 (0.0011) [2023-12-27 14:26:12,959][20381] Updated weights for policy 1, policy_version 1267446 (0.0010) [2023-12-27 14:26:13,373][20347] Updated weights for policy 0, policy_version 1268469 (0.0008) [2023-12-27 14:26:13,422][20347] Updated weights for policy 0, policy_version 1268479 (0.0007) [2023-12-27 14:26:13,474][20347] Updated weights for policy 0, policy_version 1268489 (0.0008) [2023-12-27 14:26:13,698][20381] Updated weights for policy 1, policy_version 1267456 (0.0009) [2023-12-27 14:26:13,747][20381] Updated weights for policy 1, policy_version 1267466 (0.0008) [2023-12-27 14:26:13,796][20381] Updated weights for policy 1, policy_version 1267476 (0.0007) [2023-12-27 14:26:14,077][20347] Updated weights for policy 0, policy_version 1268499 (0.0008) [2023-12-27 14:26:14,134][20347] Updated weights for policy 0, policy_version 1268509 (0.0008) [2023-12-27 14:26:14,202][20347] Updated weights for policy 0, policy_version 1268519 (0.0008) [2023-12-27 14:26:14,459][20381] Updated weights for policy 1, policy_version 1267486 (0.0006) [2023-12-27 14:26:14,526][20381] Updated weights for policy 1, policy_version 1267496 (0.0007) [2023-12-27 14:26:14,595][20381] Updated weights for policy 1, policy_version 1267506 (0.0007) [2023-12-27 14:26:14,834][20347] Updated weights for policy 0, policy_version 1268529 (0.0006) [2023-12-27 14:26:14,897][20347] Updated weights for policy 0, policy_version 1268539 (0.0009) [2023-12-27 14:26:14,961][20347] Updated weights for policy 0, policy_version 1268549 (0.0007) [2023-12-27 14:26:15,022][20347] Updated weights for policy 0, policy_version 1268559 (0.0007) [2023-12-27 14:26:15,196][20381] Updated weights for policy 1, policy_version 1267516 (0.0008) [2023-12-27 14:26:15,256][20381] Updated weights for policy 1, policy_version 1267526 (0.0011) [2023-12-27 14:26:15,314][20381] Updated weights for policy 1, policy_version 1267536 (0.0010) [2023-12-27 14:26:15,733][20347] Updated weights for policy 0, policy_version 1268569 (0.0008) [2023-12-27 14:26:15,789][20347] Updated weights for policy 0, policy_version 1268579 (0.0007) [2023-12-27 14:26:15,842][20347] Updated weights for policy 0, policy_version 1268589 (0.0009) [2023-12-27 14:26:15,893][20381] Updated weights for policy 1, policy_version 1267546 (0.0010) [2023-12-27 14:26:15,944][20381] Updated weights for policy 1, policy_version 1267556 (0.0005) [2023-12-27 14:26:16,005][20381] Updated weights for policy 1, policy_version 1267566 (0.0009) [2023-12-27 14:26:16,055][20381] Updated weights for policy 1, policy_version 1267576 (0.0009) [2023-12-27 14:26:16,510][20347] Updated weights for policy 0, policy_version 1268599 (0.0007) [2023-12-27 14:26:16,563][20347] Updated weights for policy 0, policy_version 1268609 (0.0005) [2023-12-27 14:26:16,614][20381] Updated weights for policy 1, policy_version 1267586 (0.0011) [2023-12-27 14:26:16,620][20347] Updated weights for policy 0, policy_version 1268619 (0.0006) [2023-12-27 14:26:16,669][20381] Updated weights for policy 1, policy_version 1267596 (0.0010) [2023-12-27 14:26:16,733][20381] Updated weights for policy 1, policy_version 1267606 (0.0007) [2023-12-27 14:26:17,222][20347] Updated weights for policy 0, policy_version 1268629 (0.0007) [2023-12-27 14:26:17,273][20347] Updated weights for policy 0, policy_version 1268639 (0.0007) [2023-12-27 14:26:17,323][20347] Updated weights for policy 0, policy_version 1268649 (0.0008) [2023-12-27 14:26:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 649371648. Throughput: 0: 10290.2, 1: 10420.2. Samples: 649380636. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:17,363][19170] Avg episode reward: [(0, '804.560'), (1, '738.740')] [2023-12-27 14:26:17,394][20381] Updated weights for policy 1, policy_version 1267616 (0.0008) [2023-12-27 14:26:17,440][20381] Updated weights for policy 1, policy_version 1267626 (0.0008) [2023-12-27 14:26:17,495][20381] Updated weights for policy 1, policy_version 1267636 (0.0007) [2023-12-27 14:26:17,990][20347] Updated weights for policy 0, policy_version 1268659 (0.0007) [2023-12-27 14:26:18,052][20347] Updated weights for policy 0, policy_version 1268669 (0.0007) [2023-12-27 14:26:18,114][20347] Updated weights for policy 0, policy_version 1268679 (0.0007) [2023-12-27 14:26:18,200][20381] Updated weights for policy 1, policy_version 1267646 (0.0008) [2023-12-27 14:26:18,256][20381] Updated weights for policy 1, policy_version 1267656 (0.0009) [2023-12-27 14:26:18,317][20381] Updated weights for policy 1, policy_version 1267666 (0.0008) [2023-12-27 14:26:18,772][20347] Updated weights for policy 0, policy_version 1268689 (0.0006) [2023-12-27 14:26:18,837][20347] Updated weights for policy 0, policy_version 1268699 (0.0007) [2023-12-27 14:26:18,904][20347] Updated weights for policy 0, policy_version 1268709 (0.0007) [2023-12-27 14:26:18,965][20347] Updated weights for policy 0, policy_version 1268719 (0.0008) [2023-12-27 14:26:19,013][20381] Updated weights for policy 1, policy_version 1267676 (0.0009) [2023-12-27 14:26:19,077][20381] Updated weights for policy 1, policy_version 1267686 (0.0008) [2023-12-27 14:26:19,139][20381] Updated weights for policy 1, policy_version 1267696 (0.0008) [2023-12-27 14:26:19,702][20347] Updated weights for policy 0, policy_version 1268729 (0.0007) [2023-12-27 14:26:19,766][20347] Updated weights for policy 0, policy_version 1268739 (0.0008) [2023-12-27 14:26:19,828][20347] Updated weights for policy 0, policy_version 1268749 (0.0008) [2023-12-27 14:26:19,885][20381] Updated weights for policy 1, policy_version 1267706 (0.0009) [2023-12-27 14:26:19,949][20381] Updated weights for policy 1, policy_version 1267716 (0.0011) [2023-12-27 14:26:19,995][20381] Updated weights for policy 1, policy_version 1267726 (0.0011) [2023-12-27 14:26:20,052][20381] Updated weights for policy 1, policy_version 1267736 (0.0011) [2023-12-27 14:26:20,550][20347] Updated weights for policy 0, policy_version 1268759 (0.0006) [2023-12-27 14:26:20,617][20347] Updated weights for policy 0, policy_version 1268769 (0.0008) [2023-12-27 14:26:20,664][20347] Updated weights for policy 0, policy_version 1268779 (0.0007) [2023-12-27 14:26:20,794][20381] Updated weights for policy 1, policy_version 1267746 (0.0008) [2023-12-27 14:26:20,850][20381] Updated weights for policy 1, policy_version 1267756 (0.0008) [2023-12-27 14:26:20,905][20381] Updated weights for policy 1, policy_version 1267766 (0.0007) [2023-12-27 14:26:21,396][20347] Updated weights for policy 0, policy_version 1268789 (0.0008) [2023-12-27 14:26:21,463][20347] Updated weights for policy 0, policy_version 1268799 (0.0009) [2023-12-27 14:26:21,524][20347] Updated weights for policy 0, policy_version 1268809 (0.0009) [2023-12-27 14:26:21,619][20381] Updated weights for policy 1, policy_version 1267776 (0.0010) [2023-12-27 14:26:21,685][20381] Updated weights for policy 1, policy_version 1267786 (0.0009) [2023-12-27 14:26:21,757][20381] Updated weights for policy 1, policy_version 1267796 (0.0008) [2023-12-27 14:26:22,171][20347] Updated weights for policy 0, policy_version 1268819 (0.0007) [2023-12-27 14:26:22,229][20347] Updated weights for policy 0, policy_version 1268829 (0.0007) [2023-12-27 14:26:22,290][20347] Updated weights for policy 0, policy_version 1268839 (0.0007) [2023-12-27 14:26:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 649469952. Throughput: 0: 10258.1, 1: 10392.6. Samples: 649438656. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:22,363][19170] Avg episode reward: [(0, '765.130'), (1, '719.290')] [2023-12-27 14:26:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001267800_324583424.pth... [2023-12-27 14:26:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001268848_324886528.pth... [2023-12-27 14:26:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001266584_324272128.pth [2023-12-27 14:26:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001267632_324575232.pth [2023-12-27 14:26:22,531][20381] Updated weights for policy 1, policy_version 1267806 (0.0009) [2023-12-27 14:26:22,581][20381] Updated weights for policy 1, policy_version 1267816 (0.0008) [2023-12-27 14:26:22,640][20381] Updated weights for policy 1, policy_version 1267826 (0.0008) [2023-12-27 14:26:22,989][20347] Updated weights for policy 0, policy_version 1268849 (0.0007) [2023-12-27 14:26:23,048][20347] Updated weights for policy 0, policy_version 1268859 (0.0008) [2023-12-27 14:26:23,106][20347] Updated weights for policy 0, policy_version 1268869 (0.0008) [2023-12-27 14:26:23,160][20347] Updated weights for policy 0, policy_version 1268879 (0.0006) [2023-12-27 14:26:23,327][20381] Updated weights for policy 1, policy_version 1267836 (0.0007) [2023-12-27 14:26:23,392][20381] Updated weights for policy 1, policy_version 1267846 (0.0006) [2023-12-27 14:26:23,451][20381] Updated weights for policy 1, policy_version 1267856 (0.0008) [2023-12-27 14:26:23,798][20347] Updated weights for policy 0, policy_version 1268889 (0.0005) [2023-12-27 14:26:23,844][20347] Updated weights for policy 0, policy_version 1268899 (0.0005) [2023-12-27 14:26:23,901][20347] Updated weights for policy 0, policy_version 1268909 (0.0005) [2023-12-27 14:26:24,048][20381] Updated weights for policy 1, policy_version 1267866 (0.0009) [2023-12-27 14:26:24,097][20381] Updated weights for policy 1, policy_version 1267876 (0.0005) [2023-12-27 14:26:24,149][20381] Updated weights for policy 1, policy_version 1267886 (0.0005) [2023-12-27 14:26:24,196][20381] Updated weights for policy 1, policy_version 1267896 (0.0005) [2023-12-27 14:26:24,469][20347] Updated weights for policy 0, policy_version 1268919 (0.0006) [2023-12-27 14:26:24,531][20347] Updated weights for policy 0, policy_version 1268929 (0.0005) [2023-12-27 14:26:24,595][20347] Updated weights for policy 0, policy_version 1268939 (0.0005) [2023-12-27 14:26:24,870][20381] Updated weights for policy 1, policy_version 1267906 (0.0008) [2023-12-27 14:26:24,912][20381] Updated weights for policy 1, policy_version 1267916 (0.0007) [2023-12-27 14:26:24,960][20381] Updated weights for policy 1, policy_version 1267926 (0.0006) [2023-12-27 14:26:25,192][20347] Updated weights for policy 0, policy_version 1268949 (0.0008) [2023-12-27 14:26:25,237][20347] Updated weights for policy 0, policy_version 1268959 (0.0010) [2023-12-27 14:26:25,285][20347] Updated weights for policy 0, policy_version 1268969 (0.0010) [2023-12-27 14:26:25,594][20381] Updated weights for policy 1, policy_version 1267936 (0.0010) [2023-12-27 14:26:25,640][20381] Updated weights for policy 1, policy_version 1267946 (0.0010) [2023-12-27 14:26:25,692][20381] Updated weights for policy 1, policy_version 1267956 (0.0010) [2023-12-27 14:26:25,907][20347] Updated weights for policy 0, policy_version 1268979 (0.0006) [2023-12-27 14:26:25,958][20347] Updated weights for policy 0, policy_version 1268989 (0.0007) [2023-12-27 14:26:26,012][20347] Updated weights for policy 0, policy_version 1268999 (0.0007) [2023-12-27 14:26:26,399][20381] Updated weights for policy 1, policy_version 1267966 (0.0008) [2023-12-27 14:26:26,458][20381] Updated weights for policy 1, policy_version 1267976 (0.0005) [2023-12-27 14:26:26,516][20381] Updated weights for policy 1, policy_version 1267986 (0.0005) [2023-12-27 14:26:26,700][20347] Updated weights for policy 0, policy_version 1269009 (0.0008) [2023-12-27 14:26:26,746][20347] Updated weights for policy 0, policy_version 1269019 (0.0008) [2023-12-27 14:26:26,796][20347] Updated weights for policy 0, policy_version 1269029 (0.0006) [2023-12-27 14:26:26,842][20347] Updated weights for policy 0, policy_version 1269039 (0.0007) [2023-12-27 14:26:27,045][20381] Updated weights for policy 1, policy_version 1267996 (0.0005) [2023-12-27 14:26:27,095][20381] Updated weights for policy 1, policy_version 1268006 (0.0005) [2023-12-27 14:26:27,151][20381] Updated weights for policy 1, policy_version 1268016 (0.0005) [2023-12-27 14:26:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 649576448. Throughput: 0: 10291.0, 1: 10327.4. Samples: 649562932. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:27,363][19170] Avg episode reward: [(0, '716.860'), (1, '701.450')] [2023-12-27 14:26:27,603][20347] Updated weights for policy 0, policy_version 1269049 (0.0008) [2023-12-27 14:26:27,662][20347] Updated weights for policy 0, policy_version 1269059 (0.0007) [2023-12-27 14:26:27,718][20347] Updated weights for policy 0, policy_version 1269069 (0.0007) [2023-12-27 14:26:27,729][20381] Updated weights for policy 1, policy_version 1268026 (0.0006) [2023-12-27 14:26:27,785][20381] Updated weights for policy 1, policy_version 1268036 (0.0008) [2023-12-27 14:26:27,840][20381] Updated weights for policy 1, policy_version 1268046 (0.0007) [2023-12-27 14:26:27,887][20381] Updated weights for policy 1, policy_version 1268056 (0.0007) [2023-12-27 14:26:28,381][20347] Updated weights for policy 0, policy_version 1269079 (0.0006) [2023-12-27 14:26:28,437][20347] Updated weights for policy 0, policy_version 1269089 (0.0007) [2023-12-27 14:26:28,498][20347] Updated weights for policy 0, policy_version 1269099 (0.0008) [2023-12-27 14:26:28,580][20381] Updated weights for policy 1, policy_version 1268066 (0.0007) [2023-12-27 14:26:28,637][20381] Updated weights for policy 1, policy_version 1268076 (0.0007) [2023-12-27 14:26:28,692][20381] Updated weights for policy 1, policy_version 1268086 (0.0005) [2023-12-27 14:26:29,161][20347] Updated weights for policy 0, policy_version 1269109 (0.0010) [2023-12-27 14:26:29,215][20347] Updated weights for policy 0, policy_version 1269119 (0.0010) [2023-12-27 14:26:29,274][20347] Updated weights for policy 0, policy_version 1269129 (0.0008) [2023-12-27 14:26:29,334][20381] Updated weights for policy 1, policy_version 1268096 (0.0009) [2023-12-27 14:26:29,397][20381] Updated weights for policy 1, policy_version 1268106 (0.0010) [2023-12-27 14:26:29,446][20381] Updated weights for policy 1, policy_version 1268116 (0.0011) [2023-12-27 14:26:29,991][20347] Updated weights for policy 0, policy_version 1269139 (0.0008) [2023-12-27 14:26:30,058][20347] Updated weights for policy 0, policy_version 1269149 (0.0008) [2023-12-27 14:26:30,113][20347] Updated weights for policy 0, policy_version 1269159 (0.0007) [2023-12-27 14:26:30,146][20381] Updated weights for policy 1, policy_version 1268126 (0.0007) [2023-12-27 14:26:30,204][20381] Updated weights for policy 1, policy_version 1268136 (0.0007) [2023-12-27 14:26:30,263][20381] Updated weights for policy 1, policy_version 1268146 (0.0009) [2023-12-27 14:26:30,712][20347] Updated weights for policy 0, policy_version 1269169 (0.0006) [2023-12-27 14:26:30,767][20347] Updated weights for policy 0, policy_version 1269179 (0.0005) [2023-12-27 14:26:30,827][20347] Updated weights for policy 0, policy_version 1269189 (0.0006) [2023-12-27 14:26:30,874][20347] Updated weights for policy 0, policy_version 1269199 (0.0005) [2023-12-27 14:26:30,941][20381] Updated weights for policy 1, policy_version 1268156 (0.0007) [2023-12-27 14:26:30,995][20381] Updated weights for policy 1, policy_version 1268166 (0.0007) [2023-12-27 14:26:31,060][20381] Updated weights for policy 1, policy_version 1268176 (0.0009) [2023-12-27 14:26:31,518][20347] Updated weights for policy 0, policy_version 1269209 (0.0006) [2023-12-27 14:26:31,573][20347] Updated weights for policy 0, policy_version 1269219 (0.0005) [2023-12-27 14:26:31,628][20347] Updated weights for policy 0, policy_version 1269229 (0.0008) [2023-12-27 14:26:31,752][20381] Updated weights for policy 1, policy_version 1268186 (0.0006) [2023-12-27 14:26:31,811][20381] Updated weights for policy 1, policy_version 1268196 (0.0008) [2023-12-27 14:26:31,867][20381] Updated weights for policy 1, policy_version 1268206 (0.0008) [2023-12-27 14:26:31,923][20381] Updated weights for policy 1, policy_version 1268216 (0.0008) [2023-12-27 14:26:32,319][20347] Updated weights for policy 0, policy_version 1269239 (0.0008) [2023-12-27 14:26:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 649674752. Throughput: 0: 10247.5, 1: 10352.6. Samples: 649687736. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:32,363][19170] Avg episode reward: [(0, '677.530'), (1, '673.420')] [2023-12-27 14:26:32,385][20347] Updated weights for policy 0, policy_version 1269249 (0.0008) [2023-12-27 14:26:32,447][20347] Updated weights for policy 0, policy_version 1269259 (0.0008) [2023-12-27 14:26:32,644][20381] Updated weights for policy 1, policy_version 1268226 (0.0007) [2023-12-27 14:26:32,698][20381] Updated weights for policy 1, policy_version 1268236 (0.0007) [2023-12-27 14:26:32,753][20381] Updated weights for policy 1, policy_version 1268246 (0.0008) [2023-12-27 14:26:33,097][20347] Updated weights for policy 0, policy_version 1269269 (0.0007) [2023-12-27 14:26:33,142][20347] Updated weights for policy 0, policy_version 1269279 (0.0005) [2023-12-27 14:26:33,199][20347] Updated weights for policy 0, policy_version 1269289 (0.0008) [2023-12-27 14:26:33,466][20381] Updated weights for policy 1, policy_version 1268256 (0.0008) [2023-12-27 14:26:33,522][20381] Updated weights for policy 1, policy_version 1268266 (0.0006) [2023-12-27 14:26:33,569][20381] Updated weights for policy 1, policy_version 1268276 (0.0005) [2023-12-27 14:26:33,755][20347] Updated weights for policy 0, policy_version 1269299 (0.0010) [2023-12-27 14:26:33,818][20347] Updated weights for policy 0, policy_version 1269309 (0.0010) [2023-12-27 14:26:33,876][20347] Updated weights for policy 0, policy_version 1269319 (0.0008) [2023-12-27 14:26:34,312][20381] Updated weights for policy 1, policy_version 1268286 (0.0007) [2023-12-27 14:26:34,371][20381] Updated weights for policy 1, policy_version 1268296 (0.0008) [2023-12-27 14:26:34,424][20381] Updated weights for policy 1, policy_version 1268306 (0.0008) [2023-12-27 14:26:34,480][20347] Updated weights for policy 0, policy_version 1269329 (0.0006) [2023-12-27 14:26:34,540][20347] Updated weights for policy 0, policy_version 1269339 (0.0010) [2023-12-27 14:26:34,603][20347] Updated weights for policy 0, policy_version 1269349 (0.0011) [2023-12-27 14:26:34,671][20347] Updated weights for policy 0, policy_version 1269359 (0.0010) [2023-12-27 14:26:35,109][20381] Updated weights for policy 1, policy_version 1268316 (0.0007) [2023-12-27 14:26:35,174][20381] Updated weights for policy 1, policy_version 1268326 (0.0007) [2023-12-27 14:26:35,230][20381] Updated weights for policy 1, policy_version 1268336 (0.0007) [2023-12-27 14:26:35,406][20347] Updated weights for policy 0, policy_version 1269369 (0.0006) [2023-12-27 14:26:35,455][20347] Updated weights for policy 0, policy_version 1269379 (0.0007) [2023-12-27 14:26:35,517][20347] Updated weights for policy 0, policy_version 1269389 (0.0007) [2023-12-27 14:26:35,956][20381] Updated weights for policy 1, policy_version 1268346 (0.0007) [2023-12-27 14:26:36,009][20381] Updated weights for policy 1, policy_version 1268356 (0.0009) [2023-12-27 14:26:36,063][20381] Updated weights for policy 1, policy_version 1268366 (0.0010) [2023-12-27 14:26:36,121][20381] Updated weights for policy 1, policy_version 1268376 (0.0010) [2023-12-27 14:26:36,212][20347] Updated weights for policy 0, policy_version 1269399 (0.0009) [2023-12-27 14:26:36,279][20347] Updated weights for policy 0, policy_version 1269409 (0.0011) [2023-12-27 14:26:36,345][20347] Updated weights for policy 0, policy_version 1269419 (0.0011) [2023-12-27 14:26:36,803][20381] Updated weights for policy 1, policy_version 1268386 (0.0010) [2023-12-27 14:26:36,856][20381] Updated weights for policy 1, policy_version 1268396 (0.0006) [2023-12-27 14:26:36,918][20381] Updated weights for policy 1, policy_version 1268406 (0.0005) [2023-12-27 14:26:37,017][20347] Updated weights for policy 0, policy_version 1269429 (0.0010) [2023-12-27 14:26:37,075][20347] Updated weights for policy 0, policy_version 1269439 (0.0005) [2023-12-27 14:26:37,142][20347] Updated weights for policy 0, policy_version 1269449 (0.0005) [2023-12-27 14:26:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 649781248. Throughput: 0: 10300.7, 1: 10336.9. Samples: 649749792. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:37,363][19170] Avg episode reward: [(0, '794.860'), (1, '672.530')] [2023-12-27 14:26:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001269456_325042176.pth... [2023-12-27 14:26:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001268408_324739072.pth... [2023-12-27 14:26:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001267224_324435968.pth [2023-12-27 14:26:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001268240_324730880.pth [2023-12-27 14:26:37,584][20381] Updated weights for policy 1, policy_version 1268416 (0.0007) [2023-12-27 14:26:37,641][20381] Updated weights for policy 1, policy_version 1268426 (0.0008) [2023-12-27 14:26:37,702][20381] Updated weights for policy 1, policy_version 1268436 (0.0008) [2023-12-27 14:26:37,770][20347] Updated weights for policy 0, policy_version 1269459 (0.0009) [2023-12-27 14:26:37,826][20347] Updated weights for policy 0, policy_version 1269469 (0.0007) [2023-12-27 14:26:37,887][20347] Updated weights for policy 0, policy_version 1269479 (0.0008) [2023-12-27 14:26:38,347][20381] Updated weights for policy 1, policy_version 1268446 (0.0009) [2023-12-27 14:26:38,400][20381] Updated weights for policy 1, policy_version 1268456 (0.0008) [2023-12-27 14:26:38,456][20381] Updated weights for policy 1, policy_version 1268466 (0.0007) [2023-12-27 14:26:38,603][20347] Updated weights for policy 0, policy_version 1269489 (0.0008) [2023-12-27 14:26:38,659][20347] Updated weights for policy 0, policy_version 1269499 (0.0005) [2023-12-27 14:26:38,718][20347] Updated weights for policy 0, policy_version 1269509 (0.0005) [2023-12-27 14:26:38,777][20347] Updated weights for policy 0, policy_version 1269519 (0.0005) [2023-12-27 14:26:39,163][20381] Updated weights for policy 1, policy_version 1268476 (0.0008) [2023-12-27 14:26:39,223][20381] Updated weights for policy 1, policy_version 1268486 (0.0008) [2023-12-27 14:26:39,285][20381] Updated weights for policy 1, policy_version 1268496 (0.0007) [2023-12-27 14:26:39,431][20347] Updated weights for policy 0, policy_version 1269529 (0.0007) [2023-12-27 14:26:39,499][20347] Updated weights for policy 0, policy_version 1269539 (0.0006) [2023-12-27 14:26:39,567][20347] Updated weights for policy 0, policy_version 1269549 (0.0007) [2023-12-27 14:26:39,983][20381] Updated weights for policy 1, policy_version 1268506 (0.0006) [2023-12-27 14:26:40,049][20381] Updated weights for policy 1, policy_version 1268516 (0.0008) [2023-12-27 14:26:40,109][20381] Updated weights for policy 1, policy_version 1268526 (0.0007) [2023-12-27 14:26:40,171][20381] Updated weights for policy 1, policy_version 1268536 (0.0011) [2023-12-27 14:26:40,245][20347] Updated weights for policy 0, policy_version 1269559 (0.0007) [2023-12-27 14:26:40,298][20347] Updated weights for policy 0, policy_version 1269569 (0.0008) [2023-12-27 14:26:40,353][20347] Updated weights for policy 0, policy_version 1269579 (0.0007) [2023-12-27 14:26:40,856][20381] Updated weights for policy 1, policy_version 1268546 (0.0005) [2023-12-27 14:26:40,913][20381] Updated weights for policy 1, policy_version 1268556 (0.0005) [2023-12-27 14:26:40,972][20381] Updated weights for policy 1, policy_version 1268566 (0.0005) [2023-12-27 14:26:41,087][20347] Updated weights for policy 0, policy_version 1269589 (0.0008) [2023-12-27 14:26:41,149][20347] Updated weights for policy 0, policy_version 1269599 (0.0009) [2023-12-27 14:26:41,210][20347] Updated weights for policy 0, policy_version 1269609 (0.0007) [2023-12-27 14:26:41,596][20381] Updated weights for policy 1, policy_version 1268576 (0.0005) [2023-12-27 14:26:41,666][20381] Updated weights for policy 1, policy_version 1268586 (0.0007) [2023-12-27 14:26:41,718][20381] Updated weights for policy 1, policy_version 1268596 (0.0008) [2023-12-27 14:26:41,969][20347] Updated weights for policy 0, policy_version 1269619 (0.0007) [2023-12-27 14:26:42,028][20347] Updated weights for policy 0, policy_version 1269629 (0.0008) [2023-12-27 14:26:42,090][20347] Updated weights for policy 0, policy_version 1269639 (0.0006) [2023-12-27 14:26:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 649879552. Throughput: 0: 10274.1, 1: 10269.4. Samples: 649871516. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:42,363][19170] Avg episode reward: [(0, '745.630'), (1, '656.530')] [2023-12-27 14:26:42,415][20381] Updated weights for policy 1, policy_version 1268606 (0.0009) [2023-12-27 14:26:42,474][20381] Updated weights for policy 1, policy_version 1268616 (0.0008) [2023-12-27 14:26:42,523][20381] Updated weights for policy 1, policy_version 1268626 (0.0007) [2023-12-27 14:26:42,776][20347] Updated weights for policy 0, policy_version 1269649 (0.0008) [2023-12-27 14:26:42,833][20347] Updated weights for policy 0, policy_version 1269659 (0.0010) [2023-12-27 14:26:42,887][20347] Updated weights for policy 0, policy_version 1269669 (0.0010) [2023-12-27 14:26:42,940][20347] Updated weights for policy 0, policy_version 1269679 (0.0010) [2023-12-27 14:26:43,214][20381] Updated weights for policy 1, policy_version 1268636 (0.0007) [2023-12-27 14:26:43,269][20381] Updated weights for policy 1, policy_version 1268646 (0.0005) [2023-12-27 14:26:43,335][20381] Updated weights for policy 1, policy_version 1268656 (0.0006) [2023-12-27 14:26:43,623][20347] Updated weights for policy 0, policy_version 1269689 (0.0008) [2023-12-27 14:26:43,682][20347] Updated weights for policy 0, policy_version 1269699 (0.0006) [2023-12-27 14:26:43,743][20347] Updated weights for policy 0, policy_version 1269709 (0.0005) [2023-12-27 14:26:43,914][20381] Updated weights for policy 1, policy_version 1268666 (0.0008) [2023-12-27 14:26:43,962][20381] Updated weights for policy 1, policy_version 1268676 (0.0005) [2023-12-27 14:26:44,008][20381] Updated weights for policy 1, policy_version 1268686 (0.0005) [2023-12-27 14:26:44,054][20381] Updated weights for policy 1, policy_version 1268696 (0.0005) [2023-12-27 14:26:44,306][20347] Updated weights for policy 0, policy_version 1269719 (0.0006) [2023-12-27 14:26:44,358][20347] Updated weights for policy 0, policy_version 1269729 (0.0010) [2023-12-27 14:26:44,413][20347] Updated weights for policy 0, policy_version 1269739 (0.0010) [2023-12-27 14:26:44,647][20381] Updated weights for policy 1, policy_version 1268706 (0.0005) [2023-12-27 14:26:44,700][20381] Updated weights for policy 1, policy_version 1268716 (0.0007) [2023-12-27 14:26:44,752][20381] Updated weights for policy 1, policy_version 1268726 (0.0007) [2023-12-27 14:26:45,052][20347] Updated weights for policy 0, policy_version 1269749 (0.0010) [2023-12-27 14:26:45,115][20347] Updated weights for policy 0, policy_version 1269759 (0.0010) [2023-12-27 14:26:45,174][20347] Updated weights for policy 0, policy_version 1269769 (0.0008) [2023-12-27 14:26:45,405][20381] Updated weights for policy 1, policy_version 1268736 (0.0008) [2023-12-27 14:26:45,452][20381] Updated weights for policy 1, policy_version 1268746 (0.0007) [2023-12-27 14:26:45,500][20381] Updated weights for policy 1, policy_version 1268756 (0.0007) [2023-12-27 14:26:45,826][20347] Updated weights for policy 0, policy_version 1269779 (0.0007) [2023-12-27 14:26:45,887][20347] Updated weights for policy 0, policy_version 1269789 (0.0010) [2023-12-27 14:26:45,938][20347] Updated weights for policy 0, policy_version 1269799 (0.0010) [2023-12-27 14:26:46,223][20381] Updated weights for policy 1, policy_version 1268766 (0.0007) [2023-12-27 14:26:46,277][20381] Updated weights for policy 1, policy_version 1268776 (0.0008) [2023-12-27 14:26:46,332][20381] Updated weights for policy 1, policy_version 1268786 (0.0006) [2023-12-27 14:26:46,609][20347] Updated weights for policy 0, policy_version 1269809 (0.0010) [2023-12-27 14:26:46,668][20347] Updated weights for policy 0, policy_version 1269819 (0.0005) [2023-12-27 14:26:46,721][20347] Updated weights for policy 0, policy_version 1269829 (0.0007) [2023-12-27 14:26:46,773][20347] Updated weights for policy 0, policy_version 1269839 (0.0010) [2023-12-27 14:26:46,953][20381] Updated weights for policy 1, policy_version 1268796 (0.0007) [2023-12-27 14:26:47,018][20381] Updated weights for policy 1, policy_version 1268806 (0.0010) [2023-12-27 14:26:47,076][20381] Updated weights for policy 1, policy_version 1268816 (0.0010) [2023-12-27 14:26:47,345][20347] Updated weights for policy 0, policy_version 1269849 (0.0010) [2023-12-27 14:26:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 649986048. Throughput: 0: 10324.6, 1: 10305.0. Samples: 649997672. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:47,363][19170] Avg episode reward: [(0, '795.390'), (1, '742.600')] [2023-12-27 14:26:47,398][20347] Updated weights for policy 0, policy_version 1269859 (0.0008) [2023-12-27 14:26:47,453][20347] Updated weights for policy 0, policy_version 1269869 (0.0005) [2023-12-27 14:26:47,657][20381] Updated weights for policy 1, policy_version 1268826 (0.0009) [2023-12-27 14:26:47,706][20381] Updated weights for policy 1, policy_version 1268836 (0.0005) [2023-12-27 14:26:47,769][20381] Updated weights for policy 1, policy_version 1268846 (0.0008) [2023-12-27 14:26:47,834][20381] Updated weights for policy 1, policy_version 1268856 (0.0009) [2023-12-27 14:26:48,046][20347] Updated weights for policy 0, policy_version 1269879 (0.0005) [2023-12-27 14:26:48,096][20347] Updated weights for policy 0, policy_version 1269889 (0.0005) [2023-12-27 14:26:48,150][20347] Updated weights for policy 0, policy_version 1269899 (0.0005) [2023-12-27 14:26:48,469][20381] Updated weights for policy 1, policy_version 1268866 (0.0006) [2023-12-27 14:26:48,533][20381] Updated weights for policy 1, policy_version 1268876 (0.0006) [2023-12-27 14:26:48,590][20381] Updated weights for policy 1, policy_version 1268886 (0.0006) [2023-12-27 14:26:48,800][20347] Updated weights for policy 0, policy_version 1269909 (0.0007) [2023-12-27 14:26:48,860][20347] Updated weights for policy 0, policy_version 1269919 (0.0008) [2023-12-27 14:26:48,916][20347] Updated weights for policy 0, policy_version 1269929 (0.0008) [2023-12-27 14:26:49,134][20381] Updated weights for policy 1, policy_version 1268896 (0.0005) [2023-12-27 14:26:49,181][20381] Updated weights for policy 1, policy_version 1268906 (0.0005) [2023-12-27 14:26:49,236][20381] Updated weights for policy 1, policy_version 1268916 (0.0006) [2023-12-27 14:26:49,668][20347] Updated weights for policy 0, policy_version 1269939 (0.0007) [2023-12-27 14:26:49,732][20347] Updated weights for policy 0, policy_version 1269949 (0.0007) [2023-12-27 14:26:49,803][20347] Updated weights for policy 0, policy_version 1269959 (0.0007) [2023-12-27 14:26:50,038][20381] Updated weights for policy 1, policy_version 1268926 (0.0007) [2023-12-27 14:26:50,106][20381] Updated weights for policy 1, policy_version 1268936 (0.0007) [2023-12-27 14:26:50,175][20381] Updated weights for policy 1, policy_version 1268946 (0.0008) [2023-12-27 14:26:50,524][20347] Updated weights for policy 0, policy_version 1269969 (0.0008) [2023-12-27 14:26:50,586][20347] Updated weights for policy 0, policy_version 1269979 (0.0011) [2023-12-27 14:26:50,639][20347] Updated weights for policy 0, policy_version 1269989 (0.0009) [2023-12-27 14:26:50,691][20347] Updated weights for policy 0, policy_version 1269999 (0.0010) [2023-12-27 14:26:50,893][20381] Updated weights for policy 1, policy_version 1268956 (0.0007) [2023-12-27 14:26:50,956][20381] Updated weights for policy 1, policy_version 1268966 (0.0008) [2023-12-27 14:26:51,014][20381] Updated weights for policy 1, policy_version 1268976 (0.0008) [2023-12-27 14:26:51,423][20347] Updated weights for policy 0, policy_version 1270009 (0.0009) [2023-12-27 14:26:51,487][20347] Updated weights for policy 0, policy_version 1270019 (0.0008) [2023-12-27 14:26:51,551][20347] Updated weights for policy 0, policy_version 1270029 (0.0008) [2023-12-27 14:26:51,709][20381] Updated weights for policy 1, policy_version 1268986 (0.0007) [2023-12-27 14:26:51,772][20381] Updated weights for policy 1, policy_version 1268996 (0.0009) [2023-12-27 14:26:51,839][20381] Updated weights for policy 1, policy_version 1269006 (0.0007) [2023-12-27 14:26:51,904][20381] Updated weights for policy 1, policy_version 1269016 (0.0008) [2023-12-27 14:26:52,169][20347] Updated weights for policy 0, policy_version 1270039 (0.0010) [2023-12-27 14:26:52,213][20291] KL-divergence is very high: 105.5562 [2023-12-27 14:26:52,224][20347] Updated weights for policy 0, policy_version 1270049 (0.0010) [2023-12-27 14:26:52,286][20347] Updated weights for policy 0, policy_version 1270059 (0.0007) [2023-12-27 14:26:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 650092544. Throughput: 0: 10342.3, 1: 10361.1. Samples: 650060504. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:52,363][19170] Avg episode reward: [(0, '795.430'), (1, '818.770')] [2023-12-27 14:26:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001269016_324894720.pth... [2023-12-27 14:26:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001270064_325197824.pth... [2023-12-27 14:26:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001267800_324583424.pth [2023-12-27 14:26:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001268848_324886528.pth [2023-12-27 14:26:52,564][20381] Updated weights for policy 1, policy_version 1269026 (0.0011) [2023-12-27 14:26:52,620][20381] Updated weights for policy 1, policy_version 1269036 (0.0009) [2023-12-27 14:26:52,679][20381] Updated weights for policy 1, policy_version 1269046 (0.0007) [2023-12-27 14:26:52,947][20347] Updated weights for policy 0, policy_version 1270069 (0.0008) [2023-12-27 14:26:53,002][20347] Updated weights for policy 0, policy_version 1270079 (0.0006) [2023-12-27 14:26:53,060][20347] Updated weights for policy 0, policy_version 1270089 (0.0007) [2023-12-27 14:26:53,305][20381] Updated weights for policy 1, policy_version 1269056 (0.0005) [2023-12-27 14:26:53,360][20381] Updated weights for policy 1, policy_version 1269066 (0.0005) [2023-12-27 14:26:53,413][20381] Updated weights for policy 1, policy_version 1269076 (0.0005) [2023-12-27 14:26:53,606][20347] Updated weights for policy 0, policy_version 1270099 (0.0006) [2023-12-27 14:26:53,657][20347] Updated weights for policy 0, policy_version 1270109 (0.0005) [2023-12-27 14:26:53,707][20347] Updated weights for policy 0, policy_version 1270119 (0.0005) [2023-12-27 14:26:54,063][20381] Updated weights for policy 1, policy_version 1269086 (0.0006) [2023-12-27 14:26:54,114][20381] Updated weights for policy 1, policy_version 1269096 (0.0006) [2023-12-27 14:26:54,166][20381] Updated weights for policy 1, policy_version 1269106 (0.0008) [2023-12-27 14:26:54,343][20347] Updated weights for policy 0, policy_version 1270129 (0.0007) [2023-12-27 14:26:54,395][20347] Updated weights for policy 0, policy_version 1270139 (0.0005) [2023-12-27 14:26:54,446][20347] Updated weights for policy 0, policy_version 1270149 (0.0007) [2023-12-27 14:26:54,494][20347] Updated weights for policy 0, policy_version 1270159 (0.0007) [2023-12-27 14:26:54,810][20381] Updated weights for policy 1, policy_version 1269116 (0.0007) [2023-12-27 14:26:54,859][20381] Updated weights for policy 1, policy_version 1269126 (0.0005) [2023-12-27 14:26:54,905][20381] Updated weights for policy 1, policy_version 1269136 (0.0007) [2023-12-27 14:26:55,170][20347] Updated weights for policy 0, policy_version 1270169 (0.0010) [2023-12-27 14:26:55,225][20347] Updated weights for policy 0, policy_version 1270179 (0.0010) [2023-12-27 14:26:55,279][20347] Updated weights for policy 0, policy_version 1270189 (0.0010) [2023-12-27 14:26:55,536][20381] Updated weights for policy 1, policy_version 1269146 (0.0007) [2023-12-27 14:26:55,583][20381] Updated weights for policy 1, policy_version 1269156 (0.0005) [2023-12-27 14:26:55,630][20381] Updated weights for policy 1, policy_version 1269166 (0.0005) [2023-12-27 14:26:55,679][20381] Updated weights for policy 1, policy_version 1269176 (0.0005) [2023-12-27 14:26:55,933][20347] Updated weights for policy 0, policy_version 1270199 (0.0010) [2023-12-27 14:26:55,984][20347] Updated weights for policy 0, policy_version 1270209 (0.0010) [2023-12-27 14:26:56,034][20347] Updated weights for policy 0, policy_version 1270219 (0.0010) [2023-12-27 14:26:56,279][20381] Updated weights for policy 1, policy_version 1269186 (0.0005) [2023-12-27 14:26:56,337][20381] Updated weights for policy 1, policy_version 1269196 (0.0005) [2023-12-27 14:26:56,385][20381] Updated weights for policy 1, policy_version 1269206 (0.0008) [2023-12-27 14:26:56,596][20347] Updated weights for policy 0, policy_version 1270229 (0.0008) [2023-12-27 14:26:56,641][20347] Updated weights for policy 0, policy_version 1270239 (0.0005) [2023-12-27 14:26:56,689][20347] Updated weights for policy 0, policy_version 1270249 (0.0009) [2023-12-27 14:26:56,978][20381] Updated weights for policy 1, policy_version 1269216 (0.0007) [2023-12-27 14:26:57,029][20381] Updated weights for policy 1, policy_version 1269226 (0.0005) [2023-12-27 14:26:57,082][20381] Updated weights for policy 1, policy_version 1269236 (0.0005) [2023-12-27 14:26:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 650199040. Throughput: 0: 10351.1, 1: 10376.4. Samples: 650187312. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:26:57,364][19170] Avg episode reward: [(0, '745.880'), (1, '836.420')] [2023-12-27 14:26:57,365][20347] Updated weights for policy 0, policy_version 1270259 (0.0010) [2023-12-27 14:26:57,432][20347] Updated weights for policy 0, policy_version 1270269 (0.0007) [2023-12-27 14:26:57,496][20347] Updated weights for policy 0, policy_version 1270279 (0.0006) [2023-12-27 14:26:57,604][20381] Updated weights for policy 1, policy_version 1269246 (0.0007) [2023-12-27 14:26:57,659][20381] Updated weights for policy 1, policy_version 1269256 (0.0007) [2023-12-27 14:26:57,719][20381] Updated weights for policy 1, policy_version 1269266 (0.0007) [2023-12-27 14:26:58,072][20347] Updated weights for policy 0, policy_version 1270289 (0.0006) [2023-12-27 14:26:58,124][20347] Updated weights for policy 0, policy_version 1270299 (0.0005) [2023-12-27 14:26:58,177][20347] Updated weights for policy 0, policy_version 1270309 (0.0005) [2023-12-27 14:26:58,239][20347] Updated weights for policy 0, policy_version 1270319 (0.0008) [2023-12-27 14:26:58,409][20381] Updated weights for policy 1, policy_version 1269276 (0.0009) [2023-12-27 14:26:58,471][20381] Updated weights for policy 1, policy_version 1269286 (0.0008) [2023-12-27 14:26:58,535][20381] Updated weights for policy 1, policy_version 1269296 (0.0008) [2023-12-27 14:26:59,008][20347] Updated weights for policy 0, policy_version 1270329 (0.0008) [2023-12-27 14:26:59,066][20347] Updated weights for policy 0, policy_version 1270339 (0.0008) [2023-12-27 14:26:59,128][20347] Updated weights for policy 0, policy_version 1270349 (0.0006) [2023-12-27 14:26:59,289][20381] Updated weights for policy 1, policy_version 1269306 (0.0007) [2023-12-27 14:26:59,350][20381] Updated weights for policy 1, policy_version 1269316 (0.0008) [2023-12-27 14:26:59,420][20381] Updated weights for policy 1, policy_version 1269326 (0.0008) [2023-12-27 14:26:59,478][20381] Updated weights for policy 1, policy_version 1269336 (0.0007) [2023-12-27 14:26:59,878][20347] Updated weights for policy 0, policy_version 1270359 (0.0008) [2023-12-27 14:26:59,938][20347] Updated weights for policy 0, policy_version 1270369 (0.0008) [2023-12-27 14:26:59,988][20347] Updated weights for policy 0, policy_version 1270379 (0.0006) [2023-12-27 14:27:00,160][20381] Updated weights for policy 1, policy_version 1269346 (0.0008) [2023-12-27 14:27:00,215][20381] Updated weights for policy 1, policy_version 1269356 (0.0008) [2023-12-27 14:27:00,267][20381] Updated weights for policy 1, policy_version 1269366 (0.0008) [2023-12-27 14:27:00,689][20347] Updated weights for policy 0, policy_version 1270389 (0.0007) [2023-12-27 14:27:00,754][20347] Updated weights for policy 0, policy_version 1270400 (0.0006) [2023-12-27 14:27:00,818][20347] Updated weights for policy 0, policy_version 1270410 (0.0008) [2023-12-27 14:27:00,958][20381] Updated weights for policy 1, policy_version 1269376 (0.0010) [2023-12-27 14:27:01,004][20381] Updated weights for policy 1, policy_version 1269386 (0.0010) [2023-12-27 14:27:01,073][20381] Updated weights for policy 1, policy_version 1269396 (0.0011) [2023-12-27 14:27:01,486][20347] Updated weights for policy 0, policy_version 1270420 (0.0007) [2023-12-27 14:27:01,531][20291] KL-divergence is very high: 106.5909 [2023-12-27 14:27:01,542][20347] Updated weights for policy 0, policy_version 1270430 (0.0008) [2023-12-27 14:27:01,603][20347] Updated weights for policy 0, policy_version 1270440 (0.0008) [2023-12-27 14:27:01,799][20381] Updated weights for policy 1, policy_version 1269406 (0.0010) [2023-12-27 14:27:01,851][20381] Updated weights for policy 1, policy_version 1269416 (0.0010) [2023-12-27 14:27:01,909][20381] Updated weights for policy 1, policy_version 1269426 (0.0011) [2023-12-27 14:27:02,293][20347] Updated weights for policy 0, policy_version 1270450 (0.0008) [2023-12-27 14:27:02,342][20347] Updated weights for policy 0, policy_version 1270460 (0.0007) [2023-12-27 14:27:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 650297344. Throughput: 0: 10345.1, 1: 10336.1. Samples: 650311292. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:02,363][19170] Avg episode reward: [(0, '755.860'), (1, '815.170')] [2023-12-27 14:27:02,400][20347] Updated weights for policy 0, policy_version 1270470 (0.0008) [2023-12-27 14:27:02,455][20347] Updated weights for policy 0, policy_version 1270480 (0.0007) [2023-12-27 14:27:02,574][20381] Updated weights for policy 1, policy_version 1269436 (0.0010) [2023-12-27 14:27:02,632][20381] Updated weights for policy 1, policy_version 1269446 (0.0011) [2023-12-27 14:27:02,687][20381] Updated weights for policy 1, policy_version 1269456 (0.0008) [2023-12-27 14:27:03,155][20347] Updated weights for policy 0, policy_version 1270490 (0.0007) [2023-12-27 14:27:03,199][20347] Updated weights for policy 0, policy_version 1270500 (0.0008) [2023-12-27 14:27:03,251][20347] Updated weights for policy 0, policy_version 1270510 (0.0009) [2023-12-27 14:27:03,253][20381] Updated weights for policy 1, policy_version 1269466 (0.0006) [2023-12-27 14:27:03,306][20381] Updated weights for policy 1, policy_version 1269476 (0.0005) [2023-12-27 14:27:03,357][20381] Updated weights for policy 1, policy_version 1269486 (0.0007) [2023-12-27 14:27:03,414][20381] Updated weights for policy 1, policy_version 1269496 (0.0007) [2023-12-27 14:27:03,985][20347] Updated weights for policy 0, policy_version 1270520 (0.0010) [2023-12-27 14:27:04,046][20347] Updated weights for policy 0, policy_version 1270530 (0.0011) [2023-12-27 14:27:04,106][20381] Updated weights for policy 1, policy_version 1269506 (0.0006) [2023-12-27 14:27:04,111][20347] Updated weights for policy 0, policy_version 1270540 (0.0009) [2023-12-27 14:27:04,168][20381] Updated weights for policy 1, policy_version 1269516 (0.0007) [2023-12-27 14:27:04,229][20381] Updated weights for policy 1, policy_version 1269526 (0.0008) [2023-12-27 14:27:04,753][20347] Updated weights for policy 0, policy_version 1270550 (0.0010) [2023-12-27 14:27:04,813][20347] Updated weights for policy 0, policy_version 1270560 (0.0010) [2023-12-27 14:27:04,868][20381] Updated weights for policy 1, policy_version 1269536 (0.0006) [2023-12-27 14:27:04,869][20347] Updated weights for policy 0, policy_version 1270570 (0.0010) [2023-12-27 14:27:04,926][20381] Updated weights for policy 1, policy_version 1269546 (0.0006) [2023-12-27 14:27:04,982][20381] Updated weights for policy 1, policy_version 1269556 (0.0008) [2023-12-27 14:27:05,567][20347] Updated weights for policy 0, policy_version 1270580 (0.0009) [2023-12-27 14:27:05,627][20347] Updated weights for policy 0, policy_version 1270590 (0.0010) [2023-12-27 14:27:05,684][20347] Updated weights for policy 0, policy_version 1270600 (0.0010) [2023-12-27 14:27:05,689][20381] Updated weights for policy 1, policy_version 1269566 (0.0007) [2023-12-27 14:27:05,739][20381] Updated weights for policy 1, policy_version 1269576 (0.0005) [2023-12-27 14:27:05,791][20381] Updated weights for policy 1, policy_version 1269586 (0.0005) [2023-12-27 14:27:06,359][20347] Updated weights for policy 0, policy_version 1270610 (0.0010) [2023-12-27 14:27:06,425][20347] Updated weights for policy 0, policy_version 1270620 (0.0010) [2023-12-27 14:27:06,487][20347] Updated weights for policy 0, policy_version 1270630 (0.0009) [2023-12-27 14:27:06,489][20381] Updated weights for policy 1, policy_version 1269596 (0.0005) [2023-12-27 14:27:06,544][20381] Updated weights for policy 1, policy_version 1269606 (0.0005) [2023-12-27 14:27:06,549][20347] Updated weights for policy 0, policy_version 1270640 (0.0010) [2023-12-27 14:27:06,594][20381] Updated weights for policy 1, policy_version 1269616 (0.0009) [2023-12-27 14:27:07,187][20347] Updated weights for policy 0, policy_version 1270650 (0.0005) [2023-12-27 14:27:07,255][20347] Updated weights for policy 0, policy_version 1270660 (0.0010) [2023-12-27 14:27:07,316][20381] Updated weights for policy 1, policy_version 1269626 (0.0008) [2023-12-27 14:27:07,317][20347] Updated weights for policy 0, policy_version 1270670 (0.0010) [2023-12-27 14:27:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 650403840. Throughput: 0: 10364.5, 1: 10413.6. Samples: 650373672. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:07,363][19170] Avg episode reward: [(0, '775.700'), (1, '786.920')] [2023-12-27 14:27:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001270672_325353472.pth... [2023-12-27 14:27:07,370][20381] Updated weights for policy 1, policy_version 1269636 (0.0010) [2023-12-27 14:27:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001269456_325042176.pth [2023-12-27 14:27:07,427][20381] Updated weights for policy 1, policy_version 1269646 (0.0010) [2023-12-27 14:27:07,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001269656_325058560.pth... [2023-12-27 14:27:07,478][20381] Updated weights for policy 1, policy_version 1269656 (0.0006) [2023-12-27 14:27:07,481][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001268408_324739072.pth [2023-12-27 14:27:07,898][20347] Updated weights for policy 0, policy_version 1270680 (0.0006) [2023-12-27 14:27:07,949][20347] Updated weights for policy 0, policy_version 1270690 (0.0009) [2023-12-27 14:27:07,997][20347] Updated weights for policy 0, policy_version 1270700 (0.0010) [2023-12-27 14:27:08,066][20381] Updated weights for policy 1, policy_version 1269666 (0.0011) [2023-12-27 14:27:08,128][20381] Updated weights for policy 1, policy_version 1269676 (0.0011) [2023-12-27 14:27:08,183][20381] Updated weights for policy 1, policy_version 1269686 (0.0009) [2023-12-27 14:27:08,679][20347] Updated weights for policy 0, policy_version 1270710 (0.0010) [2023-12-27 14:27:08,733][20347] Updated weights for policy 0, policy_version 1270720 (0.0010) [2023-12-27 14:27:08,792][20347] Updated weights for policy 0, policy_version 1270730 (0.0010) [2023-12-27 14:27:08,824][20381] Updated weights for policy 1, policy_version 1269696 (0.0006) [2023-12-27 14:27:08,880][20381] Updated weights for policy 1, policy_version 1269706 (0.0007) [2023-12-27 14:27:08,928][20381] Updated weights for policy 1, policy_version 1269716 (0.0005) [2023-12-27 14:27:09,513][20347] Updated weights for policy 0, policy_version 1270740 (0.0010) [2023-12-27 14:27:09,547][20381] Updated weights for policy 1, policy_version 1269726 (0.0005) [2023-12-27 14:27:09,565][20347] Updated weights for policy 0, policy_version 1270750 (0.0008) [2023-12-27 14:27:09,607][20381] Updated weights for policy 1, policy_version 1269736 (0.0005) [2023-12-27 14:27:09,621][20347] Updated weights for policy 0, policy_version 1270760 (0.0008) [2023-12-27 14:27:09,666][20381] Updated weights for policy 1, policy_version 1269746 (0.0006) [2023-12-27 14:27:10,347][20347] Updated weights for policy 0, policy_version 1270770 (0.0007) [2023-12-27 14:27:10,378][20381] Updated weights for policy 1, policy_version 1269756 (0.0008) [2023-12-27 14:27:10,404][20347] Updated weights for policy 0, policy_version 1270780 (0.0006) [2023-12-27 14:27:10,433][20381] Updated weights for policy 1, policy_version 1269766 (0.0007) [2023-12-27 14:27:10,458][20347] Updated weights for policy 0, policy_version 1270790 (0.0006) [2023-12-27 14:27:10,496][20381] Updated weights for policy 1, policy_version 1269776 (0.0008) [2023-12-27 14:27:10,514][20347] Updated weights for policy 0, policy_version 1270800 (0.0006) [2023-12-27 14:27:11,212][20381] Updated weights for policy 1, policy_version 1269786 (0.0007) [2023-12-27 14:27:11,243][20347] Updated weights for policy 0, policy_version 1270810 (0.0006) [2023-12-27 14:27:11,273][20381] Updated weights for policy 1, policy_version 1269796 (0.0008) [2023-12-27 14:27:11,303][20347] Updated weights for policy 0, policy_version 1270820 (0.0007) [2023-12-27 14:27:11,340][20381] Updated weights for policy 1, policy_version 1269806 (0.0008) [2023-12-27 14:27:11,356][20347] Updated weights for policy 0, policy_version 1270830 (0.0007) [2023-12-27 14:27:11,412][20381] Updated weights for policy 1, policy_version 1269816 (0.0008) [2023-12-27 14:27:12,093][20292] KL-divergence is very high: 131.7294 [2023-12-27 14:27:12,123][20347] Updated weights for policy 0, policy_version 1270840 (0.0006) [2023-12-27 14:27:12,129][20381] Updated weights for policy 1, policy_version 1269826 (0.0007) [2023-12-27 14:27:12,143][20292] KL-divergence is very high: 199.9653 [2023-12-27 14:27:12,176][20347] Updated weights for policy 0, policy_version 1270850 (0.0006) [2023-12-27 14:27:12,189][20381] Updated weights for policy 1, policy_version 1269836 (0.0008) [2023-12-27 14:27:12,236][20347] Updated weights for policy 0, policy_version 1270860 (0.0008) [2023-12-27 14:27:12,250][20381] Updated weights for policy 1, policy_version 1269846 (0.0006) [2023-12-27 14:27:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 650510336. Throughput: 0: 10322.4, 1: 10431.0. Samples: 650496832. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:12,363][19170] Avg episode reward: [(0, '834.030'), (1, '750.980')] [2023-12-27 14:27:12,892][20347] Updated weights for policy 0, policy_version 1270870 (0.0006) [2023-12-27 14:27:12,916][20381] Updated weights for policy 1, policy_version 1269856 (0.0008) [2023-12-27 14:27:12,949][20347] Updated weights for policy 0, policy_version 1270880 (0.0005) [2023-12-27 14:27:12,971][20381] Updated weights for policy 1, policy_version 1269866 (0.0007) [2023-12-27 14:27:13,000][20347] Updated weights for policy 0, policy_version 1270890 (0.0008) [2023-12-27 14:27:13,026][20381] Updated weights for policy 1, policy_version 1269876 (0.0010) [2023-12-27 14:27:13,629][20347] Updated weights for policy 0, policy_version 1270900 (0.0010) [2023-12-27 14:27:13,683][20347] Updated weights for policy 0, policy_version 1270910 (0.0010) [2023-12-27 14:27:13,728][20381] Updated weights for policy 1, policy_version 1269886 (0.0006) [2023-12-27 14:27:13,740][20347] Updated weights for policy 0, policy_version 1270920 (0.0010) [2023-12-27 14:27:13,781][20381] Updated weights for policy 1, policy_version 1269896 (0.0006) [2023-12-27 14:27:13,835][20381] Updated weights for policy 1, policy_version 1269906 (0.0007) [2023-12-27 14:27:14,412][20347] Updated weights for policy 0, policy_version 1270930 (0.0008) [2023-12-27 14:27:14,467][20381] Updated weights for policy 1, policy_version 1269916 (0.0007) [2023-12-27 14:27:14,468][20347] Updated weights for policy 0, policy_version 1270940 (0.0005) [2023-12-27 14:27:14,482][20291] KL-divergence is very high: 551.1978 [2023-12-27 14:27:14,487][20291] KL-divergence is very high: 339.8994 [2023-12-27 14:27:14,494][20291] KL-divergence is very high: 348.3419 [2023-12-27 14:27:14,516][20381] Updated weights for policy 1, policy_version 1269926 (0.0006) [2023-12-27 14:27:14,527][20347] Updated weights for policy 0, policy_version 1270950 (0.0007) [2023-12-27 14:27:14,529][20291] KL-divergence is very high: 327.1519 [2023-12-27 14:27:14,536][20291] KL-divergence is very high: 112.1637 [2023-12-27 14:27:14,576][20381] Updated weights for policy 1, policy_version 1269936 (0.0006) [2023-12-27 14:27:14,590][20347] Updated weights for policy 0, policy_version 1270960 (0.0005) [2023-12-27 14:27:15,185][20291] KL-divergence is very high: 447.2311 [2023-12-27 14:27:15,232][20347] Updated weights for policy 0, policy_version 1270970 (0.0007) [2023-12-27 14:27:15,233][20291] KL-divergence is very high: 532.5215 [2023-12-27 14:27:15,245][20381] Updated weights for policy 1, policy_version 1269946 (0.0005) [2023-12-27 14:27:15,278][20291] KL-divergence is very high: 679.1772 [2023-12-27 14:27:15,289][20347] Updated weights for policy 0, policy_version 1270980 (0.0006) [2023-12-27 14:27:15,299][20381] Updated weights for policy 1, policy_version 1269956 (0.0006) [2023-12-27 14:27:15,319][20291] KL-divergence is very high: 453.9642 [2023-12-27 14:27:15,343][20347] Updated weights for policy 0, policy_version 1270990 (0.0006) [2023-12-27 14:27:15,353][20381] Updated weights for policy 1, policy_version 1269966 (0.0006) [2023-12-27 14:27:15,406][20381] Updated weights for policy 1, policy_version 1269976 (0.0008) [2023-12-27 14:27:16,018][20381] Updated weights for policy 1, policy_version 1269986 (0.0005) [2023-12-27 14:27:16,018][20347] Updated weights for policy 0, policy_version 1271000 (0.0006) [2023-12-27 14:27:16,067][20381] Updated weights for policy 1, policy_version 1269996 (0.0005) [2023-12-27 14:27:16,075][20347] Updated weights for policy 0, policy_version 1271010 (0.0007) [2023-12-27 14:27:16,128][20381] Updated weights for policy 1, policy_version 1270006 (0.0007) [2023-12-27 14:27:16,139][20347] Updated weights for policy 0, policy_version 1271020 (0.0006) [2023-12-27 14:27:16,681][20381] Updated weights for policy 1, policy_version 1270016 (0.0007) [2023-12-27 14:27:16,740][20381] Updated weights for policy 1, policy_version 1270026 (0.0006) [2023-12-27 14:27:16,798][20381] Updated weights for policy 1, policy_version 1270036 (0.0005) [2023-12-27 14:27:16,841][20347] Updated weights for policy 0, policy_version 1271030 (0.0008) [2023-12-27 14:27:16,888][20347] Updated weights for policy 0, policy_version 1271040 (0.0008) [2023-12-27 14:27:16,941][20347] Updated weights for policy 0, policy_version 1271050 (0.0007) [2023-12-27 14:27:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 650608640. Throughput: 0: 10304.0, 1: 10433.6. Samples: 650620928. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:17,363][19170] Avg episode reward: [(0, '766.210'), (1, '706.330')] [2023-12-27 14:27:17,397][20381] Updated weights for policy 1, policy_version 1270046 (0.0008) [2023-12-27 14:27:17,451][20381] Updated weights for policy 1, policy_version 1270056 (0.0009) [2023-12-27 14:27:17,500][20381] Updated weights for policy 1, policy_version 1270066 (0.0007) [2023-12-27 14:27:17,559][20347] Updated weights for policy 0, policy_version 1271060 (0.0006) [2023-12-27 14:27:17,610][20347] Updated weights for policy 0, policy_version 1271070 (0.0005) [2023-12-27 14:27:17,661][20347] Updated weights for policy 0, policy_version 1271080 (0.0005) [2023-12-27 14:27:18,174][20381] Updated weights for policy 1, policy_version 1270076 (0.0009) [2023-12-27 14:27:18,226][20381] Updated weights for policy 1, policy_version 1270086 (0.0010) [2023-12-27 14:27:18,232][20347] Updated weights for policy 0, policy_version 1271090 (0.0008) [2023-12-27 14:27:18,275][20381] Updated weights for policy 1, policy_version 1270096 (0.0010) [2023-12-27 14:27:18,281][20347] Updated weights for policy 0, policy_version 1271100 (0.0010) [2023-12-27 14:27:18,338][20347] Updated weights for policy 0, policy_version 1271110 (0.0010) [2023-12-27 14:27:18,404][20347] Updated weights for policy 0, policy_version 1271120 (0.0010) [2023-12-27 14:27:18,851][20381] Updated weights for policy 1, policy_version 1270106 (0.0009) [2023-12-27 14:27:18,900][20381] Updated weights for policy 1, policy_version 1270116 (0.0006) [2023-12-27 14:27:18,959][20381] Updated weights for policy 1, policy_version 1270126 (0.0006) [2023-12-27 14:27:19,013][20381] Updated weights for policy 1, policy_version 1270136 (0.0006) [2023-12-27 14:27:19,144][20347] Updated weights for policy 0, policy_version 1271130 (0.0010) [2023-12-27 14:27:19,195][20347] Updated weights for policy 0, policy_version 1271140 (0.0010) [2023-12-27 14:27:19,256][20347] Updated weights for policy 0, policy_version 1271150 (0.0009) [2023-12-27 14:27:19,710][20381] Updated weights for policy 1, policy_version 1270146 (0.0007) [2023-12-27 14:27:19,766][20381] Updated weights for policy 1, policy_version 1270156 (0.0007) [2023-12-27 14:27:19,825][20381] Updated weights for policy 1, policy_version 1270166 (0.0008) [2023-12-27 14:27:19,974][20347] Updated weights for policy 0, policy_version 1271160 (0.0008) [2023-12-27 14:27:20,035][20347] Updated weights for policy 0, policy_version 1271170 (0.0010) [2023-12-27 14:27:20,093][20347] Updated weights for policy 0, policy_version 1271180 (0.0009) [2023-12-27 14:27:20,583][20381] Updated weights for policy 1, policy_version 1270176 (0.0007) [2023-12-27 14:27:20,641][20381] Updated weights for policy 1, policy_version 1270186 (0.0008) [2023-12-27 14:27:20,703][20381] Updated weights for policy 1, policy_version 1270196 (0.0008) [2023-12-27 14:27:20,871][20347] Updated weights for policy 0, policy_version 1271190 (0.0007) [2023-12-27 14:27:20,916][20291] KL-divergence is very high: 118.9786 [2023-12-27 14:27:20,935][20347] Updated weights for policy 0, policy_version 1271200 (0.0007) [2023-12-27 14:27:20,995][20347] Updated weights for policy 0, policy_version 1271210 (0.0007) [2023-12-27 14:27:21,445][20381] Updated weights for policy 1, policy_version 1270206 (0.0009) [2023-12-27 14:27:21,509][20381] Updated weights for policy 1, policy_version 1270216 (0.0010) [2023-12-27 14:27:21,569][20381] Updated weights for policy 1, policy_version 1270226 (0.0011) [2023-12-27 14:27:21,741][20347] Updated weights for policy 0, policy_version 1271220 (0.0009) [2023-12-27 14:27:21,806][20347] Updated weights for policy 0, policy_version 1271230 (0.0006) [2023-12-27 14:27:21,864][20347] Updated weights for policy 0, policy_version 1271240 (0.0006) [2023-12-27 14:27:21,904][20291] KL-divergence is very high: 142.7282 [2023-12-27 14:27:22,319][20381] Updated weights for policy 1, policy_version 1270236 (0.0010) [2023-12-27 14:27:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 650706944. Throughput: 0: 10298.7, 1: 10461.6. Samples: 650684004. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:22,363][19170] Avg episode reward: [(0, '508.470'), (1, '648.160')] [2023-12-27 14:27:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001271248_325500928.pth... [2023-12-27 14:27:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001270064_325197824.pth [2023-12-27 14:27:22,383][20381] Updated weights for policy 1, policy_version 1270246 (0.0009) [2023-12-27 14:27:22,439][20381] Updated weights for policy 1, policy_version 1270256 (0.0008) [2023-12-27 14:27:22,476][20347] Updated weights for policy 0, policy_version 1271250 (0.0006) [2023-12-27 14:27:22,487][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001270264_325214208.pth... [2023-12-27 14:27:22,491][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001269016_324894720.pth [2023-12-27 14:27:22,511][20291] KL-divergence is very high: 168.9737 [2023-12-27 14:27:22,533][20347] Updated weights for policy 0, policy_version 1271260 (0.0007) [2023-12-27 14:27:22,586][20347] Updated weights for policy 0, policy_version 1271270 (0.0008) [2023-12-27 14:27:22,639][20347] Updated weights for policy 0, policy_version 1271280 (0.0008) [2023-12-27 14:27:23,152][20381] Updated weights for policy 1, policy_version 1270266 (0.0007) [2023-12-27 14:27:23,211][20381] Updated weights for policy 1, policy_version 1270276 (0.0008) [2023-12-27 14:27:23,268][20381] Updated weights for policy 1, policy_version 1270286 (0.0008) [2023-12-27 14:27:23,296][20291] KL-divergence is very high: 128.4381 [2023-12-27 14:27:23,300][20347] Updated weights for policy 0, policy_version 1271290 (0.0006) [2023-12-27 14:27:23,324][20381] Updated weights for policy 1, policy_version 1270296 (0.0005) [2023-12-27 14:27:23,341][20291] KL-divergence is very high: 110.1721 [2023-12-27 14:27:23,354][20347] Updated weights for policy 0, policy_version 1271300 (0.0007) [2023-12-27 14:27:23,399][20347] Updated weights for policy 0, policy_version 1271310 (0.0009) [2023-12-27 14:27:23,925][20381] Updated weights for policy 1, policy_version 1270306 (0.0010) [2023-12-27 14:27:23,982][20381] Updated weights for policy 1, policy_version 1270316 (0.0005) [2023-12-27 14:27:24,033][20381] Updated weights for policy 1, policy_version 1270326 (0.0010) [2023-12-27 14:27:24,134][20347] Updated weights for policy 0, policy_version 1271320 (0.0007) [2023-12-27 14:27:24,187][20347] Updated weights for policy 0, policy_version 1271330 (0.0007) [2023-12-27 14:27:24,238][20347] Updated weights for policy 0, policy_version 1271340 (0.0007) [2023-12-27 14:27:24,701][20381] Updated weights for policy 1, policy_version 1270336 (0.0008) [2023-12-27 14:27:24,751][20381] Updated weights for policy 1, policy_version 1270346 (0.0007) [2023-12-27 14:27:24,802][20381] Updated weights for policy 1, policy_version 1270356 (0.0008) [2023-12-27 14:27:24,949][20347] Updated weights for policy 0, policy_version 1271350 (0.0007) [2023-12-27 14:27:25,010][20347] Updated weights for policy 0, policy_version 1271360 (0.0007) [2023-12-27 14:27:25,067][20347] Updated weights for policy 0, policy_version 1271370 (0.0006) [2023-12-27 14:27:25,431][20381] Updated weights for policy 1, policy_version 1270366 (0.0006) [2023-12-27 14:27:25,477][20381] Updated weights for policy 1, policy_version 1270376 (0.0007) [2023-12-27 14:27:25,549][20381] Updated weights for policy 1, policy_version 1270386 (0.0005) [2023-12-27 14:27:25,755][20347] Updated weights for policy 0, policy_version 1271380 (0.0008) [2023-12-27 14:27:25,803][20347] Updated weights for policy 0, policy_version 1271390 (0.0007) [2023-12-27 14:27:25,850][20347] Updated weights for policy 0, policy_version 1271400 (0.0007) [2023-12-27 14:27:26,169][20381] Updated weights for policy 1, policy_version 1270396 (0.0007) [2023-12-27 14:27:26,224][20381] Updated weights for policy 1, policy_version 1270406 (0.0010) [2023-12-27 14:27:26,285][20381] Updated weights for policy 1, policy_version 1270416 (0.0005) [2023-12-27 14:27:26,504][20347] Updated weights for policy 0, policy_version 1271410 (0.0007) [2023-12-27 14:27:26,562][20347] Updated weights for policy 0, policy_version 1271420 (0.0007) [2023-12-27 14:27:26,617][20347] Updated weights for policy 0, policy_version 1271430 (0.0007) [2023-12-27 14:27:26,673][20347] Updated weights for policy 0, policy_version 1271440 (0.0007) [2023-12-27 14:27:26,866][20381] Updated weights for policy 1, policy_version 1270426 (0.0006) [2023-12-27 14:27:26,910][20381] Updated weights for policy 1, policy_version 1270436 (0.0010) [2023-12-27 14:27:26,955][20381] Updated weights for policy 1, policy_version 1270446 (0.0010) [2023-12-27 14:27:27,000][20381] Updated weights for policy 1, policy_version 1270456 (0.0010) [2023-12-27 14:27:27,341][20347] Updated weights for policy 0, policy_version 1271450 (0.0007) [2023-12-27 14:27:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 650813440. Throughput: 0: 10262.8, 1: 10471.7. Samples: 650804568. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:27,363][19170] Avg episode reward: [(0, '629.400'), (1, '673.960')] [2023-12-27 14:27:27,390][20347] Updated weights for policy 0, policy_version 1271460 (0.0007) [2023-12-27 14:27:27,439][20347] Updated weights for policy 0, policy_version 1271470 (0.0007) [2023-12-27 14:27:27,633][20381] Updated weights for policy 1, policy_version 1270466 (0.0010) [2023-12-27 14:27:27,681][20381] Updated weights for policy 1, policy_version 1270476 (0.0010) [2023-12-27 14:27:27,735][20381] Updated weights for policy 1, policy_version 1270486 (0.0010) [2023-12-27 14:27:28,066][20347] Updated weights for policy 0, policy_version 1271480 (0.0007) [2023-12-27 14:27:28,119][20347] Updated weights for policy 0, policy_version 1271490 (0.0007) [2023-12-27 14:27:28,165][20347] Updated weights for policy 0, policy_version 1271500 (0.0007) [2023-12-27 14:27:28,411][20381] Updated weights for policy 1, policy_version 1270496 (0.0008) [2023-12-27 14:27:28,471][20381] Updated weights for policy 1, policy_version 1270506 (0.0007) [2023-12-27 14:27:28,518][20381] Updated weights for policy 1, policy_version 1270516 (0.0008) [2023-12-27 14:27:28,865][20347] Updated weights for policy 0, policy_version 1271510 (0.0007) [2023-12-27 14:27:28,912][20347] Updated weights for policy 0, policy_version 1271520 (0.0007) [2023-12-27 14:27:28,965][20347] Updated weights for policy 0, policy_version 1271530 (0.0007) [2023-12-27 14:27:29,211][20381] Updated weights for policy 1, policy_version 1270526 (0.0010) [2023-12-27 14:27:29,273][20381] Updated weights for policy 1, policy_version 1270536 (0.0009) [2023-12-27 14:27:29,325][20381] Updated weights for policy 1, policy_version 1270546 (0.0010) [2023-12-27 14:27:29,675][20347] Updated weights for policy 0, policy_version 1271540 (0.0007) [2023-12-27 14:27:29,731][20347] Updated weights for policy 0, policy_version 1271550 (0.0008) [2023-12-27 14:27:29,792][20347] Updated weights for policy 0, policy_version 1271560 (0.0008) [2023-12-27 14:27:30,032][20381] Updated weights for policy 1, policy_version 1270556 (0.0008) [2023-12-27 14:27:30,096][20381] Updated weights for policy 1, policy_version 1270566 (0.0005) [2023-12-27 14:27:30,153][20381] Updated weights for policy 1, policy_version 1270576 (0.0007) [2023-12-27 14:27:30,514][20347] Updated weights for policy 0, policy_version 1271570 (0.0007) [2023-12-27 14:27:30,567][20347] Updated weights for policy 0, policy_version 1271580 (0.0009) [2023-12-27 14:27:30,612][20347] Updated weights for policy 0, policy_version 1271590 (0.0008) [2023-12-27 14:27:30,659][20347] Updated weights for policy 0, policy_version 1271600 (0.0008) [2023-12-27 14:27:30,713][20381] Updated weights for policy 1, policy_version 1270586 (0.0007) [2023-12-27 14:27:30,765][20381] Updated weights for policy 1, policy_version 1270596 (0.0008) [2023-12-27 14:27:30,819][20381] Updated weights for policy 1, policy_version 1270606 (0.0005) [2023-12-27 14:27:30,869][20381] Updated weights for policy 1, policy_version 1270616 (0.0007) [2023-12-27 14:27:31,407][20347] Updated weights for policy 0, policy_version 1271610 (0.0008) [2023-12-27 14:27:31,462][20347] Updated weights for policy 0, policy_version 1271620 (0.0007) [2023-12-27 14:27:31,515][20347] Updated weights for policy 0, policy_version 1271630 (0.0007) [2023-12-27 14:27:31,628][20381] Updated weights for policy 1, policy_version 1270626 (0.0010) [2023-12-27 14:27:31,691][20381] Updated weights for policy 1, policy_version 1270636 (0.0007) [2023-12-27 14:27:31,757][20381] Updated weights for policy 1, policy_version 1270646 (0.0008) [2023-12-27 14:27:32,225][20347] Updated weights for policy 0, policy_version 1271640 (0.0009) [2023-12-27 14:27:32,281][20347] Updated weights for policy 0, policy_version 1271650 (0.0008) [2023-12-27 14:27:32,336][20347] Updated weights for policy 0, policy_version 1271660 (0.0008) [2023-12-27 14:27:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 650919936. Throughput: 0: 10222.6, 1: 10468.5. Samples: 650928772. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:32,363][19170] Avg episode reward: [(0, '609.390'), (1, '613.130')] [2023-12-27 14:27:32,403][20381] Updated weights for policy 1, policy_version 1270656 (0.0008) [2023-12-27 14:27:32,459][20381] Updated weights for policy 1, policy_version 1270666 (0.0008) [2023-12-27 14:27:32,526][20381] Updated weights for policy 1, policy_version 1270676 (0.0009) [2023-12-27 14:27:33,031][20347] Updated weights for policy 0, policy_version 1271670 (0.0010) [2023-12-27 14:27:33,085][20347] Updated weights for policy 0, policy_version 1271680 (0.0010) [2023-12-27 14:27:33,136][20347] Updated weights for policy 0, policy_version 1271690 (0.0010) [2023-12-27 14:27:33,152][20381] Updated weights for policy 1, policy_version 1270686 (0.0007) [2023-12-27 14:27:33,202][20381] Updated weights for policy 1, policy_version 1270696 (0.0007) [2023-12-27 14:27:33,251][20381] Updated weights for policy 1, policy_version 1270706 (0.0005) [2023-12-27 14:27:33,747][20347] Updated weights for policy 0, policy_version 1271700 (0.0008) [2023-12-27 14:27:33,800][20347] Updated weights for policy 0, policy_version 1271710 (0.0008) [2023-12-27 14:27:33,804][20381] Updated weights for policy 1, policy_version 1270716 (0.0005) [2023-12-27 14:27:33,845][20347] Updated weights for policy 0, policy_version 1271720 (0.0010) [2023-12-27 14:27:33,852][20381] Updated weights for policy 1, policy_version 1270726 (0.0005) [2023-12-27 14:27:33,895][20381] Updated weights for policy 1, policy_version 1270736 (0.0005) [2023-12-27 14:27:34,499][20347] Updated weights for policy 0, policy_version 1271730 (0.0009) [2023-12-27 14:27:34,560][20381] Updated weights for policy 1, policy_version 1270746 (0.0005) [2023-12-27 14:27:34,562][20347] Updated weights for policy 0, policy_version 1271740 (0.0011) [2023-12-27 14:27:34,609][20381] Updated weights for policy 1, policy_version 1270756 (0.0005) [2023-12-27 14:27:34,623][20347] Updated weights for policy 0, policy_version 1271750 (0.0010) [2023-12-27 14:27:34,665][20381] Updated weights for policy 1, policy_version 1270766 (0.0005) [2023-12-27 14:27:34,678][20347] Updated weights for policy 0, policy_version 1271760 (0.0010) [2023-12-27 14:27:34,713][20381] Updated weights for policy 1, policy_version 1270776 (0.0007) [2023-12-27 14:27:35,344][20347] Updated weights for policy 0, policy_version 1271770 (0.0007) [2023-12-27 14:27:35,354][20381] Updated weights for policy 1, policy_version 1270786 (0.0010) [2023-12-27 14:27:35,404][20347] Updated weights for policy 0, policy_version 1271780 (0.0006) [2023-12-27 14:27:35,412][20381] Updated weights for policy 1, policy_version 1270796 (0.0010) [2023-12-27 14:27:35,467][20347] Updated weights for policy 0, policy_version 1271790 (0.0006) [2023-12-27 14:27:35,472][20381] Updated weights for policy 1, policy_version 1270806 (0.0011) [2023-12-27 14:27:36,121][20381] Updated weights for policy 1, policy_version 1270818 (0.0010) [2023-12-27 14:27:36,136][20347] Updated weights for policy 0, policy_version 1271800 (0.0008) [2023-12-27 14:27:36,181][20381] Updated weights for policy 1, policy_version 1270828 (0.0011) [2023-12-27 14:27:36,197][20347] Updated weights for policy 0, policy_version 1271810 (0.0008) [2023-12-27 14:27:36,243][20381] Updated weights for policy 1, policy_version 1270838 (0.0011) [2023-12-27 14:27:36,255][20347] Updated weights for policy 0, policy_version 1271820 (0.0008) [2023-12-27 14:27:36,936][20381] Updated weights for policy 1, policy_version 1270848 (0.0011) [2023-12-27 14:27:36,943][20347] Updated weights for policy 0, policy_version 1271830 (0.0009) [2023-12-27 14:27:36,993][20381] Updated weights for policy 1, policy_version 1270858 (0.0006) [2023-12-27 14:27:36,998][20347] Updated weights for policy 0, policy_version 1271840 (0.0007) [2023-12-27 14:27:37,048][20347] Updated weights for policy 0, policy_version 1271850 (0.0007) [2023-12-27 14:27:37,050][20381] Updated weights for policy 1, policy_version 1270868 (0.0005) [2023-12-27 14:27:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 651026432. Throughput: 0: 10226.8, 1: 10512.1. Samples: 650993752. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:37,363][19170] Avg episode reward: [(0, '669.840'), (1, '615.900')] [2023-12-27 14:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001270872_325369856.pth... [2023-12-27 14:27:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001271856_325656576.pth... [2023-12-27 14:27:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001270672_325353472.pth [2023-12-27 14:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001269656_325058560.pth [2023-12-27 14:27:37,645][20381] Updated weights for policy 1, policy_version 1270878 (0.0006) [2023-12-27 14:27:37,710][20381] Updated weights for policy 1, policy_version 1270888 (0.0006) [2023-12-27 14:27:37,736][20347] Updated weights for policy 0, policy_version 1271860 (0.0007) [2023-12-27 14:27:37,764][20381] Updated weights for policy 1, policy_version 1270898 (0.0008) [2023-12-27 14:27:37,791][20347] Updated weights for policy 0, policy_version 1271870 (0.0007) [2023-12-27 14:27:37,837][20347] Updated weights for policy 0, policy_version 1271880 (0.0007) [2023-12-27 14:27:38,428][20381] Updated weights for policy 1, policy_version 1270908 (0.0010) [2023-12-27 14:27:38,487][20381] Updated weights for policy 1, policy_version 1270918 (0.0011) [2023-12-27 14:27:38,504][20347] Updated weights for policy 0, policy_version 1271890 (0.0007) [2023-12-27 14:27:38,544][20381] Updated weights for policy 1, policy_version 1270928 (0.0011) [2023-12-27 14:27:38,552][20347] Updated weights for policy 0, policy_version 1271900 (0.0008) [2023-12-27 14:27:38,611][20347] Updated weights for policy 0, policy_version 1271910 (0.0008) [2023-12-27 14:27:38,658][20347] Updated weights for policy 0, policy_version 1271920 (0.0008) [2023-12-27 14:27:39,182][20381] Updated weights for policy 1, policy_version 1270938 (0.0010) [2023-12-27 14:27:39,242][20381] Updated weights for policy 1, policy_version 1270948 (0.0010) [2023-12-27 14:27:39,304][20381] Updated weights for policy 1, policy_version 1270958 (0.0008) [2023-12-27 14:27:39,363][20381] Updated weights for policy 1, policy_version 1270968 (0.0008) [2023-12-27 14:27:39,381][20347] Updated weights for policy 0, policy_version 1271930 (0.0008) [2023-12-27 14:27:39,444][20347] Updated weights for policy 0, policy_version 1271940 (0.0008) [2023-12-27 14:27:39,497][20347] Updated weights for policy 0, policy_version 1271950 (0.0008) [2023-12-27 14:27:40,104][20381] Updated weights for policy 1, policy_version 1270978 (0.0006) [2023-12-27 14:27:40,169][20381] Updated weights for policy 1, policy_version 1270988 (0.0007) [2023-12-27 14:27:40,227][20381] Updated weights for policy 1, policy_version 1270998 (0.0008) [2023-12-27 14:27:40,272][20347] Updated weights for policy 0, policy_version 1271960 (0.0007) [2023-12-27 14:27:40,337][20347] Updated weights for policy 0, policy_version 1271970 (0.0007) [2023-12-27 14:27:40,417][20347] Updated weights for policy 0, policy_version 1271980 (0.0008) [2023-12-27 14:27:40,876][20381] Updated weights for policy 1, policy_version 1271008 (0.0008) [2023-12-27 14:27:40,928][20381] Updated weights for policy 1, policy_version 1271018 (0.0009) [2023-12-27 14:27:40,977][20381] Updated weights for policy 1, policy_version 1271028 (0.0007) [2023-12-27 14:27:40,978][20347] Updated weights for policy 0, policy_version 1271990 (0.0006) [2023-12-27 14:27:41,044][20347] Updated weights for policy 0, policy_version 1272000 (0.0007) [2023-12-27 14:27:41,098][20347] Updated weights for policy 0, policy_version 1272010 (0.0008) [2023-12-27 14:27:41,727][20381] Updated weights for policy 1, policy_version 1271038 (0.0008) [2023-12-27 14:27:41,795][20381] Updated weights for policy 1, policy_version 1271048 (0.0008) [2023-12-27 14:27:41,825][20347] Updated weights for policy 0, policy_version 1272020 (0.0007) [2023-12-27 14:27:41,850][20381] Updated weights for policy 1, policy_version 1271058 (0.0007) [2023-12-27 14:27:41,883][20347] Updated weights for policy 0, policy_version 1272030 (0.0006) [2023-12-27 14:27:41,938][20347] Updated weights for policy 0, policy_version 1272040 (0.0007) [2023-12-27 14:27:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 651124736. Throughput: 0: 10180.1, 1: 10466.6. Samples: 651116412. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:42,363][19170] Avg episode reward: [(0, '630.680'), (1, '691.390')] [2023-12-27 14:27:42,573][20381] Updated weights for policy 1, policy_version 1271068 (0.0007) [2023-12-27 14:27:42,627][20347] Updated weights for policy 0, policy_version 1272050 (0.0007) [2023-12-27 14:27:42,632][20381] Updated weights for policy 1, policy_version 1271078 (0.0008) [2023-12-27 14:27:42,680][20347] Updated weights for policy 0, policy_version 1272060 (0.0006) [2023-12-27 14:27:42,686][20381] Updated weights for policy 1, policy_version 1271088 (0.0006) [2023-12-27 14:27:42,732][20347] Updated weights for policy 0, policy_version 1272070 (0.0005) [2023-12-27 14:27:42,787][20347] Updated weights for policy 0, policy_version 1272080 (0.0006) [2023-12-27 14:27:43,347][20381] Updated weights for policy 1, policy_version 1271098 (0.0006) [2023-12-27 14:27:43,398][20381] Updated weights for policy 1, policy_version 1271108 (0.0006) [2023-12-27 14:27:43,417][20347] Updated weights for policy 0, policy_version 1272090 (0.0007) [2023-12-27 14:27:43,447][20381] Updated weights for policy 1, policy_version 1271118 (0.0006) [2023-12-27 14:27:43,472][20347] Updated weights for policy 0, policy_version 1272100 (0.0010) [2023-12-27 14:27:43,499][20381] Updated weights for policy 1, policy_version 1271128 (0.0005) [2023-12-27 14:27:43,524][20347] Updated weights for policy 0, policy_version 1272110 (0.0010) [2023-12-27 14:27:44,182][20381] Updated weights for policy 1, policy_version 1271138 (0.0008) [2023-12-27 14:27:44,233][20347] Updated weights for policy 0, policy_version 1272120 (0.0010) [2023-12-27 14:27:44,238][20381] Updated weights for policy 1, policy_version 1271148 (0.0006) [2023-12-27 14:27:44,285][20347] Updated weights for policy 0, policy_version 1272130 (0.0010) [2023-12-27 14:27:44,301][20381] Updated weights for policy 1, policy_version 1271158 (0.0006) [2023-12-27 14:27:44,344][20347] Updated weights for policy 0, policy_version 1272140 (0.0007) [2023-12-27 14:27:44,963][20347] Updated weights for policy 0, policy_version 1272150 (0.0007) [2023-12-27 14:27:45,016][20347] Updated weights for policy 0, policy_version 1272160 (0.0008) [2023-12-27 14:27:45,039][20381] Updated weights for policy 1, policy_version 1271168 (0.0007) [2023-12-27 14:27:45,076][20347] Updated weights for policy 0, policy_version 1272170 (0.0008) [2023-12-27 14:27:45,100][20381] Updated weights for policy 1, policy_version 1271178 (0.0007) [2023-12-27 14:27:45,156][20381] Updated weights for policy 1, policy_version 1271188 (0.0007) [2023-12-27 14:27:45,719][20347] Updated weights for policy 0, policy_version 1272180 (0.0007) [2023-12-27 14:27:45,774][20347] Updated weights for policy 0, policy_version 1272190 (0.0005) [2023-12-27 14:27:45,820][20347] Updated weights for policy 0, policy_version 1272200 (0.0006) [2023-12-27 14:27:45,876][20381] Updated weights for policy 1, policy_version 1271198 (0.0007) [2023-12-27 14:27:45,930][20381] Updated weights for policy 1, policy_version 1271208 (0.0006) [2023-12-27 14:27:45,992][20381] Updated weights for policy 1, policy_version 1271218 (0.0005) [2023-12-27 14:27:46,434][20347] Updated weights for policy 0, policy_version 1272210 (0.0006) [2023-12-27 14:27:46,499][20347] Updated weights for policy 0, policy_version 1272220 (0.0007) [2023-12-27 14:27:46,557][20347] Updated weights for policy 0, policy_version 1272230 (0.0007) [2023-12-27 14:27:46,586][20381] Updated weights for policy 1, policy_version 1271228 (0.0005) [2023-12-27 14:27:46,614][20347] Updated weights for policy 0, policy_version 1272240 (0.0007) [2023-12-27 14:27:46,636][20381] Updated weights for policy 1, policy_version 1271238 (0.0006) [2023-12-27 14:27:46,688][20381] Updated weights for policy 1, policy_version 1271248 (0.0007) [2023-12-27 14:27:47,262][20347] Updated weights for policy 0, policy_version 1272250 (0.0005) [2023-12-27 14:27:47,310][20347] Updated weights for policy 0, policy_version 1272260 (0.0005) [2023-12-27 14:27:47,359][20347] Updated weights for policy 0, policy_version 1272270 (0.0005) [2023-12-27 14:27:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 651223040. Throughput: 0: 10195.3, 1: 10412.0. Samples: 651238620. Policy #0 lag: (min: 23.0, avg: 30.6, max: 55.0) [2023-12-27 14:27:47,363][19170] Avg episode reward: [(0, '701.680'), (1, '708.290')] [2023-12-27 14:27:47,420][20381] Updated weights for policy 1, policy_version 1271258 (0.0007) [2023-12-27 14:27:47,465][20381] Updated weights for policy 1, policy_version 1271268 (0.0009) [2023-12-27 14:27:47,513][20381] Updated weights for policy 1, policy_version 1271278 (0.0008) [2023-12-27 14:27:47,569][20381] Updated weights for policy 1, policy_version 1271288 (0.0008) [2023-12-27 14:27:47,969][20347] Updated weights for policy 0, policy_version 1272280 (0.0007) [2023-12-27 14:27:48,033][20347] Updated weights for policy 0, policy_version 1272290 (0.0007) [2023-12-27 14:27:48,078][20347] Updated weights for policy 0, policy_version 1272300 (0.0008) [2023-12-27 14:27:48,236][20381] Updated weights for policy 1, policy_version 1271298 (0.0005) [2023-12-27 14:27:48,298][20381] Updated weights for policy 1, policy_version 1271308 (0.0006) [2023-12-27 14:27:48,360][20381] Updated weights for policy 1, policy_version 1271318 (0.0008) [2023-12-27 14:27:48,795][20347] Updated weights for policy 0, policy_version 1272310 (0.0009) [2023-12-27 14:27:48,854][20347] Updated weights for policy 0, policy_version 1272320 (0.0008) [2023-12-27 14:27:48,910][20347] Updated weights for policy 0, policy_version 1272330 (0.0008) [2023-12-27 14:27:49,035][20381] Updated weights for policy 1, policy_version 1271328 (0.0006) [2023-12-27 14:27:49,095][20381] Updated weights for policy 1, policy_version 1271338 (0.0008) [2023-12-27 14:27:49,154][20381] Updated weights for policy 1, policy_version 1271348 (0.0008) [2023-12-27 14:27:49,620][20347] Updated weights for policy 0, policy_version 1272340 (0.0008) [2023-12-27 14:27:49,683][20347] Updated weights for policy 0, policy_version 1272350 (0.0007) [2023-12-27 14:27:49,743][20347] Updated weights for policy 0, policy_version 1272360 (0.0008) [2023-12-27 14:27:49,850][20381] Updated weights for policy 1, policy_version 1271358 (0.0008) [2023-12-27 14:27:49,914][20381] Updated weights for policy 1, policy_version 1271368 (0.0007) [2023-12-27 14:27:49,975][20381] Updated weights for policy 1, policy_version 1271378 (0.0007) [2023-12-27 14:27:50,414][20347] Updated weights for policy 0, policy_version 1272370 (0.0008) [2023-12-27 14:27:50,468][20347] Updated weights for policy 0, policy_version 1272380 (0.0008) [2023-12-27 14:27:50,516][20347] Updated weights for policy 0, policy_version 1272390 (0.0007) [2023-12-27 14:27:50,578][20347] Updated weights for policy 0, policy_version 1272400 (0.0006) [2023-12-27 14:27:50,626][20381] Updated weights for policy 1, policy_version 1271388 (0.0007) [2023-12-27 14:27:50,682][20381] Updated weights for policy 1, policy_version 1271398 (0.0008) [2023-12-27 14:27:50,735][20381] Updated weights for policy 1, policy_version 1271408 (0.0008) [2023-12-27 14:27:51,307][20347] Updated weights for policy 0, policy_version 1272410 (0.0007) [2023-12-27 14:27:51,382][20347] Updated weights for policy 0, policy_version 1272420 (0.0008) [2023-12-27 14:27:51,441][20347] Updated weights for policy 0, policy_version 1272430 (0.0007) [2023-12-27 14:27:51,534][20381] Updated weights for policy 1, policy_version 1271418 (0.0008) [2023-12-27 14:27:51,589][20381] Updated weights for policy 1, policy_version 1271428 (0.0008) [2023-12-27 14:27:51,656][20381] Updated weights for policy 1, policy_version 1271438 (0.0009) [2023-12-27 14:27:51,719][20381] Updated weights for policy 1, policy_version 1271448 (0.0008) [2023-12-27 14:27:52,132][20347] Updated weights for policy 0, policy_version 1272440 (0.0008) [2023-12-27 14:27:52,196][20347] Updated weights for policy 0, policy_version 1272450 (0.0008) [2023-12-27 14:27:52,253][20347] Updated weights for policy 0, policy_version 1272460 (0.0008) [2023-12-27 14:27:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 651329536. Throughput: 0: 10203.4, 1: 10372.3. Samples: 651299576. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:27:52,363][19170] Avg episode reward: [(0, '669.340'), (1, '707.550')] [2023-12-27 14:27:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001272464_325812224.pth... [2023-12-27 14:27:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001271448_325517312.pth... [2023-12-27 14:27:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001271248_325500928.pth [2023-12-27 14:27:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001270264_325214208.pth [2023-12-27 14:27:52,480][20381] Updated weights for policy 1, policy_version 1271458 (0.0008) [2023-12-27 14:27:52,534][20381] Updated weights for policy 1, policy_version 1271468 (0.0008) [2023-12-27 14:27:52,590][20381] Updated weights for policy 1, policy_version 1271478 (0.0009) [2023-12-27 14:27:52,929][20347] Updated weights for policy 0, policy_version 1272470 (0.0008) [2023-12-27 14:27:52,986][20347] Updated weights for policy 0, policy_version 1272480 (0.0009) [2023-12-27 14:27:53,052][20347] Updated weights for policy 0, policy_version 1272490 (0.0007) [2023-12-27 14:27:53,269][20381] Updated weights for policy 1, policy_version 1271488 (0.0008) [2023-12-27 14:27:53,312][20381] Updated weights for policy 1, policy_version 1271498 (0.0006) [2023-12-27 14:27:53,363][20381] Updated weights for policy 1, policy_version 1271508 (0.0005) [2023-12-27 14:27:53,703][20347] Updated weights for policy 0, policy_version 1272500 (0.0008) [2023-12-27 14:27:53,749][20347] Updated weights for policy 0, policy_version 1272510 (0.0008) [2023-12-27 14:27:53,801][20347] Updated weights for policy 0, policy_version 1272520 (0.0008) [2023-12-27 14:27:54,023][20381] Updated weights for policy 1, policy_version 1271518 (0.0005) [2023-12-27 14:27:54,086][20381] Updated weights for policy 1, policy_version 1271528 (0.0006) [2023-12-27 14:27:54,145][20381] Updated weights for policy 1, policy_version 1271538 (0.0008) [2023-12-27 14:27:54,483][20347] Updated weights for policy 0, policy_version 1272530 (0.0007) [2023-12-27 14:27:54,544][20347] Updated weights for policy 0, policy_version 1272540 (0.0008) [2023-12-27 14:27:54,599][20347] Updated weights for policy 0, policy_version 1272550 (0.0008) [2023-12-27 14:27:54,649][20347] Updated weights for policy 0, policy_version 1272560 (0.0008) [2023-12-27 14:27:54,769][20381] Updated weights for policy 1, policy_version 1271548 (0.0007) [2023-12-27 14:27:54,819][20381] Updated weights for policy 1, policy_version 1271558 (0.0005) [2023-12-27 14:27:54,878][20381] Updated weights for policy 1, policy_version 1271568 (0.0005) [2023-12-27 14:27:55,395][20347] Updated weights for policy 0, policy_version 1272570 (0.0009) [2023-12-27 14:27:55,458][20347] Updated weights for policy 0, policy_version 1272580 (0.0006) [2023-12-27 14:27:55,458][20381] Updated weights for policy 1, policy_version 1271578 (0.0005) [2023-12-27 14:27:55,509][20381] Updated weights for policy 1, policy_version 1271588 (0.0007) [2023-12-27 14:27:55,511][20347] Updated weights for policy 0, policy_version 1272590 (0.0006) [2023-12-27 14:27:55,556][20381] Updated weights for policy 1, policy_version 1271598 (0.0007) [2023-12-27 14:27:55,608][20381] Updated weights for policy 1, policy_version 1271608 (0.0005) [2023-12-27 14:27:56,203][20347] Updated weights for policy 0, policy_version 1272600 (0.0007) [2023-12-27 14:27:56,241][20381] Updated weights for policy 1, policy_version 1271618 (0.0008) [2023-12-27 14:27:56,254][20347] Updated weights for policy 0, policy_version 1272610 (0.0006) [2023-12-27 14:27:56,298][20381] Updated weights for policy 1, policy_version 1271628 (0.0005) [2023-12-27 14:27:56,307][20347] Updated weights for policy 0, policy_version 1272620 (0.0007) [2023-12-27 14:27:56,352][20381] Updated weights for policy 1, policy_version 1271638 (0.0008) [2023-12-27 14:27:56,953][20347] Updated weights for policy 0, policy_version 1272630 (0.0008) [2023-12-27 14:27:56,996][20347] Updated weights for policy 0, policy_version 1272640 (0.0007) [2023-12-27 14:27:57,041][20347] Updated weights for policy 0, policy_version 1272650 (0.0007) [2023-12-27 14:27:57,043][20381] Updated weights for policy 1, policy_version 1271648 (0.0010) [2023-12-27 14:27:57,094][20381] Updated weights for policy 1, policy_version 1271658 (0.0010) [2023-12-27 14:27:57,141][20381] Updated weights for policy 1, policy_version 1271668 (0.0010) [2023-12-27 14:27:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 651436032. Throughput: 0: 10183.4, 1: 10369.2. Samples: 651421700. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:27:57,363][19170] Avg episode reward: [(0, '747.040'), (1, '718.560')] [2023-12-27 14:27:57,772][20347] Updated weights for policy 0, policy_version 1272660 (0.0006) [2023-12-27 14:27:57,820][20347] Updated weights for policy 0, policy_version 1272670 (0.0006) [2023-12-27 14:27:57,828][20381] Updated weights for policy 1, policy_version 1271678 (0.0010) [2023-12-27 14:27:57,864][20347] Updated weights for policy 0, policy_version 1272680 (0.0007) [2023-12-27 14:27:57,882][20381] Updated weights for policy 1, policy_version 1271688 (0.0010) [2023-12-27 14:27:57,933][20381] Updated weights for policy 1, policy_version 1271698 (0.0010) [2023-12-27 14:27:58,586][20347] Updated weights for policy 0, policy_version 1272690 (0.0006) [2023-12-27 14:27:58,652][20347] Updated weights for policy 0, policy_version 1272700 (0.0007) [2023-12-27 14:27:58,677][20381] Updated weights for policy 1, policy_version 1271708 (0.0009) [2023-12-27 14:27:58,724][20347] Updated weights for policy 0, policy_version 1272710 (0.0007) [2023-12-27 14:27:58,737][20381] Updated weights for policy 1, policy_version 1271718 (0.0007) [2023-12-27 14:27:58,799][20347] Updated weights for policy 0, policy_version 1272720 (0.0007) [2023-12-27 14:27:58,804][20381] Updated weights for policy 1, policy_version 1271728 (0.0010) [2023-12-27 14:27:59,539][20381] Updated weights for policy 1, policy_version 1271738 (0.0008) [2023-12-27 14:27:59,539][20347] Updated weights for policy 0, policy_version 1272730 (0.0006) [2023-12-27 14:27:59,573][20291] KL-divergence is very high: 104.6385 [2023-12-27 14:27:59,596][20347] Updated weights for policy 0, policy_version 1272740 (0.0007) [2023-12-27 14:27:59,601][20381] Updated weights for policy 1, policy_version 1271748 (0.0007) [2023-12-27 14:27:59,651][20347] Updated weights for policy 0, policy_version 1272750 (0.0005) [2023-12-27 14:27:59,657][20381] Updated weights for policy 1, policy_version 1271758 (0.0006) [2023-12-27 14:27:59,715][20381] Updated weights for policy 1, policy_version 1271768 (0.0007) [2023-12-27 14:28:00,296][20347] Updated weights for policy 0, policy_version 1272760 (0.0006) [2023-12-27 14:28:00,353][20347] Updated weights for policy 0, policy_version 1272770 (0.0005) [2023-12-27 14:28:00,378][20381] Updated weights for policy 1, policy_version 1271778 (0.0006) [2023-12-27 14:28:00,409][20347] Updated weights for policy 0, policy_version 1272780 (0.0006) [2023-12-27 14:28:00,443][20381] Updated weights for policy 1, policy_version 1271788 (0.0006) [2023-12-27 14:28:00,511][20381] Updated weights for policy 1, policy_version 1271798 (0.0005) [2023-12-27 14:28:01,075][20347] Updated weights for policy 0, policy_version 1272790 (0.0007) [2023-12-27 14:28:01,077][20381] Updated weights for policy 1, policy_version 1271808 (0.0006) [2023-12-27 14:28:01,131][20381] Updated weights for policy 1, policy_version 1271818 (0.0009) [2023-12-27 14:28:01,134][20347] Updated weights for policy 0, policy_version 1272800 (0.0006) [2023-12-27 14:28:01,193][20381] Updated weights for policy 1, policy_version 1271828 (0.0010) [2023-12-27 14:28:01,196][20347] Updated weights for policy 0, policy_version 1272810 (0.0007) [2023-12-27 14:28:01,856][20381] Updated weights for policy 1, policy_version 1271838 (0.0007) [2023-12-27 14:28:01,876][20347] Updated weights for policy 0, policy_version 1272820 (0.0008) [2023-12-27 14:28:01,914][20381] Updated weights for policy 1, policy_version 1271848 (0.0006) [2023-12-27 14:28:01,932][20347] Updated weights for policy 0, policy_version 1272830 (0.0010) [2023-12-27 14:28:01,974][20381] Updated weights for policy 1, policy_version 1271858 (0.0008) [2023-12-27 14:28:01,992][20347] Updated weights for policy 0, policy_version 1272840 (0.0010) [2023-12-27 14:28:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 651534336. Throughput: 0: 10173.7, 1: 10319.9. Samples: 651543140. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:02,363][19170] Avg episode reward: [(0, '796.080'), (1, '796.430')] [2023-12-27 14:28:02,621][20381] Updated weights for policy 1, policy_version 1271868 (0.0006) [2023-12-27 14:28:02,677][20381] Updated weights for policy 1, policy_version 1271878 (0.0007) [2023-12-27 14:28:02,690][20347] Updated weights for policy 0, policy_version 1272850 (0.0010) [2023-12-27 14:28:02,733][20381] Updated weights for policy 1, policy_version 1271888 (0.0006) [2023-12-27 14:28:02,752][20347] Updated weights for policy 0, policy_version 1272860 (0.0011) [2023-12-27 14:28:02,806][20347] Updated weights for policy 0, policy_version 1272870 (0.0009) [2023-12-27 14:28:02,870][20347] Updated weights for policy 0, policy_version 1272880 (0.0007) [2023-12-27 14:28:03,392][20381] Updated weights for policy 1, policy_version 1271898 (0.0008) [2023-12-27 14:28:03,449][20381] Updated weights for policy 1, policy_version 1271908 (0.0007) [2023-12-27 14:28:03,488][20347] Updated weights for policy 0, policy_version 1272890 (0.0005) [2023-12-27 14:28:03,497][20381] Updated weights for policy 1, policy_version 1271918 (0.0010) [2023-12-27 14:28:03,531][20347] Updated weights for policy 0, policy_version 1272900 (0.0005) [2023-12-27 14:28:03,543][20381] Updated weights for policy 1, policy_version 1271928 (0.0010) [2023-12-27 14:28:03,579][20347] Updated weights for policy 0, policy_version 1272910 (0.0005) [2023-12-27 14:28:04,243][20381] Updated weights for policy 1, policy_version 1271938 (0.0006) [2023-12-27 14:28:04,258][20347] Updated weights for policy 0, policy_version 1272920 (0.0007) [2023-12-27 14:28:04,307][20381] Updated weights for policy 1, policy_version 1271948 (0.0006) [2023-12-27 14:28:04,309][20347] Updated weights for policy 0, policy_version 1272930 (0.0007) [2023-12-27 14:28:04,363][20347] Updated weights for policy 0, policy_version 1272940 (0.0006) [2023-12-27 14:28:04,365][20381] Updated weights for policy 1, policy_version 1271958 (0.0008) [2023-12-27 14:28:05,012][20347] Updated weights for policy 0, policy_version 1272950 (0.0008) [2023-12-27 14:28:05,047][20381] Updated weights for policy 1, policy_version 1271968 (0.0005) [2023-12-27 14:28:05,060][20347] Updated weights for policy 0, policy_version 1272960 (0.0008) [2023-12-27 14:28:05,093][20381] Updated weights for policy 1, policy_version 1271978 (0.0005) [2023-12-27 14:28:05,109][20347] Updated weights for policy 0, policy_version 1272970 (0.0007) [2023-12-27 14:28:05,160][20381] Updated weights for policy 1, policy_version 1271988 (0.0005) [2023-12-27 14:28:05,826][20381] Updated weights for policy 1, policy_version 1271998 (0.0006) [2023-12-27 14:28:05,849][20347] Updated weights for policy 0, policy_version 1272980 (0.0007) [2023-12-27 14:28:05,878][20381] Updated weights for policy 1, policy_version 1272008 (0.0006) [2023-12-27 14:28:05,909][20347] Updated weights for policy 0, policy_version 1272990 (0.0008) [2023-12-27 14:28:05,940][20381] Updated weights for policy 1, policy_version 1272018 (0.0006) [2023-12-27 14:28:05,966][20347] Updated weights for policy 0, policy_version 1273000 (0.0008) [2023-12-27 14:28:06,569][20381] Updated weights for policy 1, policy_version 1272028 (0.0007) [2023-12-27 14:28:06,630][20381] Updated weights for policy 1, policy_version 1272038 (0.0008) [2023-12-27 14:28:06,676][20347] Updated weights for policy 0, policy_version 1273010 (0.0007) [2023-12-27 14:28:06,689][20381] Updated weights for policy 1, policy_version 1272048 (0.0007) [2023-12-27 14:28:06,733][20347] Updated weights for policy 0, policy_version 1273020 (0.0006) [2023-12-27 14:28:06,786][20347] Updated weights for policy 0, policy_version 1273030 (0.0006) [2023-12-27 14:28:06,842][20347] Updated weights for policy 0, policy_version 1273040 (0.0005) [2023-12-27 14:28:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 651632640. Throughput: 0: 10176.7, 1: 10325.4. Samples: 651606600. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:07,363][19170] Avg episode reward: [(0, '804.970'), (1, '747.010')] [2023-12-27 14:28:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001272056_325672960.pth... [2023-12-27 14:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001273040_325959680.pth... [2023-12-27 14:28:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001270872_325369856.pth [2023-12-27 14:28:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001271856_325656576.pth [2023-12-27 14:28:07,423][20381] Updated weights for policy 1, policy_version 1272058 (0.0006) [2023-12-27 14:28:07,446][20347] Updated weights for policy 0, policy_version 1273050 (0.0007) [2023-12-27 14:28:07,482][20381] Updated weights for policy 1, policy_version 1272068 (0.0005) [2023-12-27 14:28:07,503][20347] Updated weights for policy 0, policy_version 1273060 (0.0006) [2023-12-27 14:28:07,544][20381] Updated weights for policy 1, policy_version 1272078 (0.0007) [2023-12-27 14:28:07,557][20347] Updated weights for policy 0, policy_version 1273070 (0.0006) [2023-12-27 14:28:07,594][20381] Updated weights for policy 1, policy_version 1272088 (0.0007) [2023-12-27 14:28:08,231][20347] Updated weights for policy 0, policy_version 1273080 (0.0007) [2023-12-27 14:28:08,278][20347] Updated weights for policy 0, policy_version 1273090 (0.0007) [2023-12-27 14:28:08,291][20381] Updated weights for policy 1, policy_version 1272098 (0.0007) [2023-12-27 14:28:08,333][20347] Updated weights for policy 0, policy_version 1273100 (0.0007) [2023-12-27 14:28:08,353][20381] Updated weights for policy 1, policy_version 1272108 (0.0007) [2023-12-27 14:28:08,412][20381] Updated weights for policy 1, policy_version 1272118 (0.0009) [2023-12-27 14:28:09,037][20347] Updated weights for policy 0, policy_version 1273110 (0.0008) [2023-12-27 14:28:09,094][20347] Updated weights for policy 0, policy_version 1273120 (0.0006) [2023-12-27 14:28:09,107][20381] Updated weights for policy 1, policy_version 1272128 (0.0006) [2023-12-27 14:28:09,153][20347] Updated weights for policy 0, policy_version 1273130 (0.0006) [2023-12-27 14:28:09,162][20381] Updated weights for policy 1, policy_version 1272138 (0.0006) [2023-12-27 14:28:09,218][20381] Updated weights for policy 1, policy_version 1272148 (0.0008) [2023-12-27 14:28:09,897][20347] Updated weights for policy 0, policy_version 1273140 (0.0007) [2023-12-27 14:28:09,933][20381] Updated weights for policy 1, policy_version 1272158 (0.0008) [2023-12-27 14:28:09,959][20347] Updated weights for policy 0, policy_version 1273150 (0.0007) [2023-12-27 14:28:09,994][20381] Updated weights for policy 1, policy_version 1272168 (0.0009) [2023-12-27 14:28:10,020][20347] Updated weights for policy 0, policy_version 1273160 (0.0007) [2023-12-27 14:28:10,054][20381] Updated weights for policy 1, policy_version 1272178 (0.0008) [2023-12-27 14:28:10,665][20347] Updated weights for policy 0, policy_version 1273170 (0.0006) [2023-12-27 14:28:10,717][20347] Updated weights for policy 0, policy_version 1273180 (0.0005) [2023-12-27 14:28:10,764][20347] Updated weights for policy 0, policy_version 1273190 (0.0008) [2023-12-27 14:28:10,777][20381] Updated weights for policy 1, policy_version 1272188 (0.0007) [2023-12-27 14:28:10,817][20347] Updated weights for policy 0, policy_version 1273200 (0.0005) [2023-12-27 14:28:10,834][20381] Updated weights for policy 1, policy_version 1272198 (0.0007) [2023-12-27 14:28:10,889][20381] Updated weights for policy 1, policy_version 1272208 (0.0007) [2023-12-27 14:28:11,530][20347] Updated weights for policy 0, policy_version 1273210 (0.0007) [2023-12-27 14:28:11,552][20291] KL-divergence is very high: 130.8596 [2023-12-27 14:28:11,570][20291] KL-divergence is very high: 145.6117 [2023-12-27 14:28:11,588][20347] Updated weights for policy 0, policy_version 1273220 (0.0007) [2023-12-27 14:28:11,599][20291] KL-divergence is very high: 101.1780 [2023-12-27 14:28:11,650][20347] Updated weights for policy 0, policy_version 1273230 (0.0008) [2023-12-27 14:28:11,679][20381] Updated weights for policy 1, policy_version 1272218 (0.0007) [2023-12-27 14:28:11,752][20381] Updated weights for policy 1, policy_version 1272228 (0.0008) [2023-12-27 14:28:11,812][20381] Updated weights for policy 1, policy_version 1272238 (0.0008) [2023-12-27 14:28:11,874][20381] Updated weights for policy 1, policy_version 1272248 (0.0008) [2023-12-27 14:28:12,360][20347] Updated weights for policy 0, policy_version 1273240 (0.0007) [2023-12-27 14:28:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 651730944. Throughput: 0: 10207.1, 1: 10271.3. Samples: 651726096. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:12,363][19170] Avg episode reward: [(0, '755.960'), (1, '746.660')] [2023-12-27 14:28:12,412][20347] Updated weights for policy 0, policy_version 1273250 (0.0008) [2023-12-27 14:28:12,471][20347] Updated weights for policy 0, policy_version 1273260 (0.0008) [2023-12-27 14:28:12,565][20381] Updated weights for policy 1, policy_version 1272258 (0.0008) [2023-12-27 14:28:12,631][20381] Updated weights for policy 1, policy_version 1272268 (0.0008) [2023-12-27 14:28:12,686][20381] Updated weights for policy 1, policy_version 1272278 (0.0007) [2023-12-27 14:28:13,121][20347] Updated weights for policy 0, policy_version 1273270 (0.0009) [2023-12-27 14:28:13,161][20291] KL-divergence is very high: 128.8411 [2023-12-27 14:28:13,168][20291] KL-divergence is very high: 136.8932 [2023-12-27 14:28:13,179][20347] Updated weights for policy 0, policy_version 1273280 (0.0008) [2023-12-27 14:28:13,208][20291] KL-divergence is very high: 131.3690 [2023-12-27 14:28:13,214][20291] KL-divergence is very high: 121.8032 [2023-12-27 14:28:13,237][20347] Updated weights for policy 0, policy_version 1273290 (0.0008) [2023-12-27 14:28:13,397][20381] Updated weights for policy 1, policy_version 1272288 (0.0008) [2023-12-27 14:28:13,455][20381] Updated weights for policy 1, policy_version 1272298 (0.0007) [2023-12-27 14:28:13,513][20381] Updated weights for policy 1, policy_version 1272308 (0.0007) [2023-12-27 14:28:13,907][20347] Updated weights for policy 0, policy_version 1273300 (0.0006) [2023-12-27 14:28:13,955][20347] Updated weights for policy 0, policy_version 1273310 (0.0005) [2023-12-27 14:28:14,004][20347] Updated weights for policy 0, policy_version 1273320 (0.0005) [2023-12-27 14:28:14,151][20381] Updated weights for policy 1, policy_version 1272318 (0.0007) [2023-12-27 14:28:14,215][20381] Updated weights for policy 1, policy_version 1272328 (0.0006) [2023-12-27 14:28:14,266][20381] Updated weights for policy 1, policy_version 1272338 (0.0005) [2023-12-27 14:28:14,611][20347] Updated weights for policy 0, policy_version 1273330 (0.0005) [2023-12-27 14:28:14,666][20347] Updated weights for policy 0, policy_version 1273340 (0.0006) [2023-12-27 14:28:14,710][20347] Updated weights for policy 0, policy_version 1273350 (0.0007) [2023-12-27 14:28:14,762][20347] Updated weights for policy 0, policy_version 1273360 (0.0007) [2023-12-27 14:28:14,962][20381] Updated weights for policy 1, policy_version 1272348 (0.0008) [2023-12-27 14:28:15,028][20381] Updated weights for policy 1, policy_version 1272358 (0.0008) [2023-12-27 14:28:15,091][20381] Updated weights for policy 1, policy_version 1272368 (0.0007) [2023-12-27 14:28:15,519][20347] Updated weights for policy 0, policy_version 1273370 (0.0007) [2023-12-27 14:28:15,570][20347] Updated weights for policy 0, policy_version 1273380 (0.0007) [2023-12-27 14:28:15,633][20347] Updated weights for policy 0, policy_version 1273390 (0.0006) [2023-12-27 14:28:15,676][20381] Updated weights for policy 1, policy_version 1272378 (0.0007) [2023-12-27 14:28:15,730][20381] Updated weights for policy 1, policy_version 1272388 (0.0006) [2023-12-27 14:28:15,780][20381] Updated weights for policy 1, policy_version 1272398 (0.0006) [2023-12-27 14:28:15,833][20381] Updated weights for policy 1, policy_version 1272408 (0.0007) [2023-12-27 14:28:16,328][20347] Updated weights for policy 0, policy_version 1273400 (0.0007) [2023-12-27 14:28:16,377][20347] Updated weights for policy 0, policy_version 1273410 (0.0008) [2023-12-27 14:28:16,431][20347] Updated weights for policy 0, policy_version 1273420 (0.0007) [2023-12-27 14:28:16,484][20381] Updated weights for policy 1, policy_version 1272418 (0.0005) [2023-12-27 14:28:16,547][20381] Updated weights for policy 1, policy_version 1272428 (0.0005) [2023-12-27 14:28:16,611][20381] Updated weights for policy 1, policy_version 1272438 (0.0006) [2023-12-27 14:28:17,148][20347] Updated weights for policy 0, policy_version 1273430 (0.0007) [2023-12-27 14:28:17,200][20347] Updated weights for policy 0, policy_version 1273440 (0.0007) [2023-12-27 14:28:17,211][20381] Updated weights for policy 1, policy_version 1272448 (0.0006) [2023-12-27 14:28:17,251][20347] Updated weights for policy 0, policy_version 1273450 (0.0006) [2023-12-27 14:28:17,257][20381] Updated weights for policy 1, policy_version 1272458 (0.0006) [2023-12-27 14:28:17,313][20381] Updated weights for policy 1, policy_version 1272468 (0.0008) [2023-12-27 14:28:17,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 651845632. Throughput: 0: 10212.7, 1: 10233.0. Samples: 651848824. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:17,363][19170] Avg episode reward: [(0, '765.830'), (1, '771.800')] [2023-12-27 14:28:17,887][20347] Updated weights for policy 0, policy_version 1273460 (0.0007) [2023-12-27 14:28:17,896][20381] Updated weights for policy 1, policy_version 1272478 (0.0006) [2023-12-27 14:28:17,942][20347] Updated weights for policy 0, policy_version 1273470 (0.0010) [2023-12-27 14:28:17,948][20381] Updated weights for policy 1, policy_version 1272488 (0.0008) [2023-12-27 14:28:17,998][20381] Updated weights for policy 1, policy_version 1272498 (0.0006) [2023-12-27 14:28:18,000][20347] Updated weights for policy 0, policy_version 1273480 (0.0010) [2023-12-27 14:28:18,644][20381] Updated weights for policy 1, policy_version 1272508 (0.0010) [2023-12-27 14:28:18,647][20347] Updated weights for policy 0, policy_version 1273490 (0.0006) [2023-12-27 14:28:18,696][20381] Updated weights for policy 1, policy_version 1272518 (0.0010) [2023-12-27 14:28:18,709][20347] Updated weights for policy 0, policy_version 1273500 (0.0011) [2023-12-27 14:28:18,757][20381] Updated weights for policy 1, policy_version 1272528 (0.0007) [2023-12-27 14:28:18,771][20347] Updated weights for policy 0, policy_version 1273510 (0.0010) [2023-12-27 14:28:18,824][20347] Updated weights for policy 0, policy_version 1273520 (0.0010) [2023-12-27 14:28:19,343][20381] Updated weights for policy 1, policy_version 1272538 (0.0006) [2023-12-27 14:28:19,412][20381] Updated weights for policy 1, policy_version 1272548 (0.0010) [2023-12-27 14:28:19,476][20381] Updated weights for policy 1, policy_version 1272558 (0.0008) [2023-12-27 14:28:19,541][20381] Updated weights for policy 1, policy_version 1272568 (0.0008) [2023-12-27 14:28:19,576][20347] Updated weights for policy 0, policy_version 1273530 (0.0008) [2023-12-27 14:28:19,629][20347] Updated weights for policy 0, policy_version 1273540 (0.0007) [2023-12-27 14:28:19,694][20347] Updated weights for policy 0, policy_version 1273550 (0.0010) [2023-12-27 14:28:20,355][20381] Updated weights for policy 1, policy_version 1272578 (0.0009) [2023-12-27 14:28:20,417][20381] Updated weights for policy 1, policy_version 1272588 (0.0009) [2023-12-27 14:28:20,476][20381] Updated weights for policy 1, policy_version 1272598 (0.0008) [2023-12-27 14:28:20,484][20347] Updated weights for policy 0, policy_version 1273560 (0.0010) [2023-12-27 14:28:20,540][20347] Updated weights for policy 0, policy_version 1273570 (0.0011) [2023-12-27 14:28:20,604][20347] Updated weights for policy 0, policy_version 1273580 (0.0011) [2023-12-27 14:28:21,291][20347] Updated weights for policy 0, policy_version 1273590 (0.0008) [2023-12-27 14:28:21,353][20347] Updated weights for policy 0, policy_version 1273600 (0.0009) [2023-12-27 14:28:21,360][20381] Updated weights for policy 1, policy_version 1272608 (0.0009) [2023-12-27 14:28:21,423][20381] Updated weights for policy 1, policy_version 1272618 (0.0008) [2023-12-27 14:28:21,424][20347] Updated weights for policy 0, policy_version 1273610 (0.0009) [2023-12-27 14:28:21,483][20381] Updated weights for policy 1, policy_version 1272628 (0.0009) [2023-12-27 14:28:22,157][20347] Updated weights for policy 0, policy_version 1273620 (0.0008) [2023-12-27 14:28:22,211][20347] Updated weights for policy 0, policy_version 1273630 (0.0007) [2023-12-27 14:28:22,282][20347] Updated weights for policy 0, policy_version 1273641 (0.0007) [2023-12-27 14:28:22,333][20381] Updated weights for policy 1, policy_version 1272638 (0.0007) [2023-12-27 14:28:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 651935744. Throughput: 0: 10196.4, 1: 10188.6. Samples: 651911080. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:22,363][19170] Avg episode reward: [(0, '814.640'), (1, '764.060')] [2023-12-27 14:28:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001273648_326115328.pth... [2023-12-27 14:28:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001272464_325812224.pth [2023-12-27 14:28:22,393][20381] Updated weights for policy 1, policy_version 1272648 (0.0007) [2023-12-27 14:28:22,420][20292] KL-divergence is very high: 128.8538 [2023-12-27 14:28:22,445][20381] Updated weights for policy 1, policy_version 1272658 (0.0008) [2023-12-27 14:28:22,458][20292] KL-divergence is very high: 136.8264 [2023-12-27 14:28:22,470][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001272664_325828608.pth... [2023-12-27 14:28:22,473][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001271448_325517312.pth [2023-12-27 14:28:22,940][20347] Updated weights for policy 0, policy_version 1273651 (0.0007) [2023-12-27 14:28:22,994][20347] Updated weights for policy 0, policy_version 1273661 (0.0005) [2023-12-27 14:28:23,039][20347] Updated weights for policy 0, policy_version 1273671 (0.0007) [2023-12-27 14:28:23,149][20292] KL-divergence is very high: 113.8135 [2023-12-27 14:28:23,171][20381] Updated weights for policy 1, policy_version 1272668 (0.0007) [2023-12-27 14:28:23,220][20381] Updated weights for policy 1, policy_version 1272678 (0.0007) [2023-12-27 14:28:23,276][20381] Updated weights for policy 1, policy_version 1272688 (0.0009) [2023-12-27 14:28:23,617][20347] Updated weights for policy 0, policy_version 1273681 (0.0006) [2023-12-27 14:28:23,684][20347] Updated weights for policy 0, policy_version 1273691 (0.0007) [2023-12-27 14:28:23,747][20347] Updated weights for policy 0, policy_version 1273701 (0.0008) [2023-12-27 14:28:23,808][20347] Updated weights for policy 0, policy_version 1273711 (0.0007) [2023-12-27 14:28:23,989][20381] Updated weights for policy 1, policy_version 1272698 (0.0008) [2023-12-27 14:28:24,046][20381] Updated weights for policy 1, policy_version 1272708 (0.0008) [2023-12-27 14:28:24,100][20381] Updated weights for policy 1, policy_version 1272718 (0.0008) [2023-12-27 14:28:24,154][20381] Updated weights for policy 1, policy_version 1272728 (0.0007) [2023-12-27 14:28:24,417][20347] Updated weights for policy 0, policy_version 1273721 (0.0008) [2023-12-27 14:28:24,474][20347] Updated weights for policy 0, policy_version 1273731 (0.0008) [2023-12-27 14:28:24,536][20347] Updated weights for policy 0, policy_version 1273741 (0.0007) [2023-12-27 14:28:24,783][20381] Updated weights for policy 1, policy_version 1272738 (0.0009) [2023-12-27 14:28:24,831][20381] Updated weights for policy 1, policy_version 1272748 (0.0008) [2023-12-27 14:28:24,893][20381] Updated weights for policy 1, policy_version 1272758 (0.0008) [2023-12-27 14:28:25,135][20347] Updated weights for policy 0, policy_version 1273751 (0.0007) [2023-12-27 14:28:25,184][20347] Updated weights for policy 0, policy_version 1273761 (0.0007) [2023-12-27 14:28:25,230][20347] Updated weights for policy 0, policy_version 1273771 (0.0008) [2023-12-27 14:28:25,614][20381] Updated weights for policy 1, policy_version 1272768 (0.0007) [2023-12-27 14:28:25,662][20381] Updated weights for policy 1, policy_version 1272778 (0.0007) [2023-12-27 14:28:25,708][20381] Updated weights for policy 1, policy_version 1272788 (0.0006) [2023-12-27 14:28:25,932][20347] Updated weights for policy 0, policy_version 1273781 (0.0006) [2023-12-27 14:28:25,979][20347] Updated weights for policy 0, policy_version 1273791 (0.0005) [2023-12-27 14:28:26,029][20347] Updated weights for policy 0, policy_version 1273801 (0.0005) [2023-12-27 14:28:26,400][20381] Updated weights for policy 1, policy_version 1272798 (0.0007) [2023-12-27 14:28:26,461][20381] Updated weights for policy 1, policy_version 1272808 (0.0007) [2023-12-27 14:28:26,518][20381] Updated weights for policy 1, policy_version 1272818 (0.0007) [2023-12-27 14:28:26,644][20347] Updated weights for policy 0, policy_version 1273811 (0.0005) [2023-12-27 14:28:26,701][20347] Updated weights for policy 0, policy_version 1273821 (0.0005) [2023-12-27 14:28:26,752][20347] Updated weights for policy 0, policy_version 1273831 (0.0008) [2023-12-27 14:28:27,206][20381] Updated weights for policy 1, policy_version 1272828 (0.0007) [2023-12-27 14:28:27,263][20381] Updated weights for policy 1, policy_version 1272838 (0.0007) [2023-12-27 14:28:27,317][20381] Updated weights for policy 1, policy_version 1272848 (0.0007) [2023-12-27 14:28:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 652042240. Throughput: 0: 10208.6, 1: 10084.1. Samples: 652029580. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:27,363][19170] Avg episode reward: [(0, '843.500'), (1, '814.720')] [2023-12-27 14:28:27,387][20347] Updated weights for policy 0, policy_version 1273841 (0.0010) [2023-12-27 14:28:27,446][20347] Updated weights for policy 0, policy_version 1273851 (0.0010) [2023-12-27 14:28:27,502][20347] Updated weights for policy 0, policy_version 1273861 (0.0010) [2023-12-27 14:28:27,551][20347] Updated weights for policy 0, policy_version 1273871 (0.0009) [2023-12-27 14:28:28,026][20381] Updated weights for policy 1, policy_version 1272858 (0.0007) [2023-12-27 14:28:28,081][20381] Updated weights for policy 1, policy_version 1272868 (0.0009) [2023-12-27 14:28:28,129][20381] Updated weights for policy 1, policy_version 1272878 (0.0007) [2023-12-27 14:28:28,147][20347] Updated weights for policy 0, policy_version 1273881 (0.0005) [2023-12-27 14:28:28,181][20381] Updated weights for policy 1, policy_version 1272888 (0.0008) [2023-12-27 14:28:28,206][20347] Updated weights for policy 0, policy_version 1273891 (0.0005) [2023-12-27 14:28:28,268][20347] Updated weights for policy 0, policy_version 1273901 (0.0008) [2023-12-27 14:28:28,888][20347] Updated weights for policy 0, policy_version 1273911 (0.0010) [2023-12-27 14:28:28,909][20381] Updated weights for policy 1, policy_version 1272898 (0.0005) [2023-12-27 14:28:28,937][20347] Updated weights for policy 0, policy_version 1273921 (0.0010) [2023-12-27 14:28:28,969][20381] Updated weights for policy 1, policy_version 1272908 (0.0005) [2023-12-27 14:28:28,989][20347] Updated weights for policy 0, policy_version 1273931 (0.0009) [2023-12-27 14:28:29,018][20381] Updated weights for policy 1, policy_version 1272918 (0.0005) [2023-12-27 14:28:29,692][20381] Updated weights for policy 1, policy_version 1272928 (0.0006) [2023-12-27 14:28:29,700][20347] Updated weights for policy 0, policy_version 1273941 (0.0008) [2023-12-27 14:28:29,745][20381] Updated weights for policy 1, policy_version 1272938 (0.0007) [2023-12-27 14:28:29,758][20347] Updated weights for policy 0, policy_version 1273951 (0.0007) [2023-12-27 14:28:29,800][20381] Updated weights for policy 1, policy_version 1272948 (0.0007) [2023-12-27 14:28:29,806][20347] Updated weights for policy 0, policy_version 1273961 (0.0006) [2023-12-27 14:28:30,479][20381] Updated weights for policy 1, policy_version 1272958 (0.0008) [2023-12-27 14:28:30,532][20347] Updated weights for policy 0, policy_version 1273971 (0.0008) [2023-12-27 14:28:30,534][20381] Updated weights for policy 1, policy_version 1272968 (0.0007) [2023-12-27 14:28:30,582][20347] Updated weights for policy 0, policy_version 1273981 (0.0010) [2023-12-27 14:28:30,584][20381] Updated weights for policy 1, policy_version 1272978 (0.0006) [2023-12-27 14:28:30,634][20347] Updated weights for policy 0, policy_version 1273991 (0.0005) [2023-12-27 14:28:31,256][20347] Updated weights for policy 0, policy_version 1274001 (0.0006) [2023-12-27 14:28:31,310][20347] Updated weights for policy 0, policy_version 1274011 (0.0010) [2023-12-27 14:28:31,316][20381] Updated weights for policy 1, policy_version 1272988 (0.0007) [2023-12-27 14:28:31,369][20347] Updated weights for policy 0, policy_version 1274021 (0.0010) [2023-12-27 14:28:31,375][20381] Updated weights for policy 1, policy_version 1272998 (0.0006) [2023-12-27 14:28:31,432][20381] Updated weights for policy 1, policy_version 1273008 (0.0007) [2023-12-27 14:28:31,437][20347] Updated weights for policy 0, policy_version 1274031 (0.0008) [2023-12-27 14:28:32,154][20347] Updated weights for policy 0, policy_version 1274041 (0.0007) [2023-12-27 14:28:32,185][20381] Updated weights for policy 1, policy_version 1273018 (0.0006) [2023-12-27 14:28:32,211][20347] Updated weights for policy 0, policy_version 1274051 (0.0007) [2023-12-27 14:28:32,238][20381] Updated weights for policy 1, policy_version 1273028 (0.0007) [2023-12-27 14:28:32,270][20347] Updated weights for policy 0, policy_version 1274061 (0.0006) [2023-12-27 14:28:32,304][20381] Updated weights for policy 1, policy_version 1273038 (0.0006) [2023-12-27 14:28:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 652140544. Throughput: 0: 10233.7, 1: 10067.2. Samples: 652152160. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:32,363][19170] Avg episode reward: [(0, '853.520'), (1, '766.290')] [2023-12-27 14:28:32,365][20381] Updated weights for policy 1, policy_version 1273048 (0.0008) [2023-12-27 14:28:32,943][20347] Updated weights for policy 0, policy_version 1274071 (0.0007) [2023-12-27 14:28:32,997][20347] Updated weights for policy 0, policy_version 1274081 (0.0006) [2023-12-27 14:28:33,053][20347] Updated weights for policy 0, policy_version 1274091 (0.0005) [2023-12-27 14:28:33,094][20381] Updated weights for policy 1, policy_version 1273058 (0.0007) [2023-12-27 14:28:33,152][20381] Updated weights for policy 1, policy_version 1273068 (0.0009) [2023-12-27 14:28:33,200][20381] Updated weights for policy 1, policy_version 1273078 (0.0007) [2023-12-27 14:28:33,656][20347] Updated weights for policy 0, policy_version 1274101 (0.0006) [2023-12-27 14:28:33,714][20347] Updated weights for policy 0, policy_version 1274111 (0.0007) [2023-12-27 14:28:33,768][20347] Updated weights for policy 0, policy_version 1274121 (0.0007) [2023-12-27 14:28:33,878][20381] Updated weights for policy 1, policy_version 1273088 (0.0006) [2023-12-27 14:28:33,935][20381] Updated weights for policy 1, policy_version 1273098 (0.0008) [2023-12-27 14:28:33,988][20381] Updated weights for policy 1, policy_version 1273108 (0.0007) [2023-12-27 14:28:34,490][20347] Updated weights for policy 0, policy_version 1274131 (0.0007) [2023-12-27 14:28:34,550][20347] Updated weights for policy 0, policy_version 1274141 (0.0009) [2023-12-27 14:28:34,615][20347] Updated weights for policy 0, policy_version 1274151 (0.0008) [2023-12-27 14:28:34,670][20381] Updated weights for policy 1, policy_version 1273118 (0.0008) [2023-12-27 14:28:34,730][20381] Updated weights for policy 1, policy_version 1273128 (0.0008) [2023-12-27 14:28:34,788][20381] Updated weights for policy 1, policy_version 1273138 (0.0007) [2023-12-27 14:28:35,306][20347] Updated weights for policy 0, policy_version 1274161 (0.0006) [2023-12-27 14:28:35,367][20347] Updated weights for policy 0, policy_version 1274171 (0.0008) [2023-12-27 14:28:35,425][20347] Updated weights for policy 0, policy_version 1274181 (0.0007) [2023-12-27 14:28:35,460][20381] Updated weights for policy 1, policy_version 1273148 (0.0008) [2023-12-27 14:28:35,487][20347] Updated weights for policy 0, policy_version 1274191 (0.0006) [2023-12-27 14:28:35,521][20381] Updated weights for policy 1, policy_version 1273158 (0.0008) [2023-12-27 14:28:35,577][20381] Updated weights for policy 1, policy_version 1273168 (0.0008) [2023-12-27 14:28:36,228][20347] Updated weights for policy 0, policy_version 1274201 (0.0008) [2023-12-27 14:28:36,280][20347] Updated weights for policy 0, policy_version 1274211 (0.0008) [2023-12-27 14:28:36,284][20381] Updated weights for policy 1, policy_version 1273178 (0.0010) [2023-12-27 14:28:36,337][20347] Updated weights for policy 0, policy_version 1274221 (0.0008) [2023-12-27 14:28:36,343][20381] Updated weights for policy 1, policy_version 1273188 (0.0008) [2023-12-27 14:28:36,403][20381] Updated weights for policy 1, policy_version 1273198 (0.0011) [2023-12-27 14:28:36,462][20381] Updated weights for policy 1, policy_version 1273208 (0.0011) [2023-12-27 14:28:36,997][20347] Updated weights for policy 0, policy_version 1274231 (0.0008) [2023-12-27 14:28:37,057][20347] Updated weights for policy 0, policy_version 1274241 (0.0008) [2023-12-27 14:28:37,116][20347] Updated weights for policy 0, policy_version 1274251 (0.0008) [2023-12-27 14:28:37,132][20381] Updated weights for policy 1, policy_version 1273218 (0.0005) [2023-12-27 14:28:37,181][20381] Updated weights for policy 1, policy_version 1273228 (0.0007) [2023-12-27 14:28:37,243][20381] Updated weights for policy 1, policy_version 1273238 (0.0006) [2023-12-27 14:28:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 652247040. Throughput: 0: 10238.5, 1: 10058.3. Samples: 652212936. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:37,364][19170] Avg episode reward: [(0, '814.520'), (1, '784.900')] [2023-12-27 14:28:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001273240_325976064.pth... [2023-12-27 14:28:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001274256_326270976.pth... [2023-12-27 14:28:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001272056_325672960.pth [2023-12-27 14:28:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001273040_325959680.pth [2023-12-27 14:28:37,774][20347] Updated weights for policy 0, policy_version 1274261 (0.0007) [2023-12-27 14:28:37,779][20381] Updated weights for policy 1, policy_version 1273248 (0.0009) [2023-12-27 14:28:37,830][20381] Updated weights for policy 1, policy_version 1273258 (0.0009) [2023-12-27 14:28:37,836][20347] Updated weights for policy 0, policy_version 1274271 (0.0006) [2023-12-27 14:28:37,879][20381] Updated weights for policy 1, policy_version 1273268 (0.0010) [2023-12-27 14:28:37,902][20347] Updated weights for policy 0, policy_version 1274281 (0.0006) [2023-12-27 14:28:38,519][20381] Updated weights for policy 1, policy_version 1273278 (0.0009) [2023-12-27 14:28:38,567][20347] Updated weights for policy 0, policy_version 1274291 (0.0007) [2023-12-27 14:28:38,586][20381] Updated weights for policy 1, policy_version 1273288 (0.0008) [2023-12-27 14:28:38,629][20347] Updated weights for policy 0, policy_version 1274301 (0.0006) [2023-12-27 14:28:38,650][20381] Updated weights for policy 1, policy_version 1273298 (0.0007) [2023-12-27 14:28:38,686][20347] Updated weights for policy 0, policy_version 1274311 (0.0006) [2023-12-27 14:28:39,303][20381] Updated weights for policy 1, policy_version 1273308 (0.0007) [2023-12-27 14:28:39,360][20381] Updated weights for policy 1, policy_version 1273318 (0.0008) [2023-12-27 14:28:39,391][20347] Updated weights for policy 0, policy_version 1274321 (0.0008) [2023-12-27 14:28:39,430][20381] Updated weights for policy 1, policy_version 1273328 (0.0008) [2023-12-27 14:28:39,462][20347] Updated weights for policy 0, policy_version 1274331 (0.0009) [2023-12-27 14:28:39,526][20347] Updated weights for policy 0, policy_version 1274341 (0.0009) [2023-12-27 14:28:39,583][20347] Updated weights for policy 0, policy_version 1274351 (0.0009) [2023-12-27 14:28:40,196][20381] Updated weights for policy 1, policy_version 1273338 (0.0008) [2023-12-27 14:28:40,260][20381] Updated weights for policy 1, policy_version 1273348 (0.0007) [2023-12-27 14:28:40,324][20381] Updated weights for policy 1, policy_version 1273358 (0.0007) [2023-12-27 14:28:40,386][20381] Updated weights for policy 1, policy_version 1273368 (0.0006) [2023-12-27 14:28:40,388][20347] Updated weights for policy 0, policy_version 1274361 (0.0007) [2023-12-27 14:28:40,450][20347] Updated weights for policy 0, policy_version 1274371 (0.0008) [2023-12-27 14:28:40,509][20347] Updated weights for policy 0, policy_version 1274381 (0.0008) [2023-12-27 14:28:40,994][20381] Updated weights for policy 1, policy_version 1273378 (0.0011) [2023-12-27 14:28:41,060][20381] Updated weights for policy 1, policy_version 1273388 (0.0009) [2023-12-27 14:28:41,126][20381] Updated weights for policy 1, policy_version 1273398 (0.0009) [2023-12-27 14:28:41,230][20347] Updated weights for policy 0, policy_version 1274391 (0.0007) [2023-12-27 14:28:41,293][20347] Updated weights for policy 0, policy_version 1274401 (0.0008) [2023-12-27 14:28:41,354][20347] Updated weights for policy 0, policy_version 1274411 (0.0007) [2023-12-27 14:28:41,841][20381] Updated weights for policy 1, policy_version 1273408 (0.0007) [2023-12-27 14:28:41,905][20381] Updated weights for policy 1, policy_version 1273418 (0.0008) [2023-12-27 14:28:41,962][20381] Updated weights for policy 1, policy_version 1273428 (0.0007) [2023-12-27 14:28:42,041][20347] Updated weights for policy 0, policy_version 1274421 (0.0008) [2023-12-27 14:28:42,096][20347] Updated weights for policy 0, policy_version 1274431 (0.0008) [2023-12-27 14:28:42,155][20347] Updated weights for policy 0, policy_version 1274441 (0.0008) [2023-12-27 14:28:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 652345344. Throughput: 0: 10206.0, 1: 10076.0. Samples: 652334392. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:42,363][19170] Avg episode reward: [(0, '833.930'), (1, '843.930')] [2023-12-27 14:28:42,638][20381] Updated weights for policy 1, policy_version 1273438 (0.0009) [2023-12-27 14:28:42,698][20381] Updated weights for policy 1, policy_version 1273448 (0.0007) [2023-12-27 14:28:42,747][20381] Updated weights for policy 1, policy_version 1273458 (0.0007) [2023-12-27 14:28:42,883][20347] Updated weights for policy 0, policy_version 1274451 (0.0007) [2023-12-27 14:28:42,931][20347] Updated weights for policy 0, policy_version 1274461 (0.0006) [2023-12-27 14:28:42,979][20347] Updated weights for policy 0, policy_version 1274471 (0.0005) [2023-12-27 14:28:43,350][20381] Updated weights for policy 1, policy_version 1273468 (0.0009) [2023-12-27 14:28:43,407][20381] Updated weights for policy 1, policy_version 1273478 (0.0010) [2023-12-27 14:28:43,470][20381] Updated weights for policy 1, policy_version 1273488 (0.0011) [2023-12-27 14:28:43,567][20347] Updated weights for policy 0, policy_version 1274481 (0.0005) [2023-12-27 14:28:43,632][20347] Updated weights for policy 0, policy_version 1274491 (0.0007) [2023-12-27 14:28:43,692][20347] Updated weights for policy 0, policy_version 1274501 (0.0007) [2023-12-27 14:28:43,755][20347] Updated weights for policy 0, policy_version 1274511 (0.0005) [2023-12-27 14:28:44,110][20381] Updated weights for policy 1, policy_version 1273498 (0.0011) [2023-12-27 14:28:44,155][20381] Updated weights for policy 1, policy_version 1273508 (0.0010) [2023-12-27 14:28:44,203][20381] Updated weights for policy 1, policy_version 1273518 (0.0010) [2023-12-27 14:28:44,258][20381] Updated weights for policy 1, policy_version 1273528 (0.0010) [2023-12-27 14:28:44,356][20347] Updated weights for policy 0, policy_version 1274521 (0.0007) [2023-12-27 14:28:44,417][20347] Updated weights for policy 0, policy_version 1274531 (0.0005) [2023-12-27 14:28:44,478][20347] Updated weights for policy 0, policy_version 1274541 (0.0007) [2023-12-27 14:28:44,977][20381] Updated weights for policy 1, policy_version 1273538 (0.0007) [2023-12-27 14:28:45,047][20381] Updated weights for policy 1, policy_version 1273548 (0.0007) [2023-12-27 14:28:45,111][20381] Updated weights for policy 1, policy_version 1273558 (0.0008) [2023-12-27 14:28:45,147][20347] Updated weights for policy 0, policy_version 1274551 (0.0009) [2023-12-27 14:28:45,206][20347] Updated weights for policy 0, policy_version 1274561 (0.0010) [2023-12-27 14:28:45,260][20347] Updated weights for policy 0, policy_version 1274571 (0.0009) [2023-12-27 14:28:45,736][20381] Updated weights for policy 1, policy_version 1273568 (0.0009) [2023-12-27 14:28:45,785][20381] Updated weights for policy 1, policy_version 1273578 (0.0010) [2023-12-27 14:28:45,843][20381] Updated weights for policy 1, policy_version 1273588 (0.0010) [2023-12-27 14:28:45,886][20347] Updated weights for policy 0, policy_version 1274581 (0.0007) [2023-12-27 14:28:45,933][20347] Updated weights for policy 0, policy_version 1274591 (0.0005) [2023-12-27 14:28:45,990][20347] Updated weights for policy 0, policy_version 1274601 (0.0005) [2023-12-27 14:28:46,530][20381] Updated weights for policy 1, policy_version 1273598 (0.0008) [2023-12-27 14:28:46,591][20381] Updated weights for policy 1, policy_version 1273608 (0.0007) [2023-12-27 14:28:46,636][20347] Updated weights for policy 0, policy_version 1274611 (0.0005) [2023-12-27 14:28:46,645][20381] Updated weights for policy 1, policy_version 1273618 (0.0007) [2023-12-27 14:28:46,696][20347] Updated weights for policy 0, policy_version 1274621 (0.0007) [2023-12-27 14:28:46,740][20347] Updated weights for policy 0, policy_version 1274631 (0.0007) [2023-12-27 14:28:47,318][20381] Updated weights for policy 1, policy_version 1273628 (0.0006) [2023-12-27 14:28:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 652443648. Throughput: 0: 10257.4, 1: 10090.6. Samples: 652458800. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:47,363][19170] Avg episode reward: [(0, '882.950'), (1, '823.560')] [2023-12-27 14:28:47,374][20381] Updated weights for policy 1, policy_version 1273638 (0.0007) [2023-12-27 14:28:47,429][20381] Updated weights for policy 1, policy_version 1273648 (0.0008) [2023-12-27 14:28:47,449][20347] Updated weights for policy 0, policy_version 1274641 (0.0008) [2023-12-27 14:28:47,493][20291] KL-divergence is very high: 137.9272 [2023-12-27 14:28:47,503][20347] Updated weights for policy 0, policy_version 1274651 (0.0006) [2023-12-27 14:28:47,539][20291] KL-divergence is very high: 136.4381 [2023-12-27 14:28:47,566][20347] Updated weights for policy 0, policy_version 1274661 (0.0006) [2023-12-27 14:28:47,625][20347] Updated weights for policy 0, policy_version 1274671 (0.0009) [2023-12-27 14:28:48,095][20381] Updated weights for policy 1, policy_version 1273658 (0.0007) [2023-12-27 14:28:48,146][20381] Updated weights for policy 1, policy_version 1273668 (0.0008) [2023-12-27 14:28:48,202][20381] Updated weights for policy 1, policy_version 1273678 (0.0008) [2023-12-27 14:28:48,223][20347] Updated weights for policy 0, policy_version 1274681 (0.0006) [2023-12-27 14:28:48,255][20381] Updated weights for policy 1, policy_version 1273688 (0.0007) [2023-12-27 14:28:48,269][20347] Updated weights for policy 0, policy_version 1274691 (0.0006) [2023-12-27 14:28:48,321][20347] Updated weights for policy 0, policy_version 1274701 (0.0008) [2023-12-27 14:28:48,913][20381] Updated weights for policy 1, policy_version 1273698 (0.0008) [2023-12-27 14:28:48,966][20381] Updated weights for policy 1, policy_version 1273708 (0.0007) [2023-12-27 14:28:49,003][20347] Updated weights for policy 0, policy_version 1274711 (0.0007) [2023-12-27 14:28:49,017][20381] Updated weights for policy 1, policy_version 1273718 (0.0005) [2023-12-27 14:28:49,067][20347] Updated weights for policy 0, policy_version 1274721 (0.0008) [2023-12-27 14:28:49,123][20347] Updated weights for policy 0, policy_version 1274731 (0.0008) [2023-12-27 14:28:49,722][20381] Updated weights for policy 1, policy_version 1273728 (0.0007) [2023-12-27 14:28:49,769][20381] Updated weights for policy 1, policy_version 1273738 (0.0007) [2023-12-27 14:28:49,781][20347] Updated weights for policy 0, policy_version 1274741 (0.0007) [2023-12-27 14:28:49,827][20381] Updated weights for policy 1, policy_version 1273748 (0.0007) [2023-12-27 14:28:49,838][20347] Updated weights for policy 0, policy_version 1274751 (0.0007) [2023-12-27 14:28:49,900][20347] Updated weights for policy 0, policy_version 1274761 (0.0006) [2023-12-27 14:28:50,568][20347] Updated weights for policy 0, policy_version 1274771 (0.0008) [2023-12-27 14:28:50,571][20381] Updated weights for policy 1, policy_version 1273758 (0.0009) [2023-12-27 14:28:50,623][20347] Updated weights for policy 0, policy_version 1274781 (0.0006) [2023-12-27 14:28:50,634][20381] Updated weights for policy 1, policy_version 1273768 (0.0008) [2023-12-27 14:28:50,681][20347] Updated weights for policy 0, policy_version 1274791 (0.0008) [2023-12-27 14:28:50,684][20381] Updated weights for policy 1, policy_version 1273778 (0.0009) [2023-12-27 14:28:51,393][20347] Updated weights for policy 0, policy_version 1274801 (0.0007) [2023-12-27 14:28:51,421][20381] Updated weights for policy 1, policy_version 1273788 (0.0009) [2023-12-27 14:28:51,454][20347] Updated weights for policy 0, policy_version 1274811 (0.0007) [2023-12-27 14:28:51,476][20381] Updated weights for policy 1, policy_version 1273798 (0.0006) [2023-12-27 14:28:51,512][20347] Updated weights for policy 0, policy_version 1274821 (0.0007) [2023-12-27 14:28:51,536][20381] Updated weights for policy 1, policy_version 1273808 (0.0005) [2023-12-27 14:28:51,568][20347] Updated weights for policy 0, policy_version 1274831 (0.0007) [2023-12-27 14:28:52,170][20381] Updated weights for policy 1, policy_version 1273818 (0.0007) [2023-12-27 14:28:52,231][20381] Updated weights for policy 1, policy_version 1273828 (0.0005) [2023-12-27 14:28:52,290][20381] Updated weights for policy 1, policy_version 1273838 (0.0007) [2023-12-27 14:28:52,346][20381] Updated weights for policy 1, policy_version 1273848 (0.0009) [2023-12-27 14:28:52,350][20347] Updated weights for policy 0, policy_version 1274841 (0.0008) [2023-12-27 14:28:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 652550144. Throughput: 0: 10252.7, 1: 10068.5. Samples: 652521056. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:52,363][19170] Avg episode reward: [(0, '834.010'), (1, '785.850')] [2023-12-27 14:28:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001273848_326131712.pth... [2023-12-27 14:28:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001272664_325828608.pth [2023-12-27 14:28:52,416][20347] Updated weights for policy 0, policy_version 1274851 (0.0008) [2023-12-27 14:28:52,479][20347] Updated weights for policy 0, policy_version 1274861 (0.0008) [2023-12-27 14:28:52,494][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001274864_326426624.pth... [2023-12-27 14:28:52,498][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001273648_326115328.pth [2023-12-27 14:28:53,008][20381] Updated weights for policy 1, policy_version 1273858 (0.0011) [2023-12-27 14:28:53,064][20381] Updated weights for policy 1, policy_version 1273868 (0.0010) [2023-12-27 14:28:53,119][20381] Updated weights for policy 1, policy_version 1273878 (0.0010) [2023-12-27 14:28:53,165][20347] Updated weights for policy 0, policy_version 1274871 (0.0007) [2023-12-27 14:28:53,216][20347] Updated weights for policy 0, policy_version 1274881 (0.0007) [2023-12-27 14:28:53,270][20347] Updated weights for policy 0, policy_version 1274891 (0.0007) [2023-12-27 14:28:53,794][20381] Updated weights for policy 1, policy_version 1273888 (0.0010) [2023-12-27 14:28:53,850][20381] Updated weights for policy 1, policy_version 1273898 (0.0011) [2023-12-27 14:28:53,908][20381] Updated weights for policy 1, policy_version 1273908 (0.0011) [2023-12-27 14:28:53,987][20347] Updated weights for policy 0, policy_version 1274901 (0.0007) [2023-12-27 14:28:54,044][20347] Updated weights for policy 0, policy_version 1274911 (0.0007) [2023-12-27 14:28:54,105][20347] Updated weights for policy 0, policy_version 1274921 (0.0007) [2023-12-27 14:28:54,543][20381] Updated weights for policy 1, policy_version 1273918 (0.0009) [2023-12-27 14:28:54,585][20381] Updated weights for policy 1, policy_version 1273928 (0.0007) [2023-12-27 14:28:54,643][20381] Updated weights for policy 1, policy_version 1273938 (0.0008) [2023-12-27 14:28:54,841][20347] Updated weights for policy 0, policy_version 1274931 (0.0008) [2023-12-27 14:28:54,889][20347] Updated weights for policy 0, policy_version 1274941 (0.0009) [2023-12-27 14:28:54,938][20347] Updated weights for policy 0, policy_version 1274951 (0.0008) [2023-12-27 14:28:55,289][20381] Updated weights for policy 1, policy_version 1273948 (0.0007) [2023-12-27 14:28:55,347][20381] Updated weights for policy 1, policy_version 1273958 (0.0006) [2023-12-27 14:28:55,406][20381] Updated weights for policy 1, policy_version 1273968 (0.0007) [2023-12-27 14:28:55,668][20347] Updated weights for policy 0, policy_version 1274961 (0.0007) [2023-12-27 14:28:55,717][20347] Updated weights for policy 0, policy_version 1274971 (0.0006) [2023-12-27 14:28:55,766][20347] Updated weights for policy 0, policy_version 1274981 (0.0005) [2023-12-27 14:28:55,813][20347] Updated weights for policy 0, policy_version 1274991 (0.0005) [2023-12-27 14:28:56,074][20381] Updated weights for policy 1, policy_version 1273978 (0.0008) [2023-12-27 14:28:56,127][20381] Updated weights for policy 1, policy_version 1273988 (0.0008) [2023-12-27 14:28:56,183][20381] Updated weights for policy 1, policy_version 1273998 (0.0005) [2023-12-27 14:28:56,248][20381] Updated weights for policy 1, policy_version 1274008 (0.0005) [2023-12-27 14:28:56,484][20347] Updated weights for policy 0, policy_version 1275001 (0.0010) [2023-12-27 14:28:56,526][20347] Updated weights for policy 0, policy_version 1275011 (0.0006) [2023-12-27 14:28:56,572][20347] Updated weights for policy 0, policy_version 1275021 (0.0009) [2023-12-27 14:28:56,861][20381] Updated weights for policy 1, policy_version 1274018 (0.0007) [2023-12-27 14:28:56,916][20381] Updated weights for policy 1, policy_version 1274028 (0.0007) [2023-12-27 14:28:56,970][20381] Updated weights for policy 1, policy_version 1274038 (0.0007) [2023-12-27 14:28:57,249][20347] Updated weights for policy 0, policy_version 1275031 (0.0010) [2023-12-27 14:28:57,302][20347] Updated weights for policy 0, policy_version 1275041 (0.0010) [2023-12-27 14:28:57,350][20347] Updated weights for policy 0, policy_version 1275051 (0.0005) [2023-12-27 14:28:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 652648448. Throughput: 0: 10209.3, 1: 10136.3. Samples: 652641644. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:28:57,363][19170] Avg episode reward: [(0, '814.520'), (1, '834.170')] [2023-12-27 14:28:57,616][20381] Updated weights for policy 1, policy_version 1274048 (0.0006) [2023-12-27 14:28:57,677][20381] Updated weights for policy 1, policy_version 1274058 (0.0005) [2023-12-27 14:28:57,733][20381] Updated weights for policy 1, policy_version 1274068 (0.0006) [2023-12-27 14:28:57,968][20347] Updated weights for policy 0, policy_version 1275061 (0.0006) [2023-12-27 14:28:58,035][20347] Updated weights for policy 0, policy_version 1275071 (0.0005) [2023-12-27 14:28:58,087][20347] Updated weights for policy 0, policy_version 1275081 (0.0006) [2023-12-27 14:28:58,377][20381] Updated weights for policy 1, policy_version 1274078 (0.0007) [2023-12-27 14:28:58,438][20381] Updated weights for policy 1, policy_version 1274088 (0.0007) [2023-12-27 14:28:58,498][20381] Updated weights for policy 1, policy_version 1274098 (0.0008) [2023-12-27 14:28:58,852][20347] Updated weights for policy 0, policy_version 1275091 (0.0010) [2023-12-27 14:28:58,907][20347] Updated weights for policy 0, policy_version 1275101 (0.0008) [2023-12-27 14:28:58,969][20347] Updated weights for policy 0, policy_version 1275111 (0.0008) [2023-12-27 14:28:59,274][20381] Updated weights for policy 1, policy_version 1274108 (0.0008) [2023-12-27 14:28:59,337][20381] Updated weights for policy 1, policy_version 1274118 (0.0008) [2023-12-27 14:28:59,398][20381] Updated weights for policy 1, policy_version 1274128 (0.0007) [2023-12-27 14:28:59,700][20347] Updated weights for policy 0, policy_version 1275121 (0.0008) [2023-12-27 14:28:59,756][20347] Updated weights for policy 0, policy_version 1275131 (0.0005) [2023-12-27 14:28:59,812][20347] Updated weights for policy 0, policy_version 1275141 (0.0006) [2023-12-27 14:28:59,874][20347] Updated weights for policy 0, policy_version 1275151 (0.0007) [2023-12-27 14:29:00,077][20381] Updated weights for policy 1, policy_version 1274138 (0.0007) [2023-12-27 14:29:00,135][20381] Updated weights for policy 1, policy_version 1274148 (0.0008) [2023-12-27 14:29:00,188][20381] Updated weights for policy 1, policy_version 1274158 (0.0008) [2023-12-27 14:29:00,250][20381] Updated weights for policy 1, policy_version 1274168 (0.0006) [2023-12-27 14:29:00,501][20347] Updated weights for policy 0, policy_version 1275161 (0.0007) [2023-12-27 14:29:00,560][20347] Updated weights for policy 0, policy_version 1275171 (0.0008) [2023-12-27 14:29:00,622][20347] Updated weights for policy 0, policy_version 1275181 (0.0010) [2023-12-27 14:29:00,862][20381] Updated weights for policy 1, policy_version 1274178 (0.0005) [2023-12-27 14:29:00,920][20381] Updated weights for policy 1, policy_version 1274188 (0.0006) [2023-12-27 14:29:00,975][20381] Updated weights for policy 1, policy_version 1274198 (0.0008) [2023-12-27 14:29:01,289][20347] Updated weights for policy 0, policy_version 1275191 (0.0009) [2023-12-27 14:29:01,345][20347] Updated weights for policy 0, policy_version 1275201 (0.0009) [2023-12-27 14:29:01,407][20347] Updated weights for policy 0, policy_version 1275211 (0.0009) [2023-12-27 14:29:01,422][20291] KL-divergence is very high: 105.1720 [2023-12-27 14:29:01,764][20381] Updated weights for policy 1, policy_version 1274208 (0.0008) [2023-12-27 14:29:01,817][20381] Updated weights for policy 1, policy_version 1274218 (0.0006) [2023-12-27 14:29:01,874][20381] Updated weights for policy 1, policy_version 1274228 (0.0007) [2023-12-27 14:29:02,154][20291] KL-divergence is very high: 157.5422 [2023-12-27 14:29:02,182][20347] Updated weights for policy 0, policy_version 1275221 (0.0010) [2023-12-27 14:29:02,233][20347] Updated weights for policy 0, policy_version 1275231 (0.0010) [2023-12-27 14:29:02,290][20347] Updated weights for policy 0, policy_version 1275241 (0.0010) [2023-12-27 14:29:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 652754944. Throughput: 0: 10201.1, 1: 10120.0. Samples: 652763276. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:02,363][19170] Avg episode reward: [(0, '775.830'), (1, '803.950')] [2023-12-27 14:29:02,533][20381] Updated weights for policy 1, policy_version 1274238 (0.0007) [2023-12-27 14:29:02,588][20381] Updated weights for policy 1, policy_version 1274248 (0.0008) [2023-12-27 14:29:02,641][20381] Updated weights for policy 1, policy_version 1274258 (0.0007) [2023-12-27 14:29:03,022][20347] Updated weights for policy 0, policy_version 1275251 (0.0009) [2023-12-27 14:29:03,072][20347] Updated weights for policy 0, policy_version 1275261 (0.0007) [2023-12-27 14:29:03,116][20347] Updated weights for policy 0, policy_version 1275271 (0.0007) [2023-12-27 14:29:03,256][20381] Updated weights for policy 1, policy_version 1274268 (0.0005) [2023-12-27 14:29:03,299][20381] Updated weights for policy 1, policy_version 1274278 (0.0005) [2023-12-27 14:29:03,349][20381] Updated weights for policy 1, policy_version 1274288 (0.0007) [2023-12-27 14:29:03,808][20347] Updated weights for policy 0, policy_version 1275281 (0.0007) [2023-12-27 14:29:03,865][20347] Updated weights for policy 0, policy_version 1275291 (0.0009) [2023-12-27 14:29:03,931][20347] Updated weights for policy 0, policy_version 1275301 (0.0008) [2023-12-27 14:29:03,955][20381] Updated weights for policy 1, policy_version 1274298 (0.0009) [2023-12-27 14:29:03,987][20347] Updated weights for policy 0, policy_version 1275311 (0.0007) [2023-12-27 14:29:04,009][20381] Updated weights for policy 1, policy_version 1274308 (0.0006) [2023-12-27 14:29:04,066][20381] Updated weights for policy 1, policy_version 1274318 (0.0007) [2023-12-27 14:29:04,126][20381] Updated weights for policy 1, policy_version 1274328 (0.0007) [2023-12-27 14:29:04,616][20347] Updated weights for policy 0, policy_version 1275321 (0.0005) [2023-12-27 14:29:04,675][20347] Updated weights for policy 0, policy_version 1275332 (0.0007) [2023-12-27 14:29:04,728][20347] Updated weights for policy 0, policy_version 1275342 (0.0010) [2023-12-27 14:29:04,799][20381] Updated weights for policy 1, policy_version 1274338 (0.0007) [2023-12-27 14:29:04,843][20381] Updated weights for policy 1, policy_version 1274348 (0.0007) [2023-12-27 14:29:04,890][20381] Updated weights for policy 1, policy_version 1274358 (0.0007) [2023-12-27 14:29:05,383][20347] Updated weights for policy 0, policy_version 1275352 (0.0010) [2023-12-27 14:29:05,444][20347] Updated weights for policy 0, policy_version 1275362 (0.0010) [2023-12-27 14:29:05,502][20347] Updated weights for policy 0, policy_version 1275372 (0.0009) [2023-12-27 14:29:05,593][20381] Updated weights for policy 1, policy_version 1274368 (0.0007) [2023-12-27 14:29:05,640][20381] Updated weights for policy 1, policy_version 1274378 (0.0007) [2023-12-27 14:29:05,688][20381] Updated weights for policy 1, policy_version 1274388 (0.0007) [2023-12-27 14:29:06,134][20347] Updated weights for policy 0, policy_version 1275382 (0.0008) [2023-12-27 14:29:06,203][20347] Updated weights for policy 0, policy_version 1275392 (0.0010) [2023-12-27 14:29:06,263][20347] Updated weights for policy 0, policy_version 1275402 (0.0011) [2023-12-27 14:29:06,429][20381] Updated weights for policy 1, policy_version 1274398 (0.0006) [2023-12-27 14:29:06,480][20381] Updated weights for policy 1, policy_version 1274408 (0.0007) [2023-12-27 14:29:06,530][20381] Updated weights for policy 1, policy_version 1274418 (0.0006) [2023-12-27 14:29:06,991][20347] Updated weights for policy 0, policy_version 1275412 (0.0009) [2023-12-27 14:29:07,047][20347] Updated weights for policy 0, policy_version 1275422 (0.0006) [2023-12-27 14:29:07,082][20381] Updated weights for policy 1, policy_version 1274428 (0.0007) [2023-12-27 14:29:07,103][20347] Updated weights for policy 0, policy_version 1275432 (0.0010) [2023-12-27 14:29:07,138][20381] Updated weights for policy 1, policy_version 1274438 (0.0006) [2023-12-27 14:29:07,192][20381] Updated weights for policy 1, policy_version 1274448 (0.0007) [2023-12-27 14:29:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 652861440. Throughput: 0: 10211.8, 1: 10132.8. Samples: 652826584. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:07,363][19170] Avg episode reward: [(0, '824.610'), (1, '804.170')] [2023-12-27 14:29:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001274456_326287360.pth... [2023-12-27 14:29:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001275440_326574080.pth... [2023-12-27 14:29:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001273240_325976064.pth [2023-12-27 14:29:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001274256_326270976.pth [2023-12-27 14:29:07,739][20347] Updated weights for policy 0, policy_version 1275442 (0.0010) [2023-12-27 14:29:07,797][20347] Updated weights for policy 0, policy_version 1275452 (0.0010) [2023-12-27 14:29:07,853][20347] Updated weights for policy 0, policy_version 1275462 (0.0005) [2023-12-27 14:29:07,905][20381] Updated weights for policy 1, policy_version 1274458 (0.0006) [2023-12-27 14:29:07,910][20347] Updated weights for policy 0, policy_version 1275472 (0.0007) [2023-12-27 14:29:07,958][20381] Updated weights for policy 1, policy_version 1274468 (0.0008) [2023-12-27 14:29:08,012][20381] Updated weights for policy 1, policy_version 1274478 (0.0008) [2023-12-27 14:29:08,071][20381] Updated weights for policy 1, policy_version 1274488 (0.0008) [2023-12-27 14:29:08,550][20347] Updated weights for policy 0, policy_version 1275482 (0.0007) [2023-12-27 14:29:08,601][20291] KL-divergence is very high: 134.5829 [2023-12-27 14:29:08,614][20347] Updated weights for policy 0, policy_version 1275492 (0.0006) [2023-12-27 14:29:08,633][20291] KL-divergence is very high: 114.4376 [2023-12-27 14:29:08,651][20291] KL-divergence is very high: 297.7070 [2023-12-27 14:29:08,675][20347] Updated weights for policy 0, policy_version 1275502 (0.0007) [2023-12-27 14:29:08,765][20381] Updated weights for policy 1, policy_version 1274498 (0.0007) [2023-12-27 14:29:08,831][20381] Updated weights for policy 1, policy_version 1274508 (0.0007) [2023-12-27 14:29:08,896][20381] Updated weights for policy 1, policy_version 1274518 (0.0008) [2023-12-27 14:29:09,302][20347] Updated weights for policy 0, policy_version 1275512 (0.0005) [2023-12-27 14:29:09,368][20347] Updated weights for policy 0, policy_version 1275522 (0.0007) [2023-12-27 14:29:09,439][20347] Updated weights for policy 0, policy_version 1275532 (0.0007) [2023-12-27 14:29:09,601][20381] Updated weights for policy 1, policy_version 1274528 (0.0008) [2023-12-27 14:29:09,656][20381] Updated weights for policy 1, policy_version 1274538 (0.0008) [2023-12-27 14:29:09,711][20381] Updated weights for policy 1, policy_version 1274548 (0.0008) [2023-12-27 14:29:10,135][20347] Updated weights for policy 0, policy_version 1275542 (0.0007) [2023-12-27 14:29:10,196][20347] Updated weights for policy 0, policy_version 1275552 (0.0009) [2023-12-27 14:29:10,251][20347] Updated weights for policy 0, policy_version 1275562 (0.0007) [2023-12-27 14:29:10,467][20381] Updated weights for policy 1, policy_version 1274558 (0.0008) [2023-12-27 14:29:10,526][20381] Updated weights for policy 1, policy_version 1274568 (0.0007) [2023-12-27 14:29:10,578][20381] Updated weights for policy 1, policy_version 1274578 (0.0006) [2023-12-27 14:29:10,954][20347] Updated weights for policy 0, policy_version 1275572 (0.0007) [2023-12-27 14:29:11,006][20347] Updated weights for policy 0, policy_version 1275582 (0.0008) [2023-12-27 14:29:11,068][20347] Updated weights for policy 0, policy_version 1275592 (0.0009) [2023-12-27 14:29:11,292][20381] Updated weights for policy 1, policy_version 1274588 (0.0007) [2023-12-27 14:29:11,347][20381] Updated weights for policy 1, policy_version 1274598 (0.0009) [2023-12-27 14:29:11,415][20381] Updated weights for policy 1, policy_version 1274608 (0.0008) [2023-12-27 14:29:11,892][20347] Updated weights for policy 0, policy_version 1275602 (0.0008) [2023-12-27 14:29:11,951][20347] Updated weights for policy 0, policy_version 1275612 (0.0011) [2023-12-27 14:29:12,017][20347] Updated weights for policy 0, policy_version 1275622 (0.0008) [2023-12-27 14:29:12,078][20347] Updated weights for policy 0, policy_version 1275632 (0.0008) [2023-12-27 14:29:12,132][20381] Updated weights for policy 1, policy_version 1274618 (0.0007) [2023-12-27 14:29:12,191][20381] Updated weights for policy 1, policy_version 1274628 (0.0007) [2023-12-27 14:29:12,250][20381] Updated weights for policy 1, policy_version 1274638 (0.0008) [2023-12-27 14:29:12,316][20381] Updated weights for policy 1, policy_version 1274648 (0.0008) [2023-12-27 14:29:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 652959744. Throughput: 0: 10200.3, 1: 10185.9. Samples: 652946964. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:12,363][19170] Avg episode reward: [(0, '775.630'), (1, '785.190')] [2023-12-27 14:29:12,731][20347] Updated weights for policy 0, policy_version 1275642 (0.0008) [2023-12-27 14:29:12,791][20347] Updated weights for policy 0, policy_version 1275652 (0.0006) [2023-12-27 14:29:12,844][20347] Updated weights for policy 0, policy_version 1275662 (0.0007) [2023-12-27 14:29:13,045][20381] Updated weights for policy 1, policy_version 1274658 (0.0010) [2023-12-27 14:29:13,099][20381] Updated weights for policy 1, policy_version 1274668 (0.0010) [2023-12-27 14:29:13,160][20381] Updated weights for policy 1, policy_version 1274678 (0.0010) [2023-12-27 14:29:13,553][20347] Updated weights for policy 0, policy_version 1275672 (0.0006) [2023-12-27 14:29:13,612][20347] Updated weights for policy 0, policy_version 1275682 (0.0006) [2023-12-27 14:29:13,663][20347] Updated weights for policy 0, policy_version 1275692 (0.0006) [2023-12-27 14:29:13,751][20381] Updated weights for policy 1, policy_version 1274688 (0.0011) [2023-12-27 14:29:13,806][20381] Updated weights for policy 1, policy_version 1274698 (0.0010) [2023-12-27 14:29:13,863][20381] Updated weights for policy 1, policy_version 1274708 (0.0010) [2023-12-27 14:29:14,352][20347] Updated weights for policy 0, policy_version 1275702 (0.0007) [2023-12-27 14:29:14,413][20347] Updated weights for policy 0, policy_version 1275712 (0.0007) [2023-12-27 14:29:14,466][20347] Updated weights for policy 0, policy_version 1275722 (0.0007) [2023-12-27 14:29:14,535][20381] Updated weights for policy 1, policy_version 1274718 (0.0010) [2023-12-27 14:29:14,590][20381] Updated weights for policy 1, policy_version 1274728 (0.0010) [2023-12-27 14:29:14,638][20381] Updated weights for policy 1, policy_version 1274738 (0.0010) [2023-12-27 14:29:15,177][20347] Updated weights for policy 0, policy_version 1275732 (0.0006) [2023-12-27 14:29:15,241][20347] Updated weights for policy 0, policy_version 1275742 (0.0007) [2023-12-27 14:29:15,304][20347] Updated weights for policy 0, policy_version 1275752 (0.0007) [2023-12-27 14:29:15,336][20381] Updated weights for policy 1, policy_version 1274748 (0.0010) [2023-12-27 14:29:15,391][20381] Updated weights for policy 1, policy_version 1274758 (0.0010) [2023-12-27 14:29:15,442][20381] Updated weights for policy 1, policy_version 1274768 (0.0010) [2023-12-27 14:29:15,882][20347] Updated weights for policy 0, policy_version 1275762 (0.0006) [2023-12-27 14:29:15,941][20347] Updated weights for policy 0, policy_version 1275772 (0.0006) [2023-12-27 14:29:16,002][20347] Updated weights for policy 0, policy_version 1275782 (0.0006) [2023-12-27 14:29:16,039][20381] Updated weights for policy 1, policy_version 1274778 (0.0010) [2023-12-27 14:29:16,065][20347] Updated weights for policy 0, policy_version 1275792 (0.0006) [2023-12-27 14:29:16,098][20381] Updated weights for policy 1, policy_version 1274788 (0.0011) [2023-12-27 14:29:16,153][20381] Updated weights for policy 1, policy_version 1274798 (0.0010) [2023-12-27 14:29:16,208][20381] Updated weights for policy 1, policy_version 1274808 (0.0010) [2023-12-27 14:29:16,663][20347] Updated weights for policy 0, policy_version 1275802 (0.0008) [2023-12-27 14:29:16,723][20347] Updated weights for policy 0, policy_version 1275812 (0.0007) [2023-12-27 14:29:16,771][20347] Updated weights for policy 0, policy_version 1275822 (0.0005) [2023-12-27 14:29:16,919][20381] Updated weights for policy 1, policy_version 1274818 (0.0010) [2023-12-27 14:29:16,972][20381] Updated weights for policy 1, policy_version 1274828 (0.0007) [2023-12-27 14:29:17,025][20381] Updated weights for policy 1, policy_version 1274838 (0.0005) [2023-12-27 14:29:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 653058048. Throughput: 0: 10141.3, 1: 10260.8. Samples: 653070252. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:17,363][19170] Avg episode reward: [(0, '805.130'), (1, '745.750')] [2023-12-27 14:29:17,447][20347] Updated weights for policy 0, policy_version 1275832 (0.0006) [2023-12-27 14:29:17,493][20347] Updated weights for policy 0, policy_version 1275842 (0.0010) [2023-12-27 14:29:17,554][20347] Updated weights for policy 0, policy_version 1275852 (0.0010) [2023-12-27 14:29:17,645][20381] Updated weights for policy 1, policy_version 1274848 (0.0009) [2023-12-27 14:29:17,701][20381] Updated weights for policy 1, policy_version 1274858 (0.0010) [2023-12-27 14:29:17,756][20381] Updated weights for policy 1, policy_version 1274868 (0.0010) [2023-12-27 14:29:18,179][20347] Updated weights for policy 0, policy_version 1275862 (0.0007) [2023-12-27 14:29:18,243][20347] Updated weights for policy 0, policy_version 1275872 (0.0010) [2023-12-27 14:29:18,301][20347] Updated weights for policy 0, policy_version 1275882 (0.0010) [2023-12-27 14:29:18,445][20381] Updated weights for policy 1, policy_version 1274878 (0.0009) [2023-12-27 14:29:18,498][20381] Updated weights for policy 1, policy_version 1274888 (0.0011) [2023-12-27 14:29:18,553][20381] Updated weights for policy 1, policy_version 1274898 (0.0010) [2023-12-27 14:29:18,956][20347] Updated weights for policy 0, policy_version 1275892 (0.0010) [2023-12-27 14:29:19,005][20347] Updated weights for policy 0, policy_version 1275902 (0.0007) [2023-12-27 14:29:19,058][20347] Updated weights for policy 0, policy_version 1275912 (0.0007) [2023-12-27 14:29:19,170][20381] Updated weights for policy 1, policy_version 1274908 (0.0011) [2023-12-27 14:29:19,235][20381] Updated weights for policy 1, policy_version 1274918 (0.0007) [2023-12-27 14:29:19,292][20381] Updated weights for policy 1, policy_version 1274928 (0.0008) [2023-12-27 14:29:19,822][20347] Updated weights for policy 0, policy_version 1275922 (0.0010) [2023-12-27 14:29:19,893][20347] Updated weights for policy 0, policy_version 1275932 (0.0009) [2023-12-27 14:29:19,958][20347] Updated weights for policy 0, policy_version 1275942 (0.0007) [2023-12-27 14:29:20,021][20347] Updated weights for policy 0, policy_version 1275952 (0.0007) [2023-12-27 14:29:20,085][20381] Updated weights for policy 1, policy_version 1274938 (0.0008) [2023-12-27 14:29:20,144][20381] Updated weights for policy 1, policy_version 1274948 (0.0008) [2023-12-27 14:29:20,197][20381] Updated weights for policy 1, policy_version 1274958 (0.0008) [2023-12-27 14:29:20,258][20381] Updated weights for policy 1, policy_version 1274968 (0.0008) [2023-12-27 14:29:20,716][20347] Updated weights for policy 0, policy_version 1275962 (0.0007) [2023-12-27 14:29:20,774][20347] Updated weights for policy 0, policy_version 1275972 (0.0011) [2023-12-27 14:29:20,834][20347] Updated weights for policy 0, policy_version 1275982 (0.0010) [2023-12-27 14:29:20,938][20381] Updated weights for policy 1, policy_version 1274978 (0.0008) [2023-12-27 14:29:21,004][20381] Updated weights for policy 1, policy_version 1274988 (0.0008) [2023-12-27 14:29:21,068][20381] Updated weights for policy 1, policy_version 1274998 (0.0008) [2023-12-27 14:29:21,572][20347] Updated weights for policy 0, policy_version 1275992 (0.0010) [2023-12-27 14:29:21,633][20347] Updated weights for policy 0, policy_version 1276002 (0.0011) [2023-12-27 14:29:21,702][20347] Updated weights for policy 0, policy_version 1276012 (0.0011) [2023-12-27 14:29:21,715][20291] KL-divergence is very high: 309.4822 [2023-12-27 14:29:21,911][20381] Updated weights for policy 1, policy_version 1275008 (0.0008) [2023-12-27 14:29:21,973][20381] Updated weights for policy 1, policy_version 1275018 (0.0007) [2023-12-27 14:29:22,037][20381] Updated weights for policy 1, policy_version 1275028 (0.0007) [2023-12-27 14:29:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 653156352. Throughput: 0: 10133.3, 1: 10258.2. Samples: 653130548. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:22,363][19170] Avg episode reward: [(0, '814.260'), (1, '755.560')] [2023-12-27 14:29:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001275032_326434816.pth... [2023-12-27 14:29:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001276016_326721536.pth... [2023-12-27 14:29:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001273848_326131712.pth [2023-12-27 14:29:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001274864_326426624.pth [2023-12-27 14:29:22,443][20347] Updated weights for policy 0, policy_version 1276022 (0.0009) [2023-12-27 14:29:22,501][20347] Updated weights for policy 0, policy_version 1276032 (0.0007) [2023-12-27 14:29:22,560][20347] Updated weights for policy 0, policy_version 1276042 (0.0008) [2023-12-27 14:29:22,792][20381] Updated weights for policy 1, policy_version 1275038 (0.0008) [2023-12-27 14:29:22,848][20381] Updated weights for policy 1, policy_version 1275048 (0.0008) [2023-12-27 14:29:22,911][20381] Updated weights for policy 1, policy_version 1275058 (0.0005) [2023-12-27 14:29:23,214][20347] Updated weights for policy 0, policy_version 1276052 (0.0008) [2023-12-27 14:29:23,271][20347] Updated weights for policy 0, policy_version 1276062 (0.0008) [2023-12-27 14:29:23,315][20347] Updated weights for policy 0, policy_version 1276072 (0.0008) [2023-12-27 14:29:23,521][20381] Updated weights for policy 1, policy_version 1275068 (0.0006) [2023-12-27 14:29:23,582][20381] Updated weights for policy 1, policy_version 1275078 (0.0009) [2023-12-27 14:29:23,645][20381] Updated weights for policy 1, policy_version 1275088 (0.0007) [2023-12-27 14:29:24,025][20347] Updated weights for policy 0, policy_version 1276082 (0.0007) [2023-12-27 14:29:24,079][20347] Updated weights for policy 0, policy_version 1276092 (0.0008) [2023-12-27 14:29:24,130][20347] Updated weights for policy 0, policy_version 1276102 (0.0007) [2023-12-27 14:29:24,189][20347] Updated weights for policy 0, policy_version 1276112 (0.0008) [2023-12-27 14:29:24,337][20381] Updated weights for policy 1, policy_version 1275098 (0.0008) [2023-12-27 14:29:24,394][20381] Updated weights for policy 1, policy_version 1275108 (0.0008) [2023-12-27 14:29:24,459][20381] Updated weights for policy 1, policy_version 1275118 (0.0007) [2023-12-27 14:29:24,517][20381] Updated weights for policy 1, policy_version 1275128 (0.0008) [2023-12-27 14:29:24,816][20347] Updated weights for policy 0, policy_version 1276122 (0.0006) [2023-12-27 14:29:24,874][20347] Updated weights for policy 0, policy_version 1276132 (0.0008) [2023-12-27 14:29:24,923][20347] Updated weights for policy 0, policy_version 1276142 (0.0007) [2023-12-27 14:29:25,247][20381] Updated weights for policy 1, policy_version 1275138 (0.0007) [2023-12-27 14:29:25,298][20381] Updated weights for policy 1, policy_version 1275148 (0.0008) [2023-12-27 14:29:25,346][20381] Updated weights for policy 1, policy_version 1275158 (0.0007) [2023-12-27 14:29:25,585][20347] Updated weights for policy 0, policy_version 1276152 (0.0007) [2023-12-27 14:29:25,633][20347] Updated weights for policy 0, policy_version 1276162 (0.0007) [2023-12-27 14:29:25,689][20347] Updated weights for policy 0, policy_version 1276172 (0.0008) [2023-12-27 14:29:26,029][20381] Updated weights for policy 1, policy_version 1275168 (0.0008) [2023-12-27 14:29:26,090][20381] Updated weights for policy 1, policy_version 1275178 (0.0008) [2023-12-27 14:29:26,148][20381] Updated weights for policy 1, policy_version 1275188 (0.0007) [2023-12-27 14:29:26,398][20347] Updated weights for policy 0, policy_version 1276182 (0.0008) [2023-12-27 14:29:26,458][20347] Updated weights for policy 0, policy_version 1276192 (0.0006) [2023-12-27 14:29:26,517][20347] Updated weights for policy 0, policy_version 1276202 (0.0007) [2023-12-27 14:29:26,836][20381] Updated weights for policy 1, policy_version 1275198 (0.0009) [2023-12-27 14:29:26,901][20381] Updated weights for policy 1, policy_version 1275208 (0.0008) [2023-12-27 14:29:26,949][20381] Updated weights for policy 1, policy_version 1275218 (0.0010) [2023-12-27 14:29:27,179][20347] Updated weights for policy 0, policy_version 1276212 (0.0007) [2023-12-27 14:29:27,234][20347] Updated weights for policy 0, policy_version 1276222 (0.0007) [2023-12-27 14:29:27,289][20347] Updated weights for policy 0, policy_version 1276232 (0.0007) [2023-12-27 14:29:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 653262848. Throughput: 0: 10165.6, 1: 10175.0. Samples: 653249720. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:27,363][19170] Avg episode reward: [(0, '785.080'), (1, '804.630')] [2023-12-27 14:29:27,593][20381] Updated weights for policy 1, policy_version 1275228 (0.0008) [2023-12-27 14:29:27,657][20381] Updated weights for policy 1, policy_version 1275238 (0.0006) [2023-12-27 14:29:27,704][20381] Updated weights for policy 1, policy_version 1275248 (0.0006) [2023-12-27 14:29:28,044][20347] Updated weights for policy 0, policy_version 1276242 (0.0007) [2023-12-27 14:29:28,081][20291] KL-divergence is very high: 125.1306 [2023-12-27 14:29:28,090][20347] Updated weights for policy 0, policy_version 1276252 (0.0008) [2023-12-27 14:29:28,090][20291] KL-divergence is very high: 145.3460 [2023-12-27 14:29:28,122][20291] KL-divergence is very high: 126.2658 [2023-12-27 14:29:28,131][20291] KL-divergence is very high: 106.8199 [2023-12-27 14:29:28,140][20347] Updated weights for policy 0, policy_version 1276262 (0.0008) [2023-12-27 14:29:28,190][20347] Updated weights for policy 0, policy_version 1276272 (0.0008) [2023-12-27 14:29:28,216][20381] Updated weights for policy 1, policy_version 1275258 (0.0005) [2023-12-27 14:29:28,271][20381] Updated weights for policy 1, policy_version 1275268 (0.0005) [2023-12-27 14:29:28,327][20381] Updated weights for policy 1, policy_version 1275278 (0.0008) [2023-12-27 14:29:28,393][20381] Updated weights for policy 1, policy_version 1275288 (0.0007) [2023-12-27 14:29:28,907][20347] Updated weights for policy 0, policy_version 1276282 (0.0007) [2023-12-27 14:29:28,958][20347] Updated weights for policy 0, policy_version 1276292 (0.0007) [2023-12-27 14:29:29,005][20347] Updated weights for policy 0, policy_version 1276302 (0.0008) [2023-12-27 14:29:29,055][20381] Updated weights for policy 1, policy_version 1275298 (0.0007) [2023-12-27 14:29:29,100][20381] Updated weights for policy 1, policy_version 1275308 (0.0008) [2023-12-27 14:29:29,149][20381] Updated weights for policy 1, policy_version 1275318 (0.0008) [2023-12-27 14:29:29,736][20347] Updated weights for policy 0, policy_version 1276312 (0.0007) [2023-12-27 14:29:29,793][20347] Updated weights for policy 0, policy_version 1276322 (0.0007) [2023-12-27 14:29:29,855][20347] Updated weights for policy 0, policy_version 1276332 (0.0008) [2023-12-27 14:29:29,922][20381] Updated weights for policy 1, policy_version 1275328 (0.0008) [2023-12-27 14:29:29,989][20381] Updated weights for policy 1, policy_version 1275338 (0.0009) [2023-12-27 14:29:30,054][20381] Updated weights for policy 1, policy_version 1275348 (0.0006) [2023-12-27 14:29:30,592][20347] Updated weights for policy 0, policy_version 1276342 (0.0008) [2023-12-27 14:29:30,622][20381] Updated weights for policy 1, policy_version 1275358 (0.0007) [2023-12-27 14:29:30,644][20347] Updated weights for policy 0, policy_version 1276352 (0.0007) [2023-12-27 14:29:30,676][20381] Updated weights for policy 1, policy_version 1275368 (0.0008) [2023-12-27 14:29:30,711][20347] Updated weights for policy 0, policy_version 1276362 (0.0006) [2023-12-27 14:29:30,731][20381] Updated weights for policy 1, policy_version 1275378 (0.0006) [2023-12-27 14:29:31,367][20381] Updated weights for policy 1, policy_version 1275388 (0.0007) [2023-12-27 14:29:31,425][20381] Updated weights for policy 1, policy_version 1275398 (0.0008) [2023-12-27 14:29:31,435][20347] Updated weights for policy 0, policy_version 1276372 (0.0006) [2023-12-27 14:29:31,482][20381] Updated weights for policy 1, policy_version 1275408 (0.0007) [2023-12-27 14:29:31,500][20347] Updated weights for policy 0, policy_version 1276382 (0.0007) [2023-12-27 14:29:31,565][20347] Updated weights for policy 0, policy_version 1276392 (0.0007) [2023-12-27 14:29:32,150][20381] Updated weights for policy 1, policy_version 1275418 (0.0007) [2023-12-27 14:29:32,192][20347] Updated weights for policy 0, policy_version 1276402 (0.0008) [2023-12-27 14:29:32,211][20381] Updated weights for policy 1, policy_version 1275428 (0.0008) [2023-12-27 14:29:32,247][20347] Updated weights for policy 0, policy_version 1276412 (0.0007) [2023-12-27 14:29:32,276][20381] Updated weights for policy 1, policy_version 1275438 (0.0008) [2023-12-27 14:29:32,304][20347] Updated weights for policy 0, policy_version 1276422 (0.0006) [2023-12-27 14:29:32,333][20381] Updated weights for policy 1, policy_version 1275448 (0.0011) [2023-12-27 14:29:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 653361152. Throughput: 0: 10090.0, 1: 10219.3. Samples: 653372716. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:32,363][19170] Avg episode reward: [(0, '824.710'), (1, '695.880')] [2023-12-27 14:29:32,366][20347] Updated weights for policy 0, policy_version 1276432 (0.0007) [2023-12-27 14:29:33,015][20381] Updated weights for policy 1, policy_version 1275458 (0.0007) [2023-12-27 14:29:33,061][20347] Updated weights for policy 0, policy_version 1276442 (0.0006) [2023-12-27 14:29:33,063][20381] Updated weights for policy 1, policy_version 1275468 (0.0005) [2023-12-27 14:29:33,113][20347] Updated weights for policy 0, policy_version 1276452 (0.0005) [2023-12-27 14:29:33,115][20381] Updated weights for policy 1, policy_version 1275478 (0.0005) [2023-12-27 14:29:33,166][20347] Updated weights for policy 0, policy_version 1276462 (0.0008) [2023-12-27 14:29:33,751][20381] Updated weights for policy 1, policy_version 1275488 (0.0010) [2023-12-27 14:29:33,806][20381] Updated weights for policy 1, policy_version 1275498 (0.0010) [2023-12-27 14:29:33,818][20347] Updated weights for policy 0, policy_version 1276472 (0.0010) [2023-12-27 14:29:33,861][20381] Updated weights for policy 1, policy_version 1275508 (0.0008) [2023-12-27 14:29:33,876][20347] Updated weights for policy 0, policy_version 1276482 (0.0010) [2023-12-27 14:29:33,922][20347] Updated weights for policy 0, policy_version 1276492 (0.0007) [2023-12-27 14:29:34,560][20381] Updated weights for policy 1, policy_version 1275518 (0.0009) [2023-12-27 14:29:34,580][20347] Updated weights for policy 0, policy_version 1276502 (0.0007) [2023-12-27 14:29:34,626][20381] Updated weights for policy 1, policy_version 1275528 (0.0011) [2023-12-27 14:29:34,643][20347] Updated weights for policy 0, policy_version 1276512 (0.0006) [2023-12-27 14:29:34,689][20381] Updated weights for policy 1, policy_version 1275538 (0.0011) [2023-12-27 14:29:34,700][20347] Updated weights for policy 0, policy_version 1276522 (0.0007) [2023-12-27 14:29:35,321][20381] Updated weights for policy 1, policy_version 1275548 (0.0011) [2023-12-27 14:29:35,370][20381] Updated weights for policy 1, policy_version 1275558 (0.0007) [2023-12-27 14:29:35,396][20347] Updated weights for policy 0, policy_version 1276532 (0.0007) [2023-12-27 14:29:35,432][20381] Updated weights for policy 1, policy_version 1275568 (0.0009) [2023-12-27 14:29:35,450][20347] Updated weights for policy 0, policy_version 1276542 (0.0007) [2023-12-27 14:29:35,508][20347] Updated weights for policy 0, policy_version 1276552 (0.0007) [2023-12-27 14:29:35,985][20381] Updated weights for policy 1, policy_version 1275578 (0.0010) [2023-12-27 14:29:36,040][20381] Updated weights for policy 1, policy_version 1275588 (0.0010) [2023-12-27 14:29:36,092][20381] Updated weights for policy 1, policy_version 1275598 (0.0010) [2023-12-27 14:29:36,156][20381] Updated weights for policy 1, policy_version 1275608 (0.0008) [2023-12-27 14:29:36,208][20347] Updated weights for policy 0, policy_version 1276562 (0.0008) [2023-12-27 14:29:36,264][20347] Updated weights for policy 0, policy_version 1276572 (0.0008) [2023-12-27 14:29:36,320][20347] Updated weights for policy 0, policy_version 1276582 (0.0008) [2023-12-27 14:29:36,372][20347] Updated weights for policy 0, policy_version 1276592 (0.0008) [2023-12-27 14:29:36,921][20381] Updated weights for policy 1, policy_version 1275618 (0.0005) [2023-12-27 14:29:36,981][20381] Updated weights for policy 1, policy_version 1275628 (0.0009) [2023-12-27 14:29:37,043][20381] Updated weights for policy 1, policy_version 1275638 (0.0010) [2023-12-27 14:29:37,088][20347] Updated weights for policy 0, policy_version 1276602 (0.0007) [2023-12-27 14:29:37,147][20347] Updated weights for policy 0, policy_version 1276612 (0.0008) [2023-12-27 14:29:37,213][20347] Updated weights for policy 0, policy_version 1276622 (0.0007) [2023-12-27 14:29:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 653467648. Throughput: 0: 10074.9, 1: 10238.2. Samples: 653435140. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:37,363][19170] Avg episode reward: [(0, '854.050'), (1, '793.630')] [2023-12-27 14:29:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001276624_326877184.pth... [2023-12-27 14:29:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001275640_326590464.pth... [2023-12-27 14:29:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001275440_326574080.pth [2023-12-27 14:29:37,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001274456_326287360.pth [2023-12-27 14:29:37,723][20381] Updated weights for policy 1, policy_version 1275648 (0.0011) [2023-12-27 14:29:37,785][20381] Updated weights for policy 1, policy_version 1275658 (0.0009) [2023-12-27 14:29:37,845][20381] Updated weights for policy 1, policy_version 1275668 (0.0010) [2023-12-27 14:29:37,933][20347] Updated weights for policy 0, policy_version 1276632 (0.0007) [2023-12-27 14:29:37,991][20347] Updated weights for policy 0, policy_version 1276642 (0.0007) [2023-12-27 14:29:38,044][20347] Updated weights for policy 0, policy_version 1276652 (0.0007) [2023-12-27 14:29:38,433][20381] Updated weights for policy 1, policy_version 1275678 (0.0010) [2023-12-27 14:29:38,495][20381] Updated weights for policy 1, policy_version 1275688 (0.0006) [2023-12-27 14:29:38,562][20381] Updated weights for policy 1, policy_version 1275698 (0.0006) [2023-12-27 14:29:38,810][20347] Updated weights for policy 0, policy_version 1276662 (0.0007) [2023-12-27 14:29:38,867][20347] Updated weights for policy 0, policy_version 1276672 (0.0006) [2023-12-27 14:29:38,927][20347] Updated weights for policy 0, policy_version 1276682 (0.0005) [2023-12-27 14:29:39,147][20381] Updated weights for policy 1, policy_version 1275708 (0.0008) [2023-12-27 14:29:39,192][20381] Updated weights for policy 1, policy_version 1275718 (0.0010) [2023-12-27 14:29:39,257][20381] Updated weights for policy 1, policy_version 1275728 (0.0008) [2023-12-27 14:29:39,645][20347] Updated weights for policy 0, policy_version 1276692 (0.0008) [2023-12-27 14:29:39,707][20347] Updated weights for policy 0, policy_version 1276702 (0.0008) [2023-12-27 14:29:39,770][20347] Updated weights for policy 0, policy_version 1276713 (0.0008) [2023-12-27 14:29:39,993][20381] Updated weights for policy 1, policy_version 1275738 (0.0006) [2023-12-27 14:29:40,049][20381] Updated weights for policy 1, policy_version 1275748 (0.0008) [2023-12-27 14:29:40,115][20381] Updated weights for policy 1, policy_version 1275758 (0.0008) [2023-12-27 14:29:40,176][20381] Updated weights for policy 1, policy_version 1275768 (0.0010) [2023-12-27 14:29:40,452][20347] Updated weights for policy 0, policy_version 1276723 (0.0007) [2023-12-27 14:29:40,514][20347] Updated weights for policy 0, policy_version 1276733 (0.0009) [2023-12-27 14:29:40,571][20347] Updated weights for policy 0, policy_version 1276743 (0.0008) [2023-12-27 14:29:40,828][20381] Updated weights for policy 1, policy_version 1275778 (0.0008) [2023-12-27 14:29:40,873][20381] Updated weights for policy 1, policy_version 1275788 (0.0010) [2023-12-27 14:29:40,918][20381] Updated weights for policy 1, policy_version 1275798 (0.0010) [2023-12-27 14:29:41,295][20347] Updated weights for policy 0, policy_version 1276753 (0.0009) [2023-12-27 14:29:41,356][20347] Updated weights for policy 0, policy_version 1276763 (0.0008) [2023-12-27 14:29:41,427][20347] Updated weights for policy 0, policy_version 1276773 (0.0008) [2023-12-27 14:29:41,494][20347] Updated weights for policy 0, policy_version 1276783 (0.0006) [2023-12-27 14:29:41,675][20381] Updated weights for policy 1, policy_version 1275808 (0.0008) [2023-12-27 14:29:41,737][20381] Updated weights for policy 1, policy_version 1275818 (0.0007) [2023-12-27 14:29:41,797][20381] Updated weights for policy 1, policy_version 1275828 (0.0006) [2023-12-27 14:29:42,157][20347] Updated weights for policy 0, policy_version 1276793 (0.0008) [2023-12-27 14:29:42,220][20347] Updated weights for policy 0, policy_version 1276803 (0.0007) [2023-12-27 14:29:42,282][20347] Updated weights for policy 0, policy_version 1276813 (0.0008) [2023-12-27 14:29:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 653565952. Throughput: 0: 10064.7, 1: 10251.3. Samples: 653555864. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:42,363][19170] Avg episode reward: [(0, '814.390'), (1, '766.720')] [2023-12-27 14:29:42,453][20381] Updated weights for policy 1, policy_version 1275838 (0.0007) [2023-12-27 14:29:42,502][20381] Updated weights for policy 1, policy_version 1275848 (0.0007) [2023-12-27 14:29:42,554][20381] Updated weights for policy 1, policy_version 1275858 (0.0008) [2023-12-27 14:29:43,049][20347] Updated weights for policy 0, policy_version 1276823 (0.0008) [2023-12-27 14:29:43,105][20347] Updated weights for policy 0, policy_version 1276833 (0.0009) [2023-12-27 14:29:43,162][20381] Updated weights for policy 1, policy_version 1275868 (0.0007) [2023-12-27 14:29:43,163][20347] Updated weights for policy 0, policy_version 1276843 (0.0008) [2023-12-27 14:29:43,211][20381] Updated weights for policy 1, policy_version 1275878 (0.0006) [2023-12-27 14:29:43,269][20381] Updated weights for policy 1, policy_version 1275888 (0.0010) [2023-12-27 14:29:43,816][20381] Updated weights for policy 1, policy_version 1275898 (0.0009) [2023-12-27 14:29:43,866][20381] Updated weights for policy 1, policy_version 1275908 (0.0005) [2023-12-27 14:29:43,887][20347] Updated weights for policy 0, policy_version 1276853 (0.0006) [2023-12-27 14:29:43,929][20381] Updated weights for policy 1, policy_version 1275918 (0.0007) [2023-12-27 14:29:43,932][20347] Updated weights for policy 0, policy_version 1276863 (0.0005) [2023-12-27 14:29:43,981][20381] Updated weights for policy 1, policy_version 1275928 (0.0005) [2023-12-27 14:29:43,990][20347] Updated weights for policy 0, policy_version 1276873 (0.0006) [2023-12-27 14:29:44,571][20381] Updated weights for policy 1, policy_version 1275938 (0.0007) [2023-12-27 14:29:44,622][20381] Updated weights for policy 1, policy_version 1275948 (0.0006) [2023-12-27 14:29:44,664][20347] Updated weights for policy 0, policy_version 1276883 (0.0009) [2023-12-27 14:29:44,675][20381] Updated weights for policy 1, policy_version 1275958 (0.0008) [2023-12-27 14:29:44,713][20347] Updated weights for policy 0, policy_version 1276893 (0.0006) [2023-12-27 14:29:44,777][20347] Updated weights for policy 0, policy_version 1276903 (0.0008) [2023-12-27 14:29:45,383][20381] Updated weights for policy 1, policy_version 1275968 (0.0007) [2023-12-27 14:29:45,384][20347] Updated weights for policy 0, policy_version 1276913 (0.0007) [2023-12-27 14:29:45,435][20381] Updated weights for policy 1, policy_version 1275978 (0.0006) [2023-12-27 14:29:45,440][20347] Updated weights for policy 0, policy_version 1276923 (0.0007) [2023-12-27 14:29:45,488][20381] Updated weights for policy 1, policy_version 1275988 (0.0006) [2023-12-27 14:29:45,493][20347] Updated weights for policy 0, policy_version 1276933 (0.0006) [2023-12-27 14:29:45,551][20347] Updated weights for policy 0, policy_version 1276943 (0.0007) [2023-12-27 14:29:46,168][20381] Updated weights for policy 1, policy_version 1275998 (0.0005) [2023-12-27 14:29:46,220][20347] Updated weights for policy 0, policy_version 1276953 (0.0008) [2023-12-27 14:29:46,229][20381] Updated weights for policy 1, policy_version 1276008 (0.0005) [2023-12-27 14:29:46,276][20347] Updated weights for policy 0, policy_version 1276963 (0.0008) [2023-12-27 14:29:46,289][20381] Updated weights for policy 1, policy_version 1276018 (0.0005) [2023-12-27 14:29:46,326][20347] Updated weights for policy 0, policy_version 1276973 (0.0006) [2023-12-27 14:29:46,858][20381] Updated weights for policy 1, policy_version 1276028 (0.0006) [2023-12-27 14:29:46,917][20381] Updated weights for policy 1, policy_version 1276038 (0.0007) [2023-12-27 14:29:46,969][20381] Updated weights for policy 1, policy_version 1276048 (0.0008) [2023-12-27 14:29:47,031][20347] Updated weights for policy 0, policy_version 1276983 (0.0008) [2023-12-27 14:29:47,091][20347] Updated weights for policy 0, policy_version 1276993 (0.0008) [2023-12-27 14:29:47,148][20347] Updated weights for policy 0, policy_version 1277003 (0.0007) [2023-12-27 14:29:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 653672448. Throughput: 0: 10047.0, 1: 10325.7. Samples: 653680040. Policy #0 lag: (min: 9.0, avg: 22.8, max: 41.0) [2023-12-27 14:29:47,363][19170] Avg episode reward: [(0, '784.850'), (1, '656.570')] [2023-12-27 14:29:47,654][20381] Updated weights for policy 1, policy_version 1276058 (0.0007) [2023-12-27 14:29:47,710][20381] Updated weights for policy 1, policy_version 1276068 (0.0005) [2023-12-27 14:29:47,762][20381] Updated weights for policy 1, policy_version 1276078 (0.0005) [2023-12-27 14:29:47,813][20381] Updated weights for policy 1, policy_version 1276088 (0.0005) [2023-12-27 14:29:47,853][20347] Updated weights for policy 0, policy_version 1277013 (0.0007) [2023-12-27 14:29:47,899][20347] Updated weights for policy 0, policy_version 1277023 (0.0005) [2023-12-27 14:29:47,948][20347] Updated weights for policy 0, policy_version 1277033 (0.0005) [2023-12-27 14:29:48,443][20381] Updated weights for policy 1, policy_version 1276098 (0.0007) [2023-12-27 14:29:48,501][20381] Updated weights for policy 1, policy_version 1276108 (0.0008) [2023-12-27 14:29:48,518][20347] Updated weights for policy 0, policy_version 1277043 (0.0007) [2023-12-27 14:29:48,560][20381] Updated weights for policy 1, policy_version 1276118 (0.0006) [2023-12-27 14:29:48,571][20347] Updated weights for policy 0, policy_version 1277053 (0.0010) [2023-12-27 14:29:48,623][20347] Updated weights for policy 0, policy_version 1277063 (0.0011) [2023-12-27 14:29:49,224][20381] Updated weights for policy 1, policy_version 1276128 (0.0009) [2023-12-27 14:29:49,289][20381] Updated weights for policy 1, policy_version 1276138 (0.0008) [2023-12-27 14:29:49,346][20347] Updated weights for policy 0, policy_version 1277073 (0.0010) [2023-12-27 14:29:49,354][20381] Updated weights for policy 1, policy_version 1276148 (0.0009) [2023-12-27 14:29:49,415][20347] Updated weights for policy 0, policy_version 1277083 (0.0007) [2023-12-27 14:29:49,475][20347] Updated weights for policy 0, policy_version 1277093 (0.0006) [2023-12-27 14:29:49,529][20347] Updated weights for policy 0, policy_version 1277103 (0.0006) [2023-12-27 14:29:50,116][20381] Updated weights for policy 1, policy_version 1276158 (0.0007) [2023-12-27 14:29:50,179][20381] Updated weights for policy 1, policy_version 1276168 (0.0006) [2023-12-27 14:29:50,196][20347] Updated weights for policy 0, policy_version 1277113 (0.0008) [2023-12-27 14:29:50,246][20381] Updated weights for policy 1, policy_version 1276178 (0.0008) [2023-12-27 14:29:50,251][20347] Updated weights for policy 0, policy_version 1277123 (0.0006) [2023-12-27 14:29:50,312][20347] Updated weights for policy 0, policy_version 1277133 (0.0007) [2023-12-27 14:29:50,893][20381] Updated weights for policy 1, policy_version 1276188 (0.0007) [2023-12-27 14:29:50,953][20381] Updated weights for policy 1, policy_version 1276198 (0.0007) [2023-12-27 14:29:50,993][20347] Updated weights for policy 0, policy_version 1277143 (0.0006) [2023-12-27 14:29:51,011][20381] Updated weights for policy 1, policy_version 1276208 (0.0008) [2023-12-27 14:29:51,057][20347] Updated weights for policy 0, policy_version 1277153 (0.0008) [2023-12-27 14:29:51,120][20347] Updated weights for policy 0, policy_version 1277163 (0.0006) [2023-12-27 14:29:51,758][20381] Updated weights for policy 1, policy_version 1276218 (0.0008) [2023-12-27 14:29:51,821][20381] Updated weights for policy 1, policy_version 1276228 (0.0008) [2023-12-27 14:29:51,827][20347] Updated weights for policy 0, policy_version 1277173 (0.0008) [2023-12-27 14:29:51,874][20347] Updated weights for policy 0, policy_version 1277183 (0.0008) [2023-12-27 14:29:51,876][20381] Updated weights for policy 1, policy_version 1276238 (0.0008) [2023-12-27 14:29:51,929][20347] Updated weights for policy 0, policy_version 1277193 (0.0007) [2023-12-27 14:29:51,932][20381] Updated weights for policy 1, policy_version 1276248 (0.0008) [2023-12-27 14:29:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 653770752. Throughput: 0: 10046.2, 1: 10290.2. Samples: 653741724. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:29:52,363][19170] Avg episode reward: [(0, '814.120'), (1, '707.480')] [2023-12-27 14:29:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001276248_326746112.pth... [2023-12-27 14:29:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001277200_327024640.pth... [2023-12-27 14:29:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001275032_326434816.pth [2023-12-27 14:29:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001276016_326721536.pth [2023-12-27 14:29:52,631][20347] Updated weights for policy 0, policy_version 1277203 (0.0008) [2023-12-27 14:29:52,648][20381] Updated weights for policy 1, policy_version 1276258 (0.0006) [2023-12-27 14:29:52,688][20347] Updated weights for policy 0, policy_version 1277213 (0.0011) [2023-12-27 14:29:52,703][20381] Updated weights for policy 1, policy_version 1276268 (0.0006) [2023-12-27 14:29:52,746][20347] Updated weights for policy 0, policy_version 1277223 (0.0010) [2023-12-27 14:29:52,747][20291] KL-divergence is very high: 101.5907 [2023-12-27 14:29:52,758][20381] Updated weights for policy 1, policy_version 1276278 (0.0007) [2023-12-27 14:29:52,789][20291] KL-divergence is very high: 130.4583 [2023-12-27 14:29:52,794][20291] KL-divergence is very high: 151.6087 [2023-12-27 14:29:53,408][20347] Updated weights for policy 0, policy_version 1277233 (0.0011) [2023-12-27 14:29:53,413][20291] KL-divergence is very high: 150.4059 [2023-12-27 14:29:53,465][20347] Updated weights for policy 0, policy_version 1277243 (0.0011) [2023-12-27 14:29:53,467][20291] KL-divergence is very high: 183.1548 [2023-12-27 14:29:53,484][20381] Updated weights for policy 1, policy_version 1276288 (0.0008) [2023-12-27 14:29:53,510][20291] KL-divergence is very high: 118.9715 [2023-12-27 14:29:53,517][20291] KL-divergence is very high: 169.3269 [2023-12-27 14:29:53,528][20347] Updated weights for policy 0, policy_version 1277253 (0.0009) [2023-12-27 14:29:53,542][20381] Updated weights for policy 1, policy_version 1276298 (0.0007) [2023-12-27 14:29:53,557][20291] KL-divergence is very high: 131.3349 [2023-12-27 14:29:53,591][20347] Updated weights for policy 0, policy_version 1277263 (0.0010) [2023-12-27 14:29:53,600][20381] Updated weights for policy 1, policy_version 1276308 (0.0006) [2023-12-27 14:29:54,251][20347] Updated weights for policy 0, policy_version 1277273 (0.0010) [2023-12-27 14:29:54,275][20381] Updated weights for policy 1, policy_version 1276318 (0.0010) [2023-12-27 14:29:54,312][20347] Updated weights for policy 0, policy_version 1277283 (0.0011) [2023-12-27 14:29:54,334][20381] Updated weights for policy 1, policy_version 1276328 (0.0011) [2023-12-27 14:29:54,367][20347] Updated weights for policy 0, policy_version 1277293 (0.0010) [2023-12-27 14:29:54,391][20381] Updated weights for policy 1, policy_version 1276338 (0.0011) [2023-12-27 14:29:54,984][20347] Updated weights for policy 0, policy_version 1277303 (0.0007) [2023-12-27 14:29:55,037][20347] Updated weights for policy 0, policy_version 1277313 (0.0007) [2023-12-27 14:29:55,052][20381] Updated weights for policy 1, policy_version 1276348 (0.0011) [2023-12-27 14:29:55,092][20347] Updated weights for policy 0, policy_version 1277323 (0.0010) [2023-12-27 14:29:55,104][20381] Updated weights for policy 1, policy_version 1276358 (0.0010) [2023-12-27 14:29:55,163][20381] Updated weights for policy 1, policy_version 1276368 (0.0011) [2023-12-27 14:29:55,754][20347] Updated weights for policy 0, policy_version 1277333 (0.0010) [2023-12-27 14:29:55,797][20381] Updated weights for policy 1, policy_version 1276378 (0.0010) [2023-12-27 14:29:55,808][20347] Updated weights for policy 0, policy_version 1277343 (0.0010) [2023-12-27 14:29:55,855][20381] Updated weights for policy 1, policy_version 1276388 (0.0011) [2023-12-27 14:29:55,863][20347] Updated weights for policy 0, policy_version 1277353 (0.0010) [2023-12-27 14:29:55,916][20381] Updated weights for policy 1, policy_version 1276398 (0.0007) [2023-12-27 14:29:55,978][20381] Updated weights for policy 1, policy_version 1276408 (0.0005) [2023-12-27 14:29:56,499][20381] Updated weights for policy 1, policy_version 1276418 (0.0005) [2023-12-27 14:29:56,538][20347] Updated weights for policy 0, policy_version 1277363 (0.0010) [2023-12-27 14:29:56,553][20381] Updated weights for policy 1, policy_version 1276428 (0.0010) [2023-12-27 14:29:56,596][20347] Updated weights for policy 0, policy_version 1277373 (0.0007) [2023-12-27 14:29:56,609][20381] Updated weights for policy 1, policy_version 1276438 (0.0005) [2023-12-27 14:29:56,646][20347] Updated weights for policy 0, policy_version 1277383 (0.0005) [2023-12-27 14:29:57,167][20381] Updated weights for policy 1, policy_version 1276448 (0.0006) [2023-12-27 14:29:57,222][20381] Updated weights for policy 1, policy_version 1276458 (0.0005) [2023-12-27 14:29:57,277][20381] Updated weights for policy 1, policy_version 1276468 (0.0005) [2023-12-27 14:29:57,287][20347] Updated weights for policy 0, policy_version 1277393 (0.0005) [2023-12-27 14:29:57,345][20347] Updated weights for policy 0, policy_version 1277403 (0.0006) [2023-12-27 14:29:57,362][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 653877248. Throughput: 0: 10048.8, 1: 10329.3. Samples: 653863980. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:29:57,363][19170] Avg episode reward: [(0, '745.730'), (1, '671.830')] [2023-12-27 14:29:57,389][20347] Updated weights for policy 0, policy_version 1277413 (0.0006) [2023-12-27 14:29:57,437][20347] Updated weights for policy 0, policy_version 1277423 (0.0007) [2023-12-27 14:29:57,935][20381] Updated weights for policy 1, policy_version 1276478 (0.0007) [2023-12-27 14:29:57,998][20381] Updated weights for policy 1, policy_version 1276488 (0.0008) [2023-12-27 14:29:58,068][20381] Updated weights for policy 1, policy_version 1276498 (0.0008) [2023-12-27 14:29:58,074][20347] Updated weights for policy 0, policy_version 1277433 (0.0006) [2023-12-27 14:29:58,123][20347] Updated weights for policy 0, policy_version 1277443 (0.0006) [2023-12-27 14:29:58,180][20347] Updated weights for policy 0, policy_version 1277453 (0.0007) [2023-12-27 14:29:58,852][20381] Updated weights for policy 1, policy_version 1276508 (0.0009) [2023-12-27 14:29:58,917][20381] Updated weights for policy 1, policy_version 1276518 (0.0010) [2023-12-27 14:29:58,981][20381] Updated weights for policy 1, policy_version 1276528 (0.0011) [2023-12-27 14:29:59,030][20347] Updated weights for policy 0, policy_version 1277463 (0.0009) [2023-12-27 14:29:59,089][20347] Updated weights for policy 0, policy_version 1277473 (0.0010) [2023-12-27 14:29:59,150][20347] Updated weights for policy 0, policy_version 1277483 (0.0010) [2023-12-27 14:29:59,716][20381] Updated weights for policy 1, policy_version 1276538 (0.0010) [2023-12-27 14:29:59,784][20381] Updated weights for policy 1, policy_version 1276548 (0.0008) [2023-12-27 14:29:59,845][20347] Updated weights for policy 0, policy_version 1277493 (0.0011) [2023-12-27 14:29:59,847][20381] Updated weights for policy 1, policy_version 1276558 (0.0009) [2023-12-27 14:29:59,896][20347] Updated weights for policy 0, policy_version 1277503 (0.0006) [2023-12-27 14:29:59,909][20381] Updated weights for policy 1, policy_version 1276568 (0.0006) [2023-12-27 14:29:59,957][20347] Updated weights for policy 0, policy_version 1277513 (0.0008) [2023-12-27 14:30:00,518][20347] Updated weights for policy 0, policy_version 1277523 (0.0005) [2023-12-27 14:30:00,527][20381] Updated weights for policy 1, policy_version 1276578 (0.0011) [2023-12-27 14:30:00,576][20347] Updated weights for policy 0, policy_version 1277533 (0.0006) [2023-12-27 14:30:00,582][20381] Updated weights for policy 1, policy_version 1276588 (0.0010) [2023-12-27 14:30:00,629][20381] Updated weights for policy 1, policy_version 1276598 (0.0009) [2023-12-27 14:30:00,629][20347] Updated weights for policy 0, policy_version 1277543 (0.0005) [2023-12-27 14:30:01,253][20347] Updated weights for policy 0, policy_version 1277553 (0.0005) [2023-12-27 14:30:01,267][20381] Updated weights for policy 1, policy_version 1276608 (0.0007) [2023-12-27 14:30:01,320][20347] Updated weights for policy 0, policy_version 1277563 (0.0007) [2023-12-27 14:30:01,325][20381] Updated weights for policy 1, policy_version 1276618 (0.0006) [2023-12-27 14:30:01,383][20347] Updated weights for policy 0, policy_version 1277573 (0.0007) [2023-12-27 14:30:01,388][20381] Updated weights for policy 1, policy_version 1276628 (0.0007) [2023-12-27 14:30:01,437][20347] Updated weights for policy 0, policy_version 1277583 (0.0006) [2023-12-27 14:30:02,088][20381] Updated weights for policy 1, policy_version 1276638 (0.0007) [2023-12-27 14:30:02,121][20347] Updated weights for policy 0, policy_version 1277593 (0.0007) [2023-12-27 14:30:02,143][20381] Updated weights for policy 1, policy_version 1276648 (0.0008) [2023-12-27 14:30:02,174][20347] Updated weights for policy 0, policy_version 1277603 (0.0010) [2023-12-27 14:30:02,199][20381] Updated weights for policy 1, policy_version 1276658 (0.0007) [2023-12-27 14:30:02,229][20347] Updated weights for policy 0, policy_version 1277613 (0.0006) [2023-12-27 14:30:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 653983744. Throughput: 0: 10083.3, 1: 10302.1. Samples: 653987596. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:02,363][19170] Avg episode reward: [(0, '844.110'), (1, '692.920')] [2023-12-27 14:30:02,900][20347] Updated weights for policy 0, policy_version 1277623 (0.0007) [2023-12-27 14:30:02,914][20381] Updated weights for policy 1, policy_version 1276668 (0.0007) [2023-12-27 14:30:02,956][20347] Updated weights for policy 0, policy_version 1277633 (0.0005) [2023-12-27 14:30:02,969][20381] Updated weights for policy 1, policy_version 1276678 (0.0007) [2023-12-27 14:30:03,006][20347] Updated weights for policy 0, policy_version 1277643 (0.0005) [2023-12-27 14:30:03,022][20381] Updated weights for policy 1, policy_version 1276688 (0.0006) [2023-12-27 14:30:03,630][20347] Updated weights for policy 0, policy_version 1277653 (0.0006) [2023-12-27 14:30:03,683][20381] Updated weights for policy 1, policy_version 1276698 (0.0007) [2023-12-27 14:30:03,685][20347] Updated weights for policy 0, policy_version 1277663 (0.0008) [2023-12-27 14:30:03,729][20347] Updated weights for policy 0, policy_version 1277673 (0.0005) [2023-12-27 14:30:03,731][20381] Updated weights for policy 1, policy_version 1276708 (0.0006) [2023-12-27 14:30:03,783][20381] Updated weights for policy 1, policy_version 1276718 (0.0007) [2023-12-27 14:30:03,842][20381] Updated weights for policy 1, policy_version 1276728 (0.0007) [2023-12-27 14:30:04,420][20347] Updated weights for policy 0, policy_version 1277683 (0.0005) [2023-12-27 14:30:04,483][20347] Updated weights for policy 0, policy_version 1277693 (0.0007) [2023-12-27 14:30:04,539][20347] Updated weights for policy 0, policy_version 1277703 (0.0008) [2023-12-27 14:30:04,545][20381] Updated weights for policy 1, policy_version 1276738 (0.0006) [2023-12-27 14:30:04,601][20381] Updated weights for policy 1, policy_version 1276748 (0.0005) [2023-12-27 14:30:04,666][20381] Updated weights for policy 1, policy_version 1276758 (0.0009) [2023-12-27 14:30:05,196][20347] Updated weights for policy 0, policy_version 1277713 (0.0008) [2023-12-27 14:30:05,207][20381] Updated weights for policy 1, policy_version 1276768 (0.0010) [2023-12-27 14:30:05,250][20347] Updated weights for policy 0, policy_version 1277723 (0.0008) [2023-12-27 14:30:05,263][20381] Updated weights for policy 1, policy_version 1276778 (0.0010) [2023-12-27 14:30:05,300][20347] Updated weights for policy 0, policy_version 1277733 (0.0007) [2023-12-27 14:30:05,318][20381] Updated weights for policy 1, policy_version 1276788 (0.0010) [2023-12-27 14:30:05,351][20347] Updated weights for policy 0, policy_version 1277743 (0.0010) [2023-12-27 14:30:05,952][20381] Updated weights for policy 1, policy_version 1276798 (0.0007) [2023-12-27 14:30:06,008][20381] Updated weights for policy 1, policy_version 1276808 (0.0006) [2023-12-27 14:30:06,019][20347] Updated weights for policy 0, policy_version 1277753 (0.0010) [2023-12-27 14:30:06,064][20381] Updated weights for policy 1, policy_version 1276818 (0.0007) [2023-12-27 14:30:06,068][20347] Updated weights for policy 0, policy_version 1277763 (0.0010) [2023-12-27 14:30:06,130][20347] Updated weights for policy 0, policy_version 1277773 (0.0011) [2023-12-27 14:30:06,678][20381] Updated weights for policy 1, policy_version 1276828 (0.0007) [2023-12-27 14:30:06,735][20381] Updated weights for policy 1, policy_version 1276838 (0.0006) [2023-12-27 14:30:06,789][20381] Updated weights for policy 1, policy_version 1276848 (0.0005) [2023-12-27 14:30:06,848][20347] Updated weights for policy 0, policy_version 1277783 (0.0011) [2023-12-27 14:30:06,911][20347] Updated weights for policy 0, policy_version 1277793 (0.0010) [2023-12-27 14:30:06,975][20347] Updated weights for policy 0, policy_version 1277803 (0.0008) [2023-12-27 14:30:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 654082048. Throughput: 0: 10103.7, 1: 10359.8. Samples: 654051400. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:07,363][19170] Avg episode reward: [(0, '843.210'), (1, '701.500')] [2023-12-27 14:30:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001276856_326901760.pth... [2023-12-27 14:30:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001277808_327180288.pth... [2023-12-27 14:30:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001275640_326590464.pth [2023-12-27 14:30:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001276624_326877184.pth [2023-12-27 14:30:07,501][20381] Updated weights for policy 1, policy_version 1276858 (0.0008) [2023-12-27 14:30:07,561][20381] Updated weights for policy 1, policy_version 1276868 (0.0007) [2023-12-27 14:30:07,585][20347] Updated weights for policy 0, policy_version 1277813 (0.0007) [2023-12-27 14:30:07,618][20381] Updated weights for policy 1, policy_version 1276878 (0.0006) [2023-12-27 14:30:07,638][20347] Updated weights for policy 0, policy_version 1277823 (0.0005) [2023-12-27 14:30:07,677][20381] Updated weights for policy 1, policy_version 1276888 (0.0007) [2023-12-27 14:30:07,692][20347] Updated weights for policy 0, policy_version 1277833 (0.0005) [2023-12-27 14:30:08,281][20347] Updated weights for policy 0, policy_version 1277843 (0.0005) [2023-12-27 14:30:08,340][20347] Updated weights for policy 0, policy_version 1277853 (0.0006) [2023-12-27 14:30:08,398][20347] Updated weights for policy 0, policy_version 1277863 (0.0006) [2023-12-27 14:30:08,420][20381] Updated weights for policy 1, policy_version 1276898 (0.0008) [2023-12-27 14:30:08,482][20381] Updated weights for policy 1, policy_version 1276908 (0.0008) [2023-12-27 14:30:08,544][20381] Updated weights for policy 1, policy_version 1276918 (0.0008) [2023-12-27 14:30:09,033][20347] Updated weights for policy 0, policy_version 1277873 (0.0006) [2023-12-27 14:30:09,081][20347] Updated weights for policy 0, policy_version 1277883 (0.0008) [2023-12-27 14:30:09,124][20347] Updated weights for policy 0, policy_version 1277893 (0.0007) [2023-12-27 14:30:09,171][20347] Updated weights for policy 0, policy_version 1277903 (0.0007) [2023-12-27 14:30:09,263][20381] Updated weights for policy 1, policy_version 1276928 (0.0009) [2023-12-27 14:30:09,331][20381] Updated weights for policy 1, policy_version 1276938 (0.0007) [2023-12-27 14:30:09,395][20381] Updated weights for policy 1, policy_version 1276948 (0.0008) [2023-12-27 14:30:09,923][20347] Updated weights for policy 0, policy_version 1277913 (0.0006) [2023-12-27 14:30:09,987][20347] Updated weights for policy 0, policy_version 1277923 (0.0008) [2023-12-27 14:30:10,048][20347] Updated weights for policy 0, policy_version 1277933 (0.0007) [2023-12-27 14:30:10,065][20381] Updated weights for policy 1, policy_version 1276958 (0.0009) [2023-12-27 14:30:10,121][20381] Updated weights for policy 1, policy_version 1276968 (0.0007) [2023-12-27 14:30:10,183][20381] Updated weights for policy 1, policy_version 1276978 (0.0008) [2023-12-27 14:30:10,769][20347] Updated weights for policy 0, policy_version 1277943 (0.0006) [2023-12-27 14:30:10,826][20347] Updated weights for policy 0, policy_version 1277953 (0.0008) [2023-12-27 14:30:10,879][20381] Updated weights for policy 1, policy_version 1276988 (0.0009) [2023-12-27 14:30:10,890][20347] Updated weights for policy 0, policy_version 1277963 (0.0009) [2023-12-27 14:30:10,938][20381] Updated weights for policy 1, policy_version 1276998 (0.0010) [2023-12-27 14:30:10,996][20381] Updated weights for policy 1, policy_version 1277008 (0.0010) [2023-12-27 14:30:11,610][20347] Updated weights for policy 0, policy_version 1277973 (0.0007) [2023-12-27 14:30:11,677][20347] Updated weights for policy 0, policy_version 1277983 (0.0009) [2023-12-27 14:30:11,723][20381] Updated weights for policy 1, policy_version 1277018 (0.0008) [2023-12-27 14:30:11,741][20347] Updated weights for policy 0, policy_version 1277993 (0.0008) [2023-12-27 14:30:11,785][20381] Updated weights for policy 1, policy_version 1277028 (0.0008) [2023-12-27 14:30:11,839][20381] Updated weights for policy 1, policy_version 1277038 (0.0008) [2023-12-27 14:30:11,893][20381] Updated weights for policy 1, policy_version 1277048 (0.0008) [2023-12-27 14:30:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 654180352. Throughput: 0: 10127.8, 1: 10384.3. Samples: 654172764. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:12,363][19170] Avg episode reward: [(0, '784.430'), (1, '721.120')] [2023-12-27 14:30:12,486][20347] Updated weights for policy 0, policy_version 1278003 (0.0008) [2023-12-27 14:30:12,556][20347] Updated weights for policy 0, policy_version 1278013 (0.0007) [2023-12-27 14:30:12,610][20347] Updated weights for policy 0, policy_version 1278023 (0.0006) [2023-12-27 14:30:12,639][20381] Updated weights for policy 1, policy_version 1277058 (0.0011) [2023-12-27 14:30:12,703][20381] Updated weights for policy 1, policy_version 1277068 (0.0011) [2023-12-27 14:30:12,764][20381] Updated weights for policy 1, policy_version 1277078 (0.0011) [2023-12-27 14:30:13,259][20347] Updated weights for policy 0, policy_version 1278033 (0.0006) [2023-12-27 14:30:13,307][20347] Updated weights for policy 0, policy_version 1278043 (0.0007) [2023-12-27 14:30:13,369][20347] Updated weights for policy 0, policy_version 1278053 (0.0008) [2023-12-27 14:30:13,423][20347] Updated weights for policy 0, policy_version 1278063 (0.0007) [2023-12-27 14:30:13,444][20381] Updated weights for policy 1, policy_version 1277088 (0.0007) [2023-12-27 14:30:13,493][20381] Updated weights for policy 1, policy_version 1277098 (0.0007) [2023-12-27 14:30:13,551][20381] Updated weights for policy 1, policy_version 1277108 (0.0008) [2023-12-27 14:30:14,149][20347] Updated weights for policy 0, policy_version 1278073 (0.0006) [2023-12-27 14:30:14,162][20381] Updated weights for policy 1, policy_version 1277118 (0.0007) [2023-12-27 14:30:14,198][20347] Updated weights for policy 0, policy_version 1278083 (0.0006) [2023-12-27 14:30:14,219][20381] Updated weights for policy 1, policy_version 1277128 (0.0006) [2023-12-27 14:30:14,255][20347] Updated weights for policy 0, policy_version 1278093 (0.0006) [2023-12-27 14:30:14,272][20381] Updated weights for policy 1, policy_version 1277138 (0.0007) [2023-12-27 14:30:14,964][20381] Updated weights for policy 1, policy_version 1277148 (0.0009) [2023-12-27 14:30:14,970][20347] Updated weights for policy 0, policy_version 1278103 (0.0006) [2023-12-27 14:30:15,018][20381] Updated weights for policy 1, policy_version 1277158 (0.0009) [2023-12-27 14:30:15,024][20347] Updated weights for policy 0, policy_version 1278113 (0.0006) [2023-12-27 14:30:15,074][20381] Updated weights for policy 1, policy_version 1277168 (0.0009) [2023-12-27 14:30:15,088][20347] Updated weights for policy 0, policy_version 1278123 (0.0008) [2023-12-27 14:30:15,724][20381] Updated weights for policy 1, policy_version 1277178 (0.0010) [2023-12-27 14:30:15,784][20381] Updated weights for policy 1, policy_version 1277188 (0.0008) [2023-12-27 14:30:15,796][20347] Updated weights for policy 0, policy_version 1278133 (0.0006) [2023-12-27 14:30:15,843][20381] Updated weights for policy 1, policy_version 1277198 (0.0007) [2023-12-27 14:30:15,848][20347] Updated weights for policy 0, policy_version 1278143 (0.0007) [2023-12-27 14:30:15,896][20347] Updated weights for policy 0, policy_version 1278153 (0.0005) [2023-12-27 14:30:15,898][20381] Updated weights for policy 1, policy_version 1277208 (0.0010) [2023-12-27 14:30:16,520][20381] Updated weights for policy 1, policy_version 1277218 (0.0008) [2023-12-27 14:30:16,560][20347] Updated weights for policy 0, policy_version 1278163 (0.0008) [2023-12-27 14:30:16,570][20381] Updated weights for policy 1, policy_version 1277228 (0.0006) [2023-12-27 14:30:16,618][20347] Updated weights for policy 0, policy_version 1278173 (0.0007) [2023-12-27 14:30:16,623][20381] Updated weights for policy 1, policy_version 1277238 (0.0007) [2023-12-27 14:30:16,676][20347] Updated weights for policy 0, policy_version 1278183 (0.0007) [2023-12-27 14:30:17,274][20381] Updated weights for policy 1, policy_version 1277248 (0.0008) [2023-12-27 14:30:17,318][20381] Updated weights for policy 1, policy_version 1277258 (0.0007) [2023-12-27 14:30:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 654278656. Throughput: 0: 10123.6, 1: 10344.4. Samples: 654293772. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:17,363][19170] Avg episode reward: [(0, '882.670'), (1, '670.770')] [2023-12-27 14:30:17,367][20381] Updated weights for policy 1, policy_version 1277268 (0.0006) [2023-12-27 14:30:17,376][20347] Updated weights for policy 0, policy_version 1278193 (0.0008) [2023-12-27 14:30:17,426][20347] Updated weights for policy 0, policy_version 1278203 (0.0007) [2023-12-27 14:30:17,479][20347] Updated weights for policy 0, policy_version 1278213 (0.0008) [2023-12-27 14:30:17,524][20347] Updated weights for policy 0, policy_version 1278223 (0.0007) [2023-12-27 14:30:18,084][20381] Updated weights for policy 1, policy_version 1277278 (0.0007) [2023-12-27 14:30:18,137][20381] Updated weights for policy 1, policy_version 1277288 (0.0008) [2023-12-27 14:30:18,191][20381] Updated weights for policy 1, policy_version 1277298 (0.0009) [2023-12-27 14:30:18,197][20347] Updated weights for policy 0, policy_version 1278233 (0.0009) [2023-12-27 14:30:18,248][20347] Updated weights for policy 0, policy_version 1278243 (0.0006) [2023-12-27 14:30:18,306][20347] Updated weights for policy 0, policy_version 1278253 (0.0007) [2023-12-27 14:30:18,869][20381] Updated weights for policy 1, policy_version 1277308 (0.0005) [2023-12-27 14:30:18,924][20381] Updated weights for policy 1, policy_version 1277318 (0.0005) [2023-12-27 14:30:18,981][20381] Updated weights for policy 1, policy_version 1277328 (0.0005) [2023-12-27 14:30:19,043][20347] Updated weights for policy 0, policy_version 1278263 (0.0008) [2023-12-27 14:30:19,089][20347] Updated weights for policy 0, policy_version 1278273 (0.0008) [2023-12-27 14:30:19,138][20347] Updated weights for policy 0, policy_version 1278283 (0.0008) [2023-12-27 14:30:19,655][20381] Updated weights for policy 1, policy_version 1277338 (0.0006) [2023-12-27 14:30:19,717][20381] Updated weights for policy 1, policy_version 1277348 (0.0009) [2023-12-27 14:30:19,778][20381] Updated weights for policy 1, policy_version 1277358 (0.0008) [2023-12-27 14:30:19,848][20381] Updated weights for policy 1, policy_version 1277368 (0.0008) [2023-12-27 14:30:19,889][20347] Updated weights for policy 0, policy_version 1278293 (0.0008) [2023-12-27 14:30:19,955][20347] Updated weights for policy 0, policy_version 1278303 (0.0008) [2023-12-27 14:30:20,017][20347] Updated weights for policy 0, policy_version 1278313 (0.0008) [2023-12-27 14:30:20,563][20381] Updated weights for policy 1, policy_version 1277378 (0.0008) [2023-12-27 14:30:20,624][20381] Updated weights for policy 1, policy_version 1277388 (0.0008) [2023-12-27 14:30:20,682][20381] Updated weights for policy 1, policy_version 1277398 (0.0008) [2023-12-27 14:30:20,743][20347] Updated weights for policy 0, policy_version 1278323 (0.0008) [2023-12-27 14:30:20,799][20347] Updated weights for policy 0, policy_version 1278333 (0.0006) [2023-12-27 14:30:20,862][20347] Updated weights for policy 0, policy_version 1278343 (0.0007) [2023-12-27 14:30:21,404][20381] Updated weights for policy 1, policy_version 1277408 (0.0008) [2023-12-27 14:30:21,468][20381] Updated weights for policy 1, policy_version 1277418 (0.0009) [2023-12-27 14:30:21,515][20381] Updated weights for policy 1, policy_version 1277428 (0.0011) [2023-12-27 14:30:21,663][20347] Updated weights for policy 0, policy_version 1278353 (0.0008) [2023-12-27 14:30:21,732][20347] Updated weights for policy 0, policy_version 1278363 (0.0008) [2023-12-27 14:30:21,791][20347] Updated weights for policy 0, policy_version 1278373 (0.0007) [2023-12-27 14:30:21,849][20347] Updated weights for policy 0, policy_version 1278384 (0.0009) [2023-12-27 14:30:22,274][20381] Updated weights for policy 1, policy_version 1277438 (0.0011) [2023-12-27 14:30:22,336][20381] Updated weights for policy 1, policy_version 1277448 (0.0011) [2023-12-27 14:30:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 654376960. Throughput: 0: 10084.2, 1: 10319.5. Samples: 654353308. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:22,363][19170] Avg episode reward: [(0, '902.390'), (1, '739.800')] [2023-12-27 14:30:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001278384_327327744.pth... [2023-12-27 14:30:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001277200_327024640.pth [2023-12-27 14:30:22,407][20381] Updated weights for policy 1, policy_version 1277458 (0.0012) [2023-12-27 14:30:22,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001277464_327057408.pth... [2023-12-27 14:30:22,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001276248_326746112.pth [2023-12-27 14:30:22,553][20347] Updated weights for policy 0, policy_version 1278394 (0.0006) [2023-12-27 14:30:22,615][20347] Updated weights for policy 0, policy_version 1278404 (0.0007) [2023-12-27 14:30:22,676][20347] Updated weights for policy 0, policy_version 1278414 (0.0007) [2023-12-27 14:30:23,121][20381] Updated weights for policy 1, policy_version 1277468 (0.0009) [2023-12-27 14:30:23,179][20381] Updated weights for policy 1, policy_version 1277478 (0.0008) [2023-12-27 14:30:23,241][20381] Updated weights for policy 1, policy_version 1277488 (0.0005) [2023-12-27 14:30:23,358][20347] Updated weights for policy 0, policy_version 1278424 (0.0008) [2023-12-27 14:30:23,410][20347] Updated weights for policy 0, policy_version 1278434 (0.0009) [2023-12-27 14:30:23,461][20347] Updated weights for policy 0, policy_version 1278444 (0.0008) [2023-12-27 14:30:23,808][20381] Updated weights for policy 1, policy_version 1277498 (0.0006) [2023-12-27 14:30:23,861][20381] Updated weights for policy 1, policy_version 1277508 (0.0007) [2023-12-27 14:30:23,914][20381] Updated weights for policy 1, policy_version 1277518 (0.0006) [2023-12-27 14:30:23,966][20381] Updated weights for policy 1, policy_version 1277528 (0.0006) [2023-12-27 14:30:24,191][20347] Updated weights for policy 0, policy_version 1278454 (0.0009) [2023-12-27 14:30:24,247][20347] Updated weights for policy 0, policy_version 1278464 (0.0007) [2023-12-27 14:30:24,309][20347] Updated weights for policy 0, policy_version 1278474 (0.0008) [2023-12-27 14:30:24,643][20381] Updated weights for policy 1, policy_version 1277538 (0.0008) [2023-12-27 14:30:24,695][20381] Updated weights for policy 1, policy_version 1277548 (0.0007) [2023-12-27 14:30:24,747][20381] Updated weights for policy 1, policy_version 1277558 (0.0007) [2023-12-27 14:30:24,933][20347] Updated weights for policy 0, policy_version 1278484 (0.0008) [2023-12-27 14:30:24,988][20347] Updated weights for policy 0, policy_version 1278494 (0.0005) [2023-12-27 14:30:25,045][20347] Updated weights for policy 0, policy_version 1278504 (0.0007) [2023-12-27 14:30:25,392][20381] Updated weights for policy 1, policy_version 1277568 (0.0008) [2023-12-27 14:30:25,444][20381] Updated weights for policy 1, policy_version 1277578 (0.0007) [2023-12-27 14:30:25,498][20381] Updated weights for policy 1, policy_version 1277588 (0.0008) [2023-12-27 14:30:25,709][20347] Updated weights for policy 0, policy_version 1278514 (0.0011) [2023-12-27 14:30:25,765][20347] Updated weights for policy 0, policy_version 1278524 (0.0010) [2023-12-27 14:30:25,816][20347] Updated weights for policy 0, policy_version 1278534 (0.0010) [2023-12-27 14:30:25,843][20291] KL-divergence is very high: 106.7800 [2023-12-27 14:30:25,848][20291] KL-divergence is very high: 103.2734 [2023-12-27 14:30:25,868][20347] Updated weights for policy 0, policy_version 1278544 (0.0010) [2023-12-27 14:30:26,118][20381] Updated weights for policy 1, policy_version 1277598 (0.0007) [2023-12-27 14:30:26,165][20381] Updated weights for policy 1, policy_version 1277608 (0.0005) [2023-12-27 14:30:26,214][20381] Updated weights for policy 1, policy_version 1277618 (0.0006) [2023-12-27 14:30:26,520][20347] Updated weights for policy 0, policy_version 1278554 (0.0007) [2023-12-27 14:30:26,569][20347] Updated weights for policy 0, policy_version 1278564 (0.0007) [2023-12-27 14:30:26,624][20347] Updated weights for policy 0, policy_version 1278574 (0.0008) [2023-12-27 14:30:26,815][20381] Updated weights for policy 1, policy_version 1277628 (0.0006) [2023-12-27 14:30:26,869][20381] Updated weights for policy 1, policy_version 1277638 (0.0006) [2023-12-27 14:30:26,931][20381] Updated weights for policy 1, policy_version 1277648 (0.0006) [2023-12-27 14:30:27,310][20347] Updated weights for policy 0, policy_version 1278584 (0.0008) [2023-12-27 14:30:27,361][20347] Updated weights for policy 0, policy_version 1278594 (0.0007) [2023-12-27 14:30:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 654483456. Throughput: 0: 10111.5, 1: 10305.0. Samples: 654474604. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:27,363][19170] Avg episode reward: [(0, '873.030'), (1, '820.780')] [2023-12-27 14:30:27,406][20347] Updated weights for policy 0, policy_version 1278604 (0.0008) [2023-12-27 14:30:27,568][20381] Updated weights for policy 1, policy_version 1277658 (0.0007) [2023-12-27 14:30:27,629][20381] Updated weights for policy 1, policy_version 1277668 (0.0006) [2023-12-27 14:30:27,683][20381] Updated weights for policy 1, policy_version 1277678 (0.0007) [2023-12-27 14:30:27,739][20381] Updated weights for policy 1, policy_version 1277688 (0.0007) [2023-12-27 14:30:28,103][20347] Updated weights for policy 0, policy_version 1278614 (0.0008) [2023-12-27 14:30:28,160][20347] Updated weights for policy 0, policy_version 1278624 (0.0008) [2023-12-27 14:30:28,215][20347] Updated weights for policy 0, policy_version 1278634 (0.0007) [2023-12-27 14:30:28,392][20381] Updated weights for policy 1, policy_version 1277698 (0.0008) [2023-12-27 14:30:28,447][20381] Updated weights for policy 1, policy_version 1277708 (0.0008) [2023-12-27 14:30:28,506][20381] Updated weights for policy 1, policy_version 1277718 (0.0006) [2023-12-27 14:30:28,969][20347] Updated weights for policy 0, policy_version 1278644 (0.0008) [2023-12-27 14:30:29,025][20347] Updated weights for policy 0, policy_version 1278654 (0.0007) [2023-12-27 14:30:29,082][20347] Updated weights for policy 0, policy_version 1278664 (0.0007) [2023-12-27 14:30:29,121][20381] Updated weights for policy 1, policy_version 1277728 (0.0007) [2023-12-27 14:30:29,175][20381] Updated weights for policy 1, policy_version 1277738 (0.0007) [2023-12-27 14:30:29,236][20381] Updated weights for policy 1, policy_version 1277748 (0.0008) [2023-12-27 14:30:29,792][20347] Updated weights for policy 0, policy_version 1278674 (0.0006) [2023-12-27 14:30:29,857][20347] Updated weights for policy 0, policy_version 1278684 (0.0008) [2023-12-27 14:30:29,917][20347] Updated weights for policy 0, policy_version 1278694 (0.0008) [2023-12-27 14:30:29,988][20347] Updated weights for policy 0, policy_version 1278704 (0.0008) [2023-12-27 14:30:30,044][20381] Updated weights for policy 1, policy_version 1277758 (0.0010) [2023-12-27 14:30:30,100][20381] Updated weights for policy 1, policy_version 1277768 (0.0011) [2023-12-27 14:30:30,159][20381] Updated weights for policy 1, policy_version 1277778 (0.0011) [2023-12-27 14:30:30,692][20347] Updated weights for policy 0, policy_version 1278714 (0.0008) [2023-12-27 14:30:30,741][20347] Updated weights for policy 0, policy_version 1278724 (0.0008) [2023-12-27 14:30:30,793][20347] Updated weights for policy 0, policy_version 1278734 (0.0008) [2023-12-27 14:30:30,811][20381] Updated weights for policy 1, policy_version 1277788 (0.0009) [2023-12-27 14:30:30,868][20381] Updated weights for policy 1, policy_version 1277798 (0.0008) [2023-12-27 14:30:30,927][20381] Updated weights for policy 1, policy_version 1277808 (0.0008) [2023-12-27 14:30:31,514][20347] Updated weights for policy 0, policy_version 1278744 (0.0007) [2023-12-27 14:30:31,562][20347] Updated weights for policy 0, policy_version 1278754 (0.0007) [2023-12-27 14:30:31,623][20347] Updated weights for policy 0, policy_version 1278764 (0.0007) [2023-12-27 14:30:31,698][20381] Updated weights for policy 1, policy_version 1277818 (0.0008) [2023-12-27 14:30:31,761][20381] Updated weights for policy 1, policy_version 1277828 (0.0008) [2023-12-27 14:30:31,814][20381] Updated weights for policy 1, policy_version 1277838 (0.0007) [2023-12-27 14:30:31,867][20381] Updated weights for policy 1, policy_version 1277848 (0.0008) [2023-12-27 14:30:32,314][20347] Updated weights for policy 0, policy_version 1278774 (0.0006) [2023-12-27 14:30:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 654581760. Throughput: 0: 10111.4, 1: 10228.1. Samples: 654595324. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:32,363][19170] Avg episode reward: [(0, '834.000'), (1, '757.200')] [2023-12-27 14:30:32,384][20347] Updated weights for policy 0, policy_version 1278784 (0.0011) [2023-12-27 14:30:32,442][20347] Updated weights for policy 0, policy_version 1278794 (0.0010) [2023-12-27 14:30:32,660][20381] Updated weights for policy 1, policy_version 1277858 (0.0008) [2023-12-27 14:30:32,717][20381] Updated weights for policy 1, policy_version 1277868 (0.0008) [2023-12-27 14:30:32,779][20381] Updated weights for policy 1, policy_version 1277878 (0.0008) [2023-12-27 14:30:33,096][20347] Updated weights for policy 0, policy_version 1278804 (0.0010) [2023-12-27 14:30:33,154][20347] Updated weights for policy 0, policy_version 1278814 (0.0010) [2023-12-27 14:30:33,215][20347] Updated weights for policy 0, policy_version 1278824 (0.0010) [2023-12-27 14:30:33,408][20381] Updated weights for policy 1, policy_version 1277888 (0.0008) [2023-12-27 14:30:33,467][20381] Updated weights for policy 1, policy_version 1277898 (0.0008) [2023-12-27 14:30:33,515][20381] Updated weights for policy 1, policy_version 1277908 (0.0007) [2023-12-27 14:30:33,871][20347] Updated weights for policy 0, policy_version 1278834 (0.0009) [2023-12-27 14:30:33,927][20347] Updated weights for policy 0, policy_version 1278844 (0.0005) [2023-12-27 14:30:33,983][20347] Updated weights for policy 0, policy_version 1278854 (0.0006) [2023-12-27 14:30:34,043][20347] Updated weights for policy 0, policy_version 1278864 (0.0007) [2023-12-27 14:30:34,235][20381] Updated weights for policy 1, policy_version 1277918 (0.0009) [2023-12-27 14:30:34,298][20381] Updated weights for policy 1, policy_version 1277928 (0.0009) [2023-12-27 14:30:34,361][20381] Updated weights for policy 1, policy_version 1277938 (0.0011) [2023-12-27 14:30:34,659][20347] Updated weights for policy 0, policy_version 1278874 (0.0007) [2023-12-27 14:30:34,719][20347] Updated weights for policy 0, policy_version 1278884 (0.0007) [2023-12-27 14:30:34,779][20347] Updated weights for policy 0, policy_version 1278894 (0.0005) [2023-12-27 14:30:35,072][20381] Updated weights for policy 1, policy_version 1277948 (0.0008) [2023-12-27 14:30:35,078][20292] KL-divergence is very high: 103.8579 [2023-12-27 14:30:35,095][20292] KL-divergence is very high: 145.6452 [2023-12-27 14:30:35,121][20292] KL-divergence is very high: 179.2015 [2023-12-27 14:30:35,129][20381] Updated weights for policy 1, policy_version 1277958 (0.0009) [2023-12-27 14:30:35,139][20292] KL-divergence is very high: 128.5658 [2023-12-27 14:30:35,190][20381] Updated weights for policy 1, policy_version 1277968 (0.0008) [2023-12-27 14:30:35,391][20347] Updated weights for policy 0, policy_version 1278904 (0.0007) [2023-12-27 14:30:35,445][20347] Updated weights for policy 0, policy_version 1278914 (0.0007) [2023-12-27 14:30:35,506][20347] Updated weights for policy 0, policy_version 1278924 (0.0007) [2023-12-27 14:30:35,889][20381] Updated weights for policy 1, policy_version 1277978 (0.0008) [2023-12-27 14:30:35,941][20381] Updated weights for policy 1, policy_version 1277988 (0.0008) [2023-12-27 14:30:35,999][20381] Updated weights for policy 1, policy_version 1277998 (0.0008) [2023-12-27 14:30:36,064][20381] Updated weights for policy 1, policy_version 1278008 (0.0008) [2023-12-27 14:30:36,235][20347] Updated weights for policy 0, policy_version 1278934 (0.0008) [2023-12-27 14:30:36,304][20347] Updated weights for policy 0, policy_version 1278944 (0.0007) [2023-12-27 14:30:36,374][20347] Updated weights for policy 0, policy_version 1278954 (0.0006) [2023-12-27 14:30:36,799][20381] Updated weights for policy 1, policy_version 1278018 (0.0008) [2023-12-27 14:30:36,860][20381] Updated weights for policy 1, policy_version 1278028 (0.0008) [2023-12-27 14:30:36,921][20381] Updated weights for policy 1, policy_version 1278038 (0.0008) [2023-12-27 14:30:37,060][20347] Updated weights for policy 0, policy_version 1278964 (0.0008) [2023-12-27 14:30:37,129][20347] Updated weights for policy 0, policy_version 1278974 (0.0008) [2023-12-27 14:30:37,187][20347] Updated weights for policy 0, policy_version 1278984 (0.0008) [2023-12-27 14:30:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 654688256. Throughput: 0: 10140.2, 1: 10206.1. Samples: 654657304. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:37,363][19170] Avg episode reward: [(0, '873.160'), (1, '671.900')] [2023-12-27 14:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001278040_327204864.pth... [2023-12-27 14:30:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001278992_327483392.pth... [2023-12-27 14:30:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001276856_326901760.pth [2023-12-27 14:30:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001277808_327180288.pth [2023-12-27 14:30:37,658][20381] Updated weights for policy 1, policy_version 1278048 (0.0006) [2023-12-27 14:30:37,706][20381] Updated weights for policy 1, policy_version 1278058 (0.0006) [2023-12-27 14:30:37,769][20381] Updated weights for policy 1, policy_version 1278068 (0.0007) [2023-12-27 14:30:37,863][20347] Updated weights for policy 0, policy_version 1278994 (0.0007) [2023-12-27 14:30:37,917][20347] Updated weights for policy 0, policy_version 1279004 (0.0008) [2023-12-27 14:30:37,970][20347] Updated weights for policy 0, policy_version 1279014 (0.0006) [2023-12-27 14:30:38,020][20347] Updated weights for policy 0, policy_version 1279024 (0.0008) [2023-12-27 14:30:38,414][20381] Updated weights for policy 1, policy_version 1278078 (0.0008) [2023-12-27 14:30:38,473][20381] Updated weights for policy 1, policy_version 1278088 (0.0007) [2023-12-27 14:30:38,533][20381] Updated weights for policy 1, policy_version 1278098 (0.0008) [2023-12-27 14:30:38,718][20347] Updated weights for policy 0, policy_version 1279034 (0.0008) [2023-12-27 14:30:38,776][20347] Updated weights for policy 0, policy_version 1279044 (0.0008) [2023-12-27 14:30:38,846][20347] Updated weights for policy 0, policy_version 1279054 (0.0006) [2023-12-27 14:30:39,212][20381] Updated weights for policy 1, policy_version 1278108 (0.0007) [2023-12-27 14:30:39,281][20381] Updated weights for policy 1, policy_version 1278118 (0.0006) [2023-12-27 14:30:39,345][20381] Updated weights for policy 1, policy_version 1278128 (0.0006) [2023-12-27 14:30:39,588][20347] Updated weights for policy 0, policy_version 1279064 (0.0007) [2023-12-27 14:30:39,645][20347] Updated weights for policy 0, policy_version 1279074 (0.0008) [2023-12-27 14:30:39,698][20347] Updated weights for policy 0, policy_version 1279084 (0.0007) [2023-12-27 14:30:40,047][20381] Updated weights for policy 1, policy_version 1278138 (0.0007) [2023-12-27 14:30:40,103][20381] Updated weights for policy 1, policy_version 1278148 (0.0007) [2023-12-27 14:30:40,158][20381] Updated weights for policy 1, policy_version 1278158 (0.0008) [2023-12-27 14:30:40,215][20381] Updated weights for policy 1, policy_version 1278168 (0.0008) [2023-12-27 14:30:40,444][20347] Updated weights for policy 0, policy_version 1279094 (0.0007) [2023-12-27 14:30:40,501][20347] Updated weights for policy 0, policy_version 1279104 (0.0007) [2023-12-27 14:30:40,557][20347] Updated weights for policy 0, policy_version 1279114 (0.0006) [2023-12-27 14:30:40,934][20381] Updated weights for policy 1, policy_version 1278178 (0.0006) [2023-12-27 14:30:40,983][20381] Updated weights for policy 1, policy_version 1278188 (0.0006) [2023-12-27 14:30:41,041][20381] Updated weights for policy 1, policy_version 1278198 (0.0010) [2023-12-27 14:30:41,271][20347] Updated weights for policy 0, policy_version 1279124 (0.0007) [2023-12-27 14:30:41,323][20347] Updated weights for policy 0, policy_version 1279134 (0.0006) [2023-12-27 14:30:41,394][20347] Updated weights for policy 0, policy_version 1279144 (0.0009) [2023-12-27 14:30:41,764][20381] Updated weights for policy 1, policy_version 1278208 (0.0008) [2023-12-27 14:30:41,815][20381] Updated weights for policy 1, policy_version 1278218 (0.0008) [2023-12-27 14:30:41,872][20381] Updated weights for policy 1, policy_version 1278228 (0.0007) [2023-12-27 14:30:42,136][20347] Updated weights for policy 0, policy_version 1279154 (0.0007) [2023-12-27 14:30:42,190][20347] Updated weights for policy 0, policy_version 1279164 (0.0008) [2023-12-27 14:30:42,246][20347] Updated weights for policy 0, policy_version 1279174 (0.0008) [2023-12-27 14:30:42,305][20347] Updated weights for policy 0, policy_version 1279184 (0.0008) [2023-12-27 14:30:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 654786560. Throughput: 0: 10088.5, 1: 10155.8. Samples: 654774972. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:42,363][19170] Avg episode reward: [(0, '853.360'), (1, '600.120')] [2023-12-27 14:30:42,597][20381] Updated weights for policy 1, policy_version 1278238 (0.0006) [2023-12-27 14:30:42,647][20381] Updated weights for policy 1, policy_version 1278248 (0.0006) [2023-12-27 14:30:42,694][20381] Updated weights for policy 1, policy_version 1278258 (0.0006) [2023-12-27 14:30:43,085][20347] Updated weights for policy 0, policy_version 1279194 (0.0008) [2023-12-27 14:30:43,149][20347] Updated weights for policy 0, policy_version 1279204 (0.0007) [2023-12-27 14:30:43,206][20347] Updated weights for policy 0, policy_version 1279214 (0.0007) [2023-12-27 14:30:43,366][20381] Updated weights for policy 1, policy_version 1278268 (0.0006) [2023-12-27 14:30:43,421][20381] Updated weights for policy 1, policy_version 1278278 (0.0005) [2023-12-27 14:30:43,467][20381] Updated weights for policy 1, policy_version 1278288 (0.0006) [2023-12-27 14:30:43,814][20347] Updated weights for policy 0, policy_version 1279224 (0.0007) [2023-12-27 14:30:43,876][20347] Updated weights for policy 0, policy_version 1279234 (0.0005) [2023-12-27 14:30:43,921][20347] Updated weights for policy 0, policy_version 1279244 (0.0007) [2023-12-27 14:30:44,124][20381] Updated weights for policy 1, policy_version 1278298 (0.0010) [2023-12-27 14:30:44,179][20381] Updated weights for policy 1, policy_version 1278308 (0.0007) [2023-12-27 14:30:44,235][20381] Updated weights for policy 1, policy_version 1278318 (0.0007) [2023-12-27 14:30:44,290][20381] Updated weights for policy 1, policy_version 1278328 (0.0006) [2023-12-27 14:30:44,563][20347] Updated weights for policy 0, policy_version 1279254 (0.0010) [2023-12-27 14:30:44,617][20347] Updated weights for policy 0, policy_version 1279264 (0.0010) [2023-12-27 14:30:44,671][20347] Updated weights for policy 0, policy_version 1279274 (0.0010) [2023-12-27 14:30:45,011][20381] Updated weights for policy 1, policy_version 1278338 (0.0007) [2023-12-27 14:30:45,068][20381] Updated weights for policy 1, policy_version 1278348 (0.0008) [2023-12-27 14:30:45,127][20381] Updated weights for policy 1, policy_version 1278358 (0.0007) [2023-12-27 14:30:45,346][20347] Updated weights for policy 0, policy_version 1279284 (0.0010) [2023-12-27 14:30:45,391][20347] Updated weights for policy 0, policy_version 1279294 (0.0010) [2023-12-27 14:30:45,446][20347] Updated weights for policy 0, policy_version 1279304 (0.0010) [2023-12-27 14:30:45,774][20381] Updated weights for policy 1, policy_version 1278368 (0.0007) [2023-12-27 14:30:45,829][20381] Updated weights for policy 1, policy_version 1278378 (0.0008) [2023-12-27 14:30:45,886][20381] Updated weights for policy 1, policy_version 1278388 (0.0007) [2023-12-27 14:30:46,132][20347] Updated weights for policy 0, policy_version 1279314 (0.0010) [2023-12-27 14:30:46,193][20347] Updated weights for policy 0, policy_version 1279324 (0.0010) [2023-12-27 14:30:46,252][20347] Updated weights for policy 0, policy_version 1279334 (0.0007) [2023-12-27 14:30:46,300][20347] Updated weights for policy 0, policy_version 1279344 (0.0007) [2023-12-27 14:30:46,532][20381] Updated weights for policy 1, policy_version 1278398 (0.0009) [2023-12-27 14:30:46,577][20381] Updated weights for policy 1, policy_version 1278408 (0.0010) [2023-12-27 14:30:46,625][20381] Updated weights for policy 1, policy_version 1278418 (0.0010) [2023-12-27 14:30:46,914][20347] Updated weights for policy 0, policy_version 1279354 (0.0010) [2023-12-27 14:30:46,969][20347] Updated weights for policy 0, policy_version 1279364 (0.0010) [2023-12-27 14:30:47,025][20347] Updated weights for policy 0, policy_version 1279374 (0.0006) [2023-12-27 14:30:47,283][20381] Updated weights for policy 1, policy_version 1278428 (0.0010) [2023-12-27 14:30:47,337][20381] Updated weights for policy 1, policy_version 1278438 (0.0011) [2023-12-27 14:30:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 654884864. Throughput: 0: 10058.1, 1: 10168.4. Samples: 654897788. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:47,363][19170] Avg episode reward: [(0, '863.350'), (1, '473.620')] [2023-12-27 14:30:47,383][20381] Updated weights for policy 1, policy_version 1278448 (0.0011) [2023-12-27 14:30:47,682][20347] Updated weights for policy 0, policy_version 1279384 (0.0009) [2023-12-27 14:30:47,739][20347] Updated weights for policy 0, policy_version 1279394 (0.0010) [2023-12-27 14:30:47,796][20347] Updated weights for policy 0, policy_version 1279404 (0.0010) [2023-12-27 14:30:48,090][20381] Updated weights for policy 1, policy_version 1278458 (0.0011) [2023-12-27 14:30:48,139][20381] Updated weights for policy 1, policy_version 1278468 (0.0010) [2023-12-27 14:30:48,191][20381] Updated weights for policy 1, policy_version 1278478 (0.0010) [2023-12-27 14:30:48,242][20381] Updated weights for policy 1, policy_version 1278488 (0.0011) [2023-12-27 14:30:48,488][20347] Updated weights for policy 0, policy_version 1279414 (0.0009) [2023-12-27 14:30:48,548][20347] Updated weights for policy 0, policy_version 1279424 (0.0007) [2023-12-27 14:30:48,609][20347] Updated weights for policy 0, policy_version 1279434 (0.0006) [2023-12-27 14:30:48,976][20381] Updated weights for policy 1, policy_version 1278498 (0.0005) [2023-12-27 14:30:49,034][20381] Updated weights for policy 1, policy_version 1278508 (0.0007) [2023-12-27 14:30:49,087][20381] Updated weights for policy 1, policy_version 1278518 (0.0007) [2023-12-27 14:30:49,288][20347] Updated weights for policy 0, policy_version 1279444 (0.0008) [2023-12-27 14:30:49,349][20347] Updated weights for policy 0, policy_version 1279454 (0.0009) [2023-12-27 14:30:49,411][20347] Updated weights for policy 0, policy_version 1279464 (0.0008) [2023-12-27 14:30:49,794][20381] Updated weights for policy 1, policy_version 1278528 (0.0010) [2023-12-27 14:30:49,862][20381] Updated weights for policy 1, policy_version 1278538 (0.0009) [2023-12-27 14:30:49,924][20381] Updated weights for policy 1, policy_version 1278548 (0.0008) [2023-12-27 14:30:50,025][20347] Updated weights for policy 0, policy_version 1279474 (0.0007) [2023-12-27 14:30:50,090][20347] Updated weights for policy 0, policy_version 1279484 (0.0007) [2023-12-27 14:30:50,143][20347] Updated weights for policy 0, policy_version 1279494 (0.0008) [2023-12-27 14:30:50,199][20347] Updated weights for policy 0, policy_version 1279504 (0.0007) [2023-12-27 14:30:50,568][20381] Updated weights for policy 1, policy_version 1278558 (0.0008) [2023-12-27 14:30:50,633][20381] Updated weights for policy 1, policy_version 1278568 (0.0009) [2023-12-27 14:30:50,694][20381] Updated weights for policy 1, policy_version 1278578 (0.0007) [2023-12-27 14:30:50,955][20347] Updated weights for policy 0, policy_version 1279514 (0.0009) [2023-12-27 14:30:51,018][20347] Updated weights for policy 0, policy_version 1279524 (0.0008) [2023-12-27 14:30:51,071][20347] Updated weights for policy 0, policy_version 1279534 (0.0008) [2023-12-27 14:30:51,372][20381] Updated weights for policy 1, policy_version 1278588 (0.0008) [2023-12-27 14:30:51,432][20381] Updated weights for policy 1, policy_version 1278598 (0.0008) [2023-12-27 14:30:51,484][20381] Updated weights for policy 1, policy_version 1278608 (0.0008) [2023-12-27 14:30:51,900][20347] Updated weights for policy 0, policy_version 1279544 (0.0006) [2023-12-27 14:30:51,957][20347] Updated weights for policy 0, policy_version 1279554 (0.0010) [2023-12-27 14:30:52,012][20347] Updated weights for policy 0, policy_version 1279564 (0.0010) [2023-12-27 14:30:52,260][20381] Updated weights for policy 1, policy_version 1278618 (0.0009) [2023-12-27 14:30:52,325][20381] Updated weights for policy 1, policy_version 1278628 (0.0008) [2023-12-27 14:30:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 654983168. Throughput: 0: 10049.4, 1: 10100.6. Samples: 654958152. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:52,363][19170] Avg episode reward: [(0, '931.960'), (1, '420.470')] [2023-12-27 14:30:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001279568_327630848.pth... [2023-12-27 14:30:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001278384_327327744.pth [2023-12-27 14:30:52,391][20381] Updated weights for policy 1, policy_version 1278638 (0.0007) [2023-12-27 14:30:52,453][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001278648_327360512.pth... [2023-12-27 14:30:52,457][20381] Updated weights for policy 1, policy_version 1278648 (0.0006) [2023-12-27 14:30:52,458][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001277464_327057408.pth [2023-12-27 14:30:52,696][20347] Updated weights for policy 0, policy_version 1279574 (0.0009) [2023-12-27 14:30:52,748][20347] Updated weights for policy 0, policy_version 1279584 (0.0008) [2023-12-27 14:30:52,796][20347] Updated weights for policy 0, policy_version 1279594 (0.0007) [2023-12-27 14:30:53,121][20381] Updated weights for policy 1, policy_version 1278658 (0.0005) [2023-12-27 14:30:53,178][20381] Updated weights for policy 1, policy_version 1278668 (0.0007) [2023-12-27 14:30:53,225][20381] Updated weights for policy 1, policy_version 1278678 (0.0007) [2023-12-27 14:30:53,509][20347] Updated weights for policy 0, policy_version 1279604 (0.0008) [2023-12-27 14:30:53,566][20347] Updated weights for policy 0, policy_version 1279614 (0.0009) [2023-12-27 14:30:53,618][20347] Updated weights for policy 0, policy_version 1279624 (0.0009) [2023-12-27 14:30:53,887][20381] Updated weights for policy 1, policy_version 1278688 (0.0007) [2023-12-27 14:30:53,936][20381] Updated weights for policy 1, policy_version 1278698 (0.0007) [2023-12-27 14:30:53,985][20381] Updated weights for policy 1, policy_version 1278708 (0.0007) [2023-12-27 14:30:54,276][20347] Updated weights for policy 0, policy_version 1279634 (0.0009) [2023-12-27 14:30:54,332][20347] Updated weights for policy 0, policy_version 1279644 (0.0005) [2023-12-27 14:30:54,384][20347] Updated weights for policy 0, policy_version 1279654 (0.0006) [2023-12-27 14:30:54,437][20347] Updated weights for policy 0, policy_version 1279664 (0.0008) [2023-12-27 14:30:54,696][20381] Updated weights for policy 1, policy_version 1278718 (0.0006) [2023-12-27 14:30:54,754][20381] Updated weights for policy 1, policy_version 1278728 (0.0005) [2023-12-27 14:30:54,817][20381] Updated weights for policy 1, policy_version 1278738 (0.0005) [2023-12-27 14:30:55,126][20347] Updated weights for policy 0, policy_version 1279674 (0.0007) [2023-12-27 14:30:55,182][20347] Updated weights for policy 0, policy_version 1279684 (0.0008) [2023-12-27 14:30:55,227][20347] Updated weights for policy 0, policy_version 1279694 (0.0008) [2023-12-27 14:30:55,357][20381] Updated weights for policy 1, policy_version 1278748 (0.0007) [2023-12-27 14:30:55,410][20381] Updated weights for policy 1, policy_version 1278758 (0.0009) [2023-12-27 14:30:55,461][20381] Updated weights for policy 1, policy_version 1278768 (0.0007) [2023-12-27 14:30:55,888][20347] Updated weights for policy 0, policy_version 1279704 (0.0008) [2023-12-27 14:30:55,952][20347] Updated weights for policy 0, policy_version 1279714 (0.0008) [2023-12-27 14:30:56,011][20347] Updated weights for policy 0, policy_version 1279724 (0.0007) [2023-12-27 14:30:56,131][20381] Updated weights for policy 1, policy_version 1278778 (0.0005) [2023-12-27 14:30:56,177][20381] Updated weights for policy 1, policy_version 1278788 (0.0008) [2023-12-27 14:30:56,223][20381] Updated weights for policy 1, policy_version 1278798 (0.0009) [2023-12-27 14:30:56,273][20381] Updated weights for policy 1, policy_version 1278808 (0.0008) [2023-12-27 14:30:56,678][20347] Updated weights for policy 0, policy_version 1279734 (0.0007) [2023-12-27 14:30:56,738][20347] Updated weights for policy 0, policy_version 1279744 (0.0008) [2023-12-27 14:30:56,791][20347] Updated weights for policy 0, policy_version 1279754 (0.0008) [2023-12-27 14:30:56,990][20381] Updated weights for policy 1, policy_version 1278818 (0.0007) [2023-12-27 14:30:57,044][20381] Updated weights for policy 1, policy_version 1278828 (0.0008) [2023-12-27 14:30:57,103][20381] Updated weights for policy 1, policy_version 1278838 (0.0008) [2023-12-27 14:30:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20438.4). Total num frames: 655089664. Throughput: 0: 9996.3, 1: 10158.1. Samples: 655079708. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:30:57,363][19170] Avg episode reward: [(0, '931.840'), (1, '536.340')] [2023-12-27 14:30:57,467][20347] Updated weights for policy 0, policy_version 1279764 (0.0008) [2023-12-27 14:30:57,521][20347] Updated weights for policy 0, policy_version 1279774 (0.0008) [2023-12-27 14:30:57,572][20347] Updated weights for policy 0, policy_version 1279784 (0.0007) [2023-12-27 14:30:57,799][20381] Updated weights for policy 1, policy_version 1278848 (0.0007) [2023-12-27 14:30:57,853][20381] Updated weights for policy 1, policy_version 1278858 (0.0007) [2023-12-27 14:30:57,912][20381] Updated weights for policy 1, policy_version 1278868 (0.0008) [2023-12-27 14:30:58,212][20347] Updated weights for policy 0, policy_version 1279794 (0.0005) [2023-12-27 14:30:58,272][20347] Updated weights for policy 0, policy_version 1279804 (0.0008) [2023-12-27 14:30:58,358][20347] Updated weights for policy 0, policy_version 1279814 (0.0007) [2023-12-27 14:30:58,422][20347] Updated weights for policy 0, policy_version 1279824 (0.0008) [2023-12-27 14:30:58,625][20381] Updated weights for policy 1, policy_version 1278878 (0.0008) [2023-12-27 14:30:58,685][20381] Updated weights for policy 1, policy_version 1278888 (0.0009) [2023-12-27 14:30:58,746][20381] Updated weights for policy 1, policy_version 1278898 (0.0010) [2023-12-27 14:30:59,176][20347] Updated weights for policy 0, policy_version 1279834 (0.0007) [2023-12-27 14:30:59,238][20347] Updated weights for policy 0, policy_version 1279844 (0.0008) [2023-12-27 14:30:59,308][20347] Updated weights for policy 0, policy_version 1279854 (0.0009) [2023-12-27 14:30:59,546][20381] Updated weights for policy 1, policy_version 1278908 (0.0009) [2023-12-27 14:30:59,608][20381] Updated weights for policy 1, policy_version 1278918 (0.0006) [2023-12-27 14:30:59,670][20381] Updated weights for policy 1, policy_version 1278928 (0.0008) [2023-12-27 14:31:00,057][20347] Updated weights for policy 0, policy_version 1279864 (0.0006) [2023-12-27 14:31:00,118][20347] Updated weights for policy 0, policy_version 1279874 (0.0006) [2023-12-27 14:31:00,174][20347] Updated weights for policy 0, policy_version 1279884 (0.0008) [2023-12-27 14:31:00,321][20381] Updated weights for policy 1, policy_version 1278938 (0.0008) [2023-12-27 14:31:00,382][20381] Updated weights for policy 1, policy_version 1278948 (0.0008) [2023-12-27 14:31:00,442][20381] Updated weights for policy 1, policy_version 1278958 (0.0008) [2023-12-27 14:31:00,494][20381] Updated weights for policy 1, policy_version 1278968 (0.0008) [2023-12-27 14:31:00,797][20347] Updated weights for policy 0, policy_version 1279894 (0.0006) [2023-12-27 14:31:00,840][20347] Updated weights for policy 0, policy_version 1279904 (0.0005) [2023-12-27 14:31:00,866][20291] KL-divergence is very high: 101.3430 [2023-12-27 14:31:00,884][20347] Updated weights for policy 0, policy_version 1279914 (0.0006) [2023-12-27 14:31:01,207][20381] Updated weights for policy 1, policy_version 1278978 (0.0008) [2023-12-27 14:31:01,279][20381] Updated weights for policy 1, policy_version 1278988 (0.0009) [2023-12-27 14:31:01,331][20381] Updated weights for policy 1, policy_version 1278998 (0.0008) [2023-12-27 14:31:01,617][20347] Updated weights for policy 0, policy_version 1279924 (0.0008) [2023-12-27 14:31:01,681][20347] Updated weights for policy 0, policy_version 1279934 (0.0010) [2023-12-27 14:31:01,739][20347] Updated weights for policy 0, policy_version 1279944 (0.0008) [2023-12-27 14:31:02,058][20381] Updated weights for policy 1, policy_version 1279008 (0.0010) [2023-12-27 14:31:02,121][20381] Updated weights for policy 1, policy_version 1279018 (0.0011) [2023-12-27 14:31:02,174][20381] Updated weights for policy 1, policy_version 1279028 (0.0011) [2023-12-27 14:31:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20070.4, 300 sec: 20410.6). Total num frames: 655187968. Throughput: 0: 10031.4, 1: 10083.3. Samples: 655198932. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:02,363][19170] Avg episode reward: [(0, '892.430'), (1, '599.660')] [2023-12-27 14:31:02,389][20347] Updated weights for policy 0, policy_version 1279954 (0.0007) [2023-12-27 14:31:02,438][20347] Updated weights for policy 0, policy_version 1279964 (0.0007) [2023-12-27 14:31:02,498][20347] Updated weights for policy 0, policy_version 1279974 (0.0007) [2023-12-27 14:31:02,553][20347] Updated weights for policy 0, policy_version 1279984 (0.0007) [2023-12-27 14:31:02,829][20381] Updated weights for policy 1, policy_version 1279038 (0.0008) [2023-12-27 14:31:02,884][20381] Updated weights for policy 1, policy_version 1279048 (0.0007) [2023-12-27 14:31:02,936][20381] Updated weights for policy 1, policy_version 1279058 (0.0008) [2023-12-27 14:31:03,242][20347] Updated weights for policy 0, policy_version 1279994 (0.0008) [2023-12-27 14:31:03,286][20347] Updated weights for policy 0, policy_version 1280004 (0.0009) [2023-12-27 14:31:03,330][20347] Updated weights for policy 0, policy_version 1280014 (0.0007) [2023-12-27 14:31:03,545][20381] Updated weights for policy 1, policy_version 1279068 (0.0007) [2023-12-27 14:31:03,598][20381] Updated weights for policy 1, policy_version 1279078 (0.0010) [2023-12-27 14:31:03,651][20381] Updated weights for policy 1, policy_version 1279088 (0.0009) [2023-12-27 14:31:04,009][20347] Updated weights for policy 0, policy_version 1280024 (0.0010) [2023-12-27 14:31:04,062][20347] Updated weights for policy 0, policy_version 1280034 (0.0011) [2023-12-27 14:31:04,123][20347] Updated weights for policy 0, policy_version 1280044 (0.0011) [2023-12-27 14:31:04,332][20381] Updated weights for policy 1, policy_version 1279098 (0.0010) [2023-12-27 14:31:04,388][20381] Updated weights for policy 1, policy_version 1279108 (0.0010) [2023-12-27 14:31:04,440][20381] Updated weights for policy 1, policy_version 1279118 (0.0007) [2023-12-27 14:31:04,495][20381] Updated weights for policy 1, policy_version 1279128 (0.0010) [2023-12-27 14:31:04,816][20347] Updated weights for policy 0, policy_version 1280054 (0.0007) [2023-12-27 14:31:04,866][20347] Updated weights for policy 0, policy_version 1280064 (0.0005) [2023-12-27 14:31:04,911][20347] Updated weights for policy 0, policy_version 1280074 (0.0005) [2023-12-27 14:31:05,138][20381] Updated weights for policy 1, policy_version 1279138 (0.0006) [2023-12-27 14:31:05,182][20381] Updated weights for policy 1, policy_version 1279148 (0.0010) [2023-12-27 14:31:05,235][20381] Updated weights for policy 1, policy_version 1279158 (0.0005) [2023-12-27 14:31:05,515][20347] Updated weights for policy 0, policy_version 1280084 (0.0007) [2023-12-27 14:31:05,571][20347] Updated weights for policy 0, policy_version 1280094 (0.0011) [2023-12-27 14:31:05,619][20347] Updated weights for policy 0, policy_version 1280104 (0.0010) [2023-12-27 14:31:05,875][20381] Updated weights for policy 1, policy_version 1279168 (0.0007) [2023-12-27 14:31:05,939][20381] Updated weights for policy 1, policy_version 1279178 (0.0007) [2023-12-27 14:31:05,984][20381] Updated weights for policy 1, policy_version 1279188 (0.0007) [2023-12-27 14:31:06,320][20347] Updated weights for policy 0, policy_version 1280114 (0.0010) [2023-12-27 14:31:06,374][20347] Updated weights for policy 0, policy_version 1280124 (0.0010) [2023-12-27 14:31:06,439][20347] Updated weights for policy 0, policy_version 1280134 (0.0011) [2023-12-27 14:31:06,500][20347] Updated weights for policy 0, policy_version 1280144 (0.0010) [2023-12-27 14:31:06,718][20381] Updated weights for policy 1, policy_version 1279198 (0.0008) [2023-12-27 14:31:06,774][20381] Updated weights for policy 1, policy_version 1279208 (0.0008) [2023-12-27 14:31:06,823][20381] Updated weights for policy 1, policy_version 1279218 (0.0008) [2023-12-27 14:31:07,199][20347] Updated weights for policy 0, policy_version 1280154 (0.0008) [2023-12-27 14:31:07,260][20347] Updated weights for policy 0, policy_version 1280164 (0.0005) [2023-12-27 14:31:07,320][20347] Updated weights for policy 0, policy_version 1280174 (0.0006) [2023-12-27 14:31:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 655294464. Throughput: 0: 10069.2, 1: 10127.4. Samples: 655262152. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:07,363][19170] Avg episode reward: [(0, '921.800'), (1, '680.490')] [2023-12-27 14:31:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001279224_327507968.pth... [2023-12-27 14:31:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001280176_327786496.pth... [2023-12-27 14:31:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001278040_327204864.pth [2023-12-27 14:31:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001278992_327483392.pth [2023-12-27 14:31:07,586][20381] Updated weights for policy 1, policy_version 1279228 (0.0007) [2023-12-27 14:31:07,639][20381] Updated weights for policy 1, policy_version 1279238 (0.0005) [2023-12-27 14:31:07,692][20381] Updated weights for policy 1, policy_version 1279248 (0.0006) [2023-12-27 14:31:07,918][20347] Updated weights for policy 0, policy_version 1280184 (0.0007) [2023-12-27 14:31:07,977][20347] Updated weights for policy 0, policy_version 1280194 (0.0008) [2023-12-27 14:31:07,984][20291] KL-divergence is very high: 195.3903 [2023-12-27 14:31:08,037][20291] KL-divergence is very high: 143.7446 [2023-12-27 14:31:08,043][20347] Updated weights for policy 0, policy_version 1280204 (0.0008) [2023-12-27 14:31:08,409][20381] Updated weights for policy 1, policy_version 1279258 (0.0008) [2023-12-27 14:31:08,465][20381] Updated weights for policy 1, policy_version 1279268 (0.0008) [2023-12-27 14:31:08,518][20381] Updated weights for policy 1, policy_version 1279278 (0.0008) [2023-12-27 14:31:08,575][20381] Updated weights for policy 1, policy_version 1279288 (0.0008) [2023-12-27 14:31:08,715][20347] Updated weights for policy 0, policy_version 1280214 (0.0006) [2023-12-27 14:31:08,776][20347] Updated weights for policy 0, policy_version 1280224 (0.0006) [2023-12-27 14:31:08,838][20347] Updated weights for policy 0, policy_version 1280234 (0.0006) [2023-12-27 14:31:09,285][20381] Updated weights for policy 1, policy_version 1279298 (0.0007) [2023-12-27 14:31:09,361][20381] Updated weights for policy 1, policy_version 1279308 (0.0009) [2023-12-27 14:31:09,433][20381] Updated weights for policy 1, policy_version 1279318 (0.0008) [2023-12-27 14:31:09,514][20347] Updated weights for policy 0, policy_version 1280244 (0.0007) [2023-12-27 14:31:09,566][20347] Updated weights for policy 0, policy_version 1280254 (0.0008) [2023-12-27 14:31:09,625][20347] Updated weights for policy 0, policy_version 1280264 (0.0008) [2023-12-27 14:31:10,110][20381] Updated weights for policy 1, policy_version 1279328 (0.0006) [2023-12-27 14:31:10,173][20381] Updated weights for policy 1, policy_version 1279338 (0.0006) [2023-12-27 14:31:10,235][20381] Updated weights for policy 1, policy_version 1279348 (0.0006) [2023-12-27 14:31:10,382][20347] Updated weights for policy 0, policy_version 1280274 (0.0008) [2023-12-27 14:31:10,441][20347] Updated weights for policy 0, policy_version 1280284 (0.0007) [2023-12-27 14:31:10,491][20347] Updated weights for policy 0, policy_version 1280294 (0.0007) [2023-12-27 14:31:10,542][20347] Updated weights for policy 0, policy_version 1280304 (0.0007) [2023-12-27 14:31:10,878][20381] Updated weights for policy 1, policy_version 1279358 (0.0006) [2023-12-27 14:31:10,937][20381] Updated weights for policy 1, policy_version 1279368 (0.0005) [2023-12-27 14:31:10,996][20381] Updated weights for policy 1, policy_version 1279378 (0.0006) [2023-12-27 14:31:11,207][20347] Updated weights for policy 0, policy_version 1280314 (0.0006) [2023-12-27 14:31:11,268][20347] Updated weights for policy 0, policy_version 1280324 (0.0006) [2023-12-27 14:31:11,332][20347] Updated weights for policy 0, policy_version 1280334 (0.0006) [2023-12-27 14:31:11,741][20381] Updated weights for policy 1, policy_version 1279388 (0.0008) [2023-12-27 14:31:11,811][20381] Updated weights for policy 1, policy_version 1279398 (0.0009) [2023-12-27 14:31:11,877][20381] Updated weights for policy 1, policy_version 1279408 (0.0009) [2023-12-27 14:31:11,976][20347] Updated weights for policy 0, policy_version 1280344 (0.0006) [2023-12-27 14:31:12,034][20347] Updated weights for policy 0, policy_version 1280354 (0.0008) [2023-12-27 14:31:12,092][20347] Updated weights for policy 0, policy_version 1280364 (0.0008) [2023-12-27 14:31:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 655392768. Throughput: 0: 10118.0, 1: 10081.1. Samples: 655383568. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:12,363][19170] Avg episode reward: [(0, '941.370'), (1, '761.980')] [2023-12-27 14:31:12,508][20381] Updated weights for policy 1, policy_version 1279418 (0.0007) [2023-12-27 14:31:12,582][20381] Updated weights for policy 1, policy_version 1279428 (0.0008) [2023-12-27 14:31:12,647][20381] Updated weights for policy 1, policy_version 1279438 (0.0008) [2023-12-27 14:31:12,707][20381] Updated weights for policy 1, policy_version 1279448 (0.0008) [2023-12-27 14:31:12,804][20347] Updated weights for policy 0, policy_version 1280374 (0.0009) [2023-12-27 14:31:12,862][20347] Updated weights for policy 0, policy_version 1280384 (0.0008) [2023-12-27 14:31:12,917][20347] Updated weights for policy 0, policy_version 1280394 (0.0008) [2023-12-27 14:31:13,349][20381] Updated weights for policy 1, policy_version 1279458 (0.0006) [2023-12-27 14:31:13,396][20381] Updated weights for policy 1, policy_version 1279468 (0.0005) [2023-12-27 14:31:13,447][20381] Updated weights for policy 1, policy_version 1279478 (0.0005) [2023-12-27 14:31:13,611][20347] Updated weights for policy 0, policy_version 1280404 (0.0008) [2023-12-27 14:31:13,668][20347] Updated weights for policy 0, policy_version 1280414 (0.0008) [2023-12-27 14:31:13,720][20347] Updated weights for policy 0, policy_version 1280424 (0.0009) [2023-12-27 14:31:14,046][20381] Updated weights for policy 1, policy_version 1279488 (0.0008) [2023-12-27 14:31:14,105][20381] Updated weights for policy 1, policy_version 1279498 (0.0008) [2023-12-27 14:31:14,166][20381] Updated weights for policy 1, policy_version 1279508 (0.0006) [2023-12-27 14:31:14,294][20347] Updated weights for policy 0, policy_version 1280434 (0.0009) [2023-12-27 14:31:14,352][20347] Updated weights for policy 0, policy_version 1280444 (0.0006) [2023-12-27 14:31:14,410][20347] Updated weights for policy 0, policy_version 1280454 (0.0005) [2023-12-27 14:31:14,472][20347] Updated weights for policy 0, policy_version 1280464 (0.0005) [2023-12-27 14:31:14,898][20381] Updated weights for policy 1, policy_version 1279518 (0.0007) [2023-12-27 14:31:14,947][20381] Updated weights for policy 1, policy_version 1279528 (0.0008) [2023-12-27 14:31:15,002][20381] Updated weights for policy 1, policy_version 1279538 (0.0009) [2023-12-27 14:31:15,122][20347] Updated weights for policy 0, policy_version 1280474 (0.0006) [2023-12-27 14:31:15,184][20347] Updated weights for policy 0, policy_version 1280484 (0.0006) [2023-12-27 14:31:15,246][20347] Updated weights for policy 0, policy_version 1280494 (0.0006) [2023-12-27 14:31:15,724][20381] Updated weights for policy 1, policy_version 1279548 (0.0009) [2023-12-27 14:31:15,772][20381] Updated weights for policy 1, policy_version 1279558 (0.0010) [2023-12-27 14:31:15,838][20381] Updated weights for policy 1, policy_version 1279568 (0.0009) [2023-12-27 14:31:15,876][20347] Updated weights for policy 0, policy_version 1280504 (0.0005) [2023-12-27 14:31:15,937][20347] Updated weights for policy 0, policy_version 1280514 (0.0005) [2023-12-27 14:31:15,996][20347] Updated weights for policy 0, policy_version 1280524 (0.0005) [2023-12-27 14:31:16,486][20381] Updated weights for policy 1, policy_version 1279578 (0.0011) [2023-12-27 14:31:16,540][20381] Updated weights for policy 1, policy_version 1279588 (0.0010) [2023-12-27 14:31:16,582][20347] Updated weights for policy 0, policy_version 1280534 (0.0008) [2023-12-27 14:31:16,595][20381] Updated weights for policy 1, policy_version 1279598 (0.0010) [2023-12-27 14:31:16,641][20347] Updated weights for policy 0, policy_version 1280544 (0.0010) [2023-12-27 14:31:16,648][20381] Updated weights for policy 1, policy_version 1279608 (0.0010) [2023-12-27 14:31:16,698][20347] Updated weights for policy 0, policy_version 1280554 (0.0008) [2023-12-27 14:31:17,262][20381] Updated weights for policy 1, policy_version 1279618 (0.0010) [2023-12-27 14:31:17,272][20347] Updated weights for policy 0, policy_version 1280564 (0.0010) [2023-12-27 14:31:17,313][20381] Updated weights for policy 1, policy_version 1279628 (0.0007) [2023-12-27 14:31:17,327][20347] Updated weights for policy 0, policy_version 1280574 (0.0010) [2023-12-27 14:31:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 655491072. Throughput: 0: 10207.5, 1: 10104.8. Samples: 655509380. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:17,363][19170] Avg episode reward: [(0, '951.130'), (1, '707.680')] [2023-12-27 14:31:17,368][20381] Updated weights for policy 1, policy_version 1279638 (0.0006) [2023-12-27 14:31:17,374][20347] Updated weights for policy 0, policy_version 1280584 (0.0010) [2023-12-27 14:31:17,961][20347] Updated weights for policy 0, policy_version 1280594 (0.0009) [2023-12-27 14:31:18,016][20347] Updated weights for policy 0, policy_version 1280604 (0.0005) [2023-12-27 14:31:18,067][20347] Updated weights for policy 0, policy_version 1280614 (0.0005) [2023-12-27 14:31:18,070][20381] Updated weights for policy 1, policy_version 1279648 (0.0007) [2023-12-27 14:31:18,122][20347] Updated weights for policy 0, policy_version 1280624 (0.0008) [2023-12-27 14:31:18,125][20381] Updated weights for policy 1, policy_version 1279658 (0.0007) [2023-12-27 14:31:18,179][20381] Updated weights for policy 1, policy_version 1279668 (0.0006) [2023-12-27 14:31:18,738][20347] Updated weights for policy 0, policy_version 1280634 (0.0005) [2023-12-27 14:31:18,799][20347] Updated weights for policy 0, policy_version 1280644 (0.0008) [2023-12-27 14:31:18,827][20381] Updated weights for policy 1, policy_version 1279678 (0.0006) [2023-12-27 14:31:18,855][20347] Updated weights for policy 0, policy_version 1280654 (0.0010) [2023-12-27 14:31:18,886][20381] Updated weights for policy 1, policy_version 1279688 (0.0007) [2023-12-27 14:31:18,945][20381] Updated weights for policy 1, policy_version 1279698 (0.0008) [2023-12-27 14:31:19,549][20347] Updated weights for policy 0, policy_version 1280664 (0.0010) [2023-12-27 14:31:19,602][20347] Updated weights for policy 0, policy_version 1280674 (0.0010) [2023-12-27 14:31:19,659][20347] Updated weights for policy 0, policy_version 1280684 (0.0011) [2023-12-27 14:31:19,700][20381] Updated weights for policy 1, policy_version 1279708 (0.0007) [2023-12-27 14:31:19,753][20381] Updated weights for policy 1, policy_version 1279718 (0.0007) [2023-12-27 14:31:19,812][20381] Updated weights for policy 1, policy_version 1279728 (0.0007) [2023-12-27 14:31:20,310][20347] Updated weights for policy 0, policy_version 1280694 (0.0009) [2023-12-27 14:31:20,370][20347] Updated weights for policy 0, policy_version 1280704 (0.0008) [2023-12-27 14:31:20,430][20347] Updated weights for policy 0, policy_version 1280714 (0.0008) [2023-12-27 14:31:20,583][20381] Updated weights for policy 1, policy_version 1279738 (0.0008) [2023-12-27 14:31:20,643][20381] Updated weights for policy 1, policy_version 1279748 (0.0008) [2023-12-27 14:31:20,704][20381] Updated weights for policy 1, policy_version 1279758 (0.0008) [2023-12-27 14:31:20,762][20381] Updated weights for policy 1, policy_version 1279768 (0.0008) [2023-12-27 14:31:21,176][20347] Updated weights for policy 0, policy_version 1280724 (0.0008) [2023-12-27 14:31:21,240][20347] Updated weights for policy 0, policy_version 1280734 (0.0008) [2023-12-27 14:31:21,302][20347] Updated weights for policy 0, policy_version 1280744 (0.0008) [2023-12-27 14:31:21,513][20381] Updated weights for policy 1, policy_version 1279778 (0.0006) [2023-12-27 14:31:21,578][20381] Updated weights for policy 1, policy_version 1279788 (0.0008) [2023-12-27 14:31:21,645][20381] Updated weights for policy 1, policy_version 1279798 (0.0007) [2023-12-27 14:31:21,985][20347] Updated weights for policy 0, policy_version 1280754 (0.0007) [2023-12-27 14:31:22,052][20347] Updated weights for policy 0, policy_version 1280764 (0.0008) [2023-12-27 14:31:22,115][20347] Updated weights for policy 0, policy_version 1280774 (0.0007) [2023-12-27 14:31:22,177][20347] Updated weights for policy 0, policy_version 1280784 (0.0009) [2023-12-27 14:31:22,241][20381] Updated weights for policy 1, policy_version 1279808 (0.0008) [2023-12-27 14:31:22,309][20381] Updated weights for policy 1, policy_version 1279818 (0.0009) [2023-12-27 14:31:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 655597568. Throughput: 0: 10219.6, 1: 10087.8. Samples: 655571140. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:22,363][19170] Avg episode reward: [(0, '922.010'), (1, '710.180')] [2023-12-27 14:31:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001280784_327942144.pth... [2023-12-27 14:31:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001279568_327630848.pth [2023-12-27 14:31:22,374][20381] Updated weights for policy 1, policy_version 1279828 (0.0008) [2023-12-27 14:31:22,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001279832_327663616.pth... [2023-12-27 14:31:22,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001278648_327360512.pth [2023-12-27 14:31:22,909][20347] Updated weights for policy 0, policy_version 1280794 (0.0008) [2023-12-27 14:31:22,961][20347] Updated weights for policy 0, policy_version 1280804 (0.0007) [2023-12-27 14:31:23,013][20347] Updated weights for policy 0, policy_version 1280814 (0.0008) [2023-12-27 14:31:23,093][20381] Updated weights for policy 1, policy_version 1279838 (0.0008) [2023-12-27 14:31:23,146][20381] Updated weights for policy 1, policy_version 1279848 (0.0008) [2023-12-27 14:31:23,200][20381] Updated weights for policy 1, policy_version 1279858 (0.0009) [2023-12-27 14:31:23,706][20347] Updated weights for policy 0, policy_version 1280824 (0.0008) [2023-12-27 14:31:23,767][20347] Updated weights for policy 0, policy_version 1280834 (0.0008) [2023-12-27 14:31:23,826][20347] Updated weights for policy 0, policy_version 1280844 (0.0008) [2023-12-27 14:31:23,911][20381] Updated weights for policy 1, policy_version 1279868 (0.0010) [2023-12-27 14:31:23,967][20381] Updated weights for policy 1, policy_version 1279878 (0.0008) [2023-12-27 14:31:24,012][20381] Updated weights for policy 1, policy_version 1279888 (0.0007) [2023-12-27 14:31:24,479][20347] Updated weights for policy 0, policy_version 1280854 (0.0006) [2023-12-27 14:31:24,533][20347] Updated weights for policy 0, policy_version 1280864 (0.0006) [2023-12-27 14:31:24,584][20347] Updated weights for policy 0, policy_version 1280874 (0.0007) [2023-12-27 14:31:24,756][20381] Updated weights for policy 1, policy_version 1279898 (0.0008) [2023-12-27 14:31:24,808][20381] Updated weights for policy 1, policy_version 1279908 (0.0009) [2023-12-27 14:31:24,857][20381] Updated weights for policy 1, policy_version 1279918 (0.0007) [2023-12-27 14:31:24,916][20381] Updated weights for policy 1, policy_version 1279928 (0.0007) [2023-12-27 14:31:25,140][20347] Updated weights for policy 0, policy_version 1280884 (0.0006) [2023-12-27 14:31:25,190][20347] Updated weights for policy 0, policy_version 1280894 (0.0006) [2023-12-27 14:31:25,238][20347] Updated weights for policy 0, policy_version 1280904 (0.0007) [2023-12-27 14:31:25,606][20381] Updated weights for policy 1, policy_version 1279938 (0.0008) [2023-12-27 14:31:25,658][20381] Updated weights for policy 1, policy_version 1279948 (0.0007) [2023-12-27 14:31:25,706][20381] Updated weights for policy 1, policy_version 1279958 (0.0008) [2023-12-27 14:31:25,944][20347] Updated weights for policy 0, policy_version 1280914 (0.0006) [2023-12-27 14:31:25,992][20347] Updated weights for policy 0, policy_version 1280924 (0.0007) [2023-12-27 14:31:26,006][20291] KL-divergence is very high: 177.5710 [2023-12-27 14:31:26,021][20291] KL-divergence is very high: 196.8612 [2023-12-27 14:31:26,043][20347] Updated weights for policy 0, policy_version 1280934 (0.0008) [2023-12-27 14:31:26,049][20291] KL-divergence is very high: 173.6666 [2023-12-27 14:31:26,064][20291] KL-divergence is very high: 109.7664 [2023-12-27 14:31:26,092][20347] Updated weights for policy 0, policy_version 1280944 (0.0007) [2023-12-27 14:31:26,376][20381] Updated weights for policy 1, policy_version 1279968 (0.0006) [2023-12-27 14:31:26,431][20381] Updated weights for policy 1, policy_version 1279978 (0.0007) [2023-12-27 14:31:26,479][20381] Updated weights for policy 1, policy_version 1279988 (0.0007) [2023-12-27 14:31:26,758][20347] Updated weights for policy 0, policy_version 1280954 (0.0005) [2023-12-27 14:31:26,805][20347] Updated weights for policy 0, policy_version 1280964 (0.0005) [2023-12-27 14:31:26,854][20347] Updated weights for policy 0, policy_version 1280974 (0.0007) [2023-12-27 14:31:27,151][20381] Updated weights for policy 1, policy_version 1279998 (0.0008) [2023-12-27 14:31:27,204][20381] Updated weights for policy 1, policy_version 1280008 (0.0008) [2023-12-27 14:31:27,252][20381] Updated weights for policy 1, policy_version 1280018 (0.0007) [2023-12-27 14:31:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 655704064. Throughput: 0: 10270.3, 1: 10120.2. Samples: 655692544. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:27,363][19170] Avg episode reward: [(0, '931.910'), (1, '759.250')] [2023-12-27 14:31:27,461][20347] Updated weights for policy 0, policy_version 1280984 (0.0009) [2023-12-27 14:31:27,507][20347] Updated weights for policy 0, policy_version 1280994 (0.0007) [2023-12-27 14:31:27,560][20347] Updated weights for policy 0, policy_version 1281004 (0.0007) [2023-12-27 14:31:27,924][20381] Updated weights for policy 1, policy_version 1280028 (0.0006) [2023-12-27 14:31:27,968][20381] Updated weights for policy 1, policy_version 1280038 (0.0007) [2023-12-27 14:31:28,020][20381] Updated weights for policy 1, policy_version 1280048 (0.0007) [2023-12-27 14:31:28,267][20347] Updated weights for policy 0, policy_version 1281014 (0.0008) [2023-12-27 14:31:28,317][20347] Updated weights for policy 0, policy_version 1281024 (0.0009) [2023-12-27 14:31:28,378][20347] Updated weights for policy 0, policy_version 1281034 (0.0008) [2023-12-27 14:31:28,681][20381] Updated weights for policy 1, policy_version 1280058 (0.0006) [2023-12-27 14:31:28,740][20381] Updated weights for policy 1, policy_version 1280068 (0.0008) [2023-12-27 14:31:28,806][20381] Updated weights for policy 1, policy_version 1280078 (0.0008) [2023-12-27 14:31:28,863][20381] Updated weights for policy 1, policy_version 1280088 (0.0008) [2023-12-27 14:31:28,991][20347] Updated weights for policy 0, policy_version 1281044 (0.0006) [2023-12-27 14:31:29,054][20347] Updated weights for policy 0, policy_version 1281054 (0.0005) [2023-12-27 14:31:29,109][20347] Updated weights for policy 0, policy_version 1281064 (0.0006) [2023-12-27 14:31:29,550][20381] Updated weights for policy 1, policy_version 1280098 (0.0006) [2023-12-27 14:31:29,602][20381] Updated weights for policy 1, policy_version 1280108 (0.0007) [2023-12-27 14:31:29,657][20381] Updated weights for policy 1, policy_version 1280118 (0.0008) [2023-12-27 14:31:29,718][20347] Updated weights for policy 0, policy_version 1281074 (0.0005) [2023-12-27 14:31:29,771][20347] Updated weights for policy 0, policy_version 1281084 (0.0008) [2023-12-27 14:31:29,835][20347] Updated weights for policy 0, policy_version 1281094 (0.0007) [2023-12-27 14:31:29,893][20347] Updated weights for policy 0, policy_version 1281104 (0.0008) [2023-12-27 14:31:30,350][20381] Updated weights for policy 1, policy_version 1280128 (0.0007) [2023-12-27 14:31:30,412][20381] Updated weights for policy 1, policy_version 1280138 (0.0008) [2023-12-27 14:31:30,467][20381] Updated weights for policy 1, policy_version 1280148 (0.0008) [2023-12-27 14:31:30,592][20347] Updated weights for policy 0, policy_version 1281114 (0.0008) [2023-12-27 14:31:30,656][20347] Updated weights for policy 0, policy_version 1281124 (0.0007) [2023-12-27 14:31:30,714][20347] Updated weights for policy 0, policy_version 1281134 (0.0008) [2023-12-27 14:31:31,108][20381] Updated weights for policy 1, policy_version 1280158 (0.0007) [2023-12-27 14:31:31,174][20381] Updated weights for policy 1, policy_version 1280168 (0.0008) [2023-12-27 14:31:31,227][20381] Updated weights for policy 1, policy_version 1280178 (0.0006) [2023-12-27 14:31:31,421][20347] Updated weights for policy 0, policy_version 1281144 (0.0008) [2023-12-27 14:31:31,479][20347] Updated weights for policy 0, policy_version 1281154 (0.0008) [2023-12-27 14:31:31,529][20347] Updated weights for policy 0, policy_version 1281164 (0.0008) [2023-12-27 14:31:31,895][20381] Updated weights for policy 1, policy_version 1280188 (0.0007) [2023-12-27 14:31:31,959][20381] Updated weights for policy 1, policy_version 1280198 (0.0006) [2023-12-27 14:31:32,017][20381] Updated weights for policy 1, policy_version 1280208 (0.0006) [2023-12-27 14:31:32,302][20347] Updated weights for policy 0, policy_version 1281174 (0.0009) [2023-12-27 14:31:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 655802368. Throughput: 0: 10298.7, 1: 10130.7. Samples: 655817112. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:32,363][19170] Avg episode reward: [(0, '941.680'), (1, '787.870')] [2023-12-27 14:31:32,365][20347] Updated weights for policy 0, policy_version 1281184 (0.0008) [2023-12-27 14:31:32,404][20291] KL-divergence is very high: 542.4576 [2023-12-27 14:31:32,430][20347] Updated weights for policy 0, policy_version 1281194 (0.0008) [2023-12-27 14:31:32,433][20291] KL-divergence is very high: 104.1944 [2023-12-27 14:31:32,452][20291] KL-divergence is very high: 358.9893 [2023-12-27 14:31:32,614][20381] Updated weights for policy 1, policy_version 1280218 (0.0006) [2023-12-27 14:31:32,681][20381] Updated weights for policy 1, policy_version 1280228 (0.0008) [2023-12-27 14:31:32,740][20381] Updated weights for policy 1, policy_version 1280238 (0.0008) [2023-12-27 14:31:32,798][20381] Updated weights for policy 1, policy_version 1280248 (0.0008) [2023-12-27 14:31:33,090][20347] Updated weights for policy 0, policy_version 1281204 (0.0007) [2023-12-27 14:31:33,144][20347] Updated weights for policy 0, policy_version 1281214 (0.0005) [2023-12-27 14:31:33,209][20347] Updated weights for policy 0, policy_version 1281224 (0.0007) [2023-12-27 14:31:33,413][20381] Updated weights for policy 1, policy_version 1280258 (0.0005) [2023-12-27 14:31:33,476][20381] Updated weights for policy 1, policy_version 1280268 (0.0005) [2023-12-27 14:31:33,541][20381] Updated weights for policy 1, policy_version 1280278 (0.0007) [2023-12-27 14:31:33,830][20347] Updated weights for policy 0, policy_version 1281234 (0.0007) [2023-12-27 14:31:33,878][20347] Updated weights for policy 0, policy_version 1281244 (0.0005) [2023-12-27 14:31:33,931][20347] Updated weights for policy 0, policy_version 1281254 (0.0005) [2023-12-27 14:31:33,982][20347] Updated weights for policy 0, policy_version 1281264 (0.0005) [2023-12-27 14:31:34,151][20381] Updated weights for policy 1, policy_version 1280288 (0.0008) [2023-12-27 14:31:34,219][20381] Updated weights for policy 1, policy_version 1280298 (0.0007) [2023-12-27 14:31:34,275][20381] Updated weights for policy 1, policy_version 1280308 (0.0007) [2023-12-27 14:31:34,613][20347] Updated weights for policy 0, policy_version 1281274 (0.0005) [2023-12-27 14:31:34,677][20347] Updated weights for policy 0, policy_version 1281284 (0.0006) [2023-12-27 14:31:34,739][20347] Updated weights for policy 0, policy_version 1281294 (0.0006) [2023-12-27 14:31:34,968][20381] Updated weights for policy 1, policy_version 1280318 (0.0006) [2023-12-27 14:31:35,029][20381] Updated weights for policy 1, policy_version 1280328 (0.0006) [2023-12-27 14:31:35,081][20381] Updated weights for policy 1, policy_version 1280338 (0.0007) [2023-12-27 14:31:35,355][20347] Updated weights for policy 0, policy_version 1281304 (0.0007) [2023-12-27 14:31:35,409][20347] Updated weights for policy 0, policy_version 1281314 (0.0008) [2023-12-27 14:31:35,467][20347] Updated weights for policy 0, policy_version 1281324 (0.0007) [2023-12-27 14:31:35,730][20381] Updated weights for policy 1, policy_version 1280348 (0.0008) [2023-12-27 14:31:35,790][20381] Updated weights for policy 1, policy_version 1280358 (0.0007) [2023-12-27 14:31:35,845][20381] Updated weights for policy 1, policy_version 1280368 (0.0008) [2023-12-27 14:31:36,182][20347] Updated weights for policy 0, policy_version 1281334 (0.0008) [2023-12-27 14:31:36,242][20347] Updated weights for policy 0, policy_version 1281344 (0.0008) [2023-12-27 14:31:36,299][20347] Updated weights for policy 0, policy_version 1281354 (0.0008) [2023-12-27 14:31:36,544][20381] Updated weights for policy 1, policy_version 1280378 (0.0007) [2023-12-27 14:31:36,606][20381] Updated weights for policy 1, policy_version 1280388 (0.0006) [2023-12-27 14:31:36,668][20381] Updated weights for policy 1, policy_version 1280398 (0.0006) [2023-12-27 14:31:36,731][20381] Updated weights for policy 1, policy_version 1280408 (0.0006) [2023-12-27 14:31:37,044][20347] Updated weights for policy 0, policy_version 1281364 (0.0007) [2023-12-27 14:31:37,111][20347] Updated weights for policy 0, policy_version 1281374 (0.0007) [2023-12-27 14:31:37,174][20347] Updated weights for policy 0, policy_version 1281384 (0.0006) [2023-12-27 14:31:37,358][20381] Updated weights for policy 1, policy_version 1280418 (0.0008) [2023-12-27 14:31:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 655908864. Throughput: 0: 10323.9, 1: 10185.6. Samples: 655881080. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:37,363][19170] Avg episode reward: [(0, '882.950'), (1, '759.270')] [2023-12-27 14:31:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001281392_328097792.pth... [2023-12-27 14:31:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001280176_327786496.pth [2023-12-27 14:31:37,416][20381] Updated weights for policy 1, policy_version 1280428 (0.0008) [2023-12-27 14:31:37,467][20381] Updated weights for policy 1, policy_version 1280438 (0.0008) [2023-12-27 14:31:37,476][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001280440_327819264.pth... [2023-12-27 14:31:37,479][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001279224_327507968.pth [2023-12-27 14:31:37,842][20347] Updated weights for policy 0, policy_version 1281394 (0.0006) [2023-12-27 14:31:37,903][20347] Updated weights for policy 0, policy_version 1281404 (0.0005) [2023-12-27 14:31:37,950][20347] Updated weights for policy 0, policy_version 1281414 (0.0005) [2023-12-27 14:31:38,004][20347] Updated weights for policy 0, policy_version 1281424 (0.0005) [2023-12-27 14:31:38,169][20381] Updated weights for policy 1, policy_version 1280448 (0.0007) [2023-12-27 14:31:38,230][20381] Updated weights for policy 1, policy_version 1280458 (0.0008) [2023-12-27 14:31:38,283][20381] Updated weights for policy 1, policy_version 1280468 (0.0008) [2023-12-27 14:31:38,688][20347] Updated weights for policy 0, policy_version 1281434 (0.0007) [2023-12-27 14:31:38,749][20347] Updated weights for policy 0, policy_version 1281444 (0.0007) [2023-12-27 14:31:38,806][20347] Updated weights for policy 0, policy_version 1281454 (0.0008) [2023-12-27 14:31:38,957][20381] Updated weights for policy 1, policy_version 1280478 (0.0007) [2023-12-27 14:31:39,019][20381] Updated weights for policy 1, policy_version 1280488 (0.0007) [2023-12-27 14:31:39,078][20381] Updated weights for policy 1, policy_version 1280498 (0.0008) [2023-12-27 14:31:39,539][20347] Updated weights for policy 0, policy_version 1281464 (0.0007) [2023-12-27 14:31:39,600][20347] Updated weights for policy 0, policy_version 1281474 (0.0008) [2023-12-27 14:31:39,663][20347] Updated weights for policy 0, policy_version 1281484 (0.0011) [2023-12-27 14:31:39,817][20381] Updated weights for policy 1, policy_version 1280508 (0.0008) [2023-12-27 14:31:39,886][20381] Updated weights for policy 1, policy_version 1280518 (0.0008) [2023-12-27 14:31:39,943][20381] Updated weights for policy 1, policy_version 1280528 (0.0008) [2023-12-27 14:31:40,307][20347] Updated weights for policy 0, policy_version 1281494 (0.0009) [2023-12-27 14:31:40,373][20347] Updated weights for policy 0, policy_version 1281504 (0.0008) [2023-12-27 14:31:40,436][20347] Updated weights for policy 0, policy_version 1281514 (0.0008) [2023-12-27 14:31:40,678][20381] Updated weights for policy 1, policy_version 1280538 (0.0008) [2023-12-27 14:31:40,728][20381] Updated weights for policy 1, policy_version 1280548 (0.0008) [2023-12-27 14:31:40,774][20381] Updated weights for policy 1, policy_version 1280558 (0.0008) [2023-12-27 14:31:40,822][20381] Updated weights for policy 1, policy_version 1280568 (0.0008) [2023-12-27 14:31:41,058][20347] Updated weights for policy 0, policy_version 1281524 (0.0008) [2023-12-27 14:31:41,117][20347] Updated weights for policy 0, policy_version 1281534 (0.0008) [2023-12-27 14:31:41,179][20347] Updated weights for policy 0, policy_version 1281544 (0.0008) [2023-12-27 14:31:41,637][20381] Updated weights for policy 1, policy_version 1280578 (0.0008) [2023-12-27 14:31:41,704][20381] Updated weights for policy 1, policy_version 1280588 (0.0008) [2023-12-27 14:31:41,773][20381] Updated weights for policy 1, policy_version 1280598 (0.0007) [2023-12-27 14:31:41,969][20347] Updated weights for policy 0, policy_version 1281554 (0.0008) [2023-12-27 14:31:42,022][20291] KL-divergence is very high: 354.4861 [2023-12-27 14:31:42,030][20347] Updated weights for policy 0, policy_version 1281564 (0.0008) [2023-12-27 14:31:42,087][20347] Updated weights for policy 0, policy_version 1281574 (0.0008) [2023-12-27 14:31:42,140][20347] Updated weights for policy 0, policy_version 1281584 (0.0007) [2023-12-27 14:31:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 656007168. Throughput: 0: 10350.6, 1: 10114.2. Samples: 656000628. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:42,365][19170] Avg episode reward: [(0, '883.050'), (1, '703.830')] [2023-12-27 14:31:42,483][20381] Updated weights for policy 1, policy_version 1280608 (0.0007) [2023-12-27 14:31:42,536][20381] Updated weights for policy 1, policy_version 1280618 (0.0008) [2023-12-27 14:31:42,588][20381] Updated weights for policy 1, policy_version 1280628 (0.0008) [2023-12-27 14:31:42,881][20291] KL-divergence is very high: 101.9426 [2023-12-27 14:31:42,896][20347] Updated weights for policy 0, policy_version 1281594 (0.0010) [2023-12-27 14:31:42,897][20291] KL-divergence is very high: 281.7292 [2023-12-27 14:31:42,925][20291] KL-divergence is very high: 136.3875 [2023-12-27 14:31:42,940][20291] KL-divergence is very high: 150.8042 [2023-12-27 14:31:42,951][20347] Updated weights for policy 0, policy_version 1281604 (0.0010) [2023-12-27 14:31:43,013][20347] Updated weights for policy 0, policy_version 1281614 (0.0010) [2023-12-27 14:31:43,273][20381] Updated weights for policy 1, policy_version 1280638 (0.0006) [2023-12-27 14:31:43,319][20381] Updated weights for policy 1, policy_version 1280648 (0.0005) [2023-12-27 14:31:43,364][20381] Updated weights for policy 1, policy_version 1280658 (0.0005) [2023-12-27 14:31:43,624][20347] Updated weights for policy 0, policy_version 1281624 (0.0007) [2023-12-27 14:31:43,673][20347] Updated weights for policy 0, policy_version 1281634 (0.0010) [2023-12-27 14:31:43,728][20347] Updated weights for policy 0, policy_version 1281644 (0.0010) [2023-12-27 14:31:43,928][20381] Updated weights for policy 1, policy_version 1280668 (0.0006) [2023-12-27 14:31:43,975][20381] Updated weights for policy 1, policy_version 1280678 (0.0007) [2023-12-27 14:31:44,025][20381] Updated weights for policy 1, policy_version 1280688 (0.0010) [2023-12-27 14:31:44,389][20347] Updated weights for policy 0, policy_version 1281654 (0.0007) [2023-12-27 14:31:44,442][20347] Updated weights for policy 0, policy_version 1281664 (0.0010) [2023-12-27 14:31:44,497][20347] Updated weights for policy 0, policy_version 1281674 (0.0010) [2023-12-27 14:31:44,720][20381] Updated weights for policy 1, policy_version 1280698 (0.0010) [2023-12-27 14:31:44,765][20381] Updated weights for policy 1, policy_version 1280708 (0.0007) [2023-12-27 14:31:44,818][20381] Updated weights for policy 1, policy_version 1280718 (0.0007) [2023-12-27 14:31:44,871][20381] Updated weights for policy 1, policy_version 1280728 (0.0007) [2023-12-27 14:31:45,203][20347] Updated weights for policy 0, policy_version 1281684 (0.0009) [2023-12-27 14:31:45,240][20291] KL-divergence is very high: 252.6998 [2023-12-27 14:31:45,255][20347] Updated weights for policy 0, policy_version 1281694 (0.0008) [2023-12-27 14:31:45,285][20291] KL-divergence is very high: 326.9386 [2023-12-27 14:31:45,313][20347] Updated weights for policy 0, policy_version 1281704 (0.0007) [2023-12-27 14:31:45,332][20291] KL-divergence is very high: 255.1844 [2023-12-27 14:31:45,598][20381] Updated weights for policy 1, policy_version 1280738 (0.0011) [2023-12-27 14:31:45,649][20381] Updated weights for policy 1, policy_version 1280748 (0.0011) [2023-12-27 14:31:45,702][20381] Updated weights for policy 1, policy_version 1280758 (0.0011) [2023-12-27 14:31:46,044][20347] Updated weights for policy 0, policy_version 1281714 (0.0007) [2023-12-27 14:31:46,099][20347] Updated weights for policy 0, policy_version 1281724 (0.0008) [2023-12-27 14:31:46,154][20347] Updated weights for policy 0, policy_version 1281734 (0.0006) [2023-12-27 14:31:46,210][20347] Updated weights for policy 0, policy_version 1281744 (0.0005) [2023-12-27 14:31:46,319][20381] Updated weights for policy 1, policy_version 1280768 (0.0011) [2023-12-27 14:31:46,371][20381] Updated weights for policy 1, policy_version 1280778 (0.0006) [2023-12-27 14:31:46,427][20381] Updated weights for policy 1, policy_version 1280788 (0.0005) [2023-12-27 14:31:46,898][20347] Updated weights for policy 0, policy_version 1281754 (0.0007) [2023-12-27 14:31:46,957][20347] Updated weights for policy 0, policy_version 1281764 (0.0008) [2023-12-27 14:31:47,006][20381] Updated weights for policy 1, policy_version 1280798 (0.0007) [2023-12-27 14:31:47,017][20347] Updated weights for policy 0, policy_version 1281774 (0.0007) [2023-12-27 14:31:47,058][20381] Updated weights for policy 1, policy_version 1280808 (0.0005) [2023-12-27 14:31:47,117][20381] Updated weights for policy 1, policy_version 1280818 (0.0005) [2023-12-27 14:31:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20410.6). Total num frames: 656113664. Throughput: 0: 10312.0, 1: 10221.6. Samples: 656122944. Policy #0 lag: (min: 12.0, avg: 13.4, max: 34.0) [2023-12-27 14:31:47,364][19170] Avg episode reward: [(0, '883.030'), (1, '723.010')] [2023-12-27 14:31:47,709][20381] Updated weights for policy 1, policy_version 1280828 (0.0005) [2023-12-27 14:31:47,745][20347] Updated weights for policy 0, policy_version 1281784 (0.0008) [2023-12-27 14:31:47,770][20381] Updated weights for policy 1, policy_version 1280838 (0.0006) [2023-12-27 14:31:47,794][20347] Updated weights for policy 0, policy_version 1281794 (0.0007) [2023-12-27 14:31:47,824][20381] Updated weights for policy 1, policy_version 1280848 (0.0010) [2023-12-27 14:31:47,849][20347] Updated weights for policy 0, policy_version 1281804 (0.0006) [2023-12-27 14:31:48,383][20381] Updated weights for policy 1, policy_version 1280858 (0.0007) [2023-12-27 14:31:48,438][20381] Updated weights for policy 1, policy_version 1280868 (0.0011) [2023-12-27 14:31:48,467][20347] Updated weights for policy 0, policy_version 1281814 (0.0010) [2023-12-27 14:31:48,490][20381] Updated weights for policy 1, policy_version 1280878 (0.0010) [2023-12-27 14:31:48,523][20347] Updated weights for policy 0, policy_version 1281824 (0.0008) [2023-12-27 14:31:48,543][20381] Updated weights for policy 1, policy_version 1280888 (0.0011) [2023-12-27 14:31:48,588][20347] Updated weights for policy 0, policy_version 1281834 (0.0008) [2023-12-27 14:31:49,241][20381] Updated weights for policy 1, policy_version 1280898 (0.0009) [2023-12-27 14:31:49,297][20381] Updated weights for policy 1, policy_version 1280908 (0.0007) [2023-12-27 14:31:49,338][20347] Updated weights for policy 0, policy_version 1281844 (0.0008) [2023-12-27 14:31:49,361][20381] Updated weights for policy 1, policy_version 1280918 (0.0008) [2023-12-27 14:31:49,406][20347] Updated weights for policy 0, policy_version 1281854 (0.0007) [2023-12-27 14:31:49,456][20347] Updated weights for policy 0, policy_version 1281864 (0.0007) [2023-12-27 14:31:50,045][20381] Updated weights for policy 1, policy_version 1280928 (0.0010) [2023-12-27 14:31:50,096][20381] Updated weights for policy 1, policy_version 1280938 (0.0009) [2023-12-27 14:31:50,145][20381] Updated weights for policy 1, policy_version 1280948 (0.0010) [2023-12-27 14:31:50,234][20347] Updated weights for policy 0, policy_version 1281874 (0.0008) [2023-12-27 14:31:50,282][20347] Updated weights for policy 0, policy_version 1281884 (0.0008) [2023-12-27 14:31:50,334][20347] Updated weights for policy 0, policy_version 1281894 (0.0008) [2023-12-27 14:31:50,390][20347] Updated weights for policy 0, policy_version 1281904 (0.0007) [2023-12-27 14:31:50,799][20381] Updated weights for policy 1, policy_version 1280958 (0.0009) [2023-12-27 14:31:50,858][20381] Updated weights for policy 1, policy_version 1280968 (0.0011) [2023-12-27 14:31:50,910][20381] Updated weights for policy 1, policy_version 1280978 (0.0010) [2023-12-27 14:31:51,053][20347] Updated weights for policy 0, policy_version 1281914 (0.0006) [2023-12-27 14:31:51,119][20347] Updated weights for policy 0, policy_version 1281924 (0.0007) [2023-12-27 14:31:51,181][20347] Updated weights for policy 0, policy_version 1281934 (0.0009) [2023-12-27 14:31:51,628][20381] Updated weights for policy 1, policy_version 1280988 (0.0010) [2023-12-27 14:31:51,689][20381] Updated weights for policy 1, policy_version 1280998 (0.0007) [2023-12-27 14:31:51,755][20381] Updated weights for policy 1, policy_version 1281008 (0.0006) [2023-12-27 14:31:51,856][20347] Updated weights for policy 0, policy_version 1281944 (0.0006) [2023-12-27 14:31:51,913][20347] Updated weights for policy 0, policy_version 1281954 (0.0008) [2023-12-27 14:31:51,962][20347] Updated weights for policy 0, policy_version 1281964 (0.0007) [2023-12-27 14:31:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 656211968. Throughput: 0: 10288.9, 1: 10221.9. Samples: 656185136. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:31:52,363][19170] Avg episode reward: [(0, '892.750'), (1, '720.120')] [2023-12-27 14:31:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001281968_328245248.pth... [2023-12-27 14:31:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001280784_327942144.pth [2023-12-27 14:31:52,379][20381] Updated weights for policy 1, policy_version 1281018 (0.0008) [2023-12-27 14:31:52,445][20381] Updated weights for policy 1, policy_version 1281028 (0.0009) [2023-12-27 14:31:52,508][20381] Updated weights for policy 1, policy_version 1281038 (0.0008) [2023-12-27 14:31:52,568][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001281048_327974912.pth... [2023-12-27 14:31:52,569][20381] Updated weights for policy 1, policy_version 1281048 (0.0006) [2023-12-27 14:31:52,572][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001279832_327663616.pth [2023-12-27 14:31:52,761][20347] Updated weights for policy 0, policy_version 1281974 (0.0008) [2023-12-27 14:31:52,816][20347] Updated weights for policy 0, policy_version 1281984 (0.0008) [2023-12-27 14:31:52,868][20347] Updated weights for policy 0, policy_version 1281994 (0.0008) [2023-12-27 14:31:53,135][20381] Updated weights for policy 1, policy_version 1281058 (0.0005) [2023-12-27 14:31:53,198][20381] Updated weights for policy 1, policy_version 1281068 (0.0005) [2023-12-27 14:31:53,255][20381] Updated weights for policy 1, policy_version 1281078 (0.0007) [2023-12-27 14:31:53,609][20347] Updated weights for policy 0, policy_version 1282004 (0.0008) [2023-12-27 14:31:53,665][20347] Updated weights for policy 0, policy_version 1282014 (0.0008) [2023-12-27 14:31:53,722][20347] Updated weights for policy 0, policy_version 1282024 (0.0007) [2023-12-27 14:31:53,839][20381] Updated weights for policy 1, policy_version 1281088 (0.0006) [2023-12-27 14:31:53,895][20381] Updated weights for policy 1, policy_version 1281098 (0.0005) [2023-12-27 14:31:53,958][20381] Updated weights for policy 1, policy_version 1281108 (0.0007) [2023-12-27 14:31:54,456][20347] Updated weights for policy 0, policy_version 1282034 (0.0006) [2023-12-27 14:31:54,500][20347] Updated weights for policy 0, policy_version 1282044 (0.0007) [2023-12-27 14:31:54,517][20381] Updated weights for policy 1, policy_version 1281118 (0.0007) [2023-12-27 14:31:54,548][20347] Updated weights for policy 0, policy_version 1282054 (0.0005) [2023-12-27 14:31:54,571][20381] Updated weights for policy 1, policy_version 1281128 (0.0008) [2023-12-27 14:31:54,601][20347] Updated weights for policy 0, policy_version 1282064 (0.0008) [2023-12-27 14:31:54,626][20381] Updated weights for policy 1, policy_version 1281138 (0.0005) [2023-12-27 14:31:55,245][20381] Updated weights for policy 1, policy_version 1281148 (0.0007) [2023-12-27 14:31:55,306][20381] Updated weights for policy 1, policy_version 1281158 (0.0006) [2023-12-27 14:31:55,348][20347] Updated weights for policy 0, policy_version 1282074 (0.0007) [2023-12-27 14:31:55,357][20381] Updated weights for policy 1, policy_version 1281168 (0.0005) [2023-12-27 14:31:55,409][20347] Updated weights for policy 0, policy_version 1282084 (0.0008) [2023-12-27 14:31:55,461][20347] Updated weights for policy 0, policy_version 1282094 (0.0008) [2023-12-27 14:31:55,950][20381] Updated weights for policy 1, policy_version 1281178 (0.0005) [2023-12-27 14:31:56,006][20381] Updated weights for policy 1, policy_version 1281188 (0.0010) [2023-12-27 14:31:56,065][20381] Updated weights for policy 1, policy_version 1281198 (0.0011) [2023-12-27 14:31:56,110][20381] Updated weights for policy 1, policy_version 1281208 (0.0010) [2023-12-27 14:31:56,172][20347] Updated weights for policy 0, policy_version 1282104 (0.0008) [2023-12-27 14:31:56,232][20347] Updated weights for policy 0, policy_version 1282114 (0.0007) [2023-12-27 14:31:56,286][20347] Updated weights for policy 0, policy_version 1282124 (0.0007) [2023-12-27 14:31:56,780][20381] Updated weights for policy 1, policy_version 1281218 (0.0010) [2023-12-27 14:31:56,828][20381] Updated weights for policy 1, policy_version 1281228 (0.0010) [2023-12-27 14:31:56,873][20381] Updated weights for policy 1, policy_version 1281238 (0.0010) [2023-12-27 14:31:56,977][20347] Updated weights for policy 0, policy_version 1282134 (0.0007) [2023-12-27 14:31:57,028][20347] Updated weights for policy 0, policy_version 1282144 (0.0007) [2023-12-27 14:31:57,085][20347] Updated weights for policy 0, policy_version 1282154 (0.0007) [2023-12-27 14:31:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 656318464. Throughput: 0: 10211.6, 1: 10363.4. Samples: 656309440. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:31:57,363][19170] Avg episode reward: [(0, '902.690'), (1, '798.110')] [2023-12-27 14:31:57,553][20381] Updated weights for policy 1, policy_version 1281248 (0.0009) [2023-12-27 14:31:57,604][20381] Updated weights for policy 1, policy_version 1281258 (0.0010) [2023-12-27 14:31:57,662][20381] Updated weights for policy 1, policy_version 1281268 (0.0008) [2023-12-27 14:31:57,701][20347] Updated weights for policy 0, policy_version 1282164 (0.0007) [2023-12-27 14:31:57,751][20347] Updated weights for policy 0, policy_version 1282174 (0.0005) [2023-12-27 14:31:57,798][20347] Updated weights for policy 0, policy_version 1282184 (0.0007) [2023-12-27 14:31:58,324][20381] Updated weights for policy 1, policy_version 1281278 (0.0009) [2023-12-27 14:31:58,389][20381] Updated weights for policy 1, policy_version 1281288 (0.0010) [2023-12-27 14:31:58,452][20381] Updated weights for policy 1, policy_version 1281298 (0.0008) [2023-12-27 14:31:58,569][20347] Updated weights for policy 0, policy_version 1282194 (0.0008) [2023-12-27 14:31:58,634][20347] Updated weights for policy 0, policy_version 1282204 (0.0008) [2023-12-27 14:31:58,691][20347] Updated weights for policy 0, policy_version 1282214 (0.0008) [2023-12-27 14:31:58,756][20347] Updated weights for policy 0, policy_version 1282224 (0.0008) [2023-12-27 14:31:59,235][20381] Updated weights for policy 1, policy_version 1281308 (0.0008) [2023-12-27 14:31:59,294][20381] Updated weights for policy 1, policy_version 1281318 (0.0008) [2023-12-27 14:31:59,357][20381] Updated weights for policy 1, policy_version 1281328 (0.0008) [2023-12-27 14:31:59,542][20347] Updated weights for policy 0, policy_version 1282234 (0.0008) [2023-12-27 14:31:59,596][20347] Updated weights for policy 0, policy_version 1282244 (0.0007) [2023-12-27 14:31:59,649][20347] Updated weights for policy 0, policy_version 1282254 (0.0008) [2023-12-27 14:31:59,996][20381] Updated weights for policy 1, policy_version 1281338 (0.0006) [2023-12-27 14:32:00,055][20381] Updated weights for policy 1, policy_version 1281348 (0.0008) [2023-12-27 14:32:00,107][20381] Updated weights for policy 1, policy_version 1281358 (0.0007) [2023-12-27 14:32:00,154][20381] Updated weights for policy 1, policy_version 1281368 (0.0009) [2023-12-27 14:32:00,337][20347] Updated weights for policy 0, policy_version 1282264 (0.0007) [2023-12-27 14:32:00,396][20347] Updated weights for policy 0, policy_version 1282274 (0.0005) [2023-12-27 14:32:00,446][20347] Updated weights for policy 0, policy_version 1282284 (0.0005) [2023-12-27 14:32:00,928][20381] Updated weights for policy 1, policy_version 1281378 (0.0009) [2023-12-27 14:32:00,962][20347] Updated weights for policy 0, policy_version 1282294 (0.0005) [2023-12-27 14:32:00,987][20381] Updated weights for policy 1, policy_version 1281388 (0.0007) [2023-12-27 14:32:01,008][20347] Updated weights for policy 0, policy_version 1282304 (0.0006) [2023-12-27 14:32:01,047][20381] Updated weights for policy 1, policy_version 1281398 (0.0007) [2023-12-27 14:32:01,065][20347] Updated weights for policy 0, policy_version 1282314 (0.0011) [2023-12-27 14:32:01,758][20347] Updated weights for policy 0, policy_version 1282324 (0.0009) [2023-12-27 14:32:01,791][20381] Updated weights for policy 1, policy_version 1281408 (0.0005) [2023-12-27 14:32:01,817][20347] Updated weights for policy 0, policy_version 1282334 (0.0008) [2023-12-27 14:32:01,853][20381] Updated weights for policy 1, policy_version 1281418 (0.0006) [2023-12-27 14:32:01,876][20347] Updated weights for policy 0, policy_version 1282344 (0.0007) [2023-12-27 14:32:01,911][20381] Updated weights for policy 1, policy_version 1281428 (0.0006) [2023-12-27 14:32:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 656416768. Throughput: 0: 10154.3, 1: 10299.0. Samples: 656429780. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:02,363][19170] Avg episode reward: [(0, '931.800'), (1, '747.740')] [2023-12-27 14:32:02,542][20347] Updated weights for policy 0, policy_version 1282354 (0.0010) [2023-12-27 14:32:02,571][20381] Updated weights for policy 1, policy_version 1281438 (0.0008) [2023-12-27 14:32:02,600][20347] Updated weights for policy 0, policy_version 1282364 (0.0010) [2023-12-27 14:32:02,626][20381] Updated weights for policy 1, policy_version 1281448 (0.0006) [2023-12-27 14:32:02,662][20347] Updated weights for policy 0, policy_version 1282374 (0.0010) [2023-12-27 14:32:02,680][20381] Updated weights for policy 1, policy_version 1281458 (0.0005) [2023-12-27 14:32:02,716][20347] Updated weights for policy 0, policy_version 1282384 (0.0010) [2023-12-27 14:32:03,273][20381] Updated weights for policy 1, policy_version 1281468 (0.0009) [2023-12-27 14:32:03,328][20381] Updated weights for policy 1, policy_version 1281478 (0.0006) [2023-12-27 14:32:03,378][20347] Updated weights for policy 0, policy_version 1282394 (0.0010) [2023-12-27 14:32:03,380][20381] Updated weights for policy 1, policy_version 1281488 (0.0005) [2023-12-27 14:32:03,429][20347] Updated weights for policy 0, policy_version 1282404 (0.0010) [2023-12-27 14:32:03,473][20347] Updated weights for policy 0, policy_version 1282414 (0.0010) [2023-12-27 14:32:03,941][20381] Updated weights for policy 1, policy_version 1281498 (0.0005) [2023-12-27 14:32:03,998][20381] Updated weights for policy 1, policy_version 1281508 (0.0006) [2023-12-27 14:32:04,052][20381] Updated weights for policy 1, policy_version 1281518 (0.0007) [2023-12-27 14:32:04,111][20381] Updated weights for policy 1, policy_version 1281528 (0.0006) [2023-12-27 14:32:04,201][20347] Updated weights for policy 0, policy_version 1282424 (0.0008) [2023-12-27 14:32:04,261][20347] Updated weights for policy 0, policy_version 1282434 (0.0006) [2023-12-27 14:32:04,318][20347] Updated weights for policy 0, policy_version 1282444 (0.0006) [2023-12-27 14:32:04,761][20381] Updated weights for policy 1, policy_version 1281538 (0.0008) [2023-12-27 14:32:04,817][20381] Updated weights for policy 1, policy_version 1281548 (0.0007) [2023-12-27 14:32:04,875][20381] Updated weights for policy 1, policy_version 1281558 (0.0008) [2023-12-27 14:32:04,932][20347] Updated weights for policy 0, policy_version 1282454 (0.0008) [2023-12-27 14:32:04,995][20347] Updated weights for policy 0, policy_version 1282464 (0.0009) [2023-12-27 14:32:05,055][20347] Updated weights for policy 0, policy_version 1282474 (0.0008) [2023-12-27 14:32:05,479][20381] Updated weights for policy 1, policy_version 1281568 (0.0008) [2023-12-27 14:32:05,532][20381] Updated weights for policy 1, policy_version 1281578 (0.0006) [2023-12-27 14:32:05,594][20381] Updated weights for policy 1, policy_version 1281588 (0.0005) [2023-12-27 14:32:05,737][20347] Updated weights for policy 0, policy_version 1282484 (0.0008) [2023-12-27 14:32:05,791][20347] Updated weights for policy 0, policy_version 1282494 (0.0007) [2023-12-27 14:32:05,849][20347] Updated weights for policy 0, policy_version 1282504 (0.0008) [2023-12-27 14:32:06,254][20381] Updated weights for policy 1, policy_version 1281598 (0.0006) [2023-12-27 14:32:06,315][20381] Updated weights for policy 1, policy_version 1281608 (0.0006) [2023-12-27 14:32:06,376][20381] Updated weights for policy 1, policy_version 1281618 (0.0008) [2023-12-27 14:32:06,542][20347] Updated weights for policy 0, policy_version 1282514 (0.0008) [2023-12-27 14:32:06,600][20347] Updated weights for policy 0, policy_version 1282524 (0.0008) [2023-12-27 14:32:06,654][20347] Updated weights for policy 0, policy_version 1282534 (0.0008) [2023-12-27 14:32:06,710][20347] Updated weights for policy 0, policy_version 1282544 (0.0008) [2023-12-27 14:32:06,961][20381] Updated weights for policy 1, policy_version 1281628 (0.0006) [2023-12-27 14:32:07,028][20381] Updated weights for policy 1, policy_version 1281638 (0.0008) [2023-12-27 14:32:07,087][20381] Updated weights for policy 1, policy_version 1281648 (0.0008) [2023-12-27 14:32:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 656523264. Throughput: 0: 10106.2, 1: 10402.7. Samples: 656494040. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:07,363][19170] Avg episode reward: [(0, '892.790'), (1, '659.590')] [2023-12-27 14:32:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001281656_328130560.pth... [2023-12-27 14:32:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001280440_327819264.pth [2023-12-27 14:32:07,394][20347] Updated weights for policy 0, policy_version 1282554 (0.0007) [2023-12-27 14:32:07,450][20347] Updated weights for policy 0, policy_version 1282564 (0.0006) [2023-12-27 14:32:07,507][20347] Updated weights for policy 0, policy_version 1282574 (0.0006) [2023-12-27 14:32:07,518][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001282576_328400896.pth... [2023-12-27 14:32:07,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001281392_328097792.pth [2023-12-27 14:32:07,711][20381] Updated weights for policy 1, policy_version 1281658 (0.0008) [2023-12-27 14:32:07,775][20381] Updated weights for policy 1, policy_version 1281668 (0.0006) [2023-12-27 14:32:07,836][20381] Updated weights for policy 1, policy_version 1281678 (0.0006) [2023-12-27 14:32:07,896][20381] Updated weights for policy 1, policy_version 1281688 (0.0007) [2023-12-27 14:32:08,185][20347] Updated weights for policy 0, policy_version 1282584 (0.0008) [2023-12-27 14:32:08,233][20347] Updated weights for policy 0, policy_version 1282594 (0.0010) [2023-12-27 14:32:08,285][20347] Updated weights for policy 0, policy_version 1282604 (0.0010) [2023-12-27 14:32:08,458][20381] Updated weights for policy 1, policy_version 1281698 (0.0007) [2023-12-27 14:32:08,519][20381] Updated weights for policy 1, policy_version 1281708 (0.0006) [2023-12-27 14:32:08,581][20381] Updated weights for policy 1, policy_version 1281718 (0.0008) [2023-12-27 14:32:08,993][20347] Updated weights for policy 0, policy_version 1282614 (0.0009) [2023-12-27 14:32:09,048][20347] Updated weights for policy 0, policy_version 1282624 (0.0010) [2023-12-27 14:32:09,097][20347] Updated weights for policy 0, policy_version 1282634 (0.0010) [2023-12-27 14:32:09,188][20381] Updated weights for policy 1, policy_version 1281728 (0.0008) [2023-12-27 14:32:09,247][20381] Updated weights for policy 1, policy_version 1281738 (0.0008) [2023-12-27 14:32:09,295][20381] Updated weights for policy 1, policy_version 1281748 (0.0008) [2023-12-27 14:32:09,818][20347] Updated weights for policy 0, policy_version 1282644 (0.0009) [2023-12-27 14:32:09,885][20347] Updated weights for policy 0, policy_version 1282654 (0.0008) [2023-12-27 14:32:09,944][20347] Updated weights for policy 0, policy_version 1282664 (0.0010) [2023-12-27 14:32:10,060][20381] Updated weights for policy 1, policy_version 1281758 (0.0006) [2023-12-27 14:32:10,115][20381] Updated weights for policy 1, policy_version 1281768 (0.0007) [2023-12-27 14:32:10,175][20381] Updated weights for policy 1, policy_version 1281778 (0.0008) [2023-12-27 14:32:10,598][20347] Updated weights for policy 0, policy_version 1282674 (0.0010) [2023-12-27 14:32:10,655][20347] Updated weights for policy 0, policy_version 1282684 (0.0010) [2023-12-27 14:32:10,709][20347] Updated weights for policy 0, policy_version 1282694 (0.0010) [2023-12-27 14:32:10,768][20347] Updated weights for policy 0, policy_version 1282704 (0.0010) [2023-12-27 14:32:10,844][20381] Updated weights for policy 1, policy_version 1281788 (0.0007) [2023-12-27 14:32:10,892][20381] Updated weights for policy 1, policy_version 1281798 (0.0005) [2023-12-27 14:32:10,945][20381] Updated weights for policy 1, policy_version 1281808 (0.0005) [2023-12-27 14:32:11,514][20347] Updated weights for policy 0, policy_version 1282714 (0.0008) [2023-12-27 14:32:11,572][20347] Updated weights for policy 0, policy_version 1282724 (0.0008) [2023-12-27 14:32:11,632][20347] Updated weights for policy 0, policy_version 1282734 (0.0008) [2023-12-27 14:32:11,688][20381] Updated weights for policy 1, policy_version 1281818 (0.0006) [2023-12-27 14:32:11,754][20381] Updated weights for policy 1, policy_version 1281828 (0.0008) [2023-12-27 14:32:11,810][20381] Updated weights for policy 1, policy_version 1281838 (0.0007) [2023-12-27 14:32:11,860][20381] Updated weights for policy 1, policy_version 1281848 (0.0007) [2023-12-27 14:32:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 656621568. Throughput: 0: 10081.5, 1: 10473.4. Samples: 656617516. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:12,363][19170] Avg episode reward: [(0, '892.610'), (1, '736.500')] [2023-12-27 14:32:12,382][20347] Updated weights for policy 0, policy_version 1282744 (0.0008) [2023-12-27 14:32:12,445][20347] Updated weights for policy 0, policy_version 1282754 (0.0008) [2023-12-27 14:32:12,513][20347] Updated weights for policy 0, policy_version 1282764 (0.0008) [2023-12-27 14:32:12,572][20381] Updated weights for policy 1, policy_version 1281858 (0.0007) [2023-12-27 14:32:12,628][20381] Updated weights for policy 1, policy_version 1281868 (0.0008) [2023-12-27 14:32:12,683][20381] Updated weights for policy 1, policy_version 1281878 (0.0007) [2023-12-27 14:32:13,176][20347] Updated weights for policy 0, policy_version 1282774 (0.0006) [2023-12-27 14:32:13,236][20347] Updated weights for policy 0, policy_version 1282784 (0.0008) [2023-12-27 14:32:13,292][20347] Updated weights for policy 0, policy_version 1282794 (0.0006) [2023-12-27 14:32:13,349][20381] Updated weights for policy 1, policy_version 1281888 (0.0008) [2023-12-27 14:32:13,395][20381] Updated weights for policy 1, policy_version 1281898 (0.0009) [2023-12-27 14:32:13,447][20381] Updated weights for policy 1, policy_version 1281908 (0.0007) [2023-12-27 14:32:13,853][20347] Updated weights for policy 0, policy_version 1282804 (0.0006) [2023-12-27 14:32:13,925][20347] Updated weights for policy 0, policy_version 1282814 (0.0008) [2023-12-27 14:32:13,984][20347] Updated weights for policy 0, policy_version 1282824 (0.0007) [2023-12-27 14:32:14,126][20381] Updated weights for policy 1, policy_version 1281918 (0.0006) [2023-12-27 14:32:14,173][20381] Updated weights for policy 1, policy_version 1281928 (0.0007) [2023-12-27 14:32:14,224][20381] Updated weights for policy 1, policy_version 1281938 (0.0008) [2023-12-27 14:32:14,637][20347] Updated weights for policy 0, policy_version 1282834 (0.0007) [2023-12-27 14:32:14,691][20347] Updated weights for policy 0, policy_version 1282844 (0.0008) [2023-12-27 14:32:14,737][20347] Updated weights for policy 0, policy_version 1282854 (0.0007) [2023-12-27 14:32:14,800][20347] Updated weights for policy 0, policy_version 1282864 (0.0008) [2023-12-27 14:32:14,944][20381] Updated weights for policy 1, policy_version 1281948 (0.0008) [2023-12-27 14:32:15,005][20381] Updated weights for policy 1, policy_version 1281958 (0.0008) [2023-12-27 14:32:15,069][20381] Updated weights for policy 1, policy_version 1281968 (0.0008) [2023-12-27 14:32:15,500][20347] Updated weights for policy 0, policy_version 1282874 (0.0006) [2023-12-27 14:32:15,560][20347] Updated weights for policy 0, policy_version 1282884 (0.0007) [2023-12-27 14:32:15,615][20347] Updated weights for policy 0, policy_version 1282894 (0.0007) [2023-12-27 14:32:15,761][20381] Updated weights for policy 1, policy_version 1281978 (0.0008) [2023-12-27 14:32:15,814][20381] Updated weights for policy 1, policy_version 1281988 (0.0008) [2023-12-27 14:32:15,873][20381] Updated weights for policy 1, policy_version 1281998 (0.0008) [2023-12-27 14:32:15,926][20381] Updated weights for policy 1, policy_version 1282008 (0.0008) [2023-12-27 14:32:16,290][20347] Updated weights for policy 0, policy_version 1282904 (0.0008) [2023-12-27 14:32:16,336][20347] Updated weights for policy 0, policy_version 1282914 (0.0007) [2023-12-27 14:32:16,393][20347] Updated weights for policy 0, policy_version 1282924 (0.0008) [2023-12-27 14:32:16,599][20381] Updated weights for policy 1, policy_version 1282018 (0.0008) [2023-12-27 14:32:16,655][20381] Updated weights for policy 1, policy_version 1282028 (0.0008) [2023-12-27 14:32:16,708][20381] Updated weights for policy 1, policy_version 1282038 (0.0008) [2023-12-27 14:32:17,091][20347] Updated weights for policy 0, policy_version 1282934 (0.0008) [2023-12-27 14:32:17,156][20347] Updated weights for policy 0, policy_version 1282944 (0.0008) [2023-12-27 14:32:17,207][20347] Updated weights for policy 0, policy_version 1282954 (0.0007) [2023-12-27 14:32:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 656728064. Throughput: 0: 10060.6, 1: 10424.2. Samples: 656738928. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:17,363][19170] Avg episode reward: [(0, '941.280'), (1, '768.320')] [2023-12-27 14:32:17,403][20381] Updated weights for policy 1, policy_version 1282048 (0.0008) [2023-12-27 14:32:17,458][20381] Updated weights for policy 1, policy_version 1282058 (0.0008) [2023-12-27 14:32:17,511][20381] Updated weights for policy 1, policy_version 1282068 (0.0008) [2023-12-27 14:32:17,875][20347] Updated weights for policy 0, policy_version 1282964 (0.0008) [2023-12-27 14:32:17,935][20347] Updated weights for policy 0, policy_version 1282974 (0.0007) [2023-12-27 14:32:17,991][20347] Updated weights for policy 0, policy_version 1282984 (0.0008) [2023-12-27 14:32:18,211][20381] Updated weights for policy 1, policy_version 1282078 (0.0008) [2023-12-27 14:32:18,267][20381] Updated weights for policy 1, policy_version 1282088 (0.0007) [2023-12-27 14:32:18,340][20381] Updated weights for policy 1, policy_version 1282098 (0.0007) [2023-12-27 14:32:18,705][20347] Updated weights for policy 0, policy_version 1282994 (0.0008) [2023-12-27 14:32:18,761][20347] Updated weights for policy 0, policy_version 1283004 (0.0008) [2023-12-27 14:32:18,819][20347] Updated weights for policy 0, policy_version 1283014 (0.0008) [2023-12-27 14:32:18,881][20347] Updated weights for policy 0, policy_version 1283024 (0.0008) [2023-12-27 14:32:19,029][20381] Updated weights for policy 1, policy_version 1282108 (0.0008) [2023-12-27 14:32:19,084][20381] Updated weights for policy 1, policy_version 1282118 (0.0007) [2023-12-27 14:32:19,135][20381] Updated weights for policy 1, policy_version 1282128 (0.0007) [2023-12-27 14:32:19,583][20347] Updated weights for policy 0, policy_version 1283034 (0.0007) [2023-12-27 14:32:19,642][20347] Updated weights for policy 0, policy_version 1283044 (0.0008) [2023-12-27 14:32:19,702][20347] Updated weights for policy 0, policy_version 1283054 (0.0008) [2023-12-27 14:32:19,857][20381] Updated weights for policy 1, policy_version 1282138 (0.0007) [2023-12-27 14:32:19,911][20381] Updated weights for policy 1, policy_version 1282148 (0.0008) [2023-12-27 14:32:19,972][20381] Updated weights for policy 1, policy_version 1282158 (0.0008) [2023-12-27 14:32:20,032][20381] Updated weights for policy 1, policy_version 1282168 (0.0007) [2023-12-27 14:32:20,410][20347] Updated weights for policy 0, policy_version 1283064 (0.0007) [2023-12-27 14:32:20,472][20347] Updated weights for policy 0, policy_version 1283074 (0.0008) [2023-12-27 14:32:20,531][20347] Updated weights for policy 0, policy_version 1283084 (0.0008) [2023-12-27 14:32:20,763][20381] Updated weights for policy 1, policy_version 1282178 (0.0007) [2023-12-27 14:32:20,822][20381] Updated weights for policy 1, policy_version 1282188 (0.0008) [2023-12-27 14:32:20,876][20381] Updated weights for policy 1, policy_version 1282198 (0.0008) [2023-12-27 14:32:21,238][20347] Updated weights for policy 0, policy_version 1283094 (0.0008) [2023-12-27 14:32:21,301][20347] Updated weights for policy 0, policy_version 1283104 (0.0008) [2023-12-27 14:32:21,366][20347] Updated weights for policy 0, policy_version 1283114 (0.0008) [2023-12-27 14:32:21,590][20381] Updated weights for policy 1, policy_version 1282208 (0.0006) [2023-12-27 14:32:21,656][20381] Updated weights for policy 1, policy_version 1282218 (0.0008) [2023-12-27 14:32:21,726][20381] Updated weights for policy 1, policy_version 1282228 (0.0008) [2023-12-27 14:32:22,068][20347] Updated weights for policy 0, policy_version 1283124 (0.0007) [2023-12-27 14:32:22,131][20347] Updated weights for policy 0, policy_version 1283134 (0.0007) [2023-12-27 14:32:22,197][20347] Updated weights for policy 0, policy_version 1283144 (0.0007) [2023-12-27 14:32:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 656826368. Throughput: 0: 10007.4, 1: 10367.7. Samples: 656797960. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:22,363][19170] Avg episode reward: [(0, '931.530'), (1, '825.530')] [2023-12-27 14:32:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001283152_328548352.pth... [2023-12-27 14:32:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001282232_328278016.pth... [2023-12-27 14:32:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001281968_328245248.pth [2023-12-27 14:32:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001281048_327974912.pth [2023-12-27 14:32:22,496][20381] Updated weights for policy 1, policy_version 1282238 (0.0008) [2023-12-27 14:32:22,552][20381] Updated weights for policy 1, policy_version 1282248 (0.0008) [2023-12-27 14:32:22,611][20381] Updated weights for policy 1, policy_version 1282258 (0.0008) [2023-12-27 14:32:22,888][20347] Updated weights for policy 0, policy_version 1283154 (0.0007) [2023-12-27 14:32:22,946][20347] Updated weights for policy 0, policy_version 1283164 (0.0007) [2023-12-27 14:32:23,006][20347] Updated weights for policy 0, policy_version 1283174 (0.0007) [2023-12-27 14:32:23,060][20347] Updated weights for policy 0, policy_version 1283184 (0.0007) [2023-12-27 14:32:23,409][20381] Updated weights for policy 1, policy_version 1282268 (0.0008) [2023-12-27 14:32:23,460][20381] Updated weights for policy 1, policy_version 1282278 (0.0007) [2023-12-27 14:32:23,515][20381] Updated weights for policy 1, policy_version 1282288 (0.0007) [2023-12-27 14:32:23,690][20347] Updated weights for policy 0, policy_version 1283194 (0.0007) [2023-12-27 14:32:23,742][20347] Updated weights for policy 0, policy_version 1283204 (0.0008) [2023-12-27 14:32:23,807][20347] Updated weights for policy 0, policy_version 1283214 (0.0008) [2023-12-27 14:32:24,214][20381] Updated weights for policy 1, policy_version 1282298 (0.0008) [2023-12-27 14:32:24,268][20381] Updated weights for policy 1, policy_version 1282308 (0.0008) [2023-12-27 14:32:24,327][20381] Updated weights for policy 1, policy_version 1282318 (0.0008) [2023-12-27 14:32:24,384][20381] Updated weights for policy 1, policy_version 1282328 (0.0008) [2023-12-27 14:32:24,491][20347] Updated weights for policy 0, policy_version 1283224 (0.0007) [2023-12-27 14:32:24,540][20347] Updated weights for policy 0, policy_version 1283234 (0.0008) [2023-12-27 14:32:24,585][20347] Updated weights for policy 0, policy_version 1283244 (0.0007) [2023-12-27 14:32:25,086][20381] Updated weights for policy 1, policy_version 1282338 (0.0007) [2023-12-27 14:32:25,137][20381] Updated weights for policy 1, policy_version 1282348 (0.0008) [2023-12-27 14:32:25,189][20381] Updated weights for policy 1, policy_version 1282358 (0.0007) [2023-12-27 14:32:25,263][20347] Updated weights for policy 0, policy_version 1283254 (0.0008) [2023-12-27 14:32:25,311][20347] Updated weights for policy 0, policy_version 1283264 (0.0008) [2023-12-27 14:32:25,363][20347] Updated weights for policy 0, policy_version 1283274 (0.0008) [2023-12-27 14:32:25,890][20381] Updated weights for policy 1, policy_version 1282368 (0.0008) [2023-12-27 14:32:25,943][20381] Updated weights for policy 1, policy_version 1282378 (0.0006) [2023-12-27 14:32:26,005][20381] Updated weights for policy 1, policy_version 1282388 (0.0007) [2023-12-27 14:32:26,066][20347] Updated weights for policy 0, policy_version 1283284 (0.0007) [2023-12-27 14:32:26,119][20347] Updated weights for policy 0, policy_version 1283294 (0.0008) [2023-12-27 14:32:26,167][20347] Updated weights for policy 0, policy_version 1283304 (0.0008) [2023-12-27 14:32:26,684][20381] Updated weights for policy 1, policy_version 1282398 (0.0008) [2023-12-27 14:32:26,742][20381] Updated weights for policy 1, policy_version 1282408 (0.0007) [2023-12-27 14:32:26,797][20381] Updated weights for policy 1, policy_version 1282418 (0.0007) [2023-12-27 14:32:26,830][20347] Updated weights for policy 0, policy_version 1283314 (0.0007) [2023-12-27 14:32:26,884][20347] Updated weights for policy 0, policy_version 1283324 (0.0008) [2023-12-27 14:32:26,934][20347] Updated weights for policy 0, policy_version 1283334 (0.0007) [2023-12-27 14:32:26,982][20347] Updated weights for policy 0, policy_version 1283344 (0.0007) [2023-12-27 14:32:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20355.0). Total num frames: 656924672. Throughput: 0: 10010.9, 1: 10339.8. Samples: 656916404. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:27,363][19170] Avg episode reward: [(0, '892.640'), (1, '786.900')] [2023-12-27 14:32:27,476][20381] Updated weights for policy 1, policy_version 1282428 (0.0007) [2023-12-27 14:32:27,522][20381] Updated weights for policy 1, policy_version 1282438 (0.0007) [2023-12-27 14:32:27,572][20381] Updated weights for policy 1, policy_version 1282448 (0.0007) [2023-12-27 14:32:27,605][20347] Updated weights for policy 0, policy_version 1283354 (0.0009) [2023-12-27 14:32:27,649][20347] Updated weights for policy 0, policy_version 1283364 (0.0007) [2023-12-27 14:32:27,703][20347] Updated weights for policy 0, policy_version 1283374 (0.0007) [2023-12-27 14:32:28,274][20381] Updated weights for policy 1, policy_version 1282458 (0.0007) [2023-12-27 14:32:28,336][20381] Updated weights for policy 1, policy_version 1282468 (0.0008) [2023-12-27 14:32:28,392][20381] Updated weights for policy 1, policy_version 1282478 (0.0008) [2023-12-27 14:32:28,426][20347] Updated weights for policy 0, policy_version 1283384 (0.0009) [2023-12-27 14:32:28,450][20381] Updated weights for policy 1, policy_version 1282488 (0.0009) [2023-12-27 14:32:28,483][20347] Updated weights for policy 0, policy_version 1283394 (0.0007) [2023-12-27 14:32:28,547][20347] Updated weights for policy 0, policy_version 1283404 (0.0008) [2023-12-27 14:32:29,102][20381] Updated weights for policy 1, policy_version 1282498 (0.0007) [2023-12-27 14:32:29,155][20381] Updated weights for policy 1, policy_version 1282508 (0.0007) [2023-12-27 14:32:29,210][20381] Updated weights for policy 1, policy_version 1282518 (0.0007) [2023-12-27 14:32:29,229][20347] Updated weights for policy 0, policy_version 1283414 (0.0009) [2023-12-27 14:32:29,292][20347] Updated weights for policy 0, policy_version 1283424 (0.0007) [2023-12-27 14:32:29,356][20347] Updated weights for policy 0, policy_version 1283434 (0.0008) [2023-12-27 14:32:29,945][20381] Updated weights for policy 1, policy_version 1282528 (0.0008) [2023-12-27 14:32:29,997][20381] Updated weights for policy 1, policy_version 1282538 (0.0008) [2023-12-27 14:32:30,038][20347] Updated weights for policy 0, policy_version 1283444 (0.0007) [2023-12-27 14:32:30,052][20381] Updated weights for policy 1, policy_version 1282548 (0.0007) [2023-12-27 14:32:30,093][20347] Updated weights for policy 0, policy_version 1283454 (0.0006) [2023-12-27 14:32:30,142][20347] Updated weights for policy 0, policy_version 1283464 (0.0008) [2023-12-27 14:32:30,714][20381] Updated weights for policy 1, policy_version 1282558 (0.0006) [2023-12-27 14:32:30,769][20381] Updated weights for policy 1, policy_version 1282568 (0.0006) [2023-12-27 14:32:30,828][20381] Updated weights for policy 1, policy_version 1282578 (0.0007) [2023-12-27 14:32:30,838][20347] Updated weights for policy 0, policy_version 1283474 (0.0007) [2023-12-27 14:32:30,891][20347] Updated weights for policy 0, policy_version 1283484 (0.0006) [2023-12-27 14:32:30,943][20347] Updated weights for policy 0, policy_version 1283494 (0.0008) [2023-12-27 14:32:30,994][20347] Updated weights for policy 0, policy_version 1283504 (0.0008) [2023-12-27 14:32:31,519][20381] Updated weights for policy 1, policy_version 1282588 (0.0006) [2023-12-27 14:32:31,579][20381] Updated weights for policy 1, policy_version 1282598 (0.0006) [2023-12-27 14:32:31,641][20381] Updated weights for policy 1, policy_version 1282608 (0.0007) [2023-12-27 14:32:31,742][20347] Updated weights for policy 0, policy_version 1283514 (0.0007) [2023-12-27 14:32:31,810][20347] Updated weights for policy 0, policy_version 1283524 (0.0008) [2023-12-27 14:32:31,869][20347] Updated weights for policy 0, policy_version 1283534 (0.0008) [2023-12-27 14:32:32,328][20381] Updated weights for policy 1, policy_version 1282618 (0.0008) [2023-12-27 14:32:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20327.3). Total num frames: 657022976. Throughput: 0: 10037.9, 1: 10287.7. Samples: 657037592. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:32,363][19170] Avg episode reward: [(0, '902.460'), (1, '843.950')] [2023-12-27 14:32:32,387][20381] Updated weights for policy 1, policy_version 1282628 (0.0008) [2023-12-27 14:32:32,446][20381] Updated weights for policy 1, policy_version 1282638 (0.0008) [2023-12-27 14:32:32,505][20381] Updated weights for policy 1, policy_version 1282648 (0.0008) [2023-12-27 14:32:32,598][20347] Updated weights for policy 0, policy_version 1283544 (0.0007) [2023-12-27 14:32:32,656][20347] Updated weights for policy 0, policy_version 1283554 (0.0008) [2023-12-27 14:32:32,717][20347] Updated weights for policy 0, policy_version 1283564 (0.0007) [2023-12-27 14:32:33,168][20381] Updated weights for policy 1, policy_version 1282658 (0.0005) [2023-12-27 14:32:33,233][20381] Updated weights for policy 1, policy_version 1282668 (0.0005) [2023-12-27 14:32:33,297][20381] Updated weights for policy 1, policy_version 1282678 (0.0005) [2023-12-27 14:32:33,297][20347] Updated weights for policy 0, policy_version 1283574 (0.0007) [2023-12-27 14:32:33,346][20347] Updated weights for policy 0, policy_version 1283584 (0.0005) [2023-12-27 14:32:33,402][20347] Updated weights for policy 0, policy_version 1283594 (0.0005) [2023-12-27 14:32:33,842][20381] Updated weights for policy 1, policy_version 1282688 (0.0006) [2023-12-27 14:32:33,895][20381] Updated weights for policy 1, policy_version 1282698 (0.0005) [2023-12-27 14:32:33,944][20381] Updated weights for policy 1, policy_version 1282708 (0.0005) [2023-12-27 14:32:33,962][20347] Updated weights for policy 0, policy_version 1283604 (0.0005) [2023-12-27 14:32:34,010][20347] Updated weights for policy 0, policy_version 1283614 (0.0005) [2023-12-27 14:32:34,068][20347] Updated weights for policy 0, policy_version 1283624 (0.0005) [2023-12-27 14:32:34,597][20381] Updated weights for policy 1, policy_version 1282718 (0.0008) [2023-12-27 14:32:34,646][20347] Updated weights for policy 0, policy_version 1283634 (0.0005) [2023-12-27 14:32:34,648][20291] KL-divergence is very high: 120.6163 [2023-12-27 14:32:34,656][20381] Updated weights for policy 1, policy_version 1282728 (0.0011) [2023-12-27 14:32:34,709][20347] Updated weights for policy 0, policy_version 1283644 (0.0007) [2023-12-27 14:32:34,718][20381] Updated weights for policy 1, policy_version 1282738 (0.0011) [2023-12-27 14:32:34,769][20347] Updated weights for policy 0, policy_version 1283654 (0.0006) [2023-12-27 14:32:34,822][20347] Updated weights for policy 0, policy_version 1283664 (0.0007) [2023-12-27 14:32:35,385][20381] Updated weights for policy 1, policy_version 1282748 (0.0011) [2023-12-27 14:32:35,390][20347] Updated weights for policy 0, policy_version 1283674 (0.0005) [2023-12-27 14:32:35,440][20347] Updated weights for policy 0, policy_version 1283684 (0.0005) [2023-12-27 14:32:35,441][20381] Updated weights for policy 1, policy_version 1282758 (0.0009) [2023-12-27 14:32:35,487][20347] Updated weights for policy 0, policy_version 1283694 (0.0005) [2023-12-27 14:32:35,492][20381] Updated weights for policy 1, policy_version 1282768 (0.0006) [2023-12-27 14:32:36,016][20347] Updated weights for policy 0, policy_version 1283704 (0.0007) [2023-12-27 14:32:36,074][20347] Updated weights for policy 0, policy_version 1283714 (0.0005) [2023-12-27 14:32:36,142][20347] Updated weights for policy 0, policy_version 1283724 (0.0007) [2023-12-27 14:32:36,173][20381] Updated weights for policy 1, policy_version 1282778 (0.0010) [2023-12-27 14:32:36,234][20381] Updated weights for policy 1, policy_version 1282788 (0.0011) [2023-12-27 14:32:36,291][20381] Updated weights for policy 1, policy_version 1282798 (0.0011) [2023-12-27 14:32:36,351][20381] Updated weights for policy 1, policy_version 1282808 (0.0011) [2023-12-27 14:32:36,805][20347] Updated weights for policy 0, policy_version 1283734 (0.0006) [2023-12-27 14:32:36,870][20347] Updated weights for policy 0, policy_version 1283744 (0.0007) [2023-12-27 14:32:36,929][20347] Updated weights for policy 0, policy_version 1283754 (0.0008) [2023-12-27 14:32:37,046][20381] Updated weights for policy 1, policy_version 1282818 (0.0009) [2023-12-27 14:32:37,108][20381] Updated weights for policy 1, policy_version 1282828 (0.0008) [2023-12-27 14:32:37,165][20381] Updated weights for policy 1, policy_version 1282838 (0.0005) [2023-12-27 14:32:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 657137664. Throughput: 0: 10134.6, 1: 10291.9. Samples: 657104332. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:37,364][19170] Avg episode reward: [(0, '872.980'), (1, '872.400')] [2023-12-27 14:32:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001282840_328433664.pth... [2023-12-27 14:32:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001283760_328704000.pth... [2023-12-27 14:32:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001281656_328130560.pth [2023-12-27 14:32:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001282576_328400896.pth [2023-12-27 14:32:37,652][20347] Updated weights for policy 0, policy_version 1283764 (0.0008) [2023-12-27 14:32:37,702][20347] Updated weights for policy 0, policy_version 1283774 (0.0006) [2023-12-27 14:32:37,751][20347] Updated weights for policy 0, policy_version 1283784 (0.0006) [2023-12-27 14:32:37,755][20381] Updated weights for policy 1, policy_version 1282848 (0.0008) [2023-12-27 14:32:37,803][20381] Updated weights for policy 1, policy_version 1282858 (0.0008) [2023-12-27 14:32:37,847][20381] Updated weights for policy 1, policy_version 1282868 (0.0008) [2023-12-27 14:32:38,419][20347] Updated weights for policy 0, policy_version 1283794 (0.0006) [2023-12-27 14:32:38,472][20347] Updated weights for policy 0, policy_version 1283804 (0.0008) [2023-12-27 14:32:38,523][20347] Updated weights for policy 0, policy_version 1283814 (0.0007) [2023-12-27 14:32:38,544][20381] Updated weights for policy 1, policy_version 1282878 (0.0007) [2023-12-27 14:32:38,583][20347] Updated weights for policy 0, policy_version 1283824 (0.0005) [2023-12-27 14:32:38,600][20381] Updated weights for policy 1, policy_version 1282888 (0.0006) [2023-12-27 14:32:38,659][20381] Updated weights for policy 1, policy_version 1282898 (0.0008) [2023-12-27 14:32:39,286][20347] Updated weights for policy 0, policy_version 1283834 (0.0007) [2023-12-27 14:32:39,288][20381] Updated weights for policy 1, policy_version 1282908 (0.0007) [2023-12-27 14:32:39,342][20347] Updated weights for policy 0, policy_version 1283844 (0.0006) [2023-12-27 14:32:39,345][20381] Updated weights for policy 1, policy_version 1282918 (0.0006) [2023-12-27 14:32:39,410][20347] Updated weights for policy 0, policy_version 1283854 (0.0008) [2023-12-27 14:32:39,415][20381] Updated weights for policy 1, policy_version 1282928 (0.0008) [2023-12-27 14:32:40,114][20347] Updated weights for policy 0, policy_version 1283864 (0.0007) [2023-12-27 14:32:40,116][20291] KL-divergence is very high: 176.9429 [2023-12-27 14:32:40,121][20381] Updated weights for policy 1, policy_version 1282938 (0.0008) [2023-12-27 14:32:40,162][20291] KL-divergence is very high: 219.7786 [2023-12-27 14:32:40,173][20347] Updated weights for policy 0, policy_version 1283874 (0.0008) [2023-12-27 14:32:40,190][20381] Updated weights for policy 1, policy_version 1282948 (0.0007) [2023-12-27 14:32:40,208][20291] KL-divergence is very high: 148.5297 [2023-12-27 14:32:40,232][20347] Updated weights for policy 0, policy_version 1283884 (0.0007) [2023-12-27 14:32:40,256][20381] Updated weights for policy 1, policy_version 1282958 (0.0006) [2023-12-27 14:32:40,327][20381] Updated weights for policy 1, policy_version 1282968 (0.0008) [2023-12-27 14:32:40,844][20347] Updated weights for policy 0, policy_version 1283894 (0.0007) [2023-12-27 14:32:40,935][20347] Updated weights for policy 0, policy_version 1283904 (0.0008) [2023-12-27 14:32:40,987][20347] Updated weights for policy 0, policy_version 1283914 (0.0007) [2023-12-27 14:32:41,069][20381] Updated weights for policy 1, policy_version 1282978 (0.0011) [2023-12-27 14:32:41,141][20381] Updated weights for policy 1, policy_version 1282988 (0.0007) [2023-12-27 14:32:41,207][20381] Updated weights for policy 1, policy_version 1282998 (0.0010) [2023-12-27 14:32:41,744][20347] Updated weights for policy 0, policy_version 1283924 (0.0007) [2023-12-27 14:32:41,804][20347] Updated weights for policy 0, policy_version 1283934 (0.0008) [2023-12-27 14:32:41,863][20347] Updated weights for policy 0, policy_version 1283944 (0.0007) [2023-12-27 14:32:41,889][20381] Updated weights for policy 1, policy_version 1283008 (0.0006) [2023-12-27 14:32:41,948][20381] Updated weights for policy 1, policy_version 1283018 (0.0006) [2023-12-27 14:32:42,010][20381] Updated weights for policy 1, policy_version 1283028 (0.0005) [2023-12-27 14:32:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 657235968. Throughput: 0: 10243.4, 1: 10166.0. Samples: 657227860. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:42,363][19170] Avg episode reward: [(0, '912.110'), (1, '872.250')] [2023-12-27 14:32:42,553][20347] Updated weights for policy 0, policy_version 1283954 (0.0009) [2023-12-27 14:32:42,615][20347] Updated weights for policy 0, policy_version 1283964 (0.0008) [2023-12-27 14:32:42,631][20381] Updated weights for policy 1, policy_version 1283038 (0.0006) [2023-12-27 14:32:42,669][20347] Updated weights for policy 0, policy_version 1283974 (0.0010) [2023-12-27 14:32:42,684][20381] Updated weights for policy 1, policy_version 1283048 (0.0009) [2023-12-27 14:32:42,721][20347] Updated weights for policy 0, policy_version 1283984 (0.0010) [2023-12-27 14:32:42,734][20381] Updated weights for policy 1, policy_version 1283058 (0.0006) [2023-12-27 14:32:43,394][20347] Updated weights for policy 0, policy_version 1283994 (0.0008) [2023-12-27 14:32:43,429][20381] Updated weights for policy 1, policy_version 1283068 (0.0006) [2023-12-27 14:32:43,455][20347] Updated weights for policy 0, policy_version 1284004 (0.0007) [2023-12-27 14:32:43,484][20381] Updated weights for policy 1, policy_version 1283078 (0.0007) [2023-12-27 14:32:43,507][20347] Updated weights for policy 0, policy_version 1284014 (0.0007) [2023-12-27 14:32:43,531][20381] Updated weights for policy 1, policy_version 1283088 (0.0006) [2023-12-27 14:32:44,169][20347] Updated weights for policy 0, policy_version 1284024 (0.0010) [2023-12-27 14:32:44,221][20381] Updated weights for policy 1, policy_version 1283098 (0.0007) [2023-12-27 14:32:44,230][20347] Updated weights for policy 0, policy_version 1284034 (0.0010) [2023-12-27 14:32:44,274][20381] Updated weights for policy 1, policy_version 1283108 (0.0006) [2023-12-27 14:32:44,284][20347] Updated weights for policy 0, policy_version 1284044 (0.0009) [2023-12-27 14:32:44,324][20381] Updated weights for policy 1, policy_version 1283118 (0.0007) [2023-12-27 14:32:44,377][20381] Updated weights for policy 1, policy_version 1283128 (0.0007) [2023-12-27 14:32:44,941][20347] Updated weights for policy 0, policy_version 1284054 (0.0008) [2023-12-27 14:32:45,001][20347] Updated weights for policy 0, policy_version 1284064 (0.0007) [2023-12-27 14:32:45,064][20347] Updated weights for policy 0, policy_version 1284074 (0.0008) [2023-12-27 14:32:45,076][20381] Updated weights for policy 1, policy_version 1283138 (0.0011) [2023-12-27 14:32:45,139][20381] Updated weights for policy 1, policy_version 1283148 (0.0011) [2023-12-27 14:32:45,199][20381] Updated weights for policy 1, policy_version 1283158 (0.0011) [2023-12-27 14:32:45,774][20347] Updated weights for policy 0, policy_version 1284084 (0.0009) [2023-12-27 14:32:45,830][20347] Updated weights for policy 0, policy_version 1284094 (0.0007) [2023-12-27 14:32:45,879][20347] Updated weights for policy 0, policy_version 1284104 (0.0005) [2023-12-27 14:32:45,899][20381] Updated weights for policy 1, policy_version 1283168 (0.0011) [2023-12-27 14:32:45,954][20381] Updated weights for policy 1, policy_version 1283178 (0.0011) [2023-12-27 14:32:46,013][20381] Updated weights for policy 1, policy_version 1283188 (0.0010) [2023-12-27 14:32:46,482][20347] Updated weights for policy 0, policy_version 1284114 (0.0006) [2023-12-27 14:32:46,538][20347] Updated weights for policy 0, policy_version 1284124 (0.0006) [2023-12-27 14:32:46,584][20347] Updated weights for policy 0, policy_version 1284134 (0.0006) [2023-12-27 14:32:46,630][20381] Updated weights for policy 1, policy_version 1283198 (0.0008) [2023-12-27 14:32:46,640][20347] Updated weights for policy 0, policy_version 1284144 (0.0006) [2023-12-27 14:32:46,686][20381] Updated weights for policy 1, policy_version 1283208 (0.0005) [2023-12-27 14:32:46,739][20381] Updated weights for policy 1, policy_version 1283218 (0.0005) [2023-12-27 14:32:47,256][20347] Updated weights for policy 0, policy_version 1284154 (0.0010) [2023-12-27 14:32:47,311][20347] Updated weights for policy 0, policy_version 1284164 (0.0010) [2023-12-27 14:32:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20355.0). Total num frames: 657334272. Throughput: 0: 10244.5, 1: 10227.9. Samples: 657351036. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:47,363][19170] Avg episode reward: [(0, '941.450'), (1, '823.240')] [2023-12-27 14:32:47,371][20347] Updated weights for policy 0, policy_version 1284174 (0.0006) [2023-12-27 14:32:47,384][20381] Updated weights for policy 1, policy_version 1283228 (0.0007) [2023-12-27 14:32:47,430][20381] Updated weights for policy 1, policy_version 1283238 (0.0010) [2023-12-27 14:32:47,479][20381] Updated weights for policy 1, policy_version 1283248 (0.0005) [2023-12-27 14:32:48,006][20347] Updated weights for policy 0, policy_version 1284184 (0.0005) [2023-12-27 14:32:48,064][20347] Updated weights for policy 0, policy_version 1284194 (0.0005) [2023-12-27 14:32:48,111][20347] Updated weights for policy 0, policy_version 1284204 (0.0005) [2023-12-27 14:32:48,149][20381] Updated weights for policy 1, policy_version 1283258 (0.0007) [2023-12-27 14:32:48,201][20381] Updated weights for policy 1, policy_version 1283268 (0.0005) [2023-12-27 14:32:48,255][20381] Updated weights for policy 1, policy_version 1283278 (0.0005) [2023-12-27 14:32:48,310][20381] Updated weights for policy 1, policy_version 1283288 (0.0006) [2023-12-27 14:32:48,762][20347] Updated weights for policy 0, policy_version 1284214 (0.0007) [2023-12-27 14:32:48,816][20347] Updated weights for policy 0, policy_version 1284224 (0.0008) [2023-12-27 14:32:48,870][20347] Updated weights for policy 0, policy_version 1284234 (0.0007) [2023-12-27 14:32:48,975][20381] Updated weights for policy 1, policy_version 1283298 (0.0008) [2023-12-27 14:32:49,033][20381] Updated weights for policy 1, policy_version 1283308 (0.0007) [2023-12-27 14:32:49,084][20381] Updated weights for policy 1, policy_version 1283318 (0.0007) [2023-12-27 14:32:49,582][20347] Updated weights for policy 0, policy_version 1284244 (0.0007) [2023-12-27 14:32:49,634][20347] Updated weights for policy 0, policy_version 1284254 (0.0007) [2023-12-27 14:32:49,691][20347] Updated weights for policy 0, policy_version 1284264 (0.0007) [2023-12-27 14:32:49,828][20381] Updated weights for policy 1, policy_version 1283328 (0.0010) [2023-12-27 14:32:49,896][20381] Updated weights for policy 1, policy_version 1283338 (0.0011) [2023-12-27 14:32:49,962][20381] Updated weights for policy 1, policy_version 1283348 (0.0011) [2023-12-27 14:32:50,453][20347] Updated weights for policy 0, policy_version 1284274 (0.0008) [2023-12-27 14:32:50,511][20347] Updated weights for policy 0, policy_version 1284284 (0.0007) [2023-12-27 14:32:50,558][20347] Updated weights for policy 0, policy_version 1284294 (0.0007) [2023-12-27 14:32:50,613][20347] Updated weights for policy 0, policy_version 1284304 (0.0006) [2023-12-27 14:32:50,670][20381] Updated weights for policy 1, policy_version 1283358 (0.0009) [2023-12-27 14:32:50,736][20381] Updated weights for policy 1, policy_version 1283368 (0.0005) [2023-12-27 14:32:50,804][20381] Updated weights for policy 1, policy_version 1283378 (0.0008) [2023-12-27 14:32:51,280][20347] Updated weights for policy 0, policy_version 1284314 (0.0007) [2023-12-27 14:32:51,341][20347] Updated weights for policy 0, policy_version 1284324 (0.0008) [2023-12-27 14:32:51,411][20347] Updated weights for policy 0, policy_version 1284334 (0.0007) [2023-12-27 14:32:51,438][20381] Updated weights for policy 1, policy_version 1283388 (0.0007) [2023-12-27 14:32:51,508][20381] Updated weights for policy 1, policy_version 1283398 (0.0007) [2023-12-27 14:32:51,574][20381] Updated weights for policy 1, policy_version 1283408 (0.0008) [2023-12-27 14:32:52,062][20347] Updated weights for policy 0, policy_version 1284344 (0.0007) [2023-12-27 14:32:52,115][20347] Updated weights for policy 0, policy_version 1284354 (0.0007) [2023-12-27 14:32:52,160][20347] Updated weights for policy 0, policy_version 1284364 (0.0008) [2023-12-27 14:32:52,269][20381] Updated weights for policy 1, policy_version 1283418 (0.0008) [2023-12-27 14:32:52,332][20381] Updated weights for policy 1, policy_version 1283428 (0.0006) [2023-12-27 14:32:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 657440768. Throughput: 0: 10249.7, 1: 10151.2. Samples: 657412080. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:52,363][19170] Avg episode reward: [(0, '941.540'), (1, '862.600')] [2023-12-27 14:32:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001284368_328859648.pth... [2023-12-27 14:32:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001283152_328548352.pth [2023-12-27 14:32:52,398][20381] Updated weights for policy 1, policy_version 1283438 (0.0006) [2023-12-27 14:32:52,461][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001283448_328589312.pth... [2023-12-27 14:32:52,462][20381] Updated weights for policy 1, policy_version 1283448 (0.0008) [2023-12-27 14:32:52,465][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001282232_328278016.pth [2023-12-27 14:32:52,825][20347] Updated weights for policy 0, policy_version 1284374 (0.0008) [2023-12-27 14:32:52,886][20347] Updated weights for policy 0, policy_version 1284384 (0.0007) [2023-12-27 14:32:52,944][20347] Updated weights for policy 0, policy_version 1284394 (0.0008) [2023-12-27 14:32:53,150][20381] Updated weights for policy 1, policy_version 1283458 (0.0008) [2023-12-27 14:32:53,212][20381] Updated weights for policy 1, policy_version 1283468 (0.0005) [2023-12-27 14:32:53,267][20381] Updated weights for policy 1, policy_version 1283478 (0.0006) [2023-12-27 14:32:53,632][20347] Updated weights for policy 0, policy_version 1284404 (0.0007) [2023-12-27 14:32:53,689][20347] Updated weights for policy 0, policy_version 1284414 (0.0007) [2023-12-27 14:32:53,751][20347] Updated weights for policy 0, policy_version 1284424 (0.0007) [2023-12-27 14:32:53,925][20381] Updated weights for policy 1, policy_version 1283488 (0.0009) [2023-12-27 14:32:53,987][20381] Updated weights for policy 1, policy_version 1283498 (0.0010) [2023-12-27 14:32:54,039][20381] Updated weights for policy 1, policy_version 1283508 (0.0010) [2023-12-27 14:32:54,346][20347] Updated weights for policy 0, policy_version 1284434 (0.0008) [2023-12-27 14:32:54,400][20347] Updated weights for policy 0, policy_version 1284444 (0.0008) [2023-12-27 14:32:54,451][20347] Updated weights for policy 0, policy_version 1284454 (0.0008) [2023-12-27 14:32:54,499][20347] Updated weights for policy 0, policy_version 1284464 (0.0007) [2023-12-27 14:32:54,732][20381] Updated weights for policy 1, policy_version 1283518 (0.0010) [2023-12-27 14:32:54,790][20381] Updated weights for policy 1, policy_version 1283528 (0.0008) [2023-12-27 14:32:54,847][20381] Updated weights for policy 1, policy_version 1283538 (0.0008) [2023-12-27 14:32:55,118][20347] Updated weights for policy 0, policy_version 1284474 (0.0008) [2023-12-27 14:32:55,170][20347] Updated weights for policy 0, policy_version 1284484 (0.0007) [2023-12-27 14:32:55,231][20347] Updated weights for policy 0, policy_version 1284494 (0.0007) [2023-12-27 14:32:55,544][20381] Updated weights for policy 1, policy_version 1283548 (0.0008) [2023-12-27 14:32:55,590][20381] Updated weights for policy 1, policy_version 1283558 (0.0008) [2023-12-27 14:32:55,645][20381] Updated weights for policy 1, policy_version 1283568 (0.0007) [2023-12-27 14:32:55,940][20347] Updated weights for policy 0, policy_version 1284504 (0.0007) [2023-12-27 14:32:55,985][20347] Updated weights for policy 0, policy_version 1284514 (0.0008) [2023-12-27 14:32:56,036][20347] Updated weights for policy 0, policy_version 1284524 (0.0007) [2023-12-27 14:32:56,196][20381] Updated weights for policy 1, policy_version 1283578 (0.0006) [2023-12-27 14:32:56,248][20381] Updated weights for policy 1, policy_version 1283588 (0.0010) [2023-12-27 14:32:56,297][20381] Updated weights for policy 1, policy_version 1283598 (0.0010) [2023-12-27 14:32:56,345][20381] Updated weights for policy 1, policy_version 1283608 (0.0010) [2023-12-27 14:32:56,758][20347] Updated weights for policy 0, policy_version 1284534 (0.0009) [2023-12-27 14:32:56,808][20347] Updated weights for policy 0, policy_version 1284544 (0.0008) [2023-12-27 14:32:56,858][20347] Updated weights for policy 0, policy_version 1284554 (0.0007) [2023-12-27 14:32:56,921][20381] Updated weights for policy 1, policy_version 1283618 (0.0007) [2023-12-27 14:32:56,969][20381] Updated weights for policy 1, policy_version 1283628 (0.0010) [2023-12-27 14:32:57,020][20381] Updated weights for policy 1, policy_version 1283638 (0.0010) [2023-12-27 14:32:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 657547264. Throughput: 0: 10301.3, 1: 10112.9. Samples: 657536156. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:32:57,363][19170] Avg episode reward: [(0, '970.820'), (1, '911.660')] [2023-12-27 14:32:57,507][20347] Updated weights for policy 0, policy_version 1284564 (0.0007) [2023-12-27 14:32:57,554][20347] Updated weights for policy 0, policy_version 1284574 (0.0007) [2023-12-27 14:32:57,603][20347] Updated weights for policy 0, policy_version 1284584 (0.0006) [2023-12-27 14:32:57,627][20381] Updated weights for policy 1, policy_version 1283648 (0.0010) [2023-12-27 14:32:57,680][20381] Updated weights for policy 1, policy_version 1283658 (0.0010) [2023-12-27 14:32:57,728][20381] Updated weights for policy 1, policy_version 1283668 (0.0010) [2023-12-27 14:32:58,291][20347] Updated weights for policy 0, policy_version 1284594 (0.0006) [2023-12-27 14:32:58,308][20381] Updated weights for policy 1, policy_version 1283678 (0.0008) [2023-12-27 14:32:58,356][20347] Updated weights for policy 0, policy_version 1284604 (0.0007) [2023-12-27 14:32:58,387][20381] Updated weights for policy 1, policy_version 1283688 (0.0010) [2023-12-27 14:32:58,420][20347] Updated weights for policy 0, policy_version 1284614 (0.0008) [2023-12-27 14:32:58,455][20381] Updated weights for policy 1, policy_version 1283698 (0.0008) [2023-12-27 14:32:58,483][20347] Updated weights for policy 0, policy_version 1284624 (0.0008) [2023-12-27 14:32:59,268][20381] Updated weights for policy 1, policy_version 1283708 (0.0010) [2023-12-27 14:32:59,321][20347] Updated weights for policy 0, policy_version 1284634 (0.0009) [2023-12-27 14:32:59,334][20381] Updated weights for policy 1, policy_version 1283718 (0.0007) [2023-12-27 14:32:59,394][20347] Updated weights for policy 0, policy_version 1284644 (0.0008) [2023-12-27 14:32:59,402][20381] Updated weights for policy 1, policy_version 1283728 (0.0008) [2023-12-27 14:32:59,451][20347] Updated weights for policy 0, policy_version 1284654 (0.0008) [2023-12-27 14:33:00,081][20381] Updated weights for policy 1, policy_version 1283738 (0.0006) [2023-12-27 14:33:00,138][20381] Updated weights for policy 1, policy_version 1283748 (0.0007) [2023-12-27 14:33:00,158][20347] Updated weights for policy 0, policy_version 1284664 (0.0009) [2023-12-27 14:33:00,190][20381] Updated weights for policy 1, policy_version 1283758 (0.0008) [2023-12-27 14:33:00,209][20347] Updated weights for policy 0, policy_version 1284674 (0.0007) [2023-12-27 14:33:00,252][20381] Updated weights for policy 1, policy_version 1283768 (0.0006) [2023-12-27 14:33:00,265][20347] Updated weights for policy 0, policy_version 1284684 (0.0006) [2023-12-27 14:33:00,819][20381] Updated weights for policy 1, policy_version 1283778 (0.0006) [2023-12-27 14:33:00,864][20381] Updated weights for policy 1, policy_version 1283788 (0.0005) [2023-12-27 14:33:00,914][20381] Updated weights for policy 1, policy_version 1283798 (0.0005) [2023-12-27 14:33:00,917][20347] Updated weights for policy 0, policy_version 1284694 (0.0007) [2023-12-27 14:33:00,963][20347] Updated weights for policy 0, policy_version 1284704 (0.0005) [2023-12-27 14:33:01,009][20347] Updated weights for policy 0, policy_version 1284714 (0.0005) [2023-12-27 14:33:01,608][20381] Updated weights for policy 1, policy_version 1283808 (0.0007) [2023-12-27 14:33:01,666][20381] Updated weights for policy 1, policy_version 1283818 (0.0008) [2023-12-27 14:33:01,712][20381] Updated weights for policy 1, policy_version 1283828 (0.0007) [2023-12-27 14:33:01,778][20347] Updated weights for policy 0, policy_version 1284724 (0.0008) [2023-12-27 14:33:01,838][20347] Updated weights for policy 0, policy_version 1284734 (0.0008) [2023-12-27 14:33:01,895][20347] Updated weights for policy 0, policy_version 1284744 (0.0007) [2023-12-27 14:33:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 657645568. Throughput: 0: 10249.5, 1: 10197.3. Samples: 657659032. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:02,363][19170] Avg episode reward: [(0, '970.780'), (1, '901.930')] [2023-12-27 14:33:02,381][20381] Updated weights for policy 1, policy_version 1283838 (0.0009) [2023-12-27 14:33:02,412][20292] KL-divergence is very high: 125.9340 [2023-12-27 14:33:02,432][20381] Updated weights for policy 1, policy_version 1283848 (0.0006) [2023-12-27 14:33:02,454][20292] KL-divergence is very high: 105.4516 [2023-12-27 14:33:02,486][20381] Updated weights for policy 1, policy_version 1283858 (0.0005) [2023-12-27 14:33:02,641][20347] Updated weights for policy 0, policy_version 1284754 (0.0007) [2023-12-27 14:33:02,702][20347] Updated weights for policy 0, policy_version 1284764 (0.0009) [2023-12-27 14:33:02,756][20347] Updated weights for policy 0, policy_version 1284774 (0.0006) [2023-12-27 14:33:02,813][20347] Updated weights for policy 0, policy_version 1284784 (0.0010) [2023-12-27 14:33:03,164][20381] Updated weights for policy 1, policy_version 1283868 (0.0008) [2023-12-27 14:33:03,222][20381] Updated weights for policy 1, policy_version 1283878 (0.0010) [2023-12-27 14:33:03,271][20381] Updated weights for policy 1, policy_version 1283888 (0.0011) [2023-12-27 14:33:03,446][20347] Updated weights for policy 0, policy_version 1284794 (0.0008) [2023-12-27 14:33:03,491][20347] Updated weights for policy 0, policy_version 1284804 (0.0010) [2023-12-27 14:33:03,547][20347] Updated weights for policy 0, policy_version 1284814 (0.0010) [2023-12-27 14:33:03,971][20381] Updated weights for policy 1, policy_version 1283898 (0.0010) [2023-12-27 14:33:04,025][20381] Updated weights for policy 1, policy_version 1283908 (0.0008) [2023-12-27 14:33:04,084][20381] Updated weights for policy 1, policy_version 1283918 (0.0008) [2023-12-27 14:33:04,143][20381] Updated weights for policy 1, policy_version 1283928 (0.0008) [2023-12-27 14:33:04,242][20347] Updated weights for policy 0, policy_version 1284824 (0.0011) [2023-12-27 14:33:04,304][20347] Updated weights for policy 0, policy_version 1284834 (0.0010) [2023-12-27 14:33:04,359][20347] Updated weights for policy 0, policy_version 1284844 (0.0010) [2023-12-27 14:33:04,891][20381] Updated weights for policy 1, policy_version 1283938 (0.0010) [2023-12-27 14:33:04,936][20381] Updated weights for policy 1, policy_version 1283948 (0.0010) [2023-12-27 14:33:04,968][20347] Updated weights for policy 0, policy_version 1284854 (0.0008) [2023-12-27 14:33:04,991][20381] Updated weights for policy 1, policy_version 1283958 (0.0010) [2023-12-27 14:33:05,021][20347] Updated weights for policy 0, policy_version 1284864 (0.0006) [2023-12-27 14:33:05,077][20347] Updated weights for policy 0, policy_version 1284874 (0.0005) [2023-12-27 14:33:05,615][20381] Updated weights for policy 1, policy_version 1283968 (0.0011) [2023-12-27 14:33:05,675][20347] Updated weights for policy 0, policy_version 1284884 (0.0007) [2023-12-27 14:33:05,676][20381] Updated weights for policy 1, policy_version 1283978 (0.0007) [2023-12-27 14:33:05,724][20381] Updated weights for policy 1, policy_version 1283988 (0.0005) [2023-12-27 14:33:05,726][20347] Updated weights for policy 0, policy_version 1284894 (0.0010) [2023-12-27 14:33:05,774][20347] Updated weights for policy 0, policy_version 1284904 (0.0010) [2023-12-27 14:33:06,312][20381] Updated weights for policy 1, policy_version 1283998 (0.0009) [2023-12-27 14:33:06,372][20381] Updated weights for policy 1, policy_version 1284008 (0.0011) [2023-12-27 14:33:06,420][20347] Updated weights for policy 0, policy_version 1284914 (0.0010) [2023-12-27 14:33:06,435][20381] Updated weights for policy 1, policy_version 1284018 (0.0011) [2023-12-27 14:33:06,481][20347] Updated weights for policy 0, policy_version 1284924 (0.0009) [2023-12-27 14:33:06,539][20347] Updated weights for policy 0, policy_version 1284934 (0.0010) [2023-12-27 14:33:06,605][20347] Updated weights for policy 0, policy_version 1284944 (0.0011) [2023-12-27 14:33:07,108][20381] Updated weights for policy 1, policy_version 1284028 (0.0010) [2023-12-27 14:33:07,171][20381] Updated weights for policy 1, policy_version 1284038 (0.0011) [2023-12-27 14:33:07,238][20381] Updated weights for policy 1, policy_version 1284048 (0.0008) [2023-12-27 14:33:07,266][20347] Updated weights for policy 0, policy_version 1284954 (0.0007) [2023-12-27 14:33:07,322][20347] Updated weights for policy 0, policy_version 1284964 (0.0009) [2023-12-27 14:33:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 657752064. Throughput: 0: 10294.9, 1: 10216.7. Samples: 657720984. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:07,363][19170] Avg episode reward: [(0, '951.240'), (1, '883.060')] [2023-12-27 14:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001284056_328744960.pth... [2023-12-27 14:33:07,371][20347] Updated weights for policy 0, policy_version 1284974 (0.0011) [2023-12-27 14:33:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001282840_328433664.pth [2023-12-27 14:33:07,379][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001284976_329015296.pth... [2023-12-27 14:33:07,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001283760_328704000.pth [2023-12-27 14:33:07,816][20381] Updated weights for policy 1, policy_version 1284058 (0.0007) [2023-12-27 14:33:07,871][20381] Updated weights for policy 1, policy_version 1284068 (0.0010) [2023-12-27 14:33:07,935][20381] Updated weights for policy 1, policy_version 1284078 (0.0009) [2023-12-27 14:33:08,000][20381] Updated weights for policy 1, policy_version 1284088 (0.0006) [2023-12-27 14:33:08,019][20347] Updated weights for policy 0, policy_version 1284984 (0.0010) [2023-12-27 14:33:08,074][20347] Updated weights for policy 0, policy_version 1284994 (0.0010) [2023-12-27 14:33:08,126][20347] Updated weights for policy 0, policy_version 1285004 (0.0010) [2023-12-27 14:33:08,711][20381] Updated weights for policy 1, policy_version 1284098 (0.0009) [2023-12-27 14:33:08,757][20347] Updated weights for policy 0, policy_version 1285014 (0.0007) [2023-12-27 14:33:08,767][20381] Updated weights for policy 1, policy_version 1284108 (0.0008) [2023-12-27 14:33:08,814][20347] Updated weights for policy 0, policy_version 1285024 (0.0006) [2023-12-27 14:33:08,824][20381] Updated weights for policy 1, policy_version 1284118 (0.0007) [2023-12-27 14:33:08,867][20347] Updated weights for policy 0, policy_version 1285034 (0.0007) [2023-12-27 14:33:09,495][20347] Updated weights for policy 0, policy_version 1285044 (0.0009) [2023-12-27 14:33:09,552][20347] Updated weights for policy 0, policy_version 1285054 (0.0007) [2023-12-27 14:33:09,553][20381] Updated weights for policy 1, policy_version 1284128 (0.0007) [2023-12-27 14:33:09,607][20381] Updated weights for policy 1, policy_version 1284138 (0.0006) [2023-12-27 14:33:09,609][20347] Updated weights for policy 0, policy_version 1285064 (0.0005) [2023-12-27 14:33:09,664][20381] Updated weights for policy 1, policy_version 1284148 (0.0006) [2023-12-27 14:33:10,331][20347] Updated weights for policy 0, policy_version 1285074 (0.0007) [2023-12-27 14:33:10,336][20381] Updated weights for policy 1, policy_version 1284158 (0.0006) [2023-12-27 14:33:10,399][20347] Updated weights for policy 0, policy_version 1285084 (0.0008) [2023-12-27 14:33:10,399][20381] Updated weights for policy 1, policy_version 1284168 (0.0007) [2023-12-27 14:33:10,455][20347] Updated weights for policy 0, policy_version 1285094 (0.0006) [2023-12-27 14:33:10,456][20381] Updated weights for policy 1, policy_version 1284178 (0.0008) [2023-12-27 14:33:10,509][20347] Updated weights for policy 0, policy_version 1285104 (0.0006) [2023-12-27 14:33:11,149][20381] Updated weights for policy 1, policy_version 1284188 (0.0008) [2023-12-27 14:33:11,178][20347] Updated weights for policy 0, policy_version 1285114 (0.0008) [2023-12-27 14:33:11,214][20381] Updated weights for policy 1, policy_version 1284198 (0.0009) [2023-12-27 14:33:11,243][20347] Updated weights for policy 0, policy_version 1285124 (0.0006) [2023-12-27 14:33:11,276][20381] Updated weights for policy 1, policy_version 1284208 (0.0011) [2023-12-27 14:33:11,307][20347] Updated weights for policy 0, policy_version 1285134 (0.0008) [2023-12-27 14:33:11,956][20381] Updated weights for policy 1, policy_version 1284218 (0.0010) [2023-12-27 14:33:12,011][20381] Updated weights for policy 1, policy_version 1284228 (0.0006) [2023-12-27 14:33:12,042][20347] Updated weights for policy 0, policy_version 1285144 (0.0008) [2023-12-27 14:33:12,066][20381] Updated weights for policy 1, policy_version 1284238 (0.0006) [2023-12-27 14:33:12,096][20347] Updated weights for policy 0, policy_version 1285154 (0.0006) [2023-12-27 14:33:12,117][20381] Updated weights for policy 1, policy_version 1284248 (0.0007) [2023-12-27 14:33:12,154][20347] Updated weights for policy 0, policy_version 1285164 (0.0006) [2023-12-27 14:33:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20382.8). Total num frames: 657858560. Throughput: 0: 10359.4, 1: 10312.6. Samples: 657846644. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:12,363][19170] Avg episode reward: [(0, '961.020'), (1, '862.670')] [2023-12-27 14:33:12,792][20381] Updated weights for policy 1, policy_version 1284258 (0.0007) [2023-12-27 14:33:12,855][20381] Updated weights for policy 1, policy_version 1284268 (0.0007) [2023-12-27 14:33:12,879][20347] Updated weights for policy 0, policy_version 1285174 (0.0007) [2023-12-27 14:33:12,915][20381] Updated weights for policy 1, policy_version 1284278 (0.0007) [2023-12-27 14:33:12,942][20347] Updated weights for policy 0, policy_version 1285184 (0.0007) [2023-12-27 14:33:13,004][20347] Updated weights for policy 0, policy_version 1285194 (0.0008) [2023-12-27 14:33:13,525][20381] Updated weights for policy 1, policy_version 1284288 (0.0008) [2023-12-27 14:33:13,585][20381] Updated weights for policy 1, policy_version 1284298 (0.0006) [2023-12-27 14:33:13,643][20381] Updated weights for policy 1, policy_version 1284308 (0.0006) [2023-12-27 14:33:13,649][20347] Updated weights for policy 0, policy_version 1285204 (0.0007) [2023-12-27 14:33:13,698][20347] Updated weights for policy 0, policy_version 1285214 (0.0008) [2023-12-27 14:33:13,750][20347] Updated weights for policy 0, policy_version 1285224 (0.0009) [2023-12-27 14:33:14,310][20381] Updated weights for policy 1, policy_version 1284318 (0.0006) [2023-12-27 14:33:14,366][20381] Updated weights for policy 1, policy_version 1284328 (0.0008) [2023-12-27 14:33:14,383][20347] Updated weights for policy 0, policy_version 1285234 (0.0006) [2023-12-27 14:33:14,429][20381] Updated weights for policy 1, policy_version 1284338 (0.0006) [2023-12-27 14:33:14,441][20347] Updated weights for policy 0, policy_version 1285244 (0.0005) [2023-12-27 14:33:14,503][20347] Updated weights for policy 0, policy_version 1285254 (0.0005) [2023-12-27 14:33:14,562][20347] Updated weights for policy 0, policy_version 1285264 (0.0006) [2023-12-27 14:33:15,053][20381] Updated weights for policy 1, policy_version 1284348 (0.0009) [2023-12-27 14:33:15,112][20381] Updated weights for policy 1, policy_version 1284358 (0.0011) [2023-12-27 14:33:15,172][20381] Updated weights for policy 1, policy_version 1284368 (0.0011) [2023-12-27 14:33:15,197][20347] Updated weights for policy 0, policy_version 1285274 (0.0008) [2023-12-27 14:33:15,254][20347] Updated weights for policy 0, policy_version 1285284 (0.0006) [2023-12-27 14:33:15,313][20347] Updated weights for policy 0, policy_version 1285294 (0.0008) [2023-12-27 14:33:15,767][20381] Updated weights for policy 1, policy_version 1284378 (0.0010) [2023-12-27 14:33:15,825][20381] Updated weights for policy 1, policy_version 1284388 (0.0007) [2023-12-27 14:33:15,885][20381] Updated weights for policy 1, policy_version 1284398 (0.0008) [2023-12-27 14:33:15,944][20381] Updated weights for policy 1, policy_version 1284408 (0.0008) [2023-12-27 14:33:16,031][20347] Updated weights for policy 0, policy_version 1285304 (0.0008) [2023-12-27 14:33:16,086][20347] Updated weights for policy 0, policy_version 1285314 (0.0008) [2023-12-27 14:33:16,134][20347] Updated weights for policy 0, policy_version 1285324 (0.0007) [2023-12-27 14:33:16,608][20381] Updated weights for policy 1, policy_version 1284418 (0.0006) [2023-12-27 14:33:16,659][20381] Updated weights for policy 1, policy_version 1284428 (0.0007) [2023-12-27 14:33:16,718][20381] Updated weights for policy 1, policy_version 1284438 (0.0006) [2023-12-27 14:33:16,833][20347] Updated weights for policy 0, policy_version 1285334 (0.0006) [2023-12-27 14:33:16,892][20347] Updated weights for policy 0, policy_version 1285344 (0.0006) [2023-12-27 14:33:16,938][20347] Updated weights for policy 0, policy_version 1285354 (0.0007) [2023-12-27 14:33:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 657956864. Throughput: 0: 10351.9, 1: 10353.8. Samples: 657969348. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:17,363][19170] Avg episode reward: [(0, '961.020'), (1, '852.920')] [2023-12-27 14:33:17,381][20381] Updated weights for policy 1, policy_version 1284448 (0.0009) [2023-12-27 14:33:17,432][20381] Updated weights for policy 1, policy_version 1284458 (0.0010) [2023-12-27 14:33:17,484][20381] Updated weights for policy 1, policy_version 1284468 (0.0010) [2023-12-27 14:33:17,624][20347] Updated weights for policy 0, policy_version 1285364 (0.0007) [2023-12-27 14:33:17,685][20347] Updated weights for policy 0, policy_version 1285374 (0.0007) [2023-12-27 14:33:17,733][20347] Updated weights for policy 0, policy_version 1285384 (0.0008) [2023-12-27 14:33:18,179][20381] Updated weights for policy 1, policy_version 1284478 (0.0007) [2023-12-27 14:33:18,225][20381] Updated weights for policy 1, policy_version 1284488 (0.0005) [2023-12-27 14:33:18,274][20381] Updated weights for policy 1, policy_version 1284498 (0.0005) [2023-12-27 14:33:18,416][20347] Updated weights for policy 0, policy_version 1285394 (0.0007) [2023-12-27 14:33:18,479][20347] Updated weights for policy 0, policy_version 1285404 (0.0007) [2023-12-27 14:33:18,541][20347] Updated weights for policy 0, policy_version 1285414 (0.0007) [2023-12-27 14:33:18,605][20347] Updated weights for policy 0, policy_version 1285424 (0.0008) [2023-12-27 14:33:18,982][20381] Updated weights for policy 1, policy_version 1284508 (0.0007) [2023-12-27 14:33:19,031][20381] Updated weights for policy 1, policy_version 1284518 (0.0007) [2023-12-27 14:33:19,084][20381] Updated weights for policy 1, policy_version 1284528 (0.0008) [2023-12-27 14:33:19,261][20347] Updated weights for policy 0, policy_version 1285434 (0.0008) [2023-12-27 14:33:19,312][20347] Updated weights for policy 0, policy_version 1285444 (0.0007) [2023-12-27 14:33:19,379][20347] Updated weights for policy 0, policy_version 1285454 (0.0008) [2023-12-27 14:33:19,901][20381] Updated weights for policy 1, policy_version 1284538 (0.0008) [2023-12-27 14:33:19,962][20381] Updated weights for policy 1, policy_version 1284548 (0.0008) [2023-12-27 14:33:20,023][20381] Updated weights for policy 1, policy_version 1284558 (0.0006) [2023-12-27 14:33:20,082][20381] Updated weights for policy 1, policy_version 1284568 (0.0007) [2023-12-27 14:33:20,096][20347] Updated weights for policy 0, policy_version 1285464 (0.0006) [2023-12-27 14:33:20,152][20347] Updated weights for policy 0, policy_version 1285474 (0.0006) [2023-12-27 14:33:20,212][20347] Updated weights for policy 0, policy_version 1285484 (0.0006) [2023-12-27 14:33:20,765][20381] Updated weights for policy 1, policy_version 1284578 (0.0008) [2023-12-27 14:33:20,825][20381] Updated weights for policy 1, policy_version 1284588 (0.0008) [2023-12-27 14:33:20,850][20347] Updated weights for policy 0, policy_version 1285494 (0.0006) [2023-12-27 14:33:20,877][20381] Updated weights for policy 1, policy_version 1284598 (0.0008) [2023-12-27 14:33:20,909][20347] Updated weights for policy 0, policy_version 1285504 (0.0008) [2023-12-27 14:33:20,971][20347] Updated weights for policy 0, policy_version 1285514 (0.0008) [2023-12-27 14:33:21,549][20381] Updated weights for policy 1, policy_version 1284608 (0.0007) [2023-12-27 14:33:21,605][20381] Updated weights for policy 1, policy_version 1284618 (0.0007) [2023-12-27 14:33:21,671][20381] Updated weights for policy 1, policy_version 1284628 (0.0009) [2023-12-27 14:33:21,717][20347] Updated weights for policy 0, policy_version 1285524 (0.0008) [2023-12-27 14:33:21,785][20347] Updated weights for policy 0, policy_version 1285534 (0.0006) [2023-12-27 14:33:21,843][20347] Updated weights for policy 0, policy_version 1285544 (0.0007) [2023-12-27 14:33:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 658055168. Throughput: 0: 10270.6, 1: 10291.9. Samples: 658029640. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:22,363][19170] Avg episode reward: [(0, '961.020'), (1, '833.470')] [2023-12-27 14:33:22,367][20381] Updated weights for policy 1, policy_version 1284638 (0.0008) [2023-12-27 14:33:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001285552_329162752.pth... [2023-12-27 14:33:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001284368_328859648.pth [2023-12-27 14:33:22,427][20381] Updated weights for policy 1, policy_version 1284648 (0.0007) [2023-12-27 14:33:22,491][20381] Updated weights for policy 1, policy_version 1284658 (0.0007) [2023-12-27 14:33:22,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001284664_328900608.pth... [2023-12-27 14:33:22,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001283448_328589312.pth [2023-12-27 14:33:22,624][20347] Updated weights for policy 0, policy_version 1285554 (0.0008) [2023-12-27 14:33:22,685][20347] Updated weights for policy 0, policy_version 1285564 (0.0007) [2023-12-27 14:33:22,748][20347] Updated weights for policy 0, policy_version 1285574 (0.0007) [2023-12-27 14:33:22,811][20347] Updated weights for policy 0, policy_version 1285584 (0.0007) [2023-12-27 14:33:23,204][20381] Updated weights for policy 1, policy_version 1284668 (0.0008) [2023-12-27 14:33:23,255][20381] Updated weights for policy 1, policy_version 1284678 (0.0007) [2023-12-27 14:33:23,313][20381] Updated weights for policy 1, policy_version 1284688 (0.0007) [2023-12-27 14:33:23,498][20347] Updated weights for policy 0, policy_version 1285594 (0.0008) [2023-12-27 14:33:23,552][20347] Updated weights for policy 0, policy_version 1285604 (0.0007) [2023-12-27 14:33:23,607][20347] Updated weights for policy 0, policy_version 1285614 (0.0006) [2023-12-27 14:33:23,976][20381] Updated weights for policy 1, policy_version 1284698 (0.0008) [2023-12-27 14:33:24,036][20381] Updated weights for policy 1, policy_version 1284708 (0.0008) [2023-12-27 14:33:24,097][20381] Updated weights for policy 1, policy_version 1284718 (0.0008) [2023-12-27 14:33:24,148][20381] Updated weights for policy 1, policy_version 1284728 (0.0007) [2023-12-27 14:33:24,279][20347] Updated weights for policy 0, policy_version 1285624 (0.0008) [2023-12-27 14:33:24,333][20347] Updated weights for policy 0, policy_version 1285634 (0.0007) [2023-12-27 14:33:24,389][20347] Updated weights for policy 0, policy_version 1285644 (0.0009) [2023-12-27 14:33:24,726][20381] Updated weights for policy 1, policy_version 1284738 (0.0008) [2023-12-27 14:33:24,774][20381] Updated weights for policy 1, policy_version 1284748 (0.0007) [2023-12-27 14:33:24,826][20381] Updated weights for policy 1, policy_version 1284758 (0.0007) [2023-12-27 14:33:25,012][20347] Updated weights for policy 0, policy_version 1285654 (0.0007) [2023-12-27 14:33:25,065][20347] Updated weights for policy 0, policy_version 1285664 (0.0005) [2023-12-27 14:33:25,115][20347] Updated weights for policy 0, policy_version 1285674 (0.0007) [2023-12-27 14:33:25,559][20381] Updated weights for policy 1, policy_version 1284768 (0.0009) [2023-12-27 14:33:25,616][20381] Updated weights for policy 1, policy_version 1284778 (0.0008) [2023-12-27 14:33:25,662][20381] Updated weights for policy 1, policy_version 1284788 (0.0008) [2023-12-27 14:33:25,766][20347] Updated weights for policy 0, policy_version 1285684 (0.0008) [2023-12-27 14:33:25,824][20347] Updated weights for policy 0, policy_version 1285694 (0.0009) [2023-12-27 14:33:25,878][20347] Updated weights for policy 0, policy_version 1285704 (0.0010) [2023-12-27 14:33:26,345][20381] Updated weights for policy 1, policy_version 1284798 (0.0005) [2023-12-27 14:33:26,403][20381] Updated weights for policy 1, policy_version 1284808 (0.0005) [2023-12-27 14:33:26,463][20347] Updated weights for policy 0, policy_version 1285714 (0.0009) [2023-12-27 14:33:26,466][20381] Updated weights for policy 1, policy_version 1284818 (0.0005) [2023-12-27 14:33:26,524][20347] Updated weights for policy 0, policy_version 1285724 (0.0005) [2023-12-27 14:33:26,577][20347] Updated weights for policy 0, policy_version 1285734 (0.0005) [2023-12-27 14:33:26,626][20347] Updated weights for policy 0, policy_version 1285744 (0.0005) [2023-12-27 14:33:27,066][20381] Updated weights for policy 1, policy_version 1284828 (0.0008) [2023-12-27 14:33:27,116][20381] Updated weights for policy 1, policy_version 1284838 (0.0010) [2023-12-27 14:33:27,165][20347] Updated weights for policy 0, policy_version 1285754 (0.0010) [2023-12-27 14:33:27,167][20381] Updated weights for policy 1, policy_version 1284848 (0.0010) [2023-12-27 14:33:27,213][20347] Updated weights for policy 0, policy_version 1285764 (0.0010) [2023-12-27 14:33:27,270][20347] Updated weights for policy 0, policy_version 1285774 (0.0010) [2023-12-27 14:33:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20438.3). Total num frames: 658169856. Throughput: 0: 10226.7, 1: 10310.5. Samples: 658152032. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:27,363][19170] Avg episode reward: [(0, '951.260'), (1, '882.520')] [2023-12-27 14:33:27,816][20381] Updated weights for policy 1, policy_version 1284858 (0.0009) [2023-12-27 14:33:27,873][20381] Updated weights for policy 1, policy_version 1284868 (0.0006) [2023-12-27 14:33:27,876][20347] Updated weights for policy 0, policy_version 1285784 (0.0010) [2023-12-27 14:33:27,929][20381] Updated weights for policy 1, policy_version 1284878 (0.0005) [2023-12-27 14:33:27,931][20347] Updated weights for policy 0, policy_version 1285794 (0.0010) [2023-12-27 14:33:27,980][20381] Updated weights for policy 1, policy_version 1284888 (0.0005) [2023-12-27 14:33:27,982][20347] Updated weights for policy 0, policy_version 1285804 (0.0010) [2023-12-27 14:33:28,525][20381] Updated weights for policy 1, policy_version 1284898 (0.0007) [2023-12-27 14:33:28,578][20381] Updated weights for policy 1, policy_version 1284908 (0.0010) [2023-12-27 14:33:28,630][20381] Updated weights for policy 1, policy_version 1284918 (0.0011) [2023-12-27 14:33:28,672][20347] Updated weights for policy 0, policy_version 1285814 (0.0008) [2023-12-27 14:33:28,726][20347] Updated weights for policy 0, policy_version 1285824 (0.0008) [2023-12-27 14:33:28,784][20347] Updated weights for policy 0, policy_version 1285834 (0.0008) [2023-12-27 14:33:29,251][20381] Updated weights for policy 1, policy_version 1284928 (0.0008) [2023-12-27 14:33:29,319][20381] Updated weights for policy 1, policy_version 1284938 (0.0007) [2023-12-27 14:33:29,387][20381] Updated weights for policy 1, policy_version 1284948 (0.0010) [2023-12-27 14:33:29,523][20347] Updated weights for policy 0, policy_version 1285844 (0.0007) [2023-12-27 14:33:29,585][20347] Updated weights for policy 0, policy_version 1285854 (0.0008) [2023-12-27 14:33:29,642][20347] Updated weights for policy 0, policy_version 1285864 (0.0009) [2023-12-27 14:33:30,026][20381] Updated weights for policy 1, policy_version 1284958 (0.0007) [2023-12-27 14:33:30,080][20381] Updated weights for policy 1, policy_version 1284968 (0.0008) [2023-12-27 14:33:30,130][20381] Updated weights for policy 1, policy_version 1284978 (0.0008) [2023-12-27 14:33:30,370][20347] Updated weights for policy 0, policy_version 1285874 (0.0007) [2023-12-27 14:33:30,422][20347] Updated weights for policy 0, policy_version 1285884 (0.0007) [2023-12-27 14:33:30,468][20347] Updated weights for policy 0, policy_version 1285894 (0.0008) [2023-12-27 14:33:30,518][20347] Updated weights for policy 0, policy_version 1285904 (0.0007) [2023-12-27 14:33:30,797][20381] Updated weights for policy 1, policy_version 1284988 (0.0008) [2023-12-27 14:33:30,851][20381] Updated weights for policy 1, policy_version 1284998 (0.0009) [2023-12-27 14:33:30,898][20381] Updated weights for policy 1, policy_version 1285008 (0.0010) [2023-12-27 14:33:31,196][20347] Updated weights for policy 0, policy_version 1285914 (0.0006) [2023-12-27 14:33:31,261][20347] Updated weights for policy 0, policy_version 1285924 (0.0007) [2023-12-27 14:33:31,326][20347] Updated weights for policy 0, policy_version 1285934 (0.0009) [2023-12-27 14:33:31,624][20381] Updated weights for policy 1, policy_version 1285018 (0.0009) [2023-12-27 14:33:31,688][20381] Updated weights for policy 1, policy_version 1285028 (0.0009) [2023-12-27 14:33:31,753][20381] Updated weights for policy 1, policy_version 1285038 (0.0007) [2023-12-27 14:33:31,813][20381] Updated weights for policy 1, policy_version 1285048 (0.0006) [2023-12-27 14:33:31,973][20347] Updated weights for policy 0, policy_version 1285944 (0.0007) [2023-12-27 14:33:32,030][20347] Updated weights for policy 0, policy_version 1285954 (0.0007) [2023-12-27 14:33:32,086][20347] Updated weights for policy 0, policy_version 1285964 (0.0006) [2023-12-27 14:33:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20410.6). Total num frames: 658268160. Throughput: 0: 10279.6, 1: 10369.0. Samples: 658280224. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:32,363][19170] Avg episode reward: [(0, '931.770'), (1, '892.260')] [2023-12-27 14:33:32,455][20381] Updated weights for policy 1, policy_version 1285058 (0.0011) [2023-12-27 14:33:32,507][20381] Updated weights for policy 1, policy_version 1285068 (0.0010) [2023-12-27 14:33:32,559][20381] Updated weights for policy 1, policy_version 1285078 (0.0011) [2023-12-27 14:33:32,716][20347] Updated weights for policy 0, policy_version 1285974 (0.0006) [2023-12-27 14:33:32,771][20347] Updated weights for policy 0, policy_version 1285984 (0.0006) [2023-12-27 14:33:32,831][20347] Updated weights for policy 0, policy_version 1285994 (0.0008) [2023-12-27 14:33:33,207][20381] Updated weights for policy 1, policy_version 1285088 (0.0007) [2023-12-27 14:33:33,265][20381] Updated weights for policy 1, policy_version 1285098 (0.0006) [2023-12-27 14:33:33,323][20381] Updated weights for policy 1, policy_version 1285108 (0.0008) [2023-12-27 14:33:33,492][20347] Updated weights for policy 0, policy_version 1286004 (0.0006) [2023-12-27 14:33:33,547][20347] Updated weights for policy 0, policy_version 1286014 (0.0005) [2023-12-27 14:33:33,600][20347] Updated weights for policy 0, policy_version 1286024 (0.0006) [2023-12-27 14:33:33,951][20381] Updated weights for policy 1, policy_version 1285118 (0.0010) [2023-12-27 14:33:34,005][20381] Updated weights for policy 1, policy_version 1285128 (0.0010) [2023-12-27 14:33:34,082][20381] Updated weights for policy 1, policy_version 1285138 (0.0007) [2023-12-27 14:33:34,281][20347] Updated weights for policy 0, policy_version 1286034 (0.0007) [2023-12-27 14:33:34,337][20347] Updated weights for policy 0, policy_version 1286044 (0.0007) [2023-12-27 14:33:34,396][20347] Updated weights for policy 0, policy_version 1286054 (0.0007) [2023-12-27 14:33:34,454][20347] Updated weights for policy 0, policy_version 1286064 (0.0007) [2023-12-27 14:33:34,724][20381] Updated weights for policy 1, policy_version 1285148 (0.0007) [2023-12-27 14:33:34,779][20381] Updated weights for policy 1, policy_version 1285158 (0.0010) [2023-12-27 14:33:34,836][20381] Updated weights for policy 1, policy_version 1285168 (0.0010) [2023-12-27 14:33:35,105][20347] Updated weights for policy 0, policy_version 1286074 (0.0010) [2023-12-27 14:33:35,156][20347] Updated weights for policy 0, policy_version 1286084 (0.0010) [2023-12-27 14:33:35,208][20347] Updated weights for policy 0, policy_version 1286094 (0.0009) [2023-12-27 14:33:35,403][20381] Updated weights for policy 1, policy_version 1285178 (0.0011) [2023-12-27 14:33:35,458][20381] Updated weights for policy 1, policy_version 1285188 (0.0011) [2023-12-27 14:33:35,506][20381] Updated weights for policy 1, policy_version 1285198 (0.0010) [2023-12-27 14:33:35,560][20381] Updated weights for policy 1, policy_version 1285208 (0.0010) [2023-12-27 14:33:35,844][20347] Updated weights for policy 0, policy_version 1286104 (0.0010) [2023-12-27 14:33:35,893][20347] Updated weights for policy 0, policy_version 1286114 (0.0010) [2023-12-27 14:33:35,943][20347] Updated weights for policy 0, policy_version 1286124 (0.0010) [2023-12-27 14:33:36,178][20381] Updated weights for policy 1, policy_version 1285218 (0.0010) [2023-12-27 14:33:36,238][20381] Updated weights for policy 1, policy_version 1285228 (0.0011) [2023-12-27 14:33:36,299][20381] Updated weights for policy 1, policy_version 1285238 (0.0011) [2023-12-27 14:33:36,625][20347] Updated weights for policy 0, policy_version 1286134 (0.0009) [2023-12-27 14:33:36,680][20347] Updated weights for policy 0, policy_version 1286144 (0.0007) [2023-12-27 14:33:36,740][20347] Updated weights for policy 0, policy_version 1286154 (0.0011) [2023-12-27 14:33:36,977][20381] Updated weights for policy 1, policy_version 1285248 (0.0006) [2023-12-27 14:33:37,029][20381] Updated weights for policy 1, policy_version 1285258 (0.0005) [2023-12-27 14:33:37,084][20381] Updated weights for policy 1, policy_version 1285268 (0.0009) [2023-12-27 14:33:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 658374656. Throughput: 0: 10293.7, 1: 10424.2. Samples: 658344384. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:37,363][19170] Avg episode reward: [(0, '941.600'), (1, '823.510')] [2023-12-27 14:33:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001285272_329056256.pth... [2023-12-27 14:33:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001286160_329318400.pth... [2023-12-27 14:33:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001284056_328744960.pth [2023-12-27 14:33:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001284976_329015296.pth [2023-12-27 14:33:37,447][20347] Updated weights for policy 0, policy_version 1286164 (0.0009) [2023-12-27 14:33:37,506][20347] Updated weights for policy 0, policy_version 1286174 (0.0007) [2023-12-27 14:33:37,558][20347] Updated weights for policy 0, policy_version 1286184 (0.0007) [2023-12-27 14:33:37,732][20381] Updated weights for policy 1, policy_version 1285278 (0.0010) [2023-12-27 14:33:37,788][20381] Updated weights for policy 1, policy_version 1285288 (0.0010) [2023-12-27 14:33:37,845][20381] Updated weights for policy 1, policy_version 1285298 (0.0011) [2023-12-27 14:33:38,267][20347] Updated weights for policy 0, policy_version 1286194 (0.0007) [2023-12-27 14:33:38,319][20347] Updated weights for policy 0, policy_version 1286204 (0.0008) [2023-12-27 14:33:38,387][20347] Updated weights for policy 0, policy_version 1286214 (0.0008) [2023-12-27 14:33:38,447][20347] Updated weights for policy 0, policy_version 1286224 (0.0007) [2023-12-27 14:33:38,541][20381] Updated weights for policy 1, policy_version 1285308 (0.0011) [2023-12-27 14:33:38,595][20381] Updated weights for policy 1, policy_version 1285318 (0.0011) [2023-12-27 14:33:38,648][20381] Updated weights for policy 1, policy_version 1285328 (0.0011) [2023-12-27 14:33:39,158][20347] Updated weights for policy 0, policy_version 1286234 (0.0007) [2023-12-27 14:33:39,222][20347] Updated weights for policy 0, policy_version 1286244 (0.0008) [2023-12-27 14:33:39,287][20347] Updated weights for policy 0, policy_version 1286254 (0.0007) [2023-12-27 14:33:39,368][20381] Updated weights for policy 1, policy_version 1285338 (0.0010) [2023-12-27 14:33:39,434][20381] Updated weights for policy 1, policy_version 1285348 (0.0009) [2023-12-27 14:33:39,483][20381] Updated weights for policy 1, policy_version 1285358 (0.0008) [2023-12-27 14:33:39,539][20381] Updated weights for policy 1, policy_version 1285368 (0.0008) [2023-12-27 14:33:39,928][20347] Updated weights for policy 0, policy_version 1286264 (0.0008) [2023-12-27 14:33:39,979][20347] Updated weights for policy 0, policy_version 1286274 (0.0007) [2023-12-27 14:33:40,026][20347] Updated weights for policy 0, policy_version 1286284 (0.0008) [2023-12-27 14:33:40,245][20381] Updated weights for policy 1, policy_version 1285378 (0.0006) [2023-12-27 14:33:40,300][20381] Updated weights for policy 1, policy_version 1285388 (0.0010) [2023-12-27 14:33:40,364][20381] Updated weights for policy 1, policy_version 1285398 (0.0010) [2023-12-27 14:33:40,751][20347] Updated weights for policy 0, policy_version 1286294 (0.0008) [2023-12-27 14:33:40,797][20347] Updated weights for policy 0, policy_version 1286304 (0.0008) [2023-12-27 14:33:40,847][20347] Updated weights for policy 0, policy_version 1286314 (0.0007) [2023-12-27 14:33:41,051][20381] Updated weights for policy 1, policy_version 1285408 (0.0009) [2023-12-27 14:33:41,111][20381] Updated weights for policy 1, policy_version 1285418 (0.0007) [2023-12-27 14:33:41,174][20381] Updated weights for policy 1, policy_version 1285428 (0.0008) [2023-12-27 14:33:41,626][20347] Updated weights for policy 0, policy_version 1286324 (0.0007) [2023-12-27 14:33:41,688][20347] Updated weights for policy 0, policy_version 1286334 (0.0008) [2023-12-27 14:33:41,752][20347] Updated weights for policy 0, policy_version 1286344 (0.0008) [2023-12-27 14:33:41,902][20381] Updated weights for policy 1, policy_version 1285438 (0.0006) [2023-12-27 14:33:41,964][20381] Updated weights for policy 1, policy_version 1285448 (0.0007) [2023-12-27 14:33:42,028][20381] Updated weights for policy 1, policy_version 1285458 (0.0008) [2023-12-27 14:33:42,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 658472960. Throughput: 0: 10249.4, 1: 10404.0. Samples: 658465560. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:42,363][19170] Avg episode reward: [(0, '931.570'), (1, '872.580')] [2023-12-27 14:33:42,504][20347] Updated weights for policy 0, policy_version 1286354 (0.0008) [2023-12-27 14:33:42,566][20347] Updated weights for policy 0, policy_version 1286364 (0.0009) [2023-12-27 14:33:42,626][20347] Updated weights for policy 0, policy_version 1286374 (0.0007) [2023-12-27 14:33:42,686][20347] Updated weights for policy 0, policy_version 1286384 (0.0007) [2023-12-27 14:33:42,734][20381] Updated weights for policy 1, policy_version 1285468 (0.0007) [2023-12-27 14:33:42,798][20381] Updated weights for policy 1, policy_version 1285478 (0.0009) [2023-12-27 14:33:42,856][20381] Updated weights for policy 1, policy_version 1285488 (0.0007) [2023-12-27 14:33:43,359][20347] Updated weights for policy 0, policy_version 1286394 (0.0008) [2023-12-27 14:33:43,407][20347] Updated weights for policy 0, policy_version 1286404 (0.0010) [2023-12-27 14:33:43,451][20347] Updated weights for policy 0, policy_version 1286414 (0.0010) [2023-12-27 14:33:43,468][20381] Updated weights for policy 1, policy_version 1285498 (0.0006) [2023-12-27 14:33:43,516][20381] Updated weights for policy 1, policy_version 1285508 (0.0011) [2023-12-27 14:33:43,568][20381] Updated weights for policy 1, policy_version 1285518 (0.0010) [2023-12-27 14:33:43,625][20381] Updated weights for policy 1, policy_version 1285528 (0.0010) [2023-12-27 14:33:44,146][20347] Updated weights for policy 0, policy_version 1286424 (0.0008) [2023-12-27 14:33:44,205][20347] Updated weights for policy 0, policy_version 1286434 (0.0007) [2023-12-27 14:33:44,264][20347] Updated weights for policy 0, policy_version 1286444 (0.0007) [2023-12-27 14:33:44,296][20381] Updated weights for policy 1, policy_version 1285538 (0.0010) [2023-12-27 14:33:44,351][20381] Updated weights for policy 1, policy_version 1285548 (0.0011) [2023-12-27 14:33:44,360][20292] KL-divergence is very high: 206.2781 [2023-12-27 14:33:44,403][20381] Updated weights for policy 1, policy_version 1285558 (0.0010) [2023-12-27 14:33:44,404][20292] KL-divergence is very high: 189.7463 [2023-12-27 14:33:44,943][20347] Updated weights for policy 0, policy_version 1286454 (0.0007) [2023-12-27 14:33:45,000][20347] Updated weights for policy 0, policy_version 1286464 (0.0007) [2023-12-27 14:33:45,063][20347] Updated weights for policy 0, policy_version 1286474 (0.0007) [2023-12-27 14:33:45,143][20381] Updated weights for policy 1, policy_version 1285568 (0.0008) [2023-12-27 14:33:45,203][20381] Updated weights for policy 1, policy_version 1285578 (0.0008) [2023-12-27 14:33:45,264][20381] Updated weights for policy 1, policy_version 1285588 (0.0008) [2023-12-27 14:33:45,708][20347] Updated weights for policy 0, policy_version 1286484 (0.0008) [2023-12-27 14:33:45,763][20347] Updated weights for policy 0, policy_version 1286494 (0.0010) [2023-12-27 14:33:45,816][20347] Updated weights for policy 0, policy_version 1286504 (0.0010) [2023-12-27 14:33:45,918][20381] Updated weights for policy 1, policy_version 1285598 (0.0008) [2023-12-27 14:33:45,974][20381] Updated weights for policy 1, policy_version 1285608 (0.0006) [2023-12-27 14:33:45,998][20292] KL-divergence is very high: 266.6753 [2023-12-27 14:33:46,017][20292] KL-divergence is very high: 282.3392 [2023-12-27 14:33:46,027][20381] Updated weights for policy 1, policy_version 1285618 (0.0005) [2023-12-27 14:33:46,044][20292] KL-divergence is very high: 192.1204 [2023-12-27 14:33:46,439][20347] Updated weights for policy 0, policy_version 1286514 (0.0009) [2023-12-27 14:33:46,492][20347] Updated weights for policy 0, policy_version 1286524 (0.0005) [2023-12-27 14:33:46,536][20347] Updated weights for policy 0, policy_version 1286534 (0.0005) [2023-12-27 14:33:46,582][20381] Updated weights for policy 1, policy_version 1285628 (0.0006) [2023-12-27 14:33:46,586][20347] Updated weights for policy 0, policy_version 1286544 (0.0005) [2023-12-27 14:33:46,634][20381] Updated weights for policy 1, policy_version 1285638 (0.0007) [2023-12-27 14:33:46,678][20381] Updated weights for policy 1, policy_version 1285648 (0.0008) [2023-12-27 14:33:47,186][20347] Updated weights for policy 0, policy_version 1286554 (0.0010) [2023-12-27 14:33:47,246][20347] Updated weights for policy 0, policy_version 1286564 (0.0009) [2023-12-27 14:33:47,294][20347] Updated weights for policy 0, policy_version 1286574 (0.0007) [2023-12-27 14:33:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20438.4). Total num frames: 658579456. Throughput: 0: 10308.6, 1: 10365.9. Samples: 658589384. Policy #0 lag: (min: 12.0, avg: 14.9, max: 34.0) [2023-12-27 14:33:47,363][19170] Avg episode reward: [(0, '941.240'), (1, '853.850')] [2023-12-27 14:33:47,366][20381] Updated weights for policy 1, policy_version 1285658 (0.0006) [2023-12-27 14:33:47,419][20381] Updated weights for policy 1, policy_version 1285668 (0.0005) [2023-12-27 14:33:47,471][20381] Updated weights for policy 1, policy_version 1285678 (0.0005) [2023-12-27 14:33:47,519][20381] Updated weights for policy 1, policy_version 1285688 (0.0006) [2023-12-27 14:33:47,898][20347] Updated weights for policy 0, policy_version 1286584 (0.0010) [2023-12-27 14:33:47,953][20347] Updated weights for policy 0, policy_version 1286594 (0.0006) [2023-12-27 14:33:48,015][20347] Updated weights for policy 0, policy_version 1286604 (0.0005) [2023-12-27 14:33:48,138][20381] Updated weights for policy 1, policy_version 1285698 (0.0008) [2023-12-27 14:33:48,191][20381] Updated weights for policy 1, policy_version 1285708 (0.0007) [2023-12-27 14:33:48,239][20381] Updated weights for policy 1, policy_version 1285718 (0.0007) [2023-12-27 14:33:48,669][20347] Updated weights for policy 0, policy_version 1286614 (0.0008) [2023-12-27 14:33:48,718][20347] Updated weights for policy 0, policy_version 1286624 (0.0010) [2023-12-27 14:33:48,770][20347] Updated weights for policy 0, policy_version 1286634 (0.0009) [2023-12-27 14:33:48,960][20381] Updated weights for policy 1, policy_version 1285728 (0.0006) [2023-12-27 14:33:49,017][20381] Updated weights for policy 1, policy_version 1285738 (0.0005) [2023-12-27 14:33:49,075][20381] Updated weights for policy 1, policy_version 1285748 (0.0005) [2023-12-27 14:33:49,512][20347] Updated weights for policy 0, policy_version 1286644 (0.0009) [2023-12-27 14:33:49,586][20347] Updated weights for policy 0, policy_version 1286654 (0.0010) [2023-12-27 14:33:49,639][20347] Updated weights for policy 0, policy_version 1286664 (0.0010) [2023-12-27 14:33:49,710][20381] Updated weights for policy 1, policy_version 1285758 (0.0007) [2023-12-27 14:33:49,765][20381] Updated weights for policy 1, policy_version 1285768 (0.0008) [2023-12-27 14:33:49,827][20381] Updated weights for policy 1, policy_version 1285778 (0.0008) [2023-12-27 14:33:50,374][20347] Updated weights for policy 0, policy_version 1286674 (0.0009) [2023-12-27 14:33:50,426][20347] Updated weights for policy 0, policy_version 1286684 (0.0006) [2023-12-27 14:33:50,481][20347] Updated weights for policy 0, policy_version 1286694 (0.0006) [2023-12-27 14:33:50,534][20347] Updated weights for policy 0, policy_version 1286704 (0.0007) [2023-12-27 14:33:50,601][20381] Updated weights for policy 1, policy_version 1285788 (0.0008) [2023-12-27 14:33:50,656][20381] Updated weights for policy 1, policy_version 1285798 (0.0008) [2023-12-27 14:33:50,709][20381] Updated weights for policy 1, policy_version 1285808 (0.0007) [2023-12-27 14:33:51,256][20347] Updated weights for policy 0, policy_version 1286714 (0.0007) [2023-12-27 14:33:51,314][20347] Updated weights for policy 0, policy_version 1286724 (0.0007) [2023-12-27 14:33:51,382][20347] Updated weights for policy 0, policy_version 1286734 (0.0008) [2023-12-27 14:33:51,440][20381] Updated weights for policy 1, policy_version 1285818 (0.0007) [2023-12-27 14:33:51,496][20381] Updated weights for policy 1, policy_version 1285828 (0.0008) [2023-12-27 14:33:51,552][20381] Updated weights for policy 1, policy_version 1285838 (0.0007) [2023-12-27 14:33:51,609][20381] Updated weights for policy 1, policy_version 1285848 (0.0007) [2023-12-27 14:33:52,127][20347] Updated weights for policy 0, policy_version 1286744 (0.0007) [2023-12-27 14:33:52,189][20347] Updated weights for policy 0, policy_version 1286754 (0.0008) [2023-12-27 14:33:52,248][20347] Updated weights for policy 0, policy_version 1286764 (0.0006) [2023-12-27 14:33:52,319][20381] Updated weights for policy 1, policy_version 1285858 (0.0007) [2023-12-27 14:33:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 658677760. Throughput: 0: 10277.1, 1: 10383.7. Samples: 658650720. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:33:52,363][19170] Avg episode reward: [(0, '960.840'), (1, '834.240')] [2023-12-27 14:33:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001286768_329474048.pth... [2023-12-27 14:33:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001285552_329162752.pth [2023-12-27 14:33:52,385][20381] Updated weights for policy 1, policy_version 1285868 (0.0008) [2023-12-27 14:33:52,453][20381] Updated weights for policy 1, policy_version 1285878 (0.0005) [2023-12-27 14:33:52,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001285880_329211904.pth... [2023-12-27 14:33:52,472][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001284664_328900608.pth [2023-12-27 14:33:52,909][20347] Updated weights for policy 0, policy_version 1286774 (0.0007) [2023-12-27 14:33:52,956][20347] Updated weights for policy 0, policy_version 1286784 (0.0007) [2023-12-27 14:33:53,010][20347] Updated weights for policy 0, policy_version 1286794 (0.0008) [2023-12-27 14:33:53,078][20381] Updated weights for policy 1, policy_version 1285888 (0.0006) [2023-12-27 14:33:53,129][20381] Updated weights for policy 1, policy_version 1285898 (0.0006) [2023-12-27 14:33:53,179][20381] Updated weights for policy 1, policy_version 1285908 (0.0007) [2023-12-27 14:33:53,738][20347] Updated weights for policy 0, policy_version 1286804 (0.0007) [2023-12-27 14:33:53,792][20347] Updated weights for policy 0, policy_version 1286814 (0.0008) [2023-12-27 14:33:53,809][20381] Updated weights for policy 1, policy_version 1285918 (0.0006) [2023-12-27 14:33:53,841][20347] Updated weights for policy 0, policy_version 1286824 (0.0007) [2023-12-27 14:33:53,865][20381] Updated weights for policy 1, policy_version 1285928 (0.0006) [2023-12-27 14:33:53,925][20381] Updated weights for policy 1, policy_version 1285938 (0.0007) [2023-12-27 14:33:54,560][20347] Updated weights for policy 0, policy_version 1286834 (0.0007) [2023-12-27 14:33:54,599][20381] Updated weights for policy 1, policy_version 1285948 (0.0008) [2023-12-27 14:33:54,615][20347] Updated weights for policy 0, policy_version 1286844 (0.0008) [2023-12-27 14:33:54,643][20381] Updated weights for policy 1, policy_version 1285958 (0.0006) [2023-12-27 14:33:54,670][20347] Updated weights for policy 0, policy_version 1286854 (0.0007) [2023-12-27 14:33:54,699][20381] Updated weights for policy 1, policy_version 1285968 (0.0006) [2023-12-27 14:33:54,733][20347] Updated weights for policy 0, policy_version 1286864 (0.0007) [2023-12-27 14:33:55,321][20381] Updated weights for policy 1, policy_version 1285978 (0.0006) [2023-12-27 14:33:55,376][20381] Updated weights for policy 1, policy_version 1285988 (0.0008) [2023-12-27 14:33:55,426][20381] Updated weights for policy 1, policy_version 1285998 (0.0006) [2023-12-27 14:33:55,427][20347] Updated weights for policy 0, policy_version 1286874 (0.0007) [2023-12-27 14:33:55,481][20381] Updated weights for policy 1, policy_version 1286008 (0.0005) [2023-12-27 14:33:55,483][20347] Updated weights for policy 0, policy_version 1286884 (0.0006) [2023-12-27 14:33:55,544][20347] Updated weights for policy 0, policy_version 1286894 (0.0007) [2023-12-27 14:33:56,166][20381] Updated weights for policy 1, policy_version 1286018 (0.0007) [2023-12-27 14:33:56,219][20381] Updated weights for policy 1, policy_version 1286028 (0.0007) [2023-12-27 14:33:56,229][20347] Updated weights for policy 0, policy_version 1286904 (0.0006) [2023-12-27 14:33:56,275][20381] Updated weights for policy 1, policy_version 1286038 (0.0005) [2023-12-27 14:33:56,288][20347] Updated weights for policy 0, policy_version 1286914 (0.0006) [2023-12-27 14:33:56,346][20347] Updated weights for policy 0, policy_version 1286924 (0.0007) [2023-12-27 14:33:56,926][20381] Updated weights for policy 1, policy_version 1286048 (0.0007) [2023-12-27 14:33:56,984][20381] Updated weights for policy 1, policy_version 1286058 (0.0008) [2023-12-27 14:33:57,036][20381] Updated weights for policy 1, policy_version 1286068 (0.0008) [2023-12-27 14:33:57,042][20347] Updated weights for policy 0, policy_version 1286934 (0.0006) [2023-12-27 14:33:57,097][20347] Updated weights for policy 0, policy_version 1286944 (0.0005) [2023-12-27 14:33:57,142][20347] Updated weights for policy 0, policy_version 1286954 (0.0005) [2023-12-27 14:33:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 658784256. Throughput: 0: 10174.1, 1: 10375.4. Samples: 658771372. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:33:57,363][19170] Avg episode reward: [(0, '931.690'), (1, '775.620')] [2023-12-27 14:33:57,751][20381] Updated weights for policy 1, policy_version 1286078 (0.0007) [2023-12-27 14:33:57,767][20347] Updated weights for policy 0, policy_version 1286964 (0.0006) [2023-12-27 14:33:57,803][20381] Updated weights for policy 1, policy_version 1286088 (0.0005) [2023-12-27 14:33:57,804][20292] KL-divergence is very high: 121.0713 [2023-12-27 14:33:57,822][20347] Updated weights for policy 0, policy_version 1286974 (0.0007) [2023-12-27 14:33:57,851][20381] Updated weights for policy 1, policy_version 1286098 (0.0005) [2023-12-27 14:33:57,878][20347] Updated weights for policy 0, policy_version 1286984 (0.0006) [2023-12-27 14:33:58,566][20381] Updated weights for policy 1, policy_version 1286108 (0.0005) [2023-12-27 14:33:58,609][20347] Updated weights for policy 0, policy_version 1286994 (0.0007) [2023-12-27 14:33:58,627][20381] Updated weights for policy 1, policy_version 1286118 (0.0008) [2023-12-27 14:33:58,670][20347] Updated weights for policy 0, policy_version 1287004 (0.0008) [2023-12-27 14:33:58,690][20381] Updated weights for policy 1, policy_version 1286128 (0.0008) [2023-12-27 14:33:58,738][20347] Updated weights for policy 0, policy_version 1287015 (0.0008) [2023-12-27 14:33:59,427][20381] Updated weights for policy 1, policy_version 1286138 (0.0006) [2023-12-27 14:33:59,488][20381] Updated weights for policy 1, policy_version 1286148 (0.0007) [2023-12-27 14:33:59,496][20347] Updated weights for policy 0, policy_version 1287025 (0.0009) [2023-12-27 14:33:59,549][20381] Updated weights for policy 1, policy_version 1286158 (0.0006) [2023-12-27 14:33:59,553][20347] Updated weights for policy 0, policy_version 1287035 (0.0008) [2023-12-27 14:33:59,606][20381] Updated weights for policy 1, policy_version 1286168 (0.0009) [2023-12-27 14:33:59,613][20347] Updated weights for policy 0, policy_version 1287045 (0.0009) [2023-12-27 14:33:59,672][20347] Updated weights for policy 0, policy_version 1287055 (0.0008) [2023-12-27 14:34:00,281][20381] Updated weights for policy 1, policy_version 1286178 (0.0007) [2023-12-27 14:34:00,337][20381] Updated weights for policy 1, policy_version 1286188 (0.0007) [2023-12-27 14:34:00,397][20381] Updated weights for policy 1, policy_version 1286198 (0.0006) [2023-12-27 14:34:00,436][20347] Updated weights for policy 0, policy_version 1287065 (0.0010) [2023-12-27 14:34:00,481][20347] Updated weights for policy 0, policy_version 1287075 (0.0010) [2023-12-27 14:34:00,525][20347] Updated weights for policy 0, policy_version 1287085 (0.0010) [2023-12-27 14:34:01,053][20381] Updated weights for policy 1, policy_version 1286208 (0.0007) [2023-12-27 14:34:01,107][20381] Updated weights for policy 1, policy_version 1286218 (0.0007) [2023-12-27 14:34:01,168][20381] Updated weights for policy 1, policy_version 1286228 (0.0007) [2023-12-27 14:34:01,228][20347] Updated weights for policy 0, policy_version 1287095 (0.0009) [2023-12-27 14:34:01,290][20347] Updated weights for policy 0, policy_version 1287105 (0.0007) [2023-12-27 14:34:01,338][20347] Updated weights for policy 0, policy_version 1287115 (0.0007) [2023-12-27 14:34:01,864][20381] Updated weights for policy 1, policy_version 1286238 (0.0006) [2023-12-27 14:34:01,918][20381] Updated weights for policy 1, policy_version 1286248 (0.0006) [2023-12-27 14:34:01,970][20381] Updated weights for policy 1, policy_version 1286258 (0.0007) [2023-12-27 14:34:02,084][20347] Updated weights for policy 0, policy_version 1287125 (0.0007) [2023-12-27 14:34:02,153][20347] Updated weights for policy 0, policy_version 1287135 (0.0007) [2023-12-27 14:34:02,208][20347] Updated weights for policy 0, policy_version 1287145 (0.0007) [2023-12-27 14:34:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 658882560. Throughput: 0: 10144.7, 1: 10317.3. Samples: 658890140. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:02,363][19170] Avg episode reward: [(0, '941.570'), (1, '804.210')] [2023-12-27 14:34:02,639][20381] Updated weights for policy 1, policy_version 1286268 (0.0006) [2023-12-27 14:34:02,701][20381] Updated weights for policy 1, policy_version 1286278 (0.0008) [2023-12-27 14:34:02,762][20381] Updated weights for policy 1, policy_version 1286288 (0.0008) [2023-12-27 14:34:02,836][20347] Updated weights for policy 0, policy_version 1287155 (0.0007) [2023-12-27 14:34:02,883][20347] Updated weights for policy 0, policy_version 1287165 (0.0007) [2023-12-27 14:34:02,938][20347] Updated weights for policy 0, policy_version 1287175 (0.0007) [2023-12-27 14:34:03,409][20381] Updated weights for policy 1, policy_version 1286298 (0.0006) [2023-12-27 14:34:03,457][20381] Updated weights for policy 1, policy_version 1286308 (0.0007) [2023-12-27 14:34:03,502][20381] Updated weights for policy 1, policy_version 1286318 (0.0007) [2023-12-27 14:34:03,554][20381] Updated weights for policy 1, policy_version 1286328 (0.0006) [2023-12-27 14:34:03,640][20347] Updated weights for policy 0, policy_version 1287185 (0.0007) [2023-12-27 14:34:03,694][20347] Updated weights for policy 0, policy_version 1287195 (0.0007) [2023-12-27 14:34:03,751][20347] Updated weights for policy 0, policy_version 1287205 (0.0007) [2023-12-27 14:34:03,807][20347] Updated weights for policy 0, policy_version 1287215 (0.0007) [2023-12-27 14:34:04,202][20381] Updated weights for policy 1, policy_version 1286338 (0.0008) [2023-12-27 14:34:04,261][20381] Updated weights for policy 1, policy_version 1286348 (0.0008) [2023-12-27 14:34:04,324][20381] Updated weights for policy 1, policy_version 1286358 (0.0008) [2023-12-27 14:34:04,542][20347] Updated weights for policy 0, policy_version 1287225 (0.0008) [2023-12-27 14:34:04,597][20347] Updated weights for policy 0, policy_version 1287235 (0.0007) [2023-12-27 14:34:04,651][20347] Updated weights for policy 0, policy_version 1287245 (0.0008) [2023-12-27 14:34:04,975][20381] Updated weights for policy 1, policy_version 1286368 (0.0006) [2023-12-27 14:34:05,026][20381] Updated weights for policy 1, policy_version 1286378 (0.0005) [2023-12-27 14:34:05,084][20381] Updated weights for policy 1, policy_version 1286388 (0.0005) [2023-12-27 14:34:05,388][20347] Updated weights for policy 0, policy_version 1287255 (0.0008) [2023-12-27 14:34:05,436][20347] Updated weights for policy 0, policy_version 1287265 (0.0007) [2023-12-27 14:34:05,481][20347] Updated weights for policy 0, policy_version 1287275 (0.0007) [2023-12-27 14:34:05,636][20381] Updated weights for policy 1, policy_version 1286398 (0.0007) [2023-12-27 14:34:05,682][20381] Updated weights for policy 1, policy_version 1286408 (0.0008) [2023-12-27 14:34:05,727][20381] Updated weights for policy 1, policy_version 1286418 (0.0010) [2023-12-27 14:34:06,215][20347] Updated weights for policy 0, policy_version 1287285 (0.0008) [2023-12-27 14:34:06,269][20347] Updated weights for policy 0, policy_version 1287295 (0.0008) [2023-12-27 14:34:06,327][20347] Updated weights for policy 0, policy_version 1287305 (0.0008) [2023-12-27 14:34:06,433][20381] Updated weights for policy 1, policy_version 1286428 (0.0009) [2023-12-27 14:34:06,503][20381] Updated weights for policy 1, policy_version 1286438 (0.0011) [2023-12-27 14:34:06,565][20381] Updated weights for policy 1, policy_version 1286448 (0.0010) [2023-12-27 14:34:07,021][20347] Updated weights for policy 0, policy_version 1287315 (0.0008) [2023-12-27 14:34:07,078][20347] Updated weights for policy 0, policy_version 1287325 (0.0006) [2023-12-27 14:34:07,131][20347] Updated weights for policy 0, policy_version 1287335 (0.0008) [2023-12-27 14:34:07,142][20381] Updated weights for policy 1, policy_version 1286458 (0.0006) [2023-12-27 14:34:07,195][20381] Updated weights for policy 1, policy_version 1286468 (0.0009) [2023-12-27 14:34:07,253][20381] Updated weights for policy 1, policy_version 1286478 (0.0005) [2023-12-27 14:34:07,315][20381] Updated weights for policy 1, policy_version 1286488 (0.0010) [2023-12-27 14:34:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 658989056. Throughput: 0: 10137.9, 1: 10368.9. Samples: 658952444. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:07,363][19170] Avg episode reward: [(0, '960.950'), (1, '775.810')] [2023-12-27 14:34:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001286488_329367552.pth... [2023-12-27 14:34:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001287344_329621504.pth... [2023-12-27 14:34:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001286160_329318400.pth [2023-12-27 14:34:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001285272_329056256.pth [2023-12-27 14:34:07,802][20347] Updated weights for policy 0, policy_version 1287345 (0.0007) [2023-12-27 14:34:07,850][20347] Updated weights for policy 0, policy_version 1287355 (0.0005) [2023-12-27 14:34:07,869][20381] Updated weights for policy 1, policy_version 1286498 (0.0010) [2023-12-27 14:34:07,898][20347] Updated weights for policy 0, policy_version 1287365 (0.0008) [2023-12-27 14:34:07,933][20381] Updated weights for policy 1, policy_version 1286508 (0.0005) [2023-12-27 14:34:07,949][20347] Updated weights for policy 0, policy_version 1287375 (0.0007) [2023-12-27 14:34:07,997][20381] Updated weights for policy 1, policy_version 1286518 (0.0008) [2023-12-27 14:34:08,579][20381] Updated weights for policy 1, policy_version 1286528 (0.0005) [2023-12-27 14:34:08,633][20381] Updated weights for policy 1, policy_version 1286538 (0.0006) [2023-12-27 14:34:08,695][20381] Updated weights for policy 1, policy_version 1286548 (0.0011) [2023-12-27 14:34:08,742][20347] Updated weights for policy 0, policy_version 1287385 (0.0006) [2023-12-27 14:34:08,805][20347] Updated weights for policy 0, policy_version 1287395 (0.0007) [2023-12-27 14:34:08,836][20291] KL-divergence is very high: 121.7991 [2023-12-27 14:34:08,848][20291] KL-divergence is very high: 131.9874 [2023-12-27 14:34:08,866][20347] Updated weights for policy 0, policy_version 1287405 (0.0008) [2023-12-27 14:34:09,365][20381] Updated weights for policy 1, policy_version 1286558 (0.0007) [2023-12-27 14:34:09,427][20381] Updated weights for policy 1, policy_version 1286568 (0.0008) [2023-12-27 14:34:09,484][20381] Updated weights for policy 1, policy_version 1286578 (0.0008) [2023-12-27 14:34:09,503][20347] Updated weights for policy 0, policy_version 1287415 (0.0008) [2023-12-27 14:34:09,558][20347] Updated weights for policy 0, policy_version 1287425 (0.0007) [2023-12-27 14:34:09,611][20347] Updated weights for policy 0, policy_version 1287435 (0.0008) [2023-12-27 14:34:10,243][20381] Updated weights for policy 1, policy_version 1286588 (0.0008) [2023-12-27 14:34:10,310][20381] Updated weights for policy 1, policy_version 1286598 (0.0007) [2023-12-27 14:34:10,348][20347] Updated weights for policy 0, policy_version 1287445 (0.0008) [2023-12-27 14:34:10,376][20381] Updated weights for policy 1, policy_version 1286608 (0.0008) [2023-12-27 14:34:10,407][20347] Updated weights for policy 0, policy_version 1287455 (0.0009) [2023-12-27 14:34:10,471][20347] Updated weights for policy 0, policy_version 1287465 (0.0009) [2023-12-27 14:34:11,056][20381] Updated weights for policy 1, policy_version 1286618 (0.0007) [2023-12-27 14:34:11,109][20381] Updated weights for policy 1, policy_version 1286628 (0.0006) [2023-12-27 14:34:11,178][20381] Updated weights for policy 1, policy_version 1286638 (0.0008) [2023-12-27 14:34:11,209][20347] Updated weights for policy 0, policy_version 1287475 (0.0008) [2023-12-27 14:34:11,241][20381] Updated weights for policy 1, policy_version 1286648 (0.0006) [2023-12-27 14:34:11,264][20347] Updated weights for policy 0, policy_version 1287485 (0.0010) [2023-12-27 14:34:11,329][20347] Updated weights for policy 0, policy_version 1287495 (0.0010) [2023-12-27 14:34:11,913][20381] Updated weights for policy 1, policy_version 1286658 (0.0008) [2023-12-27 14:34:11,974][20381] Updated weights for policy 1, policy_version 1286668 (0.0009) [2023-12-27 14:34:11,977][20347] Updated weights for policy 0, policy_version 1287505 (0.0008) [2023-12-27 14:34:12,035][20381] Updated weights for policy 1, policy_version 1286678 (0.0007) [2023-12-27 14:34:12,042][20347] Updated weights for policy 0, policy_version 1287515 (0.0006) [2023-12-27 14:34:12,091][20347] Updated weights for policy 0, policy_version 1287525 (0.0007) [2023-12-27 14:34:12,139][20347] Updated weights for policy 0, policy_version 1287535 (0.0007) [2023-12-27 14:34:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 659087360. Throughput: 0: 10091.6, 1: 10421.2. Samples: 659075104. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:12,363][19170] Avg episode reward: [(0, '951.100'), (1, '844.380')] [2023-12-27 14:34:12,751][20381] Updated weights for policy 1, policy_version 1286688 (0.0007) [2023-12-27 14:34:12,818][20381] Updated weights for policy 1, policy_version 1286698 (0.0007) [2023-12-27 14:34:12,883][20381] Updated weights for policy 1, policy_version 1286708 (0.0006) [2023-12-27 14:34:12,919][20347] Updated weights for policy 0, policy_version 1287545 (0.0010) [2023-12-27 14:34:12,964][20347] Updated weights for policy 0, policy_version 1287555 (0.0010) [2023-12-27 14:34:13,017][20347] Updated weights for policy 0, policy_version 1287565 (0.0010) [2023-12-27 14:34:13,522][20381] Updated weights for policy 1, policy_version 1286718 (0.0007) [2023-12-27 14:34:13,573][20381] Updated weights for policy 1, policy_version 1286728 (0.0006) [2023-12-27 14:34:13,632][20381] Updated weights for policy 1, policy_version 1286738 (0.0007) [2023-12-27 14:34:13,674][20347] Updated weights for policy 0, policy_version 1287575 (0.0009) [2023-12-27 14:34:13,732][20347] Updated weights for policy 0, policy_version 1287585 (0.0007) [2023-12-27 14:34:13,791][20347] Updated weights for policy 0, policy_version 1287595 (0.0007) [2023-12-27 14:34:14,201][20381] Updated weights for policy 1, policy_version 1286748 (0.0007) [2023-12-27 14:34:14,265][20381] Updated weights for policy 1, policy_version 1286758 (0.0005) [2023-12-27 14:34:14,325][20381] Updated weights for policy 1, policy_version 1286768 (0.0006) [2023-12-27 14:34:14,444][20347] Updated weights for policy 0, policy_version 1287605 (0.0008) [2023-12-27 14:34:14,502][20347] Updated weights for policy 0, policy_version 1287615 (0.0007) [2023-12-27 14:34:14,558][20347] Updated weights for policy 0, policy_version 1287625 (0.0007) [2023-12-27 14:34:14,987][20381] Updated weights for policy 1, policy_version 1286778 (0.0010) [2023-12-27 14:34:15,053][20381] Updated weights for policy 1, policy_version 1286788 (0.0008) [2023-12-27 14:34:15,115][20381] Updated weights for policy 1, policy_version 1286798 (0.0008) [2023-12-27 14:34:15,170][20381] Updated weights for policy 1, policy_version 1286808 (0.0006) [2023-12-27 14:34:15,277][20347] Updated weights for policy 0, policy_version 1287635 (0.0007) [2023-12-27 14:34:15,340][20347] Updated weights for policy 0, policy_version 1287645 (0.0008) [2023-12-27 14:34:15,394][20347] Updated weights for policy 0, policy_version 1287655 (0.0008) [2023-12-27 14:34:15,814][20381] Updated weights for policy 1, policy_version 1286818 (0.0007) [2023-12-27 14:34:15,871][20381] Updated weights for policy 1, policy_version 1286828 (0.0008) [2023-12-27 14:34:15,930][20381] Updated weights for policy 1, policy_version 1286838 (0.0008) [2023-12-27 14:34:16,041][20347] Updated weights for policy 0, policy_version 1287665 (0.0007) [2023-12-27 14:34:16,092][20347] Updated weights for policy 0, policy_version 1287675 (0.0005) [2023-12-27 14:34:16,148][20347] Updated weights for policy 0, policy_version 1287685 (0.0005) [2023-12-27 14:34:16,195][20347] Updated weights for policy 0, policy_version 1287695 (0.0005) [2023-12-27 14:34:16,556][20381] Updated weights for policy 1, policy_version 1286848 (0.0006) [2023-12-27 14:34:16,610][20381] Updated weights for policy 1, policy_version 1286858 (0.0006) [2023-12-27 14:34:16,667][20381] Updated weights for policy 1, policy_version 1286868 (0.0008) [2023-12-27 14:34:16,728][20347] Updated weights for policy 0, policy_version 1287705 (0.0010) [2023-12-27 14:34:16,776][20347] Updated weights for policy 0, policy_version 1287715 (0.0010) [2023-12-27 14:34:16,823][20347] Updated weights for policy 0, policy_version 1287725 (0.0010) [2023-12-27 14:34:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 659185664. Throughput: 0: 10079.1, 1: 10383.2. Samples: 659201028. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:17,364][19170] Avg episode reward: [(0, '960.860'), (1, '882.570')] [2023-12-27 14:34:17,375][20381] Updated weights for policy 1, policy_version 1286878 (0.0006) [2023-12-27 14:34:17,413][20347] Updated weights for policy 0, policy_version 1287735 (0.0010) [2023-12-27 14:34:17,429][20381] Updated weights for policy 1, policy_version 1286888 (0.0005) [2023-12-27 14:34:17,473][20347] Updated weights for policy 0, policy_version 1287745 (0.0010) [2023-12-27 14:34:17,488][20381] Updated weights for policy 1, policy_version 1286898 (0.0006) [2023-12-27 14:34:17,523][20347] Updated weights for policy 0, policy_version 1287755 (0.0006) [2023-12-27 14:34:18,131][20347] Updated weights for policy 0, policy_version 1287765 (0.0005) [2023-12-27 14:34:18,169][20381] Updated weights for policy 1, policy_version 1286908 (0.0007) [2023-12-27 14:34:18,183][20347] Updated weights for policy 0, policy_version 1287775 (0.0008) [2023-12-27 14:34:18,221][20381] Updated weights for policy 1, policy_version 1286918 (0.0006) [2023-12-27 14:34:18,232][20347] Updated weights for policy 0, policy_version 1287785 (0.0010) [2023-12-27 14:34:18,275][20381] Updated weights for policy 1, policy_version 1286928 (0.0007) [2023-12-27 14:34:18,839][20347] Updated weights for policy 0, policy_version 1287795 (0.0009) [2023-12-27 14:34:18,891][20347] Updated weights for policy 0, policy_version 1287805 (0.0006) [2023-12-27 14:34:18,953][20347] Updated weights for policy 0, policy_version 1287815 (0.0009) [2023-12-27 14:34:19,011][20381] Updated weights for policy 1, policy_version 1286938 (0.0007) [2023-12-27 14:34:19,055][20381] Updated weights for policy 1, policy_version 1286948 (0.0008) [2023-12-27 14:34:19,106][20381] Updated weights for policy 1, policy_version 1286958 (0.0008) [2023-12-27 14:34:19,153][20381] Updated weights for policy 1, policy_version 1286968 (0.0008) [2023-12-27 14:34:19,605][20347] Updated weights for policy 0, policy_version 1287825 (0.0010) [2023-12-27 14:34:19,661][20347] Updated weights for policy 0, policy_version 1287835 (0.0010) [2023-12-27 14:34:19,720][20347] Updated weights for policy 0, policy_version 1287845 (0.0010) [2023-12-27 14:34:19,785][20347] Updated weights for policy 0, policy_version 1287855 (0.0010) [2023-12-27 14:34:19,928][20381] Updated weights for policy 1, policy_version 1286978 (0.0007) [2023-12-27 14:34:19,986][20381] Updated weights for policy 1, policy_version 1286988 (0.0008) [2023-12-27 14:34:20,040][20381] Updated weights for policy 1, policy_version 1286998 (0.0008) [2023-12-27 14:34:20,464][20347] Updated weights for policy 0, policy_version 1287865 (0.0007) [2023-12-27 14:34:20,534][20347] Updated weights for policy 0, policy_version 1287875 (0.0006) [2023-12-27 14:34:20,600][20347] Updated weights for policy 0, policy_version 1287885 (0.0007) [2023-12-27 14:34:20,728][20381] Updated weights for policy 1, policy_version 1287008 (0.0008) [2023-12-27 14:34:20,782][20381] Updated weights for policy 1, policy_version 1287018 (0.0007) [2023-12-27 14:34:20,839][20381] Updated weights for policy 1, policy_version 1287028 (0.0007) [2023-12-27 14:34:21,244][20347] Updated weights for policy 0, policy_version 1287895 (0.0009) [2023-12-27 14:34:21,307][20347] Updated weights for policy 0, policy_version 1287905 (0.0010) [2023-12-27 14:34:21,375][20347] Updated weights for policy 0, policy_version 1287915 (0.0011) [2023-12-27 14:34:21,546][20381] Updated weights for policy 1, policy_version 1287038 (0.0009) [2023-12-27 14:34:21,611][20381] Updated weights for policy 1, policy_version 1287048 (0.0008) [2023-12-27 14:34:21,679][20381] Updated weights for policy 1, policy_version 1287058 (0.0008) [2023-12-27 14:34:22,068][20347] Updated weights for policy 0, policy_version 1287925 (0.0007) [2023-12-27 14:34:22,127][20347] Updated weights for policy 0, policy_version 1287935 (0.0007) [2023-12-27 14:34:22,187][20347] Updated weights for policy 0, policy_version 1287945 (0.0006) [2023-12-27 14:34:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 659292160. Throughput: 0: 10101.2, 1: 10307.6. Samples: 659262784. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:22,363][19170] Avg episode reward: [(0, '980.400'), (1, '892.360')] [2023-12-27 14:34:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001287952_329777152.pth... [2023-12-27 14:34:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001286768_329474048.pth [2023-12-27 14:34:22,375][20381] Updated weights for policy 1, policy_version 1287068 (0.0008) [2023-12-27 14:34:22,435][20381] Updated weights for policy 1, policy_version 1287078 (0.0007) [2023-12-27 14:34:22,496][20381] Updated weights for policy 1, policy_version 1287088 (0.0007) [2023-12-27 14:34:22,543][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001287096_329523200.pth... [2023-12-27 14:34:22,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001285880_329211904.pth [2023-12-27 14:34:22,878][20347] Updated weights for policy 0, policy_version 1287955 (0.0007) [2023-12-27 14:34:22,936][20347] Updated weights for policy 0, policy_version 1287965 (0.0009) [2023-12-27 14:34:22,990][20347] Updated weights for policy 0, policy_version 1287975 (0.0010) [2023-12-27 14:34:23,213][20381] Updated weights for policy 1, policy_version 1287098 (0.0007) [2023-12-27 14:34:23,278][20381] Updated weights for policy 1, policy_version 1287108 (0.0008) [2023-12-27 14:34:23,342][20381] Updated weights for policy 1, policy_version 1287118 (0.0007) [2023-12-27 14:34:23,400][20381] Updated weights for policy 1, policy_version 1287128 (0.0007) [2023-12-27 14:34:23,666][20347] Updated weights for policy 0, policy_version 1287985 (0.0010) [2023-12-27 14:34:23,716][20347] Updated weights for policy 0, policy_version 1287995 (0.0006) [2023-12-27 14:34:23,781][20347] Updated weights for policy 0, policy_version 1288005 (0.0007) [2023-12-27 14:34:23,832][20347] Updated weights for policy 0, policy_version 1288015 (0.0006) [2023-12-27 14:34:24,089][20381] Updated weights for policy 1, policy_version 1287138 (0.0007) [2023-12-27 14:34:24,151][20381] Updated weights for policy 1, policy_version 1287148 (0.0008) [2023-12-27 14:34:24,206][20381] Updated weights for policy 1, policy_version 1287158 (0.0008) [2023-12-27 14:34:24,492][20347] Updated weights for policy 0, policy_version 1288025 (0.0010) [2023-12-27 14:34:24,554][20347] Updated weights for policy 0, policy_version 1288035 (0.0010) [2023-12-27 14:34:24,612][20347] Updated weights for policy 0, policy_version 1288045 (0.0010) [2023-12-27 14:34:24,946][20381] Updated weights for policy 1, policy_version 1287168 (0.0008) [2023-12-27 14:34:24,994][20381] Updated weights for policy 1, policy_version 1287178 (0.0008) [2023-12-27 14:34:25,046][20381] Updated weights for policy 1, policy_version 1287188 (0.0007) [2023-12-27 14:34:25,244][20347] Updated weights for policy 0, policy_version 1288055 (0.0010) [2023-12-27 14:34:25,301][20347] Updated weights for policy 0, policy_version 1288065 (0.0010) [2023-12-27 14:34:25,356][20347] Updated weights for policy 0, policy_version 1288075 (0.0007) [2023-12-27 14:34:25,737][20381] Updated weights for policy 1, policy_version 1287198 (0.0008) [2023-12-27 14:34:25,798][20381] Updated weights for policy 1, policy_version 1287208 (0.0008) [2023-12-27 14:34:25,848][20381] Updated weights for policy 1, policy_version 1287218 (0.0007) [2023-12-27 14:34:25,975][20347] Updated weights for policy 0, policy_version 1288085 (0.0005) [2023-12-27 14:34:26,021][20347] Updated weights for policy 0, policy_version 1288095 (0.0008) [2023-12-27 14:34:26,076][20347] Updated weights for policy 0, policy_version 1288105 (0.0010) [2023-12-27 14:34:26,535][20381] Updated weights for policy 1, policy_version 1287228 (0.0007) [2023-12-27 14:34:26,590][20381] Updated weights for policy 1, policy_version 1287238 (0.0005) [2023-12-27 14:34:26,647][20381] Updated weights for policy 1, policy_version 1287248 (0.0005) [2023-12-27 14:34:26,747][20347] Updated weights for policy 0, policy_version 1288115 (0.0010) [2023-12-27 14:34:26,791][20347] Updated weights for policy 0, policy_version 1288125 (0.0010) [2023-12-27 14:34:26,838][20347] Updated weights for policy 0, policy_version 1288135 (0.0010) [2023-12-27 14:34:27,215][20381] Updated weights for policy 1, policy_version 1287258 (0.0006) [2023-12-27 14:34:27,268][20381] Updated weights for policy 1, policy_version 1287268 (0.0007) [2023-12-27 14:34:27,327][20381] Updated weights for policy 1, policy_version 1287278 (0.0007) [2023-12-27 14:34:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 659390464. Throughput: 0: 10143.9, 1: 10269.9. Samples: 659384180. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:27,363][19170] Avg episode reward: [(0, '990.210'), (1, '961.200')] [2023-12-27 14:34:27,378][20381] Updated weights for policy 1, policy_version 1287288 (0.0007) [2023-12-27 14:34:27,526][20347] Updated weights for policy 0, policy_version 1288145 (0.0010) [2023-12-27 14:34:27,576][20347] Updated weights for policy 0, policy_version 1288155 (0.0010) [2023-12-27 14:34:27,629][20347] Updated weights for policy 0, policy_version 1288165 (0.0008) [2023-12-27 14:34:27,676][20347] Updated weights for policy 0, policy_version 1288175 (0.0010) [2023-12-27 14:34:28,062][20381] Updated weights for policy 1, policy_version 1287298 (0.0007) [2023-12-27 14:34:28,108][20381] Updated weights for policy 1, policy_version 1287308 (0.0008) [2023-12-27 14:34:28,154][20381] Updated weights for policy 1, policy_version 1287318 (0.0010) [2023-12-27 14:34:28,291][20347] Updated weights for policy 0, policy_version 1288185 (0.0006) [2023-12-27 14:34:28,372][20347] Updated weights for policy 0, policy_version 1288195 (0.0006) [2023-12-27 14:34:28,427][20347] Updated weights for policy 0, policy_version 1288205 (0.0008) [2023-12-27 14:34:28,873][20381] Updated weights for policy 1, policy_version 1287328 (0.0006) [2023-12-27 14:34:28,925][20381] Updated weights for policy 1, policy_version 1287338 (0.0005) [2023-12-27 14:34:28,973][20381] Updated weights for policy 1, policy_version 1287348 (0.0005) [2023-12-27 14:34:29,118][20347] Updated weights for policy 0, policy_version 1288215 (0.0010) [2023-12-27 14:34:29,175][20347] Updated weights for policy 0, policy_version 1288225 (0.0010) [2023-12-27 14:34:29,233][20347] Updated weights for policy 0, policy_version 1288235 (0.0008) [2023-12-27 14:34:29,541][20381] Updated weights for policy 1, policy_version 1287358 (0.0008) [2023-12-27 14:34:29,603][20381] Updated weights for policy 1, policy_version 1287368 (0.0010) [2023-12-27 14:34:29,657][20381] Updated weights for policy 1, policy_version 1287378 (0.0010) [2023-12-27 14:34:29,975][20347] Updated weights for policy 0, policy_version 1288245 (0.0007) [2023-12-27 14:34:30,040][20347] Updated weights for policy 0, policy_version 1288255 (0.0006) [2023-12-27 14:34:30,105][20347] Updated weights for policy 0, policy_version 1288265 (0.0006) [2023-12-27 14:34:30,306][20381] Updated weights for policy 1, policy_version 1287388 (0.0010) [2023-12-27 14:34:30,358][20381] Updated weights for policy 1, policy_version 1287398 (0.0010) [2023-12-27 14:34:30,407][20381] Updated weights for policy 1, policy_version 1287408 (0.0010) [2023-12-27 14:34:30,718][20347] Updated weights for policy 0, policy_version 1288275 (0.0009) [2023-12-27 14:34:30,773][20347] Updated weights for policy 0, policy_version 1288285 (0.0010) [2023-12-27 14:34:30,837][20347] Updated weights for policy 0, policy_version 1288295 (0.0010) [2023-12-27 14:34:31,098][20381] Updated weights for policy 1, policy_version 1287418 (0.0009) [2023-12-27 14:34:31,162][20381] Updated weights for policy 1, policy_version 1287428 (0.0010) [2023-12-27 14:34:31,226][20381] Updated weights for policy 1, policy_version 1287438 (0.0008) [2023-12-27 14:34:31,294][20381] Updated weights for policy 1, policy_version 1287448 (0.0008) [2023-12-27 14:34:31,508][20347] Updated weights for policy 0, policy_version 1288305 (0.0008) [2023-12-27 14:34:31,571][20347] Updated weights for policy 0, policy_version 1288315 (0.0007) [2023-12-27 14:34:31,635][20347] Updated weights for policy 0, policy_version 1288325 (0.0008) [2023-12-27 14:34:31,689][20347] Updated weights for policy 0, policy_version 1288335 (0.0010) [2023-12-27 14:34:31,994][20381] Updated weights for policy 1, policy_version 1287458 (0.0008) [2023-12-27 14:34:32,046][20381] Updated weights for policy 1, policy_version 1287468 (0.0009) [2023-12-27 14:34:32,101][20381] Updated weights for policy 1, policy_version 1287478 (0.0011) [2023-12-27 14:34:32,362][20347] Updated weights for policy 0, policy_version 1288345 (0.0010) [2023-12-27 14:34:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 659496960. Throughput: 0: 10142.7, 1: 10295.0. Samples: 659509080. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.260')] [2023-12-27 14:34:32,426][20347] Updated weights for policy 0, policy_version 1288355 (0.0010) [2023-12-27 14:34:32,490][20347] Updated weights for policy 0, policy_version 1288365 (0.0008) [2023-12-27 14:34:32,742][20381] Updated weights for policy 1, policy_version 1287488 (0.0011) [2023-12-27 14:34:32,800][20381] Updated weights for policy 1, policy_version 1287498 (0.0010) [2023-12-27 14:34:32,860][20381] Updated weights for policy 1, policy_version 1287508 (0.0006) [2023-12-27 14:34:33,104][20347] Updated weights for policy 0, policy_version 1288375 (0.0006) [2023-12-27 14:34:33,160][20347] Updated weights for policy 0, policy_version 1288385 (0.0005) [2023-12-27 14:34:33,203][20347] Updated weights for policy 0, policy_version 1288395 (0.0005) [2023-12-27 14:34:33,516][20381] Updated weights for policy 1, policy_version 1287518 (0.0008) [2023-12-27 14:34:33,571][20381] Updated weights for policy 1, policy_version 1287528 (0.0010) [2023-12-27 14:34:33,632][20381] Updated weights for policy 1, policy_version 1287538 (0.0010) [2023-12-27 14:34:33,823][20347] Updated weights for policy 0, policy_version 1288405 (0.0010) [2023-12-27 14:34:33,867][20347] Updated weights for policy 0, policy_version 1288415 (0.0010) [2023-12-27 14:34:33,922][20347] Updated weights for policy 0, policy_version 1288425 (0.0005) [2023-12-27 14:34:34,282][20381] Updated weights for policy 1, policy_version 1287548 (0.0009) [2023-12-27 14:34:34,340][20381] Updated weights for policy 1, policy_version 1287558 (0.0008) [2023-12-27 14:34:34,395][20381] Updated weights for policy 1, policy_version 1287568 (0.0008) [2023-12-27 14:34:34,534][20347] Updated weights for policy 0, policy_version 1288435 (0.0009) [2023-12-27 14:34:34,593][20347] Updated weights for policy 0, policy_version 1288445 (0.0006) [2023-12-27 14:34:34,653][20347] Updated weights for policy 0, policy_version 1288455 (0.0010) [2023-12-27 14:34:35,106][20381] Updated weights for policy 1, policy_version 1287578 (0.0009) [2023-12-27 14:34:35,153][20381] Updated weights for policy 1, policy_version 1287588 (0.0010) [2023-12-27 14:34:35,211][20381] Updated weights for policy 1, policy_version 1287598 (0.0010) [2023-12-27 14:34:35,264][20381] Updated weights for policy 1, policy_version 1287608 (0.0010) [2023-12-27 14:34:35,314][20347] Updated weights for policy 0, policy_version 1288465 (0.0010) [2023-12-27 14:34:35,363][20291] KL-divergence is very high: 156.4604 [2023-12-27 14:34:35,369][20347] Updated weights for policy 0, policy_version 1288475 (0.0010) [2023-12-27 14:34:35,405][20291] KL-divergence is very high: 110.8482 [2023-12-27 14:34:35,425][20347] Updated weights for policy 0, policy_version 1288485 (0.0010) [2023-12-27 14:34:35,487][20347] Updated weights for policy 0, policy_version 1288495 (0.0009) [2023-12-27 14:34:35,946][20381] Updated weights for policy 1, policy_version 1287618 (0.0010) [2023-12-27 14:34:35,958][20292] KL-divergence is very high: 106.7003 [2023-12-27 14:34:36,004][20381] Updated weights for policy 1, policy_version 1287628 (0.0007) [2023-12-27 14:34:36,005][20292] KL-divergence is very high: 144.5125 [2023-12-27 14:34:36,066][20381] Updated weights for policy 1, policy_version 1287638 (0.0005) [2023-12-27 14:34:36,150][20347] Updated weights for policy 0, policy_version 1288505 (0.0007) [2023-12-27 14:34:36,208][20347] Updated weights for policy 0, policy_version 1288515 (0.0006) [2023-12-27 14:34:36,270][20347] Updated weights for policy 0, policy_version 1288525 (0.0011) [2023-12-27 14:34:36,706][20381] Updated weights for policy 1, policy_version 1287648 (0.0006) [2023-12-27 14:34:36,767][20381] Updated weights for policy 1, policy_version 1287658 (0.0006) [2023-12-27 14:34:36,829][20381] Updated weights for policy 1, policy_version 1287668 (0.0006) [2023-12-27 14:34:36,910][20347] Updated weights for policy 0, policy_version 1288535 (0.0010) [2023-12-27 14:34:36,965][20347] Updated weights for policy 0, policy_version 1288545 (0.0010) [2023-12-27 14:34:37,024][20347] Updated weights for policy 0, policy_version 1288555 (0.0010) [2023-12-27 14:34:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 659603456. Throughput: 0: 10192.6, 1: 10293.8. Samples: 659572604. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:37,363][19170] Avg episode reward: [(0, '980.480'), (1, '892.360')] [2023-12-27 14:34:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001288560_329932800.pth... [2023-12-27 14:34:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001287344_329621504.pth [2023-12-27 14:34:37,404][20381] Updated weights for policy 1, policy_version 1287678 (0.0006) [2023-12-27 14:34:37,450][20381] Updated weights for policy 1, policy_version 1287688 (0.0005) [2023-12-27 14:34:37,495][20381] Updated weights for policy 1, policy_version 1287698 (0.0005) [2023-12-27 14:34:37,531][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001287704_329678848.pth... [2023-12-27 14:34:37,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001286488_329367552.pth [2023-12-27 14:34:37,711][20347] Updated weights for policy 0, policy_version 1288565 (0.0011) [2023-12-27 14:34:37,773][20347] Updated weights for policy 0, policy_version 1288575 (0.0011) [2023-12-27 14:34:37,832][20347] Updated weights for policy 0, policy_version 1288585 (0.0010) [2023-12-27 14:34:38,101][20381] Updated weights for policy 1, policy_version 1287708 (0.0005) [2023-12-27 14:34:38,146][20381] Updated weights for policy 1, policy_version 1287718 (0.0005) [2023-12-27 14:34:38,193][20381] Updated weights for policy 1, policy_version 1287728 (0.0007) [2023-12-27 14:34:38,451][20347] Updated weights for policy 0, policy_version 1288595 (0.0009) [2023-12-27 14:34:38,498][20347] Updated weights for policy 0, policy_version 1288605 (0.0008) [2023-12-27 14:34:38,545][20347] Updated weights for policy 0, policy_version 1288615 (0.0005) [2023-12-27 14:34:38,884][20381] Updated weights for policy 1, policy_version 1287738 (0.0010) [2023-12-27 14:34:38,945][20381] Updated weights for policy 1, policy_version 1287748 (0.0009) [2023-12-27 14:34:39,011][20381] Updated weights for policy 1, policy_version 1287758 (0.0008) [2023-12-27 14:34:39,068][20381] Updated weights for policy 1, policy_version 1287768 (0.0009) [2023-12-27 14:34:39,265][20347] Updated weights for policy 0, policy_version 1288625 (0.0007) [2023-12-27 14:34:39,324][20347] Updated weights for policy 0, policy_version 1288635 (0.0010) [2023-12-27 14:34:39,385][20347] Updated weights for policy 0, policy_version 1288645 (0.0009) [2023-12-27 14:34:39,457][20347] Updated weights for policy 0, policy_version 1288655 (0.0008) [2023-12-27 14:34:39,734][20381] Updated weights for policy 1, policy_version 1287778 (0.0006) [2023-12-27 14:34:39,793][20381] Updated weights for policy 1, policy_version 1287788 (0.0007) [2023-12-27 14:34:39,859][20381] Updated weights for policy 1, policy_version 1287798 (0.0007) [2023-12-27 14:34:40,171][20347] Updated weights for policy 0, policy_version 1288665 (0.0009) [2023-12-27 14:34:40,234][20347] Updated weights for policy 0, policy_version 1288675 (0.0008) [2023-12-27 14:34:40,299][20347] Updated weights for policy 0, policy_version 1288685 (0.0008) [2023-12-27 14:34:40,525][20381] Updated weights for policy 1, policy_version 1287808 (0.0007) [2023-12-27 14:34:40,576][20381] Updated weights for policy 1, policy_version 1287818 (0.0007) [2023-12-27 14:34:40,635][20381] Updated weights for policy 1, policy_version 1287828 (0.0007) [2023-12-27 14:34:40,974][20347] Updated weights for policy 0, policy_version 1288695 (0.0007) [2023-12-27 14:34:41,035][20347] Updated weights for policy 0, policy_version 1288705 (0.0008) [2023-12-27 14:34:41,100][20347] Updated weights for policy 0, policy_version 1288715 (0.0008) [2023-12-27 14:34:41,267][20381] Updated weights for policy 1, policy_version 1287838 (0.0007) [2023-12-27 14:34:41,329][20381] Updated weights for policy 1, policy_version 1287848 (0.0007) [2023-12-27 14:34:41,401][20381] Updated weights for policy 1, policy_version 1287858 (0.0008) [2023-12-27 14:34:41,866][20347] Updated weights for policy 0, policy_version 1288725 (0.0008) [2023-12-27 14:34:41,928][20347] Updated weights for policy 0, policy_version 1288735 (0.0007) [2023-12-27 14:34:41,994][20347] Updated weights for policy 0, policy_version 1288745 (0.0007) [2023-12-27 14:34:42,141][20381] Updated weights for policy 1, policy_version 1287868 (0.0009) [2023-12-27 14:34:42,197][20381] Updated weights for policy 1, policy_version 1287878 (0.0009) [2023-12-27 14:34:42,262][20381] Updated weights for policy 1, policy_version 1287888 (0.0010) [2023-12-27 14:34:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 659709952. Throughput: 0: 10236.4, 1: 10346.5. Samples: 659697604. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:42,363][19170] Avg episode reward: [(0, '970.690'), (1, '862.990')] [2023-12-27 14:34:42,707][20347] Updated weights for policy 0, policy_version 1288755 (0.0008) [2023-12-27 14:34:42,763][20347] Updated weights for policy 0, policy_version 1288765 (0.0007) [2023-12-27 14:34:42,822][20347] Updated weights for policy 0, policy_version 1288775 (0.0008) [2023-12-27 14:34:42,913][20381] Updated weights for policy 1, policy_version 1287898 (0.0007) [2023-12-27 14:34:42,968][20381] Updated weights for policy 1, policy_version 1287908 (0.0005) [2023-12-27 14:34:43,026][20381] Updated weights for policy 1, policy_version 1287918 (0.0006) [2023-12-27 14:34:43,090][20381] Updated weights for policy 1, policy_version 1287928 (0.0005) [2023-12-27 14:34:43,465][20347] Updated weights for policy 0, policy_version 1288785 (0.0008) [2023-12-27 14:34:43,527][20347] Updated weights for policy 0, policy_version 1288795 (0.0008) [2023-12-27 14:34:43,577][20347] Updated weights for policy 0, policy_version 1288805 (0.0007) [2023-12-27 14:34:43,629][20347] Updated weights for policy 0, policy_version 1288815 (0.0008) [2023-12-27 14:34:43,729][20381] Updated weights for policy 1, policy_version 1287938 (0.0008) [2023-12-27 14:34:43,788][20381] Updated weights for policy 1, policy_version 1287948 (0.0008) [2023-12-27 14:34:43,843][20381] Updated weights for policy 1, policy_version 1287958 (0.0008) [2023-12-27 14:34:44,287][20347] Updated weights for policy 0, policy_version 1288825 (0.0010) [2023-12-27 14:34:44,340][20347] Updated weights for policy 0, policy_version 1288835 (0.0010) [2023-12-27 14:34:44,398][20347] Updated weights for policy 0, policy_version 1288845 (0.0010) [2023-12-27 14:34:44,572][20381] Updated weights for policy 1, policy_version 1287968 (0.0007) [2023-12-27 14:34:44,625][20381] Updated weights for policy 1, policy_version 1287978 (0.0008) [2023-12-27 14:34:44,685][20381] Updated weights for policy 1, policy_version 1287988 (0.0005) [2023-12-27 14:34:45,045][20347] Updated weights for policy 0, policy_version 1288855 (0.0010) [2023-12-27 14:34:45,110][20347] Updated weights for policy 0, policy_version 1288865 (0.0010) [2023-12-27 14:34:45,173][20347] Updated weights for policy 0, policy_version 1288875 (0.0010) [2023-12-27 14:34:45,369][20381] Updated weights for policy 1, policy_version 1287998 (0.0007) [2023-12-27 14:34:45,432][20381] Updated weights for policy 1, policy_version 1288008 (0.0008) [2023-12-27 14:34:45,493][20381] Updated weights for policy 1, policy_version 1288018 (0.0005) [2023-12-27 14:34:45,854][20347] Updated weights for policy 0, policy_version 1288885 (0.0006) [2023-12-27 14:34:45,902][20347] Updated weights for policy 0, policy_version 1288895 (0.0006) [2023-12-27 14:34:45,951][20347] Updated weights for policy 0, policy_version 1288905 (0.0007) [2023-12-27 14:34:46,078][20381] Updated weights for policy 1, policy_version 1288028 (0.0006) [2023-12-27 14:34:46,128][20381] Updated weights for policy 1, policy_version 1288038 (0.0007) [2023-12-27 14:34:46,180][20381] Updated weights for policy 1, policy_version 1288048 (0.0007) [2023-12-27 14:34:46,614][20347] Updated weights for policy 0, policy_version 1288915 (0.0008) [2023-12-27 14:34:46,661][20347] Updated weights for policy 0, policy_version 1288925 (0.0009) [2023-12-27 14:34:46,714][20347] Updated weights for policy 0, policy_version 1288935 (0.0005) [2023-12-27 14:34:46,888][20381] Updated weights for policy 1, policy_version 1288058 (0.0007) [2023-12-27 14:34:46,946][20381] Updated weights for policy 1, policy_version 1288068 (0.0005) [2023-12-27 14:34:47,000][20381] Updated weights for policy 1, policy_version 1288078 (0.0008) [2023-12-27 14:34:47,046][20381] Updated weights for policy 1, policy_version 1288088 (0.0008) [2023-12-27 14:34:47,347][20347] Updated weights for policy 0, policy_version 1288945 (0.0006) [2023-12-27 14:34:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 659808256. Throughput: 0: 10293.7, 1: 10355.0. Samples: 659819328. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:47,363][19170] Avg episode reward: [(0, '970.650'), (1, '902.120')] [2023-12-27 14:34:47,404][20347] Updated weights for policy 0, policy_version 1288955 (0.0005) [2023-12-27 14:34:47,465][20347] Updated weights for policy 0, policy_version 1288965 (0.0006) [2023-12-27 14:34:47,518][20347] Updated weights for policy 0, policy_version 1288975 (0.0005) [2023-12-27 14:34:47,766][20381] Updated weights for policy 1, policy_version 1288098 (0.0008) [2023-12-27 14:34:47,819][20381] Updated weights for policy 1, policy_version 1288108 (0.0008) [2023-12-27 14:34:47,872][20381] Updated weights for policy 1, policy_version 1288118 (0.0007) [2023-12-27 14:34:48,049][20347] Updated weights for policy 0, policy_version 1288985 (0.0007) [2023-12-27 14:34:48,100][20347] Updated weights for policy 0, policy_version 1288995 (0.0007) [2023-12-27 14:34:48,152][20347] Updated weights for policy 0, policy_version 1289005 (0.0008) [2023-12-27 14:34:48,576][20381] Updated weights for policy 1, policy_version 1288128 (0.0008) [2023-12-27 14:34:48,640][20381] Updated weights for policy 1, policy_version 1288138 (0.0008) [2023-12-27 14:34:48,691][20381] Updated weights for policy 1, policy_version 1288148 (0.0007) [2023-12-27 14:34:48,851][20347] Updated weights for policy 0, policy_version 1289015 (0.0006) [2023-12-27 14:34:48,921][20347] Updated weights for policy 0, policy_version 1289025 (0.0008) [2023-12-27 14:34:48,986][20347] Updated weights for policy 0, policy_version 1289035 (0.0008) [2023-12-27 14:34:49,478][20381] Updated weights for policy 1, policy_version 1288158 (0.0008) [2023-12-27 14:34:49,536][20381] Updated weights for policy 1, policy_version 1288168 (0.0006) [2023-12-27 14:34:49,600][20381] Updated weights for policy 1, policy_version 1288178 (0.0007) [2023-12-27 14:34:49,674][20347] Updated weights for policy 0, policy_version 1289045 (0.0007) [2023-12-27 14:34:49,738][20347] Updated weights for policy 0, policy_version 1289055 (0.0010) [2023-12-27 14:34:49,800][20347] Updated weights for policy 0, policy_version 1289065 (0.0010) [2023-12-27 14:34:50,234][20381] Updated weights for policy 1, policy_version 1288188 (0.0006) [2023-12-27 14:34:50,294][20381] Updated weights for policy 1, policy_version 1288198 (0.0006) [2023-12-27 14:34:50,350][20381] Updated weights for policy 1, policy_version 1288208 (0.0006) [2023-12-27 14:34:50,536][20347] Updated weights for policy 0, policy_version 1289075 (0.0009) [2023-12-27 14:34:50,591][20347] Updated weights for policy 0, policy_version 1289085 (0.0007) [2023-12-27 14:34:50,650][20347] Updated weights for policy 0, policy_version 1289095 (0.0008) [2023-12-27 14:34:51,012][20381] Updated weights for policy 1, policy_version 1288218 (0.0008) [2023-12-27 14:34:51,077][20381] Updated weights for policy 1, policy_version 1288228 (0.0008) [2023-12-27 14:34:51,137][20381] Updated weights for policy 1, policy_version 1288238 (0.0008) [2023-12-27 14:34:51,200][20381] Updated weights for policy 1, policy_version 1288248 (0.0008) [2023-12-27 14:34:51,278][20347] Updated weights for policy 0, policy_version 1289105 (0.0008) [2023-12-27 14:34:51,347][20347] Updated weights for policy 0, policy_version 1289115 (0.0007) [2023-12-27 14:34:51,412][20347] Updated weights for policy 0, policy_version 1289125 (0.0007) [2023-12-27 14:34:51,472][20347] Updated weights for policy 0, policy_version 1289135 (0.0006) [2023-12-27 14:34:51,998][20381] Updated weights for policy 1, policy_version 1288258 (0.0007) [2023-12-27 14:34:52,056][20381] Updated weights for policy 1, policy_version 1288268 (0.0007) [2023-12-27 14:34:52,107][20381] Updated weights for policy 1, policy_version 1288278 (0.0007) [2023-12-27 14:34:52,187][20347] Updated weights for policy 0, policy_version 1289145 (0.0008) [2023-12-27 14:34:52,241][20347] Updated weights for policy 0, policy_version 1289155 (0.0008) [2023-12-27 14:34:52,300][20347] Updated weights for policy 0, policy_version 1289165 (0.0008) [2023-12-27 14:34:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 659914752. Throughput: 0: 10323.9, 1: 10305.9. Samples: 659880788. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:52,363][19170] Avg episode reward: [(0, '960.950'), (1, '902.210')] [2023-12-27 14:34:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001289168_330088448.pth... [2023-12-27 14:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001288280_329826304.pth... [2023-12-27 14:34:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001287952_329777152.pth [2023-12-27 14:34:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001287096_329523200.pth [2023-12-27 14:34:52,841][20381] Updated weights for policy 1, policy_version 1288288 (0.0007) [2023-12-27 14:34:52,901][20381] Updated weights for policy 1, policy_version 1288298 (0.0008) [2023-12-27 14:34:52,965][20381] Updated weights for policy 1, policy_version 1288308 (0.0007) [2023-12-27 14:34:52,966][20347] Updated weights for policy 0, policy_version 1289175 (0.0006) [2023-12-27 14:34:53,020][20347] Updated weights for policy 0, policy_version 1289185 (0.0006) [2023-12-27 14:34:53,096][20347] Updated weights for policy 0, policy_version 1289195 (0.0005) [2023-12-27 14:34:53,675][20381] Updated weights for policy 1, policy_version 1288318 (0.0009) [2023-12-27 14:34:53,682][20347] Updated weights for policy 0, policy_version 1289205 (0.0007) [2023-12-27 14:34:53,721][20381] Updated weights for policy 1, policy_version 1288328 (0.0010) [2023-12-27 14:34:53,725][20347] Updated weights for policy 0, policy_version 1289215 (0.0008) [2023-12-27 14:34:53,772][20381] Updated weights for policy 1, policy_version 1288338 (0.0009) [2023-12-27 14:34:53,781][20347] Updated weights for policy 0, policy_version 1289225 (0.0007) [2023-12-27 14:34:54,509][20381] Updated weights for policy 1, policy_version 1288348 (0.0008) [2023-12-27 14:34:54,511][20347] Updated weights for policy 0, policy_version 1289235 (0.0007) [2023-12-27 14:34:54,562][20381] Updated weights for policy 1, policy_version 1288358 (0.0008) [2023-12-27 14:34:54,569][20347] Updated weights for policy 0, policy_version 1289245 (0.0010) [2023-12-27 14:34:54,617][20381] Updated weights for policy 1, policy_version 1288368 (0.0006) [2023-12-27 14:34:54,626][20347] Updated weights for policy 0, policy_version 1289255 (0.0011) [2023-12-27 14:34:55,292][20381] Updated weights for policy 1, policy_version 1288378 (0.0006) [2023-12-27 14:34:55,312][20347] Updated weights for policy 0, policy_version 1289265 (0.0010) [2023-12-27 14:34:55,347][20381] Updated weights for policy 1, policy_version 1288388 (0.0006) [2023-12-27 14:34:55,364][20347] Updated weights for policy 0, policy_version 1289275 (0.0010) [2023-12-27 14:34:55,402][20381] Updated weights for policy 1, policy_version 1288398 (0.0005) [2023-12-27 14:34:55,415][20347] Updated weights for policy 0, policy_version 1289285 (0.0010) [2023-12-27 14:34:55,457][20381] Updated weights for policy 1, policy_version 1288408 (0.0005) [2023-12-27 14:34:55,466][20347] Updated weights for policy 0, policy_version 1289295 (0.0010) [2023-12-27 14:34:56,129][20347] Updated weights for policy 0, policy_version 1289305 (0.0006) [2023-12-27 14:34:56,165][20381] Updated weights for policy 1, policy_version 1288418 (0.0008) [2023-12-27 14:34:56,186][20347] Updated weights for policy 0, policy_version 1289315 (0.0010) [2023-12-27 14:34:56,212][20381] Updated weights for policy 1, policy_version 1288428 (0.0006) [2023-12-27 14:34:56,233][20347] Updated weights for policy 0, policy_version 1289325 (0.0010) [2023-12-27 14:34:56,261][20381] Updated weights for policy 1, policy_version 1288438 (0.0006) [2023-12-27 14:34:56,849][20347] Updated weights for policy 0, policy_version 1289335 (0.0007) [2023-12-27 14:34:56,902][20381] Updated weights for policy 1, policy_version 1288448 (0.0008) [2023-12-27 14:34:56,908][20347] Updated weights for policy 0, policy_version 1289345 (0.0005) [2023-12-27 14:34:56,959][20381] Updated weights for policy 1, policy_version 1288458 (0.0007) [2023-12-27 14:34:56,964][20347] Updated weights for policy 0, policy_version 1289355 (0.0006) [2023-12-27 14:34:57,011][20381] Updated weights for policy 1, policy_version 1288468 (0.0006) [2023-12-27 14:34:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 660013056. Throughput: 0: 10383.9, 1: 10188.8. Samples: 660000876. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:34:57,364][19170] Avg episode reward: [(0, '921.860'), (1, '902.210')] [2023-12-27 14:34:57,598][20347] Updated weights for policy 0, policy_version 1289365 (0.0009) [2023-12-27 14:34:57,655][20347] Updated weights for policy 0, policy_version 1289375 (0.0010) [2023-12-27 14:34:57,696][20381] Updated weights for policy 1, policy_version 1288478 (0.0006) [2023-12-27 14:34:57,705][20347] Updated weights for policy 0, policy_version 1289385 (0.0010) [2023-12-27 14:34:57,747][20381] Updated weights for policy 1, policy_version 1288488 (0.0005) [2023-12-27 14:34:57,799][20381] Updated weights for policy 1, policy_version 1288498 (0.0007) [2023-12-27 14:34:58,422][20347] Updated weights for policy 0, policy_version 1289395 (0.0009) [2023-12-27 14:34:58,484][20347] Updated weights for policy 0, policy_version 1289405 (0.0008) [2023-12-27 14:34:58,544][20381] Updated weights for policy 1, policy_version 1288508 (0.0006) [2023-12-27 14:34:58,546][20347] Updated weights for policy 0, policy_version 1289415 (0.0009) [2023-12-27 14:34:58,614][20381] Updated weights for policy 1, policy_version 1288518 (0.0006) [2023-12-27 14:34:58,676][20381] Updated weights for policy 1, policy_version 1288528 (0.0008) [2023-12-27 14:34:59,282][20347] Updated weights for policy 0, policy_version 1289425 (0.0009) [2023-12-27 14:34:59,342][20347] Updated weights for policy 0, policy_version 1289435 (0.0007) [2023-12-27 14:34:59,407][20347] Updated weights for policy 0, policy_version 1289445 (0.0008) [2023-12-27 14:34:59,429][20381] Updated weights for policy 1, policy_version 1288538 (0.0008) [2023-12-27 14:34:59,466][20347] Updated weights for policy 0, policy_version 1289455 (0.0006) [2023-12-27 14:34:59,491][20381] Updated weights for policy 1, policy_version 1288548 (0.0011) [2023-12-27 14:34:59,541][20381] Updated weights for policy 1, policy_version 1288558 (0.0006) [2023-12-27 14:34:59,592][20381] Updated weights for policy 1, policy_version 1288568 (0.0006) [2023-12-27 14:35:00,199][20347] Updated weights for policy 0, policy_version 1289465 (0.0008) [2023-12-27 14:35:00,257][20347] Updated weights for policy 0, policy_version 1289475 (0.0008) [2023-12-27 14:35:00,308][20347] Updated weights for policy 0, policy_version 1289485 (0.0008) [2023-12-27 14:35:00,318][20381] Updated weights for policy 1, policy_version 1288578 (0.0011) [2023-12-27 14:35:00,383][20381] Updated weights for policy 1, policy_version 1288588 (0.0011) [2023-12-27 14:35:00,444][20381] Updated weights for policy 1, policy_version 1288598 (0.0011) [2023-12-27 14:35:00,911][20347] Updated weights for policy 0, policy_version 1289495 (0.0008) [2023-12-27 14:35:00,965][20347] Updated weights for policy 0, policy_version 1289505 (0.0007) [2023-12-27 14:35:01,015][20347] Updated weights for policy 0, policy_version 1289515 (0.0007) [2023-12-27 14:35:01,090][20381] Updated weights for policy 1, policy_version 1288608 (0.0010) [2023-12-27 14:35:01,153][20381] Updated weights for policy 1, policy_version 1288618 (0.0009) [2023-12-27 14:35:01,218][20381] Updated weights for policy 1, policy_version 1288628 (0.0008) [2023-12-27 14:35:01,714][20347] Updated weights for policy 0, policy_version 1289525 (0.0007) [2023-12-27 14:35:01,777][20347] Updated weights for policy 0, policy_version 1289535 (0.0008) [2023-12-27 14:35:01,839][20347] Updated weights for policy 0, policy_version 1289545 (0.0005) [2023-12-27 14:35:01,981][20381] Updated weights for policy 1, policy_version 1288638 (0.0008) [2023-12-27 14:35:02,038][20381] Updated weights for policy 1, policy_version 1288648 (0.0008) [2023-12-27 14:35:02,097][20381] Updated weights for policy 1, policy_version 1288658 (0.0008) [2023-12-27 14:35:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 660111360. Throughput: 0: 10339.9, 1: 10097.3. Samples: 660120696. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:02,363][19170] Avg episode reward: [(0, '941.380'), (1, '931.550')] [2023-12-27 14:35:02,539][20347] Updated weights for policy 0, policy_version 1289555 (0.0008) [2023-12-27 14:35:02,598][20347] Updated weights for policy 0, policy_version 1289565 (0.0008) [2023-12-27 14:35:02,651][20347] Updated weights for policy 0, policy_version 1289575 (0.0008) [2023-12-27 14:35:02,801][20381] Updated weights for policy 1, policy_version 1288668 (0.0007) [2023-12-27 14:35:02,848][20381] Updated weights for policy 1, policy_version 1288678 (0.0007) [2023-12-27 14:35:02,900][20381] Updated weights for policy 1, policy_version 1288688 (0.0007) [2023-12-27 14:35:03,248][20347] Updated weights for policy 0, policy_version 1289585 (0.0007) [2023-12-27 14:35:03,299][20347] Updated weights for policy 0, policy_version 1289595 (0.0005) [2023-12-27 14:35:03,344][20347] Updated weights for policy 0, policy_version 1289605 (0.0005) [2023-12-27 14:35:03,389][20347] Updated weights for policy 0, policy_version 1289615 (0.0006) [2023-12-27 14:35:03,659][20381] Updated weights for policy 1, policy_version 1288698 (0.0007) [2023-12-27 14:35:03,704][20381] Updated weights for policy 1, policy_version 1288708 (0.0006) [2023-12-27 14:35:03,745][20381] Updated weights for policy 1, policy_version 1288718 (0.0007) [2023-12-27 14:35:03,794][20381] Updated weights for policy 1, policy_version 1288728 (0.0008) [2023-12-27 14:35:04,018][20347] Updated weights for policy 0, policy_version 1289625 (0.0010) [2023-12-27 14:35:04,080][20347] Updated weights for policy 0, policy_version 1289635 (0.0011) [2023-12-27 14:35:04,133][20347] Updated weights for policy 0, policy_version 1289645 (0.0011) [2023-12-27 14:35:04,472][20381] Updated weights for policy 1, policy_version 1288738 (0.0007) [2023-12-27 14:35:04,526][20381] Updated weights for policy 1, policy_version 1288748 (0.0007) [2023-12-27 14:35:04,579][20381] Updated weights for policy 1, policy_version 1288758 (0.0008) [2023-12-27 14:35:04,839][20347] Updated weights for policy 0, policy_version 1289655 (0.0010) [2023-12-27 14:35:04,883][20347] Updated weights for policy 0, policy_version 1289665 (0.0010) [2023-12-27 14:35:04,929][20347] Updated weights for policy 0, policy_version 1289675 (0.0010) [2023-12-27 14:35:05,311][20381] Updated weights for policy 1, policy_version 1288768 (0.0010) [2023-12-27 14:35:05,373][20381] Updated weights for policy 1, policy_version 1288778 (0.0011) [2023-12-27 14:35:05,422][20381] Updated weights for policy 1, policy_version 1288788 (0.0010) [2023-12-27 14:35:05,520][20347] Updated weights for policy 0, policy_version 1289685 (0.0008) [2023-12-27 14:35:05,573][20347] Updated weights for policy 0, policy_version 1289695 (0.0009) [2023-12-27 14:35:05,624][20347] Updated weights for policy 0, policy_version 1289705 (0.0007) [2023-12-27 14:35:06,119][20381] Updated weights for policy 1, policy_version 1288798 (0.0009) [2023-12-27 14:35:06,184][20381] Updated weights for policy 1, policy_version 1288808 (0.0008) [2023-12-27 14:35:06,247][20381] Updated weights for policy 1, policy_version 1288818 (0.0006) [2023-12-27 14:35:06,330][20347] Updated weights for policy 0, policy_version 1289715 (0.0008) [2023-12-27 14:35:06,393][20347] Updated weights for policy 0, policy_version 1289725 (0.0007) [2023-12-27 14:35:06,460][20347] Updated weights for policy 0, policy_version 1289735 (0.0007) [2023-12-27 14:35:06,868][20381] Updated weights for policy 1, policy_version 1288828 (0.0006) [2023-12-27 14:35:06,924][20381] Updated weights for policy 1, policy_version 1288838 (0.0008) [2023-12-27 14:35:06,980][20381] Updated weights for policy 1, policy_version 1288848 (0.0007) [2023-12-27 14:35:07,150][20347] Updated weights for policy 0, policy_version 1289745 (0.0009) [2023-12-27 14:35:07,201][20347] Updated weights for policy 0, policy_version 1289755 (0.0008) [2023-12-27 14:35:07,251][20347] Updated weights for policy 0, policy_version 1289765 (0.0008) [2023-12-27 14:35:07,307][20347] Updated weights for policy 0, policy_version 1289775 (0.0008) [2023-12-27 14:35:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 660217856. Throughput: 0: 10322.3, 1: 10112.1. Samples: 660182332. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:07,363][19170] Avg episode reward: [(0, '951.050'), (1, '960.900')] [2023-12-27 14:35:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001288856_329973760.pth... [2023-12-27 14:35:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001289776_330244096.pth... [2023-12-27 14:35:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001287704_329678848.pth [2023-12-27 14:35:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001288560_329932800.pth [2023-12-27 14:35:07,704][20381] Updated weights for policy 1, policy_version 1288858 (0.0008) [2023-12-27 14:35:07,767][20381] Updated weights for policy 1, policy_version 1288868 (0.0008) [2023-12-27 14:35:07,825][20381] Updated weights for policy 1, policy_version 1288878 (0.0007) [2023-12-27 14:35:07,880][20381] Updated weights for policy 1, policy_version 1288888 (0.0008) [2023-12-27 14:35:08,016][20347] Updated weights for policy 0, policy_version 1289785 (0.0007) [2023-12-27 14:35:08,070][20347] Updated weights for policy 0, policy_version 1289795 (0.0007) [2023-12-27 14:35:08,126][20347] Updated weights for policy 0, policy_version 1289805 (0.0008) [2023-12-27 14:35:08,581][20381] Updated weights for policy 1, policy_version 1288898 (0.0009) [2023-12-27 14:35:08,640][20381] Updated weights for policy 1, policy_version 1288908 (0.0006) [2023-12-27 14:35:08,697][20381] Updated weights for policy 1, policy_version 1288918 (0.0005) [2023-12-27 14:35:08,813][20347] Updated weights for policy 0, policy_version 1289815 (0.0010) [2023-12-27 14:35:08,870][20347] Updated weights for policy 0, policy_version 1289825 (0.0006) [2023-12-27 14:35:08,930][20347] Updated weights for policy 0, policy_version 1289835 (0.0006) [2023-12-27 14:35:09,380][20381] Updated weights for policy 1, policy_version 1288928 (0.0007) [2023-12-27 14:35:09,442][20381] Updated weights for policy 1, policy_version 1288938 (0.0008) [2023-12-27 14:35:09,499][20381] Updated weights for policy 1, policy_version 1288948 (0.0008) [2023-12-27 14:35:09,601][20347] Updated weights for policy 0, policy_version 1289845 (0.0005) [2023-12-27 14:35:09,673][20347] Updated weights for policy 0, policy_version 1289855 (0.0005) [2023-12-27 14:35:09,744][20347] Updated weights for policy 0, policy_version 1289865 (0.0005) [2023-12-27 14:35:10,229][20381] Updated weights for policy 1, policy_version 1288958 (0.0010) [2023-12-27 14:35:10,291][20381] Updated weights for policy 1, policy_version 1288968 (0.0007) [2023-12-27 14:35:10,334][20347] Updated weights for policy 0, policy_version 1289875 (0.0009) [2023-12-27 14:35:10,351][20381] Updated weights for policy 1, policy_version 1288978 (0.0010) [2023-12-27 14:35:10,394][20347] Updated weights for policy 0, policy_version 1289885 (0.0008) [2023-12-27 14:35:10,457][20347] Updated weights for policy 0, policy_version 1289895 (0.0009) [2023-12-27 14:35:10,967][20381] Updated weights for policy 1, policy_version 1288988 (0.0008) [2023-12-27 14:35:11,018][20381] Updated weights for policy 1, policy_version 1288998 (0.0006) [2023-12-27 14:35:11,072][20347] Updated weights for policy 0, policy_version 1289905 (0.0007) [2023-12-27 14:35:11,088][20381] Updated weights for policy 1, policy_version 1289008 (0.0009) [2023-12-27 14:35:11,125][20347] Updated weights for policy 0, policy_version 1289915 (0.0007) [2023-12-27 14:35:11,188][20347] Updated weights for policy 0, policy_version 1289925 (0.0008) [2023-12-27 14:35:11,252][20347] Updated weights for policy 0, policy_version 1289935 (0.0007) [2023-12-27 14:35:11,845][20381] Updated weights for policy 1, policy_version 1289018 (0.0009) [2023-12-27 14:35:11,905][20381] Updated weights for policy 1, policy_version 1289028 (0.0006) [2023-12-27 14:35:11,929][20292] KL-divergence is very high: 292.6608 [2023-12-27 14:35:11,957][20381] Updated weights for policy 1, policy_version 1289038 (0.0007) [2023-12-27 14:35:11,970][20347] Updated weights for policy 0, policy_version 1289945 (0.0006) [2023-12-27 14:35:11,972][20292] KL-divergence is very high: 266.0920 [2023-12-27 14:35:12,008][20381] Updated weights for policy 1, policy_version 1289048 (0.0008) [2023-12-27 14:35:12,028][20347] Updated weights for policy 0, policy_version 1289955 (0.0007) [2023-12-27 14:35:12,082][20347] Updated weights for policy 0, policy_version 1289965 (0.0008) [2023-12-27 14:35:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 660316160. Throughput: 0: 10330.9, 1: 10143.7. Samples: 660305536. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:12,363][19170] Avg episode reward: [(0, '941.260'), (1, '970.650')] [2023-12-27 14:35:12,711][20381] Updated weights for policy 1, policy_version 1289058 (0.0008) [2023-12-27 14:35:12,774][20381] Updated weights for policy 1, policy_version 1289068 (0.0008) [2023-12-27 14:35:12,829][20381] Updated weights for policy 1, policy_version 1289078 (0.0007) [2023-12-27 14:35:12,849][20347] Updated weights for policy 0, policy_version 1289975 (0.0008) [2023-12-27 14:35:12,908][20347] Updated weights for policy 0, policy_version 1289985 (0.0008) [2023-12-27 14:35:12,967][20347] Updated weights for policy 0, policy_version 1289995 (0.0005) [2023-12-27 14:35:13,418][20381] Updated weights for policy 1, policy_version 1289088 (0.0008) [2023-12-27 14:35:13,479][20381] Updated weights for policy 1, policy_version 1289098 (0.0008) [2023-12-27 14:35:13,538][20381] Updated weights for policy 1, policy_version 1289108 (0.0007) [2023-12-27 14:35:13,608][20347] Updated weights for policy 0, policy_version 1290005 (0.0006) [2023-12-27 14:35:13,668][20347] Updated weights for policy 0, policy_version 1290015 (0.0005) [2023-12-27 14:35:13,730][20347] Updated weights for policy 0, policy_version 1290025 (0.0005) [2023-12-27 14:35:14,163][20381] Updated weights for policy 1, policy_version 1289118 (0.0009) [2023-12-27 14:35:14,221][20381] Updated weights for policy 1, policy_version 1289128 (0.0011) [2023-12-27 14:35:14,277][20381] Updated weights for policy 1, policy_version 1289138 (0.0011) [2023-12-27 14:35:14,349][20347] Updated weights for policy 0, policy_version 1290035 (0.0006) [2023-12-27 14:35:14,398][20347] Updated weights for policy 0, policy_version 1290045 (0.0005) [2023-12-27 14:35:14,462][20347] Updated weights for policy 0, policy_version 1290055 (0.0005) [2023-12-27 14:35:14,990][20381] Updated weights for policy 1, policy_version 1289148 (0.0008) [2023-12-27 14:35:15,049][20381] Updated weights for policy 1, policy_version 1289158 (0.0008) [2023-12-27 14:35:15,066][20347] Updated weights for policy 0, policy_version 1290065 (0.0006) [2023-12-27 14:35:15,108][20381] Updated weights for policy 1, policy_version 1289168 (0.0006) [2023-12-27 14:35:15,137][20347] Updated weights for policy 0, policy_version 1290075 (0.0008) [2023-12-27 14:35:15,204][20347] Updated weights for policy 0, policy_version 1290085 (0.0007) [2023-12-27 14:35:15,260][20347] Updated weights for policy 0, policy_version 1290095 (0.0006) [2023-12-27 14:35:15,823][20381] Updated weights for policy 1, policy_version 1289178 (0.0008) [2023-12-27 14:35:15,862][20347] Updated weights for policy 0, policy_version 1290105 (0.0006) [2023-12-27 14:35:15,883][20381] Updated weights for policy 1, policy_version 1289188 (0.0011) [2023-12-27 14:35:15,925][20347] Updated weights for policy 0, policy_version 1290115 (0.0005) [2023-12-27 14:35:15,936][20381] Updated weights for policy 1, policy_version 1289198 (0.0008) [2023-12-27 14:35:15,991][20347] Updated weights for policy 0, policy_version 1290125 (0.0005) [2023-12-27 14:35:15,991][20381] Updated weights for policy 1, policy_version 1289208 (0.0007) [2023-12-27 14:35:16,537][20347] Updated weights for policy 0, policy_version 1290135 (0.0007) [2023-12-27 14:35:16,589][20381] Updated weights for policy 1, policy_version 1289218 (0.0007) [2023-12-27 14:35:16,604][20347] Updated weights for policy 0, policy_version 1290145 (0.0006) [2023-12-27 14:35:16,641][20381] Updated weights for policy 1, policy_version 1289228 (0.0006) [2023-12-27 14:35:16,664][20347] Updated weights for policy 0, policy_version 1290155 (0.0008) [2023-12-27 14:35:16,696][20381] Updated weights for policy 1, policy_version 1289238 (0.0005) [2023-12-27 14:35:17,225][20381] Updated weights for policy 1, policy_version 1289248 (0.0006) [2023-12-27 14:35:17,235][20347] Updated weights for policy 0, policy_version 1290165 (0.0007) [2023-12-27 14:35:17,281][20347] Updated weights for policy 0, policy_version 1290175 (0.0006) [2023-12-27 14:35:17,286][20381] Updated weights for policy 1, policy_version 1289258 (0.0005) [2023-12-27 14:35:17,332][20347] Updated weights for policy 0, policy_version 1290185 (0.0006) [2023-12-27 14:35:17,341][20381] Updated weights for policy 1, policy_version 1289268 (0.0005) [2023-12-27 14:35:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 660422656. Throughput: 0: 10392.6, 1: 10153.7. Samples: 660433660. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:17,363][19170] Avg episode reward: [(0, '980.450'), (1, '980.420')] [2023-12-27 14:35:17,894][20381] Updated weights for policy 1, policy_version 1289278 (0.0005) [2023-12-27 14:35:17,941][20381] Updated weights for policy 1, policy_version 1289288 (0.0009) [2023-12-27 14:35:17,986][20381] Updated weights for policy 1, policy_version 1289298 (0.0010) [2023-12-27 14:35:17,999][20347] Updated weights for policy 0, policy_version 1290195 (0.0007) [2023-12-27 14:35:18,055][20347] Updated weights for policy 0, policy_version 1290205 (0.0008) [2023-12-27 14:35:18,109][20347] Updated weights for policy 0, policy_version 1290215 (0.0007) [2023-12-27 14:35:18,637][20381] Updated weights for policy 1, policy_version 1289308 (0.0011) [2023-12-27 14:35:18,689][20381] Updated weights for policy 1, policy_version 1289318 (0.0008) [2023-12-27 14:35:18,745][20381] Updated weights for policy 1, policy_version 1289328 (0.0005) [2023-12-27 14:35:18,856][20347] Updated weights for policy 0, policy_version 1290225 (0.0007) [2023-12-27 14:35:18,916][20347] Updated weights for policy 0, policy_version 1290235 (0.0008) [2023-12-27 14:35:18,974][20347] Updated weights for policy 0, policy_version 1290245 (0.0008) [2023-12-27 14:35:19,023][20347] Updated weights for policy 0, policy_version 1290255 (0.0007) [2023-12-27 14:35:19,382][20381] Updated weights for policy 1, policy_version 1289338 (0.0006) [2023-12-27 14:35:19,434][20381] Updated weights for policy 1, policy_version 1289348 (0.0008) [2023-12-27 14:35:19,497][20381] Updated weights for policy 1, policy_version 1289358 (0.0009) [2023-12-27 14:35:19,557][20381] Updated weights for policy 1, policy_version 1289368 (0.0007) [2023-12-27 14:35:19,759][20347] Updated weights for policy 0, policy_version 1290265 (0.0007) [2023-12-27 14:35:19,824][20347] Updated weights for policy 0, policy_version 1290275 (0.0007) [2023-12-27 14:35:19,881][20347] Updated weights for policy 0, policy_version 1290285 (0.0008) [2023-12-27 14:35:20,246][20381] Updated weights for policy 1, policy_version 1289378 (0.0010) [2023-12-27 14:35:20,310][20381] Updated weights for policy 1, policy_version 1289388 (0.0011) [2023-12-27 14:35:20,367][20381] Updated weights for policy 1, policy_version 1289398 (0.0011) [2023-12-27 14:35:20,596][20347] Updated weights for policy 0, policy_version 1290295 (0.0008) [2023-12-27 14:35:20,657][20347] Updated weights for policy 0, policy_version 1290305 (0.0009) [2023-12-27 14:35:20,716][20347] Updated weights for policy 0, policy_version 1290315 (0.0011) [2023-12-27 14:35:21,103][20381] Updated weights for policy 1, policy_version 1289408 (0.0009) [2023-12-27 14:35:21,171][20381] Updated weights for policy 1, policy_version 1289418 (0.0010) [2023-12-27 14:35:21,231][20381] Updated weights for policy 1, policy_version 1289428 (0.0011) [2023-12-27 14:35:21,476][20347] Updated weights for policy 0, policy_version 1290325 (0.0009) [2023-12-27 14:35:21,534][20347] Updated weights for policy 0, policy_version 1290335 (0.0007) [2023-12-27 14:35:21,581][20347] Updated weights for policy 0, policy_version 1290345 (0.0008) [2023-12-27 14:35:21,945][20381] Updated weights for policy 1, policy_version 1289438 (0.0009) [2023-12-27 14:35:21,997][20381] Updated weights for policy 1, policy_version 1289448 (0.0005) [2023-12-27 14:35:22,054][20381] Updated weights for policy 1, policy_version 1289458 (0.0010) [2023-12-27 14:35:22,268][20347] Updated weights for policy 0, policy_version 1290355 (0.0008) [2023-12-27 14:35:22,331][20347] Updated weights for policy 0, policy_version 1290365 (0.0008) [2023-12-27 14:35:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 660520960. Throughput: 0: 10320.5, 1: 10184.9. Samples: 660495348. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:22,363][19170] Avg episode reward: [(0, '990.190'), (1, '990.240')] [2023-12-27 14:35:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001289464_330129408.pth... [2023-12-27 14:35:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001288280_329826304.pth [2023-12-27 14:35:22,399][20347] Updated weights for policy 0, policy_version 1290375 (0.0008) [2023-12-27 14:35:22,453][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001290384_330399744.pth... [2023-12-27 14:35:22,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001289168_330088448.pth [2023-12-27 14:35:22,744][20381] Updated weights for policy 1, policy_version 1289468 (0.0010) [2023-12-27 14:35:22,804][20381] Updated weights for policy 1, policy_version 1289478 (0.0011) [2023-12-27 14:35:22,864][20381] Updated weights for policy 1, policy_version 1289488 (0.0011) [2023-12-27 14:35:23,074][20347] Updated weights for policy 0, policy_version 1290385 (0.0007) [2023-12-27 14:35:23,135][20347] Updated weights for policy 0, policy_version 1290395 (0.0006) [2023-12-27 14:35:23,195][20347] Updated weights for policy 0, policy_version 1290405 (0.0007) [2023-12-27 14:35:23,251][20347] Updated weights for policy 0, policy_version 1290415 (0.0007) [2023-12-27 14:35:23,559][20381] Updated weights for policy 1, policy_version 1289498 (0.0008) [2023-12-27 14:35:23,610][20381] Updated weights for policy 1, policy_version 1289508 (0.0010) [2023-12-27 14:35:23,655][20381] Updated weights for policy 1, policy_version 1289518 (0.0010) [2023-12-27 14:35:23,701][20381] Updated weights for policy 1, policy_version 1289528 (0.0010) [2023-12-27 14:35:23,862][20347] Updated weights for policy 0, policy_version 1290425 (0.0005) [2023-12-27 14:35:23,902][20291] KL-divergence is very high: 101.8795 [2023-12-27 14:35:23,907][20347] Updated weights for policy 0, policy_version 1290435 (0.0006) [2023-12-27 14:35:23,951][20347] Updated weights for policy 0, policy_version 1290445 (0.0007) [2023-12-27 14:35:24,442][20381] Updated weights for policy 1, policy_version 1289538 (0.0007) [2023-12-27 14:35:24,498][20381] Updated weights for policy 1, policy_version 1289548 (0.0009) [2023-12-27 14:35:24,557][20381] Updated weights for policy 1, policy_version 1289558 (0.0008) [2023-12-27 14:35:24,612][20347] Updated weights for policy 0, policy_version 1290455 (0.0009) [2023-12-27 14:35:24,680][20347] Updated weights for policy 0, policy_version 1290465 (0.0009) [2023-12-27 14:35:24,739][20347] Updated weights for policy 0, policy_version 1290475 (0.0007) [2023-12-27 14:35:25,231][20381] Updated weights for policy 1, policy_version 1289568 (0.0007) [2023-12-27 14:35:25,292][20381] Updated weights for policy 1, policy_version 1289578 (0.0007) [2023-12-27 14:35:25,344][20381] Updated weights for policy 1, policy_version 1289588 (0.0007) [2023-12-27 14:35:25,404][20347] Updated weights for policy 0, policy_version 1290485 (0.0009) [2023-12-27 14:35:25,457][20347] Updated weights for policy 0, policy_version 1290495 (0.0006) [2023-12-27 14:35:25,511][20347] Updated weights for policy 0, policy_version 1290505 (0.0008) [2023-12-27 14:35:26,052][20347] Updated weights for policy 0, policy_version 1290515 (0.0009) [2023-12-27 14:35:26,061][20381] Updated weights for policy 1, policy_version 1289598 (0.0007) [2023-12-27 14:35:26,107][20347] Updated weights for policy 0, policy_version 1290525 (0.0005) [2023-12-27 14:35:26,115][20381] Updated weights for policy 1, policy_version 1289608 (0.0008) [2023-12-27 14:35:26,156][20347] Updated weights for policy 0, policy_version 1290535 (0.0005) [2023-12-27 14:35:26,161][20381] Updated weights for policy 1, policy_version 1289618 (0.0008) [2023-12-27 14:35:26,765][20347] Updated weights for policy 0, policy_version 1290545 (0.0006) [2023-12-27 14:35:26,818][20347] Updated weights for policy 0, policy_version 1290555 (0.0010) [2023-12-27 14:35:26,871][20347] Updated weights for policy 0, policy_version 1290565 (0.0010) [2023-12-27 14:35:26,891][20381] Updated weights for policy 1, policy_version 1289628 (0.0007) [2023-12-27 14:35:26,925][20347] Updated weights for policy 0, policy_version 1290575 (0.0010) [2023-12-27 14:35:26,952][20381] Updated weights for policy 1, policy_version 1289638 (0.0006) [2023-12-27 14:35:27,002][20381] Updated weights for policy 1, policy_version 1289648 (0.0007) [2023-12-27 14:35:27,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 660627456. Throughput: 0: 10359.8, 1: 10045.9. Samples: 660615860. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:27,363][19170] Avg episode reward: [(0, '941.360'), (1, '990.240')] [2023-12-27 14:35:27,597][20347] Updated weights for policy 0, policy_version 1290585 (0.0010) [2023-12-27 14:35:27,638][20381] Updated weights for policy 1, policy_version 1289658 (0.0007) [2023-12-27 14:35:27,653][20347] Updated weights for policy 0, policy_version 1290595 (0.0010) [2023-12-27 14:35:27,682][20381] Updated weights for policy 1, policy_version 1289668 (0.0007) [2023-12-27 14:35:27,716][20347] Updated weights for policy 0, policy_version 1290605 (0.0010) [2023-12-27 14:35:27,727][20381] Updated weights for policy 1, policy_version 1289678 (0.0007) [2023-12-27 14:35:27,776][20381] Updated weights for policy 1, policy_version 1289688 (0.0007) [2023-12-27 14:35:28,362][20347] Updated weights for policy 0, policy_version 1290615 (0.0009) [2023-12-27 14:35:28,418][20347] Updated weights for policy 0, policy_version 1290625 (0.0008) [2023-12-27 14:35:28,476][20347] Updated weights for policy 0, policy_version 1290635 (0.0008) [2023-12-27 14:35:28,487][20381] Updated weights for policy 1, policy_version 1289698 (0.0006) [2023-12-27 14:35:28,539][20381] Updated weights for policy 1, policy_version 1289708 (0.0007) [2023-12-27 14:35:28,588][20381] Updated weights for policy 1, policy_version 1289718 (0.0007) [2023-12-27 14:35:29,132][20347] Updated weights for policy 0, policy_version 1290645 (0.0007) [2023-12-27 14:35:29,180][20347] Updated weights for policy 0, policy_version 1290655 (0.0005) [2023-12-27 14:35:29,233][20347] Updated weights for policy 0, policy_version 1290665 (0.0006) [2023-12-27 14:35:29,306][20381] Updated weights for policy 1, policy_version 1289728 (0.0006) [2023-12-27 14:35:29,373][20381] Updated weights for policy 1, policy_version 1289738 (0.0008) [2023-12-27 14:35:29,432][20381] Updated weights for policy 1, policy_version 1289748 (0.0007) [2023-12-27 14:35:29,946][20347] Updated weights for policy 0, policy_version 1290675 (0.0008) [2023-12-27 14:35:30,003][20347] Updated weights for policy 0, policy_version 1290685 (0.0008) [2023-12-27 14:35:30,053][20347] Updated weights for policy 0, policy_version 1290695 (0.0008) [2023-12-27 14:35:30,130][20381] Updated weights for policy 1, policy_version 1289758 (0.0007) [2023-12-27 14:35:30,206][20381] Updated weights for policy 1, policy_version 1289768 (0.0008) [2023-12-27 14:35:30,258][20381] Updated weights for policy 1, policy_version 1289778 (0.0007) [2023-12-27 14:35:30,737][20347] Updated weights for policy 0, policy_version 1290705 (0.0008) [2023-12-27 14:35:30,794][20347] Updated weights for policy 0, policy_version 1290715 (0.0010) [2023-12-27 14:35:30,848][20347] Updated weights for policy 0, policy_version 1290725 (0.0010) [2023-12-27 14:35:30,902][20347] Updated weights for policy 0, policy_version 1290735 (0.0010) [2023-12-27 14:35:30,923][20381] Updated weights for policy 1, policy_version 1289788 (0.0009) [2023-12-27 14:35:30,981][20381] Updated weights for policy 1, policy_version 1289798 (0.0010) [2023-12-27 14:35:31,041][20381] Updated weights for policy 1, policy_version 1289808 (0.0009) [2023-12-27 14:35:31,598][20347] Updated weights for policy 0, policy_version 1290745 (0.0009) [2023-12-27 14:35:31,659][20347] Updated weights for policy 0, policy_version 1290755 (0.0009) [2023-12-27 14:35:31,720][20347] Updated weights for policy 0, policy_version 1290765 (0.0010) [2023-12-27 14:35:31,761][20381] Updated weights for policy 1, policy_version 1289818 (0.0007) [2023-12-27 14:35:31,818][20381] Updated weights for policy 1, policy_version 1289828 (0.0007) [2023-12-27 14:35:31,876][20381] Updated weights for policy 1, policy_version 1289838 (0.0008) [2023-12-27 14:35:31,932][20381] Updated weights for policy 1, policy_version 1289848 (0.0008) [2023-12-27 14:35:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 660725760. Throughput: 0: 10397.6, 1: 10039.1. Samples: 660738980. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:32,363][19170] Avg episode reward: [(0, '931.490'), (1, '1000.000')] [2023-12-27 14:35:32,406][20347] Updated weights for policy 0, policy_version 1290775 (0.0010) [2023-12-27 14:35:32,468][20347] Updated weights for policy 0, policy_version 1290785 (0.0007) [2023-12-27 14:35:32,526][20347] Updated weights for policy 0, policy_version 1290795 (0.0008) [2023-12-27 14:35:32,611][20381] Updated weights for policy 1, policy_version 1289858 (0.0008) [2023-12-27 14:35:32,671][20381] Updated weights for policy 1, policy_version 1289868 (0.0011) [2023-12-27 14:35:32,731][20381] Updated weights for policy 1, policy_version 1289878 (0.0011) [2023-12-27 14:35:33,097][20347] Updated weights for policy 0, policy_version 1290805 (0.0007) [2023-12-27 14:35:33,153][20347] Updated weights for policy 0, policy_version 1290815 (0.0010) [2023-12-27 14:35:33,208][20347] Updated weights for policy 0, policy_version 1290825 (0.0009) [2023-12-27 14:35:33,399][20381] Updated weights for policy 1, policy_version 1289888 (0.0007) [2023-12-27 14:35:33,451][20381] Updated weights for policy 1, policy_version 1289898 (0.0008) [2023-12-27 14:35:33,501][20381] Updated weights for policy 1, policy_version 1289908 (0.0007) [2023-12-27 14:35:33,775][20347] Updated weights for policy 0, policy_version 1290835 (0.0006) [2023-12-27 14:35:33,831][20347] Updated weights for policy 0, policy_version 1290845 (0.0008) [2023-12-27 14:35:33,881][20347] Updated weights for policy 0, policy_version 1290855 (0.0008) [2023-12-27 14:35:34,211][20381] Updated weights for policy 1, policy_version 1289918 (0.0007) [2023-12-27 14:35:34,263][20381] Updated weights for policy 1, policy_version 1289928 (0.0007) [2023-12-27 14:35:34,318][20381] Updated weights for policy 1, policy_version 1289938 (0.0007) [2023-12-27 14:35:34,527][20347] Updated weights for policy 0, policy_version 1290865 (0.0009) [2023-12-27 14:35:34,591][20347] Updated weights for policy 0, policy_version 1290875 (0.0006) [2023-12-27 14:35:34,646][20347] Updated weights for policy 0, policy_version 1290885 (0.0007) [2023-12-27 14:35:34,705][20347] Updated weights for policy 0, policy_version 1290895 (0.0008) [2023-12-27 14:35:35,054][20381] Updated weights for policy 1, policy_version 1289948 (0.0007) [2023-12-27 14:35:35,112][20381] Updated weights for policy 1, policy_version 1289958 (0.0007) [2023-12-27 14:35:35,168][20381] Updated weights for policy 1, policy_version 1289968 (0.0007) [2023-12-27 14:35:35,367][20347] Updated weights for policy 0, policy_version 1290905 (0.0010) [2023-12-27 14:35:35,427][20347] Updated weights for policy 0, policy_version 1290915 (0.0010) [2023-12-27 14:35:35,482][20347] Updated weights for policy 0, policy_version 1290925 (0.0010) [2023-12-27 14:35:35,846][20381] Updated weights for policy 1, policy_version 1289978 (0.0008) [2023-12-27 14:35:35,893][20381] Updated weights for policy 1, policy_version 1289988 (0.0007) [2023-12-27 14:35:35,951][20381] Updated weights for policy 1, policy_version 1289998 (0.0007) [2023-12-27 14:35:36,005][20381] Updated weights for policy 1, policy_version 1290008 (0.0007) [2023-12-27 14:35:36,086][20347] Updated weights for policy 0, policy_version 1290935 (0.0010) [2023-12-27 14:35:36,153][20347] Updated weights for policy 0, policy_version 1290945 (0.0010) [2023-12-27 14:35:36,209][20347] Updated weights for policy 0, policy_version 1290955 (0.0010) [2023-12-27 14:35:36,736][20381] Updated weights for policy 1, policy_version 1290018 (0.0006) [2023-12-27 14:35:36,801][20381] Updated weights for policy 1, policy_version 1290028 (0.0006) [2023-12-27 14:35:36,855][20381] Updated weights for policy 1, policy_version 1290038 (0.0008) [2023-12-27 14:35:36,889][20347] Updated weights for policy 0, policy_version 1290965 (0.0010) [2023-12-27 14:35:36,948][20347] Updated weights for policy 0, policy_version 1290975 (0.0010) [2023-12-27 14:35:36,998][20347] Updated weights for policy 0, policy_version 1290985 (0.0011) [2023-12-27 14:35:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 660832256. Throughput: 0: 10431.1, 1: 10037.0. Samples: 660801852. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:37,363][19170] Avg episode reward: [(0, '922.240'), (1, '980.540')] [2023-12-27 14:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001290040_330276864.pth... [2023-12-27 14:35:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001290992_330555392.pth... [2023-12-27 14:35:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001289776_330244096.pth [2023-12-27 14:35:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001288856_329973760.pth [2023-12-27 14:35:37,478][20381] Updated weights for policy 1, policy_version 1290048 (0.0006) [2023-12-27 14:35:37,540][20381] Updated weights for policy 1, policy_version 1290058 (0.0006) [2023-12-27 14:35:37,594][20381] Updated weights for policy 1, policy_version 1290068 (0.0006) [2023-12-27 14:35:37,695][20347] Updated weights for policy 0, policy_version 1290995 (0.0010) [2023-12-27 14:35:37,748][20347] Updated weights for policy 0, policy_version 1291005 (0.0010) [2023-12-27 14:35:37,796][20347] Updated weights for policy 0, policy_version 1291015 (0.0010) [2023-12-27 14:35:38,203][20381] Updated weights for policy 1, policy_version 1290078 (0.0007) [2023-12-27 14:35:38,264][20381] Updated weights for policy 1, policy_version 1290088 (0.0006) [2023-12-27 14:35:38,329][20381] Updated weights for policy 1, policy_version 1290098 (0.0006) [2023-12-27 14:35:38,508][20347] Updated weights for policy 0, policy_version 1291025 (0.0010) [2023-12-27 14:35:38,572][20347] Updated weights for policy 0, policy_version 1291035 (0.0008) [2023-12-27 14:35:38,635][20347] Updated weights for policy 0, policy_version 1291045 (0.0007) [2023-12-27 14:35:38,693][20347] Updated weights for policy 0, policy_version 1291055 (0.0006) [2023-12-27 14:35:38,997][20381] Updated weights for policy 1, policy_version 1290108 (0.0008) [2023-12-27 14:35:39,059][20381] Updated weights for policy 1, policy_version 1290118 (0.0009) [2023-12-27 14:35:39,123][20381] Updated weights for policy 1, policy_version 1290128 (0.0010) [2023-12-27 14:35:39,387][20347] Updated weights for policy 0, policy_version 1291065 (0.0008) [2023-12-27 14:35:39,461][20347] Updated weights for policy 0, policy_version 1291075 (0.0009) [2023-12-27 14:35:39,524][20347] Updated weights for policy 0, policy_version 1291085 (0.0008) [2023-12-27 14:35:39,749][20381] Updated weights for policy 1, policy_version 1290138 (0.0009) [2023-12-27 14:35:39,812][20381] Updated weights for policy 1, policy_version 1290148 (0.0006) [2023-12-27 14:35:39,882][20381] Updated weights for policy 1, policy_version 1290158 (0.0007) [2023-12-27 14:35:39,948][20381] Updated weights for policy 1, policy_version 1290168 (0.0007) [2023-12-27 14:35:40,174][20347] Updated weights for policy 0, policy_version 1291095 (0.0007) [2023-12-27 14:35:40,223][20347] Updated weights for policy 0, policy_version 1291105 (0.0008) [2023-12-27 14:35:40,284][20347] Updated weights for policy 0, policy_version 1291115 (0.0006) [2023-12-27 14:35:40,585][20381] Updated weights for policy 1, policy_version 1290178 (0.0008) [2023-12-27 14:35:40,635][20381] Updated weights for policy 1, policy_version 1290188 (0.0008) [2023-12-27 14:35:40,687][20381] Updated weights for policy 1, policy_version 1290198 (0.0007) [2023-12-27 14:35:40,988][20347] Updated weights for policy 0, policy_version 1291125 (0.0007) [2023-12-27 14:35:41,049][20347] Updated weights for policy 0, policy_version 1291135 (0.0008) [2023-12-27 14:35:41,114][20347] Updated weights for policy 0, policy_version 1291145 (0.0008) [2023-12-27 14:35:41,470][20381] Updated weights for policy 1, policy_version 1290208 (0.0008) [2023-12-27 14:35:41,525][20381] Updated weights for policy 1, policy_version 1290218 (0.0008) [2023-12-27 14:35:41,585][20381] Updated weights for policy 1, policy_version 1290228 (0.0008) [2023-12-27 14:35:41,831][20347] Updated weights for policy 0, policy_version 1291155 (0.0008) [2023-12-27 14:35:41,885][20347] Updated weights for policy 0, policy_version 1291165 (0.0010) [2023-12-27 14:35:41,938][20347] Updated weights for policy 0, policy_version 1291175 (0.0010) [2023-12-27 14:35:42,338][20381] Updated weights for policy 1, policy_version 1290238 (0.0007) [2023-12-27 14:35:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 660930560. Throughput: 0: 10420.4, 1: 10126.9. Samples: 660925500. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:42,366][19170] Avg episode reward: [(0, '941.660'), (1, '970.760')] [2023-12-27 14:35:42,403][20381] Updated weights for policy 1, policy_version 1290248 (0.0008) [2023-12-27 14:35:42,462][20381] Updated weights for policy 1, policy_version 1290258 (0.0008) [2023-12-27 14:35:42,638][20347] Updated weights for policy 0, policy_version 1291185 (0.0010) [2023-12-27 14:35:42,691][20347] Updated weights for policy 0, policy_version 1291195 (0.0010) [2023-12-27 14:35:42,743][20347] Updated weights for policy 0, policy_version 1291205 (0.0010) [2023-12-27 14:35:42,798][20347] Updated weights for policy 0, policy_version 1291215 (0.0010) [2023-12-27 14:35:43,111][20381] Updated weights for policy 1, policy_version 1290268 (0.0006) [2023-12-27 14:35:43,166][20381] Updated weights for policy 1, policy_version 1290278 (0.0005) [2023-12-27 14:35:43,213][20381] Updated weights for policy 1, policy_version 1290288 (0.0008) [2023-12-27 14:35:43,456][20347] Updated weights for policy 0, policy_version 1291225 (0.0010) [2023-12-27 14:35:43,508][20347] Updated weights for policy 0, policy_version 1291235 (0.0010) [2023-12-27 14:35:43,560][20347] Updated weights for policy 0, policy_version 1291245 (0.0010) [2023-12-27 14:35:43,875][20381] Updated weights for policy 1, policy_version 1290298 (0.0007) [2023-12-27 14:35:43,932][20381] Updated weights for policy 1, policy_version 1290308 (0.0008) [2023-12-27 14:35:43,994][20381] Updated weights for policy 1, policy_version 1290318 (0.0008) [2023-12-27 14:35:44,048][20381] Updated weights for policy 1, policy_version 1290328 (0.0007) [2023-12-27 14:35:44,252][20347] Updated weights for policy 0, policy_version 1291255 (0.0009) [2023-12-27 14:35:44,304][20347] Updated weights for policy 0, policy_version 1291265 (0.0010) [2023-12-27 14:35:44,352][20347] Updated weights for policy 0, policy_version 1291275 (0.0010) [2023-12-27 14:35:44,765][20381] Updated weights for policy 1, policy_version 1290338 (0.0008) [2023-12-27 14:35:44,830][20381] Updated weights for policy 1, policy_version 1290348 (0.0007) [2023-12-27 14:35:44,889][20381] Updated weights for policy 1, policy_version 1290358 (0.0008) [2023-12-27 14:35:45,027][20347] Updated weights for policy 0, policy_version 1291285 (0.0010) [2023-12-27 14:35:45,092][20347] Updated weights for policy 0, policy_version 1291295 (0.0011) [2023-12-27 14:35:45,149][20347] Updated weights for policy 0, policy_version 1291305 (0.0011) [2023-12-27 14:35:45,564][20381] Updated weights for policy 1, policy_version 1290368 (0.0006) [2023-12-27 14:35:45,619][20381] Updated weights for policy 1, policy_version 1290378 (0.0007) [2023-12-27 14:35:45,676][20381] Updated weights for policy 1, policy_version 1290388 (0.0007) [2023-12-27 14:35:45,769][20347] Updated weights for policy 0, policy_version 1291315 (0.0008) [2023-12-27 14:35:45,827][20347] Updated weights for policy 0, policy_version 1291325 (0.0010) [2023-12-27 14:35:45,882][20347] Updated weights for policy 0, policy_version 1291335 (0.0010) [2023-12-27 14:35:46,341][20381] Updated weights for policy 1, policy_version 1290398 (0.0008) [2023-12-27 14:35:46,399][20381] Updated weights for policy 1, policy_version 1290408 (0.0010) [2023-12-27 14:35:46,458][20381] Updated weights for policy 1, policy_version 1290418 (0.0011) [2023-12-27 14:35:46,541][20347] Updated weights for policy 0, policy_version 1291345 (0.0010) [2023-12-27 14:35:46,601][20347] Updated weights for policy 0, policy_version 1291355 (0.0005) [2023-12-27 14:35:46,665][20347] Updated weights for policy 0, policy_version 1291365 (0.0005) [2023-12-27 14:35:46,723][20347] Updated weights for policy 0, policy_version 1291375 (0.0011) [2023-12-27 14:35:47,062][20381] Updated weights for policy 1, policy_version 1290428 (0.0008) [2023-12-27 14:35:47,112][20381] Updated weights for policy 1, policy_version 1290438 (0.0010) [2023-12-27 14:35:47,157][20381] Updated weights for policy 1, policy_version 1290448 (0.0011) [2023-12-27 14:35:47,308][20347] Updated weights for policy 0, policy_version 1291385 (0.0005) [2023-12-27 14:35:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 661037056. Throughput: 0: 10443.7, 1: 10175.0. Samples: 661048540. Policy #0 lag: (min: 9.0, avg: 32.8, max: 41.0) [2023-12-27 14:35:47,363][19170] Avg episode reward: [(0, '951.380'), (1, '990.220')] [2023-12-27 14:35:47,367][20347] Updated weights for policy 0, policy_version 1291395 (0.0005) [2023-12-27 14:35:47,425][20347] Updated weights for policy 0, policy_version 1291405 (0.0006) [2023-12-27 14:35:47,842][20381] Updated weights for policy 1, policy_version 1290458 (0.0010) [2023-12-27 14:35:47,894][20381] Updated weights for policy 1, policy_version 1290468 (0.0010) [2023-12-27 14:35:47,945][20381] Updated weights for policy 1, policy_version 1290478 (0.0010) [2023-12-27 14:35:47,990][20381] Updated weights for policy 1, policy_version 1290488 (0.0010) [2023-12-27 14:35:48,069][20347] Updated weights for policy 0, policy_version 1291415 (0.0007) [2023-12-27 14:35:48,125][20347] Updated weights for policy 0, policy_version 1291425 (0.0007) [2023-12-27 14:35:48,178][20347] Updated weights for policy 0, policy_version 1291435 (0.0007) [2023-12-27 14:35:48,728][20381] Updated weights for policy 1, policy_version 1290498 (0.0011) [2023-12-27 14:35:48,778][20381] Updated weights for policy 1, policy_version 1290508 (0.0011) [2023-12-27 14:35:48,815][20347] Updated weights for policy 0, policy_version 1291445 (0.0006) [2023-12-27 14:35:48,831][20381] Updated weights for policy 1, policy_version 1290518 (0.0011) [2023-12-27 14:35:48,875][20347] Updated weights for policy 0, policy_version 1291455 (0.0007) [2023-12-27 14:35:48,941][20347] Updated weights for policy 0, policy_version 1291465 (0.0008) [2023-12-27 14:35:49,538][20347] Updated weights for policy 0, policy_version 1291475 (0.0007) [2023-12-27 14:35:49,556][20381] Updated weights for policy 1, policy_version 1290528 (0.0008) [2023-12-27 14:35:49,600][20347] Updated weights for policy 0, policy_version 1291485 (0.0009) [2023-12-27 14:35:49,618][20381] Updated weights for policy 1, policy_version 1290538 (0.0009) [2023-12-27 14:35:49,658][20347] Updated weights for policy 0, policy_version 1291495 (0.0010) [2023-12-27 14:35:49,677][20381] Updated weights for policy 1, policy_version 1290548 (0.0011) [2023-12-27 14:35:50,357][20347] Updated weights for policy 0, policy_version 1291505 (0.0010) [2023-12-27 14:35:50,384][20381] Updated weights for policy 1, policy_version 1290558 (0.0011) [2023-12-27 14:35:50,426][20347] Updated weights for policy 0, policy_version 1291515 (0.0007) [2023-12-27 14:35:50,444][20381] Updated weights for policy 1, policy_version 1290568 (0.0010) [2023-12-27 14:35:50,484][20347] Updated weights for policy 0, policy_version 1291525 (0.0008) [2023-12-27 14:35:50,500][20381] Updated weights for policy 1, policy_version 1290578 (0.0011) [2023-12-27 14:35:50,545][20347] Updated weights for policy 0, policy_version 1291535 (0.0007) [2023-12-27 14:35:51,177][20381] Updated weights for policy 1, policy_version 1290588 (0.0009) [2023-12-27 14:35:51,179][20347] Updated weights for policy 0, policy_version 1291545 (0.0009) [2023-12-27 14:35:51,234][20347] Updated weights for policy 0, policy_version 1291555 (0.0010) [2023-12-27 14:35:51,236][20381] Updated weights for policy 1, policy_version 1290598 (0.0006) [2023-12-27 14:35:51,292][20347] Updated weights for policy 0, policy_version 1291565 (0.0007) [2023-12-27 14:35:51,295][20381] Updated weights for policy 1, policy_version 1290608 (0.0006) [2023-12-27 14:35:52,017][20347] Updated weights for policy 0, policy_version 1291575 (0.0010) [2023-12-27 14:35:52,036][20381] Updated weights for policy 1, policy_version 1290618 (0.0008) [2023-12-27 14:35:52,074][20347] Updated weights for policy 0, policy_version 1291585 (0.0008) [2023-12-27 14:35:52,091][20381] Updated weights for policy 1, policy_version 1290628 (0.0006) [2023-12-27 14:35:52,133][20347] Updated weights for policy 0, policy_version 1291595 (0.0010) [2023-12-27 14:35:52,142][20381] Updated weights for policy 1, policy_version 1290638 (0.0005) [2023-12-27 14:35:52,197][20381] Updated weights for policy 1, policy_version 1290648 (0.0007) [2023-12-27 14:35:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 661143552. Throughput: 0: 10449.0, 1: 10166.0. Samples: 661110004. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:35:52,363][19170] Avg episode reward: [(0, '941.460'), (1, '1000.000')] [2023-12-27 14:35:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001290648_330432512.pth... [2023-12-27 14:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001291600_330711040.pth... [2023-12-27 14:35:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001289464_330129408.pth [2023-12-27 14:35:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001290384_330399744.pth [2023-12-27 14:35:52,788][20347] Updated weights for policy 0, policy_version 1291605 (0.0010) [2023-12-27 14:35:52,806][20291] KL-divergence is very high: 116.9728 [2023-12-27 14:35:52,848][20347] Updated weights for policy 0, policy_version 1291615 (0.0010) [2023-12-27 14:35:52,853][20291] KL-divergence is very high: 179.9899 [2023-12-27 14:35:52,898][20381] Updated weights for policy 1, policy_version 1290658 (0.0007) [2023-12-27 14:35:52,899][20291] KL-divergence is very high: 134.2982 [2023-12-27 14:35:52,904][20347] Updated weights for policy 0, policy_version 1291625 (0.0010) [2023-12-27 14:35:52,949][20381] Updated weights for policy 1, policy_version 1290668 (0.0005) [2023-12-27 14:35:53,003][20381] Updated weights for policy 1, policy_version 1290678 (0.0008) [2023-12-27 14:35:53,594][20347] Updated weights for policy 0, policy_version 1291635 (0.0010) [2023-12-27 14:35:53,650][20381] Updated weights for policy 1, policy_version 1290688 (0.0007) [2023-12-27 14:35:53,656][20347] Updated weights for policy 0, policy_version 1291645 (0.0010) [2023-12-27 14:35:53,708][20381] Updated weights for policy 1, policy_version 1290698 (0.0007) [2023-12-27 14:35:53,717][20347] Updated weights for policy 0, policy_version 1291655 (0.0010) [2023-12-27 14:35:53,770][20381] Updated weights for policy 1, policy_version 1290708 (0.0010) [2023-12-27 14:35:54,397][20381] Updated weights for policy 1, policy_version 1290718 (0.0007) [2023-12-27 14:35:54,421][20347] Updated weights for policy 0, policy_version 1291665 (0.0007) [2023-12-27 14:35:54,454][20381] Updated weights for policy 1, policy_version 1290728 (0.0007) [2023-12-27 14:35:54,472][20347] Updated weights for policy 0, policy_version 1291675 (0.0006) [2023-12-27 14:35:54,515][20381] Updated weights for policy 1, policy_version 1290738 (0.0007) [2023-12-27 14:35:54,533][20347] Updated weights for policy 0, policy_version 1291685 (0.0008) [2023-12-27 14:35:54,591][20347] Updated weights for policy 0, policy_version 1291695 (0.0008) [2023-12-27 14:35:55,238][20347] Updated weights for policy 0, policy_version 1291705 (0.0006) [2023-12-27 14:35:55,246][20381] Updated weights for policy 1, policy_version 1290748 (0.0007) [2023-12-27 14:35:55,295][20347] Updated weights for policy 0, policy_version 1291715 (0.0007) [2023-12-27 14:35:55,300][20381] Updated weights for policy 1, policy_version 1290758 (0.0009) [2023-12-27 14:35:55,352][20347] Updated weights for policy 0, policy_version 1291725 (0.0006) [2023-12-27 14:35:55,355][20381] Updated weights for policy 1, policy_version 1290768 (0.0008) [2023-12-27 14:35:55,931][20381] Updated weights for policy 1, policy_version 1290778 (0.0005) [2023-12-27 14:35:55,978][20381] Updated weights for policy 1, policy_version 1290788 (0.0005) [2023-12-27 14:35:56,027][20347] Updated weights for policy 0, policy_version 1291735 (0.0009) [2023-12-27 14:35:56,036][20381] Updated weights for policy 1, policy_version 1290798 (0.0006) [2023-12-27 14:35:56,084][20381] Updated weights for policy 1, policy_version 1290808 (0.0006) [2023-12-27 14:35:56,086][20347] Updated weights for policy 0, policy_version 1291745 (0.0010) [2023-12-27 14:35:56,140][20347] Updated weights for policy 0, policy_version 1291755 (0.0010) [2023-12-27 14:35:56,650][20381] Updated weights for policy 1, policy_version 1290818 (0.0007) [2023-12-27 14:35:56,702][20381] Updated weights for policy 1, policy_version 1290828 (0.0008) [2023-12-27 14:35:56,759][20381] Updated weights for policy 1, policy_version 1290838 (0.0006) [2023-12-27 14:35:56,819][20347] Updated weights for policy 0, policy_version 1291768 (0.0010) [2023-12-27 14:35:56,868][20347] Updated weights for policy 0, policy_version 1291778 (0.0011) [2023-12-27 14:35:56,918][20347] Updated weights for policy 0, policy_version 1291788 (0.0011) [2023-12-27 14:35:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 661241856. Throughput: 0: 10412.6, 1: 10214.0. Samples: 661233736. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:35:57,363][19170] Avg episode reward: [(0, '932.090'), (1, '990.240')] [2023-12-27 14:35:57,383][20381] Updated weights for policy 1, policy_version 1290848 (0.0006) [2023-12-27 14:35:57,434][20381] Updated weights for policy 1, policy_version 1290858 (0.0005) [2023-12-27 14:35:57,492][20381] Updated weights for policy 1, policy_version 1290868 (0.0005) [2023-12-27 14:35:57,631][20347] Updated weights for policy 0, policy_version 1291798 (0.0007) [2023-12-27 14:35:57,681][20347] Updated weights for policy 0, policy_version 1291808 (0.0007) [2023-12-27 14:35:57,739][20347] Updated weights for policy 0, policy_version 1291818 (0.0008) [2023-12-27 14:35:58,050][20381] Updated weights for policy 1, policy_version 1290878 (0.0008) [2023-12-27 14:35:58,105][20381] Updated weights for policy 1, policy_version 1290888 (0.0010) [2023-12-27 14:35:58,154][20381] Updated weights for policy 1, policy_version 1290898 (0.0010) [2023-12-27 14:35:58,387][20347] Updated weights for policy 0, policy_version 1291828 (0.0007) [2023-12-27 14:35:58,452][20347] Updated weights for policy 0, policy_version 1291838 (0.0008) [2023-12-27 14:35:58,517][20347] Updated weights for policy 0, policy_version 1291848 (0.0011) [2023-12-27 14:35:58,949][20381] Updated weights for policy 1, policy_version 1290908 (0.0009) [2023-12-27 14:35:59,002][20381] Updated weights for policy 1, policy_version 1290918 (0.0011) [2023-12-27 14:35:59,058][20381] Updated weights for policy 1, policy_version 1290928 (0.0010) [2023-12-27 14:35:59,285][20347] Updated weights for policy 0, policy_version 1291858 (0.0010) [2023-12-27 14:35:59,349][20347] Updated weights for policy 0, policy_version 1291868 (0.0008) [2023-12-27 14:35:59,417][20347] Updated weights for policy 0, policy_version 1291878 (0.0008) [2023-12-27 14:35:59,475][20347] Updated weights for policy 0, policy_version 1291888 (0.0008) [2023-12-27 14:35:59,782][20381] Updated weights for policy 1, policy_version 1290938 (0.0008) [2023-12-27 14:35:59,838][20381] Updated weights for policy 1, policy_version 1290948 (0.0009) [2023-12-27 14:35:59,901][20381] Updated weights for policy 1, policy_version 1290958 (0.0008) [2023-12-27 14:35:59,962][20381] Updated weights for policy 1, policy_version 1290968 (0.0008) [2023-12-27 14:36:00,216][20347] Updated weights for policy 0, policy_version 1291898 (0.0005) [2023-12-27 14:36:00,277][20347] Updated weights for policy 0, policy_version 1291908 (0.0005) [2023-12-27 14:36:00,337][20347] Updated weights for policy 0, policy_version 1291918 (0.0005) [2023-12-27 14:36:00,653][20381] Updated weights for policy 1, policy_version 1290978 (0.0007) [2023-12-27 14:36:00,703][20381] Updated weights for policy 1, policy_version 1290988 (0.0007) [2023-12-27 14:36:00,749][20381] Updated weights for policy 1, policy_version 1290998 (0.0007) [2023-12-27 14:36:00,945][20347] Updated weights for policy 0, policy_version 1291928 (0.0009) [2023-12-27 14:36:00,996][20347] Updated weights for policy 0, policy_version 1291938 (0.0010) [2023-12-27 14:36:01,050][20347] Updated weights for policy 0, policy_version 1291948 (0.0009) [2023-12-27 14:36:01,487][20381] Updated weights for policy 1, policy_version 1291008 (0.0008) [2023-12-27 14:36:01,538][20381] Updated weights for policy 1, policy_version 1291018 (0.0008) [2023-12-27 14:36:01,598][20381] Updated weights for policy 1, policy_version 1291028 (0.0008) [2023-12-27 14:36:01,811][20347] Updated weights for policy 0, policy_version 1291958 (0.0009) [2023-12-27 14:36:01,873][20347] Updated weights for policy 0, policy_version 1291968 (0.0010) [2023-12-27 14:36:01,931][20347] Updated weights for policy 0, policy_version 1291978 (0.0010) [2023-12-27 14:36:02,244][20381] Updated weights for policy 1, policy_version 1291038 (0.0009) [2023-12-27 14:36:02,297][20381] Updated weights for policy 1, policy_version 1291048 (0.0009) [2023-12-27 14:36:02,353][20381] Updated weights for policy 1, policy_version 1291058 (0.0009) [2023-12-27 14:36:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 661340160. Throughput: 0: 10305.7, 1: 10175.3. Samples: 661355308. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:02,363][19170] Avg episode reward: [(0, '932.440'), (1, '990.240')] [2023-12-27 14:36:02,591][20347] Updated weights for policy 0, policy_version 1291988 (0.0009) [2023-12-27 14:36:02,660][20347] Updated weights for policy 0, policy_version 1291998 (0.0011) [2023-12-27 14:36:02,728][20347] Updated weights for policy 0, policy_version 1292008 (0.0006) [2023-12-27 14:36:03,127][20381] Updated weights for policy 1, policy_version 1291068 (0.0008) [2023-12-27 14:36:03,180][20381] Updated weights for policy 1, policy_version 1291078 (0.0008) [2023-12-27 14:36:03,238][20381] Updated weights for policy 1, policy_version 1291088 (0.0008) [2023-12-27 14:36:03,384][20347] Updated weights for policy 0, policy_version 1292018 (0.0006) [2023-12-27 14:36:03,432][20347] Updated weights for policy 0, policy_version 1292028 (0.0008) [2023-12-27 14:36:03,474][20291] KL-divergence is very high: 408.9274 [2023-12-27 14:36:03,486][20347] Updated weights for policy 0, policy_version 1292038 (0.0007) [2023-12-27 14:36:03,522][20291] KL-divergence is very high: 189.8795 [2023-12-27 14:36:03,545][20347] Updated weights for policy 0, policy_version 1292048 (0.0007) [2023-12-27 14:36:03,879][20381] Updated weights for policy 1, policy_version 1291098 (0.0008) [2023-12-27 14:36:03,933][20381] Updated weights for policy 1, policy_version 1291108 (0.0007) [2023-12-27 14:36:03,994][20381] Updated weights for policy 1, policy_version 1291118 (0.0008) [2023-12-27 14:36:04,060][20381] Updated weights for policy 1, policy_version 1291128 (0.0008) [2023-12-27 14:36:04,286][20347] Updated weights for policy 0, policy_version 1292058 (0.0010) [2023-12-27 14:36:04,343][20347] Updated weights for policy 0, policy_version 1292068 (0.0011) [2023-12-27 14:36:04,399][20347] Updated weights for policy 0, policy_version 1292078 (0.0011) [2023-12-27 14:36:04,723][20381] Updated weights for policy 1, policy_version 1291138 (0.0008) [2023-12-27 14:36:04,787][20381] Updated weights for policy 1, policy_version 1291148 (0.0007) [2023-12-27 14:36:04,841][20381] Updated weights for policy 1, policy_version 1291158 (0.0007) [2023-12-27 14:36:05,078][20347] Updated weights for policy 0, policy_version 1292088 (0.0010) [2023-12-27 14:36:05,132][20347] Updated weights for policy 0, policy_version 1292098 (0.0010) [2023-12-27 14:36:05,198][20347] Updated weights for policy 0, policy_version 1292108 (0.0010) [2023-12-27 14:36:05,492][20381] Updated weights for policy 1, policy_version 1291168 (0.0007) [2023-12-27 14:36:05,536][20381] Updated weights for policy 1, policy_version 1291178 (0.0007) [2023-12-27 14:36:05,587][20381] Updated weights for policy 1, policy_version 1291188 (0.0007) [2023-12-27 14:36:05,839][20347] Updated weights for policy 0, policy_version 1292118 (0.0008) [2023-12-27 14:36:05,901][20347] Updated weights for policy 0, policy_version 1292128 (0.0006) [2023-12-27 14:36:05,950][20347] Updated weights for policy 0, policy_version 1292138 (0.0009) [2023-12-27 14:36:06,317][20381] Updated weights for policy 1, policy_version 1291198 (0.0008) [2023-12-27 14:36:06,383][20381] Updated weights for policy 1, policy_version 1291208 (0.0005) [2023-12-27 14:36:06,447][20381] Updated weights for policy 1, policy_version 1291218 (0.0009) [2023-12-27 14:36:06,628][20347] Updated weights for policy 0, policy_version 1292148 (0.0010) [2023-12-27 14:36:06,687][20347] Updated weights for policy 0, policy_version 1292158 (0.0010) [2023-12-27 14:36:06,744][20347] Updated weights for policy 0, policy_version 1292168 (0.0010) [2023-12-27 14:36:07,112][20381] Updated weights for policy 1, policy_version 1291228 (0.0009) [2023-12-27 14:36:07,161][20381] Updated weights for policy 1, policy_version 1291238 (0.0005) [2023-12-27 14:36:07,223][20381] Updated weights for policy 1, policy_version 1291248 (0.0006) [2023-12-27 14:36:07,363][19170] Fps is (10 sec: 20479.1, 60 sec: 20479.8, 300 sec: 20521.6). Total num frames: 661446656. Throughput: 0: 10331.5, 1: 10124.3. Samples: 661415872. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:07,364][19170] Avg episode reward: [(0, '893.500'), (1, '1000.000')] [2023-12-27 14:36:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001291256_330588160.pth... [2023-12-27 14:36:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001292176_330858496.pth... [2023-12-27 14:36:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001290040_330276864.pth [2023-12-27 14:36:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001290992_330555392.pth [2023-12-27 14:36:07,430][20347] Updated weights for policy 0, policy_version 1292178 (0.0010) [2023-12-27 14:36:07,471][20291] KL-divergence is very high: 142.7833 [2023-12-27 14:36:07,487][20347] Updated weights for policy 0, policy_version 1292188 (0.0010) [2023-12-27 14:36:07,538][20347] Updated weights for policy 0, policy_version 1292198 (0.0010) [2023-12-27 14:36:07,587][20347] Updated weights for policy 0, policy_version 1292208 (0.0010) [2023-12-27 14:36:07,803][20381] Updated weights for policy 1, policy_version 1291258 (0.0007) [2023-12-27 14:36:07,864][20381] Updated weights for policy 1, policy_version 1291268 (0.0005) [2023-12-27 14:36:07,919][20381] Updated weights for policy 1, policy_version 1291278 (0.0007) [2023-12-27 14:36:07,971][20381] Updated weights for policy 1, policy_version 1291288 (0.0008) [2023-12-27 14:36:08,277][20347] Updated weights for policy 0, policy_version 1292218 (0.0010) [2023-12-27 14:36:08,335][20347] Updated weights for policy 0, policy_version 1292228 (0.0011) [2023-12-27 14:36:08,392][20347] Updated weights for policy 0, policy_version 1292238 (0.0010) [2023-12-27 14:36:08,648][20381] Updated weights for policy 1, policy_version 1291298 (0.0007) [2023-12-27 14:36:08,706][20381] Updated weights for policy 1, policy_version 1291308 (0.0007) [2023-12-27 14:36:08,760][20381] Updated weights for policy 1, policy_version 1291318 (0.0008) [2023-12-27 14:36:09,097][20347] Updated weights for policy 0, policy_version 1292248 (0.0010) [2023-12-27 14:36:09,154][20347] Updated weights for policy 0, policy_version 1292258 (0.0008) [2023-12-27 14:36:09,209][20347] Updated weights for policy 0, policy_version 1292268 (0.0010) [2023-12-27 14:36:09,447][20381] Updated weights for policy 1, policy_version 1291328 (0.0008) [2023-12-27 14:36:09,502][20381] Updated weights for policy 1, policy_version 1291338 (0.0008) [2023-12-27 14:36:09,562][20381] Updated weights for policy 1, policy_version 1291348 (0.0008) [2023-12-27 14:36:09,942][20347] Updated weights for policy 0, policy_version 1292278 (0.0011) [2023-12-27 14:36:10,002][20347] Updated weights for policy 0, policy_version 1292288 (0.0010) [2023-12-27 14:36:10,066][20347] Updated weights for policy 0, policy_version 1292298 (0.0011) [2023-12-27 14:36:10,309][20381] Updated weights for policy 1, policy_version 1291358 (0.0007) [2023-12-27 14:36:10,369][20381] Updated weights for policy 1, policy_version 1291368 (0.0008) [2023-12-27 14:36:10,431][20381] Updated weights for policy 1, policy_version 1291378 (0.0007) [2023-12-27 14:36:10,741][20347] Updated weights for policy 0, policy_version 1292308 (0.0011) [2023-12-27 14:36:10,793][20347] Updated weights for policy 0, policy_version 1292318 (0.0010) [2023-12-27 14:36:10,854][20347] Updated weights for policy 0, policy_version 1292328 (0.0008) [2023-12-27 14:36:11,128][20381] Updated weights for policy 1, policy_version 1291388 (0.0008) [2023-12-27 14:36:11,201][20381] Updated weights for policy 1, policy_version 1291398 (0.0007) [2023-12-27 14:36:11,266][20381] Updated weights for policy 1, policy_version 1291408 (0.0008) [2023-12-27 14:36:11,546][20347] Updated weights for policy 0, policy_version 1292338 (0.0008) [2023-12-27 14:36:11,609][20347] Updated weights for policy 0, policy_version 1292348 (0.0011) [2023-12-27 14:36:11,679][20347] Updated weights for policy 0, policy_version 1292358 (0.0010) [2023-12-27 14:36:11,744][20347] Updated weights for policy 0, policy_version 1292368 (0.0010) [2023-12-27 14:36:11,982][20381] Updated weights for policy 1, policy_version 1291418 (0.0007) [2023-12-27 14:36:12,043][20381] Updated weights for policy 1, policy_version 1291428 (0.0006) [2023-12-27 14:36:12,098][20381] Updated weights for policy 1, policy_version 1291438 (0.0006) [2023-12-27 14:36:12,153][20381] Updated weights for policy 1, policy_version 1291448 (0.0005) [2023-12-27 14:36:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 661544960. Throughput: 0: 10278.1, 1: 10187.6. Samples: 661536816. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:12,363][19170] Avg episode reward: [(0, '844.160'), (1, '1000.000')] [2023-12-27 14:36:12,455][20347] Updated weights for policy 0, policy_version 1292378 (0.0010) [2023-12-27 14:36:12,517][20347] Updated weights for policy 0, policy_version 1292388 (0.0010) [2023-12-27 14:36:12,579][20347] Updated weights for policy 0, policy_version 1292398 (0.0011) [2023-12-27 14:36:12,797][20381] Updated weights for policy 1, policy_version 1291458 (0.0008) [2023-12-27 14:36:12,845][20381] Updated weights for policy 1, policy_version 1291468 (0.0007) [2023-12-27 14:36:12,890][20381] Updated weights for policy 1, policy_version 1291478 (0.0007) [2023-12-27 14:36:13,242][20347] Updated weights for policy 0, policy_version 1292408 (0.0006) [2023-12-27 14:36:13,288][20347] Updated weights for policy 0, policy_version 1292418 (0.0005) [2023-12-27 14:36:13,335][20347] Updated weights for policy 0, policy_version 1292428 (0.0005) [2023-12-27 14:36:13,536][20381] Updated weights for policy 1, policy_version 1291488 (0.0010) [2023-12-27 14:36:13,596][20381] Updated weights for policy 1, policy_version 1291498 (0.0006) [2023-12-27 14:36:13,660][20381] Updated weights for policy 1, policy_version 1291508 (0.0008) [2023-12-27 14:36:13,924][20347] Updated weights for policy 0, policy_version 1292438 (0.0005) [2023-12-27 14:36:13,980][20347] Updated weights for policy 0, policy_version 1292448 (0.0007) [2023-12-27 14:36:14,039][20347] Updated weights for policy 0, policy_version 1292458 (0.0007) [2023-12-27 14:36:14,313][20381] Updated weights for policy 1, policy_version 1291518 (0.0007) [2023-12-27 14:36:14,372][20381] Updated weights for policy 1, policy_version 1291528 (0.0008) [2023-12-27 14:36:14,430][20381] Updated weights for policy 1, policy_version 1291538 (0.0007) [2023-12-27 14:36:14,707][20347] Updated weights for policy 0, policy_version 1292468 (0.0007) [2023-12-27 14:36:14,778][20347] Updated weights for policy 0, policy_version 1292478 (0.0008) [2023-12-27 14:36:14,849][20347] Updated weights for policy 0, policy_version 1292488 (0.0007) [2023-12-27 14:36:15,148][20381] Updated weights for policy 1, policy_version 1291548 (0.0008) [2023-12-27 14:36:15,210][20381] Updated weights for policy 1, policy_version 1291558 (0.0009) [2023-12-27 14:36:15,259][20381] Updated weights for policy 1, policy_version 1291568 (0.0009) [2023-12-27 14:36:15,492][20347] Updated weights for policy 0, policy_version 1292498 (0.0008) [2023-12-27 14:36:15,547][20347] Updated weights for policy 0, policy_version 1292508 (0.0008) [2023-12-27 14:36:15,612][20347] Updated weights for policy 0, policy_version 1292518 (0.0007) [2023-12-27 14:36:15,670][20347] Updated weights for policy 0, policy_version 1292528 (0.0007) [2023-12-27 14:36:16,003][20381] Updated weights for policy 1, policy_version 1291578 (0.0009) [2023-12-27 14:36:16,062][20381] Updated weights for policy 1, policy_version 1291588 (0.0009) [2023-12-27 14:36:16,125][20381] Updated weights for policy 1, policy_version 1291598 (0.0008) [2023-12-27 14:36:16,186][20381] Updated weights for policy 1, policy_version 1291608 (0.0007) [2023-12-27 14:36:16,293][20347] Updated weights for policy 0, policy_version 1292538 (0.0008) [2023-12-27 14:36:16,345][20347] Updated weights for policy 0, policy_version 1292548 (0.0008) [2023-12-27 14:36:16,390][20347] Updated weights for policy 0, policy_version 1292558 (0.0007) [2023-12-27 14:36:16,866][20381] Updated weights for policy 1, policy_version 1291618 (0.0008) [2023-12-27 14:36:16,922][20381] Updated weights for policy 1, policy_version 1291628 (0.0008) [2023-12-27 14:36:16,971][20381] Updated weights for policy 1, policy_version 1291638 (0.0007) [2023-12-27 14:36:17,049][20347] Updated weights for policy 0, policy_version 1292568 (0.0008) [2023-12-27 14:36:17,103][20347] Updated weights for policy 0, policy_version 1292578 (0.0008) [2023-12-27 14:36:17,160][20347] Updated weights for policy 0, policy_version 1292588 (0.0006) [2023-12-27 14:36:17,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 661651456. Throughput: 0: 10282.6, 1: 10191.6. Samples: 661660320. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:17,363][19170] Avg episode reward: [(0, '824.280'), (1, '1000.000')] [2023-12-27 14:36:17,717][20381] Updated weights for policy 1, policy_version 1291648 (0.0007) [2023-12-27 14:36:17,718][20347] Updated weights for policy 0, policy_version 1292598 (0.0006) [2023-12-27 14:36:17,775][20381] Updated weights for policy 1, policy_version 1291658 (0.0008) [2023-12-27 14:36:17,777][20347] Updated weights for policy 0, policy_version 1292608 (0.0005) [2023-12-27 14:36:17,827][20381] Updated weights for policy 1, policy_version 1291668 (0.0007) [2023-12-27 14:36:17,833][20347] Updated weights for policy 0, policy_version 1292618 (0.0005) [2023-12-27 14:36:18,379][20347] Updated weights for policy 0, policy_version 1292628 (0.0007) [2023-12-27 14:36:18,421][20291] KL-divergence is very high: 155.0771 [2023-12-27 14:36:18,431][20347] Updated weights for policy 0, policy_version 1292638 (0.0006) [2023-12-27 14:36:18,445][20291] KL-divergence is very high: 145.3558 [2023-12-27 14:36:18,460][20291] KL-divergence is very high: 124.2836 [2023-12-27 14:36:18,478][20347] Updated weights for policy 0, policy_version 1292648 (0.0007) [2023-12-27 14:36:18,484][20291] KL-divergence is very high: 188.2676 [2023-12-27 14:36:18,499][20291] KL-divergence is very high: 246.5982 [2023-12-27 14:36:18,510][20291] KL-divergence is very high: 189.1627 [2023-12-27 14:36:18,585][20381] Updated weights for policy 1, policy_version 1291678 (0.0008) [2023-12-27 14:36:18,645][20381] Updated weights for policy 1, policy_version 1291688 (0.0008) [2023-12-27 14:36:18,697][20381] Updated weights for policy 1, policy_version 1291698 (0.0008) [2023-12-27 14:36:19,102][20347] Updated weights for policy 0, policy_version 1292658 (0.0007) [2023-12-27 14:36:19,158][20347] Updated weights for policy 0, policy_version 1292668 (0.0008) [2023-12-27 14:36:19,218][20347] Updated weights for policy 0, policy_version 1292678 (0.0007) [2023-12-27 14:36:19,288][20347] Updated weights for policy 0, policy_version 1292688 (0.0007) [2023-12-27 14:36:19,452][20381] Updated weights for policy 1, policy_version 1291708 (0.0009) [2023-12-27 14:36:19,519][20381] Updated weights for policy 1, policy_version 1291718 (0.0010) [2023-12-27 14:36:19,577][20381] Updated weights for policy 1, policy_version 1291728 (0.0007) [2023-12-27 14:36:19,918][20347] Updated weights for policy 0, policy_version 1292698 (0.0006) [2023-12-27 14:36:19,975][20347] Updated weights for policy 0, policy_version 1292708 (0.0009) [2023-12-27 14:36:20,025][20347] Updated weights for policy 0, policy_version 1292718 (0.0008) [2023-12-27 14:36:20,268][20381] Updated weights for policy 1, policy_version 1291738 (0.0008) [2023-12-27 14:36:20,334][20381] Updated weights for policy 1, policy_version 1291748 (0.0011) [2023-12-27 14:36:20,393][20381] Updated weights for policy 1, policy_version 1291758 (0.0009) [2023-12-27 14:36:20,448][20381] Updated weights for policy 1, policy_version 1291768 (0.0007) [2023-12-27 14:36:20,802][20347] Updated weights for policy 0, policy_version 1292728 (0.0008) [2023-12-27 14:36:20,859][20347] Updated weights for policy 0, policy_version 1292738 (0.0008) [2023-12-27 14:36:20,922][20347] Updated weights for policy 0, policy_version 1292748 (0.0007) [2023-12-27 14:36:21,159][20381] Updated weights for policy 1, policy_version 1291778 (0.0010) [2023-12-27 14:36:21,215][20381] Updated weights for policy 1, policy_version 1291788 (0.0011) [2023-12-27 14:36:21,272][20381] Updated weights for policy 1, policy_version 1291798 (0.0009) [2023-12-27 14:36:21,600][20347] Updated weights for policy 0, policy_version 1292758 (0.0009) [2023-12-27 14:36:21,660][20347] Updated weights for policy 0, policy_version 1292768 (0.0008) [2023-12-27 14:36:21,725][20347] Updated weights for policy 0, policy_version 1292778 (0.0006) [2023-12-27 14:36:22,064][20381] Updated weights for policy 1, policy_version 1291808 (0.0010) [2023-12-27 14:36:22,125][20381] Updated weights for policy 1, policy_version 1291818 (0.0010) [2023-12-27 14:36:22,184][20381] Updated weights for policy 1, policy_version 1291828 (0.0011) [2023-12-27 14:36:22,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 661749760. Throughput: 0: 10289.4, 1: 10164.9. Samples: 661722296. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:22,364][19170] Avg episode reward: [(0, '853.860'), (1, '990.220')] [2023-12-27 14:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001291832_330735616.pth... [2023-12-27 14:36:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001292784_331014144.pth... [2023-12-27 14:36:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001290648_330432512.pth [2023-12-27 14:36:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001291600_330711040.pth [2023-12-27 14:36:22,426][20347] Updated weights for policy 0, policy_version 1292788 (0.0007) [2023-12-27 14:36:22,485][20347] Updated weights for policy 0, policy_version 1292798 (0.0008) [2023-12-27 14:36:22,542][20347] Updated weights for policy 0, policy_version 1292808 (0.0006) [2023-12-27 14:36:22,835][20381] Updated weights for policy 1, policy_version 1291838 (0.0009) [2023-12-27 14:36:22,888][20381] Updated weights for policy 1, policy_version 1291848 (0.0011) [2023-12-27 14:36:22,943][20381] Updated weights for policy 1, policy_version 1291858 (0.0010) [2023-12-27 14:36:23,273][20347] Updated weights for policy 0, policy_version 1292818 (0.0007) [2023-12-27 14:36:23,328][20347] Updated weights for policy 0, policy_version 1292828 (0.0005) [2023-12-27 14:36:23,389][20347] Updated weights for policy 0, policy_version 1292838 (0.0008) [2023-12-27 14:36:23,448][20347] Updated weights for policy 0, policy_version 1292848 (0.0007) [2023-12-27 14:36:23,542][20381] Updated weights for policy 1, policy_version 1291868 (0.0008) [2023-12-27 14:36:23,600][20381] Updated weights for policy 1, policy_version 1291878 (0.0007) [2023-12-27 14:36:23,657][20381] Updated weights for policy 1, policy_version 1291888 (0.0006) [2023-12-27 14:36:24,059][20347] Updated weights for policy 0, policy_version 1292858 (0.0006) [2023-12-27 14:36:24,119][20347] Updated weights for policy 0, policy_version 1292868 (0.0006) [2023-12-27 14:36:24,183][20347] Updated weights for policy 0, policy_version 1292878 (0.0008) [2023-12-27 14:36:24,194][20381] Updated weights for policy 1, policy_version 1291898 (0.0006) [2023-12-27 14:36:24,248][20381] Updated weights for policy 1, policy_version 1291908 (0.0011) [2023-12-27 14:36:24,301][20381] Updated weights for policy 1, policy_version 1291918 (0.0011) [2023-12-27 14:36:24,350][20381] Updated weights for policy 1, policy_version 1291928 (0.0009) [2023-12-27 14:36:24,844][20347] Updated weights for policy 0, policy_version 1292888 (0.0010) [2023-12-27 14:36:24,903][20347] Updated weights for policy 0, policy_version 1292898 (0.0010) [2023-12-27 14:36:24,965][20347] Updated weights for policy 0, policy_version 1292908 (0.0010) [2023-12-27 14:36:25,050][20381] Updated weights for policy 1, policy_version 1291938 (0.0011) [2023-12-27 14:36:25,109][20381] Updated weights for policy 1, policy_version 1291948 (0.0011) [2023-12-27 14:36:25,167][20381] Updated weights for policy 1, policy_version 1291958 (0.0010) [2023-12-27 14:36:25,562][20347] Updated weights for policy 0, policy_version 1292918 (0.0008) [2023-12-27 14:36:25,613][20347] Updated weights for policy 0, policy_version 1292928 (0.0005) [2023-12-27 14:36:25,666][20347] Updated weights for policy 0, policy_version 1292938 (0.0005) [2023-12-27 14:36:25,869][20381] Updated weights for policy 1, policy_version 1291968 (0.0006) [2023-12-27 14:36:25,924][20381] Updated weights for policy 1, policy_version 1291978 (0.0005) [2023-12-27 14:36:25,980][20381] Updated weights for policy 1, policy_version 1291988 (0.0006) [2023-12-27 14:36:26,248][20347] Updated weights for policy 0, policy_version 1292948 (0.0005) [2023-12-27 14:36:26,301][20347] Updated weights for policy 0, policy_version 1292958 (0.0006) [2023-12-27 14:36:26,358][20347] Updated weights for policy 0, policy_version 1292968 (0.0007) [2023-12-27 14:36:26,607][20381] Updated weights for policy 1, policy_version 1291998 (0.0007) [2023-12-27 14:36:26,663][20381] Updated weights for policy 1, policy_version 1292008 (0.0007) [2023-12-27 14:36:26,715][20381] Updated weights for policy 1, policy_version 1292018 (0.0008) [2023-12-27 14:36:26,982][20347] Updated weights for policy 0, policy_version 1292978 (0.0007) [2023-12-27 14:36:27,035][20347] Updated weights for policy 0, policy_version 1292988 (0.0007) [2023-12-27 14:36:27,036][20291] KL-divergence is very high: 105.3618 [2023-12-27 14:36:27,079][20291] KL-divergence is very high: 106.8425 [2023-12-27 14:36:27,088][20347] Updated weights for policy 0, policy_version 1292998 (0.0006) [2023-12-27 14:36:27,143][20347] Updated weights for policy 0, policy_version 1293008 (0.0008) [2023-12-27 14:36:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 661856256. Throughput: 0: 10307.4, 1: 10156.9. Samples: 661846396. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:27,363][19170] Avg episode reward: [(0, '902.630'), (1, '980.440')] [2023-12-27 14:36:27,424][20381] Updated weights for policy 1, policy_version 1292028 (0.0009) [2023-12-27 14:36:27,470][20381] Updated weights for policy 1, policy_version 1292038 (0.0010) [2023-12-27 14:36:27,516][20381] Updated weights for policy 1, policy_version 1292048 (0.0010) [2023-12-27 14:36:27,743][20347] Updated weights for policy 0, policy_version 1293018 (0.0007) [2023-12-27 14:36:27,802][20347] Updated weights for policy 0, policy_version 1293028 (0.0007) [2023-12-27 14:36:27,858][20347] Updated weights for policy 0, policy_version 1293038 (0.0007) [2023-12-27 14:36:28,202][20381] Updated weights for policy 1, policy_version 1292058 (0.0009) [2023-12-27 14:36:28,258][20381] Updated weights for policy 1, policy_version 1292068 (0.0010) [2023-12-27 14:36:28,315][20381] Updated weights for policy 1, policy_version 1292078 (0.0008) [2023-12-27 14:36:28,375][20381] Updated weights for policy 1, policy_version 1292088 (0.0009) [2023-12-27 14:36:28,522][20347] Updated weights for policy 0, policy_version 1293048 (0.0008) [2023-12-27 14:36:28,581][20347] Updated weights for policy 0, policy_version 1293058 (0.0007) [2023-12-27 14:36:28,640][20347] Updated weights for policy 0, policy_version 1293068 (0.0005) [2023-12-27 14:36:29,065][20381] Updated weights for policy 1, policy_version 1292098 (0.0010) [2023-12-27 14:36:29,120][20381] Updated weights for policy 1, policy_version 1292108 (0.0010) [2023-12-27 14:36:29,172][20381] Updated weights for policy 1, policy_version 1292118 (0.0008) [2023-12-27 14:36:29,263][20347] Updated weights for policy 0, policy_version 1293078 (0.0007) [2023-12-27 14:36:29,331][20347] Updated weights for policy 0, policy_version 1293088 (0.0007) [2023-12-27 14:36:29,401][20347] Updated weights for policy 0, policy_version 1293098 (0.0009) [2023-12-27 14:36:29,815][20381] Updated weights for policy 1, policy_version 1292128 (0.0006) [2023-12-27 14:36:29,877][20381] Updated weights for policy 1, policy_version 1292138 (0.0007) [2023-12-27 14:36:29,943][20381] Updated weights for policy 1, policy_version 1292148 (0.0008) [2023-12-27 14:36:30,068][20347] Updated weights for policy 0, policy_version 1293108 (0.0011) [2023-12-27 14:36:30,120][20347] Updated weights for policy 0, policy_version 1293118 (0.0010) [2023-12-27 14:36:30,178][20347] Updated weights for policy 0, policy_version 1293128 (0.0010) [2023-12-27 14:36:30,612][20381] Updated weights for policy 1, policy_version 1292158 (0.0009) [2023-12-27 14:36:30,665][20381] Updated weights for policy 1, policy_version 1292168 (0.0010) [2023-12-27 14:36:30,724][20381] Updated weights for policy 1, policy_version 1292178 (0.0006) [2023-12-27 14:36:30,795][20347] Updated weights for policy 0, policy_version 1293138 (0.0009) [2023-12-27 14:36:30,840][20347] Updated weights for policy 0, policy_version 1293148 (0.0005) [2023-12-27 14:36:30,893][20347] Updated weights for policy 0, policy_version 1293158 (0.0007) [2023-12-27 14:36:30,955][20347] Updated weights for policy 0, policy_version 1293168 (0.0010) [2023-12-27 14:36:31,307][20381] Updated weights for policy 1, policy_version 1292188 (0.0007) [2023-12-27 14:36:31,372][20381] Updated weights for policy 1, policy_version 1292198 (0.0010) [2023-12-27 14:36:31,436][20381] Updated weights for policy 1, policy_version 1292208 (0.0009) [2023-12-27 14:36:31,658][20347] Updated weights for policy 0, policy_version 1293178 (0.0008) [2023-12-27 14:36:31,716][20347] Updated weights for policy 0, policy_version 1293188 (0.0006) [2023-12-27 14:36:31,776][20347] Updated weights for policy 0, policy_version 1293198 (0.0008) [2023-12-27 14:36:32,099][20381] Updated weights for policy 1, policy_version 1292218 (0.0011) [2023-12-27 14:36:32,157][20381] Updated weights for policy 1, policy_version 1292228 (0.0010) [2023-12-27 14:36:32,215][20381] Updated weights for policy 1, policy_version 1292238 (0.0010) [2023-12-27 14:36:32,279][20381] Updated weights for policy 1, policy_version 1292248 (0.0011) [2023-12-27 14:36:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 661962752. Throughput: 0: 10346.2, 1: 10189.3. Samples: 661972640. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:32,363][19170] Avg episode reward: [(0, '902.750'), (1, '990.220')] [2023-12-27 14:36:32,457][20347] Updated weights for policy 0, policy_version 1293208 (0.0007) [2023-12-27 14:36:32,518][20347] Updated weights for policy 0, policy_version 1293218 (0.0008) [2023-12-27 14:36:32,580][20347] Updated weights for policy 0, policy_version 1293228 (0.0007) [2023-12-27 14:36:32,988][20381] Updated weights for policy 1, policy_version 1292258 (0.0010) [2023-12-27 14:36:33,039][20381] Updated weights for policy 1, policy_version 1292268 (0.0010) [2023-12-27 14:36:33,097][20381] Updated weights for policy 1, policy_version 1292278 (0.0011) [2023-12-27 14:36:33,240][20347] Updated weights for policy 0, policy_version 1293238 (0.0006) [2023-12-27 14:36:33,302][20347] Updated weights for policy 0, policy_version 1293248 (0.0009) [2023-12-27 14:36:33,362][20347] Updated weights for policy 0, policy_version 1293258 (0.0011) [2023-12-27 14:36:33,776][20381] Updated weights for policy 1, policy_version 1292288 (0.0011) [2023-12-27 14:36:33,834][20381] Updated weights for policy 1, policy_version 1292298 (0.0010) [2023-12-27 14:36:33,885][20381] Updated weights for policy 1, policy_version 1292308 (0.0010) [2023-12-27 14:36:33,964][20347] Updated weights for policy 0, policy_version 1293268 (0.0008) [2023-12-27 14:36:34,022][20347] Updated weights for policy 0, policy_version 1293278 (0.0005) [2023-12-27 14:36:34,078][20347] Updated weights for policy 0, policy_version 1293288 (0.0006) [2023-12-27 14:36:34,597][20381] Updated weights for policy 1, policy_version 1292318 (0.0011) [2023-12-27 14:36:34,656][20381] Updated weights for policy 1, policy_version 1292328 (0.0011) [2023-12-27 14:36:34,711][20381] Updated weights for policy 1, policy_version 1292338 (0.0006) [2023-12-27 14:36:34,795][20347] Updated weights for policy 0, policy_version 1293298 (0.0007) [2023-12-27 14:36:34,850][20347] Updated weights for policy 0, policy_version 1293308 (0.0007) [2023-12-27 14:36:34,905][20347] Updated weights for policy 0, policy_version 1293318 (0.0007) [2023-12-27 14:36:34,963][20347] Updated weights for policy 0, policy_version 1293328 (0.0008) [2023-12-27 14:36:35,382][20381] Updated weights for policy 1, policy_version 1292348 (0.0010) [2023-12-27 14:36:35,442][20381] Updated weights for policy 1, policy_version 1292358 (0.0010) [2023-12-27 14:36:35,493][20381] Updated weights for policy 1, policy_version 1292368 (0.0010) [2023-12-27 14:36:35,664][20347] Updated weights for policy 0, policy_version 1293338 (0.0007) [2023-12-27 14:36:35,718][20347] Updated weights for policy 0, policy_version 1293348 (0.0008) [2023-12-27 14:36:35,779][20347] Updated weights for policy 0, policy_version 1293358 (0.0008) [2023-12-27 14:36:36,140][20381] Updated weights for policy 1, policy_version 1292378 (0.0010) [2023-12-27 14:36:36,197][20381] Updated weights for policy 1, policy_version 1292388 (0.0011) [2023-12-27 14:36:36,263][20381] Updated weights for policy 1, policy_version 1292398 (0.0011) [2023-12-27 14:36:36,321][20381] Updated weights for policy 1, policy_version 1292408 (0.0011) [2023-12-27 14:36:36,456][20347] Updated weights for policy 0, policy_version 1293368 (0.0008) [2023-12-27 14:36:36,519][20347] Updated weights for policy 0, policy_version 1293378 (0.0007) [2023-12-27 14:36:36,578][20347] Updated weights for policy 0, policy_version 1293388 (0.0007) [2023-12-27 14:36:37,002][20381] Updated weights for policy 1, policy_version 1292418 (0.0011) [2023-12-27 14:36:37,069][20381] Updated weights for policy 1, policy_version 1292428 (0.0011) [2023-12-27 14:36:37,115][20381] Updated weights for policy 1, policy_version 1292438 (0.0010) [2023-12-27 14:36:37,208][20347] Updated weights for policy 0, policy_version 1293398 (0.0006) [2023-12-27 14:36:37,262][20347] Updated weights for policy 0, policy_version 1293408 (0.0006) [2023-12-27 14:36:37,310][20347] Updated weights for policy 0, policy_version 1293418 (0.0007) [2023-12-27 14:36:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 662069248. Throughput: 0: 10323.6, 1: 10210.1. Samples: 662034024. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:37,363][19170] Avg episode reward: [(0, '863.840'), (1, '990.230')] [2023-12-27 14:36:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001293424_331177984.pth... [2023-12-27 14:36:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001292440_330891264.pth... [2023-12-27 14:36:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001291256_330588160.pth [2023-12-27 14:36:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001292176_330858496.pth [2023-12-27 14:36:37,737][20381] Updated weights for policy 1, policy_version 1292448 (0.0006) [2023-12-27 14:36:37,788][20381] Updated weights for policy 1, policy_version 1292458 (0.0006) [2023-12-27 14:36:37,847][20381] Updated weights for policy 1, policy_version 1292468 (0.0007) [2023-12-27 14:36:38,013][20347] Updated weights for policy 0, policy_version 1293428 (0.0007) [2023-12-27 14:36:38,064][20347] Updated weights for policy 0, policy_version 1293438 (0.0005) [2023-12-27 14:36:38,128][20347] Updated weights for policy 0, policy_version 1293448 (0.0005) [2023-12-27 14:36:38,437][20381] Updated weights for policy 1, policy_version 1292478 (0.0008) [2023-12-27 14:36:38,490][20381] Updated weights for policy 1, policy_version 1292488 (0.0008) [2023-12-27 14:36:38,548][20381] Updated weights for policy 1, policy_version 1292498 (0.0008) [2023-12-27 14:36:38,789][20347] Updated weights for policy 0, policy_version 1293458 (0.0006) [2023-12-27 14:36:38,849][20347] Updated weights for policy 0, policy_version 1293468 (0.0007) [2023-12-27 14:36:38,911][20347] Updated weights for policy 0, policy_version 1293478 (0.0006) [2023-12-27 14:36:38,965][20347] Updated weights for policy 0, policy_version 1293488 (0.0005) [2023-12-27 14:36:39,272][20381] Updated weights for policy 1, policy_version 1292508 (0.0008) [2023-12-27 14:36:39,340][20381] Updated weights for policy 1, policy_version 1292518 (0.0008) [2023-12-27 14:36:39,409][20381] Updated weights for policy 1, policy_version 1292528 (0.0007) [2023-12-27 14:36:39,623][20347] Updated weights for policy 0, policy_version 1293498 (0.0007) [2023-12-27 14:36:39,689][20347] Updated weights for policy 0, policy_version 1293508 (0.0007) [2023-12-27 14:36:39,751][20347] Updated weights for policy 0, policy_version 1293518 (0.0007) [2023-12-27 14:36:40,195][20381] Updated weights for policy 1, policy_version 1292538 (0.0008) [2023-12-27 14:36:40,256][20381] Updated weights for policy 1, policy_version 1292548 (0.0008) [2023-12-27 14:36:40,314][20381] Updated weights for policy 1, policy_version 1292558 (0.0007) [2023-12-27 14:36:40,376][20381] Updated weights for policy 1, policy_version 1292568 (0.0008) [2023-12-27 14:36:40,491][20347] Updated weights for policy 0, policy_version 1293528 (0.0010) [2023-12-27 14:36:40,544][20347] Updated weights for policy 0, policy_version 1293538 (0.0010) [2023-12-27 14:36:40,595][20347] Updated weights for policy 0, policy_version 1293548 (0.0010) [2023-12-27 14:36:41,031][20381] Updated weights for policy 1, policy_version 1292578 (0.0006) [2023-12-27 14:36:41,097][20381] Updated weights for policy 1, policy_version 1292588 (0.0009) [2023-12-27 14:36:41,163][20381] Updated weights for policy 1, policy_version 1292598 (0.0007) [2023-12-27 14:36:41,362][20347] Updated weights for policy 0, policy_version 1293558 (0.0009) [2023-12-27 14:36:41,419][20347] Updated weights for policy 0, policy_version 1293568 (0.0008) [2023-12-27 14:36:41,472][20347] Updated weights for policy 0, policy_version 1293578 (0.0008) [2023-12-27 14:36:41,939][20381] Updated weights for policy 1, policy_version 1292608 (0.0010) [2023-12-27 14:36:41,997][20381] Updated weights for policy 1, policy_version 1292618 (0.0011) [2023-12-27 14:36:42,059][20381] Updated weights for policy 1, policy_version 1292628 (0.0010) [2023-12-27 14:36:42,256][20347] Updated weights for policy 0, policy_version 1293588 (0.0008) [2023-12-27 14:36:42,319][20347] Updated weights for policy 0, policy_version 1293598 (0.0007) [2023-12-27 14:36:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 662159360. Throughput: 0: 10324.2, 1: 10171.3. Samples: 662156032. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:42,363][19170] Avg episode reward: [(0, '863.410'), (1, '990.230')] [2023-12-27 14:36:42,387][20347] Updated weights for policy 0, policy_version 1293608 (0.0009) [2023-12-27 14:36:42,716][20381] Updated weights for policy 1, policy_version 1292638 (0.0007) [2023-12-27 14:36:42,765][20381] Updated weights for policy 1, policy_version 1292648 (0.0005) [2023-12-27 14:36:42,820][20381] Updated weights for policy 1, policy_version 1292658 (0.0011) [2023-12-27 14:36:43,073][20347] Updated weights for policy 0, policy_version 1293618 (0.0007) [2023-12-27 14:36:43,127][20347] Updated weights for policy 0, policy_version 1293628 (0.0007) [2023-12-27 14:36:43,178][20347] Updated weights for policy 0, policy_version 1293638 (0.0007) [2023-12-27 14:36:43,222][20347] Updated weights for policy 0, policy_version 1293648 (0.0007) [2023-12-27 14:36:43,486][20381] Updated weights for policy 1, policy_version 1292668 (0.0011) [2023-12-27 14:36:43,537][20381] Updated weights for policy 1, policy_version 1292678 (0.0010) [2023-12-27 14:36:43,586][20381] Updated weights for policy 1, policy_version 1292688 (0.0010) [2023-12-27 14:36:43,929][20347] Updated weights for policy 0, policy_version 1293658 (0.0006) [2023-12-27 14:36:43,979][20347] Updated weights for policy 0, policy_version 1293668 (0.0006) [2023-12-27 14:36:44,034][20347] Updated weights for policy 0, policy_version 1293678 (0.0007) [2023-12-27 14:36:44,291][20381] Updated weights for policy 1, policy_version 1292698 (0.0011) [2023-12-27 14:36:44,349][20381] Updated weights for policy 1, policy_version 1292708 (0.0010) [2023-12-27 14:36:44,394][20381] Updated weights for policy 1, policy_version 1292718 (0.0010) [2023-12-27 14:36:44,439][20381] Updated weights for policy 1, policy_version 1292728 (0.0010) [2023-12-27 14:36:44,673][20347] Updated weights for policy 0, policy_version 1293688 (0.0006) [2023-12-27 14:36:44,730][20347] Updated weights for policy 0, policy_version 1293698 (0.0006) [2023-12-27 14:36:44,799][20347] Updated weights for policy 0, policy_version 1293708 (0.0008) [2023-12-27 14:36:45,110][20381] Updated weights for policy 1, policy_version 1292738 (0.0006) [2023-12-27 14:36:45,169][20381] Updated weights for policy 1, policy_version 1292748 (0.0006) [2023-12-27 14:36:45,227][20381] Updated weights for policy 1, policy_version 1292758 (0.0005) [2023-12-27 14:36:45,514][20347] Updated weights for policy 0, policy_version 1293718 (0.0009) [2023-12-27 14:36:45,574][20347] Updated weights for policy 0, policy_version 1293728 (0.0008) [2023-12-27 14:36:45,625][20347] Updated weights for policy 0, policy_version 1293738 (0.0008) [2023-12-27 14:36:45,825][20381] Updated weights for policy 1, policy_version 1292768 (0.0008) [2023-12-27 14:36:45,876][20381] Updated weights for policy 1, policy_version 1292778 (0.0006) [2023-12-27 14:36:45,932][20381] Updated weights for policy 1, policy_version 1292788 (0.0006) [2023-12-27 14:36:46,383][20347] Updated weights for policy 0, policy_version 1293748 (0.0008) [2023-12-27 14:36:46,439][20347] Updated weights for policy 0, policy_version 1293758 (0.0009) [2023-12-27 14:36:46,479][20381] Updated weights for policy 1, policy_version 1292798 (0.0006) [2023-12-27 14:36:46,495][20347] Updated weights for policy 0, policy_version 1293768 (0.0007) [2023-12-27 14:36:46,527][20381] Updated weights for policy 1, policy_version 1292808 (0.0005) [2023-12-27 14:36:46,574][20381] Updated weights for policy 1, policy_version 1292818 (0.0007) [2023-12-27 14:36:47,202][20347] Updated weights for policy 0, policy_version 1293778 (0.0007) [2023-12-27 14:36:47,262][20347] Updated weights for policy 0, policy_version 1293788 (0.0006) [2023-12-27 14:36:47,263][20381] Updated weights for policy 1, policy_version 1292828 (0.0008) [2023-12-27 14:36:47,313][20347] Updated weights for policy 0, policy_version 1293798 (0.0006) [2023-12-27 14:36:47,318][20381] Updated weights for policy 1, policy_version 1292838 (0.0007) [2023-12-27 14:36:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 662257664. Throughput: 0: 10292.6, 1: 10202.0. Samples: 662277564. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:47,363][19170] Avg episode reward: [(0, '844.270'), (1, '1000.000')] [2023-12-27 14:36:47,368][20381] Updated weights for policy 1, policy_version 1292848 (0.0006) [2023-12-27 14:36:47,369][20347] Updated weights for policy 0, policy_version 1293808 (0.0006) [2023-12-27 14:36:48,025][20381] Updated weights for policy 1, policy_version 1292858 (0.0007) [2023-12-27 14:36:48,079][20381] Updated weights for policy 1, policy_version 1292868 (0.0007) [2023-12-27 14:36:48,084][20347] Updated weights for policy 0, policy_version 1293818 (0.0008) [2023-12-27 14:36:48,138][20381] Updated weights for policy 1, policy_version 1292878 (0.0006) [2023-12-27 14:36:48,145][20347] Updated weights for policy 0, policy_version 1293828 (0.0006) [2023-12-27 14:36:48,193][20381] Updated weights for policy 1, policy_version 1292888 (0.0006) [2023-12-27 14:36:48,204][20347] Updated weights for policy 0, policy_version 1293838 (0.0006) [2023-12-27 14:36:48,788][20381] Updated weights for policy 1, policy_version 1292898 (0.0006) [2023-12-27 14:36:48,813][20347] Updated weights for policy 0, policy_version 1293848 (0.0007) [2023-12-27 14:36:48,843][20381] Updated weights for policy 1, policy_version 1292908 (0.0007) [2023-12-27 14:36:48,871][20347] Updated weights for policy 0, policy_version 1293858 (0.0005) [2023-12-27 14:36:48,902][20381] Updated weights for policy 1, policy_version 1292918 (0.0006) [2023-12-27 14:36:48,934][20347] Updated weights for policy 0, policy_version 1293868 (0.0006) [2023-12-27 14:36:49,510][20381] Updated weights for policy 1, policy_version 1292928 (0.0007) [2023-12-27 14:36:49,552][20347] Updated weights for policy 0, policy_version 1293878 (0.0007) [2023-12-27 14:36:49,568][20381] Updated weights for policy 1, policy_version 1292938 (0.0008) [2023-12-27 14:36:49,606][20347] Updated weights for policy 0, policy_version 1293888 (0.0006) [2023-12-27 14:36:49,624][20381] Updated weights for policy 1, policy_version 1292948 (0.0009) [2023-12-27 14:36:49,664][20347] Updated weights for policy 0, policy_version 1293898 (0.0005) [2023-12-27 14:36:50,316][20347] Updated weights for policy 0, policy_version 1293908 (0.0006) [2023-12-27 14:36:50,349][20381] Updated weights for policy 1, policy_version 1292958 (0.0009) [2023-12-27 14:36:50,370][20347] Updated weights for policy 0, policy_version 1293918 (0.0007) [2023-12-27 14:36:50,405][20381] Updated weights for policy 1, policy_version 1292968 (0.0008) [2023-12-27 14:36:50,423][20347] Updated weights for policy 0, policy_version 1293928 (0.0006) [2023-12-27 14:36:50,469][20381] Updated weights for policy 1, policy_version 1292978 (0.0006) [2023-12-27 14:36:51,124][20381] Updated weights for policy 1, policy_version 1292988 (0.0008) [2023-12-27 14:36:51,183][20381] Updated weights for policy 1, policy_version 1292998 (0.0005) [2023-12-27 14:36:51,203][20347] Updated weights for policy 0, policy_version 1293938 (0.0006) [2023-12-27 14:36:51,243][20381] Updated weights for policy 1, policy_version 1293008 (0.0006) [2023-12-27 14:36:51,259][20347] Updated weights for policy 0, policy_version 1293948 (0.0007) [2023-12-27 14:36:51,321][20347] Updated weights for policy 0, policy_version 1293958 (0.0008) [2023-12-27 14:36:51,388][20347] Updated weights for policy 0, policy_version 1293968 (0.0008) [2023-12-27 14:36:51,934][20381] Updated weights for policy 1, policy_version 1293018 (0.0006) [2023-12-27 14:36:52,001][20381] Updated weights for policy 1, policy_version 1293028 (0.0008) [2023-12-27 14:36:52,063][20347] Updated weights for policy 0, policy_version 1293978 (0.0006) [2023-12-27 14:36:52,069][20381] Updated weights for policy 1, policy_version 1293038 (0.0008) [2023-12-27 14:36:52,120][20347] Updated weights for policy 0, policy_version 1293988 (0.0005) [2023-12-27 14:36:52,136][20381] Updated weights for policy 1, policy_version 1293048 (0.0008) [2023-12-27 14:36:52,178][20347] Updated weights for policy 0, policy_version 1293998 (0.0006) [2023-12-27 14:36:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 662372352. Throughput: 0: 10317.7, 1: 10233.3. Samples: 662340656. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:52,363][19170] Avg episode reward: [(0, '767.020'), (1, '1000.000')] [2023-12-27 14:36:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001293048_331046912.pth... [2023-12-27 14:36:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001294000_331325440.pth... [2023-12-27 14:36:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001291832_330735616.pth [2023-12-27 14:36:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001292784_331014144.pth [2023-12-27 14:36:52,838][20381] Updated weights for policy 1, policy_version 1293058 (0.0007) [2023-12-27 14:36:52,878][20347] Updated weights for policy 0, policy_version 1294008 (0.0007) [2023-12-27 14:36:52,894][20381] Updated weights for policy 1, policy_version 1293068 (0.0007) [2023-12-27 14:36:52,929][20347] Updated weights for policy 0, policy_version 1294018 (0.0007) [2023-12-27 14:36:52,953][20381] Updated weights for policy 1, policy_version 1293078 (0.0006) [2023-12-27 14:36:52,985][20347] Updated weights for policy 0, policy_version 1294028 (0.0006) [2023-12-27 14:36:53,588][20381] Updated weights for policy 1, policy_version 1293088 (0.0007) [2023-12-27 14:36:53,640][20347] Updated weights for policy 0, policy_version 1294038 (0.0006) [2023-12-27 14:36:53,642][20381] Updated weights for policy 1, policy_version 1293098 (0.0008) [2023-12-27 14:36:53,693][20347] Updated weights for policy 0, policy_version 1294048 (0.0005) [2023-12-27 14:36:53,693][20381] Updated weights for policy 1, policy_version 1293108 (0.0008) [2023-12-27 14:36:53,744][20347] Updated weights for policy 0, policy_version 1294058 (0.0005) [2023-12-27 14:36:54,374][20347] Updated weights for policy 0, policy_version 1294068 (0.0006) [2023-12-27 14:36:54,411][20381] Updated weights for policy 1, policy_version 1293118 (0.0008) [2023-12-27 14:36:54,422][20347] Updated weights for policy 0, policy_version 1294078 (0.0006) [2023-12-27 14:36:54,462][20381] Updated weights for policy 1, policy_version 1293128 (0.0008) [2023-12-27 14:36:54,471][20347] Updated weights for policy 0, policy_version 1294088 (0.0007) [2023-12-27 14:36:54,512][20381] Updated weights for policy 1, policy_version 1293138 (0.0007) [2023-12-27 14:36:55,149][20347] Updated weights for policy 0, policy_version 1294098 (0.0007) [2023-12-27 14:36:55,206][20347] Updated weights for policy 0, policy_version 1294108 (0.0008) [2023-12-27 14:36:55,246][20381] Updated weights for policy 1, policy_version 1293148 (0.0008) [2023-12-27 14:36:55,259][20347] Updated weights for policy 0, policy_version 1294118 (0.0007) [2023-12-27 14:36:55,303][20381] Updated weights for policy 1, policy_version 1293158 (0.0006) [2023-12-27 14:36:55,315][20347] Updated weights for policy 0, policy_version 1294128 (0.0007) [2023-12-27 14:36:55,365][20381] Updated weights for policy 1, policy_version 1293168 (0.0005) [2023-12-27 14:36:55,920][20347] Updated weights for policy 0, policy_version 1294138 (0.0008) [2023-12-27 14:36:55,979][20347] Updated weights for policy 0, policy_version 1294148 (0.0006) [2023-12-27 14:36:55,999][20381] Updated weights for policy 1, policy_version 1293178 (0.0006) [2023-12-27 14:36:56,036][20347] Updated weights for policy 0, policy_version 1294158 (0.0006) [2023-12-27 14:36:56,056][20381] Updated weights for policy 1, policy_version 1293188 (0.0007) [2023-12-27 14:36:56,113][20381] Updated weights for policy 1, policy_version 1293198 (0.0005) [2023-12-27 14:36:56,166][20381] Updated weights for policy 1, policy_version 1293208 (0.0005) [2023-12-27 14:36:56,646][20347] Updated weights for policy 0, policy_version 1294168 (0.0005) [2023-12-27 14:36:56,690][20381] Updated weights for policy 1, policy_version 1293218 (0.0007) [2023-12-27 14:36:56,699][20347] Updated weights for policy 0, policy_version 1294178 (0.0006) [2023-12-27 14:36:56,743][20381] Updated weights for policy 1, policy_version 1293228 (0.0005) [2023-12-27 14:36:56,753][20347] Updated weights for policy 0, policy_version 1294188 (0.0007) [2023-12-27 14:36:56,797][20381] Updated weights for policy 1, policy_version 1293238 (0.0005) [2023-12-27 14:36:57,360][20381] Updated weights for policy 1, policy_version 1293248 (0.0006) [2023-12-27 14:36:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 662470656. Throughput: 0: 10342.7, 1: 10238.2. Samples: 662462952. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:36:57,363][19170] Avg episode reward: [(0, '737.340'), (1, '990.310')] [2023-12-27 14:36:57,405][20381] Updated weights for policy 1, policy_version 1293258 (0.0008) [2023-12-27 14:36:57,444][20347] Updated weights for policy 0, policy_version 1294198 (0.0007) [2023-12-27 14:36:57,454][20381] Updated weights for policy 1, policy_version 1293268 (0.0006) [2023-12-27 14:36:57,508][20347] Updated weights for policy 0, policy_version 1294208 (0.0008) [2023-12-27 14:36:57,559][20347] Updated weights for policy 0, policy_version 1294218 (0.0010) [2023-12-27 14:36:58,026][20381] Updated weights for policy 1, policy_version 1293278 (0.0005) [2023-12-27 14:36:58,077][20381] Updated weights for policy 1, policy_version 1293288 (0.0005) [2023-12-27 14:36:58,131][20381] Updated weights for policy 1, policy_version 1293298 (0.0005) [2023-12-27 14:36:58,222][20347] Updated weights for policy 0, policy_version 1294228 (0.0009) [2023-12-27 14:36:58,281][20347] Updated weights for policy 0, policy_version 1294238 (0.0008) [2023-12-27 14:36:58,339][20347] Updated weights for policy 0, policy_version 1294248 (0.0008) [2023-12-27 14:36:58,886][20381] Updated weights for policy 1, policy_version 1293308 (0.0007) [2023-12-27 14:36:58,954][20381] Updated weights for policy 1, policy_version 1293318 (0.0006) [2023-12-27 14:36:59,011][20381] Updated weights for policy 1, policy_version 1293328 (0.0008) [2023-12-27 14:36:59,137][20347] Updated weights for policy 0, policy_version 1294258 (0.0007) [2023-12-27 14:36:59,202][20347] Updated weights for policy 0, policy_version 1294268 (0.0006) [2023-12-27 14:36:59,272][20347] Updated weights for policy 0, policy_version 1294278 (0.0008) [2023-12-27 14:36:59,336][20347] Updated weights for policy 0, policy_version 1294288 (0.0008) [2023-12-27 14:36:59,716][20381] Updated weights for policy 1, policy_version 1293338 (0.0008) [2023-12-27 14:36:59,768][20381] Updated weights for policy 1, policy_version 1293348 (0.0007) [2023-12-27 14:36:59,822][20381] Updated weights for policy 1, policy_version 1293358 (0.0006) [2023-12-27 14:36:59,882][20381] Updated weights for policy 1, policy_version 1293368 (0.0007) [2023-12-27 14:37:00,021][20347] Updated weights for policy 0, policy_version 1294298 (0.0006) [2023-12-27 14:37:00,084][20347] Updated weights for policy 0, policy_version 1294308 (0.0008) [2023-12-27 14:37:00,144][20347] Updated weights for policy 0, policy_version 1294318 (0.0009) [2023-12-27 14:37:00,524][20381] Updated weights for policy 1, policy_version 1293378 (0.0008) [2023-12-27 14:37:00,580][20381] Updated weights for policy 1, policy_version 1293388 (0.0007) [2023-12-27 14:37:00,637][20381] Updated weights for policy 1, policy_version 1293398 (0.0008) [2023-12-27 14:37:00,811][20347] Updated weights for policy 0, policy_version 1294328 (0.0008) [2023-12-27 14:37:00,858][20347] Updated weights for policy 0, policy_version 1294338 (0.0007) [2023-12-27 14:37:00,901][20347] Updated weights for policy 0, policy_version 1294348 (0.0007) [2023-12-27 14:37:01,357][20381] Updated weights for policy 1, policy_version 1293408 (0.0009) [2023-12-27 14:37:01,416][20381] Updated weights for policy 1, policy_version 1293418 (0.0008) [2023-12-27 14:37:01,467][20381] Updated weights for policy 1, policy_version 1293428 (0.0008) [2023-12-27 14:37:01,651][20347] Updated weights for policy 0, policy_version 1294358 (0.0007) [2023-12-27 14:37:01,709][20347] Updated weights for policy 0, policy_version 1294368 (0.0006) [2023-12-27 14:37:01,786][20347] Updated weights for policy 0, policy_version 1294378 (0.0008) [2023-12-27 14:37:02,232][20381] Updated weights for policy 1, policy_version 1293438 (0.0006) [2023-12-27 14:37:02,297][20381] Updated weights for policy 1, policy_version 1293448 (0.0007) [2023-12-27 14:37:02,353][20381] Updated weights for policy 1, policy_version 1293458 (0.0008) [2023-12-27 14:37:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 662568960. Throughput: 0: 10288.0, 1: 10296.0. Samples: 662586600. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:02,363][19170] Avg episode reward: [(0, '746.600'), (1, '980.610')] [2023-12-27 14:37:02,370][20347] Updated weights for policy 0, policy_version 1294388 (0.0008) [2023-12-27 14:37:02,431][20347] Updated weights for policy 0, policy_version 1294398 (0.0006) [2023-12-27 14:37:02,438][20291] KL-divergence is very high: 103.4393 [2023-12-27 14:37:02,483][20347] Updated weights for policy 0, policy_version 1294408 (0.0007) [2023-12-27 14:37:03,015][20381] Updated weights for policy 1, policy_version 1293468 (0.0007) [2023-12-27 14:37:03,066][20381] Updated weights for policy 1, policy_version 1293478 (0.0008) [2023-12-27 14:37:03,118][20381] Updated weights for policy 1, policy_version 1293488 (0.0006) [2023-12-27 14:37:03,149][20347] Updated weights for policy 0, policy_version 1294418 (0.0007) [2023-12-27 14:37:03,200][20347] Updated weights for policy 0, policy_version 1294428 (0.0005) [2023-12-27 14:37:03,253][20347] Updated weights for policy 0, policy_version 1294438 (0.0005) [2023-12-27 14:37:03,301][20347] Updated weights for policy 0, policy_version 1294448 (0.0005) [2023-12-27 14:37:03,758][20381] Updated weights for policy 1, policy_version 1293498 (0.0007) [2023-12-27 14:37:03,808][20381] Updated weights for policy 1, policy_version 1293508 (0.0009) [2023-12-27 14:37:03,866][20381] Updated weights for policy 1, policy_version 1293518 (0.0007) [2023-12-27 14:37:03,928][20381] Updated weights for policy 1, policy_version 1293528 (0.0006) [2023-12-27 14:37:03,958][20347] Updated weights for policy 0, policy_version 1294458 (0.0007) [2023-12-27 14:37:04,019][20347] Updated weights for policy 0, policy_version 1294468 (0.0008) [2023-12-27 14:37:04,079][20347] Updated weights for policy 0, policy_version 1294478 (0.0008) [2023-12-27 14:37:04,595][20381] Updated weights for policy 1, policy_version 1293538 (0.0009) [2023-12-27 14:37:04,658][20381] Updated weights for policy 1, policy_version 1293548 (0.0008) [2023-12-27 14:37:04,721][20381] Updated weights for policy 1, policy_version 1293558 (0.0008) [2023-12-27 14:37:04,773][20347] Updated weights for policy 0, policy_version 1294488 (0.0008) [2023-12-27 14:37:04,833][20347] Updated weights for policy 0, policy_version 1294498 (0.0007) [2023-12-27 14:37:04,892][20347] Updated weights for policy 0, policy_version 1294508 (0.0007) [2023-12-27 14:37:05,382][20381] Updated weights for policy 1, policy_version 1293568 (0.0007) [2023-12-27 14:37:05,442][20381] Updated weights for policy 1, policy_version 1293578 (0.0008) [2023-12-27 14:37:05,500][20381] Updated weights for policy 1, policy_version 1293588 (0.0007) [2023-12-27 14:37:05,550][20347] Updated weights for policy 0, policy_version 1294518 (0.0006) [2023-12-27 14:37:05,608][20347] Updated weights for policy 0, policy_version 1294528 (0.0005) [2023-12-27 14:37:05,665][20347] Updated weights for policy 0, policy_version 1294538 (0.0005) [2023-12-27 14:37:06,212][20381] Updated weights for policy 1, policy_version 1293598 (0.0009) [2023-12-27 14:37:06,271][20381] Updated weights for policy 1, policy_version 1293608 (0.0008) [2023-12-27 14:37:06,274][20347] Updated weights for policy 0, policy_version 1294548 (0.0008) [2023-12-27 14:37:06,330][20381] Updated weights for policy 1, policy_version 1293618 (0.0006) [2023-12-27 14:37:06,337][20347] Updated weights for policy 0, policy_version 1294558 (0.0011) [2023-12-27 14:37:06,403][20347] Updated weights for policy 0, policy_version 1294568 (0.0010) [2023-12-27 14:37:06,961][20381] Updated weights for policy 1, policy_version 1293628 (0.0006) [2023-12-27 14:37:07,018][20381] Updated weights for policy 1, policy_version 1293638 (0.0007) [2023-12-27 14:37:07,080][20381] Updated weights for policy 1, policy_version 1293648 (0.0008) [2023-12-27 14:37:07,089][20347] Updated weights for policy 0, policy_version 1294578 (0.0011) [2023-12-27 14:37:07,148][20347] Updated weights for policy 0, policy_version 1294588 (0.0011) [2023-12-27 14:37:07,208][20347] Updated weights for policy 0, policy_version 1294598 (0.0009) [2023-12-27 14:37:07,259][20347] Updated weights for policy 0, policy_version 1294608 (0.0010) [2023-12-27 14:37:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.7, 300 sec: 20549.4). Total num frames: 662683648. Throughput: 0: 10228.2, 1: 10358.9. Samples: 662648712. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:07,363][19170] Avg episode reward: [(0, '794.500'), (1, '960.960')] [2023-12-27 14:37:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001293656_331202560.pth... [2023-12-27 14:37:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001294608_331481088.pth... [2023-12-27 14:37:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001292440_330891264.pth [2023-12-27 14:37:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001293424_331177984.pth [2023-12-27 14:37:07,771][20381] Updated weights for policy 1, policy_version 1293658 (0.0007) [2023-12-27 14:37:07,836][20381] Updated weights for policy 1, policy_version 1293668 (0.0008) [2023-12-27 14:37:07,881][20381] Updated weights for policy 1, policy_version 1293678 (0.0006) [2023-12-27 14:37:07,898][20347] Updated weights for policy 0, policy_version 1294618 (0.0006) [2023-12-27 14:37:07,931][20381] Updated weights for policy 1, policy_version 1293688 (0.0007) [2023-12-27 14:37:07,964][20347] Updated weights for policy 0, policy_version 1294628 (0.0006) [2023-12-27 14:37:08,024][20347] Updated weights for policy 0, policy_version 1294638 (0.0007) [2023-12-27 14:37:08,621][20381] Updated weights for policy 1, policy_version 1293698 (0.0008) [2023-12-27 14:37:08,675][20381] Updated weights for policy 1, policy_version 1293708 (0.0007) [2023-12-27 14:37:08,700][20347] Updated weights for policy 0, policy_version 1294648 (0.0008) [2023-12-27 14:37:08,727][20381] Updated weights for policy 1, policy_version 1293718 (0.0006) [2023-12-27 14:37:08,762][20347] Updated weights for policy 0, policy_version 1294658 (0.0006) [2023-12-27 14:37:08,819][20347] Updated weights for policy 0, policy_version 1294668 (0.0008) [2023-12-27 14:37:09,429][20381] Updated weights for policy 1, policy_version 1293728 (0.0007) [2023-12-27 14:37:09,483][20381] Updated weights for policy 1, policy_version 1293738 (0.0008) [2023-12-27 14:37:09,538][20347] Updated weights for policy 0, policy_version 1294678 (0.0007) [2023-12-27 14:37:09,540][20381] Updated weights for policy 1, policy_version 1293748 (0.0006) [2023-12-27 14:37:09,597][20347] Updated weights for policy 0, policy_version 1294688 (0.0008) [2023-12-27 14:37:09,663][20347] Updated weights for policy 0, policy_version 1294698 (0.0008) [2023-12-27 14:37:10,223][20381] Updated weights for policy 1, policy_version 1293758 (0.0006) [2023-12-27 14:37:10,280][20381] Updated weights for policy 1, policy_version 1293768 (0.0007) [2023-12-27 14:37:10,336][20381] Updated weights for policy 1, policy_version 1293778 (0.0008) [2023-12-27 14:37:10,363][20347] Updated weights for policy 0, policy_version 1294708 (0.0006) [2023-12-27 14:37:10,431][20347] Updated weights for policy 0, policy_version 1294718 (0.0006) [2023-12-27 14:37:10,497][20347] Updated weights for policy 0, policy_version 1294728 (0.0008) [2023-12-27 14:37:10,929][20381] Updated weights for policy 1, policy_version 1293788 (0.0007) [2023-12-27 14:37:10,985][20381] Updated weights for policy 1, policy_version 1293798 (0.0007) [2023-12-27 14:37:11,049][20381] Updated weights for policy 1, policy_version 1293808 (0.0007) [2023-12-27 14:37:11,167][20347] Updated weights for policy 0, policy_version 1294738 (0.0008) [2023-12-27 14:37:11,232][20347] Updated weights for policy 0, policy_version 1294748 (0.0008) [2023-12-27 14:37:11,295][20347] Updated weights for policy 0, policy_version 1294758 (0.0008) [2023-12-27 14:37:11,358][20347] Updated weights for policy 0, policy_version 1294768 (0.0008) [2023-12-27 14:37:11,823][20381] Updated weights for policy 1, policy_version 1293818 (0.0008) [2023-12-27 14:37:11,885][20381] Updated weights for policy 1, policy_version 1293828 (0.0008) [2023-12-27 14:37:11,945][20381] Updated weights for policy 1, policy_version 1293838 (0.0010) [2023-12-27 14:37:11,999][20381] Updated weights for policy 1, policy_version 1293848 (0.0011) [2023-12-27 14:37:12,059][20347] Updated weights for policy 0, policy_version 1294778 (0.0008) [2023-12-27 14:37:12,111][20347] Updated weights for policy 0, policy_version 1294788 (0.0007) [2023-12-27 14:37:12,166][20347] Updated weights for policy 0, policy_version 1294798 (0.0007) [2023-12-27 14:37:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 662781952. Throughput: 0: 10214.9, 1: 10331.3. Samples: 662770976. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:12,363][19170] Avg episode reward: [(0, '804.040'), (1, '960.890')] [2023-12-27 14:37:12,720][20381] Updated weights for policy 1, policy_version 1293858 (0.0010) [2023-12-27 14:37:12,782][20381] Updated weights for policy 1, policy_version 1293868 (0.0011) [2023-12-27 14:37:12,831][20381] Updated weights for policy 1, policy_version 1293878 (0.0010) [2023-12-27 14:37:12,924][20347] Updated weights for policy 0, policy_version 1294808 (0.0011) [2023-12-27 14:37:12,980][20347] Updated weights for policy 0, policy_version 1294818 (0.0010) [2023-12-27 14:37:13,039][20347] Updated weights for policy 0, policy_version 1294828 (0.0010) [2023-12-27 14:37:13,508][20381] Updated weights for policy 1, policy_version 1293888 (0.0008) [2023-12-27 14:37:13,567][20381] Updated weights for policy 1, policy_version 1293898 (0.0007) [2023-12-27 14:37:13,628][20381] Updated weights for policy 1, policy_version 1293908 (0.0007) [2023-12-27 14:37:13,641][20347] Updated weights for policy 0, policy_version 1294838 (0.0010) [2023-12-27 14:37:13,689][20347] Updated weights for policy 0, policy_version 1294848 (0.0010) [2023-12-27 14:37:13,748][20347] Updated weights for policy 0, policy_version 1294858 (0.0010) [2023-12-27 14:37:14,290][20381] Updated weights for policy 1, policy_version 1293918 (0.0007) [2023-12-27 14:37:14,352][20381] Updated weights for policy 1, policy_version 1293928 (0.0008) [2023-12-27 14:37:14,391][20347] Updated weights for policy 0, policy_version 1294868 (0.0009) [2023-12-27 14:37:14,405][20381] Updated weights for policy 1, policy_version 1293938 (0.0006) [2023-12-27 14:37:14,446][20347] Updated weights for policy 0, policy_version 1294878 (0.0005) [2023-12-27 14:37:14,497][20347] Updated weights for policy 0, policy_version 1294888 (0.0010) [2023-12-27 14:37:15,129][20381] Updated weights for policy 1, policy_version 1293948 (0.0009) [2023-12-27 14:37:15,136][20347] Updated weights for policy 0, policy_version 1294898 (0.0006) [2023-12-27 14:37:15,186][20381] Updated weights for policy 1, policy_version 1293958 (0.0007) [2023-12-27 14:37:15,192][20347] Updated weights for policy 0, policy_version 1294908 (0.0008) [2023-12-27 14:37:15,244][20381] Updated weights for policy 1, policy_version 1293968 (0.0007) [2023-12-27 14:37:15,250][20347] Updated weights for policy 0, policy_version 1294918 (0.0005) [2023-12-27 14:37:15,300][20347] Updated weights for policy 0, policy_version 1294928 (0.0005) [2023-12-27 14:37:15,848][20381] Updated weights for policy 1, policy_version 1293978 (0.0007) [2023-12-27 14:37:15,907][20381] Updated weights for policy 1, policy_version 1293988 (0.0005) [2023-12-27 14:37:15,962][20381] Updated weights for policy 1, policy_version 1293998 (0.0008) [2023-12-27 14:37:15,978][20347] Updated weights for policy 0, policy_version 1294938 (0.0011) [2023-12-27 14:37:16,014][20381] Updated weights for policy 1, policy_version 1294008 (0.0010) [2023-12-27 14:37:16,030][20347] Updated weights for policy 0, policy_version 1294948 (0.0010) [2023-12-27 14:37:16,082][20347] Updated weights for policy 0, policy_version 1294958 (0.0011) [2023-12-27 14:37:16,660][20347] Updated weights for policy 0, policy_version 1294968 (0.0007) [2023-12-27 14:37:16,673][20381] Updated weights for policy 1, policy_version 1294018 (0.0011) [2023-12-27 14:37:16,712][20347] Updated weights for policy 0, policy_version 1294978 (0.0005) [2023-12-27 14:37:16,726][20381] Updated weights for policy 1, policy_version 1294028 (0.0010) [2023-12-27 14:37:16,766][20347] Updated weights for policy 0, policy_version 1294988 (0.0006) [2023-12-27 14:37:16,781][20381] Updated weights for policy 1, policy_version 1294038 (0.0007) [2023-12-27 14:37:17,358][20381] Updated weights for policy 1, policy_version 1294048 (0.0005) [2023-12-27 14:37:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 662880256. Throughput: 0: 10202.0, 1: 10317.2. Samples: 662896008. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:17,363][19170] Avg episode reward: [(0, '814.120'), (1, '980.460')] [2023-12-27 14:37:17,408][20381] Updated weights for policy 1, policy_version 1294058 (0.0007) [2023-12-27 14:37:17,435][20347] Updated weights for policy 0, policy_version 1294998 (0.0006) [2023-12-27 14:37:17,455][20381] Updated weights for policy 1, policy_version 1294068 (0.0010) [2023-12-27 14:37:17,480][20347] Updated weights for policy 0, policy_version 1295008 (0.0005) [2023-12-27 14:37:17,531][20347] Updated weights for policy 0, policy_version 1295018 (0.0005) [2023-12-27 14:37:18,003][20381] Updated weights for policy 1, policy_version 1294078 (0.0008) [2023-12-27 14:37:18,071][20381] Updated weights for policy 1, policy_version 1294088 (0.0005) [2023-12-27 14:37:18,132][20381] Updated weights for policy 1, policy_version 1294098 (0.0005) [2023-12-27 14:37:18,153][20347] Updated weights for policy 0, policy_version 1295028 (0.0006) [2023-12-27 14:37:18,198][20347] Updated weights for policy 0, policy_version 1295038 (0.0006) [2023-12-27 14:37:18,255][20347] Updated weights for policy 0, policy_version 1295048 (0.0007) [2023-12-27 14:37:18,708][20381] Updated weights for policy 1, policy_version 1294108 (0.0007) [2023-12-27 14:37:18,760][20381] Updated weights for policy 1, policy_version 1294118 (0.0010) [2023-12-27 14:37:18,807][20381] Updated weights for policy 1, policy_version 1294128 (0.0010) [2023-12-27 14:37:18,919][20347] Updated weights for policy 0, policy_version 1295058 (0.0007) [2023-12-27 14:37:18,967][20347] Updated weights for policy 0, policy_version 1295068 (0.0008) [2023-12-27 14:37:19,021][20347] Updated weights for policy 0, policy_version 1295078 (0.0007) [2023-12-27 14:37:19,078][20347] Updated weights for policy 0, policy_version 1295088 (0.0005) [2023-12-27 14:37:19,575][20381] Updated weights for policy 1, policy_version 1294138 (0.0011) [2023-12-27 14:37:19,633][20381] Updated weights for policy 1, policy_version 1294148 (0.0009) [2023-12-27 14:37:19,691][20381] Updated weights for policy 1, policy_version 1294158 (0.0005) [2023-12-27 14:37:19,753][20381] Updated weights for policy 1, policy_version 1294168 (0.0008) [2023-12-27 14:37:19,834][20347] Updated weights for policy 0, policy_version 1295098 (0.0008) [2023-12-27 14:37:19,893][20347] Updated weights for policy 0, policy_version 1295108 (0.0007) [2023-12-27 14:37:19,953][20347] Updated weights for policy 0, policy_version 1295118 (0.0007) [2023-12-27 14:37:20,464][20381] Updated weights for policy 1, policy_version 1294178 (0.0010) [2023-12-27 14:37:20,509][20381] Updated weights for policy 1, policy_version 1294188 (0.0010) [2023-12-27 14:37:20,565][20381] Updated weights for policy 1, policy_version 1294198 (0.0011) [2023-12-27 14:37:20,681][20347] Updated weights for policy 0, policy_version 1295128 (0.0007) [2023-12-27 14:37:20,745][20347] Updated weights for policy 0, policy_version 1295138 (0.0006) [2023-12-27 14:37:20,804][20347] Updated weights for policy 0, policy_version 1295148 (0.0006) [2023-12-27 14:37:21,291][20381] Updated weights for policy 1, policy_version 1294208 (0.0009) [2023-12-27 14:37:21,362][20381] Updated weights for policy 1, policy_version 1294218 (0.0009) [2023-12-27 14:37:21,397][20291] KL-divergence is very high: 149.8427 [2023-12-27 14:37:21,413][20291] KL-divergence is very high: 175.4859 [2023-12-27 14:37:21,417][20381] Updated weights for policy 1, policy_version 1294228 (0.0008) [2023-12-27 14:37:21,420][20347] Updated weights for policy 0, policy_version 1295158 (0.0007) [2023-12-27 14:37:21,479][20347] Updated weights for policy 0, policy_version 1295168 (0.0007) [2023-12-27 14:37:21,537][20347] Updated weights for policy 0, policy_version 1295178 (0.0007) [2023-12-27 14:37:22,165][20381] Updated weights for policy 1, policy_version 1294238 (0.0008) [2023-12-27 14:37:22,216][20381] Updated weights for policy 1, policy_version 1294248 (0.0008) [2023-12-27 14:37:22,257][20347] Updated weights for policy 0, policy_version 1295188 (0.0008) [2023-12-27 14:37:22,280][20381] Updated weights for policy 1, policy_version 1294258 (0.0007) [2023-12-27 14:37:22,311][20347] Updated weights for policy 0, policy_version 1295198 (0.0011) [2023-12-27 14:37:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 662986752. Throughput: 0: 10200.6, 1: 10352.0. Samples: 662958892. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:22,363][19170] Avg episode reward: [(0, '834.060'), (1, '990.300')] [2023-12-27 14:37:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001294264_331358208.pth... [2023-12-27 14:37:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001293048_331046912.pth [2023-12-27 14:37:22,379][20347] Updated weights for policy 0, policy_version 1295208 (0.0010) [2023-12-27 14:37:22,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001295216_331636736.pth... [2023-12-27 14:37:22,424][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001294000_331325440.pth [2023-12-27 14:37:22,997][20347] Updated weights for policy 0, policy_version 1295218 (0.0010) [2023-12-27 14:37:23,022][20381] Updated weights for policy 1, policy_version 1294268 (0.0008) [2023-12-27 14:37:23,052][20347] Updated weights for policy 0, policy_version 1295228 (0.0008) [2023-12-27 14:37:23,076][20381] Updated weights for policy 1, policy_version 1294278 (0.0010) [2023-12-27 14:37:23,106][20347] Updated weights for policy 0, policy_version 1295238 (0.0010) [2023-12-27 14:37:23,133][20381] Updated weights for policy 1, policy_version 1294288 (0.0010) [2023-12-27 14:37:23,161][20347] Updated weights for policy 0, policy_version 1295248 (0.0010) [2023-12-27 14:37:23,725][20381] Updated weights for policy 1, policy_version 1294298 (0.0009) [2023-12-27 14:37:23,758][20347] Updated weights for policy 0, policy_version 1295258 (0.0006) [2023-12-27 14:37:23,779][20381] Updated weights for policy 1, policy_version 1294308 (0.0005) [2023-12-27 14:37:23,815][20347] Updated weights for policy 0, policy_version 1295268 (0.0005) [2023-12-27 14:37:23,838][20381] Updated weights for policy 1, policy_version 1294318 (0.0005) [2023-12-27 14:37:23,871][20347] Updated weights for policy 0, policy_version 1295278 (0.0006) [2023-12-27 14:37:23,892][20381] Updated weights for policy 1, policy_version 1294328 (0.0005) [2023-12-27 14:37:24,476][20347] Updated weights for policy 0, policy_version 1295288 (0.0007) [2023-12-27 14:37:24,518][20381] Updated weights for policy 1, policy_version 1294338 (0.0007) [2023-12-27 14:37:24,531][20347] Updated weights for policy 0, policy_version 1295298 (0.0006) [2023-12-27 14:37:24,569][20381] Updated weights for policy 1, policy_version 1294348 (0.0010) [2023-12-27 14:37:24,582][20347] Updated weights for policy 0, policy_version 1295308 (0.0006) [2023-12-27 14:37:24,620][20381] Updated weights for policy 1, policy_version 1294358 (0.0009) [2023-12-27 14:37:25,141][20347] Updated weights for policy 0, policy_version 1295318 (0.0007) [2023-12-27 14:37:25,195][20347] Updated weights for policy 0, policy_version 1295328 (0.0008) [2023-12-27 14:37:25,225][20381] Updated weights for policy 1, policy_version 1294368 (0.0010) [2023-12-27 14:37:25,248][20347] Updated weights for policy 0, policy_version 1295338 (0.0009) [2023-12-27 14:37:25,287][20381] Updated weights for policy 1, policy_version 1294378 (0.0010) [2023-12-27 14:37:25,340][20381] Updated weights for policy 1, policy_version 1294388 (0.0010) [2023-12-27 14:37:25,950][20347] Updated weights for policy 0, policy_version 1295348 (0.0007) [2023-12-27 14:37:25,997][20347] Updated weights for policy 0, policy_version 1295358 (0.0007) [2023-12-27 14:37:26,039][20347] Updated weights for policy 0, policy_version 1295368 (0.0007) [2023-12-27 14:37:26,042][20381] Updated weights for policy 1, policy_version 1294398 (0.0011) [2023-12-27 14:37:26,098][20381] Updated weights for policy 1, policy_version 1294408 (0.0011) [2023-12-27 14:37:26,144][20381] Updated weights for policy 1, policy_version 1294418 (0.0010) [2023-12-27 14:37:26,763][20347] Updated weights for policy 0, policy_version 1295378 (0.0006) [2023-12-27 14:37:26,773][20381] Updated weights for policy 1, policy_version 1294428 (0.0008) [2023-12-27 14:37:26,821][20381] Updated weights for policy 1, policy_version 1294438 (0.0006) [2023-12-27 14:37:26,824][20347] Updated weights for policy 0, policy_version 1295388 (0.0007) [2023-12-27 14:37:26,871][20381] Updated weights for policy 1, policy_version 1294448 (0.0005) [2023-12-27 14:37:26,879][20347] Updated weights for policy 0, policy_version 1295398 (0.0007) [2023-12-27 14:37:26,937][20347] Updated weights for policy 0, policy_version 1295408 (0.0007) [2023-12-27 14:37:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 663093248. Throughput: 0: 10264.3, 1: 10361.2. Samples: 663084180. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:27,363][19170] Avg episode reward: [(0, '804.450'), (1, '990.300')] [2023-12-27 14:37:27,403][20381] Updated weights for policy 1, policy_version 1294458 (0.0006) [2023-12-27 14:37:27,461][20381] Updated weights for policy 1, policy_version 1294468 (0.0005) [2023-12-27 14:37:27,519][20381] Updated weights for policy 1, policy_version 1294478 (0.0006) [2023-12-27 14:37:27,546][20347] Updated weights for policy 0, policy_version 1295418 (0.0009) [2023-12-27 14:37:27,575][20381] Updated weights for policy 1, policy_version 1294488 (0.0006) [2023-12-27 14:37:27,604][20347] Updated weights for policy 0, policy_version 1295428 (0.0006) [2023-12-27 14:37:27,660][20347] Updated weights for policy 0, policy_version 1295438 (0.0005) [2023-12-27 14:37:28,189][20381] Updated weights for policy 1, policy_version 1294498 (0.0010) [2023-12-27 14:37:28,240][20381] Updated weights for policy 1, policy_version 1294508 (0.0010) [2023-12-27 14:37:28,244][20347] Updated weights for policy 0, policy_version 1295448 (0.0005) [2023-12-27 14:37:28,289][20381] Updated weights for policy 1, policy_version 1294518 (0.0010) [2023-12-27 14:37:28,307][20347] Updated weights for policy 0, policy_version 1295458 (0.0005) [2023-12-27 14:37:28,377][20347] Updated weights for policy 0, policy_version 1295468 (0.0007) [2023-12-27 14:37:28,959][20347] Updated weights for policy 0, policy_version 1295478 (0.0008) [2023-12-27 14:37:29,006][20347] Updated weights for policy 0, policy_version 1295488 (0.0008) [2023-12-27 14:37:29,029][20381] Updated weights for policy 1, policy_version 1294528 (0.0011) [2023-12-27 14:37:29,058][20347] Updated weights for policy 0, policy_version 1295498 (0.0007) [2023-12-27 14:37:29,083][20381] Updated weights for policy 1, policy_version 1294538 (0.0010) [2023-12-27 14:37:29,138][20381] Updated weights for policy 1, policy_version 1294548 (0.0010) [2023-12-27 14:37:29,677][20347] Updated weights for policy 0, policy_version 1295508 (0.0007) [2023-12-27 14:37:29,736][20347] Updated weights for policy 0, policy_version 1295518 (0.0005) [2023-12-27 14:37:29,796][20347] Updated weights for policy 0, policy_version 1295528 (0.0008) [2023-12-27 14:37:29,843][20381] Updated weights for policy 1, policy_version 1294558 (0.0010) [2023-12-27 14:37:29,901][20381] Updated weights for policy 1, policy_version 1294568 (0.0008) [2023-12-27 14:37:29,964][20381] Updated weights for policy 1, policy_version 1294578 (0.0008) [2023-12-27 14:37:30,370][20347] Updated weights for policy 0, policy_version 1295538 (0.0009) [2023-12-27 14:37:30,421][20347] Updated weights for policy 0, policy_version 1295548 (0.0010) [2023-12-27 14:37:30,475][20347] Updated weights for policy 0, policy_version 1295558 (0.0007) [2023-12-27 14:37:30,525][20347] Updated weights for policy 0, policy_version 1295568 (0.0005) [2023-12-27 14:37:30,617][20381] Updated weights for policy 1, policy_version 1294588 (0.0010) [2023-12-27 14:37:30,661][20381] Updated weights for policy 1, policy_version 1294598 (0.0010) [2023-12-27 14:37:30,706][20381] Updated weights for policy 1, policy_version 1294608 (0.0010) [2023-12-27 14:37:31,116][20347] Updated weights for policy 0, policy_version 1295578 (0.0008) [2023-12-27 14:37:31,183][20347] Updated weights for policy 0, policy_version 1295588 (0.0008) [2023-12-27 14:37:31,239][20347] Updated weights for policy 0, policy_version 1295598 (0.0007) [2023-12-27 14:37:31,417][20381] Updated weights for policy 1, policy_version 1294618 (0.0010) [2023-12-27 14:37:31,470][20381] Updated weights for policy 1, policy_version 1294628 (0.0009) [2023-12-27 14:37:31,522][20381] Updated weights for policy 1, policy_version 1294638 (0.0008) [2023-12-27 14:37:31,573][20381] Updated weights for policy 1, policy_version 1294648 (0.0005) [2023-12-27 14:37:31,957][20347] Updated weights for policy 0, policy_version 1295608 (0.0009) [2023-12-27 14:37:32,005][20347] Updated weights for policy 0, policy_version 1295618 (0.0010) [2023-12-27 14:37:32,055][20347] Updated weights for policy 0, policy_version 1295628 (0.0010) [2023-12-27 14:37:32,204][20381] Updated weights for policy 1, policy_version 1294658 (0.0007) [2023-12-27 14:37:32,252][20381] Updated weights for policy 1, policy_version 1294668 (0.0006) [2023-12-27 14:37:32,310][20381] Updated weights for policy 1, policy_version 1294678 (0.0011) [2023-12-27 14:37:32,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 663207936. Throughput: 0: 10419.3, 1: 10384.2. Samples: 663213720. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:32,363][19170] Avg episode reward: [(0, '804.150'), (1, '1000.000')] [2023-12-27 14:37:32,784][20347] Updated weights for policy 0, policy_version 1295638 (0.0010) [2023-12-27 14:37:32,845][20347] Updated weights for policy 0, policy_version 1295648 (0.0009) [2023-12-27 14:37:32,904][20347] Updated weights for policy 0, policy_version 1295658 (0.0007) [2023-12-27 14:37:32,925][20381] Updated weights for policy 1, policy_version 1294688 (0.0007) [2023-12-27 14:37:32,979][20381] Updated weights for policy 1, policy_version 1294698 (0.0006) [2023-12-27 14:37:33,028][20381] Updated weights for policy 1, policy_version 1294708 (0.0005) [2023-12-27 14:37:33,576][20347] Updated weights for policy 0, policy_version 1295668 (0.0007) [2023-12-27 14:37:33,623][20347] Updated weights for policy 0, policy_version 1295678 (0.0006) [2023-12-27 14:37:33,625][20381] Updated weights for policy 1, policy_version 1294718 (0.0005) [2023-12-27 14:37:33,671][20347] Updated weights for policy 0, policy_version 1295688 (0.0007) [2023-12-27 14:37:33,684][20381] Updated weights for policy 1, policy_version 1294728 (0.0005) [2023-12-27 14:37:33,739][20381] Updated weights for policy 1, policy_version 1294738 (0.0005) [2023-12-27 14:37:34,373][20381] Updated weights for policy 1, policy_version 1294748 (0.0006) [2023-12-27 14:37:34,379][20347] Updated weights for policy 0, policy_version 1295698 (0.0007) [2023-12-27 14:37:34,435][20381] Updated weights for policy 1, policy_version 1294758 (0.0006) [2023-12-27 14:37:34,438][20347] Updated weights for policy 0, policy_version 1295708 (0.0006) [2023-12-27 14:37:34,492][20381] Updated weights for policy 1, policy_version 1294768 (0.0008) [2023-12-27 14:37:34,499][20347] Updated weights for policy 0, policy_version 1295718 (0.0006) [2023-12-27 14:37:34,562][20347] Updated weights for policy 0, policy_version 1295728 (0.0008) [2023-12-27 14:37:35,174][20381] Updated weights for policy 1, policy_version 1294778 (0.0008) [2023-12-27 14:37:35,234][20381] Updated weights for policy 1, policy_version 1294788 (0.0007) [2023-12-27 14:37:35,239][20347] Updated weights for policy 0, policy_version 1295738 (0.0007) [2023-12-27 14:37:35,296][20347] Updated weights for policy 0, policy_version 1295748 (0.0007) [2023-12-27 14:37:35,298][20381] Updated weights for policy 1, policy_version 1294798 (0.0007) [2023-12-27 14:37:35,351][20381] Updated weights for policy 1, policy_version 1294808 (0.0006) [2023-12-27 14:37:35,353][20347] Updated weights for policy 0, policy_version 1295758 (0.0006) [2023-12-27 14:37:35,984][20347] Updated weights for policy 0, policy_version 1295768 (0.0008) [2023-12-27 14:37:36,006][20381] Updated weights for policy 1, policy_version 1294818 (0.0005) [2023-12-27 14:37:36,037][20347] Updated weights for policy 0, policy_version 1295778 (0.0008) [2023-12-27 14:37:36,061][20381] Updated weights for policy 1, policy_version 1294828 (0.0005) [2023-12-27 14:37:36,093][20347] Updated weights for policy 0, policy_version 1295788 (0.0007) [2023-12-27 14:37:36,120][20381] Updated weights for policy 1, policy_version 1294838 (0.0006) [2023-12-27 14:37:36,747][20381] Updated weights for policy 1, policy_version 1294848 (0.0008) [2023-12-27 14:37:36,803][20381] Updated weights for policy 1, policy_version 1294858 (0.0008) [2023-12-27 14:37:36,846][20292] KL-divergence is very high: 242.8000 [2023-12-27 14:37:36,852][20347] Updated weights for policy 0, policy_version 1295798 (0.0007) [2023-12-27 14:37:36,865][20381] Updated weights for policy 1, policy_version 1294868 (0.0006) [2023-12-27 14:37:36,907][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000005 [2023-12-27 14:37:36,909][20347] Updated weights for policy 0, policy_version 1295808 (0.0007) [2023-12-27 14:37:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 663306240. Throughput: 0: 10389.2, 1: 10399.8. Samples: 663276164. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:37,364][19170] Avg episode reward: [(0, '843.210'), (1, '980.420')] [2023-12-27 14:37:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001294872_331513856.pth... [2023-12-27 14:37:37,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001295808_331792384.pth... [2023-12-27 14:37:37,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001293656_331202560.pth [2023-12-27 14:37:37,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001294608_331481088.pth [2023-12-27 14:37:37,469][20381] Updated weights for policy 1, policy_version 1294878 (0.0006) [2023-12-27 14:37:37,535][20381] Updated weights for policy 1, policy_version 1294888 (0.0006) [2023-12-27 14:37:37,591][20381] Updated weights for policy 1, policy_version 1294898 (0.0007) [2023-12-27 14:37:37,717][20347] Updated weights for policy 0, policy_version 1295818 (0.0007) [2023-12-27 14:37:37,763][20347] Updated weights for policy 0, policy_version 1295828 (0.0008) [2023-12-27 14:37:37,808][20347] Updated weights for policy 0, policy_version 1295838 (0.0007) [2023-12-27 14:37:38,226][20381] Updated weights for policy 1, policy_version 1294908 (0.0007) [2023-12-27 14:37:38,292][20381] Updated weights for policy 1, policy_version 1294918 (0.0007) [2023-12-27 14:37:38,355][20381] Updated weights for policy 1, policy_version 1294928 (0.0008) [2023-12-27 14:37:38,556][20347] Updated weights for policy 0, policy_version 1295848 (0.0007) [2023-12-27 14:37:38,613][20347] Updated weights for policy 0, policy_version 1295858 (0.0007) [2023-12-27 14:37:38,673][20347] Updated weights for policy 0, policy_version 1295868 (0.0005) [2023-12-27 14:37:39,052][20381] Updated weights for policy 1, policy_version 1294938 (0.0010) [2023-12-27 14:37:39,107][20381] Updated weights for policy 1, policy_version 1294948 (0.0011) [2023-12-27 14:37:39,162][20381] Updated weights for policy 1, policy_version 1294958 (0.0011) [2023-12-27 14:37:39,219][20381] Updated weights for policy 1, policy_version 1294968 (0.0009) [2023-12-27 14:37:39,299][20347] Updated weights for policy 0, policy_version 1295878 (0.0008) [2023-12-27 14:37:39,369][20347] Updated weights for policy 0, policy_version 1295888 (0.0010) [2023-12-27 14:37:39,442][20347] Updated weights for policy 0, policy_version 1295898 (0.0010) [2023-12-27 14:37:40,000][20381] Updated weights for policy 1, policy_version 1294978 (0.0008) [2023-12-27 14:37:40,057][20381] Updated weights for policy 1, policy_version 1294988 (0.0008) [2023-12-27 14:37:40,115][20347] Updated weights for policy 0, policy_version 1295908 (0.0009) [2023-12-27 14:37:40,118][20381] Updated weights for policy 1, policy_version 1294998 (0.0008) [2023-12-27 14:37:40,168][20347] Updated weights for policy 0, policy_version 1295918 (0.0010) [2023-12-27 14:37:40,222][20347] Updated weights for policy 0, policy_version 1295928 (0.0006) [2023-12-27 14:37:40,823][20381] Updated weights for policy 1, policy_version 1295008 (0.0006) [2023-12-27 14:37:40,876][20381] Updated weights for policy 1, policy_version 1295018 (0.0006) [2023-12-27 14:37:40,878][20347] Updated weights for policy 0, policy_version 1295938 (0.0005) [2023-12-27 14:37:40,932][20381] Updated weights for policy 1, policy_version 1295028 (0.0006) [2023-12-27 14:37:40,938][20347] Updated weights for policy 0, policy_version 1295948 (0.0008) [2023-12-27 14:37:40,998][20347] Updated weights for policy 0, policy_version 1295958 (0.0008) [2023-12-27 14:37:41,064][20347] Updated weights for policy 0, policy_version 1295968 (0.0008) [2023-12-27 14:37:41,585][20381] Updated weights for policy 1, policy_version 1295038 (0.0006) [2023-12-27 14:37:41,654][20381] Updated weights for policy 1, policy_version 1295048 (0.0007) [2023-12-27 14:37:41,711][20381] Updated weights for policy 1, policy_version 1295058 (0.0008) [2023-12-27 14:37:41,831][20347] Updated weights for policy 0, policy_version 1295978 (0.0007) [2023-12-27 14:37:41,887][20347] Updated weights for policy 0, policy_version 1295988 (0.0007) [2023-12-27 14:37:41,952][20347] Updated weights for policy 0, policy_version 1295998 (0.0008) [2023-12-27 14:37:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 663404544. Throughput: 0: 10372.8, 1: 10402.2. Samples: 663397828. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:42,363][19170] Avg episode reward: [(0, '794.820'), (1, '960.860')] [2023-12-27 14:37:42,418][20381] Updated weights for policy 1, policy_version 1295068 (0.0008) [2023-12-27 14:37:42,471][20381] Updated weights for policy 1, policy_version 1295078 (0.0007) [2023-12-27 14:37:42,535][20381] Updated weights for policy 1, policy_version 1295088 (0.0007) [2023-12-27 14:37:42,635][20347] Updated weights for policy 0, policy_version 1296008 (0.0008) [2023-12-27 14:37:42,697][20347] Updated weights for policy 0, policy_version 1296018 (0.0007) [2023-12-27 14:37:42,756][20347] Updated weights for policy 0, policy_version 1296028 (0.0008) [2023-12-27 14:37:43,206][20381] Updated weights for policy 1, policy_version 1295098 (0.0008) [2023-12-27 14:37:43,258][20381] Updated weights for policy 1, policy_version 1295108 (0.0008) [2023-12-27 14:37:43,308][20381] Updated weights for policy 1, policy_version 1295118 (0.0008) [2023-12-27 14:37:43,360][20381] Updated weights for policy 1, policy_version 1295128 (0.0008) [2023-12-27 14:37:43,428][20347] Updated weights for policy 0, policy_version 1296038 (0.0006) [2023-12-27 14:37:43,481][20347] Updated weights for policy 0, policy_version 1296048 (0.0006) [2023-12-27 14:37:43,535][20347] Updated weights for policy 0, policy_version 1296058 (0.0005) [2023-12-27 14:37:44,079][20381] Updated weights for policy 1, policy_version 1295138 (0.0007) [2023-12-27 14:37:44,134][20381] Updated weights for policy 1, policy_version 1295148 (0.0008) [2023-12-27 14:37:44,187][20381] Updated weights for policy 1, policy_version 1295158 (0.0008) [2023-12-27 14:37:44,201][20347] Updated weights for policy 0, policy_version 1296068 (0.0006) [2023-12-27 14:37:44,260][20347] Updated weights for policy 0, policy_version 1296078 (0.0008) [2023-12-27 14:37:44,322][20347] Updated weights for policy 0, policy_version 1296088 (0.0005) [2023-12-27 14:37:44,894][20381] Updated weights for policy 1, policy_version 1295168 (0.0009) [2023-12-27 14:37:44,930][20347] Updated weights for policy 0, policy_version 1296098 (0.0005) [2023-12-27 14:37:44,953][20381] Updated weights for policy 1, policy_version 1295178 (0.0008) [2023-12-27 14:37:44,998][20347] Updated weights for policy 0, policy_version 1296108 (0.0006) [2023-12-27 14:37:45,011][20381] Updated weights for policy 1, policy_version 1295188 (0.0009) [2023-12-27 14:37:45,062][20347] Updated weights for policy 0, policy_version 1296118 (0.0008) [2023-12-27 14:37:45,118][20347] Updated weights for policy 0, policy_version 1296128 (0.0007) [2023-12-27 14:37:45,697][20381] Updated weights for policy 1, policy_version 1295198 (0.0007) [2023-12-27 14:37:45,755][20381] Updated weights for policy 1, policy_version 1295208 (0.0007) [2023-12-27 14:37:45,776][20347] Updated weights for policy 0, policy_version 1296138 (0.0007) [2023-12-27 14:37:45,810][20381] Updated weights for policy 1, policy_version 1295218 (0.0008) [2023-12-27 14:37:45,835][20347] Updated weights for policy 0, policy_version 1296148 (0.0005) [2023-12-27 14:37:45,888][20347] Updated weights for policy 0, policy_version 1296158 (0.0005) [2023-12-27 14:37:46,430][20347] Updated weights for policy 0, policy_version 1296168 (0.0009) [2023-12-27 14:37:46,480][20347] Updated weights for policy 0, policy_version 1296178 (0.0010) [2023-12-27 14:37:46,531][20381] Updated weights for policy 1, policy_version 1295228 (0.0008) [2023-12-27 14:37:46,532][20347] Updated weights for policy 0, policy_version 1296188 (0.0010) [2023-12-27 14:37:46,589][20381] Updated weights for policy 1, policy_version 1295238 (0.0008) [2023-12-27 14:37:46,639][20381] Updated weights for policy 1, policy_version 1295248 (0.0007) [2023-12-27 14:37:47,156][20347] Updated weights for policy 0, policy_version 1296198 (0.0007) [2023-12-27 14:37:47,218][20347] Updated weights for policy 0, policy_version 1296208 (0.0010) [2023-12-27 14:37:47,276][20347] Updated weights for policy 0, policy_version 1296218 (0.0010) [2023-12-27 14:37:47,297][20381] Updated weights for policy 1, policy_version 1295258 (0.0007) [2023-12-27 14:37:47,356][20381] Updated weights for policy 1, policy_version 1295268 (0.0005) [2023-12-27 14:37:47,363][19170] Fps is (10 sec: 20480.3, 60 sec: 20889.5, 300 sec: 20577.2). Total num frames: 663511040. Throughput: 0: 10429.3, 1: 10330.0. Samples: 663520772. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-12-27 14:37:47,364][19170] Avg episode reward: [(0, '795.140'), (1, '980.440')] [2023-12-27 14:37:47,414][20381] Updated weights for policy 1, policy_version 1295278 (0.0007) [2023-12-27 14:37:47,472][20381] Updated weights for policy 1, policy_version 1295288 (0.0007) [2023-12-27 14:37:47,924][20347] Updated weights for policy 0, policy_version 1296228 (0.0009) [2023-12-27 14:37:47,985][20347] Updated weights for policy 0, policy_version 1296238 (0.0007) [2023-12-27 14:37:48,008][20381] Updated weights for policy 1, policy_version 1295298 (0.0005) [2023-12-27 14:37:48,041][20347] Updated weights for policy 0, policy_version 1296248 (0.0008) [2023-12-27 14:37:48,072][20381] Updated weights for policy 1, policy_version 1295308 (0.0005) [2023-12-27 14:37:48,132][20381] Updated weights for policy 1, policy_version 1295318 (0.0005) [2023-12-27 14:37:48,676][20381] Updated weights for policy 1, policy_version 1295328 (0.0005) [2023-12-27 14:37:48,742][20381] Updated weights for policy 1, policy_version 1295338 (0.0007) [2023-12-27 14:37:48,765][20347] Updated weights for policy 0, policy_version 1296258 (0.0006) [2023-12-27 14:37:48,801][20381] Updated weights for policy 1, policy_version 1295348 (0.0008) [2023-12-27 14:37:48,826][20347] Updated weights for policy 0, policy_version 1296268 (0.0006) [2023-12-27 14:37:48,885][20347] Updated weights for policy 0, policy_version 1296278 (0.0008) [2023-12-27 14:37:48,944][20347] Updated weights for policy 0, policy_version 1296288 (0.0008) [2023-12-27 14:37:49,493][20381] Updated weights for policy 1, policy_version 1295358 (0.0009) [2023-12-27 14:37:49,561][20381] Updated weights for policy 1, policy_version 1295368 (0.0010) [2023-12-27 14:37:49,621][20381] Updated weights for policy 1, policy_version 1295378 (0.0011) [2023-12-27 14:37:49,675][20347] Updated weights for policy 0, policy_version 1296298 (0.0007) [2023-12-27 14:37:49,737][20347] Updated weights for policy 0, policy_version 1296308 (0.0008) [2023-12-27 14:37:49,801][20347] Updated weights for policy 0, policy_version 1296318 (0.0008) [2023-12-27 14:37:50,339][20381] Updated weights for policy 1, policy_version 1295388 (0.0009) [2023-12-27 14:37:50,404][20381] Updated weights for policy 1, policy_version 1295398 (0.0007) [2023-12-27 14:37:50,462][20381] Updated weights for policy 1, policy_version 1295408 (0.0008) [2023-12-27 14:37:50,548][20347] Updated weights for policy 0, policy_version 1296328 (0.0007) [2023-12-27 14:37:50,618][20347] Updated weights for policy 0, policy_version 1296339 (0.0009) [2023-12-27 14:37:50,674][20347] Updated weights for policy 0, policy_version 1296349 (0.0008) [2023-12-27 14:37:51,164][20381] Updated weights for policy 1, policy_version 1295418 (0.0007) [2023-12-27 14:37:51,225][20381] Updated weights for policy 1, policy_version 1295428 (0.0008) [2023-12-27 14:37:51,284][20381] Updated weights for policy 1, policy_version 1295438 (0.0007) [2023-12-27 14:37:51,333][20381] Updated weights for policy 1, policy_version 1295448 (0.0008) [2023-12-27 14:37:51,392][20347] Updated weights for policy 0, policy_version 1296359 (0.0007) [2023-12-27 14:37:51,444][20347] Updated weights for policy 0, policy_version 1296369 (0.0007) [2023-12-27 14:37:51,505][20347] Updated weights for policy 0, policy_version 1296379 (0.0008) [2023-12-27 14:37:52,084][20381] Updated weights for policy 1, policy_version 1295458 (0.0007) [2023-12-27 14:37:52,140][20381] Updated weights for policy 1, policy_version 1295468 (0.0006) [2023-12-27 14:37:52,199][20381] Updated weights for policy 1, policy_version 1295478 (0.0006) [2023-12-27 14:37:52,237][20347] Updated weights for policy 0, policy_version 1296389 (0.0009) [2023-12-27 14:37:52,303][20347] Updated weights for policy 0, policy_version 1296399 (0.0010) [2023-12-27 14:37:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 663609344. Throughput: 0: 10394.4, 1: 10363.5. Samples: 663582820. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:37:52,363][19170] Avg episode reward: [(0, '873.030'), (1, '980.440')] [2023-12-27 14:37:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001295480_331669504.pth... [2023-12-27 14:37:52,371][20347] Updated weights for policy 0, policy_version 1296409 (0.0011) [2023-12-27 14:37:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001294264_331358208.pth [2023-12-27 14:37:52,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001296416_331948032.pth... [2023-12-27 14:37:52,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001295216_331636736.pth [2023-12-27 14:37:52,834][20381] Updated weights for policy 1, policy_version 1295488 (0.0007) [2023-12-27 14:37:52,890][20381] Updated weights for policy 1, policy_version 1295498 (0.0009) [2023-12-27 14:37:52,946][20381] Updated weights for policy 1, policy_version 1295508 (0.0008) [2023-12-27 14:37:52,997][20347] Updated weights for policy 0, policy_version 1296419 (0.0008) [2023-12-27 14:37:53,050][20347] Updated weights for policy 0, policy_version 1296429 (0.0007) [2023-12-27 14:37:53,103][20347] Updated weights for policy 0, policy_version 1296439 (0.0008) [2023-12-27 14:37:53,660][20381] Updated weights for policy 1, policy_version 1295518 (0.0010) [2023-12-27 14:37:53,690][20347] Updated weights for policy 0, policy_version 1296449 (0.0008) [2023-12-27 14:37:53,716][20381] Updated weights for policy 1, policy_version 1295528 (0.0010) [2023-12-27 14:37:53,742][20347] Updated weights for policy 0, policy_version 1296459 (0.0009) [2023-12-27 14:37:53,768][20381] Updated weights for policy 1, policy_version 1295538 (0.0010) [2023-12-27 14:37:53,794][20347] Updated weights for policy 0, policy_version 1296469 (0.0007) [2023-12-27 14:37:53,859][20347] Updated weights for policy 0, policy_version 1296479 (0.0008) [2023-12-27 14:37:54,407][20381] Updated weights for policy 1, policy_version 1295548 (0.0009) [2023-12-27 14:37:54,452][20381] Updated weights for policy 1, policy_version 1295558 (0.0010) [2023-12-27 14:37:54,508][20381] Updated weights for policy 1, policy_version 1295568 (0.0011) [2023-12-27 14:37:54,531][20347] Updated weights for policy 0, policy_version 1296489 (0.0006) [2023-12-27 14:37:54,585][20347] Updated weights for policy 0, policy_version 1296499 (0.0005) [2023-12-27 14:37:54,639][20347] Updated weights for policy 0, policy_version 1296509 (0.0007) [2023-12-27 14:37:55,127][20381] Updated weights for policy 1, policy_version 1295578 (0.0010) [2023-12-27 14:37:55,175][20381] Updated weights for policy 1, policy_version 1295588 (0.0008) [2023-12-27 14:37:55,230][20381] Updated weights for policy 1, policy_version 1295598 (0.0007) [2023-12-27 14:37:55,292][20381] Updated weights for policy 1, policy_version 1295608 (0.0006) [2023-12-27 14:37:55,309][20347] Updated weights for policy 0, policy_version 1296519 (0.0008) [2023-12-27 14:37:55,360][20347] Updated weights for policy 0, policy_version 1296529 (0.0007) [2023-12-27 14:37:55,418][20347] Updated weights for policy 0, policy_version 1296539 (0.0008) [2023-12-27 14:37:55,969][20381] Updated weights for policy 1, policy_version 1295618 (0.0008) [2023-12-27 14:37:56,032][20381] Updated weights for policy 1, policy_version 1295628 (0.0005) [2023-12-27 14:37:56,084][20381] Updated weights for policy 1, policy_version 1295638 (0.0005) [2023-12-27 14:37:56,149][20347] Updated weights for policy 0, policy_version 1296549 (0.0008) [2023-12-27 14:37:56,207][20347] Updated weights for policy 0, policy_version 1296559 (0.0008) [2023-12-27 14:37:56,267][20347] Updated weights for policy 0, policy_version 1296569 (0.0009) [2023-12-27 14:37:56,678][20381] Updated weights for policy 1, policy_version 1295648 (0.0007) [2023-12-27 14:37:56,732][20381] Updated weights for policy 1, policy_version 1295658 (0.0005) [2023-12-27 14:37:56,783][20381] Updated weights for policy 1, policy_version 1295668 (0.0005) [2023-12-27 14:37:56,982][20347] Updated weights for policy 0, policy_version 1296579 (0.0008) [2023-12-27 14:37:57,026][20347] Updated weights for policy 0, policy_version 1296589 (0.0010) [2023-12-27 14:37:57,070][20347] Updated weights for policy 0, policy_version 1296599 (0.0010) [2023-12-27 14:37:57,343][20381] Updated weights for policy 1, policy_version 1295678 (0.0008) [2023-12-27 14:37:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 663715840. Throughput: 0: 10376.3, 1: 10394.2. Samples: 663705648. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:37:57,363][19170] Avg episode reward: [(0, '843.570'), (1, '970.710')] [2023-12-27 14:37:57,399][20381] Updated weights for policy 1, policy_version 1295688 (0.0008) [2023-12-27 14:37:57,450][20381] Updated weights for policy 1, policy_version 1295698 (0.0005) [2023-12-27 14:37:57,760][20347] Updated weights for policy 0, policy_version 1296609 (0.0009) [2023-12-27 14:37:57,824][20347] Updated weights for policy 0, policy_version 1296619 (0.0008) [2023-12-27 14:37:57,876][20347] Updated weights for policy 0, policy_version 1296629 (0.0008) [2023-12-27 14:37:57,930][20347] Updated weights for policy 0, policy_version 1296639 (0.0006) [2023-12-27 14:37:58,005][20381] Updated weights for policy 1, policy_version 1295708 (0.0007) [2023-12-27 14:37:58,053][20381] Updated weights for policy 1, policy_version 1295718 (0.0007) [2023-12-27 14:37:58,137][20381] Updated weights for policy 1, policy_version 1295728 (0.0010) [2023-12-27 14:37:58,649][20347] Updated weights for policy 0, policy_version 1296649 (0.0007) [2023-12-27 14:37:58,715][20347] Updated weights for policy 0, policy_version 1296659 (0.0007) [2023-12-27 14:37:58,789][20347] Updated weights for policy 0, policy_version 1296669 (0.0008) [2023-12-27 14:37:58,878][20381] Updated weights for policy 1, policy_version 1295738 (0.0010) [2023-12-27 14:37:58,941][20381] Updated weights for policy 1, policy_version 1295748 (0.0007) [2023-12-27 14:37:58,994][20381] Updated weights for policy 1, policy_version 1295758 (0.0008) [2023-12-27 14:37:59,049][20381] Updated weights for policy 1, policy_version 1295768 (0.0008) [2023-12-27 14:37:59,546][20347] Updated weights for policy 0, policy_version 1296679 (0.0008) [2023-12-27 14:37:59,611][20347] Updated weights for policy 0, policy_version 1296689 (0.0008) [2023-12-27 14:37:59,671][20347] Updated weights for policy 0, policy_version 1296699 (0.0008) [2023-12-27 14:37:59,742][20381] Updated weights for policy 1, policy_version 1295778 (0.0008) [2023-12-27 14:37:59,806][20381] Updated weights for policy 1, policy_version 1295788 (0.0008) [2023-12-27 14:37:59,870][20381] Updated weights for policy 1, policy_version 1295798 (0.0007) [2023-12-27 14:38:00,310][20347] Updated weights for policy 0, policy_version 1296709 (0.0006) [2023-12-27 14:38:00,368][20347] Updated weights for policy 0, policy_version 1296719 (0.0006) [2023-12-27 14:38:00,420][20347] Updated weights for policy 0, policy_version 1296729 (0.0005) [2023-12-27 14:38:00,631][20381] Updated weights for policy 1, policy_version 1295808 (0.0010) [2023-12-27 14:38:00,682][20381] Updated weights for policy 1, policy_version 1295818 (0.0010) [2023-12-27 14:38:00,733][20381] Updated weights for policy 1, policy_version 1295828 (0.0010) [2023-12-27 14:38:00,974][20347] Updated weights for policy 0, policy_version 1296739 (0.0007) [2023-12-27 14:38:01,022][20347] Updated weights for policy 0, policy_version 1296749 (0.0010) [2023-12-27 14:38:01,078][20347] Updated weights for policy 0, policy_version 1296759 (0.0010) [2023-12-27 14:38:01,425][20381] Updated weights for policy 1, policy_version 1295838 (0.0009) [2023-12-27 14:38:01,471][20381] Updated weights for policy 1, policy_version 1295848 (0.0007) [2023-12-27 14:38:01,524][20381] Updated weights for policy 1, policy_version 1295858 (0.0005) [2023-12-27 14:38:01,883][20347] Updated weights for policy 0, policy_version 1296769 (0.0010) [2023-12-27 14:38:01,931][20347] Updated weights for policy 0, policy_version 1296779 (0.0008) [2023-12-27 14:38:01,984][20347] Updated weights for policy 0, policy_version 1296789 (0.0008) [2023-12-27 14:38:02,029][20347] Updated weights for policy 0, policy_version 1296799 (0.0007) [2023-12-27 14:38:02,157][20381] Updated weights for policy 1, policy_version 1295868 (0.0007) [2023-12-27 14:38:02,216][20381] Updated weights for policy 1, policy_version 1295878 (0.0010) [2023-12-27 14:38:02,274][20381] Updated weights for policy 1, policy_version 1295888 (0.0010) [2023-12-27 14:38:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 663822336. Throughput: 0: 10293.9, 1: 10421.6. Samples: 663828208. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:02,363][19170] Avg episode reward: [(0, '843.610'), (1, '980.480')] [2023-12-27 14:38:02,800][20347] Updated weights for policy 0, policy_version 1296809 (0.0007) [2023-12-27 14:38:02,850][20347] Updated weights for policy 0, policy_version 1296819 (0.0007) [2023-12-27 14:38:02,865][20381] Updated weights for policy 1, policy_version 1295898 (0.0010) [2023-12-27 14:38:02,903][20347] Updated weights for policy 0, policy_version 1296829 (0.0008) [2023-12-27 14:38:02,916][20381] Updated weights for policy 1, policy_version 1295908 (0.0010) [2023-12-27 14:38:02,972][20381] Updated weights for policy 1, policy_version 1295918 (0.0010) [2023-12-27 14:38:03,025][20381] Updated weights for policy 1, policy_version 1295928 (0.0010) [2023-12-27 14:38:03,492][20347] Updated weights for policy 0, policy_version 1296839 (0.0005) [2023-12-27 14:38:03,545][20347] Updated weights for policy 0, policy_version 1296849 (0.0005) [2023-12-27 14:38:03,600][20347] Updated weights for policy 0, policy_version 1296859 (0.0005) [2023-12-27 14:38:03,740][20381] Updated weights for policy 1, policy_version 1295938 (0.0009) [2023-12-27 14:38:03,792][20381] Updated weights for policy 1, policy_version 1295948 (0.0006) [2023-12-27 14:38:03,847][20381] Updated weights for policy 1, policy_version 1295958 (0.0007) [2023-12-27 14:38:04,179][20347] Updated weights for policy 0, policy_version 1296869 (0.0006) [2023-12-27 14:38:04,243][20347] Updated weights for policy 0, policy_version 1296879 (0.0007) [2023-12-27 14:38:04,299][20347] Updated weights for policy 0, policy_version 1296889 (0.0007) [2023-12-27 14:38:04,580][20381] Updated weights for policy 1, policy_version 1295968 (0.0006) [2023-12-27 14:38:04,643][20381] Updated weights for policy 1, policy_version 1295978 (0.0005) [2023-12-27 14:38:04,702][20381] Updated weights for policy 1, policy_version 1295988 (0.0007) [2023-12-27 14:38:04,952][20347] Updated weights for policy 0, policy_version 1296899 (0.0009) [2023-12-27 14:38:05,001][20347] Updated weights for policy 0, policy_version 1296909 (0.0010) [2023-12-27 14:38:05,037][20291] KL-divergence is very high: 136.2080 [2023-12-27 14:38:05,051][20291] KL-divergence is very high: 139.8724 [2023-12-27 14:38:05,058][20291] KL-divergence is very high: 127.4603 [2023-12-27 14:38:05,064][20347] Updated weights for policy 0, policy_version 1296919 (0.0010) [2023-12-27 14:38:05,297][20381] Updated weights for policy 1, policy_version 1295998 (0.0006) [2023-12-27 14:38:05,346][20381] Updated weights for policy 1, policy_version 1296008 (0.0005) [2023-12-27 14:38:05,393][20381] Updated weights for policy 1, policy_version 1296018 (0.0006) [2023-12-27 14:38:05,731][20347] Updated weights for policy 0, policy_version 1296929 (0.0006) [2023-12-27 14:38:05,780][20347] Updated weights for policy 0, policy_version 1296939 (0.0010) [2023-12-27 14:38:05,836][20347] Updated weights for policy 0, policy_version 1296949 (0.0010) [2023-12-27 14:38:05,889][20347] Updated weights for policy 0, policy_version 1296959 (0.0010) [2023-12-27 14:38:06,082][20381] Updated weights for policy 1, policy_version 1296028 (0.0008) [2023-12-27 14:38:06,149][20381] Updated weights for policy 1, policy_version 1296038 (0.0008) [2023-12-27 14:38:06,207][20381] Updated weights for policy 1, policy_version 1296048 (0.0008) [2023-12-27 14:38:06,509][20347] Updated weights for policy 0, policy_version 1296969 (0.0011) [2023-12-27 14:38:06,573][20347] Updated weights for policy 0, policy_version 1296979 (0.0010) [2023-12-27 14:38:06,635][20347] Updated weights for policy 0, policy_version 1296989 (0.0010) [2023-12-27 14:38:06,842][20381] Updated weights for policy 1, policy_version 1296058 (0.0008) [2023-12-27 14:38:06,909][20381] Updated weights for policy 1, policy_version 1296068 (0.0005) [2023-12-27 14:38:06,969][20381] Updated weights for policy 1, policy_version 1296078 (0.0006) [2023-12-27 14:38:07,027][20381] Updated weights for policy 1, policy_version 1296088 (0.0007) [2023-12-27 14:38:07,274][20347] Updated weights for policy 0, policy_version 1296999 (0.0006) [2023-12-27 14:38:07,334][20347] Updated weights for policy 0, policy_version 1297009 (0.0006) [2023-12-27 14:38:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 663920640. Throughput: 0: 10329.4, 1: 10411.3. Samples: 663892224. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:07,363][19170] Avg episode reward: [(0, '873.860'), (1, '980.430')] [2023-12-27 14:38:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001296088_331825152.pth... [2023-12-27 14:38:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001294872_331513856.pth [2023-12-27 14:38:07,399][20347] Updated weights for policy 0, policy_version 1297019 (0.0006) [2023-12-27 14:38:07,427][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001297024_332103680.pth... [2023-12-27 14:38:07,431][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001295808_331792384.pth [2023-12-27 14:38:07,696][20381] Updated weights for policy 1, policy_version 1296098 (0.0009) [2023-12-27 14:38:07,754][20381] Updated weights for policy 1, policy_version 1296108 (0.0009) [2023-12-27 14:38:07,822][20381] Updated weights for policy 1, policy_version 1296118 (0.0008) [2023-12-27 14:38:07,903][20347] Updated weights for policy 0, policy_version 1297029 (0.0008) [2023-12-27 14:38:07,954][20347] Updated weights for policy 0, policy_version 1297039 (0.0010) [2023-12-27 14:38:08,006][20347] Updated weights for policy 0, policy_version 1297049 (0.0010) [2023-12-27 14:38:08,425][20381] Updated weights for policy 1, policy_version 1296128 (0.0006) [2023-12-27 14:38:08,485][20381] Updated weights for policy 1, policy_version 1296138 (0.0007) [2023-12-27 14:38:08,545][20381] Updated weights for policy 1, policy_version 1296148 (0.0008) [2023-12-27 14:38:08,671][20347] Updated weights for policy 0, policy_version 1297059 (0.0010) [2023-12-27 14:38:08,729][20347] Updated weights for policy 0, policy_version 1297069 (0.0006) [2023-12-27 14:38:08,791][20347] Updated weights for policy 0, policy_version 1297079 (0.0007) [2023-12-27 14:38:09,211][20381] Updated weights for policy 1, policy_version 1296158 (0.0010) [2023-12-27 14:38:09,274][20381] Updated weights for policy 1, policy_version 1296168 (0.0007) [2023-12-27 14:38:09,332][20381] Updated weights for policy 1, policy_version 1296178 (0.0008) [2023-12-27 14:38:09,461][20347] Updated weights for policy 0, policy_version 1297089 (0.0005) [2023-12-27 14:38:09,527][20347] Updated weights for policy 0, policy_version 1297099 (0.0008) [2023-12-27 14:38:09,594][20347] Updated weights for policy 0, policy_version 1297109 (0.0008) [2023-12-27 14:38:09,656][20347] Updated weights for policy 0, policy_version 1297119 (0.0007) [2023-12-27 14:38:10,055][20381] Updated weights for policy 1, policy_version 1296188 (0.0009) [2023-12-27 14:38:10,110][20381] Updated weights for policy 1, policy_version 1296198 (0.0008) [2023-12-27 14:38:10,168][20381] Updated weights for policy 1, policy_version 1296208 (0.0009) [2023-12-27 14:38:10,364][20347] Updated weights for policy 0, policy_version 1297129 (0.0007) [2023-12-27 14:38:10,423][20347] Updated weights for policy 0, policy_version 1297139 (0.0008) [2023-12-27 14:38:10,483][20347] Updated weights for policy 0, policy_version 1297149 (0.0008) [2023-12-27 14:38:10,905][20381] Updated weights for policy 1, policy_version 1296218 (0.0007) [2023-12-27 14:38:10,966][20381] Updated weights for policy 1, policy_version 1296228 (0.0008) [2023-12-27 14:38:11,021][20381] Updated weights for policy 1, policy_version 1296238 (0.0009) [2023-12-27 14:38:11,082][20381] Updated weights for policy 1, policy_version 1296248 (0.0008) [2023-12-27 14:38:11,227][20347] Updated weights for policy 0, policy_version 1297159 (0.0007) [2023-12-27 14:38:11,289][20347] Updated weights for policy 0, policy_version 1297169 (0.0008) [2023-12-27 14:38:11,349][20347] Updated weights for policy 0, policy_version 1297179 (0.0006) [2023-12-27 14:38:11,828][20381] Updated weights for policy 1, policy_version 1296258 (0.0006) [2023-12-27 14:38:11,886][20381] Updated weights for policy 1, policy_version 1296268 (0.0006) [2023-12-27 14:38:11,944][20381] Updated weights for policy 1, policy_version 1296278 (0.0007) [2023-12-27 14:38:12,103][20347] Updated weights for policy 0, policy_version 1297189 (0.0007) [2023-12-27 14:38:12,164][20347] Updated weights for policy 0, policy_version 1297199 (0.0006) [2023-12-27 14:38:12,214][20347] Updated weights for policy 0, policy_version 1297209 (0.0005) [2023-12-27 14:38:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 664027136. Throughput: 0: 10316.7, 1: 10387.0. Samples: 664015844. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:12,363][19170] Avg episode reward: [(0, '828.750'), (1, '970.650')] [2023-12-27 14:38:12,562][20381] Updated weights for policy 1, policy_version 1296288 (0.0006) [2023-12-27 14:38:12,611][20381] Updated weights for policy 1, policy_version 1296298 (0.0005) [2023-12-27 14:38:12,660][20381] Updated weights for policy 1, policy_version 1296308 (0.0007) [2023-12-27 14:38:12,909][20347] Updated weights for policy 0, policy_version 1297219 (0.0007) [2023-12-27 14:38:12,953][20347] Updated weights for policy 0, policy_version 1297229 (0.0007) [2023-12-27 14:38:12,999][20347] Updated weights for policy 0, policy_version 1297239 (0.0008) [2023-12-27 14:38:13,322][20381] Updated weights for policy 1, policy_version 1296318 (0.0007) [2023-12-27 14:38:13,375][20381] Updated weights for policy 1, policy_version 1296328 (0.0007) [2023-12-27 14:38:13,424][20381] Updated weights for policy 1, policy_version 1296338 (0.0007) [2023-12-27 14:38:13,647][20347] Updated weights for policy 0, policy_version 1297249 (0.0007) [2023-12-27 14:38:13,717][20347] Updated weights for policy 0, policy_version 1297259 (0.0005) [2023-12-27 14:38:13,781][20347] Updated weights for policy 0, policy_version 1297269 (0.0005) [2023-12-27 14:38:13,847][20347] Updated weights for policy 0, policy_version 1297279 (0.0005) [2023-12-27 14:38:14,044][20381] Updated weights for policy 1, policy_version 1296348 (0.0007) [2023-12-27 14:38:14,098][20381] Updated weights for policy 1, policy_version 1296358 (0.0006) [2023-12-27 14:38:14,142][20381] Updated weights for policy 1, policy_version 1296368 (0.0006) [2023-12-27 14:38:14,427][20347] Updated weights for policy 0, policy_version 1297289 (0.0006) [2023-12-27 14:38:14,484][20347] Updated weights for policy 0, policy_version 1297299 (0.0005) [2023-12-27 14:38:14,535][20347] Updated weights for policy 0, policy_version 1297309 (0.0006) [2023-12-27 14:38:14,860][20381] Updated weights for policy 1, policy_version 1296378 (0.0009) [2023-12-27 14:38:14,916][20381] Updated weights for policy 1, policy_version 1296388 (0.0011) [2023-12-27 14:38:14,977][20381] Updated weights for policy 1, policy_version 1296398 (0.0011) [2023-12-27 14:38:15,028][20381] Updated weights for policy 1, policy_version 1296408 (0.0011) [2023-12-27 14:38:15,142][20347] Updated weights for policy 0, policy_version 1297319 (0.0010) [2023-12-27 14:38:15,204][20347] Updated weights for policy 0, policy_version 1297329 (0.0011) [2023-12-27 14:38:15,264][20347] Updated weights for policy 0, policy_version 1297339 (0.0010) [2023-12-27 14:38:15,734][20381] Updated weights for policy 1, policy_version 1296418 (0.0010) [2023-12-27 14:38:15,789][20381] Updated weights for policy 1, policy_version 1296428 (0.0010) [2023-12-27 14:38:15,838][20381] Updated weights for policy 1, policy_version 1296438 (0.0010) [2023-12-27 14:38:15,893][20347] Updated weights for policy 0, policy_version 1297349 (0.0009) [2023-12-27 14:38:15,942][20347] Updated weights for policy 0, policy_version 1297359 (0.0007) [2023-12-27 14:38:16,000][20347] Updated weights for policy 0, policy_version 1297369 (0.0007) [2023-12-27 14:38:16,534][20381] Updated weights for policy 1, policy_version 1296448 (0.0011) [2023-12-27 14:38:16,592][20381] Updated weights for policy 1, policy_version 1296458 (0.0010) [2023-12-27 14:38:16,647][20381] Updated weights for policy 1, policy_version 1296468 (0.0010) [2023-12-27 14:38:16,663][20347] Updated weights for policy 0, policy_version 1297379 (0.0009) [2023-12-27 14:38:16,712][20347] Updated weights for policy 0, policy_version 1297389 (0.0010) [2023-12-27 14:38:16,763][20347] Updated weights for policy 0, policy_version 1297399 (0.0010) [2023-12-27 14:38:17,307][20381] Updated weights for policy 1, policy_version 1296478 (0.0007) [2023-12-27 14:38:17,353][20381] Updated weights for policy 1, policy_version 1296488 (0.0005) [2023-12-27 14:38:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 664125440. Throughput: 0: 10261.0, 1: 10340.6. Samples: 664140792. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:17,363][19170] Avg episode reward: [(0, '813.130'), (1, '990.220')] [2023-12-27 14:38:17,401][20381] Updated weights for policy 1, policy_version 1296498 (0.0005) [2023-12-27 14:38:17,447][20347] Updated weights for policy 0, policy_version 1297409 (0.0010) [2023-12-27 14:38:17,496][20347] Updated weights for policy 0, policy_version 1297419 (0.0010) [2023-12-27 14:38:17,544][20347] Updated weights for policy 0, policy_version 1297429 (0.0010) [2023-12-27 14:38:17,592][20347] Updated weights for policy 0, policy_version 1297439 (0.0010) [2023-12-27 14:38:17,999][20381] Updated weights for policy 1, policy_version 1296508 (0.0007) [2023-12-27 14:38:18,057][20381] Updated weights for policy 1, policy_version 1296518 (0.0011) [2023-12-27 14:38:18,110][20381] Updated weights for policy 1, policy_version 1296528 (0.0010) [2023-12-27 14:38:18,286][20347] Updated weights for policy 0, policy_version 1297449 (0.0009) [2023-12-27 14:38:18,357][20347] Updated weights for policy 0, policy_version 1297459 (0.0007) [2023-12-27 14:38:18,420][20347] Updated weights for policy 0, policy_version 1297469 (0.0007) [2023-12-27 14:38:18,780][20381] Updated weights for policy 1, policy_version 1296538 (0.0010) [2023-12-27 14:38:18,841][20381] Updated weights for policy 1, policy_version 1296548 (0.0008) [2023-12-27 14:38:18,898][20381] Updated weights for policy 1, policy_version 1296558 (0.0006) [2023-12-27 14:38:18,955][20381] Updated weights for policy 1, policy_version 1296568 (0.0007) [2023-12-27 14:38:19,108][20347] Updated weights for policy 0, policy_version 1297479 (0.0007) [2023-12-27 14:38:19,168][20347] Updated weights for policy 0, policy_version 1297489 (0.0007) [2023-12-27 14:38:19,217][20347] Updated weights for policy 0, policy_version 1297499 (0.0007) [2023-12-27 14:38:19,672][20381] Updated weights for policy 1, policy_version 1296578 (0.0006) [2023-12-27 14:38:19,738][20381] Updated weights for policy 1, policy_version 1296588 (0.0008) [2023-12-27 14:38:19,803][20381] Updated weights for policy 1, policy_version 1296598 (0.0007) [2023-12-27 14:38:19,905][20347] Updated weights for policy 0, policy_version 1297509 (0.0008) [2023-12-27 14:38:19,973][20347] Updated weights for policy 0, policy_version 1297519 (0.0007) [2023-12-27 14:38:20,038][20347] Updated weights for policy 0, policy_version 1297529 (0.0007) [2023-12-27 14:38:20,512][20381] Updated weights for policy 1, policy_version 1296608 (0.0008) [2023-12-27 14:38:20,568][20381] Updated weights for policy 1, policy_version 1296618 (0.0007) [2023-12-27 14:38:20,635][20381] Updated weights for policy 1, policy_version 1296628 (0.0008) [2023-12-27 14:38:20,738][20347] Updated weights for policy 0, policy_version 1297539 (0.0008) [2023-12-27 14:38:20,795][20347] Updated weights for policy 0, policy_version 1297549 (0.0007) [2023-12-27 14:38:20,845][20347] Updated weights for policy 0, policy_version 1297559 (0.0009) [2023-12-27 14:38:21,360][20381] Updated weights for policy 1, policy_version 1296638 (0.0008) [2023-12-27 14:38:21,429][20381] Updated weights for policy 1, policy_version 1296648 (0.0008) [2023-12-27 14:38:21,486][20381] Updated weights for policy 1, policy_version 1296658 (0.0007) [2023-12-27 14:38:21,598][20347] Updated weights for policy 0, policy_version 1297569 (0.0010) [2023-12-27 14:38:21,669][20347] Updated weights for policy 0, policy_version 1297579 (0.0009) [2023-12-27 14:38:21,732][20347] Updated weights for policy 0, policy_version 1297589 (0.0008) [2023-12-27 14:38:21,795][20347] Updated weights for policy 0, policy_version 1297599 (0.0007) [2023-12-27 14:38:22,146][20381] Updated weights for policy 1, policy_version 1296668 (0.0007) [2023-12-27 14:38:22,218][20381] Updated weights for policy 1, policy_version 1296678 (0.0006) [2023-12-27 14:38:22,282][20381] Updated weights for policy 1, policy_version 1296688 (0.0007) [2023-12-27 14:38:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 664231936. Throughput: 0: 10257.3, 1: 10313.8. Samples: 664201860. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:22,363][19170] Avg episode reward: [(0, '759.150'), (1, '1000.000')] [2023-12-27 14:38:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001297600_332251136.pth... [2023-12-27 14:38:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001296696_331980800.pth... [2023-12-27 14:38:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001296416_331948032.pth [2023-12-27 14:38:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001295480_331669504.pth [2023-12-27 14:38:22,501][20347] Updated weights for policy 0, policy_version 1297609 (0.0006) [2023-12-27 14:38:22,560][20347] Updated weights for policy 0, policy_version 1297619 (0.0007) [2023-12-27 14:38:22,622][20347] Updated weights for policy 0, policy_version 1297629 (0.0007) [2023-12-27 14:38:22,866][20381] Updated weights for policy 1, policy_version 1296698 (0.0006) [2023-12-27 14:38:22,930][20381] Updated weights for policy 1, policy_version 1296708 (0.0007) [2023-12-27 14:38:22,987][20381] Updated weights for policy 1, policy_version 1296718 (0.0007) [2023-12-27 14:38:23,045][20381] Updated weights for policy 1, policy_version 1296728 (0.0005) [2023-12-27 14:38:23,251][20347] Updated weights for policy 0, policy_version 1297639 (0.0009) [2023-12-27 14:38:23,306][20347] Updated weights for policy 0, policy_version 1297649 (0.0008) [2023-12-27 14:38:23,361][20347] Updated weights for policy 0, policy_version 1297659 (0.0007) [2023-12-27 14:38:23,665][20381] Updated weights for policy 1, policy_version 1296738 (0.0006) [2023-12-27 14:38:23,709][20381] Updated weights for policy 1, policy_version 1296748 (0.0010) [2023-12-27 14:38:23,759][20381] Updated weights for policy 1, policy_version 1296758 (0.0011) [2023-12-27 14:38:24,038][20347] Updated weights for policy 0, policy_version 1297669 (0.0009) [2023-12-27 14:38:24,081][20347] Updated weights for policy 0, policy_version 1297679 (0.0006) [2023-12-27 14:38:24,134][20347] Updated weights for policy 0, policy_version 1297689 (0.0005) [2023-12-27 14:38:24,446][20381] Updated weights for policy 1, policy_version 1296768 (0.0010) [2023-12-27 14:38:24,506][20381] Updated weights for policy 1, policy_version 1296778 (0.0005) [2023-12-27 14:38:24,560][20381] Updated weights for policy 1, policy_version 1296788 (0.0006) [2023-12-27 14:38:24,757][20347] Updated weights for policy 0, policy_version 1297699 (0.0006) [2023-12-27 14:38:24,815][20347] Updated weights for policy 0, policy_version 1297709 (0.0008) [2023-12-27 14:38:24,871][20347] Updated weights for policy 0, policy_version 1297719 (0.0007) [2023-12-27 14:38:25,117][20381] Updated weights for policy 1, policy_version 1296798 (0.0008) [2023-12-27 14:38:25,169][20381] Updated weights for policy 1, policy_version 1296808 (0.0005) [2023-12-27 14:38:25,228][20381] Updated weights for policy 1, policy_version 1296818 (0.0006) [2023-12-27 14:38:25,467][20347] Updated weights for policy 0, policy_version 1297729 (0.0006) [2023-12-27 14:38:25,514][20347] Updated weights for policy 0, policy_version 1297739 (0.0008) [2023-12-27 14:38:25,565][20347] Updated weights for policy 0, policy_version 1297749 (0.0005) [2023-12-27 14:38:25,620][20347] Updated weights for policy 0, policy_version 1297759 (0.0005) [2023-12-27 14:38:25,857][20381] Updated weights for policy 1, policy_version 1296828 (0.0011) [2023-12-27 14:38:25,915][20381] Updated weights for policy 1, policy_version 1296838 (0.0010) [2023-12-27 14:38:25,972][20381] Updated weights for policy 1, policy_version 1296848 (0.0007) [2023-12-27 14:38:26,294][20347] Updated weights for policy 0, policy_version 1297769 (0.0008) [2023-12-27 14:38:26,351][20347] Updated weights for policy 0, policy_version 1297779 (0.0008) [2023-12-27 14:38:26,410][20347] Updated weights for policy 0, policy_version 1297789 (0.0007) [2023-12-27 14:38:26,599][20381] Updated weights for policy 1, policy_version 1296858 (0.0010) [2023-12-27 14:38:26,653][20381] Updated weights for policy 1, policy_version 1296868 (0.0005) [2023-12-27 14:38:26,707][20381] Updated weights for policy 1, policy_version 1296878 (0.0007) [2023-12-27 14:38:26,764][20381] Updated weights for policy 1, policy_version 1296888 (0.0007) [2023-12-27 14:38:27,056][20347] Updated weights for policy 0, policy_version 1297799 (0.0007) [2023-12-27 14:38:27,106][20347] Updated weights for policy 0, policy_version 1297809 (0.0007) [2023-12-27 14:38:27,161][20347] Updated weights for policy 0, policy_version 1297819 (0.0007) [2023-12-27 14:38:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 664338432. Throughput: 0: 10291.0, 1: 10360.2. Samples: 664327132. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:27,363][19170] Avg episode reward: [(0, '800.400'), (1, '1000.000')] [2023-12-27 14:38:27,400][20381] Updated weights for policy 1, policy_version 1296898 (0.0009) [2023-12-27 14:38:27,451][20381] Updated weights for policy 1, policy_version 1296908 (0.0007) [2023-12-27 14:38:27,494][20381] Updated weights for policy 1, policy_version 1296918 (0.0006) [2023-12-27 14:38:27,856][20347] Updated weights for policy 0, policy_version 1297829 (0.0007) [2023-12-27 14:38:27,903][20347] Updated weights for policy 0, policy_version 1297839 (0.0008) [2023-12-27 14:38:27,949][20347] Updated weights for policy 0, policy_version 1297849 (0.0006) [2023-12-27 14:38:28,113][20381] Updated weights for policy 1, policy_version 1296928 (0.0008) [2023-12-27 14:38:28,171][20381] Updated weights for policy 1, policy_version 1296938 (0.0007) [2023-12-27 14:38:28,223][20381] Updated weights for policy 1, policy_version 1296948 (0.0008) [2023-12-27 14:38:28,626][20347] Updated weights for policy 0, policy_version 1297859 (0.0006) [2023-12-27 14:38:28,694][20347] Updated weights for policy 0, policy_version 1297869 (0.0009) [2023-12-27 14:38:28,753][20347] Updated weights for policy 0, policy_version 1297879 (0.0009) [2023-12-27 14:38:28,936][20381] Updated weights for policy 1, policy_version 1296958 (0.0007) [2023-12-27 14:38:28,987][20381] Updated weights for policy 1, policy_version 1296968 (0.0005) [2023-12-27 14:38:29,046][20381] Updated weights for policy 1, policy_version 1296978 (0.0008) [2023-12-27 14:38:29,391][20347] Updated weights for policy 0, policy_version 1297889 (0.0007) [2023-12-27 14:38:29,439][20347] Updated weights for policy 0, policy_version 1297899 (0.0010) [2023-12-27 14:38:29,486][20347] Updated weights for policy 0, policy_version 1297909 (0.0010) [2023-12-27 14:38:29,540][20347] Updated weights for policy 0, policy_version 1297919 (0.0010) [2023-12-27 14:38:29,727][20381] Updated weights for policy 1, policy_version 1296988 (0.0007) [2023-12-27 14:38:29,785][20381] Updated weights for policy 1, policy_version 1296998 (0.0008) [2023-12-27 14:38:29,843][20381] Updated weights for policy 1, policy_version 1297008 (0.0007) [2023-12-27 14:38:30,288][20347] Updated weights for policy 0, policy_version 1297929 (0.0008) [2023-12-27 14:38:30,334][20347] Updated weights for policy 0, policy_version 1297939 (0.0008) [2023-12-27 14:38:30,381][20347] Updated weights for policy 0, policy_version 1297949 (0.0008) [2023-12-27 14:38:30,554][20381] Updated weights for policy 1, policy_version 1297018 (0.0008) [2023-12-27 14:38:30,601][20381] Updated weights for policy 1, policy_version 1297028 (0.0010) [2023-12-27 14:38:30,645][20381] Updated weights for policy 1, policy_version 1297038 (0.0010) [2023-12-27 14:38:30,702][20381] Updated weights for policy 1, policy_version 1297048 (0.0006) [2023-12-27 14:38:30,980][20347] Updated weights for policy 0, policy_version 1297959 (0.0008) [2023-12-27 14:38:31,027][20347] Updated weights for policy 0, policy_version 1297969 (0.0009) [2023-12-27 14:38:31,097][20347] Updated weights for policy 0, policy_version 1297979 (0.0008) [2023-12-27 14:38:31,377][20381] Updated weights for policy 1, policy_version 1297058 (0.0008) [2023-12-27 14:38:31,432][20381] Updated weights for policy 1, policy_version 1297068 (0.0008) [2023-12-27 14:38:31,486][20381] Updated weights for policy 1, policy_version 1297078 (0.0008) [2023-12-27 14:38:31,896][20347] Updated weights for policy 0, policy_version 1297989 (0.0008) [2023-12-27 14:38:31,952][20347] Updated weights for policy 0, policy_version 1297999 (0.0007) [2023-12-27 14:38:32,008][20347] Updated weights for policy 0, policy_version 1298009 (0.0007) [2023-12-27 14:38:32,199][20381] Updated weights for policy 1, policy_version 1297088 (0.0008) [2023-12-27 14:38:32,249][20381] Updated weights for policy 1, policy_version 1297098 (0.0009) [2023-12-27 14:38:32,313][20381] Updated weights for policy 1, policy_version 1297108 (0.0008) [2023-12-27 14:38:32,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 664444928. Throughput: 0: 10255.8, 1: 10415.1. Samples: 664450964. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:32,363][19170] Avg episode reward: [(0, '859.550'), (1, '1000.000')] [2023-12-27 14:38:32,621][20347] Updated weights for policy 0, policy_version 1298019 (0.0008) [2023-12-27 14:38:32,680][20347] Updated weights for policy 0, policy_version 1298029 (0.0008) [2023-12-27 14:38:32,736][20347] Updated weights for policy 0, policy_version 1298039 (0.0008) [2023-12-27 14:38:32,980][20381] Updated weights for policy 1, policy_version 1297118 (0.0007) [2023-12-27 14:38:33,043][20381] Updated weights for policy 1, policy_version 1297128 (0.0005) [2023-12-27 14:38:33,109][20381] Updated weights for policy 1, policy_version 1297138 (0.0005) [2023-12-27 14:38:33,379][20347] Updated weights for policy 0, policy_version 1298049 (0.0007) [2023-12-27 14:38:33,433][20347] Updated weights for policy 0, policy_version 1298059 (0.0007) [2023-12-27 14:38:33,481][20347] Updated weights for policy 0, policy_version 1298069 (0.0008) [2023-12-27 14:38:33,535][20347] Updated weights for policy 0, policy_version 1298079 (0.0007) [2023-12-27 14:38:33,737][20381] Updated weights for policy 1, policy_version 1297148 (0.0006) [2023-12-27 14:38:33,788][20381] Updated weights for policy 1, policy_version 1297158 (0.0008) [2023-12-27 14:38:33,838][20381] Updated weights for policy 1, policy_version 1297168 (0.0009) [2023-12-27 14:38:34,171][20347] Updated weights for policy 0, policy_version 1298089 (0.0007) [2023-12-27 14:38:34,230][20347] Updated weights for policy 0, policy_version 1298099 (0.0008) [2023-12-27 14:38:34,290][20347] Updated weights for policy 0, policy_version 1298109 (0.0008) [2023-12-27 14:38:34,525][20381] Updated weights for policy 1, policy_version 1297178 (0.0008) [2023-12-27 14:38:34,589][20381] Updated weights for policy 1, policy_version 1297188 (0.0008) [2023-12-27 14:38:34,652][20381] Updated weights for policy 1, policy_version 1297198 (0.0005) [2023-12-27 14:38:34,707][20381] Updated weights for policy 1, policy_version 1297208 (0.0008) [2023-12-27 14:38:34,989][20347] Updated weights for policy 0, policy_version 1298119 (0.0006) [2023-12-27 14:38:35,046][20347] Updated weights for policy 0, policy_version 1298129 (0.0005) [2023-12-27 14:38:35,095][20347] Updated weights for policy 0, policy_version 1298139 (0.0006) [2023-12-27 14:38:35,323][20381] Updated weights for policy 1, policy_version 1297218 (0.0010) [2023-12-27 14:38:35,377][20381] Updated weights for policy 1, policy_version 1297228 (0.0010) [2023-12-27 14:38:35,435][20381] Updated weights for policy 1, policy_version 1297238 (0.0010) [2023-12-27 14:38:35,725][20347] Updated weights for policy 0, policy_version 1298149 (0.0007) [2023-12-27 14:38:35,784][20347] Updated weights for policy 0, policy_version 1298159 (0.0008) [2023-12-27 14:38:35,845][20347] Updated weights for policy 0, policy_version 1298169 (0.0007) [2023-12-27 14:38:36,090][20381] Updated weights for policy 1, policy_version 1297248 (0.0011) [2023-12-27 14:38:36,175][20381] Updated weights for policy 1, policy_version 1297258 (0.0011) [2023-12-27 14:38:36,238][20381] Updated weights for policy 1, policy_version 1297268 (0.0011) [2023-12-27 14:38:36,577][20347] Updated weights for policy 0, policy_version 1298179 (0.0008) [2023-12-27 14:38:36,641][20347] Updated weights for policy 0, policy_version 1298189 (0.0008) [2023-12-27 14:38:36,700][20347] Updated weights for policy 0, policy_version 1298199 (0.0007) [2023-12-27 14:38:36,923][20381] Updated weights for policy 1, policy_version 1297278 (0.0011) [2023-12-27 14:38:36,984][20381] Updated weights for policy 1, policy_version 1297288 (0.0011) [2023-12-27 14:38:37,036][20381] Updated weights for policy 1, policy_version 1297298 (0.0011) [2023-12-27 14:38:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.7, 300 sec: 20577.2). Total num frames: 664543232. Throughput: 0: 10320.0, 1: 10390.2. Samples: 664514776. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:37,363][19170] Avg episode reward: [(0, '854.070'), (1, '1000.000')] [2023-12-27 14:38:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001297304_332136448.pth... [2023-12-27 14:38:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001298208_332406784.pth... [2023-12-27 14:38:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001296088_331825152.pth [2023-12-27 14:38:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001297024_332103680.pth [2023-12-27 14:38:37,401][20347] Updated weights for policy 0, policy_version 1298209 (0.0007) [2023-12-27 14:38:37,452][20347] Updated weights for policy 0, policy_version 1298219 (0.0007) [2023-12-27 14:38:37,499][20347] Updated weights for policy 0, policy_version 1298229 (0.0007) [2023-12-27 14:38:37,546][20347] Updated weights for policy 0, policy_version 1298239 (0.0007) [2023-12-27 14:38:37,751][20381] Updated weights for policy 1, policy_version 1297308 (0.0011) [2023-12-27 14:38:37,804][20381] Updated weights for policy 1, policy_version 1297318 (0.0011) [2023-12-27 14:38:37,864][20381] Updated weights for policy 1, policy_version 1297328 (0.0011) [2023-12-27 14:38:38,267][20347] Updated weights for policy 0, policy_version 1298249 (0.0008) [2023-12-27 14:38:38,317][20347] Updated weights for policy 0, policy_version 1298259 (0.0007) [2023-12-27 14:38:38,383][20347] Updated weights for policy 0, policy_version 1298269 (0.0007) [2023-12-27 14:38:38,561][20381] Updated weights for policy 1, policy_version 1297338 (0.0009) [2023-12-27 14:38:38,623][20381] Updated weights for policy 1, policy_version 1297348 (0.0009) [2023-12-27 14:38:38,679][20381] Updated weights for policy 1, policy_version 1297358 (0.0010) [2023-12-27 14:38:38,736][20381] Updated weights for policy 1, policy_version 1297368 (0.0011) [2023-12-27 14:38:39,079][20347] Updated weights for policy 0, policy_version 1298279 (0.0007) [2023-12-27 14:38:39,129][20347] Updated weights for policy 0, policy_version 1298289 (0.0007) [2023-12-27 14:38:39,176][20347] Updated weights for policy 0, policy_version 1298299 (0.0007) [2023-12-27 14:38:39,481][20381] Updated weights for policy 1, policy_version 1297378 (0.0010) [2023-12-27 14:38:39,544][20381] Updated weights for policy 1, policy_version 1297388 (0.0010) [2023-12-27 14:38:39,597][20381] Updated weights for policy 1, policy_version 1297398 (0.0007) [2023-12-27 14:38:39,925][20347] Updated weights for policy 0, policy_version 1298309 (0.0007) [2023-12-27 14:38:39,990][20347] Updated weights for policy 0, policy_version 1298319 (0.0008) [2023-12-27 14:38:40,054][20347] Updated weights for policy 0, policy_version 1298329 (0.0008) [2023-12-27 14:38:40,320][20381] Updated weights for policy 1, policy_version 1297408 (0.0009) [2023-12-27 14:38:40,382][20381] Updated weights for policy 1, policy_version 1297418 (0.0011) [2023-12-27 14:38:40,442][20381] Updated weights for policy 1, policy_version 1297428 (0.0006) [2023-12-27 14:38:40,762][20347] Updated weights for policy 0, policy_version 1298339 (0.0007) [2023-12-27 14:38:40,816][20347] Updated weights for policy 0, policy_version 1298349 (0.0007) [2023-12-27 14:38:40,868][20347] Updated weights for policy 0, policy_version 1298359 (0.0008) [2023-12-27 14:38:41,065][20381] Updated weights for policy 1, policy_version 1297438 (0.0009) [2023-12-27 14:38:41,132][20381] Updated weights for policy 1, policy_version 1297448 (0.0010) [2023-12-27 14:38:41,197][20381] Updated weights for policy 1, policy_version 1297458 (0.0011) [2023-12-27 14:38:41,649][20347] Updated weights for policy 0, policy_version 1298369 (0.0007) [2023-12-27 14:38:41,704][20347] Updated weights for policy 0, policy_version 1298379 (0.0007) [2023-12-27 14:38:41,773][20347] Updated weights for policy 0, policy_version 1298389 (0.0007) [2023-12-27 14:38:41,836][20347] Updated weights for policy 0, policy_version 1298399 (0.0007) [2023-12-27 14:38:41,939][20381] Updated weights for policy 1, policy_version 1297468 (0.0008) [2023-12-27 14:38:42,003][20381] Updated weights for policy 1, policy_version 1297478 (0.0008) [2023-12-27 14:38:42,059][20381] Updated weights for policy 1, policy_version 1297488 (0.0010) [2023-12-27 14:38:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 664641536. Throughput: 0: 10273.3, 1: 10319.5. Samples: 664632328. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:42,363][19170] Avg episode reward: [(0, '912.180'), (1, '1000.000')] [2023-12-27 14:38:42,570][20347] Updated weights for policy 0, policy_version 1298409 (0.0008) [2023-12-27 14:38:42,634][20347] Updated weights for policy 0, policy_version 1298419 (0.0007) [2023-12-27 14:38:42,686][20347] Updated weights for policy 0, policy_version 1298429 (0.0009) [2023-12-27 14:38:42,705][20381] Updated weights for policy 1, policy_version 1297498 (0.0010) [2023-12-27 14:38:42,752][20381] Updated weights for policy 1, policy_version 1297508 (0.0005) [2023-12-27 14:38:42,805][20381] Updated weights for policy 1, policy_version 1297518 (0.0006) [2023-12-27 14:38:42,864][20381] Updated weights for policy 1, policy_version 1297528 (0.0011) [2023-12-27 14:38:43,378][20347] Updated weights for policy 0, policy_version 1298439 (0.0007) [2023-12-27 14:38:43,438][20347] Updated weights for policy 0, policy_version 1298449 (0.0005) [2023-12-27 14:38:43,496][20347] Updated weights for policy 0, policy_version 1298459 (0.0006) [2023-12-27 14:38:43,546][20381] Updated weights for policy 1, policy_version 1297538 (0.0007) [2023-12-27 14:38:43,599][20381] Updated weights for policy 1, policy_version 1297548 (0.0006) [2023-12-27 14:38:43,656][20381] Updated weights for policy 1, policy_version 1297558 (0.0007) [2023-12-27 14:38:44,122][20347] Updated weights for policy 0, policy_version 1298469 (0.0005) [2023-12-27 14:38:44,180][20347] Updated weights for policy 0, policy_version 1298479 (0.0007) [2023-12-27 14:38:44,239][20347] Updated weights for policy 0, policy_version 1298489 (0.0007) [2023-12-27 14:38:44,281][20381] Updated weights for policy 1, policy_version 1297568 (0.0009) [2023-12-27 14:38:44,327][20381] Updated weights for policy 1, policy_version 1297578 (0.0010) [2023-12-27 14:38:44,380][20381] Updated weights for policy 1, policy_version 1297588 (0.0011) [2023-12-27 14:38:44,836][20347] Updated weights for policy 0, policy_version 1298499 (0.0009) [2023-12-27 14:38:44,895][20347] Updated weights for policy 0, policy_version 1298509 (0.0007) [2023-12-27 14:38:44,952][20347] Updated weights for policy 0, policy_version 1298519 (0.0007) [2023-12-27 14:38:45,090][20381] Updated weights for policy 1, policy_version 1297598 (0.0010) [2023-12-27 14:38:45,150][20381] Updated weights for policy 1, policy_version 1297608 (0.0009) [2023-12-27 14:38:45,209][20381] Updated weights for policy 1, policy_version 1297618 (0.0008) [2023-12-27 14:38:45,639][20347] Updated weights for policy 0, policy_version 1298529 (0.0007) [2023-12-27 14:38:45,690][20347] Updated weights for policy 0, policy_version 1298539 (0.0005) [2023-12-27 14:38:45,746][20347] Updated weights for policy 0, policy_version 1298549 (0.0005) [2023-12-27 14:38:45,789][20381] Updated weights for policy 1, policy_version 1297628 (0.0006) [2023-12-27 14:38:45,800][20347] Updated weights for policy 0, policy_version 1298559 (0.0006) [2023-12-27 14:38:45,845][20381] Updated weights for policy 1, policy_version 1297638 (0.0007) [2023-12-27 14:38:45,896][20381] Updated weights for policy 1, policy_version 1297648 (0.0010) [2023-12-27 14:38:46,404][20347] Updated weights for policy 0, policy_version 1298569 (0.0005) [2023-12-27 14:38:46,463][20347] Updated weights for policy 0, policy_version 1298579 (0.0005) [2023-12-27 14:38:46,514][20381] Updated weights for policy 1, policy_version 1297658 (0.0011) [2023-12-27 14:38:46,515][20347] Updated weights for policy 0, policy_version 1298589 (0.0006) [2023-12-27 14:38:46,568][20381] Updated weights for policy 1, policy_version 1297668 (0.0010) [2023-12-27 14:38:46,629][20381] Updated weights for policy 1, policy_version 1297678 (0.0010) [2023-12-27 14:38:46,686][20381] Updated weights for policy 1, policy_version 1297688 (0.0009) [2023-12-27 14:38:47,162][20347] Updated weights for policy 0, policy_version 1298599 (0.0007) [2023-12-27 14:38:47,213][20347] Updated weights for policy 0, policy_version 1298609 (0.0007) [2023-12-27 14:38:47,237][20381] Updated weights for policy 1, policy_version 1297698 (0.0006) [2023-12-27 14:38:47,269][20347] Updated weights for policy 0, policy_version 1298619 (0.0008) [2023-12-27 14:38:47,299][20381] Updated weights for policy 1, policy_version 1297708 (0.0005) [2023-12-27 14:38:47,355][20381] Updated weights for policy 1, policy_version 1297718 (0.0005) [2023-12-27 14:38:47,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 664748032. Throughput: 0: 10335.6, 1: 10350.9. Samples: 664759104. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:47,364][19170] Avg episode reward: [(0, '931.930'), (1, '990.250')] [2023-12-27 14:38:47,916][20381] Updated weights for policy 1, policy_version 1297728 (0.0005) [2023-12-27 14:38:47,964][20347] Updated weights for policy 0, policy_version 1298629 (0.0007) [2023-12-27 14:38:47,969][20381] Updated weights for policy 1, policy_version 1297738 (0.0006) [2023-12-27 14:38:48,015][20347] Updated weights for policy 0, policy_version 1298639 (0.0007) [2023-12-27 14:38:48,018][20381] Updated weights for policy 1, policy_version 1297748 (0.0006) [2023-12-27 14:38:48,073][20347] Updated weights for policy 0, policy_version 1298649 (0.0007) [2023-12-27 14:38:48,679][20381] Updated weights for policy 1, policy_version 1297758 (0.0006) [2023-12-27 14:38:48,741][20381] Updated weights for policy 1, policy_version 1297768 (0.0006) [2023-12-27 14:38:48,769][20347] Updated weights for policy 0, policy_version 1298659 (0.0008) [2023-12-27 14:38:48,807][20381] Updated weights for policy 1, policy_version 1297778 (0.0006) [2023-12-27 14:38:48,831][20347] Updated weights for policy 0, policy_version 1298669 (0.0007) [2023-12-27 14:38:48,887][20347] Updated weights for policy 0, policy_version 1298679 (0.0007) [2023-12-27 14:38:49,473][20381] Updated weights for policy 1, policy_version 1297788 (0.0006) [2023-12-27 14:38:49,526][20347] Updated weights for policy 0, policy_version 1298689 (0.0008) [2023-12-27 14:38:49,531][20381] Updated weights for policy 1, policy_version 1297798 (0.0007) [2023-12-27 14:38:49,574][20347] Updated weights for policy 0, policy_version 1298699 (0.0008) [2023-12-27 14:38:49,584][20381] Updated weights for policy 1, policy_version 1297808 (0.0006) [2023-12-27 14:38:49,630][20347] Updated weights for policy 0, policy_version 1298709 (0.0006) [2023-12-27 14:38:49,687][20347] Updated weights for policy 0, policy_version 1298719 (0.0008) [2023-12-27 14:38:50,331][20381] Updated weights for policy 1, policy_version 1297818 (0.0007) [2023-12-27 14:38:50,365][20347] Updated weights for policy 0, policy_version 1298729 (0.0007) [2023-12-27 14:38:50,393][20381] Updated weights for policy 1, policy_version 1297828 (0.0006) [2023-12-27 14:38:50,413][20347] Updated weights for policy 0, policy_version 1298739 (0.0007) [2023-12-27 14:38:50,458][20381] Updated weights for policy 1, policy_version 1297838 (0.0006) [2023-12-27 14:38:50,462][20347] Updated weights for policy 0, policy_version 1298749 (0.0006) [2023-12-27 14:38:50,518][20381] Updated weights for policy 1, policy_version 1297848 (0.0008) [2023-12-27 14:38:51,111][20347] Updated weights for policy 0, policy_version 1298759 (0.0010) [2023-12-27 14:38:51,177][20347] Updated weights for policy 0, policy_version 1298769 (0.0011) [2023-12-27 14:38:51,219][20381] Updated weights for policy 1, policy_version 1297858 (0.0006) [2023-12-27 14:38:51,232][20347] Updated weights for policy 0, policy_version 1298779 (0.0010) [2023-12-27 14:38:51,278][20381] Updated weights for policy 1, policy_version 1297868 (0.0007) [2023-12-27 14:38:51,334][20381] Updated weights for policy 1, policy_version 1297878 (0.0007) [2023-12-27 14:38:51,982][20347] Updated weights for policy 0, policy_version 1298789 (0.0007) [2023-12-27 14:38:52,041][20347] Updated weights for policy 0, policy_version 1298799 (0.0005) [2023-12-27 14:38:52,082][20381] Updated weights for policy 1, policy_version 1297888 (0.0007) [2023-12-27 14:38:52,102][20347] Updated weights for policy 0, policy_version 1298809 (0.0006) [2023-12-27 14:38:52,133][20381] Updated weights for policy 1, policy_version 1297898 (0.0007) [2023-12-27 14:38:52,193][20381] Updated weights for policy 1, policy_version 1297908 (0.0007) [2023-12-27 14:38:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 664854528. Throughput: 0: 10297.0, 1: 10354.5. Samples: 664821544. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:52,363][19170] Avg episode reward: [(0, '863.920'), (1, '980.480')] [2023-12-27 14:38:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001297912_332292096.pth... [2023-12-27 14:38:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001298816_332562432.pth... [2023-12-27 14:38:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001296696_331980800.pth [2023-12-27 14:38:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001297600_332251136.pth [2023-12-27 14:38:52,795][20347] Updated weights for policy 0, policy_version 1298819 (0.0007) [2023-12-27 14:38:52,855][20347] Updated weights for policy 0, policy_version 1298829 (0.0007) [2023-12-27 14:38:52,899][20381] Updated weights for policy 1, policy_version 1297918 (0.0007) [2023-12-27 14:38:52,912][20347] Updated weights for policy 0, policy_version 1298839 (0.0007) [2023-12-27 14:38:52,963][20381] Updated weights for policy 1, policy_version 1297928 (0.0007) [2023-12-27 14:38:53,020][20381] Updated weights for policy 1, policy_version 1297938 (0.0010) [2023-12-27 14:38:53,609][20347] Updated weights for policy 0, policy_version 1298849 (0.0007) [2023-12-27 14:38:53,660][20347] Updated weights for policy 0, policy_version 1298859 (0.0007) [2023-12-27 14:38:53,703][20381] Updated weights for policy 1, policy_version 1297948 (0.0010) [2023-12-27 14:38:53,708][20347] Updated weights for policy 0, policy_version 1298869 (0.0007) [2023-12-27 14:38:53,755][20381] Updated weights for policy 1, policy_version 1297958 (0.0010) [2023-12-27 14:38:53,759][20347] Updated weights for policy 0, policy_version 1298879 (0.0007) [2023-12-27 14:38:53,804][20381] Updated weights for policy 1, policy_version 1297968 (0.0010) [2023-12-27 14:38:54,475][20347] Updated weights for policy 0, policy_version 1298890 (0.0009) [2023-12-27 14:38:54,491][20291] KL-divergence is very high: 238.9888 [2023-12-27 14:38:54,496][20381] Updated weights for policy 1, policy_version 1297978 (0.0010) [2023-12-27 14:38:54,530][20347] Updated weights for policy 0, policy_version 1298900 (0.0007) [2023-12-27 14:38:54,534][20291] KL-divergence is very high: 295.7475 [2023-12-27 14:38:54,555][20381] Updated weights for policy 1, policy_version 1297988 (0.0007) [2023-12-27 14:38:54,583][20291] KL-divergence is very high: 171.3172 [2023-12-27 14:38:54,589][20347] Updated weights for policy 0, policy_version 1298910 (0.0006) [2023-12-27 14:38:54,610][20381] Updated weights for policy 1, policy_version 1297998 (0.0007) [2023-12-27 14:38:54,662][20381] Updated weights for policy 1, policy_version 1298008 (0.0007) [2023-12-27 14:38:55,272][20347] Updated weights for policy 0, policy_version 1298920 (0.0009) [2023-12-27 14:38:55,318][20381] Updated weights for policy 1, policy_version 1298018 (0.0007) [2023-12-27 14:38:55,319][20347] Updated weights for policy 0, policy_version 1298930 (0.0010) [2023-12-27 14:38:55,371][20381] Updated weights for policy 1, policy_version 1298028 (0.0005) [2023-12-27 14:38:55,377][20347] Updated weights for policy 0, policy_version 1298940 (0.0010) [2023-12-27 14:38:55,424][20381] Updated weights for policy 1, policy_version 1298038 (0.0006) [2023-12-27 14:38:56,023][20381] Updated weights for policy 1, policy_version 1298048 (0.0005) [2023-12-27 14:38:56,079][20347] Updated weights for policy 0, policy_version 1298950 (0.0008) [2023-12-27 14:38:56,083][20381] Updated weights for policy 1, policy_version 1298058 (0.0007) [2023-12-27 14:38:56,139][20347] Updated weights for policy 0, policy_version 1298960 (0.0008) [2023-12-27 14:38:56,144][20381] Updated weights for policy 1, policy_version 1298068 (0.0008) [2023-12-27 14:38:56,195][20347] Updated weights for policy 0, policy_version 1298970 (0.0008) [2023-12-27 14:38:56,753][20381] Updated weights for policy 1, policy_version 1298078 (0.0005) [2023-12-27 14:38:56,806][20381] Updated weights for policy 1, policy_version 1298088 (0.0007) [2023-12-27 14:38:56,857][20381] Updated weights for policy 1, policy_version 1298098 (0.0006) [2023-12-27 14:38:56,891][20347] Updated weights for policy 0, policy_version 1298980 (0.0008) [2023-12-27 14:38:56,943][20347] Updated weights for policy 0, policy_version 1298990 (0.0009) [2023-12-27 14:38:56,993][20347] Updated weights for policy 0, policy_version 1299000 (0.0008) [2023-12-27 14:38:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 664952832. Throughput: 0: 10225.2, 1: 10372.8. Samples: 664942756. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:38:57,363][19170] Avg episode reward: [(0, '893.220'), (1, '980.440')] [2023-12-27 14:38:57,430][20381] Updated weights for policy 1, policy_version 1298108 (0.0006) [2023-12-27 14:38:57,476][20381] Updated weights for policy 1, policy_version 1298118 (0.0006) [2023-12-27 14:38:57,524][20381] Updated weights for policy 1, policy_version 1298128 (0.0005) [2023-12-27 14:38:57,684][20347] Updated weights for policy 0, policy_version 1299010 (0.0007) [2023-12-27 14:38:57,737][20347] Updated weights for policy 0, policy_version 1299020 (0.0007) [2023-12-27 14:38:57,791][20347] Updated weights for policy 0, policy_version 1299030 (0.0009) [2023-12-27 14:38:57,840][20347] Updated weights for policy 0, policy_version 1299040 (0.0008) [2023-12-27 14:38:58,099][20381] Updated weights for policy 1, policy_version 1298138 (0.0005) [2023-12-27 14:38:58,128][20292] KL-divergence is very high: 123.7698 [2023-12-27 14:38:58,149][20381] Updated weights for policy 1, policy_version 1298148 (0.0005) [2023-12-27 14:38:58,217][20381] Updated weights for policy 1, policy_version 1298158 (0.0007) [2023-12-27 14:38:58,277][20381] Updated weights for policy 1, policy_version 1298168 (0.0008) [2023-12-27 14:38:58,639][20347] Updated weights for policy 0, policy_version 1299050 (0.0008) [2023-12-27 14:38:58,703][20347] Updated weights for policy 0, policy_version 1299060 (0.0007) [2023-12-27 14:38:58,775][20347] Updated weights for policy 0, policy_version 1299070 (0.0007) [2023-12-27 14:38:59,133][20381] Updated weights for policy 1, policy_version 1298178 (0.0008) [2023-12-27 14:38:59,195][20381] Updated weights for policy 1, policy_version 1298188 (0.0007) [2023-12-27 14:38:59,265][20381] Updated weights for policy 1, policy_version 1298198 (0.0009) [2023-12-27 14:38:59,576][20347] Updated weights for policy 0, policy_version 1299080 (0.0008) [2023-12-27 14:38:59,641][20347] Updated weights for policy 0, policy_version 1299090 (0.0008) [2023-12-27 14:38:59,693][20347] Updated weights for policy 0, policy_version 1299100 (0.0009) [2023-12-27 14:38:59,953][20381] Updated weights for policy 1, policy_version 1298208 (0.0008) [2023-12-27 14:39:00,012][20381] Updated weights for policy 1, policy_version 1298218 (0.0008) [2023-12-27 14:39:00,063][20381] Updated weights for policy 1, policy_version 1298228 (0.0008) [2023-12-27 14:39:00,385][20347] Updated weights for policy 0, policy_version 1299110 (0.0007) [2023-12-27 14:39:00,433][20347] Updated weights for policy 0, policy_version 1299120 (0.0007) [2023-12-27 14:39:00,486][20347] Updated weights for policy 0, policy_version 1299130 (0.0005) [2023-12-27 14:39:00,769][20381] Updated weights for policy 1, policy_version 1298238 (0.0007) [2023-12-27 14:39:00,820][20381] Updated weights for policy 1, policy_version 1298248 (0.0010) [2023-12-27 14:39:00,868][20381] Updated weights for policy 1, policy_version 1298258 (0.0010) [2023-12-27 14:39:01,173][20347] Updated weights for policy 0, policy_version 1299140 (0.0006) [2023-12-27 14:39:01,226][20347] Updated weights for policy 0, policy_version 1299150 (0.0006) [2023-12-27 14:39:01,291][20347] Updated weights for policy 0, policy_version 1299160 (0.0008) [2023-12-27 14:39:01,548][20381] Updated weights for policy 1, policy_version 1298268 (0.0009) [2023-12-27 14:39:01,604][20381] Updated weights for policy 1, policy_version 1298278 (0.0007) [2023-12-27 14:39:01,667][20381] Updated weights for policy 1, policy_version 1298288 (0.0008) [2023-12-27 14:39:02,017][20347] Updated weights for policy 0, policy_version 1299170 (0.0007) [2023-12-27 14:39:02,079][20347] Updated weights for policy 0, policy_version 1299180 (0.0007) [2023-12-27 14:39:02,133][20347] Updated weights for policy 0, policy_version 1299190 (0.0006) [2023-12-27 14:39:02,188][20347] Updated weights for policy 0, policy_version 1299200 (0.0005) [2023-12-27 14:39:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 665051136. Throughput: 0: 10120.6, 1: 10363.2. Samples: 665062564. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:02,363][19170] Avg episode reward: [(0, '912.380'), (1, '990.210')] [2023-12-27 14:39:02,402][20381] Updated weights for policy 1, policy_version 1298298 (0.0008) [2023-12-27 14:39:02,460][20381] Updated weights for policy 1, policy_version 1298308 (0.0011) [2023-12-27 14:39:02,516][20381] Updated weights for policy 1, policy_version 1298318 (0.0011) [2023-12-27 14:39:02,565][20381] Updated weights for policy 1, policy_version 1298328 (0.0010) [2023-12-27 14:39:02,798][20347] Updated weights for policy 0, policy_version 1299210 (0.0005) [2023-12-27 14:39:02,847][20347] Updated weights for policy 0, policy_version 1299220 (0.0005) [2023-12-27 14:39:02,900][20347] Updated weights for policy 0, policy_version 1299230 (0.0006) [2023-12-27 14:39:03,212][20381] Updated weights for policy 1, policy_version 1298338 (0.0006) [2023-12-27 14:39:03,256][20381] Updated weights for policy 1, policy_version 1298348 (0.0005) [2023-12-27 14:39:03,300][20381] Updated weights for policy 1, policy_version 1298358 (0.0005) [2023-12-27 14:39:03,581][20347] Updated weights for policy 0, policy_version 1299240 (0.0008) [2023-12-27 14:39:03,629][20347] Updated weights for policy 0, policy_version 1299250 (0.0007) [2023-12-27 14:39:03,688][20347] Updated weights for policy 0, policy_version 1299260 (0.0007) [2023-12-27 14:39:03,913][20381] Updated weights for policy 1, policy_version 1298368 (0.0008) [2023-12-27 14:39:03,971][20381] Updated weights for policy 1, policy_version 1298378 (0.0011) [2023-12-27 14:39:04,028][20381] Updated weights for policy 1, policy_version 1298388 (0.0011) [2023-12-27 14:39:04,361][20347] Updated weights for policy 0, policy_version 1299270 (0.0007) [2023-12-27 14:39:04,428][20347] Updated weights for policy 0, policy_version 1299280 (0.0008) [2023-12-27 14:39:04,490][20347] Updated weights for policy 0, policy_version 1299290 (0.0008) [2023-12-27 14:39:04,706][20381] Updated weights for policy 1, policy_version 1298398 (0.0009) [2023-12-27 14:39:04,772][20381] Updated weights for policy 1, policy_version 1298408 (0.0008) [2023-12-27 14:39:04,836][20381] Updated weights for policy 1, policy_version 1298418 (0.0009) [2023-12-27 14:39:05,062][20347] Updated weights for policy 0, policy_version 1299300 (0.0009) [2023-12-27 14:39:05,123][20347] Updated weights for policy 0, policy_version 1299310 (0.0010) [2023-12-27 14:39:05,178][20347] Updated weights for policy 0, policy_version 1299320 (0.0010) [2023-12-27 14:39:05,544][20381] Updated weights for policy 1, policy_version 1298428 (0.0008) [2023-12-27 14:39:05,589][20381] Updated weights for policy 1, policy_version 1298438 (0.0007) [2023-12-27 14:39:05,635][20381] Updated weights for policy 1, policy_version 1298448 (0.0010) [2023-12-27 14:39:05,769][20347] Updated weights for policy 0, policy_version 1299330 (0.0009) [2023-12-27 14:39:05,817][20347] Updated weights for policy 0, policy_version 1299340 (0.0008) [2023-12-27 14:39:05,878][20347] Updated weights for policy 0, policy_version 1299350 (0.0006) [2023-12-27 14:39:05,932][20347] Updated weights for policy 0, policy_version 1299360 (0.0008) [2023-12-27 14:39:06,381][20381] Updated weights for policy 1, policy_version 1298458 (0.0008) [2023-12-27 14:39:06,434][20381] Updated weights for policy 1, policy_version 1298468 (0.0009) [2023-12-27 14:39:06,490][20381] Updated weights for policy 1, policy_version 1298478 (0.0010) [2023-12-27 14:39:06,552][20381] Updated weights for policy 1, policy_version 1298488 (0.0006) [2023-12-27 14:39:06,562][20347] Updated weights for policy 0, policy_version 1299370 (0.0008) [2023-12-27 14:39:06,627][20347] Updated weights for policy 0, policy_version 1299380 (0.0009) [2023-12-27 14:39:06,691][20347] Updated weights for policy 0, policy_version 1299390 (0.0009) [2023-12-27 14:39:07,237][20381] Updated weights for policy 1, policy_version 1298498 (0.0008) [2023-12-27 14:39:07,301][20381] Updated weights for policy 1, policy_version 1298508 (0.0005) [2023-12-27 14:39:07,348][20347] Updated weights for policy 0, policy_version 1299400 (0.0010) [2023-12-27 14:39:07,362][20381] Updated weights for policy 1, policy_version 1298518 (0.0006) [2023-12-27 14:39:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 665149440. Throughput: 0: 10170.0, 1: 10368.3. Samples: 665126080. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:07,363][19170] Avg episode reward: [(0, '892.990'), (1, '971.140')] [2023-12-27 14:39:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001298520_332447744.pth... [2023-12-27 14:39:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001297304_332136448.pth [2023-12-27 14:39:07,406][20347] Updated weights for policy 0, policy_version 1299410 (0.0010) [2023-12-27 14:39:07,461][20347] Updated weights for policy 0, policy_version 1299420 (0.0010) [2023-12-27 14:39:07,483][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001299424_332718080.pth... [2023-12-27 14:39:07,487][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001298208_332406784.pth [2023-12-27 14:39:08,037][20381] Updated weights for policy 1, policy_version 1298528 (0.0007) [2023-12-27 14:39:08,088][20381] Updated weights for policy 1, policy_version 1298538 (0.0007) [2023-12-27 14:39:08,102][20347] Updated weights for policy 0, policy_version 1299430 (0.0007) [2023-12-27 14:39:08,137][20381] Updated weights for policy 1, policy_version 1298548 (0.0007) [2023-12-27 14:39:08,156][20347] Updated weights for policy 0, policy_version 1299440 (0.0005) [2023-12-27 14:39:08,210][20347] Updated weights for policy 0, policy_version 1299450 (0.0007) [2023-12-27 14:39:08,844][20347] Updated weights for policy 0, policy_version 1299460 (0.0007) [2023-12-27 14:39:08,888][20381] Updated weights for policy 1, policy_version 1298558 (0.0006) [2023-12-27 14:39:08,900][20347] Updated weights for policy 0, policy_version 1299470 (0.0010) [2023-12-27 14:39:08,933][20381] Updated weights for policy 1, policy_version 1298568 (0.0006) [2023-12-27 14:39:08,952][20347] Updated weights for policy 0, policy_version 1299480 (0.0010) [2023-12-27 14:39:08,987][20381] Updated weights for policy 1, policy_version 1298578 (0.0005) [2023-12-27 14:39:09,684][20347] Updated weights for policy 0, policy_version 1299490 (0.0010) [2023-12-27 14:39:09,743][20347] Updated weights for policy 0, policy_version 1299500 (0.0010) [2023-12-27 14:39:09,770][20381] Updated weights for policy 1, policy_version 1298588 (0.0008) [2023-12-27 14:39:09,796][20347] Updated weights for policy 0, policy_version 1299510 (0.0010) [2023-12-27 14:39:09,824][20381] Updated weights for policy 1, policy_version 1298598 (0.0006) [2023-12-27 14:39:09,857][20347] Updated weights for policy 0, policy_version 1299520 (0.0010) [2023-12-27 14:39:09,890][20381] Updated weights for policy 1, policy_version 1298608 (0.0007) [2023-12-27 14:39:10,524][20347] Updated weights for policy 0, policy_version 1299530 (0.0007) [2023-12-27 14:39:10,582][20347] Updated weights for policy 0, policy_version 1299540 (0.0010) [2023-12-27 14:39:10,635][20381] Updated weights for policy 1, policy_version 1298618 (0.0008) [2023-12-27 14:39:10,641][20347] Updated weights for policy 0, policy_version 1299550 (0.0010) [2023-12-27 14:39:10,687][20381] Updated weights for policy 1, policy_version 1298628 (0.0006) [2023-12-27 14:39:10,749][20381] Updated weights for policy 1, policy_version 1298638 (0.0008) [2023-12-27 14:39:10,808][20381] Updated weights for policy 1, policy_version 1298648 (0.0008) [2023-12-27 14:39:11,322][20347] Updated weights for policy 0, policy_version 1299560 (0.0009) [2023-12-27 14:39:11,388][20347] Updated weights for policy 0, policy_version 1299570 (0.0007) [2023-12-27 14:39:11,451][20347] Updated weights for policy 0, policy_version 1299580 (0.0007) [2023-12-27 14:39:11,550][20381] Updated weights for policy 1, policy_version 1298658 (0.0007) [2023-12-27 14:39:11,616][20381] Updated weights for policy 1, policy_version 1298668 (0.0007) [2023-12-27 14:39:11,682][20381] Updated weights for policy 1, policy_version 1298678 (0.0006) [2023-12-27 14:39:12,120][20347] Updated weights for policy 0, policy_version 1299590 (0.0006) [2023-12-27 14:39:12,176][20347] Updated weights for policy 0, policy_version 1299600 (0.0008) [2023-12-27 14:39:12,235][20347] Updated weights for policy 0, policy_version 1299610 (0.0008) [2023-12-27 14:39:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 665255936. Throughput: 0: 10206.0, 1: 10257.1. Samples: 665247968. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:12,363][19170] Avg episode reward: [(0, '893.030'), (1, '980.930')] [2023-12-27 14:39:12,390][20381] Updated weights for policy 1, policy_version 1298688 (0.0009) [2023-12-27 14:39:12,450][20381] Updated weights for policy 1, policy_version 1298698 (0.0008) [2023-12-27 14:39:12,508][20381] Updated weights for policy 1, policy_version 1298708 (0.0008) [2023-12-27 14:39:12,927][20347] Updated weights for policy 0, policy_version 1299620 (0.0007) [2023-12-27 14:39:12,972][20347] Updated weights for policy 0, policy_version 1299630 (0.0007) [2023-12-27 14:39:13,021][20347] Updated weights for policy 0, policy_version 1299640 (0.0008) [2023-12-27 14:39:13,190][20381] Updated weights for policy 1, policy_version 1298718 (0.0008) [2023-12-27 14:39:13,239][20381] Updated weights for policy 1, policy_version 1298728 (0.0006) [2023-12-27 14:39:13,287][20381] Updated weights for policy 1, policy_version 1298738 (0.0007) [2023-12-27 14:39:13,703][20347] Updated weights for policy 0, policy_version 1299650 (0.0007) [2023-12-27 14:39:13,764][20347] Updated weights for policy 0, policy_version 1299660 (0.0005) [2023-12-27 14:39:13,815][20347] Updated weights for policy 0, policy_version 1299670 (0.0006) [2023-12-27 14:39:13,865][20347] Updated weights for policy 0, policy_version 1299680 (0.0007) [2023-12-27 14:39:14,013][20381] Updated weights for policy 1, policy_version 1298748 (0.0007) [2023-12-27 14:39:14,067][20381] Updated weights for policy 1, policy_version 1298758 (0.0007) [2023-12-27 14:39:14,123][20381] Updated weights for policy 1, policy_version 1298768 (0.0007) [2023-12-27 14:39:14,447][20347] Updated weights for policy 0, policy_version 1299690 (0.0007) [2023-12-27 14:39:14,499][20347] Updated weights for policy 0, policy_version 1299700 (0.0008) [2023-12-27 14:39:14,554][20347] Updated weights for policy 0, policy_version 1299710 (0.0008) [2023-12-27 14:39:14,789][20381] Updated weights for policy 1, policy_version 1298778 (0.0007) [2023-12-27 14:39:14,846][20381] Updated weights for policy 1, policy_version 1298788 (0.0008) [2023-12-27 14:39:14,911][20381] Updated weights for policy 1, policy_version 1298798 (0.0006) [2023-12-27 14:39:14,975][20381] Updated weights for policy 1, policy_version 1298808 (0.0008) [2023-12-27 14:39:15,224][20347] Updated weights for policy 0, policy_version 1299720 (0.0008) [2023-12-27 14:39:15,287][20347] Updated weights for policy 0, policy_version 1299730 (0.0007) [2023-12-27 14:39:15,340][20347] Updated weights for policy 0, policy_version 1299740 (0.0007) [2023-12-27 14:39:15,671][20381] Updated weights for policy 1, policy_version 1298818 (0.0008) [2023-12-27 14:39:15,726][20381] Updated weights for policy 1, policy_version 1298828 (0.0008) [2023-12-27 14:39:15,786][20381] Updated weights for policy 1, policy_version 1298838 (0.0007) [2023-12-27 14:39:15,992][20347] Updated weights for policy 0, policy_version 1299750 (0.0006) [2023-12-27 14:39:16,052][20347] Updated weights for policy 0, policy_version 1299760 (0.0005) [2023-12-27 14:39:16,106][20347] Updated weights for policy 0, policy_version 1299770 (0.0007) [2023-12-27 14:39:16,482][20381] Updated weights for policy 1, policy_version 1298848 (0.0007) [2023-12-27 14:39:16,535][20381] Updated weights for policy 1, policy_version 1298858 (0.0007) [2023-12-27 14:39:16,582][20381] Updated weights for policy 1, policy_version 1298868 (0.0008) [2023-12-27 14:39:16,729][20347] Updated weights for policy 0, policy_version 1299780 (0.0010) [2023-12-27 14:39:16,790][20347] Updated weights for policy 0, policy_version 1299790 (0.0010) [2023-12-27 14:39:16,843][20347] Updated weights for policy 0, policy_version 1299800 (0.0010) [2023-12-27 14:39:17,279][20381] Updated weights for policy 1, policy_version 1298878 (0.0006) [2023-12-27 14:39:17,344][20381] Updated weights for policy 1, policy_version 1298888 (0.0006) [2023-12-27 14:39:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 665354240. Throughput: 0: 10221.4, 1: 10197.8. Samples: 665369824. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:17,363][19170] Avg episode reward: [(0, '922.170'), (1, '990.240')] [2023-12-27 14:39:17,400][20381] Updated weights for policy 1, policy_version 1298898 (0.0006) [2023-12-27 14:39:17,523][20347] Updated weights for policy 0, policy_version 1299810 (0.0010) [2023-12-27 14:39:17,581][20347] Updated weights for policy 0, policy_version 1299820 (0.0005) [2023-12-27 14:39:17,644][20347] Updated weights for policy 0, policy_version 1299830 (0.0005) [2023-12-27 14:39:17,704][20347] Updated weights for policy 0, policy_version 1299840 (0.0010) [2023-12-27 14:39:17,970][20381] Updated weights for policy 1, policy_version 1298908 (0.0009) [2023-12-27 14:39:18,024][20381] Updated weights for policy 1, policy_version 1298918 (0.0005) [2023-12-27 14:39:18,081][20381] Updated weights for policy 1, policy_version 1298928 (0.0006) [2023-12-27 14:39:18,328][20347] Updated weights for policy 0, policy_version 1299850 (0.0010) [2023-12-27 14:39:18,393][20347] Updated weights for policy 0, policy_version 1299860 (0.0006) [2023-12-27 14:39:18,456][20347] Updated weights for policy 0, policy_version 1299870 (0.0007) [2023-12-27 14:39:18,735][20381] Updated weights for policy 1, policy_version 1298938 (0.0011) [2023-12-27 14:39:18,789][20381] Updated weights for policy 1, policy_version 1298948 (0.0011) [2023-12-27 14:39:18,848][20381] Updated weights for policy 1, policy_version 1298958 (0.0010) [2023-12-27 14:39:18,908][20381] Updated weights for policy 1, policy_version 1298968 (0.0011) [2023-12-27 14:39:19,198][20347] Updated weights for policy 0, policy_version 1299880 (0.0008) [2023-12-27 14:39:19,262][20347] Updated weights for policy 0, policy_version 1299890 (0.0007) [2023-12-27 14:39:19,318][20347] Updated weights for policy 0, policy_version 1299900 (0.0008) [2023-12-27 14:39:19,615][20381] Updated weights for policy 1, policy_version 1298978 (0.0009) [2023-12-27 14:39:19,670][20381] Updated weights for policy 1, policy_version 1298988 (0.0008) [2023-12-27 14:39:19,721][20381] Updated weights for policy 1, policy_version 1298998 (0.0007) [2023-12-27 14:39:20,079][20347] Updated weights for policy 0, policy_version 1299910 (0.0008) [2023-12-27 14:39:20,141][20347] Updated weights for policy 0, policy_version 1299920 (0.0007) [2023-12-27 14:39:20,197][20347] Updated weights for policy 0, policy_version 1299930 (0.0007) [2023-12-27 14:39:20,469][20381] Updated weights for policy 1, policy_version 1299008 (0.0011) [2023-12-27 14:39:20,523][20381] Updated weights for policy 1, policy_version 1299018 (0.0010) [2023-12-27 14:39:20,581][20381] Updated weights for policy 1, policy_version 1299028 (0.0010) [2023-12-27 14:39:20,909][20347] Updated weights for policy 0, policy_version 1299940 (0.0007) [2023-12-27 14:39:20,968][20347] Updated weights for policy 0, policy_version 1299950 (0.0007) [2023-12-27 14:39:21,032][20347] Updated weights for policy 0, policy_version 1299960 (0.0007) [2023-12-27 14:39:21,289][20381] Updated weights for policy 1, policy_version 1299038 (0.0011) [2023-12-27 14:39:21,347][20381] Updated weights for policy 1, policy_version 1299048 (0.0010) [2023-12-27 14:39:21,412][20381] Updated weights for policy 1, policy_version 1299058 (0.0010) [2023-12-27 14:39:21,791][20347] Updated weights for policy 0, policy_version 1299970 (0.0009) [2023-12-27 14:39:21,848][20347] Updated weights for policy 0, policy_version 1299980 (0.0007) [2023-12-27 14:39:21,901][20347] Updated weights for policy 0, policy_version 1299990 (0.0008) [2023-12-27 14:39:21,962][20347] Updated weights for policy 0, policy_version 1300000 (0.0007) [2023-12-27 14:39:22,114][20381] Updated weights for policy 1, policy_version 1299068 (0.0009) [2023-12-27 14:39:22,169][20381] Updated weights for policy 1, policy_version 1299078 (0.0008) [2023-12-27 14:39:22,227][20381] Updated weights for policy 1, policy_version 1299088 (0.0008) [2023-12-27 14:39:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 665460736. Throughput: 0: 10173.6, 1: 10183.3. Samples: 665430840. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:22,363][19170] Avg episode reward: [(0, '941.410'), (1, '990.240')] [2023-12-27 14:39:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001299096_332595200.pth... [2023-12-27 14:39:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001300000_332865536.pth... [2023-12-27 14:39:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001298816_332562432.pth [2023-12-27 14:39:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001297912_332292096.pth [2023-12-27 14:39:22,592][20347] Updated weights for policy 0, policy_version 1300010 (0.0006) [2023-12-27 14:39:22,657][20347] Updated weights for policy 0, policy_version 1300020 (0.0007) [2023-12-27 14:39:22,717][20347] Updated weights for policy 0, policy_version 1300030 (0.0006) [2023-12-27 14:39:22,949][20381] Updated weights for policy 1, policy_version 1299098 (0.0008) [2023-12-27 14:39:23,012][20381] Updated weights for policy 1, policy_version 1299108 (0.0006) [2023-12-27 14:39:23,068][20381] Updated weights for policy 1, policy_version 1299118 (0.0009) [2023-12-27 14:39:23,123][20381] Updated weights for policy 1, policy_version 1299128 (0.0007) [2023-12-27 14:39:23,358][20347] Updated weights for policy 0, policy_version 1300040 (0.0007) [2023-12-27 14:39:23,411][20347] Updated weights for policy 0, policy_version 1300050 (0.0008) [2023-12-27 14:39:23,476][20347] Updated weights for policy 0, policy_version 1300060 (0.0006) [2023-12-27 14:39:23,816][20381] Updated weights for policy 1, policy_version 1299138 (0.0007) [2023-12-27 14:39:23,872][20381] Updated weights for policy 1, policy_version 1299148 (0.0007) [2023-12-27 14:39:23,920][20381] Updated weights for policy 1, policy_version 1299158 (0.0007) [2023-12-27 14:39:24,090][20347] Updated weights for policy 0, policy_version 1300070 (0.0006) [2023-12-27 14:39:24,141][20347] Updated weights for policy 0, policy_version 1300080 (0.0007) [2023-12-27 14:39:24,190][20347] Updated weights for policy 0, policy_version 1300090 (0.0007) [2023-12-27 14:39:24,648][20381] Updated weights for policy 1, policy_version 1299168 (0.0008) [2023-12-27 14:39:24,703][20381] Updated weights for policy 1, policy_version 1299178 (0.0007) [2023-12-27 14:39:24,757][20381] Updated weights for policy 1, policy_version 1299188 (0.0008) [2023-12-27 14:39:24,829][20347] Updated weights for policy 0, policy_version 1300100 (0.0007) [2023-12-27 14:39:24,891][20347] Updated weights for policy 0, policy_version 1300110 (0.0007) [2023-12-27 14:39:24,949][20347] Updated weights for policy 0, policy_version 1300120 (0.0008) [2023-12-27 14:39:25,416][20381] Updated weights for policy 1, policy_version 1299198 (0.0009) [2023-12-27 14:39:25,478][20381] Updated weights for policy 1, policy_version 1299208 (0.0008) [2023-12-27 14:39:25,537][20381] Updated weights for policy 1, policy_version 1299218 (0.0006) [2023-12-27 14:39:25,626][20347] Updated weights for policy 0, policy_version 1300130 (0.0009) [2023-12-27 14:39:25,682][20347] Updated weights for policy 0, policy_version 1300140 (0.0008) [2023-12-27 14:39:25,733][20347] Updated weights for policy 0, policy_version 1300150 (0.0008) [2023-12-27 14:39:25,785][20347] Updated weights for policy 0, policy_version 1300160 (0.0008) [2023-12-27 14:39:26,149][20381] Updated weights for policy 1, policy_version 1299228 (0.0005) [2023-12-27 14:39:26,212][20381] Updated weights for policy 1, policy_version 1299238 (0.0006) [2023-12-27 14:39:26,263][20381] Updated weights for policy 1, policy_version 1299248 (0.0005) [2023-12-27 14:39:26,519][20347] Updated weights for policy 0, policy_version 1300170 (0.0008) [2023-12-27 14:39:26,580][20347] Updated weights for policy 0, policy_version 1300180 (0.0009) [2023-12-27 14:39:26,637][20347] Updated weights for policy 0, policy_version 1300190 (0.0010) [2023-12-27 14:39:26,851][20381] Updated weights for policy 1, policy_version 1299258 (0.0006) [2023-12-27 14:39:26,907][20381] Updated weights for policy 1, policy_version 1299268 (0.0008) [2023-12-27 14:39:26,952][20381] Updated weights for policy 1, policy_version 1299278 (0.0008) [2023-12-27 14:39:27,000][20381] Updated weights for policy 1, policy_version 1299288 (0.0007) [2023-12-27 14:39:27,213][20347] Updated weights for policy 0, policy_version 1300200 (0.0010) [2023-12-27 14:39:27,259][20347] Updated weights for policy 0, policy_version 1300210 (0.0005) [2023-12-27 14:39:27,310][20347] Updated weights for policy 0, policy_version 1300220 (0.0006) [2023-12-27 14:39:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 665567232. Throughput: 0: 10236.3, 1: 10216.8. Samples: 665552716. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:27,363][19170] Avg episode reward: [(0, '912.350'), (1, '980.460')] [2023-12-27 14:39:27,706][20381] Updated weights for policy 1, policy_version 1299298 (0.0009) [2023-12-27 14:39:27,770][20381] Updated weights for policy 1, policy_version 1299308 (0.0008) [2023-12-27 14:39:27,824][20381] Updated weights for policy 1, policy_version 1299318 (0.0007) [2023-12-27 14:39:27,904][20347] Updated weights for policy 0, policy_version 1300230 (0.0006) [2023-12-27 14:39:27,965][20347] Updated weights for policy 0, policy_version 1300240 (0.0009) [2023-12-27 14:39:28,021][20347] Updated weights for policy 0, policy_version 1300250 (0.0010) [2023-12-27 14:39:28,491][20381] Updated weights for policy 1, policy_version 1299328 (0.0009) [2023-12-27 14:39:28,551][20381] Updated weights for policy 1, policy_version 1299338 (0.0009) [2023-12-27 14:39:28,606][20381] Updated weights for policy 1, policy_version 1299348 (0.0009) [2023-12-27 14:39:28,617][20347] Updated weights for policy 0, policy_version 1300260 (0.0009) [2023-12-27 14:39:28,672][20347] Updated weights for policy 0, policy_version 1300270 (0.0009) [2023-12-27 14:39:28,729][20347] Updated weights for policy 0, policy_version 1300280 (0.0010) [2023-12-27 14:39:29,279][20381] Updated weights for policy 1, policy_version 1299358 (0.0007) [2023-12-27 14:39:29,341][20381] Updated weights for policy 1, policy_version 1299368 (0.0008) [2023-12-27 14:39:29,345][20347] Updated weights for policy 0, policy_version 1300290 (0.0011) [2023-12-27 14:39:29,405][20381] Updated weights for policy 1, policy_version 1299378 (0.0008) [2023-12-27 14:39:29,407][20347] Updated weights for policy 0, policy_version 1300300 (0.0010) [2023-12-27 14:39:29,464][20347] Updated weights for policy 0, policy_version 1300310 (0.0010) [2023-12-27 14:39:29,523][20347] Updated weights for policy 0, policy_version 1300320 (0.0010) [2023-12-27 14:39:30,143][20347] Updated weights for policy 0, policy_version 1300330 (0.0008) [2023-12-27 14:39:30,160][20381] Updated weights for policy 1, policy_version 1299388 (0.0006) [2023-12-27 14:39:30,196][20347] Updated weights for policy 0, policy_version 1300340 (0.0006) [2023-12-27 14:39:30,210][20381] Updated weights for policy 1, policy_version 1299398 (0.0005) [2023-12-27 14:39:30,253][20347] Updated weights for policy 0, policy_version 1300350 (0.0007) [2023-12-27 14:39:30,259][20381] Updated weights for policy 1, policy_version 1299408 (0.0006) [2023-12-27 14:39:30,874][20347] Updated weights for policy 0, policy_version 1300360 (0.0005) [2023-12-27 14:39:30,927][20347] Updated weights for policy 0, policy_version 1300370 (0.0007) [2023-12-27 14:39:30,975][20347] Updated weights for policy 0, policy_version 1300380 (0.0007) [2023-12-27 14:39:31,005][20381] Updated weights for policy 1, policy_version 1299418 (0.0007) [2023-12-27 14:39:31,089][20381] Updated weights for policy 1, policy_version 1299428 (0.0008) [2023-12-27 14:39:31,150][20381] Updated weights for policy 1, policy_version 1299438 (0.0009) [2023-12-27 14:39:31,211][20381] Updated weights for policy 1, policy_version 1299448 (0.0006) [2023-12-27 14:39:31,708][20347] Updated weights for policy 0, policy_version 1300390 (0.0007) [2023-12-27 14:39:31,778][20347] Updated weights for policy 0, policy_version 1300400 (0.0008) [2023-12-27 14:39:31,841][20347] Updated weights for policy 0, policy_version 1300410 (0.0008) [2023-12-27 14:39:31,884][20381] Updated weights for policy 1, policy_version 1299458 (0.0007) [2023-12-27 14:39:31,936][20381] Updated weights for policy 1, policy_version 1299468 (0.0006) [2023-12-27 14:39:31,992][20381] Updated weights for policy 1, policy_version 1299478 (0.0007) [2023-12-27 14:39:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 665665536. Throughput: 0: 10289.1, 1: 10108.8. Samples: 665677004. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:32,363][19170] Avg episode reward: [(0, '902.500'), (1, '980.460')] [2023-12-27 14:39:32,536][20347] Updated weights for policy 0, policy_version 1300420 (0.0008) [2023-12-27 14:39:32,595][20347] Updated weights for policy 0, policy_version 1300430 (0.0007) [2023-12-27 14:39:32,653][20347] Updated weights for policy 0, policy_version 1300440 (0.0010) [2023-12-27 14:39:32,658][20381] Updated weights for policy 1, policy_version 1299488 (0.0008) [2023-12-27 14:39:32,710][20381] Updated weights for policy 1, policy_version 1299498 (0.0006) [2023-12-27 14:39:32,757][20381] Updated weights for policy 1, policy_version 1299508 (0.0007) [2023-12-27 14:39:33,315][20347] Updated weights for policy 0, policy_version 1300450 (0.0010) [2023-12-27 14:39:33,372][20347] Updated weights for policy 0, policy_version 1300460 (0.0010) [2023-12-27 14:39:33,430][20347] Updated weights for policy 0, policy_version 1300470 (0.0010) [2023-12-27 14:39:33,434][20381] Updated weights for policy 1, policy_version 1299518 (0.0006) [2023-12-27 14:39:33,477][20381] Updated weights for policy 1, policy_version 1299528 (0.0005) [2023-12-27 14:39:33,491][20347] Updated weights for policy 0, policy_version 1300480 (0.0010) [2023-12-27 14:39:33,524][20381] Updated weights for policy 1, policy_version 1299538 (0.0007) [2023-12-27 14:39:34,147][20347] Updated weights for policy 0, policy_version 1300490 (0.0010) [2023-12-27 14:39:34,178][20381] Updated weights for policy 1, policy_version 1299548 (0.0010) [2023-12-27 14:39:34,212][20347] Updated weights for policy 0, policy_version 1300500 (0.0010) [2023-12-27 14:39:34,240][20381] Updated weights for policy 1, policy_version 1299558 (0.0006) [2023-12-27 14:39:34,272][20347] Updated weights for policy 0, policy_version 1300510 (0.0008) [2023-12-27 14:39:34,299][20381] Updated weights for policy 1, policy_version 1299568 (0.0006) [2023-12-27 14:39:34,926][20347] Updated weights for policy 0, policy_version 1300520 (0.0007) [2023-12-27 14:39:34,933][20381] Updated weights for policy 1, policy_version 1299578 (0.0007) [2023-12-27 14:39:34,981][20347] Updated weights for policy 0, policy_version 1300530 (0.0007) [2023-12-27 14:39:34,986][20381] Updated weights for policy 1, policy_version 1299588 (0.0005) [2023-12-27 14:39:35,035][20347] Updated weights for policy 0, policy_version 1300540 (0.0007) [2023-12-27 14:39:35,041][20381] Updated weights for policy 1, policy_version 1299598 (0.0007) [2023-12-27 14:39:35,096][20381] Updated weights for policy 1, policy_version 1299608 (0.0007) [2023-12-27 14:39:35,704][20381] Updated weights for policy 1, policy_version 1299618 (0.0007) [2023-12-27 14:39:35,733][20347] Updated weights for policy 0, policy_version 1300550 (0.0006) [2023-12-27 14:39:35,757][20381] Updated weights for policy 1, policy_version 1299628 (0.0007) [2023-12-27 14:39:35,783][20347] Updated weights for policy 0, policy_version 1300560 (0.0005) [2023-12-27 14:39:35,809][20381] Updated weights for policy 1, policy_version 1299638 (0.0007) [2023-12-27 14:39:35,833][20347] Updated weights for policy 0, policy_version 1300570 (0.0007) [2023-12-27 14:39:36,498][20347] Updated weights for policy 0, policy_version 1300580 (0.0007) [2023-12-27 14:39:36,550][20381] Updated weights for policy 1, policy_version 1299648 (0.0006) [2023-12-27 14:39:36,552][20347] Updated weights for policy 0, policy_version 1300590 (0.0007) [2023-12-27 14:39:36,604][20381] Updated weights for policy 1, policy_version 1299658 (0.0006) [2023-12-27 14:39:36,609][20347] Updated weights for policy 0, policy_version 1300600 (0.0006) [2023-12-27 14:39:36,654][20381] Updated weights for policy 1, policy_version 1299668 (0.0006) [2023-12-27 14:39:37,278][20347] Updated weights for policy 0, policy_version 1300610 (0.0006) [2023-12-27 14:39:37,282][20381] Updated weights for policy 1, policy_version 1299678 (0.0007) [2023-12-27 14:39:37,335][20347] Updated weights for policy 0, policy_version 1300620 (0.0008) [2023-12-27 14:39:37,338][20381] Updated weights for policy 1, policy_version 1299688 (0.0006) [2023-12-27 14:39:37,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 665763840. Throughput: 0: 10285.4, 1: 10113.4. Samples: 665739492. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:37,363][19170] Avg episode reward: [(0, '902.460'), (1, '990.250')] [2023-12-27 14:39:37,393][20347] Updated weights for policy 0, policy_version 1300630 (0.0007) [2023-12-27 14:39:37,397][20381] Updated weights for policy 1, policy_version 1299698 (0.0007) [2023-12-27 14:39:37,433][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001299704_332750848.pth... [2023-12-27 14:39:37,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001298520_332447744.pth [2023-12-27 14:39:37,443][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001300640_333029376.pth... [2023-12-27 14:39:37,444][20347] Updated weights for policy 0, policy_version 1300640 (0.0010) [2023-12-27 14:39:37,446][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001299424_332718080.pth [2023-12-27 14:39:38,012][20381] Updated weights for policy 1, policy_version 1299708 (0.0008) [2023-12-27 14:39:38,063][20381] Updated weights for policy 1, policy_version 1299718 (0.0007) [2023-12-27 14:39:38,115][20381] Updated weights for policy 1, policy_version 1299728 (0.0006) [2023-12-27 14:39:38,136][20347] Updated weights for policy 0, policy_version 1300650 (0.0009) [2023-12-27 14:39:38,190][20347] Updated weights for policy 0, policy_version 1300660 (0.0005) [2023-12-27 14:39:38,247][20347] Updated weights for policy 0, policy_version 1300670 (0.0009) [2023-12-27 14:39:38,810][20381] Updated weights for policy 1, policy_version 1299738 (0.0007) [2023-12-27 14:39:38,867][20381] Updated weights for policy 1, policy_version 1299748 (0.0007) [2023-12-27 14:39:38,874][20347] Updated weights for policy 0, policy_version 1300680 (0.0010) [2023-12-27 14:39:38,924][20381] Updated weights for policy 1, policy_version 1299758 (0.0006) [2023-12-27 14:39:38,930][20347] Updated weights for policy 0, policy_version 1300690 (0.0010) [2023-12-27 14:39:38,980][20347] Updated weights for policy 0, policy_version 1300700 (0.0011) [2023-12-27 14:39:38,988][20381] Updated weights for policy 1, policy_version 1299768 (0.0006) [2023-12-27 14:39:39,716][20381] Updated weights for policy 1, policy_version 1299778 (0.0008) [2023-12-27 14:39:39,741][20347] Updated weights for policy 0, policy_version 1300710 (0.0008) [2023-12-27 14:39:39,774][20381] Updated weights for policy 1, policy_version 1299788 (0.0006) [2023-12-27 14:39:39,803][20347] Updated weights for policy 0, policy_version 1300720 (0.0006) [2023-12-27 14:39:39,833][20381] Updated weights for policy 1, policy_version 1299798 (0.0008) [2023-12-27 14:39:39,863][20347] Updated weights for policy 0, policy_version 1300730 (0.0007) [2023-12-27 14:39:40,580][20381] Updated weights for policy 1, policy_version 1299808 (0.0007) [2023-12-27 14:39:40,638][20381] Updated weights for policy 1, policy_version 1299818 (0.0007) [2023-12-27 14:39:40,647][20347] Updated weights for policy 0, policy_version 1300740 (0.0008) [2023-12-27 14:39:40,685][20381] Updated weights for policy 1, policy_version 1299828 (0.0007) [2023-12-27 14:39:40,702][20347] Updated weights for policy 0, policy_version 1300750 (0.0007) [2023-12-27 14:39:40,762][20347] Updated weights for policy 0, policy_version 1300760 (0.0007) [2023-12-27 14:39:41,420][20381] Updated weights for policy 1, policy_version 1299838 (0.0006) [2023-12-27 14:39:41,475][20381] Updated weights for policy 1, policy_version 1299848 (0.0007) [2023-12-27 14:39:41,540][20381] Updated weights for policy 1, policy_version 1299858 (0.0008) [2023-12-27 14:39:41,559][20347] Updated weights for policy 0, policy_version 1300770 (0.0008) [2023-12-27 14:39:41,623][20347] Updated weights for policy 0, policy_version 1300780 (0.0009) [2023-12-27 14:39:41,700][20347] Updated weights for policy 0, policy_version 1300791 (0.0009) [2023-12-27 14:39:42,222][20381] Updated weights for policy 1, policy_version 1299868 (0.0009) [2023-12-27 14:39:42,284][20381] Updated weights for policy 1, policy_version 1299878 (0.0007) [2023-12-27 14:39:42,347][20381] Updated weights for policy 1, policy_version 1299888 (0.0007) [2023-12-27 14:39:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 665862144. Throughput: 0: 10281.3, 1: 10125.7. Samples: 665861072. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:42,363][19170] Avg episode reward: [(0, '921.870'), (1, '990.250')] [2023-12-27 14:39:42,387][20347] Updated weights for policy 0, policy_version 1300801 (0.0009) [2023-12-27 14:39:42,451][20347] Updated weights for policy 0, policy_version 1300811 (0.0008) [2023-12-27 14:39:42,513][20347] Updated weights for policy 0, policy_version 1300821 (0.0008) [2023-12-27 14:39:42,573][20347] Updated weights for policy 0, policy_version 1300831 (0.0008) [2023-12-27 14:39:42,939][20381] Updated weights for policy 1, policy_version 1299898 (0.0007) [2023-12-27 14:39:43,001][20381] Updated weights for policy 1, policy_version 1299908 (0.0008) [2023-12-27 14:39:43,057][20381] Updated weights for policy 1, policy_version 1299918 (0.0008) [2023-12-27 14:39:43,116][20381] Updated weights for policy 1, policy_version 1299928 (0.0008) [2023-12-27 14:39:43,246][20347] Updated weights for policy 0, policy_version 1300841 (0.0006) [2023-12-27 14:39:43,310][20347] Updated weights for policy 0, policy_version 1300851 (0.0007) [2023-12-27 14:39:43,364][20347] Updated weights for policy 0, policy_version 1300861 (0.0007) [2023-12-27 14:39:43,799][20381] Updated weights for policy 1, policy_version 1299938 (0.0007) [2023-12-27 14:39:43,847][20381] Updated weights for policy 1, policy_version 1299948 (0.0007) [2023-12-27 14:39:43,899][20381] Updated weights for policy 1, policy_version 1299958 (0.0007) [2023-12-27 14:39:43,987][20347] Updated weights for policy 0, policy_version 1300871 (0.0007) [2023-12-27 14:39:44,033][20347] Updated weights for policy 0, policy_version 1300881 (0.0008) [2023-12-27 14:39:44,081][20347] Updated weights for policy 0, policy_version 1300891 (0.0007) [2023-12-27 14:39:44,622][20381] Updated weights for policy 1, policy_version 1299968 (0.0007) [2023-12-27 14:39:44,669][20381] Updated weights for policy 1, policy_version 1299978 (0.0007) [2023-12-27 14:39:44,716][20381] Updated weights for policy 1, policy_version 1299988 (0.0007) [2023-12-27 14:39:44,725][20347] Updated weights for policy 0, policy_version 1300901 (0.0009) [2023-12-27 14:39:44,783][20347] Updated weights for policy 0, policy_version 1300911 (0.0010) [2023-12-27 14:39:44,840][20347] Updated weights for policy 0, policy_version 1300921 (0.0008) [2023-12-27 14:39:45,458][20381] Updated weights for policy 1, policy_version 1299998 (0.0008) [2023-12-27 14:39:45,508][20381] Updated weights for policy 1, policy_version 1300008 (0.0008) [2023-12-27 14:39:45,514][20347] Updated weights for policy 0, policy_version 1300931 (0.0009) [2023-12-27 14:39:45,562][20347] Updated weights for policy 0, policy_version 1300941 (0.0005) [2023-12-27 14:39:45,567][20381] Updated weights for policy 1, policy_version 1300018 (0.0008) [2023-12-27 14:39:45,626][20347] Updated weights for policy 0, policy_version 1300951 (0.0005) [2023-12-27 14:39:46,200][20347] Updated weights for policy 0, policy_version 1300961 (0.0005) [2023-12-27 14:39:46,259][20347] Updated weights for policy 0, policy_version 1300971 (0.0008) [2023-12-27 14:39:46,283][20381] Updated weights for policy 1, policy_version 1300028 (0.0009) [2023-12-27 14:39:46,312][20347] Updated weights for policy 0, policy_version 1300981 (0.0006) [2023-12-27 14:39:46,337][20381] Updated weights for policy 1, policy_version 1300038 (0.0005) [2023-12-27 14:39:46,368][20347] Updated weights for policy 0, policy_version 1300991 (0.0006) [2023-12-27 14:39:46,386][20381] Updated weights for policy 1, policy_version 1300048 (0.0007) [2023-12-27 14:39:46,955][20291] KL-divergence is very high: 113.3009 [2023-12-27 14:39:47,012][20347] Updated weights for policy 0, policy_version 1301001 (0.0008) [2023-12-27 14:39:47,068][20347] Updated weights for policy 0, policy_version 1301011 (0.0007) [2023-12-27 14:39:47,104][20381] Updated weights for policy 1, policy_version 1300058 (0.0007) [2023-12-27 14:39:47,127][20347] Updated weights for policy 0, policy_version 1301021 (0.0008) [2023-12-27 14:39:47,161][20381] Updated weights for policy 1, policy_version 1300068 (0.0007) [2023-12-27 14:39:47,218][20381] Updated weights for policy 1, policy_version 1300078 (0.0007) [2023-12-27 14:39:47,275][20381] Updated weights for policy 1, policy_version 1300088 (0.0008) [2023-12-27 14:39:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 665976832. Throughput: 0: 10379.7, 1: 10072.8. Samples: 665982924. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 14:39:47,363][19170] Avg episode reward: [(0, '941.520'), (1, '1000.000')] [2023-12-27 14:39:47,776][20347] Updated weights for policy 0, policy_version 1301031 (0.0009) [2023-12-27 14:39:47,830][20347] Updated weights for policy 0, policy_version 1301041 (0.0010) [2023-12-27 14:39:47,881][20347] Updated weights for policy 0, policy_version 1301051 (0.0010) [2023-12-27 14:39:47,969][20381] Updated weights for policy 1, policy_version 1300098 (0.0007) [2023-12-27 14:39:48,018][20381] Updated weights for policy 1, policy_version 1300108 (0.0007) [2023-12-27 14:39:48,066][20381] Updated weights for policy 1, policy_version 1300118 (0.0007) [2023-12-27 14:39:48,562][20347] Updated weights for policy 0, policy_version 1301061 (0.0008) [2023-12-27 14:39:48,614][20347] Updated weights for policy 0, policy_version 1301071 (0.0005) [2023-12-27 14:39:48,668][20347] Updated weights for policy 0, policy_version 1301081 (0.0006) [2023-12-27 14:39:48,823][20381] Updated weights for policy 1, policy_version 1300128 (0.0007) [2023-12-27 14:39:48,879][20381] Updated weights for policy 1, policy_version 1300138 (0.0008) [2023-12-27 14:39:48,942][20381] Updated weights for policy 1, policy_version 1300148 (0.0007) [2023-12-27 14:39:49,335][20347] Updated weights for policy 0, policy_version 1301091 (0.0009) [2023-12-27 14:39:49,399][20347] Updated weights for policy 0, policy_version 1301101 (0.0009) [2023-12-27 14:39:49,454][20347] Updated weights for policy 0, policy_version 1301111 (0.0010) [2023-12-27 14:39:49,651][20381] Updated weights for policy 1, policy_version 1300158 (0.0006) [2023-12-27 14:39:49,725][20381] Updated weights for policy 1, policy_version 1300168 (0.0006) [2023-12-27 14:39:49,794][20381] Updated weights for policy 1, policy_version 1300178 (0.0006) [2023-12-27 14:39:50,179][20347] Updated weights for policy 0, policy_version 1301121 (0.0010) [2023-12-27 14:39:50,243][20347] Updated weights for policy 0, policy_version 1301131 (0.0009) [2023-12-27 14:39:50,304][20347] Updated weights for policy 0, policy_version 1301141 (0.0008) [2023-12-27 14:39:50,361][20347] Updated weights for policy 0, policy_version 1301151 (0.0008) [2023-12-27 14:39:50,439][20381] Updated weights for policy 1, policy_version 1300188 (0.0008) [2023-12-27 14:39:50,499][20381] Updated weights for policy 1, policy_version 1300198 (0.0008) [2023-12-27 14:39:50,564][20381] Updated weights for policy 1, policy_version 1300208 (0.0008) [2023-12-27 14:39:51,074][20347] Updated weights for policy 0, policy_version 1301161 (0.0010) [2023-12-27 14:39:51,133][20347] Updated weights for policy 0, policy_version 1301171 (0.0010) [2023-12-27 14:39:51,190][20347] Updated weights for policy 0, policy_version 1301181 (0.0011) [2023-12-27 14:39:51,282][20381] Updated weights for policy 1, policy_version 1300218 (0.0008) [2023-12-27 14:39:51,343][20381] Updated weights for policy 1, policy_version 1300228 (0.0008) [2023-12-27 14:39:51,405][20381] Updated weights for policy 1, policy_version 1300238 (0.0009) [2023-12-27 14:39:51,464][20381] Updated weights for policy 1, policy_version 1300248 (0.0007) [2023-12-27 14:39:51,880][20347] Updated weights for policy 0, policy_version 1301191 (0.0010) [2023-12-27 14:39:51,926][20347] Updated weights for policy 0, policy_version 1301201 (0.0008) [2023-12-27 14:39:51,973][20347] Updated weights for policy 0, policy_version 1301211 (0.0008) [2023-12-27 14:39:52,192][20381] Updated weights for policy 1, policy_version 1300258 (0.0007) [2023-12-27 14:39:52,249][20381] Updated weights for policy 1, policy_version 1300268 (0.0008) [2023-12-27 14:39:52,302][20381] Updated weights for policy 1, policy_version 1300278 (0.0008) [2023-12-27 14:39:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 666075136. Throughput: 0: 10340.4, 1: 10040.3. Samples: 666043212. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:39:52,363][19170] Avg episode reward: [(0, '892.670'), (1, '1000.000')] [2023-12-27 14:39:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001300280_332898304.pth... [2023-12-27 14:39:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001301216_333176832.pth... [2023-12-27 14:39:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001299096_332595200.pth [2023-12-27 14:39:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001300000_332865536.pth [2023-12-27 14:39:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001301216_333176832.pth [2023-12-27 14:39:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001300280_332898304.pth [2023-12-27 14:39:52,692][20347] Updated weights for policy 0, policy_version 1301221 (0.0006) [2023-12-27 14:39:52,751][20347] Updated weights for policy 0, policy_version 1301231 (0.0008) [2023-12-27 14:39:52,804][20347] Updated weights for policy 0, policy_version 1301241 (0.0007) [2023-12-27 14:39:53,062][20381] Updated weights for policy 1, policy_version 1300288 (0.0006) [2023-12-27 14:39:53,124][20381] Updated weights for policy 1, policy_version 1300298 (0.0006) [2023-12-27 14:39:53,170][20381] Updated weights for policy 1, policy_version 1300308 (0.0007) [2023-12-27 14:39:53,493][20347] Updated weights for policy 0, policy_version 1301251 (0.0007) [2023-12-27 14:39:53,554][20347] Updated weights for policy 0, policy_version 1301261 (0.0007) [2023-12-27 14:39:53,615][20347] Updated weights for policy 0, policy_version 1301271 (0.0007) [2023-12-27 14:39:53,846][20381] Updated weights for policy 1, policy_version 1300318 (0.0009) [2023-12-27 14:39:53,901][20381] Updated weights for policy 1, policy_version 1300328 (0.0007) [2023-12-27 14:39:53,963][20381] Updated weights for policy 1, policy_version 1300338 (0.0005) [2023-12-27 14:39:54,267][20347] Updated weights for policy 0, policy_version 1301281 (0.0007) [2023-12-27 14:39:54,323][20347] Updated weights for policy 0, policy_version 1301291 (0.0007) [2023-12-27 14:39:54,374][20347] Updated weights for policy 0, policy_version 1301301 (0.0007) [2023-12-27 14:39:54,426][20347] Updated weights for policy 0, policy_version 1301311 (0.0007) [2023-12-27 14:39:54,597][20381] Updated weights for policy 1, policy_version 1300348 (0.0007) [2023-12-27 14:39:54,659][20381] Updated weights for policy 1, policy_version 1300358 (0.0007) [2023-12-27 14:39:54,713][20381] Updated weights for policy 1, policy_version 1300368 (0.0005) [2023-12-27 14:39:55,053][20347] Updated weights for policy 0, policy_version 1301321 (0.0006) [2023-12-27 14:39:55,111][20347] Updated weights for policy 0, policy_version 1301331 (0.0007) [2023-12-27 14:39:55,165][20347] Updated weights for policy 0, policy_version 1301341 (0.0007) [2023-12-27 14:39:55,285][20381] Updated weights for policy 1, policy_version 1300378 (0.0006) [2023-12-27 14:39:55,343][20381] Updated weights for policy 1, policy_version 1300388 (0.0010) [2023-12-27 14:39:55,392][20381] Updated weights for policy 1, policy_version 1300398 (0.0010) [2023-12-27 14:39:55,440][20381] Updated weights for policy 1, policy_version 1300408 (0.0010) [2023-12-27 14:39:55,869][20347] Updated weights for policy 0, policy_version 1301351 (0.0006) [2023-12-27 14:39:55,927][20347] Updated weights for policy 0, policy_version 1301361 (0.0005) [2023-12-27 14:39:55,983][20347] Updated weights for policy 0, policy_version 1301371 (0.0005) [2023-12-27 14:39:56,105][20381] Updated weights for policy 1, policy_version 1300418 (0.0008) [2023-12-27 14:39:56,158][20381] Updated weights for policy 1, policy_version 1300428 (0.0005) [2023-12-27 14:39:56,214][20381] Updated weights for policy 1, policy_version 1300438 (0.0007) [2023-12-27 14:39:56,559][20347] Updated weights for policy 0, policy_version 1301381 (0.0008) [2023-12-27 14:39:56,607][20347] Updated weights for policy 0, policy_version 1301391 (0.0010) [2023-12-27 14:39:56,673][20347] Updated weights for policy 0, policy_version 1301401 (0.0010) [2023-12-27 14:39:56,904][20381] Updated weights for policy 1, policy_version 1300448 (0.0007) [2023-12-27 14:39:56,960][20381] Updated weights for policy 1, policy_version 1300458 (0.0007) [2023-12-27 14:39:57,020][20381] Updated weights for policy 1, policy_version 1300468 (0.0009) [2023-12-27 14:39:57,314][20347] Updated weights for policy 0, policy_version 1301411 (0.0007) [2023-12-27 14:39:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 666173440. Throughput: 0: 10290.4, 1: 10113.6. Samples: 666166148. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:39:57,363][20347] Updated weights for policy 0, policy_version 1301421 (0.0006) [2023-12-27 14:39:57,363][19170] Avg episode reward: [(0, '922.030'), (1, '980.430')] [2023-12-27 14:39:57,424][20347] Updated weights for policy 0, policy_version 1301431 (0.0005) [2023-12-27 14:39:57,645][20381] Updated weights for policy 1, policy_version 1300478 (0.0008) [2023-12-27 14:39:57,696][20381] Updated weights for policy 1, policy_version 1300488 (0.0009) [2023-12-27 14:39:57,745][20381] Updated weights for policy 1, policy_version 1300498 (0.0009) [2023-12-27 14:39:57,976][20347] Updated weights for policy 0, policy_version 1301441 (0.0006) [2023-12-27 14:39:58,031][20347] Updated weights for policy 0, policy_version 1301451 (0.0005) [2023-12-27 14:39:58,074][20347] Updated weights for policy 0, policy_version 1301461 (0.0005) [2023-12-27 14:39:58,127][20347] Updated weights for policy 0, policy_version 1301471 (0.0005) [2023-12-27 14:39:58,467][20381] Updated weights for policy 1, policy_version 1300508 (0.0009) [2023-12-27 14:39:58,534][20381] Updated weights for policy 1, policy_version 1300518 (0.0011) [2023-12-27 14:39:58,591][20381] Updated weights for policy 1, policy_version 1300528 (0.0010) [2023-12-27 14:39:58,811][20347] Updated weights for policy 0, policy_version 1301481 (0.0007) [2023-12-27 14:39:58,874][20347] Updated weights for policy 0, policy_version 1301491 (0.0009) [2023-12-27 14:39:58,939][20347] Updated weights for policy 0, policy_version 1301501 (0.0007) [2023-12-27 14:39:59,370][20381] Updated weights for policy 1, policy_version 1300538 (0.0010) [2023-12-27 14:39:59,428][20381] Updated weights for policy 1, policy_version 1300548 (0.0007) [2023-12-27 14:39:59,490][20381] Updated weights for policy 1, policy_version 1300558 (0.0008) [2023-12-27 14:39:59,546][20381] Updated weights for policy 1, policy_version 1300568 (0.0007) [2023-12-27 14:39:59,698][20347] Updated weights for policy 0, policy_version 1301511 (0.0010) [2023-12-27 14:39:59,757][20347] Updated weights for policy 0, policy_version 1301521 (0.0010) [2023-12-27 14:39:59,815][20347] Updated weights for policy 0, policy_version 1301531 (0.0010) [2023-12-27 14:40:00,188][20381] Updated weights for policy 1, policy_version 1300578 (0.0011) [2023-12-27 14:40:00,240][20381] Updated weights for policy 1, policy_version 1300588 (0.0008) [2023-12-27 14:40:00,291][20381] Updated weights for policy 1, policy_version 1300598 (0.0005) [2023-12-27 14:40:00,506][20347] Updated weights for policy 0, policy_version 1301541 (0.0009) [2023-12-27 14:40:00,560][20347] Updated weights for policy 0, policy_version 1301551 (0.0010) [2023-12-27 14:40:00,616][20347] Updated weights for policy 0, policy_version 1301561 (0.0008) [2023-12-27 14:40:00,950][20381] Updated weights for policy 1, policy_version 1300608 (0.0007) [2023-12-27 14:40:00,997][20381] Updated weights for policy 1, policy_version 1300618 (0.0005) [2023-12-27 14:40:01,052][20381] Updated weights for policy 1, policy_version 1300628 (0.0007) [2023-12-27 14:40:01,222][20347] Updated weights for policy 0, policy_version 1301571 (0.0006) [2023-12-27 14:40:01,284][20347] Updated weights for policy 0, policy_version 1301581 (0.0009) [2023-12-27 14:40:01,339][20347] Updated weights for policy 0, policy_version 1301591 (0.0010) [2023-12-27 14:40:01,751][20381] Updated weights for policy 1, policy_version 1300638 (0.0007) [2023-12-27 14:40:01,810][20381] Updated weights for policy 1, policy_version 1300648 (0.0008) [2023-12-27 14:40:01,870][20381] Updated weights for policy 1, policy_version 1300658 (0.0008) [2023-12-27 14:40:02,091][20347] Updated weights for policy 0, policy_version 1301601 (0.0008) [2023-12-27 14:40:02,141][20347] Updated weights for policy 0, policy_version 1301611 (0.0008) [2023-12-27 14:40:02,194][20347] Updated weights for policy 0, policy_version 1301621 (0.0008) [2023-12-27 14:40:02,246][20347] Updated weights for policy 0, policy_version 1301631 (0.0008) [2023-12-27 14:40:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 666279936. Throughput: 0: 10298.7, 1: 10127.3. Samples: 666288996. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:02,363][19170] Avg episode reward: [(0, '951.490'), (1, '961.110')] [2023-12-27 14:40:02,570][20381] Updated weights for policy 1, policy_version 1300668 (0.0007) [2023-12-27 14:40:02,638][20381] Updated weights for policy 1, policy_version 1300678 (0.0008) [2023-12-27 14:40:02,671][20292] KL-divergence is very high: 131.4201 [2023-12-27 14:40:02,693][20381] Updated weights for policy 1, policy_version 1300688 (0.0007) [2023-12-27 14:40:02,716][20292] KL-divergence is very high: 109.9901 [2023-12-27 14:40:02,922][20347] Updated weights for policy 0, policy_version 1301641 (0.0005) [2023-12-27 14:40:02,971][20347] Updated weights for policy 0, policy_version 1301651 (0.0005) [2023-12-27 14:40:03,026][20347] Updated weights for policy 0, policy_version 1301661 (0.0007) [2023-12-27 14:40:03,323][20381] Updated weights for policy 1, policy_version 1300698 (0.0007) [2023-12-27 14:40:03,378][20381] Updated weights for policy 1, policy_version 1300708 (0.0005) [2023-12-27 14:40:03,427][20381] Updated weights for policy 1, policy_version 1300718 (0.0005) [2023-12-27 14:40:03,472][20381] Updated weights for policy 1, policy_version 1300728 (0.0007) [2023-12-27 14:40:03,634][20347] Updated weights for policy 0, policy_version 1301671 (0.0007) [2023-12-27 14:40:03,682][20347] Updated weights for policy 0, policy_version 1301681 (0.0007) [2023-12-27 14:40:03,731][20347] Updated weights for policy 0, policy_version 1301691 (0.0008) [2023-12-27 14:40:04,067][20381] Updated weights for policy 1, policy_version 1300738 (0.0006) [2023-12-27 14:40:04,123][20381] Updated weights for policy 1, policy_version 1300748 (0.0006) [2023-12-27 14:40:04,178][20381] Updated weights for policy 1, policy_version 1300758 (0.0006) [2023-12-27 14:40:04,441][20347] Updated weights for policy 0, policy_version 1301701 (0.0009) [2023-12-27 14:40:04,499][20347] Updated weights for policy 0, policy_version 1301711 (0.0010) [2023-12-27 14:40:04,557][20347] Updated weights for policy 0, policy_version 1301721 (0.0010) [2023-12-27 14:40:04,874][20381] Updated weights for policy 1, policy_version 1300768 (0.0009) [2023-12-27 14:40:04,921][20381] Updated weights for policy 1, policy_version 1300778 (0.0010) [2023-12-27 14:40:04,967][20381] Updated weights for policy 1, policy_version 1300788 (0.0010) [2023-12-27 14:40:05,181][20347] Updated weights for policy 0, policy_version 1301731 (0.0010) [2023-12-27 14:40:05,229][20347] Updated weights for policy 0, policy_version 1301741 (0.0007) [2023-12-27 14:40:05,278][20347] Updated weights for policy 0, policy_version 1301751 (0.0005) [2023-12-27 14:40:05,677][20381] Updated weights for policy 1, policy_version 1300798 (0.0008) [2023-12-27 14:40:05,728][20381] Updated weights for policy 1, policy_version 1300808 (0.0010) [2023-12-27 14:40:05,787][20381] Updated weights for policy 1, policy_version 1300818 (0.0010) [2023-12-27 14:40:05,901][20347] Updated weights for policy 0, policy_version 1301761 (0.0006) [2023-12-27 14:40:05,946][20347] Updated weights for policy 0, policy_version 1301771 (0.0010) [2023-12-27 14:40:05,995][20347] Updated weights for policy 0, policy_version 1301781 (0.0010) [2023-12-27 14:40:06,045][20347] Updated weights for policy 0, policy_version 1301791 (0.0009) [2023-12-27 14:40:06,401][20381] Updated weights for policy 1, policy_version 1300828 (0.0007) [2023-12-27 14:40:06,470][20381] Updated weights for policy 1, policy_version 1300838 (0.0008) [2023-12-27 14:40:06,533][20381] Updated weights for policy 1, policy_version 1300848 (0.0007) [2023-12-27 14:40:06,785][20347] Updated weights for policy 0, policy_version 1301801 (0.0006) [2023-12-27 14:40:06,849][20347] Updated weights for policy 0, policy_version 1301811 (0.0006) [2023-12-27 14:40:06,905][20347] Updated weights for policy 0, policy_version 1301821 (0.0007) [2023-12-27 14:40:07,182][20381] Updated weights for policy 1, policy_version 1300858 (0.0007) [2023-12-27 14:40:07,242][20381] Updated weights for policy 1, policy_version 1300868 (0.0008) [2023-12-27 14:40:07,300][20381] Updated weights for policy 1, policy_version 1300878 (0.0008) [2023-12-27 14:40:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 666386432. Throughput: 0: 10332.5, 1: 10144.8. Samples: 666352320. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:07,363][20381] Updated weights for policy 1, policy_version 1300888 (0.0008) [2023-12-27 14:40:07,363][19170] Avg episode reward: [(0, '902.560'), (1, '980.680')] [2023-12-27 14:40:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001301824_333332480.pth... [2023-12-27 14:40:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001300888_333053952.pth... [2023-12-27 14:40:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001300640_333029376.pth [2023-12-27 14:40:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001299704_332750848.pth [2023-12-27 14:40:07,530][20347] Updated weights for policy 0, policy_version 1301831 (0.0008) [2023-12-27 14:40:07,584][20347] Updated weights for policy 0, policy_version 1301841 (0.0008) [2023-12-27 14:40:07,634][20347] Updated weights for policy 0, policy_version 1301851 (0.0009) [2023-12-27 14:40:07,967][20381] Updated weights for policy 1, policy_version 1300898 (0.0008) [2023-12-27 14:40:08,025][20381] Updated weights for policy 1, policy_version 1300908 (0.0010) [2023-12-27 14:40:08,076][20381] Updated weights for policy 1, policy_version 1300918 (0.0010) [2023-12-27 14:40:08,292][20347] Updated weights for policy 0, policy_version 1301861 (0.0009) [2023-12-27 14:40:08,355][20347] Updated weights for policy 0, policy_version 1301871 (0.0009) [2023-12-27 14:40:08,418][20347] Updated weights for policy 0, policy_version 1301881 (0.0008) [2023-12-27 14:40:08,809][20381] Updated weights for policy 1, policy_version 1300928 (0.0009) [2023-12-27 14:40:08,870][20381] Updated weights for policy 1, policy_version 1300938 (0.0007) [2023-12-27 14:40:08,925][20381] Updated weights for policy 1, policy_version 1300948 (0.0006) [2023-12-27 14:40:09,082][20347] Updated weights for policy 0, policy_version 1301891 (0.0009) [2023-12-27 14:40:09,149][20347] Updated weights for policy 0, policy_version 1301901 (0.0007) [2023-12-27 14:40:09,213][20347] Updated weights for policy 0, policy_version 1301911 (0.0007) [2023-12-27 14:40:09,611][20381] Updated weights for policy 1, policy_version 1300958 (0.0007) [2023-12-27 14:40:09,671][20381] Updated weights for policy 1, policy_version 1300968 (0.0008) [2023-12-27 14:40:09,736][20381] Updated weights for policy 1, policy_version 1300978 (0.0007) [2023-12-27 14:40:09,972][20347] Updated weights for policy 0, policy_version 1301921 (0.0008) [2023-12-27 14:40:10,035][20347] Updated weights for policy 0, policy_version 1301931 (0.0007) [2023-12-27 14:40:10,102][20347] Updated weights for policy 0, policy_version 1301941 (0.0007) [2023-12-27 14:40:10,161][20347] Updated weights for policy 0, policy_version 1301951 (0.0006) [2023-12-27 14:40:10,442][20381] Updated weights for policy 1, policy_version 1300988 (0.0007) [2023-12-27 14:40:10,500][20381] Updated weights for policy 1, policy_version 1300998 (0.0006) [2023-12-27 14:40:10,557][20381] Updated weights for policy 1, policy_version 1301008 (0.0007) [2023-12-27 14:40:10,858][20347] Updated weights for policy 0, policy_version 1301961 (0.0007) [2023-12-27 14:40:10,915][20347] Updated weights for policy 0, policy_version 1301971 (0.0008) [2023-12-27 14:40:10,960][20347] Updated weights for policy 0, policy_version 1301981 (0.0008) [2023-12-27 14:40:11,237][20381] Updated weights for policy 1, policy_version 1301018 (0.0008) [2023-12-27 14:40:11,303][20381] Updated weights for policy 1, policy_version 1301028 (0.0006) [2023-12-27 14:40:11,368][20381] Updated weights for policy 1, policy_version 1301038 (0.0007) [2023-12-27 14:40:11,441][20381] Updated weights for policy 1, policy_version 1301048 (0.0007) [2023-12-27 14:40:11,808][20347] Updated weights for policy 0, policy_version 1301991 (0.0008) [2023-12-27 14:40:11,863][20347] Updated weights for policy 0, policy_version 1302001 (0.0008) [2023-12-27 14:40:11,924][20347] Updated weights for policy 0, policy_version 1302011 (0.0008) [2023-12-27 14:40:12,094][20381] Updated weights for policy 1, policy_version 1301058 (0.0008) [2023-12-27 14:40:12,153][20381] Updated weights for policy 1, policy_version 1301068 (0.0008) [2023-12-27 14:40:12,212][20381] Updated weights for policy 1, policy_version 1301078 (0.0007) [2023-12-27 14:40:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 666484736. Throughput: 0: 10329.8, 1: 10163.6. Samples: 666474920. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:12,363][19170] Avg episode reward: [(0, '902.580'), (1, '1000.000')] [2023-12-27 14:40:12,657][20347] Updated weights for policy 0, policy_version 1302021 (0.0008) [2023-12-27 14:40:12,717][20347] Updated weights for policy 0, policy_version 1302031 (0.0007) [2023-12-27 14:40:12,781][20347] Updated weights for policy 0, policy_version 1302041 (0.0008) [2023-12-27 14:40:12,928][20381] Updated weights for policy 1, policy_version 1301088 (0.0007) [2023-12-27 14:40:12,979][20381] Updated weights for policy 1, policy_version 1301098 (0.0007) [2023-12-27 14:40:13,034][20381] Updated weights for policy 1, policy_version 1301108 (0.0007) [2023-12-27 14:40:13,444][20347] Updated weights for policy 0, policy_version 1302051 (0.0007) [2023-12-27 14:40:13,500][20347] Updated weights for policy 0, policy_version 1302061 (0.0005) [2023-12-27 14:40:13,563][20347] Updated weights for policy 0, policy_version 1302071 (0.0006) [2023-12-27 14:40:13,720][20381] Updated weights for policy 1, policy_version 1301118 (0.0006) [2023-12-27 14:40:13,766][20381] Updated weights for policy 1, policy_version 1301128 (0.0005) [2023-12-27 14:40:13,809][20381] Updated weights for policy 1, policy_version 1301138 (0.0005) [2023-12-27 14:40:14,210][20347] Updated weights for policy 0, policy_version 1302081 (0.0009) [2023-12-27 14:40:14,271][20347] Updated weights for policy 0, policy_version 1302091 (0.0007) [2023-12-27 14:40:14,334][20347] Updated weights for policy 0, policy_version 1302101 (0.0007) [2023-12-27 14:40:14,392][20347] Updated weights for policy 0, policy_version 1302111 (0.0007) [2023-12-27 14:40:14,402][20381] Updated weights for policy 1, policy_version 1301148 (0.0006) [2023-12-27 14:40:14,450][20381] Updated weights for policy 1, policy_version 1301158 (0.0007) [2023-12-27 14:40:14,506][20381] Updated weights for policy 1, policy_version 1301168 (0.0008) [2023-12-27 14:40:15,072][20347] Updated weights for policy 0, policy_version 1302121 (0.0007) [2023-12-27 14:40:15,136][20347] Updated weights for policy 0, policy_version 1302131 (0.0007) [2023-12-27 14:40:15,138][20381] Updated weights for policy 1, policy_version 1301178 (0.0008) [2023-12-27 14:40:15,195][20381] Updated weights for policy 1, policy_version 1301188 (0.0006) [2023-12-27 14:40:15,196][20347] Updated weights for policy 0, policy_version 1302141 (0.0006) [2023-12-27 14:40:15,248][20381] Updated weights for policy 1, policy_version 1301198 (0.0007) [2023-12-27 14:40:15,306][20381] Updated weights for policy 1, policy_version 1301208 (0.0008) [2023-12-27 14:40:15,881][20347] Updated weights for policy 0, policy_version 1302151 (0.0008) [2023-12-27 14:40:15,931][20347] Updated weights for policy 0, policy_version 1302161 (0.0007) [2023-12-27 14:40:15,977][20347] Updated weights for policy 0, policy_version 1302171 (0.0007) [2023-12-27 14:40:16,016][20381] Updated weights for policy 1, policy_version 1301218 (0.0006) [2023-12-27 14:40:16,075][20381] Updated weights for policy 1, policy_version 1301228 (0.0011) [2023-12-27 14:40:16,130][20381] Updated weights for policy 1, policy_version 1301238 (0.0007) [2023-12-27 14:40:16,677][20347] Updated weights for policy 0, policy_version 1302181 (0.0007) [2023-12-27 14:40:16,727][20347] Updated weights for policy 0, policy_version 1302191 (0.0006) [2023-12-27 14:40:16,764][20381] Updated weights for policy 1, policy_version 1301248 (0.0007) [2023-12-27 14:40:16,777][20347] Updated weights for policy 0, policy_version 1302201 (0.0006) [2023-12-27 14:40:16,826][20381] Updated weights for policy 1, policy_version 1301258 (0.0007) [2023-12-27 14:40:16,883][20381] Updated weights for policy 1, policy_version 1301268 (0.0008) [2023-12-27 14:40:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 666583040. Throughput: 0: 10207.1, 1: 10234.7. Samples: 666596884. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:17,363][19170] Avg episode reward: [(0, '922.320'), (1, '1000.000')] [2023-12-27 14:40:17,462][20347] Updated weights for policy 0, policy_version 1302211 (0.0008) [2023-12-27 14:40:17,523][20347] Updated weights for policy 0, policy_version 1302221 (0.0010) [2023-12-27 14:40:17,530][20381] Updated weights for policy 1, policy_version 1301278 (0.0009) [2023-12-27 14:40:17,582][20347] Updated weights for policy 0, policy_version 1302231 (0.0010) [2023-12-27 14:40:17,586][20381] Updated weights for policy 1, policy_version 1301288 (0.0011) [2023-12-27 14:40:17,642][20381] Updated weights for policy 1, policy_version 1301298 (0.0011) [2023-12-27 14:40:18,253][20347] Updated weights for policy 0, policy_version 1302241 (0.0010) [2023-12-27 14:40:18,270][20381] Updated weights for policy 1, policy_version 1301308 (0.0009) [2023-12-27 14:40:18,304][20347] Updated weights for policy 0, policy_version 1302251 (0.0010) [2023-12-27 14:40:18,332][20381] Updated weights for policy 1, policy_version 1301319 (0.0006) [2023-12-27 14:40:18,362][20347] Updated weights for policy 0, policy_version 1302261 (0.0007) [2023-12-27 14:40:18,395][20381] Updated weights for policy 1, policy_version 1301329 (0.0007) [2023-12-27 14:40:18,426][20347] Updated weights for policy 0, policy_version 1302271 (0.0008) [2023-12-27 14:40:19,004][20381] Updated weights for policy 1, policy_version 1301339 (0.0007) [2023-12-27 14:40:19,060][20381] Updated weights for policy 1, policy_version 1301349 (0.0008) [2023-12-27 14:40:19,108][20381] Updated weights for policy 1, policy_version 1301359 (0.0007) [2023-12-27 14:40:19,144][20347] Updated weights for policy 0, policy_version 1302281 (0.0006) [2023-12-27 14:40:19,200][20347] Updated weights for policy 0, policy_version 1302291 (0.0008) [2023-12-27 14:40:19,266][20347] Updated weights for policy 0, policy_version 1302301 (0.0008) [2023-12-27 14:40:19,804][20381] Updated weights for policy 1, policy_version 1301369 (0.0005) [2023-12-27 14:40:19,870][20381] Updated weights for policy 1, policy_version 1301379 (0.0008) [2023-12-27 14:40:19,938][20381] Updated weights for policy 1, policy_version 1301389 (0.0009) [2023-12-27 14:40:19,993][20381] Updated weights for policy 1, policy_version 1301399 (0.0007) [2023-12-27 14:40:20,075][20347] Updated weights for policy 0, policy_version 1302311 (0.0008) [2023-12-27 14:40:20,136][20347] Updated weights for policy 0, policy_version 1302321 (0.0008) [2023-12-27 14:40:20,200][20347] Updated weights for policy 0, policy_version 1302331 (0.0007) [2023-12-27 14:40:20,710][20381] Updated weights for policy 1, policy_version 1301409 (0.0007) [2023-12-27 14:40:20,779][20381] Updated weights for policy 1, policy_version 1301419 (0.0006) [2023-12-27 14:40:20,843][20381] Updated weights for policy 1, policy_version 1301429 (0.0009) [2023-12-27 14:40:20,851][20347] Updated weights for policy 0, policy_version 1302341 (0.0009) [2023-12-27 14:40:20,911][20347] Updated weights for policy 0, policy_version 1302351 (0.0011) [2023-12-27 14:40:20,965][20347] Updated weights for policy 0, policy_version 1302361 (0.0009) [2023-12-27 14:40:21,548][20381] Updated weights for policy 1, policy_version 1301439 (0.0011) [2023-12-27 14:40:21,620][20381] Updated weights for policy 1, policy_version 1301449 (0.0006) [2023-12-27 14:40:21,680][20381] Updated weights for policy 1, policy_version 1301459 (0.0008) [2023-12-27 14:40:21,700][20347] Updated weights for policy 0, policy_version 1302371 (0.0006) [2023-12-27 14:40:21,797][20347] Updated weights for policy 0, policy_version 1302381 (0.0006) [2023-12-27 14:40:21,856][20347] Updated weights for policy 0, policy_version 1302391 (0.0008) [2023-12-27 14:40:22,331][20381] Updated weights for policy 1, policy_version 1301469 (0.0011) [2023-12-27 14:40:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 666681344. Throughput: 0: 10179.8, 1: 10218.9. Samples: 666657436. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:22,363][19170] Avg episode reward: [(0, '912.650'), (1, '990.240')] [2023-12-27 14:40:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001302400_333479936.pth... [2023-12-27 14:40:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001301216_333176832.pth [2023-12-27 14:40:22,400][20381] Updated weights for policy 1, policy_version 1301479 (0.0007) [2023-12-27 14:40:22,465][20381] Updated weights for policy 1, policy_version 1301489 (0.0006) [2023-12-27 14:40:22,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001301496_333209600.pth... [2023-12-27 14:40:22,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001300280_332898304.pth [2023-12-27 14:40:22,537][20347] Updated weights for policy 0, policy_version 1302401 (0.0008) [2023-12-27 14:40:22,588][20347] Updated weights for policy 0, policy_version 1302411 (0.0006) [2023-12-27 14:40:22,643][20347] Updated weights for policy 0, policy_version 1302421 (0.0007) [2023-12-27 14:40:22,700][20347] Updated weights for policy 0, policy_version 1302431 (0.0007) [2023-12-27 14:40:23,165][20381] Updated weights for policy 1, policy_version 1301499 (0.0008) [2023-12-27 14:40:23,221][20381] Updated weights for policy 1, policy_version 1301509 (0.0005) [2023-12-27 14:40:23,273][20381] Updated weights for policy 1, policy_version 1301519 (0.0005) [2023-12-27 14:40:23,391][20347] Updated weights for policy 0, policy_version 1302441 (0.0006) [2023-12-27 14:40:23,443][20347] Updated weights for policy 0, policy_version 1302451 (0.0005) [2023-12-27 14:40:23,507][20347] Updated weights for policy 0, policy_version 1302461 (0.0006) [2023-12-27 14:40:23,857][20381] Updated weights for policy 1, policy_version 1301529 (0.0006) [2023-12-27 14:40:23,920][20381] Updated weights for policy 1, policy_version 1301539 (0.0007) [2023-12-27 14:40:23,982][20381] Updated weights for policy 1, policy_version 1301549 (0.0007) [2023-12-27 14:40:24,029][20381] Updated weights for policy 1, policy_version 1301559 (0.0006) [2023-12-27 14:40:24,049][20347] Updated weights for policy 0, policy_version 1302471 (0.0005) [2023-12-27 14:40:24,110][20347] Updated weights for policy 0, policy_version 1302481 (0.0008) [2023-12-27 14:40:24,177][20347] Updated weights for policy 0, policy_version 1302491 (0.0010) [2023-12-27 14:40:24,604][20381] Updated weights for policy 1, policy_version 1301569 (0.0005) [2023-12-27 14:40:24,653][20381] Updated weights for policy 1, policy_version 1301579 (0.0005) [2023-12-27 14:40:24,702][20381] Updated weights for policy 1, policy_version 1301589 (0.0005) [2023-12-27 14:40:24,829][20347] Updated weights for policy 0, policy_version 1302501 (0.0009) [2023-12-27 14:40:24,876][20347] Updated weights for policy 0, policy_version 1302511 (0.0007) [2023-12-27 14:40:24,926][20347] Updated weights for policy 0, policy_version 1302521 (0.0009) [2023-12-27 14:40:25,302][20381] Updated weights for policy 1, policy_version 1301599 (0.0009) [2023-12-27 14:40:25,360][20381] Updated weights for policy 1, policy_version 1301609 (0.0010) [2023-12-27 14:40:25,415][20381] Updated weights for policy 1, policy_version 1301619 (0.0011) [2023-12-27 14:40:25,604][20347] Updated weights for policy 0, policy_version 1302531 (0.0011) [2023-12-27 14:40:25,655][20347] Updated weights for policy 0, policy_version 1302541 (0.0010) [2023-12-27 14:40:25,707][20347] Updated weights for policy 0, policy_version 1302551 (0.0010) [2023-12-27 14:40:26,093][20381] Updated weights for policy 1, policy_version 1301629 (0.0011) [2023-12-27 14:40:26,152][20381] Updated weights for policy 1, policy_version 1301639 (0.0010) [2023-12-27 14:40:26,204][20381] Updated weights for policy 1, policy_version 1301649 (0.0010) [2023-12-27 14:40:26,380][20347] Updated weights for policy 0, policy_version 1302561 (0.0010) [2023-12-27 14:40:26,427][20347] Updated weights for policy 0, policy_version 1302571 (0.0010) [2023-12-27 14:40:26,474][20347] Updated weights for policy 0, policy_version 1302581 (0.0010) [2023-12-27 14:40:26,518][20347] Updated weights for policy 0, policy_version 1302591 (0.0010) [2023-12-27 14:40:26,884][20381] Updated weights for policy 1, policy_version 1301659 (0.0010) [2023-12-27 14:40:26,943][20381] Updated weights for policy 1, policy_version 1301669 (0.0011) [2023-12-27 14:40:26,995][20381] Updated weights for policy 1, policy_version 1301679 (0.0010) [2023-12-27 14:40:27,172][20347] Updated weights for policy 0, policy_version 1302601 (0.0010) [2023-12-27 14:40:27,217][20347] Updated weights for policy 0, policy_version 1302611 (0.0010) [2023-12-27 14:40:27,275][20347] Updated weights for policy 0, policy_version 1302621 (0.0010) [2023-12-27 14:40:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 666796032. Throughput: 0: 10214.8, 1: 10259.3. Samples: 666782408. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:27,363][19170] Avg episode reward: [(0, '922.310'), (1, '990.240')] [2023-12-27 14:40:27,698][20381] Updated weights for policy 1, policy_version 1301689 (0.0011) [2023-12-27 14:40:27,751][20381] Updated weights for policy 1, policy_version 1301699 (0.0010) [2023-12-27 14:40:27,806][20381] Updated weights for policy 1, policy_version 1301709 (0.0011) [2023-12-27 14:40:27,852][20381] Updated weights for policy 1, policy_version 1301719 (0.0010) [2023-12-27 14:40:27,966][20347] Updated weights for policy 0, policy_version 1302631 (0.0010) [2023-12-27 14:40:28,023][20347] Updated weights for policy 0, policy_version 1302641 (0.0010) [2023-12-27 14:40:28,071][20347] Updated weights for policy 0, policy_version 1302651 (0.0010) [2023-12-27 14:40:28,487][20381] Updated weights for policy 1, policy_version 1301729 (0.0010) [2023-12-27 14:40:28,543][20381] Updated weights for policy 1, policy_version 1301739 (0.0011) [2023-12-27 14:40:28,604][20381] Updated weights for policy 1, policy_version 1301749 (0.0011) [2023-12-27 14:40:28,767][20347] Updated weights for policy 0, policy_version 1302661 (0.0010) [2023-12-27 14:40:28,826][20347] Updated weights for policy 0, policy_version 1302671 (0.0008) [2023-12-27 14:40:28,890][20347] Updated weights for policy 0, policy_version 1302681 (0.0009) [2023-12-27 14:40:29,321][20381] Updated weights for policy 1, policy_version 1301759 (0.0009) [2023-12-27 14:40:29,389][20381] Updated weights for policy 1, policy_version 1301769 (0.0008) [2023-12-27 14:40:29,442][20381] Updated weights for policy 1, policy_version 1301779 (0.0008) [2023-12-27 14:40:29,559][20347] Updated weights for policy 0, policy_version 1302691 (0.0007) [2023-12-27 14:40:29,608][20347] Updated weights for policy 0, policy_version 1302701 (0.0006) [2023-12-27 14:40:29,663][20347] Updated weights for policy 0, policy_version 1302711 (0.0005) [2023-12-27 14:40:30,141][20381] Updated weights for policy 1, policy_version 1301789 (0.0008) [2023-12-27 14:40:30,200][20381] Updated weights for policy 1, policy_version 1301799 (0.0007) [2023-12-27 14:40:30,260][20381] Updated weights for policy 1, policy_version 1301809 (0.0006) [2023-12-27 14:40:30,329][20347] Updated weights for policy 0, policy_version 1302721 (0.0006) [2023-12-27 14:40:30,381][20347] Updated weights for policy 0, policy_version 1302731 (0.0008) [2023-12-27 14:40:30,426][20347] Updated weights for policy 0, policy_version 1302741 (0.0008) [2023-12-27 14:40:30,478][20347] Updated weights for policy 0, policy_version 1302751 (0.0008) [2023-12-27 14:40:30,820][20381] Updated weights for policy 1, policy_version 1301819 (0.0007) [2023-12-27 14:40:30,875][20381] Updated weights for policy 1, policy_version 1301829 (0.0008) [2023-12-27 14:40:30,924][20381] Updated weights for policy 1, policy_version 1301839 (0.0008) [2023-12-27 14:40:31,187][20347] Updated weights for policy 0, policy_version 1302761 (0.0007) [2023-12-27 14:40:31,239][20347] Updated weights for policy 0, policy_version 1302771 (0.0008) [2023-12-27 14:40:31,299][20347] Updated weights for policy 0, policy_version 1302781 (0.0007) [2023-12-27 14:40:31,703][20381] Updated weights for policy 1, policy_version 1301849 (0.0008) [2023-12-27 14:40:31,770][20381] Updated weights for policy 1, policy_version 1301859 (0.0006) [2023-12-27 14:40:31,830][20381] Updated weights for policy 1, policy_version 1301869 (0.0005) [2023-12-27 14:40:31,897][20381] Updated weights for policy 1, policy_version 1301879 (0.0006) [2023-12-27 14:40:32,019][20347] Updated weights for policy 0, policy_version 1302791 (0.0006) [2023-12-27 14:40:32,082][20347] Updated weights for policy 0, policy_version 1302801 (0.0005) [2023-12-27 14:40:32,139][20347] Updated weights for policy 0, policy_version 1302811 (0.0005) [2023-12-27 14:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 666894336. Throughput: 0: 10201.6, 1: 10308.2. Samples: 666905864. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:32,363][19170] Avg episode reward: [(0, '931.650'), (1, '1000.000')] [2023-12-27 14:40:32,471][20381] Updated weights for policy 1, policy_version 1301889 (0.0006) [2023-12-27 14:40:32,517][20381] Updated weights for policy 1, policy_version 1301899 (0.0005) [2023-12-27 14:40:32,570][20381] Updated weights for policy 1, policy_version 1301909 (0.0006) [2023-12-27 14:40:32,763][20347] Updated weights for policy 0, policy_version 1302821 (0.0007) [2023-12-27 14:40:32,819][20347] Updated weights for policy 0, policy_version 1302831 (0.0008) [2023-12-27 14:40:32,873][20347] Updated weights for policy 0, policy_version 1302841 (0.0008) [2023-12-27 14:40:33,140][20381] Updated weights for policy 1, policy_version 1301919 (0.0005) [2023-12-27 14:40:33,193][20381] Updated weights for policy 1, policy_version 1301929 (0.0005) [2023-12-27 14:40:33,244][20381] Updated weights for policy 1, policy_version 1301939 (0.0006) [2023-12-27 14:40:33,615][20347] Updated weights for policy 0, policy_version 1302851 (0.0008) [2023-12-27 14:40:33,673][20347] Updated weights for policy 0, policy_version 1302861 (0.0009) [2023-12-27 14:40:33,736][20347] Updated weights for policy 0, policy_version 1302871 (0.0007) [2023-12-27 14:40:33,796][20381] Updated weights for policy 1, policy_version 1301949 (0.0006) [2023-12-27 14:40:33,848][20381] Updated weights for policy 1, policy_version 1301959 (0.0006) [2023-12-27 14:40:33,905][20381] Updated weights for policy 1, policy_version 1301969 (0.0006) [2023-12-27 14:40:34,446][20347] Updated weights for policy 0, policy_version 1302881 (0.0007) [2023-12-27 14:40:34,506][20347] Updated weights for policy 0, policy_version 1302891 (0.0008) [2023-12-27 14:40:34,552][20381] Updated weights for policy 1, policy_version 1301979 (0.0005) [2023-12-27 14:40:34,567][20347] Updated weights for policy 0, policy_version 1302901 (0.0008) [2023-12-27 14:40:34,608][20381] Updated weights for policy 1, policy_version 1301989 (0.0008) [2023-12-27 14:40:34,625][20347] Updated weights for policy 0, policy_version 1302911 (0.0008) [2023-12-27 14:40:34,663][20381] Updated weights for policy 1, policy_version 1301999 (0.0006) [2023-12-27 14:40:35,314][20347] Updated weights for policy 0, policy_version 1302921 (0.0008) [2023-12-27 14:40:35,353][20381] Updated weights for policy 1, policy_version 1302009 (0.0007) [2023-12-27 14:40:35,375][20347] Updated weights for policy 0, policy_version 1302931 (0.0008) [2023-12-27 14:40:35,410][20381] Updated weights for policy 1, policy_version 1302019 (0.0005) [2023-12-27 14:40:35,438][20347] Updated weights for policy 0, policy_version 1302941 (0.0007) [2023-12-27 14:40:35,467][20381] Updated weights for policy 1, policy_version 1302029 (0.0006) [2023-12-27 14:40:35,520][20381] Updated weights for policy 1, policy_version 1302039 (0.0005) [2023-12-27 14:40:36,059][20381] Updated weights for policy 1, policy_version 1302049 (0.0007) [2023-12-27 14:40:36,111][20381] Updated weights for policy 1, policy_version 1302059 (0.0008) [2023-12-27 14:40:36,176][20347] Updated weights for policy 0, policy_version 1302951 (0.0007) [2023-12-27 14:40:36,177][20381] Updated weights for policy 1, policy_version 1302069 (0.0007) [2023-12-27 14:40:36,236][20347] Updated weights for policy 0, policy_version 1302961 (0.0007) [2023-12-27 14:40:36,294][20347] Updated weights for policy 0, policy_version 1302971 (0.0008) [2023-12-27 14:40:36,913][20381] Updated weights for policy 1, policy_version 1302079 (0.0009) [2023-12-27 14:40:36,974][20347] Updated weights for policy 0, policy_version 1302981 (0.0007) [2023-12-27 14:40:36,976][20381] Updated weights for policy 1, policy_version 1302089 (0.0008) [2023-12-27 14:40:37,018][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-27 14:40:37,026][20347] Updated weights for policy 0, policy_version 1302991 (0.0006) [2023-12-27 14:40:37,089][20347] Updated weights for policy 0, policy_version 1303001 (0.0011) [2023-12-27 14:40:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 667000832. Throughput: 0: 10179.6, 1: 10386.7. Samples: 666968700. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:37,364][19170] Avg episode reward: [(0, '931.850'), (1, '980.550')] [2023-12-27 14:40:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001302096_333365248.pth... [2023-12-27 14:40:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001303008_333635584.pth... [2023-12-27 14:40:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001300888_333053952.pth [2023-12-27 14:40:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001301824_333332480.pth [2023-12-27 14:40:37,614][20381] Updated weights for policy 1, policy_version 1302099 (0.0008) [2023-12-27 14:40:37,664][20381] Updated weights for policy 1, policy_version 1302109 (0.0005) [2023-12-27 14:40:37,723][20381] Updated weights for policy 1, policy_version 1302119 (0.0006) [2023-12-27 14:40:37,724][20347] Updated weights for policy 0, policy_version 1303011 (0.0009) [2023-12-27 14:40:37,779][20347] Updated weights for policy 0, policy_version 1303021 (0.0005) [2023-12-27 14:40:37,837][20347] Updated weights for policy 0, policy_version 1303031 (0.0005) [2023-12-27 14:40:38,341][20381] Updated weights for policy 1, policy_version 1302129 (0.0007) [2023-12-27 14:40:38,407][20381] Updated weights for policy 1, policy_version 1302139 (0.0008) [2023-12-27 14:40:38,462][20381] Updated weights for policy 1, policy_version 1302149 (0.0008) [2023-12-27 14:40:38,499][20347] Updated weights for policy 0, policy_version 1303041 (0.0007) [2023-12-27 14:40:38,513][20381] Updated weights for policy 1, policy_version 1302159 (0.0006) [2023-12-27 14:40:38,560][20347] Updated weights for policy 0, policy_version 1303051 (0.0007) [2023-12-27 14:40:38,620][20347] Updated weights for policy 0, policy_version 1303061 (0.0008) [2023-12-27 14:40:38,677][20347] Updated weights for policy 0, policy_version 1303071 (0.0007) [2023-12-27 14:40:39,267][20381] Updated weights for policy 1, policy_version 1302169 (0.0007) [2023-12-27 14:40:39,328][20381] Updated weights for policy 1, policy_version 1302179 (0.0006) [2023-12-27 14:40:39,386][20347] Updated weights for policy 0, policy_version 1303081 (0.0008) [2023-12-27 14:40:39,402][20381] Updated weights for policy 1, policy_version 1302189 (0.0008) [2023-12-27 14:40:39,457][20347] Updated weights for policy 0, policy_version 1303091 (0.0007) [2023-12-27 14:40:39,522][20347] Updated weights for policy 0, policy_version 1303101 (0.0007) [2023-12-27 14:40:40,088][20381] Updated weights for policy 1, policy_version 1302199 (0.0008) [2023-12-27 14:40:40,148][20381] Updated weights for policy 1, policy_version 1302209 (0.0008) [2023-12-27 14:40:40,200][20381] Updated weights for policy 1, policy_version 1302219 (0.0008) [2023-12-27 14:40:40,277][20347] Updated weights for policy 0, policy_version 1303111 (0.0007) [2023-12-27 14:40:40,340][20347] Updated weights for policy 0, policy_version 1303121 (0.0007) [2023-12-27 14:40:40,406][20347] Updated weights for policy 0, policy_version 1303131 (0.0007) [2023-12-27 14:40:40,922][20381] Updated weights for policy 1, policy_version 1302229 (0.0007) [2023-12-27 14:40:40,983][20381] Updated weights for policy 1, policy_version 1302239 (0.0008) [2023-12-27 14:40:41,020][20347] Updated weights for policy 0, policy_version 1303141 (0.0007) [2023-12-27 14:40:41,048][20381] Updated weights for policy 1, policy_version 1302249 (0.0008) [2023-12-27 14:40:41,083][20347] Updated weights for policy 0, policy_version 1303151 (0.0006) [2023-12-27 14:40:41,150][20347] Updated weights for policy 0, policy_version 1303161 (0.0008) [2023-12-27 14:40:41,755][20381] Updated weights for policy 1, policy_version 1302259 (0.0007) [2023-12-27 14:40:41,813][20381] Updated weights for policy 1, policy_version 1302269 (0.0008) [2023-12-27 14:40:41,850][20347] Updated weights for policy 0, policy_version 1303171 (0.0006) [2023-12-27 14:40:41,875][20381] Updated weights for policy 1, policy_version 1302279 (0.0007) [2023-12-27 14:40:41,909][20347] Updated weights for policy 0, policy_version 1303181 (0.0006) [2023-12-27 14:40:41,974][20347] Updated weights for policy 0, policy_version 1303191 (0.0008) [2023-12-27 14:40:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 667099136. Throughput: 0: 10154.0, 1: 10390.8. Samples: 667090660. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:42,363][19170] Avg episode reward: [(0, '902.670'), (1, '980.550')] [2023-12-27 14:40:42,634][20381] Updated weights for policy 1, policy_version 1302289 (0.0006) [2023-12-27 14:40:42,659][20347] Updated weights for policy 0, policy_version 1303201 (0.0007) [2023-12-27 14:40:42,686][20381] Updated weights for policy 1, policy_version 1302299 (0.0007) [2023-12-27 14:40:42,718][20347] Updated weights for policy 0, policy_version 1303211 (0.0008) [2023-12-27 14:40:42,737][20381] Updated weights for policy 1, policy_version 1302309 (0.0008) [2023-12-27 14:40:42,768][20347] Updated weights for policy 0, policy_version 1303221 (0.0008) [2023-12-27 14:40:42,793][20381] Updated weights for policy 1, policy_version 1302319 (0.0008) [2023-12-27 14:40:42,823][20347] Updated weights for policy 0, policy_version 1303231 (0.0007) [2023-12-27 14:40:43,467][20381] Updated weights for policy 1, policy_version 1302329 (0.0010) [2023-12-27 14:40:43,526][20347] Updated weights for policy 0, policy_version 1303241 (0.0007) [2023-12-27 14:40:43,530][20381] Updated weights for policy 1, policy_version 1302339 (0.0010) [2023-12-27 14:40:43,582][20381] Updated weights for policy 1, policy_version 1302349 (0.0010) [2023-12-27 14:40:43,582][20347] Updated weights for policy 0, policy_version 1303251 (0.0007) [2023-12-27 14:40:43,639][20347] Updated weights for policy 0, policy_version 1303261 (0.0008) [2023-12-27 14:40:44,256][20381] Updated weights for policy 1, policy_version 1302359 (0.0009) [2023-12-27 14:40:44,307][20381] Updated weights for policy 1, policy_version 1302369 (0.0007) [2023-12-27 14:40:44,327][20347] Updated weights for policy 0, policy_version 1303271 (0.0007) [2023-12-27 14:40:44,362][20381] Updated weights for policy 1, policy_version 1302379 (0.0008) [2023-12-27 14:40:44,390][20347] Updated weights for policy 0, policy_version 1303281 (0.0008) [2023-12-27 14:40:44,451][20347] Updated weights for policy 0, policy_version 1303291 (0.0010) [2023-12-27 14:40:45,011][20381] Updated weights for policy 1, policy_version 1302389 (0.0007) [2023-12-27 14:40:45,074][20381] Updated weights for policy 1, policy_version 1302399 (0.0008) [2023-12-27 14:40:45,087][20347] Updated weights for policy 0, policy_version 1303301 (0.0008) [2023-12-27 14:40:45,138][20381] Updated weights for policy 1, policy_version 1302409 (0.0007) [2023-12-27 14:40:45,148][20347] Updated weights for policy 0, policy_version 1303311 (0.0007) [2023-12-27 14:40:45,206][20347] Updated weights for policy 0, policy_version 1303321 (0.0009) [2023-12-27 14:40:45,775][20381] Updated weights for policy 1, policy_version 1302419 (0.0007) [2023-12-27 14:40:45,829][20381] Updated weights for policy 1, policy_version 1302429 (0.0008) [2023-12-27 14:40:45,846][20347] Updated weights for policy 0, policy_version 1303331 (0.0010) [2023-12-27 14:40:45,884][20381] Updated weights for policy 1, policy_version 1302439 (0.0006) [2023-12-27 14:40:45,894][20347] Updated weights for policy 0, policy_version 1303341 (0.0010) [2023-12-27 14:40:45,939][20347] Updated weights for policy 0, policy_version 1303351 (0.0010) [2023-12-27 14:40:46,497][20381] Updated weights for policy 1, policy_version 1302449 (0.0006) [2023-12-27 14:40:46,555][20381] Updated weights for policy 1, policy_version 1302459 (0.0005) [2023-12-27 14:40:46,612][20381] Updated weights for policy 1, policy_version 1302469 (0.0009) [2023-12-27 14:40:46,620][20347] Updated weights for policy 0, policy_version 1303361 (0.0010) [2023-12-27 14:40:46,661][20381] Updated weights for policy 1, policy_version 1302479 (0.0010) [2023-12-27 14:40:46,670][20347] Updated weights for policy 0, policy_version 1303371 (0.0010) [2023-12-27 14:40:46,718][20347] Updated weights for policy 0, policy_version 1303381 (0.0010) [2023-12-27 14:40:46,765][20347] Updated weights for policy 0, policy_version 1303391 (0.0010) [2023-12-27 14:40:47,229][20381] Updated weights for policy 1, policy_version 1302489 (0.0009) [2023-12-27 14:40:47,276][20381] Updated weights for policy 1, policy_version 1302499 (0.0010) [2023-12-27 14:40:47,331][20381] Updated weights for policy 1, policy_version 1302509 (0.0005) [2023-12-27 14:40:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 667205632. Throughput: 0: 10121.9, 1: 10439.4. Samples: 667214260. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:47,364][19170] Avg episode reward: [(0, '921.980'), (1, '990.260')] [2023-12-27 14:40:47,443][20347] Updated weights for policy 0, policy_version 1303401 (0.0010) [2023-12-27 14:40:47,503][20347] Updated weights for policy 0, policy_version 1303411 (0.0010) [2023-12-27 14:40:47,566][20347] Updated weights for policy 0, policy_version 1303421 (0.0010) [2023-12-27 14:40:47,957][20381] Updated weights for policy 1, policy_version 1302519 (0.0006) [2023-12-27 14:40:48,024][20381] Updated weights for policy 1, policy_version 1302529 (0.0007) [2023-12-27 14:40:48,071][20381] Updated weights for policy 1, policy_version 1302539 (0.0010) [2023-12-27 14:40:48,231][20347] Updated weights for policy 0, policy_version 1303431 (0.0008) [2023-12-27 14:40:48,284][20347] Updated weights for policy 0, policy_version 1303441 (0.0008) [2023-12-27 14:40:48,341][20347] Updated weights for policy 0, policy_version 1303451 (0.0011) [2023-12-27 14:40:48,713][20381] Updated weights for policy 1, policy_version 1302549 (0.0006) [2023-12-27 14:40:48,771][20381] Updated weights for policy 1, policy_version 1302559 (0.0005) [2023-12-27 14:40:48,829][20381] Updated weights for policy 1, policy_version 1302569 (0.0007) [2023-12-27 14:40:49,028][20347] Updated weights for policy 0, policy_version 1303461 (0.0008) [2023-12-27 14:40:49,092][20347] Updated weights for policy 0, policy_version 1303471 (0.0006) [2023-12-27 14:40:49,145][20347] Updated weights for policy 0, policy_version 1303481 (0.0006) [2023-12-27 14:40:49,456][20381] Updated weights for policy 1, policy_version 1302579 (0.0007) [2023-12-27 14:40:49,507][20381] Updated weights for policy 1, policy_version 1302589 (0.0008) [2023-12-27 14:40:49,555][20381] Updated weights for policy 1, policy_version 1302599 (0.0007) [2023-12-27 14:40:49,827][20347] Updated weights for policy 0, policy_version 1303491 (0.0007) [2023-12-27 14:40:49,888][20347] Updated weights for policy 0, policy_version 1303501 (0.0008) [2023-12-27 14:40:49,954][20347] Updated weights for policy 0, policy_version 1303511 (0.0010) [2023-12-27 14:40:50,215][20381] Updated weights for policy 1, policy_version 1302609 (0.0007) [2023-12-27 14:40:50,270][20381] Updated weights for policy 1, policy_version 1302619 (0.0007) [2023-12-27 14:40:50,325][20381] Updated weights for policy 1, policy_version 1302629 (0.0007) [2023-12-27 14:40:50,383][20381] Updated weights for policy 1, policy_version 1302639 (0.0005) [2023-12-27 14:40:50,669][20347] Updated weights for policy 0, policy_version 1303521 (0.0010) [2023-12-27 14:40:50,732][20347] Updated weights for policy 0, policy_version 1303531 (0.0010) [2023-12-27 14:40:50,786][20347] Updated weights for policy 0, policy_version 1303541 (0.0011) [2023-12-27 14:40:50,842][20347] Updated weights for policy 0, policy_version 1303551 (0.0009) [2023-12-27 14:40:51,054][20381] Updated weights for policy 1, policy_version 1302649 (0.0007) [2023-12-27 14:40:51,111][20381] Updated weights for policy 1, policy_version 1302659 (0.0009) [2023-12-27 14:40:51,175][20381] Updated weights for policy 1, policy_version 1302669 (0.0007) [2023-12-27 14:40:51,572][20347] Updated weights for policy 0, policy_version 1303561 (0.0008) [2023-12-27 14:40:51,634][20347] Updated weights for policy 0, policy_version 1303571 (0.0007) [2023-12-27 14:40:51,688][20347] Updated weights for policy 0, policy_version 1303581 (0.0006) [2023-12-27 14:40:51,989][20381] Updated weights for policy 1, policy_version 1302679 (0.0010) [2023-12-27 14:40:52,048][20381] Updated weights for policy 1, policy_version 1302689 (0.0011) [2023-12-27 14:40:52,107][20381] Updated weights for policy 1, policy_version 1302699 (0.0011) [2023-12-27 14:40:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 667303936. Throughput: 0: 10094.0, 1: 10457.4. Samples: 667277132. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:52,363][19170] Avg episode reward: [(0, '951.260'), (1, '980.500')] [2023-12-27 14:40:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001302704_333520896.pth... [2023-12-27 14:40:52,372][20347] Updated weights for policy 0, policy_version 1303591 (0.0007) [2023-12-27 14:40:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001301496_333209600.pth [2023-12-27 14:40:52,433][20347] Updated weights for policy 0, policy_version 1303601 (0.0007) [2023-12-27 14:40:52,492][20347] Updated weights for policy 0, policy_version 1303611 (0.0005) [2023-12-27 14:40:52,521][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001303616_333791232.pth... [2023-12-27 14:40:52,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001302400_333479936.pth [2023-12-27 14:40:52,806][20381] Updated weights for policy 1, policy_version 1302709 (0.0009) [2023-12-27 14:40:52,865][20381] Updated weights for policy 1, policy_version 1302719 (0.0007) [2023-12-27 14:40:52,916][20381] Updated weights for policy 1, policy_version 1302729 (0.0008) [2023-12-27 14:40:53,127][20347] Updated weights for policy 0, policy_version 1303621 (0.0005) [2023-12-27 14:40:53,182][20347] Updated weights for policy 0, policy_version 1303631 (0.0006) [2023-12-27 14:40:53,230][20347] Updated weights for policy 0, policy_version 1303641 (0.0006) [2023-12-27 14:40:53,618][20381] Updated weights for policy 1, policy_version 1302739 (0.0008) [2023-12-27 14:40:53,671][20381] Updated weights for policy 1, policy_version 1302749 (0.0011) [2023-12-27 14:40:53,727][20381] Updated weights for policy 1, policy_version 1302759 (0.0011) [2023-12-27 14:40:53,888][20347] Updated weights for policy 0, policy_version 1303651 (0.0007) [2023-12-27 14:40:53,948][20347] Updated weights for policy 0, policy_version 1303661 (0.0007) [2023-12-27 14:40:54,008][20347] Updated weights for policy 0, policy_version 1303671 (0.0005) [2023-12-27 14:40:54,441][20381] Updated weights for policy 1, policy_version 1302769 (0.0011) [2023-12-27 14:40:54,501][20381] Updated weights for policy 1, policy_version 1302779 (0.0008) [2023-12-27 14:40:54,558][20381] Updated weights for policy 1, policy_version 1302789 (0.0008) [2023-12-27 14:40:54,615][20381] Updated weights for policy 1, policy_version 1302799 (0.0007) [2023-12-27 14:40:54,645][20347] Updated weights for policy 0, policy_version 1303681 (0.0006) [2023-12-27 14:40:54,692][20347] Updated weights for policy 0, policy_version 1303691 (0.0010) [2023-12-27 14:40:54,743][20347] Updated weights for policy 0, policy_version 1303701 (0.0010) [2023-12-27 14:40:54,797][20347] Updated weights for policy 0, policy_version 1303711 (0.0010) [2023-12-27 14:40:55,281][20381] Updated weights for policy 1, policy_version 1302809 (0.0008) [2023-12-27 14:40:55,332][20381] Updated weights for policy 1, policy_version 1302819 (0.0007) [2023-12-27 14:40:55,397][20381] Updated weights for policy 1, policy_version 1302829 (0.0005) [2023-12-27 14:40:55,484][20347] Updated weights for policy 0, policy_version 1303721 (0.0010) [2023-12-27 14:40:55,540][20347] Updated weights for policy 0, policy_version 1303731 (0.0010) [2023-12-27 14:40:55,603][20347] Updated weights for policy 0, policy_version 1303741 (0.0010) [2023-12-27 14:40:55,975][20381] Updated weights for policy 1, policy_version 1302839 (0.0007) [2023-12-27 14:40:56,022][20381] Updated weights for policy 1, policy_version 1302849 (0.0007) [2023-12-27 14:40:56,068][20381] Updated weights for policy 1, policy_version 1302859 (0.0005) [2023-12-27 14:40:56,273][20347] Updated weights for policy 0, policy_version 1303751 (0.0011) [2023-12-27 14:40:56,321][20347] Updated weights for policy 0, policy_version 1303761 (0.0010) [2023-12-27 14:40:56,366][20347] Updated weights for policy 0, policy_version 1303771 (0.0008) [2023-12-27 14:40:56,688][20381] Updated weights for policy 1, policy_version 1302869 (0.0006) [2023-12-27 14:40:56,747][20381] Updated weights for policy 1, policy_version 1302879 (0.0005) [2023-12-27 14:40:56,808][20381] Updated weights for policy 1, policy_version 1302889 (0.0006) [2023-12-27 14:40:57,048][20347] Updated weights for policy 0, policy_version 1303781 (0.0008) [2023-12-27 14:40:57,111][20347] Updated weights for policy 0, policy_version 1303791 (0.0010) [2023-12-27 14:40:57,160][20347] Updated weights for policy 0, policy_version 1303801 (0.0010) [2023-12-27 14:40:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 667410432. Throughput: 0: 10102.6, 1: 10449.0. Samples: 667399740. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:40:57,363][19170] Avg episode reward: [(0, '931.980'), (1, '1000.000')] [2023-12-27 14:40:57,402][20381] Updated weights for policy 1, policy_version 1302899 (0.0007) [2023-12-27 14:40:57,462][20381] Updated weights for policy 1, policy_version 1302909 (0.0008) [2023-12-27 14:40:57,513][20381] Updated weights for policy 1, policy_version 1302919 (0.0007) [2023-12-27 14:40:57,817][20347] Updated weights for policy 0, policy_version 1303811 (0.0009) [2023-12-27 14:40:57,864][20347] Updated weights for policy 0, policy_version 1303821 (0.0005) [2023-12-27 14:40:57,922][20347] Updated weights for policy 0, policy_version 1303831 (0.0008) [2023-12-27 14:40:58,120][20381] Updated weights for policy 1, policy_version 1302929 (0.0007) [2023-12-27 14:40:58,185][20381] Updated weights for policy 1, policy_version 1302939 (0.0006) [2023-12-27 14:40:58,251][20381] Updated weights for policy 1, policy_version 1302949 (0.0008) [2023-12-27 14:40:58,313][20381] Updated weights for policy 1, policy_version 1302959 (0.0008) [2023-12-27 14:40:58,604][20347] Updated weights for policy 0, policy_version 1303841 (0.0010) [2023-12-27 14:40:58,669][20347] Updated weights for policy 0, policy_version 1303851 (0.0009) [2023-12-27 14:40:58,734][20347] Updated weights for policy 0, policy_version 1303861 (0.0008) [2023-12-27 14:40:58,800][20347] Updated weights for policy 0, policy_version 1303871 (0.0008) [2023-12-27 14:40:59,020][20381] Updated weights for policy 1, policy_version 1302969 (0.0008) [2023-12-27 14:40:59,082][20381] Updated weights for policy 1, policy_version 1302979 (0.0008) [2023-12-27 14:40:59,140][20381] Updated weights for policy 1, policy_version 1302989 (0.0008) [2023-12-27 14:40:59,636][20347] Updated weights for policy 0, policy_version 1303881 (0.0008) [2023-12-27 14:40:59,692][20347] Updated weights for policy 0, policy_version 1303891 (0.0007) [2023-12-27 14:40:59,754][20347] Updated weights for policy 0, policy_version 1303901 (0.0007) [2023-12-27 14:40:59,825][20381] Updated weights for policy 1, policy_version 1302999 (0.0008) [2023-12-27 14:40:59,886][20381] Updated weights for policy 1, policy_version 1303009 (0.0008) [2023-12-27 14:40:59,953][20381] Updated weights for policy 1, policy_version 1303019 (0.0007) [2023-12-27 14:41:00,467][20347] Updated weights for policy 0, policy_version 1303911 (0.0007) [2023-12-27 14:41:00,512][20347] Updated weights for policy 0, policy_version 1303921 (0.0009) [2023-12-27 14:41:00,564][20381] Updated weights for policy 1, policy_version 1303029 (0.0005) [2023-12-27 14:41:00,569][20347] Updated weights for policy 0, policy_version 1303931 (0.0008) [2023-12-27 14:41:00,610][20381] Updated weights for policy 1, policy_version 1303039 (0.0007) [2023-12-27 14:41:00,654][20381] Updated weights for policy 1, policy_version 1303049 (0.0010) [2023-12-27 14:41:01,285][20347] Updated weights for policy 0, policy_version 1303941 (0.0008) [2023-12-27 14:41:01,321][20381] Updated weights for policy 1, policy_version 1303059 (0.0008) [2023-12-27 14:41:01,336][20347] Updated weights for policy 0, policy_version 1303951 (0.0008) [2023-12-27 14:41:01,387][20381] Updated weights for policy 1, policy_version 1303069 (0.0007) [2023-12-27 14:41:01,400][20347] Updated weights for policy 0, policy_version 1303961 (0.0011) [2023-12-27 14:41:01,445][20381] Updated weights for policy 1, policy_version 1303079 (0.0008) [2023-12-27 14:41:02,084][20381] Updated weights for policy 1, policy_version 1303089 (0.0008) [2023-12-27 14:41:02,094][20347] Updated weights for policy 0, policy_version 1303971 (0.0008) [2023-12-27 14:41:02,140][20381] Updated weights for policy 1, policy_version 1303099 (0.0011) [2023-12-27 14:41:02,146][20347] Updated weights for policy 0, policy_version 1303981 (0.0006) [2023-12-27 14:41:02,197][20381] Updated weights for policy 1, policy_version 1303109 (0.0008) [2023-12-27 14:41:02,199][20347] Updated weights for policy 0, policy_version 1303991 (0.0006) [2023-12-27 14:41:02,258][20381] Updated weights for policy 1, policy_version 1303119 (0.0010) [2023-12-27 14:41:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 667516928. Throughput: 0: 10098.3, 1: 10453.4. Samples: 667521712. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:02,363][19170] Avg episode reward: [(0, '902.940'), (1, '1000.000')] [2023-12-27 14:41:02,880][20347] Updated weights for policy 0, policy_version 1304001 (0.0006) [2023-12-27 14:41:02,934][20347] Updated weights for policy 0, policy_version 1304011 (0.0006) [2023-12-27 14:41:02,953][20381] Updated weights for policy 1, policy_version 1303129 (0.0007) [2023-12-27 14:41:02,991][20347] Updated weights for policy 0, policy_version 1304021 (0.0007) [2023-12-27 14:41:03,017][20381] Updated weights for policy 1, policy_version 1303139 (0.0008) [2023-12-27 14:41:03,039][20347] Updated weights for policy 0, policy_version 1304031 (0.0007) [2023-12-27 14:41:03,075][20381] Updated weights for policy 1, policy_version 1303149 (0.0007) [2023-12-27 14:41:03,721][20347] Updated weights for policy 0, policy_version 1304041 (0.0010) [2023-12-27 14:41:03,754][20381] Updated weights for policy 1, policy_version 1303159 (0.0007) [2023-12-27 14:41:03,780][20347] Updated weights for policy 0, policy_version 1304051 (0.0010) [2023-12-27 14:41:03,809][20381] Updated weights for policy 1, policy_version 1303169 (0.0008) [2023-12-27 14:41:03,836][20347] Updated weights for policy 0, policy_version 1304061 (0.0010) [2023-12-27 14:41:03,875][20381] Updated weights for policy 1, policy_version 1303179 (0.0009) [2023-12-27 14:41:04,547][20347] Updated weights for policy 0, policy_version 1304071 (0.0010) [2023-12-27 14:41:04,575][20381] Updated weights for policy 1, policy_version 1303189 (0.0011) [2023-12-27 14:41:04,607][20347] Updated weights for policy 0, policy_version 1304081 (0.0011) [2023-12-27 14:41:04,628][20381] Updated weights for policy 1, policy_version 1303199 (0.0011) [2023-12-27 14:41:04,663][20347] Updated weights for policy 0, policy_version 1304091 (0.0010) [2023-12-27 14:41:04,680][20381] Updated weights for policy 1, policy_version 1303209 (0.0010) [2023-12-27 14:41:05,337][20347] Updated weights for policy 0, policy_version 1304101 (0.0010) [2023-12-27 14:41:05,375][20381] Updated weights for policy 1, policy_version 1303219 (0.0009) [2023-12-27 14:41:05,389][20347] Updated weights for policy 0, policy_version 1304111 (0.0010) [2023-12-27 14:41:05,430][20381] Updated weights for policy 1, policy_version 1303229 (0.0006) [2023-12-27 14:41:05,443][20347] Updated weights for policy 0, policy_version 1304121 (0.0010) [2023-12-27 14:41:05,485][20381] Updated weights for policy 1, policy_version 1303239 (0.0006) [2023-12-27 14:41:06,101][20347] Updated weights for policy 0, policy_version 1304131 (0.0009) [2023-12-27 14:41:06,163][20347] Updated weights for policy 0, policy_version 1304141 (0.0007) [2023-12-27 14:41:06,205][20381] Updated weights for policy 1, policy_version 1303249 (0.0007) [2023-12-27 14:41:06,219][20347] Updated weights for policy 0, policy_version 1304151 (0.0008) [2023-12-27 14:41:06,268][20381] Updated weights for policy 1, policy_version 1303259 (0.0008) [2023-12-27 14:41:06,271][20292] KL-divergence is very high: 529.1870 [2023-12-27 14:41:06,318][20292] KL-divergence is very high: 370.4460 [2023-12-27 14:41:06,323][20381] Updated weights for policy 1, policy_version 1303269 (0.0007) [2023-12-27 14:41:06,387][20381] Updated weights for policy 1, policy_version 1303279 (0.0008) [2023-12-27 14:41:06,904][20347] Updated weights for policy 0, policy_version 1304161 (0.0010) [2023-12-27 14:41:06,961][20347] Updated weights for policy 0, policy_version 1304171 (0.0007) [2023-12-27 14:41:07,015][20347] Updated weights for policy 0, policy_version 1304181 (0.0006) [2023-12-27 14:41:07,075][20347] Updated weights for policy 0, policy_version 1304191 (0.0006) [2023-12-27 14:41:07,088][20381] Updated weights for policy 1, policy_version 1303289 (0.0010) [2023-12-27 14:41:07,160][20381] Updated weights for policy 1, policy_version 1303299 (0.0010) [2023-12-27 14:41:07,219][20381] Updated weights for policy 1, policy_version 1303309 (0.0011) [2023-12-27 14:41:07,363][19170] Fps is (10 sec: 20479.1, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 667615232. Throughput: 0: 10111.2, 1: 10424.8. Samples: 667581564. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:07,364][19170] Avg episode reward: [(0, '941.760'), (1, '990.230')] [2023-12-27 14:41:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001303312_333676544.pth... [2023-12-27 14:41:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001304192_333938688.pth... [2023-12-27 14:41:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001302096_333365248.pth [2023-12-27 14:41:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001303008_333635584.pth [2023-12-27 14:41:07,668][20347] Updated weights for policy 0, policy_version 1304201 (0.0005) [2023-12-27 14:41:07,721][20347] Updated weights for policy 0, policy_version 1304211 (0.0006) [2023-12-27 14:41:07,766][20347] Updated weights for policy 0, policy_version 1304221 (0.0006) [2023-12-27 14:41:07,850][20381] Updated weights for policy 1, policy_version 1303319 (0.0010) [2023-12-27 14:41:07,906][20381] Updated weights for policy 1, policy_version 1303329 (0.0007) [2023-12-27 14:41:07,952][20381] Updated weights for policy 1, policy_version 1303339 (0.0005) [2023-12-27 14:41:08,365][20347] Updated weights for policy 0, policy_version 1304231 (0.0008) [2023-12-27 14:41:08,420][20347] Updated weights for policy 0, policy_version 1304241 (0.0006) [2023-12-27 14:41:08,472][20347] Updated weights for policy 0, policy_version 1304251 (0.0008) [2023-12-27 14:41:08,583][20381] Updated weights for policy 1, policy_version 1303349 (0.0005) [2023-12-27 14:41:08,640][20381] Updated weights for policy 1, policy_version 1303359 (0.0008) [2023-12-27 14:41:08,695][20381] Updated weights for policy 1, policy_version 1303369 (0.0008) [2023-12-27 14:41:09,143][20347] Updated weights for policy 0, policy_version 1304261 (0.0008) [2023-12-27 14:41:09,190][20347] Updated weights for policy 0, policy_version 1304271 (0.0007) [2023-12-27 14:41:09,253][20347] Updated weights for policy 0, policy_version 1304281 (0.0008) [2023-12-27 14:41:09,358][20381] Updated weights for policy 1, policy_version 1303379 (0.0007) [2023-12-27 14:41:09,430][20381] Updated weights for policy 1, policy_version 1303389 (0.0008) [2023-12-27 14:41:09,492][20381] Updated weights for policy 1, policy_version 1303399 (0.0008) [2023-12-27 14:41:10,041][20347] Updated weights for policy 0, policy_version 1304291 (0.0006) [2023-12-27 14:41:10,117][20347] Updated weights for policy 0, policy_version 1304301 (0.0007) [2023-12-27 14:41:10,187][20347] Updated weights for policy 0, policy_version 1304311 (0.0008) [2023-12-27 14:41:10,209][20381] Updated weights for policy 1, policy_version 1303409 (0.0008) [2023-12-27 14:41:10,268][20381] Updated weights for policy 1, policy_version 1303419 (0.0007) [2023-12-27 14:41:10,331][20381] Updated weights for policy 1, policy_version 1303429 (0.0006) [2023-12-27 14:41:10,392][20381] Updated weights for policy 1, policy_version 1303439 (0.0006) [2023-12-27 14:41:10,788][20347] Updated weights for policy 0, policy_version 1304321 (0.0007) [2023-12-27 14:41:10,837][20347] Updated weights for policy 0, policy_version 1304331 (0.0007) [2023-12-27 14:41:10,890][20347] Updated weights for policy 0, policy_version 1304341 (0.0008) [2023-12-27 14:41:10,956][20347] Updated weights for policy 0, policy_version 1304351 (0.0005) [2023-12-27 14:41:11,065][20381] Updated weights for policy 1, policy_version 1303449 (0.0008) [2023-12-27 14:41:11,122][20381] Updated weights for policy 1, policy_version 1303459 (0.0009) [2023-12-27 14:41:11,182][20381] Updated weights for policy 1, policy_version 1303469 (0.0008) [2023-12-27 14:41:11,679][20347] Updated weights for policy 0, policy_version 1304361 (0.0008) [2023-12-27 14:41:11,741][20291] KL-divergence is very high: 110.4370 [2023-12-27 14:41:11,747][20347] Updated weights for policy 0, policy_version 1304371 (0.0008) [2023-12-27 14:41:11,810][20347] Updated weights for policy 0, policy_version 1304381 (0.0007) [2023-12-27 14:41:11,948][20381] Updated weights for policy 1, policy_version 1303479 (0.0008) [2023-12-27 14:41:12,001][20381] Updated weights for policy 1, policy_version 1303489 (0.0008) [2023-12-27 14:41:12,057][20381] Updated weights for policy 1, policy_version 1303499 (0.0008) [2023-12-27 14:41:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 667713536. Throughput: 0: 10140.7, 1: 10359.3. Samples: 667704908. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:12,363][19170] Avg episode reward: [(0, '960.940'), (1, '990.230')] [2023-12-27 14:41:12,480][20347] Updated weights for policy 0, policy_version 1304391 (0.0007) [2023-12-27 14:41:12,543][20347] Updated weights for policy 0, policy_version 1304401 (0.0007) [2023-12-27 14:41:12,610][20347] Updated weights for policy 0, policy_version 1304411 (0.0008) [2023-12-27 14:41:12,827][20381] Updated weights for policy 1, policy_version 1303509 (0.0008) [2023-12-27 14:41:12,888][20381] Updated weights for policy 1, policy_version 1303519 (0.0007) [2023-12-27 14:41:12,942][20381] Updated weights for policy 1, policy_version 1303529 (0.0007) [2023-12-27 14:41:13,290][20347] Updated weights for policy 0, policy_version 1304421 (0.0009) [2023-12-27 14:41:13,326][20291] KL-divergence is very high: 115.0882 [2023-12-27 14:41:13,340][20347] Updated weights for policy 0, policy_version 1304431 (0.0009) [2023-12-27 14:41:13,386][20347] Updated weights for policy 0, policy_version 1304441 (0.0010) [2023-12-27 14:41:13,403][20291] KL-divergence is very high: 191.2869 [2023-12-27 14:41:13,595][20381] Updated weights for policy 1, policy_version 1303539 (0.0008) [2023-12-27 14:41:13,652][20381] Updated weights for policy 1, policy_version 1303549 (0.0005) [2023-12-27 14:41:13,710][20381] Updated weights for policy 1, policy_version 1303559 (0.0005) [2023-12-27 14:41:14,084][20347] Updated weights for policy 0, policy_version 1304451 (0.0010) [2023-12-27 14:41:14,145][20347] Updated weights for policy 0, policy_version 1304461 (0.0010) [2023-12-27 14:41:14,205][20347] Updated weights for policy 0, policy_version 1304471 (0.0010) [2023-12-27 14:41:14,272][20381] Updated weights for policy 1, policy_version 1303569 (0.0005) [2023-12-27 14:41:14,335][20381] Updated weights for policy 1, policy_version 1303579 (0.0005) [2023-12-27 14:41:14,394][20381] Updated weights for policy 1, policy_version 1303589 (0.0005) [2023-12-27 14:41:14,456][20381] Updated weights for policy 1, policy_version 1303599 (0.0006) [2023-12-27 14:41:14,905][20347] Updated weights for policy 0, policy_version 1304481 (0.0010) [2023-12-27 14:41:14,954][20347] Updated weights for policy 0, policy_version 1304491 (0.0009) [2023-12-27 14:41:15,014][20347] Updated weights for policy 0, policy_version 1304501 (0.0011) [2023-12-27 14:41:15,033][20381] Updated weights for policy 1, policy_version 1303609 (0.0007) [2023-12-27 14:41:15,086][20347] Updated weights for policy 0, policy_version 1304511 (0.0008) [2023-12-27 14:41:15,091][20381] Updated weights for policy 1, policy_version 1303619 (0.0008) [2023-12-27 14:41:15,159][20381] Updated weights for policy 1, policy_version 1303629 (0.0008) [2023-12-27 14:41:15,744][20347] Updated weights for policy 0, policy_version 1304521 (0.0010) [2023-12-27 14:41:15,801][20347] Updated weights for policy 0, policy_version 1304531 (0.0010) [2023-12-27 14:41:15,843][20381] Updated weights for policy 1, policy_version 1303639 (0.0006) [2023-12-27 14:41:15,854][20347] Updated weights for policy 0, policy_version 1304541 (0.0010) [2023-12-27 14:41:15,892][20381] Updated weights for policy 1, policy_version 1303649 (0.0009) [2023-12-27 14:41:15,942][20381] Updated weights for policy 1, policy_version 1303659 (0.0007) [2023-12-27 14:41:16,517][20347] Updated weights for policy 0, policy_version 1304551 (0.0010) [2023-12-27 14:41:16,577][20347] Updated weights for policy 0, policy_version 1304561 (0.0010) [2023-12-27 14:41:16,635][20347] Updated weights for policy 0, policy_version 1304571 (0.0010) [2023-12-27 14:41:16,641][20381] Updated weights for policy 1, policy_version 1303669 (0.0006) [2023-12-27 14:41:16,690][20381] Updated weights for policy 1, policy_version 1303679 (0.0006) [2023-12-27 14:41:16,740][20381] Updated weights for policy 1, policy_version 1303689 (0.0008) [2023-12-27 14:41:17,282][20347] Updated weights for policy 0, policy_version 1304581 (0.0010) [2023-12-27 14:41:17,336][20347] Updated weights for policy 0, policy_version 1304591 (0.0010) [2023-12-27 14:41:17,362][19170] Fps is (10 sec: 19661.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 667811840. Throughput: 0: 10114.0, 1: 10372.8. Samples: 667827768. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:17,363][19170] Avg episode reward: [(0, '902.240'), (1, '1000.000')] [2023-12-27 14:41:17,391][20347] Updated weights for policy 0, policy_version 1304601 (0.0010) [2023-12-27 14:41:17,450][20381] Updated weights for policy 1, policy_version 1303699 (0.0007) [2023-12-27 14:41:17,507][20381] Updated weights for policy 1, policy_version 1303709 (0.0008) [2023-12-27 14:41:17,557][20381] Updated weights for policy 1, policy_version 1303719 (0.0007) [2023-12-27 14:41:18,053][20347] Updated weights for policy 0, policy_version 1304611 (0.0009) [2023-12-27 14:41:18,098][20347] Updated weights for policy 0, policy_version 1304621 (0.0005) [2023-12-27 14:41:18,143][20347] Updated weights for policy 0, policy_version 1304631 (0.0005) [2023-12-27 14:41:18,272][20381] Updated weights for policy 1, policy_version 1303729 (0.0008) [2023-12-27 14:41:18,318][20381] Updated weights for policy 1, policy_version 1303739 (0.0010) [2023-12-27 14:41:18,381][20381] Updated weights for policy 1, policy_version 1303749 (0.0009) [2023-12-27 14:41:18,445][20381] Updated weights for policy 1, policy_version 1303759 (0.0008) [2023-12-27 14:41:18,760][20347] Updated weights for policy 0, policy_version 1304641 (0.0006) [2023-12-27 14:41:18,818][20347] Updated weights for policy 0, policy_version 1304651 (0.0010) [2023-12-27 14:41:18,882][20347] Updated weights for policy 0, policy_version 1304661 (0.0010) [2023-12-27 14:41:18,937][20347] Updated weights for policy 0, policy_version 1304671 (0.0010) [2023-12-27 14:41:19,153][20381] Updated weights for policy 1, policy_version 1303769 (0.0007) [2023-12-27 14:41:19,201][20381] Updated weights for policy 1, policy_version 1303779 (0.0007) [2023-12-27 14:41:19,266][20381] Updated weights for policy 1, policy_version 1303789 (0.0008) [2023-12-27 14:41:19,623][20347] Updated weights for policy 0, policy_version 1304681 (0.0010) [2023-12-27 14:41:19,682][20347] Updated weights for policy 0, policy_version 1304691 (0.0010) [2023-12-27 14:41:19,739][20347] Updated weights for policy 0, policy_version 1304701 (0.0011) [2023-12-27 14:41:19,972][20381] Updated weights for policy 1, policy_version 1303799 (0.0007) [2023-12-27 14:41:20,035][20381] Updated weights for policy 1, policy_version 1303809 (0.0006) [2023-12-27 14:41:20,105][20381] Updated weights for policy 1, policy_version 1303819 (0.0006) [2023-12-27 14:41:20,449][20347] Updated weights for policy 0, policy_version 1304711 (0.0008) [2023-12-27 14:41:20,508][20347] Updated weights for policy 0, policy_version 1304721 (0.0007) [2023-12-27 14:41:20,567][20347] Updated weights for policy 0, policy_version 1304731 (0.0008) [2023-12-27 14:41:20,733][20381] Updated weights for policy 1, policy_version 1303829 (0.0006) [2023-12-27 14:41:20,800][20381] Updated weights for policy 1, policy_version 1303839 (0.0006) [2023-12-27 14:41:20,869][20381] Updated weights for policy 1, policy_version 1303849 (0.0008) [2023-12-27 14:41:21,250][20347] Updated weights for policy 0, policy_version 1304741 (0.0008) [2023-12-27 14:41:21,316][20347] Updated weights for policy 0, policy_version 1304751 (0.0006) [2023-12-27 14:41:21,389][20347] Updated weights for policy 0, policy_version 1304761 (0.0009) [2023-12-27 14:41:21,558][20381] Updated weights for policy 1, policy_version 1303859 (0.0008) [2023-12-27 14:41:21,626][20381] Updated weights for policy 1, policy_version 1303869 (0.0007) [2023-12-27 14:41:21,690][20381] Updated weights for policy 1, policy_version 1303879 (0.0007) [2023-12-27 14:41:22,117][20347] Updated weights for policy 0, policy_version 1304771 (0.0007) [2023-12-27 14:41:22,167][20347] Updated weights for policy 0, policy_version 1304781 (0.0005) [2023-12-27 14:41:22,223][20347] Updated weights for policy 0, policy_version 1304791 (0.0005) [2023-12-27 14:41:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 667918336. Throughput: 0: 10141.4, 1: 10293.2. Samples: 667888252. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:22,363][19170] Avg episode reward: [(0, '941.350'), (1, '980.430')] [2023-12-27 14:41:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001303888_333824000.pth... [2023-12-27 14:41:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001304800_334094336.pth... [2023-12-27 14:41:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001303616_333791232.pth [2023-12-27 14:41:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001302704_333520896.pth [2023-12-27 14:41:22,397][20381] Updated weights for policy 1, policy_version 1303889 (0.0006) [2023-12-27 14:41:22,456][20381] Updated weights for policy 1, policy_version 1303899 (0.0008) [2023-12-27 14:41:22,513][20381] Updated weights for policy 1, policy_version 1303909 (0.0007) [2023-12-27 14:41:22,569][20381] Updated weights for policy 1, policy_version 1303919 (0.0008) [2023-12-27 14:41:22,900][20347] Updated weights for policy 0, policy_version 1304801 (0.0006) [2023-12-27 14:41:22,950][20347] Updated weights for policy 0, policy_version 1304811 (0.0008) [2023-12-27 14:41:22,996][20347] Updated weights for policy 0, policy_version 1304821 (0.0007) [2023-12-27 14:41:23,045][20347] Updated weights for policy 0, policy_version 1304831 (0.0008) [2023-12-27 14:41:23,245][20381] Updated weights for policy 1, policy_version 1303929 (0.0006) [2023-12-27 14:41:23,294][20381] Updated weights for policy 1, policy_version 1303939 (0.0008) [2023-12-27 14:41:23,339][20381] Updated weights for policy 1, policy_version 1303949 (0.0009) [2023-12-27 14:41:23,744][20347] Updated weights for policy 0, policy_version 1304841 (0.0010) [2023-12-27 14:41:23,806][20347] Updated weights for policy 0, policy_version 1304851 (0.0010) [2023-12-27 14:41:23,858][20347] Updated weights for policy 0, policy_version 1304861 (0.0010) [2023-12-27 14:41:24,028][20381] Updated weights for policy 1, policy_version 1303959 (0.0007) [2023-12-27 14:41:24,074][20381] Updated weights for policy 1, policy_version 1303969 (0.0007) [2023-12-27 14:41:24,118][20381] Updated weights for policy 1, policy_version 1303979 (0.0007) [2023-12-27 14:41:24,490][20347] Updated weights for policy 0, policy_version 1304871 (0.0007) [2023-12-27 14:41:24,536][20347] Updated weights for policy 0, policy_version 1304881 (0.0005) [2023-12-27 14:41:24,583][20347] Updated weights for policy 0, policy_version 1304891 (0.0005) [2023-12-27 14:41:24,879][20381] Updated weights for policy 1, policy_version 1303989 (0.0008) [2023-12-27 14:41:24,927][20381] Updated weights for policy 1, policy_version 1303999 (0.0007) [2023-12-27 14:41:24,979][20381] Updated weights for policy 1, policy_version 1304009 (0.0008) [2023-12-27 14:41:25,132][20347] Updated weights for policy 0, policy_version 1304901 (0.0007) [2023-12-27 14:41:25,183][20347] Updated weights for policy 0, policy_version 1304911 (0.0007) [2023-12-27 14:41:25,245][20347] Updated weights for policy 0, policy_version 1304921 (0.0005) [2023-12-27 14:41:25,712][20381] Updated weights for policy 1, policy_version 1304019 (0.0008) [2023-12-27 14:41:25,767][20381] Updated weights for policy 1, policy_version 1304029 (0.0007) [2023-12-27 14:41:25,821][20381] Updated weights for policy 1, policy_version 1304039 (0.0007) [2023-12-27 14:41:25,901][20347] Updated weights for policy 0, policy_version 1304931 (0.0006) [2023-12-27 14:41:25,954][20347] Updated weights for policy 0, policy_version 1304941 (0.0008) [2023-12-27 14:41:25,998][20291] KL-divergence is very high: 108.4682 [2023-12-27 14:41:26,008][20347] Updated weights for policy 0, policy_version 1304951 (0.0007) [2023-12-27 14:41:26,025][20291] KL-divergence is very high: 185.4465 [2023-12-27 14:41:26,043][20291] KL-divergence is very high: 240.1771 [2023-12-27 14:41:26,509][20381] Updated weights for policy 1, policy_version 1304049 (0.0008) [2023-12-27 14:41:26,563][20381] Updated weights for policy 1, policy_version 1304059 (0.0008) [2023-12-27 14:41:26,610][20381] Updated weights for policy 1, policy_version 1304069 (0.0007) [2023-12-27 14:41:26,661][20381] Updated weights for policy 1, policy_version 1304079 (0.0008) [2023-12-27 14:41:26,710][20347] Updated weights for policy 0, policy_version 1304961 (0.0007) [2023-12-27 14:41:26,768][20347] Updated weights for policy 0, policy_version 1304971 (0.0007) [2023-12-27 14:41:26,823][20347] Updated weights for policy 0, policy_version 1304981 (0.0007) [2023-12-27 14:41:26,876][20347] Updated weights for policy 0, policy_version 1304991 (0.0007) [2023-12-27 14:41:27,350][20292] KL-divergence is very high: 363.4373 [2023-12-27 14:41:27,355][20381] Updated weights for policy 1, policy_version 1304089 (0.0008) [2023-12-27 14:41:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 668016640. Throughput: 0: 10196.2, 1: 10249.0. Samples: 668010700. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:27,363][19170] Avg episode reward: [(0, '961.010'), (1, '960.890')] [2023-12-27 14:41:27,397][20292] KL-divergence is very high: 272.5308 [2023-12-27 14:41:27,416][20381] Updated weights for policy 1, policy_version 1304099 (0.0007) [2023-12-27 14:41:27,440][20292] KL-divergence is very high: 107.6334 [2023-12-27 14:41:27,465][20381] Updated weights for policy 1, policy_version 1304109 (0.0008) [2023-12-27 14:41:27,582][20347] Updated weights for policy 0, policy_version 1305001 (0.0008) [2023-12-27 14:41:27,637][20347] Updated weights for policy 0, policy_version 1305011 (0.0008) [2023-12-27 14:41:27,700][20347] Updated weights for policy 0, policy_version 1305021 (0.0008) [2023-12-27 14:41:28,095][20381] Updated weights for policy 1, policy_version 1304119 (0.0008) [2023-12-27 14:41:28,138][20381] Updated weights for policy 1, policy_version 1304129 (0.0006) [2023-12-27 14:41:28,183][20381] Updated weights for policy 1, policy_version 1304139 (0.0005) [2023-12-27 14:41:28,370][20347] Updated weights for policy 0, policy_version 1305031 (0.0007) [2023-12-27 14:41:28,429][20347] Updated weights for policy 0, policy_version 1305041 (0.0008) [2023-12-27 14:41:28,488][20347] Updated weights for policy 0, policy_version 1305051 (0.0008) [2023-12-27 14:41:28,863][20381] Updated weights for policy 1, policy_version 1304149 (0.0006) [2023-12-27 14:41:28,919][20381] Updated weights for policy 1, policy_version 1304159 (0.0009) [2023-12-27 14:41:28,969][20381] Updated weights for policy 1, policy_version 1304169 (0.0008) [2023-12-27 14:41:29,162][20347] Updated weights for policy 0, policy_version 1305061 (0.0007) [2023-12-27 14:41:29,205][20347] Updated weights for policy 0, policy_version 1305071 (0.0005) [2023-12-27 14:41:29,273][20347] Updated weights for policy 0, policy_version 1305081 (0.0008) [2023-12-27 14:41:29,699][20381] Updated weights for policy 1, policy_version 1304179 (0.0007) [2023-12-27 14:41:29,758][20381] Updated weights for policy 1, policy_version 1304189 (0.0008) [2023-12-27 14:41:29,832][20381] Updated weights for policy 1, policy_version 1304199 (0.0007) [2023-12-27 14:41:29,985][20347] Updated weights for policy 0, policy_version 1305091 (0.0008) [2023-12-27 14:41:30,044][20347] Updated weights for policy 0, policy_version 1305101 (0.0008) [2023-12-27 14:41:30,106][20347] Updated weights for policy 0, policy_version 1305111 (0.0008) [2023-12-27 14:41:30,467][20381] Updated weights for policy 1, policy_version 1304209 (0.0007) [2023-12-27 14:41:30,519][20381] Updated weights for policy 1, policy_version 1304219 (0.0010) [2023-12-27 14:41:30,563][20381] Updated weights for policy 1, policy_version 1304229 (0.0010) [2023-12-27 14:41:30,613][20381] Updated weights for policy 1, policy_version 1304239 (0.0011) [2023-12-27 14:41:30,827][20347] Updated weights for policy 0, policy_version 1305121 (0.0008) [2023-12-27 14:41:30,877][20347] Updated weights for policy 0, policy_version 1305131 (0.0009) [2023-12-27 14:41:30,925][20347] Updated weights for policy 0, policy_version 1305141 (0.0007) [2023-12-27 14:41:30,972][20347] Updated weights for policy 0, policy_version 1305151 (0.0007) [2023-12-27 14:41:31,316][20381] Updated weights for policy 1, policy_version 1304249 (0.0011) [2023-12-27 14:41:31,383][20381] Updated weights for policy 1, policy_version 1304259 (0.0011) [2023-12-27 14:41:31,438][20381] Updated weights for policy 1, policy_version 1304269 (0.0008) [2023-12-27 14:41:31,688][20347] Updated weights for policy 0, policy_version 1305161 (0.0010) [2023-12-27 14:41:31,756][20347] Updated weights for policy 0, policy_version 1305171 (0.0010) [2023-12-27 14:41:31,824][20347] Updated weights for policy 0, policy_version 1305181 (0.0006) [2023-12-27 14:41:32,142][20381] Updated weights for policy 1, policy_version 1304279 (0.0009) [2023-12-27 14:41:32,193][20381] Updated weights for policy 1, policy_version 1304289 (0.0010) [2023-12-27 14:41:32,243][20381] Updated weights for policy 1, policy_version 1304299 (0.0010) [2023-12-27 14:41:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 668123136. Throughput: 0: 10187.4, 1: 10218.3. Samples: 668132512. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:32,363][19170] Avg episode reward: [(0, '892.780'), (1, '970.720')] [2023-12-27 14:41:32,464][20347] Updated weights for policy 0, policy_version 1305191 (0.0006) [2023-12-27 14:41:32,508][20347] Updated weights for policy 0, policy_version 1305201 (0.0008) [2023-12-27 14:41:32,559][20347] Updated weights for policy 0, policy_version 1305211 (0.0008) [2023-12-27 14:41:32,892][20381] Updated weights for policy 1, policy_version 1304309 (0.0009) [2023-12-27 14:41:32,948][20381] Updated weights for policy 1, policy_version 1304319 (0.0007) [2023-12-27 14:41:32,999][20381] Updated weights for policy 1, policy_version 1304329 (0.0007) [2023-12-27 14:41:33,243][20347] Updated weights for policy 0, policy_version 1305221 (0.0006) [2023-12-27 14:41:33,293][20347] Updated weights for policy 0, policy_version 1305231 (0.0005) [2023-12-27 14:41:33,339][20347] Updated weights for policy 0, policy_version 1305241 (0.0007) [2023-12-27 14:41:33,639][20381] Updated weights for policy 1, policy_version 1304339 (0.0007) [2023-12-27 14:41:33,693][20381] Updated weights for policy 1, policy_version 1304349 (0.0005) [2023-12-27 14:41:33,749][20381] Updated weights for policy 1, policy_version 1304359 (0.0005) [2023-12-27 14:41:33,996][20347] Updated weights for policy 0, policy_version 1305251 (0.0007) [2023-12-27 14:41:34,026][20291] KL-divergence is very high: 888.5840 [2023-12-27 14:41:34,031][20291] KL-divergence is very high: 838.4318 [2023-12-27 14:41:34,048][20347] Updated weights for policy 0, policy_version 1305261 (0.0008) [2023-12-27 14:41:34,074][20291] KL-divergence is very high: 151.5441 [2023-12-27 14:41:34,079][20291] KL-divergence is very high: 108.0954 [2023-12-27 14:41:34,101][20347] Updated weights for policy 0, policy_version 1305271 (0.0008) [2023-12-27 14:41:34,304][20381] Updated weights for policy 1, policy_version 1304369 (0.0005) [2023-12-27 14:41:34,350][20381] Updated weights for policy 1, policy_version 1304379 (0.0008) [2023-12-27 14:41:34,402][20381] Updated weights for policy 1, policy_version 1304389 (0.0008) [2023-12-27 14:41:34,460][20381] Updated weights for policy 1, policy_version 1304399 (0.0007) [2023-12-27 14:41:34,830][20347] Updated weights for policy 0, policy_version 1305281 (0.0006) [2023-12-27 14:41:34,882][20347] Updated weights for policy 0, policy_version 1305291 (0.0007) [2023-12-27 14:41:34,941][20347] Updated weights for policy 0, policy_version 1305301 (0.0007) [2023-12-27 14:41:34,996][20347] Updated weights for policy 0, policy_version 1305311 (0.0008) [2023-12-27 14:41:35,180][20381] Updated weights for policy 1, policy_version 1304409 (0.0008) [2023-12-27 14:41:35,236][20381] Updated weights for policy 1, policy_version 1304419 (0.0008) [2023-12-27 14:41:35,290][20381] Updated weights for policy 1, policy_version 1304429 (0.0007) [2023-12-27 14:41:35,685][20347] Updated weights for policy 0, policy_version 1305321 (0.0007) [2023-12-27 14:41:35,734][20347] Updated weights for policy 0, policy_version 1305331 (0.0008) [2023-12-27 14:41:35,792][20347] Updated weights for policy 0, policy_version 1305341 (0.0006) [2023-12-27 14:41:35,972][20381] Updated weights for policy 1, policy_version 1304439 (0.0010) [2023-12-27 14:41:36,017][20381] Updated weights for policy 1, policy_version 1304449 (0.0007) [2023-12-27 14:41:36,067][20381] Updated weights for policy 1, policy_version 1304459 (0.0008) [2023-12-27 14:41:36,502][20347] Updated weights for policy 0, policy_version 1305351 (0.0008) [2023-12-27 14:41:36,527][20291] KL-divergence is very high: 365.3916 [2023-12-27 14:41:36,534][20291] KL-divergence is very high: 331.5792 [2023-12-27 14:41:36,563][20347] Updated weights for policy 0, policy_version 1305361 (0.0008) [2023-12-27 14:41:36,572][20291] KL-divergence is very high: 168.8110 [2023-12-27 14:41:36,577][20291] KL-divergence is very high: 118.4556 [2023-12-27 14:41:36,617][20347] Updated weights for policy 0, policy_version 1305371 (0.0007) [2023-12-27 14:41:36,775][20381] Updated weights for policy 1, policy_version 1304469 (0.0006) [2023-12-27 14:41:36,827][20381] Updated weights for policy 1, policy_version 1304479 (0.0005) [2023-12-27 14:41:36,885][20381] Updated weights for policy 1, policy_version 1304489 (0.0007) [2023-12-27 14:41:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.6, 300 sec: 20549.4). Total num frames: 668221440. Throughput: 0: 10194.0, 1: 10212.4. Samples: 668195416. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:37,363][19170] Avg episode reward: [(0, '863.460'), (1, '931.630')] [2023-12-27 14:41:37,364][20347] Updated weights for policy 0, policy_version 1305381 (0.0007) [2023-12-27 14:41:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001304496_333979648.pth... [2023-12-27 14:41:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001303312_333676544.pth [2023-12-27 14:41:37,428][20347] Updated weights for policy 0, policy_version 1305391 (0.0007) [2023-12-27 14:41:37,482][20381] Updated weights for policy 1, policy_version 1304499 (0.0009) [2023-12-27 14:41:37,491][20347] Updated weights for policy 0, policy_version 1305401 (0.0007) [2023-12-27 14:41:37,528][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001305408_334249984.pth... [2023-12-27 14:41:37,531][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001304192_333938688.pth [2023-12-27 14:41:37,539][20381] Updated weights for policy 1, policy_version 1304509 (0.0009) [2023-12-27 14:41:37,589][20381] Updated weights for policy 1, policy_version 1304519 (0.0005) [2023-12-27 14:41:38,125][20381] Updated weights for policy 1, policy_version 1304529 (0.0007) [2023-12-27 14:41:38,173][20381] Updated weights for policy 1, policy_version 1304539 (0.0006) [2023-12-27 14:41:38,209][20347] Updated weights for policy 0, policy_version 1305411 (0.0006) [2023-12-27 14:41:38,223][20381] Updated weights for policy 1, policy_version 1304549 (0.0010) [2023-12-27 14:41:38,261][20347] Updated weights for policy 0, policy_version 1305421 (0.0006) [2023-12-27 14:41:38,277][20381] Updated weights for policy 1, policy_version 1304559 (0.0010) [2023-12-27 14:41:38,317][20347] Updated weights for policy 0, policy_version 1305431 (0.0007) [2023-12-27 14:41:38,975][20381] Updated weights for policy 1, policy_version 1304569 (0.0011) [2023-12-27 14:41:39,022][20347] Updated weights for policy 0, policy_version 1305441 (0.0008) [2023-12-27 14:41:39,034][20381] Updated weights for policy 1, policy_version 1304579 (0.0011) [2023-12-27 14:41:39,078][20347] Updated weights for policy 0, policy_version 1305451 (0.0005) [2023-12-27 14:41:39,095][20381] Updated weights for policy 1, policy_version 1304589 (0.0010) [2023-12-27 14:41:39,131][20347] Updated weights for policy 0, policy_version 1305461 (0.0005) [2023-12-27 14:41:39,177][20347] Updated weights for policy 0, policy_version 1305471 (0.0005) [2023-12-27 14:41:39,826][20347] Updated weights for policy 0, policy_version 1305481 (0.0007) [2023-12-27 14:41:39,859][20381] Updated weights for policy 1, policy_version 1304599 (0.0010) [2023-12-27 14:41:39,892][20347] Updated weights for policy 0, policy_version 1305491 (0.0008) [2023-12-27 14:41:39,917][20381] Updated weights for policy 1, policy_version 1304609 (0.0007) [2023-12-27 14:41:39,959][20347] Updated weights for policy 0, policy_version 1305501 (0.0008) [2023-12-27 14:41:39,982][20381] Updated weights for policy 1, policy_version 1304619 (0.0007) [2023-12-27 14:41:40,689][20347] Updated weights for policy 0, policy_version 1305511 (0.0008) [2023-12-27 14:41:40,724][20381] Updated weights for policy 1, policy_version 1304629 (0.0008) [2023-12-27 14:41:40,749][20347] Updated weights for policy 0, policy_version 1305521 (0.0007) [2023-12-27 14:41:40,784][20381] Updated weights for policy 1, policy_version 1304639 (0.0006) [2023-12-27 14:41:40,802][20347] Updated weights for policy 0, policy_version 1305531 (0.0008) [2023-12-27 14:41:40,850][20381] Updated weights for policy 1, policy_version 1304649 (0.0008) [2023-12-27 14:41:41,548][20347] Updated weights for policy 0, policy_version 1305541 (0.0009) [2023-12-27 14:41:41,598][20381] Updated weights for policy 1, policy_version 1304659 (0.0009) [2023-12-27 14:41:41,609][20347] Updated weights for policy 0, policy_version 1305551 (0.0011) [2023-12-27 14:41:41,672][20347] Updated weights for policy 0, policy_version 1305561 (0.0008) [2023-12-27 14:41:41,680][20381] Updated weights for policy 1, policy_version 1304669 (0.0008) [2023-12-27 14:41:41,745][20381] Updated weights for policy 1, policy_version 1304679 (0.0008) [2023-12-27 14:41:42,352][20347] Updated weights for policy 0, policy_version 1305571 (0.0007) [2023-12-27 14:41:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 668319744. Throughput: 0: 10141.2, 1: 10216.0. Samples: 668315812. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:42,363][19170] Avg episode reward: [(0, '882.970'), (1, '911.970')] [2023-12-27 14:41:42,415][20347] Updated weights for policy 0, policy_version 1305581 (0.0008) [2023-12-27 14:41:42,451][20381] Updated weights for policy 1, policy_version 1304689 (0.0008) [2023-12-27 14:41:42,482][20347] Updated weights for policy 0, policy_version 1305591 (0.0008) [2023-12-27 14:41:42,504][20381] Updated weights for policy 1, policy_version 1304699 (0.0010) [2023-12-27 14:41:42,558][20381] Updated weights for policy 1, policy_version 1304709 (0.0010) [2023-12-27 14:41:42,604][20381] Updated weights for policy 1, policy_version 1304719 (0.0010) [2023-12-27 14:41:43,211][20347] Updated weights for policy 0, policy_version 1305601 (0.0007) [2023-12-27 14:41:43,276][20347] Updated weights for policy 0, policy_version 1305611 (0.0007) [2023-12-27 14:41:43,300][20381] Updated weights for policy 1, policy_version 1304729 (0.0010) [2023-12-27 14:41:43,332][20347] Updated weights for policy 0, policy_version 1305621 (0.0006) [2023-12-27 14:41:43,349][20381] Updated weights for policy 1, policy_version 1304739 (0.0010) [2023-12-27 14:41:43,388][20347] Updated weights for policy 0, policy_version 1305631 (0.0005) [2023-12-27 14:41:43,400][20381] Updated weights for policy 1, policy_version 1304749 (0.0010) [2023-12-27 14:41:44,071][20347] Updated weights for policy 0, policy_version 1305641 (0.0006) [2023-12-27 14:41:44,081][20381] Updated weights for policy 1, policy_version 1304759 (0.0008) [2023-12-27 14:41:44,131][20347] Updated weights for policy 0, policy_version 1305651 (0.0007) [2023-12-27 14:41:44,143][20381] Updated weights for policy 1, policy_version 1304769 (0.0005) [2023-12-27 14:41:44,190][20347] Updated weights for policy 0, policy_version 1305661 (0.0006) [2023-12-27 14:41:44,196][20381] Updated weights for policy 1, policy_version 1304779 (0.0005) [2023-12-27 14:41:44,767][20347] Updated weights for policy 0, policy_version 1305671 (0.0007) [2023-12-27 14:41:44,813][20381] Updated weights for policy 1, policy_version 1304789 (0.0008) [2023-12-27 14:41:44,833][20347] Updated weights for policy 0, policy_version 1305681 (0.0006) [2023-12-27 14:41:44,878][20381] Updated weights for policy 1, policy_version 1304799 (0.0010) [2023-12-27 14:41:44,894][20347] Updated weights for policy 0, policy_version 1305691 (0.0006) [2023-12-27 14:41:44,938][20381] Updated weights for policy 1, policy_version 1304809 (0.0011) [2023-12-27 14:41:45,509][20347] Updated weights for policy 0, policy_version 1305701 (0.0005) [2023-12-27 14:41:45,571][20347] Updated weights for policy 0, policy_version 1305711 (0.0006) [2023-12-27 14:41:45,581][20381] Updated weights for policy 1, policy_version 1304819 (0.0010) [2023-12-27 14:41:45,627][20347] Updated weights for policy 0, policy_version 1305721 (0.0005) [2023-12-27 14:41:45,639][20381] Updated weights for policy 1, policy_version 1304829 (0.0010) [2023-12-27 14:41:45,696][20381] Updated weights for policy 1, policy_version 1304839 (0.0006) [2023-12-27 14:41:46,228][20347] Updated weights for policy 0, policy_version 1305731 (0.0008) [2023-12-27 14:41:46,254][20381] Updated weights for policy 1, policy_version 1304849 (0.0006) [2023-12-27 14:41:46,283][20347] Updated weights for policy 0, policy_version 1305741 (0.0010) [2023-12-27 14:41:46,316][20381] Updated weights for policy 1, policy_version 1304859 (0.0005) [2023-12-27 14:41:46,329][20347] Updated weights for policy 0, policy_version 1305751 (0.0008) [2023-12-27 14:41:46,370][20381] Updated weights for policy 1, policy_version 1304869 (0.0009) [2023-12-27 14:41:46,415][20381] Updated weights for policy 1, policy_version 1304879 (0.0010) [2023-12-27 14:41:46,964][20347] Updated weights for policy 0, policy_version 1305761 (0.0005) [2023-12-27 14:41:47,032][20347] Updated weights for policy 0, policy_version 1305771 (0.0006) [2023-12-27 14:41:47,071][20381] Updated weights for policy 1, policy_version 1304889 (0.0006) [2023-12-27 14:41:47,081][20347] Updated weights for policy 0, policy_version 1305781 (0.0010) [2023-12-27 14:41:47,122][20381] Updated weights for policy 1, policy_version 1304899 (0.0005) [2023-12-27 14:41:47,135][20347] Updated weights for policy 0, policy_version 1305791 (0.0006) [2023-12-27 14:41:47,179][20381] Updated weights for policy 1, policy_version 1304909 (0.0010) [2023-12-27 14:41:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 668434432. Throughput: 0: 10222.1, 1: 10211.2. Samples: 668441208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) [2023-12-27 14:41:47,363][19170] Avg episode reward: [(0, '941.580'), (1, '941.270')] [2023-12-27 14:41:47,714][20347] Updated weights for policy 0, policy_version 1305801 (0.0010) [2023-12-27 14:41:47,774][20347] Updated weights for policy 0, policy_version 1305811 (0.0008) [2023-12-27 14:41:47,832][20347] Updated weights for policy 0, policy_version 1305821 (0.0005) [2023-12-27 14:41:47,835][20381] Updated weights for policy 1, policy_version 1304919 (0.0010) [2023-12-27 14:41:47,893][20381] Updated weights for policy 1, policy_version 1304929 (0.0010) [2023-12-27 14:41:47,946][20381] Updated weights for policy 1, policy_version 1304939 (0.0009) [2023-12-27 14:41:48,462][20347] Updated weights for policy 0, policy_version 1305831 (0.0009) [2023-12-27 14:41:48,513][20347] Updated weights for policy 0, policy_version 1305841 (0.0010) [2023-12-27 14:41:48,568][20291] KL-divergence is very high: 343.5827 [2023-12-27 14:41:48,581][20347] Updated weights for policy 0, policy_version 1305851 (0.0010) [2023-12-27 14:41:48,631][20381] Updated weights for policy 1, policy_version 1304949 (0.0005) [2023-12-27 14:41:48,694][20381] Updated weights for policy 1, policy_version 1304959 (0.0007) [2023-12-27 14:41:48,756][20381] Updated weights for policy 1, policy_version 1304969 (0.0006) [2023-12-27 14:41:49,253][20347] Updated weights for policy 0, policy_version 1305861 (0.0009) [2023-12-27 14:41:49,314][20347] Updated weights for policy 0, policy_version 1305871 (0.0007) [2023-12-27 14:41:49,343][20381] Updated weights for policy 1, policy_version 1304979 (0.0007) [2023-12-27 14:41:49,380][20347] Updated weights for policy 0, policy_version 1305881 (0.0008) [2023-12-27 14:41:49,408][20381] Updated weights for policy 1, policy_version 1304989 (0.0009) [2023-12-27 14:41:49,464][20381] Updated weights for policy 1, policy_version 1304999 (0.0008) [2023-12-27 14:41:50,066][20347] Updated weights for policy 0, policy_version 1305891 (0.0007) [2023-12-27 14:41:50,116][20381] Updated weights for policy 1, policy_version 1305009 (0.0005) [2023-12-27 14:41:50,124][20347] Updated weights for policy 0, policy_version 1305901 (0.0008) [2023-12-27 14:41:50,166][20381] Updated weights for policy 1, policy_version 1305019 (0.0006) [2023-12-27 14:41:50,186][20347] Updated weights for policy 0, policy_version 1305911 (0.0007) [2023-12-27 14:41:50,221][20381] Updated weights for policy 1, policy_version 1305029 (0.0007) [2023-12-27 14:41:50,269][20381] Updated weights for policy 1, policy_version 1305039 (0.0007) [2023-12-27 14:41:50,865][20347] Updated weights for policy 0, policy_version 1305921 (0.0007) [2023-12-27 14:41:50,934][20347] Updated weights for policy 0, policy_version 1305931 (0.0005) [2023-12-27 14:41:50,976][20381] Updated weights for policy 1, policy_version 1305049 (0.0008) [2023-12-27 14:41:50,998][20347] Updated weights for policy 0, policy_version 1305941 (0.0006) [2023-12-27 14:41:51,043][20381] Updated weights for policy 1, policy_version 1305059 (0.0007) [2023-12-27 14:41:51,065][20347] Updated weights for policy 0, policy_version 1305951 (0.0007) [2023-12-27 14:41:51,104][20381] Updated weights for policy 1, policy_version 1305069 (0.0008) [2023-12-27 14:41:51,680][20347] Updated weights for policy 0, policy_version 1305961 (0.0006) [2023-12-27 14:41:51,745][20347] Updated weights for policy 0, policy_version 1305971 (0.0006) [2023-12-27 14:41:51,806][20347] Updated weights for policy 0, policy_version 1305981 (0.0007) [2023-12-27 14:41:51,873][20381] Updated weights for policy 1, policy_version 1305079 (0.0008) [2023-12-27 14:41:51,934][20381] Updated weights for policy 1, policy_version 1305089 (0.0008) [2023-12-27 14:41:51,994][20381] Updated weights for policy 1, policy_version 1305099 (0.0007) [2023-12-27 14:41:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 668532736. Throughput: 0: 10260.9, 1: 10249.0. Samples: 668504500. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:41:52,363][19170] Avg episode reward: [(0, '960.990'), (1, '960.910')] [2023-12-27 14:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001305104_334135296.pth... [2023-12-27 14:41:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001305984_334397440.pth... [2023-12-27 14:41:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001303888_333824000.pth [2023-12-27 14:41:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001304800_334094336.pth [2023-12-27 14:41:52,467][20347] Updated weights for policy 0, policy_version 1305991 (0.0008) [2023-12-27 14:41:52,529][20347] Updated weights for policy 0, policy_version 1306001 (0.0007) [2023-12-27 14:41:52,585][20347] Updated weights for policy 0, policy_version 1306011 (0.0006) [2023-12-27 14:41:52,669][20381] Updated weights for policy 1, policy_version 1305109 (0.0008) [2023-12-27 14:41:52,730][20381] Updated weights for policy 1, policy_version 1305119 (0.0007) [2023-12-27 14:41:52,790][20381] Updated weights for policy 1, policy_version 1305129 (0.0008) [2023-12-27 14:41:53,262][20347] Updated weights for policy 0, policy_version 1306021 (0.0007) [2023-12-27 14:41:53,317][20347] Updated weights for policy 0, policy_version 1306031 (0.0008) [2023-12-27 14:41:53,369][20347] Updated weights for policy 0, policy_version 1306041 (0.0007) [2023-12-27 14:41:53,482][20381] Updated weights for policy 1, policy_version 1305139 (0.0007) [2023-12-27 14:41:53,546][20381] Updated weights for policy 1, policy_version 1305149 (0.0007) [2023-12-27 14:41:53,594][20381] Updated weights for policy 1, policy_version 1305159 (0.0007) [2023-12-27 14:41:54,051][20347] Updated weights for policy 0, policy_version 1306051 (0.0007) [2023-12-27 14:41:54,112][20347] Updated weights for policy 0, policy_version 1306061 (0.0010) [2023-12-27 14:41:54,164][20347] Updated weights for policy 0, policy_version 1306071 (0.0010) [2023-12-27 14:41:54,247][20381] Updated weights for policy 1, policy_version 1305169 (0.0006) [2023-12-27 14:41:54,292][20381] Updated weights for policy 1, policy_version 1305179 (0.0008) [2023-12-27 14:41:54,349][20381] Updated weights for policy 1, policy_version 1305189 (0.0007) [2023-12-27 14:41:54,397][20381] Updated weights for policy 1, policy_version 1305199 (0.0007) [2023-12-27 14:41:54,769][20347] Updated weights for policy 0, policy_version 1306081 (0.0010) [2023-12-27 14:41:54,816][20347] Updated weights for policy 0, policy_version 1306091 (0.0005) [2023-12-27 14:41:54,862][20347] Updated weights for policy 0, policy_version 1306101 (0.0006) [2023-12-27 14:41:54,904][20347] Updated weights for policy 0, policy_version 1306111 (0.0006) [2023-12-27 14:41:55,145][20381] Updated weights for policy 1, policy_version 1305209 (0.0007) [2023-12-27 14:41:55,194][20381] Updated weights for policy 1, policy_version 1305219 (0.0007) [2023-12-27 14:41:55,241][20381] Updated weights for policy 1, policy_version 1305229 (0.0006) [2023-12-27 14:41:55,521][20347] Updated weights for policy 0, policy_version 1306121 (0.0006) [2023-12-27 14:41:55,567][20347] Updated weights for policy 0, policy_version 1306131 (0.0005) [2023-12-27 14:41:55,614][20347] Updated weights for policy 0, policy_version 1306141 (0.0005) [2023-12-27 14:41:55,974][20381] Updated weights for policy 1, policy_version 1305239 (0.0007) [2023-12-27 14:41:56,027][20381] Updated weights for policy 1, policy_version 1305249 (0.0009) [2023-12-27 14:41:56,083][20381] Updated weights for policy 1, policy_version 1305259 (0.0007) [2023-12-27 14:41:56,261][20347] Updated weights for policy 0, policy_version 1306151 (0.0007) [2023-12-27 14:41:56,312][20347] Updated weights for policy 0, policy_version 1306161 (0.0010) [2023-12-27 14:41:56,359][20347] Updated weights for policy 0, policy_version 1306171 (0.0010) [2023-12-27 14:41:56,711][20381] Updated weights for policy 1, policy_version 1305269 (0.0006) [2023-12-27 14:41:56,767][20381] Updated weights for policy 1, policy_version 1305279 (0.0007) [2023-12-27 14:41:56,823][20381] Updated weights for policy 1, policy_version 1305289 (0.0007) [2023-12-27 14:41:56,980][20347] Updated weights for policy 0, policy_version 1306182 (0.0011) [2023-12-27 14:41:57,046][20347] Updated weights for policy 0, policy_version 1306192 (0.0010) [2023-12-27 14:41:57,101][20347] Updated weights for policy 0, policy_version 1306202 (0.0010) [2023-12-27 14:41:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 668639232. Throughput: 0: 10271.0, 1: 10233.2. Samples: 668627596. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:41:57,363][19170] Avg episode reward: [(0, '970.950'), (1, '990.240')] [2023-12-27 14:41:57,423][20381] Updated weights for policy 1, policy_version 1305299 (0.0007) [2023-12-27 14:41:57,472][20381] Updated weights for policy 1, policy_version 1305309 (0.0005) [2023-12-27 14:41:57,522][20381] Updated weights for policy 1, policy_version 1305319 (0.0006) [2023-12-27 14:41:57,750][20347] Updated weights for policy 0, policy_version 1306212 (0.0008) [2023-12-27 14:41:57,815][20347] Updated weights for policy 0, policy_version 1306222 (0.0005) [2023-12-27 14:41:57,872][20347] Updated weights for policy 0, policy_version 1306232 (0.0010) [2023-12-27 14:41:58,159][20381] Updated weights for policy 1, policy_version 1305329 (0.0007) [2023-12-27 14:41:58,226][20381] Updated weights for policy 1, policy_version 1305339 (0.0007) [2023-12-27 14:41:58,295][20381] Updated weights for policy 1, policy_version 1305349 (0.0006) [2023-12-27 14:41:58,366][20381] Updated weights for policy 1, policy_version 1305359 (0.0008) [2023-12-27 14:41:58,527][20347] Updated weights for policy 0, policy_version 1306242 (0.0007) [2023-12-27 14:41:58,586][20347] Updated weights for policy 0, policy_version 1306252 (0.0008) [2023-12-27 14:41:58,646][20347] Updated weights for policy 0, policy_version 1306262 (0.0008) [2023-12-27 14:41:58,714][20347] Updated weights for policy 0, policy_version 1306272 (0.0008) [2023-12-27 14:41:59,116][20381] Updated weights for policy 1, policy_version 1305369 (0.0008) [2023-12-27 14:41:59,168][20381] Updated weights for policy 1, policy_version 1305379 (0.0008) [2023-12-27 14:41:59,251][20381] Updated weights for policy 1, policy_version 1305389 (0.0008) [2023-12-27 14:41:59,510][20347] Updated weights for policy 0, policy_version 1306282 (0.0009) [2023-12-27 14:41:59,571][20347] Updated weights for policy 0, policy_version 1306292 (0.0007) [2023-12-27 14:41:59,625][20347] Updated weights for policy 0, policy_version 1306302 (0.0009) [2023-12-27 14:41:59,953][20381] Updated weights for policy 1, policy_version 1305399 (0.0007) [2023-12-27 14:42:00,008][20381] Updated weights for policy 1, policy_version 1305409 (0.0006) [2023-12-27 14:42:00,065][20381] Updated weights for policy 1, policy_version 1305419 (0.0007) [2023-12-27 14:42:00,287][20347] Updated weights for policy 0, policy_version 1306312 (0.0010) [2023-12-27 14:42:00,345][20347] Updated weights for policy 0, policy_version 1306322 (0.0010) [2023-12-27 14:42:00,406][20347] Updated weights for policy 0, policy_version 1306332 (0.0010) [2023-12-27 14:42:00,760][20381] Updated weights for policy 1, policy_version 1305429 (0.0007) [2023-12-27 14:42:00,820][20381] Updated weights for policy 1, policy_version 1305439 (0.0005) [2023-12-27 14:42:00,880][20381] Updated weights for policy 1, policy_version 1305449 (0.0005) [2023-12-27 14:42:01,011][20347] Updated weights for policy 0, policy_version 1306342 (0.0010) [2023-12-27 14:42:01,071][20347] Updated weights for policy 0, policy_version 1306352 (0.0010) [2023-12-27 14:42:01,130][20347] Updated weights for policy 0, policy_version 1306362 (0.0010) [2023-12-27 14:42:01,504][20381] Updated weights for policy 1, policy_version 1305459 (0.0006) [2023-12-27 14:42:01,556][20381] Updated weights for policy 1, policy_version 1305469 (0.0009) [2023-12-27 14:42:01,617][20381] Updated weights for policy 1, policy_version 1305479 (0.0007) [2023-12-27 14:42:01,792][20347] Updated weights for policy 0, policy_version 1306372 (0.0009) [2023-12-27 14:42:01,847][20347] Updated weights for policy 0, policy_version 1306382 (0.0008) [2023-12-27 14:42:01,902][20347] Updated weights for policy 0, policy_version 1306392 (0.0007) [2023-12-27 14:42:02,348][20381] Updated weights for policy 1, policy_version 1305489 (0.0009) [2023-12-27 14:42:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 668737536. Throughput: 0: 10290.8, 1: 10217.4. Samples: 668750636. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:02,363][19170] Avg episode reward: [(0, '922.050'), (1, '990.200')] [2023-12-27 14:42:02,414][20381] Updated weights for policy 1, policy_version 1305499 (0.0007) [2023-12-27 14:42:02,480][20381] Updated weights for policy 1, policy_version 1305509 (0.0007) [2023-12-27 14:42:02,540][20381] Updated weights for policy 1, policy_version 1305519 (0.0007) [2023-12-27 14:42:02,610][20347] Updated weights for policy 0, policy_version 1306402 (0.0008) [2023-12-27 14:42:02,671][20347] Updated weights for policy 0, policy_version 1306412 (0.0008) [2023-12-27 14:42:02,731][20347] Updated weights for policy 0, policy_version 1306422 (0.0008) [2023-12-27 14:42:02,791][20347] Updated weights for policy 0, policy_version 1306432 (0.0008) [2023-12-27 14:42:03,208][20381] Updated weights for policy 1, policy_version 1305529 (0.0010) [2023-12-27 14:42:03,262][20381] Updated weights for policy 1, policy_version 1305539 (0.0010) [2023-12-27 14:42:03,317][20381] Updated weights for policy 1, policy_version 1305549 (0.0010) [2023-12-27 14:42:03,403][20347] Updated weights for policy 0, policy_version 1306442 (0.0009) [2023-12-27 14:42:03,457][20347] Updated weights for policy 0, policy_version 1306452 (0.0010) [2023-12-27 14:42:03,503][20347] Updated weights for policy 0, policy_version 1306462 (0.0009) [2023-12-27 14:42:03,877][20381] Updated weights for policy 1, policy_version 1305559 (0.0010) [2023-12-27 14:42:03,940][20381] Updated weights for policy 1, policy_version 1305569 (0.0010) [2023-12-27 14:42:04,002][20381] Updated weights for policy 1, policy_version 1305579 (0.0011) [2023-12-27 14:42:04,231][20347] Updated weights for policy 0, policy_version 1306472 (0.0008) [2023-12-27 14:42:04,290][20347] Updated weights for policy 0, policy_version 1306482 (0.0008) [2023-12-27 14:42:04,352][20347] Updated weights for policy 0, policy_version 1306492 (0.0008) [2023-12-27 14:42:04,667][20381] Updated weights for policy 1, policy_version 1305589 (0.0009) [2023-12-27 14:42:04,730][20381] Updated weights for policy 1, policy_version 1305599 (0.0009) [2023-12-27 14:42:04,786][20381] Updated weights for policy 1, policy_version 1305609 (0.0007) [2023-12-27 14:42:05,015][20347] Updated weights for policy 0, policy_version 1306502 (0.0007) [2023-12-27 14:42:05,065][20347] Updated weights for policy 0, policy_version 1306512 (0.0007) [2023-12-27 14:42:05,120][20347] Updated weights for policy 0, policy_version 1306522 (0.0008) [2023-12-27 14:42:05,503][20381] Updated weights for policy 1, policy_version 1305619 (0.0006) [2023-12-27 14:42:05,559][20381] Updated weights for policy 1, policy_version 1305629 (0.0005) [2023-12-27 14:42:05,620][20381] Updated weights for policy 1, policy_version 1305639 (0.0007) [2023-12-27 14:42:05,761][20347] Updated weights for policy 0, policy_version 1306532 (0.0009) [2023-12-27 14:42:05,819][20347] Updated weights for policy 0, policy_version 1306542 (0.0010) [2023-12-27 14:42:05,875][20347] Updated weights for policy 0, policy_version 1306552 (0.0010) [2023-12-27 14:42:06,263][20381] Updated weights for policy 1, policy_version 1305649 (0.0006) [2023-12-27 14:42:06,321][20381] Updated weights for policy 1, policy_version 1305659 (0.0008) [2023-12-27 14:42:06,378][20381] Updated weights for policy 1, policy_version 1305669 (0.0008) [2023-12-27 14:42:06,438][20381] Updated weights for policy 1, policy_version 1305679 (0.0008) [2023-12-27 14:42:06,542][20347] Updated weights for policy 0, policy_version 1306562 (0.0010) [2023-12-27 14:42:06,602][20347] Updated weights for policy 0, policy_version 1306572 (0.0007) [2023-12-27 14:42:06,658][20347] Updated weights for policy 0, policy_version 1306582 (0.0007) [2023-12-27 14:42:06,715][20347] Updated weights for policy 0, policy_version 1306592 (0.0008) [2023-12-27 14:42:07,073][20381] Updated weights for policy 1, policy_version 1305689 (0.0006) [2023-12-27 14:42:07,137][20381] Updated weights for policy 1, policy_version 1305699 (0.0005) [2023-12-27 14:42:07,194][20381] Updated weights for policy 1, policy_version 1305709 (0.0005) [2023-12-27 14:42:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 668844032. Throughput: 0: 10299.6, 1: 10252.6. Samples: 668813100. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:07,363][19170] Avg episode reward: [(0, '902.600'), (1, '980.410')] [2023-12-27 14:42:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001305712_334290944.pth... [2023-12-27 14:42:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001306592_334553088.pth... [2023-12-27 14:42:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001305408_334249984.pth [2023-12-27 14:42:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001304496_333979648.pth [2023-12-27 14:42:07,469][20347] Updated weights for policy 0, policy_version 1306602 (0.0008) [2023-12-27 14:42:07,529][20347] Updated weights for policy 0, policy_version 1306612 (0.0008) [2023-12-27 14:42:07,590][20347] Updated weights for policy 0, policy_version 1306622 (0.0008) [2023-12-27 14:42:07,776][20381] Updated weights for policy 1, policy_version 1305719 (0.0006) [2023-12-27 14:42:07,844][20381] Updated weights for policy 1, policy_version 1305729 (0.0005) [2023-12-27 14:42:07,901][20381] Updated weights for policy 1, policy_version 1305739 (0.0005) [2023-12-27 14:42:08,301][20347] Updated weights for policy 0, policy_version 1306632 (0.0008) [2023-12-27 14:42:08,362][20347] Updated weights for policy 0, policy_version 1306642 (0.0008) [2023-12-27 14:42:08,410][20347] Updated weights for policy 0, policy_version 1306652 (0.0008) [2023-12-27 14:42:08,471][20381] Updated weights for policy 1, policy_version 1305749 (0.0005) [2023-12-27 14:42:08,525][20381] Updated weights for policy 1, policy_version 1305759 (0.0005) [2023-12-27 14:42:08,577][20381] Updated weights for policy 1, policy_version 1305769 (0.0005) [2023-12-27 14:42:09,121][20347] Updated weights for policy 0, policy_version 1306662 (0.0007) [2023-12-27 14:42:09,181][20347] Updated weights for policy 0, policy_version 1306672 (0.0007) [2023-12-27 14:42:09,207][20381] Updated weights for policy 1, policy_version 1305779 (0.0007) [2023-12-27 14:42:09,246][20347] Updated weights for policy 0, policy_version 1306682 (0.0008) [2023-12-27 14:42:09,273][20381] Updated weights for policy 1, policy_version 1305789 (0.0008) [2023-12-27 14:42:09,327][20381] Updated weights for policy 1, policy_version 1305799 (0.0009) [2023-12-27 14:42:09,992][20347] Updated weights for policy 0, policy_version 1306692 (0.0009) [2023-12-27 14:42:10,053][20347] Updated weights for policy 0, policy_version 1306702 (0.0007) [2023-12-27 14:42:10,059][20381] Updated weights for policy 1, policy_version 1305809 (0.0009) [2023-12-27 14:42:10,112][20347] Updated weights for policy 0, policy_version 1306712 (0.0011) [2023-12-27 14:42:10,113][20381] Updated weights for policy 1, policy_version 1305819 (0.0006) [2023-12-27 14:42:10,172][20381] Updated weights for policy 1, policy_version 1305829 (0.0006) [2023-12-27 14:42:10,222][20381] Updated weights for policy 1, policy_version 1305839 (0.0008) [2023-12-27 14:42:10,733][20347] Updated weights for policy 0, policy_version 1306722 (0.0010) [2023-12-27 14:42:10,797][20347] Updated weights for policy 0, policy_version 1306732 (0.0007) [2023-12-27 14:42:10,856][20347] Updated weights for policy 0, policy_version 1306742 (0.0006) [2023-12-27 14:42:10,918][20381] Updated weights for policy 1, policy_version 1305849 (0.0007) [2023-12-27 14:42:10,919][20347] Updated weights for policy 0, policy_version 1306752 (0.0006) [2023-12-27 14:42:10,975][20381] Updated weights for policy 1, policy_version 1305859 (0.0009) [2023-12-27 14:42:11,051][20381] Updated weights for policy 1, policy_version 1305869 (0.0009) [2023-12-27 14:42:11,669][20347] Updated weights for policy 0, policy_version 1306762 (0.0011) [2023-12-27 14:42:11,741][20347] Updated weights for policy 0, policy_version 1306772 (0.0010) [2023-12-27 14:42:11,809][20347] Updated weights for policy 0, policy_version 1306782 (0.0010) [2023-12-27 14:42:11,874][20381] Updated weights for policy 1, policy_version 1305879 (0.0008) [2023-12-27 14:42:11,931][20381] Updated weights for policy 1, policy_version 1305889 (0.0008) [2023-12-27 14:42:11,990][20381] Updated weights for policy 1, policy_version 1305899 (0.0007) [2023-12-27 14:42:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 668942336. Throughput: 0: 10243.3, 1: 10313.1. Samples: 668935740. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:12,363][19170] Avg episode reward: [(0, '941.740'), (1, '921.720')] [2023-12-27 14:42:12,494][20347] Updated weights for policy 0, policy_version 1306792 (0.0006) [2023-12-27 14:42:12,545][20347] Updated weights for policy 0, policy_version 1306802 (0.0005) [2023-12-27 14:42:12,597][20347] Updated weights for policy 0, policy_version 1306812 (0.0006) [2023-12-27 14:42:12,736][20381] Updated weights for policy 1, policy_version 1305909 (0.0008) [2023-12-27 14:42:12,791][20381] Updated weights for policy 1, policy_version 1305919 (0.0008) [2023-12-27 14:42:12,849][20381] Updated weights for policy 1, policy_version 1305929 (0.0008) [2023-12-27 14:42:13,157][20347] Updated weights for policy 0, policy_version 1306822 (0.0005) [2023-12-27 14:42:13,213][20347] Updated weights for policy 0, policy_version 1306832 (0.0005) [2023-12-27 14:42:13,277][20347] Updated weights for policy 0, policy_version 1306842 (0.0005) [2023-12-27 14:42:13,551][20381] Updated weights for policy 1, policy_version 1305939 (0.0006) [2023-12-27 14:42:13,597][20381] Updated weights for policy 1, policy_version 1305949 (0.0007) [2023-12-27 14:42:13,648][20381] Updated weights for policy 1, policy_version 1305959 (0.0006) [2023-12-27 14:42:13,807][20347] Updated weights for policy 0, policy_version 1306852 (0.0006) [2023-12-27 14:42:13,871][20347] Updated weights for policy 0, policy_version 1306862 (0.0005) [2023-12-27 14:42:13,930][20347] Updated weights for policy 0, policy_version 1306872 (0.0005) [2023-12-27 14:42:14,380][20381] Updated weights for policy 1, policy_version 1305969 (0.0008) [2023-12-27 14:42:14,429][20381] Updated weights for policy 1, policy_version 1305979 (0.0008) [2023-12-27 14:42:14,484][20381] Updated weights for policy 1, policy_version 1305989 (0.0008) [2023-12-27 14:42:14,488][20347] Updated weights for policy 0, policy_version 1306882 (0.0007) [2023-12-27 14:42:14,538][20347] Updated weights for policy 0, policy_version 1306892 (0.0006) [2023-12-27 14:42:14,540][20381] Updated weights for policy 1, policy_version 1305999 (0.0007) [2023-12-27 14:42:14,599][20347] Updated weights for policy 0, policy_version 1306902 (0.0010) [2023-12-27 14:42:14,658][20347] Updated weights for policy 0, policy_version 1306912 (0.0007) [2023-12-27 14:42:15,228][20381] Updated weights for policy 1, policy_version 1306009 (0.0006) [2023-12-27 14:42:15,293][20381] Updated weights for policy 1, policy_version 1306019 (0.0007) [2023-12-27 14:42:15,330][20347] Updated weights for policy 0, policy_version 1306922 (0.0006) [2023-12-27 14:42:15,352][20381] Updated weights for policy 1, policy_version 1306029 (0.0005) [2023-12-27 14:42:15,388][20347] Updated weights for policy 0, policy_version 1306932 (0.0007) [2023-12-27 14:42:15,447][20347] Updated weights for policy 0, policy_version 1306942 (0.0007) [2023-12-27 14:42:15,892][20381] Updated weights for policy 1, policy_version 1306039 (0.0010) [2023-12-27 14:42:15,950][20381] Updated weights for policy 1, policy_version 1306049 (0.0011) [2023-12-27 14:42:16,013][20381] Updated weights for policy 1, policy_version 1306059 (0.0010) [2023-12-27 14:42:16,102][20347] Updated weights for policy 0, policy_version 1306952 (0.0007) [2023-12-27 14:42:16,147][20347] Updated weights for policy 0, policy_version 1306962 (0.0007) [2023-12-27 14:42:16,199][20347] Updated weights for policy 0, policy_version 1306972 (0.0008) [2023-12-27 14:42:16,563][20381] Updated weights for policy 1, policy_version 1306069 (0.0008) [2023-12-27 14:42:16,614][20381] Updated weights for policy 1, policy_version 1306079 (0.0007) [2023-12-27 14:42:16,659][20381] Updated weights for policy 1, policy_version 1306089 (0.0007) [2023-12-27 14:42:16,878][20347] Updated weights for policy 0, policy_version 1306982 (0.0008) [2023-12-27 14:42:16,923][20347] Updated weights for policy 0, policy_version 1306992 (0.0009) [2023-12-27 14:42:16,970][20347] Updated weights for policy 0, policy_version 1307002 (0.0010) [2023-12-27 14:42:17,234][20381] Updated weights for policy 1, policy_version 1306099 (0.0007) [2023-12-27 14:42:17,295][20381] Updated weights for policy 1, policy_version 1306109 (0.0005) [2023-12-27 14:42:17,344][20381] Updated weights for policy 1, policy_version 1306119 (0.0005) [2023-12-27 14:42:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 669048832. Throughput: 0: 10312.8, 1: 10353.5. Samples: 669062496. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:17,363][19170] Avg episode reward: [(0, '912.410'), (1, '843.700')] [2023-12-27 14:42:17,752][20347] Updated weights for policy 0, policy_version 1307012 (0.0009) [2023-12-27 14:42:17,801][20347] Updated weights for policy 0, policy_version 1307022 (0.0007) [2023-12-27 14:42:17,862][20347] Updated weights for policy 0, policy_version 1307032 (0.0007) [2023-12-27 14:42:17,922][20381] Updated weights for policy 1, policy_version 1306129 (0.0005) [2023-12-27 14:42:17,979][20381] Updated weights for policy 1, policy_version 1306139 (0.0008) [2023-12-27 14:42:18,036][20381] Updated weights for policy 1, policy_version 1306149 (0.0008) [2023-12-27 14:42:18,089][20381] Updated weights for policy 1, policy_version 1306159 (0.0008) [2023-12-27 14:42:18,548][20347] Updated weights for policy 0, policy_version 1307042 (0.0007) [2023-12-27 14:42:18,617][20347] Updated weights for policy 0, policy_version 1307052 (0.0006) [2023-12-27 14:42:18,672][20347] Updated weights for policy 0, policy_version 1307062 (0.0007) [2023-12-27 14:42:18,721][20347] Updated weights for policy 0, policy_version 1307072 (0.0007) [2023-12-27 14:42:18,826][20381] Updated weights for policy 1, policy_version 1306169 (0.0008) [2023-12-27 14:42:18,887][20381] Updated weights for policy 1, policy_version 1306179 (0.0008) [2023-12-27 14:42:18,944][20381] Updated weights for policy 1, policy_version 1306189 (0.0008) [2023-12-27 14:42:19,411][20347] Updated weights for policy 0, policy_version 1307082 (0.0008) [2023-12-27 14:42:19,475][20347] Updated weights for policy 0, policy_version 1307092 (0.0008) [2023-12-27 14:42:19,544][20347] Updated weights for policy 0, policy_version 1307102 (0.0007) [2023-12-27 14:42:19,653][20381] Updated weights for policy 1, policy_version 1306199 (0.0008) [2023-12-27 14:42:19,711][20381] Updated weights for policy 1, policy_version 1306209 (0.0008) [2023-12-27 14:42:19,775][20381] Updated weights for policy 1, policy_version 1306219 (0.0008) [2023-12-27 14:42:20,267][20347] Updated weights for policy 0, policy_version 1307112 (0.0008) [2023-12-27 14:42:20,330][20347] Updated weights for policy 0, policy_version 1307122 (0.0008) [2023-12-27 14:42:20,384][20347] Updated weights for policy 0, policy_version 1307132 (0.0007) [2023-12-27 14:42:20,495][20381] Updated weights for policy 1, policy_version 1306229 (0.0007) [2023-12-27 14:42:20,552][20381] Updated weights for policy 1, policy_version 1306239 (0.0005) [2023-12-27 14:42:20,627][20381] Updated weights for policy 1, policy_version 1306249 (0.0008) [2023-12-27 14:42:21,034][20291] KL-divergence is very high: 146.7665 [2023-12-27 14:42:21,064][20347] Updated weights for policy 0, policy_version 1307142 (0.0008) [2023-12-27 14:42:21,087][20291] KL-divergence is very high: 251.9686 [2023-12-27 14:42:21,125][20347] Updated weights for policy 0, policy_version 1307152 (0.0008) [2023-12-27 14:42:21,137][20291] KL-divergence is very high: 173.9868 [2023-12-27 14:42:21,190][20347] Updated weights for policy 0, policy_version 1307162 (0.0008) [2023-12-27 14:42:21,362][20381] Updated weights for policy 1, policy_version 1306259 (0.0008) [2023-12-27 14:42:21,437][20381] Updated weights for policy 1, policy_version 1306269 (0.0008) [2023-12-27 14:42:21,502][20381] Updated weights for policy 1, policy_version 1306279 (0.0006) [2023-12-27 14:42:21,525][20292] KL-divergence is very high: 133.2337 [2023-12-27 14:42:21,918][20347] Updated weights for policy 0, policy_version 1307172 (0.0008) [2023-12-27 14:42:21,975][20347] Updated weights for policy 0, policy_version 1307182 (0.0008) [2023-12-27 14:42:22,036][20347] Updated weights for policy 0, policy_version 1307192 (0.0008) [2023-12-27 14:42:22,146][20381] Updated weights for policy 1, policy_version 1306289 (0.0006) [2023-12-27 14:42:22,204][20381] Updated weights for policy 1, policy_version 1306299 (0.0008) [2023-12-27 14:42:22,262][20381] Updated weights for policy 1, policy_version 1306309 (0.0007) [2023-12-27 14:42:22,321][20381] Updated weights for policy 1, policy_version 1306319 (0.0008) [2023-12-27 14:42:22,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 669155328. Throughput: 0: 10289.4, 1: 10293.0. Samples: 669121624. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:22,363][19170] Avg episode reward: [(0, '883.430'), (1, '892.550')] [2023-12-27 14:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001307200_334708736.pth... [2023-12-27 14:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001306320_334446592.pth... [2023-12-27 14:42:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001305984_334397440.pth [2023-12-27 14:42:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001305104_334135296.pth [2023-12-27 14:42:22,788][20347] Updated weights for policy 0, policy_version 1307202 (0.0008) [2023-12-27 14:42:22,848][20347] Updated weights for policy 0, policy_version 1307212 (0.0008) [2023-12-27 14:42:22,876][20291] KL-divergence is very high: 160.5826 [2023-12-27 14:42:22,909][20347] Updated weights for policy 0, policy_version 1307222 (0.0008) [2023-12-27 14:42:22,925][20291] KL-divergence is very high: 141.7220 [2023-12-27 14:42:22,969][20347] Updated weights for policy 0, policy_version 1307232 (0.0007) [2023-12-27 14:42:23,025][20381] Updated weights for policy 1, policy_version 1306329 (0.0007) [2023-12-27 14:42:23,085][20381] Updated weights for policy 1, policy_version 1306339 (0.0008) [2023-12-27 14:42:23,137][20381] Updated weights for policy 1, policy_version 1306349 (0.0008) [2023-12-27 14:42:23,630][20347] Updated weights for policy 0, policy_version 1307242 (0.0007) [2023-12-27 14:42:23,693][20347] Updated weights for policy 0, policy_version 1307252 (0.0008) [2023-12-27 14:42:23,759][20347] Updated weights for policy 0, policy_version 1307262 (0.0007) [2023-12-27 14:42:23,783][20381] Updated weights for policy 1, policy_version 1306359 (0.0006) [2023-12-27 14:42:23,842][20381] Updated weights for policy 1, policy_version 1306369 (0.0008) [2023-12-27 14:42:23,903][20381] Updated weights for policy 1, policy_version 1306379 (0.0008) [2023-12-27 14:42:24,452][20347] Updated weights for policy 0, policy_version 1307272 (0.0008) [2023-12-27 14:42:24,503][20347] Updated weights for policy 0, policy_version 1307282 (0.0008) [2023-12-27 14:42:24,557][20347] Updated weights for policy 0, policy_version 1307292 (0.0007) [2023-12-27 14:42:24,570][20381] Updated weights for policy 1, policy_version 1306389 (0.0007) [2023-12-27 14:42:24,621][20381] Updated weights for policy 1, policy_version 1306399 (0.0006) [2023-12-27 14:42:24,677][20381] Updated weights for policy 1, policy_version 1306409 (0.0005) [2023-12-27 14:42:25,161][20347] Updated weights for policy 0, policy_version 1307302 (0.0006) [2023-12-27 14:42:25,217][20347] Updated weights for policy 0, policy_version 1307312 (0.0005) [2023-12-27 14:42:25,261][20381] Updated weights for policy 1, policy_version 1306419 (0.0006) [2023-12-27 14:42:25,270][20347] Updated weights for policy 0, policy_version 1307322 (0.0006) [2023-12-27 14:42:25,310][20381] Updated weights for policy 1, policy_version 1306429 (0.0007) [2023-12-27 14:42:25,364][20381] Updated weights for policy 1, policy_version 1306439 (0.0008) [2023-12-27 14:42:25,865][20347] Updated weights for policy 0, policy_version 1307332 (0.0007) [2023-12-27 14:42:25,928][20347] Updated weights for policy 0, policy_version 1307342 (0.0006) [2023-12-27 14:42:25,944][20381] Updated weights for policy 1, policy_version 1306449 (0.0007) [2023-12-27 14:42:25,986][20347] Updated weights for policy 0, policy_version 1307352 (0.0009) [2023-12-27 14:42:25,999][20381] Updated weights for policy 1, policy_version 1306459 (0.0005) [2023-12-27 14:42:26,048][20381] Updated weights for policy 1, policy_version 1306469 (0.0007) [2023-12-27 14:42:26,101][20381] Updated weights for policy 1, policy_version 1306479 (0.0006) [2023-12-27 14:42:26,627][20347] Updated weights for policy 0, policy_version 1307362 (0.0010) [2023-12-27 14:42:26,686][20347] Updated weights for policy 0, policy_version 1307372 (0.0010) [2023-12-27 14:42:26,743][20347] Updated weights for policy 0, policy_version 1307382 (0.0010) [2023-12-27 14:42:26,756][20381] Updated weights for policy 1, policy_version 1306489 (0.0005) [2023-12-27 14:42:26,797][20347] Updated weights for policy 0, policy_version 1307392 (0.0010) [2023-12-27 14:42:26,805][20381] Updated weights for policy 1, policy_version 1306499 (0.0009) [2023-12-27 14:42:26,846][20381] Updated weights for policy 1, policy_version 1306509 (0.0007) [2023-12-27 14:42:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 669253632. Throughput: 0: 10322.1, 1: 10338.9. Samples: 669245552. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:27,363][19170] Avg episode reward: [(0, '902.670'), (1, '912.120')] [2023-12-27 14:42:27,462][20347] Updated weights for policy 0, policy_version 1307402 (0.0010) [2023-12-27 14:42:27,464][20381] Updated weights for policy 1, policy_version 1306519 (0.0006) [2023-12-27 14:42:27,515][20381] Updated weights for policy 1, policy_version 1306529 (0.0005) [2023-12-27 14:42:27,516][20347] Updated weights for policy 0, policy_version 1307412 (0.0010) [2023-12-27 14:42:27,564][20347] Updated weights for policy 0, policy_version 1307422 (0.0010) [2023-12-27 14:42:27,569][20381] Updated weights for policy 1, policy_version 1306539 (0.0005) [2023-12-27 14:42:28,233][20347] Updated weights for policy 0, policy_version 1307432 (0.0010) [2023-12-27 14:42:28,281][20381] Updated weights for policy 1, policy_version 1306549 (0.0009) [2023-12-27 14:42:28,281][20347] Updated weights for policy 0, policy_version 1307442 (0.0010) [2023-12-27 14:42:28,340][20381] Updated weights for policy 1, policy_version 1306559 (0.0011) [2023-12-27 14:42:28,342][20347] Updated weights for policy 0, policy_version 1307452 (0.0009) [2023-12-27 14:42:28,406][20381] Updated weights for policy 1, policy_version 1306569 (0.0008) [2023-12-27 14:42:28,978][20347] Updated weights for policy 0, policy_version 1307462 (0.0009) [2023-12-27 14:42:29,027][20381] Updated weights for policy 1, policy_version 1306579 (0.0007) [2023-12-27 14:42:29,034][20347] Updated weights for policy 0, policy_version 1307472 (0.0006) [2023-12-27 14:42:29,078][20381] Updated weights for policy 1, policy_version 1306589 (0.0007) [2023-12-27 14:42:29,087][20347] Updated weights for policy 0, policy_version 1307482 (0.0006) [2023-12-27 14:42:29,134][20381] Updated weights for policy 1, policy_version 1306599 (0.0007) [2023-12-27 14:42:29,792][20381] Updated weights for policy 1, policy_version 1306609 (0.0007) [2023-12-27 14:42:29,794][20347] Updated weights for policy 0, policy_version 1307492 (0.0008) [2023-12-27 14:42:29,858][20381] Updated weights for policy 1, policy_version 1306619 (0.0007) [2023-12-27 14:42:29,858][20347] Updated weights for policy 0, policy_version 1307502 (0.0011) [2023-12-27 14:42:29,914][20381] Updated weights for policy 1, policy_version 1306629 (0.0008) [2023-12-27 14:42:29,921][20347] Updated weights for policy 0, policy_version 1307512 (0.0011) [2023-12-27 14:42:29,983][20381] Updated weights for policy 1, policy_version 1306639 (0.0008) [2023-12-27 14:42:30,584][20381] Updated weights for policy 1, policy_version 1306649 (0.0010) [2023-12-27 14:42:30,598][20347] Updated weights for policy 0, policy_version 1307522 (0.0011) [2023-12-27 14:42:30,638][20381] Updated weights for policy 1, policy_version 1306659 (0.0010) [2023-12-27 14:42:30,652][20347] Updated weights for policy 0, policy_version 1307532 (0.0010) [2023-12-27 14:42:30,692][20381] Updated weights for policy 1, policy_version 1306669 (0.0010) [2023-12-27 14:42:30,706][20347] Updated weights for policy 0, policy_version 1307542 (0.0010) [2023-12-27 14:42:30,760][20347] Updated weights for policy 0, policy_version 1307552 (0.0010) [2023-12-27 14:42:31,408][20347] Updated weights for policy 0, policy_version 1307562 (0.0008) [2023-12-27 14:42:31,420][20381] Updated weights for policy 1, policy_version 1306679 (0.0010) [2023-12-27 14:42:31,467][20347] Updated weights for policy 0, policy_version 1307572 (0.0006) [2023-12-27 14:42:31,476][20381] Updated weights for policy 1, policy_version 1306689 (0.0011) [2023-12-27 14:42:31,483][20291] KL-divergence is very high: 101.3996 [2023-12-27 14:42:31,518][20347] Updated weights for policy 0, policy_version 1307582 (0.0005) [2023-12-27 14:42:31,535][20381] Updated weights for policy 1, policy_version 1306699 (0.0010) [2023-12-27 14:42:32,198][20347] Updated weights for policy 0, policy_version 1307592 (0.0005) [2023-12-27 14:42:32,256][20381] Updated weights for policy 1, policy_version 1306709 (0.0008) [2023-12-27 14:42:32,259][20347] Updated weights for policy 0, policy_version 1307602 (0.0007) [2023-12-27 14:42:32,323][20381] Updated weights for policy 1, policy_version 1306719 (0.0007) [2023-12-27 14:42:32,325][20347] Updated weights for policy 0, policy_version 1307612 (0.0007) [2023-12-27 14:42:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 669360128. Throughput: 0: 10306.4, 1: 10332.8. Samples: 669369972. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:32,363][19170] Avg episode reward: [(0, '855.130'), (1, '912.140')] [2023-12-27 14:42:32,390][20381] Updated weights for policy 1, policy_version 1306729 (0.0009) [2023-12-27 14:42:33,006][20347] Updated weights for policy 0, policy_version 1307622 (0.0007) [2023-12-27 14:42:33,059][20381] Updated weights for policy 1, policy_version 1306739 (0.0009) [2023-12-27 14:42:33,064][20347] Updated weights for policy 0, policy_version 1307632 (0.0006) [2023-12-27 14:42:33,108][20381] Updated weights for policy 1, policy_version 1306749 (0.0010) [2023-12-27 14:42:33,118][20347] Updated weights for policy 0, policy_version 1307642 (0.0006) [2023-12-27 14:42:33,156][20381] Updated weights for policy 1, policy_version 1306759 (0.0010) [2023-12-27 14:42:33,719][20381] Updated weights for policy 1, policy_version 1306769 (0.0010) [2023-12-27 14:42:33,772][20381] Updated weights for policy 1, policy_version 1306779 (0.0005) [2023-12-27 14:42:33,784][20347] Updated weights for policy 0, policy_version 1307652 (0.0007) [2023-12-27 14:42:33,818][20381] Updated weights for policy 1, policy_version 1306789 (0.0009) [2023-12-27 14:42:33,831][20347] Updated weights for policy 0, policy_version 1307662 (0.0007) [2023-12-27 14:42:33,867][20381] Updated weights for policy 1, policy_version 1306799 (0.0010) [2023-12-27 14:42:33,885][20347] Updated weights for policy 0, policy_version 1307672 (0.0006) [2023-12-27 14:42:34,571][20381] Updated weights for policy 1, policy_version 1306809 (0.0011) [2023-12-27 14:42:34,612][20347] Updated weights for policy 0, policy_version 1307682 (0.0007) [2023-12-27 14:42:34,625][20381] Updated weights for policy 1, policy_version 1306819 (0.0010) [2023-12-27 14:42:34,675][20347] Updated weights for policy 0, policy_version 1307692 (0.0006) [2023-12-27 14:42:34,685][20381] Updated weights for policy 1, policy_version 1306829 (0.0009) [2023-12-27 14:42:34,729][20347] Updated weights for policy 0, policy_version 1307702 (0.0007) [2023-12-27 14:42:34,782][20347] Updated weights for policy 0, policy_version 1307712 (0.0008) [2023-12-27 14:42:35,367][20381] Updated weights for policy 1, policy_version 1306839 (0.0007) [2023-12-27 14:42:35,416][20381] Updated weights for policy 1, policy_version 1306849 (0.0009) [2023-12-27 14:42:35,476][20381] Updated weights for policy 1, policy_version 1306859 (0.0006) [2023-12-27 14:42:35,489][20347] Updated weights for policy 0, policy_version 1307723 (0.0007) [2023-12-27 14:42:35,539][20347] Updated weights for policy 0, policy_version 1307733 (0.0006) [2023-12-27 14:42:35,589][20347] Updated weights for policy 0, policy_version 1307743 (0.0006) [2023-12-27 14:42:36,032][20381] Updated weights for policy 1, policy_version 1306869 (0.0007) [2023-12-27 14:42:36,085][20381] Updated weights for policy 1, policy_version 1306879 (0.0006) [2023-12-27 14:42:36,148][20381] Updated weights for policy 1, policy_version 1306889 (0.0010) [2023-12-27 14:42:36,340][20347] Updated weights for policy 0, policy_version 1307753 (0.0010) [2023-12-27 14:42:36,386][20347] Updated weights for policy 0, policy_version 1307763 (0.0010) [2023-12-27 14:42:36,432][20347] Updated weights for policy 0, policy_version 1307773 (0.0010) [2023-12-27 14:42:36,836][20381] Updated weights for policy 1, policy_version 1306899 (0.0010) [2023-12-27 14:42:36,892][20381] Updated weights for policy 1, policy_version 1306909 (0.0011) [2023-12-27 14:42:36,941][20381] Updated weights for policy 1, policy_version 1306919 (0.0010) [2023-12-27 14:42:37,091][20347] Updated weights for policy 0, policy_version 1307783 (0.0009) [2023-12-27 14:42:37,150][20347] Updated weights for policy 0, policy_version 1307793 (0.0010) [2023-12-27 14:42:37,202][20347] Updated weights for policy 0, policy_version 1307803 (0.0010) [2023-12-27 14:42:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 669466624. Throughput: 0: 10275.5, 1: 10331.2. Samples: 669431804. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:37,363][19170] Avg episode reward: [(0, '825.400'), (1, '892.580')] [2023-12-27 14:42:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001306928_334602240.pth... [2023-12-27 14:42:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001307808_334864384.pth... [2023-12-27 14:42:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001305712_334290944.pth [2023-12-27 14:42:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001306592_334553088.pth [2023-12-27 14:42:37,634][20381] Updated weights for policy 1, policy_version 1306929 (0.0011) [2023-12-27 14:42:37,691][20381] Updated weights for policy 1, policy_version 1306939 (0.0011) [2023-12-27 14:42:37,745][20381] Updated weights for policy 1, policy_version 1306949 (0.0009) [2023-12-27 14:42:37,798][20381] Updated weights for policy 1, policy_version 1306959 (0.0009) [2023-12-27 14:42:37,890][20347] Updated weights for policy 0, policy_version 1307813 (0.0008) [2023-12-27 14:42:37,948][20347] Updated weights for policy 0, policy_version 1307823 (0.0005) [2023-12-27 14:42:38,001][20347] Updated weights for policy 0, policy_version 1307833 (0.0006) [2023-12-27 14:42:38,463][20381] Updated weights for policy 1, policy_version 1306969 (0.0011) [2023-12-27 14:42:38,519][20381] Updated weights for policy 1, policy_version 1306979 (0.0011) [2023-12-27 14:42:38,570][20347] Updated weights for policy 0, policy_version 1307843 (0.0008) [2023-12-27 14:42:38,579][20381] Updated weights for policy 1, policy_version 1306989 (0.0011) [2023-12-27 14:42:38,630][20347] Updated weights for policy 0, policy_version 1307853 (0.0006) [2023-12-27 14:42:38,685][20347] Updated weights for policy 0, policy_version 1307863 (0.0007) [2023-12-27 14:42:39,254][20381] Updated weights for policy 1, policy_version 1306999 (0.0011) [2023-12-27 14:42:39,304][20292] KL-divergence is very high: 141.2069 [2023-12-27 14:42:39,308][20381] Updated weights for policy 1, policy_version 1307009 (0.0007) [2023-12-27 14:42:39,354][20292] KL-divergence is very high: 103.9409 [2023-12-27 14:42:39,379][20381] Updated weights for policy 1, policy_version 1307019 (0.0010) [2023-12-27 14:42:39,446][20347] Updated weights for policy 0, policy_version 1307873 (0.0007) [2023-12-27 14:42:39,501][20347] Updated weights for policy 0, policy_version 1307883 (0.0005) [2023-12-27 14:42:39,568][20347] Updated weights for policy 0, policy_version 1307893 (0.0006) [2023-12-27 14:42:39,634][20347] Updated weights for policy 0, policy_version 1307903 (0.0008) [2023-12-27 14:42:40,109][20381] Updated weights for policy 1, policy_version 1307029 (0.0007) [2023-12-27 14:42:40,171][20381] Updated weights for policy 1, policy_version 1307039 (0.0009) [2023-12-27 14:42:40,229][20381] Updated weights for policy 1, policy_version 1307049 (0.0008) [2023-12-27 14:42:40,317][20347] Updated weights for policy 0, policy_version 1307913 (0.0008) [2023-12-27 14:42:40,379][20347] Updated weights for policy 0, policy_version 1307923 (0.0007) [2023-12-27 14:42:40,435][20347] Updated weights for policy 0, policy_version 1307933 (0.0008) [2023-12-27 14:42:40,936][20381] Updated weights for policy 1, policy_version 1307059 (0.0007) [2023-12-27 14:42:40,990][20381] Updated weights for policy 1, policy_version 1307069 (0.0008) [2023-12-27 14:42:41,051][20381] Updated weights for policy 1, policy_version 1307079 (0.0008) [2023-12-27 14:42:41,163][20347] Updated weights for policy 0, policy_version 1307943 (0.0008) [2023-12-27 14:42:41,215][20347] Updated weights for policy 0, policy_version 1307953 (0.0007) [2023-12-27 14:42:41,277][20347] Updated weights for policy 0, policy_version 1307963 (0.0007) [2023-12-27 14:42:41,814][20381] Updated weights for policy 1, policy_version 1307089 (0.0006) [2023-12-27 14:42:41,874][20381] Updated weights for policy 1, policy_version 1307099 (0.0008) [2023-12-27 14:42:41,938][20381] Updated weights for policy 1, policy_version 1307109 (0.0007) [2023-12-27 14:42:41,994][20381] Updated weights for policy 1, policy_version 1307119 (0.0008) [2023-12-27 14:42:42,040][20347] Updated weights for policy 0, policy_version 1307973 (0.0008) [2023-12-27 14:42:42,097][20347] Updated weights for policy 0, policy_version 1307983 (0.0008) [2023-12-27 14:42:42,156][20347] Updated weights for policy 0, policy_version 1307993 (0.0008) [2023-12-27 14:42:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 669564928. Throughput: 0: 10221.5, 1: 10355.7. Samples: 669553572. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:42,363][19170] Avg episode reward: [(0, '844.110'), (1, '892.600')] [2023-12-27 14:42:42,704][20381] Updated weights for policy 1, policy_version 1307129 (0.0007) [2023-12-27 14:42:42,759][20381] Updated weights for policy 1, policy_version 1307139 (0.0007) [2023-12-27 14:42:42,813][20381] Updated weights for policy 1, policy_version 1307149 (0.0007) [2023-12-27 14:42:42,925][20347] Updated weights for policy 0, policy_version 1308003 (0.0008) [2023-12-27 14:42:42,985][20347] Updated weights for policy 0, policy_version 1308013 (0.0008) [2023-12-27 14:42:43,046][20347] Updated weights for policy 0, policy_version 1308023 (0.0007) [2023-12-27 14:42:43,530][20381] Updated weights for policy 1, policy_version 1307159 (0.0009) [2023-12-27 14:42:43,583][20381] Updated weights for policy 1, policy_version 1307169 (0.0008) [2023-12-27 14:42:43,632][20381] Updated weights for policy 1, policy_version 1307179 (0.0008) [2023-12-27 14:42:43,722][20347] Updated weights for policy 0, policy_version 1308033 (0.0009) [2023-12-27 14:42:43,779][20347] Updated weights for policy 0, policy_version 1308043 (0.0008) [2023-12-27 14:42:43,830][20347] Updated weights for policy 0, policy_version 1308053 (0.0008) [2023-12-27 14:42:43,879][20347] Updated weights for policy 0, policy_version 1308063 (0.0008) [2023-12-27 14:42:44,325][20381] Updated weights for policy 1, policy_version 1307189 (0.0008) [2023-12-27 14:42:44,378][20381] Updated weights for policy 1, policy_version 1307199 (0.0005) [2023-12-27 14:42:44,424][20381] Updated weights for policy 1, policy_version 1307209 (0.0005) [2023-12-27 14:42:44,586][20347] Updated weights for policy 0, policy_version 1308073 (0.0007) [2023-12-27 14:42:44,641][20347] Updated weights for policy 0, policy_version 1308083 (0.0005) [2023-12-27 14:42:44,701][20347] Updated weights for policy 0, policy_version 1308093 (0.0006) [2023-12-27 14:42:45,129][20381] Updated weights for policy 1, policy_version 1307219 (0.0006) [2023-12-27 14:42:45,194][20381] Updated weights for policy 1, policy_version 1307229 (0.0008) [2023-12-27 14:42:45,258][20381] Updated weights for policy 1, policy_version 1307239 (0.0008) [2023-12-27 14:42:45,378][20347] Updated weights for policy 0, policy_version 1308103 (0.0008) [2023-12-27 14:42:45,430][20347] Updated weights for policy 0, policy_version 1308113 (0.0008) [2023-12-27 14:42:45,475][20347] Updated weights for policy 0, policy_version 1308123 (0.0007) [2023-12-27 14:42:45,924][20381] Updated weights for policy 1, policy_version 1307249 (0.0008) [2023-12-27 14:42:45,983][20381] Updated weights for policy 1, policy_version 1307259 (0.0007) [2023-12-27 14:42:46,036][20381] Updated weights for policy 1, policy_version 1307269 (0.0005) [2023-12-27 14:42:46,085][20381] Updated weights for policy 1, policy_version 1307279 (0.0006) [2023-12-27 14:42:46,186][20347] Updated weights for policy 0, policy_version 1308133 (0.0007) [2023-12-27 14:42:46,235][20347] Updated weights for policy 0, policy_version 1308143 (0.0010) [2023-12-27 14:42:46,283][20347] Updated weights for policy 0, policy_version 1308153 (0.0010) [2023-12-27 14:42:46,741][20381] Updated weights for policy 1, policy_version 1307289 (0.0008) [2023-12-27 14:42:46,794][20381] Updated weights for policy 1, policy_version 1307299 (0.0007) [2023-12-27 14:42:46,850][20381] Updated weights for policy 1, policy_version 1307309 (0.0008) [2023-12-27 14:42:46,885][20347] Updated weights for policy 0, policy_version 1308163 (0.0009) [2023-12-27 14:42:46,938][20347] Updated weights for policy 0, policy_version 1308173 (0.0005) [2023-12-27 14:42:46,989][20347] Updated weights for policy 0, policy_version 1308183 (0.0007) [2023-12-27 14:42:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 669663232. Throughput: 0: 10180.1, 1: 10318.5. Samples: 669673072. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:47,363][19170] Avg episode reward: [(0, '844.130'), (1, '882.880')] [2023-12-27 14:42:47,583][20347] Updated weights for policy 0, policy_version 1308193 (0.0007) [2023-12-27 14:42:47,604][20381] Updated weights for policy 1, policy_version 1307319 (0.0007) [2023-12-27 14:42:47,640][20347] Updated weights for policy 0, policy_version 1308203 (0.0006) [2023-12-27 14:42:47,652][20381] Updated weights for policy 1, policy_version 1307329 (0.0007) [2023-12-27 14:42:47,659][20292] KL-divergence is very high: 134.6245 [2023-12-27 14:42:47,686][20292] KL-divergence is very high: 141.1594 [2023-12-27 14:42:47,696][20347] Updated weights for policy 0, policy_version 1308213 (0.0006) [2023-12-27 14:42:47,701][20292] KL-divergence is very high: 141.2139 [2023-12-27 14:42:47,708][20381] Updated weights for policy 1, policy_version 1307339 (0.0008) [2023-12-27 14:42:47,729][20292] KL-divergence is very high: 104.8399 [2023-12-27 14:42:47,753][20347] Updated weights for policy 0, policy_version 1308223 (0.0005) [2023-12-27 14:42:48,316][20347] Updated weights for policy 0, policy_version 1308233 (0.0007) [2023-12-27 14:42:48,378][20347] Updated weights for policy 0, policy_version 1308243 (0.0007) [2023-12-27 14:42:48,435][20347] Updated weights for policy 0, policy_version 1308253 (0.0006) [2023-12-27 14:42:48,449][20292] KL-divergence is very high: 109.9148 [2023-12-27 14:42:48,480][20381] Updated weights for policy 1, policy_version 1307349 (0.0007) [2023-12-27 14:42:48,533][20381] Updated weights for policy 1, policy_version 1307359 (0.0008) [2023-12-27 14:42:48,558][20292] KL-divergence is very high: 196.5172 [2023-12-27 14:42:48,592][20381] Updated weights for policy 1, policy_version 1307369 (0.0008) [2023-12-27 14:42:49,112][20347] Updated weights for policy 0, policy_version 1308263 (0.0006) [2023-12-27 14:42:49,174][20347] Updated weights for policy 0, policy_version 1308273 (0.0008) [2023-12-27 14:42:49,224][20347] Updated weights for policy 0, policy_version 1308283 (0.0008) [2023-12-27 14:42:49,330][20381] Updated weights for policy 1, policy_version 1307379 (0.0008) [2023-12-27 14:42:49,391][20292] KL-divergence is very high: 102.4314 [2023-12-27 14:42:49,396][20381] Updated weights for policy 1, policy_version 1307389 (0.0009) [2023-12-27 14:42:49,459][20381] Updated weights for policy 1, policy_version 1307399 (0.0006) [2023-12-27 14:42:49,959][20347] Updated weights for policy 0, policy_version 1308293 (0.0007) [2023-12-27 14:42:50,018][20347] Updated weights for policy 0, policy_version 1308303 (0.0008) [2023-12-27 14:42:50,072][20347] Updated weights for policy 0, policy_version 1308313 (0.0007) [2023-12-27 14:42:50,149][20381] Updated weights for policy 1, policy_version 1307409 (0.0007) [2023-12-27 14:42:50,201][20381] Updated weights for policy 1, policy_version 1307419 (0.0007) [2023-12-27 14:42:50,257][20381] Updated weights for policy 1, policy_version 1307429 (0.0008) [2023-12-27 14:42:50,308][20381] Updated weights for policy 1, policy_version 1307439 (0.0008) [2023-12-27 14:42:50,785][20347] Updated weights for policy 0, policy_version 1308323 (0.0009) [2023-12-27 14:42:50,848][20347] Updated weights for policy 0, policy_version 1308333 (0.0009) [2023-12-27 14:42:50,936][20347] Updated weights for policy 0, policy_version 1308343 (0.0008) [2023-12-27 14:42:50,988][20381] Updated weights for policy 1, policy_version 1307449 (0.0005) [2023-12-27 14:42:51,051][20381] Updated weights for policy 1, policy_version 1307459 (0.0007) [2023-12-27 14:42:51,110][20381] Updated weights for policy 1, policy_version 1307469 (0.0008) [2023-12-27 14:42:51,592][20347] Updated weights for policy 0, policy_version 1308353 (0.0007) [2023-12-27 14:42:51,655][20347] Updated weights for policy 0, policy_version 1308363 (0.0009) [2023-12-27 14:42:51,719][20347] Updated weights for policy 0, policy_version 1308373 (0.0010) [2023-12-27 14:42:51,781][20347] Updated weights for policy 0, policy_version 1308383 (0.0007) [2023-12-27 14:42:51,853][20381] Updated weights for policy 1, policy_version 1307479 (0.0007) [2023-12-27 14:42:51,918][20381] Updated weights for policy 1, policy_version 1307489 (0.0007) [2023-12-27 14:42:51,976][20381] Updated weights for policy 1, policy_version 1307499 (0.0009) [2023-12-27 14:42:52,363][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 669761536. Throughput: 0: 10210.1, 1: 10266.0. Samples: 669734528. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:52,363][19170] Avg episode reward: [(0, '765.580'), (1, '844.040')] [2023-12-27 14:42:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001307504_334749696.pth... [2023-12-27 14:42:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001306320_334446592.pth [2023-12-27 14:42:52,406][20347] Updated weights for policy 0, policy_version 1308393 (0.0007) [2023-12-27 14:42:52,469][20347] Updated weights for policy 0, policy_version 1308403 (0.0007) [2023-12-27 14:42:52,533][20347] Updated weights for policy 0, policy_version 1308413 (0.0008) [2023-12-27 14:42:52,548][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001308416_335020032.pth... [2023-12-27 14:42:52,552][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001307200_334708736.pth [2023-12-27 14:42:52,674][20381] Updated weights for policy 1, policy_version 1307509 (0.0008) [2023-12-27 14:42:52,735][20381] Updated weights for policy 1, policy_version 1307519 (0.0008) [2023-12-27 14:42:52,796][20381] Updated weights for policy 1, policy_version 1307529 (0.0007) [2023-12-27 14:42:53,163][20347] Updated weights for policy 0, policy_version 1308423 (0.0008) [2023-12-27 14:42:53,223][20347] Updated weights for policy 0, policy_version 1308433 (0.0007) [2023-12-27 14:42:53,283][20347] Updated weights for policy 0, policy_version 1308443 (0.0008) [2023-12-27 14:42:53,559][20381] Updated weights for policy 1, policy_version 1307539 (0.0008) [2023-12-27 14:42:53,611][20381] Updated weights for policy 1, policy_version 1307549 (0.0009) [2023-12-27 14:42:53,668][20381] Updated weights for policy 1, policy_version 1307559 (0.0008) [2023-12-27 14:42:53,836][20347] Updated weights for policy 0, policy_version 1308453 (0.0007) [2023-12-27 14:42:53,888][20347] Updated weights for policy 0, policy_version 1308463 (0.0007) [2023-12-27 14:42:53,951][20347] Updated weights for policy 0, policy_version 1308473 (0.0009) [2023-12-27 14:42:54,323][20381] Updated weights for policy 1, policy_version 1307569 (0.0008) [2023-12-27 14:42:54,380][20381] Updated weights for policy 1, policy_version 1307579 (0.0008) [2023-12-27 14:42:54,438][20381] Updated weights for policy 1, policy_version 1307589 (0.0008) [2023-12-27 14:42:54,489][20381] Updated weights for policy 1, policy_version 1307599 (0.0007) [2023-12-27 14:42:54,623][20347] Updated weights for policy 0, policy_version 1308483 (0.0010) [2023-12-27 14:42:54,675][20347] Updated weights for policy 0, policy_version 1308493 (0.0006) [2023-12-27 14:42:54,733][20347] Updated weights for policy 0, policy_version 1308503 (0.0009) [2023-12-27 14:42:55,201][20381] Updated weights for policy 1, policy_version 1307609 (0.0008) [2023-12-27 14:42:55,263][20381] Updated weights for policy 1, policy_version 1307619 (0.0008) [2023-12-27 14:42:55,317][20381] Updated weights for policy 1, policy_version 1307629 (0.0008) [2023-12-27 14:42:55,356][20347] Updated weights for policy 0, policy_version 1308513 (0.0010) [2023-12-27 14:42:55,416][20347] Updated weights for policy 0, policy_version 1308523 (0.0010) [2023-12-27 14:42:55,472][20347] Updated weights for policy 0, policy_version 1308533 (0.0009) [2023-12-27 14:42:55,529][20347] Updated weights for policy 0, policy_version 1308543 (0.0006) [2023-12-27 14:42:55,961][20381] Updated weights for policy 1, policy_version 1307639 (0.0006) [2023-12-27 14:42:56,016][20381] Updated weights for policy 1, policy_version 1307649 (0.0006) [2023-12-27 14:42:56,064][20381] Updated weights for policy 1, policy_version 1307659 (0.0006) [2023-12-27 14:42:56,081][20347] Updated weights for policy 0, policy_version 1308553 (0.0006) [2023-12-27 14:42:56,142][20347] Updated weights for policy 0, policy_version 1308563 (0.0005) [2023-12-27 14:42:56,201][20347] Updated weights for policy 0, policy_version 1308573 (0.0006) [2023-12-27 14:42:56,636][20381] Updated weights for policy 1, policy_version 1307669 (0.0006) [2023-12-27 14:42:56,684][20381] Updated weights for policy 1, policy_version 1307679 (0.0009) [2023-12-27 14:42:56,734][20381] Updated weights for policy 1, policy_version 1307689 (0.0008) [2023-12-27 14:42:56,772][20347] Updated weights for policy 0, policy_version 1308583 (0.0009) [2023-12-27 14:42:56,832][20347] Updated weights for policy 0, policy_version 1308593 (0.0010) [2023-12-27 14:42:56,888][20347] Updated weights for policy 0, policy_version 1308603 (0.0010) [2023-12-27 14:42:57,246][20381] Updated weights for policy 1, policy_version 1307699 (0.0005) [2023-12-27 14:42:57,296][20381] Updated weights for policy 1, policy_version 1307709 (0.0005) [2023-12-27 14:42:57,351][20381] Updated weights for policy 1, policy_version 1307719 (0.0006) [2023-12-27 14:42:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 669868032. Throughput: 0: 10309.5, 1: 10216.3. Samples: 669859400. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:42:57,363][19170] Avg episode reward: [(0, '824.240'), (1, '649.610')] [2023-12-27 14:42:57,549][20347] Updated weights for policy 0, policy_version 1308613 (0.0008) [2023-12-27 14:42:57,596][20347] Updated weights for policy 0, policy_version 1308623 (0.0010) [2023-12-27 14:42:57,644][20347] Updated weights for policy 0, policy_version 1308633 (0.0010) [2023-12-27 14:42:58,004][20381] Updated weights for policy 1, policy_version 1307729 (0.0011) [2023-12-27 14:42:58,061][20381] Updated weights for policy 1, policy_version 1307739 (0.0010) [2023-12-27 14:42:58,119][20381] Updated weights for policy 1, policy_version 1307749 (0.0010) [2023-12-27 14:42:58,180][20381] Updated weights for policy 1, policy_version 1307759 (0.0011) [2023-12-27 14:42:58,378][20347] Updated weights for policy 0, policy_version 1308643 (0.0009) [2023-12-27 14:42:58,441][20347] Updated weights for policy 0, policy_version 1308653 (0.0007) [2023-12-27 14:42:58,503][20347] Updated weights for policy 0, policy_version 1308663 (0.0007) [2023-12-27 14:42:59,026][20381] Updated weights for policy 1, policy_version 1307769 (0.0010) [2023-12-27 14:42:59,073][20381] Updated weights for policy 1, policy_version 1307779 (0.0012) [2023-12-27 14:42:59,125][20381] Updated weights for policy 1, policy_version 1307789 (0.0010) [2023-12-27 14:42:59,205][20347] Updated weights for policy 0, policy_version 1308673 (0.0008) [2023-12-27 14:42:59,269][20347] Updated weights for policy 0, policy_version 1308683 (0.0008) [2023-12-27 14:42:59,327][20347] Updated weights for policy 0, policy_version 1308693 (0.0009) [2023-12-27 14:42:59,389][20347] Updated weights for policy 0, policy_version 1308703 (0.0011) [2023-12-27 14:42:59,893][20381] Updated weights for policy 1, policy_version 1307799 (0.0009) [2023-12-27 14:42:59,957][20381] Updated weights for policy 1, policy_version 1307809 (0.0008) [2023-12-27 14:43:00,011][20381] Updated weights for policy 1, policy_version 1307819 (0.0007) [2023-12-27 14:43:00,039][20347] Updated weights for policy 0, policy_version 1308713 (0.0008) [2023-12-27 14:43:00,095][20347] Updated weights for policy 0, policy_version 1308723 (0.0007) [2023-12-27 14:43:00,153][20347] Updated weights for policy 0, policy_version 1308733 (0.0008) [2023-12-27 14:43:00,669][20381] Updated weights for policy 1, policy_version 1307829 (0.0006) [2023-12-27 14:43:00,733][20381] Updated weights for policy 1, policy_version 1307839 (0.0006) [2023-12-27 14:43:00,784][20381] Updated weights for policy 1, policy_version 1307849 (0.0007) [2023-12-27 14:43:00,864][20347] Updated weights for policy 0, policy_version 1308743 (0.0007) [2023-12-27 14:43:00,915][20347] Updated weights for policy 0, policy_version 1308753 (0.0007) [2023-12-27 14:43:00,964][20347] Updated weights for policy 0, policy_version 1308763 (0.0007) [2023-12-27 14:43:01,441][20381] Updated weights for policy 1, policy_version 1307859 (0.0007) [2023-12-27 14:43:01,489][20381] Updated weights for policy 1, policy_version 1307869 (0.0005) [2023-12-27 14:43:01,542][20381] Updated weights for policy 1, policy_version 1307879 (0.0005) [2023-12-27 14:43:01,730][20347] Updated weights for policy 0, policy_version 1308773 (0.0007) [2023-12-27 14:43:01,795][20347] Updated weights for policy 0, policy_version 1308783 (0.0005) [2023-12-27 14:43:01,854][20347] Updated weights for policy 0, policy_version 1308793 (0.0006) [2023-12-27 14:43:02,188][20381] Updated weights for policy 1, policy_version 1307889 (0.0007) [2023-12-27 14:43:02,249][20381] Updated weights for policy 1, policy_version 1307899 (0.0008) [2023-12-27 14:43:02,309][20381] Updated weights for policy 1, policy_version 1307909 (0.0007) [2023-12-27 14:43:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 669966336. Throughput: 0: 10245.7, 1: 10202.3. Samples: 669982656. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:02,363][19170] Avg episode reward: [(0, '824.920'), (1, '678.720')] [2023-12-27 14:43:02,372][20381] Updated weights for policy 1, policy_version 1307919 (0.0009) [2023-12-27 14:43:02,491][20347] Updated weights for policy 0, policy_version 1308803 (0.0007) [2023-12-27 14:43:02,544][20347] Updated weights for policy 0, policy_version 1308813 (0.0007) [2023-12-27 14:43:02,600][20347] Updated weights for policy 0, policy_version 1308823 (0.0008) [2023-12-27 14:43:03,005][20381] Updated weights for policy 1, policy_version 1307929 (0.0009) [2023-12-27 14:43:03,064][20381] Updated weights for policy 1, policy_version 1307939 (0.0008) [2023-12-27 14:43:03,122][20381] Updated weights for policy 1, policy_version 1307949 (0.0010) [2023-12-27 14:43:03,263][20347] Updated weights for policy 0, policy_version 1308833 (0.0007) [2023-12-27 14:43:03,320][20347] Updated weights for policy 0, policy_version 1308843 (0.0005) [2023-12-27 14:43:03,377][20347] Updated weights for policy 0, policy_version 1308853 (0.0005) [2023-12-27 14:43:03,434][20347] Updated weights for policy 0, policy_version 1308863 (0.0008) [2023-12-27 14:43:03,736][20381] Updated weights for policy 1, policy_version 1307959 (0.0009) [2023-12-27 14:43:03,789][20381] Updated weights for policy 1, policy_version 1307969 (0.0005) [2023-12-27 14:43:03,856][20381] Updated weights for policy 1, policy_version 1307979 (0.0007) [2023-12-27 14:43:04,034][20347] Updated weights for policy 0, policy_version 1308873 (0.0008) [2023-12-27 14:43:04,094][20347] Updated weights for policy 0, policy_version 1308883 (0.0008) [2023-12-27 14:43:04,157][20347] Updated weights for policy 0, policy_version 1308893 (0.0008) [2023-12-27 14:43:04,549][20381] Updated weights for policy 1, policy_version 1307989 (0.0007) [2023-12-27 14:43:04,607][20381] Updated weights for policy 1, policy_version 1307999 (0.0007) [2023-12-27 14:43:04,652][20381] Updated weights for policy 1, policy_version 1308009 (0.0007) [2023-12-27 14:43:04,876][20347] Updated weights for policy 0, policy_version 1308903 (0.0008) [2023-12-27 14:43:04,937][20347] Updated weights for policy 0, policy_version 1308913 (0.0007) [2023-12-27 14:43:04,997][20347] Updated weights for policy 0, policy_version 1308923 (0.0005) [2023-12-27 14:43:05,316][20381] Updated weights for policy 1, policy_version 1308019 (0.0006) [2023-12-27 14:43:05,366][20381] Updated weights for policy 1, policy_version 1308029 (0.0008) [2023-12-27 14:43:05,416][20381] Updated weights for policy 1, policy_version 1308039 (0.0005) [2023-12-27 14:43:05,645][20347] Updated weights for policy 0, policy_version 1308933 (0.0006) [2023-12-27 14:43:05,700][20347] Updated weights for policy 0, policy_version 1308943 (0.0008) [2023-12-27 14:43:05,753][20347] Updated weights for policy 0, policy_version 1308953 (0.0007) [2023-12-27 14:43:06,050][20381] Updated weights for policy 1, policy_version 1308049 (0.0006) [2023-12-27 14:43:06,108][20381] Updated weights for policy 1, policy_version 1308059 (0.0006) [2023-12-27 14:43:06,168][20381] Updated weights for policy 1, policy_version 1308069 (0.0008) [2023-12-27 14:43:06,230][20381] Updated weights for policy 1, policy_version 1308079 (0.0008) [2023-12-27 14:43:06,508][20347] Updated weights for policy 0, policy_version 1308963 (0.0007) [2023-12-27 14:43:06,567][20347] Updated weights for policy 0, policy_version 1308973 (0.0007) [2023-12-27 14:43:06,579][20291] KL-divergence is very high: 137.7980 [2023-12-27 14:43:06,596][20291] KL-divergence is very high: 127.7464 [2023-12-27 14:43:06,608][20291] KL-divergence is very high: 102.9295 [2023-12-27 14:43:06,625][20347] Updated weights for policy 0, policy_version 1308983 (0.0008) [2023-12-27 14:43:06,891][20381] Updated weights for policy 1, policy_version 1308089 (0.0010) [2023-12-27 14:43:06,946][20381] Updated weights for policy 1, policy_version 1308099 (0.0011) [2023-12-27 14:43:07,001][20381] Updated weights for policy 1, policy_version 1308109 (0.0010) [2023-12-27 14:43:07,317][20347] Updated weights for policy 0, policy_version 1308993 (0.0007) [2023-12-27 14:43:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 670072832. Throughput: 0: 10279.5, 1: 10250.8. Samples: 670045488. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:07,363][19170] Avg episode reward: [(0, '775.980'), (1, '756.600')] [2023-12-27 14:43:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001308112_334905344.pth... [2023-12-27 14:43:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001306928_334602240.pth [2023-12-27 14:43:07,373][20347] Updated weights for policy 0, policy_version 1309003 (0.0008) [2023-12-27 14:43:07,421][20347] Updated weights for policy 0, policy_version 1309013 (0.0008) [2023-12-27 14:43:07,475][20347] Updated weights for policy 0, policy_version 1309023 (0.0007) [2023-12-27 14:43:07,481][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001309024_335175680.pth... [2023-12-27 14:43:07,485][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001307808_334864384.pth [2023-12-27 14:43:07,679][20381] Updated weights for policy 1, policy_version 1308119 (0.0010) [2023-12-27 14:43:07,736][20381] Updated weights for policy 1, policy_version 1308129 (0.0010) [2023-12-27 14:43:07,790][20381] Updated weights for policy 1, policy_version 1308139 (0.0010) [2023-12-27 14:43:08,153][20347] Updated weights for policy 0, policy_version 1309033 (0.0008) [2023-12-27 14:43:08,207][20347] Updated weights for policy 0, policy_version 1309043 (0.0007) [2023-12-27 14:43:08,255][20347] Updated weights for policy 0, policy_version 1309053 (0.0007) [2023-12-27 14:43:08,449][20381] Updated weights for policy 1, policy_version 1308149 (0.0010) [2023-12-27 14:43:08,507][20381] Updated weights for policy 1, policy_version 1308159 (0.0010) [2023-12-27 14:43:08,568][20381] Updated weights for policy 1, policy_version 1308169 (0.0011) [2023-12-27 14:43:08,959][20347] Updated weights for policy 0, policy_version 1309063 (0.0006) [2023-12-27 14:43:09,026][20347] Updated weights for policy 0, policy_version 1309073 (0.0006) [2023-12-27 14:43:09,087][20347] Updated weights for policy 0, policy_version 1309083 (0.0005) [2023-12-27 14:43:09,230][20381] Updated weights for policy 1, policy_version 1308179 (0.0010) [2023-12-27 14:43:09,292][20381] Updated weights for policy 1, policy_version 1308189 (0.0006) [2023-12-27 14:43:09,353][20381] Updated weights for policy 1, policy_version 1308199 (0.0008) [2023-12-27 14:43:09,738][20347] Updated weights for policy 0, policy_version 1309093 (0.0007) [2023-12-27 14:43:09,799][20347] Updated weights for policy 0, policy_version 1309103 (0.0007) [2023-12-27 14:43:09,864][20347] Updated weights for policy 0, policy_version 1309113 (0.0011) [2023-12-27 14:43:09,902][20291] KL-divergence is very high: 277.3530 [2023-12-27 14:43:10,092][20381] Updated weights for policy 1, policy_version 1308209 (0.0009) [2023-12-27 14:43:10,152][20381] Updated weights for policy 1, policy_version 1308219 (0.0008) [2023-12-27 14:43:10,215][20381] Updated weights for policy 1, policy_version 1308229 (0.0007) [2023-12-27 14:43:10,274][20381] Updated weights for policy 1, policy_version 1308239 (0.0008) [2023-12-27 14:43:10,548][20291] KL-divergence is very high: 318.1165 [2023-12-27 14:43:10,562][20347] Updated weights for policy 0, policy_version 1309123 (0.0011) [2023-12-27 14:43:10,601][20291] KL-divergence is very high: 130.8100 [2023-12-27 14:43:10,621][20291] KL-divergence is very high: 100.0371 [2023-12-27 14:43:10,627][20347] Updated weights for policy 0, policy_version 1309133 (0.0011) [2023-12-27 14:43:10,653][20291] KL-divergence is very high: 129.7509 [2023-12-27 14:43:10,689][20347] Updated weights for policy 0, policy_version 1309143 (0.0010) [2023-12-27 14:43:10,702][20291] KL-divergence is very high: 210.8702 [2023-12-27 14:43:10,981][20381] Updated weights for policy 1, policy_version 1308249 (0.0007) [2023-12-27 14:43:11,037][20381] Updated weights for policy 1, policy_version 1308259 (0.0008) [2023-12-27 14:43:11,100][20381] Updated weights for policy 1, policy_version 1308269 (0.0007) [2023-12-27 14:43:11,476][20347] Updated weights for policy 0, policy_version 1309153 (0.0010) [2023-12-27 14:43:11,533][20347] Updated weights for policy 0, policy_version 1309163 (0.0009) [2023-12-27 14:43:11,582][20347] Updated weights for policy 0, policy_version 1309173 (0.0009) [2023-12-27 14:43:11,641][20347] Updated weights for policy 0, policy_version 1309183 (0.0008) [2023-12-27 14:43:11,922][20381] Updated weights for policy 1, policy_version 1308279 (0.0010) [2023-12-27 14:43:11,975][20381] Updated weights for policy 1, policy_version 1308289 (0.0010) [2023-12-27 14:43:12,028][20381] Updated weights for policy 1, policy_version 1308299 (0.0011) [2023-12-27 14:43:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 670171136. Throughput: 0: 10263.5, 1: 10184.2. Samples: 670165700. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:12,363][19170] Avg episode reward: [(0, '755.640'), (1, '668.980')] [2023-12-27 14:43:12,412][20347] Updated weights for policy 0, policy_version 1309193 (0.0007) [2023-12-27 14:43:12,474][20347] Updated weights for policy 0, policy_version 1309203 (0.0007) [2023-12-27 14:43:12,536][20347] Updated weights for policy 0, policy_version 1309213 (0.0007) [2023-12-27 14:43:12,791][20381] Updated weights for policy 1, policy_version 1308309 (0.0009) [2023-12-27 14:43:12,843][20381] Updated weights for policy 1, policy_version 1308319 (0.0008) [2023-12-27 14:43:12,903][20381] Updated weights for policy 1, policy_version 1308329 (0.0008) [2023-12-27 14:43:13,177][20347] Updated weights for policy 0, policy_version 1309223 (0.0007) [2023-12-27 14:43:13,230][20347] Updated weights for policy 0, policy_version 1309233 (0.0006) [2023-12-27 14:43:13,290][20347] Updated weights for policy 0, policy_version 1309243 (0.0009) [2023-12-27 14:43:13,658][20381] Updated weights for policy 1, policy_version 1308339 (0.0008) [2023-12-27 14:43:13,712][20381] Updated weights for policy 1, policy_version 1308349 (0.0008) [2023-12-27 14:43:13,761][20381] Updated weights for policy 1, policy_version 1308359 (0.0008) [2023-12-27 14:43:13,989][20291] KL-divergence is very high: 139.2185 [2023-12-27 14:43:13,998][20347] Updated weights for policy 0, policy_version 1309253 (0.0008) [2023-12-27 14:43:14,034][20291] KL-divergence is very high: 186.3994 [2023-12-27 14:43:14,055][20347] Updated weights for policy 0, policy_version 1309263 (0.0008) [2023-12-27 14:43:14,075][20291] KL-divergence is very high: 113.0410 [2023-12-27 14:43:14,110][20347] Updated weights for policy 0, policy_version 1309273 (0.0007) [2023-12-27 14:43:14,478][20381] Updated weights for policy 1, policy_version 1308369 (0.0007) [2023-12-27 14:43:14,537][20381] Updated weights for policy 1, policy_version 1308379 (0.0008) [2023-12-27 14:43:14,586][20381] Updated weights for policy 1, policy_version 1308389 (0.0008) [2023-12-27 14:43:14,647][20381] Updated weights for policy 1, policy_version 1308399 (0.0008) [2023-12-27 14:43:14,805][20347] Updated weights for policy 0, policy_version 1309283 (0.0007) [2023-12-27 14:43:14,866][20347] Updated weights for policy 0, policy_version 1309293 (0.0008) [2023-12-27 14:43:14,928][20347] Updated weights for policy 0, policy_version 1309303 (0.0009) [2023-12-27 14:43:15,302][20381] Updated weights for policy 1, policy_version 1308409 (0.0007) [2023-12-27 14:43:15,365][20381] Updated weights for policy 1, policy_version 1308419 (0.0008) [2023-12-27 14:43:15,421][20381] Updated weights for policy 1, policy_version 1308429 (0.0008) [2023-12-27 14:43:15,615][20347] Updated weights for policy 0, policy_version 1309313 (0.0008) [2023-12-27 14:43:15,670][20347] Updated weights for policy 0, policy_version 1309323 (0.0005) [2023-12-27 14:43:15,723][20347] Updated weights for policy 0, policy_version 1309333 (0.0005) [2023-12-27 14:43:15,772][20347] Updated weights for policy 0, policy_version 1309343 (0.0005) [2023-12-27 14:43:16,125][20381] Updated weights for policy 1, policy_version 1308439 (0.0006) [2023-12-27 14:43:16,188][20381] Updated weights for policy 1, policy_version 1308449 (0.0006) [2023-12-27 14:43:16,251][20381] Updated weights for policy 1, policy_version 1308459 (0.0006) [2023-12-27 14:43:16,338][20347] Updated weights for policy 0, policy_version 1309353 (0.0005) [2023-12-27 14:43:16,343][20291] KL-divergence is very high: 101.2521 [2023-12-27 14:43:16,392][20347] Updated weights for policy 0, policy_version 1309363 (0.0005) [2023-12-27 14:43:16,444][20347] Updated weights for policy 0, policy_version 1309373 (0.0006) [2023-12-27 14:43:16,915][20381] Updated weights for policy 1, policy_version 1308469 (0.0007) [2023-12-27 14:43:16,976][20381] Updated weights for policy 1, policy_version 1308479 (0.0007) [2023-12-27 14:43:17,027][20381] Updated weights for policy 1, policy_version 1308489 (0.0007) [2023-12-27 14:43:17,055][20347] Updated weights for policy 0, policy_version 1309383 (0.0006) [2023-12-27 14:43:17,106][20347] Updated weights for policy 0, policy_version 1309393 (0.0005) [2023-12-27 14:43:17,154][20347] Updated weights for policy 0, policy_version 1309403 (0.0005) [2023-12-27 14:43:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 670277632. Throughput: 0: 10241.6, 1: 10093.6. Samples: 670285056. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:17,363][19170] Avg episode reward: [(0, '736.030'), (1, '736.990')] [2023-12-27 14:43:17,716][20381] Updated weights for policy 1, policy_version 1308499 (0.0007) [2023-12-27 14:43:17,774][20381] Updated weights for policy 1, policy_version 1308509 (0.0008) [2023-12-27 14:43:17,803][20347] Updated weights for policy 0, policy_version 1309413 (0.0005) [2023-12-27 14:43:17,830][20381] Updated weights for policy 1, policy_version 1308519 (0.0008) [2023-12-27 14:43:17,857][20347] Updated weights for policy 0, policy_version 1309423 (0.0007) [2023-12-27 14:43:17,907][20347] Updated weights for policy 0, policy_version 1309433 (0.0007) [2023-12-27 14:43:18,529][20381] Updated weights for policy 1, policy_version 1308529 (0.0005) [2023-12-27 14:43:18,585][20381] Updated weights for policy 1, policy_version 1308539 (0.0008) [2023-12-27 14:43:18,598][20347] Updated weights for policy 0, policy_version 1309443 (0.0007) [2023-12-27 14:43:18,638][20381] Updated weights for policy 1, policy_version 1308549 (0.0005) [2023-12-27 14:43:18,651][20347] Updated weights for policy 0, policy_version 1309453 (0.0006) [2023-12-27 14:43:18,691][20381] Updated weights for policy 1, policy_version 1308559 (0.0005) [2023-12-27 14:43:18,703][20347] Updated weights for policy 0, policy_version 1309463 (0.0006) [2023-12-27 14:43:19,384][20381] Updated weights for policy 1, policy_version 1308569 (0.0007) [2023-12-27 14:43:19,439][20381] Updated weights for policy 1, policy_version 1308579 (0.0008) [2023-12-27 14:43:19,443][20347] Updated weights for policy 0, policy_version 1309473 (0.0007) [2023-12-27 14:43:19,497][20381] Updated weights for policy 1, policy_version 1308589 (0.0009) [2023-12-27 14:43:19,510][20347] Updated weights for policy 0, policy_version 1309483 (0.0007) [2023-12-27 14:43:19,566][20347] Updated weights for policy 0, policy_version 1309493 (0.0007) [2023-12-27 14:43:19,628][20347] Updated weights for policy 0, policy_version 1309503 (0.0008) [2023-12-27 14:43:20,182][20381] Updated weights for policy 1, policy_version 1308599 (0.0010) [2023-12-27 14:43:20,234][20381] Updated weights for policy 1, policy_version 1308609 (0.0010) [2023-12-27 14:43:20,295][20381] Updated weights for policy 1, policy_version 1308619 (0.0011) [2023-12-27 14:43:20,358][20347] Updated weights for policy 0, policy_version 1309513 (0.0007) [2023-12-27 14:43:20,416][20347] Updated weights for policy 0, policy_version 1309523 (0.0008) [2023-12-27 14:43:20,473][20347] Updated weights for policy 0, policy_version 1309533 (0.0006) [2023-12-27 14:43:20,945][20381] Updated weights for policy 1, policy_version 1308629 (0.0009) [2023-12-27 14:43:21,006][20381] Updated weights for policy 1, policy_version 1308639 (0.0007) [2023-12-27 14:43:21,073][20381] Updated weights for policy 1, policy_version 1308649 (0.0007) [2023-12-27 14:43:21,079][20347] Updated weights for policy 0, policy_version 1309543 (0.0009) [2023-12-27 14:43:21,145][20347] Updated weights for policy 0, policy_version 1309553 (0.0008) [2023-12-27 14:43:21,209][20347] Updated weights for policy 0, policy_version 1309563 (0.0008) [2023-12-27 14:43:21,783][20381] Updated weights for policy 1, policy_version 1308659 (0.0006) [2023-12-27 14:43:21,847][20381] Updated weights for policy 1, policy_version 1308669 (0.0008) [2023-12-27 14:43:21,907][20381] Updated weights for policy 1, policy_version 1308679 (0.0011) [2023-12-27 14:43:21,944][20347] Updated weights for policy 0, policy_version 1309573 (0.0008) [2023-12-27 14:43:22,003][20347] Updated weights for policy 0, policy_version 1309583 (0.0007) [2023-12-27 14:43:22,064][20347] Updated weights for policy 0, policy_version 1309593 (0.0005) [2023-12-27 14:43:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 670375936. Throughput: 0: 10237.8, 1: 10058.0. Samples: 670345112. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:22,363][19170] Avg episode reward: [(0, '755.680'), (1, '765.950')] [2023-12-27 14:43:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001308688_335052800.pth... [2023-12-27 14:43:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001309600_335323136.pth... [2023-12-27 14:43:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001307504_334749696.pth [2023-12-27 14:43:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001308416_335020032.pth [2023-12-27 14:43:22,587][20381] Updated weights for policy 1, policy_version 1308689 (0.0006) [2023-12-27 14:43:22,649][20381] Updated weights for policy 1, policy_version 1308699 (0.0007) [2023-12-27 14:43:22,712][20381] Updated weights for policy 1, policy_version 1308709 (0.0006) [2023-12-27 14:43:22,714][20347] Updated weights for policy 0, policy_version 1309603 (0.0006) [2023-12-27 14:43:22,770][20381] Updated weights for policy 1, policy_version 1308719 (0.0006) [2023-12-27 14:43:22,771][20347] Updated weights for policy 0, policy_version 1309613 (0.0008) [2023-12-27 14:43:22,828][20347] Updated weights for policy 0, policy_version 1309623 (0.0008) [2023-12-27 14:43:23,391][20381] Updated weights for policy 1, policy_version 1308729 (0.0007) [2023-12-27 14:43:23,451][20381] Updated weights for policy 1, policy_version 1308739 (0.0008) [2023-12-27 14:43:23,508][20381] Updated weights for policy 1, policy_version 1308749 (0.0008) [2023-12-27 14:43:23,534][20347] Updated weights for policy 0, policy_version 1309633 (0.0007) [2023-12-27 14:43:23,598][20347] Updated weights for policy 0, policy_version 1309643 (0.0005) [2023-12-27 14:43:23,654][20347] Updated weights for policy 0, policy_version 1309653 (0.0007) [2023-12-27 14:43:23,710][20347] Updated weights for policy 0, policy_version 1309663 (0.0007) [2023-12-27 14:43:24,218][20381] Updated weights for policy 1, policy_version 1308759 (0.0008) [2023-12-27 14:43:24,271][20381] Updated weights for policy 1, policy_version 1308769 (0.0008) [2023-12-27 14:43:24,327][20381] Updated weights for policy 1, policy_version 1308779 (0.0008) [2023-12-27 14:43:24,367][20347] Updated weights for policy 0, policy_version 1309673 (0.0007) [2023-12-27 14:43:24,422][20347] Updated weights for policy 0, policy_version 1309683 (0.0007) [2023-12-27 14:43:24,472][20347] Updated weights for policy 0, policy_version 1309693 (0.0006) [2023-12-27 14:43:24,968][20381] Updated weights for policy 1, policy_version 1308789 (0.0005) [2023-12-27 14:43:25,026][20381] Updated weights for policy 1, policy_version 1308799 (0.0006) [2023-12-27 14:43:25,076][20381] Updated weights for policy 1, policy_version 1308809 (0.0007) [2023-12-27 14:43:25,089][20347] Updated weights for policy 0, policy_version 1309703 (0.0006) [2023-12-27 14:43:25,135][20347] Updated weights for policy 0, policy_version 1309713 (0.0005) [2023-12-27 14:43:25,184][20347] Updated weights for policy 0, policy_version 1309723 (0.0005) [2023-12-27 14:43:25,767][20347] Updated weights for policy 0, policy_version 1309733 (0.0005) [2023-12-27 14:43:25,795][20381] Updated weights for policy 1, policy_version 1308819 (0.0008) [2023-12-27 14:43:25,828][20347] Updated weights for policy 0, policy_version 1309743 (0.0006) [2023-12-27 14:43:25,853][20381] Updated weights for policy 1, policy_version 1308829 (0.0008) [2023-12-27 14:43:25,888][20347] Updated weights for policy 0, policy_version 1309753 (0.0007) [2023-12-27 14:43:25,915][20381] Updated weights for policy 1, policy_version 1308839 (0.0006) [2023-12-27 14:43:26,429][20347] Updated weights for policy 0, policy_version 1309763 (0.0007) [2023-12-27 14:43:26,475][20347] Updated weights for policy 0, policy_version 1309773 (0.0006) [2023-12-27 14:43:26,522][20347] Updated weights for policy 0, policy_version 1309783 (0.0005) [2023-12-27 14:43:26,581][20381] Updated weights for policy 1, policy_version 1308849 (0.0008) [2023-12-27 14:43:26,633][20381] Updated weights for policy 1, policy_version 1308859 (0.0007) [2023-12-27 14:43:26,680][20381] Updated weights for policy 1, policy_version 1308869 (0.0007) [2023-12-27 14:43:26,724][20381] Updated weights for policy 1, policy_version 1308879 (0.0006) [2023-12-27 14:43:27,189][20347] Updated weights for policy 0, policy_version 1309793 (0.0005) [2023-12-27 14:43:27,249][20347] Updated weights for policy 0, policy_version 1309803 (0.0010) [2023-12-27 14:43:27,309][20347] Updated weights for policy 0, policy_version 1309813 (0.0010) [2023-12-27 14:43:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 670474240. Throughput: 0: 10283.3, 1: 10058.1. Samples: 670468932. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:27,363][19170] Avg episode reward: [(0, '699.300'), (1, '814.860')] [2023-12-27 14:43:27,368][20381] Updated weights for policy 1, policy_version 1308889 (0.0010) [2023-12-27 14:43:27,368][20347] Updated weights for policy 0, policy_version 1309823 (0.0010) [2023-12-27 14:43:27,423][20381] Updated weights for policy 1, policy_version 1308899 (0.0010) [2023-12-27 14:43:27,482][20381] Updated weights for policy 1, policy_version 1308909 (0.0010) [2023-12-27 14:43:28,015][20347] Updated weights for policy 0, policy_version 1309833 (0.0006) [2023-12-27 14:43:28,070][20347] Updated weights for policy 0, policy_version 1309843 (0.0005) [2023-12-27 14:43:28,109][20381] Updated weights for policy 1, policy_version 1308919 (0.0007) [2023-12-27 14:43:28,122][20347] Updated weights for policy 0, policy_version 1309853 (0.0009) [2023-12-27 14:43:28,163][20381] Updated weights for policy 1, policy_version 1308929 (0.0007) [2023-12-27 14:43:28,216][20381] Updated weights for policy 1, policy_version 1308939 (0.0008) [2023-12-27 14:43:28,765][20347] Updated weights for policy 0, policy_version 1309863 (0.0010) [2023-12-27 14:43:28,823][20347] Updated weights for policy 0, policy_version 1309873 (0.0010) [2023-12-27 14:43:28,883][20347] Updated weights for policy 0, policy_version 1309883 (0.0011) [2023-12-27 14:43:28,904][20381] Updated weights for policy 1, policy_version 1308949 (0.0010) [2023-12-27 14:43:28,965][20381] Updated weights for policy 1, policy_version 1308959 (0.0010) [2023-12-27 14:43:29,025][20381] Updated weights for policy 1, policy_version 1308969 (0.0010) [2023-12-27 14:43:29,570][20347] Updated weights for policy 0, policy_version 1309893 (0.0008) [2023-12-27 14:43:29,624][20347] Updated weights for policy 0, policy_version 1309903 (0.0005) [2023-12-27 14:43:29,666][20381] Updated weights for policy 1, policy_version 1308979 (0.0009) [2023-12-27 14:43:29,681][20347] Updated weights for policy 0, policy_version 1309913 (0.0005) [2023-12-27 14:43:29,721][20381] Updated weights for policy 1, policy_version 1308989 (0.0010) [2023-12-27 14:43:29,773][20381] Updated weights for policy 1, policy_version 1308999 (0.0008) [2023-12-27 14:43:30,296][20347] Updated weights for policy 0, policy_version 1309923 (0.0006) [2023-12-27 14:43:30,356][20347] Updated weights for policy 0, policy_version 1309933 (0.0007) [2023-12-27 14:43:30,400][20291] KL-divergence is very high: 200.8883 [2023-12-27 14:43:30,413][20347] Updated weights for policy 0, policy_version 1309943 (0.0008) [2023-12-27 14:43:30,436][20381] Updated weights for policy 1, policy_version 1309009 (0.0006) [2023-12-27 14:43:30,449][20291] KL-divergence is very high: 120.7864 [2023-12-27 14:43:30,492][20381] Updated weights for policy 1, policy_version 1309019 (0.0006) [2023-12-27 14:43:30,549][20381] Updated weights for policy 1, policy_version 1309029 (0.0010) [2023-12-27 14:43:30,606][20381] Updated weights for policy 1, policy_version 1309039 (0.0010) [2023-12-27 14:43:31,056][20347] Updated weights for policy 0, policy_version 1309953 (0.0007) [2023-12-27 14:43:31,105][20347] Updated weights for policy 0, policy_version 1309963 (0.0008) [2023-12-27 14:43:31,167][20347] Updated weights for policy 0, policy_version 1309973 (0.0008) [2023-12-27 14:43:31,224][20347] Updated weights for policy 0, policy_version 1309983 (0.0007) [2023-12-27 14:43:31,276][20381] Updated weights for policy 1, policy_version 1309049 (0.0007) [2023-12-27 14:43:31,340][20381] Updated weights for policy 1, policy_version 1309059 (0.0006) [2023-12-27 14:43:31,407][20381] Updated weights for policy 1, policy_version 1309069 (0.0007) [2023-12-27 14:43:32,010][20347] Updated weights for policy 0, policy_version 1309993 (0.0008) [2023-12-27 14:43:32,053][20381] Updated weights for policy 1, policy_version 1309079 (0.0007) [2023-12-27 14:43:32,067][20347] Updated weights for policy 0, policy_version 1310003 (0.0007) [2023-12-27 14:43:32,113][20381] Updated weights for policy 1, policy_version 1309089 (0.0007) [2023-12-27 14:43:32,128][20347] Updated weights for policy 0, policy_version 1310013 (0.0008) [2023-12-27 14:43:32,167][20381] Updated weights for policy 1, policy_version 1309099 (0.0005) [2023-12-27 14:43:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 670588928. Throughput: 0: 10355.9, 1: 10134.0. Samples: 670595116. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:32,363][19170] Avg episode reward: [(0, '688.830'), (1, '698.730')] [2023-12-27 14:43:32,806][20381] Updated weights for policy 1, policy_version 1309109 (0.0007) [2023-12-27 14:43:32,859][20347] Updated weights for policy 0, policy_version 1310023 (0.0005) [2023-12-27 14:43:32,866][20381] Updated weights for policy 1, policy_version 1309119 (0.0007) [2023-12-27 14:43:32,917][20381] Updated weights for policy 1, policy_version 1309129 (0.0007) [2023-12-27 14:43:32,925][20347] Updated weights for policy 0, policy_version 1310033 (0.0009) [2023-12-27 14:43:32,993][20347] Updated weights for policy 0, policy_version 1310043 (0.0008) [2023-12-27 14:43:33,584][20381] Updated weights for policy 1, policy_version 1309139 (0.0006) [2023-12-27 14:43:33,616][20347] Updated weights for policy 0, policy_version 1310053 (0.0006) [2023-12-27 14:43:33,636][20381] Updated weights for policy 1, policy_version 1309149 (0.0009) [2023-12-27 14:43:33,667][20347] Updated weights for policy 0, policy_version 1310063 (0.0005) [2023-12-27 14:43:33,687][20381] Updated weights for policy 1, policy_version 1309159 (0.0010) [2023-12-27 14:43:33,722][20347] Updated weights for policy 0, policy_version 1310073 (0.0006) [2023-12-27 14:43:34,346][20381] Updated weights for policy 1, policy_version 1309169 (0.0010) [2023-12-27 14:43:34,409][20381] Updated weights for policy 1, policy_version 1309179 (0.0010) [2023-12-27 14:43:34,422][20347] Updated weights for policy 0, policy_version 1310083 (0.0007) [2023-12-27 14:43:34,468][20381] Updated weights for policy 1, policy_version 1309189 (0.0010) [2023-12-27 14:43:34,482][20347] Updated weights for policy 0, policy_version 1310093 (0.0007) [2023-12-27 14:43:34,528][20381] Updated weights for policy 1, policy_version 1309199 (0.0011) [2023-12-27 14:43:34,549][20347] Updated weights for policy 0, policy_version 1310103 (0.0007) [2023-12-27 14:43:35,192][20347] Updated weights for policy 0, policy_version 1310113 (0.0008) [2023-12-27 14:43:35,201][20381] Updated weights for policy 1, policy_version 1309209 (0.0010) [2023-12-27 14:43:35,246][20347] Updated weights for policy 0, policy_version 1310123 (0.0006) [2023-12-27 14:43:35,247][20381] Updated weights for policy 1, policy_version 1309219 (0.0010) [2023-12-27 14:43:35,301][20347] Updated weights for policy 0, policy_version 1310133 (0.0005) [2023-12-27 14:43:35,302][20381] Updated weights for policy 1, policy_version 1309229 (0.0010) [2023-12-27 14:43:35,350][20347] Updated weights for policy 0, policy_version 1310143 (0.0007) [2023-12-27 14:43:35,994][20347] Updated weights for policy 0, policy_version 1310153 (0.0008) [2023-12-27 14:43:35,995][20381] Updated weights for policy 1, policy_version 1309239 (0.0008) [2023-12-27 14:43:36,045][20347] Updated weights for policy 0, policy_version 1310163 (0.0005) [2023-12-27 14:43:36,051][20381] Updated weights for policy 1, policy_version 1309249 (0.0010) [2023-12-27 14:43:36,101][20347] Updated weights for policy 0, policy_version 1310173 (0.0006) [2023-12-27 14:43:36,114][20381] Updated weights for policy 1, policy_version 1309259 (0.0011) [2023-12-27 14:43:36,745][20381] Updated weights for policy 1, policy_version 1309269 (0.0007) [2023-12-27 14:43:36,810][20381] Updated weights for policy 1, policy_version 1309279 (0.0007) [2023-12-27 14:43:36,845][20347] Updated weights for policy 0, policy_version 1310183 (0.0006) [2023-12-27 14:43:36,864][20381] Updated weights for policy 1, policy_version 1309289 (0.0007) [2023-12-27 14:43:36,904][20347] Updated weights for policy 0, policy_version 1310193 (0.0006) [2023-12-27 14:43:36,964][20347] Updated weights for policy 0, policy_version 1310203 (0.0006) [2023-12-27 14:43:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 670687232. Throughput: 0: 10317.0, 1: 10185.9. Samples: 670657156. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:37,363][19170] Avg episode reward: [(0, '650.230'), (1, '659.550')] [2023-12-27 14:43:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001309296_335208448.pth... [2023-12-27 14:43:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001310208_335478784.pth... [2023-12-27 14:43:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001308112_334905344.pth [2023-12-27 14:43:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001309024_335175680.pth [2023-12-27 14:43:37,518][20381] Updated weights for policy 1, policy_version 1309299 (0.0007) [2023-12-27 14:43:37,579][20381] Updated weights for policy 1, policy_version 1309309 (0.0008) [2023-12-27 14:43:37,619][20347] Updated weights for policy 0, policy_version 1310213 (0.0006) [2023-12-27 14:43:37,632][20381] Updated weights for policy 1, policy_version 1309319 (0.0006) [2023-12-27 14:43:37,673][20347] Updated weights for policy 0, policy_version 1310223 (0.0005) [2023-12-27 14:43:37,709][20291] KL-divergence is very high: 182.7999 [2023-12-27 14:43:37,723][20291] KL-divergence is very high: 203.4242 [2023-12-27 14:43:37,737][20347] Updated weights for policy 0, policy_version 1310233 (0.0007) [2023-12-27 14:43:37,765][20291] KL-divergence is very high: 194.3627 [2023-12-27 14:43:37,778][20291] KL-divergence is very high: 161.4837 [2023-12-27 14:43:38,261][20381] Updated weights for policy 1, policy_version 1309329 (0.0006) [2023-12-27 14:43:38,310][20381] Updated weights for policy 1, policy_version 1309339 (0.0006) [2023-12-27 14:43:38,375][20381] Updated weights for policy 1, policy_version 1309349 (0.0009) [2023-12-27 14:43:38,426][20347] Updated weights for policy 0, policy_version 1310243 (0.0005) [2023-12-27 14:43:38,426][20381] Updated weights for policy 1, policy_version 1309359 (0.0008) [2023-12-27 14:43:38,482][20347] Updated weights for policy 0, policy_version 1310253 (0.0006) [2023-12-27 14:43:38,538][20347] Updated weights for policy 0, policy_version 1310263 (0.0007) [2023-12-27 14:43:39,036][20381] Updated weights for policy 1, policy_version 1309369 (0.0008) [2023-12-27 14:43:39,096][20381] Updated weights for policy 1, policy_version 1309379 (0.0008) [2023-12-27 14:43:39,155][20381] Updated weights for policy 1, policy_version 1309389 (0.0008) [2023-12-27 14:43:39,207][20347] Updated weights for policy 0, policy_version 1310273 (0.0008) [2023-12-27 14:43:39,272][20347] Updated weights for policy 0, policy_version 1310283 (0.0010) [2023-12-27 14:43:39,333][20347] Updated weights for policy 0, policy_version 1310293 (0.0008) [2023-12-27 14:43:39,405][20347] Updated weights for policy 0, policy_version 1310303 (0.0008) [2023-12-27 14:43:39,879][20381] Updated weights for policy 1, policy_version 1309399 (0.0008) [2023-12-27 14:43:39,945][20381] Updated weights for policy 1, policy_version 1309409 (0.0008) [2023-12-27 14:43:40,006][20381] Updated weights for policy 1, policy_version 1309419 (0.0008) [2023-12-27 14:43:40,097][20347] Updated weights for policy 0, policy_version 1310313 (0.0008) [2023-12-27 14:43:40,153][20347] Updated weights for policy 0, policy_version 1310323 (0.0008) [2023-12-27 14:43:40,209][20347] Updated weights for policy 0, policy_version 1310333 (0.0007) [2023-12-27 14:43:40,638][20381] Updated weights for policy 1, policy_version 1309429 (0.0008) [2023-12-27 14:43:40,686][20381] Updated weights for policy 1, policy_version 1309439 (0.0008) [2023-12-27 14:43:40,744][20381] Updated weights for policy 1, policy_version 1309449 (0.0007) [2023-12-27 14:43:40,926][20347] Updated weights for policy 0, policy_version 1310343 (0.0007) [2023-12-27 14:43:40,979][20347] Updated weights for policy 0, policy_version 1310353 (0.0006) [2023-12-27 14:43:41,040][20347] Updated weights for policy 0, policy_version 1310363 (0.0009) [2023-12-27 14:43:41,449][20381] Updated weights for policy 1, policy_version 1309459 (0.0009) [2023-12-27 14:43:41,510][20381] Updated weights for policy 1, policy_version 1309469 (0.0008) [2023-12-27 14:43:41,561][20381] Updated weights for policy 1, policy_version 1309479 (0.0008) [2023-12-27 14:43:41,860][20347] Updated weights for policy 0, policy_version 1310373 (0.0009) [2023-12-27 14:43:41,921][20347] Updated weights for policy 0, policy_version 1310383 (0.0011) [2023-12-27 14:43:41,979][20347] Updated weights for policy 0, policy_version 1310393 (0.0007) [2023-12-27 14:43:42,296][20381] Updated weights for policy 1, policy_version 1309489 (0.0008) [2023-12-27 14:43:42,359][20381] Updated weights for policy 1, policy_version 1309499 (0.0008) [2023-12-27 14:43:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 670785536. Throughput: 0: 10205.6, 1: 10248.5. Samples: 670779836. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:42,363][19170] Avg episode reward: [(0, '738.240'), (1, '697.210')] [2023-12-27 14:43:42,421][20381] Updated weights for policy 1, policy_version 1309509 (0.0006) [2023-12-27 14:43:42,482][20381] Updated weights for policy 1, policy_version 1309519 (0.0006) [2023-12-27 14:43:42,643][20347] Updated weights for policy 0, policy_version 1310403 (0.0009) [2023-12-27 14:43:42,705][20347] Updated weights for policy 0, policy_version 1310413 (0.0010) [2023-12-27 14:43:42,766][20347] Updated weights for policy 0, policy_version 1310423 (0.0010) [2023-12-27 14:43:43,174][20381] Updated weights for policy 1, policy_version 1309529 (0.0007) [2023-12-27 14:43:43,223][20381] Updated weights for policy 1, policy_version 1309539 (0.0008) [2023-12-27 14:43:43,279][20381] Updated weights for policy 1, policy_version 1309549 (0.0007) [2023-12-27 14:43:43,451][20347] Updated weights for policy 0, policy_version 1310433 (0.0010) [2023-12-27 14:43:43,515][20347] Updated weights for policy 0, policy_version 1310443 (0.0009) [2023-12-27 14:43:43,574][20347] Updated weights for policy 0, policy_version 1310453 (0.0007) [2023-12-27 14:43:43,640][20347] Updated weights for policy 0, policy_version 1310463 (0.0008) [2023-12-27 14:43:44,010][20381] Updated weights for policy 1, policy_version 1309559 (0.0007) [2023-12-27 14:43:44,074][20381] Updated weights for policy 1, policy_version 1309569 (0.0008) [2023-12-27 14:43:44,129][20381] Updated weights for policy 1, policy_version 1309579 (0.0008) [2023-12-27 14:43:44,281][20347] Updated weights for policy 0, policy_version 1310473 (0.0005) [2023-12-27 14:43:44,337][20291] KL-divergence is very high: 102.4495 [2023-12-27 14:43:44,341][20347] Updated weights for policy 0, policy_version 1310483 (0.0006) [2023-12-27 14:43:44,403][20347] Updated weights for policy 0, policy_version 1310493 (0.0005) [2023-12-27 14:43:44,893][20381] Updated weights for policy 1, policy_version 1309589 (0.0008) [2023-12-27 14:43:44,947][20381] Updated weights for policy 1, policy_version 1309599 (0.0007) [2023-12-27 14:43:44,988][20347] Updated weights for policy 0, policy_version 1310503 (0.0006) [2023-12-27 14:43:45,010][20381] Updated weights for policy 1, policy_version 1309609 (0.0006) [2023-12-27 14:43:45,047][20347] Updated weights for policy 0, policy_version 1310513 (0.0007) [2023-12-27 14:43:45,109][20347] Updated weights for policy 0, policy_version 1310523 (0.0008) [2023-12-27 14:43:45,728][20381] Updated weights for policy 1, policy_version 1309619 (0.0008) [2023-12-27 14:43:45,763][20347] Updated weights for policy 0, policy_version 1310533 (0.0007) [2023-12-27 14:43:45,779][20381] Updated weights for policy 1, policy_version 1309629 (0.0009) [2023-12-27 14:43:45,815][20347] Updated weights for policy 0, policy_version 1310543 (0.0006) [2023-12-27 14:43:45,828][20381] Updated weights for policy 1, policy_version 1309639 (0.0006) [2023-12-27 14:43:45,858][20347] Updated weights for policy 0, policy_version 1310553 (0.0006) [2023-12-27 14:43:46,484][20381] Updated weights for policy 1, policy_version 1309649 (0.0007) [2023-12-27 14:43:46,542][20381] Updated weights for policy 1, policy_version 1309659 (0.0007) [2023-12-27 14:43:46,573][20347] Updated weights for policy 0, policy_version 1310563 (0.0007) [2023-12-27 14:43:46,607][20381] Updated weights for policy 1, policy_version 1309669 (0.0005) [2023-12-27 14:43:46,627][20347] Updated weights for policy 0, policy_version 1310573 (0.0007) [2023-12-27 14:43:46,659][20291] KL-divergence is very high: 117.6780 [2023-12-27 14:43:46,668][20381] Updated weights for policy 1, policy_version 1309679 (0.0005) [2023-12-27 14:43:46,683][20347] Updated weights for policy 0, policy_version 1310583 (0.0008) [2023-12-27 14:43:47,332][20381] Updated weights for policy 1, policy_version 1309689 (0.0007) [2023-12-27 14:43:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20438.4). Total num frames: 670883840. Throughput: 0: 10211.4, 1: 10166.6. Samples: 670899664. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:47,363][19170] Avg episode reward: [(0, '656.090'), (1, '726.810')] [2023-12-27 14:43:47,375][20347] Updated weights for policy 0, policy_version 1310593 (0.0010) [2023-12-27 14:43:47,386][20381] Updated weights for policy 1, policy_version 1309699 (0.0008) [2023-12-27 14:43:47,427][20347] Updated weights for policy 0, policy_version 1310603 (0.0005) [2023-12-27 14:43:47,439][20381] Updated weights for policy 1, policy_version 1309709 (0.0007) [2023-12-27 14:43:47,490][20347] Updated weights for policy 0, policy_version 1310613 (0.0006) [2023-12-27 14:43:47,551][20347] Updated weights for policy 0, policy_version 1310623 (0.0005) [2023-12-27 14:43:47,987][20381] Updated weights for policy 1, policy_version 1309719 (0.0005) [2023-12-27 14:43:48,036][20381] Updated weights for policy 1, policy_version 1309729 (0.0006) [2023-12-27 14:43:48,087][20381] Updated weights for policy 1, policy_version 1309739 (0.0007) [2023-12-27 14:43:48,135][20347] Updated weights for policy 0, policy_version 1310633 (0.0007) [2023-12-27 14:43:48,188][20347] Updated weights for policy 0, policy_version 1310643 (0.0008) [2023-12-27 14:43:48,248][20347] Updated weights for policy 0, policy_version 1310653 (0.0006) [2023-12-27 14:43:48,723][20381] Updated weights for policy 1, policy_version 1309749 (0.0009) [2023-12-27 14:43:48,779][20381] Updated weights for policy 1, policy_version 1309759 (0.0010) [2023-12-27 14:43:48,838][20381] Updated weights for policy 1, policy_version 1309769 (0.0011) [2023-12-27 14:43:48,876][20347] Updated weights for policy 0, policy_version 1310663 (0.0005) [2023-12-27 14:43:48,933][20291] KL-divergence is very high: 114.3095 [2023-12-27 14:43:48,934][20347] Updated weights for policy 0, policy_version 1310673 (0.0007) [2023-12-27 14:43:48,990][20347] Updated weights for policy 0, policy_version 1310683 (0.0007) [2023-12-27 14:43:49,540][20381] Updated weights for policy 1, policy_version 1309779 (0.0010) [2023-12-27 14:43:49,603][20381] Updated weights for policy 1, policy_version 1309789 (0.0007) [2023-12-27 14:43:49,661][20347] Updated weights for policy 0, policy_version 1310693 (0.0009) [2023-12-27 14:43:49,663][20381] Updated weights for policy 1, policy_version 1309799 (0.0007) [2023-12-27 14:43:49,716][20347] Updated weights for policy 0, policy_version 1310703 (0.0011) [2023-12-27 14:43:49,772][20347] Updated weights for policy 0, policy_version 1310713 (0.0010) [2023-12-27 14:43:50,338][20381] Updated weights for policy 1, policy_version 1309809 (0.0007) [2023-12-27 14:43:50,384][20381] Updated weights for policy 1, policy_version 1309819 (0.0006) [2023-12-27 14:43:50,435][20381] Updated weights for policy 1, policy_version 1309829 (0.0006) [2023-12-27 14:43:50,481][20381] Updated weights for policy 1, policy_version 1309839 (0.0006) [2023-12-27 14:43:50,529][20347] Updated weights for policy 0, policy_version 1310723 (0.0009) [2023-12-27 14:43:50,593][20347] Updated weights for policy 0, policy_version 1310733 (0.0008) [2023-12-27 14:43:50,646][20347] Updated weights for policy 0, policy_version 1310743 (0.0008) [2023-12-27 14:43:51,101][20381] Updated weights for policy 1, policy_version 1309849 (0.0006) [2023-12-27 14:43:51,165][20381] Updated weights for policy 1, policy_version 1309859 (0.0006) [2023-12-27 14:43:51,230][20381] Updated weights for policy 1, policy_version 1309869 (0.0006) [2023-12-27 14:43:51,328][20347] Updated weights for policy 0, policy_version 1310753 (0.0007) [2023-12-27 14:43:51,403][20347] Updated weights for policy 0, policy_version 1310763 (0.0008) [2023-12-27 14:43:51,458][20347] Updated weights for policy 0, policy_version 1310773 (0.0007) [2023-12-27 14:43:51,513][20347] Updated weights for policy 0, policy_version 1310783 (0.0006) [2023-12-27 14:43:51,882][20381] Updated weights for policy 1, policy_version 1309879 (0.0006) [2023-12-27 14:43:51,935][20381] Updated weights for policy 1, policy_version 1309889 (0.0006) [2023-12-27 14:43:51,996][20381] Updated weights for policy 1, policy_version 1309899 (0.0006) [2023-12-27 14:43:52,259][20347] Updated weights for policy 0, policy_version 1310793 (0.0007) [2023-12-27 14:43:52,318][20347] Updated weights for policy 0, policy_version 1310803 (0.0006) [2023-12-27 14:43:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 670990336. Throughput: 0: 10224.5, 1: 10178.8. Samples: 670963640. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 14:43:52,363][19170] Avg episode reward: [(0, '529.680'), (1, '719.040')] [2023-12-27 14:43:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001309904_335364096.pth... [2023-12-27 14:43:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001308688_335052800.pth [2023-12-27 14:43:52,384][20347] Updated weights for policy 0, policy_version 1310813 (0.0007) [2023-12-27 14:43:52,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001310816_335634432.pth... [2023-12-27 14:43:52,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001309600_335323136.pth [2023-12-27 14:43:52,617][20381] Updated weights for policy 1, policy_version 1309909 (0.0007) [2023-12-27 14:43:52,679][20381] Updated weights for policy 1, policy_version 1309919 (0.0008) [2023-12-27 14:43:52,734][20381] Updated weights for policy 1, policy_version 1309929 (0.0008) [2023-12-27 14:43:53,058][20347] Updated weights for policy 0, policy_version 1310823 (0.0008) [2023-12-27 14:43:53,112][20347] Updated weights for policy 0, policy_version 1310833 (0.0008) [2023-12-27 14:43:53,129][20291] KL-divergence is very high: 114.1254 [2023-12-27 14:43:53,166][20347] Updated weights for policy 0, policy_version 1310843 (0.0007) [2023-12-27 14:43:53,359][20381] Updated weights for policy 1, policy_version 1309939 (0.0006) [2023-12-27 14:43:53,408][20381] Updated weights for policy 1, policy_version 1309949 (0.0007) [2023-12-27 14:43:53,469][20381] Updated weights for policy 1, policy_version 1309959 (0.0006) [2023-12-27 14:43:53,875][20347] Updated weights for policy 0, policy_version 1310853 (0.0009) [2023-12-27 14:43:53,924][20347] Updated weights for policy 0, policy_version 1310863 (0.0010) [2023-12-27 14:43:53,968][20347] Updated weights for policy 0, policy_version 1310873 (0.0010) [2023-12-27 14:43:54,107][20381] Updated weights for policy 1, policy_version 1309969 (0.0006) [2023-12-27 14:43:54,164][20381] Updated weights for policy 1, policy_version 1309979 (0.0006) [2023-12-27 14:43:54,222][20381] Updated weights for policy 1, policy_version 1309989 (0.0006) [2023-12-27 14:43:54,273][20381] Updated weights for policy 1, policy_version 1309999 (0.0007) [2023-12-27 14:43:54,652][20347] Updated weights for policy 0, policy_version 1310883 (0.0009) [2023-12-27 14:43:54,703][20347] Updated weights for policy 0, policy_version 1310893 (0.0005) [2023-12-27 14:43:54,759][20347] Updated weights for policy 0, policy_version 1310903 (0.0006) [2023-12-27 14:43:54,883][20381] Updated weights for policy 1, policy_version 1310009 (0.0009) [2023-12-27 14:43:54,946][20381] Updated weights for policy 1, policy_version 1310019 (0.0007) [2023-12-27 14:43:55,010][20381] Updated weights for policy 1, policy_version 1310029 (0.0007) [2023-12-27 14:43:55,358][20347] Updated weights for policy 0, policy_version 1310913 (0.0007) [2023-12-27 14:43:55,416][20347] Updated weights for policy 0, policy_version 1310923 (0.0008) [2023-12-27 14:43:55,479][20347] Updated weights for policy 0, policy_version 1310933 (0.0006) [2023-12-27 14:43:55,530][20347] Updated weights for policy 0, policy_version 1310943 (0.0005) [2023-12-27 14:43:55,573][20381] Updated weights for policy 1, policy_version 1310039 (0.0007) [2023-12-27 14:43:55,619][20381] Updated weights for policy 1, policy_version 1310049 (0.0005) [2023-12-27 14:43:55,674][20381] Updated weights for policy 1, policy_version 1310059 (0.0005) [2023-12-27 14:43:56,115][20347] Updated weights for policy 0, policy_version 1310953 (0.0006) [2023-12-27 14:43:56,175][20347] Updated weights for policy 0, policy_version 1310963 (0.0006) [2023-12-27 14:43:56,233][20347] Updated weights for policy 0, policy_version 1310973 (0.0005) [2023-12-27 14:43:56,382][20381] Updated weights for policy 1, policy_version 1310069 (0.0007) [2023-12-27 14:43:56,444][20381] Updated weights for policy 1, policy_version 1310079 (0.0007) [2023-12-27 14:43:56,498][20381] Updated weights for policy 1, policy_version 1310089 (0.0008) [2023-12-27 14:43:56,784][20347] Updated weights for policy 0, policy_version 1310983 (0.0006) [2023-12-27 14:43:56,839][20347] Updated weights for policy 0, policy_version 1310993 (0.0007) [2023-12-27 14:43:56,893][20347] Updated weights for policy 0, policy_version 1311003 (0.0007) [2023-12-27 14:43:57,206][20381] Updated weights for policy 1, policy_version 1310099 (0.0008) [2023-12-27 14:43:57,257][20381] Updated weights for policy 1, policy_version 1310109 (0.0007) [2023-12-27 14:43:57,313][20381] Updated weights for policy 1, policy_version 1310119 (0.0007) [2023-12-27 14:43:57,363][19170] Fps is (10 sec: 22118.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 671105024. Throughput: 0: 10264.1, 1: 10296.3. Samples: 671090920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:43:57,364][19170] Avg episode reward: [(0, '618.980'), (1, '795.800')] [2023-12-27 14:43:57,550][20347] Updated weights for policy 0, policy_version 1311013 (0.0007) [2023-12-27 14:43:57,604][20347] Updated weights for policy 0, policy_version 1311023 (0.0007) [2023-12-27 14:43:57,648][20347] Updated weights for policy 0, policy_version 1311033 (0.0008) [2023-12-27 14:43:58,015][20381] Updated weights for policy 1, policy_version 1310129 (0.0007) [2023-12-27 14:43:58,060][20381] Updated weights for policy 1, policy_version 1310139 (0.0008) [2023-12-27 14:43:58,104][20381] Updated weights for policy 1, policy_version 1310149 (0.0008) [2023-12-27 14:43:58,160][20381] Updated weights for policy 1, policy_version 1310159 (0.0009) [2023-12-27 14:43:58,273][20347] Updated weights for policy 0, policy_version 1311043 (0.0006) [2023-12-27 14:43:58,336][20347] Updated weights for policy 0, policy_version 1311053 (0.0008) [2023-12-27 14:43:58,401][20347] Updated weights for policy 0, policy_version 1311063 (0.0008) [2023-12-27 14:43:58,970][20381] Updated weights for policy 1, policy_version 1310169 (0.0008) [2023-12-27 14:43:59,030][20381] Updated weights for policy 1, policy_version 1310179 (0.0008) [2023-12-27 14:43:59,079][20381] Updated weights for policy 1, policy_version 1310189 (0.0005) [2023-12-27 14:43:59,149][20347] Updated weights for policy 0, policy_version 1311073 (0.0008) [2023-12-27 14:43:59,207][20347] Updated weights for policy 0, policy_version 1311083 (0.0007) [2023-12-27 14:43:59,264][20347] Updated weights for policy 0, policy_version 1311093 (0.0008) [2023-12-27 14:43:59,326][20347] Updated weights for policy 0, policy_version 1311103 (0.0007) [2023-12-27 14:43:59,757][20381] Updated weights for policy 1, policy_version 1310199 (0.0009) [2023-12-27 14:43:59,820][20381] Updated weights for policy 1, policy_version 1310209 (0.0010) [2023-12-27 14:43:59,885][20381] Updated weights for policy 1, policy_version 1310219 (0.0010) [2023-12-27 14:44:00,006][20347] Updated weights for policy 0, policy_version 1311113 (0.0008) [2023-12-27 14:44:00,066][20347] Updated weights for policy 0, policy_version 1311123 (0.0007) [2023-12-27 14:44:00,128][20347] Updated weights for policy 0, policy_version 1311133 (0.0007) [2023-12-27 14:44:00,605][20381] Updated weights for policy 1, policy_version 1310229 (0.0009) [2023-12-27 14:44:00,653][20381] Updated weights for policy 1, policy_version 1310239 (0.0010) [2023-12-27 14:44:00,697][20381] Updated weights for policy 1, policy_version 1310249 (0.0010) [2023-12-27 14:44:00,738][20347] Updated weights for policy 0, policy_version 1311143 (0.0008) [2023-12-27 14:44:00,795][20347] Updated weights for policy 0, policy_version 1311153 (0.0007) [2023-12-27 14:44:00,853][20347] Updated weights for policy 0, policy_version 1311163 (0.0007) [2023-12-27 14:44:01,393][20381] Updated weights for policy 1, policy_version 1310259 (0.0009) [2023-12-27 14:44:01,446][20381] Updated weights for policy 1, policy_version 1310269 (0.0008) [2023-12-27 14:44:01,504][20381] Updated weights for policy 1, policy_version 1310279 (0.0008) [2023-12-27 14:44:01,585][20347] Updated weights for policy 0, policy_version 1311173 (0.0007) [2023-12-27 14:44:01,644][20347] Updated weights for policy 0, policy_version 1311183 (0.0007) [2023-12-27 14:44:01,702][20347] Updated weights for policy 0, policy_version 1311193 (0.0008) [2023-12-27 14:44:02,237][20381] Updated weights for policy 1, policy_version 1310289 (0.0009) [2023-12-27 14:44:02,298][20381] Updated weights for policy 1, policy_version 1310299 (0.0008) [2023-12-27 14:44:02,362][20381] Updated weights for policy 1, policy_version 1310309 (0.0008) [2023-12-27 14:44:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 671195136. Throughput: 0: 10290.4, 1: 10299.1. Samples: 671211584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:02,363][19170] Avg episode reward: [(0, '595.370'), (1, '814.800')] [2023-12-27 14:44:02,382][20347] Updated weights for policy 0, policy_version 1311203 (0.0008) [2023-12-27 14:44:02,411][20381] Updated weights for policy 1, policy_version 1310319 (0.0005) [2023-12-27 14:44:02,431][20347] Updated weights for policy 0, policy_version 1311213 (0.0008) [2023-12-27 14:44:02,485][20347] Updated weights for policy 0, policy_version 1311223 (0.0007) [2023-12-27 14:44:03,122][20381] Updated weights for policy 1, policy_version 1310329 (0.0007) [2023-12-27 14:44:03,131][20347] Updated weights for policy 0, policy_version 1311233 (0.0008) [2023-12-27 14:44:03,171][20381] Updated weights for policy 1, policy_version 1310339 (0.0006) [2023-12-27 14:44:03,194][20347] Updated weights for policy 0, policy_version 1311243 (0.0006) [2023-12-27 14:44:03,221][20381] Updated weights for policy 1, policy_version 1310349 (0.0006) [2023-12-27 14:44:03,249][20347] Updated weights for policy 0, policy_version 1311253 (0.0006) [2023-12-27 14:44:03,307][20347] Updated weights for policy 0, policy_version 1311263 (0.0008) [2023-12-27 14:44:03,853][20381] Updated weights for policy 1, policy_version 1310359 (0.0007) [2023-12-27 14:44:03,918][20381] Updated weights for policy 1, policy_version 1310369 (0.0007) [2023-12-27 14:44:03,985][20381] Updated weights for policy 1, policy_version 1310379 (0.0007) [2023-12-27 14:44:04,009][20347] Updated weights for policy 0, policy_version 1311273 (0.0007) [2023-12-27 14:44:04,073][20347] Updated weights for policy 0, policy_version 1311283 (0.0006) [2023-12-27 14:44:04,137][20347] Updated weights for policy 0, policy_version 1311293 (0.0008) [2023-12-27 14:44:04,685][20381] Updated weights for policy 1, policy_version 1310389 (0.0007) [2023-12-27 14:44:04,732][20347] Updated weights for policy 0, policy_version 1311303 (0.0006) [2023-12-27 14:44:04,738][20381] Updated weights for policy 1, policy_version 1310399 (0.0007) [2023-12-27 14:44:04,778][20347] Updated weights for policy 0, policy_version 1311313 (0.0005) [2023-12-27 14:44:04,783][20381] Updated weights for policy 1, policy_version 1310409 (0.0005) [2023-12-27 14:44:04,830][20347] Updated weights for policy 0, policy_version 1311323 (0.0006) [2023-12-27 14:44:05,484][20347] Updated weights for policy 0, policy_version 1311333 (0.0007) [2023-12-27 14:44:05,502][20381] Updated weights for policy 1, policy_version 1310419 (0.0005) [2023-12-27 14:44:05,541][20347] Updated weights for policy 0, policy_version 1311343 (0.0007) [2023-12-27 14:44:05,562][20381] Updated weights for policy 1, policy_version 1310429 (0.0006) [2023-12-27 14:44:05,592][20347] Updated weights for policy 0, policy_version 1311353 (0.0005) [2023-12-27 14:44:05,623][20381] Updated weights for policy 1, policy_version 1310439 (0.0007) [2023-12-27 14:44:06,208][20347] Updated weights for policy 0, policy_version 1311363 (0.0006) [2023-12-27 14:44:06,235][20381] Updated weights for policy 1, policy_version 1310449 (0.0006) [2023-12-27 14:44:06,268][20347] Updated weights for policy 0, policy_version 1311373 (0.0007) [2023-12-27 14:44:06,300][20381] Updated weights for policy 1, policy_version 1310459 (0.0011) [2023-12-27 14:44:06,333][20347] Updated weights for policy 0, policy_version 1311383 (0.0006) [2023-12-27 14:44:06,362][20381] Updated weights for policy 1, policy_version 1310469 (0.0006) [2023-12-27 14:44:06,433][20381] Updated weights for policy 1, policy_version 1310479 (0.0006) [2023-12-27 14:44:07,034][20347] Updated weights for policy 0, policy_version 1311393 (0.0007) [2023-12-27 14:44:07,069][20381] Updated weights for policy 1, policy_version 1310489 (0.0005) [2023-12-27 14:44:07,091][20347] Updated weights for policy 0, policy_version 1311403 (0.0008) [2023-12-27 14:44:07,124][20381] Updated weights for policy 1, policy_version 1310499 (0.0005) [2023-12-27 14:44:07,141][20347] Updated weights for policy 0, policy_version 1311413 (0.0007) [2023-12-27 14:44:07,175][20381] Updated weights for policy 1, policy_version 1310509 (0.0008) [2023-12-27 14:44:07,199][20347] Updated weights for policy 0, policy_version 1311423 (0.0006) [2023-12-27 14:44:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 671309824. Throughput: 0: 10321.4, 1: 10302.6. Samples: 671273192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:07,363][19170] Avg episode reward: [(0, '679.380'), (1, '766.380')] [2023-12-27 14:44:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001310512_335519744.pth... [2023-12-27 14:44:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001311424_335790080.pth... [2023-12-27 14:44:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001309296_335208448.pth [2023-12-27 14:44:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001310208_335478784.pth [2023-12-27 14:44:07,801][20381] Updated weights for policy 1, policy_version 1310519 (0.0010) [2023-12-27 14:44:07,857][20381] Updated weights for policy 1, policy_version 1310529 (0.0006) [2023-12-27 14:44:07,874][20347] Updated weights for policy 0, policy_version 1311433 (0.0010) [2023-12-27 14:44:07,912][20381] Updated weights for policy 1, policy_version 1310539 (0.0005) [2023-12-27 14:44:07,932][20347] Updated weights for policy 0, policy_version 1311443 (0.0010) [2023-12-27 14:44:07,981][20347] Updated weights for policy 0, policy_version 1311453 (0.0010) [2023-12-27 14:44:08,572][20381] Updated weights for policy 1, policy_version 1310549 (0.0006) [2023-12-27 14:44:08,639][20381] Updated weights for policy 1, policy_version 1310559 (0.0007) [2023-12-27 14:44:08,663][20347] Updated weights for policy 0, policy_version 1311463 (0.0010) [2023-12-27 14:44:08,699][20381] Updated weights for policy 1, policy_version 1310569 (0.0006) [2023-12-27 14:44:08,725][20347] Updated weights for policy 0, policy_version 1311473 (0.0008) [2023-12-27 14:44:08,777][20347] Updated weights for policy 0, policy_version 1311483 (0.0007) [2023-12-27 14:44:09,416][20347] Updated weights for policy 0, policy_version 1311493 (0.0008) [2023-12-27 14:44:09,475][20381] Updated weights for policy 1, policy_version 1310579 (0.0007) [2023-12-27 14:44:09,478][20347] Updated weights for policy 0, policy_version 1311503 (0.0006) [2023-12-27 14:44:09,533][20347] Updated weights for policy 0, policy_version 1311513 (0.0006) [2023-12-27 14:44:09,534][20381] Updated weights for policy 1, policy_version 1310589 (0.0009) [2023-12-27 14:44:09,594][20381] Updated weights for policy 1, policy_version 1310599 (0.0007) [2023-12-27 14:44:10,283][20347] Updated weights for policy 0, policy_version 1311523 (0.0006) [2023-12-27 14:44:10,288][20381] Updated weights for policy 1, policy_version 1310609 (0.0007) [2023-12-27 14:44:10,341][20381] Updated weights for policy 1, policy_version 1310619 (0.0007) [2023-12-27 14:44:10,343][20347] Updated weights for policy 0, policy_version 1311533 (0.0006) [2023-12-27 14:44:10,402][20381] Updated weights for policy 1, policy_version 1310629 (0.0007) [2023-12-27 14:44:10,404][20347] Updated weights for policy 0, policy_version 1311543 (0.0006) [2023-12-27 14:44:10,462][20381] Updated weights for policy 1, policy_version 1310639 (0.0008) [2023-12-27 14:44:10,970][20347] Updated weights for policy 0, policy_version 1311553 (0.0006) [2023-12-27 14:44:11,033][20347] Updated weights for policy 0, policy_version 1311563 (0.0010) [2023-12-27 14:44:11,101][20347] Updated weights for policy 0, policy_version 1311573 (0.0007) [2023-12-27 14:44:11,165][20347] Updated weights for policy 0, policy_version 1311583 (0.0007) [2023-12-27 14:44:11,186][20381] Updated weights for policy 1, policy_version 1310649 (0.0009) [2023-12-27 14:44:11,255][20381] Updated weights for policy 1, policy_version 1310659 (0.0008) [2023-12-27 14:44:11,317][20381] Updated weights for policy 1, policy_version 1310669 (0.0008) [2023-12-27 14:44:11,836][20347] Updated weights for policy 0, policy_version 1311593 (0.0010) [2023-12-27 14:44:11,899][20347] Updated weights for policy 0, policy_version 1311603 (0.0006) [2023-12-27 14:44:11,959][20347] Updated weights for policy 0, policy_version 1311613 (0.0009) [2023-12-27 14:44:12,056][20381] Updated weights for policy 1, policy_version 1310679 (0.0007) [2023-12-27 14:44:12,117][20381] Updated weights for policy 1, policy_version 1310689 (0.0007) [2023-12-27 14:44:12,178][20381] Updated weights for policy 1, policy_version 1310699 (0.0007) [2023-12-27 14:44:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 671408128. Throughput: 0: 10318.1, 1: 10312.2. Samples: 671397300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:12,363][19170] Avg episode reward: [(0, '708.640'), (1, '747.830')] [2023-12-27 14:44:12,623][20347] Updated weights for policy 0, policy_version 1311623 (0.0007) [2023-12-27 14:44:12,674][20347] Updated weights for policy 0, policy_version 1311633 (0.0006) [2023-12-27 14:44:12,725][20347] Updated weights for policy 0, policy_version 1311643 (0.0006) [2023-12-27 14:44:12,892][20381] Updated weights for policy 1, policy_version 1310709 (0.0007) [2023-12-27 14:44:12,940][20381] Updated weights for policy 1, policy_version 1310719 (0.0006) [2023-12-27 14:44:12,990][20381] Updated weights for policy 1, policy_version 1310729 (0.0006) [2023-12-27 14:44:13,344][20347] Updated weights for policy 0, policy_version 1311653 (0.0006) [2023-12-27 14:44:13,398][20347] Updated weights for policy 0, policy_version 1311663 (0.0005) [2023-12-27 14:44:13,446][20347] Updated weights for policy 0, policy_version 1311673 (0.0005) [2023-12-27 14:44:13,576][20381] Updated weights for policy 1, policy_version 1310739 (0.0005) [2023-12-27 14:44:13,627][20381] Updated weights for policy 1, policy_version 1310749 (0.0005) [2023-12-27 14:44:13,685][20381] Updated weights for policy 1, policy_version 1310759 (0.0006) [2023-12-27 14:44:14,085][20347] Updated weights for policy 0, policy_version 1311683 (0.0006) [2023-12-27 14:44:14,150][20347] Updated weights for policy 0, policy_version 1311693 (0.0007) [2023-12-27 14:44:14,218][20347] Updated weights for policy 0, policy_version 1311703 (0.0007) [2023-12-27 14:44:14,336][20381] Updated weights for policy 1, policy_version 1310769 (0.0008) [2023-12-27 14:44:14,396][20381] Updated weights for policy 1, policy_version 1310779 (0.0007) [2023-12-27 14:44:14,450][20381] Updated weights for policy 1, policy_version 1310789 (0.0007) [2023-12-27 14:44:14,506][20381] Updated weights for policy 1, policy_version 1310799 (0.0007) [2023-12-27 14:44:14,890][20347] Updated weights for policy 0, policy_version 1311713 (0.0007) [2023-12-27 14:44:14,949][20347] Updated weights for policy 0, policy_version 1311723 (0.0006) [2023-12-27 14:44:15,009][20347] Updated weights for policy 0, policy_version 1311733 (0.0007) [2023-12-27 14:44:15,062][20347] Updated weights for policy 0, policy_version 1311743 (0.0008) [2023-12-27 14:44:15,244][20381] Updated weights for policy 1, policy_version 1310809 (0.0008) [2023-12-27 14:44:15,298][20381] Updated weights for policy 1, policy_version 1310819 (0.0008) [2023-12-27 14:44:15,353][20381] Updated weights for policy 1, policy_version 1310829 (0.0007) [2023-12-27 14:44:15,754][20347] Updated weights for policy 0, policy_version 1311753 (0.0008) [2023-12-27 14:44:15,811][20347] Updated weights for policy 0, policy_version 1311763 (0.0010) [2023-12-27 14:44:15,861][20347] Updated weights for policy 0, policy_version 1311773 (0.0010) [2023-12-27 14:44:15,959][20381] Updated weights for policy 1, policy_version 1310839 (0.0007) [2023-12-27 14:44:16,011][20381] Updated weights for policy 1, policy_version 1310849 (0.0008) [2023-12-27 14:44:16,060][20381] Updated weights for policy 1, policy_version 1310859 (0.0009) [2023-12-27 14:44:16,530][20347] Updated weights for policy 0, policy_version 1311783 (0.0009) [2023-12-27 14:44:16,593][20347] Updated weights for policy 0, policy_version 1311793 (0.0010) [2023-12-27 14:44:16,644][20347] Updated weights for policy 0, policy_version 1311803 (0.0010) [2023-12-27 14:44:16,713][20381] Updated weights for policy 1, policy_version 1310869 (0.0009) [2023-12-27 14:44:16,766][20381] Updated weights for policy 1, policy_version 1310879 (0.0008) [2023-12-27 14:44:16,822][20381] Updated weights for policy 1, policy_version 1310889 (0.0010) [2023-12-27 14:44:17,209][20347] Updated weights for policy 0, policy_version 1311813 (0.0008) [2023-12-27 14:44:17,262][20347] Updated weights for policy 0, policy_version 1311823 (0.0005) [2023-12-27 14:44:17,319][20347] Updated weights for policy 0, policy_version 1311833 (0.0006) [2023-12-27 14:44:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 671514624. Throughput: 0: 10317.3, 1: 10277.1. Samples: 671521864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:17,363][19170] Avg episode reward: [(0, '686.840'), (1, '776.120')] [2023-12-27 14:44:17,487][20381] Updated weights for policy 1, policy_version 1310899 (0.0010) [2023-12-27 14:44:17,540][20381] Updated weights for policy 1, policy_version 1310909 (0.0007) [2023-12-27 14:44:17,587][20381] Updated weights for policy 1, policy_version 1310919 (0.0005) [2023-12-27 14:44:17,859][20347] Updated weights for policy 0, policy_version 1311843 (0.0010) [2023-12-27 14:44:17,910][20347] Updated weights for policy 0, policy_version 1311853 (0.0010) [2023-12-27 14:44:17,958][20347] Updated weights for policy 0, policy_version 1311863 (0.0010) [2023-12-27 14:44:18,130][20381] Updated weights for policy 1, policy_version 1310929 (0.0005) [2023-12-27 14:44:18,183][20381] Updated weights for policy 1, policy_version 1310939 (0.0005) [2023-12-27 14:44:18,242][20381] Updated weights for policy 1, policy_version 1310949 (0.0006) [2023-12-27 14:44:18,294][20381] Updated weights for policy 1, policy_version 1310959 (0.0005) [2023-12-27 14:44:18,640][20347] Updated weights for policy 0, policy_version 1311873 (0.0008) [2023-12-27 14:44:18,692][20347] Updated weights for policy 0, policy_version 1311883 (0.0007) [2023-12-27 14:44:18,741][20347] Updated weights for policy 0, policy_version 1311893 (0.0008) [2023-12-27 14:44:18,794][20347] Updated weights for policy 0, policy_version 1311903 (0.0008) [2023-12-27 14:44:18,939][20381] Updated weights for policy 1, policy_version 1310969 (0.0007) [2023-12-27 14:44:18,984][20381] Updated weights for policy 1, policy_version 1310979 (0.0008) [2023-12-27 14:44:19,044][20381] Updated weights for policy 1, policy_version 1310989 (0.0008) [2023-12-27 14:44:19,474][20347] Updated weights for policy 0, policy_version 1311913 (0.0008) [2023-12-27 14:44:19,534][20347] Updated weights for policy 0, policy_version 1311923 (0.0008) [2023-12-27 14:44:19,586][20347] Updated weights for policy 0, policy_version 1311933 (0.0008) [2023-12-27 14:44:19,796][20381] Updated weights for policy 1, policy_version 1310999 (0.0008) [2023-12-27 14:44:19,865][20381] Updated weights for policy 1, policy_version 1311009 (0.0008) [2023-12-27 14:44:19,926][20381] Updated weights for policy 1, policy_version 1311019 (0.0008) [2023-12-27 14:44:20,271][20347] Updated weights for policy 0, policy_version 1311943 (0.0006) [2023-12-27 14:44:20,326][20347] Updated weights for policy 0, policy_version 1311953 (0.0007) [2023-12-27 14:44:20,374][20347] Updated weights for policy 0, policy_version 1311963 (0.0007) [2023-12-27 14:44:20,654][20381] Updated weights for policy 1, policy_version 1311029 (0.0008) [2023-12-27 14:44:20,713][20381] Updated weights for policy 1, policy_version 1311039 (0.0008) [2023-12-27 14:44:20,779][20381] Updated weights for policy 1, policy_version 1311049 (0.0008) [2023-12-27 14:44:21,074][20347] Updated weights for policy 0, policy_version 1311973 (0.0008) [2023-12-27 14:44:21,145][20347] Updated weights for policy 0, policy_version 1311983 (0.0006) [2023-12-27 14:44:21,209][20347] Updated weights for policy 0, policy_version 1311993 (0.0008) [2023-12-27 14:44:21,588][20381] Updated weights for policy 1, policy_version 1311059 (0.0009) [2023-12-27 14:44:21,658][20381] Updated weights for policy 1, policy_version 1311069 (0.0008) [2023-12-27 14:44:21,718][20381] Updated weights for policy 1, policy_version 1311079 (0.0009) [2023-12-27 14:44:21,885][20347] Updated weights for policy 0, policy_version 1312003 (0.0008) [2023-12-27 14:44:21,943][20347] Updated weights for policy 0, policy_version 1312013 (0.0010) [2023-12-27 14:44:21,999][20347] Updated weights for policy 0, policy_version 1312023 (0.0010) [2023-12-27 14:44:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 671612928. Throughput: 0: 10357.0, 1: 10269.9. Samples: 671585368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:22,363][19170] Avg episode reward: [(0, '434.440'), (1, '834.220')] [2023-12-27 14:44:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001312032_335945728.pth... [2023-12-27 14:44:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001311088_335667200.pth... [2023-12-27 14:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001309904_335364096.pth [2023-12-27 14:44:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001310816_335634432.pth [2023-12-27 14:44:22,402][20381] Updated weights for policy 1, policy_version 1311089 (0.0008) [2023-12-27 14:44:22,467][20381] Updated weights for policy 1, policy_version 1311099 (0.0008) [2023-12-27 14:44:22,527][20381] Updated weights for policy 1, policy_version 1311109 (0.0011) [2023-12-27 14:44:22,590][20381] Updated weights for policy 1, policy_version 1311119 (0.0010) [2023-12-27 14:44:22,631][20347] Updated weights for policy 0, policy_version 1312033 (0.0010) [2023-12-27 14:44:22,696][20347] Updated weights for policy 0, policy_version 1312043 (0.0009) [2023-12-27 14:44:22,763][20347] Updated weights for policy 0, policy_version 1312053 (0.0010) [2023-12-27 14:44:22,831][20347] Updated weights for policy 0, policy_version 1312063 (0.0010) [2023-12-27 14:44:23,185][20381] Updated weights for policy 1, policy_version 1311129 (0.0009) [2023-12-27 14:44:23,243][20381] Updated weights for policy 1, policy_version 1311139 (0.0008) [2023-12-27 14:44:23,295][20381] Updated weights for policy 1, policy_version 1311149 (0.0006) [2023-12-27 14:44:23,505][20347] Updated weights for policy 0, policy_version 1312073 (0.0007) [2023-12-27 14:44:23,568][20347] Updated weights for policy 0, policy_version 1312083 (0.0006) [2023-12-27 14:44:23,622][20347] Updated weights for policy 0, policy_version 1312093 (0.0005) [2023-12-27 14:44:23,914][20381] Updated weights for policy 1, policy_version 1311159 (0.0007) [2023-12-27 14:44:23,976][20381] Updated weights for policy 1, policy_version 1311169 (0.0008) [2023-12-27 14:44:24,040][20381] Updated weights for policy 1, policy_version 1311179 (0.0007) [2023-12-27 14:44:24,192][20347] Updated weights for policy 0, policy_version 1312103 (0.0005) [2023-12-27 14:44:24,246][20347] Updated weights for policy 0, policy_version 1312113 (0.0007) [2023-12-27 14:44:24,300][20347] Updated weights for policy 0, policy_version 1312123 (0.0005) [2023-12-27 14:44:24,689][20381] Updated weights for policy 1, policy_version 1311189 (0.0008) [2023-12-27 14:44:24,747][20381] Updated weights for policy 1, policy_version 1311199 (0.0008) [2023-12-27 14:44:24,816][20381] Updated weights for policy 1, policy_version 1311209 (0.0010) [2023-12-27 14:44:24,930][20347] Updated weights for policy 0, policy_version 1312133 (0.0007) [2023-12-27 14:44:24,985][20347] Updated weights for policy 0, policy_version 1312143 (0.0007) [2023-12-27 14:44:25,035][20347] Updated weights for policy 0, policy_version 1312153 (0.0007) [2023-12-27 14:44:25,443][20381] Updated weights for policy 1, policy_version 1311219 (0.0009) [2023-12-27 14:44:25,497][20381] Updated weights for policy 1, policy_version 1311229 (0.0010) [2023-12-27 14:44:25,546][20381] Updated weights for policy 1, policy_version 1311239 (0.0010) [2023-12-27 14:44:25,757][20347] Updated weights for policy 0, policy_version 1312163 (0.0008) [2023-12-27 14:44:25,817][20347] Updated weights for policy 0, policy_version 1312173 (0.0006) [2023-12-27 14:44:25,882][20347] Updated weights for policy 0, policy_version 1312183 (0.0005) [2023-12-27 14:44:26,153][20381] Updated weights for policy 1, policy_version 1311249 (0.0010) [2023-12-27 14:44:26,204][20381] Updated weights for policy 1, policy_version 1311259 (0.0005) [2023-12-27 14:44:26,264][20381] Updated weights for policy 1, policy_version 1311269 (0.0005) [2023-12-27 14:44:26,321][20381] Updated weights for policy 1, policy_version 1311279 (0.0009) [2023-12-27 14:44:26,491][20347] Updated weights for policy 0, policy_version 1312193 (0.0007) [2023-12-27 14:44:26,541][20347] Updated weights for policy 0, policy_version 1312203 (0.0005) [2023-12-27 14:44:26,597][20347] Updated weights for policy 0, policy_version 1312213 (0.0006) [2023-12-27 14:44:26,649][20347] Updated weights for policy 0, policy_version 1312223 (0.0006) [2023-12-27 14:44:26,966][20381] Updated weights for policy 1, policy_version 1311289 (0.0010) [2023-12-27 14:44:27,024][20381] Updated weights for policy 1, policy_version 1311299 (0.0010) [2023-12-27 14:44:27,069][20381] Updated weights for policy 1, policy_version 1311309 (0.0010) [2023-12-27 14:44:27,250][20347] Updated weights for policy 0, policy_version 1312233 (0.0008) [2023-12-27 14:44:27,307][20347] Updated weights for policy 0, policy_version 1312243 (0.0007) [2023-12-27 14:44:27,361][20347] Updated weights for policy 0, policy_version 1312253 (0.0007) [2023-12-27 14:44:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 671719424. Throughput: 0: 10414.8, 1: 10260.9. Samples: 671710240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:27,363][19170] Avg episode reward: [(0, '465.600'), (1, '922.110')] [2023-12-27 14:44:27,668][20381] Updated weights for policy 1, policy_version 1311319 (0.0010) [2023-12-27 14:44:27,716][20381] Updated weights for policy 1, policy_version 1311329 (0.0010) [2023-12-27 14:44:27,770][20381] Updated weights for policy 1, policy_version 1311339 (0.0010) [2023-12-27 14:44:28,006][20347] Updated weights for policy 0, policy_version 1312263 (0.0006) [2023-12-27 14:44:28,063][20347] Updated weights for policy 0, policy_version 1312273 (0.0007) [2023-12-27 14:44:28,120][20347] Updated weights for policy 0, policy_version 1312283 (0.0007) [2023-12-27 14:44:28,467][20381] Updated weights for policy 1, policy_version 1311349 (0.0010) [2023-12-27 14:44:28,549][20381] Updated weights for policy 1, policy_version 1311359 (0.0010) [2023-12-27 14:44:28,604][20381] Updated weights for policy 1, policy_version 1311369 (0.0010) [2023-12-27 14:44:28,783][20347] Updated weights for policy 0, policy_version 1312293 (0.0008) [2023-12-27 14:44:28,833][20347] Updated weights for policy 0, policy_version 1312303 (0.0007) [2023-12-27 14:44:28,891][20347] Updated weights for policy 0, policy_version 1312313 (0.0007) [2023-12-27 14:44:29,228][20381] Updated weights for policy 1, policy_version 1311379 (0.0010) [2023-12-27 14:44:29,282][20381] Updated weights for policy 1, policy_version 1311389 (0.0006) [2023-12-27 14:44:29,355][20381] Updated weights for policy 1, policy_version 1311399 (0.0008) [2023-12-27 14:44:29,594][20347] Updated weights for policy 0, policy_version 1312323 (0.0008) [2023-12-27 14:44:29,644][20347] Updated weights for policy 0, policy_version 1312333 (0.0008) [2023-12-27 14:44:29,695][20347] Updated weights for policy 0, policy_version 1312343 (0.0007) [2023-12-27 14:44:29,988][20381] Updated weights for policy 1, policy_version 1311409 (0.0008) [2023-12-27 14:44:30,054][20381] Updated weights for policy 1, policy_version 1311419 (0.0008) [2023-12-27 14:44:30,118][20381] Updated weights for policy 1, policy_version 1311429 (0.0007) [2023-12-27 14:44:30,170][20381] Updated weights for policy 1, policy_version 1311439 (0.0007) [2023-12-27 14:44:30,416][20347] Updated weights for policy 0, policy_version 1312353 (0.0008) [2023-12-27 14:44:30,469][20347] Updated weights for policy 0, policy_version 1312363 (0.0008) [2023-12-27 14:44:30,524][20347] Updated weights for policy 0, policy_version 1312373 (0.0008) [2023-12-27 14:44:30,581][20347] Updated weights for policy 0, policy_version 1312383 (0.0007) [2023-12-27 14:44:30,810][20381] Updated weights for policy 1, policy_version 1311449 (0.0008) [2023-12-27 14:44:30,859][20381] Updated weights for policy 1, policy_version 1311459 (0.0010) [2023-12-27 14:44:30,905][20381] Updated weights for policy 1, policy_version 1311469 (0.0010) [2023-12-27 14:44:31,233][20347] Updated weights for policy 0, policy_version 1312393 (0.0007) [2023-12-27 14:44:31,292][20347] Updated weights for policy 0, policy_version 1312403 (0.0008) [2023-12-27 14:44:31,347][20347] Updated weights for policy 0, policy_version 1312413 (0.0006) [2023-12-27 14:44:31,643][20381] Updated weights for policy 1, policy_version 1311479 (0.0009) [2023-12-27 14:44:31,704][20381] Updated weights for policy 1, policy_version 1311489 (0.0008) [2023-12-27 14:44:31,762][20381] Updated weights for policy 1, policy_version 1311499 (0.0007) [2023-12-27 14:44:32,073][20347] Updated weights for policy 0, policy_version 1312423 (0.0008) [2023-12-27 14:44:32,138][20347] Updated weights for policy 0, policy_version 1312433 (0.0007) [2023-12-27 14:44:32,194][20347] Updated weights for policy 0, policy_version 1312443 (0.0008) [2023-12-27 14:44:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 671825920. Throughput: 0: 10441.0, 1: 10355.0. Samples: 671835484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:32,363][19170] Avg episode reward: [(0, '496.670'), (1, '922.000')] [2023-12-27 14:44:32,377][20381] Updated weights for policy 1, policy_version 1311509 (0.0007) [2023-12-27 14:44:32,436][20381] Updated weights for policy 1, policy_version 1311519 (0.0006) [2023-12-27 14:44:32,497][20381] Updated weights for policy 1, policy_version 1311529 (0.0005) [2023-12-27 14:44:32,936][20347] Updated weights for policy 0, policy_version 1312453 (0.0008) [2023-12-27 14:44:32,991][20347] Updated weights for policy 0, policy_version 1312463 (0.0008) [2023-12-27 14:44:33,039][20381] Updated weights for policy 1, policy_version 1311539 (0.0005) [2023-12-27 14:44:33,040][20347] Updated weights for policy 0, policy_version 1312473 (0.0007) [2023-12-27 14:44:33,087][20381] Updated weights for policy 1, policy_version 1311549 (0.0005) [2023-12-27 14:44:33,138][20381] Updated weights for policy 1, policy_version 1311559 (0.0005) [2023-12-27 14:44:33,680][20381] Updated weights for policy 1, policy_version 1311569 (0.0006) [2023-12-27 14:44:33,733][20381] Updated weights for policy 1, policy_version 1311579 (0.0008) [2023-12-27 14:44:33,778][20381] Updated weights for policy 1, policy_version 1311589 (0.0007) [2023-12-27 14:44:33,796][20347] Updated weights for policy 0, policy_version 1312483 (0.0007) [2023-12-27 14:44:33,828][20381] Updated weights for policy 1, policy_version 1311599 (0.0006) [2023-12-27 14:44:33,849][20347] Updated weights for policy 0, policy_version 1312493 (0.0006) [2023-12-27 14:44:33,911][20347] Updated weights for policy 0, policy_version 1312503 (0.0008) [2023-12-27 14:44:34,493][20381] Updated weights for policy 1, policy_version 1311609 (0.0006) [2023-12-27 14:44:34,550][20381] Updated weights for policy 1, policy_version 1311619 (0.0005) [2023-12-27 14:44:34,611][20381] Updated weights for policy 1, policy_version 1311629 (0.0005) [2023-12-27 14:44:34,656][20347] Updated weights for policy 0, policy_version 1312513 (0.0009) [2023-12-27 14:44:34,716][20347] Updated weights for policy 0, policy_version 1312523 (0.0008) [2023-12-27 14:44:34,776][20347] Updated weights for policy 0, policy_version 1312533 (0.0008) [2023-12-27 14:44:34,831][20347] Updated weights for policy 0, policy_version 1312543 (0.0008) [2023-12-27 14:44:35,137][20381] Updated weights for policy 1, policy_version 1311639 (0.0007) [2023-12-27 14:44:35,183][20381] Updated weights for policy 1, policy_version 1311649 (0.0008) [2023-12-27 14:44:35,239][20381] Updated weights for policy 1, policy_version 1311659 (0.0008) [2023-12-27 14:44:35,545][20347] Updated weights for policy 0, policy_version 1312553 (0.0007) [2023-12-27 14:44:35,599][20347] Updated weights for policy 0, policy_version 1312563 (0.0007) [2023-12-27 14:44:35,652][20347] Updated weights for policy 0, policy_version 1312573 (0.0007) [2023-12-27 14:44:35,945][20381] Updated weights for policy 1, policy_version 1311669 (0.0007) [2023-12-27 14:44:35,998][20381] Updated weights for policy 1, policy_version 1311679 (0.0007) [2023-12-27 14:44:36,050][20381] Updated weights for policy 1, policy_version 1311689 (0.0008) [2023-12-27 14:44:36,364][20347] Updated weights for policy 0, policy_version 1312583 (0.0007) [2023-12-27 14:44:36,420][20347] Updated weights for policy 0, policy_version 1312593 (0.0008) [2023-12-27 14:44:36,480][20347] Updated weights for policy 0, policy_version 1312603 (0.0008) [2023-12-27 14:44:36,754][20381] Updated weights for policy 1, policy_version 1311699 (0.0008) [2023-12-27 14:44:36,808][20381] Updated weights for policy 1, policy_version 1311709 (0.0007) [2023-12-27 14:44:36,857][20381] Updated weights for policy 1, policy_version 1311719 (0.0005) [2023-12-27 14:44:37,151][20347] Updated weights for policy 0, policy_version 1312613 (0.0007) [2023-12-27 14:44:37,214][20347] Updated weights for policy 0, policy_version 1312623 (0.0007) [2023-12-27 14:44:37,264][20347] Updated weights for policy 0, policy_version 1312633 (0.0009) [2023-12-27 14:44:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 671932416. Throughput: 0: 10359.0, 1: 10415.5. Samples: 671898492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:37,363][19170] Avg episode reward: [(0, '543.170'), (1, '893.230')] [2023-12-27 14:44:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001311728_335831040.pth... [2023-12-27 14:44:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001312640_336101376.pth... [2023-12-27 14:44:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001311424_335790080.pth [2023-12-27 14:44:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001310512_335519744.pth [2023-12-27 14:44:37,451][20381] Updated weights for policy 1, policy_version 1311729 (0.0006) [2023-12-27 14:44:37,509][20381] Updated weights for policy 1, policy_version 1311739 (0.0008) [2023-12-27 14:44:37,563][20381] Updated weights for policy 1, policy_version 1311749 (0.0008) [2023-12-27 14:44:37,614][20381] Updated weights for policy 1, policy_version 1311759 (0.0008) [2023-12-27 14:44:37,921][20347] Updated weights for policy 0, policy_version 1312643 (0.0008) [2023-12-27 14:44:37,975][20347] Updated weights for policy 0, policy_version 1312653 (0.0009) [2023-12-27 14:44:38,020][20347] Updated weights for policy 0, policy_version 1312663 (0.0010) [2023-12-27 14:44:38,368][20381] Updated weights for policy 1, policy_version 1311769 (0.0007) [2023-12-27 14:44:38,426][20381] Updated weights for policy 1, policy_version 1311779 (0.0007) [2023-12-27 14:44:38,480][20381] Updated weights for policy 1, policy_version 1311789 (0.0009) [2023-12-27 14:44:38,674][20347] Updated weights for policy 0, policy_version 1312673 (0.0010) [2023-12-27 14:44:38,726][20347] Updated weights for policy 0, policy_version 1312683 (0.0010) [2023-12-27 14:44:38,779][20347] Updated weights for policy 0, policy_version 1312693 (0.0007) [2023-12-27 14:44:38,837][20347] Updated weights for policy 0, policy_version 1312703 (0.0006) [2023-12-27 14:44:39,234][20381] Updated weights for policy 1, policy_version 1311799 (0.0008) [2023-12-27 14:44:39,291][20381] Updated weights for policy 1, policy_version 1311809 (0.0007) [2023-12-27 14:44:39,352][20381] Updated weights for policy 1, policy_version 1311819 (0.0008) [2023-12-27 14:44:39,519][20347] Updated weights for policy 0, policy_version 1312713 (0.0008) [2023-12-27 14:44:39,589][20347] Updated weights for policy 0, policy_version 1312723 (0.0008) [2023-12-27 14:44:39,650][20347] Updated weights for policy 0, policy_version 1312733 (0.0008) [2023-12-27 14:44:40,111][20381] Updated weights for policy 1, policy_version 1311829 (0.0007) [2023-12-27 14:44:40,170][20381] Updated weights for policy 1, policy_version 1311839 (0.0008) [2023-12-27 14:44:40,228][20381] Updated weights for policy 1, policy_version 1311849 (0.0008) [2023-12-27 14:44:40,379][20347] Updated weights for policy 0, policy_version 1312743 (0.0008) [2023-12-27 14:44:40,440][20347] Updated weights for policy 0, policy_version 1312753 (0.0008) [2023-12-27 14:44:40,497][20347] Updated weights for policy 0, policy_version 1312763 (0.0007) [2023-12-27 14:44:40,951][20381] Updated weights for policy 1, policy_version 1311859 (0.0008) [2023-12-27 14:44:41,001][20381] Updated weights for policy 1, policy_version 1311869 (0.0010) [2023-12-27 14:44:41,069][20381] Updated weights for policy 1, policy_version 1311879 (0.0010) [2023-12-27 14:44:41,200][20347] Updated weights for policy 0, policy_version 1312773 (0.0006) [2023-12-27 14:44:41,261][20347] Updated weights for policy 0, policy_version 1312783 (0.0007) [2023-12-27 14:44:41,319][20347] Updated weights for policy 0, policy_version 1312793 (0.0006) [2023-12-27 14:44:41,911][20381] Updated weights for policy 1, policy_version 1311889 (0.0009) [2023-12-27 14:44:41,971][20381] Updated weights for policy 1, policy_version 1311899 (0.0009) [2023-12-27 14:44:42,026][20381] Updated weights for policy 1, policy_version 1311909 (0.0008) [2023-12-27 14:44:42,032][20347] Updated weights for policy 0, policy_version 1312803 (0.0006) [2023-12-27 14:44:42,083][20381] Updated weights for policy 1, policy_version 1311919 (0.0005) [2023-12-27 14:44:42,088][20347] Updated weights for policy 0, policy_version 1312813 (0.0006) [2023-12-27 14:44:42,143][20347] Updated weights for policy 0, policy_version 1312823 (0.0007) [2023-12-27 14:44:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 672030720. Throughput: 0: 10348.0, 1: 10263.0. Samples: 672018412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:42,363][19170] Avg episode reward: [(0, '614.230'), (1, '902.890')] [2023-12-27 14:44:42,828][20347] Updated weights for policy 0, policy_version 1312833 (0.0007) [2023-12-27 14:44:42,853][20381] Updated weights for policy 1, policy_version 1311929 (0.0008) [2023-12-27 14:44:42,882][20347] Updated weights for policy 0, policy_version 1312843 (0.0009) [2023-12-27 14:44:42,914][20381] Updated weights for policy 1, policy_version 1311939 (0.0008) [2023-12-27 14:44:42,944][20347] Updated weights for policy 0, policy_version 1312853 (0.0005) [2023-12-27 14:44:42,964][20381] Updated weights for policy 1, policy_version 1311949 (0.0007) [2023-12-27 14:44:42,999][20347] Updated weights for policy 0, policy_version 1312863 (0.0009) [2023-12-27 14:44:43,606][20347] Updated weights for policy 0, policy_version 1312873 (0.0010) [2023-12-27 14:44:43,653][20347] Updated weights for policy 0, policy_version 1312883 (0.0010) [2023-12-27 14:44:43,667][20381] Updated weights for policy 1, policy_version 1311959 (0.0006) [2023-12-27 14:44:43,703][20347] Updated weights for policy 0, policy_version 1312893 (0.0009) [2023-12-27 14:44:43,712][20381] Updated weights for policy 1, policy_version 1311969 (0.0005) [2023-12-27 14:44:43,740][20292] KL-divergence is very high: 107.3942 [2023-12-27 14:44:43,763][20381] Updated weights for policy 1, policy_version 1311979 (0.0005) [2023-12-27 14:44:44,367][20381] Updated weights for policy 1, policy_version 1311989 (0.0006) [2023-12-27 14:44:44,420][20347] Updated weights for policy 0, policy_version 1312903 (0.0008) [2023-12-27 14:44:44,427][20381] Updated weights for policy 1, policy_version 1311999 (0.0008) [2023-12-27 14:44:44,480][20347] Updated weights for policy 0, policy_version 1312913 (0.0009) [2023-12-27 14:44:44,486][20381] Updated weights for policy 1, policy_version 1312009 (0.0007) [2023-12-27 14:44:44,538][20347] Updated weights for policy 0, policy_version 1312923 (0.0006) [2023-12-27 14:44:45,212][20381] Updated weights for policy 1, policy_version 1312019 (0.0007) [2023-12-27 14:44:45,255][20347] Updated weights for policy 0, policy_version 1312933 (0.0007) [2023-12-27 14:44:45,272][20381] Updated weights for policy 1, policy_version 1312029 (0.0008) [2023-12-27 14:44:45,314][20347] Updated weights for policy 0, policy_version 1312943 (0.0009) [2023-12-27 14:44:45,332][20381] Updated weights for policy 1, policy_version 1312039 (0.0008) [2023-12-27 14:44:45,371][20347] Updated weights for policy 0, policy_version 1312953 (0.0006) [2023-12-27 14:44:46,036][20347] Updated weights for policy 0, policy_version 1312963 (0.0007) [2023-12-27 14:44:46,039][20381] Updated weights for policy 1, policy_version 1312049 (0.0006) [2023-12-27 14:44:46,086][20347] Updated weights for policy 0, policy_version 1312973 (0.0005) [2023-12-27 14:44:46,094][20381] Updated weights for policy 1, policy_version 1312059 (0.0007) [2023-12-27 14:44:46,137][20347] Updated weights for policy 0, policy_version 1312983 (0.0007) [2023-12-27 14:44:46,149][20381] Updated weights for policy 1, policy_version 1312069 (0.0006) [2023-12-27 14:44:46,203][20381] Updated weights for policy 1, policy_version 1312079 (0.0008) [2023-12-27 14:44:46,823][20381] Updated weights for policy 1, policy_version 1312089 (0.0008) [2023-12-27 14:44:46,836][20347] Updated weights for policy 0, policy_version 1312993 (0.0008) [2023-12-27 14:44:46,871][20381] Updated weights for policy 1, policy_version 1312099 (0.0006) [2023-12-27 14:44:46,891][20347] Updated weights for policy 0, policy_version 1313003 (0.0006) [2023-12-27 14:44:46,930][20381] Updated weights for policy 1, policy_version 1312109 (0.0006) [2023-12-27 14:44:46,940][20347] Updated weights for policy 0, policy_version 1313013 (0.0006) [2023-12-27 14:44:46,988][20347] Updated weights for policy 0, policy_version 1313023 (0.0005) [2023-12-27 14:44:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 672129024. Throughput: 0: 10313.3, 1: 10284.1. Samples: 672138468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:47,363][19170] Avg episode reward: [(0, '641.480'), (1, '902.350')] [2023-12-27 14:44:47,592][20347] Updated weights for policy 0, policy_version 1313033 (0.0005) [2023-12-27 14:44:47,636][20347] Updated weights for policy 0, policy_version 1313043 (0.0007) [2023-12-27 14:44:47,666][20381] Updated weights for policy 1, policy_version 1312119 (0.0007) [2023-12-27 14:44:47,682][20347] Updated weights for policy 0, policy_version 1313053 (0.0007) [2023-12-27 14:44:47,722][20381] Updated weights for policy 1, policy_version 1312129 (0.0007) [2023-12-27 14:44:47,770][20381] Updated weights for policy 1, policy_version 1312139 (0.0007) [2023-12-27 14:44:48,379][20347] Updated weights for policy 0, policy_version 1313063 (0.0007) [2023-12-27 14:44:48,436][20347] Updated weights for policy 0, policy_version 1313073 (0.0008) [2023-12-27 14:44:48,463][20381] Updated weights for policy 1, policy_version 1312149 (0.0008) [2023-12-27 14:44:48,494][20347] Updated weights for policy 0, policy_version 1313083 (0.0007) [2023-12-27 14:44:48,514][20381] Updated weights for policy 1, policy_version 1312159 (0.0008) [2023-12-27 14:44:48,570][20381] Updated weights for policy 1, policy_version 1312169 (0.0008) [2023-12-27 14:44:49,118][20347] Updated weights for policy 0, policy_version 1313093 (0.0006) [2023-12-27 14:44:49,172][20347] Updated weights for policy 0, policy_version 1313103 (0.0006) [2023-12-27 14:44:49,229][20347] Updated weights for policy 0, policy_version 1313113 (0.0008) [2023-12-27 14:44:49,368][20381] Updated weights for policy 1, policy_version 1312179 (0.0008) [2023-12-27 14:44:49,420][20381] Updated weights for policy 1, policy_version 1312189 (0.0008) [2023-12-27 14:44:49,479][20381] Updated weights for policy 1, policy_version 1312199 (0.0009) [2023-12-27 14:44:49,890][20347] Updated weights for policy 0, policy_version 1313123 (0.0008) [2023-12-27 14:44:49,960][20347] Updated weights for policy 0, policy_version 1313133 (0.0008) [2023-12-27 14:44:50,016][20347] Updated weights for policy 0, policy_version 1313143 (0.0008) [2023-12-27 14:44:50,158][20381] Updated weights for policy 1, policy_version 1312209 (0.0008) [2023-12-27 14:44:50,218][20381] Updated weights for policy 1, policy_version 1312219 (0.0007) [2023-12-27 14:44:50,218][20292] KL-divergence is very high: 199.8023 [2023-12-27 14:44:50,265][20292] KL-divergence is very high: 221.2989 [2023-12-27 14:44:50,276][20381] Updated weights for policy 1, policy_version 1312229 (0.0008) [2023-12-27 14:44:50,313][20292] KL-divergence is very high: 125.1979 [2023-12-27 14:44:50,337][20381] Updated weights for policy 1, policy_version 1312239 (0.0008) [2023-12-27 14:44:50,691][20347] Updated weights for policy 0, policy_version 1313153 (0.0007) [2023-12-27 14:44:50,745][20347] Updated weights for policy 0, policy_version 1313163 (0.0008) [2023-12-27 14:44:50,802][20347] Updated weights for policy 0, policy_version 1313173 (0.0007) [2023-12-27 14:44:50,850][20347] Updated weights for policy 0, policy_version 1313183 (0.0006) [2023-12-27 14:44:51,077][20381] Updated weights for policy 1, policy_version 1312249 (0.0008) [2023-12-27 14:44:51,135][20381] Updated weights for policy 1, policy_version 1312259 (0.0008) [2023-12-27 14:44:51,200][20381] Updated weights for policy 1, policy_version 1312269 (0.0008) [2023-12-27 14:44:51,555][20347] Updated weights for policy 0, policy_version 1313193 (0.0008) [2023-12-27 14:44:51,614][20347] Updated weights for policy 0, policy_version 1313203 (0.0008) [2023-12-27 14:44:51,678][20347] Updated weights for policy 0, policy_version 1313213 (0.0008) [2023-12-27 14:44:51,936][20381] Updated weights for policy 1, policy_version 1312279 (0.0008) [2023-12-27 14:44:51,995][20381] Updated weights for policy 1, policy_version 1312289 (0.0008) [2023-12-27 14:44:52,060][20381] Updated weights for policy 1, policy_version 1312299 (0.0008) [2023-12-27 14:44:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 672227328. Throughput: 0: 10315.1, 1: 10263.8. Samples: 672199244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:52,363][19170] Avg episode reward: [(0, '542.980'), (1, '873.080')] [2023-12-27 14:44:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001312304_335978496.pth... [2023-12-27 14:44:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001311088_335667200.pth [2023-12-27 14:44:52,383][20347] Updated weights for policy 0, policy_version 1313223 (0.0007) [2023-12-27 14:44:52,444][20347] Updated weights for policy 0, policy_version 1313233 (0.0006) [2023-12-27 14:44:52,507][20347] Updated weights for policy 0, policy_version 1313243 (0.0007) [2023-12-27 14:44:52,534][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001313248_336257024.pth... [2023-12-27 14:44:52,538][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001312032_335945728.pth [2023-12-27 14:44:52,760][20381] Updated weights for policy 1, policy_version 1312309 (0.0007) [2023-12-27 14:44:52,806][20381] Updated weights for policy 1, policy_version 1312319 (0.0007) [2023-12-27 14:44:52,861][20381] Updated weights for policy 1, policy_version 1312329 (0.0007) [2023-12-27 14:44:53,123][20347] Updated weights for policy 0, policy_version 1313253 (0.0007) [2023-12-27 14:44:53,172][20347] Updated weights for policy 0, policy_version 1313263 (0.0007) [2023-12-27 14:44:53,220][20347] Updated weights for policy 0, policy_version 1313273 (0.0007) [2023-12-27 14:44:53,525][20381] Updated weights for policy 1, policy_version 1312339 (0.0007) [2023-12-27 14:44:53,583][20381] Updated weights for policy 1, policy_version 1312349 (0.0006) [2023-12-27 14:44:53,638][20381] Updated weights for policy 1, policy_version 1312359 (0.0008) [2023-12-27 14:44:53,899][20347] Updated weights for policy 0, policy_version 1313283 (0.0008) [2023-12-27 14:44:53,951][20347] Updated weights for policy 0, policy_version 1313293 (0.0010) [2023-12-27 14:44:54,006][20347] Updated weights for policy 0, policy_version 1313303 (0.0010) [2023-12-27 14:44:54,304][20381] Updated weights for policy 1, policy_version 1312369 (0.0007) [2023-12-27 14:44:54,352][20381] Updated weights for policy 1, policy_version 1312379 (0.0007) [2023-12-27 14:44:54,402][20381] Updated weights for policy 1, policy_version 1312389 (0.0007) [2023-12-27 14:44:54,455][20381] Updated weights for policy 1, policy_version 1312399 (0.0008) [2023-12-27 14:44:54,669][20347] Updated weights for policy 0, policy_version 1313313 (0.0010) [2023-12-27 14:44:54,713][20347] Updated weights for policy 0, policy_version 1313323 (0.0007) [2023-12-27 14:44:54,758][20347] Updated weights for policy 0, policy_version 1313333 (0.0007) [2023-12-27 14:44:54,815][20347] Updated weights for policy 0, policy_version 1313343 (0.0008) [2023-12-27 14:44:55,084][20381] Updated weights for policy 1, policy_version 1312409 (0.0006) [2023-12-27 14:44:55,132][20381] Updated weights for policy 1, policy_version 1312419 (0.0006) [2023-12-27 14:44:55,178][20381] Updated weights for policy 1, policy_version 1312429 (0.0008) [2023-12-27 14:44:55,504][20347] Updated weights for policy 0, policy_version 1313353 (0.0008) [2023-12-27 14:44:55,565][20347] Updated weights for policy 0, policy_version 1313363 (0.0008) [2023-12-27 14:44:55,623][20347] Updated weights for policy 0, policy_version 1313373 (0.0007) [2023-12-27 14:44:55,825][20381] Updated weights for policy 1, policy_version 1312439 (0.0007) [2023-12-27 14:44:55,875][20381] Updated weights for policy 1, policy_version 1312449 (0.0008) [2023-12-27 14:44:55,936][20381] Updated weights for policy 1, policy_version 1312459 (0.0005) [2023-12-27 14:44:56,193][20347] Updated weights for policy 0, policy_version 1313383 (0.0006) [2023-12-27 14:44:56,244][20347] Updated weights for policy 0, policy_version 1313393 (0.0005) [2023-12-27 14:44:56,293][20347] Updated weights for policy 0, policy_version 1313403 (0.0005) [2023-12-27 14:44:56,499][20381] Updated weights for policy 1, policy_version 1312469 (0.0008) [2023-12-27 14:44:56,557][20381] Updated weights for policy 1, policy_version 1312479 (0.0010) [2023-12-27 14:44:56,606][20381] Updated weights for policy 1, policy_version 1312489 (0.0010) [2023-12-27 14:44:56,865][20347] Updated weights for policy 0, policy_version 1313413 (0.0006) [2023-12-27 14:44:56,914][20347] Updated weights for policy 0, policy_version 1313423 (0.0007) [2023-12-27 14:44:56,968][20347] Updated weights for policy 0, policy_version 1313433 (0.0007) [2023-12-27 14:44:57,295][20381] Updated weights for policy 1, policy_version 1312499 (0.0009) [2023-12-27 14:44:57,348][20381] Updated weights for policy 1, policy_version 1312509 (0.0006) [2023-12-27 14:44:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 672333824. Throughput: 0: 10309.9, 1: 10270.3. Samples: 672323408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:44:57,363][19170] Avg episode reward: [(0, '359.640'), (1, '872.910')] [2023-12-27 14:44:57,402][20381] Updated weights for policy 1, policy_version 1312519 (0.0006) [2023-12-27 14:44:57,550][20347] Updated weights for policy 0, policy_version 1313443 (0.0006) [2023-12-27 14:44:57,603][20347] Updated weights for policy 0, policy_version 1313453 (0.0007) [2023-12-27 14:44:57,670][20347] Updated weights for policy 0, policy_version 1313463 (0.0009) [2023-12-27 14:44:57,981][20381] Updated weights for policy 1, policy_version 1312529 (0.0005) [2023-12-27 14:44:58,034][20381] Updated weights for policy 1, policy_version 1312539 (0.0005) [2023-12-27 14:44:58,084][20381] Updated weights for policy 1, policy_version 1312549 (0.0005) [2023-12-27 14:44:58,133][20381] Updated weights for policy 1, policy_version 1312559 (0.0005) [2023-12-27 14:44:58,502][20347] Updated weights for policy 0, policy_version 1313473 (0.0007) [2023-12-27 14:44:58,565][20347] Updated weights for policy 0, policy_version 1313483 (0.0006) [2023-12-27 14:44:58,628][20347] Updated weights for policy 0, policy_version 1313493 (0.0008) [2023-12-27 14:44:58,685][20347] Updated weights for policy 0, policy_version 1313503 (0.0007) [2023-12-27 14:44:58,864][20381] Updated weights for policy 1, policy_version 1312569 (0.0008) [2023-12-27 14:44:58,924][20381] Updated weights for policy 1, policy_version 1312579 (0.0008) [2023-12-27 14:44:58,984][20381] Updated weights for policy 1, policy_version 1312589 (0.0008) [2023-12-27 14:44:59,443][20347] Updated weights for policy 0, policy_version 1313513 (0.0005) [2023-12-27 14:44:59,500][20347] Updated weights for policy 0, policy_version 1313523 (0.0005) [2023-12-27 14:44:59,549][20347] Updated weights for policy 0, policy_version 1313533 (0.0005) [2023-12-27 14:44:59,767][20381] Updated weights for policy 1, policy_version 1312599 (0.0006) [2023-12-27 14:44:59,832][20381] Updated weights for policy 1, policy_version 1312609 (0.0007) [2023-12-27 14:44:59,890][20381] Updated weights for policy 1, policy_version 1312619 (0.0007) [2023-12-27 14:45:00,190][20347] Updated weights for policy 0, policy_version 1313543 (0.0006) [2023-12-27 14:45:00,244][20347] Updated weights for policy 0, policy_version 1313553 (0.0006) [2023-12-27 14:45:00,300][20347] Updated weights for policy 0, policy_version 1313563 (0.0007) [2023-12-27 14:45:00,546][20381] Updated weights for policy 1, policy_version 1312629 (0.0006) [2023-12-27 14:45:00,596][20381] Updated weights for policy 1, policy_version 1312639 (0.0005) [2023-12-27 14:45:00,653][20381] Updated weights for policy 1, policy_version 1312649 (0.0005) [2023-12-27 14:45:00,933][20347] Updated weights for policy 0, policy_version 1313573 (0.0007) [2023-12-27 14:45:00,985][20347] Updated weights for policy 0, policy_version 1313583 (0.0007) [2023-12-27 14:45:01,038][20347] Updated weights for policy 0, policy_version 1313593 (0.0008) [2023-12-27 14:45:01,320][20381] Updated weights for policy 1, policy_version 1312659 (0.0007) [2023-12-27 14:45:01,388][20381] Updated weights for policy 1, policy_version 1312669 (0.0008) [2023-12-27 14:45:01,448][20381] Updated weights for policy 1, policy_version 1312679 (0.0008) [2023-12-27 14:45:01,718][20347] Updated weights for policy 0, policy_version 1313603 (0.0007) [2023-12-27 14:45:01,782][20347] Updated weights for policy 0, policy_version 1313613 (0.0009) [2023-12-27 14:45:01,844][20347] Updated weights for policy 0, policy_version 1313623 (0.0007) [2023-12-27 14:45:02,120][20381] Updated weights for policy 1, policy_version 1312689 (0.0008) [2023-12-27 14:45:02,172][20381] Updated weights for policy 1, policy_version 1312699 (0.0008) [2023-12-27 14:45:02,230][20381] Updated weights for policy 1, policy_version 1312709 (0.0008) [2023-12-27 14:45:02,287][20381] Updated weights for policy 1, policy_version 1312719 (0.0008) [2023-12-27 14:45:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 672440320. Throughput: 0: 10297.7, 1: 10275.2. Samples: 672447644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:02,363][19170] Avg episode reward: [(0, '463.900'), (1, '882.800')] [2023-12-27 14:45:02,566][20347] Updated weights for policy 0, policy_version 1313633 (0.0006) [2023-12-27 14:45:02,614][20347] Updated weights for policy 0, policy_version 1313643 (0.0007) [2023-12-27 14:45:02,670][20347] Updated weights for policy 0, policy_version 1313653 (0.0008) [2023-12-27 14:45:02,721][20347] Updated weights for policy 0, policy_version 1313663 (0.0008) [2023-12-27 14:45:02,892][20381] Updated weights for policy 1, policy_version 1312729 (0.0008) [2023-12-27 14:45:02,938][20381] Updated weights for policy 1, policy_version 1312739 (0.0008) [2023-12-27 14:45:02,987][20381] Updated weights for policy 1, policy_version 1312749 (0.0007) [2023-12-27 14:45:03,387][20347] Updated weights for policy 0, policy_version 1313673 (0.0005) [2023-12-27 14:45:03,437][20347] Updated weights for policy 0, policy_version 1313683 (0.0005) [2023-12-27 14:45:03,488][20347] Updated weights for policy 0, policy_version 1313693 (0.0006) [2023-12-27 14:45:03,700][20381] Updated weights for policy 1, policy_version 1312759 (0.0007) [2023-12-27 14:45:03,754][20381] Updated weights for policy 1, policy_version 1312769 (0.0008) [2023-12-27 14:45:03,802][20381] Updated weights for policy 1, policy_version 1312779 (0.0008) [2023-12-27 14:45:04,080][20347] Updated weights for policy 0, policy_version 1313703 (0.0007) [2023-12-27 14:45:04,139][20347] Updated weights for policy 0, policy_version 1313713 (0.0007) [2023-12-27 14:45:04,202][20347] Updated weights for policy 0, policy_version 1313723 (0.0008) [2023-12-27 14:45:04,519][20381] Updated weights for policy 1, policy_version 1312789 (0.0007) [2023-12-27 14:45:04,572][20381] Updated weights for policy 1, policy_version 1312799 (0.0006) [2023-12-27 14:45:04,640][20381] Updated weights for policy 1, policy_version 1312809 (0.0008) [2023-12-27 14:45:04,827][20347] Updated weights for policy 0, policy_version 1313733 (0.0007) [2023-12-27 14:45:04,881][20347] Updated weights for policy 0, policy_version 1313743 (0.0007) [2023-12-27 14:45:04,934][20347] Updated weights for policy 0, policy_version 1313753 (0.0008) [2023-12-27 14:45:05,296][20381] Updated weights for policy 1, policy_version 1312819 (0.0007) [2023-12-27 14:45:05,349][20381] Updated weights for policy 1, policy_version 1312829 (0.0007) [2023-12-27 14:45:05,394][20381] Updated weights for policy 1, policy_version 1312839 (0.0008) [2023-12-27 14:45:05,627][20347] Updated weights for policy 0, policy_version 1313763 (0.0009) [2023-12-27 14:45:05,676][20347] Updated weights for policy 0, policy_version 1313773 (0.0008) [2023-12-27 14:45:05,724][20347] Updated weights for policy 0, policy_version 1313783 (0.0007) [2023-12-27 14:45:06,097][20381] Updated weights for policy 1, policy_version 1312849 (0.0007) [2023-12-27 14:45:06,163][20381] Updated weights for policy 1, policy_version 1312859 (0.0009) [2023-12-27 14:45:06,223][20381] Updated weights for policy 1, policy_version 1312869 (0.0008) [2023-12-27 14:45:06,286][20381] Updated weights for policy 1, policy_version 1312879 (0.0007) [2023-12-27 14:45:06,405][20347] Updated weights for policy 0, policy_version 1313793 (0.0007) [2023-12-27 14:45:06,461][20347] Updated weights for policy 0, policy_version 1313803 (0.0008) [2023-12-27 14:45:06,530][20347] Updated weights for policy 0, policy_version 1313813 (0.0008) [2023-12-27 14:45:06,592][20347] Updated weights for policy 0, policy_version 1313823 (0.0008) [2023-12-27 14:45:06,910][20381] Updated weights for policy 1, policy_version 1312889 (0.0006) [2023-12-27 14:45:06,967][20381] Updated weights for policy 1, policy_version 1312899 (0.0005) [2023-12-27 14:45:07,018][20381] Updated weights for policy 1, policy_version 1312909 (0.0005) [2023-12-27 14:45:07,354][20347] Updated weights for policy 0, policy_version 1313833 (0.0008) [2023-12-27 14:45:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 672538624. Throughput: 0: 10279.2, 1: 10278.1. Samples: 672510444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:07,363][19170] Avg episode reward: [(0, '456.680'), (1, '834.020')] [2023-12-27 14:45:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001312912_336134144.pth... [2023-12-27 14:45:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001311728_335831040.pth [2023-12-27 14:45:07,403][20347] Updated weights for policy 0, policy_version 1313843 (0.0009) [2023-12-27 14:45:07,448][20347] Updated weights for policy 0, policy_version 1313853 (0.0007) [2023-12-27 14:45:07,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001313856_336412672.pth... [2023-12-27 14:45:07,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001312640_336101376.pth [2023-12-27 14:45:07,533][20381] Updated weights for policy 1, policy_version 1312919 (0.0005) [2023-12-27 14:45:07,582][20381] Updated weights for policy 1, policy_version 1312929 (0.0006) [2023-12-27 14:45:07,637][20381] Updated weights for policy 1, policy_version 1312939 (0.0006) [2023-12-27 14:45:08,197][20381] Updated weights for policy 1, policy_version 1312949 (0.0006) [2023-12-27 14:45:08,224][20347] Updated weights for policy 0, policy_version 1313863 (0.0008) [2023-12-27 14:45:08,251][20381] Updated weights for policy 1, policy_version 1312959 (0.0008) [2023-12-27 14:45:08,281][20347] Updated weights for policy 0, policy_version 1313873 (0.0007) [2023-12-27 14:45:08,303][20381] Updated weights for policy 1, policy_version 1312969 (0.0007) [2023-12-27 14:45:08,327][20347] Updated weights for policy 0, policy_version 1313883 (0.0007) [2023-12-27 14:45:08,902][20381] Updated weights for policy 1, policy_version 1312979 (0.0008) [2023-12-27 14:45:08,958][20381] Updated weights for policy 1, policy_version 1312989 (0.0007) [2023-12-27 14:45:09,021][20381] Updated weights for policy 1, policy_version 1312999 (0.0007) [2023-12-27 14:45:09,058][20347] Updated weights for policy 0, policy_version 1313893 (0.0008) [2023-12-27 14:45:09,113][20347] Updated weights for policy 0, policy_version 1313903 (0.0007) [2023-12-27 14:45:09,164][20347] Updated weights for policy 0, policy_version 1313913 (0.0008) [2023-12-27 14:45:09,757][20381] Updated weights for policy 1, policy_version 1313009 (0.0005) [2023-12-27 14:45:09,816][20381] Updated weights for policy 1, policy_version 1313019 (0.0006) [2023-12-27 14:45:09,857][20347] Updated weights for policy 0, policy_version 1313923 (0.0007) [2023-12-27 14:45:09,885][20381] Updated weights for policy 1, policy_version 1313029 (0.0007) [2023-12-27 14:45:09,913][20347] Updated weights for policy 0, policy_version 1313933 (0.0008) [2023-12-27 14:45:09,943][20381] Updated weights for policy 1, policy_version 1313039 (0.0007) [2023-12-27 14:45:09,977][20347] Updated weights for policy 0, policy_version 1313943 (0.0008) [2023-12-27 14:45:10,637][20381] Updated weights for policy 1, policy_version 1313049 (0.0007) [2023-12-27 14:45:10,688][20347] Updated weights for policy 0, policy_version 1313953 (0.0008) [2023-12-27 14:45:10,689][20381] Updated weights for policy 1, policy_version 1313059 (0.0005) [2023-12-27 14:45:10,736][20381] Updated weights for policy 1, policy_version 1313069 (0.0005) [2023-12-27 14:45:10,740][20347] Updated weights for policy 0, policy_version 1313963 (0.0007) [2023-12-27 14:45:10,795][20347] Updated weights for policy 0, policy_version 1313973 (0.0008) [2023-12-27 14:45:10,854][20347] Updated weights for policy 0, policy_version 1313983 (0.0007) [2023-12-27 14:45:11,440][20381] Updated weights for policy 1, policy_version 1313079 (0.0008) [2023-12-27 14:45:11,508][20381] Updated weights for policy 1, policy_version 1313089 (0.0006) [2023-12-27 14:45:11,569][20381] Updated weights for policy 1, policy_version 1313099 (0.0008) [2023-12-27 14:45:11,678][20347] Updated weights for policy 0, policy_version 1313993 (0.0008) [2023-12-27 14:45:11,746][20347] Updated weights for policy 0, policy_version 1314003 (0.0008) [2023-12-27 14:45:11,808][20347] Updated weights for policy 0, policy_version 1314013 (0.0009) [2023-12-27 14:45:12,336][20381] Updated weights for policy 1, policy_version 1313109 (0.0007) [2023-12-27 14:45:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 672636928. Throughput: 0: 10193.7, 1: 10329.5. Samples: 672633784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:12,363][19170] Avg episode reward: [(0, '601.120'), (1, '834.040')] [2023-12-27 14:45:12,398][20381] Updated weights for policy 1, policy_version 1313119 (0.0009) [2023-12-27 14:45:12,451][20381] Updated weights for policy 1, policy_version 1313129 (0.0009) [2023-12-27 14:45:12,555][20347] Updated weights for policy 0, policy_version 1314023 (0.0006) [2023-12-27 14:45:12,606][20347] Updated weights for policy 0, policy_version 1314033 (0.0007) [2023-12-27 14:45:12,655][20347] Updated weights for policy 0, policy_version 1314043 (0.0005) [2023-12-27 14:45:13,155][20381] Updated weights for policy 1, policy_version 1313139 (0.0009) [2023-12-27 14:45:13,208][20381] Updated weights for policy 1, policy_version 1313149 (0.0007) [2023-12-27 14:45:13,263][20381] Updated weights for policy 1, policy_version 1313159 (0.0006) [2023-12-27 14:45:13,304][20347] Updated weights for policy 0, policy_version 1314053 (0.0007) [2023-12-27 14:45:13,359][20347] Updated weights for policy 0, policy_version 1314063 (0.0007) [2023-12-27 14:45:13,422][20347] Updated weights for policy 0, policy_version 1314073 (0.0007) [2023-12-27 14:45:13,838][20381] Updated weights for policy 1, policy_version 1313169 (0.0006) [2023-12-27 14:45:13,890][20381] Updated weights for policy 1, policy_version 1313179 (0.0007) [2023-12-27 14:45:13,947][20381] Updated weights for policy 1, policy_version 1313189 (0.0005) [2023-12-27 14:45:14,001][20381] Updated weights for policy 1, policy_version 1313199 (0.0007) [2023-12-27 14:45:14,111][20347] Updated weights for policy 0, policy_version 1314083 (0.0007) [2023-12-27 14:45:14,166][20347] Updated weights for policy 0, policy_version 1314093 (0.0007) [2023-12-27 14:45:14,219][20347] Updated weights for policy 0, policy_version 1314103 (0.0007) [2023-12-27 14:45:14,660][20381] Updated weights for policy 1, policy_version 1313209 (0.0008) [2023-12-27 14:45:14,716][20381] Updated weights for policy 1, policy_version 1313219 (0.0011) [2023-12-27 14:45:14,782][20381] Updated weights for policy 1, policy_version 1313229 (0.0011) [2023-12-27 14:45:14,893][20347] Updated weights for policy 0, policy_version 1314113 (0.0009) [2023-12-27 14:45:14,954][20347] Updated weights for policy 0, policy_version 1314123 (0.0007) [2023-12-27 14:45:15,020][20347] Updated weights for policy 0, policy_version 1314133 (0.0008) [2023-12-27 14:45:15,081][20347] Updated weights for policy 0, policy_version 1314143 (0.0007) [2023-12-27 14:45:15,484][20381] Updated weights for policy 1, policy_version 1313239 (0.0009) [2023-12-27 14:45:15,547][20381] Updated weights for policy 1, policy_version 1313249 (0.0008) [2023-12-27 14:45:15,602][20381] Updated weights for policy 1, policy_version 1313259 (0.0007) [2023-12-27 14:45:15,746][20347] Updated weights for policy 0, policy_version 1314153 (0.0006) [2023-12-27 14:45:15,806][20347] Updated weights for policy 0, policy_version 1314163 (0.0005) [2023-12-27 14:45:15,855][20347] Updated weights for policy 0, policy_version 1314173 (0.0005) [2023-12-27 14:45:16,309][20381] Updated weights for policy 1, policy_version 1313269 (0.0006) [2023-12-27 14:45:16,363][20381] Updated weights for policy 1, policy_version 1313279 (0.0007) [2023-12-27 14:45:16,425][20381] Updated weights for policy 1, policy_version 1313289 (0.0008) [2023-12-27 14:45:16,523][20347] Updated weights for policy 0, policy_version 1314183 (0.0007) [2023-12-27 14:45:16,570][20347] Updated weights for policy 0, policy_version 1314193 (0.0007) [2023-12-27 14:45:16,626][20347] Updated weights for policy 0, policy_version 1314203 (0.0008) [2023-12-27 14:45:17,100][20381] Updated weights for policy 1, policy_version 1313299 (0.0008) [2023-12-27 14:45:17,149][20381] Updated weights for policy 1, policy_version 1313309 (0.0011) [2023-12-27 14:45:17,199][20381] Updated weights for policy 1, policy_version 1313319 (0.0011) [2023-12-27 14:45:17,233][20347] Updated weights for policy 0, policy_version 1314213 (0.0007) [2023-12-27 14:45:17,287][20347] Updated weights for policy 0, policy_version 1314223 (0.0007) [2023-12-27 14:45:17,344][20347] Updated weights for policy 0, policy_version 1314233 (0.0007) [2023-12-27 14:45:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 672743424. Throughput: 0: 10152.1, 1: 10259.4. Samples: 672754008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:17,364][19170] Avg episode reward: [(0, '673.880'), (1, '873.120')] [2023-12-27 14:45:17,855][20381] Updated weights for policy 1, policy_version 1313329 (0.0010) [2023-12-27 14:45:17,908][20381] Updated weights for policy 1, policy_version 1313339 (0.0005) [2023-12-27 14:45:17,960][20381] Updated weights for policy 1, policy_version 1313349 (0.0007) [2023-12-27 14:45:18,014][20381] Updated weights for policy 1, policy_version 1313359 (0.0007) [2023-12-27 14:45:18,059][20347] Updated weights for policy 0, policy_version 1314243 (0.0008) [2023-12-27 14:45:18,115][20347] Updated weights for policy 0, policy_version 1314253 (0.0007) [2023-12-27 14:45:18,170][20347] Updated weights for policy 0, policy_version 1314263 (0.0008) [2023-12-27 14:45:18,729][20381] Updated weights for policy 1, policy_version 1313369 (0.0007) [2023-12-27 14:45:18,787][20381] Updated weights for policy 1, policy_version 1313379 (0.0005) [2023-12-27 14:45:18,844][20381] Updated weights for policy 1, policy_version 1313389 (0.0005) [2023-12-27 14:45:18,911][20347] Updated weights for policy 0, policy_version 1314273 (0.0007) [2023-12-27 14:45:18,963][20347] Updated weights for policy 0, policy_version 1314283 (0.0008) [2023-12-27 14:45:19,018][20347] Updated weights for policy 0, policy_version 1314293 (0.0007) [2023-12-27 14:45:19,095][20347] Updated weights for policy 0, policy_version 1314303 (0.0008) [2023-12-27 14:45:19,508][20381] Updated weights for policy 1, policy_version 1313399 (0.0007) [2023-12-27 14:45:19,566][20381] Updated weights for policy 1, policy_version 1313409 (0.0010) [2023-12-27 14:45:19,620][20381] Updated weights for policy 1, policy_version 1313419 (0.0008) [2023-12-27 14:45:19,868][20347] Updated weights for policy 0, policy_version 1314313 (0.0008) [2023-12-27 14:45:19,929][20347] Updated weights for policy 0, policy_version 1314323 (0.0007) [2023-12-27 14:45:19,992][20347] Updated weights for policy 0, policy_version 1314333 (0.0008) [2023-12-27 14:45:20,351][20381] Updated weights for policy 1, policy_version 1313429 (0.0008) [2023-12-27 14:45:20,402][20381] Updated weights for policy 1, policy_version 1313439 (0.0008) [2023-12-27 14:45:20,459][20381] Updated weights for policy 1, policy_version 1313449 (0.0007) [2023-12-27 14:45:20,714][20347] Updated weights for policy 0, policy_version 1314343 (0.0007) [2023-12-27 14:45:20,776][20347] Updated weights for policy 0, policy_version 1314353 (0.0008) [2023-12-27 14:45:20,838][20347] Updated weights for policy 0, policy_version 1314363 (0.0008) [2023-12-27 14:45:21,155][20381] Updated weights for policy 1, policy_version 1313459 (0.0008) [2023-12-27 14:45:21,208][20381] Updated weights for policy 1, policy_version 1313469 (0.0007) [2023-12-27 14:45:21,272][20381] Updated weights for policy 1, policy_version 1313479 (0.0008) [2023-12-27 14:45:21,571][20347] Updated weights for policy 0, policy_version 1314373 (0.0007) [2023-12-27 14:45:21,639][20347] Updated weights for policy 0, policy_version 1314383 (0.0008) [2023-12-27 14:45:21,703][20347] Updated weights for policy 0, policy_version 1314393 (0.0006) [2023-12-27 14:45:21,984][20381] Updated weights for policy 1, policy_version 1313489 (0.0007) [2023-12-27 14:45:22,042][20381] Updated weights for policy 1, policy_version 1313499 (0.0007) [2023-12-27 14:45:22,095][20381] Updated weights for policy 1, policy_version 1313509 (0.0007) [2023-12-27 14:45:22,153][20381] Updated weights for policy 1, policy_version 1313519 (0.0007) [2023-12-27 14:45:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 672841728. Throughput: 0: 10172.8, 1: 10169.3. Samples: 672813884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:22,363][19170] Avg episode reward: [(0, '691.280'), (1, '912.220')] [2023-12-27 14:45:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001314400_336551936.pth... [2023-12-27 14:45:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001313520_336289792.pth... [2023-12-27 14:45:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001312304_335978496.pth [2023-12-27 14:45:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001313248_336257024.pth [2023-12-27 14:45:22,453][20347] Updated weights for policy 0, policy_version 1314403 (0.0008) [2023-12-27 14:45:22,513][20347] Updated weights for policy 0, policy_version 1314413 (0.0008) [2023-12-27 14:45:22,566][20347] Updated weights for policy 0, policy_version 1314423 (0.0008) [2023-12-27 14:45:22,838][20381] Updated weights for policy 1, policy_version 1313529 (0.0008) [2023-12-27 14:45:22,887][20381] Updated weights for policy 1, policy_version 1313539 (0.0008) [2023-12-27 14:45:22,952][20381] Updated weights for policy 1, policy_version 1313549 (0.0007) [2023-12-27 14:45:23,246][20347] Updated weights for policy 0, policy_version 1314433 (0.0009) [2023-12-27 14:45:23,308][20347] Updated weights for policy 0, policy_version 1314443 (0.0008) [2023-12-27 14:45:23,364][20347] Updated weights for policy 0, policy_version 1314453 (0.0007) [2023-12-27 14:45:23,427][20347] Updated weights for policy 0, policy_version 1314463 (0.0008) [2023-12-27 14:45:23,591][20381] Updated weights for policy 1, policy_version 1313559 (0.0007) [2023-12-27 14:45:23,647][20381] Updated weights for policy 1, policy_version 1313569 (0.0007) [2023-12-27 14:45:23,696][20381] Updated weights for policy 1, policy_version 1313579 (0.0005) [2023-12-27 14:45:24,155][20347] Updated weights for policy 0, policy_version 1314473 (0.0007) [2023-12-27 14:45:24,220][20347] Updated weights for policy 0, policy_version 1314483 (0.0008) [2023-12-27 14:45:24,285][20347] Updated weights for policy 0, policy_version 1314493 (0.0006) [2023-12-27 14:45:24,320][20381] Updated weights for policy 1, policy_version 1313589 (0.0006) [2023-12-27 14:45:24,371][20381] Updated weights for policy 1, policy_version 1313599 (0.0007) [2023-12-27 14:45:24,422][20381] Updated weights for policy 1, policy_version 1313609 (0.0007) [2023-12-27 14:45:24,956][20347] Updated weights for policy 0, policy_version 1314503 (0.0007) [2023-12-27 14:45:25,002][20347] Updated weights for policy 0, policy_version 1314513 (0.0010) [2023-12-27 14:45:25,027][20381] Updated weights for policy 1, policy_version 1313619 (0.0008) [2023-12-27 14:45:25,049][20347] Updated weights for policy 0, policy_version 1314523 (0.0007) [2023-12-27 14:45:25,072][20381] Updated weights for policy 1, policy_version 1313629 (0.0009) [2023-12-27 14:45:25,120][20381] Updated weights for policy 1, policy_version 1313639 (0.0007) [2023-12-27 14:45:25,750][20347] Updated weights for policy 0, policy_version 1314533 (0.0008) [2023-12-27 14:45:25,806][20347] Updated weights for policy 0, policy_version 1314543 (0.0006) [2023-12-27 14:45:25,819][20381] Updated weights for policy 1, policy_version 1313649 (0.0008) [2023-12-27 14:45:25,853][20347] Updated weights for policy 0, policy_version 1314553 (0.0005) [2023-12-27 14:45:25,874][20381] Updated weights for policy 1, policy_version 1313659 (0.0008) [2023-12-27 14:45:25,921][20381] Updated weights for policy 1, policy_version 1313669 (0.0008) [2023-12-27 14:45:25,974][20381] Updated weights for policy 1, policy_version 1313679 (0.0009) [2023-12-27 14:45:26,420][20347] Updated weights for policy 0, policy_version 1314563 (0.0006) [2023-12-27 14:45:26,472][20347] Updated weights for policy 0, policy_version 1314573 (0.0007) [2023-12-27 14:45:26,526][20347] Updated weights for policy 0, policy_version 1314583 (0.0007) [2023-12-27 14:45:26,678][20381] Updated weights for policy 1, policy_version 1313689 (0.0005) [2023-12-27 14:45:26,725][20381] Updated weights for policy 1, policy_version 1313699 (0.0005) [2023-12-27 14:45:26,777][20381] Updated weights for policy 1, policy_version 1313709 (0.0006) [2023-12-27 14:45:27,263][20347] Updated weights for policy 0, policy_version 1314593 (0.0007) [2023-12-27 14:45:27,313][20347] Updated weights for policy 0, policy_version 1314603 (0.0007) [2023-12-27 14:45:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 672940032. Throughput: 0: 10127.5, 1: 10237.4. Samples: 672934832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:27,363][19170] Avg episode reward: [(0, '748.680'), (1, '892.650')] [2023-12-27 14:45:27,364][20347] Updated weights for policy 0, policy_version 1314613 (0.0007) [2023-12-27 14:45:27,396][20381] Updated weights for policy 1, policy_version 1313719 (0.0010) [2023-12-27 14:45:27,412][20347] Updated weights for policy 0, policy_version 1314623 (0.0005) [2023-12-27 14:45:27,445][20381] Updated weights for policy 1, policy_version 1313729 (0.0011) [2023-12-27 14:45:27,494][20381] Updated weights for policy 1, policy_version 1313739 (0.0011) [2023-12-27 14:45:28,059][20381] Updated weights for policy 1, policy_version 1313749 (0.0011) [2023-12-27 14:45:28,062][20347] Updated weights for policy 0, policy_version 1314633 (0.0006) [2023-12-27 14:45:28,107][20381] Updated weights for policy 1, policy_version 1313759 (0.0010) [2023-12-27 14:45:28,116][20347] Updated weights for policy 0, policy_version 1314643 (0.0005) [2023-12-27 14:45:28,154][20381] Updated weights for policy 1, policy_version 1313769 (0.0010) [2023-12-27 14:45:28,170][20347] Updated weights for policy 0, policy_version 1314653 (0.0005) [2023-12-27 14:45:28,184][20292] KL-divergence is very high: 355.7996 [2023-12-27 14:45:28,813][20381] Updated weights for policy 1, policy_version 1313779 (0.0010) [2023-12-27 14:45:28,879][20381] Updated weights for policy 1, policy_version 1313789 (0.0011) [2023-12-27 14:45:28,893][20347] Updated weights for policy 0, policy_version 1314663 (0.0006) [2023-12-27 14:45:28,942][20381] Updated weights for policy 1, policy_version 1313799 (0.0011) [2023-12-27 14:45:28,948][20347] Updated weights for policy 0, policy_version 1314673 (0.0006) [2023-12-27 14:45:28,997][20347] Updated weights for policy 0, policy_version 1314683 (0.0005) [2023-12-27 14:45:29,619][20381] Updated weights for policy 1, policy_version 1313809 (0.0008) [2023-12-27 14:45:29,674][20381] Updated weights for policy 1, policy_version 1313819 (0.0006) [2023-12-27 14:45:29,733][20381] Updated weights for policy 1, policy_version 1313829 (0.0007) [2023-12-27 14:45:29,749][20347] Updated weights for policy 0, policy_version 1314693 (0.0006) [2023-12-27 14:45:29,794][20381] Updated weights for policy 1, policy_version 1313839 (0.0006) [2023-12-27 14:45:29,817][20347] Updated weights for policy 0, policy_version 1314703 (0.0006) [2023-12-27 14:45:29,890][20347] Updated weights for policy 0, policy_version 1314713 (0.0008) [2023-12-27 14:45:30,440][20381] Updated weights for policy 1, policy_version 1313849 (0.0008) [2023-12-27 14:45:30,502][20381] Updated weights for policy 1, policy_version 1313859 (0.0008) [2023-12-27 14:45:30,561][20381] Updated weights for policy 1, policy_version 1313869 (0.0008) [2023-12-27 14:45:30,577][20347] Updated weights for policy 0, policy_version 1314723 (0.0008) [2023-12-27 14:45:30,634][20347] Updated weights for policy 0, policy_version 1314733 (0.0010) [2023-12-27 14:45:30,700][20347] Updated weights for policy 0, policy_version 1314743 (0.0009) [2023-12-27 14:45:31,271][20381] Updated weights for policy 1, policy_version 1313879 (0.0008) [2023-12-27 14:45:31,326][20381] Updated weights for policy 1, policy_version 1313889 (0.0008) [2023-12-27 14:45:31,390][20381] Updated weights for policy 1, policy_version 1313899 (0.0008) [2023-12-27 14:45:31,405][20347] Updated weights for policy 0, policy_version 1314753 (0.0008) [2023-12-27 14:45:31,462][20347] Updated weights for policy 0, policy_version 1314763 (0.0007) [2023-12-27 14:45:31,523][20347] Updated weights for policy 0, policy_version 1314773 (0.0007) [2023-12-27 14:45:31,580][20347] Updated weights for policy 0, policy_version 1314783 (0.0008) [2023-12-27 14:45:32,086][20381] Updated weights for policy 1, policy_version 1313909 (0.0007) [2023-12-27 14:45:32,146][20381] Updated weights for policy 1, policy_version 1313919 (0.0005) [2023-12-27 14:45:32,202][20381] Updated weights for policy 1, policy_version 1313929 (0.0006) [2023-12-27 14:45:32,315][20347] Updated weights for policy 0, policy_version 1314793 (0.0010) [2023-12-27 14:45:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 673046528. Throughput: 0: 10112.5, 1: 10318.1. Samples: 673057840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:32,363][19170] Avg episode reward: [(0, '634.120'), (1, '902.510')] [2023-12-27 14:45:32,376][20347] Updated weights for policy 0, policy_version 1314803 (0.0010) [2023-12-27 14:45:32,437][20347] Updated weights for policy 0, policy_version 1314813 (0.0008) [2023-12-27 14:45:32,828][20381] Updated weights for policy 1, policy_version 1313939 (0.0006) [2023-12-27 14:45:32,888][20381] Updated weights for policy 1, policy_version 1313949 (0.0005) [2023-12-27 14:45:32,946][20381] Updated weights for policy 1, policy_version 1313959 (0.0005) [2023-12-27 14:45:33,068][20347] Updated weights for policy 0, policy_version 1314823 (0.0007) [2023-12-27 14:45:33,126][20347] Updated weights for policy 0, policy_version 1314833 (0.0007) [2023-12-27 14:45:33,186][20347] Updated weights for policy 0, policy_version 1314843 (0.0008) [2023-12-27 14:45:33,573][20381] Updated weights for policy 1, policy_version 1313969 (0.0008) [2023-12-27 14:45:33,624][20381] Updated weights for policy 1, policy_version 1313979 (0.0009) [2023-12-27 14:45:33,685][20381] Updated weights for policy 1, policy_version 1313989 (0.0006) [2023-12-27 14:45:33,735][20381] Updated weights for policy 1, policy_version 1313999 (0.0007) [2023-12-27 14:45:33,802][20347] Updated weights for policy 0, policy_version 1314853 (0.0009) [2023-12-27 14:45:33,860][20347] Updated weights for policy 0, policy_version 1314863 (0.0010) [2023-12-27 14:45:33,921][20347] Updated weights for policy 0, policy_version 1314873 (0.0010) [2023-12-27 14:45:34,416][20381] Updated weights for policy 1, policy_version 1314009 (0.0010) [2023-12-27 14:45:34,483][20381] Updated weights for policy 1, policy_version 1314019 (0.0011) [2023-12-27 14:45:34,544][20381] Updated weights for policy 1, policy_version 1314029 (0.0010) [2023-12-27 14:45:34,647][20347] Updated weights for policy 0, policy_version 1314883 (0.0008) [2023-12-27 14:45:34,710][20347] Updated weights for policy 0, policy_version 1314893 (0.0008) [2023-12-27 14:45:34,773][20347] Updated weights for policy 0, policy_version 1314903 (0.0008) [2023-12-27 14:45:35,255][20381] Updated weights for policy 1, policy_version 1314039 (0.0011) [2023-12-27 14:45:35,304][20381] Updated weights for policy 1, policy_version 1314049 (0.0010) [2023-12-27 14:45:35,359][20381] Updated weights for policy 1, policy_version 1314059 (0.0010) [2023-12-27 14:45:35,398][20347] Updated weights for policy 0, policy_version 1314913 (0.0007) [2023-12-27 14:45:35,453][20347] Updated weights for policy 0, policy_version 1314923 (0.0007) [2023-12-27 14:45:35,508][20347] Updated weights for policy 0, policy_version 1314933 (0.0008) [2023-12-27 14:45:35,559][20347] Updated weights for policy 0, policy_version 1314943 (0.0007) [2023-12-27 14:45:36,052][20381] Updated weights for policy 1, policy_version 1314069 (0.0010) [2023-12-27 14:45:36,112][20381] Updated weights for policy 1, policy_version 1314079 (0.0010) [2023-12-27 14:45:36,177][20381] Updated weights for policy 1, policy_version 1314089 (0.0009) [2023-12-27 14:45:36,299][20347] Updated weights for policy 0, policy_version 1314953 (0.0009) [2023-12-27 14:45:36,364][20347] Updated weights for policy 0, policy_version 1314963 (0.0011) [2023-12-27 14:45:36,426][20347] Updated weights for policy 0, policy_version 1314973 (0.0010) [2023-12-27 14:45:36,869][20381] Updated weights for policy 1, policy_version 1314099 (0.0008) [2023-12-27 14:45:36,928][20381] Updated weights for policy 1, policy_version 1314109 (0.0007) [2023-12-27 14:45:36,985][20381] Updated weights for policy 1, policy_version 1314119 (0.0007) [2023-12-27 14:45:37,128][20347] Updated weights for policy 0, policy_version 1314983 (0.0008) [2023-12-27 14:45:37,182][20347] Updated weights for policy 0, policy_version 1314993 (0.0008) [2023-12-27 14:45:37,233][20347] Updated weights for policy 0, policy_version 1315003 (0.0007) [2023-12-27 14:45:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 673153024. Throughput: 0: 10095.1, 1: 10362.3. Samples: 673119828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:37,363][19170] Avg episode reward: [(0, '519.630'), (1, '941.480')] [2023-12-27 14:45:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001314128_336445440.pth... [2023-12-27 14:45:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001315008_336707584.pth... [2023-12-27 14:45:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001312912_336134144.pth [2023-12-27 14:45:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001313856_336412672.pth [2023-12-27 14:45:37,698][20381] Updated weights for policy 1, policy_version 1314129 (0.0007) [2023-12-27 14:45:37,757][20381] Updated weights for policy 1, policy_version 1314139 (0.0007) [2023-12-27 14:45:37,819][20381] Updated weights for policy 1, policy_version 1314149 (0.0008) [2023-12-27 14:45:37,876][20381] Updated weights for policy 1, policy_version 1314159 (0.0007) [2023-12-27 14:45:37,957][20347] Updated weights for policy 0, policy_version 1315013 (0.0008) [2023-12-27 14:45:38,010][20347] Updated weights for policy 0, policy_version 1315023 (0.0008) [2023-12-27 14:45:38,062][20347] Updated weights for policy 0, policy_version 1315033 (0.0006) [2023-12-27 14:45:38,579][20381] Updated weights for policy 1, policy_version 1314169 (0.0008) [2023-12-27 14:45:38,633][20381] Updated weights for policy 1, policy_version 1314179 (0.0008) [2023-12-27 14:45:38,694][20381] Updated weights for policy 1, policy_version 1314189 (0.0006) [2023-12-27 14:45:38,761][20347] Updated weights for policy 0, policy_version 1315043 (0.0007) [2023-12-27 14:45:38,829][20347] Updated weights for policy 0, policy_version 1315053 (0.0009) [2023-12-27 14:45:38,896][20347] Updated weights for policy 0, policy_version 1315063 (0.0009) [2023-12-27 14:45:39,391][20381] Updated weights for policy 1, policy_version 1314199 (0.0007) [2023-12-27 14:45:39,457][20381] Updated weights for policy 1, policy_version 1314209 (0.0008) [2023-12-27 14:45:39,506][20381] Updated weights for policy 1, policy_version 1314219 (0.0009) [2023-12-27 14:45:39,662][20347] Updated weights for policy 0, policy_version 1315073 (0.0008) [2023-12-27 14:45:39,728][20347] Updated weights for policy 0, policy_version 1315083 (0.0008) [2023-12-27 14:45:39,792][20347] Updated weights for policy 0, policy_version 1315093 (0.0007) [2023-12-27 14:45:39,858][20347] Updated weights for policy 0, policy_version 1315103 (0.0008) [2023-12-27 14:45:40,279][20381] Updated weights for policy 1, policy_version 1314229 (0.0008) [2023-12-27 14:45:40,338][20381] Updated weights for policy 1, policy_version 1314239 (0.0008) [2023-12-27 14:45:40,394][20381] Updated weights for policy 1, policy_version 1314249 (0.0007) [2023-12-27 14:45:40,531][20347] Updated weights for policy 0, policy_version 1315113 (0.0006) [2023-12-27 14:45:40,591][20347] Updated weights for policy 0, policy_version 1315123 (0.0006) [2023-12-27 14:45:40,651][20347] Updated weights for policy 0, policy_version 1315133 (0.0008) [2023-12-27 14:45:41,132][20381] Updated weights for policy 1, policy_version 1314259 (0.0007) [2023-12-27 14:45:41,203][20381] Updated weights for policy 1, policy_version 1314269 (0.0009) [2023-12-27 14:45:41,265][20381] Updated weights for policy 1, policy_version 1314279 (0.0009) [2023-12-27 14:45:41,291][20347] Updated weights for policy 0, policy_version 1315143 (0.0008) [2023-12-27 14:45:41,353][20347] Updated weights for policy 0, policy_version 1315153 (0.0007) [2023-12-27 14:45:41,413][20347] Updated weights for policy 0, policy_version 1315163 (0.0008) [2023-12-27 14:45:42,020][20381] Updated weights for policy 1, policy_version 1314289 (0.0009) [2023-12-27 14:45:42,076][20381] Updated weights for policy 1, policy_version 1314299 (0.0009) [2023-12-27 14:45:42,125][20347] Updated weights for policy 0, policy_version 1315173 (0.0007) [2023-12-27 14:45:42,135][20381] Updated weights for policy 1, policy_version 1314309 (0.0007) [2023-12-27 14:45:42,185][20347] Updated weights for policy 0, policy_version 1315183 (0.0008) [2023-12-27 14:45:42,195][20381] Updated weights for policy 1, policy_version 1314319 (0.0006) [2023-12-27 14:45:42,242][20347] Updated weights for policy 0, policy_version 1315193 (0.0007) [2023-12-27 14:45:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 673251328. Throughput: 0: 10023.9, 1: 10278.8. Samples: 673237032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:42,363][19170] Avg episode reward: [(0, '653.490'), (1, '931.590')] [2023-12-27 14:45:42,903][20347] Updated weights for policy 0, policy_version 1315203 (0.0008) [2023-12-27 14:45:42,962][20347] Updated weights for policy 0, policy_version 1315213 (0.0007) [2023-12-27 14:45:42,963][20381] Updated weights for policy 1, policy_version 1314329 (0.0005) [2023-12-27 14:45:43,016][20347] Updated weights for policy 0, policy_version 1315223 (0.0008) [2023-12-27 14:45:43,017][20381] Updated weights for policy 1, policy_version 1314339 (0.0005) [2023-12-27 14:45:43,068][20381] Updated weights for policy 1, policy_version 1314349 (0.0005) [2023-12-27 14:45:43,688][20347] Updated weights for policy 0, policy_version 1315233 (0.0008) [2023-12-27 14:45:43,701][20381] Updated weights for policy 1, policy_version 1314359 (0.0006) [2023-12-27 14:45:43,749][20347] Updated weights for policy 0, policy_version 1315243 (0.0008) [2023-12-27 14:45:43,754][20381] Updated weights for policy 1, policy_version 1314369 (0.0008) [2023-12-27 14:45:43,803][20381] Updated weights for policy 1, policy_version 1314379 (0.0005) [2023-12-27 14:45:43,808][20347] Updated weights for policy 0, policy_version 1315253 (0.0006) [2023-12-27 14:45:43,865][20347] Updated weights for policy 0, policy_version 1315263 (0.0007) [2023-12-27 14:45:44,485][20381] Updated weights for policy 1, policy_version 1314389 (0.0006) [2023-12-27 14:45:44,544][20381] Updated weights for policy 1, policy_version 1314399 (0.0007) [2023-12-27 14:45:44,546][20347] Updated weights for policy 0, policy_version 1315273 (0.0007) [2023-12-27 14:45:44,595][20347] Updated weights for policy 0, policy_version 1315283 (0.0005) [2023-12-27 14:45:44,599][20381] Updated weights for policy 1, policy_version 1314409 (0.0006) [2023-12-27 14:45:44,667][20347] Updated weights for policy 0, policy_version 1315293 (0.0007) [2023-12-27 14:45:45,346][20381] Updated weights for policy 1, policy_version 1314419 (0.0008) [2023-12-27 14:45:45,385][20347] Updated weights for policy 0, policy_version 1315303 (0.0006) [2023-12-27 14:45:45,407][20381] Updated weights for policy 1, policy_version 1314429 (0.0006) [2023-12-27 14:45:45,435][20347] Updated weights for policy 0, policy_version 1315313 (0.0006) [2023-12-27 14:45:45,463][20381] Updated weights for policy 1, policy_version 1314439 (0.0008) [2023-12-27 14:45:45,487][20347] Updated weights for policy 0, policy_version 1315323 (0.0006) [2023-12-27 14:45:46,084][20347] Updated weights for policy 0, policy_version 1315333 (0.0005) [2023-12-27 14:45:46,144][20347] Updated weights for policy 0, policy_version 1315343 (0.0005) [2023-12-27 14:45:46,202][20347] Updated weights for policy 0, policy_version 1315353 (0.0005) [2023-12-27 14:45:46,203][20381] Updated weights for policy 1, policy_version 1314449 (0.0006) [2023-12-27 14:45:46,256][20381] Updated weights for policy 1, policy_version 1314459 (0.0008) [2023-12-27 14:45:46,309][20381] Updated weights for policy 1, policy_version 1314469 (0.0008) [2023-12-27 14:45:46,366][20381] Updated weights for policy 1, policy_version 1314479 (0.0008) [2023-12-27 14:45:46,847][20347] Updated weights for policy 0, policy_version 1315363 (0.0006) [2023-12-27 14:45:46,920][20347] Updated weights for policy 0, policy_version 1315373 (0.0008) [2023-12-27 14:45:46,980][20347] Updated weights for policy 0, policy_version 1315383 (0.0008) [2023-12-27 14:45:47,080][20381] Updated weights for policy 1, policy_version 1314489 (0.0008) [2023-12-27 14:45:47,132][20381] Updated weights for policy 1, policy_version 1314499 (0.0007) [2023-12-27 14:45:47,185][20381] Updated weights for policy 1, policy_version 1314509 (0.0007) [2023-12-27 14:45:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 673349632. Throughput: 0: 9998.1, 1: 10195.6. Samples: 673356356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:47,363][19170] Avg episode reward: [(0, '681.620'), (1, '892.550')] [2023-12-27 14:45:47,639][20347] Updated weights for policy 0, policy_version 1315393 (0.0008) [2023-12-27 14:45:47,692][20347] Updated weights for policy 0, policy_version 1315403 (0.0007) [2023-12-27 14:45:47,749][20347] Updated weights for policy 0, policy_version 1315413 (0.0007) [2023-12-27 14:45:47,804][20347] Updated weights for policy 0, policy_version 1315423 (0.0007) [2023-12-27 14:45:47,880][20381] Updated weights for policy 1, policy_version 1314519 (0.0006) [2023-12-27 14:45:47,942][20381] Updated weights for policy 1, policy_version 1314529 (0.0006) [2023-12-27 14:45:47,999][20381] Updated weights for policy 1, policy_version 1314539 (0.0006) [2023-12-27 14:45:48,389][20347] Updated weights for policy 0, policy_version 1315433 (0.0007) [2023-12-27 14:45:48,447][20347] Updated weights for policy 0, policy_version 1315443 (0.0005) [2023-12-27 14:45:48,509][20347] Updated weights for policy 0, policy_version 1315453 (0.0005) [2023-12-27 14:45:48,549][20381] Updated weights for policy 1, policy_version 1314549 (0.0010) [2023-12-27 14:45:48,609][20381] Updated weights for policy 1, policy_version 1314559 (0.0011) [2023-12-27 14:45:48,669][20381] Updated weights for policy 1, policy_version 1314569 (0.0011) [2023-12-27 14:45:49,239][20347] Updated weights for policy 0, policy_version 1315463 (0.0008) [2023-12-27 14:45:49,271][20381] Updated weights for policy 1, policy_version 1314579 (0.0009) [2023-12-27 14:45:49,303][20347] Updated weights for policy 0, policy_version 1315473 (0.0006) [2023-12-27 14:45:49,333][20381] Updated weights for policy 1, policy_version 1314589 (0.0006) [2023-12-27 14:45:49,364][20347] Updated weights for policy 0, policy_version 1315483 (0.0006) [2023-12-27 14:45:49,398][20381] Updated weights for policy 1, policy_version 1314599 (0.0008) [2023-12-27 14:45:49,974][20347] Updated weights for policy 0, policy_version 1315493 (0.0008) [2023-12-27 14:45:50,041][20347] Updated weights for policy 0, policy_version 1315503 (0.0007) [2023-12-27 14:45:50,105][20347] Updated weights for policy 0, policy_version 1315513 (0.0008) [2023-12-27 14:45:50,123][20381] Updated weights for policy 1, policy_version 1314609 (0.0006) [2023-12-27 14:45:50,180][20381] Updated weights for policy 1, policy_version 1314619 (0.0007) [2023-12-27 14:45:50,242][20381] Updated weights for policy 1, policy_version 1314629 (0.0006) [2023-12-27 14:45:50,304][20381] Updated weights for policy 1, policy_version 1314639 (0.0006) [2023-12-27 14:45:50,784][20347] Updated weights for policy 0, policy_version 1315523 (0.0007) [2023-12-27 14:45:50,846][20347] Updated weights for policy 0, policy_version 1315533 (0.0006) [2023-12-27 14:45:50,898][20347] Updated weights for policy 0, policy_version 1315543 (0.0007) [2023-12-27 14:45:50,926][20381] Updated weights for policy 1, policy_version 1314649 (0.0007) [2023-12-27 14:45:50,980][20381] Updated weights for policy 1, policy_version 1314659 (0.0006) [2023-12-27 14:45:51,037][20381] Updated weights for policy 1, policy_version 1314669 (0.0007) [2023-12-27 14:45:51,670][20347] Updated weights for policy 0, policy_version 1315553 (0.0007) [2023-12-27 14:45:51,734][20347] Updated weights for policy 0, policy_version 1315563 (0.0009) [2023-12-27 14:45:51,753][20381] Updated weights for policy 1, policy_version 1314679 (0.0009) [2023-12-27 14:45:51,805][20347] Updated weights for policy 0, policy_version 1315573 (0.0009) [2023-12-27 14:45:51,807][20381] Updated weights for policy 1, policy_version 1314689 (0.0006) [2023-12-27 14:45:51,859][20347] Updated weights for policy 0, policy_version 1315583 (0.0006) [2023-12-27 14:45:51,865][20381] Updated weights for policy 1, policy_version 1314699 (0.0006) [2023-12-27 14:45:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 673447936. Throughput: 0: 10004.5, 1: 10220.2. Samples: 673420556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:45:52,363][19170] Avg episode reward: [(0, '723.080'), (1, '843.630')] [2023-12-27 14:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001314704_336592896.pth... [2023-12-27 14:45:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001315584_336855040.pth... [2023-12-27 14:45:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001313520_336289792.pth [2023-12-27 14:45:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001314400_336551936.pth [2023-12-27 14:45:52,545][20347] Updated weights for policy 0, policy_version 1315593 (0.0007) [2023-12-27 14:45:52,583][20381] Updated weights for policy 1, policy_version 1314709 (0.0008) [2023-12-27 14:45:52,603][20347] Updated weights for policy 0, policy_version 1315603 (0.0007) [2023-12-27 14:45:52,641][20381] Updated weights for policy 1, policy_version 1314719 (0.0007) [2023-12-27 14:45:52,659][20347] Updated weights for policy 0, policy_version 1315613 (0.0008) [2023-12-27 14:45:52,691][20381] Updated weights for policy 1, policy_version 1314729 (0.0007) [2023-12-27 14:45:53,243][20347] Updated weights for policy 0, policy_version 1315623 (0.0006) [2023-12-27 14:45:53,291][20381] Updated weights for policy 1, policy_version 1314739 (0.0008) [2023-12-27 14:45:53,299][20347] Updated weights for policy 0, policy_version 1315633 (0.0005) [2023-12-27 14:45:53,325][20292] KL-divergence is very high: 132.9415 [2023-12-27 14:45:53,345][20347] Updated weights for policy 0, policy_version 1315643 (0.0005) [2023-12-27 14:45:53,350][20381] Updated weights for policy 1, policy_version 1314749 (0.0007) [2023-12-27 14:45:53,372][20292] KL-divergence is very high: 119.6121 [2023-12-27 14:45:53,408][20381] Updated weights for policy 1, policy_version 1314759 (0.0006) [2023-12-27 14:45:54,010][20347] Updated weights for policy 0, policy_version 1315653 (0.0006) [2023-12-27 14:45:54,066][20381] Updated weights for policy 1, policy_version 1314769 (0.0005) [2023-12-27 14:45:54,066][20347] Updated weights for policy 0, policy_version 1315663 (0.0008) [2023-12-27 14:45:54,122][20347] Updated weights for policy 0, policy_version 1315673 (0.0007) [2023-12-27 14:45:54,123][20381] Updated weights for policy 1, policy_version 1314779 (0.0006) [2023-12-27 14:45:54,180][20381] Updated weights for policy 1, policy_version 1314789 (0.0007) [2023-12-27 14:45:54,240][20381] Updated weights for policy 1, policy_version 1314799 (0.0005) [2023-12-27 14:45:54,804][20347] Updated weights for policy 0, policy_version 1315683 (0.0005) [2023-12-27 14:45:54,805][20381] Updated weights for policy 1, policy_version 1314809 (0.0005) [2023-12-27 14:45:54,858][20381] Updated weights for policy 1, policy_version 1314819 (0.0008) [2023-12-27 14:45:54,860][20347] Updated weights for policy 0, policy_version 1315693 (0.0005) [2023-12-27 14:45:54,914][20381] Updated weights for policy 1, policy_version 1314829 (0.0007) [2023-12-27 14:45:54,916][20347] Updated weights for policy 0, policy_version 1315703 (0.0005) [2023-12-27 14:45:55,550][20347] Updated weights for policy 0, policy_version 1315713 (0.0006) [2023-12-27 14:45:55,556][20381] Updated weights for policy 1, policy_version 1314839 (0.0006) [2023-12-27 14:45:55,598][20347] Updated weights for policy 0, policy_version 1315723 (0.0005) [2023-12-27 14:45:55,607][20381] Updated weights for policy 1, policy_version 1314849 (0.0006) [2023-12-27 14:45:55,650][20347] Updated weights for policy 0, policy_version 1315733 (0.0005) [2023-12-27 14:45:55,656][20381] Updated weights for policy 1, policy_version 1314859 (0.0005) [2023-12-27 14:45:55,704][20347] Updated weights for policy 0, policy_version 1315743 (0.0006) [2023-12-27 14:45:56,304][20381] Updated weights for policy 1, policy_version 1314869 (0.0007) [2023-12-27 14:45:56,353][20381] Updated weights for policy 1, policy_version 1314879 (0.0009) [2023-12-27 14:45:56,400][20381] Updated weights for policy 1, policy_version 1314889 (0.0005) [2023-12-27 14:45:56,403][20347] Updated weights for policy 0, policy_version 1315753 (0.0005) [2023-12-27 14:45:56,451][20347] Updated weights for policy 0, policy_version 1315763 (0.0005) [2023-12-27 14:45:56,497][20347] Updated weights for policy 0, policy_version 1315773 (0.0005) [2023-12-27 14:45:56,963][20381] Updated weights for policy 1, policy_version 1314899 (0.0005) [2023-12-27 14:45:57,019][20381] Updated weights for policy 1, policy_version 1314909 (0.0005) [2023-12-27 14:45:57,063][20381] Updated weights for policy 1, policy_version 1314919 (0.0006) [2023-12-27 14:45:57,167][20347] Updated weights for policy 0, policy_version 1315783 (0.0006) [2023-12-27 14:45:57,212][20347] Updated weights for policy 0, policy_version 1315793 (0.0005) [2023-12-27 14:45:57,260][20347] Updated weights for policy 0, policy_version 1315803 (0.0005) [2023-12-27 14:45:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 673562624. Throughput: 0: 10058.8, 1: 10201.4. Samples: 673545488. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:45:57,363][19170] Avg episode reward: [(0, '640.630'), (1, '863.340')] [2023-12-27 14:45:57,637][20381] Updated weights for policy 1, policy_version 1314929 (0.0009) [2023-12-27 14:45:57,686][20381] Updated weights for policy 1, policy_version 1314939 (0.0005) [2023-12-27 14:45:57,733][20381] Updated weights for policy 1, policy_version 1314949 (0.0006) [2023-12-27 14:45:57,783][20381] Updated weights for policy 1, policy_version 1314959 (0.0006) [2023-12-27 14:45:57,788][20347] Updated weights for policy 0, policy_version 1315813 (0.0005) [2023-12-27 14:45:57,831][20347] Updated weights for policy 0, policy_version 1315823 (0.0005) [2023-12-27 14:45:57,877][20347] Updated weights for policy 0, policy_version 1315833 (0.0005) [2023-12-27 14:45:58,427][20381] Updated weights for policy 1, policy_version 1314969 (0.0007) [2023-12-27 14:45:58,491][20381] Updated weights for policy 1, policy_version 1314979 (0.0008) [2023-12-27 14:45:58,555][20381] Updated weights for policy 1, policy_version 1314989 (0.0008) [2023-12-27 14:45:58,568][20347] Updated weights for policy 0, policy_version 1315843 (0.0006) [2023-12-27 14:45:58,636][20347] Updated weights for policy 0, policy_version 1315853 (0.0008) [2023-12-27 14:45:58,701][20347] Updated weights for policy 0, policy_version 1315863 (0.0006) [2023-12-27 14:45:59,399][20381] Updated weights for policy 1, policy_version 1314999 (0.0007) [2023-12-27 14:45:59,457][20381] Updated weights for policy 1, policy_version 1315009 (0.0006) [2023-12-27 14:45:59,500][20347] Updated weights for policy 0, policy_version 1315873 (0.0008) [2023-12-27 14:45:59,517][20381] Updated weights for policy 1, policy_version 1315019 (0.0008) [2023-12-27 14:45:59,549][20347] Updated weights for policy 0, policy_version 1315883 (0.0005) [2023-12-27 14:45:59,602][20347] Updated weights for policy 0, policy_version 1315893 (0.0008) [2023-12-27 14:45:59,652][20347] Updated weights for policy 0, policy_version 1315903 (0.0008) [2023-12-27 14:46:00,177][20381] Updated weights for policy 1, policy_version 1315029 (0.0007) [2023-12-27 14:46:00,241][20381] Updated weights for policy 1, policy_version 1315039 (0.0007) [2023-12-27 14:46:00,308][20381] Updated weights for policy 1, policy_version 1315049 (0.0008) [2023-12-27 14:46:00,419][20347] Updated weights for policy 0, policy_version 1315913 (0.0008) [2023-12-27 14:46:00,474][20347] Updated weights for policy 0, policy_version 1315923 (0.0009) [2023-12-27 14:46:00,534][20347] Updated weights for policy 0, policy_version 1315933 (0.0008) [2023-12-27 14:46:00,913][20381] Updated weights for policy 1, policy_version 1315059 (0.0008) [2023-12-27 14:46:00,973][20381] Updated weights for policy 1, policy_version 1315069 (0.0007) [2023-12-27 14:46:01,029][20381] Updated weights for policy 1, policy_version 1315079 (0.0008) [2023-12-27 14:46:01,201][20347] Updated weights for policy 0, policy_version 1315943 (0.0008) [2023-12-27 14:46:01,267][20347] Updated weights for policy 0, policy_version 1315953 (0.0008) [2023-12-27 14:46:01,327][20347] Updated weights for policy 0, policy_version 1315963 (0.0008) [2023-12-27 14:46:01,763][20381] Updated weights for policy 1, policy_version 1315089 (0.0008) [2023-12-27 14:46:01,809][20381] Updated weights for policy 1, policy_version 1315099 (0.0010) [2023-12-27 14:46:01,860][20381] Updated weights for policy 1, policy_version 1315109 (0.0010) [2023-12-27 14:46:01,921][20381] Updated weights for policy 1, policy_version 1315119 (0.0010) [2023-12-27 14:46:02,047][20347] Updated weights for policy 0, policy_version 1315973 (0.0008) [2023-12-27 14:46:02,102][20347] Updated weights for policy 0, policy_version 1315983 (0.0008) [2023-12-27 14:46:02,158][20347] Updated weights for policy 0, policy_version 1315993 (0.0006) [2023-12-27 14:46:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 673660928. Throughput: 0: 10071.0, 1: 10255.3. Samples: 673668688. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:02,363][19170] Avg episode reward: [(0, '631.050'), (1, '873.030')] [2023-12-27 14:46:02,638][20381] Updated weights for policy 1, policy_version 1315129 (0.0008) [2023-12-27 14:46:02,695][20381] Updated weights for policy 1, policy_version 1315139 (0.0008) [2023-12-27 14:46:02,751][20381] Updated weights for policy 1, policy_version 1315149 (0.0008) [2023-12-27 14:46:02,847][20347] Updated weights for policy 0, policy_version 1316003 (0.0007) [2023-12-27 14:46:02,905][20347] Updated weights for policy 0, policy_version 1316013 (0.0007) [2023-12-27 14:46:02,962][20347] Updated weights for policy 0, policy_version 1316023 (0.0007) [2023-12-27 14:46:03,398][20381] Updated weights for policy 1, policy_version 1315159 (0.0007) [2023-12-27 14:46:03,449][20381] Updated weights for policy 1, policy_version 1315169 (0.0007) [2023-12-27 14:46:03,503][20381] Updated weights for policy 1, policy_version 1315179 (0.0007) [2023-12-27 14:46:03,637][20347] Updated weights for policy 0, policy_version 1316033 (0.0008) [2023-12-27 14:46:03,687][20347] Updated weights for policy 0, policy_version 1316043 (0.0010) [2023-12-27 14:46:03,736][20347] Updated weights for policy 0, policy_version 1316053 (0.0010) [2023-12-27 14:46:03,790][20347] Updated weights for policy 0, policy_version 1316063 (0.0010) [2023-12-27 14:46:04,245][20381] Updated weights for policy 1, policy_version 1315189 (0.0007) [2023-12-27 14:46:04,271][20292] KL-divergence is very high: 368.2106 [2023-12-27 14:46:04,308][20381] Updated weights for policy 1, policy_version 1315199 (0.0006) [2023-12-27 14:46:04,321][20292] KL-divergence is very high: 327.7655 [2023-12-27 14:46:04,368][20381] Updated weights for policy 1, policy_version 1315209 (0.0006) [2023-12-27 14:46:04,503][20347] Updated weights for policy 0, policy_version 1316073 (0.0010) [2023-12-27 14:46:04,560][20347] Updated weights for policy 0, policy_version 1316083 (0.0011) [2023-12-27 14:46:04,618][20347] Updated weights for policy 0, policy_version 1316093 (0.0010) [2023-12-27 14:46:05,026][20381] Updated weights for policy 1, policy_version 1315219 (0.0006) [2023-12-27 14:46:05,079][20381] Updated weights for policy 1, policy_version 1315229 (0.0005) [2023-12-27 14:46:05,125][20381] Updated weights for policy 1, policy_version 1315239 (0.0005) [2023-12-27 14:46:05,190][20347] Updated weights for policy 0, policy_version 1316103 (0.0007) [2023-12-27 14:46:05,250][20347] Updated weights for policy 0, policy_version 1316113 (0.0005) [2023-12-27 14:46:05,309][20347] Updated weights for policy 0, policy_version 1316123 (0.0005) [2023-12-27 14:46:05,781][20381] Updated weights for policy 1, policy_version 1315249 (0.0005) [2023-12-27 14:46:05,805][20292] KL-divergence is very high: 180.7742 [2023-12-27 14:46:05,822][20347] Updated weights for policy 0, policy_version 1316133 (0.0008) [2023-12-27 14:46:05,822][20292] KL-divergence is very high: 199.9372 [2023-12-27 14:46:05,839][20381] Updated weights for policy 1, policy_version 1315259 (0.0006) [2023-12-27 14:46:05,850][20292] KL-divergence is very high: 124.8073 [2023-12-27 14:46:05,878][20347] Updated weights for policy 0, policy_version 1316143 (0.0010) [2023-12-27 14:46:05,895][20381] Updated weights for policy 1, policy_version 1315269 (0.0008) [2023-12-27 14:46:05,933][20347] Updated weights for policy 0, policy_version 1316153 (0.0010) [2023-12-27 14:46:05,951][20381] Updated weights for policy 1, policy_version 1315279 (0.0006) [2023-12-27 14:46:06,575][20347] Updated weights for policy 0, policy_version 1316163 (0.0010) [2023-12-27 14:46:06,637][20381] Updated weights for policy 1, policy_version 1315289 (0.0009) [2023-12-27 14:46:06,641][20347] Updated weights for policy 0, policy_version 1316173 (0.0010) [2023-12-27 14:46:06,699][20347] Updated weights for policy 0, policy_version 1316183 (0.0010) [2023-12-27 14:46:06,699][20381] Updated weights for policy 1, policy_version 1315299 (0.0010) [2023-12-27 14:46:06,758][20381] Updated weights for policy 1, policy_version 1315309 (0.0010) [2023-12-27 14:46:07,297][20347] Updated weights for policy 0, policy_version 1316193 (0.0010) [2023-12-27 14:46:07,362][20347] Updated weights for policy 0, policy_version 1316203 (0.0010) [2023-12-27 14:46:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 673759232. Throughput: 0: 10128.8, 1: 10254.8. Samples: 673731144. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:07,363][19170] Avg episode reward: [(0, '561.980'), (1, '902.280')] [2023-12-27 14:46:07,374][20381] Updated weights for policy 1, policy_version 1315319 (0.0010) [2023-12-27 14:46:07,421][20347] Updated weights for policy 0, policy_version 1316213 (0.0010) [2023-12-27 14:46:07,426][20381] Updated weights for policy 1, policy_version 1315329 (0.0011) [2023-12-27 14:46:07,477][20347] Updated weights for policy 0, policy_version 1316223 (0.0010) [2023-12-27 14:46:07,481][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001316224_337018880.pth... [2023-12-27 14:46:07,482][20381] Updated weights for policy 1, policy_version 1315339 (0.0011) [2023-12-27 14:46:07,485][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001315008_336707584.pth [2023-12-27 14:46:07,509][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001315344_336756736.pth... [2023-12-27 14:46:07,512][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001314128_336445440.pth [2023-12-27 14:46:08,124][20347] Updated weights for policy 0, policy_version 1316233 (0.0007) [2023-12-27 14:46:08,132][20381] Updated weights for policy 1, policy_version 1315349 (0.0009) [2023-12-27 14:46:08,165][20291] KL-divergence is very high: 105.0324 [2023-12-27 14:46:08,181][20347] Updated weights for policy 0, policy_version 1316243 (0.0006) [2023-12-27 14:46:08,188][20381] Updated weights for policy 1, policy_version 1315359 (0.0007) [2023-12-27 14:46:08,230][20347] Updated weights for policy 0, policy_version 1316253 (0.0010) [2023-12-27 14:46:08,243][20381] Updated weights for policy 1, policy_version 1315369 (0.0006) [2023-12-27 14:46:08,858][20347] Updated weights for policy 0, policy_version 1316263 (0.0007) [2023-12-27 14:46:08,914][20381] Updated weights for policy 1, policy_version 1315379 (0.0008) [2023-12-27 14:46:08,919][20347] Updated weights for policy 0, policy_version 1316273 (0.0006) [2023-12-27 14:46:08,974][20347] Updated weights for policy 0, policy_version 1316283 (0.0007) [2023-12-27 14:46:08,976][20381] Updated weights for policy 1, policy_version 1315389 (0.0009) [2023-12-27 14:46:09,025][20381] Updated weights for policy 1, policy_version 1315399 (0.0011) [2023-12-27 14:46:09,612][20347] Updated weights for policy 0, policy_version 1316293 (0.0007) [2023-12-27 14:46:09,672][20347] Updated weights for policy 0, policy_version 1316303 (0.0008) [2023-12-27 14:46:09,738][20347] Updated weights for policy 0, policy_version 1316313 (0.0007) [2023-12-27 14:46:09,771][20381] Updated weights for policy 1, policy_version 1315409 (0.0011) [2023-12-27 14:46:09,833][20381] Updated weights for policy 1, policy_version 1315419 (0.0010) [2023-12-27 14:46:09,897][20381] Updated weights for policy 1, policy_version 1315429 (0.0009) [2023-12-27 14:46:09,957][20381] Updated weights for policy 1, policy_version 1315439 (0.0008) [2023-12-27 14:46:10,466][20347] Updated weights for policy 0, policy_version 1316323 (0.0008) [2023-12-27 14:46:10,529][20347] Updated weights for policy 0, policy_version 1316333 (0.0008) [2023-12-27 14:46:10,588][20347] Updated weights for policy 0, policy_version 1316343 (0.0006) [2023-12-27 14:46:10,624][20381] Updated weights for policy 1, policy_version 1315449 (0.0005) [2023-12-27 14:46:10,667][20381] Updated weights for policy 1, policy_version 1315459 (0.0006) [2023-12-27 14:46:10,717][20381] Updated weights for policy 1, policy_version 1315469 (0.0008) [2023-12-27 14:46:11,230][20347] Updated weights for policy 0, policy_version 1316353 (0.0010) [2023-12-27 14:46:11,297][20347] Updated weights for policy 0, policy_version 1316363 (0.0008) [2023-12-27 14:46:11,362][20347] Updated weights for policy 0, policy_version 1316373 (0.0007) [2023-12-27 14:46:11,426][20347] Updated weights for policy 0, policy_version 1316383 (0.0007) [2023-12-27 14:46:11,498][20381] Updated weights for policy 1, policy_version 1315479 (0.0008) [2023-12-27 14:46:11,561][20381] Updated weights for policy 1, policy_version 1315489 (0.0007) [2023-12-27 14:46:11,623][20381] Updated weights for policy 1, policy_version 1315499 (0.0006) [2023-12-27 14:46:12,118][20347] Updated weights for policy 0, policy_version 1316393 (0.0007) [2023-12-27 14:46:12,180][20347] Updated weights for policy 0, policy_version 1316403 (0.0008) [2023-12-27 14:46:12,234][20347] Updated weights for policy 0, policy_version 1316413 (0.0007) [2023-12-27 14:46:12,254][20381] Updated weights for policy 1, policy_version 1315509 (0.0008) [2023-12-27 14:46:12,318][20381] Updated weights for policy 1, policy_version 1315519 (0.0009) [2023-12-27 14:46:12,334][20292] KL-divergence is very high: 250.1152 [2023-12-27 14:46:12,340][20292] KL-divergence is very high: 272.2074 [2023-12-27 14:46:12,351][20292] KL-divergence is very high: 205.8050 [2023-12-27 14:46:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 673865728. Throughput: 0: 10252.1, 1: 10236.4. Samples: 673856820. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:12,363][19170] Avg episode reward: [(0, '733.200'), (1, '853.500')] [2023-12-27 14:46:12,376][20381] Updated weights for policy 1, policy_version 1315529 (0.0007) [2023-12-27 14:46:12,386][20292] KL-divergence is very high: 142.6113 [2023-12-27 14:46:12,393][20292] KL-divergence is very high: 124.0611 [2023-12-27 14:46:12,935][20347] Updated weights for policy 0, policy_version 1316423 (0.0007) [2023-12-27 14:46:12,979][20347] Updated weights for policy 0, policy_version 1316433 (0.0008) [2023-12-27 14:46:13,024][20347] Updated weights for policy 0, policy_version 1316443 (0.0007) [2023-12-27 14:46:13,032][20292] KL-divergence is very high: 291.1889 [2023-12-27 14:46:13,037][20381] Updated weights for policy 1, policy_version 1315539 (0.0005) [2023-12-27 14:46:13,071][20292] KL-divergence is very high: 507.1186 [2023-12-27 14:46:13,087][20381] Updated weights for policy 1, policy_version 1315549 (0.0007) [2023-12-27 14:46:13,113][20292] KL-divergence is very high: 173.0807 [2023-12-27 14:46:13,143][20381] Updated weights for policy 1, policy_version 1315559 (0.0008) [2023-12-27 14:46:13,160][20292] KL-divergence is very high: 117.4548 [2023-12-27 14:46:13,746][20347] Updated weights for policy 0, policy_version 1316453 (0.0007) [2023-12-27 14:46:13,792][20347] Updated weights for policy 0, policy_version 1316463 (0.0007) [2023-12-27 14:46:13,831][20292] KL-divergence is very high: 133.4412 [2023-12-27 14:46:13,836][20381] Updated weights for policy 1, policy_version 1315569 (0.0008) [2023-12-27 14:46:13,849][20347] Updated weights for policy 0, policy_version 1316473 (0.0007) [2023-12-27 14:46:13,894][20381] Updated weights for policy 1, policy_version 1315579 (0.0006) [2023-12-27 14:46:13,965][20381] Updated weights for policy 1, policy_version 1315589 (0.0005) [2023-12-27 14:46:14,022][20381] Updated weights for policy 1, policy_version 1315599 (0.0005) [2023-12-27 14:46:14,565][20381] Updated weights for policy 1, policy_version 1315609 (0.0005) [2023-12-27 14:46:14,625][20381] Updated weights for policy 1, policy_version 1315619 (0.0005) [2023-12-27 14:46:14,636][20347] Updated weights for policy 0, policy_version 1316483 (0.0007) [2023-12-27 14:46:14,682][20381] Updated weights for policy 1, policy_version 1315629 (0.0005) [2023-12-27 14:46:14,688][20347] Updated weights for policy 0, policy_version 1316493 (0.0006) [2023-12-27 14:46:14,750][20347] Updated weights for policy 0, policy_version 1316503 (0.0006) [2023-12-27 14:46:15,385][20381] Updated weights for policy 1, policy_version 1315639 (0.0009) [2023-12-27 14:46:15,441][20381] Updated weights for policy 1, policy_version 1315649 (0.0010) [2023-12-27 14:46:15,463][20347] Updated weights for policy 0, policy_version 1316513 (0.0008) [2023-12-27 14:46:15,496][20381] Updated weights for policy 1, policy_version 1315659 (0.0011) [2023-12-27 14:46:15,518][20347] Updated weights for policy 0, policy_version 1316523 (0.0007) [2023-12-27 14:46:15,575][20347] Updated weights for policy 0, policy_version 1316533 (0.0006) [2023-12-27 14:46:15,636][20347] Updated weights for policy 0, policy_version 1316543 (0.0006) [2023-12-27 14:46:16,133][20381] Updated weights for policy 1, policy_version 1315669 (0.0011) [2023-12-27 14:46:16,182][20381] Updated weights for policy 1, policy_version 1315679 (0.0010) [2023-12-27 14:46:16,239][20381] Updated weights for policy 1, policy_version 1315689 (0.0010) [2023-12-27 14:46:16,264][20347] Updated weights for policy 0, policy_version 1316553 (0.0010) [2023-12-27 14:46:16,325][20347] Updated weights for policy 0, policy_version 1316563 (0.0010) [2023-12-27 14:46:16,385][20347] Updated weights for policy 0, policy_version 1316573 (0.0010) [2023-12-27 14:46:16,891][20381] Updated weights for policy 1, policy_version 1315699 (0.0009) [2023-12-27 14:46:16,924][20292] KL-divergence is very high: 162.4813 [2023-12-27 14:46:16,947][20381] Updated weights for policy 1, policy_version 1315709 (0.0007) [2023-12-27 14:46:16,966][20292] KL-divergence is very high: 196.9563 [2023-12-27 14:46:17,000][20381] Updated weights for policy 1, policy_version 1315719 (0.0008) [2023-12-27 14:46:17,011][20292] KL-divergence is very high: 133.3661 [2023-12-27 14:46:17,084][20347] Updated weights for policy 0, policy_version 1316583 (0.0010) [2023-12-27 14:46:17,133][20347] Updated weights for policy 0, policy_version 1316593 (0.0010) [2023-12-27 14:46:17,188][20347] Updated weights for policy 0, policy_version 1316603 (0.0010) [2023-12-27 14:46:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 673972224. Throughput: 0: 10235.6, 1: 10238.8. Samples: 673979192. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:17,363][19170] Avg episode reward: [(0, '653.870'), (1, '766.670')] [2023-12-27 14:46:17,676][20381] Updated weights for policy 1, policy_version 1315729 (0.0007) [2023-12-27 14:46:17,731][20381] Updated weights for policy 1, policy_version 1315739 (0.0010) [2023-12-27 14:46:17,790][20381] Updated weights for policy 1, policy_version 1315749 (0.0005) [2023-12-27 14:46:17,848][20381] Updated weights for policy 1, policy_version 1315759 (0.0005) [2023-12-27 14:46:17,859][20347] Updated weights for policy 0, policy_version 1316613 (0.0011) [2023-12-27 14:46:17,910][20347] Updated weights for policy 0, policy_version 1316623 (0.0008) [2023-12-27 14:46:17,959][20347] Updated weights for policy 0, policy_version 1316633 (0.0006) [2023-12-27 14:46:18,472][20381] Updated weights for policy 1, policy_version 1315769 (0.0007) [2023-12-27 14:46:18,532][20381] Updated weights for policy 1, policy_version 1315779 (0.0005) [2023-12-27 14:46:18,594][20381] Updated weights for policy 1, policy_version 1315789 (0.0005) [2023-12-27 14:46:18,622][20347] Updated weights for policy 0, policy_version 1316643 (0.0008) [2023-12-27 14:46:18,685][20347] Updated weights for policy 0, policy_version 1316653 (0.0010) [2023-12-27 14:46:18,745][20347] Updated weights for policy 0, policy_version 1316663 (0.0005) [2023-12-27 14:46:19,246][20292] KL-divergence is very high: 142.9439 [2023-12-27 14:46:19,252][20381] Updated weights for policy 1, policy_version 1315799 (0.0007) [2023-12-27 14:46:19,278][20292] KL-divergence is very high: 601.3123 [2023-12-27 14:46:19,289][20292] KL-divergence is very high: 219.6734 [2023-12-27 14:46:19,295][20292] KL-divergence is very high: 689.2275 [2023-12-27 14:46:19,313][20381] Updated weights for policy 1, policy_version 1315809 (0.0006) [2023-12-27 14:46:19,325][20292] KL-divergence is very high: 492.5752 [2023-12-27 14:46:19,332][20347] Updated weights for policy 0, policy_version 1316673 (0.0007) [2023-12-27 14:46:19,338][20292] KL-divergence is very high: 125.1495 [2023-12-27 14:46:19,345][20292] KL-divergence is very high: 304.0636 [2023-12-27 14:46:19,377][20381] Updated weights for policy 1, policy_version 1315819 (0.0008) [2023-12-27 14:46:19,399][20347] Updated weights for policy 0, policy_version 1316683 (0.0010) [2023-12-27 14:46:19,459][20347] Updated weights for policy 0, policy_version 1316693 (0.0007) [2023-12-27 14:46:19,526][20347] Updated weights for policy 0, policy_version 1316703 (0.0009) [2023-12-27 14:46:20,103][20292] KL-divergence is very high: 161.3283 [2023-12-27 14:46:20,117][20381] Updated weights for policy 1, policy_version 1315829 (0.0009) [2023-12-27 14:46:20,157][20292] KL-divergence is very high: 207.0030 [2023-12-27 14:46:20,184][20381] Updated weights for policy 1, policy_version 1315839 (0.0007) [2023-12-27 14:46:20,207][20292] KL-divergence is very high: 137.7989 [2023-12-27 14:46:20,224][20347] Updated weights for policy 0, policy_version 1316713 (0.0007) [2023-12-27 14:46:20,237][20381] Updated weights for policy 1, policy_version 1315849 (0.0005) [2023-12-27 14:46:20,273][20347] Updated weights for policy 0, policy_version 1316723 (0.0006) [2023-12-27 14:46:20,321][20347] Updated weights for policy 0, policy_version 1316733 (0.0007) [2023-12-27 14:46:20,900][20381] Updated weights for policy 1, policy_version 1315859 (0.0006) [2023-12-27 14:46:20,958][20381] Updated weights for policy 1, policy_version 1315869 (0.0010) [2023-12-27 14:46:21,013][20381] Updated weights for policy 1, policy_version 1315879 (0.0010) [2023-12-27 14:46:21,046][20347] Updated weights for policy 0, policy_version 1316743 (0.0009) [2023-12-27 14:46:21,110][20347] Updated weights for policy 0, policy_version 1316753 (0.0008) [2023-12-27 14:46:21,168][20347] Updated weights for policy 0, policy_version 1316763 (0.0008) [2023-12-27 14:46:21,727][20381] Updated weights for policy 1, policy_version 1315889 (0.0010) [2023-12-27 14:46:21,801][20381] Updated weights for policy 1, policy_version 1315899 (0.0009) [2023-12-27 14:46:21,866][20381] Updated weights for policy 1, policy_version 1315909 (0.0008) [2023-12-27 14:46:21,931][20381] Updated weights for policy 1, policy_version 1315919 (0.0009) [2023-12-27 14:46:21,934][20347] Updated weights for policy 0, policy_version 1316773 (0.0009) [2023-12-27 14:46:21,994][20347] Updated weights for policy 0, policy_version 1316783 (0.0006) [2023-12-27 14:46:22,052][20347] Updated weights for policy 0, policy_version 1316793 (0.0007) [2023-12-27 14:46:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 674070528. Throughput: 0: 10251.4, 1: 10226.0. Samples: 674041308. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:22,363][19170] Avg episode reward: [(0, '683.720'), (1, '776.110')] [2023-12-27 14:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001315920_336904192.pth... [2023-12-27 14:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001316800_337166336.pth... [2023-12-27 14:46:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001314704_336592896.pth [2023-12-27 14:46:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001315584_336855040.pth [2023-12-27 14:46:22,652][20381] Updated weights for policy 1, policy_version 1315929 (0.0007) [2023-12-27 14:46:22,708][20381] Updated weights for policy 1, policy_version 1315939 (0.0010) [2023-12-27 14:46:22,736][20347] Updated weights for policy 0, policy_version 1316803 (0.0008) [2023-12-27 14:46:22,769][20381] Updated weights for policy 1, policy_version 1315949 (0.0010) [2023-12-27 14:46:22,794][20347] Updated weights for policy 0, policy_version 1316813 (0.0006) [2023-12-27 14:46:22,853][20347] Updated weights for policy 0, policy_version 1316823 (0.0006) [2023-12-27 14:46:23,462][20381] Updated weights for policy 1, policy_version 1315959 (0.0011) [2023-12-27 14:46:23,517][20381] Updated weights for policy 1, policy_version 1315969 (0.0008) [2023-12-27 14:46:23,546][20347] Updated weights for policy 0, policy_version 1316833 (0.0007) [2023-12-27 14:46:23,572][20381] Updated weights for policy 1, policy_version 1315979 (0.0005) [2023-12-27 14:46:23,609][20347] Updated weights for policy 0, policy_version 1316843 (0.0007) [2023-12-27 14:46:23,660][20347] Updated weights for policy 0, policy_version 1316853 (0.0007) [2023-12-27 14:46:23,711][20347] Updated weights for policy 0, policy_version 1316863 (0.0007) [2023-12-27 14:46:24,168][20381] Updated weights for policy 1, policy_version 1315989 (0.0005) [2023-12-27 14:46:24,217][20381] Updated weights for policy 1, policy_version 1315999 (0.0007) [2023-12-27 14:46:24,266][20381] Updated weights for policy 1, policy_version 1316009 (0.0010) [2023-12-27 14:46:24,418][20347] Updated weights for policy 0, policy_version 1316873 (0.0008) [2023-12-27 14:46:24,475][20347] Updated weights for policy 0, policy_version 1316883 (0.0007) [2023-12-27 14:46:24,527][20347] Updated weights for policy 0, policy_version 1316893 (0.0008) [2023-12-27 14:46:24,837][20381] Updated weights for policy 1, policy_version 1316019 (0.0009) [2023-12-27 14:46:24,891][20381] Updated weights for policy 1, policy_version 1316029 (0.0009) [2023-12-27 14:46:24,939][20381] Updated weights for policy 1, policy_version 1316039 (0.0010) [2023-12-27 14:46:25,294][20347] Updated weights for policy 0, policy_version 1316903 (0.0008) [2023-12-27 14:46:25,356][20347] Updated weights for policy 0, policy_version 1316913 (0.0007) [2023-12-27 14:46:25,404][20347] Updated weights for policy 0, policy_version 1316923 (0.0010) [2023-12-27 14:46:25,517][20381] Updated weights for policy 1, policy_version 1316049 (0.0010) [2023-12-27 14:46:25,572][20381] Updated weights for policy 1, policy_version 1316059 (0.0011) [2023-12-27 14:46:25,623][20381] Updated weights for policy 1, policy_version 1316069 (0.0010) [2023-12-27 14:46:25,681][20381] Updated weights for policy 1, policy_version 1316079 (0.0011) [2023-12-27 14:46:26,039][20347] Updated weights for policy 0, policy_version 1316933 (0.0010) [2023-12-27 14:46:26,100][20347] Updated weights for policy 0, policy_version 1316943 (0.0010) [2023-12-27 14:46:26,158][20347] Updated weights for policy 0, policy_version 1316953 (0.0010) [2023-12-27 14:46:26,298][20381] Updated weights for policy 1, policy_version 1316089 (0.0006) [2023-12-27 14:46:26,358][20381] Updated weights for policy 1, policy_version 1316099 (0.0005) [2023-12-27 14:46:26,412][20381] Updated weights for policy 1, policy_version 1316109 (0.0005) [2023-12-27 14:46:26,757][20347] Updated weights for policy 0, policy_version 1316963 (0.0010) [2023-12-27 14:46:26,805][20347] Updated weights for policy 0, policy_version 1316973 (0.0010) [2023-12-27 14:46:26,852][20347] Updated weights for policy 0, policy_version 1316983 (0.0010) [2023-12-27 14:46:26,914][20381] Updated weights for policy 1, policy_version 1316119 (0.0005) [2023-12-27 14:46:26,967][20381] Updated weights for policy 1, policy_version 1316129 (0.0005) [2023-12-27 14:46:27,019][20381] Updated weights for policy 1, policy_version 1316139 (0.0006) [2023-12-27 14:46:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 674177024. Throughput: 0: 10240.3, 1: 10363.2. Samples: 674164192. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:27,364][19170] Avg episode reward: [(0, '681.530'), (1, '815.250')] [2023-12-27 14:46:27,515][20347] Updated weights for policy 0, policy_version 1316993 (0.0010) [2023-12-27 14:46:27,565][20347] Updated weights for policy 0, policy_version 1317003 (0.0010) [2023-12-27 14:46:27,595][20381] Updated weights for policy 1, policy_version 1316149 (0.0006) [2023-12-27 14:46:27,616][20347] Updated weights for policy 0, policy_version 1317013 (0.0009) [2023-12-27 14:46:27,650][20381] Updated weights for policy 1, policy_version 1316159 (0.0006) [2023-12-27 14:46:27,669][20347] Updated weights for policy 0, policy_version 1317023 (0.0005) [2023-12-27 14:46:27,704][20381] Updated weights for policy 1, policy_version 1316169 (0.0005) [2023-12-27 14:46:28,232][20347] Updated weights for policy 0, policy_version 1317033 (0.0010) [2023-12-27 14:46:28,282][20347] Updated weights for policy 0, policy_version 1317043 (0.0010) [2023-12-27 14:46:28,339][20347] Updated weights for policy 0, policy_version 1317053 (0.0010) [2023-12-27 14:46:28,390][20381] Updated weights for policy 1, policy_version 1316179 (0.0006) [2023-12-27 14:46:28,439][20381] Updated weights for policy 1, policy_version 1316189 (0.0008) [2023-12-27 14:46:28,488][20381] Updated weights for policy 1, policy_version 1316199 (0.0007) [2023-12-27 14:46:29,010][20347] Updated weights for policy 0, policy_version 1317063 (0.0010) [2023-12-27 14:46:29,067][20347] Updated weights for policy 0, policy_version 1317073 (0.0010) [2023-12-27 14:46:29,125][20347] Updated weights for policy 0, policy_version 1317083 (0.0010) [2023-12-27 14:46:29,188][20381] Updated weights for policy 1, policy_version 1316209 (0.0008) [2023-12-27 14:46:29,255][20381] Updated weights for policy 1, policy_version 1316219 (0.0007) [2023-12-27 14:46:29,308][20381] Updated weights for policy 1, policy_version 1316229 (0.0008) [2023-12-27 14:46:29,369][20381] Updated weights for policy 1, policy_version 1316239 (0.0010) [2023-12-27 14:46:29,876][20347] Updated weights for policy 0, policy_version 1317093 (0.0009) [2023-12-27 14:46:29,943][20347] Updated weights for policy 0, policy_version 1317103 (0.0008) [2023-12-27 14:46:29,998][20347] Updated weights for policy 0, policy_version 1317113 (0.0006) [2023-12-27 14:46:30,080][20381] Updated weights for policy 1, policy_version 1316249 (0.0009) [2023-12-27 14:46:30,141][20381] Updated weights for policy 1, policy_version 1316259 (0.0007) [2023-12-27 14:46:30,194][20381] Updated weights for policy 1, policy_version 1316269 (0.0006) [2023-12-27 14:46:30,674][20347] Updated weights for policy 0, policy_version 1317123 (0.0007) [2023-12-27 14:46:30,726][20347] Updated weights for policy 0, policy_version 1317133 (0.0005) [2023-12-27 14:46:30,771][20347] Updated weights for policy 0, policy_version 1317143 (0.0007) [2023-12-27 14:46:30,857][20381] Updated weights for policy 1, policy_version 1316279 (0.0008) [2023-12-27 14:46:30,910][20381] Updated weights for policy 1, policy_version 1316289 (0.0005) [2023-12-27 14:46:30,959][20381] Updated weights for policy 1, policy_version 1316299 (0.0006) [2023-12-27 14:46:31,421][20347] Updated weights for policy 0, policy_version 1317153 (0.0007) [2023-12-27 14:46:31,479][20347] Updated weights for policy 0, policy_version 1317163 (0.0007) [2023-12-27 14:46:31,538][20347] Updated weights for policy 0, policy_version 1317173 (0.0007) [2023-12-27 14:46:31,596][20347] Updated weights for policy 0, policy_version 1317183 (0.0007) [2023-12-27 14:46:31,640][20381] Updated weights for policy 1, policy_version 1316309 (0.0007) [2023-12-27 14:46:31,701][20381] Updated weights for policy 1, policy_version 1316319 (0.0007) [2023-12-27 14:46:31,764][20381] Updated weights for policy 1, policy_version 1316329 (0.0009) [2023-12-27 14:46:32,306][20347] Updated weights for policy 0, policy_version 1317193 (0.0007) [2023-12-27 14:46:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 674275328. Throughput: 0: 10280.3, 1: 10489.4. Samples: 674290996. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:32,363][19170] Avg episode reward: [(0, '632.080'), (1, '805.410')] [2023-12-27 14:46:32,366][20347] Updated weights for policy 0, policy_version 1317203 (0.0008) [2023-12-27 14:46:32,431][20347] Updated weights for policy 0, policy_version 1317213 (0.0007) [2023-12-27 14:46:32,443][20381] Updated weights for policy 1, policy_version 1316339 (0.0007) [2023-12-27 14:46:32,503][20381] Updated weights for policy 1, policy_version 1316349 (0.0008) [2023-12-27 14:46:32,563][20381] Updated weights for policy 1, policy_version 1316359 (0.0008) [2023-12-27 14:46:33,136][20347] Updated weights for policy 0, policy_version 1317223 (0.0007) [2023-12-27 14:46:33,187][20347] Updated weights for policy 0, policy_version 1317233 (0.0007) [2023-12-27 14:46:33,230][20381] Updated weights for policy 1, policy_version 1316369 (0.0008) [2023-12-27 14:46:33,246][20347] Updated weights for policy 0, policy_version 1317243 (0.0006) [2023-12-27 14:46:33,274][20381] Updated weights for policy 1, policy_version 1316379 (0.0007) [2023-12-27 14:46:33,322][20381] Updated weights for policy 1, policy_version 1316389 (0.0007) [2023-12-27 14:46:33,366][20381] Updated weights for policy 1, policy_version 1316399 (0.0007) [2023-12-27 14:46:33,924][20347] Updated weights for policy 0, policy_version 1317253 (0.0007) [2023-12-27 14:46:33,978][20347] Updated weights for policy 0, policy_version 1317263 (0.0007) [2023-12-27 14:46:34,027][20347] Updated weights for policy 0, policy_version 1317273 (0.0007) [2023-12-27 14:46:34,066][20381] Updated weights for policy 1, policy_version 1316409 (0.0007) [2023-12-27 14:46:34,119][20381] Updated weights for policy 1, policy_version 1316419 (0.0007) [2023-12-27 14:46:34,179][20381] Updated weights for policy 1, policy_version 1316429 (0.0009) [2023-12-27 14:46:34,686][20347] Updated weights for policy 0, policy_version 1317283 (0.0007) [2023-12-27 14:46:34,750][20347] Updated weights for policy 0, policy_version 1317293 (0.0006) [2023-12-27 14:46:34,813][20347] Updated weights for policy 0, policy_version 1317303 (0.0005) [2023-12-27 14:46:34,928][20381] Updated weights for policy 1, policy_version 1316439 (0.0007) [2023-12-27 14:46:34,981][20381] Updated weights for policy 1, policy_version 1316449 (0.0007) [2023-12-27 14:46:35,035][20381] Updated weights for policy 1, policy_version 1316459 (0.0007) [2023-12-27 14:46:35,373][20347] Updated weights for policy 0, policy_version 1317313 (0.0007) [2023-12-27 14:46:35,422][20347] Updated weights for policy 0, policy_version 1317323 (0.0007) [2023-12-27 14:46:35,471][20347] Updated weights for policy 0, policy_version 1317333 (0.0008) [2023-12-27 14:46:35,520][20347] Updated weights for policy 0, policy_version 1317343 (0.0007) [2023-12-27 14:46:35,750][20381] Updated weights for policy 1, policy_version 1316469 (0.0007) [2023-12-27 14:46:35,802][20381] Updated weights for policy 1, policy_version 1316479 (0.0008) [2023-12-27 14:46:35,855][20381] Updated weights for policy 1, policy_version 1316489 (0.0008) [2023-12-27 14:46:36,261][20347] Updated weights for policy 0, policy_version 1317353 (0.0007) [2023-12-27 14:46:36,318][20347] Updated weights for policy 0, policy_version 1317363 (0.0008) [2023-12-27 14:46:36,378][20347] Updated weights for policy 0, policy_version 1317373 (0.0008) [2023-12-27 14:46:36,576][20381] Updated weights for policy 1, policy_version 1316499 (0.0009) [2023-12-27 14:46:36,637][20381] Updated weights for policy 1, policy_version 1316509 (0.0008) [2023-12-27 14:46:36,699][20381] Updated weights for policy 1, policy_version 1316519 (0.0007) [2023-12-27 14:46:36,995][20347] Updated weights for policy 0, policy_version 1317383 (0.0006) [2023-12-27 14:46:37,064][20347] Updated weights for policy 0, policy_version 1317393 (0.0006) [2023-12-27 14:46:37,131][20347] Updated weights for policy 0, policy_version 1317403 (0.0007) [2023-12-27 14:46:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 674381824. Throughput: 0: 10259.3, 1: 10430.8. Samples: 674351612. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:37,363][19170] Avg episode reward: [(0, '685.210'), (1, '863.200')] [2023-12-27 14:46:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001316528_337059840.pth... [2023-12-27 14:46:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001317408_337321984.pth... [2023-12-27 14:46:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001315344_336756736.pth [2023-12-27 14:46:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001316224_337018880.pth [2023-12-27 14:46:37,401][20381] Updated weights for policy 1, policy_version 1316529 (0.0007) [2023-12-27 14:46:37,453][20381] Updated weights for policy 1, policy_version 1316539 (0.0006) [2023-12-27 14:46:37,505][20381] Updated weights for policy 1, policy_version 1316549 (0.0008) [2023-12-27 14:46:37,552][20381] Updated weights for policy 1, policy_version 1316559 (0.0007) [2023-12-27 14:46:37,793][20347] Updated weights for policy 0, policy_version 1317413 (0.0007) [2023-12-27 14:46:37,853][20347] Updated weights for policy 0, policy_version 1317423 (0.0008) [2023-12-27 14:46:37,908][20347] Updated weights for policy 0, policy_version 1317433 (0.0009) [2023-12-27 14:46:38,210][20381] Updated weights for policy 1, policy_version 1316569 (0.0008) [2023-12-27 14:46:38,259][20381] Updated weights for policy 1, policy_version 1316579 (0.0008) [2023-12-27 14:46:38,308][20381] Updated weights for policy 1, policy_version 1316589 (0.0008) [2023-12-27 14:46:38,541][20347] Updated weights for policy 0, policy_version 1317443 (0.0006) [2023-12-27 14:46:38,596][20347] Updated weights for policy 0, policy_version 1317453 (0.0005) [2023-12-27 14:46:38,656][20347] Updated weights for policy 0, policy_version 1317463 (0.0007) [2023-12-27 14:46:38,991][20381] Updated weights for policy 1, policy_version 1316599 (0.0007) [2023-12-27 14:46:39,048][20381] Updated weights for policy 1, policy_version 1316609 (0.0006) [2023-12-27 14:46:39,104][20381] Updated weights for policy 1, policy_version 1316619 (0.0005) [2023-12-27 14:46:39,301][20347] Updated weights for policy 0, policy_version 1317473 (0.0006) [2023-12-27 14:46:39,377][20347] Updated weights for policy 0, policy_version 1317483 (0.0007) [2023-12-27 14:46:39,446][20347] Updated weights for policy 0, policy_version 1317493 (0.0008) [2023-12-27 14:46:39,505][20347] Updated weights for policy 0, policy_version 1317503 (0.0008) [2023-12-27 14:46:39,736][20381] Updated weights for policy 1, policy_version 1316629 (0.0007) [2023-12-27 14:46:39,799][20381] Updated weights for policy 1, policy_version 1316639 (0.0008) [2023-12-27 14:46:39,869][20381] Updated weights for policy 1, policy_version 1316649 (0.0007) [2023-12-27 14:46:40,206][20347] Updated weights for policy 0, policy_version 1317513 (0.0008) [2023-12-27 14:46:40,267][20347] Updated weights for policy 0, policy_version 1317523 (0.0008) [2023-12-27 14:46:40,326][20347] Updated weights for policy 0, policy_version 1317533 (0.0008) [2023-12-27 14:46:40,572][20381] Updated weights for policy 1, policy_version 1316659 (0.0007) [2023-12-27 14:46:40,606][20292] KL-divergence is very high: 109.4268 [2023-12-27 14:46:40,620][20381] Updated weights for policy 1, policy_version 1316669 (0.0007) [2023-12-27 14:46:40,673][20381] Updated weights for policy 1, policy_version 1316679 (0.0007) [2023-12-27 14:46:40,988][20347] Updated weights for policy 0, policy_version 1317543 (0.0008) [2023-12-27 14:46:41,057][20347] Updated weights for policy 0, policy_version 1317553 (0.0009) [2023-12-27 14:46:41,109][20347] Updated weights for policy 0, policy_version 1317563 (0.0007) [2023-12-27 14:46:41,435][20381] Updated weights for policy 1, policy_version 1316689 (0.0008) [2023-12-27 14:46:41,490][20381] Updated weights for policy 1, policy_version 1316699 (0.0009) [2023-12-27 14:46:41,551][20381] Updated weights for policy 1, policy_version 1316709 (0.0008) [2023-12-27 14:46:41,608][20381] Updated weights for policy 1, policy_version 1316719 (0.0008) [2023-12-27 14:46:41,868][20347] Updated weights for policy 0, policy_version 1317573 (0.0008) [2023-12-27 14:46:41,923][20347] Updated weights for policy 0, policy_version 1317583 (0.0005) [2023-12-27 14:46:41,982][20347] Updated weights for policy 0, policy_version 1317593 (0.0005) [2023-12-27 14:46:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 674480128. Throughput: 0: 10279.9, 1: 10342.8. Samples: 674473512. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:42,363][19170] Avg episode reward: [(0, '741.630'), (1, '902.290')] [2023-12-27 14:46:42,392][20381] Updated weights for policy 1, policy_version 1316729 (0.0008) [2023-12-27 14:46:42,457][20381] Updated weights for policy 1, policy_version 1316739 (0.0008) [2023-12-27 14:46:42,515][20381] Updated weights for policy 1, policy_version 1316749 (0.0008) [2023-12-27 14:46:42,603][20347] Updated weights for policy 0, policy_version 1317603 (0.0007) [2023-12-27 14:46:42,653][20347] Updated weights for policy 0, policy_version 1317613 (0.0007) [2023-12-27 14:46:42,711][20347] Updated weights for policy 0, policy_version 1317623 (0.0008) [2023-12-27 14:46:43,225][20381] Updated weights for policy 1, policy_version 1316759 (0.0008) [2023-12-27 14:46:43,277][20381] Updated weights for policy 1, policy_version 1316769 (0.0008) [2023-12-27 14:46:43,323][20381] Updated weights for policy 1, policy_version 1316779 (0.0008) [2023-12-27 14:46:43,408][20347] Updated weights for policy 0, policy_version 1317633 (0.0008) [2023-12-27 14:46:43,466][20347] Updated weights for policy 0, policy_version 1317643 (0.0008) [2023-12-27 14:46:43,519][20347] Updated weights for policy 0, policy_version 1317653 (0.0007) [2023-12-27 14:46:43,575][20347] Updated weights for policy 0, policy_version 1317663 (0.0008) [2023-12-27 14:46:44,022][20381] Updated weights for policy 1, policy_version 1316789 (0.0009) [2023-12-27 14:46:44,082][20381] Updated weights for policy 1, policy_version 1316799 (0.0008) [2023-12-27 14:46:44,140][20381] Updated weights for policy 1, policy_version 1316809 (0.0008) [2023-12-27 14:46:44,208][20347] Updated weights for policy 0, policy_version 1317673 (0.0005) [2023-12-27 14:46:44,265][20347] Updated weights for policy 0, policy_version 1317683 (0.0007) [2023-12-27 14:46:44,317][20347] Updated weights for policy 0, policy_version 1317693 (0.0008) [2023-12-27 14:46:44,932][20381] Updated weights for policy 1, policy_version 1316819 (0.0008) [2023-12-27 14:46:44,967][20347] Updated weights for policy 0, policy_version 1317703 (0.0009) [2023-12-27 14:46:44,989][20381] Updated weights for policy 1, policy_version 1316829 (0.0011) [2023-12-27 14:46:45,031][20347] Updated weights for policy 0, policy_version 1317713 (0.0011) [2023-12-27 14:46:45,051][20381] Updated weights for policy 1, policy_version 1316839 (0.0011) [2023-12-27 14:46:45,075][20292] KL-divergence is very high: 142.9232 [2023-12-27 14:46:45,094][20347] Updated weights for policy 0, policy_version 1317723 (0.0011) [2023-12-27 14:46:45,820][20347] Updated weights for policy 0, policy_version 1317733 (0.0008) [2023-12-27 14:46:45,852][20381] Updated weights for policy 1, policy_version 1316849 (0.0008) [2023-12-27 14:46:45,881][20347] Updated weights for policy 0, policy_version 1317743 (0.0007) [2023-12-27 14:46:45,902][20381] Updated weights for policy 1, policy_version 1316859 (0.0007) [2023-12-27 14:46:45,933][20347] Updated weights for policy 0, policy_version 1317753 (0.0010) [2023-12-27 14:46:45,958][20381] Updated weights for policy 1, policy_version 1316869 (0.0007) [2023-12-27 14:46:46,015][20381] Updated weights for policy 1, policy_version 1316879 (0.0007) [2023-12-27 14:46:46,549][20347] Updated weights for policy 0, policy_version 1317763 (0.0009) [2023-12-27 14:46:46,607][20347] Updated weights for policy 0, policy_version 1317773 (0.0007) [2023-12-27 14:46:46,665][20347] Updated weights for policy 0, policy_version 1317783 (0.0010) [2023-12-27 14:46:46,740][20381] Updated weights for policy 1, policy_version 1316889 (0.0009) [2023-12-27 14:46:46,787][20381] Updated weights for policy 1, policy_version 1316899 (0.0010) [2023-12-27 14:46:46,844][20381] Updated weights for policy 1, policy_version 1316909 (0.0008) [2023-12-27 14:46:47,280][20347] Updated weights for policy 0, policy_version 1317793 (0.0008) [2023-12-27 14:46:47,338][20347] Updated weights for policy 0, policy_version 1317803 (0.0006) [2023-12-27 14:46:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 674578432. Throughput: 0: 10310.4, 1: 10226.2. Samples: 674592836. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:47,364][19170] Avg episode reward: [(0, '697.600'), (1, '911.970')] [2023-12-27 14:46:47,393][20347] Updated weights for policy 0, policy_version 1317813 (0.0005) [2023-12-27 14:46:47,452][20347] Updated weights for policy 0, policy_version 1317823 (0.0008) [2023-12-27 14:46:47,514][20381] Updated weights for policy 1, policy_version 1316919 (0.0007) [2023-12-27 14:46:47,566][20381] Updated weights for policy 1, policy_version 1316929 (0.0007) [2023-12-27 14:46:47,624][20381] Updated weights for policy 1, policy_version 1316939 (0.0007) [2023-12-27 14:46:48,115][20347] Updated weights for policy 0, policy_version 1317833 (0.0007) [2023-12-27 14:46:48,178][20347] Updated weights for policy 0, policy_version 1317843 (0.0008) [2023-12-27 14:46:48,236][20347] Updated weights for policy 0, policy_version 1317853 (0.0008) [2023-12-27 14:46:48,324][20381] Updated weights for policy 1, policy_version 1316949 (0.0008) [2023-12-27 14:46:48,382][20381] Updated weights for policy 1, policy_version 1316959 (0.0010) [2023-12-27 14:46:48,437][20381] Updated weights for policy 1, policy_version 1316969 (0.0008) [2023-12-27 14:46:49,000][20347] Updated weights for policy 0, policy_version 1317863 (0.0008) [2023-12-27 14:46:49,061][20347] Updated weights for policy 0, policy_version 1317873 (0.0008) [2023-12-27 14:46:49,068][20381] Updated weights for policy 1, policy_version 1316979 (0.0007) [2023-12-27 14:46:49,124][20347] Updated weights for policy 0, policy_version 1317883 (0.0006) [2023-12-27 14:46:49,132][20381] Updated weights for policy 1, policy_version 1316989 (0.0005) [2023-12-27 14:46:49,197][20381] Updated weights for policy 1, policy_version 1316999 (0.0005) [2023-12-27 14:46:49,800][20347] Updated weights for policy 0, policy_version 1317893 (0.0005) [2023-12-27 14:46:49,864][20347] Updated weights for policy 0, policy_version 1317903 (0.0007) [2023-12-27 14:46:49,873][20381] Updated weights for policy 1, policy_version 1317009 (0.0009) [2023-12-27 14:46:49,930][20347] Updated weights for policy 0, policy_version 1317913 (0.0007) [2023-12-27 14:46:49,939][20381] Updated weights for policy 1, policy_version 1317019 (0.0006) [2023-12-27 14:46:50,000][20381] Updated weights for policy 1, policy_version 1317029 (0.0006) [2023-12-27 14:46:50,059][20381] Updated weights for policy 1, policy_version 1317039 (0.0006) [2023-12-27 14:46:50,625][20347] Updated weights for policy 0, policy_version 1317923 (0.0008) [2023-12-27 14:46:50,684][20347] Updated weights for policy 0, policy_version 1317933 (0.0007) [2023-12-27 14:46:50,716][20381] Updated weights for policy 1, policy_version 1317049 (0.0006) [2023-12-27 14:46:50,744][20347] Updated weights for policy 0, policy_version 1317943 (0.0007) [2023-12-27 14:46:50,765][20381] Updated weights for policy 1, policy_version 1317059 (0.0006) [2023-12-27 14:46:50,820][20381] Updated weights for policy 1, policy_version 1317069 (0.0006) [2023-12-27 14:46:51,408][20347] Updated weights for policy 0, policy_version 1317953 (0.0006) [2023-12-27 14:46:51,472][20347] Updated weights for policy 0, policy_version 1317963 (0.0008) [2023-12-27 14:46:51,534][20347] Updated weights for policy 0, policy_version 1317973 (0.0007) [2023-12-27 14:46:51,583][20381] Updated weights for policy 1, policy_version 1317079 (0.0009) [2023-12-27 14:46:51,595][20347] Updated weights for policy 0, policy_version 1317983 (0.0007) [2023-12-27 14:46:51,647][20381] Updated weights for policy 1, policy_version 1317089 (0.0007) [2023-12-27 14:46:51,706][20381] Updated weights for policy 1, policy_version 1317099 (0.0008) [2023-12-27 14:46:52,328][20347] Updated weights for policy 0, policy_version 1317993 (0.0007) [2023-12-27 14:46:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 674676736. Throughput: 0: 10271.0, 1: 10226.0. Samples: 674653512. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:52,363][19170] Avg episode reward: [(0, '744.910'), (1, '814.360')] [2023-12-27 14:46:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001317104_337207296.pth... [2023-12-27 14:46:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001315920_336904192.pth [2023-12-27 14:46:52,394][20347] Updated weights for policy 0, policy_version 1318003 (0.0008) [2023-12-27 14:46:52,444][20347] Updated weights for policy 0, policy_version 1318013 (0.0007) [2023-12-27 14:46:52,448][20381] Updated weights for policy 1, policy_version 1317109 (0.0008) [2023-12-27 14:46:52,461][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001318016_337477632.pth... [2023-12-27 14:46:52,466][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001316800_337166336.pth [2023-12-27 14:46:52,507][20381] Updated weights for policy 1, policy_version 1317119 (0.0010) [2023-12-27 14:46:52,562][20381] Updated weights for policy 1, policy_version 1317129 (0.0010) [2023-12-27 14:46:53,068][20347] Updated weights for policy 0, policy_version 1318023 (0.0006) [2023-12-27 14:46:53,114][20347] Updated weights for policy 0, policy_version 1318033 (0.0005) [2023-12-27 14:46:53,159][20347] Updated weights for policy 0, policy_version 1318043 (0.0005) [2023-12-27 14:46:53,233][20381] Updated weights for policy 1, policy_version 1317139 (0.0009) [2023-12-27 14:46:53,286][20381] Updated weights for policy 1, policy_version 1317149 (0.0006) [2023-12-27 14:46:53,343][20381] Updated weights for policy 1, policy_version 1317159 (0.0007) [2023-12-27 14:46:53,784][20347] Updated weights for policy 0, policy_version 1318053 (0.0007) [2023-12-27 14:46:53,839][20347] Updated weights for policy 0, policy_version 1318063 (0.0006) [2023-12-27 14:46:53,887][20347] Updated weights for policy 0, policy_version 1318073 (0.0006) [2023-12-27 14:46:53,899][20381] Updated weights for policy 1, policy_version 1317169 (0.0007) [2023-12-27 14:46:53,944][20381] Updated weights for policy 1, policy_version 1317179 (0.0005) [2023-12-27 14:46:53,995][20381] Updated weights for policy 1, policy_version 1317189 (0.0005) [2023-12-27 14:46:54,053][20381] Updated weights for policy 1, policy_version 1317199 (0.0005) [2023-12-27 14:46:54,535][20347] Updated weights for policy 0, policy_version 1318083 (0.0007) [2023-12-27 14:46:54,581][20347] Updated weights for policy 0, policy_version 1318093 (0.0005) [2023-12-27 14:46:54,625][20347] Updated weights for policy 0, policy_version 1318103 (0.0005) [2023-12-27 14:46:54,702][20381] Updated weights for policy 1, policy_version 1317209 (0.0008) [2023-12-27 14:46:54,764][20381] Updated weights for policy 1, policy_version 1317219 (0.0008) [2023-12-27 14:46:54,822][20381] Updated weights for policy 1, policy_version 1317229 (0.0008) [2023-12-27 14:46:55,280][20347] Updated weights for policy 0, policy_version 1318113 (0.0005) [2023-12-27 14:46:55,342][20347] Updated weights for policy 0, policy_version 1318123 (0.0005) [2023-12-27 14:46:55,391][20347] Updated weights for policy 0, policy_version 1318133 (0.0005) [2023-12-27 14:46:55,447][20347] Updated weights for policy 0, policy_version 1318143 (0.0005) [2023-12-27 14:46:55,505][20381] Updated weights for policy 1, policy_version 1317239 (0.0007) [2023-12-27 14:46:55,559][20381] Updated weights for policy 1, policy_version 1317249 (0.0007) [2023-12-27 14:46:55,608][20381] Updated weights for policy 1, policy_version 1317259 (0.0010) [2023-12-27 14:46:56,013][20347] Updated weights for policy 0, policy_version 1318153 (0.0006) [2023-12-27 14:46:56,076][20347] Updated weights for policy 0, policy_version 1318163 (0.0008) [2023-12-27 14:46:56,129][20347] Updated weights for policy 0, policy_version 1318173 (0.0008) [2023-12-27 14:46:56,264][20381] Updated weights for policy 1, policy_version 1317269 (0.0008) [2023-12-27 14:46:56,312][20381] Updated weights for policy 1, policy_version 1317279 (0.0008) [2023-12-27 14:46:56,363][20381] Updated weights for policy 1, policy_version 1317289 (0.0007) [2023-12-27 14:46:56,775][20347] Updated weights for policy 0, policy_version 1318183 (0.0007) [2023-12-27 14:46:56,825][20347] Updated weights for policy 0, policy_version 1318193 (0.0005) [2023-12-27 14:46:56,875][20347] Updated weights for policy 0, policy_version 1318203 (0.0005) [2023-12-27 14:46:57,075][20381] Updated weights for policy 1, policy_version 1317299 (0.0008) [2023-12-27 14:46:57,133][20381] Updated weights for policy 1, policy_version 1317309 (0.0007) [2023-12-27 14:46:57,180][20381] Updated weights for policy 1, policy_version 1317319 (0.0007) [2023-12-27 14:46:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 674791424. Throughput: 0: 10249.2, 1: 10249.0. Samples: 674779236. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:46:57,363][19170] Avg episode reward: [(0, '757.640'), (1, '804.830')] [2023-12-27 14:46:57,476][20347] Updated weights for policy 0, policy_version 1318213 (0.0006) [2023-12-27 14:46:57,531][20347] Updated weights for policy 0, policy_version 1318223 (0.0005) [2023-12-27 14:46:57,574][20347] Updated weights for policy 0, policy_version 1318233 (0.0005) [2023-12-27 14:46:57,925][20381] Updated weights for policy 1, policy_version 1317329 (0.0007) [2023-12-27 14:46:57,975][20381] Updated weights for policy 1, policy_version 1317339 (0.0006) [2023-12-27 14:46:58,020][20381] Updated weights for policy 1, policy_version 1317349 (0.0008) [2023-12-27 14:46:58,063][20381] Updated weights for policy 1, policy_version 1317359 (0.0009) [2023-12-27 14:46:58,113][20347] Updated weights for policy 0, policy_version 1318243 (0.0006) [2023-12-27 14:46:58,167][20347] Updated weights for policy 0, policy_version 1318253 (0.0007) [2023-12-27 14:46:58,233][20347] Updated weights for policy 0, policy_version 1318263 (0.0007) [2023-12-27 14:46:58,849][20381] Updated weights for policy 1, policy_version 1317369 (0.0009) [2023-12-27 14:46:58,912][20381] Updated weights for policy 1, policy_version 1317379 (0.0007) [2023-12-27 14:46:58,978][20381] Updated weights for policy 1, policy_version 1317389 (0.0007) [2023-12-27 14:46:59,046][20347] Updated weights for policy 0, policy_version 1318273 (0.0007) [2023-12-27 14:46:59,100][20347] Updated weights for policy 0, policy_version 1318283 (0.0007) [2023-12-27 14:46:59,160][20347] Updated weights for policy 0, policy_version 1318293 (0.0005) [2023-12-27 14:46:59,222][20347] Updated weights for policy 0, policy_version 1318303 (0.0007) [2023-12-27 14:46:59,728][20381] Updated weights for policy 1, policy_version 1317399 (0.0008) [2023-12-27 14:46:59,783][20381] Updated weights for policy 1, policy_version 1317409 (0.0009) [2023-12-27 14:46:59,845][20381] Updated weights for policy 1, policy_version 1317419 (0.0008) [2023-12-27 14:46:59,873][20347] Updated weights for policy 0, policy_version 1318313 (0.0008) [2023-12-27 14:46:59,922][20347] Updated weights for policy 0, policy_version 1318323 (0.0007) [2023-12-27 14:46:59,984][20347] Updated weights for policy 0, policy_version 1318333 (0.0007) [2023-12-27 14:47:00,548][20381] Updated weights for policy 1, policy_version 1317429 (0.0009) [2023-12-27 14:47:00,612][20381] Updated weights for policy 1, policy_version 1317439 (0.0011) [2023-12-27 14:47:00,621][20347] Updated weights for policy 0, policy_version 1318343 (0.0006) [2023-12-27 14:47:00,668][20381] Updated weights for policy 1, policy_version 1317449 (0.0011) [2023-12-27 14:47:00,677][20347] Updated weights for policy 0, policy_version 1318353 (0.0006) [2023-12-27 14:47:00,727][20347] Updated weights for policy 0, policy_version 1318363 (0.0006) [2023-12-27 14:47:01,299][20381] Updated weights for policy 1, policy_version 1317459 (0.0011) [2023-12-27 14:47:01,352][20381] Updated weights for policy 1, policy_version 1317469 (0.0011) [2023-12-27 14:47:01,412][20381] Updated weights for policy 1, policy_version 1317479 (0.0009) [2023-12-27 14:47:01,477][20347] Updated weights for policy 0, policy_version 1318373 (0.0006) [2023-12-27 14:47:01,528][20347] Updated weights for policy 0, policy_version 1318383 (0.0007) [2023-12-27 14:47:01,588][20347] Updated weights for policy 0, policy_version 1318393 (0.0007) [2023-12-27 14:47:02,140][20381] Updated weights for policy 1, policy_version 1317489 (0.0010) [2023-12-27 14:47:02,206][20381] Updated weights for policy 1, policy_version 1317499 (0.0006) [2023-12-27 14:47:02,264][20381] Updated weights for policy 1, policy_version 1317509 (0.0007) [2023-12-27 14:47:02,276][20347] Updated weights for policy 0, policy_version 1318403 (0.0007) [2023-12-27 14:47:02,317][20381] Updated weights for policy 1, policy_version 1317519 (0.0005) [2023-12-27 14:47:02,333][20347] Updated weights for policy 0, policy_version 1318413 (0.0006) [2023-12-27 14:47:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 674889728. Throughput: 0: 10334.9, 1: 10144.0. Samples: 674900744. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:02,363][19170] Avg episode reward: [(0, '776.630'), (1, '834.270')] [2023-12-27 14:47:02,389][20347] Updated weights for policy 0, policy_version 1318423 (0.0007) [2023-12-27 14:47:02,978][20381] Updated weights for policy 1, policy_version 1317529 (0.0007) [2023-12-27 14:47:03,021][20381] Updated weights for policy 1, policy_version 1317539 (0.0008) [2023-12-27 14:47:03,049][20347] Updated weights for policy 0, policy_version 1318433 (0.0008) [2023-12-27 14:47:03,073][20381] Updated weights for policy 1, policy_version 1317549 (0.0009) [2023-12-27 14:47:03,102][20347] Updated weights for policy 0, policy_version 1318443 (0.0006) [2023-12-27 14:47:03,154][20347] Updated weights for policy 0, policy_version 1318453 (0.0009) [2023-12-27 14:47:03,208][20347] Updated weights for policy 0, policy_version 1318463 (0.0007) [2023-12-27 14:47:03,653][20381] Updated weights for policy 1, policy_version 1317559 (0.0006) [2023-12-27 14:47:03,700][20381] Updated weights for policy 1, policy_version 1317569 (0.0005) [2023-12-27 14:47:03,750][20381] Updated weights for policy 1, policy_version 1317579 (0.0005) [2023-12-27 14:47:03,960][20347] Updated weights for policy 0, policy_version 1318473 (0.0008) [2023-12-27 14:47:04,014][20347] Updated weights for policy 0, policy_version 1318483 (0.0007) [2023-12-27 14:47:04,071][20347] Updated weights for policy 0, policy_version 1318493 (0.0006) [2023-12-27 14:47:04,363][20381] Updated weights for policy 1, policy_version 1317589 (0.0005) [2023-12-27 14:47:04,419][20381] Updated weights for policy 1, policy_version 1317599 (0.0008) [2023-12-27 14:47:04,469][20292] KL-divergence is very high: 401.2245 [2023-12-27 14:47:04,482][20381] Updated weights for policy 1, policy_version 1317609 (0.0011) [2023-12-27 14:47:04,518][20292] KL-divergence is very high: 386.1447 [2023-12-27 14:47:04,738][20347] Updated weights for policy 0, policy_version 1318503 (0.0006) [2023-12-27 14:47:04,785][20347] Updated weights for policy 0, policy_version 1318513 (0.0005) [2023-12-27 14:47:04,831][20347] Updated weights for policy 0, policy_version 1318523 (0.0005) [2023-12-27 14:47:05,040][20381] Updated weights for policy 1, policy_version 1317619 (0.0010) [2023-12-27 14:47:05,091][20381] Updated weights for policy 1, policy_version 1317629 (0.0005) [2023-12-27 14:47:05,148][20381] Updated weights for policy 1, policy_version 1317639 (0.0005) [2023-12-27 14:47:05,496][20347] Updated weights for policy 0, policy_version 1318533 (0.0007) [2023-12-27 14:47:05,547][20347] Updated weights for policy 0, policy_version 1318543 (0.0005) [2023-12-27 14:47:05,596][20347] Updated weights for policy 0, policy_version 1318553 (0.0005) [2023-12-27 14:47:05,694][20381] Updated weights for policy 1, policy_version 1317649 (0.0009) [2023-12-27 14:47:05,738][20292] KL-divergence is very high: 122.5975 [2023-12-27 14:47:05,752][20381] Updated weights for policy 1, policy_version 1317659 (0.0008) [2023-12-27 14:47:05,783][20292] KL-divergence is very high: 124.7049 [2023-12-27 14:47:05,813][20381] Updated weights for policy 1, policy_version 1317669 (0.0010) [2023-12-27 14:47:05,878][20381] Updated weights for policy 1, policy_version 1317679 (0.0011) [2023-12-27 14:47:06,223][20347] Updated weights for policy 0, policy_version 1318563 (0.0006) [2023-12-27 14:47:06,287][20347] Updated weights for policy 0, policy_version 1318573 (0.0006) [2023-12-27 14:47:06,348][20347] Updated weights for policy 0, policy_version 1318583 (0.0005) [2023-12-27 14:47:06,531][20292] KL-divergence is very high: 153.4896 [2023-12-27 14:47:06,536][20381] Updated weights for policy 1, policy_version 1317689 (0.0011) [2023-12-27 14:47:06,616][20381] Updated weights for policy 1, policy_version 1317699 (0.0011) [2023-12-27 14:47:06,676][20381] Updated weights for policy 1, policy_version 1317709 (0.0011) [2023-12-27 14:47:06,919][20347] Updated weights for policy 0, policy_version 1318593 (0.0006) [2023-12-27 14:47:06,976][20347] Updated weights for policy 0, policy_version 1318603 (0.0009) [2023-12-27 14:47:07,026][20347] Updated weights for policy 0, policy_version 1318613 (0.0010) [2023-12-27 14:47:07,088][20347] Updated weights for policy 0, policy_version 1318623 (0.0010) [2023-12-27 14:47:07,310][20381] Updated weights for policy 1, policy_version 1317719 (0.0008) [2023-12-27 14:47:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 674996224. Throughput: 0: 10306.5, 1: 10220.0. Samples: 674965000. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:07,363][19170] Avg episode reward: [(0, '786.840'), (1, '824.240')] [2023-12-27 14:47:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001318624_337633280.pth... [2023-12-27 14:47:07,368][20381] Updated weights for policy 1, policy_version 1317729 (0.0007) [2023-12-27 14:47:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001317408_337321984.pth [2023-12-27 14:47:07,427][20381] Updated weights for policy 1, policy_version 1317739 (0.0007) [2023-12-27 14:47:07,456][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001317744_337371136.pth... [2023-12-27 14:47:07,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001316528_337059840.pth [2023-12-27 14:47:07,682][20347] Updated weights for policy 0, policy_version 1318633 (0.0008) [2023-12-27 14:47:07,734][20347] Updated weights for policy 0, policy_version 1318643 (0.0009) [2023-12-27 14:47:07,789][20347] Updated weights for policy 0, policy_version 1318653 (0.0007) [2023-12-27 14:47:08,059][20381] Updated weights for policy 1, policy_version 1317749 (0.0007) [2023-12-27 14:47:08,108][20381] Updated weights for policy 1, policy_version 1317759 (0.0005) [2023-12-27 14:47:08,155][20381] Updated weights for policy 1, policy_version 1317769 (0.0005) [2023-12-27 14:47:08,480][20347] Updated weights for policy 0, policy_version 1318663 (0.0008) [2023-12-27 14:47:08,539][20347] Updated weights for policy 0, policy_version 1318673 (0.0008) [2023-12-27 14:47:08,598][20347] Updated weights for policy 0, policy_version 1318683 (0.0008) [2023-12-27 14:47:08,765][20381] Updated weights for policy 1, policy_version 1317779 (0.0006) [2023-12-27 14:47:08,822][20381] Updated weights for policy 1, policy_version 1317789 (0.0007) [2023-12-27 14:47:08,884][20381] Updated weights for policy 1, policy_version 1317799 (0.0007) [2023-12-27 14:47:09,324][20347] Updated weights for policy 0, policy_version 1318693 (0.0008) [2023-12-27 14:47:09,392][20347] Updated weights for policy 0, policy_version 1318703 (0.0008) [2023-12-27 14:47:09,455][20347] Updated weights for policy 0, policy_version 1318713 (0.0007) [2023-12-27 14:47:09,640][20381] Updated weights for policy 1, policy_version 1317809 (0.0007) [2023-12-27 14:47:09,697][20381] Updated weights for policy 1, policy_version 1317819 (0.0006) [2023-12-27 14:47:09,743][20381] Updated weights for policy 1, policy_version 1317829 (0.0008) [2023-12-27 14:47:09,789][20381] Updated weights for policy 1, policy_version 1317839 (0.0008) [2023-12-27 14:47:10,118][20347] Updated weights for policy 0, policy_version 1318723 (0.0006) [2023-12-27 14:47:10,176][20347] Updated weights for policy 0, policy_version 1318733 (0.0008) [2023-12-27 14:47:10,239][20347] Updated weights for policy 0, policy_version 1318743 (0.0007) [2023-12-27 14:47:10,540][20381] Updated weights for policy 1, policy_version 1317849 (0.0008) [2023-12-27 14:47:10,585][20381] Updated weights for policy 1, policy_version 1317859 (0.0008) [2023-12-27 14:47:10,634][20381] Updated weights for policy 1, policy_version 1317869 (0.0008) [2023-12-27 14:47:10,901][20347] Updated weights for policy 0, policy_version 1318753 (0.0008) [2023-12-27 14:47:10,959][20347] Updated weights for policy 0, policy_version 1318763 (0.0008) [2023-12-27 14:47:11,013][20347] Updated weights for policy 0, policy_version 1318773 (0.0007) [2023-12-27 14:47:11,079][20347] Updated weights for policy 0, policy_version 1318783 (0.0007) [2023-12-27 14:47:11,395][20381] Updated weights for policy 1, policy_version 1317879 (0.0008) [2023-12-27 14:47:11,455][20381] Updated weights for policy 1, policy_version 1317889 (0.0006) [2023-12-27 14:47:11,519][20381] Updated weights for policy 1, policy_version 1317899 (0.0006) [2023-12-27 14:47:11,800][20347] Updated weights for policy 0, policy_version 1318793 (0.0006) [2023-12-27 14:47:11,849][20347] Updated weights for policy 0, policy_version 1318803 (0.0006) [2023-12-27 14:47:11,904][20347] Updated weights for policy 0, policy_version 1318813 (0.0007) [2023-12-27 14:47:12,234][20381] Updated weights for policy 1, policy_version 1317909 (0.0008) [2023-12-27 14:47:12,295][20381] Updated weights for policy 1, policy_version 1317919 (0.0008) [2023-12-27 14:47:12,359][20381] Updated weights for policy 1, policy_version 1317929 (0.0009) [2023-12-27 14:47:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 675094528. Throughput: 0: 10401.9, 1: 10177.7. Samples: 675090272. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:12,363][19170] Avg episode reward: [(0, '795.720'), (1, '794.700')] [2023-12-27 14:47:12,637][20347] Updated weights for policy 0, policy_version 1318823 (0.0006) [2023-12-27 14:47:12,699][20347] Updated weights for policy 0, policy_version 1318833 (0.0006) [2023-12-27 14:47:12,758][20347] Updated weights for policy 0, policy_version 1318843 (0.0006) [2023-12-27 14:47:13,071][20381] Updated weights for policy 1, policy_version 1317939 (0.0010) [2023-12-27 14:47:13,128][20381] Updated weights for policy 1, policy_version 1317949 (0.0010) [2023-12-27 14:47:13,188][20381] Updated weights for policy 1, policy_version 1317959 (0.0008) [2023-12-27 14:47:13,442][20347] Updated weights for policy 0, policy_version 1318853 (0.0007) [2023-12-27 14:47:13,497][20347] Updated weights for policy 0, policy_version 1318863 (0.0007) [2023-12-27 14:47:13,551][20347] Updated weights for policy 0, policy_version 1318873 (0.0008) [2023-12-27 14:47:13,730][20381] Updated weights for policy 1, policy_version 1317969 (0.0005) [2023-12-27 14:47:13,789][20381] Updated weights for policy 1, policy_version 1317979 (0.0005) [2023-12-27 14:47:13,847][20381] Updated weights for policy 1, policy_version 1317989 (0.0010) [2023-12-27 14:47:13,870][20292] KL-divergence is very high: 142.8311 [2023-12-27 14:47:13,883][20292] KL-divergence is very high: 141.5146 [2023-12-27 14:47:13,908][20381] Updated weights for policy 1, policy_version 1317999 (0.0010) [2023-12-27 14:47:14,122][20347] Updated weights for policy 0, policy_version 1318883 (0.0006) [2023-12-27 14:47:14,177][20347] Updated weights for policy 0, policy_version 1318894 (0.0010) [2023-12-27 14:47:14,225][20347] Updated weights for policy 0, policy_version 1318904 (0.0007) [2023-12-27 14:47:14,488][20381] Updated weights for policy 1, policy_version 1318009 (0.0010) [2023-12-27 14:47:14,544][20381] Updated weights for policy 1, policy_version 1318019 (0.0010) [2023-12-27 14:47:14,608][20381] Updated weights for policy 1, policy_version 1318029 (0.0011) [2023-12-27 14:47:14,890][20347] Updated weights for policy 0, policy_version 1318914 (0.0010) [2023-12-27 14:47:14,953][20347] Updated weights for policy 0, policy_version 1318924 (0.0010) [2023-12-27 14:47:15,012][20347] Updated weights for policy 0, policy_version 1318934 (0.0010) [2023-12-27 14:47:15,068][20347] Updated weights for policy 0, policy_version 1318944 (0.0006) [2023-12-27 14:47:15,296][20381] Updated weights for policy 1, policy_version 1318039 (0.0008) [2023-12-27 14:47:15,354][20381] Updated weights for policy 1, policy_version 1318049 (0.0007) [2023-12-27 14:47:15,408][20381] Updated weights for policy 1, policy_version 1318059 (0.0007) [2023-12-27 14:47:15,712][20347] Updated weights for policy 0, policy_version 1318954 (0.0009) [2023-12-27 14:47:15,770][20347] Updated weights for policy 0, policy_version 1318964 (0.0010) [2023-12-27 14:47:15,837][20347] Updated weights for policy 0, policy_version 1318974 (0.0009) [2023-12-27 14:47:16,079][20381] Updated weights for policy 1, policy_version 1318069 (0.0008) [2023-12-27 14:47:16,132][20381] Updated weights for policy 1, policy_version 1318079 (0.0007) [2023-12-27 14:47:16,185][20381] Updated weights for policy 1, policy_version 1318089 (0.0008) [2023-12-27 14:47:16,456][20347] Updated weights for policy 0, policy_version 1318984 (0.0006) [2023-12-27 14:47:16,517][20347] Updated weights for policy 0, policy_version 1318994 (0.0005) [2023-12-27 14:47:16,576][20347] Updated weights for policy 0, policy_version 1319004 (0.0009) [2023-12-27 14:47:16,961][20381] Updated weights for policy 1, policy_version 1318099 (0.0010) [2023-12-27 14:47:17,013][20381] Updated weights for policy 1, policy_version 1318109 (0.0010) [2023-12-27 14:47:17,071][20381] Updated weights for policy 1, policy_version 1318119 (0.0010) [2023-12-27 14:47:17,116][20347] Updated weights for policy 0, policy_version 1319014 (0.0008) [2023-12-27 14:47:17,173][20347] Updated weights for policy 0, policy_version 1319024 (0.0010) [2023-12-27 14:47:17,228][20347] Updated weights for policy 0, policy_version 1319034 (0.0010) [2023-12-27 14:47:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 675209216. Throughput: 0: 10419.3, 1: 10122.4. Samples: 675215372. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:17,363][19170] Avg episode reward: [(0, '795.100'), (1, '814.320')] [2023-12-27 14:47:17,754][20381] Updated weights for policy 1, policy_version 1318129 (0.0010) [2023-12-27 14:47:17,809][20381] Updated weights for policy 1, policy_version 1318139 (0.0011) [2023-12-27 14:47:17,873][20381] Updated weights for policy 1, policy_version 1318149 (0.0011) [2023-12-27 14:47:17,909][20347] Updated weights for policy 0, policy_version 1319044 (0.0010) [2023-12-27 14:47:17,932][20381] Updated weights for policy 1, policy_version 1318159 (0.0011) [2023-12-27 14:47:17,962][20347] Updated weights for policy 0, policy_version 1319054 (0.0010) [2023-12-27 14:47:18,014][20347] Updated weights for policy 0, policy_version 1319064 (0.0010) [2023-12-27 14:47:18,629][20381] Updated weights for policy 1, policy_version 1318169 (0.0011) [2023-12-27 14:47:18,658][20347] Updated weights for policy 0, policy_version 1319074 (0.0011) [2023-12-27 14:47:18,683][20381] Updated weights for policy 1, policy_version 1318179 (0.0011) [2023-12-27 14:47:18,716][20347] Updated weights for policy 0, policy_version 1319084 (0.0011) [2023-12-27 14:47:18,739][20381] Updated weights for policy 1, policy_version 1318189 (0.0011) [2023-12-27 14:47:18,773][20347] Updated weights for policy 0, policy_version 1319094 (0.0011) [2023-12-27 14:47:18,822][20347] Updated weights for policy 0, policy_version 1319104 (0.0010) [2023-12-27 14:47:19,447][20381] Updated weights for policy 1, policy_version 1318199 (0.0007) [2023-12-27 14:47:19,511][20381] Updated weights for policy 1, policy_version 1318209 (0.0008) [2023-12-27 14:47:19,554][20347] Updated weights for policy 0, policy_version 1319114 (0.0010) [2023-12-27 14:47:19,571][20381] Updated weights for policy 1, policy_version 1318219 (0.0008) [2023-12-27 14:47:19,609][20347] Updated weights for policy 0, policy_version 1319124 (0.0007) [2023-12-27 14:47:19,665][20347] Updated weights for policy 0, policy_version 1319134 (0.0007) [2023-12-27 14:47:20,300][20381] Updated weights for policy 1, policy_version 1318229 (0.0008) [2023-12-27 14:47:20,349][20347] Updated weights for policy 0, policy_version 1319144 (0.0010) [2023-12-27 14:47:20,359][20381] Updated weights for policy 1, policy_version 1318239 (0.0008) [2023-12-27 14:47:20,405][20347] Updated weights for policy 0, policy_version 1319154 (0.0010) [2023-12-27 14:47:20,415][20381] Updated weights for policy 1, policy_version 1318249 (0.0007) [2023-12-27 14:47:20,466][20347] Updated weights for policy 0, policy_version 1319164 (0.0007) [2023-12-27 14:47:21,142][20347] Updated weights for policy 0, policy_version 1319174 (0.0009) [2023-12-27 14:47:21,143][20381] Updated weights for policy 1, policy_version 1318259 (0.0008) [2023-12-27 14:47:21,190][20347] Updated weights for policy 0, policy_version 1319184 (0.0008) [2023-12-27 14:47:21,202][20381] Updated weights for policy 1, policy_version 1318269 (0.0007) [2023-12-27 14:47:21,246][20347] Updated weights for policy 0, policy_version 1319194 (0.0008) [2023-12-27 14:47:21,261][20381] Updated weights for policy 1, policy_version 1318279 (0.0007) [2023-12-27 14:47:21,939][20347] Updated weights for policy 0, policy_version 1319204 (0.0007) [2023-12-27 14:47:21,996][20347] Updated weights for policy 0, policy_version 1319214 (0.0008) [2023-12-27 14:47:22,014][20381] Updated weights for policy 1, policy_version 1318289 (0.0007) [2023-12-27 14:47:22,055][20347] Updated weights for policy 0, policy_version 1319224 (0.0006) [2023-12-27 14:47:22,075][20381] Updated weights for policy 1, policy_version 1318299 (0.0006) [2023-12-27 14:47:22,130][20381] Updated weights for policy 1, policy_version 1318309 (0.0006) [2023-12-27 14:47:22,176][20381] Updated weights for policy 1, policy_version 1318319 (0.0005) [2023-12-27 14:47:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 675307520. Throughput: 0: 10402.5, 1: 10121.8. Samples: 675275204. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:22,363][19170] Avg episode reward: [(0, '766.930'), (1, '882.580')] [2023-12-27 14:47:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001318320_337518592.pth... [2023-12-27 14:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001319232_337788928.pth... [2023-12-27 14:47:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001317104_337207296.pth [2023-12-27 14:47:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001318016_337477632.pth [2023-12-27 14:47:22,773][20347] Updated weights for policy 0, policy_version 1319234 (0.0006) [2023-12-27 14:47:22,833][20347] Updated weights for policy 0, policy_version 1319244 (0.0006) [2023-12-27 14:47:22,861][20381] Updated weights for policy 1, policy_version 1318329 (0.0006) [2023-12-27 14:47:22,889][20347] Updated weights for policy 0, policy_version 1319254 (0.0006) [2023-12-27 14:47:22,918][20381] Updated weights for policy 1, policy_version 1318339 (0.0008) [2023-12-27 14:47:22,940][20347] Updated weights for policy 0, policy_version 1319264 (0.0006) [2023-12-27 14:47:22,977][20381] Updated weights for policy 1, policy_version 1318349 (0.0007) [2023-12-27 14:47:23,599][20347] Updated weights for policy 0, policy_version 1319274 (0.0008) [2023-12-27 14:47:23,631][20381] Updated weights for policy 1, policy_version 1318359 (0.0006) [2023-12-27 14:47:23,648][20347] Updated weights for policy 0, policy_version 1319284 (0.0007) [2023-12-27 14:47:23,685][20381] Updated weights for policy 1, policy_version 1318369 (0.0005) [2023-12-27 14:47:23,697][20347] Updated weights for policy 0, policy_version 1319294 (0.0008) [2023-12-27 14:47:23,730][20381] Updated weights for policy 1, policy_version 1318379 (0.0005) [2023-12-27 14:47:24,307][20381] Updated weights for policy 1, policy_version 1318389 (0.0006) [2023-12-27 14:47:24,369][20381] Updated weights for policy 1, policy_version 1318399 (0.0008) [2023-12-27 14:47:24,423][20381] Updated weights for policy 1, policy_version 1318409 (0.0008) [2023-12-27 14:47:24,444][20347] Updated weights for policy 0, policy_version 1319304 (0.0006) [2023-12-27 14:47:24,502][20347] Updated weights for policy 0, policy_version 1319314 (0.0007) [2023-12-27 14:47:24,562][20347] Updated weights for policy 0, policy_version 1319324 (0.0008) [2023-12-27 14:47:25,073][20381] Updated weights for policy 1, policy_version 1318419 (0.0006) [2023-12-27 14:47:25,131][20381] Updated weights for policy 1, policy_version 1318429 (0.0007) [2023-12-27 14:47:25,189][20381] Updated weights for policy 1, policy_version 1318439 (0.0008) [2023-12-27 14:47:25,253][20347] Updated weights for policy 0, policy_version 1319334 (0.0007) [2023-12-27 14:47:25,309][20347] Updated weights for policy 0, policy_version 1319344 (0.0008) [2023-12-27 14:47:25,370][20347] Updated weights for policy 0, policy_version 1319354 (0.0006) [2023-12-27 14:47:25,930][20381] Updated weights for policy 1, policy_version 1318449 (0.0007) [2023-12-27 14:47:25,937][20347] Updated weights for policy 0, policy_version 1319364 (0.0005) [2023-12-27 14:47:25,986][20381] Updated weights for policy 1, policy_version 1318459 (0.0006) [2023-12-27 14:47:25,993][20347] Updated weights for policy 0, policy_version 1319374 (0.0005) [2023-12-27 14:47:26,037][20381] Updated weights for policy 1, policy_version 1318469 (0.0005) [2023-12-27 14:47:26,052][20347] Updated weights for policy 0, policy_version 1319384 (0.0007) [2023-12-27 14:47:26,088][20381] Updated weights for policy 1, policy_version 1318479 (0.0005) [2023-12-27 14:47:26,632][20347] Updated weights for policy 0, policy_version 1319394 (0.0007) [2023-12-27 14:47:26,690][20347] Updated weights for policy 0, policy_version 1319404 (0.0010) [2023-12-27 14:47:26,744][20347] Updated weights for policy 0, policy_version 1319414 (0.0010) [2023-12-27 14:47:26,767][20381] Updated weights for policy 1, policy_version 1318489 (0.0007) [2023-12-27 14:47:26,799][20347] Updated weights for policy 0, policy_version 1319424 (0.0010) [2023-12-27 14:47:26,826][20381] Updated weights for policy 1, policy_version 1318499 (0.0006) [2023-12-27 14:47:26,877][20381] Updated weights for policy 1, policy_version 1318509 (0.0007) [2023-12-27 14:47:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 675405824. Throughput: 0: 10395.3, 1: 10137.5. Samples: 675397488. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:27,363][19170] Avg episode reward: [(0, '698.570'), (1, '862.960')] [2023-12-27 14:47:27,465][20347] Updated weights for policy 0, policy_version 1319434 (0.0010) [2023-12-27 14:47:27,515][20347] Updated weights for policy 0, policy_version 1319444 (0.0010) [2023-12-27 14:47:27,570][20347] Updated weights for policy 0, policy_version 1319454 (0.0010) [2023-12-27 14:47:27,572][20381] Updated weights for policy 1, policy_version 1318519 (0.0006) [2023-12-27 14:47:27,621][20381] Updated weights for policy 1, policy_version 1318529 (0.0007) [2023-12-27 14:47:27,674][20381] Updated weights for policy 1, policy_version 1318539 (0.0007) [2023-12-27 14:47:28,211][20347] Updated weights for policy 0, policy_version 1319464 (0.0010) [2023-12-27 14:47:28,264][20347] Updated weights for policy 0, policy_version 1319474 (0.0010) [2023-12-27 14:47:28,320][20347] Updated weights for policy 0, policy_version 1319484 (0.0011) [2023-12-27 14:47:28,375][20381] Updated weights for policy 1, policy_version 1318549 (0.0007) [2023-12-27 14:47:28,435][20381] Updated weights for policy 1, policy_version 1318559 (0.0007) [2023-12-27 14:47:28,491][20381] Updated weights for policy 1, policy_version 1318569 (0.0007) [2023-12-27 14:47:28,935][20347] Updated weights for policy 0, policy_version 1319494 (0.0010) [2023-12-27 14:47:28,992][20347] Updated weights for policy 0, policy_version 1319504 (0.0010) [2023-12-27 14:47:29,043][20347] Updated weights for policy 0, policy_version 1319514 (0.0007) [2023-12-27 14:47:29,199][20381] Updated weights for policy 1, policy_version 1318579 (0.0008) [2023-12-27 14:47:29,261][20381] Updated weights for policy 1, policy_version 1318589 (0.0010) [2023-12-27 14:47:29,318][20381] Updated weights for policy 1, policy_version 1318599 (0.0007) [2023-12-27 14:47:29,755][20347] Updated weights for policy 0, policy_version 1319524 (0.0007) [2023-12-27 14:47:29,814][20347] Updated weights for policy 0, policy_version 1319534 (0.0010) [2023-12-27 14:47:29,876][20347] Updated weights for policy 0, policy_version 1319544 (0.0007) [2023-12-27 14:47:30,035][20381] Updated weights for policy 1, policy_version 1318609 (0.0008) [2023-12-27 14:47:30,097][20381] Updated weights for policy 1, policy_version 1318619 (0.0007) [2023-12-27 14:47:30,148][20381] Updated weights for policy 1, policy_version 1318629 (0.0007) [2023-12-27 14:47:30,211][20381] Updated weights for policy 1, policy_version 1318639 (0.0008) [2023-12-27 14:47:30,581][20347] Updated weights for policy 0, policy_version 1319554 (0.0006) [2023-12-27 14:47:30,632][20347] Updated weights for policy 0, policy_version 1319564 (0.0008) [2023-12-27 14:47:30,681][20347] Updated weights for policy 0, policy_version 1319574 (0.0009) [2023-12-27 14:47:30,741][20347] Updated weights for policy 0, policy_version 1319584 (0.0007) [2023-12-27 14:47:30,923][20381] Updated weights for policy 1, policy_version 1318649 (0.0007) [2023-12-27 14:47:30,978][20381] Updated weights for policy 1, policy_version 1318659 (0.0008) [2023-12-27 14:47:31,043][20381] Updated weights for policy 1, policy_version 1318669 (0.0009) [2023-12-27 14:47:31,478][20347] Updated weights for policy 0, policy_version 1319594 (0.0007) [2023-12-27 14:47:31,533][20347] Updated weights for policy 0, policy_version 1319604 (0.0007) [2023-12-27 14:47:31,592][20347] Updated weights for policy 0, policy_version 1319614 (0.0005) [2023-12-27 14:47:31,831][20381] Updated weights for policy 1, policy_version 1318679 (0.0008) [2023-12-27 14:47:31,888][20381] Updated weights for policy 1, policy_version 1318689 (0.0007) [2023-12-27 14:47:31,948][20381] Updated weights for policy 1, policy_version 1318699 (0.0007) [2023-12-27 14:47:32,292][20347] Updated weights for policy 0, policy_version 1319624 (0.0007) [2023-12-27 14:47:32,351][20347] Updated weights for policy 0, policy_version 1319634 (0.0008) [2023-12-27 14:47:32,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 675504128. Throughput: 0: 10394.8, 1: 10159.8. Samples: 675517792. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:32,363][19170] Avg episode reward: [(0, '698.080'), (1, '863.180')] [2023-12-27 14:47:32,418][20347] Updated weights for policy 0, policy_version 1319644 (0.0007) [2023-12-27 14:47:32,674][20381] Updated weights for policy 1, policy_version 1318709 (0.0008) [2023-12-27 14:47:32,730][20381] Updated weights for policy 1, policy_version 1318719 (0.0008) [2023-12-27 14:47:32,792][20381] Updated weights for policy 1, policy_version 1318729 (0.0008) [2023-12-27 14:47:33,063][20347] Updated weights for policy 0, policy_version 1319654 (0.0008) [2023-12-27 14:47:33,113][20347] Updated weights for policy 0, policy_version 1319664 (0.0007) [2023-12-27 14:47:33,162][20347] Updated weights for policy 0, policy_version 1319674 (0.0008) [2023-12-27 14:47:33,490][20381] Updated weights for policy 1, policy_version 1318739 (0.0008) [2023-12-27 14:47:33,539][20381] Updated weights for policy 1, policy_version 1318749 (0.0010) [2023-12-27 14:47:33,581][20381] Updated weights for policy 1, policy_version 1318759 (0.0008) [2023-12-27 14:47:33,823][20347] Updated weights for policy 0, policy_version 1319684 (0.0007) [2023-12-27 14:47:33,868][20347] Updated weights for policy 0, policy_version 1319694 (0.0007) [2023-12-27 14:47:33,922][20347] Updated weights for policy 0, policy_version 1319704 (0.0007) [2023-12-27 14:47:34,336][20381] Updated weights for policy 1, policy_version 1318769 (0.0007) [2023-12-27 14:47:34,394][20381] Updated weights for policy 1, policy_version 1318779 (0.0008) [2023-12-27 14:47:34,444][20381] Updated weights for policy 1, policy_version 1318789 (0.0008) [2023-12-27 14:47:34,494][20381] Updated weights for policy 1, policy_version 1318799 (0.0008) [2023-12-27 14:47:34,582][20347] Updated weights for policy 0, policy_version 1319714 (0.0007) [2023-12-27 14:47:34,643][20347] Updated weights for policy 0, policy_version 1319724 (0.0007) [2023-12-27 14:47:34,702][20347] Updated weights for policy 0, policy_version 1319734 (0.0008) [2023-12-27 14:47:34,756][20347] Updated weights for policy 0, policy_version 1319744 (0.0005) [2023-12-27 14:47:35,228][20381] Updated weights for policy 1, policy_version 1318809 (0.0006) [2023-12-27 14:47:35,281][20381] Updated weights for policy 1, policy_version 1318819 (0.0005) [2023-12-27 14:47:35,328][20381] Updated weights for policy 1, policy_version 1318829 (0.0005) [2023-12-27 14:47:35,381][20347] Updated weights for policy 0, policy_version 1319754 (0.0008) [2023-12-27 14:47:35,440][20347] Updated weights for policy 0, policy_version 1319764 (0.0007) [2023-12-27 14:47:35,493][20347] Updated weights for policy 0, policy_version 1319774 (0.0007) [2023-12-27 14:47:35,980][20381] Updated weights for policy 1, policy_version 1318839 (0.0007) [2023-12-27 14:47:36,035][20381] Updated weights for policy 1, policy_version 1318849 (0.0009) [2023-12-27 14:47:36,094][20381] Updated weights for policy 1, policy_version 1318859 (0.0009) [2023-12-27 14:47:36,176][20347] Updated weights for policy 0, policy_version 1319784 (0.0007) [2023-12-27 14:47:36,238][20347] Updated weights for policy 0, policy_version 1319794 (0.0008) [2023-12-27 14:47:36,292][20347] Updated weights for policy 0, policy_version 1319804 (0.0007) [2023-12-27 14:47:36,806][20381] Updated weights for policy 1, policy_version 1318869 (0.0010) [2023-12-27 14:47:36,875][20381] Updated weights for policy 1, policy_version 1318879 (0.0011) [2023-12-27 14:47:36,924][20347] Updated weights for policy 0, policy_version 1319814 (0.0006) [2023-12-27 14:47:36,933][20381] Updated weights for policy 1, policy_version 1318889 (0.0009) [2023-12-27 14:47:36,982][20347] Updated weights for policy 0, policy_version 1319824 (0.0006) [2023-12-27 14:47:37,043][20347] Updated weights for policy 0, policy_version 1319834 (0.0007) [2023-12-27 14:47:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 675610624. Throughput: 0: 10451.2, 1: 10125.2. Samples: 675579448. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:37,363][19170] Avg episode reward: [(0, '766.830'), (1, '833.880')] [2023-12-27 14:47:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001318896_337666048.pth... [2023-12-27 14:47:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001319840_337944576.pth... [2023-12-27 14:47:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001317744_337371136.pth [2023-12-27 14:47:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001318624_337633280.pth [2023-12-27 14:47:37,619][20381] Updated weights for policy 1, policy_version 1318899 (0.0011) [2023-12-27 14:47:37,657][20347] Updated weights for policy 0, policy_version 1319844 (0.0008) [2023-12-27 14:47:37,669][20381] Updated weights for policy 1, policy_version 1318909 (0.0011) [2023-12-27 14:47:37,714][20347] Updated weights for policy 0, policy_version 1319854 (0.0011) [2023-12-27 14:47:37,720][20381] Updated weights for policy 1, policy_version 1318919 (0.0010) [2023-12-27 14:47:37,764][20347] Updated weights for policy 0, policy_version 1319864 (0.0010) [2023-12-27 14:47:38,422][20381] Updated weights for policy 1, policy_version 1318929 (0.0011) [2023-12-27 14:47:38,443][20347] Updated weights for policy 0, policy_version 1319874 (0.0010) [2023-12-27 14:47:38,479][20381] Updated weights for policy 1, policy_version 1318939 (0.0011) [2023-12-27 14:47:38,499][20347] Updated weights for policy 0, policy_version 1319884 (0.0010) [2023-12-27 14:47:38,536][20381] Updated weights for policy 1, policy_version 1318949 (0.0011) [2023-12-27 14:47:38,554][20347] Updated weights for policy 0, policy_version 1319894 (0.0010) [2023-12-27 14:47:38,593][20381] Updated weights for policy 1, policy_version 1318959 (0.0010) [2023-12-27 14:47:38,614][20347] Updated weights for policy 0, policy_version 1319904 (0.0010) [2023-12-27 14:47:39,293][20381] Updated weights for policy 1, policy_version 1318969 (0.0011) [2023-12-27 14:47:39,304][20347] Updated weights for policy 0, policy_version 1319914 (0.0010) [2023-12-27 14:47:39,354][20381] Updated weights for policy 1, policy_version 1318979 (0.0010) [2023-12-27 14:47:39,365][20347] Updated weights for policy 0, policy_version 1319924 (0.0010) [2023-12-27 14:47:39,424][20381] Updated weights for policy 1, policy_version 1318989 (0.0010) [2023-12-27 14:47:39,430][20347] Updated weights for policy 0, policy_version 1319934 (0.0010) [2023-12-27 14:47:40,104][20347] Updated weights for policy 0, policy_version 1319944 (0.0011) [2023-12-27 14:47:40,119][20381] Updated weights for policy 1, policy_version 1318999 (0.0010) [2023-12-27 14:47:40,167][20347] Updated weights for policy 0, policy_version 1319954 (0.0010) [2023-12-27 14:47:40,179][20381] Updated weights for policy 1, policy_version 1319009 (0.0011) [2023-12-27 14:47:40,224][20347] Updated weights for policy 0, policy_version 1319964 (0.0011) [2023-12-27 14:47:40,243][20381] Updated weights for policy 1, policy_version 1319019 (0.0011) [2023-12-27 14:47:40,883][20347] Updated weights for policy 0, policy_version 1319974 (0.0009) [2023-12-27 14:47:40,935][20347] Updated weights for policy 0, policy_version 1319984 (0.0010) [2023-12-27 14:47:40,937][20381] Updated weights for policy 1, policy_version 1319029 (0.0009) [2023-12-27 14:47:40,984][20347] Updated weights for policy 0, policy_version 1319994 (0.0007) [2023-12-27 14:47:40,990][20381] Updated weights for policy 1, policy_version 1319039 (0.0010) [2023-12-27 14:47:41,065][20381] Updated weights for policy 1, policy_version 1319050 (0.0009) [2023-12-27 14:47:41,821][20347] Updated weights for policy 0, policy_version 1320004 (0.0007) [2023-12-27 14:47:41,870][20347] Updated weights for policy 0, policy_version 1320014 (0.0008) [2023-12-27 14:47:41,926][20347] Updated weights for policy 0, policy_version 1320024 (0.0007) [2023-12-27 14:47:41,927][20381] Updated weights for policy 1, policy_version 1319060 (0.0009) [2023-12-27 14:47:41,984][20381] Updated weights for policy 1, policy_version 1319070 (0.0008) [2023-12-27 14:47:42,044][20381] Updated weights for policy 1, policy_version 1319080 (0.0008) [2023-12-27 14:47:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 675708928. Throughput: 0: 10394.5, 1: 10069.6. Samples: 675700120. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:42,363][19170] Avg episode reward: [(0, '834.710'), (1, '872.890')] [2023-12-27 14:47:42,668][20347] Updated weights for policy 0, policy_version 1320034 (0.0009) [2023-12-27 14:47:42,693][20381] Updated weights for policy 1, policy_version 1319090 (0.0008) [2023-12-27 14:47:42,725][20347] Updated weights for policy 0, policy_version 1320044 (0.0011) [2023-12-27 14:47:42,746][20381] Updated weights for policy 1, policy_version 1319100 (0.0007) [2023-12-27 14:47:42,781][20347] Updated weights for policy 0, policy_version 1320054 (0.0010) [2023-12-27 14:47:42,796][20381] Updated weights for policy 1, policy_version 1319110 (0.0009) [2023-12-27 14:47:42,830][20347] Updated weights for policy 0, policy_version 1320064 (0.0010) [2023-12-27 14:47:42,855][20381] Updated weights for policy 1, policy_version 1319120 (0.0008) [2023-12-27 14:47:43,474][20381] Updated weights for policy 1, policy_version 1319130 (0.0007) [2023-12-27 14:47:43,502][20347] Updated weights for policy 0, policy_version 1320074 (0.0005) [2023-12-27 14:47:43,536][20381] Updated weights for policy 1, policy_version 1319140 (0.0007) [2023-12-27 14:47:43,565][20347] Updated weights for policy 0, policy_version 1320084 (0.0005) [2023-12-27 14:47:43,592][20381] Updated weights for policy 1, policy_version 1319150 (0.0005) [2023-12-27 14:47:43,620][20347] Updated weights for policy 0, policy_version 1320094 (0.0005) [2023-12-27 14:47:44,189][20347] Updated weights for policy 0, policy_version 1320104 (0.0008) [2023-12-27 14:47:44,231][20381] Updated weights for policy 1, policy_version 1319160 (0.0005) [2023-12-27 14:47:44,242][20347] Updated weights for policy 0, policy_version 1320114 (0.0008) [2023-12-27 14:47:44,281][20381] Updated weights for policy 1, policy_version 1319170 (0.0008) [2023-12-27 14:47:44,297][20347] Updated weights for policy 0, policy_version 1320124 (0.0007) [2023-12-27 14:47:44,339][20381] Updated weights for policy 1, policy_version 1319180 (0.0005) [2023-12-27 14:47:44,960][20381] Updated weights for policy 1, policy_version 1319190 (0.0006) [2023-12-27 14:47:45,025][20381] Updated weights for policy 1, policy_version 1319200 (0.0011) [2023-12-27 14:47:45,033][20347] Updated weights for policy 0, policy_version 1320134 (0.0010) [2023-12-27 14:47:45,084][20381] Updated weights for policy 1, policy_version 1319210 (0.0011) [2023-12-27 14:47:45,090][20347] Updated weights for policy 0, policy_version 1320144 (0.0011) [2023-12-27 14:47:45,149][20347] Updated weights for policy 0, policy_version 1320154 (0.0009) [2023-12-27 14:47:45,664][20381] Updated weights for policy 1, policy_version 1319220 (0.0011) [2023-12-27 14:47:45,726][20381] Updated weights for policy 1, policy_version 1319230 (0.0009) [2023-12-27 14:47:45,786][20381] Updated weights for policy 1, policy_version 1319240 (0.0010) [2023-12-27 14:47:45,821][20347] Updated weights for policy 0, policy_version 1320164 (0.0006) [2023-12-27 14:47:45,872][20347] Updated weights for policy 0, policy_version 1320174 (0.0005) [2023-12-27 14:47:45,919][20347] Updated weights for policy 0, policy_version 1320184 (0.0005) [2023-12-27 14:47:46,473][20381] Updated weights for policy 1, policy_version 1319250 (0.0010) [2023-12-27 14:47:46,534][20381] Updated weights for policy 1, policy_version 1319260 (0.0009) [2023-12-27 14:47:46,571][20347] Updated weights for policy 0, policy_version 1320194 (0.0010) [2023-12-27 14:47:46,591][20381] Updated weights for policy 1, policy_version 1319270 (0.0007) [2023-12-27 14:47:46,626][20347] Updated weights for policy 0, policy_version 1320204 (0.0010) [2023-12-27 14:47:46,647][20381] Updated weights for policy 1, policy_version 1319280 (0.0005) [2023-12-27 14:47:46,681][20347] Updated weights for policy 0, policy_version 1320214 (0.0010) [2023-12-27 14:47:46,735][20347] Updated weights for policy 0, policy_version 1320224 (0.0010) [2023-12-27 14:47:47,245][20381] Updated weights for policy 1, policy_version 1319290 (0.0010) [2023-12-27 14:47:47,300][20381] Updated weights for policy 1, policy_version 1319300 (0.0010) [2023-12-27 14:47:47,352][20381] Updated weights for policy 1, policy_version 1319310 (0.0010) [2023-12-27 14:47:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 675807232. Throughput: 0: 10337.8, 1: 10186.3. Samples: 675824328. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:47,363][19170] Avg episode reward: [(0, '785.980'), (1, '853.320')] [2023-12-27 14:47:47,421][20347] Updated weights for policy 0, policy_version 1320234 (0.0010) [2023-12-27 14:47:47,481][20347] Updated weights for policy 0, policy_version 1320244 (0.0010) [2023-12-27 14:47:47,545][20347] Updated weights for policy 0, policy_version 1320254 (0.0010) [2023-12-27 14:47:47,949][20381] Updated weights for policy 1, policy_version 1319320 (0.0011) [2023-12-27 14:47:48,005][20381] Updated weights for policy 1, policy_version 1319330 (0.0011) [2023-12-27 14:47:48,067][20381] Updated weights for policy 1, policy_version 1319340 (0.0006) [2023-12-27 14:47:48,215][20347] Updated weights for policy 0, policy_version 1320264 (0.0011) [2023-12-27 14:47:48,291][20347] Updated weights for policy 0, policy_version 1320274 (0.0011) [2023-12-27 14:47:48,355][20347] Updated weights for policy 0, policy_version 1320284 (0.0010) [2023-12-27 14:47:48,773][20381] Updated weights for policy 1, policy_version 1319350 (0.0009) [2023-12-27 14:47:48,835][20381] Updated weights for policy 1, policy_version 1319360 (0.0008) [2023-12-27 14:47:48,891][20381] Updated weights for policy 1, policy_version 1319370 (0.0007) [2023-12-27 14:47:49,098][20347] Updated weights for policy 0, policy_version 1320294 (0.0008) [2023-12-27 14:47:49,161][20347] Updated weights for policy 0, policy_version 1320304 (0.0007) [2023-12-27 14:47:49,223][20347] Updated weights for policy 0, policy_version 1320314 (0.0008) [2023-12-27 14:47:49,617][20381] Updated weights for policy 1, policy_version 1319380 (0.0008) [2023-12-27 14:47:49,685][20381] Updated weights for policy 1, policy_version 1319390 (0.0006) [2023-12-27 14:47:49,738][20381] Updated weights for policy 1, policy_version 1319400 (0.0008) [2023-12-27 14:47:49,956][20347] Updated weights for policy 0, policy_version 1320324 (0.0009) [2023-12-27 14:47:50,015][20347] Updated weights for policy 0, policy_version 1320334 (0.0006) [2023-12-27 14:47:50,076][20347] Updated weights for policy 0, policy_version 1320344 (0.0007) [2023-12-27 14:47:50,421][20381] Updated weights for policy 1, policy_version 1319410 (0.0007) [2023-12-27 14:47:50,474][20381] Updated weights for policy 1, policy_version 1319420 (0.0008) [2023-12-27 14:47:50,523][20381] Updated weights for policy 1, policy_version 1319430 (0.0007) [2023-12-27 14:47:50,587][20381] Updated weights for policy 1, policy_version 1319440 (0.0008) [2023-12-27 14:47:50,691][20347] Updated weights for policy 0, policy_version 1320354 (0.0006) [2023-12-27 14:47:50,755][20347] Updated weights for policy 0, policy_version 1320364 (0.0007) [2023-12-27 14:47:50,824][20347] Updated weights for policy 0, policy_version 1320374 (0.0006) [2023-12-27 14:47:50,894][20347] Updated weights for policy 0, policy_version 1320384 (0.0007) [2023-12-27 14:47:51,287][20381] Updated weights for policy 1, policy_version 1319450 (0.0006) [2023-12-27 14:47:51,354][20381] Updated weights for policy 1, policy_version 1319460 (0.0008) [2023-12-27 14:47:51,422][20381] Updated weights for policy 1, policy_version 1319470 (0.0006) [2023-12-27 14:47:51,538][20347] Updated weights for policy 0, policy_version 1320394 (0.0008) [2023-12-27 14:47:51,595][20347] Updated weights for policy 0, policy_version 1320404 (0.0008) [2023-12-27 14:47:51,661][20347] Updated weights for policy 0, policy_version 1320414 (0.0008) [2023-12-27 14:47:52,098][20381] Updated weights for policy 1, policy_version 1319480 (0.0008) [2023-12-27 14:47:52,161][20381] Updated weights for policy 1, policy_version 1319490 (0.0010) [2023-12-27 14:47:52,224][20381] Updated weights for policy 1, policy_version 1319500 (0.0010) [2023-12-27 14:47:52,324][20347] Updated weights for policy 0, policy_version 1320424 (0.0008) [2023-12-27 14:47:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 675913728. Throughput: 0: 10316.3, 1: 10106.0. Samples: 675884000. Policy #0 lag: (min: 10.0, avg: 20.7, max: 42.0) [2023-12-27 14:47:52,363][19170] Avg episode reward: [(0, '786.320'), (1, '833.700')] [2023-12-27 14:47:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001319504_337821696.pth... [2023-12-27 14:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001318320_337518592.pth [2023-12-27 14:47:52,386][20347] Updated weights for policy 0, policy_version 1320434 (0.0008) [2023-12-27 14:47:52,447][20347] Updated weights for policy 0, policy_version 1320444 (0.0008) [2023-12-27 14:47:52,470][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001320448_338100224.pth... [2023-12-27 14:47:52,474][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001319232_337788928.pth [2023-12-27 14:47:52,791][20381] Updated weights for policy 1, policy_version 1319510 (0.0009) [2023-12-27 14:47:52,850][20381] Updated weights for policy 1, policy_version 1319520 (0.0008) [2023-12-27 14:47:52,902][20381] Updated weights for policy 1, policy_version 1319530 (0.0007) [2023-12-27 14:47:53,185][20347] Updated weights for policy 0, policy_version 1320454 (0.0007) [2023-12-27 14:47:53,243][20347] Updated weights for policy 0, policy_version 1320464 (0.0007) [2023-12-27 14:47:53,303][20347] Updated weights for policy 0, policy_version 1320474 (0.0008) [2023-12-27 14:47:53,543][20381] Updated weights for policy 1, policy_version 1319540 (0.0008) [2023-12-27 14:47:53,596][20381] Updated weights for policy 1, policy_version 1319550 (0.0008) [2023-12-27 14:47:53,648][20381] Updated weights for policy 1, policy_version 1319560 (0.0009) [2023-12-27 14:47:53,956][20347] Updated weights for policy 0, policy_version 1320484 (0.0008) [2023-12-27 14:47:54,014][20347] Updated weights for policy 0, policy_version 1320494 (0.0008) [2023-12-27 14:47:54,070][20347] Updated weights for policy 0, policy_version 1320504 (0.0006) [2023-12-27 14:47:54,346][20381] Updated weights for policy 1, policy_version 1319570 (0.0008) [2023-12-27 14:47:54,399][20381] Updated weights for policy 1, policy_version 1319580 (0.0005) [2023-12-27 14:47:54,455][20381] Updated weights for policy 1, policy_version 1319590 (0.0006) [2023-12-27 14:47:54,506][20381] Updated weights for policy 1, policy_version 1319600 (0.0006) [2023-12-27 14:47:54,719][20347] Updated weights for policy 0, policy_version 1320514 (0.0006) [2023-12-27 14:47:54,777][20347] Updated weights for policy 0, policy_version 1320524 (0.0007) [2023-12-27 14:47:54,822][20347] Updated weights for policy 0, policy_version 1320534 (0.0007) [2023-12-27 14:47:54,879][20347] Updated weights for policy 0, policy_version 1320544 (0.0006) [2023-12-27 14:47:55,151][20381] Updated weights for policy 1, policy_version 1319610 (0.0008) [2023-12-27 14:47:55,195][20381] Updated weights for policy 1, policy_version 1319620 (0.0008) [2023-12-27 14:47:55,251][20381] Updated weights for policy 1, policy_version 1319630 (0.0008) [2023-12-27 14:47:55,579][20347] Updated weights for policy 0, policy_version 1320554 (0.0007) [2023-12-27 14:47:55,633][20347] Updated weights for policy 0, policy_version 1320564 (0.0005) [2023-12-27 14:47:55,692][20347] Updated weights for policy 0, policy_version 1320574 (0.0005) [2023-12-27 14:47:55,976][20381] Updated weights for policy 1, policy_version 1319640 (0.0008) [2023-12-27 14:47:56,028][20381] Updated weights for policy 1, policy_version 1319650 (0.0005) [2023-12-27 14:47:56,084][20381] Updated weights for policy 1, policy_version 1319660 (0.0005) [2023-12-27 14:47:56,341][20347] Updated weights for policy 0, policy_version 1320584 (0.0007) [2023-12-27 14:47:56,389][20347] Updated weights for policy 0, policy_version 1320594 (0.0008) [2023-12-27 14:47:56,440][20347] Updated weights for policy 0, policy_version 1320604 (0.0008) [2023-12-27 14:47:56,646][20381] Updated weights for policy 1, policy_version 1319670 (0.0009) [2023-12-27 14:47:56,700][20381] Updated weights for policy 1, policy_version 1319680 (0.0010) [2023-12-27 14:47:56,751][20381] Updated weights for policy 1, policy_version 1319690 (0.0010) [2023-12-27 14:47:57,114][20347] Updated weights for policy 0, policy_version 1320614 (0.0010) [2023-12-27 14:47:57,159][20347] Updated weights for policy 0, policy_version 1320624 (0.0010) [2023-12-27 14:47:57,207][20347] Updated weights for policy 0, policy_version 1320634 (0.0010) [2023-12-27 14:47:57,350][20381] Updated weights for policy 1, policy_version 1319700 (0.0009) [2023-12-27 14:47:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 676020224. Throughput: 0: 10286.2, 1: 10117.2. Samples: 676008424. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:47:57,363][19170] Avg episode reward: [(0, '787.100'), (1, '892.550')] [2023-12-27 14:47:57,398][20381] Updated weights for policy 1, policy_version 1319710 (0.0010) [2023-12-27 14:47:57,446][20381] Updated weights for policy 1, policy_version 1319720 (0.0010) [2023-12-27 14:47:57,873][20347] Updated weights for policy 0, policy_version 1320644 (0.0008) [2023-12-27 14:47:57,922][20347] Updated weights for policy 0, policy_version 1320654 (0.0007) [2023-12-27 14:47:57,963][20347] Updated weights for policy 0, policy_version 1320664 (0.0005) [2023-12-27 14:47:58,091][20381] Updated weights for policy 1, policy_version 1319730 (0.0009) [2023-12-27 14:47:58,150][20381] Updated weights for policy 1, policy_version 1319740 (0.0008) [2023-12-27 14:47:58,214][20381] Updated weights for policy 1, policy_version 1319750 (0.0010) [2023-12-27 14:47:58,273][20381] Updated weights for policy 1, policy_version 1319760 (0.0011) [2023-12-27 14:47:58,585][20347] Updated weights for policy 0, policy_version 1320674 (0.0005) [2023-12-27 14:47:58,649][20347] Updated weights for policy 0, policy_version 1320684 (0.0008) [2023-12-27 14:47:58,713][20347] Updated weights for policy 0, policy_version 1320694 (0.0007) [2023-12-27 14:47:58,777][20347] Updated weights for policy 0, policy_version 1320704 (0.0008) [2023-12-27 14:47:59,008][20381] Updated weights for policy 1, policy_version 1319770 (0.0006) [2023-12-27 14:47:59,070][20381] Updated weights for policy 1, policy_version 1319780 (0.0007) [2023-12-27 14:47:59,127][20381] Updated weights for policy 1, policy_version 1319790 (0.0007) [2023-12-27 14:47:59,569][20347] Updated weights for policy 0, policy_version 1320714 (0.0005) [2023-12-27 14:47:59,621][20347] Updated weights for policy 0, policy_version 1320724 (0.0008) [2023-12-27 14:47:59,673][20347] Updated weights for policy 0, policy_version 1320734 (0.0007) [2023-12-27 14:47:59,837][20381] Updated weights for policy 1, policy_version 1319800 (0.0010) [2023-12-27 14:47:59,901][20381] Updated weights for policy 1, policy_version 1319810 (0.0010) [2023-12-27 14:47:59,970][20381] Updated weights for policy 1, policy_version 1319820 (0.0008) [2023-12-27 14:48:00,364][20347] Updated weights for policy 0, policy_version 1320744 (0.0007) [2023-12-27 14:48:00,425][20347] Updated weights for policy 0, policy_version 1320754 (0.0007) [2023-12-27 14:48:00,483][20347] Updated weights for policy 0, policy_version 1320764 (0.0007) [2023-12-27 14:48:00,613][20381] Updated weights for policy 1, policy_version 1319830 (0.0007) [2023-12-27 14:48:00,677][20381] Updated weights for policy 1, policy_version 1319840 (0.0007) [2023-12-27 14:48:00,730][20381] Updated weights for policy 1, policy_version 1319850 (0.0006) [2023-12-27 14:48:01,175][20347] Updated weights for policy 0, policy_version 1320774 (0.0007) [2023-12-27 14:48:01,228][20347] Updated weights for policy 0, policy_version 1320784 (0.0008) [2023-12-27 14:48:01,286][20347] Updated weights for policy 0, policy_version 1320794 (0.0008) [2023-12-27 14:48:01,428][20381] Updated weights for policy 1, policy_version 1319860 (0.0006) [2023-12-27 14:48:01,496][20381] Updated weights for policy 1, policy_version 1319870 (0.0008) [2023-12-27 14:48:01,550][20381] Updated weights for policy 1, policy_version 1319880 (0.0008) [2023-12-27 14:48:01,999][20347] Updated weights for policy 0, policy_version 1320804 (0.0007) [2023-12-27 14:48:02,055][20347] Updated weights for policy 0, policy_version 1320814 (0.0008) [2023-12-27 14:48:02,108][20347] Updated weights for policy 0, policy_version 1320824 (0.0008) [2023-12-27 14:48:02,296][20381] Updated weights for policy 1, policy_version 1319890 (0.0006) [2023-12-27 14:48:02,361][20381] Updated weights for policy 1, policy_version 1319900 (0.0009) [2023-12-27 14:48:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 676118528. Throughput: 0: 10217.6, 1: 10124.7. Samples: 676130776. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:02,363][19170] Avg episode reward: [(0, '777.270'), (1, '902.340')] [2023-12-27 14:48:02,416][20381] Updated weights for policy 1, policy_version 1319910 (0.0007) [2023-12-27 14:48:02,472][20381] Updated weights for policy 1, policy_version 1319920 (0.0006) [2023-12-27 14:48:02,814][20347] Updated weights for policy 0, policy_version 1320834 (0.0007) [2023-12-27 14:48:02,881][20347] Updated weights for policy 0, policy_version 1320844 (0.0007) [2023-12-27 14:48:02,938][20347] Updated weights for policy 0, policy_version 1320854 (0.0009) [2023-12-27 14:48:02,987][20347] Updated weights for policy 0, policy_version 1320864 (0.0010) [2023-12-27 14:48:03,124][20381] Updated weights for policy 1, policy_version 1319930 (0.0007) [2023-12-27 14:48:03,178][20381] Updated weights for policy 1, policy_version 1319940 (0.0005) [2023-12-27 14:48:03,238][20381] Updated weights for policy 1, policy_version 1319950 (0.0007) [2023-12-27 14:48:03,619][20347] Updated weights for policy 0, policy_version 1320874 (0.0008) [2023-12-27 14:48:03,672][20347] Updated weights for policy 0, policy_version 1320884 (0.0007) [2023-12-27 14:48:03,718][20347] Updated weights for policy 0, policy_version 1320894 (0.0007) [2023-12-27 14:48:03,907][20381] Updated weights for policy 1, policy_version 1319960 (0.0007) [2023-12-27 14:48:03,963][20381] Updated weights for policy 1, policy_version 1319970 (0.0007) [2023-12-27 14:48:04,012][20381] Updated weights for policy 1, policy_version 1319980 (0.0005) [2023-12-27 14:48:04,398][20347] Updated weights for policy 0, policy_version 1320904 (0.0010) [2023-12-27 14:48:04,446][20347] Updated weights for policy 0, policy_version 1320914 (0.0009) [2023-12-27 14:48:04,504][20347] Updated weights for policy 0, policy_version 1320924 (0.0006) [2023-12-27 14:48:04,706][20381] Updated weights for policy 1, policy_version 1319990 (0.0007) [2023-12-27 14:48:04,766][20381] Updated weights for policy 1, policy_version 1320000 (0.0007) [2023-12-27 14:48:04,821][20381] Updated weights for policy 1, policy_version 1320010 (0.0007) [2023-12-27 14:48:05,171][20347] Updated weights for policy 0, policy_version 1320934 (0.0007) [2023-12-27 14:48:05,222][20347] Updated weights for policy 0, policy_version 1320944 (0.0010) [2023-12-27 14:48:05,279][20347] Updated weights for policy 0, policy_version 1320954 (0.0008) [2023-12-27 14:48:05,506][20381] Updated weights for policy 1, policy_version 1320020 (0.0007) [2023-12-27 14:48:05,558][20381] Updated weights for policy 1, policy_version 1320030 (0.0009) [2023-12-27 14:48:05,614][20381] Updated weights for policy 1, policy_version 1320040 (0.0009) [2023-12-27 14:48:05,872][20347] Updated weights for policy 0, policy_version 1320964 (0.0008) [2023-12-27 14:48:05,933][20347] Updated weights for policy 0, policy_version 1320974 (0.0008) [2023-12-27 14:48:05,994][20347] Updated weights for policy 0, policy_version 1320984 (0.0007) [2023-12-27 14:48:06,349][20381] Updated weights for policy 1, policy_version 1320050 (0.0008) [2023-12-27 14:48:06,413][20381] Updated weights for policy 1, policy_version 1320060 (0.0007) [2023-12-27 14:48:06,470][20381] Updated weights for policy 1, policy_version 1320070 (0.0008) [2023-12-27 14:48:06,528][20381] Updated weights for policy 1, policy_version 1320080 (0.0008) [2023-12-27 14:48:06,653][20347] Updated weights for policy 0, policy_version 1320994 (0.0005) [2023-12-27 14:48:06,700][20347] Updated weights for policy 0, policy_version 1321004 (0.0005) [2023-12-27 14:48:06,745][20347] Updated weights for policy 0, policy_version 1321014 (0.0005) [2023-12-27 14:48:06,778][20291] KL-divergence is very high: 222.9203 [2023-12-27 14:48:06,792][20347] Updated weights for policy 0, policy_version 1321024 (0.0005) [2023-12-27 14:48:07,235][20381] Updated weights for policy 1, policy_version 1320090 (0.0011) [2023-12-27 14:48:07,291][20381] Updated weights for policy 1, policy_version 1320100 (0.0011) [2023-12-27 14:48:07,351][20381] Updated weights for policy 1, policy_version 1320110 (0.0010) [2023-12-27 14:48:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 676225024. Throughput: 0: 10234.6, 1: 10145.5. Samples: 676192304. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:07,363][19170] Avg episode reward: [(0, '776.870'), (1, '921.850')] [2023-12-27 14:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001320112_337977344.pth... [2023-12-27 14:48:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001318896_337666048.pth [2023-12-27 14:48:07,383][20291] KL-divergence is very high: 272.0741 [2023-12-27 14:48:07,410][20347] Updated weights for policy 0, policy_version 1321034 (0.0005) [2023-12-27 14:48:07,427][20291] KL-divergence is very high: 223.3734 [2023-12-27 14:48:07,476][20347] Updated weights for policy 0, policy_version 1321044 (0.0005) [2023-12-27 14:48:07,478][20291] KL-divergence is very high: 262.9034 [2023-12-27 14:48:07,524][20291] KL-divergence is very high: 146.9750 [2023-12-27 14:48:07,534][20347] Updated weights for policy 0, policy_version 1321054 (0.0005) [2023-12-27 14:48:07,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001321056_338255872.pth... [2023-12-27 14:48:07,548][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001319840_337944576.pth [2023-12-27 14:48:07,979][20381] Updated weights for policy 1, policy_version 1320120 (0.0006) [2023-12-27 14:48:08,041][20381] Updated weights for policy 1, policy_version 1320130 (0.0005) [2023-12-27 14:48:08,096][20381] Updated weights for policy 1, policy_version 1320140 (0.0005) [2023-12-27 14:48:08,173][20347] Updated weights for policy 0, policy_version 1321064 (0.0010) [2023-12-27 14:48:08,224][20347] Updated weights for policy 0, policy_version 1321074 (0.0010) [2023-12-27 14:48:08,263][20291] KL-divergence is very high: 104.5139 [2023-12-27 14:48:08,279][20347] Updated weights for policy 0, policy_version 1321084 (0.0010) [2023-12-27 14:48:08,285][20291] KL-divergence is very high: 123.8687 [2023-12-27 14:48:08,603][20381] Updated weights for policy 1, policy_version 1320150 (0.0008) [2023-12-27 14:48:08,663][20381] Updated weights for policy 1, policy_version 1320160 (0.0011) [2023-12-27 14:48:08,720][20381] Updated weights for policy 1, policy_version 1320170 (0.0009) [2023-12-27 14:48:08,927][20347] Updated weights for policy 0, policy_version 1321094 (0.0009) [2023-12-27 14:48:08,983][20347] Updated weights for policy 0, policy_version 1321104 (0.0007) [2023-12-27 14:48:09,039][20347] Updated weights for policy 0, policy_version 1321114 (0.0005) [2023-12-27 14:48:09,456][20381] Updated weights for policy 1, policy_version 1320180 (0.0009) [2023-12-27 14:48:09,513][20381] Updated weights for policy 1, policy_version 1320190 (0.0011) [2023-12-27 14:48:09,576][20381] Updated weights for policy 1, policy_version 1320200 (0.0011) [2023-12-27 14:48:09,690][20347] Updated weights for policy 0, policy_version 1321124 (0.0007) [2023-12-27 14:48:09,751][20347] Updated weights for policy 0, policy_version 1321134 (0.0008) [2023-12-27 14:48:09,815][20347] Updated weights for policy 0, policy_version 1321144 (0.0008) [2023-12-27 14:48:10,217][20381] Updated weights for policy 1, policy_version 1320210 (0.0011) [2023-12-27 14:48:10,277][20381] Updated weights for policy 1, policy_version 1320220 (0.0011) [2023-12-27 14:48:10,340][20381] Updated weights for policy 1, policy_version 1320230 (0.0011) [2023-12-27 14:48:10,403][20381] Updated weights for policy 1, policy_version 1320240 (0.0011) [2023-12-27 14:48:10,518][20347] Updated weights for policy 0, policy_version 1321154 (0.0007) [2023-12-27 14:48:10,579][20347] Updated weights for policy 0, policy_version 1321164 (0.0008) [2023-12-27 14:48:10,640][20347] Updated weights for policy 0, policy_version 1321174 (0.0007) [2023-12-27 14:48:10,700][20347] Updated weights for policy 0, policy_version 1321184 (0.0008) [2023-12-27 14:48:11,051][20381] Updated weights for policy 1, policy_version 1320250 (0.0008) [2023-12-27 14:48:11,114][20381] Updated weights for policy 1, policy_version 1320260 (0.0009) [2023-12-27 14:48:11,183][20381] Updated weights for policy 1, policy_version 1320270 (0.0008) [2023-12-27 14:48:11,411][20347] Updated weights for policy 0, policy_version 1321194 (0.0007) [2023-12-27 14:48:11,478][20347] Updated weights for policy 0, policy_version 1321204 (0.0007) [2023-12-27 14:48:11,536][20347] Updated weights for policy 0, policy_version 1321214 (0.0008) [2023-12-27 14:48:11,929][20381] Updated weights for policy 1, policy_version 1320280 (0.0008) [2023-12-27 14:48:11,981][20381] Updated weights for policy 1, policy_version 1320290 (0.0007) [2023-12-27 14:48:12,029][20381] Updated weights for policy 1, policy_version 1320300 (0.0008) [2023-12-27 14:48:12,217][20347] Updated weights for policy 0, policy_version 1321224 (0.0007) [2023-12-27 14:48:12,277][20347] Updated weights for policy 0, policy_version 1321234 (0.0008) [2023-12-27 14:48:12,335][20347] Updated weights for policy 0, policy_version 1321244 (0.0008) [2023-12-27 14:48:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 676331520. Throughput: 0: 10282.5, 1: 10160.5. Samples: 676317420. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:12,363][19170] Avg episode reward: [(0, '767.190'), (1, '921.790')] [2023-12-27 14:48:12,777][20381] Updated weights for policy 1, policy_version 1320310 (0.0009) [2023-12-27 14:48:12,825][20381] Updated weights for policy 1, policy_version 1320320 (0.0008) [2023-12-27 14:48:12,882][20381] Updated weights for policy 1, policy_version 1320330 (0.0009) [2023-12-27 14:48:13,005][20347] Updated weights for policy 0, policy_version 1321254 (0.0007) [2023-12-27 14:48:13,074][20347] Updated weights for policy 0, policy_version 1321264 (0.0009) [2023-12-27 14:48:13,141][20347] Updated weights for policy 0, policy_version 1321274 (0.0009) [2023-12-27 14:48:13,473][20381] Updated weights for policy 1, policy_version 1320340 (0.0007) [2023-12-27 14:48:13,532][20381] Updated weights for policy 1, policy_version 1320350 (0.0006) [2023-12-27 14:48:13,584][20381] Updated weights for policy 1, policy_version 1320360 (0.0006) [2023-12-27 14:48:13,842][20347] Updated weights for policy 0, policy_version 1321284 (0.0007) [2023-12-27 14:48:13,900][20347] Updated weights for policy 0, policy_version 1321294 (0.0007) [2023-12-27 14:48:13,958][20347] Updated weights for policy 0, policy_version 1321304 (0.0008) [2023-12-27 14:48:14,268][20381] Updated weights for policy 1, policy_version 1320370 (0.0008) [2023-12-27 14:48:14,330][20381] Updated weights for policy 1, policy_version 1320380 (0.0008) [2023-12-27 14:48:14,391][20381] Updated weights for policy 1, policy_version 1320390 (0.0006) [2023-12-27 14:48:14,439][20381] Updated weights for policy 1, policy_version 1320400 (0.0005) [2023-12-27 14:48:14,643][20347] Updated weights for policy 0, policy_version 1321314 (0.0008) [2023-12-27 14:48:14,694][20347] Updated weights for policy 0, policy_version 1321324 (0.0008) [2023-12-27 14:48:14,749][20347] Updated weights for policy 0, policy_version 1321334 (0.0007) [2023-12-27 14:48:14,810][20347] Updated weights for policy 0, policy_version 1321344 (0.0008) [2023-12-27 14:48:15,101][20381] Updated weights for policy 1, policy_version 1320410 (0.0009) [2023-12-27 14:48:15,160][20381] Updated weights for policy 1, policy_version 1320420 (0.0005) [2023-12-27 14:48:15,214][20381] Updated weights for policy 1, policy_version 1320430 (0.0006) [2023-12-27 14:48:15,531][20347] Updated weights for policy 0, policy_version 1321354 (0.0007) [2023-12-27 14:48:15,585][20347] Updated weights for policy 0, policy_version 1321364 (0.0007) [2023-12-27 14:48:15,637][20347] Updated weights for policy 0, policy_version 1321374 (0.0007) [2023-12-27 14:48:15,822][20381] Updated weights for policy 1, policy_version 1320440 (0.0008) [2023-12-27 14:48:15,868][20381] Updated weights for policy 1, policy_version 1320450 (0.0007) [2023-12-27 14:48:15,915][20381] Updated weights for policy 1, policy_version 1320460 (0.0005) [2023-12-27 14:48:16,343][20347] Updated weights for policy 0, policy_version 1321384 (0.0007) [2023-12-27 14:48:16,395][20347] Updated weights for policy 0, policy_version 1321394 (0.0006) [2023-12-27 14:48:16,447][20347] Updated weights for policy 0, policy_version 1321404 (0.0005) [2023-12-27 14:48:16,589][20381] Updated weights for policy 1, policy_version 1320470 (0.0010) [2023-12-27 14:48:16,642][20381] Updated weights for policy 1, policy_version 1320480 (0.0011) [2023-12-27 14:48:16,695][20381] Updated weights for policy 1, policy_version 1320490 (0.0010) [2023-12-27 14:48:17,100][20347] Updated weights for policy 0, policy_version 1321414 (0.0006) [2023-12-27 14:48:17,158][20347] Updated weights for policy 0, policy_version 1321424 (0.0007) [2023-12-27 14:48:17,216][20347] Updated weights for policy 0, policy_version 1321434 (0.0008) [2023-12-27 14:48:17,309][20381] Updated weights for policy 1, policy_version 1320500 (0.0009) [2023-12-27 14:48:17,355][20381] Updated weights for policy 1, policy_version 1320510 (0.0005) [2023-12-27 14:48:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 676429824. Throughput: 0: 10228.8, 1: 10271.2. Samples: 676440292. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:17,363][19170] Avg episode reward: [(0, '747.020'), (1, '873.120')] [2023-12-27 14:48:17,400][20381] Updated weights for policy 1, policy_version 1320520 (0.0006) [2023-12-27 14:48:17,867][20347] Updated weights for policy 0, policy_version 1321444 (0.0007) [2023-12-27 14:48:17,920][20347] Updated weights for policy 0, policy_version 1321454 (0.0005) [2023-12-27 14:48:17,972][20347] Updated weights for policy 0, policy_version 1321464 (0.0005) [2023-12-27 14:48:18,071][20381] Updated weights for policy 1, policy_version 1320530 (0.0006) [2023-12-27 14:48:18,122][20381] Updated weights for policy 1, policy_version 1320540 (0.0008) [2023-12-27 14:48:18,172][20381] Updated weights for policy 1, policy_version 1320550 (0.0007) [2023-12-27 14:48:18,225][20381] Updated weights for policy 1, policy_version 1320560 (0.0006) [2023-12-27 14:48:18,614][20347] Updated weights for policy 0, policy_version 1321474 (0.0007) [2023-12-27 14:48:18,686][20347] Updated weights for policy 0, policy_version 1321484 (0.0009) [2023-12-27 14:48:18,744][20347] Updated weights for policy 0, policy_version 1321494 (0.0010) [2023-12-27 14:48:18,806][20347] Updated weights for policy 0, policy_version 1321504 (0.0005) [2023-12-27 14:48:18,947][20381] Updated weights for policy 1, policy_version 1320570 (0.0007) [2023-12-27 14:48:18,996][20381] Updated weights for policy 1, policy_version 1320580 (0.0007) [2023-12-27 14:48:19,056][20381] Updated weights for policy 1, policy_version 1320590 (0.0008) [2023-12-27 14:48:19,459][20347] Updated weights for policy 0, policy_version 1321514 (0.0010) [2023-12-27 14:48:19,527][20347] Updated weights for policy 0, policy_version 1321524 (0.0010) [2023-12-27 14:48:19,586][20347] Updated weights for policy 0, policy_version 1321534 (0.0009) [2023-12-27 14:48:19,884][20381] Updated weights for policy 1, policy_version 1320600 (0.0007) [2023-12-27 14:48:19,954][20381] Updated weights for policy 1, policy_version 1320610 (0.0008) [2023-12-27 14:48:20,014][20381] Updated weights for policy 1, policy_version 1320620 (0.0009) [2023-12-27 14:48:20,299][20347] Updated weights for policy 0, policy_version 1321544 (0.0006) [2023-12-27 14:48:20,361][20347] Updated weights for policy 0, policy_version 1321554 (0.0008) [2023-12-27 14:48:20,425][20347] Updated weights for policy 0, policy_version 1321564 (0.0007) [2023-12-27 14:48:20,734][20381] Updated weights for policy 1, policy_version 1320630 (0.0008) [2023-12-27 14:48:20,800][20381] Updated weights for policy 1, policy_version 1320640 (0.0009) [2023-12-27 14:48:20,865][20381] Updated weights for policy 1, policy_version 1320650 (0.0008) [2023-12-27 14:48:21,057][20347] Updated weights for policy 0, policy_version 1321574 (0.0008) [2023-12-27 14:48:21,110][20347] Updated weights for policy 0, policy_version 1321584 (0.0008) [2023-12-27 14:48:21,171][20347] Updated weights for policy 0, policy_version 1321594 (0.0007) [2023-12-27 14:48:21,587][20381] Updated weights for policy 1, policy_version 1320660 (0.0007) [2023-12-27 14:48:21,649][20381] Updated weights for policy 1, policy_version 1320670 (0.0006) [2023-12-27 14:48:21,711][20381] Updated weights for policy 1, policy_version 1320680 (0.0008) [2023-12-27 14:48:21,909][20347] Updated weights for policy 0, policy_version 1321604 (0.0007) [2023-12-27 14:48:21,956][20347] Updated weights for policy 0, policy_version 1321614 (0.0009) [2023-12-27 14:48:22,010][20347] Updated weights for policy 0, policy_version 1321624 (0.0009) [2023-12-27 14:48:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 676528128. Throughput: 0: 10198.4, 1: 10260.4. Samples: 676500096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:22,363][19170] Avg episode reward: [(0, '785.670'), (1, '834.020')] [2023-12-27 14:48:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001320688_338124800.pth... [2023-12-27 14:48:22,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001321632_338403328.pth... [2023-12-27 14:48:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001319504_337821696.pth [2023-12-27 14:48:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001320448_338100224.pth [2023-12-27 14:48:22,407][20381] Updated weights for policy 1, policy_version 1320690 (0.0008) [2023-12-27 14:48:22,464][20381] Updated weights for policy 1, policy_version 1320700 (0.0007) [2023-12-27 14:48:22,520][20381] Updated weights for policy 1, policy_version 1320710 (0.0008) [2023-12-27 14:48:22,579][20381] Updated weights for policy 1, policy_version 1320720 (0.0008) [2023-12-27 14:48:22,780][20347] Updated weights for policy 0, policy_version 1321634 (0.0010) [2023-12-27 14:48:22,837][20347] Updated weights for policy 0, policy_version 1321644 (0.0007) [2023-12-27 14:48:22,893][20347] Updated weights for policy 0, policy_version 1321654 (0.0008) [2023-12-27 14:48:22,894][20291] KL-divergence is very high: 145.0808 [2023-12-27 14:48:22,908][20291] KL-divergence is very high: 106.1135 [2023-12-27 14:48:22,932][20291] KL-divergence is very high: 125.5244 [2023-12-27 14:48:22,943][20347] Updated weights for policy 0, policy_version 1321664 (0.0008) [2023-12-27 14:48:23,289][20381] Updated weights for policy 1, policy_version 1320730 (0.0008) [2023-12-27 14:48:23,346][20381] Updated weights for policy 1, policy_version 1320740 (0.0008) [2023-12-27 14:48:23,403][20381] Updated weights for policy 1, policy_version 1320750 (0.0007) [2023-12-27 14:48:23,605][20347] Updated weights for policy 0, policy_version 1321674 (0.0007) [2023-12-27 14:48:23,662][20347] Updated weights for policy 0, policy_version 1321684 (0.0008) [2023-12-27 14:48:23,717][20347] Updated weights for policy 0, policy_version 1321694 (0.0007) [2023-12-27 14:48:24,079][20381] Updated weights for policy 1, policy_version 1320760 (0.0008) [2023-12-27 14:48:24,125][20381] Updated weights for policy 1, policy_version 1320770 (0.0007) [2023-12-27 14:48:24,176][20381] Updated weights for policy 1, policy_version 1320780 (0.0007) [2023-12-27 14:48:24,374][20347] Updated weights for policy 0, policy_version 1321704 (0.0007) [2023-12-27 14:48:24,430][20347] Updated weights for policy 0, policy_version 1321714 (0.0007) [2023-12-27 14:48:24,488][20347] Updated weights for policy 0, policy_version 1321724 (0.0008) [2023-12-27 14:48:24,888][20381] Updated weights for policy 1, policy_version 1320790 (0.0008) [2023-12-27 14:48:24,933][20381] Updated weights for policy 1, policy_version 1320800 (0.0007) [2023-12-27 14:48:24,978][20381] Updated weights for policy 1, policy_version 1320810 (0.0007) [2023-12-27 14:48:25,179][20347] Updated weights for policy 0, policy_version 1321734 (0.0008) [2023-12-27 14:48:25,228][20347] Updated weights for policy 0, policy_version 1321744 (0.0009) [2023-12-27 14:48:25,283][20347] Updated weights for policy 0, policy_version 1321754 (0.0006) [2023-12-27 14:48:25,662][20381] Updated weights for policy 1, policy_version 1320820 (0.0008) [2023-12-27 14:48:25,720][20381] Updated weights for policy 1, policy_version 1320830 (0.0008) [2023-12-27 14:48:25,767][20381] Updated weights for policy 1, policy_version 1320840 (0.0007) [2023-12-27 14:48:25,886][20347] Updated weights for policy 0, policy_version 1321764 (0.0006) [2023-12-27 14:48:25,930][20347] Updated weights for policy 0, policy_version 1321774 (0.0007) [2023-12-27 14:48:25,982][20347] Updated weights for policy 0, policy_version 1321784 (0.0007) [2023-12-27 14:48:26,338][20381] Updated weights for policy 1, policy_version 1320850 (0.0007) [2023-12-27 14:48:26,402][20381] Updated weights for policy 1, policy_version 1320860 (0.0005) [2023-12-27 14:48:26,462][20381] Updated weights for policy 1, policy_version 1320870 (0.0007) [2023-12-27 14:48:26,520][20381] Updated weights for policy 1, policy_version 1320880 (0.0007) [2023-12-27 14:48:26,697][20347] Updated weights for policy 0, policy_version 1321794 (0.0008) [2023-12-27 14:48:26,752][20347] Updated weights for policy 0, policy_version 1321804 (0.0007) [2023-12-27 14:48:26,806][20347] Updated weights for policy 0, policy_version 1321814 (0.0007) [2023-12-27 14:48:26,852][20347] Updated weights for policy 0, policy_version 1321824 (0.0007) [2023-12-27 14:48:27,133][20381] Updated weights for policy 1, policy_version 1320890 (0.0005) [2023-12-27 14:48:27,183][20381] Updated weights for policy 1, policy_version 1320900 (0.0005) [2023-12-27 14:48:27,228][20381] Updated weights for policy 1, policy_version 1320910 (0.0006) [2023-12-27 14:48:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 676634624. Throughput: 0: 10186.6, 1: 10289.5. Samples: 676621544. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:27,363][19170] Avg episode reward: [(0, '804.980'), (1, '843.610')] [2023-12-27 14:48:27,546][20347] Updated weights for policy 0, policy_version 1321834 (0.0005) [2023-12-27 14:48:27,591][20347] Updated weights for policy 0, policy_version 1321844 (0.0005) [2023-12-27 14:48:27,636][20347] Updated weights for policy 0, policy_version 1321854 (0.0005) [2023-12-27 14:48:27,902][20381] Updated weights for policy 1, policy_version 1320920 (0.0006) [2023-12-27 14:48:27,961][20381] Updated weights for policy 1, policy_version 1320930 (0.0005) [2023-12-27 14:48:28,011][20381] Updated weights for policy 1, policy_version 1320940 (0.0005) [2023-12-27 14:48:28,204][20347] Updated weights for policy 0, policy_version 1321864 (0.0005) [2023-12-27 14:48:28,250][20347] Updated weights for policy 0, policy_version 1321874 (0.0005) [2023-12-27 14:48:28,293][20347] Updated weights for policy 0, policy_version 1321884 (0.0005) [2023-12-27 14:48:28,531][20381] Updated weights for policy 1, policy_version 1320950 (0.0006) [2023-12-27 14:48:28,592][20381] Updated weights for policy 1, policy_version 1320960 (0.0005) [2023-12-27 14:48:28,652][20381] Updated weights for policy 1, policy_version 1320970 (0.0008) [2023-12-27 14:48:28,935][20347] Updated weights for policy 0, policy_version 1321894 (0.0008) [2023-12-27 14:48:29,003][20347] Updated weights for policy 0, policy_version 1321904 (0.0011) [2023-12-27 14:48:29,060][20347] Updated weights for policy 0, policy_version 1321914 (0.0010) [2023-12-27 14:48:29,266][20381] Updated weights for policy 1, policy_version 1320980 (0.0007) [2023-12-27 14:48:29,324][20381] Updated weights for policy 1, policy_version 1320990 (0.0007) [2023-12-27 14:48:29,387][20381] Updated weights for policy 1, policy_version 1321000 (0.0008) [2023-12-27 14:48:29,664][20347] Updated weights for policy 0, policy_version 1321924 (0.0008) [2023-12-27 14:48:29,722][20347] Updated weights for policy 0, policy_version 1321934 (0.0010) [2023-12-27 14:48:29,780][20347] Updated weights for policy 0, policy_version 1321944 (0.0010) [2023-12-27 14:48:30,104][20381] Updated weights for policy 1, policy_version 1321010 (0.0008) [2023-12-27 14:48:30,167][20381] Updated weights for policy 1, policy_version 1321020 (0.0008) [2023-12-27 14:48:30,231][20381] Updated weights for policy 1, policy_version 1321030 (0.0008) [2023-12-27 14:48:30,291][20381] Updated weights for policy 1, policy_version 1321040 (0.0008) [2023-12-27 14:48:30,371][20347] Updated weights for policy 0, policy_version 1321954 (0.0009) [2023-12-27 14:48:30,427][20347] Updated weights for policy 0, policy_version 1321964 (0.0008) [2023-12-27 14:48:30,491][20347] Updated weights for policy 0, policy_version 1321974 (0.0006) [2023-12-27 14:48:30,544][20347] Updated weights for policy 0, policy_version 1321984 (0.0007) [2023-12-27 14:48:31,024][20381] Updated weights for policy 1, policy_version 1321050 (0.0008) [2023-12-27 14:48:31,097][20381] Updated weights for policy 1, policy_version 1321060 (0.0009) [2023-12-27 14:48:31,164][20381] Updated weights for policy 1, policy_version 1321070 (0.0007) [2023-12-27 14:48:31,256][20347] Updated weights for policy 0, policy_version 1321994 (0.0007) [2023-12-27 14:48:31,316][20347] Updated weights for policy 0, policy_version 1322004 (0.0008) [2023-12-27 14:48:31,381][20347] Updated weights for policy 0, policy_version 1322014 (0.0007) [2023-12-27 14:48:31,876][20381] Updated weights for policy 1, policy_version 1321080 (0.0008) [2023-12-27 14:48:31,923][20381] Updated weights for policy 1, policy_version 1321090 (0.0008) [2023-12-27 14:48:31,970][20381] Updated weights for policy 1, policy_version 1321100 (0.0007) [2023-12-27 14:48:32,047][20347] Updated weights for policy 0, policy_version 1322024 (0.0007) [2023-12-27 14:48:32,100][20347] Updated weights for policy 0, policy_version 1322034 (0.0008) [2023-12-27 14:48:32,161][20347] Updated weights for policy 0, policy_version 1322044 (0.0008) [2023-12-27 14:48:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 676741120. Throughput: 0: 10288.0, 1: 10254.7. Samples: 676748748. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:32,363][19170] Avg episode reward: [(0, '853.580'), (1, '843.600')] [2023-12-27 14:48:32,698][20381] Updated weights for policy 1, policy_version 1321110 (0.0007) [2023-12-27 14:48:32,748][20381] Updated weights for policy 1, policy_version 1321120 (0.0008) [2023-12-27 14:48:32,795][20381] Updated weights for policy 1, policy_version 1321130 (0.0006) [2023-12-27 14:48:32,851][20347] Updated weights for policy 0, policy_version 1322054 (0.0008) [2023-12-27 14:48:32,914][20347] Updated weights for policy 0, policy_version 1322064 (0.0010) [2023-12-27 14:48:32,974][20347] Updated weights for policy 0, policy_version 1322074 (0.0010) [2023-12-27 14:48:33,343][20381] Updated weights for policy 1, policy_version 1321140 (0.0007) [2023-12-27 14:48:33,396][20381] Updated weights for policy 1, policy_version 1321150 (0.0005) [2023-12-27 14:48:33,446][20381] Updated weights for policy 1, policy_version 1321160 (0.0006) [2023-12-27 14:48:33,643][20347] Updated weights for policy 0, policy_version 1322084 (0.0010) [2023-12-27 14:48:33,705][20347] Updated weights for policy 0, policy_version 1322094 (0.0008) [2023-12-27 14:48:33,767][20347] Updated weights for policy 0, policy_version 1322104 (0.0007) [2023-12-27 14:48:34,028][20381] Updated weights for policy 1, policy_version 1321170 (0.0010) [2023-12-27 14:48:34,072][20381] Updated weights for policy 1, policy_version 1321180 (0.0008) [2023-12-27 14:48:34,116][20381] Updated weights for policy 1, policy_version 1321190 (0.0007) [2023-12-27 14:48:34,171][20381] Updated weights for policy 1, policy_version 1321200 (0.0007) [2023-12-27 14:48:34,375][20347] Updated weights for policy 0, policy_version 1322114 (0.0008) [2023-12-27 14:48:34,429][20347] Updated weights for policy 0, policy_version 1322124 (0.0005) [2023-12-27 14:48:34,488][20347] Updated weights for policy 0, policy_version 1322134 (0.0005) [2023-12-27 14:48:34,542][20347] Updated weights for policy 0, policy_version 1322144 (0.0005) [2023-12-27 14:48:34,924][20381] Updated weights for policy 1, policy_version 1321210 (0.0007) [2023-12-27 14:48:34,982][20381] Updated weights for policy 1, policy_version 1321220 (0.0008) [2023-12-27 14:48:35,039][20381] Updated weights for policy 1, policy_version 1321230 (0.0007) [2023-12-27 14:48:35,116][20347] Updated weights for policy 0, policy_version 1322154 (0.0007) [2023-12-27 14:48:35,173][20347] Updated weights for policy 0, policy_version 1322164 (0.0007) [2023-12-27 14:48:35,233][20347] Updated weights for policy 0, policy_version 1322174 (0.0007) [2023-12-27 14:48:35,716][20381] Updated weights for policy 1, policy_version 1321240 (0.0006) [2023-12-27 14:48:35,770][20381] Updated weights for policy 1, policy_version 1321250 (0.0005) [2023-12-27 14:48:35,817][20347] Updated weights for policy 0, policy_version 1322184 (0.0007) [2023-12-27 14:48:35,821][20381] Updated weights for policy 1, policy_version 1321260 (0.0007) [2023-12-27 14:48:35,866][20347] Updated weights for policy 0, policy_version 1322194 (0.0008) [2023-12-27 14:48:35,918][20347] Updated weights for policy 0, policy_version 1322204 (0.0007) [2023-12-27 14:48:36,466][20381] Updated weights for policy 1, policy_version 1321270 (0.0007) [2023-12-27 14:48:36,525][20381] Updated weights for policy 1, policy_version 1321280 (0.0011) [2023-12-27 14:48:36,585][20381] Updated weights for policy 1, policy_version 1321290 (0.0010) [2023-12-27 14:48:36,650][20347] Updated weights for policy 0, policy_version 1322214 (0.0007) [2023-12-27 14:48:36,703][20347] Updated weights for policy 0, policy_version 1322224 (0.0008) [2023-12-27 14:48:36,754][20347] Updated weights for policy 0, policy_version 1322234 (0.0008) [2023-12-27 14:48:37,212][20381] Updated weights for policy 1, policy_version 1321300 (0.0007) [2023-12-27 14:48:37,277][20381] Updated weights for policy 1, policy_version 1321310 (0.0010) [2023-12-27 14:48:37,331][20381] Updated weights for policy 1, policy_version 1321320 (0.0010) [2023-12-27 14:48:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 676839424. Throughput: 0: 10350.6, 1: 10294.3. Samples: 676813028. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:37,363][19170] Avg episode reward: [(0, '863.700'), (1, '912.130')] [2023-12-27 14:48:37,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001321328_338288640.pth... [2023-12-27 14:48:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001320112_337977344.pth [2023-12-27 14:48:37,398][20347] Updated weights for policy 0, policy_version 1322244 (0.0007) [2023-12-27 14:48:37,449][20347] Updated weights for policy 0, policy_version 1322254 (0.0007) [2023-12-27 14:48:37,503][20347] Updated weights for policy 0, policy_version 1322264 (0.0007) [2023-12-27 14:48:37,546][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001322272_338567168.pth... [2023-12-27 14:48:37,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001321056_338255872.pth [2023-12-27 14:48:37,992][20381] Updated weights for policy 1, policy_version 1321330 (0.0010) [2023-12-27 14:48:38,044][20381] Updated weights for policy 1, policy_version 1321340 (0.0011) [2023-12-27 14:48:38,096][20381] Updated weights for policy 1, policy_version 1321350 (0.0010) [2023-12-27 14:48:38,157][20381] Updated weights for policy 1, policy_version 1321360 (0.0010) [2023-12-27 14:48:38,199][20347] Updated weights for policy 0, policy_version 1322274 (0.0007) [2023-12-27 14:48:38,265][20347] Updated weights for policy 0, policy_version 1322284 (0.0005) [2023-12-27 14:48:38,328][20347] Updated weights for policy 0, policy_version 1322294 (0.0006) [2023-12-27 14:48:38,388][20347] Updated weights for policy 0, policy_version 1322304 (0.0007) [2023-12-27 14:48:38,804][20381] Updated weights for policy 1, policy_version 1321370 (0.0006) [2023-12-27 14:48:38,864][20381] Updated weights for policy 1, policy_version 1321380 (0.0007) [2023-12-27 14:48:38,918][20381] Updated weights for policy 1, policy_version 1321390 (0.0008) [2023-12-27 14:48:38,996][20347] Updated weights for policy 0, policy_version 1322314 (0.0007) [2023-12-27 14:48:39,051][20347] Updated weights for policy 0, policy_version 1322324 (0.0007) [2023-12-27 14:48:39,109][20347] Updated weights for policy 0, policy_version 1322334 (0.0010) [2023-12-27 14:48:39,674][20381] Updated weights for policy 1, policy_version 1321400 (0.0008) [2023-12-27 14:48:39,734][20381] Updated weights for policy 1, policy_version 1321410 (0.0008) [2023-12-27 14:48:39,769][20347] Updated weights for policy 0, policy_version 1322344 (0.0008) [2023-12-27 14:48:39,794][20381] Updated weights for policy 1, policy_version 1321420 (0.0007) [2023-12-27 14:48:39,828][20347] Updated weights for policy 0, policy_version 1322354 (0.0006) [2023-12-27 14:48:39,887][20347] Updated weights for policy 0, policy_version 1322364 (0.0009) [2023-12-27 14:48:40,528][20381] Updated weights for policy 1, policy_version 1321430 (0.0008) [2023-12-27 14:48:40,579][20347] Updated weights for policy 0, policy_version 1322374 (0.0008) [2023-12-27 14:48:40,584][20381] Updated weights for policy 1, policy_version 1321440 (0.0006) [2023-12-27 14:48:40,630][20347] Updated weights for policy 0, policy_version 1322384 (0.0007) [2023-12-27 14:48:40,643][20381] Updated weights for policy 1, policy_version 1321450 (0.0005) [2023-12-27 14:48:40,682][20347] Updated weights for policy 0, policy_version 1322394 (0.0007) [2023-12-27 14:48:41,371][20347] Updated weights for policy 0, policy_version 1322404 (0.0009) [2023-12-27 14:48:41,382][20381] Updated weights for policy 1, policy_version 1321460 (0.0009) [2023-12-27 14:48:41,440][20347] Updated weights for policy 0, policy_version 1322414 (0.0007) [2023-12-27 14:48:41,441][20381] Updated weights for policy 1, policy_version 1321470 (0.0010) [2023-12-27 14:48:41,487][20381] Updated weights for policy 1, policy_version 1321480 (0.0010) [2023-12-27 14:48:41,493][20347] Updated weights for policy 0, policy_version 1322424 (0.0007) [2023-12-27 14:48:42,145][20381] Updated weights for policy 1, policy_version 1321490 (0.0010) [2023-12-27 14:48:42,147][20347] Updated weights for policy 0, policy_version 1322434 (0.0007) [2023-12-27 14:48:42,205][20347] Updated weights for policy 0, policy_version 1322444 (0.0008) [2023-12-27 14:48:42,205][20381] Updated weights for policy 1, policy_version 1321500 (0.0011) [2023-12-27 14:48:42,263][20347] Updated weights for policy 0, policy_version 1322454 (0.0008) [2023-12-27 14:48:42,268][20381] Updated weights for policy 1, policy_version 1321510 (0.0011) [2023-12-27 14:48:42,333][20381] Updated weights for policy 1, policy_version 1321520 (0.0009) [2023-12-27 14:48:42,333][20347] Updated weights for policy 0, policy_version 1322464 (0.0008) [2023-12-27 14:48:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 676954112. Throughput: 0: 10346.5, 1: 10270.8. Samples: 676936200. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:42,363][19170] Avg episode reward: [(0, '796.320'), (1, '892.530')] [2023-12-27 14:48:42,935][20347] Updated weights for policy 0, policy_version 1322474 (0.0009) [2023-12-27 14:48:42,988][20347] Updated weights for policy 0, policy_version 1322484 (0.0006) [2023-12-27 14:48:43,019][20381] Updated weights for policy 1, policy_version 1321530 (0.0006) [2023-12-27 14:48:43,049][20347] Updated weights for policy 0, policy_version 1322494 (0.0006) [2023-12-27 14:48:43,070][20381] Updated weights for policy 1, policy_version 1321540 (0.0007) [2023-12-27 14:48:43,116][20381] Updated weights for policy 1, policy_version 1321550 (0.0010) [2023-12-27 14:48:43,694][20347] Updated weights for policy 0, policy_version 1322504 (0.0007) [2023-12-27 14:48:43,743][20347] Updated weights for policy 0, policy_version 1322514 (0.0009) [2023-12-27 14:48:43,790][20347] Updated weights for policy 0, policy_version 1322524 (0.0007) [2023-12-27 14:48:43,821][20381] Updated weights for policy 1, policy_version 1321560 (0.0006) [2023-12-27 14:48:43,876][20381] Updated weights for policy 1, policy_version 1321570 (0.0007) [2023-12-27 14:48:43,921][20381] Updated weights for policy 1, policy_version 1321580 (0.0008) [2023-12-27 14:48:44,513][20347] Updated weights for policy 0, policy_version 1322534 (0.0006) [2023-12-27 14:48:44,568][20347] Updated weights for policy 0, policy_version 1322544 (0.0006) [2023-12-27 14:48:44,607][20381] Updated weights for policy 1, policy_version 1321590 (0.0008) [2023-12-27 14:48:44,622][20347] Updated weights for policy 0, policy_version 1322554 (0.0007) [2023-12-27 14:48:44,671][20381] Updated weights for policy 1, policy_version 1321600 (0.0005) [2023-12-27 14:48:44,728][20381] Updated weights for policy 1, policy_version 1321610 (0.0005) [2023-12-27 14:48:45,360][20347] Updated weights for policy 0, policy_version 1322564 (0.0007) [2023-12-27 14:48:45,396][20381] Updated weights for policy 1, policy_version 1321621 (0.0006) [2023-12-27 14:48:45,424][20347] Updated weights for policy 0, policy_version 1322574 (0.0006) [2023-12-27 14:48:45,451][20381] Updated weights for policy 1, policy_version 1321631 (0.0007) [2023-12-27 14:48:45,487][20347] Updated weights for policy 0, policy_version 1322584 (0.0006) [2023-12-27 14:48:45,500][20381] Updated weights for policy 1, policy_version 1321641 (0.0006) [2023-12-27 14:48:46,096][20381] Updated weights for policy 1, policy_version 1321651 (0.0005) [2023-12-27 14:48:46,144][20381] Updated weights for policy 1, policy_version 1321661 (0.0006) [2023-12-27 14:48:46,213][20381] Updated weights for policy 1, policy_version 1321671 (0.0007) [2023-12-27 14:48:46,223][20347] Updated weights for policy 0, policy_version 1322594 (0.0006) [2023-12-27 14:48:46,278][20347] Updated weights for policy 0, policy_version 1322604 (0.0007) [2023-12-27 14:48:46,338][20347] Updated weights for policy 0, policy_version 1322614 (0.0008) [2023-12-27 14:48:46,401][20347] Updated weights for policy 0, policy_version 1322624 (0.0008) [2023-12-27 14:48:46,889][20381] Updated weights for policy 1, policy_version 1321681 (0.0007) [2023-12-27 14:48:46,940][20381] Updated weights for policy 1, policy_version 1321691 (0.0008) [2023-12-27 14:48:46,988][20381] Updated weights for policy 1, policy_version 1321701 (0.0007) [2023-12-27 14:48:47,038][20381] Updated weights for policy 1, policy_version 1321711 (0.0007) [2023-12-27 14:48:47,078][20347] Updated weights for policy 0, policy_version 1322634 (0.0007) [2023-12-27 14:48:47,127][20347] Updated weights for policy 0, policy_version 1322644 (0.0008) [2023-12-27 14:48:47,182][20347] Updated weights for policy 0, policy_version 1322654 (0.0007) [2023-12-27 14:48:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 677052416. Throughput: 0: 10347.4, 1: 10271.7. Samples: 677058636. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:47,363][19170] Avg episode reward: [(0, '786.990'), (1, '902.230')] [2023-12-27 14:48:47,703][20381] Updated weights for policy 1, policy_version 1321721 (0.0007) [2023-12-27 14:48:47,767][20381] Updated weights for policy 1, policy_version 1321731 (0.0008) [2023-12-27 14:48:47,820][20381] Updated weights for policy 1, policy_version 1321741 (0.0007) [2023-12-27 14:48:47,896][20347] Updated weights for policy 0, policy_version 1322664 (0.0008) [2023-12-27 14:48:47,945][20347] Updated weights for policy 0, policy_version 1322674 (0.0007) [2023-12-27 14:48:47,993][20347] Updated weights for policy 0, policy_version 1322684 (0.0008) [2023-12-27 14:48:48,453][20381] Updated weights for policy 1, policy_version 1321751 (0.0007) [2023-12-27 14:48:48,516][20381] Updated weights for policy 1, policy_version 1321761 (0.0007) [2023-12-27 14:48:48,571][20381] Updated weights for policy 1, policy_version 1321771 (0.0006) [2023-12-27 14:48:48,738][20347] Updated weights for policy 0, policy_version 1322694 (0.0007) [2023-12-27 14:48:48,801][20347] Updated weights for policy 0, policy_version 1322704 (0.0007) [2023-12-27 14:48:48,865][20347] Updated weights for policy 0, policy_version 1322714 (0.0007) [2023-12-27 14:48:49,183][20381] Updated weights for policy 1, policy_version 1321781 (0.0005) [2023-12-27 14:48:49,254][20381] Updated weights for policy 1, policy_version 1321791 (0.0008) [2023-12-27 14:48:49,316][20381] Updated weights for policy 1, policy_version 1321801 (0.0008) [2023-12-27 14:48:49,584][20347] Updated weights for policy 0, policy_version 1322724 (0.0008) [2023-12-27 14:48:49,647][20347] Updated weights for policy 0, policy_version 1322734 (0.0008) [2023-12-27 14:48:49,709][20347] Updated weights for policy 0, policy_version 1322744 (0.0007) [2023-12-27 14:48:50,032][20381] Updated weights for policy 1, policy_version 1321811 (0.0008) [2023-12-27 14:48:50,095][20381] Updated weights for policy 1, policy_version 1321821 (0.0007) [2023-12-27 14:48:50,159][20381] Updated weights for policy 1, policy_version 1321831 (0.0008) [2023-12-27 14:48:50,406][20347] Updated weights for policy 0, policy_version 1322754 (0.0008) [2023-12-27 14:48:50,455][20347] Updated weights for policy 0, policy_version 1322764 (0.0006) [2023-12-27 14:48:50,506][20347] Updated weights for policy 0, policy_version 1322774 (0.0005) [2023-12-27 14:48:50,561][20347] Updated weights for policy 0, policy_version 1322784 (0.0006) [2023-12-27 14:48:50,850][20381] Updated weights for policy 1, policy_version 1321841 (0.0008) [2023-12-27 14:48:50,912][20381] Updated weights for policy 1, policy_version 1321851 (0.0007) [2023-12-27 14:48:50,970][20381] Updated weights for policy 1, policy_version 1321861 (0.0007) [2023-12-27 14:48:51,039][20381] Updated weights for policy 1, policy_version 1321871 (0.0007) [2023-12-27 14:48:51,248][20347] Updated weights for policy 0, policy_version 1322794 (0.0009) [2023-12-27 14:48:51,312][20347] Updated weights for policy 0, policy_version 1322804 (0.0008) [2023-12-27 14:48:51,381][20347] Updated weights for policy 0, policy_version 1322814 (0.0009) [2023-12-27 14:48:51,777][20381] Updated weights for policy 1, policy_version 1321881 (0.0008) [2023-12-27 14:48:51,837][20381] Updated weights for policy 1, policy_version 1321891 (0.0007) [2023-12-27 14:48:51,896][20381] Updated weights for policy 1, policy_version 1321901 (0.0008) [2023-12-27 14:48:52,113][20347] Updated weights for policy 0, policy_version 1322824 (0.0007) [2023-12-27 14:48:52,179][20347] Updated weights for policy 0, policy_version 1322834 (0.0007) [2023-12-27 14:48:52,242][20347] Updated weights for policy 0, policy_version 1322844 (0.0006) [2023-12-27 14:48:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 677150720. Throughput: 0: 10301.7, 1: 10291.9. Samples: 677119016. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:52,363][19170] Avg episode reward: [(0, '824.790'), (1, '941.410')] [2023-12-27 14:48:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001321904_338436096.pth... [2023-12-27 14:48:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001322848_338714624.pth... [2023-12-27 14:48:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001320688_338124800.pth [2023-12-27 14:48:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001321632_338403328.pth [2023-12-27 14:48:52,595][20381] Updated weights for policy 1, policy_version 1321911 (0.0007) [2023-12-27 14:48:52,651][20381] Updated weights for policy 1, policy_version 1321921 (0.0007) [2023-12-27 14:48:52,711][20381] Updated weights for policy 1, policy_version 1321931 (0.0008) [2023-12-27 14:48:52,894][20347] Updated weights for policy 0, policy_version 1322854 (0.0007) [2023-12-27 14:48:52,941][20347] Updated weights for policy 0, policy_version 1322864 (0.0009) [2023-12-27 14:48:52,992][20347] Updated weights for policy 0, policy_version 1322874 (0.0010) [2023-12-27 14:48:53,407][20381] Updated weights for policy 1, policy_version 1321941 (0.0008) [2023-12-27 14:48:53,456][20381] Updated weights for policy 1, policy_version 1321951 (0.0008) [2023-12-27 14:48:53,491][20292] KL-divergence is very high: 102.2458 [2023-12-27 14:48:53,510][20292] KL-divergence is very high: 199.6652 [2023-12-27 14:48:53,516][20381] Updated weights for policy 1, policy_version 1321961 (0.0007) [2023-12-27 14:48:53,689][20347] Updated weights for policy 0, policy_version 1322884 (0.0008) [2023-12-27 14:48:53,747][20347] Updated weights for policy 0, policy_version 1322894 (0.0008) [2023-12-27 14:48:53,794][20347] Updated weights for policy 0, policy_version 1322904 (0.0008) [2023-12-27 14:48:54,189][20381] Updated weights for policy 1, policy_version 1321971 (0.0008) [2023-12-27 14:48:54,236][20381] Updated weights for policy 1, policy_version 1321981 (0.0005) [2023-12-27 14:48:54,289][20381] Updated weights for policy 1, policy_version 1321991 (0.0005) [2023-12-27 14:48:54,522][20347] Updated weights for policy 0, policy_version 1322914 (0.0007) [2023-12-27 14:48:54,579][20347] Updated weights for policy 0, policy_version 1322924 (0.0007) [2023-12-27 14:48:54,638][20347] Updated weights for policy 0, policy_version 1322934 (0.0005) [2023-12-27 14:48:54,698][20347] Updated weights for policy 0, policy_version 1322944 (0.0005) [2023-12-27 14:48:54,927][20381] Updated weights for policy 1, policy_version 1322001 (0.0006) [2023-12-27 14:48:54,978][20381] Updated weights for policy 1, policy_version 1322011 (0.0009) [2023-12-27 14:48:55,036][20381] Updated weights for policy 1, policy_version 1322021 (0.0009) [2023-12-27 14:48:55,099][20381] Updated weights for policy 1, policy_version 1322031 (0.0010) [2023-12-27 14:48:55,320][20347] Updated weights for policy 0, policy_version 1322954 (0.0005) [2023-12-27 14:48:55,379][20347] Updated weights for policy 0, policy_version 1322964 (0.0005) [2023-12-27 14:48:55,440][20347] Updated weights for policy 0, policy_version 1322974 (0.0006) [2023-12-27 14:48:55,692][20381] Updated weights for policy 1, policy_version 1322041 (0.0006) [2023-12-27 14:48:55,746][20381] Updated weights for policy 1, policy_version 1322051 (0.0005) [2023-12-27 14:48:55,803][20381] Updated weights for policy 1, policy_version 1322061 (0.0008) [2023-12-27 14:48:55,964][20347] Updated weights for policy 0, policy_version 1322984 (0.0006) [2023-12-27 14:48:56,024][20347] Updated weights for policy 0, policy_version 1322994 (0.0005) [2023-12-27 14:48:56,075][20347] Updated weights for policy 0, policy_version 1323004 (0.0005) [2023-12-27 14:48:56,411][20381] Updated weights for policy 1, policy_version 1322071 (0.0010) [2023-12-27 14:48:56,466][20381] Updated weights for policy 1, policy_version 1322081 (0.0010) [2023-12-27 14:48:56,524][20381] Updated weights for policy 1, policy_version 1322091 (0.0010) [2023-12-27 14:48:56,697][20347] Updated weights for policy 0, policy_version 1323014 (0.0006) [2023-12-27 14:48:56,740][20347] Updated weights for policy 0, policy_version 1323024 (0.0008) [2023-12-27 14:48:56,783][20347] Updated weights for policy 0, policy_version 1323034 (0.0007) [2023-12-27 14:48:57,076][20381] Updated weights for policy 1, policy_version 1322101 (0.0009) [2023-12-27 14:48:57,129][20381] Updated weights for policy 1, policy_version 1322111 (0.0008) [2023-12-27 14:48:57,179][20381] Updated weights for policy 1, policy_version 1322121 (0.0009) [2023-12-27 14:48:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 677257216. Throughput: 0: 10269.3, 1: 10301.4. Samples: 677243104. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:48:57,363][19170] Avg episode reward: [(0, '834.340'), (1, '903.160')] [2023-12-27 14:48:57,426][20347] Updated weights for policy 0, policy_version 1323044 (0.0009) [2023-12-27 14:48:57,477][20347] Updated weights for policy 0, policy_version 1323054 (0.0010) [2023-12-27 14:48:57,523][20347] Updated weights for policy 0, policy_version 1323064 (0.0010) [2023-12-27 14:48:57,800][20381] Updated weights for policy 1, policy_version 1322131 (0.0008) [2023-12-27 14:48:57,851][20381] Updated weights for policy 1, policy_version 1322141 (0.0006) [2023-12-27 14:48:57,907][20381] Updated weights for policy 1, policy_version 1322151 (0.0008) [2023-12-27 14:48:58,170][20347] Updated weights for policy 0, policy_version 1323074 (0.0010) [2023-12-27 14:48:58,231][20347] Updated weights for policy 0, policy_version 1323084 (0.0011) [2023-12-27 14:48:58,284][20347] Updated weights for policy 0, policy_version 1323094 (0.0010) [2023-12-27 14:48:58,353][20347] Updated weights for policy 0, policy_version 1323104 (0.0010) [2023-12-27 14:48:58,533][20381] Updated weights for policy 1, policy_version 1322161 (0.0007) [2023-12-27 14:48:58,596][20381] Updated weights for policy 1, policy_version 1322171 (0.0008) [2023-12-27 14:48:58,657][20381] Updated weights for policy 1, policy_version 1322181 (0.0008) [2023-12-27 14:48:58,726][20381] Updated weights for policy 1, policy_version 1322191 (0.0007) [2023-12-27 14:48:59,133][20347] Updated weights for policy 0, policy_version 1323114 (0.0008) [2023-12-27 14:48:59,194][20347] Updated weights for policy 0, policy_version 1323124 (0.0005) [2023-12-27 14:48:59,260][20347] Updated weights for policy 0, policy_version 1323134 (0.0007) [2023-12-27 14:48:59,496][20381] Updated weights for policy 1, policy_version 1322201 (0.0007) [2023-12-27 14:48:59,551][20381] Updated weights for policy 1, policy_version 1322211 (0.0007) [2023-12-27 14:48:59,605][20381] Updated weights for policy 1, policy_version 1322221 (0.0007) [2023-12-27 14:48:59,918][20347] Updated weights for policy 0, policy_version 1323144 (0.0010) [2023-12-27 14:48:59,979][20347] Updated weights for policy 0, policy_version 1323154 (0.0008) [2023-12-27 14:49:00,032][20347] Updated weights for policy 0, policy_version 1323164 (0.0008) [2023-12-27 14:49:00,296][20381] Updated weights for policy 1, policy_version 1322231 (0.0007) [2023-12-27 14:49:00,356][20381] Updated weights for policy 1, policy_version 1322241 (0.0008) [2023-12-27 14:49:00,413][20381] Updated weights for policy 1, policy_version 1322251 (0.0008) [2023-12-27 14:49:00,720][20347] Updated weights for policy 0, policy_version 1323174 (0.0006) [2023-12-27 14:49:00,769][20291] KL-divergence is very high: 103.9836 [2023-12-27 14:49:00,769][20347] Updated weights for policy 0, policy_version 1323184 (0.0009) [2023-12-27 14:49:00,820][20347] Updated weights for policy 0, policy_version 1323194 (0.0010) [2023-12-27 14:49:01,070][20381] Updated weights for policy 1, policy_version 1322261 (0.0008) [2023-12-27 14:49:01,123][20381] Updated weights for policy 1, policy_version 1322271 (0.0008) [2023-12-27 14:49:01,183][20381] Updated weights for policy 1, policy_version 1322281 (0.0007) [2023-12-27 14:49:01,525][20347] Updated weights for policy 0, policy_version 1323204 (0.0009) [2023-12-27 14:49:01,587][20347] Updated weights for policy 0, policy_version 1323214 (0.0008) [2023-12-27 14:49:01,664][20347] Updated weights for policy 0, policy_version 1323224 (0.0008) [2023-12-27 14:49:01,882][20381] Updated weights for policy 1, policy_version 1322291 (0.0006) [2023-12-27 14:49:01,939][20381] Updated weights for policy 1, policy_version 1322301 (0.0008) [2023-12-27 14:49:01,995][20381] Updated weights for policy 1, policy_version 1322311 (0.0008) [2023-12-27 14:49:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 677355520. Throughput: 0: 10299.7, 1: 10296.4. Samples: 677367120. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:02,363][19170] Avg episode reward: [(0, '775.650'), (1, '894.120')] [2023-12-27 14:49:02,389][20347] Updated weights for policy 0, policy_version 1323234 (0.0008) [2023-12-27 14:49:02,434][20347] Updated weights for policy 0, policy_version 1323244 (0.0008) [2023-12-27 14:49:02,481][20347] Updated weights for policy 0, policy_version 1323254 (0.0008) [2023-12-27 14:49:02,535][20347] Updated weights for policy 0, policy_version 1323264 (0.0008) [2023-12-27 14:49:02,653][20381] Updated weights for policy 1, policy_version 1322321 (0.0007) [2023-12-27 14:49:02,716][20381] Updated weights for policy 1, policy_version 1322331 (0.0007) [2023-12-27 14:49:02,771][20381] Updated weights for policy 1, policy_version 1322341 (0.0008) [2023-12-27 14:49:02,817][20381] Updated weights for policy 1, policy_version 1322351 (0.0007) [2023-12-27 14:49:03,224][20347] Updated weights for policy 0, policy_version 1323274 (0.0009) [2023-12-27 14:49:03,278][20347] Updated weights for policy 0, policy_version 1323284 (0.0009) [2023-12-27 14:49:03,326][20347] Updated weights for policy 0, policy_version 1323294 (0.0010) [2023-12-27 14:49:03,477][20381] Updated weights for policy 1, policy_version 1322361 (0.0007) [2023-12-27 14:49:03,529][20381] Updated weights for policy 1, policy_version 1322371 (0.0007) [2023-12-27 14:49:03,584][20381] Updated weights for policy 1, policy_version 1322381 (0.0007) [2023-12-27 14:49:04,013][20347] Updated weights for policy 0, policy_version 1323304 (0.0010) [2023-12-27 14:49:04,075][20347] Updated weights for policy 0, policy_version 1323314 (0.0010) [2023-12-27 14:49:04,142][20347] Updated weights for policy 0, policy_version 1323324 (0.0011) [2023-12-27 14:49:04,270][20381] Updated weights for policy 1, policy_version 1322391 (0.0007) [2023-12-27 14:49:04,332][20381] Updated weights for policy 1, policy_version 1322401 (0.0007) [2023-12-27 14:49:04,392][20381] Updated weights for policy 1, policy_version 1322411 (0.0008) [2023-12-27 14:49:04,820][20347] Updated weights for policy 0, policy_version 1323334 (0.0010) [2023-12-27 14:49:04,865][20347] Updated weights for policy 0, policy_version 1323344 (0.0010) [2023-12-27 14:49:04,914][20347] Updated weights for policy 0, policy_version 1323354 (0.0010) [2023-12-27 14:49:05,027][20381] Updated weights for policy 1, policy_version 1322421 (0.0008) [2023-12-27 14:49:05,090][20381] Updated weights for policy 1, policy_version 1322431 (0.0008) [2023-12-27 14:49:05,152][20381] Updated weights for policy 1, policy_version 1322441 (0.0006) [2023-12-27 14:49:05,538][20347] Updated weights for policy 0, policy_version 1323364 (0.0010) [2023-12-27 14:49:05,597][20347] Updated weights for policy 0, policy_version 1323374 (0.0011) [2023-12-27 14:49:05,651][20347] Updated weights for policy 0, policy_version 1323384 (0.0010) [2023-12-27 14:49:05,773][20381] Updated weights for policy 1, policy_version 1322451 (0.0007) [2023-12-27 14:49:05,831][20381] Updated weights for policy 1, policy_version 1322461 (0.0010) [2023-12-27 14:49:05,888][20381] Updated weights for policy 1, policy_version 1322471 (0.0010) [2023-12-27 14:49:06,358][20347] Updated weights for policy 0, policy_version 1323394 (0.0009) [2023-12-27 14:49:06,416][20347] Updated weights for policy 0, policy_version 1323404 (0.0006) [2023-12-27 14:49:06,476][20347] Updated weights for policy 0, policy_version 1323414 (0.0008) [2023-12-27 14:49:06,521][20381] Updated weights for policy 1, policy_version 1322481 (0.0009) [2023-12-27 14:49:06,535][20347] Updated weights for policy 0, policy_version 1323424 (0.0007) [2023-12-27 14:49:06,581][20381] Updated weights for policy 1, policy_version 1322491 (0.0006) [2023-12-27 14:49:06,641][20381] Updated weights for policy 1, policy_version 1322501 (0.0006) [2023-12-27 14:49:06,703][20381] Updated weights for policy 1, policy_version 1322511 (0.0006) [2023-12-27 14:49:07,240][20347] Updated weights for policy 0, policy_version 1323434 (0.0007) [2023-12-27 14:49:07,268][20381] Updated weights for policy 1, policy_version 1322521 (0.0010) [2023-12-27 14:49:07,300][20347] Updated weights for policy 0, policy_version 1323444 (0.0006) [2023-12-27 14:49:07,321][20381] Updated weights for policy 1, policy_version 1322531 (0.0011) [2023-12-27 14:49:07,353][20347] Updated weights for policy 0, policy_version 1323454 (0.0006) [2023-12-27 14:49:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 677462016. Throughput: 0: 10287.1, 1: 10355.2. Samples: 677428992. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:07,363][19170] Avg episode reward: [(0, '718.410'), (1, '902.950')] [2023-12-27 14:49:07,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001323456_338870272.pth... [2023-12-27 14:49:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001322272_338567168.pth [2023-12-27 14:49:07,375][20381] Updated weights for policy 1, policy_version 1322541 (0.0010) [2023-12-27 14:49:07,389][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001322544_338599936.pth... [2023-12-27 14:49:07,394][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001321328_338288640.pth [2023-12-27 14:49:08,002][20381] Updated weights for policy 1, policy_version 1322551 (0.0007) [2023-12-27 14:49:08,021][20347] Updated weights for policy 0, policy_version 1323464 (0.0007) [2023-12-27 14:49:08,065][20381] Updated weights for policy 1, policy_version 1322561 (0.0009) [2023-12-27 14:49:08,074][20347] Updated weights for policy 0, policy_version 1323474 (0.0007) [2023-12-27 14:49:08,121][20381] Updated weights for policy 1, policy_version 1322571 (0.0010) [2023-12-27 14:49:08,123][20347] Updated weights for policy 0, policy_version 1323484 (0.0005) [2023-12-27 14:49:08,763][20381] Updated weights for policy 1, policy_version 1322581 (0.0008) [2023-12-27 14:49:08,809][20347] Updated weights for policy 0, policy_version 1323494 (0.0006) [2023-12-27 14:49:08,821][20381] Updated weights for policy 1, policy_version 1322591 (0.0008) [2023-12-27 14:49:08,868][20347] Updated weights for policy 0, policy_version 1323504 (0.0006) [2023-12-27 14:49:08,877][20381] Updated weights for policy 1, policy_version 1322601 (0.0006) [2023-12-27 14:49:08,933][20347] Updated weights for policy 0, policy_version 1323514 (0.0006) [2023-12-27 14:49:09,530][20381] Updated weights for policy 1, policy_version 1322611 (0.0009) [2023-12-27 14:49:09,596][20381] Updated weights for policy 1, policy_version 1322621 (0.0009) [2023-12-27 14:49:09,660][20381] Updated weights for policy 1, policy_version 1322631 (0.0006) [2023-12-27 14:49:09,715][20347] Updated weights for policy 0, policy_version 1323524 (0.0008) [2023-12-27 14:49:09,764][20347] Updated weights for policy 0, policy_version 1323534 (0.0007) [2023-12-27 14:49:09,825][20347] Updated weights for policy 0, policy_version 1323544 (0.0006) [2023-12-27 14:49:10,426][20381] Updated weights for policy 1, policy_version 1322641 (0.0006) [2023-12-27 14:49:10,477][20381] Updated weights for policy 1, policy_version 1322651 (0.0009) [2023-12-27 14:49:10,534][20381] Updated weights for policy 1, policy_version 1322661 (0.0009) [2023-12-27 14:49:10,548][20347] Updated weights for policy 0, policy_version 1323554 (0.0007) [2023-12-27 14:49:10,589][20381] Updated weights for policy 1, policy_version 1322671 (0.0009) [2023-12-27 14:49:10,595][20347] Updated weights for policy 0, policy_version 1323564 (0.0006) [2023-12-27 14:49:10,639][20347] Updated weights for policy 0, policy_version 1323574 (0.0006) [2023-12-27 14:49:10,686][20347] Updated weights for policy 0, policy_version 1323584 (0.0007) [2023-12-27 14:49:11,383][20381] Updated weights for policy 1, policy_version 1322681 (0.0009) [2023-12-27 14:49:11,442][20381] Updated weights for policy 1, policy_version 1322691 (0.0007) [2023-12-27 14:49:11,497][20381] Updated weights for policy 1, policy_version 1322701 (0.0007) [2023-12-27 14:49:11,527][20347] Updated weights for policy 0, policy_version 1323594 (0.0006) [2023-12-27 14:49:11,576][20347] Updated weights for policy 0, policy_version 1323604 (0.0008) [2023-12-27 14:49:11,637][20347] Updated weights for policy 0, policy_version 1323614 (0.0008) [2023-12-27 14:49:12,215][20381] Updated weights for policy 1, policy_version 1322711 (0.0007) [2023-12-27 14:49:12,277][20381] Updated weights for policy 1, policy_version 1322721 (0.0008) [2023-12-27 14:49:12,333][20381] Updated weights for policy 1, policy_version 1322731 (0.0008) [2023-12-27 14:49:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 677552128. Throughput: 0: 10283.2, 1: 10394.2. Samples: 677552028. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:12,363][19170] Avg episode reward: [(0, '695.270'), (1, '931.480')] [2023-12-27 14:49:12,412][20347] Updated weights for policy 0, policy_version 1323624 (0.0008) [2023-12-27 14:49:12,464][20347] Updated weights for policy 0, policy_version 1323634 (0.0007) [2023-12-27 14:49:12,519][20347] Updated weights for policy 0, policy_version 1323644 (0.0010) [2023-12-27 14:49:13,023][20381] Updated weights for policy 1, policy_version 1322741 (0.0008) [2023-12-27 14:49:13,070][20381] Updated weights for policy 1, policy_version 1322751 (0.0008) [2023-12-27 14:49:13,126][20381] Updated weights for policy 1, policy_version 1322761 (0.0009) [2023-12-27 14:49:13,208][20347] Updated weights for policy 0, policy_version 1323654 (0.0007) [2023-12-27 14:49:13,253][20347] Updated weights for policy 0, policy_version 1323664 (0.0007) [2023-12-27 14:49:13,306][20347] Updated weights for policy 0, policy_version 1323674 (0.0005) [2023-12-27 14:49:13,794][20381] Updated weights for policy 1, policy_version 1322771 (0.0008) [2023-12-27 14:49:13,854][20381] Updated weights for policy 1, policy_version 1322781 (0.0008) [2023-12-27 14:49:13,886][20347] Updated weights for policy 0, policy_version 1323684 (0.0005) [2023-12-27 14:49:13,923][20381] Updated weights for policy 1, policy_version 1322791 (0.0007) [2023-12-27 14:49:13,935][20347] Updated weights for policy 0, policy_version 1323694 (0.0006) [2023-12-27 14:49:13,996][20347] Updated weights for policy 0, policy_version 1323704 (0.0008) [2023-12-27 14:49:14,474][20381] Updated weights for policy 1, policy_version 1322801 (0.0006) [2023-12-27 14:49:14,531][20381] Updated weights for policy 1, policy_version 1322811 (0.0006) [2023-12-27 14:49:14,580][20381] Updated weights for policy 1, policy_version 1322821 (0.0005) [2023-12-27 14:49:14,632][20381] Updated weights for policy 1, policy_version 1322831 (0.0005) [2023-12-27 14:49:14,646][20347] Updated weights for policy 0, policy_version 1323714 (0.0007) [2023-12-27 14:49:14,697][20347] Updated weights for policy 0, policy_version 1323724 (0.0007) [2023-12-27 14:49:14,754][20347] Updated weights for policy 0, policy_version 1323734 (0.0008) [2023-12-27 14:49:14,813][20347] Updated weights for policy 0, policy_version 1323744 (0.0008) [2023-12-27 14:49:15,284][20381] Updated weights for policy 1, policy_version 1322841 (0.0007) [2023-12-27 14:49:15,345][20381] Updated weights for policy 1, policy_version 1322851 (0.0008) [2023-12-27 14:49:15,405][20381] Updated weights for policy 1, policy_version 1322861 (0.0008) [2023-12-27 14:49:15,542][20347] Updated weights for policy 0, policy_version 1323754 (0.0007) [2023-12-27 14:49:15,606][20347] Updated weights for policy 0, policy_version 1323764 (0.0008) [2023-12-27 14:49:15,663][20347] Updated weights for policy 0, policy_version 1323774 (0.0008) [2023-12-27 14:49:16,125][20381] Updated weights for policy 1, policy_version 1322871 (0.0010) [2023-12-27 14:49:16,184][20381] Updated weights for policy 1, policy_version 1322881 (0.0011) [2023-12-27 14:49:16,242][20381] Updated weights for policy 1, policy_version 1322891 (0.0011) [2023-12-27 14:49:16,301][20347] Updated weights for policy 0, policy_version 1323784 (0.0010) [2023-12-27 14:49:16,353][20347] Updated weights for policy 0, policy_version 1323794 (0.0010) [2023-12-27 14:49:16,418][20347] Updated weights for policy 0, policy_version 1323804 (0.0008) [2023-12-27 14:49:16,894][20381] Updated weights for policy 1, policy_version 1322901 (0.0010) [2023-12-27 14:49:16,944][20381] Updated weights for policy 1, policy_version 1322911 (0.0010) [2023-12-27 14:49:17,003][20381] Updated weights for policy 1, policy_version 1322921 (0.0010) [2023-12-27 14:49:17,046][20347] Updated weights for policy 0, policy_version 1323814 (0.0007) [2023-12-27 14:49:17,106][20347] Updated weights for policy 0, policy_version 1323824 (0.0005) [2023-12-27 14:49:17,171][20347] Updated weights for policy 0, policy_version 1323834 (0.0007) [2023-12-27 14:49:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 677666816. Throughput: 0: 10211.0, 1: 10391.3. Samples: 677675852. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:17,363][19170] Avg episode reward: [(0, '704.600'), (1, '902.090')] [2023-12-27 14:49:17,677][20381] Updated weights for policy 1, policy_version 1322931 (0.0011) [2023-12-27 14:49:17,739][20381] Updated weights for policy 1, policy_version 1322941 (0.0011) [2023-12-27 14:49:17,742][20347] Updated weights for policy 0, policy_version 1323844 (0.0008) [2023-12-27 14:49:17,801][20381] Updated weights for policy 1, policy_version 1322951 (0.0010) [2023-12-27 14:49:17,804][20347] Updated weights for policy 0, policy_version 1323854 (0.0005) [2023-12-27 14:49:17,864][20347] Updated weights for policy 0, policy_version 1323864 (0.0007) [2023-12-27 14:49:18,355][20381] Updated weights for policy 1, policy_version 1322961 (0.0010) [2023-12-27 14:49:18,406][20381] Updated weights for policy 1, policy_version 1322971 (0.0008) [2023-12-27 14:49:18,430][20347] Updated weights for policy 0, policy_version 1323874 (0.0010) [2023-12-27 14:49:18,465][20381] Updated weights for policy 1, policy_version 1322981 (0.0008) [2023-12-27 14:49:18,491][20347] Updated weights for policy 0, policy_version 1323884 (0.0012) [2023-12-27 14:49:18,521][20381] Updated weights for policy 1, policy_version 1322991 (0.0008) [2023-12-27 14:49:18,539][20347] Updated weights for policy 0, policy_version 1323894 (0.0009) [2023-12-27 14:49:18,598][20347] Updated weights for policy 0, policy_version 1323904 (0.0006) [2023-12-27 14:49:19,214][20347] Updated weights for policy 0, policy_version 1323914 (0.0006) [2023-12-27 14:49:19,217][20381] Updated weights for policy 1, policy_version 1323001 (0.0007) [2023-12-27 14:49:19,280][20381] Updated weights for policy 1, policy_version 1323011 (0.0008) [2023-12-27 14:49:19,291][20347] Updated weights for policy 0, policy_version 1323924 (0.0008) [2023-12-27 14:49:19,342][20381] Updated weights for policy 1, policy_version 1323021 (0.0010) [2023-12-27 14:49:19,352][20347] Updated weights for policy 0, policy_version 1323934 (0.0008) [2023-12-27 14:49:20,025][20381] Updated weights for policy 1, policy_version 1323031 (0.0009) [2023-12-27 14:49:20,080][20381] Updated weights for policy 1, policy_version 1323041 (0.0007) [2023-12-27 14:49:20,113][20347] Updated weights for policy 0, policy_version 1323944 (0.0007) [2023-12-27 14:49:20,132][20381] Updated weights for policy 1, policy_version 1323051 (0.0007) [2023-12-27 14:49:20,170][20347] Updated weights for policy 0, policy_version 1323954 (0.0007) [2023-12-27 14:49:20,228][20347] Updated weights for policy 0, policy_version 1323964 (0.0008) [2023-12-27 14:49:20,780][20381] Updated weights for policy 1, policy_version 1323061 (0.0007) [2023-12-27 14:49:20,846][20381] Updated weights for policy 1, policy_version 1323071 (0.0008) [2023-12-27 14:49:20,914][20381] Updated weights for policy 1, policy_version 1323081 (0.0008) [2023-12-27 14:49:20,995][20347] Updated weights for policy 0, policy_version 1323974 (0.0008) [2023-12-27 14:49:21,066][20347] Updated weights for policy 0, policy_version 1323984 (0.0008) [2023-12-27 14:49:21,123][20347] Updated weights for policy 0, policy_version 1323994 (0.0007) [2023-12-27 14:49:21,630][20381] Updated weights for policy 1, policy_version 1323091 (0.0010) [2023-12-27 14:49:21,697][20381] Updated weights for policy 1, policy_version 1323101 (0.0008) [2023-12-27 14:49:21,762][20381] Updated weights for policy 1, policy_version 1323111 (0.0009) [2023-12-27 14:49:21,883][20347] Updated weights for policy 0, policy_version 1324004 (0.0008) [2023-12-27 14:49:21,938][20347] Updated weights for policy 0, policy_version 1324014 (0.0006) [2023-12-27 14:49:21,993][20347] Updated weights for policy 0, policy_version 1324024 (0.0005) [2023-12-27 14:49:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 677765120. Throughput: 0: 10220.7, 1: 10366.5. Samples: 677739448. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:22,363][19170] Avg episode reward: [(0, '715.040'), (1, '932.740')] [2023-12-27 14:49:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001323120_338747392.pth... [2023-12-27 14:49:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001324032_339017728.pth... [2023-12-27 14:49:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001321904_338436096.pth [2023-12-27 14:49:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001322848_338714624.pth [2023-12-27 14:49:22,473][20381] Updated weights for policy 1, policy_version 1323121 (0.0009) [2023-12-27 14:49:22,538][20381] Updated weights for policy 1, policy_version 1323131 (0.0007) [2023-12-27 14:49:22,596][20381] Updated weights for policy 1, policy_version 1323141 (0.0008) [2023-12-27 14:49:22,657][20381] Updated weights for policy 1, policy_version 1323151 (0.0008) [2023-12-27 14:49:22,689][20347] Updated weights for policy 0, policy_version 1324034 (0.0006) [2023-12-27 14:49:22,753][20347] Updated weights for policy 0, policy_version 1324044 (0.0008) [2023-12-27 14:49:22,807][20347] Updated weights for policy 0, policy_version 1324054 (0.0008) [2023-12-27 14:49:22,854][20347] Updated weights for policy 0, policy_version 1324064 (0.0008) [2023-12-27 14:49:23,337][20381] Updated weights for policy 1, policy_version 1323161 (0.0006) [2023-12-27 14:49:23,401][20381] Updated weights for policy 1, policy_version 1323171 (0.0007) [2023-12-27 14:49:23,456][20381] Updated weights for policy 1, policy_version 1323181 (0.0006) [2023-12-27 14:49:23,490][20347] Updated weights for policy 0, policy_version 1324074 (0.0008) [2023-12-27 14:49:23,543][20347] Updated weights for policy 0, policy_version 1324084 (0.0007) [2023-12-27 14:49:23,589][20347] Updated weights for policy 0, policy_version 1324094 (0.0008) [2023-12-27 14:49:24,056][20381] Updated weights for policy 1, policy_version 1323191 (0.0007) [2023-12-27 14:49:24,108][20381] Updated weights for policy 1, policy_version 1323201 (0.0010) [2023-12-27 14:49:24,155][20381] Updated weights for policy 1, policy_version 1323211 (0.0007) [2023-12-27 14:49:24,283][20347] Updated weights for policy 0, policy_version 1324104 (0.0008) [2023-12-27 14:49:24,341][20347] Updated weights for policy 0, policy_version 1324114 (0.0007) [2023-12-27 14:49:24,397][20347] Updated weights for policy 0, policy_version 1324124 (0.0008) [2023-12-27 14:49:24,843][20381] Updated weights for policy 1, policy_version 1323221 (0.0010) [2023-12-27 14:49:24,892][20381] Updated weights for policy 1, policy_version 1323231 (0.0011) [2023-12-27 14:49:24,950][20381] Updated weights for policy 1, policy_version 1323241 (0.0011) [2023-12-27 14:49:24,989][20347] Updated weights for policy 0, policy_version 1324134 (0.0006) [2023-12-27 14:49:25,040][20347] Updated weights for policy 0, policy_version 1324144 (0.0005) [2023-12-27 14:49:25,088][20347] Updated weights for policy 0, policy_version 1324154 (0.0005) [2023-12-27 14:49:25,590][20381] Updated weights for policy 1, policy_version 1323251 (0.0009) [2023-12-27 14:49:25,645][20381] Updated weights for policy 1, policy_version 1323261 (0.0008) [2023-12-27 14:49:25,662][20347] Updated weights for policy 0, policy_version 1324164 (0.0006) [2023-12-27 14:49:25,694][20381] Updated weights for policy 1, policy_version 1323271 (0.0011) [2023-12-27 14:49:25,714][20347] Updated weights for policy 0, policy_version 1324174 (0.0005) [2023-12-27 14:49:25,766][20347] Updated weights for policy 0, policy_version 1324184 (0.0005) [2023-12-27 14:49:26,308][20381] Updated weights for policy 1, policy_version 1323281 (0.0010) [2023-12-27 14:49:26,347][20347] Updated weights for policy 0, policy_version 1324194 (0.0006) [2023-12-27 14:49:26,379][20381] Updated weights for policy 1, policy_version 1323291 (0.0008) [2023-12-27 14:49:26,395][20347] Updated weights for policy 0, policy_version 1324204 (0.0005) [2023-12-27 14:49:26,442][20347] Updated weights for policy 0, policy_version 1324214 (0.0005) [2023-12-27 14:49:26,443][20381] Updated weights for policy 1, policy_version 1323301 (0.0008) [2023-12-27 14:49:26,494][20347] Updated weights for policy 0, policy_version 1324224 (0.0007) [2023-12-27 14:49:26,498][20381] Updated weights for policy 1, policy_version 1323311 (0.0005) [2023-12-27 14:49:27,044][20347] Updated weights for policy 0, policy_version 1324234 (0.0006) [2023-12-27 14:49:27,091][20347] Updated weights for policy 0, policy_version 1324244 (0.0006) [2023-12-27 14:49:27,136][20381] Updated weights for policy 1, policy_version 1323321 (0.0010) [2023-12-27 14:49:27,141][20347] Updated weights for policy 0, policy_version 1324254 (0.0005) [2023-12-27 14:49:27,180][20381] Updated weights for policy 1, policy_version 1323331 (0.0010) [2023-12-27 14:49:27,228][20381] Updated weights for policy 1, policy_version 1323341 (0.0010) [2023-12-27 14:49:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 677879808. Throughput: 0: 10227.9, 1: 10382.2. Samples: 677863656. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:27,363][19170] Avg episode reward: [(0, '834.640'), (1, '805.900')] [2023-12-27 14:49:27,764][20347] Updated weights for policy 0, policy_version 1324264 (0.0005) [2023-12-27 14:49:27,812][20347] Updated weights for policy 0, policy_version 1324274 (0.0006) [2023-12-27 14:49:27,864][20347] Updated weights for policy 0, policy_version 1324284 (0.0007) [2023-12-27 14:49:27,892][20381] Updated weights for policy 1, policy_version 1323351 (0.0010) [2023-12-27 14:49:27,940][20381] Updated weights for policy 1, policy_version 1323361 (0.0010) [2023-12-27 14:49:27,998][20381] Updated weights for policy 1, policy_version 1323371 (0.0010) [2023-12-27 14:49:28,468][20347] Updated weights for policy 0, policy_version 1324294 (0.0008) [2023-12-27 14:49:28,499][20291] KL-divergence is very high: 158.2108 [2023-12-27 14:49:28,512][20291] KL-divergence is very high: 169.9412 [2023-12-27 14:49:28,529][20347] Updated weights for policy 0, policy_version 1324304 (0.0008) [2023-12-27 14:49:28,549][20291] KL-divergence is very high: 215.7193 [2023-12-27 14:49:28,562][20291] KL-divergence is very high: 164.4993 [2023-12-27 14:49:28,591][20347] Updated weights for policy 0, policy_version 1324314 (0.0006) [2023-12-27 14:49:28,692][20381] Updated weights for policy 1, policy_version 1323381 (0.0008) [2023-12-27 14:49:28,749][20381] Updated weights for policy 1, policy_version 1323391 (0.0005) [2023-12-27 14:49:28,802][20381] Updated weights for policy 1, policy_version 1323401 (0.0007) [2023-12-27 14:49:29,247][20347] Updated weights for policy 0, policy_version 1324324 (0.0006) [2023-12-27 14:49:29,302][20347] Updated weights for policy 0, policy_version 1324334 (0.0007) [2023-12-27 14:49:29,364][20347] Updated weights for policy 0, policy_version 1324344 (0.0008) [2023-12-27 14:49:29,472][20381] Updated weights for policy 1, policy_version 1323411 (0.0010) [2023-12-27 14:49:29,523][20381] Updated weights for policy 1, policy_version 1323421 (0.0007) [2023-12-27 14:49:29,579][20381] Updated weights for policy 1, policy_version 1323431 (0.0008) [2023-12-27 14:49:30,075][20347] Updated weights for policy 0, policy_version 1324354 (0.0008) [2023-12-27 14:49:30,127][20347] Updated weights for policy 0, policy_version 1324364 (0.0006) [2023-12-27 14:49:30,171][20347] Updated weights for policy 0, policy_version 1324374 (0.0007) [2023-12-27 14:49:30,222][20347] Updated weights for policy 0, policy_version 1324384 (0.0006) [2023-12-27 14:49:30,313][20381] Updated weights for policy 1, policy_version 1323441 (0.0008) [2023-12-27 14:49:30,366][20381] Updated weights for policy 1, policy_version 1323451 (0.0008) [2023-12-27 14:49:30,412][20381] Updated weights for policy 1, policy_version 1323461 (0.0007) [2023-12-27 14:49:30,461][20381] Updated weights for policy 1, policy_version 1323471 (0.0008) [2023-12-27 14:49:30,872][20347] Updated weights for policy 0, policy_version 1324394 (0.0007) [2023-12-27 14:49:30,922][20347] Updated weights for policy 0, policy_version 1324404 (0.0007) [2023-12-27 14:49:30,969][20347] Updated weights for policy 0, policy_version 1324414 (0.0007) [2023-12-27 14:49:31,145][20381] Updated weights for policy 1, policy_version 1323481 (0.0007) [2023-12-27 14:49:31,213][20381] Updated weights for policy 1, policy_version 1323491 (0.0008) [2023-12-27 14:49:31,274][20381] Updated weights for policy 1, policy_version 1323501 (0.0008) [2023-12-27 14:49:31,629][20347] Updated weights for policy 0, policy_version 1324424 (0.0007) [2023-12-27 14:49:31,699][20347] Updated weights for policy 0, policy_version 1324434 (0.0008) [2023-12-27 14:49:31,753][20347] Updated weights for policy 0, policy_version 1324444 (0.0005) [2023-12-27 14:49:31,978][20381] Updated weights for policy 1, policy_version 1323511 (0.0009) [2023-12-27 14:49:32,038][20381] Updated weights for policy 1, policy_version 1323521 (0.0007) [2023-12-27 14:49:32,097][20381] Updated weights for policy 1, policy_version 1323531 (0.0008) [2023-12-27 14:49:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 677978112. Throughput: 0: 10331.8, 1: 10356.3. Samples: 677989604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:32,363][19170] Avg episode reward: [(0, '873.870'), (1, '854.170')] [2023-12-27 14:49:32,424][20347] Updated weights for policy 0, policy_version 1324454 (0.0007) [2023-12-27 14:49:32,485][20347] Updated weights for policy 0, policy_version 1324464 (0.0008) [2023-12-27 14:49:32,545][20347] Updated weights for policy 0, policy_version 1324474 (0.0007) [2023-12-27 14:49:32,830][20381] Updated weights for policy 1, policy_version 1323541 (0.0008) [2023-12-27 14:49:32,878][20381] Updated weights for policy 1, policy_version 1323551 (0.0008) [2023-12-27 14:49:32,924][20381] Updated weights for policy 1, policy_version 1323561 (0.0008) [2023-12-27 14:49:33,225][20347] Updated weights for policy 0, policy_version 1324484 (0.0007) [2023-12-27 14:49:33,272][20347] Updated weights for policy 0, policy_version 1324494 (0.0007) [2023-12-27 14:49:33,325][20347] Updated weights for policy 0, policy_version 1324504 (0.0005) [2023-12-27 14:49:33,581][20381] Updated weights for policy 1, policy_version 1323571 (0.0007) [2023-12-27 14:49:33,650][20381] Updated weights for policy 1, policy_version 1323581 (0.0006) [2023-12-27 14:49:33,709][20381] Updated weights for policy 1, policy_version 1323591 (0.0007) [2023-12-27 14:49:33,930][20347] Updated weights for policy 0, policy_version 1324514 (0.0005) [2023-12-27 14:49:33,976][20347] Updated weights for policy 0, policy_version 1324524 (0.0005) [2023-12-27 14:49:34,020][20347] Updated weights for policy 0, policy_version 1324534 (0.0005) [2023-12-27 14:49:34,065][20347] Updated weights for policy 0, policy_version 1324544 (0.0005) [2023-12-27 14:49:34,344][20381] Updated weights for policy 1, policy_version 1323601 (0.0007) [2023-12-27 14:49:34,402][20381] Updated weights for policy 1, policy_version 1323611 (0.0007) [2023-12-27 14:49:34,456][20381] Updated weights for policy 1, policy_version 1323621 (0.0008) [2023-12-27 14:49:34,513][20381] Updated weights for policy 1, policy_version 1323631 (0.0008) [2023-12-27 14:49:34,713][20347] Updated weights for policy 0, policy_version 1324554 (0.0008) [2023-12-27 14:49:34,773][20347] Updated weights for policy 0, policy_version 1324564 (0.0008) [2023-12-27 14:49:34,817][20347] Updated weights for policy 0, policy_version 1324574 (0.0005) [2023-12-27 14:49:35,168][20381] Updated weights for policy 1, policy_version 1323641 (0.0010) [2023-12-27 14:49:35,223][20381] Updated weights for policy 1, policy_version 1323651 (0.0010) [2023-12-27 14:49:35,304][20381] Updated weights for policy 1, policy_version 1323661 (0.0007) [2023-12-27 14:49:35,434][20347] Updated weights for policy 0, policy_version 1324584 (0.0009) [2023-12-27 14:49:35,485][20347] Updated weights for policy 0, policy_version 1324594 (0.0010) [2023-12-27 14:49:35,541][20347] Updated weights for policy 0, policy_version 1324604 (0.0010) [2023-12-27 14:49:35,908][20381] Updated weights for policy 1, policy_version 1323671 (0.0009) [2023-12-27 14:49:35,963][20381] Updated weights for policy 1, policy_version 1323681 (0.0010) [2023-12-27 14:49:36,017][20381] Updated weights for policy 1, policy_version 1323691 (0.0010) [2023-12-27 14:49:36,176][20347] Updated weights for policy 0, policy_version 1324614 (0.0009) [2023-12-27 14:49:36,235][20347] Updated weights for policy 0, policy_version 1324624 (0.0008) [2023-12-27 14:49:36,304][20347] Updated weights for policy 0, policy_version 1324634 (0.0009) [2023-12-27 14:49:36,671][20381] Updated weights for policy 1, policy_version 1323701 (0.0011) [2023-12-27 14:49:36,733][20381] Updated weights for policy 1, policy_version 1323711 (0.0011) [2023-12-27 14:49:36,795][20381] Updated weights for policy 1, policy_version 1323721 (0.0011) [2023-12-27 14:49:36,999][20347] Updated weights for policy 0, policy_version 1324644 (0.0009) [2023-12-27 14:49:37,056][20347] Updated weights for policy 0, policy_version 1324654 (0.0010) [2023-12-27 14:49:37,115][20347] Updated weights for policy 0, policy_version 1324664 (0.0007) [2023-12-27 14:49:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 678084608. Throughput: 0: 10401.8, 1: 10355.8. Samples: 678053108. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:37,363][19170] Avg episode reward: [(0, '796.000'), (1, '922.670')] [2023-12-27 14:49:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001323728_338903040.pth... [2023-12-27 14:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001324672_339181568.pth... [2023-12-27 14:49:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001322544_338599936.pth [2023-12-27 14:49:37,392][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001323456_338870272.pth [2023-12-27 14:49:37,472][20381] Updated weights for policy 1, policy_version 1323731 (0.0009) [2023-12-27 14:49:37,529][20381] Updated weights for policy 1, policy_version 1323741 (0.0008) [2023-12-27 14:49:37,585][20381] Updated weights for policy 1, policy_version 1323751 (0.0007) [2023-12-27 14:49:37,708][20347] Updated weights for policy 0, policy_version 1324674 (0.0006) [2023-12-27 14:49:37,767][20347] Updated weights for policy 0, policy_version 1324684 (0.0006) [2023-12-27 14:49:37,826][20347] Updated weights for policy 0, policy_version 1324694 (0.0008) [2023-12-27 14:49:37,877][20347] Updated weights for policy 0, policy_version 1324704 (0.0010) [2023-12-27 14:49:38,293][20381] Updated weights for policy 1, policy_version 1323761 (0.0009) [2023-12-27 14:49:38,357][20381] Updated weights for policy 1, policy_version 1323771 (0.0010) [2023-12-27 14:49:38,405][20381] Updated weights for policy 1, policy_version 1323781 (0.0008) [2023-12-27 14:49:38,466][20381] Updated weights for policy 1, policy_version 1323791 (0.0008) [2023-12-27 14:49:38,510][20347] Updated weights for policy 0, policy_version 1324714 (0.0006) [2023-12-27 14:49:38,563][20347] Updated weights for policy 0, policy_version 1324724 (0.0007) [2023-12-27 14:49:38,620][20347] Updated weights for policy 0, policy_version 1324734 (0.0007) [2023-12-27 14:49:39,160][20381] Updated weights for policy 1, policy_version 1323801 (0.0006) [2023-12-27 14:49:39,213][20381] Updated weights for policy 1, policy_version 1323811 (0.0007) [2023-12-27 14:49:39,277][20381] Updated weights for policy 1, policy_version 1323821 (0.0008) [2023-12-27 14:49:39,367][20347] Updated weights for policy 0, policy_version 1324744 (0.0007) [2023-12-27 14:49:39,440][20347] Updated weights for policy 0, policy_version 1324754 (0.0008) [2023-12-27 14:49:39,500][20347] Updated weights for policy 0, policy_version 1324764 (0.0008) [2023-12-27 14:49:39,961][20381] Updated weights for policy 1, policy_version 1323831 (0.0009) [2023-12-27 14:49:40,023][20381] Updated weights for policy 1, policy_version 1323841 (0.0007) [2023-12-27 14:49:40,085][20381] Updated weights for policy 1, policy_version 1323851 (0.0007) [2023-12-27 14:49:40,231][20347] Updated weights for policy 0, policy_version 1324774 (0.0007) [2023-12-27 14:49:40,293][20347] Updated weights for policy 0, policy_version 1324784 (0.0005) [2023-12-27 14:49:40,358][20347] Updated weights for policy 0, policy_version 1324794 (0.0007) [2023-12-27 14:49:40,798][20381] Updated weights for policy 1, policy_version 1323861 (0.0007) [2023-12-27 14:49:40,850][20381] Updated weights for policy 1, policy_version 1323871 (0.0008) [2023-12-27 14:49:40,910][20381] Updated weights for policy 1, policy_version 1323881 (0.0008) [2023-12-27 14:49:40,994][20347] Updated weights for policy 0, policy_version 1324804 (0.0007) [2023-12-27 14:49:41,060][20347] Updated weights for policy 0, policy_version 1324814 (0.0008) [2023-12-27 14:49:41,126][20347] Updated weights for policy 0, policy_version 1324824 (0.0006) [2023-12-27 14:49:41,678][20381] Updated weights for policy 1, policy_version 1323891 (0.0008) [2023-12-27 14:49:41,751][20381] Updated weights for policy 1, policy_version 1323901 (0.0008) [2023-12-27 14:49:41,807][20381] Updated weights for policy 1, policy_version 1323911 (0.0007) [2023-12-27 14:49:41,834][20347] Updated weights for policy 0, policy_version 1324834 (0.0008) [2023-12-27 14:49:41,897][20347] Updated weights for policy 0, policy_version 1324844 (0.0009) [2023-12-27 14:49:41,953][20347] Updated weights for policy 0, policy_version 1324854 (0.0011) [2023-12-27 14:49:42,009][20347] Updated weights for policy 0, policy_version 1324864 (0.0006) [2023-12-27 14:49:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 678182912. Throughput: 0: 10388.9, 1: 10325.9. Samples: 678175272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:42,363][19170] Avg episode reward: [(0, '766.640'), (1, '854.980')] [2023-12-27 14:49:42,458][20381] Updated weights for policy 1, policy_version 1323921 (0.0009) [2023-12-27 14:49:42,516][20381] Updated weights for policy 1, policy_version 1323931 (0.0007) [2023-12-27 14:49:42,582][20381] Updated weights for policy 1, policy_version 1323941 (0.0009) [2023-12-27 14:49:42,646][20381] Updated weights for policy 1, policy_version 1323951 (0.0006) [2023-12-27 14:49:42,735][20347] Updated weights for policy 0, policy_version 1324874 (0.0008) [2023-12-27 14:49:42,797][20347] Updated weights for policy 0, policy_version 1324884 (0.0007) [2023-12-27 14:49:42,859][20347] Updated weights for policy 0, policy_version 1324894 (0.0008) [2023-12-27 14:49:43,313][20381] Updated weights for policy 1, policy_version 1323961 (0.0008) [2023-12-27 14:49:43,374][20381] Updated weights for policy 1, policy_version 1323971 (0.0009) [2023-12-27 14:49:43,423][20381] Updated weights for policy 1, policy_version 1323981 (0.0007) [2023-12-27 14:49:43,492][20347] Updated weights for policy 0, policy_version 1324904 (0.0007) [2023-12-27 14:49:43,547][20347] Updated weights for policy 0, policy_version 1324914 (0.0006) [2023-12-27 14:49:43,593][20347] Updated weights for policy 0, policy_version 1324924 (0.0005) [2023-12-27 14:49:44,148][20381] Updated weights for policy 1, policy_version 1323991 (0.0008) [2023-12-27 14:49:44,206][20381] Updated weights for policy 1, policy_version 1324001 (0.0007) [2023-12-27 14:49:44,220][20347] Updated weights for policy 0, policy_version 1324934 (0.0007) [2023-12-27 14:49:44,259][20381] Updated weights for policy 1, policy_version 1324011 (0.0005) [2023-12-27 14:49:44,266][20347] Updated weights for policy 0, policy_version 1324944 (0.0005) [2023-12-27 14:49:44,322][20347] Updated weights for policy 0, policy_version 1324954 (0.0007) [2023-12-27 14:49:44,936][20347] Updated weights for policy 0, policy_version 1324964 (0.0007) [2023-12-27 14:49:44,994][20347] Updated weights for policy 0, policy_version 1324974 (0.0007) [2023-12-27 14:49:45,042][20381] Updated weights for policy 1, policy_version 1324021 (0.0007) [2023-12-27 14:49:45,056][20347] Updated weights for policy 0, policy_version 1324984 (0.0006) [2023-12-27 14:49:45,097][20381] Updated weights for policy 1, policy_version 1324031 (0.0007) [2023-12-27 14:49:45,154][20381] Updated weights for policy 1, policy_version 1324041 (0.0007) [2023-12-27 14:49:45,694][20347] Updated weights for policy 0, policy_version 1324994 (0.0005) [2023-12-27 14:49:45,742][20347] Updated weights for policy 0, policy_version 1325004 (0.0007) [2023-12-27 14:49:45,797][20347] Updated weights for policy 0, policy_version 1325014 (0.0007) [2023-12-27 14:49:45,849][20347] Updated weights for policy 0, policy_version 1325024 (0.0008) [2023-12-27 14:49:45,881][20381] Updated weights for policy 1, policy_version 1324051 (0.0008) [2023-12-27 14:49:45,937][20381] Updated weights for policy 1, policy_version 1324061 (0.0007) [2023-12-27 14:49:45,988][20381] Updated weights for policy 1, policy_version 1324071 (0.0007) [2023-12-27 14:49:46,517][20347] Updated weights for policy 0, policy_version 1325034 (0.0010) [2023-12-27 14:49:46,568][20347] Updated weights for policy 0, policy_version 1325044 (0.0010) [2023-12-27 14:49:46,619][20347] Updated weights for policy 0, policy_version 1325054 (0.0010) [2023-12-27 14:49:46,696][20381] Updated weights for policy 1, policy_version 1324081 (0.0007) [2023-12-27 14:49:46,765][20381] Updated weights for policy 1, policy_version 1324091 (0.0007) [2023-12-27 14:49:46,820][20381] Updated weights for policy 1, policy_version 1324101 (0.0007) [2023-12-27 14:49:46,876][20381] Updated weights for policy 1, policy_version 1324111 (0.0008) [2023-12-27 14:49:47,338][20347] Updated weights for policy 0, policy_version 1325064 (0.0009) [2023-12-27 14:49:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 678281216. Throughput: 0: 10411.4, 1: 10233.2. Samples: 678296124. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:47,363][19170] Avg episode reward: [(0, '806.040'), (1, '856.470')] [2023-12-27 14:49:47,403][20347] Updated weights for policy 0, policy_version 1325074 (0.0007) [2023-12-27 14:49:47,457][20347] Updated weights for policy 0, policy_version 1325084 (0.0006) [2023-12-27 14:49:47,581][20381] Updated weights for policy 1, policy_version 1324121 (0.0008) [2023-12-27 14:49:47,629][20381] Updated weights for policy 1, policy_version 1324131 (0.0008) [2023-12-27 14:49:47,678][20381] Updated weights for policy 1, policy_version 1324141 (0.0007) [2023-12-27 14:49:48,102][20347] Updated weights for policy 0, policy_version 1325094 (0.0009) [2023-12-27 14:49:48,116][20291] KL-divergence is very high: 121.4212 [2023-12-27 14:49:48,153][20347] Updated weights for policy 0, policy_version 1325104 (0.0010) [2023-12-27 14:49:48,211][20347] Updated weights for policy 0, policy_version 1325114 (0.0010) [2023-12-27 14:49:48,425][20381] Updated weights for policy 1, policy_version 1324151 (0.0007) [2023-12-27 14:49:48,475][20381] Updated weights for policy 1, policy_version 1324161 (0.0006) [2023-12-27 14:49:48,518][20381] Updated weights for policy 1, policy_version 1324171 (0.0007) [2023-12-27 14:49:48,934][20347] Updated weights for policy 0, policy_version 1325124 (0.0010) [2023-12-27 14:49:48,995][20347] Updated weights for policy 0, policy_version 1325134 (0.0010) [2023-12-27 14:49:49,047][20347] Updated weights for policy 0, policy_version 1325144 (0.0010) [2023-12-27 14:49:49,124][20381] Updated weights for policy 1, policy_version 1324181 (0.0008) [2023-12-27 14:49:49,183][20381] Updated weights for policy 1, policy_version 1324191 (0.0006) [2023-12-27 14:49:49,270][20381] Updated weights for policy 1, policy_version 1324201 (0.0008) [2023-12-27 14:49:49,787][20347] Updated weights for policy 0, policy_version 1325154 (0.0007) [2023-12-27 14:49:49,854][20347] Updated weights for policy 0, policy_version 1325164 (0.0008) [2023-12-27 14:49:49,913][20347] Updated weights for policy 0, policy_version 1325174 (0.0008) [2023-12-27 14:49:49,992][20347] Updated weights for policy 0, policy_version 1325184 (0.0006) [2023-12-27 14:49:50,010][20381] Updated weights for policy 1, policy_version 1324211 (0.0007) [2023-12-27 14:49:50,073][20381] Updated weights for policy 1, policy_version 1324221 (0.0008) [2023-12-27 14:49:50,136][20381] Updated weights for policy 1, policy_version 1324231 (0.0009) [2023-12-27 14:49:50,568][20347] Updated weights for policy 0, policy_version 1325194 (0.0006) [2023-12-27 14:49:50,626][20347] Updated weights for policy 0, policy_version 1325204 (0.0008) [2023-12-27 14:49:50,681][20347] Updated weights for policy 0, policy_version 1325214 (0.0007) [2023-12-27 14:49:50,863][20381] Updated weights for policy 1, policy_version 1324241 (0.0008) [2023-12-27 14:49:50,922][20381] Updated weights for policy 1, policy_version 1324251 (0.0007) [2023-12-27 14:49:50,986][20381] Updated weights for policy 1, policy_version 1324261 (0.0008) [2023-12-27 14:49:51,056][20381] Updated weights for policy 1, policy_version 1324271 (0.0007) [2023-12-27 14:49:51,466][20347] Updated weights for policy 0, policy_version 1325224 (0.0008) [2023-12-27 14:49:51,523][20347] Updated weights for policy 0, policy_version 1325234 (0.0008) [2023-12-27 14:49:51,579][20347] Updated weights for policy 0, policy_version 1325244 (0.0008) [2023-12-27 14:49:51,737][20381] Updated weights for policy 1, policy_version 1324281 (0.0008) [2023-12-27 14:49:51,794][20381] Updated weights for policy 1, policy_version 1324291 (0.0006) [2023-12-27 14:49:51,852][20381] Updated weights for policy 1, policy_version 1324301 (0.0007) [2023-12-27 14:49:52,288][20347] Updated weights for policy 0, policy_version 1325254 (0.0010) [2023-12-27 14:49:52,358][20347] Updated weights for policy 0, policy_version 1325264 (0.0011) [2023-12-27 14:49:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 678379520. Throughput: 0: 10405.3, 1: 10192.9. Samples: 678355916. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:49:52,363][19170] Avg episode reward: [(0, '825.420'), (1, '848.330')] [2023-12-27 14:49:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001324304_339050496.pth... [2023-12-27 14:49:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001323120_338747392.pth [2023-12-27 14:49:52,424][20347] Updated weights for policy 0, policy_version 1325274 (0.0007) [2023-12-27 14:49:52,462][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001325280_339337216.pth... [2023-12-27 14:49:52,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001324032_339017728.pth [2023-12-27 14:49:52,531][20381] Updated weights for policy 1, policy_version 1324311 (0.0006) [2023-12-27 14:49:52,590][20381] Updated weights for policy 1, policy_version 1324321 (0.0008) [2023-12-27 14:49:52,656][20381] Updated weights for policy 1, policy_version 1324331 (0.0011) [2023-12-27 14:49:53,053][20347] Updated weights for policy 0, policy_version 1325284 (0.0006) [2023-12-27 14:49:53,103][20347] Updated weights for policy 0, policy_version 1325294 (0.0005) [2023-12-27 14:49:53,156][20347] Updated weights for policy 0, policy_version 1325304 (0.0005) [2023-12-27 14:49:53,311][20381] Updated weights for policy 1, policy_version 1324341 (0.0008) [2023-12-27 14:49:53,379][20381] Updated weights for policy 1, policy_version 1324351 (0.0005) [2023-12-27 14:49:53,438][20381] Updated weights for policy 1, policy_version 1324361 (0.0006) [2023-12-27 14:49:53,712][20347] Updated weights for policy 0, policy_version 1325314 (0.0005) [2023-12-27 14:49:53,773][20347] Updated weights for policy 0, policy_version 1325324 (0.0005) [2023-12-27 14:49:53,841][20347] Updated weights for policy 0, policy_version 1325334 (0.0005) [2023-12-27 14:49:53,898][20347] Updated weights for policy 0, policy_version 1325344 (0.0006) [2023-12-27 14:49:53,982][20381] Updated weights for policy 1, policy_version 1324371 (0.0009) [2023-12-27 14:49:54,042][20381] Updated weights for policy 1, policy_version 1324381 (0.0006) [2023-12-27 14:49:54,088][20381] Updated weights for policy 1, policy_version 1324391 (0.0007) [2023-12-27 14:49:54,429][20347] Updated weights for policy 0, policy_version 1325354 (0.0009) [2023-12-27 14:49:54,476][20347] Updated weights for policy 0, policy_version 1325364 (0.0008) [2023-12-27 14:49:54,523][20347] Updated weights for policy 0, policy_version 1325374 (0.0008) [2023-12-27 14:49:54,706][20381] Updated weights for policy 1, policy_version 1324401 (0.0006) [2023-12-27 14:49:54,758][20381] Updated weights for policy 1, policy_version 1324411 (0.0008) [2023-12-27 14:49:54,813][20381] Updated weights for policy 1, policy_version 1324421 (0.0007) [2023-12-27 14:49:54,873][20381] Updated weights for policy 1, policy_version 1324431 (0.0008) [2023-12-27 14:49:55,255][20347] Updated weights for policy 0, policy_version 1325384 (0.0006) [2023-12-27 14:49:55,299][20347] Updated weights for policy 0, policy_version 1325394 (0.0005) [2023-12-27 14:49:55,347][20347] Updated weights for policy 0, policy_version 1325404 (0.0005) [2023-12-27 14:49:55,569][20381] Updated weights for policy 1, policy_version 1324441 (0.0008) [2023-12-27 14:49:55,617][20381] Updated weights for policy 1, policy_version 1324451 (0.0008) [2023-12-27 14:49:55,662][20381] Updated weights for policy 1, policy_version 1324461 (0.0009) [2023-12-27 14:49:55,938][20347] Updated weights for policy 0, policy_version 1325414 (0.0005) [2023-12-27 14:49:55,997][20347] Updated weights for policy 0, policy_version 1325424 (0.0005) [2023-12-27 14:49:56,054][20347] Updated weights for policy 0, policy_version 1325434 (0.0007) [2023-12-27 14:49:56,441][20381] Updated weights for policy 1, policy_version 1324471 (0.0007) [2023-12-27 14:49:56,494][20381] Updated weights for policy 1, policy_version 1324481 (0.0007) [2023-12-27 14:49:56,546][20381] Updated weights for policy 1, policy_version 1324491 (0.0009) [2023-12-27 14:49:56,591][20347] Updated weights for policy 0, policy_version 1325444 (0.0008) [2023-12-27 14:49:56,638][20347] Updated weights for policy 0, policy_version 1325454 (0.0007) [2023-12-27 14:49:56,697][20347] Updated weights for policy 0, policy_version 1325464 (0.0007) [2023-12-27 14:49:57,256][20381] Updated weights for policy 1, policy_version 1324501 (0.0008) [2023-12-27 14:49:57,305][20381] Updated weights for policy 1, policy_version 1324511 (0.0007) [2023-12-27 14:49:57,354][20381] Updated weights for policy 1, policy_version 1324521 (0.0006) [2023-12-27 14:49:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 678486016. Throughput: 0: 10492.7, 1: 10172.0. Samples: 678481940. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:49:57,363][19170] Avg episode reward: [(0, '805.170'), (1, '839.890')] [2023-12-27 14:49:57,392][20347] Updated weights for policy 0, policy_version 1325474 (0.0007) [2023-12-27 14:49:57,445][20347] Updated weights for policy 0, policy_version 1325484 (0.0005) [2023-12-27 14:49:57,492][20347] Updated weights for policy 0, policy_version 1325494 (0.0006) [2023-12-27 14:49:57,540][20347] Updated weights for policy 0, policy_version 1325504 (0.0007) [2023-12-27 14:49:58,058][20381] Updated weights for policy 1, policy_version 1324531 (0.0007) [2023-12-27 14:49:58,105][20381] Updated weights for policy 1, policy_version 1324541 (0.0008) [2023-12-27 14:49:58,160][20381] Updated weights for policy 1, policy_version 1324551 (0.0007) [2023-12-27 14:49:58,203][20347] Updated weights for policy 0, policy_version 1325514 (0.0008) [2023-12-27 14:49:58,263][20347] Updated weights for policy 0, policy_version 1325524 (0.0008) [2023-12-27 14:49:58,321][20347] Updated weights for policy 0, policy_version 1325534 (0.0008) [2023-12-27 14:49:58,994][20381] Updated weights for policy 1, policy_version 1324561 (0.0007) [2023-12-27 14:49:59,053][20381] Updated weights for policy 1, policy_version 1324571 (0.0007) [2023-12-27 14:49:59,106][20347] Updated weights for policy 0, policy_version 1325544 (0.0008) [2023-12-27 14:49:59,117][20381] Updated weights for policy 1, policy_version 1324581 (0.0006) [2023-12-27 14:49:59,159][20347] Updated weights for policy 0, policy_version 1325554 (0.0008) [2023-12-27 14:49:59,186][20381] Updated weights for policy 1, policy_version 1324591 (0.0007) [2023-12-27 14:49:59,224][20347] Updated weights for policy 0, policy_version 1325564 (0.0008) [2023-12-27 14:49:59,861][20381] Updated weights for policy 1, policy_version 1324601 (0.0008) [2023-12-27 14:49:59,919][20381] Updated weights for policy 1, policy_version 1324611 (0.0008) [2023-12-27 14:49:59,958][20347] Updated weights for policy 0, policy_version 1325574 (0.0007) [2023-12-27 14:49:59,980][20381] Updated weights for policy 1, policy_version 1324621 (0.0008) [2023-12-27 14:50:00,021][20347] Updated weights for policy 0, policy_version 1325584 (0.0006) [2023-12-27 14:50:00,075][20347] Updated weights for policy 0, policy_version 1325594 (0.0007) [2023-12-27 14:50:00,639][20381] Updated weights for policy 1, policy_version 1324631 (0.0010) [2023-12-27 14:50:00,693][20381] Updated weights for policy 1, policy_version 1324641 (0.0010) [2023-12-27 14:50:00,709][20347] Updated weights for policy 0, policy_version 1325604 (0.0007) [2023-12-27 14:50:00,746][20381] Updated weights for policy 1, policy_version 1324651 (0.0010) [2023-12-27 14:50:00,755][20347] Updated weights for policy 0, policy_version 1325614 (0.0008) [2023-12-27 14:50:00,803][20347] Updated weights for policy 0, policy_version 1325624 (0.0007) [2023-12-27 14:50:01,420][20381] Updated weights for policy 1, policy_version 1324661 (0.0010) [2023-12-27 14:50:01,479][20381] Updated weights for policy 1, policy_version 1324671 (0.0009) [2023-12-27 14:50:01,515][20347] Updated weights for policy 0, policy_version 1325634 (0.0008) [2023-12-27 14:50:01,535][20381] Updated weights for policy 1, policy_version 1324681 (0.0010) [2023-12-27 14:50:01,567][20347] Updated weights for policy 0, policy_version 1325644 (0.0007) [2023-12-27 14:50:01,621][20347] Updated weights for policy 0, policy_version 1325654 (0.0008) [2023-12-27 14:50:01,691][20347] Updated weights for policy 0, policy_version 1325664 (0.0008) [2023-12-27 14:50:02,207][20381] Updated weights for policy 1, policy_version 1324691 (0.0009) [2023-12-27 14:50:02,262][20381] Updated weights for policy 1, policy_version 1324701 (0.0006) [2023-12-27 14:50:02,320][20381] Updated weights for policy 1, policy_version 1324711 (0.0006) [2023-12-27 14:50:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 678584320. Throughput: 0: 10463.1, 1: 10105.3. Samples: 678601428. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:02,363][19170] Avg episode reward: [(0, '824.490'), (1, '904.590')] [2023-12-27 14:50:02,422][20347] Updated weights for policy 0, policy_version 1325674 (0.0008) [2023-12-27 14:50:02,473][20347] Updated weights for policy 0, policy_version 1325684 (0.0008) [2023-12-27 14:50:02,523][20347] Updated weights for policy 0, policy_version 1325694 (0.0008) [2023-12-27 14:50:02,936][20381] Updated weights for policy 1, policy_version 1324721 (0.0008) [2023-12-27 14:50:02,983][20381] Updated weights for policy 1, policy_version 1324731 (0.0007) [2023-12-27 14:50:03,048][20381] Updated weights for policy 1, policy_version 1324741 (0.0008) [2023-12-27 14:50:03,103][20381] Updated weights for policy 1, policy_version 1324751 (0.0006) [2023-12-27 14:50:03,211][20347] Updated weights for policy 0, policy_version 1325704 (0.0006) [2023-12-27 14:50:03,261][20347] Updated weights for policy 0, policy_version 1325714 (0.0008) [2023-12-27 14:50:03,305][20347] Updated weights for policy 0, policy_version 1325724 (0.0007) [2023-12-27 14:50:03,663][20292] KL-divergence is very high: 127.9613 [2023-12-27 14:50:03,686][20381] Updated weights for policy 1, policy_version 1324761 (0.0008) [2023-12-27 14:50:03,708][20292] KL-divergence is very high: 199.1565 [2023-12-27 14:50:03,742][20381] Updated weights for policy 1, policy_version 1324771 (0.0008) [2023-12-27 14:50:03,803][20381] Updated weights for policy 1, policy_version 1324781 (0.0008) [2023-12-27 14:50:03,975][20347] Updated weights for policy 0, policy_version 1325734 (0.0007) [2023-12-27 14:50:04,032][20347] Updated weights for policy 0, policy_version 1325744 (0.0008) [2023-12-27 14:50:04,092][20347] Updated weights for policy 0, policy_version 1325754 (0.0008) [2023-12-27 14:50:04,559][20381] Updated weights for policy 1, policy_version 1324791 (0.0007) [2023-12-27 14:50:04,620][20381] Updated weights for policy 1, policy_version 1324801 (0.0008) [2023-12-27 14:50:04,666][20381] Updated weights for policy 1, policy_version 1324811 (0.0007) [2023-12-27 14:50:04,711][20347] Updated weights for policy 0, policy_version 1325764 (0.0006) [2023-12-27 14:50:04,759][20347] Updated weights for policy 0, policy_version 1325774 (0.0007) [2023-12-27 14:50:04,807][20347] Updated weights for policy 0, policy_version 1325784 (0.0007) [2023-12-27 14:50:05,383][20381] Updated weights for policy 1, policy_version 1324821 (0.0008) [2023-12-27 14:50:05,437][20381] Updated weights for policy 1, policy_version 1324831 (0.0006) [2023-12-27 14:50:05,442][20347] Updated weights for policy 0, policy_version 1325794 (0.0007) [2023-12-27 14:50:05,483][20381] Updated weights for policy 1, policy_version 1324841 (0.0006) [2023-12-27 14:50:05,496][20347] Updated weights for policy 0, policy_version 1325804 (0.0006) [2023-12-27 14:50:05,555][20347] Updated weights for policy 0, policy_version 1325814 (0.0007) [2023-12-27 14:50:05,616][20347] Updated weights for policy 0, policy_version 1325824 (0.0008) [2023-12-27 14:50:06,162][20381] Updated weights for policy 1, policy_version 1324851 (0.0006) [2023-12-27 14:50:06,223][20381] Updated weights for policy 1, policy_version 1324861 (0.0007) [2023-12-27 14:50:06,280][20347] Updated weights for policy 0, policy_version 1325834 (0.0006) [2023-12-27 14:50:06,281][20381] Updated weights for policy 1, policy_version 1324871 (0.0007) [2023-12-27 14:50:06,343][20347] Updated weights for policy 0, policy_version 1325844 (0.0007) [2023-12-27 14:50:06,399][20347] Updated weights for policy 0, policy_version 1325854 (0.0008) [2023-12-27 14:50:06,985][20381] Updated weights for policy 1, policy_version 1324881 (0.0007) [2023-12-27 14:50:07,031][20381] Updated weights for policy 1, policy_version 1324891 (0.0008) [2023-12-27 14:50:07,091][20381] Updated weights for policy 1, policy_version 1324901 (0.0008) [2023-12-27 14:50:07,109][20347] Updated weights for policy 0, policy_version 1325864 (0.0007) [2023-12-27 14:50:07,143][20381] Updated weights for policy 1, policy_version 1324911 (0.0006) [2023-12-27 14:50:07,163][20347] Updated weights for policy 0, policy_version 1325874 (0.0006) [2023-12-27 14:50:07,228][20347] Updated weights for policy 0, policy_version 1325884 (0.0007) [2023-12-27 14:50:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 678699008. Throughput: 0: 10433.0, 1: 10105.9. Samples: 678663700. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:07,363][19170] Avg episode reward: [(0, '863.540'), (1, '864.650')] [2023-12-27 14:50:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001325888_339492864.pth... [2023-12-27 14:50:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001324912_339206144.pth... [2023-12-27 14:50:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001324672_339181568.pth [2023-12-27 14:50:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001323728_338903040.pth [2023-12-27 14:50:07,741][20381] Updated weights for policy 1, policy_version 1324921 (0.0005) [2023-12-27 14:50:07,794][20381] Updated weights for policy 1, policy_version 1324931 (0.0006) [2023-12-27 14:50:07,853][20381] Updated weights for policy 1, policy_version 1324941 (0.0007) [2023-12-27 14:50:07,939][20347] Updated weights for policy 0, policy_version 1325894 (0.0009) [2023-12-27 14:50:08,005][20347] Updated weights for policy 0, policy_version 1325904 (0.0007) [2023-12-27 14:50:08,062][20347] Updated weights for policy 0, policy_version 1325914 (0.0008) [2023-12-27 14:50:08,475][20381] Updated weights for policy 1, policy_version 1324951 (0.0008) [2023-12-27 14:50:08,534][20381] Updated weights for policy 1, policy_version 1324961 (0.0007) [2023-12-27 14:50:08,595][20381] Updated weights for policy 1, policy_version 1324971 (0.0007) [2023-12-27 14:50:08,757][20347] Updated weights for policy 0, policy_version 1325924 (0.0008) [2023-12-27 14:50:08,814][20347] Updated weights for policy 0, policy_version 1325934 (0.0008) [2023-12-27 14:50:08,826][20291] KL-divergence is very high: 129.9238 [2023-12-27 14:50:08,878][20347] Updated weights for policy 0, policy_version 1325944 (0.0007) [2023-12-27 14:50:09,281][20381] Updated weights for policy 1, policy_version 1324981 (0.0008) [2023-12-27 14:50:09,337][20381] Updated weights for policy 1, policy_version 1324991 (0.0008) [2023-12-27 14:50:09,338][20292] KL-divergence is very high: 204.5253 [2023-12-27 14:50:09,352][20292] KL-divergence is very high: 215.6929 [2023-12-27 14:50:09,366][20292] KL-divergence is very high: 138.5465 [2023-12-27 14:50:09,406][20381] Updated weights for policy 1, policy_version 1325001 (0.0008) [2023-12-27 14:50:09,595][20347] Updated weights for policy 0, policy_version 1325954 (0.0008) [2023-12-27 14:50:09,648][20347] Updated weights for policy 0, policy_version 1325964 (0.0007) [2023-12-27 14:50:09,700][20347] Updated weights for policy 0, policy_version 1325974 (0.0007) [2023-12-27 14:50:09,758][20347] Updated weights for policy 0, policy_version 1325984 (0.0008) [2023-12-27 14:50:10,189][20381] Updated weights for policy 1, policy_version 1325011 (0.0009) [2023-12-27 14:50:10,242][20381] Updated weights for policy 1, policy_version 1325021 (0.0007) [2023-12-27 14:50:10,299][20381] Updated weights for policy 1, policy_version 1325031 (0.0008) [2023-12-27 14:50:10,474][20347] Updated weights for policy 0, policy_version 1325994 (0.0007) [2023-12-27 14:50:10,534][20347] Updated weights for policy 0, policy_version 1326004 (0.0007) [2023-12-27 14:50:10,585][20347] Updated weights for policy 0, policy_version 1326014 (0.0007) [2023-12-27 14:50:11,004][20381] Updated weights for policy 1, policy_version 1325041 (0.0008) [2023-12-27 14:50:11,067][20381] Updated weights for policy 1, policy_version 1325051 (0.0010) [2023-12-27 14:50:11,133][20381] Updated weights for policy 1, policy_version 1325061 (0.0008) [2023-12-27 14:50:11,199][20381] Updated weights for policy 1, policy_version 1325071 (0.0009) [2023-12-27 14:50:11,298][20347] Updated weights for policy 0, policy_version 1326024 (0.0008) [2023-12-27 14:50:11,358][20347] Updated weights for policy 0, policy_version 1326034 (0.0008) [2023-12-27 14:50:11,422][20347] Updated weights for policy 0, policy_version 1326044 (0.0008) [2023-12-27 14:50:11,964][20381] Updated weights for policy 1, policy_version 1325081 (0.0009) [2023-12-27 14:50:12,026][20381] Updated weights for policy 1, policy_version 1325091 (0.0008) [2023-12-27 14:50:12,090][20381] Updated weights for policy 1, policy_version 1325101 (0.0008) [2023-12-27 14:50:12,098][20347] Updated weights for policy 0, policy_version 1326054 (0.0009) [2023-12-27 14:50:12,149][20347] Updated weights for policy 0, policy_version 1326064 (0.0010) [2023-12-27 14:50:12,205][20347] Updated weights for policy 0, policy_version 1326074 (0.0010) [2023-12-27 14:50:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 678797312. Throughput: 0: 10402.2, 1: 10063.7. Samples: 678784620. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:12,363][19170] Avg episode reward: [(0, '834.270'), (1, '844.590')] [2023-12-27 14:50:12,746][20381] Updated weights for policy 1, policy_version 1325111 (0.0008) [2023-12-27 14:50:12,806][20381] Updated weights for policy 1, policy_version 1325121 (0.0008) [2023-12-27 14:50:12,868][20381] Updated weights for policy 1, policy_version 1325131 (0.0008) [2023-12-27 14:50:12,896][20347] Updated weights for policy 0, policy_version 1326084 (0.0008) [2023-12-27 14:50:12,953][20347] Updated weights for policy 0, policy_version 1326094 (0.0005) [2023-12-27 14:50:13,009][20347] Updated weights for policy 0, policy_version 1326104 (0.0008) [2023-12-27 14:50:13,465][20381] Updated weights for policy 1, policy_version 1325141 (0.0007) [2023-12-27 14:50:13,530][20381] Updated weights for policy 1, policy_version 1325151 (0.0006) [2023-12-27 14:50:13,590][20381] Updated weights for policy 1, policy_version 1325161 (0.0009) [2023-12-27 14:50:13,675][20347] Updated weights for policy 0, policy_version 1326114 (0.0007) [2023-12-27 14:50:13,721][20347] Updated weights for policy 0, policy_version 1326124 (0.0006) [2023-12-27 14:50:13,780][20347] Updated weights for policy 0, policy_version 1326134 (0.0008) [2023-12-27 14:50:13,826][20347] Updated weights for policy 0, policy_version 1326144 (0.0009) [2023-12-27 14:50:14,186][20381] Updated weights for policy 1, policy_version 1325171 (0.0008) [2023-12-27 14:50:14,234][20381] Updated weights for policy 1, policy_version 1325181 (0.0008) [2023-12-27 14:50:14,286][20381] Updated weights for policy 1, policy_version 1325191 (0.0007) [2023-12-27 14:50:14,470][20347] Updated weights for policy 0, policy_version 1326154 (0.0008) [2023-12-27 14:50:14,519][20347] Updated weights for policy 0, policy_version 1326164 (0.0007) [2023-12-27 14:50:14,572][20347] Updated weights for policy 0, policy_version 1326174 (0.0007) [2023-12-27 14:50:14,923][20381] Updated weights for policy 1, policy_version 1325201 (0.0008) [2023-12-27 14:50:14,987][20381] Updated weights for policy 1, policy_version 1325211 (0.0008) [2023-12-27 14:50:15,051][20381] Updated weights for policy 1, policy_version 1325221 (0.0008) [2023-12-27 14:50:15,111][20381] Updated weights for policy 1, policy_version 1325231 (0.0008) [2023-12-27 14:50:15,276][20347] Updated weights for policy 0, policy_version 1326184 (0.0010) [2023-12-27 14:50:15,334][20347] Updated weights for policy 0, policy_version 1326194 (0.0006) [2023-12-27 14:50:15,391][20347] Updated weights for policy 0, policy_version 1326204 (0.0005) [2023-12-27 14:50:15,745][20381] Updated weights for policy 1, policy_version 1325241 (0.0006) [2023-12-27 14:50:15,792][20381] Updated weights for policy 1, policy_version 1325251 (0.0005) [2023-12-27 14:50:15,836][20381] Updated weights for policy 1, policy_version 1325261 (0.0005) [2023-12-27 14:50:15,941][20347] Updated weights for policy 0, policy_version 1326214 (0.0007) [2023-12-27 14:50:15,992][20347] Updated weights for policy 0, policy_version 1326224 (0.0008) [2023-12-27 14:50:16,053][20347] Updated weights for policy 0, policy_version 1326234 (0.0007) [2023-12-27 14:50:16,440][20381] Updated weights for policy 1, policy_version 1325271 (0.0007) [2023-12-27 14:50:16,503][20381] Updated weights for policy 1, policy_version 1325281 (0.0008) [2023-12-27 14:50:16,557][20381] Updated weights for policy 1, policy_version 1325291 (0.0005) [2023-12-27 14:50:16,756][20347] Updated weights for policy 0, policy_version 1326244 (0.0007) [2023-12-27 14:50:16,808][20347] Updated weights for policy 0, policy_version 1326254 (0.0008) [2023-12-27 14:50:16,856][20347] Updated weights for policy 0, policy_version 1326264 (0.0007) [2023-12-27 14:50:17,163][20381] Updated weights for policy 1, policy_version 1325301 (0.0007) [2023-12-27 14:50:17,229][20381] Updated weights for policy 1, policy_version 1325311 (0.0008) [2023-12-27 14:50:17,296][20381] Updated weights for policy 1, policy_version 1325321 (0.0008) [2023-12-27 14:50:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 678903808. Throughput: 0: 10327.3, 1: 10144.7. Samples: 678910844. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:17,363][19170] Avg episode reward: [(0, '873.220'), (1, '786.460')] [2023-12-27 14:50:17,501][20347] Updated weights for policy 0, policy_version 1326274 (0.0007) [2023-12-27 14:50:17,551][20347] Updated weights for policy 0, policy_version 1326284 (0.0008) [2023-12-27 14:50:17,599][20347] Updated weights for policy 0, policy_version 1326294 (0.0007) [2023-12-27 14:50:17,654][20347] Updated weights for policy 0, policy_version 1326304 (0.0008) [2023-12-27 14:50:17,986][20381] Updated weights for policy 1, policy_version 1325331 (0.0008) [2023-12-27 14:50:18,046][20381] Updated weights for policy 1, policy_version 1325341 (0.0007) [2023-12-27 14:50:18,106][20381] Updated weights for policy 1, policy_version 1325351 (0.0005) [2023-12-27 14:50:18,364][20347] Updated weights for policy 0, policy_version 1326314 (0.0008) [2023-12-27 14:50:18,416][20347] Updated weights for policy 0, policy_version 1326324 (0.0008) [2023-12-27 14:50:18,466][20347] Updated weights for policy 0, policy_version 1326334 (0.0007) [2023-12-27 14:50:18,754][20381] Updated weights for policy 1, policy_version 1325361 (0.0005) [2023-12-27 14:50:18,814][20381] Updated weights for policy 1, policy_version 1325371 (0.0008) [2023-12-27 14:50:18,877][20381] Updated weights for policy 1, policy_version 1325381 (0.0008) [2023-12-27 14:50:18,927][20381] Updated weights for policy 1, policy_version 1325391 (0.0008) [2023-12-27 14:50:19,140][20347] Updated weights for policy 0, policy_version 1326344 (0.0006) [2023-12-27 14:50:19,195][20347] Updated weights for policy 0, policy_version 1326354 (0.0005) [2023-12-27 14:50:19,260][20347] Updated weights for policy 0, policy_version 1326364 (0.0008) [2023-12-27 14:50:19,667][20381] Updated weights for policy 1, policy_version 1325401 (0.0007) [2023-12-27 14:50:19,723][20381] Updated weights for policy 1, policy_version 1325411 (0.0008) [2023-12-27 14:50:19,784][20381] Updated weights for policy 1, policy_version 1325421 (0.0008) [2023-12-27 14:50:19,965][20347] Updated weights for policy 0, policy_version 1326374 (0.0008) [2023-12-27 14:50:20,019][20347] Updated weights for policy 0, policy_version 1326384 (0.0007) [2023-12-27 14:50:20,074][20347] Updated weights for policy 0, policy_version 1326394 (0.0008) [2023-12-27 14:50:20,514][20381] Updated weights for policy 1, policy_version 1325431 (0.0007) [2023-12-27 14:50:20,575][20381] Updated weights for policy 1, policy_version 1325441 (0.0008) [2023-12-27 14:50:20,643][20381] Updated weights for policy 1, policy_version 1325451 (0.0008) [2023-12-27 14:50:20,790][20347] Updated weights for policy 0, policy_version 1326404 (0.0008) [2023-12-27 14:50:20,855][20347] Updated weights for policy 0, policy_version 1326414 (0.0008) [2023-12-27 14:50:20,918][20347] Updated weights for policy 0, policy_version 1326424 (0.0008) [2023-12-27 14:50:21,340][20381] Updated weights for policy 1, policy_version 1325461 (0.0008) [2023-12-27 14:50:21,413][20381] Updated weights for policy 1, policy_version 1325471 (0.0008) [2023-12-27 14:50:21,469][20381] Updated weights for policy 1, policy_version 1325481 (0.0007) [2023-12-27 14:50:21,727][20347] Updated weights for policy 0, policy_version 1326434 (0.0008) [2023-12-27 14:50:21,793][20347] Updated weights for policy 0, policy_version 1326444 (0.0008) [2023-12-27 14:50:21,844][20347] Updated weights for policy 0, policy_version 1326454 (0.0008) [2023-12-27 14:50:21,891][20347] Updated weights for policy 0, policy_version 1326464 (0.0008) [2023-12-27 14:50:22,228][20381] Updated weights for policy 1, policy_version 1325491 (0.0007) [2023-12-27 14:50:22,295][20381] Updated weights for policy 1, policy_version 1325501 (0.0007) [2023-12-27 14:50:22,363][20381] Updated weights for policy 1, policy_version 1325511 (0.0009) [2023-12-27 14:50:22,363][19170] Fps is (10 sec: 19658.9, 60 sec: 20479.7, 300 sec: 20521.6). Total num frames: 678993920. Throughput: 0: 10296.2, 1: 10109.6. Samples: 678971384. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:22,364][19170] Avg episode reward: [(0, '892.720'), (1, '680.550')] [2023-12-27 14:50:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001326464_339640320.pth... [2023-12-27 14:50:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001325280_339337216.pth [2023-12-27 14:50:22,417][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001325520_339361792.pth... [2023-12-27 14:50:22,422][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001324304_339050496.pth [2023-12-27 14:50:22,623][20347] Updated weights for policy 0, policy_version 1326474 (0.0006) [2023-12-27 14:50:22,679][20347] Updated weights for policy 0, policy_version 1326484 (0.0006) [2023-12-27 14:50:22,740][20347] Updated weights for policy 0, policy_version 1326494 (0.0008) [2023-12-27 14:50:23,125][20381] Updated weights for policy 1, policy_version 1325521 (0.0008) [2023-12-27 14:50:23,181][20381] Updated weights for policy 1, policy_version 1325531 (0.0009) [2023-12-27 14:50:23,243][20381] Updated weights for policy 1, policy_version 1325541 (0.0007) [2023-12-27 14:50:23,290][20347] Updated weights for policy 0, policy_version 1326504 (0.0006) [2023-12-27 14:50:23,296][20381] Updated weights for policy 1, policy_version 1325551 (0.0008) [2023-12-27 14:50:23,345][20347] Updated weights for policy 0, policy_version 1326514 (0.0005) [2023-12-27 14:50:23,405][20347] Updated weights for policy 0, policy_version 1326524 (0.0006) [2023-12-27 14:50:24,013][20347] Updated weights for policy 0, policy_version 1326534 (0.0006) [2023-12-27 14:50:24,038][20381] Updated weights for policy 1, policy_version 1325561 (0.0007) [2023-12-27 14:50:24,062][20347] Updated weights for policy 0, policy_version 1326544 (0.0006) [2023-12-27 14:50:24,090][20381] Updated weights for policy 1, policy_version 1325571 (0.0005) [2023-12-27 14:50:24,115][20347] Updated weights for policy 0, policy_version 1326554 (0.0006) [2023-12-27 14:50:24,146][20381] Updated weights for policy 1, policy_version 1325581 (0.0006) [2023-12-27 14:50:24,798][20347] Updated weights for policy 0, policy_version 1326564 (0.0007) [2023-12-27 14:50:24,821][20381] Updated weights for policy 1, policy_version 1325591 (0.0008) [2023-12-27 14:50:24,850][20347] Updated weights for policy 0, policy_version 1326574 (0.0011) [2023-12-27 14:50:24,874][20381] Updated weights for policy 1, policy_version 1325601 (0.0005) [2023-12-27 14:50:24,906][20347] Updated weights for policy 0, policy_version 1326584 (0.0011) [2023-12-27 14:50:24,923][20381] Updated weights for policy 1, policy_version 1325611 (0.0005) [2023-12-27 14:50:25,499][20347] Updated weights for policy 0, policy_version 1326594 (0.0009) [2023-12-27 14:50:25,551][20347] Updated weights for policy 0, policy_version 1326604 (0.0005) [2023-12-27 14:50:25,566][20381] Updated weights for policy 1, policy_version 1325621 (0.0008) [2023-12-27 14:50:25,605][20347] Updated weights for policy 0, policy_version 1326614 (0.0005) [2023-12-27 14:50:25,628][20381] Updated weights for policy 1, policy_version 1325631 (0.0010) [2023-12-27 14:50:25,671][20347] Updated weights for policy 0, policy_version 1326624 (0.0007) [2023-12-27 14:50:25,689][20381] Updated weights for policy 1, policy_version 1325641 (0.0010) [2023-12-27 14:50:26,310][20347] Updated weights for policy 0, policy_version 1326634 (0.0005) [2023-12-27 14:50:26,373][20347] Updated weights for policy 0, policy_version 1326644 (0.0009) [2023-12-27 14:50:26,387][20381] Updated weights for policy 1, policy_version 1325651 (0.0009) [2023-12-27 14:50:26,423][20347] Updated weights for policy 0, policy_version 1326654 (0.0008) [2023-12-27 14:50:26,450][20381] Updated weights for policy 1, policy_version 1325661 (0.0005) [2023-12-27 14:50:26,508][20381] Updated weights for policy 1, policy_version 1325671 (0.0006) [2023-12-27 14:50:27,106][20347] Updated weights for policy 0, policy_version 1326664 (0.0008) [2023-12-27 14:50:27,143][20381] Updated weights for policy 1, policy_version 1325681 (0.0008) [2023-12-27 14:50:27,153][20347] Updated weights for policy 0, policy_version 1326674 (0.0007) [2023-12-27 14:50:27,194][20381] Updated weights for policy 1, policy_version 1325691 (0.0006) [2023-12-27 14:50:27,207][20347] Updated weights for policy 0, policy_version 1326684 (0.0006) [2023-12-27 14:50:27,244][20381] Updated weights for policy 1, policy_version 1325701 (0.0006) [2023-12-27 14:50:27,295][20381] Updated weights for policy 1, policy_version 1325711 (0.0007) [2023-12-27 14:50:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 679108608. Throughput: 0: 10312.0, 1: 10064.7. Samples: 679092224. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:27,363][19170] Avg episode reward: [(0, '892.860'), (1, '664.890')] [2023-12-27 14:50:27,853][20347] Updated weights for policy 0, policy_version 1326694 (0.0006) [2023-12-27 14:50:27,901][20347] Updated weights for policy 0, policy_version 1326704 (0.0005) [2023-12-27 14:50:27,931][20381] Updated weights for policy 1, policy_version 1325721 (0.0007) [2023-12-27 14:50:27,954][20347] Updated weights for policy 0, policy_version 1326714 (0.0005) [2023-12-27 14:50:27,987][20381] Updated weights for policy 1, policy_version 1325731 (0.0007) [2023-12-27 14:50:28,030][20381] Updated weights for policy 1, policy_version 1325741 (0.0008) [2023-12-27 14:50:28,627][20347] Updated weights for policy 0, policy_version 1326724 (0.0007) [2023-12-27 14:50:28,688][20347] Updated weights for policy 0, policy_version 1326734 (0.0005) [2023-12-27 14:50:28,746][20347] Updated weights for policy 0, policy_version 1326744 (0.0006) [2023-12-27 14:50:28,749][20381] Updated weights for policy 1, policy_version 1325751 (0.0006) [2023-12-27 14:50:28,805][20381] Updated weights for policy 1, policy_version 1325761 (0.0007) [2023-12-27 14:50:28,859][20381] Updated weights for policy 1, policy_version 1325771 (0.0008) [2023-12-27 14:50:29,380][20347] Updated weights for policy 0, policy_version 1326754 (0.0006) [2023-12-27 14:50:29,432][20347] Updated weights for policy 0, policy_version 1326764 (0.0008) [2023-12-27 14:50:29,462][20381] Updated weights for policy 1, policy_version 1325781 (0.0007) [2023-12-27 14:50:29,483][20347] Updated weights for policy 0, policy_version 1326774 (0.0008) [2023-12-27 14:50:29,520][20381] Updated weights for policy 1, policy_version 1325791 (0.0008) [2023-12-27 14:50:29,539][20347] Updated weights for policy 0, policy_version 1326784 (0.0009) [2023-12-27 14:50:29,577][20381] Updated weights for policy 1, policy_version 1325801 (0.0008) [2023-12-27 14:50:30,219][20347] Updated weights for policy 0, policy_version 1326794 (0.0006) [2023-12-27 14:50:30,274][20347] Updated weights for policy 0, policy_version 1326804 (0.0008) [2023-12-27 14:50:30,328][20347] Updated weights for policy 0, policy_version 1326814 (0.0007) [2023-12-27 14:50:30,333][20381] Updated weights for policy 1, policy_version 1325811 (0.0008) [2023-12-27 14:50:30,390][20381] Updated weights for policy 1, policy_version 1325821 (0.0008) [2023-12-27 14:50:30,448][20381] Updated weights for policy 1, policy_version 1325831 (0.0008) [2023-12-27 14:50:30,984][20347] Updated weights for policy 0, policy_version 1326824 (0.0009) [2023-12-27 14:50:31,041][20347] Updated weights for policy 0, policy_version 1326834 (0.0010) [2023-12-27 14:50:31,101][20347] Updated weights for policy 0, policy_version 1326844 (0.0008) [2023-12-27 14:50:31,135][20381] Updated weights for policy 1, policy_version 1325841 (0.0007) [2023-12-27 14:50:31,196][20381] Updated weights for policy 1, policy_version 1325851 (0.0007) [2023-12-27 14:50:31,258][20381] Updated weights for policy 1, policy_version 1325861 (0.0007) [2023-12-27 14:50:31,321][20381] Updated weights for policy 1, policy_version 1325871 (0.0008) [2023-12-27 14:50:31,798][20347] Updated weights for policy 0, policy_version 1326854 (0.0006) [2023-12-27 14:50:31,856][20347] Updated weights for policy 0, policy_version 1326864 (0.0009) [2023-12-27 14:50:31,910][20347] Updated weights for policy 0, policy_version 1326874 (0.0010) [2023-12-27 14:50:32,009][20381] Updated weights for policy 1, policy_version 1325881 (0.0007) [2023-12-27 14:50:32,066][20381] Updated weights for policy 1, policy_version 1325891 (0.0007) [2023-12-27 14:50:32,124][20381] Updated weights for policy 1, policy_version 1325901 (0.0007) [2023-12-27 14:50:32,362][19170] Fps is (10 sec: 21301.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 679206912. Throughput: 0: 10313.2, 1: 10124.5. Samples: 679215824. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:32,363][19170] Avg episode reward: [(0, '853.790'), (1, '723.840')] [2023-12-27 14:50:32,615][20347] Updated weights for policy 0, policy_version 1326884 (0.0010) [2023-12-27 14:50:32,667][20347] Updated weights for policy 0, policy_version 1326894 (0.0010) [2023-12-27 14:50:32,724][20347] Updated weights for policy 0, policy_version 1326904 (0.0010) [2023-12-27 14:50:32,776][20381] Updated weights for policy 1, policy_version 1325911 (0.0006) [2023-12-27 14:50:32,822][20381] Updated weights for policy 1, policy_version 1325921 (0.0005) [2023-12-27 14:50:32,868][20381] Updated weights for policy 1, policy_version 1325931 (0.0005) [2023-12-27 14:50:33,343][20347] Updated weights for policy 0, policy_version 1326914 (0.0010) [2023-12-27 14:50:33,399][20347] Updated weights for policy 0, policy_version 1326924 (0.0008) [2023-12-27 14:50:33,449][20347] Updated weights for policy 0, policy_version 1326934 (0.0006) [2023-12-27 14:50:33,469][20381] Updated weights for policy 1, policy_version 1325941 (0.0006) [2023-12-27 14:50:33,501][20347] Updated weights for policy 0, policy_version 1326944 (0.0005) [2023-12-27 14:50:33,515][20381] Updated weights for policy 1, policy_version 1325951 (0.0006) [2023-12-27 14:50:33,566][20381] Updated weights for policy 1, policy_version 1325961 (0.0008) [2023-12-27 14:50:34,047][20347] Updated weights for policy 0, policy_version 1326954 (0.0005) [2023-12-27 14:50:34,090][20347] Updated weights for policy 0, policy_version 1326964 (0.0005) [2023-12-27 14:50:34,136][20347] Updated weights for policy 0, policy_version 1326974 (0.0005) [2023-12-27 14:50:34,357][20381] Updated weights for policy 1, policy_version 1325971 (0.0009) [2023-12-27 14:50:34,420][20381] Updated weights for policy 1, policy_version 1325981 (0.0008) [2023-12-27 14:50:34,476][20381] Updated weights for policy 1, policy_version 1325991 (0.0008) [2023-12-27 14:50:34,853][20347] Updated weights for policy 0, policy_version 1326984 (0.0008) [2023-12-27 14:50:34,906][20347] Updated weights for policy 0, policy_version 1326994 (0.0007) [2023-12-27 14:50:34,967][20347] Updated weights for policy 0, policy_version 1327004 (0.0007) [2023-12-27 14:50:35,162][20381] Updated weights for policy 1, policy_version 1326001 (0.0008) [2023-12-27 14:50:35,221][20381] Updated weights for policy 1, policy_version 1326011 (0.0006) [2023-12-27 14:50:35,284][20381] Updated weights for policy 1, policy_version 1326021 (0.0011) [2023-12-27 14:50:35,343][20381] Updated weights for policy 1, policy_version 1326031 (0.0011) [2023-12-27 14:50:35,592][20347] Updated weights for policy 0, policy_version 1327014 (0.0006) [2023-12-27 14:50:35,648][20347] Updated weights for policy 0, policy_version 1327024 (0.0005) [2023-12-27 14:50:35,708][20347] Updated weights for policy 0, policy_version 1327034 (0.0005) [2023-12-27 14:50:35,955][20381] Updated weights for policy 1, policy_version 1326041 (0.0006) [2023-12-27 14:50:36,011][20381] Updated weights for policy 1, policy_version 1326051 (0.0005) [2023-12-27 14:50:36,070][20381] Updated weights for policy 1, policy_version 1326061 (0.0005) [2023-12-27 14:50:36,333][20347] Updated weights for policy 0, policy_version 1327044 (0.0007) [2023-12-27 14:50:36,345][20291] KL-divergence is very high: 115.2272 [2023-12-27 14:50:36,393][20347] Updated weights for policy 0, policy_version 1327054 (0.0010) [2023-12-27 14:50:36,455][20347] Updated weights for policy 0, policy_version 1327064 (0.0010) [2023-12-27 14:50:36,662][20381] Updated weights for policy 1, policy_version 1326071 (0.0011) [2023-12-27 14:50:36,724][20381] Updated weights for policy 1, policy_version 1326081 (0.0011) [2023-12-27 14:50:36,776][20381] Updated weights for policy 1, policy_version 1326091 (0.0010) [2023-12-27 14:50:37,061][20347] Updated weights for policy 0, policy_version 1327074 (0.0011) [2023-12-27 14:50:37,122][20347] Updated weights for policy 0, policy_version 1327084 (0.0006) [2023-12-27 14:50:37,175][20347] Updated weights for policy 0, policy_version 1327094 (0.0006) [2023-12-27 14:50:37,223][20347] Updated weights for policy 0, policy_version 1327104 (0.0007) [2023-12-27 14:50:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 679313408. Throughput: 0: 10362.0, 1: 10158.1. Samples: 679279324. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:37,363][19170] Avg episode reward: [(0, '863.360'), (1, '702.930')] [2023-12-27 14:50:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001327104_339804160.pth... [2023-12-27 14:50:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001325888_339492864.pth [2023-12-27 14:50:37,381][20381] Updated weights for policy 1, policy_version 1326101 (0.0008) [2023-12-27 14:50:37,428][20381] Updated weights for policy 1, policy_version 1326111 (0.0005) [2023-12-27 14:50:37,489][20381] Updated weights for policy 1, policy_version 1326121 (0.0010) [2023-12-27 14:50:37,531][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001326128_339517440.pth... [2023-12-27 14:50:37,536][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001324912_339206144.pth [2023-12-27 14:50:37,894][20347] Updated weights for policy 0, policy_version 1327114 (0.0008) [2023-12-27 14:50:37,943][20347] Updated weights for policy 0, policy_version 1327124 (0.0007) [2023-12-27 14:50:37,998][20347] Updated weights for policy 0, policy_version 1327134 (0.0006) [2023-12-27 14:50:38,134][20381] Updated weights for policy 1, policy_version 1326131 (0.0009) [2023-12-27 14:50:38,198][20381] Updated weights for policy 1, policy_version 1326141 (0.0005) [2023-12-27 14:50:38,245][20381] Updated weights for policy 1, policy_version 1326151 (0.0005) [2023-12-27 14:50:38,661][20347] Updated weights for policy 0, policy_version 1327144 (0.0006) [2023-12-27 14:50:38,717][20347] Updated weights for policy 0, policy_version 1327154 (0.0005) [2023-12-27 14:50:38,772][20347] Updated weights for policy 0, policy_version 1327164 (0.0005) [2023-12-27 14:50:38,888][20381] Updated weights for policy 1, policy_version 1326161 (0.0005) [2023-12-27 14:50:38,942][20381] Updated weights for policy 1, policy_version 1326171 (0.0007) [2023-12-27 14:50:38,993][20381] Updated weights for policy 1, policy_version 1326181 (0.0007) [2023-12-27 14:50:39,050][20381] Updated weights for policy 1, policy_version 1326191 (0.0008) [2023-12-27 14:50:39,377][20347] Updated weights for policy 0, policy_version 1327174 (0.0007) [2023-12-27 14:50:39,443][20347] Updated weights for policy 0, policy_version 1327184 (0.0007) [2023-12-27 14:50:39,498][20347] Updated weights for policy 0, policy_version 1327194 (0.0007) [2023-12-27 14:50:39,797][20381] Updated weights for policy 1, policy_version 1326201 (0.0008) [2023-12-27 14:50:39,869][20381] Updated weights for policy 1, policy_version 1326211 (0.0008) [2023-12-27 14:50:39,938][20381] Updated weights for policy 1, policy_version 1326221 (0.0008) [2023-12-27 14:50:40,214][20347] Updated weights for policy 0, policy_version 1327204 (0.0007) [2023-12-27 14:50:40,272][20347] Updated weights for policy 0, policy_version 1327214 (0.0005) [2023-12-27 14:50:40,333][20347] Updated weights for policy 0, policy_version 1327224 (0.0005) [2023-12-27 14:50:40,632][20381] Updated weights for policy 1, policy_version 1326231 (0.0008) [2023-12-27 14:50:40,696][20381] Updated weights for policy 1, policy_version 1326241 (0.0008) [2023-12-27 14:50:40,759][20381] Updated weights for policy 1, policy_version 1326251 (0.0008) [2023-12-27 14:50:40,991][20347] Updated weights for policy 0, policy_version 1327234 (0.0008) [2023-12-27 14:50:41,055][20347] Updated weights for policy 0, policy_version 1327244 (0.0008) [2023-12-27 14:50:41,126][20347] Updated weights for policy 0, policy_version 1327254 (0.0007) [2023-12-27 14:50:41,188][20347] Updated weights for policy 0, policy_version 1327264 (0.0007) [2023-12-27 14:50:41,494][20381] Updated weights for policy 1, policy_version 1326261 (0.0008) [2023-12-27 14:50:41,520][20292] KL-divergence is very high: 221.7694 [2023-12-27 14:50:41,556][20381] Updated weights for policy 1, policy_version 1326271 (0.0008) [2023-12-27 14:50:41,570][20292] KL-divergence is very high: 273.2188 [2023-12-27 14:50:41,620][20292] KL-divergence is very high: 135.7406 [2023-12-27 14:50:41,620][20381] Updated weights for policy 1, policy_version 1326281 (0.0008) [2023-12-27 14:50:41,875][20347] Updated weights for policy 0, policy_version 1327274 (0.0007) [2023-12-27 14:50:41,935][20347] Updated weights for policy 0, policy_version 1327284 (0.0005) [2023-12-27 14:50:41,992][20347] Updated weights for policy 0, policy_version 1327294 (0.0006) [2023-12-27 14:50:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 679411712. Throughput: 0: 10339.5, 1: 10181.4. Samples: 679405380. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:42,363][19170] Avg episode reward: [(0, '873.170'), (1, '723.350')] [2023-12-27 14:50:42,376][20381] Updated weights for policy 1, policy_version 1326291 (0.0013) [2023-12-27 14:50:42,429][20381] Updated weights for policy 1, policy_version 1326301 (0.0007) [2023-12-27 14:50:42,493][20381] Updated weights for policy 1, policy_version 1326311 (0.0011) [2023-12-27 14:50:42,634][20347] Updated weights for policy 0, policy_version 1327304 (0.0007) [2023-12-27 14:50:42,695][20347] Updated weights for policy 0, policy_version 1327314 (0.0008) [2023-12-27 14:50:42,758][20347] Updated weights for policy 0, policy_version 1327324 (0.0007) [2023-12-27 14:50:43,172][20381] Updated weights for policy 1, policy_version 1326321 (0.0010) [2023-12-27 14:50:43,231][20381] Updated weights for policy 1, policy_version 1326331 (0.0005) [2023-12-27 14:50:43,291][20381] Updated weights for policy 1, policy_version 1326341 (0.0005) [2023-12-27 14:50:43,351][20381] Updated weights for policy 1, policy_version 1326351 (0.0007) [2023-12-27 14:50:43,410][20347] Updated weights for policy 0, policy_version 1327334 (0.0006) [2023-12-27 14:50:43,456][20347] Updated weights for policy 0, policy_version 1327344 (0.0005) [2023-12-27 14:50:43,512][20347] Updated weights for policy 0, policy_version 1327354 (0.0009) [2023-12-27 14:50:43,973][20381] Updated weights for policy 1, policy_version 1326361 (0.0010) [2023-12-27 14:50:44,023][20381] Updated weights for policy 1, policy_version 1326371 (0.0008) [2023-12-27 14:50:44,086][20381] Updated weights for policy 1, policy_version 1326381 (0.0010) [2023-12-27 14:50:44,155][20347] Updated weights for policy 0, policy_version 1327364 (0.0008) [2023-12-27 14:50:44,217][20347] Updated weights for policy 0, policy_version 1327374 (0.0010) [2023-12-27 14:50:44,272][20347] Updated weights for policy 0, policy_version 1327384 (0.0010) [2023-12-27 14:50:44,710][20381] Updated weights for policy 1, policy_version 1326391 (0.0007) [2023-12-27 14:50:44,771][20381] Updated weights for policy 1, policy_version 1326401 (0.0007) [2023-12-27 14:50:44,837][20381] Updated weights for policy 1, policy_version 1326411 (0.0011) [2023-12-27 14:50:44,932][20347] Updated weights for policy 0, policy_version 1327394 (0.0010) [2023-12-27 14:50:44,997][20347] Updated weights for policy 0, policy_version 1327404 (0.0011) [2023-12-27 14:50:45,059][20347] Updated weights for policy 0, policy_version 1327414 (0.0011) [2023-12-27 14:50:45,120][20347] Updated weights for policy 0, policy_version 1327424 (0.0008) [2023-12-27 14:50:45,443][20381] Updated weights for policy 1, policy_version 1326421 (0.0009) [2023-12-27 14:50:45,500][20381] Updated weights for policy 1, policy_version 1326431 (0.0006) [2023-12-27 14:50:45,550][20381] Updated weights for policy 1, policy_version 1326441 (0.0006) [2023-12-27 14:50:45,812][20347] Updated weights for policy 0, policy_version 1327434 (0.0011) [2023-12-27 14:50:45,871][20347] Updated weights for policy 0, policy_version 1327444 (0.0011) [2023-12-27 14:50:45,928][20347] Updated weights for policy 0, policy_version 1327454 (0.0011) [2023-12-27 14:50:46,153][20381] Updated weights for policy 1, policy_version 1326451 (0.0005) [2023-12-27 14:50:46,213][20381] Updated weights for policy 1, policy_version 1326461 (0.0005) [2023-12-27 14:50:46,270][20381] Updated weights for policy 1, policy_version 1326471 (0.0007) [2023-12-27 14:50:46,493][20347] Updated weights for policy 0, policy_version 1327464 (0.0005) [2023-12-27 14:50:46,539][20347] Updated weights for policy 0, policy_version 1327474 (0.0005) [2023-12-27 14:50:46,587][20347] Updated weights for policy 0, policy_version 1327484 (0.0010) [2023-12-27 14:50:46,911][20381] Updated weights for policy 1, policy_version 1326481 (0.0008) [2023-12-27 14:50:46,969][20381] Updated weights for policy 1, policy_version 1326491 (0.0010) [2023-12-27 14:50:47,023][20381] Updated weights for policy 1, policy_version 1326501 (0.0010) [2023-12-27 14:50:47,077][20381] Updated weights for policy 1, policy_version 1326511 (0.0009) [2023-12-27 14:50:47,178][20347] Updated weights for policy 0, policy_version 1327494 (0.0010) [2023-12-27 14:50:47,227][20347] Updated weights for policy 0, policy_version 1327504 (0.0009) [2023-12-27 14:50:47,289][20347] Updated weights for policy 0, policy_version 1327514 (0.0011) [2023-12-27 14:50:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 679526400. Throughput: 0: 10411.2, 1: 10257.8. Samples: 679531536. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:47,363][19170] Avg episode reward: [(0, '853.860'), (1, '693.400')] [2023-12-27 14:50:47,735][20381] Updated weights for policy 1, policy_version 1326521 (0.0010) [2023-12-27 14:50:47,791][20381] Updated weights for policy 1, policy_version 1326531 (0.0011) [2023-12-27 14:50:47,847][20381] Updated weights for policy 1, policy_version 1326541 (0.0011) [2023-12-27 14:50:47,954][20347] Updated weights for policy 0, policy_version 1327524 (0.0009) [2023-12-27 14:50:47,998][20347] Updated weights for policy 0, policy_version 1327534 (0.0008) [2023-12-27 14:50:48,042][20347] Updated weights for policy 0, policy_version 1327544 (0.0008) [2023-12-27 14:50:48,548][20381] Updated weights for policy 1, policy_version 1326551 (0.0008) [2023-12-27 14:50:48,610][20381] Updated weights for policy 1, policy_version 1326561 (0.0008) [2023-12-27 14:50:48,671][20381] Updated weights for policy 1, policy_version 1326571 (0.0008) [2023-12-27 14:50:48,742][20347] Updated weights for policy 0, policy_version 1327554 (0.0008) [2023-12-27 14:50:48,801][20347] Updated weights for policy 0, policy_version 1327564 (0.0010) [2023-12-27 14:50:48,856][20347] Updated weights for policy 0, policy_version 1327574 (0.0010) [2023-12-27 14:50:48,915][20347] Updated weights for policy 0, policy_version 1327584 (0.0010) [2023-12-27 14:50:49,395][20381] Updated weights for policy 1, policy_version 1326581 (0.0008) [2023-12-27 14:50:49,442][20381] Updated weights for policy 1, policy_version 1326591 (0.0007) [2023-12-27 14:50:49,492][20381] Updated weights for policy 1, policy_version 1326601 (0.0007) [2023-12-27 14:50:49,620][20347] Updated weights for policy 0, policy_version 1327594 (0.0008) [2023-12-27 14:50:49,674][20347] Updated weights for policy 0, policy_version 1327604 (0.0008) [2023-12-27 14:50:49,736][20347] Updated weights for policy 0, policy_version 1327614 (0.0008) [2023-12-27 14:50:50,254][20381] Updated weights for policy 1, policy_version 1326611 (0.0007) [2023-12-27 14:50:50,314][20381] Updated weights for policy 1, policy_version 1326621 (0.0008) [2023-12-27 14:50:50,364][20381] Updated weights for policy 1, policy_version 1326631 (0.0007) [2023-12-27 14:50:50,472][20347] Updated weights for policy 0, policy_version 1327624 (0.0007) [2023-12-27 14:50:50,532][20347] Updated weights for policy 0, policy_version 1327634 (0.0008) [2023-12-27 14:50:50,600][20347] Updated weights for policy 0, policy_version 1327644 (0.0007) [2023-12-27 14:50:51,030][20381] Updated weights for policy 1, policy_version 1326641 (0.0008) [2023-12-27 14:50:51,098][20381] Updated weights for policy 1, policy_version 1326651 (0.0008) [2023-12-27 14:50:51,166][20381] Updated weights for policy 1, policy_version 1326661 (0.0008) [2023-12-27 14:50:51,224][20381] Updated weights for policy 1, policy_version 1326671 (0.0006) [2023-12-27 14:50:51,350][20347] Updated weights for policy 0, policy_version 1327654 (0.0008) [2023-12-27 14:50:51,417][20347] Updated weights for policy 0, policy_version 1327664 (0.0009) [2023-12-27 14:50:51,471][20347] Updated weights for policy 0, policy_version 1327674 (0.0008) [2023-12-27 14:50:51,956][20381] Updated weights for policy 1, policy_version 1326681 (0.0008) [2023-12-27 14:50:52,025][20381] Updated weights for policy 1, policy_version 1326691 (0.0007) [2023-12-27 14:50:52,086][20381] Updated weights for policy 1, policy_version 1326701 (0.0007) [2023-12-27 14:50:52,186][20347] Updated weights for policy 0, policy_version 1327684 (0.0008) [2023-12-27 14:50:52,254][20347] Updated weights for policy 0, policy_version 1327694 (0.0011) [2023-12-27 14:50:52,313][20347] Updated weights for policy 0, policy_version 1327704 (0.0009) [2023-12-27 14:50:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 679624704. Throughput: 0: 10386.8, 1: 10219.2. Samples: 679590972. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:52,363][19170] Avg episode reward: [(0, '903.450'), (1, '750.960')] [2023-12-27 14:50:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001326704_339664896.pth... [2023-12-27 14:50:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001327712_339959808.pth... [2023-12-27 14:50:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001325520_339361792.pth [2023-12-27 14:50:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001326464_339640320.pth [2023-12-27 14:50:52,763][20381] Updated weights for policy 1, policy_version 1326711 (0.0007) [2023-12-27 14:50:52,817][20381] Updated weights for policy 1, policy_version 1326721 (0.0006) [2023-12-27 14:50:52,885][20381] Updated weights for policy 1, policy_version 1326731 (0.0006) [2023-12-27 14:50:52,999][20347] Updated weights for policy 0, policy_version 1327714 (0.0010) [2023-12-27 14:50:53,050][20347] Updated weights for policy 0, policy_version 1327724 (0.0010) [2023-12-27 14:50:53,103][20347] Updated weights for policy 0, policy_version 1327734 (0.0011) [2023-12-27 14:50:53,156][20347] Updated weights for policy 0, policy_version 1327744 (0.0010) [2023-12-27 14:50:53,476][20381] Updated weights for policy 1, policy_version 1326741 (0.0006) [2023-12-27 14:50:53,534][20381] Updated weights for policy 1, policy_version 1326751 (0.0007) [2023-12-27 14:50:53,588][20381] Updated weights for policy 1, policy_version 1326761 (0.0008) [2023-12-27 14:50:53,852][20347] Updated weights for policy 0, policy_version 1327754 (0.0010) [2023-12-27 14:50:53,916][20347] Updated weights for policy 0, policy_version 1327764 (0.0010) [2023-12-27 14:50:53,978][20347] Updated weights for policy 0, policy_version 1327774 (0.0011) [2023-12-27 14:50:54,286][20381] Updated weights for policy 1, policy_version 1326771 (0.0008) [2023-12-27 14:50:54,345][20381] Updated weights for policy 1, policy_version 1326781 (0.0009) [2023-12-27 14:50:54,402][20381] Updated weights for policy 1, policy_version 1326791 (0.0008) [2023-12-27 14:50:54,609][20347] Updated weights for policy 0, policy_version 1327784 (0.0010) [2023-12-27 14:50:54,654][20347] Updated weights for policy 0, policy_version 1327794 (0.0010) [2023-12-27 14:50:54,697][20347] Updated weights for policy 0, policy_version 1327804 (0.0007) [2023-12-27 14:50:55,017][20381] Updated weights for policy 1, policy_version 1326801 (0.0008) [2023-12-27 14:50:55,076][20381] Updated weights for policy 1, policy_version 1326811 (0.0009) [2023-12-27 14:50:55,122][20381] Updated weights for policy 1, policy_version 1326821 (0.0009) [2023-12-27 14:50:55,171][20381] Updated weights for policy 1, policy_version 1326831 (0.0009) [2023-12-27 14:50:55,292][20347] Updated weights for policy 0, policy_version 1327814 (0.0005) [2023-12-27 14:50:55,346][20347] Updated weights for policy 0, policy_version 1327824 (0.0005) [2023-12-27 14:50:55,395][20347] Updated weights for policy 0, policy_version 1327834 (0.0005) [2023-12-27 14:50:55,929][20381] Updated weights for policy 1, policy_version 1326841 (0.0006) [2023-12-27 14:50:55,972][20347] Updated weights for policy 0, policy_version 1327844 (0.0006) [2023-12-27 14:50:55,981][20381] Updated weights for policy 1, policy_version 1326851 (0.0006) [2023-12-27 14:50:56,030][20381] Updated weights for policy 1, policy_version 1326861 (0.0005) [2023-12-27 14:50:56,031][20347] Updated weights for policy 0, policy_version 1327854 (0.0008) [2023-12-27 14:50:56,080][20347] Updated weights for policy 0, policy_version 1327864 (0.0009) [2023-12-27 14:50:56,589][20381] Updated weights for policy 1, policy_version 1326871 (0.0007) [2023-12-27 14:50:56,652][20381] Updated weights for policy 1, policy_version 1326881 (0.0008) [2023-12-27 14:50:56,716][20381] Updated weights for policy 1, policy_version 1326891 (0.0008) [2023-12-27 14:50:56,776][20347] Updated weights for policy 0, policy_version 1327874 (0.0009) [2023-12-27 14:50:56,833][20347] Updated weights for policy 0, policy_version 1327884 (0.0007) [2023-12-27 14:50:56,886][20347] Updated weights for policy 0, policy_version 1327894 (0.0007) [2023-12-27 14:50:56,935][20347] Updated weights for policy 0, policy_version 1327904 (0.0009) [2023-12-27 14:50:57,320][20381] Updated weights for policy 1, policy_version 1326901 (0.0006) [2023-12-27 14:50:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 679723008. Throughput: 0: 10415.1, 1: 10245.7. Samples: 679714356. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:50:57,363][19170] Avg episode reward: [(0, '855.800'), (1, '752.920')] [2023-12-27 14:50:57,379][20381] Updated weights for policy 1, policy_version 1326911 (0.0010) [2023-12-27 14:50:57,445][20381] Updated weights for policy 1, policy_version 1326921 (0.0008) [2023-12-27 14:50:57,552][20347] Updated weights for policy 0, policy_version 1327914 (0.0008) [2023-12-27 14:50:57,596][20347] Updated weights for policy 0, policy_version 1327924 (0.0008) [2023-12-27 14:50:57,641][20347] Updated weights for policy 0, policy_version 1327934 (0.0008) [2023-12-27 14:50:58,020][20381] Updated weights for policy 1, policy_version 1326931 (0.0005) [2023-12-27 14:50:58,080][20381] Updated weights for policy 1, policy_version 1326941 (0.0006) [2023-12-27 14:50:58,138][20381] Updated weights for policy 1, policy_version 1326951 (0.0007) [2023-12-27 14:50:58,395][20347] Updated weights for policy 0, policy_version 1327944 (0.0008) [2023-12-27 14:50:58,457][20347] Updated weights for policy 0, policy_version 1327954 (0.0008) [2023-12-27 14:50:58,517][20347] Updated weights for policy 0, policy_version 1327964 (0.0008) [2023-12-27 14:50:58,811][20381] Updated weights for policy 1, policy_version 1326961 (0.0006) [2023-12-27 14:50:58,870][20381] Updated weights for policy 1, policy_version 1326971 (0.0010) [2023-12-27 14:50:58,935][20381] Updated weights for policy 1, policy_version 1326981 (0.0010) [2023-12-27 14:50:58,997][20381] Updated weights for policy 1, policy_version 1326991 (0.0008) [2023-12-27 14:50:59,346][20347] Updated weights for policy 0, policy_version 1327974 (0.0008) [2023-12-27 14:50:59,407][20347] Updated weights for policy 0, policy_version 1327984 (0.0006) [2023-12-27 14:50:59,465][20347] Updated weights for policy 0, policy_version 1327994 (0.0007) [2023-12-27 14:50:59,724][20381] Updated weights for policy 1, policy_version 1327001 (0.0008) [2023-12-27 14:50:59,781][20381] Updated weights for policy 1, policy_version 1327011 (0.0007) [2023-12-27 14:50:59,848][20381] Updated weights for policy 1, policy_version 1327021 (0.0007) [2023-12-27 14:51:00,094][20347] Updated weights for policy 0, policy_version 1328004 (0.0007) [2023-12-27 14:51:00,149][20347] Updated weights for policy 0, policy_version 1328014 (0.0007) [2023-12-27 14:51:00,206][20347] Updated weights for policy 0, policy_version 1328024 (0.0007) [2023-12-27 14:51:00,480][20381] Updated weights for policy 1, policy_version 1327031 (0.0007) [2023-12-27 14:51:00,544][20381] Updated weights for policy 1, policy_version 1327041 (0.0008) [2023-12-27 14:51:00,608][20381] Updated weights for policy 1, policy_version 1327051 (0.0008) [2023-12-27 14:51:00,784][20347] Updated weights for policy 0, policy_version 1328034 (0.0006) [2023-12-27 14:51:00,837][20347] Updated weights for policy 0, policy_version 1328044 (0.0005) [2023-12-27 14:51:00,890][20347] Updated weights for policy 0, policy_version 1328054 (0.0007) [2023-12-27 14:51:00,945][20347] Updated weights for policy 0, policy_version 1328064 (0.0008) [2023-12-27 14:51:01,313][20381] Updated weights for policy 1, policy_version 1327061 (0.0008) [2023-12-27 14:51:01,377][20381] Updated weights for policy 1, policy_version 1327071 (0.0008) [2023-12-27 14:51:01,438][20381] Updated weights for policy 1, policy_version 1327081 (0.0008) [2023-12-27 14:51:01,630][20347] Updated weights for policy 0, policy_version 1328074 (0.0007) [2023-12-27 14:51:01,693][20347] Updated weights for policy 0, policy_version 1328084 (0.0008) [2023-12-27 14:51:01,761][20347] Updated weights for policy 0, policy_version 1328094 (0.0007) [2023-12-27 14:51:02,080][20381] Updated weights for policy 1, policy_version 1327091 (0.0006) [2023-12-27 14:51:02,130][20381] Updated weights for policy 1, policy_version 1327101 (0.0008) [2023-12-27 14:51:02,188][20381] Updated weights for policy 1, policy_version 1327111 (0.0007) [2023-12-27 14:51:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 679829504. Throughput: 0: 10395.1, 1: 10214.7. Samples: 679838284. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:02,363][19170] Avg episode reward: [(0, '913.270'), (1, '752.130')] [2023-12-27 14:51:02,419][20347] Updated weights for policy 0, policy_version 1328104 (0.0007) [2023-12-27 14:51:02,476][20347] Updated weights for policy 0, policy_version 1328114 (0.0007) [2023-12-27 14:51:02,528][20347] Updated weights for policy 0, policy_version 1328124 (0.0007) [2023-12-27 14:51:02,870][20381] Updated weights for policy 1, policy_version 1327121 (0.0008) [2023-12-27 14:51:02,919][20381] Updated weights for policy 1, policy_version 1327131 (0.0007) [2023-12-27 14:51:02,971][20381] Updated weights for policy 1, policy_version 1327141 (0.0008) [2023-12-27 14:51:03,020][20381] Updated weights for policy 1, policy_version 1327151 (0.0008) [2023-12-27 14:51:03,135][20347] Updated weights for policy 0, policy_version 1328134 (0.0006) [2023-12-27 14:51:03,184][20347] Updated weights for policy 0, policy_version 1328144 (0.0005) [2023-12-27 14:51:03,233][20347] Updated weights for policy 0, policy_version 1328154 (0.0005) [2023-12-27 14:51:03,750][20381] Updated weights for policy 1, policy_version 1327161 (0.0007) [2023-12-27 14:51:03,807][20381] Updated weights for policy 1, policy_version 1327171 (0.0007) [2023-12-27 14:51:03,875][20381] Updated weights for policy 1, policy_version 1327181 (0.0006) [2023-12-27 14:51:03,878][20347] Updated weights for policy 0, policy_version 1328164 (0.0007) [2023-12-27 14:51:03,939][20347] Updated weights for policy 0, policy_version 1328174 (0.0006) [2023-12-27 14:51:04,003][20347] Updated weights for policy 0, policy_version 1328184 (0.0007) [2023-12-27 14:51:04,547][20381] Updated weights for policy 1, policy_version 1327191 (0.0010) [2023-12-27 14:51:04,600][20381] Updated weights for policy 1, policy_version 1327201 (0.0010) [2023-12-27 14:51:04,644][20347] Updated weights for policy 0, policy_version 1328194 (0.0009) [2023-12-27 14:51:04,665][20381] Updated weights for policy 1, policy_version 1327211 (0.0009) [2023-12-27 14:51:04,710][20347] Updated weights for policy 0, policy_version 1328204 (0.0008) [2023-12-27 14:51:04,771][20347] Updated weights for policy 0, policy_version 1328214 (0.0010) [2023-12-27 14:51:04,829][20347] Updated weights for policy 0, policy_version 1328224 (0.0010) [2023-12-27 14:51:05,336][20381] Updated weights for policy 1, policy_version 1327221 (0.0008) [2023-12-27 14:51:05,391][20381] Updated weights for policy 1, policy_version 1327231 (0.0010) [2023-12-27 14:51:05,435][20347] Updated weights for policy 0, policy_version 1328234 (0.0009) [2023-12-27 14:51:05,449][20381] Updated weights for policy 1, policy_version 1327241 (0.0010) [2023-12-27 14:51:05,488][20347] Updated weights for policy 0, policy_version 1328244 (0.0007) [2023-12-27 14:51:05,541][20347] Updated weights for policy 0, policy_version 1328254 (0.0007) [2023-12-27 14:51:06,128][20381] Updated weights for policy 1, policy_version 1327251 (0.0010) [2023-12-27 14:51:06,190][20381] Updated weights for policy 1, policy_version 1327261 (0.0010) [2023-12-27 14:51:06,208][20347] Updated weights for policy 0, policy_version 1328264 (0.0010) [2023-12-27 14:51:06,246][20381] Updated weights for policy 1, policy_version 1327271 (0.0008) [2023-12-27 14:51:06,268][20347] Updated weights for policy 0, policy_version 1328274 (0.0011) [2023-12-27 14:51:06,327][20347] Updated weights for policy 0, policy_version 1328284 (0.0010) [2023-12-27 14:51:06,928][20381] Updated weights for policy 1, policy_version 1327281 (0.0008) [2023-12-27 14:51:06,991][20381] Updated weights for policy 1, policy_version 1327291 (0.0006) [2023-12-27 14:51:07,009][20347] Updated weights for policy 0, policy_version 1328294 (0.0010) [2023-12-27 14:51:07,047][20381] Updated weights for policy 1, policy_version 1327301 (0.0009) [2023-12-27 14:51:07,066][20347] Updated weights for policy 0, policy_version 1328304 (0.0011) [2023-12-27 14:51:07,098][20381] Updated weights for policy 1, policy_version 1327311 (0.0011) [2023-12-27 14:51:07,123][20347] Updated weights for policy 0, policy_version 1328314 (0.0011) [2023-12-27 14:51:07,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 679936000. Throughput: 0: 10436.9, 1: 10228.7. Samples: 679901324. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:07,364][19170] Avg episode reward: [(0, '931.760'), (1, '798.600')] [2023-12-27 14:51:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001328320_340115456.pth... [2023-12-27 14:51:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001327312_339820544.pth... [2023-12-27 14:51:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001327104_339804160.pth [2023-12-27 14:51:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001326128_339517440.pth [2023-12-27 14:51:07,722][20347] Updated weights for policy 0, policy_version 1328324 (0.0011) [2023-12-27 14:51:07,760][20381] Updated weights for policy 1, policy_version 1327321 (0.0007) [2023-12-27 14:51:07,773][20347] Updated weights for policy 0, policy_version 1328334 (0.0010) [2023-12-27 14:51:07,807][20381] Updated weights for policy 1, policy_version 1327331 (0.0007) [2023-12-27 14:51:07,825][20347] Updated weights for policy 0, policy_version 1328344 (0.0010) [2023-12-27 14:51:07,856][20381] Updated weights for policy 1, policy_version 1327341 (0.0005) [2023-12-27 14:51:08,480][20347] Updated weights for policy 0, policy_version 1328354 (0.0008) [2023-12-27 14:51:08,537][20347] Updated weights for policy 0, policy_version 1328364 (0.0008) [2023-12-27 14:51:08,563][20381] Updated weights for policy 1, policy_version 1327351 (0.0009) [2023-12-27 14:51:08,599][20347] Updated weights for policy 0, policy_version 1328374 (0.0007) [2023-12-27 14:51:08,620][20381] Updated weights for policy 1, policy_version 1327361 (0.0006) [2023-12-27 14:51:08,655][20347] Updated weights for policy 0, policy_version 1328384 (0.0006) [2023-12-27 14:51:08,677][20381] Updated weights for policy 1, policy_version 1327371 (0.0006) [2023-12-27 14:51:09,349][20381] Updated weights for policy 1, policy_version 1327381 (0.0008) [2023-12-27 14:51:09,365][20347] Updated weights for policy 0, policy_version 1328394 (0.0008) [2023-12-27 14:51:09,417][20381] Updated weights for policy 1, policy_version 1327391 (0.0008) [2023-12-27 14:51:09,433][20347] Updated weights for policy 0, policy_version 1328404 (0.0008) [2023-12-27 14:51:09,480][20381] Updated weights for policy 1, policy_version 1327401 (0.0008) [2023-12-27 14:51:09,491][20347] Updated weights for policy 0, policy_version 1328414 (0.0011) [2023-12-27 14:51:10,204][20381] Updated weights for policy 1, policy_version 1327411 (0.0007) [2023-12-27 14:51:10,222][20347] Updated weights for policy 0, policy_version 1328424 (0.0009) [2023-12-27 14:51:10,267][20381] Updated weights for policy 1, policy_version 1327421 (0.0006) [2023-12-27 14:51:10,284][20347] Updated weights for policy 0, policy_version 1328434 (0.0009) [2023-12-27 14:51:10,327][20381] Updated weights for policy 1, policy_version 1327431 (0.0010) [2023-12-27 14:51:10,343][20347] Updated weights for policy 0, policy_version 1328444 (0.0010) [2023-12-27 14:51:10,907][20381] Updated weights for policy 1, policy_version 1327441 (0.0008) [2023-12-27 14:51:10,959][20381] Updated weights for policy 1, policy_version 1327451 (0.0006) [2023-12-27 14:51:11,015][20381] Updated weights for policy 1, policy_version 1327461 (0.0010) [2023-12-27 14:51:11,024][20347] Updated weights for policy 0, policy_version 1328454 (0.0010) [2023-12-27 14:51:11,087][20347] Updated weights for policy 0, policy_version 1328464 (0.0010) [2023-12-27 14:51:11,091][20381] Updated weights for policy 1, policy_version 1327471 (0.0008) [2023-12-27 14:51:11,150][20347] Updated weights for policy 0, policy_version 1328474 (0.0010) [2023-12-27 14:51:11,874][20347] Updated weights for policy 0, policy_version 1328484 (0.0008) [2023-12-27 14:51:11,917][20381] Updated weights for policy 1, policy_version 1327481 (0.0007) [2023-12-27 14:51:11,925][20347] Updated weights for policy 0, policy_version 1328494 (0.0010) [2023-12-27 14:51:11,976][20381] Updated weights for policy 1, policy_version 1327491 (0.0007) [2023-12-27 14:51:11,990][20347] Updated weights for policy 0, policy_version 1328504 (0.0008) [2023-12-27 14:51:12,037][20381] Updated weights for policy 1, policy_version 1327501 (0.0007) [2023-12-27 14:51:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 680034304. Throughput: 0: 10423.7, 1: 10285.1. Samples: 680024120. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:12,363][19170] Avg episode reward: [(0, '883.150'), (1, '874.710')] [2023-12-27 14:51:12,687][20347] Updated weights for policy 0, policy_version 1328514 (0.0006) [2023-12-27 14:51:12,747][20347] Updated weights for policy 0, policy_version 1328524 (0.0006) [2023-12-27 14:51:12,760][20381] Updated weights for policy 1, policy_version 1327511 (0.0006) [2023-12-27 14:51:12,806][20291] KL-divergence is very high: 125.9103 [2023-12-27 14:51:12,811][20347] Updated weights for policy 0, policy_version 1328534 (0.0007) [2023-12-27 14:51:12,816][20381] Updated weights for policy 1, policy_version 1327521 (0.0008) [2023-12-27 14:51:12,868][20347] Updated weights for policy 0, policy_version 1328544 (0.0005) [2023-12-27 14:51:12,873][20381] Updated weights for policy 1, policy_version 1327531 (0.0006) [2023-12-27 14:51:13,527][20347] Updated weights for policy 0, policy_version 1328554 (0.0007) [2023-12-27 14:51:13,555][20381] Updated weights for policy 1, policy_version 1327541 (0.0008) [2023-12-27 14:51:13,579][20347] Updated weights for policy 0, policy_version 1328564 (0.0006) [2023-12-27 14:51:13,607][20381] Updated weights for policy 1, policy_version 1327551 (0.0006) [2023-12-27 14:51:13,627][20347] Updated weights for policy 0, policy_version 1328574 (0.0006) [2023-12-27 14:51:13,660][20381] Updated weights for policy 1, policy_version 1327561 (0.0006) [2023-12-27 14:51:14,278][20381] Updated weights for policy 1, policy_version 1327571 (0.0008) [2023-12-27 14:51:14,328][20381] Updated weights for policy 1, policy_version 1327581 (0.0008) [2023-12-27 14:51:14,333][20347] Updated weights for policy 0, policy_version 1328584 (0.0005) [2023-12-27 14:51:14,384][20381] Updated weights for policy 1, policy_version 1327591 (0.0008) [2023-12-27 14:51:14,386][20347] Updated weights for policy 0, policy_version 1328594 (0.0005) [2023-12-27 14:51:14,439][20347] Updated weights for policy 0, policy_version 1328604 (0.0005) [2023-12-27 14:51:15,064][20381] Updated weights for policy 1, policy_version 1327601 (0.0008) [2023-12-27 14:51:15,076][20347] Updated weights for policy 0, policy_version 1328614 (0.0007) [2023-12-27 14:51:15,122][20381] Updated weights for policy 1, policy_version 1327611 (0.0007) [2023-12-27 14:51:15,135][20347] Updated weights for policy 0, policy_version 1328624 (0.0007) [2023-12-27 14:51:15,178][20381] Updated weights for policy 1, policy_version 1327621 (0.0005) [2023-12-27 14:51:15,198][20347] Updated weights for policy 0, policy_version 1328634 (0.0008) [2023-12-27 14:51:15,241][20381] Updated weights for policy 1, policy_version 1327631 (0.0006) [2023-12-27 14:51:15,834][20347] Updated weights for policy 0, policy_version 1328644 (0.0008) [2023-12-27 14:51:15,866][20381] Updated weights for policy 1, policy_version 1327641 (0.0005) [2023-12-27 14:51:15,890][20347] Updated weights for policy 0, policy_version 1328654 (0.0007) [2023-12-27 14:51:15,929][20381] Updated weights for policy 1, policy_version 1327651 (0.0006) [2023-12-27 14:51:15,944][20347] Updated weights for policy 0, policy_version 1328664 (0.0007) [2023-12-27 14:51:15,986][20381] Updated weights for policy 1, policy_version 1327661 (0.0005) [2023-12-27 14:51:16,591][20381] Updated weights for policy 1, policy_version 1327671 (0.0007) [2023-12-27 14:51:16,594][20347] Updated weights for policy 0, policy_version 1328674 (0.0006) [2023-12-27 14:51:16,649][20347] Updated weights for policy 0, policy_version 1328684 (0.0008) [2023-12-27 14:51:16,651][20381] Updated weights for policy 1, policy_version 1327681 (0.0007) [2023-12-27 14:51:16,698][20347] Updated weights for policy 0, policy_version 1328694 (0.0005) [2023-12-27 14:51:16,713][20381] Updated weights for policy 1, policy_version 1327691 (0.0008) [2023-12-27 14:51:16,757][20347] Updated weights for policy 0, policy_version 1328704 (0.0007) [2023-12-27 14:51:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 680132608. Throughput: 0: 10396.1, 1: 10292.3. Samples: 680146800. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:17,363][19170] Avg episode reward: [(0, '844.250'), (1, '808.940')] [2023-12-27 14:51:17,384][20381] Updated weights for policy 1, policy_version 1327701 (0.0009) [2023-12-27 14:51:17,438][20381] Updated weights for policy 1, policy_version 1327711 (0.0010) [2023-12-27 14:51:17,468][20347] Updated weights for policy 0, policy_version 1328714 (0.0006) [2023-12-27 14:51:17,522][20381] Updated weights for policy 1, policy_version 1327721 (0.0011) [2023-12-27 14:51:17,529][20347] Updated weights for policy 0, policy_version 1328724 (0.0007) [2023-12-27 14:51:17,594][20347] Updated weights for policy 0, policy_version 1328734 (0.0009) [2023-12-27 14:51:18,211][20381] Updated weights for policy 1, policy_version 1327731 (0.0009) [2023-12-27 14:51:18,261][20381] Updated weights for policy 1, policy_version 1327741 (0.0007) [2023-12-27 14:51:18,311][20381] Updated weights for policy 1, policy_version 1327751 (0.0007) [2023-12-27 14:51:18,320][20347] Updated weights for policy 0, policy_version 1328744 (0.0007) [2023-12-27 14:51:18,385][20347] Updated weights for policy 0, policy_version 1328754 (0.0009) [2023-12-27 14:51:18,451][20347] Updated weights for policy 0, policy_version 1328764 (0.0008) [2023-12-27 14:51:18,922][20381] Updated weights for policy 1, policy_version 1327761 (0.0009) [2023-12-27 14:51:18,977][20381] Updated weights for policy 1, policy_version 1327771 (0.0008) [2023-12-27 14:51:19,023][20381] Updated weights for policy 1, policy_version 1327781 (0.0007) [2023-12-27 14:51:19,073][20381] Updated weights for policy 1, policy_version 1327791 (0.0006) [2023-12-27 14:51:19,144][20347] Updated weights for policy 0, policy_version 1328774 (0.0010) [2023-12-27 14:51:19,212][20347] Updated weights for policy 0, policy_version 1328784 (0.0010) [2023-12-27 14:51:19,273][20347] Updated weights for policy 0, policy_version 1328794 (0.0009) [2023-12-27 14:51:19,764][20381] Updated weights for policy 1, policy_version 1327801 (0.0006) [2023-12-27 14:51:19,833][20381] Updated weights for policy 1, policy_version 1327811 (0.0006) [2023-12-27 14:51:19,897][20381] Updated weights for policy 1, policy_version 1327821 (0.0006) [2023-12-27 14:51:19,952][20347] Updated weights for policy 0, policy_version 1328804 (0.0009) [2023-12-27 14:51:20,010][20347] Updated weights for policy 0, policy_version 1328814 (0.0009) [2023-12-27 14:51:20,071][20347] Updated weights for policy 0, policy_version 1328824 (0.0008) [2023-12-27 14:51:20,499][20381] Updated weights for policy 1, policy_version 1327831 (0.0008) [2023-12-27 14:51:20,552][20381] Updated weights for policy 1, policy_version 1327841 (0.0007) [2023-12-27 14:51:20,611][20381] Updated weights for policy 1, policy_version 1327851 (0.0008) [2023-12-27 14:51:20,851][20347] Updated weights for policy 0, policy_version 1328834 (0.0008) [2023-12-27 14:51:20,910][20347] Updated weights for policy 0, policy_version 1328844 (0.0008) [2023-12-27 14:51:20,963][20347] Updated weights for policy 0, policy_version 1328854 (0.0008) [2023-12-27 14:51:21,018][20347] Updated weights for policy 0, policy_version 1328864 (0.0007) [2023-12-27 14:51:21,348][20381] Updated weights for policy 1, policy_version 1327861 (0.0008) [2023-12-27 14:51:21,407][20381] Updated weights for policy 1, policy_version 1327871 (0.0008) [2023-12-27 14:51:21,468][20381] Updated weights for policy 1, policy_version 1327881 (0.0007) [2023-12-27 14:51:21,791][20347] Updated weights for policy 0, policy_version 1328874 (0.0009) [2023-12-27 14:51:21,838][20347] Updated weights for policy 0, policy_version 1328884 (0.0008) [2023-12-27 14:51:21,894][20347] Updated weights for policy 0, policy_version 1328894 (0.0009) [2023-12-27 14:51:22,116][20381] Updated weights for policy 1, policy_version 1327891 (0.0007) [2023-12-27 14:51:22,179][20381] Updated weights for policy 1, policy_version 1327901 (0.0008) [2023-12-27 14:51:22,240][20381] Updated weights for policy 1, policy_version 1327911 (0.0008) [2023-12-27 14:51:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.3, 300 sec: 20549.4). Total num frames: 680239104. Throughput: 0: 10332.8, 1: 10300.2. Samples: 680207808. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:22,363][19170] Avg episode reward: [(0, '863.520'), (1, '820.640')] [2023-12-27 14:51:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001327920_339976192.pth... [2023-12-27 14:51:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001328896_340262912.pth... [2023-12-27 14:51:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001326704_339664896.pth [2023-12-27 14:51:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001327712_339959808.pth [2023-12-27 14:51:22,626][20347] Updated weights for policy 0, policy_version 1328904 (0.0007) [2023-12-27 14:51:22,684][20347] Updated weights for policy 0, policy_version 1328914 (0.0008) [2023-12-27 14:51:22,756][20347] Updated weights for policy 0, policy_version 1328924 (0.0008) [2023-12-27 14:51:22,990][20381] Updated weights for policy 1, policy_version 1327921 (0.0008) [2023-12-27 14:51:23,045][20381] Updated weights for policy 1, policy_version 1327931 (0.0011) [2023-12-27 14:51:23,099][20381] Updated weights for policy 1, policy_version 1327941 (0.0010) [2023-12-27 14:51:23,161][20381] Updated weights for policy 1, policy_version 1327951 (0.0009) [2023-12-27 14:51:23,403][20347] Updated weights for policy 0, policy_version 1328934 (0.0008) [2023-12-27 14:51:23,458][20347] Updated weights for policy 0, policy_version 1328944 (0.0007) [2023-12-27 14:51:23,503][20347] Updated weights for policy 0, policy_version 1328954 (0.0008) [2023-12-27 14:51:23,732][20381] Updated weights for policy 1, policy_version 1327961 (0.0009) [2023-12-27 14:51:23,784][20381] Updated weights for policy 1, policy_version 1327971 (0.0010) [2023-12-27 14:51:23,840][20381] Updated weights for policy 1, policy_version 1327981 (0.0010) [2023-12-27 14:51:24,175][20347] Updated weights for policy 0, policy_version 1328964 (0.0007) [2023-12-27 14:51:24,238][20347] Updated weights for policy 0, policy_version 1328974 (0.0007) [2023-12-27 14:51:24,300][20347] Updated weights for policy 0, policy_version 1328984 (0.0007) [2023-12-27 14:51:24,514][20381] Updated weights for policy 1, policy_version 1327991 (0.0009) [2023-12-27 14:51:24,579][20381] Updated weights for policy 1, policy_version 1328001 (0.0007) [2023-12-27 14:51:24,632][20381] Updated weights for policy 1, policy_version 1328011 (0.0008) [2023-12-27 14:51:25,001][20347] Updated weights for policy 0, policy_version 1328994 (0.0008) [2023-12-27 14:51:25,059][20347] Updated weights for policy 0, policy_version 1329004 (0.0007) [2023-12-27 14:51:25,118][20347] Updated weights for policy 0, policy_version 1329014 (0.0007) [2023-12-27 14:51:25,182][20347] Updated weights for policy 0, policy_version 1329024 (0.0007) [2023-12-27 14:51:25,295][20381] Updated weights for policy 1, policy_version 1328021 (0.0007) [2023-12-27 14:51:25,347][20381] Updated weights for policy 1, policy_version 1328031 (0.0008) [2023-12-27 14:51:25,400][20381] Updated weights for policy 1, policy_version 1328041 (0.0008) [2023-12-27 14:51:25,809][20347] Updated weights for policy 0, policy_version 1329034 (0.0007) [2023-12-27 14:51:25,860][20347] Updated weights for policy 0, policy_version 1329044 (0.0007) [2023-12-27 14:51:25,912][20347] Updated weights for policy 0, policy_version 1329054 (0.0008) [2023-12-27 14:51:26,076][20381] Updated weights for policy 1, policy_version 1328051 (0.0007) [2023-12-27 14:51:26,124][20381] Updated weights for policy 1, policy_version 1328061 (0.0005) [2023-12-27 14:51:26,174][20381] Updated weights for policy 1, policy_version 1328071 (0.0005) [2023-12-27 14:51:26,569][20347] Updated weights for policy 0, policy_version 1329064 (0.0010) [2023-12-27 14:51:26,617][20347] Updated weights for policy 0, policy_version 1329074 (0.0010) [2023-12-27 14:51:26,664][20347] Updated weights for policy 0, policy_version 1329084 (0.0010) [2023-12-27 14:51:26,857][20381] Updated weights for policy 1, policy_version 1328081 (0.0006) [2023-12-27 14:51:26,907][20381] Updated weights for policy 1, policy_version 1328091 (0.0007) [2023-12-27 14:51:26,951][20381] Updated weights for policy 1, policy_version 1328101 (0.0007) [2023-12-27 14:51:27,002][20381] Updated weights for policy 1, policy_version 1328111 (0.0006) [2023-12-27 14:51:27,334][20347] Updated weights for policy 0, policy_version 1329094 (0.0010) [2023-12-27 14:51:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 680337408. Throughput: 0: 10231.6, 1: 10281.3. Samples: 680328456. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:27,363][19170] Avg episode reward: [(0, '863.500'), (1, '904.560')] [2023-12-27 14:51:27,389][20347] Updated weights for policy 0, policy_version 1329104 (0.0010) [2023-12-27 14:51:27,449][20347] Updated weights for policy 0, policy_version 1329114 (0.0010) [2023-12-27 14:51:27,680][20381] Updated weights for policy 1, policy_version 1328121 (0.0007) [2023-12-27 14:51:27,734][20381] Updated weights for policy 1, policy_version 1328131 (0.0007) [2023-12-27 14:51:27,782][20381] Updated weights for policy 1, policy_version 1328141 (0.0007) [2023-12-27 14:51:28,119][20347] Updated weights for policy 0, policy_version 1329124 (0.0010) [2023-12-27 14:51:28,173][20347] Updated weights for policy 0, policy_version 1329134 (0.0010) [2023-12-27 14:51:28,224][20347] Updated weights for policy 0, policy_version 1329144 (0.0007) [2023-12-27 14:51:28,420][20381] Updated weights for policy 1, policy_version 1328151 (0.0008) [2023-12-27 14:51:28,485][20381] Updated weights for policy 1, policy_version 1328161 (0.0008) [2023-12-27 14:51:28,550][20381] Updated weights for policy 1, policy_version 1328171 (0.0008) [2023-12-27 14:51:28,903][20347] Updated weights for policy 0, policy_version 1329154 (0.0010) [2023-12-27 14:51:28,955][20347] Updated weights for policy 0, policy_version 1329164 (0.0010) [2023-12-27 14:51:29,005][20347] Updated weights for policy 0, policy_version 1329174 (0.0010) [2023-12-27 14:51:29,058][20347] Updated weights for policy 0, policy_version 1329184 (0.0005) [2023-12-27 14:51:29,279][20381] Updated weights for policy 1, policy_version 1328181 (0.0008) [2023-12-27 14:51:29,344][20381] Updated weights for policy 1, policy_version 1328191 (0.0008) [2023-12-27 14:51:29,404][20381] Updated weights for policy 1, policy_version 1328201 (0.0009) [2023-12-27 14:51:29,719][20347] Updated weights for policy 0, policy_version 1329194 (0.0010) [2023-12-27 14:51:29,774][20347] Updated weights for policy 0, policy_version 1329204 (0.0010) [2023-12-27 14:51:29,829][20347] Updated weights for policy 0, policy_version 1329214 (0.0012) [2023-12-27 14:51:30,093][20381] Updated weights for policy 1, policy_version 1328211 (0.0006) [2023-12-27 14:51:30,150][20381] Updated weights for policy 1, policy_version 1328221 (0.0005) [2023-12-27 14:51:30,205][20381] Updated weights for policy 1, policy_version 1328231 (0.0005) [2023-12-27 14:51:30,483][20347] Updated weights for policy 0, policy_version 1329224 (0.0009) [2023-12-27 14:51:30,541][20347] Updated weights for policy 0, policy_version 1329234 (0.0010) [2023-12-27 14:51:30,588][20347] Updated weights for policy 0, policy_version 1329244 (0.0006) [2023-12-27 14:51:30,742][20381] Updated weights for policy 1, policy_version 1328241 (0.0005) [2023-12-27 14:51:30,790][20381] Updated weights for policy 1, policy_version 1328251 (0.0007) [2023-12-27 14:51:30,848][20381] Updated weights for policy 1, policy_version 1328261 (0.0007) [2023-12-27 14:51:30,902][20381] Updated weights for policy 1, policy_version 1328271 (0.0007) [2023-12-27 14:51:31,264][20347] Updated weights for policy 0, policy_version 1329254 (0.0008) [2023-12-27 14:51:31,325][20347] Updated weights for policy 0, policy_version 1329264 (0.0010) [2023-12-27 14:51:31,389][20347] Updated weights for policy 0, policy_version 1329274 (0.0011) [2023-12-27 14:51:31,646][20381] Updated weights for policy 1, policy_version 1328281 (0.0008) [2023-12-27 14:51:31,703][20381] Updated weights for policy 1, policy_version 1328291 (0.0007) [2023-12-27 14:51:31,762][20381] Updated weights for policy 1, policy_version 1328301 (0.0007) [2023-12-27 14:51:32,066][20347] Updated weights for policy 0, policy_version 1329284 (0.0009) [2023-12-27 14:51:32,121][20347] Updated weights for policy 0, policy_version 1329294 (0.0010) [2023-12-27 14:51:32,180][20347] Updated weights for policy 0, policy_version 1329304 (0.0009) [2023-12-27 14:51:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 680443904. Throughput: 0: 10224.1, 1: 10269.1. Samples: 680453728. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:32,363][19170] Avg episode reward: [(0, '873.360'), (1, '817.770')] [2023-12-27 14:51:32,438][20381] Updated weights for policy 1, policy_version 1328311 (0.0008) [2023-12-27 14:51:32,495][20381] Updated weights for policy 1, policy_version 1328321 (0.0009) [2023-12-27 14:51:32,555][20381] Updated weights for policy 1, policy_version 1328331 (0.0011) [2023-12-27 14:51:32,858][20347] Updated weights for policy 0, policy_version 1329314 (0.0008) [2023-12-27 14:51:32,916][20347] Updated weights for policy 0, policy_version 1329324 (0.0007) [2023-12-27 14:51:32,977][20347] Updated weights for policy 0, policy_version 1329334 (0.0006) [2023-12-27 14:51:33,036][20347] Updated weights for policy 0, policy_version 1329344 (0.0007) [2023-12-27 14:51:33,173][20381] Updated weights for policy 1, policy_version 1328341 (0.0010) [2023-12-27 14:51:33,230][20381] Updated weights for policy 1, policy_version 1328351 (0.0011) [2023-12-27 14:51:33,291][20381] Updated weights for policy 1, policy_version 1328361 (0.0011) [2023-12-27 14:51:33,564][20347] Updated weights for policy 0, policy_version 1329354 (0.0005) [2023-12-27 14:51:33,615][20347] Updated weights for policy 0, policy_version 1329364 (0.0005) [2023-12-27 14:51:33,674][20347] Updated weights for policy 0, policy_version 1329374 (0.0005) [2023-12-27 14:51:33,870][20381] Updated weights for policy 1, policy_version 1328371 (0.0011) [2023-12-27 14:51:33,932][20381] Updated weights for policy 1, policy_version 1328381 (0.0008) [2023-12-27 14:51:33,976][20381] Updated weights for policy 1, policy_version 1328391 (0.0006) [2023-12-27 14:51:34,343][20347] Updated weights for policy 0, policy_version 1329384 (0.0007) [2023-12-27 14:51:34,396][20347] Updated weights for policy 0, policy_version 1329394 (0.0008) [2023-12-27 14:51:34,459][20347] Updated weights for policy 0, policy_version 1329404 (0.0008) [2023-12-27 14:51:34,594][20381] Updated weights for policy 1, policy_version 1328401 (0.0006) [2023-12-27 14:51:34,655][20381] Updated weights for policy 1, policy_version 1328411 (0.0008) [2023-12-27 14:51:34,712][20381] Updated weights for policy 1, policy_version 1328421 (0.0008) [2023-12-27 14:51:34,772][20381] Updated weights for policy 1, policy_version 1328431 (0.0007) [2023-12-27 14:51:35,196][20347] Updated weights for policy 0, policy_version 1329414 (0.0007) [2023-12-27 14:51:35,244][20347] Updated weights for policy 0, policy_version 1329424 (0.0008) [2023-12-27 14:51:35,309][20347] Updated weights for policy 0, policy_version 1329434 (0.0008) [2023-12-27 14:51:35,392][20381] Updated weights for policy 1, policy_version 1328441 (0.0007) [2023-12-27 14:51:35,452][20381] Updated weights for policy 1, policy_version 1328451 (0.0009) [2023-12-27 14:51:35,502][20381] Updated weights for policy 1, policy_version 1328461 (0.0009) [2023-12-27 14:51:35,947][20347] Updated weights for policy 0, policy_version 1329444 (0.0005) [2023-12-27 14:51:35,997][20347] Updated weights for policy 0, policy_version 1329454 (0.0006) [2023-12-27 14:51:36,047][20347] Updated weights for policy 0, policy_version 1329464 (0.0007) [2023-12-27 14:51:36,209][20381] Updated weights for policy 1, policy_version 1328471 (0.0008) [2023-12-27 14:51:36,265][20381] Updated weights for policy 1, policy_version 1328481 (0.0008) [2023-12-27 14:51:36,316][20381] Updated weights for policy 1, policy_version 1328491 (0.0008) [2023-12-27 14:51:36,729][20347] Updated weights for policy 0, policy_version 1329474 (0.0007) [2023-12-27 14:51:36,791][20347] Updated weights for policy 0, policy_version 1329484 (0.0007) [2023-12-27 14:51:36,843][20347] Updated weights for policy 0, policy_version 1329494 (0.0005) [2023-12-27 14:51:36,899][20347] Updated weights for policy 0, policy_version 1329504 (0.0006) [2023-12-27 14:51:37,068][20381] Updated weights for policy 1, policy_version 1328501 (0.0008) [2023-12-27 14:51:37,127][20381] Updated weights for policy 1, policy_version 1328511 (0.0008) [2023-12-27 14:51:37,179][20381] Updated weights for policy 1, policy_version 1328521 (0.0007) [2023-12-27 14:51:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 680550400. Throughput: 0: 10254.5, 1: 10355.9. Samples: 680518436. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:37,363][19170] Avg episode reward: [(0, '912.320'), (1, '800.410')] [2023-12-27 14:51:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001329504_340418560.pth... [2023-12-27 14:51:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001328528_340131840.pth... [2023-12-27 14:51:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001328320_340115456.pth [2023-12-27 14:51:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001327312_339820544.pth [2023-12-27 14:51:37,549][20347] Updated weights for policy 0, policy_version 1329514 (0.0007) [2023-12-27 14:51:37,609][20347] Updated weights for policy 0, policy_version 1329524 (0.0007) [2023-12-27 14:51:37,660][20347] Updated weights for policy 0, policy_version 1329534 (0.0007) [2023-12-27 14:51:37,883][20381] Updated weights for policy 1, policy_version 1328531 (0.0008) [2023-12-27 14:51:37,930][20381] Updated weights for policy 1, policy_version 1328541 (0.0007) [2023-12-27 14:51:37,978][20381] Updated weights for policy 1, policy_version 1328551 (0.0007) [2023-12-27 14:51:38,356][20347] Updated weights for policy 0, policy_version 1329544 (0.0009) [2023-12-27 14:51:38,421][20347] Updated weights for policy 0, policy_version 1329554 (0.0005) [2023-12-27 14:51:38,486][20347] Updated weights for policy 0, policy_version 1329564 (0.0007) [2023-12-27 14:51:38,689][20381] Updated weights for policy 1, policy_version 1328561 (0.0008) [2023-12-27 14:51:38,746][20381] Updated weights for policy 1, policy_version 1328571 (0.0011) [2023-12-27 14:51:38,802][20381] Updated weights for policy 1, policy_version 1328581 (0.0010) [2023-12-27 14:51:38,862][20381] Updated weights for policy 1, policy_version 1328591 (0.0011) [2023-12-27 14:51:39,186][20347] Updated weights for policy 0, policy_version 1329574 (0.0008) [2023-12-27 14:51:39,240][20347] Updated weights for policy 0, policy_version 1329584 (0.0007) [2023-12-27 14:51:39,309][20347] Updated weights for policy 0, policy_version 1329594 (0.0005) [2023-12-27 14:51:39,559][20381] Updated weights for policy 1, policy_version 1328601 (0.0008) [2023-12-27 14:51:39,626][20381] Updated weights for policy 1, policy_version 1328611 (0.0007) [2023-12-27 14:51:39,692][20381] Updated weights for policy 1, policy_version 1328621 (0.0007) [2023-12-27 14:51:40,005][20347] Updated weights for policy 0, policy_version 1329604 (0.0007) [2023-12-27 14:51:40,059][20347] Updated weights for policy 0, policy_version 1329614 (0.0007) [2023-12-27 14:51:40,112][20347] Updated weights for policy 0, policy_version 1329624 (0.0006) [2023-12-27 14:51:40,375][20381] Updated weights for policy 1, policy_version 1328631 (0.0007) [2023-12-27 14:51:40,431][20381] Updated weights for policy 1, policy_version 1328641 (0.0008) [2023-12-27 14:51:40,490][20381] Updated weights for policy 1, policy_version 1328651 (0.0008) [2023-12-27 14:51:40,792][20347] Updated weights for policy 0, policy_version 1329634 (0.0007) [2023-12-27 14:51:40,850][20347] Updated weights for policy 0, policy_version 1329644 (0.0008) [2023-12-27 14:51:40,902][20347] Updated weights for policy 0, policy_version 1329654 (0.0007) [2023-12-27 14:51:40,950][20347] Updated weights for policy 0, policy_version 1329664 (0.0007) [2023-12-27 14:51:41,189][20381] Updated weights for policy 1, policy_version 1328661 (0.0008) [2023-12-27 14:51:41,244][20381] Updated weights for policy 1, policy_version 1328671 (0.0007) [2023-12-27 14:51:41,309][20381] Updated weights for policy 1, policy_version 1328681 (0.0008) [2023-12-27 14:51:41,690][20347] Updated weights for policy 0, policy_version 1329674 (0.0008) [2023-12-27 14:51:41,761][20347] Updated weights for policy 0, policy_version 1329684 (0.0008) [2023-12-27 14:51:41,821][20347] Updated weights for policy 0, policy_version 1329694 (0.0008) [2023-12-27 14:51:42,075][20381] Updated weights for policy 1, policy_version 1328691 (0.0008) [2023-12-27 14:51:42,132][20381] Updated weights for policy 1, policy_version 1328701 (0.0007) [2023-12-27 14:51:42,186][20381] Updated weights for policy 1, policy_version 1328711 (0.0008) [2023-12-27 14:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 680648704. Throughput: 0: 10230.1, 1: 10301.2. Samples: 680638260. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:42,363][19170] Avg episode reward: [(0, '912.630'), (1, '819.900')] [2023-12-27 14:51:42,552][20347] Updated weights for policy 0, policy_version 1329704 (0.0010) [2023-12-27 14:51:42,604][20347] Updated weights for policy 0, policy_version 1329714 (0.0010) [2023-12-27 14:51:42,657][20347] Updated weights for policy 0, policy_version 1329724 (0.0010) [2023-12-27 14:51:42,905][20381] Updated weights for policy 1, policy_version 1328721 (0.0007) [2023-12-27 14:51:42,969][20381] Updated weights for policy 1, policy_version 1328731 (0.0008) [2023-12-27 14:51:43,025][20381] Updated weights for policy 1, policy_version 1328741 (0.0008) [2023-12-27 14:51:43,078][20381] Updated weights for policy 1, policy_version 1328751 (0.0007) [2023-12-27 14:51:43,353][20347] Updated weights for policy 0, policy_version 1329734 (0.0010) [2023-12-27 14:51:43,401][20347] Updated weights for policy 0, policy_version 1329744 (0.0010) [2023-12-27 14:51:43,458][20347] Updated weights for policy 0, policy_version 1329754 (0.0010) [2023-12-27 14:51:43,762][20381] Updated weights for policy 1, policy_version 1328761 (0.0007) [2023-12-27 14:51:43,817][20381] Updated weights for policy 1, policy_version 1328771 (0.0007) [2023-12-27 14:51:43,875][20381] Updated weights for policy 1, policy_version 1328781 (0.0008) [2023-12-27 14:51:44,139][20347] Updated weights for policy 0, policy_version 1329764 (0.0010) [2023-12-27 14:51:44,191][20347] Updated weights for policy 0, policy_version 1329774 (0.0010) [2023-12-27 14:51:44,242][20347] Updated weights for policy 0, policy_version 1329784 (0.0010) [2023-12-27 14:51:44,574][20381] Updated weights for policy 1, policy_version 1328791 (0.0008) [2023-12-27 14:51:44,631][20381] Updated weights for policy 1, policy_version 1328801 (0.0007) [2023-12-27 14:51:44,685][20381] Updated weights for policy 1, policy_version 1328811 (0.0007) [2023-12-27 14:51:44,931][20347] Updated weights for policy 0, policy_version 1329794 (0.0010) [2023-12-27 14:51:44,991][20347] Updated weights for policy 0, policy_version 1329804 (0.0010) [2023-12-27 14:51:45,051][20347] Updated weights for policy 0, policy_version 1329814 (0.0011) [2023-12-27 14:51:45,110][20347] Updated weights for policy 0, policy_version 1329824 (0.0009) [2023-12-27 14:51:45,431][20381] Updated weights for policy 1, policy_version 1328821 (0.0007) [2023-12-27 14:51:45,489][20381] Updated weights for policy 1, policy_version 1328831 (0.0007) [2023-12-27 14:51:45,550][20381] Updated weights for policy 1, policy_version 1328841 (0.0007) [2023-12-27 14:51:45,799][20347] Updated weights for policy 0, policy_version 1329834 (0.0010) [2023-12-27 14:51:45,856][20347] Updated weights for policy 0, policy_version 1329844 (0.0010) [2023-12-27 14:51:45,910][20347] Updated weights for policy 0, policy_version 1329854 (0.0010) [2023-12-27 14:51:46,242][20381] Updated weights for policy 1, policy_version 1328851 (0.0007) [2023-12-27 14:51:46,292][20381] Updated weights for policy 1, policy_version 1328861 (0.0007) [2023-12-27 14:51:46,343][20381] Updated weights for policy 1, policy_version 1328871 (0.0007) [2023-12-27 14:51:46,567][20347] Updated weights for policy 0, policy_version 1329864 (0.0006) [2023-12-27 14:51:46,621][20347] Updated weights for policy 0, policy_version 1329874 (0.0009) [2023-12-27 14:51:46,684][20347] Updated weights for policy 0, policy_version 1329884 (0.0010) [2023-12-27 14:51:47,048][20381] Updated weights for policy 1, policy_version 1328881 (0.0008) [2023-12-27 14:51:47,105][20381] Updated weights for policy 1, policy_version 1328891 (0.0010) [2023-12-27 14:51:47,163][20381] Updated weights for policy 1, policy_version 1328901 (0.0010) [2023-12-27 14:51:47,218][20381] Updated weights for policy 1, policy_version 1328911 (0.0010) [2023-12-27 14:51:47,345][20347] Updated weights for policy 0, policy_version 1329894 (0.0010) [2023-12-27 14:51:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 680747008. Throughput: 0: 10204.7, 1: 10223.5. Samples: 680757552. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:47,363][19170] Avg episode reward: [(0, '912.740'), (1, '780.220')] [2023-12-27 14:51:47,394][20347] Updated weights for policy 0, policy_version 1329904 (0.0010) [2023-12-27 14:51:47,442][20347] Updated weights for policy 0, policy_version 1329914 (0.0010) [2023-12-27 14:51:47,835][20381] Updated weights for policy 1, policy_version 1328921 (0.0011) [2023-12-27 14:51:47,887][20381] Updated weights for policy 1, policy_version 1328931 (0.0011) [2023-12-27 14:51:47,936][20381] Updated weights for policy 1, policy_version 1328941 (0.0010) [2023-12-27 14:51:48,087][20347] Updated weights for policy 0, policy_version 1329924 (0.0010) [2023-12-27 14:51:48,143][20347] Updated weights for policy 0, policy_version 1329934 (0.0010) [2023-12-27 14:51:48,196][20347] Updated weights for policy 0, policy_version 1329944 (0.0010) [2023-12-27 14:51:48,646][20381] Updated weights for policy 1, policy_version 1328951 (0.0011) [2023-12-27 14:51:48,707][20381] Updated weights for policy 1, policy_version 1328961 (0.0011) [2023-12-27 14:51:48,768][20381] Updated weights for policy 1, policy_version 1328971 (0.0010) [2023-12-27 14:51:48,937][20347] Updated weights for policy 0, policy_version 1329954 (0.0010) [2023-12-27 14:51:48,996][20347] Updated weights for policy 0, policy_version 1329964 (0.0007) [2023-12-27 14:51:49,056][20347] Updated weights for policy 0, policy_version 1329974 (0.0005) [2023-12-27 14:51:49,117][20347] Updated weights for policy 0, policy_version 1329984 (0.0007) [2023-12-27 14:51:49,477][20381] Updated weights for policy 1, policy_version 1328981 (0.0009) [2023-12-27 14:51:49,543][20381] Updated weights for policy 1, policy_version 1328991 (0.0011) [2023-12-27 14:51:49,592][20381] Updated weights for policy 1, policy_version 1329001 (0.0010) [2023-12-27 14:51:49,828][20347] Updated weights for policy 0, policy_version 1329994 (0.0009) [2023-12-27 14:51:49,883][20347] Updated weights for policy 0, policy_version 1330004 (0.0008) [2023-12-27 14:51:49,945][20347] Updated weights for policy 0, policy_version 1330014 (0.0009) [2023-12-27 14:51:50,293][20381] Updated weights for policy 1, policy_version 1329011 (0.0010) [2023-12-27 14:51:50,356][20381] Updated weights for policy 1, policy_version 1329021 (0.0011) [2023-12-27 14:51:50,412][20381] Updated weights for policy 1, policy_version 1329031 (0.0011) [2023-12-27 14:51:50,607][20347] Updated weights for policy 0, policy_version 1330024 (0.0010) [2023-12-27 14:51:50,663][20347] Updated weights for policy 0, policy_version 1330034 (0.0010) [2023-12-27 14:51:50,718][20347] Updated weights for policy 0, policy_version 1330044 (0.0009) [2023-12-27 14:51:51,104][20381] Updated weights for policy 1, policy_version 1329041 (0.0011) [2023-12-27 14:51:51,168][20381] Updated weights for policy 1, policy_version 1329051 (0.0011) [2023-12-27 14:51:51,218][20381] Updated weights for policy 1, policy_version 1329061 (0.0011) [2023-12-27 14:51:51,283][20381] Updated weights for policy 1, policy_version 1329071 (0.0011) [2023-12-27 14:51:51,474][20347] Updated weights for policy 0, policy_version 1330054 (0.0008) [2023-12-27 14:51:51,538][20347] Updated weights for policy 0, policy_version 1330064 (0.0007) [2023-12-27 14:51:51,597][20347] Updated weights for policy 0, policy_version 1330074 (0.0008) [2023-12-27 14:51:52,027][20381] Updated weights for policy 1, policy_version 1329081 (0.0008) [2023-12-27 14:51:52,086][20381] Updated weights for policy 1, policy_version 1329091 (0.0008) [2023-12-27 14:51:52,145][20381] Updated weights for policy 1, policy_version 1329101 (0.0008) [2023-12-27 14:51:52,305][20347] Updated weights for policy 0, policy_version 1330084 (0.0007) [2023-12-27 14:51:52,359][20347] Updated weights for policy 0, policy_version 1330094 (0.0008) [2023-12-27 14:51:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 680845312. Throughput: 0: 10146.4, 1: 10220.8. Samples: 680817840. Policy #0 lag: (min: 19.0, avg: 24.5, max: 51.0) [2023-12-27 14:51:52,363][19170] Avg episode reward: [(0, '824.510'), (1, '803.820')] [2023-12-27 14:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001329104_340279296.pth... [2023-12-27 14:51:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001327920_339976192.pth [2023-12-27 14:51:52,425][20347] Updated weights for policy 0, policy_version 1330104 (0.0008) [2023-12-27 14:51:52,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001330112_340574208.pth... [2023-12-27 14:51:52,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001328896_340262912.pth [2023-12-27 14:51:52,759][20381] Updated weights for policy 1, policy_version 1329111 (0.0007) [2023-12-27 14:51:52,821][20381] Updated weights for policy 1, policy_version 1329121 (0.0005) [2023-12-27 14:51:52,885][20381] Updated weights for policy 1, policy_version 1329131 (0.0005) [2023-12-27 14:51:53,195][20347] Updated weights for policy 0, policy_version 1330114 (0.0009) [2023-12-27 14:51:53,262][20347] Updated weights for policy 0, policy_version 1330124 (0.0008) [2023-12-27 14:51:53,314][20347] Updated weights for policy 0, policy_version 1330134 (0.0007) [2023-12-27 14:51:53,366][20347] Updated weights for policy 0, policy_version 1330144 (0.0007) [2023-12-27 14:51:53,390][20381] Updated weights for policy 1, policy_version 1329141 (0.0007) [2023-12-27 14:51:53,435][20381] Updated weights for policy 1, policy_version 1329151 (0.0009) [2023-12-27 14:51:53,486][20381] Updated weights for policy 1, policy_version 1329161 (0.0006) [2023-12-27 14:51:54,011][20347] Updated weights for policy 0, policy_version 1330154 (0.0007) [2023-12-27 14:51:54,077][20347] Updated weights for policy 0, policy_version 1330164 (0.0008) [2023-12-27 14:51:54,084][20381] Updated weights for policy 1, policy_version 1329171 (0.0006) [2023-12-27 14:51:54,130][20347] Updated weights for policy 0, policy_version 1330174 (0.0009) [2023-12-27 14:51:54,135][20381] Updated weights for policy 1, policy_version 1329181 (0.0007) [2023-12-27 14:51:54,185][20381] Updated weights for policy 1, policy_version 1329191 (0.0007) [2023-12-27 14:51:54,234][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 14:51:54,787][20347] Updated weights for policy 0, policy_version 1330184 (0.0008) [2023-12-27 14:51:54,840][20347] Updated weights for policy 0, policy_version 1330194 (0.0007) [2023-12-27 14:51:54,894][20347] Updated weights for policy 0, policy_version 1330204 (0.0006) [2023-12-27 14:51:54,898][20381] Updated weights for policy 1, policy_version 1329201 (0.0008) [2023-12-27 14:51:54,951][20381] Updated weights for policy 1, policy_version 1329211 (0.0007) [2023-12-27 14:51:55,003][20381] Updated weights for policy 1, policy_version 1329221 (0.0007) [2023-12-27 14:51:55,060][20381] Updated weights for policy 1, policy_version 1329231 (0.0005) [2023-12-27 14:51:55,605][20347] Updated weights for policy 0, policy_version 1330214 (0.0006) [2023-12-27 14:51:55,660][20347] Updated weights for policy 0, policy_version 1330224 (0.0005) [2023-12-27 14:51:55,708][20347] Updated weights for policy 0, policy_version 1330234 (0.0005) [2023-12-27 14:51:55,727][20381] Updated weights for policy 1, policy_version 1329241 (0.0010) [2023-12-27 14:51:55,780][20381] Updated weights for policy 1, policy_version 1329251 (0.0010) [2023-12-27 14:51:55,839][20381] Updated weights for policy 1, policy_version 1329261 (0.0007) [2023-12-27 14:51:56,246][20347] Updated weights for policy 0, policy_version 1330244 (0.0005) [2023-12-27 14:51:56,298][20347] Updated weights for policy 0, policy_version 1330254 (0.0005) [2023-12-27 14:51:56,346][20347] Updated weights for policy 0, policy_version 1330264 (0.0005) [2023-12-27 14:51:56,538][20381] Updated weights for policy 1, policy_version 1329271 (0.0008) [2023-12-27 14:51:56,590][20381] Updated weights for policy 1, policy_version 1329281 (0.0008) [2023-12-27 14:51:56,645][20381] Updated weights for policy 1, policy_version 1329291 (0.0007) [2023-12-27 14:51:56,925][20347] Updated weights for policy 0, policy_version 1330274 (0.0006) [2023-12-27 14:51:56,974][20347] Updated weights for policy 0, policy_version 1330284 (0.0007) [2023-12-27 14:51:57,019][20347] Updated weights for policy 0, policy_version 1330294 (0.0007) [2023-12-27 14:51:57,062][20347] Updated weights for policy 0, policy_version 1330304 (0.0008) [2023-12-27 14:51:57,210][20381] Updated weights for policy 1, policy_version 1329301 (0.0006) [2023-12-27 14:51:57,270][20381] Updated weights for policy 1, policy_version 1329311 (0.0005) [2023-12-27 14:51:57,328][20381] Updated weights for policy 1, policy_version 1329321 (0.0006) [2023-12-27 14:51:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 680951808. Throughput: 0: 10136.3, 1: 10266.7. Samples: 680942252. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:51:57,363][19170] Avg episode reward: [(0, '843.990'), (1, '803.760')] [2023-12-27 14:51:57,760][20347] Updated weights for policy 0, policy_version 1330314 (0.0010) [2023-12-27 14:51:57,813][20347] Updated weights for policy 0, policy_version 1330324 (0.0010) [2023-12-27 14:51:57,869][20347] Updated weights for policy 0, policy_version 1330334 (0.0010) [2023-12-27 14:51:57,971][20381] Updated weights for policy 1, policy_version 1329331 (0.0007) [2023-12-27 14:51:58,023][20381] Updated weights for policy 1, policy_version 1329341 (0.0005) [2023-12-27 14:51:58,073][20381] Updated weights for policy 1, policy_version 1329351 (0.0005) [2023-12-27 14:51:58,540][20347] Updated weights for policy 0, policy_version 1330344 (0.0009) [2023-12-27 14:51:58,602][20347] Updated weights for policy 0, policy_version 1330354 (0.0009) [2023-12-27 14:51:58,662][20347] Updated weights for policy 0, policy_version 1330364 (0.0009) [2023-12-27 14:51:58,834][20381] Updated weights for policy 1, policy_version 1329361 (0.0006) [2023-12-27 14:51:58,905][20381] Updated weights for policy 1, policy_version 1329371 (0.0007) [2023-12-27 14:51:58,960][20381] Updated weights for policy 1, policy_version 1329381 (0.0011) [2023-12-27 14:51:59,019][20381] Updated weights for policy 1, policy_version 1329391 (0.0010) [2023-12-27 14:51:59,460][20347] Updated weights for policy 0, policy_version 1330374 (0.0008) [2023-12-27 14:51:59,506][20347] Updated weights for policy 0, policy_version 1330384 (0.0008) [2023-12-27 14:51:59,566][20347] Updated weights for policy 0, policy_version 1330394 (0.0007) [2023-12-27 14:51:59,741][20381] Updated weights for policy 1, policy_version 1329401 (0.0008) [2023-12-27 14:51:59,807][20381] Updated weights for policy 1, policy_version 1329411 (0.0008) [2023-12-27 14:51:59,873][20381] Updated weights for policy 1, policy_version 1329421 (0.0009) [2023-12-27 14:52:00,310][20347] Updated weights for policy 0, policy_version 1330404 (0.0007) [2023-12-27 14:52:00,366][20347] Updated weights for policy 0, policy_version 1330414 (0.0007) [2023-12-27 14:52:00,419][20347] Updated weights for policy 0, policy_version 1330424 (0.0008) [2023-12-27 14:52:00,617][20381] Updated weights for policy 1, policy_version 1329431 (0.0006) [2023-12-27 14:52:00,680][20381] Updated weights for policy 1, policy_version 1329441 (0.0006) [2023-12-27 14:52:00,735][20381] Updated weights for policy 1, policy_version 1329451 (0.0006) [2023-12-27 14:52:01,009][20347] Updated weights for policy 0, policy_version 1330434 (0.0007) [2023-12-27 14:52:01,074][20347] Updated weights for policy 0, policy_version 1330444 (0.0007) [2023-12-27 14:52:01,134][20347] Updated weights for policy 0, policy_version 1330454 (0.0008) [2023-12-27 14:52:01,192][20347] Updated weights for policy 0, policy_version 1330464 (0.0008) [2023-12-27 14:52:01,337][20381] Updated weights for policy 1, policy_version 1329461 (0.0007) [2023-12-27 14:52:01,409][20381] Updated weights for policy 1, policy_version 1329471 (0.0008) [2023-12-27 14:52:01,468][20381] Updated weights for policy 1, policy_version 1329481 (0.0009) [2023-12-27 14:52:01,908][20347] Updated weights for policy 0, policy_version 1330474 (0.0007) [2023-12-27 14:52:01,955][20347] Updated weights for policy 0, policy_version 1330484 (0.0007) [2023-12-27 14:52:02,010][20347] Updated weights for policy 0, policy_version 1330494 (0.0007) [2023-12-27 14:52:02,126][20381] Updated weights for policy 1, policy_version 1329491 (0.0011) [2023-12-27 14:52:02,185][20381] Updated weights for policy 1, policy_version 1329501 (0.0011) [2023-12-27 14:52:02,240][20381] Updated weights for policy 1, policy_version 1329511 (0.0010) [2023-12-27 14:52:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 681058304. Throughput: 0: 10138.7, 1: 10246.0. Samples: 681064108. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:02,363][19170] Avg episode reward: [(0, '796.440'), (1, '789.890')] [2023-12-27 14:52:02,705][20347] Updated weights for policy 0, policy_version 1330504 (0.0006) [2023-12-27 14:52:02,763][20347] Updated weights for policy 0, policy_version 1330514 (0.0005) [2023-12-27 14:52:02,823][20347] Updated weights for policy 0, policy_version 1330524 (0.0005) [2023-12-27 14:52:02,939][20381] Updated weights for policy 1, policy_version 1329521 (0.0010) [2023-12-27 14:52:02,998][20381] Updated weights for policy 1, policy_version 1329531 (0.0009) [2023-12-27 14:52:03,056][20381] Updated weights for policy 1, policy_version 1329541 (0.0011) [2023-12-27 14:52:03,115][20381] Updated weights for policy 1, policy_version 1329551 (0.0010) [2023-12-27 14:52:03,455][20347] Updated weights for policy 0, policy_version 1330534 (0.0007) [2023-12-27 14:52:03,512][20347] Updated weights for policy 0, policy_version 1330544 (0.0007) [2023-12-27 14:52:03,569][20347] Updated weights for policy 0, policy_version 1330554 (0.0007) [2023-12-27 14:52:03,779][20381] Updated weights for policy 1, policy_version 1329561 (0.0010) [2023-12-27 14:52:03,831][20381] Updated weights for policy 1, policy_version 1329571 (0.0010) [2023-12-27 14:52:03,898][20381] Updated weights for policy 1, policy_version 1329581 (0.0007) [2023-12-27 14:52:04,232][20347] Updated weights for policy 0, policy_version 1330564 (0.0007) [2023-12-27 14:52:04,292][20347] Updated weights for policy 0, policy_version 1330574 (0.0007) [2023-12-27 14:52:04,348][20347] Updated weights for policy 0, policy_version 1330584 (0.0007) [2023-12-27 14:52:04,597][20381] Updated weights for policy 1, policy_version 1329591 (0.0008) [2023-12-27 14:52:04,658][20381] Updated weights for policy 1, policy_version 1329601 (0.0008) [2023-12-27 14:52:04,713][20381] Updated weights for policy 1, policy_version 1329611 (0.0008) [2023-12-27 14:52:05,024][20347] Updated weights for policy 0, policy_version 1330594 (0.0008) [2023-12-27 14:52:05,082][20347] Updated weights for policy 0, policy_version 1330604 (0.0010) [2023-12-27 14:52:05,135][20347] Updated weights for policy 0, policy_version 1330614 (0.0010) [2023-12-27 14:52:05,154][20291] KL-divergence is very high: 124.1079 [2023-12-27 14:52:05,159][20291] KL-divergence is very high: 145.6867 [2023-12-27 14:52:05,170][20291] KL-divergence is very high: 137.1199 [2023-12-27 14:52:05,186][20347] Updated weights for policy 0, policy_version 1330624 (0.0007) [2023-12-27 14:52:05,402][20381] Updated weights for policy 1, policy_version 1329621 (0.0006) [2023-12-27 14:52:05,455][20381] Updated weights for policy 1, policy_version 1329631 (0.0006) [2023-12-27 14:52:05,510][20381] Updated weights for policy 1, policy_version 1329641 (0.0007) [2023-12-27 14:52:05,857][20347] Updated weights for policy 0, policy_version 1330634 (0.0007) [2023-12-27 14:52:05,858][20291] KL-divergence is very high: 196.0544 [2023-12-27 14:52:05,905][20291] KL-divergence is very high: 115.0627 [2023-12-27 14:52:05,916][20347] Updated weights for policy 0, policy_version 1330644 (0.0008) [2023-12-27 14:52:05,968][20347] Updated weights for policy 0, policy_version 1330654 (0.0006) [2023-12-27 14:52:06,112][20381] Updated weights for policy 1, policy_version 1329651 (0.0011) [2023-12-27 14:52:06,172][20381] Updated weights for policy 1, policy_version 1329661 (0.0011) [2023-12-27 14:52:06,233][20381] Updated weights for policy 1, policy_version 1329671 (0.0011) [2023-12-27 14:52:06,659][20347] Updated weights for policy 0, policy_version 1330664 (0.0007) [2023-12-27 14:52:06,723][20347] Updated weights for policy 0, policy_version 1330674 (0.0007) [2023-12-27 14:52:06,781][20347] Updated weights for policy 0, policy_version 1330684 (0.0007) [2023-12-27 14:52:06,918][20381] Updated weights for policy 1, policy_version 1329681 (0.0011) [2023-12-27 14:52:06,973][20381] Updated weights for policy 1, policy_version 1329691 (0.0010) [2023-12-27 14:52:07,030][20381] Updated weights for policy 1, policy_version 1329702 (0.0010) [2023-12-27 14:52:07,084][20381] Updated weights for policy 1, policy_version 1329712 (0.0006) [2023-12-27 14:52:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 681156608. Throughput: 0: 10181.7, 1: 10208.4. Samples: 681125360. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:07,363][19170] Avg episode reward: [(0, '766.090'), (1, '771.080')] [2023-12-27 14:52:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001330688_340721664.pth... [2023-12-27 14:52:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001329712_340434944.pth... [2023-12-27 14:52:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001329504_340418560.pth [2023-12-27 14:52:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001328528_340131840.pth [2023-12-27 14:52:07,502][20347] Updated weights for policy 0, policy_version 1330694 (0.0006) [2023-12-27 14:52:07,551][20347] Updated weights for policy 0, policy_version 1330704 (0.0010) [2023-12-27 14:52:07,612][20347] Updated weights for policy 0, policy_version 1330714 (0.0007) [2023-12-27 14:52:07,667][20381] Updated weights for policy 1, policy_version 1329722 (0.0007) [2023-12-27 14:52:07,716][20381] Updated weights for policy 1, policy_version 1329732 (0.0008) [2023-12-27 14:52:07,768][20381] Updated weights for policy 1, policy_version 1329742 (0.0007) [2023-12-27 14:52:08,234][20347] Updated weights for policy 0, policy_version 1330724 (0.0006) [2023-12-27 14:52:08,285][20347] Updated weights for policy 0, policy_version 1330734 (0.0007) [2023-12-27 14:52:08,341][20347] Updated weights for policy 0, policy_version 1330744 (0.0007) [2023-12-27 14:52:08,445][20381] Updated weights for policy 1, policy_version 1329752 (0.0007) [2023-12-27 14:52:08,504][20381] Updated weights for policy 1, policy_version 1329762 (0.0008) [2023-12-27 14:52:08,553][20381] Updated weights for policy 1, policy_version 1329772 (0.0007) [2023-12-27 14:52:09,073][20347] Updated weights for policy 0, policy_version 1330754 (0.0008) [2023-12-27 14:52:09,128][20347] Updated weights for policy 0, policy_version 1330764 (0.0008) [2023-12-27 14:52:09,179][20347] Updated weights for policy 0, policy_version 1330774 (0.0007) [2023-12-27 14:52:09,243][20347] Updated weights for policy 0, policy_version 1330784 (0.0008) [2023-12-27 14:52:09,248][20381] Updated weights for policy 1, policy_version 1329782 (0.0009) [2023-12-27 14:52:09,305][20381] Updated weights for policy 1, policy_version 1329792 (0.0009) [2023-12-27 14:52:09,375][20381] Updated weights for policy 1, policy_version 1329802 (0.0008) [2023-12-27 14:52:09,999][20347] Updated weights for policy 0, policy_version 1330794 (0.0010) [2023-12-27 14:52:10,059][20347] Updated weights for policy 0, policy_version 1330804 (0.0009) [2023-12-27 14:52:10,120][20347] Updated weights for policy 0, policy_version 1330814 (0.0006) [2023-12-27 14:52:10,177][20381] Updated weights for policy 1, policy_version 1329812 (0.0008) [2023-12-27 14:52:10,230][20381] Updated weights for policy 1, policy_version 1329822 (0.0009) [2023-12-27 14:52:10,291][20381] Updated weights for policy 1, policy_version 1329832 (0.0009) [2023-12-27 14:52:10,797][20347] Updated weights for policy 0, policy_version 1330824 (0.0007) [2023-12-27 14:52:10,849][20347] Updated weights for policy 0, policy_version 1330834 (0.0007) [2023-12-27 14:52:10,906][20347] Updated weights for policy 0, policy_version 1330844 (0.0008) [2023-12-27 14:52:11,008][20381] Updated weights for policy 1, policy_version 1329842 (0.0008) [2023-12-27 14:52:11,076][20381] Updated weights for policy 1, policy_version 1329852 (0.0007) [2023-12-27 14:52:11,140][20381] Updated weights for policy 1, policy_version 1329862 (0.0008) [2023-12-27 14:52:11,202][20381] Updated weights for policy 1, policy_version 1329872 (0.0008) [2023-12-27 14:52:11,742][20347] Updated weights for policy 0, policy_version 1330854 (0.0008) [2023-12-27 14:52:11,802][20347] Updated weights for policy 0, policy_version 1330864 (0.0008) [2023-12-27 14:52:11,854][20347] Updated weights for policy 0, policy_version 1330874 (0.0008) [2023-12-27 14:52:11,968][20381] Updated weights for policy 1, policy_version 1329882 (0.0008) [2023-12-27 14:52:12,023][20381] Updated weights for policy 1, policy_version 1329892 (0.0008) [2023-12-27 14:52:12,078][20381] Updated weights for policy 1, policy_version 1329902 (0.0008) [2023-12-27 14:52:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 681254912. Throughput: 0: 10200.3, 1: 10194.6. Samples: 681246232. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:12,363][19170] Avg episode reward: [(0, '746.340'), (1, '847.710')] [2023-12-27 14:52:12,604][20347] Updated weights for policy 0, policy_version 1330884 (0.0008) [2023-12-27 14:52:12,658][20347] Updated weights for policy 0, policy_version 1330894 (0.0007) [2023-12-27 14:52:12,712][20347] Updated weights for policy 0, policy_version 1330904 (0.0007) [2023-12-27 14:52:12,845][20381] Updated weights for policy 1, policy_version 1329912 (0.0007) [2023-12-27 14:52:12,904][20381] Updated weights for policy 1, policy_version 1329922 (0.0005) [2023-12-27 14:52:12,957][20381] Updated weights for policy 1, policy_version 1329932 (0.0006) [2023-12-27 14:52:13,370][20347] Updated weights for policy 0, policy_version 1330914 (0.0007) [2023-12-27 14:52:13,420][20347] Updated weights for policy 0, policy_version 1330924 (0.0008) [2023-12-27 14:52:13,469][20347] Updated weights for policy 0, policy_version 1330934 (0.0008) [2023-12-27 14:52:13,519][20347] Updated weights for policy 0, policy_version 1330944 (0.0008) [2023-12-27 14:52:13,610][20381] Updated weights for policy 1, policy_version 1329942 (0.0006) [2023-12-27 14:52:13,653][20381] Updated weights for policy 1, policy_version 1329952 (0.0005) [2023-12-27 14:52:13,708][20381] Updated weights for policy 1, policy_version 1329962 (0.0005) [2023-12-27 14:52:14,227][20347] Updated weights for policy 0, policy_version 1330954 (0.0010) [2023-12-27 14:52:14,280][20381] Updated weights for policy 1, policy_version 1329972 (0.0005) [2023-12-27 14:52:14,282][20347] Updated weights for policy 0, policy_version 1330964 (0.0010) [2023-12-27 14:52:14,338][20381] Updated weights for policy 1, policy_version 1329982 (0.0006) [2023-12-27 14:52:14,340][20347] Updated weights for policy 0, policy_version 1330974 (0.0010) [2023-12-27 14:52:14,395][20381] Updated weights for policy 1, policy_version 1329992 (0.0006) [2023-12-27 14:52:15,017][20347] Updated weights for policy 0, policy_version 1330984 (0.0008) [2023-12-27 14:52:15,047][20381] Updated weights for policy 1, policy_version 1330002 (0.0007) [2023-12-27 14:52:15,077][20347] Updated weights for policy 0, policy_version 1330994 (0.0010) [2023-12-27 14:52:15,106][20381] Updated weights for policy 1, policy_version 1330012 (0.0006) [2023-12-27 14:52:15,134][20347] Updated weights for policy 0, policy_version 1331004 (0.0009) [2023-12-27 14:52:15,160][20381] Updated weights for policy 1, policy_version 1330022 (0.0006) [2023-12-27 14:52:15,214][20381] Updated weights for policy 1, policy_version 1330032 (0.0008) [2023-12-27 14:52:15,758][20347] Updated weights for policy 0, policy_version 1331014 (0.0009) [2023-12-27 14:52:15,808][20347] Updated weights for policy 0, policy_version 1331024 (0.0008) [2023-12-27 14:52:15,854][20381] Updated weights for policy 1, policy_version 1330042 (0.0007) [2023-12-27 14:52:15,868][20347] Updated weights for policy 0, policy_version 1331034 (0.0005) [2023-12-27 14:52:15,897][20381] Updated weights for policy 1, policy_version 1330052 (0.0008) [2023-12-27 14:52:15,943][20381] Updated weights for policy 1, policy_version 1330062 (0.0008) [2023-12-27 14:52:16,473][20347] Updated weights for policy 0, policy_version 1331044 (0.0006) [2023-12-27 14:52:16,532][20347] Updated weights for policy 0, policy_version 1331054 (0.0005) [2023-12-27 14:52:16,588][20347] Updated weights for policy 0, policy_version 1331064 (0.0005) [2023-12-27 14:52:16,679][20381] Updated weights for policy 1, policy_version 1330072 (0.0006) [2023-12-27 14:52:16,735][20381] Updated weights for policy 1, policy_version 1330082 (0.0007) [2023-12-27 14:52:16,781][20381] Updated weights for policy 1, policy_version 1330092 (0.0009) [2023-12-27 14:52:17,101][20347] Updated weights for policy 0, policy_version 1331074 (0.0005) [2023-12-27 14:52:17,154][20347] Updated weights for policy 0, policy_version 1331084 (0.0006) [2023-12-27 14:52:17,200][20347] Updated weights for policy 0, policy_version 1331094 (0.0008) [2023-12-27 14:52:17,251][20347] Updated weights for policy 0, policy_version 1331104 (0.0006) [2023-12-27 14:52:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 681361408. Throughput: 0: 10184.6, 1: 10177.9. Samples: 681370048. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:17,364][19170] Avg episode reward: [(0, '746.250'), (1, '922.340')] [2023-12-27 14:52:17,483][20381] Updated weights for policy 1, policy_version 1330102 (0.0006) [2023-12-27 14:52:17,535][20381] Updated weights for policy 1, policy_version 1330112 (0.0005) [2023-12-27 14:52:17,586][20381] Updated weights for policy 1, policy_version 1330122 (0.0006) [2023-12-27 14:52:17,945][20347] Updated weights for policy 0, policy_version 1331114 (0.0008) [2023-12-27 14:52:17,993][20347] Updated weights for policy 0, policy_version 1331124 (0.0008) [2023-12-27 14:52:18,050][20347] Updated weights for policy 0, policy_version 1331134 (0.0008) [2023-12-27 14:52:18,196][20381] Updated weights for policy 1, policy_version 1330132 (0.0007) [2023-12-27 14:52:18,254][20381] Updated weights for policy 1, policy_version 1330142 (0.0007) [2023-12-27 14:52:18,313][20381] Updated weights for policy 1, policy_version 1330152 (0.0007) [2023-12-27 14:52:18,747][20347] Updated weights for policy 0, policy_version 1331144 (0.0007) [2023-12-27 14:52:18,810][20347] Updated weights for policy 0, policy_version 1331154 (0.0008) [2023-12-27 14:52:18,866][20347] Updated weights for policy 0, policy_version 1331164 (0.0007) [2023-12-27 14:52:18,944][20381] Updated weights for policy 1, policy_version 1330162 (0.0007) [2023-12-27 14:52:18,997][20381] Updated weights for policy 1, policy_version 1330172 (0.0008) [2023-12-27 14:52:19,050][20381] Updated weights for policy 1, policy_version 1330182 (0.0008) [2023-12-27 14:52:19,100][20381] Updated weights for policy 1, policy_version 1330192 (0.0007) [2023-12-27 14:52:19,466][20347] Updated weights for policy 0, policy_version 1331174 (0.0005) [2023-12-27 14:52:19,533][20347] Updated weights for policy 0, policy_version 1331184 (0.0008) [2023-12-27 14:52:19,590][20347] Updated weights for policy 0, policy_version 1331194 (0.0008) [2023-12-27 14:52:19,881][20381] Updated weights for policy 1, policy_version 1330202 (0.0009) [2023-12-27 14:52:19,941][20381] Updated weights for policy 1, policy_version 1330212 (0.0008) [2023-12-27 14:52:20,004][20381] Updated weights for policy 1, policy_version 1330222 (0.0007) [2023-12-27 14:52:20,240][20347] Updated weights for policy 0, policy_version 1331204 (0.0008) [2023-12-27 14:52:20,297][20347] Updated weights for policy 0, policy_version 1331214 (0.0009) [2023-12-27 14:52:20,347][20347] Updated weights for policy 0, policy_version 1331224 (0.0008) [2023-12-27 14:52:20,654][20381] Updated weights for policy 1, policy_version 1330232 (0.0007) [2023-12-27 14:52:20,725][20381] Updated weights for policy 1, policy_version 1330242 (0.0007) [2023-12-27 14:52:20,795][20381] Updated weights for policy 1, policy_version 1330252 (0.0007) [2023-12-27 14:52:21,135][20347] Updated weights for policy 0, policy_version 1331234 (0.0009) [2023-12-27 14:52:21,200][20347] Updated weights for policy 0, policy_version 1331244 (0.0008) [2023-12-27 14:52:21,264][20347] Updated weights for policy 0, policy_version 1331254 (0.0008) [2023-12-27 14:52:21,325][20347] Updated weights for policy 0, policy_version 1331264 (0.0006) [2023-12-27 14:52:21,496][20381] Updated weights for policy 1, policy_version 1330262 (0.0007) [2023-12-27 14:52:21,560][20381] Updated weights for policy 1, policy_version 1330272 (0.0006) [2023-12-27 14:52:21,624][20381] Updated weights for policy 1, policy_version 1330282 (0.0007) [2023-12-27 14:52:21,994][20347] Updated weights for policy 0, policy_version 1331274 (0.0009) [2023-12-27 14:52:22,051][20347] Updated weights for policy 0, policy_version 1331284 (0.0007) [2023-12-27 14:52:22,108][20347] Updated weights for policy 0, policy_version 1331294 (0.0008) [2023-12-27 14:52:22,275][20381] Updated weights for policy 1, policy_version 1330292 (0.0007) [2023-12-27 14:52:22,338][20381] Updated weights for policy 1, policy_version 1330302 (0.0006) [2023-12-27 14:52:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 681459712. Throughput: 0: 10179.2, 1: 10113.2. Samples: 681431596. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:22,363][19170] Avg episode reward: [(0, '785.500'), (1, '921.620')] [2023-12-27 14:52:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001331296_340877312.pth... [2023-12-27 14:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001330112_340574208.pth [2023-12-27 14:52:22,403][20381] Updated weights for policy 1, policy_version 1330312 (0.0009) [2023-12-27 14:52:22,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001330320_340590592.pth... [2023-12-27 14:52:22,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001329104_340279296.pth [2023-12-27 14:52:22,835][20347] Updated weights for policy 0, policy_version 1331304 (0.0007) [2023-12-27 14:52:22,900][20347] Updated weights for policy 0, policy_version 1331314 (0.0006) [2023-12-27 14:52:22,958][20347] Updated weights for policy 0, policy_version 1331324 (0.0007) [2023-12-27 14:52:23,043][20381] Updated weights for policy 1, policy_version 1330322 (0.0008) [2023-12-27 14:52:23,094][20381] Updated weights for policy 1, policy_version 1330332 (0.0008) [2023-12-27 14:52:23,145][20381] Updated weights for policy 1, policy_version 1330342 (0.0007) [2023-12-27 14:52:23,196][20381] Updated weights for policy 1, policy_version 1330352 (0.0008) [2023-12-27 14:52:23,648][20347] Updated weights for policy 0, policy_version 1331334 (0.0008) [2023-12-27 14:52:23,700][20347] Updated weights for policy 0, policy_version 1331344 (0.0008) [2023-12-27 14:52:23,756][20347] Updated weights for policy 0, policy_version 1331354 (0.0009) [2023-12-27 14:52:23,868][20381] Updated weights for policy 1, policy_version 1330362 (0.0007) [2023-12-27 14:52:23,927][20381] Updated weights for policy 1, policy_version 1330372 (0.0008) [2023-12-27 14:52:23,985][20381] Updated weights for policy 1, policy_version 1330382 (0.0008) [2023-12-27 14:52:24,460][20347] Updated weights for policy 0, policy_version 1331364 (0.0008) [2023-12-27 14:52:24,522][20347] Updated weights for policy 0, policy_version 1331374 (0.0007) [2023-12-27 14:52:24,579][20347] Updated weights for policy 0, policy_version 1331384 (0.0005) [2023-12-27 14:52:24,667][20381] Updated weights for policy 1, policy_version 1330392 (0.0009) [2023-12-27 14:52:24,716][20381] Updated weights for policy 1, policy_version 1330402 (0.0008) [2023-12-27 14:52:24,778][20381] Updated weights for policy 1, policy_version 1330412 (0.0007) [2023-12-27 14:52:25,174][20347] Updated weights for policy 0, policy_version 1331394 (0.0005) [2023-12-27 14:52:25,223][20347] Updated weights for policy 0, policy_version 1331404 (0.0005) [2023-12-27 14:52:25,272][20347] Updated weights for policy 0, policy_version 1331414 (0.0007) [2023-12-27 14:52:25,326][20347] Updated weights for policy 0, policy_version 1331424 (0.0010) [2023-12-27 14:52:25,510][20381] Updated weights for policy 1, policy_version 1330422 (0.0007) [2023-12-27 14:52:25,564][20381] Updated weights for policy 1, policy_version 1330432 (0.0008) [2023-12-27 14:52:25,622][20381] Updated weights for policy 1, policy_version 1330442 (0.0008) [2023-12-27 14:52:25,947][20347] Updated weights for policy 0, policy_version 1331434 (0.0010) [2023-12-27 14:52:25,997][20347] Updated weights for policy 0, policy_version 1331444 (0.0009) [2023-12-27 14:52:26,057][20347] Updated weights for policy 0, policy_version 1331454 (0.0010) [2023-12-27 14:52:26,370][20381] Updated weights for policy 1, policy_version 1330452 (0.0008) [2023-12-27 14:52:26,424][20381] Updated weights for policy 1, policy_version 1330462 (0.0007) [2023-12-27 14:52:26,470][20381] Updated weights for policy 1, policy_version 1330472 (0.0009) [2023-12-27 14:52:26,682][20347] Updated weights for policy 0, policy_version 1331464 (0.0006) [2023-12-27 14:52:26,736][20347] Updated weights for policy 0, policy_version 1331474 (0.0005) [2023-12-27 14:52:26,784][20347] Updated weights for policy 0, policy_version 1331484 (0.0005) [2023-12-27 14:52:27,230][20381] Updated weights for policy 1, policy_version 1330482 (0.0007) [2023-12-27 14:52:27,287][20381] Updated weights for policy 1, policy_version 1330492 (0.0006) [2023-12-27 14:52:27,298][20347] Updated weights for policy 0, policy_version 1331494 (0.0005) [2023-12-27 14:52:27,340][20381] Updated weights for policy 1, policy_version 1330502 (0.0006) [2023-12-27 14:52:27,351][20347] Updated weights for policy 0, policy_version 1331504 (0.0005) [2023-12-27 14:52:27,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 681558016. Throughput: 0: 10178.7, 1: 10156.8. Samples: 681553356. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:27,363][19170] Avg episode reward: [(0, '785.240'), (1, '911.810')] [2023-12-27 14:52:27,402][20381] Updated weights for policy 1, policy_version 1330512 (0.0005) [2023-12-27 14:52:27,404][20347] Updated weights for policy 0, policy_version 1331514 (0.0006) [2023-12-27 14:52:27,991][20347] Updated weights for policy 0, policy_version 1331524 (0.0008) [2023-12-27 14:52:28,024][20381] Updated weights for policy 1, policy_version 1330522 (0.0007) [2023-12-27 14:52:28,037][20347] Updated weights for policy 0, policy_version 1331534 (0.0006) [2023-12-27 14:52:28,068][20381] Updated weights for policy 1, policy_version 1330532 (0.0005) [2023-12-27 14:52:28,081][20347] Updated weights for policy 0, policy_version 1331544 (0.0005) [2023-12-27 14:52:28,126][20381] Updated weights for policy 1, policy_version 1330542 (0.0005) [2023-12-27 14:52:28,746][20347] Updated weights for policy 0, policy_version 1331554 (0.0006) [2023-12-27 14:52:28,803][20347] Updated weights for policy 0, policy_version 1331564 (0.0007) [2023-12-27 14:52:28,847][20381] Updated weights for policy 1, policy_version 1330552 (0.0006) [2023-12-27 14:52:28,863][20347] Updated weights for policy 0, policy_version 1331574 (0.0008) [2023-12-27 14:52:28,904][20381] Updated weights for policy 1, policy_version 1330562 (0.0009) [2023-12-27 14:52:28,917][20347] Updated weights for policy 0, policy_version 1331584 (0.0006) [2023-12-27 14:52:28,959][20381] Updated weights for policy 1, policy_version 1330572 (0.0007) [2023-12-27 14:52:29,615][20347] Updated weights for policy 0, policy_version 1331594 (0.0008) [2023-12-27 14:52:29,635][20381] Updated weights for policy 1, policy_version 1330582 (0.0008) [2023-12-27 14:52:29,666][20347] Updated weights for policy 0, policy_version 1331604 (0.0006) [2023-12-27 14:52:29,696][20381] Updated weights for policy 1, policy_version 1330592 (0.0008) [2023-12-27 14:52:29,719][20347] Updated weights for policy 0, policy_version 1331614 (0.0005) [2023-12-27 14:52:29,759][20381] Updated weights for policy 1, policy_version 1330602 (0.0008) [2023-12-27 14:52:30,375][20347] Updated weights for policy 0, policy_version 1331624 (0.0005) [2023-12-27 14:52:30,409][20381] Updated weights for policy 1, policy_version 1330612 (0.0007) [2023-12-27 14:52:30,425][20347] Updated weights for policy 0, policy_version 1331634 (0.0007) [2023-12-27 14:52:30,464][20381] Updated weights for policy 1, policy_version 1330622 (0.0005) [2023-12-27 14:52:30,482][20347] Updated weights for policy 0, policy_version 1331644 (0.0006) [2023-12-27 14:52:30,515][20381] Updated weights for policy 1, policy_version 1330632 (0.0007) [2023-12-27 14:52:31,159][20347] Updated weights for policy 0, policy_version 1331654 (0.0008) [2023-12-27 14:52:31,190][20381] Updated weights for policy 1, policy_version 1330642 (0.0008) [2023-12-27 14:52:31,222][20347] Updated weights for policy 0, policy_version 1331664 (0.0008) [2023-12-27 14:52:31,248][20381] Updated weights for policy 1, policy_version 1330652 (0.0006) [2023-12-27 14:52:31,290][20347] Updated weights for policy 0, policy_version 1331674 (0.0008) [2023-12-27 14:52:31,308][20381] Updated weights for policy 1, policy_version 1330662 (0.0009) [2023-12-27 14:52:31,372][20381] Updated weights for policy 1, policy_version 1330672 (0.0008) [2023-12-27 14:52:32,026][20381] Updated weights for policy 1, policy_version 1330682 (0.0007) [2023-12-27 14:52:32,050][20347] Updated weights for policy 0, policy_version 1331684 (0.0009) [2023-12-27 14:52:32,082][20381] Updated weights for policy 1, policy_version 1330692 (0.0006) [2023-12-27 14:52:32,107][20347] Updated weights for policy 0, policy_version 1331694 (0.0006) [2023-12-27 14:52:32,136][20381] Updated weights for policy 1, policy_version 1330702 (0.0006) [2023-12-27 14:52:32,158][20347] Updated weights for policy 0, policy_version 1331704 (0.0006) [2023-12-27 14:52:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 681672704. Throughput: 0: 10264.2, 1: 10189.7. Samples: 681677976. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:32,363][19170] Avg episode reward: [(0, '756.340'), (1, '882.520')] [2023-12-27 14:52:32,818][20381] Updated weights for policy 1, policy_version 1330712 (0.0008) [2023-12-27 14:52:32,869][20381] Updated weights for policy 1, policy_version 1330722 (0.0008) [2023-12-27 14:52:32,907][20347] Updated weights for policy 0, policy_version 1331714 (0.0007) [2023-12-27 14:52:32,920][20381] Updated weights for policy 1, policy_version 1330732 (0.0007) [2023-12-27 14:52:32,955][20347] Updated weights for policy 0, policy_version 1331724 (0.0006) [2023-12-27 14:52:33,006][20347] Updated weights for policy 0, policy_version 1331734 (0.0007) [2023-12-27 14:52:33,057][20347] Updated weights for policy 0, policy_version 1331744 (0.0008) [2023-12-27 14:52:33,627][20381] Updated weights for policy 1, policy_version 1330742 (0.0007) [2023-12-27 14:52:33,691][20381] Updated weights for policy 1, policy_version 1330752 (0.0007) [2023-12-27 14:52:33,745][20381] Updated weights for policy 1, policy_version 1330762 (0.0007) [2023-12-27 14:52:33,772][20347] Updated weights for policy 0, policy_version 1331754 (0.0006) [2023-12-27 14:52:33,828][20347] Updated weights for policy 0, policy_version 1331764 (0.0007) [2023-12-27 14:52:33,885][20347] Updated weights for policy 0, policy_version 1331774 (0.0008) [2023-12-27 14:52:34,421][20381] Updated weights for policy 1, policy_version 1330772 (0.0006) [2023-12-27 14:52:34,478][20381] Updated weights for policy 1, policy_version 1330782 (0.0008) [2023-12-27 14:52:34,532][20381] Updated weights for policy 1, policy_version 1330792 (0.0007) [2023-12-27 14:52:34,570][20347] Updated weights for policy 0, policy_version 1331784 (0.0006) [2023-12-27 14:52:34,627][20347] Updated weights for policy 0, policy_version 1331794 (0.0007) [2023-12-27 14:52:34,688][20347] Updated weights for policy 0, policy_version 1331804 (0.0009) [2023-12-27 14:52:35,252][20347] Updated weights for policy 0, policy_version 1331814 (0.0009) [2023-12-27 14:52:35,275][20291] KL-divergence is very high: 159.5449 [2023-12-27 14:52:35,292][20381] Updated weights for policy 1, policy_version 1330802 (0.0008) [2023-12-27 14:52:35,300][20347] Updated weights for policy 0, policy_version 1331824 (0.0010) [2023-12-27 14:52:35,313][20291] KL-divergence is very high: 210.4996 [2023-12-27 14:52:35,347][20381] Updated weights for policy 1, policy_version 1330812 (0.0008) [2023-12-27 14:52:35,352][20347] Updated weights for policy 0, policy_version 1331834 (0.0010) [2023-12-27 14:52:35,360][20291] KL-divergence is very high: 140.7285 [2023-12-27 14:52:35,406][20381] Updated weights for policy 1, policy_version 1330822 (0.0007) [2023-12-27 14:52:35,452][20381] Updated weights for policy 1, policy_version 1330832 (0.0008) [2023-12-27 14:52:35,936][20347] Updated weights for policy 0, policy_version 1331844 (0.0005) [2023-12-27 14:52:35,992][20347] Updated weights for policy 0, policy_version 1331854 (0.0006) [2023-12-27 14:52:36,054][20347] Updated weights for policy 0, policy_version 1331864 (0.0007) [2023-12-27 14:52:36,210][20381] Updated weights for policy 1, policy_version 1330842 (0.0008) [2023-12-27 14:52:36,264][20381] Updated weights for policy 1, policy_version 1330852 (0.0008) [2023-12-27 14:52:36,314][20381] Updated weights for policy 1, policy_version 1330862 (0.0010) [2023-12-27 14:52:36,667][20347] Updated weights for policy 0, policy_version 1331874 (0.0008) [2023-12-27 14:52:36,729][20347] Updated weights for policy 0, policy_version 1331884 (0.0011) [2023-12-27 14:52:36,786][20347] Updated weights for policy 0, policy_version 1331894 (0.0010) [2023-12-27 14:52:36,846][20347] Updated weights for policy 0, policy_version 1331904 (0.0011) [2023-12-27 14:52:37,064][20381] Updated weights for policy 1, policy_version 1330872 (0.0008) [2023-12-27 14:52:37,128][20381] Updated weights for policy 1, policy_version 1330882 (0.0008) [2023-12-27 14:52:37,183][20381] Updated weights for policy 1, policy_version 1330892 (0.0007) [2023-12-27 14:52:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 681771008. Throughput: 0: 10274.0, 1: 10181.9. Samples: 681738356. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:37,363][19170] Avg episode reward: [(0, '834.810'), (1, '892.320')] [2023-12-27 14:52:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001330896_340738048.pth... [2023-12-27 14:52:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001331904_341032960.pth... [2023-12-27 14:52:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001329712_340434944.pth [2023-12-27 14:52:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001330688_340721664.pth [2023-12-27 14:52:37,480][20347] Updated weights for policy 0, policy_version 1331914 (0.0011) [2023-12-27 14:52:37,543][20347] Updated weights for policy 0, policy_version 1331924 (0.0006) [2023-12-27 14:52:37,611][20347] Updated weights for policy 0, policy_version 1331934 (0.0008) [2023-12-27 14:52:37,807][20381] Updated weights for policy 1, policy_version 1330902 (0.0006) [2023-12-27 14:52:37,861][20381] Updated weights for policy 1, policy_version 1330912 (0.0007) [2023-12-27 14:52:37,915][20381] Updated weights for policy 1, policy_version 1330922 (0.0007) [2023-12-27 14:52:38,243][20347] Updated weights for policy 0, policy_version 1331944 (0.0010) [2023-12-27 14:52:38,290][20347] Updated weights for policy 0, policy_version 1331954 (0.0010) [2023-12-27 14:52:38,360][20347] Updated weights for policy 0, policy_version 1331964 (0.0009) [2023-12-27 14:52:38,601][20381] Updated weights for policy 1, policy_version 1330932 (0.0007) [2023-12-27 14:52:38,661][20381] Updated weights for policy 1, policy_version 1330942 (0.0008) [2023-12-27 14:52:38,720][20381] Updated weights for policy 1, policy_version 1330952 (0.0008) [2023-12-27 14:52:39,059][20347] Updated weights for policy 0, policy_version 1331974 (0.0009) [2023-12-27 14:52:39,115][20347] Updated weights for policy 0, policy_version 1331984 (0.0010) [2023-12-27 14:52:39,174][20347] Updated weights for policy 0, policy_version 1331994 (0.0010) [2023-12-27 14:52:39,419][20381] Updated weights for policy 1, policy_version 1330962 (0.0007) [2023-12-27 14:52:39,486][20381] Updated weights for policy 1, policy_version 1330972 (0.0009) [2023-12-27 14:52:39,542][20381] Updated weights for policy 1, policy_version 1330982 (0.0008) [2023-12-27 14:52:39,602][20381] Updated weights for policy 1, policy_version 1330992 (0.0007) [2023-12-27 14:52:39,898][20347] Updated weights for policy 0, policy_version 1332004 (0.0011) [2023-12-27 14:52:39,959][20347] Updated weights for policy 0, policy_version 1332014 (0.0011) [2023-12-27 14:52:40,024][20347] Updated weights for policy 0, policy_version 1332024 (0.0011) [2023-12-27 14:52:40,338][20381] Updated weights for policy 1, policy_version 1331002 (0.0008) [2023-12-27 14:52:40,407][20381] Updated weights for policy 1, policy_version 1331012 (0.0007) [2023-12-27 14:52:40,471][20381] Updated weights for policy 1, policy_version 1331022 (0.0006) [2023-12-27 14:52:40,742][20347] Updated weights for policy 0, policy_version 1332034 (0.0009) [2023-12-27 14:52:40,790][20347] Updated weights for policy 0, policy_version 1332044 (0.0010) [2023-12-27 14:52:40,842][20347] Updated weights for policy 0, policy_version 1332054 (0.0007) [2023-12-27 14:52:40,892][20347] Updated weights for policy 0, policy_version 1332064 (0.0006) [2023-12-27 14:52:41,084][20381] Updated weights for policy 1, policy_version 1331032 (0.0010) [2023-12-27 14:52:41,145][20381] Updated weights for policy 1, policy_version 1331042 (0.0010) [2023-12-27 14:52:41,215][20381] Updated weights for policy 1, policy_version 1331052 (0.0007) [2023-12-27 14:52:41,646][20347] Updated weights for policy 0, policy_version 1332074 (0.0009) [2023-12-27 14:52:41,697][20347] Updated weights for policy 0, policy_version 1332084 (0.0009) [2023-12-27 14:52:41,766][20347] Updated weights for policy 0, policy_version 1332094 (0.0009) [2023-12-27 14:52:41,952][20381] Updated weights for policy 1, policy_version 1331062 (0.0007) [2023-12-27 14:52:42,015][20381] Updated weights for policy 1, policy_version 1331072 (0.0008) [2023-12-27 14:52:42,071][20381] Updated weights for policy 1, policy_version 1331082 (0.0007) [2023-12-27 14:52:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 681869312. Throughput: 0: 10326.6, 1: 10105.6. Samples: 681861700. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:42,363][19170] Avg episode reward: [(0, '805.070'), (1, '912.800')] [2023-12-27 14:52:42,503][20347] Updated weights for policy 0, policy_version 1332104 (0.0010) [2023-12-27 14:52:42,520][20291] KL-divergence is very high: 299.5227 [2023-12-27 14:52:42,526][20291] KL-divergence is very high: 127.8683 [2023-12-27 14:52:42,558][20347] Updated weights for policy 0, policy_version 1332114 (0.0008) [2023-12-27 14:52:42,610][20291] KL-divergence is very high: 164.6281 [2023-12-27 14:52:42,616][20347] Updated weights for policy 0, policy_version 1332124 (0.0009) [2023-12-27 14:52:42,617][20291] KL-divergence is very high: 147.8310 [2023-12-27 14:52:42,813][20381] Updated weights for policy 1, policy_version 1331092 (0.0007) [2023-12-27 14:52:42,878][20381] Updated weights for policy 1, policy_version 1331102 (0.0008) [2023-12-27 14:52:42,931][20381] Updated weights for policy 1, policy_version 1331112 (0.0010) [2023-12-27 14:52:43,220][20347] Updated weights for policy 0, policy_version 1332134 (0.0008) [2023-12-27 14:52:43,263][20291] KL-divergence is very high: 161.3595 [2023-12-27 14:52:43,274][20347] Updated weights for policy 0, policy_version 1332144 (0.0010) [2023-12-27 14:52:43,327][20347] Updated weights for policy 0, policy_version 1332154 (0.0008) [2023-12-27 14:52:43,506][20381] Updated weights for policy 1, policy_version 1331122 (0.0007) [2023-12-27 14:52:43,552][20381] Updated weights for policy 1, policy_version 1331132 (0.0008) [2023-12-27 14:52:43,598][20381] Updated weights for policy 1, policy_version 1331142 (0.0009) [2023-12-27 14:52:43,648][20381] Updated weights for policy 1, policy_version 1331152 (0.0010) [2023-12-27 14:52:43,877][20347] Updated weights for policy 0, policy_version 1332164 (0.0006) [2023-12-27 14:52:43,936][20347] Updated weights for policy 0, policy_version 1332174 (0.0007) [2023-12-27 14:52:43,986][20347] Updated weights for policy 0, policy_version 1332184 (0.0007) [2023-12-27 14:52:44,460][20381] Updated weights for policy 1, policy_version 1331162 (0.0008) [2023-12-27 14:52:44,512][20347] Updated weights for policy 0, policy_version 1332194 (0.0005) [2023-12-27 14:52:44,514][20381] Updated weights for policy 1, policy_version 1331172 (0.0009) [2023-12-27 14:52:44,570][20347] Updated weights for policy 0, policy_version 1332204 (0.0005) [2023-12-27 14:52:44,571][20381] Updated weights for policy 1, policy_version 1331182 (0.0008) [2023-12-27 14:52:44,630][20347] Updated weights for policy 0, policy_version 1332214 (0.0005) [2023-12-27 14:52:44,682][20347] Updated weights for policy 0, policy_version 1332224 (0.0006) [2023-12-27 14:52:45,302][20381] Updated weights for policy 1, policy_version 1331192 (0.0006) [2023-12-27 14:52:45,303][20347] Updated weights for policy 0, policy_version 1332234 (0.0008) [2023-12-27 14:52:45,348][20381] Updated weights for policy 1, policy_version 1331202 (0.0006) [2023-12-27 14:52:45,362][20347] Updated weights for policy 0, policy_version 1332244 (0.0008) [2023-12-27 14:52:45,402][20381] Updated weights for policy 1, policy_version 1331212 (0.0005) [2023-12-27 14:52:45,411][20347] Updated weights for policy 0, policy_version 1332254 (0.0006) [2023-12-27 14:52:46,032][20381] Updated weights for policy 1, policy_version 1331222 (0.0006) [2023-12-27 14:52:46,054][20347] Updated weights for policy 0, policy_version 1332264 (0.0005) [2023-12-27 14:52:46,078][20381] Updated weights for policy 1, policy_version 1331232 (0.0005) [2023-12-27 14:52:46,114][20347] Updated weights for policy 0, policy_version 1332274 (0.0005) [2023-12-27 14:52:46,129][20381] Updated weights for policy 1, policy_version 1331242 (0.0006) [2023-12-27 14:52:46,172][20347] Updated weights for policy 0, policy_version 1332284 (0.0007) [2023-12-27 14:52:46,718][20347] Updated weights for policy 0, policy_version 1332294 (0.0006) [2023-12-27 14:52:46,764][20347] Updated weights for policy 0, policy_version 1332304 (0.0005) [2023-12-27 14:52:46,795][20381] Updated weights for policy 1, policy_version 1331252 (0.0007) [2023-12-27 14:52:46,812][20347] Updated weights for policy 0, policy_version 1332314 (0.0005) [2023-12-27 14:52:46,852][20381] Updated weights for policy 1, policy_version 1331262 (0.0008) [2023-12-27 14:52:46,916][20381] Updated weights for policy 1, policy_version 1331272 (0.0008) [2023-12-27 14:52:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 681975808. Throughput: 0: 10449.0, 1: 10098.4. Samples: 681988740. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:47,363][19170] Avg episode reward: [(0, '756.580'), (1, '845.730')] [2023-12-27 14:52:47,378][20347] Updated weights for policy 0, policy_version 1332324 (0.0006) [2023-12-27 14:52:47,423][20347] Updated weights for policy 0, policy_version 1332334 (0.0008) [2023-12-27 14:52:47,471][20347] Updated weights for policy 0, policy_version 1332344 (0.0007) [2023-12-27 14:52:47,633][20381] Updated weights for policy 1, policy_version 1331282 (0.0008) [2023-12-27 14:52:47,682][20381] Updated weights for policy 1, policy_version 1331292 (0.0008) [2023-12-27 14:52:47,724][20381] Updated weights for policy 1, policy_version 1331302 (0.0007) [2023-12-27 14:52:47,773][20381] Updated weights for policy 1, policy_version 1331312 (0.0007) [2023-12-27 14:52:48,202][20347] Updated weights for policy 0, policy_version 1332354 (0.0008) [2023-12-27 14:52:48,257][20347] Updated weights for policy 0, policy_version 1332364 (0.0007) [2023-12-27 14:52:48,314][20347] Updated weights for policy 0, policy_version 1332374 (0.0008) [2023-12-27 14:52:48,376][20347] Updated weights for policy 0, policy_version 1332384 (0.0009) [2023-12-27 14:52:48,421][20381] Updated weights for policy 1, policy_version 1331322 (0.0009) [2023-12-27 14:52:48,483][20381] Updated weights for policy 1, policy_version 1331332 (0.0009) [2023-12-27 14:52:48,548][20381] Updated weights for policy 1, policy_version 1331342 (0.0009) [2023-12-27 14:52:49,108][20347] Updated weights for policy 0, policy_version 1332394 (0.0010) [2023-12-27 14:52:49,156][20347] Updated weights for policy 0, policy_version 1332404 (0.0010) [2023-12-27 14:52:49,204][20347] Updated weights for policy 0, policy_version 1332414 (0.0010) [2023-12-27 14:52:49,294][20381] Updated weights for policy 1, policy_version 1331352 (0.0007) [2023-12-27 14:52:49,352][20381] Updated weights for policy 1, policy_version 1331362 (0.0006) [2023-12-27 14:52:49,417][20381] Updated weights for policy 1, policy_version 1331372 (0.0008) [2023-12-27 14:52:49,903][20347] Updated weights for policy 0, policy_version 1332424 (0.0010) [2023-12-27 14:52:49,965][20347] Updated weights for policy 0, policy_version 1332434 (0.0007) [2023-12-27 14:52:50,017][20347] Updated weights for policy 0, policy_version 1332444 (0.0007) [2023-12-27 14:52:50,134][20381] Updated weights for policy 1, policy_version 1331382 (0.0007) [2023-12-27 14:52:50,193][20381] Updated weights for policy 1, policy_version 1331392 (0.0008) [2023-12-27 14:52:50,252][20381] Updated weights for policy 1, policy_version 1331402 (0.0008) [2023-12-27 14:52:50,728][20347] Updated weights for policy 0, policy_version 1332454 (0.0008) [2023-12-27 14:52:50,797][20347] Updated weights for policy 0, policy_version 1332464 (0.0008) [2023-12-27 14:52:50,860][20347] Updated weights for policy 0, policy_version 1332474 (0.0007) [2023-12-27 14:52:50,904][20381] Updated weights for policy 1, policy_version 1331412 (0.0008) [2023-12-27 14:52:50,954][20381] Updated weights for policy 1, policy_version 1331422 (0.0007) [2023-12-27 14:52:51,009][20381] Updated weights for policy 1, policy_version 1331432 (0.0008) [2023-12-27 14:52:51,505][20347] Updated weights for policy 0, policy_version 1332484 (0.0007) [2023-12-27 14:52:51,562][20347] Updated weights for policy 0, policy_version 1332494 (0.0008) [2023-12-27 14:52:51,620][20347] Updated weights for policy 0, policy_version 1332504 (0.0008) [2023-12-27 14:52:51,759][20381] Updated weights for policy 1, policy_version 1331442 (0.0009) [2023-12-27 14:52:51,821][20381] Updated weights for policy 1, policy_version 1331452 (0.0007) [2023-12-27 14:52:51,881][20381] Updated weights for policy 1, policy_version 1331462 (0.0008) [2023-12-27 14:52:51,941][20381] Updated weights for policy 1, policy_version 1331472 (0.0007) [2023-12-27 14:52:52,336][20347] Updated weights for policy 0, policy_version 1332514 (0.0008) [2023-12-27 14:52:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 682074112. Throughput: 0: 10421.2, 1: 10090.7. Samples: 682048392. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:52,363][19170] Avg episode reward: [(0, '785.570'), (1, '835.160')] [2023-12-27 14:52:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001331472_340885504.pth... [2023-12-27 14:52:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001330320_340590592.pth [2023-12-27 14:52:52,393][20347] Updated weights for policy 0, policy_version 1332524 (0.0008) [2023-12-27 14:52:52,455][20347] Updated weights for policy 0, policy_version 1332534 (0.0008) [2023-12-27 14:52:52,509][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001332544_341196800.pth... [2023-12-27 14:52:52,509][20347] Updated weights for policy 0, policy_version 1332544 (0.0008) [2023-12-27 14:52:52,513][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001331296_340877312.pth [2023-12-27 14:52:52,631][20381] Updated weights for policy 1, policy_version 1331482 (0.0008) [2023-12-27 14:52:52,695][20381] Updated weights for policy 1, policy_version 1331492 (0.0008) [2023-12-27 14:52:52,763][20381] Updated weights for policy 1, policy_version 1331502 (0.0008) [2023-12-27 14:52:53,228][20347] Updated weights for policy 0, policy_version 1332554 (0.0008) [2023-12-27 14:52:53,286][20347] Updated weights for policy 0, policy_version 1332564 (0.0005) [2023-12-27 14:52:53,323][20381] Updated weights for policy 1, policy_version 1331512 (0.0007) [2023-12-27 14:52:53,344][20347] Updated weights for policy 0, policy_version 1332574 (0.0005) [2023-12-27 14:52:53,367][20381] Updated weights for policy 1, policy_version 1331522 (0.0008) [2023-12-27 14:52:53,413][20381] Updated weights for policy 1, policy_version 1331532 (0.0006) [2023-12-27 14:52:53,931][20347] Updated weights for policy 0, policy_version 1332584 (0.0006) [2023-12-27 14:52:53,985][20347] Updated weights for policy 0, policy_version 1332594 (0.0005) [2023-12-27 14:52:54,031][20347] Updated weights for policy 0, policy_version 1332604 (0.0005) [2023-12-27 14:52:54,051][20381] Updated weights for policy 1, policy_version 1331542 (0.0009) [2023-12-27 14:52:54,102][20381] Updated weights for policy 1, policy_version 1331552 (0.0010) [2023-12-27 14:52:54,153][20381] Updated weights for policy 1, policy_version 1331562 (0.0011) [2023-12-27 14:52:54,683][20347] Updated weights for policy 0, policy_version 1332614 (0.0007) [2023-12-27 14:52:54,731][20347] Updated weights for policy 0, policy_version 1332624 (0.0007) [2023-12-27 14:52:54,780][20347] Updated weights for policy 0, policy_version 1332634 (0.0007) [2023-12-27 14:52:54,850][20381] Updated weights for policy 1, policy_version 1331572 (0.0010) [2023-12-27 14:52:54,905][20381] Updated weights for policy 1, policy_version 1331582 (0.0010) [2023-12-27 14:52:54,961][20381] Updated weights for policy 1, policy_version 1331592 (0.0010) [2023-12-27 14:52:55,429][20347] Updated weights for policy 0, policy_version 1332644 (0.0007) [2023-12-27 14:52:55,480][20347] Updated weights for policy 0, policy_version 1332654 (0.0007) [2023-12-27 14:52:55,535][20347] Updated weights for policy 0, policy_version 1332664 (0.0007) [2023-12-27 14:52:55,642][20381] Updated weights for policy 1, policy_version 1331602 (0.0011) [2023-12-27 14:52:55,700][20381] Updated weights for policy 1, policy_version 1331612 (0.0010) [2023-12-27 14:52:55,748][20381] Updated weights for policy 1, policy_version 1331622 (0.0008) [2023-12-27 14:52:55,799][20381] Updated weights for policy 1, policy_version 1331632 (0.0010) [2023-12-27 14:52:56,244][20347] Updated weights for policy 0, policy_version 1332674 (0.0008) [2023-12-27 14:52:56,292][20347] Updated weights for policy 0, policy_version 1332684 (0.0007) [2023-12-27 14:52:56,338][20347] Updated weights for policy 0, policy_version 1332694 (0.0007) [2023-12-27 14:52:56,390][20347] Updated weights for policy 0, policy_version 1332704 (0.0008) [2023-12-27 14:52:56,419][20381] Updated weights for policy 1, policy_version 1331642 (0.0010) [2023-12-27 14:52:56,468][20381] Updated weights for policy 1, policy_version 1331652 (0.0010) [2023-12-27 14:52:56,526][20381] Updated weights for policy 1, policy_version 1331662 (0.0011) [2023-12-27 14:52:57,054][20347] Updated weights for policy 0, policy_version 1332714 (0.0008) [2023-12-27 14:52:57,106][20347] Updated weights for policy 0, policy_version 1332724 (0.0007) [2023-12-27 14:52:57,145][20381] Updated weights for policy 1, policy_version 1331672 (0.0011) [2023-12-27 14:52:57,154][20347] Updated weights for policy 0, policy_version 1332734 (0.0005) [2023-12-27 14:52:57,194][20381] Updated weights for policy 1, policy_version 1331682 (0.0010) [2023-12-27 14:52:57,242][20381] Updated weights for policy 1, policy_version 1331692 (0.0010) [2023-12-27 14:52:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 682188800. Throughput: 0: 10473.2, 1: 10125.8. Samples: 682173184. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:52:57,363][19170] Avg episode reward: [(0, '844.440'), (1, '825.740')] [2023-12-27 14:52:57,784][20347] Updated weights for policy 0, policy_version 1332744 (0.0009) [2023-12-27 14:52:57,835][20347] Updated weights for policy 0, policy_version 1332754 (0.0010) [2023-12-27 14:52:57,892][20347] Updated weights for policy 0, policy_version 1332764 (0.0011) [2023-12-27 14:52:57,947][20381] Updated weights for policy 1, policy_version 1331702 (0.0010) [2023-12-27 14:52:58,005][20381] Updated weights for policy 1, policy_version 1331712 (0.0010) [2023-12-27 14:52:58,049][20381] Updated weights for policy 1, policy_version 1331722 (0.0007) [2023-12-27 14:52:58,560][20347] Updated weights for policy 0, policy_version 1332774 (0.0008) [2023-12-27 14:52:58,622][20347] Updated weights for policy 0, policy_version 1332784 (0.0008) [2023-12-27 14:52:58,687][20347] Updated weights for policy 0, policy_version 1332794 (0.0008) [2023-12-27 14:52:58,744][20381] Updated weights for policy 1, policy_version 1331732 (0.0007) [2023-12-27 14:52:58,810][20381] Updated weights for policy 1, policy_version 1331742 (0.0008) [2023-12-27 14:52:58,874][20381] Updated weights for policy 1, policy_version 1331752 (0.0007) [2023-12-27 14:52:59,442][20347] Updated weights for policy 0, policy_version 1332804 (0.0008) [2023-12-27 14:52:59,499][20347] Updated weights for policy 0, policy_version 1332814 (0.0008) [2023-12-27 14:52:59,562][20347] Updated weights for policy 0, policy_version 1332824 (0.0008) [2023-12-27 14:52:59,625][20381] Updated weights for policy 1, policy_version 1331762 (0.0009) [2023-12-27 14:52:59,689][20381] Updated weights for policy 1, policy_version 1331772 (0.0008) [2023-12-27 14:52:59,748][20381] Updated weights for policy 1, policy_version 1331782 (0.0007) [2023-12-27 14:52:59,809][20381] Updated weights for policy 1, policy_version 1331792 (0.0007) [2023-12-27 14:53:00,273][20347] Updated weights for policy 0, policy_version 1332834 (0.0008) [2023-12-27 14:53:00,333][20347] Updated weights for policy 0, policy_version 1332844 (0.0008) [2023-12-27 14:53:00,397][20347] Updated weights for policy 0, policy_version 1332854 (0.0008) [2023-12-27 14:53:00,453][20347] Updated weights for policy 0, policy_version 1332864 (0.0007) [2023-12-27 14:53:00,486][20381] Updated weights for policy 1, policy_version 1331802 (0.0008) [2023-12-27 14:53:00,549][20381] Updated weights for policy 1, policy_version 1331812 (0.0008) [2023-12-27 14:53:00,617][20381] Updated weights for policy 1, policy_version 1331822 (0.0008) [2023-12-27 14:53:01,072][20347] Updated weights for policy 0, policy_version 1332874 (0.0007) [2023-12-27 14:53:01,133][20347] Updated weights for policy 0, policy_version 1332884 (0.0008) [2023-12-27 14:53:01,201][20347] Updated weights for policy 0, policy_version 1332894 (0.0008) [2023-12-27 14:53:01,281][20381] Updated weights for policy 1, policy_version 1331832 (0.0010) [2023-12-27 14:53:01,328][20381] Updated weights for policy 1, policy_version 1331842 (0.0008) [2023-12-27 14:53:01,394][20381] Updated weights for policy 1, policy_version 1331852 (0.0008) [2023-12-27 14:53:01,937][20347] Updated weights for policy 0, policy_version 1332904 (0.0008) [2023-12-27 14:53:01,990][20347] Updated weights for policy 0, policy_version 1332914 (0.0008) [2023-12-27 14:53:02,039][20347] Updated weights for policy 0, policy_version 1332924 (0.0007) [2023-12-27 14:53:02,089][20381] Updated weights for policy 1, policy_version 1331862 (0.0008) [2023-12-27 14:53:02,142][20381] Updated weights for policy 1, policy_version 1331872 (0.0007) [2023-12-27 14:53:02,194][20381] Updated weights for policy 1, policy_version 1331882 (0.0008) [2023-12-27 14:53:02,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 682287104. Throughput: 0: 10412.7, 1: 10110.0. Samples: 682293572. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:02,364][19170] Avg episode reward: [(0, '873.690'), (1, '845.440')] [2023-12-27 14:53:02,784][20347] Updated weights for policy 0, policy_version 1332934 (0.0008) [2023-12-27 14:53:02,846][20347] Updated weights for policy 0, policy_version 1332944 (0.0008) [2023-12-27 14:53:02,879][20381] Updated weights for policy 1, policy_version 1331892 (0.0007) [2023-12-27 14:53:02,903][20347] Updated weights for policy 0, policy_version 1332954 (0.0008) [2023-12-27 14:53:02,941][20381] Updated weights for policy 1, policy_version 1331902 (0.0005) [2023-12-27 14:53:02,993][20381] Updated weights for policy 1, policy_version 1331912 (0.0008) [2023-12-27 14:53:03,511][20347] Updated weights for policy 0, policy_version 1332964 (0.0006) [2023-12-27 14:53:03,574][20347] Updated weights for policy 0, policy_version 1332974 (0.0008) [2023-12-27 14:53:03,633][20347] Updated weights for policy 0, policy_version 1332984 (0.0008) [2023-12-27 14:53:03,671][20381] Updated weights for policy 1, policy_version 1331922 (0.0007) [2023-12-27 14:53:03,735][20381] Updated weights for policy 1, policy_version 1331932 (0.0008) [2023-12-27 14:53:03,786][20381] Updated weights for policy 1, policy_version 1331942 (0.0008) [2023-12-27 14:53:03,832][20381] Updated weights for policy 1, policy_version 1331952 (0.0008) [2023-12-27 14:53:04,312][20347] Updated weights for policy 0, policy_version 1332994 (0.0007) [2023-12-27 14:53:04,372][20347] Updated weights for policy 0, policy_version 1333004 (0.0007) [2023-12-27 14:53:04,429][20347] Updated weights for policy 0, policy_version 1333014 (0.0007) [2023-12-27 14:53:04,485][20347] Updated weights for policy 0, policy_version 1333024 (0.0008) [2023-12-27 14:53:04,522][20381] Updated weights for policy 1, policy_version 1331962 (0.0007) [2023-12-27 14:53:04,576][20381] Updated weights for policy 1, policy_version 1331972 (0.0007) [2023-12-27 14:53:04,632][20381] Updated weights for policy 1, policy_version 1331982 (0.0006) [2023-12-27 14:53:05,172][20347] Updated weights for policy 0, policy_version 1333034 (0.0008) [2023-12-27 14:53:05,225][20347] Updated weights for policy 0, policy_version 1333044 (0.0009) [2023-12-27 14:53:05,283][20347] Updated weights for policy 0, policy_version 1333054 (0.0010) [2023-12-27 14:53:05,289][20381] Updated weights for policy 1, policy_version 1331992 (0.0008) [2023-12-27 14:53:05,307][20292] KL-divergence is very high: 166.1872 [2023-12-27 14:53:05,338][20381] Updated weights for policy 1, policy_version 1332002 (0.0008) [2023-12-27 14:53:05,349][20292] KL-divergence is very high: 168.2466 [2023-12-27 14:53:05,389][20381] Updated weights for policy 1, policy_version 1332012 (0.0007) [2023-12-27 14:53:05,966][20347] Updated weights for policy 0, policy_version 1333064 (0.0010) [2023-12-27 14:53:06,028][20347] Updated weights for policy 0, policy_version 1333074 (0.0007) [2023-12-27 14:53:06,091][20347] Updated weights for policy 0, policy_version 1333084 (0.0007) [2023-12-27 14:53:06,108][20381] Updated weights for policy 1, policy_version 1332022 (0.0009) [2023-12-27 14:53:06,171][20381] Updated weights for policy 1, policy_version 1332032 (0.0010) [2023-12-27 14:53:06,232][20381] Updated weights for policy 1, policy_version 1332042 (0.0011) [2023-12-27 14:53:06,758][20347] Updated weights for policy 0, policy_version 1333094 (0.0006) [2023-12-27 14:53:06,808][20347] Updated weights for policy 0, policy_version 1333104 (0.0005) [2023-12-27 14:53:06,872][20347] Updated weights for policy 0, policy_version 1333114 (0.0005) [2023-12-27 14:53:06,903][20381] Updated weights for policy 1, policy_version 1332052 (0.0011) [2023-12-27 14:53:06,956][20381] Updated weights for policy 1, policy_version 1332062 (0.0011) [2023-12-27 14:53:07,015][20381] Updated weights for policy 1, policy_version 1332072 (0.0011) [2023-12-27 14:53:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 682385408. Throughput: 0: 10399.9, 1: 10130.0. Samples: 682355444. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:07,364][19170] Avg episode reward: [(0, '863.910'), (1, '833.930')] [2023-12-27 14:53:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001332080_341041152.pth... [2023-12-27 14:53:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001333120_341344256.pth... [2023-12-27 14:53:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001330896_340738048.pth [2023-12-27 14:53:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001331904_341032960.pth [2023-12-27 14:53:07,449][20347] Updated weights for policy 0, policy_version 1333124 (0.0005) [2023-12-27 14:53:07,511][20347] Updated weights for policy 0, policy_version 1333134 (0.0005) [2023-12-27 14:53:07,572][20347] Updated weights for policy 0, policy_version 1333144 (0.0005) [2023-12-27 14:53:07,707][20381] Updated weights for policy 1, policy_version 1332082 (0.0011) [2023-12-27 14:53:07,757][20381] Updated weights for policy 1, policy_version 1332092 (0.0011) [2023-12-27 14:53:07,806][20381] Updated weights for policy 1, policy_version 1332102 (0.0010) [2023-12-27 14:53:07,858][20381] Updated weights for policy 1, policy_version 1332112 (0.0011) [2023-12-27 14:53:08,120][20347] Updated weights for policy 0, policy_version 1333154 (0.0006) [2023-12-27 14:53:08,183][20347] Updated weights for policy 0, policy_version 1333164 (0.0010) [2023-12-27 14:53:08,233][20347] Updated weights for policy 0, policy_version 1333174 (0.0008) [2023-12-27 14:53:08,280][20347] Updated weights for policy 0, policy_version 1333184 (0.0010) [2023-12-27 14:53:08,544][20381] Updated weights for policy 1, policy_version 1332122 (0.0011) [2023-12-27 14:53:08,600][20381] Updated weights for policy 1, policy_version 1332132 (0.0010) [2023-12-27 14:53:08,651][20381] Updated weights for policy 1, policy_version 1332142 (0.0010) [2023-12-27 14:53:09,014][20347] Updated weights for policy 0, policy_version 1333194 (0.0005) [2023-12-27 14:53:09,065][20347] Updated weights for policy 0, policy_version 1333204 (0.0007) [2023-12-27 14:53:09,124][20347] Updated weights for policy 0, policy_version 1333214 (0.0005) [2023-12-27 14:53:09,336][20381] Updated weights for policy 1, policy_version 1332152 (0.0007) [2023-12-27 14:53:09,409][20381] Updated weights for policy 1, policy_version 1332162 (0.0007) [2023-12-27 14:53:09,475][20381] Updated weights for policy 1, policy_version 1332172 (0.0007) [2023-12-27 14:53:09,817][20347] Updated weights for policy 0, policy_version 1333224 (0.0008) [2023-12-27 14:53:09,893][20347] Updated weights for policy 0, policy_version 1333234 (0.0008) [2023-12-27 14:53:09,955][20347] Updated weights for policy 0, policy_version 1333244 (0.0008) [2023-12-27 14:53:10,162][20381] Updated weights for policy 1, policy_version 1332182 (0.0008) [2023-12-27 14:53:10,225][20381] Updated weights for policy 1, policy_version 1332192 (0.0008) [2023-12-27 14:53:10,285][20381] Updated weights for policy 1, policy_version 1332202 (0.0008) [2023-12-27 14:53:10,639][20347] Updated weights for policy 0, policy_version 1333254 (0.0008) [2023-12-27 14:53:10,688][20347] Updated weights for policy 0, policy_version 1333264 (0.0008) [2023-12-27 14:53:10,739][20347] Updated weights for policy 0, policy_version 1333274 (0.0007) [2023-12-27 14:53:10,970][20381] Updated weights for policy 1, policy_version 1332212 (0.0007) [2023-12-27 14:53:11,033][20381] Updated weights for policy 1, policy_version 1332222 (0.0006) [2023-12-27 14:53:11,103][20381] Updated weights for policy 1, policy_version 1332232 (0.0008) [2023-12-27 14:53:11,491][20347] Updated weights for policy 0, policy_version 1333284 (0.0008) [2023-12-27 14:53:11,551][20347] Updated weights for policy 0, policy_version 1333294 (0.0008) [2023-12-27 14:53:11,610][20347] Updated weights for policy 0, policy_version 1333304 (0.0008) [2023-12-27 14:53:11,798][20381] Updated weights for policy 1, policy_version 1332242 (0.0008) [2023-12-27 14:53:11,851][20381] Updated weights for policy 1, policy_version 1332252 (0.0007) [2023-12-27 14:53:11,900][20381] Updated weights for policy 1, policy_version 1332262 (0.0007) [2023-12-27 14:53:11,951][20381] Updated weights for policy 1, policy_version 1332272 (0.0007) [2023-12-27 14:53:12,359][20347] Updated weights for policy 0, policy_version 1333314 (0.0008) [2023-12-27 14:53:12,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 682483712. Throughput: 0: 10437.8, 1: 10099.5. Samples: 682477536. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:12,363][19170] Avg episode reward: [(0, '863.990'), (1, '892.510')] [2023-12-27 14:53:12,423][20347] Updated weights for policy 0, policy_version 1333324 (0.0008) [2023-12-27 14:53:12,480][20347] Updated weights for policy 0, policy_version 1333334 (0.0007) [2023-12-27 14:53:12,536][20347] Updated weights for policy 0, policy_version 1333344 (0.0007) [2023-12-27 14:53:12,661][20381] Updated weights for policy 1, policy_version 1332282 (0.0007) [2023-12-27 14:53:12,719][20381] Updated weights for policy 1, policy_version 1332292 (0.0006) [2023-12-27 14:53:12,780][20381] Updated weights for policy 1, policy_version 1332302 (0.0006) [2023-12-27 14:53:13,256][20347] Updated weights for policy 0, policy_version 1333354 (0.0007) [2023-12-27 14:53:13,309][20347] Updated weights for policy 0, policy_version 1333364 (0.0007) [2023-12-27 14:53:13,358][20347] Updated weights for policy 0, policy_version 1333374 (0.0010) [2023-12-27 14:53:13,397][20381] Updated weights for policy 1, policy_version 1332312 (0.0009) [2023-12-27 14:53:13,455][20381] Updated weights for policy 1, policy_version 1332322 (0.0007) [2023-12-27 14:53:13,517][20381] Updated weights for policy 1, policy_version 1332332 (0.0009) [2023-12-27 14:53:14,059][20347] Updated weights for policy 0, policy_version 1333384 (0.0010) [2023-12-27 14:53:14,116][20347] Updated weights for policy 0, policy_version 1333394 (0.0010) [2023-12-27 14:53:14,165][20381] Updated weights for policy 1, policy_version 1332342 (0.0011) [2023-12-27 14:53:14,174][20347] Updated weights for policy 0, policy_version 1333404 (0.0010) [2023-12-27 14:53:14,214][20381] Updated weights for policy 1, policy_version 1332352 (0.0005) [2023-12-27 14:53:14,267][20381] Updated weights for policy 1, policy_version 1332362 (0.0005) [2023-12-27 14:53:14,864][20347] Updated weights for policy 0, policy_version 1333414 (0.0009) [2023-12-27 14:53:14,891][20381] Updated weights for policy 1, policy_version 1332372 (0.0007) [2023-12-27 14:53:14,923][20347] Updated weights for policy 0, policy_version 1333424 (0.0009) [2023-12-27 14:53:14,928][20291] KL-divergence is very high: 417.0337 [2023-12-27 14:53:14,948][20381] Updated weights for policy 1, policy_version 1332382 (0.0011) [2023-12-27 14:53:14,973][20291] KL-divergence is very high: 281.2147 [2023-12-27 14:53:14,978][20347] Updated weights for policy 0, policy_version 1333434 (0.0006) [2023-12-27 14:53:15,010][20381] Updated weights for policy 1, policy_version 1332392 (0.0009) [2023-12-27 14:53:15,654][20347] Updated weights for policy 0, policy_version 1333444 (0.0007) [2023-12-27 14:53:15,706][20347] Updated weights for policy 0, policy_version 1333454 (0.0007) [2023-12-27 14:53:15,708][20381] Updated weights for policy 1, policy_version 1332402 (0.0010) [2023-12-27 14:53:15,758][20347] Updated weights for policy 0, policy_version 1333464 (0.0005) [2023-12-27 14:53:15,763][20381] Updated weights for policy 1, policy_version 1332412 (0.0006) [2023-12-27 14:53:15,818][20381] Updated weights for policy 1, policy_version 1332422 (0.0006) [2023-12-27 14:53:15,873][20381] Updated weights for policy 1, policy_version 1332432 (0.0005) [2023-12-27 14:53:16,468][20347] Updated weights for policy 0, policy_version 1333474 (0.0006) [2023-12-27 14:53:16,504][20381] Updated weights for policy 1, policy_version 1332442 (0.0007) [2023-12-27 14:53:16,513][20347] Updated weights for policy 0, policy_version 1333484 (0.0006) [2023-12-27 14:53:16,561][20347] Updated weights for policy 0, policy_version 1333494 (0.0006) [2023-12-27 14:53:16,562][20381] Updated weights for policy 1, policy_version 1332452 (0.0007) [2023-12-27 14:53:16,573][20291] KL-divergence is very high: 448.0978 [2023-12-27 14:53:16,615][20347] Updated weights for policy 0, policy_version 1333504 (0.0008) [2023-12-27 14:53:16,620][20381] Updated weights for policy 1, policy_version 1332462 (0.0008) [2023-12-27 14:53:17,166][20381] Updated weights for policy 1, policy_version 1332472 (0.0005) [2023-12-27 14:53:17,224][20381] Updated weights for policy 1, policy_version 1332482 (0.0008) [2023-12-27 14:53:17,278][20381] Updated weights for policy 1, policy_version 1332492 (0.0007) [2023-12-27 14:53:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 682590208. Throughput: 0: 10319.8, 1: 10168.8. Samples: 682599964. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:17,363][19170] Avg episode reward: [(0, '892.810'), (1, '951.100')] [2023-12-27 14:53:17,369][20347] Updated weights for policy 0, policy_version 1333514 (0.0008) [2023-12-27 14:53:17,415][20347] Updated weights for policy 0, policy_version 1333524 (0.0008) [2023-12-27 14:53:17,463][20347] Updated weights for policy 0, policy_version 1333534 (0.0008) [2023-12-27 14:53:17,948][20381] Updated weights for policy 1, policy_version 1332502 (0.0006) [2023-12-27 14:53:18,011][20381] Updated weights for policy 1, policy_version 1332512 (0.0006) [2023-12-27 14:53:18,059][20381] Updated weights for policy 1, policy_version 1332522 (0.0007) [2023-12-27 14:53:18,171][20347] Updated weights for policy 0, policy_version 1333544 (0.0007) [2023-12-27 14:53:18,226][20347] Updated weights for policy 0, policy_version 1333554 (0.0007) [2023-12-27 14:53:18,273][20347] Updated weights for policy 0, policy_version 1333564 (0.0007) [2023-12-27 14:53:18,739][20381] Updated weights for policy 1, policy_version 1332532 (0.0008) [2023-12-27 14:53:18,793][20381] Updated weights for policy 1, policy_version 1332542 (0.0008) [2023-12-27 14:53:18,853][20381] Updated weights for policy 1, policy_version 1332552 (0.0009) [2023-12-27 14:53:19,002][20347] Updated weights for policy 0, policy_version 1333574 (0.0007) [2023-12-27 14:53:19,048][20347] Updated weights for policy 0, policy_version 1333584 (0.0007) [2023-12-27 14:53:19,102][20347] Updated weights for policy 0, policy_version 1333594 (0.0008) [2023-12-27 14:53:19,115][20291] KL-divergence is very high: 112.4616 [2023-12-27 14:53:19,539][20381] Updated weights for policy 1, policy_version 1332562 (0.0007) [2023-12-27 14:53:19,598][20381] Updated weights for policy 1, policy_version 1332572 (0.0006) [2023-12-27 14:53:19,660][20381] Updated weights for policy 1, policy_version 1332582 (0.0006) [2023-12-27 14:53:19,719][20381] Updated weights for policy 1, policy_version 1332592 (0.0005) [2023-12-27 14:53:19,843][20347] Updated weights for policy 0, policy_version 1333604 (0.0008) [2023-12-27 14:53:19,907][20347] Updated weights for policy 0, policy_version 1333614 (0.0008) [2023-12-27 14:53:19,935][20291] KL-divergence is very high: 103.1088 [2023-12-27 14:53:19,970][20347] Updated weights for policy 0, policy_version 1333624 (0.0008) [2023-12-27 14:53:20,390][20381] Updated weights for policy 1, policy_version 1332602 (0.0008) [2023-12-27 14:53:20,450][20381] Updated weights for policy 1, policy_version 1332612 (0.0009) [2023-12-27 14:53:20,513][20381] Updated weights for policy 1, policy_version 1332622 (0.0007) [2023-12-27 14:53:20,624][20347] Updated weights for policy 0, policy_version 1333634 (0.0007) [2023-12-27 14:53:20,685][20347] Updated weights for policy 0, policy_version 1333644 (0.0007) [2023-12-27 14:53:20,743][20347] Updated weights for policy 0, policy_version 1333654 (0.0007) [2023-12-27 14:53:20,804][20347] Updated weights for policy 0, policy_version 1333664 (0.0007) [2023-12-27 14:53:21,287][20381] Updated weights for policy 1, policy_version 1332632 (0.0009) [2023-12-27 14:53:21,347][20381] Updated weights for policy 1, policy_version 1332642 (0.0007) [2023-12-27 14:53:21,418][20381] Updated weights for policy 1, policy_version 1332652 (0.0008) [2023-12-27 14:53:21,550][20347] Updated weights for policy 0, policy_version 1333674 (0.0008) [2023-12-27 14:53:21,625][20347] Updated weights for policy 0, policy_version 1333684 (0.0008) [2023-12-27 14:53:21,689][20347] Updated weights for policy 0, policy_version 1333694 (0.0007) [2023-12-27 14:53:22,093][20381] Updated weights for policy 1, policy_version 1332662 (0.0007) [2023-12-27 14:53:22,163][20381] Updated weights for policy 1, policy_version 1332672 (0.0007) [2023-12-27 14:53:22,226][20381] Updated weights for policy 1, policy_version 1332682 (0.0006) [2023-12-27 14:53:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 682688512. Throughput: 0: 10286.9, 1: 10187.4. Samples: 682659696. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:22,363][19170] Avg episode reward: [(0, '363.440'), (1, '941.460')] [2023-12-27 14:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001332688_341196800.pth... [2023-12-27 14:53:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001331472_340885504.pth [2023-12-27 14:53:22,385][20347] Updated weights for policy 0, policy_version 1333704 (0.0012) [2023-12-27 14:53:22,451][20347] Updated weights for policy 0, policy_version 1333714 (0.0008) [2023-12-27 14:53:22,525][20347] Updated weights for policy 0, policy_version 1333724 (0.0008) [2023-12-27 14:53:22,549][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001333728_341499904.pth... [2023-12-27 14:53:22,553][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001332544_341196800.pth [2023-12-27 14:53:22,813][20381] Updated weights for policy 1, policy_version 1332692 (0.0008) [2023-12-27 14:53:22,870][20381] Updated weights for policy 1, policy_version 1332702 (0.0007) [2023-12-27 14:53:22,929][20381] Updated weights for policy 1, policy_version 1332712 (0.0006) [2023-12-27 14:53:23,221][20347] Updated weights for policy 0, policy_version 1333734 (0.0007) [2023-12-27 14:53:23,284][20347] Updated weights for policy 0, policy_version 1333744 (0.0008) [2023-12-27 14:53:23,346][20347] Updated weights for policy 0, policy_version 1333754 (0.0008) [2023-12-27 14:53:23,608][20381] Updated weights for policy 1, policy_version 1332722 (0.0006) [2023-12-27 14:53:23,654][20381] Updated weights for policy 1, policy_version 1332732 (0.0005) [2023-12-27 14:53:23,707][20381] Updated weights for policy 1, policy_version 1332742 (0.0005) [2023-12-27 14:53:23,759][20381] Updated weights for policy 1, policy_version 1332752 (0.0005) [2023-12-27 14:53:24,028][20347] Updated weights for policy 0, policy_version 1333764 (0.0009) [2023-12-27 14:53:24,080][20347] Updated weights for policy 0, policy_version 1333774 (0.0007) [2023-12-27 14:53:24,136][20347] Updated weights for policy 0, policy_version 1333784 (0.0007) [2023-12-27 14:53:24,386][20381] Updated weights for policy 1, policy_version 1332762 (0.0008) [2023-12-27 14:53:24,446][20381] Updated weights for policy 1, policy_version 1332772 (0.0007) [2023-12-27 14:53:24,492][20381] Updated weights for policy 1, policy_version 1332782 (0.0008) [2023-12-27 14:53:24,829][20347] Updated weights for policy 0, policy_version 1333794 (0.0008) [2023-12-27 14:53:24,887][20347] Updated weights for policy 0, policy_version 1333804 (0.0010) [2023-12-27 14:53:24,941][20347] Updated weights for policy 0, policy_version 1333814 (0.0010) [2023-12-27 14:53:25,001][20347] Updated weights for policy 0, policy_version 1333824 (0.0010) [2023-12-27 14:53:25,181][20381] Updated weights for policy 1, policy_version 1332792 (0.0010) [2023-12-27 14:53:25,229][20381] Updated weights for policy 1, policy_version 1332802 (0.0010) [2023-12-27 14:53:25,275][20381] Updated weights for policy 1, policy_version 1332812 (0.0010) [2023-12-27 14:53:25,647][20347] Updated weights for policy 0, policy_version 1333834 (0.0010) [2023-12-27 14:53:25,695][20347] Updated weights for policy 0, policy_version 1333844 (0.0010) [2023-12-27 14:53:25,743][20347] Updated weights for policy 0, policy_version 1333854 (0.0010) [2023-12-27 14:53:25,957][20381] Updated weights for policy 1, policy_version 1332822 (0.0009) [2023-12-27 14:53:26,017][20381] Updated weights for policy 1, policy_version 1332832 (0.0008) [2023-12-27 14:53:26,077][20381] Updated weights for policy 1, policy_version 1332842 (0.0006) [2023-12-27 14:53:26,379][20347] Updated weights for policy 0, policy_version 1333864 (0.0006) [2023-12-27 14:53:26,432][20347] Updated weights for policy 0, policy_version 1333874 (0.0005) [2023-12-27 14:53:26,483][20347] Updated weights for policy 0, policy_version 1333884 (0.0005) [2023-12-27 14:53:26,715][20381] Updated weights for policy 1, policy_version 1332852 (0.0006) [2023-12-27 14:53:26,767][20381] Updated weights for policy 1, policy_version 1332862 (0.0006) [2023-12-27 14:53:26,826][20381] Updated weights for policy 1, policy_version 1332872 (0.0005) [2023-12-27 14:53:27,096][20347] Updated weights for policy 0, policy_version 1333894 (0.0010) [2023-12-27 14:53:27,156][20347] Updated weights for policy 0, policy_version 1333904 (0.0010) [2023-12-27 14:53:27,212][20347] Updated weights for policy 0, policy_version 1333914 (0.0008) [2023-12-27 14:53:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 682795008. Throughput: 0: 10210.2, 1: 10219.1. Samples: 682781016. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:27,363][19170] Avg episode reward: [(0, '146.380'), (1, '941.410')] [2023-12-27 14:53:27,399][20381] Updated weights for policy 1, policy_version 1332882 (0.0006) [2023-12-27 14:53:27,454][20381] Updated weights for policy 1, policy_version 1332892 (0.0010) [2023-12-27 14:53:27,502][20381] Updated weights for policy 1, policy_version 1332902 (0.0010) [2023-12-27 14:53:27,563][20381] Updated weights for policy 1, policy_version 1332912 (0.0010) [2023-12-27 14:53:27,770][20347] Updated weights for policy 0, policy_version 1333924 (0.0005) [2023-12-27 14:53:27,829][20347] Updated weights for policy 0, policy_version 1333934 (0.0005) [2023-12-27 14:53:27,875][20347] Updated weights for policy 0, policy_version 1333944 (0.0005) [2023-12-27 14:53:28,182][20381] Updated weights for policy 1, policy_version 1332922 (0.0005) [2023-12-27 14:53:28,227][20381] Updated weights for policy 1, policy_version 1332932 (0.0005) [2023-12-27 14:53:28,277][20381] Updated weights for policy 1, policy_version 1332942 (0.0005) [2023-12-27 14:53:28,510][20347] Updated weights for policy 0, policy_version 1333954 (0.0007) [2023-12-27 14:53:28,571][20347] Updated weights for policy 0, policy_version 1333964 (0.0010) [2023-12-27 14:53:28,621][20347] Updated weights for policy 0, policy_version 1333974 (0.0010) [2023-12-27 14:53:28,676][20347] Updated weights for policy 0, policy_version 1333984 (0.0010) [2023-12-27 14:53:28,928][20381] Updated weights for policy 1, policy_version 1332952 (0.0010) [2023-12-27 14:53:28,976][20381] Updated weights for policy 1, policy_version 1332962 (0.0010) [2023-12-27 14:53:29,021][20381] Updated weights for policy 1, policy_version 1332972 (0.0010) [2023-12-27 14:53:29,329][20347] Updated weights for policy 0, policy_version 1333994 (0.0009) [2023-12-27 14:53:29,393][20347] Updated weights for policy 0, policy_version 1334004 (0.0011) [2023-12-27 14:53:29,440][20347] Updated weights for policy 0, policy_version 1334014 (0.0010) [2023-12-27 14:53:29,737][20381] Updated weights for policy 1, policy_version 1332982 (0.0009) [2023-12-27 14:53:29,795][20381] Updated weights for policy 1, policy_version 1332992 (0.0007) [2023-12-27 14:53:29,859][20381] Updated weights for policy 1, policy_version 1333002 (0.0008) [2023-12-27 14:53:30,165][20347] Updated weights for policy 0, policy_version 1334024 (0.0007) [2023-12-27 14:53:30,221][20347] Updated weights for policy 0, policy_version 1334034 (0.0006) [2023-12-27 14:53:30,270][20347] Updated weights for policy 0, policy_version 1334044 (0.0008) [2023-12-27 14:53:30,584][20381] Updated weights for policy 1, policy_version 1333012 (0.0008) [2023-12-27 14:53:30,628][20381] Updated weights for policy 1, policy_version 1333022 (0.0008) [2023-12-27 14:53:30,679][20381] Updated weights for policy 1, policy_version 1333032 (0.0007) [2023-12-27 14:53:30,881][20347] Updated weights for policy 0, policy_version 1334054 (0.0006) [2023-12-27 14:53:30,939][20347] Updated weights for policy 0, policy_version 1334064 (0.0005) [2023-12-27 14:53:30,997][20347] Updated weights for policy 0, policy_version 1334074 (0.0006) [2023-12-27 14:53:31,361][20381] Updated weights for policy 1, policy_version 1333042 (0.0006) [2023-12-27 14:53:31,425][20381] Updated weights for policy 1, policy_version 1333052 (0.0007) [2023-12-27 14:53:31,484][20381] Updated weights for policy 1, policy_version 1333062 (0.0007) [2023-12-27 14:53:31,540][20381] Updated weights for policy 1, policy_version 1333072 (0.0008) [2023-12-27 14:53:31,654][20347] Updated weights for policy 0, policy_version 1334084 (0.0008) [2023-12-27 14:53:31,720][20347] Updated weights for policy 0, policy_version 1334094 (0.0008) [2023-12-27 14:53:31,781][20347] Updated weights for policy 0, policy_version 1334104 (0.0006) [2023-12-27 14:53:32,296][20381] Updated weights for policy 1, policy_version 1333082 (0.0008) [2023-12-27 14:53:32,361][20381] Updated weights for policy 1, policy_version 1333092 (0.0007) [2023-12-27 14:53:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 682893312. Throughput: 0: 10179.4, 1: 10272.1. Samples: 682909056. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:32,363][19170] Avg episode reward: [(0, '155.770'), (1, '951.110')] [2023-12-27 14:53:32,421][20381] Updated weights for policy 1, policy_version 1333102 (0.0006) [2023-12-27 14:53:32,446][20347] Updated weights for policy 0, policy_version 1334114 (0.0006) [2023-12-27 14:53:32,505][20347] Updated weights for policy 0, policy_version 1334124 (0.0008) [2023-12-27 14:53:32,562][20347] Updated weights for policy 0, policy_version 1334134 (0.0008) [2023-12-27 14:53:32,610][20347] Updated weights for policy 0, policy_version 1334144 (0.0007) [2023-12-27 14:53:33,028][20381] Updated weights for policy 1, policy_version 1333112 (0.0008) [2023-12-27 14:53:33,082][20381] Updated weights for policy 1, policy_version 1333122 (0.0005) [2023-12-27 14:53:33,134][20381] Updated weights for policy 1, policy_version 1333132 (0.0005) [2023-12-27 14:53:33,288][20347] Updated weights for policy 0, policy_version 1334154 (0.0008) [2023-12-27 14:53:33,349][20347] Updated weights for policy 0, policy_version 1334164 (0.0008) [2023-12-27 14:53:33,390][20347] Updated weights for policy 0, policy_version 1334174 (0.0007) [2023-12-27 14:53:33,743][20381] Updated weights for policy 1, policy_version 1333142 (0.0006) [2023-12-27 14:53:33,804][20381] Updated weights for policy 1, policy_version 1333152 (0.0009) [2023-12-27 14:53:33,864][20381] Updated weights for policy 1, policy_version 1333162 (0.0009) [2023-12-27 14:53:34,047][20347] Updated weights for policy 0, policy_version 1334184 (0.0009) [2023-12-27 14:53:34,099][20347] Updated weights for policy 0, policy_version 1334194 (0.0010) [2023-12-27 14:53:34,161][20347] Updated weights for policy 0, policy_version 1334204 (0.0010) [2023-12-27 14:53:34,558][20381] Updated weights for policy 1, policy_version 1333172 (0.0007) [2023-12-27 14:53:34,622][20381] Updated weights for policy 1, policy_version 1333182 (0.0006) [2023-12-27 14:53:34,685][20381] Updated weights for policy 1, policy_version 1333192 (0.0007) [2023-12-27 14:53:34,882][20347] Updated weights for policy 0, policy_version 1334214 (0.0008) [2023-12-27 14:53:34,940][20347] Updated weights for policy 0, policy_version 1334224 (0.0008) [2023-12-27 14:53:34,989][20347] Updated weights for policy 0, policy_version 1334234 (0.0007) [2023-12-27 14:53:35,388][20381] Updated weights for policy 1, policy_version 1333202 (0.0008) [2023-12-27 14:53:35,444][20381] Updated weights for policy 1, policy_version 1333212 (0.0008) [2023-12-27 14:53:35,494][20381] Updated weights for policy 1, policy_version 1333222 (0.0008) [2023-12-27 14:53:35,544][20381] Updated weights for policy 1, policy_version 1333232 (0.0008) [2023-12-27 14:53:35,574][20347] Updated weights for policy 0, policy_version 1334244 (0.0007) [2023-12-27 14:53:35,644][20347] Updated weights for policy 0, policy_version 1334254 (0.0006) [2023-12-27 14:53:35,704][20347] Updated weights for policy 0, policy_version 1334264 (0.0011) [2023-12-27 14:53:36,266][20381] Updated weights for policy 1, policy_version 1333242 (0.0008) [2023-12-27 14:53:36,300][20347] Updated weights for policy 0, policy_version 1334274 (0.0009) [2023-12-27 14:53:36,321][20381] Updated weights for policy 1, policy_version 1333252 (0.0007) [2023-12-27 14:53:36,363][20347] Updated weights for policy 0, policy_version 1334284 (0.0007) [2023-12-27 14:53:36,368][20381] Updated weights for policy 1, policy_version 1333262 (0.0005) [2023-12-27 14:53:36,423][20347] Updated weights for policy 0, policy_version 1334294 (0.0008) [2023-12-27 14:53:36,487][20347] Updated weights for policy 0, policy_version 1334304 (0.0008) [2023-12-27 14:53:37,038][20381] Updated weights for policy 1, policy_version 1333272 (0.0007) [2023-12-27 14:53:37,099][20381] Updated weights for policy 1, policy_version 1333282 (0.0006) [2023-12-27 14:53:37,153][20381] Updated weights for policy 1, policy_version 1333292 (0.0007) [2023-12-27 14:53:37,208][20347] Updated weights for policy 0, policy_version 1334314 (0.0007) [2023-12-27 14:53:37,267][20347] Updated weights for policy 0, policy_version 1334324 (0.0008) [2023-12-27 14:53:37,324][20347] Updated weights for policy 0, policy_version 1334334 (0.0007) [2023-12-27 14:53:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 683008000. Throughput: 0: 10185.0, 1: 10315.8. Samples: 682970924. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:37,363][19170] Avg episode reward: [(0, '459.890'), (1, '921.890')] [2023-12-27 14:53:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001334336_341655552.pth... [2023-12-27 14:53:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001333296_341352448.pth... [2023-12-27 14:53:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001333120_341344256.pth [2023-12-27 14:53:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001332080_341041152.pth [2023-12-27 14:53:37,763][20381] Updated weights for policy 1, policy_version 1333302 (0.0007) [2023-12-27 14:53:37,808][20381] Updated weights for policy 1, policy_version 1333312 (0.0008) [2023-12-27 14:53:37,857][20381] Updated weights for policy 1, policy_version 1333322 (0.0008) [2023-12-27 14:53:38,007][20347] Updated weights for policy 0, policy_version 1334344 (0.0008) [2023-12-27 14:53:38,054][20347] Updated weights for policy 0, policy_version 1334354 (0.0008) [2023-12-27 14:53:38,102][20347] Updated weights for policy 0, policy_version 1334364 (0.0008) [2023-12-27 14:53:38,589][20381] Updated weights for policy 1, policy_version 1333332 (0.0008) [2023-12-27 14:53:38,636][20381] Updated weights for policy 1, policy_version 1333342 (0.0008) [2023-12-27 14:53:38,685][20381] Updated weights for policy 1, policy_version 1333352 (0.0008) [2023-12-27 14:53:38,822][20347] Updated weights for policy 0, policy_version 1334374 (0.0007) [2023-12-27 14:53:38,875][20347] Updated weights for policy 0, policy_version 1334384 (0.0007) [2023-12-27 14:53:38,930][20347] Updated weights for policy 0, policy_version 1334394 (0.0005) [2023-12-27 14:53:39,441][20381] Updated weights for policy 1, policy_version 1333362 (0.0008) [2023-12-27 14:53:39,504][20381] Updated weights for policy 1, policy_version 1333372 (0.0007) [2023-12-27 14:53:39,566][20381] Updated weights for policy 1, policy_version 1333382 (0.0006) [2023-12-27 14:53:39,575][20347] Updated weights for policy 0, policy_version 1334404 (0.0006) [2023-12-27 14:53:39,617][20381] Updated weights for policy 1, policy_version 1333392 (0.0006) [2023-12-27 14:53:39,633][20347] Updated weights for policy 0, policy_version 1334414 (0.0008) [2023-12-27 14:53:39,695][20347] Updated weights for policy 0, policy_version 1334424 (0.0008) [2023-12-27 14:53:40,303][20381] Updated weights for policy 1, policy_version 1333402 (0.0011) [2023-12-27 14:53:40,367][20381] Updated weights for policy 1, policy_version 1333412 (0.0011) [2023-12-27 14:53:40,423][20347] Updated weights for policy 0, policy_version 1334434 (0.0009) [2023-12-27 14:53:40,428][20381] Updated weights for policy 1, policy_version 1333422 (0.0011) [2023-12-27 14:53:40,474][20347] Updated weights for policy 0, policy_version 1334444 (0.0007) [2023-12-27 14:53:40,522][20347] Updated weights for policy 0, policy_version 1334454 (0.0007) [2023-12-27 14:53:40,574][20347] Updated weights for policy 0, policy_version 1334464 (0.0007) [2023-12-27 14:53:41,103][20381] Updated weights for policy 1, policy_version 1333432 (0.0011) [2023-12-27 14:53:41,168][20381] Updated weights for policy 1, policy_version 1333442 (0.0011) [2023-12-27 14:53:41,220][20381] Updated weights for policy 1, policy_version 1333452 (0.0008) [2023-12-27 14:53:41,269][20347] Updated weights for policy 0, policy_version 1334474 (0.0008) [2023-12-27 14:53:41,327][20347] Updated weights for policy 0, policy_version 1334484 (0.0008) [2023-12-27 14:53:41,395][20347] Updated weights for policy 0, policy_version 1334494 (0.0008) [2023-12-27 14:53:42,059][20381] Updated weights for policy 1, policy_version 1333462 (0.0009) [2023-12-27 14:53:42,122][20381] Updated weights for policy 1, policy_version 1333472 (0.0010) [2023-12-27 14:53:42,131][20347] Updated weights for policy 0, policy_version 1334504 (0.0005) [2023-12-27 14:53:42,182][20381] Updated weights for policy 1, policy_version 1333482 (0.0010) [2023-12-27 14:53:42,191][20347] Updated weights for policy 0, policy_version 1334514 (0.0006) [2023-12-27 14:53:42,255][20347] Updated weights for policy 0, policy_version 1334524 (0.0006) [2023-12-27 14:53:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 683106304. Throughput: 0: 10187.5, 1: 10254.9. Samples: 683093092. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:42,364][19170] Avg episode reward: [(0, '496.770'), (1, '892.760')] [2023-12-27 14:53:42,806][20381] Updated weights for policy 1, policy_version 1333492 (0.0008) [2023-12-27 14:53:42,861][20381] Updated weights for policy 1, policy_version 1333502 (0.0005) [2023-12-27 14:53:42,916][20381] Updated weights for policy 1, policy_version 1333512 (0.0006) [2023-12-27 14:53:42,929][20347] Updated weights for policy 0, policy_version 1334534 (0.0006) [2023-12-27 14:53:42,979][20347] Updated weights for policy 0, policy_version 1334544 (0.0006) [2023-12-27 14:53:43,027][20347] Updated weights for policy 0, policy_version 1334554 (0.0007) [2023-12-27 14:53:43,548][20381] Updated weights for policy 1, policy_version 1333522 (0.0009) [2023-12-27 14:53:43,597][20381] Updated weights for policy 1, policy_version 1333532 (0.0010) [2023-12-27 14:53:43,646][20381] Updated weights for policy 1, policy_version 1333542 (0.0010) [2023-12-27 14:53:43,691][20347] Updated weights for policy 0, policy_version 1334564 (0.0008) [2023-12-27 14:53:43,701][20381] Updated weights for policy 1, policy_version 1333552 (0.0010) [2023-12-27 14:53:43,749][20347] Updated weights for policy 0, policy_version 1334574 (0.0007) [2023-12-27 14:53:43,804][20347] Updated weights for policy 0, policy_version 1334584 (0.0007) [2023-12-27 14:53:44,365][20381] Updated weights for policy 1, policy_version 1333562 (0.0010) [2023-12-27 14:53:44,417][20381] Updated weights for policy 1, policy_version 1333572 (0.0010) [2023-12-27 14:53:44,471][20381] Updated weights for policy 1, policy_version 1333582 (0.0010) [2023-12-27 14:53:44,517][20347] Updated weights for policy 0, policy_version 1334594 (0.0007) [2023-12-27 14:53:44,563][20347] Updated weights for policy 0, policy_version 1334604 (0.0007) [2023-12-27 14:53:44,613][20347] Updated weights for policy 0, policy_version 1334614 (0.0007) [2023-12-27 14:53:44,660][20347] Updated weights for policy 0, policy_version 1334624 (0.0007) [2023-12-27 14:53:45,172][20381] Updated weights for policy 1, policy_version 1333592 (0.0009) [2023-12-27 14:53:45,231][20381] Updated weights for policy 1, policy_version 1333602 (0.0008) [2023-12-27 14:53:45,292][20381] Updated weights for policy 1, policy_version 1333612 (0.0008) [2023-12-27 14:53:45,410][20347] Updated weights for policy 0, policy_version 1334634 (0.0008) [2023-12-27 14:53:45,465][20347] Updated weights for policy 0, policy_version 1334644 (0.0008) [2023-12-27 14:53:45,524][20347] Updated weights for policy 0, policy_version 1334654 (0.0008) [2023-12-27 14:53:45,972][20381] Updated weights for policy 1, policy_version 1333622 (0.0009) [2023-12-27 14:53:46,020][20381] Updated weights for policy 1, policy_version 1333632 (0.0008) [2023-12-27 14:53:46,071][20381] Updated weights for policy 1, policy_version 1333642 (0.0008) [2023-12-27 14:53:46,192][20347] Updated weights for policy 0, policy_version 1334664 (0.0008) [2023-12-27 14:53:46,243][20347] Updated weights for policy 0, policy_version 1334674 (0.0007) [2023-12-27 14:53:46,299][20347] Updated weights for policy 0, policy_version 1334684 (0.0007) [2023-12-27 14:53:46,747][20381] Updated weights for policy 1, policy_version 1333652 (0.0007) [2023-12-27 14:53:46,806][20381] Updated weights for policy 1, policy_version 1333662 (0.0005) [2023-12-27 14:53:46,866][20381] Updated weights for policy 1, policy_version 1333672 (0.0009) [2023-12-27 14:53:46,965][20347] Updated weights for policy 0, policy_version 1334694 (0.0006) [2023-12-27 14:53:47,021][20347] Updated weights for policy 0, policy_version 1334704 (0.0006) [2023-12-27 14:53:47,079][20347] Updated weights for policy 0, policy_version 1334714 (0.0005) [2023-12-27 14:53:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 683204608. Throughput: 0: 10196.3, 1: 10288.6. Samples: 683215384. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:47,363][19170] Avg episode reward: [(0, '505.210'), (1, '941.350')] [2023-12-27 14:53:47,486][20381] Updated weights for policy 1, policy_version 1333682 (0.0009) [2023-12-27 14:53:47,544][20381] Updated weights for policy 1, policy_version 1333692 (0.0010) [2023-12-27 14:53:47,600][20381] Updated weights for policy 1, policy_version 1333702 (0.0006) [2023-12-27 14:53:47,658][20381] Updated weights for policy 1, policy_version 1333712 (0.0011) [2023-12-27 14:53:47,679][20347] Updated weights for policy 0, policy_version 1334724 (0.0006) [2023-12-27 14:53:47,733][20347] Updated weights for policy 0, policy_version 1334734 (0.0007) [2023-12-27 14:53:47,787][20347] Updated weights for policy 0, policy_version 1334744 (0.0007) [2023-12-27 14:53:48,326][20381] Updated weights for policy 1, policy_version 1333722 (0.0011) [2023-12-27 14:53:48,390][20381] Updated weights for policy 1, policy_version 1333732 (0.0011) [2023-12-27 14:53:48,425][20347] Updated weights for policy 0, policy_version 1334754 (0.0007) [2023-12-27 14:53:48,449][20381] Updated weights for policy 1, policy_version 1333742 (0.0011) [2023-12-27 14:53:48,486][20347] Updated weights for policy 0, policy_version 1334764 (0.0008) [2023-12-27 14:53:48,505][20291] KL-divergence is very high: 135.1877 [2023-12-27 14:53:48,512][20291] KL-divergence is very high: 142.4279 [2023-12-27 14:53:48,518][20291] KL-divergence is very high: 135.5692 [2023-12-27 14:53:48,525][20291] KL-divergence is very high: 125.2557 [2023-12-27 14:53:48,547][20347] Updated weights for policy 0, policy_version 1334774 (0.0008) [2023-12-27 14:53:48,608][20347] Updated weights for policy 0, policy_version 1334784 (0.0010) [2023-12-27 14:53:49,172][20381] Updated weights for policy 1, policy_version 1333752 (0.0008) [2023-12-27 14:53:49,233][20381] Updated weights for policy 1, policy_version 1333762 (0.0008) [2023-12-27 14:53:49,235][20347] Updated weights for policy 0, policy_version 1334794 (0.0007) [2023-12-27 14:53:49,290][20381] Updated weights for policy 1, policy_version 1333772 (0.0006) [2023-12-27 14:53:49,297][20347] Updated weights for policy 0, policy_version 1334804 (0.0008) [2023-12-27 14:53:49,340][20291] KL-divergence is very high: 134.7309 [2023-12-27 14:53:49,363][20347] Updated weights for policy 0, policy_version 1334814 (0.0009) [2023-12-27 14:53:49,976][20381] Updated weights for policy 1, policy_version 1333782 (0.0008) [2023-12-27 14:53:50,034][20381] Updated weights for policy 1, policy_version 1333792 (0.0007) [2023-12-27 14:53:50,084][20347] Updated weights for policy 0, policy_version 1334824 (0.0007) [2023-12-27 14:53:50,084][20291] KL-divergence is very high: 124.6777 [2023-12-27 14:53:50,094][20381] Updated weights for policy 1, policy_version 1333802 (0.0006) [2023-12-27 14:53:50,095][20291] KL-divergence is very high: 129.3805 [2023-12-27 14:53:50,134][20291] KL-divergence is very high: 112.6879 [2023-12-27 14:53:50,149][20347] Updated weights for policy 0, policy_version 1334834 (0.0009) [2023-12-27 14:53:50,215][20347] Updated weights for policy 0, policy_version 1334844 (0.0009) [2023-12-27 14:53:50,786][20381] Updated weights for policy 1, policy_version 1333812 (0.0007) [2023-12-27 14:53:50,845][20381] Updated weights for policy 1, policy_version 1333822 (0.0008) [2023-12-27 14:53:50,893][20347] Updated weights for policy 0, policy_version 1334854 (0.0006) [2023-12-27 14:53:50,902][20381] Updated weights for policy 1, policy_version 1333832 (0.0009) [2023-12-27 14:53:50,954][20347] Updated weights for policy 0, policy_version 1334864 (0.0006) [2023-12-27 14:53:51,002][20347] Updated weights for policy 0, policy_version 1334874 (0.0005) [2023-12-27 14:53:51,639][20381] Updated weights for policy 1, policy_version 1333842 (0.0007) [2023-12-27 14:53:51,688][20347] Updated weights for policy 0, policy_version 1334884 (0.0007) [2023-12-27 14:53:51,695][20381] Updated weights for policy 1, policy_version 1333852 (0.0006) [2023-12-27 14:53:51,756][20347] Updated weights for policy 0, policy_version 1334894 (0.0007) [2023-12-27 14:53:51,762][20381] Updated weights for policy 1, policy_version 1333862 (0.0007) [2023-12-27 14:53:51,817][20381] Updated weights for policy 1, policy_version 1333872 (0.0006) [2023-12-27 14:53:51,817][20347] Updated weights for policy 0, policy_version 1334904 (0.0007) [2023-12-27 14:53:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 683302912. Throughput: 0: 10209.0, 1: 10251.7. Samples: 683276172. Policy #0 lag: (min: 3.0, avg: 3.8, max: 21.0) [2023-12-27 14:53:52,363][19170] Avg episode reward: [(0, '542.550'), (1, '941.430')] [2023-12-27 14:53:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001333872_341499904.pth... [2023-12-27 14:53:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001334912_341803008.pth... [2023-12-27 14:53:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001333728_341499904.pth [2023-12-27 14:53:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001332688_341196800.pth [2023-12-27 14:53:52,489][20347] Updated weights for policy 0, policy_version 1334914 (0.0006) [2023-12-27 14:53:52,551][20381] Updated weights for policy 1, policy_version 1333882 (0.0006) [2023-12-27 14:53:52,553][20347] Updated weights for policy 0, policy_version 1334924 (0.0006) [2023-12-27 14:53:52,612][20381] Updated weights for policy 1, policy_version 1333892 (0.0007) [2023-12-27 14:53:52,615][20347] Updated weights for policy 0, policy_version 1334934 (0.0006) [2023-12-27 14:53:52,666][20381] Updated weights for policy 1, policy_version 1333902 (0.0008) [2023-12-27 14:53:52,678][20347] Updated weights for policy 0, policy_version 1334944 (0.0008) [2023-12-27 14:53:53,273][20347] Updated weights for policy 0, policy_version 1334954 (0.0010) [2023-12-27 14:53:53,322][20381] Updated weights for policy 1, policy_version 1333912 (0.0010) [2023-12-27 14:53:53,328][20347] Updated weights for policy 0, policy_version 1334964 (0.0010) [2023-12-27 14:53:53,371][20381] Updated weights for policy 1, policy_version 1333922 (0.0010) [2023-12-27 14:53:53,385][20347] Updated weights for policy 0, policy_version 1334974 (0.0010) [2023-12-27 14:53:53,421][20381] Updated weights for policy 1, policy_version 1333932 (0.0010) [2023-12-27 14:53:54,059][20381] Updated weights for policy 1, policy_version 1333942 (0.0007) [2023-12-27 14:53:54,063][20347] Updated weights for policy 0, policy_version 1334984 (0.0008) [2023-12-27 14:53:54,112][20381] Updated weights for policy 1, policy_version 1333952 (0.0005) [2023-12-27 14:53:54,112][20347] Updated weights for policy 0, policy_version 1334994 (0.0007) [2023-12-27 14:53:54,165][20347] Updated weights for policy 0, policy_version 1335004 (0.0008) [2023-12-27 14:53:54,168][20381] Updated weights for policy 1, policy_version 1333962 (0.0005) [2023-12-27 14:53:54,749][20381] Updated weights for policy 1, policy_version 1333972 (0.0009) [2023-12-27 14:53:54,804][20381] Updated weights for policy 1, policy_version 1333982 (0.0010) [2023-12-27 14:53:54,855][20381] Updated weights for policy 1, policy_version 1333992 (0.0010) [2023-12-27 14:53:54,901][20347] Updated weights for policy 0, policy_version 1335014 (0.0009) [2023-12-27 14:53:54,954][20347] Updated weights for policy 0, policy_version 1335024 (0.0007) [2023-12-27 14:53:55,013][20347] Updated weights for policy 0, policy_version 1335034 (0.0010) [2023-12-27 14:53:55,548][20381] Updated weights for policy 1, policy_version 1334002 (0.0010) [2023-12-27 14:53:55,600][20381] Updated weights for policy 1, policy_version 1334012 (0.0010) [2023-12-27 14:53:55,654][20381] Updated weights for policy 1, policy_version 1334022 (0.0006) [2023-12-27 14:53:55,659][20347] Updated weights for policy 0, policy_version 1335044 (0.0010) [2023-12-27 14:53:55,711][20347] Updated weights for policy 0, policy_version 1335054 (0.0010) [2023-12-27 14:53:55,713][20381] Updated weights for policy 1, policy_version 1334032 (0.0006) [2023-12-27 14:53:55,766][20347] Updated weights for policy 0, policy_version 1335064 (0.0010) [2023-12-27 14:53:56,369][20347] Updated weights for policy 0, policy_version 1335074 (0.0011) [2023-12-27 14:53:56,372][20381] Updated weights for policy 1, policy_version 1334042 (0.0007) [2023-12-27 14:53:56,417][20347] Updated weights for policy 0, policy_version 1335084 (0.0010) [2023-12-27 14:53:56,421][20381] Updated weights for policy 1, policy_version 1334052 (0.0009) [2023-12-27 14:53:56,461][20347] Updated weights for policy 0, policy_version 1335094 (0.0010) [2023-12-27 14:53:56,474][20381] Updated weights for policy 1, policy_version 1334062 (0.0007) [2023-12-27 14:53:56,506][20347] Updated weights for policy 0, policy_version 1335104 (0.0009) [2023-12-27 14:53:57,119][20347] Updated weights for policy 0, policy_version 1335114 (0.0010) [2023-12-27 14:53:57,121][20381] Updated weights for policy 1, policy_version 1334072 (0.0005) [2023-12-27 14:53:57,173][20381] Updated weights for policy 1, policy_version 1334082 (0.0005) [2023-12-27 14:53:57,174][20347] Updated weights for policy 0, policy_version 1335124 (0.0010) [2023-12-27 14:53:57,226][20381] Updated weights for policy 1, policy_version 1334092 (0.0005) [2023-12-27 14:53:57,227][20347] Updated weights for policy 0, policy_version 1335134 (0.0010) [2023-12-27 14:53:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 683417600. Throughput: 0: 10190.9, 1: 10303.4. Samples: 683399780. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:53:57,363][19170] Avg episode reward: [(0, '606.170'), (1, '931.530')] [2023-12-27 14:53:57,766][20347] Updated weights for policy 0, policy_version 1335144 (0.0009) [2023-12-27 14:53:57,776][20381] Updated weights for policy 1, policy_version 1334102 (0.0005) [2023-12-27 14:53:57,820][20347] Updated weights for policy 0, policy_version 1335154 (0.0010) [2023-12-27 14:53:57,825][20381] Updated weights for policy 1, policy_version 1334112 (0.0008) [2023-12-27 14:53:57,875][20381] Updated weights for policy 1, policy_version 1334122 (0.0005) [2023-12-27 14:53:57,881][20347] Updated weights for policy 0, policy_version 1335164 (0.0008) [2023-12-27 14:53:58,576][20381] Updated weights for policy 1, policy_version 1334132 (0.0006) [2023-12-27 14:53:58,580][20347] Updated weights for policy 0, policy_version 1335174 (0.0006) [2023-12-27 14:53:58,635][20381] Updated weights for policy 1, policy_version 1334142 (0.0007) [2023-12-27 14:53:58,644][20347] Updated weights for policy 0, policy_version 1335184 (0.0010) [2023-12-27 14:53:58,697][20381] Updated weights for policy 1, policy_version 1334152 (0.0007) [2023-12-27 14:53:58,706][20347] Updated weights for policy 0, policy_version 1335194 (0.0010) [2023-12-27 14:53:59,498][20381] Updated weights for policy 1, policy_version 1334162 (0.0009) [2023-12-27 14:53:59,557][20381] Updated weights for policy 1, policy_version 1334172 (0.0007) [2023-12-27 14:53:59,564][20347] Updated weights for policy 0, policy_version 1335204 (0.0008) [2023-12-27 14:53:59,618][20381] Updated weights for policy 1, policy_version 1334182 (0.0008) [2023-12-27 14:53:59,631][20347] Updated weights for policy 0, policy_version 1335214 (0.0007) [2023-12-27 14:53:59,672][20381] Updated weights for policy 1, policy_version 1334192 (0.0006) [2023-12-27 14:53:59,682][20347] Updated weights for policy 0, policy_version 1335224 (0.0007) [2023-12-27 14:54:00,364][20381] Updated weights for policy 1, policy_version 1334202 (0.0006) [2023-12-27 14:54:00,416][20347] Updated weights for policy 0, policy_version 1335234 (0.0007) [2023-12-27 14:54:00,424][20381] Updated weights for policy 1, policy_version 1334212 (0.0008) [2023-12-27 14:54:00,468][20347] Updated weights for policy 0, policy_version 1335244 (0.0007) [2023-12-27 14:54:00,484][20381] Updated weights for policy 1, policy_version 1334222 (0.0007) [2023-12-27 14:54:00,526][20347] Updated weights for policy 0, policy_version 1335254 (0.0006) [2023-12-27 14:54:00,585][20347] Updated weights for policy 0, policy_version 1335264 (0.0007) [2023-12-27 14:54:01,124][20381] Updated weights for policy 1, policy_version 1334232 (0.0006) [2023-12-27 14:54:01,192][20381] Updated weights for policy 1, policy_version 1334242 (0.0007) [2023-12-27 14:54:01,247][20381] Updated weights for policy 1, policy_version 1334252 (0.0006) [2023-12-27 14:54:01,253][20347] Updated weights for policy 0, policy_version 1335274 (0.0008) [2023-12-27 14:54:01,315][20347] Updated weights for policy 0, policy_version 1335284 (0.0006) [2023-12-27 14:54:01,382][20347] Updated weights for policy 0, policy_version 1335294 (0.0006) [2023-12-27 14:54:01,905][20381] Updated weights for policy 1, policy_version 1334262 (0.0008) [2023-12-27 14:54:01,954][20381] Updated weights for policy 1, policy_version 1334272 (0.0008) [2023-12-27 14:54:02,013][20381] Updated weights for policy 1, policy_version 1334282 (0.0008) [2023-12-27 14:54:02,090][20347] Updated weights for policy 0, policy_version 1335304 (0.0006) [2023-12-27 14:54:02,152][20347] Updated weights for policy 0, policy_version 1335314 (0.0006) [2023-12-27 14:54:02,211][20347] Updated weights for policy 0, policy_version 1335324 (0.0005) [2023-12-27 14:54:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20521.6). Total num frames: 683515904. Throughput: 0: 10257.8, 1: 10235.5. Samples: 683522164. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:02,363][19170] Avg episode reward: [(0, '609.820'), (1, '951.110')] [2023-12-27 14:54:02,744][20347] Updated weights for policy 0, policy_version 1335334 (0.0005) [2023-12-27 14:54:02,798][20347] Updated weights for policy 0, policy_version 1335344 (0.0005) [2023-12-27 14:54:02,811][20381] Updated weights for policy 1, policy_version 1334292 (0.0008) [2023-12-27 14:54:02,848][20347] Updated weights for policy 0, policy_version 1335354 (0.0005) [2023-12-27 14:54:02,863][20381] Updated weights for policy 1, policy_version 1334302 (0.0007) [2023-12-27 14:54:02,918][20381] Updated weights for policy 1, policy_version 1334312 (0.0007) [2023-12-27 14:54:03,412][20347] Updated weights for policy 0, policy_version 1335364 (0.0005) [2023-12-27 14:54:03,457][20347] Updated weights for policy 0, policy_version 1335374 (0.0005) [2023-12-27 14:54:03,502][20347] Updated weights for policy 0, policy_version 1335384 (0.0005) [2023-12-27 14:54:03,656][20381] Updated weights for policy 1, policy_version 1334322 (0.0008) [2023-12-27 14:54:03,722][20381] Updated weights for policy 1, policy_version 1334332 (0.0008) [2023-12-27 14:54:03,779][20381] Updated weights for policy 1, policy_version 1334342 (0.0009) [2023-12-27 14:54:03,832][20381] Updated weights for policy 1, policy_version 1334352 (0.0006) [2023-12-27 14:54:04,102][20347] Updated weights for policy 0, policy_version 1335394 (0.0005) [2023-12-27 14:54:04,157][20347] Updated weights for policy 0, policy_version 1335404 (0.0005) [2023-12-27 14:54:04,221][20347] Updated weights for policy 0, policy_version 1335414 (0.0009) [2023-12-27 14:54:04,268][20347] Updated weights for policy 0, policy_version 1335424 (0.0010) [2023-12-27 14:54:04,553][20381] Updated weights for policy 1, policy_version 1334362 (0.0006) [2023-12-27 14:54:04,612][20381] Updated weights for policy 1, policy_version 1334372 (0.0007) [2023-12-27 14:54:04,667][20381] Updated weights for policy 1, policy_version 1334382 (0.0008) [2023-12-27 14:54:04,909][20347] Updated weights for policy 0, policy_version 1335434 (0.0005) [2023-12-27 14:54:04,963][20347] Updated weights for policy 0, policy_version 1335444 (0.0007) [2023-12-27 14:54:04,978][20291] KL-divergence is very high: 112.6546 [2023-12-27 14:54:05,021][20347] Updated weights for policy 0, policy_version 1335454 (0.0007) [2023-12-27 14:54:05,267][20381] Updated weights for policy 1, policy_version 1334392 (0.0006) [2023-12-27 14:54:05,317][20381] Updated weights for policy 1, policy_version 1334402 (0.0005) [2023-12-27 14:54:05,371][20381] Updated weights for policy 1, policy_version 1334412 (0.0006) [2023-12-27 14:54:05,631][20347] Updated weights for policy 0, policy_version 1335464 (0.0006) [2023-12-27 14:54:05,696][20347] Updated weights for policy 0, policy_version 1335474 (0.0005) [2023-12-27 14:54:05,754][20347] Updated weights for policy 0, policy_version 1335484 (0.0008) [2023-12-27 14:54:05,948][20381] Updated weights for policy 1, policy_version 1334422 (0.0007) [2023-12-27 14:54:06,003][20381] Updated weights for policy 1, policy_version 1334432 (0.0008) [2023-12-27 14:54:06,052][20381] Updated weights for policy 1, policy_version 1334442 (0.0008) [2023-12-27 14:54:06,394][20347] Updated weights for policy 0, policy_version 1335494 (0.0010) [2023-12-27 14:54:06,445][20347] Updated weights for policy 0, policy_version 1335504 (0.0011) [2023-12-27 14:54:06,491][20347] Updated weights for policy 0, policy_version 1335514 (0.0010) [2023-12-27 14:54:06,764][20381] Updated weights for policy 1, policy_version 1334452 (0.0006) [2023-12-27 14:54:06,824][20381] Updated weights for policy 1, policy_version 1334462 (0.0007) [2023-12-27 14:54:06,879][20381] Updated weights for policy 1, policy_version 1334472 (0.0007) [2023-12-27 14:54:07,217][20347] Updated weights for policy 0, policy_version 1335524 (0.0010) [2023-12-27 14:54:07,273][20347] Updated weights for policy 0, policy_version 1335534 (0.0010) [2023-12-27 14:54:07,325][20347] Updated weights for policy 0, policy_version 1335544 (0.0010) [2023-12-27 14:54:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 683622400. Throughput: 0: 10382.9, 1: 10212.4. Samples: 683586480. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:07,363][19170] Avg episode reward: [(0, '521.720'), (1, '931.590')] [2023-12-27 14:54:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001335552_341966848.pth... [2023-12-27 14:54:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001334480_341655552.pth... [2023-12-27 14:54:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001334336_341655552.pth [2023-12-27 14:54:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001333296_341352448.pth [2023-12-27 14:54:07,561][20381] Updated weights for policy 1, policy_version 1334482 (0.0008) [2023-12-27 14:54:07,624][20381] Updated weights for policy 1, policy_version 1334492 (0.0007) [2023-12-27 14:54:07,682][20381] Updated weights for policy 1, policy_version 1334502 (0.0008) [2023-12-27 14:54:07,743][20381] Updated weights for policy 1, policy_version 1334512 (0.0007) [2023-12-27 14:54:08,018][20347] Updated weights for policy 0, policy_version 1335554 (0.0009) [2023-12-27 14:54:08,071][20347] Updated weights for policy 0, policy_version 1335564 (0.0008) [2023-12-27 14:54:08,120][20347] Updated weights for policy 0, policy_version 1335574 (0.0007) [2023-12-27 14:54:08,174][20347] Updated weights for policy 0, policy_version 1335584 (0.0008) [2023-12-27 14:54:08,375][20381] Updated weights for policy 1, policy_version 1334522 (0.0008) [2023-12-27 14:54:08,425][20381] Updated weights for policy 1, policy_version 1334532 (0.0007) [2023-12-27 14:54:08,483][20381] Updated weights for policy 1, policy_version 1334542 (0.0007) [2023-12-27 14:54:08,914][20347] Updated weights for policy 0, policy_version 1335594 (0.0005) [2023-12-27 14:54:08,973][20347] Updated weights for policy 0, policy_version 1335604 (0.0006) [2023-12-27 14:54:09,029][20347] Updated weights for policy 0, policy_version 1335614 (0.0007) [2023-12-27 14:54:09,196][20381] Updated weights for policy 1, policy_version 1334552 (0.0008) [2023-12-27 14:54:09,262][20381] Updated weights for policy 1, policy_version 1334562 (0.0008) [2023-12-27 14:54:09,315][20381] Updated weights for policy 1, policy_version 1334572 (0.0008) [2023-12-27 14:54:09,721][20347] Updated weights for policy 0, policy_version 1335624 (0.0007) [2023-12-27 14:54:09,781][20347] Updated weights for policy 0, policy_version 1335634 (0.0006) [2023-12-27 14:54:09,841][20347] Updated weights for policy 0, policy_version 1335644 (0.0007) [2023-12-27 14:54:10,111][20381] Updated weights for policy 1, policy_version 1334582 (0.0008) [2023-12-27 14:54:10,168][20381] Updated weights for policy 1, policy_version 1334592 (0.0008) [2023-12-27 14:54:10,231][20381] Updated weights for policy 1, policy_version 1334602 (0.0007) [2023-12-27 14:54:10,513][20347] Updated weights for policy 0, policy_version 1335654 (0.0008) [2023-12-27 14:54:10,561][20347] Updated weights for policy 0, policy_version 1335664 (0.0009) [2023-12-27 14:54:10,616][20347] Updated weights for policy 0, policy_version 1335674 (0.0008) [2023-12-27 14:54:10,895][20381] Updated weights for policy 1, policy_version 1334612 (0.0007) [2023-12-27 14:54:10,956][20381] Updated weights for policy 1, policy_version 1334622 (0.0005) [2023-12-27 14:54:11,009][20381] Updated weights for policy 1, policy_version 1334632 (0.0005) [2023-12-27 14:54:11,376][20347] Updated weights for policy 0, policy_version 1335684 (0.0007) [2023-12-27 14:54:11,442][20347] Updated weights for policy 0, policy_version 1335694 (0.0006) [2023-12-27 14:54:11,512][20347] Updated weights for policy 0, policy_version 1335704 (0.0006) [2023-12-27 14:54:11,748][20381] Updated weights for policy 1, policy_version 1334642 (0.0008) [2023-12-27 14:54:11,815][20381] Updated weights for policy 1, policy_version 1334652 (0.0008) [2023-12-27 14:54:11,817][20292] KL-divergence is very high: 109.1507 [2023-12-27 14:54:11,888][20381] Updated weights for policy 1, policy_version 1334662 (0.0008) [2023-12-27 14:54:11,958][20381] Updated weights for policy 1, policy_version 1334672 (0.0008) [2023-12-27 14:54:12,196][20347] Updated weights for policy 0, policy_version 1335714 (0.0006) [2023-12-27 14:54:12,259][20347] Updated weights for policy 0, policy_version 1335724 (0.0007) [2023-12-27 14:54:12,319][20347] Updated weights for policy 0, policy_version 1335734 (0.0008) [2023-12-27 14:54:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 683712512. Throughput: 0: 10393.6, 1: 10205.1. Samples: 683707956. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:12,363][19170] Avg episode reward: [(0, '543.820'), (1, '921.810')] [2023-12-27 14:54:12,384][20347] Updated weights for policy 0, policy_version 1335744 (0.0007) [2023-12-27 14:54:12,667][20381] Updated weights for policy 1, policy_version 1334682 (0.0006) [2023-12-27 14:54:12,721][20381] Updated weights for policy 1, policy_version 1334692 (0.0005) [2023-12-27 14:54:12,781][20381] Updated weights for policy 1, policy_version 1334702 (0.0007) [2023-12-27 14:54:13,110][20347] Updated weights for policy 0, policy_version 1335754 (0.0008) [2023-12-27 14:54:13,167][20347] Updated weights for policy 0, policy_version 1335764 (0.0006) [2023-12-27 14:54:13,222][20347] Updated weights for policy 0, policy_version 1335774 (0.0005) [2023-12-27 14:54:13,455][20381] Updated weights for policy 1, policy_version 1334712 (0.0008) [2023-12-27 14:54:13,507][20381] Updated weights for policy 1, policy_version 1334722 (0.0008) [2023-12-27 14:54:13,562][20381] Updated weights for policy 1, policy_version 1334732 (0.0007) [2023-12-27 14:54:13,802][20347] Updated weights for policy 0, policy_version 1335784 (0.0009) [2023-12-27 14:54:13,856][20347] Updated weights for policy 0, policy_version 1335794 (0.0010) [2023-12-27 14:54:13,886][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000002 [2023-12-27 14:54:14,149][20381] Updated weights for policy 1, policy_version 1334742 (0.0006) [2023-12-27 14:54:14,203][20381] Updated weights for policy 1, policy_version 1334752 (0.0007) [2023-12-27 14:54:14,257][20381] Updated weights for policy 1, policy_version 1334762 (0.0005) [2023-12-27 14:54:14,607][20347] Updated weights for policy 0, policy_version 1335804 (0.0009) [2023-12-27 14:54:14,667][20347] Updated weights for policy 0, policy_version 1335814 (0.0008) [2023-12-27 14:54:14,725][20347] Updated weights for policy 0, policy_version 1335824 (0.0008) [2023-12-27 14:54:14,906][20381] Updated weights for policy 1, policy_version 1334772 (0.0009) [2023-12-27 14:54:14,965][20381] Updated weights for policy 1, policy_version 1334782 (0.0010) [2023-12-27 14:54:15,023][20381] Updated weights for policy 1, policy_version 1334792 (0.0011) [2023-12-27 14:54:15,420][20347] Updated weights for policy 0, policy_version 1335834 (0.0007) [2023-12-27 14:54:15,472][20347] Updated weights for policy 0, policy_version 1335844 (0.0005) [2023-12-27 14:54:15,526][20347] Updated weights for policy 0, policy_version 1335854 (0.0006) [2023-12-27 14:54:15,583][20347] Updated weights for policy 0, policy_version 1335864 (0.0006) [2023-12-27 14:54:15,673][20381] Updated weights for policy 1, policy_version 1334802 (0.0009) [2023-12-27 14:54:15,730][20381] Updated weights for policy 1, policy_version 1334812 (0.0010) [2023-12-27 14:54:15,788][20381] Updated weights for policy 1, policy_version 1334822 (0.0010) [2023-12-27 14:54:15,847][20381] Updated weights for policy 1, policy_version 1334832 (0.0010) [2023-12-27 14:54:16,137][20347] Updated weights for policy 0, policy_version 1335874 (0.0007) [2023-12-27 14:54:16,186][20347] Updated weights for policy 0, policy_version 1335884 (0.0010) [2023-12-27 14:54:16,235][20347] Updated weights for policy 0, policy_version 1335894 (0.0008) [2023-12-27 14:54:16,458][20381] Updated weights for policy 1, policy_version 1334842 (0.0006) [2023-12-27 14:54:16,517][20381] Updated weights for policy 1, policy_version 1334852 (0.0005) [2023-12-27 14:54:16,572][20381] Updated weights for policy 1, policy_version 1334862 (0.0007) [2023-12-27 14:54:16,954][20347] Updated weights for policy 0, policy_version 1335904 (0.0007) [2023-12-27 14:54:17,013][20347] Updated weights for policy 0, policy_version 1335914 (0.0007) [2023-12-27 14:54:17,064][20347] Updated weights for policy 0, policy_version 1335924 (0.0007) [2023-12-27 14:54:17,212][20381] Updated weights for policy 1, policy_version 1334872 (0.0010) [2023-12-27 14:54:17,270][20381] Updated weights for policy 1, policy_version 1334882 (0.0006) [2023-12-27 14:54:17,330][20381] Updated weights for policy 1, policy_version 1334892 (0.0010) [2023-12-27 14:54:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 683827200. Throughput: 0: 10312.7, 1: 10205.4. Samples: 683832368. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:17,363][19170] Avg episode reward: [(0, '716.830'), (1, '951.050')] [2023-12-27 14:54:17,666][20347] Updated weights for policy 0, policy_version 1335934 (0.0007) [2023-12-27 14:54:17,718][20347] Updated weights for policy 0, policy_version 1335944 (0.0007) [2023-12-27 14:54:17,774][20347] Updated weights for policy 0, policy_version 1335954 (0.0008) [2023-12-27 14:54:17,944][20381] Updated weights for policy 1, policy_version 1334902 (0.0011) [2023-12-27 14:54:17,989][20381] Updated weights for policy 1, policy_version 1334912 (0.0010) [2023-12-27 14:54:18,036][20381] Updated weights for policy 1, policy_version 1334922 (0.0011) [2023-12-27 14:54:18,372][20347] Updated weights for policy 0, policy_version 1335964 (0.0007) [2023-12-27 14:54:18,424][20347] Updated weights for policy 0, policy_version 1335974 (0.0010) [2023-12-27 14:54:18,479][20347] Updated weights for policy 0, policy_version 1335984 (0.0010) [2023-12-27 14:54:18,489][20291] KL-divergence is very high: 111.6149 [2023-12-27 14:54:18,748][20381] Updated weights for policy 1, policy_version 1334932 (0.0008) [2023-12-27 14:54:18,805][20381] Updated weights for policy 1, policy_version 1334942 (0.0005) [2023-12-27 14:54:18,871][20381] Updated weights for policy 1, policy_version 1334952 (0.0008) [2023-12-27 14:54:19,071][20347] Updated weights for policy 0, policy_version 1335994 (0.0007) [2023-12-27 14:54:19,133][20347] Updated weights for policy 0, policy_version 1336004 (0.0007) [2023-12-27 14:54:19,193][20347] Updated weights for policy 0, policy_version 1336014 (0.0007) [2023-12-27 14:54:19,256][20347] Updated weights for policy 0, policy_version 1336024 (0.0010) [2023-12-27 14:54:19,552][20381] Updated weights for policy 1, policy_version 1334962 (0.0008) [2023-12-27 14:54:19,613][20381] Updated weights for policy 1, policy_version 1334972 (0.0009) [2023-12-27 14:54:19,678][20381] Updated weights for policy 1, policy_version 1334982 (0.0007) [2023-12-27 14:54:19,743][20381] Updated weights for policy 1, policy_version 1334992 (0.0006) [2023-12-27 14:54:19,906][20347] Updated weights for policy 0, policy_version 1336034 (0.0008) [2023-12-27 14:54:19,968][20347] Updated weights for policy 0, policy_version 1336044 (0.0007) [2023-12-27 14:54:20,024][20347] Updated weights for policy 0, policy_version 1336054 (0.0007) [2023-12-27 14:54:20,390][20381] Updated weights for policy 1, policy_version 1335002 (0.0005) [2023-12-27 14:54:20,446][20381] Updated weights for policy 1, policy_version 1335012 (0.0006) [2023-12-27 14:54:20,506][20381] Updated weights for policy 1, policy_version 1335022 (0.0008) [2023-12-27 14:54:20,782][20347] Updated weights for policy 0, policy_version 1336064 (0.0008) [2023-12-27 14:54:20,833][20347] Updated weights for policy 0, policy_version 1336074 (0.0008) [2023-12-27 14:54:20,886][20347] Updated weights for policy 0, policy_version 1336084 (0.0008) [2023-12-27 14:54:21,149][20381] Updated weights for policy 1, policy_version 1335032 (0.0006) [2023-12-27 14:54:21,212][20381] Updated weights for policy 1, policy_version 1335042 (0.0007) [2023-12-27 14:54:21,274][20381] Updated weights for policy 1, policy_version 1335052 (0.0007) [2023-12-27 14:54:21,738][20347] Updated weights for policy 0, policy_version 1336094 (0.0009) [2023-12-27 14:54:21,803][20347] Updated weights for policy 0, policy_version 1336104 (0.0008) [2023-12-27 14:54:21,863][20347] Updated weights for policy 0, policy_version 1336114 (0.0011) [2023-12-27 14:54:21,951][20381] Updated weights for policy 1, policy_version 1335062 (0.0007) [2023-12-27 14:54:22,013][20381] Updated weights for policy 1, policy_version 1335072 (0.0007) [2023-12-27 14:54:22,075][20381] Updated weights for policy 1, policy_version 1335082 (0.0007) [2023-12-27 14:54:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 683925504. Throughput: 0: 10382.1, 1: 10200.2. Samples: 683897132. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:22,363][19170] Avg episode reward: [(0, '746.130'), (1, '970.610')] [2023-12-27 14:54:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001335088_341811200.pth... [2023-12-27 14:54:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001336120_342114304.pth... [2023-12-27 14:54:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001333872_341499904.pth [2023-12-27 14:54:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001334912_341803008.pth [2023-12-27 14:54:22,540][20347] Updated weights for policy 0, policy_version 1336124 (0.0010) [2023-12-27 14:54:22,609][20347] Updated weights for policy 0, policy_version 1336134 (0.0007) [2023-12-27 14:54:22,675][20347] Updated weights for policy 0, policy_version 1336144 (0.0008) [2023-12-27 14:54:22,699][20381] Updated weights for policy 1, policy_version 1335092 (0.0007) [2023-12-27 14:54:22,756][20381] Updated weights for policy 1, policy_version 1335102 (0.0005) [2023-12-27 14:54:22,809][20381] Updated weights for policy 1, policy_version 1335112 (0.0006) [2023-12-27 14:54:23,262][20347] Updated weights for policy 0, policy_version 1336154 (0.0008) [2023-12-27 14:54:23,311][20347] Updated weights for policy 0, policy_version 1336164 (0.0005) [2023-12-27 14:54:23,357][20347] Updated weights for policy 0, policy_version 1336174 (0.0005) [2023-12-27 14:54:23,411][20347] Updated weights for policy 0, policy_version 1336184 (0.0005) [2023-12-27 14:54:23,489][20381] Updated weights for policy 1, policy_version 1335122 (0.0007) [2023-12-27 14:54:23,549][20381] Updated weights for policy 1, policy_version 1335132 (0.0006) [2023-12-27 14:54:23,608][20381] Updated weights for policy 1, policy_version 1335142 (0.0006) [2023-12-27 14:54:23,670][20381] Updated weights for policy 1, policy_version 1335152 (0.0005) [2023-12-27 14:54:24,077][20347] Updated weights for policy 0, policy_version 1336194 (0.0010) [2023-12-27 14:54:24,132][20347] Updated weights for policy 0, policy_version 1336204 (0.0010) [2023-12-27 14:54:24,136][20291] KL-divergence is very high: 118.6732 [2023-12-27 14:54:24,141][20291] KL-divergence is very high: 115.3594 [2023-12-27 14:54:24,178][20347] Updated weights for policy 0, policy_version 1336214 (0.0007) [2023-12-27 14:54:24,245][20381] Updated weights for policy 1, policy_version 1335162 (0.0006) [2023-12-27 14:54:24,292][20381] Updated weights for policy 1, policy_version 1335172 (0.0005) [2023-12-27 14:54:24,340][20381] Updated weights for policy 1, policy_version 1335182 (0.0006) [2023-12-27 14:54:24,826][20347] Updated weights for policy 0, policy_version 1336224 (0.0009) [2023-12-27 14:54:24,874][20347] Updated weights for policy 0, policy_version 1336234 (0.0010) [2023-12-27 14:54:24,919][20347] Updated weights for policy 0, policy_version 1336244 (0.0010) [2023-12-27 14:54:24,998][20381] Updated weights for policy 1, policy_version 1335192 (0.0007) [2023-12-27 14:54:25,049][20381] Updated weights for policy 1, policy_version 1335202 (0.0010) [2023-12-27 14:54:25,098][20381] Updated weights for policy 1, policy_version 1335212 (0.0010) [2023-12-27 14:54:25,525][20347] Updated weights for policy 0, policy_version 1336254 (0.0010) [2023-12-27 14:54:25,579][20347] Updated weights for policy 0, policy_version 1336264 (0.0010) [2023-12-27 14:54:25,626][20347] Updated weights for policy 0, policy_version 1336274 (0.0010) [2023-12-27 14:54:25,767][20381] Updated weights for policy 1, policy_version 1335222 (0.0010) [2023-12-27 14:54:25,828][20381] Updated weights for policy 1, policy_version 1335232 (0.0010) [2023-12-27 14:54:25,888][20381] Updated weights for policy 1, policy_version 1335242 (0.0010) [2023-12-27 14:54:26,269][20347] Updated weights for policy 0, policy_version 1336284 (0.0008) [2023-12-27 14:54:26,327][20347] Updated weights for policy 0, policy_version 1336294 (0.0005) [2023-12-27 14:54:26,385][20347] Updated weights for policy 0, policy_version 1336304 (0.0005) [2023-12-27 14:54:26,540][20381] Updated weights for policy 1, policy_version 1335252 (0.0008) [2023-12-27 14:54:26,592][20381] Updated weights for policy 1, policy_version 1335262 (0.0007) [2023-12-27 14:54:26,646][20381] Updated weights for policy 1, policy_version 1335272 (0.0010) [2023-12-27 14:54:26,972][20347] Updated weights for policy 0, policy_version 1336314 (0.0006) [2023-12-27 14:54:27,017][20347] Updated weights for policy 0, policy_version 1336324 (0.0010) [2023-12-27 14:54:27,062][20347] Updated weights for policy 0, policy_version 1336334 (0.0010) [2023-12-27 14:54:27,107][20347] Updated weights for policy 0, policy_version 1336344 (0.0010) [2023-12-27 14:54:27,279][20381] Updated weights for policy 1, policy_version 1335282 (0.0011) [2023-12-27 14:54:27,344][20381] Updated weights for policy 1, policy_version 1335292 (0.0011) [2023-12-27 14:54:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 684032000. Throughput: 0: 10357.4, 1: 10278.9. Samples: 684021728. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:27,363][19170] Avg episode reward: [(0, '824.530'), (1, '980.500')] [2023-12-27 14:54:27,405][20381] Updated weights for policy 1, policy_version 1335302 (0.0011) [2023-12-27 14:54:27,465][20381] Updated weights for policy 1, policy_version 1335312 (0.0011) [2023-12-27 14:54:27,777][20347] Updated weights for policy 0, policy_version 1336354 (0.0005) [2023-12-27 14:54:27,822][20347] Updated weights for policy 0, policy_version 1336364 (0.0005) [2023-12-27 14:54:27,873][20347] Updated weights for policy 0, policy_version 1336374 (0.0005) [2023-12-27 14:54:28,153][20381] Updated weights for policy 1, policy_version 1335322 (0.0011) [2023-12-27 14:54:28,204][20381] Updated weights for policy 1, policy_version 1335332 (0.0010) [2023-12-27 14:54:28,258][20381] Updated weights for policy 1, policy_version 1335342 (0.0010) [2023-12-27 14:54:28,440][20347] Updated weights for policy 0, policy_version 1336384 (0.0006) [2023-12-27 14:54:28,494][20347] Updated weights for policy 0, policy_version 1336394 (0.0007) [2023-12-27 14:54:28,555][20347] Updated weights for policy 0, policy_version 1336404 (0.0007) [2023-12-27 14:54:28,964][20381] Updated weights for policy 1, policy_version 1335352 (0.0008) [2023-12-27 14:54:29,010][20381] Updated weights for policy 1, policy_version 1335362 (0.0008) [2023-12-27 14:54:29,053][20381] Updated weights for policy 1, policy_version 1335372 (0.0006) [2023-12-27 14:54:29,260][20347] Updated weights for policy 0, policy_version 1336414 (0.0008) [2023-12-27 14:54:29,316][20347] Updated weights for policy 0, policy_version 1336424 (0.0005) [2023-12-27 14:54:29,375][20347] Updated weights for policy 0, policy_version 1336434 (0.0008) [2023-12-27 14:54:29,729][20381] Updated weights for policy 1, policy_version 1335382 (0.0007) [2023-12-27 14:54:29,790][20381] Updated weights for policy 1, policy_version 1335392 (0.0010) [2023-12-27 14:54:29,858][20381] Updated weights for policy 1, policy_version 1335402 (0.0009) [2023-12-27 14:54:30,104][20347] Updated weights for policy 0, policy_version 1336444 (0.0006) [2023-12-27 14:54:30,168][20347] Updated weights for policy 0, policy_version 1336454 (0.0006) [2023-12-27 14:54:30,233][20347] Updated weights for policy 0, policy_version 1336464 (0.0005) [2023-12-27 14:54:30,553][20381] Updated weights for policy 1, policy_version 1335412 (0.0009) [2023-12-27 14:54:30,623][20381] Updated weights for policy 1, policy_version 1335422 (0.0005) [2023-12-27 14:54:30,682][20381] Updated weights for policy 1, policy_version 1335432 (0.0008) [2023-12-27 14:54:30,819][20347] Updated weights for policy 0, policy_version 1336474 (0.0005) [2023-12-27 14:54:30,879][20347] Updated weights for policy 0, policy_version 1336484 (0.0005) [2023-12-27 14:54:30,932][20347] Updated weights for policy 0, policy_version 1336494 (0.0005) [2023-12-27 14:54:30,992][20347] Updated weights for policy 0, policy_version 1336504 (0.0007) [2023-12-27 14:54:31,259][20381] Updated weights for policy 1, policy_version 1335442 (0.0006) [2023-12-27 14:54:31,319][20381] Updated weights for policy 1, policy_version 1335452 (0.0007) [2023-12-27 14:54:31,380][20381] Updated weights for policy 1, policy_version 1335462 (0.0009) [2023-12-27 14:54:31,442][20381] Updated weights for policy 1, policy_version 1335472 (0.0009) [2023-12-27 14:54:31,593][20347] Updated weights for policy 0, policy_version 1336514 (0.0007) [2023-12-27 14:54:31,655][20347] Updated weights for policy 0, policy_version 1336524 (0.0008) [2023-12-27 14:54:31,711][20347] Updated weights for policy 0, policy_version 1336534 (0.0006) [2023-12-27 14:54:32,223][20381] Updated weights for policy 1, policy_version 1335482 (0.0009) [2023-12-27 14:54:32,291][20381] Updated weights for policy 1, policy_version 1335492 (0.0008) [2023-12-27 14:54:32,348][20381] Updated weights for policy 1, policy_version 1335502 (0.0007) [2023-12-27 14:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 684138496. Throughput: 0: 10461.1, 1: 10263.3. Samples: 684147984. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:32,363][19170] Avg episode reward: [(0, '843.670'), (1, '970.710')] [2023-12-27 14:54:32,441][20347] Updated weights for policy 0, policy_version 1336544 (0.0007) [2023-12-27 14:54:32,498][20347] Updated weights for policy 0, policy_version 1336554 (0.0006) [2023-12-27 14:54:32,553][20347] Updated weights for policy 0, policy_version 1336564 (0.0007) [2023-12-27 14:54:33,059][20381] Updated weights for policy 1, policy_version 1335512 (0.0008) [2023-12-27 14:54:33,102][20347] Updated weights for policy 0, policy_version 1336574 (0.0006) [2023-12-27 14:54:33,110][20381] Updated weights for policy 1, policy_version 1335522 (0.0008) [2023-12-27 14:54:33,163][20347] Updated weights for policy 0, policy_version 1336584 (0.0006) [2023-12-27 14:54:33,165][20381] Updated weights for policy 1, policy_version 1335532 (0.0008) [2023-12-27 14:54:33,214][20347] Updated weights for policy 0, policy_version 1336594 (0.0006) [2023-12-27 14:54:33,828][20381] Updated weights for policy 1, policy_version 1335542 (0.0009) [2023-12-27 14:54:33,885][20381] Updated weights for policy 1, policy_version 1335552 (0.0010) [2023-12-27 14:54:33,898][20347] Updated weights for policy 0, policy_version 1336604 (0.0006) [2023-12-27 14:54:33,939][20381] Updated weights for policy 1, policy_version 1335562 (0.0010) [2023-12-27 14:54:33,941][20347] Updated weights for policy 0, policy_version 1336614 (0.0005) [2023-12-27 14:54:33,964][20291] KL-divergence is very high: 100.5686 [2023-12-27 14:54:33,986][20347] Updated weights for policy 0, policy_version 1336624 (0.0006) [2023-12-27 14:54:34,646][20381] Updated weights for policy 1, policy_version 1335572 (0.0009) [2023-12-27 14:54:34,670][20347] Updated weights for policy 0, policy_version 1336634 (0.0007) [2023-12-27 14:54:34,706][20381] Updated weights for policy 1, policy_version 1335582 (0.0007) [2023-12-27 14:54:34,727][20347] Updated weights for policy 0, policy_version 1336644 (0.0005) [2023-12-27 14:54:34,770][20381] Updated weights for policy 1, policy_version 1335592 (0.0007) [2023-12-27 14:54:34,784][20347] Updated weights for policy 0, policy_version 1336654 (0.0005) [2023-12-27 14:54:34,841][20347] Updated weights for policy 0, policy_version 1336664 (0.0008) [2023-12-27 14:54:35,392][20381] Updated weights for policy 1, policy_version 1335602 (0.0008) [2023-12-27 14:54:35,449][20381] Updated weights for policy 1, policy_version 1335612 (0.0007) [2023-12-27 14:54:35,504][20381] Updated weights for policy 1, policy_version 1335622 (0.0007) [2023-12-27 14:54:35,562][20381] Updated weights for policy 1, policy_version 1335632 (0.0005) [2023-12-27 14:54:35,568][20347] Updated weights for policy 0, policy_version 1336674 (0.0008) [2023-12-27 14:54:35,626][20347] Updated weights for policy 0, policy_version 1336684 (0.0010) [2023-12-27 14:54:35,684][20347] Updated weights for policy 0, policy_version 1336694 (0.0010) [2023-12-27 14:54:36,188][20381] Updated weights for policy 1, policy_version 1335642 (0.0008) [2023-12-27 14:54:36,255][20381] Updated weights for policy 1, policy_version 1335652 (0.0008) [2023-12-27 14:54:36,318][20381] Updated weights for policy 1, policy_version 1335662 (0.0007) [2023-12-27 14:54:36,395][20347] Updated weights for policy 0, policy_version 1336704 (0.0011) [2023-12-27 14:54:36,452][20347] Updated weights for policy 0, policy_version 1336714 (0.0011) [2023-12-27 14:54:36,508][20347] Updated weights for policy 0, policy_version 1336724 (0.0011) [2023-12-27 14:54:37,018][20381] Updated weights for policy 1, policy_version 1335672 (0.0007) [2023-12-27 14:54:37,073][20381] Updated weights for policy 1, policy_version 1335682 (0.0007) [2023-12-27 14:54:37,127][20381] Updated weights for policy 1, policy_version 1335692 (0.0007) [2023-12-27 14:54:37,173][20347] Updated weights for policy 0, policy_version 1336734 (0.0009) [2023-12-27 14:54:37,224][20347] Updated weights for policy 0, policy_version 1336744 (0.0010) [2023-12-27 14:54:37,273][20347] Updated weights for policy 0, policy_version 1336754 (0.0010) [2023-12-27 14:54:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 684244992. Throughput: 0: 10462.3, 1: 10280.9. Samples: 684209620. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:37,364][19170] Avg episode reward: [(0, '844.020'), (1, '941.360')] [2023-12-27 14:54:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001335696_341966848.pth... [2023-12-27 14:54:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001336760_342278144.pth... [2023-12-27 14:54:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001334480_341655552.pth [2023-12-27 14:54:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001335552_341966848.pth [2023-12-27 14:54:37,830][20381] Updated weights for policy 1, policy_version 1335702 (0.0007) [2023-12-27 14:54:37,889][20381] Updated weights for policy 1, policy_version 1335712 (0.0008) [2023-12-27 14:54:37,940][20381] Updated weights for policy 1, policy_version 1335722 (0.0008) [2023-12-27 14:54:37,942][20347] Updated weights for policy 0, policy_version 1336764 (0.0009) [2023-12-27 14:54:37,997][20347] Updated weights for policy 0, policy_version 1336774 (0.0006) [2023-12-27 14:54:38,055][20347] Updated weights for policy 0, policy_version 1336784 (0.0009) [2023-12-27 14:54:38,655][20347] Updated weights for policy 0, policy_version 1336794 (0.0008) [2023-12-27 14:54:38,696][20381] Updated weights for policy 1, policy_version 1335732 (0.0007) [2023-12-27 14:54:38,710][20347] Updated weights for policy 0, policy_version 1336804 (0.0007) [2023-12-27 14:54:38,760][20381] Updated weights for policy 1, policy_version 1335742 (0.0008) [2023-12-27 14:54:38,770][20347] Updated weights for policy 0, policy_version 1336814 (0.0007) [2023-12-27 14:54:38,824][20347] Updated weights for policy 0, policy_version 1336824 (0.0006) [2023-12-27 14:54:38,826][20381] Updated weights for policy 1, policy_version 1335752 (0.0007) [2023-12-27 14:54:39,454][20347] Updated weights for policy 0, policy_version 1336834 (0.0009) [2023-12-27 14:54:39,516][20347] Updated weights for policy 0, policy_version 1336844 (0.0006) [2023-12-27 14:54:39,575][20347] Updated weights for policy 0, policy_version 1336854 (0.0007) [2023-12-27 14:54:39,579][20381] Updated weights for policy 1, policy_version 1335762 (0.0009) [2023-12-27 14:54:39,639][20381] Updated weights for policy 1, policy_version 1335772 (0.0009) [2023-12-27 14:54:39,695][20381] Updated weights for policy 1, policy_version 1335782 (0.0008) [2023-12-27 14:54:39,758][20381] Updated weights for policy 1, policy_version 1335792 (0.0008) [2023-12-27 14:54:40,266][20347] Updated weights for policy 0, policy_version 1336864 (0.0010) [2023-12-27 14:54:40,317][20347] Updated weights for policy 0, policy_version 1336874 (0.0008) [2023-12-27 14:54:40,374][20347] Updated weights for policy 0, policy_version 1336884 (0.0008) [2023-12-27 14:54:40,452][20381] Updated weights for policy 1, policy_version 1335802 (0.0007) [2023-12-27 14:54:40,513][20381] Updated weights for policy 1, policy_version 1335812 (0.0007) [2023-12-27 14:54:40,566][20381] Updated weights for policy 1, policy_version 1335822 (0.0008) [2023-12-27 14:54:41,134][20347] Updated weights for policy 0, policy_version 1336894 (0.0008) [2023-12-27 14:54:41,203][20347] Updated weights for policy 0, policy_version 1336904 (0.0008) [2023-12-27 14:54:41,271][20347] Updated weights for policy 0, policy_version 1336914 (0.0007) [2023-12-27 14:54:41,299][20381] Updated weights for policy 1, policy_version 1335832 (0.0008) [2023-12-27 14:54:41,368][20381] Updated weights for policy 1, policy_version 1335842 (0.0009) [2023-12-27 14:54:41,436][20381] Updated weights for policy 1, policy_version 1335852 (0.0006) [2023-12-27 14:54:42,036][20347] Updated weights for policy 0, policy_version 1336924 (0.0007) [2023-12-27 14:54:42,063][20381] Updated weights for policy 1, policy_version 1335862 (0.0009) [2023-12-27 14:54:42,092][20347] Updated weights for policy 0, policy_version 1336934 (0.0007) [2023-12-27 14:54:42,121][20381] Updated weights for policy 1, policy_version 1335872 (0.0005) [2023-12-27 14:54:42,146][20347] Updated weights for policy 0, policy_version 1336944 (0.0007) [2023-12-27 14:54:42,181][20381] Updated weights for policy 1, policy_version 1335882 (0.0006) [2023-12-27 14:54:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 684343296. Throughput: 0: 10465.1, 1: 10206.2. Samples: 684329992. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:42,363][19170] Avg episode reward: [(0, '843.910'), (1, '921.780')] [2023-12-27 14:54:42,849][20381] Updated weights for policy 1, policy_version 1335892 (0.0006) [2023-12-27 14:54:42,888][20347] Updated weights for policy 0, policy_version 1336954 (0.0007) [2023-12-27 14:54:42,902][20381] Updated weights for policy 1, policy_version 1335902 (0.0005) [2023-12-27 14:54:42,938][20347] Updated weights for policy 0, policy_version 1336964 (0.0006) [2023-12-27 14:54:42,952][20381] Updated weights for policy 1, policy_version 1335912 (0.0005) [2023-12-27 14:54:42,992][20347] Updated weights for policy 0, policy_version 1336974 (0.0006) [2023-12-27 14:54:43,041][20347] Updated weights for policy 0, policy_version 1336984 (0.0005) [2023-12-27 14:54:43,623][20381] Updated weights for policy 1, policy_version 1335922 (0.0005) [2023-12-27 14:54:43,677][20381] Updated weights for policy 1, policy_version 1335932 (0.0005) [2023-12-27 14:54:43,731][20347] Updated weights for policy 0, policy_version 1336994 (0.0006) [2023-12-27 14:54:43,736][20381] Updated weights for policy 1, policy_version 1335942 (0.0005) [2023-12-27 14:54:43,783][20347] Updated weights for policy 0, policy_version 1337004 (0.0005) [2023-12-27 14:54:43,793][20381] Updated weights for policy 1, policy_version 1335952 (0.0007) [2023-12-27 14:54:43,838][20347] Updated weights for policy 0, policy_version 1337014 (0.0008) [2023-12-27 14:54:44,417][20381] Updated weights for policy 1, policy_version 1335962 (0.0007) [2023-12-27 14:54:44,475][20381] Updated weights for policy 1, policy_version 1335972 (0.0007) [2023-12-27 14:54:44,503][20347] Updated weights for policy 0, policy_version 1337024 (0.0006) [2023-12-27 14:54:44,533][20381] Updated weights for policy 1, policy_version 1335982 (0.0006) [2023-12-27 14:54:44,558][20347] Updated weights for policy 0, policy_version 1337034 (0.0007) [2023-12-27 14:54:44,610][20347] Updated weights for policy 0, policy_version 1337044 (0.0007) [2023-12-27 14:54:45,244][20381] Updated weights for policy 1, policy_version 1335992 (0.0007) [2023-12-27 14:54:45,304][20381] Updated weights for policy 1, policy_version 1336002 (0.0007) [2023-12-27 14:54:45,311][20347] Updated weights for policy 0, policy_version 1337054 (0.0009) [2023-12-27 14:54:45,360][20381] Updated weights for policy 1, policy_version 1336012 (0.0008) [2023-12-27 14:54:45,367][20347] Updated weights for policy 0, policy_version 1337064 (0.0010) [2023-12-27 14:54:45,420][20347] Updated weights for policy 0, policy_version 1337074 (0.0010) [2023-12-27 14:54:46,068][20381] Updated weights for policy 1, policy_version 1336022 (0.0006) [2023-12-27 14:54:46,068][20347] Updated weights for policy 0, policy_version 1337084 (0.0009) [2023-12-27 14:54:46,122][20381] Updated weights for policy 1, policy_version 1336032 (0.0006) [2023-12-27 14:54:46,123][20347] Updated weights for policy 0, policy_version 1337094 (0.0008) [2023-12-27 14:54:46,172][20347] Updated weights for policy 0, policy_version 1337104 (0.0007) [2023-12-27 14:54:46,184][20381] Updated weights for policy 1, policy_version 1336042 (0.0005) [2023-12-27 14:54:46,786][20347] Updated weights for policy 0, policy_version 1337114 (0.0007) [2023-12-27 14:54:46,834][20347] Updated weights for policy 0, policy_version 1337124 (0.0005) [2023-12-27 14:54:46,868][20291] KL-divergence is very high: 117.0331 [2023-12-27 14:54:46,881][20381] Updated weights for policy 1, policy_version 1336052 (0.0007) [2023-12-27 14:54:46,883][20347] Updated weights for policy 0, policy_version 1337134 (0.0005) [2023-12-27 14:54:46,930][20381] Updated weights for policy 1, policy_version 1336062 (0.0007) [2023-12-27 14:54:46,946][20347] Updated weights for policy 0, policy_version 1337144 (0.0006) [2023-12-27 14:54:46,983][20381] Updated weights for policy 1, policy_version 1336072 (0.0008) [2023-12-27 14:54:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 684441600. Throughput: 0: 10455.8, 1: 10202.2. Samples: 684451772. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:47,363][19170] Avg episode reward: [(0, '814.290'), (1, '951.200')] [2023-12-27 14:54:47,502][20347] Updated weights for policy 0, policy_version 1337154 (0.0007) [2023-12-27 14:54:47,548][20347] Updated weights for policy 0, policy_version 1337164 (0.0007) [2023-12-27 14:54:47,595][20347] Updated weights for policy 0, policy_version 1337174 (0.0007) [2023-12-27 14:54:47,708][20381] Updated weights for policy 1, policy_version 1336082 (0.0008) [2023-12-27 14:54:47,766][20381] Updated weights for policy 1, policy_version 1336092 (0.0007) [2023-12-27 14:54:47,823][20381] Updated weights for policy 1, policy_version 1336102 (0.0007) [2023-12-27 14:54:47,877][20381] Updated weights for policy 1, policy_version 1336112 (0.0007) [2023-12-27 14:54:48,266][20347] Updated weights for policy 0, policy_version 1337184 (0.0010) [2023-12-27 14:54:48,331][20347] Updated weights for policy 0, policy_version 1337194 (0.0009) [2023-12-27 14:54:48,390][20347] Updated weights for policy 0, policy_version 1337204 (0.0008) [2023-12-27 14:54:48,548][20381] Updated weights for policy 1, policy_version 1336122 (0.0006) [2023-12-27 14:54:48,609][20381] Updated weights for policy 1, policy_version 1336132 (0.0006) [2023-12-27 14:54:48,668][20381] Updated weights for policy 1, policy_version 1336142 (0.0006) [2023-12-27 14:54:49,121][20347] Updated weights for policy 0, policy_version 1337214 (0.0008) [2023-12-27 14:54:49,168][20347] Updated weights for policy 0, policy_version 1337224 (0.0007) [2023-12-27 14:54:49,223][20347] Updated weights for policy 0, policy_version 1337234 (0.0007) [2023-12-27 14:54:49,226][20381] Updated weights for policy 1, policy_version 1336152 (0.0007) [2023-12-27 14:54:49,285][20381] Updated weights for policy 1, policy_version 1336162 (0.0008) [2023-12-27 14:54:49,340][20381] Updated weights for policy 1, policy_version 1336172 (0.0008) [2023-12-27 14:54:49,990][20347] Updated weights for policy 0, policy_version 1337244 (0.0007) [2023-12-27 14:54:50,053][20347] Updated weights for policy 0, policy_version 1337254 (0.0006) [2023-12-27 14:54:50,072][20381] Updated weights for policy 1, policy_version 1336182 (0.0010) [2023-12-27 14:54:50,112][20347] Updated weights for policy 0, policy_version 1337264 (0.0006) [2023-12-27 14:54:50,133][20381] Updated weights for policy 1, policy_version 1336192 (0.0011) [2023-12-27 14:54:50,192][20381] Updated weights for policy 1, policy_version 1336202 (0.0011) [2023-12-27 14:54:50,723][20347] Updated weights for policy 0, policy_version 1337274 (0.0007) [2023-12-27 14:54:50,749][20291] KL-divergence is very high: 109.6032 [2023-12-27 14:54:50,762][20291] KL-divergence is very high: 127.4740 [2023-12-27 14:54:50,768][20291] KL-divergence is very high: 125.3129 [2023-12-27 14:54:50,787][20347] Updated weights for policy 0, policy_version 1337284 (0.0011) [2023-12-27 14:54:50,849][20347] Updated weights for policy 0, policy_version 1337294 (0.0010) [2023-12-27 14:54:50,913][20347] Updated weights for policy 0, policy_version 1337304 (0.0011) [2023-12-27 14:54:50,945][20381] Updated weights for policy 1, policy_version 1336212 (0.0011) [2023-12-27 14:54:51,002][20381] Updated weights for policy 1, policy_version 1336222 (0.0011) [2023-12-27 14:54:51,075][20381] Updated weights for policy 1, policy_version 1336232 (0.0010) [2023-12-27 14:54:51,567][20347] Updated weights for policy 0, policy_version 1337314 (0.0010) [2023-12-27 14:54:51,630][20347] Updated weights for policy 0, policy_version 1337324 (0.0009) [2023-12-27 14:54:51,692][20347] Updated weights for policy 0, policy_version 1337334 (0.0010) [2023-12-27 14:54:51,825][20381] Updated weights for policy 1, policy_version 1336242 (0.0010) [2023-12-27 14:54:51,881][20381] Updated weights for policy 1, policy_version 1336252 (0.0008) [2023-12-27 14:54:51,939][20381] Updated weights for policy 1, policy_version 1336262 (0.0009) [2023-12-27 14:54:51,984][20381] Updated weights for policy 1, policy_version 1336272 (0.0008) [2023-12-27 14:54:52,327][20347] Updated weights for policy 0, policy_version 1337344 (0.0010) [2023-12-27 14:54:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 684539904. Throughput: 0: 10365.8, 1: 10225.8. Samples: 684513108. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:52,363][19170] Avg episode reward: [(0, '785.000'), (1, '960.980')] [2023-12-27 14:54:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001336272_342114304.pth... [2023-12-27 14:54:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001335088_341811200.pth [2023-12-27 14:54:52,395][20347] Updated weights for policy 0, policy_version 1337354 (0.0010) [2023-12-27 14:54:52,458][20347] Updated weights for policy 0, policy_version 1337364 (0.0011) [2023-12-27 14:54:52,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001337368_342433792.pth... [2023-12-27 14:54:52,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001336120_342114304.pth [2023-12-27 14:54:52,751][20381] Updated weights for policy 1, policy_version 1336282 (0.0008) [2023-12-27 14:54:52,811][20381] Updated weights for policy 1, policy_version 1336292 (0.0008) [2023-12-27 14:54:52,857][20381] Updated weights for policy 1, policy_version 1336302 (0.0010) [2023-12-27 14:54:53,056][20347] Updated weights for policy 0, policy_version 1337374 (0.0010) [2023-12-27 14:54:53,106][20347] Updated weights for policy 0, policy_version 1337384 (0.0010) [2023-12-27 14:54:53,156][20347] Updated weights for policy 0, policy_version 1337394 (0.0007) [2023-12-27 14:54:53,490][20381] Updated weights for policy 1, policy_version 1336312 (0.0011) [2023-12-27 14:54:53,538][20381] Updated weights for policy 1, policy_version 1336322 (0.0011) [2023-12-27 14:54:53,593][20381] Updated weights for policy 1, policy_version 1336332 (0.0011) [2023-12-27 14:54:53,718][20347] Updated weights for policy 0, policy_version 1337404 (0.0005) [2023-12-27 14:54:53,777][20347] Updated weights for policy 0, policy_version 1337414 (0.0005) [2023-12-27 14:54:53,831][20347] Updated weights for policy 0, policy_version 1337424 (0.0009) [2023-12-27 14:54:54,292][20381] Updated weights for policy 1, policy_version 1336342 (0.0010) [2023-12-27 14:54:54,346][20381] Updated weights for policy 1, policy_version 1336352 (0.0009) [2023-12-27 14:54:54,368][20347] Updated weights for policy 0, policy_version 1337434 (0.0010) [2023-12-27 14:54:54,407][20381] Updated weights for policy 1, policy_version 1336362 (0.0007) [2023-12-27 14:54:54,421][20347] Updated weights for policy 0, policy_version 1337444 (0.0010) [2023-12-27 14:54:54,476][20347] Updated weights for policy 0, policy_version 1337454 (0.0006) [2023-12-27 14:54:54,539][20347] Updated weights for policy 0, policy_version 1337464 (0.0010) [2023-12-27 14:54:55,009][20381] Updated weights for policy 1, policy_version 1336372 (0.0007) [2023-12-27 14:54:55,062][20381] Updated weights for policy 1, policy_version 1336382 (0.0008) [2023-12-27 14:54:55,078][20347] Updated weights for policy 0, policy_version 1337474 (0.0005) [2023-12-27 14:54:55,116][20381] Updated weights for policy 1, policy_version 1336392 (0.0008) [2023-12-27 14:54:55,129][20347] Updated weights for policy 0, policy_version 1337484 (0.0005) [2023-12-27 14:54:55,188][20347] Updated weights for policy 0, policy_version 1337494 (0.0005) [2023-12-27 14:54:55,719][20347] Updated weights for policy 0, policy_version 1337504 (0.0009) [2023-12-27 14:54:55,773][20347] Updated weights for policy 0, policy_version 1337514 (0.0010) [2023-12-27 14:54:55,807][20381] Updated weights for policy 1, policy_version 1336402 (0.0009) [2023-12-27 14:54:55,834][20347] Updated weights for policy 0, policy_version 1337524 (0.0010) [2023-12-27 14:54:55,859][20381] Updated weights for policy 1, policy_version 1336412 (0.0011) [2023-12-27 14:54:55,910][20381] Updated weights for policy 1, policy_version 1336422 (0.0010) [2023-12-27 14:54:55,971][20381] Updated weights for policy 1, policy_version 1336432 (0.0009) [2023-12-27 14:54:56,467][20347] Updated weights for policy 0, policy_version 1337534 (0.0010) [2023-12-27 14:54:56,514][20347] Updated weights for policy 0, policy_version 1337544 (0.0010) [2023-12-27 14:54:56,567][20347] Updated weights for policy 0, policy_version 1337554 (0.0010) [2023-12-27 14:54:56,581][20381] Updated weights for policy 1, policy_version 1336442 (0.0008) [2023-12-27 14:54:56,629][20381] Updated weights for policy 1, policy_version 1336452 (0.0010) [2023-12-27 14:54:56,683][20381] Updated weights for policy 1, policy_version 1336462 (0.0010) [2023-12-27 14:54:57,203][20347] Updated weights for policy 0, policy_version 1337564 (0.0008) [2023-12-27 14:54:57,253][20347] Updated weights for policy 0, policy_version 1337574 (0.0005) [2023-12-27 14:54:57,280][20381] Updated weights for policy 1, policy_version 1336472 (0.0009) [2023-12-27 14:54:57,317][20347] Updated weights for policy 0, policy_version 1337584 (0.0005) [2023-12-27 14:54:57,336][20381] Updated weights for policy 1, policy_version 1336482 (0.0008) [2023-12-27 14:54:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 684654592. Throughput: 0: 10531.9, 1: 10213.1. Samples: 684641480. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:54:57,363][19170] Avg episode reward: [(0, '843.630'), (1, '970.620')] [2023-12-27 14:54:57,388][20381] Updated weights for policy 1, policy_version 1336492 (0.0010) [2023-12-27 14:54:57,839][20347] Updated weights for policy 0, policy_version 1337594 (0.0007) [2023-12-27 14:54:57,897][20347] Updated weights for policy 0, policy_version 1337604 (0.0008) [2023-12-27 14:54:57,954][20347] Updated weights for policy 0, policy_version 1337614 (0.0007) [2023-12-27 14:54:58,006][20347] Updated weights for policy 0, policy_version 1337624 (0.0008) [2023-12-27 14:54:58,084][20381] Updated weights for policy 1, policy_version 1336502 (0.0009) [2023-12-27 14:54:58,137][20381] Updated weights for policy 1, policy_version 1336512 (0.0007) [2023-12-27 14:54:58,187][20381] Updated weights for policy 1, policy_version 1336522 (0.0007) [2023-12-27 14:54:58,769][20347] Updated weights for policy 0, policy_version 1337634 (0.0009) [2023-12-27 14:54:58,840][20347] Updated weights for policy 0, policy_version 1337644 (0.0008) [2023-12-27 14:54:58,904][20347] Updated weights for policy 0, policy_version 1337654 (0.0007) [2023-12-27 14:54:58,995][20381] Updated weights for policy 1, policy_version 1336532 (0.0008) [2023-12-27 14:54:59,057][20381] Updated weights for policy 1, policy_version 1336542 (0.0008) [2023-12-27 14:54:59,122][20381] Updated weights for policy 1, policy_version 1336552 (0.0008) [2023-12-27 14:54:59,732][20347] Updated weights for policy 0, policy_version 1337664 (0.0009) [2023-12-27 14:54:59,785][20347] Updated weights for policy 0, policy_version 1337674 (0.0009) [2023-12-27 14:54:59,839][20347] Updated weights for policy 0, policy_version 1337684 (0.0010) [2023-12-27 14:54:59,846][20381] Updated weights for policy 1, policy_version 1336562 (0.0007) [2023-12-27 14:54:59,902][20381] Updated weights for policy 1, policy_version 1336572 (0.0008) [2023-12-27 14:54:59,962][20381] Updated weights for policy 1, policy_version 1336582 (0.0007) [2023-12-27 14:55:00,019][20381] Updated weights for policy 1, policy_version 1336592 (0.0006) [2023-12-27 14:55:00,568][20347] Updated weights for policy 0, policy_version 1337694 (0.0009) [2023-12-27 14:55:00,613][20347] Updated weights for policy 0, policy_version 1337704 (0.0010) [2023-12-27 14:55:00,615][20381] Updated weights for policy 1, policy_version 1336602 (0.0005) [2023-12-27 14:55:00,657][20347] Updated weights for policy 0, policy_version 1337714 (0.0010) [2023-12-27 14:55:00,663][20381] Updated weights for policy 1, policy_version 1336612 (0.0005) [2023-12-27 14:55:00,709][20381] Updated weights for policy 1, policy_version 1336622 (0.0006) [2023-12-27 14:55:01,352][20347] Updated weights for policy 0, policy_version 1337724 (0.0009) [2023-12-27 14:55:01,363][20381] Updated weights for policy 1, policy_version 1336632 (0.0010) [2023-12-27 14:55:01,419][20347] Updated weights for policy 0, policy_version 1337734 (0.0008) [2023-12-27 14:55:01,420][20381] Updated weights for policy 1, policy_version 1336642 (0.0007) [2023-12-27 14:55:01,475][20381] Updated weights for policy 1, policy_version 1336652 (0.0009) [2023-12-27 14:55:01,478][20347] Updated weights for policy 0, policy_version 1337744 (0.0008) [2023-12-27 14:55:02,178][20381] Updated weights for policy 1, policy_version 1336662 (0.0011) [2023-12-27 14:55:02,194][20347] Updated weights for policy 0, policy_version 1337754 (0.0008) [2023-12-27 14:55:02,231][20381] Updated weights for policy 1, policy_version 1336672 (0.0007) [2023-12-27 14:55:02,245][20347] Updated weights for policy 0, policy_version 1337764 (0.0005) [2023-12-27 14:55:02,292][20381] Updated weights for policy 1, policy_version 1336682 (0.0010) [2023-12-27 14:55:02,300][20347] Updated weights for policy 0, policy_version 1337774 (0.0007) [2023-12-27 14:55:02,359][20347] Updated weights for policy 0, policy_version 1337784 (0.0008) [2023-12-27 14:55:02,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 684761088. Throughput: 0: 10500.9, 1: 10190.3. Samples: 684763472. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:02,363][19170] Avg episode reward: [(0, '892.450'), (1, '951.100')] [2023-12-27 14:55:02,927][20381] Updated weights for policy 1, policy_version 1336692 (0.0005) [2023-12-27 14:55:02,977][20381] Updated weights for policy 1, policy_version 1336702 (0.0006) [2023-12-27 14:55:03,005][20347] Updated weights for policy 0, policy_version 1337794 (0.0010) [2023-12-27 14:55:03,032][20381] Updated weights for policy 1, policy_version 1336712 (0.0005) [2023-12-27 14:55:03,060][20347] Updated weights for policy 0, policy_version 1337804 (0.0010) [2023-12-27 14:55:03,117][20347] Updated weights for policy 0, policy_version 1337814 (0.0010) [2023-12-27 14:55:03,624][20381] Updated weights for policy 1, policy_version 1336722 (0.0006) [2023-12-27 14:55:03,681][20381] Updated weights for policy 1, policy_version 1336732 (0.0010) [2023-12-27 14:55:03,697][20347] Updated weights for policy 0, policy_version 1337824 (0.0009) [2023-12-27 14:55:03,730][20381] Updated weights for policy 1, policy_version 1336742 (0.0009) [2023-12-27 14:55:03,748][20347] Updated weights for policy 0, policy_version 1337834 (0.0009) [2023-12-27 14:55:03,785][20381] Updated weights for policy 1, policy_version 1336752 (0.0005) [2023-12-27 14:55:03,787][20291] KL-divergence is very high: 246.4992 [2023-12-27 14:55:03,806][20347] Updated weights for policy 0, policy_version 1337844 (0.0009) [2023-12-27 14:55:04,427][20381] Updated weights for policy 1, policy_version 1336762 (0.0007) [2023-12-27 14:55:04,486][20381] Updated weights for policy 1, policy_version 1336772 (0.0008) [2023-12-27 14:55:04,513][20347] Updated weights for policy 0, policy_version 1337854 (0.0008) [2023-12-27 14:55:04,545][20381] Updated weights for policy 1, policy_version 1336782 (0.0008) [2023-12-27 14:55:04,570][20347] Updated weights for policy 0, policy_version 1337864 (0.0008) [2023-12-27 14:55:04,632][20347] Updated weights for policy 0, policy_version 1337874 (0.0008) [2023-12-27 14:55:05,191][20381] Updated weights for policy 1, policy_version 1336792 (0.0010) [2023-12-27 14:55:05,244][20381] Updated weights for policy 1, policy_version 1336802 (0.0006) [2023-12-27 14:55:05,287][20347] Updated weights for policy 0, policy_version 1337884 (0.0007) [2023-12-27 14:55:05,302][20381] Updated weights for policy 1, policy_version 1336812 (0.0009) [2023-12-27 14:55:05,344][20347] Updated weights for policy 0, policy_version 1337894 (0.0006) [2023-12-27 14:55:05,394][20347] Updated weights for policy 0, policy_version 1337904 (0.0007) [2023-12-27 14:55:05,926][20381] Updated weights for policy 1, policy_version 1336822 (0.0007) [2023-12-27 14:55:05,985][20381] Updated weights for policy 1, policy_version 1336832 (0.0005) [2023-12-27 14:55:05,993][20347] Updated weights for policy 0, policy_version 1337914 (0.0006) [2023-12-27 14:55:06,036][20381] Updated weights for policy 1, policy_version 1336842 (0.0005) [2023-12-27 14:55:06,058][20347] Updated weights for policy 0, policy_version 1337924 (0.0009) [2023-12-27 14:55:06,116][20347] Updated weights for policy 0, policy_version 1337934 (0.0010) [2023-12-27 14:55:06,174][20347] Updated weights for policy 0, policy_version 1337944 (0.0010) [2023-12-27 14:55:06,737][20381] Updated weights for policy 1, policy_version 1336852 (0.0006) [2023-12-27 14:55:06,788][20381] Updated weights for policy 1, policy_version 1336862 (0.0008) [2023-12-27 14:55:06,821][20347] Updated weights for policy 0, policy_version 1337954 (0.0008) [2023-12-27 14:55:06,842][20381] Updated weights for policy 1, policy_version 1336872 (0.0005) [2023-12-27 14:55:06,870][20347] Updated weights for policy 0, policy_version 1337964 (0.0009) [2023-12-27 14:55:06,932][20347] Updated weights for policy 0, policy_version 1337974 (0.0009) [2023-12-27 14:55:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 684859392. Throughput: 0: 10447.4, 1: 10212.2. Samples: 684826820. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:07,364][19170] Avg episode reward: [(0, '843.850'), (1, '960.910')] [2023-12-27 14:55:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001337976_342589440.pth... [2023-12-27 14:55:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001336880_342269952.pth... [2023-12-27 14:55:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001336760_342278144.pth [2023-12-27 14:55:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001335696_341966848.pth [2023-12-27 14:55:07,508][20381] Updated weights for policy 1, policy_version 1336882 (0.0005) [2023-12-27 14:55:07,556][20381] Updated weights for policy 1, policy_version 1336892 (0.0007) [2023-12-27 14:55:07,616][20381] Updated weights for policy 1, policy_version 1336902 (0.0009) [2023-12-27 14:55:07,636][20347] Updated weights for policy 0, policy_version 1337984 (0.0009) [2023-12-27 14:55:07,667][20381] Updated weights for policy 1, policy_version 1336912 (0.0007) [2023-12-27 14:55:07,687][20347] Updated weights for policy 0, policy_version 1337994 (0.0010) [2023-12-27 14:55:07,738][20347] Updated weights for policy 0, policy_version 1338004 (0.0008) [2023-12-27 14:55:08,273][20381] Updated weights for policy 1, policy_version 1336922 (0.0005) [2023-12-27 14:55:08,320][20347] Updated weights for policy 0, policy_version 1338014 (0.0006) [2023-12-27 14:55:08,325][20381] Updated weights for policy 1, policy_version 1336932 (0.0006) [2023-12-27 14:55:08,377][20347] Updated weights for policy 0, policy_version 1338024 (0.0009) [2023-12-27 14:55:08,380][20381] Updated weights for policy 1, policy_version 1336942 (0.0008) [2023-12-27 14:55:08,437][20347] Updated weights for policy 0, policy_version 1338034 (0.0011) [2023-12-27 14:55:09,043][20381] Updated weights for policy 1, policy_version 1336952 (0.0010) [2023-12-27 14:55:09,099][20381] Updated weights for policy 1, policy_version 1336962 (0.0011) [2023-12-27 14:55:09,126][20347] Updated weights for policy 0, policy_version 1338044 (0.0010) [2023-12-27 14:55:09,159][20381] Updated weights for policy 1, policy_version 1336972 (0.0011) [2023-12-27 14:55:09,179][20347] Updated weights for policy 0, policy_version 1338054 (0.0011) [2023-12-27 14:55:09,244][20347] Updated weights for policy 0, policy_version 1338064 (0.0011) [2023-12-27 14:55:09,971][20381] Updated weights for policy 1, policy_version 1336982 (0.0008) [2023-12-27 14:55:10,024][20381] Updated weights for policy 1, policy_version 1336992 (0.0006) [2023-12-27 14:55:10,087][20381] Updated weights for policy 1, policy_version 1337002 (0.0008) [2023-12-27 14:55:10,122][20347] Updated weights for policy 0, policy_version 1338074 (0.0010) [2023-12-27 14:55:10,185][20347] Updated weights for policy 0, policy_version 1338084 (0.0007) [2023-12-27 14:55:10,245][20347] Updated weights for policy 0, policy_version 1338094 (0.0007) [2023-12-27 14:55:10,299][20347] Updated weights for policy 0, policy_version 1338104 (0.0008) [2023-12-27 14:55:10,790][20381] Updated weights for policy 1, policy_version 1337012 (0.0007) [2023-12-27 14:55:10,850][20381] Updated weights for policy 1, policy_version 1337022 (0.0005) [2023-12-27 14:55:10,906][20381] Updated weights for policy 1, policy_version 1337032 (0.0008) [2023-12-27 14:55:10,936][20347] Updated weights for policy 0, policy_version 1338114 (0.0005) [2023-12-27 14:55:10,992][20347] Updated weights for policy 0, policy_version 1338124 (0.0005) [2023-12-27 14:55:11,045][20347] Updated weights for policy 0, policy_version 1338134 (0.0006) [2023-12-27 14:55:11,629][20381] Updated weights for policy 1, policy_version 1337042 (0.0010) [2023-12-27 14:55:11,697][20381] Updated weights for policy 1, policy_version 1337052 (0.0007) [2023-12-27 14:55:11,749][20347] Updated weights for policy 0, policy_version 1338144 (0.0008) [2023-12-27 14:55:11,760][20381] Updated weights for policy 1, policy_version 1337062 (0.0008) [2023-12-27 14:55:11,807][20347] Updated weights for policy 0, policy_version 1338154 (0.0007) [2023-12-27 14:55:11,819][20381] Updated weights for policy 1, policy_version 1337072 (0.0007) [2023-12-27 14:55:11,869][20347] Updated weights for policy 0, policy_version 1338164 (0.0006) [2023-12-27 14:55:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 684957696. Throughput: 0: 10486.0, 1: 10164.8. Samples: 684951016. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:12,363][19170] Avg episode reward: [(0, '853.820'), (1, '970.690')] [2023-12-27 14:55:12,457][20381] Updated weights for policy 1, policy_version 1337082 (0.0008) [2023-12-27 14:55:12,512][20381] Updated weights for policy 1, policy_version 1337092 (0.0007) [2023-12-27 14:55:12,531][20347] Updated weights for policy 0, policy_version 1338174 (0.0006) [2023-12-27 14:55:12,568][20381] Updated weights for policy 1, policy_version 1337102 (0.0008) [2023-12-27 14:55:12,592][20347] Updated weights for policy 0, policy_version 1338184 (0.0008) [2023-12-27 14:55:12,654][20347] Updated weights for policy 0, policy_version 1338194 (0.0006) [2023-12-27 14:55:13,222][20381] Updated weights for policy 1, policy_version 1337112 (0.0006) [2023-12-27 14:55:13,274][20381] Updated weights for policy 1, policy_version 1337122 (0.0005) [2023-12-27 14:55:13,281][20347] Updated weights for policy 0, policy_version 1338204 (0.0007) [2023-12-27 14:55:13,320][20381] Updated weights for policy 1, policy_version 1337132 (0.0005) [2023-12-27 14:55:13,336][20347] Updated weights for policy 0, policy_version 1338214 (0.0007) [2023-12-27 14:55:13,393][20347] Updated weights for policy 0, policy_version 1338224 (0.0006) [2023-12-27 14:55:13,937][20381] Updated weights for policy 1, policy_version 1337142 (0.0007) [2023-12-27 14:55:13,985][20381] Updated weights for policy 1, policy_version 1337152 (0.0007) [2023-12-27 14:55:14,030][20381] Updated weights for policy 1, policy_version 1337162 (0.0008) [2023-12-27 14:55:14,092][20347] Updated weights for policy 0, policy_version 1338234 (0.0008) [2023-12-27 14:55:14,153][20347] Updated weights for policy 0, policy_version 1338244 (0.0010) [2023-12-27 14:55:14,207][20347] Updated weights for policy 0, policy_version 1338254 (0.0005) [2023-12-27 14:55:14,259][20347] Updated weights for policy 0, policy_version 1338264 (0.0005) [2023-12-27 14:55:14,717][20381] Updated weights for policy 1, policy_version 1337172 (0.0007) [2023-12-27 14:55:14,776][20381] Updated weights for policy 1, policy_version 1337182 (0.0007) [2023-12-27 14:55:14,844][20381] Updated weights for policy 1, policy_version 1337192 (0.0008) [2023-12-27 14:55:14,883][20347] Updated weights for policy 0, policy_version 1338274 (0.0008) [2023-12-27 14:55:14,949][20347] Updated weights for policy 0, policy_version 1338284 (0.0007) [2023-12-27 14:55:15,003][20347] Updated weights for policy 0, policy_version 1338294 (0.0010) [2023-12-27 14:55:15,524][20381] Updated weights for policy 1, policy_version 1337202 (0.0007) [2023-12-27 14:55:15,577][20381] Updated weights for policy 1, policy_version 1337212 (0.0007) [2023-12-27 14:55:15,636][20381] Updated weights for policy 1, policy_version 1337222 (0.0008) [2023-12-27 14:55:15,669][20347] Updated weights for policy 0, policy_version 1338304 (0.0007) [2023-12-27 14:55:15,691][20381] Updated weights for policy 1, policy_version 1337232 (0.0008) [2023-12-27 14:55:15,720][20347] Updated weights for policy 0, policy_version 1338314 (0.0007) [2023-12-27 14:55:15,776][20347] Updated weights for policy 0, policy_version 1338324 (0.0007) [2023-12-27 14:55:16,399][20347] Updated weights for policy 0, policy_version 1338334 (0.0007) [2023-12-27 14:55:16,409][20381] Updated weights for policy 1, policy_version 1337242 (0.0005) [2023-12-27 14:55:16,447][20347] Updated weights for policy 0, policy_version 1338344 (0.0006) [2023-12-27 14:55:16,464][20381] Updated weights for policy 1, policy_version 1337252 (0.0005) [2023-12-27 14:55:16,498][20347] Updated weights for policy 0, policy_version 1338354 (0.0005) [2023-12-27 14:55:16,516][20381] Updated weights for policy 1, policy_version 1337262 (0.0007) [2023-12-27 14:55:17,167][20347] Updated weights for policy 0, policy_version 1338364 (0.0005) [2023-12-27 14:55:17,190][20381] Updated weights for policy 1, policy_version 1337272 (0.0008) [2023-12-27 14:55:17,215][20347] Updated weights for policy 0, policy_version 1338374 (0.0005) [2023-12-27 14:55:17,248][20381] Updated weights for policy 1, policy_version 1337282 (0.0008) [2023-12-27 14:55:17,265][20347] Updated weights for policy 0, policy_version 1338384 (0.0005) [2023-12-27 14:55:17,307][20381] Updated weights for policy 1, policy_version 1337292 (0.0008) [2023-12-27 14:55:17,363][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 685072384. Throughput: 0: 10427.2, 1: 10178.1. Samples: 685075224. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:17,364][19170] Avg episode reward: [(0, '873.520'), (1, '980.480')] [2023-12-27 14:55:17,772][20347] Updated weights for policy 0, policy_version 1338394 (0.0005) [2023-12-27 14:55:17,820][20347] Updated weights for policy 0, policy_version 1338404 (0.0006) [2023-12-27 14:55:17,869][20347] Updated weights for policy 0, policy_version 1338414 (0.0006) [2023-12-27 14:55:17,914][20347] Updated weights for policy 0, policy_version 1338424 (0.0005) [2023-12-27 14:55:18,068][20381] Updated weights for policy 1, policy_version 1337302 (0.0007) [2023-12-27 14:55:18,123][20381] Updated weights for policy 1, policy_version 1337312 (0.0007) [2023-12-27 14:55:18,182][20381] Updated weights for policy 1, policy_version 1337322 (0.0006) [2023-12-27 14:55:18,552][20347] Updated weights for policy 0, policy_version 1338434 (0.0007) [2023-12-27 14:55:18,609][20347] Updated weights for policy 0, policy_version 1338444 (0.0008) [2023-12-27 14:55:18,665][20347] Updated weights for policy 0, policy_version 1338454 (0.0008) [2023-12-27 14:55:18,830][20381] Updated weights for policy 1, policy_version 1337332 (0.0007) [2023-12-27 14:55:18,882][20381] Updated weights for policy 1, policy_version 1337342 (0.0007) [2023-12-27 14:55:18,931][20381] Updated weights for policy 1, policy_version 1337352 (0.0007) [2023-12-27 14:55:19,406][20347] Updated weights for policy 0, policy_version 1338464 (0.0007) [2023-12-27 14:55:19,461][20347] Updated weights for policy 0, policy_version 1338474 (0.0007) [2023-12-27 14:55:19,522][20347] Updated weights for policy 0, policy_version 1338484 (0.0008) [2023-12-27 14:55:19,610][20381] Updated weights for policy 1, policy_version 1337362 (0.0006) [2023-12-27 14:55:19,672][20381] Updated weights for policy 1, policy_version 1337372 (0.0006) [2023-12-27 14:55:19,733][20381] Updated weights for policy 1, policy_version 1337382 (0.0006) [2023-12-27 14:55:19,785][20381] Updated weights for policy 1, policy_version 1337392 (0.0006) [2023-12-27 14:55:20,189][20347] Updated weights for policy 0, policy_version 1338494 (0.0007) [2023-12-27 14:55:20,256][20347] Updated weights for policy 0, policy_version 1338504 (0.0008) [2023-12-27 14:55:20,322][20347] Updated weights for policy 0, policy_version 1338514 (0.0008) [2023-12-27 14:55:20,441][20381] Updated weights for policy 1, policy_version 1337402 (0.0008) [2023-12-27 14:55:20,500][20381] Updated weights for policy 1, policy_version 1337412 (0.0007) [2023-12-27 14:55:20,544][20381] Updated weights for policy 1, policy_version 1337422 (0.0006) [2023-12-27 14:55:21,033][20347] Updated weights for policy 0, policy_version 1338524 (0.0006) [2023-12-27 14:55:21,096][20347] Updated weights for policy 0, policy_version 1338534 (0.0007) [2023-12-27 14:55:21,162][20347] Updated weights for policy 0, policy_version 1338544 (0.0008) [2023-12-27 14:55:21,211][20381] Updated weights for policy 1, policy_version 1337432 (0.0008) [2023-12-27 14:55:21,272][20381] Updated weights for policy 1, policy_version 1337442 (0.0008) [2023-12-27 14:55:21,330][20381] Updated weights for policy 1, policy_version 1337452 (0.0006) [2023-12-27 14:55:21,890][20347] Updated weights for policy 0, policy_version 1338554 (0.0008) [2023-12-27 14:55:21,949][20347] Updated weights for policy 0, policy_version 1338564 (0.0008) [2023-12-27 14:55:22,009][20347] Updated weights for policy 0, policy_version 1338574 (0.0008) [2023-12-27 14:55:22,059][20381] Updated weights for policy 1, policy_version 1337462 (0.0007) [2023-12-27 14:55:22,070][20347] Updated weights for policy 0, policy_version 1338584 (0.0008) [2023-12-27 14:55:22,113][20381] Updated weights for policy 1, policy_version 1337472 (0.0006) [2023-12-27 14:55:22,175][20381] Updated weights for policy 1, policy_version 1337482 (0.0007) [2023-12-27 14:55:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 685170688. Throughput: 0: 10447.7, 1: 10184.0. Samples: 685138040. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:22,363][19170] Avg episode reward: [(0, '863.390'), (1, '990.220')] [2023-12-27 14:55:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001337488_342425600.pth... [2023-12-27 14:55:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001338584_342745088.pth... [2023-12-27 14:55:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001336272_342114304.pth [2023-12-27 14:55:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001337368_342433792.pth [2023-12-27 14:55:22,841][20347] Updated weights for policy 0, policy_version 1338594 (0.0007) [2023-12-27 14:55:22,847][20381] Updated weights for policy 1, policy_version 1337492 (0.0008) [2023-12-27 14:55:22,901][20381] Updated weights for policy 1, policy_version 1337502 (0.0006) [2023-12-27 14:55:22,906][20347] Updated weights for policy 0, policy_version 1338604 (0.0008) [2023-12-27 14:55:22,957][20381] Updated weights for policy 1, policy_version 1337512 (0.0006) [2023-12-27 14:55:22,967][20347] Updated weights for policy 0, policy_version 1338614 (0.0007) [2023-12-27 14:55:23,593][20381] Updated weights for policy 1, policy_version 1337522 (0.0006) [2023-12-27 14:55:23,642][20381] Updated weights for policy 1, policy_version 1337532 (0.0009) [2023-12-27 14:55:23,655][20347] Updated weights for policy 0, policy_version 1338624 (0.0006) [2023-12-27 14:55:23,691][20381] Updated weights for policy 1, policy_version 1337542 (0.0007) [2023-12-27 14:55:23,706][20347] Updated weights for policy 0, policy_version 1338634 (0.0007) [2023-12-27 14:55:23,754][20347] Updated weights for policy 0, policy_version 1338644 (0.0006) [2023-12-27 14:55:23,754][20381] Updated weights for policy 1, policy_version 1337552 (0.0008) [2023-12-27 14:55:24,377][20347] Updated weights for policy 0, policy_version 1338654 (0.0005) [2023-12-27 14:55:24,394][20381] Updated weights for policy 1, policy_version 1337562 (0.0011) [2023-12-27 14:55:24,430][20347] Updated weights for policy 0, policy_version 1338664 (0.0006) [2023-12-27 14:55:24,448][20381] Updated weights for policy 1, policy_version 1337572 (0.0010) [2023-12-27 14:55:24,491][20347] Updated weights for policy 0, policy_version 1338674 (0.0005) [2023-12-27 14:55:24,508][20381] Updated weights for policy 1, policy_version 1337582 (0.0010) [2023-12-27 14:55:25,089][20347] Updated weights for policy 0, policy_version 1338684 (0.0006) [2023-12-27 14:55:25,145][20347] Updated weights for policy 0, policy_version 1338694 (0.0007) [2023-12-27 14:55:25,189][20381] Updated weights for policy 1, policy_version 1337592 (0.0011) [2023-12-27 14:55:25,190][20347] Updated weights for policy 0, policy_version 1338704 (0.0007) [2023-12-27 14:55:25,243][20381] Updated weights for policy 1, policy_version 1337602 (0.0010) [2023-12-27 14:55:25,292][20381] Updated weights for policy 1, policy_version 1337612 (0.0010) [2023-12-27 14:55:25,914][20347] Updated weights for policy 0, policy_version 1338714 (0.0006) [2023-12-27 14:55:25,917][20381] Updated weights for policy 1, policy_version 1337622 (0.0009) [2023-12-27 14:55:25,968][20347] Updated weights for policy 0, policy_version 1338724 (0.0005) [2023-12-27 14:55:25,972][20381] Updated weights for policy 1, policy_version 1337632 (0.0010) [2023-12-27 14:55:26,025][20347] Updated weights for policy 0, policy_version 1338734 (0.0005) [2023-12-27 14:55:26,037][20381] Updated weights for policy 1, policy_version 1337642 (0.0010) [2023-12-27 14:55:26,077][20347] Updated weights for policy 0, policy_version 1338744 (0.0007) [2023-12-27 14:55:26,715][20381] Updated weights for policy 1, policy_version 1337652 (0.0010) [2023-12-27 14:55:26,738][20347] Updated weights for policy 0, policy_version 1338754 (0.0009) [2023-12-27 14:55:26,764][20381] Updated weights for policy 1, policy_version 1337662 (0.0010) [2023-12-27 14:55:26,792][20347] Updated weights for policy 0, policy_version 1338764 (0.0010) [2023-12-27 14:55:26,812][20381] Updated weights for policy 1, policy_version 1337672 (0.0010) [2023-12-27 14:55:26,845][20347] Updated weights for policy 0, policy_version 1338774 (0.0005) [2023-12-27 14:55:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 685268992. Throughput: 0: 10411.0, 1: 10255.8. Samples: 685260000. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:27,363][19170] Avg episode reward: [(0, '912.180'), (1, '1000.000')] [2023-12-27 14:55:27,430][20347] Updated weights for policy 0, policy_version 1338784 (0.0009) [2023-12-27 14:55:27,486][20347] Updated weights for policy 0, policy_version 1338794 (0.0010) [2023-12-27 14:55:27,519][20381] Updated weights for policy 1, policy_version 1337682 (0.0010) [2023-12-27 14:55:27,536][20347] Updated weights for policy 0, policy_version 1338804 (0.0010) [2023-12-27 14:55:27,575][20381] Updated weights for policy 1, policy_version 1337692 (0.0010) [2023-12-27 14:55:27,626][20381] Updated weights for policy 1, policy_version 1337702 (0.0010) [2023-12-27 14:55:27,673][20381] Updated weights for policy 1, policy_version 1337712 (0.0010) [2023-12-27 14:55:28,196][20347] Updated weights for policy 0, policy_version 1338814 (0.0010) [2023-12-27 14:55:28,250][20381] Updated weights for policy 1, policy_version 1337722 (0.0010) [2023-12-27 14:55:28,252][20347] Updated weights for policy 0, policy_version 1338824 (0.0010) [2023-12-27 14:55:28,308][20381] Updated weights for policy 1, policy_version 1337732 (0.0010) [2023-12-27 14:55:28,310][20347] Updated weights for policy 0, policy_version 1338834 (0.0009) [2023-12-27 14:55:28,367][20381] Updated weights for policy 1, policy_version 1337742 (0.0008) [2023-12-27 14:55:28,960][20347] Updated weights for policy 0, policy_version 1338844 (0.0008) [2023-12-27 14:55:29,015][20347] Updated weights for policy 0, policy_version 1338854 (0.0008) [2023-12-27 14:55:29,062][20381] Updated weights for policy 1, policy_version 1337752 (0.0009) [2023-12-27 14:55:29,063][20347] Updated weights for policy 0, policy_version 1338864 (0.0006) [2023-12-27 14:55:29,114][20381] Updated weights for policy 1, policy_version 1337762 (0.0010) [2023-12-27 14:55:29,158][20381] Updated weights for policy 1, policy_version 1337772 (0.0010) [2023-12-27 14:55:29,733][20347] Updated weights for policy 0, policy_version 1338874 (0.0005) [2023-12-27 14:55:29,795][20347] Updated weights for policy 0, policy_version 1338884 (0.0008) [2023-12-27 14:55:29,860][20347] Updated weights for policy 0, policy_version 1338894 (0.0007) [2023-12-27 14:55:29,887][20381] Updated weights for policy 1, policy_version 1337782 (0.0009) [2023-12-27 14:55:29,925][20347] Updated weights for policy 0, policy_version 1338904 (0.0009) [2023-12-27 14:55:29,951][20381] Updated weights for policy 1, policy_version 1337792 (0.0008) [2023-12-27 14:55:30,013][20381] Updated weights for policy 1, policy_version 1337802 (0.0008) [2023-12-27 14:55:30,538][20347] Updated weights for policy 0, policy_version 1338914 (0.0007) [2023-12-27 14:55:30,598][20347] Updated weights for policy 0, policy_version 1338924 (0.0008) [2023-12-27 14:55:30,652][20347] Updated weights for policy 0, policy_version 1338934 (0.0008) [2023-12-27 14:55:30,688][20381] Updated weights for policy 1, policy_version 1337812 (0.0008) [2023-12-27 14:55:30,745][20381] Updated weights for policy 1, policy_version 1337822 (0.0008) [2023-12-27 14:55:30,789][20381] Updated weights for policy 1, policy_version 1337832 (0.0007) [2023-12-27 14:55:31,387][20347] Updated weights for policy 0, policy_version 1338944 (0.0009) [2023-12-27 14:55:31,448][20347] Updated weights for policy 0, policy_version 1338954 (0.0011) [2023-12-27 14:55:31,451][20381] Updated weights for policy 1, policy_version 1337842 (0.0006) [2023-12-27 14:55:31,495][20347] Updated weights for policy 0, policy_version 1338964 (0.0010) [2023-12-27 14:55:31,500][20381] Updated weights for policy 1, policy_version 1337852 (0.0008) [2023-12-27 14:55:31,548][20381] Updated weights for policy 1, policy_version 1337862 (0.0008) [2023-12-27 14:55:31,595][20381] Updated weights for policy 1, policy_version 1337872 (0.0008) [2023-12-27 14:55:32,232][20347] Updated weights for policy 0, policy_version 1338974 (0.0008) [2023-12-27 14:55:32,288][20347] Updated weights for policy 0, policy_version 1338984 (0.0008) [2023-12-27 14:55:32,303][20381] Updated weights for policy 1, policy_version 1337882 (0.0007) [2023-12-27 14:55:32,344][20347] Updated weights for policy 0, policy_version 1338994 (0.0007) [2023-12-27 14:55:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 685367296. Throughput: 0: 10438.5, 1: 10288.2. Samples: 685384472. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:32,363][19170] Avg episode reward: [(0, '922.100'), (1, '990.240')] [2023-12-27 14:55:32,369][20381] Updated weights for policy 1, policy_version 1337892 (0.0006) [2023-12-27 14:55:32,428][20381] Updated weights for policy 1, policy_version 1337902 (0.0007) [2023-12-27 14:55:33,031][20347] Updated weights for policy 0, policy_version 1339004 (0.0008) [2023-12-27 14:55:33,077][20347] Updated weights for policy 0, policy_version 1339014 (0.0006) [2023-12-27 14:55:33,109][20381] Updated weights for policy 1, policy_version 1337912 (0.0006) [2023-12-27 14:55:33,122][20347] Updated weights for policy 0, policy_version 1339024 (0.0005) [2023-12-27 14:55:33,155][20381] Updated weights for policy 1, policy_version 1337922 (0.0006) [2023-12-27 14:55:33,213][20381] Updated weights for policy 1, policy_version 1337932 (0.0005) [2023-12-27 14:55:33,667][20347] Updated weights for policy 0, policy_version 1339034 (0.0005) [2023-12-27 14:55:33,712][20347] Updated weights for policy 0, policy_version 1339044 (0.0005) [2023-12-27 14:55:33,757][20347] Updated weights for policy 0, policy_version 1339054 (0.0005) [2023-12-27 14:55:33,801][20347] Updated weights for policy 0, policy_version 1339064 (0.0006) [2023-12-27 14:55:33,869][20381] Updated weights for policy 1, policy_version 1337942 (0.0008) [2023-12-27 14:55:33,914][20381] Updated weights for policy 1, policy_version 1337952 (0.0010) [2023-12-27 14:55:33,968][20381] Updated weights for policy 1, policy_version 1337962 (0.0010) [2023-12-27 14:55:34,494][20347] Updated weights for policy 0, policy_version 1339074 (0.0008) [2023-12-27 14:55:34,556][20347] Updated weights for policy 0, policy_version 1339084 (0.0007) [2023-12-27 14:55:34,615][20347] Updated weights for policy 0, policy_version 1339094 (0.0009) [2023-12-27 14:55:34,687][20381] Updated weights for policy 1, policy_version 1337972 (0.0011) [2023-12-27 14:55:34,743][20381] Updated weights for policy 1, policy_version 1337982 (0.0011) [2023-12-27 14:55:34,796][20381] Updated weights for policy 1, policy_version 1337992 (0.0011) [2023-12-27 14:55:35,244][20347] Updated weights for policy 0, policy_version 1339104 (0.0006) [2023-12-27 14:55:35,302][20347] Updated weights for policy 0, policy_version 1339114 (0.0010) [2023-12-27 14:55:35,359][20347] Updated weights for policy 0, policy_version 1339124 (0.0010) [2023-12-27 14:55:35,424][20381] Updated weights for policy 1, policy_version 1338002 (0.0010) [2023-12-27 14:55:35,468][20381] Updated weights for policy 1, policy_version 1338012 (0.0005) [2023-12-27 14:55:35,515][20381] Updated weights for policy 1, policy_version 1338022 (0.0005) [2023-12-27 14:55:35,563][20381] Updated weights for policy 1, policy_version 1338032 (0.0006) [2023-12-27 14:55:35,964][20347] Updated weights for policy 0, policy_version 1339134 (0.0007) [2023-12-27 14:55:36,019][20347] Updated weights for policy 0, policy_version 1339144 (0.0005) [2023-12-27 14:55:36,075][20347] Updated weights for policy 0, policy_version 1339154 (0.0005) [2023-12-27 14:55:36,173][20381] Updated weights for policy 1, policy_version 1338042 (0.0008) [2023-12-27 14:55:36,235][20381] Updated weights for policy 1, policy_version 1338052 (0.0008) [2023-12-27 14:55:36,296][20381] Updated weights for policy 1, policy_version 1338062 (0.0007) [2023-12-27 14:55:36,736][20347] Updated weights for policy 0, policy_version 1339164 (0.0009) [2023-12-27 14:55:36,791][20347] Updated weights for policy 0, policy_version 1339174 (0.0010) [2023-12-27 14:55:36,843][20347] Updated weights for policy 0, policy_version 1339184 (0.0010) [2023-12-27 14:55:36,988][20381] Updated weights for policy 1, policy_version 1338072 (0.0007) [2023-12-27 14:55:37,044][20381] Updated weights for policy 1, policy_version 1338082 (0.0007) [2023-12-27 14:55:37,102][20381] Updated weights for policy 1, policy_version 1338092 (0.0008) [2023-12-27 14:55:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 685481984. Throughput: 0: 10480.8, 1: 10282.5. Samples: 685447456. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:37,363][19170] Avg episode reward: [(0, '873.370'), (1, '990.240')] [2023-12-27 14:55:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001338096_342581248.pth... [2023-12-27 14:55:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001339192_342900736.pth... [2023-12-27 14:55:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001336880_342269952.pth [2023-12-27 14:55:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001337976_342589440.pth [2023-12-27 14:55:37,501][20347] Updated weights for policy 0, policy_version 1339194 (0.0007) [2023-12-27 14:55:37,557][20347] Updated weights for policy 0, policy_version 1339204 (0.0005) [2023-12-27 14:55:37,608][20347] Updated weights for policy 0, policy_version 1339214 (0.0006) [2023-12-27 14:55:37,665][20347] Updated weights for policy 0, policy_version 1339224 (0.0007) [2023-12-27 14:55:37,831][20381] Updated weights for policy 1, policy_version 1338102 (0.0006) [2023-12-27 14:55:37,883][20381] Updated weights for policy 1, policy_version 1338112 (0.0005) [2023-12-27 14:55:37,937][20381] Updated weights for policy 1, policy_version 1338122 (0.0005) [2023-12-27 14:55:38,298][20347] Updated weights for policy 0, policy_version 1339234 (0.0008) [2023-12-27 14:55:38,359][20347] Updated weights for policy 0, policy_version 1339244 (0.0008) [2023-12-27 14:55:38,412][20347] Updated weights for policy 0, policy_version 1339254 (0.0008) [2023-12-27 14:55:38,544][20381] Updated weights for policy 1, policy_version 1338132 (0.0006) [2023-12-27 14:55:38,593][20381] Updated weights for policy 1, policy_version 1338142 (0.0007) [2023-12-27 14:55:38,645][20381] Updated weights for policy 1, policy_version 1338152 (0.0008) [2023-12-27 14:55:39,158][20347] Updated weights for policy 0, policy_version 1339264 (0.0007) [2023-12-27 14:55:39,225][20347] Updated weights for policy 0, policy_version 1339274 (0.0008) [2023-12-27 14:55:39,282][20347] Updated weights for policy 0, policy_version 1339284 (0.0007) [2023-12-27 14:55:39,327][20381] Updated weights for policy 1, policy_version 1338162 (0.0009) [2023-12-27 14:55:39,394][20381] Updated weights for policy 1, policy_version 1338172 (0.0009) [2023-12-27 14:55:39,458][20381] Updated weights for policy 1, policy_version 1338182 (0.0008) [2023-12-27 14:55:39,521][20381] Updated weights for policy 1, policy_version 1338192 (0.0008) [2023-12-27 14:55:40,052][20347] Updated weights for policy 0, policy_version 1339294 (0.0006) [2023-12-27 14:55:40,116][20347] Updated weights for policy 0, policy_version 1339304 (0.0009) [2023-12-27 14:55:40,176][20347] Updated weights for policy 0, policy_version 1339314 (0.0006) [2023-12-27 14:55:40,282][20381] Updated weights for policy 1, policy_version 1338202 (0.0007) [2023-12-27 14:55:40,339][20381] Updated weights for policy 1, policy_version 1338212 (0.0007) [2023-12-27 14:55:40,402][20381] Updated weights for policy 1, policy_version 1338222 (0.0008) [2023-12-27 14:55:40,891][20347] Updated weights for policy 0, policy_version 1339324 (0.0007) [2023-12-27 14:55:40,946][20347] Updated weights for policy 0, policy_version 1339334 (0.0008) [2023-12-27 14:55:41,004][20347] Updated weights for policy 0, policy_version 1339344 (0.0009) [2023-12-27 14:55:41,082][20381] Updated weights for policy 1, policy_version 1338232 (0.0008) [2023-12-27 14:55:41,142][20381] Updated weights for policy 1, policy_version 1338242 (0.0009) [2023-12-27 14:55:41,214][20381] Updated weights for policy 1, policy_version 1338252 (0.0007) [2023-12-27 14:55:41,797][20347] Updated weights for policy 0, policy_version 1339354 (0.0008) [2023-12-27 14:55:41,855][20347] Updated weights for policy 0, policy_version 1339364 (0.0006) [2023-12-27 14:55:41,913][20347] Updated weights for policy 0, policy_version 1339374 (0.0006) [2023-12-27 14:55:41,970][20347] Updated weights for policy 0, policy_version 1339384 (0.0007) [2023-12-27 14:55:42,049][20381] Updated weights for policy 1, policy_version 1338262 (0.0009) [2023-12-27 14:55:42,108][20381] Updated weights for policy 1, policy_version 1338272 (0.0009) [2023-12-27 14:55:42,168][20381] Updated weights for policy 1, policy_version 1338282 (0.0008) [2023-12-27 14:55:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 685580288. Throughput: 0: 10324.9, 1: 10308.3. Samples: 685569972. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:42,363][19170] Avg episode reward: [(0, '863.290'), (1, '990.260')] [2023-12-27 14:55:42,580][20347] Updated weights for policy 0, policy_version 1339394 (0.0008) [2023-12-27 14:55:42,636][20347] Updated weights for policy 0, policy_version 1339404 (0.0007) [2023-12-27 14:55:42,691][20347] Updated weights for policy 0, policy_version 1339414 (0.0008) [2023-12-27 14:55:42,886][20381] Updated weights for policy 1, policy_version 1338292 (0.0007) [2023-12-27 14:55:42,935][20381] Updated weights for policy 1, policy_version 1338302 (0.0007) [2023-12-27 14:55:42,990][20381] Updated weights for policy 1, policy_version 1338312 (0.0007) [2023-12-27 14:55:43,402][20347] Updated weights for policy 0, policy_version 1339424 (0.0005) [2023-12-27 14:55:43,453][20347] Updated weights for policy 0, policy_version 1339434 (0.0005) [2023-12-27 14:55:43,516][20347] Updated weights for policy 0, policy_version 1339444 (0.0005) [2023-12-27 14:55:43,632][20381] Updated weights for policy 1, policy_version 1338322 (0.0008) [2023-12-27 14:55:43,690][20381] Updated weights for policy 1, policy_version 1338332 (0.0008) [2023-12-27 14:55:43,756][20381] Updated weights for policy 1, policy_version 1338342 (0.0010) [2023-12-27 14:55:43,816][20381] Updated weights for policy 1, policy_version 1338352 (0.0010) [2023-12-27 14:55:44,097][20347] Updated weights for policy 0, policy_version 1339454 (0.0007) [2023-12-27 14:55:44,156][20347] Updated weights for policy 0, policy_version 1339464 (0.0008) [2023-12-27 14:55:44,214][20347] Updated weights for policy 0, policy_version 1339474 (0.0007) [2023-12-27 14:55:44,424][20381] Updated weights for policy 1, policy_version 1338362 (0.0007) [2023-12-27 14:55:44,475][20381] Updated weights for policy 1, policy_version 1338372 (0.0008) [2023-12-27 14:55:44,534][20381] Updated weights for policy 1, policy_version 1338382 (0.0008) [2023-12-27 14:55:44,937][20347] Updated weights for policy 0, policy_version 1339484 (0.0007) [2023-12-27 14:55:44,993][20347] Updated weights for policy 0, policy_version 1339494 (0.0006) [2023-12-27 14:55:45,050][20347] Updated weights for policy 0, policy_version 1339504 (0.0006) [2023-12-27 14:55:45,220][20381] Updated weights for policy 1, policy_version 1338392 (0.0010) [2023-12-27 14:55:45,277][20381] Updated weights for policy 1, policy_version 1338402 (0.0011) [2023-12-27 14:55:45,338][20381] Updated weights for policy 1, policy_version 1338412 (0.0010) [2023-12-27 14:55:45,612][20347] Updated weights for policy 0, policy_version 1339514 (0.0006) [2023-12-27 14:55:45,642][20291] KL-divergence is very high: 107.1336 [2023-12-27 14:55:45,648][20291] KL-divergence is very high: 146.7520 [2023-12-27 14:55:45,659][20291] KL-divergence is very high: 154.6646 [2023-12-27 14:55:45,663][20347] Updated weights for policy 0, policy_version 1339524 (0.0005) [2023-12-27 14:55:45,664][20291] KL-divergence is very high: 148.8684 [2023-12-27 14:55:45,715][20347] Updated weights for policy 0, policy_version 1339534 (0.0005) [2023-12-27 14:55:45,764][20347] Updated weights for policy 0, policy_version 1339544 (0.0005) [2023-12-27 14:55:45,979][20381] Updated weights for policy 1, policy_version 1338422 (0.0010) [2023-12-27 14:55:46,034][20381] Updated weights for policy 1, policy_version 1338432 (0.0010) [2023-12-27 14:55:46,092][20381] Updated weights for policy 1, policy_version 1338442 (0.0010) [2023-12-27 14:55:46,256][20291] KL-divergence is very high: 149.5172 [2023-12-27 14:55:46,285][20291] KL-divergence is very high: 217.8231 [2023-12-27 14:55:46,301][20291] KL-divergence is very high: 156.4918 [2023-12-27 14:55:46,307][20347] Updated weights for policy 0, policy_version 1339554 (0.0006) [2023-12-27 14:55:46,356][20347] Updated weights for policy 0, policy_version 1339564 (0.0007) [2023-12-27 14:55:46,407][20347] Updated weights for policy 0, policy_version 1339574 (0.0009) [2023-12-27 14:55:46,779][20381] Updated weights for policy 1, policy_version 1338452 (0.0011) [2023-12-27 14:55:46,840][20381] Updated weights for policy 1, policy_version 1338462 (0.0011) [2023-12-27 14:55:46,904][20381] Updated weights for policy 1, policy_version 1338472 (0.0010) [2023-12-27 14:55:47,015][20347] Updated weights for policy 0, policy_version 1339584 (0.0010) [2023-12-27 14:55:47,071][20347] Updated weights for policy 0, policy_version 1339594 (0.0010) [2023-12-27 14:55:47,124][20347] Updated weights for policy 0, policy_version 1339604 (0.0005) [2023-12-27 14:55:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 685686784. Throughput: 0: 10434.5, 1: 10310.2. Samples: 685696984. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:47,363][19170] Avg episode reward: [(0, '834.100'), (1, '990.260')] [2023-12-27 14:55:47,450][20381] Updated weights for policy 1, policy_version 1338482 (0.0010) [2023-12-27 14:55:47,507][20381] Updated weights for policy 1, policy_version 1338492 (0.0010) [2023-12-27 14:55:47,560][20381] Updated weights for policy 1, policy_version 1338502 (0.0010) [2023-12-27 14:55:47,614][20381] Updated weights for policy 1, policy_version 1338512 (0.0010) [2023-12-27 14:55:47,742][20347] Updated weights for policy 0, policy_version 1339614 (0.0008) [2023-12-27 14:55:47,797][20347] Updated weights for policy 0, policy_version 1339624 (0.0010) [2023-12-27 14:55:47,850][20347] Updated weights for policy 0, policy_version 1339634 (0.0010) [2023-12-27 14:55:48,294][20381] Updated weights for policy 1, policy_version 1338522 (0.0007) [2023-12-27 14:55:48,353][20381] Updated weights for policy 1, policy_version 1338532 (0.0007) [2023-12-27 14:55:48,405][20381] Updated weights for policy 1, policy_version 1338542 (0.0007) [2023-12-27 14:55:48,507][20347] Updated weights for policy 0, policy_version 1339644 (0.0010) [2023-12-27 14:55:48,562][20347] Updated weights for policy 0, policy_version 1339654 (0.0010) [2023-12-27 14:55:48,607][20347] Updated weights for policy 0, policy_version 1339664 (0.0010) [2023-12-27 14:55:49,080][20381] Updated weights for policy 1, policy_version 1338552 (0.0008) [2023-12-27 14:55:49,133][20381] Updated weights for policy 1, policy_version 1338562 (0.0007) [2023-12-27 14:55:49,180][20381] Updated weights for policy 1, policy_version 1338572 (0.0008) [2023-12-27 14:55:49,301][20347] Updated weights for policy 0, policy_version 1339674 (0.0007) [2023-12-27 14:55:49,368][20347] Updated weights for policy 0, policy_version 1339684 (0.0008) [2023-12-27 14:55:49,426][20347] Updated weights for policy 0, policy_version 1339694 (0.0006) [2023-12-27 14:55:49,484][20347] Updated weights for policy 0, policy_version 1339704 (0.0008) [2023-12-27 14:55:49,877][20381] Updated weights for policy 1, policy_version 1338582 (0.0008) [2023-12-27 14:55:49,940][20381] Updated weights for policy 1, policy_version 1338592 (0.0008) [2023-12-27 14:55:49,993][20381] Updated weights for policy 1, policy_version 1338602 (0.0005) [2023-12-27 14:55:50,185][20347] Updated weights for policy 0, policy_version 1339714 (0.0008) [2023-12-27 14:55:50,242][20347] Updated weights for policy 0, policy_version 1339724 (0.0008) [2023-12-27 14:55:50,294][20347] Updated weights for policy 0, policy_version 1339734 (0.0008) [2023-12-27 14:55:50,664][20381] Updated weights for policy 1, policy_version 1338612 (0.0008) [2023-12-27 14:55:50,721][20381] Updated weights for policy 1, policy_version 1338622 (0.0008) [2023-12-27 14:55:50,781][20381] Updated weights for policy 1, policy_version 1338632 (0.0008) [2023-12-27 14:55:51,018][20347] Updated weights for policy 0, policy_version 1339744 (0.0007) [2023-12-27 14:55:51,081][20347] Updated weights for policy 0, policy_version 1339754 (0.0008) [2023-12-27 14:55:51,139][20347] Updated weights for policy 0, policy_version 1339764 (0.0008) [2023-12-27 14:55:51,512][20381] Updated weights for policy 1, policy_version 1338642 (0.0008) [2023-12-27 14:55:51,571][20381] Updated weights for policy 1, policy_version 1338652 (0.0006) [2023-12-27 14:55:51,632][20381] Updated weights for policy 1, policy_version 1338662 (0.0007) [2023-12-27 14:55:51,698][20381] Updated weights for policy 1, policy_version 1338672 (0.0006) [2023-12-27 14:55:51,938][20347] Updated weights for policy 0, policy_version 1339774 (0.0007) [2023-12-27 14:55:51,999][20347] Updated weights for policy 0, policy_version 1339784 (0.0006) [2023-12-27 14:55:52,057][20347] Updated weights for policy 0, policy_version 1339794 (0.0005) [2023-12-27 14:55:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 685785088. Throughput: 0: 10434.2, 1: 10280.3. Samples: 685758968. Policy #0 lag: (min: 31.0, avg: 41.0, max: 63.0) [2023-12-27 14:55:52,363][19170] Avg episode reward: [(0, '853.670'), (1, '1000.000')] [2023-12-27 14:55:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001339800_343056384.pth... [2023-12-27 14:55:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001338584_342745088.pth [2023-12-27 14:55:52,379][20381] Updated weights for policy 1, policy_version 1338682 (0.0008) [2023-12-27 14:55:52,434][20381] Updated weights for policy 1, policy_version 1338692 (0.0008) [2023-12-27 14:55:52,481][20381] Updated weights for policy 1, policy_version 1338702 (0.0008) [2023-12-27 14:55:52,488][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001338704_342736896.pth... [2023-12-27 14:55:52,491][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001337488_342425600.pth [2023-12-27 14:55:52,746][20347] Updated weights for policy 0, policy_version 1339804 (0.0006) [2023-12-27 14:55:52,793][20347] Updated weights for policy 0, policy_version 1339814 (0.0010) [2023-12-27 14:55:52,844][20347] Updated weights for policy 0, policy_version 1339824 (0.0007) [2023-12-27 14:55:53,118][20381] Updated weights for policy 1, policy_version 1338712 (0.0006) [2023-12-27 14:55:53,171][20381] Updated weights for policy 1, policy_version 1338722 (0.0005) [2023-12-27 14:55:53,231][20381] Updated weights for policy 1, policy_version 1338732 (0.0005) [2023-12-27 14:55:53,586][20347] Updated weights for policy 0, policy_version 1339834 (0.0007) [2023-12-27 14:55:53,637][20347] Updated weights for policy 0, policy_version 1339844 (0.0008) [2023-12-27 14:55:53,686][20347] Updated weights for policy 0, policy_version 1339854 (0.0007) [2023-12-27 14:55:53,730][20347] Updated weights for policy 0, policy_version 1339864 (0.0008) [2023-12-27 14:55:53,836][20381] Updated weights for policy 1, policy_version 1338742 (0.0007) [2023-12-27 14:55:53,891][20381] Updated weights for policy 1, policy_version 1338752 (0.0007) [2023-12-27 14:55:53,947][20381] Updated weights for policy 1, policy_version 1338762 (0.0008) [2023-12-27 14:55:54,388][20347] Updated weights for policy 0, policy_version 1339874 (0.0005) [2023-12-27 14:55:54,440][20347] Updated weights for policy 0, policy_version 1339884 (0.0005) [2023-12-27 14:55:54,494][20347] Updated weights for policy 0, policy_version 1339894 (0.0007) [2023-12-27 14:55:54,650][20381] Updated weights for policy 1, policy_version 1338772 (0.0007) [2023-12-27 14:55:54,703][20381] Updated weights for policy 1, policy_version 1338782 (0.0008) [2023-12-27 14:55:54,746][20381] Updated weights for policy 1, policy_version 1338792 (0.0008) [2023-12-27 14:55:55,134][20347] Updated weights for policy 0, policy_version 1339904 (0.0007) [2023-12-27 14:55:55,195][20347] Updated weights for policy 0, policy_version 1339914 (0.0008) [2023-12-27 14:55:55,250][20347] Updated weights for policy 0, policy_version 1339924 (0.0007) [2023-12-27 14:55:55,487][20381] Updated weights for policy 1, policy_version 1338802 (0.0008) [2023-12-27 14:55:55,542][20381] Updated weights for policy 1, policy_version 1338812 (0.0007) [2023-12-27 14:55:55,601][20381] Updated weights for policy 1, policy_version 1338822 (0.0007) [2023-12-27 14:55:55,656][20381] Updated weights for policy 1, policy_version 1338832 (0.0008) [2023-12-27 14:55:55,846][20347] Updated weights for policy 0, policy_version 1339934 (0.0007) [2023-12-27 14:55:55,904][20347] Updated weights for policy 0, policy_version 1339944 (0.0006) [2023-12-27 14:55:55,957][20347] Updated weights for policy 0, policy_version 1339954 (0.0006) [2023-12-27 14:55:56,401][20381] Updated weights for policy 1, policy_version 1338842 (0.0008) [2023-12-27 14:55:56,460][20381] Updated weights for policy 1, policy_version 1338852 (0.0007) [2023-12-27 14:55:56,518][20381] Updated weights for policy 1, policy_version 1338862 (0.0005) [2023-12-27 14:55:56,583][20347] Updated weights for policy 0, policy_version 1339964 (0.0006) [2023-12-27 14:55:56,628][20347] Updated weights for policy 0, policy_version 1339974 (0.0007) [2023-12-27 14:55:56,673][20347] Updated weights for policy 0, policy_version 1339984 (0.0007) [2023-12-27 14:55:57,069][20381] Updated weights for policy 1, policy_version 1338872 (0.0007) [2023-12-27 14:55:57,126][20381] Updated weights for policy 1, policy_version 1338882 (0.0008) [2023-12-27 14:55:57,182][20381] Updated weights for policy 1, policy_version 1338892 (0.0008) [2023-12-27 14:55:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 685891584. Throughput: 0: 10393.5, 1: 10259.9. Samples: 685880420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:55:57,363][19170] Avg episode reward: [(0, '882.600'), (1, '971.130')] [2023-12-27 14:55:57,384][20347] Updated weights for policy 0, policy_version 1339994 (0.0008) [2023-12-27 14:55:57,433][20347] Updated weights for policy 0, policy_version 1340004 (0.0007) [2023-12-27 14:55:57,488][20347] Updated weights for policy 0, policy_version 1340014 (0.0008) [2023-12-27 14:55:57,539][20347] Updated weights for policy 0, policy_version 1340024 (0.0007) [2023-12-27 14:55:57,881][20381] Updated weights for policy 1, policy_version 1338902 (0.0007) [2023-12-27 14:55:57,936][20381] Updated weights for policy 1, policy_version 1338912 (0.0007) [2023-12-27 14:55:57,989][20381] Updated weights for policy 1, policy_version 1338922 (0.0008) [2023-12-27 14:55:58,215][20347] Updated weights for policy 0, policy_version 1340034 (0.0007) [2023-12-27 14:55:58,276][20347] Updated weights for policy 0, policy_version 1340044 (0.0008) [2023-12-27 14:55:58,332][20347] Updated weights for policy 0, policy_version 1340054 (0.0008) [2023-12-27 14:55:58,785][20381] Updated weights for policy 1, policy_version 1338932 (0.0010) [2023-12-27 14:55:58,851][20381] Updated weights for policy 1, policy_version 1338942 (0.0009) [2023-12-27 14:55:58,910][20381] Updated weights for policy 1, policy_version 1338952 (0.0008) [2023-12-27 14:55:59,080][20347] Updated weights for policy 0, policy_version 1340064 (0.0008) [2023-12-27 14:55:59,136][20347] Updated weights for policy 0, policy_version 1340074 (0.0006) [2023-12-27 14:55:59,191][20347] Updated weights for policy 0, policy_version 1340084 (0.0007) [2023-12-27 14:55:59,685][20381] Updated weights for policy 1, policy_version 1338962 (0.0008) [2023-12-27 14:55:59,739][20381] Updated weights for policy 1, policy_version 1338972 (0.0010) [2023-12-27 14:55:59,794][20381] Updated weights for policy 1, policy_version 1338982 (0.0007) [2023-12-27 14:55:59,851][20381] Updated weights for policy 1, policy_version 1338992 (0.0009) [2023-12-27 14:55:59,969][20347] Updated weights for policy 0, policy_version 1340094 (0.0008) [2023-12-27 14:56:00,027][20347] Updated weights for policy 0, policy_version 1340104 (0.0008) [2023-12-27 14:56:00,085][20347] Updated weights for policy 0, policy_version 1340114 (0.0008) [2023-12-27 14:56:00,570][20381] Updated weights for policy 1, policy_version 1339002 (0.0007) [2023-12-27 14:56:00,629][20381] Updated weights for policy 1, policy_version 1339012 (0.0009) [2023-12-27 14:56:00,681][20381] Updated weights for policy 1, policy_version 1339022 (0.0007) [2023-12-27 14:56:00,794][20347] Updated weights for policy 0, policy_version 1340124 (0.0008) [2023-12-27 14:56:00,850][20347] Updated weights for policy 0, policy_version 1340134 (0.0007) [2023-12-27 14:56:00,909][20347] Updated weights for policy 0, policy_version 1340144 (0.0008) [2023-12-27 14:56:01,314][20381] Updated weights for policy 1, policy_version 1339032 (0.0007) [2023-12-27 14:56:01,374][20381] Updated weights for policy 1, policy_version 1339042 (0.0008) [2023-12-27 14:56:01,425][20381] Updated weights for policy 1, policy_version 1339052 (0.0006) [2023-12-27 14:56:01,636][20347] Updated weights for policy 0, policy_version 1340154 (0.0007) [2023-12-27 14:56:01,703][20347] Updated weights for policy 0, policy_version 1340164 (0.0008) [2023-12-27 14:56:01,766][20347] Updated weights for policy 0, policy_version 1340174 (0.0008) [2023-12-27 14:56:01,817][20347] Updated weights for policy 0, policy_version 1340184 (0.0007) [2023-12-27 14:56:02,115][20381] Updated weights for policy 1, policy_version 1339062 (0.0007) [2023-12-27 14:56:02,175][20381] Updated weights for policy 1, policy_version 1339072 (0.0006) [2023-12-27 14:56:02,239][20381] Updated weights for policy 1, policy_version 1339082 (0.0006) [2023-12-27 14:56:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 685989888. Throughput: 0: 10316.9, 1: 10216.6. Samples: 685999228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:02,363][19170] Avg episode reward: [(0, '853.200'), (1, '971.130')] [2023-12-27 14:56:02,464][20347] Updated weights for policy 0, policy_version 1340194 (0.0006) [2023-12-27 14:56:02,527][20347] Updated weights for policy 0, policy_version 1340204 (0.0006) [2023-12-27 14:56:02,587][20347] Updated weights for policy 0, policy_version 1340214 (0.0006) [2023-12-27 14:56:02,856][20381] Updated weights for policy 1, policy_version 1339092 (0.0006) [2023-12-27 14:56:02,907][20381] Updated weights for policy 1, policy_version 1339102 (0.0006) [2023-12-27 14:56:02,965][20381] Updated weights for policy 1, policy_version 1339112 (0.0006) [2023-12-27 14:56:03,238][20347] Updated weights for policy 0, policy_version 1340224 (0.0006) [2023-12-27 14:56:03,290][20347] Updated weights for policy 0, policy_version 1340234 (0.0006) [2023-12-27 14:56:03,335][20347] Updated weights for policy 0, policy_version 1340244 (0.0008) [2023-12-27 14:56:03,549][20381] Updated weights for policy 1, policy_version 1339122 (0.0006) [2023-12-27 14:56:03,612][20381] Updated weights for policy 1, policy_version 1339132 (0.0005) [2023-12-27 14:56:03,666][20381] Updated weights for policy 1, policy_version 1339142 (0.0006) [2023-12-27 14:56:03,710][20381] Updated weights for policy 1, policy_version 1339152 (0.0005) [2023-12-27 14:56:03,990][20347] Updated weights for policy 0, policy_version 1340254 (0.0007) [2023-12-27 14:56:04,049][20347] Updated weights for policy 0, policy_version 1340264 (0.0006) [2023-12-27 14:56:04,113][20347] Updated weights for policy 0, policy_version 1340274 (0.0006) [2023-12-27 14:56:04,306][20381] Updated weights for policy 1, policy_version 1339162 (0.0007) [2023-12-27 14:56:04,359][20381] Updated weights for policy 1, policy_version 1339172 (0.0007) [2023-12-27 14:56:04,420][20381] Updated weights for policy 1, policy_version 1339182 (0.0006) [2023-12-27 14:56:04,766][20347] Updated weights for policy 0, policy_version 1340284 (0.0005) [2023-12-27 14:56:04,821][20347] Updated weights for policy 0, policy_version 1340294 (0.0005) [2023-12-27 14:56:04,881][20347] Updated weights for policy 0, policy_version 1340304 (0.0007) [2023-12-27 14:56:05,049][20381] Updated weights for policy 1, policy_version 1339192 (0.0007) [2023-12-27 14:56:05,101][20381] Updated weights for policy 1, policy_version 1339202 (0.0007) [2023-12-27 14:56:05,152][20381] Updated weights for policy 1, policy_version 1339212 (0.0007) [2023-12-27 14:56:05,561][20347] Updated weights for policy 0, policy_version 1340314 (0.0010) [2023-12-27 14:56:05,614][20347] Updated weights for policy 0, policy_version 1340324 (0.0010) [2023-12-27 14:56:05,664][20347] Updated weights for policy 0, policy_version 1340334 (0.0010) [2023-12-27 14:56:05,719][20347] Updated weights for policy 0, policy_version 1340344 (0.0010) [2023-12-27 14:56:05,853][20381] Updated weights for policy 1, policy_version 1339222 (0.0007) [2023-12-27 14:56:05,900][20381] Updated weights for policy 1, policy_version 1339232 (0.0007) [2023-12-27 14:56:05,951][20381] Updated weights for policy 1, policy_version 1339242 (0.0007) [2023-12-27 14:56:06,407][20347] Updated weights for policy 0, policy_version 1340354 (0.0010) [2023-12-27 14:56:06,466][20347] Updated weights for policy 0, policy_version 1340364 (0.0011) [2023-12-27 14:56:06,527][20347] Updated weights for policy 0, policy_version 1340374 (0.0010) [2023-12-27 14:56:06,642][20381] Updated weights for policy 1, policy_version 1339252 (0.0007) [2023-12-27 14:56:06,701][20381] Updated weights for policy 1, policy_version 1339262 (0.0008) [2023-12-27 14:56:06,761][20381] Updated weights for policy 1, policy_version 1339272 (0.0008) [2023-12-27 14:56:07,164][20347] Updated weights for policy 0, policy_version 1340384 (0.0010) [2023-12-27 14:56:07,222][20347] Updated weights for policy 0, policy_version 1340394 (0.0010) [2023-12-27 14:56:07,278][20347] Updated weights for policy 0, policy_version 1340404 (0.0008) [2023-12-27 14:56:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 686096384. Throughput: 0: 10300.3, 1: 10281.1. Samples: 686064208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:07,363][19170] Avg episode reward: [(0, '774.730'), (1, '1000.000')] [2023-12-27 14:56:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001340408_343212032.pth... [2023-12-27 14:56:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001339280_342884352.pth... [2023-12-27 14:56:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001339192_342900736.pth [2023-12-27 14:56:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001338096_342581248.pth [2023-12-27 14:56:07,495][20381] Updated weights for policy 1, policy_version 1339282 (0.0008) [2023-12-27 14:56:07,554][20381] Updated weights for policy 1, policy_version 1339292 (0.0008) [2023-12-27 14:56:07,611][20381] Updated weights for policy 1, policy_version 1339302 (0.0007) [2023-12-27 14:56:07,663][20381] Updated weights for policy 1, policy_version 1339312 (0.0009) [2023-12-27 14:56:07,874][20347] Updated weights for policy 0, policy_version 1340414 (0.0010) [2023-12-27 14:56:07,925][20347] Updated weights for policy 0, policy_version 1340424 (0.0010) [2023-12-27 14:56:07,981][20347] Updated weights for policy 0, policy_version 1340434 (0.0010) [2023-12-27 14:56:08,392][20381] Updated weights for policy 1, policy_version 1339322 (0.0008) [2023-12-27 14:56:08,451][20381] Updated weights for policy 1, policy_version 1339332 (0.0007) [2023-12-27 14:56:08,513][20381] Updated weights for policy 1, policy_version 1339342 (0.0007) [2023-12-27 14:56:08,689][20347] Updated weights for policy 0, policy_version 1340444 (0.0011) [2023-12-27 14:56:08,753][20347] Updated weights for policy 0, policy_version 1340454 (0.0010) [2023-12-27 14:56:08,810][20347] Updated weights for policy 0, policy_version 1340464 (0.0010) [2023-12-27 14:56:09,170][20381] Updated weights for policy 1, policy_version 1339352 (0.0008) [2023-12-27 14:56:09,239][20381] Updated weights for policy 1, policy_version 1339362 (0.0008) [2023-12-27 14:56:09,305][20381] Updated weights for policy 1, policy_version 1339372 (0.0007) [2023-12-27 14:56:09,500][20347] Updated weights for policy 0, policy_version 1340474 (0.0010) [2023-12-27 14:56:09,550][20347] Updated weights for policy 0, policy_version 1340484 (0.0008) [2023-12-27 14:56:09,619][20347] Updated weights for policy 0, policy_version 1340494 (0.0007) [2023-12-27 14:56:09,682][20347] Updated weights for policy 0, policy_version 1340504 (0.0008) [2023-12-27 14:56:10,002][20381] Updated weights for policy 1, policy_version 1339382 (0.0007) [2023-12-27 14:56:10,068][20381] Updated weights for policy 1, policy_version 1339392 (0.0007) [2023-12-27 14:56:10,120][20381] Updated weights for policy 1, policy_version 1339402 (0.0007) [2023-12-27 14:56:10,396][20347] Updated weights for policy 0, policy_version 1340514 (0.0010) [2023-12-27 14:56:10,454][20347] Updated weights for policy 0, policy_version 1340524 (0.0010) [2023-12-27 14:56:10,506][20347] Updated weights for policy 0, policy_version 1340534 (0.0010) [2023-12-27 14:56:10,817][20381] Updated weights for policy 1, policy_version 1339412 (0.0008) [2023-12-27 14:56:10,884][20381] Updated weights for policy 1, policy_version 1339422 (0.0008) [2023-12-27 14:56:10,939][20381] Updated weights for policy 1, policy_version 1339432 (0.0007) [2023-12-27 14:56:11,196][20347] Updated weights for policy 0, policy_version 1340544 (0.0011) [2023-12-27 14:56:11,255][20347] Updated weights for policy 0, policy_version 1340554 (0.0010) [2023-12-27 14:56:11,314][20347] Updated weights for policy 0, policy_version 1340564 (0.0007) [2023-12-27 14:56:11,729][20381] Updated weights for policy 1, policy_version 1339442 (0.0007) [2023-12-27 14:56:11,796][20381] Updated weights for policy 1, policy_version 1339452 (0.0008) [2023-12-27 14:56:11,859][20381] Updated weights for policy 1, policy_version 1339462 (0.0008) [2023-12-27 14:56:11,923][20381] Updated weights for policy 1, policy_version 1339472 (0.0008) [2023-12-27 14:56:12,020][20347] Updated weights for policy 0, policy_version 1340574 (0.0006) [2023-12-27 14:56:12,086][20347] Updated weights for policy 0, policy_version 1340584 (0.0006) [2023-12-27 14:56:12,154][20347] Updated weights for policy 0, policy_version 1340594 (0.0006) [2023-12-27 14:56:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 686194688. Throughput: 0: 10335.8, 1: 10215.4. Samples: 686184808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:12,363][19170] Avg episode reward: [(0, '755.350'), (1, '990.280')] [2023-12-27 14:56:12,644][20381] Updated weights for policy 1, policy_version 1339482 (0.0006) [2023-12-27 14:56:12,706][20381] Updated weights for policy 1, policy_version 1339492 (0.0007) [2023-12-27 14:56:12,758][20347] Updated weights for policy 0, policy_version 1340604 (0.0007) [2023-12-27 14:56:12,766][20381] Updated weights for policy 1, policy_version 1339502 (0.0006) [2023-12-27 14:56:12,822][20347] Updated weights for policy 0, policy_version 1340614 (0.0008) [2023-12-27 14:56:12,886][20347] Updated weights for policy 0, policy_version 1340624 (0.0008) [2023-12-27 14:56:13,387][20381] Updated weights for policy 1, policy_version 1339512 (0.0007) [2023-12-27 14:56:13,446][20381] Updated weights for policy 1, policy_version 1339522 (0.0006) [2023-12-27 14:56:13,494][20381] Updated weights for policy 1, policy_version 1339532 (0.0007) [2023-12-27 14:56:13,586][20347] Updated weights for policy 0, policy_version 1340634 (0.0008) [2023-12-27 14:56:13,640][20347] Updated weights for policy 0, policy_version 1340644 (0.0008) [2023-12-27 14:56:13,696][20347] Updated weights for policy 0, policy_version 1340654 (0.0008) [2023-12-27 14:56:13,753][20347] Updated weights for policy 0, policy_version 1340664 (0.0007) [2023-12-27 14:56:14,172][20381] Updated weights for policy 1, policy_version 1339542 (0.0008) [2023-12-27 14:56:14,230][20381] Updated weights for policy 1, policy_version 1339552 (0.0007) [2023-12-27 14:56:14,285][20381] Updated weights for policy 1, policy_version 1339562 (0.0007) [2023-12-27 14:56:14,407][20347] Updated weights for policy 0, policy_version 1340674 (0.0007) [2023-12-27 14:56:14,466][20347] Updated weights for policy 0, policy_version 1340684 (0.0008) [2023-12-27 14:56:14,522][20347] Updated weights for policy 0, policy_version 1340694 (0.0008) [2023-12-27 14:56:14,989][20381] Updated weights for policy 1, policy_version 1339572 (0.0008) [2023-12-27 14:56:15,047][20381] Updated weights for policy 1, policy_version 1339582 (0.0008) [2023-12-27 14:56:15,106][20381] Updated weights for policy 1, policy_version 1339592 (0.0008) [2023-12-27 14:56:15,162][20347] Updated weights for policy 0, policy_version 1340704 (0.0006) [2023-12-27 14:56:15,228][20347] Updated weights for policy 0, policy_version 1340714 (0.0006) [2023-12-27 14:56:15,293][20347] Updated weights for policy 0, policy_version 1340724 (0.0006) [2023-12-27 14:56:15,842][20381] Updated weights for policy 1, policy_version 1339602 (0.0007) [2023-12-27 14:56:15,890][20381] Updated weights for policy 1, policy_version 1339612 (0.0008) [2023-12-27 14:56:15,910][20347] Updated weights for policy 0, policy_version 1340734 (0.0006) [2023-12-27 14:56:15,946][20381] Updated weights for policy 1, policy_version 1339622 (0.0005) [2023-12-27 14:56:15,971][20347] Updated weights for policy 0, policy_version 1340744 (0.0006) [2023-12-27 14:56:15,996][20381] Updated weights for policy 1, policy_version 1339632 (0.0005) [2023-12-27 14:56:16,039][20347] Updated weights for policy 0, policy_version 1340754 (0.0008) [2023-12-27 14:56:16,583][20381] Updated weights for policy 1, policy_version 1339642 (0.0005) [2023-12-27 14:56:16,625][20347] Updated weights for policy 0, policy_version 1340764 (0.0007) [2023-12-27 14:56:16,636][20381] Updated weights for policy 1, policy_version 1339652 (0.0005) [2023-12-27 14:56:16,670][20347] Updated weights for policy 0, policy_version 1340774 (0.0008) [2023-12-27 14:56:16,682][20381] Updated weights for policy 1, policy_version 1339662 (0.0005) [2023-12-27 14:56:16,716][20347] Updated weights for policy 0, policy_version 1340784 (0.0005) [2023-12-27 14:56:17,246][20381] Updated weights for policy 1, policy_version 1339672 (0.0007) [2023-12-27 14:56:17,298][20381] Updated weights for policy 1, policy_version 1339682 (0.0007) [2023-12-27 14:56:17,348][20381] Updated weights for policy 1, policy_version 1339692 (0.0008) [2023-12-27 14:56:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 686292992. Throughput: 0: 10351.8, 1: 10216.8. Samples: 686310060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:17,363][19170] Avg episode reward: [(0, '843.170'), (1, '990.280')] [2023-12-27 14:56:17,387][20347] Updated weights for policy 0, policy_version 1340794 (0.0006) [2023-12-27 14:56:17,435][20347] Updated weights for policy 0, policy_version 1340804 (0.0009) [2023-12-27 14:56:17,484][20347] Updated weights for policy 0, policy_version 1340814 (0.0007) [2023-12-27 14:56:17,540][20347] Updated weights for policy 0, policy_version 1340824 (0.0008) [2023-12-27 14:56:17,994][20381] Updated weights for policy 1, policy_version 1339702 (0.0009) [2023-12-27 14:56:18,043][20381] Updated weights for policy 1, policy_version 1339712 (0.0011) [2023-12-27 14:56:18,092][20381] Updated weights for policy 1, policy_version 1339722 (0.0010) [2023-12-27 14:56:18,193][20347] Updated weights for policy 0, policy_version 1340834 (0.0005) [2023-12-27 14:56:18,247][20347] Updated weights for policy 0, policy_version 1340844 (0.0005) [2023-12-27 14:56:18,309][20347] Updated weights for policy 0, policy_version 1340854 (0.0006) [2023-12-27 14:56:18,773][20381] Updated weights for policy 1, policy_version 1339732 (0.0009) [2023-12-27 14:56:18,831][20381] Updated weights for policy 1, policy_version 1339742 (0.0006) [2023-12-27 14:56:18,898][20381] Updated weights for policy 1, policy_version 1339752 (0.0007) [2023-12-27 14:56:18,955][20347] Updated weights for policy 0, policy_version 1340864 (0.0007) [2023-12-27 14:56:19,012][20347] Updated weights for policy 0, policy_version 1340874 (0.0005) [2023-12-27 14:56:19,074][20347] Updated weights for policy 0, policy_version 1340884 (0.0005) [2023-12-27 14:56:19,606][20381] Updated weights for policy 1, policy_version 1339762 (0.0007) [2023-12-27 14:56:19,668][20381] Updated weights for policy 1, policy_version 1339772 (0.0008) [2023-12-27 14:56:19,727][20347] Updated weights for policy 0, policy_version 1340894 (0.0006) [2023-12-27 14:56:19,729][20381] Updated weights for policy 1, policy_version 1339782 (0.0007) [2023-12-27 14:56:19,786][20347] Updated weights for policy 0, policy_version 1340904 (0.0005) [2023-12-27 14:56:19,788][20381] Updated weights for policy 1, policy_version 1339792 (0.0006) [2023-12-27 14:56:19,851][20347] Updated weights for policy 0, policy_version 1340914 (0.0007) [2023-12-27 14:56:20,422][20381] Updated weights for policy 1, policy_version 1339802 (0.0006) [2023-12-27 14:56:20,471][20381] Updated weights for policy 1, policy_version 1339812 (0.0008) [2023-12-27 14:56:20,518][20381] Updated weights for policy 1, policy_version 1339822 (0.0005) [2023-12-27 14:56:20,522][20347] Updated weights for policy 0, policy_version 1340924 (0.0006) [2023-12-27 14:56:20,583][20347] Updated weights for policy 0, policy_version 1340934 (0.0006) [2023-12-27 14:56:20,652][20347] Updated weights for policy 0, policy_version 1340944 (0.0008) [2023-12-27 14:56:21,101][20381] Updated weights for policy 1, policy_version 1339832 (0.0007) [2023-12-27 14:56:21,160][20381] Updated weights for policy 1, policy_version 1339842 (0.0008) [2023-12-27 14:56:21,213][20381] Updated weights for policy 1, policy_version 1339852 (0.0008) [2023-12-27 14:56:21,313][20347] Updated weights for policy 0, policy_version 1340954 (0.0006) [2023-12-27 14:56:21,374][20347] Updated weights for policy 0, policy_version 1340964 (0.0008) [2023-12-27 14:56:21,438][20347] Updated weights for policy 0, policy_version 1340974 (0.0008) [2023-12-27 14:56:21,498][20347] Updated weights for policy 0, policy_version 1340984 (0.0008) [2023-12-27 14:56:21,966][20381] Updated weights for policy 1, policy_version 1339862 (0.0007) [2023-12-27 14:56:22,028][20381] Updated weights for policy 1, policy_version 1339872 (0.0006) [2023-12-27 14:56:22,091][20381] Updated weights for policy 1, policy_version 1339882 (0.0007) [2023-12-27 14:56:22,279][20347] Updated weights for policy 0, policy_version 1340994 (0.0009) [2023-12-27 14:56:22,346][20347] Updated weights for policy 0, policy_version 1341004 (0.0007) [2023-12-27 14:56:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 686399488. Throughput: 0: 10331.7, 1: 10224.2. Samples: 686372472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:22,365][19170] Avg episode reward: [(0, '862.820'), (1, '1000.000')] [2023-12-27 14:56:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001339888_343040000.pth... [2023-12-27 14:56:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001338704_342736896.pth [2023-12-27 14:56:22,409][20347] Updated weights for policy 0, policy_version 1341014 (0.0007) [2023-12-27 14:56:22,417][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001341016_343367680.pth... [2023-12-27 14:56:22,421][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001339800_343056384.pth [2023-12-27 14:56:22,773][20381] Updated weights for policy 1, policy_version 1339892 (0.0008) [2023-12-27 14:56:22,830][20381] Updated weights for policy 1, policy_version 1339902 (0.0008) [2023-12-27 14:56:22,889][20381] Updated weights for policy 1, policy_version 1339912 (0.0008) [2023-12-27 14:56:23,061][20347] Updated weights for policy 0, policy_version 1341024 (0.0008) [2023-12-27 14:56:23,113][20347] Updated weights for policy 0, policy_version 1341034 (0.0007) [2023-12-27 14:56:23,163][20347] Updated weights for policy 0, policy_version 1341044 (0.0008) [2023-12-27 14:56:23,599][20381] Updated weights for policy 1, policy_version 1339922 (0.0010) [2023-12-27 14:56:23,661][20381] Updated weights for policy 1, policy_version 1339932 (0.0008) [2023-12-27 14:56:23,727][20381] Updated weights for policy 1, policy_version 1339942 (0.0008) [2023-12-27 14:56:23,787][20381] Updated weights for policy 1, policy_version 1339952 (0.0006) [2023-12-27 14:56:23,890][20347] Updated weights for policy 0, policy_version 1341054 (0.0009) [2023-12-27 14:56:23,937][20347] Updated weights for policy 0, policy_version 1341064 (0.0006) [2023-12-27 14:56:23,993][20347] Updated weights for policy 0, policy_version 1341074 (0.0005) [2023-12-27 14:56:24,344][20381] Updated weights for policy 1, policy_version 1339962 (0.0006) [2023-12-27 14:56:24,390][20381] Updated weights for policy 1, policy_version 1339972 (0.0005) [2023-12-27 14:56:24,433][20381] Updated weights for policy 1, policy_version 1339982 (0.0005) [2023-12-27 14:56:24,688][20347] Updated weights for policy 0, policy_version 1341084 (0.0006) [2023-12-27 14:56:24,738][20347] Updated weights for policy 0, policy_version 1341094 (0.0007) [2023-12-27 14:56:24,791][20347] Updated weights for policy 0, policy_version 1341104 (0.0007) [2023-12-27 14:56:25,085][20381] Updated weights for policy 1, policy_version 1339992 (0.0009) [2023-12-27 14:56:25,149][20381] Updated weights for policy 1, policy_version 1340002 (0.0010) [2023-12-27 14:56:25,214][20381] Updated weights for policy 1, policy_version 1340012 (0.0008) [2023-12-27 14:56:25,452][20347] Updated weights for policy 0, policy_version 1341114 (0.0007) [2023-12-27 14:56:25,505][20347] Updated weights for policy 0, policy_version 1341124 (0.0007) [2023-12-27 14:56:25,561][20347] Updated weights for policy 0, policy_version 1341134 (0.0007) [2023-12-27 14:56:25,615][20347] Updated weights for policy 0, policy_version 1341144 (0.0007) [2023-12-27 14:56:25,828][20381] Updated weights for policy 1, policy_version 1340022 (0.0007) [2023-12-27 14:56:25,886][20381] Updated weights for policy 1, policy_version 1340032 (0.0010) [2023-12-27 14:56:25,941][20381] Updated weights for policy 1, policy_version 1340042 (0.0010) [2023-12-27 14:56:26,311][20347] Updated weights for policy 0, policy_version 1341154 (0.0007) [2023-12-27 14:56:26,361][20347] Updated weights for policy 0, policy_version 1341164 (0.0006) [2023-12-27 14:56:26,416][20347] Updated weights for policy 0, policy_version 1341174 (0.0007) [2023-12-27 14:56:26,566][20381] Updated weights for policy 1, policy_version 1340052 (0.0006) [2023-12-27 14:56:26,617][20381] Updated weights for policy 1, policy_version 1340062 (0.0007) [2023-12-27 14:56:26,667][20381] Updated weights for policy 1, policy_version 1340072 (0.0007) [2023-12-27 14:56:27,066][20347] Updated weights for policy 0, policy_version 1341184 (0.0008) [2023-12-27 14:56:27,126][20347] Updated weights for policy 0, policy_version 1341194 (0.0010) [2023-12-27 14:56:27,191][20347] Updated weights for policy 0, policy_version 1341204 (0.0008) [2023-12-27 14:56:27,215][20381] Updated weights for policy 1, policy_version 1340082 (0.0007) [2023-12-27 14:56:27,272][20381] Updated weights for policy 1, policy_version 1340092 (0.0008) [2023-12-27 14:56:27,326][20381] Updated weights for policy 1, policy_version 1340102 (0.0009) [2023-12-27 14:56:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 686505984. Throughput: 0: 10314.3, 1: 10274.0. Samples: 686496444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:27,363][19170] Avg episode reward: [(0, '833.700'), (1, '990.250')] [2023-12-27 14:56:27,384][20381] Updated weights for policy 1, policy_version 1340112 (0.0009) [2023-12-27 14:56:27,801][20347] Updated weights for policy 0, policy_version 1341214 (0.0008) [2023-12-27 14:56:27,858][20347] Updated weights for policy 0, policy_version 1341224 (0.0010) [2023-12-27 14:56:27,918][20347] Updated weights for policy 0, policy_version 1341234 (0.0010) [2023-12-27 14:56:28,020][20381] Updated weights for policy 1, policy_version 1340122 (0.0010) [2023-12-27 14:56:28,066][20381] Updated weights for policy 1, policy_version 1340132 (0.0007) [2023-12-27 14:56:28,111][20381] Updated weights for policy 1, policy_version 1340142 (0.0007) [2023-12-27 14:56:28,118][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000000 [2023-12-27 14:56:28,604][20347] Updated weights for policy 0, policy_version 1341244 (0.0009) [2023-12-27 14:56:28,664][20347] Updated weights for policy 0, policy_version 1341254 (0.0008) [2023-12-27 14:56:28,722][20347] Updated weights for policy 0, policy_version 1341264 (0.0007) [2023-12-27 14:56:28,759][20381] Updated weights for policy 1, policy_version 1340152 (0.0007) [2023-12-27 14:56:28,819][20381] Updated weights for policy 1, policy_version 1340162 (0.0008) [2023-12-27 14:56:28,877][20381] Updated weights for policy 1, policy_version 1340172 (0.0008) [2023-12-27 14:56:29,437][20347] Updated weights for policy 0, policy_version 1341274 (0.0006) [2023-12-27 14:56:29,483][20347] Updated weights for policy 0, policy_version 1341284 (0.0008) [2023-12-27 14:56:29,529][20381] Updated weights for policy 1, policy_version 1340182 (0.0007) [2023-12-27 14:56:29,546][20347] Updated weights for policy 0, policy_version 1341294 (0.0007) [2023-12-27 14:56:29,558][20291] KL-divergence is very high: 144.0552 [2023-12-27 14:56:29,563][20291] KL-divergence is very high: 191.4914 [2023-12-27 14:56:29,574][20291] KL-divergence is very high: 146.6237 [2023-12-27 14:56:29,585][20381] Updated weights for policy 1, policy_version 1340192 (0.0005) [2023-12-27 14:56:29,598][20347] Updated weights for policy 0, policy_version 1341304 (0.0007) [2023-12-27 14:56:29,630][20381] Updated weights for policy 1, policy_version 1340202 (0.0008) [2023-12-27 14:56:30,314][20347] Updated weights for policy 0, policy_version 1341314 (0.0008) [2023-12-27 14:56:30,345][20381] Updated weights for policy 1, policy_version 1340212 (0.0007) [2023-12-27 14:56:30,376][20347] Updated weights for policy 0, policy_version 1341324 (0.0007) [2023-12-27 14:56:30,406][20381] Updated weights for policy 1, policy_version 1340222 (0.0006) [2023-12-27 14:56:30,436][20347] Updated weights for policy 0, policy_version 1341334 (0.0007) [2023-12-27 14:56:30,464][20381] Updated weights for policy 1, policy_version 1340232 (0.0007) [2023-12-27 14:56:31,070][20347] Updated weights for policy 0, policy_version 1341344 (0.0008) [2023-12-27 14:56:31,133][20381] Updated weights for policy 1, policy_version 1340242 (0.0008) [2023-12-27 14:56:31,136][20347] Updated weights for policy 0, policy_version 1341354 (0.0007) [2023-12-27 14:56:31,198][20381] Updated weights for policy 1, policy_version 1340252 (0.0007) [2023-12-27 14:56:31,199][20347] Updated weights for policy 0, policy_version 1341364 (0.0008) [2023-12-27 14:56:31,262][20381] Updated weights for policy 1, policy_version 1340262 (0.0007) [2023-12-27 14:56:31,322][20381] Updated weights for policy 1, policy_version 1340272 (0.0006) [2023-12-27 14:56:31,852][20347] Updated weights for policy 0, policy_version 1341374 (0.0006) [2023-12-27 14:56:31,909][20347] Updated weights for policy 0, policy_version 1341384 (0.0005) [2023-12-27 14:56:31,962][20347] Updated weights for policy 0, policy_version 1341394 (0.0005) [2023-12-27 14:56:32,045][20381] Updated weights for policy 1, policy_version 1340282 (0.0008) [2023-12-27 14:56:32,100][20381] Updated weights for policy 1, policy_version 1340292 (0.0007) [2023-12-27 14:56:32,150][20381] Updated weights for policy 1, policy_version 1340302 (0.0008) [2023-12-27 14:56:32,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 686612480. Throughput: 0: 10237.6, 1: 10287.5. Samples: 686620612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:32,363][19170] Avg episode reward: [(0, '755.310'), (1, '990.250')] [2023-12-27 14:56:32,614][20347] Updated weights for policy 0, policy_version 1341404 (0.0006) [2023-12-27 14:56:32,669][20347] Updated weights for policy 0, policy_version 1341414 (0.0007) [2023-12-27 14:56:32,731][20347] Updated weights for policy 0, policy_version 1341424 (0.0008) [2023-12-27 14:56:32,824][20381] Updated weights for policy 1, policy_version 1340312 (0.0008) [2023-12-27 14:56:32,881][20381] Updated weights for policy 1, policy_version 1340322 (0.0007) [2023-12-27 14:56:32,937][20381] Updated weights for policy 1, policy_version 1340332 (0.0008) [2023-12-27 14:56:33,392][20347] Updated weights for policy 0, policy_version 1341434 (0.0007) [2023-12-27 14:56:33,450][20347] Updated weights for policy 0, policy_version 1341444 (0.0007) [2023-12-27 14:56:33,504][20347] Updated weights for policy 0, policy_version 1341454 (0.0008) [2023-12-27 14:56:33,558][20347] Updated weights for policy 0, policy_version 1341464 (0.0008) [2023-12-27 14:56:33,637][20381] Updated weights for policy 1, policy_version 1340342 (0.0009) [2023-12-27 14:56:33,682][20381] Updated weights for policy 1, policy_version 1340352 (0.0008) [2023-12-27 14:56:33,727][20381] Updated weights for policy 1, policy_version 1340362 (0.0008) [2023-12-27 14:56:34,110][20347] Updated weights for policy 0, policy_version 1341474 (0.0008) [2023-12-27 14:56:34,182][20347] Updated weights for policy 0, policy_version 1341484 (0.0007) [2023-12-27 14:56:34,241][20347] Updated weights for policy 0, policy_version 1341494 (0.0007) [2023-12-27 14:56:34,541][20381] Updated weights for policy 1, policy_version 1340372 (0.0009) [2023-12-27 14:56:34,606][20381] Updated weights for policy 1, policy_version 1340382 (0.0007) [2023-12-27 14:56:34,668][20381] Updated weights for policy 1, policy_version 1340392 (0.0009) [2023-12-27 14:56:34,898][20347] Updated weights for policy 0, policy_version 1341504 (0.0007) [2023-12-27 14:56:34,954][20347] Updated weights for policy 0, policy_version 1341514 (0.0009) [2023-12-27 14:56:34,997][20291] KL-divergence is very high: 161.6893 [2023-12-27 14:56:35,011][20347] Updated weights for policy 0, policy_version 1341524 (0.0008) [2023-12-27 14:56:35,277][20381] Updated weights for policy 1, policy_version 1340402 (0.0008) [2023-12-27 14:56:35,335][20381] Updated weights for policy 1, policy_version 1340412 (0.0009) [2023-12-27 14:56:35,391][20381] Updated weights for policy 1, policy_version 1340422 (0.0008) [2023-12-27 14:56:35,448][20381] Updated weights for policy 1, policy_version 1340432 (0.0008) [2023-12-27 14:56:35,692][20291] KL-divergence is very high: 121.0146 [2023-12-27 14:56:35,727][20347] Updated weights for policy 0, policy_version 1341534 (0.0007) [2023-12-27 14:56:35,787][20347] Updated weights for policy 0, policy_version 1341544 (0.0008) [2023-12-27 14:56:35,843][20347] Updated weights for policy 0, policy_version 1341554 (0.0008) [2023-12-27 14:56:36,152][20381] Updated weights for policy 1, policy_version 1340442 (0.0009) [2023-12-27 14:56:36,210][20381] Updated weights for policy 1, policy_version 1340452 (0.0007) [2023-12-27 14:56:36,261][20381] Updated weights for policy 1, policy_version 1340462 (0.0007) [2023-12-27 14:56:36,607][20347] Updated weights for policy 0, policy_version 1341564 (0.0007) [2023-12-27 14:56:36,658][20347] Updated weights for policy 0, policy_version 1341574 (0.0008) [2023-12-27 14:56:36,709][20347] Updated weights for policy 0, policy_version 1341584 (0.0007) [2023-12-27 14:56:36,992][20381] Updated weights for policy 1, policy_version 1340472 (0.0008) [2023-12-27 14:56:37,038][20381] Updated weights for policy 1, policy_version 1340482 (0.0010) [2023-12-27 14:56:37,085][20381] Updated weights for policy 1, policy_version 1340492 (0.0007) [2023-12-27 14:56:37,327][20347] Updated weights for policy 0, policy_version 1341594 (0.0007) [2023-12-27 14:56:37,362][20291] KL-divergence is very high: 290.3448 [2023-12-27 14:56:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 686710784. Throughput: 0: 10251.9, 1: 10278.9. Samples: 686682856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:37,363][19170] Avg episode reward: [(0, '794.450'), (1, '1000.000')] [2023-12-27 14:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001340496_343195648.pth... [2023-12-27 14:56:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001339280_342884352.pth [2023-12-27 14:56:37,372][20347] Updated weights for policy 0, policy_version 1341604 (0.0008) [2023-12-27 14:56:37,404][20291] KL-divergence is very high: 242.2550 [2023-12-27 14:56:37,429][20347] Updated weights for policy 0, policy_version 1341614 (0.0007) [2023-12-27 14:56:37,454][20291] KL-divergence is very high: 120.2416 [2023-12-27 14:56:37,492][20347] Updated weights for policy 0, policy_version 1341624 (0.0005) [2023-12-27 14:56:37,492][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001341624_343523328.pth... [2023-12-27 14:56:37,496][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001340408_343212032.pth [2023-12-27 14:56:37,821][20381] Updated weights for policy 1, policy_version 1340502 (0.0009) [2023-12-27 14:56:37,877][20381] Updated weights for policy 1, policy_version 1340512 (0.0011) [2023-12-27 14:56:37,929][20381] Updated weights for policy 1, policy_version 1340522 (0.0011) [2023-12-27 14:56:38,113][20347] Updated weights for policy 0, policy_version 1341634 (0.0010) [2023-12-27 14:56:38,162][20347] Updated weights for policy 0, policy_version 1341644 (0.0007) [2023-12-27 14:56:38,209][20347] Updated weights for policy 0, policy_version 1341654 (0.0007) [2023-12-27 14:56:38,640][20381] Updated weights for policy 1, policy_version 1340532 (0.0008) [2023-12-27 14:56:38,696][20381] Updated weights for policy 1, policy_version 1340542 (0.0005) [2023-12-27 14:56:38,757][20381] Updated weights for policy 1, policy_version 1340552 (0.0010) [2023-12-27 14:56:38,912][20347] Updated weights for policy 0, policy_version 1341664 (0.0006) [2023-12-27 14:56:38,971][20347] Updated weights for policy 0, policy_version 1341674 (0.0008) [2023-12-27 14:56:39,026][20347] Updated weights for policy 0, policy_version 1341684 (0.0007) [2023-12-27 14:56:39,423][20381] Updated weights for policy 1, policy_version 1340562 (0.0010) [2023-12-27 14:56:39,488][20381] Updated weights for policy 1, policy_version 1340572 (0.0008) [2023-12-27 14:56:39,549][20381] Updated weights for policy 1, policy_version 1340582 (0.0008) [2023-12-27 14:56:39,610][20381] Updated weights for policy 1, policy_version 1340592 (0.0008) [2023-12-27 14:56:39,713][20347] Updated weights for policy 0, policy_version 1341694 (0.0006) [2023-12-27 14:56:39,766][20347] Updated weights for policy 0, policy_version 1341704 (0.0005) [2023-12-27 14:56:39,827][20347] Updated weights for policy 0, policy_version 1341715 (0.0008) [2023-12-27 14:56:40,327][20381] Updated weights for policy 1, policy_version 1340602 (0.0007) [2023-12-27 14:56:40,394][20381] Updated weights for policy 1, policy_version 1340612 (0.0008) [2023-12-27 14:56:40,452][20381] Updated weights for policy 1, policy_version 1340622 (0.0006) [2023-12-27 14:56:40,458][20347] Updated weights for policy 0, policy_version 1341726 (0.0008) [2023-12-27 14:56:40,519][20347] Updated weights for policy 0, policy_version 1341736 (0.0008) [2023-12-27 14:56:40,569][20347] Updated weights for policy 0, policy_version 1341746 (0.0006) [2023-12-27 14:56:41,086][20381] Updated weights for policy 1, policy_version 1340632 (0.0008) [2023-12-27 14:56:41,153][20381] Updated weights for policy 1, policy_version 1340642 (0.0008) [2023-12-27 14:56:41,214][20381] Updated weights for policy 1, policy_version 1340652 (0.0006) [2023-12-27 14:56:41,282][20347] Updated weights for policy 0, policy_version 1341756 (0.0007) [2023-12-27 14:56:41,350][20347] Updated weights for policy 0, policy_version 1341766 (0.0008) [2023-12-27 14:56:41,413][20347] Updated weights for policy 0, policy_version 1341776 (0.0010) [2023-12-27 14:56:41,965][20381] Updated weights for policy 1, policy_version 1340662 (0.0007) [2023-12-27 14:56:42,022][20381] Updated weights for policy 1, policy_version 1340672 (0.0008) [2023-12-27 14:56:42,068][20381] Updated weights for policy 1, policy_version 1340682 (0.0010) [2023-12-27 14:56:42,088][20347] Updated weights for policy 0, policy_version 1341786 (0.0007) [2023-12-27 14:56:42,145][20347] Updated weights for policy 0, policy_version 1341796 (0.0007) [2023-12-27 14:56:42,212][20347] Updated weights for policy 0, policy_version 1341806 (0.0007) [2023-12-27 14:56:42,269][20347] Updated weights for policy 0, policy_version 1341816 (0.0008) [2023-12-27 14:56:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 686817280. Throughput: 0: 10269.2, 1: 10256.1. Samples: 686804060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:42,363][19170] Avg episode reward: [(0, '804.670'), (1, '1000.000')] [2023-12-27 14:56:42,784][20381] Updated weights for policy 1, policy_version 1340692 (0.0011) [2023-12-27 14:56:42,838][20381] Updated weights for policy 1, policy_version 1340702 (0.0009) [2023-12-27 14:56:42,888][20381] Updated weights for policy 1, policy_version 1340712 (0.0009) [2023-12-27 14:56:42,980][20347] Updated weights for policy 0, policy_version 1341826 (0.0008) [2023-12-27 14:56:43,037][20347] Updated weights for policy 0, policy_version 1341836 (0.0008) [2023-12-27 14:56:43,092][20347] Updated weights for policy 0, policy_version 1341846 (0.0008) [2023-12-27 14:56:43,571][20381] Updated weights for policy 1, policy_version 1340722 (0.0009) [2023-12-27 14:56:43,622][20381] Updated weights for policy 1, policy_version 1340732 (0.0005) [2023-12-27 14:56:43,686][20381] Updated weights for policy 1, policy_version 1340742 (0.0010) [2023-12-27 14:56:43,724][20347] Updated weights for policy 0, policy_version 1341856 (0.0010) [2023-12-27 14:56:43,752][20381] Updated weights for policy 1, policy_version 1340752 (0.0010) [2023-12-27 14:56:43,775][20347] Updated weights for policy 0, policy_version 1341866 (0.0010) [2023-12-27 14:56:43,827][20347] Updated weights for policy 0, policy_version 1341876 (0.0010) [2023-12-27 14:56:44,340][20381] Updated weights for policy 1, policy_version 1340762 (0.0005) [2023-12-27 14:56:44,398][20381] Updated weights for policy 1, policy_version 1340772 (0.0007) [2023-12-27 14:56:44,447][20381] Updated weights for policy 1, policy_version 1340782 (0.0010) [2023-12-27 14:56:44,488][20347] Updated weights for policy 0, policy_version 1341886 (0.0010) [2023-12-27 14:56:44,546][20347] Updated weights for policy 0, policy_version 1341896 (0.0010) [2023-12-27 14:56:44,593][20347] Updated weights for policy 0, policy_version 1341906 (0.0009) [2023-12-27 14:56:45,112][20381] Updated weights for policy 1, policy_version 1340792 (0.0011) [2023-12-27 14:56:45,169][20381] Updated weights for policy 1, policy_version 1340802 (0.0011) [2023-12-27 14:56:45,230][20381] Updated weights for policy 1, policy_version 1340812 (0.0011) [2023-12-27 14:56:45,343][20347] Updated weights for policy 0, policy_version 1341916 (0.0007) [2023-12-27 14:56:45,403][20347] Updated weights for policy 0, policy_version 1341926 (0.0008) [2023-12-27 14:56:45,454][20347] Updated weights for policy 0, policy_version 1341936 (0.0008) [2023-12-27 14:56:45,808][20381] Updated weights for policy 1, policy_version 1340822 (0.0007) [2023-12-27 14:56:45,864][20381] Updated weights for policy 1, policy_version 1340832 (0.0005) [2023-12-27 14:56:45,924][20381] Updated weights for policy 1, policy_version 1340842 (0.0010) [2023-12-27 14:56:46,160][20347] Updated weights for policy 0, policy_version 1341946 (0.0006) [2023-12-27 14:56:46,219][20347] Updated weights for policy 0, policy_version 1341956 (0.0008) [2023-12-27 14:56:46,271][20347] Updated weights for policy 0, policy_version 1341966 (0.0007) [2023-12-27 14:56:46,320][20347] Updated weights for policy 0, policy_version 1341976 (0.0008) [2023-12-27 14:56:46,529][20381] Updated weights for policy 1, policy_version 1340852 (0.0008) [2023-12-27 14:56:46,586][20381] Updated weights for policy 1, policy_version 1340862 (0.0007) [2023-12-27 14:56:46,639][20381] Updated weights for policy 1, policy_version 1340872 (0.0006) [2023-12-27 14:56:47,057][20347] Updated weights for policy 0, policy_version 1341986 (0.0006) [2023-12-27 14:56:47,117][20347] Updated weights for policy 0, policy_version 1341996 (0.0006) [2023-12-27 14:56:47,171][20347] Updated weights for policy 0, policy_version 1342006 (0.0006) [2023-12-27 14:56:47,230][20381] Updated weights for policy 1, policy_version 1340882 (0.0006) [2023-12-27 14:56:47,282][20381] Updated weights for policy 1, policy_version 1340892 (0.0010) [2023-12-27 14:56:47,346][20381] Updated weights for policy 1, policy_version 1340902 (0.0006) [2023-12-27 14:56:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 686915584. Throughput: 0: 10294.3, 1: 10348.5. Samples: 686928156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:47,363][19170] Avg episode reward: [(0, '814.110'), (1, '970.720')] [2023-12-27 14:56:47,404][20381] Updated weights for policy 1, policy_version 1340912 (0.0005) [2023-12-27 14:56:47,744][20347] Updated weights for policy 0, policy_version 1342016 (0.0007) [2023-12-27 14:56:47,794][20347] Updated weights for policy 0, policy_version 1342026 (0.0007) [2023-12-27 14:56:47,840][20347] Updated weights for policy 0, policy_version 1342036 (0.0009) [2023-12-27 14:56:48,044][20381] Updated weights for policy 1, policy_version 1340922 (0.0011) [2023-12-27 14:56:48,123][20381] Updated weights for policy 1, policy_version 1340932 (0.0010) [2023-12-27 14:56:48,171][20381] Updated weights for policy 1, policy_version 1340942 (0.0010) [2023-12-27 14:56:48,521][20347] Updated weights for policy 0, policy_version 1342046 (0.0008) [2023-12-27 14:56:48,586][20347] Updated weights for policy 0, policy_version 1342056 (0.0009) [2023-12-27 14:56:48,639][20347] Updated weights for policy 0, policy_version 1342066 (0.0009) [2023-12-27 14:56:48,789][20381] Updated weights for policy 1, policy_version 1340952 (0.0006) [2023-12-27 14:56:48,846][20381] Updated weights for policy 1, policy_version 1340962 (0.0006) [2023-12-27 14:56:48,904][20381] Updated weights for policy 1, policy_version 1340972 (0.0008) [2023-12-27 14:56:49,347][20347] Updated weights for policy 0, policy_version 1342076 (0.0009) [2023-12-27 14:56:49,408][20347] Updated weights for policy 0, policy_version 1342086 (0.0007) [2023-12-27 14:56:49,464][20347] Updated weights for policy 0, policy_version 1342096 (0.0007) [2023-12-27 14:56:49,542][20381] Updated weights for policy 1, policy_version 1340982 (0.0011) [2023-12-27 14:56:49,599][20381] Updated weights for policy 1, policy_version 1340992 (0.0010) [2023-12-27 14:56:49,658][20381] Updated weights for policy 1, policy_version 1341002 (0.0009) [2023-12-27 14:56:50,150][20347] Updated weights for policy 0, policy_version 1342106 (0.0007) [2023-12-27 14:56:50,215][20347] Updated weights for policy 0, policy_version 1342116 (0.0006) [2023-12-27 14:56:50,277][20347] Updated weights for policy 0, policy_version 1342126 (0.0006) [2023-12-27 14:56:50,339][20347] Updated weights for policy 0, policy_version 1342136 (0.0007) [2023-12-27 14:56:50,397][20381] Updated weights for policy 1, policy_version 1341012 (0.0008) [2023-12-27 14:56:50,447][20381] Updated weights for policy 1, policy_version 1341022 (0.0009) [2023-12-27 14:56:50,503][20381] Updated weights for policy 1, policy_version 1341032 (0.0007) [2023-12-27 14:56:50,914][20347] Updated weights for policy 0, policy_version 1342146 (0.0005) [2023-12-27 14:56:50,971][20347] Updated weights for policy 0, policy_version 1342156 (0.0005) [2023-12-27 14:56:51,036][20347] Updated weights for policy 0, policy_version 1342166 (0.0007) [2023-12-27 14:56:51,242][20381] Updated weights for policy 1, policy_version 1341042 (0.0008) [2023-12-27 14:56:51,303][20381] Updated weights for policy 1, policy_version 1341052 (0.0007) [2023-12-27 14:56:51,369][20381] Updated weights for policy 1, policy_version 1341062 (0.0007) [2023-12-27 14:56:51,434][20381] Updated weights for policy 1, policy_version 1341072 (0.0008) [2023-12-27 14:56:51,774][20347] Updated weights for policy 0, policy_version 1342176 (0.0008) [2023-12-27 14:56:51,837][20347] Updated weights for policy 0, policy_version 1342186 (0.0008) [2023-12-27 14:56:51,896][20347] Updated weights for policy 0, policy_version 1342196 (0.0009) [2023-12-27 14:56:52,048][20381] Updated weights for policy 1, policy_version 1341082 (0.0007) [2023-12-27 14:56:52,101][20381] Updated weights for policy 1, policy_version 1341092 (0.0008) [2023-12-27 14:56:52,156][20381] Updated weights for policy 1, policy_version 1341102 (0.0008) [2023-12-27 14:56:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 687022080. Throughput: 0: 10281.6, 1: 10292.6. Samples: 686990048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:52,363][19170] Avg episode reward: [(0, '843.440'), (1, '952.080')] [2023-12-27 14:56:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001341104_343351296.pth... [2023-12-27 14:56:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001342200_343670784.pth... [2023-12-27 14:56:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001339888_343040000.pth [2023-12-27 14:56:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001341016_343367680.pth [2023-12-27 14:56:52,618][20347] Updated weights for policy 0, policy_version 1342206 (0.0010) [2023-12-27 14:56:52,673][20347] Updated weights for policy 0, policy_version 1342216 (0.0010) [2023-12-27 14:56:52,725][20347] Updated weights for policy 0, policy_version 1342226 (0.0010) [2023-12-27 14:56:52,843][20381] Updated weights for policy 1, policy_version 1341112 (0.0007) [2023-12-27 14:56:52,905][20381] Updated weights for policy 1, policy_version 1341122 (0.0006) [2023-12-27 14:56:52,962][20381] Updated weights for policy 1, policy_version 1341132 (0.0007) [2023-12-27 14:56:53,420][20347] Updated weights for policy 0, policy_version 1342236 (0.0008) [2023-12-27 14:56:53,466][20347] Updated weights for policy 0, policy_version 1342246 (0.0005) [2023-12-27 14:56:53,515][20347] Updated weights for policy 0, policy_version 1342256 (0.0005) [2023-12-27 14:56:53,626][20381] Updated weights for policy 1, policy_version 1341142 (0.0007) [2023-12-27 14:56:53,680][20381] Updated weights for policy 1, policy_version 1341152 (0.0005) [2023-12-27 14:56:53,726][20381] Updated weights for policy 1, policy_version 1341162 (0.0006) [2023-12-27 14:56:54,035][20347] Updated weights for policy 0, policy_version 1342266 (0.0005) [2023-12-27 14:56:54,083][20347] Updated weights for policy 0, policy_version 1342276 (0.0009) [2023-12-27 14:56:54,148][20347] Updated weights for policy 0, policy_version 1342286 (0.0010) [2023-12-27 14:56:54,201][20347] Updated weights for policy 0, policy_version 1342296 (0.0010) [2023-12-27 14:56:54,313][20381] Updated weights for policy 1, policy_version 1341172 (0.0008) [2023-12-27 14:56:54,372][20381] Updated weights for policy 1, policy_version 1341182 (0.0007) [2023-12-27 14:56:54,429][20381] Updated weights for policy 1, policy_version 1341192 (0.0008) [2023-12-27 14:56:54,848][20347] Updated weights for policy 0, policy_version 1342306 (0.0005) [2023-12-27 14:56:54,907][20347] Updated weights for policy 0, policy_version 1342316 (0.0005) [2023-12-27 14:56:54,955][20347] Updated weights for policy 0, policy_version 1342326 (0.0005) [2023-12-27 14:56:55,122][20381] Updated weights for policy 1, policy_version 1341202 (0.0007) [2023-12-27 14:56:55,173][20381] Updated weights for policy 1, policy_version 1341212 (0.0006) [2023-12-27 14:56:55,228][20381] Updated weights for policy 1, policy_version 1341222 (0.0005) [2023-12-27 14:56:55,283][20381] Updated weights for policy 1, policy_version 1341232 (0.0006) [2023-12-27 14:56:55,587][20347] Updated weights for policy 0, policy_version 1342336 (0.0006) [2023-12-27 14:56:55,645][20347] Updated weights for policy 0, policy_version 1342346 (0.0007) [2023-12-27 14:56:55,693][20347] Updated weights for policy 0, policy_version 1342356 (0.0006) [2023-12-27 14:56:55,912][20381] Updated weights for policy 1, policy_version 1341242 (0.0008) [2023-12-27 14:56:55,963][20381] Updated weights for policy 1, policy_version 1341252 (0.0008) [2023-12-27 14:56:56,014][20381] Updated weights for policy 1, policy_version 1341262 (0.0007) [2023-12-27 14:56:56,295][20347] Updated weights for policy 0, policy_version 1342366 (0.0006) [2023-12-27 14:56:56,356][20347] Updated weights for policy 0, policy_version 1342376 (0.0005) [2023-12-27 14:56:56,413][20347] Updated weights for policy 0, policy_version 1342386 (0.0006) [2023-12-27 14:56:56,598][20381] Updated weights for policy 1, policy_version 1341272 (0.0007) [2023-12-27 14:56:56,653][20381] Updated weights for policy 1, policy_version 1341282 (0.0007) [2023-12-27 14:56:56,700][20381] Updated weights for policy 1, policy_version 1341292 (0.0008) [2023-12-27 14:56:56,938][20347] Updated weights for policy 0, policy_version 1342396 (0.0010) [2023-12-27 14:56:56,986][20347] Updated weights for policy 0, policy_version 1342406 (0.0010) [2023-12-27 14:56:57,033][20347] Updated weights for policy 0, policy_version 1342416 (0.0010) [2023-12-27 14:56:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 687128576. Throughput: 0: 10329.4, 1: 10377.2. Samples: 687116600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:56:57,363][19170] Avg episode reward: [(0, '755.360'), (1, '971.620')] [2023-12-27 14:56:57,376][20381] Updated weights for policy 1, policy_version 1341302 (0.0008) [2023-12-27 14:56:57,432][20381] Updated weights for policy 1, policy_version 1341312 (0.0007) [2023-12-27 14:56:57,491][20381] Updated weights for policy 1, policy_version 1341322 (0.0007) [2023-12-27 14:56:57,689][20347] Updated weights for policy 0, policy_version 1342426 (0.0010) [2023-12-27 14:56:57,740][20347] Updated weights for policy 0, policy_version 1342436 (0.0010) [2023-12-27 14:56:57,788][20347] Updated weights for policy 0, policy_version 1342446 (0.0010) [2023-12-27 14:56:57,833][20347] Updated weights for policy 0, policy_version 1342456 (0.0010) [2023-12-27 14:56:58,173][20381] Updated weights for policy 1, policy_version 1341332 (0.0008) [2023-12-27 14:56:58,231][20381] Updated weights for policy 1, policy_version 1341342 (0.0008) [2023-12-27 14:56:58,289][20381] Updated weights for policy 1, policy_version 1341352 (0.0008) [2023-12-27 14:56:58,566][20347] Updated weights for policy 0, policy_version 1342466 (0.0008) [2023-12-27 14:56:58,629][20347] Updated weights for policy 0, policy_version 1342476 (0.0008) [2023-12-27 14:56:58,696][20347] Updated weights for policy 0, policy_version 1342486 (0.0009) [2023-12-27 14:56:59,090][20381] Updated weights for policy 1, policy_version 1341362 (0.0008) [2023-12-27 14:56:59,149][20381] Updated weights for policy 1, policy_version 1341372 (0.0007) [2023-12-27 14:56:59,153][20292] KL-divergence is very high: 469.9515 [2023-12-27 14:56:59,190][20292] KL-divergence is very high: 218.9893 [2023-12-27 14:56:59,196][20381] Updated weights for policy 1, policy_version 1341382 (0.0006) [2023-12-27 14:56:59,265][20381] Updated weights for policy 1, policy_version 1341392 (0.0007) [2023-12-27 14:56:59,481][20347] Updated weights for policy 0, policy_version 1342496 (0.0006) [2023-12-27 14:56:59,530][20347] Updated weights for policy 0, policy_version 1342506 (0.0005) [2023-12-27 14:56:59,588][20347] Updated weights for policy 0, policy_version 1342516 (0.0005) [2023-12-27 14:56:59,987][20381] Updated weights for policy 1, policy_version 1341402 (0.0007) [2023-12-27 14:57:00,050][20381] Updated weights for policy 1, policy_version 1341412 (0.0007) [2023-12-27 14:57:00,109][20381] Updated weights for policy 1, policy_version 1341422 (0.0007) [2023-12-27 14:57:00,262][20347] Updated weights for policy 0, policy_version 1342526 (0.0009) [2023-12-27 14:57:00,324][20347] Updated weights for policy 0, policy_version 1342536 (0.0010) [2023-12-27 14:57:00,414][20347] Updated weights for policy 0, policy_version 1342546 (0.0010) [2023-12-27 14:57:00,712][20381] Updated weights for policy 1, policy_version 1341432 (0.0007) [2023-12-27 14:57:00,771][20381] Updated weights for policy 1, policy_version 1341442 (0.0011) [2023-12-27 14:57:00,826][20381] Updated weights for policy 1, policy_version 1341452 (0.0008) [2023-12-27 14:57:01,008][20347] Updated weights for policy 0, policy_version 1342556 (0.0009) [2023-12-27 14:57:01,064][20347] Updated weights for policy 0, policy_version 1342566 (0.0009) [2023-12-27 14:57:01,112][20347] Updated weights for policy 0, policy_version 1342576 (0.0005) [2023-12-27 14:57:01,581][20381] Updated weights for policy 1, policy_version 1341462 (0.0006) [2023-12-27 14:57:01,646][20381] Updated weights for policy 1, policy_version 1341472 (0.0007) [2023-12-27 14:57:01,711][20381] Updated weights for policy 1, policy_version 1341482 (0.0008) [2023-12-27 14:57:01,841][20347] Updated weights for policy 0, policy_version 1342586 (0.0009) [2023-12-27 14:57:01,890][20347] Updated weights for policy 0, policy_version 1342596 (0.0008) [2023-12-27 14:57:01,948][20347] Updated weights for policy 0, policy_version 1342606 (0.0008) [2023-12-27 14:57:01,998][20347] Updated weights for policy 0, policy_version 1342616 (0.0010) [2023-12-27 14:57:02,358][20381] Updated weights for policy 1, policy_version 1341492 (0.0008) [2023-12-27 14:57:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 687226880. Throughput: 0: 10292.0, 1: 10352.4. Samples: 687239060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:02,363][19170] Avg episode reward: [(0, '804.030'), (1, '980.440')] [2023-12-27 14:57:02,414][20381] Updated weights for policy 1, policy_version 1341502 (0.0007) [2023-12-27 14:57:02,471][20381] Updated weights for policy 1, policy_version 1341512 (0.0008) [2023-12-27 14:57:02,729][20347] Updated weights for policy 0, policy_version 1342626 (0.0005) [2023-12-27 14:57:02,789][20347] Updated weights for policy 0, policy_version 1342636 (0.0010) [2023-12-27 14:57:02,843][20347] Updated weights for policy 0, policy_version 1342646 (0.0010) [2023-12-27 14:57:03,117][20381] Updated weights for policy 1, policy_version 1341522 (0.0007) [2023-12-27 14:57:03,162][20381] Updated weights for policy 1, policy_version 1341532 (0.0005) [2023-12-27 14:57:03,213][20381] Updated weights for policy 1, policy_version 1341542 (0.0006) [2023-12-27 14:57:03,270][20381] Updated weights for policy 1, policy_version 1341552 (0.0007) [2023-12-27 14:57:03,486][20347] Updated weights for policy 0, policy_version 1342656 (0.0006) [2023-12-27 14:57:03,543][20347] Updated weights for policy 0, policy_version 1342666 (0.0005) [2023-12-27 14:57:03,603][20347] Updated weights for policy 0, policy_version 1342676 (0.0005) [2023-12-27 14:57:03,973][20381] Updated weights for policy 1, policy_version 1341562 (0.0007) [2023-12-27 14:57:04,036][20381] Updated weights for policy 1, policy_version 1341572 (0.0008) [2023-12-27 14:57:04,102][20381] Updated weights for policy 1, policy_version 1341582 (0.0009) [2023-12-27 14:57:04,132][20347] Updated weights for policy 0, policy_version 1342686 (0.0006) [2023-12-27 14:57:04,193][20347] Updated weights for policy 0, policy_version 1342696 (0.0005) [2023-12-27 14:57:04,245][20347] Updated weights for policy 0, policy_version 1342706 (0.0010) [2023-12-27 14:57:04,813][20381] Updated weights for policy 1, policy_version 1341592 (0.0007) [2023-12-27 14:57:04,867][20381] Updated weights for policy 1, policy_version 1341602 (0.0006) [2023-12-27 14:57:04,908][20347] Updated weights for policy 0, policy_version 1342716 (0.0010) [2023-12-27 14:57:04,926][20381] Updated weights for policy 1, policy_version 1341612 (0.0005) [2023-12-27 14:57:04,958][20347] Updated weights for policy 0, policy_version 1342726 (0.0010) [2023-12-27 14:57:05,003][20347] Updated weights for policy 0, policy_version 1342736 (0.0010) [2023-12-27 14:57:05,570][20381] Updated weights for policy 1, policy_version 1341622 (0.0008) [2023-12-27 14:57:05,634][20381] Updated weights for policy 1, policy_version 1341632 (0.0011) [2023-12-27 14:57:05,689][20381] Updated weights for policy 1, policy_version 1341642 (0.0011) [2023-12-27 14:57:05,732][20347] Updated weights for policy 0, policy_version 1342746 (0.0009) [2023-12-27 14:57:05,770][20291] KL-divergence is very high: 111.8344 [2023-12-27 14:57:05,783][20291] KL-divergence is very high: 109.8229 [2023-12-27 14:57:05,789][20347] Updated weights for policy 0, policy_version 1342756 (0.0007) [2023-12-27 14:57:05,847][20347] Updated weights for policy 0, policy_version 1342766 (0.0007) [2023-12-27 14:57:05,907][20347] Updated weights for policy 0, policy_version 1342776 (0.0007) [2023-12-27 14:57:06,394][20381] Updated weights for policy 1, policy_version 1341652 (0.0011) [2023-12-27 14:57:06,457][20381] Updated weights for policy 1, policy_version 1341662 (0.0011) [2023-12-27 14:57:06,518][20381] Updated weights for policy 1, policy_version 1341672 (0.0007) [2023-12-27 14:57:06,644][20347] Updated weights for policy 0, policy_version 1342786 (0.0008) [2023-12-27 14:57:06,695][20347] Updated weights for policy 0, policy_version 1342796 (0.0007) [2023-12-27 14:57:06,752][20347] Updated weights for policy 0, policy_version 1342806 (0.0008) [2023-12-27 14:57:07,175][20381] Updated weights for policy 1, policy_version 1341682 (0.0007) [2023-12-27 14:57:07,224][20381] Updated weights for policy 1, policy_version 1341692 (0.0006) [2023-12-27 14:57:07,280][20381] Updated weights for policy 1, policy_version 1341702 (0.0006) [2023-12-27 14:57:07,336][20381] Updated weights for policy 1, policy_version 1341712 (0.0006) [2023-12-27 14:57:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 687333376. Throughput: 0: 10307.8, 1: 10342.2. Samples: 687301724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:07,363][19170] Avg episode reward: [(0, '785.050'), (1, '990.200')] [2023-12-27 14:57:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001341712_343506944.pth... [2023-12-27 14:57:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001340496_343195648.pth [2023-12-27 14:57:07,405][20347] Updated weights for policy 0, policy_version 1342816 (0.0007) [2023-12-27 14:57:07,454][20347] Updated weights for policy 0, policy_version 1342826 (0.0007) [2023-12-27 14:57:07,503][20347] Updated weights for policy 0, policy_version 1342836 (0.0005) [2023-12-27 14:57:07,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001342840_343834624.pth... [2023-12-27 14:57:07,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001341624_343523328.pth [2023-12-27 14:57:07,895][20381] Updated weights for policy 1, policy_version 1341722 (0.0010) [2023-12-27 14:57:07,948][20381] Updated weights for policy 1, policy_version 1341732 (0.0007) [2023-12-27 14:57:08,006][20381] Updated weights for policy 1, policy_version 1341742 (0.0005) [2023-12-27 14:57:08,219][20347] Updated weights for policy 0, policy_version 1342846 (0.0006) [2023-12-27 14:57:08,280][20347] Updated weights for policy 0, policy_version 1342856 (0.0005) [2023-12-27 14:57:08,348][20347] Updated weights for policy 0, policy_version 1342866 (0.0007) [2023-12-27 14:57:08,598][20381] Updated weights for policy 1, policy_version 1341752 (0.0007) [2023-12-27 14:57:08,650][20381] Updated weights for policy 1, policy_version 1341762 (0.0008) [2023-12-27 14:57:08,698][20381] Updated weights for policy 1, policy_version 1341772 (0.0007) [2023-12-27 14:57:08,986][20347] Updated weights for policy 0, policy_version 1342876 (0.0006) [2023-12-27 14:57:09,047][20347] Updated weights for policy 0, policy_version 1342886 (0.0007) [2023-12-27 14:57:09,108][20347] Updated weights for policy 0, policy_version 1342896 (0.0010) [2023-12-27 14:57:09,434][20381] Updated weights for policy 1, policy_version 1341782 (0.0008) [2023-12-27 14:57:09,492][20381] Updated weights for policy 1, policy_version 1341792 (0.0007) [2023-12-27 14:57:09,540][20381] Updated weights for policy 1, policy_version 1341802 (0.0007) [2023-12-27 14:57:09,854][20347] Updated weights for policy 0, policy_version 1342906 (0.0010) [2023-12-27 14:57:09,918][20347] Updated weights for policy 0, policy_version 1342916 (0.0008) [2023-12-27 14:57:09,982][20347] Updated weights for policy 0, policy_version 1342926 (0.0008) [2023-12-27 14:57:10,041][20347] Updated weights for policy 0, policy_version 1342936 (0.0008) [2023-12-27 14:57:10,276][20381] Updated weights for policy 1, policy_version 1341812 (0.0008) [2023-12-27 14:57:10,335][20381] Updated weights for policy 1, policy_version 1341822 (0.0008) [2023-12-27 14:57:10,395][20381] Updated weights for policy 1, policy_version 1341832 (0.0008) [2023-12-27 14:57:10,744][20347] Updated weights for policy 0, policy_version 1342946 (0.0007) [2023-12-27 14:57:10,798][20347] Updated weights for policy 0, policy_version 1342956 (0.0007) [2023-12-27 14:57:10,845][20347] Updated weights for policy 0, policy_version 1342966 (0.0008) [2023-12-27 14:57:11,084][20381] Updated weights for policy 1, policy_version 1341842 (0.0008) [2023-12-27 14:57:11,150][20381] Updated weights for policy 1, policy_version 1341852 (0.0007) [2023-12-27 14:57:11,216][20381] Updated weights for policy 1, policy_version 1341862 (0.0007) [2023-12-27 14:57:11,279][20381] Updated weights for policy 1, policy_version 1341872 (0.0007) [2023-12-27 14:57:11,591][20347] Updated weights for policy 0, policy_version 1342976 (0.0008) [2023-12-27 14:57:11,655][20347] Updated weights for policy 0, policy_version 1342986 (0.0008) [2023-12-27 14:57:11,717][20347] Updated weights for policy 0, policy_version 1342996 (0.0007) [2023-12-27 14:57:12,058][20381] Updated weights for policy 1, policy_version 1341882 (0.0008) [2023-12-27 14:57:12,117][20381] Updated weights for policy 1, policy_version 1341892 (0.0009) [2023-12-27 14:57:12,174][20381] Updated weights for policy 1, policy_version 1341902 (0.0008) [2023-12-27 14:57:12,296][20347] Updated weights for policy 0, policy_version 1343006 (0.0008) [2023-12-27 14:57:12,357][20347] Updated weights for policy 0, policy_version 1343016 (0.0008) [2023-12-27 14:57:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 687431680. Throughput: 0: 10300.9, 1: 10295.7. Samples: 687423292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:12,363][19170] Avg episode reward: [(0, '794.690'), (1, '990.300')] [2023-12-27 14:57:12,421][20347] Updated weights for policy 0, policy_version 1343026 (0.0007) [2023-12-27 14:57:12,908][20381] Updated weights for policy 1, policy_version 1341912 (0.0006) [2023-12-27 14:57:12,963][20381] Updated weights for policy 1, policy_version 1341922 (0.0005) [2023-12-27 14:57:13,017][20381] Updated weights for policy 1, policy_version 1341932 (0.0005) [2023-12-27 14:57:13,080][20347] Updated weights for policy 0, policy_version 1343036 (0.0008) [2023-12-27 14:57:13,126][20347] Updated weights for policy 0, policy_version 1343046 (0.0010) [2023-12-27 14:57:13,170][20347] Updated weights for policy 0, policy_version 1343056 (0.0008) [2023-12-27 14:57:13,610][20381] Updated weights for policy 1, policy_version 1341942 (0.0007) [2023-12-27 14:57:13,657][20381] Updated weights for policy 1, policy_version 1341952 (0.0007) [2023-12-27 14:57:13,704][20381] Updated weights for policy 1, policy_version 1341962 (0.0007) [2023-12-27 14:57:13,906][20347] Updated weights for policy 0, policy_version 1343066 (0.0009) [2023-12-27 14:57:13,969][20347] Updated weights for policy 0, policy_version 1343076 (0.0006) [2023-12-27 14:57:14,030][20347] Updated weights for policy 0, policy_version 1343086 (0.0005) [2023-12-27 14:57:14,099][20347] Updated weights for policy 0, policy_version 1343096 (0.0006) [2023-12-27 14:57:14,420][20381] Updated weights for policy 1, policy_version 1341972 (0.0006) [2023-12-27 14:57:14,479][20381] Updated weights for policy 1, policy_version 1341982 (0.0006) [2023-12-27 14:57:14,542][20381] Updated weights for policy 1, policy_version 1341992 (0.0005) [2023-12-27 14:57:14,685][20347] Updated weights for policy 0, policy_version 1343106 (0.0007) [2023-12-27 14:57:14,735][20347] Updated weights for policy 0, policy_version 1343116 (0.0006) [2023-12-27 14:57:14,802][20347] Updated weights for policy 0, policy_version 1343126 (0.0008) [2023-12-27 14:57:15,148][20381] Updated weights for policy 1, policy_version 1342002 (0.0006) [2023-12-27 14:57:15,216][20381] Updated weights for policy 1, policy_version 1342012 (0.0009) [2023-12-27 14:57:15,276][20381] Updated weights for policy 1, policy_version 1342022 (0.0011) [2023-12-27 14:57:15,334][20381] Updated weights for policy 1, policy_version 1342032 (0.0008) [2023-12-27 14:57:15,532][20347] Updated weights for policy 0, policy_version 1343136 (0.0007) [2023-12-27 14:57:15,583][20347] Updated weights for policy 0, policy_version 1343146 (0.0007) [2023-12-27 14:57:15,630][20347] Updated weights for policy 0, policy_version 1343156 (0.0007) [2023-12-27 14:57:15,966][20381] Updated weights for policy 1, policy_version 1342042 (0.0006) [2023-12-27 14:57:16,012][20381] Updated weights for policy 1, policy_version 1342052 (0.0005) [2023-12-27 14:57:16,063][20381] Updated weights for policy 1, policy_version 1342062 (0.0006) [2023-12-27 14:57:16,372][20347] Updated weights for policy 0, policy_version 1343166 (0.0007) [2023-12-27 14:57:16,434][20347] Updated weights for policy 0, policy_version 1343176 (0.0007) [2023-12-27 14:57:16,489][20347] Updated weights for policy 0, policy_version 1343186 (0.0007) [2023-12-27 14:57:16,720][20381] Updated weights for policy 1, policy_version 1342072 (0.0007) [2023-12-27 14:57:16,777][20381] Updated weights for policy 1, policy_version 1342082 (0.0009) [2023-12-27 14:57:16,835][20381] Updated weights for policy 1, policy_version 1342092 (0.0006) [2023-12-27 14:57:17,194][20347] Updated weights for policy 0, policy_version 1343196 (0.0008) [2023-12-27 14:57:17,256][20347] Updated weights for policy 0, policy_version 1343206 (0.0008) [2023-12-27 14:57:17,309][20347] Updated weights for policy 0, policy_version 1343217 (0.0009) [2023-12-27 14:57:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 687538176. Throughput: 0: 10280.3, 1: 10288.8. Samples: 687546220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:17,363][19170] Avg episode reward: [(0, '853.110'), (1, '961.030')] [2023-12-27 14:57:17,415][20381] Updated weights for policy 1, policy_version 1342102 (0.0006) [2023-12-27 14:57:17,466][20381] Updated weights for policy 1, policy_version 1342112 (0.0005) [2023-12-27 14:57:17,518][20381] Updated weights for policy 1, policy_version 1342122 (0.0007) [2023-12-27 14:57:17,943][20347] Updated weights for policy 0, policy_version 1343227 (0.0007) [2023-12-27 14:57:18,003][20347] Updated weights for policy 0, policy_version 1343237 (0.0005) [2023-12-27 14:57:18,064][20347] Updated weights for policy 0, policy_version 1343247 (0.0005) [2023-12-27 14:57:18,098][20381] Updated weights for policy 1, policy_version 1342132 (0.0009) [2023-12-27 14:57:18,150][20381] Updated weights for policy 1, policy_version 1342142 (0.0010) [2023-12-27 14:57:18,204][20381] Updated weights for policy 1, policy_version 1342152 (0.0010) [2023-12-27 14:57:18,709][20347] Updated weights for policy 0, policy_version 1343257 (0.0006) [2023-12-27 14:57:18,757][20347] Updated weights for policy 0, policy_version 1343267 (0.0007) [2023-12-27 14:57:18,812][20347] Updated weights for policy 0, policy_version 1343277 (0.0008) [2023-12-27 14:57:18,862][20381] Updated weights for policy 1, policy_version 1342162 (0.0007) [2023-12-27 14:57:18,875][20347] Updated weights for policy 0, policy_version 1343287 (0.0007) [2023-12-27 14:57:18,912][20381] Updated weights for policy 1, policy_version 1342172 (0.0008) [2023-12-27 14:57:18,967][20381] Updated weights for policy 1, policy_version 1342182 (0.0007) [2023-12-27 14:57:19,025][20381] Updated weights for policy 1, policy_version 1342192 (0.0010) [2023-12-27 14:57:19,653][20347] Updated weights for policy 0, policy_version 1343297 (0.0008) [2023-12-27 14:57:19,710][20347] Updated weights for policy 0, policy_version 1343307 (0.0006) [2023-12-27 14:57:19,720][20381] Updated weights for policy 1, policy_version 1342202 (0.0009) [2023-12-27 14:57:19,763][20347] Updated weights for policy 0, policy_version 1343317 (0.0006) [2023-12-27 14:57:19,773][20381] Updated weights for policy 1, policy_version 1342212 (0.0006) [2023-12-27 14:57:19,842][20381] Updated weights for policy 1, policy_version 1342222 (0.0008) [2023-12-27 14:57:20,453][20347] Updated weights for policy 0, policy_version 1343327 (0.0006) [2023-12-27 14:57:20,510][20347] Updated weights for policy 0, policy_version 1343337 (0.0007) [2023-12-27 14:57:20,571][20347] Updated weights for policy 0, policy_version 1343347 (0.0008) [2023-12-27 14:57:20,624][20381] Updated weights for policy 1, policy_version 1342232 (0.0008) [2023-12-27 14:57:20,688][20381] Updated weights for policy 1, policy_version 1342242 (0.0008) [2023-12-27 14:57:20,749][20381] Updated weights for policy 1, policy_version 1342252 (0.0008) [2023-12-27 14:57:21,270][20347] Updated weights for policy 0, policy_version 1343357 (0.0007) [2023-12-27 14:57:21,324][20347] Updated weights for policy 0, policy_version 1343367 (0.0007) [2023-12-27 14:57:21,393][20347] Updated weights for policy 0, policy_version 1343377 (0.0008) [2023-12-27 14:57:21,518][20381] Updated weights for policy 1, policy_version 1342262 (0.0007) [2023-12-27 14:57:21,580][20381] Updated weights for policy 1, policy_version 1342272 (0.0006) [2023-12-27 14:57:21,645][20381] Updated weights for policy 1, policy_version 1342282 (0.0008) [2023-12-27 14:57:22,110][20347] Updated weights for policy 0, policy_version 1343387 (0.0008) [2023-12-27 14:57:22,170][20347] Updated weights for policy 0, policy_version 1343397 (0.0008) [2023-12-27 14:57:22,228][20347] Updated weights for policy 0, policy_version 1343407 (0.0008) [2023-12-27 14:57:22,358][20381] Updated weights for policy 1, policy_version 1342292 (0.0007) [2023-12-27 14:57:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 687636480. Throughput: 0: 10252.0, 1: 10316.1. Samples: 687608420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:22,363][19170] Avg episode reward: [(0, '794.290'), (1, '960.980')] [2023-12-27 14:57:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001343416_343982080.pth... [2023-12-27 14:57:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001342200_343670784.pth [2023-12-27 14:57:22,422][20381] Updated weights for policy 1, policy_version 1342302 (0.0008) [2023-12-27 14:57:22,480][20381] Updated weights for policy 1, policy_version 1342312 (0.0008) [2023-12-27 14:57:22,525][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001342320_343662592.pth... [2023-12-27 14:57:22,529][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001341104_343351296.pth [2023-12-27 14:57:22,997][20347] Updated weights for policy 0, policy_version 1343417 (0.0008) [2023-12-27 14:57:23,059][20347] Updated weights for policy 0, policy_version 1343427 (0.0008) [2023-12-27 14:57:23,099][20381] Updated weights for policy 1, policy_version 1342322 (0.0007) [2023-12-27 14:57:23,115][20347] Updated weights for policy 0, policy_version 1343437 (0.0008) [2023-12-27 14:57:23,159][20381] Updated weights for policy 1, policy_version 1342332 (0.0005) [2023-12-27 14:57:23,171][20347] Updated weights for policy 0, policy_version 1343447 (0.0007) [2023-12-27 14:57:23,209][20381] Updated weights for policy 1, policy_version 1342342 (0.0005) [2023-12-27 14:57:23,258][20381] Updated weights for policy 1, policy_version 1342352 (0.0006) [2023-12-27 14:57:23,784][20347] Updated weights for policy 0, policy_version 1343457 (0.0006) [2023-12-27 14:57:23,843][20347] Updated weights for policy 0, policy_version 1343467 (0.0006) [2023-12-27 14:57:23,849][20381] Updated weights for policy 1, policy_version 1342362 (0.0006) [2023-12-27 14:57:23,892][20347] Updated weights for policy 0, policy_version 1343477 (0.0005) [2023-12-27 14:57:23,904][20381] Updated weights for policy 1, policy_version 1342372 (0.0006) [2023-12-27 14:57:23,959][20381] Updated weights for policy 1, policy_version 1342382 (0.0007) [2023-12-27 14:57:24,577][20381] Updated weights for policy 1, policy_version 1342392 (0.0006) [2023-12-27 14:57:24,599][20347] Updated weights for policy 0, policy_version 1343487 (0.0007) [2023-12-27 14:57:24,629][20381] Updated weights for policy 1, policy_version 1342402 (0.0008) [2023-12-27 14:57:24,650][20347] Updated weights for policy 0, policy_version 1343497 (0.0006) [2023-12-27 14:57:24,682][20381] Updated weights for policy 1, policy_version 1342412 (0.0006) [2023-12-27 14:57:24,702][20347] Updated weights for policy 0, policy_version 1343507 (0.0008) [2023-12-27 14:57:25,252][20381] Updated weights for policy 1, policy_version 1342422 (0.0006) [2023-12-27 14:57:25,305][20381] Updated weights for policy 1, policy_version 1342432 (0.0008) [2023-12-27 14:57:25,330][20347] Updated weights for policy 0, policy_version 1343517 (0.0006) [2023-12-27 14:57:25,358][20381] Updated weights for policy 1, policy_version 1342442 (0.0010) [2023-12-27 14:57:25,381][20347] Updated weights for policy 0, policy_version 1343527 (0.0005) [2023-12-27 14:57:25,433][20347] Updated weights for policy 0, policy_version 1343537 (0.0005) [2023-12-27 14:57:26,031][20347] Updated weights for policy 0, policy_version 1343547 (0.0007) [2023-12-27 14:57:26,062][20381] Updated weights for policy 1, policy_version 1342452 (0.0010) [2023-12-27 14:57:26,097][20347] Updated weights for policy 0, policy_version 1343557 (0.0005) [2023-12-27 14:57:26,117][20381] Updated weights for policy 1, policy_version 1342462 (0.0007) [2023-12-27 14:57:26,146][20347] Updated weights for policy 0, policy_version 1343567 (0.0006) [2023-12-27 14:57:26,159][20381] Updated weights for policy 1, policy_version 1342472 (0.0005) [2023-12-27 14:57:26,709][20347] Updated weights for policy 0, policy_version 1343577 (0.0005) [2023-12-27 14:57:26,767][20347] Updated weights for policy 0, policy_version 1343587 (0.0007) [2023-12-27 14:57:26,807][20381] Updated weights for policy 1, policy_version 1342482 (0.0007) [2023-12-27 14:57:26,823][20347] Updated weights for policy 0, policy_version 1343597 (0.0006) [2023-12-27 14:57:26,863][20381] Updated weights for policy 1, policy_version 1342492 (0.0006) [2023-12-27 14:57:26,887][20347] Updated weights for policy 0, policy_version 1343607 (0.0007) [2023-12-27 14:57:26,916][20381] Updated weights for policy 1, policy_version 1342502 (0.0005) [2023-12-27 14:57:26,968][20381] Updated weights for policy 1, policy_version 1342512 (0.0010) [2023-12-27 14:57:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 687742976. Throughput: 0: 10247.8, 1: 10379.0. Samples: 687732268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:27,363][19170] Avg episode reward: [(0, '774.520'), (1, '990.250')] [2023-12-27 14:57:27,447][20347] Updated weights for policy 0, policy_version 1343617 (0.0006) [2023-12-27 14:57:27,491][20347] Updated weights for policy 0, policy_version 1343627 (0.0005) [2023-12-27 14:57:27,537][20347] Updated weights for policy 0, policy_version 1343637 (0.0007) [2023-12-27 14:57:27,608][20381] Updated weights for policy 1, policy_version 1342522 (0.0010) [2023-12-27 14:57:27,653][20381] Updated weights for policy 1, policy_version 1342532 (0.0010) [2023-12-27 14:57:27,701][20381] Updated weights for policy 1, policy_version 1342542 (0.0010) [2023-12-27 14:57:28,158][20347] Updated weights for policy 0, policy_version 1343647 (0.0006) [2023-12-27 14:57:28,209][20347] Updated weights for policy 0, policy_version 1343657 (0.0005) [2023-12-27 14:57:28,261][20347] Updated weights for policy 0, policy_version 1343667 (0.0005) [2023-12-27 14:57:28,372][20381] Updated weights for policy 1, policy_version 1342552 (0.0009) [2023-12-27 14:57:28,427][20381] Updated weights for policy 1, policy_version 1342562 (0.0006) [2023-12-27 14:57:28,486][20381] Updated weights for policy 1, policy_version 1342572 (0.0007) [2023-12-27 14:57:28,925][20347] Updated weights for policy 0, policy_version 1343677 (0.0006) [2023-12-27 14:57:28,971][20347] Updated weights for policy 0, policy_version 1343687 (0.0007) [2023-12-27 14:57:29,015][20347] Updated weights for policy 0, policy_version 1343697 (0.0007) [2023-12-27 14:57:29,124][20381] Updated weights for policy 1, policy_version 1342582 (0.0007) [2023-12-27 14:57:29,178][20381] Updated weights for policy 1, policy_version 1342592 (0.0010) [2023-12-27 14:57:29,245][20381] Updated weights for policy 1, policy_version 1342602 (0.0009) [2023-12-27 14:57:29,755][20347] Updated weights for policy 0, policy_version 1343707 (0.0007) [2023-12-27 14:57:29,804][20347] Updated weights for policy 0, policy_version 1343717 (0.0008) [2023-12-27 14:57:29,869][20347] Updated weights for policy 0, policy_version 1343727 (0.0008) [2023-12-27 14:57:29,951][20381] Updated weights for policy 1, policy_version 1342612 (0.0009) [2023-12-27 14:57:30,014][20381] Updated weights for policy 1, policy_version 1342622 (0.0006) [2023-12-27 14:57:30,073][20381] Updated weights for policy 1, policy_version 1342632 (0.0008) [2023-12-27 14:57:30,553][20347] Updated weights for policy 0, policy_version 1343737 (0.0008) [2023-12-27 14:57:30,601][20347] Updated weights for policy 0, policy_version 1343747 (0.0005) [2023-12-27 14:57:30,652][20347] Updated weights for policy 0, policy_version 1343757 (0.0005) [2023-12-27 14:57:30,697][20381] Updated weights for policy 1, policy_version 1342642 (0.0007) [2023-12-27 14:57:30,710][20347] Updated weights for policy 0, policy_version 1343767 (0.0005) [2023-12-27 14:57:30,756][20381] Updated weights for policy 1, policy_version 1342652 (0.0008) [2023-12-27 14:57:30,808][20381] Updated weights for policy 1, policy_version 1342662 (0.0005) [2023-12-27 14:57:30,860][20381] Updated weights for policy 1, policy_version 1342672 (0.0005) [2023-12-27 14:57:31,356][20347] Updated weights for policy 0, policy_version 1343777 (0.0007) [2023-12-27 14:57:31,422][20347] Updated weights for policy 0, policy_version 1343787 (0.0006) [2023-12-27 14:57:31,478][20347] Updated weights for policy 0, policy_version 1343797 (0.0006) [2023-12-27 14:57:31,533][20381] Updated weights for policy 1, policy_version 1342682 (0.0008) [2023-12-27 14:57:31,591][20381] Updated weights for policy 1, policy_version 1342692 (0.0008) [2023-12-27 14:57:31,658][20381] Updated weights for policy 1, policy_version 1342702 (0.0008) [2023-12-27 14:57:32,124][20347] Updated weights for policy 0, policy_version 1343807 (0.0009) [2023-12-27 14:57:32,179][20347] Updated weights for policy 0, policy_version 1343817 (0.0010) [2023-12-27 14:57:32,235][20347] Updated weights for policy 0, policy_version 1343827 (0.0010) [2023-12-27 14:57:32,300][20381] Updated weights for policy 1, policy_version 1342712 (0.0008) [2023-12-27 14:57:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 687849472. Throughput: 0: 10342.3, 1: 10364.9. Samples: 687859980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:32,363][19170] Avg episode reward: [(0, '804.040'), (1, '1000.000')] [2023-12-27 14:57:32,365][20381] Updated weights for policy 1, policy_version 1342722 (0.0007) [2023-12-27 14:57:32,429][20381] Updated weights for policy 1, policy_version 1342732 (0.0006) [2023-12-27 14:57:33,032][20381] Updated weights for policy 1, policy_version 1342742 (0.0006) [2023-12-27 14:57:33,046][20347] Updated weights for policy 0, policy_version 1343837 (0.0008) [2023-12-27 14:57:33,083][20381] Updated weights for policy 1, policy_version 1342752 (0.0005) [2023-12-27 14:57:33,091][20347] Updated weights for policy 0, policy_version 1343847 (0.0007) [2023-12-27 14:57:33,138][20381] Updated weights for policy 1, policy_version 1342762 (0.0006) [2023-12-27 14:57:33,152][20347] Updated weights for policy 0, policy_version 1343857 (0.0008) [2023-12-27 14:57:33,768][20381] Updated weights for policy 1, policy_version 1342772 (0.0006) [2023-12-27 14:57:33,816][20381] Updated weights for policy 1, policy_version 1342782 (0.0007) [2023-12-27 14:57:33,848][20347] Updated weights for policy 0, policy_version 1343867 (0.0007) [2023-12-27 14:57:33,861][20381] Updated weights for policy 1, policy_version 1342792 (0.0006) [2023-12-27 14:57:33,896][20347] Updated weights for policy 0, policy_version 1343877 (0.0006) [2023-12-27 14:57:33,948][20347] Updated weights for policy 0, policy_version 1343887 (0.0007) [2023-12-27 14:57:34,586][20381] Updated weights for policy 1, policy_version 1342802 (0.0007) [2023-12-27 14:57:34,613][20347] Updated weights for policy 0, policy_version 1343897 (0.0007) [2023-12-27 14:57:34,648][20381] Updated weights for policy 1, policy_version 1342812 (0.0011) [2023-12-27 14:57:34,674][20347] Updated weights for policy 0, policy_version 1343907 (0.0007) [2023-12-27 14:57:34,706][20381] Updated weights for policy 1, policy_version 1342822 (0.0010) [2023-12-27 14:57:34,729][20347] Updated weights for policy 0, policy_version 1343917 (0.0007) [2023-12-27 14:57:34,767][20381] Updated weights for policy 1, policy_version 1342832 (0.0010) [2023-12-27 14:57:34,777][20347] Updated weights for policy 0, policy_version 1343927 (0.0006) [2023-12-27 14:57:35,343][20381] Updated weights for policy 1, policy_version 1342842 (0.0005) [2023-12-27 14:57:35,392][20347] Updated weights for policy 0, policy_version 1343937 (0.0006) [2023-12-27 14:57:35,406][20381] Updated weights for policy 1, policy_version 1342852 (0.0005) [2023-12-27 14:57:35,447][20347] Updated weights for policy 0, policy_version 1343947 (0.0008) [2023-12-27 14:57:35,462][20381] Updated weights for policy 1, policy_version 1342862 (0.0009) [2023-12-27 14:57:35,496][20347] Updated weights for policy 0, policy_version 1343957 (0.0005) [2023-12-27 14:57:36,019][20381] Updated weights for policy 1, policy_version 1342872 (0.0011) [2023-12-27 14:57:36,075][20381] Updated weights for policy 1, policy_version 1342882 (0.0011) [2023-12-27 14:57:36,142][20381] Updated weights for policy 1, policy_version 1342892 (0.0011) [2023-12-27 14:57:36,161][20347] Updated weights for policy 0, policy_version 1343967 (0.0009) [2023-12-27 14:57:36,223][20347] Updated weights for policy 0, policy_version 1343977 (0.0010) [2023-12-27 14:57:36,278][20347] Updated weights for policy 0, policy_version 1343987 (0.0010) [2023-12-27 14:57:36,838][20381] Updated weights for policy 1, policy_version 1342902 (0.0011) [2023-12-27 14:57:36,894][20381] Updated weights for policy 1, policy_version 1342912 (0.0011) [2023-12-27 14:57:36,950][20381] Updated weights for policy 1, policy_version 1342922 (0.0009) [2023-12-27 14:57:36,950][20347] Updated weights for policy 0, policy_version 1343997 (0.0011) [2023-12-27 14:57:37,004][20347] Updated weights for policy 0, policy_version 1344007 (0.0010) [2023-12-27 14:57:37,062][20347] Updated weights for policy 0, policy_version 1344017 (0.0010) [2023-12-27 14:57:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 687955968. Throughput: 0: 10322.9, 1: 10401.9. Samples: 687922664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:37,363][19170] Avg episode reward: [(0, '813.840'), (1, '990.240')] [2023-12-27 14:57:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001342928_343818240.pth... [2023-12-27 14:57:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001344024_344137728.pth... [2023-12-27 14:57:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001341712_343506944.pth [2023-12-27 14:57:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001342840_343834624.pth [2023-12-27 14:57:37,658][20381] Updated weights for policy 1, policy_version 1342932 (0.0006) [2023-12-27 14:57:37,715][20347] Updated weights for policy 0, policy_version 1344027 (0.0008) [2023-12-27 14:57:37,716][20381] Updated weights for policy 1, policy_version 1342942 (0.0007) [2023-12-27 14:57:37,764][20381] Updated weights for policy 1, policy_version 1342952 (0.0006) [2023-12-27 14:57:37,773][20347] Updated weights for policy 0, policy_version 1344037 (0.0009) [2023-12-27 14:57:37,828][20347] Updated weights for policy 0, policy_version 1344047 (0.0010) [2023-12-27 14:57:38,450][20347] Updated weights for policy 0, policy_version 1344057 (0.0010) [2023-12-27 14:57:38,455][20381] Updated weights for policy 1, policy_version 1342962 (0.0006) [2023-12-27 14:57:38,506][20347] Updated weights for policy 0, policy_version 1344067 (0.0007) [2023-12-27 14:57:38,513][20381] Updated weights for policy 1, policy_version 1342972 (0.0006) [2023-12-27 14:57:38,566][20347] Updated weights for policy 0, policy_version 1344077 (0.0008) [2023-12-27 14:57:38,574][20381] Updated weights for policy 1, policy_version 1342982 (0.0006) [2023-12-27 14:57:38,622][20347] Updated weights for policy 0, policy_version 1344087 (0.0006) [2023-12-27 14:57:38,631][20381] Updated weights for policy 1, policy_version 1342992 (0.0006) [2023-12-27 14:57:39,256][20347] Updated weights for policy 0, policy_version 1344097 (0.0007) [2023-12-27 14:57:39,305][20381] Updated weights for policy 1, policy_version 1343002 (0.0007) [2023-12-27 14:57:39,314][20347] Updated weights for policy 0, policy_version 1344107 (0.0005) [2023-12-27 14:57:39,372][20381] Updated weights for policy 1, policy_version 1343012 (0.0008) [2023-12-27 14:57:39,378][20347] Updated weights for policy 0, policy_version 1344117 (0.0007) [2023-12-27 14:57:39,438][20381] Updated weights for policy 1, policy_version 1343022 (0.0007) [2023-12-27 14:57:40,080][20347] Updated weights for policy 0, policy_version 1344127 (0.0009) [2023-12-27 14:57:40,138][20347] Updated weights for policy 0, policy_version 1344137 (0.0008) [2023-12-27 14:57:40,151][20381] Updated weights for policy 1, policy_version 1343032 (0.0009) [2023-12-27 14:57:40,198][20347] Updated weights for policy 0, policy_version 1344147 (0.0007) [2023-12-27 14:57:40,208][20381] Updated weights for policy 1, policy_version 1343042 (0.0006) [2023-12-27 14:57:40,260][20381] Updated weights for policy 1, policy_version 1343052 (0.0007) [2023-12-27 14:57:40,862][20347] Updated weights for policy 0, policy_version 1344157 (0.0007) [2023-12-27 14:57:40,923][20347] Updated weights for policy 0, policy_version 1344167 (0.0008) [2023-12-27 14:57:40,963][20381] Updated weights for policy 1, policy_version 1343062 (0.0007) [2023-12-27 14:57:40,979][20347] Updated weights for policy 0, policy_version 1344177 (0.0011) [2023-12-27 14:57:41,017][20381] Updated weights for policy 1, policy_version 1343072 (0.0010) [2023-12-27 14:57:41,082][20381] Updated weights for policy 1, policy_version 1343082 (0.0011) [2023-12-27 14:57:41,743][20347] Updated weights for policy 0, policy_version 1344187 (0.0008) [2023-12-27 14:57:41,809][20347] Updated weights for policy 0, policy_version 1344197 (0.0008) [2023-12-27 14:57:41,868][20347] Updated weights for policy 0, policy_version 1344207 (0.0007) [2023-12-27 14:57:41,887][20381] Updated weights for policy 1, policy_version 1343092 (0.0009) [2023-12-27 14:57:41,951][20381] Updated weights for policy 1, policy_version 1343102 (0.0006) [2023-12-27 14:57:42,012][20381] Updated weights for policy 1, policy_version 1343112 (0.0006) [2023-12-27 14:57:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 688054272. Throughput: 0: 10293.4, 1: 10343.6. Samples: 688045268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:42,363][19170] Avg episode reward: [(0, '784.380'), (1, '990.240')] [2023-12-27 14:57:42,588][20381] Updated weights for policy 1, policy_version 1343122 (0.0006) [2023-12-27 14:57:42,650][20381] Updated weights for policy 1, policy_version 1343132 (0.0006) [2023-12-27 14:57:42,659][20347] Updated weights for policy 0, policy_version 1344217 (0.0007) [2023-12-27 14:57:42,707][20381] Updated weights for policy 1, policy_version 1343142 (0.0006) [2023-12-27 14:57:42,717][20347] Updated weights for policy 0, policy_version 1344227 (0.0008) [2023-12-27 14:57:42,766][20381] Updated weights for policy 1, policy_version 1343152 (0.0005) [2023-12-27 14:57:42,771][20347] Updated weights for policy 0, policy_version 1344237 (0.0007) [2023-12-27 14:57:42,826][20347] Updated weights for policy 0, policy_version 1344247 (0.0008) [2023-12-27 14:57:43,352][20381] Updated weights for policy 1, policy_version 1343162 (0.0006) [2023-12-27 14:57:43,401][20381] Updated weights for policy 1, policy_version 1343172 (0.0008) [2023-12-27 14:57:43,449][20381] Updated weights for policy 1, policy_version 1343182 (0.0007) [2023-12-27 14:57:43,455][20347] Updated weights for policy 0, policy_version 1344257 (0.0007) [2023-12-27 14:57:43,512][20347] Updated weights for policy 0, policy_version 1344267 (0.0008) [2023-12-27 14:57:43,575][20347] Updated weights for policy 0, policy_version 1344277 (0.0007) [2023-12-27 14:57:44,121][20381] Updated weights for policy 1, policy_version 1343192 (0.0008) [2023-12-27 14:57:44,181][20381] Updated weights for policy 1, policy_version 1343202 (0.0007) [2023-12-27 14:57:44,200][20347] Updated weights for policy 0, policy_version 1344287 (0.0010) [2023-12-27 14:57:44,228][20381] Updated weights for policy 1, policy_version 1343212 (0.0007) [2023-12-27 14:57:44,258][20347] Updated weights for policy 0, policy_version 1344297 (0.0010) [2023-12-27 14:57:44,306][20347] Updated weights for policy 0, policy_version 1344307 (0.0010) [2023-12-27 14:57:44,873][20381] Updated weights for policy 1, policy_version 1343222 (0.0006) [2023-12-27 14:57:44,938][20381] Updated weights for policy 1, policy_version 1343232 (0.0007) [2023-12-27 14:57:44,983][20347] Updated weights for policy 0, policy_version 1344317 (0.0010) [2023-12-27 14:57:45,003][20381] Updated weights for policy 1, policy_version 1343242 (0.0006) [2023-12-27 14:57:45,050][20347] Updated weights for policy 0, policy_version 1344327 (0.0006) [2023-12-27 14:57:45,115][20347] Updated weights for policy 0, policy_version 1344337 (0.0006) [2023-12-27 14:57:45,618][20381] Updated weights for policy 1, policy_version 1343252 (0.0006) [2023-12-27 14:57:45,677][20381] Updated weights for policy 1, policy_version 1343262 (0.0007) [2023-12-27 14:57:45,730][20347] Updated weights for policy 0, policy_version 1344347 (0.0006) [2023-12-27 14:57:45,731][20381] Updated weights for policy 1, policy_version 1343272 (0.0007) [2023-12-27 14:57:45,783][20347] Updated weights for policy 0, policy_version 1344357 (0.0005) [2023-12-27 14:57:45,831][20347] Updated weights for policy 0, policy_version 1344367 (0.0008) [2023-12-27 14:57:46,370][20381] Updated weights for policy 1, policy_version 1343282 (0.0007) [2023-12-27 14:57:46,425][20381] Updated weights for policy 1, policy_version 1343292 (0.0005) [2023-12-27 14:57:46,473][20381] Updated weights for policy 1, policy_version 1343302 (0.0005) [2023-12-27 14:57:46,488][20347] Updated weights for policy 0, policy_version 1344377 (0.0010) [2023-12-27 14:57:46,524][20381] Updated weights for policy 1, policy_version 1343312 (0.0005) [2023-12-27 14:57:46,534][20347] Updated weights for policy 0, policy_version 1344387 (0.0010) [2023-12-27 14:57:46,583][20347] Updated weights for policy 0, policy_version 1344397 (0.0010) [2023-12-27 14:57:46,632][20347] Updated weights for policy 0, policy_version 1344407 (0.0010) [2023-12-27 14:57:47,195][20381] Updated weights for policy 1, policy_version 1343322 (0.0007) [2023-12-27 14:57:47,260][20381] Updated weights for policy 1, policy_version 1343332 (0.0006) [2023-12-27 14:57:47,287][20347] Updated weights for policy 0, policy_version 1344417 (0.0006) [2023-12-27 14:57:47,324][20381] Updated weights for policy 1, policy_version 1343342 (0.0005) [2023-12-27 14:57:47,342][20347] Updated weights for policy 0, policy_version 1344427 (0.0005) [2023-12-27 14:57:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 688160768. Throughput: 0: 10301.5, 1: 10403.1. Samples: 688170768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:47,363][19170] Avg episode reward: [(0, '784.540'), (1, '960.980')] [2023-12-27 14:57:47,401][20347] Updated weights for policy 0, policy_version 1344437 (0.0009) [2023-12-27 14:57:47,947][20381] Updated weights for policy 1, policy_version 1343352 (0.0006) [2023-12-27 14:57:48,009][20381] Updated weights for policy 1, policy_version 1343362 (0.0005) [2023-12-27 14:57:48,039][20347] Updated weights for policy 0, policy_version 1344447 (0.0010) [2023-12-27 14:57:48,067][20381] Updated weights for policy 1, policy_version 1343372 (0.0005) [2023-12-27 14:57:48,093][20347] Updated weights for policy 0, policy_version 1344457 (0.0010) [2023-12-27 14:57:48,148][20347] Updated weights for policy 0, policy_version 1344467 (0.0010) [2023-12-27 14:57:48,631][20381] Updated weights for policy 1, policy_version 1343382 (0.0006) [2023-12-27 14:57:48,693][20381] Updated weights for policy 1, policy_version 1343392 (0.0005) [2023-12-27 14:57:48,753][20381] Updated weights for policy 1, policy_version 1343402 (0.0006) [2023-12-27 14:57:48,861][20347] Updated weights for policy 0, policy_version 1344477 (0.0009) [2023-12-27 14:57:48,923][20347] Updated weights for policy 0, policy_version 1344487 (0.0005) [2023-12-27 14:57:48,982][20347] Updated weights for policy 0, policy_version 1344497 (0.0009) [2023-12-27 14:57:49,434][20381] Updated weights for policy 1, policy_version 1343412 (0.0008) [2023-12-27 14:57:49,490][20381] Updated weights for policy 1, policy_version 1343422 (0.0008) [2023-12-27 14:57:49,542][20381] Updated weights for policy 1, policy_version 1343432 (0.0007) [2023-12-27 14:57:49,617][20347] Updated weights for policy 0, policy_version 1344507 (0.0009) [2023-12-27 14:57:49,676][20347] Updated weights for policy 0, policy_version 1344517 (0.0005) [2023-12-27 14:57:49,734][20347] Updated weights for policy 0, policy_version 1344527 (0.0005) [2023-12-27 14:57:50,285][20381] Updated weights for policy 1, policy_version 1343442 (0.0007) [2023-12-27 14:57:50,338][20381] Updated weights for policy 1, policy_version 1343452 (0.0008) [2023-12-27 14:57:50,395][20347] Updated weights for policy 0, policy_version 1344537 (0.0007) [2023-12-27 14:57:50,397][20381] Updated weights for policy 1, policy_version 1343462 (0.0008) [2023-12-27 14:57:50,446][20381] Updated weights for policy 1, policy_version 1343472 (0.0005) [2023-12-27 14:57:50,451][20347] Updated weights for policy 0, policy_version 1344547 (0.0006) [2023-12-27 14:57:50,505][20347] Updated weights for policy 0, policy_version 1344557 (0.0007) [2023-12-27 14:57:50,550][20347] Updated weights for policy 0, policy_version 1344567 (0.0008) [2023-12-27 14:57:51,198][20381] Updated weights for policy 1, policy_version 1343482 (0.0007) [2023-12-27 14:57:51,215][20347] Updated weights for policy 0, policy_version 1344577 (0.0007) [2023-12-27 14:57:51,261][20381] Updated weights for policy 1, policy_version 1343492 (0.0006) [2023-12-27 14:57:51,279][20347] Updated weights for policy 0, policy_version 1344587 (0.0007) [2023-12-27 14:57:51,319][20381] Updated weights for policy 1, policy_version 1343502 (0.0007) [2023-12-27 14:57:51,339][20347] Updated weights for policy 0, policy_version 1344597 (0.0006) [2023-12-27 14:57:51,978][20381] Updated weights for policy 1, policy_version 1343512 (0.0006) [2023-12-27 14:57:52,036][20381] Updated weights for policy 1, policy_version 1343522 (0.0008) [2023-12-27 14:57:52,098][20381] Updated weights for policy 1, policy_version 1343532 (0.0008) [2023-12-27 14:57:52,145][20347] Updated weights for policy 0, policy_version 1344607 (0.0008) [2023-12-27 14:57:52,203][20347] Updated weights for policy 0, policy_version 1344617 (0.0008) [2023-12-27 14:57:52,262][20347] Updated weights for policy 0, policy_version 1344627 (0.0010) [2023-12-27 14:57:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 688267264. Throughput: 0: 10276.8, 1: 10435.9. Samples: 688233800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 14:57:52,363][19170] Avg episode reward: [(0, '754.880'), (1, '921.920')] [2023-12-27 14:57:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001343536_343973888.pth... [2023-12-27 14:57:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001344632_344293376.pth... [2023-12-27 14:57:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001343416_343982080.pth [2023-12-27 14:57:52,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001342320_343662592.pth [2023-12-27 14:57:52,742][20381] Updated weights for policy 1, policy_version 1343542 (0.0007) [2023-12-27 14:57:52,790][20381] Updated weights for policy 1, policy_version 1343552 (0.0007) [2023-12-27 14:57:52,842][20381] Updated weights for policy 1, policy_version 1343562 (0.0008) [2023-12-27 14:57:52,966][20347] Updated weights for policy 0, policy_version 1344637 (0.0008) [2023-12-27 14:57:53,025][20347] Updated weights for policy 0, policy_version 1344647 (0.0006) [2023-12-27 14:57:53,071][20347] Updated weights for policy 0, policy_version 1344657 (0.0005) [2023-12-27 14:57:53,601][20381] Updated weights for policy 1, policy_version 1343572 (0.0008) [2023-12-27 14:57:53,662][20381] Updated weights for policy 1, policy_version 1343582 (0.0006) [2023-12-27 14:57:53,702][20347] Updated weights for policy 0, policy_version 1344667 (0.0005) [2023-12-27 14:57:53,719][20381] Updated weights for policy 1, policy_version 1343592 (0.0006) [2023-12-27 14:57:53,754][20347] Updated weights for policy 0, policy_version 1344677 (0.0008) [2023-12-27 14:57:53,804][20347] Updated weights for policy 0, policy_version 1344687 (0.0007) [2023-12-27 14:57:54,391][20381] Updated weights for policy 1, policy_version 1343602 (0.0007) [2023-12-27 14:57:54,447][20381] Updated weights for policy 1, policy_version 1343612 (0.0007) [2023-12-27 14:57:54,453][20347] Updated weights for policy 0, policy_version 1344697 (0.0008) [2023-12-27 14:57:54,501][20381] Updated weights for policy 1, policy_version 1343622 (0.0007) [2023-12-27 14:57:54,501][20347] Updated weights for policy 0, policy_version 1344707 (0.0008) [2023-12-27 14:57:54,554][20347] Updated weights for policy 0, policy_version 1344717 (0.0010) [2023-12-27 14:57:54,560][20381] Updated weights for policy 1, policy_version 1343632 (0.0005) [2023-12-27 14:57:54,607][20347] Updated weights for policy 0, policy_version 1344727 (0.0007) [2023-12-27 14:57:55,233][20347] Updated weights for policy 0, policy_version 1344737 (0.0005) [2023-12-27 14:57:55,250][20381] Updated weights for policy 1, policy_version 1343642 (0.0006) [2023-12-27 14:57:55,295][20347] Updated weights for policy 0, policy_version 1344747 (0.0006) [2023-12-27 14:57:55,304][20381] Updated weights for policy 1, policy_version 1343652 (0.0007) [2023-12-27 14:57:55,350][20347] Updated weights for policy 0, policy_version 1344757 (0.0005) [2023-12-27 14:57:55,357][20381] Updated weights for policy 1, policy_version 1343662 (0.0007) [2023-12-27 14:57:55,997][20347] Updated weights for policy 0, policy_version 1344767 (0.0007) [2023-12-27 14:57:56,046][20347] Updated weights for policy 0, policy_version 1344777 (0.0010) [2023-12-27 14:57:56,059][20381] Updated weights for policy 1, policy_version 1343672 (0.0007) [2023-12-27 14:57:56,069][20292] KL-divergence is very high: 193.9945 [2023-12-27 14:57:56,099][20347] Updated weights for policy 0, policy_version 1344787 (0.0010) [2023-12-27 14:57:56,112][20381] Updated weights for policy 1, policy_version 1343682 (0.0005) [2023-12-27 14:57:56,113][20292] KL-divergence is very high: 213.2572 [2023-12-27 14:57:56,160][20292] KL-divergence is very high: 132.1884 [2023-12-27 14:57:56,172][20381] Updated weights for policy 1, policy_version 1343692 (0.0007) [2023-12-27 14:57:56,733][20347] Updated weights for policy 0, policy_version 1344797 (0.0008) [2023-12-27 14:57:56,785][20347] Updated weights for policy 0, policy_version 1344807 (0.0009) [2023-12-27 14:57:56,836][20381] Updated weights for policy 1, policy_version 1343702 (0.0006) [2023-12-27 14:57:56,839][20347] Updated weights for policy 0, policy_version 1344817 (0.0010) [2023-12-27 14:57:56,894][20381] Updated weights for policy 1, policy_version 1343712 (0.0006) [2023-12-27 14:57:56,946][20381] Updated weights for policy 1, policy_version 1343722 (0.0008) [2023-12-27 14:57:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 688365568. Throughput: 0: 10332.7, 1: 10398.5. Samples: 688356196. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:57:57,363][19170] Avg episode reward: [(0, '872.540'), (1, '921.780')] [2023-12-27 14:57:57,455][20347] Updated weights for policy 0, policy_version 1344827 (0.0006) [2023-12-27 14:57:57,513][20347] Updated weights for policy 0, policy_version 1344837 (0.0010) [2023-12-27 14:57:57,570][20347] Updated weights for policy 0, policy_version 1344847 (0.0009) [2023-12-27 14:57:57,620][20381] Updated weights for policy 1, policy_version 1343732 (0.0006) [2023-12-27 14:57:57,681][20381] Updated weights for policy 1, policy_version 1343742 (0.0006) [2023-12-27 14:57:57,736][20381] Updated weights for policy 1, policy_version 1343752 (0.0006) [2023-12-27 14:57:58,222][20347] Updated weights for policy 0, policy_version 1344857 (0.0006) [2023-12-27 14:57:58,287][20347] Updated weights for policy 0, policy_version 1344867 (0.0010) [2023-12-27 14:57:58,346][20347] Updated weights for policy 0, policy_version 1344877 (0.0010) [2023-12-27 14:57:58,409][20347] Updated weights for policy 0, policy_version 1344887 (0.0009) [2023-12-27 14:57:58,431][20381] Updated weights for policy 1, policy_version 1343762 (0.0007) [2023-12-27 14:57:58,500][20381] Updated weights for policy 1, policy_version 1343772 (0.0007) [2023-12-27 14:57:58,562][20381] Updated weights for policy 1, policy_version 1343782 (0.0008) [2023-12-27 14:57:58,627][20381] Updated weights for policy 1, policy_version 1343792 (0.0008) [2023-12-27 14:57:59,180][20347] Updated weights for policy 0, policy_version 1344897 (0.0006) [2023-12-27 14:57:59,259][20347] Updated weights for policy 0, policy_version 1344907 (0.0008) [2023-12-27 14:57:59,320][20347] Updated weights for policy 0, policy_version 1344917 (0.0008) [2023-12-27 14:57:59,369][20381] Updated weights for policy 1, policy_version 1343802 (0.0007) [2023-12-27 14:57:59,442][20381] Updated weights for policy 1, policy_version 1343812 (0.0008) [2023-12-27 14:57:59,505][20381] Updated weights for policy 1, policy_version 1343822 (0.0008) [2023-12-27 14:58:00,004][20347] Updated weights for policy 0, policy_version 1344927 (0.0009) [2023-12-27 14:58:00,062][20347] Updated weights for policy 0, policy_version 1344937 (0.0010) [2023-12-27 14:58:00,117][20347] Updated weights for policy 0, policy_version 1344947 (0.0010) [2023-12-27 14:58:00,217][20381] Updated weights for policy 1, policy_version 1343832 (0.0008) [2023-12-27 14:58:00,271][20381] Updated weights for policy 1, policy_version 1343842 (0.0007) [2023-12-27 14:58:00,322][20381] Updated weights for policy 1, policy_version 1343852 (0.0008) [2023-12-27 14:58:00,812][20347] Updated weights for policy 0, policy_version 1344957 (0.0010) [2023-12-27 14:58:00,867][20347] Updated weights for policy 0, policy_version 1344967 (0.0010) [2023-12-27 14:58:00,921][20347] Updated weights for policy 0, policy_version 1344977 (0.0010) [2023-12-27 14:58:00,976][20381] Updated weights for policy 1, policy_version 1343862 (0.0008) [2023-12-27 14:58:01,034][20381] Updated weights for policy 1, policy_version 1343872 (0.0008) [2023-12-27 14:58:01,095][20381] Updated weights for policy 1, policy_version 1343882 (0.0008) [2023-12-27 14:58:01,559][20347] Updated weights for policy 0, policy_version 1344987 (0.0007) [2023-12-27 14:58:01,623][20347] Updated weights for policy 0, policy_version 1344997 (0.0007) [2023-12-27 14:58:01,680][20347] Updated weights for policy 0, policy_version 1345007 (0.0009) [2023-12-27 14:58:01,923][20381] Updated weights for policy 1, policy_version 1343892 (0.0008) [2023-12-27 14:58:01,990][20381] Updated weights for policy 1, policy_version 1343902 (0.0007) [2023-12-27 14:58:02,044][20381] Updated weights for policy 1, policy_version 1343912 (0.0006) [2023-12-27 14:58:02,350][20347] Updated weights for policy 0, policy_version 1345017 (0.0008) [2023-12-27 14:58:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 688463872. Throughput: 0: 10345.5, 1: 10330.6. Samples: 688476644. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:02,363][19170] Avg episode reward: [(0, '892.110'), (1, '873.200')] [2023-12-27 14:58:02,405][20347] Updated weights for policy 0, policy_version 1345027 (0.0010) [2023-12-27 14:58:02,460][20347] Updated weights for policy 0, policy_version 1345037 (0.0010) [2023-12-27 14:58:02,510][20347] Updated weights for policy 0, policy_version 1345047 (0.0009) [2023-12-27 14:58:02,754][20381] Updated weights for policy 1, policy_version 1343922 (0.0008) [2023-12-27 14:58:02,806][20381] Updated weights for policy 1, policy_version 1343932 (0.0007) [2023-12-27 14:58:02,854][20381] Updated weights for policy 1, policy_version 1343942 (0.0008) [2023-12-27 14:58:02,911][20381] Updated weights for policy 1, policy_version 1343952 (0.0008) [2023-12-27 14:58:03,120][20347] Updated weights for policy 0, policy_version 1345057 (0.0005) [2023-12-27 14:58:03,167][20347] Updated weights for policy 0, policy_version 1345067 (0.0005) [2023-12-27 14:58:03,219][20347] Updated weights for policy 0, policy_version 1345077 (0.0006) [2023-12-27 14:58:03,649][20381] Updated weights for policy 1, policy_version 1343962 (0.0010) [2023-12-27 14:58:03,704][20381] Updated weights for policy 1, policy_version 1343972 (0.0010) [2023-12-27 14:58:03,761][20381] Updated weights for policy 1, policy_version 1343982 (0.0008) [2023-12-27 14:58:03,849][20347] Updated weights for policy 0, policy_version 1345087 (0.0010) [2023-12-27 14:58:03,916][20347] Updated weights for policy 0, policy_version 1345097 (0.0007) [2023-12-27 14:58:03,979][20347] Updated weights for policy 0, policy_version 1345107 (0.0010) [2023-12-27 14:58:04,432][20381] Updated weights for policy 1, policy_version 1343992 (0.0007) [2023-12-27 14:58:04,480][20381] Updated weights for policy 1, policy_version 1344002 (0.0007) [2023-12-27 14:58:04,542][20381] Updated weights for policy 1, policy_version 1344012 (0.0008) [2023-12-27 14:58:04,637][20347] Updated weights for policy 0, policy_version 1345117 (0.0010) [2023-12-27 14:58:04,694][20347] Updated weights for policy 0, policy_version 1345127 (0.0010) [2023-12-27 14:58:04,745][20347] Updated weights for policy 0, policy_version 1345137 (0.0006) [2023-12-27 14:58:05,218][20381] Updated weights for policy 1, policy_version 1344022 (0.0006) [2023-12-27 14:58:05,284][20381] Updated weights for policy 1, policy_version 1344032 (0.0005) [2023-12-27 14:58:05,347][20381] Updated weights for policy 1, policy_version 1344042 (0.0006) [2023-12-27 14:58:05,350][20347] Updated weights for policy 0, policy_version 1345147 (0.0009) [2023-12-27 14:58:05,397][20347] Updated weights for policy 0, policy_version 1345157 (0.0007) [2023-12-27 14:58:05,455][20347] Updated weights for policy 0, policy_version 1345167 (0.0010) [2023-12-27 14:58:05,884][20381] Updated weights for policy 1, policy_version 1344052 (0.0007) [2023-12-27 14:58:05,936][20381] Updated weights for policy 1, policy_version 1344062 (0.0007) [2023-12-27 14:58:05,987][20381] Updated weights for policy 1, policy_version 1344072 (0.0007) [2023-12-27 14:58:06,096][20347] Updated weights for policy 0, policy_version 1345177 (0.0010) [2023-12-27 14:58:06,166][20347] Updated weights for policy 0, policy_version 1345187 (0.0007) [2023-12-27 14:58:06,228][20347] Updated weights for policy 0, policy_version 1345197 (0.0008) [2023-12-27 14:58:06,283][20347] Updated weights for policy 0, policy_version 1345207 (0.0005) [2023-12-27 14:58:06,724][20381] Updated weights for policy 1, policy_version 1344082 (0.0008) [2023-12-27 14:58:06,788][20381] Updated weights for policy 1, policy_version 1344092 (0.0009) [2023-12-27 14:58:06,851][20381] Updated weights for policy 1, policy_version 1344102 (0.0007) [2023-12-27 14:58:06,870][20347] Updated weights for policy 0, policy_version 1345217 (0.0008) [2023-12-27 14:58:06,908][20381] Updated weights for policy 1, policy_version 1344112 (0.0006) [2023-12-27 14:58:06,927][20347] Updated weights for policy 0, policy_version 1345227 (0.0011) [2023-12-27 14:58:06,986][20347] Updated weights for policy 0, policy_version 1345237 (0.0010) [2023-12-27 14:58:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 688570368. Throughput: 0: 10384.1, 1: 10299.7. Samples: 688539188. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:07,363][19170] Avg episode reward: [(0, '823.530'), (1, '854.220')] [2023-12-27 14:58:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001344112_344121344.pth... [2023-12-27 14:58:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001345240_344449024.pth... [2023-12-27 14:58:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001342928_343818240.pth [2023-12-27 14:58:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001344024_344137728.pth [2023-12-27 14:58:07,494][20381] Updated weights for policy 1, policy_version 1344122 (0.0008) [2023-12-27 14:58:07,546][20381] Updated weights for policy 1, policy_version 1344132 (0.0007) [2023-12-27 14:58:07,593][20381] Updated weights for policy 1, policy_version 1344142 (0.0008) [2023-12-27 14:58:07,625][20347] Updated weights for policy 0, policy_version 1345247 (0.0010) [2023-12-27 14:58:07,682][20347] Updated weights for policy 0, policy_version 1345257 (0.0006) [2023-12-27 14:58:07,732][20347] Updated weights for policy 0, policy_version 1345267 (0.0007) [2023-12-27 14:58:08,266][20381] Updated weights for policy 1, policy_version 1344152 (0.0010) [2023-12-27 14:58:08,319][20347] Updated weights for policy 0, policy_version 1345277 (0.0007) [2023-12-27 14:58:08,332][20381] Updated weights for policy 1, policy_version 1344162 (0.0010) [2023-12-27 14:58:08,383][20347] Updated weights for policy 0, policy_version 1345287 (0.0007) [2023-12-27 14:58:08,402][20381] Updated weights for policy 1, policy_version 1344172 (0.0008) [2023-12-27 14:58:08,448][20347] Updated weights for policy 0, policy_version 1345297 (0.0006) [2023-12-27 14:58:09,104][20381] Updated weights for policy 1, policy_version 1344182 (0.0007) [2023-12-27 14:58:09,140][20347] Updated weights for policy 0, policy_version 1345307 (0.0008) [2023-12-27 14:58:09,157][20381] Updated weights for policy 1, policy_version 1344192 (0.0006) [2023-12-27 14:58:09,199][20291] KL-divergence is very high: 112.1855 [2023-12-27 14:58:09,201][20347] Updated weights for policy 0, policy_version 1345317 (0.0010) [2023-12-27 14:58:09,207][20381] Updated weights for policy 1, policy_version 1344202 (0.0006) [2023-12-27 14:58:09,261][20347] Updated weights for policy 0, policy_version 1345327 (0.0009) [2023-12-27 14:58:09,924][20381] Updated weights for policy 1, policy_version 1344212 (0.0008) [2023-12-27 14:58:09,992][20381] Updated weights for policy 1, policy_version 1344222 (0.0008) [2023-12-27 14:58:10,012][20347] Updated weights for policy 0, policy_version 1345337 (0.0008) [2023-12-27 14:58:10,051][20381] Updated weights for policy 1, policy_version 1344232 (0.0007) [2023-12-27 14:58:10,077][20347] Updated weights for policy 0, policy_version 1345347 (0.0007) [2023-12-27 14:58:10,138][20347] Updated weights for policy 0, policy_version 1345357 (0.0007) [2023-12-27 14:58:10,195][20347] Updated weights for policy 0, policy_version 1345367 (0.0007) [2023-12-27 14:58:10,691][20381] Updated weights for policy 1, policy_version 1344242 (0.0007) [2023-12-27 14:58:10,749][20381] Updated weights for policy 1, policy_version 1344252 (0.0010) [2023-12-27 14:58:10,754][20292] KL-divergence is very high: 106.2674 [2023-12-27 14:58:10,808][20381] Updated weights for policy 1, policy_version 1344262 (0.0008) [2023-12-27 14:58:10,866][20381] Updated weights for policy 1, policy_version 1344272 (0.0010) [2023-12-27 14:58:10,887][20347] Updated weights for policy 0, policy_version 1345377 (0.0007) [2023-12-27 14:58:10,943][20347] Updated weights for policy 0, policy_version 1345387 (0.0006) [2023-12-27 14:58:11,004][20347] Updated weights for policy 0, policy_version 1345397 (0.0005) [2023-12-27 14:58:11,659][20381] Updated weights for policy 1, policy_version 1344282 (0.0008) [2023-12-27 14:58:11,726][20381] Updated weights for policy 1, policy_version 1344292 (0.0008) [2023-12-27 14:58:11,726][20347] Updated weights for policy 0, policy_version 1345407 (0.0008) [2023-12-27 14:58:11,778][20381] Updated weights for policy 1, policy_version 1344302 (0.0008) [2023-12-27 14:58:11,783][20347] Updated weights for policy 0, policy_version 1345417 (0.0008) [2023-12-27 14:58:11,836][20347] Updated weights for policy 0, policy_version 1345427 (0.0008) [2023-12-27 14:58:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 688668672. Throughput: 0: 10419.5, 1: 10281.8. Samples: 688663828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:12,363][19170] Avg episode reward: [(0, '745.470'), (1, '804.750')] [2023-12-27 14:58:12,512][20347] Updated weights for policy 0, policy_version 1345437 (0.0008) [2023-12-27 14:58:12,568][20381] Updated weights for policy 1, policy_version 1344312 (0.0006) [2023-12-27 14:58:12,571][20347] Updated weights for policy 0, policy_version 1345447 (0.0010) [2023-12-27 14:58:12,628][20381] Updated weights for policy 1, policy_version 1344322 (0.0005) [2023-12-27 14:58:12,629][20347] Updated weights for policy 0, policy_version 1345457 (0.0010) [2023-12-27 14:58:12,677][20381] Updated weights for policy 1, policy_version 1344332 (0.0007) [2023-12-27 14:58:13,284][20347] Updated weights for policy 0, policy_version 1345467 (0.0006) [2023-12-27 14:58:13,344][20347] Updated weights for policy 0, policy_version 1345477 (0.0005) [2023-12-27 14:58:13,382][20381] Updated weights for policy 1, policy_version 1344342 (0.0007) [2023-12-27 14:58:13,402][20347] Updated weights for policy 0, policy_version 1345487 (0.0006) [2023-12-27 14:58:13,442][20381] Updated weights for policy 1, policy_version 1344352 (0.0007) [2023-12-27 14:58:13,489][20381] Updated weights for policy 1, policy_version 1344362 (0.0006) [2023-12-27 14:58:13,970][20347] Updated weights for policy 0, policy_version 1345497 (0.0006) [2023-12-27 14:58:14,028][20347] Updated weights for policy 0, policy_version 1345507 (0.0010) [2023-12-27 14:58:14,038][20381] Updated weights for policy 1, policy_version 1344372 (0.0005) [2023-12-27 14:58:14,083][20347] Updated weights for policy 0, policy_version 1345517 (0.0010) [2023-12-27 14:58:14,093][20381] Updated weights for policy 1, policy_version 1344382 (0.0006) [2023-12-27 14:58:14,139][20347] Updated weights for policy 0, policy_version 1345527 (0.0010) [2023-12-27 14:58:14,141][20381] Updated weights for policy 1, policy_version 1344392 (0.0005) [2023-12-27 14:58:14,757][20381] Updated weights for policy 1, policy_version 1344402 (0.0007) [2023-12-27 14:58:14,817][20347] Updated weights for policy 0, policy_version 1345537 (0.0009) [2023-12-27 14:58:14,822][20381] Updated weights for policy 1, policy_version 1344412 (0.0007) [2023-12-27 14:58:14,878][20347] Updated weights for policy 0, policy_version 1345547 (0.0010) [2023-12-27 14:58:14,880][20381] Updated weights for policy 1, policy_version 1344422 (0.0006) [2023-12-27 14:58:14,939][20381] Updated weights for policy 1, policy_version 1344432 (0.0006) [2023-12-27 14:58:14,940][20347] Updated weights for policy 0, policy_version 1345557 (0.0010) [2023-12-27 14:58:15,577][20381] Updated weights for policy 1, policy_version 1344442 (0.0008) [2023-12-27 14:58:15,624][20381] Updated weights for policy 1, policy_version 1344452 (0.0005) [2023-12-27 14:58:15,626][20347] Updated weights for policy 0, policy_version 1345567 (0.0009) [2023-12-27 14:58:15,675][20347] Updated weights for policy 0, policy_version 1345577 (0.0010) [2023-12-27 14:58:15,677][20381] Updated weights for policy 1, policy_version 1344462 (0.0006) [2023-12-27 14:58:15,729][20347] Updated weights for policy 0, policy_version 1345587 (0.0010) [2023-12-27 14:58:16,305][20381] Updated weights for policy 1, policy_version 1344472 (0.0006) [2023-12-27 14:58:16,356][20381] Updated weights for policy 1, policy_version 1344482 (0.0007) [2023-12-27 14:58:16,398][20347] Updated weights for policy 0, policy_version 1345597 (0.0009) [2023-12-27 14:58:16,411][20381] Updated weights for policy 1, policy_version 1344492 (0.0006) [2023-12-27 14:58:16,454][20347] Updated weights for policy 0, policy_version 1345607 (0.0010) [2023-12-27 14:58:16,505][20347] Updated weights for policy 0, policy_version 1345617 (0.0010) [2023-12-27 14:58:17,088][20347] Updated weights for policy 0, policy_version 1345627 (0.0009) [2023-12-27 14:58:17,105][20381] Updated weights for policy 1, policy_version 1344502 (0.0006) [2023-12-27 14:58:17,137][20347] Updated weights for policy 0, policy_version 1345637 (0.0005) [2023-12-27 14:58:17,151][20381] Updated weights for policy 1, policy_version 1344512 (0.0005) [2023-12-27 14:58:17,186][20347] Updated weights for policy 0, policy_version 1345647 (0.0005) [2023-12-27 14:58:17,211][20381] Updated weights for policy 1, policy_version 1344522 (0.0006) [2023-12-27 14:58:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 688783360. Throughput: 0: 10391.6, 1: 10254.9. Samples: 688789072. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:17,363][19170] Avg episode reward: [(0, '784.340'), (1, '843.670')] [2023-12-27 14:58:17,866][20347] Updated weights for policy 0, policy_version 1345657 (0.0006) [2023-12-27 14:58:17,880][20381] Updated weights for policy 1, policy_version 1344532 (0.0006) [2023-12-27 14:58:17,918][20291] KL-divergence is very high: 103.7982 [2023-12-27 14:58:17,923][20347] Updated weights for policy 0, policy_version 1345667 (0.0007) [2023-12-27 14:58:17,929][20381] Updated weights for policy 1, policy_version 1344542 (0.0008) [2023-12-27 14:58:17,963][20291] KL-divergence is very high: 104.5000 [2023-12-27 14:58:17,976][20381] Updated weights for policy 1, policy_version 1344552 (0.0010) [2023-12-27 14:58:17,978][20347] Updated weights for policy 0, policy_version 1345677 (0.0007) [2023-12-27 14:58:18,031][20347] Updated weights for policy 0, policy_version 1345687 (0.0005) [2023-12-27 14:58:18,635][20347] Updated weights for policy 0, policy_version 1345697 (0.0008) [2023-12-27 14:58:18,635][20381] Updated weights for policy 1, policy_version 1344562 (0.0011) [2023-12-27 14:58:18,690][20381] Updated weights for policy 1, policy_version 1344572 (0.0006) [2023-12-27 14:58:18,691][20347] Updated weights for policy 0, policy_version 1345707 (0.0006) [2023-12-27 14:58:18,747][20381] Updated weights for policy 1, policy_version 1344582 (0.0006) [2023-12-27 14:58:18,748][20347] Updated weights for policy 0, policy_version 1345717 (0.0008) [2023-12-27 14:58:18,820][20381] Updated weights for policy 1, policy_version 1344592 (0.0008) [2023-12-27 14:58:19,423][20347] Updated weights for policy 0, policy_version 1345727 (0.0008) [2023-12-27 14:58:19,477][20347] Updated weights for policy 0, policy_version 1345737 (0.0011) [2023-12-27 14:58:19,521][20381] Updated weights for policy 1, policy_version 1344602 (0.0007) [2023-12-27 14:58:19,546][20347] Updated weights for policy 0, policy_version 1345747 (0.0011) [2023-12-27 14:58:19,586][20381] Updated weights for policy 1, policy_version 1344612 (0.0007) [2023-12-27 14:58:19,652][20381] Updated weights for policy 1, policy_version 1344622 (0.0007) [2023-12-27 14:58:20,258][20347] Updated weights for policy 0, policy_version 1345757 (0.0009) [2023-12-27 14:58:20,317][20347] Updated weights for policy 0, policy_version 1345767 (0.0007) [2023-12-27 14:58:20,319][20381] Updated weights for policy 1, policy_version 1344632 (0.0006) [2023-12-27 14:58:20,377][20347] Updated weights for policy 0, policy_version 1345777 (0.0007) [2023-12-27 14:58:20,378][20381] Updated weights for policy 1, policy_version 1344642 (0.0006) [2023-12-27 14:58:20,435][20381] Updated weights for policy 1, policy_version 1344652 (0.0006) [2023-12-27 14:58:21,081][20381] Updated weights for policy 1, policy_version 1344662 (0.0008) [2023-12-27 14:58:21,143][20347] Updated weights for policy 0, policy_version 1345787 (0.0009) [2023-12-27 14:58:21,148][20381] Updated weights for policy 1, policy_version 1344672 (0.0009) [2023-12-27 14:58:21,210][20347] Updated weights for policy 0, policy_version 1345797 (0.0009) [2023-12-27 14:58:21,212][20381] Updated weights for policy 1, policy_version 1344682 (0.0007) [2023-12-27 14:58:21,278][20347] Updated weights for policy 0, policy_version 1345807 (0.0008) [2023-12-27 14:58:21,938][20381] Updated weights for policy 1, policy_version 1344692 (0.0007) [2023-12-27 14:58:21,998][20381] Updated weights for policy 1, policy_version 1344702 (0.0006) [2023-12-27 14:58:22,066][20381] Updated weights for policy 1, policy_version 1344712 (0.0006) [2023-12-27 14:58:22,098][20347] Updated weights for policy 0, policy_version 1345817 (0.0008) [2023-12-27 14:58:22,156][20347] Updated weights for policy 0, policy_version 1345827 (0.0007) [2023-12-27 14:58:22,208][20347] Updated weights for policy 0, policy_version 1345837 (0.0009) [2023-12-27 14:58:22,262][20347] Updated weights for policy 0, policy_version 1345847 (0.0008) [2023-12-27 14:58:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 688881664. Throughput: 0: 10415.3, 1: 10207.9. Samples: 688850708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:22,363][19170] Avg episode reward: [(0, '784.450'), (1, '863.130')] [2023-12-27 14:58:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001344720_344276992.pth... [2023-12-27 14:58:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001345848_344604672.pth... [2023-12-27 14:58:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001343536_343973888.pth [2023-12-27 14:58:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001344632_344293376.pth [2023-12-27 14:58:22,766][20381] Updated weights for policy 1, policy_version 1344722 (0.0006) [2023-12-27 14:58:22,821][20381] Updated weights for policy 1, policy_version 1344732 (0.0008) [2023-12-27 14:58:22,874][20381] Updated weights for policy 1, policy_version 1344742 (0.0008) [2023-12-27 14:58:22,928][20381] Updated weights for policy 1, policy_version 1344752 (0.0008) [2023-12-27 14:58:23,023][20347] Updated weights for policy 0, policy_version 1345857 (0.0008) [2023-12-27 14:58:23,090][20347] Updated weights for policy 0, policy_version 1345867 (0.0008) [2023-12-27 14:58:23,150][20347] Updated weights for policy 0, policy_version 1345877 (0.0008) [2023-12-27 14:58:23,544][20381] Updated weights for policy 1, policy_version 1344762 (0.0008) [2023-12-27 14:58:23,590][20381] Updated weights for policy 1, policy_version 1344772 (0.0010) [2023-12-27 14:58:23,636][20381] Updated weights for policy 1, policy_version 1344782 (0.0010) [2023-12-27 14:58:23,796][20347] Updated weights for policy 0, policy_version 1345887 (0.0007) [2023-12-27 14:58:23,848][20347] Updated weights for policy 0, policy_version 1345897 (0.0009) [2023-12-27 14:58:23,899][20347] Updated weights for policy 0, policy_version 1345907 (0.0009) [2023-12-27 14:58:24,367][20381] Updated weights for policy 1, policy_version 1344792 (0.0008) [2023-12-27 14:58:24,413][20381] Updated weights for policy 1, policy_version 1344802 (0.0008) [2023-12-27 14:58:24,465][20381] Updated weights for policy 1, policy_version 1344812 (0.0007) [2023-12-27 14:58:24,573][20347] Updated weights for policy 0, policy_version 1345917 (0.0008) [2023-12-27 14:58:24,631][20347] Updated weights for policy 0, policy_version 1345927 (0.0008) [2023-12-27 14:58:24,680][20347] Updated weights for policy 0, policy_version 1345937 (0.0008) [2023-12-27 14:58:25,158][20381] Updated weights for policy 1, policy_version 1344822 (0.0007) [2023-12-27 14:58:25,221][20381] Updated weights for policy 1, policy_version 1344832 (0.0008) [2023-12-27 14:58:25,285][20381] Updated weights for policy 1, policy_version 1344842 (0.0007) [2023-12-27 14:58:25,346][20347] Updated weights for policy 0, policy_version 1345947 (0.0006) [2023-12-27 14:58:25,412][20347] Updated weights for policy 0, policy_version 1345957 (0.0005) [2023-12-27 14:58:25,478][20347] Updated weights for policy 0, policy_version 1345967 (0.0006) [2023-12-27 14:58:25,946][20381] Updated weights for policy 1, policy_version 1344852 (0.0010) [2023-12-27 14:58:26,002][20381] Updated weights for policy 1, policy_version 1344862 (0.0007) [2023-12-27 14:58:26,058][20381] Updated weights for policy 1, policy_version 1344872 (0.0009) [2023-12-27 14:58:26,142][20347] Updated weights for policy 0, policy_version 1345977 (0.0008) [2023-12-27 14:58:26,175][20291] KL-divergence is very high: 177.9173 [2023-12-27 14:58:26,190][20347] Updated weights for policy 0, policy_version 1345987 (0.0009) [2023-12-27 14:58:26,193][20291] KL-divergence is very high: 425.0136 [2023-12-27 14:58:26,203][20291] KL-divergence is very high: 375.7703 [2023-12-27 14:58:26,216][20291] KL-divergence is very high: 312.8246 [2023-12-27 14:58:26,239][20291] KL-divergence is very high: 168.2107 [2023-12-27 14:58:26,246][20347] Updated weights for policy 0, policy_version 1345997 (0.0009) [2023-12-27 14:58:26,298][20347] Updated weights for policy 0, policy_version 1346007 (0.0009) [2023-12-27 14:58:26,793][20381] Updated weights for policy 1, policy_version 1344882 (0.0009) [2023-12-27 14:58:26,850][20381] Updated weights for policy 1, policy_version 1344892 (0.0008) [2023-12-27 14:58:26,906][20381] Updated weights for policy 1, policy_version 1344902 (0.0007) [2023-12-27 14:58:26,964][20381] Updated weights for policy 1, policy_version 1344912 (0.0008) [2023-12-27 14:58:27,015][20347] Updated weights for policy 0, policy_version 1346017 (0.0007) [2023-12-27 14:58:27,069][20347] Updated weights for policy 0, policy_version 1346027 (0.0007) [2023-12-27 14:58:27,125][20347] Updated weights for policy 0, policy_version 1346037 (0.0008) [2023-12-27 14:58:27,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 688979968. Throughput: 0: 10327.3, 1: 10224.3. Samples: 688970092. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:27,364][19170] Avg episode reward: [(0, '814.610'), (1, '785.160')] [2023-12-27 14:58:27,569][20381] Updated weights for policy 1, policy_version 1344922 (0.0008) [2023-12-27 14:58:27,614][20381] Updated weights for policy 1, policy_version 1344932 (0.0007) [2023-12-27 14:58:27,663][20381] Updated weights for policy 1, policy_version 1344942 (0.0008) [2023-12-27 14:58:27,824][20347] Updated weights for policy 0, policy_version 1346047 (0.0008) [2023-12-27 14:58:27,876][20347] Updated weights for policy 0, policy_version 1346057 (0.0007) [2023-12-27 14:58:27,922][20347] Updated weights for policy 0, policy_version 1346067 (0.0008) [2023-12-27 14:58:28,363][20381] Updated weights for policy 1, policy_version 1344952 (0.0008) [2023-12-27 14:58:28,415][20381] Updated weights for policy 1, policy_version 1344962 (0.0008) [2023-12-27 14:58:28,459][20381] Updated weights for policy 1, policy_version 1344972 (0.0008) [2023-12-27 14:58:28,629][20347] Updated weights for policy 0, policy_version 1346077 (0.0007) [2023-12-27 14:58:28,689][20347] Updated weights for policy 0, policy_version 1346087 (0.0008) [2023-12-27 14:58:28,744][20347] Updated weights for policy 0, policy_version 1346097 (0.0008) [2023-12-27 14:58:29,162][20381] Updated weights for policy 1, policy_version 1344982 (0.0008) [2023-12-27 14:58:29,215][20381] Updated weights for policy 1, policy_version 1344992 (0.0008) [2023-12-27 14:58:29,272][20381] Updated weights for policy 1, policy_version 1345002 (0.0008) [2023-12-27 14:58:29,451][20347] Updated weights for policy 0, policy_version 1346107 (0.0008) [2023-12-27 14:58:29,493][20347] Updated weights for policy 0, policy_version 1346117 (0.0008) [2023-12-27 14:58:29,549][20347] Updated weights for policy 0, policy_version 1346127 (0.0008) [2023-12-27 14:58:30,015][20381] Updated weights for policy 1, policy_version 1345012 (0.0008) [2023-12-27 14:58:30,074][20381] Updated weights for policy 1, policy_version 1345022 (0.0009) [2023-12-27 14:58:30,130][20381] Updated weights for policy 1, policy_version 1345032 (0.0009) [2023-12-27 14:58:30,232][20347] Updated weights for policy 0, policy_version 1346137 (0.0007) [2023-12-27 14:58:30,290][20347] Updated weights for policy 0, policy_version 1346147 (0.0005) [2023-12-27 14:58:30,345][20347] Updated weights for policy 0, policy_version 1346157 (0.0008) [2023-12-27 14:58:30,406][20347] Updated weights for policy 0, policy_version 1346167 (0.0008) [2023-12-27 14:58:30,803][20381] Updated weights for policy 1, policy_version 1345042 (0.0008) [2023-12-27 14:58:30,851][20381] Updated weights for policy 1, policy_version 1345052 (0.0005) [2023-12-27 14:58:30,901][20381] Updated weights for policy 1, policy_version 1345062 (0.0006) [2023-12-27 14:58:30,950][20381] Updated weights for policy 1, policy_version 1345072 (0.0008) [2023-12-27 14:58:31,070][20347] Updated weights for policy 0, policy_version 1346177 (0.0008) [2023-12-27 14:58:31,133][20347] Updated weights for policy 0, policy_version 1346187 (0.0008) [2023-12-27 14:58:31,192][20347] Updated weights for policy 0, policy_version 1346197 (0.0005) [2023-12-27 14:58:31,670][20381] Updated weights for policy 1, policy_version 1345082 (0.0011) [2023-12-27 14:58:31,726][20381] Updated weights for policy 1, policy_version 1345092 (0.0011) [2023-12-27 14:58:31,798][20381] Updated weights for policy 1, policy_version 1345102 (0.0011) [2023-12-27 14:58:31,866][20347] Updated weights for policy 0, policy_version 1346207 (0.0006) [2023-12-27 14:58:31,913][20347] Updated weights for policy 0, policy_version 1346217 (0.0007) [2023-12-27 14:58:31,964][20347] Updated weights for policy 0, policy_version 1346227 (0.0007) [2023-12-27 14:58:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 689078272. Throughput: 0: 10284.2, 1: 10158.7. Samples: 689090696. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:32,363][19170] Avg episode reward: [(0, '814.950'), (1, '815.020')] [2023-12-27 14:58:32,485][20381] Updated weights for policy 1, policy_version 1345112 (0.0008) [2023-12-27 14:58:32,541][20381] Updated weights for policy 1, policy_version 1345122 (0.0008) [2023-12-27 14:58:32,604][20381] Updated weights for policy 1, policy_version 1345132 (0.0007) [2023-12-27 14:58:32,687][20347] Updated weights for policy 0, policy_version 1346237 (0.0007) [2023-12-27 14:58:32,747][20347] Updated weights for policy 0, policy_version 1346247 (0.0008) [2023-12-27 14:58:32,796][20347] Updated weights for policy 0, policy_version 1346257 (0.0008) [2023-12-27 14:58:33,231][20381] Updated weights for policy 1, policy_version 1345142 (0.0007) [2023-12-27 14:58:33,285][20381] Updated weights for policy 1, policy_version 1345152 (0.0006) [2023-12-27 14:58:33,342][20381] Updated weights for policy 1, policy_version 1345162 (0.0005) [2023-12-27 14:58:33,471][20347] Updated weights for policy 0, policy_version 1346267 (0.0007) [2023-12-27 14:58:33,534][20347] Updated weights for policy 0, policy_version 1346277 (0.0008) [2023-12-27 14:58:33,586][20347] Updated weights for policy 0, policy_version 1346287 (0.0007) [2023-12-27 14:58:33,892][20381] Updated weights for policy 1, policy_version 1345172 (0.0007) [2023-12-27 14:58:33,943][20381] Updated weights for policy 1, policy_version 1345182 (0.0008) [2023-12-27 14:58:33,998][20381] Updated weights for policy 1, policy_version 1345192 (0.0005) [2023-12-27 14:58:34,200][20347] Updated weights for policy 0, policy_version 1346297 (0.0008) [2023-12-27 14:58:34,255][20347] Updated weights for policy 0, policy_version 1346307 (0.0008) [2023-12-27 14:58:34,315][20347] Updated weights for policy 0, policy_version 1346317 (0.0008) [2023-12-27 14:58:34,375][20347] Updated weights for policy 0, policy_version 1346327 (0.0008) [2023-12-27 14:58:34,617][20381] Updated weights for policy 1, policy_version 1345202 (0.0005) [2023-12-27 14:58:34,682][20381] Updated weights for policy 1, policy_version 1345212 (0.0005) [2023-12-27 14:58:34,745][20381] Updated weights for policy 1, policy_version 1345222 (0.0006) [2023-12-27 14:58:34,810][20381] Updated weights for policy 1, policy_version 1345232 (0.0006) [2023-12-27 14:58:35,119][20347] Updated weights for policy 0, policy_version 1346337 (0.0008) [2023-12-27 14:58:35,174][20347] Updated weights for policy 0, policy_version 1346347 (0.0007) [2023-12-27 14:58:35,229][20347] Updated weights for policy 0, policy_version 1346357 (0.0008) [2023-12-27 14:58:35,362][20381] Updated weights for policy 1, policy_version 1345242 (0.0007) [2023-12-27 14:58:35,418][20381] Updated weights for policy 1, policy_version 1345252 (0.0008) [2023-12-27 14:58:35,477][20381] Updated weights for policy 1, policy_version 1345262 (0.0008) [2023-12-27 14:58:35,921][20347] Updated weights for policy 0, policy_version 1346367 (0.0006) [2023-12-27 14:58:35,974][20347] Updated weights for policy 0, policy_version 1346377 (0.0007) [2023-12-27 14:58:36,024][20347] Updated weights for policy 0, policy_version 1346387 (0.0008) [2023-12-27 14:58:36,184][20381] Updated weights for policy 1, policy_version 1345272 (0.0010) [2023-12-27 14:58:36,255][20381] Updated weights for policy 1, policy_version 1345283 (0.0009) [2023-12-27 14:58:36,314][20381] Updated weights for policy 1, policy_version 1345293 (0.0011) [2023-12-27 14:58:36,779][20347] Updated weights for policy 0, policy_version 1346397 (0.0006) [2023-12-27 14:58:36,836][20347] Updated weights for policy 0, policy_version 1346407 (0.0005) [2023-12-27 14:58:36,837][20291] KL-divergence is very high: 248.4416 [2023-12-27 14:58:36,879][20291] KL-divergence is very high: 302.1426 [2023-12-27 14:58:36,892][20347] Updated weights for policy 0, policy_version 1346417 (0.0006) [2023-12-27 14:58:36,919][20291] KL-divergence is very high: 147.2133 [2023-12-27 14:58:36,988][20381] Updated weights for policy 1, policy_version 1345303 (0.0011) [2023-12-27 14:58:37,045][20381] Updated weights for policy 1, policy_version 1345313 (0.0008) [2023-12-27 14:58:37,103][20381] Updated weights for policy 1, policy_version 1345323 (0.0008) [2023-12-27 14:58:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 689184768. Throughput: 0: 10271.4, 1: 10203.1. Samples: 689155148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:37,363][19170] Avg episode reward: [(0, '814.270'), (1, '853.770')] [2023-12-27 14:58:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001345328_344432640.pth... [2023-12-27 14:58:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001346424_344752128.pth... [2023-12-27 14:58:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001344112_344121344.pth [2023-12-27 14:58:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001345240_344449024.pth [2023-12-27 14:58:37,566][20347] Updated weights for policy 0, policy_version 1346427 (0.0006) [2023-12-27 14:58:37,623][20347] Updated weights for policy 0, policy_version 1346437 (0.0007) [2023-12-27 14:58:37,678][20347] Updated weights for policy 0, policy_version 1346447 (0.0006) [2023-12-27 14:58:37,750][20381] Updated weights for policy 1, policy_version 1345333 (0.0008) [2023-12-27 14:58:37,806][20381] Updated weights for policy 1, policy_version 1345343 (0.0008) [2023-12-27 14:58:37,872][20381] Updated weights for policy 1, policy_version 1345353 (0.0008) [2023-12-27 14:58:38,324][20347] Updated weights for policy 0, policy_version 1346457 (0.0009) [2023-12-27 14:58:38,385][20347] Updated weights for policy 0, policy_version 1346467 (0.0008) [2023-12-27 14:58:38,446][20347] Updated weights for policy 0, policy_version 1346477 (0.0008) [2023-12-27 14:58:38,506][20347] Updated weights for policy 0, policy_version 1346487 (0.0008) [2023-12-27 14:58:38,562][20381] Updated weights for policy 1, policy_version 1345363 (0.0007) [2023-12-27 14:58:38,622][20381] Updated weights for policy 1, policy_version 1345373 (0.0008) [2023-12-27 14:58:38,684][20381] Updated weights for policy 1, policy_version 1345383 (0.0007) [2023-12-27 14:58:39,207][20347] Updated weights for policy 0, policy_version 1346497 (0.0007) [2023-12-27 14:58:39,274][20347] Updated weights for policy 0, policy_version 1346507 (0.0008) [2023-12-27 14:58:39,336][20347] Updated weights for policy 0, policy_version 1346517 (0.0008) [2023-12-27 14:58:39,385][20381] Updated weights for policy 1, policy_version 1345393 (0.0007) [2023-12-27 14:58:39,448][20381] Updated weights for policy 1, policy_version 1345403 (0.0007) [2023-12-27 14:58:39,511][20381] Updated weights for policy 1, policy_version 1345413 (0.0006) [2023-12-27 14:58:39,572][20381] Updated weights for policy 1, policy_version 1345423 (0.0007) [2023-12-27 14:58:40,097][20347] Updated weights for policy 0, policy_version 1346527 (0.0006) [2023-12-27 14:58:40,158][20347] Updated weights for policy 0, policy_version 1346537 (0.0006) [2023-12-27 14:58:40,220][20347] Updated weights for policy 0, policy_version 1346547 (0.0006) [2023-12-27 14:58:40,320][20381] Updated weights for policy 1, policy_version 1345433 (0.0008) [2023-12-27 14:58:40,368][20381] Updated weights for policy 1, policy_version 1345443 (0.0008) [2023-12-27 14:58:40,432][20381] Updated weights for policy 1, policy_version 1345453 (0.0008) [2023-12-27 14:58:40,810][20347] Updated weights for policy 0, policy_version 1346557 (0.0005) [2023-12-27 14:58:40,871][20347] Updated weights for policy 0, policy_version 1346567 (0.0007) [2023-12-27 14:58:40,930][20347] Updated weights for policy 0, policy_version 1346577 (0.0007) [2023-12-27 14:58:41,124][20381] Updated weights for policy 1, policy_version 1345463 (0.0007) [2023-12-27 14:58:41,189][20381] Updated weights for policy 1, policy_version 1345473 (0.0008) [2023-12-27 14:58:41,249][20381] Updated weights for policy 1, policy_version 1345483 (0.0007) [2023-12-27 14:58:41,678][20347] Updated weights for policy 0, policy_version 1346587 (0.0009) [2023-12-27 14:58:41,746][20347] Updated weights for policy 0, policy_version 1346597 (0.0008) [2023-12-27 14:58:41,754][20291] KL-divergence is very high: 106.7967 [2023-12-27 14:58:41,780][20291] KL-divergence is very high: 106.3383 [2023-12-27 14:58:41,807][20291] KL-divergence is very high: 108.9727 [2023-12-27 14:58:41,814][20347] Updated weights for policy 0, policy_version 1346607 (0.0008) [2023-12-27 14:58:41,900][20381] Updated weights for policy 1, policy_version 1345493 (0.0008) [2023-12-27 14:58:41,953][20381] Updated weights for policy 1, policy_version 1345503 (0.0008) [2023-12-27 14:58:42,003][20381] Updated weights for policy 1, policy_version 1345513 (0.0008) [2023-12-27 14:58:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 689283072. Throughput: 0: 10228.3, 1: 10192.7. Samples: 689275140. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:42,363][19170] Avg episode reward: [(0, '834.010'), (1, '892.280')] [2023-12-27 14:58:42,545][20347] Updated weights for policy 0, policy_version 1346617 (0.0008) [2023-12-27 14:58:42,606][20347] Updated weights for policy 0, policy_version 1346627 (0.0010) [2023-12-27 14:58:42,668][20347] Updated weights for policy 0, policy_version 1346637 (0.0010) [2023-12-27 14:58:42,726][20381] Updated weights for policy 1, policy_version 1345523 (0.0008) [2023-12-27 14:58:42,726][20347] Updated weights for policy 0, policy_version 1346647 (0.0010) [2023-12-27 14:58:42,782][20381] Updated weights for policy 1, policy_version 1345533 (0.0011) [2023-12-27 14:58:42,846][20381] Updated weights for policy 1, policy_version 1345543 (0.0006) [2023-12-27 14:58:43,320][20347] Updated weights for policy 0, policy_version 1346657 (0.0008) [2023-12-27 14:58:43,365][20347] Updated weights for policy 0, policy_version 1346667 (0.0007) [2023-12-27 14:58:43,412][20347] Updated weights for policy 0, policy_version 1346677 (0.0006) [2023-12-27 14:58:43,492][20381] Updated weights for policy 1, policy_version 1345553 (0.0006) [2023-12-27 14:58:43,555][20381] Updated weights for policy 1, policy_version 1345563 (0.0009) [2023-12-27 14:58:43,608][20381] Updated weights for policy 1, policy_version 1345573 (0.0007) [2023-12-27 14:58:43,663][20381] Updated weights for policy 1, policy_version 1345583 (0.0008) [2023-12-27 14:58:44,049][20347] Updated weights for policy 0, policy_version 1346687 (0.0009) [2023-12-27 14:58:44,104][20347] Updated weights for policy 0, policy_version 1346697 (0.0010) [2023-12-27 14:58:44,170][20347] Updated weights for policy 0, policy_version 1346707 (0.0010) [2023-12-27 14:58:44,359][20381] Updated weights for policy 1, policy_version 1345593 (0.0007) [2023-12-27 14:58:44,418][20381] Updated weights for policy 1, policy_version 1345603 (0.0008) [2023-12-27 14:58:44,463][20381] Updated weights for policy 1, policy_version 1345613 (0.0007) [2023-12-27 14:58:44,850][20347] Updated weights for policy 0, policy_version 1346717 (0.0011) [2023-12-27 14:58:44,900][20347] Updated weights for policy 0, policy_version 1346727 (0.0010) [2023-12-27 14:58:44,953][20347] Updated weights for policy 0, policy_version 1346737 (0.0010) [2023-12-27 14:58:45,237][20381] Updated weights for policy 1, policy_version 1345623 (0.0007) [2023-12-27 14:58:45,287][20381] Updated weights for policy 1, policy_version 1345633 (0.0008) [2023-12-27 14:58:45,344][20381] Updated weights for policy 1, policy_version 1345643 (0.0008) [2023-12-27 14:58:45,643][20347] Updated weights for policy 0, policy_version 1346747 (0.0009) [2023-12-27 14:58:45,703][20347] Updated weights for policy 0, policy_version 1346757 (0.0008) [2023-12-27 14:58:45,757][20347] Updated weights for policy 0, policy_version 1346767 (0.0007) [2023-12-27 14:58:45,939][20381] Updated weights for policy 1, policy_version 1345653 (0.0008) [2023-12-27 14:58:45,988][20381] Updated weights for policy 1, policy_version 1345663 (0.0007) [2023-12-27 14:58:46,036][20381] Updated weights for policy 1, policy_version 1345673 (0.0008) [2023-12-27 14:58:46,372][20347] Updated weights for policy 0, policy_version 1346777 (0.0007) [2023-12-27 14:58:46,429][20347] Updated weights for policy 0, policy_version 1346787 (0.0005) [2023-12-27 14:58:46,481][20347] Updated weights for policy 0, policy_version 1346797 (0.0006) [2023-12-27 14:58:46,538][20347] Updated weights for policy 0, policy_version 1346807 (0.0005) [2023-12-27 14:58:46,696][20381] Updated weights for policy 1, policy_version 1345683 (0.0006) [2023-12-27 14:58:46,752][20381] Updated weights for policy 1, policy_version 1345693 (0.0005) [2023-12-27 14:58:46,800][20381] Updated weights for policy 1, policy_version 1345703 (0.0005) [2023-12-27 14:58:47,059][20347] Updated weights for policy 0, policy_version 1346817 (0.0007) [2023-12-27 14:58:47,108][20347] Updated weights for policy 0, policy_version 1346827 (0.0007) [2023-12-27 14:58:47,161][20347] Updated weights for policy 0, policy_version 1346837 (0.0008) [2023-12-27 14:58:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 689389568. Throughput: 0: 10266.5, 1: 10246.3. Samples: 689399724. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:47,364][19170] Avg episode reward: [(0, '872.880'), (1, '902.030')] [2023-12-27 14:58:47,434][20381] Updated weights for policy 1, policy_version 1345713 (0.0007) [2023-12-27 14:58:47,490][20381] Updated weights for policy 1, policy_version 1345723 (0.0006) [2023-12-27 14:58:47,540][20381] Updated weights for policy 1, policy_version 1345733 (0.0011) [2023-12-27 14:58:47,601][20381] Updated weights for policy 1, policy_version 1345743 (0.0011) [2023-12-27 14:58:47,852][20347] Updated weights for policy 0, policy_version 1346847 (0.0006) [2023-12-27 14:58:47,907][20347] Updated weights for policy 0, policy_version 1346857 (0.0006) [2023-12-27 14:58:47,960][20347] Updated weights for policy 0, policy_version 1346867 (0.0008) [2023-12-27 14:58:48,256][20381] Updated weights for policy 1, policy_version 1345753 (0.0010) [2023-12-27 14:58:48,317][20381] Updated weights for policy 1, policy_version 1345763 (0.0010) [2023-12-27 14:58:48,374][20381] Updated weights for policy 1, policy_version 1345773 (0.0008) [2023-12-27 14:58:48,666][20347] Updated weights for policy 0, policy_version 1346877 (0.0008) [2023-12-27 14:58:48,721][20347] Updated weights for policy 0, policy_version 1346887 (0.0007) [2023-12-27 14:58:48,770][20347] Updated weights for policy 0, policy_version 1346897 (0.0007) [2023-12-27 14:58:49,050][20381] Updated weights for policy 1, policy_version 1345783 (0.0006) [2023-12-27 14:58:49,102][20381] Updated weights for policy 1, policy_version 1345793 (0.0006) [2023-12-27 14:58:49,152][20381] Updated weights for policy 1, policy_version 1345803 (0.0006) [2023-12-27 14:58:49,530][20347] Updated weights for policy 0, policy_version 1346907 (0.0008) [2023-12-27 14:58:49,596][20347] Updated weights for policy 0, policy_version 1346917 (0.0008) [2023-12-27 14:58:49,663][20347] Updated weights for policy 0, policy_version 1346927 (0.0008) [2023-12-27 14:58:49,782][20381] Updated weights for policy 1, policy_version 1345813 (0.0007) [2023-12-27 14:58:49,848][20381] Updated weights for policy 1, policy_version 1345823 (0.0008) [2023-12-27 14:58:49,913][20381] Updated weights for policy 1, policy_version 1345833 (0.0008) [2023-12-27 14:58:50,377][20347] Updated weights for policy 0, policy_version 1346937 (0.0008) [2023-12-27 14:58:50,435][20347] Updated weights for policy 0, policy_version 1346947 (0.0007) [2023-12-27 14:58:50,488][20347] Updated weights for policy 0, policy_version 1346957 (0.0008) [2023-12-27 14:58:50,541][20347] Updated weights for policy 0, policy_version 1346967 (0.0007) [2023-12-27 14:58:50,609][20381] Updated weights for policy 1, policy_version 1345843 (0.0007) [2023-12-27 14:58:50,669][20381] Updated weights for policy 1, policy_version 1345853 (0.0008) [2023-12-27 14:58:50,726][20381] Updated weights for policy 1, policy_version 1345863 (0.0008) [2023-12-27 14:58:51,335][20347] Updated weights for policy 0, policy_version 1346977 (0.0008) [2023-12-27 14:58:51,401][20347] Updated weights for policy 0, policy_version 1346987 (0.0008) [2023-12-27 14:58:51,431][20381] Updated weights for policy 1, policy_version 1345873 (0.0008) [2023-12-27 14:58:51,466][20347] Updated weights for policy 0, policy_version 1346997 (0.0008) [2023-12-27 14:58:51,487][20381] Updated weights for policy 1, policy_version 1345883 (0.0006) [2023-12-27 14:58:51,549][20381] Updated weights for policy 1, policy_version 1345893 (0.0007) [2023-12-27 14:58:51,606][20381] Updated weights for policy 1, policy_version 1345903 (0.0008) [2023-12-27 14:58:52,158][20347] Updated weights for policy 0, policy_version 1347007 (0.0008) [2023-12-27 14:58:52,222][20347] Updated weights for policy 0, policy_version 1347017 (0.0008) [2023-12-27 14:58:52,284][20347] Updated weights for policy 0, policy_version 1347027 (0.0008) [2023-12-27 14:58:52,329][20381] Updated weights for policy 1, policy_version 1345913 (0.0006) [2023-12-27 14:58:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 689487872. Throughput: 0: 10197.3, 1: 10267.2. Samples: 689460096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:52,363][19170] Avg episode reward: [(0, '785.030'), (1, '843.430')] [2023-12-27 14:58:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001347032_344907776.pth... [2023-12-27 14:58:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001345848_344604672.pth [2023-12-27 14:58:52,389][20381] Updated weights for policy 1, policy_version 1345923 (0.0007) [2023-12-27 14:58:52,443][20381] Updated weights for policy 1, policy_version 1345933 (0.0008) [2023-12-27 14:58:52,458][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001345936_344588288.pth... [2023-12-27 14:58:52,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001344720_344276992.pth [2023-12-27 14:58:52,967][20347] Updated weights for policy 0, policy_version 1347037 (0.0007) [2023-12-27 14:58:53,024][20347] Updated weights for policy 0, policy_version 1347047 (0.0008) [2023-12-27 14:58:53,070][20347] Updated weights for policy 0, policy_version 1347057 (0.0007) [2023-12-27 14:58:53,157][20381] Updated weights for policy 1, policy_version 1345943 (0.0007) [2023-12-27 14:58:53,217][20381] Updated weights for policy 1, policy_version 1345953 (0.0008) [2023-12-27 14:58:53,273][20381] Updated weights for policy 1, policy_version 1345963 (0.0007) [2023-12-27 14:58:53,714][20347] Updated weights for policy 0, policy_version 1347067 (0.0008) [2023-12-27 14:58:53,771][20347] Updated weights for policy 0, policy_version 1347077 (0.0007) [2023-12-27 14:58:53,826][20347] Updated weights for policy 0, policy_version 1347087 (0.0008) [2023-12-27 14:58:53,966][20381] Updated weights for policy 1, policy_version 1345973 (0.0007) [2023-12-27 14:58:54,023][20381] Updated weights for policy 1, policy_version 1345983 (0.0008) [2023-12-27 14:58:54,082][20381] Updated weights for policy 1, policy_version 1345993 (0.0008) [2023-12-27 14:58:54,498][20347] Updated weights for policy 0, policy_version 1347097 (0.0007) [2023-12-27 14:58:54,551][20347] Updated weights for policy 0, policy_version 1347107 (0.0006) [2023-12-27 14:58:54,596][20347] Updated weights for policy 0, policy_version 1347117 (0.0008) [2023-12-27 14:58:54,640][20347] Updated weights for policy 0, policy_version 1347127 (0.0006) [2023-12-27 14:58:54,767][20381] Updated weights for policy 1, policy_version 1346003 (0.0008) [2023-12-27 14:58:54,827][20381] Updated weights for policy 1, policy_version 1346013 (0.0007) [2023-12-27 14:58:54,880][20381] Updated weights for policy 1, policy_version 1346023 (0.0008) [2023-12-27 14:58:55,244][20347] Updated weights for policy 0, policy_version 1347137 (0.0005) [2023-12-27 14:58:55,297][20347] Updated weights for policy 0, policy_version 1347147 (0.0005) [2023-12-27 14:58:55,342][20347] Updated weights for policy 0, policy_version 1347157 (0.0005) [2023-12-27 14:58:55,611][20381] Updated weights for policy 1, policy_version 1346033 (0.0008) [2023-12-27 14:58:55,661][20381] Updated weights for policy 1, policy_version 1346043 (0.0009) [2023-12-27 14:58:55,718][20381] Updated weights for policy 1, policy_version 1346053 (0.0009) [2023-12-27 14:58:55,767][20381] Updated weights for policy 1, policy_version 1346063 (0.0010) [2023-12-27 14:58:55,883][20347] Updated weights for policy 0, policy_version 1347167 (0.0007) [2023-12-27 14:58:55,939][20347] Updated weights for policy 0, policy_version 1347177 (0.0007) [2023-12-27 14:58:55,996][20347] Updated weights for policy 0, policy_version 1347187 (0.0007) [2023-12-27 14:58:56,459][20381] Updated weights for policy 1, policy_version 1346073 (0.0007) [2023-12-27 14:58:56,515][20381] Updated weights for policy 1, policy_version 1346083 (0.0007) [2023-12-27 14:58:56,568][20381] Updated weights for policy 1, policy_version 1346093 (0.0008) [2023-12-27 14:58:56,645][20347] Updated weights for policy 0, policy_version 1347197 (0.0006) [2023-12-27 14:58:56,706][20347] Updated weights for policy 0, policy_version 1347207 (0.0008) [2023-12-27 14:58:56,769][20347] Updated weights for policy 0, policy_version 1347217 (0.0008) [2023-12-27 14:58:57,232][20381] Updated weights for policy 1, policy_version 1346103 (0.0006) [2023-12-27 14:58:57,278][20381] Updated weights for policy 1, policy_version 1346113 (0.0005) [2023-12-27 14:58:57,318][20347] Updated weights for policy 0, policy_version 1347227 (0.0007) [2023-12-27 14:58:57,333][20381] Updated weights for policy 1, policy_version 1346123 (0.0007) [2023-12-27 14:58:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 689594368. Throughput: 0: 10189.1, 1: 10215.0. Samples: 689582012. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:58:57,363][19170] Avg episode reward: [(0, '814.380'), (1, '853.060')] [2023-12-27 14:58:57,377][20347] Updated weights for policy 0, policy_version 1347237 (0.0010) [2023-12-27 14:58:57,427][20347] Updated weights for policy 0, policy_version 1347247 (0.0009) [2023-12-27 14:58:57,958][20381] Updated weights for policy 1, policy_version 1346133 (0.0006) [2023-12-27 14:58:58,018][20381] Updated weights for policy 1, policy_version 1346143 (0.0005) [2023-12-27 14:58:58,039][20347] Updated weights for policy 0, policy_version 1347257 (0.0010) [2023-12-27 14:58:58,069][20381] Updated weights for policy 1, policy_version 1346153 (0.0005) [2023-12-27 14:58:58,093][20347] Updated weights for policy 0, policy_version 1347267 (0.0010) [2023-12-27 14:58:58,150][20347] Updated weights for policy 0, policy_version 1347277 (0.0010) [2023-12-27 14:58:58,199][20347] Updated weights for policy 0, policy_version 1347287 (0.0010) [2023-12-27 14:58:58,725][20381] Updated weights for policy 1, policy_version 1346163 (0.0006) [2023-12-27 14:58:58,793][20381] Updated weights for policy 1, policy_version 1346173 (0.0008) [2023-12-27 14:58:58,863][20381] Updated weights for policy 1, policy_version 1346183 (0.0007) [2023-12-27 14:58:58,960][20347] Updated weights for policy 0, policy_version 1347297 (0.0008) [2023-12-27 14:58:59,025][20347] Updated weights for policy 0, policy_version 1347307 (0.0008) [2023-12-27 14:58:59,086][20347] Updated weights for policy 0, policy_version 1347317 (0.0008) [2023-12-27 14:58:59,629][20381] Updated weights for policy 1, policy_version 1346193 (0.0007) [2023-12-27 14:58:59,693][20381] Updated weights for policy 1, policy_version 1346203 (0.0005) [2023-12-27 14:58:59,757][20381] Updated weights for policy 1, policy_version 1346213 (0.0006) [2023-12-27 14:58:59,780][20347] Updated weights for policy 0, policy_version 1347327 (0.0009) [2023-12-27 14:58:59,814][20381] Updated weights for policy 1, policy_version 1346223 (0.0007) [2023-12-27 14:58:59,833][20347] Updated weights for policy 0, policy_version 1347337 (0.0010) [2023-12-27 14:58:59,887][20347] Updated weights for policy 0, policy_version 1347347 (0.0010) [2023-12-27 14:59:00,478][20381] Updated weights for policy 1, policy_version 1346233 (0.0008) [2023-12-27 14:59:00,524][20381] Updated weights for policy 1, policy_version 1346243 (0.0008) [2023-12-27 14:59:00,561][20347] Updated weights for policy 0, policy_version 1347357 (0.0008) [2023-12-27 14:59:00,577][20381] Updated weights for policy 1, policy_version 1346253 (0.0008) [2023-12-27 14:59:00,619][20347] Updated weights for policy 0, policy_version 1347367 (0.0010) [2023-12-27 14:59:00,631][20291] KL-divergence is very high: 245.4830 [2023-12-27 14:59:00,674][20347] Updated weights for policy 0, policy_version 1347377 (0.0009) [2023-12-27 14:59:00,674][20291] KL-divergence is very high: 211.2877 [2023-12-27 14:59:01,220][20381] Updated weights for policy 1, policy_version 1346263 (0.0008) [2023-12-27 14:59:01,253][20347] Updated weights for policy 0, policy_version 1347387 (0.0009) [2023-12-27 14:59:01,278][20381] Updated weights for policy 1, policy_version 1346273 (0.0009) [2023-12-27 14:59:01,313][20347] Updated weights for policy 0, policy_version 1347397 (0.0007) [2023-12-27 14:59:01,332][20381] Updated weights for policy 1, policy_version 1346283 (0.0010) [2023-12-27 14:59:01,374][20347] Updated weights for policy 0, policy_version 1347407 (0.0007) [2023-12-27 14:59:01,984][20381] Updated weights for policy 1, policy_version 1346293 (0.0008) [2023-12-27 14:59:02,044][20381] Updated weights for policy 1, policy_version 1346303 (0.0006) [2023-12-27 14:59:02,099][20347] Updated weights for policy 0, policy_version 1347417 (0.0007) [2023-12-27 14:59:02,105][20381] Updated weights for policy 1, policy_version 1346313 (0.0006) [2023-12-27 14:59:02,150][20347] Updated weights for policy 0, policy_version 1347427 (0.0008) [2023-12-27 14:59:02,205][20347] Updated weights for policy 0, policy_version 1347437 (0.0010) [2023-12-27 14:59:02,267][20347] Updated weights for policy 0, policy_version 1347447 (0.0008) [2023-12-27 14:59:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 689700864. Throughput: 0: 10180.3, 1: 10215.8. Samples: 689706896. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:02,363][19170] Avg episode reward: [(0, '795.110'), (1, '735.560')] [2023-12-27 14:59:02,790][20381] Updated weights for policy 1, policy_version 1346323 (0.0006) [2023-12-27 14:59:02,843][20381] Updated weights for policy 1, policy_version 1346333 (0.0006) [2023-12-27 14:59:02,898][20381] Updated weights for policy 1, policy_version 1346343 (0.0006) [2023-12-27 14:59:02,944][20347] Updated weights for policy 0, policy_version 1347457 (0.0006) [2023-12-27 14:59:03,003][20347] Updated weights for policy 0, policy_version 1347467 (0.0006) [2023-12-27 14:59:03,058][20347] Updated weights for policy 0, policy_version 1347477 (0.0007) [2023-12-27 14:59:03,461][20381] Updated weights for policy 1, policy_version 1346353 (0.0006) [2023-12-27 14:59:03,516][20381] Updated weights for policy 1, policy_version 1346363 (0.0005) [2023-12-27 14:59:03,561][20381] Updated weights for policy 1, policy_version 1346373 (0.0005) [2023-12-27 14:59:03,611][20381] Updated weights for policy 1, policy_version 1346383 (0.0005) [2023-12-27 14:59:03,648][20347] Updated weights for policy 0, policy_version 1347487 (0.0008) [2023-12-27 14:59:03,707][20347] Updated weights for policy 0, policy_version 1347497 (0.0008) [2023-12-27 14:59:03,761][20347] Updated weights for policy 0, policy_version 1347507 (0.0007) [2023-12-27 14:59:04,228][20381] Updated weights for policy 1, policy_version 1346393 (0.0010) [2023-12-27 14:59:04,287][20381] Updated weights for policy 1, policy_version 1346403 (0.0011) [2023-12-27 14:59:04,348][20381] Updated weights for policy 1, policy_version 1346413 (0.0010) [2023-12-27 14:59:04,487][20347] Updated weights for policy 0, policy_version 1347517 (0.0007) [2023-12-27 14:59:04,543][20347] Updated weights for policy 0, policy_version 1347527 (0.0008) [2023-12-27 14:59:04,601][20347] Updated weights for policy 0, policy_version 1347537 (0.0007) [2023-12-27 14:59:04,997][20381] Updated weights for policy 1, policy_version 1346423 (0.0008) [2023-12-27 14:59:05,051][20381] Updated weights for policy 1, policy_version 1346433 (0.0006) [2023-12-27 14:59:05,099][20381] Updated weights for policy 1, policy_version 1346443 (0.0006) [2023-12-27 14:59:05,270][20347] Updated weights for policy 0, policy_version 1347547 (0.0007) [2023-12-27 14:59:05,316][20347] Updated weights for policy 0, policy_version 1347557 (0.0008) [2023-12-27 14:59:05,365][20347] Updated weights for policy 0, policy_version 1347567 (0.0007) [2023-12-27 14:59:05,727][20381] Updated weights for policy 1, policy_version 1346453 (0.0006) [2023-12-27 14:59:05,792][20381] Updated weights for policy 1, policy_version 1346463 (0.0006) [2023-12-27 14:59:05,851][20381] Updated weights for policy 1, policy_version 1346473 (0.0006) [2023-12-27 14:59:06,000][20347] Updated weights for policy 0, policy_version 1347577 (0.0007) [2023-12-27 14:59:06,050][20347] Updated weights for policy 0, policy_version 1347587 (0.0006) [2023-12-27 14:59:06,105][20347] Updated weights for policy 0, policy_version 1347597 (0.0006) [2023-12-27 14:59:06,166][20347] Updated weights for policy 0, policy_version 1347607 (0.0007) [2023-12-27 14:59:06,441][20381] Updated weights for policy 1, policy_version 1346483 (0.0006) [2023-12-27 14:59:06,504][20381] Updated weights for policy 1, policy_version 1346493 (0.0007) [2023-12-27 14:59:06,569][20381] Updated weights for policy 1, policy_version 1346503 (0.0007) [2023-12-27 14:59:06,828][20347] Updated weights for policy 0, policy_version 1347617 (0.0006) [2023-12-27 14:59:06,883][20347] Updated weights for policy 0, policy_version 1347627 (0.0005) [2023-12-27 14:59:06,935][20347] Updated weights for policy 0, policy_version 1347637 (0.0005) [2023-12-27 14:59:07,234][20381] Updated weights for policy 1, policy_version 1346513 (0.0007) [2023-12-27 14:59:07,291][20381] Updated weights for policy 1, policy_version 1346523 (0.0005) [2023-12-27 14:59:07,348][20381] Updated weights for policy 1, policy_version 1346533 (0.0005) [2023-12-27 14:59:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 689799168. Throughput: 0: 10184.5, 1: 10270.0. Samples: 689771156. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:07,363][19170] Avg episode reward: [(0, '824.400'), (1, '726.510')] [2023-12-27 14:59:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001347640_345063424.pth... [2023-12-27 14:59:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001346424_344752128.pth [2023-12-27 14:59:07,400][20381] Updated weights for policy 1, policy_version 1346543 (0.0007) [2023-12-27 14:59:07,405][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001346544_344743936.pth... [2023-12-27 14:59:07,408][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001345328_344432640.pth [2023-12-27 14:59:07,577][20347] Updated weights for policy 0, policy_version 1347647 (0.0007) [2023-12-27 14:59:07,635][20347] Updated weights for policy 0, policy_version 1347657 (0.0007) [2023-12-27 14:59:07,688][20347] Updated weights for policy 0, policy_version 1347667 (0.0008) [2023-12-27 14:59:07,966][20381] Updated weights for policy 1, policy_version 1346553 (0.0008) [2023-12-27 14:59:08,020][20381] Updated weights for policy 1, policy_version 1346563 (0.0008) [2023-12-27 14:59:08,075][20381] Updated weights for policy 1, policy_version 1346573 (0.0009) [2023-12-27 14:59:08,327][20347] Updated weights for policy 0, policy_version 1347677 (0.0007) [2023-12-27 14:59:08,395][20347] Updated weights for policy 0, policy_version 1347687 (0.0008) [2023-12-27 14:59:08,459][20347] Updated weights for policy 0, policy_version 1347697 (0.0008) [2023-12-27 14:59:08,780][20381] Updated weights for policy 1, policy_version 1346583 (0.0010) [2023-12-27 14:59:08,841][20381] Updated weights for policy 1, policy_version 1346593 (0.0009) [2023-12-27 14:59:08,903][20381] Updated weights for policy 1, policy_version 1346603 (0.0007) [2023-12-27 14:59:09,169][20347] Updated weights for policy 0, policy_version 1347707 (0.0007) [2023-12-27 14:59:09,229][20347] Updated weights for policy 0, policy_version 1347717 (0.0008) [2023-12-27 14:59:09,288][20347] Updated weights for policy 0, policy_version 1347727 (0.0008) [2023-12-27 14:59:09,572][20381] Updated weights for policy 1, policy_version 1346613 (0.0008) [2023-12-27 14:59:09,637][20381] Updated weights for policy 1, policy_version 1346623 (0.0009) [2023-12-27 14:59:09,703][20381] Updated weights for policy 1, policy_version 1346633 (0.0008) [2023-12-27 14:59:09,998][20347] Updated weights for policy 0, policy_version 1347737 (0.0007) [2023-12-27 14:59:10,049][20347] Updated weights for policy 0, policy_version 1347747 (0.0007) [2023-12-27 14:59:10,106][20347] Updated weights for policy 0, policy_version 1347757 (0.0005) [2023-12-27 14:59:10,162][20347] Updated weights for policy 0, policy_version 1347767 (0.0007) [2023-12-27 14:59:10,384][20381] Updated weights for policy 1, policy_version 1346643 (0.0008) [2023-12-27 14:59:10,450][20381] Updated weights for policy 1, policy_version 1346653 (0.0007) [2023-12-27 14:59:10,501][20381] Updated weights for policy 1, policy_version 1346663 (0.0008) [2023-12-27 14:59:10,890][20347] Updated weights for policy 0, policy_version 1347777 (0.0007) [2023-12-27 14:59:10,948][20347] Updated weights for policy 0, policy_version 1347787 (0.0008) [2023-12-27 14:59:11,012][20347] Updated weights for policy 0, policy_version 1347797 (0.0008) [2023-12-27 14:59:11,143][20381] Updated weights for policy 1, policy_version 1346673 (0.0008) [2023-12-27 14:59:11,197][20381] Updated weights for policy 1, policy_version 1346683 (0.0007) [2023-12-27 14:59:11,262][20381] Updated weights for policy 1, policy_version 1346693 (0.0007) [2023-12-27 14:59:11,322][20381] Updated weights for policy 1, policy_version 1346703 (0.0008) [2023-12-27 14:59:11,820][20347] Updated weights for policy 0, policy_version 1347807 (0.0008) [2023-12-27 14:59:11,889][20347] Updated weights for policy 0, policy_version 1347817 (0.0008) [2023-12-27 14:59:11,955][20347] Updated weights for policy 0, policy_version 1347827 (0.0007) [2023-12-27 14:59:12,010][20381] Updated weights for policy 1, policy_version 1346713 (0.0007) [2023-12-27 14:59:12,065][20381] Updated weights for policy 1, policy_version 1346723 (0.0008) [2023-12-27 14:59:12,116][20381] Updated weights for policy 1, policy_version 1346733 (0.0008) [2023-12-27 14:59:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 689905664. Throughput: 0: 10244.8, 1: 10316.3. Samples: 689895336. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:12,363][19170] Avg episode reward: [(0, '882.890'), (1, '688.100')] [2023-12-27 14:59:12,660][20347] Updated weights for policy 0, policy_version 1347837 (0.0007) [2023-12-27 14:59:12,719][20347] Updated weights for policy 0, policy_version 1347847 (0.0008) [2023-12-27 14:59:12,785][20347] Updated weights for policy 0, policy_version 1347857 (0.0007) [2023-12-27 14:59:12,840][20381] Updated weights for policy 1, policy_version 1346743 (0.0007) [2023-12-27 14:59:12,897][20381] Updated weights for policy 1, policy_version 1346753 (0.0007) [2023-12-27 14:59:12,948][20381] Updated weights for policy 1, policy_version 1346763 (0.0008) [2023-12-27 14:59:13,424][20347] Updated weights for policy 0, policy_version 1347867 (0.0007) [2023-12-27 14:59:13,488][20347] Updated weights for policy 0, policy_version 1347877 (0.0006) [2023-12-27 14:59:13,506][20381] Updated weights for policy 1, policy_version 1346773 (0.0007) [2023-12-27 14:59:13,541][20347] Updated weights for policy 0, policy_version 1347887 (0.0005) [2023-12-27 14:59:13,561][20381] Updated weights for policy 1, policy_version 1346783 (0.0007) [2023-12-27 14:59:13,618][20381] Updated weights for policy 1, policy_version 1346793 (0.0008) [2023-12-27 14:59:14,117][20347] Updated weights for policy 0, policy_version 1347897 (0.0005) [2023-12-27 14:59:14,181][20347] Updated weights for policy 0, policy_version 1347907 (0.0005) [2023-12-27 14:59:14,238][20347] Updated weights for policy 0, policy_version 1347917 (0.0005) [2023-12-27 14:59:14,287][20347] Updated weights for policy 0, policy_version 1347927 (0.0005) [2023-12-27 14:59:14,328][20381] Updated weights for policy 1, policy_version 1346803 (0.0008) [2023-12-27 14:59:14,386][20381] Updated weights for policy 1, policy_version 1346813 (0.0005) [2023-12-27 14:59:14,448][20381] Updated weights for policy 1, policy_version 1346823 (0.0005) [2023-12-27 14:59:14,872][20347] Updated weights for policy 0, policy_version 1347937 (0.0008) [2023-12-27 14:59:14,930][20347] Updated weights for policy 0, policy_version 1347947 (0.0008) [2023-12-27 14:59:14,991][20347] Updated weights for policy 0, policy_version 1347957 (0.0008) [2023-12-27 14:59:15,030][20381] Updated weights for policy 1, policy_version 1346833 (0.0006) [2023-12-27 14:59:15,083][20381] Updated weights for policy 1, policy_version 1346843 (0.0007) [2023-12-27 14:59:15,137][20381] Updated weights for policy 1, policy_version 1346853 (0.0008) [2023-12-27 14:59:15,191][20381] Updated weights for policy 1, policy_version 1346863 (0.0007) [2023-12-27 14:59:15,698][20347] Updated weights for policy 0, policy_version 1347967 (0.0008) [2023-12-27 14:59:15,745][20347] Updated weights for policy 0, policy_version 1347977 (0.0007) [2023-12-27 14:59:15,794][20347] Updated weights for policy 0, policy_version 1347987 (0.0007) [2023-12-27 14:59:15,898][20381] Updated weights for policy 1, policy_version 1346873 (0.0008) [2023-12-27 14:59:15,949][20381] Updated weights for policy 1, policy_version 1346883 (0.0007) [2023-12-27 14:59:16,035][20381] Updated weights for policy 1, policy_version 1346893 (0.0006) [2023-12-27 14:59:16,525][20347] Updated weights for policy 0, policy_version 1347997 (0.0007) [2023-12-27 14:59:16,569][20347] Updated weights for policy 0, policy_version 1348007 (0.0007) [2023-12-27 14:59:16,624][20347] Updated weights for policy 0, policy_version 1348017 (0.0007) [2023-12-27 14:59:16,644][20381] Updated weights for policy 1, policy_version 1346903 (0.0009) [2023-12-27 14:59:16,701][20381] Updated weights for policy 1, policy_version 1346913 (0.0009) [2023-12-27 14:59:16,760][20381] Updated weights for policy 1, policy_version 1346923 (0.0006) [2023-12-27 14:59:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 690003968. Throughput: 0: 10267.3, 1: 10380.0. Samples: 690019824. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:17,363][19170] Avg episode reward: [(0, '912.220'), (1, '765.030')] [2023-12-27 14:59:17,365][20347] Updated weights for policy 0, policy_version 1348027 (0.0006) [2023-12-27 14:59:17,370][20381] Updated weights for policy 1, policy_version 1346933 (0.0005) [2023-12-27 14:59:17,414][20347] Updated weights for policy 0, policy_version 1348037 (0.0005) [2023-12-27 14:59:17,425][20381] Updated weights for policy 1, policy_version 1346943 (0.0008) [2023-12-27 14:59:17,461][20347] Updated weights for policy 0, policy_version 1348047 (0.0005) [2023-12-27 14:59:17,482][20381] Updated weights for policy 1, policy_version 1346953 (0.0008) [2023-12-27 14:59:18,082][20381] Updated weights for policy 1, policy_version 1346963 (0.0007) [2023-12-27 14:59:18,133][20381] Updated weights for policy 1, policy_version 1346973 (0.0005) [2023-12-27 14:59:18,151][20347] Updated weights for policy 0, policy_version 1348057 (0.0006) [2023-12-27 14:59:18,190][20381] Updated weights for policy 1, policy_version 1346983 (0.0006) [2023-12-27 14:59:18,217][20347] Updated weights for policy 0, policy_version 1348067 (0.0010) [2023-12-27 14:59:18,269][20347] Updated weights for policy 0, policy_version 1348077 (0.0010) [2023-12-27 14:59:18,314][20347] Updated weights for policy 0, policy_version 1348087 (0.0010) [2023-12-27 14:59:18,858][20381] Updated weights for policy 1, policy_version 1346993 (0.0006) [2023-12-27 14:59:18,915][20381] Updated weights for policy 1, policy_version 1347003 (0.0009) [2023-12-27 14:59:18,966][20381] Updated weights for policy 1, policy_version 1347013 (0.0010) [2023-12-27 14:59:19,002][20347] Updated weights for policy 0, policy_version 1348097 (0.0010) [2023-12-27 14:59:19,014][20381] Updated weights for policy 1, policy_version 1347023 (0.0010) [2023-12-27 14:59:19,053][20347] Updated weights for policy 0, policy_version 1348107 (0.0010) [2023-12-27 14:59:19,110][20347] Updated weights for policy 0, policy_version 1348117 (0.0010) [2023-12-27 14:59:19,738][20381] Updated weights for policy 1, policy_version 1347033 (0.0008) [2023-12-27 14:59:19,801][20381] Updated weights for policy 1, policy_version 1347043 (0.0008) [2023-12-27 14:59:19,857][20347] Updated weights for policy 0, policy_version 1348127 (0.0009) [2023-12-27 14:59:19,863][20381] Updated weights for policy 1, policy_version 1347053 (0.0008) [2023-12-27 14:59:19,915][20347] Updated weights for policy 0, policy_version 1348137 (0.0011) [2023-12-27 14:59:19,972][20347] Updated weights for policy 0, policy_version 1348147 (0.0010) [2023-12-27 14:59:20,544][20381] Updated weights for policy 1, policy_version 1347063 (0.0008) [2023-12-27 14:59:20,620][20381] Updated weights for policy 1, policy_version 1347073 (0.0007) [2023-12-27 14:59:20,662][20347] Updated weights for policy 0, policy_version 1348157 (0.0010) [2023-12-27 14:59:20,682][20381] Updated weights for policy 1, policy_version 1347083 (0.0010) [2023-12-27 14:59:20,723][20347] Updated weights for policy 0, policy_version 1348167 (0.0012) [2023-12-27 14:59:20,781][20347] Updated weights for policy 0, policy_version 1348177 (0.0011) [2023-12-27 14:59:21,341][20381] Updated weights for policy 1, policy_version 1347093 (0.0009) [2023-12-27 14:59:21,413][20381] Updated weights for policy 1, policy_version 1347103 (0.0008) [2023-12-27 14:59:21,478][20381] Updated weights for policy 1, policy_version 1347113 (0.0009) [2023-12-27 14:59:21,506][20347] Updated weights for policy 0, policy_version 1348187 (0.0011) [2023-12-27 14:59:21,565][20347] Updated weights for policy 0, policy_version 1348197 (0.0010) [2023-12-27 14:59:21,629][20347] Updated weights for policy 0, policy_version 1348207 (0.0006) [2023-12-27 14:59:22,163][20381] Updated weights for policy 1, policy_version 1347123 (0.0011) [2023-12-27 14:59:22,218][20381] Updated weights for policy 1, policy_version 1347133 (0.0011) [2023-12-27 14:59:22,279][20381] Updated weights for policy 1, policy_version 1347143 (0.0009) [2023-12-27 14:59:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 690110464. Throughput: 0: 10246.3, 1: 10318.6. Samples: 690080572. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:22,363][19170] Avg episode reward: [(0, '902.830'), (1, '813.980')] [2023-12-27 14:59:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001347152_344899584.pth... [2023-12-27 14:59:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001348216_345210880.pth... [2023-12-27 14:59:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001347032_344907776.pth [2023-12-27 14:59:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001345936_344588288.pth [2023-12-27 14:59:22,427][20347] Updated weights for policy 0, policy_version 1348217 (0.0008) [2023-12-27 14:59:22,476][20347] Updated weights for policy 0, policy_version 1348227 (0.0006) [2023-12-27 14:59:22,537][20347] Updated weights for policy 0, policy_version 1348237 (0.0007) [2023-12-27 14:59:22,595][20347] Updated weights for policy 0, policy_version 1348247 (0.0008) [2023-12-27 14:59:22,983][20381] Updated weights for policy 1, policy_version 1347153 (0.0008) [2023-12-27 14:59:23,044][20381] Updated weights for policy 1, policy_version 1347163 (0.0011) [2023-12-27 14:59:23,096][20381] Updated weights for policy 1, policy_version 1347173 (0.0010) [2023-12-27 14:59:23,154][20381] Updated weights for policy 1, policy_version 1347183 (0.0008) [2023-12-27 14:59:23,193][20347] Updated weights for policy 0, policy_version 1348257 (0.0006) [2023-12-27 14:59:23,247][20347] Updated weights for policy 0, policy_version 1348267 (0.0005) [2023-12-27 14:59:23,298][20347] Updated weights for policy 0, policy_version 1348277 (0.0005) [2023-12-27 14:59:23,866][20381] Updated weights for policy 1, policy_version 1347193 (0.0007) [2023-12-27 14:59:23,892][20347] Updated weights for policy 0, policy_version 1348287 (0.0009) [2023-12-27 14:59:23,913][20381] Updated weights for policy 1, policy_version 1347203 (0.0007) [2023-12-27 14:59:23,947][20347] Updated weights for policy 0, policy_version 1348297 (0.0010) [2023-12-27 14:59:23,965][20381] Updated weights for policy 1, policy_version 1347213 (0.0006) [2023-12-27 14:59:23,999][20347] Updated weights for policy 0, policy_version 1348307 (0.0010) [2023-12-27 14:59:24,644][20381] Updated weights for policy 1, policy_version 1347223 (0.0007) [2023-12-27 14:59:24,675][20347] Updated weights for policy 0, policy_version 1348317 (0.0009) [2023-12-27 14:59:24,692][20381] Updated weights for policy 1, policy_version 1347233 (0.0006) [2023-12-27 14:59:24,720][20347] Updated weights for policy 0, policy_version 1348327 (0.0006) [2023-12-27 14:59:24,748][20381] Updated weights for policy 1, policy_version 1347243 (0.0006) [2023-12-27 14:59:24,769][20347] Updated weights for policy 0, policy_version 1348337 (0.0005) [2023-12-27 14:59:25,384][20347] Updated weights for policy 0, policy_version 1348347 (0.0007) [2023-12-27 14:59:25,438][20347] Updated weights for policy 0, policy_version 1348357 (0.0005) [2023-12-27 14:59:25,488][20381] Updated weights for policy 1, policy_version 1347253 (0.0008) [2023-12-27 14:59:25,495][20347] Updated weights for policy 0, policy_version 1348367 (0.0005) [2023-12-27 14:59:25,543][20381] Updated weights for policy 1, policy_version 1347263 (0.0007) [2023-12-27 14:59:25,606][20381] Updated weights for policy 1, policy_version 1347273 (0.0008) [2023-12-27 14:59:26,058][20347] Updated weights for policy 0, policy_version 1348377 (0.0006) [2023-12-27 14:59:26,111][20347] Updated weights for policy 0, policy_version 1348387 (0.0008) [2023-12-27 14:59:26,162][20347] Updated weights for policy 0, policy_version 1348397 (0.0007) [2023-12-27 14:59:26,205][20347] Updated weights for policy 0, policy_version 1348407 (0.0006) [2023-12-27 14:59:26,310][20381] Updated weights for policy 1, policy_version 1347283 (0.0007) [2023-12-27 14:59:26,364][20381] Updated weights for policy 1, policy_version 1347293 (0.0005) [2023-12-27 14:59:26,419][20381] Updated weights for policy 1, policy_version 1347303 (0.0005) [2023-12-27 14:59:26,867][20347] Updated weights for policy 0, policy_version 1348417 (0.0006) [2023-12-27 14:59:26,923][20347] Updated weights for policy 0, policy_version 1348427 (0.0005) [2023-12-27 14:59:26,978][20347] Updated weights for policy 0, policy_version 1348437 (0.0006) [2023-12-27 14:59:26,990][20381] Updated weights for policy 1, policy_version 1347313 (0.0005) [2023-12-27 14:59:27,037][20381] Updated weights for policy 1, policy_version 1347323 (0.0005) [2023-12-27 14:59:27,085][20381] Updated weights for policy 1, policy_version 1347333 (0.0005) [2023-12-27 14:59:27,135][20381] Updated weights for policy 1, policy_version 1347343 (0.0005) [2023-12-27 14:59:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 690216960. Throughput: 0: 10312.2, 1: 10316.8. Samples: 690203444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:27,363][19170] Avg episode reward: [(0, '863.180'), (1, '776.150')] [2023-12-27 14:59:27,587][20347] Updated weights for policy 0, policy_version 1348447 (0.0007) [2023-12-27 14:59:27,646][20347] Updated weights for policy 0, policy_version 1348457 (0.0008) [2023-12-27 14:59:27,691][20347] Updated weights for policy 0, policy_version 1348467 (0.0007) [2023-12-27 14:59:27,715][20381] Updated weights for policy 1, policy_version 1347353 (0.0006) [2023-12-27 14:59:27,763][20381] Updated weights for policy 1, policy_version 1347363 (0.0007) [2023-12-27 14:59:27,819][20381] Updated weights for policy 1, policy_version 1347373 (0.0008) [2023-12-27 14:59:28,390][20347] Updated weights for policy 0, policy_version 1348477 (0.0008) [2023-12-27 14:59:28,455][20347] Updated weights for policy 0, policy_version 1348487 (0.0010) [2023-12-27 14:59:28,508][20381] Updated weights for policy 1, policy_version 1347383 (0.0006) [2023-12-27 14:59:28,510][20347] Updated weights for policy 0, policy_version 1348497 (0.0007) [2023-12-27 14:59:28,565][20381] Updated weights for policy 1, policy_version 1347393 (0.0007) [2023-12-27 14:59:28,624][20381] Updated weights for policy 1, policy_version 1347403 (0.0008) [2023-12-27 14:59:29,155][20347] Updated weights for policy 0, policy_version 1348507 (0.0005) [2023-12-27 14:59:29,213][20347] Updated weights for policy 0, policy_version 1348517 (0.0005) [2023-12-27 14:59:29,273][20347] Updated weights for policy 0, policy_version 1348527 (0.0008) [2023-12-27 14:59:29,324][20381] Updated weights for policy 1, policy_version 1347413 (0.0007) [2023-12-27 14:59:29,390][20381] Updated weights for policy 1, policy_version 1347423 (0.0008) [2023-12-27 14:59:29,447][20381] Updated weights for policy 1, policy_version 1347433 (0.0008) [2023-12-27 14:59:29,941][20347] Updated weights for policy 0, policy_version 1348537 (0.0006) [2023-12-27 14:59:29,997][20347] Updated weights for policy 0, policy_version 1348547 (0.0007) [2023-12-27 14:59:30,059][20347] Updated weights for policy 0, policy_version 1348557 (0.0008) [2023-12-27 14:59:30,120][20347] Updated weights for policy 0, policy_version 1348567 (0.0007) [2023-12-27 14:59:30,177][20381] Updated weights for policy 1, policy_version 1347443 (0.0007) [2023-12-27 14:59:30,226][20381] Updated weights for policy 1, policy_version 1347453 (0.0007) [2023-12-27 14:59:30,279][20381] Updated weights for policy 1, policy_version 1347463 (0.0008) [2023-12-27 14:59:30,800][20347] Updated weights for policy 0, policy_version 1348577 (0.0008) [2023-12-27 14:59:30,844][20347] Updated weights for policy 0, policy_version 1348587 (0.0007) [2023-12-27 14:59:30,903][20347] Updated weights for policy 0, policy_version 1348597 (0.0008) [2023-12-27 14:59:30,948][20381] Updated weights for policy 1, policy_version 1347473 (0.0008) [2023-12-27 14:59:30,993][20381] Updated weights for policy 1, policy_version 1347483 (0.0008) [2023-12-27 14:59:31,053][20381] Updated weights for policy 1, policy_version 1347493 (0.0008) [2023-12-27 14:59:31,122][20381] Updated weights for policy 1, policy_version 1347503 (0.0008) [2023-12-27 14:59:31,613][20347] Updated weights for policy 0, policy_version 1348607 (0.0009) [2023-12-27 14:59:31,674][20347] Updated weights for policy 0, policy_version 1348617 (0.0009) [2023-12-27 14:59:31,738][20347] Updated weights for policy 0, policy_version 1348627 (0.0007) [2023-12-27 14:59:31,814][20381] Updated weights for policy 1, policy_version 1347513 (0.0007) [2023-12-27 14:59:31,871][20381] Updated weights for policy 1, policy_version 1347523 (0.0008) [2023-12-27 14:59:31,929][20381] Updated weights for policy 1, policy_version 1347533 (0.0007) [2023-12-27 14:59:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 690315264. Throughput: 0: 10284.6, 1: 10347.8. Samples: 690328180. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:32,364][19170] Avg episode reward: [(0, '834.310'), (1, '786.060')] [2023-12-27 14:59:32,430][20347] Updated weights for policy 0, policy_version 1348637 (0.0006) [2023-12-27 14:59:32,486][20347] Updated weights for policy 0, policy_version 1348647 (0.0005) [2023-12-27 14:59:32,538][20347] Updated weights for policy 0, policy_version 1348657 (0.0006) [2023-12-27 14:59:32,604][20381] Updated weights for policy 1, policy_version 1347543 (0.0007) [2023-12-27 14:59:32,656][20381] Updated weights for policy 1, policy_version 1347553 (0.0008) [2023-12-27 14:59:32,712][20381] Updated weights for policy 1, policy_version 1347563 (0.0007) [2023-12-27 14:59:33,168][20347] Updated weights for policy 0, policy_version 1348667 (0.0006) [2023-12-27 14:59:33,221][20347] Updated weights for policy 0, policy_version 1348677 (0.0007) [2023-12-27 14:59:33,276][20347] Updated weights for policy 0, policy_version 1348687 (0.0005) [2023-12-27 14:59:33,395][20381] Updated weights for policy 1, policy_version 1347573 (0.0009) [2023-12-27 14:59:33,454][20381] Updated weights for policy 1, policy_version 1347583 (0.0011) [2023-12-27 14:59:33,509][20381] Updated weights for policy 1, policy_version 1347593 (0.0010) [2023-12-27 14:59:33,916][20347] Updated weights for policy 0, policy_version 1348697 (0.0005) [2023-12-27 14:59:33,966][20347] Updated weights for policy 0, policy_version 1348707 (0.0007) [2023-12-27 14:59:33,994][20291] KL-divergence is very high: 358.7736 [2023-12-27 14:59:34,017][20347] Updated weights for policy 0, policy_version 1348717 (0.0007) [2023-12-27 14:59:34,032][20291] KL-divergence is very high: 389.2501 [2023-12-27 14:59:34,067][20347] Updated weights for policy 0, policy_version 1348727 (0.0007) [2023-12-27 14:59:34,195][20381] Updated weights for policy 1, policy_version 1347603 (0.0010) [2023-12-27 14:59:34,249][20381] Updated weights for policy 1, policy_version 1347613 (0.0008) [2023-12-27 14:59:34,310][20381] Updated weights for policy 1, policy_version 1347623 (0.0008) [2023-12-27 14:59:34,842][20347] Updated weights for policy 0, policy_version 1348737 (0.0006) [2023-12-27 14:59:34,899][20347] Updated weights for policy 0, policy_version 1348747 (0.0007) [2023-12-27 14:59:34,939][20381] Updated weights for policy 1, policy_version 1347633 (0.0008) [2023-12-27 14:59:34,954][20347] Updated weights for policy 0, policy_version 1348757 (0.0008) [2023-12-27 14:59:34,992][20381] Updated weights for policy 1, policy_version 1347643 (0.0010) [2023-12-27 14:59:35,047][20381] Updated weights for policy 1, policy_version 1347653 (0.0009) [2023-12-27 14:59:35,100][20381] Updated weights for policy 1, policy_version 1347663 (0.0005) [2023-12-27 14:59:35,583][20347] Updated weights for policy 0, policy_version 1348767 (0.0006) [2023-12-27 14:59:35,634][20347] Updated weights for policy 0, policy_version 1348777 (0.0006) [2023-12-27 14:59:35,681][20347] Updated weights for policy 0, policy_version 1348787 (0.0007) [2023-12-27 14:59:35,723][20381] Updated weights for policy 1, policy_version 1347673 (0.0010) [2023-12-27 14:59:35,769][20381] Updated weights for policy 1, policy_version 1347683 (0.0010) [2023-12-27 14:59:35,811][20381] Updated weights for policy 1, policy_version 1347693 (0.0009) [2023-12-27 14:59:36,362][20347] Updated weights for policy 0, policy_version 1348797 (0.0008) [2023-12-27 14:59:36,422][20347] Updated weights for policy 0, policy_version 1348807 (0.0007) [2023-12-27 14:59:36,482][20347] Updated weights for policy 0, policy_version 1348817 (0.0007) [2023-12-27 14:59:36,532][20381] Updated weights for policy 1, policy_version 1347703 (0.0009) [2023-12-27 14:59:36,596][20381] Updated weights for policy 1, policy_version 1347713 (0.0006) [2023-12-27 14:59:36,659][20381] Updated weights for policy 1, policy_version 1347723 (0.0008) [2023-12-27 14:59:37,227][20347] Updated weights for policy 0, policy_version 1348827 (0.0007) [2023-12-27 14:59:37,282][20347] Updated weights for policy 0, policy_version 1348837 (0.0008) [2023-12-27 14:59:37,338][20347] Updated weights for policy 0, policy_version 1348847 (0.0008) [2023-12-27 14:59:37,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 690413568. Throughput: 0: 10319.8, 1: 10336.7. Samples: 690389632. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:37,363][19170] Avg episode reward: [(0, '863.490'), (1, '814.750')] [2023-12-27 14:59:37,377][20381] Updated weights for policy 1, policy_version 1347733 (0.0008) [2023-12-27 14:59:37,389][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001348856_345374720.pth... [2023-12-27 14:59:37,393][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001347640_345063424.pth [2023-12-27 14:59:37,432][20381] Updated weights for policy 1, policy_version 1347743 (0.0005) [2023-12-27 14:59:37,496][20381] Updated weights for policy 1, policy_version 1347753 (0.0005) [2023-12-27 14:59:37,538][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001347760_345055232.pth... [2023-12-27 14:59:37,543][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001346544_344743936.pth [2023-12-27 14:59:38,041][20381] Updated weights for policy 1, policy_version 1347763 (0.0005) [2023-12-27 14:59:38,046][20347] Updated weights for policy 0, policy_version 1348857 (0.0008) [2023-12-27 14:59:38,092][20347] Updated weights for policy 0, policy_version 1348867 (0.0009) [2023-12-27 14:59:38,099][20381] Updated weights for policy 1, policy_version 1347773 (0.0005) [2023-12-27 14:59:38,139][20347] Updated weights for policy 0, policy_version 1348877 (0.0008) [2023-12-27 14:59:38,159][20381] Updated weights for policy 1, policy_version 1347783 (0.0006) [2023-12-27 14:59:38,181][20291] KL-divergence is very high: 106.6554 [2023-12-27 14:59:38,191][20347] Updated weights for policy 0, policy_version 1348887 (0.0007) [2023-12-27 14:59:38,724][20381] Updated weights for policy 1, policy_version 1347793 (0.0006) [2023-12-27 14:59:38,786][20381] Updated weights for policy 1, policy_version 1347803 (0.0010) [2023-12-27 14:59:38,848][20381] Updated weights for policy 1, policy_version 1347813 (0.0010) [2023-12-27 14:59:38,907][20381] Updated weights for policy 1, policy_version 1347823 (0.0011) [2023-12-27 14:59:38,942][20347] Updated weights for policy 0, policy_version 1348897 (0.0010) [2023-12-27 14:59:39,002][20347] Updated weights for policy 0, policy_version 1348907 (0.0009) [2023-12-27 14:59:39,039][20291] KL-divergence is very high: 101.8946 [2023-12-27 14:59:39,054][20347] Updated weights for policy 0, policy_version 1348917 (0.0009) [2023-12-27 14:59:39,564][20381] Updated weights for policy 1, policy_version 1347833 (0.0008) [2023-12-27 14:59:39,628][20381] Updated weights for policy 1, policy_version 1347843 (0.0008) [2023-12-27 14:59:39,688][20381] Updated weights for policy 1, policy_version 1347853 (0.0008) [2023-12-27 14:59:39,756][20347] Updated weights for policy 0, policy_version 1348927 (0.0006) [2023-12-27 14:59:39,828][20347] Updated weights for policy 0, policy_version 1348937 (0.0007) [2023-12-27 14:59:39,889][20347] Updated weights for policy 0, policy_version 1348947 (0.0008) [2023-12-27 14:59:40,414][20381] Updated weights for policy 1, policy_version 1347863 (0.0007) [2023-12-27 14:59:40,470][20381] Updated weights for policy 1, policy_version 1347873 (0.0007) [2023-12-27 14:59:40,533][20381] Updated weights for policy 1, policy_version 1347883 (0.0008) [2023-12-27 14:59:40,618][20347] Updated weights for policy 0, policy_version 1348957 (0.0007) [2023-12-27 14:59:40,683][20347] Updated weights for policy 0, policy_version 1348967 (0.0006) [2023-12-27 14:59:40,746][20347] Updated weights for policy 0, policy_version 1348977 (0.0006) [2023-12-27 14:59:41,231][20381] Updated weights for policy 1, policy_version 1347893 (0.0007) [2023-12-27 14:59:41,293][20381] Updated weights for policy 1, policy_version 1347903 (0.0008) [2023-12-27 14:59:41,348][20381] Updated weights for policy 1, policy_version 1347913 (0.0009) [2023-12-27 14:59:41,392][20347] Updated weights for policy 0, policy_version 1348987 (0.0006) [2023-12-27 14:59:41,450][20347] Updated weights for policy 0, policy_version 1348997 (0.0007) [2023-12-27 14:59:41,508][20347] Updated weights for policy 0, policy_version 1349007 (0.0007) [2023-12-27 14:59:42,083][20381] Updated weights for policy 1, policy_version 1347923 (0.0009) [2023-12-27 14:59:42,139][20381] Updated weights for policy 1, policy_version 1347933 (0.0007) [2023-12-27 14:59:42,191][20381] Updated weights for policy 1, policy_version 1347943 (0.0007) [2023-12-27 14:59:42,243][20347] Updated weights for policy 0, policy_version 1349017 (0.0009) [2023-12-27 14:59:42,302][20347] Updated weights for policy 0, policy_version 1349027 (0.0008) [2023-12-27 14:59:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 690520064. Throughput: 0: 10262.1, 1: 10426.6. Samples: 690513004. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:42,363][19170] Avg episode reward: [(0, '882.920'), (1, '833.560')] [2023-12-27 14:59:42,363][20347] Updated weights for policy 0, policy_version 1349037 (0.0008) [2023-12-27 14:59:42,427][20347] Updated weights for policy 0, policy_version 1349047 (0.0006) [2023-12-27 14:59:42,854][20381] Updated weights for policy 1, policy_version 1347953 (0.0007) [2023-12-27 14:59:42,920][20381] Updated weights for policy 1, policy_version 1347963 (0.0007) [2023-12-27 14:59:42,976][20381] Updated weights for policy 1, policy_version 1347973 (0.0008) [2023-12-27 14:59:43,031][20381] Updated weights for policy 1, policy_version 1347983 (0.0007) [2023-12-27 14:59:43,086][20347] Updated weights for policy 0, policy_version 1349057 (0.0008) [2023-12-27 14:59:43,135][20347] Updated weights for policy 0, policy_version 1349067 (0.0007) [2023-12-27 14:59:43,184][20347] Updated weights for policy 0, policy_version 1349077 (0.0007) [2023-12-27 14:59:43,697][20381] Updated weights for policy 1, policy_version 1347993 (0.0005) [2023-12-27 14:59:43,757][20381] Updated weights for policy 1, policy_version 1348003 (0.0005) [2023-12-27 14:59:43,820][20381] Updated weights for policy 1, policy_version 1348013 (0.0006) [2023-12-27 14:59:43,909][20347] Updated weights for policy 0, policy_version 1349087 (0.0008) [2023-12-27 14:59:43,970][20347] Updated weights for policy 0, policy_version 1349097 (0.0008) [2023-12-27 14:59:44,026][20347] Updated weights for policy 0, policy_version 1349107 (0.0007) [2023-12-27 14:59:44,407][20381] Updated weights for policy 1, policy_version 1348023 (0.0005) [2023-12-27 14:59:44,469][20381] Updated weights for policy 1, policy_version 1348033 (0.0005) [2023-12-27 14:59:44,532][20381] Updated weights for policy 1, policy_version 1348043 (0.0006) [2023-12-27 14:59:44,763][20347] Updated weights for policy 0, policy_version 1349117 (0.0008) [2023-12-27 14:59:44,829][20347] Updated weights for policy 0, policy_version 1349127 (0.0006) [2023-12-27 14:59:44,890][20347] Updated weights for policy 0, policy_version 1349137 (0.0007) [2023-12-27 14:59:45,186][20381] Updated weights for policy 1, policy_version 1348053 (0.0008) [2023-12-27 14:59:45,240][20381] Updated weights for policy 1, policy_version 1348063 (0.0010) [2023-12-27 14:59:45,290][20381] Updated weights for policy 1, policy_version 1348073 (0.0011) [2023-12-27 14:59:45,515][20347] Updated weights for policy 0, policy_version 1349147 (0.0006) [2023-12-27 14:59:45,564][20347] Updated weights for policy 0, policy_version 1349157 (0.0007) [2023-12-27 14:59:45,621][20347] Updated weights for policy 0, policy_version 1349167 (0.0007) [2023-12-27 14:59:45,915][20381] Updated weights for policy 1, policy_version 1348083 (0.0009) [2023-12-27 14:59:45,961][20381] Updated weights for policy 1, policy_version 1348093 (0.0005) [2023-12-27 14:59:46,017][20381] Updated weights for policy 1, policy_version 1348103 (0.0006) [2023-12-27 14:59:46,376][20347] Updated weights for policy 0, policy_version 1349177 (0.0007) [2023-12-27 14:59:46,421][20347] Updated weights for policy 0, policy_version 1349187 (0.0008) [2023-12-27 14:59:46,473][20347] Updated weights for policy 0, policy_version 1349197 (0.0008) [2023-12-27 14:59:46,527][20347] Updated weights for policy 0, policy_version 1349207 (0.0009) [2023-12-27 14:59:46,596][20381] Updated weights for policy 1, policy_version 1348113 (0.0010) [2023-12-27 14:59:46,644][20381] Updated weights for policy 1, policy_version 1348123 (0.0010) [2023-12-27 14:59:46,696][20381] Updated weights for policy 1, policy_version 1348133 (0.0010) [2023-12-27 14:59:46,750][20381] Updated weights for policy 1, policy_version 1348143 (0.0009) [2023-12-27 14:59:47,152][20347] Updated weights for policy 0, policy_version 1349217 (0.0006) [2023-12-27 14:59:47,208][20347] Updated weights for policy 0, policy_version 1349227 (0.0005) [2023-12-27 14:59:47,264][20347] Updated weights for policy 0, policy_version 1349237 (0.0005) [2023-12-27 14:59:47,352][20381] Updated weights for policy 1, policy_version 1348153 (0.0008) [2023-12-27 14:59:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 690626560. Throughput: 0: 10195.4, 1: 10479.9. Samples: 690637284. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:47,363][19170] Avg episode reward: [(0, '883.030'), (1, '823.790')] [2023-12-27 14:59:47,411][20381] Updated weights for policy 1, policy_version 1348163 (0.0008) [2023-12-27 14:59:47,463][20381] Updated weights for policy 1, policy_version 1348173 (0.0008) [2023-12-27 14:59:47,824][20347] Updated weights for policy 0, policy_version 1349247 (0.0007) [2023-12-27 14:59:47,871][20347] Updated weights for policy 0, policy_version 1349257 (0.0007) [2023-12-27 14:59:47,926][20347] Updated weights for policy 0, policy_version 1349268 (0.0008) [2023-12-27 14:59:48,181][20381] Updated weights for policy 1, policy_version 1348183 (0.0007) [2023-12-27 14:59:48,230][20381] Updated weights for policy 1, policy_version 1348193 (0.0007) [2023-12-27 14:59:48,285][20381] Updated weights for policy 1, policy_version 1348203 (0.0007) [2023-12-27 14:59:48,543][20347] Updated weights for policy 0, policy_version 1349278 (0.0008) [2023-12-27 14:59:48,585][20291] KL-divergence is very high: 285.9526 [2023-12-27 14:59:48,594][20347] Updated weights for policy 0, policy_version 1349288 (0.0010) [2023-12-27 14:59:48,601][20291] KL-divergence is very high: 345.1975 [2023-12-27 14:59:48,631][20291] KL-divergence is very high: 315.8682 [2023-12-27 14:59:48,648][20291] KL-divergence is very high: 271.8033 [2023-12-27 14:59:48,655][20347] Updated weights for policy 0, policy_version 1349298 (0.0010) [2023-12-27 14:59:48,679][20291] KL-divergence is very high: 163.1136 [2023-12-27 14:59:48,945][20381] Updated weights for policy 1, policy_version 1348213 (0.0007) [2023-12-27 14:59:48,995][20381] Updated weights for policy 1, policy_version 1348223 (0.0005) [2023-12-27 14:59:49,046][20381] Updated weights for policy 1, policy_version 1348233 (0.0005) [2023-12-27 14:59:49,377][20347] Updated weights for policy 0, policy_version 1349308 (0.0009) [2023-12-27 14:59:49,427][20347] Updated weights for policy 0, policy_version 1349318 (0.0007) [2023-12-27 14:59:49,491][20347] Updated weights for policy 0, policy_version 1349328 (0.0008) [2023-12-27 14:59:49,801][20381] Updated weights for policy 1, policy_version 1348243 (0.0009) [2023-12-27 14:59:49,865][20381] Updated weights for policy 1, policy_version 1348253 (0.0009) [2023-12-27 14:59:49,927][20381] Updated weights for policy 1, policy_version 1348263 (0.0007) [2023-12-27 14:59:50,142][20347] Updated weights for policy 0, policy_version 1349338 (0.0008) [2023-12-27 14:59:50,200][20347] Updated weights for policy 0, policy_version 1349348 (0.0007) [2023-12-27 14:59:50,256][20347] Updated weights for policy 0, policy_version 1349358 (0.0005) [2023-12-27 14:59:50,308][20347] Updated weights for policy 0, policy_version 1349368 (0.0005) [2023-12-27 14:59:50,695][20381] Updated weights for policy 1, policy_version 1348273 (0.0008) [2023-12-27 14:59:50,761][20381] Updated weights for policy 1, policy_version 1348283 (0.0007) [2023-12-27 14:59:50,814][20381] Updated weights for policy 1, policy_version 1348293 (0.0008) [2023-12-27 14:59:50,867][20347] Updated weights for policy 0, policy_version 1349378 (0.0006) [2023-12-27 14:59:50,869][20381] Updated weights for policy 1, policy_version 1348303 (0.0007) [2023-12-27 14:59:50,916][20347] Updated weights for policy 0, policy_version 1349388 (0.0007) [2023-12-27 14:59:50,963][20347] Updated weights for policy 0, policy_version 1349398 (0.0006) [2023-12-27 14:59:51,642][20381] Updated weights for policy 1, policy_version 1348313 (0.0007) [2023-12-27 14:59:51,647][20347] Updated weights for policy 0, policy_version 1349408 (0.0009) [2023-12-27 14:59:51,670][20292] KL-divergence is very high: 248.1466 [2023-12-27 14:59:51,677][20292] KL-divergence is very high: 326.4748 [2023-12-27 14:59:51,689][20292] KL-divergence is very high: 182.6922 [2023-12-27 14:59:51,706][20381] Updated weights for policy 1, policy_version 1348323 (0.0008) [2023-12-27 14:59:51,708][20347] Updated weights for policy 0, policy_version 1349418 (0.0008) [2023-12-27 14:59:51,770][20381] Updated weights for policy 1, policy_version 1348333 (0.0008) [2023-12-27 14:59:51,776][20347] Updated weights for policy 0, policy_version 1349428 (0.0007) [2023-12-27 14:59:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 690724864. Throughput: 0: 10215.5, 1: 10393.7. Samples: 690698568. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 14:59:52,363][19170] Avg episode reward: [(0, '892.900'), (1, '794.740')] [2023-12-27 14:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001348336_345202688.pth... [2023-12-27 14:59:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001349432_345522176.pth... [2023-12-27 14:59:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001347152_344899584.pth [2023-12-27 14:59:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001348216_345210880.pth [2023-12-27 14:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001348336_345202688.pth [2023-12-27 14:59:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001349432_345522176.pth [2023-12-27 14:59:52,470][20347] Updated weights for policy 0, policy_version 1349438 (0.0007) [2023-12-27 14:59:52,525][20347] Updated weights for policy 0, policy_version 1349448 (0.0007) [2023-12-27 14:59:52,549][20381] Updated weights for policy 1, policy_version 1348343 (0.0007) [2023-12-27 14:59:52,578][20347] Updated weights for policy 0, policy_version 1349458 (0.0006) [2023-12-27 14:59:52,610][20381] Updated weights for policy 1, policy_version 1348353 (0.0007) [2023-12-27 14:59:52,660][20381] Updated weights for policy 1, policy_version 1348363 (0.0008) [2023-12-27 14:59:53,186][20347] Updated weights for policy 0, policy_version 1349468 (0.0006) [2023-12-27 14:59:53,238][20347] Updated weights for policy 0, policy_version 1349478 (0.0008) [2023-12-27 14:59:53,251][20381] Updated weights for policy 1, policy_version 1348373 (0.0007) [2023-12-27 14:59:53,292][20347] Updated weights for policy 0, policy_version 1349488 (0.0007) [2023-12-27 14:59:53,314][20381] Updated weights for policy 1, policy_version 1348383 (0.0007) [2023-12-27 14:59:53,375][20381] Updated weights for policy 1, policy_version 1348393 (0.0008) [2023-12-27 14:59:53,909][20347] Updated weights for policy 0, policy_version 1349498 (0.0007) [2023-12-27 14:59:53,955][20347] Updated weights for policy 0, policy_version 1349508 (0.0006) [2023-12-27 14:59:54,009][20347] Updated weights for policy 0, policy_version 1349518 (0.0008) [2023-12-27 14:59:54,055][20347] Updated weights for policy 0, policy_version 1349528 (0.0007) [2023-12-27 14:59:54,057][20381] Updated weights for policy 1, policy_version 1348403 (0.0008) [2023-12-27 14:59:54,113][20381] Updated weights for policy 1, policy_version 1348413 (0.0005) [2023-12-27 14:59:54,171][20381] Updated weights for policy 1, policy_version 1348423 (0.0007) [2023-12-27 14:59:54,689][20347] Updated weights for policy 0, policy_version 1349538 (0.0008) [2023-12-27 14:59:54,702][20381] Updated weights for policy 1, policy_version 1348433 (0.0006) [2023-12-27 14:59:54,744][20347] Updated weights for policy 0, policy_version 1349548 (0.0007) [2023-12-27 14:59:54,765][20381] Updated weights for policy 1, policy_version 1348443 (0.0007) [2023-12-27 14:59:54,800][20347] Updated weights for policy 0, policy_version 1349558 (0.0008) [2023-12-27 14:59:54,820][20381] Updated weights for policy 1, policy_version 1348453 (0.0010) [2023-12-27 14:59:54,873][20381] Updated weights for policy 1, policy_version 1348463 (0.0008) [2023-12-27 14:59:55,444][20381] Updated weights for policy 1, policy_version 1348473 (0.0007) [2023-12-27 14:59:55,500][20381] Updated weights for policy 1, policy_version 1348483 (0.0006) [2023-12-27 14:59:55,512][20347] Updated weights for policy 0, policy_version 1349568 (0.0010) [2023-12-27 14:59:55,555][20381] Updated weights for policy 1, policy_version 1348493 (0.0006) [2023-12-27 14:59:55,570][20347] Updated weights for policy 0, policy_version 1349578 (0.0007) [2023-12-27 14:59:55,616][20347] Updated weights for policy 0, policy_version 1349588 (0.0005) [2023-12-27 14:59:55,625][20291] KL-divergence is very high: 107.4620 [2023-12-27 14:59:56,230][20347] Updated weights for policy 0, policy_version 1349598 (0.0008) [2023-12-27 14:59:56,231][20381] Updated weights for policy 1, policy_version 1348503 (0.0006) [2023-12-27 14:59:56,288][20347] Updated weights for policy 0, policy_version 1349608 (0.0007) [2023-12-27 14:59:56,289][20381] Updated weights for policy 1, policy_version 1348513 (0.0005) [2023-12-27 14:59:56,346][20381] Updated weights for policy 1, policy_version 1348523 (0.0005) [2023-12-27 14:59:56,348][20347] Updated weights for policy 0, policy_version 1349618 (0.0006) [2023-12-27 14:59:56,901][20381] Updated weights for policy 1, policy_version 1348533 (0.0007) [2023-12-27 14:59:56,949][20381] Updated weights for policy 1, policy_version 1348543 (0.0005) [2023-12-27 14:59:56,999][20381] Updated weights for policy 1, policy_version 1348553 (0.0005) [2023-12-27 14:59:57,014][20347] Updated weights for policy 0, policy_version 1349628 (0.0006) [2023-12-27 14:59:57,070][20347] Updated weights for policy 0, policy_version 1349638 (0.0005) [2023-12-27 14:59:57,129][20347] Updated weights for policy 0, policy_version 1349648 (0.0005) [2023-12-27 14:59:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 690839552. Throughput: 0: 10294.7, 1: 10383.4. Samples: 690825848. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 14:59:57,363][19170] Avg episode reward: [(0, '863.550'), (1, '765.730')] [2023-12-27 14:59:57,606][20381] Updated weights for policy 1, policy_version 1348563 (0.0006) [2023-12-27 14:59:57,654][20381] Updated weights for policy 1, policy_version 1348573 (0.0007) [2023-12-27 14:59:57,712][20381] Updated weights for policy 1, policy_version 1348583 (0.0007) [2023-12-27 14:59:57,734][20347] Updated weights for policy 0, policy_version 1349658 (0.0007) [2023-12-27 14:59:57,784][20347] Updated weights for policy 0, policy_version 1349668 (0.0005) [2023-12-27 14:59:57,838][20347] Updated weights for policy 0, policy_version 1349678 (0.0005) [2023-12-27 14:59:57,885][20347] Updated weights for policy 0, policy_version 1349688 (0.0005) [2023-12-27 14:59:58,444][20381] Updated weights for policy 1, policy_version 1348593 (0.0009) [2023-12-27 14:59:58,507][20381] Updated weights for policy 1, policy_version 1348603 (0.0009) [2023-12-27 14:59:58,570][20347] Updated weights for policy 0, policy_version 1349698 (0.0008) [2023-12-27 14:59:58,571][20381] Updated weights for policy 1, policy_version 1348613 (0.0008) [2023-12-27 14:59:58,629][20381] Updated weights for policy 1, policy_version 1348623 (0.0007) [2023-12-27 14:59:58,630][20347] Updated weights for policy 0, policy_version 1349708 (0.0007) [2023-12-27 14:59:58,697][20347] Updated weights for policy 0, policy_version 1349718 (0.0008) [2023-12-27 14:59:59,322][20381] Updated weights for policy 1, policy_version 1348633 (0.0008) [2023-12-27 14:59:59,387][20381] Updated weights for policy 1, policy_version 1348643 (0.0007) [2023-12-27 14:59:59,447][20381] Updated weights for policy 1, policy_version 1348653 (0.0006) [2023-12-27 14:59:59,569][20347] Updated weights for policy 0, policy_version 1349728 (0.0007) [2023-12-27 14:59:59,629][20347] Updated weights for policy 0, policy_version 1349738 (0.0008) [2023-12-27 14:59:59,690][20347] Updated weights for policy 0, policy_version 1349748 (0.0008) [2023-12-27 15:00:00,187][20381] Updated weights for policy 1, policy_version 1348663 (0.0007) [2023-12-27 15:00:00,249][20381] Updated weights for policy 1, policy_version 1348673 (0.0008) [2023-12-27 15:00:00,310][20381] Updated weights for policy 1, policy_version 1348683 (0.0008) [2023-12-27 15:00:00,372][20347] Updated weights for policy 0, policy_version 1349758 (0.0007) [2023-12-27 15:00:00,425][20347] Updated weights for policy 0, policy_version 1349768 (0.0008) [2023-12-27 15:00:00,479][20347] Updated weights for policy 0, policy_version 1349778 (0.0008) [2023-12-27 15:00:00,970][20381] Updated weights for policy 1, policy_version 1348693 (0.0006) [2023-12-27 15:00:01,014][20381] Updated weights for policy 1, policy_version 1348703 (0.0005) [2023-12-27 15:00:01,073][20381] Updated weights for policy 1, policy_version 1348713 (0.0007) [2023-12-27 15:00:01,082][20347] Updated weights for policy 0, policy_version 1349788 (0.0007) [2023-12-27 15:00:01,142][20347] Updated weights for policy 0, policy_version 1349798 (0.0008) [2023-12-27 15:00:01,203][20347] Updated weights for policy 0, policy_version 1349808 (0.0008) [2023-12-27 15:00:01,812][20381] Updated weights for policy 1, policy_version 1348723 (0.0005) [2023-12-27 15:00:01,870][20381] Updated weights for policy 1, policy_version 1348733 (0.0006) [2023-12-27 15:00:01,887][20347] Updated weights for policy 0, policy_version 1349818 (0.0007) [2023-12-27 15:00:01,929][20381] Updated weights for policy 1, policy_version 1348743 (0.0007) [2023-12-27 15:00:01,945][20347] Updated weights for policy 0, policy_version 1349828 (0.0005) [2023-12-27 15:00:01,995][20347] Updated weights for policy 0, policy_version 1349838 (0.0006) [2023-12-27 15:00:02,046][20347] Updated weights for policy 0, policy_version 1349848 (0.0007) [2023-12-27 15:00:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 690937856. Throughput: 0: 10313.2, 1: 10333.9. Samples: 690948940. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:02,363][19170] Avg episode reward: [(0, '902.450'), (1, '716.940')] [2023-12-27 15:00:02,627][20381] Updated weights for policy 1, policy_version 1348753 (0.0007) [2023-12-27 15:00:02,675][20347] Updated weights for policy 0, policy_version 1349858 (0.0007) [2023-12-27 15:00:02,686][20381] Updated weights for policy 1, policy_version 1348763 (0.0005) [2023-12-27 15:00:02,731][20347] Updated weights for policy 0, policy_version 1349868 (0.0006) [2023-12-27 15:00:02,740][20381] Updated weights for policy 1, policy_version 1348773 (0.0007) [2023-12-27 15:00:02,793][20347] Updated weights for policy 0, policy_version 1349878 (0.0005) [2023-12-27 15:00:02,800][20381] Updated weights for policy 1, policy_version 1348783 (0.0008) [2023-12-27 15:00:03,374][20347] Updated weights for policy 0, policy_version 1349888 (0.0007) [2023-12-27 15:00:03,422][20347] Updated weights for policy 0, policy_version 1349898 (0.0007) [2023-12-27 15:00:03,469][20347] Updated weights for policy 0, policy_version 1349908 (0.0008) [2023-12-27 15:00:03,490][20381] Updated weights for policy 1, policy_version 1348793 (0.0006) [2023-12-27 15:00:03,545][20381] Updated weights for policy 1, policy_version 1348803 (0.0008) [2023-12-27 15:00:03,596][20381] Updated weights for policy 1, policy_version 1348813 (0.0007) [2023-12-27 15:00:04,165][20347] Updated weights for policy 0, policy_version 1349918 (0.0007) [2023-12-27 15:00:04,225][20347] Updated weights for policy 0, policy_version 1349928 (0.0008) [2023-12-27 15:00:04,270][20381] Updated weights for policy 1, policy_version 1348823 (0.0007) [2023-12-27 15:00:04,280][20347] Updated weights for policy 0, policy_version 1349938 (0.0006) [2023-12-27 15:00:04,331][20381] Updated weights for policy 1, policy_version 1348833 (0.0008) [2023-12-27 15:00:04,392][20381] Updated weights for policy 1, policy_version 1348843 (0.0008) [2023-12-27 15:00:04,886][20347] Updated weights for policy 0, policy_version 1349948 (0.0007) [2023-12-27 15:00:04,934][20347] Updated weights for policy 0, policy_version 1349958 (0.0006) [2023-12-27 15:00:04,991][20347] Updated weights for policy 0, policy_version 1349968 (0.0005) [2023-12-27 15:00:05,130][20381] Updated weights for policy 1, policy_version 1348853 (0.0007) [2023-12-27 15:00:05,179][20381] Updated weights for policy 1, policy_version 1348863 (0.0008) [2023-12-27 15:00:05,232][20381] Updated weights for policy 1, policy_version 1348873 (0.0008) [2023-12-27 15:00:05,610][20347] Updated weights for policy 0, policy_version 1349978 (0.0006) [2023-12-27 15:00:05,666][20347] Updated weights for policy 0, policy_version 1349988 (0.0008) [2023-12-27 15:00:05,720][20347] Updated weights for policy 0, policy_version 1349998 (0.0008) [2023-12-27 15:00:05,766][20347] Updated weights for policy 0, policy_version 1350008 (0.0007) [2023-12-27 15:00:05,958][20381] Updated weights for policy 1, policy_version 1348883 (0.0008) [2023-12-27 15:00:06,010][20381] Updated weights for policy 1, policy_version 1348893 (0.0008) [2023-12-27 15:00:06,068][20381] Updated weights for policy 1, policy_version 1348903 (0.0008) [2023-12-27 15:00:06,490][20347] Updated weights for policy 0, policy_version 1350018 (0.0008) [2023-12-27 15:00:06,551][20347] Updated weights for policy 0, policy_version 1350028 (0.0008) [2023-12-27 15:00:06,610][20347] Updated weights for policy 0, policy_version 1350038 (0.0007) [2023-12-27 15:00:06,757][20381] Updated weights for policy 1, policy_version 1348913 (0.0007) [2023-12-27 15:00:06,817][20381] Updated weights for policy 1, policy_version 1348923 (0.0008) [2023-12-27 15:00:06,869][20381] Updated weights for policy 1, policy_version 1348933 (0.0007) [2023-12-27 15:00:06,925][20381] Updated weights for policy 1, policy_version 1348943 (0.0007) [2023-12-27 15:00:07,343][20347] Updated weights for policy 0, policy_version 1350048 (0.0006) [2023-12-27 15:00:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 691036160. Throughput: 0: 10385.8, 1: 10293.4. Samples: 691011132. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:07,363][19170] Avg episode reward: [(0, '902.700'), (1, '746.520')] [2023-12-27 15:00:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001348944_345358336.pth... [2023-12-27 15:00:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001347760_345055232.pth [2023-12-27 15:00:07,394][20347] Updated weights for policy 0, policy_version 1350058 (0.0005) [2023-12-27 15:00:07,455][20347] Updated weights for policy 0, policy_version 1350068 (0.0005) [2023-12-27 15:00:07,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001350072_345686016.pth... [2023-12-27 15:00:07,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001348856_345374720.pth [2023-12-27 15:00:07,692][20381] Updated weights for policy 1, policy_version 1348953 (0.0010) [2023-12-27 15:00:07,745][20381] Updated weights for policy 1, policy_version 1348963 (0.0009) [2023-12-27 15:00:07,797][20381] Updated weights for policy 1, policy_version 1348973 (0.0009) [2023-12-27 15:00:07,978][20347] Updated weights for policy 0, policy_version 1350078 (0.0005) [2023-12-27 15:00:08,039][20347] Updated weights for policy 0, policy_version 1350088 (0.0007) [2023-12-27 15:00:08,094][20347] Updated weights for policy 0, policy_version 1350098 (0.0008) [2023-12-27 15:00:08,484][20381] Updated weights for policy 1, policy_version 1348983 (0.0007) [2023-12-27 15:00:08,535][20381] Updated weights for policy 1, policy_version 1348993 (0.0007) [2023-12-27 15:00:08,587][20381] Updated weights for policy 1, policy_version 1349003 (0.0008) [2023-12-27 15:00:08,762][20347] Updated weights for policy 0, policy_version 1350108 (0.0008) [2023-12-27 15:00:08,816][20347] Updated weights for policy 0, policy_version 1350118 (0.0010) [2023-12-27 15:00:08,871][20347] Updated weights for policy 0, policy_version 1350128 (0.0010) [2023-12-27 15:00:09,313][20381] Updated weights for policy 1, policy_version 1349013 (0.0007) [2023-12-27 15:00:09,386][20381] Updated weights for policy 1, policy_version 1349023 (0.0007) [2023-12-27 15:00:09,445][20381] Updated weights for policy 1, policy_version 1349033 (0.0007) [2023-12-27 15:00:09,596][20347] Updated weights for policy 0, policy_version 1350138 (0.0010) [2023-12-27 15:00:09,653][20347] Updated weights for policy 0, policy_version 1350148 (0.0008) [2023-12-27 15:00:09,714][20347] Updated weights for policy 0, policy_version 1350158 (0.0008) [2023-12-27 15:00:09,774][20347] Updated weights for policy 0, policy_version 1350168 (0.0008) [2023-12-27 15:00:10,058][20381] Updated weights for policy 1, policy_version 1349043 (0.0006) [2023-12-27 15:00:10,113][20381] Updated weights for policy 1, policy_version 1349053 (0.0008) [2023-12-27 15:00:10,173][20381] Updated weights for policy 1, policy_version 1349063 (0.0007) [2023-12-27 15:00:10,527][20347] Updated weights for policy 0, policy_version 1350178 (0.0008) [2023-12-27 15:00:10,584][20347] Updated weights for policy 0, policy_version 1350188 (0.0008) [2023-12-27 15:00:10,638][20347] Updated weights for policy 0, policy_version 1350198 (0.0007) [2023-12-27 15:00:10,875][20381] Updated weights for policy 1, policy_version 1349073 (0.0007) [2023-12-27 15:00:10,944][20381] Updated weights for policy 1, policy_version 1349083 (0.0008) [2023-12-27 15:00:10,995][20381] Updated weights for policy 1, policy_version 1349093 (0.0009) [2023-12-27 15:00:11,052][20381] Updated weights for policy 1, policy_version 1349103 (0.0007) [2023-12-27 15:00:11,371][20347] Updated weights for policy 0, policy_version 1350208 (0.0010) [2023-12-27 15:00:11,441][20347] Updated weights for policy 0, policy_version 1350218 (0.0010) [2023-12-27 15:00:11,508][20347] Updated weights for policy 0, policy_version 1350228 (0.0010) [2023-12-27 15:00:11,808][20381] Updated weights for policy 1, policy_version 1349113 (0.0009) [2023-12-27 15:00:11,873][20381] Updated weights for policy 1, policy_version 1349123 (0.0007) [2023-12-27 15:00:11,937][20381] Updated weights for policy 1, policy_version 1349133 (0.0008) [2023-12-27 15:00:12,214][20347] Updated weights for policy 0, policy_version 1350238 (0.0009) [2023-12-27 15:00:12,278][20347] Updated weights for policy 0, policy_version 1350248 (0.0008) [2023-12-27 15:00:12,340][20347] Updated weights for policy 0, policy_version 1350258 (0.0011) [2023-12-27 15:00:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 691134464. Throughput: 0: 10333.0, 1: 10284.8. Samples: 691131248. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:12,363][19170] Avg episode reward: [(0, '873.160'), (1, '755.880')] [2023-12-27 15:00:12,623][20381] Updated weights for policy 1, policy_version 1349143 (0.0006) [2023-12-27 15:00:12,683][20381] Updated weights for policy 1, policy_version 1349153 (0.0007) [2023-12-27 15:00:12,735][20381] Updated weights for policy 1, policy_version 1349163 (0.0007) [2023-12-27 15:00:13,038][20347] Updated weights for policy 0, policy_version 1350268 (0.0011) [2023-12-27 15:00:13,095][20347] Updated weights for policy 0, policy_version 1350278 (0.0010) [2023-12-27 15:00:13,146][20347] Updated weights for policy 0, policy_version 1350288 (0.0010) [2023-12-27 15:00:13,398][20381] Updated weights for policy 1, policy_version 1349173 (0.0007) [2023-12-27 15:00:13,452][20381] Updated weights for policy 1, policy_version 1349183 (0.0007) [2023-12-27 15:00:13,506][20381] Updated weights for policy 1, policy_version 1349193 (0.0007) [2023-12-27 15:00:13,822][20347] Updated weights for policy 0, policy_version 1350298 (0.0010) [2023-12-27 15:00:13,878][20347] Updated weights for policy 0, policy_version 1350308 (0.0010) [2023-12-27 15:00:13,939][20347] Updated weights for policy 0, policy_version 1350318 (0.0010) [2023-12-27 15:00:13,987][20347] Updated weights for policy 0, policy_version 1350328 (0.0010) [2023-12-27 15:00:14,204][20381] Updated weights for policy 1, policy_version 1349203 (0.0007) [2023-12-27 15:00:14,262][20381] Updated weights for policy 1, policy_version 1349213 (0.0006) [2023-12-27 15:00:14,322][20381] Updated weights for policy 1, policy_version 1349223 (0.0007) [2023-12-27 15:00:14,668][20347] Updated weights for policy 0, policy_version 1350338 (0.0010) [2023-12-27 15:00:14,722][20347] Updated weights for policy 0, policy_version 1350348 (0.0010) [2023-12-27 15:00:14,784][20347] Updated weights for policy 0, policy_version 1350358 (0.0010) [2023-12-27 15:00:14,999][20381] Updated weights for policy 1, policy_version 1349233 (0.0007) [2023-12-27 15:00:15,055][20381] Updated weights for policy 1, policy_version 1349243 (0.0006) [2023-12-27 15:00:15,110][20381] Updated weights for policy 1, policy_version 1349253 (0.0008) [2023-12-27 15:00:15,155][20381] Updated weights for policy 1, policy_version 1349263 (0.0007) [2023-12-27 15:00:15,479][20347] Updated weights for policy 0, policy_version 1350368 (0.0006) [2023-12-27 15:00:15,538][20347] Updated weights for policy 0, policy_version 1350378 (0.0006) [2023-12-27 15:00:15,592][20347] Updated weights for policy 0, policy_version 1350388 (0.0005) [2023-12-27 15:00:15,884][20381] Updated weights for policy 1, policy_version 1349273 (0.0008) [2023-12-27 15:00:15,939][20381] Updated weights for policy 1, policy_version 1349283 (0.0008) [2023-12-27 15:00:15,998][20381] Updated weights for policy 1, policy_version 1349293 (0.0007) [2023-12-27 15:00:16,189][20347] Updated weights for policy 0, policy_version 1350398 (0.0008) [2023-12-27 15:00:16,248][20347] Updated weights for policy 0, policy_version 1350408 (0.0010) [2023-12-27 15:00:16,304][20347] Updated weights for policy 0, policy_version 1350418 (0.0010) [2023-12-27 15:00:16,668][20381] Updated weights for policy 1, policy_version 1349303 (0.0007) [2023-12-27 15:00:16,731][20381] Updated weights for policy 1, policy_version 1349313 (0.0005) [2023-12-27 15:00:16,786][20381] Updated weights for policy 1, policy_version 1349323 (0.0005) [2023-12-27 15:00:16,914][20347] Updated weights for policy 0, policy_version 1350428 (0.0008) [2023-12-27 15:00:16,976][20347] Updated weights for policy 0, policy_version 1350438 (0.0005) [2023-12-27 15:00:17,019][20347] Updated weights for policy 0, policy_version 1350448 (0.0005) [2023-12-27 15:00:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 691240960. Throughput: 0: 10328.0, 1: 10242.2. Samples: 691253836. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:17,363][19170] Avg episode reward: [(0, '853.450'), (1, '716.730')] [2023-12-27 15:00:17,388][20381] Updated weights for policy 1, policy_version 1349333 (0.0005) [2023-12-27 15:00:17,443][20381] Updated weights for policy 1, policy_version 1349343 (0.0005) [2023-12-27 15:00:17,501][20381] Updated weights for policy 1, policy_version 1349353 (0.0005) [2023-12-27 15:00:17,635][20347] Updated weights for policy 0, policy_version 1350458 (0.0008) [2023-12-27 15:00:17,700][20347] Updated weights for policy 0, policy_version 1350468 (0.0011) [2023-12-27 15:00:17,768][20347] Updated weights for policy 0, policy_version 1350478 (0.0011) [2023-12-27 15:00:17,835][20347] Updated weights for policy 0, policy_version 1350488 (0.0010) [2023-12-27 15:00:18,076][20381] Updated weights for policy 1, policy_version 1349363 (0.0006) [2023-12-27 15:00:18,136][20381] Updated weights for policy 1, policy_version 1349373 (0.0008) [2023-12-27 15:00:18,190][20381] Updated weights for policy 1, policy_version 1349383 (0.0008) [2023-12-27 15:00:18,478][20347] Updated weights for policy 0, policy_version 1350498 (0.0008) [2023-12-27 15:00:18,541][20347] Updated weights for policy 0, policy_version 1350508 (0.0009) [2023-12-27 15:00:18,604][20347] Updated weights for policy 0, policy_version 1350518 (0.0010) [2023-12-27 15:00:18,862][20381] Updated weights for policy 1, policy_version 1349393 (0.0011) [2023-12-27 15:00:18,914][20381] Updated weights for policy 1, policy_version 1349403 (0.0011) [2023-12-27 15:00:18,965][20381] Updated weights for policy 1, policy_version 1349413 (0.0010) [2023-12-27 15:00:19,013][20381] Updated weights for policy 1, policy_version 1349423 (0.0005) [2023-12-27 15:00:19,222][20347] Updated weights for policy 0, policy_version 1350528 (0.0010) [2023-12-27 15:00:19,287][20347] Updated weights for policy 0, policy_version 1350538 (0.0008) [2023-12-27 15:00:19,353][20347] Updated weights for policy 0, policy_version 1350548 (0.0008) [2023-12-27 15:00:19,720][20381] Updated weights for policy 1, policy_version 1349433 (0.0007) [2023-12-27 15:00:19,781][20381] Updated weights for policy 1, policy_version 1349443 (0.0007) [2023-12-27 15:00:19,850][20381] Updated weights for policy 1, policy_version 1349453 (0.0015) [2023-12-27 15:00:20,141][20347] Updated weights for policy 0, policy_version 1350558 (0.0008) [2023-12-27 15:00:20,206][20347] Updated weights for policy 0, policy_version 1350568 (0.0008) [2023-12-27 15:00:20,265][20347] Updated weights for policy 0, policy_version 1350578 (0.0009) [2023-12-27 15:00:20,532][20381] Updated weights for policy 1, policy_version 1349463 (0.0007) [2023-12-27 15:00:20,596][20381] Updated weights for policy 1, policy_version 1349473 (0.0008) [2023-12-27 15:00:20,660][20381] Updated weights for policy 1, policy_version 1349483 (0.0007) [2023-12-27 15:00:20,959][20347] Updated weights for policy 0, policy_version 1350588 (0.0009) [2023-12-27 15:00:21,006][20347] Updated weights for policy 0, policy_version 1350598 (0.0011) [2023-12-27 15:00:21,070][20347] Updated weights for policy 0, policy_version 1350608 (0.0009) [2023-12-27 15:00:21,367][20381] Updated weights for policy 1, policy_version 1349493 (0.0008) [2023-12-27 15:00:21,433][20381] Updated weights for policy 1, policy_version 1349503 (0.0008) [2023-12-27 15:00:21,489][20381] Updated weights for policy 1, policy_version 1349513 (0.0008) [2023-12-27 15:00:21,818][20347] Updated weights for policy 0, policy_version 1350618 (0.0010) [2023-12-27 15:00:21,885][20347] Updated weights for policy 0, policy_version 1350628 (0.0006) [2023-12-27 15:00:21,943][20347] Updated weights for policy 0, policy_version 1350638 (0.0006) [2023-12-27 15:00:22,006][20347] Updated weights for policy 0, policy_version 1350648 (0.0006) [2023-12-27 15:00:22,229][20381] Updated weights for policy 1, policy_version 1349523 (0.0008) [2023-12-27 15:00:22,297][20381] Updated weights for policy 1, policy_version 1349533 (0.0008) [2023-12-27 15:00:22,363][20381] Updated weights for policy 1, policy_version 1349543 (0.0009) [2023-12-27 15:00:22,364][19170] Fps is (10 sec: 20477.7, 60 sec: 20479.6, 300 sec: 20577.1). Total num frames: 691339264. Throughput: 0: 10330.8, 1: 10246.0. Samples: 691315612. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:22,364][19170] Avg episode reward: [(0, '892.480'), (1, '687.560')] [2023-12-27 15:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001350648_345833472.pth... [2023-12-27 15:00:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001349432_345522176.pth [2023-12-27 15:00:22,415][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001349552_345513984.pth... [2023-12-27 15:00:22,418][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001348336_345202688.pth [2023-12-27 15:00:22,619][20347] Updated weights for policy 0, policy_version 1350658 (0.0009) [2023-12-27 15:00:22,682][20347] Updated weights for policy 0, policy_version 1350668 (0.0005) [2023-12-27 15:00:22,743][20347] Updated weights for policy 0, policy_version 1350678 (0.0006) [2023-12-27 15:00:23,088][20381] Updated weights for policy 1, policy_version 1349553 (0.0009) [2023-12-27 15:00:23,142][20381] Updated weights for policy 1, policy_version 1349563 (0.0010) [2023-12-27 15:00:23,192][20381] Updated weights for policy 1, policy_version 1349573 (0.0009) [2023-12-27 15:00:23,253][20381] Updated weights for policy 1, policy_version 1349583 (0.0010) [2023-12-27 15:00:23,290][20347] Updated weights for policy 0, policy_version 1350688 (0.0009) [2023-12-27 15:00:23,339][20347] Updated weights for policy 0, policy_version 1350698 (0.0007) [2023-12-27 15:00:23,395][20347] Updated weights for policy 0, policy_version 1350708 (0.0005) [2023-12-27 15:00:23,942][20381] Updated weights for policy 1, policy_version 1349593 (0.0007) [2023-12-27 15:00:24,006][20381] Updated weights for policy 1, policy_version 1349603 (0.0005) [2023-12-27 15:00:24,008][20347] Updated weights for policy 0, policy_version 1350718 (0.0006) [2023-12-27 15:00:24,060][20347] Updated weights for policy 0, policy_version 1350728 (0.0007) [2023-12-27 15:00:24,060][20381] Updated weights for policy 1, policy_version 1349613 (0.0005) [2023-12-27 15:00:24,113][20347] Updated weights for policy 0, policy_version 1350738 (0.0007) [2023-12-27 15:00:24,650][20381] Updated weights for policy 1, policy_version 1349623 (0.0009) [2023-12-27 15:00:24,716][20381] Updated weights for policy 1, policy_version 1349633 (0.0007) [2023-12-27 15:00:24,730][20347] Updated weights for policy 0, policy_version 1350748 (0.0008) [2023-12-27 15:00:24,775][20381] Updated weights for policy 1, policy_version 1349643 (0.0006) [2023-12-27 15:00:24,784][20347] Updated weights for policy 0, policy_version 1350758 (0.0007) [2023-12-27 15:00:24,837][20347] Updated weights for policy 0, policy_version 1350768 (0.0007) [2023-12-27 15:00:25,417][20381] Updated weights for policy 1, policy_version 1349653 (0.0008) [2023-12-27 15:00:25,448][20347] Updated weights for policy 0, policy_version 1350778 (0.0008) [2023-12-27 15:00:25,469][20381] Updated weights for policy 1, policy_version 1349663 (0.0009) [2023-12-27 15:00:25,482][20292] KL-divergence is very high: 110.3510 [2023-12-27 15:00:25,497][20347] Updated weights for policy 0, policy_version 1350788 (0.0010) [2023-12-27 15:00:25,516][20381] Updated weights for policy 1, policy_version 1349673 (0.0008) [2023-12-27 15:00:25,545][20347] Updated weights for policy 0, policy_version 1350798 (0.0010) [2023-12-27 15:00:25,596][20347] Updated weights for policy 0, policy_version 1350808 (0.0010) [2023-12-27 15:00:26,166][20381] Updated weights for policy 1, policy_version 1349683 (0.0007) [2023-12-27 15:00:26,212][20381] Updated weights for policy 1, policy_version 1349693 (0.0010) [2023-12-27 15:00:26,229][20347] Updated weights for policy 0, policy_version 1350818 (0.0005) [2023-12-27 15:00:26,268][20381] Updated weights for policy 1, policy_version 1349703 (0.0006) [2023-12-27 15:00:26,283][20347] Updated weights for policy 0, policy_version 1350828 (0.0005) [2023-12-27 15:00:26,331][20347] Updated weights for policy 0, policy_version 1350838 (0.0005) [2023-12-27 15:00:26,914][20381] Updated weights for policy 1, policy_version 1349713 (0.0009) [2023-12-27 15:00:26,931][20347] Updated weights for policy 0, policy_version 1350848 (0.0005) [2023-12-27 15:00:26,975][20381] Updated weights for policy 1, policy_version 1349723 (0.0010) [2023-12-27 15:00:26,983][20347] Updated weights for policy 0, policy_version 1350858 (0.0006) [2023-12-27 15:00:27,029][20381] Updated weights for policy 1, policy_version 1349733 (0.0010) [2023-12-27 15:00:27,032][20347] Updated weights for policy 0, policy_version 1350868 (0.0005) [2023-12-27 15:00:27,087][20381] Updated weights for policy 1, policy_version 1349743 (0.0010) [2023-12-27 15:00:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 691453952. Throughput: 0: 10414.9, 1: 10211.3. Samples: 691441184. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:27,363][19170] Avg episode reward: [(0, '824.400'), (1, '658.620')] [2023-12-27 15:00:27,584][20347] Updated weights for policy 0, policy_version 1350878 (0.0006) [2023-12-27 15:00:27,600][20291] KL-divergence is very high: 190.1467 [2023-12-27 15:00:27,640][20347] Updated weights for policy 0, policy_version 1350888 (0.0005) [2023-12-27 15:00:27,645][20291] KL-divergence is very high: 241.8199 [2023-12-27 15:00:27,683][20291] KL-divergence is very high: 212.1030 [2023-12-27 15:00:27,689][20347] Updated weights for policy 0, policy_version 1350898 (0.0005) [2023-12-27 15:00:27,786][20381] Updated weights for policy 1, policy_version 1349753 (0.0008) [2023-12-27 15:00:27,847][20381] Updated weights for policy 1, policy_version 1349763 (0.0008) [2023-12-27 15:00:27,912][20381] Updated weights for policy 1, policy_version 1349773 (0.0008) [2023-12-27 15:00:28,208][20347] Updated weights for policy 0, policy_version 1350908 (0.0006) [2023-12-27 15:00:28,260][20347] Updated weights for policy 0, policy_version 1350918 (0.0006) [2023-12-27 15:00:28,317][20347] Updated weights for policy 0, policy_version 1350928 (0.0006) [2023-12-27 15:00:28,526][20381] Updated weights for policy 1, policy_version 1349783 (0.0009) [2023-12-27 15:00:28,581][20381] Updated weights for policy 1, policy_version 1349793 (0.0010) [2023-12-27 15:00:28,639][20381] Updated weights for policy 1, policy_version 1349803 (0.0010) [2023-12-27 15:00:28,961][20347] Updated weights for policy 0, policy_version 1350938 (0.0008) [2023-12-27 15:00:29,013][20347] Updated weights for policy 0, policy_version 1350948 (0.0007) [2023-12-27 15:00:29,089][20347] Updated weights for policy 0, policy_version 1350958 (0.0008) [2023-12-27 15:00:29,139][20347] Updated weights for policy 0, policy_version 1350968 (0.0008) [2023-12-27 15:00:29,291][20381] Updated weights for policy 1, policy_version 1349813 (0.0010) [2023-12-27 15:00:29,349][20381] Updated weights for policy 1, policy_version 1349823 (0.0009) [2023-12-27 15:00:29,418][20381] Updated weights for policy 1, policy_version 1349833 (0.0009) [2023-12-27 15:00:29,894][20347] Updated weights for policy 0, policy_version 1350979 (0.0009) [2023-12-27 15:00:29,958][20347] Updated weights for policy 0, policy_version 1350989 (0.0007) [2023-12-27 15:00:30,017][20347] Updated weights for policy 0, policy_version 1350999 (0.0008) [2023-12-27 15:00:30,104][20381] Updated weights for policy 1, policy_version 1349843 (0.0007) [2023-12-27 15:00:30,163][20381] Updated weights for policy 1, policy_version 1349853 (0.0007) [2023-12-27 15:00:30,215][20381] Updated weights for policy 1, policy_version 1349863 (0.0006) [2023-12-27 15:00:30,762][20347] Updated weights for policy 0, policy_version 1351009 (0.0006) [2023-12-27 15:00:30,820][20347] Updated weights for policy 0, policy_version 1351019 (0.0006) [2023-12-27 15:00:30,878][20347] Updated weights for policy 0, policy_version 1351029 (0.0006) [2023-12-27 15:00:30,879][20381] Updated weights for policy 1, policy_version 1349873 (0.0006) [2023-12-27 15:00:30,937][20381] Updated weights for policy 1, policy_version 1349883 (0.0007) [2023-12-27 15:00:31,003][20381] Updated weights for policy 1, policy_version 1349893 (0.0008) [2023-12-27 15:00:31,070][20381] Updated weights for policy 1, policy_version 1349903 (0.0008) [2023-12-27 15:00:31,563][20347] Updated weights for policy 0, policy_version 1351039 (0.0009) [2023-12-27 15:00:31,626][20347] Updated weights for policy 0, policy_version 1351049 (0.0008) [2023-12-27 15:00:31,685][20347] Updated weights for policy 0, policy_version 1351059 (0.0007) [2023-12-27 15:00:31,791][20381] Updated weights for policy 1, policy_version 1349913 (0.0007) [2023-12-27 15:00:31,854][20381] Updated weights for policy 1, policy_version 1349923 (0.0007) [2023-12-27 15:00:31,917][20381] Updated weights for policy 1, policy_version 1349933 (0.0008) [2023-12-27 15:00:32,347][20347] Updated weights for policy 0, policy_version 1351069 (0.0007) [2023-12-27 15:00:32,362][19170] Fps is (10 sec: 21301.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 691552256. Throughput: 0: 10516.5, 1: 10129.5. Samples: 691566352. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:32,363][19170] Avg episode reward: [(0, '853.700'), (1, '609.520')] [2023-12-27 15:00:32,411][20347] Updated weights for policy 0, policy_version 1351079 (0.0008) [2023-12-27 15:00:32,475][20347] Updated weights for policy 0, policy_version 1351089 (0.0008) [2023-12-27 15:00:32,584][20381] Updated weights for policy 1, policy_version 1349943 (0.0008) [2023-12-27 15:00:32,642][20381] Updated weights for policy 1, policy_version 1349953 (0.0007) [2023-12-27 15:00:32,699][20381] Updated weights for policy 1, policy_version 1349963 (0.0007) [2023-12-27 15:00:33,141][20347] Updated weights for policy 0, policy_version 1351099 (0.0007) [2023-12-27 15:00:33,197][20347] Updated weights for policy 0, policy_version 1351109 (0.0005) [2023-12-27 15:00:33,255][20347] Updated weights for policy 0, policy_version 1351119 (0.0005) [2023-12-27 15:00:33,428][20381] Updated weights for policy 1, policy_version 1349973 (0.0006) [2023-12-27 15:00:33,481][20381] Updated weights for policy 1, policy_version 1349983 (0.0006) [2023-12-27 15:00:33,528][20381] Updated weights for policy 1, policy_version 1349993 (0.0007) [2023-12-27 15:00:33,875][20347] Updated weights for policy 0, policy_version 1351129 (0.0006) [2023-12-27 15:00:33,923][20347] Updated weights for policy 0, policy_version 1351139 (0.0010) [2023-12-27 15:00:33,983][20347] Updated weights for policy 0, policy_version 1351149 (0.0009) [2023-12-27 15:00:34,050][20347] Updated weights for policy 0, policy_version 1351159 (0.0010) [2023-12-27 15:00:34,179][20381] Updated weights for policy 1, policy_version 1350003 (0.0008) [2023-12-27 15:00:34,234][20381] Updated weights for policy 1, policy_version 1350013 (0.0008) [2023-12-27 15:00:34,284][20381] Updated weights for policy 1, policy_version 1350023 (0.0008) [2023-12-27 15:00:34,748][20347] Updated weights for policy 0, policy_version 1351169 (0.0008) [2023-12-27 15:00:34,813][20347] Updated weights for policy 0, policy_version 1351179 (0.0009) [2023-12-27 15:00:34,870][20347] Updated weights for policy 0, policy_version 1351189 (0.0008) [2023-12-27 15:00:35,015][20381] Updated weights for policy 1, policy_version 1350033 (0.0008) [2023-12-27 15:00:35,078][20381] Updated weights for policy 1, policy_version 1350043 (0.0008) [2023-12-27 15:00:35,135][20381] Updated weights for policy 1, policy_version 1350053 (0.0007) [2023-12-27 15:00:35,186][20381] Updated weights for policy 1, policy_version 1350063 (0.0007) [2023-12-27 15:00:35,559][20347] Updated weights for policy 0, policy_version 1351199 (0.0011) [2023-12-27 15:00:35,612][20347] Updated weights for policy 0, policy_version 1351209 (0.0010) [2023-12-27 15:00:35,661][20347] Updated weights for policy 0, policy_version 1351219 (0.0010) [2023-12-27 15:00:35,892][20381] Updated weights for policy 1, policy_version 1350073 (0.0008) [2023-12-27 15:00:35,951][20381] Updated weights for policy 1, policy_version 1350083 (0.0008) [2023-12-27 15:00:36,008][20381] Updated weights for policy 1, policy_version 1350093 (0.0007) [2023-12-27 15:00:36,392][20347] Updated weights for policy 0, policy_version 1351229 (0.0009) [2023-12-27 15:00:36,451][20347] Updated weights for policy 0, policy_version 1351239 (0.0008) [2023-12-27 15:00:36,511][20347] Updated weights for policy 0, policy_version 1351249 (0.0008) [2023-12-27 15:00:36,746][20381] Updated weights for policy 1, policy_version 1350103 (0.0007) [2023-12-27 15:00:36,796][20381] Updated weights for policy 1, policy_version 1350113 (0.0008) [2023-12-27 15:00:36,857][20381] Updated weights for policy 1, policy_version 1350123 (0.0007) [2023-12-27 15:00:37,221][20347] Updated weights for policy 0, policy_version 1351259 (0.0008) [2023-12-27 15:00:37,291][20347] Updated weights for policy 0, policy_version 1351269 (0.0008) [2023-12-27 15:00:37,355][20347] Updated weights for policy 0, policy_version 1351279 (0.0008) [2023-12-27 15:00:37,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 691650560. Throughput: 0: 10486.3, 1: 10136.1. Samples: 691626580. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:37,363][19170] Avg episode reward: [(0, '931.970'), (1, '726.630')] [2023-12-27 15:00:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001350128_345661440.pth... [2023-12-27 15:00:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001348944_345358336.pth [2023-12-27 15:00:37,411][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001351288_345997312.pth... [2023-12-27 15:00:37,415][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001350072_345686016.pth [2023-12-27 15:00:37,651][20381] Updated weights for policy 1, policy_version 1350133 (0.0008) [2023-12-27 15:00:37,713][20381] Updated weights for policy 1, policy_version 1350143 (0.0007) [2023-12-27 15:00:37,770][20381] Updated weights for policy 1, policy_version 1350153 (0.0008) [2023-12-27 15:00:38,101][20347] Updated weights for policy 0, policy_version 1351289 (0.0008) [2023-12-27 15:00:38,155][20347] Updated weights for policy 0, policy_version 1351299 (0.0006) [2023-12-27 15:00:38,212][20347] Updated weights for policy 0, policy_version 1351309 (0.0005) [2023-12-27 15:00:38,270][20347] Updated weights for policy 0, policy_version 1351319 (0.0006) [2023-12-27 15:00:38,506][20381] Updated weights for policy 1, policy_version 1350163 (0.0007) [2023-12-27 15:00:38,566][20381] Updated weights for policy 1, policy_version 1350173 (0.0006) [2023-12-27 15:00:38,619][20381] Updated weights for policy 1, policy_version 1350183 (0.0008) [2023-12-27 15:00:38,970][20347] Updated weights for policy 0, policy_version 1351329 (0.0008) [2023-12-27 15:00:39,024][20347] Updated weights for policy 0, policy_version 1351339 (0.0008) [2023-12-27 15:00:39,077][20347] Updated weights for policy 0, policy_version 1351349 (0.0008) [2023-12-27 15:00:39,312][20381] Updated weights for policy 1, policy_version 1350193 (0.0008) [2023-12-27 15:00:39,384][20381] Updated weights for policy 1, policy_version 1350203 (0.0008) [2023-12-27 15:00:39,460][20381] Updated weights for policy 1, policy_version 1350213 (0.0008) [2023-12-27 15:00:39,516][20381] Updated weights for policy 1, policy_version 1350223 (0.0007) [2023-12-27 15:00:39,839][20347] Updated weights for policy 0, policy_version 1351359 (0.0008) [2023-12-27 15:00:39,903][20347] Updated weights for policy 0, policy_version 1351369 (0.0009) [2023-12-27 15:00:39,971][20347] Updated weights for policy 0, policy_version 1351379 (0.0008) [2023-12-27 15:00:40,277][20381] Updated weights for policy 1, policy_version 1350233 (0.0009) [2023-12-27 15:00:40,346][20381] Updated weights for policy 1, policy_version 1350243 (0.0007) [2023-12-27 15:00:40,408][20381] Updated weights for policy 1, policy_version 1350253 (0.0008) [2023-12-27 15:00:40,607][20347] Updated weights for policy 0, policy_version 1351389 (0.0008) [2023-12-27 15:00:40,654][20347] Updated weights for policy 0, policy_version 1351399 (0.0010) [2023-12-27 15:00:40,710][20347] Updated weights for policy 0, policy_version 1351409 (0.0007) [2023-12-27 15:00:41,086][20381] Updated weights for policy 1, policy_version 1350263 (0.0008) [2023-12-27 15:00:41,140][20381] Updated weights for policy 1, policy_version 1350273 (0.0008) [2023-12-27 15:00:41,210][20381] Updated weights for policy 1, policy_version 1350283 (0.0008) [2023-12-27 15:00:41,485][20347] Updated weights for policy 0, policy_version 1351419 (0.0008) [2023-12-27 15:00:41,552][20347] Updated weights for policy 0, policy_version 1351429 (0.0006) [2023-12-27 15:00:41,617][20347] Updated weights for policy 0, policy_version 1351439 (0.0007) [2023-12-27 15:00:41,959][20381] Updated weights for policy 1, policy_version 1350293 (0.0008) [2023-12-27 15:00:42,017][20381] Updated weights for policy 1, policy_version 1350303 (0.0007) [2023-12-27 15:00:42,060][20292] KL-divergence is very high: 114.1088 [2023-12-27 15:00:42,078][20381] Updated weights for policy 1, policy_version 1350313 (0.0008) [2023-12-27 15:00:42,303][20347] Updated weights for policy 0, policy_version 1351449 (0.0007) [2023-12-27 15:00:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 691748864. Throughput: 0: 10330.2, 1: 10033.2. Samples: 691742204. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:42,363][19170] Avg episode reward: [(0, '892.780'), (1, '784.940')] [2023-12-27 15:00:42,373][20347] Updated weights for policy 0, policy_version 1351459 (0.0008) [2023-12-27 15:00:42,441][20347] Updated weights for policy 0, policy_version 1351469 (0.0008) [2023-12-27 15:00:42,504][20347] Updated weights for policy 0, policy_version 1351479 (0.0008) [2023-12-27 15:00:42,775][20381] Updated weights for policy 1, policy_version 1350323 (0.0007) [2023-12-27 15:00:42,828][20381] Updated weights for policy 1, policy_version 1350333 (0.0006) [2023-12-27 15:00:42,886][20381] Updated weights for policy 1, policy_version 1350343 (0.0009) [2023-12-27 15:00:43,144][20347] Updated weights for policy 0, policy_version 1351489 (0.0008) [2023-12-27 15:00:43,198][20347] Updated weights for policy 0, policy_version 1351499 (0.0006) [2023-12-27 15:00:43,253][20347] Updated weights for policy 0, policy_version 1351509 (0.0007) [2023-12-27 15:00:43,497][20381] Updated weights for policy 1, policy_version 1350353 (0.0008) [2023-12-27 15:00:43,561][20381] Updated weights for policy 1, policy_version 1350363 (0.0007) [2023-12-27 15:00:43,625][20381] Updated weights for policy 1, policy_version 1350373 (0.0009) [2023-12-27 15:00:43,682][20381] Updated weights for policy 1, policy_version 1350383 (0.0007) [2023-12-27 15:00:43,950][20347] Updated weights for policy 0, policy_version 1351519 (0.0007) [2023-12-27 15:00:43,997][20347] Updated weights for policy 0, policy_version 1351529 (0.0007) [2023-12-27 15:00:44,047][20347] Updated weights for policy 0, policy_version 1351539 (0.0006) [2023-12-27 15:00:44,306][20381] Updated weights for policy 1, policy_version 1350393 (0.0010) [2023-12-27 15:00:44,357][20381] Updated weights for policy 1, policy_version 1350403 (0.0010) [2023-12-27 15:00:44,411][20381] Updated weights for policy 1, policy_version 1350413 (0.0010) [2023-12-27 15:00:44,632][20347] Updated weights for policy 0, policy_version 1351549 (0.0008) [2023-12-27 15:00:44,677][20347] Updated weights for policy 0, policy_version 1351559 (0.0009) [2023-12-27 15:00:44,724][20347] Updated weights for policy 0, policy_version 1351569 (0.0009) [2023-12-27 15:00:45,064][20381] Updated weights for policy 1, policy_version 1350423 (0.0010) [2023-12-27 15:00:45,127][20381] Updated weights for policy 1, policy_version 1350433 (0.0011) [2023-12-27 15:00:45,190][20381] Updated weights for policy 1, policy_version 1350443 (0.0011) [2023-12-27 15:00:45,369][20347] Updated weights for policy 0, policy_version 1351579 (0.0007) [2023-12-27 15:00:45,426][20347] Updated weights for policy 0, policy_version 1351589 (0.0010) [2023-12-27 15:00:45,443][20291] KL-divergence is very high: 101.0266 [2023-12-27 15:00:45,483][20347] Updated weights for policy 0, policy_version 1351599 (0.0010) [2023-12-27 15:00:45,837][20381] Updated weights for policy 1, policy_version 1350453 (0.0008) [2023-12-27 15:00:45,887][20381] Updated weights for policy 1, policy_version 1350463 (0.0005) [2023-12-27 15:00:45,937][20381] Updated weights for policy 1, policy_version 1350473 (0.0005) [2023-12-27 15:00:46,126][20347] Updated weights for policy 0, policy_version 1351609 (0.0009) [2023-12-27 15:00:46,186][20347] Updated weights for policy 0, policy_version 1351619 (0.0005) [2023-12-27 15:00:46,238][20347] Updated weights for policy 0, policy_version 1351629 (0.0007) [2023-12-27 15:00:46,298][20347] Updated weights for policy 0, policy_version 1351639 (0.0006) [2023-12-27 15:00:46,520][20381] Updated weights for policy 1, policy_version 1350483 (0.0007) [2023-12-27 15:00:46,578][20381] Updated weights for policy 1, policy_version 1350493 (0.0009) [2023-12-27 15:00:46,630][20381] Updated weights for policy 1, policy_version 1350503 (0.0006) [2023-12-27 15:00:46,841][20347] Updated weights for policy 0, policy_version 1351649 (0.0010) [2023-12-27 15:00:46,889][20347] Updated weights for policy 0, policy_version 1351659 (0.0010) [2023-12-27 15:00:46,924][20291] KL-divergence is very high: 106.5978 [2023-12-27 15:00:46,945][20347] Updated weights for policy 0, policy_version 1351669 (0.0007) [2023-12-27 15:00:47,272][20381] Updated weights for policy 1, policy_version 1350513 (0.0005) [2023-12-27 15:00:47,331][20381] Updated weights for policy 1, policy_version 1350523 (0.0008) [2023-12-27 15:00:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 691855360. Throughput: 0: 10366.3, 1: 10084.2. Samples: 691869220. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:47,364][19170] Avg episode reward: [(0, '883.210'), (1, '775.170')] [2023-12-27 15:00:47,380][20381] Updated weights for policy 1, policy_version 1350533 (0.0009) [2023-12-27 15:00:47,430][20381] Updated weights for policy 1, policy_version 1350543 (0.0007) [2023-12-27 15:00:47,577][20347] Updated weights for policy 0, policy_version 1351679 (0.0008) [2023-12-27 15:00:47,639][20347] Updated weights for policy 0, policy_version 1351689 (0.0007) [2023-12-27 15:00:47,714][20347] Updated weights for policy 0, policy_version 1351700 (0.0008) [2023-12-27 15:00:48,163][20381] Updated weights for policy 1, policy_version 1350553 (0.0008) [2023-12-27 15:00:48,220][20381] Updated weights for policy 1, policy_version 1350563 (0.0008) [2023-12-27 15:00:48,279][20381] Updated weights for policy 1, policy_version 1350573 (0.0008) [2023-12-27 15:00:48,376][20347] Updated weights for policy 0, policy_version 1351710 (0.0008) [2023-12-27 15:00:48,432][20347] Updated weights for policy 0, policy_version 1351720 (0.0007) [2023-12-27 15:00:48,493][20347] Updated weights for policy 0, policy_version 1351730 (0.0008) [2023-12-27 15:00:48,965][20381] Updated weights for policy 1, policy_version 1350583 (0.0006) [2023-12-27 15:00:49,023][20381] Updated weights for policy 1, policy_version 1350593 (0.0007) [2023-12-27 15:00:49,081][20381] Updated weights for policy 1, policy_version 1350603 (0.0008) [2023-12-27 15:00:49,196][20347] Updated weights for policy 0, policy_version 1351740 (0.0007) [2023-12-27 15:00:49,259][20347] Updated weights for policy 0, policy_version 1351750 (0.0008) [2023-12-27 15:00:49,315][20347] Updated weights for policy 0, policy_version 1351760 (0.0008) [2023-12-27 15:00:49,847][20381] Updated weights for policy 1, policy_version 1350613 (0.0007) [2023-12-27 15:00:49,895][20381] Updated weights for policy 1, policy_version 1350623 (0.0007) [2023-12-27 15:00:49,953][20381] Updated weights for policy 1, policy_version 1350633 (0.0009) [2023-12-27 15:00:50,013][20347] Updated weights for policy 0, policy_version 1351770 (0.0008) [2023-12-27 15:00:50,068][20347] Updated weights for policy 0, policy_version 1351780 (0.0007) [2023-12-27 15:00:50,120][20347] Updated weights for policy 0, policy_version 1351790 (0.0007) [2023-12-27 15:00:50,179][20347] Updated weights for policy 0, policy_version 1351800 (0.0007) [2023-12-27 15:00:50,675][20381] Updated weights for policy 1, policy_version 1350643 (0.0006) [2023-12-27 15:00:50,733][20381] Updated weights for policy 1, policy_version 1350653 (0.0008) [2023-12-27 15:00:50,796][20381] Updated weights for policy 1, policy_version 1350663 (0.0008) [2023-12-27 15:00:50,904][20347] Updated weights for policy 0, policy_version 1351810 (0.0009) [2023-12-27 15:00:50,958][20347] Updated weights for policy 0, policy_version 1351820 (0.0008) [2023-12-27 15:00:51,013][20347] Updated weights for policy 0, policy_version 1351830 (0.0007) [2023-12-27 15:00:51,526][20381] Updated weights for policy 1, policy_version 1350673 (0.0008) [2023-12-27 15:00:51,584][20381] Updated weights for policy 1, policy_version 1350683 (0.0008) [2023-12-27 15:00:51,652][20381] Updated weights for policy 1, policy_version 1350693 (0.0007) [2023-12-27 15:00:51,709][20381] Updated weights for policy 1, policy_version 1350703 (0.0006) [2023-12-27 15:00:51,785][20347] Updated weights for policy 0, policy_version 1351840 (0.0009) [2023-12-27 15:00:51,848][20347] Updated weights for policy 0, policy_version 1351850 (0.0008) [2023-12-27 15:00:51,906][20347] Updated weights for policy 0, policy_version 1351860 (0.0008) [2023-12-27 15:00:52,308][20381] Updated weights for policy 1, policy_version 1350713 (0.0008) [2023-12-27 15:00:52,363][20381] Updated weights for policy 1, policy_version 1350723 (0.0008) [2023-12-27 15:00:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 691953664. Throughput: 0: 10327.4, 1: 10083.2. Samples: 691929608. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:52,363][19170] Avg episode reward: [(0, '892.830'), (1, '677.580')] [2023-12-27 15:00:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001351864_346144768.pth... [2023-12-27 15:00:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001350648_345833472.pth [2023-12-27 15:00:52,431][20381] Updated weights for policy 1, policy_version 1350733 (0.0008) [2023-12-27 15:00:52,447][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001350736_345817088.pth... [2023-12-27 15:00:52,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001349552_345513984.pth [2023-12-27 15:00:52,552][20347] Updated weights for policy 0, policy_version 1351870 (0.0007) [2023-12-27 15:00:52,617][20347] Updated weights for policy 0, policy_version 1351880 (0.0007) [2023-12-27 15:00:52,680][20347] Updated weights for policy 0, policy_version 1351890 (0.0008) [2023-12-27 15:00:53,135][20381] Updated weights for policy 1, policy_version 1350743 (0.0009) [2023-12-27 15:00:53,182][20381] Updated weights for policy 1, policy_version 1350753 (0.0007) [2023-12-27 15:00:53,230][20381] Updated weights for policy 1, policy_version 1350763 (0.0008) [2023-12-27 15:00:53,376][20347] Updated weights for policy 0, policy_version 1351900 (0.0008) [2023-12-27 15:00:53,427][20347] Updated weights for policy 0, policy_version 1351910 (0.0008) [2023-12-27 15:00:53,480][20347] Updated weights for policy 0, policy_version 1351920 (0.0007) [2023-12-27 15:00:53,858][20381] Updated weights for policy 1, policy_version 1350773 (0.0006) [2023-12-27 15:00:53,915][20381] Updated weights for policy 1, policy_version 1350783 (0.0009) [2023-12-27 15:00:53,970][20381] Updated weights for policy 1, policy_version 1350793 (0.0010) [2023-12-27 15:00:54,205][20347] Updated weights for policy 0, policy_version 1351930 (0.0008) [2023-12-27 15:00:54,254][20347] Updated weights for policy 0, policy_version 1351940 (0.0008) [2023-12-27 15:00:54,303][20347] Updated weights for policy 0, policy_version 1351950 (0.0007) [2023-12-27 15:00:54,358][20347] Updated weights for policy 0, policy_version 1351960 (0.0007) [2023-12-27 15:00:54,637][20381] Updated weights for policy 1, policy_version 1350803 (0.0011) [2023-12-27 15:00:54,695][20381] Updated weights for policy 1, policy_version 1350813 (0.0011) [2023-12-27 15:00:54,743][20381] Updated weights for policy 1, policy_version 1350823 (0.0010) [2023-12-27 15:00:54,998][20347] Updated weights for policy 0, policy_version 1351970 (0.0008) [2023-12-27 15:00:55,053][20347] Updated weights for policy 0, policy_version 1351980 (0.0008) [2023-12-27 15:00:55,110][20347] Updated weights for policy 0, policy_version 1351990 (0.0007) [2023-12-27 15:00:55,441][20381] Updated weights for policy 1, policy_version 1350833 (0.0009) [2023-12-27 15:00:55,489][20381] Updated weights for policy 1, policy_version 1350843 (0.0007) [2023-12-27 15:00:55,547][20381] Updated weights for policy 1, policy_version 1350853 (0.0010) [2023-12-27 15:00:55,598][20381] Updated weights for policy 1, policy_version 1350863 (0.0008) [2023-12-27 15:00:55,780][20347] Updated weights for policy 0, policy_version 1352000 (0.0007) [2023-12-27 15:00:55,840][20347] Updated weights for policy 0, policy_version 1352010 (0.0007) [2023-12-27 15:00:55,901][20347] Updated weights for policy 0, policy_version 1352020 (0.0009) [2023-12-27 15:00:56,235][20381] Updated weights for policy 1, policy_version 1350873 (0.0007) [2023-12-27 15:00:56,298][20381] Updated weights for policy 1, policy_version 1350883 (0.0008) [2023-12-27 15:00:56,355][20381] Updated weights for policy 1, policy_version 1350893 (0.0007) [2023-12-27 15:00:56,536][20347] Updated weights for policy 0, policy_version 1352030 (0.0007) [2023-12-27 15:00:56,588][20347] Updated weights for policy 0, policy_version 1352040 (0.0008) [2023-12-27 15:00:56,640][20347] Updated weights for policy 0, policy_version 1352050 (0.0008) [2023-12-27 15:00:57,024][20381] Updated weights for policy 1, policy_version 1350903 (0.0008) [2023-12-27 15:00:57,070][20381] Updated weights for policy 1, policy_version 1350913 (0.0009) [2023-12-27 15:00:57,115][20381] Updated weights for policy 1, policy_version 1350923 (0.0008) [2023-12-27 15:00:57,318][20347] Updated weights for policy 0, policy_version 1352060 (0.0008) [2023-12-27 15:00:57,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 692060160. Throughput: 0: 10295.9, 1: 10153.2. Samples: 692051456. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:00:57,363][19170] Avg episode reward: [(0, '882.690'), (1, '678.650')] [2023-12-27 15:00:57,370][20347] Updated weights for policy 0, policy_version 1352070 (0.0010) [2023-12-27 15:00:57,419][20347] Updated weights for policy 0, policy_version 1352080 (0.0010) [2023-12-27 15:00:57,791][20381] Updated weights for policy 1, policy_version 1350933 (0.0008) [2023-12-27 15:00:57,841][20381] Updated weights for policy 1, policy_version 1350943 (0.0009) [2023-12-27 15:00:57,888][20381] Updated weights for policy 1, policy_version 1350953 (0.0010) [2023-12-27 15:00:58,087][20347] Updated weights for policy 0, policy_version 1352090 (0.0010) [2023-12-27 15:00:58,145][20347] Updated weights for policy 0, policy_version 1352100 (0.0008) [2023-12-27 15:00:58,206][20347] Updated weights for policy 0, policy_version 1352110 (0.0008) [2023-12-27 15:00:58,259][20347] Updated weights for policy 0, policy_version 1352120 (0.0007) [2023-12-27 15:00:58,633][20381] Updated weights for policy 1, policy_version 1350963 (0.0009) [2023-12-27 15:00:58,700][20381] Updated weights for policy 1, policy_version 1350973 (0.0008) [2023-12-27 15:00:58,780][20381] Updated weights for policy 1, policy_version 1350983 (0.0008) [2023-12-27 15:00:59,057][20347] Updated weights for policy 0, policy_version 1352130 (0.0007) [2023-12-27 15:00:59,114][20347] Updated weights for policy 0, policy_version 1352140 (0.0007) [2023-12-27 15:00:59,173][20347] Updated weights for policy 0, policy_version 1352150 (0.0006) [2023-12-27 15:00:59,619][20381] Updated weights for policy 1, policy_version 1350993 (0.0009) [2023-12-27 15:00:59,672][20381] Updated weights for policy 1, policy_version 1351003 (0.0007) [2023-12-27 15:00:59,733][20381] Updated weights for policy 1, policy_version 1351013 (0.0006) [2023-12-27 15:00:59,789][20381] Updated weights for policy 1, policy_version 1351023 (0.0005) [2023-12-27 15:00:59,935][20347] Updated weights for policy 0, policy_version 1352160 (0.0006) [2023-12-27 15:00:59,982][20347] Updated weights for policy 0, policy_version 1352170 (0.0006) [2023-12-27 15:01:00,037][20347] Updated weights for policy 0, policy_version 1352180 (0.0007) [2023-12-27 15:01:00,494][20381] Updated weights for policy 1, policy_version 1351033 (0.0007) [2023-12-27 15:01:00,542][20381] Updated weights for policy 1, policy_version 1351043 (0.0008) [2023-12-27 15:01:00,604][20381] Updated weights for policy 1, policy_version 1351053 (0.0007) [2023-12-27 15:01:00,765][20347] Updated weights for policy 0, policy_version 1352190 (0.0007) [2023-12-27 15:01:00,815][20347] Updated weights for policy 0, policy_version 1352200 (0.0007) [2023-12-27 15:01:00,872][20347] Updated weights for policy 0, policy_version 1352210 (0.0007) [2023-12-27 15:01:01,298][20381] Updated weights for policy 1, policy_version 1351063 (0.0007) [2023-12-27 15:01:01,373][20381] Updated weights for policy 1, policy_version 1351073 (0.0006) [2023-12-27 15:01:01,429][20381] Updated weights for policy 1, policy_version 1351083 (0.0008) [2023-12-27 15:01:01,608][20347] Updated weights for policy 0, policy_version 1352220 (0.0007) [2023-12-27 15:01:01,670][20347] Updated weights for policy 0, policy_version 1352230 (0.0008) [2023-12-27 15:01:01,731][20347] Updated weights for policy 0, policy_version 1352240 (0.0008) [2023-12-27 15:01:02,166][20381] Updated weights for policy 1, policy_version 1351093 (0.0008) [2023-12-27 15:01:02,219][20381] Updated weights for policy 1, policy_version 1351103 (0.0007) [2023-12-27 15:01:02,281][20381] Updated weights for policy 1, policy_version 1351113 (0.0008) [2023-12-27 15:01:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 692158464. Throughput: 0: 10242.6, 1: 10082.1. Samples: 692168448. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:02,363][19170] Avg episode reward: [(0, '911.910'), (1, '765.830')] [2023-12-27 15:01:02,426][20347] Updated weights for policy 0, policy_version 1352250 (0.0007) [2023-12-27 15:01:02,489][20347] Updated weights for policy 0, policy_version 1352260 (0.0007) [2023-12-27 15:01:02,548][20347] Updated weights for policy 0, policy_version 1352270 (0.0007) [2023-12-27 15:01:02,600][20347] Updated weights for policy 0, policy_version 1352280 (0.0006) [2023-12-27 15:01:02,937][20381] Updated weights for policy 1, policy_version 1351124 (0.0007) [2023-12-27 15:01:02,988][20381] Updated weights for policy 1, policy_version 1351134 (0.0006) [2023-12-27 15:01:03,042][20381] Updated weights for policy 1, policy_version 1351144 (0.0007) [2023-12-27 15:01:03,174][20347] Updated weights for policy 0, policy_version 1352290 (0.0005) [2023-12-27 15:01:03,228][20347] Updated weights for policy 0, policy_version 1352300 (0.0005) [2023-12-27 15:01:03,276][20347] Updated weights for policy 0, policy_version 1352310 (0.0007) [2023-12-27 15:01:03,776][20381] Updated weights for policy 1, policy_version 1351154 (0.0007) [2023-12-27 15:01:03,839][20381] Updated weights for policy 1, policy_version 1351164 (0.0006) [2023-12-27 15:01:03,883][20347] Updated weights for policy 0, policy_version 1352320 (0.0008) [2023-12-27 15:01:03,899][20381] Updated weights for policy 1, policy_version 1351174 (0.0008) [2023-12-27 15:01:03,935][20347] Updated weights for policy 0, policy_version 1352330 (0.0008) [2023-12-27 15:01:03,959][20381] Updated weights for policy 1, policy_version 1351184 (0.0008) [2023-12-27 15:01:03,990][20347] Updated weights for policy 0, policy_version 1352340 (0.0008) [2023-12-27 15:01:04,694][20381] Updated weights for policy 1, policy_version 1351194 (0.0006) [2023-12-27 15:01:04,731][20347] Updated weights for policy 0, policy_version 1352350 (0.0006) [2023-12-27 15:01:04,751][20381] Updated weights for policy 1, policy_version 1351204 (0.0007) [2023-12-27 15:01:04,782][20347] Updated weights for policy 0, policy_version 1352360 (0.0008) [2023-12-27 15:01:04,810][20381] Updated weights for policy 1, policy_version 1351214 (0.0006) [2023-12-27 15:01:04,835][20347] Updated weights for policy 0, policy_version 1352370 (0.0006) [2023-12-27 15:01:05,452][20347] Updated weights for policy 0, policy_version 1352380 (0.0008) [2023-12-27 15:01:05,501][20347] Updated weights for policy 0, policy_version 1352390 (0.0006) [2023-12-27 15:01:05,507][20381] Updated weights for policy 1, policy_version 1351224 (0.0009) [2023-12-27 15:01:05,554][20347] Updated weights for policy 0, policy_version 1352400 (0.0005) [2023-12-27 15:01:05,562][20381] Updated weights for policy 1, policy_version 1351234 (0.0010) [2023-12-27 15:01:05,621][20381] Updated weights for policy 1, policy_version 1351244 (0.0010) [2023-12-27 15:01:06,282][20347] Updated weights for policy 0, policy_version 1352410 (0.0008) [2023-12-27 15:01:06,308][20381] Updated weights for policy 1, policy_version 1351254 (0.0009) [2023-12-27 15:01:06,346][20347] Updated weights for policy 0, policy_version 1352420 (0.0007) [2023-12-27 15:01:06,370][20381] Updated weights for policy 1, policy_version 1351264 (0.0007) [2023-12-27 15:01:06,413][20347] Updated weights for policy 0, policy_version 1352430 (0.0006) [2023-12-27 15:01:06,429][20381] Updated weights for policy 1, policy_version 1351274 (0.0008) [2023-12-27 15:01:06,466][20347] Updated weights for policy 0, policy_version 1352440 (0.0006) [2023-12-27 15:01:07,140][20381] Updated weights for policy 1, policy_version 1351284 (0.0007) [2023-12-27 15:01:07,146][20347] Updated weights for policy 0, policy_version 1352450 (0.0008) [2023-12-27 15:01:07,199][20347] Updated weights for policy 0, policy_version 1352460 (0.0006) [2023-12-27 15:01:07,200][20381] Updated weights for policy 1, policy_version 1351294 (0.0010) [2023-12-27 15:01:07,248][20347] Updated weights for policy 0, policy_version 1352470 (0.0008) [2023-12-27 15:01:07,252][20381] Updated weights for policy 1, policy_version 1351304 (0.0010) [2023-12-27 15:01:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 692264960. Throughput: 0: 10267.1, 1: 10051.5. Samples: 692229924. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:07,363][19170] Avg episode reward: [(0, '911.940'), (1, '785.040')] [2023-12-27 15:01:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001352472_346300416.pth... [2023-12-27 15:01:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001351312_345964544.pth... [2023-12-27 15:01:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001351288_345997312.pth [2023-12-27 15:01:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001350128_345661440.pth [2023-12-27 15:01:07,903][20381] Updated weights for policy 1, policy_version 1351314 (0.0007) [2023-12-27 15:01:07,965][20381] Updated weights for policy 1, policy_version 1351324 (0.0007) [2023-12-27 15:01:07,983][20347] Updated weights for policy 0, policy_version 1352480 (0.0008) [2023-12-27 15:01:08,024][20381] Updated weights for policy 1, policy_version 1351334 (0.0007) [2023-12-27 15:01:08,051][20347] Updated weights for policy 0, policy_version 1352490 (0.0008) [2023-12-27 15:01:08,058][20291] KL-divergence is very high: 344.5898 [2023-12-27 15:01:08,078][20381] Updated weights for policy 1, policy_version 1351344 (0.0010) [2023-12-27 15:01:08,110][20291] KL-divergence is very high: 261.4333 [2023-12-27 15:01:08,118][20347] Updated weights for policy 0, policy_version 1352500 (0.0008) [2023-12-27 15:01:08,780][20381] Updated weights for policy 1, policy_version 1351354 (0.0006) [2023-12-27 15:01:08,781][20347] Updated weights for policy 0, policy_version 1352510 (0.0008) [2023-12-27 15:01:08,837][20381] Updated weights for policy 1, policy_version 1351364 (0.0006) [2023-12-27 15:01:08,839][20347] Updated weights for policy 0, policy_version 1352520 (0.0008) [2023-12-27 15:01:08,897][20381] Updated weights for policy 1, policy_version 1351374 (0.0008) [2023-12-27 15:01:08,900][20347] Updated weights for policy 0, policy_version 1352530 (0.0010) [2023-12-27 15:01:09,564][20381] Updated weights for policy 1, policy_version 1351384 (0.0008) [2023-12-27 15:01:09,579][20347] Updated weights for policy 0, policy_version 1352540 (0.0008) [2023-12-27 15:01:09,626][20381] Updated weights for policy 1, policy_version 1351394 (0.0007) [2023-12-27 15:01:09,652][20347] Updated weights for policy 0, policy_version 1352550 (0.0006) [2023-12-27 15:01:09,687][20381] Updated weights for policy 1, policy_version 1351404 (0.0008) [2023-12-27 15:01:09,715][20347] Updated weights for policy 0, policy_version 1352560 (0.0006) [2023-12-27 15:01:10,353][20347] Updated weights for policy 0, policy_version 1352570 (0.0006) [2023-12-27 15:01:10,367][20381] Updated weights for policy 1, policy_version 1351414 (0.0007) [2023-12-27 15:01:10,406][20347] Updated weights for policy 0, policy_version 1352580 (0.0005) [2023-12-27 15:01:10,427][20381] Updated weights for policy 1, policy_version 1351424 (0.0007) [2023-12-27 15:01:10,459][20347] Updated weights for policy 0, policy_version 1352590 (0.0006) [2023-12-27 15:01:10,483][20381] Updated weights for policy 1, policy_version 1351434 (0.0007) [2023-12-27 15:01:10,508][20347] Updated weights for policy 0, policy_version 1352600 (0.0005) [2023-12-27 15:01:11,158][20347] Updated weights for policy 0, policy_version 1352610 (0.0007) [2023-12-27 15:01:11,162][20381] Updated weights for policy 1, policy_version 1351444 (0.0007) [2023-12-27 15:01:11,216][20347] Updated weights for policy 0, policy_version 1352620 (0.0008) [2023-12-27 15:01:11,219][20381] Updated weights for policy 1, policy_version 1351454 (0.0006) [2023-12-27 15:01:11,278][20347] Updated weights for policy 0, policy_version 1352630 (0.0008) [2023-12-27 15:01:11,285][20381] Updated weights for policy 1, policy_version 1351464 (0.0007) [2023-12-27 15:01:11,955][20381] Updated weights for policy 1, policy_version 1351474 (0.0009) [2023-12-27 15:01:12,009][20381] Updated weights for policy 1, policy_version 1351484 (0.0011) [2023-12-27 15:01:12,055][20347] Updated weights for policy 0, policy_version 1352640 (0.0005) [2023-12-27 15:01:12,057][20381] Updated weights for policy 1, policy_version 1351494 (0.0010) [2023-12-27 15:01:12,107][20381] Updated weights for policy 1, policy_version 1351504 (0.0010) [2023-12-27 15:01:12,111][20347] Updated weights for policy 0, policy_version 1352650 (0.0007) [2023-12-27 15:01:12,164][20347] Updated weights for policy 0, policy_version 1352660 (0.0006) [2023-12-27 15:01:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 692363264. Throughput: 0: 10208.8, 1: 10028.5. Samples: 692351864. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:12,363][19170] Avg episode reward: [(0, '902.190'), (1, '736.410')] [2023-12-27 15:01:12,781][20381] Updated weights for policy 1, policy_version 1351514 (0.0006) [2023-12-27 15:01:12,838][20381] Updated weights for policy 1, policy_version 1351524 (0.0005) [2023-12-27 15:01:12,863][20347] Updated weights for policy 0, policy_version 1352670 (0.0006) [2023-12-27 15:01:12,891][20381] Updated weights for policy 1, policy_version 1351534 (0.0007) [2023-12-27 15:01:12,911][20347] Updated weights for policy 0, policy_version 1352680 (0.0005) [2023-12-27 15:01:12,967][20347] Updated weights for policy 0, policy_version 1352690 (0.0006) [2023-12-27 15:01:13,514][20347] Updated weights for policy 0, policy_version 1352700 (0.0008) [2023-12-27 15:01:13,516][20381] Updated weights for policy 1, policy_version 1351544 (0.0007) [2023-12-27 15:01:13,526][20292] KL-divergence is very high: 122.1596 [2023-12-27 15:01:13,538][20292] KL-divergence is very high: 150.4639 [2023-12-27 15:01:13,573][20381] Updated weights for policy 1, policy_version 1351554 (0.0008) [2023-12-27 15:01:13,574][20347] Updated weights for policy 0, policy_version 1352710 (0.0007) [2023-12-27 15:01:13,575][20292] KL-divergence is very high: 102.1211 [2023-12-27 15:01:13,633][20381] Updated weights for policy 1, policy_version 1351564 (0.0007) [2023-12-27 15:01:13,640][20347] Updated weights for policy 0, policy_version 1352720 (0.0009) [2023-12-27 15:01:14,271][20381] Updated weights for policy 1, policy_version 1351574 (0.0006) [2023-12-27 15:01:14,287][20347] Updated weights for policy 0, policy_version 1352730 (0.0009) [2023-12-27 15:01:14,327][20381] Updated weights for policy 1, policy_version 1351584 (0.0007) [2023-12-27 15:01:14,338][20347] Updated weights for policy 0, policy_version 1352740 (0.0010) [2023-12-27 15:01:14,384][20381] Updated weights for policy 1, policy_version 1351594 (0.0007) [2023-12-27 15:01:14,387][20347] Updated weights for policy 0, policy_version 1352750 (0.0010) [2023-12-27 15:01:14,435][20347] Updated weights for policy 0, policy_version 1352760 (0.0010) [2023-12-27 15:01:14,993][20381] Updated weights for policy 1, policy_version 1351604 (0.0008) [2023-12-27 15:01:15,052][20381] Updated weights for policy 1, policy_version 1351614 (0.0008) [2023-12-27 15:01:15,108][20381] Updated weights for policy 1, policy_version 1351624 (0.0007) [2023-12-27 15:01:15,147][20347] Updated weights for policy 0, policy_version 1352770 (0.0010) [2023-12-27 15:01:15,208][20347] Updated weights for policy 0, policy_version 1352780 (0.0007) [2023-12-27 15:01:15,261][20347] Updated weights for policy 0, policy_version 1352790 (0.0010) [2023-12-27 15:01:15,825][20381] Updated weights for policy 1, policy_version 1351634 (0.0006) [2023-12-27 15:01:15,869][20381] Updated weights for policy 1, policy_version 1351644 (0.0007) [2023-12-27 15:01:15,925][20381] Updated weights for policy 1, policy_version 1351654 (0.0009) [2023-12-27 15:01:15,935][20347] Updated weights for policy 0, policy_version 1352800 (0.0010) [2023-12-27 15:01:15,985][20381] Updated weights for policy 1, policy_version 1351664 (0.0006) [2023-12-27 15:01:15,990][20347] Updated weights for policy 0, policy_version 1352810 (0.0010) [2023-12-27 15:01:16,048][20347] Updated weights for policy 0, policy_version 1352820 (0.0010) [2023-12-27 15:01:16,700][20381] Updated weights for policy 1, policy_version 1351674 (0.0007) [2023-12-27 15:01:16,713][20347] Updated weights for policy 0, policy_version 1352830 (0.0010) [2023-12-27 15:01:16,745][20381] Updated weights for policy 1, policy_version 1351684 (0.0005) [2023-12-27 15:01:16,761][20347] Updated weights for policy 0, policy_version 1352840 (0.0010) [2023-12-27 15:01:16,792][20381] Updated weights for policy 1, policy_version 1351694 (0.0005) [2023-12-27 15:01:16,816][20347] Updated weights for policy 0, policy_version 1352850 (0.0010) [2023-12-27 15:01:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 692461568. Throughput: 0: 10141.3, 1: 10053.3. Samples: 692475112. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:17,364][19170] Avg episode reward: [(0, '844.220'), (1, '775.280')] [2023-12-27 15:01:17,439][20381] Updated weights for policy 1, policy_version 1351704 (0.0005) [2023-12-27 15:01:17,496][20347] Updated weights for policy 0, policy_version 1352860 (0.0010) [2023-12-27 15:01:17,497][20381] Updated weights for policy 1, policy_version 1351714 (0.0005) [2023-12-27 15:01:17,549][20347] Updated weights for policy 0, policy_version 1352870 (0.0010) [2023-12-27 15:01:17,551][20381] Updated weights for policy 1, policy_version 1351724 (0.0005) [2023-12-27 15:01:17,604][20347] Updated weights for policy 0, policy_version 1352880 (0.0010) [2023-12-27 15:01:18,118][20381] Updated weights for policy 1, policy_version 1351734 (0.0007) [2023-12-27 15:01:18,175][20381] Updated weights for policy 1, policy_version 1351744 (0.0007) [2023-12-27 15:01:18,226][20381] Updated weights for policy 1, policy_version 1351754 (0.0007) [2023-12-27 15:01:18,293][20347] Updated weights for policy 0, policy_version 1352890 (0.0010) [2023-12-27 15:01:18,352][20347] Updated weights for policy 0, policy_version 1352900 (0.0010) [2023-12-27 15:01:18,418][20347] Updated weights for policy 0, policy_version 1352910 (0.0006) [2023-12-27 15:01:18,482][20347] Updated weights for policy 0, policy_version 1352920 (0.0005) [2023-12-27 15:01:18,977][20381] Updated weights for policy 1, policy_version 1351764 (0.0008) [2023-12-27 15:01:19,034][20347] Updated weights for policy 0, policy_version 1352930 (0.0010) [2023-12-27 15:01:19,035][20381] Updated weights for policy 1, policy_version 1351774 (0.0006) [2023-12-27 15:01:19,080][20347] Updated weights for policy 0, policy_version 1352940 (0.0010) [2023-12-27 15:01:19,088][20381] Updated weights for policy 1, policy_version 1351784 (0.0006) [2023-12-27 15:01:19,140][20347] Updated weights for policy 0, policy_version 1352950 (0.0011) [2023-12-27 15:01:19,786][20381] Updated weights for policy 1, policy_version 1351794 (0.0007) [2023-12-27 15:01:19,849][20381] Updated weights for policy 1, policy_version 1351804 (0.0007) [2023-12-27 15:01:19,908][20381] Updated weights for policy 1, policy_version 1351814 (0.0008) [2023-12-27 15:01:19,961][20347] Updated weights for policy 0, policy_version 1352960 (0.0008) [2023-12-27 15:01:19,972][20381] Updated weights for policy 1, policy_version 1351824 (0.0008) [2023-12-27 15:01:20,023][20347] Updated weights for policy 0, policy_version 1352970 (0.0008) [2023-12-27 15:01:20,067][20347] Updated weights for policy 0, policy_version 1352980 (0.0007) [2023-12-27 15:01:20,724][20381] Updated weights for policy 1, policy_version 1351834 (0.0007) [2023-12-27 15:01:20,779][20381] Updated weights for policy 1, policy_version 1351844 (0.0006) [2023-12-27 15:01:20,822][20347] Updated weights for policy 0, policy_version 1352990 (0.0005) [2023-12-27 15:01:20,835][20381] Updated weights for policy 1, policy_version 1351854 (0.0006) [2023-12-27 15:01:20,876][20347] Updated weights for policy 0, policy_version 1353000 (0.0007) [2023-12-27 15:01:20,931][20347] Updated weights for policy 0, policy_version 1353010 (0.0008) [2023-12-27 15:01:21,551][20381] Updated weights for policy 1, policy_version 1351864 (0.0007) [2023-12-27 15:01:21,613][20381] Updated weights for policy 1, policy_version 1351874 (0.0008) [2023-12-27 15:01:21,679][20381] Updated weights for policy 1, policy_version 1351884 (0.0008) [2023-12-27 15:01:21,716][20347] Updated weights for policy 0, policy_version 1353020 (0.0007) [2023-12-27 15:01:21,781][20347] Updated weights for policy 0, policy_version 1353030 (0.0007) [2023-12-27 15:01:21,844][20347] Updated weights for policy 0, policy_version 1353040 (0.0007) [2023-12-27 15:01:22,333][20381] Updated weights for policy 1, policy_version 1351894 (0.0007) [2023-12-27 15:01:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.8, 300 sec: 20521.6). Total num frames: 692559872. Throughput: 0: 10130.2, 1: 10075.5. Samples: 692535836. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:22,363][19170] Avg episode reward: [(0, '843.730'), (1, '843.640')] [2023-12-27 15:01:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001353048_346447872.pth... [2023-12-27 15:01:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001351864_346144768.pth [2023-12-27 15:01:22,405][20381] Updated weights for policy 1, policy_version 1351904 (0.0007) [2023-12-27 15:01:22,462][20381] Updated weights for policy 1, policy_version 1351914 (0.0007) [2023-12-27 15:01:22,491][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001351920_346120192.pth... [2023-12-27 15:01:22,494][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001350736_345817088.pth [2023-12-27 15:01:22,609][20347] Updated weights for policy 0, policy_version 1353050 (0.0007) [2023-12-27 15:01:22,659][20347] Updated weights for policy 0, policy_version 1353060 (0.0007) [2023-12-27 15:01:22,721][20347] Updated weights for policy 0, policy_version 1353070 (0.0006) [2023-12-27 15:01:22,777][20347] Updated weights for policy 0, policy_version 1353080 (0.0006) [2023-12-27 15:01:23,158][20381] Updated weights for policy 1, policy_version 1351924 (0.0007) [2023-12-27 15:01:23,223][20381] Updated weights for policy 1, policy_version 1351934 (0.0005) [2023-12-27 15:01:23,288][20381] Updated weights for policy 1, policy_version 1351944 (0.0005) [2023-12-27 15:01:23,410][20347] Updated weights for policy 0, policy_version 1353090 (0.0007) [2023-12-27 15:01:23,461][20347] Updated weights for policy 0, policy_version 1353100 (0.0006) [2023-12-27 15:01:23,509][20347] Updated weights for policy 0, policy_version 1353110 (0.0008) [2023-12-27 15:01:23,949][20381] Updated weights for policy 1, policy_version 1351954 (0.0007) [2023-12-27 15:01:24,007][20381] Updated weights for policy 1, policy_version 1351964 (0.0007) [2023-12-27 15:01:24,064][20381] Updated weights for policy 1, policy_version 1351974 (0.0007) [2023-12-27 15:01:24,119][20381] Updated weights for policy 1, policy_version 1351984 (0.0008) [2023-12-27 15:01:24,211][20347] Updated weights for policy 0, policy_version 1353120 (0.0009) [2023-12-27 15:01:24,263][20347] Updated weights for policy 0, policy_version 1353130 (0.0010) [2023-12-27 15:01:24,311][20347] Updated weights for policy 0, policy_version 1353140 (0.0010) [2023-12-27 15:01:24,821][20381] Updated weights for policy 1, policy_version 1351994 (0.0008) [2023-12-27 15:01:24,868][20381] Updated weights for policy 1, policy_version 1352004 (0.0007) [2023-12-27 15:01:24,918][20381] Updated weights for policy 1, policy_version 1352014 (0.0007) [2023-12-27 15:01:24,987][20347] Updated weights for policy 0, policy_version 1353150 (0.0010) [2023-12-27 15:01:25,042][20347] Updated weights for policy 0, policy_version 1353160 (0.0010) [2023-12-27 15:01:25,087][20347] Updated weights for policy 0, policy_version 1353170 (0.0010) [2023-12-27 15:01:25,500][20381] Updated weights for policy 1, policy_version 1352024 (0.0007) [2023-12-27 15:01:25,557][20381] Updated weights for policy 1, policy_version 1352034 (0.0006) [2023-12-27 15:01:25,616][20381] Updated weights for policy 1, policy_version 1352044 (0.0008) [2023-12-27 15:01:25,713][20347] Updated weights for policy 0, policy_version 1353180 (0.0009) [2023-12-27 15:01:25,774][20347] Updated weights for policy 0, policy_version 1353190 (0.0005) [2023-12-27 15:01:25,832][20347] Updated weights for policy 0, policy_version 1353200 (0.0008) [2023-12-27 15:01:26,231][20381] Updated weights for policy 1, policy_version 1352054 (0.0006) [2023-12-27 15:01:26,265][20292] KL-divergence is very high: 123.0274 [2023-12-27 15:01:26,298][20381] Updated weights for policy 1, policy_version 1352064 (0.0006) [2023-12-27 15:01:26,359][20381] Updated weights for policy 1, policy_version 1352074 (0.0009) [2023-12-27 15:01:26,449][20347] Updated weights for policy 0, policy_version 1353210 (0.0009) [2023-12-27 15:01:26,506][20347] Updated weights for policy 0, policy_version 1353220 (0.0007) [2023-12-27 15:01:26,557][20347] Updated weights for policy 0, policy_version 1353230 (0.0007) [2023-12-27 15:01:26,613][20347] Updated weights for policy 0, policy_version 1353240 (0.0007) [2023-12-27 15:01:26,995][20381] Updated weights for policy 1, policy_version 1352084 (0.0008) [2023-12-27 15:01:27,045][20381] Updated weights for policy 1, policy_version 1352094 (0.0005) [2023-12-27 15:01:27,093][20381] Updated weights for policy 1, policy_version 1352104 (0.0005) [2023-12-27 15:01:27,306][20347] Updated weights for policy 0, policy_version 1353250 (0.0006) [2023-12-27 15:01:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 692666368. Throughput: 0: 10189.9, 1: 10167.2. Samples: 692658272. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:27,363][20347] Updated weights for policy 0, policy_version 1353260 (0.0005) [2023-12-27 15:01:27,363][19170] Avg episode reward: [(0, '892.780'), (1, '717.180')] [2023-12-27 15:01:27,412][20347] Updated weights for policy 0, policy_version 1353270 (0.0005) [2023-12-27 15:01:27,758][20381] Updated weights for policy 1, policy_version 1352114 (0.0005) [2023-12-27 15:01:27,807][20381] Updated weights for policy 1, policy_version 1352124 (0.0007) [2023-12-27 15:01:27,813][20292] KL-divergence is very high: 323.2761 [2023-12-27 15:01:27,849][20292] KL-divergence is very high: 331.5522 [2023-12-27 15:01:27,854][20381] Updated weights for policy 1, policy_version 1352134 (0.0007) [2023-12-27 15:01:27,889][20292] KL-divergence is very high: 141.9117 [2023-12-27 15:01:27,905][20381] Updated weights for policy 1, policy_version 1352144 (0.0007) [2023-12-27 15:01:28,012][20347] Updated weights for policy 0, policy_version 1353280 (0.0005) [2023-12-27 15:01:28,072][20347] Updated weights for policy 0, policy_version 1353290 (0.0008) [2023-12-27 15:01:28,123][20347] Updated weights for policy 0, policy_version 1353300 (0.0010) [2023-12-27 15:01:28,593][20381] Updated weights for policy 1, policy_version 1352154 (0.0006) [2023-12-27 15:01:28,653][20381] Updated weights for policy 1, policy_version 1352164 (0.0008) [2023-12-27 15:01:28,707][20381] Updated weights for policy 1, policy_version 1352174 (0.0008) [2023-12-27 15:01:28,721][20347] Updated weights for policy 0, policy_version 1353310 (0.0007) [2023-12-27 15:01:28,773][20347] Updated weights for policy 0, policy_version 1353320 (0.0010) [2023-12-27 15:01:28,828][20347] Updated weights for policy 0, policy_version 1353330 (0.0010) [2023-12-27 15:01:29,407][20381] Updated weights for policy 1, policy_version 1352184 (0.0008) [2023-12-27 15:01:29,464][20381] Updated weights for policy 1, policy_version 1352194 (0.0006) [2023-12-27 15:01:29,524][20381] Updated weights for policy 1, policy_version 1352204 (0.0006) [2023-12-27 15:01:29,537][20347] Updated weights for policy 0, policy_version 1353340 (0.0010) [2023-12-27 15:01:29,589][20347] Updated weights for policy 0, policy_version 1353350 (0.0010) [2023-12-27 15:01:29,649][20347] Updated weights for policy 0, policy_version 1353360 (0.0010) [2023-12-27 15:01:30,192][20381] Updated weights for policy 1, policy_version 1352214 (0.0007) [2023-12-27 15:01:30,245][20381] Updated weights for policy 1, policy_version 1352224 (0.0008) [2023-12-27 15:01:30,249][20292] KL-divergence is very high: 237.3135 [2023-12-27 15:01:30,258][20292] KL-divergence is very high: 260.9538 [2023-12-27 15:01:30,292][20381] Updated weights for policy 1, policy_version 1352234 (0.0008) [2023-12-27 15:01:30,361][20347] Updated weights for policy 0, policy_version 1353370 (0.0010) [2023-12-27 15:01:30,414][20347] Updated weights for policy 0, policy_version 1353380 (0.0008) [2023-12-27 15:01:30,468][20347] Updated weights for policy 0, policy_version 1353390 (0.0009) [2023-12-27 15:01:30,525][20347] Updated weights for policy 0, policy_version 1353400 (0.0007) [2023-12-27 15:01:30,952][20381] Updated weights for policy 1, policy_version 1352244 (0.0007) [2023-12-27 15:01:30,999][20381] Updated weights for policy 1, policy_version 1352254 (0.0005) [2023-12-27 15:01:31,056][20381] Updated weights for policy 1, policy_version 1352264 (0.0007) [2023-12-27 15:01:31,191][20347] Updated weights for policy 0, policy_version 1353410 (0.0007) [2023-12-27 15:01:31,240][20347] Updated weights for policy 0, policy_version 1353420 (0.0006) [2023-12-27 15:01:31,299][20347] Updated weights for policy 0, policy_version 1353430 (0.0007) [2023-12-27 15:01:31,734][20381] Updated weights for policy 1, policy_version 1352274 (0.0009) [2023-12-27 15:01:31,793][20381] Updated weights for policy 1, policy_version 1352284 (0.0009) [2023-12-27 15:01:31,854][20381] Updated weights for policy 1, policy_version 1352294 (0.0008) [2023-12-27 15:01:31,912][20381] Updated weights for policy 1, policy_version 1352304 (0.0008) [2023-12-27 15:01:32,046][20347] Updated weights for policy 0, policy_version 1353440 (0.0008) [2023-12-27 15:01:32,100][20347] Updated weights for policy 0, policy_version 1353450 (0.0007) [2023-12-27 15:01:32,156][20347] Updated weights for policy 0, policy_version 1353460 (0.0008) [2023-12-27 15:01:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 692772864. Throughput: 0: 10160.9, 1: 10107.9. Samples: 692781308. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:32,363][19170] Avg episode reward: [(0, '892.750'), (1, '699.350')] [2023-12-27 15:01:32,624][20381] Updated weights for policy 1, policy_version 1352314 (0.0009) [2023-12-27 15:01:32,684][20381] Updated weights for policy 1, policy_version 1352324 (0.0007) [2023-12-27 15:01:32,748][20381] Updated weights for policy 1, policy_version 1352334 (0.0010) [2023-12-27 15:01:32,886][20347] Updated weights for policy 0, policy_version 1353470 (0.0008) [2023-12-27 15:01:32,935][20347] Updated weights for policy 0, policy_version 1353480 (0.0008) [2023-12-27 15:01:32,987][20347] Updated weights for policy 0, policy_version 1353490 (0.0008) [2023-12-27 15:01:33,369][20381] Updated weights for policy 1, policy_version 1352344 (0.0006) [2023-12-27 15:01:33,430][20381] Updated weights for policy 1, policy_version 1352354 (0.0008) [2023-12-27 15:01:33,484][20381] Updated weights for policy 1, policy_version 1352364 (0.0010) [2023-12-27 15:01:33,586][20347] Updated weights for policy 0, policy_version 1353500 (0.0009) [2023-12-27 15:01:33,631][20347] Updated weights for policy 0, policy_version 1353510 (0.0010) [2023-12-27 15:01:33,688][20347] Updated weights for policy 0, policy_version 1353520 (0.0010) [2023-12-27 15:01:34,103][20381] Updated weights for policy 1, policy_version 1352374 (0.0007) [2023-12-27 15:01:34,168][20381] Updated weights for policy 1, policy_version 1352384 (0.0006) [2023-12-27 15:01:34,230][20381] Updated weights for policy 1, policy_version 1352394 (0.0010) [2023-12-27 15:01:34,352][20347] Updated weights for policy 0, policy_version 1353530 (0.0010) [2023-12-27 15:01:34,401][20347] Updated weights for policy 0, policy_version 1353540 (0.0010) [2023-12-27 15:01:34,450][20347] Updated weights for policy 0, policy_version 1353550 (0.0009) [2023-12-27 15:01:34,512][20347] Updated weights for policy 0, policy_version 1353560 (0.0011) [2023-12-27 15:01:34,907][20381] Updated weights for policy 1, policy_version 1352404 (0.0010) [2023-12-27 15:01:34,955][20381] Updated weights for policy 1, policy_version 1352414 (0.0010) [2023-12-27 15:01:35,007][20381] Updated weights for policy 1, policy_version 1352424 (0.0010) [2023-12-27 15:01:35,172][20347] Updated weights for policy 0, policy_version 1353570 (0.0005) [2023-12-27 15:01:35,208][20291] KL-divergence is very high: 101.4494 [2023-12-27 15:01:35,226][20347] Updated weights for policy 0, policy_version 1353580 (0.0007) [2023-12-27 15:01:35,277][20347] Updated weights for policy 0, policy_version 1353590 (0.0010) [2023-12-27 15:01:35,692][20381] Updated weights for policy 1, policy_version 1352434 (0.0010) [2023-12-27 15:01:35,750][20381] Updated weights for policy 1, policy_version 1352444 (0.0007) [2023-12-27 15:01:35,810][20381] Updated weights for policy 1, policy_version 1352454 (0.0008) [2023-12-27 15:01:35,870][20381] Updated weights for policy 1, policy_version 1352464 (0.0007) [2023-12-27 15:01:35,919][20347] Updated weights for policy 0, policy_version 1353600 (0.0008) [2023-12-27 15:01:35,975][20347] Updated weights for policy 0, policy_version 1353610 (0.0008) [2023-12-27 15:01:36,046][20347] Updated weights for policy 0, policy_version 1353620 (0.0008) [2023-12-27 15:01:36,568][20381] Updated weights for policy 1, policy_version 1352474 (0.0008) [2023-12-27 15:01:36,631][20381] Updated weights for policy 1, policy_version 1352484 (0.0006) [2023-12-27 15:01:36,697][20381] Updated weights for policy 1, policy_version 1352494 (0.0008) [2023-12-27 15:01:36,739][20347] Updated weights for policy 0, policy_version 1353630 (0.0008) [2023-12-27 15:01:36,782][20347] Updated weights for policy 0, policy_version 1353640 (0.0008) [2023-12-27 15:01:36,840][20347] Updated weights for policy 0, policy_version 1353650 (0.0007) [2023-12-27 15:01:37,337][20381] Updated weights for policy 1, policy_version 1352504 (0.0007) [2023-12-27 15:01:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 692871168. Throughput: 0: 10166.7, 1: 10165.9. Samples: 692844576. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:37,363][19170] Avg episode reward: [(0, '824.150'), (1, '727.930')] [2023-12-27 15:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001353656_346603520.pth... [2023-12-27 15:01:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001352472_346300416.pth [2023-12-27 15:01:37,396][20381] Updated weights for policy 1, policy_version 1352514 (0.0008) [2023-12-27 15:01:37,446][20381] Updated weights for policy 1, policy_version 1352524 (0.0006) [2023-12-27 15:01:37,475][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001352528_346275840.pth... [2023-12-27 15:01:37,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001351312_345964544.pth [2023-12-27 15:01:37,560][20347] Updated weights for policy 0, policy_version 1353660 (0.0006) [2023-12-27 15:01:37,609][20347] Updated weights for policy 0, policy_version 1353670 (0.0007) [2023-12-27 15:01:37,664][20347] Updated weights for policy 0, policy_version 1353680 (0.0008) [2023-12-27 15:01:38,165][20381] Updated weights for policy 1, policy_version 1352534 (0.0006) [2023-12-27 15:01:38,216][20381] Updated weights for policy 1, policy_version 1352544 (0.0008) [2023-12-27 15:01:38,277][20381] Updated weights for policy 1, policy_version 1352554 (0.0008) [2023-12-27 15:01:38,398][20347] Updated weights for policy 0, policy_version 1353690 (0.0009) [2023-12-27 15:01:38,454][20347] Updated weights for policy 0, policy_version 1353700 (0.0006) [2023-12-27 15:01:38,517][20347] Updated weights for policy 0, policy_version 1353710 (0.0007) [2023-12-27 15:01:38,574][20347] Updated weights for policy 0, policy_version 1353720 (0.0007) [2023-12-27 15:01:38,990][20381] Updated weights for policy 1, policy_version 1352564 (0.0008) [2023-12-27 15:01:39,037][20381] Updated weights for policy 1, policy_version 1352574 (0.0010) [2023-12-27 15:01:39,086][20381] Updated weights for policy 1, policy_version 1352584 (0.0010) [2023-12-27 15:01:39,222][20347] Updated weights for policy 0, policy_version 1353730 (0.0010) [2023-12-27 15:01:39,296][20347] Updated weights for policy 0, policy_version 1353740 (0.0008) [2023-12-27 15:01:39,362][20347] Updated weights for policy 0, policy_version 1353750 (0.0009) [2023-12-27 15:01:39,871][20381] Updated weights for policy 1, policy_version 1352594 (0.0011) [2023-12-27 15:01:39,924][20381] Updated weights for policy 1, policy_version 1352604 (0.0010) [2023-12-27 15:01:39,991][20381] Updated weights for policy 1, policy_version 1352614 (0.0008) [2023-12-27 15:01:40,029][20347] Updated weights for policy 0, policy_version 1353760 (0.0007) [2023-12-27 15:01:40,052][20381] Updated weights for policy 1, policy_version 1352624 (0.0008) [2023-12-27 15:01:40,088][20347] Updated weights for policy 0, policy_version 1353770 (0.0007) [2023-12-27 15:01:40,142][20347] Updated weights for policy 0, policy_version 1353780 (0.0007) [2023-12-27 15:01:40,748][20381] Updated weights for policy 1, policy_version 1352634 (0.0008) [2023-12-27 15:01:40,804][20381] Updated weights for policy 1, policy_version 1352644 (0.0008) [2023-12-27 15:01:40,812][20347] Updated weights for policy 0, policy_version 1353790 (0.0010) [2023-12-27 15:01:40,860][20347] Updated weights for policy 0, policy_version 1353800 (0.0010) [2023-12-27 15:01:40,863][20381] Updated weights for policy 1, policy_version 1352654 (0.0011) [2023-12-27 15:01:40,922][20347] Updated weights for policy 0, policy_version 1353810 (0.0010) [2023-12-27 15:01:41,569][20381] Updated weights for policy 1, policy_version 1352664 (0.0006) [2023-12-27 15:01:41,636][20381] Updated weights for policy 1, policy_version 1352674 (0.0009) [2023-12-27 15:01:41,668][20347] Updated weights for policy 0, policy_version 1353820 (0.0009) [2023-12-27 15:01:41,697][20381] Updated weights for policy 1, policy_version 1352684 (0.0006) [2023-12-27 15:01:41,729][20347] Updated weights for policy 0, policy_version 1353830 (0.0007) [2023-12-27 15:01:41,794][20347] Updated weights for policy 0, policy_version 1353840 (0.0007) [2023-12-27 15:01:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 692969472. Throughput: 0: 10191.9, 1: 10106.5. Samples: 692964884. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:42,363][19170] Avg episode reward: [(0, '833.980'), (1, '658.980')] [2023-12-27 15:01:42,454][20381] Updated weights for policy 1, policy_version 1352694 (0.0008) [2023-12-27 15:01:42,467][20347] Updated weights for policy 0, policy_version 1353850 (0.0006) [2023-12-27 15:01:42,510][20381] Updated weights for policy 1, policy_version 1352704 (0.0006) [2023-12-27 15:01:42,516][20347] Updated weights for policy 0, policy_version 1353860 (0.0007) [2023-12-27 15:01:42,566][20347] Updated weights for policy 0, policy_version 1353870 (0.0007) [2023-12-27 15:01:42,576][20381] Updated weights for policy 1, policy_version 1352714 (0.0006) [2023-12-27 15:01:42,621][20347] Updated weights for policy 0, policy_version 1353880 (0.0008) [2023-12-27 15:01:43,257][20347] Updated weights for policy 0, policy_version 1353890 (0.0005) [2023-12-27 15:01:43,305][20381] Updated weights for policy 1, policy_version 1352724 (0.0007) [2023-12-27 15:01:43,308][20347] Updated weights for policy 0, policy_version 1353900 (0.0005) [2023-12-27 15:01:43,360][20381] Updated weights for policy 1, policy_version 1352734 (0.0007) [2023-12-27 15:01:43,364][20347] Updated weights for policy 0, policy_version 1353910 (0.0006) [2023-12-27 15:01:43,416][20381] Updated weights for policy 1, policy_version 1352744 (0.0007) [2023-12-27 15:01:43,996][20347] Updated weights for policy 0, policy_version 1353920 (0.0007) [2023-12-27 15:01:44,048][20347] Updated weights for policy 0, policy_version 1353930 (0.0007) [2023-12-27 15:01:44,072][20381] Updated weights for policy 1, policy_version 1352754 (0.0007) [2023-12-27 15:01:44,099][20347] Updated weights for policy 0, policy_version 1353940 (0.0007) [2023-12-27 15:01:44,130][20381] Updated weights for policy 1, policy_version 1352764 (0.0007) [2023-12-27 15:01:44,188][20381] Updated weights for policy 1, policy_version 1352774 (0.0010) [2023-12-27 15:01:44,244][20381] Updated weights for policy 1, policy_version 1352784 (0.0010) [2023-12-27 15:01:44,753][20347] Updated weights for policy 0, policy_version 1353950 (0.0006) [2023-12-27 15:01:44,815][20347] Updated weights for policy 0, policy_version 1353960 (0.0008) [2023-12-27 15:01:44,868][20347] Updated weights for policy 0, policy_version 1353970 (0.0007) [2023-12-27 15:01:44,932][20381] Updated weights for policy 1, policy_version 1352794 (0.0011) [2023-12-27 15:01:44,990][20381] Updated weights for policy 1, policy_version 1352804 (0.0011) [2023-12-27 15:01:45,052][20381] Updated weights for policy 1, policy_version 1352814 (0.0011) [2023-12-27 15:01:45,500][20347] Updated weights for policy 0, policy_version 1353980 (0.0007) [2023-12-27 15:01:45,551][20347] Updated weights for policy 0, policy_version 1353990 (0.0008) [2023-12-27 15:01:45,613][20347] Updated weights for policy 0, policy_version 1354000 (0.0007) [2023-12-27 15:01:45,705][20381] Updated weights for policy 1, policy_version 1352824 (0.0008) [2023-12-27 15:01:45,762][20381] Updated weights for policy 1, policy_version 1352834 (0.0009) [2023-12-27 15:01:45,818][20381] Updated weights for policy 1, policy_version 1352844 (0.0006) [2023-12-27 15:01:46,302][20347] Updated weights for policy 0, policy_version 1354010 (0.0005) [2023-12-27 15:01:46,358][20347] Updated weights for policy 0, policy_version 1354020 (0.0007) [2023-12-27 15:01:46,366][20381] Updated weights for policy 1, policy_version 1352854 (0.0006) [2023-12-27 15:01:46,417][20347] Updated weights for policy 0, policy_version 1354030 (0.0008) [2023-12-27 15:01:46,438][20381] Updated weights for policy 1, policy_version 1352864 (0.0007) [2023-12-27 15:01:46,468][20347] Updated weights for policy 0, policy_version 1354040 (0.0007) [2023-12-27 15:01:46,492][20381] Updated weights for policy 1, policy_version 1352874 (0.0007) [2023-12-27 15:01:47,152][20381] Updated weights for policy 1, policy_version 1352884 (0.0008) [2023-12-27 15:01:47,165][20347] Updated weights for policy 0, policy_version 1354050 (0.0005) [2023-12-27 15:01:47,203][20381] Updated weights for policy 1, policy_version 1352894 (0.0007) [2023-12-27 15:01:47,214][20347] Updated weights for policy 0, policy_version 1354060 (0.0005) [2023-12-27 15:01:47,247][20381] Updated weights for policy 1, policy_version 1352904 (0.0006) [2023-12-27 15:01:47,269][20347] Updated weights for policy 0, policy_version 1354070 (0.0007) [2023-12-27 15:01:47,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 693084160. Throughput: 0: 10270.0, 1: 10186.1. Samples: 693088968. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:47,363][19170] Avg episode reward: [(0, '834.040'), (1, '601.710')] [2023-12-27 15:01:47,915][20381] Updated weights for policy 1, policy_version 1352914 (0.0008) [2023-12-27 15:01:47,956][20347] Updated weights for policy 0, policy_version 1354080 (0.0009) [2023-12-27 15:01:47,969][20381] Updated weights for policy 1, policy_version 1352924 (0.0005) [2023-12-27 15:01:48,015][20347] Updated weights for policy 0, policy_version 1354090 (0.0011) [2023-12-27 15:01:48,031][20381] Updated weights for policy 1, policy_version 1352934 (0.0006) [2023-12-27 15:01:48,076][20347] Updated weights for policy 0, policy_version 1354100 (0.0010) [2023-12-27 15:01:48,086][20381] Updated weights for policy 1, policy_version 1352944 (0.0006) [2023-12-27 15:01:48,744][20381] Updated weights for policy 1, policy_version 1352954 (0.0007) [2023-12-27 15:01:48,780][20347] Updated weights for policy 0, policy_version 1354110 (0.0011) [2023-12-27 15:01:48,790][20381] Updated weights for policy 1, policy_version 1352964 (0.0007) [2023-12-27 15:01:48,833][20347] Updated weights for policy 0, policy_version 1354120 (0.0010) [2023-12-27 15:01:48,837][20381] Updated weights for policy 1, policy_version 1352974 (0.0006) [2023-12-27 15:01:48,881][20347] Updated weights for policy 0, policy_version 1354130 (0.0010) [2023-12-27 15:01:49,527][20381] Updated weights for policy 1, policy_version 1352984 (0.0008) [2023-12-27 15:01:49,560][20347] Updated weights for policy 0, policy_version 1354140 (0.0010) [2023-12-27 15:01:49,584][20381] Updated weights for policy 1, policy_version 1352994 (0.0007) [2023-12-27 15:01:49,620][20347] Updated weights for policy 0, policy_version 1354150 (0.0009) [2023-12-27 15:01:49,641][20381] Updated weights for policy 1, policy_version 1353004 (0.0005) [2023-12-27 15:01:49,680][20347] Updated weights for policy 0, policy_version 1354160 (0.0007) [2023-12-27 15:01:50,304][20381] Updated weights for policy 1, policy_version 1353014 (0.0008) [2023-12-27 15:01:50,333][20347] Updated weights for policy 0, policy_version 1354170 (0.0010) [2023-12-27 15:01:50,364][20381] Updated weights for policy 1, policy_version 1353024 (0.0008) [2023-12-27 15:01:50,393][20347] Updated weights for policy 0, policy_version 1354180 (0.0011) [2023-12-27 15:01:50,417][20381] Updated weights for policy 1, policy_version 1353034 (0.0008) [2023-12-27 15:01:50,452][20347] Updated weights for policy 0, policy_version 1354190 (0.0010) [2023-12-27 15:01:50,503][20347] Updated weights for policy 0, policy_version 1354200 (0.0010) [2023-12-27 15:01:51,137][20381] Updated weights for policy 1, policy_version 1353044 (0.0007) [2023-12-27 15:01:51,195][20347] Updated weights for policy 0, policy_version 1354210 (0.0008) [2023-12-27 15:01:51,197][20381] Updated weights for policy 1, policy_version 1353054 (0.0006) [2023-12-27 15:01:51,246][20347] Updated weights for policy 0, policy_version 1354220 (0.0007) [2023-12-27 15:01:51,247][20291] KL-divergence is very high: 139.6954 [2023-12-27 15:01:51,260][20381] Updated weights for policy 1, policy_version 1353064 (0.0006) [2023-12-27 15:01:51,303][20347] Updated weights for policy 0, policy_version 1354230 (0.0007) [2023-12-27 15:01:51,963][20347] Updated weights for policy 0, policy_version 1354240 (0.0008) [2023-12-27 15:01:52,001][20381] Updated weights for policy 1, policy_version 1353074 (0.0007) [2023-12-27 15:01:52,016][20347] Updated weights for policy 0, policy_version 1354250 (0.0007) [2023-12-27 15:01:52,056][20381] Updated weights for policy 1, policy_version 1353084 (0.0006) [2023-12-27 15:01:52,066][20347] Updated weights for policy 0, policy_version 1354260 (0.0006) [2023-12-27 15:01:52,115][20381] Updated weights for policy 1, policy_version 1353094 (0.0006) [2023-12-27 15:01:52,175][20381] Updated weights for policy 1, policy_version 1353104 (0.0008) [2023-12-27 15:01:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 693182464. Throughput: 0: 10239.2, 1: 10217.8. Samples: 693150488. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-12-27 15:01:52,363][19170] Avg episode reward: [(0, '843.890'), (1, '620.060')] [2023-12-27 15:01:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001353104_346423296.pth... [2023-12-27 15:01:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001354264_346759168.pth... [2023-12-27 15:01:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001351920_346120192.pth [2023-12-27 15:01:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001353048_346447872.pth [2023-12-27 15:01:52,747][20347] Updated weights for policy 0, policy_version 1354270 (0.0006) [2023-12-27 15:01:52,808][20347] Updated weights for policy 0, policy_version 1354280 (0.0007) [2023-12-27 15:01:52,863][20381] Updated weights for policy 1, policy_version 1353114 (0.0007) [2023-12-27 15:01:52,871][20347] Updated weights for policy 0, policy_version 1354290 (0.0007) [2023-12-27 15:01:52,923][20381] Updated weights for policy 1, policy_version 1353124 (0.0006) [2023-12-27 15:01:52,975][20381] Updated weights for policy 1, policy_version 1353134 (0.0008) [2023-12-27 15:01:53,515][20347] Updated weights for policy 0, policy_version 1354300 (0.0005) [2023-12-27 15:01:53,563][20347] Updated weights for policy 0, policy_version 1354310 (0.0005) [2023-12-27 15:01:53,614][20347] Updated weights for policy 0, policy_version 1354320 (0.0007) [2023-12-27 15:01:53,628][20381] Updated weights for policy 1, policy_version 1353144 (0.0007) [2023-12-27 15:01:53,677][20381] Updated weights for policy 1, policy_version 1353154 (0.0007) [2023-12-27 15:01:53,735][20381] Updated weights for policy 1, policy_version 1353164 (0.0007) [2023-12-27 15:01:54,299][20347] Updated weights for policy 0, policy_version 1354330 (0.0006) [2023-12-27 15:01:54,355][20347] Updated weights for policy 0, policy_version 1354340 (0.0008) [2023-12-27 15:01:54,405][20347] Updated weights for policy 0, policy_version 1354350 (0.0007) [2023-12-27 15:01:54,442][20381] Updated weights for policy 1, policy_version 1353174 (0.0009) [2023-12-27 15:01:54,452][20347] Updated weights for policy 0, policy_version 1354360 (0.0006) [2023-12-27 15:01:54,490][20381] Updated weights for policy 1, policy_version 1353184 (0.0010) [2023-12-27 15:01:54,542][20381] Updated weights for policy 1, policy_version 1353194 (0.0010) [2023-12-27 15:01:55,099][20347] Updated weights for policy 0, policy_version 1354370 (0.0008) [2023-12-27 15:01:55,147][20347] Updated weights for policy 0, policy_version 1354380 (0.0008) [2023-12-27 15:01:55,151][20291] KL-divergence is very high: 117.8158 [2023-12-27 15:01:55,185][20381] Updated weights for policy 1, policy_version 1353204 (0.0009) [2023-12-27 15:01:55,196][20291] KL-divergence is very high: 118.2143 [2023-12-27 15:01:55,200][20347] Updated weights for policy 0, policy_version 1354390 (0.0006) [2023-12-27 15:01:55,239][20381] Updated weights for policy 1, policy_version 1353214 (0.0008) [2023-12-27 15:01:55,297][20381] Updated weights for policy 1, policy_version 1353224 (0.0006) [2023-12-27 15:01:55,795][20347] Updated weights for policy 0, policy_version 1354400 (0.0009) [2023-12-27 15:01:55,847][20347] Updated weights for policy 0, policy_version 1354410 (0.0010) [2023-12-27 15:01:55,901][20347] Updated weights for policy 0, policy_version 1354420 (0.0010) [2023-12-27 15:01:55,932][20381] Updated weights for policy 1, policy_version 1353234 (0.0006) [2023-12-27 15:01:55,977][20381] Updated weights for policy 1, policy_version 1353244 (0.0007) [2023-12-27 15:01:56,026][20381] Updated weights for policy 1, policy_version 1353254 (0.0007) [2023-12-27 15:01:56,073][20381] Updated weights for policy 1, policy_version 1353264 (0.0006) [2023-12-27 15:01:56,585][20347] Updated weights for policy 0, policy_version 1354430 (0.0010) [2023-12-27 15:01:56,636][20347] Updated weights for policy 0, policy_version 1354440 (0.0010) [2023-12-27 15:01:56,699][20347] Updated weights for policy 0, policy_version 1354450 (0.0010) [2023-12-27 15:01:56,729][20381] Updated weights for policy 1, policy_version 1353274 (0.0005) [2023-12-27 15:01:56,784][20381] Updated weights for policy 1, policy_version 1353284 (0.0005) [2023-12-27 15:01:56,842][20381] Updated weights for policy 1, policy_version 1353294 (0.0005) [2023-12-27 15:01:57,351][20381] Updated weights for policy 1, policy_version 1353304 (0.0007) [2023-12-27 15:01:57,358][20347] Updated weights for policy 0, policy_version 1354460 (0.0010) [2023-12-27 15:01:57,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 693280768. Throughput: 0: 10274.4, 1: 10223.6. Samples: 693274280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:01:57,364][19170] Avg episode reward: [(0, '843.460'), (1, '658.830')] [2023-12-27 15:01:57,408][20381] Updated weights for policy 1, policy_version 1353314 (0.0010) [2023-12-27 15:01:57,409][20347] Updated weights for policy 0, policy_version 1354470 (0.0010) [2023-12-27 15:01:57,457][20381] Updated weights for policy 1, policy_version 1353324 (0.0010) [2023-12-27 15:01:57,459][20347] Updated weights for policy 0, policy_version 1354480 (0.0010) [2023-12-27 15:01:58,104][20381] Updated weights for policy 1, policy_version 1353334 (0.0011) [2023-12-27 15:01:58,117][20347] Updated weights for policy 0, policy_version 1354490 (0.0010) [2023-12-27 15:01:58,153][20381] Updated weights for policy 1, policy_version 1353344 (0.0009) [2023-12-27 15:01:58,165][20347] Updated weights for policy 0, policy_version 1354500 (0.0010) [2023-12-27 15:01:58,218][20381] Updated weights for policy 1, policy_version 1353354 (0.0009) [2023-12-27 15:01:58,223][20347] Updated weights for policy 0, policy_version 1354510 (0.0008) [2023-12-27 15:01:58,288][20347] Updated weights for policy 0, policy_version 1354520 (0.0008) [2023-12-27 15:01:59,046][20381] Updated weights for policy 1, policy_version 1353364 (0.0010) [2023-12-27 15:01:59,079][20347] Updated weights for policy 0, policy_version 1354530 (0.0009) [2023-12-27 15:01:59,105][20381] Updated weights for policy 1, policy_version 1353374 (0.0010) [2023-12-27 15:01:59,133][20347] Updated weights for policy 0, policy_version 1354540 (0.0006) [2023-12-27 15:01:59,175][20381] Updated weights for policy 1, policy_version 1353384 (0.0011) [2023-12-27 15:01:59,192][20347] Updated weights for policy 0, policy_version 1354550 (0.0005) [2023-12-27 15:01:59,879][20347] Updated weights for policy 0, policy_version 1354560 (0.0007) [2023-12-27 15:01:59,938][20381] Updated weights for policy 1, policy_version 1353394 (0.0009) [2023-12-27 15:01:59,947][20347] Updated weights for policy 0, policy_version 1354570 (0.0007) [2023-12-27 15:01:59,995][20381] Updated weights for policy 1, policy_version 1353404 (0.0007) [2023-12-27 15:02:00,010][20347] Updated weights for policy 0, policy_version 1354580 (0.0008) [2023-12-27 15:02:00,051][20381] Updated weights for policy 1, policy_version 1353414 (0.0007) [2023-12-27 15:02:00,104][20381] Updated weights for policy 1, policy_version 1353424 (0.0007) [2023-12-27 15:02:00,646][20347] Updated weights for policy 0, policy_version 1354590 (0.0007) [2023-12-27 15:02:00,699][20347] Updated weights for policy 0, policy_version 1354600 (0.0007) [2023-12-27 15:02:00,755][20347] Updated weights for policy 0, policy_version 1354610 (0.0007) [2023-12-27 15:02:00,828][20381] Updated weights for policy 1, policy_version 1353434 (0.0010) [2023-12-27 15:02:00,888][20381] Updated weights for policy 1, policy_version 1353444 (0.0010) [2023-12-27 15:02:00,946][20381] Updated weights for policy 1, policy_version 1353454 (0.0010) [2023-12-27 15:02:01,463][20347] Updated weights for policy 0, policy_version 1354620 (0.0007) [2023-12-27 15:02:01,512][20347] Updated weights for policy 0, policy_version 1354630 (0.0006) [2023-12-27 15:02:01,566][20347] Updated weights for policy 0, policy_version 1354640 (0.0008) [2023-12-27 15:02:01,650][20381] Updated weights for policy 1, policy_version 1353464 (0.0009) [2023-12-27 15:02:01,709][20381] Updated weights for policy 1, policy_version 1353474 (0.0006) [2023-12-27 15:02:01,770][20381] Updated weights for policy 1, policy_version 1353484 (0.0007) [2023-12-27 15:02:02,229][20347] Updated weights for policy 0, policy_version 1354650 (0.0007) [2023-12-27 15:02:02,291][20347] Updated weights for policy 0, policy_version 1354660 (0.0006) [2023-12-27 15:02:02,349][20347] Updated weights for policy 0, policy_version 1354670 (0.0008) [2023-12-27 15:02:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 693379072. Throughput: 0: 10265.0, 1: 10170.2. Samples: 693394692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:02,363][19170] Avg episode reward: [(0, '853.540'), (1, '697.170')] [2023-12-27 15:02:02,411][20347] Updated weights for policy 0, policy_version 1354680 (0.0007) [2023-12-27 15:02:02,516][20381] Updated weights for policy 1, policy_version 1353494 (0.0009) [2023-12-27 15:02:02,570][20381] Updated weights for policy 1, policy_version 1353504 (0.0010) [2023-12-27 15:02:02,629][20381] Updated weights for policy 1, policy_version 1353514 (0.0009) [2023-12-27 15:02:02,993][20347] Updated weights for policy 0, policy_version 1354690 (0.0005) [2023-12-27 15:02:03,050][20347] Updated weights for policy 0, policy_version 1354700 (0.0005) [2023-12-27 15:02:03,105][20347] Updated weights for policy 0, policy_version 1354710 (0.0005) [2023-12-27 15:02:03,342][20381] Updated weights for policy 1, policy_version 1353524 (0.0009) [2023-12-27 15:02:03,392][20381] Updated weights for policy 1, policy_version 1353534 (0.0006) [2023-12-27 15:02:03,442][20381] Updated weights for policy 1, policy_version 1353544 (0.0005) [2023-12-27 15:02:03,676][20347] Updated weights for policy 0, policy_version 1354720 (0.0006) [2023-12-27 15:02:03,733][20347] Updated weights for policy 0, policy_version 1354730 (0.0010) [2023-12-27 15:02:03,788][20347] Updated weights for policy 0, policy_version 1354740 (0.0006) [2023-12-27 15:02:04,049][20381] Updated weights for policy 1, policy_version 1353554 (0.0005) [2023-12-27 15:02:04,103][20381] Updated weights for policy 1, policy_version 1353564 (0.0007) [2023-12-27 15:02:04,154][20381] Updated weights for policy 1, policy_version 1353574 (0.0008) [2023-12-27 15:02:04,211][20381] Updated weights for policy 1, policy_version 1353584 (0.0006) [2023-12-27 15:02:04,537][20347] Updated weights for policy 0, policy_version 1354750 (0.0008) [2023-12-27 15:02:04,586][20347] Updated weights for policy 0, policy_version 1354760 (0.0010) [2023-12-27 15:02:04,636][20347] Updated weights for policy 0, policy_version 1354770 (0.0010) [2023-12-27 15:02:04,785][20381] Updated weights for policy 1, policy_version 1353594 (0.0007) [2023-12-27 15:02:04,847][20381] Updated weights for policy 1, policy_version 1353604 (0.0007) [2023-12-27 15:02:04,905][20381] Updated weights for policy 1, policy_version 1353614 (0.0007) [2023-12-27 15:02:05,317][20347] Updated weights for policy 0, policy_version 1354780 (0.0010) [2023-12-27 15:02:05,372][20347] Updated weights for policy 0, policy_version 1354790 (0.0008) [2023-12-27 15:02:05,430][20347] Updated weights for policy 0, policy_version 1354800 (0.0007) [2023-12-27 15:02:05,513][20381] Updated weights for policy 1, policy_version 1353624 (0.0010) [2023-12-27 15:02:05,568][20381] Updated weights for policy 1, policy_version 1353634 (0.0011) [2023-12-27 15:02:05,626][20381] Updated weights for policy 1, policy_version 1353644 (0.0011) [2023-12-27 15:02:06,038][20347] Updated weights for policy 0, policy_version 1354810 (0.0006) [2023-12-27 15:02:06,094][20347] Updated weights for policy 0, policy_version 1354820 (0.0005) [2023-12-27 15:02:06,157][20347] Updated weights for policy 0, policy_version 1354830 (0.0008) [2023-12-27 15:02:06,221][20347] Updated weights for policy 0, policy_version 1354840 (0.0007) [2023-12-27 15:02:06,333][20381] Updated weights for policy 1, policy_version 1353654 (0.0011) [2023-12-27 15:02:06,390][20381] Updated weights for policy 1, policy_version 1353664 (0.0011) [2023-12-27 15:02:06,451][20381] Updated weights for policy 1, policy_version 1353674 (0.0011) [2023-12-27 15:02:06,805][20347] Updated weights for policy 0, policy_version 1354850 (0.0006) [2023-12-27 15:02:06,858][20347] Updated weights for policy 0, policy_version 1354860 (0.0005) [2023-12-27 15:02:06,919][20347] Updated weights for policy 0, policy_version 1354870 (0.0007) [2023-12-27 15:02:07,165][20381] Updated weights for policy 1, policy_version 1353684 (0.0011) [2023-12-27 15:02:07,221][20381] Updated weights for policy 1, policy_version 1353694 (0.0011) [2023-12-27 15:02:07,278][20381] Updated weights for policy 1, policy_version 1353704 (0.0011) [2023-12-27 15:02:07,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 693493760. Throughput: 0: 10300.7, 1: 10216.1. Samples: 693459092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:07,363][19170] Avg episode reward: [(0, '873.750'), (1, '853.730')] [2023-12-27 15:02:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001354872_346914816.pth... [2023-12-27 15:02:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001353712_346578944.pth... [2023-12-27 15:02:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001352528_346275840.pth [2023-12-27 15:02:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001353656_346603520.pth [2023-12-27 15:02:07,570][20347] Updated weights for policy 0, policy_version 1354880 (0.0005) [2023-12-27 15:02:07,626][20347] Updated weights for policy 0, policy_version 1354890 (0.0005) [2023-12-27 15:02:07,681][20347] Updated weights for policy 0, policy_version 1354900 (0.0007) [2023-12-27 15:02:07,934][20381] Updated weights for policy 1, policy_version 1353714 (0.0010) [2023-12-27 15:02:07,992][20381] Updated weights for policy 1, policy_version 1353724 (0.0008) [2023-12-27 15:02:08,053][20381] Updated weights for policy 1, policy_version 1353734 (0.0008) [2023-12-27 15:02:08,113][20381] Updated weights for policy 1, policy_version 1353744 (0.0011) [2023-12-27 15:02:08,350][20347] Updated weights for policy 0, policy_version 1354910 (0.0007) [2023-12-27 15:02:08,405][20347] Updated weights for policy 0, policy_version 1354920 (0.0006) [2023-12-27 15:02:08,468][20347] Updated weights for policy 0, policy_version 1354930 (0.0010) [2023-12-27 15:02:08,825][20381] Updated weights for policy 1, policy_version 1353754 (0.0011) [2023-12-27 15:02:08,875][20381] Updated weights for policy 1, policy_version 1353764 (0.0011) [2023-12-27 15:02:08,937][20381] Updated weights for policy 1, policy_version 1353774 (0.0010) [2023-12-27 15:02:09,133][20347] Updated weights for policy 0, policy_version 1354940 (0.0010) [2023-12-27 15:02:09,179][20347] Updated weights for policy 0, policy_version 1354950 (0.0008) [2023-12-27 15:02:09,245][20347] Updated weights for policy 0, policy_version 1354960 (0.0007) [2023-12-27 15:02:09,619][20381] Updated weights for policy 1, policy_version 1353784 (0.0008) [2023-12-27 15:02:09,688][20381] Updated weights for policy 1, policy_version 1353794 (0.0008) [2023-12-27 15:02:09,759][20381] Updated weights for policy 1, policy_version 1353804 (0.0007) [2023-12-27 15:02:09,982][20347] Updated weights for policy 0, policy_version 1354970 (0.0007) [2023-12-27 15:02:10,048][20347] Updated weights for policy 0, policy_version 1354980 (0.0008) [2023-12-27 15:02:10,109][20347] Updated weights for policy 0, policy_version 1354990 (0.0009) [2023-12-27 15:02:10,173][20347] Updated weights for policy 0, policy_version 1355000 (0.0008) [2023-12-27 15:02:10,449][20381] Updated weights for policy 1, policy_version 1353814 (0.0009) [2023-12-27 15:02:10,501][20381] Updated weights for policy 1, policy_version 1353824 (0.0007) [2023-12-27 15:02:10,553][20381] Updated weights for policy 1, policy_version 1353834 (0.0007) [2023-12-27 15:02:10,832][20347] Updated weights for policy 0, policy_version 1355010 (0.0008) [2023-12-27 15:02:10,892][20347] Updated weights for policy 0, policy_version 1355020 (0.0006) [2023-12-27 15:02:10,945][20347] Updated weights for policy 0, policy_version 1355030 (0.0006) [2023-12-27 15:02:11,262][20381] Updated weights for policy 1, policy_version 1353844 (0.0008) [2023-12-27 15:02:11,326][20381] Updated weights for policy 1, policy_version 1353854 (0.0007) [2023-12-27 15:02:11,395][20381] Updated weights for policy 1, policy_version 1353864 (0.0007) [2023-12-27 15:02:11,754][20347] Updated weights for policy 0, policy_version 1355040 (0.0008) [2023-12-27 15:02:11,818][20347] Updated weights for policy 0, policy_version 1355050 (0.0008) [2023-12-27 15:02:11,875][20347] Updated weights for policy 0, policy_version 1355060 (0.0007) [2023-12-27 15:02:12,103][20381] Updated weights for policy 1, policy_version 1353874 (0.0006) [2023-12-27 15:02:12,168][20381] Updated weights for policy 1, policy_version 1353884 (0.0008) [2023-12-27 15:02:12,231][20381] Updated weights for policy 1, policy_version 1353894 (0.0010) [2023-12-27 15:02:12,295][20381] Updated weights for policy 1, policy_version 1353904 (0.0008) [2023-12-27 15:02:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 693592064. Throughput: 0: 10348.1, 1: 10187.9. Samples: 693582392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:12,363][19170] Avg episode reward: [(0, '844.220'), (1, '824.410')] [2023-12-27 15:02:12,563][20291] KL-divergence is very high: 116.7837 [2023-12-27 15:02:12,575][20347] Updated weights for policy 0, policy_version 1355070 (0.0006) [2023-12-27 15:02:12,584][20291] KL-divergence is very high: 101.7121 [2023-12-27 15:02:12,631][20291] KL-divergence is very high: 101.3402 [2023-12-27 15:02:12,646][20347] Updated weights for policy 0, policy_version 1355080 (0.0006) [2023-12-27 15:02:12,712][20347] Updated weights for policy 0, policy_version 1355090 (0.0006) [2023-12-27 15:02:12,977][20381] Updated weights for policy 1, policy_version 1353914 (0.0005) [2023-12-27 15:02:13,023][20381] Updated weights for policy 1, policy_version 1353924 (0.0005) [2023-12-27 15:02:13,072][20381] Updated weights for policy 1, policy_version 1353934 (0.0005) [2023-12-27 15:02:13,360][20347] Updated weights for policy 0, policy_version 1355100 (0.0006) [2023-12-27 15:02:13,393][20291] KL-divergence is very high: 130.0710 [2023-12-27 15:02:13,399][20291] KL-divergence is very high: 159.1059 [2023-12-27 15:02:13,409][20347] Updated weights for policy 0, policy_version 1355110 (0.0007) [2023-12-27 15:02:13,454][20347] Updated weights for policy 0, policy_version 1355120 (0.0007) [2023-12-27 15:02:13,641][20381] Updated weights for policy 1, policy_version 1353944 (0.0005) [2023-12-27 15:02:13,691][20381] Updated weights for policy 1, policy_version 1353954 (0.0005) [2023-12-27 15:02:13,747][20381] Updated weights for policy 1, policy_version 1353964 (0.0008) [2023-12-27 15:02:14,161][20347] Updated weights for policy 0, policy_version 1355130 (0.0007) [2023-12-27 15:02:14,218][20347] Updated weights for policy 0, policy_version 1355140 (0.0007) [2023-12-27 15:02:14,275][20347] Updated weights for policy 0, policy_version 1355150 (0.0007) [2023-12-27 15:02:14,329][20347] Updated weights for policy 0, policy_version 1355160 (0.0006) [2023-12-27 15:02:14,396][20381] Updated weights for policy 1, policy_version 1353974 (0.0006) [2023-12-27 15:02:14,452][20381] Updated weights for policy 1, policy_version 1353984 (0.0006) [2023-12-27 15:02:14,506][20381] Updated weights for policy 1, policy_version 1353994 (0.0010) [2023-12-27 15:02:15,048][20347] Updated weights for policy 0, policy_version 1355170 (0.0007) [2023-12-27 15:02:15,110][20347] Updated weights for policy 0, policy_version 1355180 (0.0006) [2023-12-27 15:02:15,128][20381] Updated weights for policy 1, policy_version 1354004 (0.0010) [2023-12-27 15:02:15,167][20347] Updated weights for policy 0, policy_version 1355190 (0.0006) [2023-12-27 15:02:15,188][20381] Updated weights for policy 1, policy_version 1354014 (0.0011) [2023-12-27 15:02:15,246][20381] Updated weights for policy 1, policy_version 1354024 (0.0011) [2023-12-27 15:02:15,765][20347] Updated weights for policy 0, policy_version 1355200 (0.0005) [2023-12-27 15:02:15,826][20347] Updated weights for policy 0, policy_version 1355210 (0.0008) [2023-12-27 15:02:15,872][20381] Updated weights for policy 1, policy_version 1354034 (0.0011) [2023-12-27 15:02:15,888][20347] Updated weights for policy 0, policy_version 1355220 (0.0007) [2023-12-27 15:02:15,929][20381] Updated weights for policy 1, policy_version 1354044 (0.0010) [2023-12-27 15:02:15,984][20381] Updated weights for policy 1, policy_version 1354054 (0.0010) [2023-12-27 15:02:16,045][20381] Updated weights for policy 1, policy_version 1354064 (0.0005) [2023-12-27 15:02:16,442][20347] Updated weights for policy 0, policy_version 1355230 (0.0006) [2023-12-27 15:02:16,506][20347] Updated weights for policy 0, policy_version 1355240 (0.0007) [2023-12-27 15:02:16,554][20347] Updated weights for policy 0, policy_version 1355250 (0.0010) [2023-12-27 15:02:16,634][20381] Updated weights for policy 1, policy_version 1354074 (0.0007) [2023-12-27 15:02:16,686][20381] Updated weights for policy 1, policy_version 1354084 (0.0007) [2023-12-27 15:02:16,733][20381] Updated weights for policy 1, policy_version 1354094 (0.0008) [2023-12-27 15:02:17,211][20347] Updated weights for policy 0, policy_version 1355260 (0.0009) [2023-12-27 15:02:17,272][20347] Updated weights for policy 0, policy_version 1355270 (0.0007) [2023-12-27 15:02:17,334][20347] Updated weights for policy 0, policy_version 1355280 (0.0007) [2023-12-27 15:02:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 693690368. Throughput: 0: 10338.0, 1: 10257.3. Samples: 693708096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:17,363][19170] Avg episode reward: [(0, '785.480'), (1, '814.530')] [2023-12-27 15:02:17,409][20381] Updated weights for policy 1, policy_version 1354104 (0.0010) [2023-12-27 15:02:17,465][20381] Updated weights for policy 1, policy_version 1354114 (0.0010) [2023-12-27 15:02:17,533][20381] Updated weights for policy 1, policy_version 1354124 (0.0010) [2023-12-27 15:02:18,025][20347] Updated weights for policy 0, policy_version 1355290 (0.0007) [2023-12-27 15:02:18,079][20347] Updated weights for policy 0, policy_version 1355300 (0.0007) [2023-12-27 15:02:18,143][20347] Updated weights for policy 0, policy_version 1355310 (0.0007) [2023-12-27 15:02:18,205][20347] Updated weights for policy 0, policy_version 1355320 (0.0006) [2023-12-27 15:02:18,207][20381] Updated weights for policy 1, policy_version 1354134 (0.0010) [2023-12-27 15:02:18,274][20381] Updated weights for policy 1, policy_version 1354144 (0.0010) [2023-12-27 15:02:18,337][20381] Updated weights for policy 1, policy_version 1354154 (0.0010) [2023-12-27 15:02:18,898][20347] Updated weights for policy 0, policy_version 1355330 (0.0005) [2023-12-27 15:02:18,956][20347] Updated weights for policy 0, policy_version 1355340 (0.0006) [2023-12-27 15:02:19,010][20347] Updated weights for policy 0, policy_version 1355350 (0.0007) [2023-12-27 15:02:19,051][20381] Updated weights for policy 1, policy_version 1354164 (0.0009) [2023-12-27 15:02:19,102][20381] Updated weights for policy 1, policy_version 1354174 (0.0010) [2023-12-27 15:02:19,154][20381] Updated weights for policy 1, policy_version 1354184 (0.0010) [2023-12-27 15:02:19,739][20347] Updated weights for policy 0, policy_version 1355360 (0.0008) [2023-12-27 15:02:19,795][20347] Updated weights for policy 0, policy_version 1355370 (0.0008) [2023-12-27 15:02:19,858][20347] Updated weights for policy 0, policy_version 1355380 (0.0008) [2023-12-27 15:02:19,895][20381] Updated weights for policy 1, policy_version 1354194 (0.0010) [2023-12-27 15:02:19,960][20381] Updated weights for policy 1, policy_version 1354204 (0.0011) [2023-12-27 15:02:20,019][20381] Updated weights for policy 1, policy_version 1354214 (0.0010) [2023-12-27 15:02:20,071][20381] Updated weights for policy 1, policy_version 1354224 (0.0010) [2023-12-27 15:02:20,614][20347] Updated weights for policy 0, policy_version 1355390 (0.0007) [2023-12-27 15:02:20,674][20347] Updated weights for policy 0, policy_version 1355400 (0.0008) [2023-12-27 15:02:20,728][20347] Updated weights for policy 0, policy_version 1355410 (0.0007) [2023-12-27 15:02:20,766][20381] Updated weights for policy 1, policy_version 1354234 (0.0010) [2023-12-27 15:02:20,831][20381] Updated weights for policy 1, policy_version 1354244 (0.0008) [2023-12-27 15:02:20,899][20381] Updated weights for policy 1, policy_version 1354254 (0.0006) [2023-12-27 15:02:21,432][20347] Updated weights for policy 0, policy_version 1355420 (0.0008) [2023-12-27 15:02:21,487][20347] Updated weights for policy 0, policy_version 1355430 (0.0007) [2023-12-27 15:02:21,548][20347] Updated weights for policy 0, policy_version 1355440 (0.0007) [2023-12-27 15:02:21,579][20381] Updated weights for policy 1, policy_version 1354264 (0.0008) [2023-12-27 15:02:21,647][20381] Updated weights for policy 1, policy_version 1354274 (0.0009) [2023-12-27 15:02:21,709][20381] Updated weights for policy 1, policy_version 1354284 (0.0011) [2023-12-27 15:02:22,235][20347] Updated weights for policy 0, policy_version 1355450 (0.0006) [2023-12-27 15:02:22,296][20347] Updated weights for policy 0, policy_version 1355460 (0.0006) [2023-12-27 15:02:22,351][20347] Updated weights for policy 0, policy_version 1355470 (0.0006) [2023-12-27 15:02:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 693788672. Throughput: 0: 10287.5, 1: 10191.7. Samples: 693766140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:22,363][19170] Avg episode reward: [(0, '834.600'), (1, '873.270')] [2023-12-27 15:02:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001354288_346726400.pth... [2023-12-27 15:02:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001353104_346423296.pth [2023-12-27 15:02:22,413][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001355480_347070464.pth... [2023-12-27 15:02:22,414][20347] Updated weights for policy 0, policy_version 1355480 (0.0008) [2023-12-27 15:02:22,417][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001354264_346759168.pth [2023-12-27 15:02:22,460][20381] Updated weights for policy 1, policy_version 1354294 (0.0007) [2023-12-27 15:02:22,522][20381] Updated weights for policy 1, policy_version 1354304 (0.0008) [2023-12-27 15:02:22,576][20381] Updated weights for policy 1, policy_version 1354314 (0.0008) [2023-12-27 15:02:23,107][20347] Updated weights for policy 0, policy_version 1355490 (0.0008) [2023-12-27 15:02:23,169][20347] Updated weights for policy 0, policy_version 1355500 (0.0007) [2023-12-27 15:02:23,229][20347] Updated weights for policy 0, policy_version 1355510 (0.0007) [2023-12-27 15:02:23,310][20381] Updated weights for policy 1, policy_version 1354324 (0.0007) [2023-12-27 15:02:23,369][20381] Updated weights for policy 1, policy_version 1354334 (0.0008) [2023-12-27 15:02:23,428][20381] Updated weights for policy 1, policy_version 1354344 (0.0007) [2023-12-27 15:02:23,853][20347] Updated weights for policy 0, policy_version 1355520 (0.0007) [2023-12-27 15:02:23,903][20347] Updated weights for policy 0, policy_version 1355530 (0.0007) [2023-12-27 15:02:23,947][20347] Updated weights for policy 0, policy_version 1355540 (0.0007) [2023-12-27 15:02:24,109][20381] Updated weights for policy 1, policy_version 1354354 (0.0007) [2023-12-27 15:02:24,162][20381] Updated weights for policy 1, policy_version 1354364 (0.0006) [2023-12-27 15:02:24,218][20381] Updated weights for policy 1, policy_version 1354374 (0.0010) [2023-12-27 15:02:24,268][20381] Updated weights for policy 1, policy_version 1354384 (0.0010) [2023-12-27 15:02:24,585][20347] Updated weights for policy 0, policy_version 1355550 (0.0006) [2023-12-27 15:02:24,647][20347] Updated weights for policy 0, policy_version 1355560 (0.0005) [2023-12-27 15:02:24,708][20347] Updated weights for policy 0, policy_version 1355570 (0.0005) [2023-12-27 15:02:24,908][20381] Updated weights for policy 1, policy_version 1354394 (0.0008) [2023-12-27 15:02:24,961][20381] Updated weights for policy 1, policy_version 1354404 (0.0007) [2023-12-27 15:02:25,013][20381] Updated weights for policy 1, policy_version 1354414 (0.0008) [2023-12-27 15:02:25,298][20347] Updated weights for policy 0, policy_version 1355580 (0.0007) [2023-12-27 15:02:25,349][20347] Updated weights for policy 0, policy_version 1355590 (0.0008) [2023-12-27 15:02:25,394][20347] Updated weights for policy 0, policy_version 1355600 (0.0007) [2023-12-27 15:02:25,592][20381] Updated weights for policy 1, policy_version 1354424 (0.0006) [2023-12-27 15:02:25,638][20381] Updated weights for policy 1, policy_version 1354434 (0.0005) [2023-12-27 15:02:25,689][20381] Updated weights for policy 1, policy_version 1354444 (0.0005) [2023-12-27 15:02:26,070][20347] Updated weights for policy 0, policy_version 1355610 (0.0007) [2023-12-27 15:02:26,131][20347] Updated weights for policy 0, policy_version 1355620 (0.0007) [2023-12-27 15:02:26,181][20347] Updated weights for policy 0, policy_version 1355630 (0.0007) [2023-12-27 15:02:26,232][20347] Updated weights for policy 0, policy_version 1355640 (0.0007) [2023-12-27 15:02:26,371][20381] Updated weights for policy 1, policy_version 1354454 (0.0005) [2023-12-27 15:02:26,423][20381] Updated weights for policy 1, policy_version 1354464 (0.0005) [2023-12-27 15:02:26,467][20381] Updated weights for policy 1, policy_version 1354474 (0.0005) [2023-12-27 15:02:26,936][20347] Updated weights for policy 0, policy_version 1355650 (0.0008) [2023-12-27 15:02:26,979][20347] Updated weights for policy 0, policy_version 1355660 (0.0009) [2023-12-27 15:02:27,023][20347] Updated weights for policy 0, policy_version 1355670 (0.0006) [2023-12-27 15:02:27,055][20381] Updated weights for policy 1, policy_version 1354484 (0.0006) [2023-12-27 15:02:27,100][20381] Updated weights for policy 1, policy_version 1354494 (0.0005) [2023-12-27 15:02:27,143][20381] Updated weights for policy 1, policy_version 1354504 (0.0005) [2023-12-27 15:02:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 693903360. Throughput: 0: 10312.8, 1: 10232.9. Samples: 693889440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:27,364][19170] Avg episode reward: [(0, '883.220'), (1, '873.340')] [2023-12-27 15:02:27,730][20381] Updated weights for policy 1, policy_version 1354514 (0.0005) [2023-12-27 15:02:27,768][20347] Updated weights for policy 0, policy_version 1355680 (0.0008) [2023-12-27 15:02:27,780][20381] Updated weights for policy 1, policy_version 1354524 (0.0005) [2023-12-27 15:02:27,816][20347] Updated weights for policy 0, policy_version 1355690 (0.0006) [2023-12-27 15:02:27,829][20381] Updated weights for policy 1, policy_version 1354534 (0.0005) [2023-12-27 15:02:27,868][20347] Updated weights for policy 0, policy_version 1355700 (0.0007) [2023-12-27 15:02:27,886][20381] Updated weights for policy 1, policy_version 1354544 (0.0007) [2023-12-27 15:02:28,480][20381] Updated weights for policy 1, policy_version 1354554 (0.0006) [2023-12-27 15:02:28,536][20381] Updated weights for policy 1, policy_version 1354564 (0.0005) [2023-12-27 15:02:28,596][20381] Updated weights for policy 1, policy_version 1354574 (0.0006) [2023-12-27 15:02:28,615][20347] Updated weights for policy 0, policy_version 1355710 (0.0006) [2023-12-27 15:02:28,676][20347] Updated weights for policy 0, policy_version 1355720 (0.0008) [2023-12-27 15:02:28,722][20347] Updated weights for policy 0, policy_version 1355730 (0.0008) [2023-12-27 15:02:29,144][20381] Updated weights for policy 1, policy_version 1354584 (0.0006) [2023-12-27 15:02:29,201][20381] Updated weights for policy 1, policy_version 1354594 (0.0007) [2023-12-27 15:02:29,262][20381] Updated weights for policy 1, policy_version 1354604 (0.0009) [2023-12-27 15:02:29,476][20347] Updated weights for policy 0, policy_version 1355740 (0.0007) [2023-12-27 15:02:29,535][20347] Updated weights for policy 0, policy_version 1355750 (0.0007) [2023-12-27 15:02:29,596][20347] Updated weights for policy 0, policy_version 1355760 (0.0009) [2023-12-27 15:02:29,845][20381] Updated weights for policy 1, policy_version 1354614 (0.0007) [2023-12-27 15:02:29,901][20381] Updated weights for policy 1, policy_version 1354624 (0.0008) [2023-12-27 15:02:29,968][20381] Updated weights for policy 1, policy_version 1354634 (0.0008) [2023-12-27 15:02:30,337][20347] Updated weights for policy 0, policy_version 1355770 (0.0008) [2023-12-27 15:02:30,395][20347] Updated weights for policy 0, policy_version 1355780 (0.0008) [2023-12-27 15:02:30,450][20347] Updated weights for policy 0, policy_version 1355790 (0.0007) [2023-12-27 15:02:30,505][20347] Updated weights for policy 0, policy_version 1355800 (0.0008) [2023-12-27 15:02:30,636][20381] Updated weights for policy 1, policy_version 1354644 (0.0005) [2023-12-27 15:02:30,690][20381] Updated weights for policy 1, policy_version 1354654 (0.0005) [2023-12-27 15:02:30,744][20381] Updated weights for policy 1, policy_version 1354664 (0.0005) [2023-12-27 15:02:31,261][20347] Updated weights for policy 0, policy_version 1355810 (0.0007) [2023-12-27 15:02:31,329][20347] Updated weights for policy 0, policy_version 1355820 (0.0008) [2023-12-27 15:02:31,401][20347] Updated weights for policy 0, policy_version 1355830 (0.0009) [2023-12-27 15:02:31,478][20381] Updated weights for policy 1, policy_version 1354674 (0.0007) [2023-12-27 15:02:31,538][20381] Updated weights for policy 1, policy_version 1354684 (0.0005) [2023-12-27 15:02:31,598][20381] Updated weights for policy 1, policy_version 1354694 (0.0005) [2023-12-27 15:02:31,667][20381] Updated weights for policy 1, policy_version 1354704 (0.0006) [2023-12-27 15:02:32,105][20347] Updated weights for policy 0, policy_version 1355840 (0.0008) [2023-12-27 15:02:32,165][20347] Updated weights for policy 0, policy_version 1355850 (0.0007) [2023-12-27 15:02:32,231][20347] Updated weights for policy 0, policy_version 1355860 (0.0006) [2023-12-27 15:02:32,264][20381] Updated weights for policy 1, policy_version 1354714 (0.0007) [2023-12-27 15:02:32,319][20381] Updated weights for policy 1, policy_version 1354724 (0.0007) [2023-12-27 15:02:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 694001664. Throughput: 0: 10195.4, 1: 10353.7. Samples: 694013684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:32,363][19170] Avg episode reward: [(0, '902.620'), (1, '882.650')] [2023-12-27 15:02:32,375][20381] Updated weights for policy 1, policy_version 1354734 (0.0007) [2023-12-27 15:02:32,914][20347] Updated weights for policy 0, policy_version 1355870 (0.0007) [2023-12-27 15:02:32,959][20347] Updated weights for policy 0, policy_version 1355880 (0.0007) [2023-12-27 15:02:33,008][20347] Updated weights for policy 0, policy_version 1355890 (0.0007) [2023-12-27 15:02:33,058][20381] Updated weights for policy 1, policy_version 1354744 (0.0008) [2023-12-27 15:02:33,119][20381] Updated weights for policy 1, policy_version 1354754 (0.0007) [2023-12-27 15:02:33,174][20381] Updated weights for policy 1, policy_version 1354764 (0.0007) [2023-12-27 15:02:33,637][20347] Updated weights for policy 0, policy_version 1355900 (0.0007) [2023-12-27 15:02:33,689][20347] Updated weights for policy 0, policy_version 1355910 (0.0009) [2023-12-27 15:02:33,744][20347] Updated weights for policy 0, policy_version 1355920 (0.0007) [2023-12-27 15:02:33,800][20381] Updated weights for policy 1, policy_version 1354774 (0.0006) [2023-12-27 15:02:33,855][20381] Updated weights for policy 1, policy_version 1354784 (0.0005) [2023-12-27 15:02:33,908][20381] Updated weights for policy 1, policy_version 1354794 (0.0005) [2023-12-27 15:02:34,449][20347] Updated weights for policy 0, policy_version 1355930 (0.0007) [2023-12-27 15:02:34,461][20381] Updated weights for policy 1, policy_version 1354804 (0.0005) [2023-12-27 15:02:34,499][20292] KL-divergence is very high: 194.7357 [2023-12-27 15:02:34,513][20292] KL-divergence is very high: 102.8245 [2023-12-27 15:02:34,514][20347] Updated weights for policy 0, policy_version 1355940 (0.0007) [2023-12-27 15:02:34,524][20381] Updated weights for policy 1, policy_version 1354814 (0.0006) [2023-12-27 15:02:34,549][20292] KL-divergence is very high: 221.5752 [2023-12-27 15:02:34,575][20347] Updated weights for policy 0, policy_version 1355950 (0.0007) [2023-12-27 15:02:34,584][20381] Updated weights for policy 1, policy_version 1354824 (0.0006) [2023-12-27 15:02:34,599][20292] KL-divergence is very high: 114.5371 [2023-12-27 15:02:34,634][20347] Updated weights for policy 0, policy_version 1355960 (0.0006) [2023-12-27 15:02:35,239][20381] Updated weights for policy 1, policy_version 1354834 (0.0006) [2023-12-27 15:02:35,263][20347] Updated weights for policy 0, policy_version 1355970 (0.0006) [2023-12-27 15:02:35,296][20381] Updated weights for policy 1, policy_version 1354844 (0.0007) [2023-12-27 15:02:35,313][20347] Updated weights for policy 0, policy_version 1355980 (0.0007) [2023-12-27 15:02:35,333][20292] KL-divergence is very high: 148.5759 [2023-12-27 15:02:35,354][20381] Updated weights for policy 1, policy_version 1354854 (0.0007) [2023-12-27 15:02:35,367][20347] Updated weights for policy 0, policy_version 1355990 (0.0005) [2023-12-27 15:02:35,392][20292] KL-divergence is very high: 211.9873 [2023-12-27 15:02:35,407][20381] Updated weights for policy 1, policy_version 1354864 (0.0008) [2023-12-27 15:02:35,962][20347] Updated weights for policy 0, policy_version 1356000 (0.0007) [2023-12-27 15:02:36,007][20347] Updated weights for policy 0, policy_version 1356010 (0.0007) [2023-12-27 15:02:36,055][20347] Updated weights for policy 0, policy_version 1356020 (0.0007) [2023-12-27 15:02:36,082][20381] Updated weights for policy 1, policy_version 1354874 (0.0007) [2023-12-27 15:02:36,149][20381] Updated weights for policy 1, policy_version 1354884 (0.0007) [2023-12-27 15:02:36,217][20381] Updated weights for policy 1, policy_version 1354894 (0.0006) [2023-12-27 15:02:36,775][20347] Updated weights for policy 0, policy_version 1356030 (0.0006) [2023-12-27 15:02:36,804][20381] Updated weights for policy 1, policy_version 1354904 (0.0007) [2023-12-27 15:02:36,827][20347] Updated weights for policy 0, policy_version 1356040 (0.0006) [2023-12-27 15:02:36,862][20381] Updated weights for policy 1, policy_version 1354914 (0.0006) [2023-12-27 15:02:36,886][20347] Updated weights for policy 0, policy_version 1356050 (0.0007) [2023-12-27 15:02:36,922][20381] Updated weights for policy 1, policy_version 1354924 (0.0006) [2023-12-27 15:02:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 694108160. Throughput: 0: 10201.0, 1: 10382.1. Samples: 694076724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:37,363][19170] Avg episode reward: [(0, '902.670'), (1, '814.080')] [2023-12-27 15:02:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001354928_346890240.pth... [2023-12-27 15:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001356056_347217920.pth... [2023-12-27 15:02:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001353712_346578944.pth [2023-12-27 15:02:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001354872_346914816.pth [2023-12-27 15:02:37,511][20381] Updated weights for policy 1, policy_version 1354934 (0.0007) [2023-12-27 15:02:37,570][20381] Updated weights for policy 1, policy_version 1354944 (0.0006) [2023-12-27 15:02:37,608][20347] Updated weights for policy 0, policy_version 1356060 (0.0007) [2023-12-27 15:02:37,628][20381] Updated weights for policy 1, policy_version 1354954 (0.0006) [2023-12-27 15:02:37,651][20347] Updated weights for policy 0, policy_version 1356070 (0.0008) [2023-12-27 15:02:37,697][20347] Updated weights for policy 0, policy_version 1356080 (0.0007) [2023-12-27 15:02:38,207][20381] Updated weights for policy 1, policy_version 1354964 (0.0005) [2023-12-27 15:02:38,268][20381] Updated weights for policy 1, policy_version 1354974 (0.0005) [2023-12-27 15:02:38,330][20381] Updated weights for policy 1, policy_version 1354984 (0.0008) [2023-12-27 15:02:38,384][20347] Updated weights for policy 0, policy_version 1356090 (0.0008) [2023-12-27 15:02:38,415][20291] KL-divergence is very high: 123.7394 [2023-12-27 15:02:38,434][20291] KL-divergence is very high: 109.5677 [2023-12-27 15:02:38,444][20347] Updated weights for policy 0, policy_version 1356100 (0.0007) [2023-12-27 15:02:38,461][20291] KL-divergence is very high: 112.4165 [2023-12-27 15:02:38,503][20347] Updated weights for policy 0, policy_version 1356110 (0.0007) [2023-12-27 15:02:38,560][20347] Updated weights for policy 0, policy_version 1356120 (0.0008) [2023-12-27 15:02:39,008][20381] Updated weights for policy 1, policy_version 1354994 (0.0007) [2023-12-27 15:02:39,057][20381] Updated weights for policy 1, policy_version 1355004 (0.0008) [2023-12-27 15:02:39,112][20381] Updated weights for policy 1, policy_version 1355014 (0.0009) [2023-12-27 15:02:39,170][20381] Updated weights for policy 1, policy_version 1355024 (0.0007) [2023-12-27 15:02:39,220][20347] Updated weights for policy 0, policy_version 1356130 (0.0007) [2023-12-27 15:02:39,285][20347] Updated weights for policy 0, policy_version 1356140 (0.0009) [2023-12-27 15:02:39,352][20347] Updated weights for policy 0, policy_version 1356150 (0.0008) [2023-12-27 15:02:39,900][20381] Updated weights for policy 1, policy_version 1355034 (0.0008) [2023-12-27 15:02:39,966][20381] Updated weights for policy 1, policy_version 1355044 (0.0008) [2023-12-27 15:02:40,028][20381] Updated weights for policy 1, policy_version 1355054 (0.0007) [2023-12-27 15:02:40,090][20347] Updated weights for policy 0, policy_version 1356160 (0.0008) [2023-12-27 15:02:40,150][20347] Updated weights for policy 0, policy_version 1356170 (0.0007) [2023-12-27 15:02:40,217][20347] Updated weights for policy 0, policy_version 1356180 (0.0010) [2023-12-27 15:02:40,686][20381] Updated weights for policy 1, policy_version 1355064 (0.0008) [2023-12-27 15:02:40,744][20381] Updated weights for policy 1, policy_version 1355074 (0.0007) [2023-12-27 15:02:40,799][20381] Updated weights for policy 1, policy_version 1355084 (0.0007) [2023-12-27 15:02:40,879][20347] Updated weights for policy 0, policy_version 1356190 (0.0010) [2023-12-27 15:02:40,938][20347] Updated weights for policy 0, policy_version 1356200 (0.0010) [2023-12-27 15:02:40,984][20347] Updated weights for policy 0, policy_version 1356210 (0.0010) [2023-12-27 15:02:41,557][20381] Updated weights for policy 1, policy_version 1355094 (0.0007) [2023-12-27 15:02:41,610][20381] Updated weights for policy 1, policy_version 1355104 (0.0008) [2023-12-27 15:02:41,673][20381] Updated weights for policy 1, policy_version 1355114 (0.0007) [2023-12-27 15:02:41,801][20347] Updated weights for policy 0, policy_version 1356220 (0.0010) [2023-12-27 15:02:41,867][20347] Updated weights for policy 0, policy_version 1356230 (0.0008) [2023-12-27 15:02:41,926][20347] Updated weights for policy 0, policy_version 1356240 (0.0008) [2023-12-27 15:02:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 694206464. Throughput: 0: 10157.0, 1: 10423.0. Samples: 694200376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:42,363][19170] Avg episode reward: [(0, '892.600'), (1, '804.370')] [2023-12-27 15:02:42,477][20381] Updated weights for policy 1, policy_version 1355124 (0.0008) [2023-12-27 15:02:42,530][20381] Updated weights for policy 1, policy_version 1355134 (0.0008) [2023-12-27 15:02:42,586][20381] Updated weights for policy 1, policy_version 1355144 (0.0007) [2023-12-27 15:02:42,637][20347] Updated weights for policy 0, policy_version 1356250 (0.0009) [2023-12-27 15:02:42,692][20347] Updated weights for policy 0, policy_version 1356260 (0.0010) [2023-12-27 15:02:42,751][20347] Updated weights for policy 0, policy_version 1356270 (0.0010) [2023-12-27 15:02:42,810][20347] Updated weights for policy 0, policy_version 1356280 (0.0005) [2023-12-27 15:02:43,280][20381] Updated weights for policy 1, policy_version 1355154 (0.0007) [2023-12-27 15:02:43,337][20381] Updated weights for policy 1, policy_version 1355164 (0.0007) [2023-12-27 15:02:43,393][20381] Updated weights for policy 1, policy_version 1355174 (0.0007) [2023-12-27 15:02:43,443][20347] Updated weights for policy 0, policy_version 1356290 (0.0005) [2023-12-27 15:02:43,450][20381] Updated weights for policy 1, policy_version 1355184 (0.0008) [2023-12-27 15:02:43,494][20347] Updated weights for policy 0, policy_version 1356300 (0.0005) [2023-12-27 15:02:43,549][20347] Updated weights for policy 0, policy_version 1356310 (0.0006) [2023-12-27 15:02:44,132][20347] Updated weights for policy 0, policy_version 1356320 (0.0006) [2023-12-27 15:02:44,160][20381] Updated weights for policy 1, policy_version 1355194 (0.0007) [2023-12-27 15:02:44,187][20347] Updated weights for policy 0, policy_version 1356330 (0.0006) [2023-12-27 15:02:44,217][20381] Updated weights for policy 1, policy_version 1355204 (0.0006) [2023-12-27 15:02:44,244][20347] Updated weights for policy 0, policy_version 1356340 (0.0010) [2023-12-27 15:02:44,278][20381] Updated weights for policy 1, policy_version 1355214 (0.0010) [2023-12-27 15:02:44,882][20381] Updated weights for policy 1, policy_version 1355224 (0.0008) [2023-12-27 15:02:44,893][20347] Updated weights for policy 0, policy_version 1356350 (0.0009) [2023-12-27 15:02:44,946][20381] Updated weights for policy 1, policy_version 1355234 (0.0009) [2023-12-27 15:02:44,954][20347] Updated weights for policy 0, policy_version 1356360 (0.0009) [2023-12-27 15:02:45,005][20381] Updated weights for policy 1, policy_version 1355244 (0.0006) [2023-12-27 15:02:45,011][20347] Updated weights for policy 0, policy_version 1356370 (0.0009) [2023-12-27 15:02:45,616][20381] Updated weights for policy 1, policy_version 1355254 (0.0007) [2023-12-27 15:02:45,658][20347] Updated weights for policy 0, policy_version 1356380 (0.0008) [2023-12-27 15:02:45,671][20381] Updated weights for policy 1, policy_version 1355264 (0.0007) [2023-12-27 15:02:45,710][20347] Updated weights for policy 0, policy_version 1356390 (0.0010) [2023-12-27 15:02:45,728][20381] Updated weights for policy 1, policy_version 1355274 (0.0006) [2023-12-27 15:02:45,768][20347] Updated weights for policy 0, policy_version 1356400 (0.0010) [2023-12-27 15:02:46,377][20381] Updated weights for policy 1, policy_version 1355284 (0.0006) [2023-12-27 15:02:46,383][20347] Updated weights for policy 0, policy_version 1356410 (0.0008) [2023-12-27 15:02:46,426][20381] Updated weights for policy 1, policy_version 1355294 (0.0005) [2023-12-27 15:02:46,441][20347] Updated weights for policy 0, policy_version 1356420 (0.0010) [2023-12-27 15:02:46,482][20381] Updated weights for policy 1, policy_version 1355304 (0.0005) [2023-12-27 15:02:46,498][20347] Updated weights for policy 0, policy_version 1356430 (0.0010) [2023-12-27 15:02:46,553][20347] Updated weights for policy 0, policy_version 1356440 (0.0010) [2023-12-27 15:02:47,037][20381] Updated weights for policy 1, policy_version 1355314 (0.0005) [2023-12-27 15:02:47,094][20381] Updated weights for policy 1, policy_version 1355324 (0.0010) [2023-12-27 15:02:47,153][20381] Updated weights for policy 1, policy_version 1355334 (0.0010) [2023-12-27 15:02:47,208][20381] Updated weights for policy 1, policy_version 1355344 (0.0010) [2023-12-27 15:02:47,221][20347] Updated weights for policy 0, policy_version 1356450 (0.0010) [2023-12-27 15:02:47,281][20347] Updated weights for policy 0, policy_version 1356460 (0.0010) [2023-12-27 15:02:47,329][20347] Updated weights for policy 0, policy_version 1356470 (0.0010) [2023-12-27 15:02:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 694321152. Throughput: 0: 10198.1, 1: 10476.7. Samples: 694325060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:47,363][19170] Avg episode reward: [(0, '843.510'), (1, '853.180')] [2023-12-27 15:02:47,812][20381] Updated weights for policy 1, policy_version 1355354 (0.0007) [2023-12-27 15:02:47,868][20381] Updated weights for policy 1, policy_version 1355364 (0.0008) [2023-12-27 15:02:47,920][20381] Updated weights for policy 1, policy_version 1355374 (0.0008) [2023-12-27 15:02:47,980][20347] Updated weights for policy 0, policy_version 1356480 (0.0007) [2023-12-27 15:02:48,032][20347] Updated weights for policy 0, policy_version 1356490 (0.0005) [2023-12-27 15:02:48,076][20347] Updated weights for policy 0, policy_version 1356500 (0.0005) [2023-12-27 15:02:48,621][20381] Updated weights for policy 1, policy_version 1355384 (0.0007) [2023-12-27 15:02:48,672][20381] Updated weights for policy 1, policy_version 1355394 (0.0005) [2023-12-27 15:02:48,727][20347] Updated weights for policy 0, policy_version 1356510 (0.0008) [2023-12-27 15:02:48,733][20381] Updated weights for policy 1, policy_version 1355404 (0.0006) [2023-12-27 15:02:48,784][20347] Updated weights for policy 0, policy_version 1356520 (0.0006) [2023-12-27 15:02:48,840][20347] Updated weights for policy 0, policy_version 1356530 (0.0007) [2023-12-27 15:02:49,360][20381] Updated weights for policy 1, policy_version 1355414 (0.0008) [2023-12-27 15:02:49,423][20381] Updated weights for policy 1, policy_version 1355424 (0.0006) [2023-12-27 15:02:49,481][20381] Updated weights for policy 1, policy_version 1355434 (0.0007) [2023-12-27 15:02:49,526][20347] Updated weights for policy 0, policy_version 1356540 (0.0008) [2023-12-27 15:02:49,587][20347] Updated weights for policy 0, policy_version 1356550 (0.0008) [2023-12-27 15:02:49,649][20347] Updated weights for policy 0, policy_version 1356560 (0.0008) [2023-12-27 15:02:50,147][20381] Updated weights for policy 1, policy_version 1355444 (0.0007) [2023-12-27 15:02:50,207][20381] Updated weights for policy 1, policy_version 1355454 (0.0008) [2023-12-27 15:02:50,268][20381] Updated weights for policy 1, policy_version 1355464 (0.0008) [2023-12-27 15:02:50,347][20347] Updated weights for policy 0, policy_version 1356570 (0.0007) [2023-12-27 15:02:50,399][20347] Updated weights for policy 0, policy_version 1356580 (0.0005) [2023-12-27 15:02:50,451][20347] Updated weights for policy 0, policy_version 1356590 (0.0007) [2023-12-27 15:02:50,512][20347] Updated weights for policy 0, policy_version 1356600 (0.0006) [2023-12-27 15:02:50,958][20381] Updated weights for policy 1, policy_version 1355474 (0.0008) [2023-12-27 15:02:51,024][20381] Updated weights for policy 1, policy_version 1355484 (0.0009) [2023-12-27 15:02:51,085][20347] Updated weights for policy 0, policy_version 1356610 (0.0006) [2023-12-27 15:02:51,086][20381] Updated weights for policy 1, policy_version 1355494 (0.0008) [2023-12-27 15:02:51,145][20347] Updated weights for policy 0, policy_version 1356620 (0.0007) [2023-12-27 15:02:51,154][20381] Updated weights for policy 1, policy_version 1355504 (0.0007) [2023-12-27 15:02:51,211][20347] Updated weights for policy 0, policy_version 1356630 (0.0008) [2023-12-27 15:02:51,846][20381] Updated weights for policy 1, policy_version 1355514 (0.0011) [2023-12-27 15:02:51,902][20381] Updated weights for policy 1, policy_version 1355524 (0.0011) [2023-12-27 15:02:51,904][20347] Updated weights for policy 0, policy_version 1356640 (0.0006) [2023-12-27 15:02:51,954][20347] Updated weights for policy 0, policy_version 1356650 (0.0005) [2023-12-27 15:02:51,956][20381] Updated weights for policy 1, policy_version 1355534 (0.0007) [2023-12-27 15:02:52,018][20347] Updated weights for policy 0, policy_version 1356660 (0.0008) [2023-12-27 15:02:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 694419456. Throughput: 0: 10178.6, 1: 10449.6. Samples: 694387360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:52,363][19170] Avg episode reward: [(0, '882.540'), (1, '882.420')] [2023-12-27 15:02:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001355536_347045888.pth... [2023-12-27 15:02:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001356664_347373568.pth... [2023-12-27 15:02:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001355480_347070464.pth [2023-12-27 15:02:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001354288_346726400.pth [2023-12-27 15:02:52,648][20381] Updated weights for policy 1, policy_version 1355544 (0.0007) [2023-12-27 15:02:52,698][20381] Updated weights for policy 1, policy_version 1355554 (0.0008) [2023-12-27 15:02:52,747][20347] Updated weights for policy 0, policy_version 1356670 (0.0006) [2023-12-27 15:02:52,757][20381] Updated weights for policy 1, policy_version 1355564 (0.0007) [2023-12-27 15:02:52,808][20347] Updated weights for policy 0, policy_version 1356680 (0.0007) [2023-12-27 15:02:52,866][20347] Updated weights for policy 0, policy_version 1356690 (0.0007) [2023-12-27 15:02:53,483][20381] Updated weights for policy 1, policy_version 1355574 (0.0007) [2023-12-27 15:02:53,526][20347] Updated weights for policy 0, policy_version 1356700 (0.0007) [2023-12-27 15:02:53,544][20381] Updated weights for policy 1, policy_version 1355584 (0.0007) [2023-12-27 15:02:53,577][20347] Updated weights for policy 0, policy_version 1356710 (0.0006) [2023-12-27 15:02:53,606][20381] Updated weights for policy 1, policy_version 1355594 (0.0007) [2023-12-27 15:02:53,635][20347] Updated weights for policy 0, policy_version 1356720 (0.0007) [2023-12-27 15:02:54,253][20347] Updated weights for policy 0, policy_version 1356730 (0.0006) [2023-12-27 15:02:54,264][20381] Updated weights for policy 1, policy_version 1355604 (0.0007) [2023-12-27 15:02:54,314][20347] Updated weights for policy 0, policy_version 1356740 (0.0008) [2023-12-27 15:02:54,323][20381] Updated weights for policy 1, policy_version 1355614 (0.0007) [2023-12-27 15:02:54,370][20347] Updated weights for policy 0, policy_version 1356750 (0.0005) [2023-12-27 15:02:54,383][20381] Updated weights for policy 1, policy_version 1355624 (0.0006) [2023-12-27 15:02:54,430][20347] Updated weights for policy 0, policy_version 1356760 (0.0006) [2023-12-27 15:02:55,081][20381] Updated weights for policy 1, policy_version 1355634 (0.0006) [2023-12-27 15:02:55,096][20347] Updated weights for policy 0, policy_version 1356770 (0.0008) [2023-12-27 15:02:55,139][20381] Updated weights for policy 1, policy_version 1355644 (0.0007) [2023-12-27 15:02:55,151][20347] Updated weights for policy 0, policy_version 1356780 (0.0008) [2023-12-27 15:02:55,191][20381] Updated weights for policy 1, policy_version 1355654 (0.0005) [2023-12-27 15:02:55,197][20347] Updated weights for policy 0, policy_version 1356790 (0.0006) [2023-12-27 15:02:55,239][20381] Updated weights for policy 1, policy_version 1355664 (0.0006) [2023-12-27 15:02:55,868][20347] Updated weights for policy 0, policy_version 1356800 (0.0007) [2023-12-27 15:02:55,922][20347] Updated weights for policy 0, policy_version 1356810 (0.0007) [2023-12-27 15:02:55,940][20381] Updated weights for policy 1, policy_version 1355674 (0.0007) [2023-12-27 15:02:55,981][20347] Updated weights for policy 0, policy_version 1356820 (0.0008) [2023-12-27 15:02:55,993][20381] Updated weights for policy 1, policy_version 1355684 (0.0011) [2023-12-27 15:02:56,046][20381] Updated weights for policy 1, policy_version 1355694 (0.0011) [2023-12-27 15:02:56,652][20381] Updated weights for policy 1, policy_version 1355704 (0.0009) [2023-12-27 15:02:56,669][20347] Updated weights for policy 0, policy_version 1356830 (0.0006) [2023-12-27 15:02:56,709][20381] Updated weights for policy 1, policy_version 1355714 (0.0006) [2023-12-27 15:02:56,724][20347] Updated weights for policy 0, policy_version 1356840 (0.0007) [2023-12-27 15:02:56,760][20381] Updated weights for policy 1, policy_version 1355724 (0.0005) [2023-12-27 15:02:56,778][20347] Updated weights for policy 0, policy_version 1356850 (0.0007) [2023-12-27 15:02:57,284][20381] Updated weights for policy 1, policy_version 1355734 (0.0006) [2023-12-27 15:02:57,346][20381] Updated weights for policy 1, policy_version 1355744 (0.0005) [2023-12-27 15:02:57,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 694517760. Throughput: 0: 10181.4, 1: 10440.4. Samples: 694510376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:02:57,364][19170] Avg episode reward: [(0, '902.330'), (1, '902.150')] [2023-12-27 15:02:57,404][20381] Updated weights for policy 1, policy_version 1355754 (0.0006) [2023-12-27 15:02:57,517][20347] Updated weights for policy 0, policy_version 1356860 (0.0007) [2023-12-27 15:02:57,568][20347] Updated weights for policy 0, policy_version 1356870 (0.0008) [2023-12-27 15:02:57,621][20347] Updated weights for policy 0, policy_version 1356880 (0.0007) [2023-12-27 15:02:57,996][20381] Updated weights for policy 1, policy_version 1355764 (0.0008) [2023-12-27 15:02:58,051][20381] Updated weights for policy 1, policy_version 1355774 (0.0010) [2023-12-27 15:02:58,105][20381] Updated weights for policy 1, policy_version 1355784 (0.0010) [2023-12-27 15:02:58,339][20347] Updated weights for policy 0, policy_version 1356890 (0.0007) [2023-12-27 15:02:58,403][20347] Updated weights for policy 0, policy_version 1356900 (0.0007) [2023-12-27 15:02:58,465][20347] Updated weights for policy 0, policy_version 1356910 (0.0007) [2023-12-27 15:02:58,551][20347] Updated weights for policy 0, policy_version 1356920 (0.0008) [2023-12-27 15:02:58,870][20381] Updated weights for policy 1, policy_version 1355794 (0.0010) [2023-12-27 15:02:58,943][20381] Updated weights for policy 1, policy_version 1355804 (0.0008) [2023-12-27 15:02:59,006][20381] Updated weights for policy 1, policy_version 1355814 (0.0008) [2023-12-27 15:02:59,064][20381] Updated weights for policy 1, policy_version 1355824 (0.0008) [2023-12-27 15:02:59,342][20347] Updated weights for policy 0, policy_version 1356930 (0.0008) [2023-12-27 15:02:59,406][20347] Updated weights for policy 0, policy_version 1356940 (0.0008) [2023-12-27 15:02:59,470][20347] Updated weights for policy 0, policy_version 1356950 (0.0007) [2023-12-27 15:02:59,788][20381] Updated weights for policy 1, policy_version 1355834 (0.0010) [2023-12-27 15:02:59,850][20381] Updated weights for policy 1, policy_version 1355844 (0.0011) [2023-12-27 15:02:59,902][20381] Updated weights for policy 1, policy_version 1355854 (0.0010) [2023-12-27 15:03:00,155][20347] Updated weights for policy 0, policy_version 1356960 (0.0009) [2023-12-27 15:03:00,214][20347] Updated weights for policy 0, policy_version 1356970 (0.0010) [2023-12-27 15:03:00,266][20347] Updated weights for policy 0, policy_version 1356980 (0.0010) [2023-12-27 15:03:00,495][20381] Updated weights for policy 1, policy_version 1355864 (0.0008) [2023-12-27 15:03:00,551][20381] Updated weights for policy 1, policy_version 1355874 (0.0007) [2023-12-27 15:03:00,596][20381] Updated weights for policy 1, policy_version 1355884 (0.0009) [2023-12-27 15:03:00,896][20347] Updated weights for policy 0, policy_version 1356990 (0.0007) [2023-12-27 15:03:00,946][20347] Updated weights for policy 0, policy_version 1357000 (0.0005) [2023-12-27 15:03:00,992][20347] Updated weights for policy 0, policy_version 1357010 (0.0005) [2023-12-27 15:03:01,252][20381] Updated weights for policy 1, policy_version 1355895 (0.0009) [2023-12-27 15:03:01,313][20381] Updated weights for policy 1, policy_version 1355905 (0.0007) [2023-12-27 15:03:01,371][20381] Updated weights for policy 1, policy_version 1355915 (0.0007) [2023-12-27 15:03:01,688][20347] Updated weights for policy 0, policy_version 1357020 (0.0006) [2023-12-27 15:03:01,761][20347] Updated weights for policy 0, policy_version 1357030 (0.0008) [2023-12-27 15:03:01,827][20347] Updated weights for policy 0, policy_version 1357040 (0.0007) [2023-12-27 15:03:02,087][20381] Updated weights for policy 1, policy_version 1355925 (0.0007) [2023-12-27 15:03:02,140][20381] Updated weights for policy 1, policy_version 1355935 (0.0008) [2023-12-27 15:03:02,200][20381] Updated weights for policy 1, policy_version 1355945 (0.0009) [2023-12-27 15:03:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 694624256. Throughput: 0: 10132.3, 1: 10406.7. Samples: 694632352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:02,363][19170] Avg episode reward: [(0, '912.220'), (1, '902.140')] [2023-12-27 15:03:02,406][20347] Updated weights for policy 0, policy_version 1357050 (0.0007) [2023-12-27 15:03:02,463][20347] Updated weights for policy 0, policy_version 1357060 (0.0007) [2023-12-27 15:03:02,523][20347] Updated weights for policy 0, policy_version 1357070 (0.0008) [2023-12-27 15:03:02,593][20347] Updated weights for policy 0, policy_version 1357080 (0.0009) [2023-12-27 15:03:02,888][20381] Updated weights for policy 1, policy_version 1355955 (0.0009) [2023-12-27 15:03:02,944][20381] Updated weights for policy 1, policy_version 1355965 (0.0008) [2023-12-27 15:03:02,998][20381] Updated weights for policy 1, policy_version 1355975 (0.0007) [2023-12-27 15:03:03,272][20347] Updated weights for policy 0, policy_version 1357090 (0.0010) [2023-12-27 15:03:03,328][20347] Updated weights for policy 0, policy_version 1357100 (0.0010) [2023-12-27 15:03:03,384][20347] Updated weights for policy 0, policy_version 1357110 (0.0005) [2023-12-27 15:03:03,659][20381] Updated weights for policy 1, policy_version 1355985 (0.0007) [2023-12-27 15:03:03,707][20381] Updated weights for policy 1, policy_version 1355995 (0.0005) [2023-12-27 15:03:03,766][20381] Updated weights for policy 1, policy_version 1356005 (0.0005) [2023-12-27 15:03:03,813][20381] Updated weights for policy 1, policy_version 1356015 (0.0006) [2023-12-27 15:03:03,964][20347] Updated weights for policy 0, policy_version 1357120 (0.0008) [2023-12-27 15:03:04,019][20347] Updated weights for policy 0, policy_version 1357130 (0.0008) [2023-12-27 15:03:04,074][20347] Updated weights for policy 0, policy_version 1357140 (0.0008) [2023-12-27 15:03:04,472][20381] Updated weights for policy 1, policy_version 1356025 (0.0007) [2023-12-27 15:03:04,532][20381] Updated weights for policy 1, policy_version 1356035 (0.0007) [2023-12-27 15:03:04,588][20381] Updated weights for policy 1, policy_version 1356045 (0.0006) [2023-12-27 15:03:04,770][20347] Updated weights for policy 0, policy_version 1357150 (0.0007) [2023-12-27 15:03:04,827][20347] Updated weights for policy 0, policy_version 1357160 (0.0008) [2023-12-27 15:03:04,888][20347] Updated weights for policy 0, policy_version 1357170 (0.0006) [2023-12-27 15:03:05,260][20381] Updated weights for policy 1, policy_version 1356055 (0.0009) [2023-12-27 15:03:05,308][20381] Updated weights for policy 1, policy_version 1356065 (0.0008) [2023-12-27 15:03:05,354][20381] Updated weights for policy 1, policy_version 1356075 (0.0008) [2023-12-27 15:03:05,467][20347] Updated weights for policy 0, policy_version 1357180 (0.0005) [2023-12-27 15:03:05,518][20347] Updated weights for policy 0, policy_version 1357190 (0.0005) [2023-12-27 15:03:05,574][20347] Updated weights for policy 0, policy_version 1357200 (0.0005) [2023-12-27 15:03:06,070][20381] Updated weights for policy 1, policy_version 1356085 (0.0010) [2023-12-27 15:03:06,139][20347] Updated weights for policy 0, policy_version 1357210 (0.0006) [2023-12-27 15:03:06,146][20381] Updated weights for policy 1, policy_version 1356095 (0.0009) [2023-12-27 15:03:06,170][20291] KL-divergence is very high: 101.4858 [2023-12-27 15:03:06,193][20347] Updated weights for policy 0, policy_version 1357220 (0.0006) [2023-12-27 15:03:06,196][20381] Updated weights for policy 1, policy_version 1356105 (0.0010) [2023-12-27 15:03:06,260][20347] Updated weights for policy 0, policy_version 1357230 (0.0008) [2023-12-27 15:03:06,326][20347] Updated weights for policy 0, policy_version 1357240 (0.0007) [2023-12-27 15:03:06,859][20381] Updated weights for policy 1, policy_version 1356115 (0.0008) [2023-12-27 15:03:06,918][20381] Updated weights for policy 1, policy_version 1356125 (0.0010) [2023-12-27 15:03:06,981][20381] Updated weights for policy 1, policy_version 1356135 (0.0011) [2023-12-27 15:03:07,043][20347] Updated weights for policy 0, policy_version 1357250 (0.0006) [2023-12-27 15:03:07,098][20347] Updated weights for policy 0, policy_version 1357260 (0.0007) [2023-12-27 15:03:07,164][20347] Updated weights for policy 0, policy_version 1357270 (0.0008) [2023-12-27 15:03:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 694730752. Throughput: 0: 10184.9, 1: 10473.8. Samples: 694695788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:07,364][19170] Avg episode reward: [(0, '843.380'), (1, '911.920')] [2023-12-27 15:03:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001356144_347201536.pth... [2023-12-27 15:03:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001357272_347529216.pth... [2023-12-27 15:03:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001354928_346890240.pth [2023-12-27 15:03:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001356056_347217920.pth [2023-12-27 15:03:07,635][20381] Updated weights for policy 1, policy_version 1356145 (0.0010) [2023-12-27 15:03:07,686][20381] Updated weights for policy 1, policy_version 1356155 (0.0010) [2023-12-27 15:03:07,742][20381] Updated weights for policy 1, policy_version 1356165 (0.0010) [2023-12-27 15:03:07,795][20381] Updated weights for policy 1, policy_version 1356175 (0.0006) [2023-12-27 15:03:07,801][20347] Updated weights for policy 0, policy_version 1357280 (0.0008) [2023-12-27 15:03:07,853][20347] Updated weights for policy 0, policy_version 1357290 (0.0007) [2023-12-27 15:03:07,864][20291] KL-divergence is very high: 169.0837 [2023-12-27 15:03:07,903][20291] KL-divergence is very high: 136.8330 [2023-12-27 15:03:07,904][20347] Updated weights for policy 0, policy_version 1357300 (0.0007) [2023-12-27 15:03:08,474][20381] Updated weights for policy 1, policy_version 1356185 (0.0010) [2023-12-27 15:03:08,539][20381] Updated weights for policy 1, policy_version 1356195 (0.0011) [2023-12-27 15:03:08,568][20347] Updated weights for policy 0, policy_version 1357310 (0.0006) [2023-12-27 15:03:08,599][20381] Updated weights for policy 1, policy_version 1356205 (0.0010) [2023-12-27 15:03:08,627][20347] Updated weights for policy 0, policy_version 1357320 (0.0008) [2023-12-27 15:03:08,686][20347] Updated weights for policy 0, policy_version 1357330 (0.0007) [2023-12-27 15:03:09,325][20381] Updated weights for policy 1, policy_version 1356215 (0.0011) [2023-12-27 15:03:09,340][20347] Updated weights for policy 0, policy_version 1357340 (0.0007) [2023-12-27 15:03:09,395][20381] Updated weights for policy 1, policy_version 1356225 (0.0013) [2023-12-27 15:03:09,409][20347] Updated weights for policy 0, policy_version 1357350 (0.0007) [2023-12-27 15:03:09,460][20381] Updated weights for policy 1, policy_version 1356235 (0.0009) [2023-12-27 15:03:09,472][20347] Updated weights for policy 0, policy_version 1357360 (0.0006) [2023-12-27 15:03:10,152][20381] Updated weights for policy 1, policy_version 1356245 (0.0009) [2023-12-27 15:03:10,159][20347] Updated weights for policy 0, policy_version 1357370 (0.0006) [2023-12-27 15:03:10,214][20381] Updated weights for policy 1, policy_version 1356255 (0.0007) [2023-12-27 15:03:10,219][20347] Updated weights for policy 0, policy_version 1357380 (0.0007) [2023-12-27 15:03:10,273][20381] Updated weights for policy 1, policy_version 1356265 (0.0006) [2023-12-27 15:03:10,279][20347] Updated weights for policy 0, policy_version 1357390 (0.0008) [2023-12-27 15:03:10,336][20347] Updated weights for policy 0, policy_version 1357400 (0.0007) [2023-12-27 15:03:10,896][20381] Updated weights for policy 1, policy_version 1356275 (0.0007) [2023-12-27 15:03:10,951][20381] Updated weights for policy 1, policy_version 1356285 (0.0010) [2023-12-27 15:03:11,003][20381] Updated weights for policy 1, policy_version 1356295 (0.0010) [2023-12-27 15:03:11,070][20347] Updated weights for policy 0, policy_version 1357410 (0.0009) [2023-12-27 15:03:11,133][20347] Updated weights for policy 0, policy_version 1357420 (0.0008) [2023-12-27 15:03:11,190][20347] Updated weights for policy 0, policy_version 1357430 (0.0009) [2023-12-27 15:03:11,754][20381] Updated weights for policy 1, policy_version 1356305 (0.0010) [2023-12-27 15:03:11,814][20381] Updated weights for policy 1, policy_version 1356315 (0.0008) [2023-12-27 15:03:11,864][20381] Updated weights for policy 1, policy_version 1356325 (0.0008) [2023-12-27 15:03:11,927][20381] Updated weights for policy 1, policy_version 1356335 (0.0007) [2023-12-27 15:03:11,969][20347] Updated weights for policy 0, policy_version 1357440 (0.0006) [2023-12-27 15:03:12,029][20347] Updated weights for policy 0, policy_version 1357450 (0.0007) [2023-12-27 15:03:12,083][20347] Updated weights for policy 0, policy_version 1357460 (0.0008) [2023-12-27 15:03:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 694829056. Throughput: 0: 10201.1, 1: 10456.7. Samples: 694819040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:12,363][19170] Avg episode reward: [(0, '862.860'), (1, '911.860')] [2023-12-27 15:03:12,673][20381] Updated weights for policy 1, policy_version 1356345 (0.0010) [2023-12-27 15:03:12,722][20381] Updated weights for policy 1, policy_version 1356355 (0.0009) [2023-12-27 15:03:12,766][20347] Updated weights for policy 0, policy_version 1357470 (0.0007) [2023-12-27 15:03:12,775][20381] Updated weights for policy 1, policy_version 1356365 (0.0006) [2023-12-27 15:03:12,824][20347] Updated weights for policy 0, policy_version 1357480 (0.0006) [2023-12-27 15:03:12,876][20347] Updated weights for policy 0, policy_version 1357490 (0.0007) [2023-12-27 15:03:13,447][20381] Updated weights for policy 1, policy_version 1356375 (0.0007) [2023-12-27 15:03:13,504][20381] Updated weights for policy 1, policy_version 1356385 (0.0007) [2023-12-27 15:03:13,556][20381] Updated weights for policy 1, policy_version 1356395 (0.0006) [2023-12-27 15:03:13,558][20347] Updated weights for policy 0, policy_version 1357500 (0.0009) [2023-12-27 15:03:13,622][20347] Updated weights for policy 0, policy_version 1357510 (0.0008) [2023-12-27 15:03:13,689][20347] Updated weights for policy 0, policy_version 1357520 (0.0005) [2023-12-27 15:03:14,263][20347] Updated weights for policy 0, policy_version 1357530 (0.0005) [2023-12-27 15:03:14,270][20381] Updated weights for policy 1, policy_version 1356405 (0.0006) [2023-12-27 15:03:14,316][20347] Updated weights for policy 0, policy_version 1357540 (0.0005) [2023-12-27 15:03:14,321][20381] Updated weights for policy 1, policy_version 1356415 (0.0007) [2023-12-27 15:03:14,368][20347] Updated weights for policy 0, policy_version 1357550 (0.0006) [2023-12-27 15:03:14,370][20381] Updated weights for policy 1, policy_version 1356425 (0.0005) [2023-12-27 15:03:14,423][20347] Updated weights for policy 0, policy_version 1357560 (0.0006) [2023-12-27 15:03:15,077][20381] Updated weights for policy 1, policy_version 1356435 (0.0005) [2023-12-27 15:03:15,129][20347] Updated weights for policy 0, policy_version 1357570 (0.0006) [2023-12-27 15:03:15,142][20381] Updated weights for policy 1, policy_version 1356445 (0.0008) [2023-12-27 15:03:15,183][20347] Updated weights for policy 0, policy_version 1357580 (0.0006) [2023-12-27 15:03:15,206][20381] Updated weights for policy 1, policy_version 1356455 (0.0008) [2023-12-27 15:03:15,239][20347] Updated weights for policy 0, policy_version 1357590 (0.0006) [2023-12-27 15:03:15,901][20381] Updated weights for policy 1, policy_version 1356465 (0.0007) [2023-12-27 15:03:15,904][20347] Updated weights for policy 0, policy_version 1357600 (0.0006) [2023-12-27 15:03:15,947][20381] Updated weights for policy 1, policy_version 1356475 (0.0006) [2023-12-27 15:03:15,959][20347] Updated weights for policy 0, policy_version 1357610 (0.0007) [2023-12-27 15:03:15,992][20381] Updated weights for policy 1, policy_version 1356485 (0.0005) [2023-12-27 15:03:16,008][20347] Updated weights for policy 0, policy_version 1357620 (0.0007) [2023-12-27 15:03:16,038][20381] Updated weights for policy 1, policy_version 1356495 (0.0007) [2023-12-27 15:03:16,712][20347] Updated weights for policy 0, policy_version 1357630 (0.0007) [2023-12-27 15:03:16,743][20381] Updated weights for policy 1, policy_version 1356505 (0.0008) [2023-12-27 15:03:16,763][20347] Updated weights for policy 0, policy_version 1357640 (0.0007) [2023-12-27 15:03:16,791][20381] Updated weights for policy 1, policy_version 1356515 (0.0006) [2023-12-27 15:03:16,809][20347] Updated weights for policy 0, policy_version 1357650 (0.0007) [2023-12-27 15:03:16,845][20381] Updated weights for policy 1, policy_version 1356525 (0.0005) [2023-12-27 15:03:17,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 694927360. Throughput: 0: 10278.6, 1: 10294.4. Samples: 694939468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:17,363][19170] Avg episode reward: [(0, '902.000'), (1, '882.590')] [2023-12-27 15:03:17,505][20347] Updated weights for policy 0, policy_version 1357660 (0.0007) [2023-12-27 15:03:17,537][20381] Updated weights for policy 1, policy_version 1356535 (0.0006) [2023-12-27 15:03:17,557][20347] Updated weights for policy 0, policy_version 1357670 (0.0007) [2023-12-27 15:03:17,592][20381] Updated weights for policy 1, policy_version 1356545 (0.0006) [2023-12-27 15:03:17,616][20347] Updated weights for policy 0, policy_version 1357680 (0.0006) [2023-12-27 15:03:17,647][20381] Updated weights for policy 1, policy_version 1356555 (0.0006) [2023-12-27 15:03:18,270][20381] Updated weights for policy 1, policy_version 1356565 (0.0009) [2023-12-27 15:03:18,334][20381] Updated weights for policy 1, policy_version 1356575 (0.0011) [2023-12-27 15:03:18,338][20347] Updated weights for policy 0, policy_version 1357690 (0.0007) [2023-12-27 15:03:18,394][20381] Updated weights for policy 1, policy_version 1356585 (0.0009) [2023-12-27 15:03:18,396][20347] Updated weights for policy 0, policy_version 1357700 (0.0007) [2023-12-27 15:03:18,444][20347] Updated weights for policy 0, policy_version 1357710 (0.0007) [2023-12-27 15:03:18,504][20347] Updated weights for policy 0, policy_version 1357720 (0.0007) [2023-12-27 15:03:19,116][20381] Updated weights for policy 1, policy_version 1356595 (0.0009) [2023-12-27 15:03:19,173][20381] Updated weights for policy 1, policy_version 1356605 (0.0011) [2023-12-27 15:03:19,221][20347] Updated weights for policy 0, policy_version 1357730 (0.0006) [2023-12-27 15:03:19,233][20381] Updated weights for policy 1, policy_version 1356615 (0.0011) [2023-12-27 15:03:19,284][20347] Updated weights for policy 0, policy_version 1357740 (0.0007) [2023-12-27 15:03:19,337][20347] Updated weights for policy 0, policy_version 1357750 (0.0008) [2023-12-27 15:03:19,969][20381] Updated weights for policy 1, policy_version 1356625 (0.0011) [2023-12-27 15:03:20,026][20381] Updated weights for policy 1, policy_version 1356635 (0.0011) [2023-12-27 15:03:20,083][20381] Updated weights for policy 1, policy_version 1356645 (0.0011) [2023-12-27 15:03:20,125][20347] Updated weights for policy 0, policy_version 1357760 (0.0008) [2023-12-27 15:03:20,135][20381] Updated weights for policy 1, policy_version 1356655 (0.0010) [2023-12-27 15:03:20,185][20347] Updated weights for policy 0, policy_version 1357770 (0.0008) [2023-12-27 15:03:20,245][20347] Updated weights for policy 0, policy_version 1357780 (0.0006) [2023-12-27 15:03:20,830][20381] Updated weights for policy 1, policy_version 1356665 (0.0010) [2023-12-27 15:03:20,889][20381] Updated weights for policy 1, policy_version 1356675 (0.0011) [2023-12-27 15:03:20,950][20381] Updated weights for policy 1, policy_version 1356685 (0.0010) [2023-12-27 15:03:20,960][20347] Updated weights for policy 0, policy_version 1357790 (0.0009) [2023-12-27 15:03:21,013][20347] Updated weights for policy 0, policy_version 1357800 (0.0009) [2023-12-27 15:03:21,075][20347] Updated weights for policy 0, policy_version 1357810 (0.0010) [2023-12-27 15:03:21,664][20381] Updated weights for policy 1, policy_version 1356695 (0.0008) [2023-12-27 15:03:21,733][20381] Updated weights for policy 1, policy_version 1356705 (0.0008) [2023-12-27 15:03:21,802][20381] Updated weights for policy 1, policy_version 1356715 (0.0008) [2023-12-27 15:03:21,873][20347] Updated weights for policy 0, policy_version 1357820 (0.0008) [2023-12-27 15:03:21,930][20347] Updated weights for policy 0, policy_version 1357830 (0.0008) [2023-12-27 15:03:21,993][20347] Updated weights for policy 0, policy_version 1357840 (0.0008) [2023-12-27 15:03:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 695025664. Throughput: 0: 10238.5, 1: 10235.6. Samples: 694998060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:22,363][19170] Avg episode reward: [(0, '902.260'), (1, '882.740')] [2023-12-27 15:03:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001356720_347348992.pth... [2023-12-27 15:03:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001357848_347676672.pth... [2023-12-27 15:03:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001355536_347045888.pth [2023-12-27 15:03:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001356664_347373568.pth [2023-12-27 15:03:22,513][20381] Updated weights for policy 1, policy_version 1356725 (0.0007) [2023-12-27 15:03:22,566][20381] Updated weights for policy 1, policy_version 1356735 (0.0008) [2023-12-27 15:03:22,625][20381] Updated weights for policy 1, policy_version 1356745 (0.0007) [2023-12-27 15:03:22,695][20347] Updated weights for policy 0, policy_version 1357850 (0.0007) [2023-12-27 15:03:22,758][20347] Updated weights for policy 0, policy_version 1357860 (0.0007) [2023-12-27 15:03:22,819][20347] Updated weights for policy 0, policy_version 1357870 (0.0008) [2023-12-27 15:03:22,876][20347] Updated weights for policy 0, policy_version 1357880 (0.0008) [2023-12-27 15:03:23,299][20381] Updated weights for policy 1, policy_version 1356755 (0.0006) [2023-12-27 15:03:23,353][20381] Updated weights for policy 1, policy_version 1356765 (0.0008) [2023-12-27 15:03:23,413][20381] Updated weights for policy 1, policy_version 1356775 (0.0008) [2023-12-27 15:03:23,555][20347] Updated weights for policy 0, policy_version 1357890 (0.0008) [2023-12-27 15:03:23,613][20347] Updated weights for policy 0, policy_version 1357900 (0.0007) [2023-12-27 15:03:23,670][20347] Updated weights for policy 0, policy_version 1357910 (0.0005) [2023-12-27 15:03:24,058][20381] Updated weights for policy 1, policy_version 1356785 (0.0008) [2023-12-27 15:03:24,117][20381] Updated weights for policy 1, policy_version 1356795 (0.0007) [2023-12-27 15:03:24,181][20381] Updated weights for policy 1, policy_version 1356805 (0.0008) [2023-12-27 15:03:24,247][20381] Updated weights for policy 1, policy_version 1356815 (0.0009) [2023-12-27 15:03:24,318][20347] Updated weights for policy 0, policy_version 1357920 (0.0005) [2023-12-27 15:03:24,378][20347] Updated weights for policy 0, policy_version 1357930 (0.0005) [2023-12-27 15:03:24,429][20347] Updated weights for policy 0, policy_version 1357940 (0.0005) [2023-12-27 15:03:24,886][20381] Updated weights for policy 1, policy_version 1356825 (0.0008) [2023-12-27 15:03:24,940][20381] Updated weights for policy 1, policy_version 1356835 (0.0007) [2023-12-27 15:03:24,984][20347] Updated weights for policy 0, policy_version 1357950 (0.0008) [2023-12-27 15:03:24,999][20381] Updated weights for policy 1, policy_version 1356845 (0.0008) [2023-12-27 15:03:25,028][20347] Updated weights for policy 0, policy_version 1357960 (0.0009) [2023-12-27 15:03:25,075][20347] Updated weights for policy 0, policy_version 1357970 (0.0007) [2023-12-27 15:03:25,681][20347] Updated weights for policy 0, policy_version 1357980 (0.0008) [2023-12-27 15:03:25,722][20381] Updated weights for policy 1, policy_version 1356855 (0.0009) [2023-12-27 15:03:25,731][20347] Updated weights for policy 0, policy_version 1357990 (0.0005) [2023-12-27 15:03:25,775][20381] Updated weights for policy 1, policy_version 1356865 (0.0007) [2023-12-27 15:03:25,786][20347] Updated weights for policy 0, policy_version 1358000 (0.0007) [2023-12-27 15:03:25,831][20381] Updated weights for policy 1, policy_version 1356875 (0.0006) [2023-12-27 15:03:26,388][20347] Updated weights for policy 0, policy_version 1358010 (0.0007) [2023-12-27 15:03:26,450][20347] Updated weights for policy 0, policy_version 1358020 (0.0005) [2023-12-27 15:03:26,500][20347] Updated weights for policy 0, policy_version 1358030 (0.0007) [2023-12-27 15:03:26,552][20347] Updated weights for policy 0, policy_version 1358040 (0.0006) [2023-12-27 15:03:26,584][20381] Updated weights for policy 1, policy_version 1356885 (0.0007) [2023-12-27 15:03:26,640][20381] Updated weights for policy 1, policy_version 1356895 (0.0007) [2023-12-27 15:03:26,693][20381] Updated weights for policy 1, policy_version 1356905 (0.0006) [2023-12-27 15:03:27,225][20347] Updated weights for policy 0, policy_version 1358050 (0.0007) [2023-12-27 15:03:27,279][20347] Updated weights for policy 0, policy_version 1358060 (0.0009) [2023-12-27 15:03:27,325][20381] Updated weights for policy 1, policy_version 1356915 (0.0007) [2023-12-27 15:03:27,332][20347] Updated weights for policy 0, policy_version 1358070 (0.0007) [2023-12-27 15:03:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 695132160. Throughput: 0: 10263.5, 1: 10172.6. Samples: 695120000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:27,363][19170] Avg episode reward: [(0, '863.510'), (1, '892.490')] [2023-12-27 15:03:27,372][20292] KL-divergence is very high: 110.3957 [2023-12-27 15:03:27,381][20381] Updated weights for policy 1, policy_version 1356925 (0.0007) [2023-12-27 15:03:27,433][20381] Updated weights for policy 1, policy_version 1356935 (0.0008) [2023-12-27 15:03:27,963][20347] Updated weights for policy 0, policy_version 1358080 (0.0008) [2023-12-27 15:03:28,020][20347] Updated weights for policy 0, policy_version 1358090 (0.0007) [2023-12-27 15:03:28,051][20381] Updated weights for policy 1, policy_version 1356945 (0.0007) [2023-12-27 15:03:28,067][20347] Updated weights for policy 0, policy_version 1358100 (0.0006) [2023-12-27 15:03:28,103][20381] Updated weights for policy 1, policy_version 1356955 (0.0006) [2023-12-27 15:03:28,155][20381] Updated weights for policy 1, policy_version 1356965 (0.0007) [2023-12-27 15:03:28,211][20381] Updated weights for policy 1, policy_version 1356975 (0.0006) [2023-12-27 15:03:28,813][20381] Updated weights for policy 1, policy_version 1356985 (0.0007) [2023-12-27 15:03:28,830][20347] Updated weights for policy 0, policy_version 1358110 (0.0005) [2023-12-27 15:03:28,873][20381] Updated weights for policy 1, policy_version 1356995 (0.0006) [2023-12-27 15:03:28,879][20347] Updated weights for policy 0, policy_version 1358120 (0.0007) [2023-12-27 15:03:28,929][20347] Updated weights for policy 0, policy_version 1358130 (0.0007) [2023-12-27 15:03:28,930][20381] Updated weights for policy 1, policy_version 1357005 (0.0005) [2023-12-27 15:03:29,510][20381] Updated weights for policy 1, policy_version 1357015 (0.0005) [2023-12-27 15:03:29,573][20381] Updated weights for policy 1, policy_version 1357025 (0.0005) [2023-12-27 15:03:29,632][20381] Updated weights for policy 1, policy_version 1357035 (0.0008) [2023-12-27 15:03:29,709][20347] Updated weights for policy 0, policy_version 1358140 (0.0007) [2023-12-27 15:03:29,766][20347] Updated weights for policy 0, policy_version 1358150 (0.0007) [2023-12-27 15:03:29,828][20347] Updated weights for policy 0, policy_version 1358160 (0.0008) [2023-12-27 15:03:30,269][20381] Updated weights for policy 1, policy_version 1357045 (0.0007) [2023-12-27 15:03:30,330][20381] Updated weights for policy 1, policy_version 1357055 (0.0006) [2023-12-27 15:03:30,390][20381] Updated weights for policy 1, policy_version 1357065 (0.0007) [2023-12-27 15:03:30,518][20347] Updated weights for policy 0, policy_version 1358170 (0.0007) [2023-12-27 15:03:30,568][20347] Updated weights for policy 0, policy_version 1358180 (0.0005) [2023-12-27 15:03:30,614][20347] Updated weights for policy 0, policy_version 1358190 (0.0005) [2023-12-27 15:03:30,666][20347] Updated weights for policy 0, policy_version 1358200 (0.0006) [2023-12-27 15:03:31,072][20381] Updated weights for policy 1, policy_version 1357075 (0.0006) [2023-12-27 15:03:31,143][20381] Updated weights for policy 1, policy_version 1357085 (0.0007) [2023-12-27 15:03:31,208][20381] Updated weights for policy 1, policy_version 1357095 (0.0009) [2023-12-27 15:03:31,273][20347] Updated weights for policy 0, policy_version 1358210 (0.0008) [2023-12-27 15:03:31,327][20347] Updated weights for policy 0, policy_version 1358220 (0.0008) [2023-12-27 15:03:31,395][20347] Updated weights for policy 0, policy_version 1358230 (0.0008) [2023-12-27 15:03:31,926][20381] Updated weights for policy 1, policy_version 1357105 (0.0007) [2023-12-27 15:03:31,989][20381] Updated weights for policy 1, policy_version 1357115 (0.0008) [2023-12-27 15:03:32,042][20381] Updated weights for policy 1, policy_version 1357125 (0.0008) [2023-12-27 15:03:32,080][20347] Updated weights for policy 0, policy_version 1358240 (0.0009) [2023-12-27 15:03:32,094][20381] Updated weights for policy 1, policy_version 1357135 (0.0008) [2023-12-27 15:03:32,136][20347] Updated weights for policy 0, policy_version 1358250 (0.0007) [2023-12-27 15:03:32,191][20347] Updated weights for policy 0, policy_version 1358260 (0.0008) [2023-12-27 15:03:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 695238656. Throughput: 0: 10235.2, 1: 10199.9. Samples: 695244640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:32,363][19170] Avg episode reward: [(0, '853.530'), (1, '863.200')] [2023-12-27 15:03:32,823][20381] Updated weights for policy 1, policy_version 1357145 (0.0007) [2023-12-27 15:03:32,876][20381] Updated weights for policy 1, policy_version 1357155 (0.0007) [2023-12-27 15:03:32,902][20347] Updated weights for policy 0, policy_version 1358270 (0.0009) [2023-12-27 15:03:32,929][20381] Updated weights for policy 1, policy_version 1357165 (0.0005) [2023-12-27 15:03:32,959][20347] Updated weights for policy 0, policy_version 1358280 (0.0010) [2023-12-27 15:03:33,014][20347] Updated weights for policy 0, policy_version 1358290 (0.0006) [2023-12-27 15:03:33,621][20347] Updated weights for policy 0, policy_version 1358300 (0.0007) [2023-12-27 15:03:33,635][20381] Updated weights for policy 1, policy_version 1357175 (0.0008) [2023-12-27 15:03:33,679][20347] Updated weights for policy 0, policy_version 1358310 (0.0010) [2023-12-27 15:03:33,693][20381] Updated weights for policy 1, policy_version 1357185 (0.0008) [2023-12-27 15:03:33,733][20347] Updated weights for policy 0, policy_version 1358320 (0.0010) [2023-12-27 15:03:33,745][20381] Updated weights for policy 1, policy_version 1357195 (0.0009) [2023-12-27 15:03:34,353][20347] Updated weights for policy 0, policy_version 1358330 (0.0010) [2023-12-27 15:03:34,412][20381] Updated weights for policy 1, policy_version 1357205 (0.0006) [2023-12-27 15:03:34,417][20347] Updated weights for policy 0, policy_version 1358340 (0.0009) [2023-12-27 15:03:34,471][20347] Updated weights for policy 0, policy_version 1358350 (0.0009) [2023-12-27 15:03:34,477][20381] Updated weights for policy 1, policy_version 1357215 (0.0008) [2023-12-27 15:03:34,526][20347] Updated weights for policy 0, policy_version 1358360 (0.0010) [2023-12-27 15:03:34,543][20381] Updated weights for policy 1, policy_version 1357225 (0.0009) [2023-12-27 15:03:35,167][20347] Updated weights for policy 0, policy_version 1358370 (0.0005) [2023-12-27 15:03:35,211][20347] Updated weights for policy 0, policy_version 1358380 (0.0005) [2023-12-27 15:03:35,253][20381] Updated weights for policy 1, policy_version 1357235 (0.0008) [2023-12-27 15:03:35,266][20347] Updated weights for policy 0, policy_version 1358390 (0.0005) [2023-12-27 15:03:35,308][20381] Updated weights for policy 1, policy_version 1357245 (0.0007) [2023-12-27 15:03:35,358][20381] Updated weights for policy 1, policy_version 1357255 (0.0008) [2023-12-27 15:03:35,918][20347] Updated weights for policy 0, policy_version 1358400 (0.0009) [2023-12-27 15:03:35,972][20347] Updated weights for policy 0, policy_version 1358410 (0.0010) [2023-12-27 15:03:36,017][20347] Updated weights for policy 0, policy_version 1358420 (0.0010) [2023-12-27 15:03:36,026][20381] Updated weights for policy 1, policy_version 1357265 (0.0007) [2023-12-27 15:03:36,083][20381] Updated weights for policy 1, policy_version 1357275 (0.0007) [2023-12-27 15:03:36,148][20381] Updated weights for policy 1, policy_version 1357285 (0.0007) [2023-12-27 15:03:36,205][20381] Updated weights for policy 1, policy_version 1357295 (0.0008) [2023-12-27 15:03:36,765][20347] Updated weights for policy 0, policy_version 1358430 (0.0009) [2023-12-27 15:03:36,824][20347] Updated weights for policy 0, policy_version 1358440 (0.0007) [2023-12-27 15:03:36,881][20347] Updated weights for policy 0, policy_version 1358450 (0.0006) [2023-12-27 15:03:36,882][20381] Updated weights for policy 1, policy_version 1357305 (0.0006) [2023-12-27 15:03:36,940][20381] Updated weights for policy 1, policy_version 1357315 (0.0006) [2023-12-27 15:03:36,999][20381] Updated weights for policy 1, policy_version 1357325 (0.0006) [2023-12-27 15:03:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 695336960. Throughput: 0: 10240.1, 1: 10172.5. Samples: 695305924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:37,363][19170] Avg episode reward: [(0, '892.600'), (1, '882.770')] [2023-12-27 15:03:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001357328_347504640.pth... [2023-12-27 15:03:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001358456_347832320.pth... [2023-12-27 15:03:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001357272_347529216.pth [2023-12-27 15:03:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001356144_347201536.pth [2023-12-27 15:03:37,564][20347] Updated weights for policy 0, policy_version 1358460 (0.0006) [2023-12-27 15:03:37,601][20381] Updated weights for policy 1, policy_version 1357335 (0.0006) [2023-12-27 15:03:37,614][20347] Updated weights for policy 0, policy_version 1358470 (0.0006) [2023-12-27 15:03:37,651][20381] Updated weights for policy 1, policy_version 1357345 (0.0006) [2023-12-27 15:03:37,664][20347] Updated weights for policy 0, policy_version 1358480 (0.0006) [2023-12-27 15:03:37,697][20381] Updated weights for policy 1, policy_version 1357355 (0.0006) [2023-12-27 15:03:38,347][20347] Updated weights for policy 0, policy_version 1358490 (0.0006) [2023-12-27 15:03:38,398][20347] Updated weights for policy 0, policy_version 1358500 (0.0008) [2023-12-27 15:03:38,440][20381] Updated weights for policy 1, policy_version 1357365 (0.0007) [2023-12-27 15:03:38,454][20347] Updated weights for policy 0, policy_version 1358510 (0.0007) [2023-12-27 15:03:38,495][20381] Updated weights for policy 1, policy_version 1357375 (0.0006) [2023-12-27 15:03:38,519][20347] Updated weights for policy 0, policy_version 1358520 (0.0007) [2023-12-27 15:03:38,545][20381] Updated weights for policy 1, policy_version 1357385 (0.0007) [2023-12-27 15:03:39,219][20381] Updated weights for policy 1, policy_version 1357395 (0.0008) [2023-12-27 15:03:39,277][20381] Updated weights for policy 1, policy_version 1357405 (0.0007) [2023-12-27 15:03:39,289][20347] Updated weights for policy 0, policy_version 1358530 (0.0007) [2023-12-27 15:03:39,339][20381] Updated weights for policy 1, policy_version 1357415 (0.0007) [2023-12-27 15:03:39,350][20347] Updated weights for policy 0, policy_version 1358540 (0.0007) [2023-12-27 15:03:39,413][20347] Updated weights for policy 0, policy_version 1358550 (0.0008) [2023-12-27 15:03:40,031][20381] Updated weights for policy 1, policy_version 1357425 (0.0009) [2023-12-27 15:03:40,091][20381] Updated weights for policy 1, policy_version 1357435 (0.0007) [2023-12-27 15:03:40,153][20381] Updated weights for policy 1, policy_version 1357445 (0.0008) [2023-12-27 15:03:40,171][20347] Updated weights for policy 0, policy_version 1358560 (0.0006) [2023-12-27 15:03:40,211][20381] Updated weights for policy 1, policy_version 1357455 (0.0006) [2023-12-27 15:03:40,226][20347] Updated weights for policy 0, policy_version 1358570 (0.0006) [2023-12-27 15:03:40,280][20347] Updated weights for policy 0, policy_version 1358580 (0.0008) [2023-12-27 15:03:40,862][20381] Updated weights for policy 1, policy_version 1357465 (0.0006) [2023-12-27 15:03:40,910][20381] Updated weights for policy 1, policy_version 1357475 (0.0005) [2023-12-27 15:03:40,968][20381] Updated weights for policy 1, policy_version 1357485 (0.0007) [2023-12-27 15:03:41,010][20347] Updated weights for policy 0, policy_version 1358590 (0.0007) [2023-12-27 15:03:41,072][20347] Updated weights for policy 0, policy_version 1358600 (0.0009) [2023-12-27 15:03:41,134][20347] Updated weights for policy 0, policy_version 1358610 (0.0011) [2023-12-27 15:03:41,677][20381] Updated weights for policy 1, policy_version 1357495 (0.0008) [2023-12-27 15:03:41,728][20381] Updated weights for policy 1, policy_version 1357505 (0.0007) [2023-12-27 15:03:41,801][20381] Updated weights for policy 1, policy_version 1357515 (0.0009) [2023-12-27 15:03:41,974][20347] Updated weights for policy 0, policy_version 1358620 (0.0008) [2023-12-27 15:03:42,026][20347] Updated weights for policy 0, policy_version 1358630 (0.0008) [2023-12-27 15:03:42,077][20347] Updated weights for policy 0, policy_version 1358640 (0.0007) [2023-12-27 15:03:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 695435264. Throughput: 0: 10152.1, 1: 10198.5. Samples: 695426144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:42,363][19170] Avg episode reward: [(0, '961.130'), (1, '951.100')] [2023-12-27 15:03:42,525][20381] Updated weights for policy 1, policy_version 1357525 (0.0009) [2023-12-27 15:03:42,587][20381] Updated weights for policy 1, policy_version 1357535 (0.0011) [2023-12-27 15:03:42,649][20381] Updated weights for policy 1, policy_version 1357545 (0.0011) [2023-12-27 15:03:42,836][20347] Updated weights for policy 0, policy_version 1358650 (0.0007) [2023-12-27 15:03:42,899][20347] Updated weights for policy 0, policy_version 1358660 (0.0006) [2023-12-27 15:03:42,955][20347] Updated weights for policy 0, policy_version 1358670 (0.0006) [2023-12-27 15:03:43,000][20347] Updated weights for policy 0, policy_version 1358680 (0.0008) [2023-12-27 15:03:43,296][20381] Updated weights for policy 1, policy_version 1357555 (0.0010) [2023-12-27 15:03:43,350][20381] Updated weights for policy 1, policy_version 1357565 (0.0008) [2023-12-27 15:03:43,401][20381] Updated weights for policy 1, policy_version 1357575 (0.0008) [2023-12-27 15:03:43,606][20347] Updated weights for policy 0, policy_version 1358690 (0.0009) [2023-12-27 15:03:43,654][20347] Updated weights for policy 0, policy_version 1358700 (0.0006) [2023-12-27 15:03:43,702][20347] Updated weights for policy 0, policy_version 1358710 (0.0007) [2023-12-27 15:03:44,004][20381] Updated weights for policy 1, policy_version 1357585 (0.0005) [2023-12-27 15:03:44,060][20381] Updated weights for policy 1, policy_version 1357595 (0.0005) [2023-12-27 15:03:44,114][20381] Updated weights for policy 1, policy_version 1357605 (0.0009) [2023-12-27 15:03:44,165][20381] Updated weights for policy 1, policy_version 1357615 (0.0010) [2023-12-27 15:03:44,458][20347] Updated weights for policy 0, policy_version 1358720 (0.0007) [2023-12-27 15:03:44,511][20347] Updated weights for policy 0, policy_version 1358730 (0.0008) [2023-12-27 15:03:44,564][20347] Updated weights for policy 0, policy_version 1358740 (0.0009) [2023-12-27 15:03:44,769][20381] Updated weights for policy 1, policy_version 1357625 (0.0007) [2023-12-27 15:03:44,834][20381] Updated weights for policy 1, policy_version 1357635 (0.0008) [2023-12-27 15:03:44,894][20381] Updated weights for policy 1, policy_version 1357645 (0.0007) [2023-12-27 15:03:45,316][20347] Updated weights for policy 0, policy_version 1358750 (0.0007) [2023-12-27 15:03:45,364][20347] Updated weights for policy 0, policy_version 1358760 (0.0008) [2023-12-27 15:03:45,417][20347] Updated weights for policy 0, policy_version 1358770 (0.0007) [2023-12-27 15:03:45,529][20381] Updated weights for policy 1, policy_version 1357655 (0.0005) [2023-12-27 15:03:45,585][20381] Updated weights for policy 1, policy_version 1357665 (0.0006) [2023-12-27 15:03:45,644][20381] Updated weights for policy 1, policy_version 1357675 (0.0007) [2023-12-27 15:03:46,163][20347] Updated weights for policy 0, policy_version 1358780 (0.0009) [2023-12-27 15:03:46,210][20381] Updated weights for policy 1, policy_version 1357685 (0.0010) [2023-12-27 15:03:46,210][20347] Updated weights for policy 0, policy_version 1358790 (0.0008) [2023-12-27 15:03:46,256][20347] Updated weights for policy 0, policy_version 1358800 (0.0008) [2023-12-27 15:03:46,265][20381] Updated weights for policy 1, policy_version 1357695 (0.0010) [2023-12-27 15:03:46,325][20381] Updated weights for policy 1, policy_version 1357705 (0.0010) [2023-12-27 15:03:46,916][20381] Updated weights for policy 1, policy_version 1357715 (0.0007) [2023-12-27 15:03:46,939][20347] Updated weights for policy 0, policy_version 1358810 (0.0005) [2023-12-27 15:03:46,971][20381] Updated weights for policy 1, policy_version 1357725 (0.0007) [2023-12-27 15:03:46,998][20347] Updated weights for policy 0, policy_version 1358820 (0.0006) [2023-12-27 15:03:47,034][20381] Updated weights for policy 1, policy_version 1357735 (0.0006) [2023-12-27 15:03:47,059][20347] Updated weights for policy 0, policy_version 1358830 (0.0007) [2023-12-27 15:03:47,119][20347] Updated weights for policy 0, policy_version 1358840 (0.0009) [2023-12-27 15:03:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 695541760. Throughput: 0: 10126.4, 1: 10249.2. Samples: 695549256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:47,363][19170] Avg episode reward: [(0, '980.660'), (1, '951.060')] [2023-12-27 15:03:47,681][20381] Updated weights for policy 1, policy_version 1357745 (0.0005) [2023-12-27 15:03:47,731][20381] Updated weights for policy 1, policy_version 1357755 (0.0005) [2023-12-27 15:03:47,783][20381] Updated weights for policy 1, policy_version 1357765 (0.0005) [2023-12-27 15:03:47,790][20347] Updated weights for policy 0, policy_version 1358850 (0.0008) [2023-12-27 15:03:47,841][20381] Updated weights for policy 1, policy_version 1357775 (0.0005) [2023-12-27 15:03:47,845][20347] Updated weights for policy 0, policy_version 1358860 (0.0007) [2023-12-27 15:03:47,897][20347] Updated weights for policy 0, policy_version 1358870 (0.0008) [2023-12-27 15:03:48,399][20381] Updated weights for policy 1, policy_version 1357785 (0.0010) [2023-12-27 15:03:48,456][20381] Updated weights for policy 1, policy_version 1357795 (0.0011) [2023-12-27 15:03:48,515][20381] Updated weights for policy 1, policy_version 1357805 (0.0011) [2023-12-27 15:03:48,638][20347] Updated weights for policy 0, policy_version 1358880 (0.0008) [2023-12-27 15:03:48,686][20347] Updated weights for policy 0, policy_version 1358890 (0.0007) [2023-12-27 15:03:48,746][20347] Updated weights for policy 0, policy_version 1358900 (0.0007) [2023-12-27 15:03:49,205][20381] Updated weights for policy 1, policy_version 1357815 (0.0011) [2023-12-27 15:03:49,269][20381] Updated weights for policy 1, policy_version 1357825 (0.0009) [2023-12-27 15:03:49,325][20381] Updated weights for policy 1, policy_version 1357835 (0.0008) [2023-12-27 15:03:49,512][20347] Updated weights for policy 0, policy_version 1358910 (0.0009) [2023-12-27 15:03:49,561][20347] Updated weights for policy 0, policy_version 1358920 (0.0008) [2023-12-27 15:03:49,615][20347] Updated weights for policy 0, policy_version 1358930 (0.0008) [2023-12-27 15:03:49,998][20381] Updated weights for policy 1, policy_version 1357845 (0.0010) [2023-12-27 15:03:50,047][20381] Updated weights for policy 1, policy_version 1357855 (0.0010) [2023-12-27 15:03:50,104][20381] Updated weights for policy 1, policy_version 1357865 (0.0011) [2023-12-27 15:03:50,377][20347] Updated weights for policy 0, policy_version 1358940 (0.0007) [2023-12-27 15:03:50,445][20347] Updated weights for policy 0, policy_version 1358950 (0.0007) [2023-12-27 15:03:50,512][20347] Updated weights for policy 0, policy_version 1358960 (0.0007) [2023-12-27 15:03:50,793][20381] Updated weights for policy 1, policy_version 1357875 (0.0011) [2023-12-27 15:03:50,852][20381] Updated weights for policy 1, policy_version 1357885 (0.0008) [2023-12-27 15:03:50,908][20381] Updated weights for policy 1, policy_version 1357895 (0.0008) [2023-12-27 15:03:51,239][20347] Updated weights for policy 0, policy_version 1358970 (0.0008) [2023-12-27 15:03:51,302][20347] Updated weights for policy 0, policy_version 1358980 (0.0010) [2023-12-27 15:03:51,359][20347] Updated weights for policy 0, policy_version 1358990 (0.0010) [2023-12-27 15:03:51,420][20347] Updated weights for policy 0, policy_version 1359000 (0.0010) [2023-12-27 15:03:51,612][20381] Updated weights for policy 1, policy_version 1357905 (0.0010) [2023-12-27 15:03:51,675][20381] Updated weights for policy 1, policy_version 1357915 (0.0008) [2023-12-27 15:03:51,739][20381] Updated weights for policy 1, policy_version 1357925 (0.0008) [2023-12-27 15:03:51,793][20381] Updated weights for policy 1, policy_version 1357935 (0.0008) [2023-12-27 15:03:52,215][20347] Updated weights for policy 0, policy_version 1359010 (0.0011) [2023-12-27 15:03:52,277][20347] Updated weights for policy 0, policy_version 1359020 (0.0010) [2023-12-27 15:03:52,338][20347] Updated weights for policy 0, policy_version 1359030 (0.0009) [2023-12-27 15:03:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 695640064. Throughput: 0: 10059.0, 1: 10247.5. Samples: 695609576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.320')] [2023-12-27 15:03:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001357936_347660288.pth... [2023-12-27 15:03:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001359032_347979776.pth... [2023-12-27 15:03:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001357848_347676672.pth [2023-12-27 15:03:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001356720_347348992.pth [2023-12-27 15:03:52,470][20381] Updated weights for policy 1, policy_version 1357945 (0.0008) [2023-12-27 15:03:52,527][20381] Updated weights for policy 1, policy_version 1357955 (0.0008) [2023-12-27 15:03:52,583][20381] Updated weights for policy 1, policy_version 1357965 (0.0008) [2023-12-27 15:03:53,040][20347] Updated weights for policy 0, policy_version 1359040 (0.0007) [2023-12-27 15:03:53,088][20347] Updated weights for policy 0, policy_version 1359050 (0.0010) [2023-12-27 15:03:53,148][20347] Updated weights for policy 0, policy_version 1359060 (0.0005) [2023-12-27 15:03:53,172][20381] Updated weights for policy 1, policy_version 1357975 (0.0006) [2023-12-27 15:03:53,226][20381] Updated weights for policy 1, policy_version 1357985 (0.0005) [2023-12-27 15:03:53,272][20381] Updated weights for policy 1, policy_version 1357995 (0.0005) [2023-12-27 15:03:53,801][20347] Updated weights for policy 0, policy_version 1359070 (0.0006) [2023-12-27 15:03:53,860][20347] Updated weights for policy 0, policy_version 1359080 (0.0005) [2023-12-27 15:03:53,920][20347] Updated weights for policy 0, policy_version 1359090 (0.0007) [2023-12-27 15:03:53,929][20381] Updated weights for policy 1, policy_version 1358005 (0.0005) [2023-12-27 15:03:53,997][20381] Updated weights for policy 1, policy_version 1358015 (0.0008) [2023-12-27 15:03:54,052][20381] Updated weights for policy 1, policy_version 1358025 (0.0006) [2023-12-27 15:03:54,522][20347] Updated weights for policy 0, policy_version 1359100 (0.0006) [2023-12-27 15:03:54,578][20347] Updated weights for policy 0, policy_version 1359110 (0.0008) [2023-12-27 15:03:54,636][20347] Updated weights for policy 0, policy_version 1359120 (0.0006) [2023-12-27 15:03:54,698][20381] Updated weights for policy 1, policy_version 1358035 (0.0007) [2023-12-27 15:03:54,751][20381] Updated weights for policy 1, policy_version 1358045 (0.0006) [2023-12-27 15:03:54,795][20381] Updated weights for policy 1, policy_version 1358055 (0.0007) [2023-12-27 15:03:55,213][20347] Updated weights for policy 0, policy_version 1359130 (0.0006) [2023-12-27 15:03:55,261][20347] Updated weights for policy 0, policy_version 1359140 (0.0006) [2023-12-27 15:03:55,318][20347] Updated weights for policy 0, policy_version 1359150 (0.0005) [2023-12-27 15:03:55,374][20347] Updated weights for policy 0, policy_version 1359160 (0.0005) [2023-12-27 15:03:55,471][20381] Updated weights for policy 1, policy_version 1358065 (0.0006) [2023-12-27 15:03:55,532][20381] Updated weights for policy 1, policy_version 1358075 (0.0007) [2023-12-27 15:03:55,584][20381] Updated weights for policy 1, policy_version 1358085 (0.0007) [2023-12-27 15:03:55,640][20381] Updated weights for policy 1, policy_version 1358095 (0.0008) [2023-12-27 15:03:55,967][20347] Updated weights for policy 0, policy_version 1359170 (0.0010) [2023-12-27 15:03:56,029][20347] Updated weights for policy 0, policy_version 1359180 (0.0010) [2023-12-27 15:03:56,090][20347] Updated weights for policy 0, policy_version 1359190 (0.0010) [2023-12-27 15:03:56,340][20381] Updated weights for policy 1, policy_version 1358105 (0.0006) [2023-12-27 15:03:56,399][20381] Updated weights for policy 1, policy_version 1358115 (0.0006) [2023-12-27 15:03:56,456][20381] Updated weights for policy 1, policy_version 1358125 (0.0007) [2023-12-27 15:03:56,720][20347] Updated weights for policy 0, policy_version 1359200 (0.0006) [2023-12-27 15:03:56,768][20347] Updated weights for policy 0, policy_version 1359210 (0.0007) [2023-12-27 15:03:56,820][20347] Updated weights for policy 0, policy_version 1359220 (0.0007) [2023-12-27 15:03:57,001][20381] Updated weights for policy 1, policy_version 1358135 (0.0006) [2023-12-27 15:03:57,060][20381] Updated weights for policy 1, policy_version 1358145 (0.0008) [2023-12-27 15:03:57,125][20381] Updated weights for policy 1, policy_version 1358155 (0.0008) [2023-12-27 15:03:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 695746560. Throughput: 0: 10048.1, 1: 10294.8. Samples: 695734472. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:03:57,363][19170] Avg episode reward: [(0, '990.210'), (1, '941.370')] [2023-12-27 15:03:57,460][20347] Updated weights for policy 0, policy_version 1359230 (0.0007) [2023-12-27 15:03:57,505][20347] Updated weights for policy 0, policy_version 1359240 (0.0007) [2023-12-27 15:03:57,553][20347] Updated weights for policy 0, policy_version 1359250 (0.0010) [2023-12-27 15:03:57,738][20381] Updated weights for policy 1, policy_version 1358165 (0.0005) [2023-12-27 15:03:57,789][20381] Updated weights for policy 1, policy_version 1358175 (0.0005) [2023-12-27 15:03:57,846][20381] Updated weights for policy 1, policy_version 1358185 (0.0010) [2023-12-27 15:03:58,227][20347] Updated weights for policy 0, policy_version 1359260 (0.0010) [2023-12-27 15:03:58,285][20347] Updated weights for policy 0, policy_version 1359270 (0.0010) [2023-12-27 15:03:58,353][20347] Updated weights for policy 0, policy_version 1359280 (0.0010) [2023-12-27 15:03:58,508][20381] Updated weights for policy 1, policy_version 1358195 (0.0010) [2023-12-27 15:03:58,567][20381] Updated weights for policy 1, policy_version 1358205 (0.0008) [2023-12-27 15:03:58,626][20381] Updated weights for policy 1, policy_version 1358215 (0.0007) [2023-12-27 15:03:59,120][20347] Updated weights for policy 0, policy_version 1359290 (0.0009) [2023-12-27 15:03:59,177][20347] Updated weights for policy 0, policy_version 1359300 (0.0008) [2023-12-27 15:03:59,238][20347] Updated weights for policy 0, policy_version 1359310 (0.0008) [2023-12-27 15:03:59,303][20347] Updated weights for policy 0, policy_version 1359320 (0.0008) [2023-12-27 15:03:59,412][20381] Updated weights for policy 1, policy_version 1358225 (0.0007) [2023-12-27 15:03:59,476][20381] Updated weights for policy 1, policy_version 1358235 (0.0005) [2023-12-27 15:03:59,528][20381] Updated weights for policy 1, policy_version 1358245 (0.0005) [2023-12-27 15:03:59,580][20381] Updated weights for policy 1, policy_version 1358255 (0.0007) [2023-12-27 15:04:00,054][20347] Updated weights for policy 0, policy_version 1359330 (0.0008) [2023-12-27 15:04:00,106][20347] Updated weights for policy 0, policy_version 1359340 (0.0007) [2023-12-27 15:04:00,163][20347] Updated weights for policy 0, policy_version 1359350 (0.0007) [2023-12-27 15:04:00,206][20381] Updated weights for policy 1, policy_version 1358265 (0.0008) [2023-12-27 15:04:00,266][20381] Updated weights for policy 1, policy_version 1358275 (0.0008) [2023-12-27 15:04:00,321][20381] Updated weights for policy 1, policy_version 1358285 (0.0007) [2023-12-27 15:04:00,819][20347] Updated weights for policy 0, policy_version 1359360 (0.0007) [2023-12-27 15:04:00,875][20347] Updated weights for policy 0, policy_version 1359370 (0.0006) [2023-12-27 15:04:00,930][20347] Updated weights for policy 0, policy_version 1359380 (0.0007) [2023-12-27 15:04:00,950][20381] Updated weights for policy 1, policy_version 1358295 (0.0006) [2023-12-27 15:04:00,995][20381] Updated weights for policy 1, policy_version 1358305 (0.0008) [2023-12-27 15:04:01,054][20381] Updated weights for policy 1, policy_version 1358315 (0.0007) [2023-12-27 15:04:01,620][20347] Updated weights for policy 0, policy_version 1359390 (0.0008) [2023-12-27 15:04:01,687][20347] Updated weights for policy 0, policy_version 1359400 (0.0010) [2023-12-27 15:04:01,751][20347] Updated weights for policy 0, policy_version 1359410 (0.0008) [2023-12-27 15:04:01,794][20381] Updated weights for policy 1, policy_version 1358325 (0.0007) [2023-12-27 15:04:01,851][20381] Updated weights for policy 1, policy_version 1358335 (0.0007) [2023-12-27 15:04:01,913][20381] Updated weights for policy 1, policy_version 1358345 (0.0009) [2023-12-27 15:04:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 695844864. Throughput: 0: 10028.2, 1: 10364.6. Samples: 695857144. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.690')] [2023-12-27 15:04:02,454][20347] Updated weights for policy 0, policy_version 1359420 (0.0006) [2023-12-27 15:04:02,507][20347] Updated weights for policy 0, policy_version 1359430 (0.0007) [2023-12-27 15:04:02,567][20347] Updated weights for policy 0, policy_version 1359440 (0.0006) [2023-12-27 15:04:02,604][20381] Updated weights for policy 1, policy_version 1358355 (0.0009) [2023-12-27 15:04:02,662][20381] Updated weights for policy 1, policy_version 1358365 (0.0011) [2023-12-27 15:04:02,708][20381] Updated weights for policy 1, policy_version 1358375 (0.0010) [2023-12-27 15:04:03,231][20347] Updated weights for policy 0, policy_version 1359450 (0.0006) [2023-12-27 15:04:03,277][20347] Updated weights for policy 0, policy_version 1359460 (0.0005) [2023-12-27 15:04:03,306][20381] Updated weights for policy 1, policy_version 1358385 (0.0009) [2023-12-27 15:04:03,327][20347] Updated weights for policy 0, policy_version 1359470 (0.0006) [2023-12-27 15:04:03,362][20381] Updated weights for policy 1, policy_version 1358395 (0.0005) [2023-12-27 15:04:03,372][20347] Updated weights for policy 0, policy_version 1359480 (0.0007) [2023-12-27 15:04:03,409][20381] Updated weights for policy 1, policy_version 1358405 (0.0006) [2023-12-27 15:04:03,461][20381] Updated weights for policy 1, policy_version 1358415 (0.0005) [2023-12-27 15:04:04,000][20381] Updated weights for policy 1, policy_version 1358425 (0.0006) [2023-12-27 15:04:04,021][20347] Updated weights for policy 0, policy_version 1359490 (0.0007) [2023-12-27 15:04:04,056][20381] Updated weights for policy 1, policy_version 1358435 (0.0005) [2023-12-27 15:04:04,077][20347] Updated weights for policy 0, policy_version 1359500 (0.0007) [2023-12-27 15:04:04,115][20381] Updated weights for policy 1, policy_version 1358445 (0.0006) [2023-12-27 15:04:04,138][20347] Updated weights for policy 0, policy_version 1359510 (0.0007) [2023-12-27 15:04:04,723][20381] Updated weights for policy 1, policy_version 1358455 (0.0007) [2023-12-27 15:04:04,784][20381] Updated weights for policy 1, policy_version 1358465 (0.0008) [2023-12-27 15:04:04,841][20381] Updated weights for policy 1, policy_version 1358475 (0.0007) [2023-12-27 15:04:04,901][20347] Updated weights for policy 0, policy_version 1359520 (0.0007) [2023-12-27 15:04:04,957][20347] Updated weights for policy 0, policy_version 1359530 (0.0006) [2023-12-27 15:04:05,015][20347] Updated weights for policy 0, policy_version 1359540 (0.0006) [2023-12-27 15:04:05,527][20381] Updated weights for policy 1, policy_version 1358485 (0.0008) [2023-12-27 15:04:05,554][20347] Updated weights for policy 0, policy_version 1359550 (0.0006) [2023-12-27 15:04:05,589][20381] Updated weights for policy 1, policy_version 1358495 (0.0005) [2023-12-27 15:04:05,613][20347] Updated weights for policy 0, policy_version 1359560 (0.0006) [2023-12-27 15:04:05,649][20381] Updated weights for policy 1, policy_version 1358505 (0.0006) [2023-12-27 15:04:05,674][20347] Updated weights for policy 0, policy_version 1359570 (0.0006) [2023-12-27 15:04:06,238][20381] Updated weights for policy 1, policy_version 1358515 (0.0006) [2023-12-27 15:04:06,305][20381] Updated weights for policy 1, policy_version 1358525 (0.0007) [2023-12-27 15:04:06,368][20347] Updated weights for policy 0, policy_version 1359580 (0.0008) [2023-12-27 15:04:06,369][20381] Updated weights for policy 1, policy_version 1358535 (0.0009) [2023-12-27 15:04:06,426][20347] Updated weights for policy 0, policy_version 1359590 (0.0006) [2023-12-27 15:04:06,490][20347] Updated weights for policy 0, policy_version 1359600 (0.0007) [2023-12-27 15:04:07,043][20381] Updated weights for policy 1, policy_version 1358545 (0.0008) [2023-12-27 15:04:07,101][20381] Updated weights for policy 1, policy_version 1358555 (0.0010) [2023-12-27 15:04:07,161][20381] Updated weights for policy 1, policy_version 1358565 (0.0010) [2023-12-27 15:04:07,168][20347] Updated weights for policy 0, policy_version 1359610 (0.0008) [2023-12-27 15:04:07,216][20381] Updated weights for policy 1, policy_version 1358575 (0.0011) [2023-12-27 15:04:07,224][20347] Updated weights for policy 0, policy_version 1359620 (0.0007) [2023-12-27 15:04:07,269][20347] Updated weights for policy 0, policy_version 1359630 (0.0007) [2023-12-27 15:04:07,316][20347] Updated weights for policy 0, policy_version 1359640 (0.0007) [2023-12-27 15:04:07,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 695959552. Throughput: 0: 10058.3, 1: 10440.0. Samples: 695920484. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.200')] [2023-12-27 15:04:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001358576_347824128.pth... [2023-12-27 15:04:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001359640_348135424.pth... [2023-12-27 15:04:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001358456_347832320.pth [2023-12-27 15:04:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001357328_347504640.pth [2023-12-27 15:04:07,904][20381] Updated weights for policy 1, policy_version 1358585 (0.0010) [2023-12-27 15:04:07,954][20381] Updated weights for policy 1, policy_version 1358595 (0.0011) [2023-12-27 15:04:08,012][20381] Updated weights for policy 1, policy_version 1358605 (0.0010) [2023-12-27 15:04:08,021][20347] Updated weights for policy 0, policy_version 1359650 (0.0005) [2023-12-27 15:04:08,070][20347] Updated weights for policy 0, policy_version 1359660 (0.0005) [2023-12-27 15:04:08,119][20347] Updated weights for policy 0, policy_version 1359670 (0.0005) [2023-12-27 15:04:08,675][20381] Updated weights for policy 1, policy_version 1358615 (0.0011) [2023-12-27 15:04:08,691][20347] Updated weights for policy 0, policy_version 1359680 (0.0007) [2023-12-27 15:04:08,742][20347] Updated weights for policy 0, policy_version 1359690 (0.0008) [2023-12-27 15:04:08,744][20381] Updated weights for policy 1, policy_version 1358625 (0.0011) [2023-12-27 15:04:08,798][20347] Updated weights for policy 0, policy_version 1359700 (0.0007) [2023-12-27 15:04:08,807][20381] Updated weights for policy 1, policy_version 1358635 (0.0011) [2023-12-27 15:04:09,522][20347] Updated weights for policy 0, policy_version 1359710 (0.0007) [2023-12-27 15:04:09,578][20347] Updated weights for policy 0, policy_version 1359720 (0.0009) [2023-12-27 15:04:09,592][20381] Updated weights for policy 1, policy_version 1358645 (0.0009) [2023-12-27 15:04:09,638][20347] Updated weights for policy 0, policy_version 1359730 (0.0009) [2023-12-27 15:04:09,652][20381] Updated weights for policy 1, policy_version 1358655 (0.0007) [2023-12-27 15:04:09,712][20381] Updated weights for policy 1, policy_version 1358665 (0.0008) [2023-12-27 15:04:10,349][20347] Updated weights for policy 0, policy_version 1359740 (0.0008) [2023-12-27 15:04:10,408][20347] Updated weights for policy 0, policy_version 1359750 (0.0008) [2023-12-27 15:04:10,418][20381] Updated weights for policy 1, policy_version 1358675 (0.0007) [2023-12-27 15:04:10,472][20347] Updated weights for policy 0, policy_version 1359760 (0.0008) [2023-12-27 15:04:10,476][20381] Updated weights for policy 1, policy_version 1358685 (0.0005) [2023-12-27 15:04:10,526][20381] Updated weights for policy 1, policy_version 1358695 (0.0006) [2023-12-27 15:04:11,114][20347] Updated weights for policy 0, policy_version 1359770 (0.0006) [2023-12-27 15:04:11,182][20347] Updated weights for policy 0, policy_version 1359780 (0.0008) [2023-12-27 15:04:11,203][20381] Updated weights for policy 1, policy_version 1358705 (0.0009) [2023-12-27 15:04:11,246][20347] Updated weights for policy 0, policy_version 1359790 (0.0007) [2023-12-27 15:04:11,267][20381] Updated weights for policy 1, policy_version 1358715 (0.0010) [2023-12-27 15:04:11,309][20347] Updated weights for policy 0, policy_version 1359800 (0.0006) [2023-12-27 15:04:11,315][20292] KL-divergence is very high: 257.1861 [2023-12-27 15:04:11,320][20381] Updated weights for policy 1, policy_version 1358725 (0.0010) [2023-12-27 15:04:11,371][20292] KL-divergence is very high: 148.1601 [2023-12-27 15:04:11,393][20381] Updated weights for policy 1, policy_version 1358735 (0.0008) [2023-12-27 15:04:12,045][20347] Updated weights for policy 0, policy_version 1359810 (0.0010) [2023-12-27 15:04:12,092][20381] Updated weights for policy 1, policy_version 1358745 (0.0006) [2023-12-27 15:04:12,101][20347] Updated weights for policy 0, policy_version 1359820 (0.0010) [2023-12-27 15:04:12,148][20381] Updated weights for policy 1, policy_version 1358755 (0.0005) [2023-12-27 15:04:12,157][20347] Updated weights for policy 0, policy_version 1359830 (0.0010) [2023-12-27 15:04:12,201][20381] Updated weights for policy 1, policy_version 1358765 (0.0006) [2023-12-27 15:04:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 696057856. Throughput: 0: 10074.0, 1: 10463.6. Samples: 696044196. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:12,363][19170] Avg episode reward: [(0, '990.220'), (1, '941.460')] [2023-12-27 15:04:12,864][20347] Updated weights for policy 0, policy_version 1359840 (0.0009) [2023-12-27 15:04:12,920][20347] Updated weights for policy 0, policy_version 1359850 (0.0008) [2023-12-27 15:04:12,956][20381] Updated weights for policy 1, policy_version 1358775 (0.0008) [2023-12-27 15:04:12,974][20347] Updated weights for policy 0, policy_version 1359860 (0.0008) [2023-12-27 15:04:13,013][20381] Updated weights for policy 1, policy_version 1358785 (0.0006) [2023-12-27 15:04:13,063][20381] Updated weights for policy 1, policy_version 1358795 (0.0008) [2023-12-27 15:04:13,599][20347] Updated weights for policy 0, policy_version 1359870 (0.0007) [2023-12-27 15:04:13,647][20347] Updated weights for policy 0, policy_version 1359880 (0.0006) [2023-12-27 15:04:13,700][20347] Updated weights for policy 0, policy_version 1359890 (0.0007) [2023-12-27 15:04:13,796][20381] Updated weights for policy 1, policy_version 1358805 (0.0008) [2023-12-27 15:04:13,849][20381] Updated weights for policy 1, policy_version 1358816 (0.0009) [2023-12-27 15:04:13,917][20381] Updated weights for policy 1, policy_version 1358826 (0.0007) [2023-12-27 15:04:14,301][20347] Updated weights for policy 0, policy_version 1359900 (0.0009) [2023-12-27 15:04:14,352][20347] Updated weights for policy 0, policy_version 1359910 (0.0007) [2023-12-27 15:04:14,402][20347] Updated weights for policy 0, policy_version 1359920 (0.0007) [2023-12-27 15:04:14,631][20381] Updated weights for policy 1, policy_version 1358836 (0.0009) [2023-12-27 15:04:14,681][20381] Updated weights for policy 1, policy_version 1358846 (0.0007) [2023-12-27 15:04:14,729][20381] Updated weights for policy 1, policy_version 1358856 (0.0007) [2023-12-27 15:04:15,140][20347] Updated weights for policy 0, policy_version 1359930 (0.0007) [2023-12-27 15:04:15,196][20347] Updated weights for policy 0, policy_version 1359940 (0.0008) [2023-12-27 15:04:15,251][20347] Updated weights for policy 0, policy_version 1359950 (0.0008) [2023-12-27 15:04:15,309][20347] Updated weights for policy 0, policy_version 1359960 (0.0008) [2023-12-27 15:04:15,471][20381] Updated weights for policy 1, policy_version 1358866 (0.0008) [2023-12-27 15:04:15,524][20381] Updated weights for policy 1, policy_version 1358876 (0.0008) [2023-12-27 15:04:15,583][20381] Updated weights for policy 1, policy_version 1358886 (0.0009) [2023-12-27 15:04:15,638][20381] Updated weights for policy 1, policy_version 1358896 (0.0007) [2023-12-27 15:04:15,876][20347] Updated weights for policy 0, policy_version 1359970 (0.0010) [2023-12-27 15:04:15,930][20347] Updated weights for policy 0, policy_version 1359980 (0.0009) [2023-12-27 15:04:15,975][20347] Updated weights for policy 0, policy_version 1359990 (0.0010) [2023-12-27 15:04:16,269][20381] Updated weights for policy 1, policy_version 1358906 (0.0005) [2023-12-27 15:04:16,319][20381] Updated weights for policy 1, policy_version 1358916 (0.0005) [2023-12-27 15:04:16,369][20381] Updated weights for policy 1, policy_version 1358926 (0.0005) [2023-12-27 15:04:16,667][20347] Updated weights for policy 0, policy_version 1360000 (0.0010) [2023-12-27 15:04:16,721][20347] Updated weights for policy 0, policy_version 1360010 (0.0010) [2023-12-27 15:04:16,775][20347] Updated weights for policy 0, policy_version 1360020 (0.0010) [2023-12-27 15:04:16,956][20381] Updated weights for policy 1, policy_version 1358936 (0.0007) [2023-12-27 15:04:17,004][20381] Updated weights for policy 1, policy_version 1358946 (0.0007) [2023-12-27 15:04:17,058][20381] Updated weights for policy 1, policy_version 1358956 (0.0008) [2023-12-27 15:04:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 696156160. Throughput: 0: 10098.1, 1: 10396.1. Samples: 696166880. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:17,363][19170] Avg episode reward: [(0, '990.220'), (1, '960.960')] [2023-12-27 15:04:17,426][20347] Updated weights for policy 0, policy_version 1360030 (0.0010) [2023-12-27 15:04:17,490][20347] Updated weights for policy 0, policy_version 1360040 (0.0010) [2023-12-27 15:04:17,550][20347] Updated weights for policy 0, policy_version 1360050 (0.0010) [2023-12-27 15:04:17,644][20381] Updated weights for policy 1, policy_version 1358966 (0.0006) [2023-12-27 15:04:17,692][20381] Updated weights for policy 1, policy_version 1358976 (0.0008) [2023-12-27 15:04:17,741][20381] Updated weights for policy 1, policy_version 1358986 (0.0010) [2023-12-27 15:04:18,146][20347] Updated weights for policy 0, policy_version 1360060 (0.0009) [2023-12-27 15:04:18,189][20347] Updated weights for policy 0, policy_version 1360070 (0.0007) [2023-12-27 15:04:18,240][20347] Updated weights for policy 0, policy_version 1360080 (0.0007) [2023-12-27 15:04:18,441][20381] Updated weights for policy 1, policy_version 1358996 (0.0010) [2023-12-27 15:04:18,497][20381] Updated weights for policy 1, policy_version 1359006 (0.0008) [2023-12-27 15:04:18,559][20381] Updated weights for policy 1, policy_version 1359016 (0.0008) [2023-12-27 15:04:18,930][20347] Updated weights for policy 0, policy_version 1360090 (0.0007) [2023-12-27 15:04:18,983][20347] Updated weights for policy 0, policy_version 1360100 (0.0008) [2023-12-27 15:04:19,032][20347] Updated weights for policy 0, policy_version 1360110 (0.0008) [2023-12-27 15:04:19,085][20347] Updated weights for policy 0, policy_version 1360120 (0.0008) [2023-12-27 15:04:19,235][20381] Updated weights for policy 1, policy_version 1359026 (0.0007) [2023-12-27 15:04:19,286][20381] Updated weights for policy 1, policy_version 1359036 (0.0008) [2023-12-27 15:04:19,355][20381] Updated weights for policy 1, policy_version 1359046 (0.0008) [2023-12-27 15:04:19,406][20381] Updated weights for policy 1, policy_version 1359056 (0.0006) [2023-12-27 15:04:19,857][20347] Updated weights for policy 0, policy_version 1360130 (0.0011) [2023-12-27 15:04:19,924][20347] Updated weights for policy 0, policy_version 1360140 (0.0008) [2023-12-27 15:04:19,984][20347] Updated weights for policy 0, policy_version 1360150 (0.0008) [2023-12-27 15:04:20,145][20381] Updated weights for policy 1, policy_version 1359066 (0.0007) [2023-12-27 15:04:20,205][20381] Updated weights for policy 1, policy_version 1359076 (0.0007) [2023-12-27 15:04:20,264][20381] Updated weights for policy 1, policy_version 1359086 (0.0007) [2023-12-27 15:04:20,755][20347] Updated weights for policy 0, policy_version 1360160 (0.0008) [2023-12-27 15:04:20,807][20347] Updated weights for policy 0, policy_version 1360170 (0.0007) [2023-12-27 15:04:20,863][20347] Updated weights for policy 0, policy_version 1360180 (0.0007) [2023-12-27 15:04:21,003][20381] Updated weights for policy 1, policy_version 1359096 (0.0008) [2023-12-27 15:04:21,070][20381] Updated weights for policy 1, policy_version 1359106 (0.0008) [2023-12-27 15:04:21,130][20381] Updated weights for policy 1, policy_version 1359116 (0.0008) [2023-12-27 15:04:21,583][20347] Updated weights for policy 0, policy_version 1360190 (0.0007) [2023-12-27 15:04:21,655][20347] Updated weights for policy 0, policy_version 1360200 (0.0007) [2023-12-27 15:04:21,718][20347] Updated weights for policy 0, policy_version 1360210 (0.0008) [2023-12-27 15:04:21,959][20381] Updated weights for policy 1, policy_version 1359126 (0.0008) [2023-12-27 15:04:22,023][20381] Updated weights for policy 1, policy_version 1359136 (0.0008) [2023-12-27 15:04:22,083][20381] Updated weights for policy 1, policy_version 1359146 (0.0007) [2023-12-27 15:04:22,356][20347] Updated weights for policy 0, policy_version 1360220 (0.0008) [2023-12-27 15:04:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 696254464. Throughput: 0: 10075.8, 1: 10412.5. Samples: 696227900. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.500')] [2023-12-27 15:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001359152_347971584.pth... [2023-12-27 15:04:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001357936_347660288.pth [2023-12-27 15:04:22,422][20347] Updated weights for policy 0, policy_version 1360230 (0.0008) [2023-12-27 15:04:22,485][20347] Updated weights for policy 0, policy_version 1360240 (0.0011) [2023-12-27 15:04:22,529][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001360248_348291072.pth... [2023-12-27 15:04:22,533][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001359032_347979776.pth [2023-12-27 15:04:22,795][20381] Updated weights for policy 1, policy_version 1359156 (0.0007) [2023-12-27 15:04:22,848][20381] Updated weights for policy 1, policy_version 1359166 (0.0008) [2023-12-27 15:04:22,895][20381] Updated weights for policy 1, policy_version 1359176 (0.0007) [2023-12-27 15:04:23,127][20347] Updated weights for policy 0, policy_version 1360250 (0.0009) [2023-12-27 15:04:23,179][20347] Updated weights for policy 0, policy_version 1360260 (0.0005) [2023-12-27 15:04:23,229][20347] Updated weights for policy 0, policy_version 1360270 (0.0007) [2023-12-27 15:04:23,278][20347] Updated weights for policy 0, policy_version 1360280 (0.0008) [2023-12-27 15:04:23,550][20381] Updated weights for policy 1, policy_version 1359186 (0.0006) [2023-12-27 15:04:23,600][20381] Updated weights for policy 1, policy_version 1359196 (0.0008) [2023-12-27 15:04:23,645][20381] Updated weights for policy 1, policy_version 1359206 (0.0008) [2023-12-27 15:04:23,700][20381] Updated weights for policy 1, policy_version 1359216 (0.0006) [2023-12-27 15:04:23,918][20347] Updated weights for policy 0, policy_version 1360290 (0.0007) [2023-12-27 15:04:23,966][20347] Updated weights for policy 0, policy_version 1360300 (0.0006) [2023-12-27 15:04:23,980][20291] KL-divergence is very high: 139.7434 [2023-12-27 15:04:24,027][20347] Updated weights for policy 0, policy_version 1360310 (0.0005) [2023-12-27 15:04:24,416][20381] Updated weights for policy 1, policy_version 1359226 (0.0008) [2023-12-27 15:04:24,476][20381] Updated weights for policy 1, policy_version 1359236 (0.0010) [2023-12-27 15:04:24,541][20381] Updated weights for policy 1, policy_version 1359246 (0.0010) [2023-12-27 15:04:24,695][20347] Updated weights for policy 0, policy_version 1360320 (0.0006) [2023-12-27 15:04:24,759][20347] Updated weights for policy 0, policy_version 1360330 (0.0006) [2023-12-27 15:04:24,818][20347] Updated weights for policy 0, policy_version 1360340 (0.0008) [2023-12-27 15:04:25,209][20381] Updated weights for policy 1, policy_version 1359256 (0.0010) [2023-12-27 15:04:25,256][20381] Updated weights for policy 1, policy_version 1359266 (0.0010) [2023-12-27 15:04:25,301][20381] Updated weights for policy 1, policy_version 1359276 (0.0010) [2023-12-27 15:04:25,458][20347] Updated weights for policy 0, policy_version 1360350 (0.0008) [2023-12-27 15:04:25,516][20347] Updated weights for policy 0, policy_version 1360360 (0.0010) [2023-12-27 15:04:25,567][20347] Updated weights for policy 0, policy_version 1360370 (0.0010) [2023-12-27 15:04:26,010][20381] Updated weights for policy 1, policy_version 1359286 (0.0010) [2023-12-27 15:04:26,062][20381] Updated weights for policy 1, policy_version 1359296 (0.0011) [2023-12-27 15:04:26,115][20381] Updated weights for policy 1, policy_version 1359306 (0.0009) [2023-12-27 15:04:26,161][20347] Updated weights for policy 0, policy_version 1360380 (0.0009) [2023-12-27 15:04:26,217][20347] Updated weights for policy 0, policy_version 1360390 (0.0006) [2023-12-27 15:04:26,272][20347] Updated weights for policy 0, policy_version 1360400 (0.0010) [2023-12-27 15:04:26,775][20381] Updated weights for policy 1, policy_version 1359316 (0.0007) [2023-12-27 15:04:26,822][20381] Updated weights for policy 1, policy_version 1359326 (0.0010) [2023-12-27 15:04:26,868][20381] Updated weights for policy 1, policy_version 1359336 (0.0009) [2023-12-27 15:04:26,908][20347] Updated weights for policy 0, policy_version 1360410 (0.0008) [2023-12-27 15:04:26,950][20347] Updated weights for policy 0, policy_version 1360420 (0.0007) [2023-12-27 15:04:27,009][20347] Updated weights for policy 0, policy_version 1360430 (0.0007) [2023-12-27 15:04:27,065][20347] Updated weights for policy 0, policy_version 1360440 (0.0007) [2023-12-27 15:04:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 696360960. Throughput: 0: 10154.3, 1: 10356.1. Samples: 696349112. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:27,363][19170] Avg episode reward: [(0, '990.270'), (1, '980.480')] [2023-12-27 15:04:27,540][20381] Updated weights for policy 1, policy_version 1359346 (0.0009) [2023-12-27 15:04:27,595][20381] Updated weights for policy 1, policy_version 1359356 (0.0005) [2023-12-27 15:04:27,645][20381] Updated weights for policy 1, policy_version 1359366 (0.0005) [2023-12-27 15:04:27,701][20381] Updated weights for policy 1, policy_version 1359376 (0.0005) [2023-12-27 15:04:27,746][20347] Updated weights for policy 0, policy_version 1360451 (0.0009) [2023-12-27 15:04:27,797][20347] Updated weights for policy 0, policy_version 1360461 (0.0009) [2023-12-27 15:04:27,855][20347] Updated weights for policy 0, policy_version 1360471 (0.0008) [2023-12-27 15:04:28,276][20381] Updated weights for policy 1, policy_version 1359386 (0.0011) [2023-12-27 15:04:28,338][20381] Updated weights for policy 1, policy_version 1359396 (0.0010) [2023-12-27 15:04:28,397][20381] Updated weights for policy 1, policy_version 1359406 (0.0009) [2023-12-27 15:04:28,484][20347] Updated weights for policy 0, policy_version 1360481 (0.0009) [2023-12-27 15:04:28,540][20347] Updated weights for policy 0, policy_version 1360491 (0.0010) [2023-12-27 15:04:28,596][20347] Updated weights for policy 0, policy_version 1360501 (0.0011) [2023-12-27 15:04:29,071][20381] Updated weights for policy 1, policy_version 1359416 (0.0011) [2023-12-27 15:04:29,117][20381] Updated weights for policy 1, policy_version 1359426 (0.0008) [2023-12-27 15:04:29,182][20381] Updated weights for policy 1, policy_version 1359436 (0.0005) [2023-12-27 15:04:29,254][20347] Updated weights for policy 0, policy_version 1360511 (0.0008) [2023-12-27 15:04:29,315][20347] Updated weights for policy 0, policy_version 1360521 (0.0009) [2023-12-27 15:04:29,379][20347] Updated weights for policy 0, policy_version 1360531 (0.0010) [2023-12-27 15:04:29,872][20381] Updated weights for policy 1, policy_version 1359446 (0.0006) [2023-12-27 15:04:29,931][20381] Updated weights for policy 1, policy_version 1359456 (0.0008) [2023-12-27 15:04:29,992][20381] Updated weights for policy 1, policy_version 1359466 (0.0007) [2023-12-27 15:04:30,029][20347] Updated weights for policy 0, policy_version 1360541 (0.0009) [2023-12-27 15:04:30,080][20347] Updated weights for policy 0, policy_version 1360551 (0.0010) [2023-12-27 15:04:30,135][20347] Updated weights for policy 0, policy_version 1360561 (0.0010) [2023-12-27 15:04:30,692][20381] Updated weights for policy 1, policy_version 1359476 (0.0006) [2023-12-27 15:04:30,739][20381] Updated weights for policy 1, policy_version 1359486 (0.0005) [2023-12-27 15:04:30,795][20381] Updated weights for policy 1, policy_version 1359496 (0.0005) [2023-12-27 15:04:30,822][20347] Updated weights for policy 0, policy_version 1360571 (0.0010) [2023-12-27 15:04:30,866][20347] Updated weights for policy 0, policy_version 1360581 (0.0008) [2023-12-27 15:04:30,915][20347] Updated weights for policy 0, policy_version 1360591 (0.0005) [2023-12-27 15:04:31,444][20381] Updated weights for policy 1, policy_version 1359506 (0.0006) [2023-12-27 15:04:31,505][20381] Updated weights for policy 1, policy_version 1359516 (0.0008) [2023-12-27 15:04:31,559][20381] Updated weights for policy 1, policy_version 1359526 (0.0008) [2023-12-27 15:04:31,622][20347] Updated weights for policy 0, policy_version 1360601 (0.0006) [2023-12-27 15:04:31,623][20381] Updated weights for policy 1, policy_version 1359536 (0.0006) [2023-12-27 15:04:31,680][20347] Updated weights for policy 0, policy_version 1360611 (0.0007) [2023-12-27 15:04:31,745][20347] Updated weights for policy 0, policy_version 1360621 (0.0007) [2023-12-27 15:04:31,806][20347] Updated weights for policy 0, policy_version 1360631 (0.0007) [2023-12-27 15:04:32,311][20381] Updated weights for policy 1, policy_version 1359546 (0.0007) [2023-12-27 15:04:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 696459264. Throughput: 0: 10285.5, 1: 10300.4. Samples: 696475620. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.680')] [2023-12-27 15:04:32,367][20347] Updated weights for policy 0, policy_version 1360641 (0.0007) [2023-12-27 15:04:32,375][20381] Updated weights for policy 1, policy_version 1359556 (0.0008) [2023-12-27 15:04:32,423][20347] Updated weights for policy 0, policy_version 1360651 (0.0006) [2023-12-27 15:04:32,436][20381] Updated weights for policy 1, policy_version 1359566 (0.0005) [2023-12-27 15:04:32,484][20347] Updated weights for policy 0, policy_version 1360661 (0.0005) [2023-12-27 15:04:33,014][20381] Updated weights for policy 1, policy_version 1359576 (0.0007) [2023-12-27 15:04:33,066][20381] Updated weights for policy 1, policy_version 1359586 (0.0007) [2023-12-27 15:04:33,072][20347] Updated weights for policy 0, policy_version 1360671 (0.0005) [2023-12-27 15:04:33,118][20381] Updated weights for policy 1, policy_version 1359596 (0.0006) [2023-12-27 15:04:33,120][20347] Updated weights for policy 0, policy_version 1360681 (0.0005) [2023-12-27 15:04:33,174][20347] Updated weights for policy 0, policy_version 1360691 (0.0006) [2023-12-27 15:04:33,727][20381] Updated weights for policy 1, policy_version 1359606 (0.0007) [2023-12-27 15:04:33,782][20381] Updated weights for policy 1, policy_version 1359616 (0.0006) [2023-12-27 15:04:33,839][20381] Updated weights for policy 1, policy_version 1359626 (0.0005) [2023-12-27 15:04:33,904][20347] Updated weights for policy 0, policy_version 1360701 (0.0008) [2023-12-27 15:04:33,947][20347] Updated weights for policy 0, policy_version 1360711 (0.0005) [2023-12-27 15:04:33,995][20347] Updated weights for policy 0, policy_version 1360721 (0.0005) [2023-12-27 15:04:34,421][20381] Updated weights for policy 1, policy_version 1359636 (0.0007) [2023-12-27 15:04:34,477][20381] Updated weights for policy 1, policy_version 1359646 (0.0007) [2023-12-27 15:04:34,536][20381] Updated weights for policy 1, policy_version 1359656 (0.0008) [2023-12-27 15:04:34,748][20347] Updated weights for policy 0, policy_version 1360731 (0.0008) [2023-12-27 15:04:34,801][20347] Updated weights for policy 0, policy_version 1360741 (0.0008) [2023-12-27 15:04:34,848][20347] Updated weights for policy 0, policy_version 1360751 (0.0007) [2023-12-27 15:04:35,189][20381] Updated weights for policy 1, policy_version 1359666 (0.0008) [2023-12-27 15:04:35,241][20381] Updated weights for policy 1, policy_version 1359676 (0.0008) [2023-12-27 15:04:35,291][20381] Updated weights for policy 1, policy_version 1359686 (0.0007) [2023-12-27 15:04:35,346][20381] Updated weights for policy 1, policy_version 1359696 (0.0008) [2023-12-27 15:04:35,550][20347] Updated weights for policy 0, policy_version 1360761 (0.0008) [2023-12-27 15:04:35,615][20347] Updated weights for policy 0, policy_version 1360771 (0.0006) [2023-12-27 15:04:35,674][20347] Updated weights for policy 0, policy_version 1360781 (0.0006) [2023-12-27 15:04:35,739][20347] Updated weights for policy 0, policy_version 1360791 (0.0005) [2023-12-27 15:04:36,040][20381] Updated weights for policy 1, policy_version 1359706 (0.0008) [2023-12-27 15:04:36,096][20381] Updated weights for policy 1, policy_version 1359716 (0.0008) [2023-12-27 15:04:36,155][20381] Updated weights for policy 1, policy_version 1359726 (0.0008) [2023-12-27 15:04:36,355][20347] Updated weights for policy 0, policy_version 1360801 (0.0006) [2023-12-27 15:04:36,418][20347] Updated weights for policy 0, policy_version 1360811 (0.0006) [2023-12-27 15:04:36,481][20347] Updated weights for policy 0, policy_version 1360821 (0.0006) [2023-12-27 15:04:36,940][20381] Updated weights for policy 1, policy_version 1359736 (0.0008) [2023-12-27 15:04:36,997][20381] Updated weights for policy 1, policy_version 1359746 (0.0008) [2023-12-27 15:04:37,054][20381] Updated weights for policy 1, policy_version 1359756 (0.0007) [2023-12-27 15:04:37,166][20347] Updated weights for policy 0, policy_version 1360831 (0.0008) [2023-12-27 15:04:37,225][20347] Updated weights for policy 0, policy_version 1360841 (0.0008) [2023-12-27 15:04:37,279][20347] Updated weights for policy 0, policy_version 1360851 (0.0009) [2023-12-27 15:04:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 696573952. Throughput: 0: 10333.7, 1: 10336.8. Samples: 696539748. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.900')] [2023-12-27 15:04:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001359760_348127232.pth... [2023-12-27 15:04:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001360856_348446720.pth... [2023-12-27 15:04:37,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001358576_347824128.pth [2023-12-27 15:04:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001359640_348135424.pth [2023-12-27 15:04:37,688][20381] Updated weights for policy 1, policy_version 1359766 (0.0007) [2023-12-27 15:04:37,744][20381] Updated weights for policy 1, policy_version 1359776 (0.0007) [2023-12-27 15:04:37,805][20381] Updated weights for policy 1, policy_version 1359786 (0.0007) [2023-12-27 15:04:37,998][20347] Updated weights for policy 0, policy_version 1360861 (0.0009) [2023-12-27 15:04:38,063][20347] Updated weights for policy 0, policy_version 1360871 (0.0007) [2023-12-27 15:04:38,127][20347] Updated weights for policy 0, policy_version 1360881 (0.0007) [2023-12-27 15:04:38,463][20381] Updated weights for policy 1, policy_version 1359796 (0.0008) [2023-12-27 15:04:38,518][20381] Updated weights for policy 1, policy_version 1359806 (0.0007) [2023-12-27 15:04:38,570][20381] Updated weights for policy 1, policy_version 1359816 (0.0008) [2023-12-27 15:04:38,825][20347] Updated weights for policy 0, policy_version 1360891 (0.0009) [2023-12-27 15:04:38,883][20347] Updated weights for policy 0, policy_version 1360901 (0.0008) [2023-12-27 15:04:38,942][20347] Updated weights for policy 0, policy_version 1360911 (0.0007) [2023-12-27 15:04:39,320][20381] Updated weights for policy 1, policy_version 1359826 (0.0007) [2023-12-27 15:04:39,392][20381] Updated weights for policy 1, policy_version 1359836 (0.0009) [2023-12-27 15:04:39,458][20381] Updated weights for policy 1, policy_version 1359846 (0.0008) [2023-12-27 15:04:39,524][20381] Updated weights for policy 1, policy_version 1359856 (0.0008) [2023-12-27 15:04:39,642][20347] Updated weights for policy 0, policy_version 1360921 (0.0007) [2023-12-27 15:04:39,709][20347] Updated weights for policy 0, policy_version 1360931 (0.0007) [2023-12-27 15:04:39,763][20347] Updated weights for policy 0, policy_version 1360941 (0.0008) [2023-12-27 15:04:39,830][20347] Updated weights for policy 0, policy_version 1360951 (0.0009) [2023-12-27 15:04:40,242][20381] Updated weights for policy 1, policy_version 1359866 (0.0006) [2023-12-27 15:04:40,302][20381] Updated weights for policy 1, policy_version 1359876 (0.0007) [2023-12-27 15:04:40,363][20381] Updated weights for policy 1, policy_version 1359886 (0.0008) [2023-12-27 15:04:40,570][20347] Updated weights for policy 0, policy_version 1360961 (0.0008) [2023-12-27 15:04:40,622][20347] Updated weights for policy 0, policy_version 1360971 (0.0008) [2023-12-27 15:04:40,681][20347] Updated weights for policy 0, policy_version 1360981 (0.0008) [2023-12-27 15:04:40,999][20381] Updated weights for policy 1, policy_version 1359896 (0.0007) [2023-12-27 15:04:41,065][20381] Updated weights for policy 1, policy_version 1359907 (0.0007) [2023-12-27 15:04:41,126][20381] Updated weights for policy 1, policy_version 1359917 (0.0007) [2023-12-27 15:04:41,402][20347] Updated weights for policy 0, policy_version 1360991 (0.0008) [2023-12-27 15:04:41,454][20347] Updated weights for policy 0, policy_version 1361001 (0.0008) [2023-12-27 15:04:41,506][20347] Updated weights for policy 0, policy_version 1361011 (0.0008) [2023-12-27 15:04:41,939][20381] Updated weights for policy 1, policy_version 1359927 (0.0008) [2023-12-27 15:04:42,008][20381] Updated weights for policy 1, policy_version 1359937 (0.0008) [2023-12-27 15:04:42,069][20381] Updated weights for policy 1, policy_version 1359947 (0.0008) [2023-12-27 15:04:42,192][20347] Updated weights for policy 0, policy_version 1361021 (0.0007) [2023-12-27 15:04:42,252][20347] Updated weights for policy 0, policy_version 1361031 (0.0007) [2023-12-27 15:04:42,316][20347] Updated weights for policy 0, policy_version 1361041 (0.0008) [2023-12-27 15:04:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 696664064. Throughput: 0: 10278.5, 1: 10255.8. Samples: 696658512. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.680')] [2023-12-27 15:04:42,770][20381] Updated weights for policy 1, policy_version 1359957 (0.0007) [2023-12-27 15:04:42,823][20381] Updated weights for policy 1, policy_version 1359967 (0.0008) [2023-12-27 15:04:42,872][20381] Updated weights for policy 1, policy_version 1359977 (0.0008) [2023-12-27 15:04:42,991][20347] Updated weights for policy 0, policy_version 1361051 (0.0008) [2023-12-27 15:04:43,041][20347] Updated weights for policy 0, policy_version 1361061 (0.0007) [2023-12-27 15:04:43,095][20347] Updated weights for policy 0, policy_version 1361071 (0.0008) [2023-12-27 15:04:43,581][20381] Updated weights for policy 1, policy_version 1359987 (0.0007) [2023-12-27 15:04:43,634][20381] Updated weights for policy 1, policy_version 1359997 (0.0008) [2023-12-27 15:04:43,680][20381] Updated weights for policy 1, policy_version 1360007 (0.0008) [2023-12-27 15:04:43,805][20347] Updated weights for policy 0, policy_version 1361081 (0.0008) [2023-12-27 15:04:43,861][20347] Updated weights for policy 0, policy_version 1361091 (0.0007) [2023-12-27 15:04:43,917][20347] Updated weights for policy 0, policy_version 1361101 (0.0007) [2023-12-27 15:04:43,975][20347] Updated weights for policy 0, policy_version 1361111 (0.0008) [2023-12-27 15:04:44,288][20381] Updated weights for policy 1, policy_version 1360017 (0.0007) [2023-12-27 15:04:44,339][20381] Updated weights for policy 1, policy_version 1360027 (0.0005) [2023-12-27 15:04:44,394][20381] Updated weights for policy 1, policy_version 1360037 (0.0006) [2023-12-27 15:04:44,449][20381] Updated weights for policy 1, policy_version 1360047 (0.0008) [2023-12-27 15:04:44,720][20347] Updated weights for policy 0, policy_version 1361121 (0.0006) [2023-12-27 15:04:44,780][20347] Updated weights for policy 0, policy_version 1361131 (0.0008) [2023-12-27 15:04:44,844][20347] Updated weights for policy 0, policy_version 1361141 (0.0008) [2023-12-27 15:04:45,077][20381] Updated weights for policy 1, policy_version 1360057 (0.0008) [2023-12-27 15:04:45,139][20381] Updated weights for policy 1, policy_version 1360067 (0.0009) [2023-12-27 15:04:45,203][20381] Updated weights for policy 1, policy_version 1360077 (0.0008) [2023-12-27 15:04:45,522][20347] Updated weights for policy 0, policy_version 1361151 (0.0008) [2023-12-27 15:04:45,579][20347] Updated weights for policy 0, policy_version 1361161 (0.0007) [2023-12-27 15:04:45,629][20347] Updated weights for policy 0, policy_version 1361171 (0.0007) [2023-12-27 15:04:45,885][20381] Updated weights for policy 1, policy_version 1360087 (0.0006) [2023-12-27 15:04:45,929][20381] Updated weights for policy 1, policy_version 1360097 (0.0006) [2023-12-27 15:04:45,978][20381] Updated weights for policy 1, policy_version 1360107 (0.0007) [2023-12-27 15:04:46,292][20347] Updated weights for policy 0, policy_version 1361181 (0.0006) [2023-12-27 15:04:46,342][20347] Updated weights for policy 0, policy_version 1361191 (0.0005) [2023-12-27 15:04:46,397][20347] Updated weights for policy 0, policy_version 1361201 (0.0005) [2023-12-27 15:04:46,617][20381] Updated weights for policy 1, policy_version 1360117 (0.0007) [2023-12-27 15:04:46,679][20381] Updated weights for policy 1, policy_version 1360127 (0.0008) [2023-12-27 15:04:46,731][20381] Updated weights for policy 1, policy_version 1360137 (0.0008) [2023-12-27 15:04:46,950][20347] Updated weights for policy 0, policy_version 1361211 (0.0005) [2023-12-27 15:04:47,011][20347] Updated weights for policy 0, policy_version 1361221 (0.0006) [2023-12-27 15:04:47,068][20347] Updated weights for policy 0, policy_version 1361231 (0.0005) [2023-12-27 15:04:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 696770560. Throughput: 0: 10313.2, 1: 10241.5. Samples: 696782104. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:47,363][19170] Avg episode reward: [(0, '990.380'), (1, '990.240')] [2023-12-27 15:04:47,410][20381] Updated weights for policy 1, policy_version 1360147 (0.0009) [2023-12-27 15:04:47,458][20381] Updated weights for policy 1, policy_version 1360157 (0.0007) [2023-12-27 15:04:47,511][20381] Updated weights for policy 1, policy_version 1360167 (0.0005) [2023-12-27 15:04:47,567][20347] Updated weights for policy 0, policy_version 1361241 (0.0006) [2023-12-27 15:04:47,618][20347] Updated weights for policy 0, policy_version 1361251 (0.0008) [2023-12-27 15:04:47,672][20347] Updated weights for policy 0, policy_version 1361261 (0.0008) [2023-12-27 15:04:47,730][20347] Updated weights for policy 0, policy_version 1361271 (0.0008) [2023-12-27 15:04:48,267][20381] Updated weights for policy 1, policy_version 1360177 (0.0008) [2023-12-27 15:04:48,288][20347] Updated weights for policy 0, policy_version 1361281 (0.0006) [2023-12-27 15:04:48,316][20381] Updated weights for policy 1, policy_version 1360187 (0.0007) [2023-12-27 15:04:48,355][20347] Updated weights for policy 0, policy_version 1361291 (0.0010) [2023-12-27 15:04:48,382][20381] Updated weights for policy 1, policy_version 1360197 (0.0008) [2023-12-27 15:04:48,406][20347] Updated weights for policy 0, policy_version 1361301 (0.0010) [2023-12-27 15:04:48,436][20381] Updated weights for policy 1, policy_version 1360207 (0.0006) [2023-12-27 15:04:49,013][20347] Updated weights for policy 0, policy_version 1361311 (0.0007) [2023-12-27 15:04:49,061][20347] Updated weights for policy 0, policy_version 1361321 (0.0005) [2023-12-27 15:04:49,108][20347] Updated weights for policy 0, policy_version 1361331 (0.0005) [2023-12-27 15:04:49,196][20381] Updated weights for policy 1, policy_version 1360217 (0.0006) [2023-12-27 15:04:49,264][20381] Updated weights for policy 1, policy_version 1360227 (0.0008) [2023-12-27 15:04:49,321][20381] Updated weights for policy 1, policy_version 1360237 (0.0007) [2023-12-27 15:04:49,741][20347] Updated weights for policy 0, policy_version 1361341 (0.0007) [2023-12-27 15:04:49,791][20347] Updated weights for policy 0, policy_version 1361351 (0.0005) [2023-12-27 15:04:49,855][20347] Updated weights for policy 0, policy_version 1361361 (0.0007) [2023-12-27 15:04:49,960][20381] Updated weights for policy 1, policy_version 1360247 (0.0008) [2023-12-27 15:04:50,022][20381] Updated weights for policy 1, policy_version 1360257 (0.0007) [2023-12-27 15:04:50,079][20381] Updated weights for policy 1, policy_version 1360267 (0.0008) [2023-12-27 15:04:50,575][20347] Updated weights for policy 0, policy_version 1361371 (0.0007) [2023-12-27 15:04:50,645][20347] Updated weights for policy 0, policy_version 1361381 (0.0009) [2023-12-27 15:04:50,708][20347] Updated weights for policy 0, policy_version 1361391 (0.0008) [2023-12-27 15:04:50,762][20381] Updated weights for policy 1, policy_version 1360277 (0.0007) [2023-12-27 15:04:50,823][20381] Updated weights for policy 1, policy_version 1360287 (0.0008) [2023-12-27 15:04:50,880][20381] Updated weights for policy 1, policy_version 1360297 (0.0008) [2023-12-27 15:04:51,451][20347] Updated weights for policy 0, policy_version 1361401 (0.0008) [2023-12-27 15:04:51,490][20291] KL-divergence is very high: 371.8430 [2023-12-27 15:04:51,514][20347] Updated weights for policy 0, policy_version 1361411 (0.0007) [2023-12-27 15:04:51,514][20291] KL-divergence is very high: 547.2099 [2023-12-27 15:04:51,538][20291] KL-divergence is very high: 486.1371 [2023-12-27 15:04:51,562][20291] KL-divergence is very high: 156.7847 [2023-12-27 15:04:51,573][20347] Updated weights for policy 0, policy_version 1361421 (0.0008) [2023-12-27 15:04:51,609][20381] Updated weights for policy 1, policy_version 1360307 (0.0008) [2023-12-27 15:04:51,636][20291] KL-divergence is very high: 234.6932 [2023-12-27 15:04:51,637][20347] Updated weights for policy 0, policy_version 1361431 (0.0007) [2023-12-27 15:04:51,687][20381] Updated weights for policy 1, policy_version 1360317 (0.0009) [2023-12-27 15:04:51,767][20381] Updated weights for policy 1, policy_version 1360327 (0.0007) [2023-12-27 15:04:52,292][20347] Updated weights for policy 0, policy_version 1361441 (0.0007) [2023-12-27 15:04:52,349][20347] Updated weights for policy 0, policy_version 1361451 (0.0007) [2023-12-27 15:04:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 696868864. Throughput: 0: 10407.5, 1: 10146.2. Samples: 696845404. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:52,363][19170] Avg episode reward: [(0, '980.590'), (1, '990.220')] [2023-12-27 15:04:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001360336_348274688.pth... [2023-12-27 15:04:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001359152_347971584.pth [2023-12-27 15:04:52,416][20347] Updated weights for policy 0, policy_version 1361461 (0.0006) [2023-12-27 15:04:52,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001361464_348602368.pth... [2023-12-27 15:04:52,440][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001360248_348291072.pth [2023-12-27 15:04:52,478][20381] Updated weights for policy 1, policy_version 1360337 (0.0008) [2023-12-27 15:04:52,533][20381] Updated weights for policy 1, policy_version 1360347 (0.0010) [2023-12-27 15:04:52,588][20381] Updated weights for policy 1, policy_version 1360357 (0.0008) [2023-12-27 15:04:52,645][20381] Updated weights for policy 1, policy_version 1360367 (0.0007) [2023-12-27 15:04:53,093][20347] Updated weights for policy 0, policy_version 1361471 (0.0007) [2023-12-27 15:04:53,151][20347] Updated weights for policy 0, policy_version 1361481 (0.0007) [2023-12-27 15:04:53,204][20347] Updated weights for policy 0, policy_version 1361491 (0.0009) [2023-12-27 15:04:53,213][20381] Updated weights for policy 1, policy_version 1360377 (0.0005) [2023-12-27 15:04:53,262][20381] Updated weights for policy 1, policy_version 1360387 (0.0005) [2023-12-27 15:04:53,314][20381] Updated weights for policy 1, policy_version 1360397 (0.0006) [2023-12-27 15:04:53,911][20347] Updated weights for policy 0, policy_version 1361501 (0.0007) [2023-12-27 15:04:53,974][20347] Updated weights for policy 0, policy_version 1361511 (0.0006) [2023-12-27 15:04:53,990][20381] Updated weights for policy 1, policy_version 1360407 (0.0008) [2023-12-27 15:04:54,033][20347] Updated weights for policy 0, policy_version 1361521 (0.0006) [2023-12-27 15:04:54,051][20381] Updated weights for policy 1, policy_version 1360417 (0.0007) [2023-12-27 15:04:54,099][20381] Updated weights for policy 1, policy_version 1360427 (0.0009) [2023-12-27 15:04:54,620][20347] Updated weights for policy 0, policy_version 1361531 (0.0006) [2023-12-27 15:04:54,680][20347] Updated weights for policy 0, policy_version 1361541 (0.0008) [2023-12-27 15:04:54,729][20347] Updated weights for policy 0, policy_version 1361551 (0.0008) [2023-12-27 15:04:54,831][20381] Updated weights for policy 1, policy_version 1360437 (0.0009) [2023-12-27 15:04:54,881][20381] Updated weights for policy 1, policy_version 1360447 (0.0007) [2023-12-27 15:04:54,938][20381] Updated weights for policy 1, policy_version 1360457 (0.0007) [2023-12-27 15:04:55,451][20347] Updated weights for policy 0, policy_version 1361561 (0.0008) [2023-12-27 15:04:55,507][20347] Updated weights for policy 0, policy_version 1361571 (0.0008) [2023-12-27 15:04:55,562][20347] Updated weights for policy 0, policy_version 1361581 (0.0007) [2023-12-27 15:04:55,612][20347] Updated weights for policy 0, policy_version 1361591 (0.0007) [2023-12-27 15:04:55,648][20381] Updated weights for policy 1, policy_version 1360467 (0.0008) [2023-12-27 15:04:55,698][20381] Updated weights for policy 1, policy_version 1360477 (0.0008) [2023-12-27 15:04:55,756][20381] Updated weights for policy 1, policy_version 1360487 (0.0007) [2023-12-27 15:04:56,263][20347] Updated weights for policy 0, policy_version 1361601 (0.0006) [2023-12-27 15:04:56,326][20347] Updated weights for policy 0, policy_version 1361611 (0.0005) [2023-12-27 15:04:56,385][20347] Updated weights for policy 0, policy_version 1361621 (0.0005) [2023-12-27 15:04:56,519][20381] Updated weights for policy 1, policy_version 1360497 (0.0007) [2023-12-27 15:04:56,580][20381] Updated weights for policy 1, policy_version 1360507 (0.0007) [2023-12-27 15:04:56,640][20381] Updated weights for policy 1, policy_version 1360517 (0.0006) [2023-12-27 15:04:56,701][20381] Updated weights for policy 1, policy_version 1360527 (0.0006) [2023-12-27 15:04:56,936][20347] Updated weights for policy 0, policy_version 1361631 (0.0007) [2023-12-27 15:04:56,996][20347] Updated weights for policy 0, policy_version 1361641 (0.0007) [2023-12-27 15:04:57,054][20347] Updated weights for policy 0, policy_version 1361651 (0.0007) [2023-12-27 15:04:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 696975360. Throughput: 0: 10355.3, 1: 10124.3. Samples: 696965780. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:04:57,363][19170] Avg episode reward: [(0, '980.450'), (1, '980.440')] [2023-12-27 15:04:57,379][20381] Updated weights for policy 1, policy_version 1360537 (0.0006) [2023-12-27 15:04:57,427][20381] Updated weights for policy 1, policy_version 1360547 (0.0006) [2023-12-27 15:04:57,480][20381] Updated weights for policy 1, policy_version 1360557 (0.0008) [2023-12-27 15:04:57,636][20347] Updated weights for policy 0, policy_version 1361661 (0.0008) [2023-12-27 15:04:57,694][20347] Updated weights for policy 0, policy_version 1361671 (0.0007) [2023-12-27 15:04:57,748][20347] Updated weights for policy 0, policy_version 1361681 (0.0007) [2023-12-27 15:04:58,207][20381] Updated weights for policy 1, policy_version 1360567 (0.0009) [2023-12-27 15:04:58,263][20381] Updated weights for policy 1, policy_version 1360577 (0.0007) [2023-12-27 15:04:58,324][20381] Updated weights for policy 1, policy_version 1360587 (0.0008) [2023-12-27 15:04:58,366][20347] Updated weights for policy 0, policy_version 1361691 (0.0008) [2023-12-27 15:04:58,430][20347] Updated weights for policy 0, policy_version 1361701 (0.0010) [2023-12-27 15:04:58,486][20347] Updated weights for policy 0, policy_version 1361711 (0.0010) [2023-12-27 15:04:59,087][20381] Updated weights for policy 1, policy_version 1360598 (0.0008) [2023-12-27 15:04:59,143][20381] Updated weights for policy 1, policy_version 1360608 (0.0008) [2023-12-27 15:04:59,206][20381] Updated weights for policy 1, policy_version 1360618 (0.0007) [2023-12-27 15:04:59,223][20347] Updated weights for policy 0, policy_version 1361721 (0.0010) [2023-12-27 15:04:59,295][20347] Updated weights for policy 0, policy_version 1361731 (0.0008) [2023-12-27 15:04:59,360][20347] Updated weights for policy 0, policy_version 1361741 (0.0009) [2023-12-27 15:04:59,425][20347] Updated weights for policy 0, policy_version 1361751 (0.0008) [2023-12-27 15:04:59,869][20381] Updated weights for policy 1, policy_version 1360628 (0.0008) [2023-12-27 15:04:59,925][20381] Updated weights for policy 1, policy_version 1360638 (0.0006) [2023-12-27 15:04:59,984][20381] Updated weights for policy 1, policy_version 1360648 (0.0008) [2023-12-27 15:05:00,169][20347] Updated weights for policy 0, policy_version 1361761 (0.0007) [2023-12-27 15:05:00,221][20347] Updated weights for policy 0, policy_version 1361771 (0.0007) [2023-12-27 15:05:00,270][20347] Updated weights for policy 0, policy_version 1361781 (0.0008) [2023-12-27 15:05:00,689][20381] Updated weights for policy 1, policy_version 1360658 (0.0007) [2023-12-27 15:05:00,735][20381] Updated weights for policy 1, policy_version 1360668 (0.0006) [2023-12-27 15:05:00,787][20381] Updated weights for policy 1, policy_version 1360678 (0.0006) [2023-12-27 15:05:00,839][20381] Updated weights for policy 1, policy_version 1360688 (0.0008) [2023-12-27 15:05:01,007][20347] Updated weights for policy 0, policy_version 1361791 (0.0007) [2023-12-27 15:05:01,077][20347] Updated weights for policy 0, policy_version 1361801 (0.0008) [2023-12-27 15:05:01,137][20347] Updated weights for policy 0, policy_version 1361811 (0.0008) [2023-12-27 15:05:01,548][20381] Updated weights for policy 1, policy_version 1360698 (0.0008) [2023-12-27 15:05:01,598][20381] Updated weights for policy 1, policy_version 1360708 (0.0008) [2023-12-27 15:05:01,662][20381] Updated weights for policy 1, policy_version 1360718 (0.0009) [2023-12-27 15:05:01,879][20347] Updated weights for policy 0, policy_version 1361821 (0.0007) [2023-12-27 15:05:01,935][20347] Updated weights for policy 0, policy_version 1361831 (0.0007) [2023-12-27 15:05:01,988][20347] Updated weights for policy 0, policy_version 1361841 (0.0008) [2023-12-27 15:05:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 697073664. Throughput: 0: 10331.3, 1: 10093.7. Samples: 697086004. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:02,363][19170] Avg episode reward: [(0, '990.240'), (1, '960.900')] [2023-12-27 15:05:02,426][20381] Updated weights for policy 1, policy_version 1360728 (0.0008) [2023-12-27 15:05:02,481][20381] Updated weights for policy 1, policy_version 1360738 (0.0008) [2023-12-27 15:05:02,533][20381] Updated weights for policy 1, policy_version 1360748 (0.0008) [2023-12-27 15:05:02,643][20347] Updated weights for policy 0, policy_version 1361851 (0.0007) [2023-12-27 15:05:02,692][20347] Updated weights for policy 0, policy_version 1361861 (0.0007) [2023-12-27 15:05:02,743][20347] Updated weights for policy 0, policy_version 1361871 (0.0007) [2023-12-27 15:05:03,205][20381] Updated weights for policy 1, policy_version 1360758 (0.0007) [2023-12-27 15:05:03,251][20381] Updated weights for policy 1, policy_version 1360768 (0.0008) [2023-12-27 15:05:03,299][20381] Updated weights for policy 1, policy_version 1360778 (0.0007) [2023-12-27 15:05:03,444][20347] Updated weights for policy 0, policy_version 1361881 (0.0008) [2023-12-27 15:05:03,489][20347] Updated weights for policy 0, policy_version 1361891 (0.0007) [2023-12-27 15:05:03,536][20347] Updated weights for policy 0, policy_version 1361901 (0.0007) [2023-12-27 15:05:03,591][20347] Updated weights for policy 0, policy_version 1361911 (0.0008) [2023-12-27 15:05:04,021][20381] Updated weights for policy 1, policy_version 1360788 (0.0007) [2023-12-27 15:05:04,091][20381] Updated weights for policy 1, policy_version 1360798 (0.0008) [2023-12-27 15:05:04,150][20381] Updated weights for policy 1, policy_version 1360808 (0.0009) [2023-12-27 15:05:04,303][20347] Updated weights for policy 0, policy_version 1361921 (0.0008) [2023-12-27 15:05:04,364][20347] Updated weights for policy 0, policy_version 1361931 (0.0008) [2023-12-27 15:05:04,420][20347] Updated weights for policy 0, policy_version 1361941 (0.0007) [2023-12-27 15:05:04,850][20381] Updated weights for policy 1, policy_version 1360818 (0.0008) [2023-12-27 15:05:04,901][20381] Updated weights for policy 1, policy_version 1360828 (0.0007) [2023-12-27 15:05:04,957][20381] Updated weights for policy 1, policy_version 1360838 (0.0008) [2023-12-27 15:05:05,014][20381] Updated weights for policy 1, policy_version 1360848 (0.0008) [2023-12-27 15:05:05,105][20347] Updated weights for policy 0, policy_version 1361951 (0.0007) [2023-12-27 15:05:05,163][20347] Updated weights for policy 0, policy_version 1361961 (0.0008) [2023-12-27 15:05:05,218][20347] Updated weights for policy 0, policy_version 1361971 (0.0007) [2023-12-27 15:05:05,719][20381] Updated weights for policy 1, policy_version 1360858 (0.0007) [2023-12-27 15:05:05,774][20381] Updated weights for policy 1, policy_version 1360868 (0.0008) [2023-12-27 15:05:05,835][20381] Updated weights for policy 1, policy_version 1360878 (0.0008) [2023-12-27 15:05:05,902][20347] Updated weights for policy 0, policy_version 1361981 (0.0007) [2023-12-27 15:05:05,950][20347] Updated weights for policy 0, policy_version 1361991 (0.0007) [2023-12-27 15:05:06,004][20347] Updated weights for policy 0, policy_version 1362001 (0.0008) [2023-12-27 15:05:06,539][20381] Updated weights for policy 1, policy_version 1360888 (0.0007) [2023-12-27 15:05:06,594][20381] Updated weights for policy 1, policy_version 1360898 (0.0008) [2023-12-27 15:05:06,658][20381] Updated weights for policy 1, policy_version 1360908 (0.0008) [2023-12-27 15:05:06,740][20347] Updated weights for policy 0, policy_version 1362011 (0.0008) [2023-12-27 15:05:06,810][20347] Updated weights for policy 0, policy_version 1362021 (0.0008) [2023-12-27 15:05:06,864][20347] Updated weights for policy 0, policy_version 1362031 (0.0005) [2023-12-27 15:05:07,321][20381] Updated weights for policy 1, policy_version 1360918 (0.0008) [2023-12-27 15:05:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 697171968. Throughput: 0: 10342.9, 1: 10066.7. Samples: 697146332. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:07,363][19170] Avg episode reward: [(0, '980.440'), (1, '960.920')] [2023-12-27 15:05:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001362040_348749824.pth... [2023-12-27 15:05:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001360856_348446720.pth [2023-12-27 15:05:07,381][20381] Updated weights for policy 1, policy_version 1360928 (0.0008) [2023-12-27 15:05:07,445][20381] Updated weights for policy 1, policy_version 1360938 (0.0005) [2023-12-27 15:05:07,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001360944_348430336.pth... [2023-12-27 15:05:07,486][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001359760_348127232.pth [2023-12-27 15:05:07,496][20347] Updated weights for policy 0, policy_version 1362041 (0.0007) [2023-12-27 15:05:07,553][20347] Updated weights for policy 0, policy_version 1362051 (0.0008) [2023-12-27 15:05:07,607][20347] Updated weights for policy 0, policy_version 1362061 (0.0007) [2023-12-27 15:05:07,663][20347] Updated weights for policy 0, policy_version 1362071 (0.0007) [2023-12-27 15:05:08,052][20381] Updated weights for policy 1, policy_version 1360948 (0.0005) [2023-12-27 15:05:08,108][20381] Updated weights for policy 1, policy_version 1360958 (0.0007) [2023-12-27 15:05:08,156][20381] Updated weights for policy 1, policy_version 1360968 (0.0007) [2023-12-27 15:05:08,324][20347] Updated weights for policy 0, policy_version 1362081 (0.0010) [2023-12-27 15:05:08,389][20347] Updated weights for policy 0, policy_version 1362091 (0.0010) [2023-12-27 15:05:08,446][20347] Updated weights for policy 0, policy_version 1362101 (0.0011) [2023-12-27 15:05:08,864][20381] Updated weights for policy 1, policy_version 1360978 (0.0008) [2023-12-27 15:05:08,923][20381] Updated weights for policy 1, policy_version 1360988 (0.0009) [2023-12-27 15:05:08,983][20381] Updated weights for policy 1, policy_version 1360998 (0.0006) [2023-12-27 15:05:09,039][20381] Updated weights for policy 1, policy_version 1361008 (0.0008) [2023-12-27 15:05:09,108][20347] Updated weights for policy 0, policy_version 1362111 (0.0008) [2023-12-27 15:05:09,157][20347] Updated weights for policy 0, policy_version 1362121 (0.0008) [2023-12-27 15:05:09,211][20347] Updated weights for policy 0, policy_version 1362131 (0.0007) [2023-12-27 15:05:09,800][20381] Updated weights for policy 1, policy_version 1361018 (0.0007) [2023-12-27 15:05:09,865][20381] Updated weights for policy 1, policy_version 1361028 (0.0008) [2023-12-27 15:05:09,925][20381] Updated weights for policy 1, policy_version 1361038 (0.0007) [2023-12-27 15:05:09,931][20347] Updated weights for policy 0, policy_version 1362141 (0.0009) [2023-12-27 15:05:09,947][20291] KL-divergence is very high: 101.8446 [2023-12-27 15:05:09,977][20291] KL-divergence is very high: 156.8042 [2023-12-27 15:05:09,990][20347] Updated weights for policy 0, policy_version 1362151 (0.0010) [2023-12-27 15:05:09,996][20291] KL-divergence is very high: 106.4725 [2023-12-27 15:05:10,053][20347] Updated weights for policy 0, policy_version 1362161 (0.0011) [2023-12-27 15:05:10,692][20381] Updated weights for policy 1, policy_version 1361048 (0.0008) [2023-12-27 15:05:10,700][20347] Updated weights for policy 0, policy_version 1362171 (0.0009) [2023-12-27 15:05:10,739][20381] Updated weights for policy 1, policy_version 1361058 (0.0007) [2023-12-27 15:05:10,750][20347] Updated weights for policy 0, policy_version 1362181 (0.0005) [2023-12-27 15:05:10,786][20381] Updated weights for policy 1, policy_version 1361068 (0.0006) [2023-12-27 15:05:10,803][20347] Updated weights for policy 0, policy_version 1362191 (0.0006) [2023-12-27 15:05:11,498][20381] Updated weights for policy 1, policy_version 1361078 (0.0007) [2023-12-27 15:05:11,525][20347] Updated weights for policy 0, policy_version 1362201 (0.0007) [2023-12-27 15:05:11,550][20381] Updated weights for policy 1, policy_version 1361088 (0.0007) [2023-12-27 15:05:11,585][20347] Updated weights for policy 0, policy_version 1362211 (0.0008) [2023-12-27 15:05:11,607][20381] Updated weights for policy 1, policy_version 1361098 (0.0006) [2023-12-27 15:05:11,642][20347] Updated weights for policy 0, policy_version 1362221 (0.0008) [2023-12-27 15:05:11,704][20347] Updated weights for policy 0, policy_version 1362231 (0.0008) [2023-12-27 15:05:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 697270272. Throughput: 0: 10314.4, 1: 10074.7. Samples: 697266620. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:12,363][19170] Avg episode reward: [(0, '960.870'), (1, '951.180')] [2023-12-27 15:05:12,373][20381] Updated weights for policy 1, policy_version 1361108 (0.0008) [2023-12-27 15:05:12,431][20381] Updated weights for policy 1, policy_version 1361118 (0.0007) [2023-12-27 15:05:12,481][20347] Updated weights for policy 0, policy_version 1362241 (0.0010) [2023-12-27 15:05:12,491][20381] Updated weights for policy 1, policy_version 1361128 (0.0006) [2023-12-27 15:05:12,537][20347] Updated weights for policy 0, policy_version 1362251 (0.0011) [2023-12-27 15:05:12,598][20347] Updated weights for policy 0, policy_version 1362261 (0.0009) [2023-12-27 15:05:13,222][20381] Updated weights for policy 1, policy_version 1361138 (0.0006) [2023-12-27 15:05:13,269][20347] Updated weights for policy 0, policy_version 1362271 (0.0009) [2023-12-27 15:05:13,278][20381] Updated weights for policy 1, policy_version 1361148 (0.0006) [2023-12-27 15:05:13,320][20347] Updated weights for policy 0, policy_version 1362281 (0.0010) [2023-12-27 15:05:13,333][20381] Updated weights for policy 1, policy_version 1361158 (0.0005) [2023-12-27 15:05:13,369][20347] Updated weights for policy 0, policy_version 1362291 (0.0010) [2023-12-27 15:05:13,393][20381] Updated weights for policy 1, policy_version 1361168 (0.0005) [2023-12-27 15:05:14,011][20381] Updated weights for policy 1, policy_version 1361178 (0.0010) [2023-12-27 15:05:14,063][20381] Updated weights for policy 1, policy_version 1361188 (0.0010) [2023-12-27 15:05:14,083][20347] Updated weights for policy 0, policy_version 1362301 (0.0010) [2023-12-27 15:05:14,122][20381] Updated weights for policy 1, policy_version 1361198 (0.0011) [2023-12-27 15:05:14,142][20347] Updated weights for policy 0, policy_version 1362311 (0.0006) [2023-12-27 15:05:14,199][20347] Updated weights for policy 0, policy_version 1362321 (0.0005) [2023-12-27 15:05:14,780][20347] Updated weights for policy 0, policy_version 1362331 (0.0006) [2023-12-27 15:05:14,823][20381] Updated weights for policy 1, policy_version 1361208 (0.0008) [2023-12-27 15:05:14,842][20347] Updated weights for policy 0, policy_version 1362341 (0.0010) [2023-12-27 15:05:14,878][20381] Updated weights for policy 1, policy_version 1361218 (0.0007) [2023-12-27 15:05:14,898][20347] Updated weights for policy 0, policy_version 1362351 (0.0010) [2023-12-27 15:05:14,934][20381] Updated weights for policy 1, policy_version 1361228 (0.0006) [2023-12-27 15:05:15,610][20347] Updated weights for policy 0, policy_version 1362361 (0.0010) [2023-12-27 15:05:15,652][20381] Updated weights for policy 1, policy_version 1361238 (0.0007) [2023-12-27 15:05:15,665][20347] Updated weights for policy 0, policy_version 1362371 (0.0010) [2023-12-27 15:05:15,696][20381] Updated weights for policy 1, policy_version 1361248 (0.0005) [2023-12-27 15:05:15,719][20347] Updated weights for policy 0, policy_version 1362381 (0.0010) [2023-12-27 15:05:15,741][20381] Updated weights for policy 1, policy_version 1361258 (0.0007) [2023-12-27 15:05:15,755][20292] KL-divergence is very high: 125.6397 [2023-12-27 15:05:15,773][20347] Updated weights for policy 0, policy_version 1362391 (0.0010) [2023-12-27 15:05:16,453][20381] Updated weights for policy 1, policy_version 1361268 (0.0007) [2023-12-27 15:05:16,464][20347] Updated weights for policy 0, policy_version 1362401 (0.0010) [2023-12-27 15:05:16,503][20381] Updated weights for policy 1, policy_version 1361278 (0.0006) [2023-12-27 15:05:16,519][20347] Updated weights for policy 0, policy_version 1362411 (0.0010) [2023-12-27 15:05:16,554][20381] Updated weights for policy 1, policy_version 1361288 (0.0005) [2023-12-27 15:05:16,573][20347] Updated weights for policy 0, policy_version 1362421 (0.0010) [2023-12-27 15:05:17,252][20347] Updated weights for policy 0, policy_version 1362431 (0.0010) [2023-12-27 15:05:17,258][20381] Updated weights for policy 1, policy_version 1361298 (0.0006) [2023-12-27 15:05:17,303][20381] Updated weights for policy 1, policy_version 1361308 (0.0008) [2023-12-27 15:05:17,310][20347] Updated weights for policy 0, policy_version 1362441 (0.0010) [2023-12-27 15:05:17,358][20381] Updated weights for policy 1, policy_version 1361318 (0.0010) [2023-12-27 15:05:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20206.9, 300 sec: 20438.4). Total num frames: 697368576. Throughput: 0: 10226.1, 1: 10013.9. Samples: 697386424. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:17,364][19170] Avg episode reward: [(0, '921.850'), (1, '912.070')] [2023-12-27 15:05:17,364][20347] Updated weights for policy 0, policy_version 1362451 (0.0010) [2023-12-27 15:05:17,411][20381] Updated weights for policy 1, policy_version 1361328 (0.0007) [2023-12-27 15:05:18,044][20347] Updated weights for policy 0, policy_version 1362461 (0.0009) [2023-12-27 15:05:18,098][20347] Updated weights for policy 0, policy_version 1362471 (0.0006) [2023-12-27 15:05:18,111][20381] Updated weights for policy 1, policy_version 1361338 (0.0005) [2023-12-27 15:05:18,156][20347] Updated weights for policy 0, policy_version 1362481 (0.0007) [2023-12-27 15:05:18,162][20381] Updated weights for policy 1, policy_version 1361348 (0.0005) [2023-12-27 15:05:18,217][20381] Updated weights for policy 1, policy_version 1361358 (0.0006) [2023-12-27 15:05:18,836][20347] Updated weights for policy 0, policy_version 1362491 (0.0008) [2023-12-27 15:05:18,845][20381] Updated weights for policy 1, policy_version 1361368 (0.0007) [2023-12-27 15:05:18,893][20347] Updated weights for policy 0, policy_version 1362501 (0.0007) [2023-12-27 15:05:18,900][20381] Updated weights for policy 1, policy_version 1361378 (0.0005) [2023-12-27 15:05:18,953][20347] Updated weights for policy 0, policy_version 1362511 (0.0008) [2023-12-27 15:05:18,959][20381] Updated weights for policy 1, policy_version 1361388 (0.0007) [2023-12-27 15:05:19,605][20347] Updated weights for policy 0, policy_version 1362521 (0.0008) [2023-12-27 15:05:19,658][20347] Updated weights for policy 0, policy_version 1362531 (0.0008) [2023-12-27 15:05:19,698][20381] Updated weights for policy 1, policy_version 1361398 (0.0007) [2023-12-27 15:05:19,719][20347] Updated weights for policy 0, policy_version 1362541 (0.0007) [2023-12-27 15:05:19,759][20381] Updated weights for policy 1, policy_version 1361408 (0.0006) [2023-12-27 15:05:19,777][20347] Updated weights for policy 0, policy_version 1362551 (0.0005) [2023-12-27 15:05:19,819][20381] Updated weights for policy 1, policy_version 1361418 (0.0007) [2023-12-27 15:05:20,438][20347] Updated weights for policy 0, policy_version 1362561 (0.0007) [2023-12-27 15:05:20,495][20347] Updated weights for policy 0, policy_version 1362571 (0.0007) [2023-12-27 15:05:20,542][20381] Updated weights for policy 1, policy_version 1361428 (0.0007) [2023-12-27 15:05:20,555][20347] Updated weights for policy 0, policy_version 1362581 (0.0007) [2023-12-27 15:05:20,609][20381] Updated weights for policy 1, policy_version 1361438 (0.0008) [2023-12-27 15:05:20,676][20381] Updated weights for policy 1, policy_version 1361448 (0.0006) [2023-12-27 15:05:21,302][20347] Updated weights for policy 0, policy_version 1362591 (0.0008) [2023-12-27 15:05:21,343][20381] Updated weights for policy 1, policy_version 1361458 (0.0008) [2023-12-27 15:05:21,369][20347] Updated weights for policy 0, policy_version 1362601 (0.0007) [2023-12-27 15:05:21,417][20381] Updated weights for policy 1, policy_version 1361468 (0.0008) [2023-12-27 15:05:21,434][20347] Updated weights for policy 0, policy_version 1362611 (0.0008) [2023-12-27 15:05:21,477][20381] Updated weights for policy 1, policy_version 1361478 (0.0008) [2023-12-27 15:05:21,531][20381] Updated weights for policy 1, policy_version 1361488 (0.0009) [2023-12-27 15:05:22,162][20347] Updated weights for policy 0, policy_version 1362621 (0.0008) [2023-12-27 15:05:22,225][20347] Updated weights for policy 0, policy_version 1362631 (0.0007) [2023-12-27 15:05:22,293][20347] Updated weights for policy 0, policy_version 1362641 (0.0009) [2023-12-27 15:05:22,354][20381] Updated weights for policy 1, policy_version 1361498 (0.0009) [2023-12-27 15:05:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 697475072. Throughput: 0: 10221.8, 1: 9940.8. Samples: 697447064. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:22,363][19170] Avg episode reward: [(0, '931.670'), (1, '970.710')] [2023-12-27 15:05:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001362648_348905472.pth... [2023-12-27 15:05:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001361464_348602368.pth [2023-12-27 15:05:22,424][20381] Updated weights for policy 1, policy_version 1361508 (0.0008) [2023-12-27 15:05:22,486][20381] Updated weights for policy 1, policy_version 1361518 (0.0007) [2023-12-27 15:05:22,496][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001361520_348577792.pth... [2023-12-27 15:05:22,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001360336_348274688.pth [2023-12-27 15:05:22,998][20347] Updated weights for policy 0, policy_version 1362651 (0.0009) [2023-12-27 15:05:23,051][20347] Updated weights for policy 0, policy_version 1362661 (0.0007) [2023-12-27 15:05:23,105][20347] Updated weights for policy 0, policy_version 1362671 (0.0005) [2023-12-27 15:05:23,220][20381] Updated weights for policy 1, policy_version 1361528 (0.0007) [2023-12-27 15:05:23,270][20381] Updated weights for policy 1, policy_version 1361538 (0.0008) [2023-12-27 15:05:23,331][20381] Updated weights for policy 1, policy_version 1361548 (0.0009) [2023-12-27 15:05:23,718][20347] Updated weights for policy 0, policy_version 1362681 (0.0006) [2023-12-27 15:05:23,773][20347] Updated weights for policy 0, policy_version 1362691 (0.0008) [2023-12-27 15:05:23,829][20347] Updated weights for policy 0, policy_version 1362701 (0.0008) [2023-12-27 15:05:23,885][20347] Updated weights for policy 0, policy_version 1362711 (0.0008) [2023-12-27 15:05:23,995][20381] Updated weights for policy 1, policy_version 1361558 (0.0008) [2023-12-27 15:05:24,054][20381] Updated weights for policy 1, policy_version 1361568 (0.0008) [2023-12-27 15:05:24,119][20381] Updated weights for policy 1, policy_version 1361578 (0.0009) [2023-12-27 15:05:24,497][20347] Updated weights for policy 0, policy_version 1362721 (0.0006) [2023-12-27 15:05:24,557][20347] Updated weights for policy 0, policy_version 1362731 (0.0005) [2023-12-27 15:05:24,620][20347] Updated weights for policy 0, policy_version 1362741 (0.0005) [2023-12-27 15:05:24,865][20381] Updated weights for policy 1, policy_version 1361588 (0.0007) [2023-12-27 15:05:24,927][20381] Updated weights for policy 1, policy_version 1361598 (0.0007) [2023-12-27 15:05:24,989][20381] Updated weights for policy 1, policy_version 1361608 (0.0008) [2023-12-27 15:05:25,183][20347] Updated weights for policy 0, policy_version 1362751 (0.0009) [2023-12-27 15:05:25,235][20347] Updated weights for policy 0, policy_version 1362761 (0.0010) [2023-12-27 15:05:25,290][20347] Updated weights for policy 0, policy_version 1362771 (0.0010) [2023-12-27 15:05:25,688][20381] Updated weights for policy 1, policy_version 1361618 (0.0007) [2023-12-27 15:05:25,737][20381] Updated weights for policy 1, policy_version 1361628 (0.0008) [2023-12-27 15:05:25,790][20381] Updated weights for policy 1, policy_version 1361638 (0.0009) [2023-12-27 15:05:25,900][20347] Updated weights for policy 0, policy_version 1362781 (0.0008) [2023-12-27 15:05:25,957][20347] Updated weights for policy 0, policy_version 1362791 (0.0005) [2023-12-27 15:05:25,958][20291] KL-divergence is very high: 553.6352 [2023-12-27 15:05:26,004][20291] KL-divergence is very high: 406.0367 [2023-12-27 15:05:26,014][20347] Updated weights for policy 0, policy_version 1362801 (0.0005) [2023-12-27 15:05:26,384][20381] Updated weights for policy 1, policy_version 1361649 (0.0008) [2023-12-27 15:05:26,430][20381] Updated weights for policy 1, policy_version 1361659 (0.0005) [2023-12-27 15:05:26,473][20381] Updated weights for policy 1, policy_version 1361669 (0.0005) [2023-12-27 15:05:26,516][20381] Updated weights for policy 1, policy_version 1361679 (0.0005) [2023-12-27 15:05:26,607][20347] Updated weights for policy 0, policy_version 1362811 (0.0005) [2023-12-27 15:05:26,662][20347] Updated weights for policy 0, policy_version 1362821 (0.0005) [2023-12-27 15:05:26,713][20347] Updated weights for policy 0, policy_version 1362831 (0.0005) [2023-12-27 15:05:27,156][20381] Updated weights for policy 1, policy_version 1361689 (0.0010) [2023-12-27 15:05:27,200][20381] Updated weights for policy 1, policy_version 1361699 (0.0009) [2023-12-27 15:05:27,255][20381] Updated weights for policy 1, policy_version 1361709 (0.0005) [2023-12-27 15:05:27,299][20347] Updated weights for policy 0, policy_version 1362841 (0.0006) [2023-12-27 15:05:27,353][20347] Updated weights for policy 0, policy_version 1362851 (0.0010) [2023-12-27 15:05:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 697581568. Throughput: 0: 10290.9, 1: 9940.9. Samples: 697568940. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:27,363][19170] Avg episode reward: [(0, '931.550'), (1, '990.240')] [2023-12-27 15:05:27,406][20347] Updated weights for policy 0, policy_version 1362861 (0.0006) [2023-12-27 15:05:27,486][20347] Updated weights for policy 0, policy_version 1362871 (0.0008) [2023-12-27 15:05:27,873][20381] Updated weights for policy 1, policy_version 1361719 (0.0005) [2023-12-27 15:05:27,928][20381] Updated weights for policy 1, policy_version 1361729 (0.0007) [2023-12-27 15:05:27,979][20381] Updated weights for policy 1, policy_version 1361739 (0.0010) [2023-12-27 15:05:28,036][20347] Updated weights for policy 0, policy_version 1362881 (0.0007) [2023-12-27 15:05:28,083][20347] Updated weights for policy 0, policy_version 1362891 (0.0010) [2023-12-27 15:05:28,131][20347] Updated weights for policy 0, policy_version 1362901 (0.0010) [2023-12-27 15:05:28,612][20381] Updated weights for policy 1, policy_version 1361749 (0.0009) [2023-12-27 15:05:28,669][20381] Updated weights for policy 1, policy_version 1361759 (0.0007) [2023-12-27 15:05:28,729][20381] Updated weights for policy 1, policy_version 1361769 (0.0007) [2023-12-27 15:05:28,842][20347] Updated weights for policy 0, policy_version 1362911 (0.0008) [2023-12-27 15:05:28,893][20347] Updated weights for policy 0, policy_version 1362921 (0.0008) [2023-12-27 15:05:28,942][20347] Updated weights for policy 0, policy_version 1362931 (0.0008) [2023-12-27 15:05:29,371][20381] Updated weights for policy 1, policy_version 1361779 (0.0006) [2023-12-27 15:05:29,428][20381] Updated weights for policy 1, policy_version 1361789 (0.0007) [2023-12-27 15:05:29,491][20381] Updated weights for policy 1, policy_version 1361799 (0.0007) [2023-12-27 15:05:29,659][20347] Updated weights for policy 0, policy_version 1362941 (0.0006) [2023-12-27 15:05:29,712][20347] Updated weights for policy 0, policy_version 1362951 (0.0005) [2023-12-27 15:05:29,771][20347] Updated weights for policy 0, policy_version 1362961 (0.0005) [2023-12-27 15:05:30,158][20381] Updated weights for policy 1, policy_version 1361809 (0.0007) [2023-12-27 15:05:30,212][20381] Updated weights for policy 1, policy_version 1361819 (0.0006) [2023-12-27 15:05:30,275][20381] Updated weights for policy 1, policy_version 1361829 (0.0006) [2023-12-27 15:05:30,331][20381] Updated weights for policy 1, policy_version 1361839 (0.0007) [2023-12-27 15:05:30,495][20347] Updated weights for policy 0, policy_version 1362971 (0.0008) [2023-12-27 15:05:30,544][20347] Updated weights for policy 0, policy_version 1362981 (0.0009) [2023-12-27 15:05:30,606][20347] Updated weights for policy 0, policy_version 1362991 (0.0007) [2023-12-27 15:05:30,924][20381] Updated weights for policy 1, policy_version 1361849 (0.0007) [2023-12-27 15:05:30,978][20381] Updated weights for policy 1, policy_version 1361859 (0.0006) [2023-12-27 15:05:31,034][20381] Updated weights for policy 1, policy_version 1361869 (0.0006) [2023-12-27 15:05:31,204][20347] Updated weights for policy 0, policy_version 1363001 (0.0005) [2023-12-27 15:05:31,266][20347] Updated weights for policy 0, policy_version 1363011 (0.0007) [2023-12-27 15:05:31,316][20347] Updated weights for policy 0, policy_version 1363021 (0.0006) [2023-12-27 15:05:31,372][20347] Updated weights for policy 0, policy_version 1363031 (0.0008) [2023-12-27 15:05:31,688][20381] Updated weights for policy 1, policy_version 1361879 (0.0007) [2023-12-27 15:05:31,750][20381] Updated weights for policy 1, policy_version 1361889 (0.0008) [2023-12-27 15:05:31,808][20381] Updated weights for policy 1, policy_version 1361899 (0.0006) [2023-12-27 15:05:32,093][20347] Updated weights for policy 0, policy_version 1363041 (0.0010) [2023-12-27 15:05:32,152][20347] Updated weights for policy 0, policy_version 1363051 (0.0010) [2023-12-27 15:05:32,167][20291] KL-divergence is very high: 168.6262 [2023-12-27 15:05:32,216][20291] KL-divergence is very high: 175.9193 [2023-12-27 15:05:32,218][20347] Updated weights for policy 0, policy_version 1363061 (0.0010) [2023-12-27 15:05:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 697688064. Throughput: 0: 10350.7, 1: 10014.0. Samples: 697698520. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:32,363][19170] Avg episode reward: [(0, '921.760'), (1, '990.240')] [2023-12-27 15:05:32,446][20381] Updated weights for policy 1, policy_version 1361909 (0.0007) [2023-12-27 15:05:32,509][20381] Updated weights for policy 1, policy_version 1361919 (0.0007) [2023-12-27 15:05:32,567][20381] Updated weights for policy 1, policy_version 1361929 (0.0007) [2023-12-27 15:05:32,901][20347] Updated weights for policy 0, policy_version 1363071 (0.0007) [2023-12-27 15:05:32,946][20347] Updated weights for policy 0, policy_version 1363081 (0.0005) [2023-12-27 15:05:32,989][20347] Updated weights for policy 0, policy_version 1363091 (0.0009) [2023-12-27 15:05:33,169][20381] Updated weights for policy 1, policy_version 1361939 (0.0006) [2023-12-27 15:05:33,227][20381] Updated weights for policy 1, policy_version 1361949 (0.0006) [2023-12-27 15:05:33,280][20381] Updated weights for policy 1, policy_version 1361959 (0.0007) [2023-12-27 15:05:33,675][20347] Updated weights for policy 0, policy_version 1363101 (0.0008) [2023-12-27 15:05:33,721][20347] Updated weights for policy 0, policy_version 1363111 (0.0005) [2023-12-27 15:05:33,766][20347] Updated weights for policy 0, policy_version 1363121 (0.0005) [2023-12-27 15:05:33,888][20381] Updated weights for policy 1, policy_version 1361969 (0.0008) [2023-12-27 15:05:33,937][20381] Updated weights for policy 1, policy_version 1361979 (0.0008) [2023-12-27 15:05:33,986][20381] Updated weights for policy 1, policy_version 1361989 (0.0005) [2023-12-27 15:05:34,031][20381] Updated weights for policy 1, policy_version 1361999 (0.0009) [2023-12-27 15:05:34,332][20347] Updated weights for policy 0, policy_version 1363131 (0.0007) [2023-12-27 15:05:34,389][20347] Updated weights for policy 0, policy_version 1363141 (0.0010) [2023-12-27 15:05:34,447][20347] Updated weights for policy 0, policy_version 1363151 (0.0010) [2023-12-27 15:05:34,754][20381] Updated weights for policy 1, policy_version 1362009 (0.0008) [2023-12-27 15:05:34,804][20381] Updated weights for policy 1, policy_version 1362019 (0.0008) [2023-12-27 15:05:34,856][20381] Updated weights for policy 1, policy_version 1362029 (0.0005) [2023-12-27 15:05:35,147][20347] Updated weights for policy 0, policy_version 1363161 (0.0009) [2023-12-27 15:05:35,192][20347] Updated weights for policy 0, policy_version 1363171 (0.0008) [2023-12-27 15:05:35,249][20347] Updated weights for policy 0, policy_version 1363181 (0.0007) [2023-12-27 15:05:35,297][20347] Updated weights for policy 0, policy_version 1363191 (0.0007) [2023-12-27 15:05:35,498][20381] Updated weights for policy 1, policy_version 1362039 (0.0008) [2023-12-27 15:05:35,548][20381] Updated weights for policy 1, policy_version 1362049 (0.0009) [2023-12-27 15:05:35,597][20381] Updated weights for policy 1, policy_version 1362059 (0.0007) [2023-12-27 15:05:35,866][20347] Updated weights for policy 0, policy_version 1363201 (0.0008) [2023-12-27 15:05:35,917][20347] Updated weights for policy 0, policy_version 1363211 (0.0010) [2023-12-27 15:05:35,966][20347] Updated weights for policy 0, policy_version 1363221 (0.0010) [2023-12-27 15:05:36,281][20381] Updated weights for policy 1, policy_version 1362069 (0.0009) [2023-12-27 15:05:36,342][20381] Updated weights for policy 1, policy_version 1362079 (0.0007) [2023-12-27 15:05:36,404][20381] Updated weights for policy 1, policy_version 1362089 (0.0009) [2023-12-27 15:05:36,635][20347] Updated weights for policy 0, policy_version 1363231 (0.0006) [2023-12-27 15:05:36,697][20347] Updated weights for policy 0, policy_version 1363241 (0.0007) [2023-12-27 15:05:36,751][20347] Updated weights for policy 0, policy_version 1363251 (0.0007) [2023-12-27 15:05:37,146][20381] Updated weights for policy 1, policy_version 1362099 (0.0008) [2023-12-27 15:05:37,205][20381] Updated weights for policy 1, policy_version 1362109 (0.0007) [2023-12-27 15:05:37,261][20381] Updated weights for policy 1, policy_version 1362119 (0.0006) [2023-12-27 15:05:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 697794560. Throughput: 0: 10294.2, 1: 10089.0. Samples: 697762648. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:37,363][19170] Avg episode reward: [(0, '921.690'), (1, '961.020')] [2023-12-27 15:05:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001363256_349061120.pth... [2023-12-27 15:05:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001362128_348733440.pth... [2023-12-27 15:05:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001362040_348749824.pth [2023-12-27 15:05:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001360944_348430336.pth [2023-12-27 15:05:37,422][20347] Updated weights for policy 0, policy_version 1363261 (0.0008) [2023-12-27 15:05:37,480][20347] Updated weights for policy 0, policy_version 1363271 (0.0007) [2023-12-27 15:05:37,549][20347] Updated weights for policy 0, policy_version 1363281 (0.0006) [2023-12-27 15:05:37,844][20381] Updated weights for policy 1, policy_version 1362129 (0.0005) [2023-12-27 15:05:37,890][20381] Updated weights for policy 1, policy_version 1362139 (0.0005) [2023-12-27 15:05:37,944][20381] Updated weights for policy 1, policy_version 1362149 (0.0005) [2023-12-27 15:05:38,003][20381] Updated weights for policy 1, policy_version 1362159 (0.0005) [2023-12-27 15:05:38,086][20347] Updated weights for policy 0, policy_version 1363291 (0.0006) [2023-12-27 15:05:38,141][20347] Updated weights for policy 0, policy_version 1363301 (0.0007) [2023-12-27 15:05:38,193][20347] Updated weights for policy 0, policy_version 1363311 (0.0008) [2023-12-27 15:05:38,577][20381] Updated weights for policy 1, policy_version 1362169 (0.0007) [2023-12-27 15:05:38,633][20381] Updated weights for policy 1, policy_version 1362179 (0.0008) [2023-12-27 15:05:38,689][20381] Updated weights for policy 1, policy_version 1362189 (0.0007) [2023-12-27 15:05:38,919][20347] Updated weights for policy 0, policy_version 1363321 (0.0010) [2023-12-27 15:05:38,978][20347] Updated weights for policy 0, policy_version 1363331 (0.0008) [2023-12-27 15:05:39,038][20347] Updated weights for policy 0, policy_version 1363341 (0.0007) [2023-12-27 15:05:39,096][20347] Updated weights for policy 0, policy_version 1363351 (0.0008) [2023-12-27 15:05:39,403][20381] Updated weights for policy 1, policy_version 1362199 (0.0006) [2023-12-27 15:05:39,466][20381] Updated weights for policy 1, policy_version 1362209 (0.0006) [2023-12-27 15:05:39,518][20381] Updated weights for policy 1, policy_version 1362219 (0.0008) [2023-12-27 15:05:39,848][20347] Updated weights for policy 0, policy_version 1363361 (0.0008) [2023-12-27 15:05:39,900][20347] Updated weights for policy 0, policy_version 1363371 (0.0008) [2023-12-27 15:05:39,969][20347] Updated weights for policy 0, policy_version 1363381 (0.0009) [2023-12-27 15:05:40,257][20381] Updated weights for policy 1, policy_version 1362229 (0.0007) [2023-12-27 15:05:40,317][20381] Updated weights for policy 1, policy_version 1362239 (0.0005) [2023-12-27 15:05:40,382][20381] Updated weights for policy 1, policy_version 1362249 (0.0007) [2023-12-27 15:05:40,723][20347] Updated weights for policy 0, policy_version 1363391 (0.0008) [2023-12-27 15:05:40,787][20347] Updated weights for policy 0, policy_version 1363401 (0.0008) [2023-12-27 15:05:40,845][20347] Updated weights for policy 0, policy_version 1363411 (0.0007) [2023-12-27 15:05:41,033][20381] Updated weights for policy 1, policy_version 1362259 (0.0008) [2023-12-27 15:05:41,103][20381] Updated weights for policy 1, policy_version 1362269 (0.0008) [2023-12-27 15:05:41,178][20381] Updated weights for policy 1, policy_version 1362279 (0.0009) [2023-12-27 15:05:41,575][20347] Updated weights for policy 0, policy_version 1363421 (0.0008) [2023-12-27 15:05:41,637][20347] Updated weights for policy 0, policy_version 1363431 (0.0010) [2023-12-27 15:05:41,690][20291] KL-divergence is very high: 109.4354 [2023-12-27 15:05:41,704][20347] Updated weights for policy 0, policy_version 1363441 (0.0010) [2023-12-27 15:05:41,943][20381] Updated weights for policy 1, policy_version 1362289 (0.0007) [2023-12-27 15:05:41,996][20381] Updated weights for policy 1, policy_version 1362299 (0.0007) [2023-12-27 15:05:42,058][20381] Updated weights for policy 1, policy_version 1362309 (0.0007) [2023-12-27 15:05:42,112][20381] Updated weights for policy 1, policy_version 1362319 (0.0007) [2023-12-27 15:05:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 697892864. Throughput: 0: 10323.5, 1: 10125.8. Samples: 697885996. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:42,363][19170] Avg episode reward: [(0, '931.660'), (1, '961.010')] [2023-12-27 15:05:42,394][20347] Updated weights for policy 0, policy_version 1363451 (0.0010) [2023-12-27 15:05:42,451][20347] Updated weights for policy 0, policy_version 1363461 (0.0010) [2023-12-27 15:05:42,504][20347] Updated weights for policy 0, policy_version 1363471 (0.0010) [2023-12-27 15:05:42,870][20381] Updated weights for policy 1, policy_version 1362329 (0.0007) [2023-12-27 15:05:42,917][20381] Updated weights for policy 1, policy_version 1362339 (0.0006) [2023-12-27 15:05:42,972][20381] Updated weights for policy 1, policy_version 1362349 (0.0005) [2023-12-27 15:05:43,201][20347] Updated weights for policy 0, policy_version 1363481 (0.0010) [2023-12-27 15:05:43,251][20347] Updated weights for policy 0, policy_version 1363491 (0.0007) [2023-12-27 15:05:43,304][20347] Updated weights for policy 0, policy_version 1363501 (0.0006) [2023-12-27 15:05:43,355][20347] Updated weights for policy 0, policy_version 1363511 (0.0008) [2023-12-27 15:05:43,578][20381] Updated weights for policy 1, policy_version 1362359 (0.0006) [2023-12-27 15:05:43,632][20381] Updated weights for policy 1, policy_version 1362369 (0.0008) [2023-12-27 15:05:43,688][20381] Updated weights for policy 1, policy_version 1362379 (0.0008) [2023-12-27 15:05:44,030][20347] Updated weights for policy 0, policy_version 1363521 (0.0005) [2023-12-27 15:05:44,078][20347] Updated weights for policy 0, policy_version 1363531 (0.0005) [2023-12-27 15:05:44,129][20347] Updated weights for policy 0, policy_version 1363541 (0.0006) [2023-12-27 15:05:44,343][20381] Updated weights for policy 1, policy_version 1362389 (0.0008) [2023-12-27 15:05:44,410][20381] Updated weights for policy 1, policy_version 1362399 (0.0009) [2023-12-27 15:05:44,474][20381] Updated weights for policy 1, policy_version 1362409 (0.0007) [2023-12-27 15:05:44,798][20347] Updated weights for policy 0, policy_version 1363551 (0.0009) [2023-12-27 15:05:44,859][20347] Updated weights for policy 0, policy_version 1363561 (0.0010) [2023-12-27 15:05:44,910][20347] Updated weights for policy 0, policy_version 1363571 (0.0010) [2023-12-27 15:05:45,085][20381] Updated weights for policy 1, policy_version 1362419 (0.0008) [2023-12-27 15:05:45,129][20381] Updated weights for policy 1, policy_version 1362429 (0.0007) [2023-12-27 15:05:45,181][20381] Updated weights for policy 1, policy_version 1362439 (0.0008) [2023-12-27 15:05:45,621][20347] Updated weights for policy 0, policy_version 1363581 (0.0011) [2023-12-27 15:05:45,679][20347] Updated weights for policy 0, policy_version 1363591 (0.0010) [2023-12-27 15:05:45,730][20347] Updated weights for policy 0, policy_version 1363601 (0.0010) [2023-12-27 15:05:45,907][20381] Updated weights for policy 1, policy_version 1362449 (0.0007) [2023-12-27 15:05:45,955][20381] Updated weights for policy 1, policy_version 1362459 (0.0007) [2023-12-27 15:05:46,017][20381] Updated weights for policy 1, policy_version 1362469 (0.0007) [2023-12-27 15:05:46,068][20381] Updated weights for policy 1, policy_version 1362479 (0.0008) [2023-12-27 15:05:46,400][20347] Updated weights for policy 0, policy_version 1363611 (0.0009) [2023-12-27 15:05:46,448][20347] Updated weights for policy 0, policy_version 1363621 (0.0008) [2023-12-27 15:05:46,506][20347] Updated weights for policy 0, policy_version 1363631 (0.0007) [2023-12-27 15:05:46,772][20381] Updated weights for policy 1, policy_version 1362489 (0.0007) [2023-12-27 15:05:46,822][20381] Updated weights for policy 1, policy_version 1362499 (0.0009) [2023-12-27 15:05:46,874][20381] Updated weights for policy 1, policy_version 1362509 (0.0008) [2023-12-27 15:05:47,153][20347] Updated weights for policy 0, policy_version 1363641 (0.0006) [2023-12-27 15:05:47,208][20347] Updated weights for policy 0, policy_version 1363651 (0.0008) [2023-12-27 15:05:47,271][20347] Updated weights for policy 0, policy_version 1363661 (0.0007) [2023-12-27 15:05:47,330][20347] Updated weights for policy 0, policy_version 1363671 (0.0007) [2023-12-27 15:05:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 697999360. Throughput: 0: 10306.5, 1: 10162.2. Samples: 698007096. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:47,363][19170] Avg episode reward: [(0, '902.250'), (1, '980.480')] [2023-12-27 15:05:47,581][20381] Updated weights for policy 1, policy_version 1362519 (0.0008) [2023-12-27 15:05:47,638][20381] Updated weights for policy 1, policy_version 1362529 (0.0008) [2023-12-27 15:05:47,691][20381] Updated weights for policy 1, policy_version 1362539 (0.0007) [2023-12-27 15:05:47,992][20347] Updated weights for policy 0, policy_version 1363681 (0.0006) [2023-12-27 15:05:48,044][20347] Updated weights for policy 0, policy_version 1363691 (0.0005) [2023-12-27 15:05:48,100][20347] Updated weights for policy 0, policy_version 1363701 (0.0005) [2023-12-27 15:05:48,432][20381] Updated weights for policy 1, policy_version 1362549 (0.0008) [2023-12-27 15:05:48,495][20381] Updated weights for policy 1, policy_version 1362559 (0.0008) [2023-12-27 15:05:48,562][20381] Updated weights for policy 1, policy_version 1362569 (0.0008) [2023-12-27 15:05:48,694][20347] Updated weights for policy 0, policy_version 1363711 (0.0007) [2023-12-27 15:05:48,758][20347] Updated weights for policy 0, policy_version 1363721 (0.0007) [2023-12-27 15:05:48,826][20347] Updated weights for policy 0, policy_version 1363731 (0.0008) [2023-12-27 15:05:49,250][20381] Updated weights for policy 1, policy_version 1362579 (0.0008) [2023-12-27 15:05:49,309][20381] Updated weights for policy 1, policy_version 1362589 (0.0010) [2023-12-27 15:05:49,385][20381] Updated weights for policy 1, policy_version 1362599 (0.0008) [2023-12-27 15:05:49,465][20347] Updated weights for policy 0, policy_version 1363741 (0.0010) [2023-12-27 15:05:49,514][20347] Updated weights for policy 0, policy_version 1363751 (0.0010) [2023-12-27 15:05:49,563][20347] Updated weights for policy 0, policy_version 1363761 (0.0010) [2023-12-27 15:05:50,122][20381] Updated weights for policy 1, policy_version 1362609 (0.0008) [2023-12-27 15:05:50,181][20381] Updated weights for policy 1, policy_version 1362619 (0.0008) [2023-12-27 15:05:50,239][20381] Updated weights for policy 1, policy_version 1362629 (0.0008) [2023-12-27 15:05:50,299][20347] Updated weights for policy 0, policy_version 1363771 (0.0010) [2023-12-27 15:05:50,300][20381] Updated weights for policy 1, policy_version 1362639 (0.0008) [2023-12-27 15:05:50,352][20347] Updated weights for policy 0, policy_version 1363781 (0.0007) [2023-12-27 15:05:50,412][20347] Updated weights for policy 0, policy_version 1363791 (0.0008) [2023-12-27 15:05:51,041][20381] Updated weights for policy 1, policy_version 1362649 (0.0010) [2023-12-27 15:05:51,103][20381] Updated weights for policy 1, policy_version 1362659 (0.0011) [2023-12-27 15:05:51,170][20381] Updated weights for policy 1, policy_version 1362669 (0.0011) [2023-12-27 15:05:51,183][20347] Updated weights for policy 0, policy_version 1363801 (0.0007) [2023-12-27 15:05:51,252][20347] Updated weights for policy 0, policy_version 1363811 (0.0007) [2023-12-27 15:05:51,311][20347] Updated weights for policy 0, policy_version 1363821 (0.0007) [2023-12-27 15:05:51,377][20347] Updated weights for policy 0, policy_version 1363831 (0.0009) [2023-12-27 15:05:51,886][20381] Updated weights for policy 1, policy_version 1362679 (0.0009) [2023-12-27 15:05:51,947][20381] Updated weights for policy 1, policy_version 1362689 (0.0007) [2023-12-27 15:05:52,008][20381] Updated weights for policy 1, policy_version 1362699 (0.0008) [2023-12-27 15:05:52,034][20347] Updated weights for policy 0, policy_version 1363841 (0.0007) [2023-12-27 15:05:52,093][20347] Updated weights for policy 0, policy_version 1363851 (0.0007) [2023-12-27 15:05:52,154][20347] Updated weights for policy 0, policy_version 1363861 (0.0007) [2023-12-27 15:05:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 698097664. Throughput: 0: 10318.3, 1: 10147.1. Samples: 698067276. Policy #0 lag: (min: 20.0, avg: 21.5, max: 47.0) [2023-12-27 15:05:52,364][19170] Avg episode reward: [(0, '941.290'), (1, '990.250')] [2023-12-27 15:05:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001362704_348880896.pth... [2023-12-27 15:05:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001363864_349216768.pth... [2023-12-27 15:05:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001361520_348577792.pth [2023-12-27 15:05:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001362648_348905472.pth [2023-12-27 15:05:52,699][20381] Updated weights for policy 1, policy_version 1362709 (0.0007) [2023-12-27 15:05:52,758][20381] Updated weights for policy 1, policy_version 1362719 (0.0006) [2023-12-27 15:05:52,818][20381] Updated weights for policy 1, policy_version 1362729 (0.0008) [2023-12-27 15:05:52,861][20347] Updated weights for policy 0, policy_version 1363871 (0.0010) [2023-12-27 15:05:52,910][20347] Updated weights for policy 0, policy_version 1363881 (0.0006) [2023-12-27 15:05:52,964][20347] Updated weights for policy 0, policy_version 1363891 (0.0005) [2023-12-27 15:05:53,492][20381] Updated weights for policy 1, policy_version 1362739 (0.0006) [2023-12-27 15:05:53,532][20347] Updated weights for policy 0, policy_version 1363901 (0.0007) [2023-12-27 15:05:53,550][20381] Updated weights for policy 1, policy_version 1362749 (0.0007) [2023-12-27 15:05:53,585][20347] Updated weights for policy 0, policy_version 1363911 (0.0010) [2023-12-27 15:05:53,598][20381] Updated weights for policy 1, policy_version 1362759 (0.0005) [2023-12-27 15:05:53,633][20347] Updated weights for policy 0, policy_version 1363921 (0.0010) [2023-12-27 15:05:54,208][20381] Updated weights for policy 1, policy_version 1362769 (0.0008) [2023-12-27 15:05:54,260][20347] Updated weights for policy 0, policy_version 1363931 (0.0009) [2023-12-27 15:05:54,264][20381] Updated weights for policy 1, policy_version 1362779 (0.0011) [2023-12-27 15:05:54,312][20347] Updated weights for policy 0, policy_version 1363941 (0.0005) [2023-12-27 15:05:54,320][20381] Updated weights for policy 1, policy_version 1362789 (0.0011) [2023-12-27 15:05:54,373][20347] Updated weights for policy 0, policy_version 1363951 (0.0005) [2023-12-27 15:05:54,374][20381] Updated weights for policy 1, policy_version 1362799 (0.0011) [2023-12-27 15:05:54,917][20347] Updated weights for policy 0, policy_version 1363961 (0.0005) [2023-12-27 15:05:54,975][20347] Updated weights for policy 0, policy_version 1363971 (0.0006) [2023-12-27 15:05:55,040][20347] Updated weights for policy 0, policy_version 1363981 (0.0008) [2023-12-27 15:05:55,044][20381] Updated weights for policy 1, policy_version 1362809 (0.0008) [2023-12-27 15:05:55,097][20347] Updated weights for policy 0, policy_version 1363991 (0.0006) [2023-12-27 15:05:55,102][20381] Updated weights for policy 1, policy_version 1362819 (0.0008) [2023-12-27 15:05:55,160][20381] Updated weights for policy 1, policy_version 1362829 (0.0007) [2023-12-27 15:05:55,775][20347] Updated weights for policy 0, policy_version 1364001 (0.0010) [2023-12-27 15:05:55,830][20347] Updated weights for policy 0, policy_version 1364011 (0.0010) [2023-12-27 15:05:55,837][20381] Updated weights for policy 1, policy_version 1362839 (0.0010) [2023-12-27 15:05:55,884][20347] Updated weights for policy 0, policy_version 1364021 (0.0010) [2023-12-27 15:05:55,890][20381] Updated weights for policy 1, policy_version 1362849 (0.0007) [2023-12-27 15:05:55,947][20381] Updated weights for policy 1, policy_version 1362859 (0.0005) [2023-12-27 15:05:56,522][20347] Updated weights for policy 0, policy_version 1364031 (0.0007) [2023-12-27 15:05:56,576][20347] Updated weights for policy 0, policy_version 1364041 (0.0005) [2023-12-27 15:05:56,592][20381] Updated weights for policy 1, policy_version 1362869 (0.0005) [2023-12-27 15:05:56,621][20347] Updated weights for policy 0, policy_version 1364051 (0.0005) [2023-12-27 15:05:56,645][20381] Updated weights for policy 1, policy_version 1362879 (0.0005) [2023-12-27 15:05:56,695][20381] Updated weights for policy 1, policy_version 1362889 (0.0005) [2023-12-27 15:05:57,184][20347] Updated weights for policy 0, policy_version 1364061 (0.0005) [2023-12-27 15:05:57,228][20347] Updated weights for policy 0, policy_version 1364071 (0.0005) [2023-12-27 15:05:57,260][20381] Updated weights for policy 1, policy_version 1362899 (0.0007) [2023-12-27 15:05:57,280][20347] Updated weights for policy 0, policy_version 1364081 (0.0009) [2023-12-27 15:05:57,309][20381] Updated weights for policy 1, policy_version 1362909 (0.0010) [2023-12-27 15:05:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 698204160. Throughput: 0: 10367.9, 1: 10183.7. Samples: 698191444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:05:57,363][19170] Avg episode reward: [(0, '931.710'), (1, '990.240')] [2023-12-27 15:05:57,368][20381] Updated weights for policy 1, policy_version 1362919 (0.0010) [2023-12-27 15:05:57,909][20347] Updated weights for policy 0, policy_version 1364091 (0.0007) [2023-12-27 15:05:57,964][20347] Updated weights for policy 0, policy_version 1364101 (0.0009) [2023-12-27 15:05:58,024][20347] Updated weights for policy 0, policy_version 1364111 (0.0010) [2023-12-27 15:05:58,049][20381] Updated weights for policy 1, policy_version 1362929 (0.0010) [2023-12-27 15:05:58,101][20381] Updated weights for policy 1, policy_version 1362939 (0.0007) [2023-12-27 15:05:58,145][20381] Updated weights for policy 1, policy_version 1362949 (0.0006) [2023-12-27 15:05:58,206][20381] Updated weights for policy 1, policy_version 1362959 (0.0006) [2023-12-27 15:05:58,709][20347] Updated weights for policy 0, policy_version 1364121 (0.0010) [2023-12-27 15:05:58,772][20347] Updated weights for policy 0, policy_version 1364131 (0.0008) [2023-12-27 15:05:58,838][20347] Updated weights for policy 0, policy_version 1364141 (0.0007) [2023-12-27 15:05:58,900][20347] Updated weights for policy 0, policy_version 1364151 (0.0007) [2023-12-27 15:05:59,034][20381] Updated weights for policy 1, policy_version 1362969 (0.0010) [2023-12-27 15:05:59,093][20381] Updated weights for policy 1, policy_version 1362979 (0.0011) [2023-12-27 15:05:59,159][20381] Updated weights for policy 1, policy_version 1362989 (0.0010) [2023-12-27 15:05:59,605][20347] Updated weights for policy 0, policy_version 1364161 (0.0006) [2023-12-27 15:05:59,657][20347] Updated weights for policy 0, policy_version 1364171 (0.0008) [2023-12-27 15:05:59,712][20347] Updated weights for policy 0, policy_version 1364181 (0.0006) [2023-12-27 15:05:59,883][20381] Updated weights for policy 1, policy_version 1362999 (0.0008) [2023-12-27 15:05:59,934][20381] Updated weights for policy 1, policy_version 1363009 (0.0007) [2023-12-27 15:05:59,987][20381] Updated weights for policy 1, policy_version 1363019 (0.0008) [2023-12-27 15:06:00,336][20347] Updated weights for policy 0, policy_version 1364191 (0.0007) [2023-12-27 15:06:00,393][20347] Updated weights for policy 0, policy_version 1364201 (0.0006) [2023-12-27 15:06:00,448][20347] Updated weights for policy 0, policy_version 1364211 (0.0005) [2023-12-27 15:06:00,685][20381] Updated weights for policy 1, policy_version 1363029 (0.0006) [2023-12-27 15:06:00,743][20381] Updated weights for policy 1, policy_version 1363039 (0.0005) [2023-12-27 15:06:00,798][20381] Updated weights for policy 1, policy_version 1363049 (0.0006) [2023-12-27 15:06:01,119][20347] Updated weights for policy 0, policy_version 1364221 (0.0008) [2023-12-27 15:06:01,183][20347] Updated weights for policy 0, policy_version 1364231 (0.0009) [2023-12-27 15:06:01,242][20347] Updated weights for policy 0, policy_version 1364241 (0.0007) [2023-12-27 15:06:01,398][20381] Updated weights for policy 1, policy_version 1363059 (0.0006) [2023-12-27 15:06:01,458][20381] Updated weights for policy 1, policy_version 1363069 (0.0008) [2023-12-27 15:06:01,519][20381] Updated weights for policy 1, policy_version 1363079 (0.0005) [2023-12-27 15:06:01,957][20347] Updated weights for policy 0, policy_version 1364251 (0.0008) [2023-12-27 15:06:02,010][20347] Updated weights for policy 0, policy_version 1364261 (0.0010) [2023-12-27 15:06:02,070][20347] Updated weights for policy 0, policy_version 1364271 (0.0010) [2023-12-27 15:06:02,152][20381] Updated weights for policy 1, policy_version 1363089 (0.0006) [2023-12-27 15:06:02,208][20381] Updated weights for policy 1, policy_version 1363099 (0.0010) [2023-12-27 15:06:02,264][20381] Updated weights for policy 1, policy_version 1363109 (0.0009) [2023-12-27 15:06:02,324][20381] Updated weights for policy 1, policy_version 1363119 (0.0008) [2023-12-27 15:06:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 698310656. Throughput: 0: 10443.5, 1: 10219.4. Samples: 698316252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:02,363][19170] Avg episode reward: [(0, '892.690'), (1, '990.240')] [2023-12-27 15:06:02,690][20347] Updated weights for policy 0, policy_version 1364281 (0.0009) [2023-12-27 15:06:02,755][20347] Updated weights for policy 0, policy_version 1364291 (0.0011) [2023-12-27 15:06:02,813][20347] Updated weights for policy 0, policy_version 1364301 (0.0010) [2023-12-27 15:06:02,864][20347] Updated weights for policy 0, policy_version 1364311 (0.0010) [2023-12-27 15:06:03,045][20381] Updated weights for policy 1, policy_version 1363129 (0.0007) [2023-12-27 15:06:03,088][20381] Updated weights for policy 1, policy_version 1363139 (0.0007) [2023-12-27 15:06:03,137][20381] Updated weights for policy 1, policy_version 1363149 (0.0007) [2023-12-27 15:06:03,543][20347] Updated weights for policy 0, policy_version 1364321 (0.0010) [2023-12-27 15:06:03,594][20347] Updated weights for policy 0, policy_version 1364331 (0.0010) [2023-12-27 15:06:03,652][20347] Updated weights for policy 0, policy_version 1364341 (0.0010) [2023-12-27 15:06:03,817][20381] Updated weights for policy 1, policy_version 1363159 (0.0006) [2023-12-27 15:06:03,882][20381] Updated weights for policy 1, policy_version 1363169 (0.0008) [2023-12-27 15:06:03,942][20381] Updated weights for policy 1, policy_version 1363179 (0.0006) [2023-12-27 15:06:04,339][20347] Updated weights for policy 0, policy_version 1364351 (0.0009) [2023-12-27 15:06:04,412][20347] Updated weights for policy 0, policy_version 1364361 (0.0008) [2023-12-27 15:06:04,472][20347] Updated weights for policy 0, policy_version 1364371 (0.0010) [2023-12-27 15:06:04,619][20381] Updated weights for policy 1, policy_version 1363189 (0.0008) [2023-12-27 15:06:04,671][20381] Updated weights for policy 1, policy_version 1363199 (0.0007) [2023-12-27 15:06:04,723][20381] Updated weights for policy 1, policy_version 1363209 (0.0007) [2023-12-27 15:06:05,135][20347] Updated weights for policy 0, policy_version 1364381 (0.0010) [2023-12-27 15:06:05,191][20347] Updated weights for policy 0, policy_version 1364391 (0.0010) [2023-12-27 15:06:05,252][20347] Updated weights for policy 0, policy_version 1364401 (0.0010) [2023-12-27 15:06:05,338][20381] Updated weights for policy 1, policy_version 1363219 (0.0006) [2023-12-27 15:06:05,386][20381] Updated weights for policy 1, policy_version 1363229 (0.0007) [2023-12-27 15:06:05,433][20381] Updated weights for policy 1, policy_version 1363239 (0.0007) [2023-12-27 15:06:05,917][20347] Updated weights for policy 0, policy_version 1364411 (0.0009) [2023-12-27 15:06:05,978][20347] Updated weights for policy 0, policy_version 1364421 (0.0005) [2023-12-27 15:06:06,038][20347] Updated weights for policy 0, policy_version 1364431 (0.0005) [2023-12-27 15:06:06,047][20381] Updated weights for policy 1, policy_version 1363249 (0.0005) [2023-12-27 15:06:06,112][20381] Updated weights for policy 1, policy_version 1363259 (0.0008) [2023-12-27 15:06:06,171][20381] Updated weights for policy 1, policy_version 1363269 (0.0007) [2023-12-27 15:06:06,232][20381] Updated weights for policy 1, policy_version 1363279 (0.0007) [2023-12-27 15:06:06,719][20347] Updated weights for policy 0, policy_version 1364441 (0.0006) [2023-12-27 15:06:06,780][20347] Updated weights for policy 0, policy_version 1364451 (0.0008) [2023-12-27 15:06:06,832][20347] Updated weights for policy 0, policy_version 1364461 (0.0008) [2023-12-27 15:06:06,892][20347] Updated weights for policy 0, policy_version 1364471 (0.0007) [2023-12-27 15:06:06,901][20381] Updated weights for policy 1, policy_version 1363289 (0.0005) [2023-12-27 15:06:06,952][20381] Updated weights for policy 1, policy_version 1363299 (0.0007) [2023-12-27 15:06:07,008][20381] Updated weights for policy 1, policy_version 1363309 (0.0005) [2023-12-27 15:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 698408960. Throughput: 0: 10453.2, 1: 10234.4. Samples: 698378004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:07,363][19170] Avg episode reward: [(0, '912.040'), (1, '1000.000')] [2023-12-27 15:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001364472_349372416.pth... [2023-12-27 15:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001363312_349036544.pth... [2023-12-27 15:06:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001363256_349061120.pth [2023-12-27 15:06:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001362128_348733440.pth [2023-12-27 15:06:07,597][20347] Updated weights for policy 0, policy_version 1364481 (0.0010) [2023-12-27 15:06:07,639][20381] Updated weights for policy 1, policy_version 1363319 (0.0008) [2023-12-27 15:06:07,649][20347] Updated weights for policy 0, policy_version 1364491 (0.0010) [2023-12-27 15:06:07,684][20381] Updated weights for policy 1, policy_version 1363329 (0.0005) [2023-12-27 15:06:07,694][20347] Updated weights for policy 0, policy_version 1364501 (0.0010) [2023-12-27 15:06:07,731][20381] Updated weights for policy 1, policy_version 1363339 (0.0006) [2023-12-27 15:06:08,364][20347] Updated weights for policy 0, policy_version 1364511 (0.0009) [2023-12-27 15:06:08,422][20347] Updated weights for policy 0, policy_version 1364521 (0.0008) [2023-12-27 15:06:08,442][20381] Updated weights for policy 1, policy_version 1363349 (0.0008) [2023-12-27 15:06:08,486][20347] Updated weights for policy 0, policy_version 1364531 (0.0011) [2023-12-27 15:06:08,495][20381] Updated weights for policy 1, policy_version 1363359 (0.0006) [2023-12-27 15:06:08,550][20381] Updated weights for policy 1, policy_version 1363369 (0.0008) [2023-12-27 15:06:09,172][20347] Updated weights for policy 0, policy_version 1364541 (0.0010) [2023-12-27 15:06:09,175][20381] Updated weights for policy 1, policy_version 1363379 (0.0008) [2023-12-27 15:06:09,228][20381] Updated weights for policy 1, policy_version 1363389 (0.0007) [2023-12-27 15:06:09,231][20347] Updated weights for policy 0, policy_version 1364551 (0.0008) [2023-12-27 15:06:09,293][20381] Updated weights for policy 1, policy_version 1363399 (0.0006) [2023-12-27 15:06:09,295][20347] Updated weights for policy 0, policy_version 1364561 (0.0009) [2023-12-27 15:06:09,997][20381] Updated weights for policy 1, policy_version 1363409 (0.0006) [2023-12-27 15:06:10,013][20347] Updated weights for policy 0, policy_version 1364571 (0.0011) [2023-12-27 15:06:10,062][20381] Updated weights for policy 1, policy_version 1363419 (0.0007) [2023-12-27 15:06:10,073][20347] Updated weights for policy 0, policy_version 1364581 (0.0010) [2023-12-27 15:06:10,131][20381] Updated weights for policy 1, policy_version 1363429 (0.0008) [2023-12-27 15:06:10,153][20347] Updated weights for policy 0, policy_version 1364591 (0.0009) [2023-12-27 15:06:10,178][20292] KL-divergence is very high: 107.5797 [2023-12-27 15:06:10,196][20381] Updated weights for policy 1, policy_version 1363439 (0.0007) [2023-12-27 15:06:10,207][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000008 [2023-12-27 15:06:10,762][20347] Updated weights for policy 0, policy_version 1364601 (0.0008) [2023-12-27 15:06:10,823][20347] Updated weights for policy 0, policy_version 1364611 (0.0006) [2023-12-27 15:06:10,882][20347] Updated weights for policy 0, policy_version 1364621 (0.0008) [2023-12-27 15:06:10,930][20381] Updated weights for policy 1, policy_version 1363449 (0.0008) [2023-12-27 15:06:10,936][20347] Updated weights for policy 0, policy_version 1364631 (0.0006) [2023-12-27 15:06:10,987][20381] Updated weights for policy 1, policy_version 1363459 (0.0009) [2023-12-27 15:06:11,050][20381] Updated weights for policy 1, policy_version 1363469 (0.0007) [2023-12-27 15:06:11,720][20347] Updated weights for policy 0, policy_version 1364641 (0.0008) [2023-12-27 15:06:11,788][20347] Updated weights for policy 0, policy_version 1364651 (0.0008) [2023-12-27 15:06:11,837][20381] Updated weights for policy 1, policy_version 1363479 (0.0006) [2023-12-27 15:06:11,848][20347] Updated weights for policy 0, policy_version 1364661 (0.0008) [2023-12-27 15:06:11,903][20381] Updated weights for policy 1, policy_version 1363489 (0.0008) [2023-12-27 15:06:11,961][20381] Updated weights for policy 1, policy_version 1363499 (0.0007) [2023-12-27 15:06:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 698507264. Throughput: 0: 10407.4, 1: 10282.8. Samples: 698500000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:12,363][19170] Avg episode reward: [(0, '872.920'), (1, '990.230')] [2023-12-27 15:06:12,629][20347] Updated weights for policy 0, policy_version 1364671 (0.0008) [2023-12-27 15:06:12,631][20381] Updated weights for policy 1, policy_version 1363509 (0.0007) [2023-12-27 15:06:12,688][20347] Updated weights for policy 0, policy_version 1364681 (0.0008) [2023-12-27 15:06:12,694][20381] Updated weights for policy 1, policy_version 1363519 (0.0006) [2023-12-27 15:06:12,739][20347] Updated weights for policy 0, policy_version 1364691 (0.0006) [2023-12-27 15:06:12,757][20381] Updated weights for policy 1, policy_version 1363529 (0.0007) [2023-12-27 15:06:13,393][20381] Updated weights for policy 1, policy_version 1363539 (0.0007) [2023-12-27 15:06:13,426][20347] Updated weights for policy 0, policy_version 1364701 (0.0006) [2023-12-27 15:06:13,450][20381] Updated weights for policy 1, policy_version 1363549 (0.0005) [2023-12-27 15:06:13,487][20347] Updated weights for policy 0, policy_version 1364711 (0.0008) [2023-12-27 15:06:13,507][20381] Updated weights for policy 1, policy_version 1363559 (0.0005) [2023-12-27 15:06:13,539][20347] Updated weights for policy 0, policy_version 1364721 (0.0008) [2023-12-27 15:06:14,040][20381] Updated weights for policy 1, policy_version 1363569 (0.0005) [2023-12-27 15:06:14,090][20381] Updated weights for policy 1, policy_version 1363579 (0.0008) [2023-12-27 15:06:14,137][20381] Updated weights for policy 1, policy_version 1363589 (0.0007) [2023-12-27 15:06:14,189][20347] Updated weights for policy 0, policy_version 1364731 (0.0007) [2023-12-27 15:06:14,195][20381] Updated weights for policy 1, policy_version 1363599 (0.0007) [2023-12-27 15:06:14,240][20347] Updated weights for policy 0, policy_version 1364741 (0.0007) [2023-12-27 15:06:14,283][20347] Updated weights for policy 0, policy_version 1364751 (0.0007) [2023-12-27 15:06:14,889][20381] Updated weights for policy 1, policy_version 1363609 (0.0008) [2023-12-27 15:06:14,949][20381] Updated weights for policy 1, policy_version 1363619 (0.0008) [2023-12-27 15:06:15,007][20347] Updated weights for policy 0, policy_version 1364761 (0.0007) [2023-12-27 15:06:15,017][20381] Updated weights for policy 1, policy_version 1363629 (0.0008) [2023-12-27 15:06:15,073][20347] Updated weights for policy 0, policy_version 1364771 (0.0007) [2023-12-27 15:06:15,134][20347] Updated weights for policy 0, policy_version 1364781 (0.0007) [2023-12-27 15:06:15,187][20347] Updated weights for policy 0, policy_version 1364791 (0.0008) [2023-12-27 15:06:15,702][20381] Updated weights for policy 1, policy_version 1363639 (0.0007) [2023-12-27 15:06:15,750][20381] Updated weights for policy 1, policy_version 1363649 (0.0007) [2023-12-27 15:06:15,803][20381] Updated weights for policy 1, policy_version 1363659 (0.0007) [2023-12-27 15:06:15,901][20347] Updated weights for policy 0, policy_version 1364801 (0.0008) [2023-12-27 15:06:15,961][20347] Updated weights for policy 0, policy_version 1364811 (0.0008) [2023-12-27 15:06:16,018][20347] Updated weights for policy 0, policy_version 1364821 (0.0009) [2023-12-27 15:06:16,346][20381] Updated weights for policy 1, policy_version 1363669 (0.0005) [2023-12-27 15:06:16,396][20381] Updated weights for policy 1, policy_version 1363679 (0.0005) [2023-12-27 15:06:16,452][20381] Updated weights for policy 1, policy_version 1363689 (0.0005) [2023-12-27 15:06:16,762][20347] Updated weights for policy 0, policy_version 1364831 (0.0007) [2023-12-27 15:06:16,819][20347] Updated weights for policy 0, policy_version 1364841 (0.0007) [2023-12-27 15:06:16,876][20347] Updated weights for policy 0, policy_version 1364851 (0.0008) [2023-12-27 15:06:17,062][20381] Updated weights for policy 1, policy_version 1363699 (0.0006) [2023-12-27 15:06:17,120][20381] Updated weights for policy 1, policy_version 1363709 (0.0008) [2023-12-27 15:06:17,179][20381] Updated weights for policy 1, policy_version 1363719 (0.0008) [2023-12-27 15:06:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 698613760. Throughput: 0: 10264.6, 1: 10259.6. Samples: 698622112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:17,364][19170] Avg episode reward: [(0, '872.960'), (1, '990.230')] [2023-12-27 15:06:17,563][20347] Updated weights for policy 0, policy_version 1364861 (0.0007) [2023-12-27 15:06:17,624][20347] Updated weights for policy 0, policy_version 1364871 (0.0007) [2023-12-27 15:06:17,680][20347] Updated weights for policy 0, policy_version 1364881 (0.0008) [2023-12-27 15:06:17,866][20381] Updated weights for policy 1, policy_version 1363729 (0.0008) [2023-12-27 15:06:17,925][20381] Updated weights for policy 1, policy_version 1363739 (0.0008) [2023-12-27 15:06:17,979][20381] Updated weights for policy 1, policy_version 1363749 (0.0008) [2023-12-27 15:06:18,034][20381] Updated weights for policy 1, policy_version 1363759 (0.0008) [2023-12-27 15:06:18,278][20347] Updated weights for policy 0, policy_version 1364891 (0.0008) [2023-12-27 15:06:18,332][20347] Updated weights for policy 0, policy_version 1364901 (0.0010) [2023-12-27 15:06:18,397][20347] Updated weights for policy 0, policy_version 1364911 (0.0007) [2023-12-27 15:06:18,774][20381] Updated weights for policy 1, policy_version 1363769 (0.0008) [2023-12-27 15:06:18,840][20381] Updated weights for policy 1, policy_version 1363779 (0.0008) [2023-12-27 15:06:18,901][20381] Updated weights for policy 1, policy_version 1363789 (0.0007) [2023-12-27 15:06:19,027][20347] Updated weights for policy 0, policy_version 1364921 (0.0009) [2023-12-27 15:06:19,087][20347] Updated weights for policy 0, policy_version 1364931 (0.0009) [2023-12-27 15:06:19,148][20347] Updated weights for policy 0, policy_version 1364941 (0.0010) [2023-12-27 15:06:19,216][20347] Updated weights for policy 0, policy_version 1364951 (0.0006) [2023-12-27 15:06:19,553][20381] Updated weights for policy 1, policy_version 1363799 (0.0008) [2023-12-27 15:06:19,611][20381] Updated weights for policy 1, policy_version 1363809 (0.0008) [2023-12-27 15:06:19,665][20381] Updated weights for policy 1, policy_version 1363819 (0.0007) [2023-12-27 15:06:19,892][20347] Updated weights for policy 0, policy_version 1364961 (0.0007) [2023-12-27 15:06:19,961][20347] Updated weights for policy 0, policy_version 1364971 (0.0008) [2023-12-27 15:06:20,022][20347] Updated weights for policy 0, policy_version 1364981 (0.0007) [2023-12-27 15:06:20,389][20381] Updated weights for policy 1, policy_version 1363829 (0.0006) [2023-12-27 15:06:20,450][20381] Updated weights for policy 1, policy_version 1363839 (0.0008) [2023-12-27 15:06:20,506][20381] Updated weights for policy 1, policy_version 1363849 (0.0008) [2023-12-27 15:06:20,742][20347] Updated weights for policy 0, policy_version 1364991 (0.0008) [2023-12-27 15:06:20,802][20347] Updated weights for policy 0, policy_version 1365001 (0.0006) [2023-12-27 15:06:20,865][20347] Updated weights for policy 0, policy_version 1365011 (0.0006) [2023-12-27 15:06:21,234][20381] Updated weights for policy 1, policy_version 1363859 (0.0008) [2023-12-27 15:06:21,291][20381] Updated weights for policy 1, policy_version 1363869 (0.0008) [2023-12-27 15:06:21,351][20381] Updated weights for policy 1, policy_version 1363879 (0.0008) [2023-12-27 15:06:21,638][20347] Updated weights for policy 0, policy_version 1365021 (0.0007) [2023-12-27 15:06:21,701][20347] Updated weights for policy 0, policy_version 1365031 (0.0008) [2023-12-27 15:06:21,774][20347] Updated weights for policy 0, policy_version 1365041 (0.0008) [2023-12-27 15:06:22,168][20381] Updated weights for policy 1, policy_version 1363889 (0.0008) [2023-12-27 15:06:22,227][20381] Updated weights for policy 1, policy_version 1363899 (0.0008) [2023-12-27 15:06:22,262][20292] KL-divergence is very high: 108.3194 [2023-12-27 15:06:22,292][20381] Updated weights for policy 1, policy_version 1363909 (0.0008) [2023-12-27 15:06:22,350][20381] Updated weights for policy 1, policy_version 1363919 (0.0008) [2023-12-27 15:06:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 698712064. Throughput: 0: 10234.7, 1: 10210.6. Samples: 698682684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:22,363][19170] Avg episode reward: [(0, '921.680'), (1, '990.250')] [2023-12-27 15:06:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001363920_349192192.pth... [2023-12-27 15:06:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001365048_349519872.pth... [2023-12-27 15:06:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001362704_348880896.pth [2023-12-27 15:06:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001363864_349216768.pth [2023-12-27 15:06:22,400][20347] Updated weights for policy 0, policy_version 1365051 (0.0006) [2023-12-27 15:06:22,457][20347] Updated weights for policy 0, policy_version 1365061 (0.0007) [2023-12-27 15:06:22,519][20347] Updated weights for policy 0, policy_version 1365071 (0.0008) [2023-12-27 15:06:23,072][20381] Updated weights for policy 1, policy_version 1363929 (0.0008) [2023-12-27 15:06:23,126][20381] Updated weights for policy 1, policy_version 1363939 (0.0007) [2023-12-27 15:06:23,176][20381] Updated weights for policy 1, policy_version 1363949 (0.0007) [2023-12-27 15:06:23,213][20347] Updated weights for policy 0, policy_version 1365081 (0.0008) [2023-12-27 15:06:23,270][20347] Updated weights for policy 0, policy_version 1365091 (0.0010) [2023-12-27 15:06:23,324][20347] Updated weights for policy 0, policy_version 1365101 (0.0010) [2023-12-27 15:06:23,380][20347] Updated weights for policy 0, policy_version 1365111 (0.0009) [2023-12-27 15:06:23,770][20381] Updated weights for policy 1, policy_version 1363959 (0.0006) [2023-12-27 15:06:23,820][20381] Updated weights for policy 1, policy_version 1363969 (0.0005) [2023-12-27 15:06:23,864][20381] Updated weights for policy 1, policy_version 1363979 (0.0005) [2023-12-27 15:06:23,985][20347] Updated weights for policy 0, policy_version 1365121 (0.0008) [2023-12-27 15:06:24,033][20347] Updated weights for policy 0, policy_version 1365131 (0.0007) [2023-12-27 15:06:24,082][20347] Updated weights for policy 0, policy_version 1365141 (0.0005) [2023-12-27 15:06:24,534][20381] Updated weights for policy 1, policy_version 1363989 (0.0006) [2023-12-27 15:06:24,594][20381] Updated weights for policy 1, policy_version 1363999 (0.0007) [2023-12-27 15:06:24,648][20381] Updated weights for policy 1, policy_version 1364009 (0.0006) [2023-12-27 15:06:24,722][20347] Updated weights for policy 0, policy_version 1365151 (0.0007) [2023-12-27 15:06:24,783][20347] Updated weights for policy 0, policy_version 1365161 (0.0010) [2023-12-27 15:06:24,832][20347] Updated weights for policy 0, policy_version 1365171 (0.0010) [2023-12-27 15:06:25,283][20381] Updated weights for policy 1, policy_version 1364019 (0.0007) [2023-12-27 15:06:25,346][20381] Updated weights for policy 1, policy_version 1364029 (0.0010) [2023-12-27 15:06:25,403][20381] Updated weights for policy 1, policy_version 1364039 (0.0009) [2023-12-27 15:06:25,495][20347] Updated weights for policy 0, policy_version 1365181 (0.0009) [2023-12-27 15:06:25,554][20347] Updated weights for policy 0, policy_version 1365191 (0.0007) [2023-12-27 15:06:25,610][20347] Updated weights for policy 0, policy_version 1365201 (0.0008) [2023-12-27 15:06:26,086][20381] Updated weights for policy 1, policy_version 1364049 (0.0007) [2023-12-27 15:06:26,135][20381] Updated weights for policy 1, policy_version 1364059 (0.0010) [2023-12-27 15:06:26,190][20381] Updated weights for policy 1, policy_version 1364069 (0.0010) [2023-12-27 15:06:26,249][20381] Updated weights for policy 1, policy_version 1364079 (0.0006) [2023-12-27 15:06:26,327][20347] Updated weights for policy 0, policy_version 1365211 (0.0007) [2023-12-27 15:06:26,387][20347] Updated weights for policy 0, policy_version 1365221 (0.0005) [2023-12-27 15:06:26,436][20347] Updated weights for policy 0, policy_version 1365231 (0.0005) [2023-12-27 15:06:26,838][20381] Updated weights for policy 1, policy_version 1364089 (0.0005) [2023-12-27 15:06:26,891][20381] Updated weights for policy 1, policy_version 1364099 (0.0005) [2023-12-27 15:06:26,952][20381] Updated weights for policy 1, policy_version 1364109 (0.0005) [2023-12-27 15:06:27,003][20347] Updated weights for policy 0, policy_version 1365241 (0.0006) [2023-12-27 15:06:27,041][20291] KL-divergence is very high: 212.2365 [2023-12-27 15:06:27,050][20347] Updated weights for policy 0, policy_version 1365251 (0.0010) [2023-12-27 15:06:27,082][20291] KL-divergence is very high: 194.8399 [2023-12-27 15:06:27,104][20347] Updated weights for policy 0, policy_version 1365261 (0.0010) [2023-12-27 15:06:27,151][20347] Updated weights for policy 0, policy_version 1365271 (0.0010) [2023-12-27 15:06:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 698818560. Throughput: 0: 10231.8, 1: 10178.9. Samples: 698804480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:27,363][19170] Avg episode reward: [(0, '951.360'), (1, '990.250')] [2023-12-27 15:06:27,470][20381] Updated weights for policy 1, policy_version 1364119 (0.0005) [2023-12-27 15:06:27,521][20381] Updated weights for policy 1, policy_version 1364129 (0.0009) [2023-12-27 15:06:27,568][20381] Updated weights for policy 1, policy_version 1364139 (0.0006) [2023-12-27 15:06:27,775][20347] Updated weights for policy 0, policy_version 1365281 (0.0010) [2023-12-27 15:06:27,836][20347] Updated weights for policy 0, policy_version 1365291 (0.0010) [2023-12-27 15:06:27,895][20347] Updated weights for policy 0, policy_version 1365301 (0.0010) [2023-12-27 15:06:28,229][20381] Updated weights for policy 1, policy_version 1364149 (0.0006) [2023-12-27 15:06:28,293][20381] Updated weights for policy 1, policy_version 1364159 (0.0007) [2023-12-27 15:06:28,343][20381] Updated weights for policy 1, policy_version 1364169 (0.0008) [2023-12-27 15:06:28,565][20347] Updated weights for policy 0, policy_version 1365311 (0.0008) [2023-12-27 15:06:28,619][20347] Updated weights for policy 0, policy_version 1365321 (0.0008) [2023-12-27 15:06:28,676][20347] Updated weights for policy 0, policy_version 1365331 (0.0007) [2023-12-27 15:06:29,030][20381] Updated weights for policy 1, policy_version 1364179 (0.0007) [2023-12-27 15:06:29,073][20381] Updated weights for policy 1, policy_version 1364189 (0.0005) [2023-12-27 15:06:29,117][20381] Updated weights for policy 1, policy_version 1364199 (0.0005) [2023-12-27 15:06:29,406][20347] Updated weights for policy 0, policy_version 1365341 (0.0007) [2023-12-27 15:06:29,470][20347] Updated weights for policy 0, policy_version 1365351 (0.0006) [2023-12-27 15:06:29,530][20347] Updated weights for policy 0, policy_version 1365361 (0.0005) [2023-12-27 15:06:29,763][20381] Updated weights for policy 1, policy_version 1364209 (0.0005) [2023-12-27 15:06:29,829][20381] Updated weights for policy 1, policy_version 1364219 (0.0007) [2023-12-27 15:06:29,886][20381] Updated weights for policy 1, policy_version 1364229 (0.0008) [2023-12-27 15:06:29,942][20381] Updated weights for policy 1, policy_version 1364239 (0.0009) [2023-12-27 15:06:30,157][20347] Updated weights for policy 0, policy_version 1365371 (0.0006) [2023-12-27 15:06:30,207][20347] Updated weights for policy 0, policy_version 1365381 (0.0007) [2023-12-27 15:06:30,262][20347] Updated weights for policy 0, policy_version 1365391 (0.0008) [2023-12-27 15:06:30,598][20381] Updated weights for policy 1, policy_version 1364249 (0.0006) [2023-12-27 15:06:30,656][20381] Updated weights for policy 1, policy_version 1364259 (0.0005) [2023-12-27 15:06:30,715][20381] Updated weights for policy 1, policy_version 1364269 (0.0005) [2023-12-27 15:06:30,934][20347] Updated weights for policy 0, policy_version 1365401 (0.0008) [2023-12-27 15:06:30,982][20347] Updated weights for policy 0, policy_version 1365411 (0.0005) [2023-12-27 15:06:31,039][20347] Updated weights for policy 0, policy_version 1365421 (0.0009) [2023-12-27 15:06:31,099][20347] Updated weights for policy 0, policy_version 1365431 (0.0010) [2023-12-27 15:06:31,266][20381] Updated weights for policy 1, policy_version 1364279 (0.0006) [2023-12-27 15:06:31,316][20381] Updated weights for policy 1, policy_version 1364289 (0.0007) [2023-12-27 15:06:31,383][20381] Updated weights for policy 1, policy_version 1364299 (0.0008) [2023-12-27 15:06:31,823][20347] Updated weights for policy 0, policy_version 1365441 (0.0008) [2023-12-27 15:06:31,884][20347] Updated weights for policy 0, policy_version 1365451 (0.0009) [2023-12-27 15:06:31,944][20347] Updated weights for policy 0, policy_version 1365461 (0.0007) [2023-12-27 15:06:32,137][20381] Updated weights for policy 1, policy_version 1364309 (0.0008) [2023-12-27 15:06:32,196][20381] Updated weights for policy 1, policy_version 1364319 (0.0007) [2023-12-27 15:06:32,253][20381] Updated weights for policy 1, policy_version 1364329 (0.0008) [2023-12-27 15:06:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 698925056. Throughput: 0: 10292.4, 1: 10269.2. Samples: 698932364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:32,363][19170] Avg episode reward: [(0, '941.640'), (1, '1000.000')] [2023-12-27 15:06:32,555][20347] Updated weights for policy 0, policy_version 1365471 (0.0009) [2023-12-27 15:06:32,617][20347] Updated weights for policy 0, policy_version 1365481 (0.0011) [2023-12-27 15:06:32,670][20347] Updated weights for policy 0, policy_version 1365491 (0.0010) [2023-12-27 15:06:33,017][20381] Updated weights for policy 1, policy_version 1364339 (0.0008) [2023-12-27 15:06:33,077][20381] Updated weights for policy 1, policy_version 1364349 (0.0009) [2023-12-27 15:06:33,129][20381] Updated weights for policy 1, policy_version 1364359 (0.0009) [2023-12-27 15:06:33,245][20347] Updated weights for policy 0, policy_version 1365501 (0.0008) [2023-12-27 15:06:33,299][20347] Updated weights for policy 0, policy_version 1365511 (0.0010) [2023-12-27 15:06:33,353][20347] Updated weights for policy 0, policy_version 1365521 (0.0010) [2023-12-27 15:06:33,795][20381] Updated weights for policy 1, policy_version 1364369 (0.0006) [2023-12-27 15:06:33,847][20381] Updated weights for policy 1, policy_version 1364379 (0.0007) [2023-12-27 15:06:33,902][20381] Updated weights for policy 1, policy_version 1364389 (0.0008) [2023-12-27 15:06:33,959][20381] Updated weights for policy 1, policy_version 1364399 (0.0008) [2023-12-27 15:06:33,961][20347] Updated weights for policy 0, policy_version 1365531 (0.0010) [2023-12-27 15:06:34,017][20347] Updated weights for policy 0, policy_version 1365541 (0.0010) [2023-12-27 15:06:34,067][20347] Updated weights for policy 0, policy_version 1365551 (0.0005) [2023-12-27 15:06:34,683][20381] Updated weights for policy 1, policy_version 1364409 (0.0008) [2023-12-27 15:06:34,701][20292] KL-divergence is very high: 161.5353 [2023-12-27 15:06:34,739][20381] Updated weights for policy 1, policy_version 1364419 (0.0008) [2023-12-27 15:06:34,754][20347] Updated weights for policy 0, policy_version 1365561 (0.0005) [2023-12-27 15:06:34,792][20381] Updated weights for policy 1, policy_version 1364429 (0.0007) [2023-12-27 15:06:34,809][20347] Updated weights for policy 0, policy_version 1365571 (0.0006) [2023-12-27 15:06:34,860][20347] Updated weights for policy 0, policy_version 1365581 (0.0007) [2023-12-27 15:06:34,908][20347] Updated weights for policy 0, policy_version 1365591 (0.0007) [2023-12-27 15:06:35,519][20381] Updated weights for policy 1, policy_version 1364439 (0.0006) [2023-12-27 15:06:35,550][20347] Updated weights for policy 0, policy_version 1365601 (0.0008) [2023-12-27 15:06:35,572][20381] Updated weights for policy 1, policy_version 1364449 (0.0009) [2023-12-27 15:06:35,604][20347] Updated weights for policy 0, policy_version 1365611 (0.0006) [2023-12-27 15:06:35,631][20381] Updated weights for policy 1, policy_version 1364459 (0.0007) [2023-12-27 15:06:35,656][20347] Updated weights for policy 0, policy_version 1365621 (0.0005) [2023-12-27 15:06:36,334][20347] Updated weights for policy 0, policy_version 1365631 (0.0006) [2023-12-27 15:06:36,374][20381] Updated weights for policy 1, policy_version 1364469 (0.0007) [2023-12-27 15:06:36,391][20347] Updated weights for policy 0, policy_version 1365641 (0.0007) [2023-12-27 15:06:36,432][20381] Updated weights for policy 1, policy_version 1364479 (0.0006) [2023-12-27 15:06:36,452][20347] Updated weights for policy 0, policy_version 1365651 (0.0007) [2023-12-27 15:06:36,495][20381] Updated weights for policy 1, policy_version 1364489 (0.0007) [2023-12-27 15:06:37,127][20347] Updated weights for policy 0, policy_version 1365661 (0.0007) [2023-12-27 15:06:37,183][20347] Updated weights for policy 0, policy_version 1365671 (0.0008) [2023-12-27 15:06:37,212][20381] Updated weights for policy 1, policy_version 1364499 (0.0008) [2023-12-27 15:06:37,241][20347] Updated weights for policy 0, policy_version 1365681 (0.0007) [2023-12-27 15:06:37,269][20381] Updated weights for policy 1, policy_version 1364509 (0.0006) [2023-12-27 15:06:37,316][20381] Updated weights for policy 1, policy_version 1364519 (0.0007) [2023-12-27 15:06:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 699031552. Throughput: 0: 10327.8, 1: 10277.7. Samples: 698994528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:37,364][19170] Avg episode reward: [(0, '970.650'), (1, '990.230')] [2023-12-27 15:06:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001364528_349347840.pth... [2023-12-27 15:06:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001365688_349683712.pth... [2023-12-27 15:06:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001364472_349372416.pth [2023-12-27 15:06:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001363312_349036544.pth [2023-12-27 15:06:37,854][20347] Updated weights for policy 0, policy_version 1365691 (0.0005) [2023-12-27 15:06:37,905][20347] Updated weights for policy 0, policy_version 1365701 (0.0009) [2023-12-27 15:06:37,956][20347] Updated weights for policy 0, policy_version 1365711 (0.0010) [2023-12-27 15:06:38,063][20381] Updated weights for policy 1, policy_version 1364529 (0.0008) [2023-12-27 15:06:38,122][20381] Updated weights for policy 1, policy_version 1364539 (0.0005) [2023-12-27 15:06:38,182][20381] Updated weights for policy 1, policy_version 1364549 (0.0006) [2023-12-27 15:06:38,234][20381] Updated weights for policy 1, policy_version 1364559 (0.0007) [2023-12-27 15:06:38,615][20347] Updated weights for policy 0, policy_version 1365721 (0.0010) [2023-12-27 15:06:38,671][20347] Updated weights for policy 0, policy_version 1365731 (0.0009) [2023-12-27 15:06:38,726][20347] Updated weights for policy 0, policy_version 1365741 (0.0008) [2023-12-27 15:06:38,780][20347] Updated weights for policy 0, policy_version 1365751 (0.0008) [2023-12-27 15:06:38,925][20381] Updated weights for policy 1, policy_version 1364569 (0.0008) [2023-12-27 15:06:38,983][20381] Updated weights for policy 1, policy_version 1364579 (0.0008) [2023-12-27 15:06:39,045][20381] Updated weights for policy 1, policy_version 1364589 (0.0007) [2023-12-27 15:06:39,540][20347] Updated weights for policy 0, policy_version 1365761 (0.0009) [2023-12-27 15:06:39,594][20347] Updated weights for policy 0, policy_version 1365771 (0.0008) [2023-12-27 15:06:39,654][20347] Updated weights for policy 0, policy_version 1365781 (0.0008) [2023-12-27 15:06:39,723][20381] Updated weights for policy 1, policy_version 1364599 (0.0008) [2023-12-27 15:06:39,784][20381] Updated weights for policy 1, policy_version 1364609 (0.0007) [2023-12-27 15:06:39,844][20381] Updated weights for policy 1, policy_version 1364619 (0.0008) [2023-12-27 15:06:40,398][20347] Updated weights for policy 0, policy_version 1365791 (0.0007) [2023-12-27 15:06:40,461][20347] Updated weights for policy 0, policy_version 1365801 (0.0007) [2023-12-27 15:06:40,522][20347] Updated weights for policy 0, policy_version 1365811 (0.0007) [2023-12-27 15:06:40,590][20381] Updated weights for policy 1, policy_version 1364629 (0.0009) [2023-12-27 15:06:40,651][20381] Updated weights for policy 1, policy_version 1364639 (0.0008) [2023-12-27 15:06:40,715][20381] Updated weights for policy 1, policy_version 1364649 (0.0007) [2023-12-27 15:06:41,214][20347] Updated weights for policy 0, policy_version 1365821 (0.0008) [2023-12-27 15:06:41,276][20347] Updated weights for policy 0, policy_version 1365831 (0.0008) [2023-12-27 15:06:41,335][20347] Updated weights for policy 0, policy_version 1365841 (0.0009) [2023-12-27 15:06:41,425][20381] Updated weights for policy 1, policy_version 1364659 (0.0008) [2023-12-27 15:06:41,484][20381] Updated weights for policy 1, policy_version 1364669 (0.0007) [2023-12-27 15:06:41,551][20381] Updated weights for policy 1, policy_version 1364679 (0.0005) [2023-12-27 15:06:42,109][20347] Updated weights for policy 0, policy_version 1365851 (0.0007) [2023-12-27 15:06:42,158][20347] Updated weights for policy 0, policy_version 1365861 (0.0008) [2023-12-27 15:06:42,208][20347] Updated weights for policy 0, policy_version 1365871 (0.0008) [2023-12-27 15:06:42,260][20381] Updated weights for policy 1, policy_version 1364689 (0.0006) [2023-12-27 15:06:42,327][20381] Updated weights for policy 1, policy_version 1364699 (0.0007) [2023-12-27 15:06:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 699121664. Throughput: 0: 10285.0, 1: 10221.3. Samples: 699114232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:42,363][19170] Avg episode reward: [(0, '970.660'), (1, '990.230')] [2023-12-27 15:06:42,395][20381] Updated weights for policy 1, policy_version 1364709 (0.0010) [2023-12-27 15:06:42,455][20381] Updated weights for policy 1, policy_version 1364719 (0.0008) [2023-12-27 15:06:42,946][20347] Updated weights for policy 0, policy_version 1365881 (0.0007) [2023-12-27 15:06:42,999][20347] Updated weights for policy 0, policy_version 1365891 (0.0006) [2023-12-27 15:06:43,051][20347] Updated weights for policy 0, policy_version 1365901 (0.0007) [2023-12-27 15:06:43,104][20347] Updated weights for policy 0, policy_version 1365911 (0.0007) [2023-12-27 15:06:43,107][20381] Updated weights for policy 1, policy_version 1364729 (0.0006) [2023-12-27 15:06:43,152][20381] Updated weights for policy 1, policy_version 1364739 (0.0005) [2023-12-27 15:06:43,200][20381] Updated weights for policy 1, policy_version 1364749 (0.0009) [2023-12-27 15:06:43,739][20347] Updated weights for policy 0, policy_version 1365921 (0.0006) [2023-12-27 15:06:43,803][20347] Updated weights for policy 0, policy_version 1365931 (0.0005) [2023-12-27 15:06:43,859][20381] Updated weights for policy 1, policy_version 1364759 (0.0010) [2023-12-27 15:06:43,869][20347] Updated weights for policy 0, policy_version 1365941 (0.0005) [2023-12-27 15:06:43,910][20381] Updated weights for policy 1, policy_version 1364769 (0.0010) [2023-12-27 15:06:43,964][20381] Updated weights for policy 1, policy_version 1364779 (0.0010) [2023-12-27 15:06:44,437][20347] Updated weights for policy 0, policy_version 1365951 (0.0007) [2023-12-27 15:06:44,492][20347] Updated weights for policy 0, policy_version 1365961 (0.0006) [2023-12-27 15:06:44,552][20347] Updated weights for policy 0, policy_version 1365971 (0.0007) [2023-12-27 15:06:44,597][20381] Updated weights for policy 1, policy_version 1364789 (0.0008) [2023-12-27 15:06:44,648][20381] Updated weights for policy 1, policy_version 1364799 (0.0007) [2023-12-27 15:06:44,708][20381] Updated weights for policy 1, policy_version 1364809 (0.0008) [2023-12-27 15:06:45,192][20347] Updated weights for policy 0, policy_version 1365981 (0.0006) [2023-12-27 15:06:45,250][20347] Updated weights for policy 0, policy_version 1365991 (0.0006) [2023-12-27 15:06:45,306][20347] Updated weights for policy 0, policy_version 1366001 (0.0006) [2023-12-27 15:06:45,392][20381] Updated weights for policy 1, policy_version 1364819 (0.0008) [2023-12-27 15:06:45,457][20381] Updated weights for policy 1, policy_version 1364829 (0.0008) [2023-12-27 15:06:45,520][20381] Updated weights for policy 1, policy_version 1364839 (0.0007) [2023-12-27 15:06:45,932][20347] Updated weights for policy 0, policy_version 1366011 (0.0007) [2023-12-27 15:06:45,991][20347] Updated weights for policy 0, policy_version 1366021 (0.0008) [2023-12-27 15:06:46,051][20347] Updated weights for policy 0, policy_version 1366031 (0.0008) [2023-12-27 15:06:46,208][20381] Updated weights for policy 1, policy_version 1364849 (0.0008) [2023-12-27 15:06:46,261][20381] Updated weights for policy 1, policy_version 1364859 (0.0005) [2023-12-27 15:06:46,312][20381] Updated weights for policy 1, policy_version 1364869 (0.0007) [2023-12-27 15:06:46,363][20381] Updated weights for policy 1, policy_version 1364879 (0.0005) [2023-12-27 15:06:46,725][20347] Updated weights for policy 0, policy_version 1366041 (0.0007) [2023-12-27 15:06:46,785][20347] Updated weights for policy 0, policy_version 1366051 (0.0005) [2023-12-27 15:06:46,838][20347] Updated weights for policy 0, policy_version 1366061 (0.0005) [2023-12-27 15:06:46,892][20347] Updated weights for policy 0, policy_version 1366071 (0.0005) [2023-12-27 15:06:46,988][20381] Updated weights for policy 1, policy_version 1364889 (0.0005) [2023-12-27 15:06:47,046][20381] Updated weights for policy 1, policy_version 1364899 (0.0006) [2023-12-27 15:06:47,100][20381] Updated weights for policy 1, policy_version 1364909 (0.0007) [2023-12-27 15:06:47,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 699228160. Throughput: 0: 10257.5, 1: 10229.7. Samples: 699238180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:47,364][19170] Avg episode reward: [(0, '941.360'), (1, '1000.000')] [2023-12-27 15:06:47,520][20347] Updated weights for policy 0, policy_version 1366081 (0.0005) [2023-12-27 15:06:47,572][20347] Updated weights for policy 0, policy_version 1366091 (0.0005) [2023-12-27 15:06:47,621][20347] Updated weights for policy 0, policy_version 1366101 (0.0008) [2023-12-27 15:06:47,801][20381] Updated weights for policy 1, policy_version 1364919 (0.0006) [2023-12-27 15:06:47,859][20381] Updated weights for policy 1, policy_version 1364929 (0.0005) [2023-12-27 15:06:47,918][20381] Updated weights for policy 1, policy_version 1364939 (0.0005) [2023-12-27 15:06:48,265][20347] Updated weights for policy 0, policy_version 1366111 (0.0007) [2023-12-27 15:06:48,332][20347] Updated weights for policy 0, policy_version 1366121 (0.0008) [2023-12-27 15:06:48,395][20347] Updated weights for policy 0, policy_version 1366131 (0.0010) [2023-12-27 15:06:48,557][20381] Updated weights for policy 1, policy_version 1364949 (0.0006) [2023-12-27 15:06:48,612][20381] Updated weights for policy 1, policy_version 1364959 (0.0007) [2023-12-27 15:06:48,666][20381] Updated weights for policy 1, policy_version 1364969 (0.0007) [2023-12-27 15:06:49,045][20347] Updated weights for policy 0, policy_version 1366141 (0.0010) [2023-12-27 15:06:49,092][20347] Updated weights for policy 0, policy_version 1366151 (0.0010) [2023-12-27 15:06:49,141][20347] Updated weights for policy 0, policy_version 1366161 (0.0010) [2023-12-27 15:06:49,288][20381] Updated weights for policy 1, policy_version 1364979 (0.0007) [2023-12-27 15:06:49,344][20381] Updated weights for policy 1, policy_version 1364989 (0.0008) [2023-12-27 15:06:49,407][20381] Updated weights for policy 1, policy_version 1364999 (0.0007) [2023-12-27 15:06:49,852][20347] Updated weights for policy 0, policy_version 1366171 (0.0009) [2023-12-27 15:06:49,881][20291] KL-divergence is very high: 263.3011 [2023-12-27 15:06:49,909][20347] Updated weights for policy 0, policy_version 1366181 (0.0008) [2023-12-27 15:06:49,928][20291] KL-divergence is very high: 246.5014 [2023-12-27 15:06:49,969][20347] Updated weights for policy 0, policy_version 1366191 (0.0007) [2023-12-27 15:06:50,109][20381] Updated weights for policy 1, policy_version 1365009 (0.0007) [2023-12-27 15:06:50,169][20381] Updated weights for policy 1, policy_version 1365019 (0.0011) [2023-12-27 15:06:50,219][20381] Updated weights for policy 1, policy_version 1365029 (0.0010) [2023-12-27 15:06:50,273][20381] Updated weights for policy 1, policy_version 1365039 (0.0011) [2023-12-27 15:06:50,647][20347] Updated weights for policy 0, policy_version 1366201 (0.0007) [2023-12-27 15:06:50,708][20347] Updated weights for policy 0, policy_version 1366211 (0.0010) [2023-12-27 15:06:50,771][20347] Updated weights for policy 0, policy_version 1366221 (0.0010) [2023-12-27 15:06:50,830][20347] Updated weights for policy 0, policy_version 1366231 (0.0010) [2023-12-27 15:06:50,962][20381] Updated weights for policy 1, policy_version 1365049 (0.0010) [2023-12-27 15:06:51,022][20381] Updated weights for policy 1, policy_version 1365059 (0.0010) [2023-12-27 15:06:51,083][20381] Updated weights for policy 1, policy_version 1365069 (0.0007) [2023-12-27 15:06:51,528][20347] Updated weights for policy 0, policy_version 1366241 (0.0007) [2023-12-27 15:06:51,589][20347] Updated weights for policy 0, policy_version 1366251 (0.0008) [2023-12-27 15:06:51,659][20347] Updated weights for policy 0, policy_version 1366261 (0.0011) [2023-12-27 15:06:51,772][20381] Updated weights for policy 1, policy_version 1365079 (0.0009) [2023-12-27 15:06:51,830][20381] Updated weights for policy 1, policy_version 1365089 (0.0011) [2023-12-27 15:06:51,886][20381] Updated weights for policy 1, policy_version 1365099 (0.0011) [2023-12-27 15:06:52,336][20347] Updated weights for policy 0, policy_version 1366271 (0.0010) [2023-12-27 15:06:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 699326464. Throughput: 0: 10254.3, 1: 10234.6. Samples: 699300008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:52,366][19170] Avg episode reward: [(0, '872.960'), (1, '1000.000')] [2023-12-27 15:06:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001365104_349495296.pth... [2023-12-27 15:06:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001363920_349192192.pth [2023-12-27 15:06:52,401][20347] Updated weights for policy 0, policy_version 1366281 (0.0007) [2023-12-27 15:06:52,461][20347] Updated weights for policy 0, policy_version 1366291 (0.0008) [2023-12-27 15:06:52,487][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001366296_349839360.pth... [2023-12-27 15:06:52,491][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001365048_349519872.pth [2023-12-27 15:06:52,594][20381] Updated weights for policy 1, policy_version 1365109 (0.0011) [2023-12-27 15:06:52,659][20381] Updated weights for policy 1, policy_version 1365119 (0.0010) [2023-12-27 15:06:52,723][20381] Updated weights for policy 1, policy_version 1365129 (0.0010) [2023-12-27 15:06:53,043][20347] Updated weights for policy 0, policy_version 1366301 (0.0008) [2023-12-27 15:06:53,066][20291] KL-divergence is very high: 160.2919 [2023-12-27 15:06:53,094][20347] Updated weights for policy 0, policy_version 1366311 (0.0010) [2023-12-27 15:06:53,144][20347] Updated weights for policy 0, policy_version 1366321 (0.0008) [2023-12-27 15:06:53,340][20381] Updated weights for policy 1, policy_version 1365139 (0.0009) [2023-12-27 15:06:53,402][20381] Updated weights for policy 1, policy_version 1365149 (0.0009) [2023-12-27 15:06:53,463][20381] Updated weights for policy 1, policy_version 1365159 (0.0006) [2023-12-27 15:06:53,749][20347] Updated weights for policy 0, policy_version 1366331 (0.0007) [2023-12-27 15:06:53,808][20347] Updated weights for policy 0, policy_version 1366341 (0.0006) [2023-12-27 15:06:53,873][20347] Updated weights for policy 0, policy_version 1366351 (0.0008) [2023-12-27 15:06:54,054][20381] Updated weights for policy 1, policy_version 1365169 (0.0006) [2023-12-27 15:06:54,103][20381] Updated weights for policy 1, policy_version 1365179 (0.0008) [2023-12-27 15:06:54,162][20381] Updated weights for policy 1, policy_version 1365189 (0.0007) [2023-12-27 15:06:54,218][20381] Updated weights for policy 1, policy_version 1365199 (0.0007) [2023-12-27 15:06:54,514][20347] Updated weights for policy 0, policy_version 1366361 (0.0007) [2023-12-27 15:06:54,573][20347] Updated weights for policy 0, policy_version 1366371 (0.0009) [2023-12-27 15:06:54,628][20347] Updated weights for policy 0, policy_version 1366381 (0.0010) [2023-12-27 15:06:54,683][20347] Updated weights for policy 0, policy_version 1366391 (0.0010) [2023-12-27 15:06:54,898][20381] Updated weights for policy 1, policy_version 1365209 (0.0007) [2023-12-27 15:06:54,955][20381] Updated weights for policy 1, policy_version 1365219 (0.0008) [2023-12-27 15:06:55,012][20381] Updated weights for policy 1, policy_version 1365229 (0.0008) [2023-12-27 15:06:55,365][20347] Updated weights for policy 0, policy_version 1366401 (0.0008) [2023-12-27 15:06:55,417][20347] Updated weights for policy 0, policy_version 1366411 (0.0007) [2023-12-27 15:06:55,475][20347] Updated weights for policy 0, policy_version 1366421 (0.0008) [2023-12-27 15:06:55,713][20381] Updated weights for policy 1, policy_version 1365239 (0.0007) [2023-12-27 15:06:55,765][20381] Updated weights for policy 1, policy_version 1365249 (0.0008) [2023-12-27 15:06:55,825][20381] Updated weights for policy 1, policy_version 1365259 (0.0008) [2023-12-27 15:06:56,187][20347] Updated weights for policy 0, policy_version 1366431 (0.0008) [2023-12-27 15:06:56,243][20347] Updated weights for policy 0, policy_version 1366441 (0.0007) [2023-12-27 15:06:56,297][20347] Updated weights for policy 0, policy_version 1366451 (0.0008) [2023-12-27 15:06:56,471][20381] Updated weights for policy 1, policy_version 1365269 (0.0006) [2023-12-27 15:06:56,522][20381] Updated weights for policy 1, policy_version 1365279 (0.0005) [2023-12-27 15:06:56,576][20381] Updated weights for policy 1, policy_version 1365289 (0.0005) [2023-12-27 15:06:56,915][20347] Updated weights for policy 0, policy_version 1366461 (0.0007) [2023-12-27 15:06:56,973][20347] Updated weights for policy 0, policy_version 1366471 (0.0005) [2023-12-27 15:06:57,034][20347] Updated weights for policy 0, policy_version 1366481 (0.0005) [2023-12-27 15:06:57,262][20381] Updated weights for policy 1, policy_version 1365299 (0.0006) [2023-12-27 15:06:57,309][20381] Updated weights for policy 1, policy_version 1365309 (0.0006) [2023-12-27 15:06:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 699432960. Throughput: 0: 10286.5, 1: 10250.2. Samples: 699424156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:06:57,363][19170] Avg episode reward: [(0, '911.930'), (1, '1000.000')] [2023-12-27 15:06:57,366][20381] Updated weights for policy 1, policy_version 1365319 (0.0005) [2023-12-27 15:06:57,611][20347] Updated weights for policy 0, policy_version 1366491 (0.0007) [2023-12-27 15:06:57,672][20347] Updated weights for policy 0, policy_version 1366501 (0.0010) [2023-12-27 15:06:57,727][20347] Updated weights for policy 0, policy_version 1366511 (0.0010) [2023-12-27 15:06:57,910][20381] Updated weights for policy 1, policy_version 1365329 (0.0006) [2023-12-27 15:06:57,958][20381] Updated weights for policy 1, policy_version 1365339 (0.0008) [2023-12-27 15:06:58,003][20381] Updated weights for policy 1, policy_version 1365349 (0.0008) [2023-12-27 15:06:58,048][20381] Updated weights for policy 1, policy_version 1365359 (0.0009) [2023-12-27 15:06:58,386][20347] Updated weights for policy 0, policy_version 1366521 (0.0010) [2023-12-27 15:06:58,444][20347] Updated weights for policy 0, policy_version 1366531 (0.0008) [2023-12-27 15:06:58,499][20347] Updated weights for policy 0, policy_version 1366541 (0.0011) [2023-12-27 15:06:58,556][20347] Updated weights for policy 0, policy_version 1366551 (0.0010) [2023-12-27 15:06:58,902][20381] Updated weights for policy 1, policy_version 1365369 (0.0008) [2023-12-27 15:06:58,964][20381] Updated weights for policy 1, policy_version 1365379 (0.0008) [2023-12-27 15:06:59,031][20381] Updated weights for policy 1, policy_version 1365389 (0.0006) [2023-12-27 15:06:59,355][20347] Updated weights for policy 0, policy_version 1366561 (0.0011) [2023-12-27 15:06:59,417][20347] Updated weights for policy 0, policy_version 1366571 (0.0010) [2023-12-27 15:06:59,471][20347] Updated weights for policy 0, policy_version 1366581 (0.0010) [2023-12-27 15:06:59,703][20381] Updated weights for policy 1, policy_version 1365399 (0.0007) [2023-12-27 15:06:59,754][20381] Updated weights for policy 1, policy_version 1365409 (0.0007) [2023-12-27 15:06:59,801][20381] Updated weights for policy 1, policy_version 1365419 (0.0007) [2023-12-27 15:07:00,174][20347] Updated weights for policy 0, policy_version 1366591 (0.0008) [2023-12-27 15:07:00,220][20347] Updated weights for policy 0, policy_version 1366601 (0.0007) [2023-12-27 15:07:00,271][20347] Updated weights for policy 0, policy_version 1366611 (0.0008) [2023-12-27 15:07:00,532][20381] Updated weights for policy 1, policy_version 1365429 (0.0006) [2023-12-27 15:07:00,577][20381] Updated weights for policy 1, policy_version 1365439 (0.0005) [2023-12-27 15:07:00,630][20381] Updated weights for policy 1, policy_version 1365449 (0.0005) [2023-12-27 15:07:01,010][20347] Updated weights for policy 0, policy_version 1366621 (0.0008) [2023-12-27 15:07:01,063][20347] Updated weights for policy 0, policy_version 1366631 (0.0008) [2023-12-27 15:07:01,114][20347] Updated weights for policy 0, policy_version 1366641 (0.0008) [2023-12-27 15:07:01,264][20381] Updated weights for policy 1, policy_version 1365459 (0.0006) [2023-12-27 15:07:01,318][20381] Updated weights for policy 1, policy_version 1365469 (0.0007) [2023-12-27 15:07:01,379][20381] Updated weights for policy 1, policy_version 1365479 (0.0008) [2023-12-27 15:07:01,885][20347] Updated weights for policy 0, policy_version 1366651 (0.0008) [2023-12-27 15:07:01,934][20347] Updated weights for policy 0, policy_version 1366661 (0.0010) [2023-12-27 15:07:01,986][20347] Updated weights for policy 0, policy_version 1366671 (0.0010) [2023-12-27 15:07:02,072][20381] Updated weights for policy 1, policy_version 1365489 (0.0007) [2023-12-27 15:07:02,126][20381] Updated weights for policy 1, policy_version 1365499 (0.0007) [2023-12-27 15:07:02,174][20381] Updated weights for policy 1, policy_version 1365509 (0.0007) [2023-12-27 15:07:02,233][20381] Updated weights for policy 1, policy_version 1365519 (0.0007) [2023-12-27 15:07:02,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 699539456. Throughput: 0: 10336.3, 1: 10185.8. Samples: 699545604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:02,363][19170] Avg episode reward: [(0, '951.050'), (1, '990.230')] [2023-12-27 15:07:02,711][20347] Updated weights for policy 0, policy_version 1366681 (0.0010) [2023-12-27 15:07:02,770][20347] Updated weights for policy 0, policy_version 1366691 (0.0010) [2023-12-27 15:07:02,826][20347] Updated weights for policy 0, policy_version 1366701 (0.0011) [2023-12-27 15:07:02,884][20347] Updated weights for policy 0, policy_version 1366711 (0.0010) [2023-12-27 15:07:02,946][20381] Updated weights for policy 1, policy_version 1365529 (0.0005) [2023-12-27 15:07:03,009][20381] Updated weights for policy 1, policy_version 1365539 (0.0006) [2023-12-27 15:07:03,072][20381] Updated weights for policy 1, policy_version 1365549 (0.0006) [2023-12-27 15:07:03,477][20347] Updated weights for policy 0, policy_version 1366721 (0.0010) [2023-12-27 15:07:03,538][20347] Updated weights for policy 0, policy_version 1366731 (0.0010) [2023-12-27 15:07:03,594][20347] Updated weights for policy 0, policy_version 1366741 (0.0009) [2023-12-27 15:07:03,733][20381] Updated weights for policy 1, policy_version 1365559 (0.0007) [2023-12-27 15:07:03,778][20381] Updated weights for policy 1, policy_version 1365569 (0.0009) [2023-12-27 15:07:03,825][20381] Updated weights for policy 1, policy_version 1365579 (0.0008) [2023-12-27 15:07:04,154][20347] Updated weights for policy 0, policy_version 1366751 (0.0005) [2023-12-27 15:07:04,224][20347] Updated weights for policy 0, policy_version 1366761 (0.0008) [2023-12-27 15:07:04,289][20347] Updated weights for policy 0, policy_version 1366771 (0.0008) [2023-12-27 15:07:04,580][20381] Updated weights for policy 1, policy_version 1365589 (0.0008) [2023-12-27 15:07:04,640][20381] Updated weights for policy 1, policy_version 1365599 (0.0008) [2023-12-27 15:07:04,698][20381] Updated weights for policy 1, policy_version 1365609 (0.0008) [2023-12-27 15:07:04,894][20347] Updated weights for policy 0, policy_version 1366781 (0.0010) [2023-12-27 15:07:04,948][20347] Updated weights for policy 0, policy_version 1366791 (0.0010) [2023-12-27 15:07:05,002][20347] Updated weights for policy 0, policy_version 1366801 (0.0010) [2023-12-27 15:07:05,403][20381] Updated weights for policy 1, policy_version 1365619 (0.0007) [2023-12-27 15:07:05,450][20381] Updated weights for policy 1, policy_version 1365629 (0.0005) [2023-12-27 15:07:05,503][20381] Updated weights for policy 1, policy_version 1365639 (0.0005) [2023-12-27 15:07:05,682][20347] Updated weights for policy 0, policy_version 1366811 (0.0008) [2023-12-27 15:07:05,732][20347] Updated weights for policy 0, policy_version 1366821 (0.0005) [2023-12-27 15:07:05,794][20347] Updated weights for policy 0, policy_version 1366831 (0.0005) [2023-12-27 15:07:06,193][20381] Updated weights for policy 1, policy_version 1365649 (0.0006) [2023-12-27 15:07:06,253][20381] Updated weights for policy 1, policy_version 1365659 (0.0008) [2023-12-27 15:07:06,311][20381] Updated weights for policy 1, policy_version 1365669 (0.0008) [2023-12-27 15:07:06,372][20381] Updated weights for policy 1, policy_version 1365679 (0.0008) [2023-12-27 15:07:06,393][20347] Updated weights for policy 0, policy_version 1366841 (0.0006) [2023-12-27 15:07:06,452][20347] Updated weights for policy 0, policy_version 1366851 (0.0007) [2023-12-27 15:07:06,503][20347] Updated weights for policy 0, policy_version 1366861 (0.0006) [2023-12-27 15:07:06,560][20347] Updated weights for policy 0, policy_version 1366871 (0.0008) [2023-12-27 15:07:07,102][20381] Updated weights for policy 1, policy_version 1365689 (0.0008) [2023-12-27 15:07:07,148][20381] Updated weights for policy 1, policy_version 1365699 (0.0008) [2023-12-27 15:07:07,199][20381] Updated weights for policy 1, policy_version 1365709 (0.0008) [2023-12-27 15:07:07,216][20347] Updated weights for policy 0, policy_version 1366881 (0.0008) [2023-12-27 15:07:07,271][20347] Updated weights for policy 0, policy_version 1366891 (0.0008) [2023-12-27 15:07:07,328][20347] Updated weights for policy 0, policy_version 1366901 (0.0007) [2023-12-27 15:07:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 699645952. Throughput: 0: 10387.3, 1: 10176.9. Samples: 699608076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:07,364][19170] Avg episode reward: [(0, '980.460'), (1, '980.550')] [2023-12-27 15:07:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001365712_349650944.pth... [2023-12-27 15:07:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001366904_349995008.pth... [2023-12-27 15:07:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001364528_349347840.pth [2023-12-27 15:07:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001365688_349683712.pth [2023-12-27 15:07:07,932][20381] Updated weights for policy 1, policy_version 1365719 (0.0006) [2023-12-27 15:07:07,986][20381] Updated weights for policy 1, policy_version 1365729 (0.0006) [2023-12-27 15:07:07,995][20347] Updated weights for policy 0, policy_version 1366911 (0.0008) [2023-12-27 15:07:08,038][20381] Updated weights for policy 1, policy_version 1365739 (0.0005) [2023-12-27 15:07:08,049][20347] Updated weights for policy 0, policy_version 1366921 (0.0007) [2023-12-27 15:07:08,097][20347] Updated weights for policy 0, policy_version 1366931 (0.0007) [2023-12-27 15:07:08,723][20381] Updated weights for policy 1, policy_version 1365749 (0.0008) [2023-12-27 15:07:08,772][20381] Updated weights for policy 1, policy_version 1365759 (0.0008) [2023-12-27 15:07:08,815][20347] Updated weights for policy 0, policy_version 1366941 (0.0006) [2023-12-27 15:07:08,825][20381] Updated weights for policy 1, policy_version 1365769 (0.0007) [2023-12-27 15:07:08,876][20347] Updated weights for policy 0, policy_version 1366951 (0.0008) [2023-12-27 15:07:08,938][20347] Updated weights for policy 0, policy_version 1366961 (0.0007) [2023-12-27 15:07:09,591][20381] Updated weights for policy 1, policy_version 1365779 (0.0008) [2023-12-27 15:07:09,637][20347] Updated weights for policy 0, policy_version 1366971 (0.0008) [2023-12-27 15:07:09,651][20381] Updated weights for policy 1, policy_version 1365789 (0.0007) [2023-12-27 15:07:09,694][20347] Updated weights for policy 0, policy_version 1366981 (0.0006) [2023-12-27 15:07:09,715][20381] Updated weights for policy 1, policy_version 1365799 (0.0006) [2023-12-27 15:07:09,756][20347] Updated weights for policy 0, policy_version 1366991 (0.0007) [2023-12-27 15:07:10,425][20381] Updated weights for policy 1, policy_version 1365809 (0.0006) [2023-12-27 15:07:10,445][20347] Updated weights for policy 0, policy_version 1367001 (0.0009) [2023-12-27 15:07:10,489][20381] Updated weights for policy 1, policy_version 1365819 (0.0008) [2023-12-27 15:07:10,502][20347] Updated weights for policy 0, policy_version 1367011 (0.0006) [2023-12-27 15:07:10,554][20381] Updated weights for policy 1, policy_version 1365829 (0.0008) [2023-12-27 15:07:10,557][20347] Updated weights for policy 0, policy_version 1367021 (0.0007) [2023-12-27 15:07:10,606][20347] Updated weights for policy 0, policy_version 1367031 (0.0005) [2023-12-27 15:07:10,624][20381] Updated weights for policy 1, policy_version 1365839 (0.0009) [2023-12-27 15:07:11,339][20347] Updated weights for policy 0, policy_version 1367041 (0.0008) [2023-12-27 15:07:11,345][20381] Updated weights for policy 1, policy_version 1365849 (0.0007) [2023-12-27 15:07:11,402][20347] Updated weights for policy 0, policy_version 1367051 (0.0008) [2023-12-27 15:07:11,410][20381] Updated weights for policy 1, policy_version 1365859 (0.0009) [2023-12-27 15:07:11,459][20347] Updated weights for policy 0, policy_version 1367061 (0.0008) [2023-12-27 15:07:11,471][20381] Updated weights for policy 1, policy_version 1365869 (0.0007) [2023-12-27 15:07:12,199][20381] Updated weights for policy 1, policy_version 1365879 (0.0008) [2023-12-27 15:07:12,201][20347] Updated weights for policy 0, policy_version 1367071 (0.0007) [2023-12-27 15:07:12,258][20381] Updated weights for policy 1, policy_version 1365889 (0.0007) [2023-12-27 15:07:12,259][20347] Updated weights for policy 0, policy_version 1367081 (0.0006) [2023-12-27 15:07:12,315][20347] Updated weights for policy 0, policy_version 1367091 (0.0008) [2023-12-27 15:07:12,320][20381] Updated weights for policy 1, policy_version 1365899 (0.0007) [2023-12-27 15:07:12,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 699744256. Throughput: 0: 10385.2, 1: 10136.9. Samples: 699727976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:12,363][19170] Avg episode reward: [(0, '970.720'), (1, '990.320')] [2023-12-27 15:07:13,003][20381] Updated weights for policy 1, policy_version 1365909 (0.0006) [2023-12-27 15:07:13,024][20347] Updated weights for policy 0, policy_version 1367101 (0.0008) [2023-12-27 15:07:13,059][20381] Updated weights for policy 1, policy_version 1365919 (0.0005) [2023-12-27 15:07:13,083][20347] Updated weights for policy 0, policy_version 1367111 (0.0007) [2023-12-27 15:07:13,119][20381] Updated weights for policy 1, policy_version 1365929 (0.0006) [2023-12-27 15:07:13,143][20347] Updated weights for policy 0, policy_version 1367121 (0.0006) [2023-12-27 15:07:13,782][20381] Updated weights for policy 1, policy_version 1365939 (0.0006) [2023-12-27 15:07:13,840][20381] Updated weights for policy 1, policy_version 1365949 (0.0007) [2023-12-27 15:07:13,842][20347] Updated weights for policy 0, policy_version 1367131 (0.0007) [2023-12-27 15:07:13,898][20381] Updated weights for policy 1, policy_version 1365959 (0.0006) [2023-12-27 15:07:13,907][20347] Updated weights for policy 0, policy_version 1367141 (0.0007) [2023-12-27 15:07:13,965][20347] Updated weights for policy 0, policy_version 1367151 (0.0008) [2023-12-27 15:07:14,459][20381] Updated weights for policy 1, policy_version 1365969 (0.0006) [2023-12-27 15:07:14,507][20381] Updated weights for policy 1, policy_version 1365979 (0.0008) [2023-12-27 15:07:14,563][20381] Updated weights for policy 1, policy_version 1365989 (0.0008) [2023-12-27 15:07:14,605][20381] Updated weights for policy 1, policy_version 1365999 (0.0006) [2023-12-27 15:07:14,707][20347] Updated weights for policy 0, policy_version 1367161 (0.0008) [2023-12-27 15:07:14,765][20347] Updated weights for policy 0, policy_version 1367171 (0.0009) [2023-12-27 15:07:14,835][20347] Updated weights for policy 0, policy_version 1367181 (0.0008) [2023-12-27 15:07:14,899][20347] Updated weights for policy 0, policy_version 1367191 (0.0006) [2023-12-27 15:07:15,240][20381] Updated weights for policy 1, policy_version 1366009 (0.0007) [2023-12-27 15:07:15,298][20381] Updated weights for policy 1, policy_version 1366019 (0.0008) [2023-12-27 15:07:15,355][20381] Updated weights for policy 1, policy_version 1366029 (0.0008) [2023-12-27 15:07:15,595][20347] Updated weights for policy 0, policy_version 1367201 (0.0008) [2023-12-27 15:07:15,647][20347] Updated weights for policy 0, policy_version 1367211 (0.0008) [2023-12-27 15:07:15,703][20347] Updated weights for policy 0, policy_version 1367221 (0.0008) [2023-12-27 15:07:15,984][20381] Updated weights for policy 1, policy_version 1366039 (0.0008) [2023-12-27 15:07:16,039][20381] Updated weights for policy 1, policy_version 1366049 (0.0008) [2023-12-27 15:07:16,100][20381] Updated weights for policy 1, policy_version 1366059 (0.0009) [2023-12-27 15:07:16,426][20347] Updated weights for policy 0, policy_version 1367231 (0.0008) [2023-12-27 15:07:16,486][20347] Updated weights for policy 0, policy_version 1367241 (0.0007) [2023-12-27 15:07:16,548][20347] Updated weights for policy 0, policy_version 1367251 (0.0007) [2023-12-27 15:07:16,784][20381] Updated weights for policy 1, policy_version 1366069 (0.0010) [2023-12-27 15:07:16,837][20381] Updated weights for policy 1, policy_version 1366079 (0.0006) [2023-12-27 15:07:16,891][20381] Updated weights for policy 1, policy_version 1366089 (0.0006) [2023-12-27 15:07:17,206][20347] Updated weights for policy 0, policy_version 1367261 (0.0006) [2023-12-27 15:07:17,267][20347] Updated weights for policy 0, policy_version 1367271 (0.0005) [2023-12-27 15:07:17,325][20347] Updated weights for policy 0, policy_version 1367281 (0.0005) [2023-12-27 15:07:17,362][19170] Fps is (10 sec: 18842.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 699834368. Throughput: 0: 10262.8, 1: 10092.4. Samples: 699848348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:17,363][19170] Avg episode reward: [(0, '980.490'), (1, '990.330')] [2023-12-27 15:07:17,570][20381] Updated weights for policy 1, policy_version 1366099 (0.0007) [2023-12-27 15:07:17,623][20381] Updated weights for policy 1, policy_version 1366109 (0.0005) [2023-12-27 15:07:17,683][20381] Updated weights for policy 1, policy_version 1366119 (0.0005) [2023-12-27 15:07:17,898][20347] Updated weights for policy 0, policy_version 1367291 (0.0007) [2023-12-27 15:07:17,953][20347] Updated weights for policy 0, policy_version 1367301 (0.0010) [2023-12-27 15:07:18,004][20347] Updated weights for policy 0, policy_version 1367311 (0.0010) [2023-12-27 15:07:18,193][20381] Updated weights for policy 1, policy_version 1366129 (0.0005) [2023-12-27 15:07:18,257][20381] Updated weights for policy 1, policy_version 1366139 (0.0005) [2023-12-27 15:07:18,310][20381] Updated weights for policy 1, policy_version 1366149 (0.0005) [2023-12-27 15:07:18,375][20381] Updated weights for policy 1, policy_version 1366159 (0.0009) [2023-12-27 15:07:18,672][20347] Updated weights for policy 0, policy_version 1367321 (0.0010) [2023-12-27 15:07:18,731][20347] Updated weights for policy 0, policy_version 1367331 (0.0010) [2023-12-27 15:07:18,794][20347] Updated weights for policy 0, policy_version 1367341 (0.0011) [2023-12-27 15:07:18,853][20347] Updated weights for policy 0, policy_version 1367351 (0.0010) [2023-12-27 15:07:19,069][20381] Updated weights for policy 1, policy_version 1366169 (0.0011) [2023-12-27 15:07:19,133][20381] Updated weights for policy 1, policy_version 1366179 (0.0010) [2023-12-27 15:07:19,194][20381] Updated weights for policy 1, policy_version 1366189 (0.0010) [2023-12-27 15:07:19,538][20347] Updated weights for policy 0, policy_version 1367361 (0.0008) [2023-12-27 15:07:19,597][20347] Updated weights for policy 0, policy_version 1367371 (0.0008) [2023-12-27 15:07:19,663][20347] Updated weights for policy 0, policy_version 1367381 (0.0008) [2023-12-27 15:07:19,899][20381] Updated weights for policy 1, policy_version 1366199 (0.0007) [2023-12-27 15:07:19,963][20381] Updated weights for policy 1, policy_version 1366209 (0.0009) [2023-12-27 15:07:20,021][20381] Updated weights for policy 1, policy_version 1366219 (0.0010) [2023-12-27 15:07:20,423][20347] Updated weights for policy 0, policy_version 1367391 (0.0010) [2023-12-27 15:07:20,487][20347] Updated weights for policy 0, policy_version 1367401 (0.0010) [2023-12-27 15:07:20,543][20347] Updated weights for policy 0, policy_version 1367411 (0.0010) [2023-12-27 15:07:20,706][20381] Updated weights for policy 1, policy_version 1366229 (0.0011) [2023-12-27 15:07:20,765][20381] Updated weights for policy 1, policy_version 1366239 (0.0010) [2023-12-27 15:07:20,821][20381] Updated weights for policy 1, policy_version 1366249 (0.0010) [2023-12-27 15:07:21,284][20347] Updated weights for policy 0, policy_version 1367421 (0.0009) [2023-12-27 15:07:21,342][20347] Updated weights for policy 0, policy_version 1367431 (0.0008) [2023-12-27 15:07:21,412][20347] Updated weights for policy 0, policy_version 1367441 (0.0009) [2023-12-27 15:07:21,557][20381] Updated weights for policy 1, policy_version 1366259 (0.0011) [2023-12-27 15:07:21,616][20381] Updated weights for policy 1, policy_version 1366269 (0.0011) [2023-12-27 15:07:21,679][20381] Updated weights for policy 1, policy_version 1366279 (0.0011) [2023-12-27 15:07:22,143][20347] Updated weights for policy 0, policy_version 1367451 (0.0007) [2023-12-27 15:07:22,202][20347] Updated weights for policy 0, policy_version 1367461 (0.0007) [2023-12-27 15:07:22,265][20347] Updated weights for policy 0, policy_version 1367471 (0.0008) [2023-12-27 15:07:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 699940864. Throughput: 0: 10221.2, 1: 10142.2. Samples: 699910876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:22,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 15:07:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001366288_349798400.pth... [2023-12-27 15:07:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001367480_350142464.pth... [2023-12-27 15:07:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001365104_349495296.pth [2023-12-27 15:07:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001366296_349839360.pth [2023-12-27 15:07:22,408][20381] Updated weights for policy 1, policy_version 1366289 (0.0009) [2023-12-27 15:07:22,467][20381] Updated weights for policy 1, policy_version 1366299 (0.0007) [2023-12-27 15:07:22,520][20381] Updated weights for policy 1, policy_version 1366309 (0.0008) [2023-12-27 15:07:22,579][20381] Updated weights for policy 1, policy_version 1366319 (0.0007) [2023-12-27 15:07:22,986][20347] Updated weights for policy 0, policy_version 1367481 (0.0008) [2023-12-27 15:07:23,040][20347] Updated weights for policy 0, policy_version 1367491 (0.0007) [2023-12-27 15:07:23,100][20347] Updated weights for policy 0, policy_version 1367501 (0.0008) [2023-12-27 15:07:23,144][20347] Updated weights for policy 0, policy_version 1367511 (0.0007) [2023-12-27 15:07:23,270][20381] Updated weights for policy 1, policy_version 1366329 (0.0007) [2023-12-27 15:07:23,323][20381] Updated weights for policy 1, policy_version 1366339 (0.0010) [2023-12-27 15:07:23,378][20381] Updated weights for policy 1, policy_version 1366349 (0.0007) [2023-12-27 15:07:23,747][20347] Updated weights for policy 0, policy_version 1367521 (0.0007) [2023-12-27 15:07:23,792][20347] Updated weights for policy 0, policy_version 1367531 (0.0007) [2023-12-27 15:07:23,845][20347] Updated weights for policy 0, policy_version 1367541 (0.0006) [2023-12-27 15:07:24,068][20381] Updated weights for policy 1, policy_version 1366359 (0.0009) [2023-12-27 15:07:24,130][20381] Updated weights for policy 1, policy_version 1366369 (0.0011) [2023-12-27 15:07:24,190][20381] Updated weights for policy 1, policy_version 1366379 (0.0011) [2023-12-27 15:07:24,429][20347] Updated weights for policy 0, policy_version 1367551 (0.0006) [2023-12-27 15:07:24,492][20347] Updated weights for policy 0, policy_version 1367561 (0.0006) [2023-12-27 15:07:24,549][20347] Updated weights for policy 0, policy_version 1367571 (0.0007) [2023-12-27 15:07:24,875][20381] Updated weights for policy 1, policy_version 1366389 (0.0011) [2023-12-27 15:07:24,934][20381] Updated weights for policy 1, policy_version 1366399 (0.0011) [2023-12-27 15:07:24,989][20381] Updated weights for policy 1, policy_version 1366409 (0.0011) [2023-12-27 15:07:25,188][20347] Updated weights for policy 0, policy_version 1367581 (0.0008) [2023-12-27 15:07:25,245][20347] Updated weights for policy 0, policy_version 1367591 (0.0007) [2023-12-27 15:07:25,302][20347] Updated weights for policy 0, policy_version 1367601 (0.0007) [2023-12-27 15:07:25,682][20381] Updated weights for policy 1, policy_version 1366419 (0.0009) [2023-12-27 15:07:25,745][20381] Updated weights for policy 1, policy_version 1366429 (0.0007) [2023-12-27 15:07:25,804][20381] Updated weights for policy 1, policy_version 1366439 (0.0010) [2023-12-27 15:07:25,941][20347] Updated weights for policy 0, policy_version 1367611 (0.0008) [2023-12-27 15:07:25,996][20347] Updated weights for policy 0, policy_version 1367621 (0.0007) [2023-12-27 15:07:26,060][20347] Updated weights for policy 0, policy_version 1367631 (0.0007) [2023-12-27 15:07:26,446][20381] Updated weights for policy 1, policy_version 1366449 (0.0007) [2023-12-27 15:07:26,509][20381] Updated weights for policy 1, policy_version 1366459 (0.0007) [2023-12-27 15:07:26,568][20381] Updated weights for policy 1, policy_version 1366469 (0.0008) [2023-12-27 15:07:26,624][20381] Updated weights for policy 1, policy_version 1366479 (0.0008) [2023-12-27 15:07:26,696][20347] Updated weights for policy 0, policy_version 1367641 (0.0008) [2023-12-27 15:07:26,745][20347] Updated weights for policy 0, policy_version 1367651 (0.0005) [2023-12-27 15:07:26,803][20347] Updated weights for policy 0, policy_version 1367661 (0.0005) [2023-12-27 15:07:26,866][20347] Updated weights for policy 0, policy_version 1367671 (0.0005) [2023-12-27 15:07:27,210][20381] Updated weights for policy 1, policy_version 1366489 (0.0005) [2023-12-27 15:07:27,256][20381] Updated weights for policy 1, policy_version 1366499 (0.0005) [2023-12-27 15:07:27,303][20381] Updated weights for policy 1, policy_version 1366509 (0.0007) [2023-12-27 15:07:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 700047360. Throughput: 0: 10240.9, 1: 10166.2. Samples: 700032552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:27,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 15:07:27,385][20347] Updated weights for policy 0, policy_version 1367681 (0.0008) [2023-12-27 15:07:27,446][20347] Updated weights for policy 0, policy_version 1367691 (0.0007) [2023-12-27 15:07:27,498][20347] Updated weights for policy 0, policy_version 1367701 (0.0005) [2023-12-27 15:07:27,988][20381] Updated weights for policy 1, policy_version 1366519 (0.0007) [2023-12-27 15:07:28,037][20381] Updated weights for policy 1, policy_version 1366529 (0.0005) [2023-12-27 15:07:28,088][20381] Updated weights for policy 1, policy_version 1366539 (0.0005) [2023-12-27 15:07:28,118][20347] Updated weights for policy 0, policy_version 1367711 (0.0006) [2023-12-27 15:07:28,164][20347] Updated weights for policy 0, policy_version 1367721 (0.0009) [2023-12-27 15:07:28,209][20347] Updated weights for policy 0, policy_version 1367731 (0.0008) [2023-12-27 15:07:28,663][20381] Updated weights for policy 1, policy_version 1366549 (0.0006) [2023-12-27 15:07:28,725][20381] Updated weights for policy 1, policy_version 1366559 (0.0008) [2023-12-27 15:07:28,777][20381] Updated weights for policy 1, policy_version 1366569 (0.0011) [2023-12-27 15:07:28,930][20347] Updated weights for policy 0, policy_version 1367741 (0.0006) [2023-12-27 15:07:28,981][20347] Updated weights for policy 0, policy_version 1367751 (0.0007) [2023-12-27 15:07:29,032][20347] Updated weights for policy 0, policy_version 1367761 (0.0007) [2023-12-27 15:07:29,494][20381] Updated weights for policy 1, policy_version 1366579 (0.0009) [2023-12-27 15:07:29,545][20381] Updated weights for policy 1, policy_version 1366589 (0.0011) [2023-12-27 15:07:29,600][20381] Updated weights for policy 1, policy_version 1366599 (0.0011) [2023-12-27 15:07:29,706][20347] Updated weights for policy 0, policy_version 1367771 (0.0008) [2023-12-27 15:07:29,766][20347] Updated weights for policy 0, policy_version 1367781 (0.0010) [2023-12-27 15:07:29,817][20347] Updated weights for policy 0, policy_version 1367791 (0.0009) [2023-12-27 15:07:30,304][20381] Updated weights for policy 1, policy_version 1366609 (0.0011) [2023-12-27 15:07:30,363][20381] Updated weights for policy 1, policy_version 1366619 (0.0011) [2023-12-27 15:07:30,418][20381] Updated weights for policy 1, policy_version 1366629 (0.0011) [2023-12-27 15:07:30,471][20381] Updated weights for policy 1, policy_version 1366639 (0.0009) [2023-12-27 15:07:30,534][20347] Updated weights for policy 0, policy_version 1367801 (0.0007) [2023-12-27 15:07:30,577][20347] Updated weights for policy 0, policy_version 1367811 (0.0006) [2023-12-27 15:07:30,626][20347] Updated weights for policy 0, policy_version 1367821 (0.0005) [2023-12-27 15:07:30,673][20347] Updated weights for policy 0, policy_version 1367831 (0.0006) [2023-12-27 15:07:31,188][20381] Updated weights for policy 1, policy_version 1366649 (0.0008) [2023-12-27 15:07:31,251][20381] Updated weights for policy 1, policy_version 1366659 (0.0011) [2023-12-27 15:07:31,304][20381] Updated weights for policy 1, policy_version 1366669 (0.0009) [2023-12-27 15:07:31,331][20347] Updated weights for policy 0, policy_version 1367841 (0.0006) [2023-12-27 15:07:31,397][20347] Updated weights for policy 0, policy_version 1367851 (0.0008) [2023-12-27 15:07:31,451][20347] Updated weights for policy 0, policy_version 1367861 (0.0007) [2023-12-27 15:07:32,025][20381] Updated weights for policy 1, policy_version 1366679 (0.0009) [2023-12-27 15:07:32,080][20381] Updated weights for policy 1, policy_version 1366689 (0.0007) [2023-12-27 15:07:32,104][20347] Updated weights for policy 0, policy_version 1367871 (0.0006) [2023-12-27 15:07:32,129][20381] Updated weights for policy 1, policy_version 1366699 (0.0005) [2023-12-27 15:07:32,153][20347] Updated weights for policy 0, policy_version 1367881 (0.0006) [2023-12-27 15:07:32,213][20347] Updated weights for policy 0, policy_version 1367891 (0.0009) [2023-12-27 15:07:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 700153856. Throughput: 0: 10278.5, 1: 10175.6. Samples: 700158608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:32,363][19170] Avg episode reward: [(0, '980.520'), (1, '1000.000')] [2023-12-27 15:07:32,854][20381] Updated weights for policy 1, policy_version 1366709 (0.0007) [2023-12-27 15:07:32,903][20347] Updated weights for policy 0, policy_version 1367901 (0.0007) [2023-12-27 15:07:32,909][20381] Updated weights for policy 1, policy_version 1366719 (0.0006) [2023-12-27 15:07:32,955][20347] Updated weights for policy 0, policy_version 1367911 (0.0006) [2023-12-27 15:07:32,963][20381] Updated weights for policy 1, policy_version 1366729 (0.0007) [2023-12-27 15:07:33,013][20347] Updated weights for policy 0, policy_version 1367921 (0.0006) [2023-12-27 15:07:33,637][20381] Updated weights for policy 1, policy_version 1366739 (0.0005) [2023-12-27 15:07:33,644][20347] Updated weights for policy 0, policy_version 1367931 (0.0007) [2023-12-27 15:07:33,686][20347] Updated weights for policy 0, policy_version 1367941 (0.0005) [2023-12-27 15:07:33,687][20381] Updated weights for policy 1, policy_version 1366749 (0.0007) [2023-12-27 15:07:33,735][20347] Updated weights for policy 0, policy_version 1367951 (0.0005) [2023-12-27 15:07:33,736][20381] Updated weights for policy 1, policy_version 1366759 (0.0006) [2023-12-27 15:07:34,297][20347] Updated weights for policy 0, policy_version 1367961 (0.0005) [2023-12-27 15:07:34,360][20347] Updated weights for policy 0, policy_version 1367971 (0.0006) [2023-12-27 15:07:34,425][20347] Updated weights for policy 0, policy_version 1367981 (0.0008) [2023-12-27 15:07:34,448][20381] Updated weights for policy 1, policy_version 1366769 (0.0005) [2023-12-27 15:07:34,490][20347] Updated weights for policy 0, policy_version 1367991 (0.0008) [2023-12-27 15:07:34,511][20381] Updated weights for policy 1, policy_version 1366779 (0.0007) [2023-12-27 15:07:34,574][20381] Updated weights for policy 1, policy_version 1366789 (0.0008) [2023-12-27 15:07:34,635][20381] Updated weights for policy 1, policy_version 1366799 (0.0008) [2023-12-27 15:07:35,158][20347] Updated weights for policy 0, policy_version 1368001 (0.0005) [2023-12-27 15:07:35,205][20347] Updated weights for policy 0, policy_version 1368011 (0.0006) [2023-12-27 15:07:35,257][20347] Updated weights for policy 0, policy_version 1368021 (0.0005) [2023-12-27 15:07:35,295][20381] Updated weights for policy 1, policy_version 1366809 (0.0008) [2023-12-27 15:07:35,344][20381] Updated weights for policy 1, policy_version 1366819 (0.0009) [2023-12-27 15:07:35,393][20381] Updated weights for policy 1, policy_version 1366829 (0.0009) [2023-12-27 15:07:35,809][20347] Updated weights for policy 0, policy_version 1368031 (0.0006) [2023-12-27 15:07:35,863][20347] Updated weights for policy 0, policy_version 1368041 (0.0005) [2023-12-27 15:07:35,908][20347] Updated weights for policy 0, policy_version 1368051 (0.0006) [2023-12-27 15:07:36,107][20381] Updated weights for policy 1, policy_version 1366839 (0.0008) [2023-12-27 15:07:36,164][20381] Updated weights for policy 1, policy_version 1366849 (0.0008) [2023-12-27 15:07:36,219][20381] Updated weights for policy 1, policy_version 1366859 (0.0009) [2023-12-27 15:07:36,604][20347] Updated weights for policy 0, policy_version 1368061 (0.0008) [2023-12-27 15:07:36,658][20347] Updated weights for policy 0, policy_version 1368071 (0.0008) [2023-12-27 15:07:36,714][20347] Updated weights for policy 0, policy_version 1368081 (0.0008) [2023-12-27 15:07:36,910][20381] Updated weights for policy 1, policy_version 1366869 (0.0008) [2023-12-27 15:07:36,963][20381] Updated weights for policy 1, policy_version 1366879 (0.0006) [2023-12-27 15:07:37,017][20381] Updated weights for policy 1, policy_version 1366889 (0.0008) [2023-12-27 15:07:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.6, 300 sec: 20493.9). Total num frames: 700252160. Throughput: 0: 10313.5, 1: 10164.6. Samples: 700221516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:37,363][19170] Avg episode reward: [(0, '960.920'), (1, '1000.000')] [2023-12-27 15:07:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001366896_349954048.pth... [2023-12-27 15:07:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001368088_350298112.pth... [2023-12-27 15:07:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001365712_349650944.pth [2023-12-27 15:07:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001366904_349995008.pth [2023-12-27 15:07:37,403][20347] Updated weights for policy 0, policy_version 1368091 (0.0009) [2023-12-27 15:07:37,465][20347] Updated weights for policy 0, policy_version 1368101 (0.0009) [2023-12-27 15:07:37,521][20347] Updated weights for policy 0, policy_version 1368111 (0.0011) [2023-12-27 15:07:37,677][20381] Updated weights for policy 1, policy_version 1366899 (0.0007) [2023-12-27 15:07:37,728][20381] Updated weights for policy 1, policy_version 1366909 (0.0007) [2023-12-27 15:07:37,785][20381] Updated weights for policy 1, policy_version 1366919 (0.0008) [2023-12-27 15:07:38,172][20347] Updated weights for policy 0, policy_version 1368121 (0.0010) [2023-12-27 15:07:38,224][20347] Updated weights for policy 0, policy_version 1368131 (0.0007) [2023-12-27 15:07:38,274][20347] Updated weights for policy 0, policy_version 1368141 (0.0008) [2023-12-27 15:07:38,322][20347] Updated weights for policy 0, policy_version 1368151 (0.0006) [2023-12-27 15:07:38,397][20381] Updated weights for policy 1, policy_version 1366929 (0.0007) [2023-12-27 15:07:38,456][20381] Updated weights for policy 1, policy_version 1366939 (0.0009) [2023-12-27 15:07:38,513][20381] Updated weights for policy 1, policy_version 1366949 (0.0007) [2023-12-27 15:07:38,571][20381] Updated weights for policy 1, policy_version 1366959 (0.0006) [2023-12-27 15:07:39,069][20347] Updated weights for policy 0, policy_version 1368161 (0.0007) [2023-12-27 15:07:39,118][20347] Updated weights for policy 0, policy_version 1368171 (0.0005) [2023-12-27 15:07:39,170][20347] Updated weights for policy 0, policy_version 1368181 (0.0005) [2023-12-27 15:07:39,210][20381] Updated weights for policy 1, policy_version 1366969 (0.0008) [2023-12-27 15:07:39,278][20381] Updated weights for policy 1, policy_version 1366979 (0.0008) [2023-12-27 15:07:39,346][20381] Updated weights for policy 1, policy_version 1366989 (0.0007) [2023-12-27 15:07:39,777][20347] Updated weights for policy 0, policy_version 1368191 (0.0007) [2023-12-27 15:07:39,835][20347] Updated weights for policy 0, policy_version 1368201 (0.0008) [2023-12-27 15:07:39,901][20347] Updated weights for policy 0, policy_version 1368211 (0.0009) [2023-12-27 15:07:40,059][20381] Updated weights for policy 1, policy_version 1366999 (0.0008) [2023-12-27 15:07:40,122][20381] Updated weights for policy 1, policy_version 1367009 (0.0008) [2023-12-27 15:07:40,184][20381] Updated weights for policy 1, policy_version 1367019 (0.0008) [2023-12-27 15:07:40,611][20347] Updated weights for policy 0, policy_version 1368221 (0.0007) [2023-12-27 15:07:40,668][20347] Updated weights for policy 0, policy_version 1368231 (0.0008) [2023-12-27 15:07:40,717][20347] Updated weights for policy 0, policy_version 1368241 (0.0008) [2023-12-27 15:07:40,840][20381] Updated weights for policy 1, policy_version 1367029 (0.0008) [2023-12-27 15:07:40,894][20381] Updated weights for policy 1, policy_version 1367039 (0.0007) [2023-12-27 15:07:40,945][20381] Updated weights for policy 1, policy_version 1367049 (0.0007) [2023-12-27 15:07:41,479][20347] Updated weights for policy 0, policy_version 1368251 (0.0009) [2023-12-27 15:07:41,540][20347] Updated weights for policy 0, policy_version 1368261 (0.0008) [2023-12-27 15:07:41,601][20347] Updated weights for policy 0, policy_version 1368271 (0.0008) [2023-12-27 15:07:41,726][20381] Updated weights for policy 1, policy_version 1367059 (0.0008) [2023-12-27 15:07:41,785][20381] Updated weights for policy 1, policy_version 1367069 (0.0008) [2023-12-27 15:07:41,847][20381] Updated weights for policy 1, policy_version 1367079 (0.0007) [2023-12-27 15:07:42,345][20347] Updated weights for policy 0, policy_version 1368281 (0.0008) [2023-12-27 15:07:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 700350464. Throughput: 0: 10314.2, 1: 10152.6. Samples: 700345156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:42,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 15:07:42,410][20347] Updated weights for policy 0, policy_version 1368291 (0.0008) [2023-12-27 15:07:42,462][20347] Updated weights for policy 0, policy_version 1368301 (0.0007) [2023-12-27 15:07:42,511][20381] Updated weights for policy 1, policy_version 1367089 (0.0006) [2023-12-27 15:07:42,514][20347] Updated weights for policy 0, policy_version 1368311 (0.0007) [2023-12-27 15:07:42,563][20381] Updated weights for policy 1, policy_version 1367099 (0.0008) [2023-12-27 15:07:42,617][20381] Updated weights for policy 1, policy_version 1367109 (0.0008) [2023-12-27 15:07:42,678][20381] Updated weights for policy 1, policy_version 1367119 (0.0007) [2023-12-27 15:07:43,206][20347] Updated weights for policy 0, policy_version 1368321 (0.0006) [2023-12-27 15:07:43,261][20347] Updated weights for policy 0, policy_version 1368331 (0.0006) [2023-12-27 15:07:43,318][20347] Updated weights for policy 0, policy_version 1368341 (0.0006) [2023-12-27 15:07:43,393][20381] Updated weights for policy 1, policy_version 1367129 (0.0008) [2023-12-27 15:07:43,443][20381] Updated weights for policy 1, policy_version 1367139 (0.0005) [2023-12-27 15:07:43,499][20381] Updated weights for policy 1, policy_version 1367149 (0.0006) [2023-12-27 15:07:43,913][20347] Updated weights for policy 0, policy_version 1368351 (0.0009) [2023-12-27 15:07:43,973][20347] Updated weights for policy 0, policy_version 1368361 (0.0010) [2023-12-27 15:07:44,037][20347] Updated weights for policy 0, policy_version 1368371 (0.0011) [2023-12-27 15:07:44,153][20381] Updated weights for policy 1, policy_version 1367159 (0.0007) [2023-12-27 15:07:44,208][20381] Updated weights for policy 1, policy_version 1367169 (0.0006) [2023-12-27 15:07:44,262][20381] Updated weights for policy 1, policy_version 1367179 (0.0005) [2023-12-27 15:07:44,698][20347] Updated weights for policy 0, policy_version 1368381 (0.0008) [2023-12-27 15:07:44,755][20347] Updated weights for policy 0, policy_version 1368391 (0.0005) [2023-12-27 15:07:44,817][20347] Updated weights for policy 0, policy_version 1368401 (0.0008) [2023-12-27 15:07:44,873][20381] Updated weights for policy 1, policy_version 1367189 (0.0005) [2023-12-27 15:07:44,931][20381] Updated weights for policy 1, policy_version 1367199 (0.0008) [2023-12-27 15:07:44,991][20381] Updated weights for policy 1, policy_version 1367209 (0.0008) [2023-12-27 15:07:45,524][20347] Updated weights for policy 0, policy_version 1368411 (0.0006) [2023-12-27 15:07:45,583][20347] Updated weights for policy 0, policy_version 1368421 (0.0005) [2023-12-27 15:07:45,638][20347] Updated weights for policy 0, policy_version 1368431 (0.0005) [2023-12-27 15:07:45,677][20381] Updated weights for policy 1, policy_version 1367219 (0.0008) [2023-12-27 15:07:45,733][20381] Updated weights for policy 1, policy_version 1367229 (0.0009) [2023-12-27 15:07:45,783][20381] Updated weights for policy 1, policy_version 1367239 (0.0007) [2023-12-27 15:07:46,241][20347] Updated weights for policy 0, policy_version 1368441 (0.0006) [2023-12-27 15:07:46,293][20347] Updated weights for policy 0, policy_version 1368451 (0.0005) [2023-12-27 15:07:46,344][20347] Updated weights for policy 0, policy_version 1368461 (0.0005) [2023-12-27 15:07:46,400][20347] Updated weights for policy 0, policy_version 1368471 (0.0009) [2023-12-27 15:07:46,435][20381] Updated weights for policy 1, policy_version 1367249 (0.0007) [2023-12-27 15:07:46,480][20381] Updated weights for policy 1, policy_version 1367259 (0.0005) [2023-12-27 15:07:46,528][20381] Updated weights for policy 1, policy_version 1367269 (0.0005) [2023-12-27 15:07:46,578][20381] Updated weights for policy 1, policy_version 1367279 (0.0005) [2023-12-27 15:07:47,048][20347] Updated weights for policy 0, policy_version 1368481 (0.0010) [2023-12-27 15:07:47,106][20347] Updated weights for policy 0, policy_version 1368491 (0.0010) [2023-12-27 15:07:47,165][20347] Updated weights for policy 0, policy_version 1368501 (0.0010) [2023-12-27 15:07:47,237][20381] Updated weights for policy 1, policy_version 1367289 (0.0010) [2023-12-27 15:07:47,297][20381] Updated weights for policy 1, policy_version 1367299 (0.0011) [2023-12-27 15:07:47,358][20381] Updated weights for policy 1, policy_version 1367309 (0.0010) [2023-12-27 15:07:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 700456960. Throughput: 0: 10347.7, 1: 10173.0. Samples: 700469036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:47,364][19170] Avg episode reward: [(0, '961.030'), (1, '1000.000')] [2023-12-27 15:07:47,837][20347] Updated weights for policy 0, policy_version 1368511 (0.0007) [2023-12-27 15:07:47,894][20347] Updated weights for policy 0, policy_version 1368521 (0.0005) [2023-12-27 15:07:47,956][20347] Updated weights for policy 0, policy_version 1368531 (0.0006) [2023-12-27 15:07:47,998][20381] Updated weights for policy 1, policy_version 1367319 (0.0007) [2023-12-27 15:07:48,065][20381] Updated weights for policy 1, policy_version 1367329 (0.0009) [2023-12-27 15:07:48,113][20381] Updated weights for policy 1, policy_version 1367339 (0.0008) [2023-12-27 15:07:48,559][20347] Updated weights for policy 0, policy_version 1368541 (0.0010) [2023-12-27 15:07:48,620][20347] Updated weights for policy 0, policy_version 1368551 (0.0010) [2023-12-27 15:07:48,676][20347] Updated weights for policy 0, policy_version 1368561 (0.0010) [2023-12-27 15:07:48,747][20381] Updated weights for policy 1, policy_version 1367349 (0.0007) [2023-12-27 15:07:48,809][20381] Updated weights for policy 1, policy_version 1367359 (0.0008) [2023-12-27 15:07:48,870][20381] Updated weights for policy 1, policy_version 1367369 (0.0007) [2023-12-27 15:07:49,299][20347] Updated weights for policy 0, policy_version 1368571 (0.0010) [2023-12-27 15:07:49,372][20347] Updated weights for policy 0, policy_version 1368581 (0.0010) [2023-12-27 15:07:49,429][20347] Updated weights for policy 0, policy_version 1368591 (0.0008) [2023-12-27 15:07:49,658][20381] Updated weights for policy 1, policy_version 1367379 (0.0007) [2023-12-27 15:07:49,719][20381] Updated weights for policy 1, policy_version 1367389 (0.0007) [2023-12-27 15:07:49,776][20381] Updated weights for policy 1, policy_version 1367399 (0.0008) [2023-12-27 15:07:50,150][20347] Updated weights for policy 0, policy_version 1368601 (0.0009) [2023-12-27 15:07:50,205][20347] Updated weights for policy 0, policy_version 1368611 (0.0007) [2023-12-27 15:07:50,266][20347] Updated weights for policy 0, policy_version 1368621 (0.0008) [2023-12-27 15:07:50,326][20347] Updated weights for policy 0, policy_version 1368631 (0.0008) [2023-12-27 15:07:50,422][20381] Updated weights for policy 1, policy_version 1367409 (0.0007) [2023-12-27 15:07:50,476][20381] Updated weights for policy 1, policy_version 1367419 (0.0005) [2023-12-27 15:07:50,527][20381] Updated weights for policy 1, policy_version 1367429 (0.0005) [2023-12-27 15:07:50,580][20381] Updated weights for policy 1, policy_version 1367439 (0.0007) [2023-12-27 15:07:50,995][20347] Updated weights for policy 0, policy_version 1368641 (0.0009) [2023-12-27 15:07:51,058][20347] Updated weights for policy 0, policy_version 1368651 (0.0010) [2023-12-27 15:07:51,119][20347] Updated weights for policy 0, policy_version 1368661 (0.0010) [2023-12-27 15:07:51,273][20381] Updated weights for policy 1, policy_version 1367449 (0.0008) [2023-12-27 15:07:51,332][20381] Updated weights for policy 1, policy_version 1367459 (0.0006) [2023-12-27 15:07:51,395][20381] Updated weights for policy 1, policy_version 1367469 (0.0008) [2023-12-27 15:07:51,872][20347] Updated weights for policy 0, policy_version 1368671 (0.0009) [2023-12-27 15:07:51,935][20347] Updated weights for policy 0, policy_version 1368681 (0.0008) [2023-12-27 15:07:51,984][20347] Updated weights for policy 0, policy_version 1368691 (0.0008) [2023-12-27 15:07:52,072][20381] Updated weights for policy 1, policy_version 1367479 (0.0008) [2023-12-27 15:07:52,121][20381] Updated weights for policy 1, policy_version 1367489 (0.0007) [2023-12-27 15:07:52,176][20381] Updated weights for policy 1, policy_version 1367499 (0.0006) [2023-12-27 15:07:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 700563456. Throughput: 0: 10313.9, 1: 10182.9. Samples: 700530428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:52,363][19170] Avg episode reward: [(0, '951.160'), (1, '990.380')] [2023-12-27 15:07:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001368696_350453760.pth... [2023-12-27 15:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001367504_350109696.pth... [2023-12-27 15:07:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001366288_349798400.pth [2023-12-27 15:07:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001367480_350142464.pth [2023-12-27 15:07:52,653][20347] Updated weights for policy 0, policy_version 1368701 (0.0006) [2023-12-27 15:07:52,713][20347] Updated weights for policy 0, policy_version 1368711 (0.0007) [2023-12-27 15:07:52,766][20347] Updated weights for policy 0, policy_version 1368721 (0.0008) [2023-12-27 15:07:52,960][20381] Updated weights for policy 1, policy_version 1367509 (0.0008) [2023-12-27 15:07:53,015][20381] Updated weights for policy 1, policy_version 1367519 (0.0007) [2023-12-27 15:07:53,067][20381] Updated weights for policy 1, policy_version 1367529 (0.0008) [2023-12-27 15:07:53,386][20347] Updated weights for policy 0, policy_version 1368731 (0.0008) [2023-12-27 15:07:53,434][20347] Updated weights for policy 0, policy_version 1368741 (0.0010) [2023-12-27 15:07:53,488][20347] Updated weights for policy 0, policy_version 1368751 (0.0010) [2023-12-27 15:07:53,740][20381] Updated weights for policy 1, policy_version 1367539 (0.0010) [2023-12-27 15:07:53,801][20381] Updated weights for policy 1, policy_version 1367549 (0.0010) [2023-12-27 15:07:53,859][20381] Updated weights for policy 1, policy_version 1367559 (0.0009) [2023-12-27 15:07:54,102][20347] Updated weights for policy 0, policy_version 1368761 (0.0010) [2023-12-27 15:07:54,166][20347] Updated weights for policy 0, policy_version 1368771 (0.0005) [2023-12-27 15:07:54,230][20347] Updated weights for policy 0, policy_version 1368781 (0.0005) [2023-12-27 15:07:54,293][20347] Updated weights for policy 0, policy_version 1368791 (0.0005) [2023-12-27 15:07:54,503][20381] Updated weights for policy 1, policy_version 1367569 (0.0009) [2023-12-27 15:07:54,559][20381] Updated weights for policy 1, policy_version 1367579 (0.0008) [2023-12-27 15:07:54,614][20381] Updated weights for policy 1, policy_version 1367589 (0.0006) [2023-12-27 15:07:54,663][20381] Updated weights for policy 1, policy_version 1367599 (0.0008) [2023-12-27 15:07:54,805][20347] Updated weights for policy 0, policy_version 1368801 (0.0005) [2023-12-27 15:07:54,853][20347] Updated weights for policy 0, policy_version 1368811 (0.0005) [2023-12-27 15:07:54,907][20347] Updated weights for policy 0, policy_version 1368821 (0.0005) [2023-12-27 15:07:55,275][20381] Updated weights for policy 1, policy_version 1367609 (0.0006) [2023-12-27 15:07:55,330][20381] Updated weights for policy 1, policy_version 1367619 (0.0005) [2023-12-27 15:07:55,391][20381] Updated weights for policy 1, policy_version 1367629 (0.0005) [2023-12-27 15:07:55,454][20347] Updated weights for policy 0, policy_version 1368831 (0.0006) [2023-12-27 15:07:55,507][20347] Updated weights for policy 0, policy_version 1368841 (0.0005) [2023-12-27 15:07:55,552][20347] Updated weights for policy 0, policy_version 1368851 (0.0007) [2023-12-27 15:07:55,981][20381] Updated weights for policy 1, policy_version 1367639 (0.0006) [2023-12-27 15:07:56,031][20381] Updated weights for policy 1, policy_version 1367649 (0.0008) [2023-12-27 15:07:56,081][20381] Updated weights for policy 1, policy_version 1367659 (0.0008) [2023-12-27 15:07:56,267][20347] Updated weights for policy 0, policy_version 1368861 (0.0008) [2023-12-27 15:07:56,332][20347] Updated weights for policy 0, policy_version 1368871 (0.0008) [2023-12-27 15:07:56,381][20347] Updated weights for policy 0, policy_version 1368881 (0.0010) [2023-12-27 15:07:56,743][20381] Updated weights for policy 1, policy_version 1367669 (0.0008) [2023-12-27 15:07:56,801][20381] Updated weights for policy 1, policy_version 1367679 (0.0008) [2023-12-27 15:07:56,862][20381] Updated weights for policy 1, policy_version 1367689 (0.0007) [2023-12-27 15:07:57,075][20347] Updated weights for policy 0, policy_version 1368891 (0.0010) [2023-12-27 15:07:57,129][20347] Updated weights for policy 0, policy_version 1368901 (0.0007) [2023-12-27 15:07:57,187][20347] Updated weights for policy 0, policy_version 1368911 (0.0008) [2023-12-27 15:07:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 700669952. Throughput: 0: 10383.5, 1: 10271.4. Samples: 700657444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:07:57,363][19170] Avg episode reward: [(0, '980.460'), (1, '990.380')] [2023-12-27 15:07:57,549][20381] Updated weights for policy 1, policy_version 1367699 (0.0007) [2023-12-27 15:07:57,593][20381] Updated weights for policy 1, policy_version 1367709 (0.0008) [2023-12-27 15:07:57,651][20381] Updated weights for policy 1, policy_version 1367719 (0.0006) [2023-12-27 15:07:57,844][20347] Updated weights for policy 0, policy_version 1368921 (0.0008) [2023-12-27 15:07:57,896][20347] Updated weights for policy 0, policy_version 1368931 (0.0008) [2023-12-27 15:07:57,960][20347] Updated weights for policy 0, policy_version 1368942 (0.0007) [2023-12-27 15:07:58,019][20347] Updated weights for policy 0, policy_version 1368952 (0.0005) [2023-12-27 15:07:58,186][20381] Updated weights for policy 1, policy_version 1367729 (0.0005) [2023-12-27 15:07:58,247][20381] Updated weights for policy 1, policy_version 1367739 (0.0008) [2023-12-27 15:07:58,303][20381] Updated weights for policy 1, policy_version 1367749 (0.0007) [2023-12-27 15:07:58,369][20381] Updated weights for policy 1, policy_version 1367759 (0.0008) [2023-12-27 15:07:58,679][20347] Updated weights for policy 0, policy_version 1368963 (0.0008) [2023-12-27 15:07:58,744][20347] Updated weights for policy 0, policy_version 1368973 (0.0007) [2023-12-27 15:07:58,810][20347] Updated weights for policy 0, policy_version 1368983 (0.0009) [2023-12-27 15:07:59,155][20381] Updated weights for policy 1, policy_version 1367769 (0.0008) [2023-12-27 15:07:59,218][20381] Updated weights for policy 1, policy_version 1367779 (0.0007) [2023-12-27 15:07:59,280][20381] Updated weights for policy 1, policy_version 1367789 (0.0007) [2023-12-27 15:07:59,539][20347] Updated weights for policy 0, policy_version 1368993 (0.0010) [2023-12-27 15:07:59,594][20347] Updated weights for policy 0, policy_version 1369003 (0.0010) [2023-12-27 15:07:59,659][20347] Updated weights for policy 0, policy_version 1369013 (0.0010) [2023-12-27 15:07:59,926][20381] Updated weights for policy 1, policy_version 1367799 (0.0007) [2023-12-27 15:07:59,989][20381] Updated weights for policy 1, policy_version 1367809 (0.0008) [2023-12-27 15:08:00,045][20381] Updated weights for policy 1, policy_version 1367819 (0.0008) [2023-12-27 15:08:00,377][20347] Updated weights for policy 0, policy_version 1369023 (0.0008) [2023-12-27 15:08:00,435][20347] Updated weights for policy 0, policy_version 1369033 (0.0005) [2023-12-27 15:08:00,486][20347] Updated weights for policy 0, policy_version 1369043 (0.0008) [2023-12-27 15:08:00,705][20381] Updated weights for policy 1, policy_version 1367829 (0.0006) [2023-12-27 15:08:00,758][20381] Updated weights for policy 1, policy_version 1367839 (0.0008) [2023-12-27 15:08:00,820][20381] Updated weights for policy 1, policy_version 1367849 (0.0005) [2023-12-27 15:08:01,097][20347] Updated weights for policy 0, policy_version 1369053 (0.0007) [2023-12-27 15:08:01,161][20347] Updated weights for policy 0, policy_version 1369063 (0.0008) [2023-12-27 15:08:01,217][20347] Updated weights for policy 0, policy_version 1369073 (0.0009) [2023-12-27 15:08:01,436][20381] Updated weights for policy 1, policy_version 1367859 (0.0006) [2023-12-27 15:08:01,493][20381] Updated weights for policy 1, policy_version 1367869 (0.0008) [2023-12-27 15:08:01,547][20381] Updated weights for policy 1, policy_version 1367879 (0.0007) [2023-12-27 15:08:01,905][20347] Updated weights for policy 0, policy_version 1369083 (0.0010) [2023-12-27 15:08:01,958][20347] Updated weights for policy 0, policy_version 1369093 (0.0010) [2023-12-27 15:08:02,017][20347] Updated weights for policy 0, policy_version 1369103 (0.0011) [2023-12-27 15:08:02,215][20381] Updated weights for policy 1, policy_version 1367889 (0.0006) [2023-12-27 15:08:02,282][20381] Updated weights for policy 1, policy_version 1367899 (0.0006) [2023-12-27 15:08:02,337][20381] Updated weights for policy 1, policy_version 1367909 (0.0007) [2023-12-27 15:08:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 700768256. Throughput: 0: 10440.9, 1: 10280.6. Samples: 700780816. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '990.250')] [2023-12-27 15:08:02,398][20381] Updated weights for policy 1, policy_version 1367919 (0.0009) [2023-12-27 15:08:02,709][20347] Updated weights for policy 0, policy_version 1369113 (0.0010) [2023-12-27 15:08:02,761][20347] Updated weights for policy 0, policy_version 1369123 (0.0010) [2023-12-27 15:08:02,809][20347] Updated weights for policy 0, policy_version 1369133 (0.0010) [2023-12-27 15:08:02,857][20347] Updated weights for policy 0, policy_version 1369143 (0.0007) [2023-12-27 15:08:03,012][20381] Updated weights for policy 1, policy_version 1367929 (0.0009) [2023-12-27 15:08:03,070][20381] Updated weights for policy 1, policy_version 1367939 (0.0007) [2023-12-27 15:08:03,127][20381] Updated weights for policy 1, policy_version 1367949 (0.0006) [2023-12-27 15:08:03,514][20347] Updated weights for policy 0, policy_version 1369153 (0.0010) [2023-12-27 15:08:03,564][20347] Updated weights for policy 0, policy_version 1369163 (0.0009) [2023-12-27 15:08:03,617][20347] Updated weights for policy 0, policy_version 1369173 (0.0010) [2023-12-27 15:08:03,715][20381] Updated weights for policy 1, policy_version 1367959 (0.0007) [2023-12-27 15:08:03,769][20381] Updated weights for policy 1, policy_version 1367969 (0.0007) [2023-12-27 15:08:03,829][20381] Updated weights for policy 1, policy_version 1367979 (0.0007) [2023-12-27 15:08:04,336][20347] Updated weights for policy 0, policy_version 1369183 (0.0010) [2023-12-27 15:08:04,403][20347] Updated weights for policy 0, policy_version 1369193 (0.0011) [2023-12-27 15:08:04,459][20347] Updated weights for policy 0, policy_version 1369203 (0.0010) [2023-12-27 15:08:04,510][20381] Updated weights for policy 1, policy_version 1367989 (0.0008) [2023-12-27 15:08:04,564][20381] Updated weights for policy 1, policy_version 1367999 (0.0005) [2023-12-27 15:08:04,617][20381] Updated weights for policy 1, policy_version 1368009 (0.0005) [2023-12-27 15:08:05,107][20347] Updated weights for policy 0, policy_version 1369213 (0.0010) [2023-12-27 15:08:05,158][20347] Updated weights for policy 0, policy_version 1369223 (0.0010) [2023-12-27 15:08:05,189][20381] Updated weights for policy 1, policy_version 1368019 (0.0007) [2023-12-27 15:08:05,217][20347] Updated weights for policy 0, policy_version 1369233 (0.0010) [2023-12-27 15:08:05,242][20381] Updated weights for policy 1, policy_version 1368029 (0.0005) [2023-12-27 15:08:05,296][20381] Updated weights for policy 1, policy_version 1368039 (0.0007) [2023-12-27 15:08:05,903][20347] Updated weights for policy 0, policy_version 1369243 (0.0010) [2023-12-27 15:08:05,947][20381] Updated weights for policy 1, policy_version 1368049 (0.0007) [2023-12-27 15:08:05,964][20347] Updated weights for policy 0, policy_version 1369253 (0.0010) [2023-12-27 15:08:06,000][20381] Updated weights for policy 1, policy_version 1368059 (0.0006) [2023-12-27 15:08:06,030][20347] Updated weights for policy 0, policy_version 1369263 (0.0010) [2023-12-27 15:08:06,050][20381] Updated weights for policy 1, policy_version 1368069 (0.0008) [2023-12-27 15:08:06,107][20381] Updated weights for policy 1, policy_version 1368079 (0.0007) [2023-12-27 15:08:06,734][20347] Updated weights for policy 0, policy_version 1369273 (0.0010) [2023-12-27 15:08:06,792][20347] Updated weights for policy 0, policy_version 1369283 (0.0010) [2023-12-27 15:08:06,803][20381] Updated weights for policy 1, policy_version 1368089 (0.0006) [2023-12-27 15:08:06,852][20347] Updated weights for policy 0, policy_version 1369293 (0.0008) [2023-12-27 15:08:06,861][20381] Updated weights for policy 1, policy_version 1368099 (0.0008) [2023-12-27 15:08:06,907][20347] Updated weights for policy 0, policy_version 1369303 (0.0005) [2023-12-27 15:08:06,918][20381] Updated weights for policy 1, policy_version 1368109 (0.0008) [2023-12-27 15:08:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 700874752. Throughput: 0: 10445.5, 1: 10314.5. Samples: 700845076. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:07,363][19170] Avg episode reward: [(0, '951.400'), (1, '990.250')] [2023-12-27 15:08:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001368112_350265344.pth... [2023-12-27 15:08:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001369304_350609408.pth... [2023-12-27 15:08:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001366896_349954048.pth [2023-12-27 15:08:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001368088_350298112.pth [2023-12-27 15:08:07,551][20347] Updated weights for policy 0, policy_version 1369313 (0.0005) [2023-12-27 15:08:07,569][20381] Updated weights for policy 1, policy_version 1368119 (0.0007) [2023-12-27 15:08:07,610][20347] Updated weights for policy 0, policy_version 1369323 (0.0005) [2023-12-27 15:08:07,617][20381] Updated weights for policy 1, policy_version 1368129 (0.0008) [2023-12-27 15:08:07,660][20347] Updated weights for policy 0, policy_version 1369333 (0.0005) [2023-12-27 15:08:07,670][20381] Updated weights for policy 1, policy_version 1368139 (0.0008) [2023-12-27 15:08:08,237][20347] Updated weights for policy 0, policy_version 1369343 (0.0009) [2023-12-27 15:08:08,288][20347] Updated weights for policy 0, policy_version 1369353 (0.0010) [2023-12-27 15:08:08,330][20381] Updated weights for policy 1, policy_version 1368149 (0.0007) [2023-12-27 15:08:08,341][20347] Updated weights for policy 0, policy_version 1369363 (0.0010) [2023-12-27 15:08:08,397][20381] Updated weights for policy 1, policy_version 1368159 (0.0007) [2023-12-27 15:08:08,452][20381] Updated weights for policy 1, policy_version 1368169 (0.0007) [2023-12-27 15:08:09,053][20381] Updated weights for policy 1, policy_version 1368179 (0.0007) [2023-12-27 15:08:09,092][20347] Updated weights for policy 0, policy_version 1369373 (0.0008) [2023-12-27 15:08:09,110][20381] Updated weights for policy 1, policy_version 1368189 (0.0005) [2023-12-27 15:08:09,148][20347] Updated weights for policy 0, policy_version 1369383 (0.0008) [2023-12-27 15:08:09,162][20381] Updated weights for policy 1, policy_version 1368199 (0.0005) [2023-12-27 15:08:09,204][20347] Updated weights for policy 0, policy_version 1369393 (0.0007) [2023-12-27 15:08:09,872][20381] Updated weights for policy 1, policy_version 1368209 (0.0007) [2023-12-27 15:08:09,888][20347] Updated weights for policy 0, policy_version 1369403 (0.0007) [2023-12-27 15:08:09,935][20381] Updated weights for policy 1, policy_version 1368219 (0.0008) [2023-12-27 15:08:09,964][20347] Updated weights for policy 0, policy_version 1369413 (0.0008) [2023-12-27 15:08:09,995][20381] Updated weights for policy 1, policy_version 1368229 (0.0008) [2023-12-27 15:08:10,021][20347] Updated weights for policy 0, policy_version 1369423 (0.0005) [2023-12-27 15:08:10,052][20381] Updated weights for policy 1, policy_version 1368239 (0.0008) [2023-12-27 15:08:10,693][20381] Updated weights for policy 1, policy_version 1368249 (0.0010) [2023-12-27 15:08:10,725][20347] Updated weights for policy 0, policy_version 1369433 (0.0008) [2023-12-27 15:08:10,743][20381] Updated weights for policy 1, policy_version 1368259 (0.0010) [2023-12-27 15:08:10,761][20291] KL-divergence is very high: 150.6039 [2023-12-27 15:08:10,778][20347] Updated weights for policy 0, policy_version 1369443 (0.0007) [2023-12-27 15:08:10,796][20381] Updated weights for policy 1, policy_version 1368269 (0.0010) [2023-12-27 15:08:10,809][20291] KL-divergence is very high: 149.3101 [2023-12-27 15:08:10,838][20347] Updated weights for policy 0, policy_version 1369453 (0.0007) [2023-12-27 15:08:10,904][20347] Updated weights for policy 0, policy_version 1369463 (0.0007) [2023-12-27 15:08:11,512][20381] Updated weights for policy 1, policy_version 1368279 (0.0011) [2023-12-27 15:08:11,569][20381] Updated weights for policy 1, policy_version 1368289 (0.0011) [2023-12-27 15:08:11,632][20381] Updated weights for policy 1, policy_version 1368299 (0.0010) [2023-12-27 15:08:11,670][20347] Updated weights for policy 0, policy_version 1369473 (0.0008) [2023-12-27 15:08:11,737][20347] Updated weights for policy 0, policy_version 1369483 (0.0008) [2023-12-27 15:08:11,802][20347] Updated weights for policy 0, policy_version 1369493 (0.0007) [2023-12-27 15:08:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 700973056. Throughput: 0: 10397.9, 1: 10389.1. Samples: 700967964. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:12,363][19170] Avg episode reward: [(0, '892.590'), (1, '1000.000')] [2023-12-27 15:08:12,409][20381] Updated weights for policy 1, policy_version 1368309 (0.0008) [2023-12-27 15:08:12,466][20381] Updated weights for policy 1, policy_version 1368319 (0.0008) [2023-12-27 15:08:12,525][20381] Updated weights for policy 1, policy_version 1368329 (0.0006) [2023-12-27 15:08:12,619][20347] Updated weights for policy 0, policy_version 1369503 (0.0006) [2023-12-27 15:08:12,682][20347] Updated weights for policy 0, policy_version 1369513 (0.0005) [2023-12-27 15:08:12,741][20347] Updated weights for policy 0, policy_version 1369523 (0.0006) [2023-12-27 15:08:13,222][20381] Updated weights for policy 1, policy_version 1368339 (0.0009) [2023-12-27 15:08:13,279][20381] Updated weights for policy 1, policy_version 1368349 (0.0008) [2023-12-27 15:08:13,337][20381] Updated weights for policy 1, policy_version 1368359 (0.0007) [2023-12-27 15:08:13,362][20347] Updated weights for policy 0, policy_version 1369533 (0.0005) [2023-12-27 15:08:13,411][20347] Updated weights for policy 0, policy_version 1369543 (0.0007) [2023-12-27 15:08:13,459][20347] Updated weights for policy 0, policy_version 1369553 (0.0007) [2023-12-27 15:08:14,016][20381] Updated weights for policy 1, policy_version 1368369 (0.0006) [2023-12-27 15:08:14,070][20381] Updated weights for policy 1, policy_version 1368379 (0.0007) [2023-12-27 15:08:14,119][20347] Updated weights for policy 0, policy_version 1369563 (0.0007) [2023-12-27 15:08:14,121][20381] Updated weights for policy 1, policy_version 1368389 (0.0007) [2023-12-27 15:08:14,177][20347] Updated weights for policy 0, policy_version 1369573 (0.0006) [2023-12-27 15:08:14,179][20381] Updated weights for policy 1, policy_version 1368399 (0.0006) [2023-12-27 15:08:14,230][20347] Updated weights for policy 0, policy_version 1369583 (0.0007) [2023-12-27 15:08:14,900][20347] Updated weights for policy 0, policy_version 1369593 (0.0007) [2023-12-27 15:08:14,902][20381] Updated weights for policy 1, policy_version 1368409 (0.0007) [2023-12-27 15:08:14,957][20347] Updated weights for policy 0, policy_version 1369603 (0.0010) [2023-12-27 15:08:14,958][20381] Updated weights for policy 1, policy_version 1368419 (0.0006) [2023-12-27 15:08:15,015][20381] Updated weights for policy 1, policy_version 1368429 (0.0006) [2023-12-27 15:08:15,017][20347] Updated weights for policy 0, policy_version 1369613 (0.0011) [2023-12-27 15:08:15,076][20347] Updated weights for policy 0, policy_version 1369623 (0.0010) [2023-12-27 15:08:15,692][20381] Updated weights for policy 1, policy_version 1368439 (0.0006) [2023-12-27 15:08:15,712][20347] Updated weights for policy 0, policy_version 1369633 (0.0010) [2023-12-27 15:08:15,747][20381] Updated weights for policy 1, policy_version 1368449 (0.0005) [2023-12-27 15:08:15,764][20347] Updated weights for policy 0, policy_version 1369643 (0.0010) [2023-12-27 15:08:15,805][20381] Updated weights for policy 1, policy_version 1368459 (0.0006) [2023-12-27 15:08:15,815][20347] Updated weights for policy 0, policy_version 1369653 (0.0010) [2023-12-27 15:08:16,445][20347] Updated weights for policy 0, policy_version 1369663 (0.0010) [2023-12-27 15:08:16,499][20381] Updated weights for policy 1, policy_version 1368469 (0.0006) [2023-12-27 15:08:16,501][20347] Updated weights for policy 0, policy_version 1369673 (0.0011) [2023-12-27 15:08:16,553][20347] Updated weights for policy 0, policy_version 1369683 (0.0005) [2023-12-27 15:08:16,554][20381] Updated weights for policy 1, policy_version 1368479 (0.0008) [2023-12-27 15:08:16,615][20381] Updated weights for policy 1, policy_version 1368489 (0.0008) [2023-12-27 15:08:17,121][20347] Updated weights for policy 0, policy_version 1369693 (0.0008) [2023-12-27 15:08:17,182][20347] Updated weights for policy 0, policy_version 1369703 (0.0010) [2023-12-27 15:08:17,240][20347] Updated weights for policy 0, policy_version 1369713 (0.0010) [2023-12-27 15:08:17,353][20381] Updated weights for policy 1, policy_version 1368499 (0.0008) [2023-12-27 15:08:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 701079552. Throughput: 0: 10371.4, 1: 10318.2. Samples: 701089640. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:17,363][19170] Avg episode reward: [(0, '912.060'), (1, '1000.000')] [2023-12-27 15:08:17,411][20381] Updated weights for policy 1, policy_version 1368509 (0.0007) [2023-12-27 15:08:17,466][20381] Updated weights for policy 1, policy_version 1368519 (0.0007) [2023-12-27 15:08:17,882][20347] Updated weights for policy 0, policy_version 1369723 (0.0010) [2023-12-27 15:08:17,947][20347] Updated weights for policy 0, policy_version 1369733 (0.0011) [2023-12-27 15:08:18,009][20347] Updated weights for policy 0, policy_version 1369743 (0.0010) [2023-12-27 15:08:18,176][20381] Updated weights for policy 1, policy_version 1368529 (0.0007) [2023-12-27 15:08:18,232][20381] Updated weights for policy 1, policy_version 1368539 (0.0007) [2023-12-27 15:08:18,291][20381] Updated weights for policy 1, policy_version 1368549 (0.0008) [2023-12-27 15:08:18,349][20381] Updated weights for policy 1, policy_version 1368559 (0.0009) [2023-12-27 15:08:18,657][20347] Updated weights for policy 0, policy_version 1369753 (0.0010) [2023-12-27 15:08:18,723][20347] Updated weights for policy 0, policy_version 1369763 (0.0008) [2023-12-27 15:08:18,784][20347] Updated weights for policy 0, policy_version 1369773 (0.0008) [2023-12-27 15:08:18,839][20347] Updated weights for policy 0, policy_version 1369783 (0.0007) [2023-12-27 15:08:19,034][20381] Updated weights for policy 1, policy_version 1368569 (0.0008) [2023-12-27 15:08:19,083][20381] Updated weights for policy 1, policy_version 1368579 (0.0008) [2023-12-27 15:08:19,139][20381] Updated weights for policy 1, policy_version 1368589 (0.0008) [2023-12-27 15:08:19,553][20347] Updated weights for policy 0, policy_version 1369793 (0.0008) [2023-12-27 15:08:19,612][20347] Updated weights for policy 0, policy_version 1369803 (0.0006) [2023-12-27 15:08:19,667][20347] Updated weights for policy 0, policy_version 1369813 (0.0008) [2023-12-27 15:08:19,805][20381] Updated weights for policy 1, policy_version 1368599 (0.0006) [2023-12-27 15:08:19,877][20381] Updated weights for policy 1, policy_version 1368609 (0.0008) [2023-12-27 15:08:19,945][20381] Updated weights for policy 1, policy_version 1368619 (0.0008) [2023-12-27 15:08:20,317][20347] Updated weights for policy 0, policy_version 1369823 (0.0006) [2023-12-27 15:08:20,381][20347] Updated weights for policy 0, policy_version 1369833 (0.0006) [2023-12-27 15:08:20,438][20347] Updated weights for policy 0, policy_version 1369843 (0.0008) [2023-12-27 15:08:20,670][20381] Updated weights for policy 1, policy_version 1368629 (0.0007) [2023-12-27 15:08:20,730][20381] Updated weights for policy 1, policy_version 1368639 (0.0008) [2023-12-27 15:08:20,786][20381] Updated weights for policy 1, policy_version 1368649 (0.0008) [2023-12-27 15:08:21,099][20347] Updated weights for policy 0, policy_version 1369853 (0.0008) [2023-12-27 15:08:21,166][20347] Updated weights for policy 0, policy_version 1369863 (0.0009) [2023-12-27 15:08:21,232][20347] Updated weights for policy 0, policy_version 1369873 (0.0008) [2023-12-27 15:08:21,544][20381] Updated weights for policy 1, policy_version 1368659 (0.0008) [2023-12-27 15:08:21,602][20381] Updated weights for policy 1, policy_version 1368669 (0.0007) [2023-12-27 15:08:21,669][20381] Updated weights for policy 1, policy_version 1368679 (0.0007) [2023-12-27 15:08:21,987][20347] Updated weights for policy 0, policy_version 1369883 (0.0008) [2023-12-27 15:08:22,043][20347] Updated weights for policy 0, policy_version 1369893 (0.0008) [2023-12-27 15:08:22,103][20347] Updated weights for policy 0, policy_version 1369903 (0.0008) [2023-12-27 15:08:22,341][20381] Updated weights for policy 1, policy_version 1368689 (0.0007) [2023-12-27 15:08:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 701177856. Throughput: 0: 10332.0, 1: 10303.6. Samples: 701150120. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:22,363][19170] Avg episode reward: [(0, '960.860'), (1, '1000.000')] [2023-12-27 15:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001369912_350765056.pth... [2023-12-27 15:08:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001368696_350453760.pth [2023-12-27 15:08:22,407][20381] Updated weights for policy 1, policy_version 1368699 (0.0007) [2023-12-27 15:08:22,460][20381] Updated weights for policy 1, policy_version 1368709 (0.0006) [2023-12-27 15:08:22,519][20381] Updated weights for policy 1, policy_version 1368719 (0.0007) [2023-12-27 15:08:22,525][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001368720_350420992.pth... [2023-12-27 15:08:22,528][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001367504_350109696.pth [2023-12-27 15:08:22,900][20347] Updated weights for policy 0, policy_version 1369913 (0.0008) [2023-12-27 15:08:22,966][20347] Updated weights for policy 0, policy_version 1369923 (0.0008) [2023-12-27 15:08:23,034][20347] Updated weights for policy 0, policy_version 1369933 (0.0008) [2023-12-27 15:08:23,095][20347] Updated weights for policy 0, policy_version 1369943 (0.0007) [2023-12-27 15:08:23,164][20381] Updated weights for policy 1, policy_version 1368729 (0.0007) [2023-12-27 15:08:23,222][20381] Updated weights for policy 1, policy_version 1368739 (0.0008) [2023-12-27 15:08:23,277][20381] Updated weights for policy 1, policy_version 1368749 (0.0008) [2023-12-27 15:08:23,732][20347] Updated weights for policy 0, policy_version 1369953 (0.0005) [2023-12-27 15:08:23,786][20347] Updated weights for policy 0, policy_version 1369963 (0.0005) [2023-12-27 15:08:23,832][20347] Updated weights for policy 0, policy_version 1369973 (0.0005) [2023-12-27 15:08:23,970][20381] Updated weights for policy 1, policy_version 1368759 (0.0009) [2023-12-27 15:08:24,020][20381] Updated weights for policy 1, policy_version 1368769 (0.0008) [2023-12-27 15:08:24,072][20381] Updated weights for policy 1, policy_version 1368779 (0.0008) [2023-12-27 15:08:24,433][20347] Updated weights for policy 0, policy_version 1369983 (0.0006) [2023-12-27 15:08:24,478][20347] Updated weights for policy 0, policy_version 1369993 (0.0009) [2023-12-27 15:08:24,524][20347] Updated weights for policy 0, policy_version 1370003 (0.0007) [2023-12-27 15:08:24,705][20381] Updated weights for policy 1, policy_version 1368789 (0.0007) [2023-12-27 15:08:24,761][20381] Updated weights for policy 1, policy_version 1368799 (0.0007) [2023-12-27 15:08:24,822][20381] Updated weights for policy 1, policy_version 1368809 (0.0008) [2023-12-27 15:08:25,239][20347] Updated weights for policy 0, policy_version 1370013 (0.0008) [2023-12-27 15:08:25,288][20347] Updated weights for policy 0, policy_version 1370023 (0.0007) [2023-12-27 15:08:25,337][20347] Updated weights for policy 0, policy_version 1370033 (0.0007) [2023-12-27 15:08:25,511][20381] Updated weights for policy 1, policy_version 1368819 (0.0009) [2023-12-27 15:08:25,571][20381] Updated weights for policy 1, policy_version 1368829 (0.0008) [2023-12-27 15:08:25,626][20381] Updated weights for policy 1, policy_version 1368839 (0.0008) [2023-12-27 15:08:26,043][20347] Updated weights for policy 0, policy_version 1370043 (0.0007) [2023-12-27 15:08:26,100][20347] Updated weights for policy 0, policy_version 1370053 (0.0007) [2023-12-27 15:08:26,152][20347] Updated weights for policy 0, policy_version 1370063 (0.0006) [2023-12-27 15:08:26,247][20381] Updated weights for policy 1, policy_version 1368849 (0.0010) [2023-12-27 15:08:26,300][20381] Updated weights for policy 1, policy_version 1368859 (0.0006) [2023-12-27 15:08:26,358][20381] Updated weights for policy 1, policy_version 1368869 (0.0005) [2023-12-27 15:08:26,415][20381] Updated weights for policy 1, policy_version 1368879 (0.0007) [2023-12-27 15:08:26,848][20347] Updated weights for policy 0, policy_version 1370073 (0.0007) [2023-12-27 15:08:26,910][20347] Updated weights for policy 0, policy_version 1370083 (0.0005) [2023-12-27 15:08:26,972][20347] Updated weights for policy 0, policy_version 1370093 (0.0006) [2023-12-27 15:08:27,031][20347] Updated weights for policy 0, policy_version 1370103 (0.0008) [2023-12-27 15:08:27,097][20381] Updated weights for policy 1, policy_version 1368889 (0.0009) [2023-12-27 15:08:27,159][20381] Updated weights for policy 1, policy_version 1368899 (0.0011) [2023-12-27 15:08:27,205][20381] Updated weights for policy 1, policy_version 1368909 (0.0010) [2023-12-27 15:08:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 701284352. Throughput: 0: 10283.5, 1: 10294.3. Samples: 701271160. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:27,363][19170] Avg episode reward: [(0, '951.190'), (1, '1000.000')] [2023-12-27 15:08:27,666][20347] Updated weights for policy 0, policy_version 1370113 (0.0007) [2023-12-27 15:08:27,718][20347] Updated weights for policy 0, policy_version 1370123 (0.0007) [2023-12-27 15:08:27,782][20347] Updated weights for policy 0, policy_version 1370133 (0.0008) [2023-12-27 15:08:27,838][20381] Updated weights for policy 1, policy_version 1368919 (0.0007) [2023-12-27 15:08:27,884][20381] Updated weights for policy 1, policy_version 1368929 (0.0005) [2023-12-27 15:08:27,939][20381] Updated weights for policy 1, policy_version 1368939 (0.0005) [2023-12-27 15:08:28,540][20347] Updated weights for policy 0, policy_version 1370143 (0.0009) [2023-12-27 15:08:28,575][20381] Updated weights for policy 1, policy_version 1368949 (0.0007) [2023-12-27 15:08:28,599][20347] Updated weights for policy 0, policy_version 1370153 (0.0010) [2023-12-27 15:08:28,630][20381] Updated weights for policy 1, policy_version 1368959 (0.0005) [2023-12-27 15:08:28,660][20347] Updated weights for policy 0, policy_version 1370163 (0.0010) [2023-12-27 15:08:28,687][20381] Updated weights for policy 1, policy_version 1368969 (0.0010) [2023-12-27 15:08:29,257][20347] Updated weights for policy 0, policy_version 1370173 (0.0009) [2023-12-27 15:08:29,308][20347] Updated weights for policy 0, policy_version 1370183 (0.0007) [2023-12-27 15:08:29,369][20347] Updated weights for policy 0, policy_version 1370193 (0.0007) [2023-12-27 15:08:29,376][20381] Updated weights for policy 1, policy_version 1368979 (0.0008) [2023-12-27 15:08:29,441][20381] Updated weights for policy 1, policy_version 1368989 (0.0011) [2023-12-27 15:08:29,504][20381] Updated weights for policy 1, policy_version 1368999 (0.0007) [2023-12-27 15:08:30,023][20347] Updated weights for policy 0, policy_version 1370203 (0.0007) [2023-12-27 15:08:30,070][20347] Updated weights for policy 0, policy_version 1370213 (0.0007) [2023-12-27 15:08:30,119][20347] Updated weights for policy 0, policy_version 1370223 (0.0008) [2023-12-27 15:08:30,179][20381] Updated weights for policy 1, policy_version 1369009 (0.0010) [2023-12-27 15:08:30,245][20381] Updated weights for policy 1, policy_version 1369019 (0.0005) [2023-12-27 15:08:30,308][20381] Updated weights for policy 1, policy_version 1369029 (0.0007) [2023-12-27 15:08:30,355][20381] Updated weights for policy 1, policy_version 1369039 (0.0008) [2023-12-27 15:08:30,832][20347] Updated weights for policy 0, policy_version 1370233 (0.0009) [2023-12-27 15:08:30,876][20347] Updated weights for policy 0, policy_version 1370243 (0.0007) [2023-12-27 15:08:30,931][20347] Updated weights for policy 0, policy_version 1370253 (0.0008) [2023-12-27 15:08:30,992][20381] Updated weights for policy 1, policy_version 1369049 (0.0007) [2023-12-27 15:08:30,997][20347] Updated weights for policy 0, policy_version 1370263 (0.0006) [2023-12-27 15:08:31,076][20381] Updated weights for policy 1, policy_version 1369064 (0.0007) [2023-12-27 15:08:31,681][20347] Updated weights for policy 0, policy_version 1370273 (0.0008) [2023-12-27 15:08:31,734][20347] Updated weights for policy 0, policy_version 1370283 (0.0007) [2023-12-27 15:08:31,736][20381] Updated weights for policy 1, policy_version 1369074 (0.0006) [2023-12-27 15:08:31,799][20347] Updated weights for policy 0, policy_version 1370293 (0.0007) [2023-12-27 15:08:31,806][20381] Updated weights for policy 1, policy_version 1369084 (0.0008) [2023-12-27 15:08:31,865][20381] Updated weights for policy 1, policy_version 1369094 (0.0007) [2023-12-27 15:08:31,926][20381] Updated weights for policy 1, policy_version 1369104 (0.0008) [2023-12-27 15:08:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 701382656. Throughput: 0: 10277.4, 1: 10305.5. Samples: 701395264. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:32,363][19170] Avg episode reward: [(0, '931.560'), (1, '1000.000')] [2023-12-27 15:08:32,497][20347] Updated weights for policy 0, policy_version 1370303 (0.0007) [2023-12-27 15:08:32,553][20347] Updated weights for policy 0, policy_version 1370313 (0.0007) [2023-12-27 15:08:32,580][20381] Updated weights for policy 1, policy_version 1369114 (0.0007) [2023-12-27 15:08:32,617][20347] Updated weights for policy 0, policy_version 1370323 (0.0006) [2023-12-27 15:08:32,641][20381] Updated weights for policy 1, policy_version 1369124 (0.0006) [2023-12-27 15:08:32,700][20381] Updated weights for policy 1, policy_version 1369134 (0.0007) [2023-12-27 15:08:33,317][20347] Updated weights for policy 0, policy_version 1370333 (0.0006) [2023-12-27 15:08:33,376][20347] Updated weights for policy 0, policy_version 1370343 (0.0006) [2023-12-27 15:08:33,389][20381] Updated weights for policy 1, policy_version 1369144 (0.0010) [2023-12-27 15:08:33,423][20347] Updated weights for policy 0, policy_version 1370353 (0.0006) [2023-12-27 15:08:33,434][20381] Updated weights for policy 1, policy_version 1369154 (0.0010) [2023-12-27 15:08:33,478][20381] Updated weights for policy 1, policy_version 1369164 (0.0010) [2023-12-27 15:08:34,101][20347] Updated weights for policy 0, policy_version 1370363 (0.0006) [2023-12-27 15:08:34,158][20347] Updated weights for policy 0, policy_version 1370373 (0.0007) [2023-12-27 15:08:34,184][20381] Updated weights for policy 1, policy_version 1369174 (0.0010) [2023-12-27 15:08:34,221][20347] Updated weights for policy 0, policy_version 1370383 (0.0007) [2023-12-27 15:08:34,236][20381] Updated weights for policy 1, policy_version 1369184 (0.0006) [2023-12-27 15:08:34,295][20381] Updated weights for policy 1, policy_version 1369194 (0.0007) [2023-12-27 15:08:34,919][20381] Updated weights for policy 1, policy_version 1369204 (0.0008) [2023-12-27 15:08:34,954][20347] Updated weights for policy 0, policy_version 1370393 (0.0008) [2023-12-27 15:08:34,975][20381] Updated weights for policy 1, policy_version 1369214 (0.0006) [2023-12-27 15:08:35,007][20347] Updated weights for policy 0, policy_version 1370403 (0.0006) [2023-12-27 15:08:35,031][20381] Updated weights for policy 1, policy_version 1369224 (0.0005) [2023-12-27 15:08:35,063][20347] Updated weights for policy 0, policy_version 1370413 (0.0007) [2023-12-27 15:08:35,121][20347] Updated weights for policy 0, policy_version 1370423 (0.0008) [2023-12-27 15:08:35,712][20381] Updated weights for policy 1, policy_version 1369234 (0.0006) [2023-12-27 15:08:35,765][20381] Updated weights for policy 1, policy_version 1369244 (0.0006) [2023-12-27 15:08:35,771][20347] Updated weights for policy 0, policy_version 1370433 (0.0007) [2023-12-27 15:08:35,814][20381] Updated weights for policy 1, policy_version 1369254 (0.0006) [2023-12-27 15:08:35,852][20347] Updated weights for policy 0, policy_version 1370443 (0.0007) [2023-12-27 15:08:35,866][20381] Updated weights for policy 1, policy_version 1369264 (0.0008) [2023-12-27 15:08:35,906][20347] Updated weights for policy 0, policy_version 1370453 (0.0007) [2023-12-27 15:08:36,510][20347] Updated weights for policy 0, policy_version 1370463 (0.0006) [2023-12-27 15:08:36,561][20291] KL-divergence is very high: 102.0362 [2023-12-27 15:08:36,574][20347] Updated weights for policy 0, policy_version 1370473 (0.0007) [2023-12-27 15:08:36,610][20291] KL-divergence is very high: 103.8400 [2023-12-27 15:08:36,628][20381] Updated weights for policy 1, policy_version 1369274 (0.0007) [2023-12-27 15:08:36,634][20347] Updated weights for policy 0, policy_version 1370483 (0.0008) [2023-12-27 15:08:36,681][20381] Updated weights for policy 1, policy_version 1369284 (0.0007) [2023-12-27 15:08:36,729][20381] Updated weights for policy 1, policy_version 1369294 (0.0007) [2023-12-27 15:08:37,331][20347] Updated weights for policy 0, policy_version 1370493 (0.0007) [2023-12-27 15:08:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 701480960. Throughput: 0: 10239.4, 1: 10326.2. Samples: 701455876. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:37,363][19170] Avg episode reward: [(0, '931.690'), (1, '1000.000')] [2023-12-27 15:08:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001369296_350568448.pth... [2023-12-27 15:08:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001368112_350265344.pth [2023-12-27 15:08:37,378][20347] Updated weights for policy 0, policy_version 1370503 (0.0006) [2023-12-27 15:08:37,433][20347] Updated weights for policy 0, policy_version 1370513 (0.0010) [2023-12-27 15:08:37,450][20381] Updated weights for policy 1, policy_version 1369304 (0.0006) [2023-12-27 15:08:37,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001370520_350920704.pth... [2023-12-27 15:08:37,468][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001369304_350609408.pth [2023-12-27 15:08:37,505][20381] Updated weights for policy 1, policy_version 1369314 (0.0007) [2023-12-27 15:08:37,563][20381] Updated weights for policy 1, policy_version 1369324 (0.0007) [2023-12-27 15:08:38,081][20347] Updated weights for policy 0, policy_version 1370523 (0.0009) [2023-12-27 15:08:38,130][20347] Updated weights for policy 0, policy_version 1370533 (0.0005) [2023-12-27 15:08:38,178][20347] Updated weights for policy 0, policy_version 1370543 (0.0005) [2023-12-27 15:08:38,269][20381] Updated weights for policy 1, policy_version 1369334 (0.0007) [2023-12-27 15:08:38,315][20381] Updated weights for policy 1, policy_version 1369344 (0.0008) [2023-12-27 15:08:38,387][20381] Updated weights for policy 1, policy_version 1369354 (0.0007) [2023-12-27 15:08:38,823][20347] Updated weights for policy 0, policy_version 1370553 (0.0007) [2023-12-27 15:08:38,867][20291] KL-divergence is very high: 163.9418 [2023-12-27 15:08:38,885][20347] Updated weights for policy 0, policy_version 1370563 (0.0008) [2023-12-27 15:08:38,886][20291] KL-divergence is very high: 257.5233 [2023-12-27 15:08:38,912][20291] KL-divergence is very high: 215.5228 [2023-12-27 15:08:38,929][20291] KL-divergence is very high: 170.9057 [2023-12-27 15:08:38,940][20347] Updated weights for policy 0, policy_version 1370573 (0.0007) [2023-12-27 15:08:38,994][20347] Updated weights for policy 0, policy_version 1370583 (0.0008) [2023-12-27 15:08:39,144][20381] Updated weights for policy 1, policy_version 1369364 (0.0008) [2023-12-27 15:08:39,199][20381] Updated weights for policy 1, policy_version 1369374 (0.0007) [2023-12-27 15:08:39,264][20381] Updated weights for policy 1, policy_version 1369384 (0.0008) [2023-12-27 15:08:39,725][20347] Updated weights for policy 0, policy_version 1370593 (0.0006) [2023-12-27 15:08:39,788][20347] Updated weights for policy 0, policy_version 1370603 (0.0006) [2023-12-27 15:08:39,854][20347] Updated weights for policy 0, policy_version 1370613 (0.0008) [2023-12-27 15:08:40,030][20381] Updated weights for policy 1, policy_version 1369394 (0.0008) [2023-12-27 15:08:40,087][20381] Updated weights for policy 1, policy_version 1369404 (0.0008) [2023-12-27 15:08:40,148][20381] Updated weights for policy 1, policy_version 1369414 (0.0007) [2023-12-27 15:08:40,209][20381] Updated weights for policy 1, policy_version 1369424 (0.0006) [2023-12-27 15:08:40,547][20347] Updated weights for policy 0, policy_version 1370623 (0.0008) [2023-12-27 15:08:40,611][20347] Updated weights for policy 0, policy_version 1370633 (0.0007) [2023-12-27 15:08:40,677][20347] Updated weights for policy 0, policy_version 1370643 (0.0005) [2023-12-27 15:08:40,859][20381] Updated weights for policy 1, policy_version 1369434 (0.0005) [2023-12-27 15:08:40,921][20381] Updated weights for policy 1, policy_version 1369444 (0.0005) [2023-12-27 15:08:40,981][20381] Updated weights for policy 1, policy_version 1369454 (0.0006) [2023-12-27 15:08:41,338][20347] Updated weights for policy 0, policy_version 1370653 (0.0008) [2023-12-27 15:08:41,412][20347] Updated weights for policy 0, policy_version 1370663 (0.0010) [2023-12-27 15:08:41,468][20347] Updated weights for policy 0, policy_version 1370673 (0.0010) [2023-12-27 15:08:41,626][20381] Updated weights for policy 1, policy_version 1369464 (0.0008) [2023-12-27 15:08:41,697][20381] Updated weights for policy 1, policy_version 1369474 (0.0008) [2023-12-27 15:08:41,761][20381] Updated weights for policy 1, policy_version 1369484 (0.0008) [2023-12-27 15:08:42,147][20347] Updated weights for policy 0, policy_version 1370683 (0.0010) [2023-12-27 15:08:42,196][20347] Updated weights for policy 0, policy_version 1370693 (0.0007) [2023-12-27 15:08:42,248][20347] Updated weights for policy 0, policy_version 1370703 (0.0007) [2023-12-27 15:08:42,364][19170] Fps is (10 sec: 20477.0, 60 sec: 20616.0, 300 sec: 20493.8). Total num frames: 701587456. Throughput: 0: 10195.1, 1: 10234.0. Samples: 701576784. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:42,365][19170] Avg episode reward: [(0, '902.720'), (1, '1000.000')] [2023-12-27 15:08:42,478][20381] Updated weights for policy 1, policy_version 1369494 (0.0008) [2023-12-27 15:08:42,532][20381] Updated weights for policy 1, policy_version 1369504 (0.0007) [2023-12-27 15:08:42,593][20381] Updated weights for policy 1, policy_version 1369514 (0.0007) [2023-12-27 15:08:42,873][20347] Updated weights for policy 0, policy_version 1370713 (0.0008) [2023-12-27 15:08:42,932][20347] Updated weights for policy 0, policy_version 1370723 (0.0008) [2023-12-27 15:08:42,991][20347] Updated weights for policy 0, policy_version 1370733 (0.0010) [2023-12-27 15:08:43,050][20347] Updated weights for policy 0, policy_version 1370743 (0.0010) [2023-12-27 15:08:43,354][20381] Updated weights for policy 1, policy_version 1369524 (0.0008) [2023-12-27 15:08:43,406][20381] Updated weights for policy 1, policy_version 1369534 (0.0008) [2023-12-27 15:08:43,461][20381] Updated weights for policy 1, policy_version 1369544 (0.0007) [2023-12-27 15:08:43,654][20347] Updated weights for policy 0, policy_version 1370753 (0.0006) [2023-12-27 15:08:43,707][20347] Updated weights for policy 0, policy_version 1370763 (0.0007) [2023-12-27 15:08:43,762][20347] Updated weights for policy 0, policy_version 1370773 (0.0010) [2023-12-27 15:08:44,112][20381] Updated weights for policy 1, policy_version 1369554 (0.0007) [2023-12-27 15:08:44,162][20381] Updated weights for policy 1, policy_version 1369564 (0.0006) [2023-12-27 15:08:44,221][20381] Updated weights for policy 1, policy_version 1369574 (0.0005) [2023-12-27 15:08:44,279][20381] Updated weights for policy 1, policy_version 1369584 (0.0005) [2023-12-27 15:08:44,369][20347] Updated weights for policy 0, policy_version 1370783 (0.0010) [2023-12-27 15:08:44,427][20347] Updated weights for policy 0, policy_version 1370793 (0.0010) [2023-12-27 15:08:44,479][20347] Updated weights for policy 0, policy_version 1370803 (0.0009) [2023-12-27 15:08:44,822][20381] Updated weights for policy 1, policy_version 1369594 (0.0008) [2023-12-27 15:08:44,889][20381] Updated weights for policy 1, policy_version 1369604 (0.0008) [2023-12-27 15:08:44,947][20381] Updated weights for policy 1, policy_version 1369614 (0.0008) [2023-12-27 15:08:45,136][20347] Updated weights for policy 0, policy_version 1370813 (0.0008) [2023-12-27 15:08:45,195][20347] Updated weights for policy 0, policy_version 1370823 (0.0006) [2023-12-27 15:08:45,253][20347] Updated weights for policy 0, policy_version 1370833 (0.0007) [2023-12-27 15:08:45,591][20381] Updated weights for policy 1, policy_version 1369624 (0.0008) [2023-12-27 15:08:45,637][20381] Updated weights for policy 1, policy_version 1369634 (0.0008) [2023-12-27 15:08:45,683][20381] Updated weights for policy 1, policy_version 1369644 (0.0007) [2023-12-27 15:08:45,928][20347] Updated weights for policy 0, policy_version 1370843 (0.0007) [2023-12-27 15:08:45,974][20347] Updated weights for policy 0, policy_version 1370853 (0.0008) [2023-12-27 15:08:46,029][20347] Updated weights for policy 0, policy_version 1370863 (0.0008) [2023-12-27 15:08:46,328][20381] Updated weights for policy 1, policy_version 1369654 (0.0008) [2023-12-27 15:08:46,384][20381] Updated weights for policy 1, policy_version 1369664 (0.0008) [2023-12-27 15:08:46,436][20381] Updated weights for policy 1, policy_version 1369674 (0.0007) [2023-12-27 15:08:46,685][20347] Updated weights for policy 0, policy_version 1370873 (0.0007) [2023-12-27 15:08:46,735][20347] Updated weights for policy 0, policy_version 1370883 (0.0007) [2023-12-27 15:08:46,786][20347] Updated weights for policy 0, policy_version 1370893 (0.0006) [2023-12-27 15:08:46,834][20347] Updated weights for policy 0, policy_version 1370903 (0.0008) [2023-12-27 15:08:47,050][20381] Updated weights for policy 1, policy_version 1369684 (0.0008) [2023-12-27 15:08:47,101][20381] Updated weights for policy 1, policy_version 1369694 (0.0007) [2023-12-27 15:08:47,153][20381] Updated weights for policy 1, policy_version 1369704 (0.0007) [2023-12-27 15:08:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 701693952. Throughput: 0: 10251.1, 1: 10237.5. Samples: 701702804. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:47,363][19170] Avg episode reward: [(0, '922.060'), (1, '1000.000')] [2023-12-27 15:08:47,521][20347] Updated weights for policy 0, policy_version 1370913 (0.0008) [2023-12-27 15:08:47,583][20347] Updated weights for policy 0, policy_version 1370923 (0.0007) [2023-12-27 15:08:47,632][20347] Updated weights for policy 0, policy_version 1370933 (0.0008) [2023-12-27 15:08:47,787][20381] Updated weights for policy 1, policy_version 1369714 (0.0008) [2023-12-27 15:08:47,839][20381] Updated weights for policy 1, policy_version 1369724 (0.0009) [2023-12-27 15:08:47,895][20381] Updated weights for policy 1, policy_version 1369734 (0.0008) [2023-12-27 15:08:47,958][20381] Updated weights for policy 1, policy_version 1369744 (0.0006) [2023-12-27 15:08:48,329][20347] Updated weights for policy 0, policy_version 1370943 (0.0007) [2023-12-27 15:08:48,394][20347] Updated weights for policy 0, policy_version 1370953 (0.0007) [2023-12-27 15:08:48,450][20347] Updated weights for policy 0, policy_version 1370963 (0.0006) [2023-12-27 15:08:48,626][20381] Updated weights for policy 1, policy_version 1369754 (0.0008) [2023-12-27 15:08:48,685][20381] Updated weights for policy 1, policy_version 1369764 (0.0008) [2023-12-27 15:08:48,749][20381] Updated weights for policy 1, policy_version 1369774 (0.0008) [2023-12-27 15:08:49,080][20347] Updated weights for policy 0, policy_version 1370973 (0.0006) [2023-12-27 15:08:49,126][20347] Updated weights for policy 0, policy_version 1370983 (0.0005) [2023-12-27 15:08:49,175][20347] Updated weights for policy 0, policy_version 1370993 (0.0005) [2023-12-27 15:08:49,502][20381] Updated weights for policy 1, policy_version 1369784 (0.0007) [2023-12-27 15:08:49,560][20381] Updated weights for policy 1, policy_version 1369794 (0.0008) [2023-12-27 15:08:49,613][20381] Updated weights for policy 1, policy_version 1369804 (0.0007) [2023-12-27 15:08:49,793][20347] Updated weights for policy 0, policy_version 1371003 (0.0006) [2023-12-27 15:08:49,847][20347] Updated weights for policy 0, policy_version 1371013 (0.0008) [2023-12-27 15:08:49,910][20347] Updated weights for policy 0, policy_version 1371023 (0.0007) [2023-12-27 15:08:50,319][20381] Updated weights for policy 1, policy_version 1369814 (0.0006) [2023-12-27 15:08:50,378][20381] Updated weights for policy 1, policy_version 1369824 (0.0005) [2023-12-27 15:08:50,438][20381] Updated weights for policy 1, policy_version 1369834 (0.0006) [2023-12-27 15:08:50,662][20347] Updated weights for policy 0, policy_version 1371033 (0.0008) [2023-12-27 15:08:50,722][20347] Updated weights for policy 0, policy_version 1371043 (0.0008) [2023-12-27 15:08:50,773][20347] Updated weights for policy 0, policy_version 1371053 (0.0009) [2023-12-27 15:08:50,834][20347] Updated weights for policy 0, policy_version 1371063 (0.0010) [2023-12-27 15:08:51,086][20381] Updated weights for policy 1, policy_version 1369844 (0.0008) [2023-12-27 15:08:51,150][20381] Updated weights for policy 1, policy_version 1369854 (0.0010) [2023-12-27 15:08:51,205][20381] Updated weights for policy 1, policy_version 1369864 (0.0007) [2023-12-27 15:08:51,465][20347] Updated weights for policy 0, policy_version 1371073 (0.0006) [2023-12-27 15:08:51,512][20347] Updated weights for policy 0, policy_version 1371083 (0.0005) [2023-12-27 15:08:51,558][20347] Updated weights for policy 0, policy_version 1371093 (0.0007) [2023-12-27 15:08:51,928][20381] Updated weights for policy 1, policy_version 1369874 (0.0006) [2023-12-27 15:08:51,987][20381] Updated weights for policy 1, policy_version 1369884 (0.0007) [2023-12-27 15:08:52,041][20381] Updated weights for policy 1, policy_version 1369894 (0.0008) [2023-12-27 15:08:52,100][20381] Updated weights for policy 1, policy_version 1369904 (0.0007) [2023-12-27 15:08:52,341][20347] Updated weights for policy 0, policy_version 1371103 (0.0011) [2023-12-27 15:08:52,362][19170] Fps is (10 sec: 20482.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 701792256. Throughput: 0: 10252.3, 1: 10171.7. Samples: 701764156. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:52,363][19170] Avg episode reward: [(0, '970.630'), (1, '1000.000')] [2023-12-27 15:08:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001369904_350724096.pth... [2023-12-27 15:08:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001368720_350420992.pth [2023-12-27 15:08:52,406][20347] Updated weights for policy 0, policy_version 1371113 (0.0011) [2023-12-27 15:08:52,466][20347] Updated weights for policy 0, policy_version 1371123 (0.0009) [2023-12-27 15:08:52,493][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001371128_351076352.pth... [2023-12-27 15:08:52,497][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001369912_350765056.pth [2023-12-27 15:08:52,816][20381] Updated weights for policy 1, policy_version 1369914 (0.0006) [2023-12-27 15:08:52,871][20381] Updated weights for policy 1, policy_version 1369924 (0.0007) [2023-12-27 15:08:52,930][20381] Updated weights for policy 1, policy_version 1369934 (0.0008) [2023-12-27 15:08:53,176][20347] Updated weights for policy 0, policy_version 1371133 (0.0009) [2023-12-27 15:08:53,228][20347] Updated weights for policy 0, policy_version 1371143 (0.0009) [2023-12-27 15:08:53,280][20347] Updated weights for policy 0, policy_version 1371153 (0.0008) [2023-12-27 15:08:53,604][20381] Updated weights for policy 1, policy_version 1369944 (0.0007) [2023-12-27 15:08:53,654][20381] Updated weights for policy 1, policy_version 1369954 (0.0007) [2023-12-27 15:08:53,698][20381] Updated weights for policy 1, policy_version 1369964 (0.0005) [2023-12-27 15:08:53,978][20347] Updated weights for policy 0, policy_version 1371163 (0.0007) [2023-12-27 15:08:54,035][20347] Updated weights for policy 0, policy_version 1371173 (0.0006) [2023-12-27 15:08:54,096][20347] Updated weights for policy 0, policy_version 1371183 (0.0009) [2023-12-27 15:08:54,302][20381] Updated weights for policy 1, policy_version 1369974 (0.0007) [2023-12-27 15:08:54,350][20381] Updated weights for policy 1, policy_version 1369984 (0.0008) [2023-12-27 15:08:54,399][20381] Updated weights for policy 1, policy_version 1369994 (0.0008) [2023-12-27 15:08:54,747][20347] Updated weights for policy 0, policy_version 1371193 (0.0010) [2023-12-27 15:08:54,806][20347] Updated weights for policy 0, policy_version 1371203 (0.0010) [2023-12-27 15:08:54,861][20347] Updated weights for policy 0, policy_version 1371213 (0.0010) [2023-12-27 15:08:54,909][20347] Updated weights for policy 0, policy_version 1371223 (0.0010) [2023-12-27 15:08:55,012][20381] Updated weights for policy 1, policy_version 1370004 (0.0007) [2023-12-27 15:08:55,062][20381] Updated weights for policy 1, policy_version 1370014 (0.0005) [2023-12-27 15:08:55,117][20381] Updated weights for policy 1, policy_version 1370024 (0.0005) [2023-12-27 15:08:55,536][20347] Updated weights for policy 0, policy_version 1371233 (0.0006) [2023-12-27 15:08:55,583][20347] Updated weights for policy 0, policy_version 1371243 (0.0006) [2023-12-27 15:08:55,631][20347] Updated weights for policy 0, policy_version 1371253 (0.0010) [2023-12-27 15:08:55,639][20381] Updated weights for policy 1, policy_version 1370034 (0.0006) [2023-12-27 15:08:55,691][20381] Updated weights for policy 1, policy_version 1370044 (0.0010) [2023-12-27 15:08:55,746][20381] Updated weights for policy 1, policy_version 1370054 (0.0007) [2023-12-27 15:08:55,792][20381] Updated weights for policy 1, policy_version 1370064 (0.0010) [2023-12-27 15:08:56,291][20347] Updated weights for policy 0, policy_version 1371263 (0.0010) [2023-12-27 15:08:56,342][20347] Updated weights for policy 0, policy_version 1371273 (0.0010) [2023-12-27 15:08:56,393][20347] Updated weights for policy 0, policy_version 1371283 (0.0010) [2023-12-27 15:08:56,479][20381] Updated weights for policy 1, policy_version 1370074 (0.0005) [2023-12-27 15:08:56,524][20381] Updated weights for policy 1, policy_version 1370084 (0.0005) [2023-12-27 15:08:56,569][20381] Updated weights for policy 1, policy_version 1370094 (0.0005) [2023-12-27 15:08:57,092][20347] Updated weights for policy 0, policy_version 1371293 (0.0010) [2023-12-27 15:08:57,151][20347] Updated weights for policy 0, policy_version 1371303 (0.0010) [2023-12-27 15:08:57,162][20381] Updated weights for policy 1, policy_version 1370104 (0.0010) [2023-12-27 15:08:57,207][20347] Updated weights for policy 0, policy_version 1371313 (0.0011) [2023-12-27 15:08:57,218][20381] Updated weights for policy 1, policy_version 1370114 (0.0011) [2023-12-27 15:08:57,279][20381] Updated weights for policy 1, policy_version 1370124 (0.0011) [2023-12-27 15:08:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 701906944. Throughput: 0: 10268.5, 1: 10197.7. Samples: 701888944. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:08:57,363][19170] Avg episode reward: [(0, '970.670'), (1, '1000.000')] [2023-12-27 15:08:57,883][20347] Updated weights for policy 0, policy_version 1371323 (0.0010) [2023-12-27 15:08:57,932][20347] Updated weights for policy 0, policy_version 1371333 (0.0008) [2023-12-27 15:08:57,958][20381] Updated weights for policy 1, policy_version 1370134 (0.0011) [2023-12-27 15:08:57,988][20347] Updated weights for policy 0, policy_version 1371343 (0.0010) [2023-12-27 15:08:58,009][20381] Updated weights for policy 1, policy_version 1370144 (0.0010) [2023-12-27 15:08:58,057][20381] Updated weights for policy 1, policy_version 1370154 (0.0010) [2023-12-27 15:08:58,679][20347] Updated weights for policy 0, policy_version 1371353 (0.0010) [2023-12-27 15:08:58,744][20347] Updated weights for policy 0, policy_version 1371363 (0.0008) [2023-12-27 15:08:58,810][20347] Updated weights for policy 0, policy_version 1371373 (0.0008) [2023-12-27 15:08:58,877][20381] Updated weights for policy 1, policy_version 1370164 (0.0009) [2023-12-27 15:08:58,883][20347] Updated weights for policy 0, policy_version 1371383 (0.0008) [2023-12-27 15:08:58,950][20381] Updated weights for policy 1, policy_version 1370174 (0.0007) [2023-12-27 15:08:59,016][20381] Updated weights for policy 1, policy_version 1370184 (0.0008) [2023-12-27 15:08:59,628][20347] Updated weights for policy 0, policy_version 1371393 (0.0006) [2023-12-27 15:08:59,689][20347] Updated weights for policy 0, policy_version 1371403 (0.0007) [2023-12-27 15:08:59,744][20347] Updated weights for policy 0, policy_version 1371413 (0.0008) [2023-12-27 15:08:59,770][20381] Updated weights for policy 1, policy_version 1370194 (0.0007) [2023-12-27 15:08:59,829][20381] Updated weights for policy 1, policy_version 1370204 (0.0008) [2023-12-27 15:08:59,890][20381] Updated weights for policy 1, policy_version 1370214 (0.0007) [2023-12-27 15:08:59,956][20381] Updated weights for policy 1, policy_version 1370224 (0.0010) [2023-12-27 15:09:00,420][20347] Updated weights for policy 0, policy_version 1371423 (0.0007) [2023-12-27 15:09:00,448][20291] KL-divergence is very high: 408.9114 [2023-12-27 15:09:00,466][20291] KL-divergence is very high: 146.8171 [2023-12-27 15:09:00,472][20347] Updated weights for policy 0, policy_version 1371433 (0.0007) [2023-12-27 15:09:00,495][20291] KL-divergence is very high: 119.0416 [2023-12-27 15:09:00,532][20347] Updated weights for policy 0, policy_version 1371443 (0.0010) [2023-12-27 15:09:00,636][20381] Updated weights for policy 1, policy_version 1370234 (0.0010) [2023-12-27 15:09:00,700][20381] Updated weights for policy 1, policy_version 1370244 (0.0010) [2023-12-27 15:09:00,757][20381] Updated weights for policy 1, policy_version 1370254 (0.0008) [2023-12-27 15:09:01,260][20347] Updated weights for policy 0, policy_version 1371453 (0.0009) [2023-12-27 15:09:01,317][20347] Updated weights for policy 0, policy_version 1371463 (0.0007) [2023-12-27 15:09:01,381][20381] Updated weights for policy 1, policy_version 1370264 (0.0008) [2023-12-27 15:09:01,382][20347] Updated weights for policy 0, policy_version 1371473 (0.0008) [2023-12-27 15:09:01,442][20381] Updated weights for policy 1, policy_version 1370274 (0.0008) [2023-12-27 15:09:01,498][20381] Updated weights for policy 1, policy_version 1370284 (0.0008) [2023-12-27 15:09:02,076][20347] Updated weights for policy 0, policy_version 1371483 (0.0006) [2023-12-27 15:09:02,141][20347] Updated weights for policy 0, policy_version 1371493 (0.0005) [2023-12-27 15:09:02,203][20347] Updated weights for policy 0, policy_version 1371503 (0.0005) [2023-12-27 15:09:02,247][20381] Updated weights for policy 1, policy_version 1370294 (0.0008) [2023-12-27 15:09:02,306][20381] Updated weights for policy 1, policy_version 1370304 (0.0005) [2023-12-27 15:09:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 701997056. Throughput: 0: 10198.5, 1: 10218.2. Samples: 702008392. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:02,363][19170] Avg episode reward: [(0, '912.360'), (1, '1000.000')] [2023-12-27 15:09:02,377][20381] Updated weights for policy 1, policy_version 1370314 (0.0008) [2023-12-27 15:09:02,824][20347] Updated weights for policy 0, policy_version 1371513 (0.0006) [2023-12-27 15:09:02,879][20347] Updated weights for policy 0, policy_version 1371523 (0.0010) [2023-12-27 15:09:02,927][20347] Updated weights for policy 0, policy_version 1371533 (0.0010) [2023-12-27 15:09:02,980][20347] Updated weights for policy 0, policy_version 1371543 (0.0010) [2023-12-27 15:09:03,035][20381] Updated weights for policy 1, policy_version 1370324 (0.0008) [2023-12-27 15:09:03,093][20381] Updated weights for policy 1, policy_version 1370334 (0.0008) [2023-12-27 15:09:03,150][20381] Updated weights for policy 1, policy_version 1370344 (0.0007) [2023-12-27 15:09:03,656][20347] Updated weights for policy 0, policy_version 1371553 (0.0006) [2023-12-27 15:09:03,703][20347] Updated weights for policy 0, policy_version 1371563 (0.0010) [2023-12-27 15:09:03,759][20347] Updated weights for policy 0, policy_version 1371573 (0.0010) [2023-12-27 15:09:03,789][20381] Updated weights for policy 1, policy_version 1370354 (0.0008) [2023-12-27 15:09:03,847][20381] Updated weights for policy 1, policy_version 1370364 (0.0010) [2023-12-27 15:09:03,908][20381] Updated weights for policy 1, policy_version 1370374 (0.0009) [2023-12-27 15:09:03,968][20381] Updated weights for policy 1, policy_version 1370384 (0.0011) [2023-12-27 15:09:04,425][20347] Updated weights for policy 0, policy_version 1371583 (0.0008) [2023-12-27 15:09:04,483][20347] Updated weights for policy 0, policy_version 1371593 (0.0008) [2023-12-27 15:09:04,538][20347] Updated weights for policy 0, policy_version 1371603 (0.0008) [2023-12-27 15:09:04,679][20381] Updated weights for policy 1, policy_version 1370394 (0.0010) [2023-12-27 15:09:04,727][20381] Updated weights for policy 1, policy_version 1370404 (0.0010) [2023-12-27 15:09:04,784][20381] Updated weights for policy 1, policy_version 1370414 (0.0010) [2023-12-27 15:09:05,239][20347] Updated weights for policy 0, policy_version 1371613 (0.0009) [2023-12-27 15:09:05,294][20347] Updated weights for policy 0, policy_version 1371623 (0.0011) [2023-12-27 15:09:05,316][20291] KL-divergence is very high: 134.4337 [2023-12-27 15:09:05,348][20347] Updated weights for policy 0, policy_version 1371633 (0.0010) [2023-12-27 15:09:05,430][20381] Updated weights for policy 1, policy_version 1370424 (0.0009) [2023-12-27 15:09:05,482][20381] Updated weights for policy 1, policy_version 1370434 (0.0009) [2023-12-27 15:09:05,536][20381] Updated weights for policy 1, policy_version 1370444 (0.0010) [2023-12-27 15:09:06,018][20347] Updated weights for policy 0, policy_version 1371643 (0.0010) [2023-12-27 15:09:06,066][20347] Updated weights for policy 0, policy_version 1371653 (0.0010) [2023-12-27 15:09:06,130][20347] Updated weights for policy 0, policy_version 1371663 (0.0010) [2023-12-27 15:09:06,207][20381] Updated weights for policy 1, policy_version 1370454 (0.0009) [2023-12-27 15:09:06,275][20381] Updated weights for policy 1, policy_version 1370464 (0.0006) [2023-12-27 15:09:06,338][20381] Updated weights for policy 1, policy_version 1370474 (0.0008) [2023-12-27 15:09:06,864][20347] Updated weights for policy 0, policy_version 1371673 (0.0009) [2023-12-27 15:09:06,923][20347] Updated weights for policy 0, policy_version 1371683 (0.0010) [2023-12-27 15:09:06,977][20381] Updated weights for policy 1, policy_version 1370484 (0.0008) [2023-12-27 15:09:06,981][20347] Updated weights for policy 0, policy_version 1371693 (0.0007) [2023-12-27 15:09:07,040][20347] Updated weights for policy 0, policy_version 1371703 (0.0009) [2023-12-27 15:09:07,041][20381] Updated weights for policy 1, policy_version 1370494 (0.0007) [2023-12-27 15:09:07,100][20381] Updated weights for policy 1, policy_version 1370504 (0.0008) [2023-12-27 15:09:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 702103552. Throughput: 0: 10218.5, 1: 10243.4. Samples: 702070904. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:07,363][19170] Avg episode reward: [(0, '883.010'), (1, '1000.000')] [2023-12-27 15:09:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001371704_351223808.pth... [2023-12-27 15:09:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001370512_350879744.pth... [2023-12-27 15:09:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001369296_350568448.pth [2023-12-27 15:09:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001370520_350920704.pth [2023-12-27 15:09:07,655][20347] Updated weights for policy 0, policy_version 1371713 (0.0006) [2023-12-27 15:09:07,682][20381] Updated weights for policy 1, policy_version 1370514 (0.0008) [2023-12-27 15:09:07,711][20347] Updated weights for policy 0, policy_version 1371723 (0.0006) [2023-12-27 15:09:07,740][20381] Updated weights for policy 1, policy_version 1370524 (0.0007) [2023-12-27 15:09:07,762][20347] Updated weights for policy 0, policy_version 1371733 (0.0008) [2023-12-27 15:09:07,795][20381] Updated weights for policy 1, policy_version 1370534 (0.0009) [2023-12-27 15:09:07,854][20381] Updated weights for policy 1, policy_version 1370544 (0.0011) [2023-12-27 15:09:08,316][20347] Updated weights for policy 0, policy_version 1371743 (0.0007) [2023-12-27 15:09:08,383][20347] Updated weights for policy 0, policy_version 1371753 (0.0007) [2023-12-27 15:09:08,447][20347] Updated weights for policy 0, policy_version 1371763 (0.0008) [2023-12-27 15:09:08,523][20381] Updated weights for policy 1, policy_version 1370554 (0.0005) [2023-12-27 15:09:08,576][20381] Updated weights for policy 1, policy_version 1370564 (0.0010) [2023-12-27 15:09:08,632][20381] Updated weights for policy 1, policy_version 1370574 (0.0010) [2023-12-27 15:09:09,078][20347] Updated weights for policy 0, policy_version 1371773 (0.0008) [2023-12-27 15:09:09,136][20347] Updated weights for policy 0, policy_version 1371783 (0.0005) [2023-12-27 15:09:09,195][20347] Updated weights for policy 0, policy_version 1371793 (0.0005) [2023-12-27 15:09:09,310][20381] Updated weights for policy 1, policy_version 1370584 (0.0008) [2023-12-27 15:09:09,381][20381] Updated weights for policy 1, policy_version 1370594 (0.0008) [2023-12-27 15:09:09,448][20381] Updated weights for policy 1, policy_version 1370604 (0.0008) [2023-12-27 15:09:09,897][20347] Updated weights for policy 0, policy_version 1371803 (0.0007) [2023-12-27 15:09:09,968][20291] KL-divergence is very high: 534.6594 [2023-12-27 15:09:09,969][20347] Updated weights for policy 0, policy_version 1371813 (0.0007) [2023-12-27 15:09:09,975][20291] KL-divergence is very high: 616.9714 [2023-12-27 15:09:10,023][20291] KL-divergence is very high: 315.8166 [2023-12-27 15:09:10,031][20291] KL-divergence is very high: 303.1632 [2023-12-27 15:09:10,036][20347] Updated weights for policy 0, policy_version 1371823 (0.0008) [2023-12-27 15:09:10,151][20381] Updated weights for policy 1, policy_version 1370614 (0.0008) [2023-12-27 15:09:10,209][20381] Updated weights for policy 1, policy_version 1370624 (0.0009) [2023-12-27 15:09:10,268][20381] Updated weights for policy 1, policy_version 1370634 (0.0007) [2023-12-27 15:09:10,655][20347] Updated weights for policy 0, policy_version 1371833 (0.0006) [2023-12-27 15:09:10,660][20291] KL-divergence is very high: 408.3729 [2023-12-27 15:09:10,702][20291] KL-divergence is very high: 250.5349 [2023-12-27 15:09:10,707][20347] Updated weights for policy 0, policy_version 1371843 (0.0006) [2023-12-27 15:09:10,726][20291] KL-divergence is very high: 135.8373 [2023-12-27 15:09:10,742][20291] KL-divergence is very high: 106.1767 [2023-12-27 15:09:10,759][20347] Updated weights for policy 0, policy_version 1371853 (0.0006) [2023-12-27 15:09:10,768][20291] KL-divergence is very high: 202.9030 [2023-12-27 15:09:10,787][20291] KL-divergence is very high: 103.7625 [2023-12-27 15:09:10,815][20347] Updated weights for policy 0, policy_version 1371863 (0.0006) [2023-12-27 15:09:10,816][20291] KL-divergence is very high: 127.5572 [2023-12-27 15:09:10,998][20381] Updated weights for policy 1, policy_version 1370644 (0.0007) [2023-12-27 15:09:11,112][20381] Updated weights for policy 1, policy_version 1370654 (0.0008) [2023-12-27 15:09:11,173][20381] Updated weights for policy 1, policy_version 1370664 (0.0007) [2023-12-27 15:09:11,604][20347] Updated weights for policy 0, policy_version 1371873 (0.0008) [2023-12-27 15:09:11,671][20347] Updated weights for policy 0, policy_version 1371883 (0.0009) [2023-12-27 15:09:11,732][20347] Updated weights for policy 0, policy_version 1371893 (0.0009) [2023-12-27 15:09:11,854][20381] Updated weights for policy 1, policy_version 1370674 (0.0009) [2023-12-27 15:09:11,917][20381] Updated weights for policy 1, policy_version 1370684 (0.0007) [2023-12-27 15:09:11,970][20381] Updated weights for policy 1, policy_version 1370694 (0.0008) [2023-12-27 15:09:12,020][20381] Updated weights for policy 1, policy_version 1370704 (0.0007) [2023-12-27 15:09:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 702201856. Throughput: 0: 10287.1, 1: 10246.3. Samples: 702195164. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:12,363][19170] Avg episode reward: [(0, '873.920'), (1, '1000.000')] [2023-12-27 15:09:12,466][20347] Updated weights for policy 0, policy_version 1371903 (0.0008) [2023-12-27 15:09:12,514][20347] Updated weights for policy 0, policy_version 1371913 (0.0008) [2023-12-27 15:09:12,569][20347] Updated weights for policy 0, policy_version 1371923 (0.0008) [2023-12-27 15:09:12,657][20381] Updated weights for policy 1, policy_version 1370714 (0.0006) [2023-12-27 15:09:12,716][20381] Updated weights for policy 1, policy_version 1370724 (0.0007) [2023-12-27 15:09:12,775][20381] Updated weights for policy 1, policy_version 1370734 (0.0007) [2023-12-27 15:09:13,288][20347] Updated weights for policy 0, policy_version 1371933 (0.0008) [2023-12-27 15:09:13,337][20347] Updated weights for policy 0, policy_version 1371943 (0.0009) [2023-12-27 15:09:13,386][20347] Updated weights for policy 0, policy_version 1371953 (0.0007) [2023-12-27 15:09:13,461][20381] Updated weights for policy 1, policy_version 1370744 (0.0006) [2023-12-27 15:09:13,523][20381] Updated weights for policy 1, policy_version 1370754 (0.0005) [2023-12-27 15:09:13,587][20381] Updated weights for policy 1, policy_version 1370764 (0.0008) [2023-12-27 15:09:14,062][20347] Updated weights for policy 0, policy_version 1371963 (0.0008) [2023-12-27 15:09:14,126][20347] Updated weights for policy 0, policy_version 1371973 (0.0008) [2023-12-27 15:09:14,165][20381] Updated weights for policy 1, policy_version 1370774 (0.0006) [2023-12-27 15:09:14,187][20347] Updated weights for policy 0, policy_version 1371983 (0.0007) [2023-12-27 15:09:14,224][20381] Updated weights for policy 1, policy_version 1370784 (0.0006) [2023-12-27 15:09:14,277][20381] Updated weights for policy 1, policy_version 1370794 (0.0007) [2023-12-27 15:09:14,910][20347] Updated weights for policy 0, policy_version 1371993 (0.0006) [2023-12-27 15:09:14,949][20381] Updated weights for policy 1, policy_version 1370804 (0.0007) [2023-12-27 15:09:14,974][20347] Updated weights for policy 0, policy_version 1372003 (0.0008) [2023-12-27 15:09:15,006][20381] Updated weights for policy 1, policy_version 1370814 (0.0007) [2023-12-27 15:09:15,026][20347] Updated weights for policy 0, policy_version 1372013 (0.0008) [2023-12-27 15:09:15,073][20381] Updated weights for policy 1, policy_version 1370824 (0.0008) [2023-12-27 15:09:15,082][20347] Updated weights for policy 0, policy_version 1372023 (0.0008) [2023-12-27 15:09:15,772][20381] Updated weights for policy 1, policy_version 1370834 (0.0006) [2023-12-27 15:09:15,785][20347] Updated weights for policy 0, policy_version 1372033 (0.0007) [2023-12-27 15:09:15,827][20381] Updated weights for policy 1, policy_version 1370844 (0.0008) [2023-12-27 15:09:15,840][20347] Updated weights for policy 0, policy_version 1372043 (0.0008) [2023-12-27 15:09:15,887][20381] Updated weights for policy 1, policy_version 1370854 (0.0006) [2023-12-27 15:09:15,901][20347] Updated weights for policy 0, policy_version 1372053 (0.0006) [2023-12-27 15:09:15,948][20381] Updated weights for policy 1, policy_version 1370864 (0.0008) [2023-12-27 15:09:16,606][20347] Updated weights for policy 0, policy_version 1372063 (0.0007) [2023-12-27 15:09:16,634][20381] Updated weights for policy 1, policy_version 1370874 (0.0006) [2023-12-27 15:09:16,657][20347] Updated weights for policy 0, policy_version 1372073 (0.0005) [2023-12-27 15:09:16,682][20381] Updated weights for policy 1, policy_version 1370884 (0.0006) [2023-12-27 15:09:16,710][20347] Updated weights for policy 0, policy_version 1372083 (0.0005) [2023-12-27 15:09:16,736][20381] Updated weights for policy 1, policy_version 1370894 (0.0006) [2023-12-27 15:09:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 702300160. Throughput: 0: 10212.1, 1: 10234.7. Samples: 702315368. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:17,363][19170] Avg episode reward: [(0, '882.900'), (1, '1000.000')] [2023-12-27 15:09:17,396][20381] Updated weights for policy 1, policy_version 1370904 (0.0007) [2023-12-27 15:09:17,426][20347] Updated weights for policy 0, policy_version 1372093 (0.0005) [2023-12-27 15:09:17,454][20381] Updated weights for policy 1, policy_version 1370914 (0.0007) [2023-12-27 15:09:17,481][20347] Updated weights for policy 0, policy_version 1372103 (0.0007) [2023-12-27 15:09:17,507][20381] Updated weights for policy 1, policy_version 1370924 (0.0007) [2023-12-27 15:09:17,535][20347] Updated weights for policy 0, policy_version 1372113 (0.0007) [2023-12-27 15:09:18,178][20381] Updated weights for policy 1, policy_version 1370934 (0.0007) [2023-12-27 15:09:18,224][20347] Updated weights for policy 0, policy_version 1372123 (0.0008) [2023-12-27 15:09:18,235][20381] Updated weights for policy 1, policy_version 1370944 (0.0007) [2023-12-27 15:09:18,282][20347] Updated weights for policy 0, policy_version 1372133 (0.0008) [2023-12-27 15:09:18,294][20381] Updated weights for policy 1, policy_version 1370954 (0.0006) [2023-12-27 15:09:18,346][20347] Updated weights for policy 0, policy_version 1372143 (0.0007) [2023-12-27 15:09:18,966][20381] Updated weights for policy 1, policy_version 1370964 (0.0007) [2023-12-27 15:09:19,034][20381] Updated weights for policy 1, policy_version 1370974 (0.0010) [2023-12-27 15:09:19,081][20347] Updated weights for policy 0, policy_version 1372153 (0.0007) [2023-12-27 15:09:19,100][20381] Updated weights for policy 1, policy_version 1370984 (0.0009) [2023-12-27 15:09:19,141][20347] Updated weights for policy 0, policy_version 1372163 (0.0007) [2023-12-27 15:09:19,196][20347] Updated weights for policy 0, policy_version 1372173 (0.0008) [2023-12-27 15:09:19,268][20347] Updated weights for policy 0, policy_version 1372183 (0.0008) [2023-12-27 15:09:19,775][20381] Updated weights for policy 1, policy_version 1370994 (0.0007) [2023-12-27 15:09:19,835][20381] Updated weights for policy 1, policy_version 1371004 (0.0008) [2023-12-27 15:09:19,897][20381] Updated weights for policy 1, policy_version 1371014 (0.0008) [2023-12-27 15:09:19,965][20381] Updated weights for policy 1, policy_version 1371024 (0.0008) [2023-12-27 15:09:20,065][20347] Updated weights for policy 0, policy_version 1372193 (0.0009) [2023-12-27 15:09:20,086][20291] KL-divergence is very high: 123.0693 [2023-12-27 15:09:20,136][20347] Updated weights for policy 0, policy_version 1372203 (0.0008) [2023-12-27 15:09:20,143][20291] KL-divergence is very high: 124.1378 [2023-12-27 15:09:20,202][20347] Updated weights for policy 0, policy_version 1372213 (0.0010) [2023-12-27 15:09:20,614][20381] Updated weights for policy 1, policy_version 1371034 (0.0008) [2023-12-27 15:09:20,671][20381] Updated weights for policy 1, policy_version 1371044 (0.0008) [2023-12-27 15:09:20,724][20381] Updated weights for policy 1, policy_version 1371054 (0.0008) [2023-12-27 15:09:20,886][20347] Updated weights for policy 0, policy_version 1372223 (0.0007) [2023-12-27 15:09:20,953][20347] Updated weights for policy 0, policy_version 1372233 (0.0006) [2023-12-27 15:09:21,016][20347] Updated weights for policy 0, policy_version 1372243 (0.0008) [2023-12-27 15:09:21,424][20381] Updated weights for policy 1, policy_version 1371064 (0.0008) [2023-12-27 15:09:21,486][20381] Updated weights for policy 1, policy_version 1371074 (0.0009) [2023-12-27 15:09:21,541][20381] Updated weights for policy 1, policy_version 1371084 (0.0009) [2023-12-27 15:09:21,667][20347] Updated weights for policy 0, policy_version 1372253 (0.0007) [2023-12-27 15:09:21,728][20347] Updated weights for policy 0, policy_version 1372263 (0.0007) [2023-12-27 15:09:21,796][20347] Updated weights for policy 0, policy_version 1372273 (0.0007) [2023-12-27 15:09:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 702398464. Throughput: 0: 10188.1, 1: 10214.5. Samples: 702373992. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:22,363][19170] Avg episode reward: [(0, '834.410'), (1, '1000.000')] [2023-12-27 15:09:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001372280_351371264.pth... [2023-12-27 15:09:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001371128_351076352.pth [2023-12-27 15:09:22,379][20381] Updated weights for policy 1, policy_version 1371094 (0.0009) [2023-12-27 15:09:22,435][20381] Updated weights for policy 1, policy_version 1371104 (0.0008) [2023-12-27 15:09:22,486][20347] Updated weights for policy 0, policy_version 1372283 (0.0008) [2023-12-27 15:09:22,489][20381] Updated weights for policy 1, policy_version 1371114 (0.0008) [2023-12-27 15:09:22,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001371120_351035392.pth... [2023-12-27 15:09:22,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001369904_350724096.pth [2023-12-27 15:09:22,550][20347] Updated weights for policy 0, policy_version 1372293 (0.0007) [2023-12-27 15:09:22,616][20347] Updated weights for policy 0, policy_version 1372303 (0.0008) [2023-12-27 15:09:23,150][20381] Updated weights for policy 1, policy_version 1371124 (0.0007) [2023-12-27 15:09:23,207][20381] Updated weights for policy 1, policy_version 1371134 (0.0007) [2023-12-27 15:09:23,264][20381] Updated weights for policy 1, policy_version 1371144 (0.0005) [2023-12-27 15:09:23,280][20347] Updated weights for policy 0, policy_version 1372313 (0.0007) [2023-12-27 15:09:23,343][20347] Updated weights for policy 0, policy_version 1372323 (0.0007) [2023-12-27 15:09:23,400][20347] Updated weights for policy 0, policy_version 1372333 (0.0005) [2023-12-27 15:09:23,457][20347] Updated weights for policy 0, policy_version 1372343 (0.0008) [2023-12-27 15:09:23,857][20381] Updated weights for policy 1, policy_version 1371154 (0.0007) [2023-12-27 15:09:23,913][20381] Updated weights for policy 1, policy_version 1371164 (0.0005) [2023-12-27 15:09:23,969][20381] Updated weights for policy 1, policy_version 1371174 (0.0005) [2023-12-27 15:09:24,027][20381] Updated weights for policy 1, policy_version 1371184 (0.0006) [2023-12-27 15:09:24,092][20347] Updated weights for policy 0, policy_version 1372353 (0.0007) [2023-12-27 15:09:24,145][20347] Updated weights for policy 0, policy_version 1372363 (0.0008) [2023-12-27 15:09:24,194][20347] Updated weights for policy 0, policy_version 1372373 (0.0007) [2023-12-27 15:09:24,677][20381] Updated weights for policy 1, policy_version 1371194 (0.0007) [2023-12-27 15:09:24,735][20381] Updated weights for policy 1, policy_version 1371204 (0.0007) [2023-12-27 15:09:24,792][20381] Updated weights for policy 1, policy_version 1371214 (0.0005) [2023-12-27 15:09:24,839][20347] Updated weights for policy 0, policy_version 1372383 (0.0008) [2023-12-27 15:09:24,884][20347] Updated weights for policy 0, policy_version 1372393 (0.0009) [2023-12-27 15:09:24,934][20347] Updated weights for policy 0, policy_version 1372403 (0.0009) [2023-12-27 15:09:25,382][20381] Updated weights for policy 1, policy_version 1371224 (0.0007) [2023-12-27 15:09:25,429][20381] Updated weights for policy 1, policy_version 1371234 (0.0008) [2023-12-27 15:09:25,473][20381] Updated weights for policy 1, policy_version 1371244 (0.0008) [2023-12-27 15:09:25,670][20347] Updated weights for policy 0, policy_version 1372413 (0.0008) [2023-12-27 15:09:25,729][20347] Updated weights for policy 0, policy_version 1372423 (0.0007) [2023-12-27 15:09:25,777][20347] Updated weights for policy 0, policy_version 1372433 (0.0007) [2023-12-27 15:09:26,155][20381] Updated weights for policy 1, policy_version 1371254 (0.0006) [2023-12-27 15:09:26,204][20381] Updated weights for policy 1, policy_version 1371264 (0.0007) [2023-12-27 15:09:26,258][20381] Updated weights for policy 1, policy_version 1371274 (0.0009) [2023-12-27 15:09:26,399][20347] Updated weights for policy 0, policy_version 1372443 (0.0008) [2023-12-27 15:09:26,466][20347] Updated weights for policy 0, policy_version 1372453 (0.0006) [2023-12-27 15:09:26,529][20347] Updated weights for policy 0, policy_version 1372463 (0.0010) [2023-12-27 15:09:26,888][20381] Updated weights for policy 1, policy_version 1371284 (0.0009) [2023-12-27 15:09:26,947][20381] Updated weights for policy 1, policy_version 1371294 (0.0005) [2023-12-27 15:09:27,013][20381] Updated weights for policy 1, policy_version 1371304 (0.0009) [2023-12-27 15:09:27,147][20347] Updated weights for policy 0, policy_version 1372473 (0.0009) [2023-12-27 15:09:27,202][20347] Updated weights for policy 0, policy_version 1372483 (0.0010) [2023-12-27 15:09:27,252][20347] Updated weights for policy 0, policy_version 1372493 (0.0010) [2023-12-27 15:09:27,307][20347] Updated weights for policy 0, policy_version 1372503 (0.0010) [2023-12-27 15:09:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 702513152. Throughput: 0: 10149.3, 1: 10302.8. Samples: 702497100. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:27,363][19170] Avg episode reward: [(0, '824.420'), (1, '1000.000')] [2023-12-27 15:09:27,725][20381] Updated weights for policy 1, policy_version 1371314 (0.0007) [2023-12-27 15:09:27,771][20381] Updated weights for policy 1, policy_version 1371324 (0.0009) [2023-12-27 15:09:27,814][20381] Updated weights for policy 1, policy_version 1371334 (0.0007) [2023-12-27 15:09:27,858][20381] Updated weights for policy 1, policy_version 1371344 (0.0005) [2023-12-27 15:09:27,909][20347] Updated weights for policy 0, policy_version 1372513 (0.0008) [2023-12-27 15:09:27,951][20291] KL-divergence is very high: 181.8030 [2023-12-27 15:09:27,961][20347] Updated weights for policy 0, policy_version 1372523 (0.0010) [2023-12-27 15:09:27,988][20291] KL-divergence is very high: 297.6400 [2023-12-27 15:09:28,011][20347] Updated weights for policy 0, policy_version 1372533 (0.0010) [2023-12-27 15:09:28,482][20381] Updated weights for policy 1, policy_version 1371354 (0.0007) [2023-12-27 15:09:28,539][20381] Updated weights for policy 1, policy_version 1371364 (0.0007) [2023-12-27 15:09:28,592][20381] Updated weights for policy 1, policy_version 1371374 (0.0006) [2023-12-27 15:09:28,675][20347] Updated weights for policy 0, policy_version 1372543 (0.0007) [2023-12-27 15:09:28,735][20347] Updated weights for policy 0, policy_version 1372553 (0.0008) [2023-12-27 15:09:28,793][20347] Updated weights for policy 0, policy_version 1372563 (0.0010) [2023-12-27 15:09:29,288][20381] Updated weights for policy 1, policy_version 1371384 (0.0008) [2023-12-27 15:09:29,349][20381] Updated weights for policy 1, policy_version 1371394 (0.0008) [2023-12-27 15:09:29,413][20381] Updated weights for policy 1, policy_version 1371404 (0.0008) [2023-12-27 15:09:29,441][20347] Updated weights for policy 0, policy_version 1372573 (0.0008) [2023-12-27 15:09:29,499][20347] Updated weights for policy 0, policy_version 1372583 (0.0007) [2023-12-27 15:09:29,554][20347] Updated weights for policy 0, policy_version 1372593 (0.0010) [2023-12-27 15:09:30,141][20381] Updated weights for policy 1, policy_version 1371414 (0.0008) [2023-12-27 15:09:30,194][20347] Updated weights for policy 0, policy_version 1372603 (0.0009) [2023-12-27 15:09:30,200][20381] Updated weights for policy 1, policy_version 1371424 (0.0007) [2023-12-27 15:09:30,250][20347] Updated weights for policy 0, policy_version 1372613 (0.0006) [2023-12-27 15:09:30,252][20381] Updated weights for policy 1, policy_version 1371434 (0.0007) [2023-12-27 15:09:30,281][20291] KL-divergence is very high: 117.4907 [2023-12-27 15:09:30,308][20347] Updated weights for policy 0, policy_version 1372623 (0.0007) [2023-12-27 15:09:30,326][20291] KL-divergence is very high: 136.3857 [2023-12-27 15:09:30,871][20347] Updated weights for policy 0, policy_version 1372633 (0.0007) [2023-12-27 15:09:30,925][20381] Updated weights for policy 1, policy_version 1371444 (0.0006) [2023-12-27 15:09:30,927][20347] Updated weights for policy 0, policy_version 1372643 (0.0005) [2023-12-27 15:09:30,973][20381] Updated weights for policy 1, policy_version 1371454 (0.0006) [2023-12-27 15:09:30,975][20347] Updated weights for policy 0, policy_version 1372653 (0.0005) [2023-12-27 15:09:31,018][20381] Updated weights for policy 1, policy_version 1371464 (0.0007) [2023-12-27 15:09:31,031][20347] Updated weights for policy 0, policy_version 1372663 (0.0007) [2023-12-27 15:09:31,670][20347] Updated weights for policy 0, policy_version 1372673 (0.0008) [2023-12-27 15:09:31,736][20347] Updated weights for policy 0, policy_version 1372683 (0.0007) [2023-12-27 15:09:31,757][20291] KL-divergence is very high: 154.0289 [2023-12-27 15:09:31,759][20381] Updated weights for policy 1, policy_version 1371474 (0.0008) [2023-12-27 15:09:31,790][20347] Updated weights for policy 0, policy_version 1372693 (0.0008) [2023-12-27 15:09:31,801][20291] KL-divergence is very high: 104.9157 [2023-12-27 15:09:31,811][20381] Updated weights for policy 1, policy_version 1371484 (0.0006) [2023-12-27 15:09:31,866][20381] Updated weights for policy 1, policy_version 1371494 (0.0006) [2023-12-27 15:09:31,923][20381] Updated weights for policy 1, policy_version 1371504 (0.0008) [2023-12-27 15:09:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 702611456. Throughput: 0: 10214.8, 1: 10274.1. Samples: 702624804. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:32,363][19170] Avg episode reward: [(0, '853.600'), (1, '1000.000')] [2023-12-27 15:09:32,366][20347] Updated weights for policy 0, policy_version 1372703 (0.0007) [2023-12-27 15:09:32,418][20347] Updated weights for policy 0, policy_version 1372713 (0.0008) [2023-12-27 15:09:32,473][20347] Updated weights for policy 0, policy_version 1372723 (0.0007) [2023-12-27 15:09:32,557][20381] Updated weights for policy 1, policy_version 1371514 (0.0008) [2023-12-27 15:09:32,611][20381] Updated weights for policy 1, policy_version 1371524 (0.0008) [2023-12-27 15:09:32,658][20381] Updated weights for policy 1, policy_version 1371534 (0.0008) [2023-12-27 15:09:33,195][20347] Updated weights for policy 0, policy_version 1372733 (0.0008) [2023-12-27 15:09:33,247][20347] Updated weights for policy 0, policy_version 1372743 (0.0008) [2023-12-27 15:09:33,298][20347] Updated weights for policy 0, policy_version 1372753 (0.0008) [2023-12-27 15:09:33,412][20381] Updated weights for policy 1, policy_version 1371544 (0.0010) [2023-12-27 15:09:33,460][20381] Updated weights for policy 1, policy_version 1371554 (0.0010) [2023-12-27 15:09:33,514][20381] Updated weights for policy 1, policy_version 1371564 (0.0011) [2023-12-27 15:09:34,032][20347] Updated weights for policy 0, policy_version 1372763 (0.0009) [2023-12-27 15:09:34,080][20347] Updated weights for policy 0, policy_version 1372773 (0.0006) [2023-12-27 15:09:34,093][20381] Updated weights for policy 1, policy_version 1371574 (0.0011) [2023-12-27 15:09:34,136][20347] Updated weights for policy 0, policy_version 1372783 (0.0006) [2023-12-27 15:09:34,167][20381] Updated weights for policy 1, policy_version 1371584 (0.0010) [2023-12-27 15:09:34,228][20381] Updated weights for policy 1, policy_version 1371594 (0.0010) [2023-12-27 15:09:34,843][20347] Updated weights for policy 0, policy_version 1372793 (0.0007) [2023-12-27 15:09:34,890][20347] Updated weights for policy 0, policy_version 1372803 (0.0007) [2023-12-27 15:09:34,911][20381] Updated weights for policy 1, policy_version 1371604 (0.0011) [2023-12-27 15:09:34,946][20347] Updated weights for policy 0, policy_version 1372813 (0.0006) [2023-12-27 15:09:34,966][20381] Updated weights for policy 1, policy_version 1371614 (0.0010) [2023-12-27 15:09:35,001][20347] Updated weights for policy 0, policy_version 1372823 (0.0005) [2023-12-27 15:09:35,019][20381] Updated weights for policy 1, policy_version 1371624 (0.0011) [2023-12-27 15:09:35,666][20347] Updated weights for policy 0, policy_version 1372833 (0.0007) [2023-12-27 15:09:35,714][20381] Updated weights for policy 1, policy_version 1371634 (0.0010) [2023-12-27 15:09:35,716][20347] Updated weights for policy 0, policy_version 1372843 (0.0007) [2023-12-27 15:09:35,769][20381] Updated weights for policy 1, policy_version 1371644 (0.0006) [2023-12-27 15:09:35,771][20347] Updated weights for policy 0, policy_version 1372853 (0.0005) [2023-12-27 15:09:35,827][20381] Updated weights for policy 1, policy_version 1371654 (0.0006) [2023-12-27 15:09:35,883][20381] Updated weights for policy 1, policy_version 1371664 (0.0005) [2023-12-27 15:09:36,488][20381] Updated weights for policy 1, policy_version 1371674 (0.0008) [2023-12-27 15:09:36,503][20347] Updated weights for policy 0, policy_version 1372863 (0.0008) [2023-12-27 15:09:36,544][20381] Updated weights for policy 1, policy_version 1371684 (0.0009) [2023-12-27 15:09:36,557][20347] Updated weights for policy 0, policy_version 1372873 (0.0008) [2023-12-27 15:09:36,606][20381] Updated weights for policy 1, policy_version 1371694 (0.0008) [2023-12-27 15:09:36,617][20347] Updated weights for policy 0, policy_version 1372883 (0.0007) [2023-12-27 15:09:37,213][20347] Updated weights for policy 0, policy_version 1372893 (0.0007) [2023-12-27 15:09:37,268][20347] Updated weights for policy 0, policy_version 1372903 (0.0005) [2023-12-27 15:09:37,319][20347] Updated weights for policy 0, policy_version 1372913 (0.0005) [2023-12-27 15:09:37,322][20381] Updated weights for policy 1, policy_version 1371704 (0.0008) [2023-12-27 15:09:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 702717952. Throughput: 0: 10184.4, 1: 10284.2. Samples: 702685244. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:37,363][19170] Avg episode reward: [(0, '884.080'), (1, '1000.000')] [2023-12-27 15:09:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001372920_351535104.pth... [2023-12-27 15:09:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001371704_351223808.pth [2023-12-27 15:09:37,395][20381] Updated weights for policy 1, policy_version 1371714 (0.0008) [2023-12-27 15:09:37,460][20381] Updated weights for policy 1, policy_version 1371724 (0.0008) [2023-12-27 15:09:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001371728_351191040.pth... [2023-12-27 15:09:37,489][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001370512_350879744.pth [2023-12-27 15:09:37,861][20347] Updated weights for policy 0, policy_version 1372923 (0.0005) [2023-12-27 15:09:37,916][20347] Updated weights for policy 0, policy_version 1372933 (0.0005) [2023-12-27 15:09:37,975][20347] Updated weights for policy 0, policy_version 1372943 (0.0005) [2023-12-27 15:09:38,128][20381] Updated weights for policy 1, policy_version 1371734 (0.0007) [2023-12-27 15:09:38,186][20381] Updated weights for policy 1, policy_version 1371744 (0.0006) [2023-12-27 15:09:38,241][20381] Updated weights for policy 1, policy_version 1371754 (0.0005) [2023-12-27 15:09:38,556][20347] Updated weights for policy 0, policy_version 1372953 (0.0005) [2023-12-27 15:09:38,623][20347] Updated weights for policy 0, policy_version 1372963 (0.0008) [2023-12-27 15:09:38,691][20347] Updated weights for policy 0, policy_version 1372973 (0.0008) [2023-12-27 15:09:38,755][20347] Updated weights for policy 0, policy_version 1372983 (0.0008) [2023-12-27 15:09:38,820][20381] Updated weights for policy 1, policy_version 1371764 (0.0006) [2023-12-27 15:09:38,878][20381] Updated weights for policy 1, policy_version 1371774 (0.0007) [2023-12-27 15:09:38,931][20381] Updated weights for policy 1, policy_version 1371784 (0.0008) [2023-12-27 15:09:39,415][20347] Updated weights for policy 0, policy_version 1372993 (0.0009) [2023-12-27 15:09:39,478][20347] Updated weights for policy 0, policy_version 1373003 (0.0006) [2023-12-27 15:09:39,539][20347] Updated weights for policy 0, policy_version 1373013 (0.0007) [2023-12-27 15:09:39,676][20381] Updated weights for policy 1, policy_version 1371794 (0.0007) [2023-12-27 15:09:39,738][20381] Updated weights for policy 1, policy_version 1371804 (0.0006) [2023-12-27 15:09:39,798][20381] Updated weights for policy 1, policy_version 1371814 (0.0006) [2023-12-27 15:09:39,863][20381] Updated weights for policy 1, policy_version 1371824 (0.0008) [2023-12-27 15:09:40,161][20347] Updated weights for policy 0, policy_version 1373023 (0.0006) [2023-12-27 15:09:40,225][20347] Updated weights for policy 0, policy_version 1373033 (0.0006) [2023-12-27 15:09:40,290][20347] Updated weights for policy 0, policy_version 1373043 (0.0006) [2023-12-27 15:09:40,615][20381] Updated weights for policy 1, policy_version 1371834 (0.0008) [2023-12-27 15:09:40,661][20381] Updated weights for policy 1, policy_version 1371844 (0.0008) [2023-12-27 15:09:40,711][20381] Updated weights for policy 1, policy_version 1371854 (0.0008) [2023-12-27 15:09:40,902][20347] Updated weights for policy 0, policy_version 1373053 (0.0007) [2023-12-27 15:09:40,959][20347] Updated weights for policy 0, policy_version 1373063 (0.0006) [2023-12-27 15:09:41,016][20347] Updated weights for policy 0, policy_version 1373073 (0.0006) [2023-12-27 15:09:41,494][20381] Updated weights for policy 1, policy_version 1371864 (0.0011) [2023-12-27 15:09:41,549][20381] Updated weights for policy 1, policy_version 1371874 (0.0011) [2023-12-27 15:09:41,610][20381] Updated weights for policy 1, policy_version 1371884 (0.0011) [2023-12-27 15:09:41,727][20347] Updated weights for policy 0, policy_version 1373083 (0.0008) [2023-12-27 15:09:41,791][20347] Updated weights for policy 0, policy_version 1373093 (0.0008) [2023-12-27 15:09:41,853][20347] Updated weights for policy 0, policy_version 1373103 (0.0007) [2023-12-27 15:09:42,314][20381] Updated weights for policy 1, policy_version 1371894 (0.0009) [2023-12-27 15:09:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.5, 300 sec: 20493.9). Total num frames: 702816256. Throughput: 0: 10280.5, 1: 10205.4. Samples: 702810812. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:42,363][19170] Avg episode reward: [(0, '893.610'), (1, '1000.000')] [2023-12-27 15:09:42,381][20381] Updated weights for policy 1, policy_version 1371904 (0.0009) [2023-12-27 15:09:42,440][20381] Updated weights for policy 1, policy_version 1371914 (0.0008) [2023-12-27 15:09:42,572][20347] Updated weights for policy 0, policy_version 1373113 (0.0008) [2023-12-27 15:09:42,631][20347] Updated weights for policy 0, policy_version 1373123 (0.0007) [2023-12-27 15:09:42,683][20347] Updated weights for policy 0, policy_version 1373133 (0.0008) [2023-12-27 15:09:42,728][20347] Updated weights for policy 0, policy_version 1373143 (0.0007) [2023-12-27 15:09:43,125][20381] Updated weights for policy 1, policy_version 1371924 (0.0007) [2023-12-27 15:09:43,186][20381] Updated weights for policy 1, policy_version 1371934 (0.0010) [2023-12-27 15:09:43,240][20381] Updated weights for policy 1, policy_version 1371944 (0.0010) [2023-12-27 15:09:43,381][20347] Updated weights for policy 0, policy_version 1373153 (0.0007) [2023-12-27 15:09:43,440][20347] Updated weights for policy 0, policy_version 1373163 (0.0008) [2023-12-27 15:09:43,494][20347] Updated weights for policy 0, policy_version 1373173 (0.0005) [2023-12-27 15:09:43,784][20381] Updated weights for policy 1, policy_version 1371954 (0.0010) [2023-12-27 15:09:43,847][20381] Updated weights for policy 1, policy_version 1371964 (0.0010) [2023-12-27 15:09:43,898][20381] Updated weights for policy 1, policy_version 1371974 (0.0010) [2023-12-27 15:09:43,952][20381] Updated weights for policy 1, policy_version 1371984 (0.0010) [2023-12-27 15:09:44,091][20347] Updated weights for policy 0, policy_version 1373183 (0.0008) [2023-12-27 15:09:44,146][20347] Updated weights for policy 0, policy_version 1373193 (0.0008) [2023-12-27 15:09:44,197][20347] Updated weights for policy 0, policy_version 1373203 (0.0009) [2023-12-27 15:09:44,598][20381] Updated weights for policy 1, policy_version 1371994 (0.0011) [2023-12-27 15:09:44,657][20381] Updated weights for policy 1, policy_version 1372004 (0.0010) [2023-12-27 15:09:44,717][20381] Updated weights for policy 1, policy_version 1372014 (0.0010) [2023-12-27 15:09:44,788][20347] Updated weights for policy 0, policy_version 1373213 (0.0009) [2023-12-27 15:09:44,847][20347] Updated weights for policy 0, policy_version 1373223 (0.0007) [2023-12-27 15:09:44,898][20347] Updated weights for policy 0, policy_version 1373233 (0.0010) [2023-12-27 15:09:45,409][20381] Updated weights for policy 1, policy_version 1372024 (0.0011) [2023-12-27 15:09:45,467][20381] Updated weights for policy 1, policy_version 1372034 (0.0011) [2023-12-27 15:09:45,516][20381] Updated weights for policy 1, policy_version 1372044 (0.0010) [2023-12-27 15:09:45,628][20347] Updated weights for policy 0, policy_version 1373243 (0.0010) [2023-12-27 15:09:45,686][20347] Updated weights for policy 0, policy_version 1373253 (0.0011) [2023-12-27 15:09:45,736][20347] Updated weights for policy 0, policy_version 1373263 (0.0007) [2023-12-27 15:09:46,167][20381] Updated weights for policy 1, policy_version 1372054 (0.0008) [2023-12-27 15:09:46,234][20381] Updated weights for policy 1, policy_version 1372064 (0.0006) [2023-12-27 15:09:46,302][20381] Updated weights for policy 1, policy_version 1372074 (0.0005) [2023-12-27 15:09:46,317][20347] Updated weights for policy 0, policy_version 1373273 (0.0006) [2023-12-27 15:09:46,374][20347] Updated weights for policy 0, policy_version 1373283 (0.0005) [2023-12-27 15:09:46,435][20347] Updated weights for policy 0, policy_version 1373293 (0.0006) [2023-12-27 15:09:46,493][20347] Updated weights for policy 0, policy_version 1373303 (0.0011) [2023-12-27 15:09:46,915][20381] Updated weights for policy 1, policy_version 1372084 (0.0009) [2023-12-27 15:09:46,961][20381] Updated weights for policy 1, policy_version 1372094 (0.0005) [2023-12-27 15:09:47,010][20381] Updated weights for policy 1, policy_version 1372104 (0.0005) [2023-12-27 15:09:47,067][20347] Updated weights for policy 0, policy_version 1373313 (0.0008) [2023-12-27 15:09:47,135][20347] Updated weights for policy 0, policy_version 1373323 (0.0010) [2023-12-27 15:09:47,197][20347] Updated weights for policy 0, policy_version 1373333 (0.0010) [2023-12-27 15:09:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 702930944. Throughput: 0: 10386.9, 1: 10272.9. Samples: 702938088. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:47,364][19170] Avg episode reward: [(0, '882.660'), (1, '1000.000')] [2023-12-27 15:09:47,589][20381] Updated weights for policy 1, policy_version 1372114 (0.0007) [2023-12-27 15:09:47,640][20381] Updated weights for policy 1, policy_version 1372124 (0.0010) [2023-12-27 15:09:47,691][20381] Updated weights for policy 1, policy_version 1372134 (0.0010) [2023-12-27 15:09:47,751][20381] Updated weights for policy 1, policy_version 1372144 (0.0009) [2023-12-27 15:09:47,817][20347] Updated weights for policy 0, policy_version 1373343 (0.0007) [2023-12-27 15:09:47,873][20347] Updated weights for policy 0, policy_version 1373353 (0.0005) [2023-12-27 15:09:47,930][20347] Updated weights for policy 0, policy_version 1373363 (0.0005) [2023-12-27 15:09:48,371][20381] Updated weights for policy 1, policy_version 1372154 (0.0008) [2023-12-27 15:09:48,428][20381] Updated weights for policy 1, policy_version 1372164 (0.0008) [2023-12-27 15:09:48,488][20381] Updated weights for policy 1, policy_version 1372174 (0.0008) [2023-12-27 15:09:48,567][20347] Updated weights for policy 0, policy_version 1373373 (0.0005) [2023-12-27 15:09:48,628][20347] Updated weights for policy 0, policy_version 1373383 (0.0009) [2023-12-27 15:09:48,685][20347] Updated weights for policy 0, policy_version 1373393 (0.0010) [2023-12-27 15:09:49,233][20381] Updated weights for policy 1, policy_version 1372184 (0.0010) [2023-12-27 15:09:49,294][20347] Updated weights for policy 0, policy_version 1373403 (0.0009) [2023-12-27 15:09:49,297][20381] Updated weights for policy 1, policy_version 1372194 (0.0011) [2023-12-27 15:09:49,355][20347] Updated weights for policy 0, policy_version 1373413 (0.0008) [2023-12-27 15:09:49,364][20381] Updated weights for policy 1, policy_version 1372204 (0.0011) [2023-12-27 15:09:49,424][20347] Updated weights for policy 0, policy_version 1373423 (0.0008) [2023-12-27 15:09:50,061][20381] Updated weights for policy 1, policy_version 1372214 (0.0010) [2023-12-27 15:09:50,116][20347] Updated weights for policy 0, policy_version 1373433 (0.0007) [2023-12-27 15:09:50,125][20381] Updated weights for policy 1, policy_version 1372224 (0.0009) [2023-12-27 15:09:50,171][20347] Updated weights for policy 0, policy_version 1373443 (0.0006) [2023-12-27 15:09:50,188][20381] Updated weights for policy 1, policy_version 1372234 (0.0006) [2023-12-27 15:09:50,230][20347] Updated weights for policy 0, policy_version 1373453 (0.0008) [2023-12-27 15:09:50,290][20347] Updated weights for policy 0, policy_version 1373463 (0.0010) [2023-12-27 15:09:50,860][20381] Updated weights for policy 1, policy_version 1372244 (0.0009) [2023-12-27 15:09:50,914][20381] Updated weights for policy 1, policy_version 1372254 (0.0011) [2023-12-27 15:09:50,973][20381] Updated weights for policy 1, policy_version 1372264 (0.0010) [2023-12-27 15:09:50,977][20347] Updated weights for policy 0, policy_version 1373473 (0.0007) [2023-12-27 15:09:51,043][20347] Updated weights for policy 0, policy_version 1373483 (0.0009) [2023-12-27 15:09:51,107][20347] Updated weights for policy 0, policy_version 1373493 (0.0008) [2023-12-27 15:09:51,750][20381] Updated weights for policy 1, policy_version 1372274 (0.0010) [2023-12-27 15:09:51,817][20381] Updated weights for policy 1, policy_version 1372284 (0.0007) [2023-12-27 15:09:51,881][20381] Updated weights for policy 1, policy_version 1372294 (0.0008) [2023-12-27 15:09:51,894][20347] Updated weights for policy 0, policy_version 1373503 (0.0008) [2023-12-27 15:09:51,944][20381] Updated weights for policy 1, policy_version 1372304 (0.0007) [2023-12-27 15:09:51,952][20347] Updated weights for policy 0, policy_version 1373513 (0.0006) [2023-12-27 15:09:52,003][20347] Updated weights for policy 0, policy_version 1373523 (0.0008) [2023-12-27 15:09:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 703029248. Throughput: 0: 10397.1, 1: 10270.4. Samples: 703000940. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:52,363][19170] Avg episode reward: [(0, '892.480'), (1, '1000.000')] [2023-12-27 15:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001372304_351338496.pth... [2023-12-27 15:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001373528_351690752.pth... [2023-12-27 15:09:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001372280_351371264.pth [2023-12-27 15:09:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001371120_351035392.pth [2023-12-27 15:09:52,603][20381] Updated weights for policy 1, policy_version 1372314 (0.0007) [2023-12-27 15:09:52,662][20381] Updated weights for policy 1, policy_version 1372324 (0.0008) [2023-12-27 15:09:52,722][20381] Updated weights for policy 1, policy_version 1372334 (0.0006) [2023-12-27 15:09:52,727][20347] Updated weights for policy 0, policy_version 1373533 (0.0008) [2023-12-27 15:09:52,787][20347] Updated weights for policy 0, policy_version 1373543 (0.0006) [2023-12-27 15:09:52,844][20347] Updated weights for policy 0, policy_version 1373553 (0.0008) [2023-12-27 15:09:53,343][20381] Updated weights for policy 1, policy_version 1372344 (0.0008) [2023-12-27 15:09:53,402][20381] Updated weights for policy 1, policy_version 1372354 (0.0008) [2023-12-27 15:09:53,459][20381] Updated weights for policy 1, policy_version 1372364 (0.0006) [2023-12-27 15:09:53,487][20347] Updated weights for policy 0, policy_version 1373563 (0.0007) [2023-12-27 15:09:53,536][20347] Updated weights for policy 0, policy_version 1373573 (0.0006) [2023-12-27 15:09:53,584][20347] Updated weights for policy 0, policy_version 1373583 (0.0008) [2023-12-27 15:09:54,023][20381] Updated weights for policy 1, policy_version 1372374 (0.0008) [2023-12-27 15:09:54,068][20381] Updated weights for policy 1, policy_version 1372384 (0.0010) [2023-12-27 15:09:54,113][20381] Updated weights for policy 1, policy_version 1372394 (0.0010) [2023-12-27 15:09:54,320][20347] Updated weights for policy 0, policy_version 1373593 (0.0008) [2023-12-27 15:09:54,379][20347] Updated weights for policy 0, policy_version 1373603 (0.0007) [2023-12-27 15:09:54,434][20347] Updated weights for policy 0, policy_version 1373613 (0.0008) [2023-12-27 15:09:54,496][20347] Updated weights for policy 0, policy_version 1373623 (0.0007) [2023-12-27 15:09:54,769][20381] Updated weights for policy 1, policy_version 1372404 (0.0008) [2023-12-27 15:09:54,800][20292] KL-divergence is very high: 139.4282 [2023-12-27 15:09:54,814][20381] Updated weights for policy 1, policy_version 1372414 (0.0005) [2023-12-27 15:09:54,824][20292] KL-divergence is very high: 156.9207 [2023-12-27 15:09:54,838][20292] KL-divergence is very high: 151.3308 [2023-12-27 15:09:54,860][20381] Updated weights for policy 1, policy_version 1372424 (0.0006) [2023-12-27 15:09:55,190][20347] Updated weights for policy 0, policy_version 1373633 (0.0009) [2023-12-27 15:09:55,238][20347] Updated weights for policy 0, policy_version 1373643 (0.0007) [2023-12-27 15:09:55,292][20347] Updated weights for policy 0, policy_version 1373653 (0.0007) [2023-12-27 15:09:55,514][20381] Updated weights for policy 1, policy_version 1372434 (0.0008) [2023-12-27 15:09:55,566][20381] Updated weights for policy 1, policy_version 1372444 (0.0008) [2023-12-27 15:09:55,616][20381] Updated weights for policy 1, policy_version 1372454 (0.0010) [2023-12-27 15:09:55,668][20381] Updated weights for policy 1, policy_version 1372464 (0.0008) [2023-12-27 15:09:55,977][20347] Updated weights for policy 0, policy_version 1373663 (0.0007) [2023-12-27 15:09:56,037][20347] Updated weights for policy 0, policy_version 1373673 (0.0007) [2023-12-27 15:09:56,097][20347] Updated weights for policy 0, policy_version 1373683 (0.0008) [2023-12-27 15:09:56,335][20381] Updated weights for policy 1, policy_version 1372474 (0.0005) [2023-12-27 15:09:56,388][20381] Updated weights for policy 1, policy_version 1372484 (0.0005) [2023-12-27 15:09:56,443][20381] Updated weights for policy 1, policy_version 1372494 (0.0005) [2023-12-27 15:09:56,796][20347] Updated weights for policy 0, policy_version 1373693 (0.0008) [2023-12-27 15:09:56,848][20347] Updated weights for policy 0, policy_version 1373703 (0.0007) [2023-12-27 15:09:56,906][20347] Updated weights for policy 0, policy_version 1373713 (0.0008) [2023-12-27 15:09:57,010][20381] Updated weights for policy 1, policy_version 1372504 (0.0006) [2023-12-27 15:09:57,059][20381] Updated weights for policy 1, policy_version 1372514 (0.0008) [2023-12-27 15:09:57,112][20381] Updated weights for policy 1, policy_version 1372524 (0.0008) [2023-12-27 15:09:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 703135744. Throughput: 0: 10308.1, 1: 10307.0. Samples: 703122844. Policy #0 lag: (min: 24.0, avg: 47.1, max: 48.0) [2023-12-27 15:09:57,363][19170] Avg episode reward: [(0, '873.100'), (1, '990.230')] [2023-12-27 15:09:57,621][20347] Updated weights for policy 0, policy_version 1373723 (0.0007) [2023-12-27 15:09:57,672][20347] Updated weights for policy 0, policy_version 1373733 (0.0007) [2023-12-27 15:09:57,726][20347] Updated weights for policy 0, policy_version 1373743 (0.0007) [2023-12-27 15:09:57,798][20381] Updated weights for policy 1, policy_version 1372534 (0.0010) [2023-12-27 15:09:57,849][20381] Updated weights for policy 1, policy_version 1372544 (0.0010) [2023-12-27 15:09:57,916][20381] Updated weights for policy 1, policy_version 1372554 (0.0010) [2023-12-27 15:09:58,303][20347] Updated weights for policy 0, policy_version 1373753 (0.0005) [2023-12-27 15:09:58,384][20347] Updated weights for policy 0, policy_version 1373763 (0.0008) [2023-12-27 15:09:58,449][20347] Updated weights for policy 0, policy_version 1373773 (0.0008) [2023-12-27 15:09:58,512][20347] Updated weights for policy 0, policy_version 1373783 (0.0007) [2023-12-27 15:09:58,644][20381] Updated weights for policy 1, policy_version 1372564 (0.0010) [2023-12-27 15:09:58,701][20381] Updated weights for policy 1, policy_version 1372574 (0.0009) [2023-12-27 15:09:58,773][20381] Updated weights for policy 1, policy_version 1372584 (0.0007) [2023-12-27 15:09:59,253][20347] Updated weights for policy 0, policy_version 1373793 (0.0008) [2023-12-27 15:09:59,309][20347] Updated weights for policy 0, policy_version 1373803 (0.0007) [2023-12-27 15:09:59,365][20347] Updated weights for policy 0, policy_version 1373813 (0.0008) [2023-12-27 15:09:59,564][20381] Updated weights for policy 1, policy_version 1372594 (0.0010) [2023-12-27 15:09:59,623][20381] Updated weights for policy 1, policy_version 1372604 (0.0011) [2023-12-27 15:09:59,687][20381] Updated weights for policy 1, policy_version 1372614 (0.0011) [2023-12-27 15:09:59,748][20381] Updated weights for policy 1, policy_version 1372624 (0.0011) [2023-12-27 15:10:00,026][20347] Updated weights for policy 0, policy_version 1373823 (0.0008) [2023-12-27 15:10:00,070][20347] Updated weights for policy 0, policy_version 1373833 (0.0008) [2023-12-27 15:10:00,124][20347] Updated weights for policy 0, policy_version 1373844 (0.0008) [2023-12-27 15:10:00,393][20381] Updated weights for policy 1, policy_version 1372634 (0.0005) [2023-12-27 15:10:00,450][20381] Updated weights for policy 1, policy_version 1372644 (0.0007) [2023-12-27 15:10:00,499][20381] Updated weights for policy 1, policy_version 1372654 (0.0010) [2023-12-27 15:10:00,773][20347] Updated weights for policy 0, policy_version 1373854 (0.0008) [2023-12-27 15:10:00,818][20347] Updated weights for policy 0, policy_version 1373864 (0.0008) [2023-12-27 15:10:00,871][20347] Updated weights for policy 0, policy_version 1373874 (0.0007) [2023-12-27 15:10:01,066][20381] Updated weights for policy 1, policy_version 1372664 (0.0010) [2023-12-27 15:10:01,125][20381] Updated weights for policy 1, policy_version 1372674 (0.0008) [2023-12-27 15:10:01,182][20381] Updated weights for policy 1, policy_version 1372684 (0.0008) [2023-12-27 15:10:01,644][20347] Updated weights for policy 0, policy_version 1373884 (0.0007) [2023-12-27 15:10:01,703][20347] Updated weights for policy 0, policy_version 1373894 (0.0005) [2023-12-27 15:10:01,763][20347] Updated weights for policy 0, policy_version 1373904 (0.0008) [2023-12-27 15:10:01,894][20381] Updated weights for policy 1, policy_version 1372694 (0.0007) [2023-12-27 15:10:01,944][20381] Updated weights for policy 1, policy_version 1372704 (0.0006) [2023-12-27 15:10:01,991][20381] Updated weights for policy 1, policy_version 1372714 (0.0008) [2023-12-27 15:10:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 703234048. Throughput: 0: 10370.5, 1: 10290.6. Samples: 703245116. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:02,363][19170] Avg episode reward: [(0, '902.360'), (1, '990.230')] [2023-12-27 15:10:02,441][20347] Updated weights for policy 0, policy_version 1373914 (0.0007) [2023-12-27 15:10:02,507][20347] Updated weights for policy 0, policy_version 1373924 (0.0005) [2023-12-27 15:10:02,573][20347] Updated weights for policy 0, policy_version 1373934 (0.0005) [2023-12-27 15:10:02,636][20347] Updated weights for policy 0, policy_version 1373944 (0.0005) [2023-12-27 15:10:02,669][20381] Updated weights for policy 1, policy_version 1372724 (0.0008) [2023-12-27 15:10:02,723][20381] Updated weights for policy 1, policy_version 1372734 (0.0010) [2023-12-27 15:10:02,778][20381] Updated weights for policy 1, policy_version 1372744 (0.0009) [2023-12-27 15:10:03,253][20347] Updated weights for policy 0, policy_version 1373954 (0.0010) [2023-12-27 15:10:03,297][20347] Updated weights for policy 0, policy_version 1373964 (0.0010) [2023-12-27 15:10:03,347][20347] Updated weights for policy 0, policy_version 1373974 (0.0006) [2023-12-27 15:10:03,466][20381] Updated weights for policy 1, policy_version 1372754 (0.0010) [2023-12-27 15:10:03,520][20381] Updated weights for policy 1, policy_version 1372764 (0.0010) [2023-12-27 15:10:03,572][20381] Updated weights for policy 1, policy_version 1372774 (0.0010) [2023-12-27 15:10:03,619][20381] Updated weights for policy 1, policy_version 1372784 (0.0010) [2023-12-27 15:10:03,996][20347] Updated weights for policy 0, policy_version 1373984 (0.0009) [2023-12-27 15:10:04,057][20347] Updated weights for policy 0, policy_version 1373994 (0.0010) [2023-12-27 15:10:04,121][20347] Updated weights for policy 0, policy_version 1374004 (0.0010) [2023-12-27 15:10:04,339][20381] Updated weights for policy 1, policy_version 1372794 (0.0010) [2023-12-27 15:10:04,401][20381] Updated weights for policy 1, policy_version 1372804 (0.0011) [2023-12-27 15:10:04,461][20381] Updated weights for policy 1, policy_version 1372814 (0.0011) [2023-12-27 15:10:04,702][20347] Updated weights for policy 0, policy_version 1374014 (0.0009) [2023-12-27 15:10:04,748][20347] Updated weights for policy 0, policy_version 1374024 (0.0010) [2023-12-27 15:10:04,800][20347] Updated weights for policy 0, policy_version 1374034 (0.0010) [2023-12-27 15:10:05,139][20381] Updated weights for policy 1, policy_version 1372824 (0.0011) [2023-12-27 15:10:05,196][20381] Updated weights for policy 1, policy_version 1372834 (0.0010) [2023-12-27 15:10:05,248][20381] Updated weights for policy 1, policy_version 1372844 (0.0010) [2023-12-27 15:10:05,453][20347] Updated weights for policy 0, policy_version 1374044 (0.0010) [2023-12-27 15:10:05,511][20347] Updated weights for policy 0, policy_version 1374054 (0.0010) [2023-12-27 15:10:05,568][20347] Updated weights for policy 0, policy_version 1374064 (0.0010) [2023-12-27 15:10:05,942][20381] Updated weights for policy 1, policy_version 1372854 (0.0011) [2023-12-27 15:10:05,994][20381] Updated weights for policy 1, policy_version 1372864 (0.0011) [2023-12-27 15:10:06,053][20381] Updated weights for policy 1, policy_version 1372874 (0.0010) [2023-12-27 15:10:06,215][20347] Updated weights for policy 0, policy_version 1374074 (0.0006) [2023-12-27 15:10:06,283][20347] Updated weights for policy 0, policy_version 1374084 (0.0009) [2023-12-27 15:10:06,342][20347] Updated weights for policy 0, policy_version 1374094 (0.0008) [2023-12-27 15:10:06,401][20347] Updated weights for policy 0, policy_version 1374104 (0.0008) [2023-12-27 15:10:06,773][20381] Updated weights for policy 1, policy_version 1372884 (0.0011) [2023-12-27 15:10:06,824][20381] Updated weights for policy 1, policy_version 1372894 (0.0010) [2023-12-27 15:10:06,880][20381] Updated weights for policy 1, policy_version 1372904 (0.0010) [2023-12-27 15:10:07,073][20347] Updated weights for policy 0, policy_version 1374114 (0.0005) [2023-12-27 15:10:07,129][20347] Updated weights for policy 0, policy_version 1374124 (0.0005) [2023-12-27 15:10:07,188][20347] Updated weights for policy 0, policy_version 1374134 (0.0005) [2023-12-27 15:10:07,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 703340544. Throughput: 0: 10447.3, 1: 10299.3. Samples: 703307588. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:07,363][19170] Avg episode reward: [(0, '951.050'), (1, '1000.000')] [2023-12-27 15:10:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001372912_351494144.pth... [2023-12-27 15:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001374136_351846400.pth... [2023-12-27 15:10:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001371728_351191040.pth [2023-12-27 15:10:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001372920_351535104.pth [2023-12-27 15:10:07,595][20381] Updated weights for policy 1, policy_version 1372914 (0.0010) [2023-12-27 15:10:07,651][20381] Updated weights for policy 1, policy_version 1372924 (0.0011) [2023-12-27 15:10:07,704][20381] Updated weights for policy 1, policy_version 1372934 (0.0011) [2023-12-27 15:10:07,744][20347] Updated weights for policy 0, policy_version 1374144 (0.0009) [2023-12-27 15:10:07,754][20381] Updated weights for policy 1, policy_version 1372944 (0.0011) [2023-12-27 15:10:07,796][20347] Updated weights for policy 0, policy_version 1374154 (0.0010) [2023-12-27 15:10:07,841][20347] Updated weights for policy 0, policy_version 1374164 (0.0010) [2023-12-27 15:10:08,453][20381] Updated weights for policy 1, policy_version 1372954 (0.0011) [2023-12-27 15:10:08,511][20381] Updated weights for policy 1, policy_version 1372964 (0.0011) [2023-12-27 15:10:08,544][20347] Updated weights for policy 0, policy_version 1374174 (0.0009) [2023-12-27 15:10:08,567][20381] Updated weights for policy 1, policy_version 1372974 (0.0011) [2023-12-27 15:10:08,595][20347] Updated weights for policy 0, policy_version 1374184 (0.0007) [2023-12-27 15:10:08,649][20347] Updated weights for policy 0, policy_version 1374194 (0.0007) [2023-12-27 15:10:09,195][20381] Updated weights for policy 1, policy_version 1372984 (0.0007) [2023-12-27 15:10:09,261][20381] Updated weights for policy 1, policy_version 1372994 (0.0008) [2023-12-27 15:10:09,319][20381] Updated weights for policy 1, policy_version 1373004 (0.0008) [2023-12-27 15:10:09,357][20347] Updated weights for policy 0, policy_version 1374204 (0.0009) [2023-12-27 15:10:09,426][20347] Updated weights for policy 0, policy_version 1374214 (0.0011) [2023-12-27 15:10:09,481][20347] Updated weights for policy 0, policy_version 1374224 (0.0010) [2023-12-27 15:10:10,078][20381] Updated weights for policy 1, policy_version 1373014 (0.0008) [2023-12-27 15:10:10,139][20381] Updated weights for policy 1, policy_version 1373024 (0.0008) [2023-12-27 15:10:10,196][20381] Updated weights for policy 1, policy_version 1373034 (0.0008) [2023-12-27 15:10:10,231][20347] Updated weights for policy 0, policy_version 1374234 (0.0009) [2023-12-27 15:10:10,289][20347] Updated weights for policy 0, policy_version 1374244 (0.0007) [2023-12-27 15:10:10,348][20347] Updated weights for policy 0, policy_version 1374254 (0.0008) [2023-12-27 15:10:10,398][20347] Updated weights for policy 0, policy_version 1374264 (0.0008) [2023-12-27 15:10:10,921][20381] Updated weights for policy 1, policy_version 1373044 (0.0007) [2023-12-27 15:10:10,988][20381] Updated weights for policy 1, policy_version 1373054 (0.0008) [2023-12-27 15:10:11,046][20347] Updated weights for policy 0, policy_version 1374274 (0.0007) [2023-12-27 15:10:11,054][20381] Updated weights for policy 1, policy_version 1373064 (0.0007) [2023-12-27 15:10:11,106][20347] Updated weights for policy 0, policy_version 1374284 (0.0008) [2023-12-27 15:10:11,170][20347] Updated weights for policy 0, policy_version 1374294 (0.0008) [2023-12-27 15:10:11,821][20381] Updated weights for policy 1, policy_version 1373074 (0.0007) [2023-12-27 15:10:11,879][20381] Updated weights for policy 1, policy_version 1373084 (0.0008) [2023-12-27 15:10:11,908][20347] Updated weights for policy 0, policy_version 1374304 (0.0007) [2023-12-27 15:10:11,935][20381] Updated weights for policy 1, policy_version 1373094 (0.0008) [2023-12-27 15:10:11,965][20347] Updated weights for policy 0, policy_version 1374314 (0.0008) [2023-12-27 15:10:11,992][20381] Updated weights for policy 1, policy_version 1373104 (0.0008) [2023-12-27 15:10:12,018][20347] Updated weights for policy 0, policy_version 1374324 (0.0007) [2023-12-27 15:10:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 703438848. Throughput: 0: 10479.9, 1: 10231.3. Samples: 703429104. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:12,363][19170] Avg episode reward: [(0, '941.420'), (1, '1000.000')] [2023-12-27 15:10:12,710][20347] Updated weights for policy 0, policy_version 1374334 (0.0006) [2023-12-27 15:10:12,711][20381] Updated weights for policy 1, policy_version 1373114 (0.0007) [2023-12-27 15:10:12,765][20381] Updated weights for policy 1, policy_version 1373124 (0.0006) [2023-12-27 15:10:12,770][20347] Updated weights for policy 0, policy_version 1374344 (0.0007) [2023-12-27 15:10:12,832][20347] Updated weights for policy 0, policy_version 1374354 (0.0006) [2023-12-27 15:10:12,834][20381] Updated weights for policy 1, policy_version 1373134 (0.0006) [2023-12-27 15:10:13,494][20347] Updated weights for policy 0, policy_version 1374364 (0.0007) [2023-12-27 15:10:13,544][20381] Updated weights for policy 1, policy_version 1373144 (0.0008) [2023-12-27 15:10:13,547][20347] Updated weights for policy 0, policy_version 1374374 (0.0007) [2023-12-27 15:10:13,597][20381] Updated weights for policy 1, policy_version 1373154 (0.0008) [2023-12-27 15:10:13,613][20347] Updated weights for policy 0, policy_version 1374384 (0.0005) [2023-12-27 15:10:13,651][20381] Updated weights for policy 1, policy_version 1373164 (0.0008) [2023-12-27 15:10:14,264][20347] Updated weights for policy 0, policy_version 1374394 (0.0006) [2023-12-27 15:10:14,324][20347] Updated weights for policy 0, policy_version 1374404 (0.0007) [2023-12-27 15:10:14,361][20381] Updated weights for policy 1, policy_version 1373174 (0.0006) [2023-12-27 15:10:14,381][20347] Updated weights for policy 0, policy_version 1374414 (0.0006) [2023-12-27 15:10:14,419][20381] Updated weights for policy 1, policy_version 1373184 (0.0006) [2023-12-27 15:10:14,446][20347] Updated weights for policy 0, policy_version 1374424 (0.0007) [2023-12-27 15:10:14,472][20381] Updated weights for policy 1, policy_version 1373194 (0.0008) [2023-12-27 15:10:15,125][20347] Updated weights for policy 0, policy_version 1374434 (0.0008) [2023-12-27 15:10:15,191][20347] Updated weights for policy 0, policy_version 1374444 (0.0008) [2023-12-27 15:10:15,191][20381] Updated weights for policy 1, policy_version 1373204 (0.0007) [2023-12-27 15:10:15,246][20381] Updated weights for policy 1, policy_version 1373214 (0.0006) [2023-12-27 15:10:15,247][20347] Updated weights for policy 0, policy_version 1374454 (0.0006) [2023-12-27 15:10:15,302][20381] Updated weights for policy 1, policy_version 1373224 (0.0007) [2023-12-27 15:10:15,837][20347] Updated weights for policy 0, policy_version 1374464 (0.0006) [2023-12-27 15:10:15,897][20347] Updated weights for policy 0, policy_version 1374474 (0.0005) [2023-12-27 15:10:15,948][20347] Updated weights for policy 0, policy_version 1374484 (0.0005) [2023-12-27 15:10:15,964][20381] Updated weights for policy 1, policy_version 1373234 (0.0009) [2023-12-27 15:10:16,013][20381] Updated weights for policy 1, policy_version 1373244 (0.0008) [2023-12-27 15:10:16,058][20381] Updated weights for policy 1, policy_version 1373254 (0.0008) [2023-12-27 15:10:16,115][20381] Updated weights for policy 1, policy_version 1373264 (0.0008) [2023-12-27 15:10:16,518][20347] Updated weights for policy 0, policy_version 1374494 (0.0005) [2023-12-27 15:10:16,574][20347] Updated weights for policy 0, policy_version 1374504 (0.0005) [2023-12-27 15:10:16,622][20347] Updated weights for policy 0, policy_version 1374514 (0.0005) [2023-12-27 15:10:16,837][20381] Updated weights for policy 1, policy_version 1373274 (0.0008) [2023-12-27 15:10:16,899][20381] Updated weights for policy 1, policy_version 1373284 (0.0007) [2023-12-27 15:10:16,949][20381] Updated weights for policy 1, policy_version 1373294 (0.0007) [2023-12-27 15:10:17,175][20347] Updated weights for policy 0, policy_version 1374524 (0.0005) [2023-12-27 15:10:17,231][20347] Updated weights for policy 0, policy_version 1374534 (0.0005) [2023-12-27 15:10:17,283][20347] Updated weights for policy 0, policy_version 1374544 (0.0007) [2023-12-27 15:10:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 703545344. Throughput: 0: 10438.9, 1: 10164.4. Samples: 703551960. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:17,364][19170] Avg episode reward: [(0, '931.710'), (1, '1000.000')] [2023-12-27 15:10:17,641][20381] Updated weights for policy 1, policy_version 1373304 (0.0006) [2023-12-27 15:10:17,697][20381] Updated weights for policy 1, policy_version 1373314 (0.0006) [2023-12-27 15:10:17,753][20381] Updated weights for policy 1, policy_version 1373324 (0.0008) [2023-12-27 15:10:17,855][20347] Updated weights for policy 0, policy_version 1374554 (0.0007) [2023-12-27 15:10:17,914][20347] Updated weights for policy 0, policy_version 1374564 (0.0008) [2023-12-27 15:10:17,972][20347] Updated weights for policy 0, policy_version 1374574 (0.0007) [2023-12-27 15:10:18,032][20347] Updated weights for policy 0, policy_version 1374584 (0.0008) [2023-12-27 15:10:18,394][20381] Updated weights for policy 1, policy_version 1373334 (0.0008) [2023-12-27 15:10:18,456][20381] Updated weights for policy 1, policy_version 1373344 (0.0008) [2023-12-27 15:10:18,513][20381] Updated weights for policy 1, policy_version 1373354 (0.0006) [2023-12-27 15:10:18,715][20347] Updated weights for policy 0, policy_version 1374594 (0.0007) [2023-12-27 15:10:18,771][20347] Updated weights for policy 0, policy_version 1374604 (0.0008) [2023-12-27 15:10:18,830][20347] Updated weights for policy 0, policy_version 1374614 (0.0007) [2023-12-27 15:10:19,169][20381] Updated weights for policy 1, policy_version 1373364 (0.0006) [2023-12-27 15:10:19,221][20381] Updated weights for policy 1, policy_version 1373374 (0.0005) [2023-12-27 15:10:19,283][20381] Updated weights for policy 1, policy_version 1373384 (0.0006) [2023-12-27 15:10:19,536][20347] Updated weights for policy 0, policy_version 1374624 (0.0009) [2023-12-27 15:10:19,601][20347] Updated weights for policy 0, policy_version 1374634 (0.0006) [2023-12-27 15:10:19,660][20347] Updated weights for policy 0, policy_version 1374644 (0.0006) [2023-12-27 15:10:19,977][20381] Updated weights for policy 1, policy_version 1373394 (0.0007) [2023-12-27 15:10:20,033][20381] Updated weights for policy 1, policy_version 1373404 (0.0006) [2023-12-27 15:10:20,097][20381] Updated weights for policy 1, policy_version 1373414 (0.0006) [2023-12-27 15:10:20,161][20381] Updated weights for policy 1, policy_version 1373424 (0.0009) [2023-12-27 15:10:20,313][20347] Updated weights for policy 0, policy_version 1374654 (0.0008) [2023-12-27 15:10:20,375][20347] Updated weights for policy 0, policy_version 1374664 (0.0011) [2023-12-27 15:10:20,438][20347] Updated weights for policy 0, policy_version 1374674 (0.0010) [2023-12-27 15:10:20,945][20381] Updated weights for policy 1, policy_version 1373434 (0.0008) [2023-12-27 15:10:21,005][20381] Updated weights for policy 1, policy_version 1373444 (0.0008) [2023-12-27 15:10:21,073][20381] Updated weights for policy 1, policy_version 1373454 (0.0008) [2023-12-27 15:10:21,112][20347] Updated weights for policy 0, policy_version 1374684 (0.0010) [2023-12-27 15:10:21,177][20347] Updated weights for policy 0, policy_version 1374694 (0.0009) [2023-12-27 15:10:21,231][20347] Updated weights for policy 0, policy_version 1374704 (0.0007) [2023-12-27 15:10:21,838][20381] Updated weights for policy 1, policy_version 1373464 (0.0006) [2023-12-27 15:10:21,907][20381] Updated weights for policy 1, policy_version 1373474 (0.0006) [2023-12-27 15:10:21,956][20347] Updated weights for policy 0, policy_version 1374714 (0.0007) [2023-12-27 15:10:21,966][20381] Updated weights for policy 1, policy_version 1373484 (0.0007) [2023-12-27 15:10:22,019][20347] Updated weights for policy 0, policy_version 1374724 (0.0008) [2023-12-27 15:10:22,074][20347] Updated weights for policy 0, policy_version 1374734 (0.0008) [2023-12-27 15:10:22,136][20347] Updated weights for policy 0, policy_version 1374744 (0.0007) [2023-12-27 15:10:22,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 703643648. Throughput: 0: 10473.8, 1: 10168.0. Samples: 703614128. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:22,363][19170] Avg episode reward: [(0, '951.120'), (1, '961.070')] [2023-12-27 15:10:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001373488_351641600.pth... [2023-12-27 15:10:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001374744_352002048.pth... [2023-12-27 15:10:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001372304_351338496.pth [2023-12-27 15:10:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001373528_351690752.pth [2023-12-27 15:10:22,633][20381] Updated weights for policy 1, policy_version 1373494 (0.0009) [2023-12-27 15:10:22,688][20381] Updated weights for policy 1, policy_version 1373504 (0.0007) [2023-12-27 15:10:22,744][20381] Updated weights for policy 1, policy_version 1373514 (0.0008) [2023-12-27 15:10:22,918][20347] Updated weights for policy 0, policy_version 1374754 (0.0008) [2023-12-27 15:10:22,977][20347] Updated weights for policy 0, policy_version 1374764 (0.0008) [2023-12-27 15:10:23,032][20347] Updated weights for policy 0, policy_version 1374774 (0.0008) [2023-12-27 15:10:23,464][20381] Updated weights for policy 1, policy_version 1373524 (0.0008) [2023-12-27 15:10:23,514][20381] Updated weights for policy 1, policy_version 1373534 (0.0010) [2023-12-27 15:10:23,565][20381] Updated weights for policy 1, policy_version 1373544 (0.0010) [2023-12-27 15:10:23,627][20347] Updated weights for policy 0, policy_version 1374784 (0.0006) [2023-12-27 15:10:23,680][20347] Updated weights for policy 0, policy_version 1374794 (0.0005) [2023-12-27 15:10:23,736][20347] Updated weights for policy 0, policy_version 1374804 (0.0005) [2023-12-27 15:10:24,191][20381] Updated weights for policy 1, policy_version 1373554 (0.0008) [2023-12-27 15:10:24,251][20381] Updated weights for policy 1, policy_version 1373564 (0.0005) [2023-12-27 15:10:24,302][20381] Updated weights for policy 1, policy_version 1373574 (0.0005) [2023-12-27 15:10:24,350][20347] Updated weights for policy 0, policy_version 1374814 (0.0005) [2023-12-27 15:10:24,352][20381] Updated weights for policy 1, policy_version 1373584 (0.0005) [2023-12-27 15:10:24,404][20347] Updated weights for policy 0, policy_version 1374824 (0.0009) [2023-12-27 15:10:24,459][20347] Updated weights for policy 0, policy_version 1374834 (0.0008) [2023-12-27 15:10:24,903][20381] Updated weights for policy 1, policy_version 1373594 (0.0007) [2023-12-27 15:10:24,953][20381] Updated weights for policy 1, policy_version 1373604 (0.0007) [2023-12-27 15:10:25,009][20381] Updated weights for policy 1, policy_version 1373614 (0.0006) [2023-12-27 15:10:25,056][20347] Updated weights for policy 0, policy_version 1374844 (0.0009) [2023-12-27 15:10:25,112][20347] Updated weights for policy 0, policy_version 1374854 (0.0008) [2023-12-27 15:10:25,172][20347] Updated weights for policy 0, policy_version 1374864 (0.0006) [2023-12-27 15:10:25,660][20381] Updated weights for policy 1, policy_version 1373624 (0.0005) [2023-12-27 15:10:25,706][20381] Updated weights for policy 1, policy_version 1373634 (0.0005) [2023-12-27 15:10:25,761][20381] Updated weights for policy 1, policy_version 1373644 (0.0006) [2023-12-27 15:10:25,805][20347] Updated weights for policy 0, policy_version 1374874 (0.0006) [2023-12-27 15:10:25,869][20347] Updated weights for policy 0, policy_version 1374884 (0.0007) [2023-12-27 15:10:25,924][20347] Updated weights for policy 0, policy_version 1374894 (0.0008) [2023-12-27 15:10:25,972][20347] Updated weights for policy 0, policy_version 1374904 (0.0008) [2023-12-27 15:10:26,419][20381] Updated weights for policy 1, policy_version 1373654 (0.0006) [2023-12-27 15:10:26,478][20381] Updated weights for policy 1, policy_version 1373664 (0.0006) [2023-12-27 15:10:26,528][20347] Updated weights for policy 0, policy_version 1374914 (0.0007) [2023-12-27 15:10:26,534][20381] Updated weights for policy 1, policy_version 1373674 (0.0007) [2023-12-27 15:10:26,580][20347] Updated weights for policy 0, policy_version 1374924 (0.0006) [2023-12-27 15:10:26,631][20347] Updated weights for policy 0, policy_version 1374934 (0.0008) [2023-12-27 15:10:27,132][20381] Updated weights for policy 1, policy_version 1373684 (0.0005) [2023-12-27 15:10:27,180][20381] Updated weights for policy 1, policy_version 1373694 (0.0005) [2023-12-27 15:10:27,230][20381] Updated weights for policy 1, policy_version 1373704 (0.0007) [2023-12-27 15:10:27,355][20347] Updated weights for policy 0, policy_version 1374944 (0.0008) [2023-12-27 15:10:27,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 703750144. Throughput: 0: 10401.4, 1: 10191.9. Samples: 703737512. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:27,363][19170] Avg episode reward: [(0, '980.430'), (1, '961.070')] [2023-12-27 15:10:27,397][20347] Updated weights for policy 0, policy_version 1374954 (0.0007) [2023-12-27 15:10:27,457][20347] Updated weights for policy 0, policy_version 1374964 (0.0008) [2023-12-27 15:10:27,878][20381] Updated weights for policy 1, policy_version 1373714 (0.0008) [2023-12-27 15:10:27,928][20381] Updated weights for policy 1, policy_version 1373724 (0.0007) [2023-12-27 15:10:27,982][20381] Updated weights for policy 1, policy_version 1373734 (0.0008) [2023-12-27 15:10:28,038][20381] Updated weights for policy 1, policy_version 1373744 (0.0008) [2023-12-27 15:10:28,149][20347] Updated weights for policy 0, policy_version 1374974 (0.0007) [2023-12-27 15:10:28,199][20347] Updated weights for policy 0, policy_version 1374984 (0.0008) [2023-12-27 15:10:28,255][20347] Updated weights for policy 0, policy_version 1374994 (0.0007) [2023-12-27 15:10:28,677][20381] Updated weights for policy 1, policy_version 1373754 (0.0010) [2023-12-27 15:10:28,730][20381] Updated weights for policy 1, policy_version 1373764 (0.0010) [2023-12-27 15:10:28,795][20381] Updated weights for policy 1, policy_version 1373774 (0.0011) [2023-12-27 15:10:28,969][20347] Updated weights for policy 0, policy_version 1375004 (0.0006) [2023-12-27 15:10:29,034][20347] Updated weights for policy 0, policy_version 1375014 (0.0007) [2023-12-27 15:10:29,088][20347] Updated weights for policy 0, policy_version 1375024 (0.0008) [2023-12-27 15:10:29,490][20381] Updated weights for policy 1, policy_version 1373784 (0.0006) [2023-12-27 15:10:29,533][20381] Updated weights for policy 1, policy_version 1373794 (0.0005) [2023-12-27 15:10:29,579][20381] Updated weights for policy 1, policy_version 1373804 (0.0007) [2023-12-27 15:10:29,745][20347] Updated weights for policy 0, policy_version 1375034 (0.0007) [2023-12-27 15:10:29,795][20347] Updated weights for policy 0, policy_version 1375044 (0.0006) [2023-12-27 15:10:29,857][20347] Updated weights for policy 0, policy_version 1375054 (0.0009) [2023-12-27 15:10:29,910][20347] Updated weights for policy 0, policy_version 1375064 (0.0010) [2023-12-27 15:10:30,272][20381] Updated weights for policy 1, policy_version 1373814 (0.0008) [2023-12-27 15:10:30,325][20381] Updated weights for policy 1, policy_version 1373824 (0.0007) [2023-12-27 15:10:30,383][20381] Updated weights for policy 1, policy_version 1373834 (0.0007) [2023-12-27 15:10:30,596][20347] Updated weights for policy 0, policy_version 1375074 (0.0010) [2023-12-27 15:10:30,657][20347] Updated weights for policy 0, policy_version 1375084 (0.0008) [2023-12-27 15:10:30,719][20347] Updated weights for policy 0, policy_version 1375094 (0.0010) [2023-12-27 15:10:30,995][20381] Updated weights for policy 1, policy_version 1373844 (0.0006) [2023-12-27 15:10:31,054][20381] Updated weights for policy 1, policy_version 1373854 (0.0009) [2023-12-27 15:10:31,114][20381] Updated weights for policy 1, policy_version 1373864 (0.0010) [2023-12-27 15:10:31,371][20347] Updated weights for policy 0, policy_version 1375104 (0.0008) [2023-12-27 15:10:31,432][20347] Updated weights for policy 0, policy_version 1375114 (0.0008) [2023-12-27 15:10:31,499][20347] Updated weights for policy 0, policy_version 1375124 (0.0005) [2023-12-27 15:10:31,788][20381] Updated weights for policy 1, policy_version 1373874 (0.0009) [2023-12-27 15:10:31,839][20381] Updated weights for policy 1, policy_version 1373884 (0.0008) [2023-12-27 15:10:31,894][20381] Updated weights for policy 1, policy_version 1373894 (0.0008) [2023-12-27 15:10:31,945][20381] Updated weights for policy 1, policy_version 1373904 (0.0008) [2023-12-27 15:10:32,133][20347] Updated weights for policy 0, policy_version 1375134 (0.0008) [2023-12-27 15:10:32,204][20347] Updated weights for policy 0, policy_version 1375144 (0.0009) [2023-12-27 15:10:32,268][20347] Updated weights for policy 0, policy_version 1375154 (0.0008) [2023-12-27 15:10:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 703856640. Throughput: 0: 10367.9, 1: 10199.6. Samples: 703863620. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:32,363][19170] Avg episode reward: [(0, '980.440'), (1, '990.280')] [2023-12-27 15:10:32,674][20381] Updated weights for policy 1, policy_version 1373914 (0.0008) [2023-12-27 15:10:32,733][20381] Updated weights for policy 1, policy_version 1373924 (0.0008) [2023-12-27 15:10:32,734][20292] KL-divergence is very high: 103.4321 [2023-12-27 15:10:32,794][20381] Updated weights for policy 1, policy_version 1373934 (0.0007) [2023-12-27 15:10:32,908][20347] Updated weights for policy 0, policy_version 1375164 (0.0007) [2023-12-27 15:10:32,961][20347] Updated weights for policy 0, policy_version 1375174 (0.0006) [2023-12-27 15:10:33,006][20347] Updated weights for policy 0, policy_version 1375184 (0.0007) [2023-12-27 15:10:33,455][20381] Updated weights for policy 1, policy_version 1373944 (0.0007) [2023-12-27 15:10:33,514][20381] Updated weights for policy 1, policy_version 1373954 (0.0006) [2023-12-27 15:10:33,558][20381] Updated weights for policy 1, policy_version 1373964 (0.0008) [2023-12-27 15:10:33,659][20347] Updated weights for policy 0, policy_version 1375194 (0.0007) [2023-12-27 15:10:33,703][20347] Updated weights for policy 0, policy_version 1375204 (0.0007) [2023-12-27 15:10:33,751][20347] Updated weights for policy 0, policy_version 1375214 (0.0007) [2023-12-27 15:10:33,795][20347] Updated weights for policy 0, policy_version 1375224 (0.0007) [2023-12-27 15:10:34,249][20381] Updated weights for policy 1, policy_version 1373974 (0.0008) [2023-12-27 15:10:34,313][20381] Updated weights for policy 1, policy_version 1373984 (0.0008) [2023-12-27 15:10:34,375][20381] Updated weights for policy 1, policy_version 1373994 (0.0007) [2023-12-27 15:10:34,490][20347] Updated weights for policy 0, policy_version 1375234 (0.0006) [2023-12-27 15:10:34,540][20347] Updated weights for policy 0, policy_version 1375244 (0.0007) [2023-12-27 15:10:34,598][20347] Updated weights for policy 0, policy_version 1375254 (0.0008) [2023-12-27 15:10:35,043][20381] Updated weights for policy 1, policy_version 1374004 (0.0008) [2023-12-27 15:10:35,098][20381] Updated weights for policy 1, policy_version 1374014 (0.0008) [2023-12-27 15:10:35,149][20381] Updated weights for policy 1, policy_version 1374024 (0.0008) [2023-12-27 15:10:35,284][20347] Updated weights for policy 0, policy_version 1375264 (0.0006) [2023-12-27 15:10:35,332][20347] Updated weights for policy 0, policy_version 1375274 (0.0005) [2023-12-27 15:10:35,375][20347] Updated weights for policy 0, policy_version 1375284 (0.0005) [2023-12-27 15:10:35,781][20381] Updated weights for policy 1, policy_version 1374034 (0.0007) [2023-12-27 15:10:35,836][20381] Updated weights for policy 1, policy_version 1374044 (0.0005) [2023-12-27 15:10:35,887][20381] Updated weights for policy 1, policy_version 1374054 (0.0007) [2023-12-27 15:10:35,937][20381] Updated weights for policy 1, policy_version 1374064 (0.0007) [2023-12-27 15:10:35,970][20347] Updated weights for policy 0, policy_version 1375294 (0.0008) [2023-12-27 15:10:36,028][20347] Updated weights for policy 0, policy_version 1375304 (0.0010) [2023-12-27 15:10:36,079][20347] Updated weights for policy 0, policy_version 1375314 (0.0010) [2023-12-27 15:10:36,666][20381] Updated weights for policy 1, policy_version 1374074 (0.0007) [2023-12-27 15:10:36,714][20381] Updated weights for policy 1, policy_version 1374084 (0.0007) [2023-12-27 15:10:36,765][20381] Updated weights for policy 1, policy_version 1374094 (0.0008) [2023-12-27 15:10:36,826][20347] Updated weights for policy 0, policy_version 1375324 (0.0010) [2023-12-27 15:10:36,874][20347] Updated weights for policy 0, policy_version 1375334 (0.0010) [2023-12-27 15:10:36,926][20347] Updated weights for policy 0, policy_version 1375344 (0.0010) [2023-12-27 15:10:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 703954944. Throughput: 0: 10346.7, 1: 10179.5. Samples: 703924616. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:37,363][19170] Avg episode reward: [(0, '980.400'), (1, '970.770')] [2023-12-27 15:10:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001374096_351797248.pth... [2023-12-27 15:10:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001375352_352157696.pth... [2023-12-27 15:10:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001372912_351494144.pth [2023-12-27 15:10:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001374136_351846400.pth [2023-12-27 15:10:37,443][20381] Updated weights for policy 1, policy_version 1374104 (0.0006) [2023-12-27 15:10:37,499][20381] Updated weights for policy 1, policy_version 1374114 (0.0007) [2023-12-27 15:10:37,548][20381] Updated weights for policy 1, policy_version 1374124 (0.0008) [2023-12-27 15:10:37,643][20347] Updated weights for policy 0, policy_version 1375354 (0.0009) [2023-12-27 15:10:37,700][20347] Updated weights for policy 0, policy_version 1375364 (0.0007) [2023-12-27 15:10:37,756][20347] Updated weights for policy 0, policy_version 1375374 (0.0007) [2023-12-27 15:10:37,814][20347] Updated weights for policy 0, policy_version 1375384 (0.0007) [2023-12-27 15:10:38,234][20381] Updated weights for policy 1, policy_version 1374134 (0.0008) [2023-12-27 15:10:38,278][20381] Updated weights for policy 1, policy_version 1374144 (0.0007) [2023-12-27 15:10:38,329][20381] Updated weights for policy 1, policy_version 1374154 (0.0007) [2023-12-27 15:10:38,497][20347] Updated weights for policy 0, policy_version 1375394 (0.0008) [2023-12-27 15:10:38,554][20347] Updated weights for policy 0, policy_version 1375404 (0.0008) [2023-12-27 15:10:38,608][20347] Updated weights for policy 0, policy_version 1375414 (0.0008) [2023-12-27 15:10:39,064][20381] Updated weights for policy 1, policy_version 1374164 (0.0007) [2023-12-27 15:10:39,123][20381] Updated weights for policy 1, policy_version 1374174 (0.0008) [2023-12-27 15:10:39,185][20381] Updated weights for policy 1, policy_version 1374184 (0.0008) [2023-12-27 15:10:39,346][20347] Updated weights for policy 0, policy_version 1375424 (0.0008) [2023-12-27 15:10:39,416][20347] Updated weights for policy 0, policy_version 1375434 (0.0008) [2023-12-27 15:10:39,470][20347] Updated weights for policy 0, policy_version 1375444 (0.0008) [2023-12-27 15:10:39,956][20381] Updated weights for policy 1, policy_version 1374194 (0.0008) [2023-12-27 15:10:40,017][20381] Updated weights for policy 1, policy_version 1374204 (0.0008) [2023-12-27 15:10:40,073][20381] Updated weights for policy 1, policy_version 1374214 (0.0008) [2023-12-27 15:10:40,135][20381] Updated weights for policy 1, policy_version 1374224 (0.0008) [2023-12-27 15:10:40,185][20347] Updated weights for policy 0, policy_version 1375454 (0.0008) [2023-12-27 15:10:40,245][20347] Updated weights for policy 0, policy_version 1375464 (0.0007) [2023-12-27 15:10:40,300][20347] Updated weights for policy 0, policy_version 1375474 (0.0007) [2023-12-27 15:10:40,835][20381] Updated weights for policy 1, policy_version 1374234 (0.0009) [2023-12-27 15:10:40,885][20381] Updated weights for policy 1, policy_version 1374244 (0.0009) [2023-12-27 15:10:40,935][20381] Updated weights for policy 1, policy_version 1374254 (0.0009) [2023-12-27 15:10:40,969][20347] Updated weights for policy 0, policy_version 1375484 (0.0008) [2023-12-27 15:10:41,033][20347] Updated weights for policy 0, policy_version 1375494 (0.0009) [2023-12-27 15:10:41,094][20347] Updated weights for policy 0, policy_version 1375504 (0.0009) [2023-12-27 15:10:41,675][20381] Updated weights for policy 1, policy_version 1374264 (0.0007) [2023-12-27 15:10:41,738][20381] Updated weights for policy 1, policy_version 1374274 (0.0008) [2023-12-27 15:10:41,794][20381] Updated weights for policy 1, policy_version 1374284 (0.0006) [2023-12-27 15:10:41,816][20347] Updated weights for policy 0, policy_version 1375514 (0.0009) [2023-12-27 15:10:41,875][20347] Updated weights for policy 0, policy_version 1375524 (0.0008) [2023-12-27 15:10:41,931][20347] Updated weights for policy 0, policy_version 1375534 (0.0007) [2023-12-27 15:10:41,994][20347] Updated weights for policy 0, policy_version 1375544 (0.0008) [2023-12-27 15:10:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 704053248. Throughput: 0: 10378.4, 1: 10131.1. Samples: 704045772. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:42,363][19170] Avg episode reward: [(0, '980.440'), (1, '970.720')] [2023-12-27 15:10:42,413][20381] Updated weights for policy 1, policy_version 1374294 (0.0008) [2023-12-27 15:10:42,470][20381] Updated weights for policy 1, policy_version 1374304 (0.0007) [2023-12-27 15:10:42,532][20381] Updated weights for policy 1, policy_version 1374314 (0.0008) [2023-12-27 15:10:42,726][20347] Updated weights for policy 0, policy_version 1375554 (0.0007) [2023-12-27 15:10:42,780][20347] Updated weights for policy 0, policy_version 1375564 (0.0007) [2023-12-27 15:10:42,830][20347] Updated weights for policy 0, policy_version 1375574 (0.0008) [2023-12-27 15:10:43,250][20381] Updated weights for policy 1, policy_version 1374324 (0.0008) [2023-12-27 15:10:43,295][20381] Updated weights for policy 1, policy_version 1374334 (0.0007) [2023-12-27 15:10:43,344][20381] Updated weights for policy 1, policy_version 1374344 (0.0007) [2023-12-27 15:10:43,537][20347] Updated weights for policy 0, policy_version 1375584 (0.0007) [2023-12-27 15:10:43,593][20347] Updated weights for policy 0, policy_version 1375594 (0.0007) [2023-12-27 15:10:43,648][20347] Updated weights for policy 0, policy_version 1375604 (0.0007) [2023-12-27 15:10:44,081][20381] Updated weights for policy 1, policy_version 1374354 (0.0008) [2023-12-27 15:10:44,129][20381] Updated weights for policy 1, policy_version 1374364 (0.0006) [2023-12-27 15:10:44,184][20381] Updated weights for policy 1, policy_version 1374374 (0.0007) [2023-12-27 15:10:44,236][20381] Updated weights for policy 1, policy_version 1374384 (0.0007) [2023-12-27 15:10:44,275][20347] Updated weights for policy 0, policy_version 1375614 (0.0007) [2023-12-27 15:10:44,327][20347] Updated weights for policy 0, policy_version 1375624 (0.0007) [2023-12-27 15:10:44,373][20347] Updated weights for policy 0, policy_version 1375634 (0.0007) [2023-12-27 15:10:44,906][20381] Updated weights for policy 1, policy_version 1374394 (0.0008) [2023-12-27 15:10:44,967][20381] Updated weights for policy 1, policy_version 1374404 (0.0008) [2023-12-27 15:10:45,025][20381] Updated weights for policy 1, policy_version 1374414 (0.0008) [2023-12-27 15:10:45,106][20347] Updated weights for policy 0, policy_version 1375644 (0.0008) [2023-12-27 15:10:45,167][20347] Updated weights for policy 0, policy_version 1375654 (0.0008) [2023-12-27 15:10:45,231][20347] Updated weights for policy 0, policy_version 1375664 (0.0008) [2023-12-27 15:10:45,727][20381] Updated weights for policy 1, policy_version 1374424 (0.0008) [2023-12-27 15:10:45,783][20381] Updated weights for policy 1, policy_version 1374434 (0.0008) [2023-12-27 15:10:45,837][20381] Updated weights for policy 1, policy_version 1374444 (0.0007) [2023-12-27 15:10:45,902][20347] Updated weights for policy 0, policy_version 1375674 (0.0008) [2023-12-27 15:10:45,950][20347] Updated weights for policy 0, policy_version 1375684 (0.0005) [2023-12-27 15:10:45,997][20347] Updated weights for policy 0, policy_version 1375694 (0.0005) [2023-12-27 15:10:46,050][20347] Updated weights for policy 0, policy_version 1375704 (0.0005) [2023-12-27 15:10:46,565][20381] Updated weights for policy 1, policy_version 1374454 (0.0008) [2023-12-27 15:10:46,621][20381] Updated weights for policy 1, policy_version 1374464 (0.0007) [2023-12-27 15:10:46,676][20381] Updated weights for policy 1, policy_version 1374474 (0.0006) [2023-12-27 15:10:46,706][20347] Updated weights for policy 0, policy_version 1375714 (0.0008) [2023-12-27 15:10:46,761][20347] Updated weights for policy 0, policy_version 1375724 (0.0005) [2023-12-27 15:10:46,815][20347] Updated weights for policy 0, policy_version 1375734 (0.0006) [2023-12-27 15:10:47,323][20381] Updated weights for policy 1, policy_version 1374484 (0.0006) [2023-12-27 15:10:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 704151552. Throughput: 0: 10367.0, 1: 10094.8. Samples: 704165896. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:47,363][19170] Avg episode reward: [(0, '941.350'), (1, '970.680')] [2023-12-27 15:10:47,370][20381] Updated weights for policy 1, policy_version 1374494 (0.0005) [2023-12-27 15:10:47,427][20381] Updated weights for policy 1, policy_version 1374504 (0.0005) [2023-12-27 15:10:47,484][20347] Updated weights for policy 0, policy_version 1375744 (0.0009) [2023-12-27 15:10:47,545][20347] Updated weights for policy 0, policy_version 1375754 (0.0008) [2023-12-27 15:10:47,608][20347] Updated weights for policy 0, policy_version 1375764 (0.0008) [2023-12-27 15:10:48,090][20381] Updated weights for policy 1, policy_version 1374514 (0.0009) [2023-12-27 15:10:48,143][20381] Updated weights for policy 1, policy_version 1374524 (0.0008) [2023-12-27 15:10:48,202][20381] Updated weights for policy 1, policy_version 1374534 (0.0009) [2023-12-27 15:10:48,246][20347] Updated weights for policy 0, policy_version 1375774 (0.0008) [2023-12-27 15:10:48,259][20381] Updated weights for policy 1, policy_version 1374544 (0.0005) [2023-12-27 15:10:48,311][20347] Updated weights for policy 0, policy_version 1375784 (0.0008) [2023-12-27 15:10:48,380][20347] Updated weights for policy 0, policy_version 1375794 (0.0007) [2023-12-27 15:10:48,908][20381] Updated weights for policy 1, policy_version 1374554 (0.0007) [2023-12-27 15:10:48,965][20381] Updated weights for policy 1, policy_version 1374564 (0.0008) [2023-12-27 15:10:49,023][20381] Updated weights for policy 1, policy_version 1374574 (0.0006) [2023-12-27 15:10:49,047][20347] Updated weights for policy 0, policy_version 1375804 (0.0007) [2023-12-27 15:10:49,096][20347] Updated weights for policy 0, policy_version 1375814 (0.0005) [2023-12-27 15:10:49,157][20347] Updated weights for policy 0, policy_version 1375824 (0.0005) [2023-12-27 15:10:49,725][20381] Updated weights for policy 1, policy_version 1374584 (0.0008) [2023-12-27 15:10:49,791][20381] Updated weights for policy 1, policy_version 1374594 (0.0008) [2023-12-27 15:10:49,838][20347] Updated weights for policy 0, policy_version 1375834 (0.0006) [2023-12-27 15:10:49,852][20381] Updated weights for policy 1, policy_version 1374604 (0.0008) [2023-12-27 15:10:49,896][20347] Updated weights for policy 0, policy_version 1375844 (0.0011) [2023-12-27 15:10:49,960][20347] Updated weights for policy 0, policy_version 1375854 (0.0010) [2023-12-27 15:10:50,016][20347] Updated weights for policy 0, policy_version 1375864 (0.0010) [2023-12-27 15:10:50,554][20381] Updated weights for policy 1, policy_version 1374614 (0.0005) [2023-12-27 15:10:50,612][20381] Updated weights for policy 1, policy_version 1374624 (0.0007) [2023-12-27 15:10:50,672][20381] Updated weights for policy 1, policy_version 1374634 (0.0008) [2023-12-27 15:10:50,727][20347] Updated weights for policy 0, policy_version 1375874 (0.0011) [2023-12-27 15:10:50,788][20347] Updated weights for policy 0, policy_version 1375884 (0.0012) [2023-12-27 15:10:50,847][20347] Updated weights for policy 0, policy_version 1375894 (0.0010) [2023-12-27 15:10:51,378][20381] Updated weights for policy 1, policy_version 1374644 (0.0007) [2023-12-27 15:10:51,441][20381] Updated weights for policy 1, policy_version 1374654 (0.0008) [2023-12-27 15:10:51,496][20381] Updated weights for policy 1, policy_version 1374664 (0.0007) [2023-12-27 15:10:51,556][20347] Updated weights for policy 0, policy_version 1375904 (0.0008) [2023-12-27 15:10:51,614][20347] Updated weights for policy 0, policy_version 1375914 (0.0008) [2023-12-27 15:10:51,681][20347] Updated weights for policy 0, policy_version 1375924 (0.0008) [2023-12-27 15:10:52,164][20381] Updated weights for policy 1, policy_version 1374674 (0.0006) [2023-12-27 15:10:52,220][20381] Updated weights for policy 1, policy_version 1374684 (0.0007) [2023-12-27 15:10:52,289][20381] Updated weights for policy 1, policy_version 1374694 (0.0007) [2023-12-27 15:10:52,345][20381] Updated weights for policy 1, policy_version 1374704 (0.0007) [2023-12-27 15:10:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 704258048. Throughput: 0: 10328.5, 1: 10092.0. Samples: 704226508. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:52,363][19170] Avg episode reward: [(0, '951.290'), (1, '970.680')] [2023-12-27 15:10:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001374704_351952896.pth... [2023-12-27 15:10:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001375928_352305152.pth... [2023-12-27 15:10:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001373488_351641600.pth [2023-12-27 15:10:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001374744_352002048.pth [2023-12-27 15:10:52,436][20347] Updated weights for policy 0, policy_version 1375934 (0.0007) [2023-12-27 15:10:52,486][20347] Updated weights for policy 0, policy_version 1375944 (0.0008) [2023-12-27 15:10:52,531][20347] Updated weights for policy 0, policy_version 1375954 (0.0008) [2023-12-27 15:10:52,988][20381] Updated weights for policy 1, policy_version 1374714 (0.0009) [2023-12-27 15:10:53,045][20381] Updated weights for policy 1, policy_version 1374724 (0.0007) [2023-12-27 15:10:53,095][20381] Updated weights for policy 1, policy_version 1374734 (0.0007) [2023-12-27 15:10:53,265][20347] Updated weights for policy 0, policy_version 1375964 (0.0008) [2023-12-27 15:10:53,314][20347] Updated weights for policy 0, policy_version 1375974 (0.0010) [2023-12-27 15:10:53,360][20347] Updated weights for policy 0, policy_version 1375984 (0.0010) [2023-12-27 15:10:53,734][20381] Updated weights for policy 1, policy_version 1374744 (0.0008) [2023-12-27 15:10:53,789][20381] Updated weights for policy 1, policy_version 1374754 (0.0008) [2023-12-27 15:10:53,846][20381] Updated weights for policy 1, policy_version 1374764 (0.0007) [2023-12-27 15:10:54,061][20347] Updated weights for policy 0, policy_version 1375994 (0.0010) [2023-12-27 15:10:54,116][20347] Updated weights for policy 0, policy_version 1376004 (0.0010) [2023-12-27 15:10:54,174][20347] Updated weights for policy 0, policy_version 1376014 (0.0010) [2023-12-27 15:10:54,234][20347] Updated weights for policy 0, policy_version 1376024 (0.0010) [2023-12-27 15:10:54,486][20381] Updated weights for policy 1, policy_version 1374774 (0.0008) [2023-12-27 15:10:54,538][20381] Updated weights for policy 1, policy_version 1374784 (0.0007) [2023-12-27 15:10:54,590][20381] Updated weights for policy 1, policy_version 1374794 (0.0007) [2023-12-27 15:10:54,890][20347] Updated weights for policy 0, policy_version 1376034 (0.0008) [2023-12-27 15:10:54,937][20347] Updated weights for policy 0, policy_version 1376044 (0.0007) [2023-12-27 15:10:54,995][20347] Updated weights for policy 0, policy_version 1376054 (0.0007) [2023-12-27 15:10:55,307][20381] Updated weights for policy 1, policy_version 1374804 (0.0008) [2023-12-27 15:10:55,368][20381] Updated weights for policy 1, policy_version 1374814 (0.0007) [2023-12-27 15:10:55,418][20381] Updated weights for policy 1, policy_version 1374824 (0.0007) [2023-12-27 15:10:55,669][20347] Updated weights for policy 0, policy_version 1376064 (0.0007) [2023-12-27 15:10:55,721][20347] Updated weights for policy 0, policy_version 1376074 (0.0007) [2023-12-27 15:10:55,774][20347] Updated weights for policy 0, policy_version 1376084 (0.0005) [2023-12-27 15:10:56,125][20381] Updated weights for policy 1, policy_version 1374834 (0.0008) [2023-12-27 15:10:56,183][20381] Updated weights for policy 1, policy_version 1374844 (0.0008) [2023-12-27 15:10:56,237][20381] Updated weights for policy 1, policy_version 1374854 (0.0008) [2023-12-27 15:10:56,289][20381] Updated weights for policy 1, policy_version 1374864 (0.0007) [2023-12-27 15:10:56,442][20347] Updated weights for policy 0, policy_version 1376094 (0.0007) [2023-12-27 15:10:56,487][20347] Updated weights for policy 0, policy_version 1376104 (0.0007) [2023-12-27 15:10:56,532][20347] Updated weights for policy 0, policy_version 1376114 (0.0007) [2023-12-27 15:10:56,971][20381] Updated weights for policy 1, policy_version 1374874 (0.0007) [2023-12-27 15:10:57,026][20381] Updated weights for policy 1, policy_version 1374884 (0.0008) [2023-12-27 15:10:57,077][20381] Updated weights for policy 1, policy_version 1374894 (0.0007) [2023-12-27 15:10:57,207][20347] Updated weights for policy 0, policy_version 1376124 (0.0008) [2023-12-27 15:10:57,265][20347] Updated weights for policy 0, policy_version 1376134 (0.0008) [2023-12-27 15:10:57,319][20347] Updated weights for policy 0, policy_version 1376144 (0.0007) [2023-12-27 15:10:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 704364544. Throughput: 0: 10272.7, 1: 10146.3. Samples: 704347960. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:10:57,364][19170] Avg episode reward: [(0, '980.720'), (1, '990.230')] [2023-12-27 15:10:57,768][20381] Updated weights for policy 1, policy_version 1374904 (0.0006) [2023-12-27 15:10:57,826][20381] Updated weights for policy 1, policy_version 1374914 (0.0008) [2023-12-27 15:10:57,887][20381] Updated weights for policy 1, policy_version 1374924 (0.0010) [2023-12-27 15:10:57,966][20347] Updated weights for policy 0, policy_version 1376154 (0.0007) [2023-12-27 15:10:58,018][20347] Updated weights for policy 0, policy_version 1376164 (0.0008) [2023-12-27 15:10:58,062][20347] Updated weights for policy 0, policy_version 1376174 (0.0006) [2023-12-27 15:10:58,117][20347] Updated weights for policy 0, policy_version 1376184 (0.0007) [2023-12-27 15:10:58,604][20381] Updated weights for policy 1, policy_version 1374934 (0.0009) [2023-12-27 15:10:58,669][20381] Updated weights for policy 1, policy_version 1374944 (0.0008) [2023-12-27 15:10:58,732][20381] Updated weights for policy 1, policy_version 1374954 (0.0008) [2023-12-27 15:10:58,983][20347] Updated weights for policy 0, policy_version 1376194 (0.0008) [2023-12-27 15:10:59,051][20347] Updated weights for policy 0, policy_version 1376204 (0.0006) [2023-12-27 15:10:59,110][20347] Updated weights for policy 0, policy_version 1376214 (0.0008) [2023-12-27 15:10:59,464][20381] Updated weights for policy 1, policy_version 1374964 (0.0007) [2023-12-27 15:10:59,526][20381] Updated weights for policy 1, policy_version 1374974 (0.0008) [2023-12-27 15:10:59,592][20381] Updated weights for policy 1, policy_version 1374984 (0.0008) [2023-12-27 15:10:59,857][20347] Updated weights for policy 0, policy_version 1376224 (0.0009) [2023-12-27 15:10:59,913][20347] Updated weights for policy 0, policy_version 1376234 (0.0008) [2023-12-27 15:10:59,980][20347] Updated weights for policy 0, policy_version 1376244 (0.0008) [2023-12-27 15:11:00,297][20381] Updated weights for policy 1, policy_version 1374994 (0.0007) [2023-12-27 15:11:00,347][20381] Updated weights for policy 1, policy_version 1375004 (0.0008) [2023-12-27 15:11:00,404][20381] Updated weights for policy 1, policy_version 1375014 (0.0008) [2023-12-27 15:11:00,459][20381] Updated weights for policy 1, policy_version 1375024 (0.0007) [2023-12-27 15:11:00,662][20347] Updated weights for policy 0, policy_version 1376254 (0.0007) [2023-12-27 15:11:00,725][20347] Updated weights for policy 0, policy_version 1376264 (0.0007) [2023-12-27 15:11:00,785][20347] Updated weights for policy 0, policy_version 1376274 (0.0008) [2023-12-27 15:11:01,104][20381] Updated weights for policy 1, policy_version 1375034 (0.0007) [2023-12-27 15:11:01,166][20381] Updated weights for policy 1, policy_version 1375044 (0.0011) [2023-12-27 15:11:01,216][20381] Updated weights for policy 1, policy_version 1375054 (0.0010) [2023-12-27 15:11:01,445][20347] Updated weights for policy 0, policy_version 1376284 (0.0007) [2023-12-27 15:11:01,497][20347] Updated weights for policy 0, policy_version 1376294 (0.0008) [2023-12-27 15:11:01,556][20347] Updated weights for policy 0, policy_version 1376304 (0.0008) [2023-12-27 15:11:01,953][20381] Updated weights for policy 1, policy_version 1375064 (0.0008) [2023-12-27 15:11:02,000][20381] Updated weights for policy 1, policy_version 1375074 (0.0007) [2023-12-27 15:11:02,056][20381] Updated weights for policy 1, policy_version 1375084 (0.0008) [2023-12-27 15:11:02,310][20347] Updated weights for policy 0, policy_version 1376314 (0.0008) [2023-12-27 15:11:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 704454656. Throughput: 0: 10153.2, 1: 10153.8. Samples: 704465768. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:02,363][19170] Avg episode reward: [(0, '960.940'), (1, '990.330')] [2023-12-27 15:11:02,364][20347] Updated weights for policy 0, policy_version 1376324 (0.0010) [2023-12-27 15:11:02,424][20347] Updated weights for policy 0, policy_version 1376334 (0.0008) [2023-12-27 15:11:02,468][20347] Updated weights for policy 0, policy_version 1376344 (0.0006) [2023-12-27 15:11:02,784][20381] Updated weights for policy 1, policy_version 1375094 (0.0009) [2023-12-27 15:11:02,836][20381] Updated weights for policy 1, policy_version 1375104 (0.0009) [2023-12-27 15:11:02,894][20381] Updated weights for policy 1, policy_version 1375114 (0.0005) [2023-12-27 15:11:03,218][20347] Updated weights for policy 0, policy_version 1376354 (0.0007) [2023-12-27 15:11:03,265][20347] Updated weights for policy 0, policy_version 1376364 (0.0007) [2023-12-27 15:11:03,313][20347] Updated weights for policy 0, policy_version 1376374 (0.0007) [2023-12-27 15:11:03,514][20381] Updated weights for policy 1, policy_version 1375124 (0.0007) [2023-12-27 15:11:03,571][20381] Updated weights for policy 1, policy_version 1375134 (0.0010) [2023-12-27 15:11:03,632][20381] Updated weights for policy 1, policy_version 1375144 (0.0010) [2023-12-27 15:11:04,008][20347] Updated weights for policy 0, policy_version 1376384 (0.0007) [2023-12-27 15:11:04,054][20347] Updated weights for policy 0, policy_version 1376394 (0.0007) [2023-12-27 15:11:04,129][20347] Updated weights for policy 0, policy_version 1376404 (0.0008) [2023-12-27 15:11:04,325][20381] Updated weights for policy 1, policy_version 1375154 (0.0010) [2023-12-27 15:11:04,385][20381] Updated weights for policy 1, policy_version 1375164 (0.0008) [2023-12-27 15:11:04,447][20381] Updated weights for policy 1, policy_version 1375174 (0.0008) [2023-12-27 15:11:04,504][20381] Updated weights for policy 1, policy_version 1375184 (0.0008) [2023-12-27 15:11:04,833][20347] Updated weights for policy 0, policy_version 1376414 (0.0008) [2023-12-27 15:11:04,886][20347] Updated weights for policy 0, policy_version 1376424 (0.0008) [2023-12-27 15:11:04,932][20347] Updated weights for policy 0, policy_version 1376434 (0.0008) [2023-12-27 15:11:05,148][20381] Updated weights for policy 1, policy_version 1375194 (0.0005) [2023-12-27 15:11:05,205][20381] Updated weights for policy 1, policy_version 1375204 (0.0006) [2023-12-27 15:11:05,256][20381] Updated weights for policy 1, policy_version 1375214 (0.0007) [2023-12-27 15:11:05,671][20347] Updated weights for policy 0, policy_version 1376444 (0.0006) [2023-12-27 15:11:05,720][20347] Updated weights for policy 0, policy_version 1376454 (0.0006) [2023-12-27 15:11:05,774][20347] Updated weights for policy 0, policy_version 1376464 (0.0005) [2023-12-27 15:11:05,859][20381] Updated weights for policy 1, policy_version 1375224 (0.0006) [2023-12-27 15:11:05,922][20381] Updated weights for policy 1, policy_version 1375234 (0.0005) [2023-12-27 15:11:05,983][20381] Updated weights for policy 1, policy_version 1375244 (0.0005) [2023-12-27 15:11:06,342][20347] Updated weights for policy 0, policy_version 1376474 (0.0006) [2023-12-27 15:11:06,404][20347] Updated weights for policy 0, policy_version 1376484 (0.0007) [2023-12-27 15:11:06,465][20347] Updated weights for policy 0, policy_version 1376494 (0.0008) [2023-12-27 15:11:06,515][20347] Updated weights for policy 0, policy_version 1376504 (0.0007) [2023-12-27 15:11:06,596][20381] Updated weights for policy 1, policy_version 1375254 (0.0006) [2023-12-27 15:11:06,655][20381] Updated weights for policy 1, policy_version 1375264 (0.0009) [2023-12-27 15:11:06,709][20381] Updated weights for policy 1, policy_version 1375274 (0.0008) [2023-12-27 15:11:07,214][20347] Updated weights for policy 0, policy_version 1376514 (0.0008) [2023-12-27 15:11:07,262][20347] Updated weights for policy 0, policy_version 1376524 (0.0008) [2023-12-27 15:11:07,310][20347] Updated weights for policy 0, policy_version 1376534 (0.0007) [2023-12-27 15:11:07,322][20381] Updated weights for policy 1, policy_version 1375284 (0.0009) [2023-12-27 15:11:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 704561152. Throughput: 0: 10102.9, 1: 10170.9. Samples: 704526448. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:07,363][19170] Avg episode reward: [(0, '931.700'), (1, '970.820')] [2023-12-27 15:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001376536_352460800.pth... [2023-12-27 15:11:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001375352_352157696.pth [2023-12-27 15:11:07,379][20381] Updated weights for policy 1, policy_version 1375294 (0.0008) [2023-12-27 15:11:07,434][20381] Updated weights for policy 1, policy_version 1375304 (0.0006) [2023-12-27 15:11:07,474][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001375312_352108544.pth... [2023-12-27 15:11:07,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001374096_351797248.pth [2023-12-27 15:11:08,031][20381] Updated weights for policy 1, policy_version 1375314 (0.0008) [2023-12-27 15:11:08,066][20347] Updated weights for policy 0, policy_version 1376544 (0.0007) [2023-12-27 15:11:08,083][20381] Updated weights for policy 1, policy_version 1375324 (0.0006) [2023-12-27 15:11:08,126][20347] Updated weights for policy 0, policy_version 1376554 (0.0007) [2023-12-27 15:11:08,135][20381] Updated weights for policy 1, policy_version 1375334 (0.0006) [2023-12-27 15:11:08,174][20347] Updated weights for policy 0, policy_version 1376564 (0.0005) [2023-12-27 15:11:08,187][20381] Updated weights for policy 1, policy_version 1375344 (0.0006) [2023-12-27 15:11:08,829][20347] Updated weights for policy 0, policy_version 1376574 (0.0007) [2023-12-27 15:11:08,887][20347] Updated weights for policy 0, policy_version 1376584 (0.0007) [2023-12-27 15:11:08,936][20381] Updated weights for policy 1, policy_version 1375354 (0.0011) [2023-12-27 15:11:08,941][20347] Updated weights for policy 0, policy_version 1376594 (0.0006) [2023-12-27 15:11:08,994][20381] Updated weights for policy 1, policy_version 1375364 (0.0010) [2023-12-27 15:11:09,052][20381] Updated weights for policy 1, policy_version 1375374 (0.0010) [2023-12-27 15:11:09,622][20347] Updated weights for policy 0, policy_version 1376604 (0.0007) [2023-12-27 15:11:09,685][20347] Updated weights for policy 0, policy_version 1376614 (0.0007) [2023-12-27 15:11:09,751][20347] Updated weights for policy 0, policy_version 1376624 (0.0008) [2023-12-27 15:11:09,802][20381] Updated weights for policy 1, policy_version 1375384 (0.0007) [2023-12-27 15:11:09,865][20381] Updated weights for policy 1, policy_version 1375394 (0.0008) [2023-12-27 15:11:09,929][20381] Updated weights for policy 1, policy_version 1375404 (0.0006) [2023-12-27 15:11:10,498][20347] Updated weights for policy 0, policy_version 1376634 (0.0006) [2023-12-27 15:11:10,555][20347] Updated weights for policy 0, policy_version 1376644 (0.0010) [2023-12-27 15:11:10,588][20381] Updated weights for policy 1, policy_version 1375414 (0.0008) [2023-12-27 15:11:10,608][20347] Updated weights for policy 0, policy_version 1376654 (0.0008) [2023-12-27 15:11:10,648][20381] Updated weights for policy 1, policy_version 1375424 (0.0008) [2023-12-27 15:11:10,664][20347] Updated weights for policy 0, policy_version 1376664 (0.0005) [2023-12-27 15:11:10,708][20381] Updated weights for policy 1, policy_version 1375434 (0.0007) [2023-12-27 15:11:11,388][20347] Updated weights for policy 0, policy_version 1376674 (0.0010) [2023-12-27 15:11:11,435][20381] Updated weights for policy 1, policy_version 1375444 (0.0007) [2023-12-27 15:11:11,452][20347] Updated weights for policy 0, policy_version 1376684 (0.0007) [2023-12-27 15:11:11,497][20381] Updated weights for policy 1, policy_version 1375454 (0.0007) [2023-12-27 15:11:11,516][20347] Updated weights for policy 0, policy_version 1376694 (0.0009) [2023-12-27 15:11:11,556][20381] Updated weights for policy 1, policy_version 1375464 (0.0009) [2023-12-27 15:11:12,176][20347] Updated weights for policy 0, policy_version 1376704 (0.0009) [2023-12-27 15:11:12,244][20347] Updated weights for policy 0, policy_version 1376714 (0.0010) [2023-12-27 15:11:12,270][20381] Updated weights for policy 1, policy_version 1375474 (0.0008) [2023-12-27 15:11:12,312][20347] Updated weights for policy 0, policy_version 1376724 (0.0009) [2023-12-27 15:11:12,332][20381] Updated weights for policy 1, policy_version 1375484 (0.0008) [2023-12-27 15:11:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 704659456. Throughput: 0: 10091.8, 1: 10202.5. Samples: 704650756. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:12,363][19170] Avg episode reward: [(0, '960.950'), (1, '980.490')] [2023-12-27 15:11:12,398][20381] Updated weights for policy 1, policy_version 1375494 (0.0006) [2023-12-27 15:11:12,453][20381] Updated weights for policy 1, policy_version 1375504 (0.0007) [2023-12-27 15:11:12,970][20347] Updated weights for policy 0, policy_version 1376734 (0.0010) [2023-12-27 15:11:13,017][20347] Updated weights for policy 0, policy_version 1376744 (0.0010) [2023-12-27 15:11:13,068][20347] Updated weights for policy 0, policy_version 1376754 (0.0008) [2023-12-27 15:11:13,126][20381] Updated weights for policy 1, policy_version 1375514 (0.0008) [2023-12-27 15:11:13,183][20381] Updated weights for policy 1, policy_version 1375524 (0.0008) [2023-12-27 15:11:13,239][20381] Updated weights for policy 1, policy_version 1375534 (0.0008) [2023-12-27 15:11:13,738][20347] Updated weights for policy 0, policy_version 1376764 (0.0006) [2023-12-27 15:11:13,796][20347] Updated weights for policy 0, policy_version 1376774 (0.0008) [2023-12-27 15:11:13,851][20347] Updated weights for policy 0, policy_version 1376784 (0.0005) [2023-12-27 15:11:13,979][20381] Updated weights for policy 1, policy_version 1375544 (0.0007) [2023-12-27 15:11:14,047][20381] Updated weights for policy 1, policy_version 1375554 (0.0006) [2023-12-27 15:11:14,103][20381] Updated weights for policy 1, policy_version 1375564 (0.0008) [2023-12-27 15:11:14,465][20347] Updated weights for policy 0, policy_version 1376794 (0.0006) [2023-12-27 15:11:14,516][20347] Updated weights for policy 0, policy_version 1376804 (0.0005) [2023-12-27 15:11:14,570][20347] Updated weights for policy 0, policy_version 1376814 (0.0006) [2023-12-27 15:11:14,623][20347] Updated weights for policy 0, policy_version 1376824 (0.0005) [2023-12-27 15:11:14,746][20381] Updated weights for policy 1, policy_version 1375574 (0.0006) [2023-12-27 15:11:14,806][20381] Updated weights for policy 1, policy_version 1375584 (0.0008) [2023-12-27 15:11:14,859][20381] Updated weights for policy 1, policy_version 1375594 (0.0007) [2023-12-27 15:11:15,239][20347] Updated weights for policy 0, policy_version 1376834 (0.0007) [2023-12-27 15:11:15,298][20347] Updated weights for policy 0, policy_version 1376844 (0.0009) [2023-12-27 15:11:15,357][20347] Updated weights for policy 0, policy_version 1376854 (0.0007) [2023-12-27 15:11:15,498][20381] Updated weights for policy 1, policy_version 1375604 (0.0007) [2023-12-27 15:11:15,553][20381] Updated weights for policy 1, policy_version 1375614 (0.0007) [2023-12-27 15:11:15,608][20381] Updated weights for policy 1, policy_version 1375624 (0.0008) [2023-12-27 15:11:16,012][20347] Updated weights for policy 0, policy_version 1376864 (0.0009) [2023-12-27 15:11:16,069][20347] Updated weights for policy 0, policy_version 1376874 (0.0006) [2023-12-27 15:11:16,131][20347] Updated weights for policy 0, policy_version 1376884 (0.0006) [2023-12-27 15:11:16,218][20381] Updated weights for policy 1, policy_version 1375634 (0.0007) [2023-12-27 15:11:16,270][20381] Updated weights for policy 1, policy_version 1375644 (0.0008) [2023-12-27 15:11:16,319][20381] Updated weights for policy 1, policy_version 1375655 (0.0008) [2023-12-27 15:11:16,737][20347] Updated weights for policy 0, policy_version 1376894 (0.0006) [2023-12-27 15:11:16,786][20347] Updated weights for policy 0, policy_version 1376904 (0.0005) [2023-12-27 15:11:16,840][20347] Updated weights for policy 0, policy_version 1376914 (0.0006) [2023-12-27 15:11:17,072][20381] Updated weights for policy 1, policy_version 1375665 (0.0009) [2023-12-27 15:11:17,118][20381] Updated weights for policy 1, policy_version 1375675 (0.0010) [2023-12-27 15:11:17,165][20381] Updated weights for policy 1, policy_version 1375685 (0.0009) [2023-12-27 15:11:17,216][20381] Updated weights for policy 1, policy_version 1375695 (0.0008) [2023-12-27 15:11:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 704765952. Throughput: 0: 10107.8, 1: 10129.1. Samples: 704774284. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.480')] [2023-12-27 15:11:17,405][20347] Updated weights for policy 0, policy_version 1376924 (0.0007) [2023-12-27 15:11:17,451][20347] Updated weights for policy 0, policy_version 1376934 (0.0005) [2023-12-27 15:11:17,497][20347] Updated weights for policy 0, policy_version 1376944 (0.0005) [2023-12-27 15:11:18,020][20381] Updated weights for policy 1, policy_version 1375705 (0.0008) [2023-12-27 15:11:18,078][20381] Updated weights for policy 1, policy_version 1375715 (0.0008) [2023-12-27 15:11:18,080][20347] Updated weights for policy 0, policy_version 1376954 (0.0006) [2023-12-27 15:11:18,132][20381] Updated weights for policy 1, policy_version 1375725 (0.0005) [2023-12-27 15:11:18,147][20347] Updated weights for policy 0, policy_version 1376964 (0.0006) [2023-12-27 15:11:18,206][20347] Updated weights for policy 0, policy_version 1376974 (0.0008) [2023-12-27 15:11:18,257][20347] Updated weights for policy 0, policy_version 1376984 (0.0008) [2023-12-27 15:11:18,845][20381] Updated weights for policy 1, policy_version 1375735 (0.0007) [2023-12-27 15:11:18,901][20381] Updated weights for policy 1, policy_version 1375745 (0.0008) [2023-12-27 15:11:18,930][20347] Updated weights for policy 0, policy_version 1376994 (0.0005) [2023-12-27 15:11:18,958][20381] Updated weights for policy 1, policy_version 1375755 (0.0008) [2023-12-27 15:11:18,991][20347] Updated weights for policy 0, policy_version 1377004 (0.0005) [2023-12-27 15:11:19,053][20347] Updated weights for policy 0, policy_version 1377014 (0.0005) [2023-12-27 15:11:19,713][20347] Updated weights for policy 0, policy_version 1377024 (0.0006) [2023-12-27 15:11:19,722][20381] Updated weights for policy 1, policy_version 1375765 (0.0008) [2023-12-27 15:11:19,773][20347] Updated weights for policy 0, policy_version 1377034 (0.0007) [2023-12-27 15:11:19,782][20381] Updated weights for policy 1, policy_version 1375775 (0.0006) [2023-12-27 15:11:19,844][20347] Updated weights for policy 0, policy_version 1377044 (0.0007) [2023-12-27 15:11:19,846][20381] Updated weights for policy 1, policy_version 1375785 (0.0008) [2023-12-27 15:11:20,540][20381] Updated weights for policy 1, policy_version 1375795 (0.0007) [2023-12-27 15:11:20,552][20347] Updated weights for policy 0, policy_version 1377054 (0.0007) [2023-12-27 15:11:20,598][20381] Updated weights for policy 1, policy_version 1375805 (0.0007) [2023-12-27 15:11:20,610][20347] Updated weights for policy 0, policy_version 1377064 (0.0006) [2023-12-27 15:11:20,660][20381] Updated weights for policy 1, policy_version 1375815 (0.0007) [2023-12-27 15:11:20,673][20347] Updated weights for policy 0, policy_version 1377074 (0.0007) [2023-12-27 15:11:21,402][20381] Updated weights for policy 1, policy_version 1375825 (0.0007) [2023-12-27 15:11:21,409][20347] Updated weights for policy 0, policy_version 1377084 (0.0007) [2023-12-27 15:11:21,463][20381] Updated weights for policy 1, policy_version 1375835 (0.0007) [2023-12-27 15:11:21,475][20347] Updated weights for policy 0, policy_version 1377094 (0.0007) [2023-12-27 15:11:21,519][20381] Updated weights for policy 1, policy_version 1375845 (0.0008) [2023-12-27 15:11:21,536][20347] Updated weights for policy 0, policy_version 1377104 (0.0006) [2023-12-27 15:11:21,582][20381] Updated weights for policy 1, policy_version 1375855 (0.0006) [2023-12-27 15:11:22,222][20347] Updated weights for policy 0, policy_version 1377114 (0.0006) [2023-12-27 15:11:22,286][20347] Updated weights for policy 0, policy_version 1377124 (0.0006) [2023-12-27 15:11:22,303][20381] Updated weights for policy 1, policy_version 1375865 (0.0007) [2023-12-27 15:11:22,348][20347] Updated weights for policy 0, policy_version 1377134 (0.0007) [2023-12-27 15:11:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 704856064. Throughput: 0: 10140.5, 1: 10098.3. Samples: 704835360. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:22,363][19170] Avg episode reward: [(0, '970.680'), (1, '941.370')] [2023-12-27 15:11:22,363][20381] Updated weights for policy 1, policy_version 1375875 (0.0008) [2023-12-27 15:11:22,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001377144_352616448.pth... [2023-12-27 15:11:22,418][20347] Updated weights for policy 0, policy_version 1377144 (0.0008) [2023-12-27 15:11:22,421][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001375928_352305152.pth [2023-12-27 15:11:22,431][20381] Updated weights for policy 1, policy_version 1375885 (0.0007) [2023-12-27 15:11:22,446][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001375888_352256000.pth... [2023-12-27 15:11:22,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001374704_351952896.pth [2023-12-27 15:11:23,138][20347] Updated weights for policy 0, policy_version 1377154 (0.0006) [2023-12-27 15:11:23,152][20381] Updated weights for policy 1, policy_version 1375895 (0.0008) [2023-12-27 15:11:23,194][20347] Updated weights for policy 0, policy_version 1377164 (0.0006) [2023-12-27 15:11:23,206][20381] Updated weights for policy 1, policy_version 1375905 (0.0011) [2023-12-27 15:11:23,245][20347] Updated weights for policy 0, policy_version 1377174 (0.0005) [2023-12-27 15:11:23,261][20381] Updated weights for policy 1, policy_version 1375915 (0.0008) [2023-12-27 15:11:23,874][20347] Updated weights for policy 0, policy_version 1377184 (0.0007) [2023-12-27 15:11:23,902][20381] Updated weights for policy 1, policy_version 1375925 (0.0008) [2023-12-27 15:11:23,934][20347] Updated weights for policy 0, policy_version 1377194 (0.0008) [2023-12-27 15:11:23,963][20381] Updated weights for policy 1, policy_version 1375935 (0.0007) [2023-12-27 15:11:23,987][20347] Updated weights for policy 0, policy_version 1377204 (0.0005) [2023-12-27 15:11:24,023][20381] Updated weights for policy 1, policy_version 1375945 (0.0008) [2023-12-27 15:11:24,578][20347] Updated weights for policy 0, policy_version 1377214 (0.0007) [2023-12-27 15:11:24,631][20347] Updated weights for policy 0, policy_version 1377224 (0.0007) [2023-12-27 15:11:24,685][20347] Updated weights for policy 0, policy_version 1377234 (0.0007) [2023-12-27 15:11:24,687][20381] Updated weights for policy 1, policy_version 1375955 (0.0008) [2023-12-27 15:11:24,744][20381] Updated weights for policy 1, policy_version 1375965 (0.0006) [2023-12-27 15:11:24,802][20381] Updated weights for policy 1, policy_version 1375975 (0.0008) [2023-12-27 15:11:25,378][20347] Updated weights for policy 0, policy_version 1377244 (0.0007) [2023-12-27 15:11:25,433][20347] Updated weights for policy 0, policy_version 1377254 (0.0008) [2023-12-27 15:11:25,446][20381] Updated weights for policy 1, policy_version 1375985 (0.0008) [2023-12-27 15:11:25,485][20347] Updated weights for policy 0, policy_version 1377264 (0.0010) [2023-12-27 15:11:25,517][20381] Updated weights for policy 1, policy_version 1375995 (0.0006) [2023-12-27 15:11:25,580][20381] Updated weights for policy 1, policy_version 1376005 (0.0007) [2023-12-27 15:11:25,638][20381] Updated weights for policy 1, policy_version 1376015 (0.0008) [2023-12-27 15:11:26,190][20347] Updated weights for policy 0, policy_version 1377274 (0.0009) [2023-12-27 15:11:26,246][20347] Updated weights for policy 0, policy_version 1377284 (0.0010) [2023-12-27 15:11:26,293][20347] Updated weights for policy 0, policy_version 1377294 (0.0010) [2023-12-27 15:11:26,298][20381] Updated weights for policy 1, policy_version 1376025 (0.0007) [2023-12-27 15:11:26,338][20347] Updated weights for policy 0, policy_version 1377304 (0.0010) [2023-12-27 15:11:26,353][20381] Updated weights for policy 1, policy_version 1376035 (0.0006) [2023-12-27 15:11:26,412][20381] Updated weights for policy 1, policy_version 1376045 (0.0008) [2023-12-27 15:11:26,943][20347] Updated weights for policy 0, policy_version 1377314 (0.0005) [2023-12-27 15:11:27,006][20347] Updated weights for policy 0, policy_version 1377324 (0.0005) [2023-12-27 15:11:27,061][20347] Updated weights for policy 0, policy_version 1377334 (0.0007) [2023-12-27 15:11:27,071][20381] Updated weights for policy 1, policy_version 1376055 (0.0007) [2023-12-27 15:11:27,128][20381] Updated weights for policy 1, policy_version 1376065 (0.0008) [2023-12-27 15:11:27,193][20381] Updated weights for policy 1, policy_version 1376075 (0.0008) [2023-12-27 15:11:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 704970752. Throughput: 0: 10156.0, 1: 10090.9. Samples: 704956884. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:27,363][19170] Avg episode reward: [(0, '921.830'), (1, '921.830')] [2023-12-27 15:11:27,635][20347] Updated weights for policy 0, policy_version 1377344 (0.0006) [2023-12-27 15:11:27,700][20347] Updated weights for policy 0, policy_version 1377354 (0.0007) [2023-12-27 15:11:27,760][20347] Updated weights for policy 0, policy_version 1377364 (0.0007) [2023-12-27 15:11:27,834][20381] Updated weights for policy 1, policy_version 1376085 (0.0007) [2023-12-27 15:11:27,888][20381] Updated weights for policy 1, policy_version 1376095 (0.0005) [2023-12-27 15:11:27,940][20381] Updated weights for policy 1, policy_version 1376105 (0.0007) [2023-12-27 15:11:28,429][20347] Updated weights for policy 0, policy_version 1377374 (0.0007) [2023-12-27 15:11:28,484][20347] Updated weights for policy 0, policy_version 1377384 (0.0005) [2023-12-27 15:11:28,535][20347] Updated weights for policy 0, policy_version 1377394 (0.0005) [2023-12-27 15:11:28,602][20381] Updated weights for policy 1, policy_version 1376115 (0.0008) [2023-12-27 15:11:28,660][20381] Updated weights for policy 1, policy_version 1376125 (0.0009) [2023-12-27 15:11:28,714][20381] Updated weights for policy 1, policy_version 1376135 (0.0006) [2023-12-27 15:11:29,146][20347] Updated weights for policy 0, policy_version 1377404 (0.0006) [2023-12-27 15:11:29,198][20347] Updated weights for policy 0, policy_version 1377414 (0.0008) [2023-12-27 15:11:29,199][20291] KL-divergence is very high: 185.7373 [2023-12-27 15:11:29,211][20291] KL-divergence is very high: 187.7914 [2023-12-27 15:11:29,277][20347] Updated weights for policy 0, policy_version 1377424 (0.0007) [2023-12-27 15:11:29,433][20381] Updated weights for policy 1, policy_version 1376145 (0.0006) [2023-12-27 15:11:29,490][20381] Updated weights for policy 1, policy_version 1376155 (0.0007) [2023-12-27 15:11:29,545][20381] Updated weights for policy 1, policy_version 1376165 (0.0008) [2023-12-27 15:11:29,597][20381] Updated weights for policy 1, policy_version 1376175 (0.0007) [2023-12-27 15:11:29,952][20347] Updated weights for policy 0, policy_version 1377434 (0.0007) [2023-12-27 15:11:30,017][20347] Updated weights for policy 0, policy_version 1377444 (0.0008) [2023-12-27 15:11:30,079][20347] Updated weights for policy 0, policy_version 1377454 (0.0010) [2023-12-27 15:11:30,140][20347] Updated weights for policy 0, policy_version 1377464 (0.0010) [2023-12-27 15:11:30,318][20381] Updated weights for policy 1, policy_version 1376185 (0.0008) [2023-12-27 15:11:30,376][20381] Updated weights for policy 1, policy_version 1376195 (0.0008) [2023-12-27 15:11:30,433][20381] Updated weights for policy 1, policy_version 1376205 (0.0007) [2023-12-27 15:11:30,799][20347] Updated weights for policy 0, policy_version 1377474 (0.0010) [2023-12-27 15:11:30,851][20347] Updated weights for policy 0, policy_version 1377484 (0.0007) [2023-12-27 15:11:30,905][20347] Updated weights for policy 0, policy_version 1377494 (0.0005) [2023-12-27 15:11:31,069][20381] Updated weights for policy 1, policy_version 1376215 (0.0007) [2023-12-27 15:11:31,126][20381] Updated weights for policy 1, policy_version 1376225 (0.0007) [2023-12-27 15:11:31,182][20381] Updated weights for policy 1, policy_version 1376235 (0.0007) [2023-12-27 15:11:31,547][20347] Updated weights for policy 0, policy_version 1377504 (0.0007) [2023-12-27 15:11:31,610][20347] Updated weights for policy 0, policy_version 1377514 (0.0008) [2023-12-27 15:11:31,669][20347] Updated weights for policy 0, policy_version 1377524 (0.0007) [2023-12-27 15:11:31,875][20381] Updated weights for policy 1, policy_version 1376245 (0.0007) [2023-12-27 15:11:31,929][20381] Updated weights for policy 1, policy_version 1376255 (0.0008) [2023-12-27 15:11:31,983][20381] Updated weights for policy 1, policy_version 1376265 (0.0007) [2023-12-27 15:11:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 705069056. Throughput: 0: 10228.7, 1: 10113.6. Samples: 705081300. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:32,363][19170] Avg episode reward: [(0, '892.530'), (1, '912.050')] [2023-12-27 15:11:32,421][20347] Updated weights for policy 0, policy_version 1377534 (0.0006) [2023-12-27 15:11:32,482][20347] Updated weights for policy 0, policy_version 1377544 (0.0008) [2023-12-27 15:11:32,500][20291] KL-divergence is very high: 177.1673 [2023-12-27 15:11:32,513][20291] KL-divergence is very high: 146.9053 [2023-12-27 15:11:32,545][20347] Updated weights for policy 0, policy_version 1377554 (0.0008) [2023-12-27 15:11:32,549][20291] KL-divergence is very high: 194.4100 [2023-12-27 15:11:32,686][20381] Updated weights for policy 1, policy_version 1376275 (0.0007) [2023-12-27 15:11:32,736][20381] Updated weights for policy 1, policy_version 1376285 (0.0008) [2023-12-27 15:11:32,790][20381] Updated weights for policy 1, policy_version 1376295 (0.0008) [2023-12-27 15:11:33,248][20347] Updated weights for policy 0, policy_version 1377564 (0.0007) [2023-12-27 15:11:33,304][20347] Updated weights for policy 0, policy_version 1377574 (0.0007) [2023-12-27 15:11:33,354][20347] Updated weights for policy 0, policy_version 1377584 (0.0007) [2023-12-27 15:11:33,442][20381] Updated weights for policy 1, policy_version 1376305 (0.0008) [2023-12-27 15:11:33,497][20381] Updated weights for policy 1, policy_version 1376315 (0.0008) [2023-12-27 15:11:33,542][20381] Updated weights for policy 1, policy_version 1376325 (0.0005) [2023-12-27 15:11:33,590][20381] Updated weights for policy 1, policy_version 1376335 (0.0005) [2023-12-27 15:11:34,063][20347] Updated weights for policy 0, policy_version 1377594 (0.0007) [2023-12-27 15:11:34,114][20347] Updated weights for policy 0, policy_version 1377604 (0.0007) [2023-12-27 15:11:34,172][20347] Updated weights for policy 0, policy_version 1377614 (0.0008) [2023-12-27 15:11:34,197][20381] Updated weights for policy 1, policy_version 1376345 (0.0008) [2023-12-27 15:11:34,228][20347] Updated weights for policy 0, policy_version 1377624 (0.0008) [2023-12-27 15:11:34,252][20381] Updated weights for policy 1, policy_version 1376355 (0.0007) [2023-12-27 15:11:34,312][20381] Updated weights for policy 1, policy_version 1376365 (0.0007) [2023-12-27 15:11:34,891][20347] Updated weights for policy 0, policy_version 1377634 (0.0009) [2023-12-27 15:11:34,944][20347] Updated weights for policy 0, policy_version 1377644 (0.0006) [2023-12-27 15:11:35,000][20347] Updated weights for policy 0, policy_version 1377654 (0.0006) [2023-12-27 15:11:35,033][20381] Updated weights for policy 1, policy_version 1376375 (0.0009) [2023-12-27 15:11:35,100][20381] Updated weights for policy 1, policy_version 1376385 (0.0010) [2023-12-27 15:11:35,158][20381] Updated weights for policy 1, policy_version 1376395 (0.0010) [2023-12-27 15:11:35,677][20347] Updated weights for policy 0, policy_version 1377664 (0.0009) [2023-12-27 15:11:35,744][20347] Updated weights for policy 0, policy_version 1377674 (0.0010) [2023-12-27 15:11:35,802][20381] Updated weights for policy 1, policy_version 1376405 (0.0010) [2023-12-27 15:11:35,802][20347] Updated weights for policy 0, policy_version 1377684 (0.0010) [2023-12-27 15:11:35,860][20381] Updated weights for policy 1, policy_version 1376415 (0.0010) [2023-12-27 15:11:35,919][20381] Updated weights for policy 1, policy_version 1376425 (0.0006) [2023-12-27 15:11:36,447][20347] Updated weights for policy 0, policy_version 1377694 (0.0010) [2023-12-27 15:11:36,503][20347] Updated weights for policy 0, policy_version 1377704 (0.0011) [2023-12-27 15:11:36,562][20381] Updated weights for policy 1, policy_version 1376435 (0.0007) [2023-12-27 15:11:36,568][20347] Updated weights for policy 0, policy_version 1377714 (0.0008) [2023-12-27 15:11:36,618][20381] Updated weights for policy 1, policy_version 1376445 (0.0007) [2023-12-27 15:11:36,677][20381] Updated weights for policy 1, policy_version 1376455 (0.0008) [2023-12-27 15:11:37,217][20347] Updated weights for policy 0, policy_version 1377724 (0.0008) [2023-12-27 15:11:37,276][20381] Updated weights for policy 1, policy_version 1376465 (0.0008) [2023-12-27 15:11:37,277][20347] Updated weights for policy 0, policy_version 1377734 (0.0008) [2023-12-27 15:11:37,330][20381] Updated weights for policy 1, policy_version 1376475 (0.0005) [2023-12-27 15:11:37,335][20347] Updated weights for policy 0, policy_version 1377744 (0.0007) [2023-12-27 15:11:37,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 705167360. Throughput: 0: 10218.1, 1: 10136.5. Samples: 705142468. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:37,363][19170] Avg episode reward: [(0, '951.130'), (1, '951.140')] [2023-12-27 15:11:37,382][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001377752_352772096.pth... [2023-12-27 15:11:37,387][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001376536_352460800.pth [2023-12-27 15:11:37,388][20381] Updated weights for policy 1, policy_version 1376485 (0.0005) [2023-12-27 15:11:37,445][20381] Updated weights for policy 1, policy_version 1376495 (0.0006) [2023-12-27 15:11:37,450][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001376496_352411648.pth... [2023-12-27 15:11:37,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001375312_352108544.pth [2023-12-27 15:11:37,927][20347] Updated weights for policy 0, policy_version 1377754 (0.0006) [2023-12-27 15:11:37,988][20347] Updated weights for policy 0, policy_version 1377764 (0.0009) [2023-12-27 15:11:38,006][20381] Updated weights for policy 1, policy_version 1376505 (0.0006) [2023-12-27 15:11:38,041][20347] Updated weights for policy 0, policy_version 1377774 (0.0005) [2023-12-27 15:11:38,056][20381] Updated weights for policy 1, policy_version 1376515 (0.0005) [2023-12-27 15:11:38,093][20347] Updated weights for policy 0, policy_version 1377784 (0.0007) [2023-12-27 15:11:38,115][20381] Updated weights for policy 1, policy_version 1376525 (0.0005) [2023-12-27 15:11:38,734][20347] Updated weights for policy 0, policy_version 1377794 (0.0006) [2023-12-27 15:11:38,735][20381] Updated weights for policy 1, policy_version 1376535 (0.0005) [2023-12-27 15:11:38,794][20381] Updated weights for policy 1, policy_version 1376545 (0.0006) [2023-12-27 15:11:38,798][20347] Updated weights for policy 0, policy_version 1377804 (0.0007) [2023-12-27 15:11:38,847][20381] Updated weights for policy 1, policy_version 1376555 (0.0007) [2023-12-27 15:11:38,862][20347] Updated weights for policy 0, policy_version 1377814 (0.0009) [2023-12-27 15:11:39,508][20381] Updated weights for policy 1, policy_version 1376565 (0.0007) [2023-12-27 15:11:39,554][20381] Updated weights for policy 1, policy_version 1376575 (0.0007) [2023-12-27 15:11:39,564][20347] Updated weights for policy 0, policy_version 1377824 (0.0009) [2023-12-27 15:11:39,605][20381] Updated weights for policy 1, policy_version 1376585 (0.0006) [2023-12-27 15:11:39,624][20347] Updated weights for policy 0, policy_version 1377834 (0.0008) [2023-12-27 15:11:39,680][20347] Updated weights for policy 0, policy_version 1377844 (0.0008) [2023-12-27 15:11:40,299][20381] Updated weights for policy 1, policy_version 1376595 (0.0007) [2023-12-27 15:11:40,355][20381] Updated weights for policy 1, policy_version 1376605 (0.0007) [2023-12-27 15:11:40,414][20381] Updated weights for policy 1, policy_version 1376615 (0.0008) [2023-12-27 15:11:40,436][20347] Updated weights for policy 0, policy_version 1377854 (0.0008) [2023-12-27 15:11:40,491][20347] Updated weights for policy 0, policy_version 1377864 (0.0005) [2023-12-27 15:11:40,551][20347] Updated weights for policy 0, policy_version 1377874 (0.0005) [2023-12-27 15:11:41,099][20381] Updated weights for policy 1, policy_version 1376625 (0.0007) [2023-12-27 15:11:41,141][20347] Updated weights for policy 0, policy_version 1377884 (0.0009) [2023-12-27 15:11:41,165][20381] Updated weights for policy 1, policy_version 1376635 (0.0009) [2023-12-27 15:11:41,209][20347] Updated weights for policy 0, policy_version 1377894 (0.0008) [2023-12-27 15:11:41,217][20381] Updated weights for policy 1, policy_version 1376645 (0.0008) [2023-12-27 15:11:41,264][20347] Updated weights for policy 0, policy_version 1377904 (0.0007) [2023-12-27 15:11:41,277][20381] Updated weights for policy 1, policy_version 1376655 (0.0008) [2023-12-27 15:11:41,931][20347] Updated weights for policy 0, policy_version 1377914 (0.0008) [2023-12-27 15:11:41,994][20347] Updated weights for policy 0, policy_version 1377924 (0.0007) [2023-12-27 15:11:42,015][20381] Updated weights for policy 1, policy_version 1376665 (0.0007) [2023-12-27 15:11:42,054][20347] Updated weights for policy 0, policy_version 1377934 (0.0008) [2023-12-27 15:11:42,071][20381] Updated weights for policy 1, policy_version 1376675 (0.0008) [2023-12-27 15:11:42,117][20347] Updated weights for policy 0, policy_version 1377944 (0.0007) [2023-12-27 15:11:42,127][20381] Updated weights for policy 1, policy_version 1376685 (0.0007) [2023-12-27 15:11:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 705282048. Throughput: 0: 10283.0, 1: 10208.2. Samples: 705270060. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:42,363][19170] Avg episode reward: [(0, '941.470'), (1, '970.680')] [2023-12-27 15:11:42,800][20347] Updated weights for policy 0, policy_version 1377954 (0.0005) [2023-12-27 15:11:42,863][20347] Updated weights for policy 0, policy_version 1377964 (0.0007) [2023-12-27 15:11:42,878][20381] Updated weights for policy 1, policy_version 1376695 (0.0006) [2023-12-27 15:11:42,923][20347] Updated weights for policy 0, policy_version 1377974 (0.0008) [2023-12-27 15:11:42,933][20381] Updated weights for policy 1, policy_version 1376705 (0.0006) [2023-12-27 15:11:42,992][20381] Updated weights for policy 1, policy_version 1376715 (0.0007) [2023-12-27 15:11:43,590][20347] Updated weights for policy 0, policy_version 1377984 (0.0007) [2023-12-27 15:11:43,649][20347] Updated weights for policy 0, policy_version 1377994 (0.0007) [2023-12-27 15:11:43,672][20381] Updated weights for policy 1, policy_version 1376725 (0.0006) [2023-12-27 15:11:43,711][20347] Updated weights for policy 0, policy_version 1378004 (0.0006) [2023-12-27 15:11:43,732][20381] Updated weights for policy 1, policy_version 1376735 (0.0008) [2023-12-27 15:11:43,783][20381] Updated weights for policy 1, policy_version 1376745 (0.0010) [2023-12-27 15:11:44,354][20347] Updated weights for policy 0, policy_version 1378014 (0.0007) [2023-12-27 15:11:44,403][20347] Updated weights for policy 0, policy_version 1378024 (0.0007) [2023-12-27 15:11:44,462][20347] Updated weights for policy 0, policy_version 1378034 (0.0008) [2023-12-27 15:11:44,462][20381] Updated weights for policy 1, policy_version 1376755 (0.0010) [2023-12-27 15:11:44,517][20381] Updated weights for policy 1, policy_version 1376765 (0.0010) [2023-12-27 15:11:44,575][20381] Updated weights for policy 1, policy_version 1376775 (0.0006) [2023-12-27 15:11:45,129][20347] Updated weights for policy 0, policy_version 1378044 (0.0007) [2023-12-27 15:11:45,183][20347] Updated weights for policy 0, policy_version 1378054 (0.0006) [2023-12-27 15:11:45,195][20381] Updated weights for policy 1, policy_version 1376785 (0.0006) [2023-12-27 15:11:45,233][20347] Updated weights for policy 0, policy_version 1378064 (0.0008) [2023-12-27 15:11:45,245][20381] Updated weights for policy 1, policy_version 1376795 (0.0010) [2023-12-27 15:11:45,298][20381] Updated weights for policy 1, policy_version 1376805 (0.0011) [2023-12-27 15:11:45,351][20381] Updated weights for policy 1, policy_version 1376815 (0.0011) [2023-12-27 15:11:45,944][20347] Updated weights for policy 0, policy_version 1378074 (0.0006) [2023-12-27 15:11:45,947][20381] Updated weights for policy 1, policy_version 1376825 (0.0006) [2023-12-27 15:11:45,998][20347] Updated weights for policy 0, policy_version 1378084 (0.0007) [2023-12-27 15:11:46,001][20381] Updated weights for policy 1, policy_version 1376835 (0.0005) [2023-12-27 15:11:46,049][20381] Updated weights for policy 1, policy_version 1376845 (0.0007) [2023-12-27 15:11:46,049][20347] Updated weights for policy 0, policy_version 1378094 (0.0008) [2023-12-27 15:11:46,109][20347] Updated weights for policy 0, policy_version 1378104 (0.0008) [2023-12-27 15:11:46,638][20381] Updated weights for policy 1, policy_version 1376855 (0.0007) [2023-12-27 15:11:46,669][20292] KL-divergence is very high: 229.0689 [2023-12-27 15:11:46,675][20292] KL-divergence is very high: 154.2898 [2023-12-27 15:11:46,686][20381] Updated weights for policy 1, policy_version 1376865 (0.0007) [2023-12-27 15:11:46,747][20381] Updated weights for policy 1, policy_version 1376875 (0.0007) [2023-12-27 15:11:46,825][20347] Updated weights for policy 0, policy_version 1378114 (0.0008) [2023-12-27 15:11:46,877][20347] Updated weights for policy 0, policy_version 1378124 (0.0007) [2023-12-27 15:11:46,934][20347] Updated weights for policy 0, policy_version 1378134 (0.0007) [2023-12-27 15:11:47,341][20381] Updated weights for policy 1, policy_version 1376885 (0.0007) [2023-12-27 15:11:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 705380352. Throughput: 0: 10305.2, 1: 10302.2. Samples: 705393100. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:47,363][19170] Avg episode reward: [(0, '921.950'), (1, '961.030')] [2023-12-27 15:11:47,399][20381] Updated weights for policy 1, policy_version 1376895 (0.0008) [2023-12-27 15:11:47,460][20381] Updated weights for policy 1, policy_version 1376905 (0.0008) [2023-12-27 15:11:47,656][20347] Updated weights for policy 0, policy_version 1378144 (0.0008) [2023-12-27 15:11:47,712][20347] Updated weights for policy 0, policy_version 1378154 (0.0007) [2023-12-27 15:11:47,768][20347] Updated weights for policy 0, policy_version 1378164 (0.0008) [2023-12-27 15:11:48,112][20381] Updated weights for policy 1, policy_version 1376915 (0.0007) [2023-12-27 15:11:48,164][20381] Updated weights for policy 1, policy_version 1376925 (0.0005) [2023-12-27 15:11:48,206][20292] KL-divergence is very high: 135.1628 [2023-12-27 15:11:48,217][20381] Updated weights for policy 1, policy_version 1376935 (0.0006) [2023-12-27 15:11:48,510][20347] Updated weights for policy 0, policy_version 1378174 (0.0008) [2023-12-27 15:11:48,574][20347] Updated weights for policy 0, policy_version 1378184 (0.0008) [2023-12-27 15:11:48,642][20347] Updated weights for policy 0, policy_version 1378194 (0.0008) [2023-12-27 15:11:48,897][20381] Updated weights for policy 1, policy_version 1376945 (0.0008) [2023-12-27 15:11:48,959][20381] Updated weights for policy 1, policy_version 1376955 (0.0008) [2023-12-27 15:11:49,013][20381] Updated weights for policy 1, policy_version 1376965 (0.0008) [2023-12-27 15:11:49,062][20381] Updated weights for policy 1, policy_version 1376975 (0.0008) [2023-12-27 15:11:49,350][20347] Updated weights for policy 0, policy_version 1378204 (0.0007) [2023-12-27 15:11:49,414][20347] Updated weights for policy 0, policy_version 1378214 (0.0006) [2023-12-27 15:11:49,477][20347] Updated weights for policy 0, policy_version 1378224 (0.0006) [2023-12-27 15:11:49,486][20291] KL-divergence is very high: 519.8889 [2023-12-27 15:11:49,781][20381] Updated weights for policy 1, policy_version 1376985 (0.0008) [2023-12-27 15:11:49,847][20381] Updated weights for policy 1, policy_version 1376995 (0.0007) [2023-12-27 15:11:49,903][20381] Updated weights for policy 1, policy_version 1377005 (0.0008) [2023-12-27 15:11:50,124][20347] Updated weights for policy 0, policy_version 1378234 (0.0006) [2023-12-27 15:11:50,183][20347] Updated weights for policy 0, policy_version 1378244 (0.0007) [2023-12-27 15:11:50,243][20347] Updated weights for policy 0, policy_version 1378254 (0.0007) [2023-12-27 15:11:50,289][20347] Updated weights for policy 0, policy_version 1378264 (0.0007) [2023-12-27 15:11:50,632][20381] Updated weights for policy 1, policy_version 1377015 (0.0008) [2023-12-27 15:11:50,688][20381] Updated weights for policy 1, policy_version 1377025 (0.0009) [2023-12-27 15:11:50,743][20381] Updated weights for policy 1, policy_version 1377035 (0.0008) [2023-12-27 15:11:50,985][20347] Updated weights for policy 0, policy_version 1378274 (0.0008) [2023-12-27 15:11:51,050][20347] Updated weights for policy 0, policy_version 1378284 (0.0008) [2023-12-27 15:11:51,105][20347] Updated weights for policy 0, policy_version 1378294 (0.0007) [2023-12-27 15:11:51,463][20381] Updated weights for policy 1, policy_version 1377045 (0.0006) [2023-12-27 15:11:51,517][20381] Updated weights for policy 1, policy_version 1377055 (0.0005) [2023-12-27 15:11:51,578][20381] Updated weights for policy 1, policy_version 1377065 (0.0006) [2023-12-27 15:11:51,807][20347] Updated weights for policy 0, policy_version 1378304 (0.0008) [2023-12-27 15:11:51,868][20347] Updated weights for policy 0, policy_version 1378314 (0.0006) [2023-12-27 15:11:51,924][20347] Updated weights for policy 0, policy_version 1378324 (0.0007) [2023-12-27 15:11:52,261][20381] Updated weights for policy 1, policy_version 1377075 (0.0009) [2023-12-27 15:11:52,323][20381] Updated weights for policy 1, policy_version 1377085 (0.0007) [2023-12-27 15:11:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 705478656. Throughput: 0: 10315.2, 1: 10269.4. Samples: 705452756. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:52,363][19170] Avg episode reward: [(0, '941.480'), (1, '941.870')] [2023-12-27 15:11:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001378328_352919552.pth... [2023-12-27 15:11:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001377144_352616448.pth [2023-12-27 15:11:52,390][20381] Updated weights for policy 1, policy_version 1377095 (0.0008) [2023-12-27 15:11:52,443][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001377104_352567296.pth... [2023-12-27 15:11:52,448][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001375888_352256000.pth [2023-12-27 15:11:52,585][20347] Updated weights for policy 0, policy_version 1378334 (0.0008) [2023-12-27 15:11:52,639][20347] Updated weights for policy 0, policy_version 1378344 (0.0006) [2023-12-27 15:11:52,700][20347] Updated weights for policy 0, policy_version 1378354 (0.0005) [2023-12-27 15:11:53,111][20381] Updated weights for policy 1, policy_version 1377105 (0.0007) [2023-12-27 15:11:53,157][20381] Updated weights for policy 1, policy_version 1377115 (0.0007) [2023-12-27 15:11:53,205][20381] Updated weights for policy 1, policy_version 1377125 (0.0008) [2023-12-27 15:11:53,261][20381] Updated weights for policy 1, policy_version 1377135 (0.0008) [2023-12-27 15:11:53,271][20347] Updated weights for policy 0, policy_version 1378364 (0.0006) [2023-12-27 15:11:53,322][20347] Updated weights for policy 0, policy_version 1378374 (0.0007) [2023-12-27 15:11:53,381][20347] Updated weights for policy 0, policy_version 1378384 (0.0008) [2023-12-27 15:11:53,969][20381] Updated weights for policy 1, policy_version 1377145 (0.0006) [2023-12-27 15:11:54,032][20381] Updated weights for policy 1, policy_version 1377155 (0.0008) [2023-12-27 15:11:54,041][20347] Updated weights for policy 0, policy_version 1378394 (0.0006) [2023-12-27 15:11:54,086][20381] Updated weights for policy 1, policy_version 1377165 (0.0007) [2023-12-27 15:11:54,103][20347] Updated weights for policy 0, policy_version 1378404 (0.0010) [2023-12-27 15:11:54,158][20347] Updated weights for policy 0, policy_version 1378414 (0.0010) [2023-12-27 15:11:54,216][20347] Updated weights for policy 0, policy_version 1378424 (0.0010) [2023-12-27 15:11:54,727][20381] Updated weights for policy 1, policy_version 1377175 (0.0007) [2023-12-27 15:11:54,770][20381] Updated weights for policy 1, policy_version 1377185 (0.0008) [2023-12-27 15:11:54,803][20347] Updated weights for policy 0, policy_version 1378434 (0.0007) [2023-12-27 15:11:54,816][20381] Updated weights for policy 1, policy_version 1377195 (0.0006) [2023-12-27 15:11:54,861][20347] Updated weights for policy 0, policy_version 1378444 (0.0007) [2023-12-27 15:11:54,925][20347] Updated weights for policy 0, policy_version 1378454 (0.0007) [2023-12-27 15:11:55,494][20347] Updated weights for policy 0, policy_version 1378464 (0.0008) [2023-12-27 15:11:55,556][20381] Updated weights for policy 1, policy_version 1377205 (0.0005) [2023-12-27 15:11:55,557][20347] Updated weights for policy 0, policy_version 1378474 (0.0007) [2023-12-27 15:11:55,606][20347] Updated weights for policy 0, policy_version 1378484 (0.0006) [2023-12-27 15:11:55,608][20381] Updated weights for policy 1, policy_version 1377215 (0.0006) [2023-12-27 15:11:55,667][20381] Updated weights for policy 1, policy_version 1377225 (0.0007) [2023-12-27 15:11:56,238][20381] Updated weights for policy 1, policy_version 1377235 (0.0008) [2023-12-27 15:11:56,287][20292] KL-divergence is very high: 450.2108 [2023-12-27 15:11:56,293][20381] Updated weights for policy 1, policy_version 1377245 (0.0010) [2023-12-27 15:11:56,297][20347] Updated weights for policy 0, policy_version 1378494 (0.0005) [2023-12-27 15:11:56,327][20292] KL-divergence is very high: 320.6510 [2023-12-27 15:11:56,341][20381] Updated weights for policy 1, policy_version 1377255 (0.0010) [2023-12-27 15:11:56,351][20347] Updated weights for policy 0, policy_version 1378504 (0.0005) [2023-12-27 15:11:56,406][20347] Updated weights for policy 0, policy_version 1378514 (0.0005) [2023-12-27 15:11:57,019][20381] Updated weights for policy 1, policy_version 1377265 (0.0010) [2023-12-27 15:11:57,038][20347] Updated weights for policy 0, policy_version 1378524 (0.0007) [2023-12-27 15:11:57,076][20381] Updated weights for policy 1, policy_version 1377275 (0.0007) [2023-12-27 15:11:57,094][20347] Updated weights for policy 0, policy_version 1378534 (0.0010) [2023-12-27 15:11:57,131][20381] Updated weights for policy 1, policy_version 1377285 (0.0010) [2023-12-27 15:11:57,142][20347] Updated weights for policy 0, policy_version 1378544 (0.0010) [2023-12-27 15:11:57,193][20381] Updated weights for policy 1, policy_version 1377295 (0.0011) [2023-12-27 15:11:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 705593344. Throughput: 0: 10364.9, 1: 10234.6. Samples: 705577732. Policy #0 lag: (min: 10.0, avg: 20.3, max: 42.0) [2023-12-27 15:11:57,363][19170] Avg episode reward: [(0, '960.950'), (1, '932.080')] [2023-12-27 15:11:57,785][20347] Updated weights for policy 0, policy_version 1378554 (0.0009) [2023-12-27 15:11:57,842][20347] Updated weights for policy 0, policy_version 1378564 (0.0006) [2023-12-27 15:11:57,863][20381] Updated weights for policy 1, policy_version 1377305 (0.0010) [2023-12-27 15:11:57,900][20347] Updated weights for policy 0, policy_version 1378574 (0.0005) [2023-12-27 15:11:57,922][20381] Updated weights for policy 1, policy_version 1377315 (0.0011) [2023-12-27 15:11:57,953][20347] Updated weights for policy 0, policy_version 1378584 (0.0006) [2023-12-27 15:11:57,982][20381] Updated weights for policy 1, policy_version 1377325 (0.0005) [2023-12-27 15:11:58,624][20347] Updated weights for policy 0, policy_version 1378595 (0.0008) [2023-12-27 15:11:58,690][20347] Updated weights for policy 0, policy_version 1378605 (0.0007) [2023-12-27 15:11:58,705][20381] Updated weights for policy 1, policy_version 1377335 (0.0007) [2023-12-27 15:11:58,759][20347] Updated weights for policy 0, policy_version 1378615 (0.0007) [2023-12-27 15:11:58,769][20381] Updated weights for policy 1, policy_version 1377345 (0.0008) [2023-12-27 15:11:58,838][20381] Updated weights for policy 1, policy_version 1377355 (0.0008) [2023-12-27 15:11:59,546][20347] Updated weights for policy 0, policy_version 1378625 (0.0008) [2023-12-27 15:11:59,602][20381] Updated weights for policy 1, policy_version 1377365 (0.0006) [2023-12-27 15:11:59,607][20347] Updated weights for policy 0, policy_version 1378635 (0.0007) [2023-12-27 15:11:59,650][20381] Updated weights for policy 1, policy_version 1377375 (0.0006) [2023-12-27 15:11:59,654][20292] KL-divergence is very high: 194.4969 [2023-12-27 15:11:59,665][20292] KL-divergence is very high: 199.3884 [2023-12-27 15:11:59,667][20347] Updated weights for policy 0, policy_version 1378645 (0.0007) [2023-12-27 15:11:59,696][20292] KL-divergence is very high: 175.9301 [2023-12-27 15:11:59,702][20381] Updated weights for policy 1, policy_version 1377385 (0.0006) [2023-12-27 15:11:59,708][20292] KL-divergence is very high: 133.2475 [2023-12-27 15:12:00,364][20347] Updated weights for policy 0, policy_version 1378655 (0.0009) [2023-12-27 15:12:00,384][20381] Updated weights for policy 1, policy_version 1377395 (0.0008) [2023-12-27 15:12:00,410][20347] Updated weights for policy 0, policy_version 1378665 (0.0010) [2023-12-27 15:12:00,444][20381] Updated weights for policy 1, policy_version 1377405 (0.0006) [2023-12-27 15:12:00,469][20347] Updated weights for policy 0, policy_version 1378675 (0.0011) [2023-12-27 15:12:00,503][20381] Updated weights for policy 1, policy_version 1377415 (0.0005) [2023-12-27 15:12:01,149][20381] Updated weights for policy 1, policy_version 1377425 (0.0011) [2023-12-27 15:12:01,165][20347] Updated weights for policy 0, policy_version 1378685 (0.0010) [2023-12-27 15:12:01,207][20381] Updated weights for policy 1, policy_version 1377435 (0.0010) [2023-12-27 15:12:01,219][20347] Updated weights for policy 0, policy_version 1378695 (0.0008) [2023-12-27 15:12:01,263][20381] Updated weights for policy 1, policy_version 1377445 (0.0010) [2023-12-27 15:12:01,280][20347] Updated weights for policy 0, policy_version 1378705 (0.0008) [2023-12-27 15:12:01,312][20381] Updated weights for policy 1, policy_version 1377455 (0.0006) [2023-12-27 15:12:01,941][20381] Updated weights for policy 1, policy_version 1377465 (0.0007) [2023-12-27 15:12:01,994][20381] Updated weights for policy 1, policy_version 1377475 (0.0007) [2023-12-27 15:12:02,002][20347] Updated weights for policy 0, policy_version 1378715 (0.0009) [2023-12-27 15:12:02,053][20381] Updated weights for policy 1, policy_version 1377485 (0.0007) [2023-12-27 15:12:02,059][20347] Updated weights for policy 0, policy_version 1378725 (0.0006) [2023-12-27 15:12:02,117][20347] Updated weights for policy 0, policy_version 1378735 (0.0005) [2023-12-27 15:12:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 705691648. Throughput: 0: 10297.3, 1: 10236.6. Samples: 705698312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:02,363][19170] Avg episode reward: [(0, '970.840'), (1, '922.080')] [2023-12-27 15:12:02,727][20347] Updated weights for policy 0, policy_version 1378745 (0.0005) [2023-12-27 15:12:02,766][20381] Updated weights for policy 1, policy_version 1377495 (0.0007) [2023-12-27 15:12:02,781][20347] Updated weights for policy 0, policy_version 1378755 (0.0005) [2023-12-27 15:12:02,815][20381] Updated weights for policy 1, policy_version 1377505 (0.0007) [2023-12-27 15:12:02,835][20347] Updated weights for policy 0, policy_version 1378765 (0.0009) [2023-12-27 15:12:02,870][20381] Updated weights for policy 1, policy_version 1377515 (0.0006) [2023-12-27 15:12:02,893][20347] Updated weights for policy 0, policy_version 1378775 (0.0010) [2023-12-27 15:12:03,462][20381] Updated weights for policy 1, policy_version 1377525 (0.0007) [2023-12-27 15:12:03,522][20381] Updated weights for policy 1, policy_version 1377535 (0.0007) [2023-12-27 15:12:03,552][20347] Updated weights for policy 0, policy_version 1378785 (0.0010) [2023-12-27 15:12:03,579][20381] Updated weights for policy 1, policy_version 1377545 (0.0005) [2023-12-27 15:12:03,611][20347] Updated weights for policy 0, policy_version 1378795 (0.0010) [2023-12-27 15:12:03,659][20347] Updated weights for policy 0, policy_version 1378805 (0.0010) [2023-12-27 15:12:04,269][20381] Updated weights for policy 1, policy_version 1377555 (0.0006) [2023-12-27 15:12:04,330][20381] Updated weights for policy 1, policy_version 1377565 (0.0008) [2023-12-27 15:12:04,341][20347] Updated weights for policy 0, policy_version 1378815 (0.0010) [2023-12-27 15:12:04,387][20381] Updated weights for policy 1, policy_version 1377575 (0.0005) [2023-12-27 15:12:04,397][20347] Updated weights for policy 0, policy_version 1378825 (0.0010) [2023-12-27 15:12:04,453][20347] Updated weights for policy 0, policy_version 1378835 (0.0009) [2023-12-27 15:12:05,050][20381] Updated weights for policy 1, policy_version 1377585 (0.0006) [2023-12-27 15:12:05,085][20347] Updated weights for policy 0, policy_version 1378845 (0.0006) [2023-12-27 15:12:05,112][20381] Updated weights for policy 1, policy_version 1377595 (0.0010) [2023-12-27 15:12:05,145][20347] Updated weights for policy 0, policy_version 1378855 (0.0007) [2023-12-27 15:12:05,173][20381] Updated weights for policy 1, policy_version 1377605 (0.0010) [2023-12-27 15:12:05,202][20347] Updated weights for policy 0, policy_version 1378865 (0.0007) [2023-12-27 15:12:05,233][20381] Updated weights for policy 1, policy_version 1377615 (0.0011) [2023-12-27 15:12:05,779][20347] Updated weights for policy 0, policy_version 1378875 (0.0007) [2023-12-27 15:12:05,826][20381] Updated weights for policy 1, policy_version 1377625 (0.0006) [2023-12-27 15:12:05,833][20347] Updated weights for policy 0, policy_version 1378885 (0.0007) [2023-12-27 15:12:05,873][20381] Updated weights for policy 1, policy_version 1377635 (0.0007) [2023-12-27 15:12:05,889][20347] Updated weights for policy 0, policy_version 1378895 (0.0007) [2023-12-27 15:12:05,923][20381] Updated weights for policy 1, policy_version 1377645 (0.0006) [2023-12-27 15:12:06,478][20347] Updated weights for policy 0, policy_version 1378905 (0.0005) [2023-12-27 15:12:06,539][20347] Updated weights for policy 0, policy_version 1378915 (0.0008) [2023-12-27 15:12:06,540][20381] Updated weights for policy 1, policy_version 1377655 (0.0007) [2023-12-27 15:12:06,595][20381] Updated weights for policy 1, policy_version 1377665 (0.0006) [2023-12-27 15:12:06,598][20347] Updated weights for policy 0, policy_version 1378925 (0.0008) [2023-12-27 15:12:06,650][20381] Updated weights for policy 1, policy_version 1377675 (0.0006) [2023-12-27 15:12:06,656][20347] Updated weights for policy 0, policy_version 1378935 (0.0006) [2023-12-27 15:12:07,255][20381] Updated weights for policy 1, policy_version 1377685 (0.0010) [2023-12-27 15:12:07,306][20347] Updated weights for policy 0, policy_version 1378945 (0.0007) [2023-12-27 15:12:07,309][20381] Updated weights for policy 1, policy_version 1377695 (0.0008) [2023-12-27 15:12:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 705789952. Throughput: 0: 10289.4, 1: 10295.1. Samples: 705761660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:07,363][19170] Avg episode reward: [(0, '961.060'), (1, '941.600')] [2023-12-27 15:12:07,367][20347] Updated weights for policy 0, policy_version 1378955 (0.0007) [2023-12-27 15:12:07,370][20381] Updated weights for policy 1, policy_version 1377705 (0.0006) [2023-12-27 15:12:07,411][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001377712_352722944.pth... [2023-12-27 15:12:07,415][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001376496_352411648.pth [2023-12-27 15:12:07,423][20347] Updated weights for policy 0, policy_version 1378965 (0.0010) [2023-12-27 15:12:07,437][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001378968_353083392.pth... [2023-12-27 15:12:07,441][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001377752_352772096.pth [2023-12-27 15:12:07,954][20381] Updated weights for policy 1, policy_version 1377715 (0.0005) [2023-12-27 15:12:08,016][20381] Updated weights for policy 1, policy_version 1377725 (0.0009) [2023-12-27 15:12:08,045][20347] Updated weights for policy 0, policy_version 1378975 (0.0008) [2023-12-27 15:12:08,073][20381] Updated weights for policy 1, policy_version 1377735 (0.0008) [2023-12-27 15:12:08,094][20347] Updated weights for policy 0, policy_version 1378985 (0.0008) [2023-12-27 15:12:08,142][20347] Updated weights for policy 0, policy_version 1378995 (0.0010) [2023-12-27 15:12:08,753][20381] Updated weights for policy 1, policy_version 1377745 (0.0010) [2023-12-27 15:12:08,806][20381] Updated weights for policy 1, policy_version 1377755 (0.0011) [2023-12-27 15:12:08,838][20347] Updated weights for policy 0, policy_version 1379005 (0.0007) [2023-12-27 15:12:08,861][20381] Updated weights for policy 1, policy_version 1377765 (0.0009) [2023-12-27 15:12:08,896][20347] Updated weights for policy 0, policy_version 1379015 (0.0006) [2023-12-27 15:12:08,924][20381] Updated weights for policy 1, policy_version 1377775 (0.0009) [2023-12-27 15:12:08,951][20347] Updated weights for policy 0, policy_version 1379025 (0.0008) [2023-12-27 15:12:09,593][20347] Updated weights for policy 0, policy_version 1379035 (0.0007) [2023-12-27 15:12:09,645][20381] Updated weights for policy 1, policy_version 1377785 (0.0008) [2023-12-27 15:12:09,653][20347] Updated weights for policy 0, policy_version 1379045 (0.0005) [2023-12-27 15:12:09,702][20381] Updated weights for policy 1, policy_version 1377795 (0.0007) [2023-12-27 15:12:09,714][20347] Updated weights for policy 0, policy_version 1379055 (0.0006) [2023-12-27 15:12:09,760][20381] Updated weights for policy 1, policy_version 1377805 (0.0008) [2023-12-27 15:12:10,363][20347] Updated weights for policy 0, policy_version 1379065 (0.0006) [2023-12-27 15:12:10,404][20381] Updated weights for policy 1, policy_version 1377815 (0.0006) [2023-12-27 15:12:10,421][20347] Updated weights for policy 0, policy_version 1379075 (0.0007) [2023-12-27 15:12:10,464][20381] Updated weights for policy 1, policy_version 1377825 (0.0007) [2023-12-27 15:12:10,481][20347] Updated weights for policy 0, policy_version 1379085 (0.0006) [2023-12-27 15:12:10,522][20381] Updated weights for policy 1, policy_version 1377835 (0.0007) [2023-12-27 15:12:10,536][20347] Updated weights for policy 0, policy_version 1379095 (0.0005) [2023-12-27 15:12:11,179][20347] Updated weights for policy 0, policy_version 1379105 (0.0008) [2023-12-27 15:12:11,231][20347] Updated weights for policy 0, policy_version 1379115 (0.0008) [2023-12-27 15:12:11,272][20381] Updated weights for policy 1, policy_version 1377845 (0.0006) [2023-12-27 15:12:11,294][20347] Updated weights for policy 0, policy_version 1379125 (0.0008) [2023-12-27 15:12:11,334][20381] Updated weights for policy 1, policy_version 1377855 (0.0006) [2023-12-27 15:12:11,412][20381] Updated weights for policy 1, policy_version 1377865 (0.0008) [2023-12-27 15:12:12,046][20347] Updated weights for policy 0, policy_version 1379135 (0.0009) [2023-12-27 15:12:12,105][20347] Updated weights for policy 0, policy_version 1379145 (0.0008) [2023-12-27 15:12:12,158][20381] Updated weights for policy 1, policy_version 1377875 (0.0008) [2023-12-27 15:12:12,162][20347] Updated weights for policy 0, policy_version 1379155 (0.0006) [2023-12-27 15:12:12,217][20381] Updated weights for policy 1, policy_version 1377885 (0.0007) [2023-12-27 15:12:12,275][20381] Updated weights for policy 1, policy_version 1377895 (0.0007) [2023-12-27 15:12:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 705904640. Throughput: 0: 10362.1, 1: 10384.6. Samples: 705890484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:12,363][19170] Avg episode reward: [(0, '951.400'), (1, '960.970')] [2023-12-27 15:12:12,837][20347] Updated weights for policy 0, policy_version 1379165 (0.0007) [2023-12-27 15:12:12,895][20347] Updated weights for policy 0, policy_version 1379175 (0.0007) [2023-12-27 15:12:12,952][20347] Updated weights for policy 0, policy_version 1379185 (0.0008) [2023-12-27 15:12:12,960][20381] Updated weights for policy 1, policy_version 1377905 (0.0008) [2023-12-27 15:12:13,016][20381] Updated weights for policy 1, policy_version 1377915 (0.0007) [2023-12-27 15:12:13,068][20381] Updated weights for policy 1, policy_version 1377925 (0.0007) [2023-12-27 15:12:13,126][20381] Updated weights for policy 1, policy_version 1377935 (0.0006) [2023-12-27 15:12:13,674][20347] Updated weights for policy 0, policy_version 1379195 (0.0008) [2023-12-27 15:12:13,727][20347] Updated weights for policy 0, policy_version 1379205 (0.0008) [2023-12-27 15:12:13,789][20347] Updated weights for policy 0, policy_version 1379215 (0.0008) [2023-12-27 15:12:13,823][20381] Updated weights for policy 1, policy_version 1377945 (0.0010) [2023-12-27 15:12:13,879][20381] Updated weights for policy 1, policy_version 1377955 (0.0011) [2023-12-27 15:12:13,937][20381] Updated weights for policy 1, policy_version 1377965 (0.0011) [2023-12-27 15:12:14,499][20347] Updated weights for policy 0, policy_version 1379225 (0.0007) [2023-12-27 15:12:14,554][20347] Updated weights for policy 0, policy_version 1379235 (0.0007) [2023-12-27 15:12:14,604][20347] Updated weights for policy 0, policy_version 1379245 (0.0007) [2023-12-27 15:12:14,631][20381] Updated weights for policy 1, policy_version 1377975 (0.0011) [2023-12-27 15:12:14,651][20347] Updated weights for policy 0, policy_version 1379255 (0.0006) [2023-12-27 15:12:14,676][20381] Updated weights for policy 1, policy_version 1377985 (0.0010) [2023-12-27 15:12:14,731][20381] Updated weights for policy 1, policy_version 1377995 (0.0010) [2023-12-27 15:12:15,368][20347] Updated weights for policy 0, policy_version 1379265 (0.0007) [2023-12-27 15:12:15,417][20347] Updated weights for policy 0, policy_version 1379275 (0.0008) [2023-12-27 15:12:15,447][20381] Updated weights for policy 1, policy_version 1378005 (0.0008) [2023-12-27 15:12:15,463][20347] Updated weights for policy 0, policy_version 1379285 (0.0007) [2023-12-27 15:12:15,514][20381] Updated weights for policy 1, policy_version 1378015 (0.0008) [2023-12-27 15:12:15,576][20381] Updated weights for policy 1, policy_version 1378025 (0.0008) [2023-12-27 15:12:16,045][20347] Updated weights for policy 0, policy_version 1379295 (0.0006) [2023-12-27 15:12:16,088][20347] Updated weights for policy 0, policy_version 1379305 (0.0007) [2023-12-27 15:12:16,136][20347] Updated weights for policy 0, policy_version 1379315 (0.0007) [2023-12-27 15:12:16,288][20381] Updated weights for policy 1, policy_version 1378035 (0.0010) [2023-12-27 15:12:16,333][20381] Updated weights for policy 1, policy_version 1378045 (0.0010) [2023-12-27 15:12:16,381][20381] Updated weights for policy 1, policy_version 1378055 (0.0010) [2023-12-27 15:12:16,855][20347] Updated weights for policy 0, policy_version 1379325 (0.0007) [2023-12-27 15:12:16,921][20347] Updated weights for policy 0, policy_version 1379335 (0.0006) [2023-12-27 15:12:16,984][20347] Updated weights for policy 0, policy_version 1379345 (0.0006) [2023-12-27 15:12:17,027][20381] Updated weights for policy 1, policy_version 1378065 (0.0009) [2023-12-27 15:12:17,087][20381] Updated weights for policy 1, policy_version 1378075 (0.0009) [2023-12-27 15:12:17,136][20381] Updated weights for policy 1, policy_version 1378085 (0.0010) [2023-12-27 15:12:17,191][20381] Updated weights for policy 1, policy_version 1378095 (0.0010) [2023-12-27 15:12:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 706002944. Throughput: 0: 10300.6, 1: 10348.6. Samples: 706010512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:17,363][19170] Avg episode reward: [(0, '961.090'), (1, '941.400')] [2023-12-27 15:12:17,625][20347] Updated weights for policy 0, policy_version 1379355 (0.0008) [2023-12-27 15:12:17,684][20347] Updated weights for policy 0, policy_version 1379365 (0.0010) [2023-12-27 15:12:17,739][20347] Updated weights for policy 0, policy_version 1379375 (0.0010) [2023-12-27 15:12:17,835][20381] Updated weights for policy 1, policy_version 1378105 (0.0006) [2023-12-27 15:12:17,885][20381] Updated weights for policy 1, policy_version 1378115 (0.0008) [2023-12-27 15:12:17,935][20381] Updated weights for policy 1, policy_version 1378125 (0.0011) [2023-12-27 15:12:18,416][20347] Updated weights for policy 0, policy_version 1379385 (0.0009) [2023-12-27 15:12:18,478][20347] Updated weights for policy 0, policy_version 1379395 (0.0008) [2023-12-27 15:12:18,534][20347] Updated weights for policy 0, policy_version 1379405 (0.0006) [2023-12-27 15:12:18,591][20347] Updated weights for policy 0, policy_version 1379415 (0.0011) [2023-12-27 15:12:18,608][20381] Updated weights for policy 1, policy_version 1378135 (0.0010) [2023-12-27 15:12:18,658][20381] Updated weights for policy 1, policy_version 1378145 (0.0011) [2023-12-27 15:12:18,711][20381] Updated weights for policy 1, policy_version 1378155 (0.0011) [2023-12-27 15:12:19,237][20347] Updated weights for policy 0, policy_version 1379425 (0.0010) [2023-12-27 15:12:19,300][20347] Updated weights for policy 0, policy_version 1379435 (0.0010) [2023-12-27 15:12:19,364][20347] Updated weights for policy 0, policy_version 1379445 (0.0010) [2023-12-27 15:12:19,408][20381] Updated weights for policy 1, policy_version 1378165 (0.0011) [2023-12-27 15:12:19,466][20381] Updated weights for policy 1, policy_version 1378175 (0.0009) [2023-12-27 15:12:19,531][20381] Updated weights for policy 1, policy_version 1378185 (0.0008) [2023-12-27 15:12:20,064][20347] Updated weights for policy 0, policy_version 1379455 (0.0011) [2023-12-27 15:12:20,130][20347] Updated weights for policy 0, policy_version 1379465 (0.0010) [2023-12-27 15:12:20,189][20347] Updated weights for policy 0, policy_version 1379475 (0.0011) [2023-12-27 15:12:20,299][20381] Updated weights for policy 1, policy_version 1378195 (0.0009) [2023-12-27 15:12:20,366][20381] Updated weights for policy 1, policy_version 1378205 (0.0010) [2023-12-27 15:12:20,431][20381] Updated weights for policy 1, policy_version 1378215 (0.0008) [2023-12-27 15:12:20,817][20347] Updated weights for policy 0, policy_version 1379485 (0.0009) [2023-12-27 15:12:20,877][20347] Updated weights for policy 0, policy_version 1379495 (0.0008) [2023-12-27 15:12:20,937][20347] Updated weights for policy 0, policy_version 1379505 (0.0008) [2023-12-27 15:12:21,164][20381] Updated weights for policy 1, policy_version 1378225 (0.0006) [2023-12-27 15:12:21,227][20381] Updated weights for policy 1, policy_version 1378235 (0.0008) [2023-12-27 15:12:21,286][20381] Updated weights for policy 1, policy_version 1378245 (0.0006) [2023-12-27 15:12:21,349][20381] Updated weights for policy 1, policy_version 1378255 (0.0007) [2023-12-27 15:12:21,683][20347] Updated weights for policy 0, policy_version 1379515 (0.0007) [2023-12-27 15:12:21,749][20347] Updated weights for policy 0, policy_version 1379525 (0.0007) [2023-12-27 15:12:21,814][20347] Updated weights for policy 0, policy_version 1379535 (0.0007) [2023-12-27 15:12:22,031][20381] Updated weights for policy 1, policy_version 1378265 (0.0008) [2023-12-27 15:12:22,082][20381] Updated weights for policy 1, policy_version 1378275 (0.0009) [2023-12-27 15:12:22,134][20381] Updated weights for policy 1, policy_version 1378285 (0.0007) [2023-12-27 15:12:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 706101248. Throughput: 0: 10320.4, 1: 10331.0. Samples: 706071780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:22,363][19170] Avg episode reward: [(0, '970.720'), (1, '931.670')] [2023-12-27 15:12:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001378288_352870400.pth... [2023-12-27 15:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001379544_353230848.pth... [2023-12-27 15:12:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001377104_352567296.pth [2023-12-27 15:12:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001378328_352919552.pth [2023-12-27 15:12:22,535][20347] Updated weights for policy 0, policy_version 1379545 (0.0007) [2023-12-27 15:12:22,592][20347] Updated weights for policy 0, policy_version 1379555 (0.0009) [2023-12-27 15:12:22,658][20347] Updated weights for policy 0, policy_version 1379565 (0.0008) [2023-12-27 15:12:22,724][20347] Updated weights for policy 0, policy_version 1379575 (0.0008) [2023-12-27 15:12:22,918][20381] Updated weights for policy 1, policy_version 1378295 (0.0010) [2023-12-27 15:12:22,967][20381] Updated weights for policy 1, policy_version 1378305 (0.0010) [2023-12-27 15:12:23,024][20381] Updated weights for policy 1, policy_version 1378315 (0.0011) [2023-12-27 15:12:23,442][20347] Updated weights for policy 0, policy_version 1379585 (0.0007) [2023-12-27 15:12:23,490][20347] Updated weights for policy 0, policy_version 1379595 (0.0007) [2023-12-27 15:12:23,535][20347] Updated weights for policy 0, policy_version 1379605 (0.0008) [2023-12-27 15:12:23,727][20381] Updated weights for policy 1, policy_version 1378325 (0.0011) [2023-12-27 15:12:23,788][20381] Updated weights for policy 1, policy_version 1378335 (0.0010) [2023-12-27 15:12:23,839][20381] Updated weights for policy 1, policy_version 1378345 (0.0010) [2023-12-27 15:12:24,252][20347] Updated weights for policy 0, policy_version 1379615 (0.0007) [2023-12-27 15:12:24,303][20347] Updated weights for policy 0, policy_version 1379625 (0.0007) [2023-12-27 15:12:24,351][20347] Updated weights for policy 0, policy_version 1379635 (0.0007) [2023-12-27 15:12:24,513][20381] Updated weights for policy 1, policy_version 1378355 (0.0011) [2023-12-27 15:12:24,567][20381] Updated weights for policy 1, policy_version 1378365 (0.0011) [2023-12-27 15:12:24,619][20381] Updated weights for policy 1, policy_version 1378375 (0.0010) [2023-12-27 15:12:25,073][20347] Updated weights for policy 0, policy_version 1379645 (0.0007) [2023-12-27 15:12:25,138][20347] Updated weights for policy 0, policy_version 1379655 (0.0007) [2023-12-27 15:12:25,200][20347] Updated weights for policy 0, policy_version 1379665 (0.0007) [2023-12-27 15:12:25,318][20381] Updated weights for policy 1, policy_version 1378385 (0.0011) [2023-12-27 15:12:25,369][20381] Updated weights for policy 1, policy_version 1378395 (0.0010) [2023-12-27 15:12:25,424][20381] Updated weights for policy 1, policy_version 1378405 (0.0010) [2023-12-27 15:12:25,479][20381] Updated weights for policy 1, policy_version 1378415 (0.0010) [2023-12-27 15:12:25,884][20347] Updated weights for policy 0, policy_version 1379675 (0.0007) [2023-12-27 15:12:25,934][20347] Updated weights for policy 0, policy_version 1379685 (0.0007) [2023-12-27 15:12:25,993][20347] Updated weights for policy 0, policy_version 1379695 (0.0008) [2023-12-27 15:12:26,163][20381] Updated weights for policy 1, policy_version 1378425 (0.0006) [2023-12-27 15:12:26,222][20381] Updated weights for policy 1, policy_version 1378435 (0.0005) [2023-12-27 15:12:26,276][20381] Updated weights for policy 1, policy_version 1378445 (0.0005) [2023-12-27 15:12:26,705][20347] Updated weights for policy 0, policy_version 1379705 (0.0007) [2023-12-27 15:12:26,750][20347] Updated weights for policy 0, policy_version 1379715 (0.0008) [2023-12-27 15:12:26,804][20347] Updated weights for policy 0, policy_version 1379725 (0.0008) [2023-12-27 15:12:26,837][20381] Updated weights for policy 1, policy_version 1378455 (0.0006) [2023-12-27 15:12:26,856][20347] Updated weights for policy 0, policy_version 1379735 (0.0008) [2023-12-27 15:12:26,891][20381] Updated weights for policy 1, policy_version 1378465 (0.0007) [2023-12-27 15:12:26,940][20381] Updated weights for policy 1, policy_version 1378475 (0.0008) [2023-12-27 15:12:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 706199552. Throughput: 0: 10212.3, 1: 10218.9. Samples: 706189460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:27,363][19170] Avg episode reward: [(0, '970.730'), (1, '960.990')] [2023-12-27 15:12:27,511][20381] Updated weights for policy 1, policy_version 1378485 (0.0007) [2023-12-27 15:12:27,565][20347] Updated weights for policy 0, policy_version 1379745 (0.0005) [2023-12-27 15:12:27,574][20381] Updated weights for policy 1, policy_version 1378495 (0.0008) [2023-12-27 15:12:27,610][20347] Updated weights for policy 0, policy_version 1379755 (0.0007) [2023-12-27 15:12:27,631][20381] Updated weights for policy 1, policy_version 1378505 (0.0008) [2023-12-27 15:12:27,666][20347] Updated weights for policy 0, policy_version 1379765 (0.0008) [2023-12-27 15:12:28,253][20381] Updated weights for policy 1, policy_version 1378515 (0.0007) [2023-12-27 15:12:28,307][20381] Updated weights for policy 1, policy_version 1378525 (0.0010) [2023-12-27 15:12:28,368][20381] Updated weights for policy 1, policy_version 1378535 (0.0009) [2023-12-27 15:12:28,381][20347] Updated weights for policy 0, policy_version 1379775 (0.0007) [2023-12-27 15:12:28,438][20347] Updated weights for policy 0, policy_version 1379785 (0.0006) [2023-12-27 15:12:28,498][20347] Updated weights for policy 0, policy_version 1379795 (0.0008) [2023-12-27 15:12:29,062][20381] Updated weights for policy 1, policy_version 1378545 (0.0010) [2023-12-27 15:12:29,120][20381] Updated weights for policy 1, policy_version 1378555 (0.0011) [2023-12-27 15:12:29,169][20381] Updated weights for policy 1, policy_version 1378565 (0.0011) [2023-12-27 15:12:29,181][20347] Updated weights for policy 0, policy_version 1379805 (0.0009) [2023-12-27 15:12:29,227][20381] Updated weights for policy 1, policy_version 1378575 (0.0010) [2023-12-27 15:12:29,245][20347] Updated weights for policy 0, policy_version 1379815 (0.0009) [2023-12-27 15:12:29,303][20347] Updated weights for policy 0, policy_version 1379825 (0.0010) [2023-12-27 15:12:29,950][20381] Updated weights for policy 1, policy_version 1378585 (0.0009) [2023-12-27 15:12:30,010][20381] Updated weights for policy 1, policy_version 1378595 (0.0007) [2023-12-27 15:12:30,013][20347] Updated weights for policy 0, policy_version 1379835 (0.0008) [2023-12-27 15:12:30,065][20381] Updated weights for policy 1, policy_version 1378605 (0.0008) [2023-12-27 15:12:30,067][20347] Updated weights for policy 0, policy_version 1379845 (0.0006) [2023-12-27 15:12:30,122][20347] Updated weights for policy 0, policy_version 1379855 (0.0007) [2023-12-27 15:12:30,722][20381] Updated weights for policy 1, policy_version 1378615 (0.0006) [2023-12-27 15:12:30,759][20347] Updated weights for policy 0, policy_version 1379865 (0.0007) [2023-12-27 15:12:30,768][20381] Updated weights for policy 1, policy_version 1378625 (0.0005) [2023-12-27 15:12:30,814][20347] Updated weights for policy 0, policy_version 1379875 (0.0006) [2023-12-27 15:12:30,826][20381] Updated weights for policy 1, policy_version 1378635 (0.0007) [2023-12-27 15:12:30,864][20347] Updated weights for policy 0, policy_version 1379885 (0.0006) [2023-12-27 15:12:30,908][20347] Updated weights for policy 0, policy_version 1379895 (0.0008) [2023-12-27 15:12:31,509][20381] Updated weights for policy 1, policy_version 1378645 (0.0010) [2023-12-27 15:12:31,573][20381] Updated weights for policy 1, policy_version 1378655 (0.0011) [2023-12-27 15:12:31,639][20381] Updated weights for policy 1, policy_version 1378665 (0.0011) [2023-12-27 15:12:31,664][20347] Updated weights for policy 0, policy_version 1379905 (0.0007) [2023-12-27 15:12:31,721][20347] Updated weights for policy 0, policy_version 1379915 (0.0007) [2023-12-27 15:12:31,788][20347] Updated weights for policy 0, policy_version 1379925 (0.0008) [2023-12-27 15:12:32,309][20381] Updated weights for policy 1, policy_version 1378675 (0.0011) [2023-12-27 15:12:32,362][20381] Updated weights for policy 1, policy_version 1378685 (0.0007) [2023-12-27 15:12:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 706297856. Throughput: 0: 10238.0, 1: 10221.0. Samples: 706313756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:32,363][19170] Avg episode reward: [(0, '970.700'), (1, '970.750')] [2023-12-27 15:12:32,428][20381] Updated weights for policy 1, policy_version 1378695 (0.0008) [2023-12-27 15:12:32,439][20347] Updated weights for policy 0, policy_version 1379935 (0.0008) [2023-12-27 15:12:32,496][20347] Updated weights for policy 0, policy_version 1379945 (0.0006) [2023-12-27 15:12:32,551][20347] Updated weights for policy 0, policy_version 1379955 (0.0007) [2023-12-27 15:12:33,094][20381] Updated weights for policy 1, policy_version 1378705 (0.0006) [2023-12-27 15:12:33,143][20347] Updated weights for policy 0, policy_version 1379965 (0.0006) [2023-12-27 15:12:33,150][20381] Updated weights for policy 1, policy_version 1378715 (0.0007) [2023-12-27 15:12:33,198][20381] Updated weights for policy 1, policy_version 1378725 (0.0010) [2023-12-27 15:12:33,202][20347] Updated weights for policy 0, policy_version 1379975 (0.0005) [2023-12-27 15:12:33,249][20381] Updated weights for policy 1, policy_version 1378735 (0.0010) [2023-12-27 15:12:33,258][20347] Updated weights for policy 0, policy_version 1379985 (0.0005) [2023-12-27 15:12:33,761][20347] Updated weights for policy 0, policy_version 1379995 (0.0005) [2023-12-27 15:12:33,817][20347] Updated weights for policy 0, policy_version 1380005 (0.0005) [2023-12-27 15:12:33,865][20347] Updated weights for policy 0, policy_version 1380015 (0.0005) [2023-12-27 15:12:33,949][20381] Updated weights for policy 1, policy_version 1378745 (0.0007) [2023-12-27 15:12:34,008][20381] Updated weights for policy 1, policy_version 1378755 (0.0008) [2023-12-27 15:12:34,062][20381] Updated weights for policy 1, policy_version 1378765 (0.0006) [2023-12-27 15:12:34,474][20347] Updated weights for policy 0, policy_version 1380025 (0.0009) [2023-12-27 15:12:34,533][20347] Updated weights for policy 0, policy_version 1380035 (0.0010) [2023-12-27 15:12:34,592][20347] Updated weights for policy 0, policy_version 1380045 (0.0009) [2023-12-27 15:12:34,654][20347] Updated weights for policy 0, policy_version 1380055 (0.0010) [2023-12-27 15:12:34,715][20381] Updated weights for policy 1, policy_version 1378775 (0.0006) [2023-12-27 15:12:34,770][20381] Updated weights for policy 1, policy_version 1378785 (0.0006) [2023-12-27 15:12:34,820][20381] Updated weights for policy 1, policy_version 1378795 (0.0005) [2023-12-27 15:12:35,244][20347] Updated weights for policy 0, policy_version 1380065 (0.0010) [2023-12-27 15:12:35,303][20347] Updated weights for policy 0, policy_version 1380075 (0.0010) [2023-12-27 15:12:35,357][20347] Updated weights for policy 0, policy_version 1380085 (0.0010) [2023-12-27 15:12:35,364][20381] Updated weights for policy 1, policy_version 1378805 (0.0006) [2023-12-27 15:12:35,418][20381] Updated weights for policy 1, policy_version 1378815 (0.0008) [2023-12-27 15:12:35,473][20381] Updated weights for policy 1, policy_version 1378825 (0.0008) [2023-12-27 15:12:35,963][20347] Updated weights for policy 0, policy_version 1380095 (0.0007) [2023-12-27 15:12:36,018][20347] Updated weights for policy 0, policy_version 1380105 (0.0008) [2023-12-27 15:12:36,074][20347] Updated weights for policy 0, policy_version 1380115 (0.0006) [2023-12-27 15:12:36,076][20381] Updated weights for policy 1, policy_version 1378835 (0.0007) [2023-12-27 15:12:36,137][20381] Updated weights for policy 1, policy_version 1378845 (0.0008) [2023-12-27 15:12:36,195][20381] Updated weights for policy 1, policy_version 1378855 (0.0008) [2023-12-27 15:12:36,764][20347] Updated weights for policy 0, policy_version 1380125 (0.0009) [2023-12-27 15:12:36,829][20347] Updated weights for policy 0, policy_version 1380135 (0.0008) [2023-12-27 15:12:36,886][20347] Updated weights for policy 0, policy_version 1380145 (0.0008) [2023-12-27 15:12:36,887][20381] Updated weights for policy 1, policy_version 1378865 (0.0008) [2023-12-27 15:12:36,930][20292] KL-divergence is very high: 528.0312 [2023-12-27 15:12:36,934][20381] Updated weights for policy 1, policy_version 1378875 (0.0006) [2023-12-27 15:12:36,940][20292] KL-divergence is very high: 503.1075 [2023-12-27 15:12:36,967][20292] KL-divergence is very high: 124.3515 [2023-12-27 15:12:36,982][20381] Updated weights for policy 1, policy_version 1378885 (0.0006) [2023-12-27 15:12:37,026][20381] Updated weights for policy 1, policy_version 1378895 (0.0006) [2023-12-27 15:12:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 706412544. Throughput: 0: 10348.1, 1: 10260.6. Samples: 706380148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:37,363][19170] Avg episode reward: [(0, '990.240'), (1, '970.780')] [2023-12-27 15:12:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001378896_353026048.pth... [2023-12-27 15:12:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001380152_353386496.pth... [2023-12-27 15:12:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001377712_352722944.pth [2023-12-27 15:12:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001378968_353083392.pth [2023-12-27 15:12:37,524][20347] Updated weights for policy 0, policy_version 1380155 (0.0006) [2023-12-27 15:12:37,587][20347] Updated weights for policy 0, policy_version 1380165 (0.0005) [2023-12-27 15:12:37,647][20347] Updated weights for policy 0, policy_version 1380175 (0.0005) [2023-12-27 15:12:37,716][20381] Updated weights for policy 1, policy_version 1378905 (0.0007) [2023-12-27 15:12:37,768][20381] Updated weights for policy 1, policy_version 1378915 (0.0007) [2023-12-27 15:12:37,813][20381] Updated weights for policy 1, policy_version 1378925 (0.0005) [2023-12-27 15:12:38,171][20347] Updated weights for policy 0, policy_version 1380185 (0.0006) [2023-12-27 15:12:38,217][20347] Updated weights for policy 0, policy_version 1380195 (0.0008) [2023-12-27 15:12:38,275][20347] Updated weights for policy 0, policy_version 1380205 (0.0006) [2023-12-27 15:12:38,335][20347] Updated weights for policy 0, policy_version 1380215 (0.0006) [2023-12-27 15:12:38,457][20381] Updated weights for policy 1, policy_version 1378935 (0.0007) [2023-12-27 15:12:38,512][20381] Updated weights for policy 1, policy_version 1378945 (0.0008) [2023-12-27 15:12:38,573][20381] Updated weights for policy 1, policy_version 1378955 (0.0008) [2023-12-27 15:12:38,967][20347] Updated weights for policy 0, policy_version 1380225 (0.0008) [2023-12-27 15:12:39,034][20347] Updated weights for policy 0, policy_version 1380235 (0.0007) [2023-12-27 15:12:39,092][20347] Updated weights for policy 0, policy_version 1380245 (0.0008) [2023-12-27 15:12:39,201][20381] Updated weights for policy 1, policy_version 1378965 (0.0008) [2023-12-27 15:12:39,264][20381] Updated weights for policy 1, policy_version 1378975 (0.0008) [2023-12-27 15:12:39,320][20381] Updated weights for policy 1, policy_version 1378985 (0.0006) [2023-12-27 15:12:39,805][20347] Updated weights for policy 0, policy_version 1380255 (0.0008) [2023-12-27 15:12:39,879][20347] Updated weights for policy 0, policy_version 1380265 (0.0008) [2023-12-27 15:12:39,941][20347] Updated weights for policy 0, policy_version 1380275 (0.0008) [2023-12-27 15:12:40,073][20381] Updated weights for policy 1, policy_version 1378995 (0.0008) [2023-12-27 15:12:40,124][20381] Updated weights for policy 1, policy_version 1379005 (0.0008) [2023-12-27 15:12:40,179][20381] Updated weights for policy 1, policy_version 1379015 (0.0008) [2023-12-27 15:12:40,627][20347] Updated weights for policy 0, policy_version 1380285 (0.0009) [2023-12-27 15:12:40,690][20347] Updated weights for policy 0, policy_version 1380295 (0.0010) [2023-12-27 15:12:40,750][20347] Updated weights for policy 0, policy_version 1380305 (0.0011) [2023-12-27 15:12:40,905][20381] Updated weights for policy 1, policy_version 1379025 (0.0008) [2023-12-27 15:12:40,952][20381] Updated weights for policy 1, policy_version 1379035 (0.0006) [2023-12-27 15:12:40,999][20381] Updated weights for policy 1, policy_version 1379045 (0.0009) [2023-12-27 15:12:41,059][20381] Updated weights for policy 1, policy_version 1379055 (0.0007) [2023-12-27 15:12:41,487][20347] Updated weights for policy 0, policy_version 1380315 (0.0007) [2023-12-27 15:12:41,541][20347] Updated weights for policy 0, policy_version 1380325 (0.0009) [2023-12-27 15:12:41,603][20347] Updated weights for policy 0, policy_version 1380335 (0.0009) [2023-12-27 15:12:41,917][20381] Updated weights for policy 1, policy_version 1379065 (0.0008) [2023-12-27 15:12:41,972][20381] Updated weights for policy 1, policy_version 1379075 (0.0007) [2023-12-27 15:12:42,031][20381] Updated weights for policy 1, policy_version 1379085 (0.0006) [2023-12-27 15:12:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 706510848. Throughput: 0: 10352.4, 1: 10265.7. Samples: 706505548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:42,363][19170] Avg episode reward: [(0, '990.270'), (1, '980.530')] [2023-12-27 15:12:42,391][20347] Updated weights for policy 0, policy_version 1380345 (0.0008) [2023-12-27 15:12:42,447][20347] Updated weights for policy 0, policy_version 1380355 (0.0005) [2023-12-27 15:12:42,517][20347] Updated weights for policy 0, policy_version 1380365 (0.0009) [2023-12-27 15:12:42,581][20347] Updated weights for policy 0, policy_version 1380375 (0.0010) [2023-12-27 15:12:42,683][20381] Updated weights for policy 1, policy_version 1379095 (0.0007) [2023-12-27 15:12:42,742][20381] Updated weights for policy 1, policy_version 1379105 (0.0007) [2023-12-27 15:12:42,798][20381] Updated weights for policy 1, policy_version 1379115 (0.0008) [2023-12-27 15:12:43,190][20347] Updated weights for policy 0, policy_version 1380385 (0.0006) [2023-12-27 15:12:43,255][20347] Updated weights for policy 0, policy_version 1380395 (0.0009) [2023-12-27 15:12:43,302][20347] Updated weights for policy 0, policy_version 1380405 (0.0010) [2023-12-27 15:12:43,440][20381] Updated weights for policy 1, policy_version 1379125 (0.0009) [2023-12-27 15:12:43,492][20381] Updated weights for policy 1, policy_version 1379135 (0.0011) [2023-12-27 15:12:43,543][20381] Updated weights for policy 1, policy_version 1379145 (0.0010) [2023-12-27 15:12:43,940][20347] Updated weights for policy 0, policy_version 1380415 (0.0007) [2023-12-27 15:12:44,002][20347] Updated weights for policy 0, policy_version 1380425 (0.0010) [2023-12-27 15:12:44,059][20347] Updated weights for policy 0, policy_version 1380435 (0.0010) [2023-12-27 15:12:44,183][20381] Updated weights for policy 1, policy_version 1379155 (0.0011) [2023-12-27 15:12:44,240][20381] Updated weights for policy 1, policy_version 1379165 (0.0010) [2023-12-27 15:12:44,299][20381] Updated weights for policy 1, policy_version 1379175 (0.0011) [2023-12-27 15:12:44,723][20347] Updated weights for policy 0, policy_version 1380445 (0.0010) [2023-12-27 15:12:44,786][20347] Updated weights for policy 0, policy_version 1380455 (0.0010) [2023-12-27 15:12:44,839][20347] Updated weights for policy 0, policy_version 1380465 (0.0010) [2023-12-27 15:12:45,023][20381] Updated weights for policy 1, policy_version 1379185 (0.0010) [2023-12-27 15:12:45,089][20381] Updated weights for policy 1, policy_version 1379195 (0.0007) [2023-12-27 15:12:45,152][20381] Updated weights for policy 1, policy_version 1379205 (0.0006) [2023-12-27 15:12:45,210][20381] Updated weights for policy 1, policy_version 1379215 (0.0007) [2023-12-27 15:12:45,530][20347] Updated weights for policy 0, policy_version 1380475 (0.0010) [2023-12-27 15:12:45,583][20347] Updated weights for policy 0, policy_version 1380485 (0.0010) [2023-12-27 15:12:45,631][20347] Updated weights for policy 0, policy_version 1380495 (0.0010) [2023-12-27 15:12:45,828][20381] Updated weights for policy 1, policy_version 1379225 (0.0007) [2023-12-27 15:12:45,886][20381] Updated weights for policy 1, policy_version 1379235 (0.0005) [2023-12-27 15:12:45,945][20381] Updated weights for policy 1, policy_version 1379245 (0.0005) [2023-12-27 15:12:46,310][20347] Updated weights for policy 0, policy_version 1380505 (0.0010) [2023-12-27 15:12:46,367][20347] Updated weights for policy 0, policy_version 1380515 (0.0006) [2023-12-27 15:12:46,416][20347] Updated weights for policy 0, policy_version 1380525 (0.0009) [2023-12-27 15:12:46,467][20347] Updated weights for policy 0, policy_version 1380535 (0.0006) [2023-12-27 15:12:46,518][20381] Updated weights for policy 1, policy_version 1379255 (0.0005) [2023-12-27 15:12:46,568][20381] Updated weights for policy 1, policy_version 1379265 (0.0008) [2023-12-27 15:12:46,619][20381] Updated weights for policy 1, policy_version 1379275 (0.0006) [2023-12-27 15:12:47,123][20347] Updated weights for policy 0, policy_version 1380545 (0.0010) [2023-12-27 15:12:47,175][20347] Updated weights for policy 0, policy_version 1380555 (0.0010) [2023-12-27 15:12:47,220][20381] Updated weights for policy 1, policy_version 1379285 (0.0006) [2023-12-27 15:12:47,233][20347] Updated weights for policy 0, policy_version 1380565 (0.0010) [2023-12-27 15:12:47,269][20381] Updated weights for policy 1, policy_version 1379295 (0.0006) [2023-12-27 15:12:47,322][20381] Updated weights for policy 1, policy_version 1379305 (0.0007) [2023-12-27 15:12:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 706625536. Throughput: 0: 10353.2, 1: 10336.4. Samples: 706629340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:47,363][19170] Avg episode reward: [(0, '990.270'), (1, '970.820')] [2023-12-27 15:12:47,911][20347] Updated weights for policy 0, policy_version 1380575 (0.0010) [2023-12-27 15:12:47,919][20381] Updated weights for policy 1, policy_version 1379315 (0.0006) [2023-12-27 15:12:47,969][20347] Updated weights for policy 0, policy_version 1380585 (0.0010) [2023-12-27 15:12:47,974][20381] Updated weights for policy 1, policy_version 1379325 (0.0006) [2023-12-27 15:12:48,022][20347] Updated weights for policy 0, policy_version 1380595 (0.0010) [2023-12-27 15:12:48,034][20381] Updated weights for policy 1, policy_version 1379335 (0.0005) [2023-12-27 15:12:48,664][20347] Updated weights for policy 0, policy_version 1380605 (0.0010) [2023-12-27 15:12:48,715][20381] Updated weights for policy 1, policy_version 1379345 (0.0007) [2023-12-27 15:12:48,725][20347] Updated weights for policy 0, policy_version 1380615 (0.0011) [2023-12-27 15:12:48,778][20381] Updated weights for policy 1, policy_version 1379355 (0.0011) [2023-12-27 15:12:48,791][20347] Updated weights for policy 0, policy_version 1380625 (0.0010) [2023-12-27 15:12:48,841][20381] Updated weights for policy 1, policy_version 1379365 (0.0011) [2023-12-27 15:12:48,900][20381] Updated weights for policy 1, policy_version 1379375 (0.0011) [2023-12-27 15:12:49,476][20347] Updated weights for policy 0, policy_version 1380635 (0.0009) [2023-12-27 15:12:49,534][20347] Updated weights for policy 0, policy_version 1380645 (0.0006) [2023-12-27 15:12:49,536][20381] Updated weights for policy 1, policy_version 1379385 (0.0008) [2023-12-27 15:12:49,589][20381] Updated weights for policy 1, policy_version 1379395 (0.0007) [2023-12-27 15:12:49,594][20347] Updated weights for policy 0, policy_version 1380655 (0.0006) [2023-12-27 15:12:49,640][20381] Updated weights for policy 1, policy_version 1379405 (0.0008) [2023-12-27 15:12:50,237][20347] Updated weights for policy 0, policy_version 1380665 (0.0007) [2023-12-27 15:12:50,298][20347] Updated weights for policy 0, policy_version 1380675 (0.0006) [2023-12-27 15:12:50,357][20347] Updated weights for policy 0, policy_version 1380685 (0.0007) [2023-12-27 15:12:50,389][20381] Updated weights for policy 1, policy_version 1379415 (0.0010) [2023-12-27 15:12:50,409][20347] Updated weights for policy 0, policy_version 1380695 (0.0005) [2023-12-27 15:12:50,441][20381] Updated weights for policy 1, policy_version 1379425 (0.0010) [2023-12-27 15:12:50,496][20381] Updated weights for policy 1, policy_version 1379435 (0.0010) [2023-12-27 15:12:51,117][20347] Updated weights for policy 0, policy_version 1380705 (0.0008) [2023-12-27 15:12:51,186][20347] Updated weights for policy 0, policy_version 1380715 (0.0008) [2023-12-27 15:12:51,200][20381] Updated weights for policy 1, policy_version 1379445 (0.0011) [2023-12-27 15:12:51,237][20347] Updated weights for policy 0, policy_version 1380725 (0.0007) [2023-12-27 15:12:51,282][20381] Updated weights for policy 1, policy_version 1379455 (0.0011) [2023-12-27 15:12:51,339][20381] Updated weights for policy 1, policy_version 1379465 (0.0010) [2023-12-27 15:12:52,004][20347] Updated weights for policy 0, policy_version 1380735 (0.0007) [2023-12-27 15:12:52,067][20347] Updated weights for policy 0, policy_version 1380745 (0.0007) [2023-12-27 15:12:52,069][20381] Updated weights for policy 1, policy_version 1379475 (0.0009) [2023-12-27 15:12:52,122][20381] Updated weights for policy 1, policy_version 1379485 (0.0008) [2023-12-27 15:12:52,124][20347] Updated weights for policy 0, policy_version 1380755 (0.0006) [2023-12-27 15:12:52,172][20292] KL-divergence is very high: 128.7700 [2023-12-27 15:12:52,172][20381] Updated weights for policy 1, policy_version 1379495 (0.0008) [2023-12-27 15:12:52,218][20292] KL-divergence is very high: 158.0554 [2023-12-27 15:12:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 706723840. Throughput: 0: 10319.6, 1: 10324.5. Samples: 706690644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:52,363][19170] Avg episode reward: [(0, '990.210'), (1, '952.360')] [2023-12-27 15:12:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001379504_353181696.pth... [2023-12-27 15:12:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001380760_353542144.pth... [2023-12-27 15:12:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001378288_352870400.pth [2023-12-27 15:12:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001379544_353230848.pth [2023-12-27 15:12:52,751][20347] Updated weights for policy 0, policy_version 1380765 (0.0007) [2023-12-27 15:12:52,808][20347] Updated weights for policy 0, policy_version 1380775 (0.0006) [2023-12-27 15:12:52,872][20347] Updated weights for policy 0, policy_version 1380785 (0.0007) [2023-12-27 15:12:52,945][20381] Updated weights for policy 1, policy_version 1379505 (0.0006) [2023-12-27 15:12:52,997][20381] Updated weights for policy 1, policy_version 1379515 (0.0008) [2023-12-27 15:12:53,048][20381] Updated weights for policy 1, policy_version 1379525 (0.0008) [2023-12-27 15:12:53,111][20381] Updated weights for policy 1, policy_version 1379535 (0.0009) [2023-12-27 15:12:53,548][20347] Updated weights for policy 0, policy_version 1380795 (0.0006) [2023-12-27 15:12:53,596][20347] Updated weights for policy 0, policy_version 1380805 (0.0007) [2023-12-27 15:12:53,650][20347] Updated weights for policy 0, policy_version 1380815 (0.0006) [2023-12-27 15:12:53,779][20381] Updated weights for policy 1, policy_version 1379545 (0.0008) [2023-12-27 15:12:53,834][20381] Updated weights for policy 1, policy_version 1379555 (0.0006) [2023-12-27 15:12:53,891][20381] Updated weights for policy 1, policy_version 1379565 (0.0008) [2023-12-27 15:12:54,255][20347] Updated weights for policy 0, policy_version 1380825 (0.0005) [2023-12-27 15:12:54,304][20347] Updated weights for policy 0, policy_version 1380835 (0.0007) [2023-12-27 15:12:54,355][20347] Updated weights for policy 0, policy_version 1380845 (0.0007) [2023-12-27 15:12:54,408][20347] Updated weights for policy 0, policy_version 1380855 (0.0008) [2023-12-27 15:12:54,502][20381] Updated weights for policy 1, policy_version 1379575 (0.0006) [2023-12-27 15:12:54,548][20381] Updated weights for policy 1, policy_version 1379585 (0.0005) [2023-12-27 15:12:54,596][20381] Updated weights for policy 1, policy_version 1379595 (0.0008) [2023-12-27 15:12:55,016][20347] Updated weights for policy 0, policy_version 1380865 (0.0006) [2023-12-27 15:12:55,075][20347] Updated weights for policy 0, policy_version 1380875 (0.0006) [2023-12-27 15:12:55,127][20347] Updated weights for policy 0, policy_version 1380885 (0.0007) [2023-12-27 15:12:55,302][20381] Updated weights for policy 1, policy_version 1379605 (0.0008) [2023-12-27 15:12:55,353][20381] Updated weights for policy 1, policy_version 1379615 (0.0006) [2023-12-27 15:12:55,406][20381] Updated weights for policy 1, policy_version 1379625 (0.0005) [2023-12-27 15:12:55,759][20347] Updated weights for policy 0, policy_version 1380895 (0.0007) [2023-12-27 15:12:55,813][20347] Updated weights for policy 0, policy_version 1380905 (0.0005) [2023-12-27 15:12:55,864][20347] Updated weights for policy 0, policy_version 1380915 (0.0005) [2023-12-27 15:12:55,984][20381] Updated weights for policy 1, policy_version 1379635 (0.0008) [2023-12-27 15:12:56,035][20381] Updated weights for policy 1, policy_version 1379645 (0.0010) [2023-12-27 15:12:56,088][20381] Updated weights for policy 1, policy_version 1379655 (0.0007) [2023-12-27 15:12:56,465][20347] Updated weights for policy 0, policy_version 1380925 (0.0005) [2023-12-27 15:12:56,531][20347] Updated weights for policy 0, policy_version 1380935 (0.0007) [2023-12-27 15:12:56,588][20347] Updated weights for policy 0, policy_version 1380945 (0.0008) [2023-12-27 15:12:56,700][20381] Updated weights for policy 1, policy_version 1379665 (0.0007) [2023-12-27 15:12:56,748][20381] Updated weights for policy 1, policy_version 1379675 (0.0010) [2023-12-27 15:12:56,799][20381] Updated weights for policy 1, policy_version 1379685 (0.0010) [2023-12-27 15:12:56,850][20381] Updated weights for policy 1, policy_version 1379695 (0.0010) [2023-12-27 15:12:57,155][20347] Updated weights for policy 0, policy_version 1380955 (0.0005) [2023-12-27 15:12:57,214][20347] Updated weights for policy 0, policy_version 1380965 (0.0007) [2023-12-27 15:12:57,259][20347] Updated weights for policy 0, policy_version 1380975 (0.0006) [2023-12-27 15:12:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 706830336. Throughput: 0: 10291.0, 1: 10261.8. Samples: 706815360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:12:57,363][19170] Avg episode reward: [(0, '970.760'), (1, '990.270')] [2023-12-27 15:12:57,438][20381] Updated weights for policy 1, policy_version 1379705 (0.0006) [2023-12-27 15:12:57,503][20381] Updated weights for policy 1, policy_version 1379715 (0.0005) [2023-12-27 15:12:57,556][20381] Updated weights for policy 1, policy_version 1379725 (0.0010) [2023-12-27 15:12:57,820][20347] Updated weights for policy 0, policy_version 1380985 (0.0007) [2023-12-27 15:12:57,870][20347] Updated weights for policy 0, policy_version 1380995 (0.0005) [2023-12-27 15:12:57,925][20347] Updated weights for policy 0, policy_version 1381005 (0.0005) [2023-12-27 15:12:57,970][20347] Updated weights for policy 0, policy_version 1381015 (0.0005) [2023-12-27 15:12:58,115][20381] Updated weights for policy 1, policy_version 1379735 (0.0008) [2023-12-27 15:12:58,165][20381] Updated weights for policy 1, policy_version 1379745 (0.0007) [2023-12-27 15:12:58,229][20381] Updated weights for policy 1, policy_version 1379755 (0.0007) [2023-12-27 15:12:58,684][20347] Updated weights for policy 0, policy_version 1381025 (0.0007) [2023-12-27 15:12:58,743][20347] Updated weights for policy 0, policy_version 1381035 (0.0007) [2023-12-27 15:12:58,821][20347] Updated weights for policy 0, policy_version 1381045 (0.0008) [2023-12-27 15:12:58,982][20381] Updated weights for policy 1, policy_version 1379765 (0.0007) [2023-12-27 15:12:59,039][20381] Updated weights for policy 1, policy_version 1379775 (0.0007) [2023-12-27 15:12:59,102][20381] Updated weights for policy 1, policy_version 1379785 (0.0007) [2023-12-27 15:12:59,642][20347] Updated weights for policy 0, policy_version 1381055 (0.0008) [2023-12-27 15:12:59,694][20347] Updated weights for policy 0, policy_version 1381065 (0.0008) [2023-12-27 15:12:59,752][20347] Updated weights for policy 0, policy_version 1381075 (0.0007) [2023-12-27 15:12:59,812][20381] Updated weights for policy 1, policy_version 1379795 (0.0008) [2023-12-27 15:12:59,879][20381] Updated weights for policy 1, policy_version 1379805 (0.0007) [2023-12-27 15:12:59,950][20381] Updated weights for policy 1, policy_version 1379815 (0.0008) [2023-12-27 15:13:00,431][20347] Updated weights for policy 0, policy_version 1381085 (0.0006) [2023-12-27 15:13:00,481][20347] Updated weights for policy 0, policy_version 1381095 (0.0005) [2023-12-27 15:13:00,537][20347] Updated weights for policy 0, policy_version 1381105 (0.0008) [2023-12-27 15:13:00,673][20381] Updated weights for policy 1, policy_version 1379825 (0.0008) [2023-12-27 15:13:00,725][20381] Updated weights for policy 1, policy_version 1379835 (0.0008) [2023-12-27 15:13:00,784][20381] Updated weights for policy 1, policy_version 1379845 (0.0008) [2023-12-27 15:13:00,845][20381] Updated weights for policy 1, policy_version 1379855 (0.0008) [2023-12-27 15:13:01,143][20347] Updated weights for policy 0, policy_version 1381115 (0.0007) [2023-12-27 15:13:01,201][20347] Updated weights for policy 0, policy_version 1381125 (0.0008) [2023-12-27 15:13:01,256][20347] Updated weights for policy 0, policy_version 1381135 (0.0008) [2023-12-27 15:13:01,586][20381] Updated weights for policy 1, policy_version 1379865 (0.0008) [2023-12-27 15:13:01,649][20381] Updated weights for policy 1, policy_version 1379875 (0.0008) [2023-12-27 15:13:01,717][20381] Updated weights for policy 1, policy_version 1379885 (0.0008) [2023-12-27 15:13:02,033][20347] Updated weights for policy 0, policy_version 1381145 (0.0008) [2023-12-27 15:13:02,082][20347] Updated weights for policy 0, policy_version 1381155 (0.0008) [2023-12-27 15:13:02,134][20347] Updated weights for policy 0, policy_version 1381165 (0.0007) [2023-12-27 15:13:02,186][20347] Updated weights for policy 0, policy_version 1381175 (0.0008) [2023-12-27 15:13:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 706928640. Throughput: 0: 10326.8, 1: 10304.6. Samples: 706938928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:02,363][19170] Avg episode reward: [(0, '970.880'), (1, '980.490')] [2023-12-27 15:13:02,413][20381] Updated weights for policy 1, policy_version 1379895 (0.0007) [2023-12-27 15:13:02,468][20381] Updated weights for policy 1, policy_version 1379905 (0.0008) [2023-12-27 15:13:02,527][20381] Updated weights for policy 1, policy_version 1379915 (0.0008) [2023-12-27 15:13:02,870][20347] Updated weights for policy 0, policy_version 1381185 (0.0008) [2023-12-27 15:13:02,927][20347] Updated weights for policy 0, policy_version 1381195 (0.0007) [2023-12-27 15:13:02,984][20347] Updated weights for policy 0, policy_version 1381205 (0.0008) [2023-12-27 15:13:03,208][20381] Updated weights for policy 1, policy_version 1379925 (0.0007) [2023-12-27 15:13:03,260][20381] Updated weights for policy 1, policy_version 1379935 (0.0008) [2023-12-27 15:13:03,322][20381] Updated weights for policy 1, policy_version 1379945 (0.0008) [2023-12-27 15:13:03,667][20347] Updated weights for policy 0, policy_version 1381215 (0.0008) [2023-12-27 15:13:03,729][20347] Updated weights for policy 0, policy_version 1381225 (0.0007) [2023-12-27 15:13:03,788][20347] Updated weights for policy 0, policy_version 1381235 (0.0008) [2023-12-27 15:13:03,972][20381] Updated weights for policy 1, policy_version 1379955 (0.0008) [2023-12-27 15:13:04,022][20381] Updated weights for policy 1, policy_version 1379965 (0.0008) [2023-12-27 15:13:04,077][20381] Updated weights for policy 1, policy_version 1379975 (0.0008) [2023-12-27 15:13:04,537][20347] Updated weights for policy 0, policy_version 1381245 (0.0008) [2023-12-27 15:13:04,586][20347] Updated weights for policy 0, policy_version 1381255 (0.0008) [2023-12-27 15:13:04,641][20347] Updated weights for policy 0, policy_version 1381265 (0.0008) [2023-12-27 15:13:04,742][20381] Updated weights for policy 1, policy_version 1379985 (0.0008) [2023-12-27 15:13:04,798][20381] Updated weights for policy 1, policy_version 1379995 (0.0007) [2023-12-27 15:13:04,856][20381] Updated weights for policy 1, policy_version 1380005 (0.0008) [2023-12-27 15:13:04,914][20381] Updated weights for policy 1, policy_version 1380015 (0.0008) [2023-12-27 15:13:05,357][20347] Updated weights for policy 0, policy_version 1381275 (0.0007) [2023-12-27 15:13:05,422][20347] Updated weights for policy 0, policy_version 1381285 (0.0007) [2023-12-27 15:13:05,477][20347] Updated weights for policy 0, policy_version 1381295 (0.0008) [2023-12-27 15:13:05,549][20381] Updated weights for policy 1, policy_version 1380025 (0.0006) [2023-12-27 15:13:05,607][20381] Updated weights for policy 1, policy_version 1380035 (0.0005) [2023-12-27 15:13:05,664][20381] Updated weights for policy 1, policy_version 1380045 (0.0006) [2023-12-27 15:13:06,174][20347] Updated weights for policy 0, policy_version 1381305 (0.0007) [2023-12-27 15:13:06,235][20347] Updated weights for policy 0, policy_version 1381315 (0.0007) [2023-12-27 15:13:06,283][20381] Updated weights for policy 1, policy_version 1380055 (0.0011) [2023-12-27 15:13:06,295][20347] Updated weights for policy 0, policy_version 1381325 (0.0008) [2023-12-27 15:13:06,344][20381] Updated weights for policy 1, policy_version 1380065 (0.0007) [2023-12-27 15:13:06,354][20347] Updated weights for policy 0, policy_version 1381335 (0.0007) [2023-12-27 15:13:06,410][20381] Updated weights for policy 1, policy_version 1380075 (0.0011) [2023-12-27 15:13:07,063][20381] Updated weights for policy 1, policy_version 1380085 (0.0010) [2023-12-27 15:13:07,077][20347] Updated weights for policy 0, policy_version 1381345 (0.0007) [2023-12-27 15:13:07,121][20381] Updated weights for policy 1, policy_version 1380095 (0.0005) [2023-12-27 15:13:07,140][20347] Updated weights for policy 0, policy_version 1381355 (0.0008) [2023-12-27 15:13:07,178][20381] Updated weights for policy 1, policy_version 1380105 (0.0005) [2023-12-27 15:13:07,197][20347] Updated weights for policy 0, policy_version 1381365 (0.0007) [2023-12-27 15:13:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 707035136. Throughput: 0: 10308.7, 1: 10320.6. Samples: 707000100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:07,363][19170] Avg episode reward: [(0, '980.630'), (1, '970.710')] [2023-12-27 15:13:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001380112_353337344.pth... [2023-12-27 15:13:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001381368_353697792.pth... [2023-12-27 15:13:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001378896_353026048.pth [2023-12-27 15:13:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001380152_353386496.pth [2023-12-27 15:13:07,815][20381] Updated weights for policy 1, policy_version 1380115 (0.0005) [2023-12-27 15:13:07,870][20381] Updated weights for policy 1, policy_version 1380125 (0.0005) [2023-12-27 15:13:07,908][20347] Updated weights for policy 0, policy_version 1381375 (0.0006) [2023-12-27 15:13:07,917][20381] Updated weights for policy 1, policy_version 1380135 (0.0006) [2023-12-27 15:13:07,961][20347] Updated weights for policy 0, policy_version 1381385 (0.0007) [2023-12-27 15:13:08,012][20347] Updated weights for policy 0, policy_version 1381395 (0.0007) [2023-12-27 15:13:08,564][20381] Updated weights for policy 1, policy_version 1380145 (0.0008) [2023-12-27 15:13:08,634][20381] Updated weights for policy 1, policy_version 1380155 (0.0008) [2023-12-27 15:13:08,689][20381] Updated weights for policy 1, policy_version 1380165 (0.0007) [2023-12-27 15:13:08,737][20347] Updated weights for policy 0, policy_version 1381405 (0.0007) [2023-12-27 15:13:08,740][20381] Updated weights for policy 1, policy_version 1380175 (0.0008) [2023-12-27 15:13:08,798][20347] Updated weights for policy 0, policy_version 1381415 (0.0010) [2023-12-27 15:13:08,853][20347] Updated weights for policy 0, policy_version 1381425 (0.0010) [2023-12-27 15:13:09,456][20381] Updated weights for policy 1, policy_version 1380185 (0.0008) [2023-12-27 15:13:09,508][20381] Updated weights for policy 1, policy_version 1380195 (0.0008) [2023-12-27 15:13:09,555][20347] Updated weights for policy 0, policy_version 1381435 (0.0008) [2023-12-27 15:13:09,568][20381] Updated weights for policy 1, policy_version 1380205 (0.0010) [2023-12-27 15:13:09,607][20347] Updated weights for policy 0, policy_version 1381445 (0.0008) [2023-12-27 15:13:09,665][20347] Updated weights for policy 0, policy_version 1381455 (0.0007) [2023-12-27 15:13:10,326][20381] Updated weights for policy 1, policy_version 1380215 (0.0011) [2023-12-27 15:13:10,375][20347] Updated weights for policy 0, policy_version 1381465 (0.0007) [2023-12-27 15:13:10,389][20381] Updated weights for policy 1, policy_version 1380225 (0.0011) [2023-12-27 15:13:10,428][20347] Updated weights for policy 0, policy_version 1381475 (0.0011) [2023-12-27 15:13:10,445][20381] Updated weights for policy 1, policy_version 1380235 (0.0011) [2023-12-27 15:13:10,483][20347] Updated weights for policy 0, policy_version 1381485 (0.0010) [2023-12-27 15:13:10,538][20347] Updated weights for policy 0, policy_version 1381495 (0.0010) [2023-12-27 15:13:11,111][20381] Updated weights for policy 1, policy_version 1380245 (0.0011) [2023-12-27 15:13:11,177][20381] Updated weights for policy 1, policy_version 1380255 (0.0009) [2023-12-27 15:13:11,178][20347] Updated weights for policy 0, policy_version 1381505 (0.0010) [2023-12-27 15:13:11,235][20347] Updated weights for policy 0, policy_version 1381515 (0.0006) [2023-12-27 15:13:11,246][20381] Updated weights for policy 1, policy_version 1380265 (0.0007) [2023-12-27 15:13:11,304][20347] Updated weights for policy 0, policy_version 1381525 (0.0007) [2023-12-27 15:13:11,967][20381] Updated weights for policy 1, policy_version 1380275 (0.0008) [2023-12-27 15:13:12,025][20347] Updated weights for policy 0, policy_version 1381535 (0.0007) [2023-12-27 15:13:12,031][20381] Updated weights for policy 1, policy_version 1380285 (0.0008) [2023-12-27 15:13:12,086][20347] Updated weights for policy 0, policy_version 1381545 (0.0006) [2023-12-27 15:13:12,088][20381] Updated weights for policy 1, policy_version 1380295 (0.0007) [2023-12-27 15:13:12,143][20347] Updated weights for policy 0, policy_version 1381555 (0.0006) [2023-12-27 15:13:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 707133440. Throughput: 0: 10334.6, 1: 10379.0. Samples: 707121576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.320')] [2023-12-27 15:13:12,731][20381] Updated weights for policy 1, policy_version 1380305 (0.0008) [2023-12-27 15:13:12,789][20381] Updated weights for policy 1, policy_version 1380315 (0.0007) [2023-12-27 15:13:12,848][20381] Updated weights for policy 1, policy_version 1380325 (0.0005) [2023-12-27 15:13:12,870][20347] Updated weights for policy 0, policy_version 1381565 (0.0008) [2023-12-27 15:13:12,907][20381] Updated weights for policy 1, policy_version 1380335 (0.0005) [2023-12-27 15:13:12,923][20347] Updated weights for policy 0, policy_version 1381575 (0.0008) [2023-12-27 15:13:12,978][20347] Updated weights for policy 0, policy_version 1381585 (0.0007) [2023-12-27 15:13:13,503][20381] Updated weights for policy 1, policy_version 1380345 (0.0005) [2023-12-27 15:13:13,556][20381] Updated weights for policy 1, policy_version 1380355 (0.0005) [2023-12-27 15:13:13,610][20381] Updated weights for policy 1, policy_version 1380365 (0.0005) [2023-12-27 15:13:13,710][20347] Updated weights for policy 0, policy_version 1381595 (0.0007) [2023-12-27 15:13:13,762][20347] Updated weights for policy 0, policy_version 1381605 (0.0009) [2023-12-27 15:13:13,811][20347] Updated weights for policy 0, policy_version 1381615 (0.0009) [2023-12-27 15:13:14,250][20381] Updated weights for policy 1, policy_version 1380375 (0.0008) [2023-12-27 15:13:14,303][20381] Updated weights for policy 1, policy_version 1380385 (0.0010) [2023-12-27 15:13:14,352][20381] Updated weights for policy 1, policy_version 1380395 (0.0010) [2023-12-27 15:13:14,521][20347] Updated weights for policy 0, policy_version 1381625 (0.0007) [2023-12-27 15:13:14,577][20347] Updated weights for policy 0, policy_version 1381635 (0.0008) [2023-12-27 15:13:14,633][20347] Updated weights for policy 0, policy_version 1381645 (0.0007) [2023-12-27 15:13:14,688][20347] Updated weights for policy 0, policy_version 1381655 (0.0007) [2023-12-27 15:13:15,039][20381] Updated weights for policy 1, policy_version 1380405 (0.0010) [2023-12-27 15:13:15,098][20381] Updated weights for policy 1, policy_version 1380415 (0.0006) [2023-12-27 15:13:15,151][20381] Updated weights for policy 1, policy_version 1380425 (0.0011) [2023-12-27 15:13:15,456][20347] Updated weights for policy 0, policy_version 1381665 (0.0007) [2023-12-27 15:13:15,516][20347] Updated weights for policy 0, policy_version 1381675 (0.0007) [2023-12-27 15:13:15,578][20347] Updated weights for policy 0, policy_version 1381685 (0.0007) [2023-12-27 15:13:15,825][20381] Updated weights for policy 1, policy_version 1380435 (0.0011) [2023-12-27 15:13:15,872][20381] Updated weights for policy 1, policy_version 1380445 (0.0010) [2023-12-27 15:13:15,917][20381] Updated weights for policy 1, policy_version 1380455 (0.0010) [2023-12-27 15:13:16,262][20347] Updated weights for policy 0, policy_version 1381695 (0.0008) [2023-12-27 15:13:16,309][20347] Updated weights for policy 0, policy_version 1381705 (0.0007) [2023-12-27 15:13:16,358][20347] Updated weights for policy 0, policy_version 1381715 (0.0007) [2023-12-27 15:13:16,617][20381] Updated weights for policy 1, policy_version 1380465 (0.0010) [2023-12-27 15:13:16,662][20381] Updated weights for policy 1, policy_version 1380475 (0.0010) [2023-12-27 15:13:16,716][20381] Updated weights for policy 1, policy_version 1380485 (0.0010) [2023-12-27 15:13:16,778][20381] Updated weights for policy 1, policy_version 1380495 (0.0011) [2023-12-27 15:13:17,063][20347] Updated weights for policy 0, policy_version 1381725 (0.0006) [2023-12-27 15:13:17,116][20347] Updated weights for policy 0, policy_version 1381735 (0.0007) [2023-12-27 15:13:17,167][20347] Updated weights for policy 0, policy_version 1381745 (0.0007) [2023-12-27 15:13:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 707231744. Throughput: 0: 10280.1, 1: 10343.0. Samples: 707241792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:17,363][19170] Avg episode reward: [(0, '990.220'), (1, '951.350')] [2023-12-27 15:13:17,464][20381] Updated weights for policy 1, policy_version 1380505 (0.0011) [2023-12-27 15:13:17,529][20381] Updated weights for policy 1, policy_version 1380515 (0.0011) [2023-12-27 15:13:17,577][20381] Updated weights for policy 1, policy_version 1380525 (0.0010) [2023-12-27 15:13:17,840][20347] Updated weights for policy 0, policy_version 1381756 (0.0008) [2023-12-27 15:13:17,894][20347] Updated weights for policy 0, policy_version 1381766 (0.0007) [2023-12-27 15:13:17,937][20347] Updated weights for policy 0, policy_version 1381776 (0.0008) [2023-12-27 15:13:18,248][20381] Updated weights for policy 1, policy_version 1380535 (0.0010) [2023-12-27 15:13:18,294][20381] Updated weights for policy 1, policy_version 1380545 (0.0008) [2023-12-27 15:13:18,352][20381] Updated weights for policy 1, policy_version 1380555 (0.0007) [2023-12-27 15:13:18,543][20347] Updated weights for policy 0, policy_version 1381786 (0.0006) [2023-12-27 15:13:18,604][20347] Updated weights for policy 0, policy_version 1381796 (0.0007) [2023-12-27 15:13:18,670][20347] Updated weights for policy 0, policy_version 1381806 (0.0006) [2023-12-27 15:13:18,733][20347] Updated weights for policy 0, policy_version 1381816 (0.0007) [2023-12-27 15:13:19,020][20381] Updated weights for policy 1, policy_version 1380565 (0.0008) [2023-12-27 15:13:19,071][20381] Updated weights for policy 1, policy_version 1380575 (0.0005) [2023-12-27 15:13:19,120][20381] Updated weights for policy 1, policy_version 1380585 (0.0005) [2023-12-27 15:13:19,342][20347] Updated weights for policy 0, policy_version 1381826 (0.0007) [2023-12-27 15:13:19,407][20347] Updated weights for policy 0, policy_version 1381836 (0.0007) [2023-12-27 15:13:19,469][20347] Updated weights for policy 0, policy_version 1381846 (0.0008) [2023-12-27 15:13:19,823][20381] Updated weights for policy 1, policy_version 1380595 (0.0006) [2023-12-27 15:13:19,889][20381] Updated weights for policy 1, policy_version 1380605 (0.0007) [2023-12-27 15:13:19,957][20381] Updated weights for policy 1, policy_version 1380615 (0.0008) [2023-12-27 15:13:20,172][20347] Updated weights for policy 0, policy_version 1381856 (0.0008) [2023-12-27 15:13:20,233][20347] Updated weights for policy 0, policy_version 1381866 (0.0008) [2023-12-27 15:13:20,291][20347] Updated weights for policy 0, policy_version 1381876 (0.0008) [2023-12-27 15:13:20,641][20381] Updated weights for policy 1, policy_version 1380625 (0.0007) [2023-12-27 15:13:20,688][20381] Updated weights for policy 1, policy_version 1380635 (0.0007) [2023-12-27 15:13:20,741][20381] Updated weights for policy 1, policy_version 1380645 (0.0008) [2023-12-27 15:13:20,791][20381] Updated weights for policy 1, policy_version 1380655 (0.0008) [2023-12-27 15:13:21,038][20347] Updated weights for policy 0, policy_version 1381886 (0.0007) [2023-12-27 15:13:21,100][20347] Updated weights for policy 0, policy_version 1381896 (0.0007) [2023-12-27 15:13:21,166][20347] Updated weights for policy 0, policy_version 1381906 (0.0007) [2023-12-27 15:13:21,558][20381] Updated weights for policy 1, policy_version 1380665 (0.0008) [2023-12-27 15:13:21,619][20381] Updated weights for policy 1, policy_version 1380675 (0.0008) [2023-12-27 15:13:21,682][20381] Updated weights for policy 1, policy_version 1380685 (0.0009) [2023-12-27 15:13:21,905][20347] Updated weights for policy 0, policy_version 1381916 (0.0008) [2023-12-27 15:13:21,964][20347] Updated weights for policy 0, policy_version 1381926 (0.0008) [2023-12-27 15:13:22,020][20347] Updated weights for policy 0, policy_version 1381936 (0.0008) [2023-12-27 15:13:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 707330048. Throughput: 0: 10219.8, 1: 10301.5. Samples: 707303604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:22,363][19170] Avg episode reward: [(0, '941.490'), (1, '990.250')] [2023-12-27 15:13:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001381944_353845248.pth... [2023-12-27 15:13:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001380760_353542144.pth [2023-12-27 15:13:22,397][20381] Updated weights for policy 1, policy_version 1380695 (0.0007) [2023-12-27 15:13:22,454][20381] Updated weights for policy 1, policy_version 1380705 (0.0008) [2023-12-27 15:13:22,511][20381] Updated weights for policy 1, policy_version 1380715 (0.0006) [2023-12-27 15:13:22,543][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001380720_353492992.pth... [2023-12-27 15:13:22,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001379504_353181696.pth [2023-12-27 15:13:22,736][20347] Updated weights for policy 0, policy_version 1381946 (0.0007) [2023-12-27 15:13:22,793][20347] Updated weights for policy 0, policy_version 1381956 (0.0006) [2023-12-27 15:13:22,859][20347] Updated weights for policy 0, policy_version 1381966 (0.0008) [2023-12-27 15:13:22,922][20347] Updated weights for policy 0, policy_version 1381976 (0.0008) [2023-12-27 15:13:23,227][20381] Updated weights for policy 1, policy_version 1380725 (0.0007) [2023-12-27 15:13:23,275][20381] Updated weights for policy 1, policy_version 1380735 (0.0006) [2023-12-27 15:13:23,327][20381] Updated weights for policy 1, policy_version 1380745 (0.0009) [2023-12-27 15:13:23,510][20347] Updated weights for policy 0, policy_version 1381986 (0.0011) [2023-12-27 15:13:23,568][20347] Updated weights for policy 0, policy_version 1381996 (0.0010) [2023-12-27 15:13:23,627][20347] Updated weights for policy 0, policy_version 1382006 (0.0010) [2023-12-27 15:13:24,071][20381] Updated weights for policy 1, policy_version 1380755 (0.0009) [2023-12-27 15:13:24,123][20381] Updated weights for policy 1, policy_version 1380765 (0.0007) [2023-12-27 15:13:24,178][20381] Updated weights for policy 1, policy_version 1380775 (0.0007) [2023-12-27 15:13:24,208][20347] Updated weights for policy 0, policy_version 1382016 (0.0010) [2023-12-27 15:13:24,266][20347] Updated weights for policy 0, policy_version 1382026 (0.0010) [2023-12-27 15:13:24,320][20347] Updated weights for policy 0, policy_version 1382036 (0.0010) [2023-12-27 15:13:24,874][20347] Updated weights for policy 0, policy_version 1382046 (0.0007) [2023-12-27 15:13:24,932][20347] Updated weights for policy 0, policy_version 1382056 (0.0006) [2023-12-27 15:13:24,936][20381] Updated weights for policy 1, policy_version 1380785 (0.0007) [2023-12-27 15:13:24,985][20347] Updated weights for policy 0, policy_version 1382066 (0.0007) [2023-12-27 15:13:25,003][20381] Updated weights for policy 1, policy_version 1380795 (0.0006) [2023-12-27 15:13:25,053][20381] Updated weights for policy 1, policy_version 1380805 (0.0007) [2023-12-27 15:13:25,099][20381] Updated weights for policy 1, policy_version 1380815 (0.0008) [2023-12-27 15:13:25,611][20347] Updated weights for policy 0, policy_version 1382076 (0.0007) [2023-12-27 15:13:25,677][20347] Updated weights for policy 0, policy_version 1382086 (0.0008) [2023-12-27 15:13:25,743][20347] Updated weights for policy 0, policy_version 1382096 (0.0007) [2023-12-27 15:13:25,782][20381] Updated weights for policy 1, policy_version 1380825 (0.0006) [2023-12-27 15:13:25,834][20381] Updated weights for policy 1, policy_version 1380835 (0.0008) [2023-12-27 15:13:25,887][20381] Updated weights for policy 1, policy_version 1380845 (0.0009) [2023-12-27 15:13:26,314][20347] Updated weights for policy 0, policy_version 1382106 (0.0008) [2023-12-27 15:13:26,361][20347] Updated weights for policy 0, policy_version 1382116 (0.0007) [2023-12-27 15:13:26,412][20347] Updated weights for policy 0, policy_version 1382126 (0.0007) [2023-12-27 15:13:26,472][20347] Updated weights for policy 0, policy_version 1382136 (0.0007) [2023-12-27 15:13:26,630][20381] Updated weights for policy 1, policy_version 1380855 (0.0009) [2023-12-27 15:13:26,678][20381] Updated weights for policy 1, policy_version 1380865 (0.0007) [2023-12-27 15:13:26,730][20381] Updated weights for policy 1, policy_version 1380875 (0.0007) [2023-12-27 15:13:27,015][20347] Updated weights for policy 0, policy_version 1382146 (0.0007) [2023-12-27 15:13:27,066][20347] Updated weights for policy 0, policy_version 1382156 (0.0008) [2023-12-27 15:13:27,124][20347] Updated weights for policy 0, policy_version 1382166 (0.0008) [2023-12-27 15:13:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 707436544. Throughput: 0: 10199.5, 1: 10222.7. Samples: 707424544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:27,363][19170] Avg episode reward: [(0, '951.270'), (1, '980.560')] [2023-12-27 15:13:27,410][20381] Updated weights for policy 1, policy_version 1380885 (0.0008) [2023-12-27 15:13:27,461][20381] Updated weights for policy 1, policy_version 1380895 (0.0007) [2023-12-27 15:13:27,509][20381] Updated weights for policy 1, policy_version 1380905 (0.0007) [2023-12-27 15:13:27,829][20347] Updated weights for policy 0, policy_version 1382176 (0.0008) [2023-12-27 15:13:27,880][20347] Updated weights for policy 0, policy_version 1382186 (0.0007) [2023-12-27 15:13:27,926][20347] Updated weights for policy 0, policy_version 1382196 (0.0007) [2023-12-27 15:13:28,221][20381] Updated weights for policy 1, policy_version 1380915 (0.0008) [2023-12-27 15:13:28,263][20381] Updated weights for policy 1, policy_version 1380925 (0.0008) [2023-12-27 15:13:28,307][20381] Updated weights for policy 1, policy_version 1380935 (0.0007) [2023-12-27 15:13:28,678][20347] Updated weights for policy 0, policy_version 1382206 (0.0008) [2023-12-27 15:13:28,727][20347] Updated weights for policy 0, policy_version 1382216 (0.0008) [2023-12-27 15:13:28,780][20347] Updated weights for policy 0, policy_version 1382226 (0.0008) [2023-12-27 15:13:28,973][20381] Updated weights for policy 1, policy_version 1380945 (0.0007) [2023-12-27 15:13:29,019][20381] Updated weights for policy 1, policy_version 1380955 (0.0008) [2023-12-27 15:13:29,064][20381] Updated weights for policy 1, policy_version 1380965 (0.0007) [2023-12-27 15:13:29,119][20381] Updated weights for policy 1, policy_version 1380975 (0.0007) [2023-12-27 15:13:29,514][20347] Updated weights for policy 0, policy_version 1382236 (0.0007) [2023-12-27 15:13:29,570][20347] Updated weights for policy 0, policy_version 1382246 (0.0008) [2023-12-27 15:13:29,593][20291] KL-divergence is very high: 187.7826 [2023-12-27 15:13:29,627][20347] Updated weights for policy 0, policy_version 1382256 (0.0007) [2023-12-27 15:13:29,638][20291] KL-divergence is very high: 253.7012 [2023-12-27 15:13:29,858][20381] Updated weights for policy 1, policy_version 1380985 (0.0007) [2023-12-27 15:13:29,918][20381] Updated weights for policy 1, policy_version 1380995 (0.0008) [2023-12-27 15:13:29,976][20381] Updated weights for policy 1, policy_version 1381005 (0.0008) [2023-12-27 15:13:30,332][20347] Updated weights for policy 0, policy_version 1382266 (0.0008) [2023-12-27 15:13:30,390][20347] Updated weights for policy 0, policy_version 1382276 (0.0008) [2023-12-27 15:13:30,445][20347] Updated weights for policy 0, policy_version 1382286 (0.0007) [2023-12-27 15:13:30,496][20347] Updated weights for policy 0, policy_version 1382296 (0.0007) [2023-12-27 15:13:30,619][20381] Updated weights for policy 1, policy_version 1381015 (0.0006) [2023-12-27 15:13:30,671][20381] Updated weights for policy 1, policy_version 1381025 (0.0007) [2023-12-27 15:13:30,724][20381] Updated weights for policy 1, policy_version 1381035 (0.0008) [2023-12-27 15:13:31,233][20347] Updated weights for policy 0, policy_version 1382306 (0.0006) [2023-12-27 15:13:31,292][20347] Updated weights for policy 0, policy_version 1382316 (0.0006) [2023-12-27 15:13:31,356][20347] Updated weights for policy 0, policy_version 1382326 (0.0007) [2023-12-27 15:13:31,515][20381] Updated weights for policy 1, policy_version 1381045 (0.0007) [2023-12-27 15:13:31,570][20381] Updated weights for policy 1, policy_version 1381055 (0.0008) [2023-12-27 15:13:31,626][20381] Updated weights for policy 1, policy_version 1381065 (0.0008) [2023-12-27 15:13:32,074][20347] Updated weights for policy 0, policy_version 1382336 (0.0008) [2023-12-27 15:13:32,139][20347] Updated weights for policy 0, policy_version 1382346 (0.0008) [2023-12-27 15:13:32,200][20347] Updated weights for policy 0, policy_version 1382356 (0.0007) [2023-12-27 15:13:32,298][20381] Updated weights for policy 1, policy_version 1381075 (0.0007) [2023-12-27 15:13:32,356][20381] Updated weights for policy 1, policy_version 1381085 (0.0006) [2023-12-27 15:13:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 707534848. Throughput: 0: 10211.2, 1: 10159.4. Samples: 707546016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:32,363][19170] Avg episode reward: [(0, '990.300'), (1, '951.340')] [2023-12-27 15:13:32,418][20381] Updated weights for policy 1, policy_version 1381095 (0.0008) [2023-12-27 15:13:32,881][20347] Updated weights for policy 0, policy_version 1382366 (0.0007) [2023-12-27 15:13:32,928][20347] Updated weights for policy 0, policy_version 1382376 (0.0008) [2023-12-27 15:13:32,971][20347] Updated weights for policy 0, policy_version 1382386 (0.0007) [2023-12-27 15:13:33,091][20381] Updated weights for policy 1, policy_version 1381105 (0.0007) [2023-12-27 15:13:33,148][20381] Updated weights for policy 1, policy_version 1381115 (0.0008) [2023-12-27 15:13:33,205][20381] Updated weights for policy 1, policy_version 1381125 (0.0008) [2023-12-27 15:13:33,254][20381] Updated weights for policy 1, policy_version 1381135 (0.0007) [2023-12-27 15:13:33,684][20347] Updated weights for policy 0, policy_version 1382396 (0.0007) [2023-12-27 15:13:33,732][20347] Updated weights for policy 0, policy_version 1382406 (0.0007) [2023-12-27 15:13:33,780][20347] Updated weights for policy 0, policy_version 1382416 (0.0008) [2023-12-27 15:13:33,943][20381] Updated weights for policy 1, policy_version 1381145 (0.0008) [2023-12-27 15:13:33,995][20381] Updated weights for policy 1, policy_version 1381155 (0.0008) [2023-12-27 15:13:34,044][20381] Updated weights for policy 1, policy_version 1381165 (0.0008) [2023-12-27 15:13:34,527][20347] Updated weights for policy 0, policy_version 1382426 (0.0007) [2023-12-27 15:13:34,587][20347] Updated weights for policy 0, policy_version 1382436 (0.0006) [2023-12-27 15:13:34,645][20347] Updated weights for policy 0, policy_version 1382446 (0.0008) [2023-12-27 15:13:34,703][20347] Updated weights for policy 0, policy_version 1382456 (0.0008) [2023-12-27 15:13:34,723][20381] Updated weights for policy 1, policy_version 1381175 (0.0006) [2023-12-27 15:13:34,780][20381] Updated weights for policy 1, policy_version 1381185 (0.0007) [2023-12-27 15:13:34,836][20381] Updated weights for policy 1, policy_version 1381195 (0.0007) [2023-12-27 15:13:35,402][20381] Updated weights for policy 1, policy_version 1381205 (0.0006) [2023-12-27 15:13:35,434][20347] Updated weights for policy 0, policy_version 1382466 (0.0008) [2023-12-27 15:13:35,451][20381] Updated weights for policy 1, policy_version 1381215 (0.0007) [2023-12-27 15:13:35,495][20347] Updated weights for policy 0, policy_version 1382476 (0.0006) [2023-12-27 15:13:35,512][20381] Updated weights for policy 1, policy_version 1381225 (0.0005) [2023-12-27 15:13:35,554][20347] Updated weights for policy 0, policy_version 1382486 (0.0007) [2023-12-27 15:13:36,211][20381] Updated weights for policy 1, policy_version 1381235 (0.0006) [2023-12-27 15:13:36,240][20347] Updated weights for policy 0, policy_version 1382496 (0.0007) [2023-12-27 15:13:36,268][20381] Updated weights for policy 1, policy_version 1381245 (0.0006) [2023-12-27 15:13:36,293][20347] Updated weights for policy 0, policy_version 1382506 (0.0005) [2023-12-27 15:13:36,325][20381] Updated weights for policy 1, policy_version 1381255 (0.0006) [2023-12-27 15:13:36,350][20347] Updated weights for policy 0, policy_version 1382516 (0.0005) [2023-12-27 15:13:36,947][20347] Updated weights for policy 0, policy_version 1382526 (0.0005) [2023-12-27 15:13:37,001][20347] Updated weights for policy 0, policy_version 1382536 (0.0005) [2023-12-27 15:13:37,053][20347] Updated weights for policy 0, policy_version 1382546 (0.0005) [2023-12-27 15:13:37,090][20381] Updated weights for policy 1, policy_version 1381265 (0.0007) [2023-12-27 15:13:37,140][20381] Updated weights for policy 1, policy_version 1381275 (0.0008) [2023-12-27 15:13:37,194][20381] Updated weights for policy 1, policy_version 1381285 (0.0005) [2023-12-27 15:13:37,253][20381] Updated weights for policy 1, policy_version 1381295 (0.0006) [2023-12-27 15:13:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20521.8). Total num frames: 707641344. Throughput: 0: 10187.6, 1: 10174.4. Samples: 707606932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:37,363][19170] Avg episode reward: [(0, '990.300'), (1, '931.880')] [2023-12-27 15:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001381296_353640448.pth... [2023-12-27 15:13:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001382552_354000896.pth... [2023-12-27 15:13:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001380112_353337344.pth [2023-12-27 15:13:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001381368_353697792.pth [2023-12-27 15:13:37,685][20347] Updated weights for policy 0, policy_version 1382556 (0.0006) [2023-12-27 15:13:37,736][20347] Updated weights for policy 0, policy_version 1382566 (0.0007) [2023-12-27 15:13:37,785][20347] Updated weights for policy 0, policy_version 1382576 (0.0007) [2023-12-27 15:13:37,916][20381] Updated weights for policy 1, policy_version 1381305 (0.0007) [2023-12-27 15:13:37,968][20381] Updated weights for policy 1, policy_version 1381315 (0.0007) [2023-12-27 15:13:38,026][20381] Updated weights for policy 1, policy_version 1381325 (0.0008) [2023-12-27 15:13:38,534][20347] Updated weights for policy 0, policy_version 1382586 (0.0007) [2023-12-27 15:13:38,596][20347] Updated weights for policy 0, policy_version 1382596 (0.0007) [2023-12-27 15:13:38,652][20347] Updated weights for policy 0, policy_version 1382606 (0.0005) [2023-12-27 15:13:38,704][20381] Updated weights for policy 1, policy_version 1381335 (0.0006) [2023-12-27 15:13:38,717][20347] Updated weights for policy 0, policy_version 1382616 (0.0005) [2023-12-27 15:13:38,767][20381] Updated weights for policy 1, policy_version 1381345 (0.0006) [2023-12-27 15:13:38,830][20381] Updated weights for policy 1, policy_version 1381355 (0.0006) [2023-12-27 15:13:39,367][20347] Updated weights for policy 0, policy_version 1382626 (0.0008) [2023-12-27 15:13:39,434][20347] Updated weights for policy 0, policy_version 1382636 (0.0008) [2023-12-27 15:13:39,491][20347] Updated weights for policy 0, policy_version 1382646 (0.0007) [2023-12-27 15:13:39,524][20381] Updated weights for policy 1, policy_version 1381365 (0.0007) [2023-12-27 15:13:39,589][20381] Updated weights for policy 1, policy_version 1381375 (0.0008) [2023-12-27 15:13:39,614][20292] KL-divergence is very high: 125.6370 [2023-12-27 15:13:39,652][20381] Updated weights for policy 1, policy_version 1381385 (0.0007) [2023-12-27 15:13:40,204][20347] Updated weights for policy 0, policy_version 1382656 (0.0007) [2023-12-27 15:13:40,262][20347] Updated weights for policy 0, policy_version 1382666 (0.0008) [2023-12-27 15:13:40,278][20381] Updated weights for policy 1, policy_version 1381395 (0.0007) [2023-12-27 15:13:40,319][20347] Updated weights for policy 0, policy_version 1382676 (0.0006) [2023-12-27 15:13:40,339][20381] Updated weights for policy 1, policy_version 1381405 (0.0007) [2023-12-27 15:13:40,393][20381] Updated weights for policy 1, policy_version 1381415 (0.0009) [2023-12-27 15:13:40,940][20347] Updated weights for policy 0, policy_version 1382686 (0.0006) [2023-12-27 15:13:40,999][20347] Updated weights for policy 0, policy_version 1382696 (0.0005) [2023-12-27 15:13:41,061][20347] Updated weights for policy 0, policy_version 1382706 (0.0008) [2023-12-27 15:13:41,167][20381] Updated weights for policy 1, policy_version 1381425 (0.0008) [2023-12-27 15:13:41,220][20381] Updated weights for policy 1, policy_version 1381435 (0.0008) [2023-12-27 15:13:41,286][20381] Updated weights for policy 1, policy_version 1381445 (0.0008) [2023-12-27 15:13:41,353][20381] Updated weights for policy 1, policy_version 1381455 (0.0008) [2023-12-27 15:13:41,772][20347] Updated weights for policy 0, policy_version 1382716 (0.0008) [2023-12-27 15:13:41,830][20347] Updated weights for policy 0, policy_version 1382726 (0.0007) [2023-12-27 15:13:41,892][20347] Updated weights for policy 0, policy_version 1382736 (0.0008) [2023-12-27 15:13:42,107][20381] Updated weights for policy 1, policy_version 1381465 (0.0007) [2023-12-27 15:13:42,160][20381] Updated weights for policy 1, policy_version 1381475 (0.0008) [2023-12-27 15:13:42,219][20381] Updated weights for policy 1, policy_version 1381485 (0.0008) [2023-12-27 15:13:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 707739648. Throughput: 0: 10157.8, 1: 10143.7. Samples: 707728928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.810')] [2023-12-27 15:13:42,543][20347] Updated weights for policy 0, policy_version 1382746 (0.0007) [2023-12-27 15:13:42,599][20347] Updated weights for policy 0, policy_version 1382756 (0.0006) [2023-12-27 15:13:42,653][20347] Updated weights for policy 0, policy_version 1382766 (0.0008) [2023-12-27 15:13:42,703][20347] Updated weights for policy 0, policy_version 1382776 (0.0007) [2023-12-27 15:13:42,919][20381] Updated weights for policy 1, policy_version 1381495 (0.0007) [2023-12-27 15:13:42,977][20381] Updated weights for policy 1, policy_version 1381505 (0.0008) [2023-12-27 15:13:43,041][20381] Updated weights for policy 1, policy_version 1381515 (0.0010) [2023-12-27 15:13:43,315][20347] Updated weights for policy 0, policy_version 1382786 (0.0005) [2023-12-27 15:13:43,381][20347] Updated weights for policy 0, policy_version 1382796 (0.0005) [2023-12-27 15:13:43,442][20347] Updated weights for policy 0, policy_version 1382806 (0.0005) [2023-12-27 15:13:43,671][20381] Updated weights for policy 1, policy_version 1381525 (0.0008) [2023-12-27 15:13:43,735][20381] Updated weights for policy 1, policy_version 1381535 (0.0005) [2023-12-27 15:13:43,790][20381] Updated weights for policy 1, policy_version 1381545 (0.0006) [2023-12-27 15:13:44,038][20347] Updated weights for policy 0, policy_version 1382816 (0.0009) [2023-12-27 15:13:44,087][20347] Updated weights for policy 0, policy_version 1382826 (0.0010) [2023-12-27 15:13:44,138][20347] Updated weights for policy 0, policy_version 1382836 (0.0010) [2023-12-27 15:13:44,355][20381] Updated weights for policy 1, policy_version 1381555 (0.0006) [2023-12-27 15:13:44,416][20381] Updated weights for policy 1, policy_version 1381565 (0.0006) [2023-12-27 15:13:44,476][20381] Updated weights for policy 1, policy_version 1381575 (0.0008) [2023-12-27 15:13:44,852][20347] Updated weights for policy 0, policy_version 1382846 (0.0008) [2023-12-27 15:13:44,904][20347] Updated weights for policy 0, policy_version 1382856 (0.0007) [2023-12-27 15:13:44,956][20347] Updated weights for policy 0, policy_version 1382866 (0.0008) [2023-12-27 15:13:45,151][20381] Updated weights for policy 1, policy_version 1381585 (0.0010) [2023-12-27 15:13:45,213][20381] Updated weights for policy 1, policy_version 1381595 (0.0008) [2023-12-27 15:13:45,282][20381] Updated weights for policy 1, policy_version 1381605 (0.0008) [2023-12-27 15:13:45,346][20381] Updated weights for policy 1, policy_version 1381615 (0.0008) [2023-12-27 15:13:45,682][20347] Updated weights for policy 0, policy_version 1382876 (0.0008) [2023-12-27 15:13:45,732][20347] Updated weights for policy 0, policy_version 1382886 (0.0008) [2023-12-27 15:13:45,784][20347] Updated weights for policy 0, policy_version 1382896 (0.0007) [2023-12-27 15:13:45,985][20381] Updated weights for policy 1, policy_version 1381625 (0.0005) [2023-12-27 15:13:46,036][20381] Updated weights for policy 1, policy_version 1381635 (0.0005) [2023-12-27 15:13:46,091][20381] Updated weights for policy 1, policy_version 1381645 (0.0005) [2023-12-27 15:13:46,467][20347] Updated weights for policy 0, policy_version 1382906 (0.0007) [2023-12-27 15:13:46,513][20347] Updated weights for policy 0, policy_version 1382916 (0.0005) [2023-12-27 15:13:46,562][20347] Updated weights for policy 0, policy_version 1382926 (0.0009) [2023-12-27 15:13:46,612][20347] Updated weights for policy 0, policy_version 1382936 (0.0008) [2023-12-27 15:13:46,636][20381] Updated weights for policy 1, policy_version 1381655 (0.0005) [2023-12-27 15:13:46,687][20381] Updated weights for policy 1, policy_version 1381665 (0.0005) [2023-12-27 15:13:46,746][20381] Updated weights for policy 1, policy_version 1381675 (0.0005) [2023-12-27 15:13:47,286][20381] Updated weights for policy 1, policy_version 1381685 (0.0008) [2023-12-27 15:13:47,304][20347] Updated weights for policy 0, policy_version 1382946 (0.0010) [2023-12-27 15:13:47,338][20381] Updated weights for policy 1, policy_version 1381695 (0.0006) [2023-12-27 15:13:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 707837952. Throughput: 0: 10144.9, 1: 10229.6. Samples: 707855780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.710')] [2023-12-27 15:13:47,367][20347] Updated weights for policy 0, policy_version 1382956 (0.0010) [2023-12-27 15:13:47,390][20381] Updated weights for policy 1, policy_version 1381705 (0.0007) [2023-12-27 15:13:47,418][20347] Updated weights for policy 0, policy_version 1382966 (0.0010) [2023-12-27 15:13:47,964][20381] Updated weights for policy 1, policy_version 1381715 (0.0005) [2023-12-27 15:13:48,016][20381] Updated weights for policy 1, policy_version 1381725 (0.0005) [2023-12-27 15:13:48,075][20381] Updated weights for policy 1, policy_version 1381735 (0.0005) [2023-12-27 15:13:48,095][20347] Updated weights for policy 0, policy_version 1382976 (0.0008) [2023-12-27 15:13:48,153][20347] Updated weights for policy 0, policy_version 1382986 (0.0010) [2023-12-27 15:13:48,213][20347] Updated weights for policy 0, policy_version 1382996 (0.0010) [2023-12-27 15:13:48,735][20381] Updated weights for policy 1, policy_version 1381745 (0.0006) [2023-12-27 15:13:48,786][20381] Updated weights for policy 1, policy_version 1381755 (0.0007) [2023-12-27 15:13:48,836][20381] Updated weights for policy 1, policy_version 1381765 (0.0008) [2023-12-27 15:13:48,837][20347] Updated weights for policy 0, policy_version 1383006 (0.0010) [2023-12-27 15:13:48,890][20381] Updated weights for policy 1, policy_version 1381775 (0.0009) [2023-12-27 15:13:48,893][20347] Updated weights for policy 0, policy_version 1383016 (0.0010) [2023-12-27 15:13:48,950][20347] Updated weights for policy 0, policy_version 1383026 (0.0008) [2023-12-27 15:13:49,606][20381] Updated weights for policy 1, policy_version 1381785 (0.0008) [2023-12-27 15:13:49,627][20347] Updated weights for policy 0, policy_version 1383036 (0.0006) [2023-12-27 15:13:49,663][20381] Updated weights for policy 1, policy_version 1381795 (0.0008) [2023-12-27 15:13:49,684][20347] Updated weights for policy 0, policy_version 1383046 (0.0005) [2023-12-27 15:13:49,723][20381] Updated weights for policy 1, policy_version 1381805 (0.0008) [2023-12-27 15:13:49,744][20347] Updated weights for policy 0, policy_version 1383056 (0.0006) [2023-12-27 15:13:50,425][20381] Updated weights for policy 1, policy_version 1381815 (0.0007) [2023-12-27 15:13:50,431][20347] Updated weights for policy 0, policy_version 1383066 (0.0007) [2023-12-27 15:13:50,483][20381] Updated weights for policy 1, policy_version 1381825 (0.0007) [2023-12-27 15:13:50,495][20347] Updated weights for policy 0, policy_version 1383076 (0.0006) [2023-12-27 15:13:50,543][20381] Updated weights for policy 1, policy_version 1381835 (0.0009) [2023-12-27 15:13:50,553][20347] Updated weights for policy 0, policy_version 1383086 (0.0006) [2023-12-27 15:13:50,620][20347] Updated weights for policy 0, policy_version 1383096 (0.0008) [2023-12-27 15:13:51,241][20381] Updated weights for policy 1, policy_version 1381845 (0.0008) [2023-12-27 15:13:51,302][20381] Updated weights for policy 1, policy_version 1381855 (0.0006) [2023-12-27 15:13:51,365][20347] Updated weights for policy 0, policy_version 1383106 (0.0008) [2023-12-27 15:13:51,372][20381] Updated weights for policy 1, policy_version 1381865 (0.0007) [2023-12-27 15:13:51,422][20347] Updated weights for policy 0, policy_version 1383116 (0.0008) [2023-12-27 15:13:51,478][20347] Updated weights for policy 0, policy_version 1383126 (0.0009) [2023-12-27 15:13:51,956][20381] Updated weights for policy 1, policy_version 1381875 (0.0007) [2023-12-27 15:13:52,023][20381] Updated weights for policy 1, policy_version 1381885 (0.0006) [2023-12-27 15:13:52,081][20381] Updated weights for policy 1, policy_version 1381895 (0.0006) [2023-12-27 15:13:52,230][20347] Updated weights for policy 0, policy_version 1383136 (0.0007) [2023-12-27 15:13:52,295][20347] Updated weights for policy 0, policy_version 1383146 (0.0007) [2023-12-27 15:13:52,359][20347] Updated weights for policy 0, policy_version 1383156 (0.0011) [2023-12-27 15:13:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 707944448. Throughput: 0: 10160.3, 1: 10233.0. Samples: 707917804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:52,363][19170] Avg episode reward: [(0, '980.480'), (1, '980.470')] [2023-12-27 15:13:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001381904_353796096.pth... [2023-12-27 15:13:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001380720_353492992.pth [2023-12-27 15:13:52,385][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001383160_354156544.pth... [2023-12-27 15:13:52,388][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001381944_353845248.pth [2023-12-27 15:13:52,745][20381] Updated weights for policy 1, policy_version 1381905 (0.0008) [2023-12-27 15:13:52,809][20381] Updated weights for policy 1, policy_version 1381915 (0.0011) [2023-12-27 15:13:52,862][20381] Updated weights for policy 1, policy_version 1381925 (0.0009) [2023-12-27 15:13:52,905][20381] Updated weights for policy 1, policy_version 1381935 (0.0005) [2023-12-27 15:13:52,972][20347] Updated weights for policy 0, policy_version 1383166 (0.0009) [2023-12-27 15:13:53,031][20347] Updated weights for policy 0, policy_version 1383176 (0.0010) [2023-12-27 15:13:53,092][20347] Updated weights for policy 0, policy_version 1383186 (0.0010) [2023-12-27 15:13:53,500][20381] Updated weights for policy 1, policy_version 1381945 (0.0010) [2023-12-27 15:13:53,551][20381] Updated weights for policy 1, policy_version 1381955 (0.0011) [2023-12-27 15:13:53,603][20381] Updated weights for policy 1, policy_version 1381965 (0.0010) [2023-12-27 15:13:53,701][20347] Updated weights for policy 0, policy_version 1383196 (0.0008) [2023-12-27 15:13:53,750][20347] Updated weights for policy 0, policy_version 1383206 (0.0005) [2023-12-27 15:13:53,801][20347] Updated weights for policy 0, policy_version 1383216 (0.0007) [2023-12-27 15:13:54,299][20381] Updated weights for policy 1, policy_version 1381975 (0.0009) [2023-12-27 15:13:54,347][20381] Updated weights for policy 1, policy_version 1381985 (0.0006) [2023-12-27 15:13:54,394][20381] Updated weights for policy 1, policy_version 1381995 (0.0007) [2023-12-27 15:13:54,454][20347] Updated weights for policy 0, policy_version 1383226 (0.0006) [2023-12-27 15:13:54,509][20347] Updated weights for policy 0, policy_version 1383236 (0.0007) [2023-12-27 15:13:54,564][20347] Updated weights for policy 0, policy_version 1383246 (0.0008) [2023-12-27 15:13:54,624][20347] Updated weights for policy 0, policy_version 1383256 (0.0008) [2023-12-27 15:13:55,085][20381] Updated weights for policy 1, policy_version 1382005 (0.0007) [2023-12-27 15:13:55,145][20381] Updated weights for policy 1, policy_version 1382015 (0.0005) [2023-12-27 15:13:55,205][20381] Updated weights for policy 1, policy_version 1382025 (0.0007) [2023-12-27 15:13:55,333][20347] Updated weights for policy 0, policy_version 1383266 (0.0007) [2023-12-27 15:13:55,380][20347] Updated weights for policy 0, policy_version 1383276 (0.0010) [2023-12-27 15:13:55,443][20347] Updated weights for policy 0, policy_version 1383286 (0.0010) [2023-12-27 15:13:55,804][20381] Updated weights for policy 1, policy_version 1382035 (0.0007) [2023-12-27 15:13:55,861][20381] Updated weights for policy 1, policy_version 1382045 (0.0005) [2023-12-27 15:13:55,910][20381] Updated weights for policy 1, policy_version 1382055 (0.0005) [2023-12-27 15:13:56,104][20347] Updated weights for policy 0, policy_version 1383296 (0.0010) [2023-12-27 15:13:56,161][20347] Updated weights for policy 0, policy_version 1383306 (0.0010) [2023-12-27 15:13:56,222][20347] Updated weights for policy 0, policy_version 1383316 (0.0010) [2023-12-27 15:13:56,556][20381] Updated weights for policy 1, policy_version 1382065 (0.0006) [2023-12-27 15:13:56,604][20381] Updated weights for policy 1, policy_version 1382075 (0.0010) [2023-12-27 15:13:56,645][20381] Updated weights for policy 1, policy_version 1382085 (0.0010) [2023-12-27 15:13:56,699][20381] Updated weights for policy 1, policy_version 1382095 (0.0010) [2023-12-27 15:13:56,868][20347] Updated weights for policy 0, policy_version 1383326 (0.0007) [2023-12-27 15:13:56,918][20347] Updated weights for policy 0, policy_version 1383336 (0.0008) [2023-12-27 15:13:56,963][20347] Updated weights for policy 0, policy_version 1383346 (0.0010) [2023-12-27 15:13:57,341][20381] Updated weights for policy 1, policy_version 1382105 (0.0007) [2023-12-27 15:13:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 708050944. Throughput: 0: 10210.5, 1: 10251.3. Samples: 708042356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:13:57,363][19170] Avg episode reward: [(0, '980.480'), (1, '941.540')] [2023-12-27 15:13:57,398][20381] Updated weights for policy 1, policy_version 1382115 (0.0007) [2023-12-27 15:13:57,448][20381] Updated weights for policy 1, policy_version 1382125 (0.0005) [2023-12-27 15:13:57,627][20347] Updated weights for policy 0, policy_version 1383356 (0.0008) [2023-12-27 15:13:57,683][20347] Updated weights for policy 0, policy_version 1383366 (0.0006) [2023-12-27 15:13:57,736][20347] Updated weights for policy 0, policy_version 1383376 (0.0006) [2023-12-27 15:13:58,060][20381] Updated weights for policy 1, policy_version 1382135 (0.0007) [2023-12-27 15:13:58,124][20381] Updated weights for policy 1, policy_version 1382145 (0.0008) [2023-12-27 15:13:58,188][20381] Updated weights for policy 1, policy_version 1382155 (0.0008) [2023-12-27 15:13:58,329][20347] Updated weights for policy 0, policy_version 1383386 (0.0006) [2023-12-27 15:13:58,398][20347] Updated weights for policy 0, policy_version 1383396 (0.0008) [2023-12-27 15:13:58,460][20347] Updated weights for policy 0, policy_version 1383406 (0.0008) [2023-12-27 15:13:58,520][20347] Updated weights for policy 0, policy_version 1383416 (0.0008) [2023-12-27 15:13:58,937][20381] Updated weights for policy 1, policy_version 1382165 (0.0006) [2023-12-27 15:13:59,001][20381] Updated weights for policy 1, policy_version 1382175 (0.0009) [2023-12-27 15:13:59,066][20381] Updated weights for policy 1, policy_version 1382185 (0.0009) [2023-12-27 15:13:59,232][20347] Updated weights for policy 0, policy_version 1383426 (0.0008) [2023-12-27 15:13:59,295][20347] Updated weights for policy 0, policy_version 1383436 (0.0008) [2023-12-27 15:13:59,351][20347] Updated weights for policy 0, policy_version 1383446 (0.0008) [2023-12-27 15:13:59,779][20381] Updated weights for policy 1, policy_version 1382195 (0.0009) [2023-12-27 15:13:59,851][20381] Updated weights for policy 1, policy_version 1382205 (0.0009) [2023-12-27 15:13:59,914][20381] Updated weights for policy 1, policy_version 1382215 (0.0010) [2023-12-27 15:13:59,992][20347] Updated weights for policy 0, policy_version 1383456 (0.0006) [2023-12-27 15:14:00,051][20347] Updated weights for policy 0, policy_version 1383466 (0.0006) [2023-12-27 15:14:00,106][20347] Updated weights for policy 0, policy_version 1383476 (0.0005) [2023-12-27 15:14:00,585][20381] Updated weights for policy 1, policy_version 1382225 (0.0010) [2023-12-27 15:14:00,634][20381] Updated weights for policy 1, policy_version 1382235 (0.0008) [2023-12-27 15:14:00,676][20381] Updated weights for policy 1, policy_version 1382245 (0.0006) [2023-12-27 15:14:00,713][20347] Updated weights for policy 0, policy_version 1383486 (0.0005) [2023-12-27 15:14:00,720][20381] Updated weights for policy 1, policy_version 1382255 (0.0006) [2023-12-27 15:14:00,761][20347] Updated weights for policy 0, policy_version 1383496 (0.0005) [2023-12-27 15:14:00,818][20347] Updated weights for policy 0, policy_version 1383506 (0.0005) [2023-12-27 15:14:01,410][20381] Updated weights for policy 1, policy_version 1382265 (0.0008) [2023-12-27 15:14:01,423][20347] Updated weights for policy 0, policy_version 1383516 (0.0006) [2023-12-27 15:14:01,463][20381] Updated weights for policy 1, policy_version 1382275 (0.0008) [2023-12-27 15:14:01,477][20347] Updated weights for policy 0, policy_version 1383526 (0.0006) [2023-12-27 15:14:01,514][20381] Updated weights for policy 1, policy_version 1382285 (0.0008) [2023-12-27 15:14:01,538][20347] Updated weights for policy 0, policy_version 1383536 (0.0006) [2023-12-27 15:14:02,182][20381] Updated weights for policy 1, policy_version 1382295 (0.0010) [2023-12-27 15:14:02,243][20381] Updated weights for policy 1, policy_version 1382305 (0.0011) [2023-12-27 15:14:02,261][20347] Updated weights for policy 0, policy_version 1383546 (0.0007) [2023-12-27 15:14:02,307][20381] Updated weights for policy 1, policy_version 1382316 (0.0008) [2023-12-27 15:14:02,326][20347] Updated weights for policy 0, policy_version 1383556 (0.0006) [2023-12-27 15:14:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 708157440. Throughput: 0: 10315.8, 1: 10231.6. Samples: 708166428. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.070')] [2023-12-27 15:14:02,384][20347] Updated weights for policy 0, policy_version 1383566 (0.0008) [2023-12-27 15:14:02,442][20347] Updated weights for policy 0, policy_version 1383576 (0.0007) [2023-12-27 15:14:02,931][20381] Updated weights for policy 1, policy_version 1382326 (0.0010) [2023-12-27 15:14:02,990][20381] Updated weights for policy 1, policy_version 1382336 (0.0010) [2023-12-27 15:14:03,045][20381] Updated weights for policy 1, policy_version 1382346 (0.0008) [2023-12-27 15:14:03,160][20347] Updated weights for policy 0, policy_version 1383586 (0.0007) [2023-12-27 15:14:03,208][20347] Updated weights for policy 0, policy_version 1383596 (0.0006) [2023-12-27 15:14:03,257][20347] Updated weights for policy 0, policy_version 1383606 (0.0007) [2023-12-27 15:14:03,682][20381] Updated weights for policy 1, policy_version 1382356 (0.0008) [2023-12-27 15:14:03,743][20381] Updated weights for policy 1, policy_version 1382366 (0.0005) [2023-12-27 15:14:03,802][20381] Updated weights for policy 1, policy_version 1382376 (0.0007) [2023-12-27 15:14:03,985][20347] Updated weights for policy 0, policy_version 1383616 (0.0006) [2023-12-27 15:14:04,048][20347] Updated weights for policy 0, policy_version 1383626 (0.0005) [2023-12-27 15:14:04,105][20347] Updated weights for policy 0, policy_version 1383636 (0.0006) [2023-12-27 15:14:04,506][20381] Updated weights for policy 1, policy_version 1382386 (0.0008) [2023-12-27 15:14:04,562][20381] Updated weights for policy 1, policy_version 1382396 (0.0008) [2023-12-27 15:14:04,621][20381] Updated weights for policy 1, policy_version 1382406 (0.0008) [2023-12-27 15:14:04,681][20381] Updated weights for policy 1, policy_version 1382416 (0.0008) [2023-12-27 15:14:04,709][20347] Updated weights for policy 0, policy_version 1383646 (0.0007) [2023-12-27 15:14:04,758][20347] Updated weights for policy 0, policy_version 1383656 (0.0008) [2023-12-27 15:14:04,818][20347] Updated weights for policy 0, policy_version 1383666 (0.0008) [2023-12-27 15:14:05,325][20381] Updated weights for policy 1, policy_version 1382426 (0.0007) [2023-12-27 15:14:05,382][20381] Updated weights for policy 1, policy_version 1382436 (0.0008) [2023-12-27 15:14:05,445][20381] Updated weights for policy 1, policy_version 1382446 (0.0005) [2023-12-27 15:14:05,455][20347] Updated weights for policy 0, policy_version 1383676 (0.0007) [2023-12-27 15:14:05,510][20347] Updated weights for policy 0, policy_version 1383686 (0.0007) [2023-12-27 15:14:05,568][20347] Updated weights for policy 0, policy_version 1383696 (0.0008) [2023-12-27 15:14:06,024][20381] Updated weights for policy 1, policy_version 1382456 (0.0005) [2023-12-27 15:14:06,078][20381] Updated weights for policy 1, policy_version 1382466 (0.0006) [2023-12-27 15:14:06,138][20381] Updated weights for policy 1, policy_version 1382476 (0.0011) [2023-12-27 15:14:06,297][20347] Updated weights for policy 0, policy_version 1383706 (0.0007) [2023-12-27 15:14:06,362][20347] Updated weights for policy 0, policy_version 1383716 (0.0006) [2023-12-27 15:14:06,429][20347] Updated weights for policy 0, policy_version 1383726 (0.0006) [2023-12-27 15:14:06,496][20347] Updated weights for policy 0, policy_version 1383736 (0.0006) [2023-12-27 15:14:06,799][20381] Updated weights for policy 1, policy_version 1382486 (0.0011) [2023-12-27 15:14:06,848][20381] Updated weights for policy 1, policy_version 1382496 (0.0011) [2023-12-27 15:14:06,900][20381] Updated weights for policy 1, policy_version 1382506 (0.0010) [2023-12-27 15:14:07,072][20347] Updated weights for policy 0, policy_version 1383746 (0.0008) [2023-12-27 15:14:07,142][20347] Updated weights for policy 0, policy_version 1383756 (0.0008) [2023-12-27 15:14:07,201][20347] Updated weights for policy 0, policy_version 1383766 (0.0008) [2023-12-27 15:14:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 708263936. Throughput: 0: 10287.5, 1: 10272.4. Samples: 708228796. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:07,363][19170] Avg episode reward: [(0, '990.230'), (1, '990.290')] [2023-12-27 15:14:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001383768_354312192.pth... [2023-12-27 15:14:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001382512_353951744.pth... [2023-12-27 15:14:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001382552_354000896.pth [2023-12-27 15:14:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001381296_353640448.pth [2023-12-27 15:14:07,513][20381] Updated weights for policy 1, policy_version 1382516 (0.0011) [2023-12-27 15:14:07,564][20381] Updated weights for policy 1, policy_version 1382526 (0.0010) [2023-12-27 15:14:07,624][20381] Updated weights for policy 1, policy_version 1382536 (0.0010) [2023-12-27 15:14:07,852][20347] Updated weights for policy 0, policy_version 1383776 (0.0007) [2023-12-27 15:14:07,906][20347] Updated weights for policy 0, policy_version 1383786 (0.0008) [2023-12-27 15:14:07,958][20347] Updated weights for policy 0, policy_version 1383796 (0.0009) [2023-12-27 15:14:08,200][20381] Updated weights for policy 1, policy_version 1382546 (0.0008) [2023-12-27 15:14:08,262][20381] Updated weights for policy 1, policy_version 1382556 (0.0005) [2023-12-27 15:14:08,313][20381] Updated weights for policy 1, policy_version 1382566 (0.0005) [2023-12-27 15:14:08,382][20381] Updated weights for policy 1, policy_version 1382576 (0.0009) [2023-12-27 15:14:08,677][20347] Updated weights for policy 0, policy_version 1383806 (0.0007) [2023-12-27 15:14:08,726][20347] Updated weights for policy 0, policy_version 1383816 (0.0006) [2023-12-27 15:14:08,786][20347] Updated weights for policy 0, policy_version 1383826 (0.0007) [2023-12-27 15:14:09,036][20381] Updated weights for policy 1, policy_version 1382586 (0.0010) [2023-12-27 15:14:09,094][20381] Updated weights for policy 1, policy_version 1382596 (0.0011) [2023-12-27 15:14:09,147][20381] Updated weights for policy 1, policy_version 1382606 (0.0010) [2023-12-27 15:14:09,570][20347] Updated weights for policy 0, policy_version 1383836 (0.0007) [2023-12-27 15:14:09,629][20347] Updated weights for policy 0, policy_version 1383846 (0.0008) [2023-12-27 15:14:09,686][20347] Updated weights for policy 0, policy_version 1383856 (0.0008) [2023-12-27 15:14:09,891][20381] Updated weights for policy 1, policy_version 1382616 (0.0011) [2023-12-27 15:14:09,961][20381] Updated weights for policy 1, policy_version 1382626 (0.0010) [2023-12-27 15:14:10,028][20381] Updated weights for policy 1, policy_version 1382636 (0.0009) [2023-12-27 15:14:10,438][20347] Updated weights for policy 0, policy_version 1383866 (0.0008) [2023-12-27 15:14:10,499][20347] Updated weights for policy 0, policy_version 1383876 (0.0008) [2023-12-27 15:14:10,549][20347] Updated weights for policy 0, policy_version 1383886 (0.0007) [2023-12-27 15:14:10,608][20347] Updated weights for policy 0, policy_version 1383896 (0.0007) [2023-12-27 15:14:10,660][20381] Updated weights for policy 1, policy_version 1382646 (0.0007) [2023-12-27 15:14:10,722][20381] Updated weights for policy 1, policy_version 1382656 (0.0006) [2023-12-27 15:14:10,788][20381] Updated weights for policy 1, policy_version 1382666 (0.0005) [2023-12-27 15:14:11,385][20347] Updated weights for policy 0, policy_version 1383906 (0.0008) [2023-12-27 15:14:11,389][20381] Updated weights for policy 1, policy_version 1382676 (0.0007) [2023-12-27 15:14:11,448][20381] Updated weights for policy 1, policy_version 1382686 (0.0008) [2023-12-27 15:14:11,448][20347] Updated weights for policy 0, policy_version 1383916 (0.0008) [2023-12-27 15:14:11,499][20381] Updated weights for policy 1, policy_version 1382696 (0.0008) [2023-12-27 15:14:11,504][20347] Updated weights for policy 0, policy_version 1383926 (0.0008) [2023-12-27 15:14:12,204][20347] Updated weights for policy 0, policy_version 1383936 (0.0006) [2023-12-27 15:14:12,269][20347] Updated weights for policy 0, policy_version 1383946 (0.0007) [2023-12-27 15:14:12,302][20381] Updated weights for policy 1, policy_version 1382706 (0.0008) [2023-12-27 15:14:12,332][20347] Updated weights for policy 0, policy_version 1383956 (0.0006) [2023-12-27 15:14:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 708362240. Throughput: 0: 10218.8, 1: 10418.2. Samples: 708353208. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:12,363][19170] Avg episode reward: [(0, '970.780'), (1, '990.290')] [2023-12-27 15:14:12,367][20381] Updated weights for policy 1, policy_version 1382716 (0.0007) [2023-12-27 15:14:12,426][20381] Updated weights for policy 1, policy_version 1382726 (0.0007) [2023-12-27 15:14:12,493][20381] Updated weights for policy 1, policy_version 1382736 (0.0007) [2023-12-27 15:14:12,892][20347] Updated weights for policy 0, policy_version 1383966 (0.0007) [2023-12-27 15:14:12,945][20347] Updated weights for policy 0, policy_version 1383976 (0.0006) [2023-12-27 15:14:13,002][20347] Updated weights for policy 0, policy_version 1383986 (0.0005) [2023-12-27 15:14:13,217][20381] Updated weights for policy 1, policy_version 1382746 (0.0008) [2023-12-27 15:14:13,276][20381] Updated weights for policy 1, policy_version 1382756 (0.0007) [2023-12-27 15:14:13,333][20381] Updated weights for policy 1, policy_version 1382766 (0.0008) [2023-12-27 15:14:13,682][20347] Updated weights for policy 0, policy_version 1383996 (0.0007) [2023-12-27 15:14:13,739][20347] Updated weights for policy 0, policy_version 1384006 (0.0008) [2023-12-27 15:14:13,804][20347] Updated weights for policy 0, policy_version 1384016 (0.0006) [2023-12-27 15:14:14,000][20381] Updated weights for policy 1, policy_version 1382776 (0.0006) [2023-12-27 15:14:14,050][20381] Updated weights for policy 1, policy_version 1382786 (0.0005) [2023-12-27 15:14:14,111][20381] Updated weights for policy 1, policy_version 1382796 (0.0009) [2023-12-27 15:14:14,451][20347] Updated weights for policy 0, policy_version 1384026 (0.0010) [2023-12-27 15:14:14,513][20347] Updated weights for policy 0, policy_version 1384036 (0.0010) [2023-12-27 15:14:14,574][20347] Updated weights for policy 0, policy_version 1384046 (0.0009) [2023-12-27 15:14:14,633][20347] Updated weights for policy 0, policy_version 1384056 (0.0008) [2023-12-27 15:14:14,747][20381] Updated weights for policy 1, policy_version 1382806 (0.0010) [2023-12-27 15:14:14,812][20381] Updated weights for policy 1, policy_version 1382816 (0.0009) [2023-12-27 15:14:14,873][20381] Updated weights for policy 1, policy_version 1382826 (0.0009) [2023-12-27 15:14:15,295][20347] Updated weights for policy 0, policy_version 1384066 (0.0011) [2023-12-27 15:14:15,351][20347] Updated weights for policy 0, policy_version 1384076 (0.0010) [2023-12-27 15:14:15,406][20347] Updated weights for policy 0, policy_version 1384086 (0.0010) [2023-12-27 15:14:15,490][20381] Updated weights for policy 1, policy_version 1382836 (0.0007) [2023-12-27 15:14:15,540][20381] Updated weights for policy 1, policy_version 1382846 (0.0010) [2023-12-27 15:14:15,588][20381] Updated weights for policy 1, policy_version 1382856 (0.0010) [2023-12-27 15:14:16,098][20347] Updated weights for policy 0, policy_version 1384096 (0.0006) [2023-12-27 15:14:16,155][20347] Updated weights for policy 0, policy_version 1384106 (0.0005) [2023-12-27 15:14:16,204][20381] Updated weights for policy 1, policy_version 1382866 (0.0010) [2023-12-27 15:14:16,212][20347] Updated weights for policy 0, policy_version 1384116 (0.0007) [2023-12-27 15:14:16,255][20381] Updated weights for policy 1, policy_version 1382876 (0.0010) [2023-12-27 15:14:16,309][20381] Updated weights for policy 1, policy_version 1382886 (0.0010) [2023-12-27 15:14:16,356][20381] Updated weights for policy 1, policy_version 1382896 (0.0007) [2023-12-27 15:14:16,870][20347] Updated weights for policy 0, policy_version 1384126 (0.0007) [2023-12-27 15:14:16,919][20347] Updated weights for policy 0, policy_version 1384136 (0.0005) [2023-12-27 15:14:16,977][20347] Updated weights for policy 0, policy_version 1384146 (0.0005) [2023-12-27 15:14:16,985][20381] Updated weights for policy 1, policy_version 1382906 (0.0007) [2023-12-27 15:14:17,043][20381] Updated weights for policy 1, policy_version 1382916 (0.0008) [2023-12-27 15:14:17,104][20381] Updated weights for policy 1, policy_version 1382926 (0.0007) [2023-12-27 15:14:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 708468736. Throughput: 0: 10243.5, 1: 10446.2. Samples: 708477048. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:17,363][19170] Avg episode reward: [(0, '980.550'), (1, '990.270')] [2023-12-27 15:14:17,591][20347] Updated weights for policy 0, policy_version 1384156 (0.0005) [2023-12-27 15:14:17,651][20347] Updated weights for policy 0, policy_version 1384166 (0.0010) [2023-12-27 15:14:17,704][20347] Updated weights for policy 0, policy_version 1384176 (0.0007) [2023-12-27 15:14:17,707][20381] Updated weights for policy 1, policy_version 1382936 (0.0007) [2023-12-27 15:14:17,762][20381] Updated weights for policy 1, policy_version 1382946 (0.0006) [2023-12-27 15:14:17,810][20381] Updated weights for policy 1, policy_version 1382956 (0.0005) [2023-12-27 15:14:18,367][20347] Updated weights for policy 0, policy_version 1384186 (0.0006) [2023-12-27 15:14:18,425][20347] Updated weights for policy 0, policy_version 1384196 (0.0008) [2023-12-27 15:14:18,458][20291] KL-divergence is very high: 101.0553 [2023-12-27 15:14:18,461][20381] Updated weights for policy 1, policy_version 1382966 (0.0005) [2023-12-27 15:14:18,482][20347] Updated weights for policy 0, policy_version 1384206 (0.0008) [2023-12-27 15:14:18,517][20381] Updated weights for policy 1, policy_version 1382976 (0.0007) [2023-12-27 15:14:18,537][20347] Updated weights for policy 0, policy_version 1384216 (0.0006) [2023-12-27 15:14:18,569][20381] Updated weights for policy 1, policy_version 1382986 (0.0008) [2023-12-27 15:14:19,201][20381] Updated weights for policy 1, policy_version 1382996 (0.0007) [2023-12-27 15:14:19,210][20347] Updated weights for policy 0, policy_version 1384226 (0.0010) [2023-12-27 15:14:19,260][20381] Updated weights for policy 1, policy_version 1383006 (0.0007) [2023-12-27 15:14:19,273][20347] Updated weights for policy 0, policy_version 1384236 (0.0009) [2023-12-27 15:14:19,319][20381] Updated weights for policy 1, policy_version 1383016 (0.0006) [2023-12-27 15:14:19,335][20347] Updated weights for policy 0, policy_version 1384246 (0.0010) [2023-12-27 15:14:19,959][20347] Updated weights for policy 0, policy_version 1384256 (0.0007) [2023-12-27 15:14:20,018][20347] Updated weights for policy 0, policy_version 1384266 (0.0005) [2023-12-27 15:14:20,077][20347] Updated weights for policy 0, policy_version 1384276 (0.0006) [2023-12-27 15:14:20,086][20381] Updated weights for policy 1, policy_version 1383026 (0.0008) [2023-12-27 15:14:20,142][20381] Updated weights for policy 1, policy_version 1383036 (0.0008) [2023-12-27 15:14:20,195][20381] Updated weights for policy 1, policy_version 1383046 (0.0008) [2023-12-27 15:14:20,255][20381] Updated weights for policy 1, policy_version 1383056 (0.0008) [2023-12-27 15:14:20,776][20347] Updated weights for policy 0, policy_version 1384286 (0.0006) [2023-12-27 15:14:20,842][20347] Updated weights for policy 0, policy_version 1384296 (0.0006) [2023-12-27 15:14:20,904][20347] Updated weights for policy 0, policy_version 1384306 (0.0008) [2023-12-27 15:14:20,998][20381] Updated weights for policy 1, policy_version 1383066 (0.0008) [2023-12-27 15:14:21,064][20381] Updated weights for policy 1, policy_version 1383076 (0.0009) [2023-12-27 15:14:21,129][20381] Updated weights for policy 1, policy_version 1383086 (0.0009) [2023-12-27 15:14:21,604][20347] Updated weights for policy 0, policy_version 1384316 (0.0008) [2023-12-27 15:14:21,677][20347] Updated weights for policy 0, policy_version 1384326 (0.0007) [2023-12-27 15:14:21,742][20347] Updated weights for policy 0, policy_version 1384336 (0.0009) [2023-12-27 15:14:21,866][20381] Updated weights for policy 1, policy_version 1383096 (0.0008) [2023-12-27 15:14:21,918][20381] Updated weights for policy 1, policy_version 1383106 (0.0008) [2023-12-27 15:14:21,973][20381] Updated weights for policy 1, policy_version 1383116 (0.0007) [2023-12-27 15:14:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 708567040. Throughput: 0: 10291.1, 1: 10439.4. Samples: 708539804. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:22,363][19170] Avg episode reward: [(0, '980.470'), (1, '990.270')] [2023-12-27 15:14:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001383120_354107392.pth... [2023-12-27 15:14:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001384344_354459648.pth... [2023-12-27 15:14:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001381904_353796096.pth [2023-12-27 15:14:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001383160_354156544.pth [2023-12-27 15:14:22,447][20347] Updated weights for policy 0, policy_version 1384346 (0.0007) [2023-12-27 15:14:22,508][20347] Updated weights for policy 0, policy_version 1384356 (0.0007) [2023-12-27 15:14:22,563][20347] Updated weights for policy 0, policy_version 1384366 (0.0008) [2023-12-27 15:14:22,617][20347] Updated weights for policy 0, policy_version 1384376 (0.0008) [2023-12-27 15:14:22,674][20381] Updated weights for policy 1, policy_version 1383126 (0.0007) [2023-12-27 15:14:22,742][20381] Updated weights for policy 1, policy_version 1383136 (0.0007) [2023-12-27 15:14:22,801][20381] Updated weights for policy 1, policy_version 1383146 (0.0007) [2023-12-27 15:14:23,296][20347] Updated weights for policy 0, policy_version 1384386 (0.0008) [2023-12-27 15:14:23,340][20347] Updated weights for policy 0, policy_version 1384396 (0.0007) [2023-12-27 15:14:23,386][20347] Updated weights for policy 0, policy_version 1384406 (0.0005) [2023-12-27 15:14:23,488][20381] Updated weights for policy 1, policy_version 1383156 (0.0007) [2023-12-27 15:14:23,537][20381] Updated weights for policy 1, policy_version 1383166 (0.0008) [2023-12-27 15:14:23,582][20381] Updated weights for policy 1, policy_version 1383176 (0.0008) [2023-12-27 15:14:24,094][20347] Updated weights for policy 0, policy_version 1384416 (0.0007) [2023-12-27 15:14:24,154][20347] Updated weights for policy 0, policy_version 1384426 (0.0008) [2023-12-27 15:14:24,189][20381] Updated weights for policy 1, policy_version 1383186 (0.0006) [2023-12-27 15:14:24,217][20347] Updated weights for policy 0, policy_version 1384436 (0.0007) [2023-12-27 15:14:24,246][20381] Updated weights for policy 1, policy_version 1383196 (0.0006) [2023-12-27 15:14:24,305][20381] Updated weights for policy 1, policy_version 1383206 (0.0005) [2023-12-27 15:14:24,362][20381] Updated weights for policy 1, policy_version 1383216 (0.0005) [2023-12-27 15:14:24,936][20347] Updated weights for policy 0, policy_version 1384446 (0.0006) [2023-12-27 15:14:24,959][20381] Updated weights for policy 1, policy_version 1383226 (0.0007) [2023-12-27 15:14:24,988][20347] Updated weights for policy 0, policy_version 1384456 (0.0005) [2023-12-27 15:14:25,015][20381] Updated weights for policy 1, policy_version 1383236 (0.0006) [2023-12-27 15:14:25,042][20347] Updated weights for policy 0, policy_version 1384466 (0.0008) [2023-12-27 15:14:25,069][20381] Updated weights for policy 1, policy_version 1383246 (0.0006) [2023-12-27 15:14:25,631][20381] Updated weights for policy 1, policy_version 1383256 (0.0005) [2023-12-27 15:14:25,679][20381] Updated weights for policy 1, policy_version 1383266 (0.0005) [2023-12-27 15:14:25,735][20381] Updated weights for policy 1, policy_version 1383276 (0.0007) [2023-12-27 15:14:25,783][20347] Updated weights for policy 0, policy_version 1384476 (0.0007) [2023-12-27 15:14:25,837][20347] Updated weights for policy 0, policy_version 1384486 (0.0008) [2023-12-27 15:14:25,885][20347] Updated weights for policy 0, policy_version 1384496 (0.0011) [2023-12-27 15:14:26,350][20381] Updated weights for policy 1, policy_version 1383286 (0.0007) [2023-12-27 15:14:26,400][20381] Updated weights for policy 1, policy_version 1383296 (0.0007) [2023-12-27 15:14:26,450][20381] Updated weights for policy 1, policy_version 1383306 (0.0006) [2023-12-27 15:14:26,617][20347] Updated weights for policy 0, policy_version 1384506 (0.0009) [2023-12-27 15:14:26,671][20347] Updated weights for policy 0, policy_version 1384516 (0.0007) [2023-12-27 15:14:26,726][20347] Updated weights for policy 0, policy_version 1384526 (0.0008) [2023-12-27 15:14:26,770][20347] Updated weights for policy 0, policy_version 1384536 (0.0008) [2023-12-27 15:14:27,043][20381] Updated weights for policy 1, policy_version 1383316 (0.0005) [2023-12-27 15:14:27,086][20381] Updated weights for policy 1, policy_version 1383326 (0.0005) [2023-12-27 15:14:27,132][20381] Updated weights for policy 1, policy_version 1383336 (0.0006) [2023-12-27 15:14:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 708673536. Throughput: 0: 10204.4, 1: 10511.4. Samples: 708661140. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:27,363][19170] Avg episode reward: [(0, '970.780'), (1, '1000.000')] [2023-12-27 15:14:27,492][20347] Updated weights for policy 0, policy_version 1384546 (0.0008) [2023-12-27 15:14:27,550][20347] Updated weights for policy 0, policy_version 1384556 (0.0008) [2023-12-27 15:14:27,608][20347] Updated weights for policy 0, policy_version 1384566 (0.0008) [2023-12-27 15:14:27,772][20381] Updated weights for policy 1, policy_version 1383346 (0.0008) [2023-12-27 15:14:27,824][20381] Updated weights for policy 1, policy_version 1383356 (0.0008) [2023-12-27 15:14:27,880][20381] Updated weights for policy 1, policy_version 1383366 (0.0008) [2023-12-27 15:14:27,924][20381] Updated weights for policy 1, policy_version 1383376 (0.0007) [2023-12-27 15:14:28,296][20347] Updated weights for policy 0, policy_version 1384576 (0.0007) [2023-12-27 15:14:28,357][20347] Updated weights for policy 0, policy_version 1384586 (0.0008) [2023-12-27 15:14:28,412][20347] Updated weights for policy 0, policy_version 1384596 (0.0008) [2023-12-27 15:14:28,611][20381] Updated weights for policy 1, policy_version 1383386 (0.0007) [2023-12-27 15:14:28,666][20381] Updated weights for policy 1, policy_version 1383396 (0.0008) [2023-12-27 15:14:28,722][20381] Updated weights for policy 1, policy_version 1383406 (0.0008) [2023-12-27 15:14:29,120][20347] Updated weights for policy 0, policy_version 1384606 (0.0008) [2023-12-27 15:14:29,177][20347] Updated weights for policy 0, policy_version 1384616 (0.0007) [2023-12-27 15:14:29,237][20347] Updated weights for policy 0, policy_version 1384626 (0.0007) [2023-12-27 15:14:29,416][20381] Updated weights for policy 1, policy_version 1383416 (0.0008) [2023-12-27 15:14:29,473][20381] Updated weights for policy 1, policy_version 1383426 (0.0005) [2023-12-27 15:14:29,519][20381] Updated weights for policy 1, policy_version 1383436 (0.0007) [2023-12-27 15:14:29,962][20347] Updated weights for policy 0, policy_version 1384636 (0.0008) [2023-12-27 15:14:30,024][20347] Updated weights for policy 0, policy_version 1384646 (0.0007) [2023-12-27 15:14:30,078][20347] Updated weights for policy 0, policy_version 1384656 (0.0007) [2023-12-27 15:14:30,190][20381] Updated weights for policy 1, policy_version 1383446 (0.0008) [2023-12-27 15:14:30,240][20381] Updated weights for policy 1, policy_version 1383456 (0.0008) [2023-12-27 15:14:30,290][20381] Updated weights for policy 1, policy_version 1383466 (0.0008) [2023-12-27 15:14:30,740][20347] Updated weights for policy 0, policy_version 1384666 (0.0008) [2023-12-27 15:14:30,791][20347] Updated weights for policy 0, policy_version 1384676 (0.0007) [2023-12-27 15:14:30,844][20347] Updated weights for policy 0, policy_version 1384686 (0.0008) [2023-12-27 15:14:30,899][20347] Updated weights for policy 0, policy_version 1384696 (0.0007) [2023-12-27 15:14:30,957][20381] Updated weights for policy 1, policy_version 1383476 (0.0007) [2023-12-27 15:14:31,007][20381] Updated weights for policy 1, policy_version 1383486 (0.0006) [2023-12-27 15:14:31,073][20381] Updated weights for policy 1, policy_version 1383496 (0.0007) [2023-12-27 15:14:31,670][20347] Updated weights for policy 0, policy_version 1384706 (0.0007) [2023-12-27 15:14:31,731][20347] Updated weights for policy 0, policy_version 1384716 (0.0008) [2023-12-27 15:14:31,786][20347] Updated weights for policy 0, policy_version 1384726 (0.0007) [2023-12-27 15:14:31,807][20381] Updated weights for policy 1, policy_version 1383506 (0.0009) [2023-12-27 15:14:31,860][20381] Updated weights for policy 1, policy_version 1383516 (0.0011) [2023-12-27 15:14:31,916][20381] Updated weights for policy 1, policy_version 1383526 (0.0010) [2023-12-27 15:14:31,974][20381] Updated weights for policy 1, policy_version 1383536 (0.0010) [2023-12-27 15:14:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 708771840. Throughput: 0: 10148.5, 1: 10462.8. Samples: 708783288. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:32,363][19170] Avg episode reward: [(0, '990.230'), (1, '980.510')] [2023-12-27 15:14:32,500][20347] Updated weights for policy 0, policy_version 1384736 (0.0007) [2023-12-27 15:14:32,562][20347] Updated weights for policy 0, policy_version 1384746 (0.0006) [2023-12-27 15:14:32,622][20347] Updated weights for policy 0, policy_version 1384756 (0.0006) [2023-12-27 15:14:32,691][20381] Updated weights for policy 1, policy_version 1383546 (0.0007) [2023-12-27 15:14:32,740][20381] Updated weights for policy 1, policy_version 1383556 (0.0007) [2023-12-27 15:14:32,786][20381] Updated weights for policy 1, policy_version 1383566 (0.0010) [2023-12-27 15:14:33,162][20347] Updated weights for policy 0, policy_version 1384766 (0.0006) [2023-12-27 15:14:33,230][20347] Updated weights for policy 0, policy_version 1384776 (0.0006) [2023-12-27 15:14:33,293][20347] Updated weights for policy 0, policy_version 1384786 (0.0007) [2023-12-27 15:14:33,503][20381] Updated weights for policy 1, policy_version 1383576 (0.0009) [2023-12-27 15:14:33,555][20381] Updated weights for policy 1, policy_version 1383586 (0.0008) [2023-12-27 15:14:33,612][20381] Updated weights for policy 1, policy_version 1383596 (0.0007) [2023-12-27 15:14:33,902][20347] Updated weights for policy 0, policy_version 1384796 (0.0006) [2023-12-27 15:14:33,953][20347] Updated weights for policy 0, policy_version 1384806 (0.0005) [2023-12-27 15:14:34,006][20347] Updated weights for policy 0, policy_version 1384816 (0.0006) [2023-12-27 15:14:34,293][20381] Updated weights for policy 1, policy_version 1383606 (0.0006) [2023-12-27 15:14:34,357][20381] Updated weights for policy 1, policy_version 1383616 (0.0006) [2023-12-27 15:14:34,418][20381] Updated weights for policy 1, policy_version 1383626 (0.0007) [2023-12-27 15:14:34,661][20347] Updated weights for policy 0, policy_version 1384826 (0.0006) [2023-12-27 15:14:34,722][20347] Updated weights for policy 0, policy_version 1384836 (0.0008) [2023-12-27 15:14:34,779][20347] Updated weights for policy 0, policy_version 1384846 (0.0008) [2023-12-27 15:14:34,831][20347] Updated weights for policy 0, policy_version 1384856 (0.0007) [2023-12-27 15:14:35,032][20381] Updated weights for policy 1, policy_version 1383636 (0.0007) [2023-12-27 15:14:35,090][20381] Updated weights for policy 1, policy_version 1383646 (0.0007) [2023-12-27 15:14:35,145][20381] Updated weights for policy 1, policy_version 1383656 (0.0008) [2023-12-27 15:14:35,555][20347] Updated weights for policy 0, policy_version 1384866 (0.0010) [2023-12-27 15:14:35,611][20347] Updated weights for policy 0, policy_version 1384876 (0.0010) [2023-12-27 15:14:35,666][20347] Updated weights for policy 0, policy_version 1384886 (0.0010) [2023-12-27 15:14:35,819][20381] Updated weights for policy 1, policy_version 1383666 (0.0008) [2023-12-27 15:14:35,870][20381] Updated weights for policy 1, policy_version 1383676 (0.0007) [2023-12-27 15:14:35,912][20381] Updated weights for policy 1, policy_version 1383686 (0.0007) [2023-12-27 15:14:35,967][20381] Updated weights for policy 1, policy_version 1383696 (0.0008) [2023-12-27 15:14:36,318][20347] Updated weights for policy 0, policy_version 1384896 (0.0006) [2023-12-27 15:14:36,382][20347] Updated weights for policy 0, policy_version 1384906 (0.0007) [2023-12-27 15:14:36,447][20347] Updated weights for policy 0, policy_version 1384916 (0.0008) [2023-12-27 15:14:36,703][20381] Updated weights for policy 1, policy_version 1383706 (0.0009) [2023-12-27 15:14:36,761][20381] Updated weights for policy 1, policy_version 1383716 (0.0007) [2023-12-27 15:14:36,822][20381] Updated weights for policy 1, policy_version 1383726 (0.0005) [2023-12-27 15:14:37,078][20347] Updated weights for policy 0, policy_version 1384926 (0.0007) [2023-12-27 15:14:37,148][20347] Updated weights for policy 0, policy_version 1384936 (0.0008) [2023-12-27 15:14:37,208][20347] Updated weights for policy 0, policy_version 1384946 (0.0008) [2023-12-27 15:14:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 708878336. Throughput: 0: 10178.2, 1: 10440.9. Samples: 708845664. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:37,363][19170] Avg episode reward: [(0, '970.860'), (1, '980.550')] [2023-12-27 15:14:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001384952_354615296.pth... [2023-12-27 15:14:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001383768_354312192.pth [2023-12-27 15:14:37,395][20381] Updated weights for policy 1, policy_version 1383736 (0.0005) [2023-12-27 15:14:37,451][20381] Updated weights for policy 1, policy_version 1383746 (0.0006) [2023-12-27 15:14:37,502][20381] Updated weights for policy 1, policy_version 1383756 (0.0006) [2023-12-27 15:14:37,525][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001383760_354271232.pth... [2023-12-27 15:14:37,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001382512_353951744.pth [2023-12-27 15:14:37,772][20291] KL-divergence is very high: 179.4883 [2023-12-27 15:14:37,791][20347] Updated weights for policy 0, policy_version 1384956 (0.0007) [2023-12-27 15:14:37,818][20291] KL-divergence is very high: 105.8527 [2023-12-27 15:14:37,843][20347] Updated weights for policy 0, policy_version 1384966 (0.0005) [2023-12-27 15:14:37,889][20347] Updated weights for policy 0, policy_version 1384976 (0.0005) [2023-12-27 15:14:38,109][20381] Updated weights for policy 1, policy_version 1383766 (0.0005) [2023-12-27 15:14:38,157][20381] Updated weights for policy 1, policy_version 1383776 (0.0005) [2023-12-27 15:14:38,209][20381] Updated weights for policy 1, policy_version 1383786 (0.0005) [2023-12-27 15:14:38,524][20347] Updated weights for policy 0, policy_version 1384986 (0.0006) [2023-12-27 15:14:38,582][20347] Updated weights for policy 0, policy_version 1384996 (0.0007) [2023-12-27 15:14:38,618][20291] KL-divergence is very high: 184.9489 [2023-12-27 15:14:38,643][20347] Updated weights for policy 0, policy_version 1385006 (0.0008) [2023-12-27 15:14:38,667][20291] KL-divergence is very high: 137.2490 [2023-12-27 15:14:38,701][20347] Updated weights for policy 0, policy_version 1385016 (0.0008) [2023-12-27 15:14:38,900][20381] Updated weights for policy 1, policy_version 1383796 (0.0006) [2023-12-27 15:14:38,944][20381] Updated weights for policy 1, policy_version 1383806 (0.0005) [2023-12-27 15:14:38,992][20381] Updated weights for policy 1, policy_version 1383816 (0.0005) [2023-12-27 15:14:39,467][20347] Updated weights for policy 0, policy_version 1385026 (0.0007) [2023-12-27 15:14:39,530][20347] Updated weights for policy 0, policy_version 1385036 (0.0007) [2023-12-27 15:14:39,589][20347] Updated weights for policy 0, policy_version 1385046 (0.0007) [2023-12-27 15:14:39,674][20381] Updated weights for policy 1, policy_version 1383826 (0.0006) [2023-12-27 15:14:39,731][20381] Updated weights for policy 1, policy_version 1383836 (0.0006) [2023-12-27 15:14:39,797][20381] Updated weights for policy 1, policy_version 1383846 (0.0008) [2023-12-27 15:14:39,870][20381] Updated weights for policy 1, policy_version 1383856 (0.0010) [2023-12-27 15:14:40,286][20347] Updated weights for policy 0, policy_version 1385056 (0.0008) [2023-12-27 15:14:40,341][20347] Updated weights for policy 0, policy_version 1385066 (0.0008) [2023-12-27 15:14:40,402][20347] Updated weights for policy 0, policy_version 1385076 (0.0006) [2023-12-27 15:14:40,561][20381] Updated weights for policy 1, policy_version 1383866 (0.0011) [2023-12-27 15:14:40,615][20381] Updated weights for policy 1, policy_version 1383876 (0.0010) [2023-12-27 15:14:40,676][20381] Updated weights for policy 1, policy_version 1383886 (0.0007) [2023-12-27 15:14:41,097][20347] Updated weights for policy 0, policy_version 1385086 (0.0006) [2023-12-27 15:14:41,155][20347] Updated weights for policy 0, policy_version 1385096 (0.0008) [2023-12-27 15:14:41,218][20347] Updated weights for policy 0, policy_version 1385106 (0.0008) [2023-12-27 15:14:41,315][20381] Updated weights for policy 1, policy_version 1383896 (0.0010) [2023-12-27 15:14:41,390][20381] Updated weights for policy 1, policy_version 1383906 (0.0009) [2023-12-27 15:14:41,450][20381] Updated weights for policy 1, policy_version 1383916 (0.0007) [2023-12-27 15:14:41,950][20347] Updated weights for policy 0, policy_version 1385116 (0.0007) [2023-12-27 15:14:42,009][20347] Updated weights for policy 0, policy_version 1385126 (0.0008) [2023-12-27 15:14:42,067][20347] Updated weights for policy 0, policy_version 1385136 (0.0008) [2023-12-27 15:14:42,140][20381] Updated weights for policy 1, policy_version 1383926 (0.0006) [2023-12-27 15:14:42,204][20381] Updated weights for policy 1, policy_version 1383936 (0.0007) [2023-12-27 15:14:42,265][20381] Updated weights for policy 1, policy_version 1383946 (0.0006) [2023-12-27 15:14:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 708984832. Throughput: 0: 10198.1, 1: 10449.9. Samples: 708971516. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:42,363][19170] Avg episode reward: [(0, '931.910'), (1, '980.560')] [2023-12-27 15:14:42,736][20347] Updated weights for policy 0, policy_version 1385146 (0.0007) [2023-12-27 15:14:42,797][20347] Updated weights for policy 0, policy_version 1385156 (0.0006) [2023-12-27 15:14:42,857][20347] Updated weights for policy 0, policy_version 1385166 (0.0007) [2023-12-27 15:14:42,918][20347] Updated weights for policy 0, policy_version 1385176 (0.0007) [2023-12-27 15:14:42,932][20381] Updated weights for policy 1, policy_version 1383956 (0.0008) [2023-12-27 15:14:42,992][20381] Updated weights for policy 1, policy_version 1383966 (0.0011) [2023-12-27 15:14:43,041][20381] Updated weights for policy 1, policy_version 1383976 (0.0011) [2023-12-27 15:14:43,466][20347] Updated weights for policy 0, policy_version 1385186 (0.0008) [2023-12-27 15:14:43,530][20347] Updated weights for policy 0, policy_version 1385196 (0.0008) [2023-12-27 15:14:43,575][20347] Updated weights for policy 0, policy_version 1385206 (0.0008) [2023-12-27 15:14:43,695][20381] Updated weights for policy 1, policy_version 1383986 (0.0010) [2023-12-27 15:14:43,754][20381] Updated weights for policy 1, policy_version 1383996 (0.0006) [2023-12-27 15:14:43,808][20381] Updated weights for policy 1, policy_version 1384006 (0.0005) [2023-12-27 15:14:43,854][20381] Updated weights for policy 1, policy_version 1384016 (0.0005) [2023-12-27 15:14:44,233][20347] Updated weights for policy 0, policy_version 1385216 (0.0009) [2023-12-27 15:14:44,288][20347] Updated weights for policy 0, policy_version 1385226 (0.0010) [2023-12-27 15:14:44,342][20347] Updated weights for policy 0, policy_version 1385236 (0.0009) [2023-12-27 15:14:44,399][20381] Updated weights for policy 1, policy_version 1384026 (0.0006) [2023-12-27 15:14:44,452][20381] Updated weights for policy 1, policy_version 1384036 (0.0005) [2023-12-27 15:14:44,507][20381] Updated weights for policy 1, policy_version 1384046 (0.0005) [2023-12-27 15:14:45,120][20347] Updated weights for policy 0, policy_version 1385246 (0.0007) [2023-12-27 15:14:45,160][20381] Updated weights for policy 1, policy_version 1384056 (0.0010) [2023-12-27 15:14:45,175][20347] Updated weights for policy 0, policy_version 1385256 (0.0009) [2023-12-27 15:14:45,209][20381] Updated weights for policy 1, policy_version 1384066 (0.0010) [2023-12-27 15:14:45,223][20347] Updated weights for policy 0, policy_version 1385266 (0.0008) [2023-12-27 15:14:45,268][20381] Updated weights for policy 1, policy_version 1384076 (0.0010) [2023-12-27 15:14:45,839][20381] Updated weights for policy 1, policy_version 1384086 (0.0008) [2023-12-27 15:14:45,894][20381] Updated weights for policy 1, policy_version 1384096 (0.0008) [2023-12-27 15:14:45,947][20381] Updated weights for policy 1, policy_version 1384106 (0.0007) [2023-12-27 15:14:45,974][20347] Updated weights for policy 0, policy_version 1385276 (0.0008) [2023-12-27 15:14:46,033][20347] Updated weights for policy 0, policy_version 1385286 (0.0007) [2023-12-27 15:14:46,092][20347] Updated weights for policy 0, policy_version 1385296 (0.0008) [2023-12-27 15:14:46,566][20381] Updated weights for policy 1, policy_version 1384116 (0.0008) [2023-12-27 15:14:46,615][20381] Updated weights for policy 1, policy_version 1384126 (0.0010) [2023-12-27 15:14:46,679][20381] Updated weights for policy 1, policy_version 1384136 (0.0010) [2023-12-27 15:14:46,811][20347] Updated weights for policy 0, policy_version 1385306 (0.0007) [2023-12-27 15:14:46,856][20347] Updated weights for policy 0, policy_version 1385316 (0.0008) [2023-12-27 15:14:46,902][20347] Updated weights for policy 0, policy_version 1385326 (0.0008) [2023-12-27 15:14:46,950][20347] Updated weights for policy 0, policy_version 1385336 (0.0008) [2023-12-27 15:14:47,290][20381] Updated weights for policy 1, policy_version 1384146 (0.0007) [2023-12-27 15:14:47,342][20381] Updated weights for policy 1, policy_version 1384156 (0.0005) [2023-12-27 15:14:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 709083136. Throughput: 0: 10110.1, 1: 10538.7. Samples: 709095624. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:47,363][19170] Avg episode reward: [(0, '922.060'), (1, '980.530')] [2023-12-27 15:14:47,396][20381] Updated weights for policy 1, policy_version 1384166 (0.0005) [2023-12-27 15:14:47,450][20381] Updated weights for policy 1, policy_version 1384176 (0.0005) [2023-12-27 15:14:47,749][20347] Updated weights for policy 0, policy_version 1385346 (0.0007) [2023-12-27 15:14:47,812][20347] Updated weights for policy 0, policy_version 1385356 (0.0007) [2023-12-27 15:14:47,871][20347] Updated weights for policy 0, policy_version 1385366 (0.0008) [2023-12-27 15:14:48,015][20381] Updated weights for policy 1, policy_version 1384186 (0.0007) [2023-12-27 15:14:48,075][20381] Updated weights for policy 1, policy_version 1384196 (0.0007) [2023-12-27 15:14:48,129][20381] Updated weights for policy 1, policy_version 1384206 (0.0008) [2023-12-27 15:14:48,587][20347] Updated weights for policy 0, policy_version 1385376 (0.0008) [2023-12-27 15:14:48,650][20347] Updated weights for policy 0, policy_version 1385386 (0.0008) [2023-12-27 15:14:48,709][20347] Updated weights for policy 0, policy_version 1385396 (0.0008) [2023-12-27 15:14:48,810][20381] Updated weights for policy 1, policy_version 1384216 (0.0008) [2023-12-27 15:14:48,867][20381] Updated weights for policy 1, policy_version 1384226 (0.0007) [2023-12-27 15:14:48,927][20381] Updated weights for policy 1, policy_version 1384236 (0.0008) [2023-12-27 15:14:49,429][20347] Updated weights for policy 0, policy_version 1385406 (0.0007) [2023-12-27 15:14:49,485][20347] Updated weights for policy 0, policy_version 1385416 (0.0006) [2023-12-27 15:14:49,545][20347] Updated weights for policy 0, policy_version 1385426 (0.0008) [2023-12-27 15:14:49,684][20381] Updated weights for policy 1, policy_version 1384246 (0.0008) [2023-12-27 15:14:49,740][20381] Updated weights for policy 1, policy_version 1384256 (0.0007) [2023-12-27 15:14:49,793][20381] Updated weights for policy 1, policy_version 1384266 (0.0007) [2023-12-27 15:14:50,282][20347] Updated weights for policy 0, policy_version 1385436 (0.0008) [2023-12-27 15:14:50,335][20347] Updated weights for policy 0, policy_version 1385446 (0.0006) [2023-12-27 15:14:50,394][20347] Updated weights for policy 0, policy_version 1385456 (0.0005) [2023-12-27 15:14:50,594][20381] Updated weights for policy 1, policy_version 1384276 (0.0008) [2023-12-27 15:14:50,653][20381] Updated weights for policy 1, policy_version 1384286 (0.0008) [2023-12-27 15:14:50,710][20381] Updated weights for policy 1, policy_version 1384296 (0.0008) [2023-12-27 15:14:51,032][20347] Updated weights for policy 0, policy_version 1385466 (0.0006) [2023-12-27 15:14:51,090][20347] Updated weights for policy 0, policy_version 1385476 (0.0008) [2023-12-27 15:14:51,153][20347] Updated weights for policy 0, policy_version 1385486 (0.0008) [2023-12-27 15:14:51,207][20347] Updated weights for policy 0, policy_version 1385496 (0.0009) [2023-12-27 15:14:51,336][20381] Updated weights for policy 1, policy_version 1384306 (0.0008) [2023-12-27 15:14:51,402][20381] Updated weights for policy 1, policy_version 1384316 (0.0008) [2023-12-27 15:14:51,469][20381] Updated weights for policy 1, policy_version 1384326 (0.0007) [2023-12-27 15:14:51,536][20381] Updated weights for policy 1, policy_version 1384336 (0.0006) [2023-12-27 15:14:51,939][20347] Updated weights for policy 0, policy_version 1385506 (0.0006) [2023-12-27 15:14:51,994][20347] Updated weights for policy 0, policy_version 1385516 (0.0007) [2023-12-27 15:14:52,047][20347] Updated weights for policy 0, policy_version 1385526 (0.0007) [2023-12-27 15:14:52,249][20381] Updated weights for policy 1, policy_version 1384346 (0.0007) [2023-12-27 15:14:52,258][20292] KL-divergence is very high: 299.4830 [2023-12-27 15:14:52,308][20292] KL-divergence is very high: 140.1020 [2023-12-27 15:14:52,314][20381] Updated weights for policy 1, policy_version 1384356 (0.0008) [2023-12-27 15:14:52,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 709181440. Throughput: 0: 10078.3, 1: 10505.2. Samples: 709155056. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:52,363][19170] Avg episode reward: [(0, '931.690'), (1, '970.760')] [2023-12-27 15:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001385528_354762752.pth... [2023-12-27 15:14:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001384344_354459648.pth [2023-12-27 15:14:52,378][20381] Updated weights for policy 1, policy_version 1384366 (0.0008) [2023-12-27 15:14:52,388][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001384368_354426880.pth... [2023-12-27 15:14:52,391][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001383120_354107392.pth [2023-12-27 15:14:52,744][20347] Updated weights for policy 0, policy_version 1385536 (0.0006) [2023-12-27 15:14:52,797][20347] Updated weights for policy 0, policy_version 1385546 (0.0005) [2023-12-27 15:14:52,858][20347] Updated weights for policy 0, policy_version 1385556 (0.0006) [2023-12-27 15:14:53,100][20381] Updated weights for policy 1, policy_version 1384376 (0.0008) [2023-12-27 15:14:53,155][20381] Updated weights for policy 1, policy_version 1384386 (0.0008) [2023-12-27 15:14:53,204][20381] Updated weights for policy 1, policy_version 1384396 (0.0008) [2023-12-27 15:14:53,402][20347] Updated weights for policy 0, policy_version 1385566 (0.0007) [2023-12-27 15:14:53,462][20347] Updated weights for policy 0, policy_version 1385576 (0.0005) [2023-12-27 15:14:53,523][20347] Updated weights for policy 0, policy_version 1385586 (0.0005) [2023-12-27 15:14:53,962][20381] Updated weights for policy 1, policy_version 1384406 (0.0007) [2023-12-27 15:14:54,011][20381] Updated weights for policy 1, policy_version 1384416 (0.0008) [2023-12-27 15:14:54,080][20381] Updated weights for policy 1, policy_version 1384426 (0.0007) [2023-12-27 15:14:54,109][20347] Updated weights for policy 0, policy_version 1385596 (0.0005) [2023-12-27 15:14:54,160][20347] Updated weights for policy 0, policy_version 1385606 (0.0007) [2023-12-27 15:14:54,217][20347] Updated weights for policy 0, policy_version 1385616 (0.0007) [2023-12-27 15:14:54,776][20381] Updated weights for policy 1, policy_version 1384436 (0.0007) [2023-12-27 15:14:54,836][20381] Updated weights for policy 1, policy_version 1384446 (0.0008) [2023-12-27 15:14:54,889][20381] Updated weights for policy 1, policy_version 1384456 (0.0007) [2023-12-27 15:14:54,902][20347] Updated weights for policy 0, policy_version 1385626 (0.0007) [2023-12-27 15:14:54,952][20347] Updated weights for policy 0, policy_version 1385636 (0.0005) [2023-12-27 15:14:55,013][20347] Updated weights for policy 0, policy_version 1385646 (0.0006) [2023-12-27 15:14:55,065][20347] Updated weights for policy 0, policy_version 1385656 (0.0008) [2023-12-27 15:14:55,605][20381] Updated weights for policy 1, policy_version 1384466 (0.0008) [2023-12-27 15:14:55,658][20381] Updated weights for policy 1, policy_version 1384476 (0.0006) [2023-12-27 15:14:55,671][20347] Updated weights for policy 0, policy_version 1385666 (0.0006) [2023-12-27 15:14:55,710][20381] Updated weights for policy 1, policy_version 1384486 (0.0005) [2023-12-27 15:14:55,723][20347] Updated weights for policy 0, policy_version 1385676 (0.0006) [2023-12-27 15:14:55,742][20291] KL-divergence is very high: 170.6709 [2023-12-27 15:14:55,759][20381] Updated weights for policy 1, policy_version 1384496 (0.0005) [2023-12-27 15:14:55,776][20347] Updated weights for policy 0, policy_version 1385686 (0.0006) [2023-12-27 15:14:56,407][20347] Updated weights for policy 0, policy_version 1385696 (0.0006) [2023-12-27 15:14:56,460][20347] Updated weights for policy 0, policy_version 1385706 (0.0006) [2023-12-27 15:14:56,508][20347] Updated weights for policy 0, policy_version 1385716 (0.0006) [2023-12-27 15:14:56,520][20381] Updated weights for policy 1, policy_version 1384506 (0.0006) [2023-12-27 15:14:56,571][20381] Updated weights for policy 1, policy_version 1384516 (0.0005) [2023-12-27 15:14:56,627][20381] Updated weights for policy 1, policy_version 1384526 (0.0005) [2023-12-27 15:14:57,090][20347] Updated weights for policy 0, policy_version 1385726 (0.0006) [2023-12-27 15:14:57,143][20347] Updated weights for policy 0, policy_version 1385736 (0.0005) [2023-12-27 15:14:57,198][20347] Updated weights for policy 0, policy_version 1385746 (0.0009) [2023-12-27 15:14:57,259][20381] Updated weights for policy 1, policy_version 1384536 (0.0008) [2023-12-27 15:14:57,319][20381] Updated weights for policy 1, policy_version 1384546 (0.0008) [2023-12-27 15:14:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 709287936. Throughput: 0: 10167.7, 1: 10359.0. Samples: 709276912. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:14:57,363][19170] Avg episode reward: [(0, '960.860'), (1, '980.510')] [2023-12-27 15:14:57,371][20381] Updated weights for policy 1, policy_version 1384556 (0.0007) [2023-12-27 15:14:57,800][20347] Updated weights for policy 0, policy_version 1385756 (0.0010) [2023-12-27 15:14:57,864][20347] Updated weights for policy 0, policy_version 1385766 (0.0010) [2023-12-27 15:14:57,912][20347] Updated weights for policy 0, policy_version 1385776 (0.0010) [2023-12-27 15:14:57,929][20381] Updated weights for policy 1, policy_version 1384566 (0.0006) [2023-12-27 15:14:57,972][20381] Updated weights for policy 1, policy_version 1384576 (0.0005) [2023-12-27 15:14:58,028][20381] Updated weights for policy 1, policy_version 1384586 (0.0008) [2023-12-27 15:14:58,567][20347] Updated weights for policy 0, policy_version 1385786 (0.0007) [2023-12-27 15:14:58,621][20347] Updated weights for policy 0, policy_version 1385796 (0.0007) [2023-12-27 15:14:58,688][20347] Updated weights for policy 0, policy_version 1385806 (0.0008) [2023-12-27 15:14:58,756][20347] Updated weights for policy 0, policy_version 1385816 (0.0008) [2023-12-27 15:14:58,795][20381] Updated weights for policy 1, policy_version 1384596 (0.0007) [2023-12-27 15:14:58,864][20381] Updated weights for policy 1, policy_version 1384606 (0.0008) [2023-12-27 15:14:58,923][20381] Updated weights for policy 1, policy_version 1384616 (0.0010) [2023-12-27 15:14:59,523][20347] Updated weights for policy 0, policy_version 1385826 (0.0005) [2023-12-27 15:14:59,574][20347] Updated weights for policy 0, policy_version 1385836 (0.0007) [2023-12-27 15:14:59,633][20347] Updated weights for policy 0, policy_version 1385846 (0.0007) [2023-12-27 15:14:59,718][20381] Updated weights for policy 1, policy_version 1384626 (0.0011) [2023-12-27 15:14:59,769][20381] Updated weights for policy 1, policy_version 1384636 (0.0005) [2023-12-27 15:14:59,820][20381] Updated weights for policy 1, policy_version 1384646 (0.0008) [2023-12-27 15:14:59,886][20381] Updated weights for policy 1, policy_version 1384656 (0.0008) [2023-12-27 15:15:00,306][20347] Updated weights for policy 0, policy_version 1385856 (0.0006) [2023-12-27 15:15:00,366][20347] Updated weights for policy 0, policy_version 1385866 (0.0005) [2023-12-27 15:15:00,423][20347] Updated weights for policy 0, policy_version 1385876 (0.0006) [2023-12-27 15:15:00,487][20381] Updated weights for policy 1, policy_version 1384666 (0.0009) [2023-12-27 15:15:00,544][20381] Updated weights for policy 1, policy_version 1384676 (0.0005) [2023-12-27 15:15:00,601][20381] Updated weights for policy 1, policy_version 1384686 (0.0006) [2023-12-27 15:15:01,030][20347] Updated weights for policy 0, policy_version 1385886 (0.0007) [2023-12-27 15:15:01,091][20347] Updated weights for policy 0, policy_version 1385896 (0.0007) [2023-12-27 15:15:01,152][20347] Updated weights for policy 0, policy_version 1385906 (0.0007) [2023-12-27 15:15:01,254][20381] Updated weights for policy 1, policy_version 1384696 (0.0008) [2023-12-27 15:15:01,321][20381] Updated weights for policy 1, policy_version 1384706 (0.0009) [2023-12-27 15:15:01,384][20381] Updated weights for policy 1, policy_version 1384716 (0.0008) [2023-12-27 15:15:01,848][20347] Updated weights for policy 0, policy_version 1385916 (0.0007) [2023-12-27 15:15:01,899][20347] Updated weights for policy 0, policy_version 1385926 (0.0006) [2023-12-27 15:15:01,948][20347] Updated weights for policy 0, policy_version 1385936 (0.0005) [2023-12-27 15:15:02,090][20381] Updated weights for policy 1, policy_version 1384726 (0.0008) [2023-12-27 15:15:02,100][20292] KL-divergence is very high: 262.3985 [2023-12-27 15:15:02,138][20381] Updated weights for policy 1, policy_version 1384736 (0.0007) [2023-12-27 15:15:02,139][20292] KL-divergence is very high: 171.8709 [2023-12-27 15:15:02,197][20381] Updated weights for policy 1, policy_version 1384746 (0.0007) [2023-12-27 15:15:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 709394432. Throughput: 0: 10190.9, 1: 10345.9. Samples: 709401208. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:02,363][19170] Avg episode reward: [(0, '941.710'), (1, '970.860')] [2023-12-27 15:15:02,597][20347] Updated weights for policy 0, policy_version 1385946 (0.0006) [2023-12-27 15:15:02,653][20347] Updated weights for policy 0, policy_version 1385956 (0.0008) [2023-12-27 15:15:02,711][20347] Updated weights for policy 0, policy_version 1385966 (0.0007) [2023-12-27 15:15:02,743][20291] KL-divergence is very high: 129.4940 [2023-12-27 15:15:02,766][20347] Updated weights for policy 0, policy_version 1385976 (0.0007) [2023-12-27 15:15:02,934][20381] Updated weights for policy 1, policy_version 1384756 (0.0007) [2023-12-27 15:15:02,987][20381] Updated weights for policy 1, policy_version 1384766 (0.0007) [2023-12-27 15:15:03,035][20381] Updated weights for policy 1, policy_version 1384776 (0.0007) [2023-12-27 15:15:03,429][20347] Updated weights for policy 0, policy_version 1385986 (0.0005) [2023-12-27 15:15:03,482][20347] Updated weights for policy 0, policy_version 1385996 (0.0005) [2023-12-27 15:15:03,534][20347] Updated weights for policy 0, policy_version 1386006 (0.0005) [2023-12-27 15:15:03,728][20381] Updated weights for policy 1, policy_version 1384786 (0.0007) [2023-12-27 15:15:03,782][20381] Updated weights for policy 1, policy_version 1384796 (0.0010) [2023-12-27 15:15:03,837][20381] Updated weights for policy 1, policy_version 1384806 (0.0010) [2023-12-27 15:15:03,899][20381] Updated weights for policy 1, policy_version 1384816 (0.0009) [2023-12-27 15:15:04,174][20347] Updated weights for policy 0, policy_version 1386016 (0.0008) [2023-12-27 15:15:04,235][20347] Updated weights for policy 0, policy_version 1386026 (0.0007) [2023-12-27 15:15:04,298][20347] Updated weights for policy 0, policy_version 1386036 (0.0007) [2023-12-27 15:15:04,570][20381] Updated weights for policy 1, policy_version 1384826 (0.0007) [2023-12-27 15:15:04,624][20381] Updated weights for policy 1, policy_version 1384836 (0.0006) [2023-12-27 15:15:04,669][20381] Updated weights for policy 1, policy_version 1384846 (0.0005) [2023-12-27 15:15:04,999][20347] Updated weights for policy 0, policy_version 1386046 (0.0009) [2023-12-27 15:15:05,058][20347] Updated weights for policy 0, policy_version 1386056 (0.0010) [2023-12-27 15:15:05,119][20347] Updated weights for policy 0, policy_version 1386066 (0.0010) [2023-12-27 15:15:05,343][20381] Updated weights for policy 1, policy_version 1384856 (0.0007) [2023-12-27 15:15:05,400][20381] Updated weights for policy 1, policy_version 1384866 (0.0005) [2023-12-27 15:15:05,457][20381] Updated weights for policy 1, policy_version 1384876 (0.0007) [2023-12-27 15:15:05,791][20347] Updated weights for policy 0, policy_version 1386076 (0.0010) [2023-12-27 15:15:05,854][20347] Updated weights for policy 0, policy_version 1386086 (0.0010) [2023-12-27 15:15:05,909][20347] Updated weights for policy 0, policy_version 1386096 (0.0010) [2023-12-27 15:15:06,067][20381] Updated weights for policy 1, policy_version 1384886 (0.0007) [2023-12-27 15:15:06,130][20381] Updated weights for policy 1, policy_version 1384896 (0.0008) [2023-12-27 15:15:06,200][20381] Updated weights for policy 1, policy_version 1384906 (0.0008) [2023-12-27 15:15:06,635][20347] Updated weights for policy 0, policy_version 1386106 (0.0010) [2023-12-27 15:15:06,696][20347] Updated weights for policy 0, policy_version 1386116 (0.0006) [2023-12-27 15:15:06,756][20347] Updated weights for policy 0, policy_version 1386126 (0.0009) [2023-12-27 15:15:06,817][20347] Updated weights for policy 0, policy_version 1386136 (0.0005) [2023-12-27 15:15:06,851][20381] Updated weights for policy 1, policy_version 1384916 (0.0009) [2023-12-27 15:15:06,899][20381] Updated weights for policy 1, policy_version 1384926 (0.0006) [2023-12-27 15:15:06,956][20381] Updated weights for policy 1, policy_version 1384936 (0.0005) [2023-12-27 15:15:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 709492736. Throughput: 0: 10173.6, 1: 10341.5. Samples: 709462984. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:07,363][19170] Avg episode reward: [(0, '961.290'), (1, '970.860')] [2023-12-27 15:15:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001384944_354574336.pth... [2023-12-27 15:15:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001383760_354271232.pth [2023-12-27 15:15:07,439][20347] Updated weights for policy 0, policy_version 1386146 (0.0006) [2023-12-27 15:15:07,495][20347] Updated weights for policy 0, policy_version 1386156 (0.0007) [2023-12-27 15:15:07,496][20381] Updated weights for policy 1, policy_version 1384946 (0.0006) [2023-12-27 15:15:07,545][20381] Updated weights for policy 1, policy_version 1384956 (0.0010) [2023-12-27 15:15:07,557][20347] Updated weights for policy 0, policy_version 1386166 (0.0006) [2023-12-27 15:15:07,566][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001386168_354926592.pth... [2023-12-27 15:15:07,571][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001384952_354615296.pth [2023-12-27 15:15:07,600][20381] Updated weights for policy 1, policy_version 1384966 (0.0008) [2023-12-27 15:15:07,659][20381] Updated weights for policy 1, policy_version 1384976 (0.0005) [2023-12-27 15:15:08,133][20347] Updated weights for policy 0, policy_version 1386176 (0.0006) [2023-12-27 15:15:08,197][20347] Updated weights for policy 0, policy_version 1386186 (0.0005) [2023-12-27 15:15:08,260][20347] Updated weights for policy 0, policy_version 1386196 (0.0005) [2023-12-27 15:15:08,315][20381] Updated weights for policy 1, policy_version 1384986 (0.0011) [2023-12-27 15:15:08,376][20381] Updated weights for policy 1, policy_version 1384996 (0.0008) [2023-12-27 15:15:08,434][20381] Updated weights for policy 1, policy_version 1385006 (0.0007) [2023-12-27 15:15:08,799][20347] Updated weights for policy 0, policy_version 1386206 (0.0008) [2023-12-27 15:15:08,860][20347] Updated weights for policy 0, policy_version 1386216 (0.0007) [2023-12-27 15:15:08,921][20347] Updated weights for policy 0, policy_version 1386226 (0.0006) [2023-12-27 15:15:09,082][20381] Updated weights for policy 1, policy_version 1385016 (0.0010) [2023-12-27 15:15:09,135][20381] Updated weights for policy 1, policy_version 1385026 (0.0010) [2023-12-27 15:15:09,192][20381] Updated weights for policy 1, policy_version 1385036 (0.0007) [2023-12-27 15:15:09,616][20347] Updated weights for policy 0, policy_version 1386236 (0.0006) [2023-12-27 15:15:09,681][20347] Updated weights for policy 0, policy_version 1386246 (0.0005) [2023-12-27 15:15:09,745][20347] Updated weights for policy 0, policy_version 1386256 (0.0007) [2023-12-27 15:15:09,959][20381] Updated weights for policy 1, policy_version 1385046 (0.0008) [2023-12-27 15:15:10,014][20381] Updated weights for policy 1, policy_version 1385056 (0.0007) [2023-12-27 15:15:10,077][20381] Updated weights for policy 1, policy_version 1385066 (0.0008) [2023-12-27 15:15:10,407][20347] Updated weights for policy 0, policy_version 1386266 (0.0008) [2023-12-27 15:15:10,464][20347] Updated weights for policy 0, policy_version 1386276 (0.0007) [2023-12-27 15:15:10,521][20347] Updated weights for policy 0, policy_version 1386286 (0.0008) [2023-12-27 15:15:10,584][20347] Updated weights for policy 0, policy_version 1386296 (0.0008) [2023-12-27 15:15:10,780][20381] Updated weights for policy 1, policy_version 1385076 (0.0008) [2023-12-27 15:15:10,832][20381] Updated weights for policy 1, policy_version 1385086 (0.0006) [2023-12-27 15:15:10,889][20381] Updated weights for policy 1, policy_version 1385096 (0.0005) [2023-12-27 15:15:11,309][20347] Updated weights for policy 0, policy_version 1386306 (0.0006) [2023-12-27 15:15:11,386][20347] Updated weights for policy 0, policy_version 1386316 (0.0007) [2023-12-27 15:15:11,450][20347] Updated weights for policy 0, policy_version 1386326 (0.0006) [2023-12-27 15:15:11,575][20381] Updated weights for policy 1, policy_version 1385106 (0.0005) [2023-12-27 15:15:11,644][20381] Updated weights for policy 1, policy_version 1385116 (0.0008) [2023-12-27 15:15:11,707][20381] Updated weights for policy 1, policy_version 1385126 (0.0007) [2023-12-27 15:15:11,779][20381] Updated weights for policy 1, policy_version 1385136 (0.0008) [2023-12-27 15:15:12,109][20347] Updated weights for policy 0, policy_version 1386336 (0.0007) [2023-12-27 15:15:12,171][20347] Updated weights for policy 0, policy_version 1386346 (0.0007) [2023-12-27 15:15:12,230][20347] Updated weights for policy 0, policy_version 1386356 (0.0008) [2023-12-27 15:15:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 709599232. Throughput: 0: 10275.4, 1: 10329.2. Samples: 709588344. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:12,363][19170] Avg episode reward: [(0, '990.320'), (1, '991.200')] [2023-12-27 15:15:12,473][20381] Updated weights for policy 1, policy_version 1385146 (0.0008) [2023-12-27 15:15:12,535][20381] Updated weights for policy 1, policy_version 1385156 (0.0007) [2023-12-27 15:15:12,594][20381] Updated weights for policy 1, policy_version 1385166 (0.0008) [2023-12-27 15:15:12,928][20347] Updated weights for policy 0, policy_version 1386366 (0.0006) [2023-12-27 15:15:12,984][20347] Updated weights for policy 0, policy_version 1386376 (0.0006) [2023-12-27 15:15:13,043][20347] Updated weights for policy 0, policy_version 1386386 (0.0008) [2023-12-27 15:15:13,295][20381] Updated weights for policy 1, policy_version 1385176 (0.0008) [2023-12-27 15:15:13,350][20381] Updated weights for policy 1, policy_version 1385186 (0.0007) [2023-12-27 15:15:13,405][20381] Updated weights for policy 1, policy_version 1385196 (0.0007) [2023-12-27 15:15:13,664][20347] Updated weights for policy 0, policy_version 1386396 (0.0008) [2023-12-27 15:15:13,722][20347] Updated weights for policy 0, policy_version 1386406 (0.0006) [2023-12-27 15:15:13,784][20347] Updated weights for policy 0, policy_version 1386416 (0.0005) [2023-12-27 15:15:14,089][20381] Updated weights for policy 1, policy_version 1385206 (0.0009) [2023-12-27 15:15:14,141][20381] Updated weights for policy 1, policy_version 1385216 (0.0008) [2023-12-27 15:15:14,198][20381] Updated weights for policy 1, policy_version 1385226 (0.0009) [2023-12-27 15:15:14,380][20347] Updated weights for policy 0, policy_version 1386426 (0.0008) [2023-12-27 15:15:14,449][20347] Updated weights for policy 0, policy_version 1386436 (0.0009) [2023-12-27 15:15:14,501][20347] Updated weights for policy 0, policy_version 1386446 (0.0010) [2023-12-27 15:15:14,549][20347] Updated weights for policy 0, policy_version 1386456 (0.0010) [2023-12-27 15:15:14,810][20381] Updated weights for policy 1, policy_version 1385236 (0.0010) [2023-12-27 15:15:14,869][20381] Updated weights for policy 1, policy_version 1385246 (0.0011) [2023-12-27 15:15:14,928][20381] Updated weights for policy 1, policy_version 1385256 (0.0011) [2023-12-27 15:15:15,248][20347] Updated weights for policy 0, policy_version 1386466 (0.0011) [2023-12-27 15:15:15,313][20347] Updated weights for policy 0, policy_version 1386476 (0.0010) [2023-12-27 15:15:15,376][20347] Updated weights for policy 0, policy_version 1386486 (0.0010) [2023-12-27 15:15:15,552][20381] Updated weights for policy 1, policy_version 1385266 (0.0011) [2023-12-27 15:15:15,598][20381] Updated weights for policy 1, policy_version 1385276 (0.0011) [2023-12-27 15:15:15,650][20381] Updated weights for policy 1, policy_version 1385286 (0.0010) [2023-12-27 15:15:15,708][20381] Updated weights for policy 1, policy_version 1385296 (0.0010) [2023-12-27 15:15:16,055][20347] Updated weights for policy 0, policy_version 1386496 (0.0007) [2023-12-27 15:15:16,126][20347] Updated weights for policy 0, policy_version 1386506 (0.0006) [2023-12-27 15:15:16,183][20347] Updated weights for policy 0, policy_version 1386516 (0.0008) [2023-12-27 15:15:16,345][20381] Updated weights for policy 1, policy_version 1385306 (0.0007) [2023-12-27 15:15:16,393][20381] Updated weights for policy 1, policy_version 1385316 (0.0005) [2023-12-27 15:15:16,438][20381] Updated weights for policy 1, policy_version 1385326 (0.0007) [2023-12-27 15:15:16,850][20347] Updated weights for policy 0, policy_version 1386526 (0.0007) [2023-12-27 15:15:16,896][20347] Updated weights for policy 0, policy_version 1386536 (0.0008) [2023-12-27 15:15:16,945][20347] Updated weights for policy 0, policy_version 1386546 (0.0007) [2023-12-27 15:15:17,056][20381] Updated weights for policy 1, policy_version 1385336 (0.0008) [2023-12-27 15:15:17,105][20381] Updated weights for policy 1, policy_version 1385346 (0.0007) [2023-12-27 15:15:17,163][20381] Updated weights for policy 1, policy_version 1385356 (0.0008) [2023-12-27 15:15:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 709705728. Throughput: 0: 10327.2, 1: 10316.7. Samples: 709712264. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '991.200')] [2023-12-27 15:15:17,642][20347] Updated weights for policy 0, policy_version 1386556 (0.0007) [2023-12-27 15:15:17,694][20347] Updated weights for policy 0, policy_version 1386566 (0.0005) [2023-12-27 15:15:17,741][20347] Updated weights for policy 0, policy_version 1386576 (0.0010) [2023-12-27 15:15:17,743][20381] Updated weights for policy 1, policy_version 1385366 (0.0006) [2023-12-27 15:15:17,801][20381] Updated weights for policy 1, policy_version 1385376 (0.0006) [2023-12-27 15:15:17,863][20381] Updated weights for policy 1, policy_version 1385386 (0.0006) [2023-12-27 15:15:18,440][20347] Updated weights for policy 0, policy_version 1386586 (0.0010) [2023-12-27 15:15:18,501][20347] Updated weights for policy 0, policy_version 1386596 (0.0010) [2023-12-27 15:15:18,539][20381] Updated weights for policy 1, policy_version 1385396 (0.0006) [2023-12-27 15:15:18,560][20347] Updated weights for policy 0, policy_version 1386606 (0.0011) [2023-12-27 15:15:18,594][20381] Updated weights for policy 1, policy_version 1385406 (0.0005) [2023-12-27 15:15:18,624][20347] Updated weights for policy 0, policy_version 1386616 (0.0010) [2023-12-27 15:15:18,679][20381] Updated weights for policy 1, policy_version 1385416 (0.0007) [2023-12-27 15:15:19,289][20347] Updated weights for policy 0, policy_version 1386626 (0.0009) [2023-12-27 15:15:19,350][20347] Updated weights for policy 0, policy_version 1386636 (0.0010) [2023-12-27 15:15:19,350][20381] Updated weights for policy 1, policy_version 1385426 (0.0008) [2023-12-27 15:15:19,408][20347] Updated weights for policy 0, policy_version 1386646 (0.0008) [2023-12-27 15:15:19,409][20381] Updated weights for policy 1, policy_version 1385436 (0.0009) [2023-12-27 15:15:19,464][20381] Updated weights for policy 1, policy_version 1385446 (0.0008) [2023-12-27 15:15:19,524][20381] Updated weights for policy 1, policy_version 1385456 (0.0008) [2023-12-27 15:15:20,134][20347] Updated weights for policy 0, policy_version 1386656 (0.0010) [2023-12-27 15:15:20,188][20347] Updated weights for policy 0, policy_version 1386666 (0.0011) [2023-12-27 15:15:20,242][20347] Updated weights for policy 0, policy_version 1386676 (0.0011) [2023-12-27 15:15:20,254][20381] Updated weights for policy 1, policy_version 1385466 (0.0006) [2023-12-27 15:15:20,309][20381] Updated weights for policy 1, policy_version 1385476 (0.0007) [2023-12-27 15:15:20,368][20381] Updated weights for policy 1, policy_version 1385486 (0.0007) [2023-12-27 15:15:20,981][20347] Updated weights for policy 0, policy_version 1386686 (0.0011) [2023-12-27 15:15:21,047][20347] Updated weights for policy 0, policy_version 1386696 (0.0011) [2023-12-27 15:15:21,100][20381] Updated weights for policy 1, policy_version 1385496 (0.0011) [2023-12-27 15:15:21,110][20347] Updated weights for policy 0, policy_version 1386706 (0.0009) [2023-12-27 15:15:21,164][20381] Updated weights for policy 1, policy_version 1385506 (0.0010) [2023-12-27 15:15:21,225][20381] Updated weights for policy 1, policy_version 1385516 (0.0011) [2023-12-27 15:15:21,863][20347] Updated weights for policy 0, policy_version 1386716 (0.0007) [2023-12-27 15:15:21,929][20347] Updated weights for policy 0, policy_version 1386726 (0.0008) [2023-12-27 15:15:21,949][20381] Updated weights for policy 1, policy_version 1385526 (0.0008) [2023-12-27 15:15:21,985][20347] Updated weights for policy 0, policy_version 1386736 (0.0007) [2023-12-27 15:15:22,014][20381] Updated weights for policy 1, policy_version 1385536 (0.0006) [2023-12-27 15:15:22,084][20381] Updated weights for policy 1, policy_version 1385546 (0.0007) [2023-12-27 15:15:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 709804032. Throughput: 0: 10296.4, 1: 10338.1. Samples: 709774216. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:22,363][19170] Avg episode reward: [(0, '990.200'), (1, '970.870')] [2023-12-27 15:15:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001385552_354729984.pth... [2023-12-27 15:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001386744_355074048.pth... [2023-12-27 15:15:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001385528_354762752.pth [2023-12-27 15:15:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001384368_354426880.pth [2023-12-27 15:15:22,719][20347] Updated weights for policy 0, policy_version 1386746 (0.0006) [2023-12-27 15:15:22,772][20381] Updated weights for policy 1, policy_version 1385556 (0.0008) [2023-12-27 15:15:22,779][20347] Updated weights for policy 0, policy_version 1386756 (0.0007) [2023-12-27 15:15:22,827][20381] Updated weights for policy 1, policy_version 1385566 (0.0006) [2023-12-27 15:15:22,843][20347] Updated weights for policy 0, policy_version 1386766 (0.0009) [2023-12-27 15:15:22,892][20381] Updated weights for policy 1, policy_version 1385576 (0.0006) [2023-12-27 15:15:22,905][20347] Updated weights for policy 0, policy_version 1386776 (0.0007) [2023-12-27 15:15:23,567][20381] Updated weights for policy 1, policy_version 1385586 (0.0006) [2023-12-27 15:15:23,587][20347] Updated weights for policy 0, policy_version 1386786 (0.0007) [2023-12-27 15:15:23,625][20381] Updated weights for policy 1, policy_version 1385596 (0.0007) [2023-12-27 15:15:23,651][20347] Updated weights for policy 0, policy_version 1386796 (0.0006) [2023-12-27 15:15:23,676][20381] Updated weights for policy 1, policy_version 1385606 (0.0007) [2023-12-27 15:15:23,706][20347] Updated weights for policy 0, policy_version 1386806 (0.0005) [2023-12-27 15:15:23,726][20381] Updated weights for policy 1, policy_version 1385616 (0.0008) [2023-12-27 15:15:24,370][20381] Updated weights for policy 1, policy_version 1385626 (0.0005) [2023-12-27 15:15:24,397][20347] Updated weights for policy 0, policy_version 1386816 (0.0008) [2023-12-27 15:15:24,424][20381] Updated weights for policy 1, policy_version 1385636 (0.0006) [2023-12-27 15:15:24,446][20347] Updated weights for policy 0, policy_version 1386826 (0.0008) [2023-12-27 15:15:24,476][20381] Updated weights for policy 1, policy_version 1385646 (0.0007) [2023-12-27 15:15:24,493][20347] Updated weights for policy 0, policy_version 1386836 (0.0005) [2023-12-27 15:15:25,128][20381] Updated weights for policy 1, policy_version 1385656 (0.0008) [2023-12-27 15:15:25,155][20347] Updated weights for policy 0, policy_version 1386846 (0.0005) [2023-12-27 15:15:25,186][20381] Updated weights for policy 1, policy_version 1385666 (0.0006) [2023-12-27 15:15:25,210][20347] Updated weights for policy 0, policy_version 1386856 (0.0006) [2023-12-27 15:15:25,241][20381] Updated weights for policy 1, policy_version 1385676 (0.0006) [2023-12-27 15:15:25,258][20347] Updated weights for policy 0, policy_version 1386866 (0.0005) [2023-12-27 15:15:25,931][20381] Updated weights for policy 1, policy_version 1385686 (0.0007) [2023-12-27 15:15:25,965][20347] Updated weights for policy 0, policy_version 1386876 (0.0008) [2023-12-27 15:15:25,990][20381] Updated weights for policy 1, policy_version 1385696 (0.0007) [2023-12-27 15:15:26,035][20347] Updated weights for policy 0, policy_version 1386886 (0.0006) [2023-12-27 15:15:26,048][20381] Updated weights for policy 1, policy_version 1385706 (0.0008) [2023-12-27 15:15:26,090][20347] Updated weights for policy 0, policy_version 1386896 (0.0006) [2023-12-27 15:15:26,719][20381] Updated weights for policy 1, policy_version 1385716 (0.0006) [2023-12-27 15:15:26,758][20347] Updated weights for policy 0, policy_version 1386906 (0.0008) [2023-12-27 15:15:26,771][20381] Updated weights for policy 1, policy_version 1385726 (0.0007) [2023-12-27 15:15:26,812][20347] Updated weights for policy 0, policy_version 1386916 (0.0007) [2023-12-27 15:15:26,825][20381] Updated weights for policy 1, policy_version 1385736 (0.0006) [2023-12-27 15:15:26,870][20347] Updated weights for policy 0, policy_version 1386926 (0.0007) [2023-12-27 15:15:26,925][20347] Updated weights for policy 0, policy_version 1386936 (0.0008) [2023-12-27 15:15:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 709902336. Throughput: 0: 10210.2, 1: 10271.9. Samples: 709893208. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:27,363][19170] Avg episode reward: [(0, '980.450'), (1, '961.140')] [2023-12-27 15:15:27,537][20347] Updated weights for policy 0, policy_version 1386946 (0.0006) [2023-12-27 15:15:27,539][20381] Updated weights for policy 1, policy_version 1385746 (0.0005) [2023-12-27 15:15:27,584][20347] Updated weights for policy 0, policy_version 1386956 (0.0006) [2023-12-27 15:15:27,590][20381] Updated weights for policy 1, policy_version 1385756 (0.0005) [2023-12-27 15:15:27,635][20347] Updated weights for policy 0, policy_version 1386966 (0.0005) [2023-12-27 15:15:27,641][20381] Updated weights for policy 1, policy_version 1385766 (0.0005) [2023-12-27 15:15:27,694][20381] Updated weights for policy 1, policy_version 1385776 (0.0007) [2023-12-27 15:15:28,272][20347] Updated weights for policy 0, policy_version 1386976 (0.0007) [2023-12-27 15:15:28,326][20347] Updated weights for policy 0, policy_version 1386986 (0.0007) [2023-12-27 15:15:28,386][20347] Updated weights for policy 0, policy_version 1386996 (0.0006) [2023-12-27 15:15:28,397][20381] Updated weights for policy 1, policy_version 1385786 (0.0007) [2023-12-27 15:15:28,459][20381] Updated weights for policy 1, policy_version 1385796 (0.0008) [2023-12-27 15:15:28,515][20381] Updated weights for policy 1, policy_version 1385806 (0.0007) [2023-12-27 15:15:29,009][20347] Updated weights for policy 0, policy_version 1387006 (0.0008) [2023-12-27 15:15:29,057][20347] Updated weights for policy 0, policy_version 1387016 (0.0008) [2023-12-27 15:15:29,107][20347] Updated weights for policy 0, policy_version 1387026 (0.0008) [2023-12-27 15:15:29,150][20381] Updated weights for policy 1, policy_version 1385816 (0.0007) [2023-12-27 15:15:29,209][20381] Updated weights for policy 1, policy_version 1385826 (0.0008) [2023-12-27 15:15:29,273][20381] Updated weights for policy 1, policy_version 1385836 (0.0008) [2023-12-27 15:15:29,801][20347] Updated weights for policy 0, policy_version 1387036 (0.0007) [2023-12-27 15:15:29,858][20347] Updated weights for policy 0, policy_version 1387046 (0.0008) [2023-12-27 15:15:29,920][20347] Updated weights for policy 0, policy_version 1387056 (0.0008) [2023-12-27 15:15:30,025][20381] Updated weights for policy 1, policy_version 1385846 (0.0007) [2023-12-27 15:15:30,079][20381] Updated weights for policy 1, policy_version 1385856 (0.0007) [2023-12-27 15:15:30,145][20381] Updated weights for policy 1, policy_version 1385866 (0.0007) [2023-12-27 15:15:30,579][20347] Updated weights for policy 0, policy_version 1387066 (0.0006) [2023-12-27 15:15:30,622][20347] Updated weights for policy 0, policy_version 1387076 (0.0005) [2023-12-27 15:15:30,675][20347] Updated weights for policy 0, policy_version 1387086 (0.0006) [2023-12-27 15:15:30,719][20347] Updated weights for policy 0, policy_version 1387096 (0.0007) [2023-12-27 15:15:30,840][20381] Updated weights for policy 1, policy_version 1385876 (0.0007) [2023-12-27 15:15:30,896][20381] Updated weights for policy 1, policy_version 1385886 (0.0007) [2023-12-27 15:15:30,961][20381] Updated weights for policy 1, policy_version 1385897 (0.0006) [2023-12-27 15:15:31,386][20347] Updated weights for policy 0, policy_version 1387106 (0.0009) [2023-12-27 15:15:31,449][20347] Updated weights for policy 0, policy_version 1387116 (0.0008) [2023-12-27 15:15:31,511][20347] Updated weights for policy 0, policy_version 1387126 (0.0005) [2023-12-27 15:15:31,652][20381] Updated weights for policy 1, policy_version 1385907 (0.0009) [2023-12-27 15:15:31,714][20381] Updated weights for policy 1, policy_version 1385917 (0.0010) [2023-12-27 15:15:31,775][20381] Updated weights for policy 1, policy_version 1385927 (0.0009) [2023-12-27 15:15:32,217][20347] Updated weights for policy 0, policy_version 1387136 (0.0007) [2023-12-27 15:15:32,268][20347] Updated weights for policy 0, policy_version 1387146 (0.0009) [2023-12-27 15:15:32,332][20347] Updated weights for policy 0, policy_version 1387156 (0.0007) [2023-12-27 15:15:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 710008832. Throughput: 0: 10293.7, 1: 10150.8. Samples: 710015624. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:32,363][19170] Avg episode reward: [(0, '980.470'), (1, '941.660')] [2023-12-27 15:15:32,436][20381] Updated weights for policy 1, policy_version 1385937 (0.0007) [2023-12-27 15:15:32,506][20381] Updated weights for policy 1, policy_version 1385947 (0.0008) [2023-12-27 15:15:32,573][20381] Updated weights for policy 1, policy_version 1385957 (0.0008) [2023-12-27 15:15:32,637][20381] Updated weights for policy 1, policy_version 1385967 (0.0008) [2023-12-27 15:15:33,006][20347] Updated weights for policy 0, policy_version 1387166 (0.0006) [2023-12-27 15:15:33,061][20347] Updated weights for policy 0, policy_version 1387176 (0.0007) [2023-12-27 15:15:33,109][20347] Updated weights for policy 0, policy_version 1387186 (0.0008) [2023-12-27 15:15:33,174][20381] Updated weights for policy 1, policy_version 1385977 (0.0007) [2023-12-27 15:15:33,229][20381] Updated weights for policy 1, policy_version 1385987 (0.0007) [2023-12-27 15:15:33,282][20381] Updated weights for policy 1, policy_version 1385997 (0.0008) [2023-12-27 15:15:33,785][20347] Updated weights for policy 0, policy_version 1387196 (0.0007) [2023-12-27 15:15:33,835][20347] Updated weights for policy 0, policy_version 1387206 (0.0007) [2023-12-27 15:15:33,887][20347] Updated weights for policy 0, policy_version 1387216 (0.0007) [2023-12-27 15:15:33,968][20381] Updated weights for policy 1, policy_version 1386007 (0.0008) [2023-12-27 15:15:34,024][20381] Updated weights for policy 1, policy_version 1386017 (0.0008) [2023-12-27 15:15:34,076][20381] Updated weights for policy 1, policy_version 1386027 (0.0006) [2023-12-27 15:15:34,619][20347] Updated weights for policy 0, policy_version 1387226 (0.0007) [2023-12-27 15:15:34,676][20347] Updated weights for policy 0, policy_version 1387236 (0.0007) [2023-12-27 15:15:34,737][20347] Updated weights for policy 0, policy_version 1387246 (0.0007) [2023-12-27 15:15:34,791][20347] Updated weights for policy 0, policy_version 1387256 (0.0006) [2023-12-27 15:15:34,797][20381] Updated weights for policy 1, policy_version 1386037 (0.0006) [2023-12-27 15:15:34,855][20381] Updated weights for policy 1, policy_version 1386047 (0.0008) [2023-12-27 15:15:34,906][20381] Updated weights for policy 1, policy_version 1386057 (0.0008) [2023-12-27 15:15:35,519][20381] Updated weights for policy 1, policy_version 1386067 (0.0008) [2023-12-27 15:15:35,531][20347] Updated weights for policy 0, policy_version 1387266 (0.0008) [2023-12-27 15:15:35,578][20381] Updated weights for policy 1, policy_version 1386077 (0.0007) [2023-12-27 15:15:35,589][20347] Updated weights for policy 0, policy_version 1387276 (0.0008) [2023-12-27 15:15:35,633][20381] Updated weights for policy 1, policy_version 1386087 (0.0010) [2023-12-27 15:15:35,647][20347] Updated weights for policy 0, policy_version 1387286 (0.0006) [2023-12-27 15:15:36,281][20381] Updated weights for policy 1, policy_version 1386097 (0.0010) [2023-12-27 15:15:36,344][20381] Updated weights for policy 1, policy_version 1386107 (0.0010) [2023-12-27 15:15:36,374][20347] Updated weights for policy 0, policy_version 1387296 (0.0007) [2023-12-27 15:15:36,406][20381] Updated weights for policy 1, policy_version 1386117 (0.0010) [2023-12-27 15:15:36,435][20347] Updated weights for policy 0, policy_version 1387306 (0.0009) [2023-12-27 15:15:36,467][20381] Updated weights for policy 1, policy_version 1386127 (0.0011) [2023-12-27 15:15:36,491][20347] Updated weights for policy 0, policy_version 1387316 (0.0009) [2023-12-27 15:15:37,146][20381] Updated weights for policy 1, policy_version 1386137 (0.0011) [2023-12-27 15:15:37,202][20381] Updated weights for policy 1, policy_version 1386147 (0.0011) [2023-12-27 15:15:37,238][20347] Updated weights for policy 0, policy_version 1387326 (0.0010) [2023-12-27 15:15:37,252][20381] Updated weights for policy 1, policy_version 1386157 (0.0011) [2023-12-27 15:15:37,290][20347] Updated weights for policy 0, policy_version 1387336 (0.0010) [2023-12-27 15:15:37,352][20347] Updated weights for policy 0, policy_version 1387346 (0.0010) [2023-12-27 15:15:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 710107136. Throughput: 0: 10309.3, 1: 10174.9. Samples: 710076844. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:37,363][19170] Avg episode reward: [(0, '970.840'), (1, '931.900')] [2023-12-27 15:15:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001386160_354885632.pth... [2023-12-27 15:15:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001384944_354574336.pth [2023-12-27 15:15:37,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001387352_355229696.pth... [2023-12-27 15:15:37,393][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001386168_354926592.pth [2023-12-27 15:15:37,968][20381] Updated weights for policy 1, policy_version 1386167 (0.0010) [2023-12-27 15:15:38,034][20381] Updated weights for policy 1, policy_version 1386177 (0.0010) [2023-12-27 15:15:38,062][20347] Updated weights for policy 0, policy_version 1387356 (0.0010) [2023-12-27 15:15:38,090][20381] Updated weights for policy 1, policy_version 1386187 (0.0009) [2023-12-27 15:15:38,114][20347] Updated weights for policy 0, policy_version 1387366 (0.0008) [2023-12-27 15:15:38,166][20347] Updated weights for policy 0, policy_version 1387376 (0.0007) [2023-12-27 15:15:38,793][20381] Updated weights for policy 1, policy_version 1386197 (0.0009) [2023-12-27 15:15:38,857][20381] Updated weights for policy 1, policy_version 1386207 (0.0010) [2023-12-27 15:15:38,885][20347] Updated weights for policy 0, policy_version 1387386 (0.0007) [2023-12-27 15:15:38,925][20381] Updated weights for policy 1, policy_version 1386217 (0.0010) [2023-12-27 15:15:38,949][20347] Updated weights for policy 0, policy_version 1387396 (0.0010) [2023-12-27 15:15:39,005][20347] Updated weights for policy 0, policy_version 1387406 (0.0011) [2023-12-27 15:15:39,053][20347] Updated weights for policy 0, policy_version 1387416 (0.0010) [2023-12-27 15:15:39,664][20381] Updated weights for policy 1, policy_version 1386227 (0.0007) [2023-12-27 15:15:39,696][20292] KL-divergence is very high: 156.4333 [2023-12-27 15:15:39,732][20381] Updated weights for policy 1, policy_version 1386237 (0.0007) [2023-12-27 15:15:39,752][20292] KL-divergence is very high: 152.5030 [2023-12-27 15:15:39,792][20381] Updated weights for policy 1, policy_version 1386247 (0.0007) [2023-12-27 15:15:39,832][20347] Updated weights for policy 0, policy_version 1387426 (0.0008) [2023-12-27 15:15:39,914][20347] Updated weights for policy 0, policy_version 1387436 (0.0008) [2023-12-27 15:15:40,001][20347] Updated weights for policy 0, policy_version 1387446 (0.0009) [2023-12-27 15:15:40,524][20381] Updated weights for policy 1, policy_version 1386257 (0.0009) [2023-12-27 15:15:40,587][20381] Updated weights for policy 1, policy_version 1386267 (0.0008) [2023-12-27 15:15:40,645][20381] Updated weights for policy 1, policy_version 1386277 (0.0008) [2023-12-27 15:15:40,696][20381] Updated weights for policy 1, policy_version 1386287 (0.0008) [2023-12-27 15:15:40,721][20347] Updated weights for policy 0, policy_version 1387456 (0.0008) [2023-12-27 15:15:40,777][20347] Updated weights for policy 0, policy_version 1387466 (0.0010) [2023-12-27 15:15:40,829][20347] Updated weights for policy 0, policy_version 1387476 (0.0010) [2023-12-27 15:15:41,380][20381] Updated weights for policy 1, policy_version 1386297 (0.0008) [2023-12-27 15:15:41,443][20381] Updated weights for policy 1, policy_version 1386307 (0.0010) [2023-12-27 15:15:41,501][20381] Updated weights for policy 1, policy_version 1386317 (0.0007) [2023-12-27 15:15:41,658][20347] Updated weights for policy 0, policy_version 1387486 (0.0008) [2023-12-27 15:15:41,708][20347] Updated weights for policy 0, policy_version 1387496 (0.0007) [2023-12-27 15:15:41,773][20347] Updated weights for policy 0, policy_version 1387506 (0.0009) [2023-12-27 15:15:42,217][20381] Updated weights for policy 1, policy_version 1386327 (0.0008) [2023-12-27 15:15:42,277][20381] Updated weights for policy 1, policy_version 1386337 (0.0008) [2023-12-27 15:15:42,344][20381] Updated weights for policy 1, policy_version 1386347 (0.0008) [2023-12-27 15:15:42,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 710197248. Throughput: 0: 10157.7, 1: 10219.3. Samples: 710193876. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:42,363][19170] Avg episode reward: [(0, '970.840'), (1, '951.290')] [2023-12-27 15:15:42,495][20347] Updated weights for policy 0, policy_version 1387516 (0.0006) [2023-12-27 15:15:42,560][20347] Updated weights for policy 0, policy_version 1387526 (0.0007) [2023-12-27 15:15:42,626][20347] Updated weights for policy 0, policy_version 1387536 (0.0007) [2023-12-27 15:15:43,088][20381] Updated weights for policy 1, policy_version 1386357 (0.0008) [2023-12-27 15:15:43,151][20381] Updated weights for policy 1, policy_version 1386367 (0.0008) [2023-12-27 15:15:43,207][20381] Updated weights for policy 1, policy_version 1386377 (0.0008) [2023-12-27 15:15:43,328][20347] Updated weights for policy 0, policy_version 1387546 (0.0008) [2023-12-27 15:15:43,392][20347] Updated weights for policy 0, policy_version 1387556 (0.0008) [2023-12-27 15:15:43,449][20347] Updated weights for policy 0, policy_version 1387566 (0.0008) [2023-12-27 15:15:43,505][20347] Updated weights for policy 0, policy_version 1387576 (0.0007) [2023-12-27 15:15:43,879][20381] Updated weights for policy 1, policy_version 1386387 (0.0007) [2023-12-27 15:15:43,932][20381] Updated weights for policy 1, policy_version 1386397 (0.0007) [2023-12-27 15:15:43,992][20381] Updated weights for policy 1, policy_version 1386407 (0.0008) [2023-12-27 15:15:44,168][20291] KL-divergence is very high: 101.1196 [2023-12-27 15:15:44,180][20347] Updated weights for policy 0, policy_version 1387586 (0.0007) [2023-12-27 15:15:44,214][20291] KL-divergence is very high: 177.1317 [2023-12-27 15:15:44,239][20347] Updated weights for policy 0, policy_version 1387596 (0.0005) [2023-12-27 15:15:44,270][20291] KL-divergence is very high: 170.0309 [2023-12-27 15:15:44,304][20347] Updated weights for policy 0, policy_version 1387606 (0.0005) [2023-12-27 15:15:44,744][20381] Updated weights for policy 1, policy_version 1386417 (0.0009) [2023-12-27 15:15:44,808][20381] Updated weights for policy 1, policy_version 1386427 (0.0009) [2023-12-27 15:15:44,862][20381] Updated weights for policy 1, policy_version 1386437 (0.0008) [2023-12-27 15:15:44,926][20381] Updated weights for policy 1, policy_version 1386447 (0.0009) [2023-12-27 15:15:44,945][20347] Updated weights for policy 0, policy_version 1387616 (0.0009) [2023-12-27 15:15:45,009][20347] Updated weights for policy 0, policy_version 1387626 (0.0008) [2023-12-27 15:15:45,066][20347] Updated weights for policy 0, policy_version 1387636 (0.0006) [2023-12-27 15:15:45,567][20381] Updated weights for policy 1, policy_version 1386457 (0.0008) [2023-12-27 15:15:45,627][20381] Updated weights for policy 1, policy_version 1386467 (0.0009) [2023-12-27 15:15:45,685][20381] Updated weights for policy 1, policy_version 1386477 (0.0007) [2023-12-27 15:15:45,699][20347] Updated weights for policy 0, policy_version 1387646 (0.0007) [2023-12-27 15:15:45,755][20347] Updated weights for policy 0, policy_version 1387656 (0.0007) [2023-12-27 15:15:45,811][20347] Updated weights for policy 0, policy_version 1387666 (0.0008) [2023-12-27 15:15:46,412][20381] Updated weights for policy 1, policy_version 1386487 (0.0006) [2023-12-27 15:15:46,414][20347] Updated weights for policy 0, policy_version 1387676 (0.0007) [2023-12-27 15:15:46,464][20381] Updated weights for policy 1, policy_version 1386497 (0.0005) [2023-12-27 15:15:46,476][20347] Updated weights for policy 0, policy_version 1387686 (0.0008) [2023-12-27 15:15:46,523][20381] Updated weights for policy 1, policy_version 1386507 (0.0006) [2023-12-27 15:15:46,528][20347] Updated weights for policy 0, policy_version 1387696 (0.0005) [2023-12-27 15:15:47,175][20347] Updated weights for policy 0, policy_version 1387706 (0.0006) [2023-12-27 15:15:47,226][20347] Updated weights for policy 0, policy_version 1387716 (0.0005) [2023-12-27 15:15:47,239][20381] Updated weights for policy 1, policy_version 1386517 (0.0006) [2023-12-27 15:15:47,252][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 15:15:47,305][20381] Updated weights for policy 1, policy_version 1386527 (0.0007) [2023-12-27 15:15:47,351][20381] Updated weights for policy 1, policy_version 1386537 (0.0008) [2023-12-27 15:15:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 710303744. Throughput: 0: 10102.6, 1: 10160.8. Samples: 710313060. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:47,363][19170] Avg episode reward: [(0, '960.940'), (1, '951.290')] [2023-12-27 15:15:47,867][20347] Updated weights for policy 0, policy_version 1387726 (0.0005) [2023-12-27 15:15:47,916][20347] Updated weights for policy 0, policy_version 1387736 (0.0005) [2023-12-27 15:15:47,965][20347] Updated weights for policy 0, policy_version 1387746 (0.0010) [2023-12-27 15:15:48,063][20381] Updated weights for policy 1, policy_version 1386547 (0.0008) [2023-12-27 15:15:48,117][20381] Updated weights for policy 1, policy_version 1386557 (0.0011) [2023-12-27 15:15:48,169][20381] Updated weights for policy 1, policy_version 1386567 (0.0010) [2023-12-27 15:15:48,640][20347] Updated weights for policy 0, policy_version 1387756 (0.0007) [2023-12-27 15:15:48,698][20347] Updated weights for policy 0, policy_version 1387766 (0.0008) [2023-12-27 15:15:48,754][20347] Updated weights for policy 0, policy_version 1387776 (0.0009) [2023-12-27 15:15:48,900][20381] Updated weights for policy 1, policy_version 1386577 (0.0010) [2023-12-27 15:15:48,952][20381] Updated weights for policy 1, policy_version 1386587 (0.0011) [2023-12-27 15:15:49,004][20381] Updated weights for policy 1, policy_version 1386597 (0.0011) [2023-12-27 15:15:49,055][20381] Updated weights for policy 1, policy_version 1386607 (0.0010) [2023-12-27 15:15:49,478][20347] Updated weights for policy 0, policy_version 1387786 (0.0010) [2023-12-27 15:15:49,537][20347] Updated weights for policy 0, policy_version 1387796 (0.0007) [2023-12-27 15:15:49,589][20347] Updated weights for policy 0, policy_version 1387806 (0.0008) [2023-12-27 15:15:49,636][20347] Updated weights for policy 0, policy_version 1387816 (0.0008) [2023-12-27 15:15:49,750][20381] Updated weights for policy 1, policy_version 1386617 (0.0008) [2023-12-27 15:15:49,806][20381] Updated weights for policy 1, policy_version 1386627 (0.0008) [2023-12-27 15:15:49,870][20381] Updated weights for policy 1, policy_version 1386637 (0.0007) [2023-12-27 15:15:50,396][20347] Updated weights for policy 0, policy_version 1387826 (0.0008) [2023-12-27 15:15:50,447][20291] KL-divergence is very high: 111.5566 [2023-12-27 15:15:50,458][20347] Updated weights for policy 0, policy_version 1387836 (0.0007) [2023-12-27 15:15:50,517][20347] Updated weights for policy 0, policy_version 1387846 (0.0007) [2023-12-27 15:15:50,541][20381] Updated weights for policy 1, policy_version 1386647 (0.0007) [2023-12-27 15:15:50,599][20381] Updated weights for policy 1, policy_version 1386657 (0.0007) [2023-12-27 15:15:50,655][20381] Updated weights for policy 1, policy_version 1386667 (0.0008) [2023-12-27 15:15:51,197][20347] Updated weights for policy 0, policy_version 1387856 (0.0007) [2023-12-27 15:15:51,258][20347] Updated weights for policy 0, policy_version 1387866 (0.0006) [2023-12-27 15:15:51,310][20347] Updated weights for policy 0, policy_version 1387876 (0.0007) [2023-12-27 15:15:51,428][20381] Updated weights for policy 1, policy_version 1386677 (0.0008) [2023-12-27 15:15:51,491][20381] Updated weights for policy 1, policy_version 1386687 (0.0009) [2023-12-27 15:15:51,552][20381] Updated weights for policy 1, policy_version 1386697 (0.0009) [2023-12-27 15:15:51,991][20347] Updated weights for policy 0, policy_version 1387886 (0.0008) [2023-12-27 15:15:52,052][20347] Updated weights for policy 0, policy_version 1387896 (0.0006) [2023-12-27 15:15:52,113][20347] Updated weights for policy 0, policy_version 1387906 (0.0005) [2023-12-27 15:15:52,281][20381] Updated weights for policy 1, policy_version 1386707 (0.0008) [2023-12-27 15:15:52,336][20381] Updated weights for policy 1, policy_version 1386717 (0.0008) [2023-12-27 15:15:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 710402048. Throughput: 0: 10119.2, 1: 10140.4. Samples: 710374664. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:52,363][19170] Avg episode reward: [(0, '931.720'), (1, '951.210')] [2023-12-27 15:15:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001387912_355377152.pth... [2023-12-27 15:15:52,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001386744_355074048.pth [2023-12-27 15:15:52,402][20381] Updated weights for policy 1, policy_version 1386727 (0.0007) [2023-12-27 15:15:52,451][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001386736_355033088.pth... [2023-12-27 15:15:52,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001385552_354729984.pth [2023-12-27 15:15:52,782][20347] Updated weights for policy 0, policy_version 1387916 (0.0006) [2023-12-27 15:15:52,822][20291] KL-divergence is very high: 115.5885 [2023-12-27 15:15:52,837][20347] Updated weights for policy 0, policy_version 1387926 (0.0008) [2023-12-27 15:15:52,862][20291] KL-divergence is very high: 100.9577 [2023-12-27 15:15:52,890][20347] Updated weights for policy 0, policy_version 1387936 (0.0008) [2023-12-27 15:15:53,103][20381] Updated weights for policy 1, policy_version 1386737 (0.0006) [2023-12-27 15:15:53,156][20381] Updated weights for policy 1, policy_version 1386747 (0.0006) [2023-12-27 15:15:53,207][20381] Updated weights for policy 1, policy_version 1386757 (0.0006) [2023-12-27 15:15:53,253][20381] Updated weights for policy 1, policy_version 1386767 (0.0007) [2023-12-27 15:15:53,611][20347] Updated weights for policy 0, policy_version 1387946 (0.0008) [2023-12-27 15:15:53,669][20347] Updated weights for policy 0, policy_version 1387956 (0.0008) [2023-12-27 15:15:53,721][20347] Updated weights for policy 0, policy_version 1387966 (0.0008) [2023-12-27 15:15:53,767][20347] Updated weights for policy 0, policy_version 1387976 (0.0010) [2023-12-27 15:15:53,854][20381] Updated weights for policy 1, policy_version 1386777 (0.0006) [2023-12-27 15:15:53,905][20381] Updated weights for policy 1, policy_version 1386787 (0.0006) [2023-12-27 15:15:53,962][20381] Updated weights for policy 1, policy_version 1386797 (0.0008) [2023-12-27 15:15:54,420][20347] Updated weights for policy 0, policy_version 1387986 (0.0007) [2023-12-27 15:15:54,480][20347] Updated weights for policy 0, policy_version 1387996 (0.0008) [2023-12-27 15:15:54,535][20347] Updated weights for policy 0, policy_version 1388006 (0.0008) [2023-12-27 15:15:54,590][20381] Updated weights for policy 1, policy_version 1386807 (0.0006) [2023-12-27 15:15:54,643][20381] Updated weights for policy 1, policy_version 1386817 (0.0005) [2023-12-27 15:15:54,695][20381] Updated weights for policy 1, policy_version 1386827 (0.0005) [2023-12-27 15:15:55,173][20347] Updated weights for policy 0, policy_version 1388016 (0.0005) [2023-12-27 15:15:55,228][20347] Updated weights for policy 0, policy_version 1388026 (0.0008) [2023-12-27 15:15:55,251][20381] Updated weights for policy 1, policy_version 1386837 (0.0005) [2023-12-27 15:15:55,284][20347] Updated weights for policy 0, policy_version 1388036 (0.0008) [2023-12-27 15:15:55,308][20381] Updated weights for policy 1, policy_version 1386847 (0.0005) [2023-12-27 15:15:55,356][20381] Updated weights for policy 1, policy_version 1386857 (0.0005) [2023-12-27 15:15:55,917][20381] Updated weights for policy 1, policy_version 1386867 (0.0005) [2023-12-27 15:15:55,961][20347] Updated weights for policy 0, policy_version 1388046 (0.0007) [2023-12-27 15:15:55,976][20381] Updated weights for policy 1, policy_version 1386877 (0.0005) [2023-12-27 15:15:56,015][20347] Updated weights for policy 0, policy_version 1388056 (0.0006) [2023-12-27 15:15:56,031][20381] Updated weights for policy 1, policy_version 1386887 (0.0006) [2023-12-27 15:15:56,071][20347] Updated weights for policy 0, policy_version 1388066 (0.0007) [2023-12-27 15:15:56,635][20381] Updated weights for policy 1, policy_version 1386897 (0.0005) [2023-12-27 15:15:56,649][20347] Updated weights for policy 0, policy_version 1388076 (0.0007) [2023-12-27 15:15:56,682][20381] Updated weights for policy 1, policy_version 1386907 (0.0006) [2023-12-27 15:15:56,703][20347] Updated weights for policy 0, policy_version 1388086 (0.0006) [2023-12-27 15:15:56,732][20381] Updated weights for policy 1, policy_version 1386917 (0.0006) [2023-12-27 15:15:56,751][20347] Updated weights for policy 0, policy_version 1388096 (0.0006) [2023-12-27 15:15:56,787][20381] Updated weights for policy 1, policy_version 1386927 (0.0005) [2023-12-27 15:15:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 710508544. Throughput: 0: 10093.2, 1: 10169.7. Samples: 710500176. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-12-27 15:15:57,363][19170] Avg episode reward: [(0, '951.280'), (1, '960.930')] [2023-12-27 15:15:57,405][20347] Updated weights for policy 0, policy_version 1388106 (0.0006) [2023-12-27 15:15:57,461][20381] Updated weights for policy 1, policy_version 1386937 (0.0010) [2023-12-27 15:15:57,462][20347] Updated weights for policy 0, policy_version 1388116 (0.0009) [2023-12-27 15:15:57,512][20347] Updated weights for policy 0, policy_version 1388126 (0.0006) [2023-12-27 15:15:57,519][20381] Updated weights for policy 1, policy_version 1386947 (0.0010) [2023-12-27 15:15:57,569][20347] Updated weights for policy 0, policy_version 1388136 (0.0006) [2023-12-27 15:15:57,570][20381] Updated weights for policy 1, policy_version 1386957 (0.0008) [2023-12-27 15:15:58,241][20381] Updated weights for policy 1, policy_version 1386967 (0.0007) [2023-12-27 15:15:58,256][20347] Updated weights for policy 0, policy_version 1388146 (0.0008) [2023-12-27 15:15:58,300][20381] Updated weights for policy 1, policy_version 1386977 (0.0007) [2023-12-27 15:15:58,314][20347] Updated weights for policy 0, policy_version 1388156 (0.0006) [2023-12-27 15:15:58,366][20381] Updated weights for policy 1, policy_version 1386987 (0.0007) [2023-12-27 15:15:58,377][20347] Updated weights for policy 0, policy_version 1388166 (0.0008) [2023-12-27 15:15:59,131][20347] Updated weights for policy 0, policy_version 1388176 (0.0007) [2023-12-27 15:15:59,143][20381] Updated weights for policy 1, policy_version 1386997 (0.0007) [2023-12-27 15:15:59,183][20347] Updated weights for policy 0, policy_version 1388186 (0.0006) [2023-12-27 15:15:59,199][20381] Updated weights for policy 1, policy_version 1387007 (0.0007) [2023-12-27 15:15:59,248][20347] Updated weights for policy 0, policy_version 1388196 (0.0008) [2023-12-27 15:15:59,265][20381] Updated weights for policy 1, policy_version 1387017 (0.0007) [2023-12-27 15:15:59,901][20347] Updated weights for policy 0, policy_version 1388206 (0.0009) [2023-12-27 15:15:59,964][20347] Updated weights for policy 0, policy_version 1388216 (0.0009) [2023-12-27 15:16:00,024][20381] Updated weights for policy 1, policy_version 1387027 (0.0007) [2023-12-27 15:16:00,025][20347] Updated weights for policy 0, policy_version 1388226 (0.0008) [2023-12-27 15:16:00,081][20381] Updated weights for policy 1, policy_version 1387037 (0.0007) [2023-12-27 15:16:00,141][20381] Updated weights for policy 1, policy_version 1387047 (0.0008) [2023-12-27 15:16:00,690][20347] Updated weights for policy 0, policy_version 1388236 (0.0008) [2023-12-27 15:16:00,742][20347] Updated weights for policy 0, policy_version 1388246 (0.0005) [2023-12-27 15:16:00,802][20347] Updated weights for policy 0, policy_version 1388256 (0.0005) [2023-12-27 15:16:00,831][20381] Updated weights for policy 1, policy_version 1387057 (0.0007) [2023-12-27 15:16:00,900][20381] Updated weights for policy 1, policy_version 1387067 (0.0011) [2023-12-27 15:16:00,958][20381] Updated weights for policy 1, policy_version 1387077 (0.0011) [2023-12-27 15:16:01,011][20381] Updated weights for policy 1, policy_version 1387087 (0.0010) [2023-12-27 15:16:01,462][20347] Updated weights for policy 0, policy_version 1388266 (0.0008) [2023-12-27 15:16:01,513][20347] Updated weights for policy 0, policy_version 1388276 (0.0010) [2023-12-27 15:16:01,572][20347] Updated weights for policy 0, policy_version 1388286 (0.0008) [2023-12-27 15:16:01,626][20347] Updated weights for policy 0, policy_version 1388296 (0.0008) [2023-12-27 15:16:01,738][20381] Updated weights for policy 1, policy_version 1387097 (0.0010) [2023-12-27 15:16:01,799][20381] Updated weights for policy 1, policy_version 1387107 (0.0008) [2023-12-27 15:16:01,858][20381] Updated weights for policy 1, policy_version 1387117 (0.0005) [2023-12-27 15:16:02,322][20347] Updated weights for policy 0, policy_version 1388306 (0.0010) [2023-12-27 15:16:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 710606848. Throughput: 0: 10096.8, 1: 10113.5. Samples: 710621728. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:02,363][19170] Avg episode reward: [(0, '961.060'), (1, '960.890')] [2023-12-27 15:16:02,370][20347] Updated weights for policy 0, policy_version 1388316 (0.0010) [2023-12-27 15:16:02,438][20347] Updated weights for policy 0, policy_version 1388326 (0.0007) [2023-12-27 15:16:02,527][20381] Updated weights for policy 1, policy_version 1387127 (0.0010) [2023-12-27 15:16:02,586][20381] Updated weights for policy 1, policy_version 1387137 (0.0010) [2023-12-27 15:16:02,643][20381] Updated weights for policy 1, policy_version 1387147 (0.0010) [2023-12-27 15:16:03,091][20347] Updated weights for policy 0, policy_version 1388336 (0.0009) [2023-12-27 15:16:03,139][20347] Updated weights for policy 0, policy_version 1388346 (0.0010) [2023-12-27 15:16:03,193][20347] Updated weights for policy 0, policy_version 1388356 (0.0010) [2023-12-27 15:16:03,315][20381] Updated weights for policy 1, policy_version 1387157 (0.0010) [2023-12-27 15:16:03,374][20381] Updated weights for policy 1, policy_version 1387167 (0.0007) [2023-12-27 15:16:03,435][20381] Updated weights for policy 1, policy_version 1387177 (0.0010) [2023-12-27 15:16:03,891][20347] Updated weights for policy 0, policy_version 1388366 (0.0010) [2023-12-27 15:16:03,942][20347] Updated weights for policy 0, policy_version 1388376 (0.0011) [2023-12-27 15:16:04,005][20347] Updated weights for policy 0, policy_version 1388386 (0.0010) [2023-12-27 15:16:04,076][20381] Updated weights for policy 1, policy_version 1387187 (0.0009) [2023-12-27 15:16:04,142][20381] Updated weights for policy 1, policy_version 1387197 (0.0006) [2023-12-27 15:16:04,201][20381] Updated weights for policy 1, policy_version 1387207 (0.0007) [2023-12-27 15:16:04,699][20347] Updated weights for policy 0, policy_version 1388396 (0.0010) [2023-12-27 15:16:04,760][20347] Updated weights for policy 0, policy_version 1388406 (0.0010) [2023-12-27 15:16:04,821][20347] Updated weights for policy 0, policy_version 1388416 (0.0010) [2023-12-27 15:16:04,882][20381] Updated weights for policy 1, policy_version 1387217 (0.0007) [2023-12-27 15:16:04,937][20381] Updated weights for policy 1, policy_version 1387227 (0.0006) [2023-12-27 15:16:04,987][20381] Updated weights for policy 1, policy_version 1387237 (0.0006) [2023-12-27 15:16:05,052][20381] Updated weights for policy 1, policy_version 1387247 (0.0007) [2023-12-27 15:16:05,499][20347] Updated weights for policy 0, policy_version 1388426 (0.0010) [2023-12-27 15:16:05,549][20347] Updated weights for policy 0, policy_version 1388436 (0.0010) [2023-12-27 15:16:05,604][20347] Updated weights for policy 0, policy_version 1388446 (0.0010) [2023-12-27 15:16:05,656][20381] Updated weights for policy 1, policy_version 1387257 (0.0005) [2023-12-27 15:16:05,657][20292] KL-divergence is very high: 165.6080 [2023-12-27 15:16:05,661][20347] Updated weights for policy 0, policy_version 1388456 (0.0010) [2023-12-27 15:16:05,691][20292] KL-divergence is very high: 184.5931 [2023-12-27 15:16:05,700][20381] Updated weights for policy 1, policy_version 1387267 (0.0005) [2023-12-27 15:16:05,726][20292] KL-divergence is very high: 108.2638 [2023-12-27 15:16:05,742][20381] Updated weights for policy 1, policy_version 1387277 (0.0005) [2023-12-27 15:16:06,375][20347] Updated weights for policy 0, policy_version 1388466 (0.0010) [2023-12-27 15:16:06,407][20381] Updated weights for policy 1, policy_version 1387287 (0.0008) [2023-12-27 15:16:06,440][20347] Updated weights for policy 0, policy_version 1388476 (0.0011) [2023-12-27 15:16:06,468][20381] Updated weights for policy 1, policy_version 1387297 (0.0007) [2023-12-27 15:16:06,493][20347] Updated weights for policy 0, policy_version 1388486 (0.0009) [2023-12-27 15:16:06,519][20381] Updated weights for policy 1, policy_version 1387307 (0.0006) [2023-12-27 15:16:07,156][20347] Updated weights for policy 0, policy_version 1388496 (0.0008) [2023-12-27 15:16:07,182][20381] Updated weights for policy 1, policy_version 1387317 (0.0007) [2023-12-27 15:16:07,213][20347] Updated weights for policy 0, policy_version 1388506 (0.0007) [2023-12-27 15:16:07,234][20381] Updated weights for policy 1, policy_version 1387327 (0.0005) [2023-12-27 15:16:07,277][20347] Updated weights for policy 0, policy_version 1388516 (0.0007) [2023-12-27 15:16:07,286][20381] Updated weights for policy 1, policy_version 1387337 (0.0005) [2023-12-27 15:16:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 710721536. Throughput: 0: 10098.6, 1: 10097.0. Samples: 710683016. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:07,363][19170] Avg episode reward: [(0, '961.060'), (1, '912.070')] [2023-12-27 15:16:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001387344_355188736.pth... [2023-12-27 15:16:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001388520_355532800.pth... [2023-12-27 15:16:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001386160_354885632.pth [2023-12-27 15:16:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001387352_355229696.pth [2023-12-27 15:16:07,929][20347] Updated weights for policy 0, policy_version 1388526 (0.0008) [2023-12-27 15:16:07,931][20381] Updated weights for policy 1, policy_version 1387347 (0.0008) [2023-12-27 15:16:07,993][20381] Updated weights for policy 1, policy_version 1387357 (0.0006) [2023-12-27 15:16:07,995][20347] Updated weights for policy 0, policy_version 1388536 (0.0006) [2023-12-27 15:16:08,042][20381] Updated weights for policy 1, policy_version 1387367 (0.0006) [2023-12-27 15:16:08,052][20347] Updated weights for policy 0, policy_version 1388546 (0.0006) [2023-12-27 15:16:08,666][20347] Updated weights for policy 0, policy_version 1388556 (0.0006) [2023-12-27 15:16:08,722][20347] Updated weights for policy 0, policy_version 1388566 (0.0008) [2023-12-27 15:16:08,748][20381] Updated weights for policy 1, policy_version 1387377 (0.0006) [2023-12-27 15:16:08,784][20347] Updated weights for policy 0, policy_version 1388576 (0.0006) [2023-12-27 15:16:08,808][20381] Updated weights for policy 1, policy_version 1387387 (0.0006) [2023-12-27 15:16:08,870][20381] Updated weights for policy 1, policy_version 1387397 (0.0007) [2023-12-27 15:16:08,936][20381] Updated weights for policy 1, policy_version 1387407 (0.0008) [2023-12-27 15:16:09,514][20347] Updated weights for policy 0, policy_version 1388586 (0.0006) [2023-12-27 15:16:09,572][20347] Updated weights for policy 0, policy_version 1388596 (0.0007) [2023-12-27 15:16:09,602][20381] Updated weights for policy 1, policy_version 1387417 (0.0007) [2023-12-27 15:16:09,633][20347] Updated weights for policy 0, policy_version 1388606 (0.0007) [2023-12-27 15:16:09,662][20381] Updated weights for policy 1, policy_version 1387427 (0.0006) [2023-12-27 15:16:09,688][20347] Updated weights for policy 0, policy_version 1388616 (0.0006) [2023-12-27 15:16:09,721][20381] Updated weights for policy 1, policy_version 1387437 (0.0009) [2023-12-27 15:16:10,416][20381] Updated weights for policy 1, policy_version 1387447 (0.0007) [2023-12-27 15:16:10,437][20347] Updated weights for policy 0, policy_version 1388626 (0.0006) [2023-12-27 15:16:10,479][20381] Updated weights for policy 1, policy_version 1387457 (0.0007) [2023-12-27 15:16:10,494][20347] Updated weights for policy 0, policy_version 1388636 (0.0008) [2023-12-27 15:16:10,547][20381] Updated weights for policy 1, policy_version 1387467 (0.0007) [2023-12-27 15:16:10,554][20347] Updated weights for policy 0, policy_version 1388646 (0.0007) [2023-12-27 15:16:11,264][20347] Updated weights for policy 0, policy_version 1388656 (0.0008) [2023-12-27 15:16:11,285][20381] Updated weights for policy 1, policy_version 1387477 (0.0008) [2023-12-27 15:16:11,332][20347] Updated weights for policy 0, policy_version 1388666 (0.0006) [2023-12-27 15:16:11,344][20381] Updated weights for policy 1, policy_version 1387487 (0.0008) [2023-12-27 15:16:11,399][20347] Updated weights for policy 0, policy_version 1388676 (0.0006) [2023-12-27 15:16:11,413][20381] Updated weights for policy 1, policy_version 1387497 (0.0007) [2023-12-27 15:16:12,160][20347] Updated weights for policy 0, policy_version 1388686 (0.0007) [2023-12-27 15:16:12,166][20381] Updated weights for policy 1, policy_version 1387507 (0.0008) [2023-12-27 15:16:12,215][20347] Updated weights for policy 0, policy_version 1388696 (0.0007) [2023-12-27 15:16:12,217][20381] Updated weights for policy 1, policy_version 1387517 (0.0007) [2023-12-27 15:16:12,267][20381] Updated weights for policy 1, policy_version 1387527 (0.0006) [2023-12-27 15:16:12,281][20347] Updated weights for policy 0, policy_version 1388706 (0.0006) [2023-12-27 15:16:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 710819840. Throughput: 0: 10151.5, 1: 10119.2. Samples: 710805392. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:12,363][19170] Avg episode reward: [(0, '970.880'), (1, '921.850')] [2023-12-27 15:16:12,951][20381] Updated weights for policy 1, policy_version 1387537 (0.0008) [2023-12-27 15:16:13,002][20347] Updated weights for policy 0, policy_version 1388716 (0.0008) [2023-12-27 15:16:13,003][20381] Updated weights for policy 1, policy_version 1387547 (0.0005) [2023-12-27 15:16:13,060][20347] Updated weights for policy 0, policy_version 1388726 (0.0007) [2023-12-27 15:16:13,062][20381] Updated weights for policy 1, policy_version 1387557 (0.0006) [2023-12-27 15:16:13,117][20347] Updated weights for policy 0, policy_version 1388736 (0.0007) [2023-12-27 15:16:13,126][20381] Updated weights for policy 1, policy_version 1387567 (0.0005) [2023-12-27 15:16:13,782][20381] Updated weights for policy 1, policy_version 1387577 (0.0007) [2023-12-27 15:16:13,827][20347] Updated weights for policy 0, policy_version 1388746 (0.0007) [2023-12-27 15:16:13,843][20381] Updated weights for policy 1, policy_version 1387587 (0.0007) [2023-12-27 15:16:13,882][20347] Updated weights for policy 0, policy_version 1388756 (0.0008) [2023-12-27 15:16:13,892][20381] Updated weights for policy 1, policy_version 1387597 (0.0006) [2023-12-27 15:16:13,930][20347] Updated weights for policy 0, policy_version 1388766 (0.0008) [2023-12-27 15:16:13,983][20347] Updated weights for policy 0, policy_version 1388776 (0.0007) [2023-12-27 15:16:14,570][20381] Updated weights for policy 1, policy_version 1387607 (0.0007) [2023-12-27 15:16:14,622][20381] Updated weights for policy 1, policy_version 1387617 (0.0006) [2023-12-27 15:16:14,669][20381] Updated weights for policy 1, policy_version 1387627 (0.0005) [2023-12-27 15:16:14,676][20347] Updated weights for policy 0, policy_version 1388786 (0.0007) [2023-12-27 15:16:14,723][20347] Updated weights for policy 0, policy_version 1388796 (0.0009) [2023-12-27 15:16:14,779][20347] Updated weights for policy 0, policy_version 1388806 (0.0008) [2023-12-27 15:16:15,280][20381] Updated weights for policy 1, policy_version 1387637 (0.0006) [2023-12-27 15:16:15,336][20381] Updated weights for policy 1, policy_version 1387647 (0.0008) [2023-12-27 15:16:15,395][20381] Updated weights for policy 1, policy_version 1387657 (0.0007) [2023-12-27 15:16:15,482][20347] Updated weights for policy 0, policy_version 1388816 (0.0008) [2023-12-27 15:16:15,547][20347] Updated weights for policy 0, policy_version 1388826 (0.0007) [2023-12-27 15:16:15,611][20347] Updated weights for policy 0, policy_version 1388836 (0.0006) [2023-12-27 15:16:16,089][20381] Updated weights for policy 1, policy_version 1387667 (0.0007) [2023-12-27 15:16:16,137][20381] Updated weights for policy 1, policy_version 1387677 (0.0008) [2023-12-27 15:16:16,190][20381] Updated weights for policy 1, policy_version 1387687 (0.0008) [2023-12-27 15:16:16,254][20347] Updated weights for policy 0, policy_version 1388846 (0.0007) [2023-12-27 15:16:16,316][20347] Updated weights for policy 0, policy_version 1388856 (0.0008) [2023-12-27 15:16:16,375][20347] Updated weights for policy 0, policy_version 1388866 (0.0008) [2023-12-27 15:16:16,912][20381] Updated weights for policy 1, policy_version 1387697 (0.0007) [2023-12-27 15:16:16,920][20347] Updated weights for policy 0, policy_version 1388876 (0.0005) [2023-12-27 15:16:16,968][20381] Updated weights for policy 1, policy_version 1387707 (0.0007) [2023-12-27 15:16:16,972][20347] Updated weights for policy 0, policy_version 1388886 (0.0005) [2023-12-27 15:16:17,029][20381] Updated weights for policy 1, policy_version 1387717 (0.0008) [2023-12-27 15:16:17,029][20347] Updated weights for policy 0, policy_version 1388896 (0.0006) [2023-12-27 15:16:17,087][20381] Updated weights for policy 1, policy_version 1387727 (0.0008) [2023-12-27 15:16:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 710918144. Throughput: 0: 10106.2, 1: 10141.7. Samples: 710926780. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:17,363][19170] Avg episode reward: [(0, '931.790'), (1, '941.390')] [2023-12-27 15:16:17,686][20347] Updated weights for policy 0, policy_version 1388906 (0.0007) [2023-12-27 15:16:17,733][20347] Updated weights for policy 0, policy_version 1388916 (0.0007) [2023-12-27 15:16:17,773][20381] Updated weights for policy 1, policy_version 1387737 (0.0008) [2023-12-27 15:16:17,785][20347] Updated weights for policy 0, policy_version 1388926 (0.0008) [2023-12-27 15:16:17,828][20381] Updated weights for policy 1, policy_version 1387747 (0.0006) [2023-12-27 15:16:17,841][20347] Updated weights for policy 0, policy_version 1388936 (0.0006) [2023-12-27 15:16:17,884][20381] Updated weights for policy 1, policy_version 1387757 (0.0007) [2023-12-27 15:16:18,519][20347] Updated weights for policy 0, policy_version 1388946 (0.0009) [2023-12-27 15:16:18,576][20381] Updated weights for policy 1, policy_version 1387767 (0.0009) [2023-12-27 15:16:18,578][20347] Updated weights for policy 0, policy_version 1388956 (0.0010) [2023-12-27 15:16:18,631][20347] Updated weights for policy 0, policy_version 1388966 (0.0010) [2023-12-27 15:16:18,632][20381] Updated weights for policy 1, policy_version 1387777 (0.0011) [2023-12-27 15:16:18,691][20381] Updated weights for policy 1, policy_version 1387787 (0.0011) [2023-12-27 15:16:19,319][20347] Updated weights for policy 0, policy_version 1388976 (0.0007) [2023-12-27 15:16:19,391][20347] Updated weights for policy 0, policy_version 1388986 (0.0007) [2023-12-27 15:16:19,410][20381] Updated weights for policy 1, policy_version 1387797 (0.0008) [2023-12-27 15:16:19,454][20347] Updated weights for policy 0, policy_version 1388996 (0.0008) [2023-12-27 15:16:19,474][20381] Updated weights for policy 1, policy_version 1387807 (0.0007) [2023-12-27 15:16:19,542][20381] Updated weights for policy 1, policy_version 1387817 (0.0008) [2023-12-27 15:16:20,172][20347] Updated weights for policy 0, policy_version 1389006 (0.0008) [2023-12-27 15:16:20,232][20347] Updated weights for policy 0, policy_version 1389016 (0.0007) [2023-12-27 15:16:20,233][20381] Updated weights for policy 1, policy_version 1387827 (0.0008) [2023-12-27 15:16:20,289][20381] Updated weights for policy 1, policy_version 1387837 (0.0006) [2023-12-27 15:16:20,292][20347] Updated weights for policy 0, policy_version 1389026 (0.0006) [2023-12-27 15:16:20,345][20381] Updated weights for policy 1, policy_version 1387847 (0.0006) [2023-12-27 15:16:20,976][20381] Updated weights for policy 1, policy_version 1387857 (0.0006) [2023-12-27 15:16:21,021][20347] Updated weights for policy 0, policy_version 1389036 (0.0007) [2023-12-27 15:16:21,039][20381] Updated weights for policy 1, policy_version 1387867 (0.0007) [2023-12-27 15:16:21,090][20347] Updated weights for policy 0, policy_version 1389046 (0.0008) [2023-12-27 15:16:21,104][20381] Updated weights for policy 1, policy_version 1387877 (0.0010) [2023-12-27 15:16:21,152][20347] Updated weights for policy 0, policy_version 1389056 (0.0007) [2023-12-27 15:16:21,172][20381] Updated weights for policy 1, policy_version 1387887 (0.0006) [2023-12-27 15:16:21,847][20347] Updated weights for policy 0, policy_version 1389066 (0.0009) [2023-12-27 15:16:21,901][20347] Updated weights for policy 0, policy_version 1389076 (0.0010) [2023-12-27 15:16:21,906][20381] Updated weights for policy 1, policy_version 1387897 (0.0010) [2023-12-27 15:16:21,956][20347] Updated weights for policy 0, policy_version 1389086 (0.0010) [2023-12-27 15:16:21,967][20381] Updated weights for policy 1, policy_version 1387907 (0.0010) [2023-12-27 15:16:22,013][20347] Updated weights for policy 0, policy_version 1389096 (0.0010) [2023-12-27 15:16:22,023][20381] Updated weights for policy 1, policy_version 1387917 (0.0008) [2023-12-27 15:16:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 711016448. Throughput: 0: 10108.6, 1: 10101.3. Samples: 710986292. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:22,363][19170] Avg episode reward: [(0, '931.650'), (1, '960.940')] [2023-12-27 15:16:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001387920_355336192.pth... [2023-12-27 15:16:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001389096_355680256.pth... [2023-12-27 15:16:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001386736_355033088.pth [2023-12-27 15:16:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001387912_355377152.pth [2023-12-27 15:16:22,751][20381] Updated weights for policy 1, policy_version 1387927 (0.0008) [2023-12-27 15:16:22,755][20347] Updated weights for policy 0, policy_version 1389106 (0.0007) [2023-12-27 15:16:22,813][20381] Updated weights for policy 1, policy_version 1387937 (0.0009) [2023-12-27 15:16:22,815][20347] Updated weights for policy 0, policy_version 1389116 (0.0006) [2023-12-27 15:16:22,870][20381] Updated weights for policy 1, policy_version 1387947 (0.0011) [2023-12-27 15:16:22,871][20347] Updated weights for policy 0, policy_version 1389126 (0.0005) [2023-12-27 15:16:23,454][20381] Updated weights for policy 1, policy_version 1387957 (0.0008) [2023-12-27 15:16:23,505][20381] Updated weights for policy 1, policy_version 1387967 (0.0006) [2023-12-27 15:16:23,548][20347] Updated weights for policy 0, policy_version 1389136 (0.0007) [2023-12-27 15:16:23,553][20381] Updated weights for policy 1, policy_version 1387977 (0.0010) [2023-12-27 15:16:23,605][20347] Updated weights for policy 0, policy_version 1389146 (0.0005) [2023-12-27 15:16:23,669][20347] Updated weights for policy 0, policy_version 1389156 (0.0007) [2023-12-27 15:16:24,223][20381] Updated weights for policy 1, policy_version 1387987 (0.0009) [2023-12-27 15:16:24,284][20381] Updated weights for policy 1, policy_version 1387997 (0.0007) [2023-12-27 15:16:24,347][20381] Updated weights for policy 1, policy_version 1388007 (0.0006) [2023-12-27 15:16:24,354][20347] Updated weights for policy 0, policy_version 1389166 (0.0007) [2023-12-27 15:16:24,410][20347] Updated weights for policy 0, policy_version 1389176 (0.0008) [2023-12-27 15:16:24,469][20347] Updated weights for policy 0, policy_version 1389186 (0.0008) [2023-12-27 15:16:25,004][20381] Updated weights for policy 1, policy_version 1388017 (0.0005) [2023-12-27 15:16:25,059][20381] Updated weights for policy 1, policy_version 1388027 (0.0006) [2023-12-27 15:16:25,074][20347] Updated weights for policy 0, policy_version 1389196 (0.0007) [2023-12-27 15:16:25,110][20381] Updated weights for policy 1, policy_version 1388037 (0.0007) [2023-12-27 15:16:25,131][20347] Updated weights for policy 0, policy_version 1389206 (0.0005) [2023-12-27 15:16:25,175][20381] Updated weights for policy 1, policy_version 1388047 (0.0008) [2023-12-27 15:16:25,180][20347] Updated weights for policy 0, policy_version 1389216 (0.0005) [2023-12-27 15:16:25,713][20347] Updated weights for policy 0, policy_version 1389226 (0.0006) [2023-12-27 15:16:25,768][20347] Updated weights for policy 0, policy_version 1389236 (0.0006) [2023-12-27 15:16:25,816][20347] Updated weights for policy 0, policy_version 1389246 (0.0006) [2023-12-27 15:16:25,864][20347] Updated weights for policy 0, policy_version 1389256 (0.0006) [2023-12-27 15:16:25,906][20381] Updated weights for policy 1, policy_version 1388057 (0.0005) [2023-12-27 15:16:25,965][20381] Updated weights for policy 1, policy_version 1388067 (0.0007) [2023-12-27 15:16:26,014][20381] Updated weights for policy 1, policy_version 1388077 (0.0008) [2023-12-27 15:16:26,510][20347] Updated weights for policy 0, policy_version 1389266 (0.0007) [2023-12-27 15:16:26,558][20347] Updated weights for policy 0, policy_version 1389276 (0.0005) [2023-12-27 15:16:26,609][20347] Updated weights for policy 0, policy_version 1389286 (0.0005) [2023-12-27 15:16:26,680][20381] Updated weights for policy 1, policy_version 1388087 (0.0007) [2023-12-27 15:16:26,725][20381] Updated weights for policy 1, policy_version 1388097 (0.0008) [2023-12-27 15:16:26,778][20381] Updated weights for policy 1, policy_version 1388107 (0.0008) [2023-12-27 15:16:27,195][20347] Updated weights for policy 0, policy_version 1389296 (0.0006) [2023-12-27 15:16:27,255][20347] Updated weights for policy 0, policy_version 1389306 (0.0006) [2023-12-27 15:16:27,315][20347] Updated weights for policy 0, policy_version 1389316 (0.0008) [2023-12-27 15:16:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 711122944. Throughput: 0: 10214.7, 1: 10128.0. Samples: 711109296. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:27,363][19170] Avg episode reward: [(0, '931.620'), (1, '941.490')] [2023-12-27 15:16:27,470][20381] Updated weights for policy 1, policy_version 1388117 (0.0008) [2023-12-27 15:16:27,530][20381] Updated weights for policy 1, policy_version 1388127 (0.0008) [2023-12-27 15:16:27,586][20381] Updated weights for policy 1, policy_version 1388137 (0.0006) [2023-12-27 15:16:27,931][20347] Updated weights for policy 0, policy_version 1389326 (0.0007) [2023-12-27 15:16:27,984][20347] Updated weights for policy 0, policy_version 1389336 (0.0008) [2023-12-27 15:16:28,038][20347] Updated weights for policy 0, policy_version 1389346 (0.0006) [2023-12-27 15:16:28,234][20381] Updated weights for policy 1, policy_version 1388147 (0.0006) [2023-12-27 15:16:28,284][20381] Updated weights for policy 1, policy_version 1388157 (0.0007) [2023-12-27 15:16:28,338][20381] Updated weights for policy 1, policy_version 1388167 (0.0007) [2023-12-27 15:16:28,665][20347] Updated weights for policy 0, policy_version 1389356 (0.0008) [2023-12-27 15:16:28,723][20347] Updated weights for policy 0, policy_version 1389366 (0.0007) [2023-12-27 15:16:28,779][20347] Updated weights for policy 0, policy_version 1389376 (0.0008) [2023-12-27 15:16:29,075][20381] Updated weights for policy 1, policy_version 1388177 (0.0008) [2023-12-27 15:16:29,137][20381] Updated weights for policy 1, policy_version 1388187 (0.0008) [2023-12-27 15:16:29,188][20381] Updated weights for policy 1, policy_version 1388197 (0.0008) [2023-12-27 15:16:29,254][20381] Updated weights for policy 1, policy_version 1388207 (0.0008) [2023-12-27 15:16:29,496][20347] Updated weights for policy 0, policy_version 1389386 (0.0008) [2023-12-27 15:16:29,560][20347] Updated weights for policy 0, policy_version 1389396 (0.0010) [2023-12-27 15:16:29,619][20347] Updated weights for policy 0, policy_version 1389406 (0.0008) [2023-12-27 15:16:29,675][20347] Updated weights for policy 0, policy_version 1389416 (0.0005) [2023-12-27 15:16:29,970][20381] Updated weights for policy 1, policy_version 1388217 (0.0008) [2023-12-27 15:16:30,019][20381] Updated weights for policy 1, policy_version 1388227 (0.0007) [2023-12-27 15:16:30,068][20381] Updated weights for policy 1, policy_version 1388237 (0.0006) [2023-12-27 15:16:30,323][20347] Updated weights for policy 0, policy_version 1389426 (0.0008) [2023-12-27 15:16:30,381][20347] Updated weights for policy 0, policy_version 1389436 (0.0008) [2023-12-27 15:16:30,435][20347] Updated weights for policy 0, policy_version 1389446 (0.0008) [2023-12-27 15:16:30,789][20381] Updated weights for policy 1, policy_version 1388247 (0.0009) [2023-12-27 15:16:30,851][20381] Updated weights for policy 1, policy_version 1388257 (0.0010) [2023-12-27 15:16:30,911][20381] Updated weights for policy 1, policy_version 1388267 (0.0010) [2023-12-27 15:16:31,100][20347] Updated weights for policy 0, policy_version 1389456 (0.0006) [2023-12-27 15:16:31,164][20347] Updated weights for policy 0, policy_version 1389466 (0.0010) [2023-12-27 15:16:31,228][20347] Updated weights for policy 0, policy_version 1389476 (0.0008) [2023-12-27 15:16:31,589][20381] Updated weights for policy 1, policy_version 1388277 (0.0010) [2023-12-27 15:16:31,647][20381] Updated weights for policy 1, policy_version 1388287 (0.0010) [2023-12-27 15:16:31,710][20381] Updated weights for policy 1, policy_version 1388297 (0.0007) [2023-12-27 15:16:31,945][20347] Updated weights for policy 0, policy_version 1389486 (0.0008) [2023-12-27 15:16:31,995][20347] Updated weights for policy 0, policy_version 1389496 (0.0008) [2023-12-27 15:16:32,045][20347] Updated weights for policy 0, policy_version 1389506 (0.0008) [2023-12-27 15:16:32,320][20381] Updated weights for policy 1, policy_version 1388307 (0.0006) [2023-12-27 15:16:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 711221248. Throughput: 0: 10253.7, 1: 10176.3. Samples: 711232412. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:32,363][19170] Avg episode reward: [(0, '951.140'), (1, '941.510')] [2023-12-27 15:16:32,387][20381] Updated weights for policy 1, policy_version 1388317 (0.0007) [2023-12-27 15:16:32,437][20381] Updated weights for policy 1, policy_version 1388327 (0.0007) [2023-12-27 15:16:32,788][20347] Updated weights for policy 0, policy_version 1389516 (0.0007) [2023-12-27 15:16:32,844][20347] Updated weights for policy 0, policy_version 1389526 (0.0007) [2023-12-27 15:16:32,899][20347] Updated weights for policy 0, policy_version 1389536 (0.0008) [2023-12-27 15:16:33,084][20381] Updated weights for policy 1, policy_version 1388337 (0.0007) [2023-12-27 15:16:33,129][20381] Updated weights for policy 1, policy_version 1388347 (0.0010) [2023-12-27 15:16:33,182][20381] Updated weights for policy 1, policy_version 1388357 (0.0010) [2023-12-27 15:16:33,232][20381] Updated weights for policy 1, policy_version 1388367 (0.0010) [2023-12-27 15:16:33,534][20347] Updated weights for policy 0, policy_version 1389546 (0.0007) [2023-12-27 15:16:33,588][20347] Updated weights for policy 0, policy_version 1389556 (0.0006) [2023-12-27 15:16:33,639][20347] Updated weights for policy 0, policy_version 1389566 (0.0008) [2023-12-27 15:16:33,700][20347] Updated weights for policy 0, policy_version 1389576 (0.0007) [2023-12-27 15:16:33,834][20381] Updated weights for policy 1, policy_version 1388377 (0.0007) [2023-12-27 15:16:33,880][20381] Updated weights for policy 1, policy_version 1388387 (0.0008) [2023-12-27 15:16:33,928][20381] Updated weights for policy 1, policy_version 1388397 (0.0007) [2023-12-27 15:16:34,458][20347] Updated weights for policy 0, policy_version 1389586 (0.0008) [2023-12-27 15:16:34,525][20347] Updated weights for policy 0, policy_version 1389596 (0.0007) [2023-12-27 15:16:34,585][20381] Updated weights for policy 1, policy_version 1388407 (0.0007) [2023-12-27 15:16:34,586][20347] Updated weights for policy 0, policy_version 1389606 (0.0007) [2023-12-27 15:16:34,641][20381] Updated weights for policy 1, policy_version 1388417 (0.0006) [2023-12-27 15:16:34,695][20381] Updated weights for policy 1, policy_version 1388427 (0.0006) [2023-12-27 15:16:35,288][20381] Updated weights for policy 1, policy_version 1388437 (0.0006) [2023-12-27 15:16:35,288][20347] Updated weights for policy 0, policy_version 1389616 (0.0007) [2023-12-27 15:16:35,341][20381] Updated weights for policy 1, policy_version 1388447 (0.0005) [2023-12-27 15:16:35,344][20347] Updated weights for policy 0, policy_version 1389626 (0.0007) [2023-12-27 15:16:35,394][20381] Updated weights for policy 1, policy_version 1388457 (0.0006) [2023-12-27 15:16:35,408][20347] Updated weights for policy 0, policy_version 1389636 (0.0007) [2023-12-27 15:16:35,981][20381] Updated weights for policy 1, policy_version 1388467 (0.0007) [2023-12-27 15:16:36,033][20381] Updated weights for policy 1, policy_version 1388477 (0.0010) [2023-12-27 15:16:36,091][20381] Updated weights for policy 1, policy_version 1388487 (0.0011) [2023-12-27 15:16:36,151][20347] Updated weights for policy 0, policy_version 1389646 (0.0006) [2023-12-27 15:16:36,210][20347] Updated weights for policy 0, policy_version 1389656 (0.0007) [2023-12-27 15:16:36,272][20347] Updated weights for policy 0, policy_version 1389666 (0.0007) [2023-12-27 15:16:36,783][20381] Updated weights for policy 1, policy_version 1388497 (0.0010) [2023-12-27 15:16:36,840][20381] Updated weights for policy 1, policy_version 1388507 (0.0005) [2023-12-27 15:16:36,894][20381] Updated weights for policy 1, policy_version 1388517 (0.0005) [2023-12-27 15:16:36,954][20381] Updated weights for policy 1, policy_version 1388527 (0.0005) [2023-12-27 15:16:36,971][20347] Updated weights for policy 0, policy_version 1389676 (0.0008) [2023-12-27 15:16:37,034][20347] Updated weights for policy 0, policy_version 1389686 (0.0007) [2023-12-27 15:16:37,082][20347] Updated weights for policy 0, policy_version 1389696 (0.0008) [2023-12-27 15:16:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 711327744. Throughput: 0: 10211.6, 1: 10245.4. Samples: 711295228. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:37,363][19170] Avg episode reward: [(0, '932.070'), (1, '912.320')] [2023-12-27 15:16:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001389704_355835904.pth... [2023-12-27 15:16:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001388528_355491840.pth... [2023-12-27 15:16:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001388520_355532800.pth [2023-12-27 15:16:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001387344_355188736.pth [2023-12-27 15:16:37,502][20381] Updated weights for policy 1, policy_version 1388537 (0.0010) [2023-12-27 15:16:37,548][20381] Updated weights for policy 1, policy_version 1388547 (0.0010) [2023-12-27 15:16:37,594][20381] Updated weights for policy 1, policy_version 1388557 (0.0011) [2023-12-27 15:16:37,819][20347] Updated weights for policy 0, policy_version 1389706 (0.0008) [2023-12-27 15:16:37,869][20347] Updated weights for policy 0, policy_version 1389716 (0.0008) [2023-12-27 15:16:37,927][20347] Updated weights for policy 0, policy_version 1389726 (0.0009) [2023-12-27 15:16:37,982][20347] Updated weights for policy 0, policy_version 1389736 (0.0006) [2023-12-27 15:16:38,192][20381] Updated weights for policy 1, policy_version 1388567 (0.0007) [2023-12-27 15:16:38,248][20381] Updated weights for policy 1, policy_version 1388577 (0.0005) [2023-12-27 15:16:38,301][20381] Updated weights for policy 1, policy_version 1388587 (0.0006) [2023-12-27 15:16:38,705][20347] Updated weights for policy 0, policy_version 1389746 (0.0005) [2023-12-27 15:16:38,757][20347] Updated weights for policy 0, policy_version 1389756 (0.0006) [2023-12-27 15:16:38,820][20347] Updated weights for policy 0, policy_version 1389766 (0.0006) [2023-12-27 15:16:38,936][20381] Updated weights for policy 1, policy_version 1388597 (0.0007) [2023-12-27 15:16:38,987][20381] Updated weights for policy 1, policy_version 1388607 (0.0005) [2023-12-27 15:16:39,038][20381] Updated weights for policy 1, policy_version 1388617 (0.0006) [2023-12-27 15:16:39,496][20347] Updated weights for policy 0, policy_version 1389776 (0.0007) [2023-12-27 15:16:39,548][20347] Updated weights for policy 0, policy_version 1389786 (0.0008) [2023-12-27 15:16:39,601][20347] Updated weights for policy 0, policy_version 1389796 (0.0007) [2023-12-27 15:16:39,753][20381] Updated weights for policy 1, policy_version 1388627 (0.0008) [2023-12-27 15:16:39,809][20381] Updated weights for policy 1, policy_version 1388637 (0.0011) [2023-12-27 15:16:39,880][20381] Updated weights for policy 1, policy_version 1388647 (0.0010) [2023-12-27 15:16:40,369][20347] Updated weights for policy 0, policy_version 1389806 (0.0007) [2023-12-27 15:16:40,425][20347] Updated weights for policy 0, policy_version 1389816 (0.0008) [2023-12-27 15:16:40,490][20347] Updated weights for policy 0, policy_version 1389826 (0.0007) [2023-12-27 15:16:40,560][20381] Updated weights for policy 1, policy_version 1388657 (0.0009) [2023-12-27 15:16:40,624][20381] Updated weights for policy 1, policy_version 1388667 (0.0006) [2023-12-27 15:16:40,682][20381] Updated weights for policy 1, policy_version 1388677 (0.0008) [2023-12-27 15:16:40,742][20381] Updated weights for policy 1, policy_version 1388687 (0.0005) [2023-12-27 15:16:41,253][20347] Updated weights for policy 0, policy_version 1389836 (0.0008) [2023-12-27 15:16:41,317][20347] Updated weights for policy 0, policy_version 1389846 (0.0007) [2023-12-27 15:16:41,320][20381] Updated weights for policy 1, policy_version 1388697 (0.0010) [2023-12-27 15:16:41,387][20347] Updated weights for policy 0, policy_version 1389856 (0.0007) [2023-12-27 15:16:41,393][20381] Updated weights for policy 1, policy_version 1388707 (0.0009) [2023-12-27 15:16:41,459][20381] Updated weights for policy 1, policy_version 1388717 (0.0011) [2023-12-27 15:16:42,043][20347] Updated weights for policy 0, policy_version 1389866 (0.0007) [2023-12-27 15:16:42,099][20347] Updated weights for policy 0, policy_version 1389876 (0.0008) [2023-12-27 15:16:42,149][20347] Updated weights for policy 0, policy_version 1389886 (0.0007) [2023-12-27 15:16:42,203][20381] Updated weights for policy 1, policy_version 1388727 (0.0007) [2023-12-27 15:16:42,205][20347] Updated weights for policy 0, policy_version 1389896 (0.0007) [2023-12-27 15:16:42,263][20381] Updated weights for policy 1, policy_version 1388737 (0.0006) [2023-12-27 15:16:42,319][20381] Updated weights for policy 1, policy_version 1388747 (0.0008) [2023-12-27 15:16:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 711434240. Throughput: 0: 10140.3, 1: 10280.3. Samples: 711419104. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:42,363][19170] Avg episode reward: [(0, '902.760'), (1, '892.820')] [2023-12-27 15:16:42,946][20347] Updated weights for policy 0, policy_version 1389906 (0.0006) [2023-12-27 15:16:42,994][20347] Updated weights for policy 0, policy_version 1389916 (0.0005) [2023-12-27 15:16:42,999][20381] Updated weights for policy 1, policy_version 1388757 (0.0007) [2023-12-27 15:16:43,051][20347] Updated weights for policy 0, policy_version 1389926 (0.0006) [2023-12-27 15:16:43,053][20381] Updated weights for policy 1, policy_version 1388767 (0.0006) [2023-12-27 15:16:43,107][20381] Updated weights for policy 1, policy_version 1388777 (0.0008) [2023-12-27 15:16:43,739][20381] Updated weights for policy 1, policy_version 1388787 (0.0008) [2023-12-27 15:16:43,757][20347] Updated weights for policy 0, policy_version 1389936 (0.0006) [2023-12-27 15:16:43,785][20381] Updated weights for policy 1, policy_version 1388797 (0.0006) [2023-12-27 15:16:43,792][20291] KL-divergence is very high: 214.5625 [2023-12-27 15:16:43,805][20347] Updated weights for policy 0, policy_version 1389946 (0.0007) [2023-12-27 15:16:43,831][20381] Updated weights for policy 1, policy_version 1388807 (0.0006) [2023-12-27 15:16:43,833][20291] KL-divergence is very high: 233.5393 [2023-12-27 15:16:43,860][20347] Updated weights for policy 0, policy_version 1389956 (0.0008) [2023-12-27 15:16:43,877][20291] KL-divergence is very high: 113.7242 [2023-12-27 15:16:44,517][20381] Updated weights for policy 1, policy_version 1388817 (0.0008) [2023-12-27 15:16:44,554][20347] Updated weights for policy 0, policy_version 1389966 (0.0006) [2023-12-27 15:16:44,571][20381] Updated weights for policy 1, policy_version 1388827 (0.0006) [2023-12-27 15:16:44,615][20347] Updated weights for policy 0, policy_version 1389976 (0.0007) [2023-12-27 15:16:44,634][20381] Updated weights for policy 1, policy_version 1388837 (0.0006) [2023-12-27 15:16:44,675][20347] Updated weights for policy 0, policy_version 1389986 (0.0008) [2023-12-27 15:16:44,685][20381] Updated weights for policy 1, policy_version 1388847 (0.0005) [2023-12-27 15:16:45,348][20381] Updated weights for policy 1, policy_version 1388857 (0.0005) [2023-12-27 15:16:45,396][20381] Updated weights for policy 1, policy_version 1388867 (0.0005) [2023-12-27 15:16:45,409][20347] Updated weights for policy 0, policy_version 1389996 (0.0007) [2023-12-27 15:16:45,436][20292] KL-divergence is very high: 129.1402 [2023-12-27 15:16:45,447][20381] Updated weights for policy 1, policy_version 1388877 (0.0005) [2023-12-27 15:16:45,457][20292] KL-divergence is very high: 197.6068 [2023-12-27 15:16:45,466][20347] Updated weights for policy 0, policy_version 1390006 (0.0007) [2023-12-27 15:16:45,517][20347] Updated weights for policy 0, policy_version 1390016 (0.0007) [2023-12-27 15:16:46,126][20381] Updated weights for policy 1, policy_version 1388887 (0.0007) [2023-12-27 15:16:46,186][20381] Updated weights for policy 1, policy_version 1388897 (0.0008) [2023-12-27 15:16:46,235][20347] Updated weights for policy 0, policy_version 1390026 (0.0007) [2023-12-27 15:16:46,248][20381] Updated weights for policy 1, policy_version 1388907 (0.0007) [2023-12-27 15:16:46,288][20347] Updated weights for policy 0, policy_version 1390036 (0.0006) [2023-12-27 15:16:46,336][20347] Updated weights for policy 0, policy_version 1390046 (0.0008) [2023-12-27 15:16:46,388][20347] Updated weights for policy 0, policy_version 1390056 (0.0007) [2023-12-27 15:16:46,872][20381] Updated weights for policy 1, policy_version 1388917 (0.0006) [2023-12-27 15:16:46,926][20381] Updated weights for policy 1, policy_version 1388927 (0.0005) [2023-12-27 15:16:46,978][20381] Updated weights for policy 1, policy_version 1388937 (0.0007) [2023-12-27 15:16:47,109][20347] Updated weights for policy 0, policy_version 1390066 (0.0005) [2023-12-27 15:16:47,160][20347] Updated weights for policy 0, policy_version 1390076 (0.0005) [2023-12-27 15:16:47,210][20347] Updated weights for policy 0, policy_version 1390086 (0.0005) [2023-12-27 15:16:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 711532544. Throughput: 0: 10075.3, 1: 10337.9. Samples: 711540324. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:47,363][19170] Avg episode reward: [(0, '951.230'), (1, '882.890')] [2023-12-27 15:16:47,632][20381] Updated weights for policy 1, policy_version 1388947 (0.0007) [2023-12-27 15:16:47,687][20381] Updated weights for policy 1, policy_version 1388957 (0.0007) [2023-12-27 15:16:47,742][20381] Updated weights for policy 1, policy_version 1388967 (0.0007) [2023-12-27 15:16:47,842][20347] Updated weights for policy 0, policy_version 1390096 (0.0007) [2023-12-27 15:16:47,904][20347] Updated weights for policy 0, policy_version 1390106 (0.0005) [2023-12-27 15:16:47,968][20347] Updated weights for policy 0, policy_version 1390116 (0.0006) [2023-12-27 15:16:48,499][20381] Updated weights for policy 1, policy_version 1388977 (0.0007) [2023-12-27 15:16:48,536][20347] Updated weights for policy 0, policy_version 1390126 (0.0006) [2023-12-27 15:16:48,561][20381] Updated weights for policy 1, policy_version 1388987 (0.0008) [2023-12-27 15:16:48,596][20347] Updated weights for policy 0, policy_version 1390136 (0.0005) [2023-12-27 15:16:48,617][20381] Updated weights for policy 1, policy_version 1388997 (0.0007) [2023-12-27 15:16:48,655][20347] Updated weights for policy 0, policy_version 1390146 (0.0010) [2023-12-27 15:16:48,672][20381] Updated weights for policy 1, policy_version 1389007 (0.0005) [2023-12-27 15:16:49,336][20347] Updated weights for policy 0, policy_version 1390156 (0.0009) [2023-12-27 15:16:49,373][20381] Updated weights for policy 1, policy_version 1389017 (0.0007) [2023-12-27 15:16:49,399][20347] Updated weights for policy 0, policy_version 1390166 (0.0009) [2023-12-27 15:16:49,427][20381] Updated weights for policy 1, policy_version 1389027 (0.0006) [2023-12-27 15:16:49,452][20347] Updated weights for policy 0, policy_version 1390176 (0.0010) [2023-12-27 15:16:49,487][20381] Updated weights for policy 1, policy_version 1389037 (0.0005) [2023-12-27 15:16:50,066][20381] Updated weights for policy 1, policy_version 1389047 (0.0007) [2023-12-27 15:16:50,133][20381] Updated weights for policy 1, policy_version 1389057 (0.0008) [2023-12-27 15:16:50,165][20347] Updated weights for policy 0, policy_version 1390186 (0.0011) [2023-12-27 15:16:50,196][20381] Updated weights for policy 1, policy_version 1389067 (0.0008) [2023-12-27 15:16:50,221][20347] Updated weights for policy 0, policy_version 1390196 (0.0010) [2023-12-27 15:16:50,275][20347] Updated weights for policy 0, policy_version 1390206 (0.0011) [2023-12-27 15:16:50,335][20347] Updated weights for policy 0, policy_version 1390216 (0.0010) [2023-12-27 15:16:50,870][20381] Updated weights for policy 1, policy_version 1389077 (0.0007) [2023-12-27 15:16:50,921][20381] Updated weights for policy 1, policy_version 1389087 (0.0006) [2023-12-27 15:16:50,970][20381] Updated weights for policy 1, policy_version 1389097 (0.0006) [2023-12-27 15:16:50,980][20347] Updated weights for policy 0, policy_version 1390226 (0.0010) [2023-12-27 15:16:51,040][20347] Updated weights for policy 0, policy_version 1390236 (0.0009) [2023-12-27 15:16:51,099][20347] Updated weights for policy 0, policy_version 1390246 (0.0007) [2023-12-27 15:16:51,670][20381] Updated weights for policy 1, policy_version 1389107 (0.0008) [2023-12-27 15:16:51,731][20381] Updated weights for policy 1, policy_version 1389117 (0.0009) [2023-12-27 15:16:51,790][20381] Updated weights for policy 1, policy_version 1389127 (0.0008) [2023-12-27 15:16:51,870][20347] Updated weights for policy 0, policy_version 1390256 (0.0006) [2023-12-27 15:16:51,925][20347] Updated weights for policy 0, policy_version 1390266 (0.0006) [2023-12-27 15:16:51,990][20347] Updated weights for policy 0, policy_version 1390276 (0.0008) [2023-12-27 15:16:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 711630848. Throughput: 0: 10101.1, 1: 10330.1. Samples: 711602424. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:52,363][19170] Avg episode reward: [(0, '970.780'), (1, '873.090')] [2023-12-27 15:16:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001390280_355983360.pth... [2023-12-27 15:16:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001389136_355647488.pth... [2023-12-27 15:16:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001389096_355680256.pth [2023-12-27 15:16:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001387920_355336192.pth [2023-12-27 15:16:52,496][20381] Updated weights for policy 1, policy_version 1389137 (0.0008) [2023-12-27 15:16:52,548][20381] Updated weights for policy 1, policy_version 1389147 (0.0007) [2023-12-27 15:16:52,597][20381] Updated weights for policy 1, policy_version 1389157 (0.0008) [2023-12-27 15:16:52,603][20347] Updated weights for policy 0, policy_version 1390286 (0.0007) [2023-12-27 15:16:52,662][20381] Updated weights for policy 1, policy_version 1389167 (0.0007) [2023-12-27 15:16:52,664][20347] Updated weights for policy 0, policy_version 1390296 (0.0006) [2023-12-27 15:16:52,723][20347] Updated weights for policy 0, policy_version 1390306 (0.0009) [2023-12-27 15:16:53,323][20347] Updated weights for policy 0, policy_version 1390316 (0.0007) [2023-12-27 15:16:53,345][20381] Updated weights for policy 1, policy_version 1389177 (0.0005) [2023-12-27 15:16:53,383][20347] Updated weights for policy 0, policy_version 1390326 (0.0006) [2023-12-27 15:16:53,401][20381] Updated weights for policy 1, policy_version 1389187 (0.0007) [2023-12-27 15:16:53,439][20347] Updated weights for policy 0, policy_version 1390336 (0.0005) [2023-12-27 15:16:53,445][20381] Updated weights for policy 1, policy_version 1389197 (0.0008) [2023-12-27 15:16:54,044][20347] Updated weights for policy 0, policy_version 1390346 (0.0006) [2023-12-27 15:16:54,095][20347] Updated weights for policy 0, policy_version 1390356 (0.0007) [2023-12-27 15:16:54,148][20347] Updated weights for policy 0, policy_version 1390366 (0.0008) [2023-12-27 15:16:54,149][20381] Updated weights for policy 1, policy_version 1389207 (0.0008) [2023-12-27 15:16:54,203][20347] Updated weights for policy 0, policy_version 1390376 (0.0006) [2023-12-27 15:16:54,208][20381] Updated weights for policy 1, policy_version 1389217 (0.0006) [2023-12-27 15:16:54,252][20381] Updated weights for policy 1, policy_version 1389227 (0.0007) [2023-12-27 15:16:54,864][20347] Updated weights for policy 0, policy_version 1390386 (0.0006) [2023-12-27 15:16:54,910][20381] Updated weights for policy 1, policy_version 1389237 (0.0006) [2023-12-27 15:16:54,913][20347] Updated weights for policy 0, policy_version 1390396 (0.0009) [2023-12-27 15:16:54,962][20381] Updated weights for policy 1, policy_version 1389247 (0.0006) [2023-12-27 15:16:54,971][20347] Updated weights for policy 0, policy_version 1390406 (0.0011) [2023-12-27 15:16:55,017][20381] Updated weights for policy 1, policy_version 1389257 (0.0008) [2023-12-27 15:16:55,635][20347] Updated weights for policy 0, policy_version 1390416 (0.0010) [2023-12-27 15:16:55,691][20381] Updated weights for policy 1, policy_version 1389267 (0.0007) [2023-12-27 15:16:55,694][20347] Updated weights for policy 0, policy_version 1390426 (0.0010) [2023-12-27 15:16:55,747][20381] Updated weights for policy 1, policy_version 1389277 (0.0005) [2023-12-27 15:16:55,752][20347] Updated weights for policy 0, policy_version 1390436 (0.0010) [2023-12-27 15:16:55,806][20381] Updated weights for policy 1, policy_version 1389287 (0.0006) [2023-12-27 15:16:56,380][20347] Updated weights for policy 0, policy_version 1390446 (0.0007) [2023-12-27 15:16:56,393][20381] Updated weights for policy 1, policy_version 1389297 (0.0007) [2023-12-27 15:16:56,435][20347] Updated weights for policy 0, policy_version 1390456 (0.0010) [2023-12-27 15:16:56,445][20381] Updated weights for policy 1, policy_version 1389307 (0.0005) [2023-12-27 15:16:56,486][20347] Updated weights for policy 0, policy_version 1390466 (0.0010) [2023-12-27 15:16:56,489][20381] Updated weights for policy 1, policy_version 1389317 (0.0005) [2023-12-27 15:16:56,538][20381] Updated weights for policy 1, policy_version 1389327 (0.0007) [2023-12-27 15:16:57,162][20347] Updated weights for policy 0, policy_version 1390476 (0.0010) [2023-12-27 15:16:57,225][20347] Updated weights for policy 0, policy_version 1390486 (0.0010) [2023-12-27 15:16:57,231][20291] KL-divergence is very high: 215.6345 [2023-12-27 15:16:57,235][20381] Updated weights for policy 1, policy_version 1389337 (0.0005) [2023-12-27 15:16:57,273][20291] KL-divergence is very high: 334.2129 [2023-12-27 15:16:57,279][20347] Updated weights for policy 0, policy_version 1390496 (0.0010) [2023-12-27 15:16:57,280][20381] Updated weights for policy 1, policy_version 1389347 (0.0007) [2023-12-27 15:16:57,320][20291] KL-divergence is very high: 302.2708 [2023-12-27 15:16:57,333][20381] Updated weights for policy 1, policy_version 1389357 (0.0007) [2023-12-27 15:16:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 711745536. Throughput: 0: 10158.0, 1: 10340.3. Samples: 711727816. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:16:57,363][19170] Avg episode reward: [(0, '970.940'), (1, '892.680')] [2023-12-27 15:16:57,877][20347] Updated weights for policy 0, policy_version 1390506 (0.0011) [2023-12-27 15:16:57,938][20347] Updated weights for policy 0, policy_version 1390516 (0.0010) [2023-12-27 15:16:57,997][20347] Updated weights for policy 0, policy_version 1390526 (0.0010) [2023-12-27 15:16:58,053][20347] Updated weights for policy 0, policy_version 1390536 (0.0010) [2023-12-27 15:16:58,054][20381] Updated weights for policy 1, policy_version 1389367 (0.0007) [2023-12-27 15:16:58,106][20381] Updated weights for policy 1, policy_version 1389377 (0.0006) [2023-12-27 15:16:58,167][20381] Updated weights for policy 1, policy_version 1389387 (0.0009) [2023-12-27 15:16:58,768][20347] Updated weights for policy 0, policy_version 1390546 (0.0008) [2023-12-27 15:16:58,836][20347] Updated weights for policy 0, policy_version 1390556 (0.0007) [2023-12-27 15:16:58,909][20347] Updated weights for policy 0, policy_version 1390566 (0.0007) [2023-12-27 15:16:58,959][20381] Updated weights for policy 1, policy_version 1389397 (0.0009) [2023-12-27 15:16:59,026][20381] Updated weights for policy 1, policy_version 1389407 (0.0008) [2023-12-27 15:16:59,086][20381] Updated weights for policy 1, policy_version 1389417 (0.0007) [2023-12-27 15:16:59,646][20347] Updated weights for policy 0, policy_version 1390576 (0.0008) [2023-12-27 15:16:59,703][20347] Updated weights for policy 0, policy_version 1390586 (0.0009) [2023-12-27 15:16:59,762][20347] Updated weights for policy 0, policy_version 1390596 (0.0009) [2023-12-27 15:16:59,781][20381] Updated weights for policy 1, policy_version 1389427 (0.0007) [2023-12-27 15:16:59,836][20381] Updated weights for policy 1, policy_version 1389437 (0.0007) [2023-12-27 15:16:59,900][20381] Updated weights for policy 1, policy_version 1389447 (0.0008) [2023-12-27 15:17:00,467][20347] Updated weights for policy 0, policy_version 1390606 (0.0007) [2023-12-27 15:17:00,524][20347] Updated weights for policy 0, policy_version 1390616 (0.0007) [2023-12-27 15:17:00,582][20347] Updated weights for policy 0, policy_version 1390626 (0.0008) [2023-12-27 15:17:00,595][20381] Updated weights for policy 1, policy_version 1389457 (0.0009) [2023-12-27 15:17:00,655][20381] Updated weights for policy 1, policy_version 1389467 (0.0008) [2023-12-27 15:17:00,709][20381] Updated weights for policy 1, policy_version 1389477 (0.0010) [2023-12-27 15:17:00,756][20381] Updated weights for policy 1, policy_version 1389487 (0.0010) [2023-12-27 15:17:01,224][20347] Updated weights for policy 0, policy_version 1390636 (0.0007) [2023-12-27 15:17:01,277][20291] KL-divergence is very high: 120.2636 [2023-12-27 15:17:01,284][20347] Updated weights for policy 0, policy_version 1390646 (0.0008) [2023-12-27 15:17:01,316][20291] KL-divergence is very high: 101.1029 [2023-12-27 15:17:01,330][20347] Updated weights for policy 0, policy_version 1390656 (0.0009) [2023-12-27 15:17:01,397][20381] Updated weights for policy 1, policy_version 1389497 (0.0008) [2023-12-27 15:17:01,450][20381] Updated weights for policy 1, policy_version 1389507 (0.0007) [2023-12-27 15:17:01,501][20381] Updated weights for policy 1, policy_version 1389517 (0.0008) [2023-12-27 15:17:02,107][20347] Updated weights for policy 0, policy_version 1390666 (0.0007) [2023-12-27 15:17:02,127][20381] Updated weights for policy 1, policy_version 1389527 (0.0008) [2023-12-27 15:17:02,163][20347] Updated weights for policy 0, policy_version 1390676 (0.0006) [2023-12-27 15:17:02,181][20381] Updated weights for policy 1, policy_version 1389537 (0.0005) [2023-12-27 15:17:02,192][20291] KL-divergence is very high: 102.7860 [2023-12-27 15:17:02,221][20347] Updated weights for policy 0, policy_version 1390686 (0.0007) [2023-12-27 15:17:02,235][20381] Updated weights for policy 1, policy_version 1389547 (0.0005) [2023-12-27 15:17:02,281][20347] Updated weights for policy 0, policy_version 1390696 (0.0007) [2023-12-27 15:17:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 711843840. Throughput: 0: 10131.5, 1: 10333.3. Samples: 711847700. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:02,363][19170] Avg episode reward: [(0, '902.380'), (1, '922.030')] [2023-12-27 15:17:02,832][20381] Updated weights for policy 1, policy_version 1389557 (0.0006) [2023-12-27 15:17:02,878][20381] Updated weights for policy 1, policy_version 1389567 (0.0006) [2023-12-27 15:17:02,928][20381] Updated weights for policy 1, policy_version 1389577 (0.0006) [2023-12-27 15:17:02,994][20347] Updated weights for policy 0, policy_version 1390706 (0.0007) [2023-12-27 15:17:03,043][20347] Updated weights for policy 0, policy_version 1390716 (0.0007) [2023-12-27 15:17:03,101][20347] Updated weights for policy 0, policy_version 1390726 (0.0007) [2023-12-27 15:17:03,507][20381] Updated weights for policy 1, policy_version 1389587 (0.0006) [2023-12-27 15:17:03,558][20381] Updated weights for policy 1, policy_version 1389597 (0.0009) [2023-12-27 15:17:03,605][20381] Updated weights for policy 1, policy_version 1389607 (0.0010) [2023-12-27 15:17:03,757][20347] Updated weights for policy 0, policy_version 1390736 (0.0007) [2023-12-27 15:17:03,811][20347] Updated weights for policy 0, policy_version 1390746 (0.0005) [2023-12-27 15:17:03,867][20347] Updated weights for policy 0, policy_version 1390756 (0.0006) [2023-12-27 15:17:04,302][20381] Updated weights for policy 1, policy_version 1389617 (0.0010) [2023-12-27 15:17:04,355][20381] Updated weights for policy 1, policy_version 1389627 (0.0008) [2023-12-27 15:17:04,408][20381] Updated weights for policy 1, policy_version 1389637 (0.0008) [2023-12-27 15:17:04,466][20381] Updated weights for policy 1, policy_version 1389647 (0.0008) [2023-12-27 15:17:04,492][20347] Updated weights for policy 0, policy_version 1390766 (0.0007) [2023-12-27 15:17:04,556][20347] Updated weights for policy 0, policy_version 1390776 (0.0008) [2023-12-27 15:17:04,622][20347] Updated weights for policy 0, policy_version 1390786 (0.0010) [2023-12-27 15:17:05,179][20381] Updated weights for policy 1, policy_version 1389657 (0.0009) [2023-12-27 15:17:05,240][20381] Updated weights for policy 1, policy_version 1389667 (0.0009) [2023-12-27 15:17:05,259][20347] Updated weights for policy 0, policy_version 1390796 (0.0008) [2023-12-27 15:17:05,298][20381] Updated weights for policy 1, policy_version 1389677 (0.0008) [2023-12-27 15:17:05,310][20347] Updated weights for policy 0, policy_version 1390806 (0.0005) [2023-12-27 15:17:05,366][20347] Updated weights for policy 0, policy_version 1390816 (0.0008) [2023-12-27 15:17:05,912][20381] Updated weights for policy 1, policy_version 1389687 (0.0008) [2023-12-27 15:17:05,967][20381] Updated weights for policy 1, policy_version 1389697 (0.0007) [2023-12-27 15:17:05,991][20347] Updated weights for policy 0, policy_version 1390826 (0.0010) [2023-12-27 15:17:06,022][20381] Updated weights for policy 1, policy_version 1389707 (0.0007) [2023-12-27 15:17:06,046][20347] Updated weights for policy 0, policy_version 1390836 (0.0007) [2023-12-27 15:17:06,102][20347] Updated weights for policy 0, policy_version 1390846 (0.0006) [2023-12-27 15:17:06,167][20347] Updated weights for policy 0, policy_version 1390856 (0.0010) [2023-12-27 15:17:06,743][20381] Updated weights for policy 1, policy_version 1389717 (0.0008) [2023-12-27 15:17:06,797][20381] Updated weights for policy 1, policy_version 1389727 (0.0010) [2023-12-27 15:17:06,854][20381] Updated weights for policy 1, policy_version 1389737 (0.0011) [2023-12-27 15:17:06,861][20347] Updated weights for policy 0, policy_version 1390866 (0.0011) [2023-12-27 15:17:06,916][20347] Updated weights for policy 0, policy_version 1390876 (0.0010) [2023-12-27 15:17:06,974][20347] Updated weights for policy 0, policy_version 1390886 (0.0010) [2023-12-27 15:17:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 711942144. Throughput: 0: 10154.6, 1: 10393.8. Samples: 711910968. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:07,363][19170] Avg episode reward: [(0, '922.020'), (1, '892.890')] [2023-12-27 15:17:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001390888_356139008.pth... [2023-12-27 15:17:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001389744_355803136.pth... [2023-12-27 15:17:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001389704_355835904.pth [2023-12-27 15:17:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001388528_355491840.pth [2023-12-27 15:17:07,504][20381] Updated weights for policy 1, policy_version 1389747 (0.0010) [2023-12-27 15:17:07,556][20381] Updated weights for policy 1, policy_version 1389757 (0.0010) [2023-12-27 15:17:07,601][20381] Updated weights for policy 1, policy_version 1389767 (0.0010) [2023-12-27 15:17:07,657][20347] Updated weights for policy 0, policy_version 1390896 (0.0006) [2023-12-27 15:17:07,706][20347] Updated weights for policy 0, policy_version 1390906 (0.0005) [2023-12-27 15:17:07,761][20347] Updated weights for policy 0, policy_version 1390916 (0.0005) [2023-12-27 15:17:08,279][20381] Updated weights for policy 1, policy_version 1389777 (0.0010) [2023-12-27 15:17:08,331][20381] Updated weights for policy 1, policy_version 1389787 (0.0011) [2023-12-27 15:17:08,396][20381] Updated weights for policy 1, policy_version 1389797 (0.0010) [2023-12-27 15:17:08,427][20347] Updated weights for policy 0, policy_version 1390926 (0.0009) [2023-12-27 15:17:08,457][20381] Updated weights for policy 1, policy_version 1389807 (0.0011) [2023-12-27 15:17:08,492][20347] Updated weights for policy 0, policy_version 1390936 (0.0011) [2023-12-27 15:17:08,551][20347] Updated weights for policy 0, policy_version 1390946 (0.0008) [2023-12-27 15:17:09,166][20381] Updated weights for policy 1, policy_version 1389817 (0.0011) [2023-12-27 15:17:09,227][20381] Updated weights for policy 1, policy_version 1389827 (0.0011) [2023-12-27 15:17:09,234][20347] Updated weights for policy 0, policy_version 1390956 (0.0008) [2023-12-27 15:17:09,288][20381] Updated weights for policy 1, policy_version 1389837 (0.0007) [2023-12-27 15:17:09,297][20347] Updated weights for policy 0, policy_version 1390966 (0.0006) [2023-12-27 15:17:09,359][20347] Updated weights for policy 0, policy_version 1390976 (0.0006) [2023-12-27 15:17:10,019][20381] Updated weights for policy 1, policy_version 1389847 (0.0009) [2023-12-27 15:17:10,069][20347] Updated weights for policy 0, policy_version 1390986 (0.0008) [2023-12-27 15:17:10,079][20381] Updated weights for policy 1, policy_version 1389857 (0.0007) [2023-12-27 15:17:10,128][20347] Updated weights for policy 0, policy_version 1390996 (0.0006) [2023-12-27 15:17:10,138][20381] Updated weights for policy 1, policy_version 1389867 (0.0006) [2023-12-27 15:17:10,190][20347] Updated weights for policy 0, policy_version 1391006 (0.0007) [2023-12-27 15:17:10,246][20347] Updated weights for policy 0, policy_version 1391016 (0.0007) [2023-12-27 15:17:10,850][20381] Updated weights for policy 1, policy_version 1389877 (0.0006) [2023-12-27 15:17:10,914][20381] Updated weights for policy 1, policy_version 1389887 (0.0008) [2023-12-27 15:17:10,943][20347] Updated weights for policy 0, policy_version 1391026 (0.0006) [2023-12-27 15:17:10,974][20381] Updated weights for policy 1, policy_version 1389897 (0.0011) [2023-12-27 15:17:10,995][20347] Updated weights for policy 0, policy_version 1391036 (0.0005) [2023-12-27 15:17:11,052][20347] Updated weights for policy 0, policy_version 1391046 (0.0007) [2023-12-27 15:17:11,723][20381] Updated weights for policy 1, policy_version 1389907 (0.0010) [2023-12-27 15:17:11,793][20381] Updated weights for policy 1, policy_version 1389917 (0.0008) [2023-12-27 15:17:11,849][20381] Updated weights for policy 1, policy_version 1389927 (0.0008) [2023-12-27 15:17:11,880][20347] Updated weights for policy 0, policy_version 1391056 (0.0007) [2023-12-27 15:17:11,937][20347] Updated weights for policy 0, policy_version 1391066 (0.0007) [2023-12-27 15:17:11,988][20347] Updated weights for policy 0, policy_version 1391076 (0.0007) [2023-12-27 15:17:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 712040448. Throughput: 0: 10146.6, 1: 10367.9. Samples: 712032448. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:12,363][19170] Avg episode reward: [(0, '941.810'), (1, '785.290')] [2023-12-27 15:17:12,597][20381] Updated weights for policy 1, policy_version 1389937 (0.0008) [2023-12-27 15:17:12,656][20381] Updated weights for policy 1, policy_version 1389947 (0.0006) [2023-12-27 15:17:12,707][20381] Updated weights for policy 1, policy_version 1389957 (0.0007) [2023-12-27 15:17:12,712][20347] Updated weights for policy 0, policy_version 1391086 (0.0007) [2023-12-27 15:17:12,758][20381] Updated weights for policy 1, policy_version 1389967 (0.0008) [2023-12-27 15:17:12,772][20347] Updated weights for policy 0, policy_version 1391096 (0.0006) [2023-12-27 15:17:12,840][20347] Updated weights for policy 0, policy_version 1391106 (0.0008) [2023-12-27 15:17:13,398][20381] Updated weights for policy 1, policy_version 1389977 (0.0006) [2023-12-27 15:17:13,448][20347] Updated weights for policy 0, policy_version 1391116 (0.0006) [2023-12-27 15:17:13,451][20381] Updated weights for policy 1, policy_version 1389987 (0.0005) [2023-12-27 15:17:13,500][20347] Updated weights for policy 0, policy_version 1391126 (0.0005) [2023-12-27 15:17:13,502][20381] Updated weights for policy 1, policy_version 1389997 (0.0005) [2023-12-27 15:17:13,563][20347] Updated weights for policy 0, policy_version 1391136 (0.0009) [2023-12-27 15:17:14,043][20381] Updated weights for policy 1, policy_version 1390007 (0.0006) [2023-12-27 15:17:14,095][20381] Updated weights for policy 1, policy_version 1390017 (0.0008) [2023-12-27 15:17:14,156][20381] Updated weights for policy 1, policy_version 1390027 (0.0007) [2023-12-27 15:17:14,200][20347] Updated weights for policy 0, policy_version 1391146 (0.0010) [2023-12-27 15:17:14,253][20347] Updated weights for policy 0, policy_version 1391156 (0.0008) [2023-12-27 15:17:14,306][20347] Updated weights for policy 0, policy_version 1391166 (0.0008) [2023-12-27 15:17:14,362][20347] Updated weights for policy 0, policy_version 1391176 (0.0008) [2023-12-27 15:17:14,750][20381] Updated weights for policy 1, policy_version 1390037 (0.0006) [2023-12-27 15:17:14,809][20381] Updated weights for policy 1, policy_version 1390047 (0.0009) [2023-12-27 15:17:14,871][20381] Updated weights for policy 1, policy_version 1390057 (0.0006) [2023-12-27 15:17:15,054][20347] Updated weights for policy 0, policy_version 1391186 (0.0008) [2023-12-27 15:17:15,117][20347] Updated weights for policy 0, policy_version 1391196 (0.0008) [2023-12-27 15:17:15,181][20347] Updated weights for policy 0, policy_version 1391206 (0.0008) [2023-12-27 15:17:15,491][20381] Updated weights for policy 1, policy_version 1390067 (0.0008) [2023-12-27 15:17:15,546][20381] Updated weights for policy 1, policy_version 1390077 (0.0011) [2023-12-27 15:17:15,602][20381] Updated weights for policy 1, policy_version 1390087 (0.0011) [2023-12-27 15:17:15,896][20347] Updated weights for policy 0, policy_version 1391216 (0.0006) [2023-12-27 15:17:15,960][20347] Updated weights for policy 0, policy_version 1391226 (0.0007) [2023-12-27 15:17:16,022][20347] Updated weights for policy 0, policy_version 1391236 (0.0007) [2023-12-27 15:17:16,236][20381] Updated weights for policy 1, policy_version 1390097 (0.0008) [2023-12-27 15:17:16,297][20381] Updated weights for policy 1, policy_version 1390107 (0.0006) [2023-12-27 15:17:16,355][20381] Updated weights for policy 1, policy_version 1390117 (0.0007) [2023-12-27 15:17:16,415][20381] Updated weights for policy 1, policy_version 1390127 (0.0011) [2023-12-27 15:17:16,723][20347] Updated weights for policy 0, policy_version 1391246 (0.0007) [2023-12-27 15:17:16,773][20347] Updated weights for policy 0, policy_version 1391256 (0.0007) [2023-12-27 15:17:16,824][20347] Updated weights for policy 0, policy_version 1391266 (0.0007) [2023-12-27 15:17:17,067][20381] Updated weights for policy 1, policy_version 1390137 (0.0010) [2023-12-27 15:17:17,128][20381] Updated weights for policy 1, policy_version 1390147 (0.0010) [2023-12-27 15:17:17,186][20381] Updated weights for policy 1, policy_version 1390157 (0.0010) [2023-12-27 15:17:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 712146944. Throughput: 0: 10081.5, 1: 10445.8. Samples: 712156144. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:17,364][19170] Avg episode reward: [(0, '902.680'), (1, '873.040')] [2023-12-27 15:17:17,526][20347] Updated weights for policy 0, policy_version 1391276 (0.0007) [2023-12-27 15:17:17,590][20347] Updated weights for policy 0, policy_version 1391286 (0.0007) [2023-12-27 15:17:17,644][20347] Updated weights for policy 0, policy_version 1391296 (0.0008) [2023-12-27 15:17:17,865][20381] Updated weights for policy 1, policy_version 1390167 (0.0010) [2023-12-27 15:17:17,916][20381] Updated weights for policy 1, policy_version 1390177 (0.0010) [2023-12-27 15:17:17,962][20381] Updated weights for policy 1, policy_version 1390187 (0.0010) [2023-12-27 15:17:18,331][20347] Updated weights for policy 0, policy_version 1391306 (0.0008) [2023-12-27 15:17:18,388][20347] Updated weights for policy 0, policy_version 1391316 (0.0008) [2023-12-27 15:17:18,446][20347] Updated weights for policy 0, policy_version 1391327 (0.0009) [2023-12-27 15:17:18,589][20381] Updated weights for policy 1, policy_version 1390197 (0.0010) [2023-12-27 15:17:18,638][20381] Updated weights for policy 1, policy_version 1390207 (0.0010) [2023-12-27 15:17:18,692][20381] Updated weights for policy 1, policy_version 1390217 (0.0011) [2023-12-27 15:17:19,150][20347] Updated weights for policy 0, policy_version 1391337 (0.0009) [2023-12-27 15:17:19,208][20347] Updated weights for policy 0, policy_version 1391347 (0.0010) [2023-12-27 15:17:19,269][20347] Updated weights for policy 0, policy_version 1391357 (0.0008) [2023-12-27 15:17:19,339][20347] Updated weights for policy 0, policy_version 1391367 (0.0007) [2023-12-27 15:17:19,371][20381] Updated weights for policy 1, policy_version 1390227 (0.0009) [2023-12-27 15:17:19,432][20381] Updated weights for policy 1, policy_version 1390237 (0.0010) [2023-12-27 15:17:19,494][20381] Updated weights for policy 1, policy_version 1390247 (0.0009) [2023-12-27 15:17:20,064][20347] Updated weights for policy 0, policy_version 1391377 (0.0007) [2023-12-27 15:17:20,125][20347] Updated weights for policy 0, policy_version 1391387 (0.0007) [2023-12-27 15:17:20,175][20347] Updated weights for policy 0, policy_version 1391397 (0.0007) [2023-12-27 15:17:20,225][20381] Updated weights for policy 1, policy_version 1390257 (0.0007) [2023-12-27 15:17:20,283][20381] Updated weights for policy 1, policy_version 1390267 (0.0009) [2023-12-27 15:17:20,347][20381] Updated weights for policy 1, policy_version 1390277 (0.0008) [2023-12-27 15:17:20,406][20381] Updated weights for policy 1, policy_version 1390287 (0.0008) [2023-12-27 15:17:20,867][20347] Updated weights for policy 0, policy_version 1391407 (0.0010) [2023-12-27 15:17:20,932][20347] Updated weights for policy 0, policy_version 1391417 (0.0008) [2023-12-27 15:17:21,002][20347] Updated weights for policy 0, policy_version 1391427 (0.0007) [2023-12-27 15:17:21,121][20381] Updated weights for policy 1, policy_version 1390297 (0.0007) [2023-12-27 15:17:21,191][20381] Updated weights for policy 1, policy_version 1390307 (0.0007) [2023-12-27 15:17:21,254][20381] Updated weights for policy 1, policy_version 1390317 (0.0007) [2023-12-27 15:17:21,715][20347] Updated weights for policy 0, policy_version 1391437 (0.0008) [2023-12-27 15:17:21,786][20347] Updated weights for policy 0, policy_version 1391447 (0.0010) [2023-12-27 15:17:21,842][20347] Updated weights for policy 0, policy_version 1391457 (0.0008) [2023-12-27 15:17:21,898][20381] Updated weights for policy 1, policy_version 1390327 (0.0007) [2023-12-27 15:17:21,946][20381] Updated weights for policy 1, policy_version 1390337 (0.0007) [2023-12-27 15:17:21,994][20381] Updated weights for policy 1, policy_version 1390347 (0.0007) [2023-12-27 15:17:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 712245248. Throughput: 0: 10079.6, 1: 10391.9. Samples: 712216448. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:22,363][19170] Avg episode reward: [(0, '921.950'), (1, '922.020')] [2023-12-27 15:17:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001390352_355958784.pth... [2023-12-27 15:17:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001391464_356286464.pth... [2023-12-27 15:17:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001389136_355647488.pth [2023-12-27 15:17:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001390280_355983360.pth [2023-12-27 15:17:22,559][20347] Updated weights for policy 0, policy_version 1391467 (0.0011) [2023-12-27 15:17:22,622][20347] Updated weights for policy 0, policy_version 1391477 (0.0010) [2023-12-27 15:17:22,661][20381] Updated weights for policy 1, policy_version 1390357 (0.0010) [2023-12-27 15:17:22,682][20347] Updated weights for policy 0, policy_version 1391487 (0.0010) [2023-12-27 15:17:22,714][20381] Updated weights for policy 1, policy_version 1390367 (0.0011) [2023-12-27 15:17:22,768][20381] Updated weights for policy 1, policy_version 1390377 (0.0011) [2023-12-27 15:17:23,293][20347] Updated weights for policy 0, policy_version 1391497 (0.0010) [2023-12-27 15:17:23,354][20347] Updated weights for policy 0, policy_version 1391507 (0.0010) [2023-12-27 15:17:23,399][20347] Updated weights for policy 0, policy_version 1391517 (0.0010) [2023-12-27 15:17:23,426][20381] Updated weights for policy 1, policy_version 1390387 (0.0009) [2023-12-27 15:17:23,458][20347] Updated weights for policy 0, policy_version 1391527 (0.0010) [2023-12-27 15:17:23,479][20381] Updated weights for policy 1, policy_version 1390397 (0.0005) [2023-12-27 15:17:23,532][20381] Updated weights for policy 1, policy_version 1390407 (0.0007) [2023-12-27 15:17:24,120][20347] Updated weights for policy 0, policy_version 1391537 (0.0009) [2023-12-27 15:17:24,136][20381] Updated weights for policy 1, policy_version 1390417 (0.0010) [2023-12-27 15:17:24,173][20347] Updated weights for policy 0, policy_version 1391547 (0.0007) [2023-12-27 15:17:24,187][20381] Updated weights for policy 1, policy_version 1390427 (0.0006) [2023-12-27 15:17:24,230][20347] Updated weights for policy 0, policy_version 1391557 (0.0005) [2023-12-27 15:17:24,241][20381] Updated weights for policy 1, policy_version 1390437 (0.0005) [2023-12-27 15:17:24,298][20381] Updated weights for policy 1, policy_version 1390447 (0.0005) [2023-12-27 15:17:24,774][20347] Updated weights for policy 0, policy_version 1391567 (0.0007) [2023-12-27 15:17:24,841][20347] Updated weights for policy 0, policy_version 1391577 (0.0008) [2023-12-27 15:17:24,898][20381] Updated weights for policy 1, policy_version 1390457 (0.0008) [2023-12-27 15:17:24,903][20347] Updated weights for policy 0, policy_version 1391587 (0.0008) [2023-12-27 15:17:24,956][20381] Updated weights for policy 1, policy_version 1390467 (0.0011) [2023-12-27 15:17:25,015][20381] Updated weights for policy 1, policy_version 1390477 (0.0011) [2023-12-27 15:17:25,482][20347] Updated weights for policy 0, policy_version 1391597 (0.0005) [2023-12-27 15:17:25,543][20347] Updated weights for policy 0, policy_version 1391607 (0.0005) [2023-12-27 15:17:25,601][20347] Updated weights for policy 0, policy_version 1391617 (0.0005) [2023-12-27 15:17:25,683][20381] Updated weights for policy 1, policy_version 1390487 (0.0011) [2023-12-27 15:17:25,735][20381] Updated weights for policy 1, policy_version 1390497 (0.0011) [2023-12-27 15:17:25,783][20381] Updated weights for policy 1, policy_version 1390507 (0.0010) [2023-12-27 15:17:26,173][20347] Updated weights for policy 0, policy_version 1391627 (0.0007) [2023-12-27 15:17:26,219][20347] Updated weights for policy 0, policy_version 1391637 (0.0009) [2023-12-27 15:17:26,275][20347] Updated weights for policy 0, policy_version 1391647 (0.0008) [2023-12-27 15:17:26,426][20381] Updated weights for policy 1, policy_version 1390517 (0.0010) [2023-12-27 15:17:26,474][20381] Updated weights for policy 1, policy_version 1390527 (0.0010) [2023-12-27 15:17:26,527][20381] Updated weights for policy 1, policy_version 1390537 (0.0010) [2023-12-27 15:17:26,926][20347] Updated weights for policy 0, policy_version 1391657 (0.0010) [2023-12-27 15:17:26,984][20347] Updated weights for policy 0, policy_version 1391667 (0.0005) [2023-12-27 15:17:27,035][20347] Updated weights for policy 0, policy_version 1391677 (0.0005) [2023-12-27 15:17:27,083][20347] Updated weights for policy 0, policy_version 1391687 (0.0005) [2023-12-27 15:17:27,241][20381] Updated weights for policy 1, policy_version 1390547 (0.0007) [2023-12-27 15:17:27,294][20381] Updated weights for policy 1, policy_version 1390557 (0.0009) [2023-12-27 15:17:27,346][20381] Updated weights for policy 1, policy_version 1390567 (0.0010) [2023-12-27 15:17:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 712351744. Throughput: 0: 10201.1, 1: 10332.2. Samples: 712343100. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:27,363][19170] Avg episode reward: [(0, '970.710'), (1, '931.710')] [2023-12-27 15:17:27,687][20347] Updated weights for policy 0, policy_version 1391697 (0.0005) [2023-12-27 15:17:27,737][20347] Updated weights for policy 0, policy_version 1391707 (0.0008) [2023-12-27 15:17:27,783][20347] Updated weights for policy 0, policy_version 1391717 (0.0010) [2023-12-27 15:17:27,910][20381] Updated weights for policy 1, policy_version 1390577 (0.0010) [2023-12-27 15:17:27,971][20381] Updated weights for policy 1, policy_version 1390587 (0.0006) [2023-12-27 15:17:28,027][20381] Updated weights for policy 1, policy_version 1390597 (0.0010) [2023-12-27 15:17:28,084][20381] Updated weights for policy 1, policy_version 1390607 (0.0010) [2023-12-27 15:17:28,452][20347] Updated weights for policy 0, policy_version 1391727 (0.0007) [2023-12-27 15:17:28,519][20347] Updated weights for policy 0, policy_version 1391737 (0.0005) [2023-12-27 15:17:28,580][20347] Updated weights for policy 0, policy_version 1391747 (0.0005) [2023-12-27 15:17:28,786][20381] Updated weights for policy 1, policy_version 1390617 (0.0009) [2023-12-27 15:17:28,851][20381] Updated weights for policy 1, policy_version 1390627 (0.0005) [2023-12-27 15:17:28,917][20381] Updated weights for policy 1, policy_version 1390637 (0.0005) [2023-12-27 15:17:29,112][20347] Updated weights for policy 0, policy_version 1391757 (0.0006) [2023-12-27 15:17:29,163][20347] Updated weights for policy 0, policy_version 1391767 (0.0007) [2023-12-27 15:17:29,222][20347] Updated weights for policy 0, policy_version 1391777 (0.0008) [2023-12-27 15:17:29,500][20381] Updated weights for policy 1, policy_version 1390647 (0.0007) [2023-12-27 15:17:29,559][20381] Updated weights for policy 1, policy_version 1390657 (0.0008) [2023-12-27 15:17:29,614][20381] Updated weights for policy 1, policy_version 1390667 (0.0008) [2023-12-27 15:17:29,915][20347] Updated weights for policy 0, policy_version 1391787 (0.0007) [2023-12-27 15:17:29,975][20347] Updated weights for policy 0, policy_version 1391797 (0.0008) [2023-12-27 15:17:30,038][20347] Updated weights for policy 0, policy_version 1391807 (0.0007) [2023-12-27 15:17:30,284][20381] Updated weights for policy 1, policy_version 1390677 (0.0008) [2023-12-27 15:17:30,337][20381] Updated weights for policy 1, policy_version 1390687 (0.0008) [2023-12-27 15:17:30,389][20381] Updated weights for policy 1, policy_version 1390697 (0.0011) [2023-12-27 15:17:30,699][20347] Updated weights for policy 0, policy_version 1391817 (0.0007) [2023-12-27 15:17:30,755][20347] Updated weights for policy 0, policy_version 1391827 (0.0010) [2023-12-27 15:17:30,810][20347] Updated weights for policy 0, policy_version 1391837 (0.0008) [2023-12-27 15:17:30,863][20347] Updated weights for policy 0, policy_version 1391847 (0.0009) [2023-12-27 15:17:30,958][20381] Updated weights for policy 1, policy_version 1390707 (0.0009) [2023-12-27 15:17:31,019][20381] Updated weights for policy 1, policy_version 1390717 (0.0006) [2023-12-27 15:17:31,084][20381] Updated weights for policy 1, policy_version 1390727 (0.0010) [2023-12-27 15:17:31,572][20347] Updated weights for policy 0, policy_version 1391857 (0.0005) [2023-12-27 15:17:31,634][20347] Updated weights for policy 0, policy_version 1391867 (0.0007) [2023-12-27 15:17:31,691][20347] Updated weights for policy 0, policy_version 1391877 (0.0008) [2023-12-27 15:17:31,847][20381] Updated weights for policy 1, policy_version 1390737 (0.0010) [2023-12-27 15:17:31,904][20381] Updated weights for policy 1, policy_version 1390747 (0.0011) [2023-12-27 15:17:31,966][20381] Updated weights for policy 1, policy_version 1390757 (0.0011) [2023-12-27 15:17:32,024][20381] Updated weights for policy 1, policy_version 1390767 (0.0011) [2023-12-27 15:17:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 712458240. Throughput: 0: 10303.1, 1: 10361.8. Samples: 712470244. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:32,363][19170] Avg episode reward: [(0, '990.260'), (1, '892.700')] [2023-12-27 15:17:32,379][20347] Updated weights for policy 0, policy_version 1391887 (0.0008) [2023-12-27 15:17:32,437][20347] Updated weights for policy 0, policy_version 1391897 (0.0008) [2023-12-27 15:17:32,493][20347] Updated weights for policy 0, policy_version 1391907 (0.0007) [2023-12-27 15:17:32,700][20381] Updated weights for policy 1, policy_version 1390777 (0.0011) [2023-12-27 15:17:32,761][20381] Updated weights for policy 1, policy_version 1390787 (0.0010) [2023-12-27 15:17:32,818][20381] Updated weights for policy 1, policy_version 1390797 (0.0010) [2023-12-27 15:17:33,153][20347] Updated weights for policy 0, policy_version 1391917 (0.0007) [2023-12-27 15:17:33,200][20347] Updated weights for policy 0, policy_version 1391927 (0.0007) [2023-12-27 15:17:33,263][20347] Updated weights for policy 0, policy_version 1391937 (0.0007) [2023-12-27 15:17:33,431][20381] Updated weights for policy 1, policy_version 1390807 (0.0010) [2023-12-27 15:17:33,478][20381] Updated weights for policy 1, policy_version 1390817 (0.0010) [2023-12-27 15:17:33,529][20381] Updated weights for policy 1, policy_version 1390827 (0.0010) [2023-12-27 15:17:33,957][20347] Updated weights for policy 0, policy_version 1391947 (0.0007) [2023-12-27 15:17:34,019][20347] Updated weights for policy 0, policy_version 1391957 (0.0007) [2023-12-27 15:17:34,079][20347] Updated weights for policy 0, policy_version 1391967 (0.0008) [2023-12-27 15:17:34,198][20381] Updated weights for policy 1, policy_version 1390837 (0.0009) [2023-12-27 15:17:34,258][20381] Updated weights for policy 1, policy_version 1390847 (0.0007) [2023-12-27 15:17:34,321][20381] Updated weights for policy 1, policy_version 1390857 (0.0007) [2023-12-27 15:17:34,751][20347] Updated weights for policy 0, policy_version 1391977 (0.0007) [2023-12-27 15:17:34,805][20347] Updated weights for policy 0, policy_version 1391987 (0.0006) [2023-12-27 15:17:34,858][20347] Updated weights for policy 0, policy_version 1391997 (0.0008) [2023-12-27 15:17:34,902][20347] Updated weights for policy 0, policy_version 1392007 (0.0007) [2023-12-27 15:17:35,008][20381] Updated weights for policy 1, policy_version 1390867 (0.0009) [2023-12-27 15:17:35,059][20381] Updated weights for policy 1, policy_version 1390877 (0.0006) [2023-12-27 15:17:35,121][20381] Updated weights for policy 1, policy_version 1390887 (0.0008) [2023-12-27 15:17:35,603][20347] Updated weights for policy 0, policy_version 1392017 (0.0009) [2023-12-27 15:17:35,651][20347] Updated weights for policy 0, policy_version 1392027 (0.0007) [2023-12-27 15:17:35,694][20381] Updated weights for policy 1, policy_version 1390897 (0.0007) [2023-12-27 15:17:35,705][20347] Updated weights for policy 0, policy_version 1392037 (0.0010) [2023-12-27 15:17:35,752][20381] Updated weights for policy 1, policy_version 1390907 (0.0007) [2023-12-27 15:17:35,814][20381] Updated weights for policy 1, policy_version 1390917 (0.0006) [2023-12-27 15:17:35,872][20381] Updated weights for policy 1, policy_version 1390927 (0.0008) [2023-12-27 15:17:36,382][20347] Updated weights for policy 0, policy_version 1392047 (0.0010) [2023-12-27 15:17:36,438][20347] Updated weights for policy 0, policy_version 1392057 (0.0010) [2023-12-27 15:17:36,496][20347] Updated weights for policy 0, policy_version 1392067 (0.0010) [2023-12-27 15:17:36,535][20381] Updated weights for policy 1, policy_version 1390937 (0.0007) [2023-12-27 15:17:36,593][20381] Updated weights for policy 1, policy_version 1390947 (0.0007) [2023-12-27 15:17:36,648][20381] Updated weights for policy 1, policy_version 1390957 (0.0008) [2023-12-27 15:17:37,129][20347] Updated weights for policy 0, policy_version 1392077 (0.0010) [2023-12-27 15:17:37,178][20347] Updated weights for policy 0, policy_version 1392087 (0.0010) [2023-12-27 15:17:37,228][20347] Updated weights for policy 0, policy_version 1392097 (0.0007) [2023-12-27 15:17:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.4, 300 sec: 20521.6). Total num frames: 712564736. Throughput: 0: 10272.5, 1: 10374.0. Samples: 712531520. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:37,364][19170] Avg episode reward: [(0, '941.330'), (1, '922.020')] [2023-12-27 15:17:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001392104_356450304.pth... [2023-12-27 15:17:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001390888_356139008.pth [2023-12-27 15:17:37,395][20381] Updated weights for policy 1, policy_version 1390967 (0.0008) [2023-12-27 15:17:37,447][20381] Updated weights for policy 1, policy_version 1390977 (0.0009) [2023-12-27 15:17:37,505][20381] Updated weights for policy 1, policy_version 1390987 (0.0009) [2023-12-27 15:17:37,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001390992_356122624.pth... [2023-12-27 15:17:37,539][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001389744_355803136.pth [2023-12-27 15:17:37,852][20347] Updated weights for policy 0, policy_version 1392107 (0.0007) [2023-12-27 15:17:37,910][20347] Updated weights for policy 0, policy_version 1392117 (0.0007) [2023-12-27 15:17:37,956][20347] Updated weights for policy 0, policy_version 1392127 (0.0007) [2023-12-27 15:17:38,247][20381] Updated weights for policy 1, policy_version 1390997 (0.0008) [2023-12-27 15:17:38,296][20381] Updated weights for policy 1, policy_version 1391007 (0.0008) [2023-12-27 15:17:38,360][20381] Updated weights for policy 1, policy_version 1391017 (0.0009) [2023-12-27 15:17:38,567][20347] Updated weights for policy 0, policy_version 1392137 (0.0006) [2023-12-27 15:17:38,630][20347] Updated weights for policy 0, policy_version 1392147 (0.0006) [2023-12-27 15:17:38,691][20347] Updated weights for policy 0, policy_version 1392157 (0.0007) [2023-12-27 15:17:38,751][20347] Updated weights for policy 0, policy_version 1392167 (0.0007) [2023-12-27 15:17:39,076][20381] Updated weights for policy 1, policy_version 1391027 (0.0008) [2023-12-27 15:17:39,131][20381] Updated weights for policy 1, policy_version 1391037 (0.0008) [2023-12-27 15:17:39,187][20381] Updated weights for policy 1, policy_version 1391047 (0.0008) [2023-12-27 15:17:39,438][20347] Updated weights for policy 0, policy_version 1392177 (0.0008) [2023-12-27 15:17:39,493][20347] Updated weights for policy 0, policy_version 1392187 (0.0009) [2023-12-27 15:17:39,549][20347] Updated weights for policy 0, policy_version 1392197 (0.0009) [2023-12-27 15:17:39,912][20381] Updated weights for policy 1, policy_version 1391057 (0.0008) [2023-12-27 15:17:39,976][20381] Updated weights for policy 1, policy_version 1391067 (0.0008) [2023-12-27 15:17:40,035][20381] Updated weights for policy 1, policy_version 1391077 (0.0008) [2023-12-27 15:17:40,097][20381] Updated weights for policy 1, policy_version 1391087 (0.0008) [2023-12-27 15:17:40,291][20347] Updated weights for policy 0, policy_version 1392207 (0.0006) [2023-12-27 15:17:40,359][20347] Updated weights for policy 0, policy_version 1392217 (0.0006) [2023-12-27 15:17:40,416][20347] Updated weights for policy 0, policy_version 1392227 (0.0006) [2023-12-27 15:17:40,749][20381] Updated weights for policy 1, policy_version 1391097 (0.0006) [2023-12-27 15:17:40,811][20381] Updated weights for policy 1, policy_version 1391107 (0.0008) [2023-12-27 15:17:40,871][20381] Updated weights for policy 1, policy_version 1391117 (0.0008) [2023-12-27 15:17:41,018][20347] Updated weights for policy 0, policy_version 1392237 (0.0007) [2023-12-27 15:17:41,095][20347] Updated weights for policy 0, policy_version 1392247 (0.0007) [2023-12-27 15:17:41,168][20347] Updated weights for policy 0, policy_version 1392257 (0.0008) [2023-12-27 15:17:41,595][20381] Updated weights for policy 1, policy_version 1391127 (0.0008) [2023-12-27 15:17:41,666][20381] Updated weights for policy 1, policy_version 1391137 (0.0008) [2023-12-27 15:17:41,724][20381] Updated weights for policy 1, policy_version 1391147 (0.0008) [2023-12-27 15:17:41,872][20347] Updated weights for policy 0, policy_version 1392267 (0.0006) [2023-12-27 15:17:41,931][20347] Updated weights for policy 0, policy_version 1392277 (0.0006) [2023-12-27 15:17:41,990][20347] Updated weights for policy 0, policy_version 1392287 (0.0005) [2023-12-27 15:17:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 712663040. Throughput: 0: 10262.7, 1: 10335.1. Samples: 712654720. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:42,363][19170] Avg episode reward: [(0, '921.780'), (1, '941.560')] [2023-12-27 15:17:42,375][20381] Updated weights for policy 1, policy_version 1391157 (0.0008) [2023-12-27 15:17:42,434][20381] Updated weights for policy 1, policy_version 1391167 (0.0008) [2023-12-27 15:17:42,490][20381] Updated weights for policy 1, policy_version 1391177 (0.0005) [2023-12-27 15:17:42,664][20347] Updated weights for policy 0, policy_version 1392297 (0.0006) [2023-12-27 15:17:42,724][20347] Updated weights for policy 0, policy_version 1392307 (0.0005) [2023-12-27 15:17:42,772][20347] Updated weights for policy 0, policy_version 1392317 (0.0007) [2023-12-27 15:17:42,818][20347] Updated weights for policy 0, policy_version 1392327 (0.0006) [2023-12-27 15:17:43,168][20381] Updated weights for policy 1, policy_version 1391187 (0.0008) [2023-12-27 15:17:43,227][20381] Updated weights for policy 1, policy_version 1391197 (0.0007) [2023-12-27 15:17:43,285][20381] Updated weights for policy 1, policy_version 1391207 (0.0005) [2023-12-27 15:17:43,413][20347] Updated weights for policy 0, policy_version 1392337 (0.0007) [2023-12-27 15:17:43,414][20291] KL-divergence is very high: 107.3288 [2023-12-27 15:17:43,464][20291] KL-divergence is very high: 136.0028 [2023-12-27 15:17:43,476][20347] Updated weights for policy 0, policy_version 1392347 (0.0008) [2023-12-27 15:17:43,499][20291] KL-divergence is very high: 156.4644 [2023-12-27 15:17:43,513][20291] KL-divergence is very high: 163.3611 [2023-12-27 15:17:43,539][20347] Updated weights for policy 0, policy_version 1392357 (0.0008) [2023-12-27 15:17:43,550][20291] KL-divergence is very high: 144.3518 [2023-12-27 15:17:43,864][20381] Updated weights for policy 1, policy_version 1391217 (0.0007) [2023-12-27 15:17:43,916][20381] Updated weights for policy 1, policy_version 1391227 (0.0005) [2023-12-27 15:17:43,972][20381] Updated weights for policy 1, policy_version 1391237 (0.0005) [2023-12-27 15:17:44,025][20381] Updated weights for policy 1, policy_version 1391247 (0.0005) [2023-12-27 15:17:44,248][20347] Updated weights for policy 0, policy_version 1392367 (0.0008) [2023-12-27 15:17:44,312][20347] Updated weights for policy 0, policy_version 1392377 (0.0007) [2023-12-27 15:17:44,367][20347] Updated weights for policy 0, policy_version 1392387 (0.0008) [2023-12-27 15:17:44,624][20381] Updated weights for policy 1, policy_version 1391257 (0.0007) [2023-12-27 15:17:44,687][20381] Updated weights for policy 1, policy_version 1391267 (0.0005) [2023-12-27 15:17:44,752][20381] Updated weights for policy 1, policy_version 1391277 (0.0006) [2023-12-27 15:17:45,072][20347] Updated weights for policy 0, policy_version 1392397 (0.0009) [2023-12-27 15:17:45,135][20347] Updated weights for policy 0, policy_version 1392407 (0.0011) [2023-12-27 15:17:45,194][20347] Updated weights for policy 0, policy_version 1392417 (0.0010) [2023-12-27 15:17:45,333][20381] Updated weights for policy 1, policy_version 1391287 (0.0007) [2023-12-27 15:17:45,395][20381] Updated weights for policy 1, policy_version 1391297 (0.0005) [2023-12-27 15:17:45,445][20381] Updated weights for policy 1, policy_version 1391307 (0.0005) [2023-12-27 15:17:45,763][20347] Updated weights for policy 0, policy_version 1392427 (0.0009) [2023-12-27 15:17:45,794][20291] KL-divergence is very high: 159.5186 [2023-12-27 15:17:45,801][20291] KL-divergence is very high: 294.3800 [2023-12-27 15:17:45,806][20291] KL-divergence is very high: 325.9491 [2023-12-27 15:17:45,818][20347] Updated weights for policy 0, policy_version 1392437 (0.0005) [2023-12-27 15:17:45,817][20291] KL-divergence is very high: 226.1028 [2023-12-27 15:17:45,823][20291] KL-divergence is very high: 235.9214 [2023-12-27 15:17:45,872][20347] Updated weights for policy 0, policy_version 1392447 (0.0005) [2023-12-27 15:17:46,013][20381] Updated weights for policy 1, policy_version 1391317 (0.0006) [2023-12-27 15:17:46,064][20381] Updated weights for policy 1, policy_version 1391327 (0.0005) [2023-12-27 15:17:46,121][20381] Updated weights for policy 1, policy_version 1391337 (0.0006) [2023-12-27 15:17:46,435][20347] Updated weights for policy 0, policy_version 1392457 (0.0006) [2023-12-27 15:17:46,497][20347] Updated weights for policy 0, policy_version 1392467 (0.0010) [2023-12-27 15:17:46,560][20347] Updated weights for policy 0, policy_version 1392477 (0.0010) [2023-12-27 15:17:46,620][20347] Updated weights for policy 0, policy_version 1392487 (0.0011) [2023-12-27 15:17:46,801][20381] Updated weights for policy 1, policy_version 1391347 (0.0007) [2023-12-27 15:17:46,855][20381] Updated weights for policy 1, policy_version 1391357 (0.0007) [2023-12-27 15:17:46,915][20381] Updated weights for policy 1, policy_version 1391367 (0.0006) [2023-12-27 15:17:47,290][20347] Updated weights for policy 0, policy_version 1392497 (0.0010) [2023-12-27 15:17:47,347][20347] Updated weights for policy 0, policy_version 1392507 (0.0010) [2023-12-27 15:17:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 712769536. Throughput: 0: 10330.7, 1: 10437.6. Samples: 712782272. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:47,363][19170] Avg episode reward: [(0, '912.120'), (1, '932.280')] [2023-12-27 15:17:47,407][20347] Updated weights for policy 0, policy_version 1392517 (0.0010) [2023-12-27 15:17:47,594][20381] Updated weights for policy 1, policy_version 1391377 (0.0008) [2023-12-27 15:17:47,648][20381] Updated weights for policy 1, policy_version 1391387 (0.0008) [2023-12-27 15:17:47,708][20381] Updated weights for policy 1, policy_version 1391397 (0.0008) [2023-12-27 15:17:47,765][20381] Updated weights for policy 1, policy_version 1391407 (0.0008) [2023-12-27 15:17:48,069][20347] Updated weights for policy 0, policy_version 1392527 (0.0007) [2023-12-27 15:17:48,124][20347] Updated weights for policy 0, policy_version 1392537 (0.0010) [2023-12-27 15:17:48,185][20347] Updated weights for policy 0, policy_version 1392547 (0.0006) [2023-12-27 15:17:48,401][20381] Updated weights for policy 1, policy_version 1391417 (0.0007) [2023-12-27 15:17:48,459][20381] Updated weights for policy 1, policy_version 1391427 (0.0007) [2023-12-27 15:17:48,461][20292] KL-divergence is very high: 197.1065 [2023-12-27 15:17:48,507][20292] KL-divergence is very high: 113.2396 [2023-12-27 15:17:48,517][20381] Updated weights for policy 1, policy_version 1391437 (0.0007) [2023-12-27 15:17:48,858][20347] Updated weights for policy 0, policy_version 1392557 (0.0008) [2023-12-27 15:17:48,916][20347] Updated weights for policy 0, policy_version 1392567 (0.0010) [2023-12-27 15:17:48,970][20347] Updated weights for policy 0, policy_version 1392577 (0.0010) [2023-12-27 15:17:49,214][20381] Updated weights for policy 1, policy_version 1391447 (0.0008) [2023-12-27 15:17:49,275][20381] Updated weights for policy 1, policy_version 1391457 (0.0007) [2023-12-27 15:17:49,301][20292] KL-divergence is very high: 101.6142 [2023-12-27 15:17:49,333][20381] Updated weights for policy 1, policy_version 1391467 (0.0007) [2023-12-27 15:17:49,676][20347] Updated weights for policy 0, policy_version 1392587 (0.0010) [2023-12-27 15:17:49,728][20347] Updated weights for policy 0, policy_version 1392597 (0.0005) [2023-12-27 15:17:49,785][20347] Updated weights for policy 0, policy_version 1392607 (0.0005) [2023-12-27 15:17:49,991][20381] Updated weights for policy 1, policy_version 1391477 (0.0009) [2023-12-27 15:17:50,045][20381] Updated weights for policy 1, policy_version 1391487 (0.0008) [2023-12-27 15:17:50,112][20381] Updated weights for policy 1, policy_version 1391497 (0.0009) [2023-12-27 15:17:50,417][20347] Updated weights for policy 0, policy_version 1392617 (0.0007) [2023-12-27 15:17:50,475][20347] Updated weights for policy 0, policy_version 1392627 (0.0007) [2023-12-27 15:17:50,532][20347] Updated weights for policy 0, policy_version 1392637 (0.0008) [2023-12-27 15:17:50,589][20347] Updated weights for policy 0, policy_version 1392647 (0.0007) [2023-12-27 15:17:50,809][20381] Updated weights for policy 1, policy_version 1391507 (0.0009) [2023-12-27 15:17:50,859][20381] Updated weights for policy 1, policy_version 1391517 (0.0007) [2023-12-27 15:17:50,914][20381] Updated weights for policy 1, policy_version 1391527 (0.0008) [2023-12-27 15:17:51,247][20347] Updated weights for policy 0, policy_version 1392657 (0.0008) [2023-12-27 15:17:51,311][20347] Updated weights for policy 0, policy_version 1392667 (0.0010) [2023-12-27 15:17:51,381][20347] Updated weights for policy 0, policy_version 1392677 (0.0011) [2023-12-27 15:17:51,615][20381] Updated weights for policy 1, policy_version 1391537 (0.0006) [2023-12-27 15:17:51,676][20381] Updated weights for policy 1, policy_version 1391547 (0.0008) [2023-12-27 15:17:51,732][20381] Updated weights for policy 1, policy_version 1391557 (0.0008) [2023-12-27 15:17:51,779][20292] KL-divergence is very high: 188.7908 [2023-12-27 15:17:51,797][20381] Updated weights for policy 1, policy_version 1391567 (0.0007) [2023-12-27 15:17:52,134][20347] Updated weights for policy 0, policy_version 1392687 (0.0010) [2023-12-27 15:17:52,188][20347] Updated weights for policy 0, policy_version 1392697 (0.0005) [2023-12-27 15:17:52,237][20347] Updated weights for policy 0, policy_version 1392707 (0.0006) [2023-12-27 15:17:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 712876032. Throughput: 0: 10322.6, 1: 10400.0. Samples: 712843484. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:52,363][19170] Avg episode reward: [(0, '931.540'), (1, '873.020')] [2023-12-27 15:17:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001392712_356605952.pth... [2023-12-27 15:17:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001391568_356270080.pth... [2023-12-27 15:17:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001391464_356286464.pth [2023-12-27 15:17:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001390352_355958784.pth [2023-12-27 15:17:52,479][20381] Updated weights for policy 1, policy_version 1391577 (0.0007) [2023-12-27 15:17:52,534][20381] Updated weights for policy 1, policy_version 1391587 (0.0007) [2023-12-27 15:17:52,596][20381] Updated weights for policy 1, policy_version 1391597 (0.0006) [2023-12-27 15:17:52,914][20347] Updated weights for policy 0, policy_version 1392717 (0.0008) [2023-12-27 15:17:52,970][20347] Updated weights for policy 0, policy_version 1392727 (0.0007) [2023-12-27 15:17:53,026][20347] Updated weights for policy 0, policy_version 1392737 (0.0008) [2023-12-27 15:17:53,289][20381] Updated weights for policy 1, policy_version 1391607 (0.0007) [2023-12-27 15:17:53,350][20381] Updated weights for policy 1, policy_version 1391617 (0.0006) [2023-12-27 15:17:53,406][20381] Updated weights for policy 1, policy_version 1391627 (0.0005) [2023-12-27 15:17:53,636][20347] Updated weights for policy 0, policy_version 1392747 (0.0007) [2023-12-27 15:17:53,687][20347] Updated weights for policy 0, policy_version 1392757 (0.0005) [2023-12-27 15:17:53,747][20347] Updated weights for policy 0, policy_version 1392767 (0.0005) [2023-12-27 15:17:53,994][20381] Updated weights for policy 1, policy_version 1391637 (0.0005) [2023-12-27 15:17:54,014][20292] KL-divergence is very high: 117.6189 [2023-12-27 15:17:54,024][20292] KL-divergence is very high: 120.2304 [2023-12-27 15:17:54,044][20381] Updated weights for policy 1, policy_version 1391647 (0.0007) [2023-12-27 15:17:54,101][20381] Updated weights for policy 1, policy_version 1391657 (0.0006) [2023-12-27 15:17:54,410][20347] Updated weights for policy 0, policy_version 1392777 (0.0007) [2023-12-27 15:17:54,464][20347] Updated weights for policy 0, policy_version 1392787 (0.0009) [2023-12-27 15:17:54,525][20347] Updated weights for policy 0, policy_version 1392797 (0.0008) [2023-12-27 15:17:54,578][20347] Updated weights for policy 0, policy_version 1392807 (0.0008) [2023-12-27 15:17:54,708][20381] Updated weights for policy 1, policy_version 1391667 (0.0007) [2023-12-27 15:17:54,758][20381] Updated weights for policy 1, policy_version 1391677 (0.0008) [2023-12-27 15:17:54,805][20381] Updated weights for policy 1, policy_version 1391687 (0.0007) [2023-12-27 15:17:55,293][20347] Updated weights for policy 0, policy_version 1392817 (0.0010) [2023-12-27 15:17:55,346][20347] Updated weights for policy 0, policy_version 1392827 (0.0010) [2023-12-27 15:17:55,398][20347] Updated weights for policy 0, policy_version 1392837 (0.0010) [2023-12-27 15:17:55,539][20381] Updated weights for policy 1, policy_version 1391697 (0.0007) [2023-12-27 15:17:55,593][20381] Updated weights for policy 1, policy_version 1391707 (0.0008) [2023-12-27 15:17:55,645][20381] Updated weights for policy 1, policy_version 1391717 (0.0008) [2023-12-27 15:17:55,701][20381] Updated weights for policy 1, policy_version 1391727 (0.0008) [2023-12-27 15:17:56,029][20347] Updated weights for policy 0, policy_version 1392847 (0.0008) [2023-12-27 15:17:56,084][20347] Updated weights for policy 0, policy_version 1392857 (0.0006) [2023-12-27 15:17:56,102][20291] KL-divergence is very high: 137.1929 [2023-12-27 15:17:56,115][20291] KL-divergence is very high: 211.8039 [2023-12-27 15:17:56,145][20347] Updated weights for policy 0, policy_version 1392867 (0.0005) [2023-12-27 15:17:56,153][20291] KL-divergence is very high: 328.5691 [2023-12-27 15:17:56,163][20291] KL-divergence is very high: 312.9406 [2023-12-27 15:17:56,447][20381] Updated weights for policy 1, policy_version 1391737 (0.0007) [2023-12-27 15:17:56,494][20381] Updated weights for policy 1, policy_version 1391747 (0.0006) [2023-12-27 15:17:56,544][20381] Updated weights for policy 1, policy_version 1391757 (0.0008) [2023-12-27 15:17:56,731][20291] KL-divergence is very high: 281.8883 [2023-12-27 15:17:56,760][20347] Updated weights for policy 0, policy_version 1392877 (0.0008) [2023-12-27 15:17:56,777][20291] KL-divergence is very high: 123.0203 [2023-12-27 15:17:56,817][20347] Updated weights for policy 0, policy_version 1392887 (0.0010) [2023-12-27 15:17:56,824][20291] KL-divergence is very high: 101.1327 [2023-12-27 15:17:56,877][20347] Updated weights for policy 0, policy_version 1392897 (0.0010) [2023-12-27 15:17:57,257][20381] Updated weights for policy 1, policy_version 1391767 (0.0009) [2023-12-27 15:17:57,310][20381] Updated weights for policy 1, policy_version 1391777 (0.0009) [2023-12-27 15:17:57,355][20381] Updated weights for policy 1, policy_version 1391787 (0.0006) [2023-12-27 15:17:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 712974336. Throughput: 0: 10339.6, 1: 10443.0. Samples: 712967664. Policy #0 lag: (min: 17.0, avg: 39.3, max: 49.0) [2023-12-27 15:17:57,363][19170] Avg episode reward: [(0, '931.380'), (1, '911.990')] [2023-12-27 15:17:57,542][20347] Updated weights for policy 0, policy_version 1392907 (0.0009) [2023-12-27 15:17:57,598][20347] Updated weights for policy 0, policy_version 1392917 (0.0007) [2023-12-27 15:17:57,608][20291] KL-divergence is very high: 487.2960 [2023-12-27 15:17:57,649][20347] Updated weights for policy 0, policy_version 1392927 (0.0010) [2023-12-27 15:17:57,649][20291] KL-divergence is very high: 421.1469 [2023-12-27 15:17:57,689][20291] KL-divergence is very high: 181.0057 [2023-12-27 15:17:57,944][20381] Updated weights for policy 1, policy_version 1391797 (0.0007) [2023-12-27 15:17:57,985][20381] Updated weights for policy 1, policy_version 1391807 (0.0010) [2023-12-27 15:17:58,034][20381] Updated weights for policy 1, policy_version 1391817 (0.0006) [2023-12-27 15:17:58,332][20347] Updated weights for policy 0, policy_version 1392937 (0.0010) [2023-12-27 15:17:58,396][20347] Updated weights for policy 0, policy_version 1392947 (0.0008) [2023-12-27 15:17:58,461][20347] Updated weights for policy 0, policy_version 1392957 (0.0008) [2023-12-27 15:17:58,526][20347] Updated weights for policy 0, policy_version 1392967 (0.0006) [2023-12-27 15:17:58,766][20381] Updated weights for policy 1, policy_version 1391827 (0.0006) [2023-12-27 15:17:58,835][20381] Updated weights for policy 1, policy_version 1391837 (0.0008) [2023-12-27 15:17:58,900][20381] Updated weights for policy 1, policy_version 1391847 (0.0008) [2023-12-27 15:17:59,266][20347] Updated weights for policy 0, policy_version 1392977 (0.0008) [2023-12-27 15:17:59,328][20347] Updated weights for policy 0, policy_version 1392987 (0.0007) [2023-12-27 15:17:59,394][20347] Updated weights for policy 0, policy_version 1392997 (0.0007) [2023-12-27 15:17:59,610][20381] Updated weights for policy 1, policy_version 1391857 (0.0009) [2023-12-27 15:17:59,667][20381] Updated weights for policy 1, policy_version 1391867 (0.0006) [2023-12-27 15:17:59,723][20381] Updated weights for policy 1, policy_version 1391877 (0.0005) [2023-12-27 15:17:59,778][20381] Updated weights for policy 1, policy_version 1391887 (0.0005) [2023-12-27 15:18:00,127][20347] Updated weights for policy 0, policy_version 1393007 (0.0008) [2023-12-27 15:18:00,186][20347] Updated weights for policy 0, policy_version 1393017 (0.0008) [2023-12-27 15:18:00,242][20347] Updated weights for policy 0, policy_version 1393027 (0.0009) [2023-12-27 15:18:00,354][20381] Updated weights for policy 1, policy_version 1391897 (0.0007) [2023-12-27 15:18:00,408][20381] Updated weights for policy 1, policy_version 1391907 (0.0008) [2023-12-27 15:18:00,465][20381] Updated weights for policy 1, policy_version 1391917 (0.0007) [2023-12-27 15:18:00,918][20347] Updated weights for policy 0, policy_version 1393037 (0.0006) [2023-12-27 15:18:00,970][20347] Updated weights for policy 0, policy_version 1393047 (0.0006) [2023-12-27 15:18:01,023][20347] Updated weights for policy 0, policy_version 1393057 (0.0007) [2023-12-27 15:18:01,118][20381] Updated weights for policy 1, policy_version 1391927 (0.0007) [2023-12-27 15:18:01,184][20381] Updated weights for policy 1, policy_version 1391937 (0.0006) [2023-12-27 15:18:01,242][20381] Updated weights for policy 1, policy_version 1391947 (0.0006) [2023-12-27 15:18:01,738][20347] Updated weights for policy 0, policy_version 1393067 (0.0008) [2023-12-27 15:18:01,799][20347] Updated weights for policy 0, policy_version 1393077 (0.0007) [2023-12-27 15:18:01,844][20291] KL-divergence is very high: 139.9302 [2023-12-27 15:18:01,855][20347] Updated weights for policy 0, policy_version 1393087 (0.0005) [2023-12-27 15:18:01,875][20291] KL-divergence is very high: 148.4955 [2023-12-27 15:18:01,895][20291] KL-divergence is very high: 130.2598 [2023-12-27 15:18:01,985][20381] Updated weights for policy 1, policy_version 1391957 (0.0006) [2023-12-27 15:18:02,048][20381] Updated weights for policy 1, policy_version 1391967 (0.0007) [2023-12-27 15:18:02,110][20381] Updated weights for policy 1, policy_version 1391977 (0.0008) [2023-12-27 15:18:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 713080832. Throughput: 0: 10366.0, 1: 10384.6. Samples: 713089916. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:02,363][19170] Avg episode reward: [(0, '921.620'), (1, '922.200')] [2023-12-27 15:18:02,461][20347] Updated weights for policy 0, policy_version 1393097 (0.0006) [2023-12-27 15:18:02,518][20347] Updated weights for policy 0, policy_version 1393107 (0.0007) [2023-12-27 15:18:02,581][20347] Updated weights for policy 0, policy_version 1393117 (0.0008) [2023-12-27 15:18:02,639][20347] Updated weights for policy 0, policy_version 1393127 (0.0008) [2023-12-27 15:18:02,808][20381] Updated weights for policy 1, policy_version 1391987 (0.0008) [2023-12-27 15:18:02,856][20381] Updated weights for policy 1, policy_version 1391997 (0.0008) [2023-12-27 15:18:02,913][20381] Updated weights for policy 1, policy_version 1392007 (0.0007) [2023-12-27 15:18:03,311][20347] Updated weights for policy 0, policy_version 1393137 (0.0007) [2023-12-27 15:18:03,361][20347] Updated weights for policy 0, policy_version 1393147 (0.0007) [2023-12-27 15:18:03,415][20347] Updated weights for policy 0, policy_version 1393157 (0.0007) [2023-12-27 15:18:03,613][20381] Updated weights for policy 1, policy_version 1392017 (0.0008) [2023-12-27 15:18:03,665][20381] Updated weights for policy 1, policy_version 1392027 (0.0008) [2023-12-27 15:18:03,710][20381] Updated weights for policy 1, policy_version 1392037 (0.0007) [2023-12-27 15:18:03,754][20381] Updated weights for policy 1, policy_version 1392047 (0.0007) [2023-12-27 15:18:04,077][20347] Updated weights for policy 0, policy_version 1393167 (0.0008) [2023-12-27 15:18:04,132][20347] Updated weights for policy 0, policy_version 1393177 (0.0007) [2023-12-27 15:18:04,191][20347] Updated weights for policy 0, policy_version 1393187 (0.0007) [2023-12-27 15:18:04,494][20381] Updated weights for policy 1, policy_version 1392057 (0.0008) [2023-12-27 15:18:04,548][20381] Updated weights for policy 1, policy_version 1392067 (0.0007) [2023-12-27 15:18:04,603][20381] Updated weights for policy 1, policy_version 1392077 (0.0006) [2023-12-27 15:18:04,912][20347] Updated weights for policy 0, policy_version 1393197 (0.0007) [2023-12-27 15:18:04,966][20347] Updated weights for policy 0, policy_version 1393207 (0.0008) [2023-12-27 15:18:05,011][20347] Updated weights for policy 0, policy_version 1393217 (0.0007) [2023-12-27 15:18:05,248][20381] Updated weights for policy 1, policy_version 1392087 (0.0007) [2023-12-27 15:18:05,296][20381] Updated weights for policy 1, policy_version 1392097 (0.0008) [2023-12-27 15:18:05,351][20381] Updated weights for policy 1, policy_version 1392107 (0.0007) [2023-12-27 15:18:05,717][20347] Updated weights for policy 0, policy_version 1393227 (0.0008) [2023-12-27 15:18:05,763][20347] Updated weights for policy 0, policy_version 1393237 (0.0007) [2023-12-27 15:18:05,808][20347] Updated weights for policy 0, policy_version 1393247 (0.0010) [2023-12-27 15:18:05,977][20381] Updated weights for policy 1, policy_version 1392117 (0.0008) [2023-12-27 15:18:06,036][20381] Updated weights for policy 1, policy_version 1392127 (0.0008) [2023-12-27 15:18:06,088][20381] Updated weights for policy 1, policy_version 1392137 (0.0008) [2023-12-27 15:18:06,550][20347] Updated weights for policy 0, policy_version 1393257 (0.0010) [2023-12-27 15:18:06,611][20347] Updated weights for policy 0, policy_version 1393267 (0.0007) [2023-12-27 15:18:06,678][20347] Updated weights for policy 0, policy_version 1393277 (0.0008) [2023-12-27 15:18:06,703][20381] Updated weights for policy 1, policy_version 1392147 (0.0008) [2023-12-27 15:18:06,739][20347] Updated weights for policy 0, policy_version 1393287 (0.0008) [2023-12-27 15:18:06,767][20381] Updated weights for policy 1, policy_version 1392157 (0.0006) [2023-12-27 15:18:06,819][20381] Updated weights for policy 1, policy_version 1392167 (0.0007) [2023-12-27 15:18:07,348][20381] Updated weights for policy 1, policy_version 1392177 (0.0006) [2023-12-27 15:18:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 713179136. Throughput: 0: 10383.4, 1: 10382.1. Samples: 713150896. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:07,363][19170] Avg episode reward: [(0, '931.500'), (1, '903.450')] [2023-12-27 15:18:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001393288_356753408.pth... [2023-12-27 15:18:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001392104_356450304.pth [2023-12-27 15:18:07,404][20381] Updated weights for policy 1, policy_version 1392187 (0.0007) [2023-12-27 15:18:07,440][20347] Updated weights for policy 0, policy_version 1393297 (0.0008) [2023-12-27 15:18:07,457][20381] Updated weights for policy 1, policy_version 1392197 (0.0007) [2023-12-27 15:18:07,495][20347] Updated weights for policy 0, policy_version 1393307 (0.0008) [2023-12-27 15:18:07,516][20381] Updated weights for policy 1, policy_version 1392207 (0.0006) [2023-12-27 15:18:07,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001392208_356433920.pth... [2023-12-27 15:18:07,524][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001390992_356122624.pth [2023-12-27 15:18:07,554][20347] Updated weights for policy 0, policy_version 1393317 (0.0008) [2023-12-27 15:18:08,125][20381] Updated weights for policy 1, policy_version 1392217 (0.0009) [2023-12-27 15:18:08,170][20381] Updated weights for policy 1, policy_version 1392227 (0.0008) [2023-12-27 15:18:08,190][20347] Updated weights for policy 0, policy_version 1393327 (0.0006) [2023-12-27 15:18:08,224][20381] Updated weights for policy 1, policy_version 1392237 (0.0007) [2023-12-27 15:18:08,243][20347] Updated weights for policy 0, policy_version 1393337 (0.0005) [2023-12-27 15:18:08,295][20347] Updated weights for policy 0, policy_version 1393347 (0.0005) [2023-12-27 15:18:08,862][20347] Updated weights for policy 0, policy_version 1393357 (0.0008) [2023-12-27 15:18:08,922][20347] Updated weights for policy 0, policy_version 1393367 (0.0006) [2023-12-27 15:18:08,979][20347] Updated weights for policy 0, policy_version 1393377 (0.0010) [2023-12-27 15:18:09,004][20381] Updated weights for policy 1, policy_version 1392247 (0.0006) [2023-12-27 15:18:09,059][20381] Updated weights for policy 1, policy_version 1392257 (0.0007) [2023-12-27 15:18:09,115][20381] Updated weights for policy 1, policy_version 1392267 (0.0008) [2023-12-27 15:18:09,611][20347] Updated weights for policy 0, policy_version 1393387 (0.0010) [2023-12-27 15:18:09,672][20347] Updated weights for policy 0, policy_version 1393397 (0.0010) [2023-12-27 15:18:09,728][20347] Updated weights for policy 0, policy_version 1393407 (0.0010) [2023-12-27 15:18:09,862][20381] Updated weights for policy 1, policy_version 1392277 (0.0008) [2023-12-27 15:18:09,932][20381] Updated weights for policy 1, policy_version 1392287 (0.0008) [2023-12-27 15:18:09,992][20381] Updated weights for policy 1, policy_version 1392297 (0.0008) [2023-12-27 15:18:10,426][20347] Updated weights for policy 0, policy_version 1393417 (0.0011) [2023-12-27 15:18:10,486][20347] Updated weights for policy 0, policy_version 1393427 (0.0010) [2023-12-27 15:18:10,546][20347] Updated weights for policy 0, policy_version 1393437 (0.0010) [2023-12-27 15:18:10,598][20347] Updated weights for policy 0, policy_version 1393447 (0.0010) [2023-12-27 15:18:10,614][20381] Updated weights for policy 1, policy_version 1392307 (0.0007) [2023-12-27 15:18:10,663][20381] Updated weights for policy 1, policy_version 1392317 (0.0007) [2023-12-27 15:18:10,716][20381] Updated weights for policy 1, policy_version 1392327 (0.0007) [2023-12-27 15:18:11,316][20347] Updated weights for policy 0, policy_version 1393457 (0.0011) [2023-12-27 15:18:11,381][20347] Updated weights for policy 0, policy_version 1393467 (0.0010) [2023-12-27 15:18:11,428][20347] Updated weights for policy 0, policy_version 1393477 (0.0007) [2023-12-27 15:18:11,483][20381] Updated weights for policy 1, policy_version 1392337 (0.0006) [2023-12-27 15:18:11,545][20381] Updated weights for policy 1, policy_version 1392347 (0.0008) [2023-12-27 15:18:11,601][20381] Updated weights for policy 1, policy_version 1392357 (0.0008) [2023-12-27 15:18:11,668][20381] Updated weights for policy 1, policy_version 1392367 (0.0008) [2023-12-27 15:18:12,118][20347] Updated weights for policy 0, policy_version 1393487 (0.0010) [2023-12-27 15:18:12,179][20347] Updated weights for policy 0, policy_version 1393497 (0.0010) [2023-12-27 15:18:12,239][20347] Updated weights for policy 0, policy_version 1393507 (0.0011) [2023-12-27 15:18:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 713285632. Throughput: 0: 10345.2, 1: 10390.4. Samples: 713276200. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:12,363][19170] Avg episode reward: [(0, '921.830'), (1, '922.840')] [2023-12-27 15:18:12,418][20381] Updated weights for policy 1, policy_version 1392377 (0.0006) [2023-12-27 15:18:12,467][20381] Updated weights for policy 1, policy_version 1392387 (0.0006) [2023-12-27 15:18:12,514][20381] Updated weights for policy 1, policy_version 1392397 (0.0007) [2023-12-27 15:18:12,940][20347] Updated weights for policy 0, policy_version 1393517 (0.0010) [2023-12-27 15:18:13,001][20347] Updated weights for policy 0, policy_version 1393527 (0.0010) [2023-12-27 15:18:13,056][20347] Updated weights for policy 0, policy_version 1393537 (0.0010) [2023-12-27 15:18:13,193][20381] Updated weights for policy 1, policy_version 1392407 (0.0006) [2023-12-27 15:18:13,255][20381] Updated weights for policy 1, policy_version 1392417 (0.0006) [2023-12-27 15:18:13,317][20381] Updated weights for policy 1, policy_version 1392427 (0.0006) [2023-12-27 15:18:13,629][20347] Updated weights for policy 0, policy_version 1393547 (0.0009) [2023-12-27 15:18:13,682][20347] Updated weights for policy 0, policy_version 1393557 (0.0005) [2023-12-27 15:18:13,728][20347] Updated weights for policy 0, policy_version 1393567 (0.0005) [2023-12-27 15:18:13,903][20381] Updated weights for policy 1, policy_version 1392437 (0.0008) [2023-12-27 15:18:13,960][20381] Updated weights for policy 1, policy_version 1392447 (0.0010) [2023-12-27 15:18:13,988][20292] KL-divergence is very high: 243.2562 [2023-12-27 15:18:13,995][20292] KL-divergence is very high: 246.2981 [2023-12-27 15:18:14,021][20381] Updated weights for policy 1, policy_version 1392457 (0.0010) [2023-12-27 15:18:14,039][20292] KL-divergence is very high: 131.5470 [2023-12-27 15:18:14,044][20292] KL-divergence is very high: 109.3714 [2023-12-27 15:18:14,313][20347] Updated weights for policy 0, policy_version 1393577 (0.0005) [2023-12-27 15:18:14,373][20347] Updated weights for policy 0, policy_version 1393587 (0.0005) [2023-12-27 15:18:14,423][20291] KL-divergence is very high: 104.4169 [2023-12-27 15:18:14,429][20347] Updated weights for policy 0, policy_version 1393597 (0.0007) [2023-12-27 15:18:14,493][20347] Updated weights for policy 0, policy_version 1393607 (0.0010) [2023-12-27 15:18:14,655][20381] Updated weights for policy 1, policy_version 1392467 (0.0010) [2023-12-27 15:18:14,710][20381] Updated weights for policy 1, policy_version 1392477 (0.0008) [2023-12-27 15:18:14,788][20381] Updated weights for policy 1, policy_version 1392487 (0.0007) [2023-12-27 15:18:15,057][20347] Updated weights for policy 0, policy_version 1393617 (0.0011) [2023-12-27 15:18:15,108][20347] Updated weights for policy 0, policy_version 1393627 (0.0011) [2023-12-27 15:18:15,174][20347] Updated weights for policy 0, policy_version 1393637 (0.0010) [2023-12-27 15:18:15,500][20381] Updated weights for policy 1, policy_version 1392497 (0.0007) [2023-12-27 15:18:15,553][20381] Updated weights for policy 1, policy_version 1392507 (0.0007) [2023-12-27 15:18:15,615][20381] Updated weights for policy 1, policy_version 1392517 (0.0009) [2023-12-27 15:18:15,674][20381] Updated weights for policy 1, policy_version 1392527 (0.0010) [2023-12-27 15:18:15,799][20347] Updated weights for policy 0, policy_version 1393647 (0.0010) [2023-12-27 15:18:15,857][20347] Updated weights for policy 0, policy_version 1393657 (0.0010) [2023-12-27 15:18:15,920][20347] Updated weights for policy 0, policy_version 1393667 (0.0010) [2023-12-27 15:18:16,332][20381] Updated weights for policy 1, policy_version 1392537 (0.0008) [2023-12-27 15:18:16,387][20381] Updated weights for policy 1, policy_version 1392547 (0.0010) [2023-12-27 15:18:16,443][20381] Updated weights for policy 1, policy_version 1392557 (0.0010) [2023-12-27 15:18:16,539][20347] Updated weights for policy 0, policy_version 1393677 (0.0007) [2023-12-27 15:18:16,595][20347] Updated weights for policy 0, policy_version 1393687 (0.0008) [2023-12-27 15:18:16,662][20347] Updated weights for policy 0, policy_version 1393697 (0.0006) [2023-12-27 15:18:17,173][20381] Updated weights for policy 1, policy_version 1392567 (0.0007) [2023-12-27 15:18:17,226][20381] Updated weights for policy 1, policy_version 1392577 (0.0008) [2023-12-27 15:18:17,265][20347] Updated weights for policy 0, policy_version 1393707 (0.0007) [2023-12-27 15:18:17,282][20381] Updated weights for policy 1, policy_version 1392587 (0.0006) [2023-12-27 15:18:17,324][20347] Updated weights for policy 0, policy_version 1393717 (0.0006) [2023-12-27 15:18:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 713392128. Throughput: 0: 10385.3, 1: 10324.9. Samples: 713402200. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:17,363][19170] Avg episode reward: [(0, '921.930'), (1, '874.500')] [2023-12-27 15:18:17,380][20347] Updated weights for policy 0, policy_version 1393727 (0.0008) [2023-12-27 15:18:17,984][20381] Updated weights for policy 1, policy_version 1392597 (0.0006) [2023-12-27 15:18:18,029][20347] Updated weights for policy 0, policy_version 1393737 (0.0008) [2023-12-27 15:18:18,039][20381] Updated weights for policy 1, policy_version 1392607 (0.0007) [2023-12-27 15:18:18,082][20347] Updated weights for policy 0, policy_version 1393747 (0.0009) [2023-12-27 15:18:18,095][20381] Updated weights for policy 1, policy_version 1392617 (0.0006) [2023-12-27 15:18:18,139][20347] Updated weights for policy 0, policy_version 1393757 (0.0009) [2023-12-27 15:18:18,195][20347] Updated weights for policy 0, policy_version 1393767 (0.0008) [2023-12-27 15:18:18,716][20381] Updated weights for policy 1, policy_version 1392627 (0.0006) [2023-12-27 15:18:18,774][20381] Updated weights for policy 1, policy_version 1392637 (0.0007) [2023-12-27 15:18:18,829][20381] Updated weights for policy 1, policy_version 1392647 (0.0008) [2023-12-27 15:18:18,910][20347] Updated weights for policy 0, policy_version 1393777 (0.0010) [2023-12-27 15:18:18,971][20347] Updated weights for policy 0, policy_version 1393787 (0.0010) [2023-12-27 15:18:19,022][20347] Updated weights for policy 0, policy_version 1393797 (0.0010) [2023-12-27 15:18:19,536][20381] Updated weights for policy 1, policy_version 1392657 (0.0007) [2023-12-27 15:18:19,594][20381] Updated weights for policy 1, policy_version 1392667 (0.0008) [2023-12-27 15:18:19,656][20381] Updated weights for policy 1, policy_version 1392677 (0.0008) [2023-12-27 15:18:19,685][20347] Updated weights for policy 0, policy_version 1393807 (0.0008) [2023-12-27 15:18:19,706][20381] Updated weights for policy 1, policy_version 1392687 (0.0006) [2023-12-27 15:18:19,745][20347] Updated weights for policy 0, policy_version 1393817 (0.0005) [2023-12-27 15:18:19,808][20347] Updated weights for policy 0, policy_version 1393827 (0.0007) [2023-12-27 15:18:20,434][20381] Updated weights for policy 1, policy_version 1392697 (0.0007) [2023-12-27 15:18:20,487][20347] Updated weights for policy 0, policy_version 1393837 (0.0006) [2023-12-27 15:18:20,499][20381] Updated weights for policy 1, policy_version 1392707 (0.0006) [2023-12-27 15:18:20,537][20347] Updated weights for policy 0, policy_version 1393847 (0.0005) [2023-12-27 15:18:20,557][20381] Updated weights for policy 1, policy_version 1392717 (0.0006) [2023-12-27 15:18:20,596][20347] Updated weights for policy 0, policy_version 1393857 (0.0007) [2023-12-27 15:18:21,230][20381] Updated weights for policy 1, policy_version 1392727 (0.0006) [2023-12-27 15:18:21,293][20381] Updated weights for policy 1, policy_version 1392737 (0.0007) [2023-12-27 15:18:21,344][20347] Updated weights for policy 0, policy_version 1393867 (0.0007) [2023-12-27 15:18:21,356][20381] Updated weights for policy 1, policy_version 1392747 (0.0008) [2023-12-27 15:18:21,412][20347] Updated weights for policy 0, policy_version 1393877 (0.0008) [2023-12-27 15:18:21,495][20347] Updated weights for policy 0, policy_version 1393887 (0.0008) [2023-12-27 15:18:22,090][20381] Updated weights for policy 1, policy_version 1392757 (0.0008) [2023-12-27 15:18:22,144][20381] Updated weights for policy 1, policy_version 1392767 (0.0008) [2023-12-27 15:18:22,194][20381] Updated weights for policy 1, policy_version 1392777 (0.0007) [2023-12-27 15:18:22,198][20347] Updated weights for policy 0, policy_version 1393897 (0.0008) [2023-12-27 15:18:22,250][20347] Updated weights for policy 0, policy_version 1393907 (0.0008) [2023-12-27 15:18:22,308][20347] Updated weights for policy 0, policy_version 1393917 (0.0007) [2023-12-27 15:18:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 713490432. Throughput: 0: 10396.2, 1: 10304.4. Samples: 713463040. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:22,363][19170] Avg episode reward: [(0, '931.680'), (1, '903.620')] [2023-12-27 15:18:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001392784_356581376.pth... [2023-12-27 15:18:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001391568_356270080.pth [2023-12-27 15:18:22,375][20347] Updated weights for policy 0, policy_version 1393927 (0.0008) [2023-12-27 15:18:22,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001393928_356917248.pth... [2023-12-27 15:18:22,383][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001392712_356605952.pth [2023-12-27 15:18:22,970][20381] Updated weights for policy 1, policy_version 1392787 (0.0007) [2023-12-27 15:18:23,024][20381] Updated weights for policy 1, policy_version 1392797 (0.0007) [2023-12-27 15:18:23,049][20347] Updated weights for policy 0, policy_version 1393937 (0.0007) [2023-12-27 15:18:23,081][20381] Updated weights for policy 1, policy_version 1392807 (0.0007) [2023-12-27 15:18:23,105][20347] Updated weights for policy 0, policy_version 1393947 (0.0005) [2023-12-27 15:18:23,157][20347] Updated weights for policy 0, policy_version 1393957 (0.0005) [2023-12-27 15:18:23,739][20347] Updated weights for policy 0, policy_version 1393967 (0.0008) [2023-12-27 15:18:23,784][20347] Updated weights for policy 0, policy_version 1393977 (0.0008) [2023-12-27 15:18:23,811][20381] Updated weights for policy 1, policy_version 1392817 (0.0008) [2023-12-27 15:18:23,831][20347] Updated weights for policy 0, policy_version 1393987 (0.0006) [2023-12-27 15:18:23,860][20381] Updated weights for policy 1, policy_version 1392827 (0.0005) [2023-12-27 15:18:23,916][20381] Updated weights for policy 1, policy_version 1392837 (0.0008) [2023-12-27 15:18:23,973][20381] Updated weights for policy 1, policy_version 1392847 (0.0008) [2023-12-27 15:18:24,559][20347] Updated weights for policy 0, policy_version 1393997 (0.0006) [2023-12-27 15:18:24,607][20347] Updated weights for policy 0, policy_version 1394007 (0.0006) [2023-12-27 15:18:24,638][20381] Updated weights for policy 1, policy_version 1392857 (0.0010) [2023-12-27 15:18:24,655][20347] Updated weights for policy 0, policy_version 1394017 (0.0007) [2023-12-27 15:18:24,687][20381] Updated weights for policy 1, policy_version 1392867 (0.0010) [2023-12-27 15:18:24,745][20381] Updated weights for policy 1, policy_version 1392877 (0.0010) [2023-12-27 15:18:25,320][20347] Updated weights for policy 0, policy_version 1394027 (0.0006) [2023-12-27 15:18:25,375][20347] Updated weights for policy 0, policy_version 1394037 (0.0007) [2023-12-27 15:18:25,407][20381] Updated weights for policy 1, policy_version 1392887 (0.0010) [2023-12-27 15:18:25,430][20347] Updated weights for policy 0, policy_version 1394047 (0.0007) [2023-12-27 15:18:25,458][20381] Updated weights for policy 1, policy_version 1392897 (0.0006) [2023-12-27 15:18:25,510][20381] Updated weights for policy 1, policy_version 1392907 (0.0008) [2023-12-27 15:18:26,117][20347] Updated weights for policy 0, policy_version 1394057 (0.0006) [2023-12-27 15:18:26,170][20347] Updated weights for policy 0, policy_version 1394067 (0.0010) [2023-12-27 15:18:26,171][20381] Updated weights for policy 1, policy_version 1392917 (0.0009) [2023-12-27 15:18:26,224][20347] Updated weights for policy 0, policy_version 1394077 (0.0011) [2023-12-27 15:18:26,228][20381] Updated weights for policy 1, policy_version 1392927 (0.0007) [2023-12-27 15:18:26,284][20347] Updated weights for policy 0, policy_version 1394087 (0.0011) [2023-12-27 15:18:26,288][20381] Updated weights for policy 1, policy_version 1392937 (0.0006) [2023-12-27 15:18:26,840][20381] Updated weights for policy 1, policy_version 1392947 (0.0006) [2023-12-27 15:18:26,885][20381] Updated weights for policy 1, policy_version 1392957 (0.0006) [2023-12-27 15:18:26,937][20381] Updated weights for policy 1, policy_version 1392967 (0.0005) [2023-12-27 15:18:26,957][20347] Updated weights for policy 0, policy_version 1394097 (0.0007) [2023-12-27 15:18:27,013][20347] Updated weights for policy 0, policy_version 1394107 (0.0006) [2023-12-27 15:18:27,071][20347] Updated weights for policy 0, policy_version 1394117 (0.0007) [2023-12-27 15:18:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 713596928. Throughput: 0: 10340.6, 1: 10300.0. Samples: 713583548. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:27,363][19170] Avg episode reward: [(0, '941.480'), (1, '990.540')] [2023-12-27 15:18:27,539][20381] Updated weights for policy 1, policy_version 1392977 (0.0005) [2023-12-27 15:18:27,587][20381] Updated weights for policy 1, policy_version 1392987 (0.0007) [2023-12-27 15:18:27,633][20381] Updated weights for policy 1, policy_version 1392997 (0.0005) [2023-12-27 15:18:27,684][20381] Updated weights for policy 1, policy_version 1393007 (0.0006) [2023-12-27 15:18:27,686][20347] Updated weights for policy 0, policy_version 1394127 (0.0009) [2023-12-27 15:18:27,749][20347] Updated weights for policy 0, policy_version 1394137 (0.0007) [2023-12-27 15:18:27,810][20347] Updated weights for policy 0, policy_version 1394147 (0.0005) [2023-12-27 15:18:28,235][20381] Updated weights for policy 1, policy_version 1393017 (0.0007) [2023-12-27 15:18:28,283][20381] Updated weights for policy 1, policy_version 1393027 (0.0005) [2023-12-27 15:18:28,339][20381] Updated weights for policy 1, policy_version 1393037 (0.0005) [2023-12-27 15:18:28,419][20347] Updated weights for policy 0, policy_version 1394157 (0.0006) [2023-12-27 15:18:28,477][20347] Updated weights for policy 0, policy_version 1394167 (0.0009) [2023-12-27 15:18:28,530][20347] Updated weights for policy 0, policy_version 1394177 (0.0009) [2023-12-27 15:18:28,915][20381] Updated weights for policy 1, policy_version 1393047 (0.0006) [2023-12-27 15:18:28,975][20381] Updated weights for policy 1, policy_version 1393057 (0.0006) [2023-12-27 15:18:29,033][20381] Updated weights for policy 1, policy_version 1393067 (0.0007) [2023-12-27 15:18:29,219][20347] Updated weights for policy 0, policy_version 1394187 (0.0008) [2023-12-27 15:18:29,279][20347] Updated weights for policy 0, policy_version 1394197 (0.0007) [2023-12-27 15:18:29,337][20347] Updated weights for policy 0, policy_version 1394207 (0.0009) [2023-12-27 15:18:29,691][20381] Updated weights for policy 1, policy_version 1393077 (0.0008) [2023-12-27 15:18:29,747][20381] Updated weights for policy 1, policy_version 1393087 (0.0006) [2023-12-27 15:18:29,811][20381] Updated weights for policy 1, policy_version 1393097 (0.0007) [2023-12-27 15:18:29,991][20347] Updated weights for policy 0, policy_version 1394217 (0.0008) [2023-12-27 15:18:30,045][20347] Updated weights for policy 0, policy_version 1394227 (0.0008) [2023-12-27 15:18:30,104][20347] Updated weights for policy 0, policy_version 1394237 (0.0007) [2023-12-27 15:18:30,161][20347] Updated weights for policy 0, policy_version 1394247 (0.0007) [2023-12-27 15:18:30,407][20381] Updated weights for policy 1, policy_version 1393107 (0.0007) [2023-12-27 15:18:30,462][20381] Updated weights for policy 1, policy_version 1393117 (0.0007) [2023-12-27 15:18:30,512][20381] Updated weights for policy 1, policy_version 1393127 (0.0005) [2023-12-27 15:18:30,871][20347] Updated weights for policy 0, policy_version 1394257 (0.0010) [2023-12-27 15:18:30,919][20347] Updated weights for policy 0, policy_version 1394267 (0.0010) [2023-12-27 15:18:30,965][20347] Updated weights for policy 0, policy_version 1394277 (0.0010) [2023-12-27 15:18:31,123][20381] Updated weights for policy 1, policy_version 1393137 (0.0006) [2023-12-27 15:18:31,185][20381] Updated weights for policy 1, policy_version 1393147 (0.0011) [2023-12-27 15:18:31,235][20381] Updated weights for policy 1, policy_version 1393157 (0.0011) [2023-12-27 15:18:31,294][20381] Updated weights for policy 1, policy_version 1393167 (0.0011) [2023-12-27 15:18:31,681][20347] Updated weights for policy 0, policy_version 1394287 (0.0007) [2023-12-27 15:18:31,748][20347] Updated weights for policy 0, policy_version 1394297 (0.0007) [2023-12-27 15:18:31,813][20347] Updated weights for policy 0, policy_version 1394307 (0.0007) [2023-12-27 15:18:32,033][20381] Updated weights for policy 1, policy_version 1393177 (0.0006) [2023-12-27 15:18:32,084][20381] Updated weights for policy 1, policy_version 1393187 (0.0006) [2023-12-27 15:18:32,130][20381] Updated weights for policy 1, policy_version 1393197 (0.0008) [2023-12-27 15:18:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 713703424. Throughput: 0: 10328.6, 1: 10350.5. Samples: 713712832. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:32,363][19170] Avg episode reward: [(0, '970.720'), (1, '971.040')] [2023-12-27 15:18:32,509][20347] Updated weights for policy 0, policy_version 1394317 (0.0007) [2023-12-27 15:18:32,574][20347] Updated weights for policy 0, policy_version 1394327 (0.0005) [2023-12-27 15:18:32,627][20347] Updated weights for policy 0, policy_version 1394337 (0.0006) [2023-12-27 15:18:32,756][20381] Updated weights for policy 1, policy_version 1393207 (0.0007) [2023-12-27 15:18:32,809][20381] Updated weights for policy 1, policy_version 1393217 (0.0008) [2023-12-27 15:18:32,858][20381] Updated weights for policy 1, policy_version 1393227 (0.0007) [2023-12-27 15:18:33,143][20347] Updated weights for policy 0, policy_version 1394347 (0.0005) [2023-12-27 15:18:33,195][20347] Updated weights for policy 0, policy_version 1394357 (0.0005) [2023-12-27 15:18:33,258][20347] Updated weights for policy 0, policy_version 1394367 (0.0005) [2023-12-27 15:18:33,526][20381] Updated weights for policy 1, policy_version 1393237 (0.0009) [2023-12-27 15:18:33,576][20381] Updated weights for policy 1, policy_version 1393247 (0.0010) [2023-12-27 15:18:33,627][20381] Updated weights for policy 1, policy_version 1393257 (0.0009) [2023-12-27 15:18:33,861][20347] Updated weights for policy 0, policy_version 1394377 (0.0007) [2023-12-27 15:18:33,921][20347] Updated weights for policy 0, policy_version 1394387 (0.0007) [2023-12-27 15:18:33,986][20347] Updated weights for policy 0, policy_version 1394397 (0.0006) [2023-12-27 15:18:34,036][20347] Updated weights for policy 0, policy_version 1394407 (0.0009) [2023-12-27 15:18:34,209][20381] Updated weights for policy 1, policy_version 1393267 (0.0007) [2023-12-27 15:18:34,263][20381] Updated weights for policy 1, policy_version 1393277 (0.0010) [2023-12-27 15:18:34,321][20381] Updated weights for policy 1, policy_version 1393287 (0.0010) [2023-12-27 15:18:34,757][20347] Updated weights for policy 0, policy_version 1394417 (0.0007) [2023-12-27 15:18:34,818][20347] Updated weights for policy 0, policy_version 1394427 (0.0008) [2023-12-27 15:18:34,873][20347] Updated weights for policy 0, policy_version 1394437 (0.0008) [2023-12-27 15:18:34,994][20381] Updated weights for policy 1, policy_version 1393297 (0.0009) [2023-12-27 15:18:35,047][20381] Updated weights for policy 1, policy_version 1393307 (0.0010) [2023-12-27 15:18:35,092][20381] Updated weights for policy 1, policy_version 1393317 (0.0010) [2023-12-27 15:18:35,138][20381] Updated weights for policy 1, policy_version 1393327 (0.0010) [2023-12-27 15:18:35,565][20347] Updated weights for policy 0, policy_version 1394447 (0.0006) [2023-12-27 15:18:35,611][20347] Updated weights for policy 0, policy_version 1394457 (0.0005) [2023-12-27 15:18:35,661][20347] Updated weights for policy 0, policy_version 1394467 (0.0007) [2023-12-27 15:18:35,846][20381] Updated weights for policy 1, policy_version 1393337 (0.0010) [2023-12-27 15:18:35,902][20381] Updated weights for policy 1, policy_version 1393347 (0.0009) [2023-12-27 15:18:35,947][20381] Updated weights for policy 1, policy_version 1393357 (0.0010) [2023-12-27 15:18:36,326][20347] Updated weights for policy 0, policy_version 1394477 (0.0007) [2023-12-27 15:18:36,389][20347] Updated weights for policy 0, policy_version 1394487 (0.0007) [2023-12-27 15:18:36,456][20347] Updated weights for policy 0, policy_version 1394497 (0.0008) [2023-12-27 15:18:36,609][20381] Updated weights for policy 1, policy_version 1393367 (0.0010) [2023-12-27 15:18:36,667][20381] Updated weights for policy 1, policy_version 1393377 (0.0011) [2023-12-27 15:18:36,718][20381] Updated weights for policy 1, policy_version 1393387 (0.0009) [2023-12-27 15:18:37,190][20347] Updated weights for policy 0, policy_version 1394507 (0.0007) [2023-12-27 15:18:37,248][20347] Updated weights for policy 0, policy_version 1394517 (0.0008) [2023-12-27 15:18:37,298][20381] Updated weights for policy 1, policy_version 1393397 (0.0005) [2023-12-27 15:18:37,299][20347] Updated weights for policy 0, policy_version 1394527 (0.0007) [2023-12-27 15:18:37,355][20381] Updated weights for policy 1, policy_version 1393407 (0.0006) [2023-12-27 15:18:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.2, 300 sec: 20577.2). Total num frames: 713809920. Throughput: 0: 10359.7, 1: 10388.3. Samples: 713777144. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:37,363][19170] Avg episode reward: [(0, '970.720'), (1, '941.750')] [2023-12-27 15:18:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001394536_357072896.pth... [2023-12-27 15:18:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001393288_356753408.pth [2023-12-27 15:18:37,408][20381] Updated weights for policy 1, policy_version 1393417 (0.0008) [2023-12-27 15:18:37,447][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001393424_356745216.pth... [2023-12-27 15:18:37,451][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001392208_356433920.pth [2023-12-27 15:18:38,009][20347] Updated weights for policy 0, policy_version 1394537 (0.0007) [2023-12-27 15:18:38,039][20381] Updated weights for policy 1, policy_version 1393427 (0.0007) [2023-12-27 15:18:38,069][20347] Updated weights for policy 0, policy_version 1394547 (0.0005) [2023-12-27 15:18:38,097][20381] Updated weights for policy 1, policy_version 1393437 (0.0006) [2023-12-27 15:18:38,131][20347] Updated weights for policy 0, policy_version 1394557 (0.0007) [2023-12-27 15:18:38,156][20381] Updated weights for policy 1, policy_version 1393447 (0.0005) [2023-12-27 15:18:38,189][20347] Updated weights for policy 0, policy_version 1394567 (0.0008) [2023-12-27 15:18:38,816][20347] Updated weights for policy 0, policy_version 1394577 (0.0008) [2023-12-27 15:18:38,834][20381] Updated weights for policy 1, policy_version 1393457 (0.0006) [2023-12-27 15:18:38,877][20347] Updated weights for policy 0, policy_version 1394587 (0.0006) [2023-12-27 15:18:38,886][20381] Updated weights for policy 1, policy_version 1393467 (0.0006) [2023-12-27 15:18:38,936][20347] Updated weights for policy 0, policy_version 1394597 (0.0007) [2023-12-27 15:18:38,949][20381] Updated weights for policy 1, policy_version 1393477 (0.0006) [2023-12-27 15:18:39,004][20381] Updated weights for policy 1, policy_version 1393487 (0.0007) [2023-12-27 15:18:39,695][20347] Updated weights for policy 0, policy_version 1394607 (0.0007) [2023-12-27 15:18:39,735][20381] Updated weights for policy 1, policy_version 1393497 (0.0009) [2023-12-27 15:18:39,755][20347] Updated weights for policy 0, policy_version 1394617 (0.0008) [2023-12-27 15:18:39,785][20381] Updated weights for policy 1, policy_version 1393507 (0.0010) [2023-12-27 15:18:39,818][20347] Updated weights for policy 0, policy_version 1394627 (0.0006) [2023-12-27 15:18:39,849][20381] Updated weights for policy 1, policy_version 1393517 (0.0011) [2023-12-27 15:18:40,494][20347] Updated weights for policy 0, policy_version 1394637 (0.0008) [2023-12-27 15:18:40,549][20347] Updated weights for policy 0, policy_version 1394647 (0.0008) [2023-12-27 15:18:40,604][20381] Updated weights for policy 1, policy_version 1393527 (0.0007) [2023-12-27 15:18:40,605][20347] Updated weights for policy 0, policy_version 1394657 (0.0006) [2023-12-27 15:18:40,659][20381] Updated weights for policy 1, policy_version 1393537 (0.0006) [2023-12-27 15:18:40,723][20381] Updated weights for policy 1, policy_version 1393547 (0.0008) [2023-12-27 15:18:41,252][20347] Updated weights for policy 0, policy_version 1394667 (0.0008) [2023-12-27 15:18:41,316][20347] Updated weights for policy 0, policy_version 1394677 (0.0008) [2023-12-27 15:18:41,382][20347] Updated weights for policy 0, policy_version 1394687 (0.0008) [2023-12-27 15:18:41,439][20381] Updated weights for policy 1, policy_version 1393557 (0.0007) [2023-12-27 15:18:41,503][20381] Updated weights for policy 1, policy_version 1393567 (0.0008) [2023-12-27 15:18:41,562][20381] Updated weights for policy 1, policy_version 1393577 (0.0008) [2023-12-27 15:18:42,105][20347] Updated weights for policy 0, policy_version 1394697 (0.0007) [2023-12-27 15:18:42,166][20347] Updated weights for policy 0, policy_version 1394707 (0.0008) [2023-12-27 15:18:42,226][20347] Updated weights for policy 0, policy_version 1394717 (0.0009) [2023-12-27 15:18:42,279][20381] Updated weights for policy 1, policy_version 1393587 (0.0008) [2023-12-27 15:18:42,294][20347] Updated weights for policy 0, policy_version 1394727 (0.0006) [2023-12-27 15:18:42,339][20381] Updated weights for policy 1, policy_version 1393597 (0.0007) [2023-12-27 15:18:42,364][19170] Fps is (10 sec: 20477.6, 60 sec: 20752.7, 300 sec: 20577.1). Total num frames: 713908224. Throughput: 0: 10332.6, 1: 10379.3. Samples: 713899728. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:42,364][19170] Avg episode reward: [(0, '960.950'), (1, '942.010')] [2023-12-27 15:18:42,400][20381] Updated weights for policy 1, policy_version 1393607 (0.0008) [2023-12-27 15:18:43,007][20381] Updated weights for policy 1, policy_version 1393617 (0.0006) [2023-12-27 15:18:43,042][20347] Updated weights for policy 0, policy_version 1394737 (0.0006) [2023-12-27 15:18:43,062][20381] Updated weights for policy 1, policy_version 1393627 (0.0007) [2023-12-27 15:18:43,101][20347] Updated weights for policy 0, policy_version 1394747 (0.0006) [2023-12-27 15:18:43,121][20381] Updated weights for policy 1, policy_version 1393637 (0.0006) [2023-12-27 15:18:43,156][20347] Updated weights for policy 0, policy_version 1394757 (0.0005) [2023-12-27 15:18:43,184][20381] Updated weights for policy 1, policy_version 1393647 (0.0007) [2023-12-27 15:18:43,720][20347] Updated weights for policy 0, policy_version 1394767 (0.0006) [2023-12-27 15:18:43,783][20347] Updated weights for policy 0, policy_version 1394777 (0.0005) [2023-12-27 15:18:43,844][20347] Updated weights for policy 0, policy_version 1394787 (0.0005) [2023-12-27 15:18:43,868][20381] Updated weights for policy 1, policy_version 1393657 (0.0006) [2023-12-27 15:18:43,922][20381] Updated weights for policy 1, policy_version 1393667 (0.0008) [2023-12-27 15:18:43,978][20381] Updated weights for policy 1, policy_version 1393677 (0.0008) [2023-12-27 15:18:44,458][20347] Updated weights for policy 0, policy_version 1394797 (0.0007) [2023-12-27 15:18:44,516][20347] Updated weights for policy 0, policy_version 1394807 (0.0008) [2023-12-27 15:18:44,573][20347] Updated weights for policy 0, policy_version 1394817 (0.0008) [2023-12-27 15:18:44,632][20381] Updated weights for policy 1, policy_version 1393687 (0.0007) [2023-12-27 15:18:44,691][20381] Updated weights for policy 1, policy_version 1393697 (0.0008) [2023-12-27 15:18:44,745][20381] Updated weights for policy 1, policy_version 1393707 (0.0008) [2023-12-27 15:18:45,306][20347] Updated weights for policy 0, policy_version 1394827 (0.0007) [2023-12-27 15:18:45,354][20347] Updated weights for policy 0, policy_version 1394837 (0.0007) [2023-12-27 15:18:45,401][20347] Updated weights for policy 0, policy_version 1394847 (0.0008) [2023-12-27 15:18:45,457][20381] Updated weights for policy 1, policy_version 1393717 (0.0007) [2023-12-27 15:18:45,505][20381] Updated weights for policy 1, policy_version 1393727 (0.0005) [2023-12-27 15:18:45,555][20381] Updated weights for policy 1, policy_version 1393737 (0.0005) [2023-12-27 15:18:46,073][20347] Updated weights for policy 0, policy_version 1394857 (0.0007) [2023-12-27 15:18:46,123][20347] Updated weights for policy 0, policy_version 1394867 (0.0007) [2023-12-27 15:18:46,176][20347] Updated weights for policy 0, policy_version 1394877 (0.0007) [2023-12-27 15:18:46,213][20381] Updated weights for policy 1, policy_version 1393747 (0.0007) [2023-12-27 15:18:46,229][20347] Updated weights for policy 0, policy_version 1394887 (0.0008) [2023-12-27 15:18:46,263][20381] Updated weights for policy 1, policy_version 1393757 (0.0007) [2023-12-27 15:18:46,321][20381] Updated weights for policy 1, policy_version 1393767 (0.0009) [2023-12-27 15:18:46,952][20347] Updated weights for policy 0, policy_version 1394897 (0.0008) [2023-12-27 15:18:46,958][20381] Updated weights for policy 1, policy_version 1393777 (0.0010) [2023-12-27 15:18:47,011][20347] Updated weights for policy 0, policy_version 1394907 (0.0008) [2023-12-27 15:18:47,013][20381] Updated weights for policy 1, policy_version 1393787 (0.0008) [2023-12-27 15:18:47,067][20381] Updated weights for policy 1, policy_version 1393797 (0.0006) [2023-12-27 15:18:47,072][20347] Updated weights for policy 0, policy_version 1394917 (0.0006) [2023-12-27 15:18:47,123][20381] Updated weights for policy 1, policy_version 1393807 (0.0007) [2023-12-27 15:18:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 714014720. Throughput: 0: 10324.2, 1: 10381.6. Samples: 714021676. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:47,363][19170] Avg episode reward: [(0, '960.890'), (1, '951.170')] [2023-12-27 15:18:47,733][20381] Updated weights for policy 1, policy_version 1393817 (0.0007) [2023-12-27 15:18:47,781][20381] Updated weights for policy 1, policy_version 1393827 (0.0005) [2023-12-27 15:18:47,828][20347] Updated weights for policy 0, policy_version 1394927 (0.0006) [2023-12-27 15:18:47,837][20381] Updated weights for policy 1, policy_version 1393837 (0.0008) [2023-12-27 15:18:47,884][20347] Updated weights for policy 0, policy_version 1394937 (0.0005) [2023-12-27 15:18:47,929][20347] Updated weights for policy 0, policy_version 1394947 (0.0005) [2023-12-27 15:18:48,485][20381] Updated weights for policy 1, policy_version 1393847 (0.0010) [2023-12-27 15:18:48,548][20381] Updated weights for policy 1, policy_version 1393857 (0.0011) [2023-12-27 15:18:48,604][20347] Updated weights for policy 0, policy_version 1394957 (0.0008) [2023-12-27 15:18:48,608][20381] Updated weights for policy 1, policy_version 1393867 (0.0010) [2023-12-27 15:18:48,667][20347] Updated weights for policy 0, policy_version 1394967 (0.0010) [2023-12-27 15:18:48,726][20347] Updated weights for policy 0, policy_version 1394977 (0.0007) [2023-12-27 15:18:49,195][20381] Updated weights for policy 1, policy_version 1393877 (0.0007) [2023-12-27 15:18:49,255][20381] Updated weights for policy 1, policy_version 1393887 (0.0007) [2023-12-27 15:18:49,319][20381] Updated weights for policy 1, policy_version 1393897 (0.0006) [2023-12-27 15:18:49,495][20347] Updated weights for policy 0, policy_version 1394987 (0.0008) [2023-12-27 15:18:49,557][20347] Updated weights for policy 0, policy_version 1394997 (0.0007) [2023-12-27 15:18:49,616][20347] Updated weights for policy 0, policy_version 1395007 (0.0007) [2023-12-27 15:18:49,980][20381] Updated weights for policy 1, policy_version 1393907 (0.0008) [2023-12-27 15:18:50,042][20381] Updated weights for policy 1, policy_version 1393917 (0.0008) [2023-12-27 15:18:50,098][20381] Updated weights for policy 1, policy_version 1393927 (0.0008) [2023-12-27 15:18:50,296][20347] Updated weights for policy 0, policy_version 1395017 (0.0007) [2023-12-27 15:18:50,351][20347] Updated weights for policy 0, policy_version 1395027 (0.0006) [2023-12-27 15:18:50,410][20347] Updated weights for policy 0, policy_version 1395037 (0.0007) [2023-12-27 15:18:50,454][20347] Updated weights for policy 0, policy_version 1395047 (0.0008) [2023-12-27 15:18:50,808][20381] Updated weights for policy 1, policy_version 1393937 (0.0008) [2023-12-27 15:18:50,865][20381] Updated weights for policy 1, policy_version 1393947 (0.0007) [2023-12-27 15:18:50,925][20381] Updated weights for policy 1, policy_version 1393957 (0.0007) [2023-12-27 15:18:50,983][20381] Updated weights for policy 1, policy_version 1393967 (0.0006) [2023-12-27 15:18:51,130][20347] Updated weights for policy 0, policy_version 1395057 (0.0008) [2023-12-27 15:18:51,179][20347] Updated weights for policy 0, policy_version 1395067 (0.0008) [2023-12-27 15:18:51,235][20347] Updated weights for policy 0, policy_version 1395077 (0.0006) [2023-12-27 15:18:51,749][20381] Updated weights for policy 1, policy_version 1393977 (0.0009) [2023-12-27 15:18:51,805][20381] Updated weights for policy 1, policy_version 1393987 (0.0008) [2023-12-27 15:18:51,857][20381] Updated weights for policy 1, policy_version 1393997 (0.0008) [2023-12-27 15:18:51,930][20347] Updated weights for policy 0, policy_version 1395087 (0.0008) [2023-12-27 15:18:51,994][20347] Updated weights for policy 0, policy_version 1395097 (0.0007) [2023-12-27 15:18:52,057][20347] Updated weights for policy 0, policy_version 1395107 (0.0008) [2023-12-27 15:18:52,362][19170] Fps is (10 sec: 20482.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 714113024. Throughput: 0: 10305.8, 1: 10419.8. Samples: 714083548. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:52,363][19170] Avg episode reward: [(0, '941.470'), (1, '912.820')] [2023-12-27 15:18:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001394000_356892672.pth... [2023-12-27 15:18:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001395112_357220352.pth... [2023-12-27 15:18:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001392784_356581376.pth [2023-12-27 15:18:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001393928_356917248.pth [2023-12-27 15:18:52,512][20381] Updated weights for policy 1, policy_version 1394007 (0.0007) [2023-12-27 15:18:52,570][20381] Updated weights for policy 1, policy_version 1394017 (0.0007) [2023-12-27 15:18:52,627][20381] Updated weights for policy 1, policy_version 1394027 (0.0007) [2023-12-27 15:18:52,781][20347] Updated weights for policy 0, policy_version 1395117 (0.0007) [2023-12-27 15:18:52,834][20347] Updated weights for policy 0, policy_version 1395127 (0.0005) [2023-12-27 15:18:52,893][20347] Updated weights for policy 0, policy_version 1395137 (0.0005) [2023-12-27 15:18:53,305][20381] Updated weights for policy 1, policy_version 1394037 (0.0008) [2023-12-27 15:18:53,356][20381] Updated weights for policy 1, policy_version 1394047 (0.0007) [2023-12-27 15:18:53,405][20381] Updated weights for policy 1, policy_version 1394057 (0.0006) [2023-12-27 15:18:53,529][20347] Updated weights for policy 0, policy_version 1395147 (0.0007) [2023-12-27 15:18:53,581][20347] Updated weights for policy 0, policy_version 1395157 (0.0009) [2023-12-27 15:18:53,583][20291] KL-divergence is very high: 117.9743 [2023-12-27 15:18:53,638][20347] Updated weights for policy 0, policy_version 1395167 (0.0007) [2023-12-27 15:18:54,123][20381] Updated weights for policy 1, policy_version 1394067 (0.0008) [2023-12-27 15:18:54,190][20381] Updated weights for policy 1, policy_version 1394077 (0.0007) [2023-12-27 15:18:54,252][20347] Updated weights for policy 0, policy_version 1395177 (0.0007) [2023-12-27 15:18:54,254][20381] Updated weights for policy 1, policy_version 1394087 (0.0007) [2023-12-27 15:18:54,306][20347] Updated weights for policy 0, policy_version 1395187 (0.0005) [2023-12-27 15:18:54,358][20347] Updated weights for policy 0, policy_version 1395197 (0.0005) [2023-12-27 15:18:54,421][20347] Updated weights for policy 0, policy_version 1395207 (0.0005) [2023-12-27 15:18:54,899][20381] Updated weights for policy 1, policy_version 1394097 (0.0008) [2023-12-27 15:18:54,951][20381] Updated weights for policy 1, policy_version 1394107 (0.0005) [2023-12-27 15:18:55,006][20381] Updated weights for policy 1, policy_version 1394117 (0.0008) [2023-12-27 15:18:55,008][20347] Updated weights for policy 0, policy_version 1395217 (0.0006) [2023-12-27 15:18:55,064][20347] Updated weights for policy 0, policy_version 1395227 (0.0006) [2023-12-27 15:18:55,068][20381] Updated weights for policy 1, policy_version 1394127 (0.0011) [2023-12-27 15:18:55,118][20347] Updated weights for policy 0, policy_version 1395237 (0.0006) [2023-12-27 15:18:55,646][20381] Updated weights for policy 1, policy_version 1394137 (0.0006) [2023-12-27 15:18:55,697][20381] Updated weights for policy 1, policy_version 1394147 (0.0009) [2023-12-27 15:18:55,746][20381] Updated weights for policy 1, policy_version 1394157 (0.0010) [2023-12-27 15:18:55,774][20347] Updated weights for policy 0, policy_version 1395247 (0.0006) [2023-12-27 15:18:55,828][20347] Updated weights for policy 0, policy_version 1395257 (0.0007) [2023-12-27 15:18:55,891][20347] Updated weights for policy 0, policy_version 1395267 (0.0005) [2023-12-27 15:18:56,368][20381] Updated weights for policy 1, policy_version 1394167 (0.0010) [2023-12-27 15:18:56,429][20381] Updated weights for policy 1, policy_version 1394177 (0.0011) [2023-12-27 15:18:56,479][20347] Updated weights for policy 0, policy_version 1395277 (0.0005) [2023-12-27 15:18:56,487][20381] Updated weights for policy 1, policy_version 1394187 (0.0011) [2023-12-27 15:18:56,523][20347] Updated weights for policy 0, policy_version 1395287 (0.0005) [2023-12-27 15:18:56,566][20347] Updated weights for policy 0, policy_version 1395297 (0.0006) [2023-12-27 15:18:57,160][20347] Updated weights for policy 0, policy_version 1395307 (0.0005) [2023-12-27 15:18:57,165][20381] Updated weights for policy 1, policy_version 1394197 (0.0010) [2023-12-27 15:18:57,212][20347] Updated weights for policy 0, policy_version 1395317 (0.0005) [2023-12-27 15:18:57,214][20381] Updated weights for policy 1, policy_version 1394207 (0.0011) [2023-12-27 15:18:57,265][20347] Updated weights for policy 0, policy_version 1395327 (0.0006) [2023-12-27 15:18:57,270][20381] Updated weights for policy 1, policy_version 1394217 (0.0011) [2023-12-27 15:18:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 714227712. Throughput: 0: 10335.1, 1: 10390.9. Samples: 714208868. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:18:57,363][19170] Avg episode reward: [(0, '941.630'), (1, '922.310')] [2023-12-27 15:18:57,942][20347] Updated weights for policy 0, policy_version 1395337 (0.0006) [2023-12-27 15:18:57,967][20381] Updated weights for policy 1, policy_version 1394227 (0.0011) [2023-12-27 15:18:57,994][20347] Updated weights for policy 0, policy_version 1395347 (0.0005) [2023-12-27 15:18:58,024][20381] Updated weights for policy 1, policy_version 1394237 (0.0010) [2023-12-27 15:18:58,038][20347] Updated weights for policy 0, policy_version 1395357 (0.0005) [2023-12-27 15:18:58,076][20381] Updated weights for policy 1, policy_version 1394247 (0.0005) [2023-12-27 15:18:58,097][20347] Updated weights for policy 0, policy_version 1395367 (0.0005) [2023-12-27 15:18:58,810][20381] Updated weights for policy 1, policy_version 1394257 (0.0006) [2023-12-27 15:18:58,857][20347] Updated weights for policy 0, policy_version 1395377 (0.0008) [2023-12-27 15:18:58,878][20381] Updated weights for policy 1, policy_version 1394267 (0.0006) [2023-12-27 15:18:58,915][20347] Updated weights for policy 0, policy_version 1395387 (0.0007) [2023-12-27 15:18:58,937][20381] Updated weights for policy 1, policy_version 1394277 (0.0007) [2023-12-27 15:18:58,977][20347] Updated weights for policy 0, policy_version 1395397 (0.0007) [2023-12-27 15:18:59,000][20381] Updated weights for policy 1, policy_version 1394287 (0.0008) [2023-12-27 15:18:59,677][20347] Updated weights for policy 0, policy_version 1395407 (0.0008) [2023-12-27 15:18:59,703][20381] Updated weights for policy 1, policy_version 1394297 (0.0006) [2023-12-27 15:18:59,732][20347] Updated weights for policy 0, policy_version 1395417 (0.0007) [2023-12-27 15:18:59,767][20381] Updated weights for policy 1, policy_version 1394307 (0.0006) [2023-12-27 15:18:59,795][20347] Updated weights for policy 0, policy_version 1395427 (0.0005) [2023-12-27 15:18:59,825][20381] Updated weights for policy 1, policy_version 1394317 (0.0008) [2023-12-27 15:19:00,505][20381] Updated weights for policy 1, policy_version 1394327 (0.0008) [2023-12-27 15:19:00,522][20347] Updated weights for policy 0, policy_version 1395437 (0.0009) [2023-12-27 15:19:00,555][20381] Updated weights for policy 1, policy_version 1394337 (0.0007) [2023-12-27 15:19:00,576][20347] Updated weights for policy 0, policy_version 1395447 (0.0010) [2023-12-27 15:19:00,603][20381] Updated weights for policy 1, policy_version 1394347 (0.0005) [2023-12-27 15:19:00,634][20347] Updated weights for policy 0, policy_version 1395457 (0.0010) [2023-12-27 15:19:01,313][20347] Updated weights for policy 0, policy_version 1395467 (0.0010) [2023-12-27 15:19:01,326][20381] Updated weights for policy 1, policy_version 1394357 (0.0006) [2023-12-27 15:19:01,380][20347] Updated weights for policy 0, policy_version 1395477 (0.0009) [2023-12-27 15:19:01,390][20381] Updated weights for policy 1, policy_version 1394367 (0.0007) [2023-12-27 15:19:01,440][20347] Updated weights for policy 0, policy_version 1395487 (0.0010) [2023-12-27 15:19:01,441][20381] Updated weights for policy 1, policy_version 1394377 (0.0008) [2023-12-27 15:19:02,153][20347] Updated weights for policy 0, policy_version 1395497 (0.0010) [2023-12-27 15:19:02,162][20381] Updated weights for policy 1, policy_version 1394387 (0.0008) [2023-12-27 15:19:02,207][20347] Updated weights for policy 0, policy_version 1395507 (0.0006) [2023-12-27 15:19:02,212][20381] Updated weights for policy 1, policy_version 1394397 (0.0005) [2023-12-27 15:19:02,268][20347] Updated weights for policy 0, policy_version 1395517 (0.0008) [2023-12-27 15:19:02,274][20381] Updated weights for policy 1, policy_version 1394407 (0.0006) [2023-12-27 15:19:02,322][20347] Updated weights for policy 0, policy_version 1395527 (0.0006) [2023-12-27 15:19:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 714326016. Throughput: 0: 10244.9, 1: 10367.5. Samples: 714329760. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:02,363][19170] Avg episode reward: [(0, '970.820'), (1, '941.500')] [2023-12-27 15:19:02,914][20381] Updated weights for policy 1, policy_version 1394417 (0.0007) [2023-12-27 15:19:02,962][20381] Updated weights for policy 1, policy_version 1394427 (0.0008) [2023-12-27 15:19:03,014][20381] Updated weights for policy 1, policy_version 1394437 (0.0006) [2023-12-27 15:19:03,034][20347] Updated weights for policy 0, policy_version 1395537 (0.0006) [2023-12-27 15:19:03,062][20381] Updated weights for policy 1, policy_version 1394447 (0.0005) [2023-12-27 15:19:03,084][20347] Updated weights for policy 0, policy_version 1395547 (0.0006) [2023-12-27 15:19:03,089][20291] KL-divergence is very high: 207.0870 [2023-12-27 15:19:03,136][20291] KL-divergence is very high: 276.4467 [2023-12-27 15:19:03,140][20347] Updated weights for policy 0, policy_version 1395557 (0.0007) [2023-12-27 15:19:03,740][20381] Updated weights for policy 1, policy_version 1394457 (0.0007) [2023-12-27 15:19:03,787][20291] KL-divergence is very high: 231.9822 [2023-12-27 15:19:03,795][20381] Updated weights for policy 1, policy_version 1394467 (0.0007) [2023-12-27 15:19:03,819][20347] Updated weights for policy 0, policy_version 1395567 (0.0006) [2023-12-27 15:19:03,826][20291] KL-divergence is very high: 108.0761 [2023-12-27 15:19:03,852][20381] Updated weights for policy 1, policy_version 1394477 (0.0006) [2023-12-27 15:19:03,882][20347] Updated weights for policy 0, policy_version 1395577 (0.0008) [2023-12-27 15:19:03,940][20347] Updated weights for policy 0, policy_version 1395587 (0.0008) [2023-12-27 15:19:04,495][20381] Updated weights for policy 1, policy_version 1394487 (0.0006) [2023-12-27 15:19:04,556][20381] Updated weights for policy 1, policy_version 1394497 (0.0007) [2023-12-27 15:19:04,617][20381] Updated weights for policy 1, policy_version 1394507 (0.0006) [2023-12-27 15:19:04,678][20347] Updated weights for policy 0, policy_version 1395597 (0.0007) [2023-12-27 15:19:04,739][20347] Updated weights for policy 0, policy_version 1395607 (0.0008) [2023-12-27 15:19:04,799][20347] Updated weights for policy 0, policy_version 1395617 (0.0007) [2023-12-27 15:19:05,174][20381] Updated weights for policy 1, policy_version 1394517 (0.0006) [2023-12-27 15:19:05,236][20381] Updated weights for policy 1, policy_version 1394527 (0.0007) [2023-12-27 15:19:05,281][20381] Updated weights for policy 1, policy_version 1394537 (0.0006) [2023-12-27 15:19:05,485][20347] Updated weights for policy 0, policy_version 1395627 (0.0007) [2023-12-27 15:19:05,538][20347] Updated weights for policy 0, policy_version 1395637 (0.0007) [2023-12-27 15:19:05,599][20347] Updated weights for policy 0, policy_version 1395647 (0.0006) [2023-12-27 15:19:05,872][20381] Updated weights for policy 1, policy_version 1394547 (0.0005) [2023-12-27 15:19:05,922][20381] Updated weights for policy 1, policy_version 1394557 (0.0005) [2023-12-27 15:19:05,975][20381] Updated weights for policy 1, policy_version 1394567 (0.0005) [2023-12-27 15:19:06,319][20347] Updated weights for policy 0, policy_version 1395657 (0.0008) [2023-12-27 15:19:06,372][20291] KL-divergence is very high: 476.0469 [2023-12-27 15:19:06,385][20347] Updated weights for policy 0, policy_version 1395667 (0.0007) [2023-12-27 15:19:06,424][20291] KL-divergence is very high: 588.3749 [2023-12-27 15:19:06,451][20347] Updated weights for policy 0, policy_version 1395677 (0.0007) [2023-12-27 15:19:06,472][20291] KL-divergence is very high: 474.1884 [2023-12-27 15:19:06,509][20347] Updated weights for policy 0, policy_version 1395687 (0.0007) [2023-12-27 15:19:06,627][20381] Updated weights for policy 1, policy_version 1394577 (0.0006) [2023-12-27 15:19:06,682][20381] Updated weights for policy 1, policy_version 1394587 (0.0010) [2023-12-27 15:19:06,736][20381] Updated weights for policy 1, policy_version 1394597 (0.0010) [2023-12-27 15:19:06,793][20381] Updated weights for policy 1, policy_version 1394607 (0.0009) [2023-12-27 15:19:07,195][20347] Updated weights for policy 0, policy_version 1395697 (0.0008) [2023-12-27 15:19:07,243][20347] Updated weights for policy 0, policy_version 1395707 (0.0008) [2023-12-27 15:19:07,293][20347] Updated weights for policy 0, policy_version 1395717 (0.0005) [2023-12-27 15:19:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 714424320. Throughput: 0: 10204.8, 1: 10428.6. Samples: 714391544. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:07,363][19170] Avg episode reward: [(0, '941.470'), (1, '951.300')] [2023-12-27 15:19:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001394608_357048320.pth... [2023-12-27 15:19:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001395720_357376000.pth... [2023-12-27 15:19:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001394536_357072896.pth [2023-12-27 15:19:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001393424_356745216.pth [2023-12-27 15:19:07,450][20381] Updated weights for policy 1, policy_version 1394617 (0.0007) [2023-12-27 15:19:07,495][20381] Updated weights for policy 1, policy_version 1394627 (0.0007) [2023-12-27 15:19:07,543][20381] Updated weights for policy 1, policy_version 1394637 (0.0005) [2023-12-27 15:19:07,978][20347] Updated weights for policy 0, policy_version 1395727 (0.0007) [2023-12-27 15:19:08,027][20347] Updated weights for policy 0, policy_version 1395737 (0.0008) [2023-12-27 15:19:08,089][20347] Updated weights for policy 0, policy_version 1395747 (0.0007) [2023-12-27 15:19:08,235][20381] Updated weights for policy 1, policy_version 1394647 (0.0006) [2023-12-27 15:19:08,282][20381] Updated weights for policy 1, policy_version 1394657 (0.0006) [2023-12-27 15:19:08,340][20381] Updated weights for policy 1, policy_version 1394667 (0.0006) [2023-12-27 15:19:08,823][20347] Updated weights for policy 0, policy_version 1395757 (0.0007) [2023-12-27 15:19:08,883][20347] Updated weights for policy 0, policy_version 1395767 (0.0008) [2023-12-27 15:19:08,939][20347] Updated weights for policy 0, policy_version 1395777 (0.0008) [2023-12-27 15:19:08,990][20381] Updated weights for policy 1, policy_version 1394677 (0.0008) [2023-12-27 15:19:09,043][20381] Updated weights for policy 1, policy_version 1394687 (0.0007) [2023-12-27 15:19:09,096][20381] Updated weights for policy 1, policy_version 1394697 (0.0006) [2023-12-27 15:19:09,678][20347] Updated weights for policy 0, policy_version 1395787 (0.0007) [2023-12-27 15:19:09,736][20347] Updated weights for policy 0, policy_version 1395797 (0.0007) [2023-12-27 15:19:09,781][20381] Updated weights for policy 1, policy_version 1394707 (0.0006) [2023-12-27 15:19:09,796][20347] Updated weights for policy 0, policy_version 1395807 (0.0006) [2023-12-27 15:19:09,850][20381] Updated weights for policy 1, policy_version 1394717 (0.0008) [2023-12-27 15:19:09,907][20381] Updated weights for policy 1, policy_version 1394727 (0.0008) [2023-12-27 15:19:10,455][20347] Updated weights for policy 0, policy_version 1395817 (0.0007) [2023-12-27 15:19:10,518][20347] Updated weights for policy 0, policy_version 1395827 (0.0008) [2023-12-27 15:19:10,571][20347] Updated weights for policy 0, policy_version 1395837 (0.0008) [2023-12-27 15:19:10,620][20381] Updated weights for policy 1, policy_version 1394737 (0.0008) [2023-12-27 15:19:10,630][20347] Updated weights for policy 0, policy_version 1395847 (0.0007) [2023-12-27 15:19:10,682][20381] Updated weights for policy 1, policy_version 1394747 (0.0007) [2023-12-27 15:19:10,734][20381] Updated weights for policy 1, policy_version 1394757 (0.0008) [2023-12-27 15:19:10,790][20381] Updated weights for policy 1, policy_version 1394767 (0.0008) [2023-12-27 15:19:11,360][20347] Updated weights for policy 0, policy_version 1395857 (0.0008) [2023-12-27 15:19:11,427][20347] Updated weights for policy 0, policy_version 1395867 (0.0008) [2023-12-27 15:19:11,491][20347] Updated weights for policy 0, policy_version 1395877 (0.0007) [2023-12-27 15:19:11,547][20381] Updated weights for policy 1, policy_version 1394777 (0.0008) [2023-12-27 15:19:11,604][20381] Updated weights for policy 1, policy_version 1394787 (0.0008) [2023-12-27 15:19:11,672][20381] Updated weights for policy 1, policy_version 1394797 (0.0008) [2023-12-27 15:19:12,246][20347] Updated weights for policy 0, policy_version 1395887 (0.0007) [2023-12-27 15:19:12,304][20347] Updated weights for policy 0, policy_version 1395897 (0.0008) [2023-12-27 15:19:12,362][19170] Fps is (10 sec: 18841.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 714514432. Throughput: 0: 10185.3, 1: 10474.8. Samples: 714513248. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:12,363][19170] Avg episode reward: [(0, '960.900'), (1, '980.510')] [2023-12-27 15:19:12,375][20347] Updated weights for policy 0, policy_version 1395907 (0.0008) [2023-12-27 15:19:12,387][20381] Updated weights for policy 1, policy_version 1394807 (0.0008) [2023-12-27 15:19:12,442][20381] Updated weights for policy 1, policy_version 1394817 (0.0008) [2023-12-27 15:19:12,500][20381] Updated weights for policy 1, policy_version 1394827 (0.0008) [2023-12-27 15:19:13,116][20347] Updated weights for policy 0, policy_version 1395917 (0.0008) [2023-12-27 15:19:13,171][20381] Updated weights for policy 1, policy_version 1394837 (0.0007) [2023-12-27 15:19:13,178][20347] Updated weights for policy 0, policy_version 1395927 (0.0008) [2023-12-27 15:19:13,220][20381] Updated weights for policy 1, policy_version 1394847 (0.0010) [2023-12-27 15:19:13,237][20347] Updated weights for policy 0, policy_version 1395937 (0.0007) [2023-12-27 15:19:13,276][20381] Updated weights for policy 1, policy_version 1394857 (0.0008) [2023-12-27 15:19:13,820][20347] Updated weights for policy 0, policy_version 1395947 (0.0006) [2023-12-27 15:19:13,871][20347] Updated weights for policy 0, policy_version 1395957 (0.0008) [2023-12-27 15:19:13,901][20381] Updated weights for policy 1, policy_version 1394867 (0.0005) [2023-12-27 15:19:13,918][20347] Updated weights for policy 0, policy_version 1395967 (0.0006) [2023-12-27 15:19:13,954][20381] Updated weights for policy 1, policy_version 1394877 (0.0006) [2023-12-27 15:19:14,010][20381] Updated weights for policy 1, policy_version 1394887 (0.0006) [2023-12-27 15:19:14,572][20347] Updated weights for policy 0, policy_version 1395977 (0.0007) [2023-12-27 15:19:14,624][20347] Updated weights for policy 0, policy_version 1395987 (0.0005) [2023-12-27 15:19:14,642][20381] Updated weights for policy 1, policy_version 1394897 (0.0008) [2023-12-27 15:19:14,684][20347] Updated weights for policy 0, policy_version 1395997 (0.0006) [2023-12-27 15:19:14,693][20381] Updated weights for policy 1, policy_version 1394907 (0.0007) [2023-12-27 15:19:14,739][20347] Updated weights for policy 0, policy_version 1396007 (0.0007) [2023-12-27 15:19:14,746][20381] Updated weights for policy 1, policy_version 1394917 (0.0006) [2023-12-27 15:19:14,811][20381] Updated weights for policy 1, policy_version 1394927 (0.0008) [2023-12-27 15:19:15,412][20347] Updated weights for policy 0, policy_version 1396017 (0.0006) [2023-12-27 15:19:15,426][20381] Updated weights for policy 1, policy_version 1394937 (0.0006) [2023-12-27 15:19:15,470][20347] Updated weights for policy 0, policy_version 1396027 (0.0005) [2023-12-27 15:19:15,483][20381] Updated weights for policy 1, policy_version 1394947 (0.0007) [2023-12-27 15:19:15,527][20347] Updated weights for policy 0, policy_version 1396037 (0.0005) [2023-12-27 15:19:15,538][20381] Updated weights for policy 1, policy_version 1394957 (0.0005) [2023-12-27 15:19:16,116][20347] Updated weights for policy 0, policy_version 1396047 (0.0007) [2023-12-27 15:19:16,141][20381] Updated weights for policy 1, policy_version 1394967 (0.0009) [2023-12-27 15:19:16,180][20347] Updated weights for policy 0, policy_version 1396057 (0.0005) [2023-12-27 15:19:16,189][20381] Updated weights for policy 1, policy_version 1394977 (0.0009) [2023-12-27 15:19:16,237][20347] Updated weights for policy 0, policy_version 1396067 (0.0008) [2023-12-27 15:19:16,247][20381] Updated weights for policy 1, policy_version 1394987 (0.0008) [2023-12-27 15:19:16,860][20347] Updated weights for policy 0, policy_version 1396077 (0.0007) [2023-12-27 15:19:16,908][20347] Updated weights for policy 0, policy_version 1396087 (0.0008) [2023-12-27 15:19:16,937][20381] Updated weights for policy 1, policy_version 1394997 (0.0006) [2023-12-27 15:19:16,961][20347] Updated weights for policy 0, policy_version 1396097 (0.0008) [2023-12-27 15:19:16,993][20381] Updated weights for policy 1, policy_version 1395007 (0.0005) [2023-12-27 15:19:17,056][20381] Updated weights for policy 1, policy_version 1395017 (0.0007) [2023-12-27 15:19:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 714629120. Throughput: 0: 10172.0, 1: 10387.3. Samples: 714638000. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:17,363][19170] Avg episode reward: [(0, '990.200'), (1, '961.400')] [2023-12-27 15:19:17,694][20347] Updated weights for policy 0, policy_version 1396107 (0.0008) [2023-12-27 15:19:17,703][20381] Updated weights for policy 1, policy_version 1395027 (0.0007) [2023-12-27 15:19:17,750][20347] Updated weights for policy 0, policy_version 1396117 (0.0007) [2023-12-27 15:19:17,759][20381] Updated weights for policy 1, policy_version 1395037 (0.0005) [2023-12-27 15:19:17,800][20347] Updated weights for policy 0, policy_version 1396127 (0.0008) [2023-12-27 15:19:17,814][20381] Updated weights for policy 1, policy_version 1395047 (0.0006) [2023-12-27 15:19:18,397][20347] Updated weights for policy 0, policy_version 1396137 (0.0008) [2023-12-27 15:19:18,439][20381] Updated weights for policy 1, policy_version 1395057 (0.0006) [2023-12-27 15:19:18,463][20347] Updated weights for policy 0, policy_version 1396147 (0.0008) [2023-12-27 15:19:18,493][20381] Updated weights for policy 1, policy_version 1395067 (0.0005) [2023-12-27 15:19:18,526][20347] Updated weights for policy 0, policy_version 1396157 (0.0010) [2023-12-27 15:19:18,552][20381] Updated weights for policy 1, policy_version 1395077 (0.0008) [2023-12-27 15:19:18,592][20347] Updated weights for policy 0, policy_version 1396167 (0.0010) [2023-12-27 15:19:18,608][20381] Updated weights for policy 1, policy_version 1395087 (0.0007) [2023-12-27 15:19:19,223][20381] Updated weights for policy 1, policy_version 1395097 (0.0008) [2023-12-27 15:19:19,230][20347] Updated weights for policy 0, policy_version 1396177 (0.0008) [2023-12-27 15:19:19,281][20381] Updated weights for policy 1, policy_version 1395107 (0.0008) [2023-12-27 15:19:19,289][20347] Updated weights for policy 0, policy_version 1396187 (0.0010) [2023-12-27 15:19:19,344][20381] Updated weights for policy 1, policy_version 1395117 (0.0008) [2023-12-27 15:19:19,345][20347] Updated weights for policy 0, policy_version 1396197 (0.0009) [2023-12-27 15:19:20,038][20381] Updated weights for policy 1, policy_version 1395127 (0.0007) [2023-12-27 15:19:20,052][20347] Updated weights for policy 0, policy_version 1396207 (0.0007) [2023-12-27 15:19:20,096][20381] Updated weights for policy 1, policy_version 1395137 (0.0006) [2023-12-27 15:19:20,109][20347] Updated weights for policy 0, policy_version 1396217 (0.0007) [2023-12-27 15:19:20,157][20381] Updated weights for policy 1, policy_version 1395147 (0.0006) [2023-12-27 15:19:20,177][20347] Updated weights for policy 0, policy_version 1396227 (0.0008) [2023-12-27 15:19:20,754][20381] Updated weights for policy 1, policy_version 1395157 (0.0007) [2023-12-27 15:19:20,814][20381] Updated weights for policy 1, policy_version 1395167 (0.0006) [2023-12-27 15:19:20,860][20347] Updated weights for policy 0, policy_version 1396237 (0.0009) [2023-12-27 15:19:20,877][20381] Updated weights for policy 1, policy_version 1395177 (0.0006) [2023-12-27 15:19:20,919][20347] Updated weights for policy 0, policy_version 1396247 (0.0007) [2023-12-27 15:19:20,981][20347] Updated weights for policy 0, policy_version 1396257 (0.0008) [2023-12-27 15:19:21,565][20381] Updated weights for policy 1, policy_version 1395187 (0.0006) [2023-12-27 15:19:21,633][20381] Updated weights for policy 1, policy_version 1395197 (0.0008) [2023-12-27 15:19:21,699][20381] Updated weights for policy 1, policy_version 1395207 (0.0009) [2023-12-27 15:19:21,745][20347] Updated weights for policy 0, policy_version 1396267 (0.0008) [2023-12-27 15:19:21,798][20347] Updated weights for policy 0, policy_version 1396277 (0.0008) [2023-12-27 15:19:21,858][20347] Updated weights for policy 0, policy_version 1396287 (0.0008) [2023-12-27 15:19:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 714727424. Throughput: 0: 10145.8, 1: 10376.9. Samples: 714700664. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.980')] [2023-12-27 15:19:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001395216_357203968.pth... [2023-12-27 15:19:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001396296_357523456.pth... [2023-12-27 15:19:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001394000_356892672.pth [2023-12-27 15:19:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001395112_357220352.pth [2023-12-27 15:19:22,442][20381] Updated weights for policy 1, policy_version 1395217 (0.0008) [2023-12-27 15:19:22,494][20381] Updated weights for policy 1, policy_version 1395227 (0.0007) [2023-12-27 15:19:22,556][20381] Updated weights for policy 1, policy_version 1395237 (0.0008) [2023-12-27 15:19:22,559][20347] Updated weights for policy 0, policy_version 1396297 (0.0008) [2023-12-27 15:19:22,618][20381] Updated weights for policy 1, policy_version 1395247 (0.0009) [2023-12-27 15:19:22,619][20347] Updated weights for policy 0, policy_version 1396307 (0.0005) [2023-12-27 15:19:22,679][20347] Updated weights for policy 0, policy_version 1396317 (0.0005) [2023-12-27 15:19:22,742][20347] Updated weights for policy 0, policy_version 1396327 (0.0005) [2023-12-27 15:19:23,301][20381] Updated weights for policy 1, policy_version 1395257 (0.0010) [2023-12-27 15:19:23,352][20381] Updated weights for policy 1, policy_version 1395267 (0.0010) [2023-12-27 15:19:23,383][20347] Updated weights for policy 0, policy_version 1396337 (0.0005) [2023-12-27 15:19:23,407][20381] Updated weights for policy 1, policy_version 1395277 (0.0010) [2023-12-27 15:19:23,434][20347] Updated weights for policy 0, policy_version 1396347 (0.0006) [2023-12-27 15:19:23,483][20347] Updated weights for policy 0, policy_version 1396357 (0.0007) [2023-12-27 15:19:24,005][20381] Updated weights for policy 1, policy_version 1395287 (0.0011) [2023-12-27 15:19:24,054][20381] Updated weights for policy 1, policy_version 1395297 (0.0010) [2023-12-27 15:19:24,116][20381] Updated weights for policy 1, policy_version 1395307 (0.0011) [2023-12-27 15:19:24,171][20347] Updated weights for policy 0, policy_version 1396367 (0.0009) [2023-12-27 15:19:24,232][20347] Updated weights for policy 0, policy_version 1396377 (0.0010) [2023-12-27 15:19:24,294][20347] Updated weights for policy 0, policy_version 1396387 (0.0010) [2023-12-27 15:19:24,753][20381] Updated weights for policy 1, policy_version 1395317 (0.0010) [2023-12-27 15:19:24,815][20381] Updated weights for policy 1, policy_version 1395327 (0.0011) [2023-12-27 15:19:24,874][20381] Updated weights for policy 1, policy_version 1395337 (0.0011) [2023-12-27 15:19:24,904][20347] Updated weights for policy 0, policy_version 1396397 (0.0009) [2023-12-27 15:19:24,958][20347] Updated weights for policy 0, policy_version 1396407 (0.0007) [2023-12-27 15:19:25,017][20347] Updated weights for policy 0, policy_version 1396417 (0.0007) [2023-12-27 15:19:25,519][20381] Updated weights for policy 1, policy_version 1395347 (0.0009) [2023-12-27 15:19:25,576][20381] Updated weights for policy 1, policy_version 1395357 (0.0005) [2023-12-27 15:19:25,635][20381] Updated weights for policy 1, policy_version 1395367 (0.0005) [2023-12-27 15:19:25,686][20347] Updated weights for policy 0, policy_version 1396427 (0.0007) [2023-12-27 15:19:25,742][20347] Updated weights for policy 0, policy_version 1396437 (0.0008) [2023-12-27 15:19:25,797][20347] Updated weights for policy 0, policy_version 1396447 (0.0005) [2023-12-27 15:19:26,270][20381] Updated weights for policy 1, policy_version 1395377 (0.0006) [2023-12-27 15:19:26,330][20381] Updated weights for policy 1, policy_version 1395387 (0.0010) [2023-12-27 15:19:26,391][20381] Updated weights for policy 1, policy_version 1395397 (0.0010) [2023-12-27 15:19:26,406][20347] Updated weights for policy 0, policy_version 1396457 (0.0007) [2023-12-27 15:19:26,445][20381] Updated weights for policy 1, policy_version 1395407 (0.0011) [2023-12-27 15:19:26,454][20347] Updated weights for policy 0, policy_version 1396467 (0.0005) [2023-12-27 15:19:26,505][20347] Updated weights for policy 0, policy_version 1396477 (0.0005) [2023-12-27 15:19:26,555][20347] Updated weights for policy 0, policy_version 1396487 (0.0005) [2023-12-27 15:19:27,089][20347] Updated weights for policy 0, policy_version 1396497 (0.0005) [2023-12-27 15:19:27,130][20381] Updated weights for policy 1, policy_version 1395417 (0.0006) [2023-12-27 15:19:27,146][20347] Updated weights for policy 0, policy_version 1396507 (0.0006) [2023-12-27 15:19:27,177][20381] Updated weights for policy 1, policy_version 1395427 (0.0005) [2023-12-27 15:19:27,203][20347] Updated weights for policy 0, policy_version 1396517 (0.0007) [2023-12-27 15:19:27,229][20381] Updated weights for policy 1, policy_version 1395437 (0.0006) [2023-12-27 15:19:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 714842112. Throughput: 0: 10135.9, 1: 10405.0. Samples: 714824044. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:27,363][19170] Avg episode reward: [(0, '980.600'), (1, '980.500')] [2023-12-27 15:19:27,793][20381] Updated weights for policy 1, policy_version 1395447 (0.0006) [2023-12-27 15:19:27,847][20381] Updated weights for policy 1, policy_version 1395457 (0.0005) [2023-12-27 15:19:27,875][20347] Updated weights for policy 0, policy_version 1396527 (0.0007) [2023-12-27 15:19:27,901][20381] Updated weights for policy 1, policy_version 1395467 (0.0006) [2023-12-27 15:19:27,926][20347] Updated weights for policy 0, policy_version 1396537 (0.0006) [2023-12-27 15:19:27,986][20347] Updated weights for policy 0, policy_version 1396547 (0.0008) [2023-12-27 15:19:28,566][20381] Updated weights for policy 1, policy_version 1395477 (0.0009) [2023-12-27 15:19:28,624][20381] Updated weights for policy 1, policy_version 1395487 (0.0007) [2023-12-27 15:19:28,682][20381] Updated weights for policy 1, policy_version 1395497 (0.0008) [2023-12-27 15:19:28,718][20347] Updated weights for policy 0, policy_version 1396557 (0.0008) [2023-12-27 15:19:28,774][20347] Updated weights for policy 0, policy_version 1396567 (0.0008) [2023-12-27 15:19:28,828][20347] Updated weights for policy 0, policy_version 1396577 (0.0007) [2023-12-27 15:19:29,326][20381] Updated weights for policy 1, policy_version 1395507 (0.0008) [2023-12-27 15:19:29,389][20381] Updated weights for policy 1, policy_version 1395517 (0.0008) [2023-12-27 15:19:29,449][20381] Updated weights for policy 1, policy_version 1395527 (0.0008) [2023-12-27 15:19:29,514][20347] Updated weights for policy 0, policy_version 1396587 (0.0007) [2023-12-27 15:19:29,574][20347] Updated weights for policy 0, policy_version 1396597 (0.0008) [2023-12-27 15:19:29,630][20347] Updated weights for policy 0, policy_version 1396607 (0.0008) [2023-12-27 15:19:30,151][20381] Updated weights for policy 1, policy_version 1395537 (0.0008) [2023-12-27 15:19:30,201][20381] Updated weights for policy 1, policy_version 1395547 (0.0009) [2023-12-27 15:19:30,251][20381] Updated weights for policy 1, policy_version 1395557 (0.0009) [2023-12-27 15:19:30,296][20347] Updated weights for policy 0, policy_version 1396617 (0.0007) [2023-12-27 15:19:30,297][20381] Updated weights for policy 1, policy_version 1395567 (0.0009) [2023-12-27 15:19:30,343][20347] Updated weights for policy 0, policy_version 1396627 (0.0006) [2023-12-27 15:19:30,396][20347] Updated weights for policy 0, policy_version 1396637 (0.0006) [2023-12-27 15:19:30,450][20347] Updated weights for policy 0, policy_version 1396647 (0.0006) [2023-12-27 15:19:30,938][20381] Updated weights for policy 1, policy_version 1395577 (0.0005) [2023-12-27 15:19:30,982][20381] Updated weights for policy 1, policy_version 1395587 (0.0005) [2023-12-27 15:19:31,032][20381] Updated weights for policy 1, policy_version 1395597 (0.0007) [2023-12-27 15:19:31,106][20347] Updated weights for policy 0, policy_version 1396657 (0.0006) [2023-12-27 15:19:31,172][20347] Updated weights for policy 0, policy_version 1396667 (0.0007) [2023-12-27 15:19:31,237][20347] Updated weights for policy 0, policy_version 1396677 (0.0006) [2023-12-27 15:19:31,743][20381] Updated weights for policy 1, policy_version 1395607 (0.0009) [2023-12-27 15:19:31,814][20381] Updated weights for policy 1, policy_version 1395617 (0.0008) [2023-12-27 15:19:31,879][20381] Updated weights for policy 1, policy_version 1395627 (0.0007) [2023-12-27 15:19:31,962][20347] Updated weights for policy 0, policy_version 1396687 (0.0007) [2023-12-27 15:19:32,015][20347] Updated weights for policy 0, policy_version 1396697 (0.0008) [2023-12-27 15:19:32,066][20347] Updated weights for policy 0, policy_version 1396707 (0.0008) [2023-12-27 15:19:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 714940416. Throughput: 0: 10195.3, 1: 10427.4. Samples: 714949696. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:32,363][19170] Avg episode reward: [(0, '980.600'), (1, '951.250')] [2023-12-27 15:19:32,546][20381] Updated weights for policy 1, policy_version 1395637 (0.0006) [2023-12-27 15:19:32,608][20381] Updated weights for policy 1, policy_version 1395647 (0.0008) [2023-12-27 15:19:32,667][20381] Updated weights for policy 1, policy_version 1395657 (0.0007) [2023-12-27 15:19:32,778][20347] Updated weights for policy 0, policy_version 1396717 (0.0008) [2023-12-27 15:19:32,837][20347] Updated weights for policy 0, policy_version 1396727 (0.0008) [2023-12-27 15:19:32,887][20347] Updated weights for policy 0, policy_version 1396737 (0.0007) [2023-12-27 15:19:33,269][20381] Updated weights for policy 1, policy_version 1395667 (0.0008) [2023-12-27 15:19:33,319][20381] Updated weights for policy 1, policy_version 1395677 (0.0005) [2023-12-27 15:19:33,371][20381] Updated weights for policy 1, policy_version 1395687 (0.0005) [2023-12-27 15:19:33,598][20347] Updated weights for policy 0, policy_version 1396747 (0.0008) [2023-12-27 15:19:33,655][20347] Updated weights for policy 0, policy_version 1396757 (0.0010) [2023-12-27 15:19:33,705][20347] Updated weights for policy 0, policy_version 1396767 (0.0009) [2023-12-27 15:19:33,971][20381] Updated weights for policy 1, policy_version 1395697 (0.0007) [2023-12-27 15:19:34,028][20381] Updated weights for policy 1, policy_version 1395707 (0.0010) [2023-12-27 15:19:34,080][20381] Updated weights for policy 1, policy_version 1395717 (0.0008) [2023-12-27 15:19:34,149][20381] Updated weights for policy 1, policy_version 1395727 (0.0006) [2023-12-27 15:19:34,263][20347] Updated weights for policy 0, policy_version 1396777 (0.0005) [2023-12-27 15:19:34,326][20347] Updated weights for policy 0, policy_version 1396787 (0.0010) [2023-12-27 15:19:34,383][20347] Updated weights for policy 0, policy_version 1396797 (0.0010) [2023-12-27 15:19:34,441][20347] Updated weights for policy 0, policy_version 1396807 (0.0009) [2023-12-27 15:19:34,816][20381] Updated weights for policy 1, policy_version 1395737 (0.0008) [2023-12-27 15:19:34,871][20381] Updated weights for policy 1, policy_version 1395747 (0.0005) [2023-12-27 15:19:34,932][20381] Updated weights for policy 1, policy_version 1395757 (0.0005) [2023-12-27 15:19:35,087][20347] Updated weights for policy 0, policy_version 1396817 (0.0008) [2023-12-27 15:19:35,131][20347] Updated weights for policy 0, policy_version 1396827 (0.0006) [2023-12-27 15:19:35,181][20347] Updated weights for policy 0, policy_version 1396837 (0.0006) [2023-12-27 15:19:35,470][20381] Updated weights for policy 1, policy_version 1395767 (0.0007) [2023-12-27 15:19:35,520][20381] Updated weights for policy 1, policy_version 1395777 (0.0007) [2023-12-27 15:19:35,565][20381] Updated weights for policy 1, policy_version 1395787 (0.0009) [2023-12-27 15:19:35,800][20347] Updated weights for policy 0, policy_version 1396847 (0.0009) [2023-12-27 15:19:35,847][20347] Updated weights for policy 0, policy_version 1396857 (0.0006) [2023-12-27 15:19:35,902][20347] Updated weights for policy 0, policy_version 1396867 (0.0006) [2023-12-27 15:19:36,275][20381] Updated weights for policy 1, policy_version 1395797 (0.0008) [2023-12-27 15:19:36,335][20381] Updated weights for policy 1, policy_version 1395807 (0.0008) [2023-12-27 15:19:36,398][20381] Updated weights for policy 1, policy_version 1395817 (0.0008) [2023-12-27 15:19:36,534][20347] Updated weights for policy 0, policy_version 1396877 (0.0006) [2023-12-27 15:19:36,600][20347] Updated weights for policy 0, policy_version 1396887 (0.0006) [2023-12-27 15:19:36,665][20347] Updated weights for policy 0, policy_version 1396897 (0.0006) [2023-12-27 15:19:37,056][20381] Updated weights for policy 1, policy_version 1395827 (0.0008) [2023-12-27 15:19:37,113][20381] Updated weights for policy 1, policy_version 1395837 (0.0009) [2023-12-27 15:19:37,170][20381] Updated weights for policy 1, policy_version 1395847 (0.0008) [2023-12-27 15:19:37,217][20347] Updated weights for policy 0, policy_version 1396907 (0.0007) [2023-12-27 15:19:37,274][20347] Updated weights for policy 0, policy_version 1396917 (0.0011) [2023-12-27 15:19:37,327][20347] Updated weights for policy 0, policy_version 1396927 (0.0010) [2023-12-27 15:19:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 715046912. Throughput: 0: 10242.2, 1: 10447.7. Samples: 715014592. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:37,363][19170] Avg episode reward: [(0, '990.280'), (1, '951.310')] [2023-12-27 15:19:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001395856_357367808.pth... [2023-12-27 15:19:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001394608_357048320.pth [2023-12-27 15:19:37,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001396936_357687296.pth... [2023-12-27 15:19:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001395720_357376000.pth [2023-12-27 15:19:37,878][20381] Updated weights for policy 1, policy_version 1395857 (0.0007) [2023-12-27 15:19:37,929][20381] Updated weights for policy 1, policy_version 1395867 (0.0006) [2023-12-27 15:19:37,957][20347] Updated weights for policy 0, policy_version 1396937 (0.0007) [2023-12-27 15:19:37,988][20381] Updated weights for policy 1, policy_version 1395877 (0.0008) [2023-12-27 15:19:38,014][20347] Updated weights for policy 0, policy_version 1396947 (0.0006) [2023-12-27 15:19:38,043][20381] Updated weights for policy 1, policy_version 1395887 (0.0006) [2023-12-27 15:19:38,065][20347] Updated weights for policy 0, policy_version 1396957 (0.0006) [2023-12-27 15:19:38,117][20347] Updated weights for policy 0, policy_version 1396967 (0.0005) [2023-12-27 15:19:38,733][20381] Updated weights for policy 1, policy_version 1395897 (0.0008) [2023-12-27 15:19:38,782][20347] Updated weights for policy 0, policy_version 1396977 (0.0007) [2023-12-27 15:19:38,787][20381] Updated weights for policy 1, policy_version 1395907 (0.0007) [2023-12-27 15:19:38,844][20381] Updated weights for policy 1, policy_version 1395917 (0.0005) [2023-12-27 15:19:38,846][20347] Updated weights for policy 0, policy_version 1396987 (0.0008) [2023-12-27 15:19:38,905][20347] Updated weights for policy 0, policy_version 1396997 (0.0006) [2023-12-27 15:19:39,592][20381] Updated weights for policy 1, policy_version 1395927 (0.0006) [2023-12-27 15:19:39,615][20347] Updated weights for policy 0, policy_version 1397007 (0.0007) [2023-12-27 15:19:39,645][20381] Updated weights for policy 1, policy_version 1395937 (0.0006) [2023-12-27 15:19:39,674][20347] Updated weights for policy 0, policy_version 1397017 (0.0006) [2023-12-27 15:19:39,709][20381] Updated weights for policy 1, policy_version 1395947 (0.0006) [2023-12-27 15:19:39,734][20347] Updated weights for policy 0, policy_version 1397027 (0.0007) [2023-12-27 15:19:40,360][20381] Updated weights for policy 1, policy_version 1395957 (0.0007) [2023-12-27 15:19:40,423][20381] Updated weights for policy 1, policy_version 1395967 (0.0009) [2023-12-27 15:19:40,482][20381] Updated weights for policy 1, policy_version 1395977 (0.0010) [2023-12-27 15:19:40,492][20347] Updated weights for policy 0, policy_version 1397037 (0.0006) [2023-12-27 15:19:40,552][20347] Updated weights for policy 0, policy_version 1397047 (0.0006) [2023-12-27 15:19:40,609][20347] Updated weights for policy 0, policy_version 1397057 (0.0007) [2023-12-27 15:19:41,159][20381] Updated weights for policy 1, policy_version 1395987 (0.0010) [2023-12-27 15:19:41,229][20381] Updated weights for policy 1, policy_version 1395997 (0.0008) [2023-12-27 15:19:41,294][20381] Updated weights for policy 1, policy_version 1396007 (0.0008) [2023-12-27 15:19:41,295][20347] Updated weights for policy 0, policy_version 1397067 (0.0007) [2023-12-27 15:19:41,360][20347] Updated weights for policy 0, policy_version 1397077 (0.0006) [2023-12-27 15:19:41,423][20347] Updated weights for policy 0, policy_version 1397087 (0.0007) [2023-12-27 15:19:42,042][20347] Updated weights for policy 0, policy_version 1397097 (0.0006) [2023-12-27 15:19:42,074][20381] Updated weights for policy 1, policy_version 1396017 (0.0007) [2023-12-27 15:19:42,098][20347] Updated weights for policy 0, policy_version 1397107 (0.0008) [2023-12-27 15:19:42,127][20381] Updated weights for policy 1, policy_version 1396027 (0.0006) [2023-12-27 15:19:42,154][20347] Updated weights for policy 0, policy_version 1397117 (0.0011) [2023-12-27 15:19:42,184][20381] Updated weights for policy 1, policy_version 1396037 (0.0011) [2023-12-27 15:19:42,209][20347] Updated weights for policy 0, policy_version 1397127 (0.0010) [2023-12-27 15:19:42,241][20381] Updated weights for policy 1, policy_version 1396047 (0.0011) [2023-12-27 15:19:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.5, 300 sec: 20577.2). Total num frames: 715153408. Throughput: 0: 10239.7, 1: 10415.8. Samples: 715138368. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:42,363][19170] Avg episode reward: [(0, '990.280'), (1, '990.230')] [2023-12-27 15:19:42,889][20347] Updated weights for policy 0, policy_version 1397137 (0.0006) [2023-12-27 15:19:42,946][20347] Updated weights for policy 0, policy_version 1397147 (0.0005) [2023-12-27 15:19:42,990][20381] Updated weights for policy 1, policy_version 1396057 (0.0009) [2023-12-27 15:19:42,999][20347] Updated weights for policy 0, policy_version 1397157 (0.0005) [2023-12-27 15:19:43,036][20381] Updated weights for policy 1, policy_version 1396067 (0.0005) [2023-12-27 15:19:43,090][20381] Updated weights for policy 1, policy_version 1396077 (0.0007) [2023-12-27 15:19:43,620][20347] Updated weights for policy 0, policy_version 1397167 (0.0009) [2023-12-27 15:19:43,676][20347] Updated weights for policy 0, policy_version 1397177 (0.0010) [2023-12-27 15:19:43,687][20381] Updated weights for policy 1, policy_version 1396087 (0.0008) [2023-12-27 15:19:43,731][20347] Updated weights for policy 0, policy_version 1397187 (0.0010) [2023-12-27 15:19:43,746][20381] Updated weights for policy 1, policy_version 1396097 (0.0007) [2023-12-27 15:19:43,799][20381] Updated weights for policy 1, policy_version 1396107 (0.0007) [2023-12-27 15:19:44,349][20347] Updated weights for policy 0, policy_version 1397197 (0.0008) [2023-12-27 15:19:44,406][20347] Updated weights for policy 0, policy_version 1397207 (0.0007) [2023-12-27 15:19:44,458][20347] Updated weights for policy 0, policy_version 1397217 (0.0007) [2023-12-27 15:19:44,470][20381] Updated weights for policy 1, policy_version 1396117 (0.0009) [2023-12-27 15:19:44,525][20381] Updated weights for policy 1, policy_version 1396127 (0.0010) [2023-12-27 15:19:44,583][20381] Updated weights for policy 1, policy_version 1396137 (0.0010) [2023-12-27 15:19:45,107][20347] Updated weights for policy 0, policy_version 1397227 (0.0006) [2023-12-27 15:19:45,165][20347] Updated weights for policy 0, policy_version 1397237 (0.0007) [2023-12-27 15:19:45,224][20347] Updated weights for policy 0, policy_version 1397247 (0.0007) [2023-12-27 15:19:45,337][20381] Updated weights for policy 1, policy_version 1396147 (0.0007) [2023-12-27 15:19:45,394][20381] Updated weights for policy 1, policy_version 1396157 (0.0008) [2023-12-27 15:19:45,450][20381] Updated weights for policy 1, policy_version 1396167 (0.0006) [2023-12-27 15:19:45,906][20347] Updated weights for policy 0, policy_version 1397257 (0.0007) [2023-12-27 15:19:45,952][20347] Updated weights for policy 0, policy_version 1397267 (0.0005) [2023-12-27 15:19:46,010][20347] Updated weights for policy 0, policy_version 1397277 (0.0005) [2023-12-27 15:19:46,043][20381] Updated weights for policy 1, policy_version 1396177 (0.0005) [2023-12-27 15:19:46,063][20347] Updated weights for policy 0, policy_version 1397287 (0.0007) [2023-12-27 15:19:46,104][20381] Updated weights for policy 1, policy_version 1396187 (0.0007) [2023-12-27 15:19:46,161][20381] Updated weights for policy 1, policy_version 1396197 (0.0009) [2023-12-27 15:19:46,222][20381] Updated weights for policy 1, policy_version 1396207 (0.0009) [2023-12-27 15:19:46,596][20347] Updated weights for policy 0, policy_version 1397297 (0.0006) [2023-12-27 15:19:46,653][20347] Updated weights for policy 0, policy_version 1397307 (0.0005) [2023-12-27 15:19:46,699][20347] Updated weights for policy 0, policy_version 1397317 (0.0005) [2023-12-27 15:19:46,878][20381] Updated weights for policy 1, policy_version 1396217 (0.0006) [2023-12-27 15:19:46,943][20381] Updated weights for policy 1, policy_version 1396227 (0.0005) [2023-12-27 15:19:47,011][20381] Updated weights for policy 1, policy_version 1396237 (0.0005) [2023-12-27 15:19:47,361][20347] Updated weights for policy 0, policy_version 1397327 (0.0007) [2023-12-27 15:19:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 715251712. Throughput: 0: 10321.0, 1: 10461.3. Samples: 715264960. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:47,363][19170] Avg episode reward: [(0, '980.480'), (1, '970.910')] [2023-12-27 15:19:47,423][20347] Updated weights for policy 0, policy_version 1397337 (0.0008) [2023-12-27 15:19:47,484][20347] Updated weights for policy 0, policy_version 1397347 (0.0006) [2023-12-27 15:19:47,553][20381] Updated weights for policy 1, policy_version 1396247 (0.0007) [2023-12-27 15:19:47,610][20381] Updated weights for policy 1, policy_version 1396257 (0.0007) [2023-12-27 15:19:47,668][20381] Updated weights for policy 1, policy_version 1396267 (0.0008) [2023-12-27 15:19:48,106][20347] Updated weights for policy 0, policy_version 1397357 (0.0006) [2023-12-27 15:19:48,168][20347] Updated weights for policy 0, policy_version 1397367 (0.0005) [2023-12-27 15:19:48,229][20347] Updated weights for policy 0, policy_version 1397377 (0.0006) [2023-12-27 15:19:48,339][20381] Updated weights for policy 1, policy_version 1396277 (0.0007) [2023-12-27 15:19:48,396][20381] Updated weights for policy 1, policy_version 1396287 (0.0007) [2023-12-27 15:19:48,446][20381] Updated weights for policy 1, policy_version 1396297 (0.0008) [2023-12-27 15:19:48,799][20347] Updated weights for policy 0, policy_version 1397387 (0.0006) [2023-12-27 15:19:48,861][20347] Updated weights for policy 0, policy_version 1397397 (0.0008) [2023-12-27 15:19:48,919][20347] Updated weights for policy 0, policy_version 1397407 (0.0008) [2023-12-27 15:19:49,095][20381] Updated weights for policy 1, policy_version 1396307 (0.0008) [2023-12-27 15:19:49,163][20381] Updated weights for policy 1, policy_version 1396317 (0.0005) [2023-12-27 15:19:49,228][20381] Updated weights for policy 1, policy_version 1396327 (0.0007) [2023-12-27 15:19:49,640][20347] Updated weights for policy 0, policy_version 1397417 (0.0007) [2023-12-27 15:19:49,702][20347] Updated weights for policy 0, policy_version 1397427 (0.0007) [2023-12-27 15:19:49,758][20347] Updated weights for policy 0, policy_version 1397437 (0.0008) [2023-12-27 15:19:49,817][20347] Updated weights for policy 0, policy_version 1397447 (0.0007) [2023-12-27 15:19:49,865][20381] Updated weights for policy 1, policy_version 1396337 (0.0007) [2023-12-27 15:19:49,927][20381] Updated weights for policy 1, policy_version 1396347 (0.0008) [2023-12-27 15:19:49,989][20381] Updated weights for policy 1, policy_version 1396357 (0.0007) [2023-12-27 15:19:50,054][20381] Updated weights for policy 1, policy_version 1396367 (0.0008) [2023-12-27 15:19:50,544][20347] Updated weights for policy 0, policy_version 1397457 (0.0010) [2023-12-27 15:19:50,609][20347] Updated weights for policy 0, policy_version 1397467 (0.0011) [2023-12-27 15:19:50,661][20347] Updated weights for policy 0, policy_version 1397477 (0.0006) [2023-12-27 15:19:50,731][20381] Updated weights for policy 1, policy_version 1396377 (0.0008) [2023-12-27 15:19:50,735][20292] KL-divergence is very high: 571.8312 [2023-12-27 15:19:50,783][20292] KL-divergence is very high: 425.0624 [2023-12-27 15:19:50,788][20381] Updated weights for policy 1, policy_version 1396387 (0.0008) [2023-12-27 15:19:50,852][20381] Updated weights for policy 1, policy_version 1396397 (0.0008) [2023-12-27 15:19:51,270][20347] Updated weights for policy 0, policy_version 1397487 (0.0009) [2023-12-27 15:19:51,328][20347] Updated weights for policy 0, policy_version 1397497 (0.0010) [2023-12-27 15:19:51,398][20347] Updated weights for policy 0, policy_version 1397507 (0.0011) [2023-12-27 15:19:51,615][20381] Updated weights for policy 1, policy_version 1396407 (0.0007) [2023-12-27 15:19:51,683][20381] Updated weights for policy 1, policy_version 1396417 (0.0008) [2023-12-27 15:19:51,744][20381] Updated weights for policy 1, policy_version 1396427 (0.0009) [2023-12-27 15:19:52,059][20347] Updated weights for policy 0, policy_version 1397517 (0.0007) [2023-12-27 15:19:52,115][20347] Updated weights for policy 0, policy_version 1397527 (0.0008) [2023-12-27 15:19:52,179][20347] Updated weights for policy 0, policy_version 1397537 (0.0008) [2023-12-27 15:19:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 715358208. Throughput: 0: 10368.4, 1: 10418.9. Samples: 715326976. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:52,363][19170] Avg episode reward: [(0, '960.940'), (1, '931.920')] [2023-12-27 15:19:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001396432_357515264.pth... [2023-12-27 15:19:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001397544_357842944.pth... [2023-12-27 15:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001395216_357203968.pth [2023-12-27 15:19:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001396296_357523456.pth [2023-12-27 15:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001396432_357515264.pth [2023-12-27 15:19:52,373][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001397544_357842944.pth [2023-12-27 15:19:52,507][20381] Updated weights for policy 1, policy_version 1396437 (0.0008) [2023-12-27 15:19:52,565][20381] Updated weights for policy 1, policy_version 1396447 (0.0010) [2023-12-27 15:19:52,619][20381] Updated weights for policy 1, policy_version 1396457 (0.0009) [2023-12-27 15:19:52,910][20347] Updated weights for policy 0, policy_version 1397547 (0.0008) [2023-12-27 15:19:52,967][20347] Updated weights for policy 0, policy_version 1397557 (0.0010) [2023-12-27 15:19:53,028][20347] Updated weights for policy 0, policy_version 1397567 (0.0010) [2023-12-27 15:19:53,278][20381] Updated weights for policy 1, policy_version 1396467 (0.0005) [2023-12-27 15:19:53,337][20381] Updated weights for policy 1, policy_version 1396477 (0.0006) [2023-12-27 15:19:53,389][20381] Updated weights for policy 1, policy_version 1396487 (0.0005) [2023-12-27 15:19:53,648][20347] Updated weights for policy 0, policy_version 1397577 (0.0010) [2023-12-27 15:19:53,703][20347] Updated weights for policy 0, policy_version 1397587 (0.0008) [2023-12-27 15:19:53,756][20347] Updated weights for policy 0, policy_version 1397597 (0.0007) [2023-12-27 15:19:53,809][20347] Updated weights for policy 0, policy_version 1397607 (0.0005) [2023-12-27 15:19:54,019][20381] Updated weights for policy 1, policy_version 1396497 (0.0005) [2023-12-27 15:19:54,085][20381] Updated weights for policy 1, policy_version 1396507 (0.0008) [2023-12-27 15:19:54,140][20381] Updated weights for policy 1, policy_version 1396517 (0.0007) [2023-12-27 15:19:54,195][20381] Updated weights for policy 1, policy_version 1396527 (0.0007) [2023-12-27 15:19:54,391][20347] Updated weights for policy 0, policy_version 1397617 (0.0010) [2023-12-27 15:19:54,449][20347] Updated weights for policy 0, policy_version 1397627 (0.0008) [2023-12-27 15:19:54,512][20347] Updated weights for policy 0, policy_version 1397637 (0.0007) [2023-12-27 15:19:54,897][20381] Updated weights for policy 1, policy_version 1396537 (0.0005) [2023-12-27 15:19:54,949][20381] Updated weights for policy 1, policy_version 1396547 (0.0005) [2023-12-27 15:19:55,014][20381] Updated weights for policy 1, policy_version 1396557 (0.0007) [2023-12-27 15:19:55,205][20347] Updated weights for policy 0, policy_version 1397647 (0.0007) [2023-12-27 15:19:55,260][20347] Updated weights for policy 0, policy_version 1397657 (0.0007) [2023-12-27 15:19:55,313][20347] Updated weights for policy 0, policy_version 1397667 (0.0007) [2023-12-27 15:19:55,618][20381] Updated weights for policy 1, policy_version 1396567 (0.0008) [2023-12-27 15:19:55,673][20381] Updated weights for policy 1, policy_version 1396577 (0.0005) [2023-12-27 15:19:55,724][20381] Updated weights for policy 1, policy_version 1396587 (0.0006) [2023-12-27 15:19:55,968][20347] Updated weights for policy 0, policy_version 1397677 (0.0006) [2023-12-27 15:19:56,024][20347] Updated weights for policy 0, policy_version 1397687 (0.0008) [2023-12-27 15:19:56,075][20347] Updated weights for policy 0, policy_version 1397697 (0.0005) [2023-12-27 15:19:56,342][20381] Updated weights for policy 1, policy_version 1396597 (0.0006) [2023-12-27 15:19:56,400][20381] Updated weights for policy 1, policy_version 1396607 (0.0007) [2023-12-27 15:19:56,457][20381] Updated weights for policy 1, policy_version 1396617 (0.0007) [2023-12-27 15:19:56,732][20347] Updated weights for policy 0, policy_version 1397707 (0.0008) [2023-12-27 15:19:56,776][20347] Updated weights for policy 0, policy_version 1397717 (0.0010) [2023-12-27 15:19:56,821][20347] Updated weights for policy 0, policy_version 1397727 (0.0010) [2023-12-27 15:19:57,050][20381] Updated weights for policy 1, policy_version 1396627 (0.0007) [2023-12-27 15:19:57,102][20381] Updated weights for policy 1, policy_version 1396637 (0.0005) [2023-12-27 15:19:57,155][20381] Updated weights for policy 1, policy_version 1396647 (0.0007) [2023-12-27 15:19:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 715464704. Throughput: 0: 10440.2, 1: 10403.0. Samples: 715451192. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-12-27 15:19:57,363][19170] Avg episode reward: [(0, '980.460'), (1, '961.010')] [2023-12-27 15:19:57,412][20347] Updated weights for policy 0, policy_version 1397737 (0.0010) [2023-12-27 15:19:57,463][20347] Updated weights for policy 0, policy_version 1397747 (0.0005) [2023-12-27 15:19:57,521][20347] Updated weights for policy 0, policy_version 1397757 (0.0005) [2023-12-27 15:19:57,574][20347] Updated weights for policy 0, policy_version 1397767 (0.0005) [2023-12-27 15:19:57,752][20381] Updated weights for policy 1, policy_version 1396657 (0.0008) [2023-12-27 15:19:57,807][20381] Updated weights for policy 1, policy_version 1396667 (0.0006) [2023-12-27 15:19:57,857][20381] Updated weights for policy 1, policy_version 1396677 (0.0005) [2023-12-27 15:19:57,912][20381] Updated weights for policy 1, policy_version 1396687 (0.0005) [2023-12-27 15:19:58,084][20347] Updated weights for policy 0, policy_version 1397777 (0.0009) [2023-12-27 15:19:58,128][20347] Updated weights for policy 0, policy_version 1397787 (0.0010) [2023-12-27 15:19:58,186][20347] Updated weights for policy 0, policy_version 1397797 (0.0010) [2023-12-27 15:19:58,581][20381] Updated weights for policy 1, policy_version 1396697 (0.0010) [2023-12-27 15:19:58,647][20381] Updated weights for policy 1, policy_version 1396707 (0.0007) [2023-12-27 15:19:58,709][20381] Updated weights for policy 1, policy_version 1396717 (0.0008) [2023-12-27 15:19:59,009][20347] Updated weights for policy 0, policy_version 1397807 (0.0008) [2023-12-27 15:19:59,073][20347] Updated weights for policy 0, policy_version 1397817 (0.0009) [2023-12-27 15:19:59,137][20347] Updated weights for policy 0, policy_version 1397827 (0.0008) [2023-12-27 15:19:59,576][20381] Updated weights for policy 1, policy_version 1396727 (0.0007) [2023-12-27 15:19:59,636][20381] Updated weights for policy 1, policy_version 1396737 (0.0007) [2023-12-27 15:19:59,687][20381] Updated weights for policy 1, policy_version 1396747 (0.0008) [2023-12-27 15:19:59,833][20347] Updated weights for policy 0, policy_version 1397837 (0.0007) [2023-12-27 15:19:59,886][20347] Updated weights for policy 0, policy_version 1397847 (0.0007) [2023-12-27 15:19:59,939][20347] Updated weights for policy 0, policy_version 1397857 (0.0006) [2023-12-27 15:20:00,405][20381] Updated weights for policy 1, policy_version 1396757 (0.0007) [2023-12-27 15:20:00,471][20381] Updated weights for policy 1, policy_version 1396767 (0.0007) [2023-12-27 15:20:00,529][20381] Updated weights for policy 1, policy_version 1396777 (0.0006) [2023-12-27 15:20:00,585][20347] Updated weights for policy 0, policy_version 1397867 (0.0008) [2023-12-27 15:20:00,646][20347] Updated weights for policy 0, policy_version 1397877 (0.0008) [2023-12-27 15:20:00,703][20347] Updated weights for policy 0, policy_version 1397887 (0.0008) [2023-12-27 15:20:01,109][20381] Updated weights for policy 1, policy_version 1396787 (0.0007) [2023-12-27 15:20:01,173][20381] Updated weights for policy 1, policy_version 1396797 (0.0008) [2023-12-27 15:20:01,227][20381] Updated weights for policy 1, policy_version 1396807 (0.0008) [2023-12-27 15:20:01,457][20347] Updated weights for policy 0, policy_version 1397897 (0.0007) [2023-12-27 15:20:01,515][20347] Updated weights for policy 0, policy_version 1397907 (0.0008) [2023-12-27 15:20:01,575][20347] Updated weights for policy 0, policy_version 1397917 (0.0008) [2023-12-27 15:20:01,657][20347] Updated weights for policy 0, policy_version 1397927 (0.0008) [2023-12-27 15:20:01,956][20381] Updated weights for policy 1, policy_version 1396817 (0.0008) [2023-12-27 15:20:02,019][20381] Updated weights for policy 1, policy_version 1396827 (0.0007) [2023-12-27 15:20:02,083][20381] Updated weights for policy 1, policy_version 1396837 (0.0008) [2023-12-27 15:20:02,143][20381] Updated weights for policy 1, policy_version 1396847 (0.0009) [2023-12-27 15:20:02,347][20347] Updated weights for policy 0, policy_version 1397937 (0.0007) [2023-12-27 15:20:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 715563008. Throughput: 0: 10447.3, 1: 10349.6. Samples: 715573860. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '980.430')] [2023-12-27 15:20:02,409][20347] Updated weights for policy 0, policy_version 1397947 (0.0007) [2023-12-27 15:20:02,473][20347] Updated weights for policy 0, policy_version 1397957 (0.0008) [2023-12-27 15:20:02,857][20381] Updated weights for policy 1, policy_version 1396857 (0.0008) [2023-12-27 15:20:02,909][20381] Updated weights for policy 1, policy_version 1396867 (0.0005) [2023-12-27 15:20:02,966][20381] Updated weights for policy 1, policy_version 1396877 (0.0006) [2023-12-27 15:20:03,137][20347] Updated weights for policy 0, policy_version 1397967 (0.0006) [2023-12-27 15:20:03,195][20347] Updated weights for policy 0, policy_version 1397977 (0.0005) [2023-12-27 15:20:03,251][20347] Updated weights for policy 0, policy_version 1397987 (0.0005) [2023-12-27 15:20:03,558][20381] Updated weights for policy 1, policy_version 1396887 (0.0007) [2023-12-27 15:20:03,616][20381] Updated weights for policy 1, policy_version 1396897 (0.0005) [2023-12-27 15:20:03,674][20381] Updated weights for policy 1, policy_version 1396907 (0.0007) [2023-12-27 15:20:03,943][20347] Updated weights for policy 0, policy_version 1397997 (0.0009) [2023-12-27 15:20:04,009][20347] Updated weights for policy 0, policy_version 1398007 (0.0010) [2023-12-27 15:20:04,074][20347] Updated weights for policy 0, policy_version 1398017 (0.0010) [2023-12-27 15:20:04,303][20381] Updated weights for policy 1, policy_version 1396917 (0.0007) [2023-12-27 15:20:04,362][20381] Updated weights for policy 1, policy_version 1396927 (0.0007) [2023-12-27 15:20:04,428][20381] Updated weights for policy 1, policy_version 1396937 (0.0006) [2023-12-27 15:20:04,723][20347] Updated weights for policy 0, policy_version 1398027 (0.0009) [2023-12-27 15:20:04,782][20347] Updated weights for policy 0, policy_version 1398037 (0.0009) [2023-12-27 15:20:04,846][20347] Updated weights for policy 0, policy_version 1398047 (0.0010) [2023-12-27 15:20:05,034][20381] Updated weights for policy 1, policy_version 1396947 (0.0005) [2023-12-27 15:20:05,093][20381] Updated weights for policy 1, policy_version 1396957 (0.0006) [2023-12-27 15:20:05,151][20381] Updated weights for policy 1, policy_version 1396967 (0.0006) [2023-12-27 15:20:05,489][20347] Updated weights for policy 0, policy_version 1398057 (0.0010) [2023-12-27 15:20:05,545][20347] Updated weights for policy 0, policy_version 1398067 (0.0005) [2023-12-27 15:20:05,607][20347] Updated weights for policy 0, policy_version 1398077 (0.0008) [2023-12-27 15:20:05,664][20347] Updated weights for policy 0, policy_version 1398087 (0.0010) [2023-12-27 15:20:05,798][20381] Updated weights for policy 1, policy_version 1396977 (0.0007) [2023-12-27 15:20:05,852][20381] Updated weights for policy 1, policy_version 1396987 (0.0007) [2023-12-27 15:20:05,903][20381] Updated weights for policy 1, policy_version 1396997 (0.0007) [2023-12-27 15:20:05,951][20381] Updated weights for policy 1, policy_version 1397007 (0.0008) [2023-12-27 15:20:06,326][20347] Updated weights for policy 0, policy_version 1398097 (0.0006) [2023-12-27 15:20:06,382][20347] Updated weights for policy 0, policy_version 1398107 (0.0008) [2023-12-27 15:20:06,443][20347] Updated weights for policy 0, policy_version 1398117 (0.0008) [2023-12-27 15:20:06,673][20381] Updated weights for policy 1, policy_version 1397017 (0.0007) [2023-12-27 15:20:06,728][20381] Updated weights for policy 1, policy_version 1397027 (0.0007) [2023-12-27 15:20:06,788][20381] Updated weights for policy 1, policy_version 1397037 (0.0008) [2023-12-27 15:20:07,087][20347] Updated weights for policy 0, policy_version 1398127 (0.0008) [2023-12-27 15:20:07,136][20347] Updated weights for policy 0, policy_version 1398137 (0.0007) [2023-12-27 15:20:07,182][20347] Updated weights for policy 0, policy_version 1398147 (0.0007) [2023-12-27 15:20:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 715669504. Throughput: 0: 10443.6, 1: 10358.3. Samples: 715636748. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:07,363][19170] Avg episode reward: [(0, '990.220'), (1, '970.720')] [2023-12-27 15:20:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001397040_357670912.pth... [2023-12-27 15:20:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001398152_357998592.pth... [2023-12-27 15:20:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001395856_357367808.pth [2023-12-27 15:20:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001396936_357687296.pth [2023-12-27 15:20:07,498][20381] Updated weights for policy 1, policy_version 1397047 (0.0008) [2023-12-27 15:20:07,548][20381] Updated weights for policy 1, policy_version 1397057 (0.0007) [2023-12-27 15:20:07,596][20381] Updated weights for policy 1, policy_version 1397067 (0.0008) [2023-12-27 15:20:07,903][20347] Updated weights for policy 0, policy_version 1398157 (0.0009) [2023-12-27 15:20:07,963][20347] Updated weights for policy 0, policy_version 1398167 (0.0007) [2023-12-27 15:20:08,022][20347] Updated weights for policy 0, policy_version 1398177 (0.0005) [2023-12-27 15:20:08,348][20381] Updated weights for policy 1, policy_version 1397077 (0.0008) [2023-12-27 15:20:08,406][20381] Updated weights for policy 1, policy_version 1397087 (0.0008) [2023-12-27 15:20:08,408][20292] KL-divergence is very high: 110.4425 [2023-12-27 15:20:08,460][20292] KL-divergence is very high: 107.8475 [2023-12-27 15:20:08,474][20381] Updated weights for policy 1, policy_version 1397097 (0.0007) [2023-12-27 15:20:08,624][20347] Updated weights for policy 0, policy_version 1398187 (0.0006) [2023-12-27 15:20:08,693][20347] Updated weights for policy 0, policy_version 1398197 (0.0010) [2023-12-27 15:20:08,753][20347] Updated weights for policy 0, policy_version 1398207 (0.0010) [2023-12-27 15:20:09,162][20381] Updated weights for policy 1, policy_version 1397107 (0.0008) [2023-12-27 15:20:09,209][20381] Updated weights for policy 1, policy_version 1397117 (0.0009) [2023-12-27 15:20:09,275][20381] Updated weights for policy 1, policy_version 1397127 (0.0008) [2023-12-27 15:20:09,474][20347] Updated weights for policy 0, policy_version 1398217 (0.0009) [2023-12-27 15:20:09,527][20347] Updated weights for policy 0, policy_version 1398227 (0.0008) [2023-12-27 15:20:09,589][20347] Updated weights for policy 0, policy_version 1398237 (0.0007) [2023-12-27 15:20:09,640][20347] Updated weights for policy 0, policy_version 1398247 (0.0007) [2023-12-27 15:20:10,035][20381] Updated weights for policy 1, policy_version 1397137 (0.0007) [2023-12-27 15:20:10,087][20381] Updated weights for policy 1, policy_version 1397147 (0.0007) [2023-12-27 15:20:10,141][20381] Updated weights for policy 1, policy_version 1397157 (0.0006) [2023-12-27 15:20:10,196][20381] Updated weights for policy 1, policy_version 1397167 (0.0005) [2023-12-27 15:20:10,369][20347] Updated weights for policy 0, policy_version 1398257 (0.0008) [2023-12-27 15:20:10,429][20347] Updated weights for policy 0, policy_version 1398267 (0.0008) [2023-12-27 15:20:10,490][20347] Updated weights for policy 0, policy_version 1398277 (0.0008) [2023-12-27 15:20:10,871][20381] Updated weights for policy 1, policy_version 1397177 (0.0008) [2023-12-27 15:20:10,923][20381] Updated weights for policy 1, policy_version 1397187 (0.0007) [2023-12-27 15:20:10,969][20381] Updated weights for policy 1, policy_version 1397197 (0.0006) [2023-12-27 15:20:11,145][20347] Updated weights for policy 0, policy_version 1398287 (0.0008) [2023-12-27 15:20:11,209][20347] Updated weights for policy 0, policy_version 1398297 (0.0006) [2023-12-27 15:20:11,271][20347] Updated weights for policy 0, policy_version 1398307 (0.0006) [2023-12-27 15:20:11,795][20381] Updated weights for policy 1, policy_version 1397207 (0.0007) [2023-12-27 15:20:11,862][20381] Updated weights for policy 1, policy_version 1397217 (0.0006) [2023-12-27 15:20:11,923][20381] Updated weights for policy 1, policy_version 1397227 (0.0009) [2023-12-27 15:20:11,954][20347] Updated weights for policy 0, policy_version 1398317 (0.0008) [2023-12-27 15:20:12,016][20347] Updated weights for policy 0, policy_version 1398327 (0.0007) [2023-12-27 15:20:12,076][20347] Updated weights for policy 0, policy_version 1398337 (0.0008) [2023-12-27 15:20:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 715767808. Throughput: 0: 10472.3, 1: 10266.1. Samples: 715757272. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:12,363][19170] Avg episode reward: [(0, '980.500'), (1, '931.970')] [2023-12-27 15:20:12,520][20381] Updated weights for policy 1, policy_version 1397237 (0.0006) [2023-12-27 15:20:12,583][20381] Updated weights for policy 1, policy_version 1397247 (0.0006) [2023-12-27 15:20:12,647][20381] Updated weights for policy 1, policy_version 1397257 (0.0006) [2023-12-27 15:20:12,849][20347] Updated weights for policy 0, policy_version 1398347 (0.0008) [2023-12-27 15:20:12,898][20347] Updated weights for policy 0, policy_version 1398357 (0.0008) [2023-12-27 15:20:12,950][20347] Updated weights for policy 0, policy_version 1398367 (0.0008) [2023-12-27 15:20:13,276][20381] Updated weights for policy 1, policy_version 1397267 (0.0008) [2023-12-27 15:20:13,312][20292] KL-divergence is very high: 199.7712 [2023-12-27 15:20:13,335][20381] Updated weights for policy 1, policy_version 1397277 (0.0007) [2023-12-27 15:20:13,348][20292] KL-divergence is very high: 163.5475 [2023-12-27 15:20:13,381][20381] Updated weights for policy 1, policy_version 1397287 (0.0008) [2023-12-27 15:20:13,658][20347] Updated weights for policy 0, policy_version 1398377 (0.0007) [2023-12-27 15:20:13,716][20347] Updated weights for policy 0, policy_version 1398387 (0.0008) [2023-12-27 15:20:13,771][20347] Updated weights for policy 0, policy_version 1398397 (0.0008) [2023-12-27 15:20:13,834][20347] Updated weights for policy 0, policy_version 1398407 (0.0008) [2023-12-27 15:20:14,027][20381] Updated weights for policy 1, policy_version 1397297 (0.0008) [2023-12-27 15:20:14,083][20381] Updated weights for policy 1, policy_version 1397307 (0.0009) [2023-12-27 15:20:14,140][20381] Updated weights for policy 1, policy_version 1397317 (0.0011) [2023-12-27 15:20:14,201][20381] Updated weights for policy 1, policy_version 1397327 (0.0011) [2023-12-27 15:20:14,472][20347] Updated weights for policy 0, policy_version 1398417 (0.0009) [2023-12-27 15:20:14,522][20347] Updated weights for policy 0, policy_version 1398427 (0.0009) [2023-12-27 15:20:14,579][20347] Updated weights for policy 0, policy_version 1398437 (0.0010) [2023-12-27 15:20:14,901][20381] Updated weights for policy 1, policy_version 1397337 (0.0011) [2023-12-27 15:20:14,960][20381] Updated weights for policy 1, policy_version 1397347 (0.0011) [2023-12-27 15:20:15,023][20381] Updated weights for policy 1, policy_version 1397357 (0.0008) [2023-12-27 15:20:15,282][20347] Updated weights for policy 0, policy_version 1398447 (0.0010) [2023-12-27 15:20:15,334][20347] Updated weights for policy 0, policy_version 1398457 (0.0010) [2023-12-27 15:20:15,392][20347] Updated weights for policy 0, policy_version 1398467 (0.0010) [2023-12-27 15:20:15,669][20381] Updated weights for policy 1, policy_version 1397367 (0.0009) [2023-12-27 15:20:15,718][20381] Updated weights for policy 1, policy_version 1397377 (0.0010) [2023-12-27 15:20:15,766][20381] Updated weights for policy 1, policy_version 1397387 (0.0010) [2023-12-27 15:20:16,031][20347] Updated weights for policy 0, policy_version 1398477 (0.0010) [2023-12-27 15:20:16,095][20347] Updated weights for policy 0, policy_version 1398487 (0.0006) [2023-12-27 15:20:16,143][20347] Updated weights for policy 0, policy_version 1398497 (0.0006) [2023-12-27 15:20:16,388][20381] Updated weights for policy 1, policy_version 1397397 (0.0008) [2023-12-27 15:20:16,441][20381] Updated weights for policy 1, policy_version 1397407 (0.0006) [2023-12-27 15:20:16,490][20381] Updated weights for policy 1, policy_version 1397417 (0.0006) [2023-12-27 15:20:16,721][20347] Updated weights for policy 0, policy_version 1398507 (0.0010) [2023-12-27 15:20:16,772][20347] Updated weights for policy 0, policy_version 1398517 (0.0010) [2023-12-27 15:20:16,827][20347] Updated weights for policy 0, policy_version 1398527 (0.0010) [2023-12-27 15:20:17,144][20381] Updated weights for policy 1, policy_version 1397427 (0.0009) [2023-12-27 15:20:17,200][20381] Updated weights for policy 1, policy_version 1397437 (0.0005) [2023-12-27 15:20:17,253][20381] Updated weights for policy 1, policy_version 1397447 (0.0005) [2023-12-27 15:20:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 715874304. Throughput: 0: 10433.9, 1: 10289.2. Samples: 715882232. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:17,363][19170] Avg episode reward: [(0, '990.280'), (1, '902.450')] [2023-12-27 15:20:17,518][20347] Updated weights for policy 0, policy_version 1398537 (0.0010) [2023-12-27 15:20:17,572][20347] Updated weights for policy 0, policy_version 1398547 (0.0010) [2023-12-27 15:20:17,635][20347] Updated weights for policy 0, policy_version 1398557 (0.0008) [2023-12-27 15:20:17,699][20347] Updated weights for policy 0, policy_version 1398567 (0.0007) [2023-12-27 15:20:17,838][20381] Updated weights for policy 1, policy_version 1397457 (0.0005) [2023-12-27 15:20:17,884][20381] Updated weights for policy 1, policy_version 1397467 (0.0005) [2023-12-27 15:20:17,942][20381] Updated weights for policy 1, policy_version 1397477 (0.0011) [2023-12-27 15:20:18,002][20381] Updated weights for policy 1, policy_version 1397487 (0.0011) [2023-12-27 15:20:18,420][20347] Updated weights for policy 0, policy_version 1398577 (0.0007) [2023-12-27 15:20:18,477][20347] Updated weights for policy 0, policy_version 1398587 (0.0007) [2023-12-27 15:20:18,536][20347] Updated weights for policy 0, policy_version 1398597 (0.0007) [2023-12-27 15:20:18,699][20381] Updated weights for policy 1, policy_version 1397497 (0.0008) [2023-12-27 15:20:18,753][20381] Updated weights for policy 1, policy_version 1397507 (0.0008) [2023-12-27 15:20:18,810][20381] Updated weights for policy 1, policy_version 1397517 (0.0008) [2023-12-27 15:20:19,260][20347] Updated weights for policy 0, policy_version 1398607 (0.0007) [2023-12-27 15:20:19,319][20347] Updated weights for policy 0, policy_version 1398617 (0.0008) [2023-12-27 15:20:19,390][20347] Updated weights for policy 0, policy_version 1398627 (0.0008) [2023-12-27 15:20:19,487][20381] Updated weights for policy 1, policy_version 1397527 (0.0007) [2023-12-27 15:20:19,553][20381] Updated weights for policy 1, policy_version 1397537 (0.0009) [2023-12-27 15:20:19,613][20381] Updated weights for policy 1, policy_version 1397547 (0.0008) [2023-12-27 15:20:20,074][20347] Updated weights for policy 0, policy_version 1398637 (0.0009) [2023-12-27 15:20:20,132][20347] Updated weights for policy 0, policy_version 1398647 (0.0007) [2023-12-27 15:20:20,186][20347] Updated weights for policy 0, policy_version 1398657 (0.0006) [2023-12-27 15:20:20,376][20381] Updated weights for policy 1, policy_version 1397557 (0.0007) [2023-12-27 15:20:20,435][20381] Updated weights for policy 1, policy_version 1397567 (0.0007) [2023-12-27 15:20:20,495][20381] Updated weights for policy 1, policy_version 1397577 (0.0011) [2023-12-27 15:20:20,822][20347] Updated weights for policy 0, policy_version 1398667 (0.0006) [2023-12-27 15:20:20,886][20347] Updated weights for policy 0, policy_version 1398677 (0.0007) [2023-12-27 15:20:20,948][20347] Updated weights for policy 0, policy_version 1398687 (0.0008) [2023-12-27 15:20:21,226][20381] Updated weights for policy 1, policy_version 1397587 (0.0011) [2023-12-27 15:20:21,296][20381] Updated weights for policy 1, policy_version 1397597 (0.0010) [2023-12-27 15:20:21,372][20381] Updated weights for policy 1, policy_version 1397607 (0.0009) [2023-12-27 15:20:21,752][20347] Updated weights for policy 0, policy_version 1398697 (0.0008) [2023-12-27 15:20:21,811][20347] Updated weights for policy 0, policy_version 1398707 (0.0007) [2023-12-27 15:20:21,879][20347] Updated weights for policy 0, policy_version 1398717 (0.0007) [2023-12-27 15:20:21,939][20347] Updated weights for policy 0, policy_version 1398727 (0.0008) [2023-12-27 15:20:22,094][20381] Updated weights for policy 1, policy_version 1397617 (0.0010) [2023-12-27 15:20:22,144][20381] Updated weights for policy 1, policy_version 1397627 (0.0010) [2023-12-27 15:20:22,189][20381] Updated weights for policy 1, policy_version 1397637 (0.0010) [2023-12-27 15:20:22,245][20381] Updated weights for policy 1, policy_version 1397647 (0.0010) [2023-12-27 15:20:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 715972608. Throughput: 0: 10384.0, 1: 10224.4. Samples: 715941968. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:22,363][19170] Avg episode reward: [(0, '990.230'), (1, '951.200')] [2023-12-27 15:20:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001397648_357826560.pth... [2023-12-27 15:20:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001398728_358146048.pth... [2023-12-27 15:20:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001396432_357515264.pth [2023-12-27 15:20:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001397544_357842944.pth [2023-12-27 15:20:22,691][20347] Updated weights for policy 0, policy_version 1398737 (0.0007) [2023-12-27 15:20:22,752][20347] Updated weights for policy 0, policy_version 1398747 (0.0006) [2023-12-27 15:20:22,821][20347] Updated weights for policy 0, policy_version 1398757 (0.0010) [2023-12-27 15:20:22,976][20381] Updated weights for policy 1, policy_version 1397657 (0.0010) [2023-12-27 15:20:23,028][20381] Updated weights for policy 1, policy_version 1397667 (0.0007) [2023-12-27 15:20:23,080][20381] Updated weights for policy 1, policy_version 1397677 (0.0007) [2023-12-27 15:20:23,473][20347] Updated weights for policy 0, policy_version 1398767 (0.0009) [2023-12-27 15:20:23,529][20347] Updated weights for policy 0, policy_version 1398777 (0.0005) [2023-12-27 15:20:23,575][20347] Updated weights for policy 0, policy_version 1398787 (0.0005) [2023-12-27 15:20:23,779][20381] Updated weights for policy 1, policy_version 1397687 (0.0010) [2023-12-27 15:20:23,831][20381] Updated weights for policy 1, policy_version 1397697 (0.0010) [2023-12-27 15:20:23,875][20381] Updated weights for policy 1, policy_version 1397707 (0.0009) [2023-12-27 15:20:24,165][20347] Updated weights for policy 0, policy_version 1398797 (0.0007) [2023-12-27 15:20:24,212][20347] Updated weights for policy 0, policy_version 1398807 (0.0007) [2023-12-27 15:20:24,260][20347] Updated weights for policy 0, policy_version 1398817 (0.0010) [2023-12-27 15:20:24,568][20381] Updated weights for policy 1, policy_version 1397717 (0.0008) [2023-12-27 15:20:24,633][20381] Updated weights for policy 1, policy_version 1397727 (0.0010) [2023-12-27 15:20:24,693][20381] Updated weights for policy 1, policy_version 1397737 (0.0011) [2023-12-27 15:20:24,897][20347] Updated weights for policy 0, policy_version 1398827 (0.0009) [2023-12-27 15:20:24,949][20347] Updated weights for policy 0, policy_version 1398837 (0.0005) [2023-12-27 15:20:24,956][20291] KL-divergence is very high: 227.1643 [2023-12-27 15:20:24,997][20291] KL-divergence is very high: 222.2127 [2023-12-27 15:20:25,004][20347] Updated weights for policy 0, policy_version 1398847 (0.0008) [2023-12-27 15:20:25,038][20291] KL-divergence is very high: 137.4101 [2023-12-27 15:20:25,368][20381] Updated weights for policy 1, policy_version 1397747 (0.0009) [2023-12-27 15:20:25,423][20381] Updated weights for policy 1, policy_version 1397757 (0.0010) [2023-12-27 15:20:25,475][20381] Updated weights for policy 1, policy_version 1397767 (0.0010) [2023-12-27 15:20:25,557][20347] Updated weights for policy 0, policy_version 1398857 (0.0010) [2023-12-27 15:20:25,606][20347] Updated weights for policy 0, policy_version 1398867 (0.0010) [2023-12-27 15:20:25,662][20347] Updated weights for policy 0, policy_version 1398877 (0.0009) [2023-12-27 15:20:25,718][20347] Updated weights for policy 0, policy_version 1398887 (0.0005) [2023-12-27 15:20:26,189][20381] Updated weights for policy 1, policy_version 1397777 (0.0011) [2023-12-27 15:20:26,249][20381] Updated weights for policy 1, policy_version 1397787 (0.0010) [2023-12-27 15:20:26,303][20381] Updated weights for policy 1, policy_version 1397797 (0.0010) [2023-12-27 15:20:26,338][20347] Updated weights for policy 0, policy_version 1398897 (0.0010) [2023-12-27 15:20:26,356][20381] Updated weights for policy 1, policy_version 1397807 (0.0010) [2023-12-27 15:20:26,385][20347] Updated weights for policy 0, policy_version 1398907 (0.0010) [2023-12-27 15:20:26,442][20347] Updated weights for policy 0, policy_version 1398917 (0.0010) [2023-12-27 15:20:27,037][20381] Updated weights for policy 1, policy_version 1397817 (0.0010) [2023-12-27 15:20:27,091][20381] Updated weights for policy 1, policy_version 1397827 (0.0010) [2023-12-27 15:20:27,133][20347] Updated weights for policy 0, policy_version 1398927 (0.0010) [2023-12-27 15:20:27,145][20381] Updated weights for policy 1, policy_version 1397837 (0.0010) [2023-12-27 15:20:27,184][20347] Updated weights for policy 0, policy_version 1398937 (0.0010) [2023-12-27 15:20:27,232][20347] Updated weights for policy 0, policy_version 1398947 (0.0010) [2023-12-27 15:20:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 716079104. Throughput: 0: 10383.0, 1: 10196.9. Samples: 716064460. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:27,363][19170] Avg episode reward: [(0, '961.030'), (1, '980.480')] [2023-12-27 15:20:27,821][20381] Updated weights for policy 1, policy_version 1397847 (0.0011) [2023-12-27 15:20:27,855][20347] Updated weights for policy 0, policy_version 1398957 (0.0010) [2023-12-27 15:20:27,878][20381] Updated weights for policy 1, policy_version 1397857 (0.0011) [2023-12-27 15:20:27,902][20347] Updated weights for policy 0, policy_version 1398967 (0.0010) [2023-12-27 15:20:27,926][20381] Updated weights for policy 1, policy_version 1397867 (0.0010) [2023-12-27 15:20:27,950][20347] Updated weights for policy 0, policy_version 1398977 (0.0010) [2023-12-27 15:20:28,613][20347] Updated weights for policy 0, policy_version 1398987 (0.0009) [2023-12-27 15:20:28,623][20381] Updated weights for policy 1, policy_version 1397877 (0.0009) [2023-12-27 15:20:28,664][20347] Updated weights for policy 0, policy_version 1398997 (0.0010) [2023-12-27 15:20:28,681][20381] Updated weights for policy 1, policy_version 1397887 (0.0006) [2023-12-27 15:20:28,716][20347] Updated weights for policy 0, policy_version 1399007 (0.0010) [2023-12-27 15:20:28,739][20381] Updated weights for policy 1, policy_version 1397897 (0.0007) [2023-12-27 15:20:29,422][20347] Updated weights for policy 0, policy_version 1399017 (0.0010) [2023-12-27 15:20:29,438][20381] Updated weights for policy 1, policy_version 1397907 (0.0011) [2023-12-27 15:20:29,473][20347] Updated weights for policy 0, policy_version 1399027 (0.0005) [2023-12-27 15:20:29,496][20381] Updated weights for policy 1, policy_version 1397917 (0.0011) [2023-12-27 15:20:29,523][20347] Updated weights for policy 0, policy_version 1399037 (0.0009) [2023-12-27 15:20:29,552][20381] Updated weights for policy 1, policy_version 1397927 (0.0011) [2023-12-27 15:20:29,580][20347] Updated weights for policy 0, policy_version 1399047 (0.0006) [2023-12-27 15:20:30,219][20381] Updated weights for policy 1, policy_version 1397937 (0.0011) [2023-12-27 15:20:30,275][20381] Updated weights for policy 1, policy_version 1397947 (0.0011) [2023-12-27 15:20:30,326][20347] Updated weights for policy 0, policy_version 1399057 (0.0008) [2023-12-27 15:20:30,327][20381] Updated weights for policy 1, policy_version 1397957 (0.0010) [2023-12-27 15:20:30,377][20381] Updated weights for policy 1, policy_version 1397967 (0.0011) [2023-12-27 15:20:30,383][20347] Updated weights for policy 0, policy_version 1399067 (0.0008) [2023-12-27 15:20:30,445][20347] Updated weights for policy 0, policy_version 1399077 (0.0008) [2023-12-27 15:20:31,034][20347] Updated weights for policy 0, policy_version 1399087 (0.0007) [2023-12-27 15:20:31,069][20381] Updated weights for policy 1, policy_version 1397977 (0.0011) [2023-12-27 15:20:31,097][20347] Updated weights for policy 0, policy_version 1399097 (0.0009) [2023-12-27 15:20:31,131][20381] Updated weights for policy 1, policy_version 1397987 (0.0011) [2023-12-27 15:20:31,161][20347] Updated weights for policy 0, policy_version 1399107 (0.0008) [2023-12-27 15:20:31,196][20381] Updated weights for policy 1, policy_version 1397997 (0.0011) [2023-12-27 15:20:31,871][20347] Updated weights for policy 0, policy_version 1399117 (0.0008) [2023-12-27 15:20:31,896][20381] Updated weights for policy 1, policy_version 1398007 (0.0009) [2023-12-27 15:20:31,926][20347] Updated weights for policy 0, policy_version 1399127 (0.0007) [2023-12-27 15:20:31,951][20381] Updated weights for policy 1, policy_version 1398017 (0.0008) [2023-12-27 15:20:31,976][20347] Updated weights for policy 0, policy_version 1399137 (0.0006) [2023-12-27 15:20:32,004][20381] Updated weights for policy 1, policy_version 1398027 (0.0008) [2023-12-27 15:20:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 716177408. Throughput: 0: 10316.3, 1: 10154.4. Samples: 716186144. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:32,363][19170] Avg episode reward: [(0, '961.040'), (1, '980.480')] [2023-12-27 15:20:32,668][20347] Updated weights for policy 0, policy_version 1399147 (0.0006) [2023-12-27 15:20:32,728][20347] Updated weights for policy 0, policy_version 1399157 (0.0006) [2023-12-27 15:20:32,737][20381] Updated weights for policy 1, policy_version 1398037 (0.0007) [2023-12-27 15:20:32,791][20347] Updated weights for policy 0, policy_version 1399167 (0.0006) [2023-12-27 15:20:32,797][20381] Updated weights for policy 1, policy_version 1398047 (0.0008) [2023-12-27 15:20:32,859][20381] Updated weights for policy 1, policy_version 1398057 (0.0007) [2023-12-27 15:20:33,467][20347] Updated weights for policy 0, policy_version 1399177 (0.0006) [2023-12-27 15:20:33,492][20381] Updated weights for policy 1, policy_version 1398067 (0.0007) [2023-12-27 15:20:33,524][20347] Updated weights for policy 0, policy_version 1399187 (0.0007) [2023-12-27 15:20:33,550][20381] Updated weights for policy 1, policy_version 1398077 (0.0005) [2023-12-27 15:20:33,578][20347] Updated weights for policy 0, policy_version 1399197 (0.0010) [2023-12-27 15:20:33,608][20381] Updated weights for policy 1, policy_version 1398087 (0.0005) [2023-12-27 15:20:33,632][20347] Updated weights for policy 0, policy_version 1399207 (0.0010) [2023-12-27 15:20:34,213][20347] Updated weights for policy 0, policy_version 1399217 (0.0007) [2023-12-27 15:20:34,219][20381] Updated weights for policy 1, policy_version 1398097 (0.0005) [2023-12-27 15:20:34,268][20347] Updated weights for policy 0, policy_version 1399227 (0.0006) [2023-12-27 15:20:34,279][20381] Updated weights for policy 1, policy_version 1398107 (0.0008) [2023-12-27 15:20:34,324][20347] Updated weights for policy 0, policy_version 1399237 (0.0007) [2023-12-27 15:20:34,338][20381] Updated weights for policy 1, policy_version 1398117 (0.0008) [2023-12-27 15:20:34,396][20381] Updated weights for policy 1, policy_version 1398127 (0.0008) [2023-12-27 15:20:34,962][20291] KL-divergence is very high: 181.5620 [2023-12-27 15:20:34,979][20347] Updated weights for policy 0, policy_version 1399247 (0.0007) [2023-12-27 15:20:35,008][20291] KL-divergence is very high: 403.6840 [2023-12-27 15:20:35,036][20347] Updated weights for policy 0, policy_version 1399257 (0.0010) [2023-12-27 15:20:35,054][20291] KL-divergence is very high: 157.7211 [2023-12-27 15:20:35,066][20381] Updated weights for policy 1, policy_version 1398137 (0.0010) [2023-12-27 15:20:35,094][20347] Updated weights for policy 0, policy_version 1399267 (0.0010) [2023-12-27 15:20:35,123][20381] Updated weights for policy 1, policy_version 1398147 (0.0010) [2023-12-27 15:20:35,183][20381] Updated weights for policy 1, policy_version 1398157 (0.0010) [2023-12-27 15:20:35,759][20347] Updated weights for policy 0, policy_version 1399277 (0.0009) [2023-12-27 15:20:35,802][20347] Updated weights for policy 0, policy_version 1399287 (0.0007) [2023-12-27 15:20:35,844][20381] Updated weights for policy 1, policy_version 1398167 (0.0011) [2023-12-27 15:20:35,851][20347] Updated weights for policy 0, policy_version 1399297 (0.0007) [2023-12-27 15:20:35,896][20381] Updated weights for policy 1, policy_version 1398177 (0.0010) [2023-12-27 15:20:35,944][20381] Updated weights for policy 1, policy_version 1398187 (0.0010) [2023-12-27 15:20:36,543][20347] Updated weights for policy 0, policy_version 1399307 (0.0006) [2023-12-27 15:20:36,598][20347] Updated weights for policy 0, policy_version 1399317 (0.0007) [2023-12-27 15:20:36,656][20347] Updated weights for policy 0, policy_version 1399327 (0.0007) [2023-12-27 15:20:36,658][20381] Updated weights for policy 1, policy_version 1398197 (0.0010) [2023-12-27 15:20:36,718][20381] Updated weights for policy 1, policy_version 1398207 (0.0011) [2023-12-27 15:20:36,773][20381] Updated weights for policy 1, policy_version 1398217 (0.0011) [2023-12-27 15:20:37,352][20347] Updated weights for policy 0, policy_version 1399337 (0.0006) [2023-12-27 15:20:37,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 716275712. Throughput: 0: 10329.7, 1: 10170.3. Samples: 716249476. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:37,363][19170] Avg episode reward: [(0, '961.020'), (1, '892.560')] [2023-12-27 15:20:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001398224_357974016.pth... [2023-12-27 15:20:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001397040_357670912.pth [2023-12-27 15:20:37,403][20347] Updated weights for policy 0, policy_version 1399347 (0.0008) [2023-12-27 15:20:37,454][20347] Updated weights for policy 0, policy_version 1399357 (0.0007) [2023-12-27 15:20:37,465][20381] Updated weights for policy 1, policy_version 1398227 (0.0011) [2023-12-27 15:20:37,509][20347] Updated weights for policy 0, policy_version 1399367 (0.0006) [2023-12-27 15:20:37,512][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001399368_358309888.pth... [2023-12-27 15:20:37,515][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001398152_357998592.pth [2023-12-27 15:20:37,515][20381] Updated weights for policy 1, policy_version 1398237 (0.0011) [2023-12-27 15:20:37,561][20381] Updated weights for policy 1, policy_version 1398247 (0.0010) [2023-12-27 15:20:38,159][20347] Updated weights for policy 0, policy_version 1399377 (0.0010) [2023-12-27 15:20:38,207][20347] Updated weights for policy 0, policy_version 1399387 (0.0010) [2023-12-27 15:20:38,253][20347] Updated weights for policy 0, policy_version 1399397 (0.0009) [2023-12-27 15:20:38,265][20381] Updated weights for policy 1, policy_version 1398257 (0.0011) [2023-12-27 15:20:38,313][20381] Updated weights for policy 1, policy_version 1398267 (0.0010) [2023-12-27 15:20:38,372][20381] Updated weights for policy 1, policy_version 1398277 (0.0008) [2023-12-27 15:20:38,421][20381] Updated weights for policy 1, policy_version 1398287 (0.0011) [2023-12-27 15:20:38,974][20347] Updated weights for policy 0, policy_version 1399407 (0.0007) [2023-12-27 15:20:39,039][20347] Updated weights for policy 0, policy_version 1399417 (0.0008) [2023-12-27 15:20:39,090][20347] Updated weights for policy 0, policy_version 1399427 (0.0008) [2023-12-27 15:20:39,112][20381] Updated weights for policy 1, policy_version 1398297 (0.0011) [2023-12-27 15:20:39,163][20381] Updated weights for policy 1, policy_version 1398307 (0.0010) [2023-12-27 15:20:39,221][20381] Updated weights for policy 1, policy_version 1398317 (0.0011) [2023-12-27 15:20:39,796][20347] Updated weights for policy 0, policy_version 1399437 (0.0005) [2023-12-27 15:20:39,866][20347] Updated weights for policy 0, policy_version 1399447 (0.0009) [2023-12-27 15:20:39,923][20347] Updated weights for policy 0, policy_version 1399457 (0.0011) [2023-12-27 15:20:40,012][20381] Updated weights for policy 1, policy_version 1398327 (0.0011) [2023-12-27 15:20:40,075][20381] Updated weights for policy 1, policy_version 1398337 (0.0011) [2023-12-27 15:20:40,139][20381] Updated weights for policy 1, policy_version 1398347 (0.0011) [2023-12-27 15:20:40,580][20347] Updated weights for policy 0, policy_version 1399467 (0.0010) [2023-12-27 15:20:40,641][20347] Updated weights for policy 0, policy_version 1399477 (0.0010) [2023-12-27 15:20:40,706][20347] Updated weights for policy 0, policy_version 1399487 (0.0010) [2023-12-27 15:20:40,856][20381] Updated weights for policy 1, policy_version 1398357 (0.0011) [2023-12-27 15:20:40,913][20381] Updated weights for policy 1, policy_version 1398367 (0.0010) [2023-12-27 15:20:40,959][20381] Updated weights for policy 1, policy_version 1398377 (0.0010) [2023-12-27 15:20:41,378][20347] Updated weights for policy 0, policy_version 1399497 (0.0008) [2023-12-27 15:20:41,443][20347] Updated weights for policy 0, policy_version 1399507 (0.0011) [2023-12-27 15:20:41,505][20347] Updated weights for policy 0, policy_version 1399517 (0.0010) [2023-12-27 15:20:41,558][20347] Updated weights for policy 0, policy_version 1399527 (0.0010) [2023-12-27 15:20:41,681][20381] Updated weights for policy 1, policy_version 1398387 (0.0011) [2023-12-27 15:20:41,751][20381] Updated weights for policy 1, policy_version 1398397 (0.0011) [2023-12-27 15:20:41,804][20381] Updated weights for policy 1, policy_version 1398407 (0.0007) [2023-12-27 15:20:42,262][20347] Updated weights for policy 0, policy_version 1399537 (0.0010) [2023-12-27 15:20:42,316][20347] Updated weights for policy 0, policy_version 1399547 (0.0010) [2023-12-27 15:20:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 716374016. Throughput: 0: 10288.8, 1: 10120.5. Samples: 716369612. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:42,363][19170] Avg episode reward: [(0, '970.780'), (1, '863.400')] [2023-12-27 15:20:42,399][20347] Updated weights for policy 0, policy_version 1399557 (0.0010) [2023-12-27 15:20:42,552][20381] Updated weights for policy 1, policy_version 1398417 (0.0008) [2023-12-27 15:20:42,611][20381] Updated weights for policy 1, policy_version 1398427 (0.0011) [2023-12-27 15:20:42,673][20381] Updated weights for policy 1, policy_version 1398437 (0.0011) [2023-12-27 15:20:42,731][20381] Updated weights for policy 1, policy_version 1398447 (0.0009) [2023-12-27 15:20:43,088][20347] Updated weights for policy 0, policy_version 1399567 (0.0010) [2023-12-27 15:20:43,148][20347] Updated weights for policy 0, policy_version 1399577 (0.0010) [2023-12-27 15:20:43,202][20347] Updated weights for policy 0, policy_version 1399587 (0.0010) [2023-12-27 15:20:43,371][20381] Updated weights for policy 1, policy_version 1398457 (0.0010) [2023-12-27 15:20:43,419][20381] Updated weights for policy 1, policy_version 1398467 (0.0010) [2023-12-27 15:20:43,464][20381] Updated weights for policy 1, policy_version 1398477 (0.0010) [2023-12-27 15:20:43,867][20347] Updated weights for policy 0, policy_version 1399597 (0.0010) [2023-12-27 15:20:43,920][20347] Updated weights for policy 0, policy_version 1399607 (0.0010) [2023-12-27 15:20:43,971][20347] Updated weights for policy 0, policy_version 1399617 (0.0010) [2023-12-27 15:20:44,132][20381] Updated weights for policy 1, policy_version 1398487 (0.0010) [2023-12-27 15:20:44,191][20381] Updated weights for policy 1, policy_version 1398497 (0.0010) [2023-12-27 15:20:44,250][20381] Updated weights for policy 1, policy_version 1398507 (0.0010) [2023-12-27 15:20:44,613][20347] Updated weights for policy 0, policy_version 1399627 (0.0009) [2023-12-27 15:20:44,666][20347] Updated weights for policy 0, policy_version 1399637 (0.0005) [2023-12-27 15:20:44,715][20347] Updated weights for policy 0, policy_version 1399647 (0.0005) [2023-12-27 15:20:44,930][20381] Updated weights for policy 1, policy_version 1398517 (0.0011) [2023-12-27 15:20:44,993][20381] Updated weights for policy 1, policy_version 1398527 (0.0010) [2023-12-27 15:20:45,053][20381] Updated weights for policy 1, policy_version 1398537 (0.0011) [2023-12-27 15:20:45,309][20347] Updated weights for policy 0, policy_version 1399657 (0.0006) [2023-12-27 15:20:45,361][20347] Updated weights for policy 0, policy_version 1399667 (0.0005) [2023-12-27 15:20:45,427][20347] Updated weights for policy 0, policy_version 1399677 (0.0009) [2023-12-27 15:20:45,475][20347] Updated weights for policy 0, policy_version 1399687 (0.0010) [2023-12-27 15:20:45,701][20381] Updated weights for policy 1, policy_version 1398547 (0.0010) [2023-12-27 15:20:45,756][20381] Updated weights for policy 1, policy_version 1398557 (0.0010) [2023-12-27 15:20:45,809][20381] Updated weights for policy 1, policy_version 1398567 (0.0011) [2023-12-27 15:20:46,121][20347] Updated weights for policy 0, policy_version 1399697 (0.0009) [2023-12-27 15:20:46,174][20347] Updated weights for policy 0, policy_version 1399707 (0.0010) [2023-12-27 15:20:46,224][20347] Updated weights for policy 0, policy_version 1399717 (0.0007) [2023-12-27 15:20:46,491][20381] Updated weights for policy 1, policy_version 1398577 (0.0009) [2023-12-27 15:20:46,545][20381] Updated weights for policy 1, policy_version 1398587 (0.0007) [2023-12-27 15:20:46,589][20381] Updated weights for policy 1, policy_version 1398597 (0.0006) [2023-12-27 15:20:46,645][20381] Updated weights for policy 1, policy_version 1398607 (0.0007) [2023-12-27 15:20:46,814][20347] Updated weights for policy 0, policy_version 1399727 (0.0005) [2023-12-27 15:20:46,873][20347] Updated weights for policy 0, policy_version 1399737 (0.0005) [2023-12-27 15:20:46,919][20347] Updated weights for policy 0, policy_version 1399747 (0.0005) [2023-12-27 15:20:47,336][20381] Updated weights for policy 1, policy_version 1398617 (0.0010) [2023-12-27 15:20:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 716480512. Throughput: 0: 10330.7, 1: 10127.7. Samples: 716494492. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '922.100')] [2023-12-27 15:20:47,384][20381] Updated weights for policy 1, policy_version 1398627 (0.0010) [2023-12-27 15:20:47,436][20381] Updated weights for policy 1, policy_version 1398637 (0.0010) [2023-12-27 15:20:47,514][20347] Updated weights for policy 0, policy_version 1399757 (0.0006) [2023-12-27 15:20:47,569][20347] Updated weights for policy 0, policy_version 1399767 (0.0008) [2023-12-27 15:20:47,616][20347] Updated weights for policy 0, policy_version 1399777 (0.0007) [2023-12-27 15:20:48,122][20381] Updated weights for policy 1, policy_version 1398647 (0.0011) [2023-12-27 15:20:48,184][20381] Updated weights for policy 1, policy_version 1398657 (0.0011) [2023-12-27 15:20:48,247][20381] Updated weights for policy 1, policy_version 1398667 (0.0010) [2023-12-27 15:20:48,301][20347] Updated weights for policy 0, policy_version 1399787 (0.0007) [2023-12-27 15:20:48,368][20347] Updated weights for policy 0, policy_version 1399797 (0.0008) [2023-12-27 15:20:48,425][20347] Updated weights for policy 0, policy_version 1399807 (0.0007) [2023-12-27 15:20:48,892][20381] Updated weights for policy 1, policy_version 1398677 (0.0009) [2023-12-27 15:20:48,951][20381] Updated weights for policy 1, policy_version 1398687 (0.0005) [2023-12-27 15:20:49,010][20381] Updated weights for policy 1, policy_version 1398697 (0.0006) [2023-12-27 15:20:49,148][20347] Updated weights for policy 0, policy_version 1399817 (0.0008) [2023-12-27 15:20:49,197][20347] Updated weights for policy 0, policy_version 1399827 (0.0008) [2023-12-27 15:20:49,257][20347] Updated weights for policy 0, policy_version 1399837 (0.0008) [2023-12-27 15:20:49,317][20347] Updated weights for policy 0, policy_version 1399847 (0.0007) [2023-12-27 15:20:49,652][20381] Updated weights for policy 1, policy_version 1398707 (0.0008) [2023-12-27 15:20:49,713][20381] Updated weights for policy 1, policy_version 1398717 (0.0011) [2023-12-27 15:20:49,773][20381] Updated weights for policy 1, policy_version 1398727 (0.0011) [2023-12-27 15:20:50,008][20347] Updated weights for policy 0, policy_version 1399857 (0.0007) [2023-12-27 15:20:50,065][20347] Updated weights for policy 0, policy_version 1399867 (0.0009) [2023-12-27 15:20:50,120][20347] Updated weights for policy 0, policy_version 1399877 (0.0009) [2023-12-27 15:20:50,395][20381] Updated weights for policy 1, policy_version 1398737 (0.0011) [2023-12-27 15:20:50,444][20381] Updated weights for policy 1, policy_version 1398747 (0.0011) [2023-12-27 15:20:50,499][20381] Updated weights for policy 1, policy_version 1398757 (0.0010) [2023-12-27 15:20:50,550][20381] Updated weights for policy 1, policy_version 1398767 (0.0010) [2023-12-27 15:20:50,841][20347] Updated weights for policy 0, policy_version 1399887 (0.0008) [2023-12-27 15:20:50,901][20347] Updated weights for policy 0, policy_version 1399897 (0.0007) [2023-12-27 15:20:50,962][20347] Updated weights for policy 0, policy_version 1399907 (0.0007) [2023-12-27 15:20:51,294][20381] Updated weights for policy 1, policy_version 1398777 (0.0009) [2023-12-27 15:20:51,354][20381] Updated weights for policy 1, policy_version 1398787 (0.0010) [2023-12-27 15:20:51,423][20381] Updated weights for policy 1, policy_version 1398797 (0.0010) [2023-12-27 15:20:51,739][20347] Updated weights for policy 0, policy_version 1399917 (0.0008) [2023-12-27 15:20:51,798][20347] Updated weights for policy 0, policy_version 1399927 (0.0008) [2023-12-27 15:20:51,855][20347] Updated weights for policy 0, policy_version 1399937 (0.0008) [2023-12-27 15:20:52,141][20381] Updated weights for policy 1, policy_version 1398807 (0.0011) [2023-12-27 15:20:52,204][20381] Updated weights for policy 1, policy_version 1398817 (0.0010) [2023-12-27 15:20:52,264][20381] Updated weights for policy 1, policy_version 1398827 (0.0009) [2023-12-27 15:20:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 716587008. Throughput: 0: 10317.1, 1: 10114.7. Samples: 716556176. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.480')] [2023-12-27 15:20:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001398832_358129664.pth... [2023-12-27 15:20:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001399944_358457344.pth... [2023-12-27 15:20:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001397648_357826560.pth [2023-12-27 15:20:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001398728_358146048.pth [2023-12-27 15:20:52,636][20347] Updated weights for policy 0, policy_version 1399947 (0.0009) [2023-12-27 15:20:52,698][20347] Updated weights for policy 0, policy_version 1399957 (0.0008) [2023-12-27 15:20:52,752][20347] Updated weights for policy 0, policy_version 1399967 (0.0007) [2023-12-27 15:20:52,982][20381] Updated weights for policy 1, policy_version 1398837 (0.0011) [2023-12-27 15:20:53,039][20381] Updated weights for policy 1, policy_version 1398847 (0.0008) [2023-12-27 15:20:53,096][20381] Updated weights for policy 1, policy_version 1398857 (0.0008) [2023-12-27 15:20:53,493][20347] Updated weights for policy 0, policy_version 1399977 (0.0008) [2023-12-27 15:20:53,547][20347] Updated weights for policy 0, policy_version 1399987 (0.0009) [2023-12-27 15:20:53,595][20347] Updated weights for policy 0, policy_version 1399997 (0.0007) [2023-12-27 15:20:53,637][20347] Updated weights for policy 0, policy_version 1400007 (0.0007) [2023-12-27 15:20:53,698][20381] Updated weights for policy 1, policy_version 1398867 (0.0008) [2023-12-27 15:20:53,743][20381] Updated weights for policy 1, policy_version 1398877 (0.0005) [2023-12-27 15:20:53,792][20381] Updated weights for policy 1, policy_version 1398887 (0.0005) [2023-12-27 15:20:54,314][20347] Updated weights for policy 0, policy_version 1400017 (0.0008) [2023-12-27 15:20:54,371][20347] Updated weights for policy 0, policy_version 1400027 (0.0007) [2023-12-27 15:20:54,430][20347] Updated weights for policy 0, policy_version 1400037 (0.0008) [2023-12-27 15:20:54,460][20381] Updated weights for policy 1, policy_version 1398897 (0.0006) [2023-12-27 15:20:54,520][20381] Updated weights for policy 1, policy_version 1398907 (0.0006) [2023-12-27 15:20:54,579][20381] Updated weights for policy 1, policy_version 1398917 (0.0005) [2023-12-27 15:20:54,626][20381] Updated weights for policy 1, policy_version 1398927 (0.0005) [2023-12-27 15:20:55,063][20347] Updated weights for policy 0, policy_version 1400047 (0.0008) [2023-12-27 15:20:55,131][20347] Updated weights for policy 0, policy_version 1400057 (0.0008) [2023-12-27 15:20:55,155][20381] Updated weights for policy 1, policy_version 1398937 (0.0007) [2023-12-27 15:20:55,193][20347] Updated weights for policy 0, policy_version 1400067 (0.0008) [2023-12-27 15:20:55,218][20381] Updated weights for policy 1, policy_version 1398947 (0.0005) [2023-12-27 15:20:55,272][20381] Updated weights for policy 1, policy_version 1398957 (0.0006) [2023-12-27 15:20:55,786][20347] Updated weights for policy 0, policy_version 1400077 (0.0005) [2023-12-27 15:20:55,833][20381] Updated weights for policy 1, policy_version 1398967 (0.0005) [2023-12-27 15:20:55,842][20347] Updated weights for policy 0, policy_version 1400087 (0.0009) [2023-12-27 15:20:55,881][20381] Updated weights for policy 1, policy_version 1398977 (0.0005) [2023-12-27 15:20:55,890][20347] Updated weights for policy 0, policy_version 1400097 (0.0010) [2023-12-27 15:20:55,933][20381] Updated weights for policy 1, policy_version 1398987 (0.0007) [2023-12-27 15:20:56,539][20347] Updated weights for policy 0, policy_version 1400107 (0.0010) [2023-12-27 15:20:56,600][20347] Updated weights for policy 0, policy_version 1400117 (0.0007) [2023-12-27 15:20:56,628][20381] Updated weights for policy 1, policy_version 1398997 (0.0006) [2023-12-27 15:20:56,654][20347] Updated weights for policy 0, policy_version 1400127 (0.0007) [2023-12-27 15:20:56,677][20381] Updated weights for policy 1, policy_version 1399007 (0.0008) [2023-12-27 15:20:56,730][20381] Updated weights for policy 1, policy_version 1399017 (0.0007) [2023-12-27 15:20:57,327][20347] Updated weights for policy 0, policy_version 1400137 (0.0006) [2023-12-27 15:20:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 716685312. Throughput: 0: 10270.3, 1: 10219.6. Samples: 716679316. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:20:57,363][19170] Avg episode reward: [(0, '990.230'), (1, '922.440')] [2023-12-27 15:20:57,379][20347] Updated weights for policy 0, policy_version 1400147 (0.0008) [2023-12-27 15:20:57,392][20381] Updated weights for policy 1, policy_version 1399027 (0.0007) [2023-12-27 15:20:57,435][20347] Updated weights for policy 0, policy_version 1400157 (0.0007) [2023-12-27 15:20:57,441][20381] Updated weights for policy 1, policy_version 1399037 (0.0007) [2023-12-27 15:20:57,489][20381] Updated weights for policy 1, policy_version 1399047 (0.0009) [2023-12-27 15:20:57,493][20347] Updated weights for policy 0, policy_version 1400167 (0.0007) [2023-12-27 15:20:58,128][20381] Updated weights for policy 1, policy_version 1399057 (0.0007) [2023-12-27 15:20:58,175][20381] Updated weights for policy 1, policy_version 1399067 (0.0010) [2023-12-27 15:20:58,176][20347] Updated weights for policy 0, policy_version 1400177 (0.0006) [2023-12-27 15:20:58,236][20381] Updated weights for policy 1, policy_version 1399077 (0.0008) [2023-12-27 15:20:58,243][20347] Updated weights for policy 0, policy_version 1400187 (0.0007) [2023-12-27 15:20:58,303][20347] Updated weights for policy 0, policy_version 1400197 (0.0006) [2023-12-27 15:20:58,305][20381] Updated weights for policy 1, policy_version 1399087 (0.0008) [2023-12-27 15:20:59,062][20347] Updated weights for policy 0, policy_version 1400207 (0.0007) [2023-12-27 15:20:59,076][20381] Updated weights for policy 1, policy_version 1399097 (0.0007) [2023-12-27 15:20:59,130][20347] Updated weights for policy 0, policy_version 1400217 (0.0010) [2023-12-27 15:20:59,135][20381] Updated weights for policy 1, policy_version 1399107 (0.0007) [2023-12-27 15:20:59,187][20347] Updated weights for policy 0, policy_version 1400227 (0.0011) [2023-12-27 15:20:59,195][20381] Updated weights for policy 1, policy_version 1399117 (0.0007) [2023-12-27 15:20:59,889][20347] Updated weights for policy 0, policy_version 1400237 (0.0009) [2023-12-27 15:20:59,890][20381] Updated weights for policy 1, policy_version 1399127 (0.0008) [2023-12-27 15:20:59,948][20347] Updated weights for policy 0, policy_version 1400247 (0.0008) [2023-12-27 15:20:59,951][20381] Updated weights for policy 1, policy_version 1399137 (0.0007) [2023-12-27 15:20:59,996][20347] Updated weights for policy 0, policy_version 1400257 (0.0008) [2023-12-27 15:21:00,008][20381] Updated weights for policy 1, policy_version 1399147 (0.0007) [2023-12-27 15:21:00,662][20381] Updated weights for policy 1, policy_version 1399157 (0.0007) [2023-12-27 15:21:00,718][20381] Updated weights for policy 1, policy_version 1399167 (0.0007) [2023-12-27 15:21:00,736][20347] Updated weights for policy 0, policy_version 1400267 (0.0008) [2023-12-27 15:21:00,769][20381] Updated weights for policy 1, policy_version 1399177 (0.0008) [2023-12-27 15:21:00,788][20347] Updated weights for policy 0, policy_version 1400277 (0.0006) [2023-12-27 15:21:00,836][20347] Updated weights for policy 0, policy_version 1400287 (0.0009) [2023-12-27 15:21:01,421][20381] Updated weights for policy 1, policy_version 1399187 (0.0006) [2023-12-27 15:21:01,479][20381] Updated weights for policy 1, policy_version 1399197 (0.0008) [2023-12-27 15:21:01,536][20381] Updated weights for policy 1, policy_version 1399207 (0.0008) [2023-12-27 15:21:01,559][20347] Updated weights for policy 0, policy_version 1400297 (0.0009) [2023-12-27 15:21:01,620][20347] Updated weights for policy 0, policy_version 1400307 (0.0007) [2023-12-27 15:21:01,682][20347] Updated weights for policy 0, policy_version 1400317 (0.0007) [2023-12-27 15:21:01,741][20347] Updated weights for policy 0, policy_version 1400327 (0.0008) [2023-12-27 15:21:02,252][20381] Updated weights for policy 1, policy_version 1399217 (0.0006) [2023-12-27 15:21:02,309][20381] Updated weights for policy 1, policy_version 1399227 (0.0007) [2023-12-27 15:21:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 716783616. Throughput: 0: 10202.9, 1: 10166.4. Samples: 716798852. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:02,363][19170] Avg episode reward: [(0, '980.520'), (1, '922.520')] [2023-12-27 15:21:02,376][20381] Updated weights for policy 1, policy_version 1399237 (0.0007) [2023-12-27 15:21:02,438][20381] Updated weights for policy 1, policy_version 1399247 (0.0007) [2023-12-27 15:21:02,501][20347] Updated weights for policy 0, policy_version 1400337 (0.0008) [2023-12-27 15:21:02,559][20347] Updated weights for policy 0, policy_version 1400347 (0.0007) [2023-12-27 15:21:02,618][20347] Updated weights for policy 0, policy_version 1400357 (0.0008) [2023-12-27 15:21:03,077][20381] Updated weights for policy 1, policy_version 1399257 (0.0010) [2023-12-27 15:21:03,136][20381] Updated weights for policy 1, policy_version 1399267 (0.0008) [2023-12-27 15:21:03,196][20381] Updated weights for policy 1, policy_version 1399277 (0.0006) [2023-12-27 15:21:03,302][20347] Updated weights for policy 0, policy_version 1400367 (0.0007) [2023-12-27 15:21:03,313][20291] KL-divergence is very high: 772.5294 [2023-12-27 15:21:03,354][20291] KL-divergence is very high: 453.3466 [2023-12-27 15:21:03,355][20347] Updated weights for policy 0, policy_version 1400377 (0.0007) [2023-12-27 15:21:03,412][20347] Updated weights for policy 0, policy_version 1400387 (0.0006) [2023-12-27 15:21:03,909][20381] Updated weights for policy 1, policy_version 1399287 (0.0007) [2023-12-27 15:21:03,975][20381] Updated weights for policy 1, policy_version 1399297 (0.0007) [2023-12-27 15:21:04,036][20381] Updated weights for policy 1, policy_version 1399307 (0.0007) [2023-12-27 15:21:04,047][20347] Updated weights for policy 0, policy_version 1400397 (0.0007) [2023-12-27 15:21:04,100][20347] Updated weights for policy 0, policy_version 1400407 (0.0008) [2023-12-27 15:21:04,156][20347] Updated weights for policy 0, policy_version 1400417 (0.0008) [2023-12-27 15:21:04,704][20381] Updated weights for policy 1, policy_version 1399317 (0.0006) [2023-12-27 15:21:04,762][20381] Updated weights for policy 1, policy_version 1399327 (0.0005) [2023-12-27 15:21:04,823][20381] Updated weights for policy 1, policy_version 1399337 (0.0005) [2023-12-27 15:21:04,907][20347] Updated weights for policy 0, policy_version 1400427 (0.0008) [2023-12-27 15:21:04,955][20347] Updated weights for policy 0, policy_version 1400437 (0.0007) [2023-12-27 15:21:05,009][20347] Updated weights for policy 0, policy_version 1400447 (0.0007) [2023-12-27 15:21:05,434][20381] Updated weights for policy 1, policy_version 1399347 (0.0008) [2023-12-27 15:21:05,489][20381] Updated weights for policy 1, policy_version 1399357 (0.0007) [2023-12-27 15:21:05,538][20381] Updated weights for policy 1, policy_version 1399367 (0.0008) [2023-12-27 15:21:05,652][20347] Updated weights for policy 0, policy_version 1400457 (0.0006) [2023-12-27 15:21:05,704][20347] Updated weights for policy 0, policy_version 1400467 (0.0007) [2023-12-27 15:21:05,762][20347] Updated weights for policy 0, policy_version 1400477 (0.0008) [2023-12-27 15:21:05,816][20347] Updated weights for policy 0, policy_version 1400487 (0.0008) [2023-12-27 15:21:06,195][20381] Updated weights for policy 1, policy_version 1399377 (0.0008) [2023-12-27 15:21:06,259][20381] Updated weights for policy 1, policy_version 1399387 (0.0008) [2023-12-27 15:21:06,318][20381] Updated weights for policy 1, policy_version 1399397 (0.0008) [2023-12-27 15:21:06,382][20381] Updated weights for policy 1, policy_version 1399407 (0.0008) [2023-12-27 15:21:06,498][20347] Updated weights for policy 0, policy_version 1400497 (0.0007) [2023-12-27 15:21:06,556][20347] Updated weights for policy 0, policy_version 1400507 (0.0007) [2023-12-27 15:21:06,617][20347] Updated weights for policy 0, policy_version 1400517 (0.0006) [2023-12-27 15:21:07,035][20381] Updated weights for policy 1, policy_version 1399417 (0.0007) [2023-12-27 15:21:07,093][20381] Updated weights for policy 1, policy_version 1399427 (0.0007) [2023-12-27 15:21:07,144][20381] Updated weights for policy 1, policy_version 1399437 (0.0007) [2023-12-27 15:21:07,315][20347] Updated weights for policy 0, policy_version 1400527 (0.0007) [2023-12-27 15:21:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 716890112. Throughput: 0: 10201.6, 1: 10179.0. Samples: 716859096. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:07,363][19170] Avg episode reward: [(0, '980.530'), (1, '931.630')] [2023-12-27 15:21:07,367][20347] Updated weights for policy 0, policy_version 1400537 (0.0005) [2023-12-27 15:21:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001399440_358285312.pth... [2023-12-27 15:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001398224_357974016.pth [2023-12-27 15:21:07,420][20347] Updated weights for policy 0, policy_version 1400547 (0.0005) [2023-12-27 15:21:07,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001400552_358612992.pth... [2023-12-27 15:21:07,450][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001399368_358309888.pth [2023-12-27 15:21:07,791][20381] Updated weights for policy 1, policy_version 1399447 (0.0007) [2023-12-27 15:21:07,838][20381] Updated weights for policy 1, policy_version 1399457 (0.0008) [2023-12-27 15:21:07,891][20381] Updated weights for policy 1, policy_version 1399467 (0.0008) [2023-12-27 15:21:07,989][20347] Updated weights for policy 0, policy_version 1400557 (0.0007) [2023-12-27 15:21:08,043][20347] Updated weights for policy 0, policy_version 1400567 (0.0005) [2023-12-27 15:21:08,093][20347] Updated weights for policy 0, policy_version 1400577 (0.0007) [2023-12-27 15:21:08,567][20381] Updated weights for policy 1, policy_version 1399477 (0.0007) [2023-12-27 15:21:08,622][20381] Updated weights for policy 1, policy_version 1399487 (0.0008) [2023-12-27 15:21:08,680][20381] Updated weights for policy 1, policy_version 1399497 (0.0007) [2023-12-27 15:21:08,799][20347] Updated weights for policy 0, policy_version 1400587 (0.0007) [2023-12-27 15:21:08,867][20347] Updated weights for policy 0, policy_version 1400597 (0.0009) [2023-12-27 15:21:08,933][20347] Updated weights for policy 0, policy_version 1400607 (0.0010) [2023-12-27 15:21:09,317][20381] Updated weights for policy 1, policy_version 1399507 (0.0007) [2023-12-27 15:21:09,384][20381] Updated weights for policy 1, policy_version 1399517 (0.0008) [2023-12-27 15:21:09,452][20381] Updated weights for policy 1, policy_version 1399527 (0.0008) [2023-12-27 15:21:09,573][20347] Updated weights for policy 0, policy_version 1400617 (0.0008) [2023-12-27 15:21:09,625][20347] Updated weights for policy 0, policy_version 1400627 (0.0008) [2023-12-27 15:21:09,678][20347] Updated weights for policy 0, policy_version 1400637 (0.0006) [2023-12-27 15:21:09,735][20347] Updated weights for policy 0, policy_version 1400647 (0.0006) [2023-12-27 15:21:10,147][20381] Updated weights for policy 1, policy_version 1399537 (0.0008) [2023-12-27 15:21:10,207][20381] Updated weights for policy 1, policy_version 1399547 (0.0008) [2023-12-27 15:21:10,271][20381] Updated weights for policy 1, policy_version 1399557 (0.0008) [2023-12-27 15:21:10,334][20381] Updated weights for policy 1, policy_version 1399567 (0.0007) [2023-12-27 15:21:10,349][20347] Updated weights for policy 0, policy_version 1400657 (0.0007) [2023-12-27 15:21:10,410][20347] Updated weights for policy 0, policy_version 1400667 (0.0007) [2023-12-27 15:21:10,461][20347] Updated weights for policy 0, policy_version 1400677 (0.0005) [2023-12-27 15:21:10,929][20381] Updated weights for policy 1, policy_version 1399577 (0.0010) [2023-12-27 15:21:10,993][20381] Updated weights for policy 1, policy_version 1399587 (0.0009) [2023-12-27 15:21:11,045][20347] Updated weights for policy 0, policy_version 1400687 (0.0006) [2023-12-27 15:21:11,054][20381] Updated weights for policy 1, policy_version 1399597 (0.0009) [2023-12-27 15:21:11,106][20347] Updated weights for policy 0, policy_version 1400697 (0.0007) [2023-12-27 15:21:11,166][20347] Updated weights for policy 0, policy_version 1400707 (0.0008) [2023-12-27 15:21:11,811][20381] Updated weights for policy 1, policy_version 1399607 (0.0008) [2023-12-27 15:21:11,872][20381] Updated weights for policy 1, policy_version 1399617 (0.0007) [2023-12-27 15:21:11,931][20381] Updated weights for policy 1, policy_version 1399627 (0.0006) [2023-12-27 15:21:11,959][20347] Updated weights for policy 0, policy_version 1400717 (0.0008) [2023-12-27 15:21:12,020][20347] Updated weights for policy 0, policy_version 1400727 (0.0006) [2023-12-27 15:21:12,076][20347] Updated weights for policy 0, policy_version 1400737 (0.0005) [2023-12-27 15:21:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 716996608. Throughput: 0: 10233.1, 1: 10245.8. Samples: 716986012. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:12,363][19170] Avg episode reward: [(0, '990.240'), (1, '961.060')] [2023-12-27 15:21:12,641][20381] Updated weights for policy 1, policy_version 1399637 (0.0007) [2023-12-27 15:21:12,695][20381] Updated weights for policy 1, policy_version 1399647 (0.0008) [2023-12-27 15:21:12,735][20347] Updated weights for policy 0, policy_version 1400747 (0.0006) [2023-12-27 15:21:12,753][20381] Updated weights for policy 1, policy_version 1399657 (0.0008) [2023-12-27 15:21:12,802][20347] Updated weights for policy 0, policy_version 1400757 (0.0007) [2023-12-27 15:21:12,868][20347] Updated weights for policy 0, policy_version 1400767 (0.0005) [2023-12-27 15:21:13,443][20381] Updated weights for policy 1, policy_version 1399667 (0.0008) [2023-12-27 15:21:13,495][20381] Updated weights for policy 1, policy_version 1399677 (0.0010) [2023-12-27 15:21:13,532][20347] Updated weights for policy 0, policy_version 1400777 (0.0007) [2023-12-27 15:21:13,547][20381] Updated weights for policy 1, policy_version 1399687 (0.0011) [2023-12-27 15:21:13,583][20347] Updated weights for policy 0, policy_version 1400787 (0.0007) [2023-12-27 15:21:13,630][20347] Updated weights for policy 0, policy_version 1400797 (0.0006) [2023-12-27 15:21:13,675][20347] Updated weights for policy 0, policy_version 1400807 (0.0005) [2023-12-27 15:21:14,221][20381] Updated weights for policy 1, policy_version 1399697 (0.0010) [2023-12-27 15:21:14,256][20347] Updated weights for policy 0, policy_version 1400817 (0.0007) [2023-12-27 15:21:14,269][20381] Updated weights for policy 1, policy_version 1399707 (0.0010) [2023-12-27 15:21:14,313][20347] Updated weights for policy 0, policy_version 1400827 (0.0006) [2023-12-27 15:21:14,326][20381] Updated weights for policy 1, policy_version 1399717 (0.0008) [2023-12-27 15:21:14,364][20347] Updated weights for policy 0, policy_version 1400837 (0.0005) [2023-12-27 15:21:14,374][20381] Updated weights for policy 1, policy_version 1399727 (0.0005) [2023-12-27 15:21:14,960][20347] Updated weights for policy 0, policy_version 1400847 (0.0006) [2023-12-27 15:21:14,976][20381] Updated weights for policy 1, policy_version 1399737 (0.0008) [2023-12-27 15:21:15,013][20347] Updated weights for policy 0, policy_version 1400857 (0.0007) [2023-12-27 15:21:15,025][20381] Updated weights for policy 1, policy_version 1399747 (0.0008) [2023-12-27 15:21:15,070][20347] Updated weights for policy 0, policy_version 1400867 (0.0007) [2023-12-27 15:21:15,077][20381] Updated weights for policy 1, policy_version 1399757 (0.0008) [2023-12-27 15:21:15,777][20381] Updated weights for policy 1, policy_version 1399767 (0.0007) [2023-12-27 15:21:15,778][20347] Updated weights for policy 0, policy_version 1400877 (0.0006) [2023-12-27 15:21:15,832][20381] Updated weights for policy 1, policy_version 1399777 (0.0006) [2023-12-27 15:21:15,839][20347] Updated weights for policy 0, policy_version 1400887 (0.0007) [2023-12-27 15:21:15,889][20381] Updated weights for policy 1, policy_version 1399787 (0.0008) [2023-12-27 15:21:15,894][20347] Updated weights for policy 0, policy_version 1400897 (0.0010) [2023-12-27 15:21:16,528][20381] Updated weights for policy 1, policy_version 1399797 (0.0009) [2023-12-27 15:21:16,549][20347] Updated weights for policy 0, policy_version 1400907 (0.0010) [2023-12-27 15:21:16,579][20381] Updated weights for policy 1, policy_version 1399807 (0.0006) [2023-12-27 15:21:16,612][20347] Updated weights for policy 0, policy_version 1400917 (0.0010) [2023-12-27 15:21:16,627][20381] Updated weights for policy 1, policy_version 1399817 (0.0006) [2023-12-27 15:21:16,670][20347] Updated weights for policy 0, policy_version 1400927 (0.0010) [2023-12-27 15:21:17,226][20381] Updated weights for policy 1, policy_version 1399827 (0.0006) [2023-12-27 15:21:17,280][20381] Updated weights for policy 1, policy_version 1399837 (0.0008) [2023-12-27 15:21:17,331][20381] Updated weights for policy 1, policy_version 1399847 (0.0007) [2023-12-27 15:21:17,350][20347] Updated weights for policy 0, policy_version 1400937 (0.0010) [2023-12-27 15:21:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 717094912. Throughput: 0: 10248.2, 1: 10308.7. Samples: 717111204. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.320')] [2023-12-27 15:21:17,396][20347] Updated weights for policy 0, policy_version 1400947 (0.0010) [2023-12-27 15:21:17,441][20347] Updated weights for policy 0, policy_version 1400957 (0.0010) [2023-12-27 15:21:17,489][20347] Updated weights for policy 0, policy_version 1400967 (0.0005) [2023-12-27 15:21:18,074][20381] Updated weights for policy 1, policy_version 1399857 (0.0007) [2023-12-27 15:21:18,078][20347] Updated weights for policy 0, policy_version 1400977 (0.0005) [2023-12-27 15:21:18,119][20381] Updated weights for policy 1, policy_version 1399867 (0.0008) [2023-12-27 15:21:18,127][20347] Updated weights for policy 0, policy_version 1400987 (0.0005) [2023-12-27 15:21:18,174][20381] Updated weights for policy 1, policy_version 1399877 (0.0007) [2023-12-27 15:21:18,186][20347] Updated weights for policy 0, policy_version 1400997 (0.0008) [2023-12-27 15:21:18,230][20381] Updated weights for policy 1, policy_version 1399887 (0.0008) [2023-12-27 15:21:18,887][20381] Updated weights for policy 1, policy_version 1399897 (0.0008) [2023-12-27 15:21:18,898][20347] Updated weights for policy 0, policy_version 1401007 (0.0009) [2023-12-27 15:21:18,945][20381] Updated weights for policy 1, policy_version 1399907 (0.0007) [2023-12-27 15:21:18,953][20347] Updated weights for policy 0, policy_version 1401017 (0.0010) [2023-12-27 15:21:19,004][20347] Updated weights for policy 0, policy_version 1401027 (0.0010) [2023-12-27 15:21:19,006][20381] Updated weights for policy 1, policy_version 1399917 (0.0006) [2023-12-27 15:21:19,723][20381] Updated weights for policy 1, policy_version 1399927 (0.0009) [2023-12-27 15:21:19,726][20347] Updated weights for policy 0, policy_version 1401037 (0.0010) [2023-12-27 15:21:19,777][20347] Updated weights for policy 0, policy_version 1401047 (0.0010) [2023-12-27 15:21:19,782][20381] Updated weights for policy 1, policy_version 1399937 (0.0011) [2023-12-27 15:21:19,831][20347] Updated weights for policy 0, policy_version 1401057 (0.0010) [2023-12-27 15:21:19,849][20381] Updated weights for policy 1, policy_version 1399947 (0.0011) [2023-12-27 15:21:20,533][20347] Updated weights for policy 0, policy_version 1401067 (0.0010) [2023-12-27 15:21:20,567][20381] Updated weights for policy 1, policy_version 1399957 (0.0009) [2023-12-27 15:21:20,598][20347] Updated weights for policy 0, policy_version 1401077 (0.0008) [2023-12-27 15:21:20,629][20381] Updated weights for policy 1, policy_version 1399967 (0.0007) [2023-12-27 15:21:20,654][20347] Updated weights for policy 0, policy_version 1401087 (0.0007) [2023-12-27 15:21:20,681][20381] Updated weights for policy 1, policy_version 1399977 (0.0007) [2023-12-27 15:21:21,274][20347] Updated weights for policy 0, policy_version 1401097 (0.0006) [2023-12-27 15:21:21,341][20347] Updated weights for policy 0, policy_version 1401107 (0.0008) [2023-12-27 15:21:21,413][20347] Updated weights for policy 0, policy_version 1401117 (0.0009) [2023-12-27 15:21:21,474][20347] Updated weights for policy 0, policy_version 1401127 (0.0007) [2023-12-27 15:21:21,476][20381] Updated weights for policy 1, policy_version 1399987 (0.0009) [2023-12-27 15:21:21,539][20381] Updated weights for policy 1, policy_version 1399997 (0.0007) [2023-12-27 15:21:21,600][20381] Updated weights for policy 1, policy_version 1400007 (0.0009) [2023-12-27 15:21:22,197][20347] Updated weights for policy 0, policy_version 1401137 (0.0007) [2023-12-27 15:21:22,250][20347] Updated weights for policy 0, policy_version 1401147 (0.0008) [2023-12-27 15:21:22,276][20381] Updated weights for policy 1, policy_version 1400017 (0.0008) [2023-12-27 15:21:22,323][20347] Updated weights for policy 0, policy_version 1401157 (0.0007) [2023-12-27 15:21:22,335][20381] Updated weights for policy 1, policy_version 1400027 (0.0008) [2023-12-27 15:21:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 717201408. Throughput: 0: 10235.7, 1: 10262.9. Samples: 717171908. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.870')] [2023-12-27 15:21:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001401160_358768640.pth... [2023-12-27 15:21:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001399944_358457344.pth [2023-12-27 15:21:22,403][20381] Updated weights for policy 1, policy_version 1400037 (0.0008) [2023-12-27 15:21:22,463][20381] Updated weights for policy 1, policy_version 1400047 (0.0007) [2023-12-27 15:21:22,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001400048_358440960.pth... [2023-12-27 15:21:22,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001398832_358129664.pth [2023-12-27 15:21:23,034][20347] Updated weights for policy 0, policy_version 1401167 (0.0007) [2023-12-27 15:21:23,081][20347] Updated weights for policy 0, policy_version 1401177 (0.0007) [2023-12-27 15:21:23,139][20347] Updated weights for policy 0, policy_version 1401187 (0.0007) [2023-12-27 15:21:23,149][20381] Updated weights for policy 1, policy_version 1400057 (0.0008) [2023-12-27 15:21:23,208][20381] Updated weights for policy 1, policy_version 1400067 (0.0007) [2023-12-27 15:21:23,267][20381] Updated weights for policy 1, policy_version 1400077 (0.0011) [2023-12-27 15:21:23,768][20347] Updated weights for policy 0, policy_version 1401197 (0.0006) [2023-12-27 15:21:23,824][20347] Updated weights for policy 0, policy_version 1401207 (0.0007) [2023-12-27 15:21:23,883][20347] Updated weights for policy 0, policy_version 1401217 (0.0007) [2023-12-27 15:21:23,953][20381] Updated weights for policy 1, policy_version 1400087 (0.0009) [2023-12-27 15:21:24,011][20381] Updated weights for policy 1, policy_version 1400097 (0.0006) [2023-12-27 15:21:24,061][20381] Updated weights for policy 1, policy_version 1400107 (0.0011) [2023-12-27 15:21:24,539][20347] Updated weights for policy 0, policy_version 1401227 (0.0007) [2023-12-27 15:21:24,595][20347] Updated weights for policy 0, policy_version 1401237 (0.0007) [2023-12-27 15:21:24,652][20347] Updated weights for policy 0, policy_version 1401247 (0.0007) [2023-12-27 15:21:24,726][20381] Updated weights for policy 1, policy_version 1400117 (0.0009) [2023-12-27 15:21:24,771][20381] Updated weights for policy 1, policy_version 1400127 (0.0011) [2023-12-27 15:21:24,823][20381] Updated weights for policy 1, policy_version 1400137 (0.0010) [2023-12-27 15:21:25,347][20347] Updated weights for policy 0, policy_version 1401257 (0.0007) [2023-12-27 15:21:25,397][20347] Updated weights for policy 0, policy_version 1401267 (0.0008) [2023-12-27 15:21:25,445][20347] Updated weights for policy 0, policy_version 1401277 (0.0009) [2023-12-27 15:21:25,464][20381] Updated weights for policy 1, policy_version 1400147 (0.0009) [2023-12-27 15:21:25,496][20347] Updated weights for policy 0, policy_version 1401287 (0.0007) [2023-12-27 15:21:25,520][20381] Updated weights for policy 1, policy_version 1400157 (0.0005) [2023-12-27 15:21:25,576][20381] Updated weights for policy 1, policy_version 1400167 (0.0006) [2023-12-27 15:21:26,211][20381] Updated weights for policy 1, policy_version 1400177 (0.0006) [2023-12-27 15:21:26,237][20347] Updated weights for policy 0, policy_version 1401297 (0.0005) [2023-12-27 15:21:26,253][20381] Updated weights for policy 1, policy_version 1400187 (0.0010) [2023-12-27 15:21:26,286][20347] Updated weights for policy 0, policy_version 1401307 (0.0005) [2023-12-27 15:21:26,299][20381] Updated weights for policy 1, policy_version 1400197 (0.0010) [2023-12-27 15:21:26,337][20347] Updated weights for policy 0, policy_version 1401317 (0.0007) [2023-12-27 15:21:26,345][20381] Updated weights for policy 1, policy_version 1400207 (0.0010) [2023-12-27 15:21:26,966][20381] Updated weights for policy 1, policy_version 1400217 (0.0011) [2023-12-27 15:21:26,979][20347] Updated weights for policy 0, policy_version 1401327 (0.0007) [2023-12-27 15:21:27,011][20381] Updated weights for policy 1, policy_version 1400227 (0.0010) [2023-12-27 15:21:27,032][20347] Updated weights for policy 0, policy_version 1401337 (0.0006) [2023-12-27 15:21:27,063][20381] Updated weights for policy 1, policy_version 1400237 (0.0010) [2023-12-27 15:21:27,091][20347] Updated weights for policy 0, policy_version 1401347 (0.0006) [2023-12-27 15:21:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 717307904. Throughput: 0: 10233.0, 1: 10301.8. Samples: 717293676. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.460')] [2023-12-27 15:21:27,695][20381] Updated weights for policy 1, policy_version 1400247 (0.0008) [2023-12-27 15:21:27,758][20381] Updated weights for policy 1, policy_version 1400257 (0.0006) [2023-12-27 15:21:27,799][20347] Updated weights for policy 0, policy_version 1401357 (0.0006) [2023-12-27 15:21:27,812][20381] Updated weights for policy 1, policy_version 1400267 (0.0010) [2023-12-27 15:21:27,848][20347] Updated weights for policy 0, policy_version 1401367 (0.0006) [2023-12-27 15:21:27,894][20347] Updated weights for policy 0, policy_version 1401377 (0.0007) [2023-12-27 15:21:28,480][20381] Updated weights for policy 1, policy_version 1400277 (0.0011) [2023-12-27 15:21:28,543][20381] Updated weights for policy 1, policy_version 1400287 (0.0011) [2023-12-27 15:21:28,598][20347] Updated weights for policy 0, policy_version 1401387 (0.0006) [2023-12-27 15:21:28,603][20381] Updated weights for policy 1, policy_version 1400297 (0.0011) [2023-12-27 15:21:28,653][20347] Updated weights for policy 0, policy_version 1401397 (0.0005) [2023-12-27 15:21:28,713][20347] Updated weights for policy 0, policy_version 1401407 (0.0007) [2023-12-27 15:21:29,240][20381] Updated weights for policy 1, policy_version 1400307 (0.0011) [2023-12-27 15:21:29,302][20381] Updated weights for policy 1, policy_version 1400317 (0.0008) [2023-12-27 15:21:29,372][20381] Updated weights for policy 1, policy_version 1400327 (0.0009) [2023-12-27 15:21:29,385][20347] Updated weights for policy 0, policy_version 1401417 (0.0005) [2023-12-27 15:21:29,435][20347] Updated weights for policy 0, policy_version 1401427 (0.0007) [2023-12-27 15:21:29,483][20347] Updated weights for policy 0, policy_version 1401437 (0.0008) [2023-12-27 15:21:29,531][20347] Updated weights for policy 0, policy_version 1401447 (0.0008) [2023-12-27 15:21:30,034][20381] Updated weights for policy 1, policy_version 1400337 (0.0010) [2023-12-27 15:21:30,095][20381] Updated weights for policy 1, policy_version 1400347 (0.0010) [2023-12-27 15:21:30,156][20381] Updated weights for policy 1, policy_version 1400357 (0.0008) [2023-12-27 15:21:30,211][20381] Updated weights for policy 1, policy_version 1400367 (0.0007) [2023-12-27 15:21:30,245][20347] Updated weights for policy 0, policy_version 1401457 (0.0007) [2023-12-27 15:21:30,290][20291] KL-divergence is very high: 123.7317 [2023-12-27 15:21:30,300][20347] Updated weights for policy 0, policy_version 1401467 (0.0008) [2023-12-27 15:21:30,357][20347] Updated weights for policy 0, policy_version 1401477 (0.0006) [2023-12-27 15:21:30,843][20381] Updated weights for policy 1, policy_version 1400377 (0.0007) [2023-12-27 15:21:30,898][20381] Updated weights for policy 1, policy_version 1400387 (0.0009) [2023-12-27 15:21:30,954][20381] Updated weights for policy 1, policy_version 1400397 (0.0010) [2023-12-27 15:21:30,968][20347] Updated weights for policy 0, policy_version 1401487 (0.0006) [2023-12-27 15:21:31,020][20347] Updated weights for policy 0, policy_version 1401497 (0.0007) [2023-12-27 15:21:31,078][20347] Updated weights for policy 0, policy_version 1401507 (0.0009) [2023-12-27 15:21:31,677][20381] Updated weights for policy 1, policy_version 1400407 (0.0007) [2023-12-27 15:21:31,746][20381] Updated weights for policy 1, policy_version 1400417 (0.0007) [2023-12-27 15:21:31,805][20381] Updated weights for policy 1, policy_version 1400427 (0.0008) [2023-12-27 15:21:31,850][20347] Updated weights for policy 0, policy_version 1401517 (0.0007) [2023-12-27 15:21:31,907][20347] Updated weights for policy 0, policy_version 1401527 (0.0008) [2023-12-27 15:21:31,960][20347] Updated weights for policy 0, policy_version 1401537 (0.0007) [2023-12-27 15:21:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 717406208. Throughput: 0: 10166.9, 1: 10342.6. Samples: 717417416. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:32,363][19170] Avg episode reward: [(0, '980.420'), (1, '961.060')] [2023-12-27 15:21:32,456][20381] Updated weights for policy 1, policy_version 1400437 (0.0008) [2023-12-27 15:21:32,520][20381] Updated weights for policy 1, policy_version 1400447 (0.0007) [2023-12-27 15:21:32,585][20381] Updated weights for policy 1, policy_version 1400457 (0.0008) [2023-12-27 15:21:32,639][20347] Updated weights for policy 0, policy_version 1401547 (0.0007) [2023-12-27 15:21:32,699][20347] Updated weights for policy 0, policy_version 1401557 (0.0008) [2023-12-27 15:21:32,758][20347] Updated weights for policy 0, policy_version 1401567 (0.0008) [2023-12-27 15:21:33,235][20381] Updated weights for policy 1, policy_version 1400467 (0.0007) [2023-12-27 15:21:33,290][20381] Updated weights for policy 1, policy_version 1400477 (0.0006) [2023-12-27 15:21:33,360][20381] Updated weights for policy 1, policy_version 1400487 (0.0006) [2023-12-27 15:21:33,452][20347] Updated weights for policy 0, policy_version 1401577 (0.0008) [2023-12-27 15:21:33,507][20347] Updated weights for policy 0, policy_version 1401587 (0.0007) [2023-12-27 15:21:33,566][20347] Updated weights for policy 0, policy_version 1401597 (0.0008) [2023-12-27 15:21:33,619][20347] Updated weights for policy 0, policy_version 1401607 (0.0010) [2023-12-27 15:21:33,990][20381] Updated weights for policy 1, policy_version 1400497 (0.0007) [2023-12-27 15:21:34,044][20381] Updated weights for policy 1, policy_version 1400507 (0.0007) [2023-12-27 15:21:34,094][20381] Updated weights for policy 1, policy_version 1400517 (0.0007) [2023-12-27 15:21:34,148][20381] Updated weights for policy 1, policy_version 1400527 (0.0008) [2023-12-27 15:21:34,333][20347] Updated weights for policy 0, policy_version 1401617 (0.0005) [2023-12-27 15:21:34,388][20347] Updated weights for policy 0, policy_version 1401627 (0.0006) [2023-12-27 15:21:34,451][20347] Updated weights for policy 0, policy_version 1401637 (0.0007) [2023-12-27 15:21:34,890][20381] Updated weights for policy 1, policy_version 1400537 (0.0008) [2023-12-27 15:21:34,940][20381] Updated weights for policy 1, policy_version 1400547 (0.0008) [2023-12-27 15:21:34,999][20381] Updated weights for policy 1, policy_version 1400557 (0.0008) [2023-12-27 15:21:35,058][20347] Updated weights for policy 0, policy_version 1401647 (0.0008) [2023-12-27 15:21:35,119][20347] Updated weights for policy 0, policy_version 1401657 (0.0007) [2023-12-27 15:21:35,173][20347] Updated weights for policy 0, policy_version 1401667 (0.0007) [2023-12-27 15:21:35,681][20381] Updated weights for policy 1, policy_version 1400567 (0.0006) [2023-12-27 15:21:35,688][20292] KL-divergence is very high: 165.1826 [2023-12-27 15:21:35,735][20292] KL-divergence is very high: 211.8740 [2023-12-27 15:21:35,741][20381] Updated weights for policy 1, policy_version 1400577 (0.0007) [2023-12-27 15:21:35,781][20292] KL-divergence is very high: 127.9777 [2023-12-27 15:21:35,799][20381] Updated weights for policy 1, policy_version 1400587 (0.0008) [2023-12-27 15:21:35,803][20347] Updated weights for policy 0, policy_version 1401677 (0.0005) [2023-12-27 15:21:35,857][20347] Updated weights for policy 0, policy_version 1401687 (0.0005) [2023-12-27 15:21:35,908][20347] Updated weights for policy 0, policy_version 1401697 (0.0005) [2023-12-27 15:21:36,461][20381] Updated weights for policy 1, policy_version 1400597 (0.0007) [2023-12-27 15:21:36,520][20381] Updated weights for policy 1, policy_version 1400607 (0.0006) [2023-12-27 15:21:36,579][20381] Updated weights for policy 1, policy_version 1400617 (0.0007) [2023-12-27 15:21:36,596][20347] Updated weights for policy 0, policy_version 1401707 (0.0005) [2023-12-27 15:21:36,657][20347] Updated weights for policy 0, policy_version 1401717 (0.0005) [2023-12-27 15:21:36,722][20347] Updated weights for policy 0, policy_version 1401727 (0.0006) [2023-12-27 15:21:37,205][20381] Updated weights for policy 1, policy_version 1400627 (0.0008) [2023-12-27 15:21:37,260][20381] Updated weights for policy 1, policy_version 1400637 (0.0008) [2023-12-27 15:21:37,323][20381] Updated weights for policy 1, policy_version 1400647 (0.0008) [2023-12-27 15:21:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 717504512. Throughput: 0: 10189.3, 1: 10316.8. Samples: 717478952. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:37,363][19170] Avg episode reward: [(0, '980.420'), (1, '921.880')] [2023-12-27 15:21:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001400656_358596608.pth... [2023-12-27 15:21:37,376][20347] Updated weights for policy 0, policy_version 1401737 (0.0006) [2023-12-27 15:21:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001399440_358285312.pth [2023-12-27 15:21:37,425][20347] Updated weights for policy 0, policy_version 1401747 (0.0007) [2023-12-27 15:21:37,474][20347] Updated weights for policy 0, policy_version 1401757 (0.0008) [2023-12-27 15:21:37,524][20347] Updated weights for policy 0, policy_version 1401767 (0.0008) [2023-12-27 15:21:37,527][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001401768_358924288.pth... [2023-12-27 15:21:37,530][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001400552_358612992.pth [2023-12-27 15:21:38,003][20381] Updated weights for policy 1, policy_version 1400657 (0.0008) [2023-12-27 15:21:38,055][20381] Updated weights for policy 1, policy_version 1400667 (0.0008) [2023-12-27 15:21:38,110][20381] Updated weights for policy 1, policy_version 1400677 (0.0009) [2023-12-27 15:21:38,160][20381] Updated weights for policy 1, policy_version 1400687 (0.0007) [2023-12-27 15:21:38,220][20347] Updated weights for policy 0, policy_version 1401777 (0.0005) [2023-12-27 15:21:38,281][20347] Updated weights for policy 0, policy_version 1401787 (0.0006) [2023-12-27 15:21:38,333][20347] Updated weights for policy 0, policy_version 1401797 (0.0008) [2023-12-27 15:21:38,838][20381] Updated weights for policy 1, policy_version 1400697 (0.0008) [2023-12-27 15:21:38,891][20381] Updated weights for policy 1, policy_version 1400707 (0.0008) [2023-12-27 15:21:38,944][20347] Updated weights for policy 0, policy_version 1401807 (0.0006) [2023-12-27 15:21:38,949][20381] Updated weights for policy 1, policy_version 1400717 (0.0007) [2023-12-27 15:21:39,000][20347] Updated weights for policy 0, policy_version 1401817 (0.0005) [2023-12-27 15:21:39,058][20347] Updated weights for policy 0, policy_version 1401827 (0.0005) [2023-12-27 15:21:39,692][20381] Updated weights for policy 1, policy_version 1400727 (0.0007) [2023-12-27 15:21:39,750][20381] Updated weights for policy 1, policy_version 1400737 (0.0009) [2023-12-27 15:21:39,752][20347] Updated weights for policy 0, policy_version 1401837 (0.0005) [2023-12-27 15:21:39,814][20347] Updated weights for policy 0, policy_version 1401847 (0.0006) [2023-12-27 15:21:39,820][20381] Updated weights for policy 1, policy_version 1400747 (0.0008) [2023-12-27 15:21:39,875][20347] Updated weights for policy 0, policy_version 1401857 (0.0007) [2023-12-27 15:21:40,508][20381] Updated weights for policy 1, policy_version 1400757 (0.0007) [2023-12-27 15:21:40,569][20381] Updated weights for policy 1, policy_version 1400767 (0.0007) [2023-12-27 15:21:40,582][20347] Updated weights for policy 0, policy_version 1401867 (0.0006) [2023-12-27 15:21:40,625][20381] Updated weights for policy 1, policy_version 1400777 (0.0006) [2023-12-27 15:21:40,638][20347] Updated weights for policy 0, policy_version 1401877 (0.0005) [2023-12-27 15:21:40,701][20347] Updated weights for policy 0, policy_version 1401887 (0.0006) [2023-12-27 15:21:41,328][20381] Updated weights for policy 1, policy_version 1400787 (0.0007) [2023-12-27 15:21:41,398][20381] Updated weights for policy 1, policy_version 1400797 (0.0009) [2023-12-27 15:21:41,408][20347] Updated weights for policy 0, policy_version 1401897 (0.0008) [2023-12-27 15:21:41,461][20381] Updated weights for policy 1, policy_version 1400807 (0.0007) [2023-12-27 15:21:41,465][20347] Updated weights for policy 0, policy_version 1401907 (0.0006) [2023-12-27 15:21:41,527][20347] Updated weights for policy 0, policy_version 1401917 (0.0008) [2023-12-27 15:21:41,598][20347] Updated weights for policy 0, policy_version 1401927 (0.0008) [2023-12-27 15:21:42,200][20381] Updated weights for policy 1, policy_version 1400817 (0.0007) [2023-12-27 15:21:42,261][20381] Updated weights for policy 1, policy_version 1400827 (0.0006) [2023-12-27 15:21:42,288][20347] Updated weights for policy 0, policy_version 1401937 (0.0007) [2023-12-27 15:21:42,316][20381] Updated weights for policy 1, policy_version 1400837 (0.0006) [2023-12-27 15:21:42,350][20347] Updated weights for policy 0, policy_version 1401947 (0.0008) [2023-12-27 15:21:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 717602816. Throughput: 0: 10243.9, 1: 10263.3. Samples: 717602140. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:42,363][19170] Avg episode reward: [(0, '990.220'), (1, '882.730')] [2023-12-27 15:21:42,384][20381] Updated weights for policy 1, policy_version 1400847 (0.0008) [2023-12-27 15:21:42,412][20347] Updated weights for policy 0, policy_version 1401957 (0.0007) [2023-12-27 15:21:42,952][20381] Updated weights for policy 1, policy_version 1400857 (0.0008) [2023-12-27 15:21:43,005][20381] Updated weights for policy 1, policy_version 1400867 (0.0007) [2023-12-27 15:21:43,054][20381] Updated weights for policy 1, policy_version 1400877 (0.0007) [2023-12-27 15:21:43,085][20347] Updated weights for policy 0, policy_version 1401967 (0.0008) [2023-12-27 15:21:43,143][20347] Updated weights for policy 0, policy_version 1401977 (0.0007) [2023-12-27 15:21:43,198][20347] Updated weights for policy 0, policy_version 1401987 (0.0008) [2023-12-27 15:21:43,715][20381] Updated weights for policy 1, policy_version 1400887 (0.0010) [2023-12-27 15:21:43,766][20381] Updated weights for policy 1, policy_version 1400897 (0.0010) [2023-12-27 15:21:43,810][20347] Updated weights for policy 0, policy_version 1401997 (0.0006) [2023-12-27 15:21:43,821][20381] Updated weights for policy 1, policy_version 1400907 (0.0010) [2023-12-27 15:21:43,873][20347] Updated weights for policy 0, policy_version 1402007 (0.0006) [2023-12-27 15:21:43,928][20347] Updated weights for policy 0, policy_version 1402017 (0.0006) [2023-12-27 15:21:44,460][20347] Updated weights for policy 0, policy_version 1402027 (0.0006) [2023-12-27 15:21:44,515][20381] Updated weights for policy 1, policy_version 1400917 (0.0010) [2023-12-27 15:21:44,526][20347] Updated weights for policy 0, policy_version 1402037 (0.0007) [2023-12-27 15:21:44,574][20381] Updated weights for policy 1, policy_version 1400927 (0.0006) [2023-12-27 15:21:44,587][20347] Updated weights for policy 0, policy_version 1402047 (0.0008) [2023-12-27 15:21:44,640][20381] Updated weights for policy 1, policy_version 1400937 (0.0007) [2023-12-27 15:21:45,258][20347] Updated weights for policy 0, policy_version 1402057 (0.0006) [2023-12-27 15:21:45,312][20381] Updated weights for policy 1, policy_version 1400947 (0.0007) [2023-12-27 15:21:45,317][20347] Updated weights for policy 0, policy_version 1402067 (0.0008) [2023-12-27 15:21:45,367][20347] Updated weights for policy 0, policy_version 1402077 (0.0006) [2023-12-27 15:21:45,372][20381] Updated weights for policy 1, policy_version 1400957 (0.0006) [2023-12-27 15:21:45,418][20347] Updated weights for policy 0, policy_version 1402087 (0.0005) [2023-12-27 15:21:45,425][20381] Updated weights for policy 1, policy_version 1400967 (0.0008) [2023-12-27 15:21:46,031][20381] Updated weights for policy 1, policy_version 1400977 (0.0007) [2023-12-27 15:21:46,051][20347] Updated weights for policy 0, policy_version 1402097 (0.0005) [2023-12-27 15:21:46,085][20381] Updated weights for policy 1, policy_version 1400987 (0.0009) [2023-12-27 15:21:46,101][20347] Updated weights for policy 0, policy_version 1402107 (0.0006) [2023-12-27 15:21:46,131][20381] Updated weights for policy 1, policy_version 1400997 (0.0010) [2023-12-27 15:21:46,150][20347] Updated weights for policy 0, policy_version 1402117 (0.0006) [2023-12-27 15:21:46,182][20381] Updated weights for policy 1, policy_version 1401007 (0.0010) [2023-12-27 15:21:46,750][20347] Updated weights for policy 0, policy_version 1402127 (0.0007) [2023-12-27 15:21:46,799][20347] Updated weights for policy 0, policy_version 1402137 (0.0006) [2023-12-27 15:21:46,848][20347] Updated weights for policy 0, policy_version 1402147 (0.0005) [2023-12-27 15:21:46,871][20381] Updated weights for policy 1, policy_version 1401017 (0.0008) [2023-12-27 15:21:46,920][20381] Updated weights for policy 1, policy_version 1401027 (0.0006) [2023-12-27 15:21:46,969][20381] Updated weights for policy 1, policy_version 1401037 (0.0005) [2023-12-27 15:21:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 717717504. Throughput: 0: 10376.5, 1: 10285.3. Samples: 717728636. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:47,363][19170] Avg episode reward: [(0, '990.220'), (1, '941.420')] [2023-12-27 15:21:47,435][20347] Updated weights for policy 0, policy_version 1402157 (0.0006) [2023-12-27 15:21:47,497][20347] Updated weights for policy 0, policy_version 1402167 (0.0009) [2023-12-27 15:21:47,552][20347] Updated weights for policy 0, policy_version 1402177 (0.0010) [2023-12-27 15:21:47,702][20381] Updated weights for policy 1, policy_version 1401047 (0.0009) [2023-12-27 15:21:47,758][20381] Updated weights for policy 1, policy_version 1401057 (0.0010) [2023-12-27 15:21:47,813][20381] Updated weights for policy 1, policy_version 1401067 (0.0010) [2023-12-27 15:21:48,175][20347] Updated weights for policy 0, policy_version 1402187 (0.0009) [2023-12-27 15:21:48,223][20347] Updated weights for policy 0, policy_version 1402197 (0.0008) [2023-12-27 15:21:48,272][20347] Updated weights for policy 0, policy_version 1402207 (0.0007) [2023-12-27 15:21:48,380][20381] Updated weights for policy 1, policy_version 1401077 (0.0010) [2023-12-27 15:21:48,439][20381] Updated weights for policy 1, policy_version 1401087 (0.0011) [2023-12-27 15:21:48,504][20381] Updated weights for policy 1, policy_version 1401097 (0.0009) [2023-12-27 15:21:48,932][20347] Updated weights for policy 0, policy_version 1402217 (0.0007) [2023-12-27 15:21:48,994][20347] Updated weights for policy 0, policy_version 1402227 (0.0007) [2023-12-27 15:21:49,052][20347] Updated weights for policy 0, policy_version 1402237 (0.0008) [2023-12-27 15:21:49,109][20347] Updated weights for policy 0, policy_version 1402247 (0.0006) [2023-12-27 15:21:49,173][20381] Updated weights for policy 1, policy_version 1401107 (0.0010) [2023-12-27 15:21:49,236][20381] Updated weights for policy 1, policy_version 1401117 (0.0009) [2023-12-27 15:21:49,290][20381] Updated weights for policy 1, policy_version 1401127 (0.0009) [2023-12-27 15:21:49,771][20347] Updated weights for policy 0, policy_version 1402257 (0.0008) [2023-12-27 15:21:49,822][20347] Updated weights for policy 0, policy_version 1402267 (0.0009) [2023-12-27 15:21:49,885][20347] Updated weights for policy 0, policy_version 1402277 (0.0008) [2023-12-27 15:21:49,989][20381] Updated weights for policy 1, policy_version 1401137 (0.0007) [2023-12-27 15:21:50,047][20381] Updated weights for policy 1, policy_version 1401147 (0.0008) [2023-12-27 15:21:50,108][20381] Updated weights for policy 1, policy_version 1401157 (0.0008) [2023-12-27 15:21:50,161][20381] Updated weights for policy 1, policy_version 1401167 (0.0007) [2023-12-27 15:21:50,537][20347] Updated weights for policy 0, policy_version 1402287 (0.0007) [2023-12-27 15:21:50,599][20347] Updated weights for policy 0, policy_version 1402297 (0.0007) [2023-12-27 15:21:50,654][20347] Updated weights for policy 0, policy_version 1402307 (0.0008) [2023-12-27 15:21:50,852][20381] Updated weights for policy 1, policy_version 1401177 (0.0007) [2023-12-27 15:21:50,909][20381] Updated weights for policy 1, policy_version 1401187 (0.0007) [2023-12-27 15:21:50,968][20381] Updated weights for policy 1, policy_version 1401197 (0.0007) [2023-12-27 15:21:51,383][20347] Updated weights for policy 0, policy_version 1402317 (0.0008) [2023-12-27 15:21:51,442][20347] Updated weights for policy 0, policy_version 1402327 (0.0006) [2023-12-27 15:21:51,497][20347] Updated weights for policy 0, policy_version 1402337 (0.0006) [2023-12-27 15:21:51,707][20381] Updated weights for policy 1, policy_version 1401207 (0.0007) [2023-12-27 15:21:51,772][20381] Updated weights for policy 1, policy_version 1401217 (0.0008) [2023-12-27 15:21:51,821][20381] Updated weights for policy 1, policy_version 1401227 (0.0008) [2023-12-27 15:21:52,216][20347] Updated weights for policy 0, policy_version 1402347 (0.0007) [2023-12-27 15:21:52,272][20347] Updated weights for policy 0, policy_version 1402357 (0.0008) [2023-12-27 15:21:52,324][20347] Updated weights for policy 0, policy_version 1402367 (0.0007) [2023-12-27 15:21:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 717815808. Throughput: 0: 10433.2, 1: 10284.9. Samples: 717791416. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.190')] [2023-12-27 15:21:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001401232_358744064.pth... [2023-12-27 15:21:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001400048_358440960.pth [2023-12-27 15:21:52,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001402376_359079936.pth... [2023-12-27 15:21:52,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001401160_358768640.pth [2023-12-27 15:21:52,500][20381] Updated weights for policy 1, policy_version 1401237 (0.0008) [2023-12-27 15:21:52,550][20381] Updated weights for policy 1, policy_version 1401247 (0.0007) [2023-12-27 15:21:52,600][20381] Updated weights for policy 1, policy_version 1401257 (0.0008) [2023-12-27 15:21:52,948][20347] Updated weights for policy 0, policy_version 1402377 (0.0008) [2023-12-27 15:21:53,002][20347] Updated weights for policy 0, policy_version 1402387 (0.0005) [2023-12-27 15:21:53,058][20347] Updated weights for policy 0, policy_version 1402397 (0.0005) [2023-12-27 15:21:53,115][20347] Updated weights for policy 0, policy_version 1402407 (0.0005) [2023-12-27 15:21:53,309][20381] Updated weights for policy 1, policy_version 1401267 (0.0008) [2023-12-27 15:21:53,365][20381] Updated weights for policy 1, policy_version 1401277 (0.0008) [2023-12-27 15:21:53,417][20381] Updated weights for policy 1, policy_version 1401287 (0.0007) [2023-12-27 15:21:53,691][20347] Updated weights for policy 0, policy_version 1402417 (0.0005) [2023-12-27 15:21:53,739][20347] Updated weights for policy 0, policy_version 1402427 (0.0008) [2023-12-27 15:21:53,788][20347] Updated weights for policy 0, policy_version 1402437 (0.0010) [2023-12-27 15:21:54,160][20381] Updated weights for policy 1, policy_version 1401297 (0.0007) [2023-12-27 15:21:54,222][20381] Updated weights for policy 1, policy_version 1401307 (0.0007) [2023-12-27 15:21:54,273][20381] Updated weights for policy 1, policy_version 1401317 (0.0007) [2023-12-27 15:21:54,325][20381] Updated weights for policy 1, policy_version 1401327 (0.0008) [2023-12-27 15:21:54,434][20347] Updated weights for policy 0, policy_version 1402447 (0.0007) [2023-12-27 15:21:54,495][20347] Updated weights for policy 0, policy_version 1402457 (0.0006) [2023-12-27 15:21:54,549][20347] Updated weights for policy 0, policy_version 1402467 (0.0005) [2023-12-27 15:21:55,050][20381] Updated weights for policy 1, policy_version 1401337 (0.0007) [2023-12-27 15:21:55,107][20381] Updated weights for policy 1, policy_version 1401347 (0.0007) [2023-12-27 15:21:55,132][20347] Updated weights for policy 0, policy_version 1402477 (0.0006) [2023-12-27 15:21:55,167][20381] Updated weights for policy 1, policy_version 1401357 (0.0007) [2023-12-27 15:21:55,189][20347] Updated weights for policy 0, policy_version 1402487 (0.0006) [2023-12-27 15:21:55,239][20347] Updated weights for policy 0, policy_version 1402497 (0.0008) [2023-12-27 15:21:55,835][20381] Updated weights for policy 1, policy_version 1401367 (0.0008) [2023-12-27 15:21:55,883][20381] Updated weights for policy 1, policy_version 1401377 (0.0007) [2023-12-27 15:21:55,938][20347] Updated weights for policy 0, policy_version 1402507 (0.0006) [2023-12-27 15:21:55,941][20381] Updated weights for policy 1, policy_version 1401387 (0.0008) [2023-12-27 15:21:56,002][20347] Updated weights for policy 0, policy_version 1402517 (0.0007) [2023-12-27 15:21:56,055][20347] Updated weights for policy 0, policy_version 1402527 (0.0007) [2023-12-27 15:21:56,669][20381] Updated weights for policy 1, policy_version 1401397 (0.0008) [2023-12-27 15:21:56,687][20347] Updated weights for policy 0, policy_version 1402537 (0.0007) [2023-12-27 15:21:56,735][20381] Updated weights for policy 1, policy_version 1401407 (0.0007) [2023-12-27 15:21:56,755][20347] Updated weights for policy 0, policy_version 1402548 (0.0006) [2023-12-27 15:21:56,786][20381] Updated weights for policy 1, policy_version 1401417 (0.0007) [2023-12-27 15:21:56,811][20347] Updated weights for policy 0, policy_version 1402558 (0.0006) [2023-12-27 15:21:56,869][20347] Updated weights for policy 0, policy_version 1402568 (0.0008) [2023-12-27 15:21:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 717922304. Throughput: 0: 10418.0, 1: 10214.3. Samples: 717914464. Policy #0 lag: (min: 28.0, avg: 31.8, max: 60.0) [2023-12-27 15:21:57,363][19170] Avg episode reward: [(0, '990.220'), (1, '921.880')] [2023-12-27 15:21:57,445][20381] Updated weights for policy 1, policy_version 1401427 (0.0008) [2023-12-27 15:21:57,494][20381] Updated weights for policy 1, policy_version 1401437 (0.0006) [2023-12-27 15:21:57,503][20347] Updated weights for policy 0, policy_version 1402578 (0.0006) [2023-12-27 15:21:57,542][20381] Updated weights for policy 1, policy_version 1401447 (0.0005) [2023-12-27 15:21:57,554][20347] Updated weights for policy 0, policy_version 1402588 (0.0006) [2023-12-27 15:21:57,607][20347] Updated weights for policy 0, policy_version 1402598 (0.0006) [2023-12-27 15:21:58,255][20381] Updated weights for policy 1, policy_version 1401457 (0.0005) [2023-12-27 15:21:58,295][20347] Updated weights for policy 0, policy_version 1402608 (0.0008) [2023-12-27 15:21:58,308][20381] Updated weights for policy 1, policy_version 1401467 (0.0007) [2023-12-27 15:21:58,353][20347] Updated weights for policy 0, policy_version 1402618 (0.0008) [2023-12-27 15:21:58,372][20381] Updated weights for policy 1, policy_version 1401477 (0.0008) [2023-12-27 15:21:58,414][20347] Updated weights for policy 0, policy_version 1402628 (0.0007) [2023-12-27 15:21:58,424][20381] Updated weights for policy 1, policy_version 1401487 (0.0008) [2023-12-27 15:21:59,160][20347] Updated weights for policy 0, policy_version 1402638 (0.0009) [2023-12-27 15:21:59,219][20347] Updated weights for policy 0, policy_version 1402648 (0.0008) [2023-12-27 15:21:59,266][20381] Updated weights for policy 1, policy_version 1401497 (0.0009) [2023-12-27 15:21:59,289][20347] Updated weights for policy 0, policy_version 1402658 (0.0008) [2023-12-27 15:21:59,328][20381] Updated weights for policy 1, policy_version 1401507 (0.0010) [2023-12-27 15:21:59,393][20381] Updated weights for policy 1, policy_version 1401517 (0.0010) [2023-12-27 15:22:00,042][20347] Updated weights for policy 0, policy_version 1402668 (0.0009) [2023-12-27 15:22:00,089][20347] Updated weights for policy 0, policy_version 1402678 (0.0010) [2023-12-27 15:22:00,101][20381] Updated weights for policy 1, policy_version 1401527 (0.0010) [2023-12-27 15:22:00,141][20347] Updated weights for policy 0, policy_version 1402688 (0.0010) [2023-12-27 15:22:00,156][20381] Updated weights for policy 1, policy_version 1401537 (0.0010) [2023-12-27 15:22:00,210][20381] Updated weights for policy 1, policy_version 1401547 (0.0011) [2023-12-27 15:22:00,774][20347] Updated weights for policy 0, policy_version 1402698 (0.0009) [2023-12-27 15:22:00,820][20347] Updated weights for policy 0, policy_version 1402708 (0.0008) [2023-12-27 15:22:00,860][20381] Updated weights for policy 1, policy_version 1401557 (0.0008) [2023-12-27 15:22:00,877][20347] Updated weights for policy 0, policy_version 1402718 (0.0010) [2023-12-27 15:22:00,914][20381] Updated weights for policy 1, policy_version 1401567 (0.0007) [2023-12-27 15:22:00,931][20347] Updated weights for policy 0, policy_version 1402728 (0.0010) [2023-12-27 15:22:00,967][20381] Updated weights for policy 1, policy_version 1401577 (0.0009) [2023-12-27 15:22:01,644][20381] Updated weights for policy 1, policy_version 1401587 (0.0006) [2023-12-27 15:22:01,660][20347] Updated weights for policy 0, policy_version 1402738 (0.0008) [2023-12-27 15:22:01,702][20381] Updated weights for policy 1, policy_version 1401597 (0.0008) [2023-12-27 15:22:01,710][20347] Updated weights for policy 0, policy_version 1402748 (0.0008) [2023-12-27 15:22:01,771][20381] Updated weights for policy 1, policy_version 1401607 (0.0007) [2023-12-27 15:22:01,780][20347] Updated weights for policy 0, policy_version 1402758 (0.0009) [2023-12-27 15:22:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 718020608. Throughput: 0: 10362.0, 1: 10138.0. Samples: 718033704. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '951.150')] [2023-12-27 15:22:02,463][20381] Updated weights for policy 1, policy_version 1401617 (0.0007) [2023-12-27 15:22:02,485][20347] Updated weights for policy 0, policy_version 1402768 (0.0011) [2023-12-27 15:22:02,525][20381] Updated weights for policy 1, policy_version 1401627 (0.0006) [2023-12-27 15:22:02,538][20347] Updated weights for policy 0, policy_version 1402778 (0.0010) [2023-12-27 15:22:02,584][20381] Updated weights for policy 1, policy_version 1401637 (0.0005) [2023-12-27 15:22:02,586][20347] Updated weights for policy 0, policy_version 1402788 (0.0010) [2023-12-27 15:22:02,638][20381] Updated weights for policy 1, policy_version 1401647 (0.0005) [2023-12-27 15:22:03,267][20347] Updated weights for policy 0, policy_version 1402798 (0.0007) [2023-12-27 15:22:03,269][20381] Updated weights for policy 1, policy_version 1401657 (0.0008) [2023-12-27 15:22:03,323][20347] Updated weights for policy 0, policy_version 1402808 (0.0005) [2023-12-27 15:22:03,325][20381] Updated weights for policy 1, policy_version 1401667 (0.0008) [2023-12-27 15:22:03,374][20347] Updated weights for policy 0, policy_version 1402818 (0.0007) [2023-12-27 15:22:03,379][20381] Updated weights for policy 1, policy_version 1401677 (0.0007) [2023-12-27 15:22:04,018][20347] Updated weights for policy 0, policy_version 1402828 (0.0009) [2023-12-27 15:22:04,085][20347] Updated weights for policy 0, policy_version 1402838 (0.0008) [2023-12-27 15:22:04,114][20381] Updated weights for policy 1, policy_version 1401687 (0.0006) [2023-12-27 15:22:04,149][20347] Updated weights for policy 0, policy_version 1402848 (0.0008) [2023-12-27 15:22:04,173][20381] Updated weights for policy 1, policy_version 1401697 (0.0007) [2023-12-27 15:22:04,231][20381] Updated weights for policy 1, policy_version 1401707 (0.0008) [2023-12-27 15:22:04,824][20347] Updated weights for policy 0, policy_version 1402858 (0.0007) [2023-12-27 15:22:04,830][20381] Updated weights for policy 1, policy_version 1401717 (0.0007) [2023-12-27 15:22:04,882][20347] Updated weights for policy 0, policy_version 1402868 (0.0006) [2023-12-27 15:22:04,883][20381] Updated weights for policy 1, policy_version 1401727 (0.0005) [2023-12-27 15:22:04,934][20381] Updated weights for policy 1, policy_version 1401737 (0.0005) [2023-12-27 15:22:04,939][20347] Updated weights for policy 0, policy_version 1402878 (0.0008) [2023-12-27 15:22:04,994][20347] Updated weights for policy 0, policy_version 1402888 (0.0007) [2023-12-27 15:22:05,549][20381] Updated weights for policy 1, policy_version 1401747 (0.0008) [2023-12-27 15:22:05,603][20381] Updated weights for policy 1, policy_version 1401757 (0.0008) [2023-12-27 15:22:05,657][20381] Updated weights for policy 1, policy_version 1401767 (0.0008) [2023-12-27 15:22:05,686][20347] Updated weights for policy 0, policy_version 1402898 (0.0005) [2023-12-27 15:22:05,734][20347] Updated weights for policy 0, policy_version 1402908 (0.0005) [2023-12-27 15:22:05,793][20347] Updated weights for policy 0, policy_version 1402918 (0.0005) [2023-12-27 15:22:06,331][20381] Updated weights for policy 1, policy_version 1401777 (0.0008) [2023-12-27 15:22:06,391][20381] Updated weights for policy 1, policy_version 1401787 (0.0008) [2023-12-27 15:22:06,435][20347] Updated weights for policy 0, policy_version 1402928 (0.0006) [2023-12-27 15:22:06,445][20381] Updated weights for policy 1, policy_version 1401797 (0.0006) [2023-12-27 15:22:06,486][20347] Updated weights for policy 0, policy_version 1402938 (0.0006) [2023-12-27 15:22:06,499][20381] Updated weights for policy 1, policy_version 1401807 (0.0006) [2023-12-27 15:22:06,544][20347] Updated weights for policy 0, policy_version 1402948 (0.0007) [2023-12-27 15:22:07,202][20347] Updated weights for policy 0, policy_version 1402958 (0.0006) [2023-12-27 15:22:07,212][20381] Updated weights for policy 1, policy_version 1401817 (0.0010) [2023-12-27 15:22:07,256][20347] Updated weights for policy 0, policy_version 1402968 (0.0005) [2023-12-27 15:22:07,265][20381] Updated weights for policy 1, policy_version 1401827 (0.0011) [2023-12-27 15:22:07,304][20347] Updated weights for policy 0, policy_version 1402978 (0.0005) [2023-12-27 15:22:07,318][20381] Updated weights for policy 1, policy_version 1401837 (0.0011) [2023-12-27 15:22:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 718135296. Throughput: 0: 10346.7, 1: 10198.5. Samples: 718096440. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:07,363][19170] Avg episode reward: [(0, '990.210'), (1, '970.740')] [2023-12-27 15:22:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001401840_358899712.pth... [2023-12-27 15:22:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001402984_359235584.pth... [2023-12-27 15:22:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001400656_358596608.pth [2023-12-27 15:22:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001401768_358924288.pth [2023-12-27 15:22:07,944][20381] Updated weights for policy 1, policy_version 1401847 (0.0008) [2023-12-27 15:22:07,986][20347] Updated weights for policy 0, policy_version 1402988 (0.0005) [2023-12-27 15:22:07,995][20381] Updated weights for policy 1, policy_version 1401857 (0.0008) [2023-12-27 15:22:08,043][20347] Updated weights for policy 0, policy_version 1402998 (0.0005) [2023-12-27 15:22:08,059][20381] Updated weights for policy 1, policy_version 1401867 (0.0006) [2023-12-27 15:22:08,103][20347] Updated weights for policy 0, policy_version 1403008 (0.0006) [2023-12-27 15:22:08,652][20381] Updated weights for policy 1, policy_version 1401877 (0.0008) [2023-12-27 15:22:08,688][20347] Updated weights for policy 0, policy_version 1403018 (0.0006) [2023-12-27 15:22:08,711][20381] Updated weights for policy 1, policy_version 1401887 (0.0008) [2023-12-27 15:22:08,749][20347] Updated weights for policy 0, policy_version 1403028 (0.0010) [2023-12-27 15:22:08,771][20381] Updated weights for policy 1, policy_version 1401897 (0.0006) [2023-12-27 15:22:08,808][20347] Updated weights for policy 0, policy_version 1403038 (0.0010) [2023-12-27 15:22:08,870][20347] Updated weights for policy 0, policy_version 1403048 (0.0010) [2023-12-27 15:22:09,454][20381] Updated weights for policy 1, policy_version 1401907 (0.0006) [2023-12-27 15:22:09,514][20381] Updated weights for policy 1, policy_version 1401917 (0.0008) [2023-12-27 15:22:09,573][20381] Updated weights for policy 1, policy_version 1401927 (0.0008) [2023-12-27 15:22:09,590][20347] Updated weights for policy 0, policy_version 1403058 (0.0011) [2023-12-27 15:22:09,649][20347] Updated weights for policy 0, policy_version 1403068 (0.0010) [2023-12-27 15:22:09,709][20347] Updated weights for policy 0, policy_version 1403078 (0.0008) [2023-12-27 15:22:10,309][20381] Updated weights for policy 1, policy_version 1401937 (0.0006) [2023-12-27 15:22:10,374][20381] Updated weights for policy 1, policy_version 1401947 (0.0008) [2023-12-27 15:22:10,436][20381] Updated weights for policy 1, policy_version 1401957 (0.0007) [2023-12-27 15:22:10,467][20347] Updated weights for policy 0, policy_version 1403088 (0.0008) [2023-12-27 15:22:10,496][20381] Updated weights for policy 1, policy_version 1401967 (0.0007) [2023-12-27 15:22:10,524][20347] Updated weights for policy 0, policy_version 1403098 (0.0007) [2023-12-27 15:22:10,581][20347] Updated weights for policy 0, policy_version 1403108 (0.0006) [2023-12-27 15:22:11,195][20347] Updated weights for policy 0, policy_version 1403118 (0.0007) [2023-12-27 15:22:11,260][20347] Updated weights for policy 0, policy_version 1403128 (0.0006) [2023-12-27 15:22:11,309][20381] Updated weights for policy 1, policy_version 1401977 (0.0007) [2023-12-27 15:22:11,322][20347] Updated weights for policy 0, policy_version 1403138 (0.0006) [2023-12-27 15:22:11,374][20381] Updated weights for policy 1, policy_version 1401987 (0.0006) [2023-12-27 15:22:11,433][20381] Updated weights for policy 1, policy_version 1401997 (0.0009) [2023-12-27 15:22:11,972][20347] Updated weights for policy 0, policy_version 1403148 (0.0007) [2023-12-27 15:22:12,032][20347] Updated weights for policy 0, policy_version 1403158 (0.0006) [2023-12-27 15:22:12,088][20347] Updated weights for policy 0, policy_version 1403168 (0.0006) [2023-12-27 15:22:12,179][20381] Updated weights for policy 1, policy_version 1402007 (0.0008) [2023-12-27 15:22:12,228][20381] Updated weights for policy 1, policy_version 1402017 (0.0009) [2023-12-27 15:22:12,278][20381] Updated weights for policy 1, policy_version 1402027 (0.0009) [2023-12-27 15:22:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 718233600. Throughput: 0: 10395.9, 1: 10197.0. Samples: 718220356. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:12,363][19170] Avg episode reward: [(0, '990.210'), (1, '970.730')] [2023-12-27 15:22:12,742][20347] Updated weights for policy 0, policy_version 1403178 (0.0008) [2023-12-27 15:22:12,803][20347] Updated weights for policy 0, policy_version 1403188 (0.0008) [2023-12-27 15:22:12,864][20347] Updated weights for policy 0, policy_version 1403198 (0.0007) [2023-12-27 15:22:12,921][20347] Updated weights for policy 0, policy_version 1403208 (0.0008) [2023-12-27 15:22:13,011][20381] Updated weights for policy 1, policy_version 1402037 (0.0009) [2023-12-27 15:22:13,065][20381] Updated weights for policy 1, policy_version 1402047 (0.0007) [2023-12-27 15:22:13,117][20381] Updated weights for policy 1, policy_version 1402057 (0.0008) [2023-12-27 15:22:13,531][20347] Updated weights for policy 0, policy_version 1403218 (0.0007) [2023-12-27 15:22:13,589][20347] Updated weights for policy 0, policy_version 1403228 (0.0007) [2023-12-27 15:22:13,649][20347] Updated weights for policy 0, policy_version 1403238 (0.0007) [2023-12-27 15:22:13,813][20381] Updated weights for policy 1, policy_version 1402067 (0.0007) [2023-12-27 15:22:13,872][20381] Updated weights for policy 1, policy_version 1402077 (0.0005) [2023-12-27 15:22:13,925][20381] Updated weights for policy 1, policy_version 1402087 (0.0006) [2023-12-27 15:22:14,323][20347] Updated weights for policy 0, policy_version 1403248 (0.0010) [2023-12-27 15:22:14,381][20347] Updated weights for policy 0, policy_version 1403258 (0.0010) [2023-12-27 15:22:14,435][20347] Updated weights for policy 0, policy_version 1403268 (0.0010) [2023-12-27 15:22:14,557][20381] Updated weights for policy 1, policy_version 1402097 (0.0008) [2023-12-27 15:22:14,607][20381] Updated weights for policy 1, policy_version 1402107 (0.0008) [2023-12-27 15:22:14,664][20381] Updated weights for policy 1, policy_version 1402117 (0.0009) [2023-12-27 15:22:14,710][20381] Updated weights for policy 1, policy_version 1402127 (0.0008) [2023-12-27 15:22:15,031][20347] Updated weights for policy 0, policy_version 1403278 (0.0010) [2023-12-27 15:22:15,090][20347] Updated weights for policy 0, policy_version 1403288 (0.0010) [2023-12-27 15:22:15,139][20347] Updated weights for policy 0, policy_version 1403298 (0.0010) [2023-12-27 15:22:15,485][20381] Updated weights for policy 1, policy_version 1402137 (0.0007) [2023-12-27 15:22:15,538][20381] Updated weights for policy 1, policy_version 1402147 (0.0008) [2023-12-27 15:22:15,597][20381] Updated weights for policy 1, policy_version 1402157 (0.0006) [2023-12-27 15:22:15,832][20347] Updated weights for policy 0, policy_version 1403308 (0.0008) [2023-12-27 15:22:15,887][20347] Updated weights for policy 0, policy_version 1403318 (0.0008) [2023-12-27 15:22:15,946][20347] Updated weights for policy 0, policy_version 1403328 (0.0008) [2023-12-27 15:22:16,242][20381] Updated weights for policy 1, policy_version 1402167 (0.0008) [2023-12-27 15:22:16,292][20381] Updated weights for policy 1, policy_version 1402177 (0.0009) [2023-12-27 15:22:16,341][20381] Updated weights for policy 1, policy_version 1402187 (0.0008) [2023-12-27 15:22:16,497][20347] Updated weights for policy 0, policy_version 1403338 (0.0006) [2023-12-27 15:22:16,546][20347] Updated weights for policy 0, policy_version 1403348 (0.0008) [2023-12-27 15:22:16,602][20347] Updated weights for policy 0, policy_version 1403358 (0.0007) [2023-12-27 15:22:16,662][20347] Updated weights for policy 0, policy_version 1403368 (0.0008) [2023-12-27 15:22:17,089][20381] Updated weights for policy 1, policy_version 1402197 (0.0009) [2023-12-27 15:22:17,148][20381] Updated weights for policy 1, policy_version 1402207 (0.0008) [2023-12-27 15:22:17,201][20381] Updated weights for policy 1, policy_version 1402217 (0.0007) [2023-12-27 15:22:17,282][20347] Updated weights for policy 0, policy_version 1403378 (0.0005) [2023-12-27 15:22:17,331][20347] Updated weights for policy 0, policy_version 1403388 (0.0005) [2023-12-27 15:22:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 718331904. Throughput: 0: 10472.5, 1: 10106.6. Samples: 718343480. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '990.280')] [2023-12-27 15:22:17,396][20347] Updated weights for policy 0, policy_version 1403398 (0.0007) [2023-12-27 15:22:17,884][20381] Updated weights for policy 1, policy_version 1402227 (0.0009) [2023-12-27 15:22:17,932][20381] Updated weights for policy 1, policy_version 1402237 (0.0010) [2023-12-27 15:22:17,984][20381] Updated weights for policy 1, policy_version 1402247 (0.0010) [2023-12-27 15:22:18,000][20347] Updated weights for policy 0, policy_version 1403408 (0.0009) [2023-12-27 15:22:18,050][20347] Updated weights for policy 0, policy_version 1403418 (0.0006) [2023-12-27 15:22:18,100][20347] Updated weights for policy 0, policy_version 1403428 (0.0007) [2023-12-27 15:22:18,596][20381] Updated weights for policy 1, policy_version 1402257 (0.0011) [2023-12-27 15:22:18,662][20381] Updated weights for policy 1, policy_version 1402267 (0.0008) [2023-12-27 15:22:18,720][20381] Updated weights for policy 1, policy_version 1402277 (0.0008) [2023-12-27 15:22:18,773][20347] Updated weights for policy 0, policy_version 1403438 (0.0007) [2023-12-27 15:22:18,791][20381] Updated weights for policy 1, policy_version 1402287 (0.0007) [2023-12-27 15:22:18,833][20347] Updated weights for policy 0, policy_version 1403448 (0.0008) [2023-12-27 15:22:18,896][20347] Updated weights for policy 0, policy_version 1403458 (0.0006) [2023-12-27 15:22:19,451][20381] Updated weights for policy 1, policy_version 1402297 (0.0008) [2023-12-27 15:22:19,494][20347] Updated weights for policy 0, policy_version 1403468 (0.0007) [2023-12-27 15:22:19,520][20381] Updated weights for policy 1, policy_version 1402307 (0.0008) [2023-12-27 15:22:19,551][20347] Updated weights for policy 0, policy_version 1403478 (0.0007) [2023-12-27 15:22:19,581][20381] Updated weights for policy 1, policy_version 1402317 (0.0009) [2023-12-27 15:22:19,601][20347] Updated weights for policy 0, policy_version 1403488 (0.0007) [2023-12-27 15:22:20,283][20381] Updated weights for policy 1, policy_version 1402327 (0.0006) [2023-12-27 15:22:20,332][20347] Updated weights for policy 0, policy_version 1403498 (0.0009) [2023-12-27 15:22:20,344][20381] Updated weights for policy 1, policy_version 1402337 (0.0008) [2023-12-27 15:22:20,387][20347] Updated weights for policy 0, policy_version 1403508 (0.0006) [2023-12-27 15:22:20,400][20381] Updated weights for policy 1, policy_version 1402347 (0.0011) [2023-12-27 15:22:20,437][20347] Updated weights for policy 0, policy_version 1403518 (0.0006) [2023-12-27 15:22:20,491][20347] Updated weights for policy 0, policy_version 1403528 (0.0007) [2023-12-27 15:22:21,113][20381] Updated weights for policy 1, policy_version 1402357 (0.0011) [2023-12-27 15:22:21,174][20381] Updated weights for policy 1, policy_version 1402367 (0.0011) [2023-12-27 15:22:21,221][20347] Updated weights for policy 0, policy_version 1403538 (0.0010) [2023-12-27 15:22:21,237][20381] Updated weights for policy 1, policy_version 1402377 (0.0011) [2023-12-27 15:22:21,281][20347] Updated weights for policy 0, policy_version 1403548 (0.0010) [2023-12-27 15:22:21,346][20347] Updated weights for policy 0, policy_version 1403558 (0.0011) [2023-12-27 15:22:22,004][20381] Updated weights for policy 1, policy_version 1402387 (0.0010) [2023-12-27 15:22:22,059][20381] Updated weights for policy 1, policy_version 1402397 (0.0007) [2023-12-27 15:22:22,083][20347] Updated weights for policy 0, policy_version 1403568 (0.0009) [2023-12-27 15:22:22,121][20381] Updated weights for policy 1, policy_version 1402407 (0.0006) [2023-12-27 15:22:22,131][20347] Updated weights for policy 0, policy_version 1403578 (0.0010) [2023-12-27 15:22:22,180][20347] Updated weights for policy 0, policy_version 1403588 (0.0010) [2023-12-27 15:22:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 718438400. Throughput: 0: 10490.6, 1: 10119.1. Samples: 718406388. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:22,363][19170] Avg episode reward: [(0, '990.330'), (1, '970.690')] [2023-12-27 15:22:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001403592_359391232.pth... [2023-12-27 15:22:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001402416_359047168.pth... [2023-12-27 15:22:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001402376_359079936.pth [2023-12-27 15:22:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001401232_358744064.pth [2023-12-27 15:22:22,845][20381] Updated weights for policy 1, policy_version 1402417 (0.0005) [2023-12-27 15:22:22,893][20381] Updated weights for policy 1, policy_version 1402427 (0.0008) [2023-12-27 15:22:22,901][20347] Updated weights for policy 0, policy_version 1403598 (0.0006) [2023-12-27 15:22:22,944][20381] Updated weights for policy 1, policy_version 1402437 (0.0007) [2023-12-27 15:22:22,963][20347] Updated weights for policy 0, policy_version 1403608 (0.0005) [2023-12-27 15:22:23,002][20381] Updated weights for policy 1, policy_version 1402447 (0.0007) [2023-12-27 15:22:23,016][20347] Updated weights for policy 0, policy_version 1403618 (0.0006) [2023-12-27 15:22:23,708][20347] Updated weights for policy 0, policy_version 1403628 (0.0008) [2023-12-27 15:22:23,719][20381] Updated weights for policy 1, policy_version 1402457 (0.0007) [2023-12-27 15:22:23,768][20381] Updated weights for policy 1, policy_version 1402467 (0.0007) [2023-12-27 15:22:23,771][20347] Updated weights for policy 0, policy_version 1403638 (0.0008) [2023-12-27 15:22:23,823][20381] Updated weights for policy 1, policy_version 1402477 (0.0007) [2023-12-27 15:22:23,824][20347] Updated weights for policy 0, policy_version 1403648 (0.0007) [2023-12-27 15:22:24,452][20381] Updated weights for policy 1, policy_version 1402487 (0.0006) [2023-12-27 15:22:24,508][20381] Updated weights for policy 1, policy_version 1402497 (0.0007) [2023-12-27 15:22:24,529][20347] Updated weights for policy 0, policy_version 1403658 (0.0006) [2023-12-27 15:22:24,569][20381] Updated weights for policy 1, policy_version 1402507 (0.0008) [2023-12-27 15:22:24,584][20347] Updated weights for policy 0, policy_version 1403668 (0.0008) [2023-12-27 15:22:24,650][20347] Updated weights for policy 0, policy_version 1403678 (0.0007) [2023-12-27 15:22:24,710][20347] Updated weights for policy 0, policy_version 1403688 (0.0007) [2023-12-27 15:22:25,200][20381] Updated weights for policy 1, policy_version 1402517 (0.0008) [2023-12-27 15:22:25,254][20381] Updated weights for policy 1, policy_version 1402527 (0.0010) [2023-12-27 15:22:25,304][20381] Updated weights for policy 1, policy_version 1402537 (0.0010) [2023-12-27 15:22:25,376][20347] Updated weights for policy 0, policy_version 1403698 (0.0008) [2023-12-27 15:22:25,424][20347] Updated weights for policy 0, policy_version 1403708 (0.0009) [2023-12-27 15:22:25,474][20347] Updated weights for policy 0, policy_version 1403718 (0.0008) [2023-12-27 15:22:25,863][20381] Updated weights for policy 1, policy_version 1402547 (0.0009) [2023-12-27 15:22:25,925][20381] Updated weights for policy 1, policy_version 1402557 (0.0005) [2023-12-27 15:22:25,986][20381] Updated weights for policy 1, policy_version 1402567 (0.0008) [2023-12-27 15:22:26,164][20347] Updated weights for policy 0, policy_version 1403728 (0.0005) [2023-12-27 15:22:26,220][20347] Updated weights for policy 0, policy_version 1403738 (0.0005) [2023-12-27 15:22:26,274][20347] Updated weights for policy 0, policy_version 1403748 (0.0005) [2023-12-27 15:22:26,627][20381] Updated weights for policy 1, policy_version 1402577 (0.0011) [2023-12-27 15:22:26,678][20381] Updated weights for policy 1, policy_version 1402587 (0.0009) [2023-12-27 15:22:26,735][20381] Updated weights for policy 1, policy_version 1402597 (0.0008) [2023-12-27 15:22:26,787][20381] Updated weights for policy 1, policy_version 1402607 (0.0011) [2023-12-27 15:22:26,924][20347] Updated weights for policy 0, policy_version 1403758 (0.0006) [2023-12-27 15:22:26,976][20347] Updated weights for policy 0, policy_version 1403768 (0.0005) [2023-12-27 15:22:27,037][20347] Updated weights for policy 0, policy_version 1403778 (0.0007) [2023-12-27 15:22:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 718536704. Throughput: 0: 10431.0, 1: 10133.4. Samples: 718527536. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:27,363][19170] Avg episode reward: [(0, '990.330'), (1, '951.290')] [2023-12-27 15:22:27,398][20381] Updated weights for policy 1, policy_version 1402617 (0.0010) [2023-12-27 15:22:27,452][20381] Updated weights for policy 1, policy_version 1402627 (0.0010) [2023-12-27 15:22:27,457][20292] KL-divergence is very high: 106.8743 [2023-12-27 15:22:27,480][20292] KL-divergence is very high: 122.6903 [2023-12-27 15:22:27,505][20292] KL-divergence is very high: 128.9650 [2023-12-27 15:22:27,509][20381] Updated weights for policy 1, policy_version 1402637 (0.0005) [2023-12-27 15:22:27,693][20347] Updated weights for policy 0, policy_version 1403788 (0.0007) [2023-12-27 15:22:27,742][20347] Updated weights for policy 0, policy_version 1403798 (0.0007) [2023-12-27 15:22:27,788][20347] Updated weights for policy 0, policy_version 1403808 (0.0008) [2023-12-27 15:22:28,070][20381] Updated weights for policy 1, policy_version 1402647 (0.0007) [2023-12-27 15:22:28,127][20381] Updated weights for policy 1, policy_version 1402657 (0.0007) [2023-12-27 15:22:28,171][20381] Updated weights for policy 1, policy_version 1402667 (0.0007) [2023-12-27 15:22:28,452][20347] Updated weights for policy 0, policy_version 1403818 (0.0008) [2023-12-27 15:22:28,509][20347] Updated weights for policy 0, policy_version 1403828 (0.0007) [2023-12-27 15:22:28,567][20347] Updated weights for policy 0, policy_version 1403838 (0.0008) [2023-12-27 15:22:28,632][20347] Updated weights for policy 0, policy_version 1403848 (0.0008) [2023-12-27 15:22:28,857][20381] Updated weights for policy 1, policy_version 1402677 (0.0006) [2023-12-27 15:22:28,905][20381] Updated weights for policy 1, policy_version 1402687 (0.0005) [2023-12-27 15:22:28,950][20381] Updated weights for policy 1, policy_version 1402697 (0.0005) [2023-12-27 15:22:29,393][20347] Updated weights for policy 0, policy_version 1403858 (0.0009) [2023-12-27 15:22:29,445][20347] Updated weights for policy 0, policy_version 1403868 (0.0007) [2023-12-27 15:22:29,504][20347] Updated weights for policy 0, policy_version 1403878 (0.0007) [2023-12-27 15:22:29,526][20381] Updated weights for policy 1, policy_version 1402707 (0.0005) [2023-12-27 15:22:29,587][20381] Updated weights for policy 1, policy_version 1402717 (0.0005) [2023-12-27 15:22:29,646][20381] Updated weights for policy 1, policy_version 1402727 (0.0006) [2023-12-27 15:22:30,124][20347] Updated weights for policy 0, policy_version 1403888 (0.0006) [2023-12-27 15:22:30,164][20291] KL-divergence is very high: 846.6960 [2023-12-27 15:22:30,186][20347] Updated weights for policy 0, policy_version 1403898 (0.0005) [2023-12-27 15:22:30,224][20381] Updated weights for policy 1, policy_version 1402737 (0.0006) [2023-12-27 15:22:30,246][20347] Updated weights for policy 0, policy_version 1403908 (0.0007) [2023-12-27 15:22:30,258][20291] KL-divergence is very high: 162.1603 [2023-12-27 15:22:30,288][20381] Updated weights for policy 1, policy_version 1402747 (0.0005) [2023-12-27 15:22:30,348][20381] Updated weights for policy 1, policy_version 1402757 (0.0005) [2023-12-27 15:22:30,410][20381] Updated weights for policy 1, policy_version 1402767 (0.0007) [2023-12-27 15:22:30,893][20347] Updated weights for policy 0, policy_version 1403918 (0.0008) [2023-12-27 15:22:30,936][20381] Updated weights for policy 1, policy_version 1402777 (0.0006) [2023-12-27 15:22:30,949][20347] Updated weights for policy 0, policy_version 1403928 (0.0011) [2023-12-27 15:22:30,980][20381] Updated weights for policy 1, policy_version 1402787 (0.0005) [2023-12-27 15:22:31,003][20347] Updated weights for policy 0, policy_version 1403938 (0.0007) [2023-12-27 15:22:31,038][20381] Updated weights for policy 1, policy_version 1402797 (0.0007) [2023-12-27 15:22:31,710][20347] Updated weights for policy 0, policy_version 1403948 (0.0007) [2023-12-27 15:22:31,764][20381] Updated weights for policy 1, policy_version 1402807 (0.0007) [2023-12-27 15:22:31,776][20347] Updated weights for policy 0, policy_version 1403958 (0.0010) [2023-12-27 15:22:31,818][20381] Updated weights for policy 1, policy_version 1402817 (0.0007) [2023-12-27 15:22:31,836][20347] Updated weights for policy 0, policy_version 1403968 (0.0009) [2023-12-27 15:22:31,874][20381] Updated weights for policy 1, policy_version 1402827 (0.0007) [2023-12-27 15:22:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 718643200. Throughput: 0: 10365.0, 1: 10226.0. Samples: 718655236. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:32,363][19170] Avg episode reward: [(0, '970.730'), (1, '980.600')] [2023-12-27 15:22:32,527][20347] Updated weights for policy 0, policy_version 1403978 (0.0006) [2023-12-27 15:22:32,573][20381] Updated weights for policy 1, policy_version 1402837 (0.0006) [2023-12-27 15:22:32,587][20347] Updated weights for policy 0, policy_version 1403988 (0.0006) [2023-12-27 15:22:32,635][20381] Updated weights for policy 1, policy_version 1402847 (0.0006) [2023-12-27 15:22:32,648][20347] Updated weights for policy 0, policy_version 1403998 (0.0006) [2023-12-27 15:22:32,688][20381] Updated weights for policy 1, policy_version 1402857 (0.0005) [2023-12-27 15:22:32,701][20347] Updated weights for policy 0, policy_version 1404008 (0.0006) [2023-12-27 15:22:33,289][20347] Updated weights for policy 0, policy_version 1404018 (0.0007) [2023-12-27 15:22:33,335][20347] Updated weights for policy 0, policy_version 1404028 (0.0007) [2023-12-27 15:22:33,377][20381] Updated weights for policy 1, policy_version 1402867 (0.0008) [2023-12-27 15:22:33,383][20347] Updated weights for policy 0, policy_version 1404038 (0.0006) [2023-12-27 15:22:33,431][20381] Updated weights for policy 1, policy_version 1402877 (0.0007) [2023-12-27 15:22:33,484][20381] Updated weights for policy 1, policy_version 1402887 (0.0007) [2023-12-27 15:22:34,021][20347] Updated weights for policy 0, policy_version 1404048 (0.0006) [2023-12-27 15:22:34,078][20347] Updated weights for policy 0, policy_version 1404058 (0.0009) [2023-12-27 15:22:34,143][20347] Updated weights for policy 0, policy_version 1404068 (0.0007) [2023-12-27 15:22:34,205][20381] Updated weights for policy 1, policy_version 1402897 (0.0008) [2023-12-27 15:22:34,259][20381] Updated weights for policy 1, policy_version 1402907 (0.0006) [2023-12-27 15:22:34,318][20381] Updated weights for policy 1, policy_version 1402917 (0.0007) [2023-12-27 15:22:34,378][20381] Updated weights for policy 1, policy_version 1402927 (0.0008) [2023-12-27 15:22:34,814][20347] Updated weights for policy 0, policy_version 1404078 (0.0007) [2023-12-27 15:22:34,863][20347] Updated weights for policy 0, policy_version 1404088 (0.0005) [2023-12-27 15:22:34,910][20347] Updated weights for policy 0, policy_version 1404098 (0.0005) [2023-12-27 15:22:35,108][20381] Updated weights for policy 1, policy_version 1402937 (0.0007) [2023-12-27 15:22:35,160][20381] Updated weights for policy 1, policy_version 1402947 (0.0007) [2023-12-27 15:22:35,213][20381] Updated weights for policy 1, policy_version 1402957 (0.0007) [2023-12-27 15:22:35,470][20347] Updated weights for policy 0, policy_version 1404108 (0.0006) [2023-12-27 15:22:35,549][20347] Updated weights for policy 0, policy_version 1404118 (0.0007) [2023-12-27 15:22:35,606][20347] Updated weights for policy 0, policy_version 1404128 (0.0007) [2023-12-27 15:22:35,856][20381] Updated weights for policy 1, policy_version 1402967 (0.0007) [2023-12-27 15:22:35,917][20381] Updated weights for policy 1, policy_version 1402977 (0.0005) [2023-12-27 15:22:35,962][20381] Updated weights for policy 1, policy_version 1402987 (0.0008) [2023-12-27 15:22:36,273][20347] Updated weights for policy 0, policy_version 1404138 (0.0009) [2023-12-27 15:22:36,333][20347] Updated weights for policy 0, policy_version 1404148 (0.0008) [2023-12-27 15:22:36,391][20347] Updated weights for policy 0, policy_version 1404158 (0.0007) [2023-12-27 15:22:36,453][20347] Updated weights for policy 0, policy_version 1404168 (0.0007) [2023-12-27 15:22:36,629][20381] Updated weights for policy 1, policy_version 1402997 (0.0008) [2023-12-27 15:22:36,690][20381] Updated weights for policy 1, policy_version 1403007 (0.0008) [2023-12-27 15:22:36,747][20381] Updated weights for policy 1, policy_version 1403017 (0.0008) [2023-12-27 15:22:37,177][20347] Updated weights for policy 0, policy_version 1404178 (0.0007) [2023-12-27 15:22:37,242][20347] Updated weights for policy 0, policy_version 1404188 (0.0008) [2023-12-27 15:22:37,291][20347] Updated weights for policy 0, policy_version 1404198 (0.0008) [2023-12-27 15:22:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 718749696. Throughput: 0: 10373.7, 1: 10187.2. Samples: 718716656. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:37,363][19170] Avg episode reward: [(0, '951.200'), (1, '1000.000')] [2023-12-27 15:22:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001404200_359546880.pth... [2023-12-27 15:22:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001402984_359235584.pth [2023-12-27 15:22:37,389][20381] Updated weights for policy 1, policy_version 1403027 (0.0007) [2023-12-27 15:22:37,432][20381] Updated weights for policy 1, policy_version 1403037 (0.0008) [2023-12-27 15:22:37,485][20381] Updated weights for policy 1, policy_version 1403047 (0.0007) [2023-12-27 15:22:37,534][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001403056_359211008.pth... [2023-12-27 15:22:37,539][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001401840_358899712.pth [2023-12-27 15:22:37,970][20347] Updated weights for policy 0, policy_version 1404208 (0.0008) [2023-12-27 15:22:38,021][20347] Updated weights for policy 0, policy_version 1404218 (0.0007) [2023-12-27 15:22:38,084][20347] Updated weights for policy 0, policy_version 1404228 (0.0007) [2023-12-27 15:22:38,204][20381] Updated weights for policy 1, policy_version 1403057 (0.0008) [2023-12-27 15:22:38,255][20381] Updated weights for policy 1, policy_version 1403067 (0.0008) [2023-12-27 15:22:38,301][20381] Updated weights for policy 1, policy_version 1403077 (0.0007) [2023-12-27 15:22:38,373][20381] Updated weights for policy 1, policy_version 1403087 (0.0008) [2023-12-27 15:22:38,740][20347] Updated weights for policy 0, policy_version 1404238 (0.0008) [2023-12-27 15:22:38,796][20347] Updated weights for policy 0, policy_version 1404248 (0.0007) [2023-12-27 15:22:38,852][20347] Updated weights for policy 0, policy_version 1404258 (0.0008) [2023-12-27 15:22:39,086][20381] Updated weights for policy 1, policy_version 1403097 (0.0010) [2023-12-27 15:22:39,135][20381] Updated weights for policy 1, policy_version 1403107 (0.0009) [2023-12-27 15:22:39,181][20381] Updated weights for policy 1, policy_version 1403117 (0.0007) [2023-12-27 15:22:39,572][20347] Updated weights for policy 0, policy_version 1404268 (0.0007) [2023-12-27 15:22:39,632][20347] Updated weights for policy 0, policy_version 1404278 (0.0007) [2023-12-27 15:22:39,694][20347] Updated weights for policy 0, policy_version 1404288 (0.0007) [2023-12-27 15:22:39,980][20381] Updated weights for policy 1, policy_version 1403127 (0.0008) [2023-12-27 15:22:40,039][20381] Updated weights for policy 1, policy_version 1403137 (0.0008) [2023-12-27 15:22:40,103][20381] Updated weights for policy 1, policy_version 1403147 (0.0008) [2023-12-27 15:22:40,396][20347] Updated weights for policy 0, policy_version 1404298 (0.0008) [2023-12-27 15:22:40,451][20347] Updated weights for policy 0, policy_version 1404308 (0.0006) [2023-12-27 15:22:40,505][20347] Updated weights for policy 0, policy_version 1404318 (0.0008) [2023-12-27 15:22:40,551][20347] Updated weights for policy 0, policy_version 1404328 (0.0008) [2023-12-27 15:22:40,823][20381] Updated weights for policy 1, policy_version 1403157 (0.0007) [2023-12-27 15:22:40,889][20381] Updated weights for policy 1, policy_version 1403167 (0.0008) [2023-12-27 15:22:40,949][20381] Updated weights for policy 1, policy_version 1403177 (0.0008) [2023-12-27 15:22:41,229][20347] Updated weights for policy 0, policy_version 1404338 (0.0006) [2023-12-27 15:22:41,294][20347] Updated weights for policy 0, policy_version 1404348 (0.0007) [2023-12-27 15:22:41,361][20347] Updated weights for policy 0, policy_version 1404358 (0.0008) [2023-12-27 15:22:41,686][20381] Updated weights for policy 1, policy_version 1403187 (0.0007) [2023-12-27 15:22:41,758][20381] Updated weights for policy 1, policy_version 1403197 (0.0007) [2023-12-27 15:22:41,826][20381] Updated weights for policy 1, policy_version 1403207 (0.0007) [2023-12-27 15:22:42,126][20347] Updated weights for policy 0, policy_version 1404368 (0.0008) [2023-12-27 15:22:42,185][20347] Updated weights for policy 0, policy_version 1404378 (0.0007) [2023-12-27 15:22:42,247][20347] Updated weights for policy 0, policy_version 1404388 (0.0008) [2023-12-27 15:22:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 718848000. Throughput: 0: 10318.1, 1: 10202.5. Samples: 718837892. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:42,363][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 15:22:42,523][20381] Updated weights for policy 1, policy_version 1403217 (0.0007) [2023-12-27 15:22:42,579][20381] Updated weights for policy 1, policy_version 1403227 (0.0007) [2023-12-27 15:22:42,641][20381] Updated weights for policy 1, policy_version 1403237 (0.0007) [2023-12-27 15:22:42,696][20381] Updated weights for policy 1, policy_version 1403247 (0.0008) [2023-12-27 15:22:42,960][20347] Updated weights for policy 0, policy_version 1404398 (0.0007) [2023-12-27 15:22:43,012][20347] Updated weights for policy 0, policy_version 1404408 (0.0007) [2023-12-27 15:22:43,074][20347] Updated weights for policy 0, policy_version 1404418 (0.0008) [2023-12-27 15:22:43,331][20381] Updated weights for policy 1, policy_version 1403257 (0.0007) [2023-12-27 15:22:43,381][20381] Updated weights for policy 1, policy_version 1403267 (0.0007) [2023-12-27 15:22:43,438][20381] Updated weights for policy 1, policy_version 1403277 (0.0007) [2023-12-27 15:22:43,792][20347] Updated weights for policy 0, policy_version 1404428 (0.0007) [2023-12-27 15:22:43,856][20347] Updated weights for policy 0, policy_version 1404438 (0.0008) [2023-12-27 15:22:43,910][20347] Updated weights for policy 0, policy_version 1404448 (0.0008) [2023-12-27 15:22:44,048][20381] Updated weights for policy 1, policy_version 1403287 (0.0008) [2023-12-27 15:22:44,119][20381] Updated weights for policy 1, policy_version 1403297 (0.0008) [2023-12-27 15:22:44,175][20381] Updated weights for policy 1, policy_version 1403307 (0.0007) [2023-12-27 15:22:44,540][20347] Updated weights for policy 0, policy_version 1404458 (0.0009) [2023-12-27 15:22:44,590][20347] Updated weights for policy 0, policy_version 1404468 (0.0007) [2023-12-27 15:22:44,635][20347] Updated weights for policy 0, policy_version 1404478 (0.0008) [2023-12-27 15:22:44,684][20347] Updated weights for policy 0, policy_version 1404488 (0.0007) [2023-12-27 15:22:44,891][20381] Updated weights for policy 1, policy_version 1403317 (0.0008) [2023-12-27 15:22:44,954][20381] Updated weights for policy 1, policy_version 1403327 (0.0008) [2023-12-27 15:22:45,008][20381] Updated weights for policy 1, policy_version 1403337 (0.0008) [2023-12-27 15:22:45,452][20347] Updated weights for policy 0, policy_version 1404498 (0.0008) [2023-12-27 15:22:45,497][20347] Updated weights for policy 0, policy_version 1404508 (0.0008) [2023-12-27 15:22:45,545][20347] Updated weights for policy 0, policy_version 1404518 (0.0007) [2023-12-27 15:22:45,686][20381] Updated weights for policy 1, policy_version 1403347 (0.0007) [2023-12-27 15:22:45,746][20381] Updated weights for policy 1, policy_version 1403357 (0.0008) [2023-12-27 15:22:45,796][20381] Updated weights for policy 1, policy_version 1403367 (0.0008) [2023-12-27 15:22:46,125][20347] Updated weights for policy 0, policy_version 1404528 (0.0007) [2023-12-27 15:22:46,182][20347] Updated weights for policy 0, policy_version 1404538 (0.0006) [2023-12-27 15:22:46,238][20347] Updated weights for policy 0, policy_version 1404548 (0.0005) [2023-12-27 15:22:46,541][20381] Updated weights for policy 1, policy_version 1403377 (0.0007) [2023-12-27 15:22:46,599][20381] Updated weights for policy 1, policy_version 1403387 (0.0007) [2023-12-27 15:22:46,648][20381] Updated weights for policy 1, policy_version 1403397 (0.0007) [2023-12-27 15:22:46,691][20381] Updated weights for policy 1, policy_version 1403407 (0.0008) [2023-12-27 15:22:46,825][20347] Updated weights for policy 0, policy_version 1404558 (0.0006) [2023-12-27 15:22:46,879][20347] Updated weights for policy 0, policy_version 1404568 (0.0007) [2023-12-27 15:22:46,925][20347] Updated weights for policy 0, policy_version 1404578 (0.0005) [2023-12-27 15:22:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 718946304. Throughput: 0: 10349.5, 1: 10226.7. Samples: 718959632. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.240')] [2023-12-27 15:22:47,414][20381] Updated weights for policy 1, policy_version 1403417 (0.0007) [2023-12-27 15:22:47,466][20381] Updated weights for policy 1, policy_version 1403427 (0.0008) [2023-12-27 15:22:47,511][20347] Updated weights for policy 0, policy_version 1404588 (0.0005) [2023-12-27 15:22:47,513][20381] Updated weights for policy 1, policy_version 1403437 (0.0007) [2023-12-27 15:22:47,567][20347] Updated weights for policy 0, policy_version 1404598 (0.0008) [2023-12-27 15:22:47,629][20347] Updated weights for policy 0, policy_version 1404608 (0.0007) [2023-12-27 15:22:48,196][20381] Updated weights for policy 1, policy_version 1403447 (0.0008) [2023-12-27 15:22:48,259][20381] Updated weights for policy 1, policy_version 1403457 (0.0008) [2023-12-27 15:22:48,301][20347] Updated weights for policy 0, policy_version 1404618 (0.0008) [2023-12-27 15:22:48,325][20381] Updated weights for policy 1, policy_version 1403467 (0.0007) [2023-12-27 15:22:48,368][20347] Updated weights for policy 0, policy_version 1404628 (0.0008) [2023-12-27 15:22:48,425][20347] Updated weights for policy 0, policy_version 1404638 (0.0005) [2023-12-27 15:22:48,491][20347] Updated weights for policy 0, policy_version 1404648 (0.0005) [2023-12-27 15:22:49,059][20381] Updated weights for policy 1, policy_version 1403477 (0.0008) [2023-12-27 15:22:49,084][20347] Updated weights for policy 0, policy_version 1404658 (0.0006) [2023-12-27 15:22:49,114][20381] Updated weights for policy 1, policy_version 1403487 (0.0006) [2023-12-27 15:22:49,135][20347] Updated weights for policy 0, policy_version 1404668 (0.0006) [2023-12-27 15:22:49,164][20381] Updated weights for policy 1, policy_version 1403497 (0.0006) [2023-12-27 15:22:49,188][20347] Updated weights for policy 0, policy_version 1404678 (0.0005) [2023-12-27 15:22:49,898][20381] Updated weights for policy 1, policy_version 1403507 (0.0006) [2023-12-27 15:22:49,911][20347] Updated weights for policy 0, policy_version 1404688 (0.0009) [2023-12-27 15:22:49,966][20381] Updated weights for policy 1, policy_version 1403517 (0.0009) [2023-12-27 15:22:49,973][20347] Updated weights for policy 0, policy_version 1404698 (0.0007) [2023-12-27 15:22:50,018][20381] Updated weights for policy 1, policy_version 1403527 (0.0009) [2023-12-27 15:22:50,033][20347] Updated weights for policy 0, policy_version 1404708 (0.0008) [2023-12-27 15:22:50,667][20347] Updated weights for policy 0, policy_version 1404718 (0.0009) [2023-12-27 15:22:50,712][20381] Updated weights for policy 1, policy_version 1403537 (0.0008) [2023-12-27 15:22:50,728][20347] Updated weights for policy 0, policy_version 1404728 (0.0008) [2023-12-27 15:22:50,772][20381] Updated weights for policy 1, policy_version 1403547 (0.0006) [2023-12-27 15:22:50,791][20347] Updated weights for policy 0, policy_version 1404738 (0.0007) [2023-12-27 15:22:50,834][20381] Updated weights for policy 1, policy_version 1403557 (0.0006) [2023-12-27 15:22:50,893][20381] Updated weights for policy 1, policy_version 1403567 (0.0008) [2023-12-27 15:22:51,475][20347] Updated weights for policy 0, policy_version 1404748 (0.0007) [2023-12-27 15:22:51,532][20347] Updated weights for policy 0, policy_version 1404758 (0.0008) [2023-12-27 15:22:51,589][20381] Updated weights for policy 1, policy_version 1403577 (0.0006) [2023-12-27 15:22:51,590][20347] Updated weights for policy 0, policy_version 1404768 (0.0007) [2023-12-27 15:22:51,648][20381] Updated weights for policy 1, policy_version 1403587 (0.0007) [2023-12-27 15:22:51,703][20381] Updated weights for policy 1, policy_version 1403597 (0.0006) [2023-12-27 15:22:52,226][20347] Updated weights for policy 0, policy_version 1404778 (0.0007) [2023-12-27 15:22:52,270][20291] KL-divergence is very high: 170.6535 [2023-12-27 15:22:52,287][20347] Updated weights for policy 0, policy_version 1404788 (0.0010) [2023-12-27 15:22:52,300][20291] KL-divergence is very high: 1588.5366 [2023-12-27 15:22:52,317][20291] KL-divergence is very high: 714.3931 [2023-12-27 15:22:52,336][20381] Updated weights for policy 1, policy_version 1403607 (0.0007) [2023-12-27 15:22:52,346][20347] Updated weights for policy 0, policy_version 1404798 (0.0008) [2023-12-27 15:22:52,349][20291] KL-divergence is very high: 1157.5931 [2023-12-27 15:22:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 719044608. Throughput: 0: 10368.5, 1: 10153.5. Samples: 719019932. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:52,363][19170] Avg episode reward: [(0, '970.750'), (1, '980.500')] [2023-12-27 15:22:52,369][20291] KL-divergence is very high: 249.9339 [2023-12-27 15:22:52,401][20291] KL-divergence is very high: 283.1515 [2023-12-27 15:22:52,401][20381] Updated weights for policy 1, policy_version 1403617 (0.0006) [2023-12-27 15:22:52,412][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001404808_359702528.pth... [2023-12-27 15:22:52,414][20347] Updated weights for policy 0, policy_version 1404808 (0.0009) [2023-12-27 15:22:52,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001403592_359391232.pth [2023-12-27 15:22:52,457][20381] Updated weights for policy 1, policy_version 1403627 (0.0007) [2023-12-27 15:22:52,490][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001403632_359358464.pth... [2023-12-27 15:22:52,494][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001402416_359047168.pth [2023-12-27 15:22:53,058][20347] Updated weights for policy 0, policy_version 1404818 (0.0007) [2023-12-27 15:22:53,107][20347] Updated weights for policy 0, policy_version 1404828 (0.0008) [2023-12-27 15:22:53,151][20381] Updated weights for policy 1, policy_version 1403637 (0.0005) [2023-12-27 15:22:53,164][20347] Updated weights for policy 0, policy_version 1404838 (0.0007) [2023-12-27 15:22:53,202][20381] Updated weights for policy 1, policy_version 1403647 (0.0007) [2023-12-27 15:22:53,253][20381] Updated weights for policy 1, policy_version 1403657 (0.0008) [2023-12-27 15:22:53,766][20347] Updated weights for policy 0, policy_version 1404848 (0.0006) [2023-12-27 15:22:53,813][20347] Updated weights for policy 0, policy_version 1404858 (0.0006) [2023-12-27 15:22:53,864][20347] Updated weights for policy 0, policy_version 1404868 (0.0005) [2023-12-27 15:22:53,978][20381] Updated weights for policy 1, policy_version 1403667 (0.0008) [2023-12-27 15:22:54,023][20381] Updated weights for policy 1, policy_version 1403677 (0.0008) [2023-12-27 15:22:54,078][20381] Updated weights for policy 1, policy_version 1403687 (0.0007) [2023-12-27 15:22:54,552][20347] Updated weights for policy 0, policy_version 1404878 (0.0009) [2023-12-27 15:22:54,614][20347] Updated weights for policy 0, policy_version 1404888 (0.0010) [2023-12-27 15:22:54,665][20347] Updated weights for policy 0, policy_version 1404898 (0.0010) [2023-12-27 15:22:54,725][20381] Updated weights for policy 1, policy_version 1403697 (0.0008) [2023-12-27 15:22:54,771][20381] Updated weights for policy 1, policy_version 1403707 (0.0009) [2023-12-27 15:22:54,816][20381] Updated weights for policy 1, policy_version 1403717 (0.0010) [2023-12-27 15:22:54,864][20381] Updated weights for policy 1, policy_version 1403727 (0.0010) [2023-12-27 15:22:55,270][20347] Updated weights for policy 0, policy_version 1404908 (0.0008) [2023-12-27 15:22:55,316][20347] Updated weights for policy 0, policy_version 1404918 (0.0009) [2023-12-27 15:22:55,365][20347] Updated weights for policy 0, policy_version 1404928 (0.0010) [2023-12-27 15:22:55,557][20381] Updated weights for policy 1, policy_version 1403737 (0.0007) [2023-12-27 15:22:55,615][20381] Updated weights for policy 1, policy_version 1403747 (0.0007) [2023-12-27 15:22:55,665][20381] Updated weights for policy 1, policy_version 1403757 (0.0007) [2023-12-27 15:22:56,040][20347] Updated weights for policy 0, policy_version 1404938 (0.0009) [2023-12-27 15:22:56,093][20347] Updated weights for policy 0, policy_version 1404948 (0.0010) [2023-12-27 15:22:56,147][20347] Updated weights for policy 0, policy_version 1404958 (0.0010) [2023-12-27 15:22:56,181][20291] KL-divergence is very high: 106.4547 [2023-12-27 15:22:56,205][20347] Updated weights for policy 0, policy_version 1404968 (0.0010) [2023-12-27 15:22:56,322][20381] Updated weights for policy 1, policy_version 1403767 (0.0006) [2023-12-27 15:22:56,367][20381] Updated weights for policy 1, policy_version 1403777 (0.0006) [2023-12-27 15:22:56,428][20381] Updated weights for policy 1, policy_version 1403787 (0.0006) [2023-12-27 15:22:56,882][20347] Updated weights for policy 0, policy_version 1404978 (0.0008) [2023-12-27 15:22:56,931][20347] Updated weights for policy 0, policy_version 1404988 (0.0010) [2023-12-27 15:22:56,983][20347] Updated weights for policy 0, policy_version 1404998 (0.0010) [2023-12-27 15:22:57,008][20381] Updated weights for policy 1, policy_version 1403797 (0.0008) [2023-12-27 15:22:57,062][20381] Updated weights for policy 1, policy_version 1403807 (0.0005) [2023-12-27 15:22:57,107][20381] Updated weights for policy 1, policy_version 1403817 (0.0006) [2023-12-27 15:22:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 719159296. Throughput: 0: 10398.4, 1: 10181.5. Samples: 719146452. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:22:57,363][19170] Avg episode reward: [(0, '951.170'), (1, '990.260')] [2023-12-27 15:22:57,669][20347] Updated weights for policy 0, policy_version 1405008 (0.0010) [2023-12-27 15:22:57,730][20347] Updated weights for policy 0, policy_version 1405018 (0.0010) [2023-12-27 15:22:57,760][20381] Updated weights for policy 1, policy_version 1403827 (0.0007) [2023-12-27 15:22:57,787][20347] Updated weights for policy 0, policy_version 1405028 (0.0010) [2023-12-27 15:22:57,808][20381] Updated weights for policy 1, policy_version 1403837 (0.0005) [2023-12-27 15:22:57,856][20381] Updated weights for policy 1, policy_version 1403847 (0.0007) [2023-12-27 15:22:58,526][20347] Updated weights for policy 0, policy_version 1405038 (0.0008) [2023-12-27 15:22:58,588][20347] Updated weights for policy 0, policy_version 1405048 (0.0008) [2023-12-27 15:22:58,618][20381] Updated weights for policy 1, policy_version 1403857 (0.0006) [2023-12-27 15:22:58,654][20347] Updated weights for policy 0, policy_version 1405058 (0.0008) [2023-12-27 15:22:58,676][20381] Updated weights for policy 1, policy_version 1403867 (0.0006) [2023-12-27 15:22:58,746][20381] Updated weights for policy 1, policy_version 1403877 (0.0008) [2023-12-27 15:22:58,809][20381] Updated weights for policy 1, policy_version 1403887 (0.0007) [2023-12-27 15:22:59,444][20347] Updated weights for policy 0, policy_version 1405068 (0.0009) [2023-12-27 15:22:59,510][20347] Updated weights for policy 0, policy_version 1405078 (0.0008) [2023-12-27 15:22:59,557][20381] Updated weights for policy 1, policy_version 1403897 (0.0007) [2023-12-27 15:22:59,567][20347] Updated weights for policy 0, policy_version 1405088 (0.0005) [2023-12-27 15:22:59,616][20381] Updated weights for policy 1, policy_version 1403907 (0.0006) [2023-12-27 15:22:59,665][20381] Updated weights for policy 1, policy_version 1403917 (0.0007) [2023-12-27 15:23:00,236][20347] Updated weights for policy 0, policy_version 1405098 (0.0007) [2023-12-27 15:23:00,289][20347] Updated weights for policy 0, policy_version 1405108 (0.0008) [2023-12-27 15:23:00,345][20347] Updated weights for policy 0, policy_version 1405118 (0.0006) [2023-12-27 15:23:00,392][20381] Updated weights for policy 1, policy_version 1403927 (0.0008) [2023-12-27 15:23:00,406][20347] Updated weights for policy 0, policy_version 1405128 (0.0006) [2023-12-27 15:23:00,456][20381] Updated weights for policy 1, policy_version 1403937 (0.0007) [2023-12-27 15:23:00,510][20381] Updated weights for policy 1, policy_version 1403947 (0.0007) [2023-12-27 15:23:00,978][20347] Updated weights for policy 0, policy_version 1405138 (0.0008) [2023-12-27 15:23:01,037][20347] Updated weights for policy 0, policy_version 1405148 (0.0007) [2023-12-27 15:23:01,091][20347] Updated weights for policy 0, policy_version 1405158 (0.0007) [2023-12-27 15:23:01,188][20381] Updated weights for policy 1, policy_version 1403957 (0.0006) [2023-12-27 15:23:01,248][20381] Updated weights for policy 1, policy_version 1403967 (0.0008) [2023-12-27 15:23:01,307][20381] Updated weights for policy 1, policy_version 1403977 (0.0005) [2023-12-27 15:23:01,827][20347] Updated weights for policy 0, policy_version 1405168 (0.0009) [2023-12-27 15:23:01,880][20347] Updated weights for policy 0, policy_version 1405178 (0.0009) [2023-12-27 15:23:01,938][20347] Updated weights for policy 0, policy_version 1405188 (0.0008) [2023-12-27 15:23:02,002][20381] Updated weights for policy 1, policy_version 1403987 (0.0006) [2023-12-27 15:23:02,049][20381] Updated weights for policy 1, policy_version 1403997 (0.0007) [2023-12-27 15:23:02,100][20381] Updated weights for policy 1, policy_version 1404007 (0.0008) [2023-12-27 15:23:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 719257600. Throughput: 0: 10288.7, 1: 10206.8. Samples: 719265776. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:02,363][19170] Avg episode reward: [(0, '902.430'), (1, '961.010')] [2023-12-27 15:23:02,560][20347] Updated weights for policy 0, policy_version 1405198 (0.0007) [2023-12-27 15:23:02,615][20347] Updated weights for policy 0, policy_version 1405208 (0.0008) [2023-12-27 15:23:02,671][20347] Updated weights for policy 0, policy_version 1405218 (0.0007) [2023-12-27 15:23:02,872][20381] Updated weights for policy 1, policy_version 1404017 (0.0008) [2023-12-27 15:23:02,930][20381] Updated weights for policy 1, policy_version 1404027 (0.0008) [2023-12-27 15:23:02,990][20381] Updated weights for policy 1, policy_version 1404037 (0.0009) [2023-12-27 15:23:03,048][20381] Updated weights for policy 1, policy_version 1404047 (0.0008) [2023-12-27 15:23:03,299][20347] Updated weights for policy 0, policy_version 1405228 (0.0006) [2023-12-27 15:23:03,310][20291] KL-divergence is very high: 117.4063 [2023-12-27 15:23:03,352][20347] Updated weights for policy 0, policy_version 1405238 (0.0005) [2023-12-27 15:23:03,403][20347] Updated weights for policy 0, policy_version 1405248 (0.0005) [2023-12-27 15:23:03,783][20381] Updated weights for policy 1, policy_version 1404057 (0.0006) [2023-12-27 15:23:03,839][20381] Updated weights for policy 1, policy_version 1404067 (0.0006) [2023-12-27 15:23:03,906][20381] Updated weights for policy 1, policy_version 1404077 (0.0007) [2023-12-27 15:23:04,064][20347] Updated weights for policy 0, policy_version 1405258 (0.0007) [2023-12-27 15:23:04,122][20347] Updated weights for policy 0, policy_version 1405268 (0.0009) [2023-12-27 15:23:04,183][20347] Updated weights for policy 0, policy_version 1405278 (0.0008) [2023-12-27 15:23:04,248][20347] Updated weights for policy 0, policy_version 1405288 (0.0006) [2023-12-27 15:23:04,577][20381] Updated weights for policy 1, policy_version 1404087 (0.0006) [2023-12-27 15:23:04,636][20381] Updated weights for policy 1, policy_version 1404097 (0.0006) [2023-12-27 15:23:04,694][20381] Updated weights for policy 1, policy_version 1404107 (0.0006) [2023-12-27 15:23:04,920][20347] Updated weights for policy 0, policy_version 1405298 (0.0007) [2023-12-27 15:23:04,979][20347] Updated weights for policy 0, policy_version 1405308 (0.0005) [2023-12-27 15:23:05,035][20347] Updated weights for policy 0, policy_version 1405318 (0.0006) [2023-12-27 15:23:05,317][20381] Updated weights for policy 1, policy_version 1404117 (0.0007) [2023-12-27 15:23:05,363][20381] Updated weights for policy 1, policy_version 1404127 (0.0008) [2023-12-27 15:23:05,415][20381] Updated weights for policy 1, policy_version 1404137 (0.0008) [2023-12-27 15:23:05,695][20347] Updated weights for policy 0, policy_version 1405328 (0.0008) [2023-12-27 15:23:05,754][20347] Updated weights for policy 0, policy_version 1405338 (0.0010) [2023-12-27 15:23:05,803][20347] Updated weights for policy 0, policy_version 1405348 (0.0010) [2023-12-27 15:23:06,137][20381] Updated weights for policy 1, policy_version 1404147 (0.0009) [2023-12-27 15:23:06,198][20381] Updated weights for policy 1, policy_version 1404157 (0.0007) [2023-12-27 15:23:06,262][20381] Updated weights for policy 1, policy_version 1404167 (0.0009) [2023-12-27 15:23:06,473][20347] Updated weights for policy 0, policy_version 1405358 (0.0007) [2023-12-27 15:23:06,532][20347] Updated weights for policy 0, policy_version 1405368 (0.0006) [2023-12-27 15:23:06,591][20347] Updated weights for policy 0, policy_version 1405378 (0.0010) [2023-12-27 15:23:06,907][20381] Updated weights for policy 1, policy_version 1404177 (0.0008) [2023-12-27 15:23:06,972][20381] Updated weights for policy 1, policy_version 1404187 (0.0011) [2023-12-27 15:23:07,022][20381] Updated weights for policy 1, policy_version 1404197 (0.0011) [2023-12-27 15:23:07,072][20381] Updated weights for policy 1, policy_version 1404207 (0.0011) [2023-12-27 15:23:07,186][20347] Updated weights for policy 0, policy_version 1405388 (0.0011) [2023-12-27 15:23:07,247][20347] Updated weights for policy 0, policy_version 1405398 (0.0010) [2023-12-27 15:23:07,299][20347] Updated weights for policy 0, policy_version 1405408 (0.0010) [2023-12-27 15:23:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 719364096. Throughput: 0: 10294.3, 1: 10188.5. Samples: 719328112. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:07,363][19170] Avg episode reward: [(0, '834.090'), (1, '941.520')] [2023-12-27 15:23:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001404208_359505920.pth... [2023-12-27 15:23:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001405416_359858176.pth... [2023-12-27 15:23:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001403056_359211008.pth [2023-12-27 15:23:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001404200_359546880.pth [2023-12-27 15:23:07,736][20381] Updated weights for policy 1, policy_version 1404217 (0.0008) [2023-12-27 15:23:07,801][20381] Updated weights for policy 1, policy_version 1404227 (0.0007) [2023-12-27 15:23:07,847][20381] Updated weights for policy 1, policy_version 1404237 (0.0007) [2023-12-27 15:23:07,919][20347] Updated weights for policy 0, policy_version 1405418 (0.0009) [2023-12-27 15:23:07,976][20347] Updated weights for policy 0, policy_version 1405428 (0.0010) [2023-12-27 15:23:08,034][20347] Updated weights for policy 0, policy_version 1405438 (0.0009) [2023-12-27 15:23:08,085][20347] Updated weights for policy 0, policy_version 1405448 (0.0010) [2023-12-27 15:23:08,535][20381] Updated weights for policy 1, policy_version 1404247 (0.0008) [2023-12-27 15:23:08,594][20381] Updated weights for policy 1, policy_version 1404257 (0.0007) [2023-12-27 15:23:08,648][20381] Updated weights for policy 1, policy_version 1404267 (0.0007) [2023-12-27 15:23:08,747][20347] Updated weights for policy 0, policy_version 1405458 (0.0009) [2023-12-27 15:23:08,806][20347] Updated weights for policy 0, policy_version 1405468 (0.0005) [2023-12-27 15:23:08,861][20347] Updated weights for policy 0, policy_version 1405478 (0.0005) [2023-12-27 15:23:09,360][20381] Updated weights for policy 1, policy_version 1404277 (0.0007) [2023-12-27 15:23:09,427][20381] Updated weights for policy 1, policy_version 1404287 (0.0007) [2023-12-27 15:23:09,487][20381] Updated weights for policy 1, policy_version 1404297 (0.0008) [2023-12-27 15:23:09,525][20347] Updated weights for policy 0, policy_version 1405488 (0.0008) [2023-12-27 15:23:09,574][20347] Updated weights for policy 0, policy_version 1405498 (0.0009) [2023-12-27 15:23:09,627][20347] Updated weights for policy 0, policy_version 1405508 (0.0009) [2023-12-27 15:23:10,225][20381] Updated weights for policy 1, policy_version 1404307 (0.0008) [2023-12-27 15:23:10,277][20381] Updated weights for policy 1, policy_version 1404317 (0.0007) [2023-12-27 15:23:10,338][20381] Updated weights for policy 1, policy_version 1404327 (0.0007) [2023-12-27 15:23:10,372][20347] Updated weights for policy 0, policy_version 1405518 (0.0009) [2023-12-27 15:23:10,428][20347] Updated weights for policy 0, policy_version 1405528 (0.0008) [2023-12-27 15:23:10,486][20347] Updated weights for policy 0, policy_version 1405538 (0.0006) [2023-12-27 15:23:10,958][20381] Updated weights for policy 1, policy_version 1404337 (0.0010) [2023-12-27 15:23:11,017][20381] Updated weights for policy 1, policy_version 1404347 (0.0006) [2023-12-27 15:23:11,082][20381] Updated weights for policy 1, policy_version 1404357 (0.0008) [2023-12-27 15:23:11,139][20347] Updated weights for policy 0, policy_version 1405548 (0.0006) [2023-12-27 15:23:11,151][20381] Updated weights for policy 1, policy_version 1404367 (0.0011) [2023-12-27 15:23:11,203][20347] Updated weights for policy 0, policy_version 1405558 (0.0008) [2023-12-27 15:23:11,268][20347] Updated weights for policy 0, policy_version 1405568 (0.0008) [2023-12-27 15:23:11,920][20381] Updated weights for policy 1, policy_version 1404377 (0.0011) [2023-12-27 15:23:11,981][20381] Updated weights for policy 1, policy_version 1404387 (0.0011) [2023-12-27 15:23:11,991][20347] Updated weights for policy 0, policy_version 1405578 (0.0006) [2023-12-27 15:23:12,039][20381] Updated weights for policy 1, policy_version 1404397 (0.0010) [2023-12-27 15:23:12,048][20347] Updated weights for policy 0, policy_version 1405588 (0.0006) [2023-12-27 15:23:12,100][20347] Updated weights for policy 0, policy_version 1405598 (0.0007) [2023-12-27 15:23:12,150][20347] Updated weights for policy 0, policy_version 1405608 (0.0007) [2023-12-27 15:23:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 719462400. Throughput: 0: 10366.2, 1: 10153.3. Samples: 719450916. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:12,363][19170] Avg episode reward: [(0, '843.580'), (1, '951.270')] [2023-12-27 15:23:12,714][20381] Updated weights for policy 1, policy_version 1404407 (0.0008) [2023-12-27 15:23:12,771][20381] Updated weights for policy 1, policy_version 1404417 (0.0005) [2023-12-27 15:23:12,831][20381] Updated weights for policy 1, policy_version 1404427 (0.0005) [2023-12-27 15:23:12,915][20347] Updated weights for policy 0, policy_version 1405618 (0.0008) [2023-12-27 15:23:12,971][20347] Updated weights for policy 0, policy_version 1405628 (0.0008) [2023-12-27 15:23:13,019][20347] Updated weights for policy 0, policy_version 1405638 (0.0007) [2023-12-27 15:23:13,414][20381] Updated weights for policy 1, policy_version 1404437 (0.0006) [2023-12-27 15:23:13,468][20381] Updated weights for policy 1, policy_version 1404447 (0.0005) [2023-12-27 15:23:13,530][20381] Updated weights for policy 1, policy_version 1404457 (0.0006) [2023-12-27 15:23:13,705][20347] Updated weights for policy 0, policy_version 1405648 (0.0008) [2023-12-27 15:23:13,755][20347] Updated weights for policy 0, policy_version 1405658 (0.0007) [2023-12-27 15:23:13,807][20347] Updated weights for policy 0, policy_version 1405668 (0.0008) [2023-12-27 15:23:14,119][20381] Updated weights for policy 1, policy_version 1404467 (0.0007) [2023-12-27 15:23:14,174][20381] Updated weights for policy 1, policy_version 1404477 (0.0006) [2023-12-27 15:23:14,233][20381] Updated weights for policy 1, policy_version 1404487 (0.0008) [2023-12-27 15:23:14,488][20347] Updated weights for policy 0, policy_version 1405678 (0.0008) [2023-12-27 15:23:14,543][20347] Updated weights for policy 0, policy_version 1405688 (0.0007) [2023-12-27 15:23:14,606][20347] Updated weights for policy 0, policy_version 1405698 (0.0007) [2023-12-27 15:23:14,882][20381] Updated weights for policy 1, policy_version 1404497 (0.0007) [2023-12-27 15:23:14,942][20381] Updated weights for policy 1, policy_version 1404507 (0.0006) [2023-12-27 15:23:14,998][20381] Updated weights for policy 1, policy_version 1404517 (0.0005) [2023-12-27 15:23:15,058][20381] Updated weights for policy 1, policy_version 1404527 (0.0007) [2023-12-27 15:23:15,364][20347] Updated weights for policy 0, policy_version 1405708 (0.0008) [2023-12-27 15:23:15,422][20347] Updated weights for policy 0, policy_version 1405718 (0.0007) [2023-12-27 15:23:15,482][20347] Updated weights for policy 0, policy_version 1405728 (0.0008) [2023-12-27 15:23:15,712][20381] Updated weights for policy 1, policy_version 1404537 (0.0007) [2023-12-27 15:23:15,766][20381] Updated weights for policy 1, policy_version 1404547 (0.0008) [2023-12-27 15:23:15,826][20381] Updated weights for policy 1, policy_version 1404557 (0.0007) [2023-12-27 15:23:16,173][20347] Updated weights for policy 0, policy_version 1405738 (0.0008) [2023-12-27 15:23:16,236][20347] Updated weights for policy 0, policy_version 1405748 (0.0008) [2023-12-27 15:23:16,299][20347] Updated weights for policy 0, policy_version 1405758 (0.0005) [2023-12-27 15:23:16,353][20347] Updated weights for policy 0, policy_version 1405768 (0.0005) [2023-12-27 15:23:16,461][20381] Updated weights for policy 1, policy_version 1404567 (0.0009) [2023-12-27 15:23:16,508][20381] Updated weights for policy 1, policy_version 1404577 (0.0007) [2023-12-27 15:23:16,558][20381] Updated weights for policy 1, policy_version 1404587 (0.0006) [2023-12-27 15:23:16,898][20347] Updated weights for policy 0, policy_version 1405778 (0.0007) [2023-12-27 15:23:16,947][20347] Updated weights for policy 0, policy_version 1405788 (0.0007) [2023-12-27 15:23:16,999][20347] Updated weights for policy 0, policy_version 1405798 (0.0007) [2023-12-27 15:23:17,272][20381] Updated weights for policy 1, policy_version 1404597 (0.0007) [2023-12-27 15:23:17,333][20381] Updated weights for policy 1, policy_version 1404607 (0.0007) [2023-12-27 15:23:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 719560704. Throughput: 0: 10333.1, 1: 10081.3. Samples: 719573884. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:17,363][19170] Avg episode reward: [(0, '882.550'), (1, '941.670')] [2023-12-27 15:23:17,388][20381] Updated weights for policy 1, policy_version 1404617 (0.0008) [2023-12-27 15:23:17,696][20347] Updated weights for policy 0, policy_version 1405808 (0.0006) [2023-12-27 15:23:17,757][20347] Updated weights for policy 0, policy_version 1405818 (0.0007) [2023-12-27 15:23:17,816][20347] Updated weights for policy 0, policy_version 1405828 (0.0008) [2023-12-27 15:23:18,029][20381] Updated weights for policy 1, policy_version 1404627 (0.0007) [2023-12-27 15:23:18,078][20381] Updated weights for policy 1, policy_version 1404637 (0.0005) [2023-12-27 15:23:18,129][20381] Updated weights for policy 1, policy_version 1404647 (0.0005) [2023-12-27 15:23:18,428][20347] Updated weights for policy 0, policy_version 1405838 (0.0009) [2023-12-27 15:23:18,485][20347] Updated weights for policy 0, policy_version 1405848 (0.0010) [2023-12-27 15:23:18,542][20347] Updated weights for policy 0, policy_version 1405858 (0.0010) [2023-12-27 15:23:18,715][20381] Updated weights for policy 1, policy_version 1404657 (0.0009) [2023-12-27 15:23:18,771][20381] Updated weights for policy 1, policy_version 1404667 (0.0007) [2023-12-27 15:23:18,819][20381] Updated weights for policy 1, policy_version 1404677 (0.0008) [2023-12-27 15:23:18,868][20381] Updated weights for policy 1, policy_version 1404687 (0.0007) [2023-12-27 15:23:19,164][20347] Updated weights for policy 0, policy_version 1405868 (0.0010) [2023-12-27 15:23:19,222][20347] Updated weights for policy 0, policy_version 1405878 (0.0008) [2023-12-27 15:23:19,288][20347] Updated weights for policy 0, policy_version 1405888 (0.0008) [2023-12-27 15:23:19,633][20381] Updated weights for policy 1, policy_version 1404697 (0.0008) [2023-12-27 15:23:19,691][20381] Updated weights for policy 1, policy_version 1404707 (0.0006) [2023-12-27 15:23:19,740][20381] Updated weights for policy 1, policy_version 1404717 (0.0007) [2023-12-27 15:23:20,065][20347] Updated weights for policy 0, policy_version 1405898 (0.0008) [2023-12-27 15:23:20,121][20347] Updated weights for policy 0, policy_version 1405908 (0.0008) [2023-12-27 15:23:20,176][20347] Updated weights for policy 0, policy_version 1405918 (0.0008) [2023-12-27 15:23:20,236][20347] Updated weights for policy 0, policy_version 1405928 (0.0007) [2023-12-27 15:23:20,441][20381] Updated weights for policy 1, policy_version 1404727 (0.0007) [2023-12-27 15:23:20,498][20381] Updated weights for policy 1, policy_version 1404737 (0.0007) [2023-12-27 15:23:20,558][20381] Updated weights for policy 1, policy_version 1404747 (0.0008) [2023-12-27 15:23:20,969][20347] Updated weights for policy 0, policy_version 1405938 (0.0011) [2023-12-27 15:23:21,034][20347] Updated weights for policy 0, policy_version 1405948 (0.0011) [2023-12-27 15:23:21,100][20347] Updated weights for policy 0, policy_version 1405958 (0.0011) [2023-12-27 15:23:21,318][20381] Updated weights for policy 1, policy_version 1404757 (0.0008) [2023-12-27 15:23:21,386][20381] Updated weights for policy 1, policy_version 1404767 (0.0010) [2023-12-27 15:23:21,447][20381] Updated weights for policy 1, policy_version 1404777 (0.0007) [2023-12-27 15:23:21,877][20347] Updated weights for policy 0, policy_version 1405968 (0.0008) [2023-12-27 15:23:21,904][20291] KL-divergence is very high: 110.4608 [2023-12-27 15:23:21,928][20347] Updated weights for policy 0, policy_version 1405978 (0.0005) [2023-12-27 15:23:21,942][20291] KL-divergence is very high: 152.3690 [2023-12-27 15:23:21,978][20347] Updated weights for policy 0, policy_version 1405988 (0.0008) [2023-12-27 15:23:22,154][20381] Updated weights for policy 1, policy_version 1404787 (0.0008) [2023-12-27 15:23:22,211][20381] Updated weights for policy 1, policy_version 1404797 (0.0007) [2023-12-27 15:23:22,277][20381] Updated weights for policy 1, policy_version 1404807 (0.0007) [2023-12-27 15:23:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 719667200. Throughput: 0: 10301.4, 1: 10132.5. Samples: 719636184. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:22,363][19170] Avg episode reward: [(0, '882.760'), (1, '961.110')] [2023-12-27 15:23:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001404816_359661568.pth... [2023-12-27 15:23:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001405992_360005632.pth... [2023-12-27 15:23:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001404808_359702528.pth [2023-12-27 15:23:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001403632_359358464.pth [2023-12-27 15:23:22,712][20347] Updated weights for policy 0, policy_version 1405998 (0.0009) [2023-12-27 15:23:22,768][20347] Updated weights for policy 0, policy_version 1406008 (0.0007) [2023-12-27 15:23:22,824][20347] Updated weights for policy 0, policy_version 1406018 (0.0009) [2023-12-27 15:23:22,965][20381] Updated weights for policy 1, policy_version 1404817 (0.0008) [2023-12-27 15:23:23,017][20381] Updated weights for policy 1, policy_version 1404827 (0.0007) [2023-12-27 15:23:23,079][20381] Updated weights for policy 1, policy_version 1404837 (0.0007) [2023-12-27 15:23:23,131][20381] Updated weights for policy 1, policy_version 1404847 (0.0007) [2023-12-27 15:23:23,465][20347] Updated weights for policy 0, policy_version 1406028 (0.0008) [2023-12-27 15:23:23,530][20347] Updated weights for policy 0, policy_version 1406038 (0.0005) [2023-12-27 15:23:23,590][20347] Updated weights for policy 0, policy_version 1406048 (0.0006) [2023-12-27 15:23:23,894][20381] Updated weights for policy 1, policy_version 1404857 (0.0007) [2023-12-27 15:23:23,958][20381] Updated weights for policy 1, policy_version 1404867 (0.0007) [2023-12-27 15:23:24,007][20381] Updated weights for policy 1, policy_version 1404877 (0.0006) [2023-12-27 15:23:24,124][20347] Updated weights for policy 0, policy_version 1406058 (0.0006) [2023-12-27 15:23:24,179][20347] Updated weights for policy 0, policy_version 1406068 (0.0010) [2023-12-27 15:23:24,231][20347] Updated weights for policy 0, policy_version 1406078 (0.0010) [2023-12-27 15:23:24,279][20347] Updated weights for policy 0, policy_version 1406088 (0.0010) [2023-12-27 15:23:24,707][20381] Updated weights for policy 1, policy_version 1404887 (0.0007) [2023-12-27 15:23:24,752][20381] Updated weights for policy 1, policy_version 1404897 (0.0008) [2023-12-27 15:23:24,798][20381] Updated weights for policy 1, policy_version 1404907 (0.0007) [2023-12-27 15:23:24,938][20347] Updated weights for policy 0, policy_version 1406098 (0.0010) [2023-12-27 15:23:24,983][20347] Updated weights for policy 0, policy_version 1406108 (0.0010) [2023-12-27 15:23:25,034][20347] Updated weights for policy 0, policy_version 1406118 (0.0010) [2023-12-27 15:23:25,423][20381] Updated weights for policy 1, policy_version 1404917 (0.0005) [2023-12-27 15:23:25,471][20381] Updated weights for policy 1, policy_version 1404927 (0.0005) [2023-12-27 15:23:25,515][20381] Updated weights for policy 1, policy_version 1404937 (0.0006) [2023-12-27 15:23:25,733][20347] Updated weights for policy 0, policy_version 1406128 (0.0007) [2023-12-27 15:23:25,785][20347] Updated weights for policy 0, policy_version 1406138 (0.0006) [2023-12-27 15:23:25,837][20347] Updated weights for policy 0, policy_version 1406148 (0.0010) [2023-12-27 15:23:26,110][20381] Updated weights for policy 1, policy_version 1404947 (0.0009) [2023-12-27 15:23:26,169][20381] Updated weights for policy 1, policy_version 1404957 (0.0005) [2023-12-27 15:23:26,213][20381] Updated weights for policy 1, policy_version 1404967 (0.0005) [2023-12-27 15:23:26,500][20347] Updated weights for policy 0, policy_version 1406158 (0.0010) [2023-12-27 15:23:26,545][20347] Updated weights for policy 0, policy_version 1406168 (0.0010) [2023-12-27 15:23:26,602][20347] Updated weights for policy 0, policy_version 1406178 (0.0010) [2023-12-27 15:23:26,859][20381] Updated weights for policy 1, policy_version 1404977 (0.0006) [2023-12-27 15:23:26,922][20381] Updated weights for policy 1, policy_version 1404987 (0.0010) [2023-12-27 15:23:26,978][20381] Updated weights for policy 1, policy_version 1404997 (0.0010) [2023-12-27 15:23:27,033][20381] Updated weights for policy 1, policy_version 1405007 (0.0010) [2023-12-27 15:23:27,278][20347] Updated weights for policy 0, policy_version 1406188 (0.0010) [2023-12-27 15:23:27,341][20347] Updated weights for policy 0, policy_version 1406198 (0.0007) [2023-12-27 15:23:27,341][20291] KL-divergence is very high: 355.0849 [2023-12-27 15:23:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 719765504. Throughput: 0: 10292.9, 1: 10151.4. Samples: 719757884. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:27,363][19170] Avg episode reward: [(0, '873.170'), (1, '990.320')] [2023-12-27 15:23:27,387][20291] KL-divergence is very high: 354.3258 [2023-12-27 15:23:27,400][20347] Updated weights for policy 0, policy_version 1406208 (0.0008) [2023-12-27 15:23:27,439][20291] KL-divergence is very high: 130.1507 [2023-12-27 15:23:27,646][20381] Updated weights for policy 1, policy_version 1405017 (0.0007) [2023-12-27 15:23:27,694][20381] Updated weights for policy 1, policy_version 1405027 (0.0007) [2023-12-27 15:23:27,749][20381] Updated weights for policy 1, policy_version 1405037 (0.0008) [2023-12-27 15:23:28,067][20347] Updated weights for policy 0, policy_version 1406218 (0.0009) [2023-12-27 15:23:28,114][20347] Updated weights for policy 0, policy_version 1406228 (0.0008) [2023-12-27 15:23:28,166][20347] Updated weights for policy 0, policy_version 1406238 (0.0009) [2023-12-27 15:23:28,216][20347] Updated weights for policy 0, policy_version 1406248 (0.0006) [2023-12-27 15:23:28,374][20381] Updated weights for policy 1, policy_version 1405047 (0.0007) [2023-12-27 15:23:28,436][20381] Updated weights for policy 1, policy_version 1405057 (0.0008) [2023-12-27 15:23:28,492][20381] Updated weights for policy 1, policy_version 1405067 (0.0008) [2023-12-27 15:23:28,796][20291] KL-divergence is very high: 127.9905 [2023-12-27 15:23:28,817][20291] KL-divergence is very high: 201.7299 [2023-12-27 15:23:28,833][20291] KL-divergence is very high: 316.1986 [2023-12-27 15:23:28,840][20347] Updated weights for policy 0, policy_version 1406258 (0.0006) [2023-12-27 15:23:28,843][20291] KL-divergence is very high: 484.8858 [2023-12-27 15:23:28,858][20291] KL-divergence is very high: 186.1186 [2023-12-27 15:23:28,884][20291] KL-divergence is very high: 104.7037 [2023-12-27 15:23:28,889][20347] Updated weights for policy 0, policy_version 1406268 (0.0008) [2023-12-27 15:23:28,940][20347] Updated weights for policy 0, policy_version 1406278 (0.0007) [2023-12-27 15:23:29,197][20381] Updated weights for policy 1, policy_version 1405077 (0.0007) [2023-12-27 15:23:29,258][20381] Updated weights for policy 1, policy_version 1405087 (0.0007) [2023-12-27 15:23:29,321][20381] Updated weights for policy 1, policy_version 1405097 (0.0007) [2023-12-27 15:23:29,625][20291] KL-divergence is very high: 489.4413 [2023-12-27 15:23:29,631][20291] KL-divergence is very high: 220.8165 [2023-12-27 15:23:29,670][20291] KL-divergence is very high: 299.5513 [2023-12-27 15:23:29,671][20347] Updated weights for policy 0, policy_version 1406288 (0.0007) [2023-12-27 15:23:29,676][20291] KL-divergence is very high: 104.5582 [2023-12-27 15:23:29,718][20291] KL-divergence is very high: 203.8844 [2023-12-27 15:23:29,731][20347] Updated weights for policy 0, policy_version 1406298 (0.0008) [2023-12-27 15:23:29,766][20291] KL-divergence is very high: 218.9186 [2023-12-27 15:23:29,788][20347] Updated weights for policy 0, policy_version 1406308 (0.0010) [2023-12-27 15:23:30,016][20381] Updated weights for policy 1, policy_version 1405107 (0.0008) [2023-12-27 15:23:30,068][20381] Updated weights for policy 1, policy_version 1405117 (0.0008) [2023-12-27 15:23:30,120][20381] Updated weights for policy 1, policy_version 1405127 (0.0007) [2023-12-27 15:23:30,527][20347] Updated weights for policy 0, policy_version 1406318 (0.0010) [2023-12-27 15:23:30,585][20347] Updated weights for policy 0, policy_version 1406328 (0.0010) [2023-12-27 15:23:30,634][20347] Updated weights for policy 0, policy_version 1406338 (0.0010) [2023-12-27 15:23:30,800][20381] Updated weights for policy 1, policy_version 1405137 (0.0007) [2023-12-27 15:23:30,855][20381] Updated weights for policy 1, policy_version 1405147 (0.0005) [2023-12-27 15:23:30,914][20381] Updated weights for policy 1, policy_version 1405157 (0.0007) [2023-12-27 15:23:30,964][20381] Updated weights for policy 1, policy_version 1405167 (0.0010) [2023-12-27 15:23:31,314][20347] Updated weights for policy 0, policy_version 1406348 (0.0009) [2023-12-27 15:23:31,384][20347] Updated weights for policy 0, policy_version 1406358 (0.0008) [2023-12-27 15:23:31,436][20347] Updated weights for policy 0, policy_version 1406368 (0.0008) [2023-12-27 15:23:31,631][20381] Updated weights for policy 1, policy_version 1405177 (0.0007) [2023-12-27 15:23:31,688][20381] Updated weights for policy 1, policy_version 1405187 (0.0007) [2023-12-27 15:23:31,754][20381] Updated weights for policy 1, policy_version 1405197 (0.0008) [2023-12-27 15:23:32,175][20347] Updated weights for policy 0, policy_version 1406378 (0.0008) [2023-12-27 15:23:32,234][20347] Updated weights for policy 0, policy_version 1406388 (0.0007) [2023-12-27 15:23:32,294][20347] Updated weights for policy 0, policy_version 1406398 (0.0007) [2023-12-27 15:23:32,349][20347] Updated weights for policy 0, policy_version 1406408 (0.0008) [2023-12-27 15:23:32,359][20381] Updated weights for policy 1, policy_version 1405207 (0.0007) [2023-12-27 15:23:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 719872000. Throughput: 0: 10272.6, 1: 10202.1. Samples: 719880996. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:32,363][19170] Avg episode reward: [(0, '814.360'), (1, '961.010')] [2023-12-27 15:23:32,418][20381] Updated weights for policy 1, policy_version 1405217 (0.0007) [2023-12-27 15:23:32,476][20381] Updated weights for policy 1, policy_version 1405227 (0.0007) [2023-12-27 15:23:33,106][20347] Updated weights for policy 0, policy_version 1406418 (0.0005) [2023-12-27 15:23:33,114][20381] Updated weights for policy 1, policy_version 1405237 (0.0007) [2023-12-27 15:23:33,162][20381] Updated weights for policy 1, policy_version 1405247 (0.0007) [2023-12-27 15:23:33,166][20347] Updated weights for policy 0, policy_version 1406428 (0.0005) [2023-12-27 15:23:33,210][20381] Updated weights for policy 1, policy_version 1405257 (0.0007) [2023-12-27 15:23:33,227][20347] Updated weights for policy 0, policy_version 1406438 (0.0005) [2023-12-27 15:23:33,849][20381] Updated weights for policy 1, policy_version 1405267 (0.0007) [2023-12-27 15:23:33,883][20347] Updated weights for policy 0, policy_version 1406448 (0.0005) [2023-12-27 15:23:33,900][20381] Updated weights for policy 1, policy_version 1405277 (0.0007) [2023-12-27 15:23:33,932][20347] Updated weights for policy 0, policy_version 1406458 (0.0005) [2023-12-27 15:23:33,955][20381] Updated weights for policy 1, policy_version 1405287 (0.0007) [2023-12-27 15:23:33,982][20347] Updated weights for policy 0, policy_version 1406468 (0.0007) [2023-12-27 15:23:34,679][20347] Updated weights for policy 0, policy_version 1406478 (0.0007) [2023-12-27 15:23:34,702][20381] Updated weights for policy 1, policy_version 1405297 (0.0006) [2023-12-27 15:23:34,737][20291] KL-divergence is very high: 139.9587 [2023-12-27 15:23:34,738][20347] Updated weights for policy 0, policy_version 1406488 (0.0008) [2023-12-27 15:23:34,756][20381] Updated weights for policy 1, policy_version 1405307 (0.0007) [2023-12-27 15:23:34,784][20291] KL-divergence is very high: 102.3962 [2023-12-27 15:23:34,794][20347] Updated weights for policy 0, policy_version 1406498 (0.0007) [2023-12-27 15:23:34,807][20381] Updated weights for policy 1, policy_version 1405317 (0.0007) [2023-12-27 15:23:34,851][20381] Updated weights for policy 1, policy_version 1405327 (0.0006) [2023-12-27 15:23:35,438][20347] Updated weights for policy 0, policy_version 1406508 (0.0005) [2023-12-27 15:23:35,487][20347] Updated weights for policy 0, policy_version 1406518 (0.0005) [2023-12-27 15:23:35,538][20347] Updated weights for policy 0, policy_version 1406528 (0.0005) [2023-12-27 15:23:35,542][20381] Updated weights for policy 1, policy_version 1405337 (0.0007) [2023-12-27 15:23:35,600][20381] Updated weights for policy 1, policy_version 1405347 (0.0007) [2023-12-27 15:23:35,655][20381] Updated weights for policy 1, policy_version 1405357 (0.0005) [2023-12-27 15:23:36,180][20347] Updated weights for policy 0, policy_version 1406538 (0.0006) [2023-12-27 15:23:36,239][20347] Updated weights for policy 0, policy_version 1406548 (0.0008) [2023-12-27 15:23:36,303][20347] Updated weights for policy 0, policy_version 1406558 (0.0007) [2023-12-27 15:23:36,314][20381] Updated weights for policy 1, policy_version 1405367 (0.0008) [2023-12-27 15:23:36,369][20347] Updated weights for policy 0, policy_version 1406568 (0.0007) [2023-12-27 15:23:36,372][20381] Updated weights for policy 1, policy_version 1405377 (0.0008) [2023-12-27 15:23:36,431][20381] Updated weights for policy 1, policy_version 1405387 (0.0011) [2023-12-27 15:23:37,062][20347] Updated weights for policy 0, policy_version 1406578 (0.0008) [2023-12-27 15:23:37,066][20381] Updated weights for policy 1, policy_version 1405397 (0.0008) [2023-12-27 15:23:37,118][20347] Updated weights for policy 0, policy_version 1406588 (0.0009) [2023-12-27 15:23:37,121][20381] Updated weights for policy 1, policy_version 1405407 (0.0011) [2023-12-27 15:23:37,172][20347] Updated weights for policy 0, policy_version 1406598 (0.0006) [2023-12-27 15:23:37,174][20381] Updated weights for policy 1, policy_version 1405417 (0.0010) [2023-12-27 15:23:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.3). Total num frames: 719978496. Throughput: 0: 10256.3, 1: 10238.6. Samples: 719942200. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:37,363][19170] Avg episode reward: [(0, '834.250'), (1, '961.010')] [2023-12-27 15:23:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001405424_359817216.pth... [2023-12-27 15:23:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001406600_360161280.pth... [2023-12-27 15:23:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001404208_359505920.pth [2023-12-27 15:23:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001405416_359858176.pth [2023-12-27 15:23:37,812][20347] Updated weights for policy 0, policy_version 1406608 (0.0005) [2023-12-27 15:23:37,867][20381] Updated weights for policy 1, policy_version 1405427 (0.0010) [2023-12-27 15:23:37,879][20347] Updated weights for policy 0, policy_version 1406618 (0.0005) [2023-12-27 15:23:37,923][20381] Updated weights for policy 1, policy_version 1405437 (0.0010) [2023-12-27 15:23:37,942][20347] Updated weights for policy 0, policy_version 1406628 (0.0005) [2023-12-27 15:23:37,978][20381] Updated weights for policy 1, policy_version 1405447 (0.0011) [2023-12-27 15:23:38,589][20347] Updated weights for policy 0, policy_version 1406638 (0.0007) [2023-12-27 15:23:38,619][20381] Updated weights for policy 1, policy_version 1405457 (0.0010) [2023-12-27 15:23:38,644][20347] Updated weights for policy 0, policy_version 1406648 (0.0006) [2023-12-27 15:23:38,676][20381] Updated weights for policy 1, policy_version 1405467 (0.0005) [2023-12-27 15:23:38,704][20347] Updated weights for policy 0, policy_version 1406658 (0.0007) [2023-12-27 15:23:38,734][20381] Updated weights for policy 1, policy_version 1405477 (0.0007) [2023-12-27 15:23:38,791][20381] Updated weights for policy 1, policy_version 1405487 (0.0007) [2023-12-27 15:23:39,409][20347] Updated weights for policy 0, policy_version 1406668 (0.0007) [2023-12-27 15:23:39,466][20347] Updated weights for policy 0, policy_version 1406678 (0.0008) [2023-12-27 15:23:39,510][20381] Updated weights for policy 1, policy_version 1405497 (0.0006) [2023-12-27 15:23:39,522][20347] Updated weights for policy 0, policy_version 1406688 (0.0008) [2023-12-27 15:23:39,570][20381] Updated weights for policy 1, policy_version 1405507 (0.0005) [2023-12-27 15:23:39,629][20381] Updated weights for policy 1, policy_version 1405517 (0.0007) [2023-12-27 15:23:40,299][20381] Updated weights for policy 1, policy_version 1405527 (0.0010) [2023-12-27 15:23:40,317][20347] Updated weights for policy 0, policy_version 1406698 (0.0007) [2023-12-27 15:23:40,357][20381] Updated weights for policy 1, policy_version 1405537 (0.0011) [2023-12-27 15:23:40,376][20347] Updated weights for policy 0, policy_version 1406708 (0.0006) [2023-12-27 15:23:40,415][20381] Updated weights for policy 1, policy_version 1405547 (0.0011) [2023-12-27 15:23:40,426][20347] Updated weights for policy 0, policy_version 1406718 (0.0007) [2023-12-27 15:23:40,484][20347] Updated weights for policy 0, policy_version 1406728 (0.0008) [2023-12-27 15:23:41,015][20381] Updated weights for policy 1, policy_version 1405557 (0.0011) [2023-12-27 15:23:41,091][20381] Updated weights for policy 1, policy_version 1405567 (0.0008) [2023-12-27 15:23:41,159][20381] Updated weights for policy 1, policy_version 1405577 (0.0014) [2023-12-27 15:23:41,167][20347] Updated weights for policy 0, policy_version 1406738 (0.0009) [2023-12-27 15:23:41,229][20347] Updated weights for policy 0, policy_version 1406748 (0.0007) [2023-12-27 15:23:41,291][20347] Updated weights for policy 0, policy_version 1406758 (0.0006) [2023-12-27 15:23:41,865][20381] Updated weights for policy 1, policy_version 1405587 (0.0010) [2023-12-27 15:23:41,925][20381] Updated weights for policy 1, policy_version 1405597 (0.0011) [2023-12-27 15:23:41,986][20381] Updated weights for policy 1, policy_version 1405607 (0.0011) [2023-12-27 15:23:41,995][20347] Updated weights for policy 0, policy_version 1406768 (0.0008) [2023-12-27 15:23:42,051][20347] Updated weights for policy 0, policy_version 1406778 (0.0008) [2023-12-27 15:23:42,117][20347] Updated weights for policy 0, policy_version 1406788 (0.0008) [2023-12-27 15:23:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 720076800. Throughput: 0: 10176.1, 1: 10256.0. Samples: 720065900. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:42,363][19170] Avg episode reward: [(0, '834.040'), (1, '990.230')] [2023-12-27 15:23:42,661][20381] Updated weights for policy 1, policy_version 1405617 (0.0009) [2023-12-27 15:23:42,713][20381] Updated weights for policy 1, policy_version 1405627 (0.0008) [2023-12-27 15:23:42,772][20381] Updated weights for policy 1, policy_version 1405637 (0.0011) [2023-12-27 15:23:42,835][20381] Updated weights for policy 1, policy_version 1405647 (0.0011) [2023-12-27 15:23:42,837][20347] Updated weights for policy 0, policy_version 1406798 (0.0007) [2023-12-27 15:23:42,881][20347] Updated weights for policy 0, policy_version 1406808 (0.0007) [2023-12-27 15:23:42,931][20347] Updated weights for policy 0, policy_version 1406818 (0.0008) [2023-12-27 15:23:43,527][20381] Updated weights for policy 1, policy_version 1405657 (0.0011) [2023-12-27 15:23:43,585][20381] Updated weights for policy 1, policy_version 1405667 (0.0010) [2023-12-27 15:23:43,603][20347] Updated weights for policy 0, policy_version 1406828 (0.0009) [2023-12-27 15:23:43,637][20381] Updated weights for policy 1, policy_version 1405677 (0.0011) [2023-12-27 15:23:43,656][20347] Updated weights for policy 0, policy_version 1406838 (0.0010) [2023-12-27 15:23:43,714][20347] Updated weights for policy 0, policy_version 1406848 (0.0010) [2023-12-27 15:23:44,256][20381] Updated weights for policy 1, policy_version 1405687 (0.0010) [2023-12-27 15:23:44,313][20347] Updated weights for policy 0, policy_version 1406858 (0.0010) [2023-12-27 15:23:44,321][20381] Updated weights for policy 1, policy_version 1405697 (0.0008) [2023-12-27 15:23:44,370][20347] Updated weights for policy 0, policy_version 1406868 (0.0005) [2023-12-27 15:23:44,382][20381] Updated weights for policy 1, policy_version 1405707 (0.0008) [2023-12-27 15:23:44,431][20347] Updated weights for policy 0, policy_version 1406878 (0.0006) [2023-12-27 15:23:44,486][20347] Updated weights for policy 0, policy_version 1406888 (0.0007) [2023-12-27 15:23:45,102][20347] Updated weights for policy 0, policy_version 1406898 (0.0008) [2023-12-27 15:23:45,156][20381] Updated weights for policy 1, policy_version 1405717 (0.0008) [2023-12-27 15:23:45,165][20347] Updated weights for policy 0, policy_version 1406908 (0.0008) [2023-12-27 15:23:45,219][20381] Updated weights for policy 1, policy_version 1405727 (0.0008) [2023-12-27 15:23:45,229][20347] Updated weights for policy 0, policy_version 1406918 (0.0008) [2023-12-27 15:23:45,284][20381] Updated weights for policy 1, policy_version 1405737 (0.0007) [2023-12-27 15:23:45,904][20347] Updated weights for policy 0, policy_version 1406928 (0.0007) [2023-12-27 15:23:45,913][20381] Updated weights for policy 1, policy_version 1405747 (0.0007) [2023-12-27 15:23:45,957][20347] Updated weights for policy 0, policy_version 1406938 (0.0007) [2023-12-27 15:23:45,962][20381] Updated weights for policy 1, policy_version 1405757 (0.0005) [2023-12-27 15:23:46,006][20347] Updated weights for policy 0, policy_version 1406948 (0.0006) [2023-12-27 15:23:46,015][20381] Updated weights for policy 1, policy_version 1405767 (0.0009) [2023-12-27 15:23:46,617][20381] Updated weights for policy 1, policy_version 1405777 (0.0010) [2023-12-27 15:23:46,640][20347] Updated weights for policy 0, policy_version 1406958 (0.0005) [2023-12-27 15:23:46,668][20381] Updated weights for policy 1, policy_version 1405787 (0.0010) [2023-12-27 15:23:46,693][20347] Updated weights for policy 0, policy_version 1406968 (0.0006) [2023-12-27 15:23:46,719][20381] Updated weights for policy 1, policy_version 1405797 (0.0008) [2023-12-27 15:23:46,748][20347] Updated weights for policy 0, policy_version 1406978 (0.0005) [2023-12-27 15:23:46,766][20381] Updated weights for policy 1, policy_version 1405807 (0.0008) [2023-12-27 15:23:47,358][20347] Updated weights for policy 0, policy_version 1406988 (0.0007) [2023-12-27 15:23:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 720175104. Throughput: 0: 10259.8, 1: 10292.4. Samples: 720190624. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:47,363][19170] Avg episode reward: [(0, '843.580'), (1, '980.480')] [2023-12-27 15:23:47,409][20347] Updated weights for policy 0, policy_version 1406998 (0.0010) [2023-12-27 15:23:47,439][20381] Updated weights for policy 1, policy_version 1405817 (0.0010) [2023-12-27 15:23:47,464][20347] Updated weights for policy 0, policy_version 1407008 (0.0010) [2023-12-27 15:23:47,489][20381] Updated weights for policy 1, policy_version 1405827 (0.0007) [2023-12-27 15:23:47,536][20381] Updated weights for policy 1, policy_version 1405837 (0.0005) [2023-12-27 15:23:48,136][20347] Updated weights for policy 0, policy_version 1407018 (0.0010) [2023-12-27 15:23:48,190][20347] Updated weights for policy 0, policy_version 1407028 (0.0010) [2023-12-27 15:23:48,223][20381] Updated weights for policy 1, policy_version 1405847 (0.0010) [2023-12-27 15:23:48,242][20347] Updated weights for policy 0, policy_version 1407038 (0.0010) [2023-12-27 15:23:48,275][20381] Updated weights for policy 1, policy_version 1405857 (0.0010) [2023-12-27 15:23:48,299][20347] Updated weights for policy 0, policy_version 1407048 (0.0010) [2023-12-27 15:23:48,332][20381] Updated weights for policy 1, policy_version 1405867 (0.0010) [2023-12-27 15:23:48,967][20347] Updated weights for policy 0, policy_version 1407058 (0.0010) [2023-12-27 15:23:49,025][20347] Updated weights for policy 0, policy_version 1407068 (0.0010) [2023-12-27 15:23:49,048][20381] Updated weights for policy 1, policy_version 1405877 (0.0011) [2023-12-27 15:23:49,089][20347] Updated weights for policy 0, policy_version 1407078 (0.0010) [2023-12-27 15:23:49,105][20381] Updated weights for policy 1, policy_version 1405887 (0.0008) [2023-12-27 15:23:49,153][20381] Updated weights for policy 1, policy_version 1405897 (0.0005) [2023-12-27 15:23:49,782][20347] Updated weights for policy 0, policy_version 1407088 (0.0009) [2023-12-27 15:23:49,831][20381] Updated weights for policy 1, policy_version 1405907 (0.0007) [2023-12-27 15:23:49,855][20347] Updated weights for policy 0, policy_version 1407098 (0.0007) [2023-12-27 15:23:49,888][20381] Updated weights for policy 1, policy_version 1405917 (0.0008) [2023-12-27 15:23:49,912][20347] Updated weights for policy 0, policy_version 1407108 (0.0006) [2023-12-27 15:23:49,940][20381] Updated weights for policy 1, policy_version 1405927 (0.0008) [2023-12-27 15:23:50,600][20347] Updated weights for policy 0, policy_version 1407118 (0.0008) [2023-12-27 15:23:50,664][20347] Updated weights for policy 0, policy_version 1407128 (0.0007) [2023-12-27 15:23:50,677][20381] Updated weights for policy 1, policy_version 1405937 (0.0008) [2023-12-27 15:23:50,717][20347] Updated weights for policy 0, policy_version 1407138 (0.0007) [2023-12-27 15:23:50,727][20381] Updated weights for policy 1, policy_version 1405947 (0.0005) [2023-12-27 15:23:50,780][20381] Updated weights for policy 1, policy_version 1405957 (0.0007) [2023-12-27 15:23:50,834][20381] Updated weights for policy 1, policy_version 1405967 (0.0008) [2023-12-27 15:23:51,429][20347] Updated weights for policy 0, policy_version 1407148 (0.0006) [2023-12-27 15:23:51,488][20347] Updated weights for policy 0, policy_version 1407158 (0.0007) [2023-12-27 15:23:51,548][20347] Updated weights for policy 0, policy_version 1407168 (0.0006) [2023-12-27 15:23:51,573][20381] Updated weights for policy 1, policy_version 1405977 (0.0007) [2023-12-27 15:23:51,632][20381] Updated weights for policy 1, policy_version 1405987 (0.0008) [2023-12-27 15:23:51,693][20381] Updated weights for policy 1, policy_version 1405997 (0.0006) [2023-12-27 15:23:52,239][20347] Updated weights for policy 0, policy_version 1407178 (0.0006) [2023-12-27 15:23:52,305][20347] Updated weights for policy 0, policy_version 1407188 (0.0007) [2023-12-27 15:23:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 720273408. Throughput: 0: 10214.8, 1: 10293.7. Samples: 720251000. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:52,363][19170] Avg episode reward: [(0, '892.450'), (1, '980.510')] [2023-12-27 15:23:52,369][20347] Updated weights for policy 0, policy_version 1407198 (0.0007) [2023-12-27 15:23:52,413][20381] Updated weights for policy 1, policy_version 1406007 (0.0008) [2023-12-27 15:23:52,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001407208_360316928.pth... [2023-12-27 15:23:52,431][20347] Updated weights for policy 0, policy_version 1407208 (0.0007) [2023-12-27 15:23:52,435][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001405992_360005632.pth [2023-12-27 15:23:52,472][20381] Updated weights for policy 1, policy_version 1406017 (0.0009) [2023-12-27 15:23:52,520][20381] Updated weights for policy 1, policy_version 1406027 (0.0009) [2023-12-27 15:23:52,544][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001406032_359972864.pth... [2023-12-27 15:23:52,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001404816_359661568.pth [2023-12-27 15:23:53,010][20347] Updated weights for policy 0, policy_version 1407218 (0.0005) [2023-12-27 15:23:53,065][20347] Updated weights for policy 0, policy_version 1407228 (0.0005) [2023-12-27 15:23:53,121][20347] Updated weights for policy 0, policy_version 1407238 (0.0005) [2023-12-27 15:23:53,256][20381] Updated weights for policy 1, policy_version 1406037 (0.0009) [2023-12-27 15:23:53,308][20381] Updated weights for policy 1, policy_version 1406047 (0.0009) [2023-12-27 15:23:53,372][20381] Updated weights for policy 1, policy_version 1406057 (0.0007) [2023-12-27 15:23:53,665][20347] Updated weights for policy 0, policy_version 1407248 (0.0005) [2023-12-27 15:23:53,719][20347] Updated weights for policy 0, policy_version 1407258 (0.0007) [2023-12-27 15:23:53,767][20347] Updated weights for policy 0, policy_version 1407268 (0.0010) [2023-12-27 15:23:54,048][20381] Updated weights for policy 1, policy_version 1406067 (0.0007) [2023-12-27 15:23:54,103][20381] Updated weights for policy 1, policy_version 1406077 (0.0005) [2023-12-27 15:23:54,154][20381] Updated weights for policy 1, policy_version 1406087 (0.0006) [2023-12-27 15:23:54,374][20347] Updated weights for policy 0, policy_version 1407278 (0.0010) [2023-12-27 15:23:54,426][20347] Updated weights for policy 0, policy_version 1407288 (0.0010) [2023-12-27 15:23:54,487][20347] Updated weights for policy 0, policy_version 1407298 (0.0010) [2023-12-27 15:23:54,803][20381] Updated weights for policy 1, policy_version 1406097 (0.0008) [2023-12-27 15:23:54,849][20381] Updated weights for policy 1, policy_version 1406107 (0.0010) [2023-12-27 15:23:54,900][20381] Updated weights for policy 1, policy_version 1406117 (0.0006) [2023-12-27 15:23:54,964][20381] Updated weights for policy 1, policy_version 1406127 (0.0005) [2023-12-27 15:23:55,169][20347] Updated weights for policy 0, policy_version 1407308 (0.0008) [2023-12-27 15:23:55,219][20347] Updated weights for policy 0, policy_version 1407318 (0.0010) [2023-12-27 15:23:55,269][20347] Updated weights for policy 0, policy_version 1407328 (0.0007) [2023-12-27 15:23:55,568][20381] Updated weights for policy 1, policy_version 1406137 (0.0010) [2023-12-27 15:23:55,622][20381] Updated weights for policy 1, policy_version 1406147 (0.0010) [2023-12-27 15:23:55,680][20381] Updated weights for policy 1, policy_version 1406157 (0.0010) [2023-12-27 15:23:55,896][20347] Updated weights for policy 0, policy_version 1407338 (0.0006) [2023-12-27 15:23:55,950][20347] Updated weights for policy 0, policy_version 1407348 (0.0005) [2023-12-27 15:23:56,009][20347] Updated weights for policy 0, policy_version 1407358 (0.0009) [2023-12-27 15:23:56,067][20347] Updated weights for policy 0, policy_version 1407368 (0.0010) [2023-12-27 15:23:56,359][20381] Updated weights for policy 1, policy_version 1406167 (0.0008) [2023-12-27 15:23:56,414][20381] Updated weights for policy 1, policy_version 1406177 (0.0007) [2023-12-27 15:23:56,471][20381] Updated weights for policy 1, policy_version 1406187 (0.0006) [2023-12-27 15:23:56,720][20347] Updated weights for policy 0, policy_version 1407378 (0.0005) [2023-12-27 15:23:56,771][20347] Updated weights for policy 0, policy_version 1407388 (0.0005) [2023-12-27 15:23:56,820][20347] Updated weights for policy 0, policy_version 1407398 (0.0006) [2023-12-27 15:23:57,104][20381] Updated weights for policy 1, policy_version 1406197 (0.0006) [2023-12-27 15:23:57,152][20381] Updated weights for policy 1, policy_version 1406207 (0.0009) [2023-12-27 15:23:57,213][20381] Updated weights for policy 1, policy_version 1406217 (0.0007) [2023-12-27 15:23:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 720388096. Throughput: 0: 10239.7, 1: 10313.7. Samples: 720375816. Policy #0 lag: (min: 16.0, avg: 42.8, max: 48.0) [2023-12-27 15:23:57,363][19170] Avg episode reward: [(0, '951.230'), (1, '990.260')] [2023-12-27 15:23:57,390][20347] Updated weights for policy 0, policy_version 1407408 (0.0007) [2023-12-27 15:23:57,436][20291] KL-divergence is very high: 1491.7317 [2023-12-27 15:23:57,447][20347] Updated weights for policy 0, policy_version 1407418 (0.0007) [2023-12-27 15:23:57,482][20291] KL-divergence is very high: 2588.8247 [2023-12-27 15:23:57,505][20347] Updated weights for policy 0, policy_version 1407428 (0.0007) [2023-12-27 15:23:57,922][20381] Updated weights for policy 1, policy_version 1406227 (0.0007) [2023-12-27 15:23:57,977][20381] Updated weights for policy 1, policy_version 1406237 (0.0005) [2023-12-27 15:23:58,030][20381] Updated weights for policy 1, policy_version 1406247 (0.0005) [2023-12-27 15:23:58,176][20347] Updated weights for policy 0, policy_version 1407438 (0.0009) [2023-12-27 15:23:58,238][20347] Updated weights for policy 0, policy_version 1407448 (0.0008) [2023-12-27 15:23:58,297][20347] Updated weights for policy 0, policy_version 1407458 (0.0008) [2023-12-27 15:23:58,701][20381] Updated weights for policy 1, policy_version 1406257 (0.0006) [2023-12-27 15:23:58,768][20381] Updated weights for policy 1, policy_version 1406267 (0.0007) [2023-12-27 15:23:58,837][20381] Updated weights for policy 1, policy_version 1406277 (0.0007) [2023-12-27 15:23:58,908][20381] Updated weights for policy 1, policy_version 1406287 (0.0007) [2023-12-27 15:23:59,104][20347] Updated weights for policy 0, policy_version 1407468 (0.0008) [2023-12-27 15:23:59,135][20291] KL-divergence is very high: 127.0966 [2023-12-27 15:23:59,165][20347] Updated weights for policy 0, policy_version 1407478 (0.0008) [2023-12-27 15:23:59,186][20291] KL-divergence is very high: 142.5088 [2023-12-27 15:23:59,232][20347] Updated weights for policy 0, policy_version 1407488 (0.0008) [2023-12-27 15:23:59,692][20381] Updated weights for policy 1, policy_version 1406297 (0.0008) [2023-12-27 15:23:59,745][20381] Updated weights for policy 1, policy_version 1406307 (0.0008) [2023-12-27 15:23:59,796][20381] Updated weights for policy 1, policy_version 1406317 (0.0007) [2023-12-27 15:23:59,966][20347] Updated weights for policy 0, policy_version 1407498 (0.0013) [2023-12-27 15:24:00,021][20347] Updated weights for policy 0, policy_version 1407508 (0.0008) [2023-12-27 15:24:00,077][20347] Updated weights for policy 0, policy_version 1407518 (0.0008) [2023-12-27 15:24:00,126][20347] Updated weights for policy 0, policy_version 1407528 (0.0007) [2023-12-27 15:24:00,492][20381] Updated weights for policy 1, policy_version 1406327 (0.0007) [2023-12-27 15:24:00,547][20381] Updated weights for policy 1, policy_version 1406337 (0.0007) [2023-12-27 15:24:00,605][20381] Updated weights for policy 1, policy_version 1406347 (0.0008) [2023-12-27 15:24:00,822][20347] Updated weights for policy 0, policy_version 1407538 (0.0007) [2023-12-27 15:24:00,879][20347] Updated weights for policy 0, policy_version 1407548 (0.0006) [2023-12-27 15:24:00,938][20347] Updated weights for policy 0, policy_version 1407558 (0.0006) [2023-12-27 15:24:01,315][20381] Updated weights for policy 1, policy_version 1406357 (0.0008) [2023-12-27 15:24:01,384][20381] Updated weights for policy 1, policy_version 1406367 (0.0008) [2023-12-27 15:24:01,443][20381] Updated weights for policy 1, policy_version 1406377 (0.0008) [2023-12-27 15:24:01,631][20347] Updated weights for policy 0, policy_version 1407568 (0.0008) [2023-12-27 15:24:01,690][20347] Updated weights for policy 0, policy_version 1407578 (0.0007) [2023-12-27 15:24:01,762][20347] Updated weights for policy 0, policy_version 1407588 (0.0008) [2023-12-27 15:24:02,222][20381] Updated weights for policy 1, policy_version 1406387 (0.0007) [2023-12-27 15:24:02,282][20381] Updated weights for policy 1, policy_version 1406397 (0.0008) [2023-12-27 15:24:02,344][20381] Updated weights for policy 1, policy_version 1406407 (0.0007) [2023-12-27 15:24:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 720478208. Throughput: 0: 10242.2, 1: 10207.9. Samples: 720494140. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:02,363][19170] Avg episode reward: [(0, '912.100'), (1, '990.250')] [2023-12-27 15:24:02,441][20347] Updated weights for policy 0, policy_version 1407598 (0.0008) [2023-12-27 15:24:02,504][20347] Updated weights for policy 0, policy_version 1407608 (0.0008) [2023-12-27 15:24:02,564][20347] Updated weights for policy 0, policy_version 1407618 (0.0008) [2023-12-27 15:24:02,981][20381] Updated weights for policy 1, policy_version 1406417 (0.0007) [2023-12-27 15:24:03,033][20381] Updated weights for policy 1, policy_version 1406427 (0.0008) [2023-12-27 15:24:03,088][20381] Updated weights for policy 1, policy_version 1406437 (0.0007) [2023-12-27 15:24:03,151][20381] Updated weights for policy 1, policy_version 1406447 (0.0008) [2023-12-27 15:24:03,178][20347] Updated weights for policy 0, policy_version 1407628 (0.0005) [2023-12-27 15:24:03,232][20347] Updated weights for policy 0, policy_version 1407638 (0.0006) [2023-12-27 15:24:03,290][20347] Updated weights for policy 0, policy_version 1407648 (0.0006) [2023-12-27 15:24:03,862][20347] Updated weights for policy 0, policy_version 1407658 (0.0006) [2023-12-27 15:24:03,863][20381] Updated weights for policy 1, policy_version 1406457 (0.0007) [2023-12-27 15:24:03,915][20347] Updated weights for policy 0, policy_version 1407668 (0.0008) [2023-12-27 15:24:03,920][20381] Updated weights for policy 1, policy_version 1406467 (0.0007) [2023-12-27 15:24:03,974][20381] Updated weights for policy 1, policy_version 1406477 (0.0008) [2023-12-27 15:24:03,976][20347] Updated weights for policy 0, policy_version 1407678 (0.0009) [2023-12-27 15:24:04,029][20347] Updated weights for policy 0, policy_version 1407688 (0.0006) [2023-12-27 15:24:04,653][20347] Updated weights for policy 0, policy_version 1407698 (0.0008) [2023-12-27 15:24:04,718][20347] Updated weights for policy 0, policy_version 1407708 (0.0007) [2023-12-27 15:24:04,723][20381] Updated weights for policy 1, policy_version 1406487 (0.0008) [2023-12-27 15:24:04,767][20347] Updated weights for policy 0, policy_version 1407718 (0.0006) [2023-12-27 15:24:04,785][20381] Updated weights for policy 1, policy_version 1406497 (0.0007) [2023-12-27 15:24:04,842][20381] Updated weights for policy 1, policy_version 1406507 (0.0007) [2023-12-27 15:24:05,480][20347] Updated weights for policy 0, policy_version 1407728 (0.0008) [2023-12-27 15:24:05,520][20381] Updated weights for policy 1, policy_version 1406517 (0.0007) [2023-12-27 15:24:05,541][20347] Updated weights for policy 0, policy_version 1407738 (0.0008) [2023-12-27 15:24:05,584][20381] Updated weights for policy 1, policy_version 1406527 (0.0007) [2023-12-27 15:24:05,605][20347] Updated weights for policy 0, policy_version 1407748 (0.0008) [2023-12-27 15:24:05,631][20381] Updated weights for policy 1, policy_version 1406537 (0.0006) [2023-12-27 15:24:06,258][20347] Updated weights for policy 0, policy_version 1407758 (0.0007) [2023-12-27 15:24:06,269][20381] Updated weights for policy 1, policy_version 1406547 (0.0008) [2023-12-27 15:24:06,320][20347] Updated weights for policy 0, policy_version 1407768 (0.0008) [2023-12-27 15:24:06,335][20381] Updated weights for policy 1, policy_version 1406557 (0.0009) [2023-12-27 15:24:06,384][20347] Updated weights for policy 0, policy_version 1407778 (0.0008) [2023-12-27 15:24:06,401][20381] Updated weights for policy 1, policy_version 1406567 (0.0008) [2023-12-27 15:24:07,050][20381] Updated weights for policy 1, policy_version 1406577 (0.0008) [2023-12-27 15:24:07,065][20347] Updated weights for policy 0, policy_version 1407788 (0.0007) [2023-12-27 15:24:07,103][20381] Updated weights for policy 1, policy_version 1406587 (0.0006) [2023-12-27 15:24:07,115][20347] Updated weights for policy 0, policy_version 1407798 (0.0005) [2023-12-27 15:24:07,156][20381] Updated weights for policy 1, policy_version 1406597 (0.0011) [2023-12-27 15:24:07,173][20347] Updated weights for policy 0, policy_version 1407808 (0.0006) [2023-12-27 15:24:07,206][20381] Updated weights for policy 1, policy_version 1406607 (0.0010) [2023-12-27 15:24:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 720592896. Throughput: 0: 10274.4, 1: 10170.7. Samples: 720556216. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:07,363][19170] Avg episode reward: [(0, '784.530'), (1, '980.510')] [2023-12-27 15:24:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001406608_360120320.pth... [2023-12-27 15:24:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001407816_360472576.pth... [2023-12-27 15:24:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001405424_359817216.pth [2023-12-27 15:24:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001406600_360161280.pth [2023-12-27 15:24:07,786][20347] Updated weights for policy 0, policy_version 1407818 (0.0006) [2023-12-27 15:24:07,802][20381] Updated weights for policy 1, policy_version 1406617 (0.0011) [2023-12-27 15:24:07,843][20347] Updated weights for policy 0, policy_version 1407828 (0.0005) [2023-12-27 15:24:07,857][20381] Updated weights for policy 1, policy_version 1406627 (0.0010) [2023-12-27 15:24:07,862][20291] KL-divergence is very high: 188.3399 [2023-12-27 15:24:07,879][20291] KL-divergence is very high: 227.5154 [2023-12-27 15:24:07,889][20291] KL-divergence is very high: 218.2453 [2023-12-27 15:24:07,898][20347] Updated weights for policy 0, policy_version 1407838 (0.0009) [2023-12-27 15:24:07,903][20291] KL-divergence is very high: 191.4530 [2023-12-27 15:24:07,906][20381] Updated weights for policy 1, policy_version 1406637 (0.0007) [2023-12-27 15:24:07,956][20347] Updated weights for policy 0, policy_version 1407848 (0.0010) [2023-12-27 15:24:08,570][20381] Updated weights for policy 1, policy_version 1406647 (0.0011) [2023-12-27 15:24:08,632][20347] Updated weights for policy 0, policy_version 1407858 (0.0005) [2023-12-27 15:24:08,632][20381] Updated weights for policy 1, policy_version 1406657 (0.0011) [2023-12-27 15:24:08,687][20381] Updated weights for policy 1, policy_version 1406667 (0.0011) [2023-12-27 15:24:08,691][20347] Updated weights for policy 0, policy_version 1407868 (0.0010) [2023-12-27 15:24:08,746][20347] Updated weights for policy 0, policy_version 1407878 (0.0010) [2023-12-27 15:24:09,332][20347] Updated weights for policy 0, policy_version 1407888 (0.0008) [2023-12-27 15:24:09,387][20381] Updated weights for policy 1, policy_version 1406677 (0.0010) [2023-12-27 15:24:09,409][20347] Updated weights for policy 0, policy_version 1407898 (0.0009) [2023-12-27 15:24:09,448][20381] Updated weights for policy 1, policy_version 1406687 (0.0008) [2023-12-27 15:24:09,468][20347] Updated weights for policy 0, policy_version 1407908 (0.0009) [2023-12-27 15:24:09,511][20381] Updated weights for policy 1, policy_version 1406697 (0.0008) [2023-12-27 15:24:10,166][20347] Updated weights for policy 0, policy_version 1407918 (0.0007) [2023-12-27 15:24:10,195][20381] Updated weights for policy 1, policy_version 1406707 (0.0007) [2023-12-27 15:24:10,229][20347] Updated weights for policy 0, policy_version 1407928 (0.0007) [2023-12-27 15:24:10,253][20381] Updated weights for policy 1, policy_version 1406717 (0.0007) [2023-12-27 15:24:10,291][20347] Updated weights for policy 0, policy_version 1407938 (0.0007) [2023-12-27 15:24:10,316][20381] Updated weights for policy 1, policy_version 1406727 (0.0007) [2023-12-27 15:24:10,971][20381] Updated weights for policy 1, policy_version 1406737 (0.0007) [2023-12-27 15:24:10,988][20347] Updated weights for policy 0, policy_version 1407948 (0.0008) [2023-12-27 15:24:11,022][20381] Updated weights for policy 1, policy_version 1406747 (0.0009) [2023-12-27 15:24:11,032][20291] KL-divergence is very high: 122.8497 [2023-12-27 15:24:11,053][20347] Updated weights for policy 0, policy_version 1407958 (0.0007) [2023-12-27 15:24:11,059][20291] KL-divergence is very high: 222.8993 [2023-12-27 15:24:11,086][20291] KL-divergence is very high: 216.4443 [2023-12-27 15:24:11,086][20381] Updated weights for policy 1, policy_version 1406757 (0.0010) [2023-12-27 15:24:11,115][20291] KL-divergence is very high: 135.9542 [2023-12-27 15:24:11,121][20347] Updated weights for policy 0, policy_version 1407968 (0.0008) [2023-12-27 15:24:11,153][20381] Updated weights for policy 1, policy_version 1406767 (0.0009) [2023-12-27 15:24:11,820][20347] Updated weights for policy 0, policy_version 1407978 (0.0008) [2023-12-27 15:24:11,878][20347] Updated weights for policy 0, policy_version 1407988 (0.0009) [2023-12-27 15:24:11,936][20347] Updated weights for policy 0, policy_version 1407998 (0.0011) [2023-12-27 15:24:11,941][20381] Updated weights for policy 1, policy_version 1406777 (0.0006) [2023-12-27 15:24:11,986][20347] Updated weights for policy 0, policy_version 1408008 (0.0011) [2023-12-27 15:24:11,995][20381] Updated weights for policy 1, policy_version 1406787 (0.0006) [2023-12-27 15:24:12,053][20381] Updated weights for policy 1, policy_version 1406797 (0.0008) [2023-12-27 15:24:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 720691200. Throughput: 0: 10299.6, 1: 10211.7. Samples: 720680896. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:12,363][19170] Avg episode reward: [(0, '755.190'), (1, '990.260')] [2023-12-27 15:24:12,772][20381] Updated weights for policy 1, policy_version 1406807 (0.0010) [2023-12-27 15:24:12,773][20347] Updated weights for policy 0, policy_version 1408018 (0.0006) [2023-12-27 15:24:12,822][20381] Updated weights for policy 1, policy_version 1406817 (0.0011) [2023-12-27 15:24:12,829][20347] Updated weights for policy 0, policy_version 1408028 (0.0007) [2023-12-27 15:24:12,883][20381] Updated weights for policy 1, policy_version 1406827 (0.0011) [2023-12-27 15:24:12,884][20347] Updated weights for policy 0, policy_version 1408038 (0.0009) [2023-12-27 15:24:13,444][20347] Updated weights for policy 0, policy_version 1408048 (0.0006) [2023-12-27 15:24:13,491][20347] Updated weights for policy 0, policy_version 1408058 (0.0005) [2023-12-27 15:24:13,539][20347] Updated weights for policy 0, policy_version 1408068 (0.0006) [2023-12-27 15:24:13,576][20381] Updated weights for policy 1, policy_version 1406837 (0.0011) [2023-12-27 15:24:13,637][20381] Updated weights for policy 1, policy_version 1406847 (0.0011) [2023-12-27 15:24:13,705][20381] Updated weights for policy 1, policy_version 1406857 (0.0010) [2023-12-27 15:24:14,116][20347] Updated weights for policy 0, policy_version 1408078 (0.0007) [2023-12-27 15:24:14,166][20347] Updated weights for policy 0, policy_version 1408088 (0.0007) [2023-12-27 15:24:14,216][20347] Updated weights for policy 0, policy_version 1408098 (0.0007) [2023-12-27 15:24:14,374][20381] Updated weights for policy 1, policy_version 1406867 (0.0009) [2023-12-27 15:24:14,432][20381] Updated weights for policy 1, policy_version 1406877 (0.0008) [2023-12-27 15:24:14,496][20381] Updated weights for policy 1, policy_version 1406887 (0.0006) [2023-12-27 15:24:14,860][20347] Updated weights for policy 0, policy_version 1408108 (0.0007) [2023-12-27 15:24:14,924][20347] Updated weights for policy 0, policy_version 1408118 (0.0007) [2023-12-27 15:24:14,987][20347] Updated weights for policy 0, policy_version 1408128 (0.0008) [2023-12-27 15:24:15,029][20291] KL-divergence is very high: 125.8100 [2023-12-27 15:24:15,183][20381] Updated weights for policy 1, policy_version 1406897 (0.0007) [2023-12-27 15:24:15,249][20381] Updated weights for policy 1, policy_version 1406907 (0.0006) [2023-12-27 15:24:15,310][20381] Updated weights for policy 1, policy_version 1406917 (0.0008) [2023-12-27 15:24:15,371][20381] Updated weights for policy 1, policy_version 1406927 (0.0008) [2023-12-27 15:24:15,712][20347] Updated weights for policy 0, policy_version 1408138 (0.0008) [2023-12-27 15:24:15,764][20347] Updated weights for policy 0, policy_version 1408148 (0.0008) [2023-12-27 15:24:15,819][20347] Updated weights for policy 0, policy_version 1408158 (0.0007) [2023-12-27 15:24:15,843][20291] KL-divergence is very high: 235.2083 [2023-12-27 15:24:15,853][20291] KL-divergence is very high: 476.6589 [2023-12-27 15:24:15,865][20347] Updated weights for policy 0, policy_version 1408168 (0.0006) [2023-12-27 15:24:15,983][20381] Updated weights for policy 1, policy_version 1406937 (0.0007) [2023-12-27 15:24:16,031][20381] Updated weights for policy 1, policy_version 1406947 (0.0007) [2023-12-27 15:24:16,078][20381] Updated weights for policy 1, policy_version 1406957 (0.0007) [2023-12-27 15:24:16,499][20291] KL-divergence is very high: 265.4658 [2023-12-27 15:24:16,544][20291] KL-divergence is very high: 351.7147 [2023-12-27 15:24:16,553][20347] Updated weights for policy 0, policy_version 1408178 (0.0007) [2023-12-27 15:24:16,593][20291] KL-divergence is very high: 315.8359 [2023-12-27 15:24:16,617][20347] Updated weights for policy 0, policy_version 1408188 (0.0007) [2023-12-27 15:24:16,639][20291] KL-divergence is very high: 245.4670 [2023-12-27 15:24:16,672][20347] Updated weights for policy 0, policy_version 1408198 (0.0008) [2023-12-27 15:24:16,761][20381] Updated weights for policy 1, policy_version 1406967 (0.0006) [2023-12-27 15:24:16,815][20381] Updated weights for policy 1, policy_version 1406977 (0.0008) [2023-12-27 15:24:16,878][20381] Updated weights for policy 1, policy_version 1406987 (0.0007) [2023-12-27 15:24:17,256][20347] Updated weights for policy 0, policy_version 1408208 (0.0009) [2023-12-27 15:24:17,301][20347] Updated weights for policy 0, policy_version 1408218 (0.0010) [2023-12-27 15:24:17,353][20347] Updated weights for policy 0, policy_version 1408228 (0.0010) [2023-12-27 15:24:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 720789504. Throughput: 0: 10356.5, 1: 10161.4. Samples: 720804304. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:17,363][19170] Avg episode reward: [(0, '774.880'), (1, '1000.000')] [2023-12-27 15:24:17,561][20381] Updated weights for policy 1, policy_version 1406997 (0.0008) [2023-12-27 15:24:17,613][20381] Updated weights for policy 1, policy_version 1407007 (0.0010) [2023-12-27 15:24:17,658][20381] Updated weights for policy 1, policy_version 1407017 (0.0010) [2023-12-27 15:24:17,972][20347] Updated weights for policy 0, policy_version 1408238 (0.0010) [2023-12-27 15:24:18,025][20347] Updated weights for policy 0, policy_version 1408248 (0.0010) [2023-12-27 15:24:18,080][20347] Updated weights for policy 0, policy_version 1408258 (0.0010) [2023-12-27 15:24:18,310][20381] Updated weights for policy 1, policy_version 1407027 (0.0010) [2023-12-27 15:24:18,376][20381] Updated weights for policy 1, policy_version 1407037 (0.0010) [2023-12-27 15:24:18,428][20381] Updated weights for policy 1, policy_version 1407047 (0.0010) [2023-12-27 15:24:18,747][20347] Updated weights for policy 0, policy_version 1408268 (0.0010) [2023-12-27 15:24:18,809][20347] Updated weights for policy 0, policy_version 1408278 (0.0011) [2023-12-27 15:24:18,864][20347] Updated weights for policy 0, policy_version 1408288 (0.0010) [2023-12-27 15:24:19,076][20381] Updated weights for policy 1, policy_version 1407057 (0.0010) [2023-12-27 15:24:19,136][20381] Updated weights for policy 1, policy_version 1407067 (0.0006) [2023-12-27 15:24:19,197][20381] Updated weights for policy 1, policy_version 1407077 (0.0006) [2023-12-27 15:24:19,264][20381] Updated weights for policy 1, policy_version 1407087 (0.0007) [2023-12-27 15:24:19,589][20347] Updated weights for policy 0, policy_version 1408298 (0.0009) [2023-12-27 15:24:19,654][20347] Updated weights for policy 0, policy_version 1408308 (0.0006) [2023-12-27 15:24:19,722][20347] Updated weights for policy 0, policy_version 1408318 (0.0006) [2023-12-27 15:24:19,783][20347] Updated weights for policy 0, policy_version 1408328 (0.0006) [2023-12-27 15:24:19,919][20381] Updated weights for policy 1, policy_version 1407097 (0.0007) [2023-12-27 15:24:19,981][20381] Updated weights for policy 1, policy_version 1407107 (0.0008) [2023-12-27 15:24:20,035][20381] Updated weights for policy 1, policy_version 1407117 (0.0008) [2023-12-27 15:24:20,430][20347] Updated weights for policy 0, policy_version 1408338 (0.0009) [2023-12-27 15:24:20,477][20347] Updated weights for policy 0, policy_version 1408348 (0.0008) [2023-12-27 15:24:20,531][20347] Updated weights for policy 0, policy_version 1408358 (0.0008) [2023-12-27 15:24:20,743][20381] Updated weights for policy 1, policy_version 1407127 (0.0010) [2023-12-27 15:24:20,804][20381] Updated weights for policy 1, policy_version 1407137 (0.0011) [2023-12-27 15:24:20,864][20381] Updated weights for policy 1, policy_version 1407147 (0.0011) [2023-12-27 15:24:21,221][20347] Updated weights for policy 0, policy_version 1408368 (0.0007) [2023-12-27 15:24:21,286][20347] Updated weights for policy 0, policy_version 1408378 (0.0008) [2023-12-27 15:24:21,349][20347] Updated weights for policy 0, policy_version 1408388 (0.0007) [2023-12-27 15:24:21,651][20381] Updated weights for policy 1, policy_version 1407157 (0.0009) [2023-12-27 15:24:21,713][20381] Updated weights for policy 1, policy_version 1407167 (0.0008) [2023-12-27 15:24:21,770][20381] Updated weights for policy 1, policy_version 1407177 (0.0006) [2023-12-27 15:24:22,010][20347] Updated weights for policy 0, policy_version 1408398 (0.0006) [2023-12-27 15:24:22,070][20347] Updated weights for policy 0, policy_version 1408408 (0.0006) [2023-12-27 15:24:22,138][20347] Updated weights for policy 0, policy_version 1408418 (0.0006) [2023-12-27 15:24:22,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 720896000. Throughput: 0: 10384.2, 1: 10164.0. Samples: 720866872. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:22,363][19170] Avg episode reward: [(0, '843.290'), (1, '1000.000')] [2023-12-27 15:24:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001407184_360267776.pth... [2023-12-27 15:24:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001408424_360628224.pth... [2023-12-27 15:24:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001407208_360316928.pth [2023-12-27 15:24:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001406032_359972864.pth [2023-12-27 15:24:22,423][20381] Updated weights for policy 1, policy_version 1407187 (0.0007) [2023-12-27 15:24:22,475][20381] Updated weights for policy 1, policy_version 1407197 (0.0008) [2023-12-27 15:24:22,534][20381] Updated weights for policy 1, policy_version 1407207 (0.0008) [2023-12-27 15:24:22,808][20347] Updated weights for policy 0, policy_version 1408428 (0.0009) [2023-12-27 15:24:22,855][20347] Updated weights for policy 0, policy_version 1408438 (0.0009) [2023-12-27 15:24:22,907][20347] Updated weights for policy 0, policy_version 1408448 (0.0007) [2023-12-27 15:24:23,211][20381] Updated weights for policy 1, policy_version 1407217 (0.0006) [2023-12-27 15:24:23,259][20381] Updated weights for policy 1, policy_version 1407227 (0.0007) [2023-12-27 15:24:23,310][20381] Updated weights for policy 1, policy_version 1407237 (0.0006) [2023-12-27 15:24:23,368][20381] Updated weights for policy 1, policy_version 1407247 (0.0007) [2023-12-27 15:24:23,582][20347] Updated weights for policy 0, policy_version 1408458 (0.0007) [2023-12-27 15:24:23,638][20347] Updated weights for policy 0, policy_version 1408468 (0.0007) [2023-12-27 15:24:23,691][20347] Updated weights for policy 0, policy_version 1408478 (0.0007) [2023-12-27 15:24:23,746][20347] Updated weights for policy 0, policy_version 1408488 (0.0006) [2023-12-27 15:24:24,036][20381] Updated weights for policy 1, policy_version 1407257 (0.0010) [2023-12-27 15:24:24,093][20381] Updated weights for policy 1, policy_version 1407267 (0.0010) [2023-12-27 15:24:24,148][20381] Updated weights for policy 1, policy_version 1407277 (0.0010) [2023-12-27 15:24:24,405][20347] Updated weights for policy 0, policy_version 1408498 (0.0006) [2023-12-27 15:24:24,463][20347] Updated weights for policy 0, policy_version 1408508 (0.0008) [2023-12-27 15:24:24,521][20347] Updated weights for policy 0, policy_version 1408518 (0.0008) [2023-12-27 15:24:24,776][20381] Updated weights for policy 1, policy_version 1407287 (0.0007) [2023-12-27 15:24:24,839][20381] Updated weights for policy 1, policy_version 1407297 (0.0005) [2023-12-27 15:24:24,897][20381] Updated weights for policy 1, policy_version 1407307 (0.0005) [2023-12-27 15:24:25,241][20347] Updated weights for policy 0, policy_version 1408528 (0.0008) [2023-12-27 15:24:25,285][20347] Updated weights for policy 0, policy_version 1408538 (0.0008) [2023-12-27 15:24:25,329][20347] Updated weights for policy 0, policy_version 1408548 (0.0008) [2023-12-27 15:24:25,388][20381] Updated weights for policy 1, policy_version 1407317 (0.0005) [2023-12-27 15:24:25,444][20381] Updated weights for policy 1, policy_version 1407327 (0.0005) [2023-12-27 15:24:25,504][20381] Updated weights for policy 1, policy_version 1407337 (0.0005) [2023-12-27 15:24:26,039][20381] Updated weights for policy 1, policy_version 1407347 (0.0007) [2023-12-27 15:24:26,090][20347] Updated weights for policy 0, policy_version 1408558 (0.0008) [2023-12-27 15:24:26,091][20381] Updated weights for policy 1, policy_version 1407357 (0.0010) [2023-12-27 15:24:26,141][20381] Updated weights for policy 1, policy_version 1407367 (0.0010) [2023-12-27 15:24:26,146][20347] Updated weights for policy 0, policy_version 1408568 (0.0006) [2023-12-27 15:24:26,202][20347] Updated weights for policy 0, policy_version 1408578 (0.0007) [2023-12-27 15:24:26,763][20347] Updated weights for policy 0, policy_version 1408588 (0.0006) [2023-12-27 15:24:26,809][20347] Updated weights for policy 0, policy_version 1408598 (0.0005) [2023-12-27 15:24:26,824][20381] Updated weights for policy 1, policy_version 1407377 (0.0010) [2023-12-27 15:24:26,855][20347] Updated weights for policy 0, policy_version 1408608 (0.0006) [2023-12-27 15:24:26,872][20381] Updated weights for policy 1, policy_version 1407387 (0.0010) [2023-12-27 15:24:26,919][20381] Updated weights for policy 1, policy_version 1407397 (0.0010) [2023-12-27 15:24:26,965][20381] Updated weights for policy 1, policy_version 1407407 (0.0010) [2023-12-27 15:24:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 721002496. Throughput: 0: 10377.6, 1: 10195.5. Samples: 720991688. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:27,363][19170] Avg episode reward: [(0, '872.520'), (1, '1000.000')] [2023-12-27 15:24:27,494][20347] Updated weights for policy 0, policy_version 1408618 (0.0005) [2023-12-27 15:24:27,545][20347] Updated weights for policy 0, policy_version 1408628 (0.0005) [2023-12-27 15:24:27,593][20347] Updated weights for policy 0, policy_version 1408638 (0.0007) [2023-12-27 15:24:27,602][20381] Updated weights for policy 1, policy_version 1407417 (0.0006) [2023-12-27 15:24:27,648][20347] Updated weights for policy 0, policy_version 1408648 (0.0007) [2023-12-27 15:24:27,651][20381] Updated weights for policy 1, policy_version 1407427 (0.0005) [2023-12-27 15:24:27,696][20381] Updated weights for policy 1, policy_version 1407437 (0.0005) [2023-12-27 15:24:28,248][20381] Updated weights for policy 1, policy_version 1407447 (0.0005) [2023-12-27 15:24:28,270][20347] Updated weights for policy 0, policy_version 1408658 (0.0008) [2023-12-27 15:24:28,308][20381] Updated weights for policy 1, policy_version 1407457 (0.0008) [2023-12-27 15:24:28,322][20347] Updated weights for policy 0, policy_version 1408668 (0.0007) [2023-12-27 15:24:28,365][20381] Updated weights for policy 1, policy_version 1407467 (0.0007) [2023-12-27 15:24:28,380][20347] Updated weights for policy 0, policy_version 1408678 (0.0007) [2023-12-27 15:24:29,032][20381] Updated weights for policy 1, policy_version 1407477 (0.0007) [2023-12-27 15:24:29,056][20347] Updated weights for policy 0, policy_version 1408688 (0.0006) [2023-12-27 15:24:29,083][20381] Updated weights for policy 1, policy_version 1407487 (0.0006) [2023-12-27 15:24:29,100][20347] Updated weights for policy 0, policy_version 1408698 (0.0006) [2023-12-27 15:24:29,135][20381] Updated weights for policy 1, policy_version 1407497 (0.0006) [2023-12-27 15:24:29,141][20347] Updated weights for policy 0, policy_version 1408708 (0.0005) [2023-12-27 15:24:29,862][20381] Updated weights for policy 1, policy_version 1407507 (0.0006) [2023-12-27 15:24:29,877][20347] Updated weights for policy 0, policy_version 1408718 (0.0006) [2023-12-27 15:24:29,919][20381] Updated weights for policy 1, policy_version 1407517 (0.0006) [2023-12-27 15:24:29,945][20347] Updated weights for policy 0, policy_version 1408728 (0.0008) [2023-12-27 15:24:29,982][20381] Updated weights for policy 1, policy_version 1407527 (0.0007) [2023-12-27 15:24:30,005][20347] Updated weights for policy 0, policy_version 1408738 (0.0007) [2023-12-27 15:24:30,579][20381] Updated weights for policy 1, policy_version 1407537 (0.0006) [2023-12-27 15:24:30,634][20381] Updated weights for policy 1, policy_version 1407547 (0.0008) [2023-12-27 15:24:30,664][20347] Updated weights for policy 0, policy_version 1408748 (0.0007) [2023-12-27 15:24:30,690][20381] Updated weights for policy 1, policy_version 1407557 (0.0008) [2023-12-27 15:24:30,716][20347] Updated weights for policy 0, policy_version 1408758 (0.0005) [2023-12-27 15:24:30,746][20381] Updated weights for policy 1, policy_version 1407567 (0.0007) [2023-12-27 15:24:30,772][20347] Updated weights for policy 0, policy_version 1408768 (0.0005) [2023-12-27 15:24:31,405][20347] Updated weights for policy 0, policy_version 1408778 (0.0008) [2023-12-27 15:24:31,432][20381] Updated weights for policy 1, policy_version 1407577 (0.0008) [2023-12-27 15:24:31,473][20347] Updated weights for policy 0, policy_version 1408788 (0.0008) [2023-12-27 15:24:31,494][20381] Updated weights for policy 1, policy_version 1407587 (0.0007) [2023-12-27 15:24:31,530][20347] Updated weights for policy 0, policy_version 1408798 (0.0006) [2023-12-27 15:24:31,551][20381] Updated weights for policy 1, policy_version 1407597 (0.0008) [2023-12-27 15:24:31,582][20347] Updated weights for policy 0, policy_version 1408808 (0.0005) [2023-12-27 15:24:32,167][20381] Updated weights for policy 1, policy_version 1407607 (0.0007) [2023-12-27 15:24:32,229][20381] Updated weights for policy 1, policy_version 1407617 (0.0008) [2023-12-27 15:24:32,281][20347] Updated weights for policy 0, policy_version 1408818 (0.0007) [2023-12-27 15:24:32,287][20381] Updated weights for policy 1, policy_version 1407627 (0.0007) [2023-12-27 15:24:32,344][20347] Updated weights for policy 0, policy_version 1408828 (0.0008) [2023-12-27 15:24:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 721108992. Throughput: 0: 10396.4, 1: 10261.7. Samples: 721120236. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:32,363][19170] Avg episode reward: [(0, '797.370'), (1, '1000.000')] [2023-12-27 15:24:32,415][20347] Updated weights for policy 0, policy_version 1408838 (0.0008) [2023-12-27 15:24:33,017][20381] Updated weights for policy 1, policy_version 1407637 (0.0006) [2023-12-27 15:24:33,049][20347] Updated weights for policy 0, policy_version 1408848 (0.0008) [2023-12-27 15:24:33,070][20381] Updated weights for policy 1, policy_version 1407647 (0.0005) [2023-12-27 15:24:33,104][20347] Updated weights for policy 0, policy_version 1408858 (0.0007) [2023-12-27 15:24:33,128][20381] Updated weights for policy 1, policy_version 1407657 (0.0005) [2023-12-27 15:24:33,155][20347] Updated weights for policy 0, policy_version 1408868 (0.0008) [2023-12-27 15:24:33,706][20381] Updated weights for policy 1, policy_version 1407667 (0.0007) [2023-12-27 15:24:33,763][20381] Updated weights for policy 1, policy_version 1407677 (0.0010) [2023-12-27 15:24:33,814][20381] Updated weights for policy 1, policy_version 1407687 (0.0010) [2023-12-27 15:24:33,830][20347] Updated weights for policy 0, policy_version 1408878 (0.0006) [2023-12-27 15:24:33,875][20347] Updated weights for policy 0, policy_version 1408888 (0.0006) [2023-12-27 15:24:33,919][20291] KL-divergence is very high: 151.5945 [2023-12-27 15:24:33,922][20347] Updated weights for policy 0, policy_version 1408898 (0.0005) [2023-12-27 15:24:34,445][20381] Updated weights for policy 1, policy_version 1407697 (0.0008) [2023-12-27 15:24:34,502][20381] Updated weights for policy 1, policy_version 1407707 (0.0006) [2023-12-27 15:24:34,534][20347] Updated weights for policy 0, policy_version 1408908 (0.0007) [2023-12-27 15:24:34,572][20381] Updated weights for policy 1, policy_version 1407717 (0.0008) [2023-12-27 15:24:34,573][20291] KL-divergence is very high: 138.5255 [2023-12-27 15:24:34,578][20291] KL-divergence is very high: 171.9072 [2023-12-27 15:24:34,589][20347] Updated weights for policy 0, policy_version 1408918 (0.0007) [2023-12-27 15:24:34,590][20291] KL-divergence is very high: 208.2995 [2023-12-27 15:24:34,619][20291] KL-divergence is very high: 213.2127 [2023-12-27 15:24:34,624][20291] KL-divergence is very high: 198.3140 [2023-12-27 15:24:34,636][20291] KL-divergence is very high: 151.0157 [2023-12-27 15:24:34,641][20381] Updated weights for policy 1, policy_version 1407727 (0.0008) [2023-12-27 15:24:34,649][20347] Updated weights for policy 0, policy_version 1408928 (0.0007) [2023-12-27 15:24:35,240][20381] Updated weights for policy 1, policy_version 1407737 (0.0007) [2023-12-27 15:24:35,295][20381] Updated weights for policy 1, policy_version 1407747 (0.0007) [2023-12-27 15:24:35,296][20347] Updated weights for policy 0, policy_version 1408938 (0.0008) [2023-12-27 15:24:35,352][20347] Updated weights for policy 0, policy_version 1408948 (0.0006) [2023-12-27 15:24:35,354][20381] Updated weights for policy 1, policy_version 1407757 (0.0005) [2023-12-27 15:24:35,400][20347] Updated weights for policy 0, policy_version 1408958 (0.0007) [2023-12-27 15:24:35,454][20347] Updated weights for policy 0, policy_version 1408968 (0.0007) [2023-12-27 15:24:36,040][20381] Updated weights for policy 1, policy_version 1407767 (0.0006) [2023-12-27 15:24:36,096][20381] Updated weights for policy 1, policy_version 1407777 (0.0007) [2023-12-27 15:24:36,158][20381] Updated weights for policy 1, policy_version 1407787 (0.0008) [2023-12-27 15:24:36,209][20347] Updated weights for policy 0, policy_version 1408978 (0.0007) [2023-12-27 15:24:36,268][20347] Updated weights for policy 0, policy_version 1408988 (0.0007) [2023-12-27 15:24:36,327][20347] Updated weights for policy 0, policy_version 1408998 (0.0008) [2023-12-27 15:24:36,874][20381] Updated weights for policy 1, policy_version 1407797 (0.0007) [2023-12-27 15:24:36,928][20381] Updated weights for policy 1, policy_version 1407807 (0.0007) [2023-12-27 15:24:36,961][20347] Updated weights for policy 0, policy_version 1409008 (0.0007) [2023-12-27 15:24:36,996][20381] Updated weights for policy 1, policy_version 1407817 (0.0008) [2023-12-27 15:24:37,023][20347] Updated weights for policy 0, policy_version 1409018 (0.0008) [2023-12-27 15:24:37,082][20347] Updated weights for policy 0, policy_version 1409028 (0.0010) [2023-12-27 15:24:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 721215488. Throughput: 0: 10435.5, 1: 10302.2. Samples: 721184196. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:37,363][19170] Avg episode reward: [(0, '790.250'), (1, '1000.000')] [2023-12-27 15:24:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001407824_360431616.pth... [2023-12-27 15:24:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001409032_360783872.pth... [2023-12-27 15:24:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001406608_360120320.pth [2023-12-27 15:24:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001407816_360472576.pth [2023-12-27 15:24:37,698][20347] Updated weights for policy 0, policy_version 1409038 (0.0008) [2023-12-27 15:24:37,726][20381] Updated weights for policy 1, policy_version 1407827 (0.0007) [2023-12-27 15:24:37,752][20347] Updated weights for policy 0, policy_version 1409048 (0.0006) [2023-12-27 15:24:37,782][20381] Updated weights for policy 1, policy_version 1407837 (0.0007) [2023-12-27 15:24:37,806][20347] Updated weights for policy 0, policy_version 1409058 (0.0007) [2023-12-27 15:24:37,846][20381] Updated weights for policy 1, policy_version 1407847 (0.0007) [2023-12-27 15:24:38,451][20347] Updated weights for policy 0, policy_version 1409068 (0.0006) [2023-12-27 15:24:38,496][20381] Updated weights for policy 1, policy_version 1407857 (0.0007) [2023-12-27 15:24:38,512][20347] Updated weights for policy 0, policy_version 1409078 (0.0006) [2023-12-27 15:24:38,555][20381] Updated weights for policy 1, policy_version 1407867 (0.0008) [2023-12-27 15:24:38,571][20347] Updated weights for policy 0, policy_version 1409088 (0.0005) [2023-12-27 15:24:38,609][20381] Updated weights for policy 1, policy_version 1407877 (0.0006) [2023-12-27 15:24:38,662][20381] Updated weights for policy 1, policy_version 1407887 (0.0008) [2023-12-27 15:24:39,191][20347] Updated weights for policy 0, policy_version 1409098 (0.0007) [2023-12-27 15:24:39,253][20347] Updated weights for policy 0, policy_version 1409108 (0.0007) [2023-12-27 15:24:39,316][20347] Updated weights for policy 0, policy_version 1409118 (0.0006) [2023-12-27 15:24:39,380][20347] Updated weights for policy 0, policy_version 1409128 (0.0011) [2023-12-27 15:24:39,386][20381] Updated weights for policy 1, policy_version 1407897 (0.0007) [2023-12-27 15:24:39,450][20381] Updated weights for policy 1, policy_version 1407907 (0.0009) [2023-12-27 15:24:39,507][20381] Updated weights for policy 1, policy_version 1407917 (0.0007) [2023-12-27 15:24:40,021][20347] Updated weights for policy 0, policy_version 1409138 (0.0008) [2023-12-27 15:24:40,079][20347] Updated weights for policy 0, policy_version 1409148 (0.0008) [2023-12-27 15:24:40,137][20347] Updated weights for policy 0, policy_version 1409158 (0.0008) [2023-12-27 15:24:40,236][20381] Updated weights for policy 1, policy_version 1407927 (0.0007) [2023-12-27 15:24:40,295][20381] Updated weights for policy 1, policy_version 1407937 (0.0008) [2023-12-27 15:24:40,355][20381] Updated weights for policy 1, policy_version 1407947 (0.0009) [2023-12-27 15:24:40,863][20347] Updated weights for policy 0, policy_version 1409168 (0.0008) [2023-12-27 15:24:40,921][20347] Updated weights for policy 0, policy_version 1409178 (0.0008) [2023-12-27 15:24:40,976][20347] Updated weights for policy 0, policy_version 1409188 (0.0008) [2023-12-27 15:24:41,074][20381] Updated weights for policy 1, policy_version 1407957 (0.0008) [2023-12-27 15:24:41,150][20381] Updated weights for policy 1, policy_version 1407967 (0.0008) [2023-12-27 15:24:41,212][20381] Updated weights for policy 1, policy_version 1407977 (0.0007) [2023-12-27 15:24:41,768][20347] Updated weights for policy 0, policy_version 1409202 (0.0007) [2023-12-27 15:24:41,833][20347] Updated weights for policy 0, policy_version 1409212 (0.0007) [2023-12-27 15:24:41,890][20347] Updated weights for policy 0, policy_version 1409222 (0.0006) [2023-12-27 15:24:41,957][20381] Updated weights for policy 1, policy_version 1407987 (0.0007) [2023-12-27 15:24:42,016][20381] Updated weights for policy 1, policy_version 1407997 (0.0005) [2023-12-27 15:24:42,072][20381] Updated weights for policy 1, policy_version 1408007 (0.0006) [2023-12-27 15:24:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 721313792. Throughput: 0: 10389.8, 1: 10249.6. Samples: 721304588. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:42,363][19170] Avg episode reward: [(0, '838.270'), (1, '990.310')] [2023-12-27 15:24:42,576][20347] Updated weights for policy 0, policy_version 1409232 (0.0008) [2023-12-27 15:24:42,629][20347] Updated weights for policy 0, policy_version 1409242 (0.0007) [2023-12-27 15:24:42,689][20347] Updated weights for policy 0, policy_version 1409252 (0.0008) [2023-12-27 15:24:42,765][20381] Updated weights for policy 1, policy_version 1408017 (0.0008) [2023-12-27 15:24:42,821][20381] Updated weights for policy 1, policy_version 1408027 (0.0006) [2023-12-27 15:24:42,874][20381] Updated weights for policy 1, policy_version 1408037 (0.0005) [2023-12-27 15:24:42,930][20381] Updated weights for policy 1, policy_version 1408047 (0.0005) [2023-12-27 15:24:43,348][20347] Updated weights for policy 0, policy_version 1409262 (0.0006) [2023-12-27 15:24:43,404][20347] Updated weights for policy 0, policy_version 1409272 (0.0006) [2023-12-27 15:24:43,455][20291] KL-divergence is very high: 168.9552 [2023-12-27 15:24:43,467][20347] Updated weights for policy 0, policy_version 1409282 (0.0006) [2023-12-27 15:24:43,520][20381] Updated weights for policy 1, policy_version 1408057 (0.0007) [2023-12-27 15:24:43,585][20381] Updated weights for policy 1, policy_version 1408067 (0.0006) [2023-12-27 15:24:43,642][20381] Updated weights for policy 1, policy_version 1408077 (0.0009) [2023-12-27 15:24:44,024][20347] Updated weights for policy 0, policy_version 1409292 (0.0008) [2023-12-27 15:24:44,077][20347] Updated weights for policy 0, policy_version 1409302 (0.0010) [2023-12-27 15:24:44,129][20347] Updated weights for policy 0, policy_version 1409312 (0.0010) [2023-12-27 15:24:44,321][20381] Updated weights for policy 1, policy_version 1408087 (0.0006) [2023-12-27 15:24:44,379][20381] Updated weights for policy 1, policy_version 1408097 (0.0010) [2023-12-27 15:24:44,440][20381] Updated weights for policy 1, policy_version 1408107 (0.0007) [2023-12-27 15:24:44,766][20347] Updated weights for policy 0, policy_version 1409322 (0.0010) [2023-12-27 15:24:44,826][20347] Updated weights for policy 0, policy_version 1409332 (0.0011) [2023-12-27 15:24:44,850][20291] KL-divergence is very high: 136.5417 [2023-12-27 15:24:44,856][20291] KL-divergence is very high: 134.8765 [2023-12-27 15:24:44,885][20347] Updated weights for policy 0, policy_version 1409342 (0.0010) [2023-12-27 15:24:44,950][20347] Updated weights for policy 0, policy_version 1409352 (0.0008) [2023-12-27 15:24:45,091][20381] Updated weights for policy 1, policy_version 1408117 (0.0008) [2023-12-27 15:24:45,152][20381] Updated weights for policy 1, policy_version 1408127 (0.0011) [2023-12-27 15:24:45,215][20381] Updated weights for policy 1, policy_version 1408137 (0.0008) [2023-12-27 15:24:45,563][20347] Updated weights for policy 0, policy_version 1409362 (0.0009) [2023-12-27 15:24:45,616][20347] Updated weights for policy 0, policy_version 1409372 (0.0005) [2023-12-27 15:24:45,671][20347] Updated weights for policy 0, policy_version 1409382 (0.0007) [2023-12-27 15:24:45,910][20381] Updated weights for policy 1, policy_version 1408147 (0.0009) [2023-12-27 15:24:45,972][20381] Updated weights for policy 1, policy_version 1408157 (0.0011) [2023-12-27 15:24:46,030][20381] Updated weights for policy 1, policy_version 1408167 (0.0008) [2023-12-27 15:24:46,338][20347] Updated weights for policy 0, policy_version 1409392 (0.0010) [2023-12-27 15:24:46,388][20347] Updated weights for policy 0, policy_version 1409402 (0.0010) [2023-12-27 15:24:46,439][20347] Updated weights for policy 0, policy_version 1409412 (0.0010) [2023-12-27 15:24:46,615][20381] Updated weights for policy 1, policy_version 1408177 (0.0006) [2023-12-27 15:24:46,670][20381] Updated weights for policy 1, policy_version 1408187 (0.0008) [2023-12-27 15:24:46,729][20381] Updated weights for policy 1, policy_version 1408197 (0.0008) [2023-12-27 15:24:46,786][20381] Updated weights for policy 1, policy_version 1408207 (0.0008) [2023-12-27 15:24:47,046][20347] Updated weights for policy 0, policy_version 1409422 (0.0007) [2023-12-27 15:24:47,100][20347] Updated weights for policy 0, policy_version 1409432 (0.0006) [2023-12-27 15:24:47,161][20347] Updated weights for policy 0, policy_version 1409442 (0.0010) [2023-12-27 15:24:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 721420288. Throughput: 0: 10489.3, 1: 10321.9. Samples: 721430640. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:47,363][19170] Avg episode reward: [(0, '871.990'), (1, '990.310')] [2023-12-27 15:24:47,529][20381] Updated weights for policy 1, policy_version 1408217 (0.0007) [2023-12-27 15:24:47,583][20381] Updated weights for policy 1, policy_version 1408227 (0.0007) [2023-12-27 15:24:47,631][20381] Updated weights for policy 1, policy_version 1408237 (0.0007) [2023-12-27 15:24:47,753][20347] Updated weights for policy 0, policy_version 1409452 (0.0010) [2023-12-27 15:24:47,810][20347] Updated weights for policy 0, policy_version 1409462 (0.0010) [2023-12-27 15:24:47,864][20347] Updated weights for policy 0, policy_version 1409472 (0.0010) [2023-12-27 15:24:48,324][20381] Updated weights for policy 1, policy_version 1408247 (0.0007) [2023-12-27 15:24:48,391][20381] Updated weights for policy 1, policy_version 1408257 (0.0007) [2023-12-27 15:24:48,455][20381] Updated weights for policy 1, policy_version 1408267 (0.0008) [2023-12-27 15:24:48,555][20347] Updated weights for policy 0, policy_version 1409482 (0.0010) [2023-12-27 15:24:48,617][20347] Updated weights for policy 0, policy_version 1409492 (0.0010) [2023-12-27 15:24:48,678][20347] Updated weights for policy 0, policy_version 1409502 (0.0010) [2023-12-27 15:24:48,738][20347] Updated weights for policy 0, policy_version 1409512 (0.0008) [2023-12-27 15:24:49,143][20381] Updated weights for policy 1, policy_version 1408277 (0.0008) [2023-12-27 15:24:49,199][20381] Updated weights for policy 1, policy_version 1408287 (0.0007) [2023-12-27 15:24:49,263][20381] Updated weights for policy 1, policy_version 1408297 (0.0009) [2023-12-27 15:24:49,439][20347] Updated weights for policy 0, policy_version 1409522 (0.0007) [2023-12-27 15:24:49,487][20347] Updated weights for policy 0, policy_version 1409532 (0.0007) [2023-12-27 15:24:49,531][20347] Updated weights for policy 0, policy_version 1409542 (0.0007) [2023-12-27 15:24:49,965][20381] Updated weights for policy 1, policy_version 1408307 (0.0011) [2023-12-27 15:24:50,026][20381] Updated weights for policy 1, policy_version 1408317 (0.0010) [2023-12-27 15:24:50,085][20381] Updated weights for policy 1, policy_version 1408327 (0.0010) [2023-12-27 15:24:50,242][20347] Updated weights for policy 0, policy_version 1409552 (0.0007) [2023-12-27 15:24:50,295][20347] Updated weights for policy 0, policy_version 1409562 (0.0008) [2023-12-27 15:24:50,352][20347] Updated weights for policy 0, policy_version 1409572 (0.0007) [2023-12-27 15:24:50,750][20381] Updated weights for policy 1, policy_version 1408337 (0.0011) [2023-12-27 15:24:50,799][20381] Updated weights for policy 1, policy_version 1408347 (0.0008) [2023-12-27 15:24:50,858][20381] Updated weights for policy 1, policy_version 1408357 (0.0008) [2023-12-27 15:24:50,914][20381] Updated weights for policy 1, policy_version 1408367 (0.0007) [2023-12-27 15:24:51,061][20347] Updated weights for policy 0, policy_version 1409582 (0.0008) [2023-12-27 15:24:51,125][20347] Updated weights for policy 0, policy_version 1409592 (0.0006) [2023-12-27 15:24:51,193][20347] Updated weights for policy 0, policy_version 1409602 (0.0005) [2023-12-27 15:24:51,703][20381] Updated weights for policy 1, policy_version 1408377 (0.0011) [2023-12-27 15:24:51,758][20381] Updated weights for policy 1, policy_version 1408387 (0.0011) [2023-12-27 15:24:51,810][20347] Updated weights for policy 0, policy_version 1409612 (0.0006) [2023-12-27 15:24:51,818][20381] Updated weights for policy 1, policy_version 1408397 (0.0010) [2023-12-27 15:24:51,866][20347] Updated weights for policy 0, policy_version 1409622 (0.0007) [2023-12-27 15:24:51,917][20347] Updated weights for policy 0, policy_version 1409632 (0.0007) [2023-12-27 15:24:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 721518592. Throughput: 0: 10437.5, 1: 10317.6. Samples: 721490196. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:52,363][19170] Avg episode reward: [(0, '794.140'), (1, '1000.000')] [2023-12-27 15:24:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001408400_360579072.pth... [2023-12-27 15:24:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001409640_360939520.pth... [2023-12-27 15:24:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001407184_360267776.pth [2023-12-27 15:24:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001408424_360628224.pth [2023-12-27 15:24:52,536][20381] Updated weights for policy 1, policy_version 1408407 (0.0010) [2023-12-27 15:24:52,602][20381] Updated weights for policy 1, policy_version 1408417 (0.0009) [2023-12-27 15:24:52,632][20347] Updated weights for policy 0, policy_version 1409642 (0.0007) [2023-12-27 15:24:52,661][20381] Updated weights for policy 1, policy_version 1408427 (0.0007) [2023-12-27 15:24:52,690][20347] Updated weights for policy 0, policy_version 1409652 (0.0005) [2023-12-27 15:24:52,752][20347] Updated weights for policy 0, policy_version 1409662 (0.0008) [2023-12-27 15:24:52,810][20347] Updated weights for policy 0, policy_version 1409672 (0.0008) [2023-12-27 15:24:53,364][20381] Updated weights for policy 1, policy_version 1408437 (0.0008) [2023-12-27 15:24:53,423][20381] Updated weights for policy 1, policy_version 1408447 (0.0010) [2023-12-27 15:24:53,475][20381] Updated weights for policy 1, policy_version 1408457 (0.0010) [2023-12-27 15:24:53,476][20347] Updated weights for policy 0, policy_version 1409682 (0.0008) [2023-12-27 15:24:53,531][20347] Updated weights for policy 0, policy_version 1409692 (0.0005) [2023-12-27 15:24:53,587][20347] Updated weights for policy 0, policy_version 1409702 (0.0005) [2023-12-27 15:24:54,119][20381] Updated weights for policy 1, policy_version 1408467 (0.0009) [2023-12-27 15:24:54,185][20381] Updated weights for policy 1, policy_version 1408477 (0.0008) [2023-12-27 15:24:54,227][20347] Updated weights for policy 0, policy_version 1409712 (0.0005) [2023-12-27 15:24:54,248][20381] Updated weights for policy 1, policy_version 1408487 (0.0010) [2023-12-27 15:24:54,285][20347] Updated weights for policy 0, policy_version 1409722 (0.0005) [2023-12-27 15:24:54,341][20347] Updated weights for policy 0, policy_version 1409732 (0.0007) [2023-12-27 15:24:54,858][20381] Updated weights for policy 1, policy_version 1408497 (0.0010) [2023-12-27 15:24:54,885][20347] Updated weights for policy 0, policy_version 1409742 (0.0005) [2023-12-27 15:24:54,914][20381] Updated weights for policy 1, policy_version 1408507 (0.0008) [2023-12-27 15:24:54,942][20347] Updated weights for policy 0, policy_version 1409752 (0.0005) [2023-12-27 15:24:54,975][20381] Updated weights for policy 1, policy_version 1408517 (0.0008) [2023-12-27 15:24:55,002][20347] Updated weights for policy 0, policy_version 1409762 (0.0007) [2023-12-27 15:24:55,037][20381] Updated weights for policy 1, policy_version 1408527 (0.0008) [2023-12-27 15:24:55,616][20347] Updated weights for policy 0, policy_version 1409772 (0.0008) [2023-12-27 15:24:55,677][20347] Updated weights for policy 0, policy_version 1409782 (0.0010) [2023-12-27 15:24:55,686][20381] Updated weights for policy 1, policy_version 1408537 (0.0006) [2023-12-27 15:24:55,732][20347] Updated weights for policy 0, policy_version 1409792 (0.0010) [2023-12-27 15:24:55,744][20381] Updated weights for policy 1, policy_version 1408547 (0.0007) [2023-12-27 15:24:55,794][20381] Updated weights for policy 1, policy_version 1408557 (0.0008) [2023-12-27 15:24:56,321][20291] KL-divergence is very high: 134.5676 [2023-12-27 15:24:56,331][20347] Updated weights for policy 0, policy_version 1409802 (0.0009) [2023-12-27 15:24:56,336][20291] KL-divergence is very high: 156.0939 [2023-12-27 15:24:56,360][20291] KL-divergence is very high: 226.4827 [2023-12-27 15:24:56,367][20381] Updated weights for policy 1, policy_version 1408567 (0.0007) [2023-12-27 15:24:56,375][20291] KL-divergence is very high: 204.1781 [2023-12-27 15:24:56,378][20347] Updated weights for policy 0, policy_version 1409812 (0.0005) [2023-12-27 15:24:56,399][20291] KL-divergence is very high: 184.5789 [2023-12-27 15:24:56,413][20381] Updated weights for policy 1, policy_version 1408577 (0.0007) [2023-12-27 15:24:56,413][20291] KL-divergence is very high: 141.2409 [2023-12-27 15:24:56,428][20347] Updated weights for policy 0, policy_version 1409822 (0.0005) [2023-12-27 15:24:56,436][20291] KL-divergence is very high: 118.6950 [2023-12-27 15:24:56,458][20381] Updated weights for policy 1, policy_version 1408587 (0.0008) [2023-12-27 15:24:56,476][20347] Updated weights for policy 0, policy_version 1409832 (0.0005) [2023-12-27 15:24:57,081][20347] Updated weights for policy 0, policy_version 1409842 (0.0005) [2023-12-27 15:24:57,129][20347] Updated weights for policy 0, policy_version 1409852 (0.0005) [2023-12-27 15:24:57,157][20381] Updated weights for policy 1, policy_version 1408597 (0.0007) [2023-12-27 15:24:57,177][20347] Updated weights for policy 0, policy_version 1409862 (0.0005) [2023-12-27 15:24:57,215][20381] Updated weights for policy 1, policy_version 1408607 (0.0007) [2023-12-27 15:24:57,265][20381] Updated weights for policy 1, policy_version 1408617 (0.0008) [2023-12-27 15:24:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 721633280. Throughput: 0: 10475.3, 1: 10297.3. Samples: 721615668. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:24:57,364][19170] Avg episode reward: [(0, '813.650'), (1, '1000.000')] [2023-12-27 15:24:57,735][20347] Updated weights for policy 0, policy_version 1409872 (0.0005) [2023-12-27 15:24:57,789][20347] Updated weights for policy 0, policy_version 1409882 (0.0005) [2023-12-27 15:24:57,836][20347] Updated weights for policy 0, policy_version 1409892 (0.0009) [2023-12-27 15:24:57,945][20381] Updated weights for policy 1, policy_version 1408627 (0.0005) [2023-12-27 15:24:57,998][20381] Updated weights for policy 1, policy_version 1408637 (0.0005) [2023-12-27 15:24:58,065][20381] Updated weights for policy 1, policy_version 1408647 (0.0005) [2023-12-27 15:24:58,500][20347] Updated weights for policy 0, policy_version 1409902 (0.0009) [2023-12-27 15:24:58,561][20347] Updated weights for policy 0, policy_version 1409912 (0.0006) [2023-12-27 15:24:58,623][20347] Updated weights for policy 0, policy_version 1409922 (0.0008) [2023-12-27 15:24:58,755][20381] Updated weights for policy 1, policy_version 1408657 (0.0005) [2023-12-27 15:24:58,824][20381] Updated weights for policy 1, policy_version 1408667 (0.0008) [2023-12-27 15:24:58,887][20381] Updated weights for policy 1, policy_version 1408677 (0.0006) [2023-12-27 15:24:58,954][20381] Updated weights for policy 1, policy_version 1408687 (0.0008) [2023-12-27 15:24:59,482][20347] Updated weights for policy 0, policy_version 1409932 (0.0008) [2023-12-27 15:24:59,543][20347] Updated weights for policy 0, policy_version 1409942 (0.0006) [2023-12-27 15:24:59,599][20347] Updated weights for policy 0, policy_version 1409952 (0.0005) [2023-12-27 15:24:59,653][20381] Updated weights for policy 1, policy_version 1408697 (0.0007) [2023-12-27 15:24:59,711][20381] Updated weights for policy 1, policy_version 1408707 (0.0010) [2023-12-27 15:24:59,770][20381] Updated weights for policy 1, policy_version 1408717 (0.0011) [2023-12-27 15:25:00,280][20347] Updated weights for policy 0, policy_version 1409962 (0.0006) [2023-12-27 15:25:00,332][20347] Updated weights for policy 0, policy_version 1409972 (0.0007) [2023-12-27 15:25:00,382][20347] Updated weights for policy 0, policy_version 1409982 (0.0008) [2023-12-27 15:25:00,435][20347] Updated weights for policy 0, policy_version 1409992 (0.0006) [2023-12-27 15:25:00,449][20381] Updated weights for policy 1, policy_version 1408727 (0.0008) [2023-12-27 15:25:00,510][20381] Updated weights for policy 1, policy_version 1408737 (0.0007) [2023-12-27 15:25:00,553][20381] Updated weights for policy 1, policy_version 1408747 (0.0005) [2023-12-27 15:25:01,078][20347] Updated weights for policy 0, policy_version 1410002 (0.0007) [2023-12-27 15:25:01,142][20347] Updated weights for policy 0, policy_version 1410012 (0.0006) [2023-12-27 15:25:01,152][20381] Updated weights for policy 1, policy_version 1408757 (0.0007) [2023-12-27 15:25:01,207][20347] Updated weights for policy 0, policy_version 1410022 (0.0006) [2023-12-27 15:25:01,212][20381] Updated weights for policy 1, policy_version 1408767 (0.0007) [2023-12-27 15:25:01,274][20381] Updated weights for policy 1, policy_version 1408777 (0.0008) [2023-12-27 15:25:01,910][20347] Updated weights for policy 0, policy_version 1410032 (0.0007) [2023-12-27 15:25:01,958][20347] Updated weights for policy 0, policy_version 1410042 (0.0007) [2023-12-27 15:25:01,986][20381] Updated weights for policy 1, policy_version 1408787 (0.0009) [2023-12-27 15:25:02,010][20347] Updated weights for policy 0, policy_version 1410052 (0.0006) [2023-12-27 15:25:02,040][20381] Updated weights for policy 1, policy_version 1408797 (0.0010) [2023-12-27 15:25:02,092][20381] Updated weights for policy 1, policy_version 1408807 (0.0010) [2023-12-27 15:25:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 721731584. Throughput: 0: 10469.8, 1: 10307.2. Samples: 721739268. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:02,363][19170] Avg episode reward: [(0, '872.680'), (1, '1000.000')] [2023-12-27 15:25:02,718][20347] Updated weights for policy 0, policy_version 1410062 (0.0007) [2023-12-27 15:25:02,772][20347] Updated weights for policy 0, policy_version 1410072 (0.0007) [2023-12-27 15:25:02,818][20347] Updated weights for policy 0, policy_version 1410082 (0.0007) [2023-12-27 15:25:02,823][20381] Updated weights for policy 1, policy_version 1408817 (0.0010) [2023-12-27 15:25:02,871][20381] Updated weights for policy 1, policy_version 1408827 (0.0007) [2023-12-27 15:25:02,921][20381] Updated weights for policy 1, policy_version 1408837 (0.0007) [2023-12-27 15:25:02,984][20381] Updated weights for policy 1, policy_version 1408847 (0.0007) [2023-12-27 15:25:03,444][20347] Updated weights for policy 0, policy_version 1410092 (0.0007) [2023-12-27 15:25:03,494][20347] Updated weights for policy 0, policy_version 1410102 (0.0005) [2023-12-27 15:25:03,545][20347] Updated weights for policy 0, policy_version 1410112 (0.0005) [2023-12-27 15:25:03,669][20381] Updated weights for policy 1, policy_version 1408857 (0.0011) [2023-12-27 15:25:03,717][20381] Updated weights for policy 1, policy_version 1408867 (0.0010) [2023-12-27 15:25:03,775][20381] Updated weights for policy 1, policy_version 1408877 (0.0008) [2023-12-27 15:25:04,169][20347] Updated weights for policy 0, policy_version 1410122 (0.0006) [2023-12-27 15:25:04,231][20347] Updated weights for policy 0, policy_version 1410132 (0.0011) [2023-12-27 15:25:04,301][20347] Updated weights for policy 0, policy_version 1410142 (0.0011) [2023-12-27 15:25:04,369][20347] Updated weights for policy 0, policy_version 1410152 (0.0011) [2023-12-27 15:25:04,458][20381] Updated weights for policy 1, policy_version 1408887 (0.0009) [2023-12-27 15:25:04,518][20381] Updated weights for policy 1, policy_version 1408897 (0.0010) [2023-12-27 15:25:04,579][20381] Updated weights for policy 1, policy_version 1408907 (0.0010) [2023-12-27 15:25:05,076][20347] Updated weights for policy 0, policy_version 1410162 (0.0010) [2023-12-27 15:25:05,129][20347] Updated weights for policy 0, policy_version 1410172 (0.0010) [2023-12-27 15:25:05,181][20347] Updated weights for policy 0, policy_version 1410182 (0.0010) [2023-12-27 15:25:05,259][20381] Updated weights for policy 1, policy_version 1408917 (0.0011) [2023-12-27 15:25:05,320][20381] Updated weights for policy 1, policy_version 1408927 (0.0010) [2023-12-27 15:25:05,379][20381] Updated weights for policy 1, policy_version 1408937 (0.0010) [2023-12-27 15:25:05,922][20347] Updated weights for policy 0, policy_version 1410192 (0.0010) [2023-12-27 15:25:05,974][20347] Updated weights for policy 0, policy_version 1410202 (0.0010) [2023-12-27 15:25:06,023][20381] Updated weights for policy 1, policy_version 1408947 (0.0010) [2023-12-27 15:25:06,025][20347] Updated weights for policy 0, policy_version 1410212 (0.0010) [2023-12-27 15:25:06,078][20381] Updated weights for policy 1, policy_version 1408957 (0.0010) [2023-12-27 15:25:06,148][20381] Updated weights for policy 1, policy_version 1408967 (0.0009) [2023-12-27 15:25:06,740][20347] Updated weights for policy 0, policy_version 1410222 (0.0011) [2023-12-27 15:25:06,798][20347] Updated weights for policy 0, policy_version 1410232 (0.0010) [2023-12-27 15:25:06,851][20381] Updated weights for policy 1, policy_version 1408977 (0.0011) [2023-12-27 15:25:06,858][20347] Updated weights for policy 0, policy_version 1410242 (0.0009) [2023-12-27 15:25:06,907][20381] Updated weights for policy 1, policy_version 1408987 (0.0009) [2023-12-27 15:25:06,966][20381] Updated weights for policy 1, policy_version 1408997 (0.0005) [2023-12-27 15:25:07,021][20381] Updated weights for policy 1, policy_version 1409007 (0.0006) [2023-12-27 15:25:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 721829888. Throughput: 0: 10464.3, 1: 10300.8. Samples: 721801308. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:07,364][19170] Avg episode reward: [(0, '892.560'), (1, '1000.000')] [2023-12-27 15:25:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001409008_360734720.pth... [2023-12-27 15:25:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001410248_361095168.pth... [2023-12-27 15:25:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001407824_360431616.pth [2023-12-27 15:25:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001409032_360783872.pth [2023-12-27 15:25:07,558][20347] Updated weights for policy 0, policy_version 1410252 (0.0008) [2023-12-27 15:25:07,622][20347] Updated weights for policy 0, policy_version 1410262 (0.0007) [2023-12-27 15:25:07,677][20381] Updated weights for policy 1, policy_version 1409017 (0.0008) [2023-12-27 15:25:07,685][20347] Updated weights for policy 0, policy_version 1410272 (0.0006) [2023-12-27 15:25:07,740][20381] Updated weights for policy 1, policy_version 1409027 (0.0008) [2023-12-27 15:25:07,809][20381] Updated weights for policy 1, policy_version 1409037 (0.0008) [2023-12-27 15:25:08,400][20347] Updated weights for policy 0, policy_version 1410282 (0.0006) [2023-12-27 15:25:08,453][20381] Updated weights for policy 1, policy_version 1409047 (0.0009) [2023-12-27 15:25:08,460][20347] Updated weights for policy 0, policy_version 1410292 (0.0009) [2023-12-27 15:25:08,516][20381] Updated weights for policy 1, policy_version 1409057 (0.0008) [2023-12-27 15:25:08,517][20347] Updated weights for policy 0, policy_version 1410302 (0.0006) [2023-12-27 15:25:08,575][20347] Updated weights for policy 0, policy_version 1410312 (0.0006) [2023-12-27 15:25:08,576][20381] Updated weights for policy 1, policy_version 1409067 (0.0008) [2023-12-27 15:25:09,241][20347] Updated weights for policy 0, policy_version 1410322 (0.0008) [2023-12-27 15:25:09,246][20381] Updated weights for policy 1, policy_version 1409077 (0.0009) [2023-12-27 15:25:09,298][20347] Updated weights for policy 0, policy_version 1410332 (0.0006) [2023-12-27 15:25:09,307][20381] Updated weights for policy 1, policy_version 1409087 (0.0008) [2023-12-27 15:25:09,361][20347] Updated weights for policy 0, policy_version 1410342 (0.0007) [2023-12-27 15:25:09,374][20381] Updated weights for policy 1, policy_version 1409097 (0.0008) [2023-12-27 15:25:10,073][20347] Updated weights for policy 0, policy_version 1410352 (0.0007) [2023-12-27 15:25:10,134][20347] Updated weights for policy 0, policy_version 1410362 (0.0008) [2023-12-27 15:25:10,193][20347] Updated weights for policy 0, policy_version 1410372 (0.0008) [2023-12-27 15:25:10,209][20381] Updated weights for policy 1, policy_version 1409107 (0.0008) [2023-12-27 15:25:10,257][20381] Updated weights for policy 1, policy_version 1409117 (0.0008) [2023-12-27 15:25:10,309][20381] Updated weights for policy 1, policy_version 1409127 (0.0008) [2023-12-27 15:25:10,808][20347] Updated weights for policy 0, policy_version 1410382 (0.0008) [2023-12-27 15:25:10,861][20347] Updated weights for policy 0, policy_version 1410392 (0.0007) [2023-12-27 15:25:10,917][20347] Updated weights for policy 0, policy_version 1410402 (0.0008) [2023-12-27 15:25:11,053][20381] Updated weights for policy 1, policy_version 1409137 (0.0007) [2023-12-27 15:25:11,118][20381] Updated weights for policy 1, policy_version 1409147 (0.0007) [2023-12-27 15:25:11,178][20381] Updated weights for policy 1, policy_version 1409157 (0.0008) [2023-12-27 15:25:11,233][20381] Updated weights for policy 1, policy_version 1409167 (0.0008) [2023-12-27 15:25:11,688][20347] Updated weights for policy 0, policy_version 1410412 (0.0007) [2023-12-27 15:25:11,753][20347] Updated weights for policy 0, policy_version 1410422 (0.0007) [2023-12-27 15:25:11,818][20347] Updated weights for policy 0, policy_version 1410432 (0.0007) [2023-12-27 15:25:11,940][20381] Updated weights for policy 1, policy_version 1409177 (0.0006) [2023-12-27 15:25:11,999][20381] Updated weights for policy 1, policy_version 1409187 (0.0006) [2023-12-27 15:25:12,056][20381] Updated weights for policy 1, policy_version 1409197 (0.0006) [2023-12-27 15:25:12,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 721928192. Throughput: 0: 10436.0, 1: 10178.8. Samples: 721919352. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:12,363][19170] Avg episode reward: [(0, '921.930'), (1, '1000.000')] [2023-12-27 15:25:12,525][20347] Updated weights for policy 0, policy_version 1410442 (0.0008) [2023-12-27 15:25:12,591][20347] Updated weights for policy 0, policy_version 1410452 (0.0011) [2023-12-27 15:25:12,650][20347] Updated weights for policy 0, policy_version 1410462 (0.0011) [2023-12-27 15:25:12,653][20381] Updated weights for policy 1, policy_version 1409207 (0.0009) [2023-12-27 15:25:12,710][20347] Updated weights for policy 0, policy_version 1410472 (0.0010) [2023-12-27 15:25:12,713][20381] Updated weights for policy 1, policy_version 1409217 (0.0011) [2023-12-27 15:25:12,769][20381] Updated weights for policy 1, policy_version 1409227 (0.0011) [2023-12-27 15:25:13,349][20347] Updated weights for policy 0, policy_version 1410482 (0.0010) [2023-12-27 15:25:13,407][20347] Updated weights for policy 0, policy_version 1410492 (0.0010) [2023-12-27 15:25:13,464][20381] Updated weights for policy 1, policy_version 1409237 (0.0011) [2023-12-27 15:25:13,468][20347] Updated weights for policy 0, policy_version 1410502 (0.0006) [2023-12-27 15:25:13,516][20381] Updated weights for policy 1, policy_version 1409247 (0.0010) [2023-12-27 15:25:13,584][20381] Updated weights for policy 1, policy_version 1409257 (0.0011) [2023-12-27 15:25:14,026][20347] Updated weights for policy 0, policy_version 1410512 (0.0009) [2023-12-27 15:25:14,084][20347] Updated weights for policy 0, policy_version 1410522 (0.0010) [2023-12-27 15:25:14,139][20347] Updated weights for policy 0, policy_version 1410532 (0.0010) [2023-12-27 15:25:14,279][20381] Updated weights for policy 1, policy_version 1409267 (0.0010) [2023-12-27 15:25:14,333][20381] Updated weights for policy 1, policy_version 1409277 (0.0009) [2023-12-27 15:25:14,384][20381] Updated weights for policy 1, policy_version 1409287 (0.0010) [2023-12-27 15:25:14,753][20347] Updated weights for policy 0, policy_version 1410542 (0.0009) [2023-12-27 15:25:14,816][20347] Updated weights for policy 0, policy_version 1410552 (0.0008) [2023-12-27 15:25:14,876][20347] Updated weights for policy 0, policy_version 1410562 (0.0008) [2023-12-27 15:25:15,135][20381] Updated weights for policy 1, policy_version 1409297 (0.0008) [2023-12-27 15:25:15,201][20381] Updated weights for policy 1, policy_version 1409307 (0.0007) [2023-12-27 15:25:15,259][20381] Updated weights for policy 1, policy_version 1409317 (0.0006) [2023-12-27 15:25:15,318][20381] Updated weights for policy 1, policy_version 1409327 (0.0005) [2023-12-27 15:25:15,583][20347] Updated weights for policy 0, policy_version 1410572 (0.0010) [2023-12-27 15:25:15,652][20347] Updated weights for policy 0, policy_version 1410582 (0.0010) [2023-12-27 15:25:15,722][20347] Updated weights for policy 0, policy_version 1410592 (0.0010) [2023-12-27 15:25:15,873][20381] Updated weights for policy 1, policy_version 1409337 (0.0006) [2023-12-27 15:25:15,931][20381] Updated weights for policy 1, policy_version 1409347 (0.0007) [2023-12-27 15:25:15,941][20292] KL-divergence is very high: 250.0025 [2023-12-27 15:25:15,988][20381] Updated weights for policy 1, policy_version 1409357 (0.0007) [2023-12-27 15:25:15,990][20292] KL-divergence is very high: 179.6654 [2023-12-27 15:25:16,350][20347] Updated weights for policy 0, policy_version 1410602 (0.0007) [2023-12-27 15:25:16,417][20347] Updated weights for policy 0, policy_version 1410612 (0.0010) [2023-12-27 15:25:16,466][20347] Updated weights for policy 0, policy_version 1410622 (0.0010) [2023-12-27 15:25:16,516][20347] Updated weights for policy 0, policy_version 1410632 (0.0010) [2023-12-27 15:25:16,600][20381] Updated weights for policy 1, policy_version 1409367 (0.0006) [2023-12-27 15:25:16,661][20381] Updated weights for policy 1, policy_version 1409377 (0.0005) [2023-12-27 15:25:16,722][20381] Updated weights for policy 1, policy_version 1409387 (0.0005) [2023-12-27 15:25:17,191][20347] Updated weights for policy 0, policy_version 1410642 (0.0010) [2023-12-27 15:25:17,258][20347] Updated weights for policy 0, policy_version 1410652 (0.0007) [2023-12-27 15:25:17,311][20347] Updated weights for policy 0, policy_version 1410662 (0.0007) [2023-12-27 15:25:17,345][20381] Updated weights for policy 1, policy_version 1409397 (0.0008) [2023-12-27 15:25:17,362][19170] Fps is (10 sec: 20480.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 722034688. Throughput: 0: 10387.4, 1: 10136.3. Samples: 722043800. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:17,363][19170] Avg episode reward: [(0, '941.350'), (1, '990.260')] [2023-12-27 15:25:17,406][20381] Updated weights for policy 1, policy_version 1409407 (0.0010) [2023-12-27 15:25:17,460][20381] Updated weights for policy 1, policy_version 1409417 (0.0010) [2023-12-27 15:25:17,922][20347] Updated weights for policy 0, policy_version 1410672 (0.0010) [2023-12-27 15:25:17,986][20347] Updated weights for policy 0, policy_version 1410682 (0.0010) [2023-12-27 15:25:18,043][20347] Updated weights for policy 0, policy_version 1410692 (0.0010) [2023-12-27 15:25:18,146][20381] Updated weights for policy 1, policy_version 1409427 (0.0010) [2023-12-27 15:25:18,198][20381] Updated weights for policy 1, policy_version 1409437 (0.0010) [2023-12-27 15:25:18,253][20381] Updated weights for policy 1, policy_version 1409447 (0.0010) [2023-12-27 15:25:18,719][20347] Updated weights for policy 0, policy_version 1410702 (0.0008) [2023-12-27 15:25:18,770][20347] Updated weights for policy 0, policy_version 1410712 (0.0007) [2023-12-27 15:25:18,819][20347] Updated weights for policy 0, policy_version 1410722 (0.0008) [2023-12-27 15:25:18,953][20381] Updated weights for policy 1, policy_version 1409457 (0.0010) [2023-12-27 15:25:19,010][20381] Updated weights for policy 1, policy_version 1409467 (0.0008) [2023-12-27 15:25:19,064][20381] Updated weights for policy 1, policy_version 1409477 (0.0007) [2023-12-27 15:25:19,112][20381] Updated weights for policy 1, policy_version 1409487 (0.0007) [2023-12-27 15:25:19,560][20347] Updated weights for policy 0, policy_version 1410732 (0.0007) [2023-12-27 15:25:19,624][20347] Updated weights for policy 0, policy_version 1410742 (0.0006) [2023-12-27 15:25:19,691][20347] Updated weights for policy 0, policy_version 1410752 (0.0007) [2023-12-27 15:25:19,860][20381] Updated weights for policy 1, policy_version 1409497 (0.0008) [2023-12-27 15:25:19,923][20381] Updated weights for policy 1, policy_version 1409507 (0.0006) [2023-12-27 15:25:19,991][20381] Updated weights for policy 1, policy_version 1409517 (0.0009) [2023-12-27 15:25:20,367][20347] Updated weights for policy 0, policy_version 1410762 (0.0008) [2023-12-27 15:25:20,432][20347] Updated weights for policy 0, policy_version 1410772 (0.0006) [2023-12-27 15:25:20,494][20347] Updated weights for policy 0, policy_version 1410782 (0.0006) [2023-12-27 15:25:20,549][20347] Updated weights for policy 0, policy_version 1410792 (0.0007) [2023-12-27 15:25:20,755][20381] Updated weights for policy 1, policy_version 1409527 (0.0009) [2023-12-27 15:25:20,817][20381] Updated weights for policy 1, policy_version 1409537 (0.0008) [2023-12-27 15:25:20,875][20381] Updated weights for policy 1, policy_version 1409547 (0.0008) [2023-12-27 15:25:21,179][20347] Updated weights for policy 0, policy_version 1410802 (0.0008) [2023-12-27 15:25:21,240][20347] Updated weights for policy 0, policy_version 1410812 (0.0007) [2023-12-27 15:25:21,307][20347] Updated weights for policy 0, policy_version 1410822 (0.0008) [2023-12-27 15:25:21,668][20381] Updated weights for policy 1, policy_version 1409557 (0.0008) [2023-12-27 15:25:21,731][20381] Updated weights for policy 1, policy_version 1409567 (0.0008) [2023-12-27 15:25:21,796][20381] Updated weights for policy 1, policy_version 1409577 (0.0007) [2023-12-27 15:25:22,002][20347] Updated weights for policy 0, policy_version 1410832 (0.0007) [2023-12-27 15:25:22,069][20347] Updated weights for policy 0, policy_version 1410842 (0.0008) [2023-12-27 15:25:22,126][20347] Updated weights for policy 0, policy_version 1410852 (0.0008) [2023-12-27 15:25:22,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 722132992. Throughput: 0: 10359.4, 1: 10070.1. Samples: 722103528. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:22,363][19170] Avg episode reward: [(0, '951.050'), (1, '990.260')] [2023-12-27 15:25:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001409584_360882176.pth... [2023-12-27 15:25:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001410856_361250816.pth... [2023-12-27 15:25:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001408400_360579072.pth [2023-12-27 15:25:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001409640_360939520.pth [2023-12-27 15:25:22,444][20381] Updated weights for policy 1, policy_version 1409587 (0.0008) [2023-12-27 15:25:22,501][20381] Updated weights for policy 1, policy_version 1409597 (0.0007) [2023-12-27 15:25:22,555][20381] Updated weights for policy 1, policy_version 1409607 (0.0005) [2023-12-27 15:25:22,860][20347] Updated weights for policy 0, policy_version 1410862 (0.0008) [2023-12-27 15:25:22,910][20347] Updated weights for policy 0, policy_version 1410872 (0.0007) [2023-12-27 15:25:22,966][20347] Updated weights for policy 0, policy_version 1410882 (0.0008) [2023-12-27 15:25:23,242][20381] Updated weights for policy 1, policy_version 1409617 (0.0007) [2023-12-27 15:25:23,294][20381] Updated weights for policy 1, policy_version 1409627 (0.0006) [2023-12-27 15:25:23,344][20381] Updated weights for policy 1, policy_version 1409637 (0.0005) [2023-12-27 15:25:23,399][20381] Updated weights for policy 1, policy_version 1409647 (0.0005) [2023-12-27 15:25:23,707][20347] Updated weights for policy 0, policy_version 1410892 (0.0007) [2023-12-27 15:25:23,755][20347] Updated weights for policy 0, policy_version 1410902 (0.0010) [2023-12-27 15:25:23,811][20347] Updated weights for policy 0, policy_version 1410912 (0.0010) [2023-12-27 15:25:23,969][20381] Updated weights for policy 1, policy_version 1409657 (0.0005) [2023-12-27 15:25:24,017][20381] Updated weights for policy 1, policy_version 1409667 (0.0005) [2023-12-27 15:25:24,062][20381] Updated weights for policy 1, policy_version 1409677 (0.0005) [2023-12-27 15:25:24,469][20347] Updated weights for policy 0, policy_version 1410922 (0.0010) [2023-12-27 15:25:24,521][20347] Updated weights for policy 0, policy_version 1410932 (0.0007) [2023-12-27 15:25:24,581][20347] Updated weights for policy 0, policy_version 1410942 (0.0008) [2023-12-27 15:25:24,618][20381] Updated weights for policy 1, policy_version 1409687 (0.0005) [2023-12-27 15:25:24,636][20347] Updated weights for policy 0, policy_version 1410952 (0.0008) [2023-12-27 15:25:24,665][20381] Updated weights for policy 1, policy_version 1409697 (0.0005) [2023-12-27 15:25:24,713][20381] Updated weights for policy 1, policy_version 1409707 (0.0005) [2023-12-27 15:25:25,332][20347] Updated weights for policy 0, policy_version 1410962 (0.0010) [2023-12-27 15:25:25,387][20381] Updated weights for policy 1, policy_version 1409717 (0.0007) [2023-12-27 15:25:25,390][20347] Updated weights for policy 0, policy_version 1410972 (0.0006) [2023-12-27 15:25:25,437][20381] Updated weights for policy 1, policy_version 1409727 (0.0006) [2023-12-27 15:25:25,440][20347] Updated weights for policy 0, policy_version 1410982 (0.0010) [2023-12-27 15:25:25,492][20381] Updated weights for policy 1, policy_version 1409737 (0.0007) [2023-12-27 15:25:26,057][20347] Updated weights for policy 0, policy_version 1410992 (0.0006) [2023-12-27 15:25:26,114][20347] Updated weights for policy 0, policy_version 1411002 (0.0005) [2023-12-27 15:25:26,167][20347] Updated weights for policy 0, policy_version 1411012 (0.0005) [2023-12-27 15:25:26,240][20381] Updated weights for policy 1, policy_version 1409747 (0.0008) [2023-12-27 15:25:26,295][20381] Updated weights for policy 1, policy_version 1409757 (0.0007) [2023-12-27 15:25:26,346][20381] Updated weights for policy 1, policy_version 1409768 (0.0009) [2023-12-27 15:25:26,686][20347] Updated weights for policy 0, policy_version 1411022 (0.0006) [2023-12-27 15:25:26,730][20347] Updated weights for policy 0, policy_version 1411032 (0.0007) [2023-12-27 15:25:26,775][20347] Updated weights for policy 0, policy_version 1411042 (0.0006) [2023-12-27 15:25:27,092][20381] Updated weights for policy 1, policy_version 1409778 (0.0009) [2023-12-27 15:25:27,142][20381] Updated weights for policy 1, policy_version 1409788 (0.0007) [2023-12-27 15:25:27,195][20381] Updated weights for policy 1, policy_version 1409798 (0.0008) [2023-12-27 15:25:27,242][20381] Updated weights for policy 1, policy_version 1409808 (0.0007) [2023-12-27 15:25:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 722239488. Throughput: 0: 10339.7, 1: 10149.3. Samples: 722226588. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:27,363][19170] Avg episode reward: [(0, '961.060'), (1, '990.310')] [2023-12-27 15:25:27,434][20347] Updated weights for policy 0, policy_version 1411052 (0.0005) [2023-12-27 15:25:27,485][20347] Updated weights for policy 0, policy_version 1411062 (0.0006) [2023-12-27 15:25:27,533][20347] Updated weights for policy 0, policy_version 1411072 (0.0009) [2023-12-27 15:25:27,846][20381] Updated weights for policy 1, policy_version 1409818 (0.0005) [2023-12-27 15:25:27,906][20381] Updated weights for policy 1, policy_version 1409828 (0.0007) [2023-12-27 15:25:27,964][20381] Updated weights for policy 1, policy_version 1409838 (0.0008) [2023-12-27 15:25:28,206][20347] Updated weights for policy 0, policy_version 1411082 (0.0010) [2023-12-27 15:25:28,261][20347] Updated weights for policy 0, policy_version 1411092 (0.0007) [2023-12-27 15:25:28,312][20347] Updated weights for policy 0, policy_version 1411102 (0.0007) [2023-12-27 15:25:28,378][20347] Updated weights for policy 0, policy_version 1411112 (0.0007) [2023-12-27 15:25:28,586][20381] Updated weights for policy 1, policy_version 1409848 (0.0008) [2023-12-27 15:25:28,638][20381] Updated weights for policy 1, policy_version 1409858 (0.0007) [2023-12-27 15:25:28,684][20381] Updated weights for policy 1, policy_version 1409868 (0.0008) [2023-12-27 15:25:29,023][20347] Updated weights for policy 0, policy_version 1411122 (0.0008) [2023-12-27 15:25:29,079][20347] Updated weights for policy 0, policy_version 1411132 (0.0008) [2023-12-27 15:25:29,137][20347] Updated weights for policy 0, policy_version 1411142 (0.0007) [2023-12-27 15:25:29,434][20381] Updated weights for policy 1, policy_version 1409878 (0.0007) [2023-12-27 15:25:29,489][20381] Updated weights for policy 1, policy_version 1409888 (0.0007) [2023-12-27 15:25:29,544][20381] Updated weights for policy 1, policy_version 1409898 (0.0008) [2023-12-27 15:25:29,863][20347] Updated weights for policy 0, policy_version 1411152 (0.0008) [2023-12-27 15:25:29,912][20347] Updated weights for policy 0, policy_version 1411162 (0.0007) [2023-12-27 15:25:29,978][20347] Updated weights for policy 0, policy_version 1411172 (0.0008) [2023-12-27 15:25:30,222][20381] Updated weights for policy 1, policy_version 1409908 (0.0006) [2023-12-27 15:25:30,269][20381] Updated weights for policy 1, policy_version 1409918 (0.0008) [2023-12-27 15:25:30,313][20381] Updated weights for policy 1, policy_version 1409928 (0.0006) [2023-12-27 15:25:30,709][20347] Updated weights for policy 0, policy_version 1411182 (0.0007) [2023-12-27 15:25:30,757][20347] Updated weights for policy 0, policy_version 1411192 (0.0006) [2023-12-27 15:25:30,802][20347] Updated weights for policy 0, policy_version 1411202 (0.0007) [2023-12-27 15:25:30,940][20381] Updated weights for policy 1, policy_version 1409938 (0.0006) [2023-12-27 15:25:31,006][20381] Updated weights for policy 1, policy_version 1409948 (0.0006) [2023-12-27 15:25:31,069][20381] Updated weights for policy 1, policy_version 1409958 (0.0007) [2023-12-27 15:25:31,128][20381] Updated weights for policy 1, policy_version 1409968 (0.0007) [2023-12-27 15:25:31,538][20347] Updated weights for policy 0, policy_version 1411212 (0.0009) [2023-12-27 15:25:31,596][20347] Updated weights for policy 0, policy_version 1411222 (0.0010) [2023-12-27 15:25:31,654][20347] Updated weights for policy 0, policy_version 1411232 (0.0010) [2023-12-27 15:25:31,816][20381] Updated weights for policy 1, policy_version 1409978 (0.0008) [2023-12-27 15:25:31,874][20381] Updated weights for policy 1, policy_version 1409988 (0.0008) [2023-12-27 15:25:31,930][20381] Updated weights for policy 1, policy_version 1409998 (0.0008) [2023-12-27 15:25:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 722337792. Throughput: 0: 10293.2, 1: 10150.0. Samples: 722350588. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:32,363][19170] Avg episode reward: [(0, '931.700'), (1, '990.310')] [2023-12-27 15:25:32,417][20347] Updated weights for policy 0, policy_version 1411242 (0.0008) [2023-12-27 15:25:32,469][20347] Updated weights for policy 0, policy_version 1411252 (0.0009) [2023-12-27 15:25:32,517][20347] Updated weights for policy 0, policy_version 1411262 (0.0007) [2023-12-27 15:25:32,571][20347] Updated weights for policy 0, policy_version 1411272 (0.0007) [2023-12-27 15:25:32,651][20381] Updated weights for policy 1, policy_version 1410008 (0.0010) [2023-12-27 15:25:32,703][20381] Updated weights for policy 1, policy_version 1410018 (0.0011) [2023-12-27 15:25:32,759][20381] Updated weights for policy 1, policy_version 1410028 (0.0010) [2023-12-27 15:25:33,265][20347] Updated weights for policy 0, policy_version 1411282 (0.0005) [2023-12-27 15:25:33,315][20347] Updated weights for policy 0, policy_version 1411292 (0.0005) [2023-12-27 15:25:33,370][20347] Updated weights for policy 0, policy_version 1411302 (0.0005) [2023-12-27 15:25:33,376][20381] Updated weights for policy 1, policy_version 1410038 (0.0007) [2023-12-27 15:25:33,422][20381] Updated weights for policy 1, policy_version 1410048 (0.0005) [2023-12-27 15:25:33,477][20381] Updated weights for policy 1, policy_version 1410058 (0.0009) [2023-12-27 15:25:33,885][20347] Updated weights for policy 0, policy_version 1411312 (0.0009) [2023-12-27 15:25:33,929][20347] Updated weights for policy 0, policy_version 1411322 (0.0010) [2023-12-27 15:25:33,979][20347] Updated weights for policy 0, policy_version 1411332 (0.0005) [2023-12-27 15:25:34,101][20381] Updated weights for policy 1, policy_version 1410068 (0.0009) [2023-12-27 15:25:34,162][20381] Updated weights for policy 1, policy_version 1410078 (0.0010) [2023-12-27 15:25:34,221][20381] Updated weights for policy 1, policy_version 1410089 (0.0008) [2023-12-27 15:25:34,629][20347] Updated weights for policy 0, policy_version 1411342 (0.0007) [2023-12-27 15:25:34,684][20347] Updated weights for policy 0, policy_version 1411352 (0.0008) [2023-12-27 15:25:34,737][20347] Updated weights for policy 0, policy_version 1411362 (0.0007) [2023-12-27 15:25:34,923][20381] Updated weights for policy 1, policy_version 1410099 (0.0008) [2023-12-27 15:25:34,991][20381] Updated weights for policy 1, policy_version 1410109 (0.0009) [2023-12-27 15:25:35,051][20381] Updated weights for policy 1, policy_version 1410119 (0.0007) [2023-12-27 15:25:35,352][20347] Updated weights for policy 0, policy_version 1411372 (0.0006) [2023-12-27 15:25:35,403][20347] Updated weights for policy 0, policy_version 1411382 (0.0005) [2023-12-27 15:25:35,452][20347] Updated weights for policy 0, policy_version 1411392 (0.0005) [2023-12-27 15:25:35,774][20381] Updated weights for policy 1, policy_version 1410129 (0.0007) [2023-12-27 15:25:35,831][20381] Updated weights for policy 1, policy_version 1410139 (0.0008) [2023-12-27 15:25:35,886][20381] Updated weights for policy 1, policy_version 1410149 (0.0008) [2023-12-27 15:25:35,941][20381] Updated weights for policy 1, policy_version 1410159 (0.0008) [2023-12-27 15:25:36,138][20347] Updated weights for policy 0, policy_version 1411402 (0.0006) [2023-12-27 15:25:36,199][20347] Updated weights for policy 0, policy_version 1411412 (0.0008) [2023-12-27 15:25:36,262][20347] Updated weights for policy 0, policy_version 1411422 (0.0007) [2023-12-27 15:25:36,325][20347] Updated weights for policy 0, policy_version 1411432 (0.0006) [2023-12-27 15:25:36,655][20381] Updated weights for policy 1, policy_version 1410169 (0.0008) [2023-12-27 15:25:36,717][20381] Updated weights for policy 1, policy_version 1410179 (0.0008) [2023-12-27 15:25:36,780][20381] Updated weights for policy 1, policy_version 1410189 (0.0008) [2023-12-27 15:25:36,983][20347] Updated weights for policy 0, policy_version 1411442 (0.0010) [2023-12-27 15:25:37,034][20347] Updated weights for policy 0, policy_version 1411453 (0.0009) [2023-12-27 15:25:37,086][20347] Updated weights for policy 0, policy_version 1411463 (0.0005) [2023-12-27 15:25:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 722444288. Throughput: 0: 10359.1, 1: 10189.1. Samples: 722414864. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:37,363][19170] Avg episode reward: [(0, '961.050'), (1, '1000.000')] [2023-12-27 15:25:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001410192_361037824.pth... [2023-12-27 15:25:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001411464_361406464.pth... [2023-12-27 15:25:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001409008_360734720.pth [2023-12-27 15:25:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001410248_361095168.pth [2023-12-27 15:25:37,440][20381] Updated weights for policy 1, policy_version 1410199 (0.0008) [2023-12-27 15:25:37,497][20381] Updated weights for policy 1, policy_version 1410209 (0.0008) [2023-12-27 15:25:37,543][20381] Updated weights for policy 1, policy_version 1410219 (0.0008) [2023-12-27 15:25:37,756][20347] Updated weights for policy 0, policy_version 1411473 (0.0007) [2023-12-27 15:25:37,810][20347] Updated weights for policy 0, policy_version 1411483 (0.0007) [2023-12-27 15:25:37,860][20347] Updated weights for policy 0, policy_version 1411493 (0.0007) [2023-12-27 15:25:38,260][20381] Updated weights for policy 1, policy_version 1410229 (0.0009) [2023-12-27 15:25:38,319][20381] Updated weights for policy 1, policy_version 1410239 (0.0008) [2023-12-27 15:25:38,391][20381] Updated weights for policy 1, policy_version 1410249 (0.0011) [2023-12-27 15:25:38,489][20347] Updated weights for policy 0, policy_version 1411503 (0.0009) [2023-12-27 15:25:38,551][20347] Updated weights for policy 0, policy_version 1411513 (0.0009) [2023-12-27 15:25:38,615][20347] Updated weights for policy 0, policy_version 1411523 (0.0010) [2023-12-27 15:25:39,076][20381] Updated weights for policy 1, policy_version 1410259 (0.0011) [2023-12-27 15:25:39,136][20381] Updated weights for policy 1, policy_version 1410269 (0.0010) [2023-12-27 15:25:39,190][20381] Updated weights for policy 1, policy_version 1410279 (0.0010) [2023-12-27 15:25:39,287][20347] Updated weights for policy 0, policy_version 1411533 (0.0008) [2023-12-27 15:25:39,343][20347] Updated weights for policy 0, policy_version 1411543 (0.0008) [2023-12-27 15:25:39,409][20347] Updated weights for policy 0, policy_version 1411553 (0.0009) [2023-12-27 15:25:39,873][20381] Updated weights for policy 1, policy_version 1410289 (0.0011) [2023-12-27 15:25:39,939][20381] Updated weights for policy 1, policy_version 1410299 (0.0008) [2023-12-27 15:25:39,998][20381] Updated weights for policy 1, policy_version 1410309 (0.0008) [2023-12-27 15:25:40,060][20381] Updated weights for policy 1, policy_version 1410319 (0.0008) [2023-12-27 15:25:40,124][20347] Updated weights for policy 0, policy_version 1411563 (0.0008) [2023-12-27 15:25:40,179][20347] Updated weights for policy 0, policy_version 1411573 (0.0008) [2023-12-27 15:25:40,236][20347] Updated weights for policy 0, policy_version 1411583 (0.0007) [2023-12-27 15:25:40,735][20381] Updated weights for policy 1, policy_version 1410329 (0.0008) [2023-12-27 15:25:40,796][20381] Updated weights for policy 1, policy_version 1410339 (0.0008) [2023-12-27 15:25:40,845][20381] Updated weights for policy 1, policy_version 1410349 (0.0007) [2023-12-27 15:25:40,955][20347] Updated weights for policy 0, policy_version 1411593 (0.0008) [2023-12-27 15:25:41,018][20347] Updated weights for policy 0, policy_version 1411603 (0.0008) [2023-12-27 15:25:41,089][20347] Updated weights for policy 0, policy_version 1411613 (0.0007) [2023-12-27 15:25:41,162][20347] Updated weights for policy 0, policy_version 1411623 (0.0009) [2023-12-27 15:25:41,595][20381] Updated weights for policy 1, policy_version 1410359 (0.0007) [2023-12-27 15:25:41,660][20381] Updated weights for policy 1, policy_version 1410369 (0.0009) [2023-12-27 15:25:41,724][20381] Updated weights for policy 1, policy_version 1410379 (0.0007) [2023-12-27 15:25:41,920][20347] Updated weights for policy 0, policy_version 1411633 (0.0006) [2023-12-27 15:25:41,979][20347] Updated weights for policy 0, policy_version 1411643 (0.0010) [2023-12-27 15:25:42,041][20347] Updated weights for policy 0, policy_version 1411653 (0.0010) [2023-12-27 15:25:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 722542592. Throughput: 0: 10298.9, 1: 10134.4. Samples: 722535164. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:42,363][19170] Avg episode reward: [(0, '960.900'), (1, '1000.000')] [2023-12-27 15:25:42,459][20381] Updated weights for policy 1, policy_version 1410389 (0.0007) [2023-12-27 15:25:42,505][20381] Updated weights for policy 1, policy_version 1410399 (0.0007) [2023-12-27 15:25:42,558][20381] Updated weights for policy 1, policy_version 1410409 (0.0008) [2023-12-27 15:25:42,731][20347] Updated weights for policy 0, policy_version 1411663 (0.0009) [2023-12-27 15:25:42,784][20347] Updated weights for policy 0, policy_version 1411673 (0.0007) [2023-12-27 15:25:42,831][20347] Updated weights for policy 0, policy_version 1411683 (0.0008) [2023-12-27 15:25:43,218][20381] Updated weights for policy 1, policy_version 1410419 (0.0007) [2023-12-27 15:25:43,275][20381] Updated weights for policy 1, policy_version 1410429 (0.0006) [2023-12-27 15:25:43,320][20381] Updated weights for policy 1, policy_version 1410439 (0.0006) [2023-12-27 15:25:43,500][20347] Updated weights for policy 0, policy_version 1411693 (0.0006) [2023-12-27 15:25:43,553][20347] Updated weights for policy 0, policy_version 1411703 (0.0007) [2023-12-27 15:25:43,605][20347] Updated weights for policy 0, policy_version 1411713 (0.0007) [2023-12-27 15:25:43,924][20381] Updated weights for policy 1, policy_version 1410449 (0.0006) [2023-12-27 15:25:43,976][20381] Updated weights for policy 1, policy_version 1410459 (0.0008) [2023-12-27 15:25:44,029][20381] Updated weights for policy 1, policy_version 1410469 (0.0008) [2023-12-27 15:25:44,078][20381] Updated weights for policy 1, policy_version 1410479 (0.0008) [2023-12-27 15:25:44,273][20347] Updated weights for policy 0, policy_version 1411723 (0.0008) [2023-12-27 15:25:44,337][20347] Updated weights for policy 0, policy_version 1411733 (0.0005) [2023-12-27 15:25:44,396][20347] Updated weights for policy 0, policy_version 1411743 (0.0007) [2023-12-27 15:25:44,749][20381] Updated weights for policy 1, policy_version 1410489 (0.0008) [2023-12-27 15:25:44,824][20381] Updated weights for policy 1, policy_version 1410500 (0.0008) [2023-12-27 15:25:44,889][20381] Updated weights for policy 1, policy_version 1410510 (0.0006) [2023-12-27 15:25:45,130][20347] Updated weights for policy 0, policy_version 1411753 (0.0007) [2023-12-27 15:25:45,188][20347] Updated weights for policy 0, policy_version 1411763 (0.0009) [2023-12-27 15:25:45,241][20347] Updated weights for policy 0, policy_version 1411773 (0.0009) [2023-12-27 15:25:45,292][20347] Updated weights for policy 0, policy_version 1411783 (0.0010) [2023-12-27 15:25:45,462][20381] Updated weights for policy 1, policy_version 1410520 (0.0007) [2023-12-27 15:25:45,514][20381] Updated weights for policy 1, policy_version 1410530 (0.0005) [2023-12-27 15:25:45,562][20381] Updated weights for policy 1, policy_version 1410540 (0.0005) [2023-12-27 15:25:45,936][20347] Updated weights for policy 0, policy_version 1411793 (0.0006) [2023-12-27 15:25:45,991][20347] Updated weights for policy 0, policy_version 1411803 (0.0007) [2023-12-27 15:25:46,049][20347] Updated weights for policy 0, policy_version 1411813 (0.0010) [2023-12-27 15:25:46,174][20381] Updated weights for policy 1, policy_version 1410550 (0.0006) [2023-12-27 15:25:46,223][20381] Updated weights for policy 1, policy_version 1410560 (0.0008) [2023-12-27 15:25:46,279][20381] Updated weights for policy 1, policy_version 1410570 (0.0008) [2023-12-27 15:25:46,700][20347] Updated weights for policy 0, policy_version 1411823 (0.0010) [2023-12-27 15:25:46,758][20347] Updated weights for policy 0, policy_version 1411833 (0.0010) [2023-12-27 15:25:46,810][20347] Updated weights for policy 0, policy_version 1411843 (0.0010) [2023-12-27 15:25:46,970][20381] Updated weights for policy 1, policy_version 1410580 (0.0008) [2023-12-27 15:25:47,014][20381] Updated weights for policy 1, policy_version 1410590 (0.0008) [2023-12-27 15:25:47,062][20381] Updated weights for policy 1, policy_version 1410600 (0.0007) [2023-12-27 15:25:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 722649088. Throughput: 0: 10249.2, 1: 10185.7. Samples: 722658836. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:47,363][19170] Avg episode reward: [(0, '951.080'), (1, '1000.000')] [2023-12-27 15:25:47,444][20347] Updated weights for policy 0, policy_version 1411853 (0.0008) [2023-12-27 15:25:47,497][20347] Updated weights for policy 0, policy_version 1411863 (0.0005) [2023-12-27 15:25:47,543][20347] Updated weights for policy 0, policy_version 1411873 (0.0005) [2023-12-27 15:25:47,694][20381] Updated weights for policy 1, policy_version 1410610 (0.0007) [2023-12-27 15:25:47,748][20381] Updated weights for policy 1, policy_version 1410620 (0.0008) [2023-12-27 15:25:47,800][20381] Updated weights for policy 1, policy_version 1410630 (0.0007) [2023-12-27 15:25:47,850][20381] Updated weights for policy 1, policy_version 1410640 (0.0008) [2023-12-27 15:25:48,191][20347] Updated weights for policy 0, policy_version 1411883 (0.0008) [2023-12-27 15:25:48,241][20347] Updated weights for policy 0, policy_version 1411893 (0.0008) [2023-12-27 15:25:48,288][20347] Updated weights for policy 0, policy_version 1411903 (0.0007) [2023-12-27 15:25:48,558][20381] Updated weights for policy 1, policy_version 1410650 (0.0008) [2023-12-27 15:25:48,612][20381] Updated weights for policy 1, policy_version 1410660 (0.0007) [2023-12-27 15:25:48,667][20381] Updated weights for policy 1, policy_version 1410670 (0.0005) [2023-12-27 15:25:49,009][20347] Updated weights for policy 0, policy_version 1411913 (0.0007) [2023-12-27 15:25:49,070][20347] Updated weights for policy 0, policy_version 1411923 (0.0005) [2023-12-27 15:25:49,125][20347] Updated weights for policy 0, policy_version 1411933 (0.0005) [2023-12-27 15:25:49,178][20347] Updated weights for policy 0, policy_version 1411943 (0.0005) [2023-12-27 15:25:49,299][20381] Updated weights for policy 1, policy_version 1410680 (0.0008) [2023-12-27 15:25:49,359][20381] Updated weights for policy 1, policy_version 1410690 (0.0007) [2023-12-27 15:25:49,430][20381] Updated weights for policy 1, policy_version 1410700 (0.0009) [2023-12-27 15:25:49,808][20347] Updated weights for policy 0, policy_version 1411953 (0.0008) [2023-12-27 15:25:49,879][20347] Updated weights for policy 0, policy_version 1411963 (0.0011) [2023-12-27 15:25:49,942][20347] Updated weights for policy 0, policy_version 1411973 (0.0011) [2023-12-27 15:25:50,153][20381] Updated weights for policy 1, policy_version 1410710 (0.0009) [2023-12-27 15:25:50,211][20381] Updated weights for policy 1, policy_version 1410720 (0.0007) [2023-12-27 15:25:50,260][20381] Updated weights for policy 1, policy_version 1410730 (0.0007) [2023-12-27 15:25:50,549][20347] Updated weights for policy 0, policy_version 1411983 (0.0007) [2023-12-27 15:25:50,616][20347] Updated weights for policy 0, policy_version 1411993 (0.0009) [2023-12-27 15:25:50,671][20347] Updated weights for policy 0, policy_version 1412003 (0.0008) [2023-12-27 15:25:50,996][20381] Updated weights for policy 1, policy_version 1410740 (0.0009) [2023-12-27 15:25:51,056][20381] Updated weights for policy 1, policy_version 1410750 (0.0008) [2023-12-27 15:25:51,109][20381] Updated weights for policy 1, policy_version 1410760 (0.0008) [2023-12-27 15:25:51,364][20347] Updated weights for policy 0, policy_version 1412013 (0.0009) [2023-12-27 15:25:51,431][20347] Updated weights for policy 0, policy_version 1412023 (0.0007) [2023-12-27 15:25:51,490][20347] Updated weights for policy 0, policy_version 1412033 (0.0008) [2023-12-27 15:25:51,835][20381] Updated weights for policy 1, policy_version 1410770 (0.0007) [2023-12-27 15:25:51,884][20381] Updated weights for policy 1, policy_version 1410780 (0.0006) [2023-12-27 15:25:51,938][20381] Updated weights for policy 1, policy_version 1410790 (0.0006) [2023-12-27 15:25:51,988][20381] Updated weights for policy 1, policy_version 1410800 (0.0006) [2023-12-27 15:25:52,200][20347] Updated weights for policy 0, policy_version 1412043 (0.0007) [2023-12-27 15:25:52,263][20347] Updated weights for policy 0, policy_version 1412053 (0.0008) [2023-12-27 15:25:52,323][20347] Updated weights for policy 0, policy_version 1412063 (0.0007) [2023-12-27 15:25:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 722747392. Throughput: 0: 10256.0, 1: 10196.7. Samples: 722721672. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:52,363][19170] Avg episode reward: [(0, '970.660'), (1, '1000.000')] [2023-12-27 15:25:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001410800_361193472.pth... [2023-12-27 15:25:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001409584_360882176.pth [2023-12-27 15:25:52,385][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001412072_361562112.pth... [2023-12-27 15:25:52,390][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001410856_361250816.pth [2023-12-27 15:25:52,665][20381] Updated weights for policy 1, policy_version 1410810 (0.0007) [2023-12-27 15:25:52,721][20381] Updated weights for policy 1, policy_version 1410820 (0.0008) [2023-12-27 15:25:52,776][20381] Updated weights for policy 1, policy_version 1410830 (0.0007) [2023-12-27 15:25:53,031][20347] Updated weights for policy 0, policy_version 1412073 (0.0008) [2023-12-27 15:25:53,085][20347] Updated weights for policy 0, policy_version 1412083 (0.0008) [2023-12-27 15:25:53,140][20347] Updated weights for policy 0, policy_version 1412093 (0.0010) [2023-12-27 15:25:53,194][20347] Updated weights for policy 0, policy_version 1412103 (0.0010) [2023-12-27 15:25:53,482][20381] Updated weights for policy 1, policy_version 1410840 (0.0007) [2023-12-27 15:25:53,537][20381] Updated weights for policy 1, policy_version 1410850 (0.0008) [2023-12-27 15:25:53,592][20381] Updated weights for policy 1, policy_version 1410860 (0.0007) [2023-12-27 15:25:53,903][20347] Updated weights for policy 0, policy_version 1412113 (0.0008) [2023-12-27 15:25:53,962][20347] Updated weights for policy 0, policy_version 1412123 (0.0007) [2023-12-27 15:25:54,019][20347] Updated weights for policy 0, policy_version 1412133 (0.0007) [2023-12-27 15:25:54,248][20381] Updated weights for policy 1, policy_version 1410870 (0.0008) [2023-12-27 15:25:54,301][20381] Updated weights for policy 1, policy_version 1410880 (0.0008) [2023-12-27 15:25:54,363][20381] Updated weights for policy 1, policy_version 1410890 (0.0008) [2023-12-27 15:25:54,642][20347] Updated weights for policy 0, policy_version 1412143 (0.0007) [2023-12-27 15:25:54,698][20347] Updated weights for policy 0, policy_version 1412153 (0.0009) [2023-12-27 15:25:54,754][20347] Updated weights for policy 0, policy_version 1412163 (0.0008) [2023-12-27 15:25:54,995][20381] Updated weights for policy 1, policy_version 1410900 (0.0008) [2023-12-27 15:25:55,058][20381] Updated weights for policy 1, policy_version 1410910 (0.0008) [2023-12-27 15:25:55,119][20381] Updated weights for policy 1, policy_version 1410920 (0.0007) [2023-12-27 15:25:55,390][20347] Updated weights for policy 0, policy_version 1412173 (0.0006) [2023-12-27 15:25:55,447][20347] Updated weights for policy 0, policy_version 1412183 (0.0005) [2023-12-27 15:25:55,498][20347] Updated weights for policy 0, policy_version 1412193 (0.0005) [2023-12-27 15:25:55,802][20381] Updated weights for policy 1, policy_version 1410930 (0.0009) [2023-12-27 15:25:55,857][20381] Updated weights for policy 1, policy_version 1410940 (0.0008) [2023-12-27 15:25:55,909][20381] Updated weights for policy 1, policy_version 1410950 (0.0007) [2023-12-27 15:25:55,963][20381] Updated weights for policy 1, policy_version 1410960 (0.0006) [2023-12-27 15:25:56,147][20347] Updated weights for policy 0, policy_version 1412203 (0.0006) [2023-12-27 15:25:56,210][20347] Updated weights for policy 0, policy_version 1412213 (0.0008) [2023-12-27 15:25:56,273][20347] Updated weights for policy 0, policy_version 1412223 (0.0008) [2023-12-27 15:25:56,549][20381] Updated weights for policy 1, policy_version 1410970 (0.0010) [2023-12-27 15:25:56,594][20381] Updated weights for policy 1, policy_version 1410980 (0.0008) [2023-12-27 15:25:56,647][20381] Updated weights for policy 1, policy_version 1410990 (0.0008) [2023-12-27 15:25:56,934][20347] Updated weights for policy 0, policy_version 1412233 (0.0008) [2023-12-27 15:25:56,996][20347] Updated weights for policy 0, policy_version 1412243 (0.0005) [2023-12-27 15:25:57,047][20347] Updated weights for policy 0, policy_version 1412253 (0.0005) [2023-12-27 15:25:57,099][20347] Updated weights for policy 0, policy_version 1412263 (0.0005) [2023-12-27 15:25:57,340][20381] Updated weights for policy 1, policy_version 1411000 (0.0008) [2023-12-27 15:25:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 722853888. Throughput: 0: 10311.4, 1: 10254.3. Samples: 722844808. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-12-27 15:25:57,363][19170] Avg episode reward: [(0, '961.230'), (1, '1000.000')] [2023-12-27 15:25:57,391][20381] Updated weights for policy 1, policy_version 1411010 (0.0008) [2023-12-27 15:25:57,455][20381] Updated weights for policy 1, policy_version 1411020 (0.0007) [2023-12-27 15:25:57,697][20347] Updated weights for policy 0, policy_version 1412273 (0.0007) [2023-12-27 15:25:57,750][20347] Updated weights for policy 0, policy_version 1412283 (0.0006) [2023-12-27 15:25:57,799][20347] Updated weights for policy 0, policy_version 1412293 (0.0005) [2023-12-27 15:25:58,140][20381] Updated weights for policy 1, policy_version 1411030 (0.0008) [2023-12-27 15:25:58,199][20381] Updated weights for policy 1, policy_version 1411040 (0.0007) [2023-12-27 15:25:58,262][20381] Updated weights for policy 1, policy_version 1411050 (0.0007) [2023-12-27 15:25:58,490][20347] Updated weights for policy 0, policy_version 1412303 (0.0008) [2023-12-27 15:25:58,554][20347] Updated weights for policy 0, policy_version 1412313 (0.0008) [2023-12-27 15:25:58,618][20347] Updated weights for policy 0, policy_version 1412323 (0.0008) [2023-12-27 15:25:59,044][20381] Updated weights for policy 1, policy_version 1411060 (0.0006) [2023-12-27 15:25:59,104][20381] Updated weights for policy 1, policy_version 1411070 (0.0007) [2023-12-27 15:25:59,163][20381] Updated weights for policy 1, policy_version 1411080 (0.0006) [2023-12-27 15:25:59,363][20347] Updated weights for policy 0, policy_version 1412333 (0.0008) [2023-12-27 15:25:59,428][20347] Updated weights for policy 0, policy_version 1412343 (0.0008) [2023-12-27 15:25:59,480][20347] Updated weights for policy 0, policy_version 1412353 (0.0007) [2023-12-27 15:25:59,869][20381] Updated weights for policy 1, policy_version 1411090 (0.0007) [2023-12-27 15:25:59,931][20381] Updated weights for policy 1, policy_version 1411100 (0.0008) [2023-12-27 15:25:59,989][20381] Updated weights for policy 1, policy_version 1411110 (0.0008) [2023-12-27 15:26:00,043][20381] Updated weights for policy 1, policy_version 1411120 (0.0008) [2023-12-27 15:26:00,208][20347] Updated weights for policy 0, policy_version 1412363 (0.0008) [2023-12-27 15:26:00,257][20347] Updated weights for policy 0, policy_version 1412373 (0.0008) [2023-12-27 15:26:00,309][20347] Updated weights for policy 0, policy_version 1412383 (0.0006) [2023-12-27 15:26:00,625][20381] Updated weights for policy 1, policy_version 1411130 (0.0006) [2023-12-27 15:26:00,678][20381] Updated weights for policy 1, policy_version 1411140 (0.0005) [2023-12-27 15:26:00,732][20381] Updated weights for policy 1, policy_version 1411150 (0.0006) [2023-12-27 15:26:01,043][20347] Updated weights for policy 0, policy_version 1412393 (0.0006) [2023-12-27 15:26:01,107][20347] Updated weights for policy 0, policy_version 1412403 (0.0007) [2023-12-27 15:26:01,165][20347] Updated weights for policy 0, policy_version 1412413 (0.0008) [2023-12-27 15:26:01,224][20347] Updated weights for policy 0, policy_version 1412423 (0.0008) [2023-12-27 15:26:01,412][20381] Updated weights for policy 1, policy_version 1411160 (0.0008) [2023-12-27 15:26:01,463][20381] Updated weights for policy 1, policy_version 1411170 (0.0007) [2023-12-27 15:26:01,522][20381] Updated weights for policy 1, policy_version 1411180 (0.0007) [2023-12-27 15:26:01,962][20347] Updated weights for policy 0, policy_version 1412433 (0.0008) [2023-12-27 15:26:02,021][20347] Updated weights for policy 0, policy_version 1412443 (0.0008) [2023-12-27 15:26:02,082][20347] Updated weights for policy 0, policy_version 1412453 (0.0008) [2023-12-27 15:26:02,152][20381] Updated weights for policy 1, policy_version 1411190 (0.0009) [2023-12-27 15:26:02,209][20381] Updated weights for policy 1, policy_version 1411200 (0.0010) [2023-12-27 15:26:02,273][20381] Updated weights for policy 1, policy_version 1411210 (0.0011) [2023-12-27 15:26:02,364][19170] Fps is (10 sec: 21296.5, 60 sec: 20479.5, 300 sec: 20577.1). Total num frames: 722960384. Throughput: 0: 10246.2, 1: 10228.8. Samples: 722965204. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:02,365][19170] Avg episode reward: [(0, '961.210'), (1, '1000.000')] [2023-12-27 15:26:02,792][20347] Updated weights for policy 0, policy_version 1412463 (0.0008) [2023-12-27 15:26:02,850][20347] Updated weights for policy 0, policy_version 1412473 (0.0007) [2023-12-27 15:26:02,912][20347] Updated weights for policy 0, policy_version 1412483 (0.0008) [2023-12-27 15:26:02,955][20381] Updated weights for policy 1, policy_version 1411220 (0.0009) [2023-12-27 15:26:03,013][20381] Updated weights for policy 1, policy_version 1411230 (0.0008) [2023-12-27 15:26:03,071][20381] Updated weights for policy 1, policy_version 1411240 (0.0007) [2023-12-27 15:26:03,598][20347] Updated weights for policy 0, policy_version 1412493 (0.0007) [2023-12-27 15:26:03,646][20347] Updated weights for policy 0, policy_version 1412503 (0.0008) [2023-12-27 15:26:03,702][20347] Updated weights for policy 0, policy_version 1412513 (0.0008) [2023-12-27 15:26:03,764][20381] Updated weights for policy 1, policy_version 1411250 (0.0007) [2023-12-27 15:26:03,811][20381] Updated weights for policy 1, policy_version 1411260 (0.0007) [2023-12-27 15:26:03,876][20381] Updated weights for policy 1, policy_version 1411270 (0.0008) [2023-12-27 15:26:03,935][20381] Updated weights for policy 1, policy_version 1411280 (0.0007) [2023-12-27 15:26:04,447][20347] Updated weights for policy 0, policy_version 1412523 (0.0008) [2023-12-27 15:26:04,496][20347] Updated weights for policy 0, policy_version 1412533 (0.0008) [2023-12-27 15:26:04,543][20347] Updated weights for policy 0, policy_version 1412543 (0.0010) [2023-12-27 15:26:04,617][20381] Updated weights for policy 1, policy_version 1411290 (0.0006) [2023-12-27 15:26:04,671][20381] Updated weights for policy 1, policy_version 1411300 (0.0006) [2023-12-27 15:26:04,720][20381] Updated weights for policy 1, policy_version 1411310 (0.0005) [2023-12-27 15:26:05,301][20381] Updated weights for policy 1, policy_version 1411320 (0.0005) [2023-12-27 15:26:05,301][20347] Updated weights for policy 0, policy_version 1412553 (0.0007) [2023-12-27 15:26:05,359][20347] Updated weights for policy 0, policy_version 1412563 (0.0007) [2023-12-27 15:26:05,362][20381] Updated weights for policy 1, policy_version 1411330 (0.0005) [2023-12-27 15:26:05,410][20347] Updated weights for policy 0, policy_version 1412573 (0.0007) [2023-12-27 15:26:05,418][20381] Updated weights for policy 1, policy_version 1411340 (0.0008) [2023-12-27 15:26:05,461][20347] Updated weights for policy 0, policy_version 1412583 (0.0007) [2023-12-27 15:26:06,068][20381] Updated weights for policy 1, policy_version 1411350 (0.0008) [2023-12-27 15:26:06,135][20381] Updated weights for policy 1, policy_version 1411360 (0.0008) [2023-12-27 15:26:06,143][20347] Updated weights for policy 0, policy_version 1412593 (0.0008) [2023-12-27 15:26:06,155][20291] KL-divergence is very high: 424.1905 [2023-12-27 15:26:06,195][20381] Updated weights for policy 1, policy_version 1411370 (0.0008) [2023-12-27 15:26:06,202][20347] Updated weights for policy 0, policy_version 1412603 (0.0006) [2023-12-27 15:26:06,203][20291] KL-divergence is very high: 441.6530 [2023-12-27 15:26:06,253][20291] KL-divergence is very high: 230.7083 [2023-12-27 15:26:06,266][20347] Updated weights for policy 0, policy_version 1412613 (0.0006) [2023-12-27 15:26:06,857][20381] Updated weights for policy 1, policy_version 1411380 (0.0008) [2023-12-27 15:26:06,912][20381] Updated weights for policy 1, policy_version 1411390 (0.0007) [2023-12-27 15:26:06,918][20347] Updated weights for policy 0, policy_version 1412623 (0.0006) [2023-12-27 15:26:06,970][20381] Updated weights for policy 1, policy_version 1411400 (0.0006) [2023-12-27 15:26:06,973][20347] Updated weights for policy 0, policy_version 1412633 (0.0006) [2023-12-27 15:26:07,031][20347] Updated weights for policy 0, policy_version 1412643 (0.0008) [2023-12-27 15:26:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 723058688. Throughput: 0: 10217.6, 1: 10283.8. Samples: 723026084. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:07,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 15:26:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001411408_361349120.pth... [2023-12-27 15:26:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001412648_361709568.pth... [2023-12-27 15:26:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001410192_361037824.pth [2023-12-27 15:26:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001411464_361406464.pth [2023-12-27 15:26:07,663][20381] Updated weights for policy 1, policy_version 1411410 (0.0007) [2023-12-27 15:26:07,714][20381] Updated weights for policy 1, policy_version 1411420 (0.0007) [2023-12-27 15:26:07,757][20347] Updated weights for policy 0, policy_version 1412653 (0.0007) [2023-12-27 15:26:07,762][20381] Updated weights for policy 1, policy_version 1411430 (0.0007) [2023-12-27 15:26:07,803][20347] Updated weights for policy 0, policy_version 1412663 (0.0006) [2023-12-27 15:26:07,807][20381] Updated weights for policy 1, policy_version 1411440 (0.0005) [2023-12-27 15:26:07,852][20347] Updated weights for policy 0, policy_version 1412673 (0.0009) [2023-12-27 15:26:08,459][20381] Updated weights for policy 1, policy_version 1411450 (0.0006) [2023-12-27 15:26:08,516][20381] Updated weights for policy 1, policy_version 1411460 (0.0005) [2023-12-27 15:26:08,564][20347] Updated weights for policy 0, policy_version 1412683 (0.0009) [2023-12-27 15:26:08,571][20381] Updated weights for policy 1, policy_version 1411470 (0.0006) [2023-12-27 15:26:08,623][20347] Updated weights for policy 0, policy_version 1412693 (0.0007) [2023-12-27 15:26:08,677][20347] Updated weights for policy 0, policy_version 1412703 (0.0008) [2023-12-27 15:26:09,194][20381] Updated weights for policy 1, policy_version 1411480 (0.0008) [2023-12-27 15:26:09,255][20381] Updated weights for policy 1, policy_version 1411490 (0.0008) [2023-12-27 15:26:09,318][20381] Updated weights for policy 1, policy_version 1411500 (0.0008) [2023-12-27 15:26:09,353][20347] Updated weights for policy 0, policy_version 1412713 (0.0008) [2023-12-27 15:26:09,421][20347] Updated weights for policy 0, policy_version 1412723 (0.0008) [2023-12-27 15:26:09,479][20347] Updated weights for policy 0, policy_version 1412733 (0.0008) [2023-12-27 15:26:09,537][20347] Updated weights for policy 0, policy_version 1412743 (0.0008) [2023-12-27 15:26:10,049][20381] Updated weights for policy 1, policy_version 1411510 (0.0009) [2023-12-27 15:26:10,112][20381] Updated weights for policy 1, policy_version 1411520 (0.0009) [2023-12-27 15:26:10,172][20381] Updated weights for policy 1, policy_version 1411530 (0.0009) [2023-12-27 15:26:10,257][20347] Updated weights for policy 0, policy_version 1412753 (0.0007) [2023-12-27 15:26:10,315][20347] Updated weights for policy 0, policy_version 1412763 (0.0008) [2023-12-27 15:26:10,375][20347] Updated weights for policy 0, policy_version 1412773 (0.0009) [2023-12-27 15:26:10,770][20381] Updated weights for policy 1, policy_version 1411540 (0.0008) [2023-12-27 15:26:10,823][20381] Updated weights for policy 1, policy_version 1411550 (0.0007) [2023-12-27 15:26:10,876][20381] Updated weights for policy 1, policy_version 1411560 (0.0007) [2023-12-27 15:26:11,070][20347] Updated weights for policy 0, policy_version 1412783 (0.0008) [2023-12-27 15:26:11,128][20347] Updated weights for policy 0, policy_version 1412793 (0.0008) [2023-12-27 15:26:11,189][20347] Updated weights for policy 0, policy_version 1412803 (0.0008) [2023-12-27 15:26:11,636][20381] Updated weights for policy 1, policy_version 1411570 (0.0008) [2023-12-27 15:26:11,699][20381] Updated weights for policy 1, policy_version 1411580 (0.0007) [2023-12-27 15:26:11,768][20381] Updated weights for policy 1, policy_version 1411590 (0.0009) [2023-12-27 15:26:11,831][20381] Updated weights for policy 1, policy_version 1411600 (0.0007) [2023-12-27 15:26:11,924][20347] Updated weights for policy 0, policy_version 1412813 (0.0007) [2023-12-27 15:26:11,983][20347] Updated weights for policy 0, policy_version 1412823 (0.0007) [2023-12-27 15:26:12,043][20347] Updated weights for policy 0, policy_version 1412833 (0.0007) [2023-12-27 15:26:12,362][19170] Fps is (10 sec: 19663.4, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 723156992. Throughput: 0: 10190.8, 1: 10275.5. Samples: 723147572. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:12,363][19170] Avg episode reward: [(0, '931.770'), (1, '1000.000')] [2023-12-27 15:26:12,491][20381] Updated weights for policy 1, policy_version 1411610 (0.0009) [2023-12-27 15:26:12,548][20381] Updated weights for policy 1, policy_version 1411620 (0.0009) [2023-12-27 15:26:12,603][20381] Updated weights for policy 1, policy_version 1411630 (0.0007) [2023-12-27 15:26:12,816][20347] Updated weights for policy 0, policy_version 1412843 (0.0008) [2023-12-27 15:26:12,875][20347] Updated weights for policy 0, policy_version 1412853 (0.0007) [2023-12-27 15:26:12,929][20347] Updated weights for policy 0, policy_version 1412863 (0.0007) [2023-12-27 15:26:13,294][20381] Updated weights for policy 1, policy_version 1411640 (0.0008) [2023-12-27 15:26:13,351][20381] Updated weights for policy 1, policy_version 1411650 (0.0007) [2023-12-27 15:26:13,405][20292] KL-divergence is very high: 255.6768 [2023-12-27 15:26:13,412][20381] Updated weights for policy 1, policy_version 1411660 (0.0008) [2023-12-27 15:26:13,568][20347] Updated weights for policy 0, policy_version 1412873 (0.0007) [2023-12-27 15:26:13,627][20347] Updated weights for policy 0, policy_version 1412883 (0.0005) [2023-12-27 15:26:13,675][20347] Updated weights for policy 0, policy_version 1412893 (0.0005) [2023-12-27 15:26:13,731][20347] Updated weights for policy 0, policy_version 1412903 (0.0006) [2023-12-27 15:26:14,030][20381] Updated weights for policy 1, policy_version 1411670 (0.0009) [2023-12-27 15:26:14,082][20292] KL-divergence is very high: 102.9163 [2023-12-27 15:26:14,087][20381] Updated weights for policy 1, policy_version 1411680 (0.0005) [2023-12-27 15:26:14,134][20381] Updated weights for policy 1, policy_version 1411690 (0.0005) [2023-12-27 15:26:14,296][20347] Updated weights for policy 0, policy_version 1412913 (0.0008) [2023-12-27 15:26:14,375][20347] Updated weights for policy 0, policy_version 1412923 (0.0007) [2023-12-27 15:26:14,427][20347] Updated weights for policy 0, policy_version 1412933 (0.0008) [2023-12-27 15:26:14,761][20381] Updated weights for policy 1, policy_version 1411700 (0.0006) [2023-12-27 15:26:14,828][20381] Updated weights for policy 1, policy_version 1411710 (0.0007) [2023-12-27 15:26:14,892][20381] Updated weights for policy 1, policy_version 1411720 (0.0008) [2023-12-27 15:26:15,039][20347] Updated weights for policy 0, policy_version 1412943 (0.0006) [2023-12-27 15:26:15,101][20347] Updated weights for policy 0, policy_version 1412953 (0.0006) [2023-12-27 15:26:15,156][20347] Updated weights for policy 0, policy_version 1412963 (0.0007) [2023-12-27 15:26:15,629][20381] Updated weights for policy 1, policy_version 1411730 (0.0008) [2023-12-27 15:26:15,685][20381] Updated weights for policy 1, policy_version 1411740 (0.0008) [2023-12-27 15:26:15,705][20347] Updated weights for policy 0, policy_version 1412973 (0.0007) [2023-12-27 15:26:15,738][20381] Updated weights for policy 1, policy_version 1411750 (0.0007) [2023-12-27 15:26:15,760][20347] Updated weights for policy 0, policy_version 1412983 (0.0005) [2023-12-27 15:26:15,784][20381] Updated weights for policy 1, policy_version 1411760 (0.0006) [2023-12-27 15:26:15,814][20347] Updated weights for policy 0, policy_version 1412993 (0.0005) [2023-12-27 15:26:16,409][20347] Updated weights for policy 0, policy_version 1413003 (0.0006) [2023-12-27 15:26:16,472][20347] Updated weights for policy 0, policy_version 1413013 (0.0008) [2023-12-27 15:26:16,482][20381] Updated weights for policy 1, policy_version 1411770 (0.0007) [2023-12-27 15:26:16,524][20347] Updated weights for policy 0, policy_version 1413023 (0.0006) [2023-12-27 15:26:16,534][20381] Updated weights for policy 1, policy_version 1411780 (0.0006) [2023-12-27 15:26:16,594][20381] Updated weights for policy 1, policy_version 1411790 (0.0008) [2023-12-27 15:26:17,188][20347] Updated weights for policy 0, policy_version 1413033 (0.0006) [2023-12-27 15:26:17,223][20381] Updated weights for policy 1, policy_version 1411800 (0.0006) [2023-12-27 15:26:17,245][20347] Updated weights for policy 0, policy_version 1413043 (0.0006) [2023-12-27 15:26:17,269][20381] Updated weights for policy 1, policy_version 1411810 (0.0006) [2023-12-27 15:26:17,294][20347] Updated weights for policy 0, policy_version 1413053 (0.0005) [2023-12-27 15:26:17,315][20381] Updated weights for policy 1, policy_version 1411820 (0.0006) [2023-12-27 15:26:17,344][20347] Updated weights for policy 0, policy_version 1413063 (0.0006) [2023-12-27 15:26:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 723271680. Throughput: 0: 10222.6, 1: 10291.2. Samples: 723273708. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:17,363][19170] Avg episode reward: [(0, '941.430'), (1, '980.570')] [2023-12-27 15:26:17,984][20381] Updated weights for policy 1, policy_version 1411830 (0.0007) [2023-12-27 15:26:18,038][20381] Updated weights for policy 1, policy_version 1411840 (0.0007) [2023-12-27 15:26:18,040][20347] Updated weights for policy 0, policy_version 1413073 (0.0005) [2023-12-27 15:26:18,087][20381] Updated weights for policy 1, policy_version 1411850 (0.0005) [2023-12-27 15:26:18,096][20347] Updated weights for policy 0, policy_version 1413083 (0.0006) [2023-12-27 15:26:18,151][20347] Updated weights for policy 0, policy_version 1413093 (0.0007) [2023-12-27 15:26:18,808][20381] Updated weights for policy 1, policy_version 1411860 (0.0006) [2023-12-27 15:26:18,811][20347] Updated weights for policy 0, policy_version 1413103 (0.0006) [2023-12-27 15:26:18,866][20381] Updated weights for policy 1, policy_version 1411870 (0.0007) [2023-12-27 15:26:18,875][20347] Updated weights for policy 0, policy_version 1413113 (0.0006) [2023-12-27 15:26:18,926][20381] Updated weights for policy 1, policy_version 1411880 (0.0007) [2023-12-27 15:26:18,932][20347] Updated weights for policy 0, policy_version 1413123 (0.0006) [2023-12-27 15:26:19,633][20381] Updated weights for policy 1, policy_version 1411890 (0.0008) [2023-12-27 15:26:19,648][20347] Updated weights for policy 0, policy_version 1413133 (0.0008) [2023-12-27 15:26:19,692][20381] Updated weights for policy 1, policy_version 1411900 (0.0011) [2023-12-27 15:26:19,707][20347] Updated weights for policy 0, policy_version 1413143 (0.0011) [2023-12-27 15:26:19,752][20381] Updated weights for policy 1, policy_version 1411910 (0.0011) [2023-12-27 15:26:19,767][20347] Updated weights for policy 0, policy_version 1413153 (0.0011) [2023-12-27 15:26:19,813][20381] Updated weights for policy 1, policy_version 1411920 (0.0011) [2023-12-27 15:26:20,492][20381] Updated weights for policy 1, policy_version 1411930 (0.0011) [2023-12-27 15:26:20,496][20347] Updated weights for policy 0, policy_version 1413163 (0.0010) [2023-12-27 15:26:20,551][20381] Updated weights for policy 1, policy_version 1411940 (0.0011) [2023-12-27 15:26:20,552][20347] Updated weights for policy 0, policy_version 1413173 (0.0006) [2023-12-27 15:26:20,616][20381] Updated weights for policy 1, policy_version 1411950 (0.0011) [2023-12-27 15:26:20,618][20347] Updated weights for policy 0, policy_version 1413183 (0.0006) [2023-12-27 15:26:21,252][20381] Updated weights for policy 1, policy_version 1411960 (0.0009) [2023-12-27 15:26:21,315][20381] Updated weights for policy 1, policy_version 1411970 (0.0008) [2023-12-27 15:26:21,387][20381] Updated weights for policy 1, policy_version 1411980 (0.0009) [2023-12-27 15:26:21,391][20347] Updated weights for policy 0, policy_version 1413193 (0.0006) [2023-12-27 15:26:21,450][20347] Updated weights for policy 0, policy_version 1413203 (0.0007) [2023-12-27 15:26:21,507][20347] Updated weights for policy 0, policy_version 1413213 (0.0005) [2023-12-27 15:26:21,574][20347] Updated weights for policy 0, policy_version 1413223 (0.0005) [2023-12-27 15:26:22,084][20381] Updated weights for policy 1, policy_version 1411990 (0.0007) [2023-12-27 15:26:22,142][20381] Updated weights for policy 1, policy_version 1412000 (0.0005) [2023-12-27 15:26:22,199][20381] Updated weights for policy 1, policy_version 1412010 (0.0005) [2023-12-27 15:26:22,304][20347] Updated weights for policy 0, policy_version 1413233 (0.0008) [2023-12-27 15:26:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 723361792. Throughput: 0: 10154.0, 1: 10265.7. Samples: 723333752. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:22,363][19170] Avg episode reward: [(0, '912.130'), (1, '990.310')] [2023-12-27 15:26:22,365][20347] Updated weights for policy 0, policy_version 1413243 (0.0008) [2023-12-27 15:26:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001412016_361504768.pth... [2023-12-27 15:26:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001410800_361193472.pth [2023-12-27 15:26:22,425][20347] Updated weights for policy 0, policy_version 1413253 (0.0008) [2023-12-27 15:26:22,440][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001413256_361865216.pth... [2023-12-27 15:26:22,444][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001412072_361562112.pth [2023-12-27 15:26:22,852][20381] Updated weights for policy 1, policy_version 1412020 (0.0006) [2023-12-27 15:26:22,901][20381] Updated weights for policy 1, policy_version 1412030 (0.0007) [2023-12-27 15:26:22,950][20381] Updated weights for policy 1, policy_version 1412040 (0.0008) [2023-12-27 15:26:23,084][20347] Updated weights for policy 0, policy_version 1413263 (0.0006) [2023-12-27 15:26:23,135][20347] Updated weights for policy 0, policy_version 1413273 (0.0005) [2023-12-27 15:26:23,196][20347] Updated weights for policy 0, policy_version 1413283 (0.0007) [2023-12-27 15:26:23,666][20381] Updated weights for policy 1, policy_version 1412050 (0.0008) [2023-12-27 15:26:23,717][20381] Updated weights for policy 1, policy_version 1412060 (0.0005) [2023-12-27 15:26:23,775][20347] Updated weights for policy 0, policy_version 1413293 (0.0008) [2023-12-27 15:26:23,778][20381] Updated weights for policy 1, policy_version 1412070 (0.0005) [2023-12-27 15:26:23,834][20347] Updated weights for policy 0, policy_version 1413303 (0.0007) [2023-12-27 15:26:23,836][20381] Updated weights for policy 1, policy_version 1412080 (0.0005) [2023-12-27 15:26:23,896][20347] Updated weights for policy 0, policy_version 1413313 (0.0006) [2023-12-27 15:26:24,436][20381] Updated weights for policy 1, policy_version 1412090 (0.0006) [2023-12-27 15:26:24,490][20381] Updated weights for policy 1, policy_version 1412100 (0.0006) [2023-12-27 15:26:24,507][20347] Updated weights for policy 0, policy_version 1413323 (0.0006) [2023-12-27 15:26:24,545][20381] Updated weights for policy 1, policy_version 1412110 (0.0010) [2023-12-27 15:26:24,561][20347] Updated weights for policy 0, policy_version 1413333 (0.0010) [2023-12-27 15:26:24,610][20347] Updated weights for policy 0, policy_version 1413343 (0.0010) [2023-12-27 15:26:25,201][20381] Updated weights for policy 1, policy_version 1412120 (0.0008) [2023-12-27 15:26:25,246][20347] Updated weights for policy 0, policy_version 1413353 (0.0010) [2023-12-27 15:26:25,253][20381] Updated weights for policy 1, policy_version 1412130 (0.0008) [2023-12-27 15:26:25,301][20347] Updated weights for policy 0, policy_version 1413363 (0.0005) [2023-12-27 15:26:25,304][20381] Updated weights for policy 1, policy_version 1412140 (0.0007) [2023-12-27 15:26:25,358][20347] Updated weights for policy 0, policy_version 1413373 (0.0005) [2023-12-27 15:26:25,406][20347] Updated weights for policy 0, policy_version 1413383 (0.0005) [2023-12-27 15:26:25,953][20347] Updated weights for policy 0, policy_version 1413393 (0.0005) [2023-12-27 15:26:26,002][20347] Updated weights for policy 0, policy_version 1413403 (0.0006) [2023-12-27 15:26:26,021][20381] Updated weights for policy 1, policy_version 1412150 (0.0010) [2023-12-27 15:26:26,055][20347] Updated weights for policy 0, policy_version 1413413 (0.0005) [2023-12-27 15:26:26,077][20381] Updated weights for policy 1, policy_version 1412160 (0.0010) [2023-12-27 15:26:26,132][20381] Updated weights for policy 1, policy_version 1412170 (0.0010) [2023-12-27 15:26:26,611][20347] Updated weights for policy 0, policy_version 1413423 (0.0006) [2023-12-27 15:26:26,666][20347] Updated weights for policy 0, policy_version 1413433 (0.0006) [2023-12-27 15:26:26,686][20381] Updated weights for policy 1, policy_version 1412180 (0.0008) [2023-12-27 15:26:26,724][20347] Updated weights for policy 0, policy_version 1413443 (0.0006) [2023-12-27 15:26:26,739][20381] Updated weights for policy 1, policy_version 1412190 (0.0005) [2023-12-27 15:26:26,794][20381] Updated weights for policy 1, policy_version 1412200 (0.0005) [2023-12-27 15:26:27,251][20347] Updated weights for policy 0, policy_version 1413453 (0.0006) [2023-12-27 15:26:27,298][20347] Updated weights for policy 0, policy_version 1413463 (0.0008) [2023-12-27 15:26:27,362][20347] Updated weights for policy 0, policy_version 1413473 (0.0008) [2023-12-27 15:26:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 723468288. Throughput: 0: 10206.3, 1: 10323.9. Samples: 723459024. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:27,363][19170] Avg episode reward: [(0, '902.070'), (1, '1000.000')] [2023-12-27 15:26:27,378][20381] Updated weights for policy 1, policy_version 1412210 (0.0005) [2023-12-27 15:26:27,439][20381] Updated weights for policy 1, policy_version 1412220 (0.0005) [2023-12-27 15:26:27,499][20381] Updated weights for policy 1, policy_version 1412230 (0.0005) [2023-12-27 15:26:27,547][20381] Updated weights for policy 1, policy_version 1412240 (0.0008) [2023-12-27 15:26:27,932][20347] Updated weights for policy 0, policy_version 1413483 (0.0006) [2023-12-27 15:26:27,985][20347] Updated weights for policy 0, policy_version 1413493 (0.0008) [2023-12-27 15:26:28,033][20347] Updated weights for policy 0, policy_version 1413503 (0.0008) [2023-12-27 15:26:28,165][20381] Updated weights for policy 1, policy_version 1412250 (0.0006) [2023-12-27 15:26:28,219][20381] Updated weights for policy 1, policy_version 1412260 (0.0007) [2023-12-27 15:26:28,267][20381] Updated weights for policy 1, policy_version 1412270 (0.0010) [2023-12-27 15:26:28,716][20347] Updated weights for policy 0, policy_version 1413513 (0.0008) [2023-12-27 15:26:28,779][20347] Updated weights for policy 0, policy_version 1413523 (0.0006) [2023-12-27 15:26:28,833][20347] Updated weights for policy 0, policy_version 1413533 (0.0007) [2023-12-27 15:26:28,890][20347] Updated weights for policy 0, policy_version 1413543 (0.0008) [2023-12-27 15:26:28,955][20381] Updated weights for policy 1, policy_version 1412280 (0.0008) [2023-12-27 15:26:29,009][20381] Updated weights for policy 1, policy_version 1412290 (0.0008) [2023-12-27 15:26:29,064][20381] Updated weights for policy 1, policy_version 1412300 (0.0007) [2023-12-27 15:26:29,567][20347] Updated weights for policy 0, policy_version 1413553 (0.0006) [2023-12-27 15:26:29,627][20347] Updated weights for policy 0, policy_version 1413563 (0.0006) [2023-12-27 15:26:29,686][20347] Updated weights for policy 0, policy_version 1413573 (0.0007) [2023-12-27 15:26:29,783][20381] Updated weights for policy 1, policy_version 1412310 (0.0007) [2023-12-27 15:26:29,840][20381] Updated weights for policy 1, policy_version 1412320 (0.0007) [2023-12-27 15:26:29,897][20381] Updated weights for policy 1, policy_version 1412330 (0.0007) [2023-12-27 15:26:30,339][20347] Updated weights for policy 0, policy_version 1413583 (0.0008) [2023-12-27 15:26:30,397][20347] Updated weights for policy 0, policy_version 1413593 (0.0007) [2023-12-27 15:26:30,451][20347] Updated weights for policy 0, policy_version 1413603 (0.0007) [2023-12-27 15:26:30,623][20381] Updated weights for policy 1, policy_version 1412340 (0.0009) [2023-12-27 15:26:30,676][20381] Updated weights for policy 1, policy_version 1412350 (0.0010) [2023-12-27 15:26:30,730][20381] Updated weights for policy 1, policy_version 1412360 (0.0010) [2023-12-27 15:26:31,170][20347] Updated weights for policy 0, policy_version 1413613 (0.0008) [2023-12-27 15:26:31,229][20347] Updated weights for policy 0, policy_version 1413623 (0.0006) [2023-12-27 15:26:31,285][20347] Updated weights for policy 0, policy_version 1413633 (0.0007) [2023-12-27 15:26:31,461][20381] Updated weights for policy 1, policy_version 1412370 (0.0010) [2023-12-27 15:26:31,530][20381] Updated weights for policy 1, policy_version 1412380 (0.0008) [2023-12-27 15:26:31,592][20381] Updated weights for policy 1, policy_version 1412390 (0.0007) [2023-12-27 15:26:31,659][20381] Updated weights for policy 1, policy_version 1412400 (0.0010) [2023-12-27 15:26:31,952][20347] Updated weights for policy 0, policy_version 1413643 (0.0007) [2023-12-27 15:26:32,011][20347] Updated weights for policy 0, policy_version 1413653 (0.0008) [2023-12-27 15:26:32,069][20347] Updated weights for policy 0, policy_version 1413663 (0.0007) [2023-12-27 15:26:32,263][20381] Updated weights for policy 1, policy_version 1412410 (0.0010) [2023-12-27 15:26:32,329][20381] Updated weights for policy 1, policy_version 1412420 (0.0007) [2023-12-27 15:26:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 723574784. Throughput: 0: 10300.8, 1: 10340.1. Samples: 723587676. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:32,363][19170] Avg episode reward: [(0, '921.730'), (1, '990.270')] [2023-12-27 15:26:32,398][20381] Updated weights for policy 1, policy_version 1412430 (0.0008) [2023-12-27 15:26:32,707][20347] Updated weights for policy 0, policy_version 1413673 (0.0007) [2023-12-27 15:26:32,770][20347] Updated weights for policy 0, policy_version 1413683 (0.0006) [2023-12-27 15:26:32,826][20347] Updated weights for policy 0, policy_version 1413693 (0.0006) [2023-12-27 15:26:32,881][20347] Updated weights for policy 0, policy_version 1413703 (0.0006) [2023-12-27 15:26:33,138][20381] Updated weights for policy 1, policy_version 1412440 (0.0010) [2023-12-27 15:26:33,195][20381] Updated weights for policy 1, policy_version 1412450 (0.0010) [2023-12-27 15:26:33,255][20381] Updated weights for policy 1, policy_version 1412460 (0.0010) [2023-12-27 15:26:33,430][20347] Updated weights for policy 0, policy_version 1413713 (0.0005) [2023-12-27 15:26:33,488][20347] Updated weights for policy 0, policy_version 1413723 (0.0005) [2023-12-27 15:26:33,548][20347] Updated weights for policy 0, policy_version 1413733 (0.0005) [2023-12-27 15:26:33,837][20381] Updated weights for policy 1, policy_version 1412470 (0.0008) [2023-12-27 15:26:33,899][20381] Updated weights for policy 1, policy_version 1412480 (0.0011) [2023-12-27 15:26:33,956][20381] Updated weights for policy 1, policy_version 1412490 (0.0010) [2023-12-27 15:26:34,089][20347] Updated weights for policy 0, policy_version 1413743 (0.0006) [2023-12-27 15:26:34,145][20347] Updated weights for policy 0, policy_version 1413753 (0.0007) [2023-12-27 15:26:34,208][20347] Updated weights for policy 0, policy_version 1413763 (0.0008) [2023-12-27 15:26:34,598][20381] Updated weights for policy 1, policy_version 1412500 (0.0009) [2023-12-27 15:26:34,658][20381] Updated weights for policy 1, policy_version 1412510 (0.0008) [2023-12-27 15:26:34,712][20381] Updated weights for policy 1, policy_version 1412520 (0.0008) [2023-12-27 15:26:34,896][20347] Updated weights for policy 0, policy_version 1413773 (0.0007) [2023-12-27 15:26:34,951][20347] Updated weights for policy 0, policy_version 1413783 (0.0008) [2023-12-27 15:26:35,006][20347] Updated weights for policy 0, policy_version 1413793 (0.0008) [2023-12-27 15:26:35,395][20381] Updated weights for policy 1, policy_version 1412530 (0.0008) [2023-12-27 15:26:35,444][20381] Updated weights for policy 1, policy_version 1412540 (0.0007) [2023-12-27 15:26:35,489][20381] Updated weights for policy 1, policy_version 1412550 (0.0007) [2023-12-27 15:26:35,538][20381] Updated weights for policy 1, policy_version 1412560 (0.0006) [2023-12-27 15:26:35,650][20347] Updated weights for policy 0, policy_version 1413803 (0.0009) [2023-12-27 15:26:35,698][20347] Updated weights for policy 0, policy_version 1413813 (0.0010) [2023-12-27 15:26:35,743][20347] Updated weights for policy 0, policy_version 1413823 (0.0010) [2023-12-27 15:26:36,202][20381] Updated weights for policy 1, policy_version 1412570 (0.0008) [2023-12-27 15:26:36,263][20381] Updated weights for policy 1, policy_version 1412580 (0.0007) [2023-12-27 15:26:36,326][20381] Updated weights for policy 1, policy_version 1412590 (0.0007) [2023-12-27 15:26:36,465][20347] Updated weights for policy 0, policy_version 1413833 (0.0010) [2023-12-27 15:26:36,520][20347] Updated weights for policy 0, policy_version 1413843 (0.0007) [2023-12-27 15:26:36,576][20347] Updated weights for policy 0, policy_version 1413853 (0.0009) [2023-12-27 15:26:36,629][20347] Updated weights for policy 0, policy_version 1413863 (0.0008) [2023-12-27 15:26:36,901][20381] Updated weights for policy 1, policy_version 1412600 (0.0006) [2023-12-27 15:26:36,950][20381] Updated weights for policy 1, policy_version 1412610 (0.0007) [2023-12-27 15:26:37,001][20381] Updated weights for policy 1, policy_version 1412620 (0.0007) [2023-12-27 15:26:37,277][20347] Updated weights for policy 0, policy_version 1413873 (0.0010) [2023-12-27 15:26:37,329][20347] Updated weights for policy 0, policy_version 1413883 (0.0010) [2023-12-27 15:26:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 723681280. Throughput: 0: 10330.7, 1: 10344.6. Samples: 723652060. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:37,363][19170] Avg episode reward: [(0, '921.740'), (1, '980.530')] [2023-12-27 15:26:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001412624_361660416.pth... [2023-12-27 15:26:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001411408_361349120.pth [2023-12-27 15:26:37,384][20347] Updated weights for policy 0, policy_version 1413893 (0.0010) [2023-12-27 15:26:37,401][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001413896_362029056.pth... [2023-12-27 15:26:37,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001412648_361709568.pth [2023-12-27 15:26:37,689][20381] Updated weights for policy 1, policy_version 1412630 (0.0008) [2023-12-27 15:26:37,746][20381] Updated weights for policy 1, policy_version 1412640 (0.0008) [2023-12-27 15:26:37,804][20381] Updated weights for policy 1, policy_version 1412650 (0.0007) [2023-12-27 15:26:38,073][20347] Updated weights for policy 0, policy_version 1413903 (0.0010) [2023-12-27 15:26:38,117][20347] Updated weights for policy 0, policy_version 1413913 (0.0007) [2023-12-27 15:26:38,167][20347] Updated weights for policy 0, policy_version 1413923 (0.0007) [2023-12-27 15:26:38,465][20381] Updated weights for policy 1, policy_version 1412660 (0.0008) [2023-12-27 15:26:38,526][20381] Updated weights for policy 1, policy_version 1412670 (0.0007) [2023-12-27 15:26:38,597][20381] Updated weights for policy 1, policy_version 1412680 (0.0005) [2023-12-27 15:26:38,881][20347] Updated weights for policy 0, policy_version 1413933 (0.0011) [2023-12-27 15:26:38,935][20347] Updated weights for policy 0, policy_version 1413943 (0.0011) [2023-12-27 15:26:38,994][20347] Updated weights for policy 0, policy_version 1413953 (0.0010) [2023-12-27 15:26:39,228][20381] Updated weights for policy 1, policy_version 1412690 (0.0007) [2023-12-27 15:26:39,290][20381] Updated weights for policy 1, policy_version 1412700 (0.0008) [2023-12-27 15:26:39,355][20381] Updated weights for policy 1, policy_version 1412710 (0.0009) [2023-12-27 15:26:39,422][20381] Updated weights for policy 1, policy_version 1412720 (0.0007) [2023-12-27 15:26:39,811][20347] Updated weights for policy 0, policy_version 1413963 (0.0010) [2023-12-27 15:26:39,873][20347] Updated weights for policy 0, policy_version 1413973 (0.0008) [2023-12-27 15:26:39,933][20347] Updated weights for policy 0, policy_version 1413983 (0.0009) [2023-12-27 15:26:40,192][20381] Updated weights for policy 1, policy_version 1412730 (0.0011) [2023-12-27 15:26:40,254][20381] Updated weights for policy 1, policy_version 1412740 (0.0010) [2023-12-27 15:26:40,318][20381] Updated weights for policy 1, policy_version 1412750 (0.0008) [2023-12-27 15:26:40,664][20347] Updated weights for policy 0, policy_version 1413993 (0.0007) [2023-12-27 15:26:40,714][20347] Updated weights for policy 0, policy_version 1414003 (0.0008) [2023-12-27 15:26:40,765][20347] Updated weights for policy 0, policy_version 1414013 (0.0007) [2023-12-27 15:26:40,825][20347] Updated weights for policy 0, policy_version 1414023 (0.0006) [2023-12-27 15:26:40,987][20381] Updated weights for policy 1, policy_version 1412760 (0.0010) [2023-12-27 15:26:41,059][20381] Updated weights for policy 1, policy_version 1412770 (0.0011) [2023-12-27 15:26:41,112][20381] Updated weights for policy 1, policy_version 1412780 (0.0011) [2023-12-27 15:26:41,579][20347] Updated weights for policy 0, policy_version 1414033 (0.0009) [2023-12-27 15:26:41,644][20347] Updated weights for policy 0, policy_version 1414043 (0.0009) [2023-12-27 15:26:41,700][20347] Updated weights for policy 0, policy_version 1414053 (0.0011) [2023-12-27 15:26:41,815][20381] Updated weights for policy 1, policy_version 1412790 (0.0009) [2023-12-27 15:26:41,878][20381] Updated weights for policy 1, policy_version 1412800 (0.0008) [2023-12-27 15:26:41,935][20381] Updated weights for policy 1, policy_version 1412810 (0.0008) [2023-12-27 15:26:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 723779584. Throughput: 0: 10280.2, 1: 10329.6. Samples: 723772252. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:42,363][19170] Avg episode reward: [(0, '941.310'), (1, '990.260')] [2023-12-27 15:26:42,459][20347] Updated weights for policy 0, policy_version 1414063 (0.0008) [2023-12-27 15:26:42,513][20347] Updated weights for policy 0, policy_version 1414073 (0.0008) [2023-12-27 15:26:42,572][20347] Updated weights for policy 0, policy_version 1414083 (0.0007) [2023-12-27 15:26:42,658][20381] Updated weights for policy 1, policy_version 1412820 (0.0008) [2023-12-27 15:26:42,714][20381] Updated weights for policy 1, policy_version 1412830 (0.0008) [2023-12-27 15:26:42,781][20381] Updated weights for policy 1, policy_version 1412840 (0.0007) [2023-12-27 15:26:43,194][20347] Updated weights for policy 0, policy_version 1414093 (0.0006) [2023-12-27 15:26:43,242][20347] Updated weights for policy 0, policy_version 1414103 (0.0005) [2023-12-27 15:26:43,291][20347] Updated weights for policy 0, policy_version 1414113 (0.0005) [2023-12-27 15:26:43,456][20381] Updated weights for policy 1, policy_version 1412850 (0.0007) [2023-12-27 15:26:43,519][20381] Updated weights for policy 1, policy_version 1412860 (0.0005) [2023-12-27 15:26:43,591][20381] Updated weights for policy 1, policy_version 1412870 (0.0005) [2023-12-27 15:26:43,648][20381] Updated weights for policy 1, policy_version 1412880 (0.0005) [2023-12-27 15:26:43,891][20347] Updated weights for policy 0, policy_version 1414123 (0.0006) [2023-12-27 15:26:43,942][20347] Updated weights for policy 0, policy_version 1414133 (0.0007) [2023-12-27 15:26:43,993][20347] Updated weights for policy 0, policy_version 1414143 (0.0008) [2023-12-27 15:26:44,173][20381] Updated weights for policy 1, policy_version 1412890 (0.0005) [2023-12-27 15:26:44,234][20381] Updated weights for policy 1, policy_version 1412900 (0.0006) [2023-12-27 15:26:44,294][20381] Updated weights for policy 1, policy_version 1412910 (0.0007) [2023-12-27 15:26:44,707][20347] Updated weights for policy 0, policy_version 1414153 (0.0009) [2023-12-27 15:26:44,774][20347] Updated weights for policy 0, policy_version 1414163 (0.0006) [2023-12-27 15:26:44,832][20347] Updated weights for policy 0, policy_version 1414173 (0.0007) [2023-12-27 15:26:44,890][20347] Updated weights for policy 0, policy_version 1414183 (0.0006) [2023-12-27 15:26:44,907][20381] Updated weights for policy 1, policy_version 1412920 (0.0009) [2023-12-27 15:26:44,972][20381] Updated weights for policy 1, policy_version 1412930 (0.0006) [2023-12-27 15:26:45,032][20381] Updated weights for policy 1, policy_version 1412940 (0.0006) [2023-12-27 15:26:45,530][20347] Updated weights for policy 0, policy_version 1414193 (0.0007) [2023-12-27 15:26:45,579][20347] Updated weights for policy 0, policy_version 1414203 (0.0006) [2023-12-27 15:26:45,624][20347] Updated weights for policy 0, policy_version 1414213 (0.0007) [2023-12-27 15:26:45,645][20381] Updated weights for policy 1, policy_version 1412950 (0.0007) [2023-12-27 15:26:45,700][20381] Updated weights for policy 1, policy_version 1412960 (0.0007) [2023-12-27 15:26:45,748][20381] Updated weights for policy 1, policy_version 1412970 (0.0007) [2023-12-27 15:26:46,343][20347] Updated weights for policy 0, policy_version 1414223 (0.0008) [2023-12-27 15:26:46,395][20381] Updated weights for policy 1, policy_version 1412980 (0.0007) [2023-12-27 15:26:46,397][20347] Updated weights for policy 0, policy_version 1414233 (0.0008) [2023-12-27 15:26:46,451][20381] Updated weights for policy 1, policy_version 1412990 (0.0007) [2023-12-27 15:26:46,453][20347] Updated weights for policy 0, policy_version 1414243 (0.0005) [2023-12-27 15:26:46,509][20381] Updated weights for policy 1, policy_version 1413000 (0.0007) [2023-12-27 15:26:47,103][20347] Updated weights for policy 0, policy_version 1414253 (0.0008) [2023-12-27 15:26:47,129][20381] Updated weights for policy 1, policy_version 1413010 (0.0007) [2023-12-27 15:26:47,156][20347] Updated weights for policy 0, policy_version 1414263 (0.0006) [2023-12-27 15:26:47,180][20381] Updated weights for policy 1, policy_version 1413020 (0.0005) [2023-12-27 15:26:47,208][20347] Updated weights for policy 0, policy_version 1414273 (0.0006) [2023-12-27 15:26:47,232][20381] Updated weights for policy 1, policy_version 1413030 (0.0005) [2023-12-27 15:26:47,292][20381] Updated weights for policy 1, policy_version 1413040 (0.0007) [2023-12-27 15:26:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 723894272. Throughput: 0: 10348.9, 1: 10381.8. Samples: 723898060. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:47,363][19170] Avg episode reward: [(0, '960.770'), (1, '1000.000')] [2023-12-27 15:26:47,906][20381] Updated weights for policy 1, policy_version 1413050 (0.0007) [2023-12-27 15:26:47,951][20381] Updated weights for policy 1, policy_version 1413060 (0.0007) [2023-12-27 15:26:47,955][20347] Updated weights for policy 0, policy_version 1414283 (0.0006) [2023-12-27 15:26:48,007][20381] Updated weights for policy 1, policy_version 1413070 (0.0007) [2023-12-27 15:26:48,013][20347] Updated weights for policy 0, policy_version 1414293 (0.0006) [2023-12-27 15:26:48,071][20347] Updated weights for policy 0, policy_version 1414303 (0.0007) [2023-12-27 15:26:48,680][20347] Updated weights for policy 0, policy_version 1414313 (0.0006) [2023-12-27 15:26:48,725][20381] Updated weights for policy 1, policy_version 1413080 (0.0007) [2023-12-27 15:26:48,738][20347] Updated weights for policy 0, policy_version 1414323 (0.0007) [2023-12-27 15:26:48,785][20381] Updated weights for policy 1, policy_version 1413090 (0.0006) [2023-12-27 15:26:48,798][20347] Updated weights for policy 0, policy_version 1414333 (0.0007) [2023-12-27 15:26:48,841][20381] Updated weights for policy 1, policy_version 1413100 (0.0007) [2023-12-27 15:26:48,858][20347] Updated weights for policy 0, policy_version 1414343 (0.0006) [2023-12-27 15:26:49,485][20347] Updated weights for policy 0, policy_version 1414353 (0.0008) [2023-12-27 15:26:49,546][20347] Updated weights for policy 0, policy_version 1414363 (0.0008) [2023-12-27 15:26:49,551][20381] Updated weights for policy 1, policy_version 1413110 (0.0006) [2023-12-27 15:26:49,603][20347] Updated weights for policy 0, policy_version 1414373 (0.0008) [2023-12-27 15:26:49,611][20381] Updated weights for policy 1, policy_version 1413120 (0.0007) [2023-12-27 15:26:49,669][20381] Updated weights for policy 1, policy_version 1413130 (0.0007) [2023-12-27 15:26:50,368][20347] Updated weights for policy 0, policy_version 1414383 (0.0009) [2023-12-27 15:26:50,407][20381] Updated weights for policy 1, policy_version 1413140 (0.0006) [2023-12-27 15:26:50,423][20347] Updated weights for policy 0, policy_version 1414393 (0.0006) [2023-12-27 15:26:50,469][20381] Updated weights for policy 1, policy_version 1413150 (0.0009) [2023-12-27 15:26:50,470][20347] Updated weights for policy 0, policy_version 1414403 (0.0005) [2023-12-27 15:26:50,534][20381] Updated weights for policy 1, policy_version 1413160 (0.0008) [2023-12-27 15:26:51,178][20347] Updated weights for policy 0, policy_version 1414413 (0.0006) [2023-12-27 15:26:51,182][20381] Updated weights for policy 1, policy_version 1413170 (0.0008) [2023-12-27 15:26:51,239][20381] Updated weights for policy 1, policy_version 1413180 (0.0008) [2023-12-27 15:26:51,240][20347] Updated weights for policy 0, policy_version 1414423 (0.0007) [2023-12-27 15:26:51,292][20347] Updated weights for policy 0, policy_version 1414433 (0.0009) [2023-12-27 15:26:51,295][20381] Updated weights for policy 1, policy_version 1413190 (0.0009) [2023-12-27 15:26:51,359][20381] Updated weights for policy 1, policy_version 1413200 (0.0009) [2023-12-27 15:26:52,086][20347] Updated weights for policy 0, policy_version 1414443 (0.0009) [2023-12-27 15:26:52,128][20381] Updated weights for policy 1, policy_version 1413210 (0.0006) [2023-12-27 15:26:52,143][20347] Updated weights for policy 0, policy_version 1414453 (0.0007) [2023-12-27 15:26:52,183][20381] Updated weights for policy 1, policy_version 1413220 (0.0007) [2023-12-27 15:26:52,207][20347] Updated weights for policy 0, policy_version 1414463 (0.0006) [2023-12-27 15:26:52,229][20381] Updated weights for policy 1, policy_version 1413230 (0.0007) [2023-12-27 15:26:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 723992576. Throughput: 0: 10366.9, 1: 10350.4. Samples: 723958364. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:52,363][19170] Avg episode reward: [(0, '951.030'), (1, '1000.000')] [2023-12-27 15:26:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001413232_361816064.pth... [2023-12-27 15:26:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001414472_362176512.pth... [2023-12-27 15:26:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001413256_361865216.pth [2023-12-27 15:26:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001412016_361504768.pth [2023-12-27 15:26:52,921][20347] Updated weights for policy 0, policy_version 1414473 (0.0007) [2023-12-27 15:26:52,952][20381] Updated weights for policy 1, policy_version 1413240 (0.0008) [2023-12-27 15:26:52,973][20347] Updated weights for policy 0, policy_version 1414483 (0.0009) [2023-12-27 15:26:53,008][20381] Updated weights for policy 1, policy_version 1413250 (0.0008) [2023-12-27 15:26:53,031][20347] Updated weights for policy 0, policy_version 1414493 (0.0010) [2023-12-27 15:26:53,065][20381] Updated weights for policy 1, policy_version 1413260 (0.0006) [2023-12-27 15:26:53,080][20347] Updated weights for policy 0, policy_version 1414503 (0.0010) [2023-12-27 15:26:53,656][20381] Updated weights for policy 1, policy_version 1413270 (0.0007) [2023-12-27 15:26:53,683][20347] Updated weights for policy 0, policy_version 1414513 (0.0010) [2023-12-27 15:26:53,724][20381] Updated weights for policy 1, policy_version 1413280 (0.0005) [2023-12-27 15:26:53,743][20347] Updated weights for policy 0, policy_version 1414523 (0.0010) [2023-12-27 15:26:53,781][20381] Updated weights for policy 1, policy_version 1413290 (0.0005) [2023-12-27 15:26:53,797][20347] Updated weights for policy 0, policy_version 1414533 (0.0009) [2023-12-27 15:26:54,400][20381] Updated weights for policy 1, policy_version 1413300 (0.0008) [2023-12-27 15:26:54,428][20347] Updated weights for policy 0, policy_version 1414543 (0.0006) [2023-12-27 15:26:54,449][20381] Updated weights for policy 1, policy_version 1413310 (0.0010) [2023-12-27 15:26:54,484][20347] Updated weights for policy 0, policy_version 1414553 (0.0005) [2023-12-27 15:26:54,498][20381] Updated weights for policy 1, policy_version 1413320 (0.0010) [2023-12-27 15:26:54,538][20347] Updated weights for policy 0, policy_version 1414563 (0.0006) [2023-12-27 15:26:55,164][20347] Updated weights for policy 0, policy_version 1414573 (0.0009) [2023-12-27 15:26:55,192][20381] Updated weights for policy 1, policy_version 1413330 (0.0008) [2023-12-27 15:26:55,215][20347] Updated weights for policy 0, policy_version 1414583 (0.0010) [2023-12-27 15:26:55,259][20381] Updated weights for policy 1, policy_version 1413340 (0.0009) [2023-12-27 15:26:55,267][20347] Updated weights for policy 0, policy_version 1414593 (0.0010) [2023-12-27 15:26:55,313][20381] Updated weights for policy 1, policy_version 1413350 (0.0010) [2023-12-27 15:26:55,377][20381] Updated weights for policy 1, policy_version 1413360 (0.0010) [2023-12-27 15:26:55,906][20347] Updated weights for policy 0, policy_version 1414603 (0.0009) [2023-12-27 15:26:55,964][20347] Updated weights for policy 0, policy_version 1414613 (0.0007) [2023-12-27 15:26:56,030][20347] Updated weights for policy 0, policy_version 1414623 (0.0006) [2023-12-27 15:26:56,047][20381] Updated weights for policy 1, policy_version 1413370 (0.0011) [2023-12-27 15:26:56,098][20381] Updated weights for policy 1, policy_version 1413380 (0.0011) [2023-12-27 15:26:56,157][20381] Updated weights for policy 1, policy_version 1413390 (0.0011) [2023-12-27 15:26:56,664][20347] Updated weights for policy 0, policy_version 1414633 (0.0006) [2023-12-27 15:26:56,722][20347] Updated weights for policy 0, policy_version 1414643 (0.0007) [2023-12-27 15:26:56,770][20347] Updated weights for policy 0, policy_version 1414653 (0.0006) [2023-12-27 15:26:56,818][20347] Updated weights for policy 0, policy_version 1414663 (0.0006) [2023-12-27 15:26:56,828][20381] Updated weights for policy 1, policy_version 1413400 (0.0006) [2023-12-27 15:26:56,874][20381] Updated weights for policy 1, policy_version 1413410 (0.0008) [2023-12-27 15:26:56,922][20381] Updated weights for policy 1, policy_version 1413420 (0.0010) [2023-12-27 15:26:57,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 724090880. Throughput: 0: 10405.0, 1: 10337.8. Samples: 724081004. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:26:57,363][19170] Avg episode reward: [(0, '960.840'), (1, '1000.000')] [2023-12-27 15:26:57,469][20381] Updated weights for policy 1, policy_version 1413430 (0.0006) [2023-12-27 15:26:57,494][20347] Updated weights for policy 0, policy_version 1414673 (0.0008) [2023-12-27 15:26:57,519][20381] Updated weights for policy 1, policy_version 1413440 (0.0006) [2023-12-27 15:26:57,548][20347] Updated weights for policy 0, policy_version 1414683 (0.0008) [2023-12-27 15:26:57,565][20381] Updated weights for policy 1, policy_version 1413450 (0.0005) [2023-12-27 15:26:57,601][20347] Updated weights for policy 0, policy_version 1414693 (0.0006) [2023-12-27 15:26:58,164][20381] Updated weights for policy 1, policy_version 1413460 (0.0009) [2023-12-27 15:26:58,228][20381] Updated weights for policy 1, policy_version 1413470 (0.0007) [2023-12-27 15:26:58,295][20381] Updated weights for policy 1, policy_version 1413480 (0.0007) [2023-12-27 15:26:58,319][20347] Updated weights for policy 0, policy_version 1414703 (0.0008) [2023-12-27 15:26:58,385][20347] Updated weights for policy 0, policy_version 1414713 (0.0008) [2023-12-27 15:26:58,451][20347] Updated weights for policy 0, policy_version 1414723 (0.0008) [2023-12-27 15:26:59,072][20381] Updated weights for policy 1, policy_version 1413490 (0.0007) [2023-12-27 15:26:59,126][20381] Updated weights for policy 1, policy_version 1413500 (0.0005) [2023-12-27 15:26:59,186][20381] Updated weights for policy 1, policy_version 1413510 (0.0005) [2023-12-27 15:26:59,246][20381] Updated weights for policy 1, policy_version 1413520 (0.0007) [2023-12-27 15:26:59,252][20347] Updated weights for policy 0, policy_version 1414733 (0.0008) [2023-12-27 15:26:59,304][20291] KL-divergence is very high: 251.9623 [2023-12-27 15:26:59,316][20291] KL-divergence is very high: 434.9421 [2023-12-27 15:26:59,317][20347] Updated weights for policy 0, policy_version 1414743 (0.0007) [2023-12-27 15:26:59,358][20291] KL-divergence is very high: 307.9365 [2023-12-27 15:26:59,372][20291] KL-divergence is very high: 364.4202 [2023-12-27 15:26:59,385][20347] Updated weights for policy 0, policy_version 1414753 (0.0008) [2023-12-27 15:26:59,417][20291] KL-divergence is very high: 113.7394 [2023-12-27 15:26:59,944][20381] Updated weights for policy 1, policy_version 1413530 (0.0007) [2023-12-27 15:27:00,010][20381] Updated weights for policy 1, policy_version 1413540 (0.0008) [2023-12-27 15:27:00,071][20381] Updated weights for policy 1, policy_version 1413550 (0.0008) [2023-12-27 15:27:00,097][20347] Updated weights for policy 0, policy_version 1414763 (0.0007) [2023-12-27 15:27:00,152][20347] Updated weights for policy 0, policy_version 1414773 (0.0009) [2023-12-27 15:27:00,213][20347] Updated weights for policy 0, policy_version 1414783 (0.0008) [2023-12-27 15:27:00,766][20381] Updated weights for policy 1, policy_version 1413560 (0.0009) [2023-12-27 15:27:00,816][20347] Updated weights for policy 0, policy_version 1414793 (0.0007) [2023-12-27 15:27:00,817][20381] Updated weights for policy 1, policy_version 1413570 (0.0009) [2023-12-27 15:27:00,873][20347] Updated weights for policy 0, policy_version 1414803 (0.0006) [2023-12-27 15:27:00,875][20381] Updated weights for policy 1, policy_version 1413580 (0.0007) [2023-12-27 15:27:00,928][20347] Updated weights for policy 0, policy_version 1414813 (0.0005) [2023-12-27 15:27:00,985][20347] Updated weights for policy 0, policy_version 1414823 (0.0005) [2023-12-27 15:27:01,615][20381] Updated weights for policy 1, policy_version 1413590 (0.0006) [2023-12-27 15:27:01,637][20347] Updated weights for policy 0, policy_version 1414833 (0.0008) [2023-12-27 15:27:01,664][20291] KL-divergence is very high: 117.1914 [2023-12-27 15:27:01,681][20381] Updated weights for policy 1, policy_version 1413601 (0.0007) [2023-12-27 15:27:01,698][20347] Updated weights for policy 0, policy_version 1414843 (0.0009) [2023-12-27 15:27:01,716][20291] KL-divergence is very high: 146.1381 [2023-12-27 15:27:01,744][20381] Updated weights for policy 1, policy_version 1413611 (0.0008) [2023-12-27 15:27:01,764][20347] Updated weights for policy 0, policy_version 1414854 (0.0010) [2023-12-27 15:27:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.5, 300 sec: 20521.6). Total num frames: 724189184. Throughput: 0: 10328.1, 1: 10332.5. Samples: 724203436. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:02,363][19170] Avg episode reward: [(0, '941.440'), (1, '990.290')] [2023-12-27 15:27:02,374][20381] Updated weights for policy 1, policy_version 1413621 (0.0008) [2023-12-27 15:27:02,431][20381] Updated weights for policy 1, policy_version 1413631 (0.0007) [2023-12-27 15:27:02,463][20347] Updated weights for policy 0, policy_version 1414864 (0.0005) [2023-12-27 15:27:02,489][20381] Updated weights for policy 1, policy_version 1413641 (0.0008) [2023-12-27 15:27:02,516][20347] Updated weights for policy 0, policy_version 1414874 (0.0008) [2023-12-27 15:27:02,574][20347] Updated weights for policy 0, policy_version 1414884 (0.0008) [2023-12-27 15:27:03,105][20381] Updated weights for policy 1, policy_version 1413651 (0.0010) [2023-12-27 15:27:03,157][20381] Updated weights for policy 1, policy_version 1413661 (0.0008) [2023-12-27 15:27:03,205][20381] Updated weights for policy 1, policy_version 1413671 (0.0007) [2023-12-27 15:27:03,285][20347] Updated weights for policy 0, policy_version 1414894 (0.0007) [2023-12-27 15:27:03,330][20347] Updated weights for policy 0, policy_version 1414904 (0.0008) [2023-12-27 15:27:03,377][20347] Updated weights for policy 0, policy_version 1414914 (0.0007) [2023-12-27 15:27:03,880][20381] Updated weights for policy 1, policy_version 1413681 (0.0007) [2023-12-27 15:27:03,951][20381] Updated weights for policy 1, policy_version 1413691 (0.0009) [2023-12-27 15:27:04,013][20381] Updated weights for policy 1, policy_version 1413701 (0.0008) [2023-12-27 15:27:04,046][20347] Updated weights for policy 0, policy_version 1414924 (0.0008) [2023-12-27 15:27:04,077][20381] Updated weights for policy 1, policy_version 1413711 (0.0008) [2023-12-27 15:27:04,107][20347] Updated weights for policy 0, policy_version 1414934 (0.0008) [2023-12-27 15:27:04,156][20347] Updated weights for policy 0, policy_version 1414944 (0.0008) [2023-12-27 15:27:04,670][20381] Updated weights for policy 1, policy_version 1413721 (0.0009) [2023-12-27 15:27:04,721][20381] Updated weights for policy 1, policy_version 1413731 (0.0009) [2023-12-27 15:27:04,778][20381] Updated weights for policy 1, policy_version 1413741 (0.0005) [2023-12-27 15:27:04,870][20347] Updated weights for policy 0, policy_version 1414954 (0.0009) [2023-12-27 15:27:04,921][20347] Updated weights for policy 0, policy_version 1414964 (0.0010) [2023-12-27 15:27:04,973][20347] Updated weights for policy 0, policy_version 1414974 (0.0010) [2023-12-27 15:27:05,018][20347] Updated weights for policy 0, policy_version 1414984 (0.0010) [2023-12-27 15:27:05,429][20381] Updated weights for policy 1, policy_version 1413751 (0.0005) [2023-12-27 15:27:05,483][20381] Updated weights for policy 1, policy_version 1413761 (0.0008) [2023-12-27 15:27:05,538][20381] Updated weights for policy 1, policy_version 1413771 (0.0010) [2023-12-27 15:27:05,672][20347] Updated weights for policy 0, policy_version 1414994 (0.0007) [2023-12-27 15:27:05,718][20347] Updated weights for policy 0, policy_version 1415004 (0.0005) [2023-12-27 15:27:05,761][20347] Updated weights for policy 0, policy_version 1415014 (0.0006) [2023-12-27 15:27:06,224][20381] Updated weights for policy 1, policy_version 1413781 (0.0009) [2023-12-27 15:27:06,285][20381] Updated weights for policy 1, policy_version 1413791 (0.0007) [2023-12-27 15:27:06,339][20381] Updated weights for policy 1, policy_version 1413801 (0.0009) [2023-12-27 15:27:06,414][20347] Updated weights for policy 0, policy_version 1415024 (0.0008) [2023-12-27 15:27:06,480][20347] Updated weights for policy 0, policy_version 1415034 (0.0007) [2023-12-27 15:27:06,542][20347] Updated weights for policy 0, policy_version 1415044 (0.0007) [2023-12-27 15:27:07,017][20381] Updated weights for policy 1, policy_version 1413811 (0.0006) [2023-12-27 15:27:07,072][20381] Updated weights for policy 1, policy_version 1413821 (0.0007) [2023-12-27 15:27:07,131][20381] Updated weights for policy 1, policy_version 1413831 (0.0006) [2023-12-27 15:27:07,243][20347] Updated weights for policy 0, policy_version 1415054 (0.0007) [2023-12-27 15:27:07,295][20347] Updated weights for policy 0, policy_version 1415064 (0.0006) [2023-12-27 15:27:07,353][20347] Updated weights for policy 0, policy_version 1415074 (0.0005) [2023-12-27 15:27:07,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 724295680. Throughput: 0: 10333.7, 1: 10381.4. Samples: 724265932. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:07,363][19170] Avg episode reward: [(0, '941.480'), (1, '990.290')] [2023-12-27 15:27:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001413840_361971712.pth... [2023-12-27 15:27:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001412624_361660416.pth [2023-12-27 15:27:07,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001415080_362332160.pth... [2023-12-27 15:27:07,394][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001413896_362029056.pth [2023-12-27 15:27:07,777][20381] Updated weights for policy 1, policy_version 1413841 (0.0005) [2023-12-27 15:27:07,833][20381] Updated weights for policy 1, policy_version 1413851 (0.0007) [2023-12-27 15:27:07,882][20381] Updated weights for policy 1, policy_version 1413861 (0.0008) [2023-12-27 15:27:07,931][20347] Updated weights for policy 0, policy_version 1415084 (0.0006) [2023-12-27 15:27:07,933][20381] Updated weights for policy 1, policy_version 1413871 (0.0007) [2023-12-27 15:27:07,977][20347] Updated weights for policy 0, policy_version 1415094 (0.0007) [2023-12-27 15:27:08,034][20347] Updated weights for policy 0, policy_version 1415104 (0.0008) [2023-12-27 15:27:08,667][20381] Updated weights for policy 1, policy_version 1413881 (0.0007) [2023-12-27 15:27:08,733][20381] Updated weights for policy 1, policy_version 1413891 (0.0008) [2023-12-27 15:27:08,739][20347] Updated weights for policy 0, policy_version 1415114 (0.0008) [2023-12-27 15:27:08,791][20381] Updated weights for policy 1, policy_version 1413901 (0.0006) [2023-12-27 15:27:08,797][20347] Updated weights for policy 0, policy_version 1415124 (0.0006) [2023-12-27 15:27:08,847][20347] Updated weights for policy 0, policy_version 1415134 (0.0005) [2023-12-27 15:27:08,897][20347] Updated weights for policy 0, policy_version 1415144 (0.0005) [2023-12-27 15:27:09,520][20381] Updated weights for policy 1, policy_version 1413911 (0.0007) [2023-12-27 15:27:09,579][20381] Updated weights for policy 1, policy_version 1413921 (0.0007) [2023-12-27 15:27:09,613][20347] Updated weights for policy 0, policy_version 1415154 (0.0008) [2023-12-27 15:27:09,632][20381] Updated weights for policy 1, policy_version 1413931 (0.0007) [2023-12-27 15:27:09,664][20347] Updated weights for policy 0, policy_version 1415164 (0.0008) [2023-12-27 15:27:09,720][20347] Updated weights for policy 0, policy_version 1415174 (0.0009) [2023-12-27 15:27:10,317][20381] Updated weights for policy 1, policy_version 1413941 (0.0008) [2023-12-27 15:27:10,380][20381] Updated weights for policy 1, policy_version 1413951 (0.0008) [2023-12-27 15:27:10,441][20381] Updated weights for policy 1, policy_version 1413961 (0.0007) [2023-12-27 15:27:10,483][20347] Updated weights for policy 0, policy_version 1415184 (0.0006) [2023-12-27 15:27:10,544][20347] Updated weights for policy 0, policy_version 1415194 (0.0007) [2023-12-27 15:27:10,606][20347] Updated weights for policy 0, policy_version 1415204 (0.0008) [2023-12-27 15:27:11,124][20381] Updated weights for policy 1, policy_version 1413971 (0.0006) [2023-12-27 15:27:11,188][20381] Updated weights for policy 1, policy_version 1413981 (0.0008) [2023-12-27 15:27:11,252][20381] Updated weights for policy 1, policy_version 1413991 (0.0007) [2023-12-27 15:27:11,430][20347] Updated weights for policy 0, policy_version 1415214 (0.0007) [2023-12-27 15:27:11,490][20347] Updated weights for policy 0, policy_version 1415224 (0.0007) [2023-12-27 15:27:11,548][20347] Updated weights for policy 0, policy_version 1415234 (0.0008) [2023-12-27 15:27:11,978][20381] Updated weights for policy 1, policy_version 1414001 (0.0007) [2023-12-27 15:27:12,042][20381] Updated weights for policy 1, policy_version 1414011 (0.0008) [2023-12-27 15:27:12,105][20381] Updated weights for policy 1, policy_version 1414021 (0.0009) [2023-12-27 15:27:12,158][20381] Updated weights for policy 1, policy_version 1414031 (0.0007) [2023-12-27 15:27:12,285][20347] Updated weights for policy 0, policy_version 1415244 (0.0008) [2023-12-27 15:27:12,344][20347] Updated weights for policy 0, policy_version 1415254 (0.0008) [2023-12-27 15:27:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 724393984. Throughput: 0: 10282.3, 1: 10341.4. Samples: 724387092. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:12,363][19170] Avg episode reward: [(0, '970.690'), (1, '1000.000')] [2023-12-27 15:27:12,406][20347] Updated weights for policy 0, policy_version 1415264 (0.0008) [2023-12-27 15:27:12,911][20381] Updated weights for policy 1, policy_version 1414041 (0.0006) [2023-12-27 15:27:12,959][20381] Updated weights for policy 1, policy_version 1414051 (0.0007) [2023-12-27 15:27:13,011][20381] Updated weights for policy 1, policy_version 1414061 (0.0008) [2023-12-27 15:27:13,084][20347] Updated weights for policy 0, policy_version 1415274 (0.0007) [2023-12-27 15:27:13,137][20347] Updated weights for policy 0, policy_version 1415284 (0.0005) [2023-12-27 15:27:13,190][20347] Updated weights for policy 0, policy_version 1415294 (0.0006) [2023-12-27 15:27:13,236][20347] Updated weights for policy 0, policy_version 1415304 (0.0007) [2023-12-27 15:27:13,670][20381] Updated weights for policy 1, policy_version 1414071 (0.0008) [2023-12-27 15:27:13,723][20381] Updated weights for policy 1, policy_version 1414081 (0.0010) [2023-12-27 15:27:13,782][20381] Updated weights for policy 1, policy_version 1414091 (0.0005) [2023-12-27 15:27:13,890][20347] Updated weights for policy 0, policy_version 1415314 (0.0005) [2023-12-27 15:27:13,936][20347] Updated weights for policy 0, policy_version 1415324 (0.0006) [2023-12-27 15:27:13,981][20347] Updated weights for policy 0, policy_version 1415334 (0.0005) [2023-12-27 15:27:14,412][20381] Updated weights for policy 1, policy_version 1414101 (0.0008) [2023-12-27 15:27:14,466][20381] Updated weights for policy 1, policy_version 1414111 (0.0010) [2023-12-27 15:27:14,524][20381] Updated weights for policy 1, policy_version 1414121 (0.0010) [2023-12-27 15:27:14,616][20347] Updated weights for policy 0, policy_version 1415344 (0.0009) [2023-12-27 15:27:14,667][20347] Updated weights for policy 0, policy_version 1415354 (0.0010) [2023-12-27 15:27:14,716][20347] Updated weights for policy 0, policy_version 1415364 (0.0010) [2023-12-27 15:27:15,240][20381] Updated weights for policy 1, policy_version 1414131 (0.0007) [2023-12-27 15:27:15,290][20381] Updated weights for policy 1, policy_version 1414141 (0.0008) [2023-12-27 15:27:15,340][20381] Updated weights for policy 1, policy_version 1414151 (0.0008) [2023-12-27 15:27:15,428][20347] Updated weights for policy 0, policy_version 1415374 (0.0008) [2023-12-27 15:27:15,479][20347] Updated weights for policy 0, policy_version 1415384 (0.0007) [2023-12-27 15:27:15,529][20347] Updated weights for policy 0, policy_version 1415394 (0.0008) [2023-12-27 15:27:16,065][20381] Updated weights for policy 1, policy_version 1414161 (0.0008) [2023-12-27 15:27:16,114][20381] Updated weights for policy 1, policy_version 1414171 (0.0007) [2023-12-27 15:27:16,163][20381] Updated weights for policy 1, policy_version 1414181 (0.0007) [2023-12-27 15:27:16,191][20347] Updated weights for policy 0, policy_version 1415404 (0.0007) [2023-12-27 15:27:16,222][20381] Updated weights for policy 1, policy_version 1414191 (0.0006) [2023-12-27 15:27:16,244][20347] Updated weights for policy 0, policy_version 1415414 (0.0006) [2023-12-27 15:27:16,295][20347] Updated weights for policy 0, policy_version 1415424 (0.0007) [2023-12-27 15:27:16,885][20381] Updated weights for policy 1, policy_version 1414201 (0.0008) [2023-12-27 15:27:16,897][20347] Updated weights for policy 0, policy_version 1415434 (0.0007) [2023-12-27 15:27:16,937][20381] Updated weights for policy 1, policy_version 1414211 (0.0006) [2023-12-27 15:27:16,955][20347] Updated weights for policy 0, policy_version 1415444 (0.0007) [2023-12-27 15:27:16,991][20381] Updated weights for policy 1, policy_version 1414221 (0.0006) [2023-12-27 15:27:17,011][20347] Updated weights for policy 0, policy_version 1415454 (0.0010) [2023-12-27 15:27:17,058][20347] Updated weights for policy 0, policy_version 1415464 (0.0009) [2023-12-27 15:27:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 724500480. Throughput: 0: 10213.4, 1: 10271.4. Samples: 724509492. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:17,363][19170] Avg episode reward: [(0, '951.100'), (1, '1000.000')] [2023-12-27 15:27:17,670][20347] Updated weights for policy 0, policy_version 1415474 (0.0005) [2023-12-27 15:27:17,708][20381] Updated weights for policy 1, policy_version 1414231 (0.0007) [2023-12-27 15:27:17,730][20347] Updated weights for policy 0, policy_version 1415484 (0.0006) [2023-12-27 15:27:17,756][20381] Updated weights for policy 1, policy_version 1414241 (0.0007) [2023-12-27 15:27:17,774][20347] Updated weights for policy 0, policy_version 1415494 (0.0005) [2023-12-27 15:27:17,804][20381] Updated weights for policy 1, policy_version 1414251 (0.0008) [2023-12-27 15:27:18,379][20347] Updated weights for policy 0, policy_version 1415504 (0.0010) [2023-12-27 15:27:18,435][20347] Updated weights for policy 0, policy_version 1415514 (0.0008) [2023-12-27 15:27:18,492][20347] Updated weights for policy 0, policy_version 1415524 (0.0006) [2023-12-27 15:27:18,548][20381] Updated weights for policy 1, policy_version 1414261 (0.0007) [2023-12-27 15:27:18,608][20381] Updated weights for policy 1, policy_version 1414271 (0.0010) [2023-12-27 15:27:18,669][20381] Updated weights for policy 1, policy_version 1414281 (0.0010) [2023-12-27 15:27:19,178][20347] Updated weights for policy 0, policy_version 1415535 (0.0009) [2023-12-27 15:27:19,254][20347] Updated weights for policy 0, policy_version 1415545 (0.0008) [2023-12-27 15:27:19,314][20347] Updated weights for policy 0, policy_version 1415555 (0.0010) [2023-12-27 15:27:19,366][20381] Updated weights for policy 1, policy_version 1414291 (0.0010) [2023-12-27 15:27:19,426][20381] Updated weights for policy 1, policy_version 1414301 (0.0008) [2023-12-27 15:27:19,491][20381] Updated weights for policy 1, policy_version 1414311 (0.0007) [2023-12-27 15:27:19,996][20347] Updated weights for policy 0, policy_version 1415565 (0.0009) [2023-12-27 15:27:20,054][20347] Updated weights for policy 0, policy_version 1415575 (0.0007) [2023-12-27 15:27:20,108][20347] Updated weights for policy 0, policy_version 1415585 (0.0008) [2023-12-27 15:27:20,198][20381] Updated weights for policy 1, policy_version 1414321 (0.0007) [2023-12-27 15:27:20,258][20381] Updated weights for policy 1, policy_version 1414331 (0.0008) [2023-12-27 15:27:20,310][20381] Updated weights for policy 1, policy_version 1414341 (0.0008) [2023-12-27 15:27:20,371][20381] Updated weights for policy 1, policy_version 1414351 (0.0007) [2023-12-27 15:27:20,861][20347] Updated weights for policy 0, policy_version 1415595 (0.0007) [2023-12-27 15:27:20,919][20347] Updated weights for policy 0, policy_version 1415605 (0.0008) [2023-12-27 15:27:20,980][20347] Updated weights for policy 0, policy_version 1415615 (0.0008) [2023-12-27 15:27:21,107][20381] Updated weights for policy 1, policy_version 1414361 (0.0008) [2023-12-27 15:27:21,172][20381] Updated weights for policy 1, policy_version 1414371 (0.0008) [2023-12-27 15:27:21,238][20381] Updated weights for policy 1, policy_version 1414381 (0.0008) [2023-12-27 15:27:21,829][20347] Updated weights for policy 0, policy_version 1415625 (0.0008) [2023-12-27 15:27:21,891][20347] Updated weights for policy 0, policy_version 1415635 (0.0009) [2023-12-27 15:27:21,954][20347] Updated weights for policy 0, policy_version 1415645 (0.0008) [2023-12-27 15:27:21,968][20381] Updated weights for policy 1, policy_version 1414391 (0.0006) [2023-12-27 15:27:22,008][20347] Updated weights for policy 0, policy_version 1415655 (0.0011) [2023-12-27 15:27:22,030][20381] Updated weights for policy 1, policy_version 1414401 (0.0006) [2023-12-27 15:27:22,087][20381] Updated weights for policy 1, policy_version 1414411 (0.0006) [2023-12-27 15:27:22,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 724598784. Throughput: 0: 10177.9, 1: 10227.8. Samples: 724570320. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:22,363][19170] Avg episode reward: [(0, '941.450'), (1, '1000.000')] [2023-12-27 15:27:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001414416_362119168.pth... [2023-12-27 15:27:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001415656_362479616.pth... [2023-12-27 15:27:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001413232_361816064.pth [2023-12-27 15:27:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001414472_362176512.pth [2023-12-27 15:27:22,656][20347] Updated weights for policy 0, policy_version 1415665 (0.0010) [2023-12-27 15:27:22,716][20347] Updated weights for policy 0, policy_version 1415675 (0.0010) [2023-12-27 15:27:22,788][20347] Updated weights for policy 0, policy_version 1415685 (0.0011) [2023-12-27 15:27:22,804][20381] Updated weights for policy 1, policy_version 1414421 (0.0007) [2023-12-27 15:27:22,855][20381] Updated weights for policy 1, policy_version 1414431 (0.0007) [2023-12-27 15:27:22,911][20381] Updated weights for policy 1, policy_version 1414441 (0.0008) [2023-12-27 15:27:23,427][20347] Updated weights for policy 0, policy_version 1415695 (0.0007) [2023-12-27 15:27:23,489][20347] Updated weights for policy 0, policy_version 1415705 (0.0007) [2023-12-27 15:27:23,550][20347] Updated weights for policy 0, policy_version 1415715 (0.0008) [2023-12-27 15:27:23,613][20381] Updated weights for policy 1, policy_version 1414451 (0.0007) [2023-12-27 15:27:23,667][20381] Updated weights for policy 1, policy_version 1414461 (0.0008) [2023-12-27 15:27:23,726][20381] Updated weights for policy 1, policy_version 1414471 (0.0008) [2023-12-27 15:27:24,150][20347] Updated weights for policy 0, policy_version 1415725 (0.0007) [2023-12-27 15:27:24,214][20347] Updated weights for policy 0, policy_version 1415735 (0.0007) [2023-12-27 15:27:24,269][20347] Updated weights for policy 0, policy_version 1415745 (0.0007) [2023-12-27 15:27:24,436][20381] Updated weights for policy 1, policy_version 1414481 (0.0007) [2023-12-27 15:27:24,489][20381] Updated weights for policy 1, policy_version 1414492 (0.0009) [2023-12-27 15:27:24,539][20381] Updated weights for policy 1, policy_version 1414502 (0.0009) [2023-12-27 15:27:24,595][20381] Updated weights for policy 1, policy_version 1414512 (0.0008) [2023-12-27 15:27:24,894][20347] Updated weights for policy 0, policy_version 1415755 (0.0007) [2023-12-27 15:27:24,951][20347] Updated weights for policy 0, policy_version 1415765 (0.0006) [2023-12-27 15:27:25,002][20347] Updated weights for policy 0, policy_version 1415775 (0.0007) [2023-12-27 15:27:25,302][20381] Updated weights for policy 1, policy_version 1414522 (0.0007) [2023-12-27 15:27:25,351][20381] Updated weights for policy 1, policy_version 1414532 (0.0009) [2023-12-27 15:27:25,402][20381] Updated weights for policy 1, policy_version 1414542 (0.0008) [2023-12-27 15:27:25,585][20347] Updated weights for policy 0, policy_version 1415785 (0.0007) [2023-12-27 15:27:25,633][20347] Updated weights for policy 0, policy_version 1415795 (0.0005) [2023-12-27 15:27:25,679][20347] Updated weights for policy 0, policy_version 1415805 (0.0005) [2023-12-27 15:27:25,728][20347] Updated weights for policy 0, policy_version 1415815 (0.0005) [2023-12-27 15:27:26,035][20381] Updated weights for policy 1, policy_version 1414552 (0.0006) [2023-12-27 15:27:26,089][20381] Updated weights for policy 1, policy_version 1414562 (0.0005) [2023-12-27 15:27:26,141][20381] Updated weights for policy 1, policy_version 1414572 (0.0006) [2023-12-27 15:27:26,424][20347] Updated weights for policy 0, policy_version 1415825 (0.0006) [2023-12-27 15:27:26,478][20347] Updated weights for policy 0, policy_version 1415835 (0.0007) [2023-12-27 15:27:26,537][20347] Updated weights for policy 0, policy_version 1415845 (0.0008) [2023-12-27 15:27:26,672][20381] Updated weights for policy 1, policy_version 1414582 (0.0006) [2023-12-27 15:27:26,728][20381] Updated weights for policy 1, policy_version 1414592 (0.0008) [2023-12-27 15:27:26,786][20381] Updated weights for policy 1, policy_version 1414602 (0.0009) [2023-12-27 15:27:27,247][20347] Updated weights for policy 0, policy_version 1415855 (0.0008) [2023-12-27 15:27:27,306][20347] Updated weights for policy 0, policy_version 1415865 (0.0008) [2023-12-27 15:27:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 724697088. Throughput: 0: 10232.6, 1: 10209.4. Samples: 724692144. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:27,363][19170] Avg episode reward: [(0, '961.080'), (1, '1000.000')] [2023-12-27 15:27:27,363][20347] Updated weights for policy 0, policy_version 1415875 (0.0010) [2023-12-27 15:27:27,374][20381] Updated weights for policy 1, policy_version 1414612 (0.0007) [2023-12-27 15:27:27,426][20381] Updated weights for policy 1, policy_version 1414622 (0.0005) [2023-12-27 15:27:27,484][20381] Updated weights for policy 1, policy_version 1414632 (0.0005) [2023-12-27 15:27:27,964][20347] Updated weights for policy 0, policy_version 1415885 (0.0010) [2023-12-27 15:27:27,997][20381] Updated weights for policy 1, policy_version 1414642 (0.0006) [2023-12-27 15:27:28,022][20347] Updated weights for policy 0, policy_version 1415895 (0.0010) [2023-12-27 15:27:28,052][20381] Updated weights for policy 1, policy_version 1414652 (0.0010) [2023-12-27 15:27:28,073][20347] Updated weights for policy 0, policy_version 1415905 (0.0010) [2023-12-27 15:27:28,100][20381] Updated weights for policy 1, policy_version 1414662 (0.0010) [2023-12-27 15:27:28,151][20381] Updated weights for policy 1, policy_version 1414672 (0.0010) [2023-12-27 15:27:28,736][20347] Updated weights for policy 0, policy_version 1415915 (0.0009) [2023-12-27 15:27:28,795][20347] Updated weights for policy 0, policy_version 1415925 (0.0007) [2023-12-27 15:27:28,834][20381] Updated weights for policy 1, policy_version 1414682 (0.0008) [2023-12-27 15:27:28,848][20347] Updated weights for policy 0, policy_version 1415935 (0.0005) [2023-12-27 15:27:28,883][20381] Updated weights for policy 1, policy_version 1414692 (0.0007) [2023-12-27 15:27:28,937][20381] Updated weights for policy 1, policy_version 1414702 (0.0007) [2023-12-27 15:27:29,492][20347] Updated weights for policy 0, policy_version 1415945 (0.0006) [2023-12-27 15:27:29,543][20347] Updated weights for policy 0, policy_version 1415955 (0.0008) [2023-12-27 15:27:29,595][20347] Updated weights for policy 0, policy_version 1415965 (0.0008) [2023-12-27 15:27:29,646][20347] Updated weights for policy 0, policy_version 1415975 (0.0007) [2023-12-27 15:27:29,677][20381] Updated weights for policy 1, policy_version 1414712 (0.0007) [2023-12-27 15:27:29,729][20381] Updated weights for policy 1, policy_version 1414722 (0.0008) [2023-12-27 15:27:29,782][20381] Updated weights for policy 1, policy_version 1414732 (0.0010) [2023-12-27 15:27:30,317][20347] Updated weights for policy 0, policy_version 1415985 (0.0007) [2023-12-27 15:27:30,387][20347] Updated weights for policy 0, policy_version 1415995 (0.0007) [2023-12-27 15:27:30,441][20347] Updated weights for policy 0, policy_version 1416005 (0.0010) [2023-12-27 15:27:30,515][20381] Updated weights for policy 1, policy_version 1414742 (0.0008) [2023-12-27 15:27:30,560][20381] Updated weights for policy 1, policy_version 1414752 (0.0005) [2023-12-27 15:27:30,608][20381] Updated weights for policy 1, policy_version 1414762 (0.0008) [2023-12-27 15:27:31,070][20347] Updated weights for policy 0, policy_version 1416015 (0.0009) [2023-12-27 15:27:31,123][20347] Updated weights for policy 0, policy_version 1416025 (0.0010) [2023-12-27 15:27:31,190][20347] Updated weights for policy 0, policy_version 1416035 (0.0007) [2023-12-27 15:27:31,306][20381] Updated weights for policy 1, policy_version 1414772 (0.0011) [2023-12-27 15:27:31,367][20381] Updated weights for policy 1, policy_version 1414782 (0.0011) [2023-12-27 15:27:31,427][20381] Updated weights for policy 1, policy_version 1414792 (0.0008) [2023-12-27 15:27:31,881][20347] Updated weights for policy 0, policy_version 1416045 (0.0007) [2023-12-27 15:27:31,943][20347] Updated weights for policy 0, policy_version 1416055 (0.0007) [2023-12-27 15:27:32,008][20347] Updated weights for policy 0, policy_version 1416065 (0.0007) [2023-12-27 15:27:32,066][20381] Updated weights for policy 1, policy_version 1414802 (0.0009) [2023-12-27 15:27:32,119][20381] Updated weights for policy 1, policy_version 1414812 (0.0006) [2023-12-27 15:27:32,184][20381] Updated weights for policy 1, policy_version 1414822 (0.0008) [2023-12-27 15:27:32,245][20381] Updated weights for policy 1, policy_version 1414832 (0.0008) [2023-12-27 15:27:32,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20616.3, 300 sec: 20549.4). Total num frames: 724811776. Throughput: 0: 10239.3, 1: 10215.9. Samples: 724818560. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:32,364][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 15:27:32,722][20347] Updated weights for policy 0, policy_version 1416075 (0.0009) [2023-12-27 15:27:32,774][20347] Updated weights for policy 0, policy_version 1416085 (0.0008) [2023-12-27 15:27:32,830][20347] Updated weights for policy 0, policy_version 1416095 (0.0008) [2023-12-27 15:27:32,937][20381] Updated weights for policy 1, policy_version 1414842 (0.0005) [2023-12-27 15:27:32,989][20381] Updated weights for policy 1, policy_version 1414852 (0.0005) [2023-12-27 15:27:33,034][20381] Updated weights for policy 1, policy_version 1414862 (0.0005) [2023-12-27 15:27:33,543][20347] Updated weights for policy 0, policy_version 1416105 (0.0007) [2023-12-27 15:27:33,572][20381] Updated weights for policy 1, policy_version 1414872 (0.0007) [2023-12-27 15:27:33,601][20347] Updated weights for policy 0, policy_version 1416115 (0.0006) [2023-12-27 15:27:33,621][20381] Updated weights for policy 1, policy_version 1414882 (0.0006) [2023-12-27 15:27:33,661][20347] Updated weights for policy 0, policy_version 1416125 (0.0007) [2023-12-27 15:27:33,669][20381] Updated weights for policy 1, policy_version 1414892 (0.0007) [2023-12-27 15:27:33,712][20347] Updated weights for policy 0, policy_version 1416135 (0.0007) [2023-12-27 15:27:34,344][20381] Updated weights for policy 1, policy_version 1414902 (0.0006) [2023-12-27 15:27:34,367][20347] Updated weights for policy 0, policy_version 1416145 (0.0007) [2023-12-27 15:27:34,400][20381] Updated weights for policy 1, policy_version 1414912 (0.0006) [2023-12-27 15:27:34,424][20347] Updated weights for policy 0, policy_version 1416155 (0.0007) [2023-12-27 15:27:34,461][20381] Updated weights for policy 1, policy_version 1414922 (0.0007) [2023-12-27 15:27:34,482][20347] Updated weights for policy 0, policy_version 1416165 (0.0005) [2023-12-27 15:27:35,125][20381] Updated weights for policy 1, policy_version 1414932 (0.0006) [2023-12-27 15:27:35,149][20347] Updated weights for policy 0, policy_version 1416175 (0.0007) [2023-12-27 15:27:35,170][20381] Updated weights for policy 1, policy_version 1414942 (0.0006) [2023-12-27 15:27:35,199][20347] Updated weights for policy 0, policy_version 1416185 (0.0007) [2023-12-27 15:27:35,220][20381] Updated weights for policy 1, policy_version 1414952 (0.0006) [2023-12-27 15:27:35,247][20347] Updated weights for policy 0, policy_version 1416195 (0.0007) [2023-12-27 15:27:35,880][20381] Updated weights for policy 1, policy_version 1414962 (0.0005) [2023-12-27 15:27:35,918][20347] Updated weights for policy 0, policy_version 1416205 (0.0007) [2023-12-27 15:27:35,937][20381] Updated weights for policy 1, policy_version 1414972 (0.0008) [2023-12-27 15:27:35,978][20347] Updated weights for policy 0, policy_version 1416215 (0.0006) [2023-12-27 15:27:35,987][20381] Updated weights for policy 1, policy_version 1414982 (0.0006) [2023-12-27 15:27:36,033][20347] Updated weights for policy 0, policy_version 1416225 (0.0006) [2023-12-27 15:27:36,036][20381] Updated weights for policy 1, policy_version 1414992 (0.0005) [2023-12-27 15:27:36,725][20347] Updated weights for policy 0, policy_version 1416235 (0.0007) [2023-12-27 15:27:36,780][20347] Updated weights for policy 0, policy_version 1416245 (0.0006) [2023-12-27 15:27:36,810][20381] Updated weights for policy 1, policy_version 1415002 (0.0006) [2023-12-27 15:27:36,839][20347] Updated weights for policy 0, policy_version 1416255 (0.0007) [2023-12-27 15:27:36,872][20381] Updated weights for policy 1, policy_version 1415012 (0.0007) [2023-12-27 15:27:36,935][20381] Updated weights for policy 1, policy_version 1415022 (0.0007) [2023-12-27 15:27:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 724910080. Throughput: 0: 10244.9, 1: 10272.4. Samples: 724881644. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:37,364][19170] Avg episode reward: [(0, '970.760'), (1, '990.310')] [2023-12-27 15:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001415024_362274816.pth... [2023-12-27 15:27:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001416264_362635264.pth... [2023-12-27 15:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001413840_361971712.pth [2023-12-27 15:27:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001415080_362332160.pth [2023-12-27 15:27:37,456][20347] Updated weights for policy 0, policy_version 1416265 (0.0007) [2023-12-27 15:27:37,516][20347] Updated weights for policy 0, policy_version 1416275 (0.0007) [2023-12-27 15:27:37,520][20381] Updated weights for policy 1, policy_version 1415032 (0.0009) [2023-12-27 15:27:37,562][20347] Updated weights for policy 0, policy_version 1416285 (0.0007) [2023-12-27 15:27:37,569][20381] Updated weights for policy 1, policy_version 1415042 (0.0009) [2023-12-27 15:27:37,610][20347] Updated weights for policy 0, policy_version 1416295 (0.0005) [2023-12-27 15:27:37,621][20381] Updated weights for policy 1, policy_version 1415052 (0.0008) [2023-12-27 15:27:38,259][20347] Updated weights for policy 0, policy_version 1416305 (0.0005) [2023-12-27 15:27:38,320][20347] Updated weights for policy 0, policy_version 1416315 (0.0006) [2023-12-27 15:27:38,333][20381] Updated weights for policy 1, policy_version 1415062 (0.0010) [2023-12-27 15:27:38,387][20347] Updated weights for policy 0, policy_version 1416325 (0.0008) [2023-12-27 15:27:38,399][20381] Updated weights for policy 1, policy_version 1415072 (0.0008) [2023-12-27 15:27:38,455][20381] Updated weights for policy 1, policy_version 1415082 (0.0011) [2023-12-27 15:27:38,969][20347] Updated weights for policy 0, policy_version 1416335 (0.0008) [2023-12-27 15:27:39,024][20347] Updated weights for policy 0, policy_version 1416345 (0.0007) [2023-12-27 15:27:39,080][20347] Updated weights for policy 0, policy_version 1416355 (0.0008) [2023-12-27 15:27:39,080][20381] Updated weights for policy 1, policy_version 1415092 (0.0011) [2023-12-27 15:27:39,140][20381] Updated weights for policy 1, policy_version 1415102 (0.0011) [2023-12-27 15:27:39,196][20381] Updated weights for policy 1, policy_version 1415112 (0.0011) [2023-12-27 15:27:39,850][20347] Updated weights for policy 0, policy_version 1416365 (0.0007) [2023-12-27 15:27:39,915][20347] Updated weights for policy 0, policy_version 1416375 (0.0010) [2023-12-27 15:27:39,941][20381] Updated weights for policy 1, policy_version 1415122 (0.0008) [2023-12-27 15:27:39,979][20347] Updated weights for policy 0, policy_version 1416385 (0.0007) [2023-12-27 15:27:40,004][20381] Updated weights for policy 1, policy_version 1415132 (0.0010) [2023-12-27 15:27:40,060][20381] Updated weights for policy 1, policy_version 1415142 (0.0008) [2023-12-27 15:27:40,120][20381] Updated weights for policy 1, policy_version 1415152 (0.0010) [2023-12-27 15:27:40,679][20347] Updated weights for policy 0, policy_version 1416395 (0.0006) [2023-12-27 15:27:40,738][20347] Updated weights for policy 0, policy_version 1416405 (0.0005) [2023-12-27 15:27:40,799][20347] Updated weights for policy 0, policy_version 1416415 (0.0006) [2023-12-27 15:27:40,810][20381] Updated weights for policy 1, policy_version 1415162 (0.0010) [2023-12-27 15:27:40,868][20381] Updated weights for policy 1, policy_version 1415172 (0.0011) [2023-12-27 15:27:40,924][20381] Updated weights for policy 1, policy_version 1415182 (0.0005) [2023-12-27 15:27:41,493][20347] Updated weights for policy 0, policy_version 1416425 (0.0006) [2023-12-27 15:27:41,561][20347] Updated weights for policy 0, policy_version 1416435 (0.0008) [2023-12-27 15:27:41,624][20381] Updated weights for policy 1, policy_version 1415192 (0.0007) [2023-12-27 15:27:41,627][20347] Updated weights for policy 0, policy_version 1416445 (0.0007) [2023-12-27 15:27:41,687][20381] Updated weights for policy 1, policy_version 1415202 (0.0007) [2023-12-27 15:27:41,693][20347] Updated weights for policy 0, policy_version 1416455 (0.0007) [2023-12-27 15:27:41,753][20381] Updated weights for policy 1, policy_version 1415212 (0.0008) [2023-12-27 15:27:42,362][19170] Fps is (10 sec: 19662.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 725008384. Throughput: 0: 10263.9, 1: 10264.6. Samples: 725004784. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:42,363][19170] Avg episode reward: [(0, '960.950'), (1, '980.580')] [2023-12-27 15:27:42,423][20347] Updated weights for policy 0, policy_version 1416465 (0.0008) [2023-12-27 15:27:42,483][20347] Updated weights for policy 0, policy_version 1416475 (0.0008) [2023-12-27 15:27:42,493][20381] Updated weights for policy 1, policy_version 1415222 (0.0007) [2023-12-27 15:27:42,542][20347] Updated weights for policy 0, policy_version 1416485 (0.0006) [2023-12-27 15:27:42,554][20381] Updated weights for policy 1, policy_version 1415232 (0.0006) [2023-12-27 15:27:42,615][20381] Updated weights for policy 1, policy_version 1415242 (0.0007) [2023-12-27 15:27:43,243][20347] Updated weights for policy 0, policy_version 1416495 (0.0007) [2023-12-27 15:27:43,268][20381] Updated weights for policy 1, policy_version 1415252 (0.0007) [2023-12-27 15:27:43,298][20347] Updated weights for policy 0, policy_version 1416505 (0.0006) [2023-12-27 15:27:43,318][20381] Updated weights for policy 1, policy_version 1415262 (0.0007) [2023-12-27 15:27:43,360][20347] Updated weights for policy 0, policy_version 1416515 (0.0005) [2023-12-27 15:27:43,371][20381] Updated weights for policy 1, policy_version 1415272 (0.0006) [2023-12-27 15:27:43,976][20347] Updated weights for policy 0, policy_version 1416525 (0.0006) [2023-12-27 15:27:44,019][20347] Updated weights for policy 0, policy_version 1416535 (0.0007) [2023-12-27 15:27:44,070][20347] Updated weights for policy 0, policy_version 1416545 (0.0007) [2023-12-27 15:27:44,084][20381] Updated weights for policy 1, policy_version 1415282 (0.0008) [2023-12-27 15:27:44,141][20381] Updated weights for policy 1, policy_version 1415292 (0.0006) [2023-12-27 15:27:44,202][20381] Updated weights for policy 1, policy_version 1415302 (0.0007) [2023-12-27 15:27:44,253][20381] Updated weights for policy 1, policy_version 1415312 (0.0008) [2023-12-27 15:27:44,774][20347] Updated weights for policy 0, policy_version 1416555 (0.0006) [2023-12-27 15:27:44,840][20347] Updated weights for policy 0, policy_version 1416565 (0.0008) [2023-12-27 15:27:44,896][20347] Updated weights for policy 0, policy_version 1416575 (0.0008) [2023-12-27 15:27:44,941][20381] Updated weights for policy 1, policy_version 1415322 (0.0007) [2023-12-27 15:27:45,002][20381] Updated weights for policy 1, policy_version 1415332 (0.0008) [2023-12-27 15:27:45,062][20381] Updated weights for policy 1, policy_version 1415342 (0.0008) [2023-12-27 15:27:45,558][20347] Updated weights for policy 0, policy_version 1416585 (0.0007) [2023-12-27 15:27:45,608][20347] Updated weights for policy 0, policy_version 1416595 (0.0007) [2023-12-27 15:27:45,656][20347] Updated weights for policy 0, policy_version 1416605 (0.0007) [2023-12-27 15:27:45,701][20347] Updated weights for policy 0, policy_version 1416615 (0.0007) [2023-12-27 15:27:45,720][20381] Updated weights for policy 1, policy_version 1415352 (0.0006) [2023-12-27 15:27:45,778][20381] Updated weights for policy 1, policy_version 1415362 (0.0005) [2023-12-27 15:27:45,828][20381] Updated weights for policy 1, policy_version 1415372 (0.0005) [2023-12-27 15:27:46,435][20381] Updated weights for policy 1, policy_version 1415382 (0.0007) [2023-12-27 15:27:46,452][20347] Updated weights for policy 0, policy_version 1416625 (0.0007) [2023-12-27 15:27:46,494][20381] Updated weights for policy 1, policy_version 1415392 (0.0009) [2023-12-27 15:27:46,503][20347] Updated weights for policy 0, policy_version 1416635 (0.0005) [2023-12-27 15:27:46,552][20381] Updated weights for policy 1, policy_version 1415402 (0.0011) [2023-12-27 15:27:46,562][20347] Updated weights for policy 0, policy_version 1416645 (0.0005) [2023-12-27 15:27:47,140][20381] Updated weights for policy 1, policy_version 1415412 (0.0010) [2023-12-27 15:27:47,189][20381] Updated weights for policy 1, policy_version 1415422 (0.0010) [2023-12-27 15:27:47,201][20347] Updated weights for policy 0, policy_version 1416655 (0.0005) [2023-12-27 15:27:47,245][20381] Updated weights for policy 1, policy_version 1415432 (0.0010) [2023-12-27 15:27:47,254][20347] Updated weights for policy 0, policy_version 1416665 (0.0007) [2023-12-27 15:27:47,303][20347] Updated weights for policy 0, policy_version 1416675 (0.0005) [2023-12-27 15:27:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 725123072. Throughput: 0: 10258.0, 1: 10248.6. Samples: 725126236. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:47,363][19170] Avg episode reward: [(0, '921.790'), (1, '942.570')] [2023-12-27 15:27:47,903][20347] Updated weights for policy 0, policy_version 1416685 (0.0006) [2023-12-27 15:27:47,906][20381] Updated weights for policy 1, policy_version 1415442 (0.0010) [2023-12-27 15:27:47,962][20347] Updated weights for policy 0, policy_version 1416695 (0.0006) [2023-12-27 15:27:47,968][20381] Updated weights for policy 1, policy_version 1415452 (0.0011) [2023-12-27 15:27:48,021][20347] Updated weights for policy 0, policy_version 1416705 (0.0006) [2023-12-27 15:27:48,023][20381] Updated weights for policy 1, policy_version 1415462 (0.0011) [2023-12-27 15:27:48,078][20381] Updated weights for policy 1, policy_version 1415472 (0.0011) [2023-12-27 15:27:48,691][20347] Updated weights for policy 0, policy_version 1416715 (0.0006) [2023-12-27 15:27:48,751][20347] Updated weights for policy 0, policy_version 1416725 (0.0007) [2023-12-27 15:27:48,758][20381] Updated weights for policy 1, policy_version 1415482 (0.0006) [2023-12-27 15:27:48,809][20347] Updated weights for policy 0, policy_version 1416735 (0.0007) [2023-12-27 15:27:48,811][20381] Updated weights for policy 1, policy_version 1415492 (0.0006) [2023-12-27 15:27:48,868][20381] Updated weights for policy 1, policy_version 1415502 (0.0007) [2023-12-27 15:27:49,547][20347] Updated weights for policy 0, policy_version 1416745 (0.0006) [2023-12-27 15:27:49,559][20381] Updated weights for policy 1, policy_version 1415512 (0.0009) [2023-12-27 15:27:49,606][20347] Updated weights for policy 0, policy_version 1416755 (0.0007) [2023-12-27 15:27:49,619][20381] Updated weights for policy 1, policy_version 1415522 (0.0011) [2023-12-27 15:27:49,668][20347] Updated weights for policy 0, policy_version 1416765 (0.0006) [2023-12-27 15:27:49,679][20381] Updated weights for policy 1, policy_version 1415532 (0.0011) [2023-12-27 15:27:49,734][20347] Updated weights for policy 0, policy_version 1416775 (0.0008) [2023-12-27 15:27:50,403][20347] Updated weights for policy 0, policy_version 1416785 (0.0007) [2023-12-27 15:27:50,407][20381] Updated weights for policy 1, policy_version 1415542 (0.0009) [2023-12-27 15:27:50,463][20347] Updated weights for policy 0, policy_version 1416795 (0.0007) [2023-12-27 15:27:50,468][20381] Updated weights for policy 1, policy_version 1415552 (0.0006) [2023-12-27 15:27:50,510][20347] Updated weights for policy 0, policy_version 1416805 (0.0005) [2023-12-27 15:27:50,519][20381] Updated weights for policy 1, policy_version 1415562 (0.0010) [2023-12-27 15:27:51,193][20381] Updated weights for policy 1, policy_version 1415572 (0.0010) [2023-12-27 15:27:51,212][20347] Updated weights for policy 0, policy_version 1416815 (0.0006) [2023-12-27 15:27:51,256][20381] Updated weights for policy 1, policy_version 1415582 (0.0009) [2023-12-27 15:27:51,277][20347] Updated weights for policy 0, policy_version 1416825 (0.0006) [2023-12-27 15:27:51,314][20381] Updated weights for policy 1, policy_version 1415592 (0.0009) [2023-12-27 15:27:51,325][20347] Updated weights for policy 0, policy_version 1416835 (0.0006) [2023-12-27 15:27:52,000][20381] Updated weights for policy 1, policy_version 1415602 (0.0008) [2023-12-27 15:27:52,059][20381] Updated weights for policy 1, policy_version 1415612 (0.0009) [2023-12-27 15:27:52,065][20347] Updated weights for policy 0, policy_version 1416845 (0.0006) [2023-12-27 15:27:52,115][20381] Updated weights for policy 1, policy_version 1415622 (0.0011) [2023-12-27 15:27:52,124][20347] Updated weights for policy 0, policy_version 1416855 (0.0005) [2023-12-27 15:27:52,174][20381] Updated weights for policy 1, policy_version 1415632 (0.0011) [2023-12-27 15:27:52,184][20347] Updated weights for policy 0, policy_version 1416865 (0.0005) [2023-12-27 15:27:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 725221376. Throughput: 0: 10273.5, 1: 10202.8. Samples: 725187368. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:52,363][19170] Avg episode reward: [(0, '960.840'), (1, '952.260')] [2023-12-27 15:27:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001415632_362430464.pth... [2023-12-27 15:27:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001416872_362790912.pth... [2023-12-27 15:27:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001414416_362119168.pth [2023-12-27 15:27:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001415656_362479616.pth [2023-12-27 15:27:52,851][20381] Updated weights for policy 1, policy_version 1415642 (0.0009) [2023-12-27 15:27:52,901][20347] Updated weights for policy 0, policy_version 1416875 (0.0007) [2023-12-27 15:27:52,911][20381] Updated weights for policy 1, policy_version 1415652 (0.0007) [2023-12-27 15:27:52,958][20347] Updated weights for policy 0, policy_version 1416885 (0.0006) [2023-12-27 15:27:52,964][20381] Updated weights for policy 1, policy_version 1415662 (0.0005) [2023-12-27 15:27:53,005][20347] Updated weights for policy 0, policy_version 1416895 (0.0005) [2023-12-27 15:27:53,559][20381] Updated weights for policy 1, policy_version 1415672 (0.0007) [2023-12-27 15:27:53,605][20381] Updated weights for policy 1, policy_version 1415682 (0.0007) [2023-12-27 15:27:53,659][20381] Updated weights for policy 1, policy_version 1415692 (0.0007) [2023-12-27 15:27:53,711][20347] Updated weights for policy 0, policy_version 1416905 (0.0006) [2023-12-27 15:27:53,766][20347] Updated weights for policy 0, policy_version 1416915 (0.0007) [2023-12-27 15:27:53,822][20347] Updated weights for policy 0, policy_version 1416925 (0.0008) [2023-12-27 15:27:53,875][20347] Updated weights for policy 0, policy_version 1416935 (0.0007) [2023-12-27 15:27:54,383][20381] Updated weights for policy 1, policy_version 1415702 (0.0008) [2023-12-27 15:27:54,430][20381] Updated weights for policy 1, policy_version 1415712 (0.0008) [2023-12-27 15:27:54,478][20381] Updated weights for policy 1, policy_version 1415722 (0.0005) [2023-12-27 15:27:54,483][20347] Updated weights for policy 0, policy_version 1416945 (0.0006) [2023-12-27 15:27:54,540][20347] Updated weights for policy 0, policy_version 1416955 (0.0006) [2023-12-27 15:27:54,600][20347] Updated weights for policy 0, policy_version 1416965 (0.0007) [2023-12-27 15:27:55,149][20347] Updated weights for policy 0, policy_version 1416975 (0.0006) [2023-12-27 15:27:55,208][20347] Updated weights for policy 0, policy_version 1416985 (0.0005) [2023-12-27 15:27:55,240][20381] Updated weights for policy 1, policy_version 1415732 (0.0006) [2023-12-27 15:27:55,263][20347] Updated weights for policy 0, policy_version 1416995 (0.0005) [2023-12-27 15:27:55,288][20381] Updated weights for policy 1, policy_version 1415742 (0.0007) [2023-12-27 15:27:55,338][20381] Updated weights for policy 1, policy_version 1415752 (0.0008) [2023-12-27 15:27:55,886][20347] Updated weights for policy 0, policy_version 1417005 (0.0006) [2023-12-27 15:27:55,935][20347] Updated weights for policy 0, policy_version 1417015 (0.0005) [2023-12-27 15:27:55,968][20381] Updated weights for policy 1, policy_version 1415762 (0.0009) [2023-12-27 15:27:55,988][20347] Updated weights for policy 0, policy_version 1417025 (0.0005) [2023-12-27 15:27:56,013][20381] Updated weights for policy 1, policy_version 1415772 (0.0006) [2023-12-27 15:27:56,060][20381] Updated weights for policy 1, policy_version 1415782 (0.0005) [2023-12-27 15:27:56,114][20381] Updated weights for policy 1, policy_version 1415792 (0.0005) [2023-12-27 15:27:56,682][20347] Updated weights for policy 0, policy_version 1417035 (0.0007) [2023-12-27 15:27:56,726][20381] Updated weights for policy 1, policy_version 1415802 (0.0010) [2023-12-27 15:27:56,736][20347] Updated weights for policy 0, policy_version 1417045 (0.0008) [2023-12-27 15:27:56,776][20381] Updated weights for policy 1, policy_version 1415812 (0.0007) [2023-12-27 15:27:56,789][20347] Updated weights for policy 0, policy_version 1417055 (0.0006) [2023-12-27 15:27:56,832][20381] Updated weights for policy 1, policy_version 1415822 (0.0005) [2023-12-27 15:27:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 725319680. Throughput: 0: 10296.2, 1: 10242.4. Samples: 725311328. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:27:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.230')] [2023-12-27 15:27:57,434][20381] Updated weights for policy 1, policy_version 1415832 (0.0009) [2023-12-27 15:27:57,458][20347] Updated weights for policy 0, policy_version 1417065 (0.0007) [2023-12-27 15:27:57,487][20381] Updated weights for policy 1, policy_version 1415842 (0.0008) [2023-12-27 15:27:57,505][20347] Updated weights for policy 0, policy_version 1417075 (0.0007) [2023-12-27 15:27:57,543][20381] Updated weights for policy 1, policy_version 1415852 (0.0005) [2023-12-27 15:27:57,553][20347] Updated weights for policy 0, policy_version 1417085 (0.0007) [2023-12-27 15:27:57,602][20347] Updated weights for policy 0, policy_version 1417095 (0.0007) [2023-12-27 15:27:58,094][20381] Updated weights for policy 1, policy_version 1415862 (0.0008) [2023-12-27 15:27:58,143][20381] Updated weights for policy 1, policy_version 1415872 (0.0011) [2023-12-27 15:27:58,198][20381] Updated weights for policy 1, policy_version 1415882 (0.0011) [2023-12-27 15:27:58,199][20347] Updated weights for policy 0, policy_version 1417105 (0.0008) [2023-12-27 15:27:58,269][20347] Updated weights for policy 0, policy_version 1417115 (0.0006) [2023-12-27 15:27:58,326][20347] Updated weights for policy 0, policy_version 1417125 (0.0010) [2023-12-27 15:27:59,026][20381] Updated weights for policy 1, policy_version 1415892 (0.0008) [2023-12-27 15:27:59,081][20381] Updated weights for policy 1, policy_version 1415902 (0.0008) [2023-12-27 15:27:59,103][20347] Updated weights for policy 0, policy_version 1417135 (0.0009) [2023-12-27 15:27:59,136][20381] Updated weights for policy 1, policy_version 1415912 (0.0010) [2023-12-27 15:27:59,157][20347] Updated weights for policy 0, policy_version 1417145 (0.0007) [2023-12-27 15:27:59,212][20347] Updated weights for policy 0, policy_version 1417155 (0.0007) [2023-12-27 15:27:59,862][20381] Updated weights for policy 1, policy_version 1415922 (0.0010) [2023-12-27 15:27:59,871][20347] Updated weights for policy 0, policy_version 1417165 (0.0008) [2023-12-27 15:27:59,924][20347] Updated weights for policy 0, policy_version 1417175 (0.0008) [2023-12-27 15:27:59,924][20381] Updated weights for policy 1, policy_version 1415932 (0.0008) [2023-12-27 15:27:59,984][20381] Updated weights for policy 1, policy_version 1415942 (0.0006) [2023-12-27 15:27:59,987][20347] Updated weights for policy 0, policy_version 1417185 (0.0008) [2023-12-27 15:28:00,043][20381] Updated weights for policy 1, policy_version 1415952 (0.0007) [2023-12-27 15:28:00,646][20347] Updated weights for policy 0, policy_version 1417195 (0.0007) [2023-12-27 15:28:00,690][20381] Updated weights for policy 1, policy_version 1415962 (0.0005) [2023-12-27 15:28:00,700][20347] Updated weights for policy 0, policy_version 1417205 (0.0005) [2023-12-27 15:28:00,734][20381] Updated weights for policy 1, policy_version 1415972 (0.0006) [2023-12-27 15:28:00,758][20347] Updated weights for policy 0, policy_version 1417215 (0.0005) [2023-12-27 15:28:00,786][20381] Updated weights for policy 1, policy_version 1415982 (0.0007) [2023-12-27 15:28:01,356][20347] Updated weights for policy 0, policy_version 1417225 (0.0006) [2023-12-27 15:28:01,422][20347] Updated weights for policy 0, policy_version 1417235 (0.0007) [2023-12-27 15:28:01,480][20347] Updated weights for policy 0, policy_version 1417245 (0.0007) [2023-12-27 15:28:01,510][20381] Updated weights for policy 1, policy_version 1415992 (0.0008) [2023-12-27 15:28:01,540][20347] Updated weights for policy 0, policy_version 1417255 (0.0006) [2023-12-27 15:28:01,568][20381] Updated weights for policy 1, policy_version 1416002 (0.0008) [2023-12-27 15:28:01,624][20381] Updated weights for policy 1, policy_version 1416012 (0.0008) [2023-12-27 15:28:02,226][20347] Updated weights for policy 0, policy_version 1417265 (0.0007) [2023-12-27 15:28:02,274][20381] Updated weights for policy 1, policy_version 1416022 (0.0008) [2023-12-27 15:28:02,284][20347] Updated weights for policy 0, policy_version 1417275 (0.0008) [2023-12-27 15:28:02,331][20381] Updated weights for policy 1, policy_version 1416032 (0.0005) [2023-12-27 15:28:02,343][20347] Updated weights for policy 0, policy_version 1417285 (0.0007) [2023-12-27 15:28:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 725417984. Throughput: 0: 10318.7, 1: 10294.5. Samples: 725437088. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 15:28:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '1000.000')] [2023-12-27 15:28:02,390][20381] Updated weights for policy 1, policy_version 1416042 (0.0006) [2023-12-27 15:28:03,008][20381] Updated weights for policy 1, policy_version 1416052 (0.0005) [2023-12-27 15:28:03,032][20347] Updated weights for policy 0, policy_version 1417295 (0.0007) [2023-12-27 15:28:03,056][20381] Updated weights for policy 1, policy_version 1416062 (0.0005) [2023-12-27 15:28:03,087][20347] Updated weights for policy 0, policy_version 1417305 (0.0007) [2023-12-27 15:28:03,109][20381] Updated weights for policy 1, policy_version 1416072 (0.0005) [2023-12-27 15:28:03,143][20347] Updated weights for policy 0, policy_version 1417315 (0.0007) [2023-12-27 15:28:03,712][20381] Updated weights for policy 1, policy_version 1416082 (0.0005) [2023-12-27 15:28:03,762][20381] Updated weights for policy 1, policy_version 1416092 (0.0006) [2023-12-27 15:28:03,814][20381] Updated weights for policy 1, policy_version 1416102 (0.0008) [2023-12-27 15:28:03,861][20347] Updated weights for policy 0, policy_version 1417325 (0.0007) [2023-12-27 15:28:03,873][20381] Updated weights for policy 1, policy_version 1416112 (0.0009) [2023-12-27 15:28:03,923][20347] Updated weights for policy 0, policy_version 1417335 (0.0007) [2023-12-27 15:28:03,978][20347] Updated weights for policy 0, policy_version 1417345 (0.0008) [2023-12-27 15:28:04,557][20381] Updated weights for policy 1, policy_version 1416122 (0.0005) [2023-12-27 15:28:04,612][20381] Updated weights for policy 1, policy_version 1416132 (0.0008) [2023-12-27 15:28:04,661][20381] Updated weights for policy 1, policy_version 1416142 (0.0008) [2023-12-27 15:28:04,698][20347] Updated weights for policy 0, policy_version 1417355 (0.0008) [2023-12-27 15:28:04,749][20347] Updated weights for policy 0, policy_version 1417365 (0.0007) [2023-12-27 15:28:04,798][20347] Updated weights for policy 0, policy_version 1417375 (0.0005) [2023-12-27 15:28:05,266][20381] Updated weights for policy 1, policy_version 1416152 (0.0006) [2023-12-27 15:28:05,321][20381] Updated weights for policy 1, policy_version 1416162 (0.0005) [2023-12-27 15:28:05,371][20381] Updated weights for policy 1, policy_version 1416172 (0.0005) [2023-12-27 15:28:05,540][20347] Updated weights for policy 0, policy_version 1417385 (0.0007) [2023-12-27 15:28:05,602][20347] Updated weights for policy 0, policy_version 1417395 (0.0008) [2023-12-27 15:28:05,657][20347] Updated weights for policy 0, policy_version 1417405 (0.0007) [2023-12-27 15:28:05,708][20347] Updated weights for policy 0, policy_version 1417415 (0.0007) [2023-12-27 15:28:05,986][20381] Updated weights for policy 1, policy_version 1416182 (0.0005) [2023-12-27 15:28:06,034][20381] Updated weights for policy 1, policy_version 1416192 (0.0005) [2023-12-27 15:28:06,090][20381] Updated weights for policy 1, policy_version 1416202 (0.0005) [2023-12-27 15:28:06,445][20347] Updated weights for policy 0, policy_version 1417425 (0.0006) [2023-12-27 15:28:06,501][20291] KL-divergence is very high: 106.7146 [2023-12-27 15:28:06,508][20291] KL-divergence is very high: 139.8370 [2023-12-27 15:28:06,514][20347] Updated weights for policy 0, policy_version 1417435 (0.0005) [2023-12-27 15:28:06,528][20291] KL-divergence is very high: 102.2892 [2023-12-27 15:28:06,555][20291] KL-divergence is very high: 145.1568 [2023-12-27 15:28:06,561][20291] KL-divergence is very high: 148.9460 [2023-12-27 15:28:06,576][20347] Updated weights for policy 0, policy_version 1417445 (0.0007) [2023-12-27 15:28:06,664][20381] Updated weights for policy 1, policy_version 1416212 (0.0007) [2023-12-27 15:28:06,728][20381] Updated weights for policy 1, policy_version 1416222 (0.0007) [2023-12-27 15:28:06,788][20381] Updated weights for policy 1, policy_version 1416232 (0.0008) [2023-12-27 15:28:07,209][20347] Updated weights for policy 0, policy_version 1417455 (0.0007) [2023-12-27 15:28:07,259][20347] Updated weights for policy 0, policy_version 1417465 (0.0008) [2023-12-27 15:28:07,317][20347] Updated weights for policy 0, policy_version 1417475 (0.0008) [2023-12-27 15:28:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 725532672. Throughput: 0: 10265.1, 1: 10380.7. Samples: 725499380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:07,363][19170] Avg episode reward: [(0, '941.530'), (1, '1000.000')] [2023-12-27 15:28:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001416240_362586112.pth... [2023-12-27 15:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001417480_362946560.pth... [2023-12-27 15:28:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001415024_362274816.pth [2023-12-27 15:28:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001416264_362635264.pth [2023-12-27 15:28:07,448][20381] Updated weights for policy 1, policy_version 1416242 (0.0008) [2023-12-27 15:28:07,509][20381] Updated weights for policy 1, policy_version 1416252 (0.0005) [2023-12-27 15:28:07,559][20381] Updated weights for policy 1, policy_version 1416262 (0.0005) [2023-12-27 15:28:07,609][20381] Updated weights for policy 1, policy_version 1416272 (0.0005) [2023-12-27 15:28:08,086][20347] Updated weights for policy 0, policy_version 1417485 (0.0007) [2023-12-27 15:28:08,137][20347] Updated weights for policy 0, policy_version 1417495 (0.0007) [2023-12-27 15:28:08,186][20347] Updated weights for policy 0, policy_version 1417505 (0.0007) [2023-12-27 15:28:08,187][20381] Updated weights for policy 1, policy_version 1416282 (0.0006) [2023-12-27 15:28:08,235][20381] Updated weights for policy 1, policy_version 1416292 (0.0007) [2023-12-27 15:28:08,281][20381] Updated weights for policy 1, policy_version 1416302 (0.0008) [2023-12-27 15:28:08,835][20347] Updated weights for policy 0, policy_version 1417515 (0.0007) [2023-12-27 15:28:08,897][20347] Updated weights for policy 0, policy_version 1417525 (0.0006) [2023-12-27 15:28:08,952][20347] Updated weights for policy 0, policy_version 1417535 (0.0006) [2023-12-27 15:28:09,074][20381] Updated weights for policy 1, policy_version 1416312 (0.0006) [2023-12-27 15:28:09,129][20381] Updated weights for policy 1, policy_version 1416322 (0.0005) [2023-12-27 15:28:09,195][20381] Updated weights for policy 1, policy_version 1416332 (0.0006) [2023-12-27 15:28:09,643][20347] Updated weights for policy 0, policy_version 1417545 (0.0006) [2023-12-27 15:28:09,703][20347] Updated weights for policy 0, policy_version 1417555 (0.0008) [2023-12-27 15:28:09,760][20347] Updated weights for policy 0, policy_version 1417565 (0.0008) [2023-12-27 15:28:09,808][20347] Updated weights for policy 0, policy_version 1417575 (0.0007) [2023-12-27 15:28:09,850][20381] Updated weights for policy 1, policy_version 1416342 (0.0008) [2023-12-27 15:28:09,899][20381] Updated weights for policy 1, policy_version 1416352 (0.0007) [2023-12-27 15:28:09,959][20381] Updated weights for policy 1, policy_version 1416362 (0.0008) [2023-12-27 15:28:10,530][20347] Updated weights for policy 0, policy_version 1417585 (0.0008) [2023-12-27 15:28:10,580][20347] Updated weights for policy 0, policy_version 1417595 (0.0009) [2023-12-27 15:28:10,632][20347] Updated weights for policy 0, policy_version 1417605 (0.0008) [2023-12-27 15:28:10,635][20381] Updated weights for policy 1, policy_version 1416372 (0.0008) [2023-12-27 15:28:10,691][20381] Updated weights for policy 1, policy_version 1416382 (0.0008) [2023-12-27 15:28:10,748][20381] Updated weights for policy 1, policy_version 1416392 (0.0008) [2023-12-27 15:28:11,412][20347] Updated weights for policy 0, policy_version 1417615 (0.0006) [2023-12-27 15:28:11,462][20347] Updated weights for policy 0, policy_version 1417625 (0.0005) [2023-12-27 15:28:11,511][20381] Updated weights for policy 1, policy_version 1416402 (0.0007) [2023-12-27 15:28:11,525][20347] Updated weights for policy 0, policy_version 1417635 (0.0007) [2023-12-27 15:28:11,569][20381] Updated weights for policy 1, policy_version 1416412 (0.0007) [2023-12-27 15:28:11,635][20381] Updated weights for policy 1, policy_version 1416422 (0.0007) [2023-12-27 15:28:11,698][20381] Updated weights for policy 1, policy_version 1416432 (0.0006) [2023-12-27 15:28:12,278][20347] Updated weights for policy 0, policy_version 1417645 (0.0007) [2023-12-27 15:28:12,339][20347] Updated weights for policy 0, policy_version 1417655 (0.0007) [2023-12-27 15:28:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 725622784. Throughput: 0: 10196.5, 1: 10451.5. Samples: 725621308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:12,363][19170] Avg episode reward: [(0, '951.280'), (1, '1000.000')] [2023-12-27 15:28:12,404][20347] Updated weights for policy 0, policy_version 1417665 (0.0007) [2023-12-27 15:28:12,413][20381] Updated weights for policy 1, policy_version 1416442 (0.0009) [2023-12-27 15:28:12,460][20381] Updated weights for policy 1, policy_version 1416452 (0.0008) [2023-12-27 15:28:12,513][20381] Updated weights for policy 1, policy_version 1416462 (0.0007) [2023-12-27 15:28:13,021][20347] Updated weights for policy 0, policy_version 1417675 (0.0006) [2023-12-27 15:28:13,072][20347] Updated weights for policy 0, policy_version 1417685 (0.0005) [2023-12-27 15:28:13,133][20347] Updated weights for policy 0, policy_version 1417695 (0.0005) [2023-12-27 15:28:13,269][20381] Updated weights for policy 1, policy_version 1416472 (0.0008) [2023-12-27 15:28:13,313][20381] Updated weights for policy 1, policy_version 1416482 (0.0008) [2023-12-27 15:28:13,365][20381] Updated weights for policy 1, policy_version 1416492 (0.0008) [2023-12-27 15:28:13,685][20347] Updated weights for policy 0, policy_version 1417705 (0.0005) [2023-12-27 15:28:13,750][20347] Updated weights for policy 0, policy_version 1417715 (0.0005) [2023-12-27 15:28:13,816][20347] Updated weights for policy 0, policy_version 1417725 (0.0005) [2023-12-27 15:28:13,875][20347] Updated weights for policy 0, policy_version 1417735 (0.0005) [2023-12-27 15:28:14,122][20381] Updated weights for policy 1, policy_version 1416502 (0.0008) [2023-12-27 15:28:14,163][20381] Updated weights for policy 1, policy_version 1416512 (0.0008) [2023-12-27 15:28:14,213][20381] Updated weights for policy 1, policy_version 1416522 (0.0007) [2023-12-27 15:28:14,493][20347] Updated weights for policy 0, policy_version 1417745 (0.0006) [2023-12-27 15:28:14,554][20347] Updated weights for policy 0, policy_version 1417755 (0.0007) [2023-12-27 15:28:14,608][20347] Updated weights for policy 0, policy_version 1417765 (0.0006) [2023-12-27 15:28:14,970][20381] Updated weights for policy 1, policy_version 1416532 (0.0007) [2023-12-27 15:28:15,027][20381] Updated weights for policy 1, policy_version 1416542 (0.0007) [2023-12-27 15:28:15,087][20381] Updated weights for policy 1, policy_version 1416552 (0.0006) [2023-12-27 15:28:15,260][20347] Updated weights for policy 0, policy_version 1417775 (0.0006) [2023-12-27 15:28:15,317][20347] Updated weights for policy 0, policy_version 1417785 (0.0007) [2023-12-27 15:28:15,375][20347] Updated weights for policy 0, policy_version 1417795 (0.0007) [2023-12-27 15:28:15,770][20381] Updated weights for policy 1, policy_version 1416562 (0.0007) [2023-12-27 15:28:15,824][20381] Updated weights for policy 1, policy_version 1416572 (0.0007) [2023-12-27 15:28:15,875][20381] Updated weights for policy 1, policy_version 1416582 (0.0007) [2023-12-27 15:28:15,925][20381] Updated weights for policy 1, policy_version 1416592 (0.0007) [2023-12-27 15:28:16,061][20347] Updated weights for policy 0, policy_version 1417805 (0.0009) [2023-12-27 15:28:16,113][20347] Updated weights for policy 0, policy_version 1417815 (0.0010) [2023-12-27 15:28:16,168][20347] Updated weights for policy 0, policy_version 1417825 (0.0010) [2023-12-27 15:28:16,568][20381] Updated weights for policy 1, policy_version 1416602 (0.0007) [2023-12-27 15:28:16,622][20381] Updated weights for policy 1, policy_version 1416612 (0.0007) [2023-12-27 15:28:16,679][20381] Updated weights for policy 1, policy_version 1416622 (0.0007) [2023-12-27 15:28:16,848][20347] Updated weights for policy 0, policy_version 1417835 (0.0010) [2023-12-27 15:28:16,899][20347] Updated weights for policy 0, policy_version 1417845 (0.0010) [2023-12-27 15:28:16,954][20347] Updated weights for policy 0, policy_version 1417855 (0.0010) [2023-12-27 15:28:17,293][20381] Updated weights for policy 1, policy_version 1416632 (0.0008) [2023-12-27 15:28:17,340][20381] Updated weights for policy 1, policy_version 1416642 (0.0008) [2023-12-27 15:28:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 725729280. Throughput: 0: 10195.0, 1: 10346.0. Samples: 725742888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 15:28:17,385][20381] Updated weights for policy 1, policy_version 1416652 (0.0007) [2023-12-27 15:28:17,599][20347] Updated weights for policy 0, policy_version 1417865 (0.0010) [2023-12-27 15:28:17,650][20347] Updated weights for policy 0, policy_version 1417875 (0.0010) [2023-12-27 15:28:17,695][20347] Updated weights for policy 0, policy_version 1417885 (0.0010) [2023-12-27 15:28:17,753][20347] Updated weights for policy 0, policy_version 1417895 (0.0011) [2023-12-27 15:28:18,113][20381] Updated weights for policy 1, policy_version 1416662 (0.0008) [2023-12-27 15:28:18,165][20381] Updated weights for policy 1, policy_version 1416672 (0.0008) [2023-12-27 15:28:18,221][20381] Updated weights for policy 1, policy_version 1416682 (0.0008) [2023-12-27 15:28:18,443][20347] Updated weights for policy 0, policy_version 1417905 (0.0009) [2023-12-27 15:28:18,498][20347] Updated weights for policy 0, policy_version 1417915 (0.0009) [2023-12-27 15:28:18,557][20347] Updated weights for policy 0, policy_version 1417925 (0.0009) [2023-12-27 15:28:18,909][20381] Updated weights for policy 1, policy_version 1416692 (0.0007) [2023-12-27 15:28:18,960][20381] Updated weights for policy 1, policy_version 1416702 (0.0007) [2023-12-27 15:28:19,010][20381] Updated weights for policy 1, policy_version 1416712 (0.0007) [2023-12-27 15:28:19,319][20347] Updated weights for policy 0, policy_version 1417935 (0.0008) [2023-12-27 15:28:19,384][20347] Updated weights for policy 0, policy_version 1417945 (0.0006) [2023-12-27 15:28:19,434][20347] Updated weights for policy 0, policy_version 1417955 (0.0008) [2023-12-27 15:28:19,762][20381] Updated weights for policy 1, policy_version 1416722 (0.0008) [2023-12-27 15:28:19,818][20381] Updated weights for policy 1, policy_version 1416732 (0.0009) [2023-12-27 15:28:19,886][20381] Updated weights for policy 1, policy_version 1416742 (0.0008) [2023-12-27 15:28:19,950][20381] Updated weights for policy 1, policy_version 1416752 (0.0008) [2023-12-27 15:28:20,101][20347] Updated weights for policy 0, policy_version 1417965 (0.0007) [2023-12-27 15:28:20,161][20347] Updated weights for policy 0, policy_version 1417975 (0.0008) [2023-12-27 15:28:20,222][20347] Updated weights for policy 0, policy_version 1417985 (0.0007) [2023-12-27 15:28:20,653][20381] Updated weights for policy 1, policy_version 1416762 (0.0007) [2023-12-27 15:28:20,711][20381] Updated weights for policy 1, policy_version 1416772 (0.0008) [2023-12-27 15:28:20,773][20381] Updated weights for policy 1, policy_version 1416782 (0.0008) [2023-12-27 15:28:20,938][20347] Updated weights for policy 0, policy_version 1417995 (0.0007) [2023-12-27 15:28:20,994][20347] Updated weights for policy 0, policy_version 1418005 (0.0005) [2023-12-27 15:28:21,060][20347] Updated weights for policy 0, policy_version 1418015 (0.0008) [2023-12-27 15:28:21,450][20381] Updated weights for policy 1, policy_version 1416792 (0.0007) [2023-12-27 15:28:21,507][20381] Updated weights for policy 1, policy_version 1416802 (0.0008) [2023-12-27 15:28:21,570][20381] Updated weights for policy 1, policy_version 1416812 (0.0006) [2023-12-27 15:28:21,827][20347] Updated weights for policy 0, policy_version 1418025 (0.0007) [2023-12-27 15:28:21,889][20347] Updated weights for policy 0, policy_version 1418035 (0.0008) [2023-12-27 15:28:21,955][20347] Updated weights for policy 0, policy_version 1418045 (0.0006) [2023-12-27 15:28:22,015][20347] Updated weights for policy 0, policy_version 1418055 (0.0007) [2023-12-27 15:28:22,324][20381] Updated weights for policy 1, policy_version 1416822 (0.0007) [2023-12-27 15:28:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 725827584. Throughput: 0: 10197.7, 1: 10261.4. Samples: 725802296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:22,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 15:28:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001418056_363094016.pth... [2023-12-27 15:28:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001416872_362790912.pth [2023-12-27 15:28:22,385][20381] Updated weights for policy 1, policy_version 1416832 (0.0008) [2023-12-27 15:28:22,444][20381] Updated weights for policy 1, policy_version 1416842 (0.0009) [2023-12-27 15:28:22,479][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001416848_362741760.pth... [2023-12-27 15:28:22,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001415632_362430464.pth [2023-12-27 15:28:22,713][20347] Updated weights for policy 0, policy_version 1418065 (0.0008) [2023-12-27 15:28:22,775][20347] Updated weights for policy 0, policy_version 1418075 (0.0007) [2023-12-27 15:28:22,836][20347] Updated weights for policy 0, policy_version 1418085 (0.0007) [2023-12-27 15:28:23,126][20381] Updated weights for policy 1, policy_version 1416852 (0.0006) [2023-12-27 15:28:23,179][20381] Updated weights for policy 1, policy_version 1416862 (0.0007) [2023-12-27 15:28:23,237][20381] Updated weights for policy 1, policy_version 1416872 (0.0007) [2023-12-27 15:28:23,504][20347] Updated weights for policy 0, policy_version 1418095 (0.0008) [2023-12-27 15:28:23,555][20347] Updated weights for policy 0, policy_version 1418105 (0.0007) [2023-12-27 15:28:23,603][20347] Updated weights for policy 0, policy_version 1418115 (0.0007) [2023-12-27 15:28:23,951][20381] Updated weights for policy 1, policy_version 1416882 (0.0008) [2023-12-27 15:28:24,004][20381] Updated weights for policy 1, policy_version 1416892 (0.0008) [2023-12-27 15:28:24,057][20381] Updated weights for policy 1, policy_version 1416902 (0.0008) [2023-12-27 15:28:24,110][20381] Updated weights for policy 1, policy_version 1416912 (0.0008) [2023-12-27 15:28:24,274][20347] Updated weights for policy 0, policy_version 1418125 (0.0007) [2023-12-27 15:28:24,330][20347] Updated weights for policy 0, policy_version 1418135 (0.0008) [2023-12-27 15:28:24,387][20347] Updated weights for policy 0, policy_version 1418145 (0.0007) [2023-12-27 15:28:24,778][20381] Updated weights for policy 1, policy_version 1416922 (0.0008) [2023-12-27 15:28:24,838][20381] Updated weights for policy 1, policy_version 1416932 (0.0008) [2023-12-27 15:28:24,894][20381] Updated weights for policy 1, policy_version 1416942 (0.0008) [2023-12-27 15:28:25,092][20347] Updated weights for policy 0, policy_version 1418155 (0.0007) [2023-12-27 15:28:25,150][20347] Updated weights for policy 0, policy_version 1418165 (0.0008) [2023-12-27 15:28:25,205][20347] Updated weights for policy 0, policy_version 1418175 (0.0007) [2023-12-27 15:28:25,588][20381] Updated weights for policy 1, policy_version 1416952 (0.0008) [2023-12-27 15:28:25,644][20381] Updated weights for policy 1, policy_version 1416962 (0.0007) [2023-12-27 15:28:25,689][20381] Updated weights for policy 1, policy_version 1416972 (0.0007) [2023-12-27 15:28:25,893][20347] Updated weights for policy 0, policy_version 1418185 (0.0007) [2023-12-27 15:28:25,947][20347] Updated weights for policy 0, policy_version 1418195 (0.0005) [2023-12-27 15:28:25,953][20291] KL-divergence is very high: 123.0248 [2023-12-27 15:28:26,002][20291] KL-divergence is very high: 179.9090 [2023-12-27 15:28:26,008][20347] Updated weights for policy 0, policy_version 1418205 (0.0006) [2023-12-27 15:28:26,044][20291] KL-divergence is very high: 124.9582 [2023-12-27 15:28:26,060][20347] Updated weights for policy 0, policy_version 1418215 (0.0007) [2023-12-27 15:28:26,381][20381] Updated weights for policy 1, policy_version 1416982 (0.0006) [2023-12-27 15:28:26,431][20381] Updated weights for policy 1, policy_version 1416992 (0.0007) [2023-12-27 15:28:26,486][20381] Updated weights for policy 1, policy_version 1417002 (0.0008) [2023-12-27 15:28:26,687][20347] Updated weights for policy 0, policy_version 1418225 (0.0006) [2023-12-27 15:28:26,741][20347] Updated weights for policy 0, policy_version 1418235 (0.0009) [2023-12-27 15:28:26,796][20347] Updated weights for policy 0, policy_version 1418245 (0.0007) [2023-12-27 15:28:27,167][20381] Updated weights for policy 1, policy_version 1417012 (0.0007) [2023-12-27 15:28:27,216][20381] Updated weights for policy 1, policy_version 1417022 (0.0008) [2023-12-27 15:28:27,262][20381] Updated weights for policy 1, policy_version 1417032 (0.0008) [2023-12-27 15:28:27,355][20347] Updated weights for policy 0, policy_version 1418255 (0.0006) [2023-12-27 15:28:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 725934080. Throughput: 0: 10136.3, 1: 10231.9. Samples: 725921352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:27,363][19170] Avg episode reward: [(0, '980.440'), (1, '990.260')] [2023-12-27 15:28:27,413][20347] Updated weights for policy 0, policy_version 1418265 (0.0006) [2023-12-27 15:28:27,470][20347] Updated weights for policy 0, policy_version 1418275 (0.0005) [2023-12-27 15:28:27,979][20347] Updated weights for policy 0, policy_version 1418285 (0.0005) [2023-12-27 15:28:28,030][20381] Updated weights for policy 1, policy_version 1417042 (0.0007) [2023-12-27 15:28:28,031][20347] Updated weights for policy 0, policy_version 1418295 (0.0006) [2023-12-27 15:28:28,083][20347] Updated weights for policy 0, policy_version 1418305 (0.0007) [2023-12-27 15:28:28,088][20381] Updated weights for policy 1, policy_version 1417052 (0.0006) [2023-12-27 15:28:28,137][20381] Updated weights for policy 1, policy_version 1417062 (0.0006) [2023-12-27 15:28:28,187][20381] Updated weights for policy 1, policy_version 1417072 (0.0007) [2023-12-27 15:28:28,734][20347] Updated weights for policy 0, policy_version 1418315 (0.0006) [2023-12-27 15:28:28,784][20347] Updated weights for policy 0, policy_version 1418325 (0.0006) [2023-12-27 15:28:28,837][20347] Updated weights for policy 0, policy_version 1418335 (0.0005) [2023-12-27 15:28:28,901][20381] Updated weights for policy 1, policy_version 1417082 (0.0006) [2023-12-27 15:28:28,950][20381] Updated weights for policy 1, policy_version 1417092 (0.0006) [2023-12-27 15:28:29,008][20381] Updated weights for policy 1, policy_version 1417102 (0.0008) [2023-12-27 15:28:29,507][20347] Updated weights for policy 0, policy_version 1418345 (0.0007) [2023-12-27 15:28:29,552][20291] KL-divergence is very high: 108.6208 [2023-12-27 15:28:29,570][20347] Updated weights for policy 0, policy_version 1418355 (0.0008) [2023-12-27 15:28:29,599][20291] KL-divergence is very high: 105.7061 [2023-12-27 15:28:29,628][20347] Updated weights for policy 0, policy_version 1418365 (0.0007) [2023-12-27 15:28:29,684][20381] Updated weights for policy 1, policy_version 1417112 (0.0006) [2023-12-27 15:28:29,689][20347] Updated weights for policy 0, policy_version 1418375 (0.0006) [2023-12-27 15:28:29,746][20381] Updated weights for policy 1, policy_version 1417122 (0.0006) [2023-12-27 15:28:29,812][20381] Updated weights for policy 1, policy_version 1417132 (0.0009) [2023-12-27 15:28:30,343][20347] Updated weights for policy 0, policy_version 1418385 (0.0006) [2023-12-27 15:28:30,403][20347] Updated weights for policy 0, policy_version 1418395 (0.0007) [2023-12-27 15:28:30,459][20347] Updated weights for policy 0, policy_version 1418405 (0.0008) [2023-12-27 15:28:30,547][20381] Updated weights for policy 1, policy_version 1417142 (0.0008) [2023-12-27 15:28:30,607][20381] Updated weights for policy 1, policy_version 1417152 (0.0007) [2023-12-27 15:28:30,662][20381] Updated weights for policy 1, policy_version 1417162 (0.0008) [2023-12-27 15:28:31,110][20347] Updated weights for policy 0, policy_version 1418415 (0.0008) [2023-12-27 15:28:31,176][20347] Updated weights for policy 0, policy_version 1418425 (0.0008) [2023-12-27 15:28:31,243][20347] Updated weights for policy 0, policy_version 1418435 (0.0006) [2023-12-27 15:28:31,411][20381] Updated weights for policy 1, policy_version 1417172 (0.0008) [2023-12-27 15:28:31,471][20381] Updated weights for policy 1, policy_version 1417182 (0.0009) [2023-12-27 15:28:31,524][20381] Updated weights for policy 1, policy_version 1417192 (0.0008) [2023-12-27 15:28:31,935][20347] Updated weights for policy 0, policy_version 1418445 (0.0007) [2023-12-27 15:28:31,998][20347] Updated weights for policy 0, policy_version 1418455 (0.0008) [2023-12-27 15:28:32,057][20347] Updated weights for policy 0, policy_version 1418465 (0.0008) [2023-12-27 15:28:32,204][20381] Updated weights for policy 1, policy_version 1417202 (0.0007) [2023-12-27 15:28:32,265][20381] Updated weights for policy 1, policy_version 1417212 (0.0007) [2023-12-27 15:28:32,318][20381] Updated weights for policy 1, policy_version 1417222 (0.0007) [2023-12-27 15:28:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.7, 300 sec: 20521.7). Total num frames: 726032384. Throughput: 0: 10245.2, 1: 10187.3. Samples: 726045696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:32,363][19170] Avg episode reward: [(0, '951.070'), (1, '990.260')] [2023-12-27 15:28:32,386][20381] Updated weights for policy 1, policy_version 1417232 (0.0008) [2023-12-27 15:28:32,815][20347] Updated weights for policy 0, policy_version 1418475 (0.0007) [2023-12-27 15:28:32,883][20347] Updated weights for policy 0, policy_version 1418485 (0.0008) [2023-12-27 15:28:32,937][20347] Updated weights for policy 0, policy_version 1418495 (0.0008) [2023-12-27 15:28:32,953][20381] Updated weights for policy 1, policy_version 1417242 (0.0007) [2023-12-27 15:28:33,010][20381] Updated weights for policy 1, policy_version 1417252 (0.0007) [2023-12-27 15:28:33,070][20381] Updated weights for policy 1, policy_version 1417262 (0.0007) [2023-12-27 15:28:33,541][20347] Updated weights for policy 0, policy_version 1418505 (0.0005) [2023-12-27 15:28:33,592][20347] Updated weights for policy 0, policy_version 1418515 (0.0005) [2023-12-27 15:28:33,650][20347] Updated weights for policy 0, policy_version 1418525 (0.0005) [2023-12-27 15:28:33,707][20347] Updated weights for policy 0, policy_version 1418535 (0.0008) [2023-12-27 15:28:33,754][20381] Updated weights for policy 1, policy_version 1417272 (0.0005) [2023-12-27 15:28:33,797][20381] Updated weights for policy 1, policy_version 1417282 (0.0005) [2023-12-27 15:28:33,852][20381] Updated weights for policy 1, policy_version 1417292 (0.0005) [2023-12-27 15:28:34,360][20347] Updated weights for policy 0, policy_version 1418545 (0.0008) [2023-12-27 15:28:34,394][20381] Updated weights for policy 1, policy_version 1417302 (0.0006) [2023-12-27 15:28:34,420][20347] Updated weights for policy 0, policy_version 1418555 (0.0008) [2023-12-27 15:28:34,455][20381] Updated weights for policy 1, policy_version 1417312 (0.0006) [2023-12-27 15:28:34,480][20347] Updated weights for policy 0, policy_version 1418565 (0.0008) [2023-12-27 15:28:34,511][20381] Updated weights for policy 1, policy_version 1417322 (0.0006) [2023-12-27 15:28:35,154][20347] Updated weights for policy 0, policy_version 1418575 (0.0008) [2023-12-27 15:28:35,188][20381] Updated weights for policy 1, policy_version 1417332 (0.0007) [2023-12-27 15:28:35,212][20347] Updated weights for policy 0, policy_version 1418585 (0.0008) [2023-12-27 15:28:35,238][20381] Updated weights for policy 1, policy_version 1417342 (0.0006) [2023-12-27 15:28:35,263][20347] Updated weights for policy 0, policy_version 1418595 (0.0007) [2023-12-27 15:28:35,293][20381] Updated weights for policy 1, policy_version 1417352 (0.0009) [2023-12-27 15:28:35,877][20347] Updated weights for policy 0, policy_version 1418605 (0.0007) [2023-12-27 15:28:35,935][20347] Updated weights for policy 0, policy_version 1418615 (0.0006) [2023-12-27 15:28:35,989][20381] Updated weights for policy 1, policy_version 1417362 (0.0006) [2023-12-27 15:28:35,995][20347] Updated weights for policy 0, policy_version 1418625 (0.0007) [2023-12-27 15:28:36,046][20381] Updated weights for policy 1, policy_version 1417372 (0.0007) [2023-12-27 15:28:36,109][20381] Updated weights for policy 1, policy_version 1417382 (0.0008) [2023-12-27 15:28:36,165][20381] Updated weights for policy 1, policy_version 1417392 (0.0007) [2023-12-27 15:28:36,684][20347] Updated weights for policy 0, policy_version 1418635 (0.0007) [2023-12-27 15:28:36,751][20347] Updated weights for policy 0, policy_version 1418645 (0.0006) [2023-12-27 15:28:36,809][20347] Updated weights for policy 0, policy_version 1418655 (0.0006) [2023-12-27 15:28:36,870][20381] Updated weights for policy 1, policy_version 1417402 (0.0007) [2023-12-27 15:28:36,923][20381] Updated weights for policy 1, policy_version 1417412 (0.0008) [2023-12-27 15:28:36,987][20381] Updated weights for policy 1, policy_version 1417422 (0.0008) [2023-12-27 15:28:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 726138880. Throughput: 0: 10249.9, 1: 10249.9. Samples: 726109860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:37,363][19170] Avg episode reward: [(0, '960.910'), (1, '990.230')] [2023-12-27 15:28:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001417424_362889216.pth... [2023-12-27 15:28:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001418664_363249664.pth... [2023-12-27 15:28:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001416240_362586112.pth [2023-12-27 15:28:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001417480_362946560.pth [2023-12-27 15:28:37,454][20347] Updated weights for policy 0, policy_version 1418665 (0.0006) [2023-12-27 15:28:37,518][20347] Updated weights for policy 0, policy_version 1418675 (0.0011) [2023-12-27 15:28:37,576][20347] Updated weights for policy 0, policy_version 1418685 (0.0010) [2023-12-27 15:28:37,629][20347] Updated weights for policy 0, policy_version 1418695 (0.0010) [2023-12-27 15:28:37,679][20381] Updated weights for policy 1, policy_version 1417432 (0.0008) [2023-12-27 15:28:37,738][20381] Updated weights for policy 1, policy_version 1417442 (0.0007) [2023-12-27 15:28:37,790][20381] Updated weights for policy 1, policy_version 1417452 (0.0008) [2023-12-27 15:28:38,312][20291] KL-divergence is very high: 136.7178 [2023-12-27 15:28:38,316][20347] Updated weights for policy 0, policy_version 1418705 (0.0007) [2023-12-27 15:28:38,335][20291] KL-divergence is very high: 180.4778 [2023-12-27 15:28:38,358][20291] KL-divergence is very high: 108.6091 [2023-12-27 15:28:38,376][20347] Updated weights for policy 0, policy_version 1418715 (0.0008) [2023-12-27 15:28:38,436][20347] Updated weights for policy 0, policy_version 1418725 (0.0010) [2023-12-27 15:28:38,526][20381] Updated weights for policy 1, policy_version 1417462 (0.0007) [2023-12-27 15:28:38,579][20381] Updated weights for policy 1, policy_version 1417472 (0.0007) [2023-12-27 15:28:38,638][20381] Updated weights for policy 1, policy_version 1417482 (0.0008) [2023-12-27 15:28:39,111][20347] Updated weights for policy 0, policy_version 1418735 (0.0008) [2023-12-27 15:28:39,167][20347] Updated weights for policy 0, policy_version 1418745 (0.0008) [2023-12-27 15:28:39,238][20347] Updated weights for policy 0, policy_version 1418755 (0.0009) [2023-12-27 15:28:39,412][20381] Updated weights for policy 1, policy_version 1417492 (0.0008) [2023-12-27 15:28:39,468][20381] Updated weights for policy 1, policy_version 1417502 (0.0007) [2023-12-27 15:28:39,526][20381] Updated weights for policy 1, policy_version 1417512 (0.0008) [2023-12-27 15:28:39,929][20347] Updated weights for policy 0, policy_version 1418765 (0.0009) [2023-12-27 15:28:39,999][20347] Updated weights for policy 0, policy_version 1418775 (0.0010) [2023-12-27 15:28:40,064][20347] Updated weights for policy 0, policy_version 1418785 (0.0005) [2023-12-27 15:28:40,277][20381] Updated weights for policy 1, policy_version 1417522 (0.0009) [2023-12-27 15:28:40,344][20381] Updated weights for policy 1, policy_version 1417532 (0.0008) [2023-12-27 15:28:40,406][20381] Updated weights for policy 1, policy_version 1417542 (0.0007) [2023-12-27 15:28:40,461][20381] Updated weights for policy 1, policy_version 1417552 (0.0005) [2023-12-27 15:28:40,685][20347] Updated weights for policy 0, policy_version 1418795 (0.0006) [2023-12-27 15:28:40,742][20347] Updated weights for policy 0, policy_version 1418805 (0.0007) [2023-12-27 15:28:40,797][20347] Updated weights for policy 0, policy_version 1418815 (0.0007) [2023-12-27 15:28:41,113][20381] Updated weights for policy 1, policy_version 1417562 (0.0008) [2023-12-27 15:28:41,175][20381] Updated weights for policy 1, policy_version 1417572 (0.0009) [2023-12-27 15:28:41,241][20381] Updated weights for policy 1, policy_version 1417582 (0.0008) [2023-12-27 15:28:41,521][20347] Updated weights for policy 0, policy_version 1418825 (0.0010) [2023-12-27 15:28:41,584][20347] Updated weights for policy 0, policy_version 1418835 (0.0008) [2023-12-27 15:28:41,647][20347] Updated weights for policy 0, policy_version 1418845 (0.0008) [2023-12-27 15:28:41,705][20347] Updated weights for policy 0, policy_version 1418855 (0.0008) [2023-12-27 15:28:41,971][20381] Updated weights for policy 1, policy_version 1417592 (0.0011) [2023-12-27 15:28:42,024][20381] Updated weights for policy 1, policy_version 1417602 (0.0010) [2023-12-27 15:28:42,074][20381] Updated weights for policy 1, policy_version 1417612 (0.0007) [2023-12-27 15:28:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 726237184. Throughput: 0: 10240.7, 1: 10171.3. Samples: 726229868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:42,363][19170] Avg episode reward: [(0, '931.580'), (1, '990.230')] [2023-12-27 15:28:42,441][20347] Updated weights for policy 0, policy_version 1418865 (0.0007) [2023-12-27 15:28:42,501][20347] Updated weights for policy 0, policy_version 1418875 (0.0007) [2023-12-27 15:28:42,586][20347] Updated weights for policy 0, policy_version 1418885 (0.0008) [2023-12-27 15:28:42,778][20381] Updated weights for policy 1, policy_version 1417622 (0.0007) [2023-12-27 15:28:42,842][20381] Updated weights for policy 1, policy_version 1417632 (0.0005) [2023-12-27 15:28:42,899][20381] Updated weights for policy 1, policy_version 1417642 (0.0007) [2023-12-27 15:28:43,223][20347] Updated weights for policy 0, policy_version 1418895 (0.0006) [2023-12-27 15:28:43,283][20347] Updated weights for policy 0, policy_version 1418905 (0.0006) [2023-12-27 15:28:43,345][20347] Updated weights for policy 0, policy_version 1418915 (0.0008) [2023-12-27 15:28:43,501][20381] Updated weights for policy 1, policy_version 1417652 (0.0007) [2023-12-27 15:28:43,561][20381] Updated weights for policy 1, policy_version 1417662 (0.0007) [2023-12-27 15:28:43,607][20381] Updated weights for policy 1, policy_version 1417672 (0.0007) [2023-12-27 15:28:43,973][20347] Updated weights for policy 0, policy_version 1418925 (0.0007) [2023-12-27 15:28:44,021][20347] Updated weights for policy 0, policy_version 1418935 (0.0006) [2023-12-27 15:28:44,077][20347] Updated weights for policy 0, policy_version 1418945 (0.0005) [2023-12-27 15:28:44,307][20381] Updated weights for policy 1, policy_version 1417682 (0.0008) [2023-12-27 15:28:44,365][20381] Updated weights for policy 1, policy_version 1417692 (0.0008) [2023-12-27 15:28:44,410][20381] Updated weights for policy 1, policy_version 1417702 (0.0008) [2023-12-27 15:28:44,461][20381] Updated weights for policy 1, policy_version 1417712 (0.0007) [2023-12-27 15:28:44,641][20347] Updated weights for policy 0, policy_version 1418955 (0.0005) [2023-12-27 15:28:44,705][20347] Updated weights for policy 0, policy_version 1418965 (0.0006) [2023-12-27 15:28:44,754][20347] Updated weights for policy 0, policy_version 1418975 (0.0008) [2023-12-27 15:28:45,202][20381] Updated weights for policy 1, policy_version 1417722 (0.0008) [2023-12-27 15:28:45,256][20381] Updated weights for policy 1, policy_version 1417732 (0.0008) [2023-12-27 15:28:45,312][20381] Updated weights for policy 1, policy_version 1417742 (0.0007) [2023-12-27 15:28:45,478][20347] Updated weights for policy 0, policy_version 1418985 (0.0007) [2023-12-27 15:28:45,534][20347] Updated weights for policy 0, policy_version 1418995 (0.0006) [2023-12-27 15:28:45,592][20347] Updated weights for policy 0, policy_version 1419005 (0.0007) [2023-12-27 15:28:45,649][20347] Updated weights for policy 0, policy_version 1419015 (0.0005) [2023-12-27 15:28:45,944][20381] Updated weights for policy 1, policy_version 1417752 (0.0008) [2023-12-27 15:28:45,989][20381] Updated weights for policy 1, policy_version 1417762 (0.0008) [2023-12-27 15:28:46,036][20381] Updated weights for policy 1, policy_version 1417772 (0.0007) [2023-12-27 15:28:46,253][20347] Updated weights for policy 0, policy_version 1419025 (0.0005) [2023-12-27 15:28:46,314][20347] Updated weights for policy 0, policy_version 1419035 (0.0007) [2023-12-27 15:28:46,362][20347] Updated weights for policy 0, policy_version 1419045 (0.0009) [2023-12-27 15:28:46,612][20381] Updated weights for policy 1, policy_version 1417782 (0.0007) [2023-12-27 15:28:46,656][20381] Updated weights for policy 1, policy_version 1417792 (0.0005) [2023-12-27 15:28:46,707][20381] Updated weights for policy 1, policy_version 1417802 (0.0007) [2023-12-27 15:28:46,967][20347] Updated weights for policy 0, policy_version 1419055 (0.0008) [2023-12-27 15:28:47,026][20347] Updated weights for policy 0, policy_version 1419065 (0.0007) [2023-12-27 15:28:47,078][20347] Updated weights for policy 0, policy_version 1419075 (0.0007) [2023-12-27 15:28:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 726343680. Throughput: 0: 10239.4, 1: 10172.5. Samples: 726355620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:47,363][19170] Avg episode reward: [(0, '911.960'), (1, '971.700')] [2023-12-27 15:28:47,368][20381] Updated weights for policy 1, policy_version 1417812 (0.0007) [2023-12-27 15:28:47,421][20381] Updated weights for policy 1, policy_version 1417822 (0.0007) [2023-12-27 15:28:47,469][20381] Updated weights for policy 1, policy_version 1417832 (0.0007) [2023-12-27 15:28:47,722][20347] Updated weights for policy 0, policy_version 1419085 (0.0006) [2023-12-27 15:28:47,771][20347] Updated weights for policy 0, policy_version 1419095 (0.0005) [2023-12-27 15:28:47,825][20347] Updated weights for policy 0, policy_version 1419105 (0.0006) [2023-12-27 15:28:48,156][20381] Updated weights for policy 1, policy_version 1417842 (0.0006) [2023-12-27 15:28:48,210][20381] Updated weights for policy 1, policy_version 1417852 (0.0009) [2023-12-27 15:28:48,259][20381] Updated weights for policy 1, policy_version 1417862 (0.0009) [2023-12-27 15:28:48,309][20381] Updated weights for policy 1, policy_version 1417872 (0.0006) [2023-12-27 15:28:48,461][20347] Updated weights for policy 0, policy_version 1419115 (0.0007) [2023-12-27 15:28:48,518][20347] Updated weights for policy 0, policy_version 1419125 (0.0007) [2023-12-27 15:28:48,580][20347] Updated weights for policy 0, policy_version 1419135 (0.0006) [2023-12-27 15:28:49,054][20381] Updated weights for policy 1, policy_version 1417882 (0.0007) [2023-12-27 15:28:49,110][20381] Updated weights for policy 1, policy_version 1417892 (0.0007) [2023-12-27 15:28:49,157][20381] Updated weights for policy 1, policy_version 1417902 (0.0007) [2023-12-27 15:28:49,231][20347] Updated weights for policy 0, policy_version 1419145 (0.0010) [2023-12-27 15:28:49,294][20347] Updated weights for policy 0, policy_version 1419155 (0.0008) [2023-12-27 15:28:49,357][20347] Updated weights for policy 0, policy_version 1419165 (0.0010) [2023-12-27 15:28:49,420][20347] Updated weights for policy 0, policy_version 1419175 (0.0010) [2023-12-27 15:28:49,889][20381] Updated weights for policy 1, policy_version 1417912 (0.0006) [2023-12-27 15:28:49,954][20381] Updated weights for policy 1, policy_version 1417922 (0.0007) [2023-12-27 15:28:50,019][20381] Updated weights for policy 1, policy_version 1417932 (0.0007) [2023-12-27 15:28:50,049][20347] Updated weights for policy 0, policy_version 1419185 (0.0007) [2023-12-27 15:28:50,108][20347] Updated weights for policy 0, policy_version 1419195 (0.0009) [2023-12-27 15:28:50,171][20347] Updated weights for policy 0, policy_version 1419205 (0.0007) [2023-12-27 15:28:50,667][20381] Updated weights for policy 1, policy_version 1417942 (0.0008) [2023-12-27 15:28:50,716][20381] Updated weights for policy 1, policy_version 1417952 (0.0007) [2023-12-27 15:28:50,776][20381] Updated weights for policy 1, policy_version 1417962 (0.0008) [2023-12-27 15:28:50,831][20347] Updated weights for policy 0, policy_version 1419215 (0.0011) [2023-12-27 15:28:50,887][20347] Updated weights for policy 0, policy_version 1419225 (0.0010) [2023-12-27 15:28:50,947][20347] Updated weights for policy 0, policy_version 1419235 (0.0011) [2023-12-27 15:28:51,514][20381] Updated weights for policy 1, policy_version 1417972 (0.0007) [2023-12-27 15:28:51,565][20381] Updated weights for policy 1, policy_version 1417982 (0.0006) [2023-12-27 15:28:51,644][20381] Updated weights for policy 1, policy_version 1417992 (0.0009) [2023-12-27 15:28:51,708][20347] Updated weights for policy 0, policy_version 1419245 (0.0010) [2023-12-27 15:28:51,772][20347] Updated weights for policy 0, policy_version 1419255 (0.0009) [2023-12-27 15:28:51,834][20347] Updated weights for policy 0, policy_version 1419265 (0.0007) [2023-12-27 15:28:52,293][20381] Updated weights for policy 1, policy_version 1418002 (0.0007) [2023-12-27 15:28:52,356][20381] Updated weights for policy 1, policy_version 1418012 (0.0008) [2023-12-27 15:28:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 726441984. Throughput: 0: 10309.3, 1: 10095.6. Samples: 726417600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:52,363][19170] Avg episode reward: [(0, '941.420'), (1, '962.060')] [2023-12-27 15:28:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001419272_363405312.pth... [2023-12-27 15:28:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001418056_363094016.pth [2023-12-27 15:28:52,425][20381] Updated weights for policy 1, policy_version 1418022 (0.0008) [2023-12-27 15:28:52,474][20381] Updated weights for policy 1, policy_version 1418032 (0.0008) [2023-12-27 15:28:52,474][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001418032_363044864.pth... [2023-12-27 15:28:52,477][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001416848_362741760.pth [2023-12-27 15:28:52,573][20347] Updated weights for policy 0, policy_version 1419275 (0.0010) [2023-12-27 15:28:52,635][20347] Updated weights for policy 0, policy_version 1419285 (0.0006) [2023-12-27 15:28:52,697][20347] Updated weights for policy 0, policy_version 1419295 (0.0006) [2023-12-27 15:28:53,167][20381] Updated weights for policy 1, policy_version 1418042 (0.0007) [2023-12-27 15:28:53,215][20381] Updated weights for policy 1, policy_version 1418052 (0.0007) [2023-12-27 15:28:53,265][20381] Updated weights for policy 1, policy_version 1418062 (0.0005) [2023-12-27 15:28:53,310][20347] Updated weights for policy 0, policy_version 1419305 (0.0006) [2023-12-27 15:28:53,368][20347] Updated weights for policy 0, policy_version 1419315 (0.0005) [2023-12-27 15:28:53,427][20347] Updated weights for policy 0, policy_version 1419325 (0.0005) [2023-12-27 15:28:53,476][20347] Updated weights for policy 0, policy_version 1419335 (0.0005) [2023-12-27 15:28:53,987][20381] Updated weights for policy 1, policy_version 1418072 (0.0007) [2023-12-27 15:28:54,030][20347] Updated weights for policy 0, policy_version 1419345 (0.0006) [2023-12-27 15:28:54,043][20381] Updated weights for policy 1, policy_version 1418082 (0.0005) [2023-12-27 15:28:54,086][20347] Updated weights for policy 0, policy_version 1419355 (0.0006) [2023-12-27 15:28:54,095][20381] Updated weights for policy 1, policy_version 1418092 (0.0006) [2023-12-27 15:28:54,140][20347] Updated weights for policy 0, policy_version 1419365 (0.0006) [2023-12-27 15:28:54,788][20381] Updated weights for policy 1, policy_version 1418102 (0.0007) [2023-12-27 15:28:54,846][20381] Updated weights for policy 1, policy_version 1418112 (0.0008) [2023-12-27 15:28:54,851][20347] Updated weights for policy 0, policy_version 1419375 (0.0006) [2023-12-27 15:28:54,901][20347] Updated weights for policy 0, policy_version 1419385 (0.0005) [2023-12-27 15:28:54,901][20381] Updated weights for policy 1, policy_version 1418122 (0.0007) [2023-12-27 15:28:54,947][20347] Updated weights for policy 0, policy_version 1419395 (0.0006) [2023-12-27 15:28:55,602][20381] Updated weights for policy 1, policy_version 1418132 (0.0007) [2023-12-27 15:28:55,625][20347] Updated weights for policy 0, policy_version 1419405 (0.0008) [2023-12-27 15:28:55,657][20381] Updated weights for policy 1, policy_version 1418142 (0.0006) [2023-12-27 15:28:55,677][20347] Updated weights for policy 0, policy_version 1419415 (0.0006) [2023-12-27 15:28:55,708][20381] Updated weights for policy 1, policy_version 1418152 (0.0005) [2023-12-27 15:28:55,739][20347] Updated weights for policy 0, policy_version 1419425 (0.0005) [2023-12-27 15:28:56,394][20347] Updated weights for policy 0, policy_version 1419435 (0.0006) [2023-12-27 15:28:56,399][20381] Updated weights for policy 1, policy_version 1418162 (0.0007) [2023-12-27 15:28:56,453][20381] Updated weights for policy 1, policy_version 1418172 (0.0006) [2023-12-27 15:28:56,455][20347] Updated weights for policy 0, policy_version 1419445 (0.0006) [2023-12-27 15:28:56,504][20347] Updated weights for policy 0, policy_version 1419455 (0.0005) [2023-12-27 15:28:56,505][20381] Updated weights for policy 1, policy_version 1418182 (0.0006) [2023-12-27 15:28:56,558][20381] Updated weights for policy 1, policy_version 1418192 (0.0007) [2023-12-27 15:28:57,192][20347] Updated weights for policy 0, policy_version 1419465 (0.0005) [2023-12-27 15:28:57,244][20381] Updated weights for policy 1, policy_version 1418202 (0.0007) [2023-12-27 15:28:57,250][20347] Updated weights for policy 0, policy_version 1419475 (0.0005) [2023-12-27 15:28:57,299][20381] Updated weights for policy 1, policy_version 1418212 (0.0007) [2023-12-27 15:28:57,301][20347] Updated weights for policy 0, policy_version 1419485 (0.0005) [2023-12-27 15:28:57,353][20347] Updated weights for policy 0, policy_version 1419495 (0.0006) [2023-12-27 15:28:57,354][20381] Updated weights for policy 1, policy_version 1418222 (0.0006) [2023-12-27 15:28:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 726548480. Throughput: 0: 10369.5, 1: 10027.7. Samples: 726539184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:28:57,363][19170] Avg episode reward: [(0, '970.770'), (1, '961.070')] [2023-12-27 15:28:57,989][20347] Updated weights for policy 0, policy_version 1419505 (0.0006) [2023-12-27 15:28:58,041][20347] Updated weights for policy 0, policy_version 1419515 (0.0006) [2023-12-27 15:28:58,067][20381] Updated weights for policy 1, policy_version 1418232 (0.0006) [2023-12-27 15:28:58,088][20347] Updated weights for policy 0, policy_version 1419525 (0.0010) [2023-12-27 15:28:58,122][20381] Updated weights for policy 1, policy_version 1418242 (0.0007) [2023-12-27 15:28:58,180][20381] Updated weights for policy 1, policy_version 1418252 (0.0007) [2023-12-27 15:28:58,885][20347] Updated weights for policy 0, policy_version 1419535 (0.0009) [2023-12-27 15:28:58,954][20347] Updated weights for policy 0, policy_version 1419545 (0.0008) [2023-12-27 15:28:58,965][20381] Updated weights for policy 1, policy_version 1418262 (0.0008) [2023-12-27 15:28:59,017][20347] Updated weights for policy 0, policy_version 1419555 (0.0008) [2023-12-27 15:28:59,023][20381] Updated weights for policy 1, policy_version 1418272 (0.0006) [2023-12-27 15:28:59,088][20381] Updated weights for policy 1, policy_version 1418282 (0.0008) [2023-12-27 15:28:59,770][20347] Updated weights for policy 0, policy_version 1419565 (0.0007) [2023-12-27 15:28:59,772][20381] Updated weights for policy 1, policy_version 1418292 (0.0007) [2023-12-27 15:28:59,826][20381] Updated weights for policy 1, policy_version 1418302 (0.0009) [2023-12-27 15:28:59,827][20347] Updated weights for policy 0, policy_version 1419575 (0.0009) [2023-12-27 15:28:59,882][20381] Updated weights for policy 1, policy_version 1418312 (0.0008) [2023-12-27 15:28:59,883][20347] Updated weights for policy 0, policy_version 1419585 (0.0006) [2023-12-27 15:29:00,483][20381] Updated weights for policy 1, policy_version 1418322 (0.0008) [2023-12-27 15:29:00,542][20381] Updated weights for policy 1, policy_version 1418332 (0.0008) [2023-12-27 15:29:00,600][20381] Updated weights for policy 1, policy_version 1418342 (0.0006) [2023-12-27 15:29:00,620][20347] Updated weights for policy 0, policy_version 1419595 (0.0008) [2023-12-27 15:29:00,664][20381] Updated weights for policy 1, policy_version 1418352 (0.0006) [2023-12-27 15:29:00,675][20347] Updated weights for policy 0, policy_version 1419605 (0.0007) [2023-12-27 15:29:00,724][20347] Updated weights for policy 0, policy_version 1419615 (0.0009) [2023-12-27 15:29:01,292][20381] Updated weights for policy 1, policy_version 1418362 (0.0007) [2023-12-27 15:29:01,368][20381] Updated weights for policy 1, policy_version 1418374 (0.0008) [2023-12-27 15:29:01,425][20381] Updated weights for policy 1, policy_version 1418384 (0.0008) [2023-12-27 15:29:01,427][20347] Updated weights for policy 0, policy_version 1419625 (0.0007) [2023-12-27 15:29:01,486][20347] Updated weights for policy 0, policy_version 1419635 (0.0008) [2023-12-27 15:29:01,545][20347] Updated weights for policy 0, policy_version 1419645 (0.0008) [2023-12-27 15:29:01,599][20347] Updated weights for policy 0, policy_version 1419655 (0.0007) [2023-12-27 15:29:02,130][20381] Updated weights for policy 1, policy_version 1418394 (0.0006) [2023-12-27 15:29:02,188][20381] Updated weights for policy 1, policy_version 1418404 (0.0008) [2023-12-27 15:29:02,248][20381] Updated weights for policy 1, policy_version 1418414 (0.0007) [2023-12-27 15:29:02,326][20347] Updated weights for policy 0, policy_version 1419665 (0.0009) [2023-12-27 15:29:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 726646784. Throughput: 0: 10290.3, 1: 10056.2. Samples: 726658480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.170')] [2023-12-27 15:29:02,394][20347] Updated weights for policy 0, policy_version 1419675 (0.0008) [2023-12-27 15:29:02,456][20347] Updated weights for policy 0, policy_version 1419685 (0.0008) [2023-12-27 15:29:02,902][20381] Updated weights for policy 1, policy_version 1418424 (0.0010) [2023-12-27 15:29:02,953][20381] Updated weights for policy 1, policy_version 1418434 (0.0010) [2023-12-27 15:29:02,998][20381] Updated weights for policy 1, policy_version 1418444 (0.0009) [2023-12-27 15:29:03,151][20347] Updated weights for policy 0, policy_version 1419695 (0.0007) [2023-12-27 15:29:03,205][20347] Updated weights for policy 0, policy_version 1419705 (0.0007) [2023-12-27 15:29:03,252][20347] Updated weights for policy 0, policy_version 1419715 (0.0007) [2023-12-27 15:29:03,678][20381] Updated weights for policy 1, policy_version 1418454 (0.0010) [2023-12-27 15:29:03,741][20381] Updated weights for policy 1, policy_version 1418464 (0.0010) [2023-12-27 15:29:03,788][20381] Updated weights for policy 1, policy_version 1418474 (0.0010) [2023-12-27 15:29:03,956][20347] Updated weights for policy 0, policy_version 1419725 (0.0007) [2023-12-27 15:29:04,021][20347] Updated weights for policy 0, policy_version 1419735 (0.0007) [2023-12-27 15:29:04,080][20347] Updated weights for policy 0, policy_version 1419745 (0.0008) [2023-12-27 15:29:04,480][20381] Updated weights for policy 1, policy_version 1418484 (0.0010) [2023-12-27 15:29:04,537][20381] Updated weights for policy 1, policy_version 1418494 (0.0010) [2023-12-27 15:29:04,588][20381] Updated weights for policy 1, policy_version 1418504 (0.0010) [2023-12-27 15:29:04,786][20347] Updated weights for policy 0, policy_version 1419755 (0.0007) [2023-12-27 15:29:04,840][20347] Updated weights for policy 0, policy_version 1419765 (0.0007) [2023-12-27 15:29:04,885][20347] Updated weights for policy 0, policy_version 1419775 (0.0007) [2023-12-27 15:29:05,252][20381] Updated weights for policy 1, policy_version 1418514 (0.0009) [2023-12-27 15:29:05,310][20381] Updated weights for policy 1, policy_version 1418524 (0.0007) [2023-12-27 15:29:05,369][20381] Updated weights for policy 1, policy_version 1418534 (0.0010) [2023-12-27 15:29:05,421][20381] Updated weights for policy 1, policy_version 1418544 (0.0010) [2023-12-27 15:29:05,587][20347] Updated weights for policy 0, policy_version 1419785 (0.0008) [2023-12-27 15:29:05,633][20347] Updated weights for policy 0, policy_version 1419795 (0.0007) [2023-12-27 15:29:05,680][20347] Updated weights for policy 0, policy_version 1419805 (0.0007) [2023-12-27 15:29:05,737][20347] Updated weights for policy 0, policy_version 1419815 (0.0007) [2023-12-27 15:29:06,074][20381] Updated weights for policy 1, policy_version 1418554 (0.0005) [2023-12-27 15:29:06,135][20381] Updated weights for policy 1, policy_version 1418564 (0.0007) [2023-12-27 15:29:06,194][20381] Updated weights for policy 1, policy_version 1418574 (0.0007) [2023-12-27 15:29:06,414][20347] Updated weights for policy 0, policy_version 1419825 (0.0007) [2023-12-27 15:29:06,468][20347] Updated weights for policy 0, policy_version 1419835 (0.0006) [2023-12-27 15:29:06,525][20347] Updated weights for policy 0, policy_version 1419845 (0.0005) [2023-12-27 15:29:06,853][20381] Updated weights for policy 1, policy_version 1418584 (0.0008) [2023-12-27 15:29:06,901][20381] Updated weights for policy 1, policy_version 1418594 (0.0007) [2023-12-27 15:29:06,962][20381] Updated weights for policy 1, policy_version 1418604 (0.0008) [2023-12-27 15:29:07,203][20347] Updated weights for policy 0, policy_version 1419855 (0.0007) [2023-12-27 15:29:07,270][20347] Updated weights for policy 0, policy_version 1419865 (0.0008) [2023-12-27 15:29:07,326][20347] Updated weights for policy 0, policy_version 1419875 (0.0008) [2023-12-27 15:29:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 726753280. Throughput: 0: 10265.4, 1: 10098.6. Samples: 726718676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:07,363][19170] Avg episode reward: [(0, '980.500'), (1, '980.460')] [2023-12-27 15:29:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001418608_363192320.pth... [2023-12-27 15:29:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001419880_363560960.pth... [2023-12-27 15:29:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001417424_362889216.pth [2023-12-27 15:29:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001418664_363249664.pth [2023-12-27 15:29:07,576][20381] Updated weights for policy 1, policy_version 1418614 (0.0008) [2023-12-27 15:29:07,638][20381] Updated weights for policy 1, policy_version 1418624 (0.0008) [2023-12-27 15:29:07,698][20381] Updated weights for policy 1, policy_version 1418634 (0.0007) [2023-12-27 15:29:08,034][20347] Updated weights for policy 0, policy_version 1419885 (0.0007) [2023-12-27 15:29:08,094][20347] Updated weights for policy 0, policy_version 1419895 (0.0008) [2023-12-27 15:29:08,139][20347] Updated weights for policy 0, policy_version 1419905 (0.0008) [2023-12-27 15:29:08,296][20381] Updated weights for policy 1, policy_version 1418644 (0.0008) [2023-12-27 15:29:08,363][20381] Updated weights for policy 1, policy_version 1418654 (0.0008) [2023-12-27 15:29:08,421][20381] Updated weights for policy 1, policy_version 1418664 (0.0009) [2023-12-27 15:29:08,945][20347] Updated weights for policy 0, policy_version 1419915 (0.0007) [2023-12-27 15:29:08,992][20291] KL-divergence is very high: 131.2067 [2023-12-27 15:29:09,005][20347] Updated weights for policy 0, policy_version 1419925 (0.0007) [2023-12-27 15:29:09,031][20291] KL-divergence is very high: 101.3870 [2023-12-27 15:29:09,052][20347] Updated weights for policy 0, policy_version 1419935 (0.0007) [2023-12-27 15:29:09,148][20381] Updated weights for policy 1, policy_version 1418674 (0.0009) [2023-12-27 15:29:09,194][20381] Updated weights for policy 1, policy_version 1418684 (0.0011) [2023-12-27 15:29:09,258][20381] Updated weights for policy 1, policy_version 1418694 (0.0011) [2023-12-27 15:29:09,316][20381] Updated weights for policy 1, policy_version 1418704 (0.0011) [2023-12-27 15:29:09,852][20347] Updated weights for policy 0, policy_version 1419945 (0.0007) [2023-12-27 15:29:09,908][20347] Updated weights for policy 0, policy_version 1419955 (0.0008) [2023-12-27 15:29:09,978][20347] Updated weights for policy 0, policy_version 1419965 (0.0008) [2023-12-27 15:29:10,031][20381] Updated weights for policy 1, policy_version 1418714 (0.0007) [2023-12-27 15:29:10,039][20347] Updated weights for policy 0, policy_version 1419975 (0.0007) [2023-12-27 15:29:10,094][20381] Updated weights for policy 1, policy_version 1418724 (0.0009) [2023-12-27 15:29:10,153][20381] Updated weights for policy 1, policy_version 1418734 (0.0008) [2023-12-27 15:29:10,632][20347] Updated weights for policy 0, policy_version 1419985 (0.0005) [2023-12-27 15:29:10,683][20347] Updated weights for policy 0, policy_version 1419995 (0.0005) [2023-12-27 15:29:10,739][20347] Updated weights for policy 0, policy_version 1420005 (0.0007) [2023-12-27 15:29:10,880][20381] Updated weights for policy 1, policy_version 1418744 (0.0010) [2023-12-27 15:29:10,934][20381] Updated weights for policy 1, policy_version 1418754 (0.0011) [2023-12-27 15:29:10,994][20381] Updated weights for policy 1, policy_version 1418764 (0.0011) [2023-12-27 15:29:11,423][20347] Updated weights for policy 0, policy_version 1420015 (0.0008) [2023-12-27 15:29:11,489][20347] Updated weights for policy 0, policy_version 1420025 (0.0007) [2023-12-27 15:29:11,551][20347] Updated weights for policy 0, policy_version 1420035 (0.0007) [2023-12-27 15:29:11,738][20381] Updated weights for policy 1, policy_version 1418774 (0.0011) [2023-12-27 15:29:11,798][20381] Updated weights for policy 1, policy_version 1418784 (0.0008) [2023-12-27 15:29:11,856][20381] Updated weights for policy 1, policy_version 1418794 (0.0008) [2023-12-27 15:29:12,236][20347] Updated weights for policy 0, policy_version 1420045 (0.0007) [2023-12-27 15:29:12,297][20347] Updated weights for policy 0, policy_version 1420055 (0.0008) [2023-12-27 15:29:12,358][20347] Updated weights for policy 0, policy_version 1420065 (0.0008) [2023-12-27 15:29:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 726843392. Throughput: 0: 10264.8, 1: 10133.7. Samples: 726839284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:12,363][19170] Avg episode reward: [(0, '951.290'), (1, '981.370')] [2023-12-27 15:29:12,558][20381] Updated weights for policy 1, policy_version 1418804 (0.0008) [2023-12-27 15:29:12,618][20381] Updated weights for policy 1, policy_version 1418814 (0.0007) [2023-12-27 15:29:12,679][20381] Updated weights for policy 1, policy_version 1418824 (0.0008) [2023-12-27 15:29:13,107][20347] Updated weights for policy 0, policy_version 1420075 (0.0008) [2023-12-27 15:29:13,160][20347] Updated weights for policy 0, policy_version 1420085 (0.0008) [2023-12-27 15:29:13,204][20347] Updated weights for policy 0, policy_version 1420095 (0.0007) [2023-12-27 15:29:13,378][20381] Updated weights for policy 1, policy_version 1418834 (0.0008) [2023-12-27 15:29:13,403][20292] KL-divergence is very high: 133.5515 [2023-12-27 15:29:13,429][20381] Updated weights for policy 1, policy_version 1418844 (0.0005) [2023-12-27 15:29:13,485][20381] Updated weights for policy 1, policy_version 1418854 (0.0008) [2023-12-27 15:29:13,544][20381] Updated weights for policy 1, policy_version 1418864 (0.0008) [2023-12-27 15:29:13,937][20347] Updated weights for policy 0, policy_version 1420105 (0.0007) [2023-12-27 15:29:13,988][20347] Updated weights for policy 0, policy_version 1420115 (0.0005) [2023-12-27 15:29:14,038][20347] Updated weights for policy 0, policy_version 1420125 (0.0007) [2023-12-27 15:29:14,091][20347] Updated weights for policy 0, policy_version 1420135 (0.0007) [2023-12-27 15:29:14,180][20381] Updated weights for policy 1, policy_version 1418874 (0.0008) [2023-12-27 15:29:14,238][20381] Updated weights for policy 1, policy_version 1418884 (0.0008) [2023-12-27 15:29:14,284][20381] Updated weights for policy 1, policy_version 1418894 (0.0008) [2023-12-27 15:29:14,705][20347] Updated weights for policy 0, policy_version 1420145 (0.0010) [2023-12-27 15:29:14,760][20347] Updated weights for policy 0, policy_version 1420155 (0.0010) [2023-12-27 15:29:14,826][20347] Updated weights for policy 0, policy_version 1420165 (0.0008) [2023-12-27 15:29:15,027][20381] Updated weights for policy 1, policy_version 1418904 (0.0008) [2023-12-27 15:29:15,090][20381] Updated weights for policy 1, policy_version 1418914 (0.0008) [2023-12-27 15:29:15,146][20381] Updated weights for policy 1, policy_version 1418924 (0.0008) [2023-12-27 15:29:15,514][20347] Updated weights for policy 0, policy_version 1420175 (0.0009) [2023-12-27 15:29:15,575][20347] Updated weights for policy 0, policy_version 1420185 (0.0007) [2023-12-27 15:29:15,623][20347] Updated weights for policy 0, policy_version 1420195 (0.0010) [2023-12-27 15:29:15,806][20381] Updated weights for policy 1, policy_version 1418934 (0.0007) [2023-12-27 15:29:15,874][20381] Updated weights for policy 1, policy_version 1418944 (0.0008) [2023-12-27 15:29:15,928][20381] Updated weights for policy 1, policy_version 1418954 (0.0007) [2023-12-27 15:29:16,241][20347] Updated weights for policy 0, policy_version 1420205 (0.0008) [2023-12-27 15:29:16,304][20347] Updated weights for policy 0, policy_version 1420215 (0.0005) [2023-12-27 15:29:16,369][20347] Updated weights for policy 0, policy_version 1420225 (0.0005) [2023-12-27 15:29:16,583][20381] Updated weights for policy 1, policy_version 1418964 (0.0008) [2023-12-27 15:29:16,635][20381] Updated weights for policy 1, policy_version 1418974 (0.0011) [2023-12-27 15:29:16,693][20381] Updated weights for policy 1, policy_version 1418984 (0.0011) [2023-12-27 15:29:16,919][20347] Updated weights for policy 0, policy_version 1420235 (0.0006) [2023-12-27 15:29:16,982][20347] Updated weights for policy 0, policy_version 1420245 (0.0006) [2023-12-27 15:29:17,043][20347] Updated weights for policy 0, policy_version 1420255 (0.0005) [2023-12-27 15:29:17,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 726949888. Throughput: 0: 10199.5, 1: 10160.4. Samples: 726961896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:17,363][19170] Avg episode reward: [(0, '931.810'), (1, '971.590')] [2023-12-27 15:29:17,364][20381] Updated weights for policy 1, policy_version 1418994 (0.0010) [2023-12-27 15:29:17,419][20381] Updated weights for policy 1, policy_version 1419004 (0.0005) [2023-12-27 15:29:17,475][20381] Updated weights for policy 1, policy_version 1419014 (0.0006) [2023-12-27 15:29:17,536][20381] Updated weights for policy 1, policy_version 1419024 (0.0009) [2023-12-27 15:29:17,673][20347] Updated weights for policy 0, policy_version 1420265 (0.0005) [2023-12-27 15:29:17,732][20347] Updated weights for policy 0, policy_version 1420275 (0.0007) [2023-12-27 15:29:17,794][20347] Updated weights for policy 0, policy_version 1420285 (0.0005) [2023-12-27 15:29:17,852][20347] Updated weights for policy 0, policy_version 1420295 (0.0006) [2023-12-27 15:29:18,149][20381] Updated weights for policy 1, policy_version 1419034 (0.0011) [2023-12-27 15:29:18,211][20381] Updated weights for policy 1, policy_version 1419044 (0.0011) [2023-12-27 15:29:18,270][20381] Updated weights for policy 1, policy_version 1419054 (0.0011) [2023-12-27 15:29:18,513][20347] Updated weights for policy 0, policy_version 1420305 (0.0010) [2023-12-27 15:29:18,568][20347] Updated weights for policy 0, policy_version 1420315 (0.0008) [2023-12-27 15:29:18,623][20347] Updated weights for policy 0, policy_version 1420325 (0.0008) [2023-12-27 15:29:18,901][20381] Updated weights for policy 1, policy_version 1419064 (0.0011) [2023-12-27 15:29:18,953][20381] Updated weights for policy 1, policy_version 1419074 (0.0010) [2023-12-27 15:29:19,011][20381] Updated weights for policy 1, policy_version 1419084 (0.0011) [2023-12-27 15:29:19,301][20347] Updated weights for policy 0, policy_version 1420335 (0.0009) [2023-12-27 15:29:19,364][20347] Updated weights for policy 0, policy_version 1420345 (0.0009) [2023-12-27 15:29:19,430][20347] Updated weights for policy 0, policy_version 1420356 (0.0006) [2023-12-27 15:29:19,689][20381] Updated weights for policy 1, policy_version 1419094 (0.0009) [2023-12-27 15:29:19,753][20381] Updated weights for policy 1, policy_version 1419104 (0.0008) [2023-12-27 15:29:19,813][20381] Updated weights for policy 1, policy_version 1419114 (0.0008) [2023-12-27 15:29:20,067][20347] Updated weights for policy 0, policy_version 1420366 (0.0007) [2023-12-27 15:29:20,130][20347] Updated weights for policy 0, policy_version 1420376 (0.0008) [2023-12-27 15:29:20,188][20347] Updated weights for policy 0, policy_version 1420386 (0.0008) [2023-12-27 15:29:20,530][20381] Updated weights for policy 1, policy_version 1419124 (0.0008) [2023-12-27 15:29:20,592][20381] Updated weights for policy 1, policy_version 1419134 (0.0008) [2023-12-27 15:29:20,651][20381] Updated weights for policy 1, policy_version 1419144 (0.0008) [2023-12-27 15:29:20,868][20347] Updated weights for policy 0, policy_version 1420396 (0.0008) [2023-12-27 15:29:20,927][20347] Updated weights for policy 0, policy_version 1420406 (0.0007) [2023-12-27 15:29:20,988][20347] Updated weights for policy 0, policy_version 1420416 (0.0008) [2023-12-27 15:29:21,396][20381] Updated weights for policy 1, policy_version 1419154 (0.0008) [2023-12-27 15:29:21,454][20381] Updated weights for policy 1, policy_version 1419164 (0.0006) [2023-12-27 15:29:21,518][20381] Updated weights for policy 1, policy_version 1419174 (0.0008) [2023-12-27 15:29:21,580][20381] Updated weights for policy 1, policy_version 1419184 (0.0007) [2023-12-27 15:29:21,766][20347] Updated weights for policy 0, policy_version 1420427 (0.0009) [2023-12-27 15:29:21,837][20347] Updated weights for policy 0, policy_version 1420437 (0.0008) [2023-12-27 15:29:21,898][20347] Updated weights for policy 0, policy_version 1420447 (0.0007) [2023-12-27 15:29:22,304][20381] Updated weights for policy 1, policy_version 1419194 (0.0008) [2023-12-27 15:29:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 727048192. Throughput: 0: 10206.8, 1: 10118.5. Samples: 727024496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:22,363][19170] Avg episode reward: [(0, '951.340'), (1, '990.260')] [2023-12-27 15:29:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001420456_363708416.pth... [2023-12-27 15:29:22,372][20381] Updated weights for policy 1, policy_version 1419204 (0.0009) [2023-12-27 15:29:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001419272_363405312.pth [2023-12-27 15:29:22,431][20381] Updated weights for policy 1, policy_version 1419214 (0.0007) [2023-12-27 15:29:22,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001419216_363347968.pth... [2023-12-27 15:29:22,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001418032_363044864.pth [2023-12-27 15:29:22,625][20347] Updated weights for policy 0, policy_version 1420457 (0.0006) [2023-12-27 15:29:22,683][20347] Updated weights for policy 0, policy_version 1420467 (0.0005) [2023-12-27 15:29:22,739][20347] Updated weights for policy 0, policy_version 1420477 (0.0008) [2023-12-27 15:29:22,792][20347] Updated weights for policy 0, policy_version 1420487 (0.0010) [2023-12-27 15:29:23,171][20381] Updated weights for policy 1, policy_version 1419224 (0.0007) [2023-12-27 15:29:23,224][20381] Updated weights for policy 1, policy_version 1419234 (0.0008) [2023-12-27 15:29:23,283][20381] Updated weights for policy 1, policy_version 1419244 (0.0006) [2023-12-27 15:29:23,413][20347] Updated weights for policy 0, policy_version 1420497 (0.0010) [2023-12-27 15:29:23,470][20347] Updated weights for policy 0, policy_version 1420507 (0.0010) [2023-12-27 15:29:23,528][20347] Updated weights for policy 0, policy_version 1420517 (0.0010) [2023-12-27 15:29:23,863][20381] Updated weights for policy 1, policy_version 1419254 (0.0005) [2023-12-27 15:29:23,925][20381] Updated weights for policy 1, policy_version 1419264 (0.0005) [2023-12-27 15:29:23,985][20381] Updated weights for policy 1, policy_version 1419274 (0.0005) [2023-12-27 15:29:24,162][20347] Updated weights for policy 0, policy_version 1420527 (0.0010) [2023-12-27 15:29:24,217][20347] Updated weights for policy 0, policy_version 1420537 (0.0010) [2023-12-27 15:29:24,274][20347] Updated weights for policy 0, policy_version 1420547 (0.0010) [2023-12-27 15:29:24,581][20381] Updated weights for policy 1, policy_version 1419284 (0.0006) [2023-12-27 15:29:24,628][20381] Updated weights for policy 1, policy_version 1419294 (0.0007) [2023-12-27 15:29:24,675][20381] Updated weights for policy 1, policy_version 1419304 (0.0008) [2023-12-27 15:29:24,938][20347] Updated weights for policy 0, policy_version 1420557 (0.0010) [2023-12-27 15:29:25,003][20347] Updated weights for policy 0, policy_version 1420567 (0.0011) [2023-12-27 15:29:25,069][20347] Updated weights for policy 0, policy_version 1420577 (0.0010) [2023-12-27 15:29:25,325][20381] Updated weights for policy 1, policy_version 1419314 (0.0007) [2023-12-27 15:29:25,378][20381] Updated weights for policy 1, policy_version 1419324 (0.0007) [2023-12-27 15:29:25,431][20381] Updated weights for policy 1, policy_version 1419334 (0.0007) [2023-12-27 15:29:25,485][20381] Updated weights for policy 1, policy_version 1419344 (0.0007) [2023-12-27 15:29:25,740][20347] Updated weights for policy 0, policy_version 1420587 (0.0010) [2023-12-27 15:29:25,797][20347] Updated weights for policy 0, policy_version 1420597 (0.0010) [2023-12-27 15:29:25,852][20347] Updated weights for policy 0, policy_version 1420607 (0.0010) [2023-12-27 15:29:26,111][20381] Updated weights for policy 1, policy_version 1419354 (0.0006) [2023-12-27 15:29:26,162][20381] Updated weights for policy 1, policy_version 1419364 (0.0005) [2023-12-27 15:29:26,214][20381] Updated weights for policy 1, policy_version 1419374 (0.0005) [2023-12-27 15:29:26,508][20347] Updated weights for policy 0, policy_version 1420617 (0.0010) [2023-12-27 15:29:26,565][20347] Updated weights for policy 0, policy_version 1420627 (0.0010) [2023-12-27 15:29:26,612][20347] Updated weights for policy 0, policy_version 1420637 (0.0010) [2023-12-27 15:29:26,663][20347] Updated weights for policy 0, policy_version 1420647 (0.0010) [2023-12-27 15:29:26,755][20381] Updated weights for policy 1, policy_version 1419384 (0.0009) [2023-12-27 15:29:26,811][20381] Updated weights for policy 1, policy_version 1419394 (0.0010) [2023-12-27 15:29:26,862][20381] Updated weights for policy 1, policy_version 1419404 (0.0010) [2023-12-27 15:29:27,313][20347] Updated weights for policy 0, policy_version 1420657 (0.0006) [2023-12-27 15:29:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 727154688. Throughput: 0: 10175.8, 1: 10213.2. Samples: 727147372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:27,363][19170] Avg episode reward: [(0, '980.490'), (1, '990.310')] [2023-12-27 15:29:27,375][20347] Updated weights for policy 0, policy_version 1420667 (0.0008) [2023-12-27 15:29:27,433][20347] Updated weights for policy 0, policy_version 1420677 (0.0010) [2023-12-27 15:29:27,514][20381] Updated weights for policy 1, policy_version 1419414 (0.0010) [2023-12-27 15:29:27,565][20381] Updated weights for policy 1, policy_version 1419424 (0.0010) [2023-12-27 15:29:27,617][20381] Updated weights for policy 1, policy_version 1419434 (0.0010) [2023-12-27 15:29:28,040][20347] Updated weights for policy 0, policy_version 1420687 (0.0007) [2023-12-27 15:29:28,102][20347] Updated weights for policy 0, policy_version 1420697 (0.0005) [2023-12-27 15:29:28,161][20347] Updated weights for policy 0, policy_version 1420707 (0.0007) [2023-12-27 15:29:28,273][20381] Updated weights for policy 1, policy_version 1419444 (0.0011) [2023-12-27 15:29:28,321][20381] Updated weights for policy 1, policy_version 1419454 (0.0010) [2023-12-27 15:29:28,382][20381] Updated weights for policy 1, policy_version 1419464 (0.0008) [2023-12-27 15:29:28,712][20347] Updated weights for policy 0, policy_version 1420717 (0.0008) [2023-12-27 15:29:28,762][20347] Updated weights for policy 0, policy_version 1420727 (0.0007) [2023-12-27 15:29:28,821][20347] Updated weights for policy 0, policy_version 1420737 (0.0007) [2023-12-27 15:29:29,064][20381] Updated weights for policy 1, policy_version 1419474 (0.0010) [2023-12-27 15:29:29,120][20381] Updated weights for policy 1, policy_version 1419484 (0.0005) [2023-12-27 15:29:29,183][20381] Updated weights for policy 1, policy_version 1419494 (0.0005) [2023-12-27 15:29:29,248][20381] Updated weights for policy 1, policy_version 1419504 (0.0007) [2023-12-27 15:29:29,493][20347] Updated weights for policy 0, policy_version 1420747 (0.0008) [2023-12-27 15:29:29,546][20347] Updated weights for policy 0, policy_version 1420757 (0.0008) [2023-12-27 15:29:29,606][20347] Updated weights for policy 0, policy_version 1420767 (0.0007) [2023-12-27 15:29:29,862][20381] Updated weights for policy 1, policy_version 1419514 (0.0011) [2023-12-27 15:29:29,922][20381] Updated weights for policy 1, policy_version 1419524 (0.0010) [2023-12-27 15:29:29,973][20381] Updated weights for policy 1, policy_version 1419534 (0.0009) [2023-12-27 15:29:30,388][20347] Updated weights for policy 0, policy_version 1420777 (0.0009) [2023-12-27 15:29:30,441][20347] Updated weights for policy 0, policy_version 1420787 (0.0006) [2023-12-27 15:29:30,495][20347] Updated weights for policy 0, policy_version 1420797 (0.0007) [2023-12-27 15:29:30,546][20347] Updated weights for policy 0, policy_version 1420807 (0.0006) [2023-12-27 15:29:30,623][20381] Updated weights for policy 1, policy_version 1419544 (0.0007) [2023-12-27 15:29:30,681][20381] Updated weights for policy 1, policy_version 1419554 (0.0010) [2023-12-27 15:29:30,744][20381] Updated weights for policy 1, policy_version 1419564 (0.0010) [2023-12-27 15:29:31,165][20347] Updated weights for policy 0, policy_version 1420817 (0.0010) [2023-12-27 15:29:31,224][20347] Updated weights for policy 0, policy_version 1420827 (0.0009) [2023-12-27 15:29:31,287][20347] Updated weights for policy 0, policy_version 1420837 (0.0008) [2023-12-27 15:29:31,432][20381] Updated weights for policy 1, policy_version 1419574 (0.0010) [2023-12-27 15:29:31,484][20381] Updated weights for policy 1, policy_version 1419584 (0.0008) [2023-12-27 15:29:31,543][20381] Updated weights for policy 1, policy_version 1419594 (0.0007) [2023-12-27 15:29:31,987][20347] Updated weights for policy 0, policy_version 1420847 (0.0009) [2023-12-27 15:29:32,038][20347] Updated weights for policy 0, policy_version 1420857 (0.0007) [2023-12-27 15:29:32,089][20347] Updated weights for policy 0, policy_version 1420867 (0.0007) [2023-12-27 15:29:32,270][20381] Updated weights for policy 1, policy_version 1419604 (0.0008) [2023-12-27 15:29:32,332][20381] Updated weights for policy 1, policy_version 1419614 (0.0007) [2023-12-27 15:29:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 727261184. Throughput: 0: 10192.0, 1: 10215.3. Samples: 727273948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:32,363][19170] Avg episode reward: [(0, '941.610'), (1, '980.580')] [2023-12-27 15:29:32,397][20381] Updated weights for policy 1, policy_version 1419624 (0.0008) [2023-12-27 15:29:32,800][20347] Updated weights for policy 0, policy_version 1420877 (0.0006) [2023-12-27 15:29:32,856][20347] Updated weights for policy 0, policy_version 1420887 (0.0005) [2023-12-27 15:29:32,915][20347] Updated weights for policy 0, policy_version 1420897 (0.0005) [2023-12-27 15:29:33,017][20381] Updated weights for policy 1, policy_version 1419634 (0.0007) [2023-12-27 15:29:33,081][20381] Updated weights for policy 1, policy_version 1419644 (0.0005) [2023-12-27 15:29:33,137][20381] Updated weights for policy 1, policy_version 1419654 (0.0007) [2023-12-27 15:29:33,184][20381] Updated weights for policy 1, policy_version 1419664 (0.0007) [2023-12-27 15:29:33,488][20347] Updated weights for policy 0, policy_version 1420907 (0.0005) [2023-12-27 15:29:33,546][20347] Updated weights for policy 0, policy_version 1420917 (0.0007) [2023-12-27 15:29:33,598][20347] Updated weights for policy 0, policy_version 1420927 (0.0008) [2023-12-27 15:29:33,852][20381] Updated weights for policy 1, policy_version 1419674 (0.0007) [2023-12-27 15:29:33,901][20381] Updated weights for policy 1, policy_version 1419684 (0.0008) [2023-12-27 15:29:33,949][20381] Updated weights for policy 1, policy_version 1419694 (0.0005) [2023-12-27 15:29:34,235][20347] Updated weights for policy 0, policy_version 1420937 (0.0008) [2023-12-27 15:29:34,296][20347] Updated weights for policy 0, policy_version 1420947 (0.0007) [2023-12-27 15:29:34,353][20347] Updated weights for policy 0, policy_version 1420957 (0.0007) [2023-12-27 15:29:34,407][20347] Updated weights for policy 0, policy_version 1420967 (0.0008) [2023-12-27 15:29:34,627][20381] Updated weights for policy 1, policy_version 1419704 (0.0008) [2023-12-27 15:29:34,681][20381] Updated weights for policy 1, policy_version 1419714 (0.0008) [2023-12-27 15:29:34,732][20292] KL-divergence is very high: 104.6737 [2023-12-27 15:29:34,738][20381] Updated weights for policy 1, policy_version 1419724 (0.0008) [2023-12-27 15:29:34,742][20292] KL-divergence is very high: 103.4143 [2023-12-27 15:29:35,108][20347] Updated weights for policy 0, policy_version 1420977 (0.0007) [2023-12-27 15:29:35,172][20347] Updated weights for policy 0, policy_version 1420987 (0.0008) [2023-12-27 15:29:35,237][20347] Updated weights for policy 0, policy_version 1420997 (0.0007) [2023-12-27 15:29:35,394][20381] Updated weights for policy 1, policy_version 1419734 (0.0006) [2023-12-27 15:29:35,452][20381] Updated weights for policy 1, policy_version 1419744 (0.0005) [2023-12-27 15:29:35,517][20381] Updated weights for policy 1, policy_version 1419754 (0.0005) [2023-12-27 15:29:35,967][20347] Updated weights for policy 0, policy_version 1421007 (0.0006) [2023-12-27 15:29:36,025][20347] Updated weights for policy 0, policy_version 1421017 (0.0007) [2023-12-27 15:29:36,080][20347] Updated weights for policy 0, policy_version 1421027 (0.0010) [2023-12-27 15:29:36,086][20381] Updated weights for policy 1, policy_version 1419764 (0.0008) [2023-12-27 15:29:36,126][20292] KL-divergence is very high: 114.5212 [2023-12-27 15:29:36,152][20381] Updated weights for policy 1, policy_version 1419774 (0.0008) [2023-12-27 15:29:36,173][20292] KL-divergence is very high: 116.0078 [2023-12-27 15:29:36,206][20381] Updated weights for policy 1, policy_version 1419784 (0.0008) [2023-12-27 15:29:36,738][20347] Updated weights for policy 0, policy_version 1421037 (0.0011) [2023-12-27 15:29:36,795][20347] Updated weights for policy 0, policy_version 1421047 (0.0010) [2023-12-27 15:29:36,850][20347] Updated weights for policy 0, policy_version 1421057 (0.0010) [2023-12-27 15:29:36,906][20381] Updated weights for policy 1, policy_version 1419794 (0.0008) [2023-12-27 15:29:36,965][20381] Updated weights for policy 1, policy_version 1419804 (0.0011) [2023-12-27 15:29:37,024][20381] Updated weights for policy 1, policy_version 1419814 (0.0010) [2023-12-27 15:29:37,087][20381] Updated weights for policy 1, policy_version 1419824 (0.0011) [2023-12-27 15:29:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 727367680. Throughput: 0: 10174.0, 1: 10254.5. Samples: 727336884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:37,364][19170] Avg episode reward: [(0, '951.270'), (1, '953.110')] [2023-12-27 15:29:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001421064_363864064.pth... [2023-12-27 15:29:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001419824_363503616.pth... [2023-12-27 15:29:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001418608_363192320.pth [2023-12-27 15:29:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001419880_363560960.pth [2023-12-27 15:29:37,497][20347] Updated weights for policy 0, policy_version 1421067 (0.0010) [2023-12-27 15:29:37,559][20347] Updated weights for policy 0, policy_version 1421077 (0.0010) [2023-12-27 15:29:37,622][20347] Updated weights for policy 0, policy_version 1421087 (0.0010) [2023-12-27 15:29:37,782][20381] Updated weights for policy 1, policy_version 1419834 (0.0007) [2023-12-27 15:29:37,842][20381] Updated weights for policy 1, policy_version 1419844 (0.0008) [2023-12-27 15:29:37,898][20381] Updated weights for policy 1, policy_version 1419854 (0.0010) [2023-12-27 15:29:38,303][20347] Updated weights for policy 0, policy_version 1421097 (0.0011) [2023-12-27 15:29:38,367][20347] Updated weights for policy 0, policy_version 1421107 (0.0011) [2023-12-27 15:29:38,426][20347] Updated weights for policy 0, policy_version 1421117 (0.0011) [2023-12-27 15:29:38,485][20347] Updated weights for policy 0, policy_version 1421127 (0.0011) [2023-12-27 15:29:38,588][20381] Updated weights for policy 1, policy_version 1419864 (0.0011) [2023-12-27 15:29:38,650][20381] Updated weights for policy 1, policy_version 1419874 (0.0011) [2023-12-27 15:29:38,708][20381] Updated weights for policy 1, policy_version 1419884 (0.0010) [2023-12-27 15:29:39,167][20347] Updated weights for policy 0, policy_version 1421137 (0.0008) [2023-12-27 15:29:39,225][20347] Updated weights for policy 0, policy_version 1421147 (0.0006) [2023-12-27 15:29:39,281][20347] Updated weights for policy 0, policy_version 1421157 (0.0008) [2023-12-27 15:29:39,349][20381] Updated weights for policy 1, policy_version 1419894 (0.0009) [2023-12-27 15:29:39,389][20292] KL-divergence is very high: 152.1645 [2023-12-27 15:29:39,417][20381] Updated weights for policy 1, policy_version 1419904 (0.0012) [2023-12-27 15:29:39,446][20292] KL-divergence is very high: 142.0432 [2023-12-27 15:29:39,487][20381] Updated weights for policy 1, policy_version 1419914 (0.0011) [2023-12-27 15:29:39,968][20347] Updated weights for policy 0, policy_version 1421167 (0.0008) [2023-12-27 15:29:40,025][20347] Updated weights for policy 0, policy_version 1421177 (0.0008) [2023-12-27 15:29:40,082][20347] Updated weights for policy 0, policy_version 1421187 (0.0009) [2023-12-27 15:29:40,156][20381] Updated weights for policy 1, policy_version 1419924 (0.0009) [2023-12-27 15:29:40,225][20381] Updated weights for policy 1, policy_version 1419934 (0.0006) [2023-12-27 15:29:40,287][20381] Updated weights for policy 1, policy_version 1419944 (0.0007) [2023-12-27 15:29:40,800][20347] Updated weights for policy 0, policy_version 1421197 (0.0007) [2023-12-27 15:29:40,848][20347] Updated weights for policy 0, policy_version 1421207 (0.0005) [2023-12-27 15:29:40,896][20347] Updated weights for policy 0, policy_version 1421217 (0.0005) [2023-12-27 15:29:40,957][20381] Updated weights for policy 1, policy_version 1419954 (0.0008) [2023-12-27 15:29:41,022][20381] Updated weights for policy 1, policy_version 1419964 (0.0008) [2023-12-27 15:29:41,090][20381] Updated weights for policy 1, policy_version 1419974 (0.0007) [2023-12-27 15:29:41,156][20381] Updated weights for policy 1, policy_version 1419984 (0.0008) [2023-12-27 15:29:41,634][20347] Updated weights for policy 0, policy_version 1421227 (0.0006) [2023-12-27 15:29:41,688][20347] Updated weights for policy 0, policy_version 1421237 (0.0007) [2023-12-27 15:29:41,745][20347] Updated weights for policy 0, policy_version 1421247 (0.0007) [2023-12-27 15:29:41,955][20381] Updated weights for policy 1, policy_version 1419994 (0.0007) [2023-12-27 15:29:42,019][20381] Updated weights for policy 1, policy_version 1420004 (0.0007) [2023-12-27 15:29:42,080][20381] Updated weights for policy 1, policy_version 1420014 (0.0011) [2023-12-27 15:29:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 727465984. Throughput: 0: 10148.2, 1: 10277.8. Samples: 727458352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:42,363][19170] Avg episode reward: [(0, '980.700'), (1, '933.700')] [2023-12-27 15:29:42,483][20347] Updated weights for policy 0, policy_version 1421257 (0.0007) [2023-12-27 15:29:42,544][20347] Updated weights for policy 0, policy_version 1421267 (0.0008) [2023-12-27 15:29:42,597][20347] Updated weights for policy 0, policy_version 1421277 (0.0007) [2023-12-27 15:29:42,655][20347] Updated weights for policy 0, policy_version 1421287 (0.0007) [2023-12-27 15:29:42,779][20381] Updated weights for policy 1, policy_version 1420024 (0.0008) [2023-12-27 15:29:42,839][20381] Updated weights for policy 1, policy_version 1420034 (0.0007) [2023-12-27 15:29:42,902][20381] Updated weights for policy 1, policy_version 1420044 (0.0006) [2023-12-27 15:29:43,404][20347] Updated weights for policy 0, policy_version 1421297 (0.0008) [2023-12-27 15:29:43,441][20381] Updated weights for policy 1, policy_version 1420054 (0.0006) [2023-12-27 15:29:43,459][20347] Updated weights for policy 0, policy_version 1421307 (0.0006) [2023-12-27 15:29:43,501][20381] Updated weights for policy 1, policy_version 1420064 (0.0006) [2023-12-27 15:29:43,516][20347] Updated weights for policy 0, policy_version 1421317 (0.0006) [2023-12-27 15:29:43,557][20381] Updated weights for policy 1, policy_version 1420074 (0.0007) [2023-12-27 15:29:44,144][20347] Updated weights for policy 0, policy_version 1421327 (0.0007) [2023-12-27 15:29:44,192][20347] Updated weights for policy 0, policy_version 1421337 (0.0007) [2023-12-27 15:29:44,219][20381] Updated weights for policy 1, policy_version 1420084 (0.0009) [2023-12-27 15:29:44,245][20347] Updated weights for policy 0, policy_version 1421347 (0.0008) [2023-12-27 15:29:44,278][20381] Updated weights for policy 1, policy_version 1420094 (0.0010) [2023-12-27 15:29:44,330][20381] Updated weights for policy 1, policy_version 1420104 (0.0010) [2023-12-27 15:29:44,999][20381] Updated weights for policy 1, policy_version 1420114 (0.0009) [2023-12-27 15:29:45,034][20347] Updated weights for policy 0, policy_version 1421357 (0.0006) [2023-12-27 15:29:45,056][20381] Updated weights for policy 1, policy_version 1420124 (0.0011) [2023-12-27 15:29:45,089][20347] Updated weights for policy 0, policy_version 1421367 (0.0007) [2023-12-27 15:29:45,112][20381] Updated weights for policy 1, policy_version 1420134 (0.0010) [2023-12-27 15:29:45,151][20347] Updated weights for policy 0, policy_version 1421377 (0.0007) [2023-12-27 15:29:45,176][20381] Updated weights for policy 1, policy_version 1420144 (0.0007) [2023-12-27 15:29:45,786][20381] Updated weights for policy 1, policy_version 1420154 (0.0010) [2023-12-27 15:29:45,817][20292] KL-divergence is very high: 206.7706 [2023-12-27 15:29:45,842][20347] Updated weights for policy 0, policy_version 1421387 (0.0007) [2023-12-27 15:29:45,847][20381] Updated weights for policy 1, policy_version 1420164 (0.0010) [2023-12-27 15:29:45,865][20292] KL-divergence is very high: 234.0496 [2023-12-27 15:29:45,895][20347] Updated weights for policy 0, policy_version 1421397 (0.0005) [2023-12-27 15:29:45,905][20381] Updated weights for policy 1, policy_version 1420174 (0.0011) [2023-12-27 15:29:45,912][20292] KL-divergence is very high: 125.3392 [2023-12-27 15:29:45,947][20347] Updated weights for policy 0, policy_version 1421407 (0.0008) [2023-12-27 15:29:46,577][20381] Updated weights for policy 1, policy_version 1420184 (0.0010) [2023-12-27 15:29:46,583][20347] Updated weights for policy 0, policy_version 1421417 (0.0006) [2023-12-27 15:29:46,634][20347] Updated weights for policy 0, policy_version 1421427 (0.0006) [2023-12-27 15:29:46,635][20381] Updated weights for policy 1, policy_version 1420194 (0.0010) [2023-12-27 15:29:46,688][20347] Updated weights for policy 0, policy_version 1421437 (0.0005) [2023-12-27 15:29:46,694][20381] Updated weights for policy 1, policy_version 1420204 (0.0011) [2023-12-27 15:29:46,744][20347] Updated weights for policy 0, policy_version 1421447 (0.0005) [2023-12-27 15:29:47,348][20381] Updated weights for policy 1, policy_version 1420214 (0.0011) [2023-12-27 15:29:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 727564288. Throughput: 0: 10179.8, 1: 10314.8. Samples: 727580740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:47,363][19170] Avg episode reward: [(0, '970.950'), (1, '941.640')] [2023-12-27 15:29:47,390][20347] Updated weights for policy 0, policy_version 1421457 (0.0010) [2023-12-27 15:29:47,404][20381] Updated weights for policy 1, policy_version 1420224 (0.0011) [2023-12-27 15:29:47,450][20347] Updated weights for policy 0, policy_version 1421467 (0.0006) [2023-12-27 15:29:47,457][20381] Updated weights for policy 1, policy_version 1420234 (0.0011) [2023-12-27 15:29:47,509][20347] Updated weights for policy 0, policy_version 1421477 (0.0006) [2023-12-27 15:29:48,118][20381] Updated weights for policy 1, policy_version 1420244 (0.0011) [2023-12-27 15:29:48,173][20347] Updated weights for policy 0, policy_version 1421487 (0.0009) [2023-12-27 15:29:48,178][20381] Updated weights for policy 1, policy_version 1420254 (0.0011) [2023-12-27 15:29:48,222][20347] Updated weights for policy 0, policy_version 1421497 (0.0010) [2023-12-27 15:29:48,233][20381] Updated weights for policy 1, policy_version 1420264 (0.0011) [2023-12-27 15:29:48,272][20347] Updated weights for policy 0, policy_version 1421507 (0.0010) [2023-12-27 15:29:48,968][20347] Updated weights for policy 0, policy_version 1421517 (0.0010) [2023-12-27 15:29:48,975][20381] Updated weights for policy 1, policy_version 1420274 (0.0011) [2023-12-27 15:29:49,024][20347] Updated weights for policy 0, policy_version 1421527 (0.0010) [2023-12-27 15:29:49,033][20381] Updated weights for policy 1, policy_version 1420284 (0.0009) [2023-12-27 15:29:49,080][20347] Updated weights for policy 0, policy_version 1421537 (0.0010) [2023-12-27 15:29:49,099][20381] Updated weights for policy 1, policy_version 1420294 (0.0006) [2023-12-27 15:29:49,160][20381] Updated weights for policy 1, policy_version 1420304 (0.0005) [2023-12-27 15:29:49,786][20347] Updated weights for policy 0, policy_version 1421547 (0.0009) [2023-12-27 15:29:49,816][20291] KL-divergence is very high: 324.1112 [2023-12-27 15:29:49,838][20381] Updated weights for policy 1, policy_version 1420314 (0.0008) [2023-12-27 15:29:49,852][20347] Updated weights for policy 0, policy_version 1421557 (0.0008) [2023-12-27 15:29:49,871][20291] KL-divergence is very high: 374.0216 [2023-12-27 15:29:49,902][20381] Updated weights for policy 1, policy_version 1420324 (0.0008) [2023-12-27 15:29:49,917][20347] Updated weights for policy 0, policy_version 1421567 (0.0008) [2023-12-27 15:29:49,923][20291] KL-divergence is very high: 171.6824 [2023-12-27 15:29:49,964][20381] Updated weights for policy 1, policy_version 1420334 (0.0009) [2023-12-27 15:29:50,629][20347] Updated weights for policy 0, policy_version 1421577 (0.0010) [2023-12-27 15:29:50,688][20347] Updated weights for policy 0, policy_version 1421587 (0.0010) [2023-12-27 15:29:50,701][20381] Updated weights for policy 1, policy_version 1420344 (0.0011) [2023-12-27 15:29:50,749][20347] Updated weights for policy 0, policy_version 1421597 (0.0008) [2023-12-27 15:29:50,762][20381] Updated weights for policy 1, policy_version 1420354 (0.0011) [2023-12-27 15:29:50,809][20347] Updated weights for policy 0, policy_version 1421607 (0.0008) [2023-12-27 15:29:50,815][20381] Updated weights for policy 1, policy_version 1420364 (0.0011) [2023-12-27 15:29:51,507][20347] Updated weights for policy 0, policy_version 1421617 (0.0008) [2023-12-27 15:29:51,539][20381] Updated weights for policy 1, policy_version 1420374 (0.0009) [2023-12-27 15:29:51,571][20347] Updated weights for policy 0, policy_version 1421627 (0.0006) [2023-12-27 15:29:51,589][20381] Updated weights for policy 1, policy_version 1420384 (0.0006) [2023-12-27 15:29:51,636][20347] Updated weights for policy 0, policy_version 1421637 (0.0008) [2023-12-27 15:29:51,657][20381] Updated weights for policy 1, policy_version 1420394 (0.0008) [2023-12-27 15:29:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 727662592. Throughput: 0: 10191.5, 1: 10291.1. Samples: 727640396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:52,363][19170] Avg episode reward: [(0, '970.720'), (1, '961.060')] [2023-12-27 15:29:52,367][20347] Updated weights for policy 0, policy_version 1421649 (0.0008) [2023-12-27 15:29:52,377][20381] Updated weights for policy 1, policy_version 1420404 (0.0008) [2023-12-27 15:29:52,431][20347] Updated weights for policy 0, policy_version 1421659 (0.0006) [2023-12-27 15:29:52,432][20381] Updated weights for policy 1, policy_version 1420414 (0.0008) [2023-12-27 15:29:52,485][20347] Updated weights for policy 0, policy_version 1421669 (0.0005) [2023-12-27 15:29:52,495][20381] Updated weights for policy 1, policy_version 1420424 (0.0007) [2023-12-27 15:29:52,500][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001421672_364019712.pth... [2023-12-27 15:29:52,505][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001420456_363708416.pth [2023-12-27 15:29:52,540][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001420432_363659264.pth... [2023-12-27 15:29:52,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001419216_363347968.pth [2023-12-27 15:29:53,120][20381] Updated weights for policy 1, policy_version 1420434 (0.0008) [2023-12-27 15:29:53,162][20347] Updated weights for policy 0, policy_version 1421679 (0.0005) [2023-12-27 15:29:53,181][20381] Updated weights for policy 1, policy_version 1420444 (0.0007) [2023-12-27 15:29:53,217][20347] Updated weights for policy 0, policy_version 1421689 (0.0005) [2023-12-27 15:29:53,242][20381] Updated weights for policy 1, policy_version 1420454 (0.0008) [2023-12-27 15:29:53,277][20347] Updated weights for policy 0, policy_version 1421699 (0.0005) [2023-12-27 15:29:53,304][20381] Updated weights for policy 1, policy_version 1420464 (0.0006) [2023-12-27 15:29:53,886][20347] Updated weights for policy 0, policy_version 1421709 (0.0006) [2023-12-27 15:29:53,944][20347] Updated weights for policy 0, policy_version 1421719 (0.0007) [2023-12-27 15:29:53,991][20381] Updated weights for policy 1, policy_version 1420474 (0.0009) [2023-12-27 15:29:54,009][20347] Updated weights for policy 0, policy_version 1421729 (0.0006) [2023-12-27 15:29:54,050][20381] Updated weights for policy 1, policy_version 1420484 (0.0008) [2023-12-27 15:29:54,120][20381] Updated weights for policy 1, policy_version 1420494 (0.0007) [2023-12-27 15:29:54,694][20347] Updated weights for policy 0, policy_version 1421739 (0.0008) [2023-12-27 15:29:54,748][20347] Updated weights for policy 0, policy_version 1421749 (0.0008) [2023-12-27 15:29:54,778][20381] Updated weights for policy 1, policy_version 1420504 (0.0010) [2023-12-27 15:29:54,799][20347] Updated weights for policy 0, policy_version 1421759 (0.0005) [2023-12-27 15:29:54,824][20381] Updated weights for policy 1, policy_version 1420514 (0.0010) [2023-12-27 15:29:54,868][20381] Updated weights for policy 1, policy_version 1420524 (0.0010) [2023-12-27 15:29:55,514][20347] Updated weights for policy 0, policy_version 1421769 (0.0005) [2023-12-27 15:29:55,542][20381] Updated weights for policy 1, policy_version 1420534 (0.0008) [2023-12-27 15:29:55,566][20347] Updated weights for policy 0, policy_version 1421779 (0.0006) [2023-12-27 15:29:55,591][20381] Updated weights for policy 1, policy_version 1420544 (0.0006) [2023-12-27 15:29:55,623][20347] Updated weights for policy 0, policy_version 1421789 (0.0007) [2023-12-27 15:29:55,649][20381] Updated weights for policy 1, policy_version 1420554 (0.0008) [2023-12-27 15:29:55,679][20347] Updated weights for policy 0, policy_version 1421799 (0.0006) [2023-12-27 15:29:56,294][20381] Updated weights for policy 1, policy_version 1420564 (0.0007) [2023-12-27 15:29:56,333][20347] Updated weights for policy 0, policy_version 1421809 (0.0005) [2023-12-27 15:29:56,342][20381] Updated weights for policy 1, policy_version 1420574 (0.0006) [2023-12-27 15:29:56,380][20347] Updated weights for policy 0, policy_version 1421819 (0.0008) [2023-12-27 15:29:56,395][20381] Updated weights for policy 1, policy_version 1420584 (0.0008) [2023-12-27 15:29:56,435][20347] Updated weights for policy 0, policy_version 1421829 (0.0006) [2023-12-27 15:29:57,053][20381] Updated weights for policy 1, policy_version 1420594 (0.0007) [2023-12-27 15:29:57,113][20381] Updated weights for policy 1, policy_version 1420604 (0.0008) [2023-12-27 15:29:57,135][20347] Updated weights for policy 0, policy_version 1421839 (0.0008) [2023-12-27 15:29:57,161][20381] Updated weights for policy 1, policy_version 1420614 (0.0008) [2023-12-27 15:29:57,188][20347] Updated weights for policy 0, policy_version 1421849 (0.0007) [2023-12-27 15:29:57,207][20381] Updated weights for policy 1, policy_version 1420624 (0.0005) [2023-12-27 15:29:57,241][20347] Updated weights for policy 0, policy_version 1421859 (0.0008) [2023-12-27 15:29:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 727777280. Throughput: 0: 10222.4, 1: 10285.2. Samples: 727762124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:29:57,363][19170] Avg episode reward: [(0, '970.680'), (1, '990.220')] [2023-12-27 15:29:57,829][20381] Updated weights for policy 1, policy_version 1420634 (0.0007) [2023-12-27 15:29:57,884][20381] Updated weights for policy 1, policy_version 1420644 (0.0007) [2023-12-27 15:29:57,941][20381] Updated weights for policy 1, policy_version 1420654 (0.0007) [2023-12-27 15:29:57,951][20347] Updated weights for policy 0, policy_version 1421869 (0.0007) [2023-12-27 15:29:57,999][20347] Updated weights for policy 0, policy_version 1421879 (0.0009) [2023-12-27 15:29:58,059][20347] Updated weights for policy 0, policy_version 1421889 (0.0010) [2023-12-27 15:29:58,701][20381] Updated weights for policy 1, policy_version 1420664 (0.0006) [2023-12-27 15:29:58,768][20381] Updated weights for policy 1, policy_version 1420674 (0.0009) [2023-12-27 15:29:58,787][20347] Updated weights for policy 0, policy_version 1421899 (0.0009) [2023-12-27 15:29:58,834][20381] Updated weights for policy 1, policy_version 1420684 (0.0008) [2023-12-27 15:29:58,866][20347] Updated weights for policy 0, policy_version 1421909 (0.0007) [2023-12-27 15:29:58,932][20347] Updated weights for policy 0, policy_version 1421919 (0.0007) [2023-12-27 15:29:59,625][20347] Updated weights for policy 0, policy_version 1421929 (0.0006) [2023-12-27 15:29:59,638][20381] Updated weights for policy 1, policy_version 1420694 (0.0007) [2023-12-27 15:29:59,681][20347] Updated weights for policy 0, policy_version 1421939 (0.0005) [2023-12-27 15:29:59,698][20381] Updated weights for policy 1, policy_version 1420704 (0.0006) [2023-12-27 15:29:59,737][20347] Updated weights for policy 0, policy_version 1421949 (0.0005) [2023-12-27 15:29:59,756][20381] Updated weights for policy 1, policy_version 1420714 (0.0007) [2023-12-27 15:29:59,792][20347] Updated weights for policy 0, policy_version 1421959 (0.0007) [2023-12-27 15:30:00,382][20381] Updated weights for policy 1, policy_version 1420724 (0.0007) [2023-12-27 15:30:00,431][20381] Updated weights for policy 1, policy_version 1420734 (0.0008) [2023-12-27 15:30:00,491][20381] Updated weights for policy 1, policy_version 1420744 (0.0007) [2023-12-27 15:30:00,495][20347] Updated weights for policy 0, policy_version 1421969 (0.0007) [2023-12-27 15:30:00,551][20347] Updated weights for policy 0, policy_version 1421979 (0.0007) [2023-12-27 15:30:00,608][20347] Updated weights for policy 0, policy_version 1421989 (0.0008) [2023-12-27 15:30:01,082][20381] Updated weights for policy 1, policy_version 1420754 (0.0006) [2023-12-27 15:30:01,147][20381] Updated weights for policy 1, policy_version 1420764 (0.0008) [2023-12-27 15:30:01,208][20381] Updated weights for policy 1, policy_version 1420774 (0.0008) [2023-12-27 15:30:01,269][20381] Updated weights for policy 1, policy_version 1420784 (0.0008) [2023-12-27 15:30:01,323][20347] Updated weights for policy 0, policy_version 1421999 (0.0006) [2023-12-27 15:30:01,387][20347] Updated weights for policy 0, policy_version 1422009 (0.0007) [2023-12-27 15:30:01,444][20347] Updated weights for policy 0, policy_version 1422019 (0.0006) [2023-12-27 15:30:01,975][20381] Updated weights for policy 1, policy_version 1420794 (0.0008) [2023-12-27 15:30:02,030][20381] Updated weights for policy 1, policy_version 1420804 (0.0007) [2023-12-27 15:30:02,091][20381] Updated weights for policy 1, policy_version 1420814 (0.0007) [2023-12-27 15:30:02,129][20347] Updated weights for policy 0, policy_version 1422029 (0.0007) [2023-12-27 15:30:02,184][20347] Updated weights for policy 0, policy_version 1422039 (0.0007) [2023-12-27 15:30:02,245][20347] Updated weights for policy 0, policy_version 1422049 (0.0007) [2023-12-27 15:30:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 727875584. Throughput: 0: 10152.8, 1: 10283.0. Samples: 727881504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:30:02,363][19170] Avg episode reward: [(0, '970.680'), (1, '990.220')] [2023-12-27 15:30:02,752][20381] Updated weights for policy 1, policy_version 1420824 (0.0007) [2023-12-27 15:30:02,803][20381] Updated weights for policy 1, policy_version 1420834 (0.0007) [2023-12-27 15:30:02,855][20381] Updated weights for policy 1, policy_version 1420844 (0.0007) [2023-12-27 15:30:02,939][20347] Updated weights for policy 0, policy_version 1422059 (0.0007) [2023-12-27 15:30:02,985][20347] Updated weights for policy 0, policy_version 1422069 (0.0006) [2023-12-27 15:30:03,032][20347] Updated weights for policy 0, policy_version 1422079 (0.0007) [2023-12-27 15:30:03,474][20381] Updated weights for policy 1, policy_version 1420854 (0.0007) [2023-12-27 15:30:03,527][20381] Updated weights for policy 1, policy_version 1420864 (0.0007) [2023-12-27 15:30:03,571][20381] Updated weights for policy 1, policy_version 1420874 (0.0007) [2023-12-27 15:30:03,670][20347] Updated weights for policy 0, policy_version 1422089 (0.0007) [2023-12-27 15:30:03,718][20347] Updated weights for policy 0, policy_version 1422099 (0.0010) [2023-12-27 15:30:03,775][20347] Updated weights for policy 0, policy_version 1422109 (0.0010) [2023-12-27 15:30:03,829][20347] Updated weights for policy 0, policy_version 1422119 (0.0010) [2023-12-27 15:30:04,251][20381] Updated weights for policy 1, policy_version 1420884 (0.0007) [2023-12-27 15:30:04,313][20381] Updated weights for policy 1, policy_version 1420894 (0.0008) [2023-12-27 15:30:04,371][20381] Updated weights for policy 1, policy_version 1420904 (0.0007) [2023-12-27 15:30:04,534][20347] Updated weights for policy 0, policy_version 1422129 (0.0010) [2023-12-27 15:30:04,590][20347] Updated weights for policy 0, policy_version 1422139 (0.0011) [2023-12-27 15:30:04,643][20347] Updated weights for policy 0, policy_version 1422149 (0.0010) [2023-12-27 15:30:05,028][20381] Updated weights for policy 1, policy_version 1420914 (0.0007) [2023-12-27 15:30:05,081][20381] Updated weights for policy 1, policy_version 1420924 (0.0005) [2023-12-27 15:30:05,134][20381] Updated weights for policy 1, policy_version 1420934 (0.0005) [2023-12-27 15:30:05,184][20381] Updated weights for policy 1, policy_version 1420944 (0.0005) [2023-12-27 15:30:05,333][20347] Updated weights for policy 0, policy_version 1422159 (0.0010) [2023-12-27 15:30:05,394][20347] Updated weights for policy 0, policy_version 1422169 (0.0010) [2023-12-27 15:30:05,444][20347] Updated weights for policy 0, policy_version 1422179 (0.0010) [2023-12-27 15:30:05,837][20381] Updated weights for policy 1, policy_version 1420954 (0.0006) [2023-12-27 15:30:05,892][20381] Updated weights for policy 1, policy_version 1420964 (0.0005) [2023-12-27 15:30:05,945][20381] Updated weights for policy 1, policy_version 1420974 (0.0005) [2023-12-27 15:30:06,052][20347] Updated weights for policy 0, policy_version 1422189 (0.0008) [2023-12-27 15:30:06,109][20347] Updated weights for policy 0, policy_version 1422199 (0.0006) [2023-12-27 15:30:06,171][20347] Updated weights for policy 0, policy_version 1422209 (0.0008) [2023-12-27 15:30:06,613][20381] Updated weights for policy 1, policy_version 1420984 (0.0007) [2023-12-27 15:30:06,677][20381] Updated weights for policy 1, policy_version 1420994 (0.0006) [2023-12-27 15:30:06,735][20381] Updated weights for policy 1, policy_version 1421004 (0.0006) [2023-12-27 15:30:06,843][20347] Updated weights for policy 0, policy_version 1422219 (0.0009) [2023-12-27 15:30:06,896][20347] Updated weights for policy 0, policy_version 1422229 (0.0007) [2023-12-27 15:30:06,948][20347] Updated weights for policy 0, policy_version 1422239 (0.0007) [2023-12-27 15:30:07,329][20381] Updated weights for policy 1, policy_version 1421014 (0.0005) [2023-12-27 15:30:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 727973888. Throughput: 0: 10135.0, 1: 10306.9. Samples: 727944384. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:07,363][19170] Avg episode reward: [(0, '980.500'), (1, '990.220')] [2023-12-27 15:30:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001422248_364167168.pth... [2023-12-27 15:30:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001421064_363864064.pth [2023-12-27 15:30:07,392][20381] Updated weights for policy 1, policy_version 1421024 (0.0006) [2023-12-27 15:30:07,459][20381] Updated weights for policy 1, policy_version 1421034 (0.0007) [2023-12-27 15:30:07,495][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001421040_363814912.pth... [2023-12-27 15:30:07,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001419824_363503616.pth [2023-12-27 15:30:07,569][20347] Updated weights for policy 0, policy_version 1422249 (0.0006) [2023-12-27 15:30:07,623][20347] Updated weights for policy 0, policy_version 1422259 (0.0008) [2023-12-27 15:30:07,676][20347] Updated weights for policy 0, policy_version 1422269 (0.0007) [2023-12-27 15:30:07,741][20347] Updated weights for policy 0, policy_version 1422279 (0.0008) [2023-12-27 15:30:08,063][20381] Updated weights for policy 1, policy_version 1421044 (0.0007) [2023-12-27 15:30:08,124][20381] Updated weights for policy 1, policy_version 1421054 (0.0007) [2023-12-27 15:30:08,183][20381] Updated weights for policy 1, policy_version 1421064 (0.0008) [2023-12-27 15:30:08,483][20347] Updated weights for policy 0, policy_version 1422289 (0.0007) [2023-12-27 15:30:08,545][20347] Updated weights for policy 0, policy_version 1422299 (0.0007) [2023-12-27 15:30:08,603][20347] Updated weights for policy 0, policy_version 1422309 (0.0007) [2023-12-27 15:30:08,819][20381] Updated weights for policy 1, policy_version 1421074 (0.0008) [2023-12-27 15:30:08,877][20381] Updated weights for policy 1, policy_version 1421084 (0.0008) [2023-12-27 15:30:08,931][20381] Updated weights for policy 1, policy_version 1421094 (0.0008) [2023-12-27 15:30:08,980][20381] Updated weights for policy 1, policy_version 1421104 (0.0008) [2023-12-27 15:30:09,180][20347] Updated weights for policy 0, policy_version 1422319 (0.0007) [2023-12-27 15:30:09,255][20347] Updated weights for policy 0, policy_version 1422330 (0.0011) [2023-12-27 15:30:09,323][20347] Updated weights for policy 0, policy_version 1422340 (0.0007) [2023-12-27 15:30:09,735][20381] Updated weights for policy 1, policy_version 1421114 (0.0007) [2023-12-27 15:30:09,796][20381] Updated weights for policy 1, policy_version 1421124 (0.0008) [2023-12-27 15:30:09,864][20381] Updated weights for policy 1, policy_version 1421134 (0.0010) [2023-12-27 15:30:10,005][20347] Updated weights for policy 0, policy_version 1422350 (0.0007) [2023-12-27 15:30:10,065][20347] Updated weights for policy 0, policy_version 1422360 (0.0008) [2023-12-27 15:30:10,122][20347] Updated weights for policy 0, policy_version 1422370 (0.0007) [2023-12-27 15:30:10,579][20381] Updated weights for policy 1, policy_version 1421144 (0.0010) [2023-12-27 15:30:10,628][20381] Updated weights for policy 1, policy_version 1421154 (0.0005) [2023-12-27 15:30:10,686][20381] Updated weights for policy 1, policy_version 1421164 (0.0005) [2023-12-27 15:30:10,865][20347] Updated weights for policy 0, policy_version 1422380 (0.0008) [2023-12-27 15:30:10,921][20347] Updated weights for policy 0, policy_version 1422390 (0.0007) [2023-12-27 15:30:10,979][20347] Updated weights for policy 0, policy_version 1422400 (0.0007) [2023-12-27 15:30:11,378][20381] Updated weights for policy 1, policy_version 1421174 (0.0007) [2023-12-27 15:30:11,470][20381] Updated weights for policy 1, policy_version 1421184 (0.0009) [2023-12-27 15:30:11,536][20381] Updated weights for policy 1, policy_version 1421194 (0.0009) [2023-12-27 15:30:11,847][20347] Updated weights for policy 0, policy_version 1422410 (0.0008) [2023-12-27 15:30:11,909][20347] Updated weights for policy 0, policy_version 1422420 (0.0009) [2023-12-27 15:30:11,967][20347] Updated weights for policy 0, policy_version 1422430 (0.0009) [2023-12-27 15:30:12,017][20347] Updated weights for policy 0, policy_version 1422440 (0.0008) [2023-12-27 15:30:12,286][20381] Updated weights for policy 1, policy_version 1421204 (0.0009) [2023-12-27 15:30:12,345][20381] Updated weights for policy 1, policy_version 1421214 (0.0008) [2023-12-27 15:30:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 728072192. Throughput: 0: 10164.4, 1: 10303.8. Samples: 728068444. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:12,363][19170] Avg episode reward: [(0, '970.680'), (1, '1000.000')] [2023-12-27 15:30:12,410][20381] Updated weights for policy 1, policy_version 1421224 (0.0008) [2023-12-27 15:30:12,809][20347] Updated weights for policy 0, policy_version 1422450 (0.0006) [2023-12-27 15:30:12,868][20347] Updated weights for policy 0, policy_version 1422460 (0.0006) [2023-12-27 15:30:12,931][20347] Updated weights for policy 0, policy_version 1422470 (0.0005) [2023-12-27 15:30:13,036][20381] Updated weights for policy 1, policy_version 1421234 (0.0008) [2023-12-27 15:30:13,095][20381] Updated weights for policy 1, policy_version 1421244 (0.0007) [2023-12-27 15:30:13,154][20381] Updated weights for policy 1, policy_version 1421254 (0.0008) [2023-12-27 15:30:13,220][20381] Updated weights for policy 1, policy_version 1421264 (0.0009) [2023-12-27 15:30:13,532][20347] Updated weights for policy 0, policy_version 1422480 (0.0008) [2023-12-27 15:30:13,586][20347] Updated weights for policy 0, policy_version 1422490 (0.0010) [2023-12-27 15:30:13,638][20347] Updated weights for policy 0, policy_version 1422500 (0.0010) [2023-12-27 15:30:13,819][20381] Updated weights for policy 1, policy_version 1421274 (0.0005) [2023-12-27 15:30:13,867][20381] Updated weights for policy 1, policy_version 1421284 (0.0005) [2023-12-27 15:30:13,918][20381] Updated weights for policy 1, policy_version 1421294 (0.0005) [2023-12-27 15:30:14,234][20347] Updated weights for policy 0, policy_version 1422510 (0.0009) [2023-12-27 15:30:14,301][20347] Updated weights for policy 0, policy_version 1422520 (0.0006) [2023-12-27 15:30:14,367][20347] Updated weights for policy 0, policy_version 1422530 (0.0010) [2023-12-27 15:30:14,463][20381] Updated weights for policy 1, policy_version 1421304 (0.0009) [2023-12-27 15:30:14,521][20381] Updated weights for policy 1, policy_version 1421314 (0.0010) [2023-12-27 15:30:14,574][20381] Updated weights for policy 1, policy_version 1421324 (0.0011) [2023-12-27 15:30:14,971][20347] Updated weights for policy 0, policy_version 1422540 (0.0010) [2023-12-27 15:30:15,032][20347] Updated weights for policy 0, policy_version 1422550 (0.0011) [2023-12-27 15:30:15,100][20347] Updated weights for policy 0, policy_version 1422560 (0.0010) [2023-12-27 15:30:15,284][20381] Updated weights for policy 1, policy_version 1421334 (0.0009) [2023-12-27 15:30:15,328][20381] Updated weights for policy 1, policy_version 1421344 (0.0008) [2023-12-27 15:30:15,372][20381] Updated weights for policy 1, policy_version 1421354 (0.0008) [2023-12-27 15:30:15,782][20347] Updated weights for policy 0, policy_version 1422570 (0.0011) [2023-12-27 15:30:15,837][20347] Updated weights for policy 0, policy_version 1422580 (0.0010) [2023-12-27 15:30:15,897][20347] Updated weights for policy 0, policy_version 1422590 (0.0010) [2023-12-27 15:30:15,952][20347] Updated weights for policy 0, policy_version 1422600 (0.0010) [2023-12-27 15:30:16,115][20381] Updated weights for policy 1, policy_version 1421364 (0.0009) [2023-12-27 15:30:16,172][20381] Updated weights for policy 1, policy_version 1421374 (0.0010) [2023-12-27 15:30:16,227][20381] Updated weights for policy 1, policy_version 1421384 (0.0011) [2023-12-27 15:30:16,626][20347] Updated weights for policy 0, policy_version 1422610 (0.0007) [2023-12-27 15:30:16,679][20347] Updated weights for policy 0, policy_version 1422620 (0.0010) [2023-12-27 15:30:16,724][20347] Updated weights for policy 0, policy_version 1422630 (0.0010) [2023-12-27 15:30:16,905][20381] Updated weights for policy 1, policy_version 1421394 (0.0010) [2023-12-27 15:30:16,960][20381] Updated weights for policy 1, policy_version 1421404 (0.0010) [2023-12-27 15:30:17,021][20381] Updated weights for policy 1, policy_version 1421414 (0.0011) [2023-12-27 15:30:17,047][20292] KL-divergence is very high: 153.8261 [2023-12-27 15:30:17,062][20292] KL-divergence is very high: 166.0994 [2023-12-27 15:30:17,072][20381] Updated weights for policy 1, policy_version 1421424 (0.0010) [2023-12-27 15:30:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 728178688. Throughput: 0: 10119.6, 1: 10258.0. Samples: 728190940. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:17,363][19170] Avg episode reward: [(0, '951.090'), (1, '941.610')] [2023-12-27 15:30:17,397][20347] Updated weights for policy 0, policy_version 1422640 (0.0010) [2023-12-27 15:30:17,455][20347] Updated weights for policy 0, policy_version 1422650 (0.0010) [2023-12-27 15:30:17,512][20347] Updated weights for policy 0, policy_version 1422660 (0.0010) [2023-12-27 15:30:17,754][20381] Updated weights for policy 1, policy_version 1421434 (0.0010) [2023-12-27 15:30:17,805][20381] Updated weights for policy 1, policy_version 1421444 (0.0010) [2023-12-27 15:30:17,862][20381] Updated weights for policy 1, policy_version 1421454 (0.0010) [2023-12-27 15:30:18,103][20347] Updated weights for policy 0, policy_version 1422670 (0.0007) [2023-12-27 15:30:18,167][20347] Updated weights for policy 0, policy_version 1422680 (0.0006) [2023-12-27 15:30:18,225][20347] Updated weights for policy 0, policy_version 1422690 (0.0007) [2023-12-27 15:30:18,237][20291] KL-divergence is very high: 107.3832 [2023-12-27 15:30:18,526][20381] Updated weights for policy 1, policy_version 1421464 (0.0010) [2023-12-27 15:30:18,581][20381] Updated weights for policy 1, policy_version 1421474 (0.0010) [2023-12-27 15:30:18,639][20381] Updated weights for policy 1, policy_version 1421484 (0.0010) [2023-12-27 15:30:18,900][20347] Updated weights for policy 0, policy_version 1422700 (0.0009) [2023-12-27 15:30:18,959][20347] Updated weights for policy 0, policy_version 1422710 (0.0010) [2023-12-27 15:30:19,011][20347] Updated weights for policy 0, policy_version 1422720 (0.0010) [2023-12-27 15:30:19,035][20291] KL-divergence is very high: 183.3139 [2023-12-27 15:30:19,265][20381] Updated weights for policy 1, policy_version 1421494 (0.0008) [2023-12-27 15:30:19,319][20381] Updated weights for policy 1, policy_version 1421504 (0.0006) [2023-12-27 15:30:19,385][20381] Updated weights for policy 1, policy_version 1421514 (0.0007) [2023-12-27 15:30:19,701][20347] Updated weights for policy 0, policy_version 1422730 (0.0009) [2023-12-27 15:30:19,764][20347] Updated weights for policy 0, policy_version 1422740 (0.0005) [2023-12-27 15:30:19,829][20347] Updated weights for policy 0, policy_version 1422750 (0.0008) [2023-12-27 15:30:19,896][20347] Updated weights for policy 0, policy_version 1422760 (0.0010) [2023-12-27 15:30:20,097][20381] Updated weights for policy 1, policy_version 1421524 (0.0007) [2023-12-27 15:30:20,155][20381] Updated weights for policy 1, policy_version 1421534 (0.0010) [2023-12-27 15:30:20,206][20381] Updated weights for policy 1, policy_version 1421544 (0.0011) [2023-12-27 15:30:20,531][20347] Updated weights for policy 0, policy_version 1422770 (0.0007) [2023-12-27 15:30:20,591][20347] Updated weights for policy 0, policy_version 1422780 (0.0008) [2023-12-27 15:30:20,650][20347] Updated weights for policy 0, policy_version 1422790 (0.0010) [2023-12-27 15:30:20,939][20381] Updated weights for policy 1, policy_version 1421554 (0.0010) [2023-12-27 15:30:20,997][20381] Updated weights for policy 1, policy_version 1421564 (0.0008) [2023-12-27 15:30:21,060][20381] Updated weights for policy 1, policy_version 1421574 (0.0007) [2023-12-27 15:30:21,102][20292] KL-divergence is very high: 207.1734 [2023-12-27 15:30:21,128][20381] Updated weights for policy 1, policy_version 1421584 (0.0008) [2023-12-27 15:30:21,414][20347] Updated weights for policy 0, policy_version 1422800 (0.0009) [2023-12-27 15:30:21,471][20347] Updated weights for policy 0, policy_version 1422810 (0.0007) [2023-12-27 15:30:21,519][20347] Updated weights for policy 0, policy_version 1422820 (0.0007) [2023-12-27 15:30:21,919][20381] Updated weights for policy 1, policy_version 1421594 (0.0008) [2023-12-27 15:30:21,972][20381] Updated weights for policy 1, policy_version 1421604 (0.0007) [2023-12-27 15:30:22,030][20381] Updated weights for policy 1, policy_version 1421614 (0.0008) [2023-12-27 15:30:22,248][20347] Updated weights for policy 0, policy_version 1422830 (0.0007) [2023-12-27 15:30:22,311][20347] Updated weights for policy 0, policy_version 1422840 (0.0010) [2023-12-27 15:30:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 728276992. Throughput: 0: 10102.1, 1: 10248.9. Samples: 728252672. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:22,363][19170] Avg episode reward: [(0, '941.370'), (1, '922.930')] [2023-12-27 15:30:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001421616_363962368.pth... [2023-12-27 15:30:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001420432_363659264.pth [2023-12-27 15:30:22,377][20347] Updated weights for policy 0, policy_version 1422850 (0.0010) [2023-12-27 15:30:22,412][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001422856_364322816.pth... [2023-12-27 15:30:22,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001421672_364019712.pth [2023-12-27 15:30:22,739][20381] Updated weights for policy 1, policy_version 1421624 (0.0007) [2023-12-27 15:30:22,798][20381] Updated weights for policy 1, policy_version 1421634 (0.0007) [2023-12-27 15:30:22,858][20381] Updated weights for policy 1, policy_version 1421644 (0.0008) [2023-12-27 15:30:23,076][20347] Updated weights for policy 0, policy_version 1422860 (0.0011) [2023-12-27 15:30:23,135][20347] Updated weights for policy 0, policy_version 1422870 (0.0010) [2023-12-27 15:30:23,194][20347] Updated weights for policy 0, policy_version 1422880 (0.0009) [2023-12-27 15:30:23,508][20381] Updated weights for policy 1, policy_version 1421654 (0.0006) [2023-12-27 15:30:23,563][20381] Updated weights for policy 1, policy_version 1421664 (0.0006) [2023-12-27 15:30:23,627][20381] Updated weights for policy 1, policy_version 1421674 (0.0006) [2023-12-27 15:30:23,816][20347] Updated weights for policy 0, policy_version 1422890 (0.0009) [2023-12-27 15:30:23,879][20347] Updated weights for policy 0, policy_version 1422900 (0.0005) [2023-12-27 15:30:23,929][20347] Updated weights for policy 0, policy_version 1422910 (0.0006) [2023-12-27 15:30:23,987][20347] Updated weights for policy 0, policy_version 1422920 (0.0005) [2023-12-27 15:30:24,228][20381] Updated weights for policy 1, policy_version 1421684 (0.0008) [2023-12-27 15:30:24,283][20381] Updated weights for policy 1, policy_version 1421694 (0.0011) [2023-12-27 15:30:24,342][20381] Updated weights for policy 1, policy_version 1421704 (0.0011) [2023-12-27 15:30:24,581][20347] Updated weights for policy 0, policy_version 1422930 (0.0006) [2023-12-27 15:30:24,638][20347] Updated weights for policy 0, policy_version 1422940 (0.0006) [2023-12-27 15:30:24,702][20347] Updated weights for policy 0, policy_version 1422950 (0.0010) [2023-12-27 15:30:25,013][20381] Updated weights for policy 1, policy_version 1421714 (0.0010) [2023-12-27 15:30:25,070][20381] Updated weights for policy 1, policy_version 1421724 (0.0007) [2023-12-27 15:30:25,134][20381] Updated weights for policy 1, policy_version 1421734 (0.0007) [2023-12-27 15:30:25,178][20381] Updated weights for policy 1, policy_version 1421744 (0.0007) [2023-12-27 15:30:25,288][20347] Updated weights for policy 0, policy_version 1422960 (0.0010) [2023-12-27 15:30:25,351][20347] Updated weights for policy 0, policy_version 1422970 (0.0010) [2023-12-27 15:30:25,407][20347] Updated weights for policy 0, policy_version 1422980 (0.0006) [2023-12-27 15:30:25,823][20381] Updated weights for policy 1, policy_version 1421754 (0.0007) [2023-12-27 15:30:25,880][20381] Updated weights for policy 1, policy_version 1421764 (0.0007) [2023-12-27 15:30:25,941][20381] Updated weights for policy 1, policy_version 1421774 (0.0008) [2023-12-27 15:30:26,016][20347] Updated weights for policy 0, policy_version 1422990 (0.0006) [2023-12-27 15:30:26,078][20347] Updated weights for policy 0, policy_version 1423000 (0.0007) [2023-12-27 15:30:26,137][20347] Updated weights for policy 0, policy_version 1423010 (0.0009) [2023-12-27 15:30:26,588][20381] Updated weights for policy 1, policy_version 1421784 (0.0007) [2023-12-27 15:30:26,638][20381] Updated weights for policy 1, policy_version 1421794 (0.0007) [2023-12-27 15:30:26,682][20381] Updated weights for policy 1, policy_version 1421804 (0.0007) [2023-12-27 15:30:26,766][20347] Updated weights for policy 0, policy_version 1423020 (0.0008) [2023-12-27 15:30:26,820][20347] Updated weights for policy 0, policy_version 1423030 (0.0005) [2023-12-27 15:30:26,868][20347] Updated weights for policy 0, policy_version 1423040 (0.0006) [2023-12-27 15:30:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 728383488. Throughput: 0: 10146.8, 1: 10248.6. Samples: 728376144. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:27,363][19170] Avg episode reward: [(0, '970.770'), (1, '953.230')] [2023-12-27 15:30:27,371][20381] Updated weights for policy 1, policy_version 1421814 (0.0007) [2023-12-27 15:30:27,434][20381] Updated weights for policy 1, policy_version 1421824 (0.0005) [2023-12-27 15:30:27,467][20347] Updated weights for policy 0, policy_version 1423050 (0.0005) [2023-12-27 15:30:27,490][20381] Updated weights for policy 1, policy_version 1421834 (0.0005) [2023-12-27 15:30:27,524][20347] Updated weights for policy 0, policy_version 1423060 (0.0010) [2023-12-27 15:30:27,578][20347] Updated weights for policy 0, policy_version 1423070 (0.0010) [2023-12-27 15:30:27,634][20347] Updated weights for policy 0, policy_version 1423080 (0.0010) [2023-12-27 15:30:28,049][20381] Updated weights for policy 1, policy_version 1421844 (0.0007) [2023-12-27 15:30:28,103][20381] Updated weights for policy 1, policy_version 1421854 (0.0010) [2023-12-27 15:30:28,146][20381] Updated weights for policy 1, policy_version 1421864 (0.0008) [2023-12-27 15:30:28,204][20347] Updated weights for policy 0, policy_version 1423090 (0.0010) [2023-12-27 15:30:28,252][20347] Updated weights for policy 0, policy_version 1423100 (0.0010) [2023-12-27 15:30:28,299][20347] Updated weights for policy 0, policy_version 1423110 (0.0010) [2023-12-27 15:30:28,753][20381] Updated weights for policy 1, policy_version 1421874 (0.0005) [2023-12-27 15:30:28,812][20381] Updated weights for policy 1, policy_version 1421884 (0.0008) [2023-12-27 15:30:28,871][20381] Updated weights for policy 1, policy_version 1421894 (0.0008) [2023-12-27 15:30:28,928][20381] Updated weights for policy 1, policy_version 1421904 (0.0007) [2023-12-27 15:30:28,997][20347] Updated weights for policy 0, policy_version 1423120 (0.0010) [2023-12-27 15:30:29,042][20347] Updated weights for policy 0, policy_version 1423130 (0.0010) [2023-12-27 15:30:29,089][20347] Updated weights for policy 0, policy_version 1423140 (0.0010) [2023-12-27 15:30:29,566][20381] Updated weights for policy 1, policy_version 1421914 (0.0005) [2023-12-27 15:30:29,616][20381] Updated weights for policy 1, policy_version 1421924 (0.0006) [2023-12-27 15:30:29,661][20381] Updated weights for policy 1, policy_version 1421934 (0.0006) [2023-12-27 15:30:29,797][20347] Updated weights for policy 0, policy_version 1423150 (0.0010) [2023-12-27 15:30:29,860][20347] Updated weights for policy 0, policy_version 1423160 (0.0008) [2023-12-27 15:30:29,934][20347] Updated weights for policy 0, policy_version 1423170 (0.0008) [2023-12-27 15:30:30,384][20381] Updated weights for policy 1, policy_version 1421944 (0.0008) [2023-12-27 15:30:30,443][20381] Updated weights for policy 1, policy_version 1421954 (0.0008) [2023-12-27 15:30:30,504][20381] Updated weights for policy 1, policy_version 1421964 (0.0008) [2023-12-27 15:30:30,582][20347] Updated weights for policy 0, policy_version 1423180 (0.0009) [2023-12-27 15:30:30,641][20347] Updated weights for policy 0, policy_version 1423190 (0.0010) [2023-12-27 15:30:30,696][20347] Updated weights for policy 0, policy_version 1423200 (0.0010) [2023-12-27 15:30:31,237][20381] Updated weights for policy 1, policy_version 1421974 (0.0008) [2023-12-27 15:30:31,297][20381] Updated weights for policy 1, policy_version 1421984 (0.0008) [2023-12-27 15:30:31,351][20381] Updated weights for policy 1, policy_version 1421994 (0.0008) [2023-12-27 15:30:31,419][20347] Updated weights for policy 0, policy_version 1423210 (0.0010) [2023-12-27 15:30:31,474][20347] Updated weights for policy 0, policy_version 1423220 (0.0010) [2023-12-27 15:30:31,528][20347] Updated weights for policy 0, policy_version 1423230 (0.0010) [2023-12-27 15:30:31,595][20347] Updated weights for policy 0, policy_version 1423240 (0.0009) [2023-12-27 15:30:32,094][20381] Updated weights for policy 1, policy_version 1422004 (0.0008) [2023-12-27 15:30:32,158][20381] Updated weights for policy 1, policy_version 1422014 (0.0009) [2023-12-27 15:30:32,219][20381] Updated weights for policy 1, policy_version 1422024 (0.0008) [2023-12-27 15:30:32,240][20347] Updated weights for policy 0, policy_version 1423250 (0.0006) [2023-12-27 15:30:32,293][20347] Updated weights for policy 0, policy_version 1423260 (0.0007) [2023-12-27 15:30:32,352][20347] Updated weights for policy 0, policy_version 1423270 (0.0007) [2023-12-27 15:30:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 728489984. Throughput: 0: 10218.8, 1: 10247.1. Samples: 728501708. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:32,363][19170] Avg episode reward: [(0, '961.170'), (1, '908.450')] [2023-12-27 15:30:32,861][20381] Updated weights for policy 1, policy_version 1422034 (0.0007) [2023-12-27 15:30:32,913][20381] Updated weights for policy 1, policy_version 1422044 (0.0007) [2023-12-27 15:30:32,966][20381] Updated weights for policy 1, policy_version 1422054 (0.0007) [2023-12-27 15:30:33,012][20381] Updated weights for policy 1, policy_version 1422064 (0.0008) [2023-12-27 15:30:33,114][20347] Updated weights for policy 0, policy_version 1423280 (0.0007) [2023-12-27 15:30:33,169][20347] Updated weights for policy 0, policy_version 1423290 (0.0008) [2023-12-27 15:30:33,228][20347] Updated weights for policy 0, policy_version 1423300 (0.0007) [2023-12-27 15:30:33,682][20381] Updated weights for policy 1, policy_version 1422074 (0.0006) [2023-12-27 15:30:33,731][20381] Updated weights for policy 1, policy_version 1422084 (0.0007) [2023-12-27 15:30:33,788][20381] Updated weights for policy 1, policy_version 1422094 (0.0007) [2023-12-27 15:30:33,885][20347] Updated weights for policy 0, policy_version 1423310 (0.0008) [2023-12-27 15:30:33,936][20347] Updated weights for policy 0, policy_version 1423320 (0.0007) [2023-12-27 15:30:33,999][20347] Updated weights for policy 0, policy_version 1423330 (0.0005) [2023-12-27 15:30:34,465][20381] Updated weights for policy 1, policy_version 1422104 (0.0010) [2023-12-27 15:30:34,496][20292] KL-divergence is very high: 301.3594 [2023-12-27 15:30:34,509][20292] KL-divergence is very high: 349.8644 [2023-12-27 15:30:34,530][20292] KL-divergence is very high: 115.9602 [2023-12-27 15:30:34,538][20381] Updated weights for policy 1, policy_version 1422114 (0.0006) [2023-12-27 15:30:34,551][20292] KL-divergence is very high: 197.7162 [2023-12-27 15:30:34,564][20292] KL-divergence is very high: 138.1498 [2023-12-27 15:30:34,600][20381] Updated weights for policy 1, policy_version 1422124 (0.0009) [2023-12-27 15:30:34,708][20347] Updated weights for policy 0, policy_version 1423340 (0.0007) [2023-12-27 15:30:34,775][20347] Updated weights for policy 0, policy_version 1423350 (0.0006) [2023-12-27 15:30:34,830][20347] Updated weights for policy 0, policy_version 1423360 (0.0006) [2023-12-27 15:30:35,289][20381] Updated weights for policy 1, policy_version 1422134 (0.0010) [2023-12-27 15:30:35,346][20381] Updated weights for policy 1, policy_version 1422144 (0.0010) [2023-12-27 15:30:35,400][20381] Updated weights for policy 1, policy_version 1422154 (0.0009) [2023-12-27 15:30:35,491][20347] Updated weights for policy 0, policy_version 1423370 (0.0007) [2023-12-27 15:30:35,547][20347] Updated weights for policy 0, policy_version 1423380 (0.0007) [2023-12-27 15:30:35,605][20347] Updated weights for policy 0, policy_version 1423390 (0.0007) [2023-12-27 15:30:35,662][20347] Updated weights for policy 0, policy_version 1423400 (0.0007) [2023-12-27 15:30:36,089][20381] Updated weights for policy 1, policy_version 1422164 (0.0011) [2023-12-27 15:30:36,173][20381] Updated weights for policy 1, policy_version 1422177 (0.0010) [2023-12-27 15:30:36,239][20381] Updated weights for policy 1, policy_version 1422187 (0.0008) [2023-12-27 15:30:36,351][20347] Updated weights for policy 0, policy_version 1423410 (0.0009) [2023-12-27 15:30:36,412][20347] Updated weights for policy 0, policy_version 1423420 (0.0009) [2023-12-27 15:30:36,471][20347] Updated weights for policy 0, policy_version 1423430 (0.0009) [2023-12-27 15:30:36,922][20381] Updated weights for policy 1, policy_version 1422197 (0.0007) [2023-12-27 15:30:36,981][20381] Updated weights for policy 1, policy_version 1422207 (0.0008) [2023-12-27 15:30:37,029][20381] Updated weights for policy 1, policy_version 1422217 (0.0007) [2023-12-27 15:30:37,244][20347] Updated weights for policy 0, policy_version 1423440 (0.0010) [2023-12-27 15:30:37,295][20347] Updated weights for policy 0, policy_version 1423450 (0.0008) [2023-12-27 15:30:37,350][20347] Updated weights for policy 0, policy_version 1423460 (0.0008) [2023-12-27 15:30:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 728588288. Throughput: 0: 10230.9, 1: 10275.0. Samples: 728563164. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:37,363][19170] Avg episode reward: [(0, '961.290'), (1, '897.790')] [2023-12-27 15:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001422224_364118016.pth... [2023-12-27 15:30:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001423464_364478464.pth... [2023-12-27 15:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001421040_363814912.pth [2023-12-27 15:30:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001422248_364167168.pth [2023-12-27 15:30:37,677][20381] Updated weights for policy 1, policy_version 1422227 (0.0009) [2023-12-27 15:30:37,735][20381] Updated weights for policy 1, policy_version 1422237 (0.0008) [2023-12-27 15:30:37,790][20381] Updated weights for policy 1, policy_version 1422247 (0.0008) [2023-12-27 15:30:38,069][20347] Updated weights for policy 0, policy_version 1423470 (0.0008) [2023-12-27 15:30:38,134][20347] Updated weights for policy 0, policy_version 1423480 (0.0006) [2023-12-27 15:30:38,194][20347] Updated weights for policy 0, policy_version 1423490 (0.0007) [2023-12-27 15:30:38,546][20381] Updated weights for policy 1, policy_version 1422257 (0.0008) [2023-12-27 15:30:38,605][20381] Updated weights for policy 1, policy_version 1422267 (0.0010) [2023-12-27 15:30:38,655][20292] KL-divergence is very high: 121.1480 [2023-12-27 15:30:38,668][20381] Updated weights for policy 1, policy_version 1422277 (0.0011) [2023-12-27 15:30:38,731][20381] Updated weights for policy 1, policy_version 1422287 (0.0011) [2023-12-27 15:30:38,860][20347] Updated weights for policy 0, policy_version 1423500 (0.0007) [2023-12-27 15:30:38,921][20347] Updated weights for policy 0, policy_version 1423510 (0.0007) [2023-12-27 15:30:38,973][20347] Updated weights for policy 0, policy_version 1423520 (0.0009) [2023-12-27 15:30:39,410][20381] Updated weights for policy 1, policy_version 1422297 (0.0011) [2023-12-27 15:30:39,468][20381] Updated weights for policy 1, policy_version 1422307 (0.0006) [2023-12-27 15:30:39,532][20381] Updated weights for policy 1, policy_version 1422317 (0.0006) [2023-12-27 15:30:39,800][20347] Updated weights for policy 0, policy_version 1423530 (0.0007) [2023-12-27 15:30:39,870][20347] Updated weights for policy 0, policy_version 1423540 (0.0008) [2023-12-27 15:30:39,936][20347] Updated weights for policy 0, policy_version 1423550 (0.0008) [2023-12-27 15:30:39,994][20347] Updated weights for policy 0, policy_version 1423560 (0.0008) [2023-12-27 15:30:40,255][20381] Updated weights for policy 1, policy_version 1422327 (0.0009) [2023-12-27 15:30:40,317][20381] Updated weights for policy 1, policy_version 1422337 (0.0011) [2023-12-27 15:30:40,373][20381] Updated weights for policy 1, policy_version 1422347 (0.0010) [2023-12-27 15:30:40,698][20347] Updated weights for policy 0, policy_version 1423570 (0.0008) [2023-12-27 15:30:40,753][20347] Updated weights for policy 0, policy_version 1423580 (0.0006) [2023-12-27 15:30:40,800][20347] Updated weights for policy 0, policy_version 1423590 (0.0005) [2023-12-27 15:30:41,072][20381] Updated weights for policy 1, policy_version 1422357 (0.0011) [2023-12-27 15:30:41,144][20381] Updated weights for policy 1, policy_version 1422367 (0.0009) [2023-12-27 15:30:41,208][20381] Updated weights for policy 1, policy_version 1422377 (0.0008) [2023-12-27 15:30:41,533][20347] Updated weights for policy 0, policy_version 1423600 (0.0007) [2023-12-27 15:30:41,595][20347] Updated weights for policy 0, policy_version 1423610 (0.0008) [2023-12-27 15:30:41,660][20347] Updated weights for policy 0, policy_version 1423620 (0.0008) [2023-12-27 15:30:41,987][20381] Updated weights for policy 1, policy_version 1422387 (0.0009) [2023-12-27 15:30:42,039][20381] Updated weights for policy 1, policy_version 1422397 (0.0008) [2023-12-27 15:30:42,094][20381] Updated weights for policy 1, policy_version 1422407 (0.0008) [2023-12-27 15:30:42,348][20347] Updated weights for policy 0, policy_version 1423630 (0.0008) [2023-12-27 15:30:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 728686592. Throughput: 0: 10160.3, 1: 10228.6. Samples: 728679628. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:42,363][19170] Avg episode reward: [(0, '970.960'), (1, '913.800')] [2023-12-27 15:30:42,415][20347] Updated weights for policy 0, policy_version 1423640 (0.0007) [2023-12-27 15:30:42,482][20347] Updated weights for policy 0, policy_version 1423650 (0.0008) [2023-12-27 15:30:42,840][20381] Updated weights for policy 1, policy_version 1422417 (0.0008) [2023-12-27 15:30:42,906][20381] Updated weights for policy 1, policy_version 1422427 (0.0007) [2023-12-27 15:30:42,966][20381] Updated weights for policy 1, policy_version 1422437 (0.0005) [2023-12-27 15:30:43,022][20381] Updated weights for policy 1, policy_version 1422447 (0.0005) [2023-12-27 15:30:43,067][20347] Updated weights for policy 0, policy_version 1423660 (0.0007) [2023-12-27 15:30:43,120][20347] Updated weights for policy 0, policy_version 1423670 (0.0007) [2023-12-27 15:30:43,172][20347] Updated weights for policy 0, policy_version 1423680 (0.0007) [2023-12-27 15:30:43,632][20381] Updated weights for policy 1, policy_version 1422457 (0.0008) [2023-12-27 15:30:43,690][20381] Updated weights for policy 1, policy_version 1422467 (0.0008) [2023-12-27 15:30:43,739][20381] Updated weights for policy 1, policy_version 1422477 (0.0008) [2023-12-27 15:30:43,786][20347] Updated weights for policy 0, policy_version 1423690 (0.0008) [2023-12-27 15:30:43,839][20347] Updated weights for policy 0, policy_version 1423700 (0.0007) [2023-12-27 15:30:43,895][20347] Updated weights for policy 0, policy_version 1423710 (0.0007) [2023-12-27 15:30:43,941][20347] Updated weights for policy 0, policy_version 1423720 (0.0007) [2023-12-27 15:30:44,363][20381] Updated weights for policy 1, policy_version 1422487 (0.0008) [2023-12-27 15:30:44,426][20381] Updated weights for policy 1, policy_version 1422497 (0.0008) [2023-12-27 15:30:44,479][20381] Updated weights for policy 1, policy_version 1422507 (0.0007) [2023-12-27 15:30:44,503][20347] Updated weights for policy 0, policy_version 1423730 (0.0005) [2023-12-27 15:30:44,550][20347] Updated weights for policy 0, policy_version 1423740 (0.0005) [2023-12-27 15:30:44,610][20347] Updated weights for policy 0, policy_version 1423750 (0.0005) [2023-12-27 15:30:45,230][20381] Updated weights for policy 1, policy_version 1422517 (0.0007) [2023-12-27 15:30:45,262][20347] Updated weights for policy 0, policy_version 1423760 (0.0008) [2023-12-27 15:30:45,289][20381] Updated weights for policy 1, policy_version 1422527 (0.0007) [2023-12-27 15:30:45,318][20347] Updated weights for policy 0, policy_version 1423770 (0.0006) [2023-12-27 15:30:45,350][20381] Updated weights for policy 1, policy_version 1422537 (0.0007) [2023-12-27 15:30:45,371][20347] Updated weights for policy 0, policy_version 1423780 (0.0006) [2023-12-27 15:30:45,912][20381] Updated weights for policy 1, policy_version 1422547 (0.0007) [2023-12-27 15:30:45,967][20381] Updated weights for policy 1, policy_version 1422557 (0.0007) [2023-12-27 15:30:46,022][20381] Updated weights for policy 1, policy_version 1422567 (0.0008) [2023-12-27 15:30:46,071][20347] Updated weights for policy 0, policy_version 1423790 (0.0009) [2023-12-27 15:30:46,119][20347] Updated weights for policy 0, policy_version 1423800 (0.0010) [2023-12-27 15:30:46,171][20347] Updated weights for policy 0, policy_version 1423810 (0.0008) [2023-12-27 15:30:46,680][20381] Updated weights for policy 1, policy_version 1422577 (0.0007) [2023-12-27 15:30:46,719][20347] Updated weights for policy 0, policy_version 1423820 (0.0006) [2023-12-27 15:30:46,728][20381] Updated weights for policy 1, policy_version 1422587 (0.0007) [2023-12-27 15:30:46,776][20347] Updated weights for policy 0, policy_version 1423830 (0.0008) [2023-12-27 15:30:46,782][20381] Updated weights for policy 1, policy_version 1422597 (0.0007) [2023-12-27 15:30:46,833][20347] Updated weights for policy 0, policy_version 1423840 (0.0011) [2023-12-27 15:30:46,839][20381] Updated weights for policy 1, policy_version 1422607 (0.0006) [2023-12-27 15:30:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 728793088. Throughput: 0: 10287.7, 1: 10254.2. Samples: 728805892. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:47,363][19170] Avg episode reward: [(0, '970.740'), (1, '942.110')] [2023-12-27 15:30:47,482][20347] Updated weights for policy 0, policy_version 1423850 (0.0009) [2023-12-27 15:30:47,540][20347] Updated weights for policy 0, policy_version 1423860 (0.0005) [2023-12-27 15:30:47,547][20381] Updated weights for policy 1, policy_version 1422617 (0.0007) [2023-12-27 15:30:47,592][20347] Updated weights for policy 0, policy_version 1423870 (0.0010) [2023-12-27 15:30:47,599][20381] Updated weights for policy 1, policy_version 1422627 (0.0008) [2023-12-27 15:30:47,643][20347] Updated weights for policy 0, policy_version 1423880 (0.0010) [2023-12-27 15:30:47,652][20381] Updated weights for policy 1, policy_version 1422637 (0.0006) [2023-12-27 15:30:48,289][20347] Updated weights for policy 0, policy_version 1423890 (0.0005) [2023-12-27 15:30:48,351][20347] Updated weights for policy 0, policy_version 1423900 (0.0007) [2023-12-27 15:30:48,369][20381] Updated weights for policy 1, policy_version 1422647 (0.0008) [2023-12-27 15:30:48,409][20347] Updated weights for policy 0, policy_version 1423911 (0.0006) [2023-12-27 15:30:48,418][20381] Updated weights for policy 1, policy_version 1422657 (0.0008) [2023-12-27 15:30:48,473][20381] Updated weights for policy 1, policy_version 1422667 (0.0009) [2023-12-27 15:30:49,006][20347] Updated weights for policy 0, policy_version 1423921 (0.0010) [2023-12-27 15:30:49,060][20347] Updated weights for policy 0, policy_version 1423931 (0.0010) [2023-12-27 15:30:49,117][20347] Updated weights for policy 0, policy_version 1423941 (0.0010) [2023-12-27 15:30:49,177][20381] Updated weights for policy 1, policy_version 1422677 (0.0009) [2023-12-27 15:30:49,241][20381] Updated weights for policy 1, policy_version 1422687 (0.0009) [2023-12-27 15:30:49,308][20381] Updated weights for policy 1, policy_version 1422697 (0.0008) [2023-12-27 15:30:49,810][20347] Updated weights for policy 0, policy_version 1423951 (0.0009) [2023-12-27 15:30:49,872][20347] Updated weights for policy 0, policy_version 1423961 (0.0008) [2023-12-27 15:30:49,934][20347] Updated weights for policy 0, policy_version 1423971 (0.0008) [2023-12-27 15:30:49,985][20381] Updated weights for policy 1, policy_version 1422707 (0.0007) [2023-12-27 15:30:50,047][20381] Updated weights for policy 1, policy_version 1422717 (0.0007) [2023-12-27 15:30:50,097][20292] KL-divergence is very high: 393.1711 [2023-12-27 15:30:50,102][20381] Updated weights for policy 1, policy_version 1422727 (0.0008) [2023-12-27 15:30:50,144][20292] KL-divergence is very high: 391.3433 [2023-12-27 15:30:50,653][20347] Updated weights for policy 0, policy_version 1423981 (0.0007) [2023-12-27 15:30:50,720][20347] Updated weights for policy 0, policy_version 1423991 (0.0006) [2023-12-27 15:30:50,782][20347] Updated weights for policy 0, policy_version 1424001 (0.0008) [2023-12-27 15:30:50,788][20381] Updated weights for policy 1, policy_version 1422737 (0.0007) [2023-12-27 15:30:50,850][20381] Updated weights for policy 1, policy_version 1422747 (0.0006) [2023-12-27 15:30:50,907][20381] Updated weights for policy 1, policy_version 1422757 (0.0006) [2023-12-27 15:30:50,962][20381] Updated weights for policy 1, policy_version 1422767 (0.0005) [2023-12-27 15:30:51,529][20347] Updated weights for policy 0, policy_version 1424011 (0.0006) [2023-12-27 15:30:51,591][20347] Updated weights for policy 0, policy_version 1424021 (0.0008) [2023-12-27 15:30:51,654][20347] Updated weights for policy 0, policy_version 1424031 (0.0008) [2023-12-27 15:30:51,752][20381] Updated weights for policy 1, policy_version 1422777 (0.0008) [2023-12-27 15:30:51,803][20381] Updated weights for policy 1, policy_version 1422787 (0.0008) [2023-12-27 15:30:51,858][20381] Updated weights for policy 1, policy_version 1422797 (0.0008) [2023-12-27 15:30:52,301][20347] Updated weights for policy 0, policy_version 1424041 (0.0007) [2023-12-27 15:30:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 728891392. Throughput: 0: 10318.0, 1: 10184.9. Samples: 728867020. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:52,363][19170] Avg episode reward: [(0, '960.970'), (1, '905.190')] [2023-12-27 15:30:52,366][20347] Updated weights for policy 0, policy_version 1424051 (0.0008) [2023-12-27 15:30:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001422800_364265472.pth... [2023-12-27 15:30:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001421616_363962368.pth [2023-12-27 15:30:52,429][20347] Updated weights for policy 0, policy_version 1424061 (0.0009) [2023-12-27 15:30:52,481][20347] Updated weights for policy 0, policy_version 1424071 (0.0005) [2023-12-27 15:30:52,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001424072_364634112.pth... [2023-12-27 15:30:52,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001422856_364322816.pth [2023-12-27 15:30:52,663][20381] Updated weights for policy 1, policy_version 1422807 (0.0008) [2023-12-27 15:30:52,720][20381] Updated weights for policy 1, policy_version 1422817 (0.0008) [2023-12-27 15:30:52,768][20381] Updated weights for policy 1, policy_version 1422827 (0.0009) [2023-12-27 15:30:53,127][20347] Updated weights for policy 0, policy_version 1424081 (0.0008) [2023-12-27 15:30:53,185][20347] Updated weights for policy 0, policy_version 1424091 (0.0010) [2023-12-27 15:30:53,242][20347] Updated weights for policy 0, policy_version 1424101 (0.0011) [2023-12-27 15:30:53,490][20381] Updated weights for policy 1, policy_version 1422837 (0.0009) [2023-12-27 15:30:53,544][20381] Updated weights for policy 1, policy_version 1422847 (0.0010) [2023-12-27 15:30:53,604][20381] Updated weights for policy 1, policy_version 1422857 (0.0010) [2023-12-27 15:30:53,958][20347] Updated weights for policy 0, policy_version 1424111 (0.0007) [2023-12-27 15:30:54,004][20347] Updated weights for policy 0, policy_version 1424121 (0.0005) [2023-12-27 15:30:54,058][20347] Updated weights for policy 0, policy_version 1424131 (0.0005) [2023-12-27 15:30:54,310][20381] Updated weights for policy 1, policy_version 1422867 (0.0010) [2023-12-27 15:30:54,365][20381] Updated weights for policy 1, policy_version 1422877 (0.0010) [2023-12-27 15:30:54,426][20381] Updated weights for policy 1, policy_version 1422887 (0.0010) [2023-12-27 15:30:54,697][20347] Updated weights for policy 0, policy_version 1424141 (0.0008) [2023-12-27 15:30:54,748][20347] Updated weights for policy 0, policy_version 1424151 (0.0005) [2023-12-27 15:30:54,802][20347] Updated weights for policy 0, policy_version 1424161 (0.0005) [2023-12-27 15:30:55,082][20381] Updated weights for policy 1, policy_version 1422897 (0.0010) [2023-12-27 15:30:55,142][20381] Updated weights for policy 1, policy_version 1422907 (0.0005) [2023-12-27 15:30:55,182][20292] KL-divergence is very high: 165.7183 [2023-12-27 15:30:55,193][20381] Updated weights for policy 1, policy_version 1422917 (0.0010) [2023-12-27 15:30:55,206][20292] KL-divergence is very high: 167.6847 [2023-12-27 15:30:55,224][20292] KL-divergence is very high: 187.4718 [2023-12-27 15:30:55,247][20381] Updated weights for policy 1, policy_version 1422927 (0.0010) [2023-12-27 15:30:55,403][20347] Updated weights for policy 0, policy_version 1424171 (0.0006) [2023-12-27 15:30:55,458][20347] Updated weights for policy 0, policy_version 1424181 (0.0009) [2023-12-27 15:30:55,508][20347] Updated weights for policy 0, policy_version 1424191 (0.0009) [2023-12-27 15:30:55,827][20381] Updated weights for policy 1, policy_version 1422937 (0.0008) [2023-12-27 15:30:55,876][20381] Updated weights for policy 1, policy_version 1422947 (0.0008) [2023-12-27 15:30:55,925][20381] Updated weights for policy 1, policy_version 1422957 (0.0008) [2023-12-27 15:30:56,169][20347] Updated weights for policy 0, policy_version 1424201 (0.0008) [2023-12-27 15:30:56,226][20347] Updated weights for policy 0, policy_version 1424211 (0.0005) [2023-12-27 15:30:56,279][20347] Updated weights for policy 0, policy_version 1424221 (0.0005) [2023-12-27 15:30:56,329][20347] Updated weights for policy 0, policy_version 1424231 (0.0005) [2023-12-27 15:30:56,683][20381] Updated weights for policy 1, policy_version 1422967 (0.0007) [2023-12-27 15:30:56,734][20381] Updated weights for policy 1, policy_version 1422977 (0.0007) [2023-12-27 15:30:56,792][20381] Updated weights for policy 1, policy_version 1422987 (0.0007) [2023-12-27 15:30:56,959][20347] Updated weights for policy 0, policy_version 1424241 (0.0007) [2023-12-27 15:30:57,007][20347] Updated weights for policy 0, policy_version 1424251 (0.0008) [2023-12-27 15:30:57,057][20347] Updated weights for policy 0, policy_version 1424261 (0.0007) [2023-12-27 15:30:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.2). Total num frames: 728997888. Throughput: 0: 10312.7, 1: 10112.8. Samples: 728987592. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:30:57,363][19170] Avg episode reward: [(0, '951.320'), (1, '886.790')] [2023-12-27 15:30:57,478][20381] Updated weights for policy 1, policy_version 1422997 (0.0006) [2023-12-27 15:30:57,524][20381] Updated weights for policy 1, policy_version 1423007 (0.0008) [2023-12-27 15:30:57,533][20292] KL-divergence is very high: 102.7914 [2023-12-27 15:30:57,575][20381] Updated weights for policy 1, policy_version 1423017 (0.0007) [2023-12-27 15:30:57,747][20347] Updated weights for policy 0, policy_version 1424271 (0.0007) [2023-12-27 15:30:57,795][20347] Updated weights for policy 0, policy_version 1424281 (0.0007) [2023-12-27 15:30:57,813][20291] KL-divergence is very high: 104.1369 [2023-12-27 15:30:57,847][20347] Updated weights for policy 0, policy_version 1424291 (0.0007) [2023-12-27 15:30:58,200][20381] Updated weights for policy 1, policy_version 1423027 (0.0008) [2023-12-27 15:30:58,262][20381] Updated weights for policy 1, policy_version 1423037 (0.0006) [2023-12-27 15:30:58,324][20381] Updated weights for policy 1, policy_version 1423047 (0.0006) [2023-12-27 15:30:58,525][20347] Updated weights for policy 0, policy_version 1424301 (0.0007) [2023-12-27 15:30:58,585][20347] Updated weights for policy 0, policy_version 1424311 (0.0007) [2023-12-27 15:30:58,644][20347] Updated weights for policy 0, policy_version 1424321 (0.0008) [2023-12-27 15:30:59,035][20381] Updated weights for policy 1, policy_version 1423057 (0.0007) [2023-12-27 15:30:59,092][20381] Updated weights for policy 1, policy_version 1423067 (0.0007) [2023-12-27 15:30:59,153][20381] Updated weights for policy 1, policy_version 1423077 (0.0007) [2023-12-27 15:30:59,203][20381] Updated weights for policy 1, policy_version 1423087 (0.0008) [2023-12-27 15:30:59,378][20347] Updated weights for policy 0, policy_version 1424331 (0.0007) [2023-12-27 15:30:59,428][20347] Updated weights for policy 0, policy_version 1424341 (0.0006) [2023-12-27 15:30:59,490][20347] Updated weights for policy 0, policy_version 1424351 (0.0007) [2023-12-27 15:30:59,916][20381] Updated weights for policy 1, policy_version 1423097 (0.0009) [2023-12-27 15:30:59,977][20381] Updated weights for policy 1, policy_version 1423108 (0.0007) [2023-12-27 15:31:00,034][20381] Updated weights for policy 1, policy_version 1423118 (0.0005) [2023-12-27 15:31:00,213][20347] Updated weights for policy 0, policy_version 1424361 (0.0007) [2023-12-27 15:31:00,266][20347] Updated weights for policy 0, policy_version 1424371 (0.0008) [2023-12-27 15:31:00,327][20347] Updated weights for policy 0, policy_version 1424381 (0.0006) [2023-12-27 15:31:00,389][20347] Updated weights for policy 0, policy_version 1424391 (0.0006) [2023-12-27 15:31:00,667][20381] Updated weights for policy 1, policy_version 1423128 (0.0007) [2023-12-27 15:31:00,716][20381] Updated weights for policy 1, policy_version 1423138 (0.0008) [2023-12-27 15:31:00,774][20381] Updated weights for policy 1, policy_version 1423148 (0.0007) [2023-12-27 15:31:01,024][20347] Updated weights for policy 0, policy_version 1424401 (0.0010) [2023-12-27 15:31:01,089][20347] Updated weights for policy 0, policy_version 1424411 (0.0008) [2023-12-27 15:31:01,153][20347] Updated weights for policy 0, policy_version 1424421 (0.0008) [2023-12-27 15:31:01,491][20381] Updated weights for policy 1, policy_version 1423158 (0.0008) [2023-12-27 15:31:01,544][20381] Updated weights for policy 1, policy_version 1423168 (0.0005) [2023-12-27 15:31:01,603][20381] Updated weights for policy 1, policy_version 1423178 (0.0006) [2023-12-27 15:31:01,771][20347] Updated weights for policy 0, policy_version 1424431 (0.0007) [2023-12-27 15:31:01,826][20347] Updated weights for policy 0, policy_version 1424441 (0.0007) [2023-12-27 15:31:01,886][20347] Updated weights for policy 0, policy_version 1424451 (0.0007) [2023-12-27 15:31:02,342][20381] Updated weights for policy 1, policy_version 1423188 (0.0008) [2023-12-27 15:31:02,364][19170] Fps is (10 sec: 20477.3, 60 sec: 20343.0, 300 sec: 20466.0). Total num frames: 729096192. Throughput: 0: 10326.5, 1: 10109.1. Samples: 729110572. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:02,365][19170] Avg episode reward: [(0, '951.280'), (1, '859.810')] [2023-12-27 15:31:02,406][20381] Updated weights for policy 1, policy_version 1423198 (0.0008) [2023-12-27 15:31:02,466][20381] Updated weights for policy 1, policy_version 1423208 (0.0008) [2023-12-27 15:31:02,516][20347] Updated weights for policy 0, policy_version 1424461 (0.0006) [2023-12-27 15:31:02,576][20347] Updated weights for policy 0, policy_version 1424471 (0.0005) [2023-12-27 15:31:02,634][20347] Updated weights for policy 0, policy_version 1424481 (0.0007) [2023-12-27 15:31:03,099][20381] Updated weights for policy 1, policy_version 1423218 (0.0008) [2023-12-27 15:31:03,151][20381] Updated weights for policy 1, policy_version 1423228 (0.0009) [2023-12-27 15:31:03,200][20381] Updated weights for policy 1, policy_version 1423238 (0.0006) [2023-12-27 15:31:03,251][20381] Updated weights for policy 1, policy_version 1423248 (0.0010) [2023-12-27 15:31:03,311][20347] Updated weights for policy 0, policy_version 1424491 (0.0007) [2023-12-27 15:31:03,362][20347] Updated weights for policy 0, policy_version 1424501 (0.0010) [2023-12-27 15:31:03,414][20347] Updated weights for policy 0, policy_version 1424511 (0.0010) [2023-12-27 15:31:03,917][20381] Updated weights for policy 1, policy_version 1423258 (0.0008) [2023-12-27 15:31:03,975][20381] Updated weights for policy 1, policy_version 1423268 (0.0007) [2023-12-27 15:31:04,038][20381] Updated weights for policy 1, policy_version 1423278 (0.0008) [2023-12-27 15:31:04,111][20347] Updated weights for policy 0, policy_version 1424521 (0.0010) [2023-12-27 15:31:04,174][20347] Updated weights for policy 0, policy_version 1424531 (0.0010) [2023-12-27 15:31:04,230][20291] KL-divergence is very high: 101.5384 [2023-12-27 15:31:04,235][20347] Updated weights for policy 0, policy_version 1424541 (0.0011) [2023-12-27 15:31:04,295][20347] Updated weights for policy 0, policy_version 1424551 (0.0011) [2023-12-27 15:31:04,715][20381] Updated weights for policy 1, policy_version 1423288 (0.0010) [2023-12-27 15:31:04,771][20381] Updated weights for policy 1, policy_version 1423298 (0.0011) [2023-12-27 15:31:04,827][20381] Updated weights for policy 1, policy_version 1423308 (0.0011) [2023-12-27 15:31:04,961][20347] Updated weights for policy 0, policy_version 1424561 (0.0010) [2023-12-27 15:31:05,015][20347] Updated weights for policy 0, policy_version 1424571 (0.0007) [2023-12-27 15:31:05,077][20347] Updated weights for policy 0, policy_version 1424581 (0.0005) [2023-12-27 15:31:05,456][20381] Updated weights for policy 1, policy_version 1423318 (0.0007) [2023-12-27 15:31:05,505][20381] Updated weights for policy 1, policy_version 1423328 (0.0005) [2023-12-27 15:31:05,550][20381] Updated weights for policy 1, policy_version 1423338 (0.0006) [2023-12-27 15:31:05,701][20347] Updated weights for policy 0, policy_version 1424591 (0.0006) [2023-12-27 15:31:05,752][20347] Updated weights for policy 0, policy_version 1424601 (0.0010) [2023-12-27 15:31:05,803][20347] Updated weights for policy 0, policy_version 1424611 (0.0010) [2023-12-27 15:31:06,188][20381] Updated weights for policy 1, policy_version 1423348 (0.0007) [2023-12-27 15:31:06,250][20381] Updated weights for policy 1, policy_version 1423358 (0.0009) [2023-12-27 15:31:06,313][20381] Updated weights for policy 1, policy_version 1423368 (0.0010) [2023-12-27 15:31:06,479][20347] Updated weights for policy 0, policy_version 1424621 (0.0010) [2023-12-27 15:31:06,539][20347] Updated weights for policy 0, policy_version 1424631 (0.0011) [2023-12-27 15:31:06,605][20347] Updated weights for policy 0, policy_version 1424641 (0.0011) [2023-12-27 15:31:07,000][20381] Updated weights for policy 1, policy_version 1423378 (0.0010) [2023-12-27 15:31:07,057][20381] Updated weights for policy 1, policy_version 1423388 (0.0011) [2023-12-27 15:31:07,117][20381] Updated weights for policy 1, policy_version 1423398 (0.0009) [2023-12-27 15:31:07,172][20381] Updated weights for policy 1, policy_version 1423408 (0.0010) [2023-12-27 15:31:07,291][20347] Updated weights for policy 0, policy_version 1424651 (0.0010) [2023-12-27 15:31:07,339][20347] Updated weights for policy 0, policy_version 1424661 (0.0010) [2023-12-27 15:31:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 729202688. Throughput: 0: 10321.2, 1: 10107.7. Samples: 729171976. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:07,364][19170] Avg episode reward: [(0, '931.910'), (1, '890.280')] [2023-12-27 15:31:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001423408_364421120.pth... [2023-12-27 15:31:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001422224_364118016.pth [2023-12-27 15:31:07,391][20347] Updated weights for policy 0, policy_version 1424671 (0.0010) [2023-12-27 15:31:07,443][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001424680_364789760.pth... [2023-12-27 15:31:07,447][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001423464_364478464.pth [2023-12-27 15:31:07,735][20381] Updated weights for policy 1, policy_version 1423418 (0.0011) [2023-12-27 15:31:07,787][20381] Updated weights for policy 1, policy_version 1423428 (0.0010) [2023-12-27 15:31:07,841][20381] Updated weights for policy 1, policy_version 1423438 (0.0007) [2023-12-27 15:31:08,027][20347] Updated weights for policy 0, policy_version 1424681 (0.0010) [2023-12-27 15:31:08,079][20347] Updated weights for policy 0, policy_version 1424691 (0.0007) [2023-12-27 15:31:08,140][20347] Updated weights for policy 0, policy_version 1424701 (0.0007) [2023-12-27 15:31:08,196][20347] Updated weights for policy 0, policy_version 1424711 (0.0007) [2023-12-27 15:31:08,555][20381] Updated weights for policy 1, policy_version 1423448 (0.0007) [2023-12-27 15:31:08,607][20381] Updated weights for policy 1, policy_version 1423458 (0.0008) [2023-12-27 15:31:08,660][20381] Updated weights for policy 1, policy_version 1423468 (0.0008) [2023-12-27 15:31:08,889][20347] Updated weights for policy 0, policy_version 1424721 (0.0008) [2023-12-27 15:31:08,936][20347] Updated weights for policy 0, policy_version 1424731 (0.0007) [2023-12-27 15:31:08,984][20347] Updated weights for policy 0, policy_version 1424741 (0.0007) [2023-12-27 15:31:09,384][20381] Updated weights for policy 1, policy_version 1423478 (0.0008) [2023-12-27 15:31:09,443][20381] Updated weights for policy 1, policy_version 1423488 (0.0009) [2023-12-27 15:31:09,495][20381] Updated weights for policy 1, policy_version 1423498 (0.0008) [2023-12-27 15:31:09,702][20347] Updated weights for policy 0, policy_version 1424751 (0.0007) [2023-12-27 15:31:09,764][20347] Updated weights for policy 0, policy_version 1424761 (0.0007) [2023-12-27 15:31:09,826][20347] Updated weights for policy 0, policy_version 1424771 (0.0007) [2023-12-27 15:31:10,206][20381] Updated weights for policy 1, policy_version 1423508 (0.0008) [2023-12-27 15:31:10,266][20381] Updated weights for policy 1, policy_version 1423518 (0.0007) [2023-12-27 15:31:10,323][20381] Updated weights for policy 1, policy_version 1423528 (0.0006) [2023-12-27 15:31:10,546][20347] Updated weights for policy 0, policy_version 1424781 (0.0006) [2023-12-27 15:31:10,608][20347] Updated weights for policy 0, policy_version 1424791 (0.0007) [2023-12-27 15:31:10,656][20347] Updated weights for policy 0, policy_version 1424801 (0.0007) [2023-12-27 15:31:11,008][20381] Updated weights for policy 1, policy_version 1423538 (0.0011) [2023-12-27 15:31:11,082][20381] Updated weights for policy 1, policy_version 1423548 (0.0008) [2023-12-27 15:31:11,146][20381] Updated weights for policy 1, policy_version 1423558 (0.0009) [2023-12-27 15:31:11,211][20381] Updated weights for policy 1, policy_version 1423568 (0.0007) [2023-12-27 15:31:11,442][20347] Updated weights for policy 0, policy_version 1424811 (0.0008) [2023-12-27 15:31:11,492][20347] Updated weights for policy 0, policy_version 1424821 (0.0010) [2023-12-27 15:31:11,545][20347] Updated weights for policy 0, policy_version 1424831 (0.0010) [2023-12-27 15:31:11,938][20381] Updated weights for policy 1, policy_version 1423578 (0.0011) [2023-12-27 15:31:11,992][20381] Updated weights for policy 1, policy_version 1423588 (0.0011) [2023-12-27 15:31:12,055][20381] Updated weights for policy 1, policy_version 1423598 (0.0011) [2023-12-27 15:31:12,275][20347] Updated weights for policy 0, policy_version 1424841 (0.0008) [2023-12-27 15:31:12,362][19170] Fps is (10 sec: 20482.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 729300992. Throughput: 0: 10285.3, 1: 10118.7. Samples: 729294328. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:12,363][19170] Avg episode reward: [(0, '941.650'), (1, '946.350')] [2023-12-27 15:31:12,366][20347] Updated weights for policy 0, policy_version 1424851 (0.0010) [2023-12-27 15:31:12,430][20347] Updated weights for policy 0, policy_version 1424861 (0.0011) [2023-12-27 15:31:12,487][20347] Updated weights for policy 0, policy_version 1424871 (0.0007) [2023-12-27 15:31:12,795][20381] Updated weights for policy 1, policy_version 1423608 (0.0011) [2023-12-27 15:31:12,844][20381] Updated weights for policy 1, policy_version 1423618 (0.0010) [2023-12-27 15:31:12,897][20381] Updated weights for policy 1, policy_version 1423628 (0.0011) [2023-12-27 15:31:13,110][20347] Updated weights for policy 0, policy_version 1424881 (0.0010) [2023-12-27 15:31:13,171][20347] Updated weights for policy 0, policy_version 1424891 (0.0009) [2023-12-27 15:31:13,230][20347] Updated weights for policy 0, policy_version 1424901 (0.0011) [2023-12-27 15:31:13,569][20381] Updated weights for policy 1, policy_version 1423638 (0.0009) [2023-12-27 15:31:13,622][20381] Updated weights for policy 1, policy_version 1423648 (0.0011) [2023-12-27 15:31:13,682][20381] Updated weights for policy 1, policy_version 1423658 (0.0006) [2023-12-27 15:31:13,875][20347] Updated weights for policy 0, policy_version 1424911 (0.0007) [2023-12-27 15:31:13,937][20347] Updated weights for policy 0, policy_version 1424921 (0.0007) [2023-12-27 15:31:13,998][20347] Updated weights for policy 0, policy_version 1424931 (0.0009) [2023-12-27 15:31:14,243][20381] Updated weights for policy 1, policy_version 1423668 (0.0005) [2023-12-27 15:31:14,303][20381] Updated weights for policy 1, policy_version 1423678 (0.0007) [2023-12-27 15:31:14,361][20381] Updated weights for policy 1, policy_version 1423688 (0.0008) [2023-12-27 15:31:14,677][20347] Updated weights for policy 0, policy_version 1424941 (0.0007) [2023-12-27 15:31:14,723][20347] Updated weights for policy 0, policy_version 1424951 (0.0006) [2023-12-27 15:31:14,789][20347] Updated weights for policy 0, policy_version 1424961 (0.0008) [2023-12-27 15:31:15,065][20381] Updated weights for policy 1, policy_version 1423698 (0.0007) [2023-12-27 15:31:15,129][20381] Updated weights for policy 1, policy_version 1423708 (0.0009) [2023-12-27 15:31:15,179][20381] Updated weights for policy 1, policy_version 1423718 (0.0008) [2023-12-27 15:31:15,230][20381] Updated weights for policy 1, policy_version 1423728 (0.0008) [2023-12-27 15:31:15,485][20347] Updated weights for policy 0, policy_version 1424971 (0.0008) [2023-12-27 15:31:15,538][20347] Updated weights for policy 0, policy_version 1424981 (0.0009) [2023-12-27 15:31:15,590][20347] Updated weights for policy 0, policy_version 1424991 (0.0007) [2023-12-27 15:31:15,868][20381] Updated weights for policy 1, policy_version 1423738 (0.0006) [2023-12-27 15:31:15,892][20292] KL-divergence is very high: 109.9693 [2023-12-27 15:31:15,932][20381] Updated weights for policy 1, policy_version 1423748 (0.0006) [2023-12-27 15:31:15,943][20292] KL-divergence is very high: 118.7950 [2023-12-27 15:31:15,991][20381] Updated weights for policy 1, policy_version 1423758 (0.0006) [2023-12-27 15:31:16,306][20347] Updated weights for policy 0, policy_version 1425001 (0.0008) [2023-12-27 15:31:16,363][20347] Updated weights for policy 0, policy_version 1425011 (0.0008) [2023-12-27 15:31:16,426][20347] Updated weights for policy 0, policy_version 1425021 (0.0007) [2023-12-27 15:31:16,477][20347] Updated weights for policy 0, policy_version 1425031 (0.0005) [2023-12-27 15:31:16,634][20381] Updated weights for policy 1, policy_version 1423768 (0.0007) [2023-12-27 15:31:16,691][20381] Updated weights for policy 1, policy_version 1423778 (0.0006) [2023-12-27 15:31:16,752][20381] Updated weights for policy 1, policy_version 1423788 (0.0007) [2023-12-27 15:31:17,160][20347] Updated weights for policy 0, policy_version 1425041 (0.0008) [2023-12-27 15:31:17,218][20347] Updated weights for policy 0, policy_version 1425051 (0.0007) [2023-12-27 15:31:17,275][20347] Updated weights for policy 0, policy_version 1425061 (0.0007) [2023-12-27 15:31:17,321][20381] Updated weights for policy 1, policy_version 1423798 (0.0008) [2023-12-27 15:31:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 729407488. Throughput: 0: 10218.6, 1: 10131.1. Samples: 729417444. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:17,363][19170] Avg episode reward: [(0, '970.690'), (1, '914.280')] [2023-12-27 15:31:17,372][20381] Updated weights for policy 1, policy_version 1423808 (0.0008) [2023-12-27 15:31:17,429][20381] Updated weights for policy 1, policy_version 1423818 (0.0007) [2023-12-27 15:31:17,951][20347] Updated weights for policy 0, policy_version 1425071 (0.0006) [2023-12-27 15:31:18,000][20347] Updated weights for policy 0, policy_version 1425081 (0.0005) [2023-12-27 15:31:18,051][20347] Updated weights for policy 0, policy_version 1425091 (0.0005) [2023-12-27 15:31:18,128][20381] Updated weights for policy 1, policy_version 1423828 (0.0006) [2023-12-27 15:31:18,176][20381] Updated weights for policy 1, policy_version 1423838 (0.0005) [2023-12-27 15:31:18,232][20381] Updated weights for policy 1, policy_version 1423848 (0.0007) [2023-12-27 15:31:18,742][20347] Updated weights for policy 0, policy_version 1425101 (0.0006) [2023-12-27 15:31:18,804][20347] Updated weights for policy 0, policy_version 1425111 (0.0008) [2023-12-27 15:31:18,869][20347] Updated weights for policy 0, policy_version 1425121 (0.0008) [2023-12-27 15:31:18,921][20381] Updated weights for policy 1, policy_version 1423858 (0.0008) [2023-12-27 15:31:18,978][20381] Updated weights for policy 1, policy_version 1423868 (0.0008) [2023-12-27 15:31:19,035][20381] Updated weights for policy 1, policy_version 1423878 (0.0008) [2023-12-27 15:31:19,090][20381] Updated weights for policy 1, policy_version 1423888 (0.0008) [2023-12-27 15:31:19,581][20347] Updated weights for policy 0, policy_version 1425131 (0.0008) [2023-12-27 15:31:19,639][20347] Updated weights for policy 0, policy_version 1425141 (0.0008) [2023-12-27 15:31:19,698][20347] Updated weights for policy 0, policy_version 1425151 (0.0007) [2023-12-27 15:31:19,834][20381] Updated weights for policy 1, policy_version 1423898 (0.0008) [2023-12-27 15:31:19,889][20381] Updated weights for policy 1, policy_version 1423908 (0.0006) [2023-12-27 15:31:19,958][20381] Updated weights for policy 1, policy_version 1423918 (0.0007) [2023-12-27 15:31:20,405][20347] Updated weights for policy 0, policy_version 1425161 (0.0006) [2023-12-27 15:31:20,453][20347] Updated weights for policy 0, policy_version 1425171 (0.0009) [2023-12-27 15:31:20,505][20347] Updated weights for policy 0, policy_version 1425181 (0.0008) [2023-12-27 15:31:20,556][20347] Updated weights for policy 0, policy_version 1425191 (0.0008) [2023-12-27 15:31:20,633][20381] Updated weights for policy 1, policy_version 1423928 (0.0006) [2023-12-27 15:31:20,689][20381] Updated weights for policy 1, policy_version 1423938 (0.0006) [2023-12-27 15:31:20,750][20381] Updated weights for policy 1, policy_version 1423948 (0.0008) [2023-12-27 15:31:21,282][20291] KL-divergence is very high: 471.0625 [2023-12-27 15:31:21,298][20291] KL-divergence is very high: 617.0423 [2023-12-27 15:31:21,304][20347] Updated weights for policy 0, policy_version 1425201 (0.0008) [2023-12-27 15:31:21,328][20291] KL-divergence is very high: 682.2261 [2023-12-27 15:31:21,348][20291] KL-divergence is very high: 368.0228 [2023-12-27 15:31:21,371][20347] Updated weights for policy 0, policy_version 1425211 (0.0008) [2023-12-27 15:31:21,389][20291] KL-divergence is very high: 117.8640 [2023-12-27 15:31:21,435][20347] Updated weights for policy 0, policy_version 1425221 (0.0009) [2023-12-27 15:31:21,436][20291] KL-divergence is very high: 104.3795 [2023-12-27 15:31:21,490][20381] Updated weights for policy 1, policy_version 1423958 (0.0008) [2023-12-27 15:31:21,547][20381] Updated weights for policy 1, policy_version 1423968 (0.0008) [2023-12-27 15:31:21,604][20381] Updated weights for policy 1, policy_version 1423978 (0.0007) [2023-12-27 15:31:22,127][20291] KL-divergence is very high: 1117.2688 [2023-12-27 15:31:22,168][20347] Updated weights for policy 0, policy_version 1425231 (0.0008) [2023-12-27 15:31:22,169][20291] KL-divergence is very high: 494.1425 [2023-12-27 15:31:22,175][20291] KL-divergence is very high: 137.3849 [2023-12-27 15:31:22,219][20291] KL-divergence is very high: 798.5116 [2023-12-27 15:31:22,230][20347] Updated weights for policy 0, policy_version 1425241 (0.0008) [2023-12-27 15:31:22,266][20291] KL-divergence is very high: 490.1561 [2023-12-27 15:31:22,291][20347] Updated weights for policy 0, policy_version 1425251 (0.0008) [2023-12-27 15:31:22,314][20291] KL-divergence is very high: 237.9053 [2023-12-27 15:31:22,326][20381] Updated weights for policy 1, policy_version 1423988 (0.0008) [2023-12-27 15:31:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 729505792. Throughput: 0: 10205.4, 1: 10097.4. Samples: 729476792. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:22,363][19170] Avg episode reward: [(0, '912.510'), (1, '951.760')] [2023-12-27 15:31:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001425256_364937216.pth... [2023-12-27 15:31:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001424072_364634112.pth [2023-12-27 15:31:22,399][20381] Updated weights for policy 1, policy_version 1423998 (0.0007) [2023-12-27 15:31:22,462][20381] Updated weights for policy 1, policy_version 1424008 (0.0007) [2023-12-27 15:31:22,516][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001424016_364576768.pth... [2023-12-27 15:31:22,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001422800_364265472.pth [2023-12-27 15:31:23,023][20347] Updated weights for policy 0, policy_version 1425261 (0.0009) [2023-12-27 15:31:23,069][20381] Updated weights for policy 1, policy_version 1424018 (0.0007) [2023-12-27 15:31:23,084][20347] Updated weights for policy 0, policy_version 1425271 (0.0010) [2023-12-27 15:31:23,135][20381] Updated weights for policy 1, policy_version 1424028 (0.0009) [2023-12-27 15:31:23,140][20347] Updated weights for policy 0, policy_version 1425281 (0.0010) [2023-12-27 15:31:23,193][20381] Updated weights for policy 1, policy_version 1424038 (0.0009) [2023-12-27 15:31:23,241][20381] Updated weights for policy 1, policy_version 1424048 (0.0010) [2023-12-27 15:31:23,835][20347] Updated weights for policy 0, policy_version 1425291 (0.0009) [2023-12-27 15:31:23,872][20381] Updated weights for policy 1, policy_version 1424058 (0.0007) [2023-12-27 15:31:23,904][20347] Updated weights for policy 0, policy_version 1425301 (0.0008) [2023-12-27 15:31:23,933][20381] Updated weights for policy 1, policy_version 1424068 (0.0007) [2023-12-27 15:31:23,971][20347] Updated weights for policy 0, policy_version 1425311 (0.0006) [2023-12-27 15:31:23,984][20381] Updated weights for policy 1, policy_version 1424078 (0.0007) [2023-12-27 15:31:24,629][20347] Updated weights for policy 0, policy_version 1425321 (0.0007) [2023-12-27 15:31:24,634][20381] Updated weights for policy 1, policy_version 1424088 (0.0006) [2023-12-27 15:31:24,655][20292] KL-divergence is very high: 116.5722 [2023-12-27 15:31:24,678][20292] KL-divergence is very high: 179.5541 [2023-12-27 15:31:24,683][20292] KL-divergence is very high: 152.9836 [2023-12-27 15:31:24,684][20347] Updated weights for policy 0, policy_version 1425331 (0.0007) [2023-12-27 15:31:24,692][20381] Updated weights for policy 1, policy_version 1424098 (0.0005) [2023-12-27 15:31:24,700][20292] KL-divergence is very high: 114.1640 [2023-12-27 15:31:24,727][20292] KL-divergence is very high: 101.9024 [2023-12-27 15:31:24,738][20347] Updated weights for policy 0, policy_version 1425341 (0.0007) [2023-12-27 15:31:24,752][20381] Updated weights for policy 1, policy_version 1424108 (0.0006) [2023-12-27 15:31:24,798][20347] Updated weights for policy 0, policy_version 1425351 (0.0007) [2023-12-27 15:31:25,412][20381] Updated weights for policy 1, policy_version 1424118 (0.0006) [2023-12-27 15:31:25,458][20347] Updated weights for policy 0, policy_version 1425361 (0.0006) [2023-12-27 15:31:25,462][20381] Updated weights for policy 1, policy_version 1424128 (0.0007) [2023-12-27 15:31:25,508][20381] Updated weights for policy 1, policy_version 1424138 (0.0007) [2023-12-27 15:31:25,516][20347] Updated weights for policy 0, policy_version 1425371 (0.0005) [2023-12-27 15:31:25,573][20347] Updated weights for policy 0, policy_version 1425381 (0.0005) [2023-12-27 15:31:26,131][20347] Updated weights for policy 0, policy_version 1425391 (0.0007) [2023-12-27 15:31:26,192][20347] Updated weights for policy 0, policy_version 1425401 (0.0007) [2023-12-27 15:31:26,241][20381] Updated weights for policy 1, policy_version 1424148 (0.0006) [2023-12-27 15:31:26,250][20347] Updated weights for policy 0, policy_version 1425411 (0.0007) [2023-12-27 15:31:26,290][20381] Updated weights for policy 1, policy_version 1424158 (0.0006) [2023-12-27 15:31:26,339][20381] Updated weights for policy 1, policy_version 1424168 (0.0008) [2023-12-27 15:31:26,937][20347] Updated weights for policy 0, policy_version 1425421 (0.0007) [2023-12-27 15:31:26,992][20347] Updated weights for policy 0, policy_version 1425431 (0.0008) [2023-12-27 15:31:27,037][20381] Updated weights for policy 1, policy_version 1424178 (0.0008) [2023-12-27 15:31:27,043][20347] Updated weights for policy 0, policy_version 1425441 (0.0006) [2023-12-27 15:31:27,083][20381] Updated weights for policy 1, policy_version 1424188 (0.0006) [2023-12-27 15:31:27,137][20381] Updated weights for policy 1, policy_version 1424198 (0.0007) [2023-12-27 15:31:27,182][20381] Updated weights for policy 1, policy_version 1424208 (0.0007) [2023-12-27 15:31:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 729612288. Throughput: 0: 10266.0, 1: 10150.8. Samples: 729598380. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:27,363][19170] Avg episode reward: [(0, '931.990'), (1, '942.120')] [2023-12-27 15:31:27,733][20347] Updated weights for policy 0, policy_version 1425451 (0.0006) [2023-12-27 15:31:27,785][20347] Updated weights for policy 0, policy_version 1425461 (0.0008) [2023-12-27 15:31:27,837][20347] Updated weights for policy 0, policy_version 1425471 (0.0006) [2023-12-27 15:31:27,879][20381] Updated weights for policy 1, policy_version 1424218 (0.0006) [2023-12-27 15:31:27,931][20381] Updated weights for policy 1, policy_version 1424228 (0.0007) [2023-12-27 15:31:27,979][20381] Updated weights for policy 1, policy_version 1424238 (0.0008) [2023-12-27 15:31:28,569][20347] Updated weights for policy 0, policy_version 1425481 (0.0005) [2023-12-27 15:31:28,604][20381] Updated weights for policy 1, policy_version 1424248 (0.0006) [2023-12-27 15:31:28,628][20347] Updated weights for policy 0, policy_version 1425491 (0.0008) [2023-12-27 15:31:28,663][20381] Updated weights for policy 1, policy_version 1424258 (0.0005) [2023-12-27 15:31:28,687][20347] Updated weights for policy 0, policy_version 1425501 (0.0008) [2023-12-27 15:31:28,716][20381] Updated weights for policy 1, policy_version 1424268 (0.0005) [2023-12-27 15:31:28,742][20347] Updated weights for policy 0, policy_version 1425511 (0.0007) [2023-12-27 15:31:29,374][20381] Updated weights for policy 1, policy_version 1424278 (0.0009) [2023-12-27 15:31:29,438][20381] Updated weights for policy 1, policy_version 1424288 (0.0007) [2023-12-27 15:31:29,492][20347] Updated weights for policy 0, policy_version 1425521 (0.0006) [2023-12-27 15:31:29,496][20381] Updated weights for policy 1, policy_version 1424298 (0.0007) [2023-12-27 15:31:29,543][20347] Updated weights for policy 0, policy_version 1425531 (0.0007) [2023-12-27 15:31:29,604][20347] Updated weights for policy 0, policy_version 1425541 (0.0008) [2023-12-27 15:31:30,170][20381] Updated weights for policy 1, policy_version 1424308 (0.0009) [2023-12-27 15:31:30,229][20381] Updated weights for policy 1, policy_version 1424318 (0.0011) [2023-12-27 15:31:30,257][20347] Updated weights for policy 0, policy_version 1425551 (0.0006) [2023-12-27 15:31:30,281][20381] Updated weights for policy 1, policy_version 1424328 (0.0008) [2023-12-27 15:31:30,309][20347] Updated weights for policy 0, policy_version 1425561 (0.0006) [2023-12-27 15:31:30,371][20347] Updated weights for policy 0, policy_version 1425571 (0.0008) [2023-12-27 15:31:30,958][20381] Updated weights for policy 1, policy_version 1424338 (0.0010) [2023-12-27 15:31:30,996][20347] Updated weights for policy 0, policy_version 1425581 (0.0008) [2023-12-27 15:31:31,010][20381] Updated weights for policy 1, policy_version 1424348 (0.0010) [2023-12-27 15:31:31,056][20347] Updated weights for policy 0, policy_version 1425591 (0.0006) [2023-12-27 15:31:31,072][20381] Updated weights for policy 1, policy_version 1424358 (0.0011) [2023-12-27 15:31:31,115][20347] Updated weights for policy 0, policy_version 1425601 (0.0006) [2023-12-27 15:31:31,124][20381] Updated weights for policy 1, policy_version 1424368 (0.0010) [2023-12-27 15:31:31,791][20347] Updated weights for policy 0, policy_version 1425611 (0.0007) [2023-12-27 15:31:31,851][20347] Updated weights for policy 0, policy_version 1425621 (0.0006) [2023-12-27 15:31:31,875][20381] Updated weights for policy 1, policy_version 1424378 (0.0005) [2023-12-27 15:31:31,907][20347] Updated weights for policy 0, policy_version 1425631 (0.0007) [2023-12-27 15:31:31,932][20381] Updated weights for policy 1, policy_version 1424388 (0.0005) [2023-12-27 15:31:31,988][20381] Updated weights for policy 1, policy_version 1424398 (0.0006) [2023-12-27 15:31:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 729710592. Throughput: 0: 10171.2, 1: 10168.5. Samples: 729721180. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:32,363][19170] Avg episode reward: [(0, '931.680'), (1, '914.640')] [2023-12-27 15:31:32,516][20347] Updated weights for policy 0, policy_version 1425641 (0.0006) [2023-12-27 15:31:32,576][20347] Updated weights for policy 0, policy_version 1425651 (0.0007) [2023-12-27 15:31:32,597][20381] Updated weights for policy 1, policy_version 1424408 (0.0006) [2023-12-27 15:31:32,634][20347] Updated weights for policy 0, policy_version 1425661 (0.0006) [2023-12-27 15:31:32,653][20381] Updated weights for policy 1, policy_version 1424418 (0.0007) [2023-12-27 15:31:32,695][20347] Updated weights for policy 0, policy_version 1425671 (0.0007) [2023-12-27 15:31:32,710][20381] Updated weights for policy 1, policy_version 1424428 (0.0008) [2023-12-27 15:31:33,240][20347] Updated weights for policy 0, policy_version 1425681 (0.0005) [2023-12-27 15:31:33,288][20347] Updated weights for policy 0, policy_version 1425691 (0.0005) [2023-12-27 15:31:33,348][20347] Updated weights for policy 0, policy_version 1425701 (0.0005) [2023-12-27 15:31:33,394][20381] Updated weights for policy 1, policy_version 1424438 (0.0009) [2023-12-27 15:31:33,450][20381] Updated weights for policy 1, policy_version 1424448 (0.0007) [2023-12-27 15:31:33,500][20381] Updated weights for policy 1, policy_version 1424458 (0.0010) [2023-12-27 15:31:33,886][20347] Updated weights for policy 0, policy_version 1425711 (0.0005) [2023-12-27 15:31:33,940][20347] Updated weights for policy 0, policy_version 1425721 (0.0008) [2023-12-27 15:31:33,983][20347] Updated weights for policy 0, policy_version 1425731 (0.0006) [2023-12-27 15:31:34,244][20381] Updated weights for policy 1, policy_version 1424468 (0.0009) [2023-12-27 15:31:34,307][20381] Updated weights for policy 1, policy_version 1424478 (0.0008) [2023-12-27 15:31:34,365][20381] Updated weights for policy 1, policy_version 1424488 (0.0008) [2023-12-27 15:31:34,638][20347] Updated weights for policy 0, policy_version 1425741 (0.0007) [2023-12-27 15:31:34,698][20347] Updated weights for policy 0, policy_version 1425751 (0.0011) [2023-12-27 15:31:34,744][20291] KL-divergence is very high: 1252.9246 [2023-12-27 15:31:34,749][20291] KL-divergence is very high: 1156.9303 [2023-12-27 15:31:34,757][20347] Updated weights for policy 0, policy_version 1425761 (0.0010) [2023-12-27 15:31:34,786][20291] KL-divergence is very high: 487.8499 [2023-12-27 15:31:35,017][20381] Updated weights for policy 1, policy_version 1424498 (0.0008) [2023-12-27 15:31:35,070][20381] Updated weights for policy 1, policy_version 1424508 (0.0008) [2023-12-27 15:31:35,125][20381] Updated weights for policy 1, policy_version 1424518 (0.0008) [2023-12-27 15:31:35,185][20381] Updated weights for policy 1, policy_version 1424528 (0.0009) [2023-12-27 15:31:35,367][20347] Updated weights for policy 0, policy_version 1425771 (0.0009) [2023-12-27 15:31:35,425][20347] Updated weights for policy 0, policy_version 1425781 (0.0005) [2023-12-27 15:31:35,478][20347] Updated weights for policy 0, policy_version 1425791 (0.0006) [2023-12-27 15:31:35,843][20381] Updated weights for policy 1, policy_version 1424538 (0.0005) [2023-12-27 15:31:35,903][20381] Updated weights for policy 1, policy_version 1424548 (0.0006) [2023-12-27 15:31:35,960][20381] Updated weights for policy 1, policy_version 1424558 (0.0006) [2023-12-27 15:31:36,078][20347] Updated weights for policy 0, policy_version 1425801 (0.0010) [2023-12-27 15:31:36,080][20291] KL-divergence is very high: 184.2874 [2023-12-27 15:31:36,134][20291] KL-divergence is very high: 154.8299 [2023-12-27 15:31:36,147][20347] Updated weights for policy 0, policy_version 1425811 (0.0010) [2023-12-27 15:31:36,198][20347] Updated weights for policy 0, policy_version 1425821 (0.0007) [2023-12-27 15:31:36,257][20347] Updated weights for policy 0, policy_version 1425831 (0.0006) [2023-12-27 15:31:36,623][20381] Updated weights for policy 1, policy_version 1424568 (0.0007) [2023-12-27 15:31:36,683][20381] Updated weights for policy 1, policy_version 1424578 (0.0009) [2023-12-27 15:31:36,740][20381] Updated weights for policy 1, policy_version 1424588 (0.0007) [2023-12-27 15:31:36,834][20347] Updated weights for policy 0, policy_version 1425841 (0.0007) [2023-12-27 15:31:36,840][20291] KL-divergence is very high: 105.5815 [2023-12-27 15:31:36,891][20347] Updated weights for policy 0, policy_version 1425851 (0.0010) [2023-12-27 15:31:36,944][20347] Updated weights for policy 0, policy_version 1425861 (0.0010) [2023-12-27 15:31:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 729817088. Throughput: 0: 10225.7, 1: 10186.0. Samples: 729785552. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:37,364][19170] Avg episode reward: [(0, '873.440'), (1, '925.810')] [2023-12-27 15:31:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001424592_364724224.pth... [2023-12-27 15:31:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001425864_365092864.pth... [2023-12-27 15:31:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001423408_364421120.pth [2023-12-27 15:31:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001424680_364789760.pth [2023-12-27 15:31:37,435][20381] Updated weights for policy 1, policy_version 1424598 (0.0007) [2023-12-27 15:31:37,496][20381] Updated weights for policy 1, policy_version 1424608 (0.0006) [2023-12-27 15:31:37,554][20381] Updated weights for policy 1, policy_version 1424618 (0.0008) [2023-12-27 15:31:37,602][20347] Updated weights for policy 0, policy_version 1425871 (0.0007) [2023-12-27 15:31:37,668][20347] Updated weights for policy 0, policy_version 1425881 (0.0007) [2023-12-27 15:31:37,674][20291] KL-divergence is very high: 254.3107 [2023-12-27 15:31:37,726][20291] KL-divergence is very high: 185.4334 [2023-12-27 15:31:37,732][20347] Updated weights for policy 0, policy_version 1425891 (0.0008) [2023-12-27 15:31:38,243][20381] Updated weights for policy 1, policy_version 1424628 (0.0007) [2023-12-27 15:31:38,304][20381] Updated weights for policy 1, policy_version 1424638 (0.0008) [2023-12-27 15:31:38,337][20347] Updated weights for policy 0, policy_version 1425901 (0.0006) [2023-12-27 15:31:38,371][20381] Updated weights for policy 1, policy_version 1424648 (0.0007) [2023-12-27 15:31:38,403][20347] Updated weights for policy 0, policy_version 1425911 (0.0006) [2023-12-27 15:31:38,464][20347] Updated weights for policy 0, policy_version 1425921 (0.0006) [2023-12-27 15:31:39,088][20381] Updated weights for policy 1, policy_version 1424658 (0.0008) [2023-12-27 15:31:39,135][20347] Updated weights for policy 0, policy_version 1425931 (0.0006) [2023-12-27 15:31:39,137][20381] Updated weights for policy 1, policy_version 1424668 (0.0007) [2023-12-27 15:31:39,186][20381] Updated weights for policy 1, policy_version 1424678 (0.0005) [2023-12-27 15:31:39,188][20347] Updated weights for policy 0, policy_version 1425941 (0.0007) [2023-12-27 15:31:39,243][20347] Updated weights for policy 0, policy_version 1425951 (0.0007) [2023-12-27 15:31:39,247][20381] Updated weights for policy 1, policy_version 1424688 (0.0008) [2023-12-27 15:31:39,960][20381] Updated weights for policy 1, policy_version 1424698 (0.0008) [2023-12-27 15:31:40,015][20381] Updated weights for policy 1, policy_version 1424708 (0.0006) [2023-12-27 15:31:40,036][20347] Updated weights for policy 0, policy_version 1425961 (0.0008) [2023-12-27 15:31:40,071][20381] Updated weights for policy 1, policy_version 1424718 (0.0008) [2023-12-27 15:31:40,092][20347] Updated weights for policy 0, policy_version 1425971 (0.0006) [2023-12-27 15:31:40,150][20347] Updated weights for policy 0, policy_version 1425981 (0.0005) [2023-12-27 15:31:40,211][20347] Updated weights for policy 0, policy_version 1425991 (0.0007) [2023-12-27 15:31:40,702][20381] Updated weights for policy 1, policy_version 1424728 (0.0005) [2023-12-27 15:31:40,751][20381] Updated weights for policy 1, policy_version 1424738 (0.0005) [2023-12-27 15:31:40,807][20381] Updated weights for policy 1, policy_version 1424748 (0.0005) [2023-12-27 15:31:40,879][20347] Updated weights for policy 0, policy_version 1426001 (0.0010) [2023-12-27 15:31:40,926][20347] Updated weights for policy 0, policy_version 1426011 (0.0006) [2023-12-27 15:31:40,985][20347] Updated weights for policy 0, policy_version 1426021 (0.0005) [2023-12-27 15:31:41,541][20381] Updated weights for policy 1, policy_version 1424758 (0.0007) [2023-12-27 15:31:41,607][20381] Updated weights for policy 1, policy_version 1424768 (0.0008) [2023-12-27 15:31:41,672][20381] Updated weights for policy 1, policy_version 1424778 (0.0009) [2023-12-27 15:31:41,704][20347] Updated weights for policy 0, policy_version 1426031 (0.0007) [2023-12-27 15:31:41,767][20347] Updated weights for policy 0, policy_version 1426041 (0.0006) [2023-12-27 15:31:41,831][20347] Updated weights for policy 0, policy_version 1426051 (0.0006) [2023-12-27 15:31:42,331][20381] Updated weights for policy 1, policy_version 1424788 (0.0008) [2023-12-27 15:31:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 729915392. Throughput: 0: 10282.6, 1: 10245.1. Samples: 729911340. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:42,363][19170] Avg episode reward: [(0, '922.230'), (1, '896.130')] [2023-12-27 15:31:42,396][20381] Updated weights for policy 1, policy_version 1424798 (0.0007) [2023-12-27 15:31:42,457][20381] Updated weights for policy 1, policy_version 1424808 (0.0006) [2023-12-27 15:31:42,526][20347] Updated weights for policy 0, policy_version 1426061 (0.0006) [2023-12-27 15:31:42,588][20347] Updated weights for policy 0, policy_version 1426071 (0.0008) [2023-12-27 15:31:42,642][20347] Updated weights for policy 0, policy_version 1426081 (0.0008) [2023-12-27 15:31:43,034][20381] Updated weights for policy 1, policy_version 1424818 (0.0007) [2023-12-27 15:31:43,094][20381] Updated weights for policy 1, policy_version 1424828 (0.0006) [2023-12-27 15:31:43,149][20381] Updated weights for policy 1, policy_version 1424838 (0.0006) [2023-12-27 15:31:43,195][20381] Updated weights for policy 1, policy_version 1424848 (0.0007) [2023-12-27 15:31:43,388][20347] Updated weights for policy 0, policy_version 1426091 (0.0009) [2023-12-27 15:31:43,433][20347] Updated weights for policy 0, policy_version 1426101 (0.0007) [2023-12-27 15:31:43,482][20347] Updated weights for policy 0, policy_version 1426111 (0.0008) [2023-12-27 15:31:43,779][20381] Updated weights for policy 1, policy_version 1424858 (0.0005) [2023-12-27 15:31:43,828][20381] Updated weights for policy 1, policy_version 1424868 (0.0007) [2023-12-27 15:31:43,880][20381] Updated weights for policy 1, policy_version 1424878 (0.0008) [2023-12-27 15:31:44,206][20347] Updated weights for policy 0, policy_version 1426121 (0.0008) [2023-12-27 15:31:44,262][20347] Updated weights for policy 0, policy_version 1426131 (0.0007) [2023-12-27 15:31:44,323][20347] Updated weights for policy 0, policy_version 1426141 (0.0008) [2023-12-27 15:31:44,374][20347] Updated weights for policy 0, policy_version 1426151 (0.0007) [2023-12-27 15:31:44,490][20381] Updated weights for policy 1, policy_version 1424888 (0.0010) [2023-12-27 15:31:44,547][20381] Updated weights for policy 1, policy_version 1424898 (0.0008) [2023-12-27 15:31:44,600][20381] Updated weights for policy 1, policy_version 1424908 (0.0008) [2023-12-27 15:31:45,097][20347] Updated weights for policy 0, policy_version 1426161 (0.0007) [2023-12-27 15:31:45,158][20347] Updated weights for policy 0, policy_version 1426171 (0.0006) [2023-12-27 15:31:45,218][20347] Updated weights for policy 0, policy_version 1426181 (0.0006) [2023-12-27 15:31:45,321][20381] Updated weights for policy 1, policy_version 1424918 (0.0006) [2023-12-27 15:31:45,378][20381] Updated weights for policy 1, policy_version 1424928 (0.0006) [2023-12-27 15:31:45,466][20381] Updated weights for policy 1, policy_version 1424938 (0.0006) [2023-12-27 15:31:45,847][20347] Updated weights for policy 0, policy_version 1426191 (0.0009) [2023-12-27 15:31:45,898][20347] Updated weights for policy 0, policy_version 1426201 (0.0009) [2023-12-27 15:31:45,957][20347] Updated weights for policy 0, policy_version 1426211 (0.0007) [2023-12-27 15:31:46,062][20381] Updated weights for policy 1, policy_version 1424948 (0.0006) [2023-12-27 15:31:46,127][20381] Updated weights for policy 1, policy_version 1424958 (0.0008) [2023-12-27 15:31:46,186][20381] Updated weights for policy 1, policy_version 1424968 (0.0008) [2023-12-27 15:31:46,593][20347] Updated weights for policy 0, policy_version 1426221 (0.0007) [2023-12-27 15:31:46,643][20347] Updated weights for policy 0, policy_version 1426231 (0.0007) [2023-12-27 15:31:46,693][20347] Updated weights for policy 0, policy_version 1426241 (0.0005) [2023-12-27 15:31:46,910][20381] Updated weights for policy 1, policy_version 1424978 (0.0009) [2023-12-27 15:31:46,966][20381] Updated weights for policy 1, policy_version 1424988 (0.0008) [2023-12-27 15:31:47,027][20381] Updated weights for policy 1, policy_version 1424998 (0.0009) [2023-12-27 15:31:47,086][20381] Updated weights for policy 1, policy_version 1425008 (0.0008) [2023-12-27 15:31:47,280][20347] Updated weights for policy 0, policy_version 1426251 (0.0007) [2023-12-27 15:31:47,333][20347] Updated weights for policy 0, policy_version 1426261 (0.0006) [2023-12-27 15:31:47,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 730021888. Throughput: 0: 10256.9, 1: 10275.1. Samples: 730034480. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:47,363][19170] Avg episode reward: [(0, '941.670'), (1, '942.720')] [2023-12-27 15:31:47,390][20347] Updated weights for policy 0, policy_version 1426271 (0.0005) [2023-12-27 15:31:47,804][20381] Updated weights for policy 1, policy_version 1425018 (0.0007) [2023-12-27 15:31:47,862][20381] Updated weights for policy 1, policy_version 1425028 (0.0007) [2023-12-27 15:31:47,920][20381] Updated weights for policy 1, policy_version 1425038 (0.0006) [2023-12-27 15:31:47,983][20347] Updated weights for policy 0, policy_version 1426281 (0.0006) [2023-12-27 15:31:48,049][20347] Updated weights for policy 0, policy_version 1426291 (0.0008) [2023-12-27 15:31:48,116][20347] Updated weights for policy 0, policy_version 1426301 (0.0005) [2023-12-27 15:31:48,181][20347] Updated weights for policy 0, policy_version 1426311 (0.0008) [2023-12-27 15:31:48,646][20381] Updated weights for policy 1, policy_version 1425048 (0.0008) [2023-12-27 15:31:48,702][20381] Updated weights for policy 1, policy_version 1425058 (0.0008) [2023-12-27 15:31:48,758][20381] Updated weights for policy 1, policy_version 1425068 (0.0008) [2023-12-27 15:31:48,759][20347] Updated weights for policy 0, policy_version 1426321 (0.0006) [2023-12-27 15:31:48,816][20347] Updated weights for policy 0, policy_version 1426331 (0.0005) [2023-12-27 15:31:48,872][20347] Updated weights for policy 0, policy_version 1426341 (0.0006) [2023-12-27 15:31:49,487][20347] Updated weights for policy 0, policy_version 1426351 (0.0007) [2023-12-27 15:31:49,513][20381] Updated weights for policy 1, policy_version 1425078 (0.0007) [2023-12-27 15:31:49,535][20347] Updated weights for policy 0, policy_version 1426361 (0.0008) [2023-12-27 15:31:49,570][20381] Updated weights for policy 1, policy_version 1425088 (0.0007) [2023-12-27 15:31:49,588][20347] Updated weights for policy 0, policy_version 1426371 (0.0010) [2023-12-27 15:31:49,623][20381] Updated weights for policy 1, policy_version 1425098 (0.0006) [2023-12-27 15:31:50,252][20381] Updated weights for policy 1, policy_version 1425108 (0.0007) [2023-12-27 15:31:50,310][20381] Updated weights for policy 1, policy_version 1425118 (0.0008) [2023-12-27 15:31:50,348][20347] Updated weights for policy 0, policy_version 1426381 (0.0008) [2023-12-27 15:31:50,364][20381] Updated weights for policy 1, policy_version 1425128 (0.0006) [2023-12-27 15:31:50,398][20347] Updated weights for policy 0, policy_version 1426391 (0.0006) [2023-12-27 15:31:50,447][20347] Updated weights for policy 0, policy_version 1426401 (0.0009) [2023-12-27 15:31:51,064][20381] Updated weights for policy 1, policy_version 1425138 (0.0005) [2023-12-27 15:31:51,129][20381] Updated weights for policy 1, policy_version 1425148 (0.0008) [2023-12-27 15:31:51,189][20381] Updated weights for policy 1, policy_version 1425158 (0.0006) [2023-12-27 15:31:51,218][20347] Updated weights for policy 0, policy_version 1426411 (0.0007) [2023-12-27 15:31:51,248][20381] Updated weights for policy 1, policy_version 1425168 (0.0007) [2023-12-27 15:31:51,274][20347] Updated weights for policy 0, policy_version 1426421 (0.0007) [2023-12-27 15:31:51,335][20347] Updated weights for policy 0, policy_version 1426431 (0.0008) [2023-12-27 15:31:52,000][20381] Updated weights for policy 1, policy_version 1425178 (0.0010) [2023-12-27 15:31:52,013][20347] Updated weights for policy 0, policy_version 1426441 (0.0008) [2023-12-27 15:31:52,057][20381] Updated weights for policy 1, policy_version 1425188 (0.0007) [2023-12-27 15:31:52,075][20347] Updated weights for policy 0, policy_version 1426451 (0.0006) [2023-12-27 15:31:52,116][20381] Updated weights for policy 1, policy_version 1425198 (0.0006) [2023-12-27 15:31:52,126][20347] Updated weights for policy 0, policy_version 1426461 (0.0007) [2023-12-27 15:31:52,156][20291] KL-divergence is very high: 206.6674 [2023-12-27 15:31:52,173][20291] KL-divergence is very high: 396.8689 [2023-12-27 15:31:52,179][20291] KL-divergence is very high: 382.5394 [2023-12-27 15:31:52,185][20347] Updated weights for policy 0, policy_version 1426471 (0.0008) [2023-12-27 15:31:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 730128384. Throughput: 0: 10299.6, 1: 10260.1. Samples: 730097160. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:52,363][19170] Avg episode reward: [(0, '912.350'), (1, '980.960')] [2023-12-27 15:31:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001425200_364879872.pth... [2023-12-27 15:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001426472_365248512.pth... [2023-12-27 15:31:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001425256_364937216.pth [2023-12-27 15:31:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001424016_364576768.pth [2023-12-27 15:31:52,750][20381] Updated weights for policy 1, policy_version 1425208 (0.0007) [2023-12-27 15:31:52,810][20381] Updated weights for policy 1, policy_version 1425218 (0.0007) [2023-12-27 15:31:52,847][20291] KL-divergence is very high: 403.4017 [2023-12-27 15:31:52,865][20381] Updated weights for policy 1, policy_version 1425228 (0.0007) [2023-12-27 15:31:52,885][20347] Updated weights for policy 0, policy_version 1426481 (0.0007) [2023-12-27 15:31:52,889][20291] KL-divergence is very high: 276.3430 [2023-12-27 15:31:52,929][20291] KL-divergence is very high: 169.9958 [2023-12-27 15:31:52,933][20347] Updated weights for policy 0, policy_version 1426491 (0.0007) [2023-12-27 15:31:52,971][20291] KL-divergence is very high: 101.3902 [2023-12-27 15:31:52,987][20347] Updated weights for policy 0, policy_version 1426501 (0.0007) [2023-12-27 15:31:53,544][20381] Updated weights for policy 1, policy_version 1425238 (0.0009) [2023-12-27 15:31:53,605][20381] Updated weights for policy 1, policy_version 1425248 (0.0011) [2023-12-27 15:31:53,657][20381] Updated weights for policy 1, policy_version 1425258 (0.0011) [2023-12-27 15:31:53,706][20347] Updated weights for policy 0, policy_version 1426511 (0.0006) [2023-12-27 15:31:53,772][20347] Updated weights for policy 0, policy_version 1426521 (0.0008) [2023-12-27 15:31:53,836][20347] Updated weights for policy 0, policy_version 1426531 (0.0008) [2023-12-27 15:31:54,271][20381] Updated weights for policy 1, policy_version 1425268 (0.0010) [2023-12-27 15:31:54,336][20381] Updated weights for policy 1, policy_version 1425278 (0.0008) [2023-12-27 15:31:54,401][20381] Updated weights for policy 1, policy_version 1425288 (0.0008) [2023-12-27 15:31:54,582][20347] Updated weights for policy 0, policy_version 1426541 (0.0008) [2023-12-27 15:31:54,635][20347] Updated weights for policy 0, policy_version 1426551 (0.0008) [2023-12-27 15:31:54,688][20347] Updated weights for policy 0, policy_version 1426561 (0.0008) [2023-12-27 15:31:55,011][20381] Updated weights for policy 1, policy_version 1425298 (0.0008) [2023-12-27 15:31:55,070][20381] Updated weights for policy 1, policy_version 1425308 (0.0008) [2023-12-27 15:31:55,116][20381] Updated weights for policy 1, policy_version 1425318 (0.0007) [2023-12-27 15:31:55,166][20381] Updated weights for policy 1, policy_version 1425328 (0.0008) [2023-12-27 15:31:55,402][20347] Updated weights for policy 0, policy_version 1426571 (0.0008) [2023-12-27 15:31:55,451][20347] Updated weights for policy 0, policy_version 1426581 (0.0007) [2023-12-27 15:31:55,503][20347] Updated weights for policy 0, policy_version 1426591 (0.0008) [2023-12-27 15:31:55,891][20381] Updated weights for policy 1, policy_version 1425338 (0.0006) [2023-12-27 15:31:55,947][20381] Updated weights for policy 1, policy_version 1425348 (0.0008) [2023-12-27 15:31:56,000][20381] Updated weights for policy 1, policy_version 1425358 (0.0008) [2023-12-27 15:31:56,205][20347] Updated weights for policy 0, policy_version 1426601 (0.0007) [2023-12-27 15:31:56,258][20347] Updated weights for policy 0, policy_version 1426611 (0.0006) [2023-12-27 15:31:56,314][20347] Updated weights for policy 0, policy_version 1426621 (0.0007) [2023-12-27 15:31:56,361][20347] Updated weights for policy 0, policy_version 1426631 (0.0007) [2023-12-27 15:31:56,606][20381] Updated weights for policy 1, policy_version 1425368 (0.0008) [2023-12-27 15:31:56,651][20381] Updated weights for policy 1, policy_version 1425378 (0.0007) [2023-12-27 15:31:56,699][20381] Updated weights for policy 1, policy_version 1425388 (0.0007) [2023-12-27 15:31:57,107][20347] Updated weights for policy 0, policy_version 1426641 (0.0007) [2023-12-27 15:31:57,153][20347] Updated weights for policy 0, policy_version 1426651 (0.0005) [2023-12-27 15:31:57,198][20347] Updated weights for policy 0, policy_version 1426661 (0.0005) [2023-12-27 15:31:57,272][20381] Updated weights for policy 1, policy_version 1425398 (0.0007) [2023-12-27 15:31:57,328][20381] Updated weights for policy 1, policy_version 1425408 (0.0008) [2023-12-27 15:31:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 730226688. Throughput: 0: 10251.3, 1: 10256.3. Samples: 730217172. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:31:57,363][19170] Avg episode reward: [(0, '941.490'), (1, '980.880')] [2023-12-27 15:31:57,374][20381] Updated weights for policy 1, policy_version 1425418 (0.0006) [2023-12-27 15:31:57,774][20347] Updated weights for policy 0, policy_version 1426671 (0.0005) [2023-12-27 15:31:57,836][20347] Updated weights for policy 0, policy_version 1426681 (0.0008) [2023-12-27 15:31:57,885][20347] Updated weights for policy 0, policy_version 1426691 (0.0008) [2023-12-27 15:31:57,942][20381] Updated weights for policy 1, policy_version 1425428 (0.0006) [2023-12-27 15:31:57,994][20381] Updated weights for policy 1, policy_version 1425438 (0.0008) [2023-12-27 15:31:58,046][20381] Updated weights for policy 1, policy_version 1425448 (0.0008) [2023-12-27 15:31:58,619][20347] Updated weights for policy 0, policy_version 1426701 (0.0008) [2023-12-27 15:31:58,680][20347] Updated weights for policy 0, policy_version 1426711 (0.0008) [2023-12-27 15:31:58,744][20347] Updated weights for policy 0, policy_version 1426721 (0.0007) [2023-12-27 15:31:58,799][20381] Updated weights for policy 1, policy_version 1425458 (0.0007) [2023-12-27 15:31:58,862][20381] Updated weights for policy 1, policy_version 1425468 (0.0007) [2023-12-27 15:31:58,926][20381] Updated weights for policy 1, policy_version 1425478 (0.0009) [2023-12-27 15:31:58,979][20381] Updated weights for policy 1, policy_version 1425488 (0.0009) [2023-12-27 15:31:59,443][20347] Updated weights for policy 0, policy_version 1426731 (0.0007) [2023-12-27 15:31:59,498][20347] Updated weights for policy 0, policy_version 1426741 (0.0010) [2023-12-27 15:31:59,553][20347] Updated weights for policy 0, policy_version 1426751 (0.0010) [2023-12-27 15:31:59,710][20381] Updated weights for policy 1, policy_version 1425498 (0.0006) [2023-12-27 15:31:59,765][20381] Updated weights for policy 1, policy_version 1425508 (0.0006) [2023-12-27 15:31:59,824][20381] Updated weights for policy 1, policy_version 1425518 (0.0005) [2023-12-27 15:32:00,255][20347] Updated weights for policy 0, policy_version 1426761 (0.0010) [2023-12-27 15:32:00,310][20347] Updated weights for policy 0, policy_version 1426771 (0.0006) [2023-12-27 15:32:00,374][20347] Updated weights for policy 0, policy_version 1426781 (0.0008) [2023-12-27 15:32:00,438][20347] Updated weights for policy 0, policy_version 1426791 (0.0007) [2023-12-27 15:32:00,494][20381] Updated weights for policy 1, policy_version 1425528 (0.0007) [2023-12-27 15:32:00,539][20381] Updated weights for policy 1, policy_version 1425538 (0.0007) [2023-12-27 15:32:00,601][20381] Updated weights for policy 1, policy_version 1425548 (0.0011) [2023-12-27 15:32:01,100][20347] Updated weights for policy 0, policy_version 1426801 (0.0009) [2023-12-27 15:32:01,165][20347] Updated weights for policy 0, policy_version 1426811 (0.0008) [2023-12-27 15:32:01,218][20347] Updated weights for policy 0, policy_version 1426821 (0.0006) [2023-12-27 15:32:01,311][20381] Updated weights for policy 1, policy_version 1425558 (0.0010) [2023-12-27 15:32:01,373][20381] Updated weights for policy 1, policy_version 1425568 (0.0011) [2023-12-27 15:32:01,440][20381] Updated weights for policy 1, policy_version 1425578 (0.0008) [2023-12-27 15:32:01,927][20347] Updated weights for policy 0, policy_version 1426831 (0.0007) [2023-12-27 15:32:01,986][20347] Updated weights for policy 0, policy_version 1426841 (0.0007) [2023-12-27 15:32:02,040][20347] Updated weights for policy 0, policy_version 1426851 (0.0007) [2023-12-27 15:32:02,140][20381] Updated weights for policy 1, policy_version 1425588 (0.0008) [2023-12-27 15:32:02,195][20381] Updated weights for policy 1, policy_version 1425598 (0.0008) [2023-12-27 15:32:02,249][20381] Updated weights for policy 1, policy_version 1425608 (0.0008) [2023-12-27 15:32:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20617.1, 300 sec: 20466.1). Total num frames: 730333184. Throughput: 0: 10244.6, 1: 10243.9. Samples: 730339428. Policy #0 lag: (min: 27.0, avg: 28.0, max: 51.0) [2023-12-27 15:32:02,363][19170] Avg episode reward: [(0, '941.330'), (1, '915.040')] [2023-12-27 15:32:02,746][20347] Updated weights for policy 0, policy_version 1426861 (0.0007) [2023-12-27 15:32:02,799][20347] Updated weights for policy 0, policy_version 1426871 (0.0007) [2023-12-27 15:32:02,852][20347] Updated weights for policy 0, policy_version 1426881 (0.0007) [2023-12-27 15:32:02,964][20381] Updated weights for policy 1, policy_version 1425618 (0.0010) [2023-12-27 15:32:03,022][20381] Updated weights for policy 1, policy_version 1425628 (0.0010) [2023-12-27 15:32:03,075][20381] Updated weights for policy 1, policy_version 1425638 (0.0008) [2023-12-27 15:32:03,124][20381] Updated weights for policy 1, policy_version 1425648 (0.0007) [2023-12-27 15:32:03,540][20347] Updated weights for policy 0, policy_version 1426891 (0.0007) [2023-12-27 15:32:03,588][20347] Updated weights for policy 0, policy_version 1426901 (0.0007) [2023-12-27 15:32:03,631][20347] Updated weights for policy 0, policy_version 1426911 (0.0008) [2023-12-27 15:32:03,780][20381] Updated weights for policy 1, policy_version 1425658 (0.0007) [2023-12-27 15:32:03,832][20381] Updated weights for policy 1, policy_version 1425668 (0.0007) [2023-12-27 15:32:03,892][20381] Updated weights for policy 1, policy_version 1425678 (0.0008) [2023-12-27 15:32:04,331][20347] Updated weights for policy 0, policy_version 1426921 (0.0007) [2023-12-27 15:32:04,388][20347] Updated weights for policy 0, policy_version 1426931 (0.0007) [2023-12-27 15:32:04,446][20347] Updated weights for policy 0, policy_version 1426941 (0.0006) [2023-12-27 15:32:04,498][20347] Updated weights for policy 0, policy_version 1426951 (0.0009) [2023-12-27 15:32:04,585][20381] Updated weights for policy 1, policy_version 1425688 (0.0007) [2023-12-27 15:32:04,632][20381] Updated weights for policy 1, policy_version 1425698 (0.0007) [2023-12-27 15:32:04,680][20381] Updated weights for policy 1, policy_version 1425708 (0.0007) [2023-12-27 15:32:05,152][20347] Updated weights for policy 0, policy_version 1426961 (0.0006) [2023-12-27 15:32:05,200][20347] Updated weights for policy 0, policy_version 1426971 (0.0006) [2023-12-27 15:32:05,255][20347] Updated weights for policy 0, policy_version 1426981 (0.0005) [2023-12-27 15:32:05,417][20381] Updated weights for policy 1, policy_version 1425718 (0.0008) [2023-12-27 15:32:05,480][20381] Updated weights for policy 1, policy_version 1425728 (0.0009) [2023-12-27 15:32:05,537][20381] Updated weights for policy 1, policy_version 1425738 (0.0008) [2023-12-27 15:32:05,825][20347] Updated weights for policy 0, policy_version 1426991 (0.0006) [2023-12-27 15:32:05,890][20347] Updated weights for policy 0, policy_version 1427001 (0.0005) [2023-12-27 15:32:05,950][20347] Updated weights for policy 0, policy_version 1427011 (0.0006) [2023-12-27 15:32:06,281][20381] Updated weights for policy 1, policy_version 1425748 (0.0009) [2023-12-27 15:32:06,339][20381] Updated weights for policy 1, policy_version 1425758 (0.0008) [2023-12-27 15:32:06,396][20381] Updated weights for policy 1, policy_version 1425768 (0.0008) [2023-12-27 15:32:06,619][20347] Updated weights for policy 0, policy_version 1427021 (0.0008) [2023-12-27 15:32:06,678][20347] Updated weights for policy 0, policy_version 1427031 (0.0010) [2023-12-27 15:32:06,733][20347] Updated weights for policy 0, policy_version 1427041 (0.0010) [2023-12-27 15:32:07,124][20381] Updated weights for policy 1, policy_version 1425778 (0.0008) [2023-12-27 15:32:07,179][20381] Updated weights for policy 1, policy_version 1425788 (0.0008) [2023-12-27 15:32:07,233][20381] Updated weights for policy 1, policy_version 1425798 (0.0008) [2023-12-27 15:32:07,286][20381] Updated weights for policy 1, policy_version 1425808 (0.0008) [2023-12-27 15:32:07,361][20347] Updated weights for policy 0, policy_version 1427051 (0.0009) [2023-12-27 15:32:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 730431488. Throughput: 0: 10258.3, 1: 10254.5. Samples: 730399868. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:07,363][19170] Avg episode reward: [(0, '882.770'), (1, '924.410')] [2023-12-27 15:32:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001425808_365035520.pth... [2023-12-27 15:32:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001424592_364724224.pth [2023-12-27 15:32:07,413][20347] Updated weights for policy 0, policy_version 1427061 (0.0005) [2023-12-27 15:32:07,473][20347] Updated weights for policy 0, policy_version 1427071 (0.0005) [2023-12-27 15:32:07,524][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001427080_365404160.pth... [2023-12-27 15:32:07,528][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001425864_365092864.pth [2023-12-27 15:32:07,921][20381] Updated weights for policy 1, policy_version 1425818 (0.0005) [2023-12-27 15:32:07,969][20381] Updated weights for policy 1, policy_version 1425828 (0.0008) [2023-12-27 15:32:08,006][20347] Updated weights for policy 0, policy_version 1427081 (0.0006) [2023-12-27 15:32:08,023][20381] Updated weights for policy 1, policy_version 1425838 (0.0006) [2023-12-27 15:32:08,058][20347] Updated weights for policy 0, policy_version 1427091 (0.0010) [2023-12-27 15:32:08,114][20347] Updated weights for policy 0, policy_version 1427101 (0.0010) [2023-12-27 15:32:08,174][20347] Updated weights for policy 0, policy_version 1427111 (0.0010) [2023-12-27 15:32:08,671][20381] Updated weights for policy 1, policy_version 1425848 (0.0007) [2023-12-27 15:32:08,730][20381] Updated weights for policy 1, policy_version 1425858 (0.0008) [2023-12-27 15:32:08,789][20381] Updated weights for policy 1, policy_version 1425868 (0.0006) [2023-12-27 15:32:08,891][20347] Updated weights for policy 0, policy_version 1427121 (0.0008) [2023-12-27 15:32:08,951][20347] Updated weights for policy 0, policy_version 1427131 (0.0007) [2023-12-27 15:32:09,011][20347] Updated weights for policy 0, policy_version 1427141 (0.0006) [2023-12-27 15:32:09,530][20381] Updated weights for policy 1, policy_version 1425878 (0.0008) [2023-12-27 15:32:09,595][20381] Updated weights for policy 1, policy_version 1425888 (0.0007) [2023-12-27 15:32:09,654][20381] Updated weights for policy 1, policy_version 1425898 (0.0007) [2023-12-27 15:32:09,725][20347] Updated weights for policy 0, policy_version 1427151 (0.0007) [2023-12-27 15:32:09,791][20347] Updated weights for policy 0, policy_version 1427161 (0.0007) [2023-12-27 15:32:09,854][20347] Updated weights for policy 0, policy_version 1427171 (0.0008) [2023-12-27 15:32:10,346][20381] Updated weights for policy 1, policy_version 1425908 (0.0008) [2023-12-27 15:32:10,403][20381] Updated weights for policy 1, policy_version 1425918 (0.0008) [2023-12-27 15:32:10,461][20381] Updated weights for policy 1, policy_version 1425928 (0.0009) [2023-12-27 15:32:10,504][20347] Updated weights for policy 0, policy_version 1427181 (0.0006) [2023-12-27 15:32:10,555][20347] Updated weights for policy 0, policy_version 1427191 (0.0007) [2023-12-27 15:32:10,606][20347] Updated weights for policy 0, policy_version 1427201 (0.0006) [2023-12-27 15:32:11,230][20381] Updated weights for policy 1, policy_version 1425938 (0.0009) [2023-12-27 15:32:11,280][20347] Updated weights for policy 0, policy_version 1427211 (0.0008) [2023-12-27 15:32:11,289][20381] Updated weights for policy 1, policy_version 1425948 (0.0008) [2023-12-27 15:32:11,336][20347] Updated weights for policy 0, policy_version 1427221 (0.0008) [2023-12-27 15:32:11,350][20381] Updated weights for policy 1, policy_version 1425958 (0.0007) [2023-12-27 15:32:11,402][20347] Updated weights for policy 0, policy_version 1427231 (0.0008) [2023-12-27 15:32:11,418][20381] Updated weights for policy 1, policy_version 1425968 (0.0008) [2023-12-27 15:32:12,098][20381] Updated weights for policy 1, policy_version 1425978 (0.0007) [2023-12-27 15:32:12,156][20381] Updated weights for policy 1, policy_version 1425988 (0.0008) [2023-12-27 15:32:12,171][20347] Updated weights for policy 0, policy_version 1427241 (0.0008) [2023-12-27 15:32:12,218][20381] Updated weights for policy 1, policy_version 1425998 (0.0008) [2023-12-27 15:32:12,236][20347] Updated weights for policy 0, policy_version 1427251 (0.0008) [2023-12-27 15:32:12,299][20347] Updated weights for policy 0, policy_version 1427261 (0.0008) [2023-12-27 15:32:12,361][20347] Updated weights for policy 0, policy_version 1427271 (0.0007) [2023-12-27 15:32:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 730529792. Throughput: 0: 10353.6, 1: 10205.0. Samples: 730523520. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:12,363][19170] Avg episode reward: [(0, '882.620'), (1, '981.140')] [2023-12-27 15:32:12,858][20381] Updated weights for policy 1, policy_version 1426008 (0.0010) [2023-12-27 15:32:12,906][20381] Updated weights for policy 1, policy_version 1426018 (0.0009) [2023-12-27 15:32:12,944][20347] Updated weights for policy 0, policy_version 1427281 (0.0007) [2023-12-27 15:32:12,950][20381] Updated weights for policy 1, policy_version 1426028 (0.0006) [2023-12-27 15:32:12,994][20347] Updated weights for policy 0, policy_version 1427291 (0.0007) [2023-12-27 15:32:13,043][20347] Updated weights for policy 0, policy_version 1427301 (0.0008) [2023-12-27 15:32:13,577][20381] Updated weights for policy 1, policy_version 1426038 (0.0005) [2023-12-27 15:32:13,631][20381] Updated weights for policy 1, policy_version 1426048 (0.0006) [2023-12-27 15:32:13,681][20381] Updated weights for policy 1, policy_version 1426058 (0.0005) [2023-12-27 15:32:13,795][20347] Updated weights for policy 0, policy_version 1427311 (0.0008) [2023-12-27 15:32:13,852][20347] Updated weights for policy 0, policy_version 1427321 (0.0008) [2023-12-27 15:32:13,908][20347] Updated weights for policy 0, policy_version 1427331 (0.0008) [2023-12-27 15:32:14,220][20381] Updated weights for policy 1, policy_version 1426068 (0.0007) [2023-12-27 15:32:14,278][20381] Updated weights for policy 1, policy_version 1426078 (0.0006) [2023-12-27 15:32:14,333][20381] Updated weights for policy 1, policy_version 1426088 (0.0005) [2023-12-27 15:32:14,647][20347] Updated weights for policy 0, policy_version 1427341 (0.0008) [2023-12-27 15:32:14,699][20347] Updated weights for policy 0, policy_version 1427351 (0.0009) [2023-12-27 15:32:14,749][20347] Updated weights for policy 0, policy_version 1427361 (0.0008) [2023-12-27 15:32:14,902][20381] Updated weights for policy 1, policy_version 1426098 (0.0005) [2023-12-27 15:32:14,961][20381] Updated weights for policy 1, policy_version 1426108 (0.0008) [2023-12-27 15:32:15,026][20381] Updated weights for policy 1, policy_version 1426118 (0.0009) [2023-12-27 15:32:15,086][20381] Updated weights for policy 1, policy_version 1426128 (0.0008) [2023-12-27 15:32:15,381][20347] Updated weights for policy 0, policy_version 1427371 (0.0007) [2023-12-27 15:32:15,440][20347] Updated weights for policy 0, policy_version 1427381 (0.0007) [2023-12-27 15:32:15,502][20347] Updated weights for policy 0, policy_version 1427391 (0.0006) [2023-12-27 15:32:15,830][20381] Updated weights for policy 1, policy_version 1426138 (0.0009) [2023-12-27 15:32:15,880][20381] Updated weights for policy 1, policy_version 1426148 (0.0008) [2023-12-27 15:32:15,929][20381] Updated weights for policy 1, policy_version 1426158 (0.0007) [2023-12-27 15:32:16,100][20347] Updated weights for policy 0, policy_version 1427401 (0.0005) [2023-12-27 15:32:16,150][20347] Updated weights for policy 0, policy_version 1427411 (0.0007) [2023-12-27 15:32:16,203][20347] Updated weights for policy 0, policy_version 1427421 (0.0010) [2023-12-27 15:32:16,262][20347] Updated weights for policy 0, policy_version 1427431 (0.0008) [2023-12-27 15:32:16,641][20381] Updated weights for policy 1, policy_version 1426168 (0.0009) [2023-12-27 15:32:16,694][20381] Updated weights for policy 1, policy_version 1426178 (0.0008) [2023-12-27 15:32:16,745][20381] Updated weights for policy 1, policy_version 1426188 (0.0006) [2023-12-27 15:32:16,897][20347] Updated weights for policy 0, policy_version 1427441 (0.0005) [2023-12-27 15:32:16,949][20347] Updated weights for policy 0, policy_version 1427451 (0.0007) [2023-12-27 15:32:17,003][20347] Updated weights for policy 0, policy_version 1427461 (0.0008) [2023-12-27 15:32:17,343][20381] Updated weights for policy 1, policy_version 1426198 (0.0008) [2023-12-27 15:32:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 730636288. Throughput: 0: 10363.7, 1: 10249.2. Samples: 730648764. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:17,363][19170] Avg episode reward: [(0, '892.840'), (1, '962.140')] [2023-12-27 15:32:17,396][20381] Updated weights for policy 1, policy_version 1426208 (0.0008) [2023-12-27 15:32:17,455][20381] Updated weights for policy 1, policy_version 1426218 (0.0008) [2023-12-27 15:32:17,682][20347] Updated weights for policy 0, policy_version 1427471 (0.0006) [2023-12-27 15:32:17,724][20291] KL-divergence is very high: 193.1012 [2023-12-27 15:32:17,735][20347] Updated weights for policy 0, policy_version 1427481 (0.0008) [2023-12-27 15:32:17,745][20291] KL-divergence is very high: 506.4717 [2023-12-27 15:32:17,755][20291] KL-divergence is very high: 545.8552 [2023-12-27 15:32:17,765][20291] KL-divergence is very high: 472.9861 [2023-12-27 15:32:17,786][20291] KL-divergence is very high: 289.2477 [2023-12-27 15:32:17,786][20347] Updated weights for policy 0, policy_version 1427491 (0.0007) [2023-12-27 15:32:17,795][20291] KL-divergence is very high: 183.3823 [2023-12-27 15:32:18,168][20381] Updated weights for policy 1, policy_version 1426228 (0.0007) [2023-12-27 15:32:18,220][20381] Updated weights for policy 1, policy_version 1426238 (0.0007) [2023-12-27 15:32:18,272][20381] Updated weights for policy 1, policy_version 1426248 (0.0008) [2023-12-27 15:32:18,499][20347] Updated weights for policy 0, policy_version 1427501 (0.0008) [2023-12-27 15:32:18,515][20291] KL-divergence is very high: 116.5263 [2023-12-27 15:32:18,544][20347] Updated weights for policy 0, policy_version 1427511 (0.0007) [2023-12-27 15:32:18,596][20347] Updated weights for policy 0, policy_version 1427521 (0.0007) [2023-12-27 15:32:18,979][20381] Updated weights for policy 1, policy_version 1426258 (0.0007) [2023-12-27 15:32:19,040][20381] Updated weights for policy 1, policy_version 1426268 (0.0007) [2023-12-27 15:32:19,095][20381] Updated weights for policy 1, policy_version 1426278 (0.0007) [2023-12-27 15:32:19,145][20381] Updated weights for policy 1, policy_version 1426288 (0.0007) [2023-12-27 15:32:19,316][20347] Updated weights for policy 0, policy_version 1427531 (0.0009) [2023-12-27 15:32:19,327][20291] KL-divergence is very high: 229.0868 [2023-12-27 15:32:19,385][20347] Updated weights for policy 0, policy_version 1427541 (0.0010) [2023-12-27 15:32:19,385][20291] KL-divergence is very high: 142.7221 [2023-12-27 15:32:19,442][20347] Updated weights for policy 0, policy_version 1427551 (0.0007) [2023-12-27 15:32:19,893][20381] Updated weights for policy 1, policy_version 1426298 (0.0008) [2023-12-27 15:32:19,954][20381] Updated weights for policy 1, policy_version 1426308 (0.0008) [2023-12-27 15:32:20,012][20381] Updated weights for policy 1, policy_version 1426318 (0.0008) [2023-12-27 15:32:20,117][20347] Updated weights for policy 0, policy_version 1427561 (0.0010) [2023-12-27 15:32:20,185][20347] Updated weights for policy 0, policy_version 1427571 (0.0008) [2023-12-27 15:32:20,252][20347] Updated weights for policy 0, policy_version 1427581 (0.0008) [2023-12-27 15:32:20,317][20347] Updated weights for policy 0, policy_version 1427591 (0.0007) [2023-12-27 15:32:20,673][20381] Updated weights for policy 1, policy_version 1426328 (0.0007) [2023-12-27 15:32:20,733][20381] Updated weights for policy 1, policy_version 1426338 (0.0007) [2023-12-27 15:32:20,788][20381] Updated weights for policy 1, policy_version 1426348 (0.0005) [2023-12-27 15:32:21,033][20347] Updated weights for policy 0, policy_version 1427601 (0.0010) [2023-12-27 15:32:21,093][20347] Updated weights for policy 0, policy_version 1427611 (0.0010) [2023-12-27 15:32:21,158][20347] Updated weights for policy 0, policy_version 1427621 (0.0010) [2023-12-27 15:32:21,478][20381] Updated weights for policy 1, policy_version 1426358 (0.0006) [2023-12-27 15:32:21,534][20381] Updated weights for policy 1, policy_version 1426368 (0.0006) [2023-12-27 15:32:21,595][20381] Updated weights for policy 1, policy_version 1426378 (0.0006) [2023-12-27 15:32:21,984][20347] Updated weights for policy 0, policy_version 1427631 (0.0010) [2023-12-27 15:32:22,014][20291] KL-divergence is very high: 102.9197 [2023-12-27 15:32:22,021][20291] KL-divergence is very high: 103.9503 [2023-12-27 15:32:22,043][20347] Updated weights for policy 0, policy_version 1427641 (0.0010) [2023-12-27 15:32:22,105][20347] Updated weights for policy 0, policy_version 1427651 (0.0011) [2023-12-27 15:32:22,316][20381] Updated weights for policy 1, policy_version 1426388 (0.0008) [2023-12-27 15:32:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 730734592. Throughput: 0: 10271.0, 1: 10237.2. Samples: 730708416. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:22,363][19170] Avg episode reward: [(0, '843.870'), (1, '962.000')] [2023-12-27 15:32:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001427656_365551616.pth... [2023-12-27 15:32:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001426472_365248512.pth [2023-12-27 15:32:22,378][20381] Updated weights for policy 1, policy_version 1426398 (0.0008) [2023-12-27 15:32:22,443][20381] Updated weights for policy 1, policy_version 1426408 (0.0007) [2023-12-27 15:32:22,487][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001426416_365191168.pth... [2023-12-27 15:32:22,490][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001425200_364879872.pth [2023-12-27 15:32:22,832][20347] Updated weights for policy 0, policy_version 1427661 (0.0010) [2023-12-27 15:32:22,890][20347] Updated weights for policy 0, policy_version 1427671 (0.0010) [2023-12-27 15:32:22,947][20347] Updated weights for policy 0, policy_version 1427681 (0.0010) [2023-12-27 15:32:23,132][20381] Updated weights for policy 1, policy_version 1426418 (0.0008) [2023-12-27 15:32:23,193][20381] Updated weights for policy 1, policy_version 1426428 (0.0010) [2023-12-27 15:32:23,245][20381] Updated weights for policy 1, policy_version 1426438 (0.0010) [2023-12-27 15:32:23,301][20381] Updated weights for policy 1, policy_version 1426448 (0.0011) [2023-12-27 15:32:23,557][20347] Updated weights for policy 0, policy_version 1427691 (0.0011) [2023-12-27 15:32:23,617][20347] Updated weights for policy 0, policy_version 1427701 (0.0010) [2023-12-27 15:32:23,675][20347] Updated weights for policy 0, policy_version 1427711 (0.0010) [2023-12-27 15:32:23,902][20381] Updated weights for policy 1, policy_version 1426458 (0.0010) [2023-12-27 15:32:23,956][20381] Updated weights for policy 1, policy_version 1426468 (0.0010) [2023-12-27 15:32:24,019][20381] Updated weights for policy 1, policy_version 1426478 (0.0011) [2023-12-27 15:32:24,344][20347] Updated weights for policy 0, policy_version 1427721 (0.0010) [2023-12-27 15:32:24,406][20347] Updated weights for policy 0, policy_version 1427731 (0.0006) [2023-12-27 15:32:24,470][20347] Updated weights for policy 0, policy_version 1427741 (0.0010) [2023-12-27 15:32:24,527][20347] Updated weights for policy 0, policy_version 1427751 (0.0011) [2023-12-27 15:32:24,646][20381] Updated weights for policy 1, policy_version 1426488 (0.0010) [2023-12-27 15:32:24,703][20381] Updated weights for policy 1, policy_version 1426498 (0.0009) [2023-12-27 15:32:24,765][20381] Updated weights for policy 1, policy_version 1426508 (0.0005) [2023-12-27 15:32:25,160][20347] Updated weights for policy 0, policy_version 1427761 (0.0008) [2023-12-27 15:32:25,208][20347] Updated weights for policy 0, policy_version 1427771 (0.0010) [2023-12-27 15:32:25,263][20347] Updated weights for policy 0, policy_version 1427781 (0.0010) [2023-12-27 15:32:25,409][20381] Updated weights for policy 1, policy_version 1426518 (0.0006) [2023-12-27 15:32:25,465][20381] Updated weights for policy 1, policy_version 1426528 (0.0006) [2023-12-27 15:32:25,521][20381] Updated weights for policy 1, policy_version 1426538 (0.0006) [2023-12-27 15:32:25,943][20347] Updated weights for policy 0, policy_version 1427791 (0.0010) [2023-12-27 15:32:26,010][20347] Updated weights for policy 0, policy_version 1427801 (0.0010) [2023-12-27 15:32:26,078][20347] Updated weights for policy 0, policy_version 1427811 (0.0010) [2023-12-27 15:32:26,115][20381] Updated weights for policy 1, policy_version 1426548 (0.0006) [2023-12-27 15:32:26,172][20381] Updated weights for policy 1, policy_version 1426558 (0.0006) [2023-12-27 15:32:26,222][20381] Updated weights for policy 1, policy_version 1426568 (0.0005) [2023-12-27 15:32:26,732][20347] Updated weights for policy 0, policy_version 1427821 (0.0010) [2023-12-27 15:32:26,779][20347] Updated weights for policy 0, policy_version 1427831 (0.0010) [2023-12-27 15:32:26,801][20381] Updated weights for policy 1, policy_version 1426578 (0.0007) [2023-12-27 15:32:26,825][20347] Updated weights for policy 0, policy_version 1427841 (0.0008) [2023-12-27 15:32:26,849][20381] Updated weights for policy 1, policy_version 1426588 (0.0005) [2023-12-27 15:32:26,899][20381] Updated weights for policy 1, policy_version 1426598 (0.0007) [2023-12-27 15:32:26,953][20381] Updated weights for policy 1, policy_version 1426608 (0.0008) [2023-12-27 15:32:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 730841088. Throughput: 0: 10168.3, 1: 10273.1. Samples: 730831200. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:27,363][19170] Avg episode reward: [(0, '902.240'), (1, '951.740')] [2023-12-27 15:32:27,410][20347] Updated weights for policy 0, policy_version 1427851 (0.0009) [2023-12-27 15:32:27,460][20347] Updated weights for policy 0, policy_version 1427861 (0.0005) [2023-12-27 15:32:27,506][20347] Updated weights for policy 0, policy_version 1427871 (0.0007) [2023-12-27 15:32:27,689][20381] Updated weights for policy 1, policy_version 1426618 (0.0008) [2023-12-27 15:32:27,745][20381] Updated weights for policy 1, policy_version 1426628 (0.0007) [2023-12-27 15:32:27,796][20381] Updated weights for policy 1, policy_version 1426638 (0.0006) [2023-12-27 15:32:28,146][20347] Updated weights for policy 0, policy_version 1427881 (0.0008) [2023-12-27 15:32:28,206][20347] Updated weights for policy 0, policy_version 1427891 (0.0008) [2023-12-27 15:32:28,266][20347] Updated weights for policy 0, policy_version 1427901 (0.0008) [2023-12-27 15:32:28,329][20347] Updated weights for policy 0, policy_version 1427911 (0.0008) [2023-12-27 15:32:28,416][20381] Updated weights for policy 1, policy_version 1426648 (0.0007) [2023-12-27 15:32:28,478][20381] Updated weights for policy 1, policy_version 1426658 (0.0008) [2023-12-27 15:32:28,533][20381] Updated weights for policy 1, policy_version 1426668 (0.0007) [2023-12-27 15:32:28,967][20347] Updated weights for policy 0, policy_version 1427921 (0.0010) [2023-12-27 15:32:29,014][20347] Updated weights for policy 0, policy_version 1427931 (0.0010) [2023-12-27 15:32:29,063][20347] Updated weights for policy 0, policy_version 1427941 (0.0010) [2023-12-27 15:32:29,147][20381] Updated weights for policy 1, policy_version 1426678 (0.0007) [2023-12-27 15:32:29,205][20381] Updated weights for policy 1, policy_version 1426688 (0.0009) [2023-12-27 15:32:29,265][20381] Updated weights for policy 1, policy_version 1426698 (0.0007) [2023-12-27 15:32:29,774][20347] Updated weights for policy 0, policy_version 1427951 (0.0008) [2023-12-27 15:32:29,834][20347] Updated weights for policy 0, policy_version 1427961 (0.0008) [2023-12-27 15:32:29,900][20347] Updated weights for policy 0, policy_version 1427971 (0.0008) [2023-12-27 15:32:29,930][20381] Updated weights for policy 1, policy_version 1426708 (0.0008) [2023-12-27 15:32:29,988][20381] Updated weights for policy 1, policy_version 1426718 (0.0008) [2023-12-27 15:32:30,042][20381] Updated weights for policy 1, policy_version 1426728 (0.0007) [2023-12-27 15:32:30,609][20347] Updated weights for policy 0, policy_version 1427981 (0.0008) [2023-12-27 15:32:30,664][20347] Updated weights for policy 0, policy_version 1427991 (0.0007) [2023-12-27 15:32:30,730][20347] Updated weights for policy 0, policy_version 1428001 (0.0005) [2023-12-27 15:32:30,746][20381] Updated weights for policy 1, policy_version 1426738 (0.0006) [2023-12-27 15:32:30,790][20381] Updated weights for policy 1, policy_version 1426748 (0.0005) [2023-12-27 15:32:30,840][20381] Updated weights for policy 1, policy_version 1426758 (0.0005) [2023-12-27 15:32:30,888][20381] Updated weights for policy 1, policy_version 1426768 (0.0007) [2023-12-27 15:32:31,434][20347] Updated weights for policy 0, policy_version 1428011 (0.0007) [2023-12-27 15:32:31,493][20347] Updated weights for policy 0, policy_version 1428021 (0.0005) [2023-12-27 15:32:31,551][20347] Updated weights for policy 0, policy_version 1428031 (0.0007) [2023-12-27 15:32:31,608][20381] Updated weights for policy 1, policy_version 1426778 (0.0007) [2023-12-27 15:32:31,673][20381] Updated weights for policy 1, policy_version 1426788 (0.0007) [2023-12-27 15:32:31,723][20381] Updated weights for policy 1, policy_version 1426798 (0.0008) [2023-12-27 15:32:32,215][20347] Updated weights for policy 0, policy_version 1428041 (0.0007) [2023-12-27 15:32:32,278][20347] Updated weights for policy 0, policy_version 1428051 (0.0007) [2023-12-27 15:32:32,340][20347] Updated weights for policy 0, policy_version 1428061 (0.0007) [2023-12-27 15:32:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 730939392. Throughput: 0: 10207.4, 1: 10266.8. Samples: 730955820. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:32,363][19170] Avg episode reward: [(0, '951.290'), (1, '941.750')] [2023-12-27 15:32:32,405][20347] Updated weights for policy 0, policy_version 1428071 (0.0007) [2023-12-27 15:32:32,451][20381] Updated weights for policy 1, policy_version 1426808 (0.0009) [2023-12-27 15:32:32,516][20381] Updated weights for policy 1, policy_version 1426818 (0.0008) [2023-12-27 15:32:32,568][20381] Updated weights for policy 1, policy_version 1426828 (0.0008) [2023-12-27 15:32:33,076][20347] Updated weights for policy 0, policy_version 1428081 (0.0007) [2023-12-27 15:32:33,129][20347] Updated weights for policy 0, policy_version 1428091 (0.0007) [2023-12-27 15:32:33,181][20347] Updated weights for policy 0, policy_version 1428101 (0.0006) [2023-12-27 15:32:33,227][20381] Updated weights for policy 1, policy_version 1426838 (0.0008) [2023-12-27 15:32:33,291][20381] Updated weights for policy 1, policy_version 1426848 (0.0008) [2023-12-27 15:32:33,352][20381] Updated weights for policy 1, policy_version 1426858 (0.0008) [2023-12-27 15:32:33,883][20347] Updated weights for policy 0, policy_version 1428111 (0.0007) [2023-12-27 15:32:33,932][20347] Updated weights for policy 0, policy_version 1428121 (0.0008) [2023-12-27 15:32:33,937][20381] Updated weights for policy 1, policy_version 1426868 (0.0005) [2023-12-27 15:32:33,986][20347] Updated weights for policy 0, policy_version 1428131 (0.0007) [2023-12-27 15:32:33,993][20381] Updated weights for policy 1, policy_version 1426878 (0.0005) [2023-12-27 15:32:34,046][20381] Updated weights for policy 1, policy_version 1426888 (0.0008) [2023-12-27 15:32:34,668][20381] Updated weights for policy 1, policy_version 1426898 (0.0009) [2023-12-27 15:32:34,706][20292] KL-divergence is very high: 448.0594 [2023-12-27 15:32:34,720][20347] Updated weights for policy 0, policy_version 1428141 (0.0006) [2023-12-27 15:32:34,726][20381] Updated weights for policy 1, policy_version 1426908 (0.0006) [2023-12-27 15:32:34,749][20292] KL-divergence is very high: 595.0837 [2023-12-27 15:32:34,775][20347] Updated weights for policy 0, policy_version 1428151 (0.0008) [2023-12-27 15:32:34,784][20381] Updated weights for policy 1, policy_version 1426918 (0.0006) [2023-12-27 15:32:34,798][20292] KL-divergence is very high: 408.0188 [2023-12-27 15:32:34,830][20347] Updated weights for policy 0, policy_version 1428161 (0.0007) [2023-12-27 15:32:34,845][20381] Updated weights for policy 1, policy_version 1426928 (0.0007) [2023-12-27 15:32:35,464][20347] Updated weights for policy 0, policy_version 1428171 (0.0007) [2023-12-27 15:32:35,499][20381] Updated weights for policy 1, policy_version 1426938 (0.0006) [2023-12-27 15:32:35,512][20347] Updated weights for policy 0, policy_version 1428181 (0.0005) [2023-12-27 15:32:35,542][20381] Updated weights for policy 1, policy_version 1426948 (0.0005) [2023-12-27 15:32:35,560][20347] Updated weights for policy 0, policy_version 1428191 (0.0007) [2023-12-27 15:32:35,590][20381] Updated weights for policy 1, policy_version 1426958 (0.0005) [2023-12-27 15:32:36,229][20347] Updated weights for policy 0, policy_version 1428201 (0.0007) [2023-12-27 15:32:36,246][20381] Updated weights for policy 1, policy_version 1426968 (0.0006) [2023-12-27 15:32:36,287][20347] Updated weights for policy 0, policy_version 1428211 (0.0005) [2023-12-27 15:32:36,307][20381] Updated weights for policy 1, policy_version 1426978 (0.0006) [2023-12-27 15:32:36,343][20347] Updated weights for policy 0, policy_version 1428221 (0.0005) [2023-12-27 15:32:36,375][20381] Updated weights for policy 1, policy_version 1426988 (0.0006) [2023-12-27 15:32:36,394][20347] Updated weights for policy 0, policy_version 1428231 (0.0005) [2023-12-27 15:32:37,039][20347] Updated weights for policy 0, policy_version 1428241 (0.0006) [2023-12-27 15:32:37,091][20347] Updated weights for policy 0, policy_version 1428251 (0.0007) [2023-12-27 15:32:37,102][20381] Updated weights for policy 1, policy_version 1426998 (0.0008) [2023-12-27 15:32:37,140][20347] Updated weights for policy 0, policy_version 1428261 (0.0007) [2023-12-27 15:32:37,156][20381] Updated weights for policy 1, policy_version 1427008 (0.0007) [2023-12-27 15:32:37,207][20381] Updated weights for policy 1, policy_version 1427018 (0.0008) [2023-12-27 15:32:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 731054080. Throughput: 0: 10138.5, 1: 10316.9. Samples: 731017652. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:37,363][19170] Avg episode reward: [(0, '960.930'), (1, '922.740')] [2023-12-27 15:32:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001427024_365346816.pth... [2023-12-27 15:32:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001428264_365707264.pth... [2023-12-27 15:32:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001427080_365404160.pth [2023-12-27 15:32:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001425808_365035520.pth [2023-12-27 15:32:37,775][20347] Updated weights for policy 0, policy_version 1428271 (0.0006) [2023-12-27 15:32:37,825][20347] Updated weights for policy 0, policy_version 1428281 (0.0006) [2023-12-27 15:32:37,861][20381] Updated weights for policy 1, policy_version 1427028 (0.0008) [2023-12-27 15:32:37,878][20347] Updated weights for policy 0, policy_version 1428291 (0.0007) [2023-12-27 15:32:37,915][20381] Updated weights for policy 1, policy_version 1427038 (0.0006) [2023-12-27 15:32:37,964][20381] Updated weights for policy 1, policy_version 1427048 (0.0008) [2023-12-27 15:32:38,482][20347] Updated weights for policy 0, policy_version 1428301 (0.0007) [2023-12-27 15:32:38,542][20347] Updated weights for policy 0, policy_version 1428311 (0.0006) [2023-12-27 15:32:38,604][20347] Updated weights for policy 0, policy_version 1428321 (0.0008) [2023-12-27 15:32:38,668][20381] Updated weights for policy 1, policy_version 1427058 (0.0006) [2023-12-27 15:32:38,725][20381] Updated weights for policy 1, policy_version 1427068 (0.0007) [2023-12-27 15:32:38,783][20381] Updated weights for policy 1, policy_version 1427078 (0.0008) [2023-12-27 15:32:38,839][20381] Updated weights for policy 1, policy_version 1427088 (0.0007) [2023-12-27 15:32:39,250][20347] Updated weights for policy 0, policy_version 1428331 (0.0010) [2023-12-27 15:32:39,317][20347] Updated weights for policy 0, policy_version 1428341 (0.0007) [2023-12-27 15:32:39,380][20347] Updated weights for policy 0, policy_version 1428351 (0.0007) [2023-12-27 15:32:39,544][20381] Updated weights for policy 1, policy_version 1427098 (0.0008) [2023-12-27 15:32:39,608][20381] Updated weights for policy 1, policy_version 1427108 (0.0008) [2023-12-27 15:32:39,674][20381] Updated weights for policy 1, policy_version 1427118 (0.0007) [2023-12-27 15:32:40,114][20347] Updated weights for policy 0, policy_version 1428361 (0.0006) [2023-12-27 15:32:40,175][20347] Updated weights for policy 0, policy_version 1428371 (0.0008) [2023-12-27 15:32:40,238][20347] Updated weights for policy 0, policy_version 1428381 (0.0008) [2023-12-27 15:32:40,299][20347] Updated weights for policy 0, policy_version 1428391 (0.0008) [2023-12-27 15:32:40,404][20381] Updated weights for policy 1, policy_version 1427128 (0.0007) [2023-12-27 15:32:40,464][20381] Updated weights for policy 1, policy_version 1427138 (0.0008) [2023-12-27 15:32:40,511][20381] Updated weights for policy 1, policy_version 1427148 (0.0010) [2023-12-27 15:32:40,890][20347] Updated weights for policy 0, policy_version 1428401 (0.0006) [2023-12-27 15:32:40,940][20347] Updated weights for policy 0, policy_version 1428411 (0.0006) [2023-12-27 15:32:40,990][20347] Updated weights for policy 0, policy_version 1428421 (0.0009) [2023-12-27 15:32:41,240][20381] Updated weights for policy 1, policy_version 1427158 (0.0008) [2023-12-27 15:32:41,303][20381] Updated weights for policy 1, policy_version 1427168 (0.0009) [2023-12-27 15:32:41,358][20381] Updated weights for policy 1, policy_version 1427178 (0.0008) [2023-12-27 15:32:41,701][20347] Updated weights for policy 0, policy_version 1428431 (0.0007) [2023-12-27 15:32:41,769][20347] Updated weights for policy 0, policy_version 1428441 (0.0008) [2023-12-27 15:32:41,833][20347] Updated weights for policy 0, policy_version 1428451 (0.0008) [2023-12-27 15:32:42,124][20381] Updated weights for policy 1, policy_version 1427188 (0.0009) [2023-12-27 15:32:42,184][20381] Updated weights for policy 1, policy_version 1427198 (0.0010) [2023-12-27 15:32:42,242][20381] Updated weights for policy 1, policy_version 1427208 (0.0008) [2023-12-27 15:32:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 731152384. Throughput: 0: 10259.4, 1: 10286.0. Samples: 731141712. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:42,363][19170] Avg episode reward: [(0, '902.470'), (1, '826.740')] [2023-12-27 15:32:42,505][20347] Updated weights for policy 0, policy_version 1428461 (0.0008) [2023-12-27 15:32:42,571][20347] Updated weights for policy 0, policy_version 1428471 (0.0008) [2023-12-27 15:32:42,633][20347] Updated weights for policy 0, policy_version 1428481 (0.0008) [2023-12-27 15:32:42,975][20381] Updated weights for policy 1, policy_version 1427218 (0.0009) [2023-12-27 15:32:43,033][20381] Updated weights for policy 1, policy_version 1427228 (0.0007) [2023-12-27 15:32:43,094][20381] Updated weights for policy 1, policy_version 1427238 (0.0007) [2023-12-27 15:32:43,149][20381] Updated weights for policy 1, policy_version 1427248 (0.0007) [2023-12-27 15:32:43,303][20347] Updated weights for policy 0, policy_version 1428491 (0.0005) [2023-12-27 15:32:43,361][20347] Updated weights for policy 0, policy_version 1428501 (0.0006) [2023-12-27 15:32:43,416][20347] Updated weights for policy 0, policy_version 1428511 (0.0008) [2023-12-27 15:32:43,751][20381] Updated weights for policy 1, policy_version 1427258 (0.0008) [2023-12-27 15:32:43,802][20381] Updated weights for policy 1, policy_version 1427268 (0.0005) [2023-12-27 15:32:43,866][20381] Updated weights for policy 1, policy_version 1427278 (0.0005) [2023-12-27 15:32:44,095][20347] Updated weights for policy 0, policy_version 1428521 (0.0007) [2023-12-27 15:32:44,148][20347] Updated weights for policy 0, policy_version 1428531 (0.0006) [2023-12-27 15:32:44,198][20347] Updated weights for policy 0, policy_version 1428541 (0.0008) [2023-12-27 15:32:44,256][20347] Updated weights for policy 0, policy_version 1428551 (0.0009) [2023-12-27 15:32:44,437][20381] Updated weights for policy 1, policy_version 1427288 (0.0006) [2023-12-27 15:32:44,498][20381] Updated weights for policy 1, policy_version 1427298 (0.0005) [2023-12-27 15:32:44,558][20381] Updated weights for policy 1, policy_version 1427308 (0.0006) [2023-12-27 15:32:44,995][20347] Updated weights for policy 0, policy_version 1428561 (0.0008) [2023-12-27 15:32:45,053][20347] Updated weights for policy 0, policy_version 1428571 (0.0008) [2023-12-27 15:32:45,112][20347] Updated weights for policy 0, policy_version 1428581 (0.0008) [2023-12-27 15:32:45,215][20381] Updated weights for policy 1, policy_version 1427318 (0.0007) [2023-12-27 15:32:45,273][20381] Updated weights for policy 1, policy_version 1427328 (0.0008) [2023-12-27 15:32:45,334][20381] Updated weights for policy 1, policy_version 1427338 (0.0008) [2023-12-27 15:32:45,757][20347] Updated weights for policy 0, policy_version 1428591 (0.0006) [2023-12-27 15:32:45,810][20347] Updated weights for policy 0, policy_version 1428601 (0.0006) [2023-12-27 15:32:45,859][20347] Updated weights for policy 0, policy_version 1428611 (0.0010) [2023-12-27 15:32:45,991][20381] Updated weights for policy 1, policy_version 1427348 (0.0008) [2023-12-27 15:32:46,052][20381] Updated weights for policy 1, policy_version 1427358 (0.0007) [2023-12-27 15:32:46,114][20381] Updated weights for policy 1, policy_version 1427368 (0.0008) [2023-12-27 15:32:46,489][20347] Updated weights for policy 0, policy_version 1428621 (0.0008) [2023-12-27 15:32:46,547][20347] Updated weights for policy 0, policy_version 1428631 (0.0010) [2023-12-27 15:32:46,595][20347] Updated weights for policy 0, policy_version 1428641 (0.0010) [2023-12-27 15:32:46,831][20381] Updated weights for policy 1, policy_version 1427378 (0.0008) [2023-12-27 15:32:46,900][20381] Updated weights for policy 1, policy_version 1427388 (0.0008) [2023-12-27 15:32:46,944][20381] Updated weights for policy 1, policy_version 1427398 (0.0007) [2023-12-27 15:32:46,967][20292] KL-divergence is very high: 120.0495 [2023-12-27 15:32:46,992][20381] Updated weights for policy 1, policy_version 1427408 (0.0007) [2023-12-27 15:32:47,266][20347] Updated weights for policy 0, policy_version 1428651 (0.0010) [2023-12-27 15:32:47,326][20347] Updated weights for policy 0, policy_version 1428661 (0.0010) [2023-12-27 15:32:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 731250688. Throughput: 0: 10288.8, 1: 10261.7. Samples: 731264204. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:47,363][19170] Avg episode reward: [(0, '921.930'), (1, '847.320')] [2023-12-27 15:32:47,384][20347] Updated weights for policy 0, policy_version 1428671 (0.0010) [2023-12-27 15:32:47,649][20381] Updated weights for policy 1, policy_version 1427418 (0.0006) [2023-12-27 15:32:47,698][20381] Updated weights for policy 1, policy_version 1427428 (0.0008) [2023-12-27 15:32:47,761][20381] Updated weights for policy 1, policy_version 1427438 (0.0008) [2023-12-27 15:32:48,064][20347] Updated weights for policy 0, policy_version 1428681 (0.0010) [2023-12-27 15:32:48,125][20347] Updated weights for policy 0, policy_version 1428691 (0.0010) [2023-12-27 15:32:48,187][20347] Updated weights for policy 0, policy_version 1428701 (0.0008) [2023-12-27 15:32:48,249][20347] Updated weights for policy 0, policy_version 1428711 (0.0009) [2023-12-27 15:32:48,411][20381] Updated weights for policy 1, policy_version 1427449 (0.0008) [2023-12-27 15:32:48,463][20381] Updated weights for policy 1, policy_version 1427459 (0.0008) [2023-12-27 15:32:48,518][20381] Updated weights for policy 1, policy_version 1427469 (0.0011) [2023-12-27 15:32:48,918][20347] Updated weights for policy 0, policy_version 1428721 (0.0008) [2023-12-27 15:32:48,981][20347] Updated weights for policy 0, policy_version 1428731 (0.0007) [2023-12-27 15:32:49,037][20347] Updated weights for policy 0, policy_version 1428741 (0.0005) [2023-12-27 15:32:49,158][20381] Updated weights for policy 1, policy_version 1427479 (0.0007) [2023-12-27 15:32:49,240][20381] Updated weights for policy 1, policy_version 1427489 (0.0007) [2023-12-27 15:32:49,304][20381] Updated weights for policy 1, policy_version 1427499 (0.0008) [2023-12-27 15:32:49,651][20347] Updated weights for policy 0, policy_version 1428751 (0.0007) [2023-12-27 15:32:49,711][20347] Updated weights for policy 0, policy_version 1428761 (0.0008) [2023-12-27 15:32:49,771][20347] Updated weights for policy 0, policy_version 1428771 (0.0009) [2023-12-27 15:32:49,940][20381] Updated weights for policy 1, policy_version 1427509 (0.0008) [2023-12-27 15:32:50,000][20381] Updated weights for policy 1, policy_version 1427519 (0.0008) [2023-12-27 15:32:50,056][20381] Updated weights for policy 1, policy_version 1427529 (0.0008) [2023-12-27 15:32:50,503][20347] Updated weights for policy 0, policy_version 1428781 (0.0008) [2023-12-27 15:32:50,562][20347] Updated weights for policy 0, policy_version 1428791 (0.0007) [2023-12-27 15:32:50,627][20347] Updated weights for policy 0, policy_version 1428801 (0.0007) [2023-12-27 15:32:50,761][20381] Updated weights for policy 1, policy_version 1427539 (0.0008) [2023-12-27 15:32:50,812][20381] Updated weights for policy 1, policy_version 1427549 (0.0006) [2023-12-27 15:32:50,868][20381] Updated weights for policy 1, policy_version 1427559 (0.0005) [2023-12-27 15:32:51,330][20347] Updated weights for policy 0, policy_version 1428811 (0.0008) [2023-12-27 15:32:51,400][20347] Updated weights for policy 0, policy_version 1428821 (0.0008) [2023-12-27 15:32:51,461][20347] Updated weights for policy 0, policy_version 1428831 (0.0006) [2023-12-27 15:32:51,544][20381] Updated weights for policy 1, policy_version 1427569 (0.0005) [2023-12-27 15:32:51,603][20381] Updated weights for policy 1, policy_version 1427579 (0.0007) [2023-12-27 15:32:51,670][20381] Updated weights for policy 1, policy_version 1427589 (0.0008) [2023-12-27 15:32:51,734][20381] Updated weights for policy 1, policy_version 1427599 (0.0009) [2023-12-27 15:32:52,204][20347] Updated weights for policy 0, policy_version 1428841 (0.0007) [2023-12-27 15:32:52,253][20347] Updated weights for policy 0, policy_version 1428851 (0.0010) [2023-12-27 15:32:52,318][20347] Updated weights for policy 0, policy_version 1428861 (0.0007) [2023-12-27 15:32:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 731348992. Throughput: 0: 10284.6, 1: 10296.2. Samples: 731326004. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:52,363][19170] Avg episode reward: [(0, '951.230'), (1, '789.790')] [2023-12-27 15:32:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001427600_365494272.pth... [2023-12-27 15:32:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001426416_365191168.pth [2023-12-27 15:32:52,381][20347] Updated weights for policy 0, policy_version 1428871 (0.0009) [2023-12-27 15:32:52,386][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001428872_365862912.pth... [2023-12-27 15:32:52,390][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001427656_365551616.pth [2023-12-27 15:32:52,433][20381] Updated weights for policy 1, policy_version 1427609 (0.0007) [2023-12-27 15:32:52,485][20381] Updated weights for policy 1, policy_version 1427619 (0.0008) [2023-12-27 15:32:52,534][20381] Updated weights for policy 1, policy_version 1427629 (0.0007) [2023-12-27 15:32:53,093][20347] Updated weights for policy 0, policy_version 1428881 (0.0008) [2023-12-27 15:32:53,152][20347] Updated weights for policy 0, policy_version 1428891 (0.0008) [2023-12-27 15:32:53,212][20347] Updated weights for policy 0, policy_version 1428901 (0.0006) [2023-12-27 15:32:53,221][20381] Updated weights for policy 1, policy_version 1427639 (0.0006) [2023-12-27 15:32:53,272][20381] Updated weights for policy 1, policy_version 1427649 (0.0007) [2023-12-27 15:32:53,331][20381] Updated weights for policy 1, policy_version 1427659 (0.0008) [2023-12-27 15:32:53,806][20347] Updated weights for policy 0, policy_version 1428911 (0.0005) [2023-12-27 15:32:53,857][20347] Updated weights for policy 0, policy_version 1428921 (0.0005) [2023-12-27 15:32:53,911][20347] Updated weights for policy 0, policy_version 1428931 (0.0006) [2023-12-27 15:32:54,077][20381] Updated weights for policy 1, policy_version 1427669 (0.0008) [2023-12-27 15:32:54,144][20381] Updated weights for policy 1, policy_version 1427679 (0.0009) [2023-12-27 15:32:54,201][20381] Updated weights for policy 1, policy_version 1427689 (0.0007) [2023-12-27 15:32:54,525][20347] Updated weights for policy 0, policy_version 1428941 (0.0008) [2023-12-27 15:32:54,584][20347] Updated weights for policy 0, policy_version 1428951 (0.0007) [2023-12-27 15:32:54,640][20347] Updated weights for policy 0, policy_version 1428961 (0.0008) [2023-12-27 15:32:54,922][20381] Updated weights for policy 1, policy_version 1427699 (0.0008) [2023-12-27 15:32:54,981][20381] Updated weights for policy 1, policy_version 1427709 (0.0008) [2023-12-27 15:32:55,043][20381] Updated weights for policy 1, policy_version 1427719 (0.0007) [2023-12-27 15:32:55,290][20347] Updated weights for policy 0, policy_version 1428971 (0.0007) [2023-12-27 15:32:55,338][20347] Updated weights for policy 0, policy_version 1428981 (0.0008) [2023-12-27 15:32:55,384][20347] Updated weights for policy 0, policy_version 1428991 (0.0008) [2023-12-27 15:32:55,758][20381] Updated weights for policy 1, policy_version 1427729 (0.0008) [2023-12-27 15:32:55,809][20381] Updated weights for policy 1, policy_version 1427739 (0.0006) [2023-12-27 15:32:55,854][20381] Updated weights for policy 1, policy_version 1427749 (0.0009) [2023-12-27 15:32:55,900][20381] Updated weights for policy 1, policy_version 1427759 (0.0009) [2023-12-27 15:32:56,006][20347] Updated weights for policy 0, policy_version 1429001 (0.0008) [2023-12-27 15:32:56,067][20347] Updated weights for policy 0, policy_version 1429011 (0.0006) [2023-12-27 15:32:56,128][20347] Updated weights for policy 0, policy_version 1429021 (0.0006) [2023-12-27 15:32:56,190][20347] Updated weights for policy 0, policy_version 1429031 (0.0007) [2023-12-27 15:32:56,585][20381] Updated weights for policy 1, policy_version 1427769 (0.0007) [2023-12-27 15:32:56,627][20381] Updated weights for policy 1, policy_version 1427779 (0.0007) [2023-12-27 15:32:56,672][20381] Updated weights for policy 1, policy_version 1427789 (0.0007) [2023-12-27 15:32:56,817][20347] Updated weights for policy 0, policy_version 1429041 (0.0007) [2023-12-27 15:32:56,875][20347] Updated weights for policy 0, policy_version 1429051 (0.0008) [2023-12-27 15:32:56,935][20347] Updated weights for policy 0, policy_version 1429061 (0.0008) [2023-12-27 15:32:57,313][20381] Updated weights for policy 1, policy_version 1427799 (0.0007) [2023-12-27 15:32:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 731455488. Throughput: 0: 10226.8, 1: 10310.6. Samples: 731447700. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:32:57,363][20381] Updated weights for policy 1, policy_version 1427809 (0.0005) [2023-12-27 15:32:57,363][19170] Avg episode reward: [(0, '970.800'), (1, '807.430')] [2023-12-27 15:32:57,412][20381] Updated weights for policy 1, policy_version 1427819 (0.0005) [2023-12-27 15:32:57,592][20347] Updated weights for policy 0, policy_version 1429071 (0.0006) [2023-12-27 15:32:57,644][20347] Updated weights for policy 0, policy_version 1429081 (0.0005) [2023-12-27 15:32:57,699][20347] Updated weights for policy 0, policy_version 1429091 (0.0005) [2023-12-27 15:32:58,052][20381] Updated weights for policy 1, policy_version 1427829 (0.0008) [2023-12-27 15:32:58,100][20381] Updated weights for policy 1, policy_version 1427839 (0.0009) [2023-12-27 15:32:58,145][20381] Updated weights for policy 1, policy_version 1427849 (0.0008) [2023-12-27 15:32:58,230][20347] Updated weights for policy 0, policy_version 1429101 (0.0005) [2023-12-27 15:32:58,294][20347] Updated weights for policy 0, policy_version 1429111 (0.0007) [2023-12-27 15:32:58,363][20347] Updated weights for policy 0, policy_version 1429121 (0.0009) [2023-12-27 15:32:58,949][20381] Updated weights for policy 1, policy_version 1427859 (0.0008) [2023-12-27 15:32:59,016][20381] Updated weights for policy 1, policy_version 1427869 (0.0007) [2023-12-27 15:32:59,081][20381] Updated weights for policy 1, policy_version 1427879 (0.0010) [2023-12-27 15:32:59,109][20347] Updated weights for policy 0, policy_version 1429131 (0.0008) [2023-12-27 15:32:59,172][20347] Updated weights for policy 0, policy_version 1429141 (0.0007) [2023-12-27 15:32:59,245][20347] Updated weights for policy 0, policy_version 1429151 (0.0016) [2023-12-27 15:32:59,793][20381] Updated weights for policy 1, policy_version 1427889 (0.0009) [2023-12-27 15:32:59,849][20347] Updated weights for policy 0, policy_version 1429161 (0.0008) [2023-12-27 15:32:59,853][20381] Updated weights for policy 1, policy_version 1427899 (0.0010) [2023-12-27 15:32:59,908][20347] Updated weights for policy 0, policy_version 1429171 (0.0006) [2023-12-27 15:32:59,910][20381] Updated weights for policy 1, policy_version 1427909 (0.0011) [2023-12-27 15:32:59,964][20347] Updated weights for policy 0, policy_version 1429181 (0.0007) [2023-12-27 15:32:59,970][20381] Updated weights for policy 1, policy_version 1427919 (0.0009) [2023-12-27 15:33:00,012][20347] Updated weights for policy 0, policy_version 1429191 (0.0008) [2023-12-27 15:33:00,635][20347] Updated weights for policy 0, policy_version 1429201 (0.0006) [2023-12-27 15:33:00,648][20381] Updated weights for policy 1, policy_version 1427929 (0.0006) [2023-12-27 15:33:00,695][20347] Updated weights for policy 0, policy_version 1429211 (0.0006) [2023-12-27 15:33:00,705][20381] Updated weights for policy 1, policy_version 1427939 (0.0008) [2023-12-27 15:33:00,751][20347] Updated weights for policy 0, policy_version 1429221 (0.0005) [2023-12-27 15:33:00,757][20381] Updated weights for policy 1, policy_version 1427949 (0.0010) [2023-12-27 15:33:01,399][20347] Updated weights for policy 0, policy_version 1429231 (0.0007) [2023-12-27 15:33:01,443][20381] Updated weights for policy 1, policy_version 1427959 (0.0007) [2023-12-27 15:33:01,452][20347] Updated weights for policy 0, policy_version 1429241 (0.0006) [2023-12-27 15:33:01,496][20381] Updated weights for policy 1, policy_version 1427969 (0.0006) [2023-12-27 15:33:01,498][20347] Updated weights for policy 0, policy_version 1429251 (0.0006) [2023-12-27 15:33:01,546][20381] Updated weights for policy 1, policy_version 1427979 (0.0006) [2023-12-27 15:33:02,132][20347] Updated weights for policy 0, policy_version 1429261 (0.0006) [2023-12-27 15:33:02,179][20347] Updated weights for policy 0, policy_version 1429271 (0.0005) [2023-12-27 15:33:02,232][20347] Updated weights for policy 0, policy_version 1429281 (0.0006) [2023-12-27 15:33:02,280][20381] Updated weights for policy 1, policy_version 1427989 (0.0008) [2023-12-27 15:33:02,338][20381] Updated weights for policy 1, policy_version 1427999 (0.0006) [2023-12-27 15:33:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 731561984. Throughput: 0: 10299.9, 1: 10216.5. Samples: 731572004. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:02,363][19170] Avg episode reward: [(0, '970.730'), (1, '787.480')] [2023-12-27 15:33:02,405][20381] Updated weights for policy 1, policy_version 1428009 (0.0007) [2023-12-27 15:33:02,914][20347] Updated weights for policy 0, policy_version 1429291 (0.0007) [2023-12-27 15:33:02,969][20347] Updated weights for policy 0, policy_version 1429301 (0.0008) [2023-12-27 15:33:03,023][20347] Updated weights for policy 0, policy_version 1429311 (0.0007) [2023-12-27 15:33:03,053][20381] Updated weights for policy 1, policy_version 1428019 (0.0007) [2023-12-27 15:33:03,111][20381] Updated weights for policy 1, policy_version 1428029 (0.0008) [2023-12-27 15:33:03,170][20381] Updated weights for policy 1, policy_version 1428039 (0.0007) [2023-12-27 15:33:03,690][20347] Updated weights for policy 0, policy_version 1429321 (0.0007) [2023-12-27 15:33:03,747][20347] Updated weights for policy 0, policy_version 1429331 (0.0010) [2023-12-27 15:33:03,806][20381] Updated weights for policy 1, policy_version 1428049 (0.0008) [2023-12-27 15:33:03,810][20347] Updated weights for policy 0, policy_version 1429341 (0.0010) [2023-12-27 15:33:03,871][20381] Updated weights for policy 1, policy_version 1428059 (0.0008) [2023-12-27 15:33:03,876][20347] Updated weights for policy 0, policy_version 1429352 (0.0009) [2023-12-27 15:33:03,929][20381] Updated weights for policy 1, policy_version 1428069 (0.0007) [2023-12-27 15:33:03,986][20381] Updated weights for policy 1, policy_version 1428079 (0.0009) [2023-12-27 15:33:04,529][20347] Updated weights for policy 0, policy_version 1429362 (0.0007) [2023-12-27 15:33:04,583][20347] Updated weights for policy 0, policy_version 1429372 (0.0006) [2023-12-27 15:33:04,637][20347] Updated weights for policy 0, policy_version 1429382 (0.0007) [2023-12-27 15:33:04,699][20381] Updated weights for policy 1, policy_version 1428089 (0.0007) [2023-12-27 15:33:04,747][20381] Updated weights for policy 1, policy_version 1428099 (0.0007) [2023-12-27 15:33:04,803][20381] Updated weights for policy 1, policy_version 1428109 (0.0005) [2023-12-27 15:33:05,330][20347] Updated weights for policy 0, policy_version 1429392 (0.0006) [2023-12-27 15:33:05,380][20347] Updated weights for policy 0, policy_version 1429402 (0.0005) [2023-12-27 15:33:05,386][20381] Updated weights for policy 1, policy_version 1428119 (0.0007) [2023-12-27 15:33:05,433][20347] Updated weights for policy 0, policy_version 1429412 (0.0006) [2023-12-27 15:33:05,443][20381] Updated weights for policy 1, policy_version 1428129 (0.0006) [2023-12-27 15:33:05,494][20381] Updated weights for policy 1, policy_version 1428139 (0.0008) [2023-12-27 15:33:05,991][20347] Updated weights for policy 0, policy_version 1429422 (0.0009) [2023-12-27 15:33:06,046][20347] Updated weights for policy 0, policy_version 1429432 (0.0006) [2023-12-27 15:33:06,108][20347] Updated weights for policy 0, policy_version 1429442 (0.0009) [2023-12-27 15:33:06,256][20381] Updated weights for policy 1, policy_version 1428149 (0.0009) [2023-12-27 15:33:06,317][20381] Updated weights for policy 1, policy_version 1428159 (0.0007) [2023-12-27 15:33:06,370][20381] Updated weights for policy 1, policy_version 1428169 (0.0008) [2023-12-27 15:33:06,796][20347] Updated weights for policy 0, policy_version 1429452 (0.0008) [2023-12-27 15:33:06,857][20347] Updated weights for policy 0, policy_version 1429462 (0.0007) [2023-12-27 15:33:06,917][20347] Updated weights for policy 0, policy_version 1429472 (0.0007) [2023-12-27 15:33:07,059][20381] Updated weights for policy 1, policy_version 1428179 (0.0009) [2023-12-27 15:33:07,109][20381] Updated weights for policy 1, policy_version 1428189 (0.0010) [2023-12-27 15:33:07,164][20381] Updated weights for policy 1, policy_version 1428199 (0.0006) [2023-12-27 15:33:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 731668480. Throughput: 0: 10315.3, 1: 10253.8. Samples: 731634024. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:07,363][19170] Avg episode reward: [(0, '941.510'), (1, '786.970')] [2023-12-27 15:33:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001429480_366018560.pth... [2023-12-27 15:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001428208_365649920.pth... [2023-12-27 15:33:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001428264_365707264.pth [2023-12-27 15:33:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001427024_365346816.pth [2023-12-27 15:33:07,563][20347] Updated weights for policy 0, policy_version 1429482 (0.0008) [2023-12-27 15:33:07,617][20347] Updated weights for policy 0, policy_version 1429492 (0.0010) [2023-12-27 15:33:07,675][20347] Updated weights for policy 0, policy_version 1429502 (0.0005) [2023-12-27 15:33:07,702][20381] Updated weights for policy 1, policy_version 1428209 (0.0006) [2023-12-27 15:33:07,734][20347] Updated weights for policy 0, policy_version 1429512 (0.0005) [2023-12-27 15:33:07,763][20381] Updated weights for policy 1, policy_version 1428219 (0.0007) [2023-12-27 15:33:07,818][20381] Updated weights for policy 1, policy_version 1428229 (0.0010) [2023-12-27 15:33:07,866][20381] Updated weights for policy 1, policy_version 1428239 (0.0010) [2023-12-27 15:33:08,314][20347] Updated weights for policy 0, policy_version 1429522 (0.0010) [2023-12-27 15:33:08,378][20347] Updated weights for policy 0, policy_version 1429532 (0.0009) [2023-12-27 15:33:08,442][20347] Updated weights for policy 0, policy_version 1429542 (0.0010) [2023-12-27 15:33:08,543][20381] Updated weights for policy 1, policy_version 1428249 (0.0009) [2023-12-27 15:33:08,599][20381] Updated weights for policy 1, policy_version 1428259 (0.0010) [2023-12-27 15:33:08,658][20381] Updated weights for policy 1, policy_version 1428269 (0.0010) [2023-12-27 15:33:09,154][20347] Updated weights for policy 0, policy_version 1429552 (0.0010) [2023-12-27 15:33:09,214][20347] Updated weights for policy 0, policy_version 1429562 (0.0010) [2023-12-27 15:33:09,274][20347] Updated weights for policy 0, policy_version 1429572 (0.0008) [2023-12-27 15:33:09,324][20381] Updated weights for policy 1, policy_version 1428279 (0.0011) [2023-12-27 15:33:09,386][20381] Updated weights for policy 1, policy_version 1428289 (0.0010) [2023-12-27 15:33:09,456][20381] Updated weights for policy 1, policy_version 1428299 (0.0011) [2023-12-27 15:33:10,048][20347] Updated weights for policy 0, policy_version 1429582 (0.0007) [2023-12-27 15:33:10,109][20347] Updated weights for policy 0, policy_version 1429592 (0.0007) [2023-12-27 15:33:10,165][20347] Updated weights for policy 0, policy_version 1429602 (0.0006) [2023-12-27 15:33:10,174][20381] Updated weights for policy 1, policy_version 1428309 (0.0007) [2023-12-27 15:33:10,235][20381] Updated weights for policy 1, policy_version 1428319 (0.0006) [2023-12-27 15:33:10,301][20381] Updated weights for policy 1, policy_version 1428329 (0.0006) [2023-12-27 15:33:10,885][20347] Updated weights for policy 0, policy_version 1429612 (0.0008) [2023-12-27 15:33:10,924][20381] Updated weights for policy 1, policy_version 1428339 (0.0006) [2023-12-27 15:33:10,940][20347] Updated weights for policy 0, policy_version 1429622 (0.0007) [2023-12-27 15:33:10,977][20381] Updated weights for policy 1, policy_version 1428349 (0.0006) [2023-12-27 15:33:10,999][20347] Updated weights for policy 0, policy_version 1429632 (0.0008) [2023-12-27 15:33:11,028][20381] Updated weights for policy 1, policy_version 1428359 (0.0007) [2023-12-27 15:33:11,767][20347] Updated weights for policy 0, policy_version 1429642 (0.0007) [2023-12-27 15:33:11,822][20347] Updated weights for policy 0, policy_version 1429652 (0.0008) [2023-12-27 15:33:11,854][20381] Updated weights for policy 1, policy_version 1428369 (0.0009) [2023-12-27 15:33:11,882][20347] Updated weights for policy 0, policy_version 1429662 (0.0008) [2023-12-27 15:33:11,911][20381] Updated weights for policy 1, policy_version 1428379 (0.0007) [2023-12-27 15:33:11,940][20347] Updated weights for policy 0, policy_version 1429672 (0.0007) [2023-12-27 15:33:11,971][20381] Updated weights for policy 1, policy_version 1428389 (0.0007) [2023-12-27 15:33:12,027][20381] Updated weights for policy 1, policy_version 1428399 (0.0008) [2023-12-27 15:33:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 731766784. Throughput: 0: 10366.4, 1: 10213.4. Samples: 731757292. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:12,363][19170] Avg episode reward: [(0, '902.590'), (1, '796.090')] [2023-12-27 15:33:12,687][20347] Updated weights for policy 0, policy_version 1429682 (0.0007) [2023-12-27 15:33:12,708][20381] Updated weights for policy 1, policy_version 1428409 (0.0006) [2023-12-27 15:33:12,749][20347] Updated weights for policy 0, policy_version 1429692 (0.0007) [2023-12-27 15:33:12,770][20381] Updated weights for policy 1, policy_version 1428419 (0.0006) [2023-12-27 15:33:12,805][20347] Updated weights for policy 0, policy_version 1429702 (0.0006) [2023-12-27 15:33:12,831][20381] Updated weights for policy 1, policy_version 1428429 (0.0006) [2023-12-27 15:33:13,487][20347] Updated weights for policy 0, policy_version 1429712 (0.0007) [2023-12-27 15:33:13,509][20381] Updated weights for policy 1, policy_version 1428439 (0.0008) [2023-12-27 15:33:13,542][20347] Updated weights for policy 0, policy_version 1429722 (0.0005) [2023-12-27 15:33:13,562][20381] Updated weights for policy 1, policy_version 1428449 (0.0006) [2023-12-27 15:33:13,596][20347] Updated weights for policy 0, policy_version 1429732 (0.0005) [2023-12-27 15:33:13,613][20381] Updated weights for policy 1, policy_version 1428459 (0.0008) [2023-12-27 15:33:14,147][20347] Updated weights for policy 0, policy_version 1429742 (0.0005) [2023-12-27 15:33:14,205][20347] Updated weights for policy 0, policy_version 1429752 (0.0006) [2023-12-27 15:33:14,262][20347] Updated weights for policy 0, policy_version 1429762 (0.0008) [2023-12-27 15:33:14,363][20381] Updated weights for policy 1, policy_version 1428469 (0.0008) [2023-12-27 15:33:14,419][20381] Updated weights for policy 1, policy_version 1428479 (0.0007) [2023-12-27 15:33:14,477][20381] Updated weights for policy 1, policy_version 1428489 (0.0008) [2023-12-27 15:33:14,921][20347] Updated weights for policy 0, policy_version 1429772 (0.0008) [2023-12-27 15:33:14,975][20347] Updated weights for policy 0, policy_version 1429782 (0.0008) [2023-12-27 15:33:15,036][20347] Updated weights for policy 0, policy_version 1429792 (0.0008) [2023-12-27 15:33:15,193][20381] Updated weights for policy 1, policy_version 1428499 (0.0008) [2023-12-27 15:33:15,244][20381] Updated weights for policy 1, policy_version 1428509 (0.0008) [2023-12-27 15:33:15,298][20381] Updated weights for policy 1, policy_version 1428519 (0.0008) [2023-12-27 15:33:15,700][20347] Updated weights for policy 0, policy_version 1429802 (0.0008) [2023-12-27 15:33:15,752][20347] Updated weights for policy 0, policy_version 1429812 (0.0006) [2023-12-27 15:33:15,799][20347] Updated weights for policy 0, policy_version 1429822 (0.0006) [2023-12-27 15:33:15,852][20347] Updated weights for policy 0, policy_version 1429832 (0.0006) [2023-12-27 15:33:15,975][20381] Updated weights for policy 1, policy_version 1428529 (0.0008) [2023-12-27 15:33:16,034][20381] Updated weights for policy 1, policy_version 1428539 (0.0006) [2023-12-27 15:33:16,087][20381] Updated weights for policy 1, policy_version 1428549 (0.0007) [2023-12-27 15:33:16,139][20381] Updated weights for policy 1, policy_version 1428559 (0.0008) [2023-12-27 15:33:16,519][20347] Updated weights for policy 0, policy_version 1429842 (0.0008) [2023-12-27 15:33:16,575][20347] Updated weights for policy 0, policy_version 1429852 (0.0006) [2023-12-27 15:33:16,625][20347] Updated weights for policy 0, policy_version 1429862 (0.0008) [2023-12-27 15:33:16,769][20381] Updated weights for policy 1, policy_version 1428569 (0.0005) [2023-12-27 15:33:16,818][20381] Updated weights for policy 1, policy_version 1428579 (0.0005) [2023-12-27 15:33:16,873][20381] Updated weights for policy 1, policy_version 1428589 (0.0009) [2023-12-27 15:33:17,218][20347] Updated weights for policy 0, policy_version 1429872 (0.0010) [2023-12-27 15:33:17,271][20347] Updated weights for policy 0, policy_version 1429882 (0.0010) [2023-12-27 15:33:17,328][20347] Updated weights for policy 0, policy_version 1429892 (0.0010) [2023-12-27 15:33:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 731873280. Throughput: 0: 10356.8, 1: 10189.8. Samples: 731880416. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:17,363][19170] Avg episode reward: [(0, '922.070'), (1, '786.420')] [2023-12-27 15:33:17,501][20381] Updated weights for policy 1, policy_version 1428599 (0.0008) [2023-12-27 15:33:17,546][20381] Updated weights for policy 1, policy_version 1428609 (0.0010) [2023-12-27 15:33:17,595][20381] Updated weights for policy 1, policy_version 1428619 (0.0010) [2023-12-27 15:33:17,993][20347] Updated weights for policy 0, policy_version 1429902 (0.0008) [2023-12-27 15:33:18,051][20347] Updated weights for policy 0, policy_version 1429912 (0.0008) [2023-12-27 15:33:18,109][20347] Updated weights for policy 0, policy_version 1429922 (0.0007) [2023-12-27 15:33:18,209][20381] Updated weights for policy 1, policy_version 1428629 (0.0010) [2023-12-27 15:33:18,263][20381] Updated weights for policy 1, policy_version 1428639 (0.0010) [2023-12-27 15:33:18,322][20381] Updated weights for policy 1, policy_version 1428649 (0.0008) [2023-12-27 15:33:18,727][20347] Updated weights for policy 0, policy_version 1429932 (0.0007) [2023-12-27 15:33:18,785][20347] Updated weights for policy 0, policy_version 1429942 (0.0007) [2023-12-27 15:33:18,835][20347] Updated weights for policy 0, policy_version 1429952 (0.0008) [2023-12-27 15:33:18,994][20381] Updated weights for policy 1, policy_version 1428659 (0.0009) [2023-12-27 15:33:19,048][20381] Updated weights for policy 1, policy_version 1428669 (0.0007) [2023-12-27 15:33:19,095][20381] Updated weights for policy 1, policy_version 1428679 (0.0008) [2023-12-27 15:33:19,485][20347] Updated weights for policy 0, policy_version 1429962 (0.0008) [2023-12-27 15:33:19,547][20347] Updated weights for policy 0, policy_version 1429972 (0.0011) [2023-12-27 15:33:19,606][20347] Updated weights for policy 0, policy_version 1429982 (0.0010) [2023-12-27 15:33:19,665][20347] Updated weights for policy 0, policy_version 1429992 (0.0010) [2023-12-27 15:33:19,827][20381] Updated weights for policy 1, policy_version 1428689 (0.0008) [2023-12-27 15:33:19,893][20381] Updated weights for policy 1, policy_version 1428699 (0.0009) [2023-12-27 15:33:19,964][20381] Updated weights for policy 1, policy_version 1428709 (0.0008) [2023-12-27 15:33:20,025][20381] Updated weights for policy 1, policy_version 1428719 (0.0009) [2023-12-27 15:33:20,351][20347] Updated weights for policy 0, policy_version 1430002 (0.0010) [2023-12-27 15:33:20,416][20347] Updated weights for policy 0, policy_version 1430012 (0.0010) [2023-12-27 15:33:20,432][20291] KL-divergence is very high: 169.2688 [2023-12-27 15:33:20,453][20291] KL-divergence is very high: 130.3552 [2023-12-27 15:33:20,481][20347] Updated weights for policy 0, policy_version 1430022 (0.0010) [2023-12-27 15:33:20,770][20381] Updated weights for policy 1, policy_version 1428729 (0.0011) [2023-12-27 15:33:20,827][20381] Updated weights for policy 1, policy_version 1428739 (0.0011) [2023-12-27 15:33:20,886][20381] Updated weights for policy 1, policy_version 1428749 (0.0011) [2023-12-27 15:33:21,190][20347] Updated weights for policy 0, policy_version 1430032 (0.0010) [2023-12-27 15:33:21,253][20347] Updated weights for policy 0, policy_version 1430042 (0.0008) [2023-12-27 15:33:21,317][20347] Updated weights for policy 0, policy_version 1430052 (0.0008) [2023-12-27 15:33:21,601][20381] Updated weights for policy 1, policy_version 1428759 (0.0010) [2023-12-27 15:33:21,667][20381] Updated weights for policy 1, policy_version 1428769 (0.0007) [2023-12-27 15:33:21,733][20381] Updated weights for policy 1, policy_version 1428779 (0.0007) [2023-12-27 15:33:22,039][20347] Updated weights for policy 0, policy_version 1430062 (0.0009) [2023-12-27 15:33:22,095][20347] Updated weights for policy 0, policy_version 1430072 (0.0006) [2023-12-27 15:33:22,149][20347] Updated weights for policy 0, policy_version 1430082 (0.0008) [2023-12-27 15:33:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 731971584. Throughput: 0: 10405.1, 1: 10138.0. Samples: 731942092. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:22,363][19170] Avg episode reward: [(0, '951.220'), (1, '759.580')] [2023-12-27 15:33:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001428784_365797376.pth... [2023-12-27 15:33:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001430088_366174208.pth... [2023-12-27 15:33:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001428872_365862912.pth [2023-12-27 15:33:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001427600_365494272.pth [2023-12-27 15:33:22,451][20381] Updated weights for policy 1, policy_version 1428789 (0.0007) [2023-12-27 15:33:22,516][20381] Updated weights for policy 1, policy_version 1428799 (0.0006) [2023-12-27 15:33:22,574][20381] Updated weights for policy 1, policy_version 1428809 (0.0006) [2023-12-27 15:33:22,882][20347] Updated weights for policy 0, policy_version 1430092 (0.0008) [2023-12-27 15:33:22,935][20347] Updated weights for policy 0, policy_version 1430102 (0.0008) [2023-12-27 15:33:22,985][20347] Updated weights for policy 0, policy_version 1430112 (0.0008) [2023-12-27 15:33:23,197][20381] Updated weights for policy 1, policy_version 1428819 (0.0006) [2023-12-27 15:33:23,252][20381] Updated weights for policy 1, policy_version 1428829 (0.0005) [2023-12-27 15:33:23,311][20381] Updated weights for policy 1, policy_version 1428839 (0.0007) [2023-12-27 15:33:23,725][20347] Updated weights for policy 0, policy_version 1430122 (0.0007) [2023-12-27 15:33:23,781][20347] Updated weights for policy 0, policy_version 1430132 (0.0008) [2023-12-27 15:33:23,829][20347] Updated weights for policy 0, policy_version 1430142 (0.0008) [2023-12-27 15:33:23,880][20347] Updated weights for policy 0, policy_version 1430152 (0.0008) [2023-12-27 15:33:23,896][20381] Updated weights for policy 1, policy_version 1428849 (0.0006) [2023-12-27 15:33:23,954][20381] Updated weights for policy 1, policy_version 1428859 (0.0007) [2023-12-27 15:33:24,006][20381] Updated weights for policy 1, policy_version 1428869 (0.0005) [2023-12-27 15:33:24,054][20381] Updated weights for policy 1, policy_version 1428879 (0.0005) [2023-12-27 15:33:24,586][20347] Updated weights for policy 0, policy_version 1430162 (0.0008) [2023-12-27 15:33:24,638][20347] Updated weights for policy 0, policy_version 1430172 (0.0005) [2023-12-27 15:33:24,691][20347] Updated weights for policy 0, policy_version 1430182 (0.0006) [2023-12-27 15:33:24,715][20381] Updated weights for policy 1, policy_version 1428889 (0.0008) [2023-12-27 15:33:24,765][20381] Updated weights for policy 1, policy_version 1428899 (0.0008) [2023-12-27 15:33:24,825][20381] Updated weights for policy 1, policy_version 1428909 (0.0008) [2023-12-27 15:33:25,348][20347] Updated weights for policy 0, policy_version 1430192 (0.0007) [2023-12-27 15:33:25,405][20347] Updated weights for policy 0, policy_version 1430202 (0.0008) [2023-12-27 15:33:25,460][20347] Updated weights for policy 0, policy_version 1430212 (0.0007) [2023-12-27 15:33:25,550][20381] Updated weights for policy 1, policy_version 1428919 (0.0008) [2023-12-27 15:33:25,605][20381] Updated weights for policy 1, policy_version 1428929 (0.0008) [2023-12-27 15:33:25,655][20381] Updated weights for policy 1, policy_version 1428939 (0.0009) [2023-12-27 15:33:26,108][20347] Updated weights for policy 0, policy_version 1430222 (0.0007) [2023-12-27 15:33:26,167][20347] Updated weights for policy 0, policy_version 1430232 (0.0007) [2023-12-27 15:33:26,225][20347] Updated weights for policy 0, policy_version 1430242 (0.0007) [2023-12-27 15:33:26,357][20381] Updated weights for policy 1, policy_version 1428949 (0.0008) [2023-12-27 15:33:26,411][20381] Updated weights for policy 1, policy_version 1428959 (0.0005) [2023-12-27 15:33:26,465][20381] Updated weights for policy 1, policy_version 1428969 (0.0008) [2023-12-27 15:33:26,885][20347] Updated weights for policy 0, policy_version 1430252 (0.0007) [2023-12-27 15:33:26,931][20347] Updated weights for policy 0, policy_version 1430262 (0.0008) [2023-12-27 15:33:26,982][20347] Updated weights for policy 0, policy_version 1430272 (0.0009) [2023-12-27 15:33:27,119][20381] Updated weights for policy 1, policy_version 1428979 (0.0008) [2023-12-27 15:33:27,172][20381] Updated weights for policy 1, policy_version 1428989 (0.0008) [2023-12-27 15:33:27,224][20381] Updated weights for policy 1, policy_version 1428999 (0.0007) [2023-12-27 15:33:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 732078080. Throughput: 0: 10315.8, 1: 10153.4. Samples: 732062828. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:27,363][19170] Avg episode reward: [(0, '952.020'), (1, '787.840')] [2023-12-27 15:33:27,708][20347] Updated weights for policy 0, policy_version 1430282 (0.0007) [2023-12-27 15:33:27,766][20347] Updated weights for policy 0, policy_version 1430292 (0.0008) [2023-12-27 15:33:27,817][20347] Updated weights for policy 0, policy_version 1430302 (0.0007) [2023-12-27 15:33:27,861][20347] Updated weights for policy 0, policy_version 1430312 (0.0007) [2023-12-27 15:33:27,881][20381] Updated weights for policy 1, policy_version 1429009 (0.0008) [2023-12-27 15:33:27,935][20381] Updated weights for policy 1, policy_version 1429019 (0.0010) [2023-12-27 15:33:27,990][20381] Updated weights for policy 1, policy_version 1429029 (0.0009) [2023-12-27 15:33:28,056][20381] Updated weights for policy 1, policy_version 1429039 (0.0007) [2023-12-27 15:33:28,519][20347] Updated weights for policy 0, policy_version 1430322 (0.0009) [2023-12-27 15:33:28,572][20347] Updated weights for policy 0, policy_version 1430332 (0.0011) [2023-12-27 15:33:28,627][20347] Updated weights for policy 0, policy_version 1430342 (0.0007) [2023-12-27 15:33:28,645][20381] Updated weights for policy 1, policy_version 1429049 (0.0006) [2023-12-27 15:33:28,703][20381] Updated weights for policy 1, policy_version 1429059 (0.0006) [2023-12-27 15:33:28,760][20381] Updated weights for policy 1, policy_version 1429069 (0.0006) [2023-12-27 15:33:29,260][20347] Updated weights for policy 0, policy_version 1430352 (0.0007) [2023-12-27 15:33:29,308][20347] Updated weights for policy 0, policy_version 1430362 (0.0007) [2023-12-27 15:33:29,345][20381] Updated weights for policy 1, policy_version 1429079 (0.0006) [2023-12-27 15:33:29,367][20347] Updated weights for policy 0, policy_version 1430372 (0.0011) [2023-12-27 15:33:29,410][20381] Updated weights for policy 1, policy_version 1429089 (0.0008) [2023-12-27 15:33:29,471][20381] Updated weights for policy 1, policy_version 1429099 (0.0008) [2023-12-27 15:33:30,071][20347] Updated weights for policy 0, policy_version 1430382 (0.0008) [2023-12-27 15:33:30,131][20347] Updated weights for policy 0, policy_version 1430392 (0.0007) [2023-12-27 15:33:30,167][20381] Updated weights for policy 1, policy_version 1429109 (0.0008) [2023-12-27 15:33:30,188][20347] Updated weights for policy 0, policy_version 1430402 (0.0005) [2023-12-27 15:33:30,224][20381] Updated weights for policy 1, policy_version 1429119 (0.0010) [2023-12-27 15:33:30,280][20381] Updated weights for policy 1, policy_version 1429129 (0.0010) [2023-12-27 15:33:30,739][20347] Updated weights for policy 0, policy_version 1430412 (0.0007) [2023-12-27 15:33:30,790][20347] Updated weights for policy 0, policy_version 1430422 (0.0010) [2023-12-27 15:33:30,848][20347] Updated weights for policy 0, policy_version 1430432 (0.0010) [2023-12-27 15:33:30,950][20381] Updated weights for policy 1, policy_version 1429139 (0.0010) [2023-12-27 15:33:30,994][20381] Updated weights for policy 1, policy_version 1429149 (0.0010) [2023-12-27 15:33:31,049][20381] Updated weights for policy 1, policy_version 1429159 (0.0010) [2023-12-27 15:33:31,584][20347] Updated weights for policy 0, policy_version 1430442 (0.0010) [2023-12-27 15:33:31,641][20347] Updated weights for policy 0, policy_version 1430452 (0.0007) [2023-12-27 15:33:31,699][20347] Updated weights for policy 0, policy_version 1430462 (0.0007) [2023-12-27 15:33:31,765][20347] Updated weights for policy 0, policy_version 1430472 (0.0008) [2023-12-27 15:33:31,767][20381] Updated weights for policy 1, policy_version 1429169 (0.0010) [2023-12-27 15:33:31,819][20381] Updated weights for policy 1, policy_version 1429179 (0.0007) [2023-12-27 15:33:31,870][20381] Updated weights for policy 1, policy_version 1429189 (0.0007) [2023-12-27 15:33:31,917][20381] Updated weights for policy 1, policy_version 1429199 (0.0008) [2023-12-27 15:33:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 732176384. Throughput: 0: 10344.5, 1: 10182.6. Samples: 732187920. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:32,363][19170] Avg episode reward: [(0, '961.870'), (1, '692.010')] [2023-12-27 15:33:32,433][20347] Updated weights for policy 0, policy_version 1430482 (0.0007) [2023-12-27 15:33:32,482][20347] Updated weights for policy 0, policy_version 1430492 (0.0008) [2023-12-27 15:33:32,530][20347] Updated weights for policy 0, policy_version 1430502 (0.0007) [2023-12-27 15:33:32,653][20381] Updated weights for policy 1, policy_version 1429209 (0.0009) [2023-12-27 15:33:32,712][20381] Updated weights for policy 1, policy_version 1429219 (0.0008) [2023-12-27 15:33:32,774][20381] Updated weights for policy 1, policy_version 1429229 (0.0008) [2023-12-27 15:33:33,257][20347] Updated weights for policy 0, policy_version 1430512 (0.0006) [2023-12-27 15:33:33,305][20347] Updated weights for policy 0, policy_version 1430522 (0.0005) [2023-12-27 15:33:33,355][20347] Updated weights for policy 0, policy_version 1430532 (0.0007) [2023-12-27 15:33:33,424][20381] Updated weights for policy 1, policy_version 1429239 (0.0007) [2023-12-27 15:33:33,469][20381] Updated weights for policy 1, policy_version 1429249 (0.0008) [2023-12-27 15:33:33,517][20381] Updated weights for policy 1, policy_version 1429259 (0.0006) [2023-12-27 15:33:34,044][20347] Updated weights for policy 0, policy_version 1430542 (0.0007) [2023-12-27 15:33:34,093][20347] Updated weights for policy 0, policy_version 1430552 (0.0007) [2023-12-27 15:33:34,106][20381] Updated weights for policy 1, policy_version 1429269 (0.0006) [2023-12-27 15:33:34,158][20347] Updated weights for policy 0, policy_version 1430562 (0.0008) [2023-12-27 15:33:34,170][20381] Updated weights for policy 1, policy_version 1429279 (0.0007) [2023-12-27 15:33:34,227][20381] Updated weights for policy 1, policy_version 1429289 (0.0006) [2023-12-27 15:33:34,881][20347] Updated weights for policy 0, policy_version 1430572 (0.0009) [2023-12-27 15:33:34,906][20381] Updated weights for policy 1, policy_version 1429299 (0.0006) [2023-12-27 15:33:34,939][20347] Updated weights for policy 0, policy_version 1430582 (0.0010) [2023-12-27 15:33:34,955][20381] Updated weights for policy 1, policy_version 1429309 (0.0007) [2023-12-27 15:33:34,994][20347] Updated weights for policy 0, policy_version 1430592 (0.0010) [2023-12-27 15:33:35,013][20381] Updated weights for policy 1, policy_version 1429319 (0.0006) [2023-12-27 15:33:35,591][20347] Updated weights for policy 0, policy_version 1430602 (0.0010) [2023-12-27 15:33:35,636][20381] Updated weights for policy 1, policy_version 1429329 (0.0006) [2023-12-27 15:33:35,649][20347] Updated weights for policy 0, policy_version 1430612 (0.0010) [2023-12-27 15:33:35,684][20381] Updated weights for policy 1, policy_version 1429339 (0.0011) [2023-12-27 15:33:35,703][20347] Updated weights for policy 0, policy_version 1430622 (0.0010) [2023-12-27 15:33:35,739][20381] Updated weights for policy 1, policy_version 1429349 (0.0011) [2023-12-27 15:33:35,759][20347] Updated weights for policy 0, policy_version 1430632 (0.0010) [2023-12-27 15:33:35,795][20381] Updated weights for policy 1, policy_version 1429359 (0.0011) [2023-12-27 15:33:36,482][20347] Updated weights for policy 0, policy_version 1430642 (0.0006) [2023-12-27 15:33:36,488][20381] Updated weights for policy 1, policy_version 1429369 (0.0011) [2023-12-27 15:33:36,540][20347] Updated weights for policy 0, policy_version 1430652 (0.0006) [2023-12-27 15:33:36,544][20381] Updated weights for policy 1, policy_version 1429379 (0.0011) [2023-12-27 15:33:36,596][20381] Updated weights for policy 1, policy_version 1429389 (0.0006) [2023-12-27 15:33:36,597][20347] Updated weights for policy 0, policy_version 1430662 (0.0007) [2023-12-27 15:33:37,239][20347] Updated weights for policy 0, policy_version 1430672 (0.0008) [2023-12-27 15:33:37,293][20347] Updated weights for policy 0, policy_version 1430682 (0.0007) [2023-12-27 15:33:37,331][20381] Updated weights for policy 1, policy_version 1429399 (0.0007) [2023-12-27 15:33:37,343][20347] Updated weights for policy 0, policy_version 1430692 (0.0008) [2023-12-27 15:33:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 732274688. Throughput: 0: 10334.7, 1: 10196.8. Samples: 732249920. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:37,363][19170] Avg episode reward: [(0, '970.770'), (1, '719.520')] [2023-12-27 15:33:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001430696_366329856.pth... [2023-12-27 15:33:37,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001429480_366018560.pth [2023-12-27 15:33:37,381][20381] Updated weights for policy 1, policy_version 1429409 (0.0007) [2023-12-27 15:33:37,440][20381] Updated weights for policy 1, policy_version 1429419 (0.0007) [2023-12-27 15:33:37,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001429424_365961216.pth... [2023-12-27 15:33:37,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001428208_365649920.pth [2023-12-27 15:33:38,044][20381] Updated weights for policy 1, policy_version 1429429 (0.0006) [2023-12-27 15:33:38,089][20347] Updated weights for policy 0, policy_version 1430702 (0.0006) [2023-12-27 15:33:38,095][20381] Updated weights for policy 1, policy_version 1429439 (0.0006) [2023-12-27 15:33:38,134][20347] Updated weights for policy 0, policy_version 1430712 (0.0005) [2023-12-27 15:33:38,144][20381] Updated weights for policy 1, policy_version 1429449 (0.0005) [2023-12-27 15:33:38,186][20347] Updated weights for policy 0, policy_version 1430722 (0.0005) [2023-12-27 15:33:38,835][20347] Updated weights for policy 0, policy_version 1430732 (0.0006) [2023-12-27 15:33:38,891][20347] Updated weights for policy 0, policy_version 1430742 (0.0008) [2023-12-27 15:33:38,909][20381] Updated weights for policy 1, policy_version 1429459 (0.0006) [2023-12-27 15:33:38,950][20347] Updated weights for policy 0, policy_version 1430752 (0.0008) [2023-12-27 15:33:38,966][20381] Updated weights for policy 1, policy_version 1429469 (0.0006) [2023-12-27 15:33:39,023][20381] Updated weights for policy 1, policy_version 1429479 (0.0006) [2023-12-27 15:33:39,688][20347] Updated weights for policy 0, policy_version 1430762 (0.0006) [2023-12-27 15:33:39,745][20347] Updated weights for policy 0, policy_version 1430772 (0.0008) [2023-12-27 15:33:39,770][20381] Updated weights for policy 1, policy_version 1429489 (0.0008) [2023-12-27 15:33:39,806][20347] Updated weights for policy 0, policy_version 1430782 (0.0007) [2023-12-27 15:33:39,816][20292] KL-divergence is very high: 188.3281 [2023-12-27 15:33:39,831][20381] Updated weights for policy 1, policy_version 1429499 (0.0009) [2023-12-27 15:33:39,866][20292] KL-divergence is very high: 185.1371 [2023-12-27 15:33:39,871][20347] Updated weights for policy 0, policy_version 1430792 (0.0007) [2023-12-27 15:33:39,891][20381] Updated weights for policy 1, policy_version 1429509 (0.0008) [2023-12-27 15:33:39,953][20381] Updated weights for policy 1, policy_version 1429519 (0.0008) [2023-12-27 15:33:40,547][20347] Updated weights for policy 0, policy_version 1430802 (0.0010) [2023-12-27 15:33:40,631][20347] Updated weights for policy 0, policy_version 1430812 (0.0010) [2023-12-27 15:33:40,673][20381] Updated weights for policy 1, policy_version 1429529 (0.0006) [2023-12-27 15:33:40,683][20347] Updated weights for policy 0, policy_version 1430822 (0.0010) [2023-12-27 15:33:40,732][20381] Updated weights for policy 1, policy_version 1429539 (0.0006) [2023-12-27 15:33:40,777][20381] Updated weights for policy 1, policy_version 1429549 (0.0008) [2023-12-27 15:33:41,401][20347] Updated weights for policy 0, policy_version 1430832 (0.0010) [2023-12-27 15:33:41,466][20347] Updated weights for policy 0, policy_version 1430842 (0.0007) [2023-12-27 15:33:41,508][20381] Updated weights for policy 1, policy_version 1429559 (0.0006) [2023-12-27 15:33:41,526][20347] Updated weights for policy 0, policy_version 1430852 (0.0008) [2023-12-27 15:33:41,568][20381] Updated weights for policy 1, policy_version 1429569 (0.0009) [2023-12-27 15:33:41,639][20381] Updated weights for policy 1, policy_version 1429579 (0.0008) [2023-12-27 15:33:42,224][20347] Updated weights for policy 0, policy_version 1430862 (0.0006) [2023-12-27 15:33:42,285][20347] Updated weights for policy 0, policy_version 1430872 (0.0007) [2023-12-27 15:33:42,351][20347] Updated weights for policy 0, policy_version 1430882 (0.0007) [2023-12-27 15:33:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 732372992. Throughput: 0: 10311.3, 1: 10200.8. Samples: 732370744. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:42,363][19170] Avg episode reward: [(0, '970.770'), (1, '733.920')] [2023-12-27 15:33:42,365][20381] Updated weights for policy 1, policy_version 1429589 (0.0007) [2023-12-27 15:33:42,425][20381] Updated weights for policy 1, policy_version 1429599 (0.0007) [2023-12-27 15:33:42,481][20381] Updated weights for policy 1, policy_version 1429609 (0.0007) [2023-12-27 15:33:42,958][20347] Updated weights for policy 0, policy_version 1430892 (0.0008) [2023-12-27 15:33:43,015][20347] Updated weights for policy 0, policy_version 1430902 (0.0008) [2023-12-27 15:33:43,077][20347] Updated weights for policy 0, policy_version 1430912 (0.0008) [2023-12-27 15:33:43,082][20381] Updated weights for policy 1, policy_version 1429619 (0.0009) [2023-12-27 15:33:43,130][20381] Updated weights for policy 1, policy_version 1429629 (0.0007) [2023-12-27 15:33:43,182][20381] Updated weights for policy 1, policy_version 1429639 (0.0007) [2023-12-27 15:33:43,220][20292] KL-divergence is very high: 327.6282 [2023-12-27 15:33:43,769][20347] Updated weights for policy 0, policy_version 1430922 (0.0005) [2023-12-27 15:33:43,832][20347] Updated weights for policy 0, policy_version 1430932 (0.0006) [2023-12-27 15:33:43,869][20381] Updated weights for policy 1, policy_version 1429649 (0.0007) [2023-12-27 15:33:43,883][20292] KL-divergence is very high: 421.9203 [2023-12-27 15:33:43,891][20347] Updated weights for policy 0, policy_version 1430942 (0.0006) [2023-12-27 15:33:43,920][20381] Updated weights for policy 1, policy_version 1429659 (0.0007) [2023-12-27 15:33:43,926][20292] KL-divergence is very high: 511.2424 [2023-12-27 15:33:43,941][20347] Updated weights for policy 0, policy_version 1430952 (0.0005) [2023-12-27 15:33:43,963][20292] KL-divergence is very high: 289.0877 [2023-12-27 15:33:43,970][20381] Updated weights for policy 1, policy_version 1429669 (0.0007) [2023-12-27 15:33:44,004][20292] KL-divergence is very high: 232.9561 [2023-12-27 15:33:44,018][20381] Updated weights for policy 1, policy_version 1429679 (0.0005) [2023-12-27 15:33:44,463][20347] Updated weights for policy 0, policy_version 1430962 (0.0008) [2023-12-27 15:33:44,515][20347] Updated weights for policy 0, policy_version 1430972 (0.0010) [2023-12-27 15:33:44,557][20347] Updated weights for policy 0, policy_version 1430982 (0.0010) [2023-12-27 15:33:44,699][20381] Updated weights for policy 1, policy_version 1429689 (0.0007) [2023-12-27 15:33:44,754][20381] Updated weights for policy 1, policy_version 1429699 (0.0007) [2023-12-27 15:33:44,815][20381] Updated weights for policy 1, policy_version 1429709 (0.0008) [2023-12-27 15:33:45,283][20347] Updated weights for policy 0, policy_version 1430992 (0.0011) [2023-12-27 15:33:45,342][20347] Updated weights for policy 0, policy_version 1431002 (0.0010) [2023-12-27 15:33:45,396][20347] Updated weights for policy 0, policy_version 1431012 (0.0010) [2023-12-27 15:33:45,486][20381] Updated weights for policy 1, policy_version 1429719 (0.0009) [2023-12-27 15:33:45,538][20381] Updated weights for policy 1, policy_version 1429729 (0.0008) [2023-12-27 15:33:45,589][20381] Updated weights for policy 1, policy_version 1429739 (0.0008) [2023-12-27 15:33:46,042][20347] Updated weights for policy 0, policy_version 1431022 (0.0009) [2023-12-27 15:33:46,106][20347] Updated weights for policy 0, policy_version 1431032 (0.0008) [2023-12-27 15:33:46,161][20347] Updated weights for policy 0, policy_version 1431042 (0.0010) [2023-12-27 15:33:46,242][20381] Updated weights for policy 1, policy_version 1429749 (0.0007) [2023-12-27 15:33:46,292][20381] Updated weights for policy 1, policy_version 1429759 (0.0007) [2023-12-27 15:33:46,344][20381] Updated weights for policy 1, policy_version 1429769 (0.0007) [2023-12-27 15:33:46,796][20347] Updated weights for policy 0, policy_version 1431052 (0.0010) [2023-12-27 15:33:46,852][20347] Updated weights for policy 0, policy_version 1431062 (0.0010) [2023-12-27 15:33:46,911][20347] Updated weights for policy 0, policy_version 1431072 (0.0006) [2023-12-27 15:33:47,000][20381] Updated weights for policy 1, policy_version 1429779 (0.0006) [2023-12-27 15:33:47,060][20381] Updated weights for policy 1, policy_version 1429789 (0.0007) [2023-12-27 15:33:47,111][20381] Updated weights for policy 1, policy_version 1429799 (0.0008) [2023-12-27 15:33:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 732487680. Throughput: 0: 10283.0, 1: 10245.2. Samples: 732495772. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:47,363][19170] Avg episode reward: [(0, '990.270'), (1, '730.030')] [2023-12-27 15:33:47,576][20347] Updated weights for policy 0, policy_version 1431082 (0.0006) [2023-12-27 15:33:47,634][20347] Updated weights for policy 0, policy_version 1431092 (0.0010) [2023-12-27 15:33:47,680][20347] Updated weights for policy 0, policy_version 1431102 (0.0006) [2023-12-27 15:33:47,736][20347] Updated weights for policy 0, policy_version 1431112 (0.0007) [2023-12-27 15:33:47,799][20381] Updated weights for policy 1, policy_version 1429809 (0.0005) [2023-12-27 15:33:47,851][20381] Updated weights for policy 1, policy_version 1429819 (0.0008) [2023-12-27 15:33:47,903][20381] Updated weights for policy 1, policy_version 1429829 (0.0007) [2023-12-27 15:33:47,954][20381] Updated weights for policy 1, policy_version 1429839 (0.0007) [2023-12-27 15:33:48,364][20347] Updated weights for policy 0, policy_version 1431122 (0.0008) [2023-12-27 15:33:48,423][20347] Updated weights for policy 0, policy_version 1431132 (0.0008) [2023-12-27 15:33:48,476][20347] Updated weights for policy 0, policy_version 1431142 (0.0008) [2023-12-27 15:33:48,696][20381] Updated weights for policy 1, policy_version 1429849 (0.0008) [2023-12-27 15:33:48,752][20381] Updated weights for policy 1, policy_version 1429859 (0.0008) [2023-12-27 15:33:48,811][20381] Updated weights for policy 1, policy_version 1429869 (0.0008) [2023-12-27 15:33:49,166][20347] Updated weights for policy 0, policy_version 1431152 (0.0011) [2023-12-27 15:33:49,231][20347] Updated weights for policy 0, policy_version 1431162 (0.0010) [2023-12-27 15:33:49,295][20347] Updated weights for policy 0, policy_version 1431172 (0.0009) [2023-12-27 15:33:49,541][20381] Updated weights for policy 1, policy_version 1429879 (0.0007) [2023-12-27 15:33:49,596][20381] Updated weights for policy 1, policy_version 1429889 (0.0007) [2023-12-27 15:33:49,658][20381] Updated weights for policy 1, policy_version 1429899 (0.0007) [2023-12-27 15:33:50,014][20347] Updated weights for policy 0, policy_version 1431182 (0.0009) [2023-12-27 15:33:50,074][20347] Updated weights for policy 0, policy_version 1431192 (0.0008) [2023-12-27 15:33:50,132][20347] Updated weights for policy 0, policy_version 1431202 (0.0008) [2023-12-27 15:33:50,352][20381] Updated weights for policy 1, policy_version 1429909 (0.0006) [2023-12-27 15:33:50,409][20381] Updated weights for policy 1, policy_version 1429919 (0.0005) [2023-12-27 15:33:50,459][20381] Updated weights for policy 1, policy_version 1429929 (0.0005) [2023-12-27 15:33:50,888][20347] Updated weights for policy 0, policy_version 1431212 (0.0009) [2023-12-27 15:33:50,949][20347] Updated weights for policy 0, policy_version 1431222 (0.0010) [2023-12-27 15:33:51,005][20347] Updated weights for policy 0, policy_version 1431232 (0.0010) [2023-12-27 15:33:51,119][20381] Updated weights for policy 1, policy_version 1429939 (0.0006) [2023-12-27 15:33:51,185][20381] Updated weights for policy 1, policy_version 1429949 (0.0008) [2023-12-27 15:33:51,250][20381] Updated weights for policy 1, policy_version 1429959 (0.0008) [2023-12-27 15:33:51,790][20347] Updated weights for policy 0, policy_version 1431242 (0.0008) [2023-12-27 15:33:51,854][20347] Updated weights for policy 0, policy_version 1431252 (0.0008) [2023-12-27 15:33:51,915][20347] Updated weights for policy 0, policy_version 1431262 (0.0008) [2023-12-27 15:33:51,943][20381] Updated weights for policy 1, policy_version 1429969 (0.0008) [2023-12-27 15:33:51,974][20347] Updated weights for policy 0, policy_version 1431272 (0.0007) [2023-12-27 15:33:51,993][20381] Updated weights for policy 1, policy_version 1429979 (0.0006) [2023-12-27 15:33:52,044][20381] Updated weights for policy 1, policy_version 1429989 (0.0007) [2023-12-27 15:33:52,096][20381] Updated weights for policy 1, policy_version 1429999 (0.0009) [2023-12-27 15:33:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 732585984. Throughput: 0: 10274.3, 1: 10211.6. Samples: 732555892. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:52,363][19170] Avg episode reward: [(0, '970.830'), (1, '718.450')] [2023-12-27 15:33:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001430000_366108672.pth... [2023-12-27 15:33:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001431272_366477312.pth... [2023-12-27 15:33:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001428784_365797376.pth [2023-12-27 15:33:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001430088_366174208.pth [2023-12-27 15:33:52,635][20347] Updated weights for policy 0, policy_version 1431282 (0.0006) [2023-12-27 15:33:52,695][20347] Updated weights for policy 0, policy_version 1431292 (0.0006) [2023-12-27 15:33:52,751][20347] Updated weights for policy 0, policy_version 1431302 (0.0007) [2023-12-27 15:33:52,803][20381] Updated weights for policy 1, policy_version 1430009 (0.0008) [2023-12-27 15:33:52,861][20381] Updated weights for policy 1, policy_version 1430019 (0.0007) [2023-12-27 15:33:52,918][20381] Updated weights for policy 1, policy_version 1430029 (0.0007) [2023-12-27 15:33:53,401][20347] Updated weights for policy 0, policy_version 1431312 (0.0007) [2023-12-27 15:33:53,458][20347] Updated weights for policy 0, policy_version 1431322 (0.0008) [2023-12-27 15:33:53,514][20347] Updated weights for policy 0, policy_version 1431332 (0.0008) [2023-12-27 15:33:53,568][20381] Updated weights for policy 1, policy_version 1430039 (0.0008) [2023-12-27 15:33:53,613][20381] Updated weights for policy 1, policy_version 1430049 (0.0008) [2023-12-27 15:33:53,664][20381] Updated weights for policy 1, policy_version 1430059 (0.0007) [2023-12-27 15:33:54,191][20347] Updated weights for policy 0, policy_version 1431342 (0.0008) [2023-12-27 15:33:54,248][20347] Updated weights for policy 0, policy_version 1431352 (0.0008) [2023-12-27 15:33:54,307][20347] Updated weights for policy 0, policy_version 1431362 (0.0008) [2023-12-27 15:33:54,364][20381] Updated weights for policy 1, policy_version 1430069 (0.0008) [2023-12-27 15:33:54,420][20381] Updated weights for policy 1, policy_version 1430079 (0.0008) [2023-12-27 15:33:54,474][20381] Updated weights for policy 1, policy_version 1430089 (0.0008) [2023-12-27 15:33:55,003][20347] Updated weights for policy 0, policy_version 1431372 (0.0008) [2023-12-27 15:33:55,068][20347] Updated weights for policy 0, policy_version 1431382 (0.0008) [2023-12-27 15:33:55,124][20347] Updated weights for policy 0, policy_version 1431392 (0.0007) [2023-12-27 15:33:55,196][20381] Updated weights for policy 1, policy_version 1430099 (0.0008) [2023-12-27 15:33:55,262][20381] Updated weights for policy 1, policy_version 1430109 (0.0008) [2023-12-27 15:33:55,326][20381] Updated weights for policy 1, policy_version 1430119 (0.0008) [2023-12-27 15:33:55,763][20347] Updated weights for policy 0, policy_version 1431402 (0.0008) [2023-12-27 15:33:55,812][20347] Updated weights for policy 0, policy_version 1431412 (0.0008) [2023-12-27 15:33:55,867][20347] Updated weights for policy 0, policy_version 1431422 (0.0005) [2023-12-27 15:33:55,922][20347] Updated weights for policy 0, policy_version 1431432 (0.0005) [2023-12-27 15:33:56,037][20381] Updated weights for policy 1, policy_version 1430129 (0.0008) [2023-12-27 15:33:56,092][20381] Updated weights for policy 1, policy_version 1430139 (0.0008) [2023-12-27 15:33:56,145][20381] Updated weights for policy 1, policy_version 1430149 (0.0009) [2023-12-27 15:33:56,197][20381] Updated weights for policy 1, policy_version 1430159 (0.0009) [2023-12-27 15:33:56,463][20347] Updated weights for policy 0, policy_version 1431442 (0.0008) [2023-12-27 15:33:56,521][20347] Updated weights for policy 0, policy_version 1431452 (0.0010) [2023-12-27 15:33:56,568][20347] Updated weights for policy 0, policy_version 1431462 (0.0010) [2023-12-27 15:33:56,889][20381] Updated weights for policy 1, policy_version 1430169 (0.0009) [2023-12-27 15:33:56,939][20381] Updated weights for policy 1, policy_version 1430179 (0.0006) [2023-12-27 15:33:56,994][20381] Updated weights for policy 1, policy_version 1430189 (0.0007) [2023-12-27 15:33:57,195][20347] Updated weights for policy 0, policy_version 1431472 (0.0008) [2023-12-27 15:33:57,251][20347] Updated weights for policy 0, policy_version 1431482 (0.0008) [2023-12-27 15:33:57,295][20347] Updated weights for policy 0, policy_version 1431492 (0.0008) [2023-12-27 15:33:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 732692480. Throughput: 0: 10243.2, 1: 10173.7. Samples: 732676052. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:33:57,363][19170] Avg episode reward: [(0, '960.980'), (1, '693.900')] [2023-12-27 15:33:57,604][20381] Updated weights for policy 1, policy_version 1430199 (0.0008) [2023-12-27 15:33:57,661][20381] Updated weights for policy 1, policy_version 1430209 (0.0008) [2023-12-27 15:33:57,718][20381] Updated weights for policy 1, policy_version 1430219 (0.0008) [2023-12-27 15:33:58,010][20347] Updated weights for policy 0, policy_version 1431502 (0.0008) [2023-12-27 15:33:58,059][20347] Updated weights for policy 0, policy_version 1431512 (0.0005) [2023-12-27 15:33:58,111][20347] Updated weights for policy 0, policy_version 1431522 (0.0009) [2023-12-27 15:33:58,381][20381] Updated weights for policy 1, policy_version 1430229 (0.0008) [2023-12-27 15:33:58,438][20381] Updated weights for policy 1, policy_version 1430239 (0.0008) [2023-12-27 15:33:58,497][20381] Updated weights for policy 1, policy_version 1430249 (0.0008) [2023-12-27 15:33:58,919][20347] Updated weights for policy 0, policy_version 1431532 (0.0007) [2023-12-27 15:33:58,979][20347] Updated weights for policy 0, policy_version 1431542 (0.0008) [2023-12-27 15:33:59,043][20347] Updated weights for policy 0, policy_version 1431552 (0.0007) [2023-12-27 15:33:59,350][20381] Updated weights for policy 1, policy_version 1430259 (0.0008) [2023-12-27 15:33:59,404][20292] KL-divergence is very high: 100.3510 [2023-12-27 15:33:59,415][20381] Updated weights for policy 1, policy_version 1430269 (0.0008) [2023-12-27 15:33:59,474][20381] Updated weights for policy 1, policy_version 1430279 (0.0007) [2023-12-27 15:33:59,748][20347] Updated weights for policy 0, policy_version 1431562 (0.0007) [2023-12-27 15:33:59,811][20347] Updated weights for policy 0, policy_version 1431572 (0.0007) [2023-12-27 15:33:59,873][20347] Updated weights for policy 0, policy_version 1431582 (0.0008) [2023-12-27 15:33:59,932][20347] Updated weights for policy 0, policy_version 1431592 (0.0009) [2023-12-27 15:34:00,202][20381] Updated weights for policy 1, policy_version 1430289 (0.0007) [2023-12-27 15:34:00,269][20381] Updated weights for policy 1, policy_version 1430299 (0.0006) [2023-12-27 15:34:00,333][20381] Updated weights for policy 1, policy_version 1430309 (0.0005) [2023-12-27 15:34:00,400][20381] Updated weights for policy 1, policy_version 1430319 (0.0005) [2023-12-27 15:34:00,637][20347] Updated weights for policy 0, policy_version 1431602 (0.0010) [2023-12-27 15:34:00,694][20347] Updated weights for policy 0, policy_version 1431612 (0.0010) [2023-12-27 15:34:00,745][20347] Updated weights for policy 0, policy_version 1431622 (0.0007) [2023-12-27 15:34:00,990][20381] Updated weights for policy 1, policy_version 1430329 (0.0007) [2023-12-27 15:34:01,049][20381] Updated weights for policy 1, policy_version 1430339 (0.0008) [2023-12-27 15:34:01,102][20381] Updated weights for policy 1, policy_version 1430349 (0.0007) [2023-12-27 15:34:01,480][20347] Updated weights for policy 0, policy_version 1431632 (0.0006) [2023-12-27 15:34:01,536][20347] Updated weights for policy 0, policy_version 1431642 (0.0007) [2023-12-27 15:34:01,593][20347] Updated weights for policy 0, policy_version 1431652 (0.0005) [2023-12-27 15:34:01,847][20381] Updated weights for policy 1, policy_version 1430359 (0.0007) [2023-12-27 15:34:01,892][20381] Updated weights for policy 1, policy_version 1430369 (0.0006) [2023-12-27 15:34:01,951][20381] Updated weights for policy 1, policy_version 1430379 (0.0009) [2023-12-27 15:34:02,256][20347] Updated weights for policy 0, policy_version 1431662 (0.0007) [2023-12-27 15:34:02,313][20347] Updated weights for policy 0, policy_version 1431672 (0.0008) [2023-12-27 15:34:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 732782592. Throughput: 0: 10200.5, 1: 10173.0. Samples: 732797228. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-12-27 15:34:02,363][19170] Avg episode reward: [(0, '931.600'), (1, '674.620')] [2023-12-27 15:34:02,370][20347] Updated weights for policy 0, policy_version 1431682 (0.0008) [2023-12-27 15:34:02,625][20381] Updated weights for policy 1, policy_version 1430389 (0.0007) [2023-12-27 15:34:02,679][20381] Updated weights for policy 1, policy_version 1430399 (0.0008) [2023-12-27 15:34:02,734][20381] Updated weights for policy 1, policy_version 1430409 (0.0007) [2023-12-27 15:34:03,102][20347] Updated weights for policy 0, policy_version 1431692 (0.0009) [2023-12-27 15:34:03,153][20347] Updated weights for policy 0, policy_version 1431702 (0.0010) [2023-12-27 15:34:03,208][20347] Updated weights for policy 0, policy_version 1431712 (0.0009) [2023-12-27 15:34:03,365][20381] Updated weights for policy 1, policy_version 1430419 (0.0006) [2023-12-27 15:34:03,420][20381] Updated weights for policy 1, policy_version 1430429 (0.0005) [2023-12-27 15:34:03,486][20381] Updated weights for policy 1, policy_version 1430439 (0.0005) [2023-12-27 15:34:03,786][20347] Updated weights for policy 0, policy_version 1431722 (0.0006) [2023-12-27 15:34:03,834][20347] Updated weights for policy 0, policy_version 1431732 (0.0010) [2023-12-27 15:34:03,898][20347] Updated weights for policy 0, policy_version 1431742 (0.0011) [2023-12-27 15:34:03,954][20347] Updated weights for policy 0, policy_version 1431752 (0.0010) [2023-12-27 15:34:04,005][20381] Updated weights for policy 1, policy_version 1430449 (0.0005) [2023-12-27 15:34:04,063][20381] Updated weights for policy 1, policy_version 1430459 (0.0007) [2023-12-27 15:34:04,116][20381] Updated weights for policy 1, policy_version 1430469 (0.0008) [2023-12-27 15:34:04,177][20381] Updated weights for policy 1, policy_version 1430479 (0.0007) [2023-12-27 15:34:04,620][20347] Updated weights for policy 0, policy_version 1431762 (0.0010) [2023-12-27 15:34:04,682][20347] Updated weights for policy 0, policy_version 1431772 (0.0008) [2023-12-27 15:34:04,731][20347] Updated weights for policy 0, policy_version 1431782 (0.0005) [2023-12-27 15:34:04,804][20381] Updated weights for policy 1, policy_version 1430489 (0.0008) [2023-12-27 15:34:04,855][20381] Updated weights for policy 1, policy_version 1430499 (0.0009) [2023-12-27 15:34:04,905][20381] Updated weights for policy 1, policy_version 1430509 (0.0011) [2023-12-27 15:34:05,308][20347] Updated weights for policy 0, policy_version 1431792 (0.0005) [2023-12-27 15:34:05,354][20347] Updated weights for policy 0, policy_version 1431802 (0.0005) [2023-12-27 15:34:05,402][20347] Updated weights for policy 0, policy_version 1431812 (0.0007) [2023-12-27 15:34:05,630][20381] Updated weights for policy 1, policy_version 1430519 (0.0008) [2023-12-27 15:34:05,677][20381] Updated weights for policy 1, policy_version 1430529 (0.0007) [2023-12-27 15:34:05,729][20381] Updated weights for policy 1, policy_version 1430539 (0.0007) [2023-12-27 15:34:06,039][20347] Updated weights for policy 0, policy_version 1431822 (0.0006) [2023-12-27 15:34:06,100][20347] Updated weights for policy 0, policy_version 1431832 (0.0006) [2023-12-27 15:34:06,161][20347] Updated weights for policy 0, policy_version 1431842 (0.0010) [2023-12-27 15:34:06,393][20381] Updated weights for policy 1, policy_version 1430549 (0.0007) [2023-12-27 15:34:06,443][20381] Updated weights for policy 1, policy_version 1430559 (0.0007) [2023-12-27 15:34:06,492][20381] Updated weights for policy 1, policy_version 1430569 (0.0008) [2023-12-27 15:34:06,820][20347] Updated weights for policy 0, policy_version 1431852 (0.0010) [2023-12-27 15:34:06,875][20347] Updated weights for policy 0, policy_version 1431862 (0.0010) [2023-12-27 15:34:06,933][20347] Updated weights for policy 0, policy_version 1431872 (0.0010) [2023-12-27 15:34:07,209][20381] Updated weights for policy 1, policy_version 1430579 (0.0008) [2023-12-27 15:34:07,275][20381] Updated weights for policy 1, policy_version 1430589 (0.0008) [2023-12-27 15:34:07,327][20381] Updated weights for policy 1, policy_version 1430599 (0.0010) [2023-12-27 15:34:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 732889088. Throughput: 0: 10211.7, 1: 10229.2. Samples: 732861928. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:07,363][19170] Avg episode reward: [(0, '931.600'), (1, '799.010')] [2023-12-27 15:34:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001431880_366632960.pth... [2023-12-27 15:34:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001430696_366329856.pth [2023-12-27 15:34:07,375][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001430608_366264320.pth... [2023-12-27 15:34:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001429424_365961216.pth [2023-12-27 15:34:07,610][20347] Updated weights for policy 0, policy_version 1431882 (0.0010) [2023-12-27 15:34:07,669][20347] Updated weights for policy 0, policy_version 1431892 (0.0008) [2023-12-27 15:34:07,721][20347] Updated weights for policy 0, policy_version 1431902 (0.0007) [2023-12-27 15:34:07,779][20347] Updated weights for policy 0, policy_version 1431912 (0.0010) [2023-12-27 15:34:07,968][20381] Updated weights for policy 1, policy_version 1430609 (0.0010) [2023-12-27 15:34:08,020][20381] Updated weights for policy 1, policy_version 1430619 (0.0010) [2023-12-27 15:34:08,071][20381] Updated weights for policy 1, policy_version 1430629 (0.0010) [2023-12-27 15:34:08,133][20381] Updated weights for policy 1, policy_version 1430639 (0.0011) [2023-12-27 15:34:08,439][20347] Updated weights for policy 0, policy_version 1431922 (0.0010) [2023-12-27 15:34:08,491][20347] Updated weights for policy 0, policy_version 1431932 (0.0010) [2023-12-27 15:34:08,551][20347] Updated weights for policy 0, policy_version 1431942 (0.0010) [2023-12-27 15:34:08,810][20381] Updated weights for policy 1, policy_version 1430649 (0.0010) [2023-12-27 15:34:08,865][20381] Updated weights for policy 1, policy_version 1430659 (0.0010) [2023-12-27 15:34:08,920][20381] Updated weights for policy 1, policy_version 1430669 (0.0010) [2023-12-27 15:34:09,237][20347] Updated weights for policy 0, policy_version 1431952 (0.0010) [2023-12-27 15:34:09,299][20347] Updated weights for policy 0, policy_version 1431962 (0.0007) [2023-12-27 15:34:09,362][20347] Updated weights for policy 0, policy_version 1431972 (0.0008) [2023-12-27 15:34:09,686][20381] Updated weights for policy 1, policy_version 1430679 (0.0011) [2023-12-27 15:34:09,742][20381] Updated weights for policy 1, policy_version 1430689 (0.0011) [2023-12-27 15:34:09,803][20381] Updated weights for policy 1, policy_version 1430699 (0.0008) [2023-12-27 15:34:10,047][20347] Updated weights for policy 0, policy_version 1431982 (0.0010) [2023-12-27 15:34:10,100][20347] Updated weights for policy 0, policy_version 1431992 (0.0010) [2023-12-27 15:34:10,155][20347] Updated weights for policy 0, policy_version 1432002 (0.0010) [2023-12-27 15:34:10,501][20381] Updated weights for policy 1, policy_version 1430709 (0.0009) [2023-12-27 15:34:10,556][20381] Updated weights for policy 1, policy_version 1430719 (0.0007) [2023-12-27 15:34:10,604][20381] Updated weights for policy 1, policy_version 1430729 (0.0008) [2023-12-27 15:34:10,790][20347] Updated weights for policy 0, policy_version 1432012 (0.0009) [2023-12-27 15:34:10,839][20347] Updated weights for policy 0, policy_version 1432022 (0.0010) [2023-12-27 15:34:10,886][20347] Updated weights for policy 0, policy_version 1432032 (0.0010) [2023-12-27 15:34:11,312][20381] Updated weights for policy 1, policy_version 1430739 (0.0007) [2023-12-27 15:34:11,376][20381] Updated weights for policy 1, policy_version 1430749 (0.0009) [2023-12-27 15:34:11,445][20381] Updated weights for policy 1, policy_version 1430759 (0.0006) [2023-12-27 15:34:11,676][20347] Updated weights for policy 0, policy_version 1432042 (0.0010) [2023-12-27 15:34:11,739][20347] Updated weights for policy 0, policy_version 1432052 (0.0008) [2023-12-27 15:34:11,797][20347] Updated weights for policy 0, policy_version 1432062 (0.0008) [2023-12-27 15:34:11,853][20347] Updated weights for policy 0, policy_version 1432072 (0.0008) [2023-12-27 15:34:12,080][20381] Updated weights for policy 1, policy_version 1430769 (0.0006) [2023-12-27 15:34:12,130][20381] Updated weights for policy 1, policy_version 1430779 (0.0006) [2023-12-27 15:34:12,188][20381] Updated weights for policy 1, policy_version 1430789 (0.0005) [2023-12-27 15:34:12,248][20381] Updated weights for policy 1, policy_version 1430799 (0.0006) [2023-12-27 15:34:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 732995584. Throughput: 0: 10274.4, 1: 10217.7. Samples: 732984972. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:12,363][19170] Avg episode reward: [(0, '902.350'), (1, '768.420')] [2023-12-27 15:34:12,593][20347] Updated weights for policy 0, policy_version 1432082 (0.0005) [2023-12-27 15:34:12,651][20347] Updated weights for policy 0, policy_version 1432092 (0.0009) [2023-12-27 15:34:12,703][20347] Updated weights for policy 0, policy_version 1432102 (0.0006) [2023-12-27 15:34:12,946][20381] Updated weights for policy 1, policy_version 1430809 (0.0008) [2023-12-27 15:34:13,001][20381] Updated weights for policy 1, policy_version 1430819 (0.0008) [2023-12-27 15:34:13,055][20381] Updated weights for policy 1, policy_version 1430829 (0.0007) [2023-12-27 15:34:13,346][20347] Updated weights for policy 0, policy_version 1432112 (0.0009) [2023-12-27 15:34:13,409][20347] Updated weights for policy 0, policy_version 1432122 (0.0010) [2023-12-27 15:34:13,471][20347] Updated weights for policy 0, policy_version 1432132 (0.0006) [2023-12-27 15:34:13,719][20381] Updated weights for policy 1, policy_version 1430839 (0.0007) [2023-12-27 15:34:13,766][20381] Updated weights for policy 1, policy_version 1430849 (0.0008) [2023-12-27 15:34:13,813][20381] Updated weights for policy 1, policy_version 1430859 (0.0007) [2023-12-27 15:34:14,052][20347] Updated weights for policy 0, policy_version 1432142 (0.0005) [2023-12-27 15:34:14,104][20347] Updated weights for policy 0, policy_version 1432152 (0.0005) [2023-12-27 15:34:14,157][20347] Updated weights for policy 0, policy_version 1432162 (0.0008) [2023-12-27 15:34:14,543][20381] Updated weights for policy 1, policy_version 1430869 (0.0008) [2023-12-27 15:34:14,590][20381] Updated weights for policy 1, policy_version 1430879 (0.0008) [2023-12-27 15:34:14,638][20381] Updated weights for policy 1, policy_version 1430889 (0.0008) [2023-12-27 15:34:14,762][20347] Updated weights for policy 0, policy_version 1432172 (0.0009) [2023-12-27 15:34:14,817][20347] Updated weights for policy 0, policy_version 1432182 (0.0008) [2023-12-27 15:34:14,877][20347] Updated weights for policy 0, policy_version 1432192 (0.0008) [2023-12-27 15:34:15,345][20381] Updated weights for policy 1, policy_version 1430899 (0.0007) [2023-12-27 15:34:15,418][20381] Updated weights for policy 1, policy_version 1430909 (0.0006) [2023-12-27 15:34:15,466][20381] Updated weights for policy 1, policy_version 1430919 (0.0008) [2023-12-27 15:34:15,467][20292] KL-divergence is very high: 100.4009 [2023-12-27 15:34:15,590][20347] Updated weights for policy 0, policy_version 1432202 (0.0008) [2023-12-27 15:34:15,641][20347] Updated weights for policy 0, policy_version 1432212 (0.0007) [2023-12-27 15:34:15,695][20347] Updated weights for policy 0, policy_version 1432222 (0.0006) [2023-12-27 15:34:15,748][20347] Updated weights for policy 0, policy_version 1432232 (0.0008) [2023-12-27 15:34:16,003][20381] Updated weights for policy 1, policy_version 1430929 (0.0006) [2023-12-27 15:34:16,061][20381] Updated weights for policy 1, policy_version 1430939 (0.0005) [2023-12-27 15:34:16,124][20381] Updated weights for policy 1, policy_version 1430949 (0.0008) [2023-12-27 15:34:16,179][20381] Updated weights for policy 1, policy_version 1430959 (0.0008) [2023-12-27 15:34:16,394][20347] Updated weights for policy 0, policy_version 1432242 (0.0007) [2023-12-27 15:34:16,454][20347] Updated weights for policy 0, policy_version 1432252 (0.0008) [2023-12-27 15:34:16,521][20347] Updated weights for policy 0, policy_version 1432262 (0.0008) [2023-12-27 15:34:16,771][20381] Updated weights for policy 1, policy_version 1430969 (0.0006) [2023-12-27 15:34:16,828][20381] Updated weights for policy 1, policy_version 1430979 (0.0007) [2023-12-27 15:34:16,884][20381] Updated weights for policy 1, policy_version 1430989 (0.0007) [2023-12-27 15:34:17,182][20347] Updated weights for policy 0, policy_version 1432272 (0.0008) [2023-12-27 15:34:17,230][20291] KL-divergence is very high: 139.8954 [2023-12-27 15:34:17,231][20347] Updated weights for policy 0, policy_version 1432282 (0.0008) [2023-12-27 15:34:17,273][20291] KL-divergence is very high: 165.6502 [2023-12-27 15:34:17,283][20347] Updated weights for policy 0, policy_version 1432292 (0.0007) [2023-12-27 15:34:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 733102080. Throughput: 0: 10255.0, 1: 10224.3. Samples: 733109488. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:17,363][19170] Avg episode reward: [(0, '912.020'), (1, '670.730')] [2023-12-27 15:34:17,513][20381] Updated weights for policy 1, policy_version 1430999 (0.0006) [2023-12-27 15:34:17,574][20381] Updated weights for policy 1, policy_version 1431009 (0.0007) [2023-12-27 15:34:17,638][20381] Updated weights for policy 1, policy_version 1431019 (0.0006) [2023-12-27 15:34:17,954][20347] Updated weights for policy 0, policy_version 1432302 (0.0007) [2023-12-27 15:34:18,007][20347] Updated weights for policy 0, policy_version 1432312 (0.0006) [2023-12-27 15:34:18,059][20347] Updated weights for policy 0, policy_version 1432322 (0.0006) [2023-12-27 15:34:18,221][20381] Updated weights for policy 1, policy_version 1431029 (0.0007) [2023-12-27 15:34:18,267][20381] Updated weights for policy 1, policy_version 1431039 (0.0008) [2023-12-27 15:34:18,341][20381] Updated weights for policy 1, policy_version 1431049 (0.0008) [2023-12-27 15:34:18,775][20347] Updated weights for policy 0, policy_version 1432332 (0.0007) [2023-12-27 15:34:18,825][20347] Updated weights for policy 0, policy_version 1432342 (0.0009) [2023-12-27 15:34:18,882][20347] Updated weights for policy 0, policy_version 1432352 (0.0008) [2023-12-27 15:34:18,985][20381] Updated weights for policy 1, policy_version 1431059 (0.0007) [2023-12-27 15:34:19,038][20381] Updated weights for policy 1, policy_version 1431069 (0.0008) [2023-12-27 15:34:19,094][20381] Updated weights for policy 1, policy_version 1431079 (0.0007) [2023-12-27 15:34:19,626][20347] Updated weights for policy 0, policy_version 1432362 (0.0008) [2023-12-27 15:34:19,688][20347] Updated weights for policy 0, policy_version 1432372 (0.0011) [2023-12-27 15:34:19,749][20347] Updated weights for policy 0, policy_version 1432382 (0.0011) [2023-12-27 15:34:19,817][20347] Updated weights for policy 0, policy_version 1432392 (0.0011) [2023-12-27 15:34:19,823][20381] Updated weights for policy 1, policy_version 1431089 (0.0008) [2023-12-27 15:34:19,888][20381] Updated weights for policy 1, policy_version 1431099 (0.0009) [2023-12-27 15:34:19,953][20381] Updated weights for policy 1, policy_version 1431109 (0.0008) [2023-12-27 15:34:20,016][20381] Updated weights for policy 1, policy_version 1431119 (0.0006) [2023-12-27 15:34:20,566][20347] Updated weights for policy 0, policy_version 1432402 (0.0007) [2023-12-27 15:34:20,627][20347] Updated weights for policy 0, policy_version 1432412 (0.0008) [2023-12-27 15:34:20,685][20347] Updated weights for policy 0, policy_version 1432422 (0.0008) [2023-12-27 15:34:20,725][20381] Updated weights for policy 1, policy_version 1431129 (0.0010) [2023-12-27 15:34:20,788][20381] Updated weights for policy 1, policy_version 1431139 (0.0011) [2023-12-27 15:34:20,847][20381] Updated weights for policy 1, policy_version 1431149 (0.0011) [2023-12-27 15:34:21,400][20347] Updated weights for policy 0, policy_version 1432432 (0.0008) [2023-12-27 15:34:21,464][20347] Updated weights for policy 0, policy_version 1432442 (0.0006) [2023-12-27 15:34:21,521][20347] Updated weights for policy 0, policy_version 1432452 (0.0005) [2023-12-27 15:34:21,591][20381] Updated weights for policy 1, policy_version 1431159 (0.0008) [2023-12-27 15:34:21,665][20381] Updated weights for policy 1, policy_version 1431171 (0.0008) [2023-12-27 15:34:21,729][20381] Updated weights for policy 1, policy_version 1431181 (0.0008) [2023-12-27 15:34:22,192][20347] Updated weights for policy 0, policy_version 1432462 (0.0008) [2023-12-27 15:34:22,241][20347] Updated weights for policy 0, policy_version 1432472 (0.0007) [2023-12-27 15:34:22,303][20347] Updated weights for policy 0, policy_version 1432482 (0.0008) [2023-12-27 15:34:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 733200384. Throughput: 0: 10241.2, 1: 10203.8. Samples: 733169948. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:22,363][19170] Avg episode reward: [(0, '941.440'), (1, '787.210')] [2023-12-27 15:34:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001431184_366411776.pth... [2023-12-27 15:34:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001432488_366788608.pth... [2023-12-27 15:34:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001430000_366108672.pth [2023-12-27 15:34:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001431272_366477312.pth [2023-12-27 15:34:22,508][20381] Updated weights for policy 1, policy_version 1431191 (0.0008) [2023-12-27 15:34:22,571][20381] Updated weights for policy 1, policy_version 1431201 (0.0008) [2023-12-27 15:34:22,629][20381] Updated weights for policy 1, policy_version 1431211 (0.0007) [2023-12-27 15:34:23,030][20347] Updated weights for policy 0, policy_version 1432492 (0.0009) [2023-12-27 15:34:23,079][20347] Updated weights for policy 0, policy_version 1432502 (0.0010) [2023-12-27 15:34:23,132][20347] Updated weights for policy 0, policy_version 1432512 (0.0010) [2023-12-27 15:34:23,350][20381] Updated weights for policy 1, policy_version 1431221 (0.0009) [2023-12-27 15:34:23,395][20381] Updated weights for policy 1, policy_version 1431231 (0.0010) [2023-12-27 15:34:23,445][20381] Updated weights for policy 1, policy_version 1431241 (0.0011) [2023-12-27 15:34:23,797][20347] Updated weights for policy 0, policy_version 1432522 (0.0010) [2023-12-27 15:34:23,853][20347] Updated weights for policy 0, policy_version 1432532 (0.0010) [2023-12-27 15:34:23,916][20347] Updated weights for policy 0, policy_version 1432542 (0.0010) [2023-12-27 15:34:23,970][20347] Updated weights for policy 0, policy_version 1432552 (0.0010) [2023-12-27 15:34:24,117][20381] Updated weights for policy 1, policy_version 1431251 (0.0010) [2023-12-27 15:34:24,173][20381] Updated weights for policy 1, policy_version 1431261 (0.0006) [2023-12-27 15:34:24,229][20381] Updated weights for policy 1, policy_version 1431271 (0.0011) [2023-12-27 15:34:24,669][20347] Updated weights for policy 0, policy_version 1432562 (0.0005) [2023-12-27 15:34:24,721][20347] Updated weights for policy 0, policy_version 1432572 (0.0006) [2023-12-27 15:34:24,785][20347] Updated weights for policy 0, policy_version 1432582 (0.0010) [2023-12-27 15:34:24,912][20381] Updated weights for policy 1, policy_version 1431281 (0.0010) [2023-12-27 15:34:24,972][20381] Updated weights for policy 1, policy_version 1431291 (0.0005) [2023-12-27 15:34:25,035][20381] Updated weights for policy 1, policy_version 1431301 (0.0009) [2023-12-27 15:34:25,104][20381] Updated weights for policy 1, policy_version 1431311 (0.0011) [2023-12-27 15:34:25,420][20347] Updated weights for policy 0, policy_version 1432592 (0.0010) [2023-12-27 15:34:25,469][20347] Updated weights for policy 0, policy_version 1432602 (0.0010) [2023-12-27 15:34:25,522][20347] Updated weights for policy 0, policy_version 1432612 (0.0010) [2023-12-27 15:34:25,703][20381] Updated weights for policy 1, policy_version 1431321 (0.0011) [2023-12-27 15:34:25,761][20381] Updated weights for policy 1, policy_version 1431331 (0.0010) [2023-12-27 15:34:25,810][20381] Updated weights for policy 1, policy_version 1431341 (0.0011) [2023-12-27 15:34:26,133][20347] Updated weights for policy 0, policy_version 1432622 (0.0010) [2023-12-27 15:34:26,196][20347] Updated weights for policy 0, policy_version 1432632 (0.0011) [2023-12-27 15:34:26,249][20347] Updated weights for policy 0, policy_version 1432642 (0.0011) [2023-12-27 15:34:26,446][20381] Updated weights for policy 1, policy_version 1431351 (0.0005) [2023-12-27 15:34:26,492][20381] Updated weights for policy 1, policy_version 1431361 (0.0005) [2023-12-27 15:34:26,537][20381] Updated weights for policy 1, policy_version 1431371 (0.0007) [2023-12-27 15:34:26,868][20347] Updated weights for policy 0, policy_version 1432652 (0.0010) [2023-12-27 15:34:26,925][20347] Updated weights for policy 0, policy_version 1432662 (0.0010) [2023-12-27 15:34:26,987][20347] Updated weights for policy 0, policy_version 1432672 (0.0009) [2023-12-27 15:34:27,115][20381] Updated weights for policy 1, policy_version 1431381 (0.0010) [2023-12-27 15:34:27,169][20381] Updated weights for policy 1, policy_version 1431391 (0.0010) [2023-12-27 15:34:27,223][20381] Updated weights for policy 1, policy_version 1431401 (0.0010) [2023-12-27 15:34:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 733306880. Throughput: 0: 10251.7, 1: 10202.8. Samples: 733291196. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:27,363][19170] Avg episode reward: [(0, '961.190'), (1, '834.930')] [2023-12-27 15:34:27,604][20347] Updated weights for policy 0, policy_version 1432682 (0.0005) [2023-12-27 15:34:27,652][20347] Updated weights for policy 0, policy_version 1432692 (0.0005) [2023-12-27 15:34:27,705][20347] Updated weights for policy 0, policy_version 1432702 (0.0005) [2023-12-27 15:34:27,750][20347] Updated weights for policy 0, policy_version 1432712 (0.0005) [2023-12-27 15:34:27,838][20381] Updated weights for policy 1, policy_version 1431411 (0.0009) [2023-12-27 15:34:27,888][20381] Updated weights for policy 1, policy_version 1431421 (0.0005) [2023-12-27 15:34:27,937][20381] Updated weights for policy 1, policy_version 1431431 (0.0006) [2023-12-27 15:34:28,334][20347] Updated weights for policy 0, policy_version 1432722 (0.0010) [2023-12-27 15:34:28,391][20347] Updated weights for policy 0, policy_version 1432732 (0.0008) [2023-12-27 15:34:28,448][20347] Updated weights for policy 0, policy_version 1432742 (0.0008) [2023-12-27 15:34:28,566][20381] Updated weights for policy 1, policy_version 1431441 (0.0008) [2023-12-27 15:34:28,625][20381] Updated weights for policy 1, policy_version 1431451 (0.0006) [2023-12-27 15:34:28,683][20381] Updated weights for policy 1, policy_version 1431461 (0.0006) [2023-12-27 15:34:28,753][20381] Updated weights for policy 1, policy_version 1431471 (0.0008) [2023-12-27 15:34:29,202][20347] Updated weights for policy 0, policy_version 1432752 (0.0007) [2023-12-27 15:34:29,274][20347] Updated weights for policy 0, policy_version 1432762 (0.0007) [2023-12-27 15:34:29,336][20347] Updated weights for policy 0, policy_version 1432772 (0.0008) [2023-12-27 15:34:29,390][20381] Updated weights for policy 1, policy_version 1431481 (0.0007) [2023-12-27 15:34:29,435][20381] Updated weights for policy 1, policy_version 1431491 (0.0007) [2023-12-27 15:34:29,482][20381] Updated weights for policy 1, policy_version 1431501 (0.0007) [2023-12-27 15:34:30,018][20347] Updated weights for policy 0, policy_version 1432782 (0.0008) [2023-12-27 15:34:30,076][20347] Updated weights for policy 0, policy_version 1432792 (0.0008) [2023-12-27 15:34:30,139][20347] Updated weights for policy 0, policy_version 1432802 (0.0008) [2023-12-27 15:34:30,144][20381] Updated weights for policy 1, policy_version 1431511 (0.0006) [2023-12-27 15:34:30,197][20381] Updated weights for policy 1, policy_version 1431521 (0.0006) [2023-12-27 15:34:30,251][20381] Updated weights for policy 1, policy_version 1431531 (0.0005) [2023-12-27 15:34:30,781][20347] Updated weights for policy 0, policy_version 1432812 (0.0007) [2023-12-27 15:34:30,832][20347] Updated weights for policy 0, policy_version 1432822 (0.0005) [2023-12-27 15:34:30,882][20347] Updated weights for policy 0, policy_version 1432832 (0.0007) [2023-12-27 15:34:30,910][20381] Updated weights for policy 1, policy_version 1431541 (0.0005) [2023-12-27 15:34:30,971][20381] Updated weights for policy 1, policy_version 1431551 (0.0005) [2023-12-27 15:34:31,018][20381] Updated weights for policy 1, policy_version 1431561 (0.0007) [2023-12-27 15:34:31,556][20347] Updated weights for policy 0, policy_version 1432842 (0.0007) [2023-12-27 15:34:31,621][20347] Updated weights for policy 0, policy_version 1432852 (0.0007) [2023-12-27 15:34:31,685][20347] Updated weights for policy 0, policy_version 1432862 (0.0007) [2023-12-27 15:34:31,746][20347] Updated weights for policy 0, policy_version 1432872 (0.0008) [2023-12-27 15:34:31,761][20381] Updated weights for policy 1, policy_version 1431571 (0.0007) [2023-12-27 15:34:31,822][20381] Updated weights for policy 1, policy_version 1431581 (0.0007) [2023-12-27 15:34:31,880][20381] Updated weights for policy 1, policy_version 1431591 (0.0008) [2023-12-27 15:34:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 733405184. Throughput: 0: 10251.2, 1: 10245.2. Samples: 733418112. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:32,363][19170] Avg episode reward: [(0, '951.280'), (1, '728.500')] [2023-12-27 15:34:32,413][20347] Updated weights for policy 0, policy_version 1432882 (0.0008) [2023-12-27 15:34:32,476][20347] Updated weights for policy 0, policy_version 1432892 (0.0007) [2023-12-27 15:34:32,532][20347] Updated weights for policy 0, policy_version 1432902 (0.0008) [2023-12-27 15:34:32,602][20381] Updated weights for policy 1, policy_version 1431601 (0.0008) [2023-12-27 15:34:32,657][20381] Updated weights for policy 1, policy_version 1431611 (0.0007) [2023-12-27 15:34:32,714][20381] Updated weights for policy 1, policy_version 1431621 (0.0007) [2023-12-27 15:34:32,774][20381] Updated weights for policy 1, policy_version 1431631 (0.0008) [2023-12-27 15:34:33,134][20347] Updated weights for policy 0, policy_version 1432912 (0.0006) [2023-12-27 15:34:33,183][20347] Updated weights for policy 0, policy_version 1432922 (0.0005) [2023-12-27 15:34:33,244][20347] Updated weights for policy 0, policy_version 1432932 (0.0006) [2023-12-27 15:34:33,501][20381] Updated weights for policy 1, policy_version 1431641 (0.0007) [2023-12-27 15:34:33,557][20381] Updated weights for policy 1, policy_version 1431651 (0.0008) [2023-12-27 15:34:33,609][20381] Updated weights for policy 1, policy_version 1431661 (0.0008) [2023-12-27 15:34:33,789][20347] Updated weights for policy 0, policy_version 1432942 (0.0005) [2023-12-27 15:34:33,839][20347] Updated weights for policy 0, policy_version 1432952 (0.0005) [2023-12-27 15:34:33,895][20347] Updated weights for policy 0, policy_version 1432962 (0.0005) [2023-12-27 15:34:34,347][20381] Updated weights for policy 1, policy_version 1431671 (0.0007) [2023-12-27 15:34:34,407][20381] Updated weights for policy 1, policy_version 1431681 (0.0007) [2023-12-27 15:34:34,469][20381] Updated weights for policy 1, policy_version 1431691 (0.0007) [2023-12-27 15:34:34,541][20347] Updated weights for policy 0, policy_version 1432972 (0.0006) [2023-12-27 15:34:34,600][20347] Updated weights for policy 0, policy_version 1432982 (0.0008) [2023-12-27 15:34:34,662][20347] Updated weights for policy 0, policy_version 1432992 (0.0007) [2023-12-27 15:34:35,190][20381] Updated weights for policy 1, policy_version 1431701 (0.0008) [2023-12-27 15:34:35,248][20381] Updated weights for policy 1, policy_version 1431711 (0.0008) [2023-12-27 15:34:35,266][20347] Updated weights for policy 0, policy_version 1433002 (0.0008) [2023-12-27 15:34:35,304][20381] Updated weights for policy 1, policy_version 1431721 (0.0007) [2023-12-27 15:34:35,324][20347] Updated weights for policy 0, policy_version 1433012 (0.0006) [2023-12-27 15:34:35,374][20347] Updated weights for policy 0, policy_version 1433022 (0.0006) [2023-12-27 15:34:35,424][20347] Updated weights for policy 0, policy_version 1433032 (0.0007) [2023-12-27 15:34:36,034][20381] Updated weights for policy 1, policy_version 1431731 (0.0008) [2023-12-27 15:34:36,047][20347] Updated weights for policy 0, policy_version 1433042 (0.0006) [2023-12-27 15:34:36,087][20381] Updated weights for policy 1, policy_version 1431741 (0.0005) [2023-12-27 15:34:36,095][20347] Updated weights for policy 0, policy_version 1433052 (0.0006) [2023-12-27 15:34:36,147][20381] Updated weights for policy 1, policy_version 1431751 (0.0007) [2023-12-27 15:34:36,153][20347] Updated weights for policy 0, policy_version 1433062 (0.0007) [2023-12-27 15:34:36,874][20381] Updated weights for policy 1, policy_version 1431761 (0.0007) [2023-12-27 15:34:36,876][20347] Updated weights for policy 0, policy_version 1433072 (0.0005) [2023-12-27 15:34:36,923][20347] Updated weights for policy 0, policy_version 1433082 (0.0005) [2023-12-27 15:34:36,927][20381] Updated weights for policy 1, policy_version 1431771 (0.0008) [2023-12-27 15:34:36,984][20381] Updated weights for policy 1, policy_version 1431781 (0.0007) [2023-12-27 15:34:36,989][20347] Updated weights for policy 0, policy_version 1433092 (0.0005) [2023-12-27 15:34:37,036][20381] Updated weights for policy 1, policy_version 1431791 (0.0007) [2023-12-27 15:34:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 733511680. Throughput: 0: 10324.1, 1: 10231.0. Samples: 733480872. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:37,363][19170] Avg episode reward: [(0, '941.270'), (1, '718.190')] [2023-12-27 15:34:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001433096_366944256.pth... [2023-12-27 15:34:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001431792_366567424.pth... [2023-12-27 15:34:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001431880_366632960.pth [2023-12-27 15:34:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001430608_366264320.pth [2023-12-27 15:34:37,612][20347] Updated weights for policy 0, policy_version 1433102 (0.0007) [2023-12-27 15:34:37,665][20347] Updated weights for policy 0, policy_version 1433112 (0.0008) [2023-12-27 15:34:37,714][20347] Updated weights for policy 0, policy_version 1433122 (0.0007) [2023-12-27 15:34:37,756][20381] Updated weights for policy 1, policy_version 1431801 (0.0007) [2023-12-27 15:34:37,805][20381] Updated weights for policy 1, policy_version 1431811 (0.0008) [2023-12-27 15:34:37,854][20381] Updated weights for policy 1, policy_version 1431821 (0.0007) [2023-12-27 15:34:38,412][20347] Updated weights for policy 0, policy_version 1433132 (0.0007) [2023-12-27 15:34:38,471][20347] Updated weights for policy 0, policy_version 1433142 (0.0008) [2023-12-27 15:34:38,528][20347] Updated weights for policy 0, policy_version 1433152 (0.0006) [2023-12-27 15:34:38,561][20381] Updated weights for policy 1, policy_version 1431831 (0.0008) [2023-12-27 15:34:38,623][20381] Updated weights for policy 1, policy_version 1431841 (0.0008) [2023-12-27 15:34:38,687][20381] Updated weights for policy 1, policy_version 1431851 (0.0008) [2023-12-27 15:34:39,132][20347] Updated weights for policy 0, policy_version 1433162 (0.0006) [2023-12-27 15:34:39,193][20347] Updated weights for policy 0, policy_version 1433172 (0.0006) [2023-12-27 15:34:39,261][20347] Updated weights for policy 0, policy_version 1433182 (0.0009) [2023-12-27 15:34:39,315][20347] Updated weights for policy 0, policy_version 1433192 (0.0010) [2023-12-27 15:34:39,459][20381] Updated weights for policy 1, policy_version 1431861 (0.0007) [2023-12-27 15:34:39,522][20381] Updated weights for policy 1, policy_version 1431871 (0.0008) [2023-12-27 15:34:39,588][20381] Updated weights for policy 1, policy_version 1431881 (0.0007) [2023-12-27 15:34:40,038][20347] Updated weights for policy 0, policy_version 1433202 (0.0010) [2023-12-27 15:34:40,089][20347] Updated weights for policy 0, policy_version 1433212 (0.0010) [2023-12-27 15:34:40,156][20347] Updated weights for policy 0, policy_version 1433222 (0.0010) [2023-12-27 15:34:40,290][20381] Updated weights for policy 1, policy_version 1431891 (0.0007) [2023-12-27 15:34:40,350][20381] Updated weights for policy 1, policy_version 1431901 (0.0008) [2023-12-27 15:34:40,409][20381] Updated weights for policy 1, policy_version 1431911 (0.0006) [2023-12-27 15:34:40,844][20347] Updated weights for policy 0, policy_version 1433232 (0.0009) [2023-12-27 15:34:40,899][20347] Updated weights for policy 0, policy_version 1433242 (0.0007) [2023-12-27 15:34:40,957][20347] Updated weights for policy 0, policy_version 1433252 (0.0007) [2023-12-27 15:34:41,035][20381] Updated weights for policy 1, policy_version 1431921 (0.0006) [2023-12-27 15:34:41,104][20381] Updated weights for policy 1, policy_version 1431931 (0.0007) [2023-12-27 15:34:41,174][20381] Updated weights for policy 1, policy_version 1431941 (0.0009) [2023-12-27 15:34:41,232][20381] Updated weights for policy 1, policy_version 1431951 (0.0008) [2023-12-27 15:34:41,736][20347] Updated weights for policy 0, policy_version 1433262 (0.0008) [2023-12-27 15:34:41,794][20347] Updated weights for policy 0, policy_version 1433272 (0.0008) [2023-12-27 15:34:41,853][20347] Updated weights for policy 0, policy_version 1433282 (0.0008) [2023-12-27 15:34:41,967][20381] Updated weights for policy 1, policy_version 1431961 (0.0008) [2023-12-27 15:34:42,023][20381] Updated weights for policy 1, policy_version 1431971 (0.0008) [2023-12-27 15:34:42,079][20381] Updated weights for policy 1, policy_version 1431981 (0.0008) [2023-12-27 15:34:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 733609984. Throughput: 0: 10354.3, 1: 10192.8. Samples: 733600672. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:42,363][19170] Avg episode reward: [(0, '931.610'), (1, '697.130')] [2023-12-27 15:34:42,580][20347] Updated weights for policy 0, policy_version 1433292 (0.0008) [2023-12-27 15:34:42,640][20347] Updated weights for policy 0, policy_version 1433302 (0.0007) [2023-12-27 15:34:42,701][20347] Updated weights for policy 0, policy_version 1433312 (0.0007) [2023-12-27 15:34:42,817][20381] Updated weights for policy 1, policy_version 1431991 (0.0010) [2023-12-27 15:34:42,875][20381] Updated weights for policy 1, policy_version 1432001 (0.0010) [2023-12-27 15:34:42,937][20381] Updated weights for policy 1, policy_version 1432011 (0.0010) [2023-12-27 15:34:43,387][20347] Updated weights for policy 0, policy_version 1433322 (0.0007) [2023-12-27 15:34:43,434][20347] Updated weights for policy 0, policy_version 1433332 (0.0007) [2023-12-27 15:34:43,481][20347] Updated weights for policy 0, policy_version 1433342 (0.0007) [2023-12-27 15:34:43,533][20347] Updated weights for policy 0, policy_version 1433352 (0.0007) [2023-12-27 15:34:43,602][20381] Updated weights for policy 1, policy_version 1432021 (0.0010) [2023-12-27 15:34:43,650][20381] Updated weights for policy 1, policy_version 1432031 (0.0010) [2023-12-27 15:34:43,701][20381] Updated weights for policy 1, policy_version 1432041 (0.0010) [2023-12-27 15:34:44,269][20347] Updated weights for policy 0, policy_version 1433362 (0.0008) [2023-12-27 15:34:44,330][20381] Updated weights for policy 1, policy_version 1432051 (0.0009) [2023-12-27 15:34:44,331][20347] Updated weights for policy 0, policy_version 1433372 (0.0006) [2023-12-27 15:34:44,378][20381] Updated weights for policy 1, policy_version 1432061 (0.0007) [2023-12-27 15:34:44,386][20347] Updated weights for policy 0, policy_version 1433382 (0.0005) [2023-12-27 15:34:44,436][20381] Updated weights for policy 1, policy_version 1432071 (0.0006) [2023-12-27 15:34:45,060][20381] Updated weights for policy 1, policy_version 1432081 (0.0005) [2023-12-27 15:34:45,065][20347] Updated weights for policy 0, policy_version 1433392 (0.0008) [2023-12-27 15:34:45,118][20381] Updated weights for policy 1, policy_version 1432091 (0.0006) [2023-12-27 15:34:45,120][20347] Updated weights for policy 0, policy_version 1433402 (0.0007) [2023-12-27 15:34:45,168][20381] Updated weights for policy 1, policy_version 1432101 (0.0005) [2023-12-27 15:34:45,170][20347] Updated weights for policy 0, policy_version 1433412 (0.0006) [2023-12-27 15:34:45,227][20381] Updated weights for policy 1, policy_version 1432111 (0.0007) [2023-12-27 15:34:45,834][20381] Updated weights for policy 1, policy_version 1432121 (0.0006) [2023-12-27 15:34:45,889][20381] Updated weights for policy 1, policy_version 1432131 (0.0005) [2023-12-27 15:34:45,890][20347] Updated weights for policy 0, policy_version 1433422 (0.0009) [2023-12-27 15:34:45,934][20347] Updated weights for policy 0, policy_version 1433432 (0.0009) [2023-12-27 15:34:45,949][20381] Updated weights for policy 1, policy_version 1432141 (0.0006) [2023-12-27 15:34:45,985][20347] Updated weights for policy 0, policy_version 1433442 (0.0005) [2023-12-27 15:34:46,550][20381] Updated weights for policy 1, policy_version 1432151 (0.0006) [2023-12-27 15:34:46,590][20347] Updated weights for policy 0, policy_version 1433452 (0.0007) [2023-12-27 15:34:46,607][20381] Updated weights for policy 1, policy_version 1432161 (0.0005) [2023-12-27 15:34:46,638][20347] Updated weights for policy 0, policy_version 1433462 (0.0010) [2023-12-27 15:34:46,658][20381] Updated weights for policy 1, policy_version 1432171 (0.0005) [2023-12-27 15:34:46,697][20347] Updated weights for policy 0, policy_version 1433472 (0.0008) [2023-12-27 15:34:47,236][20381] Updated weights for policy 1, policy_version 1432181 (0.0008) [2023-12-27 15:34:47,280][20381] Updated weights for policy 1, policy_version 1432191 (0.0008) [2023-12-27 15:34:47,310][20347] Updated weights for policy 0, policy_version 1433482 (0.0008) [2023-12-27 15:34:47,329][20381] Updated weights for policy 1, policy_version 1432201 (0.0008) [2023-12-27 15:34:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 733716480. Throughput: 0: 10370.3, 1: 10264.0. Samples: 733725772. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:47,363][19170] Avg episode reward: [(0, '951.250'), (1, '727.640')] [2023-12-27 15:34:47,364][20347] Updated weights for policy 0, policy_version 1433492 (0.0010) [2023-12-27 15:34:47,422][20347] Updated weights for policy 0, policy_version 1433502 (0.0010) [2023-12-27 15:34:47,478][20347] Updated weights for policy 0, policy_version 1433512 (0.0010) [2023-12-27 15:34:47,998][20381] Updated weights for policy 1, policy_version 1432211 (0.0009) [2023-12-27 15:34:48,047][20381] Updated weights for policy 1, policy_version 1432221 (0.0010) [2023-12-27 15:34:48,097][20347] Updated weights for policy 0, policy_version 1433522 (0.0005) [2023-12-27 15:34:48,099][20381] Updated weights for policy 1, policy_version 1432231 (0.0011) [2023-12-27 15:34:48,146][20347] Updated weights for policy 0, policy_version 1433532 (0.0005) [2023-12-27 15:34:48,195][20347] Updated weights for policy 0, policy_version 1433542 (0.0007) [2023-12-27 15:34:48,768][20381] Updated weights for policy 1, policy_version 1432241 (0.0010) [2023-12-27 15:34:48,819][20381] Updated weights for policy 1, policy_version 1432251 (0.0005) [2023-12-27 15:34:48,871][20381] Updated weights for policy 1, policy_version 1432261 (0.0005) [2023-12-27 15:34:48,921][20381] Updated weights for policy 1, policy_version 1432271 (0.0006) [2023-12-27 15:34:48,960][20347] Updated weights for policy 0, policy_version 1433552 (0.0007) [2023-12-27 15:34:49,017][20347] Updated weights for policy 0, policy_version 1433562 (0.0007) [2023-12-27 15:34:49,073][20347] Updated weights for policy 0, policy_version 1433572 (0.0006) [2023-12-27 15:34:49,533][20381] Updated weights for policy 1, policy_version 1432281 (0.0006) [2023-12-27 15:34:49,595][20381] Updated weights for policy 1, policy_version 1432291 (0.0007) [2023-12-27 15:34:49,652][20381] Updated weights for policy 1, policy_version 1432301 (0.0005) [2023-12-27 15:34:49,803][20347] Updated weights for policy 0, policy_version 1433582 (0.0007) [2023-12-27 15:34:49,871][20347] Updated weights for policy 0, policy_version 1433592 (0.0008) [2023-12-27 15:34:49,930][20347] Updated weights for policy 0, policy_version 1433602 (0.0008) [2023-12-27 15:34:50,295][20381] Updated weights for policy 1, policy_version 1432311 (0.0007) [2023-12-27 15:34:50,341][20381] Updated weights for policy 1, policy_version 1432321 (0.0008) [2023-12-27 15:34:50,388][20381] Updated weights for policy 1, policy_version 1432331 (0.0008) [2023-12-27 15:34:50,645][20347] Updated weights for policy 0, policy_version 1433612 (0.0008) [2023-12-27 15:34:50,701][20347] Updated weights for policy 0, policy_version 1433622 (0.0007) [2023-12-27 15:34:50,752][20347] Updated weights for policy 0, policy_version 1433632 (0.0007) [2023-12-27 15:34:51,196][20381] Updated weights for policy 1, policy_version 1432341 (0.0008) [2023-12-27 15:34:51,250][20381] Updated weights for policy 1, policy_version 1432351 (0.0008) [2023-12-27 15:34:51,309][20381] Updated weights for policy 1, policy_version 1432361 (0.0007) [2023-12-27 15:34:51,507][20347] Updated weights for policy 0, policy_version 1433642 (0.0007) [2023-12-27 15:34:51,559][20347] Updated weights for policy 0, policy_version 1433652 (0.0007) [2023-12-27 15:34:51,611][20347] Updated weights for policy 0, policy_version 1433662 (0.0008) [2023-12-27 15:34:51,673][20347] Updated weights for policy 0, policy_version 1433672 (0.0008) [2023-12-27 15:34:52,078][20381] Updated weights for policy 1, policy_version 1432371 (0.0007) [2023-12-27 15:34:52,137][20381] Updated weights for policy 1, policy_version 1432381 (0.0008) [2023-12-27 15:34:52,195][20381] Updated weights for policy 1, policy_version 1432391 (0.0007) [2023-12-27 15:34:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 733814784. Throughput: 0: 10309.5, 1: 10248.1. Samples: 733787024. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:52,363][19170] Avg episode reward: [(0, '980.450'), (1, '734.930')] [2023-12-27 15:34:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001432400_366723072.pth... [2023-12-27 15:34:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001431184_366411776.pth [2023-12-27 15:34:52,417][20347] Updated weights for policy 0, policy_version 1433682 (0.0008) [2023-12-27 15:34:52,473][20347] Updated weights for policy 0, policy_version 1433692 (0.0008) [2023-12-27 15:34:52,528][20347] Updated weights for policy 0, policy_version 1433702 (0.0010) [2023-12-27 15:34:52,536][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001433704_367099904.pth... [2023-12-27 15:34:52,539][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001432488_366788608.pth [2023-12-27 15:34:52,921][20381] Updated weights for policy 1, policy_version 1432401 (0.0008) [2023-12-27 15:34:52,975][20381] Updated weights for policy 1, policy_version 1432411 (0.0007) [2023-12-27 15:34:53,023][20381] Updated weights for policy 1, policy_version 1432421 (0.0007) [2023-12-27 15:34:53,074][20381] Updated weights for policy 1, policy_version 1432431 (0.0009) [2023-12-27 15:34:53,204][20347] Updated weights for policy 0, policy_version 1433712 (0.0010) [2023-12-27 15:34:53,254][20347] Updated weights for policy 0, policy_version 1433722 (0.0010) [2023-12-27 15:34:53,308][20347] Updated weights for policy 0, policy_version 1433732 (0.0010) [2023-12-27 15:34:53,745][20381] Updated weights for policy 1, policy_version 1432441 (0.0005) [2023-12-27 15:34:53,801][20381] Updated weights for policy 1, policy_version 1432451 (0.0005) [2023-12-27 15:34:53,861][20381] Updated weights for policy 1, policy_version 1432461 (0.0008) [2023-12-27 15:34:53,956][20347] Updated weights for policy 0, policy_version 1433742 (0.0007) [2023-12-27 15:34:54,010][20347] Updated weights for policy 0, policy_version 1433752 (0.0005) [2023-12-27 15:34:54,063][20347] Updated weights for policy 0, policy_version 1433762 (0.0005) [2023-12-27 15:34:54,440][20381] Updated weights for policy 1, policy_version 1432471 (0.0007) [2023-12-27 15:34:54,499][20381] Updated weights for policy 1, policy_version 1432481 (0.0010) [2023-12-27 15:34:54,560][20381] Updated weights for policy 1, policy_version 1432491 (0.0011) [2023-12-27 15:34:54,638][20347] Updated weights for policy 0, policy_version 1433772 (0.0005) [2023-12-27 15:34:54,691][20347] Updated weights for policy 0, policy_version 1433782 (0.0005) [2023-12-27 15:34:54,752][20347] Updated weights for policy 0, policy_version 1433792 (0.0010) [2023-12-27 15:34:55,176][20381] Updated weights for policy 1, policy_version 1432501 (0.0010) [2023-12-27 15:34:55,234][20381] Updated weights for policy 1, policy_version 1432511 (0.0006) [2023-12-27 15:34:55,298][20381] Updated weights for policy 1, policy_version 1432521 (0.0007) [2023-12-27 15:34:55,384][20347] Updated weights for policy 0, policy_version 1433802 (0.0010) [2023-12-27 15:34:55,445][20347] Updated weights for policy 0, policy_version 1433812 (0.0008) [2023-12-27 15:34:55,504][20347] Updated weights for policy 0, policy_version 1433822 (0.0008) [2023-12-27 15:34:55,561][20347] Updated weights for policy 0, policy_version 1433832 (0.0007) [2023-12-27 15:34:55,898][20381] Updated weights for policy 1, policy_version 1432531 (0.0008) [2023-12-27 15:34:55,952][20381] Updated weights for policy 1, policy_version 1432541 (0.0010) [2023-12-27 15:34:56,013][20381] Updated weights for policy 1, policy_version 1432551 (0.0011) [2023-12-27 15:34:56,254][20347] Updated weights for policy 0, policy_version 1433842 (0.0010) [2023-12-27 15:34:56,304][20347] Updated weights for policy 0, policy_version 1433852 (0.0010) [2023-12-27 15:34:56,354][20347] Updated weights for policy 0, policy_version 1433862 (0.0010) [2023-12-27 15:34:56,699][20381] Updated weights for policy 1, policy_version 1432561 (0.0011) [2023-12-27 15:34:56,753][20381] Updated weights for policy 1, policy_version 1432571 (0.0011) [2023-12-27 15:34:56,798][20381] Updated weights for policy 1, policy_version 1432581 (0.0010) [2023-12-27 15:34:56,843][20381] Updated weights for policy 1, policy_version 1432591 (0.0010) [2023-12-27 15:34:56,995][20347] Updated weights for policy 0, policy_version 1433872 (0.0006) [2023-12-27 15:34:57,058][20347] Updated weights for policy 0, policy_version 1433882 (0.0009) [2023-12-27 15:34:57,118][20347] Updated weights for policy 0, policy_version 1433892 (0.0010) [2023-12-27 15:34:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 733921280. Throughput: 0: 10285.3, 1: 10263.2. Samples: 733909656. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:34:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '691.140')] [2023-12-27 15:34:57,546][20381] Updated weights for policy 1, policy_version 1432601 (0.0010) [2023-12-27 15:34:57,590][20381] Updated weights for policy 1, policy_version 1432611 (0.0009) [2023-12-27 15:34:57,647][20381] Updated weights for policy 1, policy_version 1432621 (0.0005) [2023-12-27 15:34:57,709][20347] Updated weights for policy 0, policy_version 1433902 (0.0010) [2023-12-27 15:34:57,756][20347] Updated weights for policy 0, policy_version 1433912 (0.0010) [2023-12-27 15:34:57,804][20347] Updated weights for policy 0, policy_version 1433922 (0.0010) [2023-12-27 15:34:58,284][20381] Updated weights for policy 1, policy_version 1432631 (0.0009) [2023-12-27 15:34:58,355][20381] Updated weights for policy 1, policy_version 1432641 (0.0010) [2023-12-27 15:34:58,415][20381] Updated weights for policy 1, policy_version 1432651 (0.0010) [2023-12-27 15:34:58,499][20347] Updated weights for policy 0, policy_version 1433932 (0.0008) [2023-12-27 15:34:58,562][20347] Updated weights for policy 0, policy_version 1433942 (0.0007) [2023-12-27 15:34:58,622][20347] Updated weights for policy 0, policy_version 1433952 (0.0008) [2023-12-27 15:34:59,246][20381] Updated weights for policy 1, policy_version 1432661 (0.0013) [2023-12-27 15:34:59,307][20381] Updated weights for policy 1, policy_version 1432671 (0.0009) [2023-12-27 15:34:59,374][20381] Updated weights for policy 1, policy_version 1432682 (0.0009) [2023-12-27 15:34:59,394][20347] Updated weights for policy 0, policy_version 1433962 (0.0007) [2023-12-27 15:34:59,452][20347] Updated weights for policy 0, policy_version 1433972 (0.0008) [2023-12-27 15:34:59,509][20347] Updated weights for policy 0, policy_version 1433982 (0.0006) [2023-12-27 15:34:59,567][20347] Updated weights for policy 0, policy_version 1433992 (0.0005) [2023-12-27 15:35:00,056][20381] Updated weights for policy 1, policy_version 1432692 (0.0012) [2023-12-27 15:35:00,122][20381] Updated weights for policy 1, policy_version 1432702 (0.0009) [2023-12-27 15:35:00,185][20381] Updated weights for policy 1, policy_version 1432712 (0.0008) [2023-12-27 15:35:00,190][20347] Updated weights for policy 0, policy_version 1434002 (0.0007) [2023-12-27 15:35:00,243][20347] Updated weights for policy 0, policy_version 1434012 (0.0006) [2023-12-27 15:35:00,291][20347] Updated weights for policy 0, policy_version 1434022 (0.0008) [2023-12-27 15:35:00,794][20381] Updated weights for policy 1, policy_version 1432722 (0.0006) [2023-12-27 15:35:00,853][20381] Updated weights for policy 1, policy_version 1432732 (0.0008) [2023-12-27 15:35:00,905][20381] Updated weights for policy 1, policy_version 1432742 (0.0008) [2023-12-27 15:35:00,943][20347] Updated weights for policy 0, policy_version 1434032 (0.0006) [2023-12-27 15:35:00,960][20381] Updated weights for policy 1, policy_version 1432752 (0.0007) [2023-12-27 15:35:01,002][20347] Updated weights for policy 0, policy_version 1434042 (0.0006) [2023-12-27 15:35:01,063][20347] Updated weights for policy 0, policy_version 1434052 (0.0010) [2023-12-27 15:35:01,631][20381] Updated weights for policy 1, policy_version 1432762 (0.0007) [2023-12-27 15:35:01,698][20381] Updated weights for policy 1, policy_version 1432773 (0.0008) [2023-12-27 15:35:01,749][20347] Updated weights for policy 0, policy_version 1434062 (0.0008) [2023-12-27 15:35:01,758][20381] Updated weights for policy 1, policy_version 1432783 (0.0010) [2023-12-27 15:35:01,803][20347] Updated weights for policy 0, policy_version 1434072 (0.0007) [2023-12-27 15:35:01,855][20347] Updated weights for policy 0, policy_version 1434082 (0.0008) [2023-12-27 15:35:02,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 734019584. Throughput: 0: 10297.8, 1: 10221.0. Samples: 734032832. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:02,363][19170] Avg episode reward: [(0, '980.480'), (1, '675.650')] [2023-12-27 15:35:02,448][20381] Updated weights for policy 1, policy_version 1432793 (0.0008) [2023-12-27 15:35:02,503][20381] Updated weights for policy 1, policy_version 1432803 (0.0011) [2023-12-27 15:35:02,536][20347] Updated weights for policy 0, policy_version 1434092 (0.0009) [2023-12-27 15:35:02,566][20381] Updated weights for policy 1, policy_version 1432813 (0.0010) [2023-12-27 15:35:02,595][20347] Updated weights for policy 0, policy_version 1434102 (0.0010) [2023-12-27 15:35:02,657][20347] Updated weights for policy 0, policy_version 1434112 (0.0010) [2023-12-27 15:35:03,253][20381] Updated weights for policy 1, policy_version 1432823 (0.0008) [2023-12-27 15:35:03,311][20381] Updated weights for policy 1, policy_version 1432833 (0.0006) [2023-12-27 15:35:03,345][20347] Updated weights for policy 0, policy_version 1434122 (0.0009) [2023-12-27 15:35:03,367][20381] Updated weights for policy 1, policy_version 1432843 (0.0006) [2023-12-27 15:35:03,400][20347] Updated weights for policy 0, policy_version 1434132 (0.0005) [2023-12-27 15:35:03,466][20347] Updated weights for policy 0, policy_version 1434142 (0.0005) [2023-12-27 15:35:03,526][20347] Updated weights for policy 0, policy_version 1434152 (0.0005) [2023-12-27 15:35:04,063][20381] Updated weights for policy 1, policy_version 1432853 (0.0006) [2023-12-27 15:35:04,123][20381] Updated weights for policy 1, policy_version 1432863 (0.0007) [2023-12-27 15:35:04,134][20347] Updated weights for policy 0, policy_version 1434162 (0.0006) [2023-12-27 15:35:04,186][20381] Updated weights for policy 1, policy_version 1432873 (0.0005) [2023-12-27 15:35:04,200][20347] Updated weights for policy 0, policy_version 1434172 (0.0007) [2023-12-27 15:35:04,255][20347] Updated weights for policy 0, policy_version 1434182 (0.0006) [2023-12-27 15:35:04,808][20381] Updated weights for policy 1, policy_version 1432883 (0.0009) [2023-12-27 15:35:04,858][20381] Updated weights for policy 1, policy_version 1432893 (0.0007) [2023-12-27 15:35:04,859][20347] Updated weights for policy 0, policy_version 1434192 (0.0007) [2023-12-27 15:35:04,911][20381] Updated weights for policy 1, policy_version 1432903 (0.0007) [2023-12-27 15:35:04,913][20347] Updated weights for policy 0, policy_version 1434202 (0.0007) [2023-12-27 15:35:04,971][20347] Updated weights for policy 0, policy_version 1434212 (0.0006) [2023-12-27 15:35:05,581][20381] Updated weights for policy 1, policy_version 1432913 (0.0007) [2023-12-27 15:35:05,614][20347] Updated weights for policy 0, policy_version 1434222 (0.0007) [2023-12-27 15:35:05,636][20381] Updated weights for policy 1, policy_version 1432923 (0.0005) [2023-12-27 15:35:05,663][20347] Updated weights for policy 0, policy_version 1434232 (0.0007) [2023-12-27 15:35:05,697][20381] Updated weights for policy 1, policy_version 1432933 (0.0005) [2023-12-27 15:35:05,719][20347] Updated weights for policy 0, policy_version 1434242 (0.0007) [2023-12-27 15:35:05,754][20381] Updated weights for policy 1, policy_version 1432943 (0.0009) [2023-12-27 15:35:06,453][20347] Updated weights for policy 0, policy_version 1434252 (0.0007) [2023-12-27 15:35:06,474][20381] Updated weights for policy 1, policy_version 1432953 (0.0007) [2023-12-27 15:35:06,510][20347] Updated weights for policy 0, policy_version 1434262 (0.0006) [2023-12-27 15:35:06,528][20381] Updated weights for policy 1, policy_version 1432963 (0.0006) [2023-12-27 15:35:06,570][20347] Updated weights for policy 0, policy_version 1434272 (0.0007) [2023-12-27 15:35:06,588][20381] Updated weights for policy 1, policy_version 1432973 (0.0006) [2023-12-27 15:35:07,255][20347] Updated weights for policy 0, policy_version 1434282 (0.0009) [2023-12-27 15:35:07,284][20381] Updated weights for policy 1, policy_version 1432983 (0.0006) [2023-12-27 15:35:07,307][20347] Updated weights for policy 0, policy_version 1434292 (0.0008) [2023-12-27 15:35:07,338][20381] Updated weights for policy 1, policy_version 1432993 (0.0007) [2023-12-27 15:35:07,363][20347] Updated weights for policy 0, policy_version 1434302 (0.0006) [2023-12-27 15:35:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 734117888. Throughput: 0: 10358.5, 1: 10209.1. Samples: 734095496. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:07,363][19170] Avg episode reward: [(0, '941.550'), (1, '738.990')] [2023-12-27 15:35:07,391][20381] Updated weights for policy 1, policy_version 1433003 (0.0006) [2023-12-27 15:35:07,416][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001433008_366878720.pth... [2023-12-27 15:35:07,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001434312_367255552.pth... [2023-12-27 15:35:07,420][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001431792_366567424.pth [2023-12-27 15:35:07,422][20347] Updated weights for policy 0, policy_version 1434312 (0.0006) [2023-12-27 15:35:07,424][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001433096_366944256.pth [2023-12-27 15:35:07,982][20347] Updated weights for policy 0, policy_version 1434322 (0.0007) [2023-12-27 15:35:08,039][20347] Updated weights for policy 0, policy_version 1434332 (0.0006) [2023-12-27 15:35:08,050][20381] Updated weights for policy 1, policy_version 1433013 (0.0006) [2023-12-27 15:35:08,095][20347] Updated weights for policy 0, policy_version 1434342 (0.0007) [2023-12-27 15:35:08,110][20381] Updated weights for policy 1, policy_version 1433023 (0.0005) [2023-12-27 15:35:08,171][20381] Updated weights for policy 1, policy_version 1433033 (0.0006) [2023-12-27 15:35:08,768][20347] Updated weights for policy 0, policy_version 1434352 (0.0006) [2023-12-27 15:35:08,834][20347] Updated weights for policy 0, policy_version 1434362 (0.0010) [2023-12-27 15:35:08,872][20381] Updated weights for policy 1, policy_version 1433043 (0.0008) [2023-12-27 15:35:08,892][20347] Updated weights for policy 0, policy_version 1434372 (0.0005) [2023-12-27 15:35:08,926][20381] Updated weights for policy 1, policy_version 1433053 (0.0008) [2023-12-27 15:35:08,994][20381] Updated weights for policy 1, policy_version 1433063 (0.0007) [2023-12-27 15:35:09,571][20347] Updated weights for policy 0, policy_version 1434382 (0.0007) [2023-12-27 15:35:09,623][20347] Updated weights for policy 0, policy_version 1434392 (0.0007) [2023-12-27 15:35:09,684][20347] Updated weights for policy 0, policy_version 1434402 (0.0008) [2023-12-27 15:35:09,707][20381] Updated weights for policy 1, policy_version 1433073 (0.0007) [2023-12-27 15:35:09,765][20381] Updated weights for policy 1, policy_version 1433083 (0.0008) [2023-12-27 15:35:09,820][20381] Updated weights for policy 1, policy_version 1433093 (0.0008) [2023-12-27 15:35:09,885][20381] Updated weights for policy 1, policy_version 1433103 (0.0007) [2023-12-27 15:35:10,447][20347] Updated weights for policy 0, policy_version 1434412 (0.0008) [2023-12-27 15:35:10,509][20347] Updated weights for policy 0, policy_version 1434422 (0.0008) [2023-12-27 15:35:10,523][20381] Updated weights for policy 1, policy_version 1433113 (0.0008) [2023-12-27 15:35:10,568][20347] Updated weights for policy 0, policy_version 1434432 (0.0007) [2023-12-27 15:35:10,574][20381] Updated weights for policy 1, policy_version 1433123 (0.0006) [2023-12-27 15:35:10,628][20381] Updated weights for policy 1, policy_version 1433133 (0.0006) [2023-12-27 15:35:11,196][20347] Updated weights for policy 0, policy_version 1434442 (0.0008) [2023-12-27 15:35:11,258][20347] Updated weights for policy 0, policy_version 1434452 (0.0010) [2023-12-27 15:35:11,305][20381] Updated weights for policy 1, policy_version 1433143 (0.0007) [2023-12-27 15:35:11,316][20347] Updated weights for policy 0, policy_version 1434462 (0.0008) [2023-12-27 15:35:11,381][20381] Updated weights for policy 1, policy_version 1433153 (0.0012) [2023-12-27 15:35:11,387][20347] Updated weights for policy 0, policy_version 1434472 (0.0007) [2023-12-27 15:35:11,446][20381] Updated weights for policy 1, policy_version 1433163 (0.0013) [2023-12-27 15:35:12,131][20381] Updated weights for policy 1, policy_version 1433173 (0.0009) [2023-12-27 15:35:12,168][20347] Updated weights for policy 0, policy_version 1434482 (0.0006) [2023-12-27 15:35:12,189][20381] Updated weights for policy 1, policy_version 1433183 (0.0006) [2023-12-27 15:35:12,218][20347] Updated weights for policy 0, policy_version 1434492 (0.0006) [2023-12-27 15:35:12,244][20381] Updated weights for policy 1, policy_version 1433193 (0.0006) [2023-12-27 15:35:12,276][20347] Updated weights for policy 0, policy_version 1434502 (0.0006) [2023-12-27 15:35:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 734232576. Throughput: 0: 10372.6, 1: 10254.5. Samples: 734219416. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:12,363][19170] Avg episode reward: [(0, '951.280'), (1, '767.840')] [2023-12-27 15:35:12,942][20347] Updated weights for policy 0, policy_version 1434512 (0.0007) [2023-12-27 15:35:12,989][20347] Updated weights for policy 0, policy_version 1434522 (0.0007) [2023-12-27 15:35:12,994][20381] Updated weights for policy 1, policy_version 1433203 (0.0008) [2023-12-27 15:35:13,040][20347] Updated weights for policy 0, policy_version 1434532 (0.0007) [2023-12-27 15:35:13,052][20381] Updated weights for policy 1, policy_version 1433213 (0.0006) [2023-12-27 15:35:13,099][20381] Updated weights for policy 1, policy_version 1433223 (0.0008) [2023-12-27 15:35:13,743][20347] Updated weights for policy 0, policy_version 1434542 (0.0007) [2023-12-27 15:35:13,775][20381] Updated weights for policy 1, policy_version 1433233 (0.0008) [2023-12-27 15:35:13,797][20347] Updated weights for policy 0, policy_version 1434552 (0.0007) [2023-12-27 15:35:13,825][20381] Updated weights for policy 1, policy_version 1433243 (0.0008) [2023-12-27 15:35:13,849][20347] Updated weights for policy 0, policy_version 1434562 (0.0006) [2023-12-27 15:35:13,880][20381] Updated weights for policy 1, policy_version 1433253 (0.0007) [2023-12-27 15:35:13,938][20381] Updated weights for policy 1, policy_version 1433263 (0.0005) [2023-12-27 15:35:14,498][20347] Updated weights for policy 0, policy_version 1434572 (0.0006) [2023-12-27 15:35:14,536][20381] Updated weights for policy 1, policy_version 1433273 (0.0010) [2023-12-27 15:35:14,546][20347] Updated weights for policy 0, policy_version 1434582 (0.0008) [2023-12-27 15:35:14,587][20381] Updated weights for policy 1, policy_version 1433283 (0.0010) [2023-12-27 15:35:14,598][20347] Updated weights for policy 0, policy_version 1434592 (0.0009) [2023-12-27 15:35:14,637][20381] Updated weights for policy 1, policy_version 1433293 (0.0009) [2023-12-27 15:35:15,309][20347] Updated weights for policy 0, policy_version 1434602 (0.0008) [2023-12-27 15:35:15,326][20381] Updated weights for policy 1, policy_version 1433303 (0.0009) [2023-12-27 15:35:15,370][20347] Updated weights for policy 0, policy_version 1434612 (0.0005) [2023-12-27 15:35:15,381][20381] Updated weights for policy 1, policy_version 1433313 (0.0011) [2023-12-27 15:35:15,427][20347] Updated weights for policy 0, policy_version 1434622 (0.0005) [2023-12-27 15:35:15,439][20381] Updated weights for policy 1, policy_version 1433323 (0.0010) [2023-12-27 15:35:15,481][20347] Updated weights for policy 0, policy_version 1434632 (0.0005) [2023-12-27 15:35:16,033][20347] Updated weights for policy 0, policy_version 1434642 (0.0006) [2023-12-27 15:35:16,091][20347] Updated weights for policy 0, policy_version 1434652 (0.0010) [2023-12-27 15:35:16,144][20381] Updated weights for policy 1, policy_version 1433333 (0.0008) [2023-12-27 15:35:16,149][20347] Updated weights for policy 0, policy_version 1434662 (0.0010) [2023-12-27 15:35:16,199][20381] Updated weights for policy 1, policy_version 1433343 (0.0010) [2023-12-27 15:35:16,266][20381] Updated weights for policy 1, policy_version 1433353 (0.0010) [2023-12-27 15:35:16,786][20347] Updated weights for policy 0, policy_version 1434672 (0.0008) [2023-12-27 15:35:16,845][20347] Updated weights for policy 0, policy_version 1434682 (0.0007) [2023-12-27 15:35:16,913][20381] Updated weights for policy 1, policy_version 1433363 (0.0009) [2023-12-27 15:35:16,915][20347] Updated weights for policy 0, policy_version 1434692 (0.0006) [2023-12-27 15:35:16,962][20381] Updated weights for policy 1, policy_version 1433373 (0.0008) [2023-12-27 15:35:17,009][20381] Updated weights for policy 1, policy_version 1433383 (0.0007) [2023-12-27 15:35:17,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 734330880. Throughput: 0: 10337.8, 1: 10181.6. Samples: 734341488. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:17,364][19170] Avg episode reward: [(0, '970.660'), (1, '778.860')] [2023-12-27 15:35:17,493][20347] Updated weights for policy 0, policy_version 1434702 (0.0006) [2023-12-27 15:35:17,549][20347] Updated weights for policy 0, policy_version 1434712 (0.0007) [2023-12-27 15:35:17,606][20347] Updated weights for policy 0, policy_version 1434722 (0.0010) [2023-12-27 15:35:17,660][20381] Updated weights for policy 1, policy_version 1433393 (0.0007) [2023-12-27 15:35:17,712][20381] Updated weights for policy 1, policy_version 1433403 (0.0009) [2023-12-27 15:35:17,775][20381] Updated weights for policy 1, policy_version 1433413 (0.0011) [2023-12-27 15:35:17,829][20381] Updated weights for policy 1, policy_version 1433423 (0.0010) [2023-12-27 15:35:18,232][20347] Updated weights for policy 0, policy_version 1434732 (0.0010) [2023-12-27 15:35:18,290][20347] Updated weights for policy 0, policy_version 1434742 (0.0010) [2023-12-27 15:35:18,356][20347] Updated weights for policy 0, policy_version 1434752 (0.0008) [2023-12-27 15:35:18,501][20381] Updated weights for policy 1, policy_version 1433433 (0.0007) [2023-12-27 15:35:18,559][20381] Updated weights for policy 1, policy_version 1433443 (0.0007) [2023-12-27 15:35:18,619][20381] Updated weights for policy 1, policy_version 1433453 (0.0007) [2023-12-27 15:35:19,022][20347] Updated weights for policy 0, policy_version 1434762 (0.0008) [2023-12-27 15:35:19,074][20347] Updated weights for policy 0, policy_version 1434772 (0.0010) [2023-12-27 15:35:19,126][20347] Updated weights for policy 0, policy_version 1434782 (0.0010) [2023-12-27 15:35:19,175][20347] Updated weights for policy 0, policy_version 1434792 (0.0010) [2023-12-27 15:35:19,301][20381] Updated weights for policy 1, policy_version 1433463 (0.0010) [2023-12-27 15:35:19,363][20381] Updated weights for policy 1, policy_version 1433473 (0.0009) [2023-12-27 15:35:19,430][20381] Updated weights for policy 1, policy_version 1433483 (0.0011) [2023-12-27 15:35:19,895][20347] Updated weights for policy 0, policy_version 1434802 (0.0009) [2023-12-27 15:35:19,957][20347] Updated weights for policy 0, policy_version 1434812 (0.0010) [2023-12-27 15:35:20,018][20347] Updated weights for policy 0, policy_version 1434822 (0.0011) [2023-12-27 15:35:20,082][20381] Updated weights for policy 1, policy_version 1433493 (0.0009) [2023-12-27 15:35:20,142][20381] Updated weights for policy 1, policy_version 1433503 (0.0008) [2023-12-27 15:35:20,203][20381] Updated weights for policy 1, policy_version 1433513 (0.0007) [2023-12-27 15:35:20,752][20347] Updated weights for policy 0, policy_version 1434832 (0.0008) [2023-12-27 15:35:20,821][20347] Updated weights for policy 0, policy_version 1434842 (0.0008) [2023-12-27 15:35:20,889][20347] Updated weights for policy 0, policy_version 1434852 (0.0008) [2023-12-27 15:35:20,896][20381] Updated weights for policy 1, policy_version 1433523 (0.0006) [2023-12-27 15:35:20,953][20381] Updated weights for policy 1, policy_version 1433533 (0.0006) [2023-12-27 15:35:21,022][20381] Updated weights for policy 1, policy_version 1433543 (0.0007) [2023-12-27 15:35:21,582][20347] Updated weights for policy 0, policy_version 1434862 (0.0006) [2023-12-27 15:35:21,601][20291] KL-divergence is very high: 116.5238 [2023-12-27 15:35:21,644][20347] Updated weights for policy 0, policy_version 1434872 (0.0008) [2023-12-27 15:35:21,711][20347] Updated weights for policy 0, policy_version 1434882 (0.0006) [2023-12-27 15:35:21,847][20381] Updated weights for policy 1, policy_version 1433553 (0.0008) [2023-12-27 15:35:21,909][20381] Updated weights for policy 1, policy_version 1433563 (0.0008) [2023-12-27 15:35:21,970][20381] Updated weights for policy 1, policy_version 1433573 (0.0008) [2023-12-27 15:35:22,029][20381] Updated weights for policy 1, policy_version 1433583 (0.0008) [2023-12-27 15:35:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 734429184. Throughput: 0: 10287.7, 1: 10227.1. Samples: 734404032. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:22,363][19170] Avg episode reward: [(0, '941.530'), (1, '770.520')] [2023-12-27 15:35:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001433584_367026176.pth... [2023-12-27 15:35:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001434888_367403008.pth... [2023-12-27 15:35:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001432400_366723072.pth [2023-12-27 15:35:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001433704_367099904.pth [2023-12-27 15:35:22,411][20347] Updated weights for policy 0, policy_version 1434892 (0.0008) [2023-12-27 15:35:22,474][20347] Updated weights for policy 0, policy_version 1434902 (0.0008) [2023-12-27 15:35:22,528][20347] Updated weights for policy 0, policy_version 1434912 (0.0008) [2023-12-27 15:35:22,742][20381] Updated weights for policy 1, policy_version 1433593 (0.0008) [2023-12-27 15:35:22,804][20381] Updated weights for policy 1, policy_version 1433603 (0.0008) [2023-12-27 15:35:22,861][20381] Updated weights for policy 1, policy_version 1433613 (0.0008) [2023-12-27 15:35:23,228][20347] Updated weights for policy 0, policy_version 1434922 (0.0008) [2023-12-27 15:35:23,290][20347] Updated weights for policy 0, policy_version 1434932 (0.0008) [2023-12-27 15:35:23,352][20347] Updated weights for policy 0, policy_version 1434942 (0.0008) [2023-12-27 15:35:23,413][20347] Updated weights for policy 0, policy_version 1434952 (0.0007) [2023-12-27 15:35:23,555][20381] Updated weights for policy 1, policy_version 1433623 (0.0006) [2023-12-27 15:35:23,612][20381] Updated weights for policy 1, policy_version 1433633 (0.0005) [2023-12-27 15:35:23,672][20381] Updated weights for policy 1, policy_version 1433643 (0.0006) [2023-12-27 15:35:24,140][20347] Updated weights for policy 0, policy_version 1434962 (0.0005) [2023-12-27 15:35:24,200][20347] Updated weights for policy 0, policy_version 1434972 (0.0006) [2023-12-27 15:35:24,232][20381] Updated weights for policy 1, policy_version 1433653 (0.0005) [2023-12-27 15:35:24,257][20347] Updated weights for policy 0, policy_version 1434982 (0.0007) [2023-12-27 15:35:24,286][20381] Updated weights for policy 1, policy_version 1433663 (0.0006) [2023-12-27 15:35:24,342][20381] Updated weights for policy 1, policy_version 1433673 (0.0008) [2023-12-27 15:35:24,927][20347] Updated weights for policy 0, policy_version 1434992 (0.0007) [2023-12-27 15:35:24,941][20381] Updated weights for policy 1, policy_version 1433683 (0.0007) [2023-12-27 15:35:24,979][20347] Updated weights for policy 0, policy_version 1435002 (0.0005) [2023-12-27 15:35:24,989][20381] Updated weights for policy 1, policy_version 1433693 (0.0005) [2023-12-27 15:35:25,026][20347] Updated weights for policy 0, policy_version 1435012 (0.0005) [2023-12-27 15:35:25,035][20381] Updated weights for policy 1, policy_version 1433703 (0.0007) [2023-12-27 15:35:25,664][20381] Updated weights for policy 1, policy_version 1433713 (0.0007) [2023-12-27 15:35:25,722][20381] Updated weights for policy 1, policy_version 1433723 (0.0007) [2023-12-27 15:35:25,738][20347] Updated weights for policy 0, policy_version 1435022 (0.0005) [2023-12-27 15:35:25,781][20381] Updated weights for policy 1, policy_version 1433733 (0.0007) [2023-12-27 15:35:25,799][20347] Updated weights for policy 0, policy_version 1435032 (0.0006) [2023-12-27 15:35:25,831][20381] Updated weights for policy 1, policy_version 1433743 (0.0006) [2023-12-27 15:35:25,855][20347] Updated weights for policy 0, policy_version 1435042 (0.0007) [2023-12-27 15:35:26,507][20381] Updated weights for policy 1, policy_version 1433753 (0.0005) [2023-12-27 15:35:26,531][20347] Updated weights for policy 0, policy_version 1435052 (0.0007) [2023-12-27 15:35:26,557][20381] Updated weights for policy 1, policy_version 1433763 (0.0005) [2023-12-27 15:35:26,584][20347] Updated weights for policy 0, policy_version 1435062 (0.0006) [2023-12-27 15:35:26,602][20381] Updated weights for policy 1, policy_version 1433773 (0.0005) [2023-12-27 15:35:26,633][20347] Updated weights for policy 0, policy_version 1435072 (0.0006) [2023-12-27 15:35:27,193][20381] Updated weights for policy 1, policy_version 1433783 (0.0006) [2023-12-27 15:35:27,228][20347] Updated weights for policy 0, policy_version 1435082 (0.0005) [2023-12-27 15:35:27,253][20381] Updated weights for policy 1, policy_version 1433793 (0.0008) [2023-12-27 15:35:27,279][20347] Updated weights for policy 0, policy_version 1435092 (0.0005) [2023-12-27 15:35:27,301][20381] Updated weights for policy 1, policy_version 1433803 (0.0006) [2023-12-27 15:35:27,327][20347] Updated weights for policy 0, policy_version 1435102 (0.0007) [2023-12-27 15:35:27,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 734535680. Throughput: 0: 10229.8, 1: 10304.1. Samples: 734524700. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:27,364][19170] Avg episode reward: [(0, '951.320'), (1, '732.380')] [2023-12-27 15:35:27,382][20347] Updated weights for policy 0, policy_version 1435112 (0.0007) [2023-12-27 15:35:27,969][20381] Updated weights for policy 1, policy_version 1433813 (0.0007) [2023-12-27 15:35:28,024][20381] Updated weights for policy 1, policy_version 1433823 (0.0007) [2023-12-27 15:35:28,046][20347] Updated weights for policy 0, policy_version 1435122 (0.0006) [2023-12-27 15:35:28,080][20381] Updated weights for policy 1, policy_version 1433833 (0.0008) [2023-12-27 15:35:28,097][20347] Updated weights for policy 0, policy_version 1435132 (0.0006) [2023-12-27 15:35:28,151][20347] Updated weights for policy 0, policy_version 1435142 (0.0007) [2023-12-27 15:35:28,715][20381] Updated weights for policy 1, policy_version 1433843 (0.0007) [2023-12-27 15:35:28,772][20381] Updated weights for policy 1, policy_version 1433853 (0.0008) [2023-12-27 15:35:28,833][20381] Updated weights for policy 1, policy_version 1433863 (0.0008) [2023-12-27 15:35:28,873][20347] Updated weights for policy 0, policy_version 1435152 (0.0007) [2023-12-27 15:35:28,934][20347] Updated weights for policy 0, policy_version 1435162 (0.0007) [2023-12-27 15:35:28,996][20347] Updated weights for policy 0, policy_version 1435172 (0.0006) [2023-12-27 15:35:29,478][20381] Updated weights for policy 1, policy_version 1433873 (0.0007) [2023-12-27 15:35:29,537][20381] Updated weights for policy 1, policy_version 1433883 (0.0007) [2023-12-27 15:35:29,586][20381] Updated weights for policy 1, policy_version 1433893 (0.0005) [2023-12-27 15:35:29,639][20381] Updated weights for policy 1, policy_version 1433903 (0.0008) [2023-12-27 15:35:29,673][20347] Updated weights for policy 0, policy_version 1435182 (0.0008) [2023-12-27 15:35:29,723][20347] Updated weights for policy 0, policy_version 1435192 (0.0008) [2023-12-27 15:35:29,775][20347] Updated weights for policy 0, policy_version 1435202 (0.0009) [2023-12-27 15:35:30,325][20381] Updated weights for policy 1, policy_version 1433913 (0.0008) [2023-12-27 15:35:30,377][20381] Updated weights for policy 1, policy_version 1433923 (0.0008) [2023-12-27 15:35:30,431][20381] Updated weights for policy 1, policy_version 1433933 (0.0008) [2023-12-27 15:35:30,481][20347] Updated weights for policy 0, policy_version 1435212 (0.0008) [2023-12-27 15:35:30,534][20347] Updated weights for policy 0, policy_version 1435222 (0.0008) [2023-12-27 15:35:30,579][20347] Updated weights for policy 0, policy_version 1435232 (0.0005) [2023-12-27 15:35:31,020][20381] Updated weights for policy 1, policy_version 1433943 (0.0007) [2023-12-27 15:35:31,087][20381] Updated weights for policy 1, policy_version 1433953 (0.0007) [2023-12-27 15:35:31,152][20381] Updated weights for policy 1, policy_version 1433963 (0.0007) [2023-12-27 15:35:31,177][20347] Updated weights for policy 0, policy_version 1435242 (0.0007) [2023-12-27 15:35:31,229][20347] Updated weights for policy 0, policy_version 1435252 (0.0010) [2023-12-27 15:35:31,290][20347] Updated weights for policy 0, policy_version 1435262 (0.0009) [2023-12-27 15:35:31,349][20347] Updated weights for policy 0, policy_version 1435272 (0.0009) [2023-12-27 15:35:31,832][20381] Updated weights for policy 1, policy_version 1433973 (0.0007) [2023-12-27 15:35:31,885][20381] Updated weights for policy 1, policy_version 1433983 (0.0008) [2023-12-27 15:35:31,943][20381] Updated weights for policy 1, policy_version 1433993 (0.0007) [2023-12-27 15:35:32,037][20347] Updated weights for policy 0, policy_version 1435282 (0.0005) [2023-12-27 15:35:32,097][20347] Updated weights for policy 0, policy_version 1435292 (0.0005) [2023-12-27 15:35:32,146][20347] Updated weights for policy 0, policy_version 1435302 (0.0010) [2023-12-27 15:35:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 734642176. Throughput: 0: 10261.4, 1: 10284.4. Samples: 734650336. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:32,363][19170] Avg episode reward: [(0, '990.240'), (1, '693.760')] [2023-12-27 15:35:32,678][20381] Updated weights for policy 1, policy_version 1434003 (0.0008) [2023-12-27 15:35:32,743][20381] Updated weights for policy 1, policy_version 1434013 (0.0008) [2023-12-27 15:35:32,801][20381] Updated weights for policy 1, policy_version 1434023 (0.0006) [2023-12-27 15:35:32,807][20347] Updated weights for policy 0, policy_version 1435312 (0.0010) [2023-12-27 15:35:32,859][20347] Updated weights for policy 0, policy_version 1435322 (0.0007) [2023-12-27 15:35:32,904][20347] Updated weights for policy 0, policy_version 1435332 (0.0005) [2023-12-27 15:35:33,387][20381] Updated weights for policy 1, policy_version 1434033 (0.0006) [2023-12-27 15:35:33,452][20381] Updated weights for policy 1, policy_version 1434043 (0.0008) [2023-12-27 15:35:33,513][20381] Updated weights for policy 1, policy_version 1434053 (0.0008) [2023-12-27 15:35:33,553][20347] Updated weights for policy 0, policy_version 1435342 (0.0010) [2023-12-27 15:35:33,566][20381] Updated weights for policy 1, policy_version 1434063 (0.0006) [2023-12-27 15:35:33,611][20347] Updated weights for policy 0, policy_version 1435352 (0.0010) [2023-12-27 15:35:33,671][20347] Updated weights for policy 0, policy_version 1435362 (0.0010) [2023-12-27 15:35:34,150][20381] Updated weights for policy 1, policy_version 1434073 (0.0006) [2023-12-27 15:35:34,217][20381] Updated weights for policy 1, policy_version 1434083 (0.0007) [2023-12-27 15:35:34,283][20381] Updated weights for policy 1, policy_version 1434093 (0.0008) [2023-12-27 15:35:34,300][20347] Updated weights for policy 0, policy_version 1435372 (0.0006) [2023-12-27 15:35:34,356][20347] Updated weights for policy 0, policy_version 1435382 (0.0009) [2023-12-27 15:35:34,416][20347] Updated weights for policy 0, policy_version 1435392 (0.0011) [2023-12-27 15:35:34,898][20381] Updated weights for policy 1, policy_version 1434103 (0.0007) [2023-12-27 15:35:34,947][20381] Updated weights for policy 1, policy_version 1434113 (0.0007) [2023-12-27 15:35:35,004][20381] Updated weights for policy 1, policy_version 1434123 (0.0009) [2023-12-27 15:35:35,093][20347] Updated weights for policy 0, policy_version 1435402 (0.0010) [2023-12-27 15:35:35,157][20347] Updated weights for policy 0, policy_version 1435412 (0.0008) [2023-12-27 15:35:35,216][20347] Updated weights for policy 0, policy_version 1435422 (0.0009) [2023-12-27 15:35:35,271][20347] Updated weights for policy 0, policy_version 1435432 (0.0010) [2023-12-27 15:35:35,696][20381] Updated weights for policy 1, policy_version 1434133 (0.0007) [2023-12-27 15:35:35,750][20381] Updated weights for policy 1, policy_version 1434143 (0.0007) [2023-12-27 15:35:35,810][20381] Updated weights for policy 1, policy_version 1434153 (0.0006) [2023-12-27 15:35:35,889][20347] Updated weights for policy 0, policy_version 1435442 (0.0010) [2023-12-27 15:35:35,937][20347] Updated weights for policy 0, policy_version 1435452 (0.0010) [2023-12-27 15:35:35,987][20347] Updated weights for policy 0, policy_version 1435462 (0.0010) [2023-12-27 15:35:36,466][20381] Updated weights for policy 1, policy_version 1434163 (0.0006) [2023-12-27 15:35:36,522][20381] Updated weights for policy 1, policy_version 1434173 (0.0005) [2023-12-27 15:35:36,572][20381] Updated weights for policy 1, policy_version 1434183 (0.0008) [2023-12-27 15:35:36,624][20347] Updated weights for policy 0, policy_version 1435472 (0.0006) [2023-12-27 15:35:36,681][20347] Updated weights for policy 0, policy_version 1435482 (0.0008) [2023-12-27 15:35:36,741][20347] Updated weights for policy 0, policy_version 1435492 (0.0008) [2023-12-27 15:35:37,173][20381] Updated weights for policy 1, policy_version 1434193 (0.0006) [2023-12-27 15:35:37,232][20381] Updated weights for policy 1, policy_version 1434203 (0.0007) [2023-12-27 15:35:37,292][20381] Updated weights for policy 1, policy_version 1434213 (0.0008) [2023-12-27 15:35:37,323][20292] KL-divergence is very high: 204.4313 [2023-12-27 15:35:37,330][20292] KL-divergence is very high: 257.6725 [2023-12-27 15:35:37,337][20292] KL-divergence is very high: 299.6337 [2023-12-27 15:35:37,349][20292] KL-divergence is very high: 292.9748 [2023-12-27 15:35:37,354][20381] Updated weights for policy 1, policy_version 1434223 (0.0005) [2023-12-27 15:35:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 734740480. Throughput: 0: 10308.0, 1: 10295.1. Samples: 734714160. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:37,363][19170] Avg episode reward: [(0, '980.440'), (1, '640.330')] [2023-12-27 15:35:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001435496_367558656.pth... [2023-12-27 15:35:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001434224_367190016.pth... [2023-12-27 15:35:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001433008_366878720.pth [2023-12-27 15:35:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001434312_367255552.pth [2023-12-27 15:35:37,472][20347] Updated weights for policy 0, policy_version 1435502 (0.0007) [2023-12-27 15:35:37,534][20347] Updated weights for policy 0, policy_version 1435512 (0.0006) [2023-12-27 15:35:37,579][20347] Updated weights for policy 0, policy_version 1435522 (0.0005) [2023-12-27 15:35:37,962][20292] KL-divergence is very high: 272.2942 [2023-12-27 15:35:37,968][20292] KL-divergence is very high: 238.4064 [2023-12-27 15:35:38,014][20292] KL-divergence is very high: 166.9510 [2023-12-27 15:35:38,021][20292] KL-divergence is very high: 156.9373 [2023-12-27 15:35:38,022][20381] Updated weights for policy 1, policy_version 1434233 (0.0007) [2023-12-27 15:35:38,066][20292] KL-divergence is very high: 105.5108 [2023-12-27 15:35:38,072][20292] KL-divergence is very high: 105.4830 [2023-12-27 15:35:38,085][20381] Updated weights for policy 1, policy_version 1434243 (0.0008) [2023-12-27 15:35:38,115][20292] KL-divergence is very high: 103.3075 [2023-12-27 15:35:38,121][20292] KL-divergence is very high: 105.5189 [2023-12-27 15:35:38,143][20381] Updated weights for policy 1, policy_version 1434253 (0.0006) [2023-12-27 15:35:38,148][20347] Updated weights for policy 0, policy_version 1435532 (0.0007) [2023-12-27 15:35:38,198][20347] Updated weights for policy 0, policy_version 1435542 (0.0006) [2023-12-27 15:35:38,246][20347] Updated weights for policy 0, policy_version 1435552 (0.0005) [2023-12-27 15:35:38,788][20381] Updated weights for policy 1, policy_version 1434263 (0.0006) [2023-12-27 15:35:38,844][20381] Updated weights for policy 1, policy_version 1434273 (0.0006) [2023-12-27 15:35:38,879][20347] Updated weights for policy 0, policy_version 1435562 (0.0006) [2023-12-27 15:35:38,904][20381] Updated weights for policy 1, policy_version 1434283 (0.0006) [2023-12-27 15:35:38,936][20347] Updated weights for policy 0, policy_version 1435572 (0.0007) [2023-12-27 15:35:38,988][20347] Updated weights for policy 0, policy_version 1435582 (0.0008) [2023-12-27 15:35:39,042][20347] Updated weights for policy 0, policy_version 1435592 (0.0007) [2023-12-27 15:35:39,617][20381] Updated weights for policy 1, policy_version 1434293 (0.0006) [2023-12-27 15:35:39,670][20381] Updated weights for policy 1, policy_version 1434303 (0.0008) [2023-12-27 15:35:39,722][20381] Updated weights for policy 1, policy_version 1434313 (0.0008) [2023-12-27 15:35:39,746][20347] Updated weights for policy 0, policy_version 1435602 (0.0008) [2023-12-27 15:35:39,804][20347] Updated weights for policy 0, policy_version 1435612 (0.0008) [2023-12-27 15:35:39,873][20347] Updated weights for policy 0, policy_version 1435622 (0.0008) [2023-12-27 15:35:40,403][20381] Updated weights for policy 1, policy_version 1434323 (0.0006) [2023-12-27 15:35:40,461][20381] Updated weights for policy 1, policy_version 1434333 (0.0008) [2023-12-27 15:35:40,520][20381] Updated weights for policy 1, policy_version 1434343 (0.0008) [2023-12-27 15:35:40,615][20347] Updated weights for policy 0, policy_version 1435632 (0.0008) [2023-12-27 15:35:40,673][20347] Updated weights for policy 0, policy_version 1435642 (0.0009) [2023-12-27 15:35:40,737][20347] Updated weights for policy 0, policy_version 1435652 (0.0007) [2023-12-27 15:35:41,163][20381] Updated weights for policy 1, policy_version 1434353 (0.0008) [2023-12-27 15:35:41,228][20381] Updated weights for policy 1, policy_version 1434363 (0.0008) [2023-12-27 15:35:41,290][20381] Updated weights for policy 1, policy_version 1434373 (0.0008) [2023-12-27 15:35:41,355][20381] Updated weights for policy 1, policy_version 1434383 (0.0008) [2023-12-27 15:35:41,505][20347] Updated weights for policy 0, policy_version 1435662 (0.0007) [2023-12-27 15:35:41,565][20347] Updated weights for policy 0, policy_version 1435672 (0.0008) [2023-12-27 15:35:41,630][20347] Updated weights for policy 0, policy_version 1435682 (0.0009) [2023-12-27 15:35:42,025][20381] Updated weights for policy 1, policy_version 1434393 (0.0006) [2023-12-27 15:35:42,079][20381] Updated weights for policy 1, policy_version 1434403 (0.0005) [2023-12-27 15:35:42,135][20381] Updated weights for policy 1, policy_version 1434413 (0.0007) [2023-12-27 15:35:42,284][20347] Updated weights for policy 0, policy_version 1435692 (0.0007) [2023-12-27 15:35:42,344][20347] Updated weights for policy 0, policy_version 1435702 (0.0007) [2023-12-27 15:35:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 734846976. Throughput: 0: 10315.8, 1: 10345.4. Samples: 734839412. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:42,363][19170] Avg episode reward: [(0, '970.650'), (1, '655.620')] [2023-12-27 15:35:42,409][20347] Updated weights for policy 0, policy_version 1435712 (0.0008) [2023-12-27 15:35:42,772][20381] Updated weights for policy 1, policy_version 1434423 (0.0008) [2023-12-27 15:35:42,829][20381] Updated weights for policy 1, policy_version 1434433 (0.0008) [2023-12-27 15:35:42,885][20381] Updated weights for policy 1, policy_version 1434443 (0.0008) [2023-12-27 15:35:43,061][20347] Updated weights for policy 0, policy_version 1435722 (0.0007) [2023-12-27 15:35:43,111][20347] Updated weights for policy 0, policy_version 1435732 (0.0008) [2023-12-27 15:35:43,161][20347] Updated weights for policy 0, policy_version 1435742 (0.0007) [2023-12-27 15:35:43,216][20347] Updated weights for policy 0, policy_version 1435752 (0.0007) [2023-12-27 15:35:43,567][20381] Updated weights for policy 1, policy_version 1434453 (0.0008) [2023-12-27 15:35:43,617][20381] Updated weights for policy 1, policy_version 1434463 (0.0008) [2023-12-27 15:35:43,673][20381] Updated weights for policy 1, policy_version 1434473 (0.0008) [2023-12-27 15:35:43,932][20347] Updated weights for policy 0, policy_version 1435762 (0.0007) [2023-12-27 15:35:43,983][20347] Updated weights for policy 0, policy_version 1435772 (0.0007) [2023-12-27 15:35:44,036][20347] Updated weights for policy 0, policy_version 1435782 (0.0007) [2023-12-27 15:35:44,339][20381] Updated weights for policy 1, policy_version 1434483 (0.0008) [2023-12-27 15:35:44,394][20381] Updated weights for policy 1, policy_version 1434493 (0.0007) [2023-12-27 15:35:44,446][20381] Updated weights for policy 1, policy_version 1434503 (0.0007) [2023-12-27 15:35:44,720][20347] Updated weights for policy 0, policy_version 1435792 (0.0009) [2023-12-27 15:35:44,775][20347] Updated weights for policy 0, policy_version 1435802 (0.0009) [2023-12-27 15:35:44,839][20347] Updated weights for policy 0, policy_version 1435812 (0.0010) [2023-12-27 15:35:45,175][20381] Updated weights for policy 1, policy_version 1434513 (0.0008) [2023-12-27 15:35:45,232][20381] Updated weights for policy 1, policy_version 1434523 (0.0007) [2023-12-27 15:35:45,294][20381] Updated weights for policy 1, policy_version 1434533 (0.0007) [2023-12-27 15:35:45,355][20381] Updated weights for policy 1, policy_version 1434543 (0.0007) [2023-12-27 15:35:45,521][20347] Updated weights for policy 0, policy_version 1435822 (0.0010) [2023-12-27 15:35:45,575][20347] Updated weights for policy 0, policy_version 1435832 (0.0010) [2023-12-27 15:35:45,630][20347] Updated weights for policy 0, policy_version 1435842 (0.0010) [2023-12-27 15:35:46,032][20381] Updated weights for policy 1, policy_version 1434553 (0.0008) [2023-12-27 15:35:46,089][20381] Updated weights for policy 1, policy_version 1434563 (0.0008) [2023-12-27 15:35:46,149][20381] Updated weights for policy 1, policy_version 1434573 (0.0008) [2023-12-27 15:35:46,253][20347] Updated weights for policy 0, policy_version 1435852 (0.0010) [2023-12-27 15:35:46,302][20347] Updated weights for policy 0, policy_version 1435862 (0.0010) [2023-12-27 15:35:46,351][20347] Updated weights for policy 0, policy_version 1435872 (0.0010) [2023-12-27 15:35:46,814][20381] Updated weights for policy 1, policy_version 1434583 (0.0006) [2023-12-27 15:35:46,875][20381] Updated weights for policy 1, policy_version 1434593 (0.0005) [2023-12-27 15:35:46,938][20381] Updated weights for policy 1, policy_version 1434603 (0.0005) [2023-12-27 15:35:46,980][20347] Updated weights for policy 0, policy_version 1435882 (0.0009) [2023-12-27 15:35:47,037][20347] Updated weights for policy 0, policy_version 1435892 (0.0007) [2023-12-27 15:35:47,090][20347] Updated weights for policy 0, policy_version 1435902 (0.0010) [2023-12-27 15:35:47,144][20347] Updated weights for policy 0, policy_version 1435912 (0.0010) [2023-12-27 15:35:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 734953472. Throughput: 0: 10310.3, 1: 10357.6. Samples: 734962888. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:47,363][19170] Avg episode reward: [(0, '990.210'), (1, '654.360')] [2023-12-27 15:35:47,493][20381] Updated weights for policy 1, policy_version 1434613 (0.0006) [2023-12-27 15:35:47,547][20381] Updated weights for policy 1, policy_version 1434623 (0.0008) [2023-12-27 15:35:47,607][20381] Updated weights for policy 1, policy_version 1434633 (0.0007) [2023-12-27 15:35:47,791][20347] Updated weights for policy 0, policy_version 1435922 (0.0006) [2023-12-27 15:35:47,847][20347] Updated weights for policy 0, policy_version 1435932 (0.0007) [2023-12-27 15:35:47,892][20347] Updated weights for policy 0, policy_version 1435942 (0.0005) [2023-12-27 15:35:48,207][20381] Updated weights for policy 1, policy_version 1434643 (0.0006) [2023-12-27 15:35:48,264][20381] Updated weights for policy 1, policy_version 1434653 (0.0008) [2023-12-27 15:35:48,322][20381] Updated weights for policy 1, policy_version 1434663 (0.0008) [2023-12-27 15:35:48,615][20347] Updated weights for policy 0, policy_version 1435952 (0.0010) [2023-12-27 15:35:48,677][20347] Updated weights for policy 0, policy_version 1435962 (0.0011) [2023-12-27 15:35:48,736][20347] Updated weights for policy 0, policy_version 1435972 (0.0010) [2023-12-27 15:35:49,007][20381] Updated weights for policy 1, policy_version 1434673 (0.0008) [2023-12-27 15:35:49,064][20381] Updated weights for policy 1, policy_version 1434683 (0.0007) [2023-12-27 15:35:49,120][20381] Updated weights for policy 1, policy_version 1434693 (0.0005) [2023-12-27 15:35:49,180][20381] Updated weights for policy 1, policy_version 1434703 (0.0005) [2023-12-27 15:35:49,388][20347] Updated weights for policy 0, policy_version 1435982 (0.0010) [2023-12-27 15:35:49,449][20347] Updated weights for policy 0, policy_version 1435992 (0.0010) [2023-12-27 15:35:49,511][20347] Updated weights for policy 0, policy_version 1436002 (0.0006) [2023-12-27 15:35:49,799][20381] Updated weights for policy 1, policy_version 1434713 (0.0007) [2023-12-27 15:35:49,859][20381] Updated weights for policy 1, policy_version 1434723 (0.0009) [2023-12-27 15:35:49,925][20381] Updated weights for policy 1, policy_version 1434733 (0.0008) [2023-12-27 15:35:50,206][20347] Updated weights for policy 0, policy_version 1436012 (0.0006) [2023-12-27 15:35:50,268][20347] Updated weights for policy 0, policy_version 1436022 (0.0006) [2023-12-27 15:35:50,325][20347] Updated weights for policy 0, policy_version 1436032 (0.0008) [2023-12-27 15:35:50,642][20381] Updated weights for policy 1, policy_version 1434743 (0.0008) [2023-12-27 15:35:50,703][20381] Updated weights for policy 1, policy_version 1434753 (0.0008) [2023-12-27 15:35:50,760][20381] Updated weights for policy 1, policy_version 1434763 (0.0008) [2023-12-27 15:35:50,942][20347] Updated weights for policy 0, policy_version 1436042 (0.0007) [2023-12-27 15:35:50,994][20347] Updated weights for policy 0, policy_version 1436052 (0.0007) [2023-12-27 15:35:51,060][20347] Updated weights for policy 0, policy_version 1436062 (0.0008) [2023-12-27 15:35:51,121][20347] Updated weights for policy 0, policy_version 1436072 (0.0008) [2023-12-27 15:35:51,553][20381] Updated weights for policy 1, policy_version 1434773 (0.0007) [2023-12-27 15:35:51,617][20381] Updated weights for policy 1, policy_version 1434783 (0.0006) [2023-12-27 15:35:51,691][20381] Updated weights for policy 1, policy_version 1434793 (0.0009) [2023-12-27 15:35:51,745][20347] Updated weights for policy 0, policy_version 1436082 (0.0007) [2023-12-27 15:35:51,809][20347] Updated weights for policy 0, policy_version 1436092 (0.0006) [2023-12-27 15:35:51,871][20347] Updated weights for policy 0, policy_version 1436102 (0.0006) [2023-12-27 15:35:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 735051776. Throughput: 0: 10279.2, 1: 10391.2. Samples: 735025664. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '581.650')] [2023-12-27 15:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001436104_367714304.pth... [2023-12-27 15:35:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001434800_367337472.pth... [2023-12-27 15:35:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001434888_367403008.pth [2023-12-27 15:35:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001433584_367026176.pth [2023-12-27 15:35:52,433][20381] Updated weights for policy 1, policy_version 1434803 (0.0009) [2023-12-27 15:35:52,491][20381] Updated weights for policy 1, policy_version 1434813 (0.0008) [2023-12-27 15:35:52,546][20381] Updated weights for policy 1, policy_version 1434823 (0.0009) [2023-12-27 15:35:52,558][20347] Updated weights for policy 0, policy_version 1436112 (0.0008) [2023-12-27 15:35:52,616][20347] Updated weights for policy 0, policy_version 1436122 (0.0007) [2023-12-27 15:35:52,670][20347] Updated weights for policy 0, policy_version 1436132 (0.0008) [2023-12-27 15:35:53,287][20381] Updated weights for policy 1, policy_version 1434833 (0.0007) [2023-12-27 15:35:53,308][20347] Updated weights for policy 0, policy_version 1436142 (0.0007) [2023-12-27 15:35:53,341][20381] Updated weights for policy 1, policy_version 1434843 (0.0006) [2023-12-27 15:35:53,367][20347] Updated weights for policy 0, policy_version 1436152 (0.0007) [2023-12-27 15:35:53,391][20381] Updated weights for policy 1, policy_version 1434853 (0.0005) [2023-12-27 15:35:53,419][20347] Updated weights for policy 0, policy_version 1436162 (0.0008) [2023-12-27 15:35:53,449][20381] Updated weights for policy 1, policy_version 1434863 (0.0006) [2023-12-27 15:35:54,061][20381] Updated weights for policy 1, policy_version 1434873 (0.0006) [2023-12-27 15:35:54,108][20347] Updated weights for policy 0, policy_version 1436172 (0.0008) [2023-12-27 15:35:54,109][20381] Updated weights for policy 1, policy_version 1434883 (0.0010) [2023-12-27 15:35:54,162][20381] Updated weights for policy 1, policy_version 1434893 (0.0010) [2023-12-27 15:35:54,170][20347] Updated weights for policy 0, policy_version 1436182 (0.0007) [2023-12-27 15:35:54,225][20347] Updated weights for policy 0, policy_version 1436192 (0.0010) [2023-12-27 15:35:54,794][20381] Updated weights for policy 1, policy_version 1434903 (0.0007) [2023-12-27 15:35:54,855][20381] Updated weights for policy 1, policy_version 1434913 (0.0007) [2023-12-27 15:35:54,871][20347] Updated weights for policy 0, policy_version 1436202 (0.0009) [2023-12-27 15:35:54,907][20381] Updated weights for policy 1, policy_version 1434923 (0.0007) [2023-12-27 15:35:54,932][20347] Updated weights for policy 0, policy_version 1436212 (0.0006) [2023-12-27 15:35:54,991][20347] Updated weights for policy 0, policy_version 1436222 (0.0008) [2023-12-27 15:35:55,040][20347] Updated weights for policy 0, policy_version 1436232 (0.0007) [2023-12-27 15:35:55,551][20381] Updated weights for policy 1, policy_version 1434933 (0.0008) [2023-12-27 15:35:55,605][20381] Updated weights for policy 1, policy_version 1434943 (0.0007) [2023-12-27 15:35:55,663][20381] Updated weights for policy 1, policy_version 1434953 (0.0007) [2023-12-27 15:35:55,731][20347] Updated weights for policy 0, policy_version 1436242 (0.0007) [2023-12-27 15:35:55,791][20347] Updated weights for policy 0, policy_version 1436252 (0.0008) [2023-12-27 15:35:55,844][20347] Updated weights for policy 0, policy_version 1436262 (0.0008) [2023-12-27 15:35:56,335][20381] Updated weights for policy 1, policy_version 1434963 (0.0006) [2023-12-27 15:35:56,389][20381] Updated weights for policy 1, policy_version 1434973 (0.0007) [2023-12-27 15:35:56,449][20381] Updated weights for policy 1, policy_version 1434983 (0.0007) [2023-12-27 15:35:56,540][20347] Updated weights for policy 0, policy_version 1436272 (0.0008) [2023-12-27 15:35:56,595][20347] Updated weights for policy 0, policy_version 1436282 (0.0007) [2023-12-27 15:35:56,646][20347] Updated weights for policy 0, policy_version 1436292 (0.0005) [2023-12-27 15:35:57,126][20381] Updated weights for policy 1, policy_version 1434993 (0.0008) [2023-12-27 15:35:57,172][20381] Updated weights for policy 1, policy_version 1435003 (0.0007) [2023-12-27 15:35:57,218][20381] Updated weights for policy 1, policy_version 1435013 (0.0007) [2023-12-27 15:35:57,228][20347] Updated weights for policy 0, policy_version 1436302 (0.0008) [2023-12-27 15:35:57,270][20381] Updated weights for policy 1, policy_version 1435023 (0.0006) [2023-12-27 15:35:57,276][20347] Updated weights for policy 0, policy_version 1436312 (0.0010) [2023-12-27 15:35:57,331][20347] Updated weights for policy 0, policy_version 1436322 (0.0010) [2023-12-27 15:35:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.5). Total num frames: 735158272. Throughput: 0: 10279.8, 1: 10349.9. Samples: 735147752. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:35:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '627.590')] [2023-12-27 15:35:57,912][20347] Updated weights for policy 0, policy_version 1436332 (0.0010) [2023-12-27 15:35:57,967][20347] Updated weights for policy 0, policy_version 1436342 (0.0010) [2023-12-27 15:35:58,011][20381] Updated weights for policy 1, policy_version 1435033 (0.0007) [2023-12-27 15:35:58,018][20347] Updated weights for policy 0, policy_version 1436352 (0.0007) [2023-12-27 15:35:58,056][20381] Updated weights for policy 1, policy_version 1435043 (0.0007) [2023-12-27 15:35:58,103][20381] Updated weights for policy 1, policy_version 1435053 (0.0007) [2023-12-27 15:35:58,687][20347] Updated weights for policy 0, policy_version 1436362 (0.0006) [2023-12-27 15:35:58,752][20347] Updated weights for policy 0, policy_version 1436372 (0.0008) [2023-12-27 15:35:58,826][20347] Updated weights for policy 0, policy_version 1436382 (0.0008) [2023-12-27 15:35:58,901][20347] Updated weights for policy 0, policy_version 1436392 (0.0008) [2023-12-27 15:35:58,985][20381] Updated weights for policy 1, policy_version 1435063 (0.0007) [2023-12-27 15:35:59,044][20381] Updated weights for policy 1, policy_version 1435073 (0.0008) [2023-12-27 15:35:59,100][20381] Updated weights for policy 1, policy_version 1435083 (0.0006) [2023-12-27 15:35:59,643][20347] Updated weights for policy 0, policy_version 1436402 (0.0006) [2023-12-27 15:35:59,705][20347] Updated weights for policy 0, policy_version 1436412 (0.0007) [2023-12-27 15:35:59,769][20347] Updated weights for policy 0, policy_version 1436422 (0.0010) [2023-12-27 15:35:59,804][20381] Updated weights for policy 1, policy_version 1435093 (0.0006) [2023-12-27 15:35:59,868][20381] Updated weights for policy 1, policy_version 1435103 (0.0009) [2023-12-27 15:35:59,934][20381] Updated weights for policy 1, policy_version 1435113 (0.0008) [2023-12-27 15:36:00,383][20347] Updated weights for policy 0, policy_version 1436432 (0.0007) [2023-12-27 15:36:00,433][20347] Updated weights for policy 0, policy_version 1436442 (0.0006) [2023-12-27 15:36:00,489][20347] Updated weights for policy 0, policy_version 1436452 (0.0006) [2023-12-27 15:36:00,627][20381] Updated weights for policy 1, policy_version 1435123 (0.0010) [2023-12-27 15:36:00,684][20381] Updated weights for policy 1, policy_version 1435133 (0.0010) [2023-12-27 15:36:00,741][20381] Updated weights for policy 1, policy_version 1435143 (0.0010) [2023-12-27 15:36:01,061][20347] Updated weights for policy 0, policy_version 1436462 (0.0008) [2023-12-27 15:36:01,118][20347] Updated weights for policy 0, policy_version 1436472 (0.0010) [2023-12-27 15:36:01,177][20347] Updated weights for policy 0, policy_version 1436482 (0.0010) [2023-12-27 15:36:01,416][20381] Updated weights for policy 1, policy_version 1435153 (0.0010) [2023-12-27 15:36:01,484][20381] Updated weights for policy 1, policy_version 1435163 (0.0009) [2023-12-27 15:36:01,541][20381] Updated weights for policy 1, policy_version 1435173 (0.0005) [2023-12-27 15:36:01,602][20381] Updated weights for policy 1, policy_version 1435183 (0.0010) [2023-12-27 15:36:01,875][20347] Updated weights for policy 0, policy_version 1436492 (0.0010) [2023-12-27 15:36:01,933][20347] Updated weights for policy 0, policy_version 1436502 (0.0009) [2023-12-27 15:36:01,987][20347] Updated weights for policy 0, policy_version 1436512 (0.0006) [2023-12-27 15:36:02,278][20381] Updated weights for policy 1, policy_version 1435193 (0.0007) [2023-12-27 15:36:02,337][20381] Updated weights for policy 1, policy_version 1435203 (0.0006) [2023-12-27 15:36:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 735256576. Throughput: 0: 10342.1, 1: 10301.3. Samples: 735270436. Policy #0 lag: (min: 26.0, avg: 41.4, max: 58.0) [2023-12-27 15:36:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '591.440')] [2023-12-27 15:36:02,404][20381] Updated weights for policy 1, policy_version 1435213 (0.0008) [2023-12-27 15:36:02,624][20347] Updated weights for policy 0, policy_version 1436522 (0.0006) [2023-12-27 15:36:02,674][20347] Updated weights for policy 0, policy_version 1436532 (0.0007) [2023-12-27 15:36:02,727][20347] Updated weights for policy 0, policy_version 1436542 (0.0008) [2023-12-27 15:36:02,783][20347] Updated weights for policy 0, policy_version 1436552 (0.0007) [2023-12-27 15:36:03,046][20381] Updated weights for policy 1, policy_version 1435223 (0.0006) [2023-12-27 15:36:03,106][20381] Updated weights for policy 1, policy_version 1435233 (0.0008) [2023-12-27 15:36:03,159][20381] Updated weights for policy 1, policy_version 1435243 (0.0007) [2023-12-27 15:36:03,469][20347] Updated weights for policy 0, policy_version 1436562 (0.0007) [2023-12-27 15:36:03,520][20347] Updated weights for policy 0, policy_version 1436572 (0.0007) [2023-12-27 15:36:03,567][20347] Updated weights for policy 0, policy_version 1436582 (0.0007) [2023-12-27 15:36:03,781][20381] Updated weights for policy 1, policy_version 1435253 (0.0006) [2023-12-27 15:36:03,853][20381] Updated weights for policy 1, policy_version 1435263 (0.0006) [2023-12-27 15:36:03,920][20381] Updated weights for policy 1, policy_version 1435273 (0.0007) [2023-12-27 15:36:04,314][20347] Updated weights for policy 0, policy_version 1436592 (0.0007) [2023-12-27 15:36:04,375][20347] Updated weights for policy 0, policy_version 1436602 (0.0008) [2023-12-27 15:36:04,426][20347] Updated weights for policy 0, policy_version 1436612 (0.0007) [2023-12-27 15:36:04,636][20381] Updated weights for policy 1, policy_version 1435283 (0.0007) [2023-12-27 15:36:04,693][20381] Updated weights for policy 1, policy_version 1435293 (0.0007) [2023-12-27 15:36:04,756][20381] Updated weights for policy 1, policy_version 1435303 (0.0008) [2023-12-27 15:36:05,067][20347] Updated weights for policy 0, policy_version 1436622 (0.0006) [2023-12-27 15:36:05,114][20347] Updated weights for policy 0, policy_version 1436632 (0.0005) [2023-12-27 15:36:05,165][20347] Updated weights for policy 0, policy_version 1436642 (0.0005) [2023-12-27 15:36:05,382][20381] Updated weights for policy 1, policy_version 1435313 (0.0008) [2023-12-27 15:36:05,442][20381] Updated weights for policy 1, policy_version 1435323 (0.0009) [2023-12-27 15:36:05,504][20381] Updated weights for policy 1, policy_version 1435333 (0.0008) [2023-12-27 15:36:05,556][20381] Updated weights for policy 1, policy_version 1435343 (0.0008) [2023-12-27 15:36:05,785][20347] Updated weights for policy 0, policy_version 1436652 (0.0006) [2023-12-27 15:36:05,835][20347] Updated weights for policy 0, policy_version 1436662 (0.0006) [2023-12-27 15:36:05,893][20347] Updated weights for policy 0, policy_version 1436672 (0.0006) [2023-12-27 15:36:06,287][20381] Updated weights for policy 1, policy_version 1435353 (0.0007) [2023-12-27 15:36:06,354][20381] Updated weights for policy 1, policy_version 1435363 (0.0009) [2023-12-27 15:36:06,416][20381] Updated weights for policy 1, policy_version 1435373 (0.0008) [2023-12-27 15:36:06,531][20347] Updated weights for policy 0, policy_version 1436682 (0.0008) [2023-12-27 15:36:06,592][20347] Updated weights for policy 0, policy_version 1436692 (0.0007) [2023-12-27 15:36:06,641][20347] Updated weights for policy 0, policy_version 1436702 (0.0007) [2023-12-27 15:36:06,692][20347] Updated weights for policy 0, policy_version 1436712 (0.0008) [2023-12-27 15:36:07,124][20381] Updated weights for policy 1, policy_version 1435383 (0.0009) [2023-12-27 15:36:07,173][20381] Updated weights for policy 1, policy_version 1435393 (0.0007) [2023-12-27 15:36:07,224][20381] Updated weights for policy 1, policy_version 1435403 (0.0008) [2023-12-27 15:36:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 735363072. Throughput: 0: 10335.5, 1: 10311.4. Samples: 735333148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '602.560')] [2023-12-27 15:36:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001435408_367493120.pth... [2023-12-27 15:36:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001434224_367190016.pth [2023-12-27 15:36:07,410][20347] Updated weights for policy 0, policy_version 1436722 (0.0007) [2023-12-27 15:36:07,462][20347] Updated weights for policy 0, policy_version 1436732 (0.0008) [2023-12-27 15:36:07,515][20347] Updated weights for policy 0, policy_version 1436742 (0.0007) [2023-12-27 15:36:07,522][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001436744_367878144.pth... [2023-12-27 15:36:07,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001435496_367558656.pth [2023-12-27 15:36:07,925][20381] Updated weights for policy 1, policy_version 1435413 (0.0008) [2023-12-27 15:36:07,977][20381] Updated weights for policy 1, policy_version 1435423 (0.0007) [2023-12-27 15:36:08,029][20381] Updated weights for policy 1, policy_version 1435433 (0.0008) [2023-12-27 15:36:08,207][20347] Updated weights for policy 0, policy_version 1436752 (0.0007) [2023-12-27 15:36:08,255][20347] Updated weights for policy 0, policy_version 1436762 (0.0007) [2023-12-27 15:36:08,302][20347] Updated weights for policy 0, policy_version 1436772 (0.0009) [2023-12-27 15:36:08,692][20381] Updated weights for policy 1, policy_version 1435443 (0.0007) [2023-12-27 15:36:08,747][20381] Updated weights for policy 1, policy_version 1435453 (0.0006) [2023-12-27 15:36:08,806][20381] Updated weights for policy 1, policy_version 1435463 (0.0008) [2023-12-27 15:36:09,064][20347] Updated weights for policy 0, policy_version 1436782 (0.0008) [2023-12-27 15:36:09,156][20347] Updated weights for policy 0, policy_version 1436792 (0.0008) [2023-12-27 15:36:09,212][20347] Updated weights for policy 0, policy_version 1436802 (0.0008) [2023-12-27 15:36:09,451][20381] Updated weights for policy 1, policy_version 1435473 (0.0007) [2023-12-27 15:36:09,507][20381] Updated weights for policy 1, policy_version 1435483 (0.0008) [2023-12-27 15:36:09,572][20381] Updated weights for policy 1, policy_version 1435493 (0.0009) [2023-12-27 15:36:09,626][20381] Updated weights for policy 1, policy_version 1435503 (0.0008) [2023-12-27 15:36:09,879][20347] Updated weights for policy 0, policy_version 1436812 (0.0008) [2023-12-27 15:36:09,943][20347] Updated weights for policy 0, policy_version 1436822 (0.0008) [2023-12-27 15:36:10,013][20347] Updated weights for policy 0, policy_version 1436832 (0.0008) [2023-12-27 15:36:10,421][20381] Updated weights for policy 1, policy_version 1435513 (0.0007) [2023-12-27 15:36:10,476][20381] Updated weights for policy 1, policy_version 1435523 (0.0009) [2023-12-27 15:36:10,533][20381] Updated weights for policy 1, policy_version 1435533 (0.0008) [2023-12-27 15:36:10,637][20347] Updated weights for policy 0, policy_version 1436842 (0.0006) [2023-12-27 15:36:10,697][20347] Updated weights for policy 0, policy_version 1436852 (0.0007) [2023-12-27 15:36:10,755][20347] Updated weights for policy 0, policy_version 1436862 (0.0005) [2023-12-27 15:36:10,798][20347] Updated weights for policy 0, policy_version 1436872 (0.0005) [2023-12-27 15:36:11,291][20381] Updated weights for policy 1, policy_version 1435543 (0.0008) [2023-12-27 15:36:11,363][20381] Updated weights for policy 1, policy_version 1435553 (0.0007) [2023-12-27 15:36:11,426][20381] Updated weights for policy 1, policy_version 1435563 (0.0007) [2023-12-27 15:36:11,486][20347] Updated weights for policy 0, policy_version 1436882 (0.0007) [2023-12-27 15:36:11,535][20347] Updated weights for policy 0, policy_version 1436892 (0.0006) [2023-12-27 15:36:11,597][20347] Updated weights for policy 0, policy_version 1436902 (0.0006) [2023-12-27 15:36:12,083][20381] Updated weights for policy 1, policy_version 1435573 (0.0007) [2023-12-27 15:36:12,139][20381] Updated weights for policy 1, policy_version 1435583 (0.0007) [2023-12-27 15:36:12,188][20381] Updated weights for policy 1, policy_version 1435593 (0.0007) [2023-12-27 15:36:12,363][20347] Updated weights for policy 0, policy_version 1436912 (0.0008) [2023-12-27 15:36:12,367][19170] Fps is (10 sec: 20471.1, 60 sec: 20478.5, 300 sec: 20521.3). Total num frames: 735461376. Throughput: 0: 10383.1, 1: 10251.2. Samples: 735453332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:12,367][19170] Avg episode reward: [(0, '990.290'), (1, '647.830')] [2023-12-27 15:36:12,430][20347] Updated weights for policy 0, policy_version 1436922 (0.0006) [2023-12-27 15:36:12,487][20347] Updated weights for policy 0, policy_version 1436932 (0.0005) [2023-12-27 15:36:12,911][20381] Updated weights for policy 1, policy_version 1435603 (0.0008) [2023-12-27 15:36:12,968][20381] Updated weights for policy 1, policy_version 1435613 (0.0008) [2023-12-27 15:36:13,024][20381] Updated weights for policy 1, policy_version 1435623 (0.0005) [2023-12-27 15:36:13,089][20347] Updated weights for policy 0, policy_version 1436942 (0.0006) [2023-12-27 15:36:13,137][20291] KL-divergence is very high: 121.4668 [2023-12-27 15:36:13,142][20347] Updated weights for policy 0, policy_version 1436952 (0.0005) [2023-12-27 15:36:13,176][20291] KL-divergence is very high: 134.6256 [2023-12-27 15:36:13,193][20347] Updated weights for policy 0, policy_version 1436962 (0.0005) [2023-12-27 15:36:13,724][20347] Updated weights for policy 0, policy_version 1436972 (0.0006) [2023-12-27 15:36:13,755][20381] Updated weights for policy 1, policy_version 1435633 (0.0006) [2023-12-27 15:36:13,777][20347] Updated weights for policy 0, policy_version 1436982 (0.0007) [2023-12-27 15:36:13,812][20381] Updated weights for policy 1, policy_version 1435643 (0.0010) [2023-12-27 15:36:13,836][20347] Updated weights for policy 0, policy_version 1436992 (0.0006) [2023-12-27 15:36:13,869][20381] Updated weights for policy 1, policy_version 1435653 (0.0011) [2023-12-27 15:36:13,925][20381] Updated weights for policy 1, policy_version 1435663 (0.0011) [2023-12-27 15:36:14,425][20347] Updated weights for policy 0, policy_version 1437002 (0.0006) [2023-12-27 15:36:14,485][20347] Updated weights for policy 0, policy_version 1437012 (0.0005) [2023-12-27 15:36:14,544][20347] Updated weights for policy 0, policy_version 1437022 (0.0008) [2023-12-27 15:36:14,593][20381] Updated weights for policy 1, policy_version 1435673 (0.0007) [2023-12-27 15:36:14,600][20347] Updated weights for policy 0, policy_version 1437032 (0.0009) [2023-12-27 15:36:14,656][20381] Updated weights for policy 1, policy_version 1435683 (0.0007) [2023-12-27 15:36:14,713][20381] Updated weights for policy 1, policy_version 1435693 (0.0010) [2023-12-27 15:36:15,220][20347] Updated weights for policy 0, policy_version 1437042 (0.0006) [2023-12-27 15:36:15,282][20347] Updated weights for policy 0, policy_version 1437052 (0.0008) [2023-12-27 15:36:15,343][20347] Updated weights for policy 0, policy_version 1437062 (0.0008) [2023-12-27 15:36:15,421][20381] Updated weights for policy 1, policy_version 1435703 (0.0009) [2023-12-27 15:36:15,479][20381] Updated weights for policy 1, policy_version 1435713 (0.0008) [2023-12-27 15:36:15,540][20381] Updated weights for policy 1, policy_version 1435723 (0.0007) [2023-12-27 15:36:16,000][20347] Updated weights for policy 0, policy_version 1437072 (0.0007) [2023-12-27 15:36:16,049][20347] Updated weights for policy 0, policy_version 1437082 (0.0007) [2023-12-27 15:36:16,094][20347] Updated weights for policy 0, policy_version 1437092 (0.0009) [2023-12-27 15:36:16,135][20381] Updated weights for policy 1, policy_version 1435733 (0.0007) [2023-12-27 15:36:16,184][20381] Updated weights for policy 1, policy_version 1435743 (0.0008) [2023-12-27 15:36:16,234][20381] Updated weights for policy 1, policy_version 1435753 (0.0009) [2023-12-27 15:36:16,680][20347] Updated weights for policy 0, policy_version 1437102 (0.0008) [2023-12-27 15:36:16,734][20347] Updated weights for policy 0, policy_version 1437112 (0.0010) [2023-12-27 15:36:16,790][20347] Updated weights for policy 0, policy_version 1437122 (0.0007) [2023-12-27 15:36:16,854][20381] Updated weights for policy 1, policy_version 1435763 (0.0007) [2023-12-27 15:36:16,919][20381] Updated weights for policy 1, policy_version 1435773 (0.0007) [2023-12-27 15:36:16,975][20381] Updated weights for policy 1, policy_version 1435783 (0.0008) [2023-12-27 15:36:17,359][20347] Updated weights for policy 0, policy_version 1437132 (0.0007) [2023-12-27 15:36:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 735567872. Throughput: 0: 10457.6, 1: 10200.4. Samples: 735579948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:17,363][19170] Avg episode reward: [(0, '990.290'), (1, '584.480')] [2023-12-27 15:36:17,410][20347] Updated weights for policy 0, policy_version 1437142 (0.0005) [2023-12-27 15:36:17,465][20347] Updated weights for policy 0, policy_version 1437152 (0.0006) [2023-12-27 15:36:17,714][20381] Updated weights for policy 1, policy_version 1435793 (0.0007) [2023-12-27 15:36:17,768][20381] Updated weights for policy 1, policy_version 1435803 (0.0007) [2023-12-27 15:36:17,816][20381] Updated weights for policy 1, policy_version 1435813 (0.0007) [2023-12-27 15:36:17,868][20381] Updated weights for policy 1, policy_version 1435823 (0.0009) [2023-12-27 15:36:18,113][20347] Updated weights for policy 0, policy_version 1437162 (0.0010) [2023-12-27 15:36:18,170][20347] Updated weights for policy 0, policy_version 1437172 (0.0008) [2023-12-27 15:36:18,228][20347] Updated weights for policy 0, policy_version 1437182 (0.0008) [2023-12-27 15:36:18,291][20347] Updated weights for policy 0, policy_version 1437192 (0.0007) [2023-12-27 15:36:18,539][20381] Updated weights for policy 1, policy_version 1435833 (0.0009) [2023-12-27 15:36:18,595][20381] Updated weights for policy 1, policy_version 1435843 (0.0011) [2023-12-27 15:36:18,651][20381] Updated weights for policy 1, policy_version 1435853 (0.0011) [2023-12-27 15:36:18,993][20347] Updated weights for policy 0, policy_version 1437202 (0.0007) [2023-12-27 15:36:19,050][20347] Updated weights for policy 0, policy_version 1437212 (0.0007) [2023-12-27 15:36:19,106][20347] Updated weights for policy 0, policy_version 1437222 (0.0008) [2023-12-27 15:36:19,335][20381] Updated weights for policy 1, policy_version 1435863 (0.0011) [2023-12-27 15:36:19,401][20381] Updated weights for policy 1, policy_version 1435873 (0.0011) [2023-12-27 15:36:19,462][20381] Updated weights for policy 1, policy_version 1435883 (0.0010) [2023-12-27 15:36:19,882][20347] Updated weights for policy 0, policy_version 1437232 (0.0007) [2023-12-27 15:36:19,944][20347] Updated weights for policy 0, policy_version 1437242 (0.0008) [2023-12-27 15:36:20,000][20347] Updated weights for policy 0, policy_version 1437252 (0.0007) [2023-12-27 15:36:20,148][20381] Updated weights for policy 1, policy_version 1435893 (0.0009) [2023-12-27 15:36:20,207][20381] Updated weights for policy 1, policy_version 1435903 (0.0008) [2023-12-27 15:36:20,266][20381] Updated weights for policy 1, policy_version 1435913 (0.0008) [2023-12-27 15:36:20,725][20347] Updated weights for policy 0, policy_version 1437262 (0.0007) [2023-12-27 15:36:20,784][20347] Updated weights for policy 0, policy_version 1437272 (0.0005) [2023-12-27 15:36:20,840][20347] Updated weights for policy 0, policy_version 1437282 (0.0006) [2023-12-27 15:36:20,942][20381] Updated weights for policy 1, policy_version 1435923 (0.0007) [2023-12-27 15:36:20,996][20381] Updated weights for policy 1, policy_version 1435933 (0.0005) [2023-12-27 15:36:21,055][20381] Updated weights for policy 1, policy_version 1435943 (0.0007) [2023-12-27 15:36:21,479][20347] Updated weights for policy 0, policy_version 1437292 (0.0008) [2023-12-27 15:36:21,545][20347] Updated weights for policy 0, policy_version 1437302 (0.0008) [2023-12-27 15:36:21,605][20347] Updated weights for policy 0, policy_version 1437312 (0.0008) [2023-12-27 15:36:21,775][20381] Updated weights for policy 1, policy_version 1435953 (0.0009) [2023-12-27 15:36:21,832][20381] Updated weights for policy 1, policy_version 1435963 (0.0007) [2023-12-27 15:36:21,890][20381] Updated weights for policy 1, policy_version 1435973 (0.0008) [2023-12-27 15:36:21,942][20381] Updated weights for policy 1, policy_version 1435983 (0.0007) [2023-12-27 15:36:22,304][20347] Updated weights for policy 0, policy_version 1437322 (0.0008) [2023-12-27 15:36:22,362][19170] Fps is (10 sec: 20488.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 735666176. Throughput: 0: 10435.8, 1: 10142.0. Samples: 735640164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:22,363][19170] Avg episode reward: [(0, '951.280'), (1, '621.050')] [2023-12-27 15:36:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001435984_367640576.pth... [2023-12-27 15:36:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001434800_367337472.pth [2023-12-27 15:36:22,372][20347] Updated weights for policy 0, policy_version 1437332 (0.0006) [2023-12-27 15:36:22,438][20347] Updated weights for policy 0, policy_version 1437342 (0.0008) [2023-12-27 15:36:22,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001437352_368033792.pth... [2023-12-27 15:36:22,508][20347] Updated weights for policy 0, policy_version 1437352 (0.0008) [2023-12-27 15:36:22,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001436104_367714304.pth [2023-12-27 15:36:22,652][20381] Updated weights for policy 1, policy_version 1435993 (0.0008) [2023-12-27 15:36:22,718][20381] Updated weights for policy 1, policy_version 1436003 (0.0007) [2023-12-27 15:36:22,783][20381] Updated weights for policy 1, policy_version 1436013 (0.0008) [2023-12-27 15:36:23,237][20347] Updated weights for policy 0, policy_version 1437362 (0.0006) [2023-12-27 15:36:23,284][20347] Updated weights for policy 0, policy_version 1437372 (0.0006) [2023-12-27 15:36:23,330][20347] Updated weights for policy 0, policy_version 1437382 (0.0005) [2023-12-27 15:36:23,490][20381] Updated weights for policy 1, policy_version 1436023 (0.0007) [2023-12-27 15:36:23,546][20381] Updated weights for policy 1, policy_version 1436033 (0.0009) [2023-12-27 15:36:23,601][20381] Updated weights for policy 1, policy_version 1436043 (0.0010) [2023-12-27 15:36:23,967][20347] Updated weights for policy 0, policy_version 1437392 (0.0008) [2023-12-27 15:36:24,013][20347] Updated weights for policy 0, policy_version 1437402 (0.0008) [2023-12-27 15:36:24,060][20347] Updated weights for policy 0, policy_version 1437412 (0.0010) [2023-12-27 15:36:24,211][20381] Updated weights for policy 1, policy_version 1436053 (0.0009) [2023-12-27 15:36:24,264][20381] Updated weights for policy 1, policy_version 1436063 (0.0008) [2023-12-27 15:36:24,323][20381] Updated weights for policy 1, policy_version 1436073 (0.0009) [2023-12-27 15:36:24,818][20347] Updated weights for policy 0, policy_version 1437422 (0.0007) [2023-12-27 15:36:24,876][20347] Updated weights for policy 0, policy_version 1437432 (0.0005) [2023-12-27 15:36:24,917][20381] Updated weights for policy 1, policy_version 1436083 (0.0007) [2023-12-27 15:36:24,935][20347] Updated weights for policy 0, policy_version 1437442 (0.0005) [2023-12-27 15:36:24,959][20381] Updated weights for policy 1, policy_version 1436093 (0.0007) [2023-12-27 15:36:25,008][20381] Updated weights for policy 1, policy_version 1436103 (0.0007) [2023-12-27 15:36:25,602][20347] Updated weights for policy 0, policy_version 1437452 (0.0007) [2023-12-27 15:36:25,660][20381] Updated weights for policy 1, policy_version 1436113 (0.0007) [2023-12-27 15:36:25,662][20347] Updated weights for policy 0, policy_version 1437462 (0.0009) [2023-12-27 15:36:25,703][20381] Updated weights for policy 1, policy_version 1436123 (0.0005) [2023-12-27 15:36:25,710][20347] Updated weights for policy 0, policy_version 1437472 (0.0007) [2023-12-27 15:36:25,757][20381] Updated weights for policy 1, policy_version 1436133 (0.0005) [2023-12-27 15:36:25,820][20381] Updated weights for policy 1, policy_version 1436143 (0.0005) [2023-12-27 15:36:26,454][20347] Updated weights for policy 0, policy_version 1437482 (0.0006) [2023-12-27 15:36:26,490][20381] Updated weights for policy 1, policy_version 1436153 (0.0006) [2023-12-27 15:36:26,500][20347] Updated weights for policy 0, policy_version 1437492 (0.0007) [2023-12-27 15:36:26,551][20381] Updated weights for policy 1, policy_version 1436163 (0.0006) [2023-12-27 15:36:26,556][20347] Updated weights for policy 0, policy_version 1437502 (0.0007) [2023-12-27 15:36:26,606][20381] Updated weights for policy 1, policy_version 1436173 (0.0006) [2023-12-27 15:36:26,614][20347] Updated weights for policy 0, policy_version 1437512 (0.0006) [2023-12-27 15:36:27,142][20381] Updated weights for policy 1, policy_version 1436183 (0.0005) [2023-12-27 15:36:27,191][20381] Updated weights for policy 1, policy_version 1436193 (0.0005) [2023-12-27 15:36:27,243][20381] Updated weights for policy 1, policy_version 1436203 (0.0006) [2023-12-27 15:36:27,344][20347] Updated weights for policy 0, policy_version 1437522 (0.0007) [2023-12-27 15:36:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 735772672. Throughput: 0: 10382.1, 1: 10129.3. Samples: 735762420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:27,363][19170] Avg episode reward: [(0, '931.700'), (1, '427.830')] [2023-12-27 15:36:27,391][20347] Updated weights for policy 0, policy_version 1437532 (0.0008) [2023-12-27 15:36:27,438][20347] Updated weights for policy 0, policy_version 1437542 (0.0007) [2023-12-27 15:36:27,851][20381] Updated weights for policy 1, policy_version 1436213 (0.0007) [2023-12-27 15:36:27,908][20381] Updated weights for policy 1, policy_version 1436223 (0.0007) [2023-12-27 15:36:27,961][20381] Updated weights for policy 1, policy_version 1436233 (0.0006) [2023-12-27 15:36:28,150][20347] Updated weights for policy 0, policy_version 1437552 (0.0007) [2023-12-27 15:36:28,199][20347] Updated weights for policy 0, policy_version 1437562 (0.0008) [2023-12-27 15:36:28,248][20347] Updated weights for policy 0, policy_version 1437572 (0.0007) [2023-12-27 15:36:28,632][20381] Updated weights for policy 1, policy_version 1436243 (0.0008) [2023-12-27 15:36:28,685][20381] Updated weights for policy 1, policy_version 1436253 (0.0008) [2023-12-27 15:36:28,742][20381] Updated weights for policy 1, policy_version 1436263 (0.0008) [2023-12-27 15:36:28,982][20347] Updated weights for policy 0, policy_version 1437582 (0.0008) [2023-12-27 15:36:29,041][20347] Updated weights for policy 0, policy_version 1437592 (0.0007) [2023-12-27 15:36:29,093][20347] Updated weights for policy 0, policy_version 1437602 (0.0008) [2023-12-27 15:36:29,427][20381] Updated weights for policy 1, policy_version 1436273 (0.0008) [2023-12-27 15:36:29,479][20381] Updated weights for policy 1, policy_version 1436283 (0.0005) [2023-12-27 15:36:29,527][20381] Updated weights for policy 1, policy_version 1436293 (0.0005) [2023-12-27 15:36:29,583][20381] Updated weights for policy 1, policy_version 1436303 (0.0005) [2023-12-27 15:36:29,846][20347] Updated weights for policy 0, policy_version 1437612 (0.0009) [2023-12-27 15:36:29,902][20347] Updated weights for policy 0, policy_version 1437622 (0.0005) [2023-12-27 15:36:29,962][20347] Updated weights for policy 0, policy_version 1437632 (0.0008) [2023-12-27 15:36:30,217][20381] Updated weights for policy 1, policy_version 1436313 (0.0008) [2023-12-27 15:36:30,274][20381] Updated weights for policy 1, policy_version 1436323 (0.0008) [2023-12-27 15:36:30,330][20381] Updated weights for policy 1, policy_version 1436333 (0.0008) [2023-12-27 15:36:30,654][20347] Updated weights for policy 0, policy_version 1437642 (0.0008) [2023-12-27 15:36:30,712][20347] Updated weights for policy 0, policy_version 1437652 (0.0007) [2023-12-27 15:36:30,767][20347] Updated weights for policy 0, policy_version 1437662 (0.0006) [2023-12-27 15:36:30,819][20347] Updated weights for policy 0, policy_version 1437672 (0.0006) [2023-12-27 15:36:31,025][20381] Updated weights for policy 1, policy_version 1436343 (0.0008) [2023-12-27 15:36:31,092][20381] Updated weights for policy 1, policy_version 1436353 (0.0006) [2023-12-27 15:36:31,156][20381] Updated weights for policy 1, policy_version 1436363 (0.0008) [2023-12-27 15:36:31,547][20347] Updated weights for policy 0, policy_version 1437682 (0.0008) [2023-12-27 15:36:31,600][20347] Updated weights for policy 0, policy_version 1437692 (0.0008) [2023-12-27 15:36:31,664][20347] Updated weights for policy 0, policy_version 1437702 (0.0007) [2023-12-27 15:36:31,817][20381] Updated weights for policy 1, policy_version 1436373 (0.0007) [2023-12-27 15:36:31,870][20381] Updated weights for policy 1, policy_version 1436383 (0.0007) [2023-12-27 15:36:31,918][20381] Updated weights for policy 1, policy_version 1436393 (0.0006) [2023-12-27 15:36:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 735870976. Throughput: 0: 10301.4, 1: 10167.4. Samples: 735883980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:32,363][19170] Avg episode reward: [(0, '960.990'), (1, '457.190')] [2023-12-27 15:36:32,394][20347] Updated weights for policy 0, policy_version 1437712 (0.0008) [2023-12-27 15:36:32,458][20347] Updated weights for policy 0, policy_version 1437722 (0.0006) [2023-12-27 15:36:32,522][20347] Updated weights for policy 0, policy_version 1437732 (0.0006) [2023-12-27 15:36:32,627][20381] Updated weights for policy 1, policy_version 1436403 (0.0008) [2023-12-27 15:36:32,683][20381] Updated weights for policy 1, policy_version 1436413 (0.0008) [2023-12-27 15:36:32,737][20381] Updated weights for policy 1, policy_version 1436423 (0.0008) [2023-12-27 15:36:33,138][20347] Updated weights for policy 0, policy_version 1437742 (0.0009) [2023-12-27 15:36:33,192][20347] Updated weights for policy 0, policy_version 1437752 (0.0010) [2023-12-27 15:36:33,249][20347] Updated weights for policy 0, policy_version 1437762 (0.0006) [2023-12-27 15:36:33,357][20381] Updated weights for policy 1, policy_version 1436433 (0.0008) [2023-12-27 15:36:33,408][20381] Updated weights for policy 1, policy_version 1436443 (0.0007) [2023-12-27 15:36:33,462][20381] Updated weights for policy 1, policy_version 1436453 (0.0005) [2023-12-27 15:36:33,517][20381] Updated weights for policy 1, policy_version 1436463 (0.0005) [2023-12-27 15:36:33,786][20347] Updated weights for policy 0, policy_version 1437772 (0.0005) [2023-12-27 15:36:33,842][20347] Updated weights for policy 0, policy_version 1437782 (0.0005) [2023-12-27 15:36:33,902][20347] Updated weights for policy 0, policy_version 1437792 (0.0006) [2023-12-27 15:36:34,176][20381] Updated weights for policy 1, policy_version 1436473 (0.0008) [2023-12-27 15:36:34,226][20381] Updated weights for policy 1, policy_version 1436483 (0.0006) [2023-12-27 15:36:34,279][20381] Updated weights for policy 1, policy_version 1436493 (0.0008) [2023-12-27 15:36:34,509][20347] Updated weights for policy 0, policy_version 1437802 (0.0007) [2023-12-27 15:36:34,567][20347] Updated weights for policy 0, policy_version 1437812 (0.0006) [2023-12-27 15:36:34,630][20347] Updated weights for policy 0, policy_version 1437822 (0.0007) [2023-12-27 15:36:34,688][20347] Updated weights for policy 0, policy_version 1437832 (0.0007) [2023-12-27 15:36:35,033][20381] Updated weights for policy 1, policy_version 1436503 (0.0009) [2023-12-27 15:36:35,078][20381] Updated weights for policy 1, policy_version 1436513 (0.0010) [2023-12-27 15:36:35,129][20381] Updated weights for policy 1, policy_version 1436523 (0.0007) [2023-12-27 15:36:35,306][20347] Updated weights for policy 0, policy_version 1437842 (0.0006) [2023-12-27 15:36:35,355][20347] Updated weights for policy 0, policy_version 1437852 (0.0005) [2023-12-27 15:36:35,401][20347] Updated weights for policy 0, policy_version 1437862 (0.0005) [2023-12-27 15:36:35,781][20381] Updated weights for policy 1, policy_version 1436533 (0.0008) [2023-12-27 15:36:35,835][20381] Updated weights for policy 1, policy_version 1436543 (0.0007) [2023-12-27 15:36:35,891][20381] Updated weights for policy 1, policy_version 1436553 (0.0006) [2023-12-27 15:36:36,010][20347] Updated weights for policy 0, policy_version 1437872 (0.0005) [2023-12-27 15:36:36,075][20347] Updated weights for policy 0, policy_version 1437882 (0.0007) [2023-12-27 15:36:36,137][20347] Updated weights for policy 0, policy_version 1437892 (0.0006) [2023-12-27 15:36:36,473][20381] Updated weights for policy 1, policy_version 1436563 (0.0006) [2023-12-27 15:36:36,537][20381] Updated weights for policy 1, policy_version 1436573 (0.0008) [2023-12-27 15:36:36,595][20381] Updated weights for policy 1, policy_version 1436583 (0.0009) [2023-12-27 15:36:36,819][20347] Updated weights for policy 0, policy_version 1437902 (0.0007) [2023-12-27 15:36:36,871][20347] Updated weights for policy 0, policy_version 1437912 (0.0005) [2023-12-27 15:36:36,929][20347] Updated weights for policy 0, policy_version 1437922 (0.0009) [2023-12-27 15:36:37,289][20381] Updated weights for policy 1, policy_version 1436593 (0.0011) [2023-12-27 15:36:37,348][20381] Updated weights for policy 1, policy_version 1436603 (0.0011) [2023-12-27 15:36:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 735977472. Throughput: 0: 10367.6, 1: 10138.7. Samples: 735948444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:37,363][19170] Avg episode reward: [(0, '960.930'), (1, '415.040')] [2023-12-27 15:36:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001437928_368181248.pth... [2023-12-27 15:36:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001436744_367878144.pth [2023-12-27 15:36:37,405][20381] Updated weights for policy 1, policy_version 1436613 (0.0011) [2023-12-27 15:36:37,467][20381] Updated weights for policy 1, policy_version 1436623 (0.0011) [2023-12-27 15:36:37,470][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001436624_367804416.pth... [2023-12-27 15:36:37,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001435408_367493120.pth [2023-12-27 15:36:37,602][20347] Updated weights for policy 0, policy_version 1437932 (0.0010) [2023-12-27 15:36:37,661][20347] Updated weights for policy 0, policy_version 1437942 (0.0011) [2023-12-27 15:36:37,719][20347] Updated weights for policy 0, policy_version 1437952 (0.0009) [2023-12-27 15:36:38,162][20381] Updated weights for policy 1, policy_version 1436633 (0.0011) [2023-12-27 15:36:38,218][20381] Updated weights for policy 1, policy_version 1436643 (0.0010) [2023-12-27 15:36:38,274][20381] Updated weights for policy 1, policy_version 1436653 (0.0010) [2023-12-27 15:36:38,382][20347] Updated weights for policy 0, policy_version 1437962 (0.0011) [2023-12-27 15:36:38,438][20347] Updated weights for policy 0, policy_version 1437972 (0.0010) [2023-12-27 15:36:38,486][20347] Updated weights for policy 0, policy_version 1437982 (0.0010) [2023-12-27 15:36:38,535][20347] Updated weights for policy 0, policy_version 1437992 (0.0010) [2023-12-27 15:36:38,953][20381] Updated weights for policy 1, policy_version 1436663 (0.0008) [2023-12-27 15:36:39,005][20381] Updated weights for policy 1, policy_version 1436673 (0.0010) [2023-12-27 15:36:39,054][20381] Updated weights for policy 1, policy_version 1436683 (0.0010) [2023-12-27 15:36:39,232][20347] Updated weights for policy 0, policy_version 1438002 (0.0006) [2023-12-27 15:36:39,297][20347] Updated weights for policy 0, policy_version 1438012 (0.0008) [2023-12-27 15:36:39,368][20347] Updated weights for policy 0, policy_version 1438022 (0.0009) [2023-12-27 15:36:39,779][20381] Updated weights for policy 1, policy_version 1436693 (0.0011) [2023-12-27 15:36:39,849][20381] Updated weights for policy 1, policy_version 1436703 (0.0011) [2023-12-27 15:36:39,910][20381] Updated weights for policy 1, policy_version 1436713 (0.0011) [2023-12-27 15:36:40,077][20347] Updated weights for policy 0, policy_version 1438032 (0.0008) [2023-12-27 15:36:40,139][20347] Updated weights for policy 0, policy_version 1438042 (0.0007) [2023-12-27 15:36:40,205][20347] Updated weights for policy 0, policy_version 1438052 (0.0008) [2023-12-27 15:36:40,613][20381] Updated weights for policy 1, policy_version 1436723 (0.0009) [2023-12-27 15:36:40,677][20381] Updated weights for policy 1, policy_version 1436733 (0.0005) [2023-12-27 15:36:40,742][20381] Updated weights for policy 1, policy_version 1436743 (0.0005) [2023-12-27 15:36:40,842][20347] Updated weights for policy 0, policy_version 1438062 (0.0008) [2023-12-27 15:36:40,894][20347] Updated weights for policy 0, policy_version 1438072 (0.0008) [2023-12-27 15:36:40,942][20347] Updated weights for policy 0, policy_version 1438082 (0.0009) [2023-12-27 15:36:41,412][20381] Updated weights for policy 1, policy_version 1436753 (0.0006) [2023-12-27 15:36:41,480][20381] Updated weights for policy 1, policy_version 1436763 (0.0009) [2023-12-27 15:36:41,528][20381] Updated weights for policy 1, policy_version 1436773 (0.0008) [2023-12-27 15:36:41,587][20381] Updated weights for policy 1, policy_version 1436783 (0.0008) [2023-12-27 15:36:41,673][20347] Updated weights for policy 0, policy_version 1438092 (0.0009) [2023-12-27 15:36:41,739][20347] Updated weights for policy 0, policy_version 1438102 (0.0006) [2023-12-27 15:36:41,806][20347] Updated weights for policy 0, policy_version 1438112 (0.0006) [2023-12-27 15:36:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 736075776. Throughput: 0: 10350.7, 1: 10177.7. Samples: 736071528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:42,363][19170] Avg episode reward: [(0, '960.850'), (1, '512.480')] [2023-12-27 15:36:42,392][20381] Updated weights for policy 1, policy_version 1436793 (0.0008) [2023-12-27 15:36:42,448][20381] Updated weights for policy 1, policy_version 1436803 (0.0008) [2023-12-27 15:36:42,468][20347] Updated weights for policy 0, policy_version 1438122 (0.0007) [2023-12-27 15:36:42,504][20381] Updated weights for policy 1, policy_version 1436813 (0.0008) [2023-12-27 15:36:42,535][20347] Updated weights for policy 0, policy_version 1438132 (0.0007) [2023-12-27 15:36:42,599][20347] Updated weights for policy 0, policy_version 1438142 (0.0007) [2023-12-27 15:36:42,661][20347] Updated weights for policy 0, policy_version 1438152 (0.0006) [2023-12-27 15:36:43,128][20381] Updated weights for policy 1, policy_version 1436823 (0.0006) [2023-12-27 15:36:43,201][20381] Updated weights for policy 1, policy_version 1436833 (0.0006) [2023-12-27 15:36:43,250][20381] Updated weights for policy 1, policy_version 1436843 (0.0007) [2023-12-27 15:36:43,264][20347] Updated weights for policy 0, policy_version 1438162 (0.0006) [2023-12-27 15:36:43,319][20347] Updated weights for policy 0, policy_version 1438172 (0.0008) [2023-12-27 15:36:43,367][20347] Updated weights for policy 0, policy_version 1438182 (0.0010) [2023-12-27 15:36:43,895][20381] Updated weights for policy 1, policy_version 1436853 (0.0007) [2023-12-27 15:36:43,974][20381] Updated weights for policy 1, policy_version 1436863 (0.0007) [2023-12-27 15:36:43,978][20347] Updated weights for policy 0, policy_version 1438192 (0.0011) [2023-12-27 15:36:44,020][20381] Updated weights for policy 1, policy_version 1436873 (0.0006) [2023-12-27 15:36:44,045][20347] Updated weights for policy 0, policy_version 1438202 (0.0010) [2023-12-27 15:36:44,100][20347] Updated weights for policy 0, policy_version 1438212 (0.0010) [2023-12-27 15:36:44,726][20381] Updated weights for policy 1, policy_version 1436883 (0.0008) [2023-12-27 15:36:44,769][20347] Updated weights for policy 0, policy_version 1438222 (0.0008) [2023-12-27 15:36:44,786][20381] Updated weights for policy 1, policy_version 1436893 (0.0007) [2023-12-27 15:36:44,821][20347] Updated weights for policy 0, policy_version 1438232 (0.0007) [2023-12-27 15:36:44,848][20381] Updated weights for policy 1, policy_version 1436903 (0.0010) [2023-12-27 15:36:44,885][20347] Updated weights for policy 0, policy_version 1438242 (0.0007) [2023-12-27 15:36:45,542][20381] Updated weights for policy 1, policy_version 1436913 (0.0011) [2023-12-27 15:36:45,566][20347] Updated weights for policy 0, policy_version 1438252 (0.0007) [2023-12-27 15:36:45,602][20381] Updated weights for policy 1, policy_version 1436923 (0.0010) [2023-12-27 15:36:45,627][20347] Updated weights for policy 0, policy_version 1438262 (0.0008) [2023-12-27 15:36:45,655][20381] Updated weights for policy 1, policy_version 1436933 (0.0009) [2023-12-27 15:36:45,680][20347] Updated weights for policy 0, policy_version 1438272 (0.0007) [2023-12-27 15:36:45,715][20381] Updated weights for policy 1, policy_version 1436943 (0.0008) [2023-12-27 15:36:46,289][20381] Updated weights for policy 1, policy_version 1436953 (0.0005) [2023-12-27 15:36:46,346][20381] Updated weights for policy 1, policy_version 1436963 (0.0005) [2023-12-27 15:36:46,353][20347] Updated weights for policy 0, policy_version 1438282 (0.0007) [2023-12-27 15:36:46,408][20381] Updated weights for policy 1, policy_version 1436973 (0.0011) [2023-12-27 15:36:46,411][20347] Updated weights for policy 0, policy_version 1438292 (0.0005) [2023-12-27 15:36:46,468][20347] Updated weights for policy 0, policy_version 1438302 (0.0005) [2023-12-27 15:36:46,530][20347] Updated weights for policy 0, policy_version 1438312 (0.0005) [2023-12-27 15:36:46,990][20381] Updated weights for policy 1, policy_version 1436983 (0.0007) [2023-12-27 15:36:47,028][20347] Updated weights for policy 0, policy_version 1438322 (0.0005) [2023-12-27 15:36:47,040][20381] Updated weights for policy 1, policy_version 1436993 (0.0009) [2023-12-27 15:36:47,084][20347] Updated weights for policy 0, policy_version 1438332 (0.0008) [2023-12-27 15:36:47,089][20381] Updated weights for policy 1, policy_version 1437003 (0.0010) [2023-12-27 15:36:47,135][20347] Updated weights for policy 0, policy_version 1438342 (0.0010) [2023-12-27 15:36:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 736190464. Throughput: 0: 10361.8, 1: 10240.1. Samples: 736197520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:47,364][19170] Avg episode reward: [(0, '941.530'), (1, '628.390')] [2023-12-27 15:36:47,706][20347] Updated weights for policy 0, policy_version 1438352 (0.0010) [2023-12-27 15:36:47,764][20347] Updated weights for policy 0, policy_version 1438362 (0.0006) [2023-12-27 15:36:47,780][20381] Updated weights for policy 1, policy_version 1437013 (0.0008) [2023-12-27 15:36:47,821][20347] Updated weights for policy 0, policy_version 1438372 (0.0010) [2023-12-27 15:36:47,831][20381] Updated weights for policy 1, policy_version 1437023 (0.0006) [2023-12-27 15:36:47,888][20381] Updated weights for policy 1, policy_version 1437033 (0.0007) [2023-12-27 15:36:48,485][20347] Updated weights for policy 0, policy_version 1438382 (0.0009) [2023-12-27 15:36:48,536][20347] Updated weights for policy 0, policy_version 1438392 (0.0008) [2023-12-27 15:36:48,598][20347] Updated weights for policy 0, policy_version 1438402 (0.0008) [2023-12-27 15:36:48,617][20381] Updated weights for policy 1, policy_version 1437043 (0.0007) [2023-12-27 15:36:48,673][20381] Updated weights for policy 1, policy_version 1437053 (0.0007) [2023-12-27 15:36:48,726][20381] Updated weights for policy 1, policy_version 1437063 (0.0006) [2023-12-27 15:36:49,323][20347] Updated weights for policy 0, policy_version 1438412 (0.0009) [2023-12-27 15:36:49,386][20381] Updated weights for policy 1, policy_version 1437073 (0.0009) [2023-12-27 15:36:49,392][20347] Updated weights for policy 0, policy_version 1438422 (0.0008) [2023-12-27 15:36:49,448][20381] Updated weights for policy 1, policy_version 1437083 (0.0007) [2023-12-27 15:36:49,449][20347] Updated weights for policy 0, policy_version 1438432 (0.0010) [2023-12-27 15:36:49,504][20381] Updated weights for policy 1, policy_version 1437093 (0.0006) [2023-12-27 15:36:49,560][20381] Updated weights for policy 1, policy_version 1437103 (0.0008) [2023-12-27 15:36:50,117][20347] Updated weights for policy 0, policy_version 1438442 (0.0009) [2023-12-27 15:36:50,181][20347] Updated weights for policy 0, policy_version 1438452 (0.0006) [2023-12-27 15:36:50,240][20347] Updated weights for policy 0, policy_version 1438462 (0.0010) [2023-12-27 15:36:50,280][20381] Updated weights for policy 1, policy_version 1437113 (0.0006) [2023-12-27 15:36:50,302][20347] Updated weights for policy 0, policy_version 1438472 (0.0010) [2023-12-27 15:36:50,337][20381] Updated weights for policy 1, policy_version 1437123 (0.0007) [2023-12-27 15:36:50,394][20381] Updated weights for policy 1, policy_version 1437133 (0.0007) [2023-12-27 15:36:50,961][20347] Updated weights for policy 0, policy_version 1438482 (0.0011) [2023-12-27 15:36:51,021][20347] Updated weights for policy 0, policy_version 1438492 (0.0010) [2023-12-27 15:36:51,045][20381] Updated weights for policy 1, policy_version 1437143 (0.0007) [2023-12-27 15:36:51,088][20347] Updated weights for policy 0, policy_version 1438502 (0.0009) [2023-12-27 15:36:51,105][20381] Updated weights for policy 1, policy_version 1437153 (0.0006) [2023-12-27 15:36:51,168][20381] Updated weights for policy 1, policy_version 1437163 (0.0009) [2023-12-27 15:36:51,815][20347] Updated weights for policy 0, policy_version 1438512 (0.0008) [2023-12-27 15:36:51,870][20381] Updated weights for policy 1, policy_version 1437173 (0.0006) [2023-12-27 15:36:51,878][20347] Updated weights for policy 0, policy_version 1438522 (0.0007) [2023-12-27 15:36:51,929][20381] Updated weights for policy 1, policy_version 1437183 (0.0006) [2023-12-27 15:36:51,939][20347] Updated weights for policy 0, policy_version 1438532 (0.0007) [2023-12-27 15:36:51,991][20381] Updated weights for policy 1, policy_version 1437193 (0.0006) [2023-12-27 15:36:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 736288768. Throughput: 0: 10362.1, 1: 10208.7. Samples: 736258828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:52,363][19170] Avg episode reward: [(0, '922.070'), (1, '571.150')] [2023-12-27 15:36:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001437200_367951872.pth... [2023-12-27 15:36:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001438536_368336896.pth... [2023-12-27 15:36:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001437352_368033792.pth [2023-12-27 15:36:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001435984_367640576.pth [2023-12-27 15:36:52,655][20347] Updated weights for policy 0, policy_version 1438542 (0.0007) [2023-12-27 15:36:52,655][20381] Updated weights for policy 1, policy_version 1437203 (0.0009) [2023-12-27 15:36:52,712][20381] Updated weights for policy 1, policy_version 1437213 (0.0005) [2023-12-27 15:36:52,715][20347] Updated weights for policy 0, policy_version 1438552 (0.0008) [2023-12-27 15:36:52,767][20381] Updated weights for policy 1, policy_version 1437223 (0.0005) [2023-12-27 15:36:52,772][20347] Updated weights for policy 0, policy_version 1438562 (0.0008) [2023-12-27 15:36:53,378][20381] Updated weights for policy 1, policy_version 1437233 (0.0006) [2023-12-27 15:36:53,434][20381] Updated weights for policy 1, policy_version 1437243 (0.0009) [2023-12-27 15:36:53,472][20347] Updated weights for policy 0, policy_version 1438572 (0.0006) [2023-12-27 15:36:53,488][20381] Updated weights for policy 1, policy_version 1437253 (0.0008) [2023-12-27 15:36:53,529][20347] Updated weights for policy 0, policy_version 1438582 (0.0006) [2023-12-27 15:36:53,545][20381] Updated weights for policy 1, policy_version 1437263 (0.0008) [2023-12-27 15:36:53,583][20347] Updated weights for policy 0, policy_version 1438592 (0.0007) [2023-12-27 15:36:54,163][20347] Updated weights for policy 0, policy_version 1438602 (0.0008) [2023-12-27 15:36:54,219][20347] Updated weights for policy 0, policy_version 1438612 (0.0006) [2023-12-27 15:36:54,246][20381] Updated weights for policy 1, policy_version 1437273 (0.0010) [2023-12-27 15:36:54,267][20347] Updated weights for policy 0, policy_version 1438622 (0.0005) [2023-12-27 15:36:54,292][20381] Updated weights for policy 1, policy_version 1437283 (0.0010) [2023-12-27 15:36:54,310][20347] Updated weights for policy 0, policy_version 1438632 (0.0007) [2023-12-27 15:36:54,344][20381] Updated weights for policy 1, policy_version 1437293 (0.0011) [2023-12-27 15:36:54,891][20347] Updated weights for policy 0, policy_version 1438642 (0.0007) [2023-12-27 15:36:54,948][20347] Updated weights for policy 0, policy_version 1438652 (0.0007) [2023-12-27 15:36:54,997][20347] Updated weights for policy 0, policy_version 1438662 (0.0007) [2023-12-27 15:36:55,032][20381] Updated weights for policy 1, policy_version 1437303 (0.0011) [2023-12-27 15:36:55,086][20381] Updated weights for policy 1, policy_version 1437313 (0.0010) [2023-12-27 15:36:55,149][20381] Updated weights for policy 1, policy_version 1437323 (0.0008) [2023-12-27 15:36:55,634][20347] Updated weights for policy 0, policy_version 1438672 (0.0005) [2023-12-27 15:36:55,690][20347] Updated weights for policy 0, policy_version 1438682 (0.0009) [2023-12-27 15:36:55,738][20347] Updated weights for policy 0, policy_version 1438692 (0.0010) [2023-12-27 15:36:55,814][20381] Updated weights for policy 1, policy_version 1437333 (0.0007) [2023-12-27 15:36:55,874][20381] Updated weights for policy 1, policy_version 1437343 (0.0006) [2023-12-27 15:36:55,934][20381] Updated weights for policy 1, policy_version 1437353 (0.0005) [2023-12-27 15:36:56,351][20347] Updated weights for policy 0, policy_version 1438702 (0.0010) [2023-12-27 15:36:56,406][20347] Updated weights for policy 0, policy_version 1438712 (0.0010) [2023-12-27 15:36:56,467][20347] Updated weights for policy 0, policy_version 1438722 (0.0010) [2023-12-27 15:36:56,491][20381] Updated weights for policy 1, policy_version 1437363 (0.0008) [2023-12-27 15:36:56,542][20381] Updated weights for policy 1, policy_version 1437373 (0.0005) [2023-12-27 15:36:56,591][20381] Updated weights for policy 1, policy_version 1437383 (0.0005) [2023-12-27 15:36:57,147][20347] Updated weights for policy 0, policy_version 1438732 (0.0010) [2023-12-27 15:36:57,204][20347] Updated weights for policy 0, policy_version 1438742 (0.0010) [2023-12-27 15:36:57,226][20381] Updated weights for policy 1, policy_version 1437393 (0.0008) [2023-12-27 15:36:57,262][20347] Updated weights for policy 0, policy_version 1438752 (0.0010) [2023-12-27 15:36:57,294][20381] Updated weights for policy 1, policy_version 1437403 (0.0006) [2023-12-27 15:36:57,351][20381] Updated weights for policy 1, policy_version 1437413 (0.0008) [2023-12-27 15:36:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 736395264. Throughput: 0: 10411.3, 1: 10297.2. Samples: 736385124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:36:57,363][19170] Avg episode reward: [(0, '941.470'), (1, '582.270')] [2023-12-27 15:36:57,408][20381] Updated weights for policy 1, policy_version 1437423 (0.0009) [2023-12-27 15:36:57,891][20347] Updated weights for policy 0, policy_version 1438762 (0.0010) [2023-12-27 15:36:57,925][20381] Updated weights for policy 1, policy_version 1437433 (0.0005) [2023-12-27 15:36:57,938][20347] Updated weights for policy 0, policy_version 1438772 (0.0010) [2023-12-27 15:36:57,981][20381] Updated weights for policy 1, policy_version 1437443 (0.0005) [2023-12-27 15:36:57,986][20347] Updated weights for policy 0, policy_version 1438782 (0.0010) [2023-12-27 15:36:58,035][20381] Updated weights for policy 1, policy_version 1437453 (0.0005) [2023-12-27 15:36:58,040][20347] Updated weights for policy 0, policy_version 1438792 (0.0010) [2023-12-27 15:36:58,682][20381] Updated weights for policy 1, policy_version 1437463 (0.0007) [2023-12-27 15:36:58,748][20381] Updated weights for policy 1, policy_version 1437473 (0.0009) [2023-12-27 15:36:58,772][20347] Updated weights for policy 0, policy_version 1438802 (0.0008) [2023-12-27 15:36:58,818][20381] Updated weights for policy 1, policy_version 1437483 (0.0008) [2023-12-27 15:36:58,838][20347] Updated weights for policy 0, policy_version 1438812 (0.0010) [2023-12-27 15:36:58,905][20347] Updated weights for policy 0, policy_version 1438822 (0.0009) [2023-12-27 15:36:59,579][20381] Updated weights for policy 1, policy_version 1437493 (0.0008) [2023-12-27 15:36:59,613][20347] Updated weights for policy 0, policy_version 1438832 (0.0007) [2023-12-27 15:36:59,637][20381] Updated weights for policy 1, policy_version 1437503 (0.0006) [2023-12-27 15:36:59,673][20347] Updated weights for policy 0, policy_version 1438842 (0.0005) [2023-12-27 15:36:59,694][20381] Updated weights for policy 1, policy_version 1437513 (0.0006) [2023-12-27 15:36:59,730][20347] Updated weights for policy 0, policy_version 1438852 (0.0005) [2023-12-27 15:37:00,368][20381] Updated weights for policy 1, policy_version 1437523 (0.0006) [2023-12-27 15:37:00,418][20381] Updated weights for policy 1, policy_version 1437533 (0.0007) [2023-12-27 15:37:00,433][20347] Updated weights for policy 0, policy_version 1438862 (0.0008) [2023-12-27 15:37:00,473][20381] Updated weights for policy 1, policy_version 1437543 (0.0008) [2023-12-27 15:37:00,489][20347] Updated weights for policy 0, policy_version 1438872 (0.0009) [2023-12-27 15:37:00,545][20347] Updated weights for policy 0, policy_version 1438882 (0.0007) [2023-12-27 15:37:01,175][20347] Updated weights for policy 0, policy_version 1438892 (0.0007) [2023-12-27 15:37:01,179][20381] Updated weights for policy 1, policy_version 1437553 (0.0007) [2023-12-27 15:37:01,234][20347] Updated weights for policy 0, policy_version 1438902 (0.0007) [2023-12-27 15:37:01,234][20381] Updated weights for policy 1, policy_version 1437563 (0.0006) [2023-12-27 15:37:01,290][20347] Updated weights for policy 0, policy_version 1438912 (0.0007) [2023-12-27 15:37:01,295][20381] Updated weights for policy 1, policy_version 1437573 (0.0007) [2023-12-27 15:37:01,350][20381] Updated weights for policy 1, policy_version 1437583 (0.0007) [2023-12-27 15:37:02,005][20347] Updated weights for policy 0, policy_version 1438922 (0.0007) [2023-12-27 15:37:02,044][20381] Updated weights for policy 1, policy_version 1437593 (0.0006) [2023-12-27 15:37:02,063][20347] Updated weights for policy 0, policy_version 1438932 (0.0007) [2023-12-27 15:37:02,098][20381] Updated weights for policy 1, policy_version 1437603 (0.0007) [2023-12-27 15:37:02,122][20347] Updated weights for policy 0, policy_version 1438942 (0.0006) [2023-12-27 15:37:02,156][20381] Updated weights for policy 1, policy_version 1437613 (0.0005) [2023-12-27 15:37:02,177][20347] Updated weights for policy 0, policy_version 1438952 (0.0007) [2023-12-27 15:37:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 736501760. Throughput: 0: 10301.7, 1: 10340.3. Samples: 736508836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:02,363][19170] Avg episode reward: [(0, '970.700'), (1, '597.170')] [2023-12-27 15:37:02,832][20381] Updated weights for policy 1, policy_version 1437623 (0.0007) [2023-12-27 15:37:02,884][20347] Updated weights for policy 0, policy_version 1438962 (0.0006) [2023-12-27 15:37:02,885][20291] KL-divergence is very high: 256.1837 [2023-12-27 15:37:02,886][20381] Updated weights for policy 1, policy_version 1437633 (0.0006) [2023-12-27 15:37:02,933][20291] KL-divergence is very high: 373.5442 [2023-12-27 15:37:02,939][20381] Updated weights for policy 1, policy_version 1437643 (0.0006) [2023-12-27 15:37:02,945][20347] Updated weights for policy 0, policy_version 1438972 (0.0008) [2023-12-27 15:37:02,979][20291] KL-divergence is very high: 305.3998 [2023-12-27 15:37:02,999][20347] Updated weights for policy 0, policy_version 1438982 (0.0007) [2023-12-27 15:37:03,482][20381] Updated weights for policy 1, policy_version 1437653 (0.0007) [2023-12-27 15:37:03,531][20381] Updated weights for policy 1, policy_version 1437663 (0.0007) [2023-12-27 15:37:03,589][20381] Updated weights for policy 1, policy_version 1437673 (0.0005) [2023-12-27 15:37:03,665][20291] KL-divergence is very high: 123.1187 [2023-12-27 15:37:03,671][20347] Updated weights for policy 0, policy_version 1438992 (0.0009) [2023-12-27 15:37:03,719][20347] Updated weights for policy 0, policy_version 1439002 (0.0010) [2023-12-27 15:37:03,777][20347] Updated weights for policy 0, policy_version 1439012 (0.0010) [2023-12-27 15:37:03,793][20291] KL-divergence is very high: 193.1033 [2023-12-27 15:37:04,262][20381] Updated weights for policy 1, policy_version 1437683 (0.0006) [2023-12-27 15:37:04,320][20381] Updated weights for policy 1, policy_version 1437693 (0.0007) [2023-12-27 15:37:04,379][20381] Updated weights for policy 1, policy_version 1437703 (0.0007) [2023-12-27 15:37:04,496][20347] Updated weights for policy 0, policy_version 1439022 (0.0010) [2023-12-27 15:37:04,547][20347] Updated weights for policy 0, policy_version 1439032 (0.0010) [2023-12-27 15:37:04,598][20347] Updated weights for policy 0, policy_version 1439042 (0.0010) [2023-12-27 15:37:05,110][20381] Updated weights for policy 1, policy_version 1437713 (0.0008) [2023-12-27 15:37:05,162][20381] Updated weights for policy 1, policy_version 1437723 (0.0009) [2023-12-27 15:37:05,206][20347] Updated weights for policy 0, policy_version 1439052 (0.0008) [2023-12-27 15:37:05,221][20381] Updated weights for policy 1, policy_version 1437733 (0.0008) [2023-12-27 15:37:05,260][20347] Updated weights for policy 0, policy_version 1439062 (0.0005) [2023-12-27 15:37:05,274][20381] Updated weights for policy 1, policy_version 1437743 (0.0008) [2023-12-27 15:37:05,314][20347] Updated weights for policy 0, policy_version 1439072 (0.0005) [2023-12-27 15:37:05,939][20347] Updated weights for policy 0, policy_version 1439082 (0.0008) [2023-12-27 15:37:05,988][20381] Updated weights for policy 1, policy_version 1437753 (0.0006) [2023-12-27 15:37:06,000][20347] Updated weights for policy 0, policy_version 1439092 (0.0010) [2023-12-27 15:37:06,034][20381] Updated weights for policy 1, policy_version 1437763 (0.0007) [2023-12-27 15:37:06,054][20347] Updated weights for policy 0, policy_version 1439102 (0.0010) [2023-12-27 15:37:06,079][20381] Updated weights for policy 1, policy_version 1437773 (0.0005) [2023-12-27 15:37:06,105][20347] Updated weights for policy 0, policy_version 1439112 (0.0010) [2023-12-27 15:37:06,740][20347] Updated weights for policy 0, policy_version 1439122 (0.0007) [2023-12-27 15:37:06,798][20347] Updated weights for policy 0, policy_version 1439132 (0.0007) [2023-12-27 15:37:06,851][20381] Updated weights for policy 1, policy_version 1437783 (0.0006) [2023-12-27 15:37:06,854][20347] Updated weights for policy 0, policy_version 1439142 (0.0010) [2023-12-27 15:37:06,903][20381] Updated weights for policy 1, policy_version 1437793 (0.0007) [2023-12-27 15:37:06,963][20381] Updated weights for policy 1, policy_version 1437803 (0.0008) [2023-12-27 15:37:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 736600064. Throughput: 0: 10304.4, 1: 10373.6. Samples: 736570672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:07,363][19170] Avg episode reward: [(0, '941.420'), (1, '617.630')] [2023-12-27 15:37:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001439144_368492544.pth... [2023-12-27 15:37:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001437808_368107520.pth... [2023-12-27 15:37:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001436624_367804416.pth [2023-12-27 15:37:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001437928_368181248.pth [2023-12-27 15:37:07,435][20347] Updated weights for policy 0, policy_version 1439152 (0.0007) [2023-12-27 15:37:07,486][20347] Updated weights for policy 0, policy_version 1439162 (0.0008) [2023-12-27 15:37:07,535][20347] Updated weights for policy 0, policy_version 1439172 (0.0007) [2023-12-27 15:37:07,656][20381] Updated weights for policy 1, policy_version 1437813 (0.0008) [2023-12-27 15:37:07,709][20381] Updated weights for policy 1, policy_version 1437823 (0.0007) [2023-12-27 15:37:07,769][20381] Updated weights for policy 1, policy_version 1437833 (0.0008) [2023-12-27 15:37:08,156][20347] Updated weights for policy 0, policy_version 1439182 (0.0009) [2023-12-27 15:37:08,207][20347] Updated weights for policy 0, policy_version 1439192 (0.0009) [2023-12-27 15:37:08,259][20347] Updated weights for policy 0, policy_version 1439202 (0.0007) [2023-12-27 15:37:08,471][20381] Updated weights for policy 1, policy_version 1437843 (0.0008) [2023-12-27 15:37:08,534][20381] Updated weights for policy 1, policy_version 1437853 (0.0009) [2023-12-27 15:37:08,597][20381] Updated weights for policy 1, policy_version 1437863 (0.0011) [2023-12-27 15:37:08,978][20347] Updated weights for policy 0, policy_version 1439212 (0.0009) [2023-12-27 15:37:09,035][20347] Updated weights for policy 0, policy_version 1439222 (0.0010) [2023-12-27 15:37:09,094][20347] Updated weights for policy 0, policy_version 1439232 (0.0010) [2023-12-27 15:37:09,099][20291] KL-divergence is very high: 322.1039 [2023-12-27 15:37:09,106][20291] KL-divergence is very high: 379.6081 [2023-12-27 15:37:09,268][20381] Updated weights for policy 1, policy_version 1437873 (0.0011) [2023-12-27 15:37:09,327][20381] Updated weights for policy 1, policy_version 1437883 (0.0010) [2023-12-27 15:37:09,395][20381] Updated weights for policy 1, policy_version 1437893 (0.0009) [2023-12-27 15:37:09,461][20381] Updated weights for policy 1, policy_version 1437903 (0.0009) [2023-12-27 15:37:09,771][20347] Updated weights for policy 0, policy_version 1439242 (0.0010) [2023-12-27 15:37:09,829][20347] Updated weights for policy 0, policy_version 1439252 (0.0007) [2023-12-27 15:37:09,901][20347] Updated weights for policy 0, policy_version 1439262 (0.0007) [2023-12-27 15:37:09,975][20347] Updated weights for policy 0, policy_version 1439272 (0.0008) [2023-12-27 15:37:10,195][20381] Updated weights for policy 1, policy_version 1437913 (0.0006) [2023-12-27 15:37:10,254][20381] Updated weights for policy 1, policy_version 1437923 (0.0008) [2023-12-27 15:37:10,311][20381] Updated weights for policy 1, policy_version 1437933 (0.0007) [2023-12-27 15:37:10,642][20347] Updated weights for policy 0, policy_version 1439282 (0.0006) [2023-12-27 15:37:10,709][20347] Updated weights for policy 0, policy_version 1439292 (0.0005) [2023-12-27 15:37:10,768][20347] Updated weights for policy 0, policy_version 1439302 (0.0006) [2023-12-27 15:37:10,987][20381] Updated weights for policy 1, policy_version 1437943 (0.0008) [2023-12-27 15:37:11,044][20381] Updated weights for policy 1, policy_version 1437953 (0.0008) [2023-12-27 15:37:11,098][20381] Updated weights for policy 1, policy_version 1437963 (0.0009) [2023-12-27 15:37:11,408][20347] Updated weights for policy 0, policy_version 1439312 (0.0008) [2023-12-27 15:37:11,473][20347] Updated weights for policy 0, policy_version 1439322 (0.0011) [2023-12-27 15:37:11,523][20347] Updated weights for policy 0, policy_version 1439332 (0.0009) [2023-12-27 15:37:11,839][20381] Updated weights for policy 1, policy_version 1437973 (0.0008) [2023-12-27 15:37:11,892][20381] Updated weights for policy 1, policy_version 1437983 (0.0009) [2023-12-27 15:37:11,946][20381] Updated weights for policy 1, policy_version 1437993 (0.0008) [2023-12-27 15:37:12,260][20347] Updated weights for policy 0, policy_version 1439342 (0.0008) [2023-12-27 15:37:12,319][20347] Updated weights for policy 0, policy_version 1439352 (0.0008) [2023-12-27 15:37:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20618.0, 300 sec: 20549.4). Total num frames: 736698368. Throughput: 0: 10408.4, 1: 10293.5. Samples: 736694008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:12,363][19170] Avg episode reward: [(0, '931.580'), (1, '610.340')] [2023-12-27 15:37:12,384][20347] Updated weights for policy 0, policy_version 1439362 (0.0008) [2023-12-27 15:37:12,697][20381] Updated weights for policy 1, policy_version 1438003 (0.0008) [2023-12-27 15:37:12,753][20381] Updated weights for policy 1, policy_version 1438013 (0.0008) [2023-12-27 15:37:12,803][20381] Updated weights for policy 1, policy_version 1438023 (0.0007) [2023-12-27 15:37:13,024][20347] Updated weights for policy 0, policy_version 1439372 (0.0007) [2023-12-27 15:37:13,074][20347] Updated weights for policy 0, policy_version 1439382 (0.0005) [2023-12-27 15:37:13,133][20347] Updated weights for policy 0, policy_version 1439392 (0.0007) [2023-12-27 15:37:13,511][20381] Updated weights for policy 1, policy_version 1438033 (0.0007) [2023-12-27 15:37:13,566][20381] Updated weights for policy 1, policy_version 1438043 (0.0008) [2023-12-27 15:37:13,625][20381] Updated weights for policy 1, policy_version 1438053 (0.0007) [2023-12-27 15:37:13,681][20381] Updated weights for policy 1, policy_version 1438063 (0.0007) [2023-12-27 15:37:13,788][20347] Updated weights for policy 0, policy_version 1439402 (0.0008) [2023-12-27 15:37:13,844][20347] Updated weights for policy 0, policy_version 1439412 (0.0007) [2023-12-27 15:37:13,906][20347] Updated weights for policy 0, policy_version 1439422 (0.0007) [2023-12-27 15:37:13,958][20347] Updated weights for policy 0, policy_version 1439432 (0.0009) [2023-12-27 15:37:14,304][20381] Updated weights for policy 1, policy_version 1438073 (0.0007) [2023-12-27 15:37:14,368][20381] Updated weights for policy 1, policy_version 1438083 (0.0007) [2023-12-27 15:37:14,417][20381] Updated weights for policy 1, policy_version 1438093 (0.0008) [2023-12-27 15:37:14,663][20347] Updated weights for policy 0, policy_version 1439442 (0.0008) [2023-12-27 15:37:14,707][20347] Updated weights for policy 0, policy_version 1439452 (0.0008) [2023-12-27 15:37:14,756][20347] Updated weights for policy 0, policy_version 1439462 (0.0007) [2023-12-27 15:37:15,120][20381] Updated weights for policy 1, policy_version 1438103 (0.0008) [2023-12-27 15:37:15,179][20381] Updated weights for policy 1, policy_version 1438113 (0.0008) [2023-12-27 15:37:15,225][20381] Updated weights for policy 1, policy_version 1438123 (0.0007) [2023-12-27 15:37:15,493][20347] Updated weights for policy 0, policy_version 1439472 (0.0007) [2023-12-27 15:37:15,543][20347] Updated weights for policy 0, policy_version 1439482 (0.0007) [2023-12-27 15:37:15,605][20347] Updated weights for policy 0, policy_version 1439492 (0.0008) [2023-12-27 15:37:15,833][20381] Updated weights for policy 1, policy_version 1438133 (0.0007) [2023-12-27 15:37:15,894][20381] Updated weights for policy 1, policy_version 1438143 (0.0006) [2023-12-27 15:37:15,953][20381] Updated weights for policy 1, policy_version 1438153 (0.0008) [2023-12-27 15:37:16,333][20347] Updated weights for policy 0, policy_version 1439502 (0.0007) [2023-12-27 15:37:16,391][20347] Updated weights for policy 0, policy_version 1439512 (0.0006) [2023-12-27 15:37:16,446][20347] Updated weights for policy 0, policy_version 1439522 (0.0005) [2023-12-27 15:37:16,591][20381] Updated weights for policy 1, policy_version 1438163 (0.0008) [2023-12-27 15:37:16,640][20381] Updated weights for policy 1, policy_version 1438173 (0.0009) [2023-12-27 15:37:16,684][20381] Updated weights for policy 1, policy_version 1438183 (0.0008) [2023-12-27 15:37:16,985][20347] Updated weights for policy 0, policy_version 1439532 (0.0005) [2023-12-27 15:37:17,034][20347] Updated weights for policy 0, policy_version 1439542 (0.0005) [2023-12-27 15:37:17,083][20347] Updated weights for policy 0, policy_version 1439552 (0.0005) [2023-12-27 15:37:17,361][20381] Updated weights for policy 1, policy_version 1438193 (0.0008) [2023-12-27 15:37:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 736804864. Throughput: 0: 10472.4, 1: 10265.3. Samples: 736817176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:17,363][19170] Avg episode reward: [(0, '951.050'), (1, '568.290')] [2023-12-27 15:37:17,416][20381] Updated weights for policy 1, policy_version 1438203 (0.0010) [2023-12-27 15:37:17,467][20381] Updated weights for policy 1, policy_version 1438213 (0.0010) [2023-12-27 15:37:17,519][20381] Updated weights for policy 1, policy_version 1438223 (0.0009) [2023-12-27 15:37:17,649][20347] Updated weights for policy 0, policy_version 1439562 (0.0006) [2023-12-27 15:37:17,707][20347] Updated weights for policy 0, policy_version 1439572 (0.0005) [2023-12-27 15:37:17,755][20347] Updated weights for policy 0, policy_version 1439582 (0.0006) [2023-12-27 15:37:17,801][20347] Updated weights for policy 0, policy_version 1439592 (0.0006) [2023-12-27 15:37:18,167][20381] Updated weights for policy 1, policy_version 1438233 (0.0005) [2023-12-27 15:37:18,236][20381] Updated weights for policy 1, policy_version 1438243 (0.0005) [2023-12-27 15:37:18,310][20381] Updated weights for policy 1, policy_version 1438253 (0.0006) [2023-12-27 15:37:18,432][20347] Updated weights for policy 0, policy_version 1439602 (0.0008) [2023-12-27 15:37:18,480][20347] Updated weights for policy 0, policy_version 1439612 (0.0008) [2023-12-27 15:37:18,532][20347] Updated weights for policy 0, policy_version 1439622 (0.0008) [2023-12-27 15:37:18,902][20381] Updated weights for policy 1, policy_version 1438263 (0.0007) [2023-12-27 15:37:18,958][20381] Updated weights for policy 1, policy_version 1438273 (0.0006) [2023-12-27 15:37:19,003][20381] Updated weights for policy 1, policy_version 1438283 (0.0005) [2023-12-27 15:37:19,298][20347] Updated weights for policy 0, policy_version 1439632 (0.0007) [2023-12-27 15:37:19,363][20347] Updated weights for policy 0, policy_version 1439642 (0.0006) [2023-12-27 15:37:19,426][20347] Updated weights for policy 0, policy_version 1439652 (0.0008) [2023-12-27 15:37:19,671][20381] Updated weights for policy 1, policy_version 1438293 (0.0006) [2023-12-27 15:37:19,729][20381] Updated weights for policy 1, policy_version 1438303 (0.0005) [2023-12-27 15:37:19,781][20381] Updated weights for policy 1, policy_version 1438313 (0.0007) [2023-12-27 15:37:20,154][20347] Updated weights for policy 0, policy_version 1439662 (0.0008) [2023-12-27 15:37:20,211][20347] Updated weights for policy 0, policy_version 1439672 (0.0009) [2023-12-27 15:37:20,271][20347] Updated weights for policy 0, policy_version 1439682 (0.0008) [2023-12-27 15:37:20,388][20381] Updated weights for policy 1, policy_version 1438323 (0.0007) [2023-12-27 15:37:20,451][20381] Updated weights for policy 1, policy_version 1438333 (0.0006) [2023-12-27 15:37:20,511][20381] Updated weights for policy 1, policy_version 1438343 (0.0006) [2023-12-27 15:37:20,988][20347] Updated weights for policy 0, policy_version 1439692 (0.0008) [2023-12-27 15:37:21,051][20347] Updated weights for policy 0, policy_version 1439702 (0.0008) [2023-12-27 15:37:21,117][20347] Updated weights for policy 0, policy_version 1439712 (0.0008) [2023-12-27 15:37:21,142][20381] Updated weights for policy 1, policy_version 1438353 (0.0006) [2023-12-27 15:37:21,207][20381] Updated weights for policy 1, policy_version 1438363 (0.0007) [2023-12-27 15:37:21,274][20381] Updated weights for policy 1, policy_version 1438373 (0.0009) [2023-12-27 15:37:21,339][20381] Updated weights for policy 1, policy_version 1438383 (0.0007) [2023-12-27 15:37:21,797][20347] Updated weights for policy 0, policy_version 1439722 (0.0006) [2023-12-27 15:37:21,855][20347] Updated weights for policy 0, policy_version 1439732 (0.0008) [2023-12-27 15:37:21,918][20347] Updated weights for policy 0, policy_version 1439742 (0.0007) [2023-12-27 15:37:21,975][20347] Updated weights for policy 0, policy_version 1439752 (0.0008) [2023-12-27 15:37:22,069][20381] Updated weights for policy 1, policy_version 1438393 (0.0010) [2023-12-27 15:37:22,131][20381] Updated weights for policy 1, policy_version 1438403 (0.0007) [2023-12-27 15:37:22,189][20381] Updated weights for policy 1, policy_version 1438413 (0.0005) [2023-12-27 15:37:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 736911360. Throughput: 0: 10412.4, 1: 10302.4. Samples: 736880608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:22,363][19170] Avg episode reward: [(0, '902.150'), (1, '476.350')] [2023-12-27 15:37:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001439752_368648192.pth... [2023-12-27 15:37:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001438416_368263168.pth... [2023-12-27 15:37:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001437200_367951872.pth [2023-12-27 15:37:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001438536_368336896.pth [2023-12-27 15:37:22,676][20347] Updated weights for policy 0, policy_version 1439762 (0.0006) [2023-12-27 15:37:22,730][20347] Updated weights for policy 0, policy_version 1439772 (0.0005) [2023-12-27 15:37:22,786][20347] Updated weights for policy 0, policy_version 1439782 (0.0006) [2023-12-27 15:37:22,842][20381] Updated weights for policy 1, policy_version 1438423 (0.0007) [2023-12-27 15:37:22,902][20381] Updated weights for policy 1, policy_version 1438433 (0.0008) [2023-12-27 15:37:22,955][20381] Updated weights for policy 1, policy_version 1438443 (0.0008) [2023-12-27 15:37:23,390][20347] Updated weights for policy 0, policy_version 1439792 (0.0007) [2023-12-27 15:37:23,447][20347] Updated weights for policy 0, policy_version 1439802 (0.0008) [2023-12-27 15:37:23,502][20347] Updated weights for policy 0, policy_version 1439812 (0.0007) [2023-12-27 15:37:23,573][20381] Updated weights for policy 1, policy_version 1438453 (0.0008) [2023-12-27 15:37:23,631][20381] Updated weights for policy 1, policy_version 1438463 (0.0009) [2023-12-27 15:37:23,687][20381] Updated weights for policy 1, policy_version 1438473 (0.0010) [2023-12-27 15:37:24,225][20347] Updated weights for policy 0, policy_version 1439822 (0.0007) [2023-12-27 15:37:24,273][20347] Updated weights for policy 0, policy_version 1439832 (0.0007) [2023-12-27 15:37:24,289][20381] Updated weights for policy 1, policy_version 1438483 (0.0011) [2023-12-27 15:37:24,327][20347] Updated weights for policy 0, policy_version 1439842 (0.0006) [2023-12-27 15:37:24,355][20381] Updated weights for policy 1, policy_version 1438493 (0.0010) [2023-12-27 15:37:24,416][20381] Updated weights for policy 1, policy_version 1438503 (0.0010) [2023-12-27 15:37:24,999][20347] Updated weights for policy 0, policy_version 1439852 (0.0005) [2023-12-27 15:37:25,016][20381] Updated weights for policy 1, policy_version 1438513 (0.0009) [2023-12-27 15:37:25,057][20347] Updated weights for policy 0, policy_version 1439862 (0.0006) [2023-12-27 15:37:25,074][20381] Updated weights for policy 1, policy_version 1438523 (0.0011) [2023-12-27 15:37:25,116][20347] Updated weights for policy 0, policy_version 1439872 (0.0007) [2023-12-27 15:37:25,130][20381] Updated weights for policy 1, policy_version 1438533 (0.0011) [2023-12-27 15:37:25,191][20381] Updated weights for policy 1, policy_version 1438543 (0.0011) [2023-12-27 15:37:25,645][20347] Updated weights for policy 0, policy_version 1439882 (0.0008) [2023-12-27 15:37:25,690][20347] Updated weights for policy 0, policy_version 1439892 (0.0010) [2023-12-27 15:37:25,747][20347] Updated weights for policy 0, policy_version 1439902 (0.0010) [2023-12-27 15:37:25,807][20347] Updated weights for policy 0, policy_version 1439912 (0.0009) [2023-12-27 15:37:25,834][20381] Updated weights for policy 1, policy_version 1438553 (0.0006) [2023-12-27 15:37:25,896][20381] Updated weights for policy 1, policy_version 1438563 (0.0005) [2023-12-27 15:37:25,955][20381] Updated weights for policy 1, policy_version 1438573 (0.0005) [2023-12-27 15:37:26,456][20381] Updated weights for policy 1, policy_version 1438583 (0.0009) [2023-12-27 15:37:26,472][20347] Updated weights for policy 0, policy_version 1439922 (0.0010) [2023-12-27 15:37:26,514][20381] Updated weights for policy 1, policy_version 1438593 (0.0009) [2023-12-27 15:37:26,528][20347] Updated weights for policy 0, policy_version 1439932 (0.0008) [2023-12-27 15:37:26,571][20381] Updated weights for policy 1, policy_version 1438603 (0.0007) [2023-12-27 15:37:26,584][20347] Updated weights for policy 0, policy_version 1439942 (0.0008) [2023-12-27 15:37:27,243][20381] Updated weights for policy 1, policy_version 1438613 (0.0007) [2023-12-27 15:37:27,271][20347] Updated weights for policy 0, policy_version 1439952 (0.0006) [2023-12-27 15:37:27,289][20381] Updated weights for policy 1, policy_version 1438623 (0.0007) [2023-12-27 15:37:27,319][20347] Updated weights for policy 0, policy_version 1439962 (0.0006) [2023-12-27 15:37:27,329][20291] KL-divergence is very high: 246.4395 [2023-12-27 15:37:27,344][20381] Updated weights for policy 1, policy_version 1438633 (0.0007) [2023-12-27 15:37:27,345][20291] KL-divergence is very high: 216.4000 [2023-12-27 15:37:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 737009664. Throughput: 0: 10427.3, 1: 10360.5. Samples: 737006980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:27,363][19170] Avg episode reward: [(0, '863.140'), (1, '493.120')] [2023-12-27 15:37:27,379][20347] Updated weights for policy 0, policy_version 1439972 (0.0007) [2023-12-27 15:37:27,381][20291] KL-divergence is very high: 172.7216 [2023-12-27 15:37:28,002][20381] Updated weights for policy 1, policy_version 1438644 (0.0007) [2023-12-27 15:37:28,028][20347] Updated weights for policy 0, policy_version 1439982 (0.0007) [2023-12-27 15:37:28,056][20381] Updated weights for policy 1, policy_version 1438654 (0.0005) [2023-12-27 15:37:28,076][20347] Updated weights for policy 0, policy_version 1439992 (0.0007) [2023-12-27 15:37:28,115][20381] Updated weights for policy 1, policy_version 1438664 (0.0007) [2023-12-27 15:37:28,126][20347] Updated weights for policy 0, policy_version 1440002 (0.0007) [2023-12-27 15:37:28,789][20381] Updated weights for policy 1, policy_version 1438674 (0.0007) [2023-12-27 15:37:28,815][20347] Updated weights for policy 0, policy_version 1440012 (0.0006) [2023-12-27 15:37:28,846][20381] Updated weights for policy 1, policy_version 1438684 (0.0006) [2023-12-27 15:37:28,871][20347] Updated weights for policy 0, policy_version 1440022 (0.0006) [2023-12-27 15:37:28,902][20381] Updated weights for policy 1, policy_version 1438694 (0.0007) [2023-12-27 15:37:28,923][20347] Updated weights for policy 0, policy_version 1440032 (0.0006) [2023-12-27 15:37:28,948][20381] Updated weights for policy 1, policy_version 1438704 (0.0006) [2023-12-27 15:37:29,612][20381] Updated weights for policy 1, policy_version 1438714 (0.0008) [2023-12-27 15:37:29,621][20347] Updated weights for policy 0, policy_version 1440042 (0.0007) [2023-12-27 15:37:29,669][20381] Updated weights for policy 1, policy_version 1438724 (0.0006) [2023-12-27 15:37:29,681][20347] Updated weights for policy 0, policy_version 1440052 (0.0006) [2023-12-27 15:37:29,725][20381] Updated weights for policy 1, policy_version 1438734 (0.0006) [2023-12-27 15:37:29,738][20347] Updated weights for policy 0, policy_version 1440062 (0.0005) [2023-12-27 15:37:29,796][20347] Updated weights for policy 0, policy_version 1440072 (0.0008) [2023-12-27 15:37:30,444][20347] Updated weights for policy 0, policy_version 1440082 (0.0007) [2023-12-27 15:37:30,473][20381] Updated weights for policy 1, policy_version 1438744 (0.0007) [2023-12-27 15:37:30,485][20291] KL-divergence is very high: 131.2952 [2023-12-27 15:37:30,502][20347] Updated weights for policy 0, policy_version 1440092 (0.0006) [2023-12-27 15:37:30,526][20381] Updated weights for policy 1, policy_version 1438754 (0.0008) [2023-12-27 15:37:30,557][20347] Updated weights for policy 0, policy_version 1440102 (0.0005) [2023-12-27 15:37:30,576][20381] Updated weights for policy 1, policy_version 1438764 (0.0007) [2023-12-27 15:37:31,116][20347] Updated weights for policy 0, policy_version 1440112 (0.0005) [2023-12-27 15:37:31,182][20347] Updated weights for policy 0, policy_version 1440122 (0.0007) [2023-12-27 15:37:31,239][20347] Updated weights for policy 0, policy_version 1440132 (0.0005) [2023-12-27 15:37:31,371][20381] Updated weights for policy 1, policy_version 1438774 (0.0008) [2023-12-27 15:37:31,429][20381] Updated weights for policy 1, policy_version 1438784 (0.0008) [2023-12-27 15:37:31,489][20381] Updated weights for policy 1, policy_version 1438794 (0.0006) [2023-12-27 15:37:31,912][20347] Updated weights for policy 0, policy_version 1440142 (0.0007) [2023-12-27 15:37:31,969][20347] Updated weights for policy 0, policy_version 1440152 (0.0007) [2023-12-27 15:37:32,025][20347] Updated weights for policy 0, policy_version 1440162 (0.0007) [2023-12-27 15:37:32,165][20381] Updated weights for policy 1, policy_version 1438804 (0.0007) [2023-12-27 15:37:32,222][20381] Updated weights for policy 1, policy_version 1438814 (0.0007) [2023-12-27 15:37:32,283][20381] Updated weights for policy 1, policy_version 1438824 (0.0008) [2023-12-27 15:37:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 737124352. Throughput: 0: 10380.6, 1: 10342.7. Samples: 737130064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:32,363][19170] Avg episode reward: [(0, '902.330'), (1, '616.370')] [2023-12-27 15:37:32,723][20347] Updated weights for policy 0, policy_version 1440172 (0.0007) [2023-12-27 15:37:32,769][20347] Updated weights for policy 0, policy_version 1440182 (0.0007) [2023-12-27 15:37:32,813][20347] Updated weights for policy 0, policy_version 1440192 (0.0008) [2023-12-27 15:37:32,960][20381] Updated weights for policy 1, policy_version 1438834 (0.0009) [2023-12-27 15:37:33,011][20381] Updated weights for policy 1, policy_version 1438844 (0.0007) [2023-12-27 15:37:33,059][20381] Updated weights for policy 1, policy_version 1438854 (0.0009) [2023-12-27 15:37:33,108][20381] Updated weights for policy 1, policy_version 1438864 (0.0009) [2023-12-27 15:37:33,449][20347] Updated weights for policy 0, policy_version 1440202 (0.0008) [2023-12-27 15:37:33,496][20347] Updated weights for policy 0, policy_version 1440212 (0.0007) [2023-12-27 15:37:33,538][20347] Updated weights for policy 0, policy_version 1440222 (0.0007) [2023-12-27 15:37:33,583][20347] Updated weights for policy 0, policy_version 1440232 (0.0008) [2023-12-27 15:37:33,764][20381] Updated weights for policy 1, policy_version 1438874 (0.0008) [2023-12-27 15:37:33,813][20381] Updated weights for policy 1, policy_version 1438884 (0.0006) [2023-12-27 15:37:33,860][20381] Updated weights for policy 1, policy_version 1438894 (0.0005) [2023-12-27 15:37:34,317][20347] Updated weights for policy 0, policy_version 1440242 (0.0005) [2023-12-27 15:37:34,379][20347] Updated weights for policy 0, policy_version 1440252 (0.0006) [2023-12-27 15:37:34,441][20347] Updated weights for policy 0, policy_version 1440262 (0.0008) [2023-12-27 15:37:34,596][20381] Updated weights for policy 1, policy_version 1438904 (0.0008) [2023-12-27 15:37:34,656][20381] Updated weights for policy 1, policy_version 1438914 (0.0007) [2023-12-27 15:37:34,715][20381] Updated weights for policy 1, policy_version 1438924 (0.0008) [2023-12-27 15:37:35,135][20347] Updated weights for policy 0, policy_version 1440272 (0.0008) [2023-12-27 15:37:35,186][20347] Updated weights for policy 0, policy_version 1440282 (0.0008) [2023-12-27 15:37:35,245][20347] Updated weights for policy 0, policy_version 1440292 (0.0007) [2023-12-27 15:37:35,300][20381] Updated weights for policy 1, policy_version 1438934 (0.0006) [2023-12-27 15:37:35,353][20381] Updated weights for policy 1, policy_version 1438944 (0.0006) [2023-12-27 15:37:35,406][20381] Updated weights for policy 1, policy_version 1438954 (0.0005) [2023-12-27 15:37:35,899][20347] Updated weights for policy 0, policy_version 1440302 (0.0008) [2023-12-27 15:37:35,926][20381] Updated weights for policy 1, policy_version 1438964 (0.0006) [2023-12-27 15:37:35,955][20347] Updated weights for policy 0, policy_version 1440312 (0.0010) [2023-12-27 15:37:35,981][20381] Updated weights for policy 1, policy_version 1438974 (0.0005) [2023-12-27 15:37:36,009][20347] Updated weights for policy 0, policy_version 1440322 (0.0010) [2023-12-27 15:37:36,033][20381] Updated weights for policy 1, policy_version 1438984 (0.0006) [2023-12-27 15:37:36,681][20381] Updated weights for policy 1, policy_version 1438994 (0.0005) [2023-12-27 15:37:36,704][20347] Updated weights for policy 0, policy_version 1440332 (0.0009) [2023-12-27 15:37:36,733][20381] Updated weights for policy 1, policy_version 1439004 (0.0007) [2023-12-27 15:37:36,767][20347] Updated weights for policy 0, policy_version 1440342 (0.0007) [2023-12-27 15:37:36,781][20381] Updated weights for policy 1, policy_version 1439014 (0.0007) [2023-12-27 15:37:36,819][20347] Updated weights for policy 0, policy_version 1440352 (0.0005) [2023-12-27 15:37:36,837][20381] Updated weights for policy 1, policy_version 1439024 (0.0008) [2023-12-27 15:37:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 737222656. Throughput: 0: 10377.0, 1: 10377.3. Samples: 737192772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:37,363][19170] Avg episode reward: [(0, '882.720'), (1, '601.030')] [2023-12-27 15:37:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001439024_368418816.pth... [2023-12-27 15:37:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001440360_368803840.pth... [2023-12-27 15:37:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001437808_368107520.pth [2023-12-27 15:37:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001439144_368492544.pth [2023-12-27 15:37:37,468][20347] Updated weights for policy 0, policy_version 1440362 (0.0006) [2023-12-27 15:37:37,522][20347] Updated weights for policy 0, policy_version 1440372 (0.0006) [2023-12-27 15:37:37,554][20381] Updated weights for policy 1, policy_version 1439034 (0.0008) [2023-12-27 15:37:37,575][20347] Updated weights for policy 0, policy_version 1440382 (0.0006) [2023-12-27 15:37:37,603][20381] Updated weights for policy 1, policy_version 1439044 (0.0006) [2023-12-27 15:37:37,631][20347] Updated weights for policy 0, policy_version 1440392 (0.0006) [2023-12-27 15:37:37,652][20381] Updated weights for policy 1, policy_version 1439054 (0.0006) [2023-12-27 15:37:38,248][20381] Updated weights for policy 1, policy_version 1439064 (0.0007) [2023-12-27 15:37:38,305][20381] Updated weights for policy 1, policy_version 1439074 (0.0008) [2023-12-27 15:37:38,367][20381] Updated weights for policy 1, policy_version 1439084 (0.0007) [2023-12-27 15:37:38,370][20347] Updated weights for policy 0, policy_version 1440402 (0.0008) [2023-12-27 15:37:38,431][20347] Updated weights for policy 0, policy_version 1440412 (0.0007) [2023-12-27 15:37:38,491][20347] Updated weights for policy 0, policy_version 1440422 (0.0008) [2023-12-27 15:37:39,002][20381] Updated weights for policy 1, policy_version 1439094 (0.0005) [2023-12-27 15:37:39,058][20381] Updated weights for policy 1, policy_version 1439104 (0.0005) [2023-12-27 15:37:39,106][20381] Updated weights for policy 1, policy_version 1439114 (0.0005) [2023-12-27 15:37:39,188][20347] Updated weights for policy 0, policy_version 1440432 (0.0007) [2023-12-27 15:37:39,249][20347] Updated weights for policy 0, policy_version 1440442 (0.0008) [2023-12-27 15:37:39,309][20347] Updated weights for policy 0, policy_version 1440452 (0.0008) [2023-12-27 15:37:39,811][20381] Updated weights for policy 1, policy_version 1439124 (0.0007) [2023-12-27 15:37:39,876][20381] Updated weights for policy 1, policy_version 1439134 (0.0007) [2023-12-27 15:37:39,941][20381] Updated weights for policy 1, policy_version 1439144 (0.0007) [2023-12-27 15:37:40,106][20347] Updated weights for policy 0, policy_version 1440462 (0.0007) [2023-12-27 15:37:40,167][20347] Updated weights for policy 0, policy_version 1440472 (0.0006) [2023-12-27 15:37:40,224][20347] Updated weights for policy 0, policy_version 1440482 (0.0008) [2023-12-27 15:37:40,675][20381] Updated weights for policy 1, policy_version 1439154 (0.0008) [2023-12-27 15:37:40,720][20381] Updated weights for policy 1, policy_version 1439164 (0.0009) [2023-12-27 15:37:40,775][20381] Updated weights for policy 1, policy_version 1439174 (0.0007) [2023-12-27 15:37:40,832][20381] Updated weights for policy 1, policy_version 1439184 (0.0008) [2023-12-27 15:37:40,841][20347] Updated weights for policy 0, policy_version 1440492 (0.0009) [2023-12-27 15:37:40,896][20347] Updated weights for policy 0, policy_version 1440502 (0.0010) [2023-12-27 15:37:40,951][20347] Updated weights for policy 0, policy_version 1440512 (0.0010) [2023-12-27 15:37:41,586][20381] Updated weights for policy 1, policy_version 1439194 (0.0011) [2023-12-27 15:37:41,648][20381] Updated weights for policy 1, policy_version 1439204 (0.0009) [2023-12-27 15:37:41,700][20347] Updated weights for policy 0, policy_version 1440522 (0.0010) [2023-12-27 15:37:41,717][20381] Updated weights for policy 1, policy_version 1439214 (0.0009) [2023-12-27 15:37:41,768][20347] Updated weights for policy 0, policy_version 1440532 (0.0007) [2023-12-27 15:37:41,825][20347] Updated weights for policy 0, policy_version 1440542 (0.0007) [2023-12-27 15:37:41,888][20347] Updated weights for policy 0, policy_version 1440552 (0.0007) [2023-12-27 15:37:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 737320960. Throughput: 0: 10300.0, 1: 10394.6. Samples: 737316380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:42,363][19170] Avg episode reward: [(0, '892.550'), (1, '555.820')] [2023-12-27 15:37:42,479][20381] Updated weights for policy 1, policy_version 1439224 (0.0007) [2023-12-27 15:37:42,500][20347] Updated weights for policy 0, policy_version 1440562 (0.0006) [2023-12-27 15:37:42,535][20381] Updated weights for policy 1, policy_version 1439234 (0.0007) [2023-12-27 15:37:42,557][20347] Updated weights for policy 0, policy_version 1440572 (0.0005) [2023-12-27 15:37:42,595][20381] Updated weights for policy 1, policy_version 1439244 (0.0007) [2023-12-27 15:37:42,616][20347] Updated weights for policy 0, policy_version 1440582 (0.0006) [2023-12-27 15:37:43,195][20381] Updated weights for policy 1, policy_version 1439254 (0.0007) [2023-12-27 15:37:43,221][20347] Updated weights for policy 0, policy_version 1440592 (0.0006) [2023-12-27 15:37:43,251][20381] Updated weights for policy 1, policy_version 1439264 (0.0008) [2023-12-27 15:37:43,273][20291] KL-divergence is very high: 106.4824 [2023-12-27 15:37:43,285][20347] Updated weights for policy 0, policy_version 1440602 (0.0005) [2023-12-27 15:37:43,307][20381] Updated weights for policy 1, policy_version 1439274 (0.0007) [2023-12-27 15:37:43,319][20291] KL-divergence is very high: 140.7464 [2023-12-27 15:37:43,339][20347] Updated weights for policy 0, policy_version 1440612 (0.0005) [2023-12-27 15:37:43,916][20347] Updated weights for policy 0, policy_version 1440622 (0.0009) [2023-12-27 15:37:43,934][20381] Updated weights for policy 1, policy_version 1439284 (0.0007) [2023-12-27 15:37:43,967][20347] Updated weights for policy 0, policy_version 1440632 (0.0010) [2023-12-27 15:37:43,991][20381] Updated weights for policy 1, policy_version 1439294 (0.0008) [2023-12-27 15:37:44,022][20347] Updated weights for policy 0, policy_version 1440642 (0.0010) [2023-12-27 15:37:44,049][20381] Updated weights for policy 1, policy_version 1439304 (0.0006) [2023-12-27 15:37:44,632][20381] Updated weights for policy 1, policy_version 1439314 (0.0007) [2023-12-27 15:37:44,684][20347] Updated weights for policy 0, policy_version 1440652 (0.0010) [2023-12-27 15:37:44,690][20381] Updated weights for policy 1, policy_version 1439324 (0.0007) [2023-12-27 15:37:44,745][20347] Updated weights for policy 0, policy_version 1440662 (0.0010) [2023-12-27 15:37:44,748][20381] Updated weights for policy 1, policy_version 1439334 (0.0006) [2023-12-27 15:37:44,807][20347] Updated weights for policy 0, policy_version 1440672 (0.0010) [2023-12-27 15:37:44,813][20381] Updated weights for policy 1, policy_version 1439344 (0.0008) [2023-12-27 15:37:45,519][20347] Updated weights for policy 0, policy_version 1440682 (0.0011) [2023-12-27 15:37:45,525][20381] Updated weights for policy 1, policy_version 1439354 (0.0007) [2023-12-27 15:37:45,564][20347] Updated weights for policy 0, policy_version 1440692 (0.0010) [2023-12-27 15:37:45,574][20381] Updated weights for policy 1, policy_version 1439364 (0.0005) [2023-12-27 15:37:45,623][20347] Updated weights for policy 0, policy_version 1440702 (0.0009) [2023-12-27 15:37:45,624][20381] Updated weights for policy 1, policy_version 1439374 (0.0005) [2023-12-27 15:37:45,682][20347] Updated weights for policy 0, policy_version 1440712 (0.0007) [2023-12-27 15:37:46,242][20381] Updated weights for policy 1, policy_version 1439384 (0.0006) [2023-12-27 15:37:46,303][20381] Updated weights for policy 1, policy_version 1439394 (0.0008) [2023-12-27 15:37:46,342][20347] Updated weights for policy 0, policy_version 1440722 (0.0010) [2023-12-27 15:37:46,357][20381] Updated weights for policy 1, policy_version 1439404 (0.0005) [2023-12-27 15:37:46,397][20347] Updated weights for policy 0, policy_version 1440732 (0.0010) [2023-12-27 15:37:46,446][20347] Updated weights for policy 0, policy_version 1440742 (0.0010) [2023-12-27 15:37:47,001][20381] Updated weights for policy 1, policy_version 1439414 (0.0009) [2023-12-27 15:37:47,059][20381] Updated weights for policy 1, policy_version 1439424 (0.0011) [2023-12-27 15:37:47,114][20347] Updated weights for policy 0, policy_version 1440752 (0.0010) [2023-12-27 15:37:47,115][20381] Updated weights for policy 1, policy_version 1439434 (0.0010) [2023-12-27 15:37:47,159][20347] Updated weights for policy 0, policy_version 1440762 (0.0010) [2023-12-27 15:37:47,207][20347] Updated weights for policy 0, policy_version 1440772 (0.0010) [2023-12-27 15:37:47,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 737435648. Throughput: 0: 10348.4, 1: 10383.7. Samples: 737441784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:47,364][19170] Avg episode reward: [(0, '892.470'), (1, '520.070')] [2023-12-27 15:37:47,787][20381] Updated weights for policy 1, policy_version 1439444 (0.0010) [2023-12-27 15:37:47,838][20381] Updated weights for policy 1, policy_version 1439454 (0.0010) [2023-12-27 15:37:47,889][20381] Updated weights for policy 1, policy_version 1439464 (0.0010) [2023-12-27 15:37:47,903][20347] Updated weights for policy 0, policy_version 1440782 (0.0010) [2023-12-27 15:37:47,960][20347] Updated weights for policy 0, policy_version 1440792 (0.0010) [2023-12-27 15:37:48,012][20347] Updated weights for policy 0, policy_version 1440802 (0.0010) [2023-12-27 15:37:48,567][20381] Updated weights for policy 1, policy_version 1439474 (0.0010) [2023-12-27 15:37:48,619][20381] Updated weights for policy 1, policy_version 1439484 (0.0005) [2023-12-27 15:37:48,677][20381] Updated weights for policy 1, policy_version 1439494 (0.0009) [2023-12-27 15:37:48,714][20347] Updated weights for policy 0, policy_version 1440812 (0.0009) [2023-12-27 15:37:48,735][20381] Updated weights for policy 1, policy_version 1439504 (0.0007) [2023-12-27 15:37:48,772][20347] Updated weights for policy 0, policy_version 1440822 (0.0008) [2023-12-27 15:37:48,830][20347] Updated weights for policy 0, policy_version 1440832 (0.0007) [2023-12-27 15:37:49,400][20381] Updated weights for policy 1, policy_version 1439514 (0.0011) [2023-12-27 15:37:49,412][20347] Updated weights for policy 0, policy_version 1440842 (0.0008) [2023-12-27 15:37:49,459][20381] Updated weights for policy 1, policy_version 1439524 (0.0011) [2023-12-27 15:37:49,468][20347] Updated weights for policy 0, policy_version 1440852 (0.0010) [2023-12-27 15:37:49,513][20381] Updated weights for policy 1, policy_version 1439534 (0.0009) [2023-12-27 15:37:49,518][20347] Updated weights for policy 0, policy_version 1440862 (0.0010) [2023-12-27 15:37:49,569][20347] Updated weights for policy 0, policy_version 1440872 (0.0009) [2023-12-27 15:37:50,207][20381] Updated weights for policy 1, policy_version 1439544 (0.0010) [2023-12-27 15:37:50,269][20381] Updated weights for policy 1, policy_version 1439554 (0.0011) [2023-12-27 15:37:50,302][20347] Updated weights for policy 0, policy_version 1440882 (0.0006) [2023-12-27 15:37:50,331][20381] Updated weights for policy 1, policy_version 1439564 (0.0007) [2023-12-27 15:37:50,358][20347] Updated weights for policy 0, policy_version 1440892 (0.0008) [2023-12-27 15:37:50,408][20347] Updated weights for policy 0, policy_version 1440902 (0.0008) [2023-12-27 15:37:51,002][20381] Updated weights for policy 1, policy_version 1439574 (0.0009) [2023-12-27 15:37:51,062][20381] Updated weights for policy 1, policy_version 1439584 (0.0009) [2023-12-27 15:37:51,120][20381] Updated weights for policy 1, policy_version 1439594 (0.0009) [2023-12-27 15:37:51,166][20347] Updated weights for policy 0, policy_version 1440912 (0.0009) [2023-12-27 15:37:51,226][20347] Updated weights for policy 0, policy_version 1440922 (0.0009) [2023-12-27 15:37:51,290][20347] Updated weights for policy 0, policy_version 1440932 (0.0008) [2023-12-27 15:37:51,885][20381] Updated weights for policy 1, policy_version 1439604 (0.0008) [2023-12-27 15:37:51,937][20381] Updated weights for policy 1, policy_version 1439614 (0.0006) [2023-12-27 15:37:51,998][20381] Updated weights for policy 1, policy_version 1439624 (0.0008) [2023-12-27 15:37:52,057][20347] Updated weights for policy 0, policy_version 1440942 (0.0007) [2023-12-27 15:37:52,121][20347] Updated weights for policy 0, policy_version 1440952 (0.0008) [2023-12-27 15:37:52,177][20347] Updated weights for policy 0, policy_version 1440962 (0.0008) [2023-12-27 15:37:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 737533952. Throughput: 0: 10350.1, 1: 10373.0. Samples: 737503212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:52,363][19170] Avg episode reward: [(0, '931.530'), (1, '595.500')] [2023-12-27 15:37:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001439632_368574464.pth... [2023-12-27 15:37:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001440968_368959488.pth... [2023-12-27 15:37:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001439752_368648192.pth [2023-12-27 15:37:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001438416_368263168.pth [2023-12-27 15:37:52,650][20381] Updated weights for policy 1, policy_version 1439634 (0.0008) [2023-12-27 15:37:52,711][20381] Updated weights for policy 1, policy_version 1439644 (0.0007) [2023-12-27 15:37:52,775][20381] Updated weights for policy 1, policy_version 1439654 (0.0006) [2023-12-27 15:37:52,835][20381] Updated weights for policy 1, policy_version 1439664 (0.0005) [2023-12-27 15:37:52,903][20347] Updated weights for policy 0, policy_version 1440972 (0.0008) [2023-12-27 15:37:52,956][20347] Updated weights for policy 0, policy_version 1440982 (0.0008) [2023-12-27 15:37:53,013][20347] Updated weights for policy 0, policy_version 1440992 (0.0007) [2023-12-27 15:37:53,438][20381] Updated weights for policy 1, policy_version 1439674 (0.0010) [2023-12-27 15:37:53,486][20381] Updated weights for policy 1, policy_version 1439684 (0.0010) [2023-12-27 15:37:53,532][20381] Updated weights for policy 1, policy_version 1439694 (0.0010) [2023-12-27 15:37:53,589][20347] Updated weights for policy 0, policy_version 1441002 (0.0008) [2023-12-27 15:37:53,647][20347] Updated weights for policy 0, policy_version 1441012 (0.0010) [2023-12-27 15:37:53,703][20347] Updated weights for policy 0, policy_version 1441022 (0.0007) [2023-12-27 15:37:53,758][20347] Updated weights for policy 0, policy_version 1441032 (0.0005) [2023-12-27 15:37:54,243][20381] Updated weights for policy 1, policy_version 1439704 (0.0010) [2023-12-27 15:37:54,306][20381] Updated weights for policy 1, policy_version 1439714 (0.0011) [2023-12-27 15:37:54,322][20347] Updated weights for policy 0, policy_version 1441042 (0.0008) [2023-12-27 15:37:54,369][20381] Updated weights for policy 1, policy_version 1439724 (0.0011) [2023-12-27 15:37:54,380][20347] Updated weights for policy 0, policy_version 1441052 (0.0007) [2023-12-27 15:37:54,434][20347] Updated weights for policy 0, policy_version 1441062 (0.0008) [2023-12-27 15:37:54,956][20381] Updated weights for policy 1, policy_version 1439734 (0.0007) [2023-12-27 15:37:55,015][20381] Updated weights for policy 1, policy_version 1439744 (0.0008) [2023-12-27 15:37:55,065][20381] Updated weights for policy 1, policy_version 1439754 (0.0008) [2023-12-27 15:37:55,114][20347] Updated weights for policy 0, policy_version 1441072 (0.0007) [2023-12-27 15:37:55,171][20347] Updated weights for policy 0, policy_version 1441082 (0.0005) [2023-12-27 15:37:55,231][20347] Updated weights for policy 0, policy_version 1441092 (0.0005) [2023-12-27 15:37:55,633][20381] Updated weights for policy 1, policy_version 1439764 (0.0007) [2023-12-27 15:37:55,691][20381] Updated weights for policy 1, policy_version 1439774 (0.0008) [2023-12-27 15:37:55,745][20381] Updated weights for policy 1, policy_version 1439784 (0.0007) [2023-12-27 15:37:55,818][20347] Updated weights for policy 0, policy_version 1441102 (0.0009) [2023-12-27 15:37:55,866][20347] Updated weights for policy 0, policy_version 1441112 (0.0010) [2023-12-27 15:37:55,917][20347] Updated weights for policy 0, policy_version 1441122 (0.0008) [2023-12-27 15:37:56,426][20381] Updated weights for policy 1, policy_version 1439794 (0.0007) [2023-12-27 15:37:56,474][20381] Updated weights for policy 1, policy_version 1439804 (0.0008) [2023-12-27 15:37:56,528][20381] Updated weights for policy 1, policy_version 1439814 (0.0006) [2023-12-27 15:37:56,583][20381] Updated weights for policy 1, policy_version 1439824 (0.0005) [2023-12-27 15:37:56,600][20347] Updated weights for policy 0, policy_version 1441132 (0.0008) [2023-12-27 15:37:56,659][20347] Updated weights for policy 0, policy_version 1441142 (0.0008) [2023-12-27 15:37:56,726][20347] Updated weights for policy 0, policy_version 1441152 (0.0008) [2023-12-27 15:37:57,255][20381] Updated weights for policy 1, policy_version 1439834 (0.0008) [2023-12-27 15:37:57,310][20381] Updated weights for policy 1, policy_version 1439844 (0.0008) [2023-12-27 15:37:57,360][20381] Updated weights for policy 1, policy_version 1439854 (0.0008) [2023-12-27 15:37:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 737632256. Throughput: 0: 10297.3, 1: 10461.8. Samples: 737628164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:37:57,363][19170] Avg episode reward: [(0, '921.680'), (1, '523.190')] [2023-12-27 15:37:57,372][20347] Updated weights for policy 0, policy_version 1441162 (0.0007) [2023-12-27 15:37:57,421][20347] Updated weights for policy 0, policy_version 1441172 (0.0005) [2023-12-27 15:37:57,470][20347] Updated weights for policy 0, policy_version 1441182 (0.0006) [2023-12-27 15:37:57,523][20347] Updated weights for policy 0, policy_version 1441192 (0.0007) [2023-12-27 15:37:57,974][20381] Updated weights for policy 1, policy_version 1439864 (0.0006) [2023-12-27 15:37:58,032][20381] Updated weights for policy 1, policy_version 1439874 (0.0006) [2023-12-27 15:37:58,084][20381] Updated weights for policy 1, policy_version 1439884 (0.0005) [2023-12-27 15:37:58,178][20347] Updated weights for policy 0, policy_version 1441202 (0.0007) [2023-12-27 15:37:58,239][20347] Updated weights for policy 0, policy_version 1441212 (0.0008) [2023-12-27 15:37:58,294][20347] Updated weights for policy 0, policy_version 1441222 (0.0008) [2023-12-27 15:37:58,838][20381] Updated weights for policy 1, policy_version 1439894 (0.0007) [2023-12-27 15:37:58,903][20381] Updated weights for policy 1, policy_version 1439904 (0.0009) [2023-12-27 15:37:58,966][20381] Updated weights for policy 1, policy_version 1439914 (0.0008) [2023-12-27 15:37:59,130][20347] Updated weights for policy 0, policy_version 1441232 (0.0009) [2023-12-27 15:37:59,192][20347] Updated weights for policy 0, policy_version 1441242 (0.0010) [2023-12-27 15:37:59,256][20347] Updated weights for policy 0, policy_version 1441252 (0.0009) [2023-12-27 15:37:59,652][20381] Updated weights for policy 1, policy_version 1439924 (0.0007) [2023-12-27 15:37:59,710][20381] Updated weights for policy 1, policy_version 1439934 (0.0008) [2023-12-27 15:37:59,774][20381] Updated weights for policy 1, policy_version 1439944 (0.0008) [2023-12-27 15:37:59,919][20347] Updated weights for policy 0, policy_version 1441262 (0.0009) [2023-12-27 15:37:59,974][20347] Updated weights for policy 0, policy_version 1441272 (0.0008) [2023-12-27 15:38:00,030][20347] Updated weights for policy 0, policy_version 1441282 (0.0008) [2023-12-27 15:38:00,393][20381] Updated weights for policy 1, policy_version 1439954 (0.0007) [2023-12-27 15:38:00,452][20381] Updated weights for policy 1, policy_version 1439964 (0.0007) [2023-12-27 15:38:00,510][20381] Updated weights for policy 1, policy_version 1439974 (0.0008) [2023-12-27 15:38:00,569][20381] Updated weights for policy 1, policy_version 1439984 (0.0007) [2023-12-27 15:38:00,737][20347] Updated weights for policy 0, policy_version 1441292 (0.0007) [2023-12-27 15:38:00,785][20347] Updated weights for policy 0, policy_version 1441302 (0.0007) [2023-12-27 15:38:00,832][20347] Updated weights for policy 0, policy_version 1441312 (0.0007) [2023-12-27 15:38:01,235][20381] Updated weights for policy 1, policy_version 1439994 (0.0008) [2023-12-27 15:38:01,289][20381] Updated weights for policy 1, policy_version 1440004 (0.0008) [2023-12-27 15:38:01,339][20381] Updated weights for policy 1, policy_version 1440014 (0.0008) [2023-12-27 15:38:01,507][20347] Updated weights for policy 0, policy_version 1441322 (0.0007) [2023-12-27 15:38:01,567][20347] Updated weights for policy 0, policy_version 1441332 (0.0006) [2023-12-27 15:38:01,628][20347] Updated weights for policy 0, policy_version 1441342 (0.0007) [2023-12-27 15:38:01,690][20347] Updated weights for policy 0, policy_version 1441352 (0.0006) [2023-12-27 15:38:02,120][20381] Updated weights for policy 1, policy_version 1440024 (0.0006) [2023-12-27 15:38:02,173][20381] Updated weights for policy 1, policy_version 1440034 (0.0005) [2023-12-27 15:38:02,235][20381] Updated weights for policy 1, policy_version 1440044 (0.0007) [2023-12-27 15:38:02,302][20347] Updated weights for policy 0, policy_version 1441362 (0.0007) [2023-12-27 15:38:02,361][20347] Updated weights for policy 0, policy_version 1441372 (0.0007) [2023-12-27 15:38:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 737738752. Throughput: 0: 10294.2, 1: 10443.0. Samples: 737750352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:38:02,363][19170] Avg episode reward: [(0, '872.930'), (1, '496.710')] [2023-12-27 15:38:02,418][20347] Updated weights for policy 0, policy_version 1441382 (0.0007) [2023-12-27 15:38:02,905][20381] Updated weights for policy 1, policy_version 1440054 (0.0007) [2023-12-27 15:38:02,966][20381] Updated weights for policy 1, policy_version 1440064 (0.0007) [2023-12-27 15:38:03,024][20381] Updated weights for policy 1, policy_version 1440074 (0.0008) [2023-12-27 15:38:03,110][20347] Updated weights for policy 0, policy_version 1441392 (0.0008) [2023-12-27 15:38:03,160][20347] Updated weights for policy 0, policy_version 1441402 (0.0009) [2023-12-27 15:38:03,215][20347] Updated weights for policy 0, policy_version 1441412 (0.0009) [2023-12-27 15:38:03,609][20381] Updated weights for policy 1, policy_version 1440084 (0.0008) [2023-12-27 15:38:03,660][20381] Updated weights for policy 1, policy_version 1440094 (0.0010) [2023-12-27 15:38:03,710][20381] Updated weights for policy 1, policy_version 1440104 (0.0010) [2023-12-27 15:38:03,847][20347] Updated weights for policy 0, policy_version 1441422 (0.0008) [2023-12-27 15:38:03,903][20347] Updated weights for policy 0, policy_version 1441432 (0.0008) [2023-12-27 15:38:03,957][20347] Updated weights for policy 0, policy_version 1441442 (0.0007) [2023-12-27 15:38:04,424][20381] Updated weights for policy 1, policy_version 1440114 (0.0010) [2023-12-27 15:38:04,486][20381] Updated weights for policy 1, policy_version 1440124 (0.0011) [2023-12-27 15:38:04,546][20381] Updated weights for policy 1, policy_version 1440134 (0.0011) [2023-12-27 15:38:04,595][20381] Updated weights for policy 1, policy_version 1440144 (0.0010) [2023-12-27 15:38:04,643][20347] Updated weights for policy 0, policy_version 1441452 (0.0007) [2023-12-27 15:38:04,703][20347] Updated weights for policy 0, policy_version 1441462 (0.0007) [2023-12-27 15:38:04,767][20347] Updated weights for policy 0, policy_version 1441472 (0.0007) [2023-12-27 15:38:05,273][20381] Updated weights for policy 1, policy_version 1440154 (0.0010) [2023-12-27 15:38:05,327][20381] Updated weights for policy 1, policy_version 1440164 (0.0010) [2023-12-27 15:38:05,384][20381] Updated weights for policy 1, policy_version 1440174 (0.0010) [2023-12-27 15:38:05,445][20347] Updated weights for policy 0, policy_version 1441482 (0.0007) [2023-12-27 15:38:05,506][20347] Updated weights for policy 0, policy_version 1441492 (0.0007) [2023-12-27 15:38:05,564][20347] Updated weights for policy 0, policy_version 1441502 (0.0007) [2023-12-27 15:38:05,621][20347] Updated weights for policy 0, policy_version 1441512 (0.0007) [2023-12-27 15:38:06,055][20381] Updated weights for policy 1, policy_version 1440184 (0.0011) [2023-12-27 15:38:06,104][20381] Updated weights for policy 1, policy_version 1440194 (0.0010) [2023-12-27 15:38:06,162][20381] Updated weights for policy 1, policy_version 1440204 (0.0011) [2023-12-27 15:38:06,336][20347] Updated weights for policy 0, policy_version 1441522 (0.0008) [2023-12-27 15:38:06,400][20347] Updated weights for policy 0, policy_version 1441532 (0.0008) [2023-12-27 15:38:06,460][20347] Updated weights for policy 0, policy_version 1441542 (0.0007) [2023-12-27 15:38:06,803][20381] Updated weights for policy 1, policy_version 1440214 (0.0007) [2023-12-27 15:38:06,859][20381] Updated weights for policy 1, policy_version 1440224 (0.0009) [2023-12-27 15:38:06,918][20381] Updated weights for policy 1, policy_version 1440234 (0.0011) [2023-12-27 15:38:07,198][20347] Updated weights for policy 0, policy_version 1441552 (0.0007) [2023-12-27 15:38:07,256][20347] Updated weights for policy 0, policy_version 1441562 (0.0008) [2023-12-27 15:38:07,314][20347] Updated weights for policy 0, policy_version 1441572 (0.0007) [2023-12-27 15:38:07,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20752.9, 300 sec: 20604.9). Total num frames: 737845248. Throughput: 0: 10298.8, 1: 10404.4. Samples: 737812260. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:07,364][19170] Avg episode reward: [(0, '843.580'), (1, '477.380')] [2023-12-27 15:38:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001441576_369115136.pth... [2023-12-27 15:38:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001440240_368730112.pth... [2023-12-27 15:38:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001440360_368803840.pth [2023-12-27 15:38:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001439024_368418816.pth [2023-12-27 15:38:07,598][20381] Updated weights for policy 1, policy_version 1440244 (0.0010) [2023-12-27 15:38:07,653][20381] Updated weights for policy 1, policy_version 1440254 (0.0008) [2023-12-27 15:38:07,697][20381] Updated weights for policy 1, policy_version 1440264 (0.0010) [2023-12-27 15:38:08,008][20347] Updated weights for policy 0, policy_version 1441582 (0.0007) [2023-12-27 15:38:08,067][20347] Updated weights for policy 0, policy_version 1441592 (0.0007) [2023-12-27 15:38:08,122][20347] Updated weights for policy 0, policy_version 1441602 (0.0007) [2023-12-27 15:38:08,366][20381] Updated weights for policy 1, policy_version 1440274 (0.0010) [2023-12-27 15:38:08,428][20381] Updated weights for policy 1, policy_version 1440284 (0.0008) [2023-12-27 15:38:08,491][20381] Updated weights for policy 1, policy_version 1440294 (0.0008) [2023-12-27 15:38:08,547][20381] Updated weights for policy 1, policy_version 1440304 (0.0007) [2023-12-27 15:38:08,840][20347] Updated weights for policy 0, policy_version 1441612 (0.0007) [2023-12-27 15:38:08,892][20347] Updated weights for policy 0, policy_version 1441622 (0.0007) [2023-12-27 15:38:08,947][20347] Updated weights for policy 0, policy_version 1441632 (0.0007) [2023-12-27 15:38:09,248][20381] Updated weights for policy 1, policy_version 1440314 (0.0010) [2023-12-27 15:38:09,310][20381] Updated weights for policy 1, policy_version 1440324 (0.0010) [2023-12-27 15:38:09,377][20381] Updated weights for policy 1, policy_version 1440334 (0.0011) [2023-12-27 15:38:09,702][20347] Updated weights for policy 0, policy_version 1441642 (0.0007) [2023-12-27 15:38:09,763][20347] Updated weights for policy 0, policy_version 1441652 (0.0008) [2023-12-27 15:38:09,819][20347] Updated weights for policy 0, policy_version 1441662 (0.0007) [2023-12-27 15:38:09,883][20347] Updated weights for policy 0, policy_version 1441672 (0.0006) [2023-12-27 15:38:10,116][20381] Updated weights for policy 1, policy_version 1440344 (0.0009) [2023-12-27 15:38:10,178][20381] Updated weights for policy 1, policy_version 1440354 (0.0009) [2023-12-27 15:38:10,246][20381] Updated weights for policy 1, policy_version 1440364 (0.0009) [2023-12-27 15:38:10,594][20347] Updated weights for policy 0, policy_version 1441682 (0.0005) [2023-12-27 15:38:10,651][20347] Updated weights for policy 0, policy_version 1441692 (0.0005) [2023-12-27 15:38:10,708][20347] Updated weights for policy 0, policy_version 1441702 (0.0008) [2023-12-27 15:38:10,949][20381] Updated weights for policy 1, policy_version 1440374 (0.0007) [2023-12-27 15:38:11,008][20381] Updated weights for policy 1, policy_version 1440384 (0.0006) [2023-12-27 15:38:11,075][20381] Updated weights for policy 1, policy_version 1440394 (0.0007) [2023-12-27 15:38:11,432][20347] Updated weights for policy 0, policy_version 1441712 (0.0008) [2023-12-27 15:38:11,495][20347] Updated weights for policy 0, policy_version 1441722 (0.0007) [2023-12-27 15:38:11,563][20347] Updated weights for policy 0, policy_version 1441732 (0.0008) [2023-12-27 15:38:11,811][20381] Updated weights for policy 1, policy_version 1440404 (0.0008) [2023-12-27 15:38:11,875][20381] Updated weights for policy 1, policy_version 1440414 (0.0009) [2023-12-27 15:38:11,952][20381] Updated weights for policy 1, policy_version 1440424 (0.0007) [2023-12-27 15:38:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 737935360. Throughput: 0: 10214.7, 1: 10314.8. Samples: 737930808. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:12,363][19170] Avg episode reward: [(0, '892.430'), (1, '429.970')] [2023-12-27 15:38:12,450][20347] Updated weights for policy 0, policy_version 1441742 (0.0008) [2023-12-27 15:38:12,507][20347] Updated weights for policy 0, policy_version 1441752 (0.0008) [2023-12-27 15:38:12,568][20347] Updated weights for policy 0, policy_version 1441762 (0.0007) [2023-12-27 15:38:12,641][20381] Updated weights for policy 1, policy_version 1440434 (0.0007) [2023-12-27 15:38:12,697][20381] Updated weights for policy 1, policy_version 1440444 (0.0008) [2023-12-27 15:38:12,750][20381] Updated weights for policy 1, policy_version 1440454 (0.0008) [2023-12-27 15:38:12,810][20381] Updated weights for policy 1, policy_version 1440464 (0.0008) [2023-12-27 15:38:13,268][20347] Updated weights for policy 0, policy_version 1441772 (0.0008) [2023-12-27 15:38:13,320][20347] Updated weights for policy 0, policy_version 1441782 (0.0007) [2023-12-27 15:38:13,376][20347] Updated weights for policy 0, policy_version 1441792 (0.0008) [2023-12-27 15:38:13,492][20381] Updated weights for policy 1, policy_version 1440474 (0.0008) [2023-12-27 15:38:13,559][20381] Updated weights for policy 1, policy_version 1440484 (0.0008) [2023-12-27 15:38:13,607][20381] Updated weights for policy 1, policy_version 1440494 (0.0010) [2023-12-27 15:38:14,004][20347] Updated weights for policy 0, policy_version 1441802 (0.0007) [2023-12-27 15:38:14,047][20347] Updated weights for policy 0, policy_version 1441812 (0.0007) [2023-12-27 15:38:14,092][20347] Updated weights for policy 0, policy_version 1441822 (0.0008) [2023-12-27 15:38:14,140][20347] Updated weights for policy 0, policy_version 1441832 (0.0007) [2023-12-27 15:38:14,227][20381] Updated weights for policy 1, policy_version 1440504 (0.0006) [2023-12-27 15:38:14,282][20381] Updated weights for policy 1, policy_version 1440514 (0.0005) [2023-12-27 15:38:14,334][20381] Updated weights for policy 1, policy_version 1440524 (0.0008) [2023-12-27 15:38:14,946][20347] Updated weights for policy 0, policy_version 1441842 (0.0008) [2023-12-27 15:38:14,960][20381] Updated weights for policy 1, policy_version 1440534 (0.0008) [2023-12-27 15:38:15,003][20347] Updated weights for policy 0, policy_version 1441852 (0.0008) [2023-12-27 15:38:15,017][20381] Updated weights for policy 1, policy_version 1440544 (0.0010) [2023-12-27 15:38:15,059][20347] Updated weights for policy 0, policy_version 1441862 (0.0008) [2023-12-27 15:38:15,076][20381] Updated weights for policy 1, policy_version 1440554 (0.0010) [2023-12-27 15:38:15,773][20381] Updated weights for policy 1, policy_version 1440564 (0.0010) [2023-12-27 15:38:15,775][20347] Updated weights for policy 0, policy_version 1441872 (0.0008) [2023-12-27 15:38:15,826][20347] Updated weights for policy 0, policy_version 1441882 (0.0008) [2023-12-27 15:38:15,836][20381] Updated weights for policy 1, policy_version 1440574 (0.0011) [2023-12-27 15:38:15,875][20347] Updated weights for policy 0, policy_version 1441892 (0.0007) [2023-12-27 15:38:15,895][20381] Updated weights for policy 1, policy_version 1440584 (0.0011) [2023-12-27 15:38:16,528][20381] Updated weights for policy 1, policy_version 1440594 (0.0009) [2023-12-27 15:38:16,581][20381] Updated weights for policy 1, policy_version 1440604 (0.0006) [2023-12-27 15:38:16,623][20347] Updated weights for policy 0, policy_version 1441902 (0.0007) [2023-12-27 15:38:16,637][20381] Updated weights for policy 1, policy_version 1440614 (0.0007) [2023-12-27 15:38:16,686][20347] Updated weights for policy 0, policy_version 1441912 (0.0006) [2023-12-27 15:38:16,688][20381] Updated weights for policy 1, policy_version 1440624 (0.0007) [2023-12-27 15:38:16,752][20347] Updated weights for policy 0, policy_version 1441922 (0.0006) [2023-12-27 15:38:17,363][19170] Fps is (10 sec: 18842.1, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 738033664. Throughput: 0: 10091.4, 1: 10348.5. Samples: 738049864. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:17,364][19170] Avg episode reward: [(0, '902.220'), (1, '459.040')] [2023-12-27 15:38:17,372][20381] Updated weights for policy 1, policy_version 1440634 (0.0008) [2023-12-27 15:38:17,430][20381] Updated weights for policy 1, policy_version 1440644 (0.0007) [2023-12-27 15:38:17,436][20347] Updated weights for policy 0, policy_version 1441932 (0.0007) [2023-12-27 15:38:17,490][20381] Updated weights for policy 1, policy_version 1440654 (0.0006) [2023-12-27 15:38:17,496][20347] Updated weights for policy 0, policy_version 1441942 (0.0006) [2023-12-27 15:38:17,557][20347] Updated weights for policy 0, policy_version 1441952 (0.0008) [2023-12-27 15:38:18,186][20347] Updated weights for policy 0, policy_version 1441962 (0.0008) [2023-12-27 15:38:18,202][20381] Updated weights for policy 1, policy_version 1440664 (0.0006) [2023-12-27 15:38:18,242][20347] Updated weights for policy 0, policy_version 1441972 (0.0006) [2023-12-27 15:38:18,251][20381] Updated weights for policy 1, policy_version 1440674 (0.0005) [2023-12-27 15:38:18,301][20347] Updated weights for policy 0, policy_version 1441982 (0.0005) [2023-12-27 15:38:18,303][20381] Updated weights for policy 1, policy_version 1440684 (0.0007) [2023-12-27 15:38:18,363][20347] Updated weights for policy 0, policy_version 1441992 (0.0007) [2023-12-27 15:38:18,906][20347] Updated weights for policy 0, policy_version 1442002 (0.0007) [2023-12-27 15:38:18,959][20347] Updated weights for policy 0, policy_version 1442012 (0.0007) [2023-12-27 15:38:18,977][20381] Updated weights for policy 1, policy_version 1440694 (0.0006) [2023-12-27 15:38:19,012][20347] Updated weights for policy 0, policy_version 1442022 (0.0008) [2023-12-27 15:38:19,024][20381] Updated weights for policy 1, policy_version 1440704 (0.0008) [2023-12-27 15:38:19,078][20381] Updated weights for policy 1, policy_version 1440714 (0.0008) [2023-12-27 15:38:19,738][20347] Updated weights for policy 0, policy_version 1442032 (0.0006) [2023-12-27 15:38:19,788][20347] Updated weights for policy 0, policy_version 1442042 (0.0007) [2023-12-27 15:38:19,806][20381] Updated weights for policy 1, policy_version 1440724 (0.0007) [2023-12-27 15:38:19,847][20347] Updated weights for policy 0, policy_version 1442052 (0.0006) [2023-12-27 15:38:19,868][20381] Updated weights for policy 1, policy_version 1440734 (0.0007) [2023-12-27 15:38:19,932][20381] Updated weights for policy 1, policy_version 1440744 (0.0008) [2023-12-27 15:38:20,536][20347] Updated weights for policy 0, policy_version 1442062 (0.0010) [2023-12-27 15:38:20,592][20347] Updated weights for policy 0, policy_version 1442072 (0.0010) [2023-12-27 15:38:20,604][20381] Updated weights for policy 1, policy_version 1440754 (0.0008) [2023-12-27 15:38:20,647][20347] Updated weights for policy 0, policy_version 1442082 (0.0010) [2023-12-27 15:38:20,664][20381] Updated weights for policy 1, policy_version 1440764 (0.0006) [2023-12-27 15:38:20,720][20381] Updated weights for policy 1, policy_version 1440774 (0.0006) [2023-12-27 15:38:20,772][20381] Updated weights for policy 1, policy_version 1440784 (0.0005) [2023-12-27 15:38:21,425][20347] Updated weights for policy 0, policy_version 1442092 (0.0006) [2023-12-27 15:38:21,428][20381] Updated weights for policy 1, policy_version 1440794 (0.0009) [2023-12-27 15:38:21,488][20381] Updated weights for policy 1, policy_version 1440804 (0.0008) [2023-12-27 15:38:21,489][20347] Updated weights for policy 0, policy_version 1442102 (0.0006) [2023-12-27 15:38:21,549][20347] Updated weights for policy 0, policy_version 1442112 (0.0006) [2023-12-27 15:38:21,551][20381] Updated weights for policy 1, policy_version 1440814 (0.0009) [2023-12-27 15:38:22,266][20347] Updated weights for policy 0, policy_version 1442122 (0.0007) [2023-12-27 15:38:22,300][20381] Updated weights for policy 1, policy_version 1440824 (0.0008) [2023-12-27 15:38:22,326][20347] Updated weights for policy 0, policy_version 1442132 (0.0006) [2023-12-27 15:38:22,356][20381] Updated weights for policy 1, policy_version 1440834 (0.0007) [2023-12-27 15:38:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 738131968. Throughput: 0: 10099.4, 1: 10301.1. Samples: 738110800. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:22,363][19170] Avg episode reward: [(0, '911.940'), (1, '382.160')] [2023-12-27 15:38:22,402][20347] Updated weights for policy 0, policy_version 1442142 (0.0007) [2023-12-27 15:38:22,428][20381] Updated weights for policy 1, policy_version 1440844 (0.0008) [2023-12-27 15:38:22,447][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001440848_368885760.pth... [2023-12-27 15:38:22,451][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001439632_368574464.pth [2023-12-27 15:38:22,453][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001442152_369262592.pth... [2023-12-27 15:38:22,454][20347] Updated weights for policy 0, policy_version 1442152 (0.0007) [2023-12-27 15:38:22,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001440968_368959488.pth [2023-12-27 15:38:23,130][20381] Updated weights for policy 1, policy_version 1440854 (0.0009) [2023-12-27 15:38:23,139][20347] Updated weights for policy 0, policy_version 1442162 (0.0006) [2023-12-27 15:38:23,188][20381] Updated weights for policy 1, policy_version 1440864 (0.0010) [2023-12-27 15:38:23,193][20347] Updated weights for policy 0, policy_version 1442172 (0.0008) [2023-12-27 15:38:23,248][20347] Updated weights for policy 0, policy_version 1442182 (0.0006) [2023-12-27 15:38:23,248][20381] Updated weights for policy 1, policy_version 1440874 (0.0011) [2023-12-27 15:38:23,926][20347] Updated weights for policy 0, policy_version 1442192 (0.0009) [2023-12-27 15:38:23,928][20381] Updated weights for policy 1, policy_version 1440884 (0.0009) [2023-12-27 15:38:23,975][20347] Updated weights for policy 0, policy_version 1442202 (0.0010) [2023-12-27 15:38:23,977][20381] Updated weights for policy 1, policy_version 1440894 (0.0005) [2023-12-27 15:38:24,024][20347] Updated weights for policy 0, policy_version 1442212 (0.0010) [2023-12-27 15:38:24,030][20381] Updated weights for policy 1, policy_version 1440904 (0.0005) [2023-12-27 15:38:24,679][20381] Updated weights for policy 1, policy_version 1440914 (0.0007) [2023-12-27 15:38:24,726][20347] Updated weights for policy 0, policy_version 1442222 (0.0007) [2023-12-27 15:38:24,732][20381] Updated weights for policy 1, policy_version 1440924 (0.0007) [2023-12-27 15:38:24,788][20381] Updated weights for policy 1, policy_version 1440934 (0.0006) [2023-12-27 15:38:24,789][20347] Updated weights for policy 0, policy_version 1442232 (0.0006) [2023-12-27 15:38:24,845][20347] Updated weights for policy 0, policy_version 1442242 (0.0008) [2023-12-27 15:38:24,847][20381] Updated weights for policy 1, policy_version 1440944 (0.0006) [2023-12-27 15:38:25,426][20381] Updated weights for policy 1, policy_version 1440954 (0.0005) [2023-12-27 15:38:25,469][20347] Updated weights for policy 0, policy_version 1442252 (0.0007) [2023-12-27 15:38:25,476][20381] Updated weights for policy 1, policy_version 1440964 (0.0007) [2023-12-27 15:38:25,517][20347] Updated weights for policy 0, policy_version 1442262 (0.0006) [2023-12-27 15:38:25,525][20381] Updated weights for policy 1, policy_version 1440974 (0.0007) [2023-12-27 15:38:25,567][20347] Updated weights for policy 0, policy_version 1442272 (0.0006) [2023-12-27 15:38:26,173][20381] Updated weights for policy 1, policy_version 1440984 (0.0009) [2023-12-27 15:38:26,180][20347] Updated weights for policy 0, policy_version 1442282 (0.0010) [2023-12-27 15:38:26,239][20347] Updated weights for policy 0, policy_version 1442292 (0.0007) [2023-12-27 15:38:26,240][20381] Updated weights for policy 1, policy_version 1440994 (0.0010) [2023-12-27 15:38:26,296][20347] Updated weights for policy 0, policy_version 1442302 (0.0007) [2023-12-27 15:38:26,305][20381] Updated weights for policy 1, policy_version 1441004 (0.0011) [2023-12-27 15:38:26,353][20347] Updated weights for policy 0, policy_version 1442312 (0.0006) [2023-12-27 15:38:26,884][20381] Updated weights for policy 1, policy_version 1441014 (0.0011) [2023-12-27 15:38:26,886][20347] Updated weights for policy 0, policy_version 1442322 (0.0007) [2023-12-27 15:38:26,940][20347] Updated weights for policy 0, policy_version 1442332 (0.0006) [2023-12-27 15:38:26,947][20381] Updated weights for policy 1, policy_version 1441024 (0.0010) [2023-12-27 15:38:26,995][20347] Updated weights for policy 0, policy_version 1442342 (0.0006) [2023-12-27 15:38:27,006][20381] Updated weights for policy 1, policy_version 1441034 (0.0009) [2023-12-27 15:38:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 738246656. Throughput: 0: 10128.3, 1: 10263.5. Samples: 738234008. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:27,363][19170] Avg episode reward: [(0, '970.640'), (1, '312.590')] [2023-12-27 15:38:27,648][20347] Updated weights for policy 0, policy_version 1442352 (0.0007) [2023-12-27 15:38:27,711][20347] Updated weights for policy 0, policy_version 1442362 (0.0007) [2023-12-27 15:38:27,729][20381] Updated weights for policy 1, policy_version 1441044 (0.0008) [2023-12-27 15:38:27,767][20347] Updated weights for policy 0, policy_version 1442372 (0.0005) [2023-12-27 15:38:27,784][20381] Updated weights for policy 1, policy_version 1441054 (0.0007) [2023-12-27 15:38:27,844][20381] Updated weights for policy 1, policy_version 1441064 (0.0005) [2023-12-27 15:38:28,381][20347] Updated weights for policy 0, policy_version 1442382 (0.0007) [2023-12-27 15:38:28,428][20347] Updated weights for policy 0, policy_version 1442392 (0.0007) [2023-12-27 15:38:28,477][20347] Updated weights for policy 0, policy_version 1442402 (0.0008) [2023-12-27 15:38:28,498][20381] Updated weights for policy 1, policy_version 1441074 (0.0005) [2023-12-27 15:38:28,556][20381] Updated weights for policy 1, policy_version 1441084 (0.0006) [2023-12-27 15:38:28,613][20381] Updated weights for policy 1, policy_version 1441094 (0.0008) [2023-12-27 15:38:28,662][20381] Updated weights for policy 1, policy_version 1441104 (0.0005) [2023-12-27 15:38:29,156][20347] Updated weights for policy 0, policy_version 1442412 (0.0007) [2023-12-27 15:38:29,219][20347] Updated weights for policy 0, policy_version 1442422 (0.0006) [2023-12-27 15:38:29,248][20381] Updated weights for policy 1, policy_version 1441114 (0.0008) [2023-12-27 15:38:29,281][20347] Updated weights for policy 0, policy_version 1442432 (0.0007) [2023-12-27 15:38:29,313][20381] Updated weights for policy 1, policy_version 1441124 (0.0006) [2023-12-27 15:38:29,377][20381] Updated weights for policy 1, policy_version 1441134 (0.0009) [2023-12-27 15:38:30,005][20347] Updated weights for policy 0, policy_version 1442442 (0.0008) [2023-12-27 15:38:30,061][20347] Updated weights for policy 0, policy_version 1442452 (0.0007) [2023-12-27 15:38:30,085][20381] Updated weights for policy 1, policy_version 1441144 (0.0008) [2023-12-27 15:38:30,116][20347] Updated weights for policy 0, policy_version 1442462 (0.0007) [2023-12-27 15:38:30,140][20381] Updated weights for policy 1, policy_version 1441154 (0.0007) [2023-12-27 15:38:30,169][20347] Updated weights for policy 0, policy_version 1442472 (0.0007) [2023-12-27 15:38:30,199][20381] Updated weights for policy 1, policy_version 1441164 (0.0008) [2023-12-27 15:38:30,792][20347] Updated weights for policy 0, policy_version 1442482 (0.0005) [2023-12-27 15:38:30,848][20347] Updated weights for policy 0, policy_version 1442492 (0.0005) [2023-12-27 15:38:30,905][20347] Updated weights for policy 0, policy_version 1442502 (0.0005) [2023-12-27 15:38:30,906][20381] Updated weights for policy 1, policy_version 1441174 (0.0008) [2023-12-27 15:38:30,952][20381] Updated weights for policy 1, policy_version 1441184 (0.0008) [2023-12-27 15:38:30,996][20381] Updated weights for policy 1, policy_version 1441194 (0.0007) [2023-12-27 15:38:31,578][20347] Updated weights for policy 0, policy_version 1442512 (0.0009) [2023-12-27 15:38:31,610][20291] KL-divergence is very high: 106.1006 [2023-12-27 15:38:31,638][20347] Updated weights for policy 0, policy_version 1442522 (0.0011) [2023-12-27 15:38:31,691][20347] Updated weights for policy 0, policy_version 1442532 (0.0008) [2023-12-27 15:38:31,758][20381] Updated weights for policy 1, policy_version 1441204 (0.0009) [2023-12-27 15:38:31,816][20381] Updated weights for policy 1, policy_version 1441214 (0.0010) [2023-12-27 15:38:31,874][20381] Updated weights for policy 1, policy_version 1441224 (0.0010) [2023-12-27 15:38:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 738344960. Throughput: 0: 10157.0, 1: 10245.4. Samples: 738359888. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:32,363][19170] Avg episode reward: [(0, '921.800'), (1, '319.700')] [2023-12-27 15:38:32,375][20347] Updated weights for policy 0, policy_version 1442542 (0.0008) [2023-12-27 15:38:32,439][20347] Updated weights for policy 0, policy_version 1442552 (0.0008) [2023-12-27 15:38:32,498][20347] Updated weights for policy 0, policy_version 1442562 (0.0007) [2023-12-27 15:38:32,602][20381] Updated weights for policy 1, policy_version 1441234 (0.0010) [2023-12-27 15:38:32,660][20381] Updated weights for policy 1, policy_version 1441244 (0.0010) [2023-12-27 15:38:32,722][20381] Updated weights for policy 1, policy_version 1441254 (0.0010) [2023-12-27 15:38:32,778][20381] Updated weights for policy 1, policy_version 1441264 (0.0007) [2023-12-27 15:38:33,093][20347] Updated weights for policy 0, policy_version 1442572 (0.0005) [2023-12-27 15:38:33,151][20347] Updated weights for policy 0, policy_version 1442582 (0.0006) [2023-12-27 15:38:33,205][20347] Updated weights for policy 0, policy_version 1442592 (0.0006) [2023-12-27 15:38:33,451][20381] Updated weights for policy 1, policy_version 1441274 (0.0008) [2023-12-27 15:38:33,503][20381] Updated weights for policy 1, policy_version 1441284 (0.0010) [2023-12-27 15:38:33,556][20381] Updated weights for policy 1, policy_version 1441294 (0.0010) [2023-12-27 15:38:33,771][20347] Updated weights for policy 0, policy_version 1442602 (0.0007) [2023-12-27 15:38:33,831][20347] Updated weights for policy 0, policy_version 1442612 (0.0006) [2023-12-27 15:38:33,888][20347] Updated weights for policy 0, policy_version 1442622 (0.0005) [2023-12-27 15:38:33,949][20347] Updated weights for policy 0, policy_version 1442632 (0.0005) [2023-12-27 15:38:34,165][20381] Updated weights for policy 1, policy_version 1441304 (0.0008) [2023-12-27 15:38:34,228][20381] Updated weights for policy 1, policy_version 1441314 (0.0008) [2023-12-27 15:38:34,281][20381] Updated weights for policy 1, policy_version 1441324 (0.0008) [2023-12-27 15:38:34,534][20347] Updated weights for policy 0, policy_version 1442642 (0.0008) [2023-12-27 15:38:34,548][20291] KL-divergence is very high: 159.1672 [2023-12-27 15:38:34,599][20347] Updated weights for policy 0, policy_version 1442652 (0.0005) [2023-12-27 15:38:34,601][20291] KL-divergence is very high: 162.8433 [2023-12-27 15:38:34,651][20347] Updated weights for policy 0, policy_version 1442662 (0.0007) [2023-12-27 15:38:35,001][20381] Updated weights for policy 1, policy_version 1441334 (0.0006) [2023-12-27 15:38:35,061][20381] Updated weights for policy 1, policy_version 1441344 (0.0005) [2023-12-27 15:38:35,115][20381] Updated weights for policy 1, policy_version 1441354 (0.0006) [2023-12-27 15:38:35,321][20347] Updated weights for policy 0, policy_version 1442672 (0.0007) [2023-12-27 15:38:35,378][20347] Updated weights for policy 0, policy_version 1442682 (0.0007) [2023-12-27 15:38:35,437][20347] Updated weights for policy 0, policy_version 1442692 (0.0008) [2023-12-27 15:38:35,761][20381] Updated weights for policy 1, policy_version 1441364 (0.0008) [2023-12-27 15:38:35,814][20381] Updated weights for policy 1, policy_version 1441374 (0.0008) [2023-12-27 15:38:35,861][20381] Updated weights for policy 1, policy_version 1441384 (0.0008) [2023-12-27 15:38:36,106][20347] Updated weights for policy 0, policy_version 1442702 (0.0008) [2023-12-27 15:38:36,165][20347] Updated weights for policy 0, policy_version 1442712 (0.0008) [2023-12-27 15:38:36,228][20347] Updated weights for policy 0, policy_version 1442722 (0.0008) [2023-12-27 15:38:36,585][20381] Updated weights for policy 1, policy_version 1441394 (0.0009) [2023-12-27 15:38:36,651][20381] Updated weights for policy 1, policy_version 1441404 (0.0007) [2023-12-27 15:38:36,707][20381] Updated weights for policy 1, policy_version 1441414 (0.0008) [2023-12-27 15:38:36,765][20381] Updated weights for policy 1, policy_version 1441424 (0.0007) [2023-12-27 15:38:36,924][20347] Updated weights for policy 0, policy_version 1442732 (0.0009) [2023-12-27 15:38:36,979][20347] Updated weights for policy 0, policy_version 1442742 (0.0010) [2023-12-27 15:38:37,038][20347] Updated weights for policy 0, policy_version 1442752 (0.0007) [2023-12-27 15:38:37,359][20381] Updated weights for policy 1, policy_version 1441434 (0.0009) [2023-12-27 15:38:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 738451456. Throughput: 0: 10211.0, 1: 10245.1. Samples: 738423736. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:37,363][19170] Avg episode reward: [(0, '902.390'), (1, '351.190')] [2023-12-27 15:38:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001442760_369418240.pth... [2023-12-27 15:38:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001441576_369115136.pth [2023-12-27 15:38:37,408][20381] Updated weights for policy 1, policy_version 1441444 (0.0009) [2023-12-27 15:38:37,454][20381] Updated weights for policy 1, policy_version 1441454 (0.0007) [2023-12-27 15:38:37,461][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001441456_369041408.pth... [2023-12-27 15:38:37,464][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001440240_368730112.pth [2023-12-27 15:38:37,631][20347] Updated weights for policy 0, policy_version 1442762 (0.0006) [2023-12-27 15:38:37,683][20347] Updated weights for policy 0, policy_version 1442772 (0.0008) [2023-12-27 15:38:37,737][20347] Updated weights for policy 0, policy_version 1442782 (0.0007) [2023-12-27 15:38:37,788][20347] Updated weights for policy 0, policy_version 1442792 (0.0008) [2023-12-27 15:38:38,176][20381] Updated weights for policy 1, policy_version 1441464 (0.0007) [2023-12-27 15:38:38,230][20381] Updated weights for policy 1, policy_version 1441474 (0.0005) [2023-12-27 15:38:38,283][20381] Updated weights for policy 1, policy_version 1441484 (0.0008) [2023-12-27 15:38:38,483][20347] Updated weights for policy 0, policy_version 1442802 (0.0008) [2023-12-27 15:38:38,553][20347] Updated weights for policy 0, policy_version 1442812 (0.0008) [2023-12-27 15:38:38,620][20347] Updated weights for policy 0, policy_version 1442822 (0.0008) [2023-12-27 15:38:38,985][20381] Updated weights for policy 1, policy_version 1441494 (0.0006) [2023-12-27 15:38:39,035][20381] Updated weights for policy 1, policy_version 1441504 (0.0007) [2023-12-27 15:38:39,081][20381] Updated weights for policy 1, policy_version 1441514 (0.0008) [2023-12-27 15:38:39,310][20347] Updated weights for policy 0, policy_version 1442832 (0.0008) [2023-12-27 15:38:39,376][20347] Updated weights for policy 0, policy_version 1442842 (0.0009) [2023-12-27 15:38:39,451][20347] Updated weights for policy 0, policy_version 1442852 (0.0008) [2023-12-27 15:38:39,892][20381] Updated weights for policy 1, policy_version 1441524 (0.0008) [2023-12-27 15:38:39,961][20381] Updated weights for policy 1, policy_version 1441534 (0.0009) [2023-12-27 15:38:40,020][20381] Updated weights for policy 1, policy_version 1441544 (0.0008) [2023-12-27 15:38:40,105][20347] Updated weights for policy 0, policy_version 1442862 (0.0008) [2023-12-27 15:38:40,170][20347] Updated weights for policy 0, policy_version 1442872 (0.0007) [2023-12-27 15:38:40,231][20347] Updated weights for policy 0, policy_version 1442882 (0.0007) [2023-12-27 15:38:40,754][20381] Updated weights for policy 1, policy_version 1441554 (0.0009) [2023-12-27 15:38:40,799][20381] Updated weights for policy 1, policy_version 1441564 (0.0010) [2023-12-27 15:38:40,863][20381] Updated weights for policy 1, policy_version 1441574 (0.0010) [2023-12-27 15:38:40,879][20347] Updated weights for policy 0, policy_version 1442892 (0.0006) [2023-12-27 15:38:40,919][20381] Updated weights for policy 1, policy_version 1441584 (0.0010) [2023-12-27 15:38:40,934][20347] Updated weights for policy 0, policy_version 1442902 (0.0005) [2023-12-27 15:38:40,990][20347] Updated weights for policy 0, policy_version 1442912 (0.0007) [2023-12-27 15:38:41,638][20381] Updated weights for policy 1, policy_version 1441594 (0.0008) [2023-12-27 15:38:41,708][20381] Updated weights for policy 1, policy_version 1441604 (0.0008) [2023-12-27 15:38:41,711][20347] Updated weights for policy 0, policy_version 1442922 (0.0008) [2023-12-27 15:38:41,773][20347] Updated weights for policy 0, policy_version 1442932 (0.0008) [2023-12-27 15:38:41,774][20381] Updated weights for policy 1, policy_version 1441614 (0.0008) [2023-12-27 15:38:41,821][20347] Updated weights for policy 0, policy_version 1442942 (0.0008) [2023-12-27 15:38:41,873][20347] Updated weights for policy 0, policy_version 1442952 (0.0008) [2023-12-27 15:38:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 738549760. Throughput: 0: 10193.9, 1: 10170.6. Samples: 738544564. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:42,363][19170] Avg episode reward: [(0, '902.570'), (1, '320.720')] [2023-12-27 15:38:42,475][20381] Updated weights for policy 1, policy_version 1441624 (0.0006) [2023-12-27 15:38:42,543][20381] Updated weights for policy 1, policy_version 1441634 (0.0006) [2023-12-27 15:38:42,590][20347] Updated weights for policy 0, policy_version 1442962 (0.0005) [2023-12-27 15:38:42,603][20381] Updated weights for policy 1, policy_version 1441644 (0.0010) [2023-12-27 15:38:42,648][20347] Updated weights for policy 0, policy_version 1442972 (0.0006) [2023-12-27 15:38:42,696][20347] Updated weights for policy 0, policy_version 1442982 (0.0007) [2023-12-27 15:38:43,259][20381] Updated weights for policy 1, policy_version 1441654 (0.0008) [2023-12-27 15:38:43,310][20381] Updated weights for policy 1, policy_version 1441664 (0.0008) [2023-12-27 15:38:43,366][20381] Updated weights for policy 1, policy_version 1441674 (0.0009) [2023-12-27 15:38:43,399][20347] Updated weights for policy 0, policy_version 1442992 (0.0007) [2023-12-27 15:38:43,448][20347] Updated weights for policy 0, policy_version 1443002 (0.0005) [2023-12-27 15:38:43,505][20347] Updated weights for policy 0, policy_version 1443012 (0.0007) [2023-12-27 15:38:44,008][20381] Updated weights for policy 1, policy_version 1441684 (0.0008) [2023-12-27 15:38:44,063][20381] Updated weights for policy 1, policy_version 1441694 (0.0005) [2023-12-27 15:38:44,122][20381] Updated weights for policy 1, policy_version 1441704 (0.0009) [2023-12-27 15:38:44,139][20347] Updated weights for policy 0, policy_version 1443022 (0.0006) [2023-12-27 15:38:44,190][20347] Updated weights for policy 0, policy_version 1443032 (0.0006) [2023-12-27 15:38:44,240][20347] Updated weights for policy 0, policy_version 1443042 (0.0006) [2023-12-27 15:38:44,712][20381] Updated weights for policy 1, policy_version 1441714 (0.0006) [2023-12-27 15:38:44,767][20381] Updated weights for policy 1, policy_version 1441724 (0.0006) [2023-12-27 15:38:44,803][20347] Updated weights for policy 0, policy_version 1443052 (0.0009) [2023-12-27 15:38:44,825][20381] Updated weights for policy 1, policy_version 1441734 (0.0008) [2023-12-27 15:38:44,861][20347] Updated weights for policy 0, policy_version 1443062 (0.0008) [2023-12-27 15:38:44,881][20381] Updated weights for policy 1, policy_version 1441744 (0.0008) [2023-12-27 15:38:44,922][20347] Updated weights for policy 0, policy_version 1443072 (0.0007) [2023-12-27 15:38:45,577][20381] Updated weights for policy 1, policy_version 1441754 (0.0007) [2023-12-27 15:38:45,626][20381] Updated weights for policy 1, policy_version 1441764 (0.0007) [2023-12-27 15:38:45,633][20347] Updated weights for policy 0, policy_version 1443082 (0.0008) [2023-12-27 15:38:45,681][20347] Updated weights for policy 0, policy_version 1443092 (0.0010) [2023-12-27 15:38:45,684][20381] Updated weights for policy 1, policy_version 1441774 (0.0007) [2023-12-27 15:38:45,743][20347] Updated weights for policy 0, policy_version 1443102 (0.0010) [2023-12-27 15:38:45,804][20347] Updated weights for policy 0, policy_version 1443112 (0.0010) [2023-12-27 15:38:46,237][20381] Updated weights for policy 1, policy_version 1441784 (0.0010) [2023-12-27 15:38:46,296][20381] Updated weights for policy 1, policy_version 1441794 (0.0011) [2023-12-27 15:38:46,344][20381] Updated weights for policy 1, policy_version 1441804 (0.0010) [2023-12-27 15:38:46,494][20347] Updated weights for policy 0, policy_version 1443122 (0.0010) [2023-12-27 15:38:46,555][20347] Updated weights for policy 0, policy_version 1443132 (0.0010) [2023-12-27 15:38:46,607][20347] Updated weights for policy 0, policy_version 1443142 (0.0010) [2023-12-27 15:38:46,971][20381] Updated weights for policy 1, policy_version 1441814 (0.0010) [2023-12-27 15:38:47,017][20381] Updated weights for policy 1, policy_version 1441824 (0.0010) [2023-12-27 15:38:47,067][20381] Updated weights for policy 1, policy_version 1441834 (0.0010) [2023-12-27 15:38:47,254][20347] Updated weights for policy 0, policy_version 1443152 (0.0008) [2023-12-27 15:38:47,301][20347] Updated weights for policy 0, policy_version 1443162 (0.0008) [2023-12-27 15:38:47,360][20347] Updated weights for policy 0, policy_version 1443172 (0.0008) [2023-12-27 15:38:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 738656256. Throughput: 0: 10214.2, 1: 10220.1. Samples: 738669896. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:47,364][19170] Avg episode reward: [(0, '922.040'), (1, '300.400')] [2023-12-27 15:38:47,691][20381] Updated weights for policy 1, policy_version 1441844 (0.0005) [2023-12-27 15:38:47,738][20381] Updated weights for policy 1, policy_version 1441854 (0.0006) [2023-12-27 15:38:47,786][20381] Updated weights for policy 1, policy_version 1441864 (0.0005) [2023-12-27 15:38:48,074][20347] Updated weights for policy 0, policy_version 1443182 (0.0008) [2023-12-27 15:38:48,132][20347] Updated weights for policy 0, policy_version 1443192 (0.0006) [2023-12-27 15:38:48,195][20347] Updated weights for policy 0, policy_version 1443202 (0.0008) [2023-12-27 15:38:48,459][20381] Updated weights for policy 1, policy_version 1441874 (0.0006) [2023-12-27 15:38:48,524][20381] Updated weights for policy 1, policy_version 1441884 (0.0008) [2023-12-27 15:38:48,587][20381] Updated weights for policy 1, policy_version 1441894 (0.0008) [2023-12-27 15:38:48,650][20381] Updated weights for policy 1, policy_version 1441904 (0.0008) [2023-12-27 15:38:48,827][20347] Updated weights for policy 0, policy_version 1443212 (0.0008) [2023-12-27 15:38:48,884][20347] Updated weights for policy 0, policy_version 1443222 (0.0006) [2023-12-27 15:38:48,942][20347] Updated weights for policy 0, policy_version 1443232 (0.0007) [2023-12-27 15:38:49,426][20381] Updated weights for policy 1, policy_version 1441914 (0.0008) [2023-12-27 15:38:49,478][20381] Updated weights for policy 1, policy_version 1441924 (0.0008) [2023-12-27 15:38:49,541][20381] Updated weights for policy 1, policy_version 1441934 (0.0008) [2023-12-27 15:38:49,640][20347] Updated weights for policy 0, policy_version 1443242 (0.0007) [2023-12-27 15:38:49,696][20347] Updated weights for policy 0, policy_version 1443252 (0.0009) [2023-12-27 15:38:49,741][20347] Updated weights for policy 0, policy_version 1443262 (0.0009) [2023-12-27 15:38:49,798][20347] Updated weights for policy 0, policy_version 1443272 (0.0008) [2023-12-27 15:38:50,201][20381] Updated weights for policy 1, policy_version 1441944 (0.0008) [2023-12-27 15:38:50,255][20381] Updated weights for policy 1, policy_version 1441954 (0.0007) [2023-12-27 15:38:50,303][20381] Updated weights for policy 1, policy_version 1441964 (0.0008) [2023-12-27 15:38:50,471][20347] Updated weights for policy 0, policy_version 1443282 (0.0007) [2023-12-27 15:38:50,520][20347] Updated weights for policy 0, policy_version 1443292 (0.0007) [2023-12-27 15:38:50,573][20347] Updated weights for policy 0, policy_version 1443302 (0.0007) [2023-12-27 15:38:51,033][20381] Updated weights for policy 1, policy_version 1441974 (0.0008) [2023-12-27 15:38:51,095][20381] Updated weights for policy 1, policy_version 1441984 (0.0007) [2023-12-27 15:38:51,155][20381] Updated weights for policy 1, policy_version 1441994 (0.0006) [2023-12-27 15:38:51,310][20347] Updated weights for policy 0, policy_version 1443312 (0.0009) [2023-12-27 15:38:51,375][20347] Updated weights for policy 0, policy_version 1443322 (0.0008) [2023-12-27 15:38:51,432][20347] Updated weights for policy 0, policy_version 1443332 (0.0007) [2023-12-27 15:38:51,857][20381] Updated weights for policy 1, policy_version 1442004 (0.0007) [2023-12-27 15:38:51,908][20381] Updated weights for policy 1, policy_version 1442014 (0.0007) [2023-12-27 15:38:51,965][20381] Updated weights for policy 1, policy_version 1442024 (0.0008) [2023-12-27 15:38:52,131][20347] Updated weights for policy 0, policy_version 1443342 (0.0008) [2023-12-27 15:38:52,177][20347] Updated weights for policy 0, policy_version 1443352 (0.0008) [2023-12-27 15:38:52,234][20347] Updated weights for policy 0, policy_version 1443362 (0.0006) [2023-12-27 15:38:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 738762752. Throughput: 0: 10180.3, 1: 10209.3. Samples: 738729780. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:52,363][19170] Avg episode reward: [(0, '902.580'), (1, '295.160')] [2023-12-27 15:38:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001442032_369188864.pth... [2023-12-27 15:38:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001443368_369573888.pth... [2023-12-27 15:38:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001440848_368885760.pth [2023-12-27 15:38:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001442152_369262592.pth [2023-12-27 15:38:52,664][20381] Updated weights for policy 1, policy_version 1442034 (0.0007) [2023-12-27 15:38:52,730][20381] Updated weights for policy 1, policy_version 1442044 (0.0007) [2023-12-27 15:38:52,794][20381] Updated weights for policy 1, policy_version 1442054 (0.0007) [2023-12-27 15:38:52,854][20381] Updated weights for policy 1, policy_version 1442064 (0.0007) [2023-12-27 15:38:53,013][20347] Updated weights for policy 0, policy_version 1443372 (0.0008) [2023-12-27 15:38:53,072][20347] Updated weights for policy 0, policy_version 1443382 (0.0005) [2023-12-27 15:38:53,120][20347] Updated weights for policy 0, policy_version 1443392 (0.0005) [2023-12-27 15:38:53,562][20381] Updated weights for policy 1, policy_version 1442074 (0.0010) [2023-12-27 15:38:53,611][20381] Updated weights for policy 1, policy_version 1442084 (0.0010) [2023-12-27 15:38:53,666][20347] Updated weights for policy 0, policy_version 1443402 (0.0006) [2023-12-27 15:38:53,667][20381] Updated weights for policy 1, policy_version 1442094 (0.0005) [2023-12-27 15:38:53,712][20347] Updated weights for policy 0, policy_version 1443412 (0.0010) [2023-12-27 15:38:53,760][20347] Updated weights for policy 0, policy_version 1443422 (0.0009) [2023-12-27 15:38:53,809][20347] Updated weights for policy 0, policy_version 1443432 (0.0005) [2023-12-27 15:38:54,336][20381] Updated weights for policy 1, policy_version 1442104 (0.0007) [2023-12-27 15:38:54,398][20381] Updated weights for policy 1, policy_version 1442114 (0.0006) [2023-12-27 15:38:54,415][20347] Updated weights for policy 0, policy_version 1443442 (0.0005) [2023-12-27 15:38:54,454][20381] Updated weights for policy 1, policy_version 1442124 (0.0006) [2023-12-27 15:38:54,470][20347] Updated weights for policy 0, policy_version 1443452 (0.0006) [2023-12-27 15:38:54,523][20347] Updated weights for policy 0, policy_version 1443462 (0.0006) [2023-12-27 15:38:55,087][20381] Updated weights for policy 1, policy_version 1442134 (0.0009) [2023-12-27 15:38:55,127][20347] Updated weights for policy 0, policy_version 1443472 (0.0010) [2023-12-27 15:38:55,142][20381] Updated weights for policy 1, policy_version 1442144 (0.0010) [2023-12-27 15:38:55,182][20347] Updated weights for policy 0, policy_version 1443482 (0.0010) [2023-12-27 15:38:55,198][20381] Updated weights for policy 1, policy_version 1442154 (0.0011) [2023-12-27 15:38:55,237][20347] Updated weights for policy 0, policy_version 1443492 (0.0010) [2023-12-27 15:38:55,830][20381] Updated weights for policy 1, policy_version 1442164 (0.0009) [2023-12-27 15:38:55,839][20347] Updated weights for policy 0, policy_version 1443502 (0.0010) [2023-12-27 15:38:55,879][20381] Updated weights for policy 1, policy_version 1442174 (0.0005) [2023-12-27 15:38:55,890][20347] Updated weights for policy 0, policy_version 1443512 (0.0011) [2023-12-27 15:38:55,936][20381] Updated weights for policy 1, policy_version 1442184 (0.0005) [2023-12-27 15:38:55,945][20347] Updated weights for policy 0, policy_version 1443522 (0.0010) [2023-12-27 15:38:56,543][20347] Updated weights for policy 0, policy_version 1443532 (0.0008) [2023-12-27 15:38:56,592][20347] Updated weights for policy 0, policy_version 1443542 (0.0005) [2023-12-27 15:38:56,594][20381] Updated weights for policy 1, policy_version 1442194 (0.0006) [2023-12-27 15:38:56,642][20381] Updated weights for policy 1, policy_version 1442204 (0.0005) [2023-12-27 15:38:56,646][20347] Updated weights for policy 0, policy_version 1443552 (0.0006) [2023-12-27 15:38:56,697][20381] Updated weights for policy 1, policy_version 1442214 (0.0005) [2023-12-27 15:38:56,752][20381] Updated weights for policy 1, policy_version 1442224 (0.0005) [2023-12-27 15:38:57,201][20347] Updated weights for policy 0, policy_version 1443562 (0.0005) [2023-12-27 15:38:57,251][20347] Updated weights for policy 0, policy_version 1443572 (0.0005) [2023-12-27 15:38:57,320][20347] Updated weights for policy 0, policy_version 1443582 (0.0007) [2023-12-27 15:38:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 738861056. Throughput: 0: 10308.7, 1: 10219.1. Samples: 738854560. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:38:57,363][19170] Avg episode reward: [(0, '882.840'), (1, '373.950')] [2023-12-27 15:38:57,376][20347] Updated weights for policy 0, policy_version 1443592 (0.0007) [2023-12-27 15:38:57,421][20381] Updated weights for policy 1, policy_version 1442234 (0.0008) [2023-12-27 15:38:57,470][20381] Updated weights for policy 1, policy_version 1442244 (0.0007) [2023-12-27 15:38:57,519][20381] Updated weights for policy 1, policy_version 1442254 (0.0009) [2023-12-27 15:38:58,001][20347] Updated weights for policy 0, policy_version 1443602 (0.0008) [2023-12-27 15:38:58,055][20347] Updated weights for policy 0, policy_version 1443612 (0.0007) [2023-12-27 15:38:58,110][20347] Updated weights for policy 0, policy_version 1443622 (0.0008) [2023-12-27 15:38:58,241][20381] Updated weights for policy 1, policy_version 1442264 (0.0007) [2023-12-27 15:38:58,297][20381] Updated weights for policy 1, policy_version 1442274 (0.0007) [2023-12-27 15:38:58,371][20381] Updated weights for policy 1, policy_version 1442284 (0.0008) [2023-12-27 15:38:58,929][20347] Updated weights for policy 0, policy_version 1443632 (0.0008) [2023-12-27 15:38:58,993][20347] Updated weights for policy 0, policy_version 1443642 (0.0011) [2023-12-27 15:38:59,056][20347] Updated weights for policy 0, policy_version 1443652 (0.0011) [2023-12-27 15:38:59,104][20381] Updated weights for policy 1, policy_version 1442294 (0.0008) [2023-12-27 15:38:59,163][20381] Updated weights for policy 1, policy_version 1442304 (0.0007) [2023-12-27 15:38:59,223][20381] Updated weights for policy 1, policy_version 1442314 (0.0008) [2023-12-27 15:38:59,773][20347] Updated weights for policy 0, policy_version 1443662 (0.0008) [2023-12-27 15:38:59,837][20347] Updated weights for policy 0, policy_version 1443672 (0.0008) [2023-12-27 15:38:59,898][20347] Updated weights for policy 0, policy_version 1443682 (0.0010) [2023-12-27 15:39:00,047][20381] Updated weights for policy 1, policy_version 1442324 (0.0009) [2023-12-27 15:39:00,101][20381] Updated weights for policy 1, policy_version 1442334 (0.0008) [2023-12-27 15:39:00,154][20381] Updated weights for policy 1, policy_version 1442344 (0.0008) [2023-12-27 15:39:00,568][20347] Updated weights for policy 0, policy_version 1443692 (0.0010) [2023-12-27 15:39:00,625][20347] Updated weights for policy 0, policy_version 1443702 (0.0010) [2023-12-27 15:39:00,678][20347] Updated weights for policy 0, policy_version 1443712 (0.0010) [2023-12-27 15:39:00,887][20381] Updated weights for policy 1, policy_version 1442354 (0.0007) [2023-12-27 15:39:00,945][20381] Updated weights for policy 1, policy_version 1442364 (0.0006) [2023-12-27 15:39:01,011][20381] Updated weights for policy 1, policy_version 1442374 (0.0006) [2023-12-27 15:39:01,073][20381] Updated weights for policy 1, policy_version 1442384 (0.0008) [2023-12-27 15:39:01,342][20347] Updated weights for policy 0, policy_version 1443722 (0.0010) [2023-12-27 15:39:01,403][20347] Updated weights for policy 0, policy_version 1443732 (0.0007) [2023-12-27 15:39:01,469][20347] Updated weights for policy 0, policy_version 1443742 (0.0007) [2023-12-27 15:39:01,497][20291] KL-divergence is very high: 116.1794 [2023-12-27 15:39:01,504][20291] KL-divergence is very high: 104.7257 [2023-12-27 15:39:01,529][20347] Updated weights for policy 0, policy_version 1443752 (0.0008) [2023-12-27 15:39:01,837][20381] Updated weights for policy 1, policy_version 1442394 (0.0005) [2023-12-27 15:39:01,892][20381] Updated weights for policy 1, policy_version 1442404 (0.0007) [2023-12-27 15:39:01,944][20381] Updated weights for policy 1, policy_version 1442414 (0.0007) [2023-12-27 15:39:02,248][20347] Updated weights for policy 0, policy_version 1443762 (0.0007) [2023-12-27 15:39:02,313][20347] Updated weights for policy 0, policy_version 1443772 (0.0006) [2023-12-27 15:39:02,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 738959360. Throughput: 0: 10419.3, 1: 10136.1. Samples: 738974856. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:02,363][19170] Avg episode reward: [(0, '853.410'), (1, '350.760')] [2023-12-27 15:39:02,376][20347] Updated weights for policy 0, policy_version 1443782 (0.0007) [2023-12-27 15:39:02,581][20381] Updated weights for policy 1, policy_version 1442424 (0.0008) [2023-12-27 15:39:02,628][20381] Updated weights for policy 1, policy_version 1442434 (0.0007) [2023-12-27 15:39:02,684][20381] Updated weights for policy 1, policy_version 1442444 (0.0008) [2023-12-27 15:39:02,996][20347] Updated weights for policy 0, policy_version 1443792 (0.0008) [2023-12-27 15:39:03,044][20347] Updated weights for policy 0, policy_version 1443802 (0.0007) [2023-12-27 15:39:03,098][20347] Updated weights for policy 0, policy_version 1443812 (0.0007) [2023-12-27 15:39:03,365][20381] Updated weights for policy 1, policy_version 1442454 (0.0008) [2023-12-27 15:39:03,413][20381] Updated weights for policy 1, policy_version 1442464 (0.0008) [2023-12-27 15:39:03,455][20381] Updated weights for policy 1, policy_version 1442474 (0.0007) [2023-12-27 15:39:03,772][20347] Updated weights for policy 0, policy_version 1443822 (0.0006) [2023-12-27 15:39:03,818][20347] Updated weights for policy 0, policy_version 1443832 (0.0005) [2023-12-27 15:39:03,876][20347] Updated weights for policy 0, policy_version 1443842 (0.0007) [2023-12-27 15:39:04,181][20381] Updated weights for policy 1, policy_version 1442484 (0.0009) [2023-12-27 15:39:04,243][20381] Updated weights for policy 1, policy_version 1442494 (0.0010) [2023-12-27 15:39:04,308][20381] Updated weights for policy 1, policy_version 1442504 (0.0008) [2023-12-27 15:39:04,525][20347] Updated weights for policy 0, policy_version 1443852 (0.0008) [2023-12-27 15:39:04,588][20347] Updated weights for policy 0, policy_version 1443862 (0.0008) [2023-12-27 15:39:04,644][20347] Updated weights for policy 0, policy_version 1443872 (0.0008) [2023-12-27 15:39:04,940][20381] Updated weights for policy 1, policy_version 1442514 (0.0007) [2023-12-27 15:39:04,994][20381] Updated weights for policy 1, policy_version 1442524 (0.0005) [2023-12-27 15:39:05,048][20381] Updated weights for policy 1, policy_version 1442534 (0.0006) [2023-12-27 15:39:05,098][20381] Updated weights for policy 1, policy_version 1442544 (0.0005) [2023-12-27 15:39:05,385][20347] Updated weights for policy 0, policy_version 1443882 (0.0007) [2023-12-27 15:39:05,442][20347] Updated weights for policy 0, policy_version 1443892 (0.0007) [2023-12-27 15:39:05,500][20347] Updated weights for policy 0, policy_version 1443902 (0.0007) [2023-12-27 15:39:05,548][20347] Updated weights for policy 0, policy_version 1443912 (0.0007) [2023-12-27 15:39:05,692][20381] Updated weights for policy 1, policy_version 1442554 (0.0007) [2023-12-27 15:39:05,737][20381] Updated weights for policy 1, policy_version 1442564 (0.0008) [2023-12-27 15:39:05,788][20381] Updated weights for policy 1, policy_version 1442574 (0.0007) [2023-12-27 15:39:06,278][20347] Updated weights for policy 0, policy_version 1443922 (0.0008) [2023-12-27 15:39:06,336][20347] Updated weights for policy 0, policy_version 1443932 (0.0007) [2023-12-27 15:39:06,400][20347] Updated weights for policy 0, policy_version 1443942 (0.0006) [2023-12-27 15:39:06,452][20381] Updated weights for policy 1, policy_version 1442584 (0.0008) [2023-12-27 15:39:06,506][20381] Updated weights for policy 1, policy_version 1442594 (0.0008) [2023-12-27 15:39:06,567][20381] Updated weights for policy 1, policy_version 1442604 (0.0008) [2023-12-27 15:39:07,012][20347] Updated weights for policy 0, policy_version 1443952 (0.0007) [2023-12-27 15:39:07,068][20347] Updated weights for policy 0, policy_version 1443962 (0.0008) [2023-12-27 15:39:07,120][20347] Updated weights for policy 0, policy_version 1443972 (0.0006) [2023-12-27 15:39:07,305][20381] Updated weights for policy 1, policy_version 1442614 (0.0008) [2023-12-27 15:39:07,361][20381] Updated weights for policy 1, policy_version 1442624 (0.0008) [2023-12-27 15:39:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.6, 300 sec: 20577.2). Total num frames: 739065856. Throughput: 0: 10411.2, 1: 10182.7. Samples: 739037524. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:07,363][19170] Avg episode reward: [(0, '814.200'), (1, '313.560')] [2023-12-27 15:39:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001443976_369729536.pth... [2023-12-27 15:39:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001442760_369418240.pth [2023-12-27 15:39:07,420][20381] Updated weights for policy 1, policy_version 1442634 (0.0008) [2023-12-27 15:39:07,454][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001442640_369344512.pth... [2023-12-27 15:39:07,459][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001441456_369041408.pth [2023-12-27 15:39:07,740][20347] Updated weights for policy 0, policy_version 1443982 (0.0006) [2023-12-27 15:39:07,789][20347] Updated weights for policy 0, policy_version 1443992 (0.0005) [2023-12-27 15:39:07,836][20347] Updated weights for policy 0, policy_version 1444002 (0.0005) [2023-12-27 15:39:08,120][20381] Updated weights for policy 1, policy_version 1442644 (0.0008) [2023-12-27 15:39:08,180][20381] Updated weights for policy 1, policy_version 1442654 (0.0008) [2023-12-27 15:39:08,233][20381] Updated weights for policy 1, policy_version 1442664 (0.0007) [2023-12-27 15:39:08,492][20347] Updated weights for policy 0, policy_version 1444012 (0.0005) [2023-12-27 15:39:08,544][20347] Updated weights for policy 0, policy_version 1444022 (0.0005) [2023-12-27 15:39:08,599][20347] Updated weights for policy 0, policy_version 1444032 (0.0006) [2023-12-27 15:39:08,937][20381] Updated weights for policy 1, policy_version 1442674 (0.0006) [2023-12-27 15:39:08,989][20381] Updated weights for policy 1, policy_version 1442684 (0.0008) [2023-12-27 15:39:09,041][20381] Updated weights for policy 1, policy_version 1442694 (0.0008) [2023-12-27 15:39:09,086][20381] Updated weights for policy 1, policy_version 1442704 (0.0008) [2023-12-27 15:39:09,197][20347] Updated weights for policy 0, policy_version 1444042 (0.0006) [2023-12-27 15:39:09,267][20347] Updated weights for policy 0, policy_version 1444052 (0.0008) [2023-12-27 15:39:09,327][20347] Updated weights for policy 0, policy_version 1444062 (0.0006) [2023-12-27 15:39:09,397][20347] Updated weights for policy 0, policy_version 1444072 (0.0008) [2023-12-27 15:39:09,837][20381] Updated weights for policy 1, policy_version 1442714 (0.0008) [2023-12-27 15:39:09,898][20381] Updated weights for policy 1, policy_version 1442724 (0.0009) [2023-12-27 15:39:09,968][20381] Updated weights for policy 1, policy_version 1442734 (0.0009) [2023-12-27 15:39:10,126][20347] Updated weights for policy 0, policy_version 1444082 (0.0008) [2023-12-27 15:39:10,184][20347] Updated weights for policy 0, policy_version 1444092 (0.0008) [2023-12-27 15:39:10,247][20347] Updated weights for policy 0, policy_version 1444102 (0.0008) [2023-12-27 15:39:10,254][20291] KL-divergence is very high: 472.0099 [2023-12-27 15:39:10,634][20381] Updated weights for policy 1, policy_version 1442744 (0.0008) [2023-12-27 15:39:10,680][20381] Updated weights for policy 1, policy_version 1442754 (0.0007) [2023-12-27 15:39:10,728][20381] Updated weights for policy 1, policy_version 1442764 (0.0008) [2023-12-27 15:39:10,921][20347] Updated weights for policy 0, policy_version 1444112 (0.0008) [2023-12-27 15:39:10,960][20291] KL-divergence is very high: 142.8725 [2023-12-27 15:39:10,993][20347] Updated weights for policy 0, policy_version 1444122 (0.0007) [2023-12-27 15:39:11,015][20291] KL-divergence is very high: 114.8901 [2023-12-27 15:39:11,066][20347] Updated weights for policy 0, policy_version 1444132 (0.0008) [2023-12-27 15:39:11,512][20381] Updated weights for policy 1, policy_version 1442774 (0.0007) [2023-12-27 15:39:11,573][20381] Updated weights for policy 1, policy_version 1442784 (0.0006) [2023-12-27 15:39:11,639][20381] Updated weights for policy 1, policy_version 1442794 (0.0008) [2023-12-27 15:39:11,834][20347] Updated weights for policy 0, policy_version 1444142 (0.0007) [2023-12-27 15:39:11,852][20291] KL-divergence is very high: 102.5041 [2023-12-27 15:39:11,891][20347] Updated weights for policy 0, policy_version 1444152 (0.0007) [2023-12-27 15:39:11,954][20347] Updated weights for policy 0, policy_version 1444162 (0.0007) [2023-12-27 15:39:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 739164160. Throughput: 0: 10432.3, 1: 10129.0. Samples: 739159264. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:12,363][19170] Avg episode reward: [(0, '802.580'), (1, '376.040')] [2023-12-27 15:39:12,389][20381] Updated weights for policy 1, policy_version 1442804 (0.0008) [2023-12-27 15:39:12,451][20381] Updated weights for policy 1, policy_version 1442814 (0.0006) [2023-12-27 15:39:12,506][20381] Updated weights for policy 1, policy_version 1442824 (0.0008) [2023-12-27 15:39:12,617][20347] Updated weights for policy 0, policy_version 1444172 (0.0009) [2023-12-27 15:39:12,668][20347] Updated weights for policy 0, policy_version 1444182 (0.0010) [2023-12-27 15:39:12,725][20347] Updated weights for policy 0, policy_version 1444192 (0.0006) [2023-12-27 15:39:13,234][20381] Updated weights for policy 1, policy_version 1442834 (0.0007) [2023-12-27 15:39:13,278][20347] Updated weights for policy 0, policy_version 1444202 (0.0005) [2023-12-27 15:39:13,294][20381] Updated weights for policy 1, policy_version 1442844 (0.0005) [2023-12-27 15:39:13,335][20347] Updated weights for policy 0, policy_version 1444212 (0.0009) [2023-12-27 15:39:13,345][20381] Updated weights for policy 1, policy_version 1442854 (0.0005) [2023-12-27 15:39:13,390][20347] Updated weights for policy 0, policy_version 1444222 (0.0010) [2023-12-27 15:39:13,398][20381] Updated weights for policy 1, policy_version 1442864 (0.0006) [2023-12-27 15:39:13,444][20347] Updated weights for policy 0, policy_version 1444232 (0.0010) [2023-12-27 15:39:14,022][20381] Updated weights for policy 1, policy_version 1442874 (0.0008) [2023-12-27 15:39:14,076][20381] Updated weights for policy 1, policy_version 1442884 (0.0008) [2023-12-27 15:39:14,102][20347] Updated weights for policy 0, policy_version 1444242 (0.0005) [2023-12-27 15:39:14,131][20381] Updated weights for policy 1, policy_version 1442894 (0.0006) [2023-12-27 15:39:14,163][20347] Updated weights for policy 0, policy_version 1444252 (0.0010) [2023-12-27 15:39:14,227][20347] Updated weights for policy 0, policy_version 1444262 (0.0010) [2023-12-27 15:39:14,768][20381] Updated weights for policy 1, policy_version 1442904 (0.0007) [2023-12-27 15:39:14,836][20381] Updated weights for policy 1, policy_version 1442914 (0.0007) [2023-12-27 15:39:14,893][20347] Updated weights for policy 0, policy_version 1444272 (0.0009) [2023-12-27 15:39:14,898][20381] Updated weights for policy 1, policy_version 1442924 (0.0007) [2023-12-27 15:39:14,953][20347] Updated weights for policy 0, policy_version 1444282 (0.0008) [2023-12-27 15:39:15,013][20347] Updated weights for policy 0, policy_version 1444292 (0.0008) [2023-12-27 15:39:15,601][20381] Updated weights for policy 1, policy_version 1442934 (0.0008) [2023-12-27 15:39:15,654][20381] Updated weights for policy 1, policy_version 1442944 (0.0008) [2023-12-27 15:39:15,686][20347] Updated weights for policy 0, policy_version 1444302 (0.0006) [2023-12-27 15:39:15,704][20381] Updated weights for policy 1, policy_version 1442954 (0.0008) [2023-12-27 15:39:15,741][20347] Updated weights for policy 0, policy_version 1444312 (0.0007) [2023-12-27 15:39:15,800][20347] Updated weights for policy 0, policy_version 1444322 (0.0006) [2023-12-27 15:39:16,387][20381] Updated weights for policy 1, policy_version 1442964 (0.0006) [2023-12-27 15:39:16,435][20347] Updated weights for policy 0, policy_version 1444332 (0.0007) [2023-12-27 15:39:16,446][20381] Updated weights for policy 1, policy_version 1442974 (0.0008) [2023-12-27 15:39:16,496][20347] Updated weights for policy 0, policy_version 1444342 (0.0010) [2023-12-27 15:39:16,503][20381] Updated weights for policy 1, policy_version 1442984 (0.0010) [2023-12-27 15:39:16,553][20347] Updated weights for policy 0, policy_version 1444352 (0.0010) [2023-12-27 15:39:17,135][20347] Updated weights for policy 0, policy_version 1444362 (0.0009) [2023-12-27 15:39:17,137][20381] Updated weights for policy 1, policy_version 1442994 (0.0010) [2023-12-27 15:39:17,186][20347] Updated weights for policy 0, policy_version 1444372 (0.0006) [2023-12-27 15:39:17,190][20381] Updated weights for policy 1, policy_version 1443004 (0.0010) [2023-12-27 15:39:17,234][20347] Updated weights for policy 0, policy_version 1444382 (0.0006) [2023-12-27 15:39:17,236][20381] Updated weights for policy 1, policy_version 1443014 (0.0010) [2023-12-27 15:39:17,282][20347] Updated weights for policy 0, policy_version 1444392 (0.0006) [2023-12-27 15:39:17,300][20381] Updated weights for policy 1, policy_version 1443024 (0.0011) [2023-12-27 15:39:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20605.0). Total num frames: 739278848. Throughput: 0: 10404.8, 1: 10113.2. Samples: 739283196. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:17,363][19170] Avg episode reward: [(0, '717.290'), (1, '293.160')] [2023-12-27 15:39:17,891][20347] Updated weights for policy 0, policy_version 1444402 (0.0005) [2023-12-27 15:39:17,940][20347] Updated weights for policy 0, policy_version 1444412 (0.0007) [2023-12-27 15:39:17,960][20381] Updated weights for policy 1, policy_version 1443034 (0.0006) [2023-12-27 15:39:17,993][20347] Updated weights for policy 0, policy_version 1444422 (0.0006) [2023-12-27 15:39:18,012][20381] Updated weights for policy 1, policy_version 1443044 (0.0006) [2023-12-27 15:39:18,069][20381] Updated weights for policy 1, policy_version 1443054 (0.0007) [2023-12-27 15:39:18,616][20347] Updated weights for policy 0, policy_version 1444432 (0.0007) [2023-12-27 15:39:18,669][20347] Updated weights for policy 0, policy_version 1444442 (0.0007) [2023-12-27 15:39:18,721][20347] Updated weights for policy 0, policy_version 1444452 (0.0006) [2023-12-27 15:39:18,736][20381] Updated weights for policy 1, policy_version 1443064 (0.0007) [2023-12-27 15:39:18,793][20381] Updated weights for policy 1, policy_version 1443074 (0.0008) [2023-12-27 15:39:18,844][20381] Updated weights for policy 1, policy_version 1443084 (0.0006) [2023-12-27 15:39:19,402][20347] Updated weights for policy 0, policy_version 1444462 (0.0006) [2023-12-27 15:39:19,467][20347] Updated weights for policy 0, policy_version 1444472 (0.0006) [2023-12-27 15:39:19,525][20347] Updated weights for policy 0, policy_version 1444482 (0.0008) [2023-12-27 15:39:19,603][20381] Updated weights for policy 1, policy_version 1443094 (0.0008) [2023-12-27 15:39:19,666][20381] Updated weights for policy 1, policy_version 1443104 (0.0008) [2023-12-27 15:39:19,725][20381] Updated weights for policy 1, policy_version 1443114 (0.0007) [2023-12-27 15:39:20,232][20347] Updated weights for policy 0, policy_version 1444492 (0.0007) [2023-12-27 15:39:20,285][20347] Updated weights for policy 0, policy_version 1444502 (0.0007) [2023-12-27 15:39:20,341][20347] Updated weights for policy 0, policy_version 1444512 (0.0008) [2023-12-27 15:39:20,441][20381] Updated weights for policy 1, policy_version 1443124 (0.0007) [2023-12-27 15:39:20,499][20381] Updated weights for policy 1, policy_version 1443134 (0.0008) [2023-12-27 15:39:20,549][20381] Updated weights for policy 1, policy_version 1443144 (0.0008) [2023-12-27 15:39:21,032][20347] Updated weights for policy 0, policy_version 1444522 (0.0007) [2023-12-27 15:39:21,094][20347] Updated weights for policy 0, policy_version 1444532 (0.0008) [2023-12-27 15:39:21,163][20347] Updated weights for policy 0, policy_version 1444542 (0.0009) [2023-12-27 15:39:21,225][20347] Updated weights for policy 0, policy_version 1444552 (0.0008) [2023-12-27 15:39:21,265][20381] Updated weights for policy 1, policy_version 1443154 (0.0008) [2023-12-27 15:39:21,320][20381] Updated weights for policy 1, policy_version 1443164 (0.0006) [2023-12-27 15:39:21,392][20381] Updated weights for policy 1, policy_version 1443174 (0.0007) [2023-12-27 15:39:21,459][20381] Updated weights for policy 1, policy_version 1443184 (0.0007) [2023-12-27 15:39:21,946][20347] Updated weights for policy 0, policy_version 1444562 (0.0008) [2023-12-27 15:39:22,006][20347] Updated weights for policy 0, policy_version 1444572 (0.0007) [2023-12-27 15:39:22,068][20347] Updated weights for policy 0, policy_version 1444582 (0.0007) [2023-12-27 15:39:22,163][20381] Updated weights for policy 1, policy_version 1443194 (0.0008) [2023-12-27 15:39:22,223][20381] Updated weights for policy 1, policy_version 1443204 (0.0007) [2023-12-27 15:39:22,283][20381] Updated weights for policy 1, policy_version 1443214 (0.0008) [2023-12-27 15:39:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 739377152. Throughput: 0: 10380.8, 1: 10094.8. Samples: 739345140. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:22,363][19170] Avg episode reward: [(0, '687.300'), (1, '106.690')] [2023-12-27 15:39:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001443216_369491968.pth... [2023-12-27 15:39:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001444584_369885184.pth... [2023-12-27 15:39:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001442032_369188864.pth [2023-12-27 15:39:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001443368_369573888.pth [2023-12-27 15:39:22,755][20347] Updated weights for policy 0, policy_version 1444592 (0.0008) [2023-12-27 15:39:22,808][20347] Updated weights for policy 0, policy_version 1444602 (0.0008) [2023-12-27 15:39:22,858][20347] Updated weights for policy 0, policy_version 1444612 (0.0008) [2023-12-27 15:39:23,016][20381] Updated weights for policy 1, policy_version 1443224 (0.0007) [2023-12-27 15:39:23,079][20381] Updated weights for policy 1, policy_version 1443234 (0.0008) [2023-12-27 15:39:23,131][20381] Updated weights for policy 1, policy_version 1443244 (0.0007) [2023-12-27 15:39:23,572][20347] Updated weights for policy 0, policy_version 1444622 (0.0008) [2023-12-27 15:39:23,631][20347] Updated weights for policy 0, policy_version 1444632 (0.0008) [2023-12-27 15:39:23,679][20347] Updated weights for policy 0, policy_version 1444642 (0.0007) [2023-12-27 15:39:23,734][20381] Updated weights for policy 1, policy_version 1443254 (0.0007) [2023-12-27 15:39:23,799][20381] Updated weights for policy 1, policy_version 1443264 (0.0008) [2023-12-27 15:39:23,851][20381] Updated weights for policy 1, policy_version 1443274 (0.0008) [2023-12-27 15:39:24,386][20347] Updated weights for policy 0, policy_version 1444652 (0.0008) [2023-12-27 15:39:24,430][20347] Updated weights for policy 0, policy_version 1444662 (0.0007) [2023-12-27 15:39:24,477][20347] Updated weights for policy 0, policy_version 1444672 (0.0007) [2023-12-27 15:39:24,522][20381] Updated weights for policy 1, policy_version 1443284 (0.0009) [2023-12-27 15:39:24,583][20381] Updated weights for policy 1, policy_version 1443294 (0.0011) [2023-12-27 15:39:24,642][20381] Updated weights for policy 1, policy_version 1443304 (0.0011) [2023-12-27 15:39:25,180][20347] Updated weights for policy 0, policy_version 1444682 (0.0008) [2023-12-27 15:39:25,231][20381] Updated weights for policy 1, policy_version 1443314 (0.0009) [2023-12-27 15:39:25,234][20347] Updated weights for policy 0, policy_version 1444692 (0.0007) [2023-12-27 15:39:25,287][20347] Updated weights for policy 0, policy_version 1444702 (0.0005) [2023-12-27 15:39:25,287][20381] Updated weights for policy 1, policy_version 1443324 (0.0008) [2023-12-27 15:39:25,335][20347] Updated weights for policy 0, policy_version 1444712 (0.0005) [2023-12-27 15:39:25,336][20381] Updated weights for policy 1, policy_version 1443334 (0.0007) [2023-12-27 15:39:25,389][20381] Updated weights for policy 1, policy_version 1443344 (0.0007) [2023-12-27 15:39:25,971][20381] Updated weights for policy 1, policy_version 1443354 (0.0008) [2023-12-27 15:39:25,978][20347] Updated weights for policy 0, policy_version 1444722 (0.0006) [2023-12-27 15:39:26,024][20381] Updated weights for policy 1, policy_version 1443364 (0.0011) [2023-12-27 15:39:26,038][20347] Updated weights for policy 0, policy_version 1444732 (0.0007) [2023-12-27 15:39:26,076][20381] Updated weights for policy 1, policy_version 1443374 (0.0011) [2023-12-27 15:39:26,093][20347] Updated weights for policy 0, policy_version 1444742 (0.0007) [2023-12-27 15:39:26,705][20381] Updated weights for policy 1, policy_version 1443384 (0.0009) [2023-12-27 15:39:26,718][20347] Updated weights for policy 0, policy_version 1444752 (0.0006) [2023-12-27 15:39:26,759][20381] Updated weights for policy 1, policy_version 1443394 (0.0008) [2023-12-27 15:39:26,771][20347] Updated weights for policy 0, policy_version 1444762 (0.0005) [2023-12-27 15:39:26,812][20381] Updated weights for policy 1, policy_version 1443404 (0.0008) [2023-12-27 15:39:26,828][20347] Updated weights for policy 0, policy_version 1444772 (0.0005) [2023-12-27 15:39:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 739475456. Throughput: 0: 10357.5, 1: 10156.9. Samples: 739467712. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:27,363][19170] Avg episode reward: [(0, '683.740'), (1, '65.430')] [2023-12-27 15:39:27,433][20381] Updated weights for policy 1, policy_version 1443414 (0.0007) [2023-12-27 15:39:27,446][20347] Updated weights for policy 0, policy_version 1444782 (0.0006) [2023-12-27 15:39:27,488][20381] Updated weights for policy 1, policy_version 1443424 (0.0007) [2023-12-27 15:39:27,494][20347] Updated weights for policy 0, policy_version 1444792 (0.0005) [2023-12-27 15:39:27,537][20381] Updated weights for policy 1, policy_version 1443434 (0.0007) [2023-12-27 15:39:27,544][20347] Updated weights for policy 0, policy_version 1444802 (0.0007) [2023-12-27 15:39:28,182][20381] Updated weights for policy 1, policy_version 1443444 (0.0008) [2023-12-27 15:39:28,227][20347] Updated weights for policy 0, policy_version 1444812 (0.0008) [2023-12-27 15:39:28,235][20381] Updated weights for policy 1, policy_version 1443454 (0.0008) [2023-12-27 15:39:28,281][20347] Updated weights for policy 0, policy_version 1444822 (0.0010) [2023-12-27 15:39:28,283][20381] Updated weights for policy 1, policy_version 1443464 (0.0005) [2023-12-27 15:39:28,346][20347] Updated weights for policy 0, policy_version 1444832 (0.0010) [2023-12-27 15:39:28,942][20381] Updated weights for policy 1, policy_version 1443474 (0.0006) [2023-12-27 15:39:28,988][20381] Updated weights for policy 1, policy_version 1443484 (0.0006) [2023-12-27 15:39:29,037][20347] Updated weights for policy 0, policy_version 1444842 (0.0010) [2023-12-27 15:39:29,039][20381] Updated weights for policy 1, policy_version 1443494 (0.0007) [2023-12-27 15:39:29,087][20381] Updated weights for policy 1, policy_version 1443504 (0.0005) [2023-12-27 15:39:29,089][20347] Updated weights for policy 0, policy_version 1444852 (0.0005) [2023-12-27 15:39:29,147][20347] Updated weights for policy 0, policy_version 1444862 (0.0007) [2023-12-27 15:39:29,207][20347] Updated weights for policy 0, policy_version 1444872 (0.0008) [2023-12-27 15:39:29,820][20347] Updated weights for policy 0, policy_version 1444882 (0.0006) [2023-12-27 15:39:29,848][20381] Updated weights for policy 1, policy_version 1443514 (0.0008) [2023-12-27 15:39:29,883][20347] Updated weights for policy 0, policy_version 1444892 (0.0006) [2023-12-27 15:39:29,908][20381] Updated weights for policy 1, policy_version 1443524 (0.0007) [2023-12-27 15:39:29,950][20347] Updated weights for policy 0, policy_version 1444902 (0.0007) [2023-12-27 15:39:29,965][20381] Updated weights for policy 1, policy_version 1443534 (0.0006) [2023-12-27 15:39:30,559][20347] Updated weights for policy 0, policy_version 1444912 (0.0008) [2023-12-27 15:39:30,615][20347] Updated weights for policy 0, policy_version 1444922 (0.0007) [2023-12-27 15:39:30,632][20381] Updated weights for policy 1, policy_version 1443544 (0.0006) [2023-12-27 15:39:30,676][20347] Updated weights for policy 0, policy_version 1444932 (0.0007) [2023-12-27 15:39:30,681][20381] Updated weights for policy 1, policy_version 1443554 (0.0007) [2023-12-27 15:39:30,729][20381] Updated weights for policy 1, policy_version 1443564 (0.0008) [2023-12-27 15:39:31,290][20291] KL-divergence is very high: 150.5194 [2023-12-27 15:39:31,301][20347] Updated weights for policy 0, policy_version 1444942 (0.0007) [2023-12-27 15:39:31,307][20291] KL-divergence is very high: 285.1357 [2023-12-27 15:39:31,313][20291] KL-divergence is very high: 302.2098 [2023-12-27 15:39:31,318][20291] KL-divergence is very high: 351.4775 [2023-12-27 15:39:31,334][20291] KL-divergence is very high: 334.6941 [2023-12-27 15:39:31,353][20291] KL-divergence is very high: 306.8774 [2023-12-27 15:39:31,360][20347] Updated weights for policy 0, policy_version 1444952 (0.0008) [2023-12-27 15:39:31,361][20291] KL-divergence is very high: 273.0135 [2023-12-27 15:39:31,369][20291] KL-divergence is very high: 269.7732 [2023-12-27 15:39:31,389][20291] KL-divergence is very high: 150.2709 [2023-12-27 15:39:31,409][20291] KL-divergence is very high: 108.4254 [2023-12-27 15:39:31,428][20381] Updated weights for policy 1, policy_version 1443574 (0.0007) [2023-12-27 15:39:31,428][20347] Updated weights for policy 0, policy_version 1444962 (0.0008) [2023-12-27 15:39:31,486][20381] Updated weights for policy 1, policy_version 1443584 (0.0006) [2023-12-27 15:39:31,542][20381] Updated weights for policy 1, policy_version 1443594 (0.0007) [2023-12-27 15:39:32,162][20381] Updated weights for policy 1, policy_version 1443604 (0.0009) [2023-12-27 15:39:32,187][20291] KL-divergence is very high: 173.8763 [2023-12-27 15:39:32,211][20347] Updated weights for policy 0, policy_version 1444972 (0.0009) [2023-12-27 15:39:32,213][20381] Updated weights for policy 1, policy_version 1443614 (0.0007) [2023-12-27 15:39:32,236][20291] KL-divergence is very high: 189.3305 [2023-12-27 15:39:32,269][20381] Updated weights for policy 1, policy_version 1443624 (0.0006) [2023-12-27 15:39:32,277][20347] Updated weights for policy 0, policy_version 1444982 (0.0010) [2023-12-27 15:39:32,289][20291] KL-divergence is very high: 284.6683 [2023-12-27 15:39:32,339][20347] Updated weights for policy 0, policy_version 1444992 (0.0008) [2023-12-27 15:39:32,340][20291] KL-divergence is very high: 288.2187 [2023-12-27 15:39:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 739581952. Throughput: 0: 10382.1, 1: 10148.4. Samples: 739593764. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:32,363][19170] Avg episode reward: [(0, '773.840'), (1, '51.850')] [2023-12-27 15:39:32,961][20347] Updated weights for policy 0, policy_version 1445002 (0.0007) [2023-12-27 15:39:32,974][20381] Updated weights for policy 1, policy_version 1443634 (0.0007) [2023-12-27 15:39:33,016][20347] Updated weights for policy 0, policy_version 1445012 (0.0005) [2023-12-27 15:39:33,023][20381] Updated weights for policy 1, policy_version 1443644 (0.0006) [2023-12-27 15:39:33,066][20347] Updated weights for policy 0, policy_version 1445022 (0.0005) [2023-12-27 15:39:33,068][20381] Updated weights for policy 1, policy_version 1443654 (0.0007) [2023-12-27 15:39:33,117][20381] Updated weights for policy 1, policy_version 1443664 (0.0008) [2023-12-27 15:39:33,118][20347] Updated weights for policy 0, policy_version 1445032 (0.0005) [2023-12-27 15:39:33,724][20347] Updated weights for policy 0, policy_version 1445042 (0.0010) [2023-12-27 15:39:33,778][20347] Updated weights for policy 0, policy_version 1445052 (0.0010) [2023-12-27 15:39:33,834][20347] Updated weights for policy 0, policy_version 1445062 (0.0007) [2023-12-27 15:39:33,836][20381] Updated weights for policy 1, policy_version 1443674 (0.0008) [2023-12-27 15:39:33,891][20381] Updated weights for policy 1, policy_version 1443684 (0.0007) [2023-12-27 15:39:33,943][20381] Updated weights for policy 1, policy_version 1443694 (0.0008) [2023-12-27 15:39:34,435][20347] Updated weights for policy 0, policy_version 1445072 (0.0006) [2023-12-27 15:39:34,498][20347] Updated weights for policy 0, policy_version 1445082 (0.0007) [2023-12-27 15:39:34,561][20347] Updated weights for policy 0, policy_version 1445092 (0.0008) [2023-12-27 15:39:34,674][20381] Updated weights for policy 1, policy_version 1443704 (0.0007) [2023-12-27 15:39:34,734][20381] Updated weights for policy 1, policy_version 1443714 (0.0009) [2023-12-27 15:39:34,789][20381] Updated weights for policy 1, policy_version 1443724 (0.0010) [2023-12-27 15:39:35,177][20347] Updated weights for policy 0, policy_version 1445102 (0.0010) [2023-12-27 15:39:35,226][20347] Updated weights for policy 0, policy_version 1445112 (0.0009) [2023-12-27 15:39:35,280][20347] Updated weights for policy 0, policy_version 1445122 (0.0008) [2023-12-27 15:39:35,425][20381] Updated weights for policy 1, policy_version 1443734 (0.0007) [2023-12-27 15:39:35,469][20381] Updated weights for policy 1, policy_version 1443744 (0.0008) [2023-12-27 15:39:35,515][20381] Updated weights for policy 1, policy_version 1443754 (0.0007) [2023-12-27 15:39:35,977][20347] Updated weights for policy 0, policy_version 1445132 (0.0006) [2023-12-27 15:39:36,027][20347] Updated weights for policy 0, policy_version 1445142 (0.0005) [2023-12-27 15:39:36,081][20347] Updated weights for policy 0, policy_version 1445152 (0.0005) [2023-12-27 15:39:36,178][20381] Updated weights for policy 1, policy_version 1443764 (0.0008) [2023-12-27 15:39:36,240][20381] Updated weights for policy 1, policy_version 1443774 (0.0010) [2023-12-27 15:39:36,296][20381] Updated weights for policy 1, policy_version 1443784 (0.0010) [2023-12-27 15:39:36,648][20347] Updated weights for policy 0, policy_version 1445162 (0.0008) [2023-12-27 15:39:36,705][20347] Updated weights for policy 0, policy_version 1445172 (0.0006) [2023-12-27 15:39:36,707][20291] KL-divergence is very high: 104.1200 [2023-12-27 15:39:36,719][20291] KL-divergence is very high: 231.6475 [2023-12-27 15:39:36,764][20347] Updated weights for policy 0, policy_version 1445182 (0.0009) [2023-12-27 15:39:36,819][20347] Updated weights for policy 0, policy_version 1445192 (0.0010) [2023-12-27 15:39:37,008][20381] Updated weights for policy 1, policy_version 1443794 (0.0011) [2023-12-27 15:39:37,073][20381] Updated weights for policy 1, policy_version 1443804 (0.0010) [2023-12-27 15:39:37,144][20381] Updated weights for policy 1, policy_version 1443814 (0.0009) [2023-12-27 15:39:37,207][20381] Updated weights for policy 1, policy_version 1443824 (0.0006) [2023-12-27 15:39:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 739688448. Throughput: 0: 10453.9, 1: 10133.8. Samples: 739656224. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:37,363][19170] Avg episode reward: [(0, '770.580'), (1, '58.790')] [2023-12-27 15:39:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001445192_370040832.pth... [2023-12-27 15:39:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001443824_369647616.pth... [2023-12-27 15:39:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001442640_369344512.pth [2023-12-27 15:39:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001443976_369729536.pth [2023-12-27 15:39:37,454][20347] Updated weights for policy 0, policy_version 1445202 (0.0007) [2023-12-27 15:39:37,517][20347] Updated weights for policy 0, policy_version 1445212 (0.0007) [2023-12-27 15:39:37,577][20347] Updated weights for policy 0, policy_version 1445222 (0.0005) [2023-12-27 15:39:37,803][20381] Updated weights for policy 1, policy_version 1443834 (0.0008) [2023-12-27 15:39:37,855][20381] Updated weights for policy 1, policy_version 1443844 (0.0006) [2023-12-27 15:39:37,908][20381] Updated weights for policy 1, policy_version 1443854 (0.0008) [2023-12-27 15:39:38,171][20347] Updated weights for policy 0, policy_version 1445232 (0.0005) [2023-12-27 15:39:38,235][20347] Updated weights for policy 0, policy_version 1445242 (0.0005) [2023-12-27 15:39:38,295][20347] Updated weights for policy 0, policy_version 1445252 (0.0005) [2023-12-27 15:39:38,604][20381] Updated weights for policy 1, policy_version 1443864 (0.0007) [2023-12-27 15:39:38,662][20381] Updated weights for policy 1, policy_version 1443874 (0.0008) [2023-12-27 15:39:38,720][20381] Updated weights for policy 1, policy_version 1443884 (0.0005) [2023-12-27 15:39:38,992][20347] Updated weights for policy 0, policy_version 1445262 (0.0008) [2023-12-27 15:39:39,057][20347] Updated weights for policy 0, policy_version 1445272 (0.0007) [2023-12-27 15:39:39,116][20347] Updated weights for policy 0, policy_version 1445282 (0.0008) [2023-12-27 15:39:39,381][20381] Updated weights for policy 1, policy_version 1443894 (0.0007) [2023-12-27 15:39:39,446][20381] Updated weights for policy 1, policy_version 1443904 (0.0008) [2023-12-27 15:39:39,508][20381] Updated weights for policy 1, policy_version 1443914 (0.0008) [2023-12-27 15:39:39,833][20347] Updated weights for policy 0, policy_version 1445292 (0.0007) [2023-12-27 15:39:39,901][20347] Updated weights for policy 0, policy_version 1445302 (0.0007) [2023-12-27 15:39:39,972][20347] Updated weights for policy 0, policy_version 1445312 (0.0008) [2023-12-27 15:39:40,185][20381] Updated weights for policy 1, policy_version 1443924 (0.0007) [2023-12-27 15:39:40,245][20381] Updated weights for policy 1, policy_version 1443934 (0.0008) [2023-12-27 15:39:40,301][20381] Updated weights for policy 1, policy_version 1443944 (0.0006) [2023-12-27 15:39:40,709][20347] Updated weights for policy 0, policy_version 1445322 (0.0008) [2023-12-27 15:39:40,767][20347] Updated weights for policy 0, policy_version 1445332 (0.0008) [2023-12-27 15:39:40,829][20347] Updated weights for policy 0, policy_version 1445342 (0.0009) [2023-12-27 15:39:40,876][20381] Updated weights for policy 1, policy_version 1443954 (0.0007) [2023-12-27 15:39:40,879][20347] Updated weights for policy 0, policy_version 1445352 (0.0007) [2023-12-27 15:39:40,931][20381] Updated weights for policy 1, policy_version 1443964 (0.0010) [2023-12-27 15:39:40,990][20381] Updated weights for policy 1, policy_version 1443974 (0.0010) [2023-12-27 15:39:41,054][20381] Updated weights for policy 1, policy_version 1443984 (0.0007) [2023-12-27 15:39:41,665][20291] KL-divergence is very high: 180.5242 [2023-12-27 15:39:41,680][20291] KL-divergence is very high: 164.7164 [2023-12-27 15:39:41,688][20347] Updated weights for policy 0, policy_version 1445362 (0.0008) [2023-12-27 15:39:41,762][20347] Updated weights for policy 0, policy_version 1445372 (0.0009) [2023-12-27 15:39:41,802][20381] Updated weights for policy 1, policy_version 1443994 (0.0011) [2023-12-27 15:39:41,821][20347] Updated weights for policy 0, policy_version 1445382 (0.0007) [2023-12-27 15:39:41,859][20381] Updated weights for policy 1, policy_version 1444004 (0.0011) [2023-12-27 15:39:41,910][20381] Updated weights for policy 1, policy_version 1444014 (0.0010) [2023-12-27 15:39:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 739786752. Throughput: 0: 10392.1, 1: 10177.6. Samples: 739780196. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:42,363][19170] Avg episode reward: [(0, '721.790'), (1, '64.870')] [2023-12-27 15:39:42,567][20347] Updated weights for policy 0, policy_version 1445392 (0.0008) [2023-12-27 15:39:42,614][20347] Updated weights for policy 0, policy_version 1445402 (0.0007) [2023-12-27 15:39:42,617][20381] Updated weights for policy 1, policy_version 1444024 (0.0006) [2023-12-27 15:39:42,673][20347] Updated weights for policy 0, policy_version 1445412 (0.0006) [2023-12-27 15:39:42,675][20381] Updated weights for policy 1, policy_version 1444034 (0.0006) [2023-12-27 15:39:42,732][20381] Updated weights for policy 1, policy_version 1444044 (0.0007) [2023-12-27 15:39:43,414][20381] Updated weights for policy 1, policy_version 1444054 (0.0007) [2023-12-27 15:39:43,421][20347] Updated weights for policy 0, policy_version 1445422 (0.0006) [2023-12-27 15:39:43,459][20381] Updated weights for policy 1, policy_version 1444064 (0.0006) [2023-12-27 15:39:43,472][20347] Updated weights for policy 0, policy_version 1445432 (0.0005) [2023-12-27 15:39:43,520][20381] Updated weights for policy 1, policy_version 1444074 (0.0006) [2023-12-27 15:39:43,530][20347] Updated weights for policy 0, policy_version 1445442 (0.0006) [2023-12-27 15:39:44,142][20347] Updated weights for policy 0, policy_version 1445452 (0.0006) [2023-12-27 15:39:44,192][20347] Updated weights for policy 0, policy_version 1445462 (0.0007) [2023-12-27 15:39:44,215][20381] Updated weights for policy 1, policy_version 1444084 (0.0006) [2023-12-27 15:39:44,239][20347] Updated weights for policy 0, policy_version 1445472 (0.0005) [2023-12-27 15:39:44,271][20381] Updated weights for policy 1, policy_version 1444094 (0.0006) [2023-12-27 15:39:44,329][20381] Updated weights for policy 1, policy_version 1444104 (0.0007) [2023-12-27 15:39:44,907][20347] Updated weights for policy 0, policy_version 1445482 (0.0006) [2023-12-27 15:39:44,969][20347] Updated weights for policy 0, policy_version 1445492 (0.0008) [2023-12-27 15:39:45,023][20381] Updated weights for policy 1, policy_version 1444114 (0.0006) [2023-12-27 15:39:45,026][20347] Updated weights for policy 0, policy_version 1445502 (0.0007) [2023-12-27 15:39:45,079][20381] Updated weights for policy 1, policy_version 1444124 (0.0006) [2023-12-27 15:39:45,085][20347] Updated weights for policy 0, policy_version 1445512 (0.0006) [2023-12-27 15:39:45,136][20381] Updated weights for policy 1, policy_version 1444134 (0.0009) [2023-12-27 15:39:45,195][20381] Updated weights for policy 1, policy_version 1444144 (0.0008) [2023-12-27 15:39:45,742][20347] Updated weights for policy 0, policy_version 1445522 (0.0006) [2023-12-27 15:39:45,801][20347] Updated weights for policy 0, policy_version 1445532 (0.0008) [2023-12-27 15:39:45,849][20347] Updated weights for policy 0, policy_version 1445542 (0.0007) [2023-12-27 15:39:45,884][20381] Updated weights for policy 1, policy_version 1444154 (0.0008) [2023-12-27 15:39:45,929][20381] Updated weights for policy 1, policy_version 1444164 (0.0005) [2023-12-27 15:39:45,984][20381] Updated weights for policy 1, policy_version 1444174 (0.0005) [2023-12-27 15:39:46,539][20347] Updated weights for policy 0, policy_version 1445552 (0.0007) [2023-12-27 15:39:46,598][20347] Updated weights for policy 0, policy_version 1445562 (0.0008) [2023-12-27 15:39:46,607][20381] Updated weights for policy 1, policy_version 1444184 (0.0006) [2023-12-27 15:39:46,654][20381] Updated weights for policy 1, policy_version 1444194 (0.0005) [2023-12-27 15:39:46,658][20347] Updated weights for policy 0, policy_version 1445572 (0.0008) [2023-12-27 15:39:46,712][20381] Updated weights for policy 1, policy_version 1444204 (0.0006) [2023-12-27 15:39:47,301][20381] Updated weights for policy 1, policy_version 1444214 (0.0009) [2023-12-27 15:39:47,356][20381] Updated weights for policy 1, policy_version 1444224 (0.0010) [2023-12-27 15:39:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 739885056. Throughput: 0: 10347.2, 1: 10249.8. Samples: 739901720. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:47,363][19170] Avg episode reward: [(0, '745.190'), (1, '63.370')] [2023-12-27 15:39:47,380][20347] Updated weights for policy 0, policy_version 1445582 (0.0006) [2023-12-27 15:39:47,414][20381] Updated weights for policy 1, policy_version 1444234 (0.0010) [2023-12-27 15:39:47,428][20347] Updated weights for policy 0, policy_version 1445592 (0.0006) [2023-12-27 15:39:47,481][20347] Updated weights for policy 0, policy_version 1445602 (0.0007) [2023-12-27 15:39:48,093][20381] Updated weights for policy 1, policy_version 1444244 (0.0008) [2023-12-27 15:39:48,139][20381] Updated weights for policy 1, policy_version 1444254 (0.0006) [2023-12-27 15:39:48,181][20347] Updated weights for policy 0, policy_version 1445612 (0.0007) [2023-12-27 15:39:48,196][20381] Updated weights for policy 1, policy_version 1444264 (0.0005) [2023-12-27 15:39:48,233][20347] Updated weights for policy 0, policy_version 1445622 (0.0007) [2023-12-27 15:39:48,281][20347] Updated weights for policy 0, policy_version 1445632 (0.0009) [2023-12-27 15:39:48,886][20381] Updated weights for policy 1, policy_version 1444274 (0.0007) [2023-12-27 15:39:48,931][20381] Updated weights for policy 1, policy_version 1444284 (0.0007) [2023-12-27 15:39:48,981][20381] Updated weights for policy 1, policy_version 1444294 (0.0007) [2023-12-27 15:39:48,995][20347] Updated weights for policy 0, policy_version 1445642 (0.0008) [2023-12-27 15:39:49,046][20381] Updated weights for policy 1, policy_version 1444304 (0.0007) [2023-12-27 15:39:49,048][20347] Updated weights for policy 0, policy_version 1445652 (0.0006) [2023-12-27 15:39:49,101][20347] Updated weights for policy 0, policy_version 1445662 (0.0005) [2023-12-27 15:39:49,158][20347] Updated weights for policy 0, policy_version 1445672 (0.0007) [2023-12-27 15:39:49,814][20347] Updated weights for policy 0, policy_version 1445682 (0.0007) [2023-12-27 15:39:49,820][20381] Updated weights for policy 1, policy_version 1444314 (0.0008) [2023-12-27 15:39:49,882][20347] Updated weights for policy 0, policy_version 1445692 (0.0008) [2023-12-27 15:39:49,887][20381] Updated weights for policy 1, policy_version 1444324 (0.0010) [2023-12-27 15:39:49,949][20347] Updated weights for policy 0, policy_version 1445702 (0.0007) [2023-12-27 15:39:49,956][20381] Updated weights for policy 1, policy_version 1444334 (0.0009) [2023-12-27 15:39:50,618][20381] Updated weights for policy 1, policy_version 1444344 (0.0009) [2023-12-27 15:39:50,678][20381] Updated weights for policy 1, policy_version 1444354 (0.0010) [2023-12-27 15:39:50,700][20347] Updated weights for policy 0, policy_version 1445712 (0.0008) [2023-12-27 15:39:50,731][20381] Updated weights for policy 1, policy_version 1444364 (0.0010) [2023-12-27 15:39:50,767][20347] Updated weights for policy 0, policy_version 1445722 (0.0008) [2023-12-27 15:39:50,830][20347] Updated weights for policy 0, policy_version 1445732 (0.0008) [2023-12-27 15:39:51,473][20381] Updated weights for policy 1, policy_version 1444374 (0.0006) [2023-12-27 15:39:51,486][20347] Updated weights for policy 0, policy_version 1445742 (0.0007) [2023-12-27 15:39:51,527][20381] Updated weights for policy 1, policy_version 1444384 (0.0007) [2023-12-27 15:39:51,549][20347] Updated weights for policy 0, policy_version 1445752 (0.0007) [2023-12-27 15:39:51,583][20381] Updated weights for policy 1, policy_version 1444394 (0.0007) [2023-12-27 15:39:51,615][20347] Updated weights for policy 0, policy_version 1445762 (0.0007) [2023-12-27 15:39:52,292][20381] Updated weights for policy 1, policy_version 1444404 (0.0009) [2023-12-27 15:39:52,319][20347] Updated weights for policy 0, policy_version 1445772 (0.0007) [2023-12-27 15:39:52,360][20381] Updated weights for policy 1, policy_version 1444414 (0.0008) [2023-12-27 15:39:52,374][19170] Fps is (10 sec: 19638.2, 60 sec: 20339.5, 300 sec: 20548.6). Total num frames: 739983360. Throughput: 0: 10316.2, 1: 10204.0. Samples: 739961172. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:52,375][19170] Avg episode reward: [(0, '676.950'), (1, '66.010')] [2023-12-27 15:39:52,386][20347] Updated weights for policy 0, policy_version 1445782 (0.0008) [2023-12-27 15:39:52,422][20381] Updated weights for policy 1, policy_version 1444424 (0.0008) [2023-12-27 15:39:52,451][20347] Updated weights for policy 0, policy_version 1445792 (0.0007) [2023-12-27 15:39:52,466][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001444432_369803264.pth... [2023-12-27 15:39:52,470][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001443216_369491968.pth [2023-12-27 15:39:52,470][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001444432_369803264.pth [2023-12-27 15:39:52,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001445800_370196480.pth... [2023-12-27 15:39:52,524][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001444584_369885184.pth [2023-12-27 15:39:52,525][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001445800_370196480.pth [2023-12-27 15:39:53,020][20381] Updated weights for policy 1, policy_version 1444434 (0.0007) [2023-12-27 15:39:53,079][20381] Updated weights for policy 1, policy_version 1444444 (0.0005) [2023-12-27 15:39:53,140][20347] Updated weights for policy 0, policy_version 1445802 (0.0005) [2023-12-27 15:39:53,141][20381] Updated weights for policy 1, policy_version 1444454 (0.0006) [2023-12-27 15:39:53,194][20347] Updated weights for policy 0, policy_version 1445812 (0.0005) [2023-12-27 15:39:53,196][20381] Updated weights for policy 1, policy_version 1444464 (0.0006) [2023-12-27 15:39:53,249][20347] Updated weights for policy 0, policy_version 1445822 (0.0005) [2023-12-27 15:39:53,305][20347] Updated weights for policy 0, policy_version 1445832 (0.0007) [2023-12-27 15:39:53,836][20381] Updated weights for policy 1, policy_version 1444474 (0.0007) [2023-12-27 15:39:53,875][20347] Updated weights for policy 0, policy_version 1445842 (0.0007) [2023-12-27 15:39:53,886][20381] Updated weights for policy 1, policy_version 1444484 (0.0005) [2023-12-27 15:39:53,933][20347] Updated weights for policy 0, policy_version 1445852 (0.0007) [2023-12-27 15:39:53,939][20381] Updated weights for policy 1, policy_version 1444494 (0.0010) [2023-12-27 15:39:53,983][20347] Updated weights for policy 0, policy_version 1445862 (0.0007) [2023-12-27 15:39:54,579][20381] Updated weights for policy 1, policy_version 1444504 (0.0010) [2023-12-27 15:39:54,635][20381] Updated weights for policy 1, policy_version 1444514 (0.0010) [2023-12-27 15:39:54,673][20347] Updated weights for policy 0, policy_version 1445872 (0.0006) [2023-12-27 15:39:54,704][20381] Updated weights for policy 1, policy_version 1444524 (0.0011) [2023-12-27 15:39:54,728][20347] Updated weights for policy 0, policy_version 1445882 (0.0005) [2023-12-27 15:39:54,782][20347] Updated weights for policy 0, policy_version 1445892 (0.0009) [2023-12-27 15:39:55,277][20381] Updated weights for policy 1, policy_version 1444534 (0.0010) [2023-12-27 15:39:55,322][20381] Updated weights for policy 1, policy_version 1444544 (0.0010) [2023-12-27 15:39:55,369][20381] Updated weights for policy 1, policy_version 1444554 (0.0010) [2023-12-27 15:39:55,452][20347] Updated weights for policy 0, policy_version 1445902 (0.0010) [2023-12-27 15:39:55,508][20347] Updated weights for policy 0, policy_version 1445912 (0.0009) [2023-12-27 15:39:55,560][20347] Updated weights for policy 0, policy_version 1445922 (0.0007) [2023-12-27 15:39:56,084][20381] Updated weights for policy 1, policy_version 1444564 (0.0011) [2023-12-27 15:39:56,130][20347] Updated weights for policy 0, policy_version 1445932 (0.0005) [2023-12-27 15:39:56,139][20381] Updated weights for policy 1, policy_version 1444574 (0.0011) [2023-12-27 15:39:56,185][20347] Updated weights for policy 0, policy_version 1445942 (0.0008) [2023-12-27 15:39:56,199][20381] Updated weights for policy 1, policy_version 1444584 (0.0011) [2023-12-27 15:39:56,239][20347] Updated weights for policy 0, policy_version 1445952 (0.0008) [2023-12-27 15:39:56,849][20381] Updated weights for policy 1, policy_version 1444594 (0.0010) [2023-12-27 15:39:56,905][20347] Updated weights for policy 0, policy_version 1445962 (0.0007) [2023-12-27 15:39:56,907][20381] Updated weights for policy 1, policy_version 1444604 (0.0008) [2023-12-27 15:39:56,958][20347] Updated weights for policy 0, policy_version 1445972 (0.0010) [2023-12-27 15:39:56,963][20381] Updated weights for policy 1, policy_version 1444614 (0.0007) [2023-12-27 15:39:57,014][20347] Updated weights for policy 0, policy_version 1445982 (0.0010) [2023-12-27 15:39:57,019][20381] Updated weights for policy 1, policy_version 1444624 (0.0006) [2023-12-27 15:39:57,069][20347] Updated weights for policy 0, policy_version 1445992 (0.0010) [2023-12-27 15:39:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 740098048. Throughput: 0: 10325.7, 1: 10275.5. Samples: 740086316. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:39:57,363][19170] Avg episode reward: [(0, '677.410'), (1, '74.430')] [2023-12-27 15:39:57,697][20347] Updated weights for policy 0, policy_version 1446002 (0.0010) [2023-12-27 15:39:57,736][20381] Updated weights for policy 1, policy_version 1444634 (0.0005) [2023-12-27 15:39:57,759][20347] Updated weights for policy 0, policy_version 1446012 (0.0010) [2023-12-27 15:39:57,792][20381] Updated weights for policy 1, policy_version 1444644 (0.0005) [2023-12-27 15:39:57,816][20347] Updated weights for policy 0, policy_version 1446022 (0.0010) [2023-12-27 15:39:57,851][20381] Updated weights for policy 1, policy_version 1444654 (0.0006) [2023-12-27 15:39:58,528][20347] Updated weights for policy 0, policy_version 1446032 (0.0008) [2023-12-27 15:39:58,546][20381] Updated weights for policy 1, policy_version 1444664 (0.0007) [2023-12-27 15:39:58,593][20347] Updated weights for policy 0, policy_version 1446042 (0.0008) [2023-12-27 15:39:58,615][20381] Updated weights for policy 1, policy_version 1444674 (0.0008) [2023-12-27 15:39:58,659][20347] Updated weights for policy 0, policy_version 1446052 (0.0008) [2023-12-27 15:39:58,678][20381] Updated weights for policy 1, policy_version 1444684 (0.0008) [2023-12-27 15:39:59,314][20347] Updated weights for policy 0, policy_version 1446062 (0.0008) [2023-12-27 15:39:59,376][20381] Updated weights for policy 1, policy_version 1444694 (0.0009) [2023-12-27 15:39:59,385][20347] Updated weights for policy 0, policy_version 1446072 (0.0008) [2023-12-27 15:39:59,426][20381] Updated weights for policy 1, policy_version 1444704 (0.0007) [2023-12-27 15:39:59,439][20347] Updated weights for policy 0, policy_version 1446082 (0.0008) [2023-12-27 15:39:59,479][20381] Updated weights for policy 1, policy_version 1444714 (0.0006) [2023-12-27 15:40:00,150][20381] Updated weights for policy 1, policy_version 1444724 (0.0006) [2023-12-27 15:40:00,177][20347] Updated weights for policy 0, policy_version 1446092 (0.0008) [2023-12-27 15:40:00,206][20381] Updated weights for policy 1, policy_version 1444734 (0.0006) [2023-12-27 15:40:00,236][20347] Updated weights for policy 0, policy_version 1446102 (0.0007) [2023-12-27 15:40:00,267][20381] Updated weights for policy 1, policy_version 1444744 (0.0009) [2023-12-27 15:40:00,284][20347] Updated weights for policy 0, policy_version 1446112 (0.0007) [2023-12-27 15:40:00,931][20347] Updated weights for policy 0, policy_version 1446122 (0.0006) [2023-12-27 15:40:00,943][20381] Updated weights for policy 1, policy_version 1444754 (0.0011) [2023-12-27 15:40:00,955][20291] KL-divergence is very high: 160.1452 [2023-12-27 15:40:00,965][20291] KL-divergence is very high: 178.7920 [2023-12-27 15:40:00,970][20291] KL-divergence is very high: 169.3481 [2023-12-27 15:40:00,982][20347] Updated weights for policy 0, policy_version 1446132 (0.0005) [2023-12-27 15:40:00,989][20381] Updated weights for policy 1, policy_version 1444764 (0.0010) [2023-12-27 15:40:00,993][20291] KL-divergence is very high: 156.2053 [2023-12-27 15:40:01,002][20291] KL-divergence is very high: 135.7038 [2023-12-27 15:40:01,008][20291] KL-divergence is very high: 111.7433 [2023-12-27 15:40:01,035][20347] Updated weights for policy 0, policy_version 1446142 (0.0007) [2023-12-27 15:40:01,039][20381] Updated weights for policy 1, policy_version 1444774 (0.0009) [2023-12-27 15:40:01,084][20347] Updated weights for policy 0, policy_version 1446152 (0.0008) [2023-12-27 15:40:01,100][20381] Updated weights for policy 1, policy_version 1444784 (0.0010) [2023-12-27 15:40:01,819][20347] Updated weights for policy 0, policy_version 1446162 (0.0008) [2023-12-27 15:40:01,859][20381] Updated weights for policy 1, policy_version 1444794 (0.0007) [2023-12-27 15:40:01,879][20347] Updated weights for policy 0, policy_version 1446172 (0.0007) [2023-12-27 15:40:01,922][20381] Updated weights for policy 1, policy_version 1444804 (0.0007) [2023-12-27 15:40:01,940][20347] Updated weights for policy 0, policy_version 1446182 (0.0007) [2023-12-27 15:40:01,986][20381] Updated weights for policy 1, policy_version 1444814 (0.0006) [2023-12-27 15:40:02,362][19170] Fps is (10 sec: 21323.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 740196352. Throughput: 0: 10274.9, 1: 10245.2. Samples: 740206604. Policy #0 lag: (min: 26.0, avg: 32.6, max: 58.0) [2023-12-27 15:40:02,363][19170] Avg episode reward: [(0, '717.010'), (1, '87.630')] [2023-12-27 15:40:02,602][20347] Updated weights for policy 0, policy_version 1446192 (0.0007) [2023-12-27 15:40:02,664][20347] Updated weights for policy 0, policy_version 1446202 (0.0005) [2023-12-27 15:40:02,707][20381] Updated weights for policy 1, policy_version 1444824 (0.0007) [2023-12-27 15:40:02,728][20347] Updated weights for policy 0, policy_version 1446212 (0.0006) [2023-12-27 15:40:02,768][20381] Updated weights for policy 1, policy_version 1444834 (0.0007) [2023-12-27 15:40:02,826][20381] Updated weights for policy 1, policy_version 1444844 (0.0007) [2023-12-27 15:40:03,378][20347] Updated weights for policy 0, policy_version 1446222 (0.0005) [2023-12-27 15:40:03,424][20347] Updated weights for policy 0, policy_version 1446232 (0.0006) [2023-12-27 15:40:03,479][20347] Updated weights for policy 0, policy_version 1446242 (0.0006) [2023-12-27 15:40:03,501][20381] Updated weights for policy 1, policy_version 1444854 (0.0007) [2023-12-27 15:40:03,552][20381] Updated weights for policy 1, policy_version 1444864 (0.0007) [2023-12-27 15:40:03,604][20381] Updated weights for policy 1, policy_version 1444874 (0.0007) [2023-12-27 15:40:04,136][20347] Updated weights for policy 0, policy_version 1446252 (0.0006) [2023-12-27 15:40:04,190][20347] Updated weights for policy 0, policy_version 1446262 (0.0007) [2023-12-27 15:40:04,251][20347] Updated weights for policy 0, policy_version 1446272 (0.0008) [2023-12-27 15:40:04,318][20381] Updated weights for policy 1, policy_version 1444884 (0.0007) [2023-12-27 15:40:04,381][20381] Updated weights for policy 1, policy_version 1444894 (0.0008) [2023-12-27 15:40:04,440][20381] Updated weights for policy 1, policy_version 1444904 (0.0008) [2023-12-27 15:40:04,982][20347] Updated weights for policy 0, policy_version 1446282 (0.0007) [2023-12-27 15:40:05,035][20347] Updated weights for policy 0, policy_version 1446292 (0.0007) [2023-12-27 15:40:05,041][20381] Updated weights for policy 1, policy_version 1444914 (0.0008) [2023-12-27 15:40:05,094][20347] Updated weights for policy 0, policy_version 1446302 (0.0005) [2023-12-27 15:40:05,103][20381] Updated weights for policy 1, policy_version 1444924 (0.0010) [2023-12-27 15:40:05,149][20347] Updated weights for policy 0, policy_version 1446312 (0.0005) [2023-12-27 15:40:05,158][20381] Updated weights for policy 1, policy_version 1444934 (0.0010) [2023-12-27 15:40:05,206][20381] Updated weights for policy 1, policy_version 1444944 (0.0010) [2023-12-27 15:40:05,795][20347] Updated weights for policy 0, policy_version 1446322 (0.0005) [2023-12-27 15:40:05,850][20347] Updated weights for policy 0, policy_version 1446332 (0.0010) [2023-12-27 15:40:05,904][20347] Updated weights for policy 0, policy_version 1446342 (0.0010) [2023-12-27 15:40:05,905][20381] Updated weights for policy 1, policy_version 1444954 (0.0010) [2023-12-27 15:40:05,957][20381] Updated weights for policy 1, policy_version 1444964 (0.0005) [2023-12-27 15:40:06,014][20381] Updated weights for policy 1, policy_version 1444974 (0.0010) [2023-12-27 15:40:06,568][20347] Updated weights for policy 0, policy_version 1446352 (0.0011) [2023-12-27 15:40:06,621][20347] Updated weights for policy 0, policy_version 1446362 (0.0011) [2023-12-27 15:40:06,678][20347] Updated weights for policy 0, policy_version 1446372 (0.0011) [2023-12-27 15:40:06,695][20381] Updated weights for policy 1, policy_version 1444984 (0.0010) [2023-12-27 15:40:06,753][20381] Updated weights for policy 1, policy_version 1444994 (0.0011) [2023-12-27 15:40:06,811][20381] Updated weights for policy 1, policy_version 1445004 (0.0010) [2023-12-27 15:40:07,277][20347] Updated weights for policy 0, policy_version 1446382 (0.0007) [2023-12-27 15:40:07,333][20347] Updated weights for policy 0, policy_version 1446392 (0.0006) [2023-12-27 15:40:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 740294656. Throughput: 0: 10252.1, 1: 10257.7. Samples: 740268080. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:07,363][19170] Avg episode reward: [(0, '677.550'), (1, '128.950')] [2023-12-27 15:40:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001445008_369950720.pth... [2023-12-27 15:40:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001443824_369647616.pth [2023-12-27 15:40:07,395][20347] Updated weights for policy 0, policy_version 1446402 (0.0005) [2023-12-27 15:40:07,429][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001446408_370352128.pth... [2023-12-27 15:40:07,433][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001445192_370040832.pth [2023-12-27 15:40:07,445][20381] Updated weights for policy 1, policy_version 1445014 (0.0007) [2023-12-27 15:40:07,503][20381] Updated weights for policy 1, policy_version 1445024 (0.0008) [2023-12-27 15:40:07,568][20381] Updated weights for policy 1, policy_version 1445034 (0.0011) [2023-12-27 15:40:08,040][20347] Updated weights for policy 0, policy_version 1446412 (0.0008) [2023-12-27 15:40:08,104][20347] Updated weights for policy 0, policy_version 1446422 (0.0009) [2023-12-27 15:40:08,161][20347] Updated weights for policy 0, policy_version 1446432 (0.0010) [2023-12-27 15:40:08,165][20381] Updated weights for policy 1, policy_version 1445044 (0.0011) [2023-12-27 15:40:08,221][20381] Updated weights for policy 1, policy_version 1445054 (0.0010) [2023-12-27 15:40:08,265][20381] Updated weights for policy 1, policy_version 1445064 (0.0009) [2023-12-27 15:40:08,810][20347] Updated weights for policy 0, policy_version 1446442 (0.0007) [2023-12-27 15:40:08,871][20347] Updated weights for policy 0, policy_version 1446452 (0.0010) [2023-12-27 15:40:08,886][20381] Updated weights for policy 1, policy_version 1445074 (0.0005) [2023-12-27 15:40:08,926][20347] Updated weights for policy 0, policy_version 1446462 (0.0010) [2023-12-27 15:40:08,931][20381] Updated weights for policy 1, policy_version 1445084 (0.0005) [2023-12-27 15:40:08,977][20347] Updated weights for policy 0, policy_version 1446472 (0.0010) [2023-12-27 15:40:08,982][20381] Updated weights for policy 1, policy_version 1445094 (0.0009) [2023-12-27 15:40:09,037][20381] Updated weights for policy 1, policy_version 1445104 (0.0011) [2023-12-27 15:40:09,651][20347] Updated weights for policy 0, policy_version 1446482 (0.0008) [2023-12-27 15:40:09,715][20347] Updated weights for policy 0, policy_version 1446492 (0.0011) [2023-12-27 15:40:09,783][20347] Updated weights for policy 0, policy_version 1446502 (0.0008) [2023-12-27 15:40:09,796][20381] Updated weights for policy 1, policy_version 1445114 (0.0007) [2023-12-27 15:40:09,873][20381] Updated weights for policy 1, policy_version 1445124 (0.0006) [2023-12-27 15:40:09,941][20381] Updated weights for policy 1, policy_version 1445134 (0.0008) [2023-12-27 15:40:10,449][20347] Updated weights for policy 0, policy_version 1446512 (0.0006) [2023-12-27 15:40:10,502][20347] Updated weights for policy 0, policy_version 1446522 (0.0008) [2023-12-27 15:40:10,553][20347] Updated weights for policy 0, policy_version 1446532 (0.0007) [2023-12-27 15:40:10,613][20381] Updated weights for policy 1, policy_version 1445144 (0.0005) [2023-12-27 15:40:10,667][20381] Updated weights for policy 1, policy_version 1445154 (0.0005) [2023-12-27 15:40:10,718][20381] Updated weights for policy 1, policy_version 1445164 (0.0005) [2023-12-27 15:40:11,311][20347] Updated weights for policy 0, policy_version 1446542 (0.0006) [2023-12-27 15:40:11,335][20381] Updated weights for policy 1, policy_version 1445174 (0.0008) [2023-12-27 15:40:11,396][20347] Updated weights for policy 0, policy_version 1446552 (0.0007) [2023-12-27 15:40:11,408][20381] Updated weights for policy 1, policy_version 1445184 (0.0009) [2023-12-27 15:40:11,447][20347] Updated weights for policy 0, policy_version 1446562 (0.0006) [2023-12-27 15:40:11,461][20381] Updated weights for policy 1, policy_version 1445194 (0.0011) [2023-12-27 15:40:12,152][20381] Updated weights for policy 1, policy_version 1445204 (0.0011) [2023-12-27 15:40:12,188][20347] Updated weights for policy 0, policy_version 1446572 (0.0006) [2023-12-27 15:40:12,210][20381] Updated weights for policy 1, policy_version 1445214 (0.0010) [2023-12-27 15:40:12,242][20347] Updated weights for policy 0, policy_version 1446582 (0.0008) [2023-12-27 15:40:12,265][20381] Updated weights for policy 1, policy_version 1445224 (0.0011) [2023-12-27 15:40:12,302][20347] Updated weights for policy 0, policy_version 1446592 (0.0008) [2023-12-27 15:40:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 740409344. Throughput: 0: 10303.7, 1: 10278.5. Samples: 740393912. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:12,363][19170] Avg episode reward: [(0, '657.830'), (1, '204.880')] [2023-12-27 15:40:12,988][20381] Updated weights for policy 1, policy_version 1445234 (0.0009) [2023-12-27 15:40:13,039][20347] Updated weights for policy 0, policy_version 1446602 (0.0008) [2023-12-27 15:40:13,049][20381] Updated weights for policy 1, policy_version 1445244 (0.0006) [2023-12-27 15:40:13,102][20347] Updated weights for policy 0, policy_version 1446612 (0.0007) [2023-12-27 15:40:13,112][20381] Updated weights for policy 1, policy_version 1445254 (0.0010) [2023-12-27 15:40:13,161][20347] Updated weights for policy 0, policy_version 1446622 (0.0006) [2023-12-27 15:40:13,176][20381] Updated weights for policy 1, policy_version 1445264 (0.0009) [2023-12-27 15:40:13,219][20347] Updated weights for policy 0, policy_version 1446632 (0.0006) [2023-12-27 15:40:13,785][20347] Updated weights for policy 0, policy_version 1446642 (0.0007) [2023-12-27 15:40:13,818][20381] Updated weights for policy 1, policy_version 1445274 (0.0011) [2023-12-27 15:40:13,841][20347] Updated weights for policy 0, policy_version 1446652 (0.0005) [2023-12-27 15:40:13,876][20381] Updated weights for policy 1, policy_version 1445284 (0.0010) [2023-12-27 15:40:13,888][20347] Updated weights for policy 0, policy_version 1446662 (0.0006) [2023-12-27 15:40:13,930][20381] Updated weights for policy 1, policy_version 1445294 (0.0010) [2023-12-27 15:40:14,461][20347] Updated weights for policy 0, policy_version 1446672 (0.0008) [2023-12-27 15:40:14,516][20347] Updated weights for policy 0, policy_version 1446682 (0.0008) [2023-12-27 15:40:14,567][20381] Updated weights for policy 1, policy_version 1445304 (0.0006) [2023-12-27 15:40:14,570][20347] Updated weights for policy 0, policy_version 1446692 (0.0009) [2023-12-27 15:40:14,623][20381] Updated weights for policy 1, policy_version 1445314 (0.0005) [2023-12-27 15:40:14,686][20381] Updated weights for policy 1, policy_version 1445324 (0.0005) [2023-12-27 15:40:15,267][20381] Updated weights for policy 1, policy_version 1445334 (0.0006) [2023-12-27 15:40:15,275][20347] Updated weights for policy 0, policy_version 1446702 (0.0008) [2023-12-27 15:40:15,320][20381] Updated weights for policy 1, policy_version 1445344 (0.0006) [2023-12-27 15:40:15,332][20347] Updated weights for policy 0, policy_version 1446712 (0.0008) [2023-12-27 15:40:15,371][20381] Updated weights for policy 1, policy_version 1445354 (0.0005) [2023-12-27 15:40:15,389][20347] Updated weights for policy 0, policy_version 1446722 (0.0006) [2023-12-27 15:40:15,947][20381] Updated weights for policy 1, policy_version 1445364 (0.0006) [2023-12-27 15:40:15,999][20381] Updated weights for policy 1, policy_version 1445374 (0.0008) [2023-12-27 15:40:16,054][20381] Updated weights for policy 1, policy_version 1445384 (0.0008) [2023-12-27 15:40:16,113][20347] Updated weights for policy 0, policy_version 1446732 (0.0008) [2023-12-27 15:40:16,172][20347] Updated weights for policy 0, policy_version 1446742 (0.0007) [2023-12-27 15:40:16,230][20347] Updated weights for policy 0, policy_version 1446752 (0.0007) [2023-12-27 15:40:16,632][20381] Updated weights for policy 1, policy_version 1445394 (0.0006) [2023-12-27 15:40:16,696][20381] Updated weights for policy 1, policy_version 1445404 (0.0007) [2023-12-27 15:40:16,751][20381] Updated weights for policy 1, policy_version 1445414 (0.0006) [2023-12-27 15:40:16,816][20381] Updated weights for policy 1, policy_version 1445424 (0.0006) [2023-12-27 15:40:16,913][20347] Updated weights for policy 0, policy_version 1446762 (0.0006) [2023-12-27 15:40:16,964][20347] Updated weights for policy 0, policy_version 1446772 (0.0010) [2023-12-27 15:40:17,026][20347] Updated weights for policy 0, policy_version 1446782 (0.0010) [2023-12-27 15:40:17,076][20347] Updated weights for policy 0, policy_version 1446792 (0.0010) [2023-12-27 15:40:17,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 740507648. Throughput: 0: 10250.9, 1: 10315.7. Samples: 740519264. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:17,363][19170] Avg episode reward: [(0, '686.530'), (1, '272.050')] [2023-12-27 15:40:17,391][20381] Updated weights for policy 1, policy_version 1445434 (0.0005) [2023-12-27 15:40:17,439][20381] Updated weights for policy 1, policy_version 1445444 (0.0006) [2023-12-27 15:40:17,484][20381] Updated weights for policy 1, policy_version 1445454 (0.0008) [2023-12-27 15:40:17,622][20347] Updated weights for policy 0, policy_version 1446802 (0.0010) [2023-12-27 15:40:17,677][20347] Updated weights for policy 0, policy_version 1446812 (0.0011) [2023-12-27 15:40:17,735][20347] Updated weights for policy 0, policy_version 1446822 (0.0010) [2023-12-27 15:40:18,117][20381] Updated weights for policy 1, policy_version 1445464 (0.0009) [2023-12-27 15:40:18,167][20381] Updated weights for policy 1, policy_version 1445474 (0.0006) [2023-12-27 15:40:18,221][20381] Updated weights for policy 1, policy_version 1445484 (0.0007) [2023-12-27 15:40:18,365][20347] Updated weights for policy 0, policy_version 1446832 (0.0010) [2023-12-27 15:40:18,423][20347] Updated weights for policy 0, policy_version 1446842 (0.0010) [2023-12-27 15:40:18,482][20347] Updated weights for policy 0, policy_version 1446852 (0.0010) [2023-12-27 15:40:18,933][20381] Updated weights for policy 1, policy_version 1445494 (0.0007) [2023-12-27 15:40:18,984][20381] Updated weights for policy 1, policy_version 1445504 (0.0008) [2023-12-27 15:40:19,031][20381] Updated weights for policy 1, policy_version 1445514 (0.0007) [2023-12-27 15:40:19,090][20347] Updated weights for policy 0, policy_version 1446862 (0.0009) [2023-12-27 15:40:19,144][20347] Updated weights for policy 0, policy_version 1446872 (0.0007) [2023-12-27 15:40:19,194][20347] Updated weights for policy 0, policy_version 1446882 (0.0006) [2023-12-27 15:40:19,797][20381] Updated weights for policy 1, policy_version 1445524 (0.0006) [2023-12-27 15:40:19,855][20381] Updated weights for policy 1, policy_version 1445534 (0.0007) [2023-12-27 15:40:19,912][20347] Updated weights for policy 0, policy_version 1446892 (0.0007) [2023-12-27 15:40:19,915][20381] Updated weights for policy 1, policy_version 1445544 (0.0008) [2023-12-27 15:40:19,981][20347] Updated weights for policy 0, policy_version 1446902 (0.0009) [2023-12-27 15:40:20,044][20347] Updated weights for policy 0, policy_version 1446912 (0.0008) [2023-12-27 15:40:20,628][20381] Updated weights for policy 1, policy_version 1445554 (0.0007) [2023-12-27 15:40:20,682][20381] Updated weights for policy 1, policy_version 1445564 (0.0008) [2023-12-27 15:40:20,741][20381] Updated weights for policy 1, policy_version 1445574 (0.0008) [2023-12-27 15:40:20,758][20347] Updated weights for policy 0, policy_version 1446922 (0.0008) [2023-12-27 15:40:20,799][20381] Updated weights for policy 1, policy_version 1445584 (0.0009) [2023-12-27 15:40:20,821][20347] Updated weights for policy 0, policy_version 1446932 (0.0011) [2023-12-27 15:40:20,877][20347] Updated weights for policy 0, policy_version 1446942 (0.0010) [2023-12-27 15:40:20,939][20347] Updated weights for policy 0, policy_version 1446952 (0.0010) [2023-12-27 15:40:21,563][20381] Updated weights for policy 1, policy_version 1445594 (0.0008) [2023-12-27 15:40:21,630][20381] Updated weights for policy 1, policy_version 1445604 (0.0008) [2023-12-27 15:40:21,691][20381] Updated weights for policy 1, policy_version 1445614 (0.0008) [2023-12-27 15:40:21,708][20347] Updated weights for policy 0, policy_version 1446962 (0.0010) [2023-12-27 15:40:21,772][20347] Updated weights for policy 0, policy_version 1446972 (0.0008) [2023-12-27 15:40:21,826][20347] Updated weights for policy 0, policy_version 1446982 (0.0007) [2023-12-27 15:40:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 740605952. Throughput: 0: 10247.0, 1: 10342.2. Samples: 740582740. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:22,363][19170] Avg episode reward: [(0, '676.610'), (1, '293.660')] [2023-12-27 15:40:22,368][20381] Updated weights for policy 1, policy_version 1445624 (0.0009) [2023-12-27 15:40:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001446984_370499584.pth... [2023-12-27 15:40:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001445800_370196480.pth [2023-12-27 15:40:22,431][20381] Updated weights for policy 1, policy_version 1445634 (0.0011) [2023-12-27 15:40:22,491][20381] Updated weights for policy 1, policy_version 1445644 (0.0009) [2023-12-27 15:40:22,515][20347] Updated weights for policy 0, policy_version 1446992 (0.0010) [2023-12-27 15:40:22,516][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001445648_370114560.pth... [2023-12-27 15:40:22,520][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001444432_369803264.pth [2023-12-27 15:40:22,562][20347] Updated weights for policy 0, policy_version 1447002 (0.0010) [2023-12-27 15:40:22,616][20347] Updated weights for policy 0, policy_version 1447012 (0.0010) [2023-12-27 15:40:23,113][20381] Updated weights for policy 1, policy_version 1445654 (0.0006) [2023-12-27 15:40:23,159][20381] Updated weights for policy 1, policy_version 1445664 (0.0005) [2023-12-27 15:40:23,216][20381] Updated weights for policy 1, policy_version 1445674 (0.0007) [2023-12-27 15:40:23,271][20347] Updated weights for policy 0, policy_version 1447022 (0.0006) [2023-12-27 15:40:23,328][20347] Updated weights for policy 0, policy_version 1447032 (0.0007) [2023-12-27 15:40:23,376][20347] Updated weights for policy 0, policy_version 1447042 (0.0009) [2023-12-27 15:40:23,817][20381] Updated weights for policy 1, policy_version 1445684 (0.0007) [2023-12-27 15:40:23,875][20381] Updated weights for policy 1, policy_version 1445694 (0.0005) [2023-12-27 15:40:23,935][20381] Updated weights for policy 1, policy_version 1445704 (0.0005) [2023-12-27 15:40:24,113][20347] Updated weights for policy 0, policy_version 1447052 (0.0008) [2023-12-27 15:40:24,172][20347] Updated weights for policy 0, policy_version 1447062 (0.0007) [2023-12-27 15:40:24,224][20347] Updated weights for policy 0, policy_version 1447072 (0.0007) [2023-12-27 15:40:24,520][20381] Updated weights for policy 1, policy_version 1445714 (0.0006) [2023-12-27 15:40:24,577][20381] Updated weights for policy 1, policy_version 1445724 (0.0007) [2023-12-27 15:40:24,635][20381] Updated weights for policy 1, policy_version 1445734 (0.0008) [2023-12-27 15:40:24,691][20381] Updated weights for policy 1, policy_version 1445744 (0.0006) [2023-12-27 15:40:24,874][20347] Updated weights for policy 0, policy_version 1447082 (0.0007) [2023-12-27 15:40:24,928][20347] Updated weights for policy 0, policy_version 1447092 (0.0005) [2023-12-27 15:40:24,939][20291] KL-divergence is very high: 172.1471 [2023-12-27 15:40:24,945][20291] KL-divergence is very high: 211.0600 [2023-12-27 15:40:24,951][20291] KL-divergence is very high: 217.4721 [2023-12-27 15:40:24,968][20291] KL-divergence is very high: 101.1959 [2023-12-27 15:40:24,984][20347] Updated weights for policy 0, policy_version 1447102 (0.0005) [2023-12-27 15:40:25,042][20347] Updated weights for policy 0, policy_version 1447112 (0.0007) [2023-12-27 15:40:25,362][20381] Updated weights for policy 1, policy_version 1445754 (0.0007) [2023-12-27 15:40:25,415][20381] Updated weights for policy 1, policy_version 1445764 (0.0008) [2023-12-27 15:40:25,467][20381] Updated weights for policy 1, policy_version 1445774 (0.0008) [2023-12-27 15:40:25,680][20347] Updated weights for policy 0, policy_version 1447122 (0.0008) [2023-12-27 15:40:25,723][20347] Updated weights for policy 0, policy_version 1447132 (0.0008) [2023-12-27 15:40:25,766][20347] Updated weights for policy 0, policy_version 1447142 (0.0007) [2023-12-27 15:40:26,055][20381] Updated weights for policy 1, policy_version 1445784 (0.0010) [2023-12-27 15:40:26,105][20381] Updated weights for policy 1, policy_version 1445794 (0.0010) [2023-12-27 15:40:26,149][20381] Updated weights for policy 1, policy_version 1445804 (0.0007) [2023-12-27 15:40:26,456][20347] Updated weights for policy 0, policy_version 1447152 (0.0007) [2023-12-27 15:40:26,506][20347] Updated weights for policy 0, policy_version 1447162 (0.0006) [2023-12-27 15:40:26,564][20347] Updated weights for policy 0, policy_version 1447172 (0.0007) [2023-12-27 15:40:26,836][20381] Updated weights for policy 1, policy_version 1445814 (0.0006) [2023-12-27 15:40:26,887][20381] Updated weights for policy 1, policy_version 1445824 (0.0008) [2023-12-27 15:40:26,938][20381] Updated weights for policy 1, policy_version 1445834 (0.0008) [2023-12-27 15:40:27,247][20347] Updated weights for policy 0, policy_version 1447182 (0.0007) [2023-12-27 15:40:27,304][20347] Updated weights for policy 0, policy_version 1447192 (0.0006) [2023-12-27 15:40:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 740712448. Throughput: 0: 10222.4, 1: 10352.8. Samples: 740706080. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:27,363][20347] Updated weights for policy 0, policy_version 1447202 (0.0007) [2023-12-27 15:40:27,363][19170] Avg episode reward: [(0, '657.090'), (1, '478.010')] [2023-12-27 15:40:27,610][20381] Updated weights for policy 1, policy_version 1445844 (0.0008) [2023-12-27 15:40:27,665][20381] Updated weights for policy 1, policy_version 1445854 (0.0008) [2023-12-27 15:40:27,714][20381] Updated weights for policy 1, policy_version 1445864 (0.0008) [2023-12-27 15:40:28,047][20347] Updated weights for policy 0, policy_version 1447212 (0.0006) [2023-12-27 15:40:28,105][20347] Updated weights for policy 0, policy_version 1447222 (0.0006) [2023-12-27 15:40:28,167][20347] Updated weights for policy 0, policy_version 1447232 (0.0009) [2023-12-27 15:40:28,409][20381] Updated weights for policy 1, policy_version 1445874 (0.0006) [2023-12-27 15:40:28,471][20381] Updated weights for policy 1, policy_version 1445884 (0.0005) [2023-12-27 15:40:28,532][20381] Updated weights for policy 1, policy_version 1445894 (0.0007) [2023-12-27 15:40:28,588][20381] Updated weights for policy 1, policy_version 1445904 (0.0008) [2023-12-27 15:40:28,837][20347] Updated weights for policy 0, policy_version 1447242 (0.0010) [2023-12-27 15:40:28,898][20347] Updated weights for policy 0, policy_version 1447252 (0.0010) [2023-12-27 15:40:28,959][20347] Updated weights for policy 0, policy_version 1447262 (0.0010) [2023-12-27 15:40:29,020][20347] Updated weights for policy 0, policy_version 1447272 (0.0010) [2023-12-27 15:40:29,160][20381] Updated weights for policy 1, policy_version 1445914 (0.0005) [2023-12-27 15:40:29,214][20381] Updated weights for policy 1, policy_version 1445924 (0.0006) [2023-12-27 15:40:29,277][20381] Updated weights for policy 1, policy_version 1445934 (0.0009) [2023-12-27 15:40:29,609][20347] Updated weights for policy 0, policy_version 1447282 (0.0008) [2023-12-27 15:40:29,668][20347] Updated weights for policy 0, policy_version 1447292 (0.0008) [2023-12-27 15:40:29,730][20347] Updated weights for policy 0, policy_version 1447302 (0.0008) [2023-12-27 15:40:29,888][20381] Updated weights for policy 1, policy_version 1445944 (0.0008) [2023-12-27 15:40:29,939][20381] Updated weights for policy 1, policy_version 1445954 (0.0008) [2023-12-27 15:40:29,991][20381] Updated weights for policy 1, policy_version 1445964 (0.0008) [2023-12-27 15:40:30,506][20347] Updated weights for policy 0, policy_version 1447312 (0.0007) [2023-12-27 15:40:30,557][20347] Updated weights for policy 0, policy_version 1447322 (0.0005) [2023-12-27 15:40:30,608][20347] Updated weights for policy 0, policy_version 1447332 (0.0005) [2023-12-27 15:40:30,643][20381] Updated weights for policy 1, policy_version 1445974 (0.0007) [2023-12-27 15:40:30,691][20381] Updated weights for policy 1, policy_version 1445984 (0.0008) [2023-12-27 15:40:30,742][20381] Updated weights for policy 1, policy_version 1445994 (0.0008) [2023-12-27 15:40:31,190][20347] Updated weights for policy 0, policy_version 1447342 (0.0006) [2023-12-27 15:40:31,246][20347] Updated weights for policy 0, policy_version 1447352 (0.0008) [2023-12-27 15:40:31,307][20347] Updated weights for policy 0, policy_version 1447362 (0.0007) [2023-12-27 15:40:31,458][20381] Updated weights for policy 1, policy_version 1446004 (0.0006) [2023-12-27 15:40:31,521][20381] Updated weights for policy 1, policy_version 1446014 (0.0005) [2023-12-27 15:40:31,581][20381] Updated weights for policy 1, policy_version 1446024 (0.0005) [2023-12-27 15:40:31,997][20347] Updated weights for policy 0, policy_version 1447372 (0.0006) [2023-12-27 15:40:32,043][20347] Updated weights for policy 0, policy_version 1447382 (0.0007) [2023-12-27 15:40:32,092][20347] Updated weights for policy 0, policy_version 1447392 (0.0007) [2023-12-27 15:40:32,151][20381] Updated weights for policy 1, policy_version 1446034 (0.0007) [2023-12-27 15:40:32,203][20381] Updated weights for policy 1, policy_version 1446044 (0.0006) [2023-12-27 15:40:32,259][20381] Updated weights for policy 1, policy_version 1446054 (0.0006) [2023-12-27 15:40:32,316][20381] Updated weights for policy 1, policy_version 1446064 (0.0006) [2023-12-27 15:40:32,362][19170] Fps is (10 sec: 22118.2, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 740827136. Throughput: 0: 10271.7, 1: 10409.2. Samples: 740832364. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:32,363][19170] Avg episode reward: [(0, '833.380'), (1, '476.790')] [2023-12-27 15:40:32,754][20347] Updated weights for policy 0, policy_version 1447402 (0.0008) [2023-12-27 15:40:32,814][20347] Updated weights for policy 0, policy_version 1447412 (0.0007) [2023-12-27 15:40:32,869][20347] Updated weights for policy 0, policy_version 1447422 (0.0007) [2023-12-27 15:40:32,927][20347] Updated weights for policy 0, policy_version 1447432 (0.0007) [2023-12-27 15:40:33,059][20381] Updated weights for policy 1, policy_version 1446074 (0.0009) [2023-12-27 15:40:33,114][20381] Updated weights for policy 1, policy_version 1446084 (0.0007) [2023-12-27 15:40:33,163][20381] Updated weights for policy 1, policy_version 1446094 (0.0008) [2023-12-27 15:40:33,506][20347] Updated weights for policy 0, policy_version 1447442 (0.0005) [2023-12-27 15:40:33,558][20347] Updated weights for policy 0, policy_version 1447453 (0.0007) [2023-12-27 15:40:33,609][20347] Updated weights for policy 0, policy_version 1447463 (0.0009) [2023-12-27 15:40:33,835][20381] Updated weights for policy 1, policy_version 1446104 (0.0006) [2023-12-27 15:40:33,882][20381] Updated weights for policy 1, policy_version 1446114 (0.0008) [2023-12-27 15:40:33,934][20381] Updated weights for policy 1, policy_version 1446124 (0.0008) [2023-12-27 15:40:34,213][20347] Updated weights for policy 0, policy_version 1447473 (0.0009) [2023-12-27 15:40:34,271][20347] Updated weights for policy 0, policy_version 1447483 (0.0009) [2023-12-27 15:40:34,327][20347] Updated weights for policy 0, policy_version 1447493 (0.0010) [2023-12-27 15:40:34,644][20381] Updated weights for policy 1, policy_version 1446134 (0.0009) [2023-12-27 15:40:34,700][20381] Updated weights for policy 1, policy_version 1446144 (0.0006) [2023-12-27 15:40:34,755][20381] Updated weights for policy 1, policy_version 1446154 (0.0008) [2023-12-27 15:40:34,951][20347] Updated weights for policy 0, policy_version 1447503 (0.0007) [2023-12-27 15:40:35,004][20347] Updated weights for policy 0, policy_version 1447513 (0.0005) [2023-12-27 15:40:35,058][20347] Updated weights for policy 0, policy_version 1447523 (0.0005) [2023-12-27 15:40:35,503][20381] Updated weights for policy 1, policy_version 1446164 (0.0007) [2023-12-27 15:40:35,550][20381] Updated weights for policy 1, policy_version 1446174 (0.0010) [2023-12-27 15:40:35,592][20347] Updated weights for policy 0, policy_version 1447533 (0.0006) [2023-12-27 15:40:35,594][20381] Updated weights for policy 1, policy_version 1446184 (0.0006) [2023-12-27 15:40:35,639][20347] Updated weights for policy 0, policy_version 1447543 (0.0005) [2023-12-27 15:40:35,690][20347] Updated weights for policy 0, policy_version 1447553 (0.0006) [2023-12-27 15:40:36,335][20347] Updated weights for policy 0, policy_version 1447563 (0.0008) [2023-12-27 15:40:36,352][20381] Updated weights for policy 1, policy_version 1446194 (0.0007) [2023-12-27 15:40:36,401][20347] Updated weights for policy 0, policy_version 1447573 (0.0010) [2023-12-27 15:40:36,408][20381] Updated weights for policy 1, policy_version 1446204 (0.0007) [2023-12-27 15:40:36,464][20347] Updated weights for policy 0, policy_version 1447583 (0.0010) [2023-12-27 15:40:36,468][20381] Updated weights for policy 1, policy_version 1446214 (0.0009) [2023-12-27 15:40:36,523][20381] Updated weights for policy 1, policy_version 1446224 (0.0007) [2023-12-27 15:40:37,117][20347] Updated weights for policy 0, policy_version 1447593 (0.0010) [2023-12-27 15:40:37,152][20381] Updated weights for policy 1, policy_version 1446234 (0.0005) [2023-12-27 15:40:37,169][20347] Updated weights for policy 0, policy_version 1447603 (0.0010) [2023-12-27 15:40:37,208][20381] Updated weights for policy 1, policy_version 1446244 (0.0006) [2023-12-27 15:40:37,221][20347] Updated weights for policy 0, policy_version 1447613 (0.0010) [2023-12-27 15:40:37,266][20381] Updated weights for policy 1, policy_version 1446254 (0.0006) [2023-12-27 15:40:37,270][20347] Updated weights for policy 0, policy_version 1447623 (0.0010) [2023-12-27 15:40:37,363][19170] Fps is (10 sec: 22118.0, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 740933632. Throughput: 0: 10357.7, 1: 10415.0. Samples: 740895708. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:37,363][19170] Avg episode reward: [(0, '668.380'), (1, '480.980')] [2023-12-27 15:40:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001446256_370270208.pth... [2023-12-27 15:40:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001447624_370663424.pth... [2023-12-27 15:40:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001445008_369950720.pth [2023-12-27 15:40:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001446408_370352128.pth [2023-12-27 15:40:37,808][20381] Updated weights for policy 1, policy_version 1446264 (0.0005) [2023-12-27 15:40:37,869][20381] Updated weights for policy 1, policy_version 1446274 (0.0005) [2023-12-27 15:40:37,917][20347] Updated weights for policy 0, policy_version 1447633 (0.0006) [2023-12-27 15:40:37,931][20381] Updated weights for policy 1, policy_version 1446284 (0.0005) [2023-12-27 15:40:37,976][20347] Updated weights for policy 0, policy_version 1447643 (0.0006) [2023-12-27 15:40:38,029][20347] Updated weights for policy 0, policy_version 1447653 (0.0010) [2023-12-27 15:40:38,514][20381] Updated weights for policy 1, policy_version 1446294 (0.0005) [2023-12-27 15:40:38,573][20381] Updated weights for policy 1, policy_version 1446304 (0.0006) [2023-12-27 15:40:38,633][20381] Updated weights for policy 1, policy_version 1446314 (0.0009) [2023-12-27 15:40:38,685][20347] Updated weights for policy 0, policy_version 1447663 (0.0010) [2023-12-27 15:40:38,734][20347] Updated weights for policy 0, policy_version 1447673 (0.0010) [2023-12-27 15:40:38,783][20347] Updated weights for policy 0, policy_version 1447683 (0.0010) [2023-12-27 15:40:39,336][20381] Updated weights for policy 1, policy_version 1446324 (0.0009) [2023-12-27 15:40:39,407][20381] Updated weights for policy 1, policy_version 1446334 (0.0009) [2023-12-27 15:40:39,476][20381] Updated weights for policy 1, policy_version 1446344 (0.0009) [2023-12-27 15:40:39,518][20347] Updated weights for policy 0, policy_version 1447693 (0.0009) [2023-12-27 15:40:39,572][20347] Updated weights for policy 0, policy_version 1447703 (0.0006) [2023-12-27 15:40:39,626][20347] Updated weights for policy 0, policy_version 1447713 (0.0007) [2023-12-27 15:40:40,208][20381] Updated weights for policy 1, policy_version 1446354 (0.0009) [2023-12-27 15:40:40,269][20381] Updated weights for policy 1, policy_version 1446364 (0.0011) [2023-12-27 15:40:40,294][20347] Updated weights for policy 0, policy_version 1447723 (0.0005) [2023-12-27 15:40:40,326][20381] Updated weights for policy 1, policy_version 1446374 (0.0011) [2023-12-27 15:40:40,346][20347] Updated weights for policy 0, policy_version 1447733 (0.0006) [2023-12-27 15:40:40,382][20381] Updated weights for policy 1, policy_version 1446384 (0.0011) [2023-12-27 15:40:40,404][20347] Updated weights for policy 0, policy_version 1447743 (0.0007) [2023-12-27 15:40:41,010][20381] Updated weights for policy 1, policy_version 1446394 (0.0006) [2023-12-27 15:40:41,071][20347] Updated weights for policy 0, policy_version 1447753 (0.0007) [2023-12-27 15:40:41,100][20381] Updated weights for policy 1, policy_version 1446404 (0.0008) [2023-12-27 15:40:41,135][20347] Updated weights for policy 0, policy_version 1447763 (0.0007) [2023-12-27 15:40:41,165][20381] Updated weights for policy 1, policy_version 1446414 (0.0008) [2023-12-27 15:40:41,202][20347] Updated weights for policy 0, policy_version 1447773 (0.0008) [2023-12-27 15:40:41,265][20347] Updated weights for policy 0, policy_version 1447783 (0.0007) [2023-12-27 15:40:41,847][20381] Updated weights for policy 1, policy_version 1446424 (0.0006) [2023-12-27 15:40:41,913][20381] Updated weights for policy 1, policy_version 1446434 (0.0008) [2023-12-27 15:40:41,981][20381] Updated weights for policy 1, policy_version 1446444 (0.0006) [2023-12-27 15:40:42,034][20347] Updated weights for policy 0, policy_version 1447793 (0.0010) [2023-12-27 15:40:42,094][20347] Updated weights for policy 0, policy_version 1447803 (0.0010) [2023-12-27 15:40:42,156][20347] Updated weights for policy 0, policy_version 1447813 (0.0010) [2023-12-27 15:40:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 741031936. Throughput: 0: 10377.6, 1: 10410.9. Samples: 741021800. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:42,363][19170] Avg episode reward: [(0, '667.360'), (1, '503.130')] [2023-12-27 15:40:42,614][20381] Updated weights for policy 1, policy_version 1446454 (0.0007) [2023-12-27 15:40:42,670][20381] Updated weights for policy 1, policy_version 1446464 (0.0008) [2023-12-27 15:40:42,726][20381] Updated weights for policy 1, policy_version 1446474 (0.0008) [2023-12-27 15:40:42,843][20347] Updated weights for policy 0, policy_version 1447823 (0.0010) [2023-12-27 15:40:42,894][20347] Updated weights for policy 0, policy_version 1447833 (0.0010) [2023-12-27 15:40:42,949][20347] Updated weights for policy 0, policy_version 1447843 (0.0010) [2023-12-27 15:40:43,352][20381] Updated weights for policy 1, policy_version 1446484 (0.0005) [2023-12-27 15:40:43,413][20381] Updated weights for policy 1, policy_version 1446494 (0.0007) [2023-12-27 15:40:43,470][20381] Updated weights for policy 1, policy_version 1446504 (0.0007) [2023-12-27 15:40:43,575][20347] Updated weights for policy 0, policy_version 1447853 (0.0010) [2023-12-27 15:40:43,636][20347] Updated weights for policy 0, policy_version 1447863 (0.0008) [2023-12-27 15:40:43,701][20347] Updated weights for policy 0, policy_version 1447873 (0.0008) [2023-12-27 15:40:44,166][20381] Updated weights for policy 1, policy_version 1446514 (0.0008) [2023-12-27 15:40:44,231][20381] Updated weights for policy 1, policy_version 1446524 (0.0008) [2023-12-27 15:40:44,292][20381] Updated weights for policy 1, policy_version 1446534 (0.0008) [2023-12-27 15:40:44,312][20347] Updated weights for policy 0, policy_version 1447883 (0.0007) [2023-12-27 15:40:44,345][20381] Updated weights for policy 1, policy_version 1446544 (0.0008) [2023-12-27 15:40:44,370][20347] Updated weights for policy 0, policy_version 1447893 (0.0005) [2023-12-27 15:40:44,423][20347] Updated weights for policy 0, policy_version 1447903 (0.0005) [2023-12-27 15:40:45,038][20381] Updated weights for policy 1, policy_version 1446554 (0.0008) [2023-12-27 15:40:45,092][20347] Updated weights for policy 0, policy_version 1447913 (0.0007) [2023-12-27 15:40:45,100][20381] Updated weights for policy 1, policy_version 1446564 (0.0006) [2023-12-27 15:40:45,149][20347] Updated weights for policy 0, policy_version 1447923 (0.0010) [2023-12-27 15:40:45,162][20381] Updated weights for policy 1, policy_version 1446574 (0.0005) [2023-12-27 15:40:45,208][20347] Updated weights for policy 0, policy_version 1447933 (0.0006) [2023-12-27 15:40:45,267][20347] Updated weights for policy 0, policy_version 1447943 (0.0007) [2023-12-27 15:40:45,842][20381] Updated weights for policy 1, policy_version 1446584 (0.0007) [2023-12-27 15:40:45,890][20381] Updated weights for policy 1, policy_version 1446594 (0.0005) [2023-12-27 15:40:45,923][20347] Updated weights for policy 0, policy_version 1447953 (0.0011) [2023-12-27 15:40:45,945][20381] Updated weights for policy 1, policy_version 1446604 (0.0005) [2023-12-27 15:40:45,978][20347] Updated weights for policy 0, policy_version 1447963 (0.0010) [2023-12-27 15:40:46,039][20347] Updated weights for policy 0, policy_version 1447973 (0.0010) [2023-12-27 15:40:46,619][20381] Updated weights for policy 1, policy_version 1446614 (0.0007) [2023-12-27 15:40:46,668][20381] Updated weights for policy 1, policy_version 1446624 (0.0007) [2023-12-27 15:40:46,715][20347] Updated weights for policy 0, policy_version 1447983 (0.0010) [2023-12-27 15:40:46,723][20381] Updated weights for policy 1, policy_version 1446634 (0.0008) [2023-12-27 15:40:46,773][20347] Updated weights for policy 0, policy_version 1447993 (0.0010) [2023-12-27 15:40:46,836][20347] Updated weights for policy 0, policy_version 1448003 (0.0010) [2023-12-27 15:40:47,357][20381] Updated weights for policy 1, policy_version 1446644 (0.0008) [2023-12-27 15:40:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 741130240. Throughput: 0: 10388.0, 1: 10464.8. Samples: 741144980. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:47,363][19170] Avg episode reward: [(0, '647.090'), (1, '478.080')] [2023-12-27 15:40:47,406][20381] Updated weights for policy 1, policy_version 1446654 (0.0010) [2023-12-27 15:40:47,461][20381] Updated weights for policy 1, policy_version 1446664 (0.0010) [2023-12-27 15:40:47,508][20347] Updated weights for policy 0, policy_version 1448013 (0.0010) [2023-12-27 15:40:47,558][20347] Updated weights for policy 0, policy_version 1448023 (0.0010) [2023-12-27 15:40:47,625][20347] Updated weights for policy 0, policy_version 1448033 (0.0010) [2023-12-27 15:40:48,142][20381] Updated weights for policy 1, policy_version 1446674 (0.0010) [2023-12-27 15:40:48,201][20381] Updated weights for policy 1, policy_version 1446684 (0.0009) [2023-12-27 15:40:48,234][20347] Updated weights for policy 0, policy_version 1448043 (0.0010) [2023-12-27 15:40:48,253][20381] Updated weights for policy 1, policy_version 1446694 (0.0010) [2023-12-27 15:40:48,292][20347] Updated weights for policy 0, policy_version 1448053 (0.0006) [2023-12-27 15:40:48,304][20381] Updated weights for policy 1, policy_version 1446704 (0.0010) [2023-12-27 15:40:48,355][20347] Updated weights for policy 0, policy_version 1448063 (0.0007) [2023-12-27 15:40:48,965][20381] Updated weights for policy 1, policy_version 1446714 (0.0010) [2023-12-27 15:40:49,017][20381] Updated weights for policy 1, policy_version 1446724 (0.0010) [2023-12-27 15:40:49,031][20347] Updated weights for policy 0, policy_version 1448073 (0.0008) [2023-12-27 15:40:49,073][20381] Updated weights for policy 1, policy_version 1446734 (0.0011) [2023-12-27 15:40:49,084][20347] Updated weights for policy 0, policy_version 1448083 (0.0005) [2023-12-27 15:40:49,144][20347] Updated weights for policy 0, policy_version 1448093 (0.0006) [2023-12-27 15:40:49,195][20347] Updated weights for policy 0, policy_version 1448103 (0.0009) [2023-12-27 15:40:49,745][20381] Updated weights for policy 1, policy_version 1446744 (0.0008) [2023-12-27 15:40:49,802][20381] Updated weights for policy 1, policy_version 1446754 (0.0007) [2023-12-27 15:40:49,869][20381] Updated weights for policy 1, policy_version 1446764 (0.0009) [2023-12-27 15:40:49,938][20347] Updated weights for policy 0, policy_version 1448113 (0.0009) [2023-12-27 15:40:50,003][20347] Updated weights for policy 0, policy_version 1448123 (0.0010) [2023-12-27 15:40:50,069][20347] Updated weights for policy 0, policy_version 1448133 (0.0010) [2023-12-27 15:40:50,591][20381] Updated weights for policy 1, policy_version 1446774 (0.0009) [2023-12-27 15:40:50,644][20381] Updated weights for policy 1, policy_version 1446784 (0.0006) [2023-12-27 15:40:50,709][20381] Updated weights for policy 1, policy_version 1446794 (0.0008) [2023-12-27 15:40:50,727][20347] Updated weights for policy 0, policy_version 1448143 (0.0008) [2023-12-27 15:40:50,773][20347] Updated weights for policy 0, policy_version 1448153 (0.0006) [2023-12-27 15:40:50,820][20347] Updated weights for policy 0, policy_version 1448163 (0.0009) [2023-12-27 15:40:51,430][20381] Updated weights for policy 1, policy_version 1446804 (0.0009) [2023-12-27 15:40:51,480][20381] Updated weights for policy 1, policy_version 1446814 (0.0011) [2023-12-27 15:40:51,540][20381] Updated weights for policy 1, policy_version 1446824 (0.0010) [2023-12-27 15:40:51,548][20347] Updated weights for policy 0, policy_version 1448173 (0.0010) [2023-12-27 15:40:51,607][20347] Updated weights for policy 0, policy_version 1448183 (0.0010) [2023-12-27 15:40:51,682][20347] Updated weights for policy 0, policy_version 1448193 (0.0008) [2023-12-27 15:40:52,267][20381] Updated weights for policy 1, policy_version 1446834 (0.0011) [2023-12-27 15:40:52,320][20381] Updated weights for policy 1, policy_version 1446844 (0.0011) [2023-12-27 15:40:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20757.1, 300 sec: 20577.2). Total num frames: 741228544. Throughput: 0: 10374.6, 1: 10464.0. Samples: 741205820. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:52,363][19170] Avg episode reward: [(0, '686.470'), (1, '517.510')] [2023-12-27 15:40:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001448200_370810880.pth... [2023-12-27 15:40:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001446984_370499584.pth [2023-12-27 15:40:52,383][20381] Updated weights for policy 1, policy_version 1446854 (0.0011) [2023-12-27 15:40:52,405][20347] Updated weights for policy 0, policy_version 1448203 (0.0008) [2023-12-27 15:40:52,443][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001446864_370425856.pth... [2023-12-27 15:40:52,446][20381] Updated weights for policy 1, policy_version 1446864 (0.0010) [2023-12-27 15:40:52,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001445648_370114560.pth [2023-12-27 15:40:52,461][20347] Updated weights for policy 0, policy_version 1448213 (0.0010) [2023-12-27 15:40:52,511][20347] Updated weights for policy 0, policy_version 1448223 (0.0010) [2023-12-27 15:40:53,148][20347] Updated weights for policy 0, policy_version 1448233 (0.0010) [2023-12-27 15:40:53,160][20381] Updated weights for policy 1, policy_version 1446874 (0.0007) [2023-12-27 15:40:53,206][20347] Updated weights for policy 0, policy_version 1448243 (0.0010) [2023-12-27 15:40:53,222][20381] Updated weights for policy 1, policy_version 1446884 (0.0005) [2023-12-27 15:40:53,264][20347] Updated weights for policy 0, policy_version 1448253 (0.0010) [2023-12-27 15:40:53,280][20381] Updated weights for policy 1, policy_version 1446894 (0.0006) [2023-12-27 15:40:53,318][20347] Updated weights for policy 0, policy_version 1448263 (0.0010) [2023-12-27 15:40:53,947][20381] Updated weights for policy 1, policy_version 1446904 (0.0010) [2023-12-27 15:40:53,990][20347] Updated weights for policy 0, policy_version 1448273 (0.0007) [2023-12-27 15:40:53,997][20381] Updated weights for policy 1, policy_version 1446914 (0.0009) [2023-12-27 15:40:54,045][20381] Updated weights for policy 1, policy_version 1446924 (0.0010) [2023-12-27 15:40:54,051][20347] Updated weights for policy 0, policy_version 1448283 (0.0007) [2023-12-27 15:40:54,112][20347] Updated weights for policy 0, policy_version 1448293 (0.0008) [2023-12-27 15:40:54,668][20381] Updated weights for policy 1, policy_version 1446934 (0.0007) [2023-12-27 15:40:54,713][20381] Updated weights for policy 1, policy_version 1446944 (0.0005) [2023-12-27 15:40:54,765][20381] Updated weights for policy 1, policy_version 1446954 (0.0010) [2023-12-27 15:40:54,768][20347] Updated weights for policy 0, policy_version 1448303 (0.0010) [2023-12-27 15:40:54,775][20291] KL-divergence is very high: 100.0407 [2023-12-27 15:40:54,816][20347] Updated weights for policy 0, policy_version 1448313 (0.0010) [2023-12-27 15:40:54,861][20347] Updated weights for policy 0, policy_version 1448323 (0.0010) [2023-12-27 15:40:55,410][20381] Updated weights for policy 1, policy_version 1446964 (0.0008) [2023-12-27 15:40:55,465][20381] Updated weights for policy 1, policy_version 1446974 (0.0006) [2023-12-27 15:40:55,527][20381] Updated weights for policy 1, policy_version 1446984 (0.0006) [2023-12-27 15:40:55,539][20347] Updated weights for policy 0, policy_version 1448333 (0.0008) [2023-12-27 15:40:55,596][20347] Updated weights for policy 0, policy_version 1448343 (0.0010) [2023-12-27 15:40:55,654][20347] Updated weights for policy 0, policy_version 1448353 (0.0010) [2023-12-27 15:40:56,138][20381] Updated weights for policy 1, policy_version 1446994 (0.0008) [2023-12-27 15:40:56,193][20381] Updated weights for policy 1, policy_version 1447004 (0.0005) [2023-12-27 15:40:56,251][20381] Updated weights for policy 1, policy_version 1447014 (0.0006) [2023-12-27 15:40:56,313][20381] Updated weights for policy 1, policy_version 1447024 (0.0010) [2023-12-27 15:40:56,337][20347] Updated weights for policy 0, policy_version 1448363 (0.0010) [2023-12-27 15:40:56,388][20347] Updated weights for policy 0, policy_version 1448373 (0.0010) [2023-12-27 15:40:56,439][20347] Updated weights for policy 0, policy_version 1448383 (0.0010) [2023-12-27 15:40:56,924][20381] Updated weights for policy 1, policy_version 1447034 (0.0005) [2023-12-27 15:40:56,977][20381] Updated weights for policy 1, policy_version 1447044 (0.0005) [2023-12-27 15:40:57,025][20381] Updated weights for policy 1, policy_version 1447054 (0.0006) [2023-12-27 15:40:57,116][20347] Updated weights for policy 0, policy_version 1448393 (0.0010) [2023-12-27 15:40:57,165][20347] Updated weights for policy 0, policy_version 1448403 (0.0005) [2023-12-27 15:40:57,210][20347] Updated weights for policy 0, policy_version 1448413 (0.0005) [2023-12-27 15:40:57,266][20347] Updated weights for policy 0, policy_version 1448423 (0.0005) [2023-12-27 15:40:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 741343232. Throughput: 0: 10350.5, 1: 10429.4. Samples: 741329008. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:40:57,363][19170] Avg episode reward: [(0, '628.270'), (1, '533.770')] [2023-12-27 15:40:57,662][20381] Updated weights for policy 1, policy_version 1447064 (0.0010) [2023-12-27 15:40:57,706][20381] Updated weights for policy 1, policy_version 1447074 (0.0010) [2023-12-27 15:40:57,754][20381] Updated weights for policy 1, policy_version 1447084 (0.0010) [2023-12-27 15:40:57,870][20347] Updated weights for policy 0, policy_version 1448433 (0.0010) [2023-12-27 15:40:57,915][20347] Updated weights for policy 0, policy_version 1448443 (0.0010) [2023-12-27 15:40:57,962][20347] Updated weights for policy 0, policy_version 1448453 (0.0010) [2023-12-27 15:40:58,522][20381] Updated weights for policy 1, policy_version 1447094 (0.0009) [2023-12-27 15:40:58,577][20381] Updated weights for policy 1, policy_version 1447104 (0.0008) [2023-12-27 15:40:58,635][20381] Updated weights for policy 1, policy_version 1447114 (0.0008) [2023-12-27 15:40:58,717][20347] Updated weights for policy 0, policy_version 1448463 (0.0008) [2023-12-27 15:40:58,787][20347] Updated weights for policy 0, policy_version 1448473 (0.0007) [2023-12-27 15:40:58,853][20347] Updated weights for policy 0, policy_version 1448483 (0.0008) [2023-12-27 15:40:59,414][20381] Updated weights for policy 1, policy_version 1447124 (0.0008) [2023-12-27 15:40:59,465][20381] Updated weights for policy 1, policy_version 1447134 (0.0008) [2023-12-27 15:40:59,517][20381] Updated weights for policy 1, policy_version 1447144 (0.0006) [2023-12-27 15:40:59,624][20347] Updated weights for policy 0, policy_version 1448493 (0.0009) [2023-12-27 15:40:59,672][20347] Updated weights for policy 0, policy_version 1448503 (0.0010) [2023-12-27 15:40:59,727][20347] Updated weights for policy 0, policy_version 1448513 (0.0010) [2023-12-27 15:41:00,163][20381] Updated weights for policy 1, policy_version 1447154 (0.0006) [2023-12-27 15:41:00,228][20381] Updated weights for policy 1, policy_version 1447164 (0.0011) [2023-12-27 15:41:00,288][20381] Updated weights for policy 1, policy_version 1447174 (0.0011) [2023-12-27 15:41:00,343][20381] Updated weights for policy 1, policy_version 1447184 (0.0010) [2023-12-27 15:41:00,451][20347] Updated weights for policy 0, policy_version 1448523 (0.0009) [2023-12-27 15:41:00,503][20347] Updated weights for policy 0, policy_version 1448533 (0.0008) [2023-12-27 15:41:00,554][20347] Updated weights for policy 0, policy_version 1448543 (0.0007) [2023-12-27 15:41:00,993][20381] Updated weights for policy 1, policy_version 1447194 (0.0011) [2023-12-27 15:41:01,044][20381] Updated weights for policy 1, policy_version 1447204 (0.0010) [2023-12-27 15:41:01,104][20381] Updated weights for policy 1, policy_version 1447214 (0.0008) [2023-12-27 15:41:01,246][20347] Updated weights for policy 0, policy_version 1448553 (0.0007) [2023-12-27 15:41:01,304][20347] Updated weights for policy 0, policy_version 1448563 (0.0008) [2023-12-27 15:41:01,358][20347] Updated weights for policy 0, policy_version 1448573 (0.0008) [2023-12-27 15:41:01,423][20347] Updated weights for policy 0, policy_version 1448583 (0.0008) [2023-12-27 15:41:01,761][20381] Updated weights for policy 1, policy_version 1447224 (0.0010) [2023-12-27 15:41:01,823][20381] Updated weights for policy 1, policy_version 1447234 (0.0006) [2023-12-27 15:41:01,882][20381] Updated weights for policy 1, policy_version 1447244 (0.0006) [2023-12-27 15:41:02,165][20347] Updated weights for policy 0, policy_version 1448593 (0.0008) [2023-12-27 15:41:02,221][20347] Updated weights for policy 0, policy_version 1448603 (0.0008) [2023-12-27 15:41:02,273][20347] Updated weights for policy 0, policy_version 1448613 (0.0007) [2023-12-27 15:41:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 741441536. Throughput: 0: 10318.1, 1: 10348.5. Samples: 741449260. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:02,363][19170] Avg episode reward: [(0, '647.380'), (1, '487.340')] [2023-12-27 15:41:02,504][20381] Updated weights for policy 1, policy_version 1447254 (0.0006) [2023-12-27 15:41:02,564][20381] Updated weights for policy 1, policy_version 1447264 (0.0005) [2023-12-27 15:41:02,618][20381] Updated weights for policy 1, policy_version 1447274 (0.0005) [2023-12-27 15:41:02,992][20347] Updated weights for policy 0, policy_version 1448623 (0.0006) [2023-12-27 15:41:03,043][20347] Updated weights for policy 0, policy_version 1448633 (0.0005) [2023-12-27 15:41:03,100][20347] Updated weights for policy 0, policy_version 1448643 (0.0006) [2023-12-27 15:41:03,210][20381] Updated weights for policy 1, policy_version 1447284 (0.0006) [2023-12-27 15:41:03,261][20381] Updated weights for policy 1, policy_version 1447294 (0.0005) [2023-12-27 15:41:03,316][20381] Updated weights for policy 1, policy_version 1447304 (0.0007) [2023-12-27 15:41:03,754][20347] Updated weights for policy 0, policy_version 1448653 (0.0010) [2023-12-27 15:41:03,810][20347] Updated weights for policy 0, policy_version 1448663 (0.0008) [2023-12-27 15:41:03,880][20347] Updated weights for policy 0, policy_version 1448673 (0.0009) [2023-12-27 15:41:04,013][20381] Updated weights for policy 1, policy_version 1447314 (0.0007) [2023-12-27 15:41:04,074][20381] Updated weights for policy 1, policy_version 1447324 (0.0006) [2023-12-27 15:41:04,137][20381] Updated weights for policy 1, policy_version 1447334 (0.0007) [2023-12-27 15:41:04,197][20381] Updated weights for policy 1, policy_version 1447344 (0.0008) [2023-12-27 15:41:04,540][20347] Updated weights for policy 0, policy_version 1448683 (0.0006) [2023-12-27 15:41:04,593][20347] Updated weights for policy 0, policy_version 1448693 (0.0007) [2023-12-27 15:41:04,649][20347] Updated weights for policy 0, policy_version 1448703 (0.0007) [2023-12-27 15:41:04,912][20381] Updated weights for policy 1, policy_version 1447354 (0.0007) [2023-12-27 15:41:04,958][20381] Updated weights for policy 1, policy_version 1447364 (0.0007) [2023-12-27 15:41:05,014][20381] Updated weights for policy 1, policy_version 1447374 (0.0008) [2023-12-27 15:41:05,304][20347] Updated weights for policy 0, policy_version 1448713 (0.0008) [2023-12-27 15:41:05,355][20291] KL-divergence is very high: 109.3586 [2023-12-27 15:41:05,362][20347] Updated weights for policy 0, policy_version 1448723 (0.0010) [2023-12-27 15:41:05,410][20347] Updated weights for policy 0, policy_version 1448733 (0.0010) [2023-12-27 15:41:05,462][20347] Updated weights for policy 0, policy_version 1448743 (0.0009) [2023-12-27 15:41:05,707][20381] Updated weights for policy 1, policy_version 1447384 (0.0008) [2023-12-27 15:41:05,759][20381] Updated weights for policy 1, policy_version 1447394 (0.0007) [2023-12-27 15:41:05,816][20381] Updated weights for policy 1, policy_version 1447404 (0.0007) [2023-12-27 15:41:06,038][20291] KL-divergence is very high: 205.4431 [2023-12-27 15:41:06,046][20291] KL-divergence is very high: 197.0285 [2023-12-27 15:41:06,081][20347] Updated weights for policy 0, policy_version 1448753 (0.0010) [2023-12-27 15:41:06,087][20291] KL-divergence is very high: 181.9948 [2023-12-27 15:41:06,093][20291] KL-divergence is very high: 160.5875 [2023-12-27 15:41:06,138][20291] KL-divergence is very high: 183.3722 [2023-12-27 15:41:06,143][20347] Updated weights for policy 0, policy_version 1448763 (0.0008) [2023-12-27 15:41:06,144][20291] KL-divergence is very high: 160.2955 [2023-12-27 15:41:06,185][20291] KL-divergence is very high: 145.5108 [2023-12-27 15:41:06,191][20291] KL-divergence is very high: 126.9894 [2023-12-27 15:41:06,204][20347] Updated weights for policy 0, policy_version 1448773 (0.0008) [2023-12-27 15:41:06,529][20381] Updated weights for policy 1, policy_version 1447414 (0.0009) [2023-12-27 15:41:06,588][20381] Updated weights for policy 1, policy_version 1447424 (0.0008) [2023-12-27 15:41:06,650][20381] Updated weights for policy 1, policy_version 1447434 (0.0009) [2023-12-27 15:41:06,856][20347] Updated weights for policy 0, policy_version 1448783 (0.0007) [2023-12-27 15:41:06,916][20347] Updated weights for policy 0, policy_version 1448793 (0.0008) [2023-12-27 15:41:06,968][20347] Updated weights for policy 0, policy_version 1448803 (0.0007) [2023-12-27 15:41:07,297][20381] Updated weights for policy 1, policy_version 1447444 (0.0008) [2023-12-27 15:41:07,350][20381] Updated weights for policy 1, policy_version 1447454 (0.0008) [2023-12-27 15:41:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20753.0, 300 sec: 20605.3). Total num frames: 741539840. Throughput: 0: 10282.4, 1: 10334.4. Samples: 741510504. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:07,364][19170] Avg episode reward: [(0, '529.750'), (1, '409.290')] [2023-12-27 15:41:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001448808_370966528.pth... [2023-12-27 15:41:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001447624_370663424.pth [2023-12-27 15:41:07,418][20381] Updated weights for policy 1, policy_version 1447464 (0.0008) [2023-12-27 15:41:07,466][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001447472_370581504.pth... [2023-12-27 15:41:07,469][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001446256_370270208.pth [2023-12-27 15:41:07,652][20347] Updated weights for policy 0, policy_version 1448813 (0.0008) [2023-12-27 15:41:07,714][20347] Updated weights for policy 0, policy_version 1448823 (0.0008) [2023-12-27 15:41:07,779][20347] Updated weights for policy 0, policy_version 1448833 (0.0009) [2023-12-27 15:41:08,030][20381] Updated weights for policy 1, policy_version 1447474 (0.0007) [2023-12-27 15:41:08,087][20381] Updated weights for policy 1, policy_version 1447484 (0.0006) [2023-12-27 15:41:08,135][20381] Updated weights for policy 1, policy_version 1447494 (0.0007) [2023-12-27 15:41:08,187][20381] Updated weights for policy 1, policy_version 1447504 (0.0008) [2023-12-27 15:41:08,447][20347] Updated weights for policy 0, policy_version 1448843 (0.0008) [2023-12-27 15:41:08,495][20347] Updated weights for policy 0, policy_version 1448853 (0.0008) [2023-12-27 15:41:08,549][20347] Updated weights for policy 0, policy_version 1448863 (0.0009) [2023-12-27 15:41:08,876][20381] Updated weights for policy 1, policy_version 1447514 (0.0007) [2023-12-27 15:41:08,928][20381] Updated weights for policy 1, policy_version 1447524 (0.0008) [2023-12-27 15:41:08,979][20381] Updated weights for policy 1, policy_version 1447534 (0.0008) [2023-12-27 15:41:09,265][20347] Updated weights for policy 0, policy_version 1448873 (0.0008) [2023-12-27 15:41:09,326][20347] Updated weights for policy 0, policy_version 1448883 (0.0007) [2023-12-27 15:41:09,388][20347] Updated weights for policy 0, policy_version 1448893 (0.0008) [2023-12-27 15:41:09,454][20347] Updated weights for policy 0, policy_version 1448903 (0.0008) [2023-12-27 15:41:09,715][20381] Updated weights for policy 1, policy_version 1447544 (0.0006) [2023-12-27 15:41:09,782][20381] Updated weights for policy 1, policy_version 1447554 (0.0008) [2023-12-27 15:41:09,847][20381] Updated weights for policy 1, policy_version 1447564 (0.0008) [2023-12-27 15:41:10,145][20347] Updated weights for policy 0, policy_version 1448913 (0.0007) [2023-12-27 15:41:10,200][20347] Updated weights for policy 0, policy_version 1448923 (0.0008) [2023-12-27 15:41:10,256][20347] Updated weights for policy 0, policy_version 1448933 (0.0009) [2023-12-27 15:41:10,529][20381] Updated weights for policy 1, policy_version 1447574 (0.0008) [2023-12-27 15:41:10,589][20381] Updated weights for policy 1, policy_version 1447584 (0.0006) [2023-12-27 15:41:10,652][20381] Updated weights for policy 1, policy_version 1447594 (0.0007) [2023-12-27 15:41:10,954][20347] Updated weights for policy 0, policy_version 1448943 (0.0009) [2023-12-27 15:41:11,020][20347] Updated weights for policy 0, policy_version 1448953 (0.0008) [2023-12-27 15:41:11,088][20347] Updated weights for policy 0, policy_version 1448963 (0.0009) [2023-12-27 15:41:11,361][20381] Updated weights for policy 1, policy_version 1447604 (0.0007) [2023-12-27 15:41:11,429][20381] Updated weights for policy 1, policy_version 1447614 (0.0009) [2023-12-27 15:41:11,488][20381] Updated weights for policy 1, policy_version 1447624 (0.0009) [2023-12-27 15:41:11,965][20347] Updated weights for policy 0, policy_version 1448973 (0.0009) [2023-12-27 15:41:12,020][20347] Updated weights for policy 0, policy_version 1448983 (0.0005) [2023-12-27 15:41:12,080][20347] Updated weights for policy 0, policy_version 1448993 (0.0005) [2023-12-27 15:41:12,162][20381] Updated weights for policy 1, policy_version 1447634 (0.0008) [2023-12-27 15:41:12,218][20381] Updated weights for policy 1, policy_version 1447644 (0.0008) [2023-12-27 15:41:12,284][20381] Updated weights for policy 1, policy_version 1447654 (0.0008) [2023-12-27 15:41:12,339][20381] Updated weights for policy 1, policy_version 1447664 (0.0008) [2023-12-27 15:41:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 741646336. Throughput: 0: 10300.7, 1: 10295.5. Samples: 741632912. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:12,363][19170] Avg episode reward: [(0, '637.240'), (1, '475.480')] [2023-12-27 15:41:12,765][20347] Updated weights for policy 0, policy_version 1449003 (0.0008) [2023-12-27 15:41:12,825][20347] Updated weights for policy 0, policy_version 1449013 (0.0007) [2023-12-27 15:41:12,880][20347] Updated weights for policy 0, policy_version 1449023 (0.0008) [2023-12-27 15:41:13,039][20381] Updated weights for policy 1, policy_version 1447674 (0.0008) [2023-12-27 15:41:13,098][20381] Updated weights for policy 1, policy_version 1447684 (0.0009) [2023-12-27 15:41:13,163][20381] Updated weights for policy 1, policy_version 1447694 (0.0008) [2023-12-27 15:41:13,515][20347] Updated weights for policy 0, policy_version 1449033 (0.0007) [2023-12-27 15:41:13,570][20347] Updated weights for policy 0, policy_version 1449043 (0.0007) [2023-12-27 15:41:13,611][20291] KL-divergence is very high: 132.9863 [2023-12-27 15:41:13,621][20347] Updated weights for policy 0, policy_version 1449053 (0.0007) [2023-12-27 15:41:13,672][20347] Updated weights for policy 0, policy_version 1449063 (0.0007) [2023-12-27 15:41:13,910][20381] Updated weights for policy 1, policy_version 1447704 (0.0008) [2023-12-27 15:41:13,974][20381] Updated weights for policy 1, policy_version 1447714 (0.0008) [2023-12-27 15:41:14,032][20381] Updated weights for policy 1, policy_version 1447724 (0.0008) [2023-12-27 15:41:14,379][20347] Updated weights for policy 0, policy_version 1449073 (0.0007) [2023-12-27 15:41:14,431][20347] Updated weights for policy 0, policy_version 1449083 (0.0008) [2023-12-27 15:41:14,483][20347] Updated weights for policy 0, policy_version 1449093 (0.0008) [2023-12-27 15:41:14,707][20381] Updated weights for policy 1, policy_version 1447734 (0.0007) [2023-12-27 15:41:14,760][20381] Updated weights for policy 1, policy_version 1447744 (0.0008) [2023-12-27 15:41:14,822][20381] Updated weights for policy 1, policy_version 1447754 (0.0011) [2023-12-27 15:41:15,171][20347] Updated weights for policy 0, policy_version 1449103 (0.0009) [2023-12-27 15:41:15,226][20347] Updated weights for policy 0, policy_version 1449113 (0.0010) [2023-12-27 15:41:15,288][20347] Updated weights for policy 0, policy_version 1449123 (0.0010) [2023-12-27 15:41:15,491][20381] Updated weights for policy 1, policy_version 1447764 (0.0009) [2023-12-27 15:41:15,541][20381] Updated weights for policy 1, policy_version 1447774 (0.0010) [2023-12-27 15:41:15,597][20381] Updated weights for policy 1, policy_version 1447784 (0.0011) [2023-12-27 15:41:15,947][20347] Updated weights for policy 0, policy_version 1449133 (0.0010) [2023-12-27 15:41:16,004][20347] Updated weights for policy 0, policy_version 1449143 (0.0010) [2023-12-27 15:41:16,059][20347] Updated weights for policy 0, policy_version 1449153 (0.0010) [2023-12-27 15:41:16,230][20381] Updated weights for policy 1, policy_version 1447794 (0.0011) [2023-12-27 15:41:16,275][20381] Updated weights for policy 1, policy_version 1447804 (0.0010) [2023-12-27 15:41:16,323][20381] Updated weights for policy 1, policy_version 1447814 (0.0010) [2023-12-27 15:41:16,369][20381] Updated weights for policy 1, policy_version 1447824 (0.0010) [2023-12-27 15:41:16,710][20347] Updated weights for policy 0, policy_version 1449163 (0.0010) [2023-12-27 15:41:16,761][20347] Updated weights for policy 0, policy_version 1449173 (0.0010) [2023-12-27 15:41:16,816][20347] Updated weights for policy 0, policy_version 1449183 (0.0010) [2023-12-27 15:41:17,093][20381] Updated weights for policy 1, policy_version 1447834 (0.0005) [2023-12-27 15:41:17,145][20381] Updated weights for policy 1, policy_version 1447844 (0.0005) [2023-12-27 15:41:17,202][20381] Updated weights for policy 1, policy_version 1447854 (0.0006) [2023-12-27 15:41:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 741744640. Throughput: 0: 10254.3, 1: 10215.5. Samples: 741753504. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:17,363][19170] Avg episode reward: [(0, '627.330'), (1, '582.680')] [2023-12-27 15:41:17,421][20347] Updated weights for policy 0, policy_version 1449193 (0.0010) [2023-12-27 15:41:17,472][20347] Updated weights for policy 0, policy_version 1449203 (0.0010) [2023-12-27 15:41:17,530][20347] Updated weights for policy 0, policy_version 1449213 (0.0006) [2023-12-27 15:41:17,592][20347] Updated weights for policy 0, policy_version 1449223 (0.0006) [2023-12-27 15:41:17,837][20381] Updated weights for policy 1, policy_version 1447864 (0.0006) [2023-12-27 15:41:17,901][20381] Updated weights for policy 1, policy_version 1447874 (0.0006) [2023-12-27 15:41:17,961][20381] Updated weights for policy 1, policy_version 1447884 (0.0007) [2023-12-27 15:41:18,177][20347] Updated weights for policy 0, policy_version 1449233 (0.0009) [2023-12-27 15:41:18,230][20347] Updated weights for policy 0, policy_version 1449243 (0.0010) [2023-12-27 15:41:18,282][20347] Updated weights for policy 0, policy_version 1449253 (0.0010) [2023-12-27 15:41:18,606][20381] Updated weights for policy 1, policy_version 1447894 (0.0008) [2023-12-27 15:41:18,666][20381] Updated weights for policy 1, policy_version 1447904 (0.0009) [2023-12-27 15:41:18,726][20381] Updated weights for policy 1, policy_version 1447914 (0.0007) [2023-12-27 15:41:18,940][20347] Updated weights for policy 0, policy_version 1449263 (0.0008) [2023-12-27 15:41:18,984][20347] Updated weights for policy 0, policy_version 1449273 (0.0007) [2023-12-27 15:41:19,031][20347] Updated weights for policy 0, policy_version 1449283 (0.0006) [2023-12-27 15:41:19,440][20381] Updated weights for policy 1, policy_version 1447924 (0.0007) [2023-12-27 15:41:19,502][20381] Updated weights for policy 1, policy_version 1447934 (0.0009) [2023-12-27 15:41:19,557][20381] Updated weights for policy 1, policy_version 1447944 (0.0008) [2023-12-27 15:41:19,666][20347] Updated weights for policy 0, policy_version 1449293 (0.0007) [2023-12-27 15:41:19,717][20347] Updated weights for policy 0, policy_version 1449303 (0.0010) [2023-12-27 15:41:19,770][20347] Updated weights for policy 0, policy_version 1449313 (0.0010) [2023-12-27 15:41:20,267][20381] Updated weights for policy 1, policy_version 1447954 (0.0007) [2023-12-27 15:41:20,322][20381] Updated weights for policy 1, policy_version 1447964 (0.0008) [2023-12-27 15:41:20,388][20381] Updated weights for policy 1, policy_version 1447974 (0.0008) [2023-12-27 15:41:20,446][20381] Updated weights for policy 1, policy_version 1447984 (0.0009) [2023-12-27 15:41:20,496][20347] Updated weights for policy 0, policy_version 1449323 (0.0010) [2023-12-27 15:41:20,551][20347] Updated weights for policy 0, policy_version 1449333 (0.0010) [2023-12-27 15:41:20,617][20347] Updated weights for policy 0, policy_version 1449343 (0.0010) [2023-12-27 15:41:21,173][20381] Updated weights for policy 1, policy_version 1447994 (0.0008) [2023-12-27 15:41:21,234][20381] Updated weights for policy 1, policy_version 1448004 (0.0008) [2023-12-27 15:41:21,294][20381] Updated weights for policy 1, policy_version 1448014 (0.0007) [2023-12-27 15:41:21,374][20347] Updated weights for policy 0, policy_version 1449353 (0.0008) [2023-12-27 15:41:21,436][20347] Updated weights for policy 0, policy_version 1449363 (0.0010) [2023-12-27 15:41:21,494][20347] Updated weights for policy 0, policy_version 1449373 (0.0008) [2023-12-27 15:41:21,563][20347] Updated weights for policy 0, policy_version 1449383 (0.0008) [2023-12-27 15:41:21,974][20381] Updated weights for policy 1, policy_version 1448024 (0.0005) [2023-12-27 15:41:22,036][20381] Updated weights for policy 1, policy_version 1448034 (0.0006) [2023-12-27 15:41:22,093][20381] Updated weights for policy 1, policy_version 1448044 (0.0008) [2023-12-27 15:41:22,354][20347] Updated weights for policy 0, policy_version 1449393 (0.0008) [2023-12-27 15:41:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 741842944. Throughput: 0: 10234.0, 1: 10229.1. Samples: 741816544. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:22,363][19170] Avg episode reward: [(0, '636.990'), (1, '533.260')] [2023-12-27 15:41:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001448048_370728960.pth... [2023-12-27 15:41:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001446864_370425856.pth [2023-12-27 15:41:22,409][20347] Updated weights for policy 0, policy_version 1449403 (0.0009) [2023-12-27 15:41:22,467][20347] Updated weights for policy 0, policy_version 1449413 (0.0008) [2023-12-27 15:41:22,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001449416_371122176.pth... [2023-12-27 15:41:22,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001448200_370810880.pth [2023-12-27 15:41:22,726][20381] Updated weights for policy 1, policy_version 1448054 (0.0008) [2023-12-27 15:41:22,786][20381] Updated weights for policy 1, policy_version 1448064 (0.0007) [2023-12-27 15:41:22,843][20381] Updated weights for policy 1, policy_version 1448074 (0.0007) [2023-12-27 15:41:23,159][20347] Updated weights for policy 0, policy_version 1449423 (0.0008) [2023-12-27 15:41:23,167][20291] KL-divergence is very high: 119.3470 [2023-12-27 15:41:23,173][20291] KL-divergence is very high: 138.6937 [2023-12-27 15:41:23,210][20347] Updated weights for policy 0, policy_version 1449433 (0.0008) [2023-12-27 15:41:23,263][20347] Updated weights for policy 0, policy_version 1449443 (0.0008) [2023-12-27 15:41:23,514][20381] Updated weights for policy 1, policy_version 1448084 (0.0007) [2023-12-27 15:41:23,586][20381] Updated weights for policy 1, policy_version 1448094 (0.0008) [2023-12-27 15:41:23,641][20381] Updated weights for policy 1, policy_version 1448104 (0.0008) [2023-12-27 15:41:23,960][20347] Updated weights for policy 0, policy_version 1449453 (0.0010) [2023-12-27 15:41:24,008][20347] Updated weights for policy 0, policy_version 1449463 (0.0010) [2023-12-27 15:41:24,058][20347] Updated weights for policy 0, policy_version 1449473 (0.0009) [2023-12-27 15:41:24,324][20381] Updated weights for policy 1, policy_version 1448114 (0.0008) [2023-12-27 15:41:24,374][20381] Updated weights for policy 1, policy_version 1448124 (0.0008) [2023-12-27 15:41:24,435][20381] Updated weights for policy 1, policy_version 1448134 (0.0007) [2023-12-27 15:41:24,482][20381] Updated weights for policy 1, policy_version 1448144 (0.0007) [2023-12-27 15:41:24,720][20347] Updated weights for policy 0, policy_version 1449483 (0.0007) [2023-12-27 15:41:24,768][20347] Updated weights for policy 0, policy_version 1449493 (0.0010) [2023-12-27 15:41:24,815][20347] Updated weights for policy 0, policy_version 1449503 (0.0010) [2023-12-27 15:41:24,853][20291] KL-divergence is very high: 161.4564 [2023-12-27 15:41:25,192][20381] Updated weights for policy 1, policy_version 1448154 (0.0007) [2023-12-27 15:41:25,245][20381] Updated weights for policy 1, policy_version 1448164 (0.0010) [2023-12-27 15:41:25,290][20381] Updated weights for policy 1, policy_version 1448174 (0.0010) [2023-12-27 15:41:25,497][20291] KL-divergence is very high: 246.0906 [2023-12-27 15:41:25,504][20347] Updated weights for policy 0, policy_version 1449513 (0.0010) [2023-12-27 15:41:25,545][20291] KL-divergence is very high: 353.8060 [2023-12-27 15:41:25,564][20347] Updated weights for policy 0, policy_version 1449523 (0.0010) [2023-12-27 15:41:25,592][20291] KL-divergence is very high: 382.3279 [2023-12-27 15:41:25,619][20347] Updated weights for policy 0, policy_version 1449533 (0.0010) [2023-12-27 15:41:25,636][20291] KL-divergence is very high: 322.4517 [2023-12-27 15:41:25,679][20347] Updated weights for policy 0, policy_version 1449543 (0.0006) [2023-12-27 15:41:25,858][20381] Updated weights for policy 1, policy_version 1448184 (0.0010) [2023-12-27 15:41:25,910][20381] Updated weights for policy 1, policy_version 1448194 (0.0010) [2023-12-27 15:41:25,965][20381] Updated weights for policy 1, policy_version 1448204 (0.0010) [2023-12-27 15:41:26,326][20347] Updated weights for policy 0, policy_version 1449553 (0.0008) [2023-12-27 15:41:26,383][20347] Updated weights for policy 0, policy_version 1449563 (0.0005) [2023-12-27 15:41:26,438][20347] Updated weights for policy 0, policy_version 1449573 (0.0006) [2023-12-27 15:41:26,569][20381] Updated weights for policy 1, policy_version 1448214 (0.0010) [2023-12-27 15:41:26,629][20381] Updated weights for policy 1, policy_version 1448224 (0.0010) [2023-12-27 15:41:26,686][20381] Updated weights for policy 1, policy_version 1448234 (0.0010) [2023-12-27 15:41:27,026][20347] Updated weights for policy 0, policy_version 1449583 (0.0007) [2023-12-27 15:41:27,083][20347] Updated weights for policy 0, policy_version 1449593 (0.0007) [2023-12-27 15:41:27,144][20347] Updated weights for policy 0, policy_version 1449603 (0.0007) [2023-12-27 15:41:27,360][20381] Updated weights for policy 1, policy_version 1448244 (0.0010) [2023-12-27 15:41:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 741949440. Throughput: 0: 10141.7, 1: 10220.6. Samples: 741938104. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:27,363][19170] Avg episode reward: [(0, '582.430'), (1, '559.900')] [2023-12-27 15:41:27,416][20381] Updated weights for policy 1, policy_version 1448254 (0.0010) [2023-12-27 15:41:27,471][20381] Updated weights for policy 1, policy_version 1448264 (0.0010) [2023-12-27 15:41:27,799][20347] Updated weights for policy 0, policy_version 1449613 (0.0006) [2023-12-27 15:41:27,847][20347] Updated weights for policy 0, policy_version 1449623 (0.0005) [2023-12-27 15:41:27,895][20347] Updated weights for policy 0, policy_version 1449633 (0.0005) [2023-12-27 15:41:28,159][20381] Updated weights for policy 1, policy_version 1448274 (0.0010) [2023-12-27 15:41:28,220][20381] Updated weights for policy 1, policy_version 1448284 (0.0010) [2023-12-27 15:41:28,285][20381] Updated weights for policy 1, policy_version 1448294 (0.0010) [2023-12-27 15:41:28,338][20381] Updated weights for policy 1, policy_version 1448304 (0.0011) [2023-12-27 15:41:28,473][20347] Updated weights for policy 0, policy_version 1449643 (0.0006) [2023-12-27 15:41:28,525][20347] Updated weights for policy 0, policy_version 1449653 (0.0007) [2023-12-27 15:41:28,578][20347] Updated weights for policy 0, policy_version 1449663 (0.0007) [2023-12-27 15:41:29,030][20381] Updated weights for policy 1, policy_version 1448314 (0.0011) [2023-12-27 15:41:29,080][20381] Updated weights for policy 1, policy_version 1448324 (0.0010) [2023-12-27 15:41:29,135][20381] Updated weights for policy 1, policy_version 1448334 (0.0007) [2023-12-27 15:41:29,310][20347] Updated weights for policy 0, policy_version 1449673 (0.0008) [2023-12-27 15:41:29,378][20347] Updated weights for policy 0, policy_version 1449683 (0.0007) [2023-12-27 15:41:29,436][20347] Updated weights for policy 0, policy_version 1449693 (0.0008) [2023-12-27 15:41:29,493][20347] Updated weights for policy 0, policy_version 1449703 (0.0010) [2023-12-27 15:41:29,828][20381] Updated weights for policy 1, policy_version 1448344 (0.0009) [2023-12-27 15:41:29,890][20381] Updated weights for policy 1, policy_version 1448354 (0.0008) [2023-12-27 15:41:29,958][20381] Updated weights for policy 1, policy_version 1448364 (0.0007) [2023-12-27 15:41:30,121][20347] Updated weights for policy 0, policy_version 1449713 (0.0007) [2023-12-27 15:41:30,182][20347] Updated weights for policy 0, policy_version 1449723 (0.0005) [2023-12-27 15:41:30,238][20347] Updated weights for policy 0, policy_version 1449733 (0.0005) [2023-12-27 15:41:30,583][20381] Updated weights for policy 1, policy_version 1448374 (0.0005) [2023-12-27 15:41:30,633][20381] Updated weights for policy 1, policy_version 1448384 (0.0005) [2023-12-27 15:41:30,680][20381] Updated weights for policy 1, policy_version 1448394 (0.0005) [2023-12-27 15:41:30,797][20347] Updated weights for policy 0, policy_version 1449743 (0.0009) [2023-12-27 15:41:30,844][20347] Updated weights for policy 0, policy_version 1449753 (0.0010) [2023-12-27 15:41:30,898][20347] Updated weights for policy 0, policy_version 1449763 (0.0010) [2023-12-27 15:41:31,265][20381] Updated weights for policy 1, policy_version 1448404 (0.0007) [2023-12-27 15:41:31,325][20381] Updated weights for policy 1, policy_version 1448414 (0.0008) [2023-12-27 15:41:31,391][20381] Updated weights for policy 1, policy_version 1448424 (0.0007) [2023-12-27 15:41:31,647][20347] Updated weights for policy 0, policy_version 1449773 (0.0010) [2023-12-27 15:41:31,699][20347] Updated weights for policy 0, policy_version 1449783 (0.0011) [2023-12-27 15:41:31,759][20347] Updated weights for policy 0, policy_version 1449793 (0.0008) [2023-12-27 15:41:32,022][20381] Updated weights for policy 1, policy_version 1448434 (0.0007) [2023-12-27 15:41:32,080][20381] Updated weights for policy 1, policy_version 1448444 (0.0007) [2023-12-27 15:41:32,143][20381] Updated weights for policy 1, policy_version 1448454 (0.0008) [2023-12-27 15:41:32,206][20381] Updated weights for policy 1, policy_version 1448464 (0.0008) [2023-12-27 15:41:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 742055936. Throughput: 0: 10208.1, 1: 10244.4. Samples: 742065340. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:32,363][19170] Avg episode reward: [(0, '572.470'), (1, '400.890')] [2023-12-27 15:41:32,386][20347] Updated weights for policy 0, policy_version 1449803 (0.0005) [2023-12-27 15:41:32,444][20347] Updated weights for policy 0, policy_version 1449813 (0.0005) [2023-12-27 15:41:32,503][20347] Updated weights for policy 0, policy_version 1449823 (0.0007) [2023-12-27 15:41:32,889][20381] Updated weights for policy 1, policy_version 1448474 (0.0008) [2023-12-27 15:41:32,946][20381] Updated weights for policy 1, policy_version 1448485 (0.0008) [2023-12-27 15:41:32,991][20381] Updated weights for policy 1, policy_version 1448495 (0.0008) [2023-12-27 15:41:33,072][20347] Updated weights for policy 0, policy_version 1449833 (0.0007) [2023-12-27 15:41:33,128][20347] Updated weights for policy 0, policy_version 1449843 (0.0005) [2023-12-27 15:41:33,185][20347] Updated weights for policy 0, policy_version 1449853 (0.0005) [2023-12-27 15:41:33,242][20347] Updated weights for policy 0, policy_version 1449863 (0.0005) [2023-12-27 15:41:33,718][20381] Updated weights for policy 1, policy_version 1448505 (0.0007) [2023-12-27 15:41:33,765][20381] Updated weights for policy 1, policy_version 1448515 (0.0008) [2023-12-27 15:41:33,798][20347] Updated weights for policy 0, policy_version 1449873 (0.0009) [2023-12-27 15:41:33,817][20381] Updated weights for policy 1, policy_version 1448525 (0.0010) [2023-12-27 15:41:33,849][20347] Updated weights for policy 0, policy_version 1449883 (0.0007) [2023-12-27 15:41:33,910][20347] Updated weights for policy 0, policy_version 1449893 (0.0007) [2023-12-27 15:41:34,530][20381] Updated weights for policy 1, policy_version 1448535 (0.0007) [2023-12-27 15:41:34,582][20347] Updated weights for policy 0, policy_version 1449903 (0.0007) [2023-12-27 15:41:34,590][20381] Updated weights for policy 1, policy_version 1448545 (0.0006) [2023-12-27 15:41:34,647][20347] Updated weights for policy 0, policy_version 1449913 (0.0007) [2023-12-27 15:41:34,657][20381] Updated weights for policy 1, policy_version 1448555 (0.0009) [2023-12-27 15:41:34,707][20347] Updated weights for policy 0, policy_version 1449923 (0.0008) [2023-12-27 15:41:35,316][20381] Updated weights for policy 1, policy_version 1448565 (0.0007) [2023-12-27 15:41:35,368][20381] Updated weights for policy 1, policy_version 1448575 (0.0007) [2023-12-27 15:41:35,406][20347] Updated weights for policy 0, policy_version 1449933 (0.0008) [2023-12-27 15:41:35,430][20381] Updated weights for policy 1, policy_version 1448585 (0.0007) [2023-12-27 15:41:35,471][20347] Updated weights for policy 0, policy_version 1449943 (0.0008) [2023-12-27 15:41:35,529][20347] Updated weights for policy 0, policy_version 1449953 (0.0008) [2023-12-27 15:41:36,075][20381] Updated weights for policy 1, policy_version 1448595 (0.0007) [2023-12-27 15:41:36,141][20381] Updated weights for policy 1, policy_version 1448605 (0.0007) [2023-12-27 15:41:36,201][20381] Updated weights for policy 1, policy_version 1448615 (0.0006) [2023-12-27 15:41:36,245][20347] Updated weights for policy 0, policy_version 1449963 (0.0008) [2023-12-27 15:41:36,315][20347] Updated weights for policy 0, policy_version 1449973 (0.0008) [2023-12-27 15:41:36,375][20347] Updated weights for policy 0, policy_version 1449983 (0.0009) [2023-12-27 15:41:36,805][20381] Updated weights for policy 1, policy_version 1448625 (0.0006) [2023-12-27 15:41:36,868][20381] Updated weights for policy 1, policy_version 1448635 (0.0008) [2023-12-27 15:41:36,928][20381] Updated weights for policy 1, policy_version 1448645 (0.0008) [2023-12-27 15:41:36,987][20381] Updated weights for policy 1, policy_version 1448655 (0.0008) [2023-12-27 15:41:37,052][20347] Updated weights for policy 0, policy_version 1449993 (0.0007) [2023-12-27 15:41:37,111][20347] Updated weights for policy 0, policy_version 1450003 (0.0005) [2023-12-27 15:41:37,170][20347] Updated weights for policy 0, policy_version 1450013 (0.0005) [2023-12-27 15:41:37,225][20347] Updated weights for policy 0, policy_version 1450023 (0.0007) [2023-12-27 15:41:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 742162432. Throughput: 0: 10267.3, 1: 10238.7. Samples: 742128588. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:37,363][19170] Avg episode reward: [(0, '525.070'), (1, '494.630')] [2023-12-27 15:41:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001448656_370884608.pth... [2023-12-27 15:41:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001450024_371277824.pth... [2023-12-27 15:41:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001447472_370581504.pth [2023-12-27 15:41:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001448808_370966528.pth [2023-12-27 15:41:37,673][20381] Updated weights for policy 1, policy_version 1448665 (0.0007) [2023-12-27 15:41:37,724][20381] Updated weights for policy 1, policy_version 1448675 (0.0007) [2023-12-27 15:41:37,783][20381] Updated weights for policy 1, policy_version 1448685 (0.0006) [2023-12-27 15:41:37,902][20347] Updated weights for policy 0, policy_version 1450033 (0.0008) [2023-12-27 15:41:37,951][20347] Updated weights for policy 0, policy_version 1450043 (0.0007) [2023-12-27 15:41:38,014][20347] Updated weights for policy 0, policy_version 1450053 (0.0006) [2023-12-27 15:41:38,512][20381] Updated weights for policy 1, policy_version 1448695 (0.0008) [2023-12-27 15:41:38,564][20381] Updated weights for policy 1, policy_version 1448705 (0.0007) [2023-12-27 15:41:38,619][20381] Updated weights for policy 1, policy_version 1448715 (0.0008) [2023-12-27 15:41:38,733][20347] Updated weights for policy 0, policy_version 1450063 (0.0007) [2023-12-27 15:41:38,786][20347] Updated weights for policy 0, policy_version 1450073 (0.0008) [2023-12-27 15:41:38,844][20347] Updated weights for policy 0, policy_version 1450083 (0.0007) [2023-12-27 15:41:39,357][20381] Updated weights for policy 1, policy_version 1448725 (0.0007) [2023-12-27 15:41:39,430][20381] Updated weights for policy 1, policy_version 1448735 (0.0008) [2023-12-27 15:41:39,492][20381] Updated weights for policy 1, policy_version 1448745 (0.0007) [2023-12-27 15:41:39,578][20347] Updated weights for policy 0, policy_version 1450093 (0.0008) [2023-12-27 15:41:39,634][20347] Updated weights for policy 0, policy_version 1450103 (0.0008) [2023-12-27 15:41:39,694][20347] Updated weights for policy 0, policy_version 1450113 (0.0010) [2023-12-27 15:41:40,222][20381] Updated weights for policy 1, policy_version 1448755 (0.0006) [2023-12-27 15:41:40,270][20381] Updated weights for policy 1, policy_version 1448765 (0.0007) [2023-12-27 15:41:40,325][20381] Updated weights for policy 1, policy_version 1448775 (0.0007) [2023-12-27 15:41:40,410][20347] Updated weights for policy 0, policy_version 1450123 (0.0011) [2023-12-27 15:41:40,460][20347] Updated weights for policy 0, policy_version 1450133 (0.0010) [2023-12-27 15:41:40,512][20347] Updated weights for policy 0, policy_version 1450143 (0.0007) [2023-12-27 15:41:41,049][20381] Updated weights for policy 1, policy_version 1448785 (0.0008) [2023-12-27 15:41:41,134][20381] Updated weights for policy 1, policy_version 1448795 (0.0008) [2023-12-27 15:41:41,181][20347] Updated weights for policy 0, policy_version 1450153 (0.0007) [2023-12-27 15:41:41,200][20381] Updated weights for policy 1, policy_version 1448805 (0.0007) [2023-12-27 15:41:41,244][20347] Updated weights for policy 0, policy_version 1450163 (0.0008) [2023-12-27 15:41:41,261][20381] Updated weights for policy 1, policy_version 1448815 (0.0007) [2023-12-27 15:41:41,295][20347] Updated weights for policy 0, policy_version 1450173 (0.0006) [2023-12-27 15:41:41,358][20347] Updated weights for policy 0, policy_version 1450183 (0.0007) [2023-12-27 15:41:41,960][20381] Updated weights for policy 1, policy_version 1448825 (0.0008) [2023-12-27 15:41:42,017][20381] Updated weights for policy 1, policy_version 1448835 (0.0007) [2023-12-27 15:41:42,079][20381] Updated weights for policy 1, policy_version 1448845 (0.0008) [2023-12-27 15:41:42,081][20347] Updated weights for policy 0, policy_version 1450193 (0.0006) [2023-12-27 15:41:42,143][20347] Updated weights for policy 0, policy_version 1450203 (0.0005) [2023-12-27 15:41:42,209][20347] Updated weights for policy 0, policy_version 1450213 (0.0005) [2023-12-27 15:41:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 742260736. Throughput: 0: 10224.5, 1: 10190.0. Samples: 742247664. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:42,363][19170] Avg episode reward: [(0, '514.260'), (1, '526.360')] [2023-12-27 15:41:42,833][20381] Updated weights for policy 1, policy_version 1448855 (0.0008) [2023-12-27 15:41:42,839][20347] Updated weights for policy 0, policy_version 1450223 (0.0006) [2023-12-27 15:41:42,892][20381] Updated weights for policy 1, policy_version 1448865 (0.0009) [2023-12-27 15:41:42,904][20347] Updated weights for policy 0, policy_version 1450233 (0.0006) [2023-12-27 15:41:42,939][20381] Updated weights for policy 1, policy_version 1448875 (0.0009) [2023-12-27 15:41:42,968][20347] Updated weights for policy 0, policy_version 1450243 (0.0006) [2023-12-27 15:41:43,523][20381] Updated weights for policy 1, policy_version 1448885 (0.0007) [2023-12-27 15:41:43,585][20381] Updated weights for policy 1, policy_version 1448895 (0.0007) [2023-12-27 15:41:43,647][20381] Updated weights for policy 1, policy_version 1448905 (0.0008) [2023-12-27 15:41:43,657][20347] Updated weights for policy 0, policy_version 1450253 (0.0007) [2023-12-27 15:41:43,716][20347] Updated weights for policy 0, policy_version 1450263 (0.0005) [2023-12-27 15:41:43,767][20347] Updated weights for policy 0, policy_version 1450273 (0.0005) [2023-12-27 15:41:44,321][20381] Updated weights for policy 1, policy_version 1448915 (0.0008) [2023-12-27 15:41:44,376][20381] Updated weights for policy 1, policy_version 1448925 (0.0007) [2023-12-27 15:41:44,426][20381] Updated weights for policy 1, policy_version 1448935 (0.0007) [2023-12-27 15:41:44,479][20347] Updated weights for policy 0, policy_version 1450283 (0.0007) [2023-12-27 15:41:44,539][20347] Updated weights for policy 0, policy_version 1450293 (0.0006) [2023-12-27 15:41:44,600][20347] Updated weights for policy 0, policy_version 1450303 (0.0006) [2023-12-27 15:41:45,161][20381] Updated weights for policy 1, policy_version 1448945 (0.0008) [2023-12-27 15:41:45,199][20347] Updated weights for policy 0, policy_version 1450313 (0.0006) [2023-12-27 15:41:45,210][20381] Updated weights for policy 1, policy_version 1448955 (0.0007) [2023-12-27 15:41:45,255][20347] Updated weights for policy 0, policy_version 1450323 (0.0008) [2023-12-27 15:41:45,260][20381] Updated weights for policy 1, policy_version 1448965 (0.0007) [2023-12-27 15:41:45,307][20347] Updated weights for policy 0, policy_version 1450333 (0.0007) [2023-12-27 15:41:45,312][20381] Updated weights for policy 1, policy_version 1448975 (0.0009) [2023-12-27 15:41:45,370][20347] Updated weights for policy 0, policy_version 1450343 (0.0006) [2023-12-27 15:41:45,962][20381] Updated weights for policy 1, policy_version 1448985 (0.0006) [2023-12-27 15:41:46,019][20381] Updated weights for policy 1, policy_version 1448995 (0.0007) [2023-12-27 15:41:46,061][20347] Updated weights for policy 0, policy_version 1450353 (0.0007) [2023-12-27 15:41:46,064][20381] Updated weights for policy 1, policy_version 1449005 (0.0006) [2023-12-27 15:41:46,116][20347] Updated weights for policy 0, policy_version 1450363 (0.0007) [2023-12-27 15:41:46,180][20347] Updated weights for policy 0, policy_version 1450373 (0.0006) [2023-12-27 15:41:46,717][20347] Updated weights for policy 0, policy_version 1450383 (0.0006) [2023-12-27 15:41:46,721][20381] Updated weights for policy 1, policy_version 1449015 (0.0009) [2023-12-27 15:41:46,773][20347] Updated weights for policy 0, policy_version 1450393 (0.0005) [2023-12-27 15:41:46,781][20381] Updated weights for policy 1, policy_version 1449025 (0.0007) [2023-12-27 15:41:46,827][20347] Updated weights for policy 0, policy_version 1450403 (0.0006) [2023-12-27 15:41:46,844][20381] Updated weights for policy 1, policy_version 1449035 (0.0008) [2023-12-27 15:41:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 742359040. Throughput: 0: 10296.8, 1: 10208.2. Samples: 742371984. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:47,363][19170] Avg episode reward: [(0, '438.450'), (1, '549.910')] [2023-12-27 15:41:47,453][20347] Updated weights for policy 0, policy_version 1450413 (0.0005) [2023-12-27 15:41:47,457][20381] Updated weights for policy 1, policy_version 1449045 (0.0005) [2023-12-27 15:41:47,505][20347] Updated weights for policy 0, policy_version 1450423 (0.0006) [2023-12-27 15:41:47,508][20381] Updated weights for policy 1, policy_version 1449055 (0.0005) [2023-12-27 15:41:47,557][20347] Updated weights for policy 0, policy_version 1450433 (0.0010) [2023-12-27 15:41:47,558][20381] Updated weights for policy 1, policy_version 1449065 (0.0005) [2023-12-27 15:41:48,088][20381] Updated weights for policy 1, policy_version 1449075 (0.0006) [2023-12-27 15:41:48,139][20381] Updated weights for policy 1, policy_version 1449085 (0.0008) [2023-12-27 15:41:48,167][20347] Updated weights for policy 0, policy_version 1450443 (0.0006) [2023-12-27 15:41:48,188][20381] Updated weights for policy 1, policy_version 1449095 (0.0007) [2023-12-27 15:41:48,224][20347] Updated weights for policy 0, policy_version 1450453 (0.0007) [2023-12-27 15:41:48,272][20347] Updated weights for policy 0, policy_version 1450463 (0.0008) [2023-12-27 15:41:48,830][20381] Updated weights for policy 1, policy_version 1449105 (0.0005) [2023-12-27 15:41:48,893][20381] Updated weights for policy 1, policy_version 1449115 (0.0009) [2023-12-27 15:41:48,947][20381] Updated weights for policy 1, policy_version 1449125 (0.0007) [2023-12-27 15:41:48,993][20347] Updated weights for policy 0, policy_version 1450473 (0.0008) [2023-12-27 15:41:48,998][20381] Updated weights for policy 1, policy_version 1449135 (0.0007) [2023-12-27 15:41:49,050][20347] Updated weights for policy 0, policy_version 1450483 (0.0007) [2023-12-27 15:41:49,098][20347] Updated weights for policy 0, policy_version 1450493 (0.0007) [2023-12-27 15:41:49,162][20347] Updated weights for policy 0, policy_version 1450503 (0.0008) [2023-12-27 15:41:49,723][20381] Updated weights for policy 1, policy_version 1449145 (0.0008) [2023-12-27 15:41:49,781][20381] Updated weights for policy 1, policy_version 1449155 (0.0008) [2023-12-27 15:41:49,825][20347] Updated weights for policy 0, policy_version 1450513 (0.0008) [2023-12-27 15:41:49,853][20381] Updated weights for policy 1, policy_version 1449165 (0.0007) [2023-12-27 15:41:49,886][20347] Updated weights for policy 0, policy_version 1450523 (0.0008) [2023-12-27 15:41:49,949][20347] Updated weights for policy 0, policy_version 1450533 (0.0008) [2023-12-27 15:41:50,460][20381] Updated weights for policy 1, policy_version 1449175 (0.0006) [2023-12-27 15:41:50,503][20381] Updated weights for policy 1, policy_version 1449185 (0.0007) [2023-12-27 15:41:50,548][20381] Updated weights for policy 1, policy_version 1449195 (0.0008) [2023-12-27 15:41:50,692][20347] Updated weights for policy 0, policy_version 1450543 (0.0008) [2023-12-27 15:41:50,746][20347] Updated weights for policy 0, policy_version 1450553 (0.0008) [2023-12-27 15:41:50,805][20347] Updated weights for policy 0, policy_version 1450563 (0.0008) [2023-12-27 15:41:51,260][20381] Updated weights for policy 1, policy_version 1449205 (0.0009) [2023-12-27 15:41:51,318][20381] Updated weights for policy 1, policy_version 1449215 (0.0006) [2023-12-27 15:41:51,385][20381] Updated weights for policy 1, policy_version 1449225 (0.0007) [2023-12-27 15:41:51,510][20347] Updated weights for policy 0, policy_version 1450573 (0.0006) [2023-12-27 15:41:51,568][20347] Updated weights for policy 0, policy_version 1450583 (0.0007) [2023-12-27 15:41:51,632][20347] Updated weights for policy 0, policy_version 1450593 (0.0007) [2023-12-27 15:41:52,049][20381] Updated weights for policy 1, policy_version 1449235 (0.0006) [2023-12-27 15:41:52,115][20381] Updated weights for policy 1, policy_version 1449245 (0.0008) [2023-12-27 15:41:52,175][20381] Updated weights for policy 1, policy_version 1449255 (0.0008) [2023-12-27 15:41:52,344][20347] Updated weights for policy 0, policy_version 1450603 (0.0007) [2023-12-27 15:41:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 742465536. Throughput: 0: 10296.4, 1: 10267.2. Samples: 742435864. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:52,363][19170] Avg episode reward: [(0, '398.170'), (1, '470.690')] [2023-12-27 15:41:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001449264_371040256.pth... [2023-12-27 15:41:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001448048_370728960.pth [2023-12-27 15:41:52,409][20347] Updated weights for policy 0, policy_version 1450613 (0.0008) [2023-12-27 15:41:52,472][20347] Updated weights for policy 0, policy_version 1450623 (0.0011) [2023-12-27 15:41:52,534][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001450632_371433472.pth... [2023-12-27 15:41:52,539][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001449416_371122176.pth [2023-12-27 15:41:52,851][20381] Updated weights for policy 1, policy_version 1449265 (0.0007) [2023-12-27 15:41:52,908][20381] Updated weights for policy 1, policy_version 1449275 (0.0008) [2023-12-27 15:41:52,963][20381] Updated weights for policy 1, policy_version 1449285 (0.0006) [2023-12-27 15:41:53,015][20381] Updated weights for policy 1, policy_version 1449295 (0.0008) [2023-12-27 15:41:53,089][20347] Updated weights for policy 0, policy_version 1450633 (0.0009) [2023-12-27 15:41:53,138][20347] Updated weights for policy 0, policy_version 1450643 (0.0008) [2023-12-27 15:41:53,183][20347] Updated weights for policy 0, policy_version 1450653 (0.0007) [2023-12-27 15:41:53,246][20347] Updated weights for policy 0, policy_version 1450663 (0.0008) [2023-12-27 15:41:53,686][20381] Updated weights for policy 1, policy_version 1449305 (0.0006) [2023-12-27 15:41:53,743][20381] Updated weights for policy 1, policy_version 1449315 (0.0006) [2023-12-27 15:41:53,788][20381] Updated weights for policy 1, policy_version 1449325 (0.0005) [2023-12-27 15:41:53,909][20347] Updated weights for policy 0, policy_version 1450673 (0.0007) [2023-12-27 15:41:53,959][20347] Updated weights for policy 0, policy_version 1450683 (0.0006) [2023-12-27 15:41:54,015][20347] Updated weights for policy 0, policy_version 1450693 (0.0007) [2023-12-27 15:41:54,424][20381] Updated weights for policy 1, policy_version 1449335 (0.0006) [2023-12-27 15:41:54,485][20381] Updated weights for policy 1, policy_version 1449345 (0.0006) [2023-12-27 15:41:54,552][20381] Updated weights for policy 1, policy_version 1449355 (0.0006) [2023-12-27 15:41:54,609][20347] Updated weights for policy 0, policy_version 1450703 (0.0009) [2023-12-27 15:41:54,655][20347] Updated weights for policy 0, policy_version 1450713 (0.0010) [2023-12-27 15:41:54,706][20347] Updated weights for policy 0, policy_version 1450723 (0.0010) [2023-12-27 15:41:55,241][20381] Updated weights for policy 1, policy_version 1449365 (0.0008) [2023-12-27 15:41:55,304][20381] Updated weights for policy 1, policy_version 1449375 (0.0007) [2023-12-27 15:41:55,364][20381] Updated weights for policy 1, policy_version 1449385 (0.0007) [2023-12-27 15:41:55,369][20347] Updated weights for policy 0, policy_version 1450733 (0.0007) [2023-12-27 15:41:55,424][20347] Updated weights for policy 0, policy_version 1450743 (0.0007) [2023-12-27 15:41:55,475][20347] Updated weights for policy 0, policy_version 1450753 (0.0005) [2023-12-27 15:41:56,074][20347] Updated weights for policy 0, policy_version 1450763 (0.0005) [2023-12-27 15:41:56,076][20381] Updated weights for policy 1, policy_version 1449395 (0.0006) [2023-12-27 15:41:56,122][20347] Updated weights for policy 0, policy_version 1450773 (0.0006) [2023-12-27 15:41:56,131][20381] Updated weights for policy 1, policy_version 1449405 (0.0008) [2023-12-27 15:41:56,177][20347] Updated weights for policy 0, policy_version 1450783 (0.0006) [2023-12-27 15:41:56,195][20381] Updated weights for policy 1, policy_version 1449415 (0.0007) [2023-12-27 15:41:56,808][20347] Updated weights for policy 0, policy_version 1450793 (0.0006) [2023-12-27 15:41:56,869][20347] Updated weights for policy 0, policy_version 1450803 (0.0007) [2023-12-27 15:41:56,901][20381] Updated weights for policy 1, policy_version 1449425 (0.0009) [2023-12-27 15:41:56,921][20347] Updated weights for policy 0, policy_version 1450813 (0.0007) [2023-12-27 15:41:56,953][20381] Updated weights for policy 1, policy_version 1449435 (0.0006) [2023-12-27 15:41:56,973][20347] Updated weights for policy 0, policy_version 1450823 (0.0005) [2023-12-27 15:41:57,007][20381] Updated weights for policy 1, policy_version 1449445 (0.0007) [2023-12-27 15:41:57,062][20381] Updated weights for policy 1, policy_version 1449455 (0.0008) [2023-12-27 15:41:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 742572032. Throughput: 0: 10347.6, 1: 10262.4. Samples: 742560360. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:41:57,363][19170] Avg episode reward: [(0, '410.390'), (1, '336.580')] [2023-12-27 15:41:57,569][20347] Updated weights for policy 0, policy_version 1450833 (0.0007) [2023-12-27 15:41:57,613][20347] Updated weights for policy 0, policy_version 1450843 (0.0007) [2023-12-27 15:41:57,671][20347] Updated weights for policy 0, policy_version 1450853 (0.0007) [2023-12-27 15:41:57,772][20381] Updated weights for policy 1, policy_version 1449465 (0.0007) [2023-12-27 15:41:57,817][20381] Updated weights for policy 1, policy_version 1449475 (0.0008) [2023-12-27 15:41:57,863][20381] Updated weights for policy 1, policy_version 1449485 (0.0008) [2023-12-27 15:41:58,297][20347] Updated weights for policy 0, policy_version 1450863 (0.0007) [2023-12-27 15:41:58,359][20347] Updated weights for policy 0, policy_version 1450873 (0.0008) [2023-12-27 15:41:58,421][20347] Updated weights for policy 0, policy_version 1450883 (0.0009) [2023-12-27 15:41:58,617][20381] Updated weights for policy 1, policy_version 1449495 (0.0008) [2023-12-27 15:41:58,686][20381] Updated weights for policy 1, policy_version 1449505 (0.0008) [2023-12-27 15:41:58,755][20381] Updated weights for policy 1, policy_version 1449515 (0.0008) [2023-12-27 15:41:59,224][20347] Updated weights for policy 0, policy_version 1450893 (0.0007) [2023-12-27 15:41:59,290][20347] Updated weights for policy 0, policy_version 1450903 (0.0008) [2023-12-27 15:41:59,361][20347] Updated weights for policy 0, policy_version 1450913 (0.0008) [2023-12-27 15:41:59,535][20381] Updated weights for policy 1, policy_version 1449525 (0.0009) [2023-12-27 15:41:59,592][20381] Updated weights for policy 1, policy_version 1449535 (0.0008) [2023-12-27 15:41:59,652][20381] Updated weights for policy 1, policy_version 1449545 (0.0008) [2023-12-27 15:42:00,112][20347] Updated weights for policy 0, policy_version 1450923 (0.0008) [2023-12-27 15:42:00,169][20347] Updated weights for policy 0, policy_version 1450933 (0.0005) [2023-12-27 15:42:00,229][20347] Updated weights for policy 0, policy_version 1450943 (0.0005) [2023-12-27 15:42:00,369][20381] Updated weights for policy 1, policy_version 1449555 (0.0007) [2023-12-27 15:42:00,431][20381] Updated weights for policy 1, policy_version 1449565 (0.0006) [2023-12-27 15:42:00,488][20381] Updated weights for policy 1, policy_version 1449575 (0.0007) [2023-12-27 15:42:00,884][20347] Updated weights for policy 0, policy_version 1450953 (0.0006) [2023-12-27 15:42:00,932][20347] Updated weights for policy 0, policy_version 1450963 (0.0008) [2023-12-27 15:42:00,989][20347] Updated weights for policy 0, policy_version 1450973 (0.0008) [2023-12-27 15:42:01,028][20381] Updated weights for policy 1, policy_version 1449585 (0.0007) [2023-12-27 15:42:01,046][20347] Updated weights for policy 0, policy_version 1450983 (0.0007) [2023-12-27 15:42:01,093][20381] Updated weights for policy 1, policy_version 1449595 (0.0008) [2023-12-27 15:42:01,153][20381] Updated weights for policy 1, policy_version 1449605 (0.0007) [2023-12-27 15:42:01,213][20381] Updated weights for policy 1, policy_version 1449615 (0.0005) [2023-12-27 15:42:01,737][20347] Updated weights for policy 0, policy_version 1450993 (0.0006) [2023-12-27 15:42:01,795][20347] Updated weights for policy 0, policy_version 1451003 (0.0008) [2023-12-27 15:42:01,850][20347] Updated weights for policy 0, policy_version 1451013 (0.0007) [2023-12-27 15:42:01,918][20381] Updated weights for policy 1, policy_version 1449625 (0.0007) [2023-12-27 15:42:01,975][20381] Updated weights for policy 1, policy_version 1449635 (0.0008) [2023-12-27 15:42:02,026][20381] Updated weights for policy 1, policy_version 1449645 (0.0007) [2023-12-27 15:42:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 742670336. Throughput: 0: 10367.8, 1: 10219.9. Samples: 742679952. Policy #0 lag: (min: 30.0, avg: 38.0, max: 62.0) [2023-12-27 15:42:02,363][19170] Avg episode reward: [(0, '385.760'), (1, '75.650')] [2023-12-27 15:42:02,531][20347] Updated weights for policy 0, policy_version 1451023 (0.0007) [2023-12-27 15:42:02,592][20347] Updated weights for policy 0, policy_version 1451033 (0.0008) [2023-12-27 15:42:02,642][20347] Updated weights for policy 0, policy_version 1451043 (0.0008) [2023-12-27 15:42:02,740][20381] Updated weights for policy 1, policy_version 1449655 (0.0007) [2023-12-27 15:42:02,792][20381] Updated weights for policy 1, policy_version 1449665 (0.0007) [2023-12-27 15:42:02,844][20381] Updated weights for policy 1, policy_version 1449675 (0.0007) [2023-12-27 15:42:03,240][20347] Updated weights for policy 0, policy_version 1451053 (0.0008) [2023-12-27 15:42:03,298][20347] Updated weights for policy 0, policy_version 1451063 (0.0008) [2023-12-27 15:42:03,346][20347] Updated weights for policy 0, policy_version 1451073 (0.0007) [2023-12-27 15:42:03,559][20381] Updated weights for policy 1, policy_version 1449685 (0.0007) [2023-12-27 15:42:03,613][20381] Updated weights for policy 1, policy_version 1449695 (0.0007) [2023-12-27 15:42:03,672][20381] Updated weights for policy 1, policy_version 1449705 (0.0008) [2023-12-27 15:42:04,036][20347] Updated weights for policy 0, policy_version 1451083 (0.0008) [2023-12-27 15:42:04,094][20347] Updated weights for policy 0, policy_version 1451093 (0.0008) [2023-12-27 15:42:04,149][20347] Updated weights for policy 0, policy_version 1451103 (0.0008) [2023-12-27 15:42:04,338][20381] Updated weights for policy 1, policy_version 1449715 (0.0007) [2023-12-27 15:42:04,402][20381] Updated weights for policy 1, policy_version 1449725 (0.0006) [2023-12-27 15:42:04,468][20381] Updated weights for policy 1, policy_version 1449735 (0.0007) [2023-12-27 15:42:04,808][20347] Updated weights for policy 0, policy_version 1451113 (0.0007) [2023-12-27 15:42:04,861][20347] Updated weights for policy 0, policy_version 1451123 (0.0005) [2023-12-27 15:42:04,905][20347] Updated weights for policy 0, policy_version 1451133 (0.0005) [2023-12-27 15:42:04,955][20347] Updated weights for policy 0, policy_version 1451143 (0.0005) [2023-12-27 15:42:05,116][20381] Updated weights for policy 1, policy_version 1449745 (0.0007) [2023-12-27 15:42:05,179][20381] Updated weights for policy 1, policy_version 1449755 (0.0010) [2023-12-27 15:42:05,228][20381] Updated weights for policy 1, policy_version 1449765 (0.0010) [2023-12-27 15:42:05,284][20381] Updated weights for policy 1, policy_version 1449775 (0.0010) [2023-12-27 15:42:05,546][20347] Updated weights for policy 0, policy_version 1451153 (0.0006) [2023-12-27 15:42:05,597][20347] Updated weights for policy 0, policy_version 1451163 (0.0006) [2023-12-27 15:42:05,643][20347] Updated weights for policy 0, policy_version 1451173 (0.0005) [2023-12-27 15:42:05,847][20381] Updated weights for policy 1, policy_version 1449785 (0.0007) [2023-12-27 15:42:05,898][20381] Updated weights for policy 1, policy_version 1449795 (0.0007) [2023-12-27 15:42:05,943][20381] Updated weights for policy 1, policy_version 1449805 (0.0008) [2023-12-27 15:42:06,301][20347] Updated weights for policy 0, policy_version 1451183 (0.0006) [2023-12-27 15:42:06,362][20347] Updated weights for policy 0, policy_version 1451193 (0.0008) [2023-12-27 15:42:06,419][20347] Updated weights for policy 0, policy_version 1451203 (0.0007) [2023-12-27 15:42:06,676][20381] Updated weights for policy 1, policy_version 1449815 (0.0006) [2023-12-27 15:42:06,735][20381] Updated weights for policy 1, policy_version 1449825 (0.0006) [2023-12-27 15:42:06,788][20381] Updated weights for policy 1, policy_version 1449835 (0.0008) [2023-12-27 15:42:07,108][20347] Updated weights for policy 0, policy_version 1451213 (0.0007) [2023-12-27 15:42:07,154][20347] Updated weights for policy 0, policy_version 1451223 (0.0008) [2023-12-27 15:42:07,200][20347] Updated weights for policy 0, policy_version 1451233 (0.0007) [2023-12-27 15:42:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 742776832. Throughput: 0: 10350.4, 1: 10228.9. Samples: 742742616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:07,364][19170] Avg episode reward: [(0, '360.050'), (1, '176.880')] [2023-12-27 15:42:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001449840_371187712.pth... [2023-12-27 15:42:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001451240_371589120.pth... [2023-12-27 15:42:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001450024_371277824.pth [2023-12-27 15:42:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001448656_370884608.pth [2023-12-27 15:42:07,427][20381] Updated weights for policy 1, policy_version 1449845 (0.0007) [2023-12-27 15:42:07,479][20381] Updated weights for policy 1, policy_version 1449855 (0.0008) [2023-12-27 15:42:07,535][20381] Updated weights for policy 1, policy_version 1449865 (0.0008) [2023-12-27 15:42:07,937][20347] Updated weights for policy 0, policy_version 1451243 (0.0007) [2023-12-27 15:42:08,000][20347] Updated weights for policy 0, policy_version 1451253 (0.0006) [2023-12-27 15:42:08,052][20347] Updated weights for policy 0, policy_version 1451263 (0.0008) [2023-12-27 15:42:08,191][20381] Updated weights for policy 1, policy_version 1449875 (0.0008) [2023-12-27 15:42:08,248][20381] Updated weights for policy 1, policy_version 1449885 (0.0005) [2023-12-27 15:42:08,303][20381] Updated weights for policy 1, policy_version 1449895 (0.0005) [2023-12-27 15:42:08,738][20347] Updated weights for policy 0, policy_version 1451273 (0.0009) [2023-12-27 15:42:08,793][20347] Updated weights for policy 0, policy_version 1451283 (0.0008) [2023-12-27 15:42:08,844][20347] Updated weights for policy 0, policy_version 1451293 (0.0008) [2023-12-27 15:42:08,892][20347] Updated weights for policy 0, policy_version 1451303 (0.0009) [2023-12-27 15:42:08,979][20381] Updated weights for policy 1, policy_version 1449905 (0.0008) [2023-12-27 15:42:09,041][20381] Updated weights for policy 1, policy_version 1449915 (0.0008) [2023-12-27 15:42:09,108][20381] Updated weights for policy 1, policy_version 1449925 (0.0008) [2023-12-27 15:42:09,171][20381] Updated weights for policy 1, policy_version 1449935 (0.0008) [2023-12-27 15:42:09,670][20347] Updated weights for policy 0, policy_version 1451313 (0.0008) [2023-12-27 15:42:09,724][20347] Updated weights for policy 0, policy_version 1451323 (0.0007) [2023-12-27 15:42:09,778][20347] Updated weights for policy 0, policy_version 1451333 (0.0010) [2023-12-27 15:42:09,895][20381] Updated weights for policy 1, policy_version 1449945 (0.0009) [2023-12-27 15:42:09,956][20381] Updated weights for policy 1, policy_version 1449955 (0.0008) [2023-12-27 15:42:10,012][20381] Updated weights for policy 1, policy_version 1449965 (0.0008) [2023-12-27 15:42:10,462][20347] Updated weights for policy 0, policy_version 1451343 (0.0010) [2023-12-27 15:42:10,517][20347] Updated weights for policy 0, policy_version 1451353 (0.0010) [2023-12-27 15:42:10,573][20347] Updated weights for policy 0, policy_version 1451363 (0.0010) [2023-12-27 15:42:10,730][20381] Updated weights for policy 1, policy_version 1449975 (0.0007) [2023-12-27 15:42:10,791][20381] Updated weights for policy 1, policy_version 1449985 (0.0007) [2023-12-27 15:42:10,846][20381] Updated weights for policy 1, policy_version 1449995 (0.0007) [2023-12-27 15:42:11,276][20347] Updated weights for policy 0, policy_version 1451373 (0.0010) [2023-12-27 15:42:11,327][20347] Updated weights for policy 0, policy_version 1451383 (0.0008) [2023-12-27 15:42:11,394][20347] Updated weights for policy 0, policy_version 1451393 (0.0008) [2023-12-27 15:42:11,584][20381] Updated weights for policy 1, policy_version 1450005 (0.0007) [2023-12-27 15:42:11,650][20381] Updated weights for policy 1, policy_version 1450015 (0.0007) [2023-12-27 15:42:11,708][20381] Updated weights for policy 1, policy_version 1450025 (0.0008) [2023-12-27 15:42:12,124][20347] Updated weights for policy 0, policy_version 1451403 (0.0007) [2023-12-27 15:42:12,180][20347] Updated weights for policy 0, policy_version 1451413 (0.0007) [2023-12-27 15:42:12,238][20347] Updated weights for policy 0, policy_version 1451423 (0.0007) [2023-12-27 15:42:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 742875136. Throughput: 0: 10392.2, 1: 10206.0. Samples: 742865020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:12,363][19170] Avg episode reward: [(0, '429.830'), (1, '298.750')] [2023-12-27 15:42:12,475][20381] Updated weights for policy 1, policy_version 1450035 (0.0008) [2023-12-27 15:42:12,535][20381] Updated weights for policy 1, policy_version 1450045 (0.0008) [2023-12-27 15:42:12,594][20381] Updated weights for policy 1, policy_version 1450055 (0.0008) [2023-12-27 15:42:12,927][20347] Updated weights for policy 0, policy_version 1451433 (0.0007) [2023-12-27 15:42:12,990][20347] Updated weights for policy 0, policy_version 1451443 (0.0007) [2023-12-27 15:42:13,037][20347] Updated weights for policy 0, policy_version 1451453 (0.0007) [2023-12-27 15:42:13,088][20347] Updated weights for policy 0, policy_version 1451463 (0.0007) [2023-12-27 15:42:13,311][20381] Updated weights for policy 1, policy_version 1450065 (0.0008) [2023-12-27 15:42:13,371][20381] Updated weights for policy 1, policy_version 1450075 (0.0008) [2023-12-27 15:42:13,427][20381] Updated weights for policy 1, policy_version 1450085 (0.0008) [2023-12-27 15:42:13,472][20381] Updated weights for policy 1, policy_version 1450095 (0.0007) [2023-12-27 15:42:13,740][20347] Updated weights for policy 0, policy_version 1451473 (0.0007) [2023-12-27 15:42:13,788][20347] Updated weights for policy 0, policy_version 1451483 (0.0007) [2023-12-27 15:42:13,846][20347] Updated weights for policy 0, policy_version 1451493 (0.0008) [2023-12-27 15:42:14,175][20381] Updated weights for policy 1, policy_version 1450105 (0.0007) [2023-12-27 15:42:14,231][20381] Updated weights for policy 1, policy_version 1450115 (0.0008) [2023-12-27 15:42:14,288][20381] Updated weights for policy 1, policy_version 1450125 (0.0007) [2023-12-27 15:42:14,597][20347] Updated weights for policy 0, policy_version 1451503 (0.0008) [2023-12-27 15:42:14,657][20347] Updated weights for policy 0, policy_version 1451513 (0.0007) [2023-12-27 15:42:14,717][20347] Updated weights for policy 0, policy_version 1451523 (0.0008) [2023-12-27 15:42:14,963][20381] Updated weights for policy 1, policy_version 1450135 (0.0010) [2023-12-27 15:42:15,030][20381] Updated weights for policy 1, policy_version 1450145 (0.0011) [2023-12-27 15:42:15,091][20381] Updated weights for policy 1, policy_version 1450155 (0.0011) [2023-12-27 15:42:15,460][20347] Updated weights for policy 0, policy_version 1451533 (0.0007) [2023-12-27 15:42:15,513][20347] Updated weights for policy 0, policy_version 1451543 (0.0008) [2023-12-27 15:42:15,517][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-27 15:42:15,776][20381] Updated weights for policy 1, policy_version 1450165 (0.0011) [2023-12-27 15:42:15,828][20381] Updated weights for policy 1, policy_version 1450175 (0.0011) [2023-12-27 15:42:15,881][20381] Updated weights for policy 1, policy_version 1450185 (0.0010) [2023-12-27 15:42:16,230][20347] Updated weights for policy 0, policy_version 1451553 (0.0010) [2023-12-27 15:42:16,288][20347] Updated weights for policy 0, policy_version 1451563 (0.0010) [2023-12-27 15:42:16,339][20347] Updated weights for policy 0, policy_version 1451573 (0.0010) [2023-12-27 15:42:16,521][20381] Updated weights for policy 1, policy_version 1450195 (0.0009) [2023-12-27 15:42:16,581][20381] Updated weights for policy 1, policy_version 1450205 (0.0005) [2023-12-27 15:42:16,637][20381] Updated weights for policy 1, policy_version 1450215 (0.0005) [2023-12-27 15:42:16,958][20347] Updated weights for policy 0, policy_version 1451583 (0.0007) [2023-12-27 15:42:17,009][20347] Updated weights for policy 0, policy_version 1451593 (0.0010) [2023-12-27 15:42:17,066][20347] Updated weights for policy 0, policy_version 1451603 (0.0010) [2023-12-27 15:42:17,318][20381] Updated weights for policy 1, policy_version 1450225 (0.0006) [2023-12-27 15:42:17,363][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 742973440. Throughput: 0: 10308.7, 1: 10135.8. Samples: 742985344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:17,364][19170] Avg episode reward: [(0, '439.010'), (1, '434.750')] [2023-12-27 15:42:17,365][20381] Updated weights for policy 1, policy_version 1450235 (0.0005) [2023-12-27 15:42:17,412][20381] Updated weights for policy 1, policy_version 1450245 (0.0007) [2023-12-27 15:42:17,460][20381] Updated weights for policy 1, policy_version 1450255 (0.0007) [2023-12-27 15:42:17,708][20347] Updated weights for policy 0, policy_version 1451613 (0.0008) [2023-12-27 15:42:17,762][20347] Updated weights for policy 0, policy_version 1451623 (0.0009) [2023-12-27 15:42:17,819][20347] Updated weights for policy 0, policy_version 1451633 (0.0010) [2023-12-27 15:42:18,058][20381] Updated weights for policy 1, policy_version 1450265 (0.0005) [2023-12-27 15:42:18,116][20381] Updated weights for policy 1, policy_version 1450275 (0.0008) [2023-12-27 15:42:18,167][20381] Updated weights for policy 1, policy_version 1450285 (0.0007) [2023-12-27 15:42:18,487][20347] Updated weights for policy 0, policy_version 1451643 (0.0010) [2023-12-27 15:42:18,545][20347] Updated weights for policy 0, policy_version 1451653 (0.0010) [2023-12-27 15:42:18,605][20347] Updated weights for policy 0, policy_version 1451663 (0.0010) [2023-12-27 15:42:18,865][20381] Updated weights for policy 1, policy_version 1450295 (0.0007) [2023-12-27 15:42:18,918][20381] Updated weights for policy 1, policy_version 1450305 (0.0008) [2023-12-27 15:42:18,976][20381] Updated weights for policy 1, policy_version 1450315 (0.0007) [2023-12-27 15:42:19,254][20347] Updated weights for policy 0, policy_version 1451673 (0.0010) [2023-12-27 15:42:19,312][20347] Updated weights for policy 0, policy_version 1451683 (0.0011) [2023-12-27 15:42:19,381][20347] Updated weights for policy 0, policy_version 1451693 (0.0010) [2023-12-27 15:42:19,443][20347] Updated weights for policy 0, policy_version 1451703 (0.0010) [2023-12-27 15:42:19,685][20381] Updated weights for policy 1, policy_version 1450325 (0.0007) [2023-12-27 15:42:19,749][20381] Updated weights for policy 1, policy_version 1450335 (0.0008) [2023-12-27 15:42:19,808][20381] Updated weights for policy 1, policy_version 1450345 (0.0008) [2023-12-27 15:42:20,157][20347] Updated weights for policy 0, policy_version 1451713 (0.0008) [2023-12-27 15:42:20,210][20347] Updated weights for policy 0, policy_version 1451723 (0.0008) [2023-12-27 15:42:20,255][20347] Updated weights for policy 0, policy_version 1451733 (0.0008) [2023-12-27 15:42:20,563][20381] Updated weights for policy 1, policy_version 1450355 (0.0008) [2023-12-27 15:42:20,630][20381] Updated weights for policy 1, policy_version 1450365 (0.0007) [2023-12-27 15:42:20,695][20381] Updated weights for policy 1, policy_version 1450375 (0.0008) [2023-12-27 15:42:20,929][20347] Updated weights for policy 0, policy_version 1451743 (0.0006) [2023-12-27 15:42:20,992][20347] Updated weights for policy 0, policy_version 1451753 (0.0006) [2023-12-27 15:42:21,068][20347] Updated weights for policy 0, policy_version 1451763 (0.0007) [2023-12-27 15:42:21,429][20381] Updated weights for policy 1, policy_version 1450385 (0.0009) [2023-12-27 15:42:21,496][20381] Updated weights for policy 1, policy_version 1450395 (0.0008) [2023-12-27 15:42:21,553][20381] Updated weights for policy 1, policy_version 1450405 (0.0008) [2023-12-27 15:42:21,611][20381] Updated weights for policy 1, policy_version 1450415 (0.0007) [2023-12-27 15:42:21,755][20347] Updated weights for policy 0, policy_version 1451773 (0.0008) [2023-12-27 15:42:21,809][20347] Updated weights for policy 0, policy_version 1451783 (0.0008) [2023-12-27 15:42:21,871][20347] Updated weights for policy 0, policy_version 1451793 (0.0007) [2023-12-27 15:42:22,286][20381] Updated weights for policy 1, policy_version 1450425 (0.0010) [2023-12-27 15:42:22,347][20381] Updated weights for policy 1, policy_version 1450435 (0.0011) [2023-12-27 15:42:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 743071744. Throughput: 0: 10265.4, 1: 10141.6. Samples: 743046904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:22,363][19170] Avg episode reward: [(0, '371.140'), (1, '536.910')] [2023-12-27 15:42:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001451800_371736576.pth... [2023-12-27 15:42:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001450632_371433472.pth [2023-12-27 15:42:22,411][20381] Updated weights for policy 1, policy_version 1450445 (0.0008) [2023-12-27 15:42:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001450448_371343360.pth... [2023-12-27 15:42:22,432][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001449264_371040256.pth [2023-12-27 15:42:22,604][20347] Updated weights for policy 0, policy_version 1451803 (0.0007) [2023-12-27 15:42:22,670][20347] Updated weights for policy 0, policy_version 1451813 (0.0007) [2023-12-27 15:42:22,735][20347] Updated weights for policy 0, policy_version 1451823 (0.0007) [2023-12-27 15:42:23,113][20381] Updated weights for policy 1, policy_version 1450455 (0.0008) [2023-12-27 15:42:23,167][20381] Updated weights for policy 1, policy_version 1450465 (0.0008) [2023-12-27 15:42:23,219][20381] Updated weights for policy 1, policy_version 1450475 (0.0007) [2023-12-27 15:42:23,451][20347] Updated weights for policy 0, policy_version 1451833 (0.0007) [2023-12-27 15:42:23,515][20347] Updated weights for policy 0, policy_version 1451843 (0.0006) [2023-12-27 15:42:23,572][20347] Updated weights for policy 0, policy_version 1451853 (0.0006) [2023-12-27 15:42:23,637][20347] Updated weights for policy 0, policy_version 1451863 (0.0005) [2023-12-27 15:42:23,894][20381] Updated weights for policy 1, policy_version 1450485 (0.0007) [2023-12-27 15:42:23,955][20381] Updated weights for policy 1, policy_version 1450495 (0.0005) [2023-12-27 15:42:24,010][20381] Updated weights for policy 1, policy_version 1450505 (0.0005) [2023-12-27 15:42:24,221][20347] Updated weights for policy 0, policy_version 1451873 (0.0008) [2023-12-27 15:42:24,279][20347] Updated weights for policy 0, policy_version 1451883 (0.0007) [2023-12-27 15:42:24,329][20347] Updated weights for policy 0, policy_version 1451893 (0.0009) [2023-12-27 15:42:24,592][20381] Updated weights for policy 1, policy_version 1450515 (0.0006) [2023-12-27 15:42:24,648][20381] Updated weights for policy 1, policy_version 1450525 (0.0008) [2023-12-27 15:42:24,700][20381] Updated weights for policy 1, policy_version 1450535 (0.0008) [2023-12-27 15:42:25,037][20347] Updated weights for policy 0, policy_version 1451903 (0.0008) [2023-12-27 15:42:25,090][20347] Updated weights for policy 0, policy_version 1451913 (0.0006) [2023-12-27 15:42:25,146][20347] Updated weights for policy 0, policy_version 1451923 (0.0008) [2023-12-27 15:42:25,372][20381] Updated weights for policy 1, policy_version 1450545 (0.0007) [2023-12-27 15:42:25,429][20381] Updated weights for policy 1, policy_version 1450555 (0.0008) [2023-12-27 15:42:25,476][20381] Updated weights for policy 1, policy_version 1450565 (0.0007) [2023-12-27 15:42:25,521][20381] Updated weights for policy 1, policy_version 1450575 (0.0007) [2023-12-27 15:42:25,775][20347] Updated weights for policy 0, policy_version 1451933 (0.0007) [2023-12-27 15:42:25,826][20347] Updated weights for policy 0, policy_version 1451943 (0.0006) [2023-12-27 15:42:25,874][20347] Updated weights for policy 0, policy_version 1451953 (0.0005) [2023-12-27 15:42:26,208][20381] Updated weights for policy 1, policy_version 1450585 (0.0011) [2023-12-27 15:42:26,261][20381] Updated weights for policy 1, policy_version 1450595 (0.0010) [2023-12-27 15:42:26,315][20381] Updated weights for policy 1, policy_version 1450605 (0.0009) [2023-12-27 15:42:26,554][20347] Updated weights for policy 0, policy_version 1451963 (0.0007) [2023-12-27 15:42:26,605][20347] Updated weights for policy 0, policy_version 1451973 (0.0007) [2023-12-27 15:42:26,654][20347] Updated weights for policy 0, policy_version 1451983 (0.0008) [2023-12-27 15:42:26,881][20381] Updated weights for policy 1, policy_version 1450615 (0.0007) [2023-12-27 15:42:26,947][20381] Updated weights for policy 1, policy_version 1450625 (0.0008) [2023-12-27 15:42:27,004][20381] Updated weights for policy 1, policy_version 1450635 (0.0010) [2023-12-27 15:42:27,317][20347] Updated weights for policy 0, policy_version 1451993 (0.0008) [2023-12-27 15:42:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 743178240. Throughput: 0: 10306.8, 1: 10191.0. Samples: 743170064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:27,363][19170] Avg episode reward: [(0, '462.700'), (1, '527.710')] [2023-12-27 15:42:27,379][20347] Updated weights for policy 0, policy_version 1452003 (0.0007) [2023-12-27 15:42:27,434][20347] Updated weights for policy 0, policy_version 1452013 (0.0008) [2023-12-27 15:42:27,491][20347] Updated weights for policy 0, policy_version 1452023 (0.0007) [2023-12-27 15:42:27,633][20381] Updated weights for policy 1, policy_version 1450645 (0.0009) [2023-12-27 15:42:27,694][20381] Updated weights for policy 1, policy_version 1450655 (0.0010) [2023-12-27 15:42:27,750][20381] Updated weights for policy 1, policy_version 1450665 (0.0009) [2023-12-27 15:42:28,161][20347] Updated weights for policy 0, policy_version 1452033 (0.0008) [2023-12-27 15:42:28,216][20347] Updated weights for policy 0, policy_version 1452043 (0.0008) [2023-12-27 15:42:28,267][20347] Updated weights for policy 0, policy_version 1452053 (0.0006) [2023-12-27 15:42:28,378][20381] Updated weights for policy 1, policy_version 1450675 (0.0007) [2023-12-27 15:42:28,433][20381] Updated weights for policy 1, policy_version 1450685 (0.0006) [2023-12-27 15:42:28,489][20381] Updated weights for policy 1, policy_version 1450695 (0.0006) [2023-12-27 15:42:29,023][20347] Updated weights for policy 0, policy_version 1452063 (0.0007) [2023-12-27 15:42:29,086][20347] Updated weights for policy 0, policy_version 1452073 (0.0008) [2023-12-27 15:42:29,093][20381] Updated weights for policy 1, policy_version 1450705 (0.0011) [2023-12-27 15:42:29,143][20347] Updated weights for policy 0, policy_version 1452083 (0.0010) [2023-12-27 15:42:29,149][20381] Updated weights for policy 1, policy_version 1450715 (0.0011) [2023-12-27 15:42:29,199][20381] Updated weights for policy 1, policy_version 1450725 (0.0010) [2023-12-27 15:42:29,262][20381] Updated weights for policy 1, policy_version 1450735 (0.0010) [2023-12-27 15:42:29,877][20347] Updated weights for policy 0, policy_version 1452093 (0.0006) [2023-12-27 15:42:29,943][20381] Updated weights for policy 1, policy_version 1450745 (0.0008) [2023-12-27 15:42:29,946][20347] Updated weights for policy 0, policy_version 1452103 (0.0007) [2023-12-27 15:42:30,008][20381] Updated weights for policy 1, policy_version 1450755 (0.0008) [2023-12-27 15:42:30,010][20347] Updated weights for policy 0, policy_version 1452113 (0.0007) [2023-12-27 15:42:30,063][20381] Updated weights for policy 1, policy_version 1450765 (0.0008) [2023-12-27 15:42:30,696][20347] Updated weights for policy 0, policy_version 1452123 (0.0007) [2023-12-27 15:42:30,749][20347] Updated weights for policy 0, policy_version 1452133 (0.0008) [2023-12-27 15:42:30,759][20381] Updated weights for policy 1, policy_version 1450775 (0.0008) [2023-12-27 15:42:30,800][20347] Updated weights for policy 0, policy_version 1452143 (0.0005) [2023-12-27 15:42:30,816][20381] Updated weights for policy 1, policy_version 1450785 (0.0010) [2023-12-27 15:42:30,863][20381] Updated weights for policy 1, policy_version 1450795 (0.0010) [2023-12-27 15:42:31,505][20347] Updated weights for policy 0, policy_version 1452153 (0.0008) [2023-12-27 15:42:31,563][20381] Updated weights for policy 1, policy_version 1450805 (0.0008) [2023-12-27 15:42:31,564][20347] Updated weights for policy 0, policy_version 1452163 (0.0007) [2023-12-27 15:42:31,619][20347] Updated weights for policy 0, policy_version 1452173 (0.0006) [2023-12-27 15:42:31,626][20381] Updated weights for policy 1, policy_version 1450815 (0.0007) [2023-12-27 15:42:31,680][20347] Updated weights for policy 0, policy_version 1452183 (0.0008) [2023-12-27 15:42:31,685][20381] Updated weights for policy 1, policy_version 1450825 (0.0009) [2023-12-27 15:42:32,283][20381] Updated weights for policy 1, policy_version 1450835 (0.0008) [2023-12-27 15:42:32,348][20381] Updated weights for policy 1, policy_version 1450845 (0.0007) [2023-12-27 15:42:32,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 743276544. Throughput: 0: 10239.4, 1: 10244.9. Samples: 743293776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:32,363][19170] Avg episode reward: [(0, '442.470'), (1, '606.920')] [2023-12-27 15:42:32,389][20347] Updated weights for policy 0, policy_version 1452193 (0.0007) [2023-12-27 15:42:32,417][20381] Updated weights for policy 1, policy_version 1450855 (0.0007) [2023-12-27 15:42:32,452][20347] Updated weights for policy 0, policy_version 1452203 (0.0006) [2023-12-27 15:42:32,511][20347] Updated weights for policy 0, policy_version 1452213 (0.0006) [2023-12-27 15:42:33,030][20381] Updated weights for policy 1, policy_version 1450865 (0.0007) [2023-12-27 15:42:33,086][20381] Updated weights for policy 1, policy_version 1450875 (0.0008) [2023-12-27 15:42:33,091][20347] Updated weights for policy 0, policy_version 1452223 (0.0005) [2023-12-27 15:42:33,141][20347] Updated weights for policy 0, policy_version 1452233 (0.0005) [2023-12-27 15:42:33,144][20381] Updated weights for policy 1, policy_version 1450885 (0.0008) [2023-12-27 15:42:33,189][20381] Updated weights for policy 1, policy_version 1450895 (0.0008) [2023-12-27 15:42:33,192][20347] Updated weights for policy 0, policy_version 1452243 (0.0005) [2023-12-27 15:42:33,833][20347] Updated weights for policy 0, policy_version 1452253 (0.0005) [2023-12-27 15:42:33,886][20381] Updated weights for policy 1, policy_version 1450905 (0.0008) [2023-12-27 15:42:33,894][20347] Updated weights for policy 0, policy_version 1452263 (0.0006) [2023-12-27 15:42:33,939][20381] Updated weights for policy 1, policy_version 1450915 (0.0007) [2023-12-27 15:42:33,951][20347] Updated weights for policy 0, policy_version 1452273 (0.0005) [2023-12-27 15:42:33,992][20381] Updated weights for policy 1, policy_version 1450925 (0.0005) [2023-12-27 15:42:34,546][20347] Updated weights for policy 0, policy_version 1452283 (0.0006) [2023-12-27 15:42:34,607][20347] Updated weights for policy 0, policy_version 1452293 (0.0006) [2023-12-27 15:42:34,663][20347] Updated weights for policy 0, policy_version 1452303 (0.0005) [2023-12-27 15:42:34,733][20381] Updated weights for policy 1, policy_version 1450935 (0.0007) [2023-12-27 15:42:34,778][20381] Updated weights for policy 1, policy_version 1450945 (0.0008) [2023-12-27 15:42:34,841][20381] Updated weights for policy 1, policy_version 1450955 (0.0008) [2023-12-27 15:42:34,867][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 15:42:35,193][20347] Updated weights for policy 0, policy_version 1452313 (0.0005) [2023-12-27 15:42:35,250][20347] Updated weights for policy 0, policy_version 1452323 (0.0005) [2023-12-27 15:42:35,295][20347] Updated weights for policy 0, policy_version 1452333 (0.0007) [2023-12-27 15:42:35,346][20347] Updated weights for policy 0, policy_version 1452343 (0.0008) [2023-12-27 15:42:35,589][20381] Updated weights for policy 1, policy_version 1450965 (0.0007) [2023-12-27 15:42:35,636][20381] Updated weights for policy 1, policy_version 1450975 (0.0005) [2023-12-27 15:42:35,681][20381] Updated weights for policy 1, policy_version 1450985 (0.0005) [2023-12-27 15:42:35,953][20347] Updated weights for policy 0, policy_version 1452353 (0.0008) [2023-12-27 15:42:36,005][20347] Updated weights for policy 0, policy_version 1452363 (0.0008) [2023-12-27 15:42:36,046][20347] Updated weights for policy 0, policy_version 1452373 (0.0006) [2023-12-27 15:42:36,370][20381] Updated weights for policy 1, policy_version 1450995 (0.0006) [2023-12-27 15:42:36,436][20381] Updated weights for policy 1, policy_version 1451005 (0.0008) [2023-12-27 15:42:36,495][20381] Updated weights for policy 1, policy_version 1451015 (0.0007) [2023-12-27 15:42:36,783][20347] Updated weights for policy 0, policy_version 1452383 (0.0006) [2023-12-27 15:42:36,828][20347] Updated weights for policy 0, policy_version 1452393 (0.0005) [2023-12-27 15:42:36,878][20347] Updated weights for policy 0, policy_version 1452403 (0.0005) [2023-12-27 15:42:37,068][20381] Updated weights for policy 1, policy_version 1451025 (0.0008) [2023-12-27 15:42:37,124][20381] Updated weights for policy 1, policy_version 1451035 (0.0005) [2023-12-27 15:42:37,181][20381] Updated weights for policy 1, policy_version 1451045 (0.0005) [2023-12-27 15:42:37,242][20381] Updated weights for policy 1, policy_version 1451055 (0.0005) [2023-12-27 15:42:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 743391232. Throughput: 0: 10304.5, 1: 10174.6. Samples: 743357420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:37,363][19170] Avg episode reward: [(0, '513.450'), (1, '710.000')] [2023-12-27 15:42:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001451056_371499008.pth... [2023-12-27 15:42:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001452408_371892224.pth... [2023-12-27 15:42:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001449840_371187712.pth [2023-12-27 15:42:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001451240_371589120.pth [2023-12-27 15:42:37,523][20347] Updated weights for policy 0, policy_version 1452413 (0.0006) [2023-12-27 15:42:37,580][20347] Updated weights for policy 0, policy_version 1452423 (0.0005) [2023-12-27 15:42:37,639][20347] Updated weights for policy 0, policy_version 1452433 (0.0007) [2023-12-27 15:42:37,759][20381] Updated weights for policy 1, policy_version 1451065 (0.0007) [2023-12-27 15:42:37,812][20381] Updated weights for policy 1, policy_version 1451075 (0.0008) [2023-12-27 15:42:37,870][20381] Updated weights for policy 1, policy_version 1451085 (0.0008) [2023-12-27 15:42:38,184][20347] Updated weights for policy 0, policy_version 1452443 (0.0007) [2023-12-27 15:42:38,237][20347] Updated weights for policy 0, policy_version 1452453 (0.0005) [2023-12-27 15:42:38,290][20347] Updated weights for policy 0, policy_version 1452463 (0.0005) [2023-12-27 15:42:38,607][20381] Updated weights for policy 1, policy_version 1451095 (0.0008) [2023-12-27 15:42:38,664][20381] Updated weights for policy 1, policy_version 1451105 (0.0008) [2023-12-27 15:42:38,716][20381] Updated weights for policy 1, policy_version 1451115 (0.0008) [2023-12-27 15:42:38,948][20347] Updated weights for policy 0, policy_version 1452473 (0.0007) [2023-12-27 15:42:39,006][20347] Updated weights for policy 0, policy_version 1452483 (0.0005) [2023-12-27 15:42:39,066][20347] Updated weights for policy 0, policy_version 1452493 (0.0007) [2023-12-27 15:42:39,112][20347] Updated weights for policy 0, policy_version 1452503 (0.0010) [2023-12-27 15:42:39,339][20381] Updated weights for policy 1, policy_version 1451125 (0.0007) [2023-12-27 15:42:39,413][20381] Updated weights for policy 1, policy_version 1451135 (0.0008) [2023-12-27 15:42:39,481][20381] Updated weights for policy 1, policy_version 1451145 (0.0007) [2023-12-27 15:42:39,908][20347] Updated weights for policy 0, policy_version 1452513 (0.0007) [2023-12-27 15:42:39,970][20347] Updated weights for policy 0, policy_version 1452523 (0.0007) [2023-12-27 15:42:40,030][20347] Updated weights for policy 0, policy_version 1452533 (0.0006) [2023-12-27 15:42:40,159][20381] Updated weights for policy 1, policy_version 1451155 (0.0008) [2023-12-27 15:42:40,219][20381] Updated weights for policy 1, policy_version 1451165 (0.0009) [2023-12-27 15:42:40,272][20381] Updated weights for policy 1, policy_version 1451175 (0.0008) [2023-12-27 15:42:40,651][20347] Updated weights for policy 0, policy_version 1452543 (0.0007) [2023-12-27 15:42:40,699][20347] Updated weights for policy 0, policy_version 1452553 (0.0007) [2023-12-27 15:42:40,749][20347] Updated weights for policy 0, policy_version 1452563 (0.0007) [2023-12-27 15:42:40,948][20381] Updated weights for policy 1, policy_version 1451185 (0.0007) [2023-12-27 15:42:41,015][20381] Updated weights for policy 1, policy_version 1451195 (0.0007) [2023-12-27 15:42:41,079][20381] Updated weights for policy 1, policy_version 1451205 (0.0010) [2023-12-27 15:42:41,145][20381] Updated weights for policy 1, policy_version 1451215 (0.0008) [2023-12-27 15:42:41,417][20347] Updated weights for policy 0, policy_version 1452573 (0.0007) [2023-12-27 15:42:41,470][20347] Updated weights for policy 0, policy_version 1452583 (0.0006) [2023-12-27 15:42:41,533][20347] Updated weights for policy 0, policy_version 1452593 (0.0007) [2023-12-27 15:42:41,907][20381] Updated weights for policy 1, policy_version 1451225 (0.0008) [2023-12-27 15:42:41,969][20381] Updated weights for policy 1, policy_version 1451235 (0.0007) [2023-12-27 15:42:42,041][20381] Updated weights for policy 1, policy_version 1451245 (0.0007) [2023-12-27 15:42:42,288][20347] Updated weights for policy 0, policy_version 1452603 (0.0007) [2023-12-27 15:42:42,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 743489536. Throughput: 0: 10304.1, 1: 10214.0. Samples: 743483672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:42,363][19170] Avg episode reward: [(0, '529.340'), (1, '659.430')] [2023-12-27 15:42:42,368][20347] Updated weights for policy 0, policy_version 1452613 (0.0006) [2023-12-27 15:42:42,426][20347] Updated weights for policy 0, policy_version 1452623 (0.0007) [2023-12-27 15:42:42,726][20381] Updated weights for policy 1, policy_version 1451255 (0.0008) [2023-12-27 15:42:42,778][20381] Updated weights for policy 1, policy_version 1451265 (0.0008) [2023-12-27 15:42:42,837][20381] Updated weights for policy 1, policy_version 1451275 (0.0007) [2023-12-27 15:42:43,036][20347] Updated weights for policy 0, policy_version 1452633 (0.0010) [2023-12-27 15:42:43,099][20347] Updated weights for policy 0, policy_version 1452643 (0.0005) [2023-12-27 15:42:43,149][20347] Updated weights for policy 0, policy_version 1452653 (0.0005) [2023-12-27 15:42:43,206][20347] Updated weights for policy 0, policy_version 1452663 (0.0005) [2023-12-27 15:42:43,527][20381] Updated weights for policy 1, policy_version 1451285 (0.0011) [2023-12-27 15:42:43,582][20381] Updated weights for policy 1, policy_version 1451295 (0.0008) [2023-12-27 15:42:43,635][20381] Updated weights for policy 1, policy_version 1451305 (0.0005) [2023-12-27 15:42:43,786][20347] Updated weights for policy 0, policy_version 1452673 (0.0005) [2023-12-27 15:42:43,830][20291] KL-divergence is very high: 108.5294 [2023-12-27 15:42:43,837][20347] Updated weights for policy 0, policy_version 1452683 (0.0006) [2023-12-27 15:42:43,887][20347] Updated weights for policy 0, policy_version 1452693 (0.0006) [2023-12-27 15:42:44,213][20381] Updated weights for policy 1, policy_version 1451315 (0.0005) [2023-12-27 15:42:44,272][20381] Updated weights for policy 1, policy_version 1451325 (0.0006) [2023-12-27 15:42:44,327][20381] Updated weights for policy 1, policy_version 1451335 (0.0006) [2023-12-27 15:42:44,543][20347] Updated weights for policy 0, policy_version 1452703 (0.0007) [2023-12-27 15:42:44,598][20347] Updated weights for policy 0, policy_version 1452713 (0.0008) [2023-12-27 15:42:44,661][20347] Updated weights for policy 0, policy_version 1452723 (0.0008) [2023-12-27 15:42:45,020][20381] Updated weights for policy 1, policy_version 1451347 (0.0008) [2023-12-27 15:42:45,074][20381] Updated weights for policy 1, policy_version 1451357 (0.0009) [2023-12-27 15:42:45,132][20381] Updated weights for policy 1, policy_version 1451367 (0.0008) [2023-12-27 15:42:45,328][20347] Updated weights for policy 0, policy_version 1452733 (0.0008) [2023-12-27 15:42:45,389][20347] Updated weights for policy 0, policy_version 1452743 (0.0008) [2023-12-27 15:42:45,442][20347] Updated weights for policy 0, policy_version 1452753 (0.0007) [2023-12-27 15:42:45,796][20381] Updated weights for policy 1, policy_version 1451377 (0.0008) [2023-12-27 15:42:45,849][20381] Updated weights for policy 1, policy_version 1451387 (0.0008) [2023-12-27 15:42:45,899][20381] Updated weights for policy 1, policy_version 1451397 (0.0005) [2023-12-27 15:42:45,952][20381] Updated weights for policy 1, policy_version 1451407 (0.0005) [2023-12-27 15:42:46,098][20347] Updated weights for policy 0, policy_version 1452763 (0.0005) [2023-12-27 15:42:46,156][20347] Updated weights for policy 0, policy_version 1452773 (0.0008) [2023-12-27 15:42:46,207][20347] Updated weights for policy 0, policy_version 1452783 (0.0010) [2023-12-27 15:42:46,635][20381] Updated weights for policy 1, policy_version 1451417 (0.0007) [2023-12-27 15:42:46,701][20381] Updated weights for policy 1, policy_version 1451427 (0.0008) [2023-12-27 15:42:46,761][20381] Updated weights for policy 1, policy_version 1451437 (0.0008) [2023-12-27 15:42:46,800][20347] Updated weights for policy 0, policy_version 1452793 (0.0009) [2023-12-27 15:42:46,846][20347] Updated weights for policy 0, policy_version 1452803 (0.0005) [2023-12-27 15:42:46,892][20347] Updated weights for policy 0, policy_version 1452813 (0.0005) [2023-12-27 15:42:46,945][20347] Updated weights for policy 0, policy_version 1452823 (0.0005) [2023-12-27 15:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 743596032. Throughput: 0: 10372.6, 1: 10264.5. Samples: 743608624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:47,363][19170] Avg episode reward: [(0, '462.510'), (1, '661.400')] [2023-12-27 15:42:47,493][20347] Updated weights for policy 0, policy_version 1452833 (0.0005) [2023-12-27 15:42:47,503][20381] Updated weights for policy 1, policy_version 1451447 (0.0007) [2023-12-27 15:42:47,552][20347] Updated weights for policy 0, policy_version 1452843 (0.0005) [2023-12-27 15:42:47,559][20381] Updated weights for policy 1, policy_version 1451457 (0.0008) [2023-12-27 15:42:47,605][20347] Updated weights for policy 0, policy_version 1452853 (0.0005) [2023-12-27 15:42:47,613][20381] Updated weights for policy 1, policy_version 1451467 (0.0008) [2023-12-27 15:42:48,141][20347] Updated weights for policy 0, policy_version 1452863 (0.0005) [2023-12-27 15:42:48,194][20347] Updated weights for policy 0, policy_version 1452873 (0.0006) [2023-12-27 15:42:48,242][20347] Updated weights for policy 0, policy_version 1452883 (0.0010) [2023-12-27 15:42:48,355][20381] Updated weights for policy 1, policy_version 1451477 (0.0007) [2023-12-27 15:42:48,414][20381] Updated weights for policy 1, policy_version 1451487 (0.0007) [2023-12-27 15:42:48,475][20381] Updated weights for policy 1, policy_version 1451497 (0.0007) [2023-12-27 15:42:48,948][20347] Updated weights for policy 0, policy_version 1452893 (0.0010) [2023-12-27 15:42:49,003][20347] Updated weights for policy 0, policy_version 1452903 (0.0010) [2023-12-27 15:42:49,064][20347] Updated weights for policy 0, policy_version 1452913 (0.0010) [2023-12-27 15:42:49,076][20381] Updated weights for policy 1, policy_version 1451507 (0.0007) [2023-12-27 15:42:49,127][20381] Updated weights for policy 1, policy_version 1451517 (0.0005) [2023-12-27 15:42:49,184][20381] Updated weights for policy 1, policy_version 1451527 (0.0005) [2023-12-27 15:42:49,783][20347] Updated weights for policy 0, policy_version 1452923 (0.0010) [2023-12-27 15:42:49,840][20381] Updated weights for policy 1, policy_version 1451537 (0.0007) [2023-12-27 15:42:49,851][20347] Updated weights for policy 0, policy_version 1452933 (0.0010) [2023-12-27 15:42:49,902][20381] Updated weights for policy 1, policy_version 1451547 (0.0006) [2023-12-27 15:42:49,912][20347] Updated weights for policy 0, policy_version 1452943 (0.0007) [2023-12-27 15:42:49,967][20381] Updated weights for policy 1, policy_version 1451557 (0.0008) [2023-12-27 15:42:50,026][20381] Updated weights for policy 1, policy_version 1451567 (0.0008) [2023-12-27 15:42:50,606][20347] Updated weights for policy 0, policy_version 1452953 (0.0009) [2023-12-27 15:42:50,675][20347] Updated weights for policy 0, policy_version 1452963 (0.0008) [2023-12-27 15:42:50,737][20347] Updated weights for policy 0, policy_version 1452973 (0.0008) [2023-12-27 15:42:50,774][20381] Updated weights for policy 1, policy_version 1451577 (0.0008) [2023-12-27 15:42:50,797][20347] Updated weights for policy 0, policy_version 1452983 (0.0008) [2023-12-27 15:42:50,836][20381] Updated weights for policy 1, policy_version 1451587 (0.0007) [2023-12-27 15:42:50,903][20381] Updated weights for policy 1, policy_version 1451597 (0.0010) [2023-12-27 15:42:51,549][20347] Updated weights for policy 0, policy_version 1452993 (0.0006) [2023-12-27 15:42:51,562][20381] Updated weights for policy 1, policy_version 1451607 (0.0008) [2023-12-27 15:42:51,617][20347] Updated weights for policy 0, policy_version 1453003 (0.0006) [2023-12-27 15:42:51,618][20381] Updated weights for policy 1, policy_version 1451617 (0.0008) [2023-12-27 15:42:51,680][20381] Updated weights for policy 1, policy_version 1451627 (0.0007) [2023-12-27 15:42:51,683][20347] Updated weights for policy 0, policy_version 1453013 (0.0007) [2023-12-27 15:42:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 743694336. Throughput: 0: 10391.0, 1: 10251.1. Samples: 743671508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:52,363][19170] Avg episode reward: [(0, '511.220'), (1, '711.650')] [2023-12-27 15:42:52,387][20381] Updated weights for policy 1, policy_version 1451637 (0.0007) [2023-12-27 15:42:52,388][20347] Updated weights for policy 0, policy_version 1453023 (0.0007) [2023-12-27 15:42:52,444][20381] Updated weights for policy 1, policy_version 1451647 (0.0008) [2023-12-27 15:42:52,453][20347] Updated weights for policy 0, policy_version 1453033 (0.0007) [2023-12-27 15:42:52,498][20381] Updated weights for policy 1, policy_version 1451657 (0.0006) [2023-12-27 15:42:52,512][20347] Updated weights for policy 0, policy_version 1453043 (0.0006) [2023-12-27 15:42:52,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001451664_371654656.pth... [2023-12-27 15:42:52,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001450448_371343360.pth [2023-12-27 15:42:52,541][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001453048_372056064.pth... [2023-12-27 15:42:52,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001451800_371736576.pth [2023-12-27 15:42:53,164][20381] Updated weights for policy 1, policy_version 1451667 (0.0006) [2023-12-27 15:42:53,205][20347] Updated weights for policy 0, policy_version 1453053 (0.0006) [2023-12-27 15:42:53,217][20381] Updated weights for policy 1, policy_version 1451677 (0.0009) [2023-12-27 15:42:53,252][20347] Updated weights for policy 0, policy_version 1453063 (0.0010) [2023-12-27 15:42:53,262][20381] Updated weights for policy 1, policy_version 1451687 (0.0008) [2023-12-27 15:42:53,301][20347] Updated weights for policy 0, policy_version 1453073 (0.0010) [2023-12-27 15:42:53,873][20381] Updated weights for policy 1, policy_version 1451697 (0.0005) [2023-12-27 15:42:53,921][20381] Updated weights for policy 1, policy_version 1451707 (0.0007) [2023-12-27 15:42:53,932][20347] Updated weights for policy 0, policy_version 1453083 (0.0010) [2023-12-27 15:42:53,977][20381] Updated weights for policy 1, policy_version 1451717 (0.0007) [2023-12-27 15:42:53,986][20347] Updated weights for policy 0, policy_version 1453093 (0.0005) [2023-12-27 15:42:54,031][20381] Updated weights for policy 1, policy_version 1451727 (0.0007) [2023-12-27 15:42:54,047][20347] Updated weights for policy 0, policy_version 1453103 (0.0005) [2023-12-27 15:42:54,659][20381] Updated weights for policy 1, policy_version 1451737 (0.0007) [2023-12-27 15:42:54,666][20347] Updated weights for policy 0, policy_version 1453113 (0.0006) [2023-12-27 15:42:54,718][20381] Updated weights for policy 1, policy_version 1451747 (0.0009) [2023-12-27 15:42:54,723][20347] Updated weights for policy 0, policy_version 1453123 (0.0011) [2023-12-27 15:42:54,775][20381] Updated weights for policy 1, policy_version 1451757 (0.0006) [2023-12-27 15:42:54,780][20347] Updated weights for policy 0, policy_version 1453133 (0.0010) [2023-12-27 15:42:54,834][20347] Updated weights for policy 0, policy_version 1453143 (0.0010) [2023-12-27 15:42:55,482][20381] Updated weights for policy 1, policy_version 1451767 (0.0007) [2023-12-27 15:42:55,486][20347] Updated weights for policy 0, policy_version 1453153 (0.0009) [2023-12-27 15:42:55,535][20381] Updated weights for policy 1, policy_version 1451777 (0.0007) [2023-12-27 15:42:55,541][20347] Updated weights for policy 0, policy_version 1453163 (0.0008) [2023-12-27 15:42:55,598][20381] Updated weights for policy 1, policy_version 1451787 (0.0008) [2023-12-27 15:42:55,600][20347] Updated weights for policy 0, policy_version 1453173 (0.0007) [2023-12-27 15:42:56,261][20347] Updated weights for policy 0, policy_version 1453183 (0.0009) [2023-12-27 15:42:56,278][20381] Updated weights for policy 1, policy_version 1451797 (0.0007) [2023-12-27 15:42:56,318][20347] Updated weights for policy 0, policy_version 1453193 (0.0010) [2023-12-27 15:42:56,338][20381] Updated weights for policy 1, policy_version 1451807 (0.0005) [2023-12-27 15:42:56,377][20347] Updated weights for policy 0, policy_version 1453203 (0.0010) [2023-12-27 15:42:56,402][20381] Updated weights for policy 1, policy_version 1451817 (0.0006) [2023-12-27 15:42:57,025][20347] Updated weights for policy 0, policy_version 1453213 (0.0010) [2023-12-27 15:42:57,059][20381] Updated weights for policy 1, policy_version 1451827 (0.0009) [2023-12-27 15:42:57,085][20347] Updated weights for policy 0, policy_version 1453223 (0.0010) [2023-12-27 15:42:57,120][20381] Updated weights for policy 1, policy_version 1451837 (0.0009) [2023-12-27 15:42:57,142][20347] Updated weights for policy 0, policy_version 1453233 (0.0011) [2023-12-27 15:42:57,177][20381] Updated weights for policy 1, policy_version 1451847 (0.0006) [2023-12-27 15:42:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 743809024. Throughput: 0: 10393.7, 1: 10268.4. Samples: 743794812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:42:57,363][19170] Avg episode reward: [(0, '608.600'), (1, '552.070')] [2023-12-27 15:42:57,797][20381] Updated weights for policy 1, policy_version 1451857 (0.0009) [2023-12-27 15:42:57,807][20347] Updated weights for policy 0, policy_version 1453243 (0.0010) [2023-12-27 15:42:57,850][20381] Updated weights for policy 1, policy_version 1451867 (0.0011) [2023-12-27 15:42:57,860][20347] Updated weights for policy 0, policy_version 1453253 (0.0010) [2023-12-27 15:42:57,901][20381] Updated weights for policy 1, policy_version 1451877 (0.0010) [2023-12-27 15:42:57,911][20347] Updated weights for policy 0, policy_version 1453263 (0.0010) [2023-12-27 15:42:57,957][20381] Updated weights for policy 1, policy_version 1451887 (0.0011) [2023-12-27 15:42:58,663][20347] Updated weights for policy 0, policy_version 1453273 (0.0010) [2023-12-27 15:42:58,704][20381] Updated weights for policy 1, policy_version 1451897 (0.0008) [2023-12-27 15:42:58,726][20347] Updated weights for policy 0, policy_version 1453283 (0.0009) [2023-12-27 15:42:58,770][20381] Updated weights for policy 1, policy_version 1451907 (0.0007) [2023-12-27 15:42:58,788][20347] Updated weights for policy 0, policy_version 1453293 (0.0009) [2023-12-27 15:42:58,839][20381] Updated weights for policy 1, policy_version 1451917 (0.0007) [2023-12-27 15:42:58,867][20347] Updated weights for policy 0, policy_version 1453303 (0.0008) [2023-12-27 15:42:59,560][20381] Updated weights for policy 1, policy_version 1451927 (0.0007) [2023-12-27 15:42:59,592][20347] Updated weights for policy 0, policy_version 1453313 (0.0006) [2023-12-27 15:42:59,618][20381] Updated weights for policy 1, policy_version 1451937 (0.0006) [2023-12-27 15:42:59,648][20347] Updated weights for policy 0, policy_version 1453323 (0.0007) [2023-12-27 15:42:59,669][20381] Updated weights for policy 1, policy_version 1451947 (0.0007) [2023-12-27 15:42:59,704][20347] Updated weights for policy 0, policy_version 1453333 (0.0005) [2023-12-27 15:43:00,321][20347] Updated weights for policy 0, policy_version 1453343 (0.0006) [2023-12-27 15:43:00,376][20347] Updated weights for policy 0, policy_version 1453353 (0.0007) [2023-12-27 15:43:00,414][20381] Updated weights for policy 1, policy_version 1451957 (0.0008) [2023-12-27 15:43:00,435][20347] Updated weights for policy 0, policy_version 1453363 (0.0005) [2023-12-27 15:43:00,466][20381] Updated weights for policy 1, policy_version 1451967 (0.0007) [2023-12-27 15:43:00,511][20381] Updated weights for policy 1, policy_version 1451977 (0.0008) [2023-12-27 15:43:01,014][20347] Updated weights for policy 0, policy_version 1453373 (0.0006) [2023-12-27 15:43:01,075][20347] Updated weights for policy 0, policy_version 1453383 (0.0006) [2023-12-27 15:43:01,141][20347] Updated weights for policy 0, policy_version 1453393 (0.0008) [2023-12-27 15:43:01,150][20381] Updated weights for policy 1, policy_version 1451987 (0.0008) [2023-12-27 15:43:01,207][20381] Updated weights for policy 1, policy_version 1451997 (0.0008) [2023-12-27 15:43:01,266][20381] Updated weights for policy 1, policy_version 1452007 (0.0007) [2023-12-27 15:43:01,794][20347] Updated weights for policy 0, policy_version 1453403 (0.0008) [2023-12-27 15:43:01,849][20347] Updated weights for policy 0, policy_version 1453413 (0.0007) [2023-12-27 15:43:01,909][20347] Updated weights for policy 0, policy_version 1453423 (0.0008) [2023-12-27 15:43:01,956][20381] Updated weights for policy 1, policy_version 1452017 (0.0008) [2023-12-27 15:43:02,015][20381] Updated weights for policy 1, policy_version 1452027 (0.0006) [2023-12-27 15:43:02,075][20381] Updated weights for policy 1, policy_version 1452037 (0.0007) [2023-12-27 15:43:02,140][20381] Updated weights for policy 1, policy_version 1452047 (0.0008) [2023-12-27 15:43:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.5). Total num frames: 743907328. Throughput: 0: 10409.3, 1: 10274.3. Samples: 743916104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:02,363][19170] Avg episode reward: [(0, '500.930'), (1, '548.360')] [2023-12-27 15:43:02,619][20347] Updated weights for policy 0, policy_version 1453433 (0.0007) [2023-12-27 15:43:02,674][20347] Updated weights for policy 0, policy_version 1453443 (0.0005) [2023-12-27 15:43:02,732][20347] Updated weights for policy 0, policy_version 1453453 (0.0005) [2023-12-27 15:43:02,786][20347] Updated weights for policy 0, policy_version 1453463 (0.0005) [2023-12-27 15:43:02,786][20381] Updated weights for policy 1, policy_version 1452057 (0.0009) [2023-12-27 15:43:02,851][20381] Updated weights for policy 1, policy_version 1452067 (0.0008) [2023-12-27 15:43:02,912][20381] Updated weights for policy 1, policy_version 1452077 (0.0008) [2023-12-27 15:43:03,378][20347] Updated weights for policy 0, policy_version 1453473 (0.0007) [2023-12-27 15:43:03,423][20347] Updated weights for policy 0, policy_version 1453483 (0.0007) [2023-12-27 15:43:03,470][20347] Updated weights for policy 0, policy_version 1453493 (0.0007) [2023-12-27 15:43:03,606][20381] Updated weights for policy 1, policy_version 1452087 (0.0006) [2023-12-27 15:43:03,654][20381] Updated weights for policy 1, policy_version 1452097 (0.0005) [2023-12-27 15:43:03,708][20381] Updated weights for policy 1, policy_version 1452107 (0.0005) [2023-12-27 15:43:04,172][20347] Updated weights for policy 0, policy_version 1453503 (0.0010) [2023-12-27 15:43:04,234][20347] Updated weights for policy 0, policy_version 1453513 (0.0011) [2023-12-27 15:43:04,289][20347] Updated weights for policy 0, policy_version 1453523 (0.0010) [2023-12-27 15:43:04,399][20381] Updated weights for policy 1, policy_version 1452117 (0.0008) [2023-12-27 15:43:04,452][20381] Updated weights for policy 1, policy_version 1452127 (0.0007) [2023-12-27 15:43:04,503][20381] Updated weights for policy 1, policy_version 1452137 (0.0007) [2023-12-27 15:43:04,987][20347] Updated weights for policy 0, policy_version 1453533 (0.0008) [2023-12-27 15:43:05,044][20347] Updated weights for policy 0, policy_version 1453543 (0.0007) [2023-12-27 15:43:05,100][20347] Updated weights for policy 0, policy_version 1453553 (0.0007) [2023-12-27 15:43:05,107][20381] Updated weights for policy 1, policy_version 1452147 (0.0006) [2023-12-27 15:43:05,166][20381] Updated weights for policy 1, policy_version 1452157 (0.0005) [2023-12-27 15:43:05,230][20381] Updated weights for policy 1, policy_version 1452167 (0.0006) [2023-12-27 15:43:05,669][20347] Updated weights for policy 0, policy_version 1453563 (0.0006) [2023-12-27 15:43:05,716][20347] Updated weights for policy 0, policy_version 1453573 (0.0007) [2023-12-27 15:43:05,764][20347] Updated weights for policy 0, policy_version 1453583 (0.0005) [2023-12-27 15:43:05,793][20381] Updated weights for policy 1, policy_version 1452177 (0.0006) [2023-12-27 15:43:05,844][20381] Updated weights for policy 1, policy_version 1452187 (0.0005) [2023-12-27 15:43:05,887][20381] Updated weights for policy 1, policy_version 1452197 (0.0005) [2023-12-27 15:43:05,941][20381] Updated weights for policy 1, policy_version 1452207 (0.0005) [2023-12-27 15:43:06,407][20347] Updated weights for policy 0, policy_version 1453593 (0.0006) [2023-12-27 15:43:06,477][20347] Updated weights for policy 0, policy_version 1453603 (0.0008) [2023-12-27 15:43:06,536][20347] Updated weights for policy 0, policy_version 1453613 (0.0008) [2023-12-27 15:43:06,591][20347] Updated weights for policy 0, policy_version 1453623 (0.0006) [2023-12-27 15:43:06,603][20381] Updated weights for policy 1, policy_version 1452217 (0.0007) [2023-12-27 15:43:06,655][20381] Updated weights for policy 1, policy_version 1452227 (0.0008) [2023-12-27 15:43:06,711][20381] Updated weights for policy 1, policy_version 1452237 (0.0008) [2023-12-27 15:43:07,233][20347] Updated weights for policy 0, policy_version 1453633 (0.0006) [2023-12-27 15:43:07,300][20347] Updated weights for policy 0, policy_version 1453643 (0.0005) [2023-12-27 15:43:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 744005632. Throughput: 0: 10412.5, 1: 10306.6. Samples: 743979268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:07,364][19170] Avg episode reward: [(0, '463.160'), (1, '546.600')] [2023-12-27 15:43:07,366][20347] Updated weights for policy 0, policy_version 1453653 (0.0005) [2023-12-27 15:43:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001452240_371802112.pth... [2023-12-27 15:43:07,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001451056_371499008.pth [2023-12-27 15:43:07,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001453656_372211712.pth... [2023-12-27 15:43:07,408][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001452408_371892224.pth [2023-12-27 15:43:07,445][20381] Updated weights for policy 1, policy_version 1452247 (0.0009) [2023-12-27 15:43:07,503][20381] Updated weights for policy 1, policy_version 1452257 (0.0008) [2023-12-27 15:43:07,565][20381] Updated weights for policy 1, policy_version 1452267 (0.0007) [2023-12-27 15:43:07,959][20347] Updated weights for policy 0, policy_version 1453663 (0.0007) [2023-12-27 15:43:08,006][20347] Updated weights for policy 0, policy_version 1453673 (0.0008) [2023-12-27 15:43:08,065][20347] Updated weights for policy 0, policy_version 1453683 (0.0007) [2023-12-27 15:43:08,265][20381] Updated weights for policy 1, policy_version 1452277 (0.0007) [2023-12-27 15:43:08,326][20381] Updated weights for policy 1, policy_version 1452287 (0.0008) [2023-12-27 15:43:08,387][20381] Updated weights for policy 1, policy_version 1452297 (0.0006) [2023-12-27 15:43:08,752][20347] Updated weights for policy 0, policy_version 1453693 (0.0007) [2023-12-27 15:43:08,810][20347] Updated weights for policy 0, policy_version 1453703 (0.0007) [2023-12-27 15:43:08,865][20347] Updated weights for policy 0, policy_version 1453713 (0.0007) [2023-12-27 15:43:09,051][20381] Updated weights for policy 1, policy_version 1452307 (0.0007) [2023-12-27 15:43:09,112][20381] Updated weights for policy 1, policy_version 1452317 (0.0010) [2023-12-27 15:43:09,166][20381] Updated weights for policy 1, policy_version 1452327 (0.0010) [2023-12-27 15:43:09,498][20347] Updated weights for policy 0, policy_version 1453723 (0.0007) [2023-12-27 15:43:09,561][20347] Updated weights for policy 0, policy_version 1453733 (0.0005) [2023-12-27 15:43:09,625][20347] Updated weights for policy 0, policy_version 1453743 (0.0006) [2023-12-27 15:43:09,884][20381] Updated weights for policy 1, policy_version 1452337 (0.0010) [2023-12-27 15:43:09,948][20381] Updated weights for policy 1, policy_version 1452347 (0.0007) [2023-12-27 15:43:10,012][20381] Updated weights for policy 1, policy_version 1452357 (0.0007) [2023-12-27 15:43:10,079][20381] Updated weights for policy 1, policy_version 1452367 (0.0007) [2023-12-27 15:43:10,259][20347] Updated weights for policy 0, policy_version 1453753 (0.0007) [2023-12-27 15:43:10,326][20347] Updated weights for policy 0, policy_version 1453763 (0.0010) [2023-12-27 15:43:10,393][20347] Updated weights for policy 0, policy_version 1453773 (0.0009) [2023-12-27 15:43:10,459][20347] Updated weights for policy 0, policy_version 1453783 (0.0008) [2023-12-27 15:43:10,699][20381] Updated weights for policy 1, policy_version 1452377 (0.0007) [2023-12-27 15:43:10,755][20381] Updated weights for policy 1, policy_version 1452387 (0.0007) [2023-12-27 15:43:10,810][20381] Updated weights for policy 1, policy_version 1452397 (0.0007) [2023-12-27 15:43:11,063][20347] Updated weights for policy 0, policy_version 1453793 (0.0010) [2023-12-27 15:43:11,120][20347] Updated weights for policy 0, policy_version 1453803 (0.0010) [2023-12-27 15:43:11,185][20347] Updated weights for policy 0, policy_version 1453813 (0.0009) [2023-12-27 15:43:11,534][20381] Updated weights for policy 1, policy_version 1452407 (0.0006) [2023-12-27 15:43:11,592][20381] Updated weights for policy 1, policy_version 1452417 (0.0006) [2023-12-27 15:43:11,660][20381] Updated weights for policy 1, policy_version 1452427 (0.0008) [2023-12-27 15:43:11,899][20347] Updated weights for policy 0, policy_version 1453823 (0.0006) [2023-12-27 15:43:11,963][20347] Updated weights for policy 0, policy_version 1453833 (0.0007) [2023-12-27 15:43:12,019][20347] Updated weights for policy 0, policy_version 1453843 (0.0007) [2023-12-27 15:43:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 744112128. Throughput: 0: 10495.6, 1: 10300.6. Samples: 744105888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:12,363][19170] Avg episode reward: [(0, '420.220'), (1, '476.460')] [2023-12-27 15:43:12,373][20381] Updated weights for policy 1, policy_version 1452437 (0.0008) [2023-12-27 15:43:12,423][20381] Updated weights for policy 1, policy_version 1452447 (0.0008) [2023-12-27 15:43:12,473][20381] Updated weights for policy 1, policy_version 1452457 (0.0008) [2023-12-27 15:43:12,714][20347] Updated weights for policy 0, policy_version 1453853 (0.0007) [2023-12-27 15:43:12,774][20347] Updated weights for policy 0, policy_version 1453863 (0.0008) [2023-12-27 15:43:12,823][20347] Updated weights for policy 0, policy_version 1453873 (0.0007) [2023-12-27 15:43:13,176][20381] Updated weights for policy 1, policy_version 1452467 (0.0008) [2023-12-27 15:43:13,221][20381] Updated weights for policy 1, policy_version 1452477 (0.0006) [2023-12-27 15:43:13,272][20381] Updated weights for policy 1, policy_version 1452487 (0.0006) [2023-12-27 15:43:13,478][20347] Updated weights for policy 0, policy_version 1453883 (0.0006) [2023-12-27 15:43:13,522][20347] Updated weights for policy 0, policy_version 1453893 (0.0007) [2023-12-27 15:43:13,575][20347] Updated weights for policy 0, policy_version 1453903 (0.0007) [2023-12-27 15:43:13,910][20381] Updated weights for policy 1, policy_version 1452497 (0.0007) [2023-12-27 15:43:13,975][20381] Updated weights for policy 1, policy_version 1452507 (0.0005) [2023-12-27 15:43:14,028][20381] Updated weights for policy 1, policy_version 1452517 (0.0005) [2023-12-27 15:43:14,079][20381] Updated weights for policy 1, policy_version 1452527 (0.0005) [2023-12-27 15:43:14,325][20347] Updated weights for policy 0, policy_version 1453913 (0.0008) [2023-12-27 15:43:14,381][20347] Updated weights for policy 0, policy_version 1453923 (0.0008) [2023-12-27 15:43:14,437][20347] Updated weights for policy 0, policy_version 1453933 (0.0007) [2023-12-27 15:43:14,495][20347] Updated weights for policy 0, policy_version 1453943 (0.0007) [2023-12-27 15:43:14,648][20381] Updated weights for policy 1, policy_version 1452537 (0.0008) [2023-12-27 15:43:14,709][20381] Updated weights for policy 1, policy_version 1452547 (0.0008) [2023-12-27 15:43:14,774][20381] Updated weights for policy 1, policy_version 1452557 (0.0007) [2023-12-27 15:43:15,210][20347] Updated weights for policy 0, policy_version 1453953 (0.0008) [2023-12-27 15:43:15,267][20347] Updated weights for policy 0, policy_version 1453963 (0.0005) [2023-12-27 15:43:15,326][20347] Updated weights for policy 0, policy_version 1453973 (0.0007) [2023-12-27 15:43:15,505][20381] Updated weights for policy 1, policy_version 1452567 (0.0007) [2023-12-27 15:43:15,564][20381] Updated weights for policy 1, policy_version 1452577 (0.0008) [2023-12-27 15:43:15,622][20381] Updated weights for policy 1, policy_version 1452587 (0.0008) [2023-12-27 15:43:15,975][20347] Updated weights for policy 0, policy_version 1453983 (0.0008) [2023-12-27 15:43:16,025][20347] Updated weights for policy 0, policy_version 1453994 (0.0008) [2023-12-27 15:43:16,074][20347] Updated weights for policy 0, policy_version 1454004 (0.0007) [2023-12-27 15:43:16,326][20381] Updated weights for policy 1, policy_version 1452597 (0.0006) [2023-12-27 15:43:16,386][20381] Updated weights for policy 1, policy_version 1452607 (0.0005) [2023-12-27 15:43:16,448][20381] Updated weights for policy 1, policy_version 1452617 (0.0005) [2023-12-27 15:43:16,696][20347] Updated weights for policy 0, policy_version 1454014 (0.0007) [2023-12-27 15:43:16,752][20347] Updated weights for policy 0, policy_version 1454024 (0.0007) [2023-12-27 15:43:16,804][20347] Updated weights for policy 0, policy_version 1454034 (0.0009) [2023-12-27 15:43:17,028][20381] Updated weights for policy 1, policy_version 1452627 (0.0007) [2023-12-27 15:43:17,080][20381] Updated weights for policy 1, policy_version 1452637 (0.0009) [2023-12-27 15:43:17,138][20381] Updated weights for policy 1, policy_version 1452647 (0.0007) [2023-12-27 15:43:17,362][19170] Fps is (10 sec: 21300.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 744218624. Throughput: 0: 10535.6, 1: 10247.3. Samples: 744229004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:17,363][19170] Avg episode reward: [(0, '237.270'), (1, '499.440')] [2023-12-27 15:43:17,367][20347] Updated weights for policy 0, policy_version 1454044 (0.0005) [2023-12-27 15:43:17,426][20347] Updated weights for policy 0, policy_version 1454054 (0.0006) [2023-12-27 15:43:17,481][20347] Updated weights for policy 0, policy_version 1454064 (0.0008) [2023-12-27 15:43:17,746][20381] Updated weights for policy 1, policy_version 1452657 (0.0008) [2023-12-27 15:43:17,807][20381] Updated weights for policy 1, policy_version 1452667 (0.0005) [2023-12-27 15:43:17,867][20381] Updated weights for policy 1, policy_version 1452677 (0.0006) [2023-12-27 15:43:17,926][20381] Updated weights for policy 1, policy_version 1452687 (0.0005) [2023-12-27 15:43:18,127][20347] Updated weights for policy 0, policy_version 1454074 (0.0009) [2023-12-27 15:43:18,190][20347] Updated weights for policy 0, policy_version 1454084 (0.0008) [2023-12-27 15:43:18,242][20347] Updated weights for policy 0, policy_version 1454094 (0.0007) [2023-12-27 15:43:18,308][20347] Updated weights for policy 0, policy_version 1454104 (0.0008) [2023-12-27 15:43:18,539][20381] Updated weights for policy 1, policy_version 1452697 (0.0008) [2023-12-27 15:43:18,597][20381] Updated weights for policy 1, policy_version 1452707 (0.0009) [2023-12-27 15:43:18,657][20381] Updated weights for policy 1, policy_version 1452717 (0.0008) [2023-12-27 15:43:18,906][20347] Updated weights for policy 0, policy_version 1454114 (0.0005) [2023-12-27 15:43:18,963][20347] Updated weights for policy 0, policy_version 1454124 (0.0005) [2023-12-27 15:43:19,022][20347] Updated weights for policy 0, policy_version 1454134 (0.0005) [2023-12-27 15:43:19,422][20381] Updated weights for policy 1, policy_version 1452727 (0.0009) [2023-12-27 15:43:19,481][20381] Updated weights for policy 1, policy_version 1452737 (0.0006) [2023-12-27 15:43:19,539][20381] Updated weights for policy 1, policy_version 1452747 (0.0007) [2023-12-27 15:43:19,661][20347] Updated weights for policy 0, policy_version 1454144 (0.0007) [2023-12-27 15:43:19,713][20347] Updated weights for policy 0, policy_version 1454154 (0.0008) [2023-12-27 15:43:19,768][20347] Updated weights for policy 0, policy_version 1454164 (0.0007) [2023-12-27 15:43:20,254][20381] Updated weights for policy 1, policy_version 1452757 (0.0010) [2023-12-27 15:43:20,309][20381] Updated weights for policy 1, policy_version 1452767 (0.0006) [2023-12-27 15:43:20,372][20381] Updated weights for policy 1, policy_version 1452777 (0.0009) [2023-12-27 15:43:20,508][20347] Updated weights for policy 0, policy_version 1454174 (0.0007) [2023-12-27 15:43:20,572][20347] Updated weights for policy 0, policy_version 1454184 (0.0006) [2023-12-27 15:43:20,633][20347] Updated weights for policy 0, policy_version 1454194 (0.0008) [2023-12-27 15:43:20,981][20381] Updated weights for policy 1, policy_version 1452787 (0.0009) [2023-12-27 15:43:21,048][20381] Updated weights for policy 1, policy_version 1452797 (0.0007) [2023-12-27 15:43:21,110][20381] Updated weights for policy 1, policy_version 1452807 (0.0007) [2023-12-27 15:43:21,341][20347] Updated weights for policy 0, policy_version 1454204 (0.0007) [2023-12-27 15:43:21,416][20347] Updated weights for policy 0, policy_version 1454214 (0.0008) [2023-12-27 15:43:21,479][20347] Updated weights for policy 0, policy_version 1454224 (0.0005) [2023-12-27 15:43:21,846][20381] Updated weights for policy 1, policy_version 1452817 (0.0009) [2023-12-27 15:43:21,904][20381] Updated weights for policy 1, policy_version 1452827 (0.0007) [2023-12-27 15:43:21,957][20381] Updated weights for policy 1, policy_version 1452837 (0.0010) [2023-12-27 15:43:22,008][20381] Updated weights for policy 1, policy_version 1452847 (0.0012) [2023-12-27 15:43:22,164][20347] Updated weights for policy 0, policy_version 1454234 (0.0006) [2023-12-27 15:43:22,221][20347] Updated weights for policy 0, policy_version 1454244 (0.0005) [2023-12-27 15:43:22,283][20347] Updated weights for policy 0, policy_version 1454254 (0.0007) [2023-12-27 15:43:22,346][20347] Updated weights for policy 0, policy_version 1454264 (0.0008) [2023-12-27 15:43:22,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 744325120. Throughput: 0: 10498.3, 1: 10268.8. Samples: 744291940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:22,363][19170] Avg episode reward: [(0, '94.010'), (1, '547.650')] [2023-12-27 15:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001452848_371957760.pth... [2023-12-27 15:43:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001454264_372367360.pth... [2023-12-27 15:43:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001453048_372056064.pth [2023-12-27 15:43:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001451664_371654656.pth [2023-12-27 15:43:22,688][20381] Updated weights for policy 1, policy_version 1452857 (0.0006) [2023-12-27 15:43:22,746][20381] Updated weights for policy 1, policy_version 1452867 (0.0010) [2023-12-27 15:43:22,812][20381] Updated weights for policy 1, policy_version 1452877 (0.0009) [2023-12-27 15:43:23,114][20347] Updated weights for policy 0, policy_version 1454274 (0.0006) [2023-12-27 15:43:23,172][20347] Updated weights for policy 0, policy_version 1454284 (0.0005) [2023-12-27 15:43:23,230][20347] Updated weights for policy 0, policy_version 1454294 (0.0008) [2023-12-27 15:43:23,449][20381] Updated weights for policy 1, policy_version 1452887 (0.0010) [2023-12-27 15:43:23,503][20381] Updated weights for policy 1, policy_version 1452897 (0.0011) [2023-12-27 15:43:23,555][20381] Updated weights for policy 1, policy_version 1452907 (0.0005) [2023-12-27 15:43:23,838][20347] Updated weights for policy 0, policy_version 1454304 (0.0008) [2023-12-27 15:43:23,893][20347] Updated weights for policy 0, policy_version 1454314 (0.0008) [2023-12-27 15:43:23,942][20347] Updated weights for policy 0, policy_version 1454324 (0.0007) [2023-12-27 15:43:24,212][20381] Updated weights for policy 1, policy_version 1452917 (0.0008) [2023-12-27 15:43:24,265][20381] Updated weights for policy 1, policy_version 1452927 (0.0008) [2023-12-27 15:43:24,311][20381] Updated weights for policy 1, policy_version 1452937 (0.0008) [2023-12-27 15:43:24,654][20347] Updated weights for policy 0, policy_version 1454334 (0.0008) [2023-12-27 15:43:24,712][20347] Updated weights for policy 0, policy_version 1454344 (0.0007) [2023-12-27 15:43:24,770][20347] Updated weights for policy 0, policy_version 1454354 (0.0007) [2023-12-27 15:43:24,969][20381] Updated weights for policy 1, policy_version 1452947 (0.0007) [2023-12-27 15:43:25,015][20381] Updated weights for policy 1, policy_version 1452957 (0.0006) [2023-12-27 15:43:25,073][20381] Updated weights for policy 1, policy_version 1452967 (0.0006) [2023-12-27 15:43:25,490][20347] Updated weights for policy 0, policy_version 1454364 (0.0008) [2023-12-27 15:43:25,540][20347] Updated weights for policy 0, policy_version 1454374 (0.0009) [2023-12-27 15:43:25,600][20347] Updated weights for policy 0, policy_version 1454384 (0.0009) [2023-12-27 15:43:25,659][20381] Updated weights for policy 1, policy_version 1452977 (0.0007) [2023-12-27 15:43:25,709][20381] Updated weights for policy 1, policy_version 1452987 (0.0005) [2023-12-27 15:43:25,762][20381] Updated weights for policy 1, policy_version 1452997 (0.0005) [2023-12-27 15:43:25,816][20381] Updated weights for policy 1, policy_version 1453007 (0.0005) [2023-12-27 15:43:26,315][20347] Updated weights for policy 0, policy_version 1454394 (0.0008) [2023-12-27 15:43:26,370][20347] Updated weights for policy 0, policy_version 1454404 (0.0010) [2023-12-27 15:43:26,428][20347] Updated weights for policy 0, policy_version 1454414 (0.0010) [2023-12-27 15:43:26,441][20381] Updated weights for policy 1, policy_version 1453017 (0.0006) [2023-12-27 15:43:26,489][20347] Updated weights for policy 0, policy_version 1454424 (0.0009) [2023-12-27 15:43:26,494][20381] Updated weights for policy 1, policy_version 1453027 (0.0010) [2023-12-27 15:43:26,546][20381] Updated weights for policy 1, policy_version 1453037 (0.0010) [2023-12-27 15:43:27,122][20347] Updated weights for policy 0, policy_version 1454434 (0.0007) [2023-12-27 15:43:27,173][20347] Updated weights for policy 0, policy_version 1454444 (0.0007) [2023-12-27 15:43:27,218][20347] Updated weights for policy 0, policy_version 1454454 (0.0006) [2023-12-27 15:43:27,224][20381] Updated weights for policy 1, policy_version 1453047 (0.0011) [2023-12-27 15:43:27,272][20381] Updated weights for policy 1, policy_version 1453057 (0.0010) [2023-12-27 15:43:27,331][20381] Updated weights for policy 1, policy_version 1453067 (0.0011) [2023-12-27 15:43:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 744431616. Throughput: 0: 10398.3, 1: 10291.3. Samples: 744414704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:27,363][19170] Avg episode reward: [(0, '117.370'), (1, '496.580')] [2023-12-27 15:43:27,888][20347] Updated weights for policy 0, policy_version 1454464 (0.0007) [2023-12-27 15:43:27,944][20347] Updated weights for policy 0, policy_version 1454474 (0.0007) [2023-12-27 15:43:28,001][20347] Updated weights for policy 0, policy_version 1454484 (0.0008) [2023-12-27 15:43:28,020][20381] Updated weights for policy 1, policy_version 1453077 (0.0011) [2023-12-27 15:43:28,070][20381] Updated weights for policy 1, policy_version 1453087 (0.0010) [2023-12-27 15:43:28,124][20381] Updated weights for policy 1, policy_version 1453097 (0.0010) [2023-12-27 15:43:28,694][20347] Updated weights for policy 0, policy_version 1454494 (0.0009) [2023-12-27 15:43:28,757][20347] Updated weights for policy 0, policy_version 1454504 (0.0008) [2023-12-27 15:43:28,825][20347] Updated weights for policy 0, policy_version 1454514 (0.0009) [2023-12-27 15:43:28,836][20381] Updated weights for policy 1, policy_version 1453107 (0.0011) [2023-12-27 15:43:28,893][20381] Updated weights for policy 1, policy_version 1453117 (0.0011) [2023-12-27 15:43:28,946][20381] Updated weights for policy 1, policy_version 1453127 (0.0011) [2023-12-27 15:43:29,458][20347] Updated weights for policy 0, policy_version 1454524 (0.0010) [2023-12-27 15:43:29,507][20347] Updated weights for policy 0, policy_version 1454534 (0.0011) [2023-12-27 15:43:29,557][20347] Updated weights for policy 0, policy_version 1454544 (0.0007) [2023-12-27 15:43:29,596][20381] Updated weights for policy 1, policy_version 1453137 (0.0010) [2023-12-27 15:43:29,649][20381] Updated weights for policy 1, policy_version 1453147 (0.0005) [2023-12-27 15:43:29,713][20381] Updated weights for policy 1, policy_version 1453157 (0.0006) [2023-12-27 15:43:29,780][20381] Updated weights for policy 1, policy_version 1453167 (0.0007) [2023-12-27 15:43:30,281][20347] Updated weights for policy 0, policy_version 1454554 (0.0006) [2023-12-27 15:43:30,343][20347] Updated weights for policy 0, policy_version 1454564 (0.0007) [2023-12-27 15:43:30,381][20381] Updated weights for policy 1, policy_version 1453177 (0.0006) [2023-12-27 15:43:30,398][20347] Updated weights for policy 0, policy_version 1454574 (0.0010) [2023-12-27 15:43:30,434][20381] Updated weights for policy 1, policy_version 1453187 (0.0005) [2023-12-27 15:43:30,452][20347] Updated weights for policy 0, policy_version 1454584 (0.0010) [2023-12-27 15:43:30,487][20381] Updated weights for policy 1, policy_version 1453197 (0.0005) [2023-12-27 15:43:31,080][20381] Updated weights for policy 1, policy_version 1453207 (0.0009) [2023-12-27 15:43:31,092][20347] Updated weights for policy 0, policy_version 1454594 (0.0010) [2023-12-27 15:43:31,146][20381] Updated weights for policy 1, policy_version 1453217 (0.0012) [2023-12-27 15:43:31,156][20347] Updated weights for policy 0, policy_version 1454604 (0.0007) [2023-12-27 15:43:31,207][20381] Updated weights for policy 1, policy_version 1453227 (0.0009) [2023-12-27 15:43:31,209][20347] Updated weights for policy 0, policy_version 1454614 (0.0007) [2023-12-27 15:43:31,976][20381] Updated weights for policy 1, policy_version 1453237 (0.0009) [2023-12-27 15:43:31,978][20347] Updated weights for policy 0, policy_version 1454624 (0.0006) [2023-12-27 15:43:32,029][20381] Updated weights for policy 1, policy_version 1453247 (0.0010) [2023-12-27 15:43:32,034][20347] Updated weights for policy 0, policy_version 1454634 (0.0005) [2023-12-27 15:43:32,085][20381] Updated weights for policy 1, policy_version 1453257 (0.0011) [2023-12-27 15:43:32,096][20347] Updated weights for policy 0, policy_version 1454644 (0.0006) [2023-12-27 15:43:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20889.7, 300 sec: 20605.0). Total num frames: 744529920. Throughput: 0: 10343.6, 1: 10305.3. Samples: 744537828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:32,363][19170] Avg episode reward: [(0, '185.650'), (1, '513.450')] [2023-12-27 15:43:32,758][20381] Updated weights for policy 1, policy_version 1453267 (0.0010) [2023-12-27 15:43:32,781][20347] Updated weights for policy 0, policy_version 1454654 (0.0009) [2023-12-27 15:43:32,814][20381] Updated weights for policy 1, policy_version 1453277 (0.0008) [2023-12-27 15:43:32,835][20347] Updated weights for policy 0, policy_version 1454664 (0.0008) [2023-12-27 15:43:32,875][20381] Updated weights for policy 1, policy_version 1453287 (0.0006) [2023-12-27 15:43:32,891][20347] Updated weights for policy 0, policy_version 1454674 (0.0005) [2023-12-27 15:43:33,408][20381] Updated weights for policy 1, policy_version 1453297 (0.0005) [2023-12-27 15:43:33,461][20381] Updated weights for policy 1, policy_version 1453307 (0.0005) [2023-12-27 15:43:33,506][20381] Updated weights for policy 1, policy_version 1453317 (0.0005) [2023-12-27 15:43:33,551][20381] Updated weights for policy 1, policy_version 1453327 (0.0005) [2023-12-27 15:43:33,637][20347] Updated weights for policy 0, policy_version 1454684 (0.0007) [2023-12-27 15:43:33,684][20347] Updated weights for policy 0, policy_version 1454694 (0.0006) [2023-12-27 15:43:33,731][20347] Updated weights for policy 0, policy_version 1454704 (0.0007) [2023-12-27 15:43:34,197][20381] Updated weights for policy 1, policy_version 1453337 (0.0006) [2023-12-27 15:43:34,261][20381] Updated weights for policy 1, policy_version 1453347 (0.0009) [2023-12-27 15:43:34,322][20381] Updated weights for policy 1, policy_version 1453357 (0.0007) [2023-12-27 15:43:34,358][20347] Updated weights for policy 0, policy_version 1454714 (0.0007) [2023-12-27 15:43:34,422][20347] Updated weights for policy 0, policy_version 1454724 (0.0007) [2023-12-27 15:43:34,487][20347] Updated weights for policy 0, policy_version 1454734 (0.0008) [2023-12-27 15:43:34,555][20347] Updated weights for policy 0, policy_version 1454744 (0.0009) [2023-12-27 15:43:34,929][20381] Updated weights for policy 1, policy_version 1453367 (0.0006) [2023-12-27 15:43:34,989][20381] Updated weights for policy 1, policy_version 1453377 (0.0005) [2023-12-27 15:43:35,053][20381] Updated weights for policy 1, policy_version 1453387 (0.0006) [2023-12-27 15:43:35,253][20347] Updated weights for policy 0, policy_version 1454754 (0.0008) [2023-12-27 15:43:35,302][20347] Updated weights for policy 0, policy_version 1454764 (0.0010) [2023-12-27 15:43:35,357][20347] Updated weights for policy 0, policy_version 1454774 (0.0010) [2023-12-27 15:43:35,650][20381] Updated weights for policy 1, policy_version 1453397 (0.0009) [2023-12-27 15:43:35,706][20381] Updated weights for policy 1, policy_version 1453407 (0.0010) [2023-12-27 15:43:35,751][20381] Updated weights for policy 1, policy_version 1453417 (0.0010) [2023-12-27 15:43:35,935][20347] Updated weights for policy 0, policy_version 1454784 (0.0006) [2023-12-27 15:43:36,000][20347] Updated weights for policy 0, policy_version 1454794 (0.0008) [2023-12-27 15:43:36,058][20347] Updated weights for policy 0, policy_version 1454804 (0.0007) [2023-12-27 15:43:36,440][20381] Updated weights for policy 1, policy_version 1453427 (0.0011) [2023-12-27 15:43:36,506][20381] Updated weights for policy 1, policy_version 1453437 (0.0011) [2023-12-27 15:43:36,572][20381] Updated weights for policy 1, policy_version 1453447 (0.0011) [2023-12-27 15:43:36,669][20347] Updated weights for policy 0, policy_version 1454814 (0.0007) [2023-12-27 15:43:36,734][20347] Updated weights for policy 0, policy_version 1454824 (0.0005) [2023-12-27 15:43:36,783][20347] Updated weights for policy 0, policy_version 1454834 (0.0005) [2023-12-27 15:43:37,253][20381] Updated weights for policy 1, policy_version 1453457 (0.0011) [2023-12-27 15:43:37,289][20347] Updated weights for policy 0, policy_version 1454844 (0.0006) [2023-12-27 15:43:37,308][20381] Updated weights for policy 1, policy_version 1453467 (0.0011) [2023-12-27 15:43:37,347][20347] Updated weights for policy 0, policy_version 1454854 (0.0007) [2023-12-27 15:43:37,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 744628224. Throughput: 0: 10288.6, 1: 10384.2. Samples: 744601788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:37,364][19170] Avg episode reward: [(0, '520.330'), (1, '460.500')] [2023-12-27 15:43:37,373][20381] Updated weights for policy 1, policy_version 1453477 (0.0011) [2023-12-27 15:43:37,402][20347] Updated weights for policy 0, policy_version 1454864 (0.0006) [2023-12-27 15:43:37,430][20381] Updated weights for policy 1, policy_version 1453487 (0.0007) [2023-12-27 15:43:37,434][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001453488_372121600.pth... [2023-12-27 15:43:37,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001452240_371802112.pth [2023-12-27 15:43:37,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001454872_372523008.pth... [2023-12-27 15:43:37,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001453656_372211712.pth [2023-12-27 15:43:38,011][20381] Updated weights for policy 1, policy_version 1453497 (0.0008) [2023-12-27 15:43:38,073][20381] Updated weights for policy 1, policy_version 1453507 (0.0005) [2023-12-27 15:43:38,076][20347] Updated weights for policy 0, policy_version 1454874 (0.0007) [2023-12-27 15:43:38,130][20347] Updated weights for policy 0, policy_version 1454884 (0.0006) [2023-12-27 15:43:38,139][20381] Updated weights for policy 1, policy_version 1453517 (0.0011) [2023-12-27 15:43:38,183][20347] Updated weights for policy 0, policy_version 1454894 (0.0006) [2023-12-27 15:43:38,238][20347] Updated weights for policy 0, policy_version 1454904 (0.0005) [2023-12-27 15:43:38,781][20381] Updated weights for policy 1, policy_version 1453527 (0.0007) [2023-12-27 15:43:38,810][20347] Updated weights for policy 0, policy_version 1454914 (0.0006) [2023-12-27 15:43:38,849][20381] Updated weights for policy 1, policy_version 1453537 (0.0006) [2023-12-27 15:43:38,861][20347] Updated weights for policy 0, policy_version 1454924 (0.0006) [2023-12-27 15:43:38,918][20347] Updated weights for policy 0, policy_version 1454934 (0.0006) [2023-12-27 15:43:38,919][20381] Updated weights for policy 1, policy_version 1453547 (0.0006) [2023-12-27 15:43:39,548][20347] Updated weights for policy 0, policy_version 1454944 (0.0008) [2023-12-27 15:43:39,613][20347] Updated weights for policy 0, policy_version 1454954 (0.0009) [2023-12-27 15:43:39,614][20381] Updated weights for policy 1, policy_version 1453557 (0.0005) [2023-12-27 15:43:39,673][20347] Updated weights for policy 0, policy_version 1454964 (0.0006) [2023-12-27 15:43:39,674][20381] Updated weights for policy 1, policy_version 1453567 (0.0008) [2023-12-27 15:43:39,729][20381] Updated weights for policy 1, policy_version 1453577 (0.0008) [2023-12-27 15:43:40,317][20347] Updated weights for policy 0, policy_version 1454974 (0.0007) [2023-12-27 15:43:40,383][20347] Updated weights for policy 0, policy_version 1454984 (0.0008) [2023-12-27 15:43:40,439][20347] Updated weights for policy 0, policy_version 1454994 (0.0008) [2023-12-27 15:43:40,476][20381] Updated weights for policy 1, policy_version 1453587 (0.0008) [2023-12-27 15:43:40,534][20381] Updated weights for policy 1, policy_version 1453597 (0.0008) [2023-12-27 15:43:40,591][20381] Updated weights for policy 1, policy_version 1453607 (0.0007) [2023-12-27 15:43:41,102][20347] Updated weights for policy 0, policy_version 1455004 (0.0009) [2023-12-27 15:43:41,171][20347] Updated weights for policy 0, policy_version 1455014 (0.0010) [2023-12-27 15:43:41,236][20347] Updated weights for policy 0, policy_version 1455024 (0.0009) [2023-12-27 15:43:41,287][20381] Updated weights for policy 1, policy_version 1453617 (0.0008) [2023-12-27 15:43:41,358][20381] Updated weights for policy 1, policy_version 1453627 (0.0006) [2023-12-27 15:43:41,416][20381] Updated weights for policy 1, policy_version 1453637 (0.0007) [2023-12-27 15:43:41,474][20381] Updated weights for policy 1, policy_version 1453647 (0.0006) [2023-12-27 15:43:42,003][20347] Updated weights for policy 0, policy_version 1455034 (0.0008) [2023-12-27 15:43:42,069][20347] Updated weights for policy 0, policy_version 1455044 (0.0007) [2023-12-27 15:43:42,131][20347] Updated weights for policy 0, policy_version 1455054 (0.0008) [2023-12-27 15:43:42,144][20381] Updated weights for policy 1, policy_version 1453657 (0.0007) [2023-12-27 15:43:42,195][20347] Updated weights for policy 0, policy_version 1455064 (0.0006) [2023-12-27 15:43:42,211][20381] Updated weights for policy 1, policy_version 1453667 (0.0008) [2023-12-27 15:43:42,278][20381] Updated weights for policy 1, policy_version 1453677 (0.0007) [2023-12-27 15:43:42,366][19170] Fps is (10 sec: 21291.9, 60 sec: 20888.4, 300 sec: 20632.5). Total num frames: 744742912. Throughput: 0: 10398.2, 1: 10373.3. Samples: 744729600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:42,366][19170] Avg episode reward: [(0, '373.310'), (1, '391.040')] [2023-12-27 15:43:42,903][20347] Updated weights for policy 0, policy_version 1455074 (0.0007) [2023-12-27 15:43:42,918][20381] Updated weights for policy 1, policy_version 1453687 (0.0007) [2023-12-27 15:43:42,962][20347] Updated weights for policy 0, policy_version 1455084 (0.0005) [2023-12-27 15:43:42,969][20381] Updated weights for policy 1, policy_version 1453697 (0.0007) [2023-12-27 15:43:43,016][20347] Updated weights for policy 0, policy_version 1455094 (0.0006) [2023-12-27 15:43:43,026][20381] Updated weights for policy 1, policy_version 1453707 (0.0005) [2023-12-27 15:43:43,592][20347] Updated weights for policy 0, policy_version 1455104 (0.0006) [2023-12-27 15:43:43,640][20347] Updated weights for policy 0, policy_version 1455114 (0.0007) [2023-12-27 15:43:43,693][20347] Updated weights for policy 0, policy_version 1455124 (0.0007) [2023-12-27 15:43:43,783][20381] Updated weights for policy 1, policy_version 1453717 (0.0008) [2023-12-27 15:43:43,842][20381] Updated weights for policy 1, policy_version 1453727 (0.0008) [2023-12-27 15:43:43,904][20381] Updated weights for policy 1, policy_version 1453737 (0.0008) [2023-12-27 15:43:44,345][20347] Updated weights for policy 0, policy_version 1455134 (0.0007) [2023-12-27 15:43:44,402][20347] Updated weights for policy 0, policy_version 1455144 (0.0007) [2023-12-27 15:43:44,459][20347] Updated weights for policy 0, policy_version 1455154 (0.0006) [2023-12-27 15:43:44,529][20381] Updated weights for policy 1, policy_version 1453747 (0.0008) [2023-12-27 15:43:44,587][20381] Updated weights for policy 1, policy_version 1453757 (0.0008) [2023-12-27 15:43:44,643][20381] Updated weights for policy 1, policy_version 1453767 (0.0007) [2023-12-27 15:43:45,178][20347] Updated weights for policy 0, policy_version 1455164 (0.0009) [2023-12-27 15:43:45,234][20347] Updated weights for policy 0, policy_version 1455174 (0.0011) [2023-12-27 15:43:45,297][20347] Updated weights for policy 0, policy_version 1455184 (0.0011) [2023-12-27 15:43:45,316][20381] Updated weights for policy 1, policy_version 1453777 (0.0008) [2023-12-27 15:43:45,368][20381] Updated weights for policy 1, policy_version 1453787 (0.0010) [2023-12-27 15:43:45,425][20381] Updated weights for policy 1, policy_version 1453797 (0.0005) [2023-12-27 15:43:45,472][20381] Updated weights for policy 1, policy_version 1453807 (0.0005) [2023-12-27 15:43:45,950][20347] Updated weights for policy 0, policy_version 1455194 (0.0009) [2023-12-27 15:43:45,999][20347] Updated weights for policy 0, policy_version 1455204 (0.0005) [2023-12-27 15:43:46,050][20347] Updated weights for policy 0, policy_version 1455214 (0.0005) [2023-12-27 15:43:46,081][20381] Updated weights for policy 1, policy_version 1453817 (0.0010) [2023-12-27 15:43:46,099][20347] Updated weights for policy 0, policy_version 1455224 (0.0008) [2023-12-27 15:43:46,132][20381] Updated weights for policy 1, policy_version 1453827 (0.0006) [2023-12-27 15:43:46,180][20381] Updated weights for policy 1, policy_version 1453837 (0.0005) [2023-12-27 15:43:46,706][20347] Updated weights for policy 0, policy_version 1455234 (0.0006) [2023-12-27 15:43:46,761][20347] Updated weights for policy 0, policy_version 1455244 (0.0008) [2023-12-27 15:43:46,829][20347] Updated weights for policy 0, policy_version 1455254 (0.0008) [2023-12-27 15:43:46,848][20381] Updated weights for policy 1, policy_version 1453847 (0.0009) [2023-12-27 15:43:46,906][20381] Updated weights for policy 1, policy_version 1453857 (0.0008) [2023-12-27 15:43:46,962][20381] Updated weights for policy 1, policy_version 1453867 (0.0008) [2023-12-27 15:43:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 744841216. Throughput: 0: 10418.2, 1: 10403.7. Samples: 744853092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:47,363][19170] Avg episode reward: [(0, '520.280'), (1, '397.250')] [2023-12-27 15:43:47,449][20347] Updated weights for policy 0, policy_version 1455264 (0.0008) [2023-12-27 15:43:47,501][20347] Updated weights for policy 0, policy_version 1455274 (0.0008) [2023-12-27 15:43:47,554][20347] Updated weights for policy 0, policy_version 1455284 (0.0007) [2023-12-27 15:43:47,666][20381] Updated weights for policy 1, policy_version 1453877 (0.0008) [2023-12-27 15:43:47,721][20381] Updated weights for policy 1, policy_version 1453887 (0.0007) [2023-12-27 15:43:47,782][20381] Updated weights for policy 1, policy_version 1453897 (0.0007) [2023-12-27 15:43:48,198][20347] Updated weights for policy 0, policy_version 1455294 (0.0008) [2023-12-27 15:43:48,258][20347] Updated weights for policy 0, policy_version 1455304 (0.0007) [2023-12-27 15:43:48,317][20347] Updated weights for policy 0, policy_version 1455314 (0.0005) [2023-12-27 15:43:48,429][20381] Updated weights for policy 1, policy_version 1453907 (0.0007) [2023-12-27 15:43:48,484][20381] Updated weights for policy 1, policy_version 1453917 (0.0005) [2023-12-27 15:43:48,537][20381] Updated weights for policy 1, policy_version 1453927 (0.0010) [2023-12-27 15:43:48,993][20347] Updated weights for policy 0, policy_version 1455324 (0.0008) [2023-12-27 15:43:49,051][20347] Updated weights for policy 0, policy_version 1455334 (0.0007) [2023-12-27 15:43:49,102][20347] Updated weights for policy 0, policy_version 1455344 (0.0007) [2023-12-27 15:43:49,156][20381] Updated weights for policy 1, policy_version 1453937 (0.0010) [2023-12-27 15:43:49,209][20381] Updated weights for policy 1, policy_version 1453947 (0.0008) [2023-12-27 15:43:49,271][20381] Updated weights for policy 1, policy_version 1453957 (0.0008) [2023-12-27 15:43:49,328][20381] Updated weights for policy 1, policy_version 1453967 (0.0006) [2023-12-27 15:43:49,846][20347] Updated weights for policy 0, policy_version 1455354 (0.0008) [2023-12-27 15:43:49,912][20347] Updated weights for policy 0, policy_version 1455364 (0.0010) [2023-12-27 15:43:49,974][20347] Updated weights for policy 0, policy_version 1455374 (0.0010) [2023-12-27 15:43:50,031][20347] Updated weights for policy 0, policy_version 1455384 (0.0007) [2023-12-27 15:43:50,041][20381] Updated weights for policy 1, policy_version 1453977 (0.0007) [2023-12-27 15:43:50,096][20381] Updated weights for policy 1, policy_version 1453987 (0.0007) [2023-12-27 15:43:50,159][20381] Updated weights for policy 1, policy_version 1453997 (0.0009) [2023-12-27 15:43:50,651][20347] Updated weights for policy 0, policy_version 1455394 (0.0009) [2023-12-27 15:43:50,697][20347] Updated weights for policy 0, policy_version 1455404 (0.0010) [2023-12-27 15:43:50,743][20347] Updated weights for policy 0, policy_version 1455414 (0.0010) [2023-12-27 15:43:50,858][20381] Updated weights for policy 1, policy_version 1454007 (0.0010) [2023-12-27 15:43:50,917][20381] Updated weights for policy 1, policy_version 1454017 (0.0011) [2023-12-27 15:43:50,972][20381] Updated weights for policy 1, policy_version 1454027 (0.0011) [2023-12-27 15:43:51,443][20347] Updated weights for policy 0, policy_version 1455424 (0.0010) [2023-12-27 15:43:51,495][20347] Updated weights for policy 0, policy_version 1455434 (0.0006) [2023-12-27 15:43:51,554][20347] Updated weights for policy 0, policy_version 1455444 (0.0009) [2023-12-27 15:43:51,729][20381] Updated weights for policy 1, policy_version 1454037 (0.0010) [2023-12-27 15:43:51,788][20381] Updated weights for policy 1, policy_version 1454047 (0.0010) [2023-12-27 15:43:51,843][20381] Updated weights for policy 1, policy_version 1454057 (0.0008) [2023-12-27 15:43:52,222][20347] Updated weights for policy 0, policy_version 1455454 (0.0008) [2023-12-27 15:43:52,282][20347] Updated weights for policy 0, policy_version 1455464 (0.0006) [2023-12-27 15:43:52,336][20347] Updated weights for policy 0, policy_version 1455474 (0.0010) [2023-12-27 15:43:52,363][19170] Fps is (10 sec: 19667.1, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 744939520. Throughput: 0: 10399.4, 1: 10389.2. Samples: 744914752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:52,363][19170] Avg episode reward: [(0, '539.520'), (1, '437.660')] [2023-12-27 15:43:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001454064_372269056.pth... [2023-12-27 15:43:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001455480_372678656.pth... [2023-12-27 15:43:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001452848_371957760.pth [2023-12-27 15:43:52,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001454264_372367360.pth [2023-12-27 15:43:52,574][20381] Updated weights for policy 1, policy_version 1454067 (0.0009) [2023-12-27 15:43:52,638][20381] Updated weights for policy 1, policy_version 1454077 (0.0008) [2023-12-27 15:43:52,698][20381] Updated weights for policy 1, policy_version 1454087 (0.0008) [2023-12-27 15:43:52,998][20347] Updated weights for policy 0, policy_version 1455484 (0.0009) [2023-12-27 15:43:53,047][20347] Updated weights for policy 0, policy_version 1455494 (0.0009) [2023-12-27 15:43:53,095][20347] Updated weights for policy 0, policy_version 1455504 (0.0008) [2023-12-27 15:43:53,354][20381] Updated weights for policy 1, policy_version 1454097 (0.0007) [2023-12-27 15:43:53,407][20381] Updated weights for policy 1, policy_version 1454107 (0.0007) [2023-12-27 15:43:53,464][20381] Updated weights for policy 1, policy_version 1454117 (0.0007) [2023-12-27 15:43:53,515][20381] Updated weights for policy 1, policy_version 1454127 (0.0007) [2023-12-27 15:43:53,728][20347] Updated weights for policy 0, policy_version 1455514 (0.0007) [2023-12-27 15:43:53,788][20347] Updated weights for policy 0, policy_version 1455524 (0.0007) [2023-12-27 15:43:53,845][20347] Updated weights for policy 0, policy_version 1455534 (0.0006) [2023-12-27 15:43:53,904][20347] Updated weights for policy 0, policy_version 1455544 (0.0007) [2023-12-27 15:43:54,255][20381] Updated weights for policy 1, policy_version 1454137 (0.0008) [2023-12-27 15:43:54,314][20381] Updated weights for policy 1, policy_version 1454147 (0.0008) [2023-12-27 15:43:54,371][20381] Updated weights for policy 1, policy_version 1454157 (0.0008) [2023-12-27 15:43:54,482][20347] Updated weights for policy 0, policy_version 1455554 (0.0011) [2023-12-27 15:43:54,537][20347] Updated weights for policy 0, policy_version 1455564 (0.0010) [2023-12-27 15:43:54,589][20347] Updated weights for policy 0, policy_version 1455574 (0.0007) [2023-12-27 15:43:55,060][20381] Updated weights for policy 1, policy_version 1454167 (0.0008) [2023-12-27 15:43:55,110][20381] Updated weights for policy 1, policy_version 1454177 (0.0007) [2023-12-27 15:43:55,162][20381] Updated weights for policy 1, policy_version 1454187 (0.0007) [2023-12-27 15:43:55,200][20347] Updated weights for policy 0, policy_version 1455584 (0.0009) [2023-12-27 15:43:55,258][20347] Updated weights for policy 0, policy_version 1455594 (0.0010) [2023-12-27 15:43:55,316][20347] Updated weights for policy 0, policy_version 1455604 (0.0010) [2023-12-27 15:43:55,858][20381] Updated weights for policy 1, policy_version 1454197 (0.0007) [2023-12-27 15:43:55,915][20381] Updated weights for policy 1, policy_version 1454207 (0.0007) [2023-12-27 15:43:55,947][20347] Updated weights for policy 0, policy_version 1455614 (0.0008) [2023-12-27 15:43:55,965][20381] Updated weights for policy 1, policy_version 1454217 (0.0006) [2023-12-27 15:43:56,002][20347] Updated weights for policy 0, policy_version 1455624 (0.0007) [2023-12-27 15:43:56,061][20347] Updated weights for policy 0, policy_version 1455634 (0.0005) [2023-12-27 15:43:56,646][20347] Updated weights for policy 0, policy_version 1455644 (0.0006) [2023-12-27 15:43:56,690][20381] Updated weights for policy 1, policy_version 1454227 (0.0006) [2023-12-27 15:43:56,692][20347] Updated weights for policy 0, policy_version 1455654 (0.0007) [2023-12-27 15:43:56,736][20381] Updated weights for policy 1, policy_version 1454237 (0.0006) [2023-12-27 15:43:56,746][20347] Updated weights for policy 0, policy_version 1455664 (0.0008) [2023-12-27 15:43:56,788][20381] Updated weights for policy 1, policy_version 1454247 (0.0008) [2023-12-27 15:43:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 745046016. Throughput: 0: 10391.4, 1: 10330.4. Samples: 745038368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:43:57,363][19170] Avg episode reward: [(0, '627.490'), (1, '345.670')] [2023-12-27 15:43:57,455][20381] Updated weights for policy 1, policy_version 1454257 (0.0008) [2023-12-27 15:43:57,457][20347] Updated weights for policy 0, policy_version 1455674 (0.0006) [2023-12-27 15:43:57,510][20381] Updated weights for policy 1, policy_version 1454267 (0.0011) [2023-12-27 15:43:57,513][20347] Updated weights for policy 0, policy_version 1455684 (0.0005) [2023-12-27 15:43:57,558][20381] Updated weights for policy 1, policy_version 1454277 (0.0010) [2023-12-27 15:43:57,575][20347] Updated weights for policy 0, policy_version 1455694 (0.0005) [2023-12-27 15:43:57,611][20381] Updated weights for policy 1, policy_version 1454287 (0.0011) [2023-12-27 15:43:57,627][20347] Updated weights for policy 0, policy_version 1455704 (0.0005) [2023-12-27 15:43:58,194][20347] Updated weights for policy 0, policy_version 1455714 (0.0006) [2023-12-27 15:43:58,253][20347] Updated weights for policy 0, policy_version 1455724 (0.0008) [2023-12-27 15:43:58,276][20381] Updated weights for policy 1, policy_version 1454297 (0.0008) [2023-12-27 15:43:58,310][20347] Updated weights for policy 0, policy_version 1455734 (0.0008) [2023-12-27 15:43:58,342][20381] Updated weights for policy 1, policy_version 1454307 (0.0008) [2023-12-27 15:43:58,407][20381] Updated weights for policy 1, policy_version 1454317 (0.0007) [2023-12-27 15:43:59,093][20347] Updated weights for policy 0, policy_version 1455744 (0.0008) [2023-12-27 15:43:59,157][20347] Updated weights for policy 0, policy_version 1455754 (0.0007) [2023-12-27 15:43:59,163][20381] Updated weights for policy 1, policy_version 1454327 (0.0008) [2023-12-27 15:43:59,217][20381] Updated weights for policy 1, policy_version 1454337 (0.0008) [2023-12-27 15:43:59,217][20347] Updated weights for policy 0, policy_version 1455764 (0.0007) [2023-12-27 15:43:59,286][20381] Updated weights for policy 1, policy_version 1454347 (0.0008) [2023-12-27 15:43:59,983][20347] Updated weights for policy 0, policy_version 1455774 (0.0009) [2023-12-27 15:44:00,004][20381] Updated weights for policy 1, policy_version 1454357 (0.0006) [2023-12-27 15:44:00,039][20347] Updated weights for policy 0, policy_version 1455784 (0.0010) [2023-12-27 15:44:00,067][20381] Updated weights for policy 1, policy_version 1454367 (0.0007) [2023-12-27 15:44:00,098][20347] Updated weights for policy 0, policy_version 1455794 (0.0010) [2023-12-27 15:44:00,126][20381] Updated weights for policy 1, policy_version 1454377 (0.0006) [2023-12-27 15:44:00,663][20381] Updated weights for policy 1, policy_version 1454387 (0.0006) [2023-12-27 15:44:00,715][20381] Updated weights for policy 1, policy_version 1454397 (0.0006) [2023-12-27 15:44:00,739][20347] Updated weights for policy 0, policy_version 1455804 (0.0010) [2023-12-27 15:44:00,769][20381] Updated weights for policy 1, policy_version 1454407 (0.0006) [2023-12-27 15:44:00,793][20347] Updated weights for policy 0, policy_version 1455814 (0.0010) [2023-12-27 15:44:00,840][20347] Updated weights for policy 0, policy_version 1455824 (0.0010) [2023-12-27 15:44:01,453][20381] Updated weights for policy 1, policy_version 1454417 (0.0006) [2023-12-27 15:44:01,511][20381] Updated weights for policy 1, policy_version 1454427 (0.0009) [2023-12-27 15:44:01,517][20347] Updated weights for policy 0, policy_version 1455834 (0.0010) [2023-12-27 15:44:01,559][20381] Updated weights for policy 1, policy_version 1454437 (0.0005) [2023-12-27 15:44:01,575][20347] Updated weights for policy 0, policy_version 1455844 (0.0010) [2023-12-27 15:44:01,609][20381] Updated weights for policy 1, policy_version 1454447 (0.0008) [2023-12-27 15:44:01,636][20347] Updated weights for policy 0, policy_version 1455854 (0.0010) [2023-12-27 15:44:01,692][20347] Updated weights for policy 0, policy_version 1455864 (0.0005) [2023-12-27 15:44:02,353][20347] Updated weights for policy 0, policy_version 1455874 (0.0007) [2023-12-27 15:44:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 745144320. Throughput: 0: 10395.5, 1: 10299.8. Samples: 745160296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:44:02,363][19170] Avg episode reward: [(0, '617.430'), (1, '357.240')] [2023-12-27 15:44:02,367][20381] Updated weights for policy 1, policy_version 1454457 (0.0007) [2023-12-27 15:44:02,420][20347] Updated weights for policy 0, policy_version 1455884 (0.0008) [2023-12-27 15:44:02,425][20381] Updated weights for policy 1, policy_version 1454467 (0.0006) [2023-12-27 15:44:02,479][20381] Updated weights for policy 1, policy_version 1454477 (0.0007) [2023-12-27 15:44:02,486][20347] Updated weights for policy 0, policy_version 1455894 (0.0008) [2023-12-27 15:44:03,061][20381] Updated weights for policy 1, policy_version 1454487 (0.0007) [2023-12-27 15:44:03,108][20381] Updated weights for policy 1, policy_version 1454497 (0.0007) [2023-12-27 15:44:03,152][20381] Updated weights for policy 1, policy_version 1454507 (0.0006) [2023-12-27 15:44:03,191][20347] Updated weights for policy 0, policy_version 1455904 (0.0006) [2023-12-27 15:44:03,240][20347] Updated weights for policy 0, policy_version 1455914 (0.0007) [2023-12-27 15:44:03,289][20347] Updated weights for policy 0, policy_version 1455924 (0.0008) [2023-12-27 15:44:03,746][20381] Updated weights for policy 1, policy_version 1454517 (0.0006) [2023-12-27 15:44:03,800][20381] Updated weights for policy 1, policy_version 1454527 (0.0005) [2023-12-27 15:44:03,860][20381] Updated weights for policy 1, policy_version 1454537 (0.0007) [2023-12-27 15:44:04,010][20347] Updated weights for policy 0, policy_version 1455934 (0.0006) [2023-12-27 15:44:04,067][20347] Updated weights for policy 0, policy_version 1455944 (0.0005) [2023-12-27 15:44:04,126][20347] Updated weights for policy 0, policy_version 1455954 (0.0007) [2023-12-27 15:44:04,549][20381] Updated weights for policy 1, policy_version 1454547 (0.0007) [2023-12-27 15:44:04,606][20381] Updated weights for policy 1, policy_version 1454557 (0.0007) [2023-12-27 15:44:04,665][20381] Updated weights for policy 1, policy_version 1454567 (0.0007) [2023-12-27 15:44:04,787][20347] Updated weights for policy 0, policy_version 1455964 (0.0008) [2023-12-27 15:44:04,835][20347] Updated weights for policy 0, policy_version 1455974 (0.0009) [2023-12-27 15:44:04,888][20347] Updated weights for policy 0, policy_version 1455984 (0.0007) [2023-12-27 15:44:05,350][20381] Updated weights for policy 1, policy_version 1454577 (0.0008) [2023-12-27 15:44:05,398][20381] Updated weights for policy 1, policy_version 1454587 (0.0008) [2023-12-27 15:44:05,447][20381] Updated weights for policy 1, policy_version 1454597 (0.0007) [2023-12-27 15:44:05,496][20381] Updated weights for policy 1, policy_version 1454607 (0.0008) [2023-12-27 15:44:05,557][20347] Updated weights for policy 0, policy_version 1455994 (0.0008) [2023-12-27 15:44:05,615][20347] Updated weights for policy 0, policy_version 1456004 (0.0008) [2023-12-27 15:44:05,670][20347] Updated weights for policy 0, policy_version 1456014 (0.0007) [2023-12-27 15:44:05,727][20347] Updated weights for policy 0, policy_version 1456024 (0.0007) [2023-12-27 15:44:06,187][20381] Updated weights for policy 1, policy_version 1454617 (0.0007) [2023-12-27 15:44:06,237][20381] Updated weights for policy 1, policy_version 1454627 (0.0007) [2023-12-27 15:44:06,293][20381] Updated weights for policy 1, policy_version 1454637 (0.0008) [2023-12-27 15:44:06,368][20347] Updated weights for policy 0, policy_version 1456034 (0.0008) [2023-12-27 15:44:06,422][20347] Updated weights for policy 0, policy_version 1456044 (0.0007) [2023-12-27 15:44:06,479][20347] Updated weights for policy 0, policy_version 1456054 (0.0008) [2023-12-27 15:44:07,004][20381] Updated weights for policy 1, policy_version 1454647 (0.0008) [2023-12-27 15:44:07,050][20381] Updated weights for policy 1, policy_version 1454657 (0.0008) [2023-12-27 15:44:07,098][20381] Updated weights for policy 1, policy_version 1454667 (0.0007) [2023-12-27 15:44:07,178][20347] Updated weights for policy 0, policy_version 1456064 (0.0008) [2023-12-27 15:44:07,223][20347] Updated weights for policy 0, policy_version 1456074 (0.0007) [2023-12-27 15:44:07,278][20347] Updated weights for policy 0, policy_version 1456084 (0.0008) [2023-12-27 15:44:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 745259008. Throughput: 0: 10339.9, 1: 10338.7. Samples: 745222480. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:07,364][19170] Avg episode reward: [(0, '647.210'), (1, '409.770')] [2023-12-27 15:44:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001454672_372424704.pth... [2023-12-27 15:44:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001456088_372834304.pth... [2023-12-27 15:44:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001454872_372523008.pth [2023-12-27 15:44:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001453488_372121600.pth [2023-12-27 15:44:07,834][20381] Updated weights for policy 1, policy_version 1454677 (0.0008) [2023-12-27 15:44:07,884][20381] Updated weights for policy 1, policy_version 1454687 (0.0007) [2023-12-27 15:44:07,942][20381] Updated weights for policy 1, policy_version 1454697 (0.0008) [2023-12-27 15:44:07,955][20347] Updated weights for policy 0, policy_version 1456094 (0.0009) [2023-12-27 15:44:08,016][20347] Updated weights for policy 0, policy_version 1456104 (0.0010) [2023-12-27 15:44:08,071][20347] Updated weights for policy 0, policy_version 1456114 (0.0010) [2023-12-27 15:44:08,655][20381] Updated weights for policy 1, policy_version 1454707 (0.0006) [2023-12-27 15:44:08,706][20381] Updated weights for policy 1, policy_version 1454717 (0.0007) [2023-12-27 15:44:08,767][20381] Updated weights for policy 1, policy_version 1454727 (0.0008) [2023-12-27 15:44:08,779][20347] Updated weights for policy 0, policy_version 1456124 (0.0010) [2023-12-27 15:44:08,826][20347] Updated weights for policy 0, policy_version 1456134 (0.0010) [2023-12-27 15:44:08,878][20347] Updated weights for policy 0, policy_version 1456144 (0.0010) [2023-12-27 15:44:09,502][20381] Updated weights for policy 1, policy_version 1454737 (0.0006) [2023-12-27 15:44:09,564][20381] Updated weights for policy 1, policy_version 1454747 (0.0007) [2023-12-27 15:44:09,625][20347] Updated weights for policy 0, policy_version 1456154 (0.0011) [2023-12-27 15:44:09,656][20381] Updated weights for policy 1, policy_version 1454757 (0.0008) [2023-12-27 15:44:09,686][20347] Updated weights for policy 0, policy_version 1456164 (0.0011) [2023-12-27 15:44:09,714][20381] Updated weights for policy 1, policy_version 1454767 (0.0005) [2023-12-27 15:44:09,753][20347] Updated weights for policy 0, policy_version 1456174 (0.0011) [2023-12-27 15:44:09,817][20347] Updated weights for policy 0, policy_version 1456184 (0.0011) [2023-12-27 15:44:10,405][20381] Updated weights for policy 1, policy_version 1454777 (0.0008) [2023-12-27 15:44:10,457][20381] Updated weights for policy 1, policy_version 1454787 (0.0008) [2023-12-27 15:44:10,511][20381] Updated weights for policy 1, policy_version 1454797 (0.0006) [2023-12-27 15:44:10,527][20347] Updated weights for policy 0, policy_version 1456194 (0.0010) [2023-12-27 15:44:10,586][20347] Updated weights for policy 0, policy_version 1456204 (0.0010) [2023-12-27 15:44:10,635][20347] Updated weights for policy 0, policy_version 1456214 (0.0008) [2023-12-27 15:44:11,283][20381] Updated weights for policy 1, policy_version 1454807 (0.0008) [2023-12-27 15:44:11,346][20347] Updated weights for policy 0, policy_version 1456224 (0.0008) [2023-12-27 15:44:11,348][20381] Updated weights for policy 1, policy_version 1454817 (0.0010) [2023-12-27 15:44:11,419][20381] Updated weights for policy 1, policy_version 1454827 (0.0007) [2023-12-27 15:44:11,419][20347] Updated weights for policy 0, policy_version 1456234 (0.0007) [2023-12-27 15:44:11,480][20347] Updated weights for policy 0, policy_version 1456244 (0.0006) [2023-12-27 15:44:12,148][20381] Updated weights for policy 1, policy_version 1454837 (0.0007) [2023-12-27 15:44:12,207][20347] Updated weights for policy 0, policy_version 1456254 (0.0007) [2023-12-27 15:44:12,208][20381] Updated weights for policy 1, policy_version 1454847 (0.0006) [2023-12-27 15:44:12,266][20347] Updated weights for policy 0, policy_version 1456264 (0.0007) [2023-12-27 15:44:12,267][20381] Updated weights for policy 1, policy_version 1454857 (0.0007) [2023-12-27 15:44:12,328][20347] Updated weights for policy 0, policy_version 1456274 (0.0006) [2023-12-27 15:44:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 745349120. Throughput: 0: 10380.9, 1: 10224.0. Samples: 745341924. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:12,363][19170] Avg episode reward: [(0, '539.670'), (1, '441.620')] [2023-12-27 15:44:12,958][20381] Updated weights for policy 1, policy_version 1454867 (0.0007) [2023-12-27 15:44:12,984][20347] Updated weights for policy 0, policy_version 1456284 (0.0007) [2023-12-27 15:44:13,012][20381] Updated weights for policy 1, policy_version 1454877 (0.0006) [2023-12-27 15:44:13,045][20347] Updated weights for policy 0, policy_version 1456294 (0.0006) [2023-12-27 15:44:13,065][20381] Updated weights for policy 1, policy_version 1454887 (0.0006) [2023-12-27 15:44:13,106][20347] Updated weights for policy 0, policy_version 1456304 (0.0007) [2023-12-27 15:44:13,724][20381] Updated weights for policy 1, policy_version 1454897 (0.0007) [2023-12-27 15:44:13,785][20381] Updated weights for policy 1, policy_version 1454907 (0.0005) [2023-12-27 15:44:13,805][20347] Updated weights for policy 0, policy_version 1456314 (0.0008) [2023-12-27 15:44:13,840][20381] Updated weights for policy 1, policy_version 1454917 (0.0006) [2023-12-27 15:44:13,864][20347] Updated weights for policy 0, policy_version 1456324 (0.0006) [2023-12-27 15:44:13,896][20381] Updated weights for policy 1, policy_version 1454927 (0.0005) [2023-12-27 15:44:13,920][20347] Updated weights for policy 0, policy_version 1456334 (0.0007) [2023-12-27 15:44:13,985][20347] Updated weights for policy 0, policy_version 1456344 (0.0008) [2023-12-27 15:44:14,423][20381] Updated weights for policy 1, policy_version 1454937 (0.0007) [2023-12-27 15:44:14,474][20381] Updated weights for policy 1, policy_version 1454947 (0.0008) [2023-12-27 15:44:14,523][20381] Updated weights for policy 1, policy_version 1454957 (0.0005) [2023-12-27 15:44:14,716][20347] Updated weights for policy 0, policy_version 1456354 (0.0007) [2023-12-27 15:44:14,776][20347] Updated weights for policy 0, policy_version 1456364 (0.0008) [2023-12-27 15:44:14,840][20347] Updated weights for policy 0, policy_version 1456374 (0.0008) [2023-12-27 15:44:15,165][20381] Updated weights for policy 1, policy_version 1454967 (0.0007) [2023-12-27 15:44:15,238][20381] Updated weights for policy 1, policy_version 1454977 (0.0008) [2023-12-27 15:44:15,298][20381] Updated weights for policy 1, policy_version 1454987 (0.0009) [2023-12-27 15:44:15,555][20347] Updated weights for policy 0, policy_version 1456384 (0.0008) [2023-12-27 15:44:15,612][20347] Updated weights for policy 0, policy_version 1456394 (0.0008) [2023-12-27 15:44:15,669][20347] Updated weights for policy 0, policy_version 1456404 (0.0007) [2023-12-27 15:44:15,933][20381] Updated weights for policy 1, policy_version 1454997 (0.0008) [2023-12-27 15:44:15,987][20381] Updated weights for policy 1, policy_version 1455007 (0.0007) [2023-12-27 15:44:16,039][20381] Updated weights for policy 1, policy_version 1455017 (0.0007) [2023-12-27 15:44:16,320][20347] Updated weights for policy 0, policy_version 1456414 (0.0007) [2023-12-27 15:44:16,365][20347] Updated weights for policy 0, policy_version 1456424 (0.0008) [2023-12-27 15:44:16,424][20347] Updated weights for policy 0, policy_version 1456434 (0.0007) [2023-12-27 15:44:16,731][20381] Updated weights for policy 1, policy_version 1455027 (0.0008) [2023-12-27 15:44:16,787][20381] Updated weights for policy 1, policy_version 1455037 (0.0007) [2023-12-27 15:44:16,845][20381] Updated weights for policy 1, policy_version 1455047 (0.0007) [2023-12-27 15:44:17,117][20347] Updated weights for policy 0, policy_version 1456444 (0.0007) [2023-12-27 15:44:17,174][20347] Updated weights for policy 0, policy_version 1456454 (0.0005) [2023-12-27 15:44:17,229][20347] Updated weights for policy 0, policy_version 1456464 (0.0006) [2023-12-27 15:44:17,363][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.4, 300 sec: 20605.0). Total num frames: 745455616. Throughput: 0: 10328.4, 1: 10242.2. Samples: 745463508. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:17,364][19170] Avg episode reward: [(0, '558.630'), (1, '525.080')] [2023-12-27 15:44:17,525][20381] Updated weights for policy 1, policy_version 1455057 (0.0008) [2023-12-27 15:44:17,583][20381] Updated weights for policy 1, policy_version 1455067 (0.0008) [2023-12-27 15:44:17,640][20381] Updated weights for policy 1, policy_version 1455077 (0.0008) [2023-12-27 15:44:17,696][20381] Updated weights for policy 1, policy_version 1455087 (0.0009) [2023-12-27 15:44:17,847][20347] Updated weights for policy 0, policy_version 1456474 (0.0006) [2023-12-27 15:44:17,910][20347] Updated weights for policy 0, policy_version 1456484 (0.0006) [2023-12-27 15:44:17,969][20347] Updated weights for policy 0, policy_version 1456494 (0.0006) [2023-12-27 15:44:18,030][20347] Updated weights for policy 0, policy_version 1456504 (0.0006) [2023-12-27 15:44:18,431][20381] Updated weights for policy 1, policy_version 1455097 (0.0008) [2023-12-27 15:44:18,490][20381] Updated weights for policy 1, policy_version 1455107 (0.0008) [2023-12-27 15:44:18,548][20381] Updated weights for policy 1, policy_version 1455117 (0.0007) [2023-12-27 15:44:18,618][20347] Updated weights for policy 0, policy_version 1456514 (0.0007) [2023-12-27 15:44:18,674][20347] Updated weights for policy 0, policy_version 1456524 (0.0007) [2023-12-27 15:44:18,740][20347] Updated weights for policy 0, policy_version 1456534 (0.0008) [2023-12-27 15:44:19,198][20381] Updated weights for policy 1, policy_version 1455127 (0.0010) [2023-12-27 15:44:19,266][20381] Updated weights for policy 1, policy_version 1455137 (0.0011) [2023-12-27 15:44:19,325][20381] Updated weights for policy 1, policy_version 1455147 (0.0010) [2023-12-27 15:44:19,442][20347] Updated weights for policy 0, policy_version 1456544 (0.0009) [2023-12-27 15:44:19,499][20347] Updated weights for policy 0, policy_version 1456554 (0.0008) [2023-12-27 15:44:19,561][20347] Updated weights for policy 0, policy_version 1456564 (0.0006) [2023-12-27 15:44:20,023][20381] Updated weights for policy 1, policy_version 1455157 (0.0007) [2023-12-27 15:44:20,077][20381] Updated weights for policy 1, policy_version 1455167 (0.0008) [2023-12-27 15:44:20,132][20381] Updated weights for policy 1, policy_version 1455177 (0.0008) [2023-12-27 15:44:20,280][20347] Updated weights for policy 0, policy_version 1456574 (0.0008) [2023-12-27 15:44:20,331][20347] Updated weights for policy 0, policy_version 1456584 (0.0008) [2023-12-27 15:44:20,386][20347] Updated weights for policy 0, policy_version 1456594 (0.0006) [2023-12-27 15:44:20,824][20381] Updated weights for policy 1, policy_version 1455187 (0.0007) [2023-12-27 15:44:20,880][20381] Updated weights for policy 1, policy_version 1455197 (0.0006) [2023-12-27 15:44:20,934][20381] Updated weights for policy 1, policy_version 1455207 (0.0008) [2023-12-27 15:44:20,996][20347] Updated weights for policy 0, policy_version 1456604 (0.0007) [2023-12-27 15:44:21,069][20347] Updated weights for policy 0, policy_version 1456614 (0.0007) [2023-12-27 15:44:21,133][20347] Updated weights for policy 0, policy_version 1456624 (0.0007) [2023-12-27 15:44:21,611][20381] Updated weights for policy 1, policy_version 1455217 (0.0007) [2023-12-27 15:44:21,677][20381] Updated weights for policy 1, policy_version 1455227 (0.0008) [2023-12-27 15:44:21,743][20381] Updated weights for policy 1, policy_version 1455237 (0.0008) [2023-12-27 15:44:21,810][20381] Updated weights for policy 1, policy_version 1455247 (0.0008) [2023-12-27 15:44:21,842][20347] Updated weights for policy 0, policy_version 1456634 (0.0006) [2023-12-27 15:44:21,901][20347] Updated weights for policy 0, policy_version 1456644 (0.0007) [2023-12-27 15:44:21,958][20347] Updated weights for policy 0, policy_version 1456654 (0.0008) [2023-12-27 15:44:22,020][20347] Updated weights for policy 0, policy_version 1456664 (0.0007) [2023-12-27 15:44:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 745553920. Throughput: 0: 10354.5, 1: 10150.6. Samples: 745524516. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:22,363][19170] Avg episode reward: [(0, '529.370'), (1, '469.410')] [2023-12-27 15:44:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001455248_372572160.pth... [2023-12-27 15:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001456664_372981760.pth... [2023-12-27 15:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001454064_372269056.pth [2023-12-27 15:44:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001455480_372678656.pth [2023-12-27 15:44:22,593][20381] Updated weights for policy 1, policy_version 1455257 (0.0006) [2023-12-27 15:44:22,646][20347] Updated weights for policy 0, policy_version 1456674 (0.0006) [2023-12-27 15:44:22,651][20381] Updated weights for policy 1, policy_version 1455267 (0.0006) [2023-12-27 15:44:22,704][20347] Updated weights for policy 0, policy_version 1456684 (0.0007) [2023-12-27 15:44:22,706][20381] Updated weights for policy 1, policy_version 1455277 (0.0006) [2023-12-27 15:44:22,762][20347] Updated weights for policy 0, policy_version 1456694 (0.0007) [2023-12-27 15:44:23,366][20381] Updated weights for policy 1, policy_version 1455287 (0.0006) [2023-12-27 15:44:23,381][20347] Updated weights for policy 0, policy_version 1456704 (0.0008) [2023-12-27 15:44:23,423][20381] Updated weights for policy 1, policy_version 1455297 (0.0005) [2023-12-27 15:44:23,430][20347] Updated weights for policy 0, policy_version 1456714 (0.0007) [2023-12-27 15:44:23,478][20381] Updated weights for policy 1, policy_version 1455307 (0.0005) [2023-12-27 15:44:23,482][20347] Updated weights for policy 0, policy_version 1456724 (0.0008) [2023-12-27 15:44:24,033][20381] Updated weights for policy 1, policy_version 1455317 (0.0008) [2023-12-27 15:44:24,082][20381] Updated weights for policy 1, policy_version 1455327 (0.0010) [2023-12-27 15:44:24,125][20347] Updated weights for policy 0, policy_version 1456734 (0.0006) [2023-12-27 15:44:24,137][20381] Updated weights for policy 1, policy_version 1455337 (0.0008) [2023-12-27 15:44:24,191][20347] Updated weights for policy 0, policy_version 1456744 (0.0005) [2023-12-27 15:44:24,258][20347] Updated weights for policy 0, policy_version 1456754 (0.0005) [2023-12-27 15:44:24,754][20347] Updated weights for policy 0, policy_version 1456764 (0.0009) [2023-12-27 15:44:24,803][20347] Updated weights for policy 0, policy_version 1456774 (0.0008) [2023-12-27 15:44:24,818][20381] Updated weights for policy 1, policy_version 1455347 (0.0007) [2023-12-27 15:44:24,859][20347] Updated weights for policy 0, policy_version 1456784 (0.0010) [2023-12-27 15:44:24,879][20381] Updated weights for policy 1, policy_version 1455357 (0.0005) [2023-12-27 15:44:24,945][20381] Updated weights for policy 1, policy_version 1455367 (0.0005) [2023-12-27 15:44:25,436][20347] Updated weights for policy 0, policy_version 1456794 (0.0009) [2023-12-27 15:44:25,466][20381] Updated weights for policy 1, policy_version 1455377 (0.0006) [2023-12-27 15:44:25,499][20347] Updated weights for policy 0, policy_version 1456804 (0.0010) [2023-12-27 15:44:25,519][20381] Updated weights for policy 1, policy_version 1455387 (0.0006) [2023-12-27 15:44:25,553][20347] Updated weights for policy 0, policy_version 1456814 (0.0010) [2023-12-27 15:44:25,569][20381] Updated weights for policy 1, policy_version 1455397 (0.0005) [2023-12-27 15:44:25,608][20347] Updated weights for policy 0, policy_version 1456824 (0.0010) [2023-12-27 15:44:25,614][20381] Updated weights for policy 1, policy_version 1455407 (0.0005) [2023-12-27 15:44:26,202][20347] Updated weights for policy 0, policy_version 1456834 (0.0005) [2023-12-27 15:44:26,253][20347] Updated weights for policy 0, policy_version 1456844 (0.0010) [2023-12-27 15:44:26,304][20347] Updated weights for policy 0, policy_version 1456854 (0.0010) [2023-12-27 15:44:26,306][20381] Updated weights for policy 1, policy_version 1455417 (0.0005) [2023-12-27 15:44:26,364][20381] Updated weights for policy 1, policy_version 1455427 (0.0005) [2023-12-27 15:44:26,422][20381] Updated weights for policy 1, policy_version 1455437 (0.0005) [2023-12-27 15:44:26,939][20347] Updated weights for policy 0, policy_version 1456864 (0.0010) [2023-12-27 15:44:26,991][20381] Updated weights for policy 1, policy_version 1455447 (0.0005) [2023-12-27 15:44:26,993][20347] Updated weights for policy 0, policy_version 1456874 (0.0010) [2023-12-27 15:44:27,044][20381] Updated weights for policy 1, policy_version 1455457 (0.0005) [2023-12-27 15:44:27,049][20347] Updated weights for policy 0, policy_version 1456884 (0.0010) [2023-12-27 15:44:27,097][20381] Updated weights for policy 1, policy_version 1455467 (0.0006) [2023-12-27 15:44:27,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 745668608. Throughput: 0: 10349.4, 1: 10195.5. Samples: 745654048. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:27,363][19170] Avg episode reward: [(0, '568.670'), (1, '467.550')] [2023-12-27 15:44:27,604][20347] Updated weights for policy 0, policy_version 1456894 (0.0010) [2023-12-27 15:44:27,657][20347] Updated weights for policy 0, policy_version 1456904 (0.0010) [2023-12-27 15:44:27,713][20347] Updated weights for policy 0, policy_version 1456914 (0.0010) [2023-12-27 15:44:27,791][20381] Updated weights for policy 1, policy_version 1455477 (0.0007) [2023-12-27 15:44:27,839][20381] Updated weights for policy 1, policy_version 1455487 (0.0006) [2023-12-27 15:44:27,886][20381] Updated weights for policy 1, policy_version 1455497 (0.0006) [2023-12-27 15:44:28,363][20347] Updated weights for policy 0, policy_version 1456924 (0.0008) [2023-12-27 15:44:28,423][20347] Updated weights for policy 0, policy_version 1456934 (0.0007) [2023-12-27 15:44:28,476][20347] Updated weights for policy 0, policy_version 1456944 (0.0007) [2023-12-27 15:44:28,507][20381] Updated weights for policy 1, policy_version 1455507 (0.0007) [2023-12-27 15:44:28,568][20381] Updated weights for policy 1, policy_version 1455517 (0.0005) [2023-12-27 15:44:28,614][20381] Updated weights for policy 1, policy_version 1455527 (0.0009) [2023-12-27 15:44:29,171][20347] Updated weights for policy 0, policy_version 1456954 (0.0007) [2023-12-27 15:44:29,239][20347] Updated weights for policy 0, policy_version 1456964 (0.0008) [2023-12-27 15:44:29,263][20381] Updated weights for policy 1, policy_version 1455537 (0.0009) [2023-12-27 15:44:29,295][20347] Updated weights for policy 0, policy_version 1456974 (0.0007) [2023-12-27 15:44:29,322][20381] Updated weights for policy 1, policy_version 1455547 (0.0007) [2023-12-27 15:44:29,362][20347] Updated weights for policy 0, policy_version 1456984 (0.0008) [2023-12-27 15:44:29,387][20381] Updated weights for policy 1, policy_version 1455557 (0.0009) [2023-12-27 15:44:29,437][20381] Updated weights for policy 1, policy_version 1455567 (0.0009) [2023-12-27 15:44:30,058][20347] Updated weights for policy 0, policy_version 1456994 (0.0008) [2023-12-27 15:44:30,124][20347] Updated weights for policy 0, policy_version 1457004 (0.0008) [2023-12-27 15:44:30,179][20381] Updated weights for policy 1, policy_version 1455577 (0.0007) [2023-12-27 15:44:30,181][20347] Updated weights for policy 0, policy_version 1457014 (0.0006) [2023-12-27 15:44:30,239][20381] Updated weights for policy 1, policy_version 1455587 (0.0007) [2023-12-27 15:44:30,301][20381] Updated weights for policy 1, policy_version 1455597 (0.0008) [2023-12-27 15:44:30,825][20347] Updated weights for policy 0, policy_version 1457024 (0.0005) [2023-12-27 15:44:30,883][20347] Updated weights for policy 0, policy_version 1457034 (0.0005) [2023-12-27 15:44:30,939][20347] Updated weights for policy 0, policy_version 1457044 (0.0005) [2023-12-27 15:44:31,032][20381] Updated weights for policy 1, policy_version 1455607 (0.0008) [2023-12-27 15:44:31,102][20381] Updated weights for policy 1, policy_version 1455617 (0.0009) [2023-12-27 15:44:31,165][20381] Updated weights for policy 1, policy_version 1455627 (0.0008) [2023-12-27 15:44:31,506][20347] Updated weights for policy 0, policy_version 1457054 (0.0005) [2023-12-27 15:44:31,566][20347] Updated weights for policy 0, policy_version 1457064 (0.0006) [2023-12-27 15:44:31,624][20347] Updated weights for policy 0, policy_version 1457074 (0.0007) [2023-12-27 15:44:31,902][20381] Updated weights for policy 1, policy_version 1455637 (0.0007) [2023-12-27 15:44:31,949][20381] Updated weights for policy 1, policy_version 1455647 (0.0008) [2023-12-27 15:44:32,005][20381] Updated weights for policy 1, policy_version 1455657 (0.0008) [2023-12-27 15:44:32,254][20347] Updated weights for policy 0, policy_version 1457084 (0.0007) [2023-12-27 15:44:32,316][20347] Updated weights for policy 0, policy_version 1457094 (0.0008) [2023-12-27 15:44:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 745766912. Throughput: 0: 10392.7, 1: 10179.8. Samples: 745778856. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:32,363][19170] Avg episode reward: [(0, '519.860'), (1, '450.290')] [2023-12-27 15:44:32,388][20347] Updated weights for policy 0, policy_version 1457104 (0.0008) [2023-12-27 15:44:32,765][20381] Updated weights for policy 1, policy_version 1455667 (0.0008) [2023-12-27 15:44:32,826][20381] Updated weights for policy 1, policy_version 1455677 (0.0007) [2023-12-27 15:44:32,893][20381] Updated weights for policy 1, policy_version 1455687 (0.0008) [2023-12-27 15:44:33,067][20347] Updated weights for policy 0, policy_version 1457114 (0.0007) [2023-12-27 15:44:33,111][20347] Updated weights for policy 0, policy_version 1457124 (0.0007) [2023-12-27 15:44:33,163][20347] Updated weights for policy 0, policy_version 1457134 (0.0008) [2023-12-27 15:44:33,209][20347] Updated weights for policy 0, policy_version 1457144 (0.0007) [2023-12-27 15:44:33,574][20381] Updated weights for policy 1, policy_version 1455697 (0.0007) [2023-12-27 15:44:33,633][20381] Updated weights for policy 1, policy_version 1455707 (0.0007) [2023-12-27 15:44:33,695][20381] Updated weights for policy 1, policy_version 1455717 (0.0006) [2023-12-27 15:44:33,758][20381] Updated weights for policy 1, policy_version 1455727 (0.0008) [2023-12-27 15:44:33,875][20347] Updated weights for policy 0, policy_version 1457154 (0.0006) [2023-12-27 15:44:33,933][20347] Updated weights for policy 0, policy_version 1457164 (0.0008) [2023-12-27 15:44:33,981][20347] Updated weights for policy 0, policy_version 1457174 (0.0008) [2023-12-27 15:44:34,463][20381] Updated weights for policy 1, policy_version 1455737 (0.0008) [2023-12-27 15:44:34,521][20381] Updated weights for policy 1, policy_version 1455747 (0.0007) [2023-12-27 15:44:34,579][20381] Updated weights for policy 1, policy_version 1455757 (0.0008) [2023-12-27 15:44:34,611][20347] Updated weights for policy 0, policy_version 1457184 (0.0008) [2023-12-27 15:44:34,680][20347] Updated weights for policy 0, policy_version 1457194 (0.0008) [2023-12-27 15:44:34,745][20347] Updated weights for policy 0, policy_version 1457204 (0.0008) [2023-12-27 15:44:35,223][20381] Updated weights for policy 1, policy_version 1455767 (0.0007) [2023-12-27 15:44:35,268][20381] Updated weights for policy 1, policy_version 1455777 (0.0007) [2023-12-27 15:44:35,319][20381] Updated weights for policy 1, policy_version 1455787 (0.0007) [2023-12-27 15:44:35,365][20347] Updated weights for policy 0, policy_version 1457214 (0.0007) [2023-12-27 15:44:35,417][20347] Updated weights for policy 0, policy_version 1457224 (0.0008) [2023-12-27 15:44:35,469][20347] Updated weights for policy 0, policy_version 1457234 (0.0008) [2023-12-27 15:44:36,017][20381] Updated weights for policy 1, policy_version 1455797 (0.0009) [2023-12-27 15:44:36,059][20347] Updated weights for policy 0, policy_version 1457244 (0.0008) [2023-12-27 15:44:36,075][20381] Updated weights for policy 1, policy_version 1455807 (0.0011) [2023-12-27 15:44:36,110][20347] Updated weights for policy 0, policy_version 1457254 (0.0006) [2023-12-27 15:44:36,152][20381] Updated weights for policy 1, policy_version 1455817 (0.0010) [2023-12-27 15:44:36,170][20347] Updated weights for policy 0, policy_version 1457264 (0.0008) [2023-12-27 15:44:36,781][20381] Updated weights for policy 1, policy_version 1455827 (0.0009) [2023-12-27 15:44:36,783][20347] Updated weights for policy 0, policy_version 1457274 (0.0008) [2023-12-27 15:44:36,830][20347] Updated weights for policy 0, policy_version 1457284 (0.0005) [2023-12-27 15:44:36,843][20381] Updated weights for policy 1, policy_version 1455837 (0.0008) [2023-12-27 15:44:36,881][20347] Updated weights for policy 0, policy_version 1457294 (0.0005) [2023-12-27 15:44:36,902][20381] Updated weights for policy 1, policy_version 1455847 (0.0005) [2023-12-27 15:44:36,940][20347] Updated weights for policy 0, policy_version 1457304 (0.0005) [2023-12-27 15:44:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20632.7). Total num frames: 745873408. Throughput: 0: 10431.0, 1: 10149.9. Samples: 745840888. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:37,363][19170] Avg episode reward: [(0, '587.870'), (1, '483.520')] [2023-12-27 15:44:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001457304_373145600.pth... [2023-12-27 15:44:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001455856_372727808.pth... [2023-12-27 15:44:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001456088_372834304.pth [2023-12-27 15:44:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001454672_372424704.pth [2023-12-27 15:44:37,513][20347] Updated weights for policy 0, policy_version 1457314 (0.0007) [2023-12-27 15:44:37,528][20381] Updated weights for policy 1, policy_version 1455857 (0.0007) [2023-12-27 15:44:37,571][20347] Updated weights for policy 0, policy_version 1457324 (0.0008) [2023-12-27 15:44:37,577][20381] Updated weights for policy 1, policy_version 1455867 (0.0011) [2023-12-27 15:44:37,626][20347] Updated weights for policy 0, policy_version 1457334 (0.0006) [2023-12-27 15:44:37,640][20381] Updated weights for policy 1, policy_version 1455877 (0.0009) [2023-12-27 15:44:37,696][20381] Updated weights for policy 1, policy_version 1455887 (0.0008) [2023-12-27 15:44:38,334][20347] Updated weights for policy 0, policy_version 1457344 (0.0008) [2023-12-27 15:44:38,389][20347] Updated weights for policy 0, policy_version 1457354 (0.0007) [2023-12-27 15:44:38,403][20381] Updated weights for policy 1, policy_version 1455897 (0.0009) [2023-12-27 15:44:38,448][20347] Updated weights for policy 0, policy_version 1457364 (0.0006) [2023-12-27 15:44:38,456][20381] Updated weights for policy 1, policy_version 1455907 (0.0008) [2023-12-27 15:44:38,507][20381] Updated weights for policy 1, policy_version 1455917 (0.0008) [2023-12-27 15:44:39,148][20381] Updated weights for policy 1, policy_version 1455927 (0.0008) [2023-12-27 15:44:39,167][20347] Updated weights for policy 0, policy_version 1457374 (0.0008) [2023-12-27 15:44:39,201][20381] Updated weights for policy 1, policy_version 1455937 (0.0007) [2023-12-27 15:44:39,239][20347] Updated weights for policy 0, policy_version 1457384 (0.0010) [2023-12-27 15:44:39,263][20381] Updated weights for policy 1, policy_version 1455947 (0.0007) [2023-12-27 15:44:39,306][20347] Updated weights for policy 0, policy_version 1457394 (0.0010) [2023-12-27 15:44:39,980][20381] Updated weights for policy 1, policy_version 1455957 (0.0008) [2023-12-27 15:44:40,030][20347] Updated weights for policy 0, policy_version 1457404 (0.0010) [2023-12-27 15:44:40,044][20381] Updated weights for policy 1, policy_version 1455967 (0.0009) [2023-12-27 15:44:40,087][20347] Updated weights for policy 0, policy_version 1457414 (0.0010) [2023-12-27 15:44:40,100][20381] Updated weights for policy 1, policy_version 1455977 (0.0010) [2023-12-27 15:44:40,144][20347] Updated weights for policy 0, policy_version 1457424 (0.0011) [2023-12-27 15:44:40,767][20381] Updated weights for policy 1, policy_version 1455987 (0.0011) [2023-12-27 15:44:40,813][20381] Updated weights for policy 1, policy_version 1455997 (0.0010) [2023-12-27 15:44:40,831][20347] Updated weights for policy 0, policy_version 1457434 (0.0010) [2023-12-27 15:44:40,868][20381] Updated weights for policy 1, policy_version 1456007 (0.0010) [2023-12-27 15:44:40,887][20347] Updated weights for policy 0, policy_version 1457444 (0.0009) [2023-12-27 15:44:40,942][20347] Updated weights for policy 0, policy_version 1457454 (0.0005) [2023-12-27 15:44:41,002][20347] Updated weights for policy 0, policy_version 1457464 (0.0010) [2023-12-27 15:44:41,685][20381] Updated weights for policy 1, policy_version 1456017 (0.0010) [2023-12-27 15:44:41,737][20347] Updated weights for policy 0, policy_version 1457474 (0.0007) [2023-12-27 15:44:41,753][20381] Updated weights for policy 1, policy_version 1456027 (0.0011) [2023-12-27 15:44:41,798][20347] Updated weights for policy 0, policy_version 1457484 (0.0007) [2023-12-27 15:44:41,815][20381] Updated weights for policy 1, policy_version 1456037 (0.0007) [2023-12-27 15:44:41,853][20347] Updated weights for policy 0, policy_version 1457494 (0.0006) [2023-12-27 15:44:41,875][20381] Updated weights for policy 1, policy_version 1456047 (0.0007) [2023-12-27 15:44:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20481.2, 300 sec: 20632.7). Total num frames: 745971712. Throughput: 0: 10388.6, 1: 10194.5. Samples: 745964608. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:42,363][19170] Avg episode reward: [(0, '552.950'), (1, '552.660')] [2023-12-27 15:44:42,490][20347] Updated weights for policy 0, policy_version 1457504 (0.0005) [2023-12-27 15:44:42,543][20347] Updated weights for policy 0, policy_version 1457514 (0.0006) [2023-12-27 15:44:42,597][20347] Updated weights for policy 0, policy_version 1457524 (0.0005) [2023-12-27 15:44:42,607][20381] Updated weights for policy 1, policy_version 1456057 (0.0007) [2023-12-27 15:44:42,665][20381] Updated weights for policy 1, policy_version 1456067 (0.0007) [2023-12-27 15:44:42,721][20381] Updated weights for policy 1, policy_version 1456077 (0.0007) [2023-12-27 15:44:43,293][20347] Updated weights for policy 0, policy_version 1457534 (0.0007) [2023-12-27 15:44:43,344][20347] Updated weights for policy 0, policy_version 1457544 (0.0007) [2023-12-27 15:44:43,393][20381] Updated weights for policy 1, policy_version 1456087 (0.0006) [2023-12-27 15:44:43,395][20347] Updated weights for policy 0, policy_version 1457554 (0.0008) [2023-12-27 15:44:43,448][20381] Updated weights for policy 1, policy_version 1456097 (0.0006) [2023-12-27 15:44:43,503][20381] Updated weights for policy 1, policy_version 1456107 (0.0009) [2023-12-27 15:44:43,970][20347] Updated weights for policy 0, policy_version 1457564 (0.0007) [2023-12-27 15:44:44,023][20347] Updated weights for policy 0, policy_version 1457574 (0.0005) [2023-12-27 15:44:44,071][20347] Updated weights for policy 0, policy_version 1457584 (0.0005) [2023-12-27 15:44:44,099][20381] Updated weights for policy 1, policy_version 1456117 (0.0007) [2023-12-27 15:44:44,161][20381] Updated weights for policy 1, policy_version 1456127 (0.0011) [2023-12-27 15:44:44,227][20381] Updated weights for policy 1, policy_version 1456137 (0.0010) [2023-12-27 15:44:44,642][20347] Updated weights for policy 0, policy_version 1457594 (0.0006) [2023-12-27 15:44:44,690][20347] Updated weights for policy 0, policy_version 1457604 (0.0010) [2023-12-27 15:44:44,739][20347] Updated weights for policy 0, policy_version 1457614 (0.0010) [2023-12-27 15:44:44,801][20347] Updated weights for policy 0, policy_version 1457624 (0.0011) [2023-12-27 15:44:44,822][20381] Updated weights for policy 1, policy_version 1456147 (0.0010) [2023-12-27 15:44:44,879][20381] Updated weights for policy 1, policy_version 1456157 (0.0011) [2023-12-27 15:44:44,935][20381] Updated weights for policy 1, policy_version 1456167 (0.0011) [2023-12-27 15:44:45,530][20347] Updated weights for policy 0, policy_version 1457634 (0.0010) [2023-12-27 15:44:45,585][20347] Updated weights for policy 0, policy_version 1457644 (0.0010) [2023-12-27 15:44:45,604][20381] Updated weights for policy 1, policy_version 1456177 (0.0009) [2023-12-27 15:44:45,645][20347] Updated weights for policy 0, policy_version 1457654 (0.0007) [2023-12-27 15:44:45,663][20381] Updated weights for policy 1, policy_version 1456187 (0.0011) [2023-12-27 15:44:45,725][20381] Updated weights for policy 1, policy_version 1456197 (0.0011) [2023-12-27 15:44:45,781][20381] Updated weights for policy 1, policy_version 1456207 (0.0011) [2023-12-27 15:44:46,347][20347] Updated weights for policy 0, policy_version 1457664 (0.0010) [2023-12-27 15:44:46,407][20347] Updated weights for policy 0, policy_version 1457674 (0.0010) [2023-12-27 15:44:46,438][20381] Updated weights for policy 1, policy_version 1456217 (0.0006) [2023-12-27 15:44:46,459][20347] Updated weights for policy 0, policy_version 1457684 (0.0009) [2023-12-27 15:44:46,492][20381] Updated weights for policy 1, policy_version 1456227 (0.0006) [2023-12-27 15:44:46,545][20381] Updated weights for policy 1, policy_version 1456237 (0.0005) [2023-12-27 15:44:47,087][20347] Updated weights for policy 0, policy_version 1457694 (0.0007) [2023-12-27 15:44:47,136][20381] Updated weights for policy 1, policy_version 1456247 (0.0006) [2023-12-27 15:44:47,137][20347] Updated weights for policy 0, policy_version 1457704 (0.0010) [2023-12-27 15:44:47,187][20381] Updated weights for policy 1, policy_version 1456257 (0.0005) [2023-12-27 15:44:47,194][20347] Updated weights for policy 0, policy_version 1457714 (0.0010) [2023-12-27 15:44:47,235][20381] Updated weights for policy 1, policy_version 1456267 (0.0007) [2023-12-27 15:44:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20689.1). Total num frames: 746086400. Throughput: 0: 10425.4, 1: 10240.7. Samples: 746090272. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:47,363][19170] Avg episode reward: [(0, '619.580'), (1, '586.110')] [2023-12-27 15:44:47,806][20381] Updated weights for policy 1, policy_version 1456277 (0.0005) [2023-12-27 15:44:47,820][20347] Updated weights for policy 0, policy_version 1457724 (0.0010) [2023-12-27 15:44:47,857][20381] Updated weights for policy 1, policy_version 1456287 (0.0005) [2023-12-27 15:44:47,869][20347] Updated weights for policy 0, policy_version 1457734 (0.0010) [2023-12-27 15:44:47,914][20381] Updated weights for policy 1, policy_version 1456297 (0.0005) [2023-12-27 15:44:47,925][20347] Updated weights for policy 0, policy_version 1457744 (0.0010) [2023-12-27 15:44:48,536][20381] Updated weights for policy 1, policy_version 1456307 (0.0008) [2023-12-27 15:44:48,593][20347] Updated weights for policy 0, policy_version 1457754 (0.0006) [2023-12-27 15:44:48,594][20381] Updated weights for policy 1, policy_version 1456317 (0.0007) [2023-12-27 15:44:48,651][20381] Updated weights for policy 1, policy_version 1456327 (0.0007) [2023-12-27 15:44:48,660][20347] Updated weights for policy 0, policy_version 1457764 (0.0009) [2023-12-27 15:44:48,717][20347] Updated weights for policy 0, policy_version 1457774 (0.0010) [2023-12-27 15:44:48,770][20347] Updated weights for policy 0, policy_version 1457784 (0.0010) [2023-12-27 15:44:49,334][20381] Updated weights for policy 1, policy_version 1456337 (0.0006) [2023-12-27 15:44:49,396][20381] Updated weights for policy 1, policy_version 1456347 (0.0007) [2023-12-27 15:44:49,444][20381] Updated weights for policy 1, policy_version 1456357 (0.0007) [2023-12-27 15:44:49,487][20347] Updated weights for policy 0, policy_version 1457794 (0.0006) [2023-12-27 15:44:49,504][20381] Updated weights for policy 1, policy_version 1456367 (0.0008) [2023-12-27 15:44:49,545][20347] Updated weights for policy 0, policy_version 1457804 (0.0006) [2023-12-27 15:44:49,604][20347] Updated weights for policy 0, policy_version 1457814 (0.0007) [2023-12-27 15:44:50,213][20381] Updated weights for policy 1, policy_version 1456377 (0.0007) [2023-12-27 15:44:50,279][20381] Updated weights for policy 1, policy_version 1456387 (0.0007) [2023-12-27 15:44:50,308][20347] Updated weights for policy 0, policy_version 1457824 (0.0006) [2023-12-27 15:44:50,345][20381] Updated weights for policy 1, policy_version 1456397 (0.0007) [2023-12-27 15:44:50,366][20347] Updated weights for policy 0, policy_version 1457834 (0.0006) [2023-12-27 15:44:50,418][20347] Updated weights for policy 0, policy_version 1457844 (0.0007) [2023-12-27 15:44:51,029][20381] Updated weights for policy 1, policy_version 1456407 (0.0007) [2023-12-27 15:44:51,058][20347] Updated weights for policy 0, policy_version 1457854 (0.0007) [2023-12-27 15:44:51,093][20381] Updated weights for policy 1, policy_version 1456417 (0.0009) [2023-12-27 15:44:51,122][20347] Updated weights for policy 0, policy_version 1457864 (0.0009) [2023-12-27 15:44:51,161][20381] Updated weights for policy 1, policy_version 1456427 (0.0008) [2023-12-27 15:44:51,185][20347] Updated weights for policy 0, policy_version 1457874 (0.0008) [2023-12-27 15:44:51,845][20381] Updated weights for policy 1, policy_version 1456437 (0.0008) [2023-12-27 15:44:51,900][20381] Updated weights for policy 1, policy_version 1456447 (0.0007) [2023-12-27 15:44:51,936][20347] Updated weights for policy 0, policy_version 1457884 (0.0009) [2023-12-27 15:44:51,955][20381] Updated weights for policy 1, policy_version 1456457 (0.0007) [2023-12-27 15:44:51,996][20347] Updated weights for policy 0, policy_version 1457894 (0.0010) [2023-12-27 15:44:52,054][20347] Updated weights for policy 0, policy_version 1457904 (0.0009) [2023-12-27 15:44:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 746184704. Throughput: 0: 10448.9, 1: 10236.6. Samples: 746153324. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:52,363][19170] Avg episode reward: [(0, '562.870'), (1, '716.560')] [2023-12-27 15:44:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001456464_372883456.pth... [2023-12-27 15:44:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001457912_373301248.pth... [2023-12-27 15:44:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001455248_372572160.pth [2023-12-27 15:44:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001456664_372981760.pth [2023-12-27 15:44:52,636][20381] Updated weights for policy 1, policy_version 1456467 (0.0008) [2023-12-27 15:44:52,700][20381] Updated weights for policy 1, policy_version 1456477 (0.0005) [2023-12-27 15:44:52,765][20381] Updated weights for policy 1, policy_version 1456487 (0.0007) [2023-12-27 15:44:52,773][20347] Updated weights for policy 0, policy_version 1457914 (0.0008) [2023-12-27 15:44:52,825][20347] Updated weights for policy 0, policy_version 1457924 (0.0006) [2023-12-27 15:44:52,844][20291] KL-divergence is very high: 111.6647 [2023-12-27 15:44:52,850][20291] KL-divergence is very high: 114.3999 [2023-12-27 15:44:52,873][20347] Updated weights for policy 0, policy_version 1457934 (0.0007) [2023-12-27 15:44:52,927][20347] Updated weights for policy 0, policy_version 1457944 (0.0008) [2023-12-27 15:44:53,398][20381] Updated weights for policy 1, policy_version 1456497 (0.0007) [2023-12-27 15:44:53,458][20381] Updated weights for policy 1, policy_version 1456507 (0.0007) [2023-12-27 15:44:53,513][20381] Updated weights for policy 1, policy_version 1456517 (0.0007) [2023-12-27 15:44:53,561][20381] Updated weights for policy 1, policy_version 1456527 (0.0007) [2023-12-27 15:44:53,636][20347] Updated weights for policy 0, policy_version 1457954 (0.0009) [2023-12-27 15:44:53,688][20347] Updated weights for policy 0, policy_version 1457964 (0.0008) [2023-12-27 15:44:53,738][20347] Updated weights for policy 0, policy_version 1457974 (0.0008) [2023-12-27 15:44:54,242][20381] Updated weights for policy 1, policy_version 1456537 (0.0007) [2023-12-27 15:44:54,298][20381] Updated weights for policy 1, policy_version 1456547 (0.0007) [2023-12-27 15:44:54,356][20381] Updated weights for policy 1, policy_version 1456557 (0.0007) [2023-12-27 15:44:54,433][20347] Updated weights for policy 0, policy_version 1457984 (0.0009) [2023-12-27 15:44:54,483][20347] Updated weights for policy 0, policy_version 1457994 (0.0008) [2023-12-27 15:44:54,530][20347] Updated weights for policy 0, policy_version 1458004 (0.0008) [2023-12-27 15:44:55,019][20381] Updated weights for policy 1, policy_version 1456567 (0.0007) [2023-12-27 15:44:55,073][20381] Updated weights for policy 1, policy_version 1456577 (0.0007) [2023-12-27 15:44:55,133][20381] Updated weights for policy 1, policy_version 1456587 (0.0008) [2023-12-27 15:44:55,203][20347] Updated weights for policy 0, policy_version 1458014 (0.0006) [2023-12-27 15:44:55,252][20347] Updated weights for policy 0, policy_version 1458024 (0.0005) [2023-12-27 15:44:55,306][20347] Updated weights for policy 0, policy_version 1458034 (0.0007) [2023-12-27 15:44:55,802][20381] Updated weights for policy 1, policy_version 1456597 (0.0006) [2023-12-27 15:44:55,848][20381] Updated weights for policy 1, policy_version 1456607 (0.0005) [2023-12-27 15:44:55,900][20381] Updated weights for policy 1, policy_version 1456617 (0.0007) [2023-12-27 15:44:55,981][20347] Updated weights for policy 0, policy_version 1458044 (0.0008) [2023-12-27 15:44:56,031][20347] Updated weights for policy 0, policy_version 1458054 (0.0007) [2023-12-27 15:44:56,093][20347] Updated weights for policy 0, policy_version 1458064 (0.0008) [2023-12-27 15:44:56,514][20381] Updated weights for policy 1, policy_version 1456627 (0.0008) [2023-12-27 15:44:56,572][20381] Updated weights for policy 1, policy_version 1456637 (0.0007) [2023-12-27 15:44:56,633][20381] Updated weights for policy 1, policy_version 1456647 (0.0008) [2023-12-27 15:44:56,810][20347] Updated weights for policy 0, policy_version 1458074 (0.0008) [2023-12-27 15:44:56,862][20347] Updated weights for policy 0, policy_version 1458084 (0.0007) [2023-12-27 15:44:56,913][20347] Updated weights for policy 0, policy_version 1458094 (0.0007) [2023-12-27 15:44:56,974][20347] Updated weights for policy 0, policy_version 1458104 (0.0007) [2023-12-27 15:44:57,296][20381] Updated weights for policy 1, policy_version 1456657 (0.0008) [2023-12-27 15:44:57,353][20381] Updated weights for policy 1, policy_version 1456667 (0.0007) [2023-12-27 15:44:57,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20616.4, 300 sec: 20632.7). Total num frames: 746283008. Throughput: 0: 10435.7, 1: 10285.5. Samples: 746274380. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:44:57,364][19170] Avg episode reward: [(0, '459.090'), (1, '738.110')] [2023-12-27 15:44:57,409][20381] Updated weights for policy 1, policy_version 1456677 (0.0008) [2023-12-27 15:44:57,457][20381] Updated weights for policy 1, policy_version 1456687 (0.0008) [2023-12-27 15:44:57,633][20347] Updated weights for policy 0, policy_version 1458114 (0.0007) [2023-12-27 15:44:57,686][20347] Updated weights for policy 0, policy_version 1458124 (0.0005) [2023-12-27 15:44:57,735][20347] Updated weights for policy 0, policy_version 1458134 (0.0007) [2023-12-27 15:44:58,113][20381] Updated weights for policy 1, policy_version 1456697 (0.0010) [2023-12-27 15:44:58,167][20381] Updated weights for policy 1, policy_version 1456707 (0.0010) [2023-12-27 15:44:58,230][20381] Updated weights for policy 1, policy_version 1456717 (0.0008) [2023-12-27 15:44:58,445][20347] Updated weights for policy 0, policy_version 1458144 (0.0010) [2023-12-27 15:44:58,504][20347] Updated weights for policy 0, policy_version 1458154 (0.0010) [2023-12-27 15:44:58,563][20347] Updated weights for policy 0, policy_version 1458164 (0.0010) [2023-12-27 15:44:59,055][20381] Updated weights for policy 1, policy_version 1456727 (0.0007) [2023-12-27 15:44:59,111][20381] Updated weights for policy 1, policy_version 1456737 (0.0008) [2023-12-27 15:44:59,167][20381] Updated weights for policy 1, policy_version 1456747 (0.0008) [2023-12-27 15:44:59,371][20347] Updated weights for policy 0, policy_version 1458174 (0.0009) [2023-12-27 15:44:59,433][20347] Updated weights for policy 0, policy_version 1458184 (0.0005) [2023-12-27 15:44:59,501][20347] Updated weights for policy 0, policy_version 1458194 (0.0005) [2023-12-27 15:44:59,897][20381] Updated weights for policy 1, policy_version 1456757 (0.0008) [2023-12-27 15:44:59,960][20381] Updated weights for policy 1, policy_version 1456767 (0.0008) [2023-12-27 15:45:00,022][20381] Updated weights for policy 1, policy_version 1456777 (0.0008) [2023-12-27 15:45:00,069][20347] Updated weights for policy 0, policy_version 1458204 (0.0005) [2023-12-27 15:45:00,123][20347] Updated weights for policy 0, policy_version 1458214 (0.0005) [2023-12-27 15:45:00,171][20347] Updated weights for policy 0, policy_version 1458224 (0.0007) [2023-12-27 15:45:00,781][20381] Updated weights for policy 1, policy_version 1456787 (0.0007) [2023-12-27 15:45:00,833][20381] Updated weights for policy 1, policy_version 1456797 (0.0006) [2023-12-27 15:45:00,876][20347] Updated weights for policy 0, policy_version 1458234 (0.0007) [2023-12-27 15:45:00,894][20381] Updated weights for policy 1, policy_version 1456807 (0.0007) [2023-12-27 15:45:00,932][20347] Updated weights for policy 0, policy_version 1458244 (0.0007) [2023-12-27 15:45:00,993][20347] Updated weights for policy 0, policy_version 1458254 (0.0008) [2023-12-27 15:45:01,051][20347] Updated weights for policy 0, policy_version 1458264 (0.0008) [2023-12-27 15:45:01,610][20381] Updated weights for policy 1, policy_version 1456817 (0.0006) [2023-12-27 15:45:01,673][20381] Updated weights for policy 1, policy_version 1456827 (0.0008) [2023-12-27 15:45:01,737][20381] Updated weights for policy 1, policy_version 1456837 (0.0009) [2023-12-27 15:45:01,797][20381] Updated weights for policy 1, policy_version 1456847 (0.0006) [2023-12-27 15:45:01,798][20347] Updated weights for policy 0, policy_version 1458274 (0.0008) [2023-12-27 15:45:01,853][20347] Updated weights for policy 0, policy_version 1458284 (0.0008) [2023-12-27 15:45:01,901][20347] Updated weights for policy 0, policy_version 1458294 (0.0008) [2023-12-27 15:45:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 746381312. Throughput: 0: 10437.3, 1: 10221.5. Samples: 746393152. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:02,363][19170] Avg episode reward: [(0, '568.990'), (1, '716.500')] [2023-12-27 15:45:02,443][20381] Updated weights for policy 1, policy_version 1456857 (0.0005) [2023-12-27 15:45:02,510][20381] Updated weights for policy 1, policy_version 1456867 (0.0006) [2023-12-27 15:45:02,568][20381] Updated weights for policy 1, policy_version 1456877 (0.0005) [2023-12-27 15:45:02,610][20347] Updated weights for policy 0, policy_version 1458304 (0.0008) [2023-12-27 15:45:02,671][20347] Updated weights for policy 0, policy_version 1458314 (0.0007) [2023-12-27 15:45:02,725][20347] Updated weights for policy 0, policy_version 1458324 (0.0007) [2023-12-27 15:45:03,129][20381] Updated weights for policy 1, policy_version 1456887 (0.0010) [2023-12-27 15:45:03,180][20381] Updated weights for policy 1, policy_version 1456897 (0.0010) [2023-12-27 15:45:03,238][20381] Updated weights for policy 1, policy_version 1456907 (0.0010) [2023-12-27 15:45:03,447][20347] Updated weights for policy 0, policy_version 1458334 (0.0008) [2023-12-27 15:45:03,500][20347] Updated weights for policy 0, policy_version 1458344 (0.0007) [2023-12-27 15:45:03,553][20347] Updated weights for policy 0, policy_version 1458354 (0.0006) [2023-12-27 15:45:03,877][20381] Updated weights for policy 1, policy_version 1456917 (0.0008) [2023-12-27 15:45:03,930][20381] Updated weights for policy 1, policy_version 1456927 (0.0006) [2023-12-27 15:45:03,992][20381] Updated weights for policy 1, policy_version 1456937 (0.0007) [2023-12-27 15:45:04,339][20347] Updated weights for policy 0, policy_version 1458364 (0.0007) [2023-12-27 15:45:04,400][20347] Updated weights for policy 0, policy_version 1458374 (0.0007) [2023-12-27 15:45:04,463][20347] Updated weights for policy 0, policy_version 1458384 (0.0007) [2023-12-27 15:45:04,667][20381] Updated weights for policy 1, policy_version 1456947 (0.0007) [2023-12-27 15:45:04,725][20381] Updated weights for policy 1, policy_version 1456957 (0.0005) [2023-12-27 15:45:04,779][20381] Updated weights for policy 1, policy_version 1456967 (0.0005) [2023-12-27 15:45:05,158][20347] Updated weights for policy 0, policy_version 1458394 (0.0008) [2023-12-27 15:45:05,211][20347] Updated weights for policy 0, policy_version 1458404 (0.0007) [2023-12-27 15:45:05,261][20347] Updated weights for policy 0, policy_version 1458414 (0.0007) [2023-12-27 15:45:05,322][20347] Updated weights for policy 0, policy_version 1458424 (0.0007) [2023-12-27 15:45:05,327][20381] Updated weights for policy 1, policy_version 1456977 (0.0005) [2023-12-27 15:45:05,383][20381] Updated weights for policy 1, policy_version 1456987 (0.0008) [2023-12-27 15:45:05,439][20381] Updated weights for policy 1, policy_version 1456997 (0.0007) [2023-12-27 15:45:05,491][20381] Updated weights for policy 1, policy_version 1457007 (0.0008) [2023-12-27 15:45:05,990][20347] Updated weights for policy 0, policy_version 1458434 (0.0007) [2023-12-27 15:45:06,046][20347] Updated weights for policy 0, policy_version 1458444 (0.0009) [2023-12-27 15:45:06,081][20381] Updated weights for policy 1, policy_version 1457017 (0.0005) [2023-12-27 15:45:06,104][20347] Updated weights for policy 0, policy_version 1458454 (0.0008) [2023-12-27 15:45:06,143][20381] Updated weights for policy 1, policy_version 1457027 (0.0006) [2023-12-27 15:45:06,205][20381] Updated weights for policy 1, policy_version 1457037 (0.0006) [2023-12-27 15:45:06,797][20347] Updated weights for policy 0, policy_version 1458464 (0.0008) [2023-12-27 15:45:06,849][20347] Updated weights for policy 0, policy_version 1458474 (0.0007) [2023-12-27 15:45:06,896][20381] Updated weights for policy 1, policy_version 1457047 (0.0008) [2023-12-27 15:45:06,907][20347] Updated weights for policy 0, policy_version 1458484 (0.0005) [2023-12-27 15:45:06,955][20381] Updated weights for policy 1, policy_version 1457057 (0.0007) [2023-12-27 15:45:07,015][20381] Updated weights for policy 1, policy_version 1457067 (0.0008) [2023-12-27 15:45:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 746487808. Throughput: 0: 10396.4, 1: 10309.1. Samples: 746456268. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:07,363][19170] Avg episode reward: [(0, '666.450'), (1, '775.070')] [2023-12-27 15:45:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001457072_373039104.pth... [2023-12-27 15:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001458488_373448704.pth... [2023-12-27 15:45:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001455856_372727808.pth [2023-12-27 15:45:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001457304_373145600.pth [2023-12-27 15:45:07,534][20347] Updated weights for policy 0, policy_version 1458494 (0.0009) [2023-12-27 15:45:07,534][20291] KL-divergence is very high: 121.7368 [2023-12-27 15:45:07,571][20291] KL-divergence is very high: 128.1481 [2023-12-27 15:45:07,579][20347] Updated weights for policy 0, policy_version 1458504 (0.0006) [2023-12-27 15:45:07,628][20347] Updated weights for policy 0, policy_version 1458514 (0.0006) [2023-12-27 15:45:07,713][20381] Updated weights for policy 1, policy_version 1457077 (0.0009) [2023-12-27 15:45:07,765][20381] Updated weights for policy 1, policy_version 1457087 (0.0005) [2023-12-27 15:45:07,821][20381] Updated weights for policy 1, policy_version 1457097 (0.0007) [2023-12-27 15:45:08,349][20347] Updated weights for policy 0, policy_version 1458524 (0.0007) [2023-12-27 15:45:08,405][20347] Updated weights for policy 0, policy_version 1458534 (0.0006) [2023-12-27 15:45:08,465][20347] Updated weights for policy 0, policy_version 1458544 (0.0008) [2023-12-27 15:45:08,495][20381] Updated weights for policy 1, policy_version 1457107 (0.0009) [2023-12-27 15:45:08,546][20381] Updated weights for policy 1, policy_version 1457117 (0.0008) [2023-12-27 15:45:08,598][20381] Updated weights for policy 1, policy_version 1457127 (0.0008) [2023-12-27 15:45:09,108][20347] Updated weights for policy 0, policy_version 1458554 (0.0009) [2023-12-27 15:45:09,168][20347] Updated weights for policy 0, policy_version 1458564 (0.0005) [2023-12-27 15:45:09,237][20347] Updated weights for policy 0, policy_version 1458574 (0.0007) [2023-12-27 15:45:09,295][20347] Updated weights for policy 0, policy_version 1458584 (0.0009) [2023-12-27 15:45:09,341][20381] Updated weights for policy 1, policy_version 1457137 (0.0007) [2023-12-27 15:45:09,410][20381] Updated weights for policy 1, policy_version 1457147 (0.0008) [2023-12-27 15:45:09,474][20381] Updated weights for policy 1, policy_version 1457157 (0.0007) [2023-12-27 15:45:09,546][20381] Updated weights for policy 1, policy_version 1457167 (0.0008) [2023-12-27 15:45:09,974][20347] Updated weights for policy 0, policy_version 1458594 (0.0008) [2023-12-27 15:45:10,031][20347] Updated weights for policy 0, policy_version 1458604 (0.0007) [2023-12-27 15:45:10,098][20347] Updated weights for policy 0, policy_version 1458614 (0.0009) [2023-12-27 15:45:10,104][20291] KL-divergence is very high: 107.4781 [2023-12-27 15:45:10,294][20381] Updated weights for policy 1, policy_version 1457177 (0.0009) [2023-12-27 15:45:10,352][20381] Updated weights for policy 1, policy_version 1457187 (0.0007) [2023-12-27 15:45:10,407][20381] Updated weights for policy 1, policy_version 1457197 (0.0008) [2023-12-27 15:45:10,762][20347] Updated weights for policy 0, policy_version 1458624 (0.0006) [2023-12-27 15:45:10,821][20347] Updated weights for policy 0, policy_version 1458634 (0.0005) [2023-12-27 15:45:10,870][20347] Updated weights for policy 0, policy_version 1458644 (0.0005) [2023-12-27 15:45:11,111][20381] Updated weights for policy 1, policy_version 1457207 (0.0008) [2023-12-27 15:45:11,179][20381] Updated weights for policy 1, policy_version 1457217 (0.0008) [2023-12-27 15:45:11,239][20381] Updated weights for policy 1, policy_version 1457227 (0.0008) [2023-12-27 15:45:11,476][20347] Updated weights for policy 0, policy_version 1458654 (0.0007) [2023-12-27 15:45:11,538][20347] Updated weights for policy 0, policy_version 1458664 (0.0008) [2023-12-27 15:45:11,594][20347] Updated weights for policy 0, policy_version 1458674 (0.0008) [2023-12-27 15:45:12,023][20381] Updated weights for policy 1, policy_version 1457237 (0.0009) [2023-12-27 15:45:12,080][20381] Updated weights for policy 1, policy_version 1457247 (0.0007) [2023-12-27 15:45:12,128][20381] Updated weights for policy 1, policy_version 1457257 (0.0008) [2023-12-27 15:45:12,309][20347] Updated weights for policy 0, policy_version 1458684 (0.0008) [2023-12-27 15:45:12,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 746586112. Throughput: 0: 10314.2, 1: 10232.0. Samples: 746578632. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:12,363][19170] Avg episode reward: [(0, '568.550'), (1, '736.930')] [2023-12-27 15:45:12,374][20347] Updated weights for policy 0, policy_version 1458694 (0.0008) [2023-12-27 15:45:12,436][20347] Updated weights for policy 0, policy_version 1458704 (0.0008) [2023-12-27 15:45:12,859][20381] Updated weights for policy 1, policy_version 1457267 (0.0008) [2023-12-27 15:45:12,910][20381] Updated weights for policy 1, policy_version 1457277 (0.0007) [2023-12-27 15:45:12,957][20381] Updated weights for policy 1, policy_version 1457287 (0.0008) [2023-12-27 15:45:13,126][20347] Updated weights for policy 0, policy_version 1458714 (0.0008) [2023-12-27 15:45:13,187][20347] Updated weights for policy 0, policy_version 1458724 (0.0006) [2023-12-27 15:45:13,236][20347] Updated weights for policy 0, policy_version 1458734 (0.0005) [2023-12-27 15:45:13,296][20347] Updated weights for policy 0, policy_version 1458744 (0.0005) [2023-12-27 15:45:13,726][20381] Updated weights for policy 1, policy_version 1457297 (0.0008) [2023-12-27 15:45:13,778][20381] Updated weights for policy 1, policy_version 1457307 (0.0008) [2023-12-27 15:45:13,834][20347] Updated weights for policy 0, policy_version 1458754 (0.0007) [2023-12-27 15:45:13,836][20381] Updated weights for policy 1, policy_version 1457317 (0.0005) [2023-12-27 15:45:13,884][20347] Updated weights for policy 0, policy_version 1458764 (0.0006) [2023-12-27 15:45:13,885][20381] Updated weights for policy 1, policy_version 1457327 (0.0006) [2023-12-27 15:45:13,938][20347] Updated weights for policy 0, policy_version 1458774 (0.0010) [2023-12-27 15:45:14,578][20381] Updated weights for policy 1, policy_version 1457337 (0.0010) [2023-12-27 15:45:14,598][20347] Updated weights for policy 0, policy_version 1458784 (0.0007) [2023-12-27 15:45:14,636][20381] Updated weights for policy 1, policy_version 1457347 (0.0010) [2023-12-27 15:45:14,649][20347] Updated weights for policy 0, policy_version 1458794 (0.0005) [2023-12-27 15:45:14,689][20381] Updated weights for policy 1, policy_version 1457357 (0.0011) [2023-12-27 15:45:14,714][20347] Updated weights for policy 0, policy_version 1458804 (0.0008) [2023-12-27 15:45:15,393][20381] Updated weights for policy 1, policy_version 1457367 (0.0011) [2023-12-27 15:45:15,439][20381] Updated weights for policy 1, policy_version 1457377 (0.0010) [2023-12-27 15:45:15,441][20347] Updated weights for policy 0, policy_version 1458814 (0.0007) [2023-12-27 15:45:15,489][20347] Updated weights for policy 0, policy_version 1458824 (0.0006) [2023-12-27 15:45:15,498][20381] Updated weights for policy 1, policy_version 1457387 (0.0010) [2023-12-27 15:45:15,537][20347] Updated weights for policy 0, policy_version 1458834 (0.0006) [2023-12-27 15:45:16,178][20381] Updated weights for policy 1, policy_version 1457397 (0.0009) [2023-12-27 15:45:16,215][20347] Updated weights for policy 0, policy_version 1458844 (0.0006) [2023-12-27 15:45:16,240][20381] Updated weights for policy 1, policy_version 1457407 (0.0007) [2023-12-27 15:45:16,268][20347] Updated weights for policy 0, policy_version 1458854 (0.0005) [2023-12-27 15:45:16,300][20381] Updated weights for policy 1, policy_version 1457417 (0.0007) [2023-12-27 15:45:16,325][20347] Updated weights for policy 0, policy_version 1458864 (0.0006) [2023-12-27 15:45:16,975][20381] Updated weights for policy 1, policy_version 1457427 (0.0006) [2023-12-27 15:45:17,028][20347] Updated weights for policy 0, policy_version 1458874 (0.0006) [2023-12-27 15:45:17,035][20381] Updated weights for policy 1, policy_version 1457437 (0.0007) [2023-12-27 15:45:17,079][20347] Updated weights for policy 0, policy_version 1458884 (0.0006) [2023-12-27 15:45:17,092][20381] Updated weights for policy 1, policy_version 1457447 (0.0007) [2023-12-27 15:45:17,128][20347] Updated weights for policy 0, policy_version 1458894 (0.0007) [2023-12-27 15:45:17,183][20347] Updated weights for policy 0, policy_version 1458904 (0.0007) [2023-12-27 15:45:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 746692608. Throughput: 0: 10262.7, 1: 10184.5. Samples: 746698976. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:17,363][19170] Avg episode reward: [(0, '667.120'), (1, '698.480')] [2023-12-27 15:45:17,730][20381] Updated weights for policy 1, policy_version 1457457 (0.0008) [2023-12-27 15:45:17,796][20381] Updated weights for policy 1, policy_version 1457467 (0.0006) [2023-12-27 15:45:17,848][20381] Updated weights for policy 1, policy_version 1457477 (0.0007) [2023-12-27 15:45:17,901][20347] Updated weights for policy 0, policy_version 1458914 (0.0006) [2023-12-27 15:45:17,902][20381] Updated weights for policy 1, policy_version 1457487 (0.0007) [2023-12-27 15:45:17,948][20347] Updated weights for policy 0, policy_version 1458924 (0.0007) [2023-12-27 15:45:18,001][20347] Updated weights for policy 0, policy_version 1458934 (0.0008) [2023-12-27 15:45:18,594][20381] Updated weights for policy 1, policy_version 1457497 (0.0008) [2023-12-27 15:45:18,649][20347] Updated weights for policy 0, policy_version 1458944 (0.0008) [2023-12-27 15:45:18,652][20381] Updated weights for policy 1, policy_version 1457507 (0.0006) [2023-12-27 15:45:18,704][20347] Updated weights for policy 0, policy_version 1458954 (0.0006) [2023-12-27 15:45:18,714][20381] Updated weights for policy 1, policy_version 1457517 (0.0009) [2023-12-27 15:45:18,765][20347] Updated weights for policy 0, policy_version 1458964 (0.0009) [2023-12-27 15:45:19,407][20381] Updated weights for policy 1, policy_version 1457527 (0.0006) [2023-12-27 15:45:19,468][20381] Updated weights for policy 1, policy_version 1457537 (0.0005) [2023-12-27 15:45:19,477][20347] Updated weights for policy 0, policy_version 1458974 (0.0008) [2023-12-27 15:45:19,536][20347] Updated weights for policy 0, policy_version 1458984 (0.0009) [2023-12-27 15:45:19,539][20381] Updated weights for policy 1, policy_version 1457547 (0.0008) [2023-12-27 15:45:19,593][20347] Updated weights for policy 0, policy_version 1458995 (0.0009) [2023-12-27 15:45:20,219][20381] Updated weights for policy 1, policy_version 1457557 (0.0007) [2023-12-27 15:45:20,283][20381] Updated weights for policy 1, policy_version 1457567 (0.0005) [2023-12-27 15:45:20,346][20381] Updated weights for policy 1, policy_version 1457577 (0.0007) [2023-12-27 15:45:20,353][20347] Updated weights for policy 0, policy_version 1459005 (0.0010) [2023-12-27 15:45:20,410][20347] Updated weights for policy 0, policy_version 1459015 (0.0007) [2023-12-27 15:45:20,465][20347] Updated weights for policy 0, policy_version 1459025 (0.0005) [2023-12-27 15:45:20,997][20381] Updated weights for policy 1, policy_version 1457587 (0.0009) [2023-12-27 15:45:21,065][20381] Updated weights for policy 1, policy_version 1457597 (0.0008) [2023-12-27 15:45:21,130][20381] Updated weights for policy 1, policy_version 1457607 (0.0007) [2023-12-27 15:45:21,215][20347] Updated weights for policy 0, policy_version 1459035 (0.0007) [2023-12-27 15:45:21,280][20347] Updated weights for policy 0, policy_version 1459045 (0.0007) [2023-12-27 15:45:21,342][20347] Updated weights for policy 0, policy_version 1459055 (0.0008) [2023-12-27 15:45:21,774][20381] Updated weights for policy 1, policy_version 1457617 (0.0006) [2023-12-27 15:45:21,833][20381] Updated weights for policy 1, policy_version 1457627 (0.0008) [2023-12-27 15:45:21,892][20381] Updated weights for policy 1, policy_version 1457637 (0.0008) [2023-12-27 15:45:21,944][20381] Updated weights for policy 1, policy_version 1457647 (0.0008) [2023-12-27 15:45:22,102][20347] Updated weights for policy 0, policy_version 1459065 (0.0009) [2023-12-27 15:45:22,149][20347] Updated weights for policy 0, policy_version 1459075 (0.0007) [2023-12-27 15:45:22,195][20347] Updated weights for policy 0, policy_version 1459085 (0.0007) [2023-12-27 15:45:22,251][20347] Updated weights for policy 0, policy_version 1459095 (0.0008) [2023-12-27 15:45:22,363][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 746790912. Throughput: 0: 10201.7, 1: 10201.0. Samples: 746759016. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:22,363][19170] Avg episode reward: [(0, '627.640'), (1, '687.110')] [2023-12-27 15:45:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001457648_373186560.pth... [2023-12-27 15:45:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001459096_373604352.pth... [2023-12-27 15:45:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001456464_372883456.pth [2023-12-27 15:45:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001457912_373301248.pth [2023-12-27 15:45:22,680][20381] Updated weights for policy 1, policy_version 1457657 (0.0011) [2023-12-27 15:45:22,732][20381] Updated weights for policy 1, policy_version 1457667 (0.0009) [2023-12-27 15:45:22,789][20381] Updated weights for policy 1, policy_version 1457677 (0.0011) [2023-12-27 15:45:23,000][20347] Updated weights for policy 0, policy_version 1459105 (0.0007) [2023-12-27 15:45:23,056][20347] Updated weights for policy 0, policy_version 1459115 (0.0007) [2023-12-27 15:45:23,109][20347] Updated weights for policy 0, policy_version 1459125 (0.0008) [2023-12-27 15:45:23,446][20381] Updated weights for policy 1, policy_version 1457687 (0.0007) [2023-12-27 15:45:23,501][20381] Updated weights for policy 1, policy_version 1457697 (0.0005) [2023-12-27 15:45:23,548][20381] Updated weights for policy 1, policy_version 1457707 (0.0005) [2023-12-27 15:45:23,763][20347] Updated weights for policy 0, policy_version 1459135 (0.0006) [2023-12-27 15:45:23,810][20347] Updated weights for policy 0, policy_version 1459145 (0.0005) [2023-12-27 15:45:23,867][20347] Updated weights for policy 0, policy_version 1459155 (0.0010) [2023-12-27 15:45:24,132][20381] Updated weights for policy 1, policy_version 1457717 (0.0006) [2023-12-27 15:45:24,194][20381] Updated weights for policy 1, policy_version 1457727 (0.0006) [2023-12-27 15:45:24,259][20381] Updated weights for policy 1, policy_version 1457737 (0.0006) [2023-12-27 15:45:24,524][20347] Updated weights for policy 0, policy_version 1459165 (0.0008) [2023-12-27 15:45:24,577][20347] Updated weights for policy 0, policy_version 1459175 (0.0006) [2023-12-27 15:45:24,641][20347] Updated weights for policy 0, policy_version 1459185 (0.0010) [2023-12-27 15:45:24,781][20381] Updated weights for policy 1, policy_version 1457747 (0.0007) [2023-12-27 15:45:24,837][20381] Updated weights for policy 1, policy_version 1457757 (0.0008) [2023-12-27 15:45:24,897][20381] Updated weights for policy 1, policy_version 1457767 (0.0006) [2023-12-27 15:45:25,192][20347] Updated weights for policy 0, policy_version 1459195 (0.0009) [2023-12-27 15:45:25,258][20347] Updated weights for policy 0, policy_version 1459205 (0.0005) [2023-12-27 15:45:25,317][20347] Updated weights for policy 0, policy_version 1459215 (0.0005) [2023-12-27 15:45:25,596][20381] Updated weights for policy 1, policy_version 1457777 (0.0008) [2023-12-27 15:45:25,646][20381] Updated weights for policy 1, policy_version 1457787 (0.0007) [2023-12-27 15:45:25,706][20381] Updated weights for policy 1, policy_version 1457797 (0.0007) [2023-12-27 15:45:25,767][20381] Updated weights for policy 1, policy_version 1457807 (0.0007) [2023-12-27 15:45:25,921][20347] Updated weights for policy 0, policy_version 1459225 (0.0010) [2023-12-27 15:45:25,978][20347] Updated weights for policy 0, policy_version 1459235 (0.0010) [2023-12-27 15:45:26,034][20347] Updated weights for policy 0, policy_version 1459245 (0.0010) [2023-12-27 15:45:26,092][20347] Updated weights for policy 0, policy_version 1459255 (0.0010) [2023-12-27 15:45:26,413][20381] Updated weights for policy 1, policy_version 1457817 (0.0007) [2023-12-27 15:45:26,467][20381] Updated weights for policy 1, policy_version 1457827 (0.0008) [2023-12-27 15:45:26,522][20381] Updated weights for policy 1, policy_version 1457837 (0.0008) [2023-12-27 15:45:26,704][20347] Updated weights for policy 0, policy_version 1459265 (0.0010) [2023-12-27 15:45:26,759][20347] Updated weights for policy 0, policy_version 1459275 (0.0010) [2023-12-27 15:45:26,826][20347] Updated weights for policy 0, policy_version 1459285 (0.0010) [2023-12-27 15:45:27,124][20381] Updated weights for policy 1, policy_version 1457847 (0.0008) [2023-12-27 15:45:27,186][20381] Updated weights for policy 1, policy_version 1457857 (0.0005) [2023-12-27 15:45:27,241][20381] Updated weights for policy 1, policy_version 1457867 (0.0005) [2023-12-27 15:45:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 746897408. Throughput: 0: 10177.0, 1: 10264.2. Samples: 746884460. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:27,363][19170] Avg episode reward: [(0, '647.880'), (1, '664.200')] [2023-12-27 15:45:27,484][20347] Updated weights for policy 0, policy_version 1459295 (0.0010) [2023-12-27 15:45:27,537][20347] Updated weights for policy 0, policy_version 1459305 (0.0010) [2023-12-27 15:45:27,581][20347] Updated weights for policy 0, policy_version 1459315 (0.0010) [2023-12-27 15:45:27,882][20381] Updated weights for policy 1, policy_version 1457877 (0.0007) [2023-12-27 15:45:27,939][20381] Updated weights for policy 1, policy_version 1457887 (0.0008) [2023-12-27 15:45:28,000][20381] Updated weights for policy 1, policy_version 1457897 (0.0008) [2023-12-27 15:45:28,273][20347] Updated weights for policy 0, policy_version 1459325 (0.0010) [2023-12-27 15:45:28,333][20347] Updated weights for policy 0, policy_version 1459335 (0.0010) [2023-12-27 15:45:28,392][20347] Updated weights for policy 0, policy_version 1459345 (0.0010) [2023-12-27 15:45:28,672][20381] Updated weights for policy 1, policy_version 1457907 (0.0007) [2023-12-27 15:45:28,720][20381] Updated weights for policy 1, policy_version 1457917 (0.0008) [2023-12-27 15:45:28,780][20381] Updated weights for policy 1, policy_version 1457927 (0.0007) [2023-12-27 15:45:29,023][20347] Updated weights for policy 0, policy_version 1459355 (0.0007) [2023-12-27 15:45:29,074][20347] Updated weights for policy 0, policy_version 1459365 (0.0008) [2023-12-27 15:45:29,126][20347] Updated weights for policy 0, policy_version 1459375 (0.0007) [2023-12-27 15:45:29,504][20381] Updated weights for policy 1, policy_version 1457937 (0.0008) [2023-12-27 15:45:29,569][20381] Updated weights for policy 1, policy_version 1457947 (0.0005) [2023-12-27 15:45:29,632][20381] Updated weights for policy 1, policy_version 1457957 (0.0006) [2023-12-27 15:45:29,691][20381] Updated weights for policy 1, policy_version 1457967 (0.0006) [2023-12-27 15:45:29,788][20347] Updated weights for policy 0, policy_version 1459385 (0.0008) [2023-12-27 15:45:29,856][20347] Updated weights for policy 0, policy_version 1459395 (0.0010) [2023-12-27 15:45:29,913][20347] Updated weights for policy 0, policy_version 1459405 (0.0010) [2023-12-27 15:45:29,974][20347] Updated weights for policy 0, policy_version 1459415 (0.0009) [2023-12-27 15:45:30,385][20381] Updated weights for policy 1, policy_version 1457977 (0.0007) [2023-12-27 15:45:30,437][20381] Updated weights for policy 1, policy_version 1457987 (0.0007) [2023-12-27 15:45:30,489][20381] Updated weights for policy 1, policy_version 1457997 (0.0007) [2023-12-27 15:45:30,668][20347] Updated weights for policy 0, policy_version 1459425 (0.0010) [2023-12-27 15:45:30,722][20347] Updated weights for policy 0, policy_version 1459435 (0.0010) [2023-12-27 15:45:30,780][20347] Updated weights for policy 0, policy_version 1459445 (0.0010) [2023-12-27 15:45:31,232][20381] Updated weights for policy 1, policy_version 1458007 (0.0006) [2023-12-27 15:45:31,300][20381] Updated weights for policy 1, policy_version 1458017 (0.0006) [2023-12-27 15:45:31,363][20381] Updated weights for policy 1, policy_version 1458027 (0.0007) [2023-12-27 15:45:31,546][20347] Updated weights for policy 0, policy_version 1459455 (0.0008) [2023-12-27 15:45:31,612][20347] Updated weights for policy 0, policy_version 1459465 (0.0007) [2023-12-27 15:45:31,672][20347] Updated weights for policy 0, policy_version 1459475 (0.0006) [2023-12-27 15:45:32,052][20381] Updated weights for policy 1, policy_version 1458037 (0.0007) [2023-12-27 15:45:32,102][20381] Updated weights for policy 1, policy_version 1458047 (0.0005) [2023-12-27 15:45:32,154][20381] Updated weights for policy 1, policy_version 1458057 (0.0007) [2023-12-27 15:45:32,278][20347] Updated weights for policy 0, policy_version 1459485 (0.0006) [2023-12-27 15:45:32,286][20291] KL-divergence is very high: 125.5449 [2023-12-27 15:45:32,346][20347] Updated weights for policy 0, policy_version 1459495 (0.0008) [2023-12-27 15:45:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 746995712. Throughput: 0: 10157.8, 1: 10232.7. Samples: 747007844. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:32,363][19170] Avg episode reward: [(0, '579.030'), (1, '633.910')] [2023-12-27 15:45:32,411][20347] Updated weights for policy 0, policy_version 1459505 (0.0008) [2023-12-27 15:45:32,822][20381] Updated weights for policy 1, policy_version 1458067 (0.0007) [2023-12-27 15:45:32,877][20381] Updated weights for policy 1, policy_version 1458077 (0.0008) [2023-12-27 15:45:32,923][20381] Updated weights for policy 1, policy_version 1458087 (0.0008) [2023-12-27 15:45:33,052][20347] Updated weights for policy 0, policy_version 1459515 (0.0006) [2023-12-27 15:45:33,111][20347] Updated weights for policy 0, policy_version 1459525 (0.0005) [2023-12-27 15:45:33,171][20347] Updated weights for policy 0, policy_version 1459535 (0.0005) [2023-12-27 15:45:33,623][20381] Updated weights for policy 1, policy_version 1458097 (0.0008) [2023-12-27 15:45:33,681][20381] Updated weights for policy 1, policy_version 1458107 (0.0008) [2023-12-27 15:45:33,727][20381] Updated weights for policy 1, policy_version 1458117 (0.0008) [2023-12-27 15:45:33,783][20381] Updated weights for policy 1, policy_version 1458127 (0.0008) [2023-12-27 15:45:33,784][20347] Updated weights for policy 0, policy_version 1459545 (0.0006) [2023-12-27 15:45:33,839][20347] Updated weights for policy 0, policy_version 1459555 (0.0007) [2023-12-27 15:45:33,885][20347] Updated weights for policy 0, policy_version 1459565 (0.0007) [2023-12-27 15:45:33,934][20347] Updated weights for policy 0, policy_version 1459575 (0.0008) [2023-12-27 15:45:34,538][20381] Updated weights for policy 1, policy_version 1458137 (0.0010) [2023-12-27 15:45:34,604][20381] Updated weights for policy 1, policy_version 1458147 (0.0009) [2023-12-27 15:45:34,650][20347] Updated weights for policy 0, policy_version 1459585 (0.0007) [2023-12-27 15:45:34,669][20381] Updated weights for policy 1, policy_version 1458157 (0.0008) [2023-12-27 15:45:34,710][20347] Updated weights for policy 0, policy_version 1459595 (0.0006) [2023-12-27 15:45:34,764][20347] Updated weights for policy 0, policy_version 1459605 (0.0007) [2023-12-27 15:45:35,280][20381] Updated weights for policy 1, policy_version 1458167 (0.0007) [2023-12-27 15:45:35,335][20381] Updated weights for policy 1, policy_version 1458177 (0.0009) [2023-12-27 15:45:35,400][20381] Updated weights for policy 1, policy_version 1458187 (0.0008) [2023-12-27 15:45:35,462][20347] Updated weights for policy 0, policy_version 1459615 (0.0008) [2023-12-27 15:45:35,551][20347] Updated weights for policy 0, policy_version 1459625 (0.0007) [2023-12-27 15:45:35,602][20347] Updated weights for policy 0, policy_version 1459635 (0.0008) [2023-12-27 15:45:36,085][20381] Updated weights for policy 1, policy_version 1458197 (0.0006) [2023-12-27 15:45:36,150][20381] Updated weights for policy 1, policy_version 1458207 (0.0007) [2023-12-27 15:45:36,207][20381] Updated weights for policy 1, policy_version 1458217 (0.0007) [2023-12-27 15:45:36,290][20347] Updated weights for policy 0, policy_version 1459645 (0.0007) [2023-12-27 15:45:36,346][20347] Updated weights for policy 0, policy_version 1459655 (0.0008) [2023-12-27 15:45:36,403][20347] Updated weights for policy 0, policy_version 1459665 (0.0008) [2023-12-27 15:45:36,898][20381] Updated weights for policy 1, policy_version 1458227 (0.0007) [2023-12-27 15:45:36,955][20381] Updated weights for policy 1, policy_version 1458237 (0.0007) [2023-12-27 15:45:37,006][20381] Updated weights for policy 1, policy_version 1458247 (0.0006) [2023-12-27 15:45:37,110][20347] Updated weights for policy 0, policy_version 1459675 (0.0007) [2023-12-27 15:45:37,162][20347] Updated weights for policy 0, policy_version 1459685 (0.0007) [2023-12-27 15:45:37,222][20347] Updated weights for policy 0, policy_version 1459695 (0.0007) [2023-12-27 15:45:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 747102208. Throughput: 0: 10161.2, 1: 10198.1. Samples: 747069488. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:37,363][19170] Avg episode reward: [(0, '627.480'), (1, '747.160')] [2023-12-27 15:45:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001459704_373760000.pth... [2023-12-27 15:45:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001458256_373342208.pth... [2023-12-27 15:45:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001458488_373448704.pth [2023-12-27 15:45:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001457072_373039104.pth [2023-12-27 15:45:37,642][20381] Updated weights for policy 1, policy_version 1458257 (0.0007) [2023-12-27 15:45:37,697][20381] Updated weights for policy 1, policy_version 1458267 (0.0008) [2023-12-27 15:45:37,755][20381] Updated weights for policy 1, policy_version 1458277 (0.0008) [2023-12-27 15:45:37,803][20381] Updated weights for policy 1, policy_version 1458287 (0.0006) [2023-12-27 15:45:37,869][20347] Updated weights for policy 0, policy_version 1459705 (0.0007) [2023-12-27 15:45:37,918][20347] Updated weights for policy 0, policy_version 1459715 (0.0007) [2023-12-27 15:45:37,973][20347] Updated weights for policy 0, policy_version 1459725 (0.0007) [2023-12-27 15:45:38,033][20347] Updated weights for policy 0, policy_version 1459735 (0.0007) [2023-12-27 15:45:38,464][20381] Updated weights for policy 1, policy_version 1458297 (0.0007) [2023-12-27 15:45:38,529][20381] Updated weights for policy 1, policy_version 1458307 (0.0008) [2023-12-27 15:45:38,583][20381] Updated weights for policy 1, policy_version 1458317 (0.0008) [2023-12-27 15:45:38,807][20347] Updated weights for policy 0, policy_version 1459745 (0.0006) [2023-12-27 15:45:38,866][20347] Updated weights for policy 0, policy_version 1459755 (0.0008) [2023-12-27 15:45:38,921][20347] Updated weights for policy 0, policy_version 1459765 (0.0008) [2023-12-27 15:45:39,298][20381] Updated weights for policy 1, policy_version 1458327 (0.0007) [2023-12-27 15:45:39,369][20381] Updated weights for policy 1, policy_version 1458337 (0.0007) [2023-12-27 15:45:39,435][20381] Updated weights for policy 1, policy_version 1458347 (0.0006) [2023-12-27 15:45:39,635][20347] Updated weights for policy 0, policy_version 1459775 (0.0008) [2023-12-27 15:45:39,696][20347] Updated weights for policy 0, policy_version 1459785 (0.0008) [2023-12-27 15:45:39,720][20291] KL-divergence is very high: 102.3473 [2023-12-27 15:45:39,726][20291] KL-divergence is very high: 103.2754 [2023-12-27 15:45:39,742][20291] KL-divergence is very high: 107.9468 [2023-12-27 15:45:39,746][20347] Updated weights for policy 0, policy_version 1459795 (0.0007) [2023-12-27 15:45:40,173][20381] Updated weights for policy 1, policy_version 1458357 (0.0008) [2023-12-27 15:45:40,244][20381] Updated weights for policy 1, policy_version 1458367 (0.0007) [2023-12-27 15:45:40,309][20381] Updated weights for policy 1, policy_version 1458377 (0.0006) [2023-12-27 15:45:40,483][20347] Updated weights for policy 0, policy_version 1459805 (0.0008) [2023-12-27 15:45:40,552][20347] Updated weights for policy 0, policy_version 1459815 (0.0008) [2023-12-27 15:45:40,615][20347] Updated weights for policy 0, policy_version 1459825 (0.0008) [2023-12-27 15:45:40,938][20381] Updated weights for policy 1, policy_version 1458387 (0.0007) [2023-12-27 15:45:40,994][20381] Updated weights for policy 1, policy_version 1458397 (0.0007) [2023-12-27 15:45:41,060][20381] Updated weights for policy 1, policy_version 1458407 (0.0008) [2023-12-27 15:45:41,443][20347] Updated weights for policy 0, policy_version 1459835 (0.0009) [2023-12-27 15:45:41,500][20347] Updated weights for policy 0, policy_version 1459845 (0.0008) [2023-12-27 15:45:41,560][20347] Updated weights for policy 0, policy_version 1459855 (0.0007) [2023-12-27 15:45:41,861][20381] Updated weights for policy 1, policy_version 1458417 (0.0008) [2023-12-27 15:45:41,922][20381] Updated weights for policy 1, policy_version 1458427 (0.0007) [2023-12-27 15:45:41,983][20381] Updated weights for policy 1, policy_version 1458437 (0.0006) [2023-12-27 15:45:42,034][20381] Updated weights for policy 1, policy_version 1458447 (0.0008) [2023-12-27 15:45:42,320][20347] Updated weights for policy 0, policy_version 1459865 (0.0008) [2023-12-27 15:45:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 747192320. Throughput: 0: 10121.7, 1: 10170.5. Samples: 747187528. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:42,363][19170] Avg episode reward: [(0, '638.400'), (1, '709.490')] [2023-12-27 15:45:42,388][20347] Updated weights for policy 0, policy_version 1459875 (0.0008) [2023-12-27 15:45:42,450][20347] Updated weights for policy 0, policy_version 1459885 (0.0008) [2023-12-27 15:45:42,509][20347] Updated weights for policy 0, policy_version 1459895 (0.0008) [2023-12-27 15:45:42,743][20381] Updated weights for policy 1, policy_version 1458457 (0.0008) [2023-12-27 15:45:42,798][20381] Updated weights for policy 1, policy_version 1458467 (0.0008) [2023-12-27 15:45:42,858][20381] Updated weights for policy 1, policy_version 1458477 (0.0008) [2023-12-27 15:45:43,205][20347] Updated weights for policy 0, policy_version 1459905 (0.0008) [2023-12-27 15:45:43,256][20347] Updated weights for policy 0, policy_version 1459915 (0.0008) [2023-12-27 15:45:43,310][20347] Updated weights for policy 0, policy_version 1459925 (0.0008) [2023-12-27 15:45:43,561][20381] Updated weights for policy 1, policy_version 1458487 (0.0008) [2023-12-27 15:45:43,623][20381] Updated weights for policy 1, policy_version 1458497 (0.0008) [2023-12-27 15:45:43,680][20381] Updated weights for policy 1, policy_version 1458507 (0.0008) [2023-12-27 15:45:44,017][20347] Updated weights for policy 0, policy_version 1459935 (0.0009) [2023-12-27 15:45:44,073][20347] Updated weights for policy 0, policy_version 1459945 (0.0010) [2023-12-27 15:45:44,134][20347] Updated weights for policy 0, policy_version 1459955 (0.0010) [2023-12-27 15:45:44,322][20381] Updated weights for policy 1, policy_version 1458517 (0.0008) [2023-12-27 15:45:44,367][20381] Updated weights for policy 1, policy_version 1458527 (0.0006) [2023-12-27 15:45:44,416][20381] Updated weights for policy 1, policy_version 1458537 (0.0005) [2023-12-27 15:45:44,757][20347] Updated weights for policy 0, policy_version 1459965 (0.0011) [2023-12-27 15:45:44,824][20347] Updated weights for policy 0, policy_version 1459975 (0.0009) [2023-12-27 15:45:44,877][20347] Updated weights for policy 0, policy_version 1459985 (0.0008) [2023-12-27 15:45:45,114][20381] Updated weights for policy 1, policy_version 1458547 (0.0006) [2023-12-27 15:45:45,168][20381] Updated weights for policy 1, policy_version 1458557 (0.0008) [2023-12-27 15:45:45,226][20381] Updated weights for policy 1, policy_version 1458567 (0.0008) [2023-12-27 15:45:45,551][20347] Updated weights for policy 0, policy_version 1459995 (0.0007) [2023-12-27 15:45:45,599][20347] Updated weights for policy 0, policy_version 1460005 (0.0005) [2023-12-27 15:45:45,647][20347] Updated weights for policy 0, policy_version 1460015 (0.0005) [2023-12-27 15:45:45,851][20381] Updated weights for policy 1, policy_version 1458577 (0.0008) [2023-12-27 15:45:45,909][20381] Updated weights for policy 1, policy_version 1458587 (0.0005) [2023-12-27 15:45:45,968][20381] Updated weights for policy 1, policy_version 1458597 (0.0006) [2023-12-27 15:45:46,034][20381] Updated weights for policy 1, policy_version 1458607 (0.0007) [2023-12-27 15:45:46,334][20347] Updated weights for policy 0, policy_version 1460025 (0.0007) [2023-12-27 15:45:46,396][20347] Updated weights for policy 0, policy_version 1460035 (0.0005) [2023-12-27 15:45:46,459][20347] Updated weights for policy 0, policy_version 1460045 (0.0007) [2023-12-27 15:45:46,520][20347] Updated weights for policy 0, policy_version 1460055 (0.0008) [2023-12-27 15:45:46,635][20381] Updated weights for policy 1, policy_version 1458617 (0.0008) [2023-12-27 15:45:46,694][20381] Updated weights for policy 1, policy_version 1458627 (0.0006) [2023-12-27 15:45:46,745][20381] Updated weights for policy 1, policy_version 1458637 (0.0007) [2023-12-27 15:45:47,100][20347] Updated weights for policy 0, policy_version 1460065 (0.0008) [2023-12-27 15:45:47,152][20347] Updated weights for policy 0, policy_version 1460075 (0.0008) [2023-12-27 15:45:47,201][20347] Updated weights for policy 0, policy_version 1460085 (0.0007) [2023-12-27 15:45:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 747298816. Throughput: 0: 10151.6, 1: 10225.5. Samples: 747310120. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:47,363][19170] Avg episode reward: [(0, '696.500'), (1, '762.510')] [2023-12-27 15:45:47,372][20381] Updated weights for policy 1, policy_version 1458647 (0.0008) [2023-12-27 15:45:47,429][20381] Updated weights for policy 1, policy_version 1458657 (0.0009) [2023-12-27 15:45:47,485][20381] Updated weights for policy 1, policy_version 1458667 (0.0007) [2023-12-27 15:45:47,811][20347] Updated weights for policy 0, policy_version 1460095 (0.0006) [2023-12-27 15:45:47,866][20347] Updated weights for policy 0, policy_version 1460105 (0.0005) [2023-12-27 15:45:47,930][20347] Updated weights for policy 0, policy_version 1460115 (0.0005) [2023-12-27 15:45:48,131][20381] Updated weights for policy 1, policy_version 1458677 (0.0007) [2023-12-27 15:45:48,179][20381] Updated weights for policy 1, policy_version 1458687 (0.0008) [2023-12-27 15:45:48,227][20381] Updated weights for policy 1, policy_version 1458697 (0.0008) [2023-12-27 15:45:48,490][20347] Updated weights for policy 0, policy_version 1460125 (0.0007) [2023-12-27 15:45:48,543][20347] Updated weights for policy 0, policy_version 1460135 (0.0007) [2023-12-27 15:45:48,596][20347] Updated weights for policy 0, policy_version 1460145 (0.0009) [2023-12-27 15:45:48,869][20381] Updated weights for policy 1, policy_version 1458707 (0.0007) [2023-12-27 15:45:48,919][20381] Updated weights for policy 1, policy_version 1458717 (0.0008) [2023-12-27 15:45:48,971][20381] Updated weights for policy 1, policy_version 1458727 (0.0007) [2023-12-27 15:45:49,335][20347] Updated weights for policy 0, policy_version 1460155 (0.0009) [2023-12-27 15:45:49,402][20347] Updated weights for policy 0, policy_version 1460165 (0.0008) [2023-12-27 15:45:49,468][20347] Updated weights for policy 0, policy_version 1460175 (0.0007) [2023-12-27 15:45:49,661][20381] Updated weights for policy 1, policy_version 1458737 (0.0008) [2023-12-27 15:45:49,720][20381] Updated weights for policy 1, policy_version 1458747 (0.0008) [2023-12-27 15:45:49,776][20381] Updated weights for policy 1, policy_version 1458757 (0.0005) [2023-12-27 15:45:49,835][20381] Updated weights for policy 1, policy_version 1458767 (0.0007) [2023-12-27 15:45:50,184][20347] Updated weights for policy 0, policy_version 1460185 (0.0008) [2023-12-27 15:45:50,237][20347] Updated weights for policy 0, policy_version 1460195 (0.0008) [2023-12-27 15:45:50,295][20347] Updated weights for policy 0, policy_version 1460205 (0.0007) [2023-12-27 15:45:50,351][20347] Updated weights for policy 0, policy_version 1460215 (0.0007) [2023-12-27 15:45:50,480][20381] Updated weights for policy 1, policy_version 1458777 (0.0007) [2023-12-27 15:45:50,536][20381] Updated weights for policy 1, policy_version 1458787 (0.0008) [2023-12-27 15:45:50,605][20381] Updated weights for policy 1, policy_version 1458797 (0.0008) [2023-12-27 15:45:51,040][20347] Updated weights for policy 0, policy_version 1460225 (0.0007) [2023-12-27 15:45:51,111][20347] Updated weights for policy 0, policy_version 1460235 (0.0008) [2023-12-27 15:45:51,172][20347] Updated weights for policy 0, policy_version 1460245 (0.0008) [2023-12-27 15:45:51,322][20381] Updated weights for policy 1, policy_version 1458807 (0.0007) [2023-12-27 15:45:51,384][20381] Updated weights for policy 1, policy_version 1458817 (0.0008) [2023-12-27 15:45:51,448][20381] Updated weights for policy 1, policy_version 1458827 (0.0008) [2023-12-27 15:45:51,949][20347] Updated weights for policy 0, policy_version 1460255 (0.0007) [2023-12-27 15:45:52,008][20347] Updated weights for policy 0, policy_version 1460265 (0.0007) [2023-12-27 15:45:52,061][20347] Updated weights for policy 0, policy_version 1460275 (0.0008) [2023-12-27 15:45:52,158][20381] Updated weights for policy 1, policy_version 1458837 (0.0008) [2023-12-27 15:45:52,208][20381] Updated weights for policy 1, policy_version 1458847 (0.0008) [2023-12-27 15:45:52,273][20381] Updated weights for policy 1, policy_version 1458857 (0.0008) [2023-12-27 15:45:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 747405312. Throughput: 0: 10190.0, 1: 10179.8. Samples: 747372912. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:52,363][19170] Avg episode reward: [(0, '677.020'), (1, '753.860')] [2023-12-27 15:45:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001460280_373907456.pth... [2023-12-27 15:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001458864_373497856.pth... [2023-12-27 15:45:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001459096_373604352.pth [2023-12-27 15:45:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001457648_373186560.pth [2023-12-27 15:45:52,797][20347] Updated weights for policy 0, policy_version 1460285 (0.0009) [2023-12-27 15:45:52,853][20347] Updated weights for policy 0, policy_version 1460295 (0.0008) [2023-12-27 15:45:52,908][20347] Updated weights for policy 0, policy_version 1460305 (0.0007) [2023-12-27 15:45:52,972][20381] Updated weights for policy 1, policy_version 1458867 (0.0008) [2023-12-27 15:45:53,021][20381] Updated weights for policy 1, policy_version 1458877 (0.0007) [2023-12-27 15:45:53,075][20381] Updated weights for policy 1, policy_version 1458887 (0.0006) [2023-12-27 15:45:53,594][20347] Updated weights for policy 0, policy_version 1460315 (0.0006) [2023-12-27 15:45:53,652][20347] Updated weights for policy 0, policy_version 1460325 (0.0007) [2023-12-27 15:45:53,708][20347] Updated weights for policy 0, policy_version 1460335 (0.0007) [2023-12-27 15:45:53,758][20381] Updated weights for policy 1, policy_version 1458897 (0.0006) [2023-12-27 15:45:53,806][20381] Updated weights for policy 1, policy_version 1458907 (0.0011) [2023-12-27 15:45:53,864][20381] Updated weights for policy 1, policy_version 1458917 (0.0011) [2023-12-27 15:45:53,922][20381] Updated weights for policy 1, policy_version 1458927 (0.0011) [2023-12-27 15:45:54,389][20347] Updated weights for policy 0, policy_version 1460345 (0.0007) [2023-12-27 15:45:54,451][20347] Updated weights for policy 0, policy_version 1460355 (0.0006) [2023-12-27 15:45:54,514][20347] Updated weights for policy 0, policy_version 1460365 (0.0008) [2023-12-27 15:45:54,544][20381] Updated weights for policy 1, policy_version 1458937 (0.0009) [2023-12-27 15:45:54,572][20347] Updated weights for policy 0, policy_version 1460375 (0.0006) [2023-12-27 15:45:54,604][20381] Updated weights for policy 1, policy_version 1458947 (0.0006) [2023-12-27 15:45:54,670][20381] Updated weights for policy 1, policy_version 1458957 (0.0005) [2023-12-27 15:45:55,213][20381] Updated weights for policy 1, policy_version 1458967 (0.0008) [2023-12-27 15:45:55,230][20347] Updated weights for policy 0, policy_version 1460385 (0.0005) [2023-12-27 15:45:55,272][20381] Updated weights for policy 1, policy_version 1458977 (0.0009) [2023-12-27 15:45:55,275][20347] Updated weights for policy 0, policy_version 1460395 (0.0006) [2023-12-27 15:45:55,322][20347] Updated weights for policy 0, policy_version 1460405 (0.0006) [2023-12-27 15:45:55,328][20381] Updated weights for policy 1, policy_version 1458987 (0.0011) [2023-12-27 15:45:55,956][20347] Updated weights for policy 0, policy_version 1460415 (0.0007) [2023-12-27 15:45:56,004][20381] Updated weights for policy 1, policy_version 1458997 (0.0009) [2023-12-27 15:45:56,011][20347] Updated weights for policy 0, policy_version 1460425 (0.0007) [2023-12-27 15:45:56,058][20381] Updated weights for policy 1, policy_version 1459007 (0.0009) [2023-12-27 15:45:56,060][20347] Updated weights for policy 0, policy_version 1460435 (0.0005) [2023-12-27 15:45:56,110][20381] Updated weights for policy 1, policy_version 1459017 (0.0011) [2023-12-27 15:45:56,763][20347] Updated weights for policy 0, policy_version 1460445 (0.0006) [2023-12-27 15:45:56,775][20381] Updated weights for policy 1, policy_version 1459027 (0.0009) [2023-12-27 15:45:56,805][20347] Updated weights for policy 0, policy_version 1460455 (0.0007) [2023-12-27 15:45:56,832][20381] Updated weights for policy 1, policy_version 1459037 (0.0008) [2023-12-27 15:45:56,849][20347] Updated weights for policy 0, policy_version 1460465 (0.0007) [2023-12-27 15:45:56,883][20381] Updated weights for policy 1, policy_version 1459047 (0.0010) [2023-12-27 15:45:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 747503616. Throughput: 0: 10133.7, 1: 10222.2. Samples: 747494644. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:45:57,363][19170] Avg episode reward: [(0, '697.100'), (1, '726.590')] [2023-12-27 15:45:57,479][20381] Updated weights for policy 1, policy_version 1459057 (0.0010) [2023-12-27 15:45:57,530][20381] Updated weights for policy 1, policy_version 1459067 (0.0007) [2023-12-27 15:45:57,562][20347] Updated weights for policy 0, policy_version 1460475 (0.0006) [2023-12-27 15:45:57,589][20381] Updated weights for policy 1, policy_version 1459077 (0.0006) [2023-12-27 15:45:57,620][20347] Updated weights for policy 0, policy_version 1460485 (0.0007) [2023-12-27 15:45:57,644][20381] Updated weights for policy 1, policy_version 1459087 (0.0006) [2023-12-27 15:45:57,673][20347] Updated weights for policy 0, policy_version 1460495 (0.0006) [2023-12-27 15:45:58,221][20381] Updated weights for policy 1, policy_version 1459097 (0.0008) [2023-12-27 15:45:58,280][20381] Updated weights for policy 1, policy_version 1459107 (0.0008) [2023-12-27 15:45:58,350][20381] Updated weights for policy 1, policy_version 1459117 (0.0008) [2023-12-27 15:45:58,421][20347] Updated weights for policy 0, policy_version 1460505 (0.0006) [2023-12-27 15:45:58,485][20347] Updated weights for policy 0, policy_version 1460515 (0.0007) [2023-12-27 15:45:58,548][20347] Updated weights for policy 0, policy_version 1460525 (0.0008) [2023-12-27 15:45:58,609][20347] Updated weights for policy 0, policy_version 1460535 (0.0011) [2023-12-27 15:45:59,104][20381] Updated weights for policy 1, policy_version 1459127 (0.0007) [2023-12-27 15:45:59,169][20381] Updated weights for policy 1, policy_version 1459137 (0.0006) [2023-12-27 15:45:59,227][20381] Updated weights for policy 1, policy_version 1459147 (0.0008) [2023-12-27 15:45:59,248][20292] KL-divergence is very high: 117.6611 [2023-12-27 15:45:59,255][20292] KL-divergence is very high: 128.1849 [2023-12-27 15:45:59,428][20347] Updated weights for policy 0, policy_version 1460545 (0.0008) [2023-12-27 15:45:59,485][20347] Updated weights for policy 0, policy_version 1460555 (0.0006) [2023-12-27 15:45:59,542][20347] Updated weights for policy 0, policy_version 1460565 (0.0007) [2023-12-27 15:45:59,974][20381] Updated weights for policy 1, policy_version 1459157 (0.0008) [2023-12-27 15:46:00,030][20381] Updated weights for policy 1, policy_version 1459167 (0.0008) [2023-12-27 15:46:00,076][20381] Updated weights for policy 1, policy_version 1459177 (0.0008) [2023-12-27 15:46:00,196][20347] Updated weights for policy 0, policy_version 1460575 (0.0007) [2023-12-27 15:46:00,249][20347] Updated weights for policy 0, policy_version 1460585 (0.0006) [2023-12-27 15:46:00,298][20347] Updated weights for policy 0, policy_version 1460595 (0.0005) [2023-12-27 15:46:00,804][20381] Updated weights for policy 1, policy_version 1459187 (0.0008) [2023-12-27 15:46:00,855][20381] Updated weights for policy 1, policy_version 1459197 (0.0008) [2023-12-27 15:46:00,901][20381] Updated weights for policy 1, policy_version 1459207 (0.0008) [2023-12-27 15:46:00,902][20347] Updated weights for policy 0, policy_version 1460605 (0.0005) [2023-12-27 15:46:00,947][20347] Updated weights for policy 0, policy_version 1460615 (0.0005) [2023-12-27 15:46:01,004][20347] Updated weights for policy 0, policy_version 1460625 (0.0005) [2023-12-27 15:46:01,639][20347] Updated weights for policy 0, policy_version 1460635 (0.0007) [2023-12-27 15:46:01,670][20381] Updated weights for policy 1, policy_version 1459217 (0.0008) [2023-12-27 15:46:01,700][20347] Updated weights for policy 0, policy_version 1460645 (0.0006) [2023-12-27 15:46:01,731][20381] Updated weights for policy 1, policy_version 1459227 (0.0010) [2023-12-27 15:46:01,768][20347] Updated weights for policy 0, policy_version 1460655 (0.0008) [2023-12-27 15:46:01,791][20381] Updated weights for policy 1, policy_version 1459237 (0.0011) [2023-12-27 15:46:01,846][20381] Updated weights for policy 1, policy_version 1459247 (0.0010) [2023-12-27 15:46:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 747601920. Throughput: 0: 10105.1, 1: 10255.4. Samples: 747615200. Policy #0 lag: (min: 13.0, avg: 18.1, max: 45.0) [2023-12-27 15:46:02,363][19170] Avg episode reward: [(0, '736.090'), (1, '777.510')] [2023-12-27 15:46:02,457][20347] Updated weights for policy 0, policy_version 1460665 (0.0008) [2023-12-27 15:46:02,519][20347] Updated weights for policy 0, policy_version 1460675 (0.0007) [2023-12-27 15:46:02,548][20381] Updated weights for policy 1, policy_version 1459257 (0.0007) [2023-12-27 15:46:02,581][20347] Updated weights for policy 0, policy_version 1460685 (0.0006) [2023-12-27 15:46:02,597][20381] Updated weights for policy 1, policy_version 1459267 (0.0006) [2023-12-27 15:46:02,641][20347] Updated weights for policy 0, policy_version 1460695 (0.0006) [2023-12-27 15:46:02,658][20381] Updated weights for policy 1, policy_version 1459277 (0.0008) [2023-12-27 15:46:03,275][20381] Updated weights for policy 1, policy_version 1459287 (0.0007) [2023-12-27 15:46:03,296][20347] Updated weights for policy 0, policy_version 1460705 (0.0006) [2023-12-27 15:46:03,324][20381] Updated weights for policy 1, policy_version 1459297 (0.0007) [2023-12-27 15:46:03,344][20347] Updated weights for policy 0, policy_version 1460715 (0.0005) [2023-12-27 15:46:03,372][20381] Updated weights for policy 1, policy_version 1459307 (0.0007) [2023-12-27 15:46:03,395][20347] Updated weights for policy 0, policy_version 1460725 (0.0005) [2023-12-27 15:46:04,011][20347] Updated weights for policy 0, policy_version 1460735 (0.0006) [2023-12-27 15:46:04,074][20347] Updated weights for policy 0, policy_version 1460745 (0.0008) [2023-12-27 15:46:04,104][20381] Updated weights for policy 1, policy_version 1459317 (0.0006) [2023-12-27 15:46:04,135][20347] Updated weights for policy 0, policy_version 1460755 (0.0006) [2023-12-27 15:46:04,171][20381] Updated weights for policy 1, policy_version 1459327 (0.0006) [2023-12-27 15:46:04,242][20381] Updated weights for policy 1, policy_version 1459337 (0.0006) [2023-12-27 15:46:04,880][20347] Updated weights for policy 0, policy_version 1460765 (0.0007) [2023-12-27 15:46:04,882][20381] Updated weights for policy 1, policy_version 1459347 (0.0007) [2023-12-27 15:46:04,934][20347] Updated weights for policy 0, policy_version 1460775 (0.0008) [2023-12-27 15:46:04,937][20381] Updated weights for policy 1, policy_version 1459357 (0.0008) [2023-12-27 15:46:04,991][20381] Updated weights for policy 1, policy_version 1459367 (0.0006) [2023-12-27 15:46:04,992][20347] Updated weights for policy 0, policy_version 1460785 (0.0007) [2023-12-27 15:46:05,572][20381] Updated weights for policy 1, policy_version 1459377 (0.0006) [2023-12-27 15:46:05,622][20381] Updated weights for policy 1, policy_version 1459387 (0.0007) [2023-12-27 15:46:05,666][20381] Updated weights for policy 1, policy_version 1459397 (0.0007) [2023-12-27 15:46:05,710][20347] Updated weights for policy 0, policy_version 1460795 (0.0007) [2023-12-27 15:46:05,712][20381] Updated weights for policy 1, policy_version 1459407 (0.0007) [2023-12-27 15:46:05,766][20347] Updated weights for policy 0, policy_version 1460805 (0.0007) [2023-12-27 15:46:05,817][20347] Updated weights for policy 0, policy_version 1460815 (0.0007) [2023-12-27 15:46:06,471][20381] Updated weights for policy 1, policy_version 1459417 (0.0009) [2023-12-27 15:46:06,520][20347] Updated weights for policy 0, policy_version 1460825 (0.0008) [2023-12-27 15:46:06,530][20381] Updated weights for policy 1, policy_version 1459427 (0.0007) [2023-12-27 15:46:06,586][20347] Updated weights for policy 0, policy_version 1460835 (0.0008) [2023-12-27 15:46:06,592][20381] Updated weights for policy 1, policy_version 1459437 (0.0006) [2023-12-27 15:46:06,646][20347] Updated weights for policy 0, policy_version 1460845 (0.0007) [2023-12-27 15:46:06,703][20347] Updated weights for policy 0, policy_version 1460855 (0.0008) [2023-12-27 15:46:07,267][20381] Updated weights for policy 1, policy_version 1459447 (0.0005) [2023-12-27 15:46:07,316][20381] Updated weights for policy 1, policy_version 1459457 (0.0006) [2023-12-27 15:46:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 747700224. Throughput: 0: 10120.1, 1: 10259.6. Samples: 747676100. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:07,363][19170] Avg episode reward: [(0, '686.790'), (1, '700.860')] [2023-12-27 15:46:07,373][20381] Updated weights for policy 1, policy_version 1459467 (0.0007) [2023-12-27 15:46:07,396][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001459472_373653504.pth... [2023-12-27 15:46:07,399][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001458256_373342208.pth [2023-12-27 15:46:07,404][20347] Updated weights for policy 0, policy_version 1460865 (0.0007) [2023-12-27 15:46:07,458][20347] Updated weights for policy 0, policy_version 1460875 (0.0006) [2023-12-27 15:46:07,521][20347] Updated weights for policy 0, policy_version 1460885 (0.0006) [2023-12-27 15:46:07,538][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001460888_374063104.pth... [2023-12-27 15:46:07,542][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001459704_373760000.pth [2023-12-27 15:46:08,045][20381] Updated weights for policy 1, policy_version 1459477 (0.0007) [2023-12-27 15:46:08,103][20381] Updated weights for policy 1, policy_version 1459487 (0.0008) [2023-12-27 15:46:08,140][20347] Updated weights for policy 0, policy_version 1460895 (0.0006) [2023-12-27 15:46:08,162][20381] Updated weights for policy 1, policy_version 1459497 (0.0007) [2023-12-27 15:46:08,196][20347] Updated weights for policy 0, policy_version 1460905 (0.0006) [2023-12-27 15:46:08,254][20347] Updated weights for policy 0, policy_version 1460915 (0.0007) [2023-12-27 15:46:08,895][20381] Updated weights for policy 1, policy_version 1459507 (0.0005) [2023-12-27 15:46:08,896][20347] Updated weights for policy 0, policy_version 1460925 (0.0006) [2023-12-27 15:46:08,945][20381] Updated weights for policy 1, policy_version 1459517 (0.0006) [2023-12-27 15:46:08,952][20347] Updated weights for policy 0, policy_version 1460935 (0.0006) [2023-12-27 15:46:08,993][20381] Updated weights for policy 1, policy_version 1459527 (0.0005) [2023-12-27 15:46:09,002][20347] Updated weights for policy 0, policy_version 1460945 (0.0006) [2023-12-27 15:46:09,632][20381] Updated weights for policy 1, policy_version 1459537 (0.0005) [2023-12-27 15:46:09,690][20347] Updated weights for policy 0, policy_version 1460955 (0.0007) [2023-12-27 15:46:09,691][20381] Updated weights for policy 1, policy_version 1459547 (0.0007) [2023-12-27 15:46:09,752][20347] Updated weights for policy 0, policy_version 1460965 (0.0007) [2023-12-27 15:46:09,753][20381] Updated weights for policy 1, policy_version 1459557 (0.0007) [2023-12-27 15:46:09,807][20381] Updated weights for policy 1, policy_version 1459567 (0.0006) [2023-12-27 15:46:09,820][20347] Updated weights for policy 0, policy_version 1460975 (0.0007) [2023-12-27 15:46:10,484][20381] Updated weights for policy 1, policy_version 1459577 (0.0010) [2023-12-27 15:46:10,546][20381] Updated weights for policy 1, policy_version 1459587 (0.0011) [2023-12-27 15:46:10,566][20347] Updated weights for policy 0, policy_version 1460985 (0.0007) [2023-12-27 15:46:10,608][20381] Updated weights for policy 1, policy_version 1459597 (0.0010) [2023-12-27 15:46:10,621][20347] Updated weights for policy 0, policy_version 1460995 (0.0005) [2023-12-27 15:46:10,678][20347] Updated weights for policy 0, policy_version 1461005 (0.0008) [2023-12-27 15:46:10,734][20347] Updated weights for policy 0, policy_version 1461015 (0.0007) [2023-12-27 15:46:11,309][20381] Updated weights for policy 1, policy_version 1459607 (0.0010) [2023-12-27 15:46:11,379][20381] Updated weights for policy 1, policy_version 1459617 (0.0011) [2023-12-27 15:46:11,435][20381] Updated weights for policy 1, policy_version 1459627 (0.0011) [2023-12-27 15:46:11,464][20347] Updated weights for policy 0, policy_version 1461025 (0.0006) [2023-12-27 15:46:11,521][20347] Updated weights for policy 0, policy_version 1461035 (0.0008) [2023-12-27 15:46:11,575][20347] Updated weights for policy 0, policy_version 1461045 (0.0007) [2023-12-27 15:46:12,145][20381] Updated weights for policy 1, policy_version 1459637 (0.0010) [2023-12-27 15:46:12,194][20381] Updated weights for policy 1, policy_version 1459647 (0.0010) [2023-12-27 15:46:12,242][20381] Updated weights for policy 1, policy_version 1459657 (0.0010) [2023-12-27 15:46:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 747806720. Throughput: 0: 10091.1, 1: 10208.8. Samples: 747797956. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:12,363][19170] Avg episode reward: [(0, '745.710'), (1, '815.610')] [2023-12-27 15:46:12,384][20347] Updated weights for policy 0, policy_version 1461055 (0.0008) [2023-12-27 15:46:12,442][20347] Updated weights for policy 0, policy_version 1461065 (0.0007) [2023-12-27 15:46:12,501][20347] Updated weights for policy 0, policy_version 1461075 (0.0007) [2023-12-27 15:46:12,963][20381] Updated weights for policy 1, policy_version 1459667 (0.0010) [2023-12-27 15:46:13,022][20381] Updated weights for policy 1, policy_version 1459677 (0.0008) [2023-12-27 15:46:13,078][20381] Updated weights for policy 1, policy_version 1459687 (0.0007) [2023-12-27 15:46:13,192][20347] Updated weights for policy 0, policy_version 1461085 (0.0009) [2023-12-27 15:46:13,249][20347] Updated weights for policy 0, policy_version 1461095 (0.0010) [2023-12-27 15:46:13,304][20347] Updated weights for policy 0, policy_version 1461105 (0.0007) [2023-12-27 15:46:13,722][20381] Updated weights for policy 1, policy_version 1459697 (0.0008) [2023-12-27 15:46:13,777][20381] Updated weights for policy 1, policy_version 1459707 (0.0011) [2023-12-27 15:46:13,827][20381] Updated weights for policy 1, policy_version 1459717 (0.0011) [2023-12-27 15:46:13,874][20381] Updated weights for policy 1, policy_version 1459727 (0.0007) [2023-12-27 15:46:13,954][20347] Updated weights for policy 0, policy_version 1461115 (0.0007) [2023-12-27 15:46:14,011][20347] Updated weights for policy 0, policy_version 1461125 (0.0010) [2023-12-27 15:46:14,067][20347] Updated weights for policy 0, policy_version 1461135 (0.0009) [2023-12-27 15:46:14,493][20381] Updated weights for policy 1, policy_version 1459737 (0.0006) [2023-12-27 15:46:14,543][20381] Updated weights for policy 1, policy_version 1459747 (0.0008) [2023-12-27 15:46:14,589][20381] Updated weights for policy 1, policy_version 1459757 (0.0008) [2023-12-27 15:46:14,740][20347] Updated weights for policy 0, policy_version 1461145 (0.0010) [2023-12-27 15:46:14,812][20347] Updated weights for policy 0, policy_version 1461155 (0.0011) [2023-12-27 15:46:14,872][20347] Updated weights for policy 0, policy_version 1461165 (0.0010) [2023-12-27 15:46:14,931][20347] Updated weights for policy 0, policy_version 1461175 (0.0010) [2023-12-27 15:46:15,183][20381] Updated weights for policy 1, policy_version 1459767 (0.0009) [2023-12-27 15:46:15,242][20381] Updated weights for policy 1, policy_version 1459777 (0.0011) [2023-12-27 15:46:15,301][20381] Updated weights for policy 1, policy_version 1459787 (0.0011) [2023-12-27 15:46:15,608][20347] Updated weights for policy 0, policy_version 1461185 (0.0010) [2023-12-27 15:46:15,665][20347] Updated weights for policy 0, policy_version 1461195 (0.0010) [2023-12-27 15:46:15,719][20347] Updated weights for policy 0, policy_version 1461205 (0.0010) [2023-12-27 15:46:15,948][20381] Updated weights for policy 1, policy_version 1459797 (0.0010) [2023-12-27 15:46:15,996][20381] Updated weights for policy 1, policy_version 1459807 (0.0010) [2023-12-27 15:46:16,055][20381] Updated weights for policy 1, policy_version 1459817 (0.0010) [2023-12-27 15:46:16,415][20347] Updated weights for policy 0, policy_version 1461215 (0.0008) [2023-12-27 15:46:16,473][20347] Updated weights for policy 0, policy_version 1461225 (0.0007) [2023-12-27 15:46:16,534][20347] Updated weights for policy 0, policy_version 1461235 (0.0009) [2023-12-27 15:46:16,704][20381] Updated weights for policy 1, policy_version 1459827 (0.0009) [2023-12-27 15:46:16,760][20381] Updated weights for policy 1, policy_version 1459837 (0.0009) [2023-12-27 15:46:16,807][20381] Updated weights for policy 1, policy_version 1459847 (0.0010) [2023-12-27 15:46:17,120][20347] Updated weights for policy 0, policy_version 1461245 (0.0006) [2023-12-27 15:46:17,179][20347] Updated weights for policy 0, policy_version 1461255 (0.0007) [2023-12-27 15:46:17,227][20347] Updated weights for policy 0, policy_version 1461265 (0.0005) [2023-12-27 15:46:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 747913216. Throughput: 0: 10055.3, 1: 10254.0. Samples: 747921764. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:17,363][19170] Avg episode reward: [(0, '745.510'), (1, '864.400')] [2023-12-27 15:46:17,441][20381] Updated weights for policy 1, policy_version 1459857 (0.0010) [2023-12-27 15:46:17,493][20381] Updated weights for policy 1, policy_version 1459867 (0.0007) [2023-12-27 15:46:17,550][20381] Updated weights for policy 1, policy_version 1459877 (0.0009) [2023-12-27 15:46:17,606][20381] Updated weights for policy 1, policy_version 1459887 (0.0007) [2023-12-27 15:46:17,771][20347] Updated weights for policy 0, policy_version 1461275 (0.0007) [2023-12-27 15:46:17,841][20347] Updated weights for policy 0, policy_version 1461285 (0.0010) [2023-12-27 15:46:17,905][20347] Updated weights for policy 0, policy_version 1461295 (0.0009) [2023-12-27 15:46:18,240][20381] Updated weights for policy 1, policy_version 1459897 (0.0006) [2023-12-27 15:46:18,288][20381] Updated weights for policy 1, policy_version 1459907 (0.0006) [2023-12-27 15:46:18,350][20381] Updated weights for policy 1, policy_version 1459917 (0.0008) [2023-12-27 15:46:18,582][20347] Updated weights for policy 0, policy_version 1461305 (0.0008) [2023-12-27 15:46:18,641][20347] Updated weights for policy 0, policy_version 1461315 (0.0011) [2023-12-27 15:46:18,696][20347] Updated weights for policy 0, policy_version 1461325 (0.0010) [2023-12-27 15:46:18,748][20347] Updated weights for policy 0, policy_version 1461335 (0.0010) [2023-12-27 15:46:19,017][20381] Updated weights for policy 1, policy_version 1459927 (0.0008) [2023-12-27 15:46:19,071][20381] Updated weights for policy 1, policy_version 1459937 (0.0005) [2023-12-27 15:46:19,128][20381] Updated weights for policy 1, policy_version 1459947 (0.0007) [2023-12-27 15:46:19,407][20347] Updated weights for policy 0, policy_version 1461345 (0.0007) [2023-12-27 15:46:19,462][20347] Updated weights for policy 0, policy_version 1461355 (0.0006) [2023-12-27 15:46:19,527][20347] Updated weights for policy 0, policy_version 1461365 (0.0007) [2023-12-27 15:46:19,875][20381] Updated weights for policy 1, policy_version 1459957 (0.0007) [2023-12-27 15:46:19,946][20381] Updated weights for policy 1, policy_version 1459967 (0.0007) [2023-12-27 15:46:20,008][20381] Updated weights for policy 1, policy_version 1459977 (0.0009) [2023-12-27 15:46:20,161][20347] Updated weights for policy 0, policy_version 1461375 (0.0006) [2023-12-27 15:46:20,229][20347] Updated weights for policy 0, policy_version 1461385 (0.0006) [2023-12-27 15:46:20,292][20347] Updated weights for policy 0, policy_version 1461395 (0.0006) [2023-12-27 15:46:20,728][20381] Updated weights for policy 1, policy_version 1459987 (0.0007) [2023-12-27 15:46:20,792][20381] Updated weights for policy 1, policy_version 1459997 (0.0006) [2023-12-27 15:46:20,856][20381] Updated weights for policy 1, policy_version 1460007 (0.0005) [2023-12-27 15:46:20,963][20347] Updated weights for policy 0, policy_version 1461405 (0.0006) [2023-12-27 15:46:21,020][20347] Updated weights for policy 0, policy_version 1461415 (0.0007) [2023-12-27 15:46:21,087][20347] Updated weights for policy 0, policy_version 1461425 (0.0008) [2023-12-27 15:46:21,571][20381] Updated weights for policy 1, policy_version 1460017 (0.0007) [2023-12-27 15:46:21,637][20381] Updated weights for policy 1, policy_version 1460027 (0.0010) [2023-12-27 15:46:21,685][20381] Updated weights for policy 1, policy_version 1460037 (0.0008) [2023-12-27 15:46:21,737][20381] Updated weights for policy 1, policy_version 1460047 (0.0009) [2023-12-27 15:46:21,786][20347] Updated weights for policy 0, policy_version 1461435 (0.0008) [2023-12-27 15:46:21,850][20347] Updated weights for policy 0, policy_version 1461445 (0.0008) [2023-12-27 15:46:21,913][20347] Updated weights for policy 0, policy_version 1461455 (0.0008) [2023-12-27 15:46:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 748011520. Throughput: 0: 10081.7, 1: 10251.6. Samples: 747984484. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:22,363][19170] Avg episode reward: [(0, '706.320'), (1, '864.100')] [2023-12-27 15:46:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001460048_373800960.pth... [2023-12-27 15:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001461464_374210560.pth... [2023-12-27 15:46:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001458864_373497856.pth [2023-12-27 15:46:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001460280_373907456.pth [2023-12-27 15:46:22,494][20381] Updated weights for policy 1, policy_version 1460057 (0.0007) [2023-12-27 15:46:22,544][20381] Updated weights for policy 1, policy_version 1460067 (0.0009) [2023-12-27 15:46:22,599][20381] Updated weights for policy 1, policy_version 1460077 (0.0008) [2023-12-27 15:46:22,647][20347] Updated weights for policy 0, policy_version 1461465 (0.0007) [2023-12-27 15:46:22,706][20347] Updated weights for policy 0, policy_version 1461475 (0.0008) [2023-12-27 15:46:22,763][20347] Updated weights for policy 0, policy_version 1461485 (0.0008) [2023-12-27 15:46:22,819][20347] Updated weights for policy 0, policy_version 1461495 (0.0007) [2023-12-27 15:46:23,322][20381] Updated weights for policy 1, policy_version 1460087 (0.0008) [2023-12-27 15:46:23,372][20381] Updated weights for policy 1, policy_version 1460097 (0.0009) [2023-12-27 15:46:23,424][20381] Updated weights for policy 1, policy_version 1460107 (0.0007) [2023-12-27 15:46:23,474][20347] Updated weights for policy 0, policy_version 1461505 (0.0007) [2023-12-27 15:46:23,523][20347] Updated weights for policy 0, policy_version 1461515 (0.0007) [2023-12-27 15:46:23,571][20347] Updated weights for policy 0, policy_version 1461525 (0.0007) [2023-12-27 15:46:24,175][20381] Updated weights for policy 1, policy_version 1460117 (0.0007) [2023-12-27 15:46:24,187][20347] Updated weights for policy 0, policy_version 1461535 (0.0006) [2023-12-27 15:46:24,234][20381] Updated weights for policy 1, policy_version 1460127 (0.0006) [2023-12-27 15:46:24,245][20347] Updated weights for policy 0, policy_version 1461545 (0.0008) [2023-12-27 15:46:24,284][20381] Updated weights for policy 1, policy_version 1460137 (0.0005) [2023-12-27 15:46:24,303][20347] Updated weights for policy 0, policy_version 1461555 (0.0007) [2023-12-27 15:46:24,855][20381] Updated weights for policy 1, policy_version 1460147 (0.0007) [2023-12-27 15:46:24,916][20381] Updated weights for policy 1, policy_version 1460157 (0.0010) [2023-12-27 15:46:24,978][20381] Updated weights for policy 1, policy_version 1460167 (0.0011) [2023-12-27 15:46:24,994][20347] Updated weights for policy 0, policy_version 1461565 (0.0007) [2023-12-27 15:46:25,045][20347] Updated weights for policy 0, policy_version 1461575 (0.0005) [2023-12-27 15:46:25,095][20347] Updated weights for policy 0, policy_version 1461585 (0.0005) [2023-12-27 15:46:25,623][20381] Updated weights for policy 1, policy_version 1460177 (0.0011) [2023-12-27 15:46:25,674][20381] Updated weights for policy 1, policy_version 1460187 (0.0010) [2023-12-27 15:46:25,729][20381] Updated weights for policy 1, policy_version 1460197 (0.0006) [2023-12-27 15:46:25,754][20347] Updated weights for policy 0, policy_version 1461595 (0.0006) [2023-12-27 15:46:25,781][20381] Updated weights for policy 1, policy_version 1460207 (0.0005) [2023-12-27 15:46:25,802][20347] Updated weights for policy 0, policy_version 1461605 (0.0007) [2023-12-27 15:46:25,852][20347] Updated weights for policy 0, policy_version 1461615 (0.0010) [2023-12-27 15:46:26,364][20381] Updated weights for policy 1, policy_version 1460217 (0.0010) [2023-12-27 15:46:26,408][20381] Updated weights for policy 1, policy_version 1460227 (0.0010) [2023-12-27 15:46:26,462][20381] Updated weights for policy 1, policy_version 1460237 (0.0010) [2023-12-27 15:46:26,575][20347] Updated weights for policy 0, policy_version 1461625 (0.0009) [2023-12-27 15:46:26,632][20347] Updated weights for policy 0, policy_version 1461635 (0.0007) [2023-12-27 15:46:26,673][20347] Updated weights for policy 0, policy_version 1461645 (0.0008) [2023-12-27 15:46:26,717][20347] Updated weights for policy 0, policy_version 1461655 (0.0008) [2023-12-27 15:46:27,084][20381] Updated weights for policy 1, policy_version 1460247 (0.0007) [2023-12-27 15:46:27,136][20381] Updated weights for policy 1, policy_version 1460257 (0.0006) [2023-12-27 15:46:27,193][20381] Updated weights for policy 1, policy_version 1460267 (0.0010) [2023-12-27 15:46:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 748118016. Throughput: 0: 10143.6, 1: 10290.6. Samples: 748107068. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:27,363][19170] Avg episode reward: [(0, '794.400'), (1, '892.620')] [2023-12-27 15:46:27,461][20347] Updated weights for policy 0, policy_version 1461665 (0.0007) [2023-12-27 15:46:27,519][20347] Updated weights for policy 0, policy_version 1461675 (0.0007) [2023-12-27 15:46:27,572][20347] Updated weights for policy 0, policy_version 1461685 (0.0008) [2023-12-27 15:46:27,803][20381] Updated weights for policy 1, policy_version 1460277 (0.0010) [2023-12-27 15:46:27,860][20381] Updated weights for policy 1, policy_version 1460287 (0.0010) [2023-12-27 15:46:27,914][20381] Updated weights for policy 1, policy_version 1460297 (0.0011) [2023-12-27 15:46:28,277][20347] Updated weights for policy 0, policy_version 1461695 (0.0008) [2023-12-27 15:46:28,331][20347] Updated weights for policy 0, policy_version 1461705 (0.0007) [2023-12-27 15:46:28,386][20347] Updated weights for policy 0, policy_version 1461715 (0.0007) [2023-12-27 15:46:28,517][20381] Updated weights for policy 1, policy_version 1460307 (0.0009) [2023-12-27 15:46:28,567][20381] Updated weights for policy 1, policy_version 1460317 (0.0010) [2023-12-27 15:46:28,619][20381] Updated weights for policy 1, policy_version 1460327 (0.0010) [2023-12-27 15:46:29,124][20347] Updated weights for policy 0, policy_version 1461725 (0.0006) [2023-12-27 15:46:29,180][20347] Updated weights for policy 0, policy_version 1461735 (0.0005) [2023-12-27 15:46:29,233][20381] Updated weights for policy 1, policy_version 1460337 (0.0009) [2023-12-27 15:46:29,256][20347] Updated weights for policy 0, policy_version 1461745 (0.0008) [2023-12-27 15:46:29,295][20381] Updated weights for policy 1, policy_version 1460347 (0.0009) [2023-12-27 15:46:29,361][20381] Updated weights for policy 1, policy_version 1460357 (0.0010) [2023-12-27 15:46:29,427][20381] Updated weights for policy 1, policy_version 1460367 (0.0008) [2023-12-27 15:46:29,852][20347] Updated weights for policy 0, policy_version 1461755 (0.0010) [2023-12-27 15:46:29,906][20347] Updated weights for policy 0, policy_version 1461765 (0.0007) [2023-12-27 15:46:29,969][20347] Updated weights for policy 0, policy_version 1461775 (0.0008) [2023-12-27 15:46:30,167][20381] Updated weights for policy 1, policy_version 1460377 (0.0010) [2023-12-27 15:46:30,219][20381] Updated weights for policy 1, policy_version 1460387 (0.0010) [2023-12-27 15:46:30,268][20381] Updated weights for policy 1, policy_version 1460397 (0.0011) [2023-12-27 15:46:30,669][20347] Updated weights for policy 0, policy_version 1461785 (0.0007) [2023-12-27 15:46:30,723][20347] Updated weights for policy 0, policy_version 1461795 (0.0007) [2023-12-27 15:46:30,769][20347] Updated weights for policy 0, policy_version 1461805 (0.0007) [2023-12-27 15:46:30,815][20347] Updated weights for policy 0, policy_version 1461815 (0.0007) [2023-12-27 15:46:30,961][20381] Updated weights for policy 1, policy_version 1460407 (0.0011) [2023-12-27 15:46:31,013][20381] Updated weights for policy 1, policy_version 1460417 (0.0010) [2023-12-27 15:46:31,079][20381] Updated weights for policy 1, policy_version 1460427 (0.0011) [2023-12-27 15:46:31,597][20347] Updated weights for policy 0, policy_version 1461825 (0.0008) [2023-12-27 15:46:31,660][20347] Updated weights for policy 0, policy_version 1461835 (0.0007) [2023-12-27 15:46:31,713][20347] Updated weights for policy 0, policy_version 1461845 (0.0005) [2023-12-27 15:46:31,802][20381] Updated weights for policy 1, policy_version 1460437 (0.0009) [2023-12-27 15:46:31,847][20381] Updated weights for policy 1, policy_version 1460447 (0.0009) [2023-12-27 15:46:31,896][20381] Updated weights for policy 1, policy_version 1460457 (0.0008) [2023-12-27 15:46:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 748216320. Throughput: 0: 10134.4, 1: 10304.1. Samples: 748229852. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:32,363][19170] Avg episode reward: [(0, '804.070'), (1, '843.840')] [2023-12-27 15:46:32,381][20347] Updated weights for policy 0, policy_version 1461855 (0.0006) [2023-12-27 15:46:32,437][20347] Updated weights for policy 0, policy_version 1461865 (0.0005) [2023-12-27 15:46:32,492][20347] Updated weights for policy 0, policy_version 1461875 (0.0008) [2023-12-27 15:46:32,640][20381] Updated weights for policy 1, policy_version 1460467 (0.0009) [2023-12-27 15:46:32,696][20381] Updated weights for policy 1, policy_version 1460477 (0.0007) [2023-12-27 15:46:32,755][20381] Updated weights for policy 1, policy_version 1460487 (0.0007) [2023-12-27 15:46:33,168][20347] Updated weights for policy 0, policy_version 1461885 (0.0007) [2023-12-27 15:46:33,230][20347] Updated weights for policy 0, policy_version 1461895 (0.0006) [2023-12-27 15:46:33,288][20347] Updated weights for policy 0, policy_version 1461905 (0.0008) [2023-12-27 15:46:33,448][20381] Updated weights for policy 1, policy_version 1460497 (0.0008) [2023-12-27 15:46:33,513][20381] Updated weights for policy 1, policy_version 1460507 (0.0006) [2023-12-27 15:46:33,571][20381] Updated weights for policy 1, policy_version 1460517 (0.0006) [2023-12-27 15:46:33,631][20381] Updated weights for policy 1, policy_version 1460527 (0.0008) [2023-12-27 15:46:33,879][20347] Updated weights for policy 0, policy_version 1461915 (0.0006) [2023-12-27 15:46:33,938][20347] Updated weights for policy 0, policy_version 1461925 (0.0006) [2023-12-27 15:46:33,991][20347] Updated weights for policy 0, policy_version 1461935 (0.0005) [2023-12-27 15:46:34,317][20381] Updated weights for policy 1, policy_version 1460537 (0.0011) [2023-12-27 15:46:34,377][20381] Updated weights for policy 1, policy_version 1460547 (0.0011) [2023-12-27 15:46:34,437][20381] Updated weights for policy 1, policy_version 1460557 (0.0011) [2023-12-27 15:46:34,605][20347] Updated weights for policy 0, policy_version 1461945 (0.0007) [2023-12-27 15:46:34,669][20347] Updated weights for policy 0, policy_version 1461955 (0.0007) [2023-12-27 15:46:34,729][20347] Updated weights for policy 0, policy_version 1461965 (0.0007) [2023-12-27 15:46:34,789][20347] Updated weights for policy 0, policy_version 1461975 (0.0007) [2023-12-27 15:46:35,074][20381] Updated weights for policy 1, policy_version 1460567 (0.0011) [2023-12-27 15:46:35,134][20381] Updated weights for policy 1, policy_version 1460577 (0.0011) [2023-12-27 15:46:35,199][20381] Updated weights for policy 1, policy_version 1460587 (0.0006) [2023-12-27 15:46:35,465][20347] Updated weights for policy 0, policy_version 1461985 (0.0007) [2023-12-27 15:46:35,523][20347] Updated weights for policy 0, policy_version 1461995 (0.0008) [2023-12-27 15:46:35,581][20347] Updated weights for policy 0, policy_version 1462005 (0.0007) [2023-12-27 15:46:35,853][20381] Updated weights for policy 1, policy_version 1460597 (0.0007) [2023-12-27 15:46:35,914][20381] Updated weights for policy 1, policy_version 1460607 (0.0008) [2023-12-27 15:46:35,966][20381] Updated weights for policy 1, policy_version 1460617 (0.0006) [2023-12-27 15:46:36,280][20347] Updated weights for policy 0, policy_version 1462015 (0.0007) [2023-12-27 15:46:36,347][20347] Updated weights for policy 0, policy_version 1462025 (0.0006) [2023-12-27 15:46:36,412][20347] Updated weights for policy 0, policy_version 1462035 (0.0007) [2023-12-27 15:46:36,692][20381] Updated weights for policy 1, policy_version 1460627 (0.0010) [2023-12-27 15:46:36,749][20381] Updated weights for policy 1, policy_version 1460637 (0.0008) [2023-12-27 15:46:36,809][20381] Updated weights for policy 1, policy_version 1460647 (0.0010) [2023-12-27 15:46:37,058][20347] Updated weights for policy 0, policy_version 1462045 (0.0008) [2023-12-27 15:46:37,121][20347] Updated weights for policy 0, policy_version 1462055 (0.0006) [2023-12-27 15:46:37,178][20347] Updated weights for policy 0, policy_version 1462065 (0.0007) [2023-12-27 15:46:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 748322816. Throughput: 0: 10144.0, 1: 10276.1. Samples: 748291816. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:37,363][19170] Avg episode reward: [(0, '715.680'), (1, '833.970')] [2023-12-27 15:46:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001462072_374366208.pth... [2023-12-27 15:46:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001460656_373956608.pth... [2023-12-27 15:46:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001460888_374063104.pth [2023-12-27 15:46:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001459472_373653504.pth [2023-12-27 15:46:37,502][20381] Updated weights for policy 1, policy_version 1460657 (0.0008) [2023-12-27 15:46:37,565][20381] Updated weights for policy 1, policy_version 1460667 (0.0009) [2023-12-27 15:46:37,625][20381] Updated weights for policy 1, policy_version 1460677 (0.0010) [2023-12-27 15:46:37,681][20381] Updated weights for policy 1, policy_version 1460687 (0.0007) [2023-12-27 15:46:37,824][20347] Updated weights for policy 0, policy_version 1462075 (0.0009) [2023-12-27 15:46:37,883][20347] Updated weights for policy 0, policy_version 1462085 (0.0008) [2023-12-27 15:46:37,943][20347] Updated weights for policy 0, policy_version 1462095 (0.0008) [2023-12-27 15:46:38,287][20381] Updated weights for policy 1, policy_version 1460697 (0.0005) [2023-12-27 15:46:38,345][20381] Updated weights for policy 1, policy_version 1460707 (0.0007) [2023-12-27 15:46:38,431][20381] Updated weights for policy 1, policy_version 1460717 (0.0008) [2023-12-27 15:46:38,584][20347] Updated weights for policy 0, policy_version 1462105 (0.0007) [2023-12-27 15:46:38,650][20347] Updated weights for policy 0, policy_version 1462115 (0.0008) [2023-12-27 15:46:38,711][20347] Updated weights for policy 0, policy_version 1462125 (0.0007) [2023-12-27 15:46:38,772][20347] Updated weights for policy 0, policy_version 1462135 (0.0007) [2023-12-27 15:46:39,031][20381] Updated weights for policy 1, policy_version 1460727 (0.0008) [2023-12-27 15:46:39,083][20381] Updated weights for policy 1, policy_version 1460737 (0.0008) [2023-12-27 15:46:39,139][20381] Updated weights for policy 1, policy_version 1460747 (0.0007) [2023-12-27 15:46:39,487][20347] Updated weights for policy 0, policy_version 1462145 (0.0008) [2023-12-27 15:46:39,546][20347] Updated weights for policy 0, policy_version 1462155 (0.0008) [2023-12-27 15:46:39,612][20347] Updated weights for policy 0, policy_version 1462165 (0.0007) [2023-12-27 15:46:39,850][20381] Updated weights for policy 1, policy_version 1460757 (0.0008) [2023-12-27 15:46:39,909][20381] Updated weights for policy 1, policy_version 1460767 (0.0007) [2023-12-27 15:46:39,963][20381] Updated weights for policy 1, policy_version 1460777 (0.0008) [2023-12-27 15:46:40,384][20347] Updated weights for policy 0, policy_version 1462175 (0.0008) [2023-12-27 15:46:40,438][20347] Updated weights for policy 0, policy_version 1462185 (0.0006) [2023-12-27 15:46:40,495][20347] Updated weights for policy 0, policy_version 1462195 (0.0005) [2023-12-27 15:46:40,691][20381] Updated weights for policy 1, policy_version 1460787 (0.0007) [2023-12-27 15:46:40,750][20381] Updated weights for policy 1, policy_version 1460797 (0.0008) [2023-12-27 15:46:40,819][20381] Updated weights for policy 1, policy_version 1460807 (0.0007) [2023-12-27 15:46:41,096][20347] Updated weights for policy 0, policy_version 1462205 (0.0006) [2023-12-27 15:46:41,162][20347] Updated weights for policy 0, policy_version 1462215 (0.0007) [2023-12-27 15:46:41,222][20347] Updated weights for policy 0, policy_version 1462225 (0.0007) [2023-12-27 15:46:41,539][20381] Updated weights for policy 1, policy_version 1460817 (0.0008) [2023-12-27 15:46:41,603][20381] Updated weights for policy 1, policy_version 1460827 (0.0007) [2023-12-27 15:46:41,678][20381] Updated weights for policy 1, policy_version 1460837 (0.0009) [2023-12-27 15:46:41,739][20381] Updated weights for policy 1, policy_version 1460847 (0.0007) [2023-12-27 15:46:41,920][20347] Updated weights for policy 0, policy_version 1462235 (0.0007) [2023-12-27 15:46:41,975][20347] Updated weights for policy 0, policy_version 1462245 (0.0007) [2023-12-27 15:46:42,034][20347] Updated weights for policy 0, policy_version 1462255 (0.0008) [2023-12-27 15:46:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 748421120. Throughput: 0: 10178.8, 1: 10250.7. Samples: 748413972. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:42,363][19170] Avg episode reward: [(0, '676.430'), (1, '843.860')] [2023-12-27 15:46:42,402][20381] Updated weights for policy 1, policy_version 1460857 (0.0008) [2023-12-27 15:46:42,460][20381] Updated weights for policy 1, policy_version 1460867 (0.0007) [2023-12-27 15:46:42,516][20381] Updated weights for policy 1, policy_version 1460877 (0.0008) [2023-12-27 15:46:42,752][20347] Updated weights for policy 0, policy_version 1462265 (0.0008) [2023-12-27 15:46:42,805][20347] Updated weights for policy 0, policy_version 1462275 (0.0007) [2023-12-27 15:46:42,869][20347] Updated weights for policy 0, policy_version 1462285 (0.0008) [2023-12-27 15:46:42,922][20347] Updated weights for policy 0, policy_version 1462295 (0.0007) [2023-12-27 15:46:43,218][20381] Updated weights for policy 1, policy_version 1460887 (0.0007) [2023-12-27 15:46:43,270][20381] Updated weights for policy 1, policy_version 1460897 (0.0008) [2023-12-27 15:46:43,325][20381] Updated weights for policy 1, policy_version 1460907 (0.0008) [2023-12-27 15:46:43,574][20347] Updated weights for policy 0, policy_version 1462305 (0.0006) [2023-12-27 15:46:43,619][20347] Updated weights for policy 0, policy_version 1462315 (0.0007) [2023-12-27 15:46:43,676][20347] Updated weights for policy 0, policy_version 1462325 (0.0007) [2023-12-27 15:46:44,060][20381] Updated weights for policy 1, policy_version 1460917 (0.0008) [2023-12-27 15:46:44,115][20381] Updated weights for policy 1, policy_version 1460927 (0.0008) [2023-12-27 15:46:44,163][20381] Updated weights for policy 1, policy_version 1460937 (0.0007) [2023-12-27 15:46:44,256][20347] Updated weights for policy 0, policy_version 1462335 (0.0007) [2023-12-27 15:46:44,306][20347] Updated weights for policy 0, policy_version 1462345 (0.0008) [2023-12-27 15:46:44,357][20347] Updated weights for policy 0, policy_version 1462355 (0.0007) [2023-12-27 15:46:44,889][20381] Updated weights for policy 1, policy_version 1460947 (0.0007) [2023-12-27 15:46:44,954][20381] Updated weights for policy 1, policy_version 1460957 (0.0006) [2023-12-27 15:46:45,016][20381] Updated weights for policy 1, policy_version 1460967 (0.0007) [2023-12-27 15:46:45,036][20347] Updated weights for policy 0, policy_version 1462365 (0.0008) [2023-12-27 15:46:45,096][20347] Updated weights for policy 0, policy_version 1462375 (0.0006) [2023-12-27 15:46:45,155][20347] Updated weights for policy 0, policy_version 1462385 (0.0007) [2023-12-27 15:46:45,692][20381] Updated weights for policy 1, policy_version 1460977 (0.0006) [2023-12-27 15:46:45,758][20381] Updated weights for policy 1, policy_version 1460987 (0.0008) [2023-12-27 15:46:45,813][20381] Updated weights for policy 1, policy_version 1460997 (0.0008) [2023-12-27 15:46:45,818][20347] Updated weights for policy 0, policy_version 1462395 (0.0006) [2023-12-27 15:46:45,866][20381] Updated weights for policy 1, policy_version 1461007 (0.0006) [2023-12-27 15:46:45,875][20347] Updated weights for policy 0, policy_version 1462405 (0.0007) [2023-12-27 15:46:45,938][20347] Updated weights for policy 0, policy_version 1462415 (0.0008) [2023-12-27 15:46:46,506][20381] Updated weights for policy 1, policy_version 1461017 (0.0008) [2023-12-27 15:46:46,571][20381] Updated weights for policy 1, policy_version 1461027 (0.0011) [2023-12-27 15:46:46,629][20347] Updated weights for policy 0, policy_version 1462425 (0.0008) [2023-12-27 15:46:46,630][20381] Updated weights for policy 1, policy_version 1461037 (0.0011) [2023-12-27 15:46:46,679][20347] Updated weights for policy 0, policy_version 1462435 (0.0008) [2023-12-27 15:46:46,737][20347] Updated weights for policy 0, policy_version 1462445 (0.0007) [2023-12-27 15:46:46,792][20347] Updated weights for policy 0, policy_version 1462455 (0.0008) [2023-12-27 15:46:47,279][20381] Updated weights for policy 1, policy_version 1461047 (0.0009) [2023-12-27 15:46:47,336][20381] Updated weights for policy 1, policy_version 1461057 (0.0006) [2023-12-27 15:46:47,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 748519424. Throughput: 0: 10196.3, 1: 10251.0. Samples: 748535332. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:47,363][19170] Avg episode reward: [(0, '706.350'), (1, '902.200')] [2023-12-27 15:46:47,389][20381] Updated weights for policy 1, policy_version 1461067 (0.0008) [2023-12-27 15:46:47,495][20347] Updated weights for policy 0, policy_version 1462465 (0.0007) [2023-12-27 15:46:47,541][20347] Updated weights for policy 0, policy_version 1462475 (0.0007) [2023-12-27 15:46:47,587][20347] Updated weights for policy 0, policy_version 1462485 (0.0006) [2023-12-27 15:46:48,077][20381] Updated weights for policy 1, policy_version 1461077 (0.0008) [2023-12-27 15:46:48,130][20381] Updated weights for policy 1, policy_version 1461087 (0.0008) [2023-12-27 15:46:48,179][20381] Updated weights for policy 1, policy_version 1461097 (0.0007) [2023-12-27 15:46:48,232][20347] Updated weights for policy 0, policy_version 1462495 (0.0006) [2023-12-27 15:46:48,284][20347] Updated weights for policy 0, policy_version 1462505 (0.0009) [2023-12-27 15:46:48,350][20347] Updated weights for policy 0, policy_version 1462515 (0.0009) [2023-12-27 15:46:48,865][20381] Updated weights for policy 1, policy_version 1461107 (0.0008) [2023-12-27 15:46:48,920][20381] Updated weights for policy 1, policy_version 1461117 (0.0011) [2023-12-27 15:46:48,978][20381] Updated weights for policy 1, policy_version 1461127 (0.0006) [2023-12-27 15:46:49,081][20347] Updated weights for policy 0, policy_version 1462525 (0.0008) [2023-12-27 15:46:49,131][20347] Updated weights for policy 0, policy_version 1462535 (0.0009) [2023-12-27 15:46:49,190][20347] Updated weights for policy 0, policy_version 1462545 (0.0005) [2023-12-27 15:46:49,600][20381] Updated weights for policy 1, policy_version 1461137 (0.0006) [2023-12-27 15:46:49,664][20381] Updated weights for policy 1, policy_version 1461147 (0.0006) [2023-12-27 15:46:49,724][20381] Updated weights for policy 1, policy_version 1461157 (0.0009) [2023-12-27 15:46:49,783][20381] Updated weights for policy 1, policy_version 1461167 (0.0007) [2023-12-27 15:46:49,839][20347] Updated weights for policy 0, policy_version 1462555 (0.0008) [2023-12-27 15:46:49,900][20347] Updated weights for policy 0, policy_version 1462565 (0.0009) [2023-12-27 15:46:49,966][20347] Updated weights for policy 0, policy_version 1462575 (0.0008) [2023-12-27 15:46:50,433][20381] Updated weights for policy 1, policy_version 1461177 (0.0007) [2023-12-27 15:46:50,481][20381] Updated weights for policy 1, policy_version 1461187 (0.0008) [2023-12-27 15:46:50,539][20381] Updated weights for policy 1, policy_version 1461197 (0.0009) [2023-12-27 15:46:50,646][20347] Updated weights for policy 0, policy_version 1462585 (0.0008) [2023-12-27 15:46:50,693][20347] Updated weights for policy 0, policy_version 1462595 (0.0007) [2023-12-27 15:46:50,743][20347] Updated weights for policy 0, policy_version 1462605 (0.0008) [2023-12-27 15:46:50,803][20347] Updated weights for policy 0, policy_version 1462615 (0.0008) [2023-12-27 15:46:51,226][20381] Updated weights for policy 1, policy_version 1461207 (0.0007) [2023-12-27 15:46:51,284][20381] Updated weights for policy 1, policy_version 1461217 (0.0008) [2023-12-27 15:46:51,340][20381] Updated weights for policy 1, policy_version 1461227 (0.0008) [2023-12-27 15:46:51,619][20347] Updated weights for policy 0, policy_version 1462625 (0.0008) [2023-12-27 15:46:51,682][20347] Updated weights for policy 0, policy_version 1462635 (0.0008) [2023-12-27 15:46:51,747][20347] Updated weights for policy 0, policy_version 1462645 (0.0008) [2023-12-27 15:46:52,067][20381] Updated weights for policy 1, policy_version 1461237 (0.0007) [2023-12-27 15:46:52,125][20381] Updated weights for policy 1, policy_version 1461247 (0.0008) [2023-12-27 15:46:52,183][20381] Updated weights for policy 1, policy_version 1461257 (0.0007) [2023-12-27 15:46:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 748625920. Throughput: 0: 10213.1, 1: 10264.0. Samples: 748597572. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:52,363][19170] Avg episode reward: [(0, '764.890'), (1, '892.590')] [2023-12-27 15:46:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001461264_374112256.pth... [2023-12-27 15:46:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001462648_374513664.pth... [2023-12-27 15:46:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001460048_373800960.pth [2023-12-27 15:46:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001461464_374210560.pth [2023-12-27 15:46:52,493][20347] Updated weights for policy 0, policy_version 1462655 (0.0008) [2023-12-27 15:46:52,552][20347] Updated weights for policy 0, policy_version 1462665 (0.0008) [2023-12-27 15:46:52,606][20347] Updated weights for policy 0, policy_version 1462675 (0.0007) [2023-12-27 15:46:52,893][20381] Updated weights for policy 1, policy_version 1461267 (0.0008) [2023-12-27 15:46:52,949][20381] Updated weights for policy 1, policy_version 1461277 (0.0011) [2023-12-27 15:46:53,003][20381] Updated weights for policy 1, policy_version 1461287 (0.0011) [2023-12-27 15:46:53,329][20347] Updated weights for policy 0, policy_version 1462685 (0.0008) [2023-12-27 15:46:53,389][20347] Updated weights for policy 0, policy_version 1462695 (0.0007) [2023-12-27 15:46:53,440][20347] Updated weights for policy 0, policy_version 1462705 (0.0007) [2023-12-27 15:46:53,648][20381] Updated weights for policy 1, policy_version 1461297 (0.0011) [2023-12-27 15:46:53,696][20381] Updated weights for policy 1, policy_version 1461307 (0.0010) [2023-12-27 15:46:53,754][20381] Updated weights for policy 1, policy_version 1461317 (0.0010) [2023-12-27 15:46:53,808][20381] Updated weights for policy 1, policy_version 1461327 (0.0010) [2023-12-27 15:46:54,156][20347] Updated weights for policy 0, policy_version 1462715 (0.0007) [2023-12-27 15:46:54,217][20347] Updated weights for policy 0, policy_version 1462725 (0.0007) [2023-12-27 15:46:54,278][20347] Updated weights for policy 0, policy_version 1462735 (0.0007) [2023-12-27 15:46:54,464][20381] Updated weights for policy 1, policy_version 1461337 (0.0006) [2023-12-27 15:46:54,509][20381] Updated weights for policy 1, policy_version 1461347 (0.0010) [2023-12-27 15:46:54,561][20381] Updated weights for policy 1, policy_version 1461357 (0.0010) [2023-12-27 15:46:54,892][20347] Updated weights for policy 0, policy_version 1462745 (0.0007) [2023-12-27 15:46:54,944][20347] Updated weights for policy 0, policy_version 1462755 (0.0005) [2023-12-27 15:46:54,995][20347] Updated weights for policy 0, policy_version 1462765 (0.0006) [2023-12-27 15:46:55,063][20347] Updated weights for policy 0, policy_version 1462775 (0.0008) [2023-12-27 15:46:55,183][20381] Updated weights for policy 1, policy_version 1461367 (0.0007) [2023-12-27 15:46:55,240][20381] Updated weights for policy 1, policy_version 1461377 (0.0008) [2023-12-27 15:46:55,293][20381] Updated weights for policy 1, policy_version 1461387 (0.0008) [2023-12-27 15:46:55,713][20347] Updated weights for policy 0, policy_version 1462785 (0.0008) [2023-12-27 15:46:55,766][20347] Updated weights for policy 0, policy_version 1462795 (0.0009) [2023-12-27 15:46:55,814][20347] Updated weights for policy 0, policy_version 1462805 (0.0009) [2023-12-27 15:46:55,919][20381] Updated weights for policy 1, policy_version 1461397 (0.0006) [2023-12-27 15:46:55,973][20381] Updated weights for policy 1, policy_version 1461407 (0.0007) [2023-12-27 15:46:56,022][20381] Updated weights for policy 1, policy_version 1461417 (0.0008) [2023-12-27 15:46:56,531][20347] Updated weights for policy 0, policy_version 1462815 (0.0008) [2023-12-27 15:46:56,588][20347] Updated weights for policy 0, policy_version 1462825 (0.0007) [2023-12-27 15:46:56,637][20347] Updated weights for policy 0, policy_version 1462835 (0.0005) [2023-12-27 15:46:56,663][20381] Updated weights for policy 1, policy_version 1461427 (0.0007) [2023-12-27 15:46:56,711][20381] Updated weights for policy 1, policy_version 1461437 (0.0005) [2023-12-27 15:46:56,759][20381] Updated weights for policy 1, policy_version 1461447 (0.0010) [2023-12-27 15:46:57,184][20347] Updated weights for policy 0, policy_version 1462845 (0.0006) [2023-12-27 15:46:57,235][20347] Updated weights for policy 0, policy_version 1462855 (0.0005) [2023-12-27 15:46:57,294][20347] Updated weights for policy 0, policy_version 1462865 (0.0005) [2023-12-27 15:46:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 748732416. Throughput: 0: 10190.2, 1: 10280.3. Samples: 748719132. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:46:57,363][19170] Avg episode reward: [(0, '794.240'), (1, '912.380')] [2023-12-27 15:46:57,501][20381] Updated weights for policy 1, policy_version 1461457 (0.0009) [2023-12-27 15:46:57,548][20381] Updated weights for policy 1, policy_version 1461467 (0.0008) [2023-12-27 15:46:57,590][20381] Updated weights for policy 1, policy_version 1461477 (0.0008) [2023-12-27 15:46:57,638][20381] Updated weights for policy 1, policy_version 1461487 (0.0005) [2023-12-27 15:46:57,925][20347] Updated weights for policy 0, policy_version 1462875 (0.0008) [2023-12-27 15:46:57,973][20347] Updated weights for policy 0, policy_version 1462885 (0.0007) [2023-12-27 15:46:58,016][20347] Updated weights for policy 0, policy_version 1462895 (0.0007) [2023-12-27 15:46:58,253][20381] Updated weights for policy 1, policy_version 1461497 (0.0008) [2023-12-27 15:46:58,312][20381] Updated weights for policy 1, policy_version 1461507 (0.0008) [2023-12-27 15:46:58,395][20381] Updated weights for policy 1, policy_version 1461517 (0.0007) [2023-12-27 15:46:58,771][20347] Updated weights for policy 0, policy_version 1462905 (0.0007) [2023-12-27 15:46:58,840][20347] Updated weights for policy 0, policy_version 1462915 (0.0007) [2023-12-27 15:46:58,895][20347] Updated weights for policy 0, policy_version 1462925 (0.0007) [2023-12-27 15:46:58,956][20347] Updated weights for policy 0, policy_version 1462935 (0.0008) [2023-12-27 15:46:59,182][20381] Updated weights for policy 1, policy_version 1461527 (0.0007) [2023-12-27 15:46:59,244][20381] Updated weights for policy 1, policy_version 1461537 (0.0008) [2023-12-27 15:46:59,302][20381] Updated weights for policy 1, policy_version 1461547 (0.0008) [2023-12-27 15:46:59,743][20347] Updated weights for policy 0, policy_version 1462945 (0.0007) [2023-12-27 15:46:59,799][20347] Updated weights for policy 0, policy_version 1462955 (0.0007) [2023-12-27 15:46:59,864][20347] Updated weights for policy 0, policy_version 1462965 (0.0008) [2023-12-27 15:46:59,952][20381] Updated weights for policy 1, policy_version 1461557 (0.0008) [2023-12-27 15:47:00,004][20381] Updated weights for policy 1, policy_version 1461567 (0.0006) [2023-12-27 15:47:00,059][20381] Updated weights for policy 1, policy_version 1461577 (0.0007) [2023-12-27 15:47:00,542][20347] Updated weights for policy 0, policy_version 1462975 (0.0008) [2023-12-27 15:47:00,588][20347] Updated weights for policy 0, policy_version 1462985 (0.0008) [2023-12-27 15:47:00,634][20347] Updated weights for policy 0, policy_version 1462995 (0.0007) [2023-12-27 15:47:00,755][20381] Updated weights for policy 1, policy_version 1461587 (0.0008) [2023-12-27 15:47:00,808][20381] Updated weights for policy 1, policy_version 1461597 (0.0007) [2023-12-27 15:47:00,865][20381] Updated weights for policy 1, policy_version 1461607 (0.0007) [2023-12-27 15:47:01,391][20347] Updated weights for policy 0, policy_version 1463005 (0.0008) [2023-12-27 15:47:01,444][20347] Updated weights for policy 0, policy_version 1463015 (0.0007) [2023-12-27 15:47:01,491][20347] Updated weights for policy 0, policy_version 1463025 (0.0007) [2023-12-27 15:47:01,581][20381] Updated weights for policy 1, policy_version 1461617 (0.0007) [2023-12-27 15:47:01,636][20381] Updated weights for policy 1, policy_version 1461627 (0.0007) [2023-12-27 15:47:01,707][20381] Updated weights for policy 1, policy_version 1461637 (0.0008) [2023-12-27 15:47:01,762][20381] Updated weights for policy 1, policy_version 1461647 (0.0008) [2023-12-27 15:47:02,156][20347] Updated weights for policy 0, policy_version 1463035 (0.0008) [2023-12-27 15:47:02,211][20347] Updated weights for policy 0, policy_version 1463045 (0.0010) [2023-12-27 15:47:02,268][20347] Updated weights for policy 0, policy_version 1463055 (0.0010) [2023-12-27 15:47:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 748830720. Throughput: 0: 10202.1, 1: 10208.9. Samples: 748840260. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:02,363][19170] Avg episode reward: [(0, '784.230'), (1, '941.780')] [2023-12-27 15:47:02,483][20381] Updated weights for policy 1, policy_version 1461657 (0.0007) [2023-12-27 15:47:02,532][20381] Updated weights for policy 1, policy_version 1461667 (0.0008) [2023-12-27 15:47:02,583][20381] Updated weights for policy 1, policy_version 1461677 (0.0007) [2023-12-27 15:47:02,956][20347] Updated weights for policy 0, policy_version 1463065 (0.0010) [2023-12-27 15:47:03,014][20347] Updated weights for policy 0, policy_version 1463075 (0.0010) [2023-12-27 15:47:03,071][20347] Updated weights for policy 0, policy_version 1463085 (0.0010) [2023-12-27 15:47:03,125][20347] Updated weights for policy 0, policy_version 1463095 (0.0010) [2023-12-27 15:47:03,307][20381] Updated weights for policy 1, policy_version 1461687 (0.0009) [2023-12-27 15:47:03,353][20381] Updated weights for policy 1, policy_version 1461697 (0.0008) [2023-12-27 15:47:03,407][20381] Updated weights for policy 1, policy_version 1461707 (0.0010) [2023-12-27 15:47:03,717][20347] Updated weights for policy 0, policy_version 1463105 (0.0006) [2023-12-27 15:47:03,780][20347] Updated weights for policy 0, policy_version 1463115 (0.0005) [2023-12-27 15:47:03,835][20347] Updated weights for policy 0, policy_version 1463125 (0.0005) [2023-12-27 15:47:03,991][20381] Updated weights for policy 1, policy_version 1461717 (0.0007) [2023-12-27 15:47:04,054][20381] Updated weights for policy 1, policy_version 1461727 (0.0008) [2023-12-27 15:47:04,116][20381] Updated weights for policy 1, policy_version 1461737 (0.0007) [2023-12-27 15:47:04,465][20347] Updated weights for policy 0, policy_version 1463135 (0.0007) [2023-12-27 15:47:04,522][20347] Updated weights for policy 0, policy_version 1463145 (0.0008) [2023-12-27 15:47:04,576][20347] Updated weights for policy 0, policy_version 1463155 (0.0007) [2023-12-27 15:47:04,805][20381] Updated weights for policy 1, policy_version 1461747 (0.0006) [2023-12-27 15:47:04,848][20381] Updated weights for policy 1, policy_version 1461757 (0.0005) [2023-12-27 15:47:04,907][20381] Updated weights for policy 1, policy_version 1461767 (0.0005) [2023-12-27 15:47:05,232][20347] Updated weights for policy 0, policy_version 1463165 (0.0006) [2023-12-27 15:47:05,291][20347] Updated weights for policy 0, policy_version 1463175 (0.0010) [2023-12-27 15:47:05,352][20347] Updated weights for policy 0, policy_version 1463185 (0.0009) [2023-12-27 15:47:05,502][20381] Updated weights for policy 1, policy_version 1461777 (0.0005) [2023-12-27 15:47:05,555][20381] Updated weights for policy 1, policy_version 1461787 (0.0009) [2023-12-27 15:47:05,611][20381] Updated weights for policy 1, policy_version 1461797 (0.0008) [2023-12-27 15:47:05,655][20381] Updated weights for policy 1, policy_version 1461807 (0.0008) [2023-12-27 15:47:05,912][20347] Updated weights for policy 0, policy_version 1463195 (0.0010) [2023-12-27 15:47:05,970][20347] Updated weights for policy 0, policy_version 1463205 (0.0010) [2023-12-27 15:47:06,024][20347] Updated weights for policy 0, policy_version 1463215 (0.0009) [2023-12-27 15:47:06,381][20381] Updated weights for policy 1, policy_version 1461817 (0.0007) [2023-12-27 15:47:06,441][20381] Updated weights for policy 1, policy_version 1461827 (0.0007) [2023-12-27 15:47:06,502][20381] Updated weights for policy 1, policy_version 1461837 (0.0008) [2023-12-27 15:47:06,731][20347] Updated weights for policy 0, policy_version 1463225 (0.0010) [2023-12-27 15:47:06,781][20347] Updated weights for policy 0, policy_version 1463235 (0.0011) [2023-12-27 15:47:06,841][20347] Updated weights for policy 0, policy_version 1463245 (0.0010) [2023-12-27 15:47:06,896][20347] Updated weights for policy 0, policy_version 1463255 (0.0011) [2023-12-27 15:47:07,232][20381] Updated weights for policy 1, policy_version 1461847 (0.0008) [2023-12-27 15:47:07,287][20381] Updated weights for policy 1, policy_version 1461857 (0.0007) [2023-12-27 15:47:07,338][20381] Updated weights for policy 1, policy_version 1461867 (0.0008) [2023-12-27 15:47:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 748929024. Throughput: 0: 10192.1, 1: 10233.6. Samples: 748903648. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:07,364][19170] Avg episode reward: [(0, '803.900'), (1, '941.640')] [2023-12-27 15:47:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001461872_374267904.pth... [2023-12-27 15:47:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001463256_374669312.pth... [2023-12-27 15:47:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001462072_374366208.pth [2023-12-27 15:47:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001460656_373956608.pth [2023-12-27 15:47:07,547][20347] Updated weights for policy 0, policy_version 1463265 (0.0011) [2023-12-27 15:47:07,595][20347] Updated weights for policy 0, policy_version 1463275 (0.0010) [2023-12-27 15:47:07,659][20347] Updated weights for policy 0, policy_version 1463285 (0.0008) [2023-12-27 15:47:07,997][20381] Updated weights for policy 1, policy_version 1461877 (0.0006) [2023-12-27 15:47:08,054][20381] Updated weights for policy 1, policy_version 1461887 (0.0005) [2023-12-27 15:47:08,102][20381] Updated weights for policy 1, policy_version 1461897 (0.0005) [2023-12-27 15:47:08,236][20347] Updated weights for policy 0, policy_version 1463295 (0.0009) [2023-12-27 15:47:08,285][20347] Updated weights for policy 0, policy_version 1463305 (0.0010) [2023-12-27 15:47:08,351][20347] Updated weights for policy 0, policy_version 1463315 (0.0009) [2023-12-27 15:47:08,782][20381] Updated weights for policy 1, policy_version 1461907 (0.0007) [2023-12-27 15:47:08,842][20381] Updated weights for policy 1, policy_version 1461917 (0.0007) [2023-12-27 15:47:08,903][20381] Updated weights for policy 1, policy_version 1461927 (0.0006) [2023-12-27 15:47:08,927][20347] Updated weights for policy 0, policy_version 1463325 (0.0007) [2023-12-27 15:47:08,979][20347] Updated weights for policy 0, policy_version 1463335 (0.0010) [2023-12-27 15:47:09,027][20347] Updated weights for policy 0, policy_version 1463345 (0.0010) [2023-12-27 15:47:09,556][20381] Updated weights for policy 1, policy_version 1461937 (0.0006) [2023-12-27 15:47:09,611][20381] Updated weights for policy 1, policy_version 1461947 (0.0007) [2023-12-27 15:47:09,667][20381] Updated weights for policy 1, policy_version 1461957 (0.0007) [2023-12-27 15:47:09,723][20381] Updated weights for policy 1, policy_version 1461967 (0.0008) [2023-12-27 15:47:09,746][20347] Updated weights for policy 0, policy_version 1463355 (0.0010) [2023-12-27 15:47:09,809][20347] Updated weights for policy 0, policy_version 1463365 (0.0011) [2023-12-27 15:47:09,882][20347] Updated weights for policy 0, policy_version 1463375 (0.0011) [2023-12-27 15:47:10,495][20381] Updated weights for policy 1, policy_version 1461977 (0.0007) [2023-12-27 15:47:10,548][20381] Updated weights for policy 1, policy_version 1461987 (0.0007) [2023-12-27 15:47:10,563][20347] Updated weights for policy 0, policy_version 1463385 (0.0009) [2023-12-27 15:47:10,592][20381] Updated weights for policy 1, policy_version 1461997 (0.0006) [2023-12-27 15:47:10,617][20347] Updated weights for policy 0, policy_version 1463395 (0.0006) [2023-12-27 15:47:10,666][20347] Updated weights for policy 0, policy_version 1463405 (0.0007) [2023-12-27 15:47:10,715][20347] Updated weights for policy 0, policy_version 1463415 (0.0008) [2023-12-27 15:47:11,305][20381] Updated weights for policy 1, policy_version 1462007 (0.0007) [2023-12-27 15:47:11,359][20381] Updated weights for policy 1, policy_version 1462017 (0.0008) [2023-12-27 15:47:11,431][20381] Updated weights for policy 1, policy_version 1462027 (0.0008) [2023-12-27 15:47:11,445][20347] Updated weights for policy 0, policy_version 1463425 (0.0007) [2023-12-27 15:47:11,505][20347] Updated weights for policy 0, policy_version 1463435 (0.0007) [2023-12-27 15:47:11,558][20347] Updated weights for policy 0, policy_version 1463445 (0.0008) [2023-12-27 15:47:12,168][20381] Updated weights for policy 1, policy_version 1462037 (0.0007) [2023-12-27 15:47:12,228][20381] Updated weights for policy 1, policy_version 1462047 (0.0008) [2023-12-27 15:47:12,250][20347] Updated weights for policy 0, policy_version 1463455 (0.0007) [2023-12-27 15:47:12,292][20381] Updated weights for policy 1, policy_version 1462057 (0.0008) [2023-12-27 15:47:12,312][20347] Updated weights for policy 0, policy_version 1463465 (0.0006) [2023-12-27 15:47:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 749035520. Throughput: 0: 10249.0, 1: 10204.9. Samples: 749027492. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:12,363][19170] Avg episode reward: [(0, '745.160'), (1, '931.600')] [2023-12-27 15:47:12,373][20347] Updated weights for policy 0, policy_version 1463475 (0.0007) [2023-12-27 15:47:13,010][20347] Updated weights for policy 0, policy_version 1463485 (0.0006) [2023-12-27 15:47:13,036][20381] Updated weights for policy 1, policy_version 1462067 (0.0008) [2023-12-27 15:47:13,063][20347] Updated weights for policy 0, policy_version 1463495 (0.0005) [2023-12-27 15:47:13,095][20381] Updated weights for policy 1, policy_version 1462077 (0.0008) [2023-12-27 15:47:13,116][20347] Updated weights for policy 0, policy_version 1463505 (0.0005) [2023-12-27 15:47:13,153][20381] Updated weights for policy 1, policy_version 1462087 (0.0007) [2023-12-27 15:47:13,772][20347] Updated weights for policy 0, policy_version 1463515 (0.0006) [2023-12-27 15:47:13,828][20347] Updated weights for policy 0, policy_version 1463525 (0.0008) [2023-12-27 15:47:13,840][20381] Updated weights for policy 1, policy_version 1462097 (0.0008) [2023-12-27 15:47:13,881][20347] Updated weights for policy 0, policy_version 1463535 (0.0007) [2023-12-27 15:47:13,901][20381] Updated weights for policy 1, policy_version 1462107 (0.0007) [2023-12-27 15:47:13,954][20381] Updated weights for policy 1, policy_version 1462117 (0.0005) [2023-12-27 15:47:14,010][20381] Updated weights for policy 1, policy_version 1462127 (0.0007) [2023-12-27 15:47:14,595][20347] Updated weights for policy 0, policy_version 1463545 (0.0007) [2023-12-27 15:47:14,607][20381] Updated weights for policy 1, policy_version 1462137 (0.0007) [2023-12-27 15:47:14,650][20347] Updated weights for policy 0, policy_version 1463555 (0.0005) [2023-12-27 15:47:14,660][20381] Updated weights for policy 1, policy_version 1462147 (0.0008) [2023-12-27 15:47:14,707][20347] Updated weights for policy 0, policy_version 1463565 (0.0006) [2023-12-27 15:47:14,709][20381] Updated weights for policy 1, policy_version 1462157 (0.0006) [2023-12-27 15:47:14,753][20347] Updated weights for policy 0, policy_version 1463575 (0.0006) [2023-12-27 15:47:15,437][20381] Updated weights for policy 1, policy_version 1462167 (0.0006) [2023-12-27 15:47:15,443][20347] Updated weights for policy 0, policy_version 1463585 (0.0010) [2023-12-27 15:47:15,488][20381] Updated weights for policy 1, policy_version 1462177 (0.0005) [2023-12-27 15:47:15,494][20347] Updated weights for policy 0, policy_version 1463595 (0.0009) [2023-12-27 15:47:15,544][20381] Updated weights for policy 1, policy_version 1462187 (0.0006) [2023-12-27 15:47:15,546][20347] Updated weights for policy 0, policy_version 1463605 (0.0005) [2023-12-27 15:47:16,231][20347] Updated weights for policy 0, policy_version 1463615 (0.0007) [2023-12-27 15:47:16,241][20381] Updated weights for policy 1, policy_version 1462197 (0.0008) [2023-12-27 15:47:16,283][20347] Updated weights for policy 0, policy_version 1463625 (0.0006) [2023-12-27 15:47:16,302][20381] Updated weights for policy 1, policy_version 1462207 (0.0011) [2023-12-27 15:47:16,338][20347] Updated weights for policy 0, policy_version 1463635 (0.0006) [2023-12-27 15:47:16,351][20381] Updated weights for policy 1, policy_version 1462217 (0.0010) [2023-12-27 15:47:16,995][20347] Updated weights for policy 0, policy_version 1463645 (0.0006) [2023-12-27 15:47:17,014][20381] Updated weights for policy 1, policy_version 1462227 (0.0009) [2023-12-27 15:47:17,053][20347] Updated weights for policy 0, policy_version 1463655 (0.0006) [2023-12-27 15:47:17,075][20381] Updated weights for policy 1, policy_version 1462237 (0.0006) [2023-12-27 15:47:17,105][20347] Updated weights for policy 0, policy_version 1463665 (0.0008) [2023-12-27 15:47:17,135][20381] Updated weights for policy 1, policy_version 1462247 (0.0006) [2023-12-27 15:47:17,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 749142016. Throughput: 0: 10275.5, 1: 10152.6. Samples: 749149116. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:17,363][19170] Avg episode reward: [(0, '735.410'), (1, '872.900')] [2023-12-27 15:47:17,711][20381] Updated weights for policy 1, policy_version 1462257 (0.0006) [2023-12-27 15:47:17,772][20381] Updated weights for policy 1, policy_version 1462267 (0.0008) [2023-12-27 15:47:17,778][20347] Updated weights for policy 0, policy_version 1463675 (0.0008) [2023-12-27 15:47:17,821][20347] Updated weights for policy 0, policy_version 1463685 (0.0008) [2023-12-27 15:47:17,827][20381] Updated weights for policy 1, policy_version 1462277 (0.0007) [2023-12-27 15:47:17,870][20347] Updated weights for policy 0, policy_version 1463695 (0.0006) [2023-12-27 15:47:17,882][20381] Updated weights for policy 1, policy_version 1462287 (0.0006) [2023-12-27 15:47:18,514][20381] Updated weights for policy 1, policy_version 1462297 (0.0005) [2023-12-27 15:47:18,581][20347] Updated weights for policy 0, policy_version 1463705 (0.0007) [2023-12-27 15:47:18,582][20381] Updated weights for policy 1, policy_version 1462307 (0.0005) [2023-12-27 15:47:18,640][20347] Updated weights for policy 0, policy_version 1463715 (0.0006) [2023-12-27 15:47:18,648][20381] Updated weights for policy 1, policy_version 1462317 (0.0006) [2023-12-27 15:47:18,700][20347] Updated weights for policy 0, policy_version 1463725 (0.0007) [2023-12-27 15:47:18,756][20347] Updated weights for policy 0, policy_version 1463735 (0.0007) [2023-12-27 15:47:19,274][20381] Updated weights for policy 1, policy_version 1462327 (0.0007) [2023-12-27 15:47:19,328][20381] Updated weights for policy 1, policy_version 1462337 (0.0007) [2023-12-27 15:47:19,362][20347] Updated weights for policy 0, policy_version 1463745 (0.0008) [2023-12-27 15:47:19,393][20381] Updated weights for policy 1, policy_version 1462347 (0.0007) [2023-12-27 15:47:19,423][20347] Updated weights for policy 0, policy_version 1463755 (0.0007) [2023-12-27 15:47:19,485][20347] Updated weights for policy 0, policy_version 1463765 (0.0009) [2023-12-27 15:47:20,110][20381] Updated weights for policy 1, policy_version 1462357 (0.0008) [2023-12-27 15:47:20,167][20381] Updated weights for policy 1, policy_version 1462367 (0.0009) [2023-12-27 15:47:20,221][20347] Updated weights for policy 0, policy_version 1463775 (0.0008) [2023-12-27 15:47:20,228][20381] Updated weights for policy 1, policy_version 1462377 (0.0009) [2023-12-27 15:47:20,280][20347] Updated weights for policy 0, policy_version 1463785 (0.0007) [2023-12-27 15:47:20,333][20347] Updated weights for policy 0, policy_version 1463795 (0.0008) [2023-12-27 15:47:20,966][20381] Updated weights for policy 1, policy_version 1462387 (0.0011) [2023-12-27 15:47:21,015][20347] Updated weights for policy 0, policy_version 1463805 (0.0009) [2023-12-27 15:47:21,023][20381] Updated weights for policy 1, policy_version 1462397 (0.0011) [2023-12-27 15:47:21,076][20381] Updated weights for policy 1, policy_version 1462407 (0.0009) [2023-12-27 15:47:21,084][20347] Updated weights for policy 0, policy_version 1463815 (0.0010) [2023-12-27 15:47:21,144][20347] Updated weights for policy 0, policy_version 1463825 (0.0009) [2023-12-27 15:47:21,761][20381] Updated weights for policy 1, policy_version 1462417 (0.0007) [2023-12-27 15:47:21,818][20381] Updated weights for policy 1, policy_version 1462427 (0.0008) [2023-12-27 15:47:21,874][20381] Updated weights for policy 1, policy_version 1462437 (0.0008) [2023-12-27 15:47:21,916][20347] Updated weights for policy 0, policy_version 1463835 (0.0008) [2023-12-27 15:47:21,933][20381] Updated weights for policy 1, policy_version 1462447 (0.0008) [2023-12-27 15:47:21,975][20347] Updated weights for policy 0, policy_version 1463845 (0.0006) [2023-12-27 15:47:22,024][20347] Updated weights for policy 0, policy_version 1463855 (0.0007) [2023-12-27 15:47:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 749240320. Throughput: 0: 10249.7, 1: 10184.5. Samples: 749211356. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:22,363][19170] Avg episode reward: [(0, '647.030'), (1, '892.560')] [2023-12-27 15:47:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001462448_374415360.pth... [2023-12-27 15:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001463864_374824960.pth... [2023-12-27 15:47:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001461264_374112256.pth [2023-12-27 15:47:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001462648_374513664.pth [2023-12-27 15:47:22,650][20347] Updated weights for policy 0, policy_version 1463865 (0.0006) [2023-12-27 15:47:22,668][20381] Updated weights for policy 1, policy_version 1462457 (0.0008) [2023-12-27 15:47:22,705][20347] Updated weights for policy 0, policy_version 1463875 (0.0006) [2023-12-27 15:47:22,730][20381] Updated weights for policy 1, policy_version 1462467 (0.0008) [2023-12-27 15:47:22,764][20347] Updated weights for policy 0, policy_version 1463885 (0.0006) [2023-12-27 15:47:22,782][20381] Updated weights for policy 1, policy_version 1462477 (0.0006) [2023-12-27 15:47:22,815][20347] Updated weights for policy 0, policy_version 1463895 (0.0006) [2023-12-27 15:47:23,359][20381] Updated weights for policy 1, policy_version 1462487 (0.0007) [2023-12-27 15:47:23,407][20381] Updated weights for policy 1, policy_version 1462497 (0.0007) [2023-12-27 15:47:23,455][20381] Updated weights for policy 1, policy_version 1462507 (0.0007) [2023-12-27 15:47:23,458][20347] Updated weights for policy 0, policy_version 1463905 (0.0005) [2023-12-27 15:47:23,512][20347] Updated weights for policy 0, policy_version 1463915 (0.0005) [2023-12-27 15:47:23,561][20347] Updated weights for policy 0, policy_version 1463925 (0.0005) [2023-12-27 15:47:24,092][20347] Updated weights for policy 0, policy_version 1463935 (0.0007) [2023-12-27 15:47:24,146][20347] Updated weights for policy 0, policy_version 1463945 (0.0006) [2023-12-27 15:47:24,193][20381] Updated weights for policy 1, policy_version 1462517 (0.0007) [2023-12-27 15:47:24,202][20347] Updated weights for policy 0, policy_version 1463955 (0.0008) [2023-12-27 15:47:24,249][20381] Updated weights for policy 1, policy_version 1462527 (0.0006) [2023-12-27 15:47:24,309][20381] Updated weights for policy 1, policy_version 1462537 (0.0006) [2023-12-27 15:47:24,871][20347] Updated weights for policy 0, policy_version 1463965 (0.0007) [2023-12-27 15:47:24,898][20381] Updated weights for policy 1, policy_version 1462547 (0.0009) [2023-12-27 15:47:24,924][20347] Updated weights for policy 0, policy_version 1463975 (0.0005) [2023-12-27 15:47:24,953][20381] Updated weights for policy 1, policy_version 1462557 (0.0010) [2023-12-27 15:47:24,981][20347] Updated weights for policy 0, policy_version 1463985 (0.0005) [2023-12-27 15:47:25,006][20381] Updated weights for policy 1, policy_version 1462567 (0.0007) [2023-12-27 15:47:25,526][20347] Updated weights for policy 0, policy_version 1463995 (0.0007) [2023-12-27 15:47:25,578][20347] Updated weights for policy 0, policy_version 1464005 (0.0008) [2023-12-27 15:47:25,629][20347] Updated weights for policy 0, policy_version 1464015 (0.0007) [2023-12-27 15:47:25,661][20381] Updated weights for policy 1, policy_version 1462577 (0.0007) [2023-12-27 15:47:25,719][20381] Updated weights for policy 1, policy_version 1462587 (0.0010) [2023-12-27 15:47:25,767][20381] Updated weights for policy 1, policy_version 1462597 (0.0010) [2023-12-27 15:47:25,815][20381] Updated weights for policy 1, policy_version 1462607 (0.0010) [2023-12-27 15:47:26,224][20347] Updated weights for policy 0, policy_version 1464025 (0.0006) [2023-12-27 15:47:26,275][20347] Updated weights for policy 0, policy_version 1464035 (0.0005) [2023-12-27 15:47:26,323][20347] Updated weights for policy 0, policy_version 1464045 (0.0005) [2023-12-27 15:47:26,374][20347] Updated weights for policy 0, policy_version 1464055 (0.0005) [2023-12-27 15:47:26,499][20381] Updated weights for policy 1, policy_version 1462617 (0.0010) [2023-12-27 15:47:26,546][20381] Updated weights for policy 1, policy_version 1462627 (0.0010) [2023-12-27 15:47:26,594][20381] Updated weights for policy 1, policy_version 1462637 (0.0010) [2023-12-27 15:47:26,926][20347] Updated weights for policy 0, policy_version 1464065 (0.0010) [2023-12-27 15:47:26,973][20347] Updated weights for policy 0, policy_version 1464075 (0.0008) [2023-12-27 15:47:27,020][20347] Updated weights for policy 0, policy_version 1464085 (0.0007) [2023-12-27 15:47:27,245][20381] Updated weights for policy 1, policy_version 1462647 (0.0010) [2023-12-27 15:47:27,297][20381] Updated weights for policy 1, policy_version 1462657 (0.0010) [2023-12-27 15:47:27,344][20381] Updated weights for policy 1, policy_version 1462667 (0.0008) [2023-12-27 15:47:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 749346816. Throughput: 0: 10313.5, 1: 10215.8. Samples: 749337792. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:27,363][19170] Avg episode reward: [(0, '715.790'), (1, '863.020')] [2023-12-27 15:47:27,619][20347] Updated weights for policy 0, policy_version 1464095 (0.0008) [2023-12-27 15:47:27,674][20347] Updated weights for policy 0, policy_version 1464105 (0.0007) [2023-12-27 15:47:27,720][20347] Updated weights for policy 0, policy_version 1464115 (0.0007) [2023-12-27 15:47:27,985][20381] Updated weights for policy 1, policy_version 1462677 (0.0008) [2023-12-27 15:47:28,037][20381] Updated weights for policy 1, policy_version 1462687 (0.0010) [2023-12-27 15:47:28,091][20381] Updated weights for policy 1, policy_version 1462697 (0.0010) [2023-12-27 15:47:28,429][20347] Updated weights for policy 0, policy_version 1464125 (0.0007) [2023-12-27 15:47:28,485][20347] Updated weights for policy 0, policy_version 1464135 (0.0007) [2023-12-27 15:47:28,541][20347] Updated weights for policy 0, policy_version 1464145 (0.0008) [2023-12-27 15:47:28,743][20381] Updated weights for policy 1, policy_version 1462707 (0.0009) [2023-12-27 15:47:28,802][20381] Updated weights for policy 1, policy_version 1462717 (0.0005) [2023-12-27 15:47:28,856][20381] Updated weights for policy 1, policy_version 1462727 (0.0005) [2023-12-27 15:47:29,201][20347] Updated weights for policy 0, policy_version 1464155 (0.0007) [2023-12-27 15:47:29,266][20347] Updated weights for policy 0, policy_version 1464165 (0.0007) [2023-12-27 15:47:29,315][20347] Updated weights for policy 0, policy_version 1464175 (0.0007) [2023-12-27 15:47:29,444][20381] Updated weights for policy 1, policy_version 1462737 (0.0006) [2023-12-27 15:47:29,498][20381] Updated weights for policy 1, policy_version 1462747 (0.0005) [2023-12-27 15:47:29,555][20381] Updated weights for policy 1, policy_version 1462757 (0.0007) [2023-12-27 15:47:29,606][20381] Updated weights for policy 1, policy_version 1462767 (0.0007) [2023-12-27 15:47:30,042][20347] Updated weights for policy 0, policy_version 1464185 (0.0008) [2023-12-27 15:47:30,101][20347] Updated weights for policy 0, policy_version 1464195 (0.0008) [2023-12-27 15:47:30,172][20347] Updated weights for policy 0, policy_version 1464205 (0.0009) [2023-12-27 15:47:30,234][20347] Updated weights for policy 0, policy_version 1464215 (0.0007) [2023-12-27 15:47:30,252][20381] Updated weights for policy 1, policy_version 1462777 (0.0008) [2023-12-27 15:47:30,304][20381] Updated weights for policy 1, policy_version 1462787 (0.0008) [2023-12-27 15:47:30,354][20381] Updated weights for policy 1, policy_version 1462797 (0.0006) [2023-12-27 15:47:30,924][20347] Updated weights for policy 0, policy_version 1464225 (0.0005) [2023-12-27 15:47:30,987][20347] Updated weights for policy 0, policy_version 1464235 (0.0005) [2023-12-27 15:47:30,990][20381] Updated weights for policy 1, policy_version 1462807 (0.0006) [2023-12-27 15:47:31,054][20347] Updated weights for policy 0, policy_version 1464245 (0.0007) [2023-12-27 15:47:31,061][20381] Updated weights for policy 1, policy_version 1462817 (0.0007) [2023-12-27 15:47:31,122][20381] Updated weights for policy 1, policy_version 1462827 (0.0009) [2023-12-27 15:47:31,715][20347] Updated weights for policy 0, policy_version 1464255 (0.0008) [2023-12-27 15:47:31,776][20347] Updated weights for policy 0, policy_version 1464265 (0.0008) [2023-12-27 15:47:31,841][20347] Updated weights for policy 0, policy_version 1464275 (0.0008) [2023-12-27 15:47:31,877][20381] Updated weights for policy 1, policy_version 1462837 (0.0009) [2023-12-27 15:47:31,940][20381] Updated weights for policy 1, policy_version 1462847 (0.0007) [2023-12-27 15:47:31,995][20381] Updated weights for policy 1, policy_version 1462857 (0.0006) [2023-12-27 15:47:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 749453312. Throughput: 0: 10349.0, 1: 10274.7. Samples: 749463396. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:32,363][19170] Avg episode reward: [(0, '754.790'), (1, '882.520')] [2023-12-27 15:47:32,534][20347] Updated weights for policy 0, policy_version 1464285 (0.0007) [2023-12-27 15:47:32,591][20347] Updated weights for policy 0, policy_version 1464295 (0.0007) [2023-12-27 15:47:32,640][20347] Updated weights for policy 0, policy_version 1464305 (0.0007) [2023-12-27 15:47:32,705][20381] Updated weights for policy 1, policy_version 1462867 (0.0008) [2023-12-27 15:47:32,760][20381] Updated weights for policy 1, policy_version 1462877 (0.0007) [2023-12-27 15:47:32,815][20381] Updated weights for policy 1, policy_version 1462887 (0.0007) [2023-12-27 15:47:33,366][20381] Updated weights for policy 1, policy_version 1462897 (0.0006) [2023-12-27 15:47:33,368][20347] Updated weights for policy 0, policy_version 1464315 (0.0007) [2023-12-27 15:47:33,412][20347] Updated weights for policy 0, policy_version 1464325 (0.0007) [2023-12-27 15:47:33,415][20381] Updated weights for policy 1, policy_version 1462907 (0.0007) [2023-12-27 15:47:33,468][20381] Updated weights for policy 1, policy_version 1462917 (0.0009) [2023-12-27 15:47:33,473][20347] Updated weights for policy 0, policy_version 1464335 (0.0007) [2023-12-27 15:47:33,521][20381] Updated weights for policy 1, policy_version 1462927 (0.0010) [2023-12-27 15:47:34,089][20347] Updated weights for policy 0, policy_version 1464345 (0.0007) [2023-12-27 15:47:34,155][20347] Updated weights for policy 0, policy_version 1464355 (0.0007) [2023-12-27 15:47:34,175][20381] Updated weights for policy 1, policy_version 1462937 (0.0010) [2023-12-27 15:47:34,215][20347] Updated weights for policy 0, policy_version 1464365 (0.0007) [2023-12-27 15:47:34,231][20381] Updated weights for policy 1, policy_version 1462947 (0.0006) [2023-12-27 15:47:34,267][20347] Updated weights for policy 0, policy_version 1464375 (0.0007) [2023-12-27 15:47:34,280][20381] Updated weights for policy 1, policy_version 1462957 (0.0006) [2023-12-27 15:47:34,912][20347] Updated weights for policy 0, policy_version 1464385 (0.0005) [2023-12-27 15:47:34,944][20381] Updated weights for policy 1, policy_version 1462967 (0.0006) [2023-12-27 15:47:34,961][20347] Updated weights for policy 0, policy_version 1464395 (0.0009) [2023-12-27 15:47:35,004][20381] Updated weights for policy 1, policy_version 1462977 (0.0005) [2023-12-27 15:47:35,010][20347] Updated weights for policy 0, policy_version 1464405 (0.0009) [2023-12-27 15:47:35,055][20381] Updated weights for policy 1, policy_version 1462987 (0.0008) [2023-12-27 15:47:35,549][20347] Updated weights for policy 0, policy_version 1464415 (0.0005) [2023-12-27 15:47:35,602][20347] Updated weights for policy 0, policy_version 1464425 (0.0007) [2023-12-27 15:47:35,654][20347] Updated weights for policy 0, policy_version 1464435 (0.0010) [2023-12-27 15:47:35,754][20381] Updated weights for policy 1, policy_version 1462997 (0.0007) [2023-12-27 15:47:35,809][20381] Updated weights for policy 1, policy_version 1463007 (0.0006) [2023-12-27 15:47:35,862][20381] Updated weights for policy 1, policy_version 1463017 (0.0005) [2023-12-27 15:47:36,302][20347] Updated weights for policy 0, policy_version 1464445 (0.0010) [2023-12-27 15:47:36,367][20347] Updated weights for policy 0, policy_version 1464455 (0.0010) [2023-12-27 15:47:36,436][20347] Updated weights for policy 0, policy_version 1464465 (0.0011) [2023-12-27 15:47:36,527][20381] Updated weights for policy 1, policy_version 1463027 (0.0007) [2023-12-27 15:47:36,584][20381] Updated weights for policy 1, policy_version 1463037 (0.0011) [2023-12-27 15:47:36,642][20381] Updated weights for policy 1, policy_version 1463047 (0.0010) [2023-12-27 15:47:37,024][20347] Updated weights for policy 0, policy_version 1464475 (0.0008) [2023-12-27 15:47:37,083][20347] Updated weights for policy 0, policy_version 1464485 (0.0010) [2023-12-27 15:47:37,144][20347] Updated weights for policy 0, policy_version 1464495 (0.0011) [2023-12-27 15:47:37,339][20381] Updated weights for policy 1, policy_version 1463057 (0.0010) [2023-12-27 15:47:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 749559808. Throughput: 0: 10358.1, 1: 10300.5. Samples: 749527204. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:37,363][19170] Avg episode reward: [(0, '666.610'), (1, '921.790')] [2023-12-27 15:47:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001464504_374988800.pth... [2023-12-27 15:47:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001463256_374669312.pth [2023-12-27 15:47:37,398][20381] Updated weights for policy 1, policy_version 1463067 (0.0007) [2023-12-27 15:47:37,460][20381] Updated weights for policy 1, policy_version 1463077 (0.0009) [2023-12-27 15:47:37,520][20381] Updated weights for policy 1, policy_version 1463087 (0.0009) [2023-12-27 15:47:37,526][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001463088_374579200.pth... [2023-12-27 15:47:37,530][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001461872_374267904.pth [2023-12-27 15:47:37,726][20347] Updated weights for policy 0, policy_version 1464505 (0.0007) [2023-12-27 15:47:37,789][20347] Updated weights for policy 0, policy_version 1464515 (0.0005) [2023-12-27 15:47:37,836][20347] Updated weights for policy 0, policy_version 1464525 (0.0005) [2023-12-27 15:47:37,880][20347] Updated weights for policy 0, policy_version 1464535 (0.0005) [2023-12-27 15:47:38,158][20381] Updated weights for policy 1, policy_version 1463097 (0.0007) [2023-12-27 15:47:38,218][20381] Updated weights for policy 1, policy_version 1463107 (0.0006) [2023-12-27 15:47:38,270][20381] Updated weights for policy 1, policy_version 1463117 (0.0010) [2023-12-27 15:47:38,551][20347] Updated weights for policy 0, policy_version 1464545 (0.0010) [2023-12-27 15:47:38,619][20347] Updated weights for policy 0, policy_version 1464555 (0.0010) [2023-12-27 15:47:38,672][20347] Updated weights for policy 0, policy_version 1464565 (0.0010) [2023-12-27 15:47:38,919][20381] Updated weights for policy 1, policy_version 1463127 (0.0007) [2023-12-27 15:47:38,979][20381] Updated weights for policy 1, policy_version 1463137 (0.0005) [2023-12-27 15:47:39,035][20381] Updated weights for policy 1, policy_version 1463147 (0.0007) [2023-12-27 15:47:39,416][20347] Updated weights for policy 0, policy_version 1464575 (0.0007) [2023-12-27 15:47:39,485][20347] Updated weights for policy 0, policy_version 1464585 (0.0006) [2023-12-27 15:47:39,544][20347] Updated weights for policy 0, policy_version 1464595 (0.0005) [2023-12-27 15:47:39,699][20381] Updated weights for policy 1, policy_version 1463157 (0.0008) [2023-12-27 15:47:39,754][20381] Updated weights for policy 1, policy_version 1463167 (0.0007) [2023-12-27 15:47:39,815][20381] Updated weights for policy 1, policy_version 1463177 (0.0007) [2023-12-27 15:47:40,138][20347] Updated weights for policy 0, policy_version 1464605 (0.0007) [2023-12-27 15:47:40,196][20347] Updated weights for policy 0, policy_version 1464615 (0.0008) [2023-12-27 15:47:40,251][20347] Updated weights for policy 0, policy_version 1464625 (0.0007) [2023-12-27 15:47:40,508][20381] Updated weights for policy 1, policy_version 1463187 (0.0008) [2023-12-27 15:47:40,568][20381] Updated weights for policy 1, policy_version 1463197 (0.0011) [2023-12-27 15:47:40,621][20381] Updated weights for policy 1, policy_version 1463207 (0.0010) [2023-12-27 15:47:40,961][20347] Updated weights for policy 0, policy_version 1464635 (0.0008) [2023-12-27 15:47:41,010][20347] Updated weights for policy 0, policy_version 1464645 (0.0008) [2023-12-27 15:47:41,074][20347] Updated weights for policy 0, policy_version 1464655 (0.0008) [2023-12-27 15:47:41,262][20381] Updated weights for policy 1, policy_version 1463217 (0.0010) [2023-12-27 15:47:41,324][20381] Updated weights for policy 1, policy_version 1463227 (0.0009) [2023-12-27 15:47:41,392][20381] Updated weights for policy 1, policy_version 1463237 (0.0007) [2023-12-27 15:47:41,457][20381] Updated weights for policy 1, policy_version 1463247 (0.0008) [2023-12-27 15:47:41,799][20347] Updated weights for policy 0, policy_version 1464665 (0.0008) [2023-12-27 15:47:41,850][20347] Updated weights for policy 0, policy_version 1464675 (0.0006) [2023-12-27 15:47:41,913][20347] Updated weights for policy 0, policy_version 1464685 (0.0006) [2023-12-27 15:47:41,965][20347] Updated weights for policy 0, policy_version 1464695 (0.0007) [2023-12-27 15:47:42,299][20381] Updated weights for policy 1, policy_version 1463257 (0.0009) [2023-12-27 15:47:42,361][20381] Updated weights for policy 1, policy_version 1463267 (0.0007) [2023-12-27 15:47:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 749658112. Throughput: 0: 10465.5, 1: 10304.6. Samples: 749653788. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:42,363][19170] Avg episode reward: [(0, '715.590'), (1, '902.400')] [2023-12-27 15:47:42,425][20381] Updated weights for policy 1, policy_version 1463277 (0.0007) [2023-12-27 15:47:42,679][20347] Updated weights for policy 0, policy_version 1464705 (0.0007) [2023-12-27 15:47:42,741][20347] Updated weights for policy 0, policy_version 1464715 (0.0008) [2023-12-27 15:47:42,803][20347] Updated weights for policy 0, policy_version 1464725 (0.0007) [2023-12-27 15:47:43,021][20381] Updated weights for policy 1, policy_version 1463287 (0.0007) [2023-12-27 15:47:43,071][20381] Updated weights for policy 1, policy_version 1463297 (0.0005) [2023-12-27 15:47:43,131][20381] Updated weights for policy 1, policy_version 1463307 (0.0008) [2023-12-27 15:47:43,423][20347] Updated weights for policy 0, policy_version 1464735 (0.0006) [2023-12-27 15:47:43,478][20347] Updated weights for policy 0, policy_version 1464745 (0.0005) [2023-12-27 15:47:43,526][20347] Updated weights for policy 0, policy_version 1464755 (0.0006) [2023-12-27 15:47:43,788][20381] Updated weights for policy 1, policy_version 1463317 (0.0011) [2023-12-27 15:47:43,836][20381] Updated weights for policy 1, policy_version 1463327 (0.0010) [2023-12-27 15:47:43,883][20381] Updated weights for policy 1, policy_version 1463337 (0.0010) [2023-12-27 15:47:44,178][20347] Updated weights for policy 0, policy_version 1464765 (0.0006) [2023-12-27 15:47:44,225][20347] Updated weights for policy 0, policy_version 1464775 (0.0007) [2023-12-27 15:47:44,270][20347] Updated weights for policy 0, policy_version 1464785 (0.0007) [2023-12-27 15:47:44,531][20381] Updated weights for policy 1, policy_version 1463347 (0.0010) [2023-12-27 15:47:44,583][20381] Updated weights for policy 1, policy_version 1463357 (0.0010) [2023-12-27 15:47:44,640][20381] Updated weights for policy 1, policy_version 1463367 (0.0010) [2023-12-27 15:47:44,951][20347] Updated weights for policy 0, policy_version 1464795 (0.0007) [2023-12-27 15:47:45,009][20347] Updated weights for policy 0, policy_version 1464805 (0.0006) [2023-12-27 15:47:45,069][20347] Updated weights for policy 0, policy_version 1464815 (0.0011) [2023-12-27 15:47:45,327][20381] Updated weights for policy 1, policy_version 1463377 (0.0010) [2023-12-27 15:47:45,376][20381] Updated weights for policy 1, policy_version 1463387 (0.0010) [2023-12-27 15:47:45,421][20381] Updated weights for policy 1, policy_version 1463397 (0.0010) [2023-12-27 15:47:45,468][20381] Updated weights for policy 1, policy_version 1463407 (0.0008) [2023-12-27 15:47:45,736][20347] Updated weights for policy 0, policy_version 1464825 (0.0010) [2023-12-27 15:47:45,790][20347] Updated weights for policy 0, policy_version 1464835 (0.0006) [2023-12-27 15:47:45,844][20347] Updated weights for policy 0, policy_version 1464845 (0.0006) [2023-12-27 15:47:45,900][20347] Updated weights for policy 0, policy_version 1464855 (0.0005) [2023-12-27 15:47:46,063][20381] Updated weights for policy 1, policy_version 1463417 (0.0006) [2023-12-27 15:47:46,119][20381] Updated weights for policy 1, policy_version 1463427 (0.0009) [2023-12-27 15:47:46,180][20381] Updated weights for policy 1, policy_version 1463437 (0.0008) [2023-12-27 15:47:46,484][20347] Updated weights for policy 0, policy_version 1464865 (0.0008) [2023-12-27 15:47:46,534][20347] Updated weights for policy 0, policy_version 1464875 (0.0009) [2023-12-27 15:47:46,601][20347] Updated weights for policy 0, policy_version 1464885 (0.0007) [2023-12-27 15:47:46,755][20381] Updated weights for policy 1, policy_version 1463447 (0.0009) [2023-12-27 15:47:46,815][20381] Updated weights for policy 1, policy_version 1463457 (0.0011) [2023-12-27 15:47:46,870][20381] Updated weights for policy 1, policy_version 1463467 (0.0011) [2023-12-27 15:47:47,312][20347] Updated weights for policy 0, policy_version 1464895 (0.0009) [2023-12-27 15:47:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 749764608. Throughput: 0: 10491.5, 1: 10377.7. Samples: 749779376. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:47,363][19170] Avg episode reward: [(0, '685.960'), (1, '863.190')] [2023-12-27 15:47:47,368][20347] Updated weights for policy 0, policy_version 1464905 (0.0008) [2023-12-27 15:47:47,429][20347] Updated weights for policy 0, policy_version 1464915 (0.0006) [2023-12-27 15:47:47,552][20381] Updated weights for policy 1, policy_version 1463477 (0.0009) [2023-12-27 15:47:47,610][20381] Updated weights for policy 1, policy_version 1463487 (0.0008) [2023-12-27 15:47:47,656][20381] Updated weights for policy 1, policy_version 1463497 (0.0007) [2023-12-27 15:47:48,088][20347] Updated weights for policy 0, policy_version 1464925 (0.0009) [2023-12-27 15:47:48,147][20347] Updated weights for policy 0, policy_version 1464935 (0.0008) [2023-12-27 15:47:48,197][20347] Updated weights for policy 0, policy_version 1464945 (0.0007) [2023-12-27 15:47:48,337][20381] Updated weights for policy 1, policy_version 1463507 (0.0007) [2023-12-27 15:47:48,403][20381] Updated weights for policy 1, policy_version 1463517 (0.0008) [2023-12-27 15:47:48,455][20381] Updated weights for policy 1, policy_version 1463527 (0.0008) [2023-12-27 15:47:48,906][20347] Updated weights for policy 0, policy_version 1464955 (0.0007) [2023-12-27 15:47:48,954][20347] Updated weights for policy 0, policy_version 1464965 (0.0005) [2023-12-27 15:47:49,008][20347] Updated weights for policy 0, policy_version 1464975 (0.0005) [2023-12-27 15:47:49,071][20381] Updated weights for policy 1, policy_version 1463537 (0.0007) [2023-12-27 15:47:49,126][20381] Updated weights for policy 1, policy_version 1463547 (0.0006) [2023-12-27 15:47:49,185][20381] Updated weights for policy 1, policy_version 1463557 (0.0008) [2023-12-27 15:47:49,250][20381] Updated weights for policy 1, policy_version 1463567 (0.0007) [2023-12-27 15:47:49,642][20347] Updated weights for policy 0, policy_version 1464985 (0.0005) [2023-12-27 15:47:49,699][20347] Updated weights for policy 0, policy_version 1464995 (0.0008) [2023-12-27 15:47:49,757][20347] Updated weights for policy 0, policy_version 1465005 (0.0007) [2023-12-27 15:47:49,808][20347] Updated weights for policy 0, policy_version 1465015 (0.0007) [2023-12-27 15:47:49,977][20381] Updated weights for policy 1, policy_version 1463577 (0.0007) [2023-12-27 15:47:50,040][20381] Updated weights for policy 1, policy_version 1463587 (0.0007) [2023-12-27 15:47:50,103][20381] Updated weights for policy 1, policy_version 1463597 (0.0007) [2023-12-27 15:47:50,463][20347] Updated weights for policy 0, policy_version 1465025 (0.0009) [2023-12-27 15:47:50,518][20347] Updated weights for policy 0, policy_version 1465035 (0.0009) [2023-12-27 15:47:50,571][20347] Updated weights for policy 0, policy_version 1465045 (0.0008) [2023-12-27 15:47:50,814][20381] Updated weights for policy 1, policy_version 1463607 (0.0007) [2023-12-27 15:47:50,876][20381] Updated weights for policy 1, policy_version 1463617 (0.0008) [2023-12-27 15:47:50,938][20381] Updated weights for policy 1, policy_version 1463627 (0.0008) [2023-12-27 15:47:51,211][20347] Updated weights for policy 0, policy_version 1465055 (0.0008) [2023-12-27 15:47:51,276][20347] Updated weights for policy 0, policy_version 1465065 (0.0009) [2023-12-27 15:47:51,335][20347] Updated weights for policy 0, policy_version 1465075 (0.0010) [2023-12-27 15:47:51,713][20381] Updated weights for policy 1, policy_version 1463637 (0.0008) [2023-12-27 15:47:51,771][20381] Updated weights for policy 1, policy_version 1463647 (0.0008) [2023-12-27 15:47:51,819][20381] Updated weights for policy 1, policy_version 1463657 (0.0007) [2023-12-27 15:47:52,023][20347] Updated weights for policy 0, policy_version 1465085 (0.0009) [2023-12-27 15:47:52,078][20347] Updated weights for policy 0, policy_version 1465095 (0.0010) [2023-12-27 15:47:52,137][20347] Updated weights for policy 0, policy_version 1465105 (0.0008) [2023-12-27 15:47:52,138][20291] KL-divergence is very high: 108.5069 [2023-12-27 15:47:52,145][20291] KL-divergence is very high: 120.5612 [2023-12-27 15:47:52,162][20291] KL-divergence is very high: 132.5968 [2023-12-27 15:47:52,173][20291] KL-divergence is very high: 127.0300 [2023-12-27 15:47:52,372][19170] Fps is (10 sec: 21278.2, 60 sec: 20749.7, 300 sec: 20548.7). Total num frames: 749871104. Throughput: 0: 10468.6, 1: 10338.7. Samples: 749840176. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:52,373][19170] Avg episode reward: [(0, '637.360'), (1, '902.190')] [2023-12-27 15:47:52,381][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001463664_374726656.pth... [2023-12-27 15:47:52,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001465112_375144448.pth... [2023-12-27 15:47:52,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001462448_374415360.pth [2023-12-27 15:47:52,386][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001463864_374824960.pth [2023-12-27 15:47:52,548][20381] Updated weights for policy 1, policy_version 1463667 (0.0008) [2023-12-27 15:47:52,596][20381] Updated weights for policy 1, policy_version 1463677 (0.0007) [2023-12-27 15:47:52,645][20381] Updated weights for policy 1, policy_version 1463687 (0.0007) [2023-12-27 15:47:52,840][20347] Updated weights for policy 0, policy_version 1465115 (0.0006) [2023-12-27 15:47:52,892][20347] Updated weights for policy 0, policy_version 1465125 (0.0007) [2023-12-27 15:47:52,949][20347] Updated weights for policy 0, policy_version 1465135 (0.0007) [2023-12-27 15:47:53,346][20381] Updated weights for policy 1, policy_version 1463697 (0.0007) [2023-12-27 15:47:53,406][20381] Updated weights for policy 1, policy_version 1463707 (0.0005) [2023-12-27 15:47:53,463][20381] Updated weights for policy 1, policy_version 1463717 (0.0007) [2023-12-27 15:47:53,529][20381] Updated weights for policy 1, policy_version 1463727 (0.0008) [2023-12-27 15:47:53,598][20347] Updated weights for policy 0, policy_version 1465145 (0.0007) [2023-12-27 15:47:53,658][20347] Updated weights for policy 0, policy_version 1465155 (0.0006) [2023-12-27 15:47:53,714][20347] Updated weights for policy 0, policy_version 1465165 (0.0005) [2023-12-27 15:47:53,769][20347] Updated weights for policy 0, policy_version 1465175 (0.0005) [2023-12-27 15:47:54,163][20381] Updated weights for policy 1, policy_version 1463737 (0.0005) [2023-12-27 15:47:54,217][20381] Updated weights for policy 1, policy_version 1463747 (0.0007) [2023-12-27 15:47:54,270][20381] Updated weights for policy 1, policy_version 1463757 (0.0007) [2023-12-27 15:47:54,386][20347] Updated weights for policy 0, policy_version 1465185 (0.0007) [2023-12-27 15:47:54,430][20347] Updated weights for policy 0, policy_version 1465195 (0.0007) [2023-12-27 15:47:54,472][20347] Updated weights for policy 0, policy_version 1465205 (0.0007) [2023-12-27 15:47:54,953][20381] Updated weights for policy 1, policy_version 1463767 (0.0007) [2023-12-27 15:47:55,017][20381] Updated weights for policy 1, policy_version 1463777 (0.0008) [2023-12-27 15:47:55,078][20381] Updated weights for policy 1, policy_version 1463787 (0.0009) [2023-12-27 15:47:55,177][20347] Updated weights for policy 0, policy_version 1465215 (0.0008) [2023-12-27 15:47:55,231][20347] Updated weights for policy 0, policy_version 1465225 (0.0007) [2023-12-27 15:47:55,289][20347] Updated weights for policy 0, policy_version 1465235 (0.0007) [2023-12-27 15:47:55,767][20381] Updated weights for policy 1, policy_version 1463797 (0.0007) [2023-12-27 15:47:55,818][20381] Updated weights for policy 1, policy_version 1463807 (0.0009) [2023-12-27 15:47:55,867][20381] Updated weights for policy 1, policy_version 1463817 (0.0007) [2023-12-27 15:47:55,909][20347] Updated weights for policy 0, policy_version 1465245 (0.0007) [2023-12-27 15:47:55,965][20347] Updated weights for policy 0, policy_version 1465255 (0.0006) [2023-12-27 15:47:56,025][20347] Updated weights for policy 0, policy_version 1465265 (0.0006) [2023-12-27 15:47:56,546][20381] Updated weights for policy 1, policy_version 1463827 (0.0008) [2023-12-27 15:47:56,599][20381] Updated weights for policy 1, policy_version 1463837 (0.0007) [2023-12-27 15:47:56,608][20347] Updated weights for policy 0, policy_version 1465275 (0.0007) [2023-12-27 15:47:56,650][20381] Updated weights for policy 1, policy_version 1463847 (0.0006) [2023-12-27 15:47:56,656][20347] Updated weights for policy 0, policy_version 1465285 (0.0005) [2023-12-27 15:47:56,708][20347] Updated weights for policy 0, policy_version 1465295 (0.0006) [2023-12-27 15:47:57,337][20381] Updated weights for policy 1, policy_version 1463857 (0.0006) [2023-12-27 15:47:57,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 749969408. Throughput: 0: 10468.7, 1: 10323.4. Samples: 749963144. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:47:57,364][19170] Avg episode reward: [(0, '657.080'), (1, '892.350')] [2023-12-27 15:47:57,400][20381] Updated weights for policy 1, policy_version 1463867 (0.0007) [2023-12-27 15:47:57,403][20347] Updated weights for policy 0, policy_version 1465305 (0.0007) [2023-12-27 15:47:57,453][20347] Updated weights for policy 0, policy_version 1465315 (0.0006) [2023-12-27 15:47:57,454][20381] Updated weights for policy 1, policy_version 1463877 (0.0007) [2023-12-27 15:47:57,504][20347] Updated weights for policy 0, policy_version 1465325 (0.0005) [2023-12-27 15:47:57,508][20381] Updated weights for policy 1, policy_version 1463887 (0.0006) [2023-12-27 15:47:57,554][20347] Updated weights for policy 0, policy_version 1465335 (0.0007) [2023-12-27 15:47:58,142][20381] Updated weights for policy 1, policy_version 1463897 (0.0005) [2023-12-27 15:47:58,177][20347] Updated weights for policy 0, policy_version 1465345 (0.0006) [2023-12-27 15:47:58,207][20381] Updated weights for policy 1, policy_version 1463907 (0.0006) [2023-12-27 15:47:58,239][20347] Updated weights for policy 0, policy_version 1465355 (0.0006) [2023-12-27 15:47:58,267][20381] Updated weights for policy 1, policy_version 1463917 (0.0007) [2023-12-27 15:47:58,297][20347] Updated weights for policy 0, policy_version 1465365 (0.0007) [2023-12-27 15:47:58,998][20381] Updated weights for policy 1, policy_version 1463927 (0.0006) [2023-12-27 15:47:59,047][20381] Updated weights for policy 1, policy_version 1463937 (0.0007) [2023-12-27 15:47:59,085][20347] Updated weights for policy 0, policy_version 1465375 (0.0009) [2023-12-27 15:47:59,103][20381] Updated weights for policy 1, policy_version 1463947 (0.0008) [2023-12-27 15:47:59,145][20347] Updated weights for policy 0, policy_version 1465385 (0.0008) [2023-12-27 15:47:59,204][20347] Updated weights for policy 0, policy_version 1465395 (0.0008) [2023-12-27 15:47:59,825][20381] Updated weights for policy 1, policy_version 1463957 (0.0009) [2023-12-27 15:47:59,835][20347] Updated weights for policy 0, policy_version 1465405 (0.0008) [2023-12-27 15:47:59,889][20381] Updated weights for policy 1, policy_version 1463967 (0.0007) [2023-12-27 15:47:59,895][20347] Updated weights for policy 0, policy_version 1465415 (0.0007) [2023-12-27 15:47:59,949][20381] Updated weights for policy 1, policy_version 1463977 (0.0008) [2023-12-27 15:47:59,952][20347] Updated weights for policy 0, policy_version 1465425 (0.0008) [2023-12-27 15:48:00,561][20381] Updated weights for policy 1, policy_version 1463987 (0.0007) [2023-12-27 15:48:00,606][20347] Updated weights for policy 0, policy_version 1465435 (0.0006) [2023-12-27 15:48:00,618][20381] Updated weights for policy 1, policy_version 1463997 (0.0007) [2023-12-27 15:48:00,657][20347] Updated weights for policy 0, policy_version 1465445 (0.0005) [2023-12-27 15:48:00,677][20381] Updated weights for policy 1, policy_version 1464007 (0.0006) [2023-12-27 15:48:00,705][20347] Updated weights for policy 0, policy_version 1465455 (0.0005) [2023-12-27 15:48:01,350][20381] Updated weights for policy 1, policy_version 1464017 (0.0005) [2023-12-27 15:48:01,414][20381] Updated weights for policy 1, policy_version 1464027 (0.0008) [2023-12-27 15:48:01,424][20347] Updated weights for policy 0, policy_version 1465465 (0.0007) [2023-12-27 15:48:01,467][20381] Updated weights for policy 1, policy_version 1464037 (0.0005) [2023-12-27 15:48:01,480][20347] Updated weights for policy 0, policy_version 1465475 (0.0007) [2023-12-27 15:48:01,525][20381] Updated weights for policy 1, policy_version 1464047 (0.0005) [2023-12-27 15:48:01,538][20347] Updated weights for policy 0, policy_version 1465485 (0.0007) [2023-12-27 15:48:01,595][20347] Updated weights for policy 0, policy_version 1465495 (0.0009) [2023-12-27 15:48:02,234][20381] Updated weights for policy 1, policy_version 1464057 (0.0008) [2023-12-27 15:48:02,241][20292] KL-divergence is very high: 173.7383 [2023-12-27 15:48:02,254][20292] KL-divergence is very high: 110.7718 [2023-12-27 15:48:02,291][20292] KL-divergence is very high: 126.9065 [2023-12-27 15:48:02,296][20381] Updated weights for policy 1, policy_version 1464067 (0.0008) [2023-12-27 15:48:02,312][20347] Updated weights for policy 0, policy_version 1465505 (0.0007) [2023-12-27 15:48:02,348][20381] Updated weights for policy 1, policy_version 1464077 (0.0005) [2023-12-27 15:48:02,362][19170] Fps is (10 sec: 19680.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 750067712. Throughput: 0: 10476.6, 1: 10340.0. Samples: 750085864. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-12-27 15:48:02,363][19170] Avg episode reward: [(0, '705.820'), (1, '911.960')] [2023-12-27 15:48:02,371][20347] Updated weights for policy 0, policy_version 1465515 (0.0006) [2023-12-27 15:48:02,433][20347] Updated weights for policy 0, policy_version 1465525 (0.0006) [2023-12-27 15:48:03,019][20381] Updated weights for policy 1, policy_version 1464087 (0.0007) [2023-12-27 15:48:03,075][20381] Updated weights for policy 1, policy_version 1464097 (0.0006) [2023-12-27 15:48:03,077][20347] Updated weights for policy 0, policy_version 1465535 (0.0006) [2023-12-27 15:48:03,125][20347] Updated weights for policy 0, policy_version 1465545 (0.0005) [2023-12-27 15:48:03,130][20381] Updated weights for policy 1, policy_version 1464107 (0.0006) [2023-12-27 15:48:03,170][20347] Updated weights for policy 0, policy_version 1465555 (0.0006) [2023-12-27 15:48:03,716][20381] Updated weights for policy 1, policy_version 1464117 (0.0006) [2023-12-27 15:48:03,765][20381] Updated weights for policy 1, policy_version 1464127 (0.0008) [2023-12-27 15:48:03,813][20381] Updated weights for policy 1, policy_version 1464137 (0.0008) [2023-12-27 15:48:03,902][20347] Updated weights for policy 0, policy_version 1465565 (0.0008) [2023-12-27 15:48:03,958][20347] Updated weights for policy 0, policy_version 1465575 (0.0009) [2023-12-27 15:48:04,048][20347] Updated weights for policy 0, policy_version 1465585 (0.0008) [2023-12-27 15:48:04,448][20381] Updated weights for policy 1, policy_version 1464147 (0.0007) [2023-12-27 15:48:04,507][20381] Updated weights for policy 1, policy_version 1464157 (0.0008) [2023-12-27 15:48:04,566][20381] Updated weights for policy 1, policy_version 1464167 (0.0006) [2023-12-27 15:48:04,774][20347] Updated weights for policy 0, policy_version 1465595 (0.0008) [2023-12-27 15:48:04,830][20347] Updated weights for policy 0, policy_version 1465605 (0.0008) [2023-12-27 15:48:04,881][20347] Updated weights for policy 0, policy_version 1465615 (0.0007) [2023-12-27 15:48:05,251][20381] Updated weights for policy 1, policy_version 1464177 (0.0007) [2023-12-27 15:48:05,323][20381] Updated weights for policy 1, policy_version 1464187 (0.0008) [2023-12-27 15:48:05,387][20381] Updated weights for policy 1, policy_version 1464197 (0.0008) [2023-12-27 15:48:05,450][20381] Updated weights for policy 1, policy_version 1464207 (0.0008) [2023-12-27 15:48:05,510][20347] Updated weights for policy 0, policy_version 1465625 (0.0007) [2023-12-27 15:48:05,567][20347] Updated weights for policy 0, policy_version 1465635 (0.0006) [2023-12-27 15:48:05,613][20347] Updated weights for policy 0, policy_version 1465645 (0.0007) [2023-12-27 15:48:05,661][20347] Updated weights for policy 0, policy_version 1465655 (0.0009) [2023-12-27 15:48:06,069][20381] Updated weights for policy 1, policy_version 1464217 (0.0008) [2023-12-27 15:48:06,142][20381] Updated weights for policy 1, policy_version 1464227 (0.0006) [2023-12-27 15:48:06,195][20381] Updated weights for policy 1, policy_version 1464237 (0.0007) [2023-12-27 15:48:06,292][20347] Updated weights for policy 0, policy_version 1465665 (0.0010) [2023-12-27 15:48:06,355][20347] Updated weights for policy 0, policy_version 1465675 (0.0011) [2023-12-27 15:48:06,410][20347] Updated weights for policy 0, policy_version 1465685 (0.0010) [2023-12-27 15:48:06,890][20381] Updated weights for policy 1, policy_version 1464247 (0.0007) [2023-12-27 15:48:06,952][20381] Updated weights for policy 1, policy_version 1464257 (0.0007) [2023-12-27 15:48:07,010][20381] Updated weights for policy 1, policy_version 1464267 (0.0007) [2023-12-27 15:48:07,089][20347] Updated weights for policy 0, policy_version 1465695 (0.0009) [2023-12-27 15:48:07,130][20291] KL-divergence is very high: 110.4960 [2023-12-27 15:48:07,134][20347] Updated weights for policy 0, policy_version 1465705 (0.0008) [2023-12-27 15:48:07,186][20347] Updated weights for policy 0, policy_version 1465715 (0.0010) [2023-12-27 15:48:07,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20889.7, 300 sec: 20577.2). Total num frames: 750182400. Throughput: 0: 10450.9, 1: 10349.1. Samples: 750147352. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:07,363][19170] Avg episode reward: [(0, '695.720'), (1, '921.790')] [2023-12-27 15:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001464272_374882304.pth... [2023-12-27 15:48:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001465720_375300096.pth... [2023-12-27 15:48:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001463088_374579200.pth [2023-12-27 15:48:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001464504_374988800.pth [2023-12-27 15:48:07,727][20381] Updated weights for policy 1, policy_version 1464277 (0.0007) [2023-12-27 15:48:07,791][20381] Updated weights for policy 1, policy_version 1464287 (0.0008) [2023-12-27 15:48:07,845][20347] Updated weights for policy 0, policy_version 1465725 (0.0008) [2023-12-27 15:48:07,847][20381] Updated weights for policy 1, policy_version 1464297 (0.0006) [2023-12-27 15:48:07,911][20347] Updated weights for policy 0, policy_version 1465735 (0.0007) [2023-12-27 15:48:07,971][20347] Updated weights for policy 0, policy_version 1465745 (0.0008) [2023-12-27 15:48:08,485][20381] Updated weights for policy 1, policy_version 1464307 (0.0007) [2023-12-27 15:48:08,543][20381] Updated weights for policy 1, policy_version 1464317 (0.0007) [2023-12-27 15:48:08,595][20381] Updated weights for policy 1, policy_version 1464327 (0.0007) [2023-12-27 15:48:08,650][20347] Updated weights for policy 0, policy_version 1465755 (0.0011) [2023-12-27 15:48:08,705][20347] Updated weights for policy 0, policy_version 1465765 (0.0011) [2023-12-27 15:48:08,758][20347] Updated weights for policy 0, policy_version 1465775 (0.0008) [2023-12-27 15:48:09,308][20381] Updated weights for policy 1, policy_version 1464337 (0.0008) [2023-12-27 15:48:09,328][20347] Updated weights for policy 0, policy_version 1465785 (0.0006) [2023-12-27 15:48:09,372][20381] Updated weights for policy 1, policy_version 1464347 (0.0008) [2023-12-27 15:48:09,405][20347] Updated weights for policy 0, policy_version 1465795 (0.0009) [2023-12-27 15:48:09,440][20381] Updated weights for policy 1, policy_version 1464357 (0.0008) [2023-12-27 15:48:09,463][20347] Updated weights for policy 0, policy_version 1465805 (0.0008) [2023-12-27 15:48:09,504][20381] Updated weights for policy 1, policy_version 1464367 (0.0008) [2023-12-27 15:48:09,520][20347] Updated weights for policy 0, policy_version 1465815 (0.0008) [2023-12-27 15:48:10,200][20381] Updated weights for policy 1, policy_version 1464377 (0.0007) [2023-12-27 15:48:10,235][20347] Updated weights for policy 0, policy_version 1465825 (0.0010) [2023-12-27 15:48:10,267][20381] Updated weights for policy 1, policy_version 1464387 (0.0007) [2023-12-27 15:48:10,292][20347] Updated weights for policy 0, policy_version 1465835 (0.0010) [2023-12-27 15:48:10,335][20381] Updated weights for policy 1, policy_version 1464397 (0.0009) [2023-12-27 15:48:10,349][20347] Updated weights for policy 0, policy_version 1465845 (0.0008) [2023-12-27 15:48:10,994][20381] Updated weights for policy 1, policy_version 1464407 (0.0008) [2023-12-27 15:48:11,044][20347] Updated weights for policy 0, policy_version 1465855 (0.0008) [2023-12-27 15:48:11,056][20381] Updated weights for policy 1, policy_version 1464417 (0.0008) [2023-12-27 15:48:11,104][20347] Updated weights for policy 0, policy_version 1465865 (0.0006) [2023-12-27 15:48:11,112][20381] Updated weights for policy 1, policy_version 1464427 (0.0006) [2023-12-27 15:48:11,172][20347] Updated weights for policy 0, policy_version 1465875 (0.0008) [2023-12-27 15:48:11,898][20381] Updated weights for policy 1, policy_version 1464437 (0.0007) [2023-12-27 15:48:11,945][20347] Updated weights for policy 0, policy_version 1465885 (0.0008) [2023-12-27 15:48:11,959][20381] Updated weights for policy 1, policy_version 1464447 (0.0007) [2023-12-27 15:48:12,012][20347] Updated weights for policy 0, policy_version 1465895 (0.0008) [2023-12-27 15:48:12,018][20381] Updated weights for policy 1, policy_version 1464457 (0.0007) [2023-12-27 15:48:12,067][20347] Updated weights for policy 0, policy_version 1465905 (0.0008) [2023-12-27 15:48:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 750280704. Throughput: 0: 10416.8, 1: 10285.2. Samples: 750269384. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:12,363][19170] Avg episode reward: [(0, '695.760'), (1, '951.120')] [2023-12-27 15:48:12,744][20381] Updated weights for policy 1, policy_version 1464467 (0.0009) [2023-12-27 15:48:12,801][20381] Updated weights for policy 1, policy_version 1464477 (0.0010) [2023-12-27 15:48:12,806][20347] Updated weights for policy 0, policy_version 1465915 (0.0007) [2023-12-27 15:48:12,856][20381] Updated weights for policy 1, policy_version 1464487 (0.0010) [2023-12-27 15:48:12,864][20347] Updated weights for policy 0, policy_version 1465925 (0.0009) [2023-12-27 15:48:12,913][20347] Updated weights for policy 0, policy_version 1465935 (0.0009) [2023-12-27 15:48:13,534][20381] Updated weights for policy 1, policy_version 1464497 (0.0010) [2023-12-27 15:48:13,585][20381] Updated weights for policy 1, policy_version 1464507 (0.0010) [2023-12-27 15:48:13,605][20347] Updated weights for policy 0, policy_version 1465945 (0.0007) [2023-12-27 15:48:13,639][20381] Updated weights for policy 1, policy_version 1464517 (0.0010) [2023-12-27 15:48:13,655][20347] Updated weights for policy 0, policy_version 1465955 (0.0005) [2023-12-27 15:48:13,686][20381] Updated weights for policy 1, policy_version 1464527 (0.0010) [2023-12-27 15:48:13,711][20347] Updated weights for policy 0, policy_version 1465965 (0.0006) [2023-12-27 15:48:13,765][20347] Updated weights for policy 0, policy_version 1465975 (0.0007) [2023-12-27 15:48:14,360][20381] Updated weights for policy 1, policy_version 1464537 (0.0006) [2023-12-27 15:48:14,414][20381] Updated weights for policy 1, policy_version 1464547 (0.0010) [2023-12-27 15:48:14,416][20347] Updated weights for policy 0, policy_version 1465985 (0.0006) [2023-12-27 15:48:14,470][20347] Updated weights for policy 0, policy_version 1465995 (0.0006) [2023-12-27 15:48:14,472][20381] Updated weights for policy 1, policy_version 1464557 (0.0010) [2023-12-27 15:48:14,522][20347] Updated weights for policy 0, policy_version 1466005 (0.0007) [2023-12-27 15:48:15,133][20381] Updated weights for policy 1, policy_version 1464567 (0.0008) [2023-12-27 15:48:15,190][20381] Updated weights for policy 1, policy_version 1464577 (0.0007) [2023-12-27 15:48:15,251][20381] Updated weights for policy 1, policy_version 1464587 (0.0008) [2023-12-27 15:48:15,280][20347] Updated weights for policy 0, policy_version 1466015 (0.0006) [2023-12-27 15:48:15,336][20347] Updated weights for policy 0, policy_version 1466025 (0.0008) [2023-12-27 15:48:15,388][20347] Updated weights for policy 0, policy_version 1466035 (0.0007) [2023-12-27 15:48:15,870][20381] Updated weights for policy 1, policy_version 1464597 (0.0006) [2023-12-27 15:48:15,922][20381] Updated weights for policy 1, policy_version 1464607 (0.0008) [2023-12-27 15:48:15,974][20381] Updated weights for policy 1, policy_version 1464617 (0.0007) [2023-12-27 15:48:16,102][20347] Updated weights for policy 0, policy_version 1466045 (0.0009) [2023-12-27 15:48:16,154][20347] Updated weights for policy 0, policy_version 1466055 (0.0010) [2023-12-27 15:48:16,215][20347] Updated weights for policy 0, policy_version 1466065 (0.0007) [2023-12-27 15:48:16,617][20381] Updated weights for policy 1, policy_version 1464627 (0.0007) [2023-12-27 15:48:16,663][20381] Updated weights for policy 1, policy_version 1464637 (0.0007) [2023-12-27 15:48:16,707][20381] Updated weights for policy 1, policy_version 1464647 (0.0005) [2023-12-27 15:48:16,865][20347] Updated weights for policy 0, policy_version 1466075 (0.0007) [2023-12-27 15:48:16,914][20347] Updated weights for policy 0, policy_version 1466085 (0.0010) [2023-12-27 15:48:16,965][20347] Updated weights for policy 0, policy_version 1466095 (0.0010) [2023-12-27 15:48:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 750379008. Throughput: 0: 10334.1, 1: 10264.2. Samples: 750390316. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:17,363][19170] Avg episode reward: [(0, '744.760'), (1, '931.610')] [2023-12-27 15:48:17,405][20381] Updated weights for policy 1, policy_version 1464657 (0.0005) [2023-12-27 15:48:17,463][20381] Updated weights for policy 1, policy_version 1464667 (0.0008) [2023-12-27 15:48:17,522][20381] Updated weights for policy 1, policy_version 1464677 (0.0008) [2023-12-27 15:48:17,587][20381] Updated weights for policy 1, policy_version 1464687 (0.0008) [2023-12-27 15:48:17,661][20347] Updated weights for policy 0, policy_version 1466105 (0.0010) [2023-12-27 15:48:17,723][20347] Updated weights for policy 0, policy_version 1466115 (0.0007) [2023-12-27 15:48:17,780][20347] Updated weights for policy 0, policy_version 1466125 (0.0008) [2023-12-27 15:48:17,837][20347] Updated weights for policy 0, policy_version 1466135 (0.0008) [2023-12-27 15:48:18,290][20381] Updated weights for policy 1, policy_version 1464697 (0.0008) [2023-12-27 15:48:18,349][20381] Updated weights for policy 1, policy_version 1464707 (0.0008) [2023-12-27 15:48:18,412][20381] Updated weights for policy 1, policy_version 1464717 (0.0008) [2023-12-27 15:48:18,544][20347] Updated weights for policy 0, policy_version 1466145 (0.0007) [2023-12-27 15:48:18,607][20347] Updated weights for policy 0, policy_version 1466155 (0.0007) [2023-12-27 15:48:18,658][20347] Updated weights for policy 0, policy_version 1466165 (0.0007) [2023-12-27 15:48:19,082][20381] Updated weights for policy 1, policy_version 1464727 (0.0006) [2023-12-27 15:48:19,141][20381] Updated weights for policy 1, policy_version 1464737 (0.0007) [2023-12-27 15:48:19,197][20381] Updated weights for policy 1, policy_version 1464747 (0.0008) [2023-12-27 15:48:19,377][20347] Updated weights for policy 0, policy_version 1466175 (0.0008) [2023-12-27 15:48:19,431][20347] Updated weights for policy 0, policy_version 1466185 (0.0008) [2023-12-27 15:48:19,487][20347] Updated weights for policy 0, policy_version 1466195 (0.0008) [2023-12-27 15:48:19,914][20381] Updated weights for policy 1, policy_version 1464757 (0.0008) [2023-12-27 15:48:19,980][20381] Updated weights for policy 1, policy_version 1464767 (0.0009) [2023-12-27 15:48:20,042][20381] Updated weights for policy 1, policy_version 1464777 (0.0008) [2023-12-27 15:48:20,226][20347] Updated weights for policy 0, policy_version 1466205 (0.0007) [2023-12-27 15:48:20,290][20347] Updated weights for policy 0, policy_version 1466215 (0.0008) [2023-12-27 15:48:20,354][20347] Updated weights for policy 0, policy_version 1466225 (0.0008) [2023-12-27 15:48:20,769][20381] Updated weights for policy 1, policy_version 1464787 (0.0008) [2023-12-27 15:48:20,830][20381] Updated weights for policy 1, policy_version 1464797 (0.0008) [2023-12-27 15:48:20,896][20381] Updated weights for policy 1, policy_version 1464807 (0.0008) [2023-12-27 15:48:21,110][20347] Updated weights for policy 0, policy_version 1466235 (0.0009) [2023-12-27 15:48:21,173][20347] Updated weights for policy 0, policy_version 1466245 (0.0009) [2023-12-27 15:48:21,237][20347] Updated weights for policy 0, policy_version 1466255 (0.0008) [2023-12-27 15:48:21,611][20381] Updated weights for policy 1, policy_version 1464817 (0.0010) [2023-12-27 15:48:21,677][20381] Updated weights for policy 1, policy_version 1464827 (0.0009) [2023-12-27 15:48:21,744][20381] Updated weights for policy 1, policy_version 1464837 (0.0010) [2023-12-27 15:48:21,801][20381] Updated weights for policy 1, policy_version 1464847 (0.0007) [2023-12-27 15:48:21,978][20347] Updated weights for policy 0, policy_version 1466265 (0.0008) [2023-12-27 15:48:22,037][20347] Updated weights for policy 0, policy_version 1466275 (0.0006) [2023-12-27 15:48:22,098][20347] Updated weights for policy 0, policy_version 1466285 (0.0006) [2023-12-27 15:48:22,149][20347] Updated weights for policy 0, policy_version 1466295 (0.0006) [2023-12-27 15:48:22,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 750477312. Throughput: 0: 10291.7, 1: 10183.7. Samples: 750448592. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:22,363][19170] Avg episode reward: [(0, '745.000'), (1, '931.550')] [2023-12-27 15:48:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001464848_375029760.pth... [2023-12-27 15:48:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001466296_375447552.pth... [2023-12-27 15:48:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001463664_374726656.pth [2023-12-27 15:48:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001465112_375144448.pth [2023-12-27 15:48:22,543][20381] Updated weights for policy 1, policy_version 1464857 (0.0007) [2023-12-27 15:48:22,604][20381] Updated weights for policy 1, policy_version 1464867 (0.0007) [2023-12-27 15:48:22,666][20381] Updated weights for policy 1, policy_version 1464877 (0.0006) [2023-12-27 15:48:22,883][20347] Updated weights for policy 0, policy_version 1466305 (0.0007) [2023-12-27 15:48:22,943][20347] Updated weights for policy 0, policy_version 1466315 (0.0007) [2023-12-27 15:48:22,992][20347] Updated weights for policy 0, policy_version 1466325 (0.0008) [2023-12-27 15:48:23,228][20381] Updated weights for policy 1, policy_version 1464887 (0.0005) [2023-12-27 15:48:23,275][20381] Updated weights for policy 1, policy_version 1464897 (0.0006) [2023-12-27 15:48:23,329][20381] Updated weights for policy 1, policy_version 1464907 (0.0005) [2023-12-27 15:48:23,756][20347] Updated weights for policy 0, policy_version 1466335 (0.0006) [2023-12-27 15:48:23,820][20347] Updated weights for policy 0, policy_version 1466345 (0.0005) [2023-12-27 15:48:23,861][20381] Updated weights for policy 1, policy_version 1464917 (0.0005) [2023-12-27 15:48:23,887][20347] Updated weights for policy 0, policy_version 1466355 (0.0005) [2023-12-27 15:48:23,916][20381] Updated weights for policy 1, policy_version 1464927 (0.0007) [2023-12-27 15:48:23,970][20381] Updated weights for policy 1, policy_version 1464937 (0.0008) [2023-12-27 15:48:24,480][20347] Updated weights for policy 0, policy_version 1466365 (0.0006) [2023-12-27 15:48:24,528][20347] Updated weights for policy 0, policy_version 1466375 (0.0007) [2023-12-27 15:48:24,585][20347] Updated weights for policy 0, policy_version 1466385 (0.0008) [2023-12-27 15:48:24,623][20381] Updated weights for policy 1, policy_version 1464947 (0.0007) [2023-12-27 15:48:24,671][20381] Updated weights for policy 1, policy_version 1464957 (0.0006) [2023-12-27 15:48:24,726][20381] Updated weights for policy 1, policy_version 1464967 (0.0007) [2023-12-27 15:48:25,242][20347] Updated weights for policy 0, policy_version 1466395 (0.0007) [2023-12-27 15:48:25,298][20347] Updated weights for policy 0, policy_version 1466405 (0.0010) [2023-12-27 15:48:25,348][20347] Updated weights for policy 0, policy_version 1466415 (0.0010) [2023-12-27 15:48:25,417][20381] Updated weights for policy 1, policy_version 1464977 (0.0008) [2023-12-27 15:48:25,462][20381] Updated weights for policy 1, policy_version 1464987 (0.0008) [2023-12-27 15:48:25,510][20381] Updated weights for policy 1, policy_version 1464997 (0.0007) [2023-12-27 15:48:25,567][20381] Updated weights for policy 1, policy_version 1465007 (0.0007) [2023-12-27 15:48:26,017][20347] Updated weights for policy 0, policy_version 1466425 (0.0010) [2023-12-27 15:48:26,075][20347] Updated weights for policy 0, policy_version 1466435 (0.0007) [2023-12-27 15:48:26,143][20347] Updated weights for policy 0, policy_version 1466445 (0.0008) [2023-12-27 15:48:26,204][20347] Updated weights for policy 0, policy_version 1466455 (0.0007) [2023-12-27 15:48:26,213][20381] Updated weights for policy 1, policy_version 1465017 (0.0006) [2023-12-27 15:48:26,261][20381] Updated weights for policy 1, policy_version 1465027 (0.0005) [2023-12-27 15:48:26,311][20381] Updated weights for policy 1, policy_version 1465037 (0.0005) [2023-12-27 15:48:26,832][20381] Updated weights for policy 1, policy_version 1465047 (0.0006) [2023-12-27 15:48:26,897][20381] Updated weights for policy 1, policy_version 1465057 (0.0005) [2023-12-27 15:48:26,923][20347] Updated weights for policy 0, policy_version 1466465 (0.0007) [2023-12-27 15:48:26,945][20381] Updated weights for policy 1, policy_version 1465067 (0.0005) [2023-12-27 15:48:26,983][20347] Updated weights for policy 0, policy_version 1466475 (0.0007) [2023-12-27 15:48:27,028][20347] Updated weights for policy 0, policy_version 1466485 (0.0008) [2023-12-27 15:48:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 750583808. Throughput: 0: 10199.8, 1: 10209.3. Samples: 750572192. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:27,363][19170] Avg episode reward: [(0, '686.350'), (1, '941.310')] [2023-12-27 15:48:27,454][20381] Updated weights for policy 1, policy_version 1465077 (0.0005) [2023-12-27 15:48:27,497][20381] Updated weights for policy 1, policy_version 1465087 (0.0005) [2023-12-27 15:48:27,542][20381] Updated weights for policy 1, policy_version 1465097 (0.0005) [2023-12-27 15:48:27,712][20347] Updated weights for policy 0, policy_version 1466495 (0.0008) [2023-12-27 15:48:27,756][20347] Updated weights for policy 0, policy_version 1466505 (0.0006) [2023-12-27 15:48:27,805][20347] Updated weights for policy 0, policy_version 1466515 (0.0006) [2023-12-27 15:48:28,124][20381] Updated weights for policy 1, policy_version 1465107 (0.0006) [2023-12-27 15:48:28,179][20381] Updated weights for policy 1, policy_version 1465117 (0.0008) [2023-12-27 15:48:28,237][20381] Updated weights for policy 1, policy_version 1465127 (0.0005) [2023-12-27 15:48:28,482][20347] Updated weights for policy 0, policy_version 1466525 (0.0007) [2023-12-27 15:48:28,539][20347] Updated weights for policy 0, policy_version 1466535 (0.0009) [2023-12-27 15:48:28,596][20347] Updated weights for policy 0, policy_version 1466545 (0.0008) [2023-12-27 15:48:28,858][20381] Updated weights for policy 1, policy_version 1465137 (0.0005) [2023-12-27 15:48:28,913][20381] Updated weights for policy 1, policy_version 1465147 (0.0008) [2023-12-27 15:48:28,968][20381] Updated weights for policy 1, policy_version 1465157 (0.0008) [2023-12-27 15:48:29,024][20381] Updated weights for policy 1, policy_version 1465167 (0.0008) [2023-12-27 15:48:29,329][20347] Updated weights for policy 0, policy_version 1466555 (0.0008) [2023-12-27 15:48:29,394][20347] Updated weights for policy 0, policy_version 1466565 (0.0008) [2023-12-27 15:48:29,444][20347] Updated weights for policy 0, policy_version 1466575 (0.0008) [2023-12-27 15:48:29,739][20381] Updated weights for policy 1, policy_version 1465177 (0.0008) [2023-12-27 15:48:29,791][20381] Updated weights for policy 1, policy_version 1465187 (0.0007) [2023-12-27 15:48:29,852][20381] Updated weights for policy 1, policy_version 1465197 (0.0007) [2023-12-27 15:48:30,111][20347] Updated weights for policy 0, policy_version 1466585 (0.0008) [2023-12-27 15:48:30,164][20347] Updated weights for policy 0, policy_version 1466595 (0.0005) [2023-12-27 15:48:30,220][20347] Updated weights for policy 0, policy_version 1466605 (0.0005) [2023-12-27 15:48:30,276][20347] Updated weights for policy 0, policy_version 1466615 (0.0006) [2023-12-27 15:48:30,587][20381] Updated weights for policy 1, policy_version 1465207 (0.0008) [2023-12-27 15:48:30,632][20381] Updated weights for policy 1, policy_version 1465217 (0.0007) [2023-12-27 15:48:30,692][20381] Updated weights for policy 1, policy_version 1465227 (0.0008) [2023-12-27 15:48:30,874][20347] Updated weights for policy 0, policy_version 1466625 (0.0008) [2023-12-27 15:48:30,920][20347] Updated weights for policy 0, policy_version 1466635 (0.0007) [2023-12-27 15:48:30,971][20347] Updated weights for policy 0, policy_version 1466645 (0.0005) [2023-12-27 15:48:31,400][20381] Updated weights for policy 1, policy_version 1465237 (0.0008) [2023-12-27 15:48:31,461][20381] Updated weights for policy 1, policy_version 1465247 (0.0007) [2023-12-27 15:48:31,513][20381] Updated weights for policy 1, policy_version 1465257 (0.0008) [2023-12-27 15:48:31,637][20347] Updated weights for policy 0, policy_version 1466655 (0.0009) [2023-12-27 15:48:31,696][20347] Updated weights for policy 0, policy_version 1466665 (0.0010) [2023-12-27 15:48:31,762][20347] Updated weights for policy 0, policy_version 1466675 (0.0010) [2023-12-27 15:48:32,267][20381] Updated weights for policy 1, policy_version 1465267 (0.0008) [2023-12-27 15:48:32,321][20381] Updated weights for policy 1, policy_version 1465277 (0.0007) [2023-12-27 15:48:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 750682112. Throughput: 0: 10177.1, 1: 10240.3. Samples: 750698156. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:32,363][19170] Avg episode reward: [(0, '626.850'), (1, '941.360')] [2023-12-27 15:48:32,393][20381] Updated weights for policy 1, policy_version 1465287 (0.0007) [2023-12-27 15:48:32,455][20347] Updated weights for policy 0, policy_version 1466685 (0.0010) [2023-12-27 15:48:32,513][20347] Updated weights for policy 0, policy_version 1466695 (0.0010) [2023-12-27 15:48:32,522][20291] KL-divergence is very high: 119.3778 [2023-12-27 15:48:32,584][20347] Updated weights for policy 0, policy_version 1466705 (0.0010) [2023-12-27 15:48:33,031][20381] Updated weights for policy 1, policy_version 1465297 (0.0006) [2023-12-27 15:48:33,082][20381] Updated weights for policy 1, policy_version 1465307 (0.0008) [2023-12-27 15:48:33,129][20381] Updated weights for policy 1, policy_version 1465317 (0.0007) [2023-12-27 15:48:33,183][20381] Updated weights for policy 1, policy_version 1465327 (0.0007) [2023-12-27 15:48:33,224][20347] Updated weights for policy 0, policy_version 1466715 (0.0007) [2023-12-27 15:48:33,278][20347] Updated weights for policy 0, policy_version 1466725 (0.0005) [2023-12-27 15:48:33,332][20347] Updated weights for policy 0, policy_version 1466735 (0.0005) [2023-12-27 15:48:33,756][20381] Updated weights for policy 1, policy_version 1465337 (0.0008) [2023-12-27 15:48:33,806][20381] Updated weights for policy 1, policy_version 1465347 (0.0008) [2023-12-27 15:48:33,851][20381] Updated weights for policy 1, policy_version 1465357 (0.0008) [2023-12-27 15:48:33,954][20347] Updated weights for policy 0, policy_version 1466745 (0.0006) [2023-12-27 15:48:34,007][20347] Updated weights for policy 0, policy_version 1466755 (0.0010) [2023-12-27 15:48:34,061][20347] Updated weights for policy 0, policy_version 1466765 (0.0010) [2023-12-27 15:48:34,115][20347] Updated weights for policy 0, policy_version 1466775 (0.0010) [2023-12-27 15:48:34,577][20381] Updated weights for policy 1, policy_version 1465367 (0.0008) [2023-12-27 15:48:34,634][20381] Updated weights for policy 1, policy_version 1465377 (0.0007) [2023-12-27 15:48:34,693][20381] Updated weights for policy 1, policy_version 1465387 (0.0007) [2023-12-27 15:48:34,827][20347] Updated weights for policy 0, policy_version 1466785 (0.0006) [2023-12-27 15:48:34,883][20347] Updated weights for policy 0, policy_version 1466795 (0.0005) [2023-12-27 15:48:34,938][20347] Updated weights for policy 0, policy_version 1466805 (0.0007) [2023-12-27 15:48:35,401][20381] Updated weights for policy 1, policy_version 1465397 (0.0007) [2023-12-27 15:48:35,456][20381] Updated weights for policy 1, policy_version 1465407 (0.0008) [2023-12-27 15:48:35,472][20347] Updated weights for policy 0, policy_version 1466815 (0.0008) [2023-12-27 15:48:35,510][20381] Updated weights for policy 1, policy_version 1465417 (0.0006) [2023-12-27 15:48:35,530][20347] Updated weights for policy 0, policy_version 1466825 (0.0010) [2023-12-27 15:48:35,578][20347] Updated weights for policy 0, policy_version 1466835 (0.0010) [2023-12-27 15:48:36,220][20347] Updated weights for policy 0, policy_version 1466845 (0.0010) [2023-12-27 15:48:36,227][20381] Updated weights for policy 1, policy_version 1465427 (0.0006) [2023-12-27 15:48:36,279][20347] Updated weights for policy 0, policy_version 1466855 (0.0010) [2023-12-27 15:48:36,287][20381] Updated weights for policy 1, policy_version 1465437 (0.0007) [2023-12-27 15:48:36,336][20347] Updated weights for policy 0, policy_version 1466865 (0.0010) [2023-12-27 15:48:36,352][20381] Updated weights for policy 1, policy_version 1465447 (0.0006) [2023-12-27 15:48:37,013][20347] Updated weights for policy 0, policy_version 1466875 (0.0010) [2023-12-27 15:48:37,041][20381] Updated weights for policy 1, policy_version 1465457 (0.0007) [2023-12-27 15:48:37,062][20347] Updated weights for policy 0, policy_version 1466885 (0.0010) [2023-12-27 15:48:37,090][20381] Updated weights for policy 1, policy_version 1465467 (0.0005) [2023-12-27 15:48:37,111][20347] Updated weights for policy 0, policy_version 1466895 (0.0010) [2023-12-27 15:48:37,139][20381] Updated weights for policy 1, policy_version 1465477 (0.0005) [2023-12-27 15:48:37,199][20381] Updated weights for policy 1, policy_version 1465487 (0.0007) [2023-12-27 15:48:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.9). Total num frames: 750796800. Throughput: 0: 10198.5, 1: 10273.5. Samples: 750761212. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:37,363][19170] Avg episode reward: [(0, '715.230'), (1, '970.670')] [2023-12-27 15:48:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001465488_375193600.pth... [2023-12-27 15:48:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001466904_375603200.pth... [2023-12-27 15:48:37,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001464272_374882304.pth [2023-12-27 15:48:37,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001465720_375300096.pth [2023-12-27 15:48:37,716][20347] Updated weights for policy 0, policy_version 1466905 (0.0010) [2023-12-27 15:48:37,769][20347] Updated weights for policy 0, policy_version 1466915 (0.0010) [2023-12-27 15:48:37,831][20347] Updated weights for policy 0, policy_version 1466925 (0.0010) [2023-12-27 15:48:37,844][20381] Updated weights for policy 1, policy_version 1465497 (0.0005) [2023-12-27 15:48:37,887][20347] Updated weights for policy 0, policy_version 1466935 (0.0010) [2023-12-27 15:48:37,893][20381] Updated weights for policy 1, policy_version 1465507 (0.0006) [2023-12-27 15:48:37,952][20381] Updated weights for policy 1, policy_version 1465517 (0.0010) [2023-12-27 15:48:38,568][20347] Updated weights for policy 0, policy_version 1466945 (0.0010) [2023-12-27 15:48:38,619][20381] Updated weights for policy 1, policy_version 1465527 (0.0011) [2023-12-27 15:48:38,630][20347] Updated weights for policy 0, policy_version 1466955 (0.0011) [2023-12-27 15:48:38,679][20381] Updated weights for policy 1, policy_version 1465537 (0.0011) [2023-12-27 15:48:38,690][20347] Updated weights for policy 0, policy_version 1466965 (0.0011) [2023-12-27 15:48:38,733][20381] Updated weights for policy 1, policy_version 1465547 (0.0011) [2023-12-27 15:48:39,334][20347] Updated weights for policy 0, policy_version 1466975 (0.0009) [2023-12-27 15:48:39,399][20347] Updated weights for policy 0, policy_version 1466985 (0.0010) [2023-12-27 15:48:39,456][20381] Updated weights for policy 1, policy_version 1465557 (0.0011) [2023-12-27 15:48:39,457][20347] Updated weights for policy 0, policy_version 1466995 (0.0008) [2023-12-27 15:48:39,516][20381] Updated weights for policy 1, policy_version 1465567 (0.0011) [2023-12-27 15:48:39,577][20381] Updated weights for policy 1, policy_version 1465577 (0.0011) [2023-12-27 15:48:40,274][20347] Updated weights for policy 0, policy_version 1467005 (0.0009) [2023-12-27 15:48:40,302][20381] Updated weights for policy 1, policy_version 1465587 (0.0009) [2023-12-27 15:48:40,338][20347] Updated weights for policy 0, policy_version 1467015 (0.0010) [2023-12-27 15:48:40,365][20381] Updated weights for policy 1, policy_version 1465597 (0.0008) [2023-12-27 15:48:40,393][20347] Updated weights for policy 0, policy_version 1467025 (0.0010) [2023-12-27 15:48:40,422][20381] Updated weights for policy 1, policy_version 1465607 (0.0011) [2023-12-27 15:48:41,063][20347] Updated weights for policy 0, policy_version 1467035 (0.0011) [2023-12-27 15:48:41,088][20381] Updated weights for policy 1, policy_version 1465617 (0.0010) [2023-12-27 15:48:41,114][20347] Updated weights for policy 0, policy_version 1467045 (0.0011) [2023-12-27 15:48:41,156][20381] Updated weights for policy 1, policy_version 1465627 (0.0010) [2023-12-27 15:48:41,180][20347] Updated weights for policy 0, policy_version 1467055 (0.0009) [2023-12-27 15:48:41,221][20381] Updated weights for policy 1, policy_version 1465637 (0.0009) [2023-12-27 15:48:41,291][20381] Updated weights for policy 1, policy_version 1465647 (0.0009) [2023-12-27 15:48:41,960][20347] Updated weights for policy 0, policy_version 1467065 (0.0010) [2023-12-27 15:48:42,016][20381] Updated weights for policy 1, policy_version 1465657 (0.0006) [2023-12-27 15:48:42,030][20347] Updated weights for policy 0, policy_version 1467075 (0.0008) [2023-12-27 15:48:42,076][20381] Updated weights for policy 1, policy_version 1465667 (0.0006) [2023-12-27 15:48:42,095][20347] Updated weights for policy 0, policy_version 1467085 (0.0007) [2023-12-27 15:48:42,136][20381] Updated weights for policy 1, policy_version 1465677 (0.0006) [2023-12-27 15:48:42,157][20347] Updated weights for policy 0, policy_version 1467095 (0.0010) [2023-12-27 15:48:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 750895104. Throughput: 0: 10170.8, 1: 10284.1. Samples: 750883608. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:42,363][19170] Avg episode reward: [(0, '715.360'), (1, '892.480')] [2023-12-27 15:48:42,795][20381] Updated weights for policy 1, policy_version 1465687 (0.0007) [2023-12-27 15:48:42,811][20347] Updated weights for policy 0, policy_version 1467105 (0.0008) [2023-12-27 15:48:42,855][20381] Updated weights for policy 1, policy_version 1465697 (0.0006) [2023-12-27 15:48:42,870][20347] Updated weights for policy 0, policy_version 1467115 (0.0007) [2023-12-27 15:48:42,913][20381] Updated weights for policy 1, policy_version 1465707 (0.0005) [2023-12-27 15:48:42,926][20347] Updated weights for policy 0, policy_version 1467125 (0.0007) [2023-12-27 15:48:43,546][20381] Updated weights for policy 1, policy_version 1465717 (0.0005) [2023-12-27 15:48:43,579][20347] Updated weights for policy 0, policy_version 1467135 (0.0006) [2023-12-27 15:48:43,611][20381] Updated weights for policy 1, policy_version 1465727 (0.0006) [2023-12-27 15:48:43,640][20347] Updated weights for policy 0, policy_version 1467145 (0.0007) [2023-12-27 15:48:43,664][20381] Updated weights for policy 1, policy_version 1465737 (0.0008) [2023-12-27 15:48:43,696][20347] Updated weights for policy 0, policy_version 1467155 (0.0006) [2023-12-27 15:48:44,270][20381] Updated weights for policy 1, policy_version 1465747 (0.0006) [2023-12-27 15:48:44,325][20381] Updated weights for policy 1, policy_version 1465757 (0.0005) [2023-12-27 15:48:44,381][20381] Updated weights for policy 1, policy_version 1465767 (0.0005) [2023-12-27 15:48:44,414][20347] Updated weights for policy 0, policy_version 1467165 (0.0008) [2023-12-27 15:48:44,474][20347] Updated weights for policy 0, policy_version 1467175 (0.0008) [2023-12-27 15:48:44,540][20347] Updated weights for policy 0, policy_version 1467185 (0.0010) [2023-12-27 15:48:45,006][20381] Updated weights for policy 1, policy_version 1465777 (0.0005) [2023-12-27 15:48:45,066][20381] Updated weights for policy 1, policy_version 1465787 (0.0007) [2023-12-27 15:48:45,131][20381] Updated weights for policy 1, policy_version 1465797 (0.0006) [2023-12-27 15:48:45,194][20381] Updated weights for policy 1, policy_version 1465807 (0.0006) [2023-12-27 15:48:45,205][20347] Updated weights for policy 0, policy_version 1467195 (0.0009) [2023-12-27 15:48:45,267][20347] Updated weights for policy 0, policy_version 1467205 (0.0007) [2023-12-27 15:48:45,330][20347] Updated weights for policy 0, policy_version 1467215 (0.0007) [2023-12-27 15:48:45,838][20381] Updated weights for policy 1, policy_version 1465817 (0.0008) [2023-12-27 15:48:45,892][20381] Updated weights for policy 1, policy_version 1465827 (0.0007) [2023-12-27 15:48:45,945][20381] Updated weights for policy 1, policy_version 1465837 (0.0006) [2023-12-27 15:48:46,013][20347] Updated weights for policy 0, policy_version 1467225 (0.0008) [2023-12-27 15:48:46,071][20347] Updated weights for policy 0, policy_version 1467235 (0.0007) [2023-12-27 15:48:46,125][20347] Updated weights for policy 0, policy_version 1467245 (0.0008) [2023-12-27 15:48:46,178][20347] Updated weights for policy 0, policy_version 1467255 (0.0007) [2023-12-27 15:48:46,635][20381] Updated weights for policy 1, policy_version 1465847 (0.0007) [2023-12-27 15:48:46,689][20381] Updated weights for policy 1, policy_version 1465857 (0.0007) [2023-12-27 15:48:46,734][20381] Updated weights for policy 1, policy_version 1465867 (0.0006) [2023-12-27 15:48:46,792][20347] Updated weights for policy 0, policy_version 1467265 (0.0008) [2023-12-27 15:48:46,847][20347] Updated weights for policy 0, policy_version 1467275 (0.0006) [2023-12-27 15:48:46,901][20347] Updated weights for policy 0, policy_version 1467285 (0.0005) [2023-12-27 15:48:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 750993408. Throughput: 0: 10131.6, 1: 10315.7. Samples: 751005992. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:47,363][19170] Avg episode reward: [(0, '705.480'), (1, '911.900')] [2023-12-27 15:48:47,387][20381] Updated weights for policy 1, policy_version 1465877 (0.0006) [2023-12-27 15:48:47,440][20381] Updated weights for policy 1, policy_version 1465887 (0.0006) [2023-12-27 15:48:47,506][20381] Updated weights for policy 1, policy_version 1465897 (0.0008) [2023-12-27 15:48:47,563][20347] Updated weights for policy 0, policy_version 1467295 (0.0007) [2023-12-27 15:48:47,616][20347] Updated weights for policy 0, policy_version 1467305 (0.0007) [2023-12-27 15:48:47,676][20347] Updated weights for policy 0, policy_version 1467315 (0.0009) [2023-12-27 15:48:48,182][20381] Updated weights for policy 1, policy_version 1465907 (0.0009) [2023-12-27 15:48:48,242][20381] Updated weights for policy 1, policy_version 1465917 (0.0007) [2023-12-27 15:48:48,302][20381] Updated weights for policy 1, policy_version 1465927 (0.0008) [2023-12-27 15:48:48,414][20347] Updated weights for policy 0, policy_version 1467325 (0.0010) [2023-12-27 15:48:48,469][20347] Updated weights for policy 0, policy_version 1467335 (0.0010) [2023-12-27 15:48:48,524][20347] Updated weights for policy 0, policy_version 1467345 (0.0010) [2023-12-27 15:48:48,991][20381] Updated weights for policy 1, policy_version 1465937 (0.0007) [2023-12-27 15:48:49,051][20381] Updated weights for policy 1, policy_version 1465947 (0.0009) [2023-12-27 15:48:49,109][20381] Updated weights for policy 1, policy_version 1465957 (0.0010) [2023-12-27 15:48:49,170][20381] Updated weights for policy 1, policy_version 1465967 (0.0011) [2023-12-27 15:48:49,220][20347] Updated weights for policy 0, policy_version 1467355 (0.0010) [2023-12-27 15:48:49,289][20347] Updated weights for policy 0, policy_version 1467365 (0.0010) [2023-12-27 15:48:49,352][20347] Updated weights for policy 0, policy_version 1467375 (0.0008) [2023-12-27 15:48:49,859][20381] Updated weights for policy 1, policy_version 1465977 (0.0009) [2023-12-27 15:48:49,915][20381] Updated weights for policy 1, policy_version 1465987 (0.0008) [2023-12-27 15:48:49,977][20381] Updated weights for policy 1, policy_version 1465997 (0.0008) [2023-12-27 15:48:50,027][20347] Updated weights for policy 0, policy_version 1467385 (0.0008) [2023-12-27 15:48:50,077][20347] Updated weights for policy 0, policy_version 1467395 (0.0010) [2023-12-27 15:48:50,136][20347] Updated weights for policy 0, policy_version 1467405 (0.0009) [2023-12-27 15:48:50,194][20347] Updated weights for policy 0, policy_version 1467415 (0.0010) [2023-12-27 15:48:50,740][20381] Updated weights for policy 1, policy_version 1466007 (0.0006) [2023-12-27 15:48:50,800][20381] Updated weights for policy 1, policy_version 1466017 (0.0006) [2023-12-27 15:48:50,867][20381] Updated weights for policy 1, policy_version 1466027 (0.0006) [2023-12-27 15:48:50,903][20347] Updated weights for policy 0, policy_version 1467425 (0.0007) [2023-12-27 15:48:50,966][20347] Updated weights for policy 0, policy_version 1467435 (0.0008) [2023-12-27 15:48:51,028][20347] Updated weights for policy 0, policy_version 1467445 (0.0009) [2023-12-27 15:48:51,510][20381] Updated weights for policy 1, policy_version 1466037 (0.0008) [2023-12-27 15:48:51,566][20381] Updated weights for policy 1, policy_version 1466047 (0.0007) [2023-12-27 15:48:51,630][20381] Updated weights for policy 1, policy_version 1466057 (0.0008) [2023-12-27 15:48:51,778][20347] Updated weights for policy 0, policy_version 1467455 (0.0008) [2023-12-27 15:48:51,834][20347] Updated weights for policy 0, policy_version 1467465 (0.0008) [2023-12-27 15:48:51,894][20347] Updated weights for policy 0, policy_version 1467475 (0.0008) [2023-12-27 15:48:52,341][20381] Updated weights for policy 1, policy_version 1466067 (0.0007) [2023-12-27 15:48:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20346.8, 300 sec: 20493.9). Total num frames: 751091712. Throughput: 0: 10143.7, 1: 10267.3. Samples: 751065848. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:52,363][19170] Avg episode reward: [(0, '617.180'), (1, '941.290')] [2023-12-27 15:48:52,383][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001467480_375750656.pth... [2023-12-27 15:48:52,388][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001466296_375447552.pth [2023-12-27 15:48:52,404][20381] Updated weights for policy 1, policy_version 1466077 (0.0008) [2023-12-27 15:48:52,468][20381] Updated weights for policy 1, policy_version 1466087 (0.0008) [2023-12-27 15:48:52,528][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001466096_375349248.pth... [2023-12-27 15:48:52,533][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001464848_375029760.pth [2023-12-27 15:48:52,620][20347] Updated weights for policy 0, policy_version 1467485 (0.0008) [2023-12-27 15:48:52,677][20347] Updated weights for policy 0, policy_version 1467495 (0.0008) [2023-12-27 15:48:52,737][20347] Updated weights for policy 0, policy_version 1467505 (0.0008) [2023-12-27 15:48:53,184][20381] Updated weights for policy 1, policy_version 1466097 (0.0008) [2023-12-27 15:48:53,231][20381] Updated weights for policy 1, policy_version 1466107 (0.0008) [2023-12-27 15:48:53,280][20381] Updated weights for policy 1, policy_version 1466117 (0.0006) [2023-12-27 15:48:53,328][20381] Updated weights for policy 1, policy_version 1466127 (0.0009) [2023-12-27 15:48:53,330][20347] Updated weights for policy 0, policy_version 1467515 (0.0007) [2023-12-27 15:48:53,385][20347] Updated weights for policy 0, policy_version 1467525 (0.0007) [2023-12-27 15:48:53,450][20347] Updated weights for policy 0, policy_version 1467535 (0.0007) [2023-12-27 15:48:54,035][20381] Updated weights for policy 1, policy_version 1466137 (0.0011) [2023-12-27 15:48:54,072][20347] Updated weights for policy 0, policy_version 1467545 (0.0007) [2023-12-27 15:48:54,087][20381] Updated weights for policy 1, policy_version 1466147 (0.0010) [2023-12-27 15:48:54,131][20347] Updated weights for policy 0, policy_version 1467555 (0.0009) [2023-12-27 15:48:54,137][20381] Updated weights for policy 1, policy_version 1466157 (0.0010) [2023-12-27 15:48:54,188][20347] Updated weights for policy 0, policy_version 1467565 (0.0010) [2023-12-27 15:48:54,244][20347] Updated weights for policy 0, policy_version 1467575 (0.0010) [2023-12-27 15:48:54,849][20381] Updated weights for policy 1, policy_version 1466167 (0.0011) [2023-12-27 15:48:54,898][20381] Updated weights for policy 1, policy_version 1466177 (0.0011) [2023-12-27 15:48:54,907][20347] Updated weights for policy 0, policy_version 1467585 (0.0010) [2023-12-27 15:48:54,953][20381] Updated weights for policy 1, policy_version 1466187 (0.0010) [2023-12-27 15:48:54,956][20347] Updated weights for policy 0, policy_version 1467595 (0.0005) [2023-12-27 15:48:55,012][20347] Updated weights for policy 0, policy_version 1467605 (0.0006) [2023-12-27 15:48:55,606][20347] Updated weights for policy 0, policy_version 1467615 (0.0006) [2023-12-27 15:48:55,641][20381] Updated weights for policy 1, policy_version 1466197 (0.0011) [2023-12-27 15:48:55,663][20347] Updated weights for policy 0, policy_version 1467625 (0.0008) [2023-12-27 15:48:55,686][20381] Updated weights for policy 1, policy_version 1466207 (0.0010) [2023-12-27 15:48:55,720][20347] Updated weights for policy 0, policy_version 1467635 (0.0010) [2023-12-27 15:48:55,735][20381] Updated weights for policy 1, policy_version 1466217 (0.0010) [2023-12-27 15:48:56,363][20347] Updated weights for policy 0, policy_version 1467645 (0.0008) [2023-12-27 15:48:56,418][20347] Updated weights for policy 0, policy_version 1467655 (0.0005) [2023-12-27 15:48:56,430][20381] Updated weights for policy 1, policy_version 1466227 (0.0010) [2023-12-27 15:48:56,471][20347] Updated weights for policy 0, policy_version 1467665 (0.0005) [2023-12-27 15:48:56,487][20381] Updated weights for policy 1, policy_version 1466237 (0.0010) [2023-12-27 15:48:56,537][20381] Updated weights for policy 1, policy_version 1466247 (0.0010) [2023-12-27 15:48:57,068][20381] Updated weights for policy 1, policy_version 1466257 (0.0005) [2023-12-27 15:48:57,098][20347] Updated weights for policy 0, policy_version 1467675 (0.0007) [2023-12-27 15:48:57,120][20381] Updated weights for policy 1, policy_version 1466267 (0.0008) [2023-12-27 15:48:57,148][20347] Updated weights for policy 0, policy_version 1467685 (0.0010) [2023-12-27 15:48:57,177][20381] Updated weights for policy 1, policy_version 1466277 (0.0011) [2023-12-27 15:48:57,198][20347] Updated weights for policy 0, policy_version 1467695 (0.0010) [2023-12-27 15:48:57,226][20381] Updated weights for policy 1, policy_version 1466287 (0.0010) [2023-12-27 15:48:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 751206400. Throughput: 0: 10134.1, 1: 10289.6. Samples: 751188448. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:48:57,363][19170] Avg episode reward: [(0, '666.190'), (1, '863.110')] [2023-12-27 15:48:57,834][20347] Updated weights for policy 0, policy_version 1467705 (0.0010) [2023-12-27 15:48:57,878][20381] Updated weights for policy 1, policy_version 1466297 (0.0007) [2023-12-27 15:48:57,883][20347] Updated weights for policy 0, policy_version 1467715 (0.0008) [2023-12-27 15:48:57,924][20381] Updated weights for policy 1, policy_version 1466307 (0.0010) [2023-12-27 15:48:57,929][20347] Updated weights for policy 0, policy_version 1467725 (0.0005) [2023-12-27 15:48:57,971][20381] Updated weights for policy 1, policy_version 1466317 (0.0010) [2023-12-27 15:48:57,976][20347] Updated weights for policy 0, policy_version 1467735 (0.0006) [2023-12-27 15:48:58,621][20381] Updated weights for policy 1, policy_version 1466327 (0.0007) [2023-12-27 15:48:58,682][20381] Updated weights for policy 1, policy_version 1466337 (0.0008) [2023-12-27 15:48:58,726][20347] Updated weights for policy 0, policy_version 1467745 (0.0008) [2023-12-27 15:48:58,743][20381] Updated weights for policy 1, policy_version 1466347 (0.0008) [2023-12-27 15:48:58,791][20347] Updated weights for policy 0, policy_version 1467755 (0.0007) [2023-12-27 15:48:58,856][20347] Updated weights for policy 0, policy_version 1467765 (0.0010) [2023-12-27 15:48:59,498][20381] Updated weights for policy 1, policy_version 1466357 (0.0007) [2023-12-27 15:48:59,562][20381] Updated weights for policy 1, policy_version 1466367 (0.0009) [2023-12-27 15:48:59,589][20347] Updated weights for policy 0, policy_version 1467775 (0.0007) [2023-12-27 15:48:59,624][20381] Updated weights for policy 1, policy_version 1466377 (0.0007) [2023-12-27 15:48:59,651][20347] Updated weights for policy 0, policy_version 1467785 (0.0007) [2023-12-27 15:48:59,707][20347] Updated weights for policy 0, policy_version 1467795 (0.0008) [2023-12-27 15:49:00,242][20381] Updated weights for policy 1, policy_version 1466387 (0.0007) [2023-12-27 15:49:00,297][20381] Updated weights for policy 1, policy_version 1466397 (0.0008) [2023-12-27 15:49:00,357][20381] Updated weights for policy 1, policy_version 1466407 (0.0008) [2023-12-27 15:49:00,382][20347] Updated weights for policy 0, policy_version 1467805 (0.0006) [2023-12-27 15:49:00,432][20347] Updated weights for policy 0, policy_version 1467815 (0.0006) [2023-12-27 15:49:00,488][20347] Updated weights for policy 0, policy_version 1467825 (0.0008) [2023-12-27 15:49:00,987][20381] Updated weights for policy 1, policy_version 1466417 (0.0006) [2023-12-27 15:49:01,041][20381] Updated weights for policy 1, policy_version 1466427 (0.0006) [2023-12-27 15:49:01,101][20381] Updated weights for policy 1, policy_version 1466437 (0.0008) [2023-12-27 15:49:01,155][20347] Updated weights for policy 0, policy_version 1467835 (0.0008) [2023-12-27 15:49:01,160][20381] Updated weights for policy 1, policy_version 1466447 (0.0008) [2023-12-27 15:49:01,218][20347] Updated weights for policy 0, policy_version 1467845 (0.0006) [2023-12-27 15:49:01,279][20347] Updated weights for policy 0, policy_version 1467855 (0.0005) [2023-12-27 15:49:01,881][20381] Updated weights for policy 1, policy_version 1466457 (0.0010) [2023-12-27 15:49:01,940][20381] Updated weights for policy 1, policy_version 1466467 (0.0010) [2023-12-27 15:49:01,953][20347] Updated weights for policy 0, policy_version 1467865 (0.0006) [2023-12-27 15:49:01,998][20381] Updated weights for policy 1, policy_version 1466477 (0.0009) [2023-12-27 15:49:02,006][20347] Updated weights for policy 0, policy_version 1467875 (0.0008) [2023-12-27 15:49:02,057][20347] Updated weights for policy 0, policy_version 1467885 (0.0008) [2023-12-27 15:49:02,116][20347] Updated weights for policy 0, policy_version 1467895 (0.0006) [2023-12-27 15:49:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 751304704. Throughput: 0: 10196.8, 1: 10306.9. Samples: 751312984. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:02,363][19170] Avg episode reward: [(0, '715.320'), (1, '843.440')] [2023-12-27 15:49:02,720][20381] Updated weights for policy 1, policy_version 1466487 (0.0007) [2023-12-27 15:49:02,778][20381] Updated weights for policy 1, policy_version 1466497 (0.0007) [2023-12-27 15:49:02,813][20347] Updated weights for policy 0, policy_version 1467905 (0.0007) [2023-12-27 15:49:02,833][20381] Updated weights for policy 1, policy_version 1466507 (0.0006) [2023-12-27 15:49:02,876][20347] Updated weights for policy 0, policy_version 1467915 (0.0008) [2023-12-27 15:49:02,940][20347] Updated weights for policy 0, policy_version 1467925 (0.0008) [2023-12-27 15:49:03,528][20347] Updated weights for policy 0, policy_version 1467935 (0.0007) [2023-12-27 15:49:03,560][20381] Updated weights for policy 1, policy_version 1466517 (0.0005) [2023-12-27 15:49:03,580][20347] Updated weights for policy 0, policy_version 1467945 (0.0005) [2023-12-27 15:49:03,608][20381] Updated weights for policy 1, policy_version 1466527 (0.0005) [2023-12-27 15:49:03,635][20347] Updated weights for policy 0, policy_version 1467955 (0.0005) [2023-12-27 15:49:03,666][20381] Updated weights for policy 1, policy_version 1466537 (0.0005) [2023-12-27 15:49:04,386][20347] Updated weights for policy 0, policy_version 1467965 (0.0006) [2023-12-27 15:49:04,419][20381] Updated weights for policy 1, policy_version 1466547 (0.0008) [2023-12-27 15:49:04,442][20347] Updated weights for policy 0, policy_version 1467975 (0.0007) [2023-12-27 15:49:04,473][20381] Updated weights for policy 1, policy_version 1466557 (0.0007) [2023-12-27 15:49:04,500][20347] Updated weights for policy 0, policy_version 1467985 (0.0007) [2023-12-27 15:49:04,531][20381] Updated weights for policy 1, policy_version 1466567 (0.0007) [2023-12-27 15:49:05,177][20347] Updated weights for policy 0, policy_version 1467995 (0.0009) [2023-12-27 15:49:05,233][20381] Updated weights for policy 1, policy_version 1466577 (0.0007) [2023-12-27 15:49:05,235][20347] Updated weights for policy 0, policy_version 1468005 (0.0007) [2023-12-27 15:49:05,287][20381] Updated weights for policy 1, policy_version 1466587 (0.0007) [2023-12-27 15:49:05,289][20347] Updated weights for policy 0, policy_version 1468015 (0.0006) [2023-12-27 15:49:05,337][20381] Updated weights for policy 1, policy_version 1466597 (0.0008) [2023-12-27 15:49:05,396][20381] Updated weights for policy 1, policy_version 1466607 (0.0008) [2023-12-27 15:49:05,975][20347] Updated weights for policy 0, policy_version 1468025 (0.0009) [2023-12-27 15:49:06,033][20347] Updated weights for policy 0, policy_version 1468035 (0.0008) [2023-12-27 15:49:06,061][20381] Updated weights for policy 1, policy_version 1466617 (0.0006) [2023-12-27 15:49:06,088][20347] Updated weights for policy 0, policy_version 1468045 (0.0008) [2023-12-27 15:49:06,121][20381] Updated weights for policy 1, policy_version 1466627 (0.0007) [2023-12-27 15:49:06,150][20347] Updated weights for policy 0, policy_version 1468055 (0.0008) [2023-12-27 15:49:06,177][20381] Updated weights for policy 1, policy_version 1466637 (0.0007) [2023-12-27 15:49:06,823][20347] Updated weights for policy 0, policy_version 1468065 (0.0006) [2023-12-27 15:49:06,880][20347] Updated weights for policy 0, policy_version 1468075 (0.0006) [2023-12-27 15:49:06,922][20381] Updated weights for policy 1, policy_version 1466647 (0.0008) [2023-12-27 15:49:06,937][20347] Updated weights for policy 0, policy_version 1468085 (0.0006) [2023-12-27 15:49:06,984][20381] Updated weights for policy 1, policy_version 1466657 (0.0008) [2023-12-27 15:49:07,044][20381] Updated weights for policy 1, policy_version 1466667 (0.0008) [2023-12-27 15:49:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 751403008. Throughput: 0: 10232.2, 1: 10315.6. Samples: 751373248. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:07,364][19170] Avg episode reward: [(0, '784.200'), (1, '862.950')] [2023-12-27 15:49:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001466672_375496704.pth... [2023-12-27 15:49:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001468088_375906304.pth... [2023-12-27 15:49:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001465488_375193600.pth [2023-12-27 15:49:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001466904_375603200.pth [2023-12-27 15:49:07,501][20347] Updated weights for policy 0, policy_version 1468095 (0.0005) [2023-12-27 15:49:07,560][20347] Updated weights for policy 0, policy_version 1468105 (0.0005) [2023-12-27 15:49:07,613][20347] Updated weights for policy 0, policy_version 1468115 (0.0011) [2023-12-27 15:49:07,778][20381] Updated weights for policy 1, policy_version 1466677 (0.0008) [2023-12-27 15:49:07,828][20381] Updated weights for policy 1, policy_version 1466687 (0.0008) [2023-12-27 15:49:07,891][20381] Updated weights for policy 1, policy_version 1466697 (0.0008) [2023-12-27 15:49:08,218][20347] Updated weights for policy 0, policy_version 1468125 (0.0008) [2023-12-27 15:49:08,265][20347] Updated weights for policy 0, policy_version 1468135 (0.0005) [2023-12-27 15:49:08,326][20347] Updated weights for policy 0, policy_version 1468145 (0.0006) [2023-12-27 15:49:08,564][20381] Updated weights for policy 1, policy_version 1466707 (0.0007) [2023-12-27 15:49:08,624][20381] Updated weights for policy 1, policy_version 1466717 (0.0007) [2023-12-27 15:49:08,680][20381] Updated weights for policy 1, policy_version 1466727 (0.0007) [2023-12-27 15:49:08,958][20347] Updated weights for policy 0, policy_version 1468155 (0.0009) [2023-12-27 15:49:09,013][20347] Updated weights for policy 0, policy_version 1468165 (0.0010) [2023-12-27 15:49:09,076][20347] Updated weights for policy 0, policy_version 1468175 (0.0011) [2023-12-27 15:49:09,350][20381] Updated weights for policy 1, policy_version 1466737 (0.0008) [2023-12-27 15:49:09,423][20381] Updated weights for policy 1, policy_version 1466747 (0.0009) [2023-12-27 15:49:09,486][20381] Updated weights for policy 1, policy_version 1466757 (0.0006) [2023-12-27 15:49:09,547][20381] Updated weights for policy 1, policy_version 1466767 (0.0008) [2023-12-27 15:49:09,790][20347] Updated weights for policy 0, policy_version 1468185 (0.0010) [2023-12-27 15:49:09,859][20347] Updated weights for policy 0, policy_version 1468195 (0.0007) [2023-12-27 15:49:09,924][20347] Updated weights for policy 0, policy_version 1468205 (0.0008) [2023-12-27 15:49:09,994][20347] Updated weights for policy 0, policy_version 1468215 (0.0008) [2023-12-27 15:49:10,176][20381] Updated weights for policy 1, policy_version 1466777 (0.0008) [2023-12-27 15:49:10,231][20381] Updated weights for policy 1, policy_version 1466787 (0.0008) [2023-12-27 15:49:10,284][20381] Updated weights for policy 1, policy_version 1466797 (0.0007) [2023-12-27 15:49:10,664][20347] Updated weights for policy 0, policy_version 1468225 (0.0007) [2023-12-27 15:49:10,723][20347] Updated weights for policy 0, policy_version 1468235 (0.0008) [2023-12-27 15:49:10,778][20347] Updated weights for policy 0, policy_version 1468245 (0.0008) [2023-12-27 15:49:11,001][20381] Updated weights for policy 1, policy_version 1466807 (0.0009) [2023-12-27 15:49:11,064][20381] Updated weights for policy 1, policy_version 1466817 (0.0008) [2023-12-27 15:49:11,127][20381] Updated weights for policy 1, policy_version 1466827 (0.0007) [2023-12-27 15:49:11,480][20347] Updated weights for policy 0, policy_version 1468255 (0.0007) [2023-12-27 15:49:11,546][20347] Updated weights for policy 0, policy_version 1468265 (0.0007) [2023-12-27 15:49:11,610][20347] Updated weights for policy 0, policy_version 1468275 (0.0008) [2023-12-27 15:49:11,922][20381] Updated weights for policy 1, policy_version 1466837 (0.0010) [2023-12-27 15:49:11,975][20381] Updated weights for policy 1, policy_version 1466847 (0.0010) [2023-12-27 15:49:12,034][20381] Updated weights for policy 1, policy_version 1466857 (0.0005) [2023-12-27 15:49:12,312][20347] Updated weights for policy 0, policy_version 1468285 (0.0007) [2023-12-27 15:49:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 751501312. Throughput: 0: 10289.5, 1: 10208.9. Samples: 751494620. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:12,363][19170] Avg episode reward: [(0, '715.190'), (1, '853.380')] [2023-12-27 15:49:12,377][20347] Updated weights for policy 0, policy_version 1468295 (0.0008) [2023-12-27 15:49:12,431][20347] Updated weights for policy 0, policy_version 1468305 (0.0006) [2023-12-27 15:49:12,746][20381] Updated weights for policy 1, policy_version 1466867 (0.0009) [2023-12-27 15:49:12,810][20381] Updated weights for policy 1, policy_version 1466877 (0.0007) [2023-12-27 15:49:12,869][20381] Updated weights for policy 1, policy_version 1466887 (0.0008) [2023-12-27 15:49:13,186][20347] Updated weights for policy 0, policy_version 1468315 (0.0008) [2023-12-27 15:49:13,243][20347] Updated weights for policy 0, policy_version 1468325 (0.0006) [2023-12-27 15:49:13,300][20347] Updated weights for policy 0, policy_version 1468335 (0.0006) [2023-12-27 15:49:13,494][20381] Updated weights for policy 1, policy_version 1466897 (0.0006) [2023-12-27 15:49:13,545][20381] Updated weights for policy 1, policy_version 1466907 (0.0006) [2023-12-27 15:49:13,591][20381] Updated weights for policy 1, policy_version 1466917 (0.0008) [2023-12-27 15:49:13,639][20381] Updated weights for policy 1, policy_version 1466927 (0.0007) [2023-12-27 15:49:13,976][20347] Updated weights for policy 0, policy_version 1468345 (0.0007) [2023-12-27 15:49:14,038][20347] Updated weights for policy 0, policy_version 1468355 (0.0008) [2023-12-27 15:49:14,096][20347] Updated weights for policy 0, policy_version 1468365 (0.0008) [2023-12-27 15:49:14,156][20347] Updated weights for policy 0, policy_version 1468375 (0.0008) [2023-12-27 15:49:14,227][20381] Updated weights for policy 1, policy_version 1466937 (0.0007) [2023-12-27 15:49:14,289][20381] Updated weights for policy 1, policy_version 1466947 (0.0006) [2023-12-27 15:49:14,345][20381] Updated weights for policy 1, policy_version 1466957 (0.0007) [2023-12-27 15:49:14,850][20347] Updated weights for policy 0, policy_version 1468385 (0.0010) [2023-12-27 15:49:14,900][20347] Updated weights for policy 0, policy_version 1468395 (0.0010) [2023-12-27 15:49:14,957][20347] Updated weights for policy 0, policy_version 1468405 (0.0011) [2023-12-27 15:49:14,993][20381] Updated weights for policy 1, policy_version 1466967 (0.0007) [2023-12-27 15:49:15,055][20381] Updated weights for policy 1, policy_version 1466977 (0.0008) [2023-12-27 15:49:15,118][20381] Updated weights for policy 1, policy_version 1466987 (0.0008) [2023-12-27 15:49:15,625][20347] Updated weights for policy 0, policy_version 1468415 (0.0007) [2023-12-27 15:49:15,681][20347] Updated weights for policy 0, policy_version 1468425 (0.0005) [2023-12-27 15:49:15,690][20381] Updated weights for policy 1, policy_version 1466997 (0.0007) [2023-12-27 15:49:15,733][20347] Updated weights for policy 0, policy_version 1468435 (0.0005) [2023-12-27 15:49:15,741][20381] Updated weights for policy 1, policy_version 1467007 (0.0008) [2023-12-27 15:49:15,791][20381] Updated weights for policy 1, policy_version 1467017 (0.0005) [2023-12-27 15:49:16,359][20347] Updated weights for policy 0, policy_version 1468445 (0.0008) [2023-12-27 15:49:16,413][20347] Updated weights for policy 0, policy_version 1468455 (0.0010) [2023-12-27 15:49:16,466][20381] Updated weights for policy 1, policy_version 1467027 (0.0006) [2023-12-27 15:49:16,470][20347] Updated weights for policy 0, policy_version 1468465 (0.0010) [2023-12-27 15:49:16,522][20381] Updated weights for policy 1, policy_version 1467037 (0.0005) [2023-12-27 15:49:16,578][20381] Updated weights for policy 1, policy_version 1467047 (0.0005) [2023-12-27 15:49:17,148][20347] Updated weights for policy 0, policy_version 1468475 (0.0010) [2023-12-27 15:49:17,208][20347] Updated weights for policy 0, policy_version 1468485 (0.0010) [2023-12-27 15:49:17,210][20381] Updated weights for policy 1, policy_version 1467057 (0.0007) [2023-12-27 15:49:17,261][20381] Updated weights for policy 1, policy_version 1467067 (0.0010) [2023-12-27 15:49:17,275][20347] Updated weights for policy 0, policy_version 1468495 (0.0010) [2023-12-27 15:49:17,317][20381] Updated weights for policy 1, policy_version 1467077 (0.0005) [2023-12-27 15:49:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 751607808. Throughput: 0: 10260.8, 1: 10193.9. Samples: 751618616. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:17,363][19170] Avg episode reward: [(0, '705.470'), (1, '853.860')] [2023-12-27 15:49:17,375][20381] Updated weights for policy 1, policy_version 1467087 (0.0005) [2023-12-27 15:49:17,941][20347] Updated weights for policy 0, policy_version 1468505 (0.0010) [2023-12-27 15:49:17,963][20381] Updated weights for policy 1, policy_version 1467097 (0.0010) [2023-12-27 15:49:18,004][20347] Updated weights for policy 0, policy_version 1468515 (0.0010) [2023-12-27 15:49:18,019][20381] Updated weights for policy 1, policy_version 1467107 (0.0006) [2023-12-27 15:49:18,065][20347] Updated weights for policy 0, policy_version 1468525 (0.0010) [2023-12-27 15:49:18,077][20381] Updated weights for policy 1, policy_version 1467117 (0.0009) [2023-12-27 15:49:18,126][20347] Updated weights for policy 0, policy_version 1468535 (0.0011) [2023-12-27 15:49:18,744][20347] Updated weights for policy 0, policy_version 1468545 (0.0009) [2023-12-27 15:49:18,761][20381] Updated weights for policy 1, policy_version 1467127 (0.0008) [2023-12-27 15:49:18,800][20347] Updated weights for policy 0, policy_version 1468555 (0.0007) [2023-12-27 15:49:18,826][20381] Updated weights for policy 1, policy_version 1467137 (0.0010) [2023-12-27 15:49:18,859][20347] Updated weights for policy 0, policy_version 1468565 (0.0007) [2023-12-27 15:49:18,890][20381] Updated weights for policy 1, policy_version 1467147 (0.0007) [2023-12-27 15:49:19,518][20347] Updated weights for policy 0, policy_version 1468575 (0.0008) [2023-12-27 15:49:19,567][20381] Updated weights for policy 1, policy_version 1467157 (0.0009) [2023-12-27 15:49:19,580][20347] Updated weights for policy 0, policy_version 1468585 (0.0006) [2023-12-27 15:49:19,627][20381] Updated weights for policy 1, policy_version 1467167 (0.0006) [2023-12-27 15:49:19,645][20347] Updated weights for policy 0, policy_version 1468595 (0.0006) [2023-12-27 15:49:19,678][20381] Updated weights for policy 1, policy_version 1467177 (0.0006) [2023-12-27 15:49:20,327][20347] Updated weights for policy 0, policy_version 1468605 (0.0008) [2023-12-27 15:49:20,381][20347] Updated weights for policy 0, policy_version 1468615 (0.0007) [2023-12-27 15:49:20,406][20381] Updated weights for policy 1, policy_version 1467187 (0.0007) [2023-12-27 15:49:20,442][20347] Updated weights for policy 0, policy_version 1468625 (0.0006) [2023-12-27 15:49:20,466][20381] Updated weights for policy 1, policy_version 1467197 (0.0006) [2023-12-27 15:49:20,520][20381] Updated weights for policy 1, policy_version 1467207 (0.0007) [2023-12-27 15:49:21,125][20381] Updated weights for policy 1, policy_version 1467217 (0.0007) [2023-12-27 15:49:21,183][20381] Updated weights for policy 1, policy_version 1467227 (0.0008) [2023-12-27 15:49:21,216][20347] Updated weights for policy 0, policy_version 1468635 (0.0006) [2023-12-27 15:49:21,248][20381] Updated weights for policy 1, policy_version 1467237 (0.0007) [2023-12-27 15:49:21,285][20347] Updated weights for policy 0, policy_version 1468645 (0.0007) [2023-12-27 15:49:21,311][20381] Updated weights for policy 1, policy_version 1467247 (0.0007) [2023-12-27 15:49:21,346][20347] Updated weights for policy 0, policy_version 1468655 (0.0008) [2023-12-27 15:49:22,052][20381] Updated weights for policy 1, policy_version 1467257 (0.0007) [2023-12-27 15:49:22,088][20347] Updated weights for policy 0, policy_version 1468665 (0.0008) [2023-12-27 15:49:22,113][20381] Updated weights for policy 1, policy_version 1467267 (0.0008) [2023-12-27 15:49:22,146][20347] Updated weights for policy 0, policy_version 1468675 (0.0006) [2023-12-27 15:49:22,177][20381] Updated weights for policy 1, policy_version 1467277 (0.0007) [2023-12-27 15:49:22,194][20347] Updated weights for policy 0, policy_version 1468685 (0.0008) [2023-12-27 15:49:22,256][20347] Updated weights for policy 0, policy_version 1468695 (0.0006) [2023-12-27 15:49:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 751714304. Throughput: 0: 10251.2, 1: 10182.6. Samples: 751680732. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:22,363][19170] Avg episode reward: [(0, '754.570'), (1, '932.130')] [2023-12-27 15:49:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001467280_375652352.pth... [2023-12-27 15:49:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001468696_376061952.pth... [2023-12-27 15:49:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001466096_375349248.pth [2023-12-27 15:49:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001467480_375750656.pth [2023-12-27 15:49:22,811][20381] Updated weights for policy 1, policy_version 1467287 (0.0009) [2023-12-27 15:49:22,858][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000006 [2023-12-27 15:49:22,917][20347] Updated weights for policy 0, policy_version 1468705 (0.0008) [2023-12-27 15:49:22,982][20347] Updated weights for policy 0, policy_version 1468715 (0.0006) [2023-12-27 15:49:23,050][20347] Updated weights for policy 0, policy_version 1468725 (0.0006) [2023-12-27 15:49:23,506][20381] Updated weights for policy 1, policy_version 1467297 (0.0006) [2023-12-27 15:49:23,563][20381] Updated weights for policy 1, policy_version 1467307 (0.0011) [2023-12-27 15:49:23,622][20381] Updated weights for policy 1, policy_version 1467317 (0.0011) [2023-12-27 15:49:23,677][20381] Updated weights for policy 1, policy_version 1467327 (0.0009) [2023-12-27 15:49:23,681][20347] Updated weights for policy 0, policy_version 1468735 (0.0006) [2023-12-27 15:49:23,738][20347] Updated weights for policy 0, policy_version 1468745 (0.0006) [2023-12-27 15:49:23,786][20347] Updated weights for policy 0, policy_version 1468755 (0.0008) [2023-12-27 15:49:24,290][20381] Updated weights for policy 1, policy_version 1467337 (0.0010) [2023-12-27 15:49:24,349][20381] Updated weights for policy 1, policy_version 1467347 (0.0011) [2023-12-27 15:49:24,389][20347] Updated weights for policy 0, policy_version 1468765 (0.0007) [2023-12-27 15:49:24,417][20381] Updated weights for policy 1, policy_version 1467357 (0.0010) [2023-12-27 15:49:24,450][20347] Updated weights for policy 0, policy_version 1468775 (0.0007) [2023-12-27 15:49:24,507][20347] Updated weights for policy 0, policy_version 1468785 (0.0005) [2023-12-27 15:49:25,003][20381] Updated weights for policy 1, policy_version 1467367 (0.0010) [2023-12-27 15:49:25,047][20347] Updated weights for policy 0, policy_version 1468795 (0.0005) [2023-12-27 15:49:25,059][20381] Updated weights for policy 1, policy_version 1467377 (0.0011) [2023-12-27 15:49:25,103][20347] Updated weights for policy 0, policy_version 1468805 (0.0005) [2023-12-27 15:49:25,107][20381] Updated weights for policy 1, policy_version 1467387 (0.0010) [2023-12-27 15:49:25,160][20347] Updated weights for policy 0, policy_version 1468815 (0.0005) [2023-12-27 15:49:25,705][20347] Updated weights for policy 0, policy_version 1468825 (0.0005) [2023-12-27 15:49:25,754][20347] Updated weights for policy 0, policy_version 1468835 (0.0005) [2023-12-27 15:49:25,762][20381] Updated weights for policy 1, policy_version 1467397 (0.0009) [2023-12-27 15:49:25,800][20347] Updated weights for policy 0, policy_version 1468845 (0.0005) [2023-12-27 15:49:25,819][20381] Updated weights for policy 1, policy_version 1467407 (0.0008) [2023-12-27 15:49:25,858][20347] Updated weights for policy 0, policy_version 1468855 (0.0008) [2023-12-27 15:49:25,880][20381] Updated weights for policy 1, policy_version 1467417 (0.0006) [2023-12-27 15:49:26,420][20347] Updated weights for policy 0, policy_version 1468865 (0.0006) [2023-12-27 15:49:26,432][20381] Updated weights for policy 1, policy_version 1467427 (0.0006) [2023-12-27 15:49:26,468][20347] Updated weights for policy 0, policy_version 1468875 (0.0005) [2023-12-27 15:49:26,492][20381] Updated weights for policy 1, policy_version 1467437 (0.0006) [2023-12-27 15:49:26,523][20347] Updated weights for policy 0, policy_version 1468885 (0.0005) [2023-12-27 15:49:26,541][20381] Updated weights for policy 1, policy_version 1467447 (0.0008) [2023-12-27 15:49:27,090][20347] Updated weights for policy 0, policy_version 1468895 (0.0005) [2023-12-27 15:49:27,143][20347] Updated weights for policy 0, policy_version 1468905 (0.0005) [2023-12-27 15:49:27,202][20347] Updated weights for policy 0, policy_version 1468915 (0.0007) [2023-12-27 15:49:27,247][20381] Updated weights for policy 1, policy_version 1467457 (0.0008) [2023-12-27 15:49:27,306][20381] Updated weights for policy 1, policy_version 1467467 (0.0008) [2023-12-27 15:49:27,358][20381] Updated weights for policy 1, policy_version 1467477 (0.0007) [2023-12-27 15:49:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 751820800. Throughput: 0: 10283.3, 1: 10286.4. Samples: 751809248. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:27,363][19170] Avg episode reward: [(0, '675.660'), (1, '970.770')] [2023-12-27 15:49:27,410][20381] Updated weights for policy 1, policy_version 1467487 (0.0007) [2023-12-27 15:49:27,793][20347] Updated weights for policy 0, policy_version 1468925 (0.0006) [2023-12-27 15:49:27,858][20347] Updated weights for policy 0, policy_version 1468935 (0.0005) [2023-12-27 15:49:27,921][20347] Updated weights for policy 0, policy_version 1468945 (0.0008) [2023-12-27 15:49:28,088][20381] Updated weights for policy 1, policy_version 1467497 (0.0007) [2023-12-27 15:49:28,134][20381] Updated weights for policy 1, policy_version 1467507 (0.0007) [2023-12-27 15:49:28,186][20381] Updated weights for policy 1, policy_version 1467517 (0.0008) [2023-12-27 15:49:28,550][20347] Updated weights for policy 0, policy_version 1468955 (0.0008) [2023-12-27 15:49:28,606][20347] Updated weights for policy 0, policy_version 1468965 (0.0008) [2023-12-27 15:49:28,663][20347] Updated weights for policy 0, policy_version 1468975 (0.0008) [2023-12-27 15:49:28,929][20381] Updated weights for policy 1, policy_version 1467527 (0.0008) [2023-12-27 15:49:28,981][20381] Updated weights for policy 1, policy_version 1467537 (0.0008) [2023-12-27 15:49:29,029][20381] Updated weights for policy 1, policy_version 1467547 (0.0008) [2023-12-27 15:49:29,297][20347] Updated weights for policy 0, policy_version 1468985 (0.0007) [2023-12-27 15:49:29,365][20347] Updated weights for policy 0, policy_version 1468995 (0.0009) [2023-12-27 15:49:29,430][20347] Updated weights for policy 0, policy_version 1469005 (0.0008) [2023-12-27 15:49:29,495][20347] Updated weights for policy 0, policy_version 1469015 (0.0008) [2023-12-27 15:49:29,767][20381] Updated weights for policy 1, policy_version 1467557 (0.0009) [2023-12-27 15:49:29,826][20381] Updated weights for policy 1, policy_version 1467567 (0.0007) [2023-12-27 15:49:29,885][20381] Updated weights for policy 1, policy_version 1467577 (0.0007) [2023-12-27 15:49:30,154][20347] Updated weights for policy 0, policy_version 1469025 (0.0006) [2023-12-27 15:49:30,210][20347] Updated weights for policy 0, policy_version 1469035 (0.0008) [2023-12-27 15:49:30,267][20347] Updated weights for policy 0, policy_version 1469045 (0.0008) [2023-12-27 15:49:30,597][20381] Updated weights for policy 1, policy_version 1467587 (0.0006) [2023-12-27 15:49:30,649][20381] Updated weights for policy 1, policy_version 1467597 (0.0008) [2023-12-27 15:49:30,705][20381] Updated weights for policy 1, policy_version 1467607 (0.0007) [2023-12-27 15:49:30,930][20347] Updated weights for policy 0, policy_version 1469055 (0.0007) [2023-12-27 15:49:30,985][20347] Updated weights for policy 0, policy_version 1469065 (0.0005) [2023-12-27 15:49:31,050][20347] Updated weights for policy 0, policy_version 1469075 (0.0006) [2023-12-27 15:49:31,443][20381] Updated weights for policy 1, policy_version 1467617 (0.0008) [2023-12-27 15:49:31,506][20381] Updated weights for policy 1, policy_version 1467627 (0.0006) [2023-12-27 15:49:31,572][20381] Updated weights for policy 1, policy_version 1467637 (0.0005) [2023-12-27 15:49:31,637][20381] Updated weights for policy 1, policy_version 1467647 (0.0008) [2023-12-27 15:49:31,710][20347] Updated weights for policy 0, policy_version 1469085 (0.0007) [2023-12-27 15:49:31,776][20347] Updated weights for policy 0, policy_version 1469095 (0.0008) [2023-12-27 15:49:31,833][20347] Updated weights for policy 0, policy_version 1469105 (0.0008) [2023-12-27 15:49:32,245][20381] Updated weights for policy 1, policy_version 1467657 (0.0008) [2023-12-27 15:49:32,303][20381] Updated weights for policy 1, policy_version 1467667 (0.0008) [2023-12-27 15:49:32,362][20381] Updated weights for policy 1, policy_version 1467677 (0.0009) [2023-12-27 15:49:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 751919104. Throughput: 0: 10385.4, 1: 10236.7. Samples: 751933988. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:32,363][19170] Avg episode reward: [(0, '734.870'), (1, '921.930')] [2023-12-27 15:49:32,510][20347] Updated weights for policy 0, policy_version 1469115 (0.0007) [2023-12-27 15:49:32,559][20347] Updated weights for policy 0, policy_version 1469125 (0.0008) [2023-12-27 15:49:32,611][20347] Updated weights for policy 0, policy_version 1469135 (0.0008) [2023-12-27 15:49:33,078][20381] Updated weights for policy 1, policy_version 1467687 (0.0008) [2023-12-27 15:49:33,127][20381] Updated weights for policy 1, policy_version 1467697 (0.0008) [2023-12-27 15:49:33,181][20381] Updated weights for policy 1, policy_version 1467707 (0.0005) [2023-12-27 15:49:33,316][20347] Updated weights for policy 0, policy_version 1469145 (0.0007) [2023-12-27 15:49:33,369][20347] Updated weights for policy 0, policy_version 1469155 (0.0008) [2023-12-27 15:49:33,423][20347] Updated weights for policy 0, policy_version 1469165 (0.0008) [2023-12-27 15:49:33,484][20347] Updated weights for policy 0, policy_version 1469175 (0.0007) [2023-12-27 15:49:33,871][20381] Updated weights for policy 1, policy_version 1467717 (0.0008) [2023-12-27 15:49:33,932][20381] Updated weights for policy 1, policy_version 1467727 (0.0008) [2023-12-27 15:49:33,993][20381] Updated weights for policy 1, policy_version 1467737 (0.0009) [2023-12-27 15:49:34,166][20347] Updated weights for policy 0, policy_version 1469185 (0.0007) [2023-12-27 15:49:34,224][20347] Updated weights for policy 0, policy_version 1469195 (0.0008) [2023-12-27 15:49:34,285][20347] Updated weights for policy 0, policy_version 1469205 (0.0007) [2023-12-27 15:49:34,740][20381] Updated weights for policy 1, policy_version 1467747 (0.0009) [2023-12-27 15:49:34,800][20381] Updated weights for policy 1, policy_version 1467757 (0.0008) [2023-12-27 15:49:34,865][20381] Updated weights for policy 1, policy_version 1467767 (0.0008) [2023-12-27 15:49:34,917][20347] Updated weights for policy 0, policy_version 1469215 (0.0006) [2023-12-27 15:49:34,972][20347] Updated weights for policy 0, policy_version 1469225 (0.0005) [2023-12-27 15:49:35,034][20347] Updated weights for policy 0, policy_version 1469235 (0.0008) [2023-12-27 15:49:35,581][20381] Updated weights for policy 1, policy_version 1467777 (0.0007) [2023-12-27 15:49:35,613][20347] Updated weights for policy 0, policy_version 1469245 (0.0008) [2023-12-27 15:49:35,631][20381] Updated weights for policy 1, policy_version 1467787 (0.0007) [2023-12-27 15:49:35,669][20347] Updated weights for policy 0, policy_version 1469255 (0.0010) [2023-12-27 15:49:35,682][20381] Updated weights for policy 1, policy_version 1467797 (0.0005) [2023-12-27 15:49:35,734][20381] Updated weights for policy 1, policy_version 1467807 (0.0007) [2023-12-27 15:49:35,736][20347] Updated weights for policy 0, policy_version 1469265 (0.0007) [2023-12-27 15:49:36,347][20347] Updated weights for policy 0, policy_version 1469275 (0.0005) [2023-12-27 15:49:36,410][20347] Updated weights for policy 0, policy_version 1469285 (0.0009) [2023-12-27 15:49:36,474][20347] Updated weights for policy 0, policy_version 1469295 (0.0008) [2023-12-27 15:49:36,492][20381] Updated weights for policy 1, policy_version 1467817 (0.0006) [2023-12-27 15:49:36,549][20381] Updated weights for policy 1, policy_version 1467827 (0.0008) [2023-12-27 15:49:36,605][20381] Updated weights for policy 1, policy_version 1467837 (0.0008) [2023-12-27 15:49:37,158][20347] Updated weights for policy 0, policy_version 1469305 (0.0011) [2023-12-27 15:49:37,204][20347] Updated weights for policy 0, policy_version 1469315 (0.0010) [2023-12-27 15:49:37,253][20347] Updated weights for policy 0, policy_version 1469325 (0.0010) [2023-12-27 15:49:37,292][20381] Updated weights for policy 1, policy_version 1467847 (0.0008) [2023-12-27 15:49:37,308][20347] Updated weights for policy 0, policy_version 1469335 (0.0010) [2023-12-27 15:49:37,349][20381] Updated weights for policy 1, policy_version 1467857 (0.0007) [2023-12-27 15:49:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 752025600. Throughput: 0: 10407.5, 1: 10218.5. Samples: 751994016. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:37,363][19170] Avg episode reward: [(0, '793.880'), (1, '902.280')] [2023-12-27 15:49:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001469336_376225792.pth... [2023-12-27 15:49:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001468088_375906304.pth [2023-12-27 15:49:37,408][20381] Updated weights for policy 1, policy_version 1467867 (0.0007) [2023-12-27 15:49:37,430][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001467872_375808000.pth... [2023-12-27 15:49:37,434][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001466672_375496704.pth [2023-12-27 15:49:37,970][20347] Updated weights for policy 0, policy_version 1469345 (0.0009) [2023-12-27 15:49:37,989][20291] KL-divergence is very high: 120.0139 [2023-12-27 15:49:38,007][20291] KL-divergence is very high: 158.7896 [2023-12-27 15:49:38,032][20347] Updated weights for policy 0, policy_version 1469355 (0.0010) [2023-12-27 15:49:38,036][20291] KL-divergence is very high: 144.3582 [2023-12-27 15:49:38,051][20291] KL-divergence is very high: 126.7194 [2023-12-27 15:49:38,080][20347] Updated weights for policy 0, policy_version 1469365 (0.0010) [2023-12-27 15:49:38,113][20381] Updated weights for policy 1, policy_version 1467877 (0.0008) [2023-12-27 15:49:38,173][20381] Updated weights for policy 1, policy_version 1467887 (0.0006) [2023-12-27 15:49:38,229][20381] Updated weights for policy 1, policy_version 1467897 (0.0007) [2023-12-27 15:49:38,763][20347] Updated weights for policy 0, policy_version 1469375 (0.0007) [2023-12-27 15:49:38,820][20347] Updated weights for policy 0, policy_version 1469385 (0.0008) [2023-12-27 15:49:38,881][20347] Updated weights for policy 0, policy_version 1469395 (0.0008) [2023-12-27 15:49:38,956][20381] Updated weights for policy 1, policy_version 1467907 (0.0008) [2023-12-27 15:49:39,014][20381] Updated weights for policy 1, policy_version 1467917 (0.0008) [2023-12-27 15:49:39,074][20381] Updated weights for policy 1, policy_version 1467927 (0.0008) [2023-12-27 15:49:39,644][20347] Updated weights for policy 0, policy_version 1469405 (0.0008) [2023-12-27 15:49:39,702][20347] Updated weights for policy 0, policy_version 1469415 (0.0008) [2023-12-27 15:49:39,713][20381] Updated weights for policy 1, policy_version 1467937 (0.0006) [2023-12-27 15:49:39,760][20347] Updated weights for policy 0, policy_version 1469425 (0.0006) [2023-12-27 15:49:39,774][20381] Updated weights for policy 1, policy_version 1467947 (0.0006) [2023-12-27 15:49:39,837][20381] Updated weights for policy 1, policy_version 1467957 (0.0006) [2023-12-27 15:49:39,908][20381] Updated weights for policy 1, policy_version 1467967 (0.0009) [2023-12-27 15:49:40,549][20347] Updated weights for policy 0, policy_version 1469435 (0.0006) [2023-12-27 15:49:40,559][20291] KL-divergence is very high: 118.0472 [2023-12-27 15:49:40,598][20381] Updated weights for policy 1, policy_version 1467977 (0.0006) [2023-12-27 15:49:40,606][20291] KL-divergence is very high: 126.4302 [2023-12-27 15:49:40,606][20347] Updated weights for policy 0, policy_version 1469445 (0.0008) [2023-12-27 15:49:40,646][20291] KL-divergence is very high: 116.3737 [2023-12-27 15:49:40,650][20381] Updated weights for policy 1, policy_version 1467987 (0.0005) [2023-12-27 15:49:40,660][20347] Updated weights for policy 0, policy_version 1469455 (0.0008) [2023-12-27 15:49:40,698][20381] Updated weights for policy 1, policy_version 1467997 (0.0005) [2023-12-27 15:49:41,301][20381] Updated weights for policy 1, policy_version 1468007 (0.0008) [2023-12-27 15:49:41,363][20381] Updated weights for policy 1, policy_version 1468017 (0.0008) [2023-12-27 15:49:41,414][20347] Updated weights for policy 0, policy_version 1469465 (0.0008) [2023-12-27 15:49:41,428][20381] Updated weights for policy 1, policy_version 1468027 (0.0007) [2023-12-27 15:49:41,473][20347] Updated weights for policy 0, policy_version 1469475 (0.0006) [2023-12-27 15:49:41,526][20347] Updated weights for policy 0, policy_version 1469485 (0.0005) [2023-12-27 15:49:41,583][20347] Updated weights for policy 0, policy_version 1469495 (0.0009) [2023-12-27 15:49:42,156][20381] Updated weights for policy 1, policy_version 1468037 (0.0008) [2023-12-27 15:49:42,219][20381] Updated weights for policy 1, policy_version 1468047 (0.0006) [2023-12-27 15:49:42,280][20381] Updated weights for policy 1, policy_version 1468057 (0.0008) [2023-12-27 15:49:42,290][20347] Updated weights for policy 0, policy_version 1469505 (0.0007) [2023-12-27 15:49:42,353][20347] Updated weights for policy 0, policy_version 1469515 (0.0008) [2023-12-27 15:49:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 752123904. Throughput: 0: 10383.3, 1: 10218.5. Samples: 752115528. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:42,365][19170] Avg episode reward: [(0, '685.570'), (1, '912.020')] [2023-12-27 15:49:42,413][20347] Updated weights for policy 0, policy_version 1469525 (0.0008) [2023-12-27 15:49:42,941][20381] Updated weights for policy 1, policy_version 1468067 (0.0007) [2023-12-27 15:49:42,999][20381] Updated weights for policy 1, policy_version 1468077 (0.0007) [2023-12-27 15:49:43,049][20381] Updated weights for policy 1, policy_version 1468087 (0.0006) [2023-12-27 15:49:43,061][20347] Updated weights for policy 0, policy_version 1469535 (0.0008) [2023-12-27 15:49:43,122][20347] Updated weights for policy 0, policy_version 1469545 (0.0006) [2023-12-27 15:49:43,182][20347] Updated weights for policy 0, policy_version 1469555 (0.0007) [2023-12-27 15:49:43,733][20381] Updated weights for policy 1, policy_version 1468097 (0.0006) [2023-12-27 15:49:43,773][20347] Updated weights for policy 0, policy_version 1469565 (0.0007) [2023-12-27 15:49:43,794][20381] Updated weights for policy 1, policy_version 1468107 (0.0007) [2023-12-27 15:49:43,835][20347] Updated weights for policy 0, policy_version 1469575 (0.0007) [2023-12-27 15:49:43,848][20381] Updated weights for policy 1, policy_version 1468117 (0.0005) [2023-12-27 15:49:43,897][20347] Updated weights for policy 0, policy_version 1469585 (0.0008) [2023-12-27 15:49:43,899][20381] Updated weights for policy 1, policy_version 1468127 (0.0006) [2023-12-27 15:49:44,573][20381] Updated weights for policy 1, policy_version 1468137 (0.0005) [2023-12-27 15:49:44,587][20347] Updated weights for policy 0, policy_version 1469595 (0.0008) [2023-12-27 15:49:44,629][20381] Updated weights for policy 1, policy_version 1468147 (0.0007) [2023-12-27 15:49:44,639][20347] Updated weights for policy 0, policy_version 1469605 (0.0008) [2023-12-27 15:49:44,685][20381] Updated weights for policy 1, policy_version 1468157 (0.0007) [2023-12-27 15:49:44,698][20347] Updated weights for policy 0, policy_version 1469615 (0.0006) [2023-12-27 15:49:45,376][20347] Updated weights for policy 0, policy_version 1469625 (0.0008) [2023-12-27 15:49:45,395][20381] Updated weights for policy 1, policy_version 1468167 (0.0007) [2023-12-27 15:49:45,440][20347] Updated weights for policy 0, policy_version 1469635 (0.0007) [2023-12-27 15:49:45,451][20381] Updated weights for policy 1, policy_version 1468177 (0.0006) [2023-12-27 15:49:45,502][20381] Updated weights for policy 1, policy_version 1468187 (0.0007) [2023-12-27 15:49:45,504][20347] Updated weights for policy 0, policy_version 1469645 (0.0007) [2023-12-27 15:49:45,555][20347] Updated weights for policy 0, policy_version 1469655 (0.0006) [2023-12-27 15:49:46,143][20347] Updated weights for policy 0, policy_version 1469665 (0.0006) [2023-12-27 15:49:46,196][20347] Updated weights for policy 0, policy_version 1469675 (0.0007) [2023-12-27 15:49:46,197][20381] Updated weights for policy 1, policy_version 1468197 (0.0008) [2023-12-27 15:49:46,253][20381] Updated weights for policy 1, policy_version 1468207 (0.0005) [2023-12-27 15:49:46,254][20347] Updated weights for policy 0, policy_version 1469685 (0.0006) [2023-12-27 15:49:46,307][20381] Updated weights for policy 1, policy_version 1468217 (0.0007) [2023-12-27 15:49:46,881][20347] Updated weights for policy 0, policy_version 1469695 (0.0005) [2023-12-27 15:49:46,924][20381] Updated weights for policy 1, policy_version 1468227 (0.0009) [2023-12-27 15:49:46,942][20347] Updated weights for policy 0, policy_version 1469705 (0.0008) [2023-12-27 15:49:46,983][20381] Updated weights for policy 1, policy_version 1468237 (0.0011) [2023-12-27 15:49:47,001][20347] Updated weights for policy 0, policy_version 1469715 (0.0011) [2023-12-27 15:49:47,028][20381] Updated weights for policy 1, policy_version 1468247 (0.0010) [2023-12-27 15:49:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 752230400. Throughput: 0: 10393.7, 1: 10182.7. Samples: 752238924. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:47,363][19170] Avg episode reward: [(0, '774.130'), (1, '911.940')] [2023-12-27 15:49:47,640][20347] Updated weights for policy 0, policy_version 1469725 (0.0008) [2023-12-27 15:49:47,689][20347] Updated weights for policy 0, policy_version 1469735 (0.0005) [2023-12-27 15:49:47,713][20381] Updated weights for policy 1, policy_version 1468257 (0.0010) [2023-12-27 15:49:47,741][20347] Updated weights for policy 0, policy_version 1469745 (0.0006) [2023-12-27 15:49:47,765][20381] Updated weights for policy 1, policy_version 1468267 (0.0010) [2023-12-27 15:49:47,816][20381] Updated weights for policy 1, policy_version 1468277 (0.0010) [2023-12-27 15:49:47,875][20381] Updated weights for policy 1, policy_version 1468287 (0.0010) [2023-12-27 15:49:48,396][20347] Updated weights for policy 0, policy_version 1469755 (0.0007) [2023-12-27 15:49:48,453][20347] Updated weights for policy 0, policy_version 1469765 (0.0007) [2023-12-27 15:49:48,514][20347] Updated weights for policy 0, policy_version 1469775 (0.0008) [2023-12-27 15:49:48,548][20381] Updated weights for policy 1, policy_version 1468297 (0.0006) [2023-12-27 15:49:48,606][20381] Updated weights for policy 1, policy_version 1468307 (0.0005) [2023-12-27 15:49:48,662][20381] Updated weights for policy 1, policy_version 1468317 (0.0008) [2023-12-27 15:49:49,208][20381] Updated weights for policy 1, policy_version 1468327 (0.0006) [2023-12-27 15:49:49,211][20347] Updated weights for policy 0, policy_version 1469785 (0.0009) [2023-12-27 15:49:49,277][20381] Updated weights for policy 1, policy_version 1468337 (0.0008) [2023-12-27 15:49:49,283][20347] Updated weights for policy 0, policy_version 1469795 (0.0008) [2023-12-27 15:49:49,345][20381] Updated weights for policy 1, policy_version 1468347 (0.0008) [2023-12-27 15:49:49,346][20347] Updated weights for policy 0, policy_version 1469805 (0.0009) [2023-12-27 15:49:49,406][20347] Updated weights for policy 0, policy_version 1469815 (0.0007) [2023-12-27 15:49:50,061][20381] Updated weights for policy 1, policy_version 1468357 (0.0010) [2023-12-27 15:49:50,087][20347] Updated weights for policy 0, policy_version 1469825 (0.0010) [2023-12-27 15:49:50,119][20381] Updated weights for policy 1, policy_version 1468367 (0.0011) [2023-12-27 15:49:50,146][20347] Updated weights for policy 0, policy_version 1469835 (0.0007) [2023-12-27 15:49:50,182][20381] Updated weights for policy 1, policy_version 1468377 (0.0010) [2023-12-27 15:49:50,207][20347] Updated weights for policy 0, policy_version 1469845 (0.0009) [2023-12-27 15:49:50,818][20381] Updated weights for policy 1, policy_version 1468387 (0.0009) [2023-12-27 15:49:50,848][20347] Updated weights for policy 0, policy_version 1469855 (0.0008) [2023-12-27 15:49:50,878][20381] Updated weights for policy 1, policy_version 1468397 (0.0011) [2023-12-27 15:49:50,906][20347] Updated weights for policy 0, policy_version 1469865 (0.0007) [2023-12-27 15:49:50,938][20381] Updated weights for policy 1, policy_version 1468407 (0.0010) [2023-12-27 15:49:50,961][20347] Updated weights for policy 0, policy_version 1469875 (0.0007) [2023-12-27 15:49:51,667][20381] Updated weights for policy 1, policy_version 1468417 (0.0007) [2023-12-27 15:49:51,721][20347] Updated weights for policy 0, policy_version 1469885 (0.0008) [2023-12-27 15:49:51,727][20381] Updated weights for policy 1, policy_version 1468427 (0.0009) [2023-12-27 15:49:51,781][20381] Updated weights for policy 1, policy_version 1468437 (0.0007) [2023-12-27 15:49:51,784][20347] Updated weights for policy 0, policy_version 1469895 (0.0008) [2023-12-27 15:49:51,839][20381] Updated weights for policy 1, policy_version 1468447 (0.0007) [2023-12-27 15:49:51,839][20347] Updated weights for policy 0, policy_version 1469905 (0.0008) [2023-12-27 15:49:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 752328704. Throughput: 0: 10394.3, 1: 10221.3. Samples: 752300948. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:52,363][19170] Avg episode reward: [(0, '823.460'), (1, '941.310')] [2023-12-27 15:49:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001468448_375955456.pth... [2023-12-27 15:49:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001469912_376373248.pth... [2023-12-27 15:49:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001467280_375652352.pth [2023-12-27 15:49:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001468696_376061952.pth [2023-12-27 15:49:52,543][20381] Updated weights for policy 1, policy_version 1468457 (0.0008) [2023-12-27 15:49:52,601][20381] Updated weights for policy 1, policy_version 1468467 (0.0008) [2023-12-27 15:49:52,610][20347] Updated weights for policy 0, policy_version 1469915 (0.0009) [2023-12-27 15:49:52,659][20381] Updated weights for policy 1, policy_version 1468477 (0.0009) [2023-12-27 15:49:52,669][20347] Updated weights for policy 0, policy_version 1469925 (0.0010) [2023-12-27 15:49:52,726][20347] Updated weights for policy 0, policy_version 1469935 (0.0010) [2023-12-27 15:49:53,328][20381] Updated weights for policy 1, policy_version 1468487 (0.0007) [2023-12-27 15:49:53,357][20347] Updated weights for policy 0, policy_version 1469945 (0.0010) [2023-12-27 15:49:53,387][20381] Updated weights for policy 1, policy_version 1468497 (0.0006) [2023-12-27 15:49:53,411][20347] Updated weights for policy 0, policy_version 1469955 (0.0005) [2023-12-27 15:49:53,444][20381] Updated weights for policy 1, policy_version 1468507 (0.0006) [2023-12-27 15:49:53,463][20347] Updated weights for policy 0, policy_version 1469965 (0.0005) [2023-12-27 15:49:53,512][20347] Updated weights for policy 0, policy_version 1469975 (0.0007) [2023-12-27 15:49:53,973][20381] Updated weights for policy 1, policy_version 1468517 (0.0006) [2023-12-27 15:49:54,028][20381] Updated weights for policy 1, policy_version 1468527 (0.0007) [2023-12-27 15:49:54,085][20381] Updated weights for policy 1, policy_version 1468537 (0.0007) [2023-12-27 15:49:54,125][20347] Updated weights for policy 0, policy_version 1469985 (0.0010) [2023-12-27 15:49:54,189][20347] Updated weights for policy 0, policy_version 1469995 (0.0010) [2023-12-27 15:49:54,253][20347] Updated weights for policy 0, policy_version 1470005 (0.0010) [2023-12-27 15:49:54,729][20381] Updated weights for policy 1, policy_version 1468547 (0.0006) [2023-12-27 15:49:54,787][20381] Updated weights for policy 1, policy_version 1468557 (0.0008) [2023-12-27 15:49:54,842][20381] Updated weights for policy 1, policy_version 1468567 (0.0007) [2023-12-27 15:49:54,923][20347] Updated weights for policy 0, policy_version 1470015 (0.0010) [2023-12-27 15:49:54,990][20347] Updated weights for policy 0, policy_version 1470025 (0.0010) [2023-12-27 15:49:55,051][20347] Updated weights for policy 0, policy_version 1470035 (0.0010) [2023-12-27 15:49:55,521][20381] Updated weights for policy 1, policy_version 1468577 (0.0008) [2023-12-27 15:49:55,577][20381] Updated weights for policy 1, policy_version 1468587 (0.0008) [2023-12-27 15:49:55,624][20381] Updated weights for policy 1, policy_version 1468597 (0.0008) [2023-12-27 15:49:55,645][20347] Updated weights for policy 0, policy_version 1470045 (0.0010) [2023-12-27 15:49:55,666][20381] Updated weights for policy 1, policy_version 1468607 (0.0005) [2023-12-27 15:49:55,699][20347] Updated weights for policy 0, policy_version 1470055 (0.0010) [2023-12-27 15:49:55,769][20347] Updated weights for policy 0, policy_version 1470065 (0.0010) [2023-12-27 15:49:56,300][20381] Updated weights for policy 1, policy_version 1468617 (0.0006) [2023-12-27 15:49:56,361][20381] Updated weights for policy 1, policy_version 1468627 (0.0006) [2023-12-27 15:49:56,406][20381] Updated weights for policy 1, policy_version 1468637 (0.0009) [2023-12-27 15:49:56,431][20347] Updated weights for policy 0, policy_version 1470075 (0.0009) [2023-12-27 15:49:56,485][20347] Updated weights for policy 0, policy_version 1470085 (0.0006) [2023-12-27 15:49:56,532][20347] Updated weights for policy 0, policy_version 1470095 (0.0010) [2023-12-27 15:49:57,055][20381] Updated weights for policy 1, policy_version 1468647 (0.0007) [2023-12-27 15:49:57,105][20381] Updated weights for policy 1, policy_version 1468657 (0.0007) [2023-12-27 15:49:57,154][20381] Updated weights for policy 1, policy_version 1468667 (0.0007) [2023-12-27 15:49:57,157][20347] Updated weights for policy 0, policy_version 1470105 (0.0010) [2023-12-27 15:49:57,220][20347] Updated weights for policy 0, policy_version 1470115 (0.0010) [2023-12-27 15:49:57,283][20347] Updated weights for policy 0, policy_version 1470125 (0.0010) [2023-12-27 15:49:57,338][20347] Updated weights for policy 0, policy_version 1470135 (0.0010) [2023-12-27 15:49:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 752443392. Throughput: 0: 10368.8, 1: 10334.3. Samples: 752426260. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:49:57,364][19170] Avg episode reward: [(0, '744.780'), (1, '872.800')] [2023-12-27 15:49:57,785][20381] Updated weights for policy 1, policy_version 1468677 (0.0007) [2023-12-27 15:49:57,845][20381] Updated weights for policy 1, policy_version 1468687 (0.0006) [2023-12-27 15:49:57,905][20381] Updated weights for policy 1, policy_version 1468697 (0.0007) [2023-12-27 15:49:58,003][20347] Updated weights for policy 0, policy_version 1470145 (0.0010) [2023-12-27 15:49:58,067][20347] Updated weights for policy 0, policy_version 1470155 (0.0010) [2023-12-27 15:49:58,121][20347] Updated weights for policy 0, policy_version 1470165 (0.0010) [2023-12-27 15:49:58,571][20381] Updated weights for policy 1, policy_version 1468707 (0.0007) [2023-12-27 15:49:58,631][20381] Updated weights for policy 1, policy_version 1468717 (0.0008) [2023-12-27 15:49:58,694][20381] Updated weights for policy 1, policy_version 1468727 (0.0008) [2023-12-27 15:49:58,929][20347] Updated weights for policy 0, policy_version 1470175 (0.0008) [2023-12-27 15:49:58,993][20347] Updated weights for policy 0, policy_version 1470185 (0.0008) [2023-12-27 15:49:59,054][20347] Updated weights for policy 0, policy_version 1470195 (0.0007) [2023-12-27 15:49:59,492][20381] Updated weights for policy 1, policy_version 1468737 (0.0008) [2023-12-27 15:49:59,551][20381] Updated weights for policy 1, policy_version 1468747 (0.0010) [2023-12-27 15:49:59,613][20381] Updated weights for policy 1, policy_version 1468757 (0.0011) [2023-12-27 15:49:59,667][20381] Updated weights for policy 1, policy_version 1468767 (0.0009) [2023-12-27 15:49:59,720][20347] Updated weights for policy 0, policy_version 1470205 (0.0007) [2023-12-27 15:49:59,781][20347] Updated weights for policy 0, policy_version 1470215 (0.0008) [2023-12-27 15:49:59,843][20347] Updated weights for policy 0, policy_version 1470225 (0.0008) [2023-12-27 15:50:00,380][20381] Updated weights for policy 1, policy_version 1468777 (0.0007) [2023-12-27 15:50:00,437][20381] Updated weights for policy 1, policy_version 1468787 (0.0011) [2023-12-27 15:50:00,498][20381] Updated weights for policy 1, policy_version 1468797 (0.0010) [2023-12-27 15:50:00,541][20347] Updated weights for policy 0, policy_version 1470235 (0.0007) [2023-12-27 15:50:00,596][20347] Updated weights for policy 0, policy_version 1470245 (0.0007) [2023-12-27 15:50:00,646][20347] Updated weights for policy 0, policy_version 1470255 (0.0007) [2023-12-27 15:50:01,082][20381] Updated weights for policy 1, policy_version 1468807 (0.0008) [2023-12-27 15:50:01,142][20381] Updated weights for policy 1, policy_version 1468817 (0.0006) [2023-12-27 15:50:01,206][20381] Updated weights for policy 1, policy_version 1468827 (0.0007) [2023-12-27 15:50:01,410][20347] Updated weights for policy 0, policy_version 1470265 (0.0007) [2023-12-27 15:50:01,470][20347] Updated weights for policy 0, policy_version 1470275 (0.0007) [2023-12-27 15:50:01,528][20347] Updated weights for policy 0, policy_version 1470285 (0.0007) [2023-12-27 15:50:01,579][20347] Updated weights for policy 0, policy_version 1470295 (0.0007) [2023-12-27 15:50:01,901][20381] Updated weights for policy 1, policy_version 1468837 (0.0006) [2023-12-27 15:50:01,962][20381] Updated weights for policy 1, policy_version 1468847 (0.0005) [2023-12-27 15:50:02,017][20381] Updated weights for policy 1, policy_version 1468857 (0.0009) [2023-12-27 15:50:02,239][20347] Updated weights for policy 0, policy_version 1470305 (0.0008) [2023-12-27 15:50:02,240][20291] KL-divergence is very high: 109.3314 [2023-12-27 15:50:02,246][20291] KL-divergence is very high: 129.8843 [2023-12-27 15:50:02,253][20291] KL-divergence is very high: 135.1420 [2023-12-27 15:50:02,260][20291] KL-divergence is very high: 113.3360 [2023-12-27 15:50:02,279][20291] KL-divergence is very high: 105.3082 [2023-12-27 15:50:02,302][20347] Updated weights for policy 0, policy_version 1470315 (0.0007) [2023-12-27 15:50:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 752533504. Throughput: 0: 10374.9, 1: 10281.0. Samples: 752548132. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:50:02,363][19170] Avg episode reward: [(0, '725.160'), (1, '843.490')] [2023-12-27 15:50:02,365][20347] Updated weights for policy 0, policy_version 1470325 (0.0007) [2023-12-27 15:50:02,687][20381] Updated weights for policy 1, policy_version 1468867 (0.0010) [2023-12-27 15:50:02,749][20381] Updated weights for policy 1, policy_version 1468877 (0.0011) [2023-12-27 15:50:02,809][20381] Updated weights for policy 1, policy_version 1468887 (0.0010) [2023-12-27 15:50:03,017][20291] KL-divergence is very high: 228.8777 [2023-12-27 15:50:03,049][20347] Updated weights for policy 0, policy_version 1470335 (0.0007) [2023-12-27 15:50:03,063][20291] KL-divergence is very high: 105.3641 [2023-12-27 15:50:03,104][20347] Updated weights for policy 0, policy_version 1470345 (0.0006) [2023-12-27 15:50:03,171][20347] Updated weights for policy 0, policy_version 1470355 (0.0008) [2023-12-27 15:50:03,461][20381] Updated weights for policy 1, policy_version 1468897 (0.0010) [2023-12-27 15:50:03,510][20381] Updated weights for policy 1, policy_version 1468907 (0.0005) [2023-12-27 15:50:03,569][20381] Updated weights for policy 1, policy_version 1468917 (0.0010) [2023-12-27 15:50:03,615][20381] Updated weights for policy 1, policy_version 1468927 (0.0008) [2023-12-27 15:50:03,856][20347] Updated weights for policy 0, policy_version 1470365 (0.0008) [2023-12-27 15:50:03,915][20347] Updated weights for policy 0, policy_version 1470375 (0.0007) [2023-12-27 15:50:03,967][20347] Updated weights for policy 0, policy_version 1470385 (0.0007) [2023-12-27 15:50:04,294][20381] Updated weights for policy 1, policy_version 1468937 (0.0010) [2023-12-27 15:50:04,345][20381] Updated weights for policy 1, policy_version 1468947 (0.0007) [2023-12-27 15:50:04,395][20381] Updated weights for policy 1, policy_version 1468957 (0.0008) [2023-12-27 15:50:04,647][20347] Updated weights for policy 0, policy_version 1470395 (0.0007) [2023-12-27 15:50:04,693][20347] Updated weights for policy 0, policy_version 1470405 (0.0007) [2023-12-27 15:50:04,738][20347] Updated weights for policy 0, policy_version 1470415 (0.0007) [2023-12-27 15:50:05,060][20381] Updated weights for policy 1, policy_version 1468967 (0.0007) [2023-12-27 15:50:05,130][20381] Updated weights for policy 1, policy_version 1468977 (0.0005) [2023-12-27 15:50:05,184][20381] Updated weights for policy 1, policy_version 1468987 (0.0005) [2023-12-27 15:50:05,461][20347] Updated weights for policy 0, policy_version 1470425 (0.0009) [2023-12-27 15:50:05,517][20347] Updated weights for policy 0, policy_version 1470435 (0.0007) [2023-12-27 15:50:05,575][20347] Updated weights for policy 0, policy_version 1470445 (0.0005) [2023-12-27 15:50:05,629][20347] Updated weights for policy 0, policy_version 1470455 (0.0007) [2023-12-27 15:50:05,797][20381] Updated weights for policy 1, policy_version 1468997 (0.0007) [2023-12-27 15:50:05,861][20381] Updated weights for policy 1, policy_version 1469007 (0.0006) [2023-12-27 15:50:05,917][20381] Updated weights for policy 1, policy_version 1469017 (0.0007) [2023-12-27 15:50:06,312][20347] Updated weights for policy 0, policy_version 1470465 (0.0006) [2023-12-27 15:50:06,374][20347] Updated weights for policy 0, policy_version 1470475 (0.0007) [2023-12-27 15:50:06,444][20347] Updated weights for policy 0, policy_version 1470485 (0.0008) [2023-12-27 15:50:06,619][20381] Updated weights for policy 1, policy_version 1469027 (0.0008) [2023-12-27 15:50:06,682][20381] Updated weights for policy 1, policy_version 1469037 (0.0008) [2023-12-27 15:50:06,738][20381] Updated weights for policy 1, policy_version 1469047 (0.0008) [2023-12-27 15:50:07,107][20347] Updated weights for policy 0, policy_version 1470495 (0.0008) [2023-12-27 15:50:07,163][20347] Updated weights for policy 0, policy_version 1470505 (0.0008) [2023-12-27 15:50:07,229][20347] Updated weights for policy 0, policy_version 1470515 (0.0007) [2023-12-27 15:50:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 752640000. Throughput: 0: 10345.8, 1: 10284.2. Samples: 752609080. Policy #0 lag: (min: 23.0, avg: 29.0, max: 55.0) [2023-12-27 15:50:07,363][19170] Avg episode reward: [(0, '764.610'), (1, '911.980')] [2023-12-27 15:50:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001469056_376111104.pth... [2023-12-27 15:50:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001470520_376528896.pth... [2023-12-27 15:50:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001469336_376225792.pth [2023-12-27 15:50:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001467872_375808000.pth [2023-12-27 15:50:07,432][20381] Updated weights for policy 1, policy_version 1469057 (0.0008) [2023-12-27 15:50:07,490][20381] Updated weights for policy 1, policy_version 1469067 (0.0006) [2023-12-27 15:50:07,541][20381] Updated weights for policy 1, policy_version 1469077 (0.0006) [2023-12-27 15:50:07,603][20381] Updated weights for policy 1, policy_version 1469087 (0.0009) [2023-12-27 15:50:07,926][20347] Updated weights for policy 0, policy_version 1470525 (0.0008) [2023-12-27 15:50:07,984][20347] Updated weights for policy 0, policy_version 1470535 (0.0008) [2023-12-27 15:50:08,049][20347] Updated weights for policy 0, policy_version 1470545 (0.0006) [2023-12-27 15:50:08,207][20381] Updated weights for policy 1, policy_version 1469097 (0.0006) [2023-12-27 15:50:08,267][20381] Updated weights for policy 1, policy_version 1469107 (0.0006) [2023-12-27 15:50:08,328][20381] Updated weights for policy 1, policy_version 1469117 (0.0007) [2023-12-27 15:50:08,783][20347] Updated weights for policy 0, policy_version 1470555 (0.0008) [2023-12-27 15:50:08,832][20347] Updated weights for policy 0, policy_version 1470565 (0.0007) [2023-12-27 15:50:08,882][20347] Updated weights for policy 0, policy_version 1470575 (0.0008) [2023-12-27 15:50:08,964][20381] Updated weights for policy 1, policy_version 1469127 (0.0008) [2023-12-27 15:50:09,020][20381] Updated weights for policy 1, policy_version 1469137 (0.0008) [2023-12-27 15:50:09,084][20381] Updated weights for policy 1, policy_version 1469147 (0.0005) [2023-12-27 15:50:09,596][20347] Updated weights for policy 0, policy_version 1470585 (0.0007) [2023-12-27 15:50:09,657][20347] Updated weights for policy 0, policy_version 1470595 (0.0008) [2023-12-27 15:50:09,726][20347] Updated weights for policy 0, policy_version 1470605 (0.0006) [2023-12-27 15:50:09,785][20381] Updated weights for policy 1, policy_version 1469157 (0.0008) [2023-12-27 15:50:09,791][20347] Updated weights for policy 0, policy_version 1470615 (0.0006) [2023-12-27 15:50:09,852][20381] Updated weights for policy 1, policy_version 1469167 (0.0008) [2023-12-27 15:50:09,909][20381] Updated weights for policy 1, policy_version 1469177 (0.0008) [2023-12-27 15:50:10,464][20347] Updated weights for policy 0, policy_version 1470625 (0.0007) [2023-12-27 15:50:10,522][20347] Updated weights for policy 0, policy_version 1470635 (0.0007) [2023-12-27 15:50:10,580][20347] Updated weights for policy 0, policy_version 1470645 (0.0007) [2023-12-27 15:50:10,625][20381] Updated weights for policy 1, policy_version 1469187 (0.0008) [2023-12-27 15:50:10,684][20381] Updated weights for policy 1, policy_version 1469197 (0.0010) [2023-12-27 15:50:10,742][20381] Updated weights for policy 1, policy_version 1469207 (0.0010) [2023-12-27 15:50:11,307][20347] Updated weights for policy 0, policy_version 1470655 (0.0007) [2023-12-27 15:50:11,372][20347] Updated weights for policy 0, policy_version 1470665 (0.0007) [2023-12-27 15:50:11,440][20347] Updated weights for policy 0, policy_version 1470675 (0.0006) [2023-12-27 15:50:11,517][20381] Updated weights for policy 1, policy_version 1469217 (0.0010) [2023-12-27 15:50:11,581][20381] Updated weights for policy 1, policy_version 1469227 (0.0010) [2023-12-27 15:50:11,649][20381] Updated weights for policy 1, policy_version 1469237 (0.0011) [2023-12-27 15:50:11,716][20381] Updated weights for policy 1, policy_version 1469247 (0.0011) [2023-12-27 15:50:12,106][20347] Updated weights for policy 0, policy_version 1470685 (0.0007) [2023-12-27 15:50:12,165][20347] Updated weights for policy 0, policy_version 1470695 (0.0007) [2023-12-27 15:50:12,228][20347] Updated weights for policy 0, policy_version 1470705 (0.0007) [2023-12-27 15:50:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 752738304. Throughput: 0: 10249.5, 1: 10198.6. Samples: 752729412. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:12,363][19170] Avg episode reward: [(0, '823.240'), (1, '970.700')] [2023-12-27 15:50:12,411][20381] Updated weights for policy 1, policy_version 1469257 (0.0008) [2023-12-27 15:50:12,473][20381] Updated weights for policy 1, policy_version 1469267 (0.0008) [2023-12-27 15:50:12,535][20381] Updated weights for policy 1, policy_version 1469277 (0.0008) [2023-12-27 15:50:12,862][20347] Updated weights for policy 0, policy_version 1470715 (0.0007) [2023-12-27 15:50:12,909][20347] Updated weights for policy 0, policy_version 1470725 (0.0007) [2023-12-27 15:50:12,956][20347] Updated weights for policy 0, policy_version 1470735 (0.0007) [2023-12-27 15:50:13,206][20381] Updated weights for policy 1, policy_version 1469287 (0.0010) [2023-12-27 15:50:13,258][20381] Updated weights for policy 1, policy_version 1469297 (0.0010) [2023-12-27 15:50:13,306][20381] Updated weights for policy 1, policy_version 1469307 (0.0010) [2023-12-27 15:50:13,644][20347] Updated weights for policy 0, policy_version 1470745 (0.0007) [2023-12-27 15:50:13,696][20347] Updated weights for policy 0, policy_version 1470755 (0.0007) [2023-12-27 15:50:13,747][20347] Updated weights for policy 0, policy_version 1470765 (0.0007) [2023-12-27 15:50:13,794][20347] Updated weights for policy 0, policy_version 1470775 (0.0007) [2023-12-27 15:50:13,955][20381] Updated weights for policy 1, policy_version 1469317 (0.0010) [2023-12-27 15:50:14,014][20381] Updated weights for policy 1, policy_version 1469327 (0.0011) [2023-12-27 15:50:14,070][20381] Updated weights for policy 1, policy_version 1469337 (0.0011) [2023-12-27 15:50:14,418][20347] Updated weights for policy 0, policy_version 1470785 (0.0006) [2023-12-27 15:50:14,468][20347] Updated weights for policy 0, policy_version 1470795 (0.0006) [2023-12-27 15:50:14,528][20347] Updated weights for policy 0, policy_version 1470805 (0.0006) [2023-12-27 15:50:14,756][20381] Updated weights for policy 1, policy_version 1469347 (0.0010) [2023-12-27 15:50:14,825][20381] Updated weights for policy 1, policy_version 1469357 (0.0008) [2023-12-27 15:50:14,878][20381] Updated weights for policy 1, policy_version 1469367 (0.0008) [2023-12-27 15:50:15,207][20347] Updated weights for policy 0, policy_version 1470815 (0.0007) [2023-12-27 15:50:15,264][20347] Updated weights for policy 0, policy_version 1470825 (0.0007) [2023-12-27 15:50:15,319][20347] Updated weights for policy 0, policy_version 1470835 (0.0007) [2023-12-27 15:50:15,578][20381] Updated weights for policy 1, policy_version 1469377 (0.0008) [2023-12-27 15:50:15,634][20381] Updated weights for policy 1, policy_version 1469387 (0.0005) [2023-12-27 15:50:15,680][20381] Updated weights for policy 1, policy_version 1469397 (0.0005) [2023-12-27 15:50:15,728][20381] Updated weights for policy 1, policy_version 1469407 (0.0005) [2023-12-27 15:50:16,006][20347] Updated weights for policy 0, policy_version 1470845 (0.0007) [2023-12-27 15:50:16,061][20347] Updated weights for policy 0, policy_version 1470855 (0.0008) [2023-12-27 15:50:16,111][20347] Updated weights for policy 0, policy_version 1470865 (0.0006) [2023-12-27 15:50:16,372][20381] Updated weights for policy 1, policy_version 1469417 (0.0007) [2023-12-27 15:50:16,426][20381] Updated weights for policy 1, policy_version 1469427 (0.0009) [2023-12-27 15:50:16,477][20381] Updated weights for policy 1, policy_version 1469437 (0.0008) [2023-12-27 15:50:16,693][20347] Updated weights for policy 0, policy_version 1470875 (0.0006) [2023-12-27 15:50:16,761][20347] Updated weights for policy 0, policy_version 1470885 (0.0010) [2023-12-27 15:50:16,817][20347] Updated weights for policy 0, policy_version 1470895 (0.0007) [2023-12-27 15:50:17,179][20381] Updated weights for policy 1, policy_version 1469447 (0.0008) [2023-12-27 15:50:17,235][20381] Updated weights for policy 1, policy_version 1469457 (0.0006) [2023-12-27 15:50:17,285][20381] Updated weights for policy 1, policy_version 1469467 (0.0008) [2023-12-27 15:50:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 752844800. Throughput: 0: 10227.9, 1: 10214.4. Samples: 752853892. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:17,364][19170] Avg episode reward: [(0, '842.870'), (1, '941.400')] [2023-12-27 15:50:17,407][20347] Updated weights for policy 0, policy_version 1470905 (0.0009) [2023-12-27 15:50:17,455][20347] Updated weights for policy 0, policy_version 1470915 (0.0010) [2023-12-27 15:50:17,510][20347] Updated weights for policy 0, policy_version 1470925 (0.0010) [2023-12-27 15:50:17,570][20347] Updated weights for policy 0, policy_version 1470935 (0.0010) [2023-12-27 15:50:17,961][20381] Updated weights for policy 1, policy_version 1469477 (0.0010) [2023-12-27 15:50:18,023][20381] Updated weights for policy 1, policy_version 1469487 (0.0011) [2023-12-27 15:50:18,084][20381] Updated weights for policy 1, policy_version 1469497 (0.0011) [2023-12-27 15:50:18,242][20347] Updated weights for policy 0, policy_version 1470945 (0.0009) [2023-12-27 15:50:18,314][20347] Updated weights for policy 0, policy_version 1470955 (0.0007) [2023-12-27 15:50:18,379][20347] Updated weights for policy 0, policy_version 1470965 (0.0008) [2023-12-27 15:50:18,767][20381] Updated weights for policy 1, policy_version 1469507 (0.0008) [2023-12-27 15:50:18,819][20292] KL-divergence is very high: 130.2495 [2023-12-27 15:50:18,825][20381] Updated weights for policy 1, policy_version 1469517 (0.0007) [2023-12-27 15:50:18,868][20292] KL-divergence is very high: 149.8426 [2023-12-27 15:50:18,886][20381] Updated weights for policy 1, policy_version 1469527 (0.0006) [2023-12-27 15:50:19,081][20347] Updated weights for policy 0, policy_version 1470975 (0.0008) [2023-12-27 15:50:19,130][20347] Updated weights for policy 0, policy_version 1470985 (0.0008) [2023-12-27 15:50:19,188][20347] Updated weights for policy 0, policy_version 1470995 (0.0007) [2023-12-27 15:50:19,581][20381] Updated weights for policy 1, policy_version 1469537 (0.0007) [2023-12-27 15:50:19,650][20381] Updated weights for policy 1, policy_version 1469547 (0.0005) [2023-12-27 15:50:19,717][20381] Updated weights for policy 1, policy_version 1469557 (0.0007) [2023-12-27 15:50:19,777][20381] Updated weights for policy 1, policy_version 1469567 (0.0006) [2023-12-27 15:50:19,923][20347] Updated weights for policy 0, policy_version 1471005 (0.0009) [2023-12-27 15:50:19,986][20347] Updated weights for policy 0, policy_version 1471015 (0.0009) [2023-12-27 15:50:20,040][20347] Updated weights for policy 0, policy_version 1471025 (0.0010) [2023-12-27 15:50:20,436][20381] Updated weights for policy 1, policy_version 1469577 (0.0006) [2023-12-27 15:50:20,486][20381] Updated weights for policy 1, policy_version 1469587 (0.0005) [2023-12-27 15:50:20,545][20381] Updated weights for policy 1, policy_version 1469597 (0.0007) [2023-12-27 15:50:20,769][20347] Updated weights for policy 0, policy_version 1471035 (0.0010) [2023-12-27 15:50:20,822][20347] Updated weights for policy 0, policy_version 1471045 (0.0008) [2023-12-27 15:50:20,882][20347] Updated weights for policy 0, policy_version 1471055 (0.0007) [2023-12-27 15:50:21,266][20381] Updated weights for policy 1, policy_version 1469607 (0.0008) [2023-12-27 15:50:21,331][20381] Updated weights for policy 1, policy_version 1469617 (0.0008) [2023-12-27 15:50:21,400][20381] Updated weights for policy 1, policy_version 1469627 (0.0009) [2023-12-27 15:50:21,604][20347] Updated weights for policy 0, policy_version 1471065 (0.0006) [2023-12-27 15:50:21,670][20347] Updated weights for policy 0, policy_version 1471075 (0.0008) [2023-12-27 15:50:21,737][20347] Updated weights for policy 0, policy_version 1471085 (0.0008) [2023-12-27 15:50:21,797][20347] Updated weights for policy 0, policy_version 1471095 (0.0007) [2023-12-27 15:50:22,110][20381] Updated weights for policy 1, policy_version 1469637 (0.0008) [2023-12-27 15:50:22,161][20381] Updated weights for policy 1, policy_version 1469647 (0.0005) [2023-12-27 15:50:22,214][20381] Updated weights for policy 1, policy_version 1469657 (0.0005) [2023-12-27 15:50:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 752943104. Throughput: 0: 10196.8, 1: 10242.0. Samples: 752913764. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:22,363][19170] Avg episode reward: [(0, '842.990'), (1, '941.320')] [2023-12-27 15:50:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001469664_376266752.pth... [2023-12-27 15:50:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001471096_376676352.pth... [2023-12-27 15:50:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001469912_376373248.pth [2023-12-27 15:50:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001468448_375955456.pth [2023-12-27 15:50:22,511][20347] Updated weights for policy 0, policy_version 1471105 (0.0008) [2023-12-27 15:50:22,570][20347] Updated weights for policy 0, policy_version 1471115 (0.0007) [2023-12-27 15:50:22,635][20347] Updated weights for policy 0, policy_version 1471125 (0.0006) [2023-12-27 15:50:22,920][20381] Updated weights for policy 1, policy_version 1469667 (0.0007) [2023-12-27 15:50:22,975][20381] Updated weights for policy 1, policy_version 1469677 (0.0008) [2023-12-27 15:50:22,989][20292] KL-divergence is very high: 127.5975 [2023-12-27 15:50:23,021][20381] Updated weights for policy 1, policy_version 1469687 (0.0008) [2023-12-27 15:50:23,026][20292] KL-divergence is very high: 145.1816 [2023-12-27 15:50:23,280][20347] Updated weights for policy 0, policy_version 1471135 (0.0006) [2023-12-27 15:50:23,329][20347] Updated weights for policy 0, policy_version 1471145 (0.0006) [2023-12-27 15:50:23,385][20347] Updated weights for policy 0, policy_version 1471155 (0.0010) [2023-12-27 15:50:23,671][20381] Updated weights for policy 1, policy_version 1469697 (0.0007) [2023-12-27 15:50:23,724][20381] Updated weights for policy 1, policy_version 1469707 (0.0007) [2023-12-27 15:50:23,769][20381] Updated weights for policy 1, policy_version 1469717 (0.0007) [2023-12-27 15:50:23,815][20381] Updated weights for policy 1, policy_version 1469727 (0.0008) [2023-12-27 15:50:24,043][20347] Updated weights for policy 0, policy_version 1471165 (0.0010) [2023-12-27 15:50:24,096][20347] Updated weights for policy 0, policy_version 1471175 (0.0008) [2023-12-27 15:50:24,154][20347] Updated weights for policy 0, policy_version 1471185 (0.0008) [2023-12-27 15:50:24,398][20381] Updated weights for policy 1, policy_version 1469737 (0.0006) [2023-12-27 15:50:24,455][20381] Updated weights for policy 1, policy_version 1469747 (0.0005) [2023-12-27 15:50:24,506][20381] Updated weights for policy 1, policy_version 1469757 (0.0005) [2023-12-27 15:50:24,832][20347] Updated weights for policy 0, policy_version 1471195 (0.0010) [2023-12-27 15:50:24,879][20347] Updated weights for policy 0, policy_version 1471205 (0.0010) [2023-12-27 15:50:24,927][20347] Updated weights for policy 0, policy_version 1471215 (0.0010) [2023-12-27 15:50:25,074][20381] Updated weights for policy 1, policy_version 1469767 (0.0007) [2023-12-27 15:50:25,128][20381] Updated weights for policy 1, policy_version 1469777 (0.0007) [2023-12-27 15:50:25,179][20381] Updated weights for policy 1, policy_version 1469787 (0.0007) [2023-12-27 15:50:25,588][20347] Updated weights for policy 0, policy_version 1471225 (0.0010) [2023-12-27 15:50:25,632][20347] Updated weights for policy 0, policy_version 1471235 (0.0010) [2023-12-27 15:50:25,678][20347] Updated weights for policy 0, policy_version 1471245 (0.0010) [2023-12-27 15:50:25,722][20347] Updated weights for policy 0, policy_version 1471255 (0.0010) [2023-12-27 15:50:25,816][20381] Updated weights for policy 1, policy_version 1469797 (0.0006) [2023-12-27 15:50:25,872][20381] Updated weights for policy 1, policy_version 1469807 (0.0005) [2023-12-27 15:50:25,925][20381] Updated weights for policy 1, policy_version 1469817 (0.0005) [2023-12-27 15:50:26,402][20347] Updated weights for policy 0, policy_version 1471265 (0.0010) [2023-12-27 15:50:26,456][20347] Updated weights for policy 0, policy_version 1471275 (0.0010) [2023-12-27 15:50:26,498][20381] Updated weights for policy 1, policy_version 1469827 (0.0005) [2023-12-27 15:50:26,507][20347] Updated weights for policy 0, policy_version 1471285 (0.0010) [2023-12-27 15:50:26,550][20381] Updated weights for policy 1, policy_version 1469837 (0.0006) [2023-12-27 15:50:26,606][20381] Updated weights for policy 1, policy_version 1469847 (0.0005) [2023-12-27 15:50:27,143][20347] Updated weights for policy 0, policy_version 1471295 (0.0010) [2023-12-27 15:50:27,200][20347] Updated weights for policy 0, policy_version 1471305 (0.0010) [2023-12-27 15:50:27,227][20381] Updated weights for policy 1, policy_version 1469857 (0.0006) [2023-12-27 15:50:27,257][20347] Updated weights for policy 0, policy_version 1471315 (0.0010) [2023-12-27 15:50:27,286][20381] Updated weights for policy 1, policy_version 1469867 (0.0010) [2023-12-27 15:50:27,342][20381] Updated weights for policy 1, policy_version 1469877 (0.0010) [2023-12-27 15:50:27,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 753049600. Throughput: 0: 10174.6, 1: 10340.0. Samples: 753038692. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:27,364][19170] Avg episode reward: [(0, '705.670'), (1, '872.980')] [2023-12-27 15:50:27,418][20381] Updated weights for policy 1, policy_version 1469887 (0.0010) [2023-12-27 15:50:27,939][20347] Updated weights for policy 0, policy_version 1471325 (0.0008) [2023-12-27 15:50:27,991][20347] Updated weights for policy 0, policy_version 1471335 (0.0005) [2023-12-27 15:50:28,013][20291] KL-divergence is very high: 161.0779 [2023-12-27 15:50:28,018][20291] KL-divergence is very high: 151.8722 [2023-12-27 15:50:28,042][20347] Updated weights for policy 0, policy_version 1471345 (0.0006) [2023-12-27 15:50:28,073][20381] Updated weights for policy 1, policy_version 1469897 (0.0010) [2023-12-27 15:50:28,124][20381] Updated weights for policy 1, policy_version 1469907 (0.0010) [2023-12-27 15:50:28,184][20381] Updated weights for policy 1, policy_version 1469917 (0.0010) [2023-12-27 15:50:28,692][20347] Updated weights for policy 0, policy_version 1471355 (0.0007) [2023-12-27 15:50:28,722][20291] KL-divergence is very high: 119.2350 [2023-12-27 15:50:28,744][20347] Updated weights for policy 0, policy_version 1471365 (0.0010) [2023-12-27 15:50:28,768][20291] KL-divergence is very high: 156.1923 [2023-12-27 15:50:28,804][20347] Updated weights for policy 0, policy_version 1471375 (0.0010) [2023-12-27 15:50:28,815][20291] KL-divergence is very high: 104.8848 [2023-12-27 15:50:28,817][20381] Updated weights for policy 1, policy_version 1469927 (0.0007) [2023-12-27 15:50:28,874][20381] Updated weights for policy 1, policy_version 1469937 (0.0006) [2023-12-27 15:50:28,932][20381] Updated weights for policy 1, policy_version 1469947 (0.0008) [2023-12-27 15:50:29,505][20347] Updated weights for policy 0, policy_version 1471385 (0.0010) [2023-12-27 15:50:29,562][20347] Updated weights for policy 0, policy_version 1471395 (0.0007) [2023-12-27 15:50:29,589][20381] Updated weights for policy 1, policy_version 1469957 (0.0009) [2023-12-27 15:50:29,617][20347] Updated weights for policy 0, policy_version 1471405 (0.0005) [2023-12-27 15:50:29,641][20381] Updated weights for policy 1, policy_version 1469967 (0.0010) [2023-12-27 15:50:29,678][20347] Updated weights for policy 0, policy_version 1471415 (0.0007) [2023-12-27 15:50:29,700][20381] Updated weights for policy 1, policy_version 1469977 (0.0010) [2023-12-27 15:50:30,365][20347] Updated weights for policy 0, policy_version 1471425 (0.0007) [2023-12-27 15:50:30,422][20347] Updated weights for policy 0, policy_version 1471435 (0.0007) [2023-12-27 15:50:30,436][20381] Updated weights for policy 1, policy_version 1469987 (0.0010) [2023-12-27 15:50:30,481][20347] Updated weights for policy 0, policy_version 1471445 (0.0006) [2023-12-27 15:50:30,494][20381] Updated weights for policy 1, policy_version 1469997 (0.0010) [2023-12-27 15:50:30,554][20381] Updated weights for policy 1, policy_version 1470007 (0.0010) [2023-12-27 15:50:31,143][20347] Updated weights for policy 0, policy_version 1471455 (0.0007) [2023-12-27 15:50:31,196][20347] Updated weights for policy 0, policy_version 1471465 (0.0007) [2023-12-27 15:50:31,211][20381] Updated weights for policy 1, policy_version 1470017 (0.0010) [2023-12-27 15:50:31,256][20347] Updated weights for policy 0, policy_version 1471475 (0.0008) [2023-12-27 15:50:31,278][20381] Updated weights for policy 1, policy_version 1470027 (0.0007) [2023-12-27 15:50:31,337][20381] Updated weights for policy 1, policy_version 1470037 (0.0012) [2023-12-27 15:50:31,395][20381] Updated weights for policy 1, policy_version 1470047 (0.0008) [2023-12-27 15:50:31,984][20347] Updated weights for policy 0, policy_version 1471485 (0.0010) [2023-12-27 15:50:32,013][20381] Updated weights for policy 1, policy_version 1470057 (0.0010) [2023-12-27 15:50:32,042][20347] Updated weights for policy 0, policy_version 1471495 (0.0010) [2023-12-27 15:50:32,065][20381] Updated weights for policy 1, policy_version 1470067 (0.0008) [2023-12-27 15:50:32,095][20347] Updated weights for policy 0, policy_version 1471505 (0.0010) [2023-12-27 15:50:32,118][20381] Updated weights for policy 1, policy_version 1470077 (0.0005) [2023-12-27 15:50:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 753156096. Throughput: 0: 10168.6, 1: 10377.2. Samples: 753163484. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:32,363][19170] Avg episode reward: [(0, '774.380'), (1, '872.930')] [2023-12-27 15:50:32,765][20347] Updated weights for policy 0, policy_version 1471515 (0.0010) [2023-12-27 15:50:32,794][20381] Updated weights for policy 1, policy_version 1470087 (0.0005) [2023-12-27 15:50:32,814][20347] Updated weights for policy 0, policy_version 1471525 (0.0008) [2023-12-27 15:50:32,850][20381] Updated weights for policy 1, policy_version 1470097 (0.0005) [2023-12-27 15:50:32,858][20347] Updated weights for policy 0, policy_version 1471535 (0.0006) [2023-12-27 15:50:32,907][20381] Updated weights for policy 1, policy_version 1470107 (0.0009) [2023-12-27 15:50:33,468][20381] Updated weights for policy 1, policy_version 1470117 (0.0009) [2023-12-27 15:50:33,470][20347] Updated weights for policy 0, policy_version 1471545 (0.0005) [2023-12-27 15:50:33,521][20347] Updated weights for policy 0, policy_version 1471555 (0.0006) [2023-12-27 15:50:33,523][20381] Updated weights for policy 1, policy_version 1470127 (0.0006) [2023-12-27 15:50:33,571][20347] Updated weights for policy 0, policy_version 1471565 (0.0005) [2023-12-27 15:50:33,578][20381] Updated weights for policy 1, policy_version 1470137 (0.0008) [2023-12-27 15:50:33,629][20347] Updated weights for policy 0, policy_version 1471575 (0.0006) [2023-12-27 15:50:34,291][20347] Updated weights for policy 0, policy_version 1471585 (0.0008) [2023-12-27 15:50:34,306][20381] Updated weights for policy 1, policy_version 1470147 (0.0007) [2023-12-27 15:50:34,350][20347] Updated weights for policy 0, policy_version 1471595 (0.0008) [2023-12-27 15:50:34,357][20381] Updated weights for policy 1, policy_version 1470157 (0.0005) [2023-12-27 15:50:34,404][20381] Updated weights for policy 1, policy_version 1470167 (0.0006) [2023-12-27 15:50:34,410][20347] Updated weights for policy 0, policy_version 1471605 (0.0006) [2023-12-27 15:50:35,014][20347] Updated weights for policy 0, policy_version 1471615 (0.0008) [2023-12-27 15:50:35,079][20347] Updated weights for policy 0, policy_version 1471625 (0.0008) [2023-12-27 15:50:35,127][20381] Updated weights for policy 1, policy_version 1470177 (0.0007) [2023-12-27 15:50:35,135][20347] Updated weights for policy 0, policy_version 1471635 (0.0007) [2023-12-27 15:50:35,182][20381] Updated weights for policy 1, policy_version 1470187 (0.0007) [2023-12-27 15:50:35,237][20381] Updated weights for policy 1, policy_version 1470197 (0.0007) [2023-12-27 15:50:35,291][20381] Updated weights for policy 1, policy_version 1470207 (0.0006) [2023-12-27 15:50:35,739][20347] Updated weights for policy 0, policy_version 1471645 (0.0010) [2023-12-27 15:50:35,793][20347] Updated weights for policy 0, policy_version 1471655 (0.0010) [2023-12-27 15:50:35,851][20347] Updated weights for policy 0, policy_version 1471665 (0.0010) [2023-12-27 15:50:35,885][20381] Updated weights for policy 1, policy_version 1470217 (0.0008) [2023-12-27 15:50:35,937][20381] Updated weights for policy 1, policy_version 1470227 (0.0010) [2023-12-27 15:50:35,992][20381] Updated weights for policy 1, policy_version 1470237 (0.0009) [2023-12-27 15:50:36,542][20347] Updated weights for policy 0, policy_version 1471675 (0.0010) [2023-12-27 15:50:36,602][20347] Updated weights for policy 0, policy_version 1471685 (0.0010) [2023-12-27 15:50:36,643][20381] Updated weights for policy 1, policy_version 1470247 (0.0009) [2023-12-27 15:50:36,658][20347] Updated weights for policy 0, policy_version 1471695 (0.0010) [2023-12-27 15:50:36,697][20381] Updated weights for policy 1, policy_version 1470257 (0.0011) [2023-12-27 15:50:36,756][20381] Updated weights for policy 1, policy_version 1470267 (0.0010) [2023-12-27 15:50:37,316][20347] Updated weights for policy 0, policy_version 1471705 (0.0010) [2023-12-27 15:50:37,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 753254400. Throughput: 0: 10207.5, 1: 10379.3. Samples: 753227352. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:37,363][19170] Avg episode reward: [(0, '852.840'), (1, '882.800')] [2023-12-27 15:50:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001470272_376422400.pth... [2023-12-27 15:50:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001469056_376111104.pth [2023-12-27 15:50:37,379][20347] Updated weights for policy 0, policy_version 1471715 (0.0006) [2023-12-27 15:50:37,441][20347] Updated weights for policy 0, policy_version 1471725 (0.0005) [2023-12-27 15:50:37,467][20381] Updated weights for policy 1, policy_version 1470277 (0.0011) [2023-12-27 15:50:37,501][20347] Updated weights for policy 0, policy_version 1471735 (0.0007) [2023-12-27 15:50:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001471736_376840192.pth... [2023-12-27 15:50:37,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001470520_376528896.pth [2023-12-27 15:50:37,527][20381] Updated weights for policy 1, policy_version 1470287 (0.0011) [2023-12-27 15:50:37,587][20381] Updated weights for policy 1, policy_version 1470297 (0.0008) [2023-12-27 15:50:38,110][20347] Updated weights for policy 0, policy_version 1471745 (0.0006) [2023-12-27 15:50:38,172][20347] Updated weights for policy 0, policy_version 1471755 (0.0005) [2023-12-27 15:50:38,226][20347] Updated weights for policy 0, policy_version 1471765 (0.0005) [2023-12-27 15:50:38,238][20381] Updated weights for policy 1, policy_version 1470307 (0.0006) [2023-12-27 15:50:38,306][20381] Updated weights for policy 1, policy_version 1470317 (0.0005) [2023-12-27 15:50:38,372][20381] Updated weights for policy 1, policy_version 1470327 (0.0008) [2023-12-27 15:50:38,885][20347] Updated weights for policy 0, policy_version 1471775 (0.0007) [2023-12-27 15:50:38,944][20347] Updated weights for policy 0, policy_version 1471785 (0.0008) [2023-12-27 15:50:39,002][20347] Updated weights for policy 0, policy_version 1471795 (0.0007) [2023-12-27 15:50:39,019][20381] Updated weights for policy 1, policy_version 1470337 (0.0008) [2023-12-27 15:50:39,080][20381] Updated weights for policy 1, policy_version 1470347 (0.0006) [2023-12-27 15:50:39,142][20381] Updated weights for policy 1, policy_version 1470357 (0.0008) [2023-12-27 15:50:39,193][20381] Updated weights for policy 1, policy_version 1470367 (0.0007) [2023-12-27 15:50:39,657][20347] Updated weights for policy 0, policy_version 1471805 (0.0006) [2023-12-27 15:50:39,716][20347] Updated weights for policy 0, policy_version 1471815 (0.0008) [2023-12-27 15:50:39,779][20347] Updated weights for policy 0, policy_version 1471825 (0.0009) [2023-12-27 15:50:39,929][20381] Updated weights for policy 1, policy_version 1470377 (0.0007) [2023-12-27 15:50:39,991][20381] Updated weights for policy 1, policy_version 1470387 (0.0008) [2023-12-27 15:50:40,062][20381] Updated weights for policy 1, policy_version 1470397 (0.0008) [2023-12-27 15:50:40,491][20347] Updated weights for policy 0, policy_version 1471835 (0.0012) [2023-12-27 15:50:40,549][20347] Updated weights for policy 0, policy_version 1471845 (0.0010) [2023-12-27 15:50:40,607][20347] Updated weights for policy 0, policy_version 1471855 (0.0010) [2023-12-27 15:50:40,706][20381] Updated weights for policy 1, policy_version 1470407 (0.0008) [2023-12-27 15:50:40,767][20381] Updated weights for policy 1, policy_version 1470417 (0.0007) [2023-12-27 15:50:40,829][20381] Updated weights for policy 1, policy_version 1470427 (0.0006) [2023-12-27 15:50:41,363][20347] Updated weights for policy 0, policy_version 1471865 (0.0010) [2023-12-27 15:50:41,429][20347] Updated weights for policy 0, policy_version 1471875 (0.0009) [2023-12-27 15:50:41,485][20347] Updated weights for policy 0, policy_version 1471885 (0.0008) [2023-12-27 15:50:41,536][20347] Updated weights for policy 0, policy_version 1471895 (0.0008) [2023-12-27 15:50:41,550][20381] Updated weights for policy 1, policy_version 1470437 (0.0006) [2023-12-27 15:50:41,618][20381] Updated weights for policy 1, policy_version 1470447 (0.0006) [2023-12-27 15:50:41,686][20381] Updated weights for policy 1, policy_version 1470457 (0.0007) [2023-12-27 15:50:42,261][20347] Updated weights for policy 0, policy_version 1471905 (0.0007) [2023-12-27 15:50:42,325][20347] Updated weights for policy 0, policy_version 1471915 (0.0008) [2023-12-27 15:50:42,333][20381] Updated weights for policy 1, policy_version 1470467 (0.0008) [2023-12-27 15:50:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 753352704. Throughput: 0: 10218.5, 1: 10331.3. Samples: 753351000. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:42,363][19170] Avg episode reward: [(0, '852.680'), (1, '882.790')] [2023-12-27 15:50:42,391][20347] Updated weights for policy 0, policy_version 1471925 (0.0008) [2023-12-27 15:50:42,394][20381] Updated weights for policy 1, policy_version 1470477 (0.0008) [2023-12-27 15:50:42,455][20381] Updated weights for policy 1, policy_version 1470487 (0.0007) [2023-12-27 15:50:43,073][20381] Updated weights for policy 1, policy_version 1470497 (0.0006) [2023-12-27 15:50:43,110][20347] Updated weights for policy 0, policy_version 1471935 (0.0006) [2023-12-27 15:50:43,129][20381] Updated weights for policy 1, policy_version 1470507 (0.0007) [2023-12-27 15:50:43,163][20347] Updated weights for policy 0, policy_version 1471945 (0.0005) [2023-12-27 15:50:43,188][20381] Updated weights for policy 1, policy_version 1470517 (0.0005) [2023-12-27 15:50:43,221][20347] Updated weights for policy 0, policy_version 1471955 (0.0005) [2023-12-27 15:50:43,247][20381] Updated weights for policy 1, policy_version 1470527 (0.0005) [2023-12-27 15:50:43,760][20347] Updated weights for policy 0, policy_version 1471965 (0.0005) [2023-12-27 15:50:43,814][20347] Updated weights for policy 0, policy_version 1471975 (0.0009) [2023-12-27 15:50:43,858][20381] Updated weights for policy 1, policy_version 1470537 (0.0010) [2023-12-27 15:50:43,865][20347] Updated weights for policy 0, policy_version 1471985 (0.0010) [2023-12-27 15:50:43,910][20381] Updated weights for policy 1, policy_version 1470547 (0.0010) [2023-12-27 15:50:43,966][20381] Updated weights for policy 1, policy_version 1470557 (0.0010) [2023-12-27 15:50:44,524][20347] Updated weights for policy 0, policy_version 1471995 (0.0010) [2023-12-27 15:50:44,532][20381] Updated weights for policy 1, policy_version 1470567 (0.0009) [2023-12-27 15:50:44,569][20347] Updated weights for policy 0, policy_version 1472005 (0.0010) [2023-12-27 15:50:44,580][20381] Updated weights for policy 1, policy_version 1470577 (0.0010) [2023-12-27 15:50:44,613][20347] Updated weights for policy 0, policy_version 1472015 (0.0007) [2023-12-27 15:50:44,637][20381] Updated weights for policy 1, policy_version 1470587 (0.0006) [2023-12-27 15:50:45,258][20347] Updated weights for policy 0, policy_version 1472025 (0.0005) [2023-12-27 15:50:45,312][20347] Updated weights for policy 0, policy_version 1472035 (0.0005) [2023-12-27 15:50:45,344][20381] Updated weights for policy 1, policy_version 1470597 (0.0009) [2023-12-27 15:50:45,368][20347] Updated weights for policy 0, policy_version 1472045 (0.0006) [2023-12-27 15:50:45,397][20381] Updated weights for policy 1, policy_version 1470607 (0.0011) [2023-12-27 15:50:45,424][20347] Updated weights for policy 0, policy_version 1472055 (0.0006) [2023-12-27 15:50:45,456][20381] Updated weights for policy 1, policy_version 1470617 (0.0010) [2023-12-27 15:50:46,048][20347] Updated weights for policy 0, policy_version 1472065 (0.0005) [2023-12-27 15:50:46,096][20347] Updated weights for policy 0, policy_version 1472075 (0.0005) [2023-12-27 15:50:46,110][20381] Updated weights for policy 1, policy_version 1470627 (0.0011) [2023-12-27 15:50:46,145][20347] Updated weights for policy 0, policy_version 1472085 (0.0007) [2023-12-27 15:50:46,163][20381] Updated weights for policy 1, policy_version 1470637 (0.0010) [2023-12-27 15:50:46,219][20381] Updated weights for policy 1, policy_version 1470647 (0.0010) [2023-12-27 15:50:46,797][20347] Updated weights for policy 0, policy_version 1472095 (0.0007) [2023-12-27 15:50:46,858][20347] Updated weights for policy 0, policy_version 1472105 (0.0007) [2023-12-27 15:50:46,912][20381] Updated weights for policy 1, policy_version 1470657 (0.0010) [2023-12-27 15:50:46,923][20347] Updated weights for policy 0, policy_version 1472115 (0.0009) [2023-12-27 15:50:46,975][20381] Updated weights for policy 1, policy_version 1470667 (0.0008) [2023-12-27 15:50:47,040][20381] Updated weights for policy 1, policy_version 1470677 (0.0008) [2023-12-27 15:50:47,093][20381] Updated weights for policy 1, policy_version 1470687 (0.0005) [2023-12-27 15:50:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 753467392. Throughput: 0: 10277.2, 1: 10382.7. Samples: 753477832. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:47,364][19170] Avg episode reward: [(0, '794.390'), (1, '892.560')] [2023-12-27 15:50:47,580][20347] Updated weights for policy 0, policy_version 1472125 (0.0010) [2023-12-27 15:50:47,631][20347] Updated weights for policy 0, policy_version 1472135 (0.0010) [2023-12-27 15:50:47,690][20347] Updated weights for policy 0, policy_version 1472145 (0.0010) [2023-12-27 15:50:47,705][20381] Updated weights for policy 1, policy_version 1470697 (0.0010) [2023-12-27 15:50:47,768][20381] Updated weights for policy 1, policy_version 1470707 (0.0010) [2023-12-27 15:50:47,828][20381] Updated weights for policy 1, policy_version 1470717 (0.0011) [2023-12-27 15:50:48,371][20347] Updated weights for policy 0, policy_version 1472155 (0.0009) [2023-12-27 15:50:48,430][20347] Updated weights for policy 0, policy_version 1472165 (0.0011) [2023-12-27 15:50:48,492][20347] Updated weights for policy 0, policy_version 1472175 (0.0010) [2023-12-27 15:50:48,534][20381] Updated weights for policy 1, policy_version 1470727 (0.0011) [2023-12-27 15:50:48,596][20381] Updated weights for policy 1, policy_version 1470737 (0.0011) [2023-12-27 15:50:48,648][20381] Updated weights for policy 1, policy_version 1470747 (0.0010) [2023-12-27 15:50:49,178][20347] Updated weights for policy 0, policy_version 1472185 (0.0010) [2023-12-27 15:50:49,242][20347] Updated weights for policy 0, policy_version 1472195 (0.0008) [2023-12-27 15:50:49,301][20347] Updated weights for policy 0, policy_version 1472205 (0.0008) [2023-12-27 15:50:49,347][20381] Updated weights for policy 1, policy_version 1470757 (0.0011) [2023-12-27 15:50:49,363][20347] Updated weights for policy 0, policy_version 1472215 (0.0007) [2023-12-27 15:50:49,410][20381] Updated weights for policy 1, policy_version 1470767 (0.0011) [2023-12-27 15:50:49,470][20381] Updated weights for policy 1, policy_version 1470777 (0.0010) [2023-12-27 15:50:50,041][20347] Updated weights for policy 0, policy_version 1472225 (0.0008) [2023-12-27 15:50:50,097][20347] Updated weights for policy 0, policy_version 1472235 (0.0007) [2023-12-27 15:50:50,136][20381] Updated weights for policy 1, policy_version 1470787 (0.0010) [2023-12-27 15:50:50,152][20347] Updated weights for policy 0, policy_version 1472245 (0.0007) [2023-12-27 15:50:50,192][20381] Updated weights for policy 1, policy_version 1470797 (0.0010) [2023-12-27 15:50:50,242][20381] Updated weights for policy 1, policy_version 1470807 (0.0010) [2023-12-27 15:50:50,853][20347] Updated weights for policy 0, policy_version 1472255 (0.0006) [2023-12-27 15:50:50,921][20347] Updated weights for policy 0, policy_version 1472265 (0.0005) [2023-12-27 15:50:50,963][20381] Updated weights for policy 1, policy_version 1470817 (0.0010) [2023-12-27 15:50:50,988][20347] Updated weights for policy 0, policy_version 1472275 (0.0005) [2023-12-27 15:50:51,024][20381] Updated weights for policy 1, policy_version 1470827 (0.0007) [2023-12-27 15:50:51,086][20381] Updated weights for policy 1, policy_version 1470837 (0.0008) [2023-12-27 15:50:51,150][20381] Updated weights for policy 1, policy_version 1470847 (0.0008) [2023-12-27 15:50:51,707][20347] Updated weights for policy 0, policy_version 1472285 (0.0007) [2023-12-27 15:50:51,775][20347] Updated weights for policy 0, policy_version 1472295 (0.0007) [2023-12-27 15:50:51,814][20381] Updated weights for policy 1, policy_version 1470857 (0.0007) [2023-12-27 15:50:51,834][20347] Updated weights for policy 0, policy_version 1472305 (0.0007) [2023-12-27 15:50:51,884][20381] Updated weights for policy 1, policy_version 1470867 (0.0006) [2023-12-27 15:50:51,942][20381] Updated weights for policy 1, policy_version 1470877 (0.0007) [2023-12-27 15:50:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 753565696. Throughput: 0: 10289.2, 1: 10370.1. Samples: 753538752. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:52,363][19170] Avg episode reward: [(0, '755.280'), (1, '902.500')] [2023-12-27 15:50:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001470880_376578048.pth... [2023-12-27 15:50:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001472312_376987648.pth... [2023-12-27 15:50:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001469664_376266752.pth [2023-12-27 15:50:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001471096_376676352.pth [2023-12-27 15:50:52,531][20347] Updated weights for policy 0, policy_version 1472315 (0.0008) [2023-12-27 15:50:52,592][20347] Updated weights for policy 0, policy_version 1472325 (0.0008) [2023-12-27 15:50:52,629][20381] Updated weights for policy 1, policy_version 1470887 (0.0006) [2023-12-27 15:50:52,646][20347] Updated weights for policy 0, policy_version 1472335 (0.0008) [2023-12-27 15:50:52,687][20381] Updated weights for policy 1, policy_version 1470897 (0.0005) [2023-12-27 15:50:52,748][20381] Updated weights for policy 1, policy_version 1470907 (0.0008) [2023-12-27 15:50:53,332][20347] Updated weights for policy 0, policy_version 1472345 (0.0008) [2023-12-27 15:50:53,394][20347] Updated weights for policy 0, policy_version 1472355 (0.0006) [2023-12-27 15:50:53,435][20381] Updated weights for policy 1, policy_version 1470917 (0.0007) [2023-12-27 15:50:53,450][20347] Updated weights for policy 0, policy_version 1472365 (0.0005) [2023-12-27 15:50:53,489][20381] Updated weights for policy 1, policy_version 1470927 (0.0007) [2023-12-27 15:50:53,507][20347] Updated weights for policy 0, policy_version 1472375 (0.0005) [2023-12-27 15:50:53,538][20381] Updated weights for policy 1, policy_version 1470937 (0.0005) [2023-12-27 15:50:54,028][20347] Updated weights for policy 0, policy_version 1472385 (0.0007) [2023-12-27 15:50:54,072][20347] Updated weights for policy 0, policy_version 1472395 (0.0007) [2023-12-27 15:50:54,126][20347] Updated weights for policy 0, policy_version 1472405 (0.0007) [2023-12-27 15:50:54,242][20381] Updated weights for policy 1, policy_version 1470947 (0.0006) [2023-12-27 15:50:54,301][20381] Updated weights for policy 1, policy_version 1470957 (0.0008) [2023-12-27 15:50:54,357][20381] Updated weights for policy 1, policy_version 1470967 (0.0008) [2023-12-27 15:50:54,734][20347] Updated weights for policy 0, policy_version 1472415 (0.0006) [2023-12-27 15:50:54,779][20347] Updated weights for policy 0, policy_version 1472425 (0.0005) [2023-12-27 15:50:54,829][20347] Updated weights for policy 0, policy_version 1472435 (0.0005) [2023-12-27 15:50:55,096][20381] Updated weights for policy 1, policy_version 1470977 (0.0009) [2023-12-27 15:50:55,152][20381] Updated weights for policy 1, policy_version 1470987 (0.0009) [2023-12-27 15:50:55,204][20381] Updated weights for policy 1, policy_version 1470997 (0.0007) [2023-12-27 15:50:55,255][20381] Updated weights for policy 1, policy_version 1471007 (0.0006) [2023-12-27 15:50:55,358][20347] Updated weights for policy 0, policy_version 1472445 (0.0007) [2023-12-27 15:50:55,413][20347] Updated weights for policy 0, policy_version 1472455 (0.0008) [2023-12-27 15:50:55,466][20347] Updated weights for policy 0, policy_version 1472465 (0.0008) [2023-12-27 15:50:55,840][20381] Updated weights for policy 1, policy_version 1471017 (0.0008) [2023-12-27 15:50:55,892][20381] Updated weights for policy 1, policy_version 1471027 (0.0007) [2023-12-27 15:50:55,954][20381] Updated weights for policy 1, policy_version 1471037 (0.0008) [2023-12-27 15:50:56,147][20347] Updated weights for policy 0, policy_version 1472475 (0.0007) [2023-12-27 15:50:56,194][20347] Updated weights for policy 0, policy_version 1472485 (0.0007) [2023-12-27 15:50:56,248][20347] Updated weights for policy 0, policy_version 1472495 (0.0007) [2023-12-27 15:50:56,636][20381] Updated weights for policy 1, policy_version 1471047 (0.0009) [2023-12-27 15:50:56,682][20381] Updated weights for policy 1, policy_version 1471057 (0.0010) [2023-12-27 15:50:56,727][20381] Updated weights for policy 1, policy_version 1471067 (0.0010) [2023-12-27 15:50:56,833][20347] Updated weights for policy 0, policy_version 1472505 (0.0007) [2023-12-27 15:50:56,890][20347] Updated weights for policy 0, policy_version 1472515 (0.0005) [2023-12-27 15:50:56,940][20347] Updated weights for policy 0, policy_version 1472525 (0.0005) [2023-12-27 15:50:56,985][20347] Updated weights for policy 0, policy_version 1472535 (0.0005) [2023-12-27 15:50:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 753672192. Throughput: 0: 10399.8, 1: 10365.9. Samples: 753663868. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:50:57,363][19170] Avg episode reward: [(0, '823.480'), (1, '941.580')] [2023-12-27 15:50:57,382][20381] Updated weights for policy 1, policy_version 1471077 (0.0008) [2023-12-27 15:50:57,437][20381] Updated weights for policy 1, policy_version 1471087 (0.0006) [2023-12-27 15:50:57,499][20381] Updated weights for policy 1, policy_version 1471097 (0.0005) [2023-12-27 15:50:57,555][20347] Updated weights for policy 0, policy_version 1472545 (0.0005) [2023-12-27 15:50:57,614][20347] Updated weights for policy 0, policy_version 1472555 (0.0005) [2023-12-27 15:50:57,672][20347] Updated weights for policy 0, policy_version 1472565 (0.0005) [2023-12-27 15:50:58,057][20381] Updated weights for policy 1, policy_version 1471107 (0.0006) [2023-12-27 15:50:58,105][20381] Updated weights for policy 1, policy_version 1471117 (0.0009) [2023-12-27 15:50:58,159][20381] Updated weights for policy 1, policy_version 1471127 (0.0009) [2023-12-27 15:50:58,227][20347] Updated weights for policy 0, policy_version 1472575 (0.0007) [2023-12-27 15:50:58,292][20347] Updated weights for policy 0, policy_version 1472585 (0.0009) [2023-12-27 15:50:58,356][20347] Updated weights for policy 0, policy_version 1472595 (0.0009) [2023-12-27 15:50:58,957][20381] Updated weights for policy 1, policy_version 1471137 (0.0009) [2023-12-27 15:50:59,017][20381] Updated weights for policy 1, policy_version 1471147 (0.0006) [2023-12-27 15:50:59,068][20381] Updated weights for policy 1, policy_version 1471157 (0.0007) [2023-12-27 15:50:59,084][20347] Updated weights for policy 0, policy_version 1472605 (0.0007) [2023-12-27 15:50:59,124][20381] Updated weights for policy 1, policy_version 1471167 (0.0006) [2023-12-27 15:50:59,141][20347] Updated weights for policy 0, policy_version 1472615 (0.0010) [2023-12-27 15:50:59,199][20347] Updated weights for policy 0, policy_version 1472625 (0.0010) [2023-12-27 15:50:59,834][20381] Updated weights for policy 1, policy_version 1471177 (0.0006) [2023-12-27 15:50:59,891][20381] Updated weights for policy 1, policy_version 1471187 (0.0006) [2023-12-27 15:50:59,950][20347] Updated weights for policy 0, policy_version 1472635 (0.0008) [2023-12-27 15:50:59,953][20381] Updated weights for policy 1, policy_version 1471197 (0.0008) [2023-12-27 15:51:00,014][20347] Updated weights for policy 0, policy_version 1472645 (0.0007) [2023-12-27 15:51:00,081][20347] Updated weights for policy 0, policy_version 1472655 (0.0007) [2023-12-27 15:51:00,602][20381] Updated weights for policy 1, policy_version 1471207 (0.0006) [2023-12-27 15:51:00,657][20381] Updated weights for policy 1, policy_version 1471217 (0.0008) [2023-12-27 15:51:00,714][20381] Updated weights for policy 1, policy_version 1471227 (0.0005) [2023-12-27 15:51:00,787][20347] Updated weights for policy 0, policy_version 1472665 (0.0008) [2023-12-27 15:51:00,838][20347] Updated weights for policy 0, policy_version 1472675 (0.0006) [2023-12-27 15:51:00,889][20347] Updated weights for policy 0, policy_version 1472685 (0.0005) [2023-12-27 15:51:00,947][20347] Updated weights for policy 0, policy_version 1472695 (0.0005) [2023-12-27 15:51:01,397][20381] Updated weights for policy 1, policy_version 1471237 (0.0007) [2023-12-27 15:51:01,465][20381] Updated weights for policy 1, policy_version 1471247 (0.0009) [2023-12-27 15:51:01,524][20381] Updated weights for policy 1, policy_version 1471257 (0.0006) [2023-12-27 15:51:01,635][20347] Updated weights for policy 0, policy_version 1472705 (0.0006) [2023-12-27 15:51:01,688][20347] Updated weights for policy 0, policy_version 1472715 (0.0008) [2023-12-27 15:51:01,748][20347] Updated weights for policy 0, policy_version 1472725 (0.0008) [2023-12-27 15:51:02,172][20381] Updated weights for policy 1, policy_version 1471267 (0.0007) [2023-12-27 15:51:02,225][20381] Updated weights for policy 1, policy_version 1471277 (0.0008) [2023-12-27 15:51:02,277][20381] Updated weights for policy 1, policy_version 1471287 (0.0008) [2023-12-27 15:51:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 753778688. Throughput: 0: 10371.1, 1: 10370.4. Samples: 753787256. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:02,363][19170] Avg episode reward: [(0, '745.290'), (1, '951.260')] [2023-12-27 15:51:02,478][20347] Updated weights for policy 0, policy_version 1472735 (0.0006) [2023-12-27 15:51:02,531][20347] Updated weights for policy 0, policy_version 1472745 (0.0005) [2023-12-27 15:51:02,584][20347] Updated weights for policy 0, policy_version 1472755 (0.0005) [2023-12-27 15:51:02,982][20381] Updated weights for policy 1, policy_version 1471297 (0.0008) [2023-12-27 15:51:03,035][20381] Updated weights for policy 1, policy_version 1471307 (0.0009) [2023-12-27 15:51:03,089][20381] Updated weights for policy 1, policy_version 1471317 (0.0009) [2023-12-27 15:51:03,148][20381] Updated weights for policy 1, policy_version 1471327 (0.0008) [2023-12-27 15:51:03,161][20347] Updated weights for policy 0, policy_version 1472765 (0.0005) [2023-12-27 15:51:03,207][20347] Updated weights for policy 0, policy_version 1472775 (0.0006) [2023-12-27 15:51:03,263][20347] Updated weights for policy 0, policy_version 1472785 (0.0005) [2023-12-27 15:51:03,877][20381] Updated weights for policy 1, policy_version 1471337 (0.0008) [2023-12-27 15:51:03,910][20347] Updated weights for policy 0, policy_version 1472795 (0.0007) [2023-12-27 15:51:03,940][20381] Updated weights for policy 1, policy_version 1471347 (0.0008) [2023-12-27 15:51:03,975][20347] Updated weights for policy 0, policy_version 1472805 (0.0007) [2023-12-27 15:51:03,999][20381] Updated weights for policy 1, policy_version 1471357 (0.0008) [2023-12-27 15:51:04,021][20347] Updated weights for policy 0, policy_version 1472815 (0.0007) [2023-12-27 15:51:04,731][20381] Updated weights for policy 1, policy_version 1471367 (0.0009) [2023-12-27 15:51:04,738][20347] Updated weights for policy 0, policy_version 1472825 (0.0007) [2023-12-27 15:51:04,784][20381] Updated weights for policy 1, policy_version 1471377 (0.0007) [2023-12-27 15:51:04,790][20347] Updated weights for policy 0, policy_version 1472835 (0.0010) [2023-12-27 15:51:04,840][20381] Updated weights for policy 1, policy_version 1471387 (0.0007) [2023-12-27 15:51:04,843][20347] Updated weights for policy 0, policy_version 1472845 (0.0010) [2023-12-27 15:51:04,893][20347] Updated weights for policy 0, policy_version 1472855 (0.0006) [2023-12-27 15:51:05,513][20381] Updated weights for policy 1, policy_version 1471397 (0.0008) [2023-12-27 15:51:05,541][20347] Updated weights for policy 0, policy_version 1472865 (0.0010) [2023-12-27 15:51:05,567][20381] Updated weights for policy 1, policy_version 1471407 (0.0005) [2023-12-27 15:51:05,595][20347] Updated weights for policy 0, policy_version 1472875 (0.0010) [2023-12-27 15:51:05,625][20381] Updated weights for policy 1, policy_version 1471417 (0.0007) [2023-12-27 15:51:05,653][20347] Updated weights for policy 0, policy_version 1472885 (0.0010) [2023-12-27 15:51:06,236][20381] Updated weights for policy 1, policy_version 1471427 (0.0007) [2023-12-27 15:51:06,293][20381] Updated weights for policy 1, policy_version 1471437 (0.0009) [2023-12-27 15:51:06,340][20347] Updated weights for policy 0, policy_version 1472895 (0.0007) [2023-12-27 15:51:06,350][20381] Updated weights for policy 1, policy_version 1471447 (0.0007) [2023-12-27 15:51:06,388][20347] Updated weights for policy 0, policy_version 1472905 (0.0006) [2023-12-27 15:51:06,448][20347] Updated weights for policy 0, policy_version 1472915 (0.0009) [2023-12-27 15:51:06,948][20381] Updated weights for policy 1, policy_version 1471457 (0.0006) [2023-12-27 15:51:07,003][20381] Updated weights for policy 1, policy_version 1471467 (0.0009) [2023-12-27 15:51:07,056][20381] Updated weights for policy 1, policy_version 1471477 (0.0010) [2023-12-27 15:51:07,104][20381] Updated weights for policy 1, policy_version 1471487 (0.0010) [2023-12-27 15:51:07,187][20347] Updated weights for policy 0, policy_version 1472925 (0.0009) [2023-12-27 15:51:07,238][20347] Updated weights for policy 0, policy_version 1472935 (0.0007) [2023-12-27 15:51:07,289][20347] Updated weights for policy 0, policy_version 1472945 (0.0007) [2023-12-27 15:51:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 753885184. Throughput: 0: 10428.0, 1: 10354.7. Samples: 753848988. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:07,363][19170] Avg episode reward: [(0, '745.510'), (1, '951.230')] [2023-12-27 15:51:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001471488_376733696.pth... [2023-12-27 15:51:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001472952_377151488.pth... [2023-12-27 15:51:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001471736_376840192.pth [2023-12-27 15:51:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001470272_376422400.pth [2023-12-27 15:51:07,730][20381] Updated weights for policy 1, policy_version 1471497 (0.0006) [2023-12-27 15:51:07,784][20381] Updated weights for policy 1, policy_version 1471507 (0.0005) [2023-12-27 15:51:07,831][20381] Updated weights for policy 1, policy_version 1471517 (0.0005) [2023-12-27 15:51:07,901][20347] Updated weights for policy 0, policy_version 1472955 (0.0007) [2023-12-27 15:51:07,959][20347] Updated weights for policy 0, policy_version 1472965 (0.0010) [2023-12-27 15:51:08,024][20347] Updated weights for policy 0, policy_version 1472975 (0.0010) [2023-12-27 15:51:08,471][20381] Updated weights for policy 1, policy_version 1471527 (0.0007) [2023-12-27 15:51:08,524][20381] Updated weights for policy 1, policy_version 1471537 (0.0008) [2023-12-27 15:51:08,576][20381] Updated weights for policy 1, policy_version 1471547 (0.0007) [2023-12-27 15:51:08,652][20347] Updated weights for policy 0, policy_version 1472985 (0.0007) [2023-12-27 15:51:08,709][20347] Updated weights for policy 0, policy_version 1472995 (0.0011) [2023-12-27 15:51:08,766][20347] Updated weights for policy 0, policy_version 1473005 (0.0010) [2023-12-27 15:51:08,833][20347] Updated weights for policy 0, policy_version 1473015 (0.0011) [2023-12-27 15:51:09,252][20381] Updated weights for policy 1, policy_version 1471557 (0.0008) [2023-12-27 15:51:09,321][20381] Updated weights for policy 1, policy_version 1471567 (0.0006) [2023-12-27 15:51:09,392][20381] Updated weights for policy 1, policy_version 1471577 (0.0008) [2023-12-27 15:51:09,502][20347] Updated weights for policy 0, policy_version 1473025 (0.0008) [2023-12-27 15:51:09,561][20347] Updated weights for policy 0, policy_version 1473035 (0.0007) [2023-12-27 15:51:09,625][20347] Updated weights for policy 0, policy_version 1473045 (0.0008) [2023-12-27 15:51:10,086][20381] Updated weights for policy 1, policy_version 1471587 (0.0008) [2023-12-27 15:51:10,140][20381] Updated weights for policy 1, policy_version 1471597 (0.0007) [2023-12-27 15:51:10,192][20381] Updated weights for policy 1, policy_version 1471607 (0.0005) [2023-12-27 15:51:10,337][20347] Updated weights for policy 0, policy_version 1473055 (0.0007) [2023-12-27 15:51:10,400][20347] Updated weights for policy 0, policy_version 1473065 (0.0009) [2023-12-27 15:51:10,460][20347] Updated weights for policy 0, policy_version 1473075 (0.0007) [2023-12-27 15:51:10,848][20381] Updated weights for policy 1, policy_version 1471617 (0.0008) [2023-12-27 15:51:10,911][20381] Updated weights for policy 1, policy_version 1471627 (0.0011) [2023-12-27 15:51:10,974][20381] Updated weights for policy 1, policy_version 1471637 (0.0010) [2023-12-27 15:51:11,040][20381] Updated weights for policy 1, policy_version 1471647 (0.0010) [2023-12-27 15:51:11,150][20347] Updated weights for policy 0, policy_version 1473085 (0.0008) [2023-12-27 15:51:11,209][20347] Updated weights for policy 0, policy_version 1473095 (0.0008) [2023-12-27 15:51:11,264][20347] Updated weights for policy 0, policy_version 1473105 (0.0007) [2023-12-27 15:51:11,775][20381] Updated weights for policy 1, policy_version 1471657 (0.0009) [2023-12-27 15:51:11,840][20381] Updated weights for policy 1, policy_version 1471667 (0.0011) [2023-12-27 15:51:11,904][20381] Updated weights for policy 1, policy_version 1471677 (0.0011) [2023-12-27 15:51:12,044][20347] Updated weights for policy 0, policy_version 1473115 (0.0006) [2023-12-27 15:51:12,104][20347] Updated weights for policy 0, policy_version 1473125 (0.0010) [2023-12-27 15:51:12,154][20347] Updated weights for policy 0, policy_version 1473135 (0.0009) [2023-12-27 15:51:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 753983488. Throughput: 0: 10454.2, 1: 10322.8. Samples: 753973652. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:12,363][19170] Avg episode reward: [(0, '754.920'), (1, '931.730')] [2023-12-27 15:51:12,547][20381] Updated weights for policy 1, policy_version 1471687 (0.0007) [2023-12-27 15:51:12,609][20381] Updated weights for policy 1, policy_version 1471697 (0.0008) [2023-12-27 15:51:12,674][20381] Updated weights for policy 1, policy_version 1471707 (0.0006) [2023-12-27 15:51:12,820][20347] Updated weights for policy 0, policy_version 1473145 (0.0010) [2023-12-27 15:51:12,880][20347] Updated weights for policy 0, policy_version 1473155 (0.0006) [2023-12-27 15:51:12,939][20347] Updated weights for policy 0, policy_version 1473165 (0.0007) [2023-12-27 15:51:13,002][20347] Updated weights for policy 0, policy_version 1473175 (0.0010) [2023-12-27 15:51:13,334][20381] Updated weights for policy 1, policy_version 1471717 (0.0006) [2023-12-27 15:51:13,389][20381] Updated weights for policy 1, policy_version 1471727 (0.0008) [2023-12-27 15:51:13,441][20381] Updated weights for policy 1, policy_version 1471737 (0.0006) [2023-12-27 15:51:13,544][20347] Updated weights for policy 0, policy_version 1473185 (0.0010) [2023-12-27 15:51:13,596][20347] Updated weights for policy 0, policy_version 1473195 (0.0008) [2023-12-27 15:51:13,648][20347] Updated weights for policy 0, policy_version 1473205 (0.0005) [2023-12-27 15:51:14,081][20381] Updated weights for policy 1, policy_version 1471747 (0.0007) [2023-12-27 15:51:14,145][20381] Updated weights for policy 1, policy_version 1471757 (0.0011) [2023-12-27 15:51:14,194][20381] Updated weights for policy 1, policy_version 1471767 (0.0011) [2023-12-27 15:51:14,275][20347] Updated weights for policy 0, policy_version 1473215 (0.0007) [2023-12-27 15:51:14,326][20347] Updated weights for policy 0, policy_version 1473225 (0.0006) [2023-12-27 15:51:14,382][20347] Updated weights for policy 0, policy_version 1473235 (0.0005) [2023-12-27 15:51:14,880][20381] Updated weights for policy 1, policy_version 1471777 (0.0010) [2023-12-27 15:51:14,933][20381] Updated weights for policy 1, policy_version 1471787 (0.0008) [2023-12-27 15:51:14,983][20381] Updated weights for policy 1, policy_version 1471797 (0.0007) [2023-12-27 15:51:14,996][20347] Updated weights for policy 0, policy_version 1473245 (0.0008) [2023-12-27 15:51:15,039][20381] Updated weights for policy 1, policy_version 1471807 (0.0006) [2023-12-27 15:51:15,053][20347] Updated weights for policy 0, policy_version 1473255 (0.0010) [2023-12-27 15:51:15,108][20347] Updated weights for policy 0, policy_version 1473265 (0.0008) [2023-12-27 15:51:15,679][20381] Updated weights for policy 1, policy_version 1471817 (0.0007) [2023-12-27 15:51:15,735][20381] Updated weights for policy 1, policy_version 1471827 (0.0007) [2023-12-27 15:51:15,790][20381] Updated weights for policy 1, policy_version 1471837 (0.0008) [2023-12-27 15:51:15,826][20347] Updated weights for policy 0, policy_version 1473275 (0.0009) [2023-12-27 15:51:15,875][20347] Updated weights for policy 0, policy_version 1473285 (0.0010) [2023-12-27 15:51:15,927][20347] Updated weights for policy 0, policy_version 1473295 (0.0010) [2023-12-27 15:51:16,451][20381] Updated weights for policy 1, policy_version 1471847 (0.0008) [2023-12-27 15:51:16,505][20381] Updated weights for policy 1, policy_version 1471857 (0.0008) [2023-12-27 15:51:16,556][20381] Updated weights for policy 1, policy_version 1471867 (0.0008) [2023-12-27 15:51:16,596][20347] Updated weights for policy 0, policy_version 1473305 (0.0010) [2023-12-27 15:51:16,647][20347] Updated weights for policy 0, policy_version 1473315 (0.0007) [2023-12-27 15:51:16,699][20347] Updated weights for policy 0, policy_version 1473325 (0.0007) [2023-12-27 15:51:16,757][20347] Updated weights for policy 0, policy_version 1473335 (0.0008) [2023-12-27 15:51:17,160][20381] Updated weights for policy 1, policy_version 1471877 (0.0008) [2023-12-27 15:51:17,209][20381] Updated weights for policy 1, policy_version 1471887 (0.0008) [2023-12-27 15:51:17,258][20381] Updated weights for policy 1, policy_version 1471897 (0.0008) [2023-12-27 15:51:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 754089984. Throughput: 0: 10466.8, 1: 10326.9. Samples: 754099204. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:17,363][19170] Avg episode reward: [(0, '833.510'), (1, '912.240')] [2023-12-27 15:51:17,379][20347] Updated weights for policy 0, policy_version 1473345 (0.0006) [2023-12-27 15:51:17,428][20347] Updated weights for policy 0, policy_version 1473355 (0.0009) [2023-12-27 15:51:17,477][20347] Updated weights for policy 0, policy_version 1473365 (0.0005) [2023-12-27 15:51:17,971][20381] Updated weights for policy 1, policy_version 1471907 (0.0009) [2023-12-27 15:51:18,032][20381] Updated weights for policy 1, policy_version 1471917 (0.0008) [2023-12-27 15:51:18,098][20381] Updated weights for policy 1, policy_version 1471927 (0.0009) [2023-12-27 15:51:18,122][20347] Updated weights for policy 0, policy_version 1473375 (0.0005) [2023-12-27 15:51:18,180][20347] Updated weights for policy 0, policy_version 1473385 (0.0005) [2023-12-27 15:51:18,233][20347] Updated weights for policy 0, policy_version 1473395 (0.0007) [2023-12-27 15:51:18,749][20381] Updated weights for policy 1, policy_version 1471937 (0.0008) [2023-12-27 15:51:18,806][20381] Updated weights for policy 1, policy_version 1471947 (0.0008) [2023-12-27 15:51:18,872][20381] Updated weights for policy 1, policy_version 1471957 (0.0007) [2023-12-27 15:51:18,878][20347] Updated weights for policy 0, policy_version 1473405 (0.0007) [2023-12-27 15:51:18,939][20381] Updated weights for policy 1, policy_version 1471967 (0.0008) [2023-12-27 15:51:18,946][20347] Updated weights for policy 0, policy_version 1473415 (0.0006) [2023-12-27 15:51:19,011][20347] Updated weights for policy 0, policy_version 1473425 (0.0006) [2023-12-27 15:51:19,692][20381] Updated weights for policy 1, policy_version 1471977 (0.0008) [2023-12-27 15:51:19,743][20347] Updated weights for policy 0, policy_version 1473435 (0.0007) [2023-12-27 15:51:19,756][20381] Updated weights for policy 1, policy_version 1471987 (0.0007) [2023-12-27 15:51:19,801][20347] Updated weights for policy 0, policy_version 1473445 (0.0006) [2023-12-27 15:51:19,814][20381] Updated weights for policy 1, policy_version 1471997 (0.0006) [2023-12-27 15:51:19,872][20347] Updated weights for policy 0, policy_version 1473455 (0.0008) [2023-12-27 15:51:20,493][20381] Updated weights for policy 1, policy_version 1472007 (0.0008) [2023-12-27 15:51:20,548][20381] Updated weights for policy 1, policy_version 1472017 (0.0008) [2023-12-27 15:51:20,552][20347] Updated weights for policy 0, policy_version 1473465 (0.0008) [2023-12-27 15:51:20,607][20381] Updated weights for policy 1, policy_version 1472027 (0.0009) [2023-12-27 15:51:20,612][20347] Updated weights for policy 0, policy_version 1473475 (0.0009) [2023-12-27 15:51:20,676][20347] Updated weights for policy 0, policy_version 1473485 (0.0007) [2023-12-27 15:51:20,734][20347] Updated weights for policy 0, policy_version 1473495 (0.0008) [2023-12-27 15:51:21,305][20381] Updated weights for policy 1, policy_version 1472037 (0.0010) [2023-12-27 15:51:21,370][20381] Updated weights for policy 1, policy_version 1472047 (0.0012) [2023-12-27 15:51:21,433][20381] Updated weights for policy 1, policy_version 1472057 (0.0008) [2023-12-27 15:51:21,488][20347] Updated weights for policy 0, policy_version 1473505 (0.0008) [2023-12-27 15:51:21,550][20347] Updated weights for policy 0, policy_version 1473515 (0.0006) [2023-12-27 15:51:21,613][20347] Updated weights for policy 0, policy_version 1473525 (0.0007) [2023-12-27 15:51:22,136][20381] Updated weights for policy 1, policy_version 1472067 (0.0007) [2023-12-27 15:51:22,189][20381] Updated weights for policy 1, policy_version 1472077 (0.0008) [2023-12-27 15:51:22,243][20381] Updated weights for policy 1, policy_version 1472087 (0.0008) [2023-12-27 15:51:22,330][20347] Updated weights for policy 0, policy_version 1473535 (0.0009) [2023-12-27 15:51:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 754188288. Throughput: 0: 10427.9, 1: 10293.7. Samples: 754159824. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:22,363][19170] Avg episode reward: [(0, '823.750'), (1, '912.230')] [2023-12-27 15:51:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001472096_376889344.pth... [2023-12-27 15:51:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001470880_376578048.pth [2023-12-27 15:51:22,390][20347] Updated weights for policy 0, policy_version 1473545 (0.0007) [2023-12-27 15:51:22,447][20347] Updated weights for policy 0, policy_version 1473555 (0.0008) [2023-12-27 15:51:22,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001473560_377307136.pth... [2023-12-27 15:51:22,478][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001472312_376987648.pth [2023-12-27 15:51:22,980][20381] Updated weights for policy 1, policy_version 1472097 (0.0007) [2023-12-27 15:51:23,042][20381] Updated weights for policy 1, policy_version 1472107 (0.0007) [2023-12-27 15:51:23,098][20381] Updated weights for policy 1, policy_version 1472117 (0.0008) [2023-12-27 15:51:23,156][20381] Updated weights for policy 1, policy_version 1472127 (0.0007) [2023-12-27 15:51:23,160][20347] Updated weights for policy 0, policy_version 1473565 (0.0009) [2023-12-27 15:51:23,215][20347] Updated weights for policy 0, policy_version 1473575 (0.0010) [2023-12-27 15:51:23,279][20347] Updated weights for policy 0, policy_version 1473585 (0.0010) [2023-12-27 15:51:23,857][20381] Updated weights for policy 1, policy_version 1472137 (0.0007) [2023-12-27 15:51:23,912][20381] Updated weights for policy 1, policy_version 1472147 (0.0008) [2023-12-27 15:51:23,951][20347] Updated weights for policy 0, policy_version 1473595 (0.0010) [2023-12-27 15:51:23,969][20381] Updated weights for policy 1, policy_version 1472157 (0.0007) [2023-12-27 15:51:23,985][20291] KL-divergence is very high: 174.2107 [2023-12-27 15:51:24,001][20347] Updated weights for policy 0, policy_version 1473605 (0.0010) [2023-12-27 15:51:24,024][20291] KL-divergence is very high: 173.0072 [2023-12-27 15:51:24,049][20347] Updated weights for policy 0, policy_version 1473615 (0.0010) [2023-12-27 15:51:24,064][20291] KL-divergence is very high: 106.2938 [2023-12-27 15:51:24,631][20381] Updated weights for policy 1, policy_version 1472167 (0.0005) [2023-12-27 15:51:24,677][20381] Updated weights for policy 1, policy_version 1472177 (0.0005) [2023-12-27 15:51:24,699][20347] Updated weights for policy 0, policy_version 1473625 (0.0010) [2023-12-27 15:51:24,731][20381] Updated weights for policy 1, policy_version 1472187 (0.0007) [2023-12-27 15:51:24,748][20347] Updated weights for policy 0, policy_version 1473635 (0.0006) [2023-12-27 15:51:24,797][20347] Updated weights for policy 0, policy_version 1473645 (0.0007) [2023-12-27 15:51:24,843][20347] Updated weights for policy 0, policy_version 1473655 (0.0007) [2023-12-27 15:51:25,403][20381] Updated weights for policy 1, policy_version 1472197 (0.0008) [2023-12-27 15:51:25,451][20381] Updated weights for policy 1, policy_version 1472207 (0.0010) [2023-12-27 15:51:25,504][20381] Updated weights for policy 1, policy_version 1472217 (0.0008) [2023-12-27 15:51:25,555][20347] Updated weights for policy 0, policy_version 1473665 (0.0006) [2023-12-27 15:51:25,618][20347] Updated weights for policy 0, policy_version 1473675 (0.0008) [2023-12-27 15:51:25,669][20347] Updated weights for policy 0, policy_version 1473685 (0.0008) [2023-12-27 15:51:26,151][20381] Updated weights for policy 1, policy_version 1472227 (0.0008) [2023-12-27 15:51:26,209][20381] Updated weights for policy 1, policy_version 1472237 (0.0010) [2023-12-27 15:51:26,266][20381] Updated weights for policy 1, policy_version 1472247 (0.0010) [2023-12-27 15:51:26,338][20347] Updated weights for policy 0, policy_version 1473695 (0.0007) [2023-12-27 15:51:26,386][20347] Updated weights for policy 0, policy_version 1473705 (0.0006) [2023-12-27 15:51:26,431][20347] Updated weights for policy 0, policy_version 1473715 (0.0005) [2023-12-27 15:51:26,908][20381] Updated weights for policy 1, policy_version 1472257 (0.0009) [2023-12-27 15:51:26,966][20381] Updated weights for policy 1, policy_version 1472267 (0.0011) [2023-12-27 15:51:27,012][20381] Updated weights for policy 1, policy_version 1472277 (0.0010) [2023-12-27 15:51:27,014][20347] Updated weights for policy 0, policy_version 1473725 (0.0007) [2023-12-27 15:51:27,057][20381] Updated weights for policy 1, policy_version 1472287 (0.0010) [2023-12-27 15:51:27,065][20347] Updated weights for policy 0, policy_version 1473735 (0.0010) [2023-12-27 15:51:27,129][20347] Updated weights for policy 0, policy_version 1473745 (0.0010) [2023-12-27 15:51:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.2, 300 sec: 20605.0). Total num frames: 754294784. Throughput: 0: 10383.2, 1: 10265.6. Samples: 754280192. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:27,363][19170] Avg episode reward: [(0, '862.910'), (1, '941.480')] [2023-12-27 15:51:27,751][20381] Updated weights for policy 1, policy_version 1472297 (0.0011) [2023-12-27 15:51:27,776][20347] Updated weights for policy 0, policy_version 1473755 (0.0009) [2023-12-27 15:51:27,796][20381] Updated weights for policy 1, policy_version 1472307 (0.0010) [2023-12-27 15:51:27,826][20347] Updated weights for policy 0, policy_version 1473765 (0.0007) [2023-12-27 15:51:27,838][20381] Updated weights for policy 1, policy_version 1472317 (0.0010) [2023-12-27 15:51:27,881][20347] Updated weights for policy 0, policy_version 1473775 (0.0007) [2023-12-27 15:51:28,544][20347] Updated weights for policy 0, policy_version 1473785 (0.0008) [2023-12-27 15:51:28,549][20381] Updated weights for policy 1, policy_version 1472327 (0.0011) [2023-12-27 15:51:28,596][20347] Updated weights for policy 0, policy_version 1473795 (0.0010) [2023-12-27 15:51:28,608][20381] Updated weights for policy 1, policy_version 1472337 (0.0011) [2023-12-27 15:51:28,647][20347] Updated weights for policy 0, policy_version 1473805 (0.0010) [2023-12-27 15:51:28,663][20381] Updated weights for policy 1, policy_version 1472347 (0.0010) [2023-12-27 15:51:28,695][20347] Updated weights for policy 0, policy_version 1473815 (0.0010) [2023-12-27 15:51:29,368][20381] Updated weights for policy 1, policy_version 1472357 (0.0010) [2023-12-27 15:51:29,400][20347] Updated weights for policy 0, policy_version 1473825 (0.0008) [2023-12-27 15:51:29,433][20381] Updated weights for policy 1, policy_version 1472367 (0.0011) [2023-12-27 15:51:29,463][20347] Updated weights for policy 0, policy_version 1473835 (0.0007) [2023-12-27 15:51:29,489][20381] Updated weights for policy 1, policy_version 1472377 (0.0011) [2023-12-27 15:51:29,517][20347] Updated weights for policy 0, policy_version 1473845 (0.0008) [2023-12-27 15:51:30,136][20347] Updated weights for policy 0, policy_version 1473855 (0.0006) [2023-12-27 15:51:30,188][20381] Updated weights for policy 1, policy_version 1472387 (0.0010) [2023-12-27 15:51:30,193][20347] Updated weights for policy 0, policy_version 1473865 (0.0010) [2023-12-27 15:51:30,242][20381] Updated weights for policy 1, policy_version 1472397 (0.0010) [2023-12-27 15:51:30,251][20347] Updated weights for policy 0, policy_version 1473875 (0.0010) [2023-12-27 15:51:30,296][20381] Updated weights for policy 1, policy_version 1472407 (0.0010) [2023-12-27 15:51:30,827][20347] Updated weights for policy 0, policy_version 1473885 (0.0008) [2023-12-27 15:51:30,885][20347] Updated weights for policy 0, policy_version 1473895 (0.0006) [2023-12-27 15:51:30,922][20381] Updated weights for policy 1, policy_version 1472417 (0.0010) [2023-12-27 15:51:30,941][20347] Updated weights for policy 0, policy_version 1473905 (0.0007) [2023-12-27 15:51:30,972][20381] Updated weights for policy 1, policy_version 1472427 (0.0005) [2023-12-27 15:51:31,029][20381] Updated weights for policy 1, policy_version 1472437 (0.0006) [2023-12-27 15:51:31,089][20381] Updated weights for policy 1, policy_version 1472447 (0.0011) [2023-12-27 15:51:31,636][20347] Updated weights for policy 0, policy_version 1473915 (0.0007) [2023-12-27 15:51:31,693][20347] Updated weights for policy 0, policy_version 1473925 (0.0008) [2023-12-27 15:51:31,757][20347] Updated weights for policy 0, policy_version 1473935 (0.0007) [2023-12-27 15:51:31,767][20381] Updated weights for policy 1, policy_version 1472457 (0.0011) [2023-12-27 15:51:31,830][20381] Updated weights for policy 1, policy_version 1472467 (0.0011) [2023-12-27 15:51:31,891][20381] Updated weights for policy 1, policy_version 1472477 (0.0009) [2023-12-27 15:51:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 754393088. Throughput: 0: 10417.0, 1: 10209.6. Samples: 754406028. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:32,364][19170] Avg episode reward: [(0, '902.190'), (1, '990.230')] [2023-12-27 15:51:32,384][20347] Updated weights for policy 0, policy_version 1473945 (0.0005) [2023-12-27 15:51:32,443][20347] Updated weights for policy 0, policy_version 1473955 (0.0006) [2023-12-27 15:51:32,493][20347] Updated weights for policy 0, policy_version 1473965 (0.0006) [2023-12-27 15:51:32,542][20347] Updated weights for policy 0, policy_version 1473975 (0.0006) [2023-12-27 15:51:32,566][20381] Updated weights for policy 1, policy_version 1472487 (0.0008) [2023-12-27 15:51:32,623][20381] Updated weights for policy 1, policy_version 1472497 (0.0008) [2023-12-27 15:51:32,685][20381] Updated weights for policy 1, policy_version 1472507 (0.0008) [2023-12-27 15:51:33,144][20347] Updated weights for policy 0, policy_version 1473985 (0.0007) [2023-12-27 15:51:33,195][20347] Updated weights for policy 0, policy_version 1473995 (0.0007) [2023-12-27 15:51:33,248][20347] Updated weights for policy 0, policy_version 1474005 (0.0008) [2023-12-27 15:51:33,331][20381] Updated weights for policy 1, policy_version 1472517 (0.0006) [2023-12-27 15:51:33,386][20381] Updated weights for policy 1, policy_version 1472527 (0.0005) [2023-12-27 15:51:33,438][20381] Updated weights for policy 1, policy_version 1472537 (0.0007) [2023-12-27 15:51:33,889][20347] Updated weights for policy 0, policy_version 1474015 (0.0009) [2023-12-27 15:51:33,946][20347] Updated weights for policy 0, policy_version 1474025 (0.0010) [2023-12-27 15:51:34,010][20347] Updated weights for policy 0, policy_version 1474035 (0.0012) [2023-12-27 15:51:34,039][20381] Updated weights for policy 1, policy_version 1472547 (0.0007) [2023-12-27 15:51:34,096][20381] Updated weights for policy 1, policy_version 1472557 (0.0007) [2023-12-27 15:51:34,159][20381] Updated weights for policy 1, policy_version 1472567 (0.0007) [2023-12-27 15:51:34,722][20347] Updated weights for policy 0, policy_version 1474045 (0.0007) [2023-12-27 15:51:34,779][20347] Updated weights for policy 0, policy_version 1474055 (0.0008) [2023-12-27 15:51:34,836][20347] Updated weights for policy 0, policy_version 1474065 (0.0006) [2023-12-27 15:51:34,841][20381] Updated weights for policy 1, policy_version 1472577 (0.0008) [2023-12-27 15:51:34,893][20381] Updated weights for policy 1, policy_version 1472587 (0.0007) [2023-12-27 15:51:34,947][20381] Updated weights for policy 1, policy_version 1472597 (0.0007) [2023-12-27 15:51:35,000][20381] Updated weights for policy 1, policy_version 1472607 (0.0008) [2023-12-27 15:51:35,496][20347] Updated weights for policy 0, policy_version 1474075 (0.0006) [2023-12-27 15:51:35,558][20347] Updated weights for policy 0, policy_version 1474085 (0.0008) [2023-12-27 15:51:35,607][20381] Updated weights for policy 1, policy_version 1472617 (0.0007) [2023-12-27 15:51:35,609][20347] Updated weights for policy 0, policy_version 1474095 (0.0005) [2023-12-27 15:51:35,663][20381] Updated weights for policy 1, policy_version 1472627 (0.0007) [2023-12-27 15:51:35,744][20381] Updated weights for policy 1, policy_version 1472637 (0.0008) [2023-12-27 15:51:36,226][20347] Updated weights for policy 0, policy_version 1474105 (0.0006) [2023-12-27 15:51:36,285][20347] Updated weights for policy 0, policy_version 1474115 (0.0007) [2023-12-27 15:51:36,345][20347] Updated weights for policy 0, policy_version 1474125 (0.0008) [2023-12-27 15:51:36,408][20347] Updated weights for policy 0, policy_version 1474135 (0.0008) [2023-12-27 15:51:36,451][20381] Updated weights for policy 1, policy_version 1472647 (0.0007) [2023-12-27 15:51:36,511][20381] Updated weights for policy 1, policy_version 1472657 (0.0007) [2023-12-27 15:51:36,568][20381] Updated weights for policy 1, policy_version 1472667 (0.0008) [2023-12-27 15:51:37,117][20347] Updated weights for policy 0, policy_version 1474145 (0.0008) [2023-12-27 15:51:37,163][20347] Updated weights for policy 0, policy_version 1474155 (0.0008) [2023-12-27 15:51:37,217][20347] Updated weights for policy 0, policy_version 1474165 (0.0008) [2023-12-27 15:51:37,253][20381] Updated weights for policy 1, policy_version 1472677 (0.0006) [2023-12-27 15:51:37,304][20381] Updated weights for policy 1, policy_version 1472687 (0.0006) [2023-12-27 15:51:37,357][20381] Updated weights for policy 1, policy_version 1472697 (0.0007) [2023-12-27 15:51:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 754499584. Throughput: 0: 10449.9, 1: 10227.5. Samples: 754469232. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:37,363][19170] Avg episode reward: [(0, '882.620'), (1, '1000.000')] [2023-12-27 15:51:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001474168_377462784.pth... [2023-12-27 15:51:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001472952_377151488.pth [2023-12-27 15:51:37,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001472704_377044992.pth... [2023-12-27 15:51:37,401][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001471488_376733696.pth [2023-12-27 15:51:37,920][20347] Updated weights for policy 0, policy_version 1474175 (0.0009) [2023-12-27 15:51:37,971][20381] Updated weights for policy 1, policy_version 1472707 (0.0006) [2023-12-27 15:51:37,973][20347] Updated weights for policy 0, policy_version 1474185 (0.0008) [2023-12-27 15:51:38,023][20347] Updated weights for policy 0, policy_version 1474195 (0.0008) [2023-12-27 15:51:38,023][20381] Updated weights for policy 1, policy_version 1472717 (0.0007) [2023-12-27 15:51:38,078][20381] Updated weights for policy 1, policy_version 1472727 (0.0007) [2023-12-27 15:51:38,719][20347] Updated weights for policy 0, policy_version 1474205 (0.0007) [2023-12-27 15:51:38,760][20381] Updated weights for policy 1, policy_version 1472737 (0.0006) [2023-12-27 15:51:38,777][20347] Updated weights for policy 0, policy_version 1474215 (0.0007) [2023-12-27 15:51:38,817][20381] Updated weights for policy 1, policy_version 1472747 (0.0006) [2023-12-27 15:51:38,836][20347] Updated weights for policy 0, policy_version 1474225 (0.0006) [2023-12-27 15:51:38,868][20381] Updated weights for policy 1, policy_version 1472757 (0.0006) [2023-12-27 15:51:38,920][20381] Updated weights for policy 1, policy_version 1472767 (0.0008) [2023-12-27 15:51:39,508][20347] Updated weights for policy 0, policy_version 1474235 (0.0007) [2023-12-27 15:51:39,564][20347] Updated weights for policy 0, policy_version 1474245 (0.0008) [2023-12-27 15:51:39,627][20347] Updated weights for policy 0, policy_version 1474255 (0.0008) [2023-12-27 15:51:39,679][20381] Updated weights for policy 1, policy_version 1472777 (0.0006) [2023-12-27 15:51:39,746][20381] Updated weights for policy 1, policy_version 1472787 (0.0008) [2023-12-27 15:51:39,809][20381] Updated weights for policy 1, policy_version 1472797 (0.0008) [2023-12-27 15:51:40,363][20347] Updated weights for policy 0, policy_version 1474265 (0.0009) [2023-12-27 15:51:40,422][20347] Updated weights for policy 0, policy_version 1474275 (0.0008) [2023-12-27 15:51:40,475][20347] Updated weights for policy 0, policy_version 1474285 (0.0008) [2023-12-27 15:51:40,532][20347] Updated weights for policy 0, policy_version 1474295 (0.0008) [2023-12-27 15:51:40,554][20381] Updated weights for policy 1, policy_version 1472807 (0.0008) [2023-12-27 15:51:40,601][20381] Updated weights for policy 1, policy_version 1472817 (0.0008) [2023-12-27 15:51:40,655][20381] Updated weights for policy 1, policy_version 1472827 (0.0007) [2023-12-27 15:51:41,289][20347] Updated weights for policy 0, policy_version 1474305 (0.0007) [2023-12-27 15:51:41,353][20347] Updated weights for policy 0, policy_version 1474315 (0.0007) [2023-12-27 15:51:41,361][20381] Updated weights for policy 1, policy_version 1472837 (0.0007) [2023-12-27 15:51:41,430][20347] Updated weights for policy 0, policy_version 1474325 (0.0007) [2023-12-27 15:51:41,434][20381] Updated weights for policy 1, policy_version 1472847 (0.0008) [2023-12-27 15:51:41,494][20381] Updated weights for policy 1, policy_version 1472857 (0.0006) [2023-12-27 15:51:42,101][20347] Updated weights for policy 0, policy_version 1474335 (0.0009) [2023-12-27 15:51:42,154][20347] Updated weights for policy 0, policy_version 1474345 (0.0008) [2023-12-27 15:51:42,211][20381] Updated weights for policy 1, policy_version 1472867 (0.0007) [2023-12-27 15:51:42,221][20347] Updated weights for policy 0, policy_version 1474355 (0.0006) [2023-12-27 15:51:42,264][20381] Updated weights for policy 1, policy_version 1472877 (0.0007) [2023-12-27 15:51:42,322][20381] Updated weights for policy 1, policy_version 1472887 (0.0011) [2023-12-27 15:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 754597888. Throughput: 0: 10360.4, 1: 10234.8. Samples: 754590656. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:42,363][19170] Avg episode reward: [(0, '892.640'), (1, '990.260')] [2023-12-27 15:51:42,943][20381] Updated weights for policy 1, policy_version 1472897 (0.0010) [2023-12-27 15:51:42,955][20347] Updated weights for policy 0, policy_version 1474365 (0.0007) [2023-12-27 15:51:42,997][20347] Updated weights for policy 0, policy_version 1474375 (0.0005) [2023-12-27 15:51:42,998][20381] Updated weights for policy 1, policy_version 1472907 (0.0011) [2023-12-27 15:51:43,045][20347] Updated weights for policy 0, policy_version 1474385 (0.0005) [2023-12-27 15:51:43,055][20381] Updated weights for policy 1, policy_version 1472917 (0.0010) [2023-12-27 15:51:43,108][20381] Updated weights for policy 1, policy_version 1472927 (0.0010) [2023-12-27 15:51:43,715][20381] Updated weights for policy 1, policy_version 1472937 (0.0006) [2023-12-27 15:51:43,746][20347] Updated weights for policy 0, policy_version 1474395 (0.0006) [2023-12-27 15:51:43,770][20381] Updated weights for policy 1, policy_version 1472947 (0.0006) [2023-12-27 15:51:43,795][20347] Updated weights for policy 0, policy_version 1474405 (0.0006) [2023-12-27 15:51:43,817][20381] Updated weights for policy 1, policy_version 1472957 (0.0007) [2023-12-27 15:51:43,843][20347] Updated weights for policy 0, policy_version 1474415 (0.0006) [2023-12-27 15:51:44,359][20381] Updated weights for policy 1, policy_version 1472967 (0.0006) [2023-12-27 15:51:44,413][20381] Updated weights for policy 1, policy_version 1472977 (0.0005) [2023-12-27 15:51:44,475][20381] Updated weights for policy 1, policy_version 1472987 (0.0006) [2023-12-27 15:51:44,485][20347] Updated weights for policy 0, policy_version 1474425 (0.0009) [2023-12-27 15:51:44,534][20347] Updated weights for policy 0, policy_version 1474435 (0.0010) [2023-12-27 15:51:44,589][20347] Updated weights for policy 0, policy_version 1474445 (0.0010) [2023-12-27 15:51:44,638][20347] Updated weights for policy 0, policy_version 1474455 (0.0010) [2023-12-27 15:51:45,050][20381] Updated weights for policy 1, policy_version 1472997 (0.0008) [2023-12-27 15:51:45,108][20381] Updated weights for policy 1, policy_version 1473007 (0.0008) [2023-12-27 15:51:45,172][20381] Updated weights for policy 1, policy_version 1473017 (0.0008) [2023-12-27 15:51:45,333][20347] Updated weights for policy 0, policy_version 1474465 (0.0007) [2023-12-27 15:51:45,388][20347] Updated weights for policy 0, policy_version 1474475 (0.0007) [2023-12-27 15:51:45,436][20347] Updated weights for policy 0, policy_version 1474485 (0.0007) [2023-12-27 15:51:45,869][20381] Updated weights for policy 1, policy_version 1473027 (0.0009) [2023-12-27 15:51:45,930][20381] Updated weights for policy 1, policy_version 1473037 (0.0011) [2023-12-27 15:51:45,982][20381] Updated weights for policy 1, policy_version 1473047 (0.0010) [2023-12-27 15:51:46,142][20347] Updated weights for policy 0, policy_version 1474495 (0.0007) [2023-12-27 15:51:46,193][20347] Updated weights for policy 0, policy_version 1474505 (0.0007) [2023-12-27 15:51:46,241][20347] Updated weights for policy 0, policy_version 1474515 (0.0006) [2023-12-27 15:51:46,660][20381] Updated weights for policy 1, policy_version 1473057 (0.0011) [2023-12-27 15:51:46,707][20381] Updated weights for policy 1, policy_version 1473067 (0.0009) [2023-12-27 15:51:46,758][20381] Updated weights for policy 1, policy_version 1473077 (0.0005) [2023-12-27 15:51:46,812][20381] Updated weights for policy 1, policy_version 1473087 (0.0005) [2023-12-27 15:51:46,967][20347] Updated weights for policy 0, policy_version 1474525 (0.0007) [2023-12-27 15:51:47,028][20347] Updated weights for policy 0, policy_version 1474535 (0.0007) [2023-12-27 15:51:47,079][20347] Updated weights for policy 0, policy_version 1474545 (0.0005) [2023-12-27 15:51:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 754704384. Throughput: 0: 10309.6, 1: 10327.2. Samples: 754715912. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:47,363][19170] Avg episode reward: [(0, '843.690'), (1, '980.510')] [2023-12-27 15:51:47,380][20381] Updated weights for policy 1, policy_version 1473097 (0.0006) [2023-12-27 15:51:47,425][20381] Updated weights for policy 1, policy_version 1473107 (0.0010) [2023-12-27 15:51:47,478][20292] KL-divergence is very high: 255.1385 [2023-12-27 15:51:47,479][20381] Updated weights for policy 1, policy_version 1473117 (0.0011) [2023-12-27 15:51:47,724][20347] Updated weights for policy 0, policy_version 1474555 (0.0007) [2023-12-27 15:51:47,784][20347] Updated weights for policy 0, policy_version 1474565 (0.0007) [2023-12-27 15:51:47,843][20347] Updated weights for policy 0, policy_version 1474575 (0.0005) [2023-12-27 15:51:48,111][20381] Updated weights for policy 1, policy_version 1473127 (0.0007) [2023-12-27 15:51:48,160][20381] Updated weights for policy 1, policy_version 1473137 (0.0008) [2023-12-27 15:51:48,207][20381] Updated weights for policy 1, policy_version 1473147 (0.0008) [2023-12-27 15:51:48,446][20347] Updated weights for policy 0, policy_version 1474585 (0.0006) [2023-12-27 15:51:48,505][20347] Updated weights for policy 0, policy_version 1474595 (0.0008) [2023-12-27 15:51:48,561][20347] Updated weights for policy 0, policy_version 1474605 (0.0008) [2023-12-27 15:51:48,619][20347] Updated weights for policy 0, policy_version 1474615 (0.0008) [2023-12-27 15:51:48,897][20381] Updated weights for policy 1, policy_version 1473157 (0.0006) [2023-12-27 15:51:48,957][20381] Updated weights for policy 1, policy_version 1473167 (0.0008) [2023-12-27 15:51:49,021][20381] Updated weights for policy 1, policy_version 1473177 (0.0008) [2023-12-27 15:51:49,304][20347] Updated weights for policy 0, policy_version 1474625 (0.0007) [2023-12-27 15:51:49,366][20347] Updated weights for policy 0, policy_version 1474635 (0.0007) [2023-12-27 15:51:49,428][20347] Updated weights for policy 0, policy_version 1474645 (0.0007) [2023-12-27 15:51:49,747][20381] Updated weights for policy 1, policy_version 1473187 (0.0006) [2023-12-27 15:51:49,793][20381] Updated weights for policy 1, policy_version 1473197 (0.0006) [2023-12-27 15:51:49,853][20381] Updated weights for policy 1, policy_version 1473207 (0.0008) [2023-12-27 15:51:50,080][20347] Updated weights for policy 0, policy_version 1474655 (0.0008) [2023-12-27 15:51:50,145][20347] Updated weights for policy 0, policy_version 1474665 (0.0008) [2023-12-27 15:51:50,208][20347] Updated weights for policy 0, policy_version 1474675 (0.0008) [2023-12-27 15:51:50,643][20381] Updated weights for policy 1, policy_version 1473217 (0.0008) [2023-12-27 15:51:50,702][20381] Updated weights for policy 1, policy_version 1473227 (0.0008) [2023-12-27 15:51:50,753][20381] Updated weights for policy 1, policy_version 1473237 (0.0010) [2023-12-27 15:51:50,809][20381] Updated weights for policy 1, policy_version 1473247 (0.0006) [2023-12-27 15:51:50,820][20347] Updated weights for policy 0, policy_version 1474685 (0.0007) [2023-12-27 15:51:50,881][20347] Updated weights for policy 0, policy_version 1474695 (0.0007) [2023-12-27 15:51:50,904][20291] KL-divergence is very high: 133.9168 [2023-12-27 15:51:50,943][20347] Updated weights for policy 0, policy_version 1474705 (0.0008) [2023-12-27 15:51:50,958][20291] KL-divergence is very high: 105.4930 [2023-12-27 15:51:51,526][20381] Updated weights for policy 1, policy_version 1473257 (0.0008) [2023-12-27 15:51:51,579][20381] Updated weights for policy 1, policy_version 1473267 (0.0008) [2023-12-27 15:51:51,640][20381] Updated weights for policy 1, policy_version 1473277 (0.0008) [2023-12-27 15:51:51,696][20347] Updated weights for policy 0, policy_version 1474715 (0.0009) [2023-12-27 15:51:51,762][20347] Updated weights for policy 0, policy_version 1474725 (0.0008) [2023-12-27 15:51:51,823][20347] Updated weights for policy 0, policy_version 1474735 (0.0007) [2023-12-27 15:51:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 754802688. Throughput: 0: 10299.2, 1: 10344.8. Samples: 754777968. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:52,363][19170] Avg episode reward: [(0, '853.520'), (1, '990.250')] [2023-12-27 15:51:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001474744_377610240.pth... [2023-12-27 15:51:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001473560_377307136.pth [2023-12-27 15:51:52,393][20381] Updated weights for policy 1, policy_version 1473287 (0.0010) [2023-12-27 15:51:52,455][20381] Updated weights for policy 1, policy_version 1473297 (0.0008) [2023-12-27 15:51:52,516][20381] Updated weights for policy 1, policy_version 1473307 (0.0008) [2023-12-27 15:51:52,539][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001473312_377200640.pth... [2023-12-27 15:51:52,541][20347] Updated weights for policy 0, policy_version 1474745 (0.0007) [2023-12-27 15:51:52,543][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001472096_376889344.pth [2023-12-27 15:51:52,595][20347] Updated weights for policy 0, policy_version 1474755 (0.0007) [2023-12-27 15:51:52,652][20347] Updated weights for policy 0, policy_version 1474765 (0.0007) [2023-12-27 15:51:52,710][20347] Updated weights for policy 0, policy_version 1474775 (0.0008) [2023-12-27 15:51:53,153][20381] Updated weights for policy 1, policy_version 1473317 (0.0009) [2023-12-27 15:51:53,208][20381] Updated weights for policy 1, policy_version 1473327 (0.0010) [2023-12-27 15:51:53,265][20381] Updated weights for policy 1, policy_version 1473337 (0.0009) [2023-12-27 15:51:53,419][20347] Updated weights for policy 0, policy_version 1474785 (0.0007) [2023-12-27 15:51:53,465][20347] Updated weights for policy 0, policy_version 1474795 (0.0007) [2023-12-27 15:51:53,520][20347] Updated weights for policy 0, policy_version 1474805 (0.0005) [2023-12-27 15:51:53,855][20381] Updated weights for policy 1, policy_version 1473347 (0.0010) [2023-12-27 15:51:53,903][20381] Updated weights for policy 1, policy_version 1473357 (0.0010) [2023-12-27 15:51:53,958][20381] Updated weights for policy 1, policy_version 1473367 (0.0010) [2023-12-27 15:51:54,071][20347] Updated weights for policy 0, policy_version 1474815 (0.0005) [2023-12-27 15:51:54,137][20347] Updated weights for policy 0, policy_version 1474825 (0.0007) [2023-12-27 15:51:54,195][20347] Updated weights for policy 0, policy_version 1474835 (0.0005) [2023-12-27 15:51:54,602][20381] Updated weights for policy 1, policy_version 1473377 (0.0010) [2023-12-27 15:51:54,663][20381] Updated weights for policy 1, policy_version 1473387 (0.0010) [2023-12-27 15:51:54,725][20381] Updated weights for policy 1, policy_version 1473397 (0.0005) [2023-12-27 15:51:54,790][20381] Updated weights for policy 1, policy_version 1473407 (0.0006) [2023-12-27 15:51:54,830][20347] Updated weights for policy 0, policy_version 1474845 (0.0007) [2023-12-27 15:51:54,879][20347] Updated weights for policy 0, policy_version 1474855 (0.0008) [2023-12-27 15:51:54,938][20347] Updated weights for policy 0, policy_version 1474865 (0.0008) [2023-12-27 15:51:55,394][20381] Updated weights for policy 1, policy_version 1473417 (0.0010) [2023-12-27 15:51:55,447][20381] Updated weights for policy 1, policy_version 1473427 (0.0010) [2023-12-27 15:51:55,499][20381] Updated weights for policy 1, policy_version 1473437 (0.0010) [2023-12-27 15:51:55,550][20347] Updated weights for policy 0, policy_version 1474875 (0.0006) [2023-12-27 15:51:55,601][20347] Updated weights for policy 0, policy_version 1474885 (0.0005) [2023-12-27 15:51:55,649][20347] Updated weights for policy 0, policy_version 1474895 (0.0005) [2023-12-27 15:51:56,147][20381] Updated weights for policy 1, policy_version 1473447 (0.0010) [2023-12-27 15:51:56,174][20347] Updated weights for policy 0, policy_version 1474905 (0.0005) [2023-12-27 15:51:56,204][20381] Updated weights for policy 1, policy_version 1473457 (0.0010) [2023-12-27 15:51:56,232][20347] Updated weights for policy 0, policy_version 1474915 (0.0006) [2023-12-27 15:51:56,256][20381] Updated weights for policy 1, policy_version 1473467 (0.0010) [2023-12-27 15:51:56,283][20347] Updated weights for policy 0, policy_version 1474925 (0.0006) [2023-12-27 15:51:56,339][20347] Updated weights for policy 0, policy_version 1474935 (0.0007) [2023-12-27 15:51:56,936][20381] Updated weights for policy 1, policy_version 1473477 (0.0010) [2023-12-27 15:51:56,983][20381] Updated weights for policy 1, policy_version 1473487 (0.0010) [2023-12-27 15:51:57,018][20347] Updated weights for policy 0, policy_version 1474945 (0.0005) [2023-12-27 15:51:57,037][20381] Updated weights for policy 1, policy_version 1473497 (0.0010) [2023-12-27 15:51:57,074][20347] Updated weights for policy 0, policy_version 1474955 (0.0005) [2023-12-27 15:51:57,123][20347] Updated weights for policy 0, policy_version 1474965 (0.0005) [2023-12-27 15:51:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 754917376. Throughput: 0: 10353.4, 1: 10321.5. Samples: 754904016. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:51:57,363][19170] Avg episode reward: [(0, '853.480'), (1, '990.260')] [2023-12-27 15:51:57,699][20381] Updated weights for policy 1, policy_version 1473507 (0.0010) [2023-12-27 15:51:57,702][20347] Updated weights for policy 0, policy_version 1474975 (0.0007) [2023-12-27 15:51:57,747][20381] Updated weights for policy 1, policy_version 1473517 (0.0010) [2023-12-27 15:51:57,756][20347] Updated weights for policy 0, policy_version 1474985 (0.0005) [2023-12-27 15:51:57,793][20381] Updated weights for policy 1, policy_version 1473527 (0.0006) [2023-12-27 15:51:57,814][20347] Updated weights for policy 0, policy_version 1474995 (0.0010) [2023-12-27 15:51:58,503][20347] Updated weights for policy 0, policy_version 1475005 (0.0008) [2023-12-27 15:51:58,532][20381] Updated weights for policy 1, policy_version 1473537 (0.0005) [2023-12-27 15:51:58,562][20347] Updated weights for policy 0, policy_version 1475015 (0.0007) [2023-12-27 15:51:58,590][20381] Updated weights for policy 1, policy_version 1473547 (0.0008) [2023-12-27 15:51:58,630][20347] Updated weights for policy 0, policy_version 1475025 (0.0007) [2023-12-27 15:51:58,656][20381] Updated weights for policy 1, policy_version 1473557 (0.0010) [2023-12-27 15:51:58,720][20381] Updated weights for policy 1, policy_version 1473567 (0.0010) [2023-12-27 15:51:59,390][20347] Updated weights for policy 0, policy_version 1475035 (0.0008) [2023-12-27 15:51:59,449][20381] Updated weights for policy 1, policy_version 1473577 (0.0008) [2023-12-27 15:51:59,475][20347] Updated weights for policy 0, policy_version 1475045 (0.0006) [2023-12-27 15:51:59,507][20381] Updated weights for policy 1, policy_version 1473587 (0.0008) [2023-12-27 15:51:59,528][20347] Updated weights for policy 0, policy_version 1475055 (0.0008) [2023-12-27 15:51:59,568][20381] Updated weights for policy 1, policy_version 1473597 (0.0007) [2023-12-27 15:52:00,099][20347] Updated weights for policy 0, policy_version 1475065 (0.0006) [2023-12-27 15:52:00,162][20347] Updated weights for policy 0, policy_version 1475075 (0.0007) [2023-12-27 15:52:00,219][20347] Updated weights for policy 0, policy_version 1475085 (0.0008) [2023-12-27 15:52:00,276][20347] Updated weights for policy 0, policy_version 1475095 (0.0007) [2023-12-27 15:52:00,285][20381] Updated weights for policy 1, policy_version 1473607 (0.0006) [2023-12-27 15:52:00,337][20381] Updated weights for policy 1, policy_version 1473617 (0.0006) [2023-12-27 15:52:00,393][20381] Updated weights for policy 1, policy_version 1473627 (0.0007) [2023-12-27 15:52:00,941][20347] Updated weights for policy 0, policy_version 1475105 (0.0008) [2023-12-27 15:52:00,996][20347] Updated weights for policy 0, policy_version 1475115 (0.0007) [2023-12-27 15:52:01,058][20347] Updated weights for policy 0, policy_version 1475125 (0.0007) [2023-12-27 15:52:01,081][20381] Updated weights for policy 1, policy_version 1473637 (0.0007) [2023-12-27 15:52:01,132][20381] Updated weights for policy 1, policy_version 1473647 (0.0007) [2023-12-27 15:52:01,196][20381] Updated weights for policy 1, policy_version 1473657 (0.0008) [2023-12-27 15:52:01,737][20347] Updated weights for policy 0, policy_version 1475135 (0.0008) [2023-12-27 15:52:01,801][20347] Updated weights for policy 0, policy_version 1475145 (0.0006) [2023-12-27 15:52:01,860][20347] Updated weights for policy 0, policy_version 1475155 (0.0006) [2023-12-27 15:52:01,914][20381] Updated weights for policy 1, policy_version 1473667 (0.0008) [2023-12-27 15:52:01,978][20381] Updated weights for policy 1, policy_version 1473677 (0.0008) [2023-12-27 15:52:02,027][20381] Updated weights for policy 1, policy_version 1473687 (0.0009) [2023-12-27 15:52:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 755015680. Throughput: 0: 10347.6, 1: 10239.4. Samples: 755025616. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:52:02,363][19170] Avg episode reward: [(0, '843.670'), (1, '990.260')] [2023-12-27 15:52:02,474][20347] Updated weights for policy 0, policy_version 1475165 (0.0008) [2023-12-27 15:52:02,520][20347] Updated weights for policy 0, policy_version 1475175 (0.0008) [2023-12-27 15:52:02,568][20347] Updated weights for policy 0, policy_version 1475185 (0.0007) [2023-12-27 15:52:02,663][20381] Updated weights for policy 1, policy_version 1473697 (0.0007) [2023-12-27 15:52:02,718][20381] Updated weights for policy 1, policy_version 1473707 (0.0008) [2023-12-27 15:52:02,777][20381] Updated weights for policy 1, policy_version 1473717 (0.0008) [2023-12-27 15:52:02,826][20381] Updated weights for policy 1, policy_version 1473727 (0.0007) [2023-12-27 15:52:03,200][20347] Updated weights for policy 0, policy_version 1475195 (0.0008) [2023-12-27 15:52:03,257][20347] Updated weights for policy 0, policy_version 1475205 (0.0005) [2023-12-27 15:52:03,314][20347] Updated weights for policy 0, policy_version 1475215 (0.0005) [2023-12-27 15:52:03,408][20381] Updated weights for policy 1, policy_version 1473737 (0.0005) [2023-12-27 15:52:03,456][20381] Updated weights for policy 1, policy_version 1473747 (0.0006) [2023-12-27 15:52:03,500][20381] Updated weights for policy 1, policy_version 1473757 (0.0007) [2023-12-27 15:52:03,927][20347] Updated weights for policy 0, policy_version 1475225 (0.0005) [2023-12-27 15:52:03,986][20347] Updated weights for policy 0, policy_version 1475235 (0.0008) [2023-12-27 15:52:04,046][20347] Updated weights for policy 0, policy_version 1475245 (0.0007) [2023-12-27 15:52:04,105][20347] Updated weights for policy 0, policy_version 1475255 (0.0008) [2023-12-27 15:52:04,210][20381] Updated weights for policy 1, policy_version 1473767 (0.0008) [2023-12-27 15:52:04,275][20381] Updated weights for policy 1, policy_version 1473777 (0.0008) [2023-12-27 15:52:04,336][20381] Updated weights for policy 1, policy_version 1473787 (0.0008) [2023-12-27 15:52:04,823][20347] Updated weights for policy 0, policy_version 1475265 (0.0007) [2023-12-27 15:52:04,882][20347] Updated weights for policy 0, policy_version 1475275 (0.0007) [2023-12-27 15:52:04,944][20347] Updated weights for policy 0, policy_version 1475285 (0.0007) [2023-12-27 15:52:05,031][20381] Updated weights for policy 1, policy_version 1473797 (0.0006) [2023-12-27 15:52:05,079][20381] Updated weights for policy 1, policy_version 1473807 (0.0008) [2023-12-27 15:52:05,137][20381] Updated weights for policy 1, policy_version 1473817 (0.0008) [2023-12-27 15:52:05,613][20347] Updated weights for policy 0, policy_version 1475295 (0.0009) [2023-12-27 15:52:05,669][20347] Updated weights for policy 0, policy_version 1475305 (0.0008) [2023-12-27 15:52:05,724][20347] Updated weights for policy 0, policy_version 1475315 (0.0010) [2023-12-27 15:52:05,801][20381] Updated weights for policy 1, policy_version 1473827 (0.0007) [2023-12-27 15:52:05,852][20381] Updated weights for policy 1, policy_version 1473837 (0.0008) [2023-12-27 15:52:05,905][20381] Updated weights for policy 1, policy_version 1473847 (0.0011) [2023-12-27 15:52:06,412][20347] Updated weights for policy 0, policy_version 1475325 (0.0008) [2023-12-27 15:52:06,471][20347] Updated weights for policy 0, policy_version 1475335 (0.0007) [2023-12-27 15:52:06,530][20347] Updated weights for policy 0, policy_version 1475345 (0.0007) [2023-12-27 15:52:06,589][20381] Updated weights for policy 1, policy_version 1473857 (0.0010) [2023-12-27 15:52:06,644][20381] Updated weights for policy 1, policy_version 1473867 (0.0010) [2023-12-27 15:52:06,697][20381] Updated weights for policy 1, policy_version 1473877 (0.0010) [2023-12-27 15:52:06,742][20381] Updated weights for policy 1, policy_version 1473887 (0.0010) [2023-12-27 15:52:07,173][20347] Updated weights for policy 0, policy_version 1475355 (0.0007) [2023-12-27 15:52:07,232][20347] Updated weights for policy 0, policy_version 1475365 (0.0007) [2023-12-27 15:52:07,288][20347] Updated weights for policy 0, policy_version 1475375 (0.0007) [2023-12-27 15:52:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 755122176. Throughput: 0: 10356.7, 1: 10287.6. Samples: 755088816. Policy #0 lag: (min: 15.0, avg: 29.4, max: 47.0) [2023-12-27 15:52:07,363][19170] Avg episode reward: [(0, '921.840'), (1, '990.300')] [2023-12-27 15:52:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001473888_377348096.pth... [2023-12-27 15:52:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001475384_377774080.pth... [2023-12-27 15:52:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001472704_377044992.pth [2023-12-27 15:52:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001474168_377462784.pth [2023-12-27 15:52:07,448][20381] Updated weights for policy 1, policy_version 1473897 (0.0007) [2023-12-27 15:52:07,504][20381] Updated weights for policy 1, policy_version 1473907 (0.0007) [2023-12-27 15:52:07,566][20381] Updated weights for policy 1, policy_version 1473917 (0.0007) [2023-12-27 15:52:08,012][20347] Updated weights for policy 0, policy_version 1475385 (0.0007) [2023-12-27 15:52:08,074][20347] Updated weights for policy 0, policy_version 1475395 (0.0008) [2023-12-27 15:52:08,143][20347] Updated weights for policy 0, policy_version 1475405 (0.0007) [2023-12-27 15:52:08,144][20381] Updated weights for policy 1, policy_version 1473927 (0.0007) [2023-12-27 15:52:08,186][20381] Updated weights for policy 1, policy_version 1473937 (0.0006) [2023-12-27 15:52:08,191][20347] Updated weights for policy 0, policy_version 1475415 (0.0005) [2023-12-27 15:52:08,238][20381] Updated weights for policy 1, policy_version 1473947 (0.0007) [2023-12-27 15:52:08,803][20347] Updated weights for policy 0, policy_version 1475425 (0.0010) [2023-12-27 15:52:08,862][20347] Updated weights for policy 0, policy_version 1475435 (0.0010) [2023-12-27 15:52:08,915][20347] Updated weights for policy 0, policy_version 1475445 (0.0010) [2023-12-27 15:52:09,008][20381] Updated weights for policy 1, policy_version 1473957 (0.0006) [2023-12-27 15:52:09,061][20381] Updated weights for policy 1, policy_version 1473967 (0.0005) [2023-12-27 15:52:09,121][20381] Updated weights for policy 1, policy_version 1473977 (0.0005) [2023-12-27 15:52:09,613][20347] Updated weights for policy 0, policy_version 1475455 (0.0008) [2023-12-27 15:52:09,677][20347] Updated weights for policy 0, policy_version 1475465 (0.0005) [2023-12-27 15:52:09,744][20347] Updated weights for policy 0, policy_version 1475475 (0.0009) [2023-12-27 15:52:09,760][20381] Updated weights for policy 1, policy_version 1473987 (0.0006) [2023-12-27 15:52:09,828][20381] Updated weights for policy 1, policy_version 1473997 (0.0008) [2023-12-27 15:52:09,896][20381] Updated weights for policy 1, policy_version 1474007 (0.0007) [2023-12-27 15:52:10,426][20347] Updated weights for policy 0, policy_version 1475485 (0.0011) [2023-12-27 15:52:10,488][20347] Updated weights for policy 0, policy_version 1475495 (0.0011) [2023-12-27 15:52:10,539][20347] Updated weights for policy 0, policy_version 1475505 (0.0006) [2023-12-27 15:52:10,588][20381] Updated weights for policy 1, policy_version 1474017 (0.0008) [2023-12-27 15:52:10,641][20381] Updated weights for policy 1, policy_version 1474027 (0.0006) [2023-12-27 15:52:10,697][20381] Updated weights for policy 1, policy_version 1474037 (0.0011) [2023-12-27 15:52:10,748][20381] Updated weights for policy 1, policy_version 1474047 (0.0009) [2023-12-27 15:52:11,226][20347] Updated weights for policy 0, policy_version 1475515 (0.0009) [2023-12-27 15:52:11,282][20347] Updated weights for policy 0, policy_version 1475525 (0.0008) [2023-12-27 15:52:11,347][20347] Updated weights for policy 0, policy_version 1475535 (0.0007) [2023-12-27 15:52:11,427][20381] Updated weights for policy 1, policy_version 1474058 (0.0010) [2023-12-27 15:52:11,490][20381] Updated weights for policy 1, policy_version 1474068 (0.0008) [2023-12-27 15:52:11,552][20381] Updated weights for policy 1, policy_version 1474078 (0.0007) [2023-12-27 15:52:12,178][20347] Updated weights for policy 0, policy_version 1475545 (0.0008) [2023-12-27 15:52:12,234][20347] Updated weights for policy 0, policy_version 1475555 (0.0009) [2023-12-27 15:52:12,300][20347] Updated weights for policy 0, policy_version 1475565 (0.0009) [2023-12-27 15:52:12,361][20347] Updated weights for policy 0, policy_version 1475575 (0.0009) [2023-12-27 15:52:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 755212288. Throughput: 0: 10397.1, 1: 10333.1. Samples: 755213052. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:12,363][19170] Avg episode reward: [(0, '941.370'), (1, '1000.000')] [2023-12-27 15:52:12,402][20381] Updated weights for policy 1, policy_version 1474088 (0.0009) [2023-12-27 15:52:12,455][20381] Updated weights for policy 1, policy_version 1474099 (0.0009) [2023-12-27 15:52:12,509][20381] Updated weights for policy 1, policy_version 1474109 (0.0009) [2023-12-27 15:52:13,051][20347] Updated weights for policy 0, policy_version 1475585 (0.0008) [2023-12-27 15:52:13,100][20347] Updated weights for policy 0, policy_version 1475595 (0.0007) [2023-12-27 15:52:13,153][20381] Updated weights for policy 1, policy_version 1474119 (0.0006) [2023-12-27 15:52:13,154][20347] Updated weights for policy 0, policy_version 1475605 (0.0005) [2023-12-27 15:52:13,217][20381] Updated weights for policy 1, policy_version 1474129 (0.0008) [2023-12-27 15:52:13,279][20381] Updated weights for policy 1, policy_version 1474139 (0.0008) [2023-12-27 15:52:13,815][20347] Updated weights for policy 0, policy_version 1475615 (0.0008) [2023-12-27 15:52:13,881][20347] Updated weights for policy 0, policy_version 1475625 (0.0010) [2023-12-27 15:52:13,893][20381] Updated weights for policy 1, policy_version 1474149 (0.0006) [2023-12-27 15:52:13,940][20347] Updated weights for policy 0, policy_version 1475635 (0.0010) [2023-12-27 15:52:13,948][20381] Updated weights for policy 1, policy_version 1474159 (0.0006) [2023-12-27 15:52:13,998][20381] Updated weights for policy 1, policy_version 1474169 (0.0007) [2023-12-27 15:52:14,613][20347] Updated weights for policy 0, policy_version 1475645 (0.0009) [2023-12-27 15:52:14,649][20381] Updated weights for policy 1, policy_version 1474179 (0.0007) [2023-12-27 15:52:14,665][20347] Updated weights for policy 0, policy_version 1475655 (0.0006) [2023-12-27 15:52:14,704][20381] Updated weights for policy 1, policy_version 1474189 (0.0006) [2023-12-27 15:52:14,727][20347] Updated weights for policy 0, policy_version 1475665 (0.0011) [2023-12-27 15:52:14,759][20381] Updated weights for policy 1, policy_version 1474199 (0.0006) [2023-12-27 15:52:15,457][20347] Updated weights for policy 0, policy_version 1475675 (0.0010) [2023-12-27 15:52:15,470][20381] Updated weights for policy 1, policy_version 1474209 (0.0008) [2023-12-27 15:52:15,512][20347] Updated weights for policy 0, policy_version 1475685 (0.0010) [2023-12-27 15:52:15,525][20381] Updated weights for policy 1, policy_version 1474219 (0.0005) [2023-12-27 15:52:15,566][20347] Updated weights for policy 0, policy_version 1475695 (0.0010) [2023-12-27 15:52:15,586][20381] Updated weights for policy 1, policy_version 1474229 (0.0005) [2023-12-27 15:52:15,642][20381] Updated weights for policy 1, policy_version 1474239 (0.0006) [2023-12-27 15:52:16,244][20381] Updated weights for policy 1, policy_version 1474249 (0.0007) [2023-12-27 15:52:16,254][20347] Updated weights for policy 0, policy_version 1475705 (0.0010) [2023-12-27 15:52:16,296][20381] Updated weights for policy 1, policy_version 1474259 (0.0007) [2023-12-27 15:52:16,314][20347] Updated weights for policy 0, policy_version 1475715 (0.0008) [2023-12-27 15:52:16,355][20381] Updated weights for policy 1, policy_version 1474269 (0.0007) [2023-12-27 15:52:16,377][20347] Updated weights for policy 0, policy_version 1475725 (0.0005) [2023-12-27 15:52:16,428][20347] Updated weights for policy 0, policy_version 1475735 (0.0006) [2023-12-27 15:52:16,955][20381] Updated weights for policy 1, policy_version 1474279 (0.0007) [2023-12-27 15:52:16,998][20381] Updated weights for policy 1, policy_version 1474289 (0.0005) [2023-12-27 15:52:17,046][20381] Updated weights for policy 1, policy_version 1474299 (0.0005) [2023-12-27 15:52:17,080][20347] Updated weights for policy 0, policy_version 1475745 (0.0007) [2023-12-27 15:52:17,131][20347] Updated weights for policy 0, policy_version 1475755 (0.0005) [2023-12-27 15:52:17,184][20347] Updated weights for policy 0, policy_version 1475765 (0.0005) [2023-12-27 15:52:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 755326976. Throughput: 0: 10279.4, 1: 10352.8. Samples: 755334480. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:17,364][19170] Avg episode reward: [(0, '883.030'), (1, '1000.000')] [2023-12-27 15:52:17,722][20381] Updated weights for policy 1, policy_version 1474309 (0.0007) [2023-12-27 15:52:17,776][20381] Updated weights for policy 1, policy_version 1474319 (0.0008) [2023-12-27 15:52:17,817][20347] Updated weights for policy 0, policy_version 1475775 (0.0005) [2023-12-27 15:52:17,833][20381] Updated weights for policy 1, policy_version 1474329 (0.0008) [2023-12-27 15:52:17,866][20347] Updated weights for policy 0, policy_version 1475785 (0.0005) [2023-12-27 15:52:17,923][20347] Updated weights for policy 0, policy_version 1475795 (0.0007) [2023-12-27 15:52:18,510][20381] Updated weights for policy 1, policy_version 1474339 (0.0008) [2023-12-27 15:52:18,566][20381] Updated weights for policy 1, policy_version 1474349 (0.0008) [2023-12-27 15:52:18,620][20347] Updated weights for policy 0, policy_version 1475805 (0.0009) [2023-12-27 15:52:18,624][20381] Updated weights for policy 1, policy_version 1474359 (0.0009) [2023-12-27 15:52:18,687][20347] Updated weights for policy 0, policy_version 1475815 (0.0007) [2023-12-27 15:52:18,755][20347] Updated weights for policy 0, policy_version 1475825 (0.0007) [2023-12-27 15:52:19,344][20381] Updated weights for policy 1, policy_version 1474369 (0.0006) [2023-12-27 15:52:19,412][20381] Updated weights for policy 1, policy_version 1474379 (0.0008) [2023-12-27 15:52:19,456][20347] Updated weights for policy 0, policy_version 1475835 (0.0009) [2023-12-27 15:52:19,475][20381] Updated weights for policy 1, policy_version 1474389 (0.0008) [2023-12-27 15:52:19,521][20347] Updated weights for policy 0, policy_version 1475845 (0.0008) [2023-12-27 15:52:19,533][20381] Updated weights for policy 1, policy_version 1474399 (0.0007) [2023-12-27 15:52:19,584][20347] Updated weights for policy 0, policy_version 1475855 (0.0008) [2023-12-27 15:52:20,211][20381] Updated weights for policy 1, policy_version 1474409 (0.0009) [2023-12-27 15:52:20,271][20381] Updated weights for policy 1, policy_version 1474419 (0.0010) [2023-12-27 15:52:20,302][20347] Updated weights for policy 0, policy_version 1475865 (0.0008) [2023-12-27 15:52:20,331][20381] Updated weights for policy 1, policy_version 1474429 (0.0011) [2023-12-27 15:52:20,359][20347] Updated weights for policy 0, policy_version 1475875 (0.0006) [2023-12-27 15:52:20,416][20347] Updated weights for policy 0, policy_version 1475885 (0.0007) [2023-12-27 15:52:20,472][20347] Updated weights for policy 0, policy_version 1475895 (0.0007) [2023-12-27 15:52:21,033][20381] Updated weights for policy 1, policy_version 1474439 (0.0011) [2023-12-27 15:52:21,094][20381] Updated weights for policy 1, policy_version 1474449 (0.0010) [2023-12-27 15:52:21,162][20381] Updated weights for policy 1, policy_version 1474459 (0.0007) [2023-12-27 15:52:21,174][20347] Updated weights for policy 0, policy_version 1475905 (0.0007) [2023-12-27 15:52:21,234][20347] Updated weights for policy 0, policy_version 1475915 (0.0008) [2023-12-27 15:52:21,287][20347] Updated weights for policy 0, policy_version 1475925 (0.0007) [2023-12-27 15:52:21,930][20381] Updated weights for policy 1, policy_version 1474469 (0.0008) [2023-12-27 15:52:21,991][20381] Updated weights for policy 1, policy_version 1474479 (0.0011) [2023-12-27 15:52:21,994][20347] Updated weights for policy 0, policy_version 1475935 (0.0008) [2023-12-27 15:52:22,052][20381] Updated weights for policy 1, policy_version 1474489 (0.0011) [2023-12-27 15:52:22,058][20347] Updated weights for policy 0, policy_version 1475945 (0.0006) [2023-12-27 15:52:22,081][20291] KL-divergence is very high: 106.4144 [2023-12-27 15:52:22,118][20347] Updated weights for policy 0, policy_version 1475955 (0.0006) [2023-12-27 15:52:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 755425280. Throughput: 0: 10241.8, 1: 10319.5. Samples: 755394496. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:22,363][19170] Avg episode reward: [(0, '843.810'), (1, '1000.000')] [2023-12-27 15:52:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001474496_377503744.pth... [2023-12-27 15:52:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001475960_377921536.pth... [2023-12-27 15:52:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001473312_377200640.pth [2023-12-27 15:52:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001474744_377610240.pth [2023-12-27 15:52:22,791][20381] Updated weights for policy 1, policy_version 1474499 (0.0011) [2023-12-27 15:52:22,838][20347] Updated weights for policy 0, policy_version 1475965 (0.0008) [2023-12-27 15:52:22,849][20381] Updated weights for policy 1, policy_version 1474509 (0.0008) [2023-12-27 15:52:22,893][20347] Updated weights for policy 0, policy_version 1475975 (0.0008) [2023-12-27 15:52:22,906][20381] Updated weights for policy 1, policy_version 1474519 (0.0008) [2023-12-27 15:52:22,955][20347] Updated weights for policy 0, policy_version 1475985 (0.0007) [2023-12-27 15:52:23,565][20381] Updated weights for policy 1, policy_version 1474529 (0.0006) [2023-12-27 15:52:23,617][20381] Updated weights for policy 1, policy_version 1474539 (0.0006) [2023-12-27 15:52:23,666][20347] Updated weights for policy 0, policy_version 1475995 (0.0008) [2023-12-27 15:52:23,675][20381] Updated weights for policy 1, policy_version 1474549 (0.0005) [2023-12-27 15:52:23,721][20347] Updated weights for policy 0, policy_version 1476005 (0.0007) [2023-12-27 15:52:23,731][20381] Updated weights for policy 1, policy_version 1474559 (0.0005) [2023-12-27 15:52:23,778][20347] Updated weights for policy 0, policy_version 1476015 (0.0008) [2023-12-27 15:52:24,284][20381] Updated weights for policy 1, policy_version 1474569 (0.0007) [2023-12-27 15:52:24,336][20381] Updated weights for policy 1, policy_version 1474579 (0.0007) [2023-12-27 15:52:24,391][20381] Updated weights for policy 1, policy_version 1474589 (0.0007) [2023-12-27 15:52:24,486][20347] Updated weights for policy 0, policy_version 1476025 (0.0009) [2023-12-27 15:52:24,540][20347] Updated weights for policy 0, policy_version 1476035 (0.0005) [2023-12-27 15:52:24,595][20347] Updated weights for policy 0, policy_version 1476045 (0.0007) [2023-12-27 15:52:24,656][20347] Updated weights for policy 0, policy_version 1476055 (0.0008) [2023-12-27 15:52:25,118][20381] Updated weights for policy 1, policy_version 1474599 (0.0008) [2023-12-27 15:52:25,177][20381] Updated weights for policy 1, policy_version 1474609 (0.0007) [2023-12-27 15:52:25,225][20381] Updated weights for policy 1, policy_version 1474619 (0.0009) [2023-12-27 15:52:25,295][20347] Updated weights for policy 0, policy_version 1476065 (0.0008) [2023-12-27 15:52:25,358][20347] Updated weights for policy 0, policy_version 1476075 (0.0007) [2023-12-27 15:52:25,409][20347] Updated weights for policy 0, policy_version 1476085 (0.0005) [2023-12-27 15:52:25,915][20381] Updated weights for policy 1, policy_version 1474629 (0.0006) [2023-12-27 15:52:25,970][20381] Updated weights for policy 1, policy_version 1474639 (0.0008) [2023-12-27 15:52:25,999][20347] Updated weights for policy 0, policy_version 1476095 (0.0006) [2023-12-27 15:52:26,016][20381] Updated weights for policy 1, policy_version 1474649 (0.0006) [2023-12-27 15:52:26,048][20347] Updated weights for policy 0, policy_version 1476105 (0.0006) [2023-12-27 15:52:26,100][20347] Updated weights for policy 0, policy_version 1476115 (0.0007) [2023-12-27 15:52:26,726][20381] Updated weights for policy 1, policy_version 1474659 (0.0007) [2023-12-27 15:52:26,776][20381] Updated weights for policy 1, policy_version 1474669 (0.0007) [2023-12-27 15:52:26,808][20347] Updated weights for policy 0, policy_version 1476125 (0.0006) [2023-12-27 15:52:26,832][20381] Updated weights for policy 1, policy_version 1474679 (0.0007) [2023-12-27 15:52:26,864][20347] Updated weights for policy 0, policy_version 1476135 (0.0006) [2023-12-27 15:52:26,911][20347] Updated weights for policy 0, policy_version 1476145 (0.0007) [2023-12-27 15:52:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 755523584. Throughput: 0: 10243.6, 1: 10295.8. Samples: 755514932. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:27,363][19170] Avg episode reward: [(0, '872.960'), (1, '1000.000')] [2023-12-27 15:52:27,418][20381] Updated weights for policy 1, policy_version 1474689 (0.0005) [2023-12-27 15:52:27,475][20381] Updated weights for policy 1, policy_version 1474699 (0.0008) [2023-12-27 15:52:27,524][20381] Updated weights for policy 1, policy_version 1474709 (0.0007) [2023-12-27 15:52:27,575][20381] Updated weights for policy 1, policy_version 1474719 (0.0007) [2023-12-27 15:52:27,631][20347] Updated weights for policy 0, policy_version 1476155 (0.0008) [2023-12-27 15:52:27,686][20347] Updated weights for policy 0, policy_version 1476165 (0.0008) [2023-12-27 15:52:27,738][20347] Updated weights for policy 0, policy_version 1476175 (0.0008) [2023-12-27 15:52:28,159][20381] Updated weights for policy 1, policy_version 1474729 (0.0010) [2023-12-27 15:52:28,217][20381] Updated weights for policy 1, policy_version 1474739 (0.0007) [2023-12-27 15:52:28,274][20381] Updated weights for policy 1, policy_version 1474749 (0.0008) [2023-12-27 15:52:28,455][20347] Updated weights for policy 0, policy_version 1476185 (0.0008) [2023-12-27 15:52:28,517][20347] Updated weights for policy 0, policy_version 1476195 (0.0008) [2023-12-27 15:52:28,571][20347] Updated weights for policy 0, policy_version 1476205 (0.0007) [2023-12-27 15:52:28,626][20347] Updated weights for policy 0, policy_version 1476215 (0.0007) [2023-12-27 15:52:28,973][20381] Updated weights for policy 1, policy_version 1474759 (0.0007) [2023-12-27 15:52:29,022][20381] Updated weights for policy 1, policy_version 1474769 (0.0010) [2023-12-27 15:52:29,070][20381] Updated weights for policy 1, policy_version 1474779 (0.0010) [2023-12-27 15:52:29,247][20347] Updated weights for policy 0, policy_version 1476225 (0.0007) [2023-12-27 15:52:29,314][20347] Updated weights for policy 0, policy_version 1476235 (0.0005) [2023-12-27 15:52:29,379][20347] Updated weights for policy 0, policy_version 1476245 (0.0008) [2023-12-27 15:52:29,715][20381] Updated weights for policy 1, policy_version 1474789 (0.0009) [2023-12-27 15:52:29,774][20381] Updated weights for policy 1, policy_version 1474799 (0.0008) [2023-12-27 15:52:29,837][20381] Updated weights for policy 1, policy_version 1474809 (0.0008) [2023-12-27 15:52:30,071][20347] Updated weights for policy 0, policy_version 1476255 (0.0008) [2023-12-27 15:52:30,122][20347] Updated weights for policy 0, policy_version 1476265 (0.0010) [2023-12-27 15:52:30,181][20347] Updated weights for policy 0, policy_version 1476275 (0.0008) [2023-12-27 15:52:30,493][20381] Updated weights for policy 1, policy_version 1474819 (0.0008) [2023-12-27 15:52:30,543][20381] Updated weights for policy 1, policy_version 1474829 (0.0008) [2023-12-27 15:52:30,587][20381] Updated weights for policy 1, policy_version 1474839 (0.0009) [2023-12-27 15:52:30,841][20347] Updated weights for policy 0, policy_version 1476285 (0.0006) [2023-12-27 15:52:30,889][20347] Updated weights for policy 0, policy_version 1476295 (0.0005) [2023-12-27 15:52:30,940][20347] Updated weights for policy 0, policy_version 1476305 (0.0005) [2023-12-27 15:52:31,319][20381] Updated weights for policy 1, policy_version 1474849 (0.0007) [2023-12-27 15:52:31,388][20381] Updated weights for policy 1, policy_version 1474859 (0.0009) [2023-12-27 15:52:31,442][20381] Updated weights for policy 1, policy_version 1474869 (0.0007) [2023-12-27 15:52:31,501][20381] Updated weights for policy 1, policy_version 1474879 (0.0007) [2023-12-27 15:52:31,643][20347] Updated weights for policy 0, policy_version 1476315 (0.0008) [2023-12-27 15:52:31,700][20347] Updated weights for policy 0, policy_version 1476325 (0.0007) [2023-12-27 15:52:31,763][20347] Updated weights for policy 0, policy_version 1476335 (0.0008) [2023-12-27 15:52:32,236][20381] Updated weights for policy 1, policy_version 1474889 (0.0008) [2023-12-27 15:52:32,297][20381] Updated weights for policy 1, policy_version 1474899 (0.0008) [2023-12-27 15:52:32,356][20381] Updated weights for policy 1, policy_version 1474909 (0.0008) [2023-12-27 15:52:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 755621888. Throughput: 0: 10254.7, 1: 10242.9. Samples: 755638308. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:32,363][19170] Avg episode reward: [(0, '902.250'), (1, '1000.000')] [2023-12-27 15:52:32,470][20347] Updated weights for policy 0, policy_version 1476345 (0.0008) [2023-12-27 15:52:32,521][20347] Updated weights for policy 0, policy_version 1476355 (0.0006) [2023-12-27 15:52:32,573][20347] Updated weights for policy 0, policy_version 1476365 (0.0007) [2023-12-27 15:52:32,624][20347] Updated weights for policy 0, policy_version 1476375 (0.0007) [2023-12-27 15:52:33,039][20381] Updated weights for policy 1, policy_version 1474919 (0.0008) [2023-12-27 15:52:33,087][20381] Updated weights for policy 1, policy_version 1474929 (0.0007) [2023-12-27 15:52:33,142][20381] Updated weights for policy 1, policy_version 1474939 (0.0005) [2023-12-27 15:52:33,198][20347] Updated weights for policy 0, policy_version 1476385 (0.0008) [2023-12-27 15:52:33,257][20347] Updated weights for policy 0, policy_version 1476395 (0.0006) [2023-12-27 15:52:33,311][20347] Updated weights for policy 0, policy_version 1476405 (0.0006) [2023-12-27 15:52:33,686][20381] Updated weights for policy 1, policy_version 1474949 (0.0005) [2023-12-27 15:52:33,746][20381] Updated weights for policy 1, policy_version 1474959 (0.0006) [2023-12-27 15:52:33,800][20381] Updated weights for policy 1, policy_version 1474969 (0.0009) [2023-12-27 15:52:33,902][20347] Updated weights for policy 0, policy_version 1476415 (0.0005) [2023-12-27 15:52:33,948][20347] Updated weights for policy 0, policy_version 1476425 (0.0005) [2023-12-27 15:52:33,990][20347] Updated weights for policy 0, policy_version 1476435 (0.0005) [2023-12-27 15:52:34,459][20381] Updated weights for policy 1, policy_version 1474979 (0.0009) [2023-12-27 15:52:34,524][20381] Updated weights for policy 1, policy_version 1474989 (0.0009) [2023-12-27 15:52:34,584][20381] Updated weights for policy 1, policy_version 1474999 (0.0008) [2023-12-27 15:52:34,656][20347] Updated weights for policy 0, policy_version 1476445 (0.0006) [2023-12-27 15:52:34,710][20347] Updated weights for policy 0, policy_version 1476455 (0.0007) [2023-12-27 15:52:34,765][20347] Updated weights for policy 0, policy_version 1476465 (0.0008) [2023-12-27 15:52:35,313][20381] Updated weights for policy 1, policy_version 1475009 (0.0008) [2023-12-27 15:52:35,368][20381] Updated weights for policy 1, policy_version 1475019 (0.0007) [2023-12-27 15:52:35,394][20347] Updated weights for policy 0, policy_version 1476475 (0.0009) [2023-12-27 15:52:35,426][20381] Updated weights for policy 1, policy_version 1475029 (0.0008) [2023-12-27 15:52:35,446][20347] Updated weights for policy 0, policy_version 1476485 (0.0010) [2023-12-27 15:52:35,483][20381] Updated weights for policy 1, policy_version 1475039 (0.0005) [2023-12-27 15:52:35,499][20347] Updated weights for policy 0, policy_version 1476495 (0.0007) [2023-12-27 15:52:36,098][20347] Updated weights for policy 0, policy_version 1476505 (0.0010) [2023-12-27 15:52:36,176][20347] Updated weights for policy 0, policy_version 1476515 (0.0008) [2023-12-27 15:52:36,179][20381] Updated weights for policy 1, policy_version 1475049 (0.0006) [2023-12-27 15:52:36,232][20381] Updated weights for policy 1, policy_version 1475059 (0.0006) [2023-12-27 15:52:36,236][20347] Updated weights for policy 0, policy_version 1476525 (0.0007) [2023-12-27 15:52:36,289][20381] Updated weights for policy 1, policy_version 1475069 (0.0008) [2023-12-27 15:52:36,296][20347] Updated weights for policy 0, policy_version 1476535 (0.0009) [2023-12-27 15:52:36,990][20347] Updated weights for policy 0, policy_version 1476545 (0.0005) [2023-12-27 15:52:36,993][20381] Updated weights for policy 1, policy_version 1475079 (0.0007) [2023-12-27 15:52:37,045][20381] Updated weights for policy 1, policy_version 1475089 (0.0006) [2023-12-27 15:52:37,047][20347] Updated weights for policy 0, policy_version 1476555 (0.0006) [2023-12-27 15:52:37,094][20381] Updated weights for policy 1, policy_version 1475099 (0.0006) [2023-12-27 15:52:37,107][20347] Updated weights for policy 0, policy_version 1476565 (0.0010) [2023-12-27 15:52:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 755736576. Throughput: 0: 10279.2, 1: 10265.9. Samples: 755702500. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:37,364][19170] Avg episode reward: [(0, '775.050'), (1, '1000.000')] [2023-12-27 15:52:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001476568_378077184.pth... [2023-12-27 15:52:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001475104_377659392.pth... [2023-12-27 15:52:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001475384_377774080.pth [2023-12-27 15:52:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001473888_377348096.pth [2023-12-27 15:52:37,707][20347] Updated weights for policy 0, policy_version 1476575 (0.0007) [2023-12-27 15:52:37,756][20347] Updated weights for policy 0, policy_version 1476585 (0.0005) [2023-12-27 15:52:37,777][20381] Updated weights for policy 1, policy_version 1475109 (0.0005) [2023-12-27 15:52:37,805][20347] Updated weights for policy 0, policy_version 1476595 (0.0006) [2023-12-27 15:52:37,826][20381] Updated weights for policy 1, policy_version 1475119 (0.0005) [2023-12-27 15:52:37,875][20381] Updated weights for policy 1, policy_version 1475129 (0.0008) [2023-12-27 15:52:38,514][20347] Updated weights for policy 0, policy_version 1476605 (0.0006) [2023-12-27 15:52:38,562][20347] Updated weights for policy 0, policy_version 1476615 (0.0007) [2023-12-27 15:52:38,609][20381] Updated weights for policy 1, policy_version 1475139 (0.0009) [2023-12-27 15:52:38,613][20347] Updated weights for policy 0, policy_version 1476625 (0.0008) [2023-12-27 15:52:38,667][20381] Updated weights for policy 1, policy_version 1475149 (0.0007) [2023-12-27 15:52:38,730][20381] Updated weights for policy 1, policy_version 1475159 (0.0008) [2023-12-27 15:52:39,245][20347] Updated weights for policy 0, policy_version 1476635 (0.0008) [2023-12-27 15:52:39,300][20347] Updated weights for policy 0, policy_version 1476645 (0.0010) [2023-12-27 15:52:39,360][20347] Updated weights for policy 0, policy_version 1476655 (0.0009) [2023-12-27 15:52:39,464][20381] Updated weights for policy 1, policy_version 1475169 (0.0010) [2023-12-27 15:52:39,527][20381] Updated weights for policy 1, policy_version 1475179 (0.0006) [2023-12-27 15:52:39,592][20381] Updated weights for policy 1, policy_version 1475189 (0.0005) [2023-12-27 15:52:39,659][20381] Updated weights for policy 1, policy_version 1475199 (0.0011) [2023-12-27 15:52:40,068][20347] Updated weights for policy 0, policy_version 1476665 (0.0010) [2023-12-27 15:52:40,126][20347] Updated weights for policy 0, policy_version 1476675 (0.0009) [2023-12-27 15:52:40,182][20347] Updated weights for policy 0, policy_version 1476685 (0.0006) [2023-12-27 15:52:40,233][20347] Updated weights for policy 0, policy_version 1476695 (0.0006) [2023-12-27 15:52:40,339][20381] Updated weights for policy 1, policy_version 1475209 (0.0008) [2023-12-27 15:52:40,401][20381] Updated weights for policy 1, policy_version 1475219 (0.0008) [2023-12-27 15:52:40,460][20381] Updated weights for policy 1, policy_version 1475229 (0.0008) [2023-12-27 15:52:40,897][20347] Updated weights for policy 0, policy_version 1476705 (0.0007) [2023-12-27 15:52:40,953][20347] Updated weights for policy 0, policy_version 1476715 (0.0008) [2023-12-27 15:52:41,013][20347] Updated weights for policy 0, policy_version 1476725 (0.0008) [2023-12-27 15:52:41,180][20381] Updated weights for policy 1, policy_version 1475239 (0.0007) [2023-12-27 15:52:41,244][20381] Updated weights for policy 1, policy_version 1475249 (0.0008) [2023-12-27 15:52:41,306][20381] Updated weights for policy 1, policy_version 1475259 (0.0008) [2023-12-27 15:52:41,807][20347] Updated weights for policy 0, policy_version 1476735 (0.0008) [2023-12-27 15:52:41,870][20347] Updated weights for policy 0, policy_version 1476745 (0.0007) [2023-12-27 15:52:41,929][20347] Updated weights for policy 0, policy_version 1476755 (0.0006) [2023-12-27 15:52:42,073][20381] Updated weights for policy 1, policy_version 1475269 (0.0007) [2023-12-27 15:52:42,132][20381] Updated weights for policy 1, policy_version 1475279 (0.0007) [2023-12-27 15:52:42,188][20381] Updated weights for policy 1, policy_version 1475289 (0.0007) [2023-12-27 15:52:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 755834880. Throughput: 0: 10259.6, 1: 10185.4. Samples: 755824044. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:42,363][19170] Avg episode reward: [(0, '814.420'), (1, '1000.000')] [2023-12-27 15:52:42,666][20347] Updated weights for policy 0, policy_version 1476765 (0.0007) [2023-12-27 15:52:42,731][20347] Updated weights for policy 0, policy_version 1476775 (0.0008) [2023-12-27 15:52:42,786][20347] Updated weights for policy 0, policy_version 1476785 (0.0007) [2023-12-27 15:52:42,849][20381] Updated weights for policy 1, policy_version 1475299 (0.0006) [2023-12-27 15:52:42,914][20381] Updated weights for policy 1, policy_version 1475309 (0.0007) [2023-12-27 15:52:42,963][20381] Updated weights for policy 1, policy_version 1475319 (0.0005) [2023-12-27 15:52:43,366][20347] Updated weights for policy 0, policy_version 1476795 (0.0007) [2023-12-27 15:52:43,412][20347] Updated weights for policy 0, policy_version 1476805 (0.0005) [2023-12-27 15:52:43,460][20347] Updated weights for policy 0, policy_version 1476815 (0.0005) [2023-12-27 15:52:43,574][20381] Updated weights for policy 1, policy_version 1475329 (0.0005) [2023-12-27 15:52:43,634][20381] Updated weights for policy 1, policy_version 1475339 (0.0007) [2023-12-27 15:52:43,684][20381] Updated weights for policy 1, policy_version 1475349 (0.0008) [2023-12-27 15:52:43,736][20381] Updated weights for policy 1, policy_version 1475359 (0.0008) [2023-12-27 15:52:44,040][20347] Updated weights for policy 0, policy_version 1476825 (0.0007) [2023-12-27 15:52:44,099][20347] Updated weights for policy 0, policy_version 1476835 (0.0006) [2023-12-27 15:52:44,152][20347] Updated weights for policy 0, policy_version 1476845 (0.0007) [2023-12-27 15:52:44,211][20347] Updated weights for policy 0, policy_version 1476855 (0.0006) [2023-12-27 15:52:44,467][20381] Updated weights for policy 1, policy_version 1475369 (0.0008) [2023-12-27 15:52:44,516][20381] Updated weights for policy 1, policy_version 1475379 (0.0007) [2023-12-27 15:52:44,567][20381] Updated weights for policy 1, policy_version 1475389 (0.0005) [2023-12-27 15:52:44,815][20347] Updated weights for policy 0, policy_version 1476865 (0.0007) [2023-12-27 15:52:44,874][20347] Updated weights for policy 0, policy_version 1476875 (0.0006) [2023-12-27 15:52:44,934][20347] Updated weights for policy 0, policy_version 1476885 (0.0005) [2023-12-27 15:52:45,289][20381] Updated weights for policy 1, policy_version 1475399 (0.0008) [2023-12-27 15:52:45,340][20381] Updated weights for policy 1, policy_version 1475409 (0.0008) [2023-12-27 15:52:45,386][20381] Updated weights for policy 1, policy_version 1475419 (0.0008) [2023-12-27 15:52:45,535][20347] Updated weights for policy 0, policy_version 1476895 (0.0006) [2023-12-27 15:52:45,584][20347] Updated weights for policy 0, policy_version 1476905 (0.0010) [2023-12-27 15:52:45,636][20347] Updated weights for policy 0, policy_version 1476915 (0.0008) [2023-12-27 15:52:45,985][20381] Updated weights for policy 1, policy_version 1475429 (0.0006) [2023-12-27 15:52:46,047][20381] Updated weights for policy 1, policy_version 1475439 (0.0005) [2023-12-27 15:52:46,111][20381] Updated weights for policy 1, policy_version 1475449 (0.0008) [2023-12-27 15:52:46,339][20347] Updated weights for policy 0, policy_version 1476925 (0.0006) [2023-12-27 15:52:46,385][20347] Updated weights for policy 0, policy_version 1476935 (0.0005) [2023-12-27 15:52:46,438][20347] Updated weights for policy 0, policy_version 1476945 (0.0005) [2023-12-27 15:52:46,797][20381] Updated weights for policy 1, policy_version 1475459 (0.0008) [2023-12-27 15:52:46,849][20381] Updated weights for policy 1, policy_version 1475469 (0.0009) [2023-12-27 15:52:46,901][20381] Updated weights for policy 1, policy_version 1475479 (0.0007) [2023-12-27 15:52:46,990][20347] Updated weights for policy 0, policy_version 1476955 (0.0007) [2023-12-27 15:52:47,043][20347] Updated weights for policy 0, policy_version 1476965 (0.0008) [2023-12-27 15:52:47,097][20347] Updated weights for policy 0, policy_version 1476975 (0.0008) [2023-12-27 15:52:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.9). Total num frames: 755941376. Throughput: 0: 10318.3, 1: 10230.0. Samples: 755950292. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:47,363][19170] Avg episode reward: [(0, '843.900'), (1, '1000.000')] [2023-12-27 15:52:47,557][20381] Updated weights for policy 1, policy_version 1475489 (0.0007) [2023-12-27 15:52:47,619][20381] Updated weights for policy 1, policy_version 1475499 (0.0005) [2023-12-27 15:52:47,681][20381] Updated weights for policy 1, policy_version 1475509 (0.0005) [2023-12-27 15:52:47,737][20381] Updated weights for policy 1, policy_version 1475519 (0.0005) [2023-12-27 15:52:47,804][20347] Updated weights for policy 0, policy_version 1476985 (0.0010) [2023-12-27 15:52:47,857][20347] Updated weights for policy 0, policy_version 1476995 (0.0008) [2023-12-27 15:52:47,918][20347] Updated weights for policy 0, policy_version 1477005 (0.0008) [2023-12-27 15:52:47,974][20347] Updated weights for policy 0, policy_version 1477015 (0.0008) [2023-12-27 15:52:48,311][20381] Updated weights for policy 1, policy_version 1475529 (0.0007) [2023-12-27 15:52:48,376][20381] Updated weights for policy 1, policy_version 1475539 (0.0008) [2023-12-27 15:52:48,438][20381] Updated weights for policy 1, policy_version 1475549 (0.0008) [2023-12-27 15:52:48,660][20347] Updated weights for policy 0, policy_version 1477025 (0.0008) [2023-12-27 15:52:48,706][20347] Updated weights for policy 0, policy_version 1477035 (0.0008) [2023-12-27 15:52:48,756][20347] Updated weights for policy 0, policy_version 1477045 (0.0007) [2023-12-27 15:52:49,122][20381] Updated weights for policy 1, policy_version 1475559 (0.0006) [2023-12-27 15:52:49,177][20381] Updated weights for policy 1, policy_version 1475569 (0.0005) [2023-12-27 15:52:49,235][20381] Updated weights for policy 1, policy_version 1475579 (0.0007) [2023-12-27 15:52:49,516][20347] Updated weights for policy 0, policy_version 1477055 (0.0008) [2023-12-27 15:52:49,573][20347] Updated weights for policy 0, policy_version 1477065 (0.0007) [2023-12-27 15:52:49,635][20347] Updated weights for policy 0, policy_version 1477075 (0.0008) [2023-12-27 15:52:49,945][20381] Updated weights for policy 1, policy_version 1475589 (0.0008) [2023-12-27 15:52:50,008][20381] Updated weights for policy 1, policy_version 1475599 (0.0008) [2023-12-27 15:52:50,064][20381] Updated weights for policy 1, policy_version 1475609 (0.0008) [2023-12-27 15:52:50,379][20347] Updated weights for policy 0, policy_version 1477085 (0.0008) [2023-12-27 15:52:50,439][20347] Updated weights for policy 0, policy_version 1477095 (0.0008) [2023-12-27 15:52:50,495][20347] Updated weights for policy 0, policy_version 1477105 (0.0008) [2023-12-27 15:52:50,768][20381] Updated weights for policy 1, policy_version 1475619 (0.0008) [2023-12-27 15:52:50,823][20381] Updated weights for policy 1, policy_version 1475629 (0.0008) [2023-12-27 15:52:50,876][20381] Updated weights for policy 1, policy_version 1475639 (0.0009) [2023-12-27 15:52:51,235][20347] Updated weights for policy 0, policy_version 1477115 (0.0008) [2023-12-27 15:52:51,291][20347] Updated weights for policy 0, policy_version 1477125 (0.0009) [2023-12-27 15:52:51,352][20347] Updated weights for policy 0, policy_version 1477135 (0.0008) [2023-12-27 15:52:51,591][20381] Updated weights for policy 1, policy_version 1475649 (0.0007) [2023-12-27 15:52:51,658][20381] Updated weights for policy 1, policy_version 1475659 (0.0007) [2023-12-27 15:52:51,725][20381] Updated weights for policy 1, policy_version 1475669 (0.0008) [2023-12-27 15:52:51,790][20381] Updated weights for policy 1, policy_version 1475679 (0.0007) [2023-12-27 15:52:52,040][20347] Updated weights for policy 0, policy_version 1477145 (0.0007) [2023-12-27 15:52:52,099][20347] Updated weights for policy 0, policy_version 1477155 (0.0009) [2023-12-27 15:52:52,158][20347] Updated weights for policy 0, policy_version 1477165 (0.0007) [2023-12-27 15:52:52,209][20347] Updated weights for policy 0, policy_version 1477175 (0.0008) [2023-12-27 15:52:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 756039680. Throughput: 0: 10270.0, 1: 10208.4. Samples: 756010348. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:52,363][19170] Avg episode reward: [(0, '912.020'), (1, '1000.000')] [2023-12-27 15:52:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001477176_378232832.pth... [2023-12-27 15:52:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001475960_377921536.pth [2023-12-27 15:52:52,396][20381] Updated weights for policy 1, policy_version 1475689 (0.0009) [2023-12-27 15:52:52,458][20381] Updated weights for policy 1, policy_version 1475699 (0.0009) [2023-12-27 15:52:52,519][20381] Updated weights for policy 1, policy_version 1475709 (0.0009) [2023-12-27 15:52:52,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001475712_377815040.pth... [2023-12-27 15:52:52,539][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001474496_377503744.pth [2023-12-27 15:52:52,909][20347] Updated weights for policy 0, policy_version 1477185 (0.0006) [2023-12-27 15:52:52,962][20347] Updated weights for policy 0, policy_version 1477195 (0.0005) [2023-12-27 15:52:53,011][20347] Updated weights for policy 0, policy_version 1477205 (0.0005) [2023-12-27 15:52:53,234][20381] Updated weights for policy 1, policy_version 1475719 (0.0006) [2023-12-27 15:52:53,286][20381] Updated weights for policy 1, policy_version 1475729 (0.0005) [2023-12-27 15:52:53,340][20381] Updated weights for policy 1, policy_version 1475739 (0.0005) [2023-12-27 15:52:53,676][20347] Updated weights for policy 0, policy_version 1477215 (0.0007) [2023-12-27 15:52:53,729][20347] Updated weights for policy 0, policy_version 1477225 (0.0007) [2023-12-27 15:52:53,777][20347] Updated weights for policy 0, policy_version 1477235 (0.0007) [2023-12-27 15:52:53,948][20381] Updated weights for policy 1, policy_version 1475749 (0.0009) [2023-12-27 15:52:53,995][20381] Updated weights for policy 1, policy_version 1475759 (0.0006) [2023-12-27 15:52:54,044][20381] Updated weights for policy 1, policy_version 1475769 (0.0010) [2023-12-27 15:52:54,515][20347] Updated weights for policy 0, policy_version 1477245 (0.0008) [2023-12-27 15:52:54,565][20347] Updated weights for policy 0, policy_version 1477255 (0.0006) [2023-12-27 15:52:54,619][20347] Updated weights for policy 0, policy_version 1477265 (0.0007) [2023-12-27 15:52:54,742][20381] Updated weights for policy 1, policy_version 1475779 (0.0010) [2023-12-27 15:52:54,801][20381] Updated weights for policy 1, policy_version 1475789 (0.0010) [2023-12-27 15:52:54,845][20381] Updated weights for policy 1, policy_version 1475799 (0.0008) [2023-12-27 15:52:55,324][20347] Updated weights for policy 0, policy_version 1477275 (0.0008) [2023-12-27 15:52:55,376][20347] Updated weights for policy 0, policy_version 1477285 (0.0009) [2023-12-27 15:52:55,421][20347] Updated weights for policy 0, policy_version 1477295 (0.0008) [2023-12-27 15:52:55,464][20381] Updated weights for policy 1, policy_version 1475809 (0.0010) [2023-12-27 15:52:55,525][20381] Updated weights for policy 1, policy_version 1475819 (0.0008) [2023-12-27 15:52:55,580][20381] Updated weights for policy 1, policy_version 1475829 (0.0008) [2023-12-27 15:52:55,634][20381] Updated weights for policy 1, policy_version 1475839 (0.0008) [2023-12-27 15:52:56,148][20347] Updated weights for policy 0, policy_version 1477305 (0.0009) [2023-12-27 15:52:56,207][20347] Updated weights for policy 0, policy_version 1477315 (0.0007) [2023-12-27 15:52:56,219][20381] Updated weights for policy 1, policy_version 1475849 (0.0007) [2023-12-27 15:52:56,255][20347] Updated weights for policy 0, policy_version 1477325 (0.0006) [2023-12-27 15:52:56,272][20381] Updated weights for policy 1, policy_version 1475859 (0.0006) [2023-12-27 15:52:56,304][20347] Updated weights for policy 0, policy_version 1477335 (0.0005) [2023-12-27 15:52:56,325][20381] Updated weights for policy 1, policy_version 1475869 (0.0006) [2023-12-27 15:52:56,965][20347] Updated weights for policy 0, policy_version 1477345 (0.0007) [2023-12-27 15:52:57,013][20347] Updated weights for policy 0, policy_version 1477355 (0.0007) [2023-12-27 15:52:57,035][20381] Updated weights for policy 1, policy_version 1475879 (0.0007) [2023-12-27 15:52:57,063][20347] Updated weights for policy 0, policy_version 1477365 (0.0006) [2023-12-27 15:52:57,095][20381] Updated weights for policy 1, policy_version 1475889 (0.0007) [2023-12-27 15:52:57,155][20381] Updated weights for policy 1, policy_version 1475899 (0.0008) [2023-12-27 15:52:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 756146176. Throughput: 0: 10213.5, 1: 10226.7. Samples: 756132860. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:52:57,363][19170] Avg episode reward: [(0, '931.450'), (1, '1000.000')] [2023-12-27 15:52:57,779][20347] Updated weights for policy 0, policy_version 1477375 (0.0007) [2023-12-27 15:52:57,813][20381] Updated weights for policy 1, policy_version 1475909 (0.0009) [2023-12-27 15:52:57,837][20347] Updated weights for policy 0, policy_version 1477385 (0.0008) [2023-12-27 15:52:57,865][20381] Updated weights for policy 1, policy_version 1475919 (0.0006) [2023-12-27 15:52:57,890][20347] Updated weights for policy 0, policy_version 1477395 (0.0007) [2023-12-27 15:52:57,919][20381] Updated weights for policy 1, policy_version 1475929 (0.0007) [2023-12-27 15:52:58,561][20347] Updated weights for policy 0, policy_version 1477405 (0.0007) [2023-12-27 15:52:58,627][20381] Updated weights for policy 1, policy_version 1475939 (0.0009) [2023-12-27 15:52:58,627][20347] Updated weights for policy 0, policy_version 1477415 (0.0008) [2023-12-27 15:52:58,692][20381] Updated weights for policy 1, policy_version 1475949 (0.0008) [2023-12-27 15:52:58,693][20347] Updated weights for policy 0, policy_version 1477425 (0.0007) [2023-12-27 15:52:58,757][20381] Updated weights for policy 1, policy_version 1475959 (0.0009) [2023-12-27 15:52:59,434][20347] Updated weights for policy 0, policy_version 1477435 (0.0008) [2023-12-27 15:52:59,498][20347] Updated weights for policy 0, policy_version 1477445 (0.0008) [2023-12-27 15:52:59,544][20381] Updated weights for policy 1, policy_version 1475969 (0.0007) [2023-12-27 15:52:59,556][20347] Updated weights for policy 0, policy_version 1477455 (0.0007) [2023-12-27 15:52:59,596][20381] Updated weights for policy 1, policy_version 1475979 (0.0007) [2023-12-27 15:52:59,661][20381] Updated weights for policy 1, policy_version 1475989 (0.0008) [2023-12-27 15:52:59,716][20381] Updated weights for policy 1, policy_version 1475999 (0.0008) [2023-12-27 15:53:00,277][20347] Updated weights for policy 0, policy_version 1477465 (0.0008) [2023-12-27 15:53:00,331][20347] Updated weights for policy 0, policy_version 1477475 (0.0007) [2023-12-27 15:53:00,333][20381] Updated weights for policy 1, policy_version 1476009 (0.0005) [2023-12-27 15:53:00,391][20347] Updated weights for policy 0, policy_version 1477485 (0.0006) [2023-12-27 15:53:00,396][20381] Updated weights for policy 1, policy_version 1476019 (0.0007) [2023-12-27 15:53:00,438][20347] Updated weights for policy 0, policy_version 1477495 (0.0007) [2023-12-27 15:53:00,450][20381] Updated weights for policy 1, policy_version 1476029 (0.0006) [2023-12-27 15:53:01,063][20381] Updated weights for policy 1, policy_version 1476039 (0.0007) [2023-12-27 15:53:01,116][20381] Updated weights for policy 1, policy_version 1476049 (0.0006) [2023-12-27 15:53:01,163][20347] Updated weights for policy 0, policy_version 1477505 (0.0008) [2023-12-27 15:53:01,177][20381] Updated weights for policy 1, policy_version 1476059 (0.0006) [2023-12-27 15:53:01,218][20347] Updated weights for policy 0, policy_version 1477515 (0.0006) [2023-12-27 15:53:01,283][20347] Updated weights for policy 0, policy_version 1477525 (0.0008) [2023-12-27 15:53:01,840][20381] Updated weights for policy 1, policy_version 1476069 (0.0006) [2023-12-27 15:53:01,892][20381] Updated weights for policy 1, policy_version 1476079 (0.0008) [2023-12-27 15:53:01,946][20381] Updated weights for policy 1, policy_version 1476089 (0.0007) [2023-12-27 15:53:01,963][20347] Updated weights for policy 0, policy_version 1477535 (0.0007) [2023-12-27 15:53:02,017][20347] Updated weights for policy 0, policy_version 1477545 (0.0007) [2023-12-27 15:53:02,075][20347] Updated weights for policy 0, policy_version 1477555 (0.0008) [2023-12-27 15:53:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 756244480. Throughput: 0: 10213.4, 1: 10194.4. Samples: 756252828. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:02,363][19170] Avg episode reward: [(0, '902.200'), (1, '980.550')] [2023-12-27 15:53:02,620][20381] Updated weights for policy 1, policy_version 1476099 (0.0006) [2023-12-27 15:53:02,666][20381] Updated weights for policy 1, policy_version 1476109 (0.0008) [2023-12-27 15:53:02,722][20381] Updated weights for policy 1, policy_version 1476119 (0.0008) [2023-12-27 15:53:02,795][20347] Updated weights for policy 0, policy_version 1477565 (0.0007) [2023-12-27 15:53:02,850][20347] Updated weights for policy 0, policy_version 1477575 (0.0007) [2023-12-27 15:53:02,900][20347] Updated weights for policy 0, policy_version 1477585 (0.0008) [2023-12-27 15:53:03,418][20381] Updated weights for policy 1, policy_version 1476129 (0.0008) [2023-12-27 15:53:03,482][20381] Updated weights for policy 1, policy_version 1476139 (0.0007) [2023-12-27 15:53:03,527][20381] Updated weights for policy 1, policy_version 1476149 (0.0006) [2023-12-27 15:53:03,570][20381] Updated weights for policy 1, policy_version 1476159 (0.0005) [2023-12-27 15:53:03,605][20347] Updated weights for policy 0, policy_version 1477595 (0.0007) [2023-12-27 15:53:03,648][20347] Updated weights for policy 0, policy_version 1477605 (0.0005) [2023-12-27 15:53:03,701][20347] Updated weights for policy 0, policy_version 1477615 (0.0005) [2023-12-27 15:53:04,148][20381] Updated weights for policy 1, policy_version 1476169 (0.0008) [2023-12-27 15:53:04,202][20381] Updated weights for policy 1, policy_version 1476179 (0.0008) [2023-12-27 15:53:04,255][20381] Updated weights for policy 1, policy_version 1476189 (0.0007) [2023-12-27 15:53:04,402][20347] Updated weights for policy 0, policy_version 1477625 (0.0005) [2023-12-27 15:53:04,459][20347] Updated weights for policy 0, policy_version 1477635 (0.0009) [2023-12-27 15:53:04,511][20347] Updated weights for policy 0, policy_version 1477645 (0.0006) [2023-12-27 15:53:04,570][20347] Updated weights for policy 0, policy_version 1477655 (0.0005) [2023-12-27 15:53:04,906][20381] Updated weights for policy 1, policy_version 1476199 (0.0008) [2023-12-27 15:53:04,965][20381] Updated weights for policy 1, policy_version 1476209 (0.0006) [2023-12-27 15:53:05,016][20381] Updated weights for policy 1, policy_version 1476219 (0.0006) [2023-12-27 15:53:05,141][20347] Updated weights for policy 0, policy_version 1477665 (0.0006) [2023-12-27 15:53:05,206][20347] Updated weights for policy 0, policy_version 1477675 (0.0008) [2023-12-27 15:53:05,263][20347] Updated weights for policy 0, policy_version 1477685 (0.0007) [2023-12-27 15:53:05,611][20381] Updated weights for policy 1, policy_version 1476229 (0.0006) [2023-12-27 15:53:05,661][20381] Updated weights for policy 1, policy_version 1476239 (0.0007) [2023-12-27 15:53:05,716][20381] Updated weights for policy 1, policy_version 1476249 (0.0006) [2023-12-27 15:53:05,959][20347] Updated weights for policy 0, policy_version 1477695 (0.0008) [2023-12-27 15:53:06,019][20347] Updated weights for policy 0, policy_version 1477705 (0.0006) [2023-12-27 15:53:06,077][20347] Updated weights for policy 0, policy_version 1477715 (0.0006) [2023-12-27 15:53:06,369][20381] Updated weights for policy 1, policy_version 1476259 (0.0006) [2023-12-27 15:53:06,424][20381] Updated weights for policy 1, policy_version 1476269 (0.0007) [2023-12-27 15:53:06,488][20381] Updated weights for policy 1, policy_version 1476279 (0.0006) [2023-12-27 15:53:06,742][20347] Updated weights for policy 0, policy_version 1477725 (0.0006) [2023-12-27 15:53:06,798][20347] Updated weights for policy 0, policy_version 1477735 (0.0005) [2023-12-27 15:53:06,805][20291] KL-divergence is very high: 369.5632 [2023-12-27 15:53:06,825][20291] KL-divergence is very high: 407.0489 [2023-12-27 15:53:06,856][20291] KL-divergence is very high: 465.7315 [2023-12-27 15:53:06,861][20347] Updated weights for policy 0, policy_version 1477745 (0.0007) [2023-12-27 15:53:06,874][20291] KL-divergence is very high: 231.8207 [2023-12-27 15:53:07,099][20381] Updated weights for policy 1, policy_version 1476289 (0.0006) [2023-12-27 15:53:07,149][20381] Updated weights for policy 1, policy_version 1476299 (0.0007) [2023-12-27 15:53:07,196][20381] Updated weights for policy 1, policy_version 1476309 (0.0008) [2023-12-27 15:53:07,247][20381] Updated weights for policy 1, policy_version 1476319 (0.0008) [2023-12-27 15:53:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 756350976. Throughput: 0: 10234.2, 1: 10257.3. Samples: 756316616. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:07,364][19170] Avg episode reward: [(0, '912.440'), (1, '951.730')] [2023-12-27 15:53:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001476320_377970688.pth... [2023-12-27 15:53:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001477752_378380288.pth... [2023-12-27 15:53:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001476568_378077184.pth [2023-12-27 15:53:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001475104_377659392.pth [2023-12-27 15:53:07,505][20347] Updated weights for policy 0, policy_version 1477755 (0.0006) [2023-12-27 15:53:07,558][20347] Updated weights for policy 0, policy_version 1477765 (0.0006) [2023-12-27 15:53:07,604][20347] Updated weights for policy 0, policy_version 1477775 (0.0005) [2023-12-27 15:53:07,968][20381] Updated weights for policy 1, policy_version 1476329 (0.0007) [2023-12-27 15:53:08,022][20381] Updated weights for policy 1, policy_version 1476339 (0.0006) [2023-12-27 15:53:08,076][20381] Updated weights for policy 1, policy_version 1476349 (0.0006) [2023-12-27 15:53:08,208][20347] Updated weights for policy 0, policy_version 1477785 (0.0005) [2023-12-27 15:53:08,266][20347] Updated weights for policy 0, policy_version 1477795 (0.0007) [2023-12-27 15:53:08,319][20347] Updated weights for policy 0, policy_version 1477805 (0.0008) [2023-12-27 15:53:08,378][20347] Updated weights for policy 0, policy_version 1477815 (0.0006) [2023-12-27 15:53:08,752][20381] Updated weights for policy 1, policy_version 1476359 (0.0007) [2023-12-27 15:53:08,813][20381] Updated weights for policy 1, policy_version 1476369 (0.0007) [2023-12-27 15:53:08,875][20381] Updated weights for policy 1, policy_version 1476379 (0.0006) [2023-12-27 15:53:09,057][20347] Updated weights for policy 0, policy_version 1477825 (0.0008) [2023-12-27 15:53:09,113][20347] Updated weights for policy 0, policy_version 1477835 (0.0008) [2023-12-27 15:53:09,167][20347] Updated weights for policy 0, policy_version 1477845 (0.0008) [2023-12-27 15:53:09,587][20381] Updated weights for policy 1, policy_version 1476389 (0.0007) [2023-12-27 15:53:09,640][20381] Updated weights for policy 1, policy_version 1476399 (0.0008) [2023-12-27 15:53:09,695][20381] Updated weights for policy 1, policy_version 1476409 (0.0007) [2023-12-27 15:53:10,009][20347] Updated weights for policy 0, policy_version 1477855 (0.0008) [2023-12-27 15:53:10,069][20347] Updated weights for policy 0, policy_version 1477865 (0.0008) [2023-12-27 15:53:10,128][20347] Updated weights for policy 0, policy_version 1477875 (0.0008) [2023-12-27 15:53:10,396][20381] Updated weights for policy 1, policy_version 1476419 (0.0007) [2023-12-27 15:53:10,459][20381] Updated weights for policy 1, policy_version 1476429 (0.0007) [2023-12-27 15:53:10,518][20381] Updated weights for policy 1, policy_version 1476439 (0.0006) [2023-12-27 15:53:10,847][20347] Updated weights for policy 0, policy_version 1477885 (0.0008) [2023-12-27 15:53:10,902][20347] Updated weights for policy 0, policy_version 1477895 (0.0006) [2023-12-27 15:53:10,955][20347] Updated weights for policy 0, policy_version 1477905 (0.0009) [2023-12-27 15:53:11,182][20381] Updated weights for policy 1, policy_version 1476449 (0.0007) [2023-12-27 15:53:11,242][20381] Updated weights for policy 1, policy_version 1476459 (0.0007) [2023-12-27 15:53:11,310][20381] Updated weights for policy 1, policy_version 1476469 (0.0007) [2023-12-27 15:53:11,382][20381] Updated weights for policy 1, policy_version 1476479 (0.0008) [2023-12-27 15:53:11,690][20347] Updated weights for policy 0, policy_version 1477915 (0.0008) [2023-12-27 15:53:11,754][20347] Updated weights for policy 0, policy_version 1477925 (0.0008) [2023-12-27 15:53:11,804][20347] Updated weights for policy 0, policy_version 1477935 (0.0008) [2023-12-27 15:53:12,081][20381] Updated weights for policy 1, policy_version 1476489 (0.0006) [2023-12-27 15:53:12,131][20381] Updated weights for policy 1, policy_version 1476499 (0.0007) [2023-12-27 15:53:12,182][20381] Updated weights for policy 1, policy_version 1476509 (0.0008) [2023-12-27 15:53:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 756449280. Throughput: 0: 10226.8, 1: 10311.9. Samples: 756439172. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:12,363][19170] Avg episode reward: [(0, '912.470'), (1, '961.390')] [2023-12-27 15:53:12,558][20347] Updated weights for policy 0, policy_version 1477945 (0.0008) [2023-12-27 15:53:12,618][20347] Updated weights for policy 0, policy_version 1477955 (0.0007) [2023-12-27 15:53:12,678][20347] Updated weights for policy 0, policy_version 1477965 (0.0008) [2023-12-27 15:53:12,742][20347] Updated weights for policy 0, policy_version 1477975 (0.0008) [2023-12-27 15:53:12,855][20381] Updated weights for policy 1, policy_version 1476519 (0.0006) [2023-12-27 15:53:12,914][20381] Updated weights for policy 1, policy_version 1476529 (0.0006) [2023-12-27 15:53:12,963][20381] Updated weights for policy 1, policy_version 1476539 (0.0005) [2023-12-27 15:53:13,402][20347] Updated weights for policy 0, policy_version 1477985 (0.0008) [2023-12-27 15:53:13,461][20347] Updated weights for policy 0, policy_version 1477995 (0.0007) [2023-12-27 15:53:13,515][20347] Updated weights for policy 0, policy_version 1478005 (0.0007) [2023-12-27 15:53:13,631][20381] Updated weights for policy 1, policy_version 1476549 (0.0008) [2023-12-27 15:53:13,688][20381] Updated weights for policy 1, policy_version 1476559 (0.0009) [2023-12-27 15:53:13,737][20381] Updated weights for policy 1, policy_version 1476569 (0.0008) [2023-12-27 15:53:14,132][20347] Updated weights for policy 0, policy_version 1478015 (0.0008) [2023-12-27 15:53:14,181][20347] Updated weights for policy 0, policy_version 1478025 (0.0009) [2023-12-27 15:53:14,228][20347] Updated weights for policy 0, policy_version 1478035 (0.0007) [2023-12-27 15:53:14,376][20381] Updated weights for policy 1, policy_version 1476579 (0.0006) [2023-12-27 15:53:14,425][20381] Updated weights for policy 1, policy_version 1476589 (0.0010) [2023-12-27 15:53:14,480][20381] Updated weights for policy 1, policy_version 1476599 (0.0010) [2023-12-27 15:53:14,896][20347] Updated weights for policy 0, policy_version 1478045 (0.0008) [2023-12-27 15:53:14,957][20347] Updated weights for policy 0, policy_version 1478055 (0.0008) [2023-12-27 15:53:15,019][20347] Updated weights for policy 0, policy_version 1478065 (0.0008) [2023-12-27 15:53:15,200][20381] Updated weights for policy 1, policy_version 1476609 (0.0010) [2023-12-27 15:53:15,255][20381] Updated weights for policy 1, policy_version 1476619 (0.0009) [2023-12-27 15:53:15,302][20381] Updated weights for policy 1, policy_version 1476629 (0.0007) [2023-12-27 15:53:15,354][20381] Updated weights for policy 1, policy_version 1476639 (0.0008) [2023-12-27 15:53:15,688][20347] Updated weights for policy 0, policy_version 1478075 (0.0008) [2023-12-27 15:53:15,746][20347] Updated weights for policy 0, policy_version 1478085 (0.0008) [2023-12-27 15:53:15,807][20347] Updated weights for policy 0, policy_version 1478095 (0.0008) [2023-12-27 15:53:15,968][20381] Updated weights for policy 1, policy_version 1476649 (0.0007) [2023-12-27 15:53:16,014][20381] Updated weights for policy 1, policy_version 1476659 (0.0008) [2023-12-27 15:53:16,059][20381] Updated weights for policy 1, policy_version 1476669 (0.0008) [2023-12-27 15:53:16,517][20347] Updated weights for policy 0, policy_version 1478105 (0.0008) [2023-12-27 15:53:16,583][20347] Updated weights for policy 0, policy_version 1478115 (0.0008) [2023-12-27 15:53:16,635][20347] Updated weights for policy 0, policy_version 1478125 (0.0007) [2023-12-27 15:53:16,686][20347] Updated weights for policy 0, policy_version 1478135 (0.0008) [2023-12-27 15:53:16,774][20381] Updated weights for policy 1, policy_version 1476679 (0.0007) [2023-12-27 15:53:16,829][20381] Updated weights for policy 1, policy_version 1476689 (0.0008) [2023-12-27 15:53:16,880][20381] Updated weights for policy 1, policy_version 1476699 (0.0007) [2023-12-27 15:53:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 756547584. Throughput: 0: 10217.2, 1: 10282.3. Samples: 756560788. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:17,363][19170] Avg episode reward: [(0, '951.290'), (1, '951.200')] [2023-12-27 15:53:17,394][20347] Updated weights for policy 0, policy_version 1478145 (0.0007) [2023-12-27 15:53:17,450][20347] Updated weights for policy 0, policy_version 1478155 (0.0008) [2023-12-27 15:53:17,501][20347] Updated weights for policy 0, policy_version 1478165 (0.0007) [2023-12-27 15:53:17,569][20381] Updated weights for policy 1, policy_version 1476709 (0.0008) [2023-12-27 15:53:17,617][20381] Updated weights for policy 1, policy_version 1476719 (0.0007) [2023-12-27 15:53:17,665][20381] Updated weights for policy 1, policy_version 1476729 (0.0007) [2023-12-27 15:53:18,206][20347] Updated weights for policy 0, policy_version 1478175 (0.0010) [2023-12-27 15:53:18,261][20347] Updated weights for policy 0, policy_version 1478185 (0.0010) [2023-12-27 15:53:18,310][20347] Updated weights for policy 0, policy_version 1478195 (0.0010) [2023-12-27 15:53:18,407][20381] Updated weights for policy 1, policy_version 1476739 (0.0008) [2023-12-27 15:53:18,469][20381] Updated weights for policy 1, policy_version 1476749 (0.0008) [2023-12-27 15:53:18,529][20381] Updated weights for policy 1, policy_version 1476759 (0.0008) [2023-12-27 15:53:18,989][20347] Updated weights for policy 0, policy_version 1478205 (0.0009) [2023-12-27 15:53:19,033][20347] Updated weights for policy 0, policy_version 1478215 (0.0006) [2023-12-27 15:53:19,076][20347] Updated weights for policy 0, policy_version 1478225 (0.0005) [2023-12-27 15:53:19,243][20381] Updated weights for policy 1, policy_version 1476769 (0.0008) [2023-12-27 15:53:19,302][20381] Updated weights for policy 1, policy_version 1476779 (0.0008) [2023-12-27 15:53:19,361][20381] Updated weights for policy 1, policy_version 1476789 (0.0007) [2023-12-27 15:53:19,425][20381] Updated weights for policy 1, policy_version 1476799 (0.0007) [2023-12-27 15:53:19,784][20347] Updated weights for policy 0, policy_version 1478235 (0.0007) [2023-12-27 15:53:19,850][20347] Updated weights for policy 0, policy_version 1478245 (0.0010) [2023-12-27 15:53:19,913][20347] Updated weights for policy 0, policy_version 1478255 (0.0010) [2023-12-27 15:53:20,111][20381] Updated weights for policy 1, policy_version 1476809 (0.0008) [2023-12-27 15:53:20,175][20381] Updated weights for policy 1, policy_version 1476819 (0.0008) [2023-12-27 15:53:20,239][20381] Updated weights for policy 1, policy_version 1476829 (0.0008) [2023-12-27 15:53:20,659][20347] Updated weights for policy 0, policy_version 1478265 (0.0008) [2023-12-27 15:53:20,720][20347] Updated weights for policy 0, policy_version 1478275 (0.0008) [2023-12-27 15:53:20,781][20347] Updated weights for policy 0, policy_version 1478285 (0.0008) [2023-12-27 15:53:20,843][20347] Updated weights for policy 0, policy_version 1478295 (0.0008) [2023-12-27 15:53:20,925][20381] Updated weights for policy 1, policy_version 1476839 (0.0008) [2023-12-27 15:53:20,988][20381] Updated weights for policy 1, policy_version 1476849 (0.0008) [2023-12-27 15:53:21,057][20381] Updated weights for policy 1, policy_version 1476859 (0.0009) [2023-12-27 15:53:21,578][20347] Updated weights for policy 0, policy_version 1478305 (0.0006) [2023-12-27 15:53:21,647][20347] Updated weights for policy 0, policy_version 1478315 (0.0010) [2023-12-27 15:53:21,708][20347] Updated weights for policy 0, policy_version 1478325 (0.0010) [2023-12-27 15:53:21,846][20381] Updated weights for policy 1, policy_version 1476869 (0.0009) [2023-12-27 15:53:21,899][20381] Updated weights for policy 1, policy_version 1476879 (0.0010) [2023-12-27 15:53:21,956][20381] Updated weights for policy 1, policy_version 1476889 (0.0011) [2023-12-27 15:53:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 756645888. Throughput: 0: 10157.2, 1: 10246.1. Samples: 756620644. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:22,363][19170] Avg episode reward: [(0, '951.310'), (1, '941.560')] [2023-12-27 15:53:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001476896_378118144.pth... [2023-12-27 15:53:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001475712_377815040.pth [2023-12-27 15:53:22,384][20347] Updated weights for policy 0, policy_version 1478335 (0.0010) [2023-12-27 15:53:22,444][20347] Updated weights for policy 0, policy_version 1478345 (0.0010) [2023-12-27 15:53:22,510][20347] Updated weights for policy 0, policy_version 1478355 (0.0010) [2023-12-27 15:53:22,540][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001478360_378535936.pth... [2023-12-27 15:53:22,545][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001477176_378232832.pth [2023-12-27 15:53:22,637][20381] Updated weights for policy 1, policy_version 1476899 (0.0009) [2023-12-27 15:53:22,696][20381] Updated weights for policy 1, policy_version 1476909 (0.0006) [2023-12-27 15:53:22,758][20381] Updated weights for policy 1, policy_version 1476919 (0.0007) [2023-12-27 15:53:23,183][20347] Updated weights for policy 0, policy_version 1478365 (0.0008) [2023-12-27 15:53:23,241][20347] Updated weights for policy 0, policy_version 1478375 (0.0007) [2023-12-27 15:53:23,289][20347] Updated weights for policy 0, policy_version 1478385 (0.0005) [2023-12-27 15:53:23,405][20381] Updated weights for policy 1, policy_version 1476929 (0.0010) [2023-12-27 15:53:23,456][20381] Updated weights for policy 1, policy_version 1476939 (0.0010) [2023-12-27 15:53:23,505][20381] Updated weights for policy 1, policy_version 1476949 (0.0010) [2023-12-27 15:53:23,552][20381] Updated weights for policy 1, policy_version 1476959 (0.0010) [2023-12-27 15:53:23,852][20347] Updated weights for policy 0, policy_version 1478395 (0.0005) [2023-12-27 15:53:23,911][20347] Updated weights for policy 0, policy_version 1478405 (0.0005) [2023-12-27 15:53:23,969][20347] Updated weights for policy 0, policy_version 1478415 (0.0006) [2023-12-27 15:53:24,145][20381] Updated weights for policy 1, policy_version 1476969 (0.0006) [2023-12-27 15:53:24,205][20381] Updated weights for policy 1, policy_version 1476979 (0.0009) [2023-12-27 15:53:24,265][20381] Updated weights for policy 1, policy_version 1476989 (0.0010) [2023-12-27 15:53:24,601][20347] Updated weights for policy 0, policy_version 1478425 (0.0007) [2023-12-27 15:53:24,659][20347] Updated weights for policy 0, policy_version 1478435 (0.0008) [2023-12-27 15:53:24,717][20347] Updated weights for policy 0, policy_version 1478445 (0.0007) [2023-12-27 15:53:24,775][20347] Updated weights for policy 0, policy_version 1478455 (0.0007) [2023-12-27 15:53:24,874][20381] Updated weights for policy 1, policy_version 1476999 (0.0008) [2023-12-27 15:53:24,942][20381] Updated weights for policy 1, policy_version 1477009 (0.0007) [2023-12-27 15:53:25,007][20381] Updated weights for policy 1, policy_version 1477019 (0.0008) [2023-12-27 15:53:25,462][20347] Updated weights for policy 0, policy_version 1478465 (0.0010) [2023-12-27 15:53:25,514][20347] Updated weights for policy 0, policy_version 1478475 (0.0010) [2023-12-27 15:53:25,567][20347] Updated weights for policy 0, policy_version 1478485 (0.0009) [2023-12-27 15:53:25,603][20381] Updated weights for policy 1, policy_version 1477029 (0.0006) [2023-12-27 15:53:25,650][20381] Updated weights for policy 1, policy_version 1477039 (0.0006) [2023-12-27 15:53:25,695][20381] Updated weights for policy 1, policy_version 1477049 (0.0010) [2023-12-27 15:53:26,229][20347] Updated weights for policy 0, policy_version 1478495 (0.0009) [2023-12-27 15:53:26,288][20347] Updated weights for policy 0, policy_version 1478505 (0.0010) [2023-12-27 15:53:26,341][20347] Updated weights for policy 0, policy_version 1478515 (0.0010) [2023-12-27 15:53:26,342][20381] Updated weights for policy 1, policy_version 1477059 (0.0010) [2023-12-27 15:53:26,402][20381] Updated weights for policy 1, policy_version 1477069 (0.0007) [2023-12-27 15:53:26,461][20381] Updated weights for policy 1, policy_version 1477079 (0.0007) [2023-12-27 15:53:26,943][20347] Updated weights for policy 0, policy_version 1478525 (0.0009) [2023-12-27 15:53:26,994][20347] Updated weights for policy 0, policy_version 1478535 (0.0010) [2023-12-27 15:53:27,031][20381] Updated weights for policy 1, policy_version 1477089 (0.0005) [2023-12-27 15:53:27,047][20347] Updated weights for policy 0, policy_version 1478545 (0.0008) [2023-12-27 15:53:27,093][20381] Updated weights for policy 1, policy_version 1477099 (0.0006) [2023-12-27 15:53:27,159][20381] Updated weights for policy 1, policy_version 1477109 (0.0005) [2023-12-27 15:53:27,212][20381] Updated weights for policy 1, policy_version 1477119 (0.0006) [2023-12-27 15:53:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 756760576. Throughput: 0: 10124.3, 1: 10344.3. Samples: 756745128. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:27,363][19170] Avg episode reward: [(0, '882.710'), (1, '970.820')] [2023-12-27 15:53:27,614][20347] Updated weights for policy 0, policy_version 1478555 (0.0007) [2023-12-27 15:53:27,657][20347] Updated weights for policy 0, policy_version 1478565 (0.0007) [2023-12-27 15:53:27,705][20347] Updated weights for policy 0, policy_version 1478575 (0.0006) [2023-12-27 15:53:27,894][20381] Updated weights for policy 1, policy_version 1477129 (0.0008) [2023-12-27 15:53:27,958][20381] Updated weights for policy 1, policy_version 1477139 (0.0007) [2023-12-27 15:53:28,014][20381] Updated weights for policy 1, policy_version 1477149 (0.0008) [2023-12-27 15:53:28,375][20347] Updated weights for policy 0, policy_version 1478585 (0.0008) [2023-12-27 15:53:28,431][20347] Updated weights for policy 0, policy_version 1478595 (0.0008) [2023-12-27 15:53:28,485][20347] Updated weights for policy 0, policy_version 1478605 (0.0007) [2023-12-27 15:53:28,540][20347] Updated weights for policy 0, policy_version 1478615 (0.0008) [2023-12-27 15:53:28,671][20381] Updated weights for policy 1, policy_version 1477159 (0.0007) [2023-12-27 15:53:28,723][20381] Updated weights for policy 1, policy_version 1477169 (0.0007) [2023-12-27 15:53:28,779][20381] Updated weights for policy 1, policy_version 1477179 (0.0007) [2023-12-27 15:53:29,227][20347] Updated weights for policy 0, policy_version 1478625 (0.0006) [2023-12-27 15:53:29,291][20347] Updated weights for policy 0, policy_version 1478635 (0.0009) [2023-12-27 15:53:29,352][20347] Updated weights for policy 0, policy_version 1478645 (0.0008) [2023-12-27 15:53:29,512][20381] Updated weights for policy 1, policy_version 1477189 (0.0008) [2023-12-27 15:53:29,574][20381] Updated weights for policy 1, policy_version 1477199 (0.0008) [2023-12-27 15:53:29,625][20381] Updated weights for policy 1, policy_version 1477209 (0.0008) [2023-12-27 15:53:30,035][20347] Updated weights for policy 0, policy_version 1478655 (0.0011) [2023-12-27 15:53:30,094][20347] Updated weights for policy 0, policy_version 1478665 (0.0008) [2023-12-27 15:53:30,152][20347] Updated weights for policy 0, policy_version 1478675 (0.0010) [2023-12-27 15:53:30,256][20381] Updated weights for policy 1, policy_version 1477219 (0.0007) [2023-12-27 15:53:30,311][20381] Updated weights for policy 1, policy_version 1477229 (0.0008) [2023-12-27 15:53:30,369][20381] Updated weights for policy 1, policy_version 1477239 (0.0007) [2023-12-27 15:53:30,782][20347] Updated weights for policy 0, policy_version 1478685 (0.0010) [2023-12-27 15:53:30,830][20347] Updated weights for policy 0, policy_version 1478695 (0.0010) [2023-12-27 15:53:30,887][20347] Updated weights for policy 0, policy_version 1478705 (0.0010) [2023-12-27 15:53:31,015][20381] Updated weights for policy 1, policy_version 1477249 (0.0007) [2023-12-27 15:53:31,077][20381] Updated weights for policy 1, policy_version 1477259 (0.0010) [2023-12-27 15:53:31,137][20381] Updated weights for policy 1, policy_version 1477269 (0.0011) [2023-12-27 15:53:31,198][20381] Updated weights for policy 1, policy_version 1477279 (0.0010) [2023-12-27 15:53:31,599][20347] Updated weights for policy 0, policy_version 1478715 (0.0009) [2023-12-27 15:53:31,660][20347] Updated weights for policy 0, policy_version 1478725 (0.0008) [2023-12-27 15:53:31,713][20347] Updated weights for policy 0, policy_version 1478735 (0.0007) [2023-12-27 15:53:31,980][20381] Updated weights for policy 1, policy_version 1477289 (0.0011) [2023-12-27 15:53:32,041][20381] Updated weights for policy 1, policy_version 1477299 (0.0011) [2023-12-27 15:53:32,099][20381] Updated weights for policy 1, policy_version 1477309 (0.0011) [2023-12-27 15:53:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 756858880. Throughput: 0: 10072.6, 1: 10347.8. Samples: 756869208. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:32,363][19170] Avg episode reward: [(0, '882.670'), (1, '1000.000')] [2023-12-27 15:53:32,473][20347] Updated weights for policy 0, policy_version 1478745 (0.0008) [2023-12-27 15:53:32,530][20347] Updated weights for policy 0, policy_version 1478755 (0.0008) [2023-12-27 15:53:32,586][20347] Updated weights for policy 0, policy_version 1478765 (0.0007) [2023-12-27 15:53:32,646][20347] Updated weights for policy 0, policy_version 1478775 (0.0006) [2023-12-27 15:53:32,807][20381] Updated weights for policy 1, policy_version 1477319 (0.0010) [2023-12-27 15:53:32,867][20381] Updated weights for policy 1, policy_version 1477329 (0.0008) [2023-12-27 15:53:32,927][20381] Updated weights for policy 1, policy_version 1477339 (0.0008) [2023-12-27 15:53:33,334][20347] Updated weights for policy 0, policy_version 1478785 (0.0007) [2023-12-27 15:53:33,387][20347] Updated weights for policy 0, policy_version 1478795 (0.0007) [2023-12-27 15:53:33,453][20347] Updated weights for policy 0, policy_version 1478805 (0.0007) [2023-12-27 15:53:33,589][20381] Updated weights for policy 1, policy_version 1477349 (0.0009) [2023-12-27 15:53:33,639][20381] Updated weights for policy 1, policy_version 1477359 (0.0008) [2023-12-27 15:53:33,688][20381] Updated weights for policy 1, policy_version 1477369 (0.0008) [2023-12-27 15:53:34,018][20347] Updated weights for policy 0, policy_version 1478815 (0.0005) [2023-12-27 15:53:34,063][20347] Updated weights for policy 0, policy_version 1478825 (0.0005) [2023-12-27 15:53:34,109][20347] Updated weights for policy 0, policy_version 1478835 (0.0006) [2023-12-27 15:53:34,355][20381] Updated weights for policy 1, policy_version 1477379 (0.0008) [2023-12-27 15:53:34,415][20381] Updated weights for policy 1, policy_version 1477389 (0.0008) [2023-12-27 15:53:34,474][20381] Updated weights for policy 1, policy_version 1477399 (0.0007) [2023-12-27 15:53:34,782][20347] Updated weights for policy 0, policy_version 1478845 (0.0006) [2023-12-27 15:53:34,836][20347] Updated weights for policy 0, policy_version 1478855 (0.0006) [2023-12-27 15:53:34,890][20347] Updated weights for policy 0, policy_version 1478865 (0.0007) [2023-12-27 15:53:35,175][20381] Updated weights for policy 1, policy_version 1477409 (0.0006) [2023-12-27 15:53:35,236][20381] Updated weights for policy 1, policy_version 1477419 (0.0008) [2023-12-27 15:53:35,285][20381] Updated weights for policy 1, policy_version 1477429 (0.0008) [2023-12-27 15:53:35,341][20381] Updated weights for policy 1, policy_version 1477439 (0.0007) [2023-12-27 15:53:35,546][20347] Updated weights for policy 0, policy_version 1478875 (0.0007) [2023-12-27 15:53:35,598][20347] Updated weights for policy 0, policy_version 1478885 (0.0005) [2023-12-27 15:53:35,646][20347] Updated weights for policy 0, policy_version 1478895 (0.0005) [2023-12-27 15:53:36,054][20381] Updated weights for policy 1, policy_version 1477449 (0.0007) [2023-12-27 15:53:36,120][20381] Updated weights for policy 1, policy_version 1477459 (0.0008) [2023-12-27 15:53:36,185][20381] Updated weights for policy 1, policy_version 1477469 (0.0008) [2023-12-27 15:53:36,257][20347] Updated weights for policy 0, policy_version 1478905 (0.0005) [2023-12-27 15:53:36,319][20347] Updated weights for policy 0, policy_version 1478915 (0.0006) [2023-12-27 15:53:36,380][20347] Updated weights for policy 0, policy_version 1478925 (0.0007) [2023-12-27 15:53:36,441][20347] Updated weights for policy 0, policy_version 1478935 (0.0007) [2023-12-27 15:53:36,869][20381] Updated weights for policy 1, policy_version 1477479 (0.0008) [2023-12-27 15:53:36,929][20381] Updated weights for policy 1, policy_version 1477489 (0.0007) [2023-12-27 15:53:36,982][20381] Updated weights for policy 1, policy_version 1477499 (0.0005) [2023-12-27 15:53:37,121][20347] Updated weights for policy 0, policy_version 1478945 (0.0007) [2023-12-27 15:53:37,176][20347] Updated weights for policy 0, policy_version 1478955 (0.0007) [2023-12-27 15:53:37,232][20347] Updated weights for policy 0, policy_version 1478965 (0.0007) [2023-12-27 15:53:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 756965376. Throughput: 0: 10122.7, 1: 10339.3. Samples: 756931136. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:37,363][19170] Avg episode reward: [(0, '912.120'), (1, '970.850')] [2023-12-27 15:53:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001477504_378273792.pth... [2023-12-27 15:53:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001478968_378691584.pth... [2023-12-27 15:53:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001476320_377970688.pth [2023-12-27 15:53:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001477752_378380288.pth [2023-12-27 15:53:37,621][20381] Updated weights for policy 1, policy_version 1477509 (0.0006) [2023-12-27 15:53:37,680][20381] Updated weights for policy 1, policy_version 1477519 (0.0009) [2023-12-27 15:53:37,744][20381] Updated weights for policy 1, policy_version 1477529 (0.0008) [2023-12-27 15:53:37,831][20347] Updated weights for policy 0, policy_version 1478975 (0.0006) [2023-12-27 15:53:37,882][20347] Updated weights for policy 0, policy_version 1478985 (0.0005) [2023-12-27 15:53:37,934][20347] Updated weights for policy 0, policy_version 1478995 (0.0007) [2023-12-27 15:53:38,456][20381] Updated weights for policy 1, policy_version 1477539 (0.0008) [2023-12-27 15:53:38,507][20381] Updated weights for policy 1, policy_version 1477549 (0.0006) [2023-12-27 15:53:38,568][20381] Updated weights for policy 1, policy_version 1477559 (0.0005) [2023-12-27 15:53:38,622][20347] Updated weights for policy 0, policy_version 1479005 (0.0008) [2023-12-27 15:53:38,674][20347] Updated weights for policy 0, policy_version 1479015 (0.0007) [2023-12-27 15:53:38,732][20347] Updated weights for policy 0, policy_version 1479025 (0.0008) [2023-12-27 15:53:39,260][20381] Updated weights for policy 1, policy_version 1477569 (0.0006) [2023-12-27 15:53:39,318][20381] Updated weights for policy 1, policy_version 1477579 (0.0008) [2023-12-27 15:53:39,385][20381] Updated weights for policy 1, policy_version 1477589 (0.0009) [2023-12-27 15:53:39,443][20347] Updated weights for policy 0, policy_version 1479035 (0.0007) [2023-12-27 15:53:39,452][20381] Updated weights for policy 1, policy_version 1477599 (0.0009) [2023-12-27 15:53:39,501][20347] Updated weights for policy 0, policy_version 1479045 (0.0006) [2023-12-27 15:53:39,559][20347] Updated weights for policy 0, policy_version 1479055 (0.0006) [2023-12-27 15:53:40,212][20347] Updated weights for policy 0, policy_version 1479065 (0.0006) [2023-12-27 15:53:40,247][20381] Updated weights for policy 1, policy_version 1477609 (0.0008) [2023-12-27 15:53:40,268][20347] Updated weights for policy 0, policy_version 1479075 (0.0006) [2023-12-27 15:53:40,305][20381] Updated weights for policy 1, policy_version 1477619 (0.0007) [2023-12-27 15:53:40,325][20347] Updated weights for policy 0, policy_version 1479085 (0.0007) [2023-12-27 15:53:40,361][20381] Updated weights for policy 1, policy_version 1477629 (0.0007) [2023-12-27 15:53:40,382][20347] Updated weights for policy 0, policy_version 1479095 (0.0008) [2023-12-27 15:53:41,049][20381] Updated weights for policy 1, policy_version 1477639 (0.0007) [2023-12-27 15:53:41,067][20347] Updated weights for policy 0, policy_version 1479105 (0.0008) [2023-12-27 15:53:41,105][20381] Updated weights for policy 1, policy_version 1477649 (0.0007) [2023-12-27 15:53:41,134][20347] Updated weights for policy 0, policy_version 1479115 (0.0007) [2023-12-27 15:53:41,169][20381] Updated weights for policy 1, policy_version 1477659 (0.0006) [2023-12-27 15:53:41,200][20347] Updated weights for policy 0, policy_version 1479125 (0.0007) [2023-12-27 15:53:41,881][20381] Updated weights for policy 1, policy_version 1477669 (0.0007) [2023-12-27 15:53:41,937][20381] Updated weights for policy 1, policy_version 1477679 (0.0006) [2023-12-27 15:53:41,998][20381] Updated weights for policy 1, policy_version 1477689 (0.0006) [2023-12-27 15:53:42,003][20347] Updated weights for policy 0, policy_version 1479135 (0.0008) [2023-12-27 15:53:42,059][20347] Updated weights for policy 0, policy_version 1479145 (0.0006) [2023-12-27 15:53:42,117][20347] Updated weights for policy 0, policy_version 1479155 (0.0007) [2023-12-27 15:53:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 757063680. Throughput: 0: 10210.1, 1: 10234.8. Samples: 757052876. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:42,363][19170] Avg episode reward: [(0, '882.930'), (1, '961.080')] [2023-12-27 15:53:42,673][20381] Updated weights for policy 1, policy_version 1477699 (0.0007) [2023-12-27 15:53:42,725][20381] Updated weights for policy 1, policy_version 1477709 (0.0008) [2023-12-27 15:53:42,785][20381] Updated weights for policy 1, policy_version 1477719 (0.0008) [2023-12-27 15:53:42,815][20347] Updated weights for policy 0, policy_version 1479165 (0.0007) [2023-12-27 15:53:42,876][20347] Updated weights for policy 0, policy_version 1479175 (0.0007) [2023-12-27 15:53:42,926][20347] Updated weights for policy 0, policy_version 1479185 (0.0008) [2023-12-27 15:53:43,472][20381] Updated weights for policy 1, policy_version 1477729 (0.0007) [2023-12-27 15:53:43,533][20381] Updated weights for policy 1, policy_version 1477739 (0.0008) [2023-12-27 15:53:43,590][20381] Updated weights for policy 1, policy_version 1477749 (0.0007) [2023-12-27 15:53:43,634][20347] Updated weights for policy 0, policy_version 1479195 (0.0006) [2023-12-27 15:53:43,648][20381] Updated weights for policy 1, policy_version 1477759 (0.0007) [2023-12-27 15:53:43,688][20347] Updated weights for policy 0, policy_version 1479205 (0.0005) [2023-12-27 15:53:43,753][20347] Updated weights for policy 0, policy_version 1479215 (0.0005) [2023-12-27 15:53:44,326][20347] Updated weights for policy 0, policy_version 1479225 (0.0006) [2023-12-27 15:53:44,338][20381] Updated weights for policy 1, policy_version 1477769 (0.0010) [2023-12-27 15:53:44,375][20347] Updated weights for policy 0, policy_version 1479235 (0.0009) [2023-12-27 15:53:44,393][20381] Updated weights for policy 1, policy_version 1477779 (0.0010) [2023-12-27 15:53:44,431][20347] Updated weights for policy 0, policy_version 1479245 (0.0007) [2023-12-27 15:53:44,443][20381] Updated weights for policy 1, policy_version 1477789 (0.0010) [2023-12-27 15:53:44,485][20347] Updated weights for policy 0, policy_version 1479255 (0.0007) [2023-12-27 15:53:45,155][20381] Updated weights for policy 1, policy_version 1477799 (0.0011) [2023-12-27 15:53:45,156][20347] Updated weights for policy 0, policy_version 1479265 (0.0010) [2023-12-27 15:53:45,207][20381] Updated weights for policy 1, policy_version 1477809 (0.0010) [2023-12-27 15:53:45,211][20347] Updated weights for policy 0, policy_version 1479275 (0.0010) [2023-12-27 15:53:45,260][20381] Updated weights for policy 1, policy_version 1477819 (0.0010) [2023-12-27 15:53:45,267][20347] Updated weights for policy 0, policy_version 1479285 (0.0010) [2023-12-27 15:53:45,960][20347] Updated weights for policy 0, policy_version 1479295 (0.0010) [2023-12-27 15:53:45,975][20381] Updated weights for policy 1, policy_version 1477829 (0.0010) [2023-12-27 15:53:46,017][20347] Updated weights for policy 0, policy_version 1479305 (0.0008) [2023-12-27 15:53:46,024][20381] Updated weights for policy 1, policy_version 1477839 (0.0010) [2023-12-27 15:53:46,065][20347] Updated weights for policy 0, policy_version 1479315 (0.0010) [2023-12-27 15:53:46,073][20381] Updated weights for policy 1, policy_version 1477849 (0.0010) [2023-12-27 15:53:46,703][20347] Updated weights for policy 0, policy_version 1479325 (0.0010) [2023-12-27 15:53:46,751][20347] Updated weights for policy 0, policy_version 1479335 (0.0010) [2023-12-27 15:53:46,774][20381] Updated weights for policy 1, policy_version 1477859 (0.0011) [2023-12-27 15:53:46,803][20347] Updated weights for policy 0, policy_version 1479345 (0.0010) [2023-12-27 15:53:46,830][20381] Updated weights for policy 1, policy_version 1477869 (0.0007) [2023-12-27 15:53:46,890][20381] Updated weights for policy 1, policy_version 1477879 (0.0006) [2023-12-27 15:53:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 757161984. Throughput: 0: 10257.2, 1: 10228.1. Samples: 757174664. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:47,363][19170] Avg episode reward: [(0, '941.310'), (1, '960.980')] [2023-12-27 15:53:47,483][20347] Updated weights for policy 0, policy_version 1479355 (0.0011) [2023-12-27 15:53:47,533][20381] Updated weights for policy 1, policy_version 1477889 (0.0008) [2023-12-27 15:53:47,538][20347] Updated weights for policy 0, policy_version 1479365 (0.0010) [2023-12-27 15:53:47,584][20381] Updated weights for policy 1, policy_version 1477899 (0.0005) [2023-12-27 15:53:47,587][20347] Updated weights for policy 0, policy_version 1479375 (0.0010) [2023-12-27 15:53:47,635][20381] Updated weights for policy 1, policy_version 1477909 (0.0005) [2023-12-27 15:53:47,686][20381] Updated weights for policy 1, policy_version 1477919 (0.0005) [2023-12-27 15:53:48,260][20381] Updated weights for policy 1, policy_version 1477929 (0.0010) [2023-12-27 15:53:48,273][20347] Updated weights for policy 0, policy_version 1479385 (0.0010) [2023-12-27 15:53:48,315][20381] Updated weights for policy 1, policy_version 1477939 (0.0010) [2023-12-27 15:53:48,338][20347] Updated weights for policy 0, policy_version 1479395 (0.0010) [2023-12-27 15:53:48,380][20381] Updated weights for policy 1, policy_version 1477949 (0.0011) [2023-12-27 15:53:48,389][20347] Updated weights for policy 0, policy_version 1479405 (0.0006) [2023-12-27 15:53:48,450][20347] Updated weights for policy 0, policy_version 1479415 (0.0008) [2023-12-27 15:53:49,015][20381] Updated weights for policy 1, policy_version 1477959 (0.0010) [2023-12-27 15:53:49,050][20347] Updated weights for policy 0, policy_version 1479425 (0.0006) [2023-12-27 15:53:49,070][20381] Updated weights for policy 1, policy_version 1477969 (0.0010) [2023-12-27 15:53:49,103][20347] Updated weights for policy 0, policy_version 1479435 (0.0005) [2023-12-27 15:53:49,121][20381] Updated weights for policy 1, policy_version 1477979 (0.0010) [2023-12-27 15:53:49,153][20347] Updated weights for policy 0, policy_version 1479445 (0.0007) [2023-12-27 15:53:49,751][20381] Updated weights for policy 1, policy_version 1477989 (0.0008) [2023-12-27 15:53:49,803][20381] Updated weights for policy 1, policy_version 1477999 (0.0005) [2023-12-27 15:53:49,865][20381] Updated weights for policy 1, policy_version 1478009 (0.0010) [2023-12-27 15:53:49,929][20347] Updated weights for policy 0, policy_version 1479455 (0.0007) [2023-12-27 15:53:49,997][20347] Updated weights for policy 0, policy_version 1479465 (0.0008) [2023-12-27 15:53:50,057][20347] Updated weights for policy 0, policy_version 1479475 (0.0007) [2023-12-27 15:53:50,599][20381] Updated weights for policy 1, policy_version 1478019 (0.0007) [2023-12-27 15:53:50,651][20381] Updated weights for policy 1, policy_version 1478029 (0.0009) [2023-12-27 15:53:50,708][20381] Updated weights for policy 1, policy_version 1478039 (0.0007) [2023-12-27 15:53:50,717][20347] Updated weights for policy 0, policy_version 1479485 (0.0006) [2023-12-27 15:53:50,770][20347] Updated weights for policy 0, policy_version 1479495 (0.0006) [2023-12-27 15:53:50,829][20347] Updated weights for policy 0, policy_version 1479505 (0.0007) [2023-12-27 15:53:51,397][20381] Updated weights for policy 1, policy_version 1478049 (0.0006) [2023-12-27 15:53:51,460][20381] Updated weights for policy 1, policy_version 1478059 (0.0009) [2023-12-27 15:53:51,522][20381] Updated weights for policy 1, policy_version 1478069 (0.0009) [2023-12-27 15:53:51,573][20347] Updated weights for policy 0, policy_version 1479515 (0.0009) [2023-12-27 15:53:51,577][20381] Updated weights for policy 1, policy_version 1478079 (0.0009) [2023-12-27 15:53:51,633][20347] Updated weights for policy 0, policy_version 1479525 (0.0008) [2023-12-27 15:53:51,690][20347] Updated weights for policy 0, policy_version 1479535 (0.0007) [2023-12-27 15:53:52,335][20381] Updated weights for policy 1, policy_version 1478089 (0.0011) [2023-12-27 15:53:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 757260288. Throughput: 0: 10236.2, 1: 10225.3. Samples: 757237380. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:52,363][19170] Avg episode reward: [(0, '941.360'), (1, '951.220')] [2023-12-27 15:53:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001479544_378839040.pth... [2023-12-27 15:53:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001478360_378535936.pth [2023-12-27 15:53:52,400][20347] Updated weights for policy 0, policy_version 1479545 (0.0007) [2023-12-27 15:53:52,404][20381] Updated weights for policy 1, policy_version 1478099 (0.0011) [2023-12-27 15:53:52,455][20347] Updated weights for policy 0, policy_version 1479555 (0.0006) [2023-12-27 15:53:52,460][20381] Updated weights for policy 1, policy_version 1478109 (0.0011) [2023-12-27 15:53:52,473][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001478112_378429440.pth... [2023-12-27 15:53:52,477][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001476896_378118144.pth [2023-12-27 15:53:52,512][20347] Updated weights for policy 0, policy_version 1479565 (0.0007) [2023-12-27 15:53:52,569][20347] Updated weights for policy 0, policy_version 1479575 (0.0007) [2023-12-27 15:53:53,145][20381] Updated weights for policy 1, policy_version 1478119 (0.0008) [2023-12-27 15:53:53,186][20347] Updated weights for policy 0, policy_version 1479585 (0.0008) [2023-12-27 15:53:53,195][20381] Updated weights for policy 1, policy_version 1478129 (0.0006) [2023-12-27 15:53:53,244][20347] Updated weights for policy 0, policy_version 1479595 (0.0010) [2023-12-27 15:53:53,246][20381] Updated weights for policy 1, policy_version 1478139 (0.0007) [2023-12-27 15:53:53,305][20347] Updated weights for policy 0, policy_version 1479605 (0.0010) [2023-12-27 15:53:53,814][20381] Updated weights for policy 1, policy_version 1478149 (0.0005) [2023-12-27 15:53:53,871][20381] Updated weights for policy 1, policy_version 1478159 (0.0005) [2023-12-27 15:53:53,928][20381] Updated weights for policy 1, policy_version 1478169 (0.0006) [2023-12-27 15:53:53,931][20347] Updated weights for policy 0, policy_version 1479615 (0.0007) [2023-12-27 15:53:53,985][20347] Updated weights for policy 0, policy_version 1479625 (0.0007) [2023-12-27 15:53:54,045][20347] Updated weights for policy 0, policy_version 1479635 (0.0007) [2023-12-27 15:53:54,449][20381] Updated weights for policy 1, policy_version 1478179 (0.0009) [2023-12-27 15:53:54,498][20381] Updated weights for policy 1, policy_version 1478189 (0.0007) [2023-12-27 15:53:54,547][20381] Updated weights for policy 1, policy_version 1478199 (0.0011) [2023-12-27 15:53:54,658][20347] Updated weights for policy 0, policy_version 1479645 (0.0005) [2023-12-27 15:53:54,720][20347] Updated weights for policy 0, policy_version 1479655 (0.0005) [2023-12-27 15:53:54,774][20347] Updated weights for policy 0, policy_version 1479665 (0.0005) [2023-12-27 15:53:55,216][20381] Updated weights for policy 1, policy_version 1478209 (0.0010) [2023-12-27 15:53:55,268][20381] Updated weights for policy 1, policy_version 1478219 (0.0010) [2023-12-27 15:53:55,319][20381] Updated weights for policy 1, policy_version 1478229 (0.0010) [2023-12-27 15:53:55,367][20347] Updated weights for policy 0, policy_version 1479675 (0.0006) [2023-12-27 15:53:55,374][20381] Updated weights for policy 1, policy_version 1478239 (0.0010) [2023-12-27 15:53:55,428][20347] Updated weights for policy 0, policy_version 1479685 (0.0008) [2023-12-27 15:53:55,481][20347] Updated weights for policy 0, policy_version 1479695 (0.0007) [2023-12-27 15:53:56,044][20381] Updated weights for policy 1, policy_version 1478249 (0.0009) [2023-12-27 15:53:56,057][20347] Updated weights for policy 0, policy_version 1479705 (0.0007) [2023-12-27 15:53:56,096][20381] Updated weights for policy 1, policy_version 1478259 (0.0010) [2023-12-27 15:53:56,111][20347] Updated weights for policy 0, policy_version 1479715 (0.0010) [2023-12-27 15:53:56,154][20381] Updated weights for policy 1, policy_version 1478269 (0.0011) [2023-12-27 15:53:56,164][20347] Updated weights for policy 0, policy_version 1479725 (0.0010) [2023-12-27 15:53:56,218][20347] Updated weights for policy 0, policy_version 1479735 (0.0010) [2023-12-27 15:53:56,800][20381] Updated weights for policy 1, policy_version 1478279 (0.0007) [2023-12-27 15:53:56,849][20347] Updated weights for policy 0, policy_version 1479745 (0.0006) [2023-12-27 15:53:56,856][20381] Updated weights for policy 1, policy_version 1478289 (0.0005) [2023-12-27 15:53:56,903][20347] Updated weights for policy 0, policy_version 1479755 (0.0008) [2023-12-27 15:53:56,910][20381] Updated weights for policy 1, policy_version 1478299 (0.0005) [2023-12-27 15:53:56,953][20347] Updated weights for policy 0, policy_version 1479765 (0.0007) [2023-12-27 15:53:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 757374976. Throughput: 0: 10311.1, 1: 10242.3. Samples: 757364076. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:53:57,363][19170] Avg episode reward: [(0, '941.340'), (1, '980.470')] [2023-12-27 15:53:57,453][20381] Updated weights for policy 1, policy_version 1478309 (0.0007) [2023-12-27 15:53:57,508][20381] Updated weights for policy 1, policy_version 1478319 (0.0008) [2023-12-27 15:53:57,567][20381] Updated weights for policy 1, policy_version 1478329 (0.0008) [2023-12-27 15:53:57,664][20347] Updated weights for policy 0, policy_version 1479775 (0.0008) [2023-12-27 15:53:57,711][20347] Updated weights for policy 0, policy_version 1479785 (0.0009) [2023-12-27 15:53:57,764][20347] Updated weights for policy 0, policy_version 1479795 (0.0008) [2023-12-27 15:53:58,176][20381] Updated weights for policy 1, policy_version 1478339 (0.0007) [2023-12-27 15:53:58,233][20381] Updated weights for policy 1, policy_version 1478349 (0.0007) [2023-12-27 15:53:58,294][20381] Updated weights for policy 1, policy_version 1478359 (0.0006) [2023-12-27 15:53:58,493][20347] Updated weights for policy 0, policy_version 1479805 (0.0008) [2023-12-27 15:53:58,554][20347] Updated weights for policy 0, policy_version 1479815 (0.0007) [2023-12-27 15:53:58,621][20347] Updated weights for policy 0, policy_version 1479825 (0.0008) [2023-12-27 15:53:59,024][20381] Updated weights for policy 1, policy_version 1478369 (0.0007) [2023-12-27 15:53:59,081][20381] Updated weights for policy 1, policy_version 1478379 (0.0008) [2023-12-27 15:53:59,136][20381] Updated weights for policy 1, policy_version 1478389 (0.0006) [2023-12-27 15:53:59,199][20381] Updated weights for policy 1, policy_version 1478399 (0.0005) [2023-12-27 15:53:59,340][20347] Updated weights for policy 0, policy_version 1479835 (0.0008) [2023-12-27 15:53:59,401][20347] Updated weights for policy 0, policy_version 1479845 (0.0009) [2023-12-27 15:53:59,459][20347] Updated weights for policy 0, policy_version 1479855 (0.0008) [2023-12-27 15:53:59,955][20381] Updated weights for policy 1, policy_version 1478409 (0.0007) [2023-12-27 15:54:00,014][20381] Updated weights for policy 1, policy_version 1478419 (0.0009) [2023-12-27 15:54:00,073][20381] Updated weights for policy 1, policy_version 1478429 (0.0007) [2023-12-27 15:54:00,175][20347] Updated weights for policy 0, policy_version 1479865 (0.0008) [2023-12-27 15:54:00,235][20347] Updated weights for policy 0, policy_version 1479875 (0.0010) [2023-12-27 15:54:00,297][20347] Updated weights for policy 0, policy_version 1479885 (0.0010) [2023-12-27 15:54:00,355][20347] Updated weights for policy 0, policy_version 1479895 (0.0010) [2023-12-27 15:54:00,729][20381] Updated weights for policy 1, policy_version 1478439 (0.0007) [2023-12-27 15:54:00,783][20381] Updated weights for policy 1, policy_version 1478449 (0.0010) [2023-12-27 15:54:00,832][20381] Updated weights for policy 1, policy_version 1478459 (0.0010) [2023-12-27 15:54:00,967][20347] Updated weights for policy 0, policy_version 1479905 (0.0010) [2023-12-27 15:54:01,025][20347] Updated weights for policy 0, policy_version 1479915 (0.0010) [2023-12-27 15:54:01,084][20347] Updated weights for policy 0, policy_version 1479925 (0.0007) [2023-12-27 15:54:01,555][20381] Updated weights for policy 1, policy_version 1478469 (0.0010) [2023-12-27 15:54:01,614][20381] Updated weights for policy 1, policy_version 1478479 (0.0010) [2023-12-27 15:54:01,691][20381] Updated weights for policy 1, policy_version 1478489 (0.0009) [2023-12-27 15:54:01,913][20347] Updated weights for policy 0, policy_version 1479935 (0.0007) [2023-12-27 15:54:01,975][20347] Updated weights for policy 0, policy_version 1479945 (0.0008) [2023-12-27 15:54:02,042][20347] Updated weights for policy 0, policy_version 1479955 (0.0006) [2023-12-27 15:54:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 757473280. Throughput: 0: 10293.1, 1: 10236.4. Samples: 757484612. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:54:02,363][19170] Avg episode reward: [(0, '960.920'), (1, '980.500')] [2023-12-27 15:54:02,451][20381] Updated weights for policy 1, policy_version 1478499 (0.0008) [2023-12-27 15:54:02,507][20381] Updated weights for policy 1, policy_version 1478509 (0.0008) [2023-12-27 15:54:02,567][20381] Updated weights for policy 1, policy_version 1478519 (0.0008) [2023-12-27 15:54:02,756][20347] Updated weights for policy 0, policy_version 1479965 (0.0007) [2023-12-27 15:54:02,812][20347] Updated weights for policy 0, policy_version 1479975 (0.0008) [2023-12-27 15:54:02,863][20347] Updated weights for policy 0, policy_version 1479985 (0.0008) [2023-12-27 15:54:03,189][20381] Updated weights for policy 1, policy_version 1478529 (0.0006) [2023-12-27 15:54:03,246][20381] Updated weights for policy 1, policy_version 1478539 (0.0007) [2023-12-27 15:54:03,291][20381] Updated weights for policy 1, policy_version 1478549 (0.0008) [2023-12-27 15:54:03,344][20381] Updated weights for policy 1, policy_version 1478559 (0.0008) [2023-12-27 15:54:03,495][20347] Updated weights for policy 0, policy_version 1479995 (0.0007) [2023-12-27 15:54:03,546][20347] Updated weights for policy 0, policy_version 1480005 (0.0007) [2023-12-27 15:54:03,604][20347] Updated weights for policy 0, policy_version 1480015 (0.0009) [2023-12-27 15:54:04,079][20381] Updated weights for policy 1, policy_version 1478570 (0.0009) [2023-12-27 15:54:04,139][20381] Updated weights for policy 1, policy_version 1478580 (0.0011) [2023-12-27 15:54:04,195][20381] Updated weights for policy 1, policy_version 1478590 (0.0011) [2023-12-27 15:54:04,238][20347] Updated weights for policy 0, policy_version 1480025 (0.0008) [2023-12-27 15:54:04,297][20347] Updated weights for policy 0, policy_version 1480035 (0.0011) [2023-12-27 15:54:04,356][20347] Updated weights for policy 0, policy_version 1480045 (0.0010) [2023-12-27 15:54:04,412][20347] Updated weights for policy 0, policy_version 1480055 (0.0010) [2023-12-27 15:54:04,861][20381] Updated weights for policy 1, policy_version 1478600 (0.0008) [2023-12-27 15:54:04,912][20381] Updated weights for policy 1, policy_version 1478610 (0.0005) [2023-12-27 15:54:04,967][20381] Updated weights for policy 1, policy_version 1478620 (0.0006) [2023-12-27 15:54:05,025][20347] Updated weights for policy 0, policy_version 1480065 (0.0010) [2023-12-27 15:54:05,084][20347] Updated weights for policy 0, policy_version 1480075 (0.0010) [2023-12-27 15:54:05,147][20347] Updated weights for policy 0, policy_version 1480085 (0.0010) [2023-12-27 15:54:05,625][20381] Updated weights for policy 1, policy_version 1478630 (0.0008) [2023-12-27 15:54:05,684][20381] Updated weights for policy 1, policy_version 1478640 (0.0009) [2023-12-27 15:54:05,747][20381] Updated weights for policy 1, policy_version 1478650 (0.0011) [2023-12-27 15:54:05,821][20347] Updated weights for policy 0, policy_version 1480095 (0.0010) [2023-12-27 15:54:05,880][20347] Updated weights for policy 0, policy_version 1480105 (0.0005) [2023-12-27 15:54:05,941][20347] Updated weights for policy 0, policy_version 1480115 (0.0006) [2023-12-27 15:54:06,386][20381] Updated weights for policy 1, policy_version 1478660 (0.0009) [2023-12-27 15:54:06,446][20381] Updated weights for policy 1, policy_version 1478670 (0.0006) [2023-12-27 15:54:06,501][20381] Updated weights for policy 1, policy_version 1478680 (0.0006) [2023-12-27 15:54:06,620][20347] Updated weights for policy 0, policy_version 1480125 (0.0006) [2023-12-27 15:54:06,679][20347] Updated weights for policy 0, policy_version 1480135 (0.0007) [2023-12-27 15:54:06,736][20347] Updated weights for policy 0, policy_version 1480145 (0.0008) [2023-12-27 15:54:07,158][20381] Updated weights for policy 1, policy_version 1478690 (0.0006) [2023-12-27 15:54:07,213][20381] Updated weights for policy 1, policy_version 1478700 (0.0006) [2023-12-27 15:54:07,264][20381] Updated weights for policy 1, policy_version 1478710 (0.0007) [2023-12-27 15:54:07,316][20381] Updated weights for policy 1, policy_version 1478720 (0.0008) [2023-12-27 15:54:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20480.0, 300 sec: 20604.9). Total num frames: 757579776. Throughput: 0: 10334.8, 1: 10272.9. Samples: 757547996. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-12-27 15:54:07,364][19170] Avg episode reward: [(0, '951.120'), (1, '951.100')] [2023-12-27 15:54:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001478720_378585088.pth... [2023-12-27 15:54:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001480152_378994688.pth... [2023-12-27 15:54:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001477504_378273792.pth [2023-12-27 15:54:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001478968_378691584.pth [2023-12-27 15:54:07,428][20347] Updated weights for policy 0, policy_version 1480155 (0.0007) [2023-12-27 15:54:07,477][20347] Updated weights for policy 0, policy_version 1480165 (0.0008) [2023-12-27 15:54:07,525][20347] Updated weights for policy 0, policy_version 1480175 (0.0007) [2023-12-27 15:54:07,985][20381] Updated weights for policy 1, policy_version 1478730 (0.0005) [2023-12-27 15:54:08,047][20381] Updated weights for policy 1, policy_version 1478740 (0.0005) [2023-12-27 15:54:08,105][20381] Updated weights for policy 1, policy_version 1478750 (0.0005) [2023-12-27 15:54:08,181][20347] Updated weights for policy 0, policy_version 1480185 (0.0007) [2023-12-27 15:54:08,240][20347] Updated weights for policy 0, policy_version 1480195 (0.0008) [2023-12-27 15:54:08,300][20347] Updated weights for policy 0, policy_version 1480205 (0.0008) [2023-12-27 15:54:08,361][20347] Updated weights for policy 0, policy_version 1480215 (0.0009) [2023-12-27 15:54:08,666][20381] Updated weights for policy 1, policy_version 1478760 (0.0007) [2023-12-27 15:54:08,725][20381] Updated weights for policy 1, policy_version 1478770 (0.0007) [2023-12-27 15:54:08,789][20381] Updated weights for policy 1, policy_version 1478780 (0.0005) [2023-12-27 15:54:09,105][20347] Updated weights for policy 0, policy_version 1480225 (0.0007) [2023-12-27 15:54:09,159][20347] Updated weights for policy 0, policy_version 1480235 (0.0007) [2023-12-27 15:54:09,208][20347] Updated weights for policy 0, policy_version 1480245 (0.0007) [2023-12-27 15:54:09,452][20381] Updated weights for policy 1, policy_version 1478790 (0.0008) [2023-12-27 15:54:09,512][20381] Updated weights for policy 1, policy_version 1478800 (0.0008) [2023-12-27 15:54:09,577][20381] Updated weights for policy 1, policy_version 1478810 (0.0008) [2023-12-27 15:54:09,975][20347] Updated weights for policy 0, policy_version 1480255 (0.0008) [2023-12-27 15:54:10,032][20347] Updated weights for policy 0, policy_version 1480265 (0.0007) [2023-12-27 15:54:10,089][20347] Updated weights for policy 0, policy_version 1480275 (0.0007) [2023-12-27 15:54:10,274][20381] Updated weights for policy 1, policy_version 1478820 (0.0008) [2023-12-27 15:54:10,338][20381] Updated weights for policy 1, policy_version 1478830 (0.0008) [2023-12-27 15:54:10,397][20381] Updated weights for policy 1, policy_version 1478840 (0.0006) [2023-12-27 15:54:10,791][20347] Updated weights for policy 0, policy_version 1480285 (0.0008) [2023-12-27 15:54:10,840][20347] Updated weights for policy 0, policy_version 1480295 (0.0007) [2023-12-27 15:54:10,894][20347] Updated weights for policy 0, policy_version 1480305 (0.0007) [2023-12-27 15:54:10,994][20381] Updated weights for policy 1, policy_version 1478850 (0.0006) [2023-12-27 15:54:11,053][20381] Updated weights for policy 1, policy_version 1478860 (0.0008) [2023-12-27 15:54:11,108][20381] Updated weights for policy 1, policy_version 1478870 (0.0007) [2023-12-27 15:54:11,175][20381] Updated weights for policy 1, policy_version 1478880 (0.0008) [2023-12-27 15:54:11,594][20347] Updated weights for policy 0, policy_version 1480315 (0.0008) [2023-12-27 15:54:11,669][20347] Updated weights for policy 0, policy_version 1480325 (0.0008) [2023-12-27 15:54:11,732][20347] Updated weights for policy 0, policy_version 1480335 (0.0008) [2023-12-27 15:54:11,914][20381] Updated weights for policy 1, policy_version 1478890 (0.0007) [2023-12-27 15:54:11,973][20381] Updated weights for policy 1, policy_version 1478900 (0.0008) [2023-12-27 15:54:12,025][20381] Updated weights for policy 1, policy_version 1478910 (0.0008) [2023-12-27 15:54:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 757678080. Throughput: 0: 10288.8, 1: 10268.8. Samples: 757670220. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:12,363][19170] Avg episode reward: [(0, '961.150'), (1, '912.220')] [2023-12-27 15:54:12,385][20347] Updated weights for policy 0, policy_version 1480345 (0.0008) [2023-12-27 15:54:12,442][20347] Updated weights for policy 0, policy_version 1480355 (0.0007) [2023-12-27 15:54:12,505][20347] Updated weights for policy 0, policy_version 1480365 (0.0007) [2023-12-27 15:54:12,563][20347] Updated weights for policy 0, policy_version 1480375 (0.0008) [2023-12-27 15:54:12,756][20381] Updated weights for policy 1, policy_version 1478920 (0.0008) [2023-12-27 15:54:12,826][20381] Updated weights for policy 1, policy_version 1478930 (0.0007) [2023-12-27 15:54:12,874][20381] Updated weights for policy 1, policy_version 1478940 (0.0009) [2023-12-27 15:54:13,271][20347] Updated weights for policy 0, policy_version 1480385 (0.0008) [2023-12-27 15:54:13,323][20347] Updated weights for policy 0, policy_version 1480395 (0.0007) [2023-12-27 15:54:13,377][20347] Updated weights for policy 0, policy_version 1480405 (0.0007) [2023-12-27 15:54:13,550][20381] Updated weights for policy 1, policy_version 1478951 (0.0010) [2023-12-27 15:54:13,607][20381] Updated weights for policy 1, policy_version 1478961 (0.0010) [2023-12-27 15:54:13,665][20381] Updated weights for policy 1, policy_version 1478971 (0.0011) [2023-12-27 15:54:13,984][20347] Updated weights for policy 0, policy_version 1480415 (0.0005) [2023-12-27 15:54:14,034][20347] Updated weights for policy 0, policy_version 1480425 (0.0005) [2023-12-27 15:54:14,095][20347] Updated weights for policy 0, policy_version 1480435 (0.0005) [2023-12-27 15:54:14,336][20381] Updated weights for policy 1, policy_version 1478981 (0.0008) [2023-12-27 15:54:14,386][20381] Updated weights for policy 1, policy_version 1478991 (0.0006) [2023-12-27 15:54:14,438][20381] Updated weights for policy 1, policy_version 1479001 (0.0005) [2023-12-27 15:54:14,685][20347] Updated weights for policy 0, policy_version 1480445 (0.0005) [2023-12-27 15:54:14,747][20347] Updated weights for policy 0, policy_version 1480455 (0.0006) [2023-12-27 15:54:14,809][20347] Updated weights for policy 0, policy_version 1480465 (0.0007) [2023-12-27 15:54:15,023][20381] Updated weights for policy 1, policy_version 1479011 (0.0006) [2023-12-27 15:54:15,083][20381] Updated weights for policy 1, policy_version 1479021 (0.0007) [2023-12-27 15:54:15,142][20381] Updated weights for policy 1, policy_version 1479031 (0.0007) [2023-12-27 15:54:15,483][20347] Updated weights for policy 0, policy_version 1480475 (0.0008) [2023-12-27 15:54:15,548][20347] Updated weights for policy 0, policy_version 1480485 (0.0005) [2023-12-27 15:54:15,603][20347] Updated weights for policy 0, policy_version 1480495 (0.0006) [2023-12-27 15:54:15,817][20381] Updated weights for policy 1, policy_version 1479041 (0.0008) [2023-12-27 15:54:15,869][20381] Updated weights for policy 1, policy_version 1479051 (0.0005) [2023-12-27 15:54:15,921][20381] Updated weights for policy 1, policy_version 1479061 (0.0005) [2023-12-27 15:54:15,973][20381] Updated weights for policy 1, policy_version 1479071 (0.0005) [2023-12-27 15:54:16,189][20347] Updated weights for policy 0, policy_version 1480505 (0.0007) [2023-12-27 15:54:16,241][20347] Updated weights for policy 0, policy_version 1480515 (0.0008) [2023-12-27 15:54:16,295][20347] Updated weights for policy 0, policy_version 1480525 (0.0009) [2023-12-27 15:54:16,349][20347] Updated weights for policy 0, policy_version 1480535 (0.0006) [2023-12-27 15:54:16,572][20381] Updated weights for policy 1, policy_version 1479081 (0.0005) [2023-12-27 15:54:16,628][20381] Updated weights for policy 1, policy_version 1479091 (0.0005) [2023-12-27 15:54:16,682][20381] Updated weights for policy 1, policy_version 1479101 (0.0006) [2023-12-27 15:54:17,064][20347] Updated weights for policy 0, policy_version 1480545 (0.0008) [2023-12-27 15:54:17,116][20347] Updated weights for policy 0, policy_version 1480555 (0.0007) [2023-12-27 15:54:17,170][20347] Updated weights for policy 0, policy_version 1480565 (0.0007) [2023-12-27 15:54:17,319][20381] Updated weights for policy 1, policy_version 1479111 (0.0008) [2023-12-27 15:54:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 757784576. Throughput: 0: 10288.2, 1: 10304.6. Samples: 757795884. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:17,363][19170] Avg episode reward: [(0, '931.910'), (1, '882.990')] [2023-12-27 15:54:17,368][20381] Updated weights for policy 1, policy_version 1479121 (0.0008) [2023-12-27 15:54:17,417][20381] Updated weights for policy 1, policy_version 1479131 (0.0008) [2023-12-27 15:54:17,798][20347] Updated weights for policy 0, policy_version 1480575 (0.0006) [2023-12-27 15:54:17,847][20347] Updated weights for policy 0, policy_version 1480585 (0.0005) [2023-12-27 15:54:17,905][20347] Updated weights for policy 0, policy_version 1480595 (0.0005) [2023-12-27 15:54:18,083][20381] Updated weights for policy 1, policy_version 1479141 (0.0007) [2023-12-27 15:54:18,129][20381] Updated weights for policy 1, policy_version 1479151 (0.0008) [2023-12-27 15:54:18,177][20381] Updated weights for policy 1, policy_version 1479161 (0.0006) [2023-12-27 15:54:18,461][20347] Updated weights for policy 0, policy_version 1480605 (0.0005) [2023-12-27 15:54:18,527][20347] Updated weights for policy 0, policy_version 1480615 (0.0006) [2023-12-27 15:54:18,585][20347] Updated weights for policy 0, policy_version 1480625 (0.0006) [2023-12-27 15:54:18,882][20381] Updated weights for policy 1, policy_version 1479171 (0.0007) [2023-12-27 15:54:18,944][20381] Updated weights for policy 1, policy_version 1479181 (0.0005) [2023-12-27 15:54:19,002][20381] Updated weights for policy 1, policy_version 1479191 (0.0007) [2023-12-27 15:54:19,246][20347] Updated weights for policy 0, policy_version 1480635 (0.0007) [2023-12-27 15:54:19,300][20347] Updated weights for policy 0, policy_version 1480645 (0.0008) [2023-12-27 15:54:19,366][20347] Updated weights for policy 0, policy_version 1480655 (0.0008) [2023-12-27 15:54:19,682][20381] Updated weights for policy 1, policy_version 1479201 (0.0008) [2023-12-27 15:54:19,736][20381] Updated weights for policy 1, policy_version 1479211 (0.0007) [2023-12-27 15:54:19,802][20381] Updated weights for policy 1, policy_version 1479221 (0.0008) [2023-12-27 15:54:19,866][20381] Updated weights for policy 1, policy_version 1479231 (0.0007) [2023-12-27 15:54:20,142][20347] Updated weights for policy 0, policy_version 1480665 (0.0009) [2023-12-27 15:54:20,202][20347] Updated weights for policy 0, policy_version 1480675 (0.0009) [2023-12-27 15:54:20,255][20347] Updated weights for policy 0, policy_version 1480685 (0.0008) [2023-12-27 15:54:20,309][20347] Updated weights for policy 0, policy_version 1480695 (0.0009) [2023-12-27 15:54:20,546][20381] Updated weights for policy 1, policy_version 1479241 (0.0007) [2023-12-27 15:54:20,600][20381] Updated weights for policy 1, policy_version 1479251 (0.0007) [2023-12-27 15:54:20,651][20381] Updated weights for policy 1, policy_version 1479261 (0.0007) [2023-12-27 15:54:21,033][20347] Updated weights for policy 0, policy_version 1480705 (0.0007) [2023-12-27 15:54:21,094][20347] Updated weights for policy 0, policy_version 1480715 (0.0006) [2023-12-27 15:54:21,155][20347] Updated weights for policy 0, policy_version 1480725 (0.0007) [2023-12-27 15:54:21,413][20381] Updated weights for policy 1, policy_version 1479271 (0.0008) [2023-12-27 15:54:21,476][20381] Updated weights for policy 1, policy_version 1479281 (0.0008) [2023-12-27 15:54:21,539][20381] Updated weights for policy 1, policy_version 1479291 (0.0008) [2023-12-27 15:54:21,864][20347] Updated weights for policy 0, policy_version 1480735 (0.0007) [2023-12-27 15:54:21,924][20347] Updated weights for policy 0, policy_version 1480745 (0.0007) [2023-12-27 15:54:21,983][20347] Updated weights for policy 0, policy_version 1480755 (0.0007) [2023-12-27 15:54:22,203][20381] Updated weights for policy 1, policy_version 1479301 (0.0007) [2023-12-27 15:54:22,259][20381] Updated weights for policy 1, policy_version 1479311 (0.0007) [2023-12-27 15:54:22,318][20381] Updated weights for policy 1, policy_version 1479321 (0.0008) [2023-12-27 15:54:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 757882880. Throughput: 0: 10295.4, 1: 10309.0. Samples: 757858340. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:22,363][19170] Avg episode reward: [(0, '912.180'), (1, '921.900')] [2023-12-27 15:54:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001479328_378740736.pth... [2023-12-27 15:54:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001480760_379150336.pth... [2023-12-27 15:54:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001479544_378839040.pth [2023-12-27 15:54:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001478112_378429440.pth [2023-12-27 15:54:22,679][20347] Updated weights for policy 0, policy_version 1480765 (0.0007) [2023-12-27 15:54:22,740][20347] Updated weights for policy 0, policy_version 1480775 (0.0008) [2023-12-27 15:54:22,805][20347] Updated weights for policy 0, policy_version 1480785 (0.0007) [2023-12-27 15:54:23,087][20381] Updated weights for policy 1, policy_version 1479331 (0.0009) [2023-12-27 15:54:23,143][20381] Updated weights for policy 1, policy_version 1479341 (0.0007) [2023-12-27 15:54:23,195][20381] Updated weights for policy 1, policy_version 1479351 (0.0009) [2023-12-27 15:54:23,388][20347] Updated weights for policy 0, policy_version 1480795 (0.0006) [2023-12-27 15:54:23,451][20347] Updated weights for policy 0, policy_version 1480805 (0.0009) [2023-12-27 15:54:23,509][20347] Updated weights for policy 0, policy_version 1480815 (0.0010) [2023-12-27 15:54:23,775][20381] Updated weights for policy 1, policy_version 1479361 (0.0006) [2023-12-27 15:54:23,829][20381] Updated weights for policy 1, policy_version 1479371 (0.0005) [2023-12-27 15:54:23,887][20381] Updated weights for policy 1, policy_version 1479381 (0.0005) [2023-12-27 15:54:23,947][20381] Updated weights for policy 1, policy_version 1479391 (0.0007) [2023-12-27 15:54:24,086][20347] Updated weights for policy 0, policy_version 1480825 (0.0007) [2023-12-27 15:54:24,153][20347] Updated weights for policy 0, policy_version 1480835 (0.0010) [2023-12-27 15:54:24,215][20347] Updated weights for policy 0, policy_version 1480845 (0.0007) [2023-12-27 15:54:24,277][20347] Updated weights for policy 0, policy_version 1480855 (0.0009) [2023-12-27 15:54:24,599][20381] Updated weights for policy 1, policy_version 1479401 (0.0008) [2023-12-27 15:54:24,643][20381] Updated weights for policy 1, policy_version 1479411 (0.0007) [2023-12-27 15:54:24,691][20381] Updated weights for policy 1, policy_version 1479421 (0.0007) [2023-12-27 15:54:24,912][20347] Updated weights for policy 0, policy_version 1480865 (0.0010) [2023-12-27 15:54:24,957][20347] Updated weights for policy 0, policy_version 1480875 (0.0010) [2023-12-27 15:54:25,005][20347] Updated weights for policy 0, policy_version 1480885 (0.0010) [2023-12-27 15:54:25,425][20381] Updated weights for policy 1, policy_version 1479431 (0.0008) [2023-12-27 15:54:25,480][20381] Updated weights for policy 1, policy_version 1479441 (0.0007) [2023-12-27 15:54:25,537][20381] Updated weights for policy 1, policy_version 1479451 (0.0007) [2023-12-27 15:54:25,682][20347] Updated weights for policy 0, policy_version 1480895 (0.0010) [2023-12-27 15:54:25,739][20347] Updated weights for policy 0, policy_version 1480905 (0.0010) [2023-12-27 15:54:25,798][20347] Updated weights for policy 0, policy_version 1480915 (0.0010) [2023-12-27 15:54:26,242][20381] Updated weights for policy 1, policy_version 1479461 (0.0008) [2023-12-27 15:54:26,301][20381] Updated weights for policy 1, policy_version 1479471 (0.0008) [2023-12-27 15:54:26,359][20381] Updated weights for policy 1, policy_version 1479481 (0.0007) [2023-12-27 15:54:26,444][20347] Updated weights for policy 0, policy_version 1480925 (0.0010) [2023-12-27 15:54:26,501][20347] Updated weights for policy 0, policy_version 1480935 (0.0010) [2023-12-27 15:54:26,558][20347] Updated weights for policy 0, policy_version 1480945 (0.0010) [2023-12-27 15:54:26,986][20381] Updated weights for policy 1, policy_version 1479491 (0.0007) [2023-12-27 15:54:27,043][20381] Updated weights for policy 1, policy_version 1479501 (0.0011) [2023-12-27 15:54:27,092][20381] Updated weights for policy 1, policy_version 1479511 (0.0010) [2023-12-27 15:54:27,223][20347] Updated weights for policy 0, policy_version 1480955 (0.0010) [2023-12-27 15:54:27,283][20347] Updated weights for policy 0, policy_version 1480965 (0.0005) [2023-12-27 15:54:27,343][20347] Updated weights for policy 0, policy_version 1480975 (0.0006) [2023-12-27 15:54:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 757989376. Throughput: 0: 10269.2, 1: 10349.2. Samples: 757980708. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:27,363][19170] Avg episode reward: [(0, '931.760'), (1, '882.840')] [2023-12-27 15:54:27,688][20381] Updated weights for policy 1, policy_version 1479521 (0.0007) [2023-12-27 15:54:27,747][20381] Updated weights for policy 1, policy_version 1479531 (0.0006) [2023-12-27 15:54:27,793][20381] Updated weights for policy 1, policy_version 1479541 (0.0007) [2023-12-27 15:54:27,851][20381] Updated weights for policy 1, policy_version 1479551 (0.0007) [2023-12-27 15:54:27,957][20347] Updated weights for policy 0, policy_version 1480985 (0.0006) [2023-12-27 15:54:28,014][20347] Updated weights for policy 0, policy_version 1480995 (0.0010) [2023-12-27 15:54:28,080][20347] Updated weights for policy 0, policy_version 1481005 (0.0009) [2023-12-27 15:54:28,164][20347] Updated weights for policy 0, policy_version 1481015 (0.0010) [2023-12-27 15:54:28,417][20381] Updated weights for policy 1, policy_version 1479561 (0.0007) [2023-12-27 15:54:28,469][20381] Updated weights for policy 1, policy_version 1479571 (0.0009) [2023-12-27 15:54:28,518][20381] Updated weights for policy 1, policy_version 1479581 (0.0008) [2023-12-27 15:54:28,785][20347] Updated weights for policy 0, policy_version 1481025 (0.0008) [2023-12-27 15:54:28,841][20347] Updated weights for policy 0, policy_version 1481035 (0.0009) [2023-12-27 15:54:28,901][20347] Updated weights for policy 0, policy_version 1481045 (0.0007) [2023-12-27 15:54:29,205][20381] Updated weights for policy 1, policy_version 1479591 (0.0008) [2023-12-27 15:54:29,266][20381] Updated weights for policy 1, policy_version 1479601 (0.0006) [2023-12-27 15:54:29,328][20381] Updated weights for policy 1, policy_version 1479611 (0.0007) [2023-12-27 15:54:29,625][20347] Updated weights for policy 0, policy_version 1481055 (0.0007) [2023-12-27 15:54:29,672][20347] Updated weights for policy 0, policy_version 1481065 (0.0007) [2023-12-27 15:54:29,722][20347] Updated weights for policy 0, policy_version 1481075 (0.0005) [2023-12-27 15:54:30,027][20381] Updated weights for policy 1, policy_version 1479621 (0.0008) [2023-12-27 15:54:30,084][20381] Updated weights for policy 1, policy_version 1479631 (0.0007) [2023-12-27 15:54:30,144][20381] Updated weights for policy 1, policy_version 1479641 (0.0006) [2023-12-27 15:54:30,412][20347] Updated weights for policy 0, policy_version 1481085 (0.0006) [2023-12-27 15:54:30,434][20291] KL-divergence is very high: 423.6856 [2023-12-27 15:54:30,471][20347] Updated weights for policy 0, policy_version 1481095 (0.0008) [2023-12-27 15:54:30,481][20291] KL-divergence is very high: 527.4596 [2023-12-27 15:54:30,518][20291] KL-divergence is very high: 303.5867 [2023-12-27 15:54:30,520][20347] Updated weights for policy 0, policy_version 1481105 (0.0011) [2023-12-27 15:54:30,793][20381] Updated weights for policy 1, policy_version 1479651 (0.0006) [2023-12-27 15:54:30,855][20381] Updated weights for policy 1, policy_version 1479661 (0.0008) [2023-12-27 15:54:30,921][20381] Updated weights for policy 1, policy_version 1479671 (0.0006) [2023-12-27 15:54:31,089][20347] Updated weights for policy 0, policy_version 1481115 (0.0008) [2023-12-27 15:54:31,149][20347] Updated weights for policy 0, policy_version 1481125 (0.0008) [2023-12-27 15:54:31,204][20347] Updated weights for policy 0, policy_version 1481135 (0.0007) [2023-12-27 15:54:31,614][20381] Updated weights for policy 1, policy_version 1479681 (0.0006) [2023-12-27 15:54:31,676][20381] Updated weights for policy 1, policy_version 1479691 (0.0009) [2023-12-27 15:54:31,747][20381] Updated weights for policy 1, policy_version 1479701 (0.0007) [2023-12-27 15:54:31,802][20381] Updated weights for policy 1, policy_version 1479711 (0.0008) [2023-12-27 15:54:31,909][20347] Updated weights for policy 0, policy_version 1481145 (0.0008) [2023-12-27 15:54:31,968][20347] Updated weights for policy 0, policy_version 1481155 (0.0008) [2023-12-27 15:54:32,031][20347] Updated weights for policy 0, policy_version 1481165 (0.0008) [2023-12-27 15:54:32,083][20347] Updated weights for policy 0, policy_version 1481175 (0.0008) [2023-12-27 15:54:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 758095872. Throughput: 0: 10303.2, 1: 10406.1. Samples: 758106584. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:32,363][19170] Avg episode reward: [(0, '951.210'), (1, '882.880')] [2023-12-27 15:54:32,478][20381] Updated weights for policy 1, policy_version 1479721 (0.0008) [2023-12-27 15:54:32,538][20381] Updated weights for policy 1, policy_version 1479731 (0.0008) [2023-12-27 15:54:32,597][20381] Updated weights for policy 1, policy_version 1479741 (0.0008) [2023-12-27 15:54:32,825][20347] Updated weights for policy 0, policy_version 1481185 (0.0007) [2023-12-27 15:54:32,880][20347] Updated weights for policy 0, policy_version 1481195 (0.0008) [2023-12-27 15:54:32,939][20347] Updated weights for policy 0, policy_version 1481205 (0.0007) [2023-12-27 15:54:33,283][20381] Updated weights for policy 1, policy_version 1479751 (0.0007) [2023-12-27 15:54:33,328][20381] Updated weights for policy 1, policy_version 1479761 (0.0007) [2023-12-27 15:54:33,374][20381] Updated weights for policy 1, policy_version 1479771 (0.0005) [2023-12-27 15:54:33,606][20347] Updated weights for policy 0, policy_version 1481215 (0.0008) [2023-12-27 15:54:33,649][20347] Updated weights for policy 0, policy_version 1481225 (0.0007) [2023-12-27 15:54:33,691][20347] Updated weights for policy 0, policy_version 1481235 (0.0008) [2023-12-27 15:54:34,029][20381] Updated weights for policy 1, policy_version 1479781 (0.0006) [2023-12-27 15:54:34,083][20381] Updated weights for policy 1, policy_version 1479791 (0.0008) [2023-12-27 15:54:34,134][20381] Updated weights for policy 1, policy_version 1479801 (0.0007) [2023-12-27 15:54:34,353][20347] Updated weights for policy 0, policy_version 1481245 (0.0007) [2023-12-27 15:54:34,409][20347] Updated weights for policy 0, policy_version 1481255 (0.0007) [2023-12-27 15:54:34,464][20347] Updated weights for policy 0, policy_version 1481265 (0.0007) [2023-12-27 15:54:34,847][20381] Updated weights for policy 1, policy_version 1479811 (0.0007) [2023-12-27 15:54:34,896][20381] Updated weights for policy 1, policy_version 1479821 (0.0008) [2023-12-27 15:54:34,948][20381] Updated weights for policy 1, policy_version 1479831 (0.0008) [2023-12-27 15:54:35,168][20347] Updated weights for policy 0, policy_version 1481275 (0.0007) [2023-12-27 15:54:35,216][20347] Updated weights for policy 0, policy_version 1481285 (0.0005) [2023-12-27 15:54:35,265][20347] Updated weights for policy 0, policy_version 1481295 (0.0005) [2023-12-27 15:54:35,664][20381] Updated weights for policy 1, policy_version 1479841 (0.0008) [2023-12-27 15:54:35,713][20381] Updated weights for policy 1, policy_version 1479851 (0.0008) [2023-12-27 15:54:35,765][20381] Updated weights for policy 1, policy_version 1479861 (0.0008) [2023-12-27 15:54:35,818][20381] Updated weights for policy 1, policy_version 1479871 (0.0006) [2023-12-27 15:54:35,914][20347] Updated weights for policy 0, policy_version 1481305 (0.0007) [2023-12-27 15:54:35,968][20347] Updated weights for policy 0, policy_version 1481315 (0.0005) [2023-12-27 15:54:36,021][20347] Updated weights for policy 0, policy_version 1481325 (0.0006) [2023-12-27 15:54:36,080][20347] Updated weights for policy 0, policy_version 1481335 (0.0007) [2023-12-27 15:54:36,527][20381] Updated weights for policy 1, policy_version 1479881 (0.0008) [2023-12-27 15:54:36,594][20381] Updated weights for policy 1, policy_version 1479891 (0.0008) [2023-12-27 15:54:36,651][20381] Updated weights for policy 1, policy_version 1479901 (0.0007) [2023-12-27 15:54:36,748][20347] Updated weights for policy 0, policy_version 1481345 (0.0006) [2023-12-27 15:54:36,801][20347] Updated weights for policy 0, policy_version 1481355 (0.0005) [2023-12-27 15:54:36,857][20347] Updated weights for policy 0, policy_version 1481365 (0.0005) [2023-12-27 15:54:37,351][20381] Updated weights for policy 1, policy_version 1479911 (0.0010) [2023-12-27 15:54:37,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 758194176. Throughput: 0: 10314.9, 1: 10358.8. Samples: 758167700. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:37,363][19170] Avg episode reward: [(0, '951.140'), (1, '921.850')] [2023-12-27 15:54:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001481368_379305984.pth... [2023-12-27 15:54:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001480152_378994688.pth [2023-12-27 15:54:37,402][20381] Updated weights for policy 1, policy_version 1479921 (0.0008) [2023-12-27 15:54:37,456][20381] Updated weights for policy 1, policy_version 1479931 (0.0007) [2023-12-27 15:54:37,462][20347] Updated weights for policy 0, policy_version 1481375 (0.0006) [2023-12-27 15:54:37,486][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001479936_378896384.pth... [2023-12-27 15:54:37,490][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001478720_378585088.pth [2023-12-27 15:54:37,520][20347] Updated weights for policy 0, policy_version 1481385 (0.0006) [2023-12-27 15:54:37,576][20347] Updated weights for policy 0, policy_version 1481395 (0.0007) [2023-12-27 15:54:38,165][20381] Updated weights for policy 1, policy_version 1479941 (0.0007) [2023-12-27 15:54:38,221][20381] Updated weights for policy 1, policy_version 1479951 (0.0007) [2023-12-27 15:54:38,249][20347] Updated weights for policy 0, policy_version 1481405 (0.0009) [2023-12-27 15:54:38,277][20381] Updated weights for policy 1, policy_version 1479961 (0.0007) [2023-12-27 15:54:38,313][20347] Updated weights for policy 0, policy_version 1481415 (0.0007) [2023-12-27 15:54:38,378][20347] Updated weights for policy 0, policy_version 1481425 (0.0008) [2023-12-27 15:54:39,006][20381] Updated weights for policy 1, policy_version 1479971 (0.0008) [2023-12-27 15:54:39,055][20381] Updated weights for policy 1, policy_version 1479981 (0.0006) [2023-12-27 15:54:39,093][20347] Updated weights for policy 0, policy_version 1481435 (0.0008) [2023-12-27 15:54:39,113][20381] Updated weights for policy 1, policy_version 1479991 (0.0007) [2023-12-27 15:54:39,150][20347] Updated weights for policy 0, policy_version 1481445 (0.0008) [2023-12-27 15:54:39,206][20347] Updated weights for policy 0, policy_version 1481455 (0.0008) [2023-12-27 15:54:39,806][20381] Updated weights for policy 1, policy_version 1480001 (0.0008) [2023-12-27 15:54:39,864][20381] Updated weights for policy 1, policy_version 1480011 (0.0007) [2023-12-27 15:54:39,928][20381] Updated weights for policy 1, policy_version 1480021 (0.0007) [2023-12-27 15:54:39,959][20347] Updated weights for policy 0, policy_version 1481465 (0.0009) [2023-12-27 15:54:39,999][20381] Updated weights for policy 1, policy_version 1480031 (0.0009) [2023-12-27 15:54:40,018][20347] Updated weights for policy 0, policy_version 1481475 (0.0007) [2023-12-27 15:54:40,077][20347] Updated weights for policy 0, policy_version 1481485 (0.0006) [2023-12-27 15:54:40,133][20347] Updated weights for policy 0, policy_version 1481495 (0.0006) [2023-12-27 15:54:40,773][20381] Updated weights for policy 1, policy_version 1480041 (0.0008) [2023-12-27 15:54:40,785][20347] Updated weights for policy 0, policy_version 1481505 (0.0006) [2023-12-27 15:54:40,823][20381] Updated weights for policy 1, policy_version 1480051 (0.0010) [2023-12-27 15:54:40,839][20347] Updated weights for policy 0, policy_version 1481515 (0.0006) [2023-12-27 15:54:40,884][20381] Updated weights for policy 1, policy_version 1480061 (0.0010) [2023-12-27 15:54:40,886][20347] Updated weights for policy 0, policy_version 1481525 (0.0007) [2023-12-27 15:54:41,597][20347] Updated weights for policy 0, policy_version 1481535 (0.0008) [2023-12-27 15:54:41,629][20381] Updated weights for policy 1, policy_version 1480071 (0.0010) [2023-12-27 15:54:41,661][20347] Updated weights for policy 0, policy_version 1481545 (0.0008) [2023-12-27 15:54:41,692][20381] Updated weights for policy 1, policy_version 1480081 (0.0009) [2023-12-27 15:54:41,721][20347] Updated weights for policy 0, policy_version 1481555 (0.0006) [2023-12-27 15:54:41,747][20381] Updated weights for policy 1, policy_version 1480091 (0.0010) [2023-12-27 15:54:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 758292480. Throughput: 0: 10275.6, 1: 10258.1. Samples: 758288096. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:42,363][19170] Avg episode reward: [(0, '931.770'), (1, '921.820')] [2023-12-27 15:54:42,453][20347] Updated weights for policy 0, policy_version 1481565 (0.0008) [2023-12-27 15:54:42,462][20381] Updated weights for policy 1, policy_version 1480101 (0.0008) [2023-12-27 15:54:42,518][20347] Updated weights for policy 0, policy_version 1481575 (0.0008) [2023-12-27 15:54:42,528][20381] Updated weights for policy 1, policy_version 1480111 (0.0006) [2023-12-27 15:54:42,574][20347] Updated weights for policy 0, policy_version 1481585 (0.0008) [2023-12-27 15:54:42,584][20381] Updated weights for policy 1, policy_version 1480121 (0.0006) [2023-12-27 15:54:43,247][20347] Updated weights for policy 0, policy_version 1481595 (0.0009) [2023-12-27 15:54:43,281][20381] Updated weights for policy 1, policy_version 1480131 (0.0007) [2023-12-27 15:54:43,307][20347] Updated weights for policy 0, policy_version 1481605 (0.0009) [2023-12-27 15:54:43,333][20381] Updated weights for policy 1, policy_version 1480141 (0.0007) [2023-12-27 15:54:43,362][20347] Updated weights for policy 0, policy_version 1481615 (0.0005) [2023-12-27 15:54:43,390][20381] Updated weights for policy 1, policy_version 1480151 (0.0008) [2023-12-27 15:54:43,966][20347] Updated weights for policy 0, policy_version 1481625 (0.0005) [2023-12-27 15:54:44,021][20347] Updated weights for policy 0, policy_version 1481635 (0.0006) [2023-12-27 15:54:44,071][20347] Updated weights for policy 0, policy_version 1481645 (0.0010) [2023-12-27 15:54:44,083][20381] Updated weights for policy 1, policy_version 1480161 (0.0007) [2023-12-27 15:54:44,124][20347] Updated weights for policy 0, policy_version 1481655 (0.0006) [2023-12-27 15:54:44,145][20381] Updated weights for policy 1, policy_version 1480171 (0.0008) [2023-12-27 15:54:44,205][20381] Updated weights for policy 1, policy_version 1480181 (0.0008) [2023-12-27 15:54:44,259][20381] Updated weights for policy 1, policy_version 1480191 (0.0007) [2023-12-27 15:54:44,754][20347] Updated weights for policy 0, policy_version 1481665 (0.0007) [2023-12-27 15:54:44,814][20347] Updated weights for policy 0, policy_version 1481675 (0.0008) [2023-12-27 15:54:44,879][20347] Updated weights for policy 0, policy_version 1481685 (0.0007) [2023-12-27 15:54:44,949][20381] Updated weights for policy 1, policy_version 1480201 (0.0008) [2023-12-27 15:54:45,015][20381] Updated weights for policy 1, policy_version 1480211 (0.0008) [2023-12-27 15:54:45,072][20381] Updated weights for policy 1, policy_version 1480221 (0.0008) [2023-12-27 15:54:45,565][20347] Updated weights for policy 0, policy_version 1481695 (0.0008) [2023-12-27 15:54:45,618][20347] Updated weights for policy 0, policy_version 1481705 (0.0006) [2023-12-27 15:54:45,674][20347] Updated weights for policy 0, policy_version 1481715 (0.0008) [2023-12-27 15:54:45,796][20381] Updated weights for policy 1, policy_version 1480231 (0.0007) [2023-12-27 15:54:45,852][20381] Updated weights for policy 1, policy_version 1480241 (0.0007) [2023-12-27 15:54:45,912][20381] Updated weights for policy 1, policy_version 1480251 (0.0005) [2023-12-27 15:54:46,299][20347] Updated weights for policy 0, policy_version 1481725 (0.0007) [2023-12-27 15:54:46,361][20347] Updated weights for policy 0, policy_version 1481735 (0.0008) [2023-12-27 15:54:46,413][20347] Updated weights for policy 0, policy_version 1481745 (0.0005) [2023-12-27 15:54:46,590][20381] Updated weights for policy 1, policy_version 1480261 (0.0008) [2023-12-27 15:54:46,653][20381] Updated weights for policy 1, policy_version 1480271 (0.0008) [2023-12-27 15:54:46,712][20381] Updated weights for policy 1, policy_version 1480281 (0.0008) [2023-12-27 15:54:46,943][20347] Updated weights for policy 0, policy_version 1481755 (0.0006) [2023-12-27 15:54:47,009][20347] Updated weights for policy 0, policy_version 1481765 (0.0008) [2023-12-27 15:54:47,075][20347] Updated weights for policy 0, policy_version 1481775 (0.0008) [2023-12-27 15:54:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 758398976. Throughput: 0: 10362.0, 1: 10197.4. Samples: 758409780. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:47,363][19170] Avg episode reward: [(0, '931.670'), (1, '931.730')] [2023-12-27 15:54:47,436][20381] Updated weights for policy 1, policy_version 1480291 (0.0008) [2023-12-27 15:54:47,487][20381] Updated weights for policy 1, policy_version 1480301 (0.0007) [2023-12-27 15:54:47,543][20381] Updated weights for policy 1, policy_version 1480311 (0.0008) [2023-12-27 15:54:47,720][20347] Updated weights for policy 0, policy_version 1481785 (0.0008) [2023-12-27 15:54:47,777][20347] Updated weights for policy 0, policy_version 1481795 (0.0007) [2023-12-27 15:54:47,839][20347] Updated weights for policy 0, policy_version 1481805 (0.0008) [2023-12-27 15:54:47,892][20347] Updated weights for policy 0, policy_version 1481815 (0.0007) [2023-12-27 15:54:48,281][20381] Updated weights for policy 1, policy_version 1480321 (0.0009) [2023-12-27 15:54:48,344][20381] Updated weights for policy 1, policy_version 1480331 (0.0008) [2023-12-27 15:54:48,406][20381] Updated weights for policy 1, policy_version 1480341 (0.0010) [2023-12-27 15:54:48,465][20381] Updated weights for policy 1, policy_version 1480351 (0.0011) [2023-12-27 15:54:48,575][20347] Updated weights for policy 0, policy_version 1481825 (0.0007) [2023-12-27 15:54:48,637][20347] Updated weights for policy 0, policy_version 1481835 (0.0008) [2023-12-27 15:54:48,695][20347] Updated weights for policy 0, policy_version 1481845 (0.0008) [2023-12-27 15:54:49,144][20381] Updated weights for policy 1, policy_version 1480361 (0.0008) [2023-12-27 15:54:49,189][20381] Updated weights for policy 1, policy_version 1480371 (0.0008) [2023-12-27 15:54:49,247][20381] Updated weights for policy 1, policy_version 1480381 (0.0008) [2023-12-27 15:54:49,348][20347] Updated weights for policy 0, policy_version 1481855 (0.0008) [2023-12-27 15:54:49,419][20347] Updated weights for policy 0, policy_version 1481865 (0.0008) [2023-12-27 15:54:49,484][20347] Updated weights for policy 0, policy_version 1481875 (0.0008) [2023-12-27 15:54:50,021][20381] Updated weights for policy 1, policy_version 1480391 (0.0007) [2023-12-27 15:54:50,080][20381] Updated weights for policy 1, policy_version 1480401 (0.0007) [2023-12-27 15:54:50,134][20381] Updated weights for policy 1, policy_version 1480411 (0.0007) [2023-12-27 15:54:50,145][20347] Updated weights for policy 0, policy_version 1481885 (0.0008) [2023-12-27 15:54:50,200][20347] Updated weights for policy 0, policy_version 1481895 (0.0007) [2023-12-27 15:54:50,246][20291] KL-divergence is very high: 299.2082 [2023-12-27 15:54:50,259][20347] Updated weights for policy 0, policy_version 1481905 (0.0008) [2023-12-27 15:54:50,290][20291] KL-divergence is very high: 260.9754 [2023-12-27 15:54:50,859][20381] Updated weights for policy 1, policy_version 1480421 (0.0007) [2023-12-27 15:54:50,927][20381] Updated weights for policy 1, policy_version 1480431 (0.0006) [2023-12-27 15:54:50,943][20347] Updated weights for policy 0, policy_version 1481915 (0.0010) [2023-12-27 15:54:50,983][20381] Updated weights for policy 1, policy_version 1480441 (0.0006) [2023-12-27 15:54:50,996][20347] Updated weights for policy 0, policy_version 1481925 (0.0010) [2023-12-27 15:54:51,050][20347] Updated weights for policy 0, policy_version 1481935 (0.0011) [2023-12-27 15:54:51,608][20381] Updated weights for policy 1, policy_version 1480451 (0.0006) [2023-12-27 15:54:51,673][20381] Updated weights for policy 1, policy_version 1480461 (0.0008) [2023-12-27 15:54:51,736][20381] Updated weights for policy 1, policy_version 1480471 (0.0008) [2023-12-27 15:54:51,804][20347] Updated weights for policy 0, policy_version 1481945 (0.0010) [2023-12-27 15:54:51,855][20347] Updated weights for policy 0, policy_version 1481955 (0.0008) [2023-12-27 15:54:51,914][20347] Updated weights for policy 0, policy_version 1481965 (0.0007) [2023-12-27 15:54:51,970][20347] Updated weights for policy 0, policy_version 1481975 (0.0008) [2023-12-27 15:54:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 758497280. Throughput: 0: 10336.8, 1: 10138.0. Samples: 758469356. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:52,363][19170] Avg episode reward: [(0, '941.570'), (1, '951.320')] [2023-12-27 15:54:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001480480_379035648.pth... [2023-12-27 15:54:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001481976_379461632.pth... [2023-12-27 15:54:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001479328_378740736.pth [2023-12-27 15:54:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001480760_379150336.pth [2023-12-27 15:54:52,494][20381] Updated weights for policy 1, policy_version 1480481 (0.0008) [2023-12-27 15:54:52,542][20381] Updated weights for policy 1, policy_version 1480491 (0.0008) [2023-12-27 15:54:52,592][20381] Updated weights for policy 1, policy_version 1480501 (0.0008) [2023-12-27 15:54:52,648][20347] Updated weights for policy 0, policy_version 1481985 (0.0007) [2023-12-27 15:54:52,652][20381] Updated weights for policy 1, policy_version 1480511 (0.0008) [2023-12-27 15:54:52,703][20347] Updated weights for policy 0, policy_version 1481995 (0.0008) [2023-12-27 15:54:52,708][20291] KL-divergence is very high: 184.9547 [2023-12-27 15:54:52,747][20291] KL-divergence is very high: 151.2227 [2023-12-27 15:54:52,754][20347] Updated weights for policy 0, policy_version 1482005 (0.0007) [2023-12-27 15:54:53,328][20381] Updated weights for policy 1, policy_version 1480521 (0.0011) [2023-12-27 15:54:53,381][20381] Updated weights for policy 1, policy_version 1480531 (0.0011) [2023-12-27 15:54:53,409][20347] Updated weights for policy 0, policy_version 1482015 (0.0006) [2023-12-27 15:54:53,431][20381] Updated weights for policy 1, policy_version 1480541 (0.0011) [2023-12-27 15:54:53,468][20347] Updated weights for policy 0, policy_version 1482025 (0.0007) [2023-12-27 15:54:53,519][20347] Updated weights for policy 0, policy_version 1482035 (0.0008) [2023-12-27 15:54:54,138][20381] Updated weights for policy 1, policy_version 1480551 (0.0011) [2023-12-27 15:54:54,161][20347] Updated weights for policy 0, policy_version 1482045 (0.0008) [2023-12-27 15:54:54,187][20381] Updated weights for policy 1, policy_version 1480561 (0.0010) [2023-12-27 15:54:54,210][20347] Updated weights for policy 0, policy_version 1482055 (0.0008) [2023-12-27 15:54:54,235][20381] Updated weights for policy 1, policy_version 1480571 (0.0010) [2023-12-27 15:54:54,258][20347] Updated weights for policy 0, policy_version 1482065 (0.0008) [2023-12-27 15:54:54,862][20381] Updated weights for policy 1, policy_version 1480581 (0.0011) [2023-12-27 15:54:54,916][20381] Updated weights for policy 1, policy_version 1480591 (0.0010) [2023-12-27 15:54:54,975][20381] Updated weights for policy 1, policy_version 1480601 (0.0011) [2023-12-27 15:54:54,984][20347] Updated weights for policy 0, policy_version 1482075 (0.0005) [2023-12-27 15:54:55,042][20347] Updated weights for policy 0, policy_version 1482085 (0.0007) [2023-12-27 15:54:55,088][20347] Updated weights for policy 0, policy_version 1482095 (0.0007) [2023-12-27 15:54:55,633][20381] Updated weights for policy 1, policy_version 1480611 (0.0010) [2023-12-27 15:54:55,681][20381] Updated weights for policy 1, policy_version 1480621 (0.0010) [2023-12-27 15:54:55,735][20381] Updated weights for policy 1, policy_version 1480631 (0.0010) [2023-12-27 15:54:55,747][20347] Updated weights for policy 0, policy_version 1482105 (0.0007) [2023-12-27 15:54:55,811][20347] Updated weights for policy 0, policy_version 1482115 (0.0008) [2023-12-27 15:54:55,871][20347] Updated weights for policy 0, policy_version 1482125 (0.0007) [2023-12-27 15:54:55,935][20347] Updated weights for policy 0, policy_version 1482135 (0.0007) [2023-12-27 15:54:56,440][20381] Updated weights for policy 1, policy_version 1480641 (0.0011) [2023-12-27 15:54:56,488][20381] Updated weights for policy 1, policy_version 1480651 (0.0010) [2023-12-27 15:54:56,547][20381] Updated weights for policy 1, policy_version 1480661 (0.0011) [2023-12-27 15:54:56,574][20347] Updated weights for policy 0, policy_version 1482145 (0.0005) [2023-12-27 15:54:56,598][20381] Updated weights for policy 1, policy_version 1480671 (0.0010) [2023-12-27 15:54:56,622][20291] KL-divergence is very high: 147.2358 [2023-12-27 15:54:56,634][20347] Updated weights for policy 0, policy_version 1482155 (0.0006) [2023-12-27 15:54:56,667][20291] KL-divergence is very high: 175.4853 [2023-12-27 15:54:56,688][20347] Updated weights for policy 0, policy_version 1482165 (0.0009) [2023-12-27 15:54:57,271][20347] Updated weights for policy 0, policy_version 1482175 (0.0009) [2023-12-27 15:54:57,282][20381] Updated weights for policy 1, policy_version 1480681 (0.0010) [2023-12-27 15:54:57,326][20347] Updated weights for policy 0, policy_version 1482185 (0.0010) [2023-12-27 15:54:57,343][20381] Updated weights for policy 1, policy_version 1480691 (0.0011) [2023-12-27 15:54:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 758595584. Throughput: 0: 10366.3, 1: 10098.7. Samples: 758591144. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:54:57,363][19170] Avg episode reward: [(0, '941.480'), (1, '941.530')] [2023-12-27 15:54:57,380][20347] Updated weights for policy 0, policy_version 1482195 (0.0009) [2023-12-27 15:54:57,394][20381] Updated weights for policy 1, policy_version 1480701 (0.0010) [2023-12-27 15:54:58,044][20347] Updated weights for policy 0, policy_version 1482205 (0.0010) [2023-12-27 15:54:58,071][20381] Updated weights for policy 1, policy_version 1480711 (0.0010) [2023-12-27 15:54:58,092][20347] Updated weights for policy 0, policy_version 1482215 (0.0010) [2023-12-27 15:54:58,122][20381] Updated weights for policy 1, policy_version 1480721 (0.0010) [2023-12-27 15:54:58,143][20347] Updated weights for policy 0, policy_version 1482225 (0.0010) [2023-12-27 15:54:58,170][20381] Updated weights for policy 1, policy_version 1480731 (0.0010) [2023-12-27 15:54:58,852][20347] Updated weights for policy 0, policy_version 1482235 (0.0010) [2023-12-27 15:54:58,923][20347] Updated weights for policy 0, policy_version 1482245 (0.0009) [2023-12-27 15:54:58,986][20347] Updated weights for policy 0, policy_version 1482255 (0.0010) [2023-12-27 15:54:58,995][20381] Updated weights for policy 1, policy_version 1480741 (0.0010) [2023-12-27 15:54:59,055][20381] Updated weights for policy 1, policy_version 1480751 (0.0007) [2023-12-27 15:54:59,121][20381] Updated weights for policy 1, policy_version 1480761 (0.0008) [2023-12-27 15:54:59,735][20347] Updated weights for policy 0, policy_version 1482265 (0.0009) [2023-12-27 15:54:59,807][20347] Updated weights for policy 0, policy_version 1482275 (0.0006) [2023-12-27 15:54:59,869][20347] Updated weights for policy 0, policy_version 1482285 (0.0008) [2023-12-27 15:54:59,897][20381] Updated weights for policy 1, policy_version 1480771 (0.0008) [2023-12-27 15:54:59,933][20347] Updated weights for policy 0, policy_version 1482295 (0.0007) [2023-12-27 15:54:59,958][20381] Updated weights for policy 1, policy_version 1480781 (0.0007) [2023-12-27 15:55:00,023][20381] Updated weights for policy 1, policy_version 1480791 (0.0007) [2023-12-27 15:55:00,605][20347] Updated weights for policy 0, policy_version 1482305 (0.0005) [2023-12-27 15:55:00,648][20291] KL-divergence is very high: 101.9517 [2023-12-27 15:55:00,653][20347] Updated weights for policy 0, policy_version 1482315 (0.0007) [2023-12-27 15:55:00,697][20347] Updated weights for policy 0, policy_version 1482325 (0.0007) [2023-12-27 15:55:00,735][20381] Updated weights for policy 1, policy_version 1480801 (0.0007) [2023-12-27 15:55:00,801][20381] Updated weights for policy 1, policy_version 1480811 (0.0010) [2023-12-27 15:55:00,859][20381] Updated weights for policy 1, policy_version 1480821 (0.0010) [2023-12-27 15:55:00,916][20381] Updated weights for policy 1, policy_version 1480831 (0.0010) [2023-12-27 15:55:01,431][20347] Updated weights for policy 0, policy_version 1482335 (0.0007) [2023-12-27 15:55:01,482][20347] Updated weights for policy 0, policy_version 1482345 (0.0008) [2023-12-27 15:55:01,538][20347] Updated weights for policy 0, policy_version 1482355 (0.0008) [2023-12-27 15:55:01,660][20381] Updated weights for policy 1, policy_version 1480841 (0.0008) [2023-12-27 15:55:01,723][20381] Updated weights for policy 1, policy_version 1480851 (0.0008) [2023-12-27 15:55:01,785][20381] Updated weights for policy 1, policy_version 1480861 (0.0008) [2023-12-27 15:55:02,280][20347] Updated weights for policy 0, policy_version 1482365 (0.0009) [2023-12-27 15:55:02,340][20347] Updated weights for policy 0, policy_version 1482375 (0.0007) [2023-12-27 15:55:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 758693888. Throughput: 0: 10331.1, 1: 9982.1. Samples: 758709980. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:02,363][19170] Avg episode reward: [(0, '921.960'), (1, '941.420')] [2023-12-27 15:55:02,403][20347] Updated weights for policy 0, policy_version 1482385 (0.0008) [2023-12-27 15:55:02,481][20381] Updated weights for policy 1, policy_version 1480871 (0.0009) [2023-12-27 15:55:02,530][20381] Updated weights for policy 1, policy_version 1480881 (0.0007) [2023-12-27 15:55:02,583][20381] Updated weights for policy 1, policy_version 1480891 (0.0007) [2023-12-27 15:55:03,091][20347] Updated weights for policy 0, policy_version 1482395 (0.0007) [2023-12-27 15:55:03,136][20347] Updated weights for policy 0, policy_version 1482405 (0.0007) [2023-12-27 15:55:03,190][20347] Updated weights for policy 0, policy_version 1482415 (0.0008) [2023-12-27 15:55:03,204][20381] Updated weights for policy 1, policy_version 1480901 (0.0008) [2023-12-27 15:55:03,254][20381] Updated weights for policy 1, policy_version 1480911 (0.0005) [2023-12-27 15:55:03,301][20381] Updated weights for policy 1, policy_version 1480921 (0.0005) [2023-12-27 15:55:03,900][20381] Updated weights for policy 1, policy_version 1480931 (0.0007) [2023-12-27 15:55:03,924][20347] Updated weights for policy 0, policy_version 1482425 (0.0007) [2023-12-27 15:55:03,959][20381] Updated weights for policy 1, policy_version 1480941 (0.0010) [2023-12-27 15:55:03,976][20347] Updated weights for policy 0, policy_version 1482435 (0.0006) [2023-12-27 15:55:04,018][20381] Updated weights for policy 1, policy_version 1480951 (0.0010) [2023-12-27 15:55:04,031][20347] Updated weights for policy 0, policy_version 1482445 (0.0006) [2023-12-27 15:55:04,083][20347] Updated weights for policy 0, policy_version 1482455 (0.0008) [2023-12-27 15:55:04,666][20347] Updated weights for policy 0, policy_version 1482465 (0.0006) [2023-12-27 15:55:04,728][20347] Updated weights for policy 0, policy_version 1482475 (0.0005) [2023-12-27 15:55:04,739][20381] Updated weights for policy 1, policy_version 1480961 (0.0010) [2023-12-27 15:55:04,770][20347] Updated weights for policy 0, policy_version 1482485 (0.0006) [2023-12-27 15:55:04,793][20381] Updated weights for policy 1, policy_version 1480971 (0.0010) [2023-12-27 15:55:04,851][20381] Updated weights for policy 1, policy_version 1480981 (0.0010) [2023-12-27 15:55:04,908][20381] Updated weights for policy 1, policy_version 1480991 (0.0010) [2023-12-27 15:55:05,332][20347] Updated weights for policy 0, policy_version 1482495 (0.0007) [2023-12-27 15:55:05,390][20347] Updated weights for policy 0, policy_version 1482505 (0.0010) [2023-12-27 15:55:05,448][20347] Updated weights for policy 0, policy_version 1482515 (0.0010) [2023-12-27 15:55:05,591][20381] Updated weights for policy 1, policy_version 1481001 (0.0007) [2023-12-27 15:55:05,649][20381] Updated weights for policy 1, policy_version 1481011 (0.0007) [2023-12-27 15:55:05,706][20381] Updated weights for policy 1, policy_version 1481021 (0.0006) [2023-12-27 15:55:06,128][20347] Updated weights for policy 0, policy_version 1482525 (0.0009) [2023-12-27 15:55:06,187][20347] Updated weights for policy 0, policy_version 1482535 (0.0006) [2023-12-27 15:55:06,245][20347] Updated weights for policy 0, policy_version 1482545 (0.0007) [2023-12-27 15:55:06,432][20381] Updated weights for policy 1, policy_version 1481031 (0.0007) [2023-12-27 15:55:06,492][20381] Updated weights for policy 1, policy_version 1481041 (0.0008) [2023-12-27 15:55:06,547][20381] Updated weights for policy 1, policy_version 1481051 (0.0008) [2023-12-27 15:55:06,906][20347] Updated weights for policy 0, policy_version 1482555 (0.0007) [2023-12-27 15:55:06,978][20347] Updated weights for policy 0, policy_version 1482565 (0.0009) [2023-12-27 15:55:07,030][20347] Updated weights for policy 0, policy_version 1482575 (0.0010) [2023-12-27 15:55:07,165][20381] Updated weights for policy 1, policy_version 1481061 (0.0007) [2023-12-27 15:55:07,221][20381] Updated weights for policy 1, policy_version 1481071 (0.0007) [2023-12-27 15:55:07,270][20381] Updated weights for policy 1, policy_version 1481081 (0.0008) [2023-12-27 15:55:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 758808576. Throughput: 0: 10323.9, 1: 10005.4. Samples: 758773160. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:07,363][19170] Avg episode reward: [(0, '902.610'), (1, '941.380')] [2023-12-27 15:55:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001481088_379191296.pth... [2023-12-27 15:55:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001482584_379617280.pth... [2023-12-27 15:55:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001479936_378896384.pth [2023-12-27 15:55:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001481368_379305984.pth [2023-12-27 15:55:07,725][20347] Updated weights for policy 0, policy_version 1482585 (0.0010) [2023-12-27 15:55:07,770][20347] Updated weights for policy 0, policy_version 1482595 (0.0010) [2023-12-27 15:55:07,828][20347] Updated weights for policy 0, policy_version 1482605 (0.0010) [2023-12-27 15:55:07,880][20347] Updated weights for policy 0, policy_version 1482615 (0.0010) [2023-12-27 15:55:07,979][20381] Updated weights for policy 1, policy_version 1481091 (0.0008) [2023-12-27 15:55:08,031][20381] Updated weights for policy 1, policy_version 1481101 (0.0007) [2023-12-27 15:55:08,082][20381] Updated weights for policy 1, policy_version 1481111 (0.0007) [2023-12-27 15:55:08,579][20347] Updated weights for policy 0, policy_version 1482625 (0.0010) [2023-12-27 15:55:08,645][20347] Updated weights for policy 0, policy_version 1482635 (0.0010) [2023-12-27 15:55:08,692][20347] Updated weights for policy 0, policy_version 1482645 (0.0010) [2023-12-27 15:55:08,794][20381] Updated weights for policy 1, policy_version 1481121 (0.0007) [2023-12-27 15:55:08,851][20381] Updated weights for policy 1, policy_version 1481131 (0.0007) [2023-12-27 15:55:08,910][20381] Updated weights for policy 1, policy_version 1481141 (0.0011) [2023-12-27 15:55:08,967][20381] Updated weights for policy 1, policy_version 1481151 (0.0010) [2023-12-27 15:55:09,434][20347] Updated weights for policy 0, policy_version 1482655 (0.0011) [2023-12-27 15:55:09,490][20347] Updated weights for policy 0, policy_version 1482665 (0.0009) [2023-12-27 15:55:09,543][20347] Updated weights for policy 0, policy_version 1482675 (0.0011) [2023-12-27 15:55:09,647][20381] Updated weights for policy 1, policy_version 1481161 (0.0010) [2023-12-27 15:55:09,716][20381] Updated weights for policy 1, policy_version 1481171 (0.0011) [2023-12-27 15:55:09,783][20381] Updated weights for policy 1, policy_version 1481181 (0.0008) [2023-12-27 15:55:10,188][20347] Updated weights for policy 0, policy_version 1482685 (0.0011) [2023-12-27 15:55:10,251][20347] Updated weights for policy 0, policy_version 1482695 (0.0010) [2023-12-27 15:55:10,310][20347] Updated weights for policy 0, policy_version 1482705 (0.0011) [2023-12-27 15:55:10,461][20381] Updated weights for policy 1, policy_version 1481191 (0.0010) [2023-12-27 15:55:10,527][20381] Updated weights for policy 1, policy_version 1481201 (0.0011) [2023-12-27 15:55:10,582][20381] Updated weights for policy 1, policy_version 1481211 (0.0010) [2023-12-27 15:55:10,985][20347] Updated weights for policy 0, policy_version 1482715 (0.0009) [2023-12-27 15:55:11,042][20347] Updated weights for policy 0, policy_version 1482725 (0.0007) [2023-12-27 15:55:11,102][20347] Updated weights for policy 0, policy_version 1482735 (0.0008) [2023-12-27 15:55:11,260][20381] Updated weights for policy 1, policy_version 1481221 (0.0009) [2023-12-27 15:55:11,315][20381] Updated weights for policy 1, policy_version 1481231 (0.0007) [2023-12-27 15:55:11,384][20381] Updated weights for policy 1, policy_version 1481241 (0.0009) [2023-12-27 15:55:11,750][20347] Updated weights for policy 0, policy_version 1482745 (0.0008) [2023-12-27 15:55:11,810][20347] Updated weights for policy 0, policy_version 1482755 (0.0009) [2023-12-27 15:55:11,870][20347] Updated weights for policy 0, policy_version 1482765 (0.0009) [2023-12-27 15:55:11,929][20347] Updated weights for policy 0, policy_version 1482775 (0.0008) [2023-12-27 15:55:12,098][20381] Updated weights for policy 1, policy_version 1481251 (0.0006) [2023-12-27 15:55:12,157][20381] Updated weights for policy 1, policy_version 1481261 (0.0007) [2023-12-27 15:55:12,211][20381] Updated weights for policy 1, policy_version 1481271 (0.0008) [2023-12-27 15:55:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 758906880. Throughput: 0: 10310.7, 1: 10005.4. Samples: 758894936. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:12,363][19170] Avg episode reward: [(0, '931.950'), (1, '882.830')] [2023-12-27 15:55:12,630][20347] Updated weights for policy 0, policy_version 1482785 (0.0010) [2023-12-27 15:55:12,676][20347] Updated weights for policy 0, policy_version 1482795 (0.0008) [2023-12-27 15:55:12,729][20347] Updated weights for policy 0, policy_version 1482805 (0.0008) [2023-12-27 15:55:12,950][20381] Updated weights for policy 1, policy_version 1481281 (0.0007) [2023-12-27 15:55:13,013][20381] Updated weights for policy 1, policy_version 1481291 (0.0007) [2023-12-27 15:55:13,069][20381] Updated weights for policy 1, policy_version 1481301 (0.0008) [2023-12-27 15:55:13,125][20381] Updated weights for policy 1, policy_version 1481311 (0.0008) [2023-12-27 15:55:13,428][20347] Updated weights for policy 0, policy_version 1482815 (0.0008) [2023-12-27 15:55:13,472][20347] Updated weights for policy 0, policy_version 1482825 (0.0007) [2023-12-27 15:55:13,516][20347] Updated weights for policy 0, policy_version 1482835 (0.0007) [2023-12-27 15:55:13,732][20381] Updated weights for policy 1, policy_version 1481321 (0.0008) [2023-12-27 15:55:13,780][20381] Updated weights for policy 1, policy_version 1481331 (0.0007) [2023-12-27 15:55:13,830][20381] Updated weights for policy 1, policy_version 1481341 (0.0006) [2023-12-27 15:55:14,266][20347] Updated weights for policy 0, policy_version 1482845 (0.0007) [2023-12-27 15:55:14,326][20347] Updated weights for policy 0, policy_version 1482855 (0.0008) [2023-12-27 15:55:14,382][20347] Updated weights for policy 0, policy_version 1482865 (0.0008) [2023-12-27 15:55:14,408][20381] Updated weights for policy 1, policy_version 1481351 (0.0005) [2023-12-27 15:55:14,470][20381] Updated weights for policy 1, policy_version 1481361 (0.0005) [2023-12-27 15:55:14,529][20381] Updated weights for policy 1, policy_version 1481371 (0.0008) [2023-12-27 15:55:15,034][20347] Updated weights for policy 0, policy_version 1482875 (0.0008) [2023-12-27 15:55:15,077][20291] KL-divergence is very high: 136.1617 [2023-12-27 15:55:15,093][20347] Updated weights for policy 0, policy_version 1482885 (0.0011) [2023-12-27 15:55:15,098][20291] KL-divergence is very high: 282.2503 [2023-12-27 15:55:15,123][20291] KL-divergence is very high: 277.8382 [2023-12-27 15:55:15,147][20291] KL-divergence is very high: 240.7995 [2023-12-27 15:55:15,152][20347] Updated weights for policy 0, policy_version 1482895 (0.0011) [2023-12-27 15:55:15,171][20291] KL-divergence is very high: 140.5293 [2023-12-27 15:55:15,188][20381] Updated weights for policy 1, policy_version 1481381 (0.0006) [2023-12-27 15:55:15,250][20381] Updated weights for policy 1, policy_version 1481391 (0.0007) [2023-12-27 15:55:15,304][20381] Updated weights for policy 1, policy_version 1481401 (0.0007) [2023-12-27 15:55:15,797][20347] Updated weights for policy 0, policy_version 1482905 (0.0011) [2023-12-27 15:55:15,848][20347] Updated weights for policy 0, policy_version 1482915 (0.0010) [2023-12-27 15:55:15,896][20347] Updated weights for policy 0, policy_version 1482925 (0.0010) [2023-12-27 15:55:15,944][20347] Updated weights for policy 0, policy_version 1482935 (0.0010) [2023-12-27 15:55:16,027][20381] Updated weights for policy 1, policy_version 1481411 (0.0008) [2023-12-27 15:55:16,081][20381] Updated weights for policy 1, policy_version 1481421 (0.0006) [2023-12-27 15:55:16,134][20381] Updated weights for policy 1, policy_version 1481431 (0.0007) [2023-12-27 15:55:16,612][20347] Updated weights for policy 0, policy_version 1482945 (0.0008) [2023-12-27 15:55:16,671][20347] Updated weights for policy 0, policy_version 1482955 (0.0008) [2023-12-27 15:55:16,739][20347] Updated weights for policy 0, policy_version 1482965 (0.0007) [2023-12-27 15:55:16,805][20381] Updated weights for policy 1, policy_version 1481441 (0.0008) [2023-12-27 15:55:16,866][20381] Updated weights for policy 1, policy_version 1481451 (0.0008) [2023-12-27 15:55:16,923][20381] Updated weights for policy 1, policy_version 1481461 (0.0009) [2023-12-27 15:55:16,981][20381] Updated weights for policy 1, policy_version 1481471 (0.0008) [2023-12-27 15:55:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 759005184. Throughput: 0: 10285.8, 1: 9968.7. Samples: 759018040. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:17,364][19170] Avg episode reward: [(0, '931.810'), (1, '882.790')] [2023-12-27 15:55:17,366][20347] Updated weights for policy 0, policy_version 1482975 (0.0007) [2023-12-27 15:55:17,418][20347] Updated weights for policy 0, policy_version 1482985 (0.0008) [2023-12-27 15:55:17,486][20347] Updated weights for policy 0, policy_version 1482995 (0.0007) [2023-12-27 15:55:17,634][20381] Updated weights for policy 1, policy_version 1481481 (0.0006) [2023-12-27 15:55:17,695][20381] Updated weights for policy 1, policy_version 1481491 (0.0005) [2023-12-27 15:55:17,757][20381] Updated weights for policy 1, policy_version 1481501 (0.0005) [2023-12-27 15:55:18,155][20347] Updated weights for policy 0, policy_version 1483005 (0.0007) [2023-12-27 15:55:18,215][20347] Updated weights for policy 0, policy_version 1483015 (0.0010) [2023-12-27 15:55:18,267][20347] Updated weights for policy 0, policy_version 1483025 (0.0010) [2023-12-27 15:55:18,315][20381] Updated weights for policy 1, policy_version 1481511 (0.0005) [2023-12-27 15:55:18,379][20381] Updated weights for policy 1, policy_version 1481521 (0.0006) [2023-12-27 15:55:18,441][20381] Updated weights for policy 1, policy_version 1481531 (0.0010) [2023-12-27 15:55:18,964][20347] Updated weights for policy 0, policy_version 1483035 (0.0010) [2023-12-27 15:55:19,013][20347] Updated weights for policy 0, policy_version 1483045 (0.0010) [2023-12-27 15:55:19,034][20381] Updated weights for policy 1, policy_version 1481541 (0.0011) [2023-12-27 15:55:19,058][20347] Updated weights for policy 0, policy_version 1483055 (0.0010) [2023-12-27 15:55:19,092][20381] Updated weights for policy 1, policy_version 1481551 (0.0011) [2023-12-27 15:55:19,151][20381] Updated weights for policy 1, policy_version 1481561 (0.0010) [2023-12-27 15:55:19,823][20347] Updated weights for policy 0, policy_version 1483065 (0.0010) [2023-12-27 15:55:19,852][20381] Updated weights for policy 1, policy_version 1481571 (0.0010) [2023-12-27 15:55:19,889][20347] Updated weights for policy 0, policy_version 1483075 (0.0009) [2023-12-27 15:55:19,911][20381] Updated weights for policy 1, policy_version 1481581 (0.0011) [2023-12-27 15:55:19,944][20347] Updated weights for policy 0, policy_version 1483085 (0.0007) [2023-12-27 15:55:19,973][20381] Updated weights for policy 1, policy_version 1481591 (0.0010) [2023-12-27 15:55:20,009][20347] Updated weights for policy 0, policy_version 1483095 (0.0006) [2023-12-27 15:55:20,661][20381] Updated weights for policy 1, policy_version 1481601 (0.0007) [2023-12-27 15:55:20,710][20381] Updated weights for policy 1, policy_version 1481611 (0.0011) [2023-12-27 15:55:20,723][20347] Updated weights for policy 0, policy_version 1483105 (0.0006) [2023-12-27 15:55:20,760][20381] Updated weights for policy 1, policy_version 1481621 (0.0010) [2023-12-27 15:55:20,785][20347] Updated weights for policy 0, policy_version 1483115 (0.0007) [2023-12-27 15:55:20,823][20381] Updated weights for policy 1, policy_version 1481631 (0.0008) [2023-12-27 15:55:20,846][20347] Updated weights for policy 0, policy_version 1483125 (0.0006) [2023-12-27 15:55:21,526][20381] Updated weights for policy 1, policy_version 1481641 (0.0011) [2023-12-27 15:55:21,588][20381] Updated weights for policy 1, policy_version 1481651 (0.0010) [2023-12-27 15:55:21,604][20347] Updated weights for policy 0, policy_version 1483135 (0.0009) [2023-12-27 15:55:21,662][20381] Updated weights for policy 1, policy_version 1481661 (0.0009) [2023-12-27 15:55:21,669][20347] Updated weights for policy 0, policy_version 1483145 (0.0009) [2023-12-27 15:55:21,721][20347] Updated weights for policy 0, policy_version 1483155 (0.0009) [2023-12-27 15:55:22,347][20381] Updated weights for policy 1, policy_version 1481671 (0.0007) [2023-12-27 15:55:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 759103488. Throughput: 0: 10253.2, 1: 10011.0. Samples: 759079588. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:22,363][19170] Avg episode reward: [(0, '931.680'), (1, '882.760')] [2023-12-27 15:55:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001483160_379764736.pth... [2023-12-27 15:55:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001481976_379461632.pth [2023-12-27 15:55:22,413][20381] Updated weights for policy 1, policy_version 1481681 (0.0007) [2023-12-27 15:55:22,470][20381] Updated weights for policy 1, policy_version 1481691 (0.0008) [2023-12-27 15:55:22,499][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001481696_379346944.pth... [2023-12-27 15:55:22,502][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001480480_379035648.pth [2023-12-27 15:55:22,523][20347] Updated weights for policy 0, policy_version 1483165 (0.0007) [2023-12-27 15:55:22,548][20291] KL-divergence is very high: 154.7181 [2023-12-27 15:55:22,586][20347] Updated weights for policy 0, policy_version 1483175 (0.0006) [2023-12-27 15:55:22,598][20291] KL-divergence is very high: 179.6055 [2023-12-27 15:55:22,649][20347] Updated weights for policy 0, policy_version 1483185 (0.0007) [2023-12-27 15:55:23,129][20381] Updated weights for policy 1, policy_version 1481701 (0.0009) [2023-12-27 15:55:23,191][20381] Updated weights for policy 1, policy_version 1481711 (0.0011) [2023-12-27 15:55:23,252][20381] Updated weights for policy 1, policy_version 1481721 (0.0010) [2023-12-27 15:55:23,368][20347] Updated weights for policy 0, policy_version 1483195 (0.0008) [2023-12-27 15:55:23,424][20347] Updated weights for policy 0, policy_version 1483205 (0.0010) [2023-12-27 15:55:23,481][20347] Updated weights for policy 0, policy_version 1483215 (0.0010) [2023-12-27 15:55:23,886][20381] Updated weights for policy 1, policy_version 1481731 (0.0007) [2023-12-27 15:55:23,939][20381] Updated weights for policy 1, policy_version 1481741 (0.0005) [2023-12-27 15:55:23,994][20381] Updated weights for policy 1, policy_version 1481751 (0.0005) [2023-12-27 15:55:24,170][20347] Updated weights for policy 0, policy_version 1483225 (0.0010) [2023-12-27 15:55:24,228][20347] Updated weights for policy 0, policy_version 1483235 (0.0010) [2023-12-27 15:55:24,283][20347] Updated weights for policy 0, policy_version 1483245 (0.0010) [2023-12-27 15:55:24,338][20347] Updated weights for policy 0, policy_version 1483255 (0.0010) [2023-12-27 15:55:24,644][20381] Updated weights for policy 1, policy_version 1481761 (0.0006) [2023-12-27 15:55:24,695][20381] Updated weights for policy 1, policy_version 1481771 (0.0010) [2023-12-27 15:55:24,752][20381] Updated weights for policy 1, policy_version 1481781 (0.0010) [2023-12-27 15:55:24,813][20381] Updated weights for policy 1, policy_version 1481791 (0.0011) [2023-12-27 15:55:25,027][20347] Updated weights for policy 0, policy_version 1483265 (0.0010) [2023-12-27 15:55:25,082][20347] Updated weights for policy 0, policy_version 1483275 (0.0010) [2023-12-27 15:55:25,133][20347] Updated weights for policy 0, policy_version 1483285 (0.0010) [2023-12-27 15:55:25,423][20381] Updated weights for policy 1, policy_version 1481801 (0.0009) [2023-12-27 15:55:25,480][20381] Updated weights for policy 1, policy_version 1481811 (0.0010) [2023-12-27 15:55:25,540][20381] Updated weights for policy 1, policy_version 1481821 (0.0010) [2023-12-27 15:55:25,804][20347] Updated weights for policy 0, policy_version 1483295 (0.0009) [2023-12-27 15:55:25,867][20347] Updated weights for policy 0, policy_version 1483305 (0.0008) [2023-12-27 15:55:25,932][20347] Updated weights for policy 0, policy_version 1483315 (0.0008) [2023-12-27 15:55:26,147][20381] Updated weights for policy 1, policy_version 1481831 (0.0008) [2023-12-27 15:55:26,203][20381] Updated weights for policy 1, policy_version 1481841 (0.0007) [2023-12-27 15:55:26,263][20381] Updated weights for policy 1, policy_version 1481851 (0.0007) [2023-12-27 15:55:26,522][20347] Updated weights for policy 0, policy_version 1483325 (0.0008) [2023-12-27 15:55:26,573][20347] Updated weights for policy 0, policy_version 1483335 (0.0005) [2023-12-27 15:55:26,626][20347] Updated weights for policy 0, policy_version 1483345 (0.0008) [2023-12-27 15:55:26,898][20381] Updated weights for policy 1, policy_version 1481861 (0.0007) [2023-12-27 15:55:26,954][20381] Updated weights for policy 1, policy_version 1481871 (0.0006) [2023-12-27 15:55:27,007][20381] Updated weights for policy 1, policy_version 1481881 (0.0005) [2023-12-27 15:55:27,211][20347] Updated weights for policy 0, policy_version 1483355 (0.0010) [2023-12-27 15:55:27,259][20347] Updated weights for policy 0, policy_version 1483365 (0.0010) [2023-12-27 15:55:27,312][20347] Updated weights for policy 0, policy_version 1483375 (0.0011) [2023-12-27 15:55:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 759218176. Throughput: 0: 10181.6, 1: 10107.6. Samples: 759201108. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:27,363][19170] Avg episode reward: [(0, '912.160'), (1, '892.500')] [2023-12-27 15:55:27,562][20381] Updated weights for policy 1, policy_version 1481891 (0.0006) [2023-12-27 15:55:27,621][20381] Updated weights for policy 1, policy_version 1481901 (0.0007) [2023-12-27 15:55:27,681][20381] Updated weights for policy 1, policy_version 1481911 (0.0008) [2023-12-27 15:55:27,911][20347] Updated weights for policy 0, policy_version 1483385 (0.0009) [2023-12-27 15:55:27,962][20347] Updated weights for policy 0, policy_version 1483395 (0.0005) [2023-12-27 15:55:28,009][20347] Updated weights for policy 0, policy_version 1483405 (0.0006) [2023-12-27 15:55:28,061][20347] Updated weights for policy 0, policy_version 1483415 (0.0005) [2023-12-27 15:55:28,461][20381] Updated weights for policy 1, policy_version 1481921 (0.0009) [2023-12-27 15:55:28,515][20381] Updated weights for policy 1, policy_version 1481931 (0.0007) [2023-12-27 15:55:28,579][20381] Updated weights for policy 1, policy_version 1481941 (0.0008) [2023-12-27 15:55:28,629][20381] Updated weights for policy 1, policy_version 1481951 (0.0006) [2023-12-27 15:55:28,634][20347] Updated weights for policy 0, policy_version 1483425 (0.0010) [2023-12-27 15:55:28,692][20347] Updated weights for policy 0, policy_version 1483435 (0.0010) [2023-12-27 15:55:28,752][20347] Updated weights for policy 0, policy_version 1483445 (0.0010) [2023-12-27 15:55:29,198][20381] Updated weights for policy 1, policy_version 1481961 (0.0006) [2023-12-27 15:55:29,258][20381] Updated weights for policy 1, policy_version 1481971 (0.0008) [2023-12-27 15:55:29,323][20381] Updated weights for policy 1, policy_version 1481981 (0.0006) [2023-12-27 15:55:29,406][20347] Updated weights for policy 0, policy_version 1483455 (0.0009) [2023-12-27 15:55:29,458][20347] Updated weights for policy 0, policy_version 1483465 (0.0010) [2023-12-27 15:55:29,511][20347] Updated weights for policy 0, policy_version 1483475 (0.0008) [2023-12-27 15:55:29,990][20381] Updated weights for policy 1, policy_version 1481991 (0.0008) [2023-12-27 15:55:30,045][20381] Updated weights for policy 1, policy_version 1482001 (0.0008) [2023-12-27 15:55:30,105][20381] Updated weights for policy 1, policy_version 1482011 (0.0006) [2023-12-27 15:55:30,150][20347] Updated weights for policy 0, policy_version 1483485 (0.0006) [2023-12-27 15:55:30,210][20347] Updated weights for policy 0, policy_version 1483495 (0.0008) [2023-12-27 15:55:30,266][20347] Updated weights for policy 0, policy_version 1483505 (0.0008) [2023-12-27 15:55:30,772][20381] Updated weights for policy 1, policy_version 1482021 (0.0007) [2023-12-27 15:55:30,824][20381] Updated weights for policy 1, policy_version 1482031 (0.0007) [2023-12-27 15:55:30,872][20347] Updated weights for policy 0, policy_version 1483515 (0.0007) [2023-12-27 15:55:30,877][20381] Updated weights for policy 1, policy_version 1482041 (0.0007) [2023-12-27 15:55:30,927][20347] Updated weights for policy 0, policy_version 1483525 (0.0005) [2023-12-27 15:55:30,988][20347] Updated weights for policy 0, policy_version 1483535 (0.0006) [2023-12-27 15:55:31,573][20381] Updated weights for policy 1, policy_version 1482051 (0.0007) [2023-12-27 15:55:31,632][20381] Updated weights for policy 1, policy_version 1482061 (0.0006) [2023-12-27 15:55:31,696][20381] Updated weights for policy 1, policy_version 1482071 (0.0008) [2023-12-27 15:55:31,724][20347] Updated weights for policy 0, policy_version 1483545 (0.0010) [2023-12-27 15:55:31,783][20347] Updated weights for policy 0, policy_version 1483555 (0.0007) [2023-12-27 15:55:31,844][20347] Updated weights for policy 0, policy_version 1483565 (0.0007) [2023-12-27 15:55:31,893][20347] Updated weights for policy 0, policy_version 1483575 (0.0010) [2023-12-27 15:55:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 759316480. Throughput: 0: 10240.3, 1: 10191.9. Samples: 759329228. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:32,363][19170] Avg episode reward: [(0, '941.440'), (1, '892.680')] [2023-12-27 15:55:32,367][20381] Updated weights for policy 1, policy_version 1482081 (0.0008) [2023-12-27 15:55:32,428][20381] Updated weights for policy 1, policy_version 1482091 (0.0006) [2023-12-27 15:55:32,479][20381] Updated weights for policy 1, policy_version 1482101 (0.0007) [2023-12-27 15:55:32,527][20381] Updated weights for policy 1, policy_version 1482111 (0.0007) [2023-12-27 15:55:32,619][20347] Updated weights for policy 0, policy_version 1483585 (0.0008) [2023-12-27 15:55:32,668][20347] Updated weights for policy 0, policy_version 1483595 (0.0008) [2023-12-27 15:55:32,723][20347] Updated weights for policy 0, policy_version 1483605 (0.0008) [2023-12-27 15:55:33,145][20381] Updated weights for policy 1, policy_version 1482121 (0.0006) [2023-12-27 15:55:33,205][20381] Updated weights for policy 1, policy_version 1482131 (0.0005) [2023-12-27 15:55:33,258][20381] Updated weights for policy 1, policy_version 1482141 (0.0008) [2023-12-27 15:55:33,417][20347] Updated weights for policy 0, policy_version 1483615 (0.0006) [2023-12-27 15:55:33,463][20347] Updated weights for policy 0, policy_version 1483625 (0.0005) [2023-12-27 15:55:33,522][20347] Updated weights for policy 0, policy_version 1483635 (0.0006) [2023-12-27 15:55:33,852][20381] Updated weights for policy 1, policy_version 1482151 (0.0007) [2023-12-27 15:55:33,905][20381] Updated weights for policy 1, policy_version 1482161 (0.0007) [2023-12-27 15:55:33,951][20381] Updated weights for policy 1, policy_version 1482171 (0.0007) [2023-12-27 15:55:34,171][20347] Updated weights for policy 0, policy_version 1483645 (0.0006) [2023-12-27 15:55:34,227][20347] Updated weights for policy 0, policy_version 1483655 (0.0007) [2023-12-27 15:55:34,287][20347] Updated weights for policy 0, policy_version 1483665 (0.0007) [2023-12-27 15:55:34,628][20381] Updated weights for policy 1, policy_version 1482181 (0.0006) [2023-12-27 15:55:34,685][20381] Updated weights for policy 1, policy_version 1482191 (0.0005) [2023-12-27 15:55:34,742][20381] Updated weights for policy 1, policy_version 1482201 (0.0008) [2023-12-27 15:55:34,985][20347] Updated weights for policy 0, policy_version 1483675 (0.0006) [2023-12-27 15:55:35,043][20347] Updated weights for policy 0, policy_version 1483685 (0.0007) [2023-12-27 15:55:35,097][20347] Updated weights for policy 0, policy_version 1483695 (0.0008) [2023-12-27 15:55:35,367][20381] Updated weights for policy 1, policy_version 1482211 (0.0011) [2023-12-27 15:55:35,422][20381] Updated weights for policy 1, policy_version 1482221 (0.0007) [2023-12-27 15:55:35,477][20381] Updated weights for policy 1, policy_version 1482231 (0.0005) [2023-12-27 15:55:35,844][20347] Updated weights for policy 0, policy_version 1483705 (0.0008) [2023-12-27 15:55:35,899][20347] Updated weights for policy 0, policy_version 1483715 (0.0008) [2023-12-27 15:55:35,954][20347] Updated weights for policy 0, policy_version 1483725 (0.0008) [2023-12-27 15:55:36,017][20347] Updated weights for policy 0, policy_version 1483735 (0.0007) [2023-12-27 15:55:36,032][20381] Updated weights for policy 1, policy_version 1482241 (0.0005) [2023-12-27 15:55:36,097][20381] Updated weights for policy 1, policy_version 1482251 (0.0010) [2023-12-27 15:55:36,160][20381] Updated weights for policy 1, policy_version 1482261 (0.0007) [2023-12-27 15:55:36,209][20381] Updated weights for policy 1, policy_version 1482271 (0.0007) [2023-12-27 15:55:36,725][20347] Updated weights for policy 0, policy_version 1483745 (0.0007) [2023-12-27 15:55:36,787][20347] Updated weights for policy 0, policy_version 1483755 (0.0008) [2023-12-27 15:55:36,837][20347] Updated weights for policy 0, policy_version 1483765 (0.0008) [2023-12-27 15:55:36,868][20381] Updated weights for policy 1, policy_version 1482281 (0.0008) [2023-12-27 15:55:36,918][20381] Updated weights for policy 1, policy_version 1482291 (0.0008) [2023-12-27 15:55:36,967][20381] Updated weights for policy 1, policy_version 1482301 (0.0010) [2023-12-27 15:55:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 759422976. Throughput: 0: 10230.7, 1: 10291.2. Samples: 759392840. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:37,363][19170] Avg episode reward: [(0, '970.730'), (1, '892.810')] [2023-12-27 15:55:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001482304_379502592.pth... [2023-12-27 15:55:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001483768_379920384.pth... [2023-12-27 15:55:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001481088_379191296.pth [2023-12-27 15:55:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001482584_379617280.pth [2023-12-27 15:55:37,472][20347] Updated weights for policy 0, policy_version 1483775 (0.0010) [2023-12-27 15:55:37,528][20347] Updated weights for policy 0, policy_version 1483785 (0.0011) [2023-12-27 15:55:37,577][20347] Updated weights for policy 0, policy_version 1483795 (0.0010) [2023-12-27 15:55:37,605][20381] Updated weights for policy 1, policy_version 1482311 (0.0007) [2023-12-27 15:55:37,668][20381] Updated weights for policy 1, policy_version 1482321 (0.0005) [2023-12-27 15:55:37,727][20381] Updated weights for policy 1, policy_version 1482331 (0.0005) [2023-12-27 15:55:38,216][20347] Updated weights for policy 0, policy_version 1483805 (0.0010) [2023-12-27 15:55:38,275][20347] Updated weights for policy 0, policy_version 1483815 (0.0010) [2023-12-27 15:55:38,330][20347] Updated weights for policy 0, policy_version 1483825 (0.0010) [2023-12-27 15:55:38,372][20381] Updated weights for policy 1, policy_version 1482341 (0.0008) [2023-12-27 15:55:38,430][20381] Updated weights for policy 1, policy_version 1482351 (0.0011) [2023-12-27 15:55:38,489][20381] Updated weights for policy 1, policy_version 1482361 (0.0007) [2023-12-27 15:55:39,005][20347] Updated weights for policy 0, policy_version 1483835 (0.0010) [2023-12-27 15:55:39,071][20347] Updated weights for policy 0, policy_version 1483845 (0.0008) [2023-12-27 15:55:39,110][20381] Updated weights for policy 1, policy_version 1482371 (0.0009) [2023-12-27 15:55:39,124][20347] Updated weights for policy 0, policy_version 1483855 (0.0005) [2023-12-27 15:55:39,169][20381] Updated weights for policy 1, policy_version 1482381 (0.0008) [2023-12-27 15:55:39,235][20381] Updated weights for policy 1, policy_version 1482391 (0.0008) [2023-12-27 15:55:39,778][20347] Updated weights for policy 0, policy_version 1483865 (0.0005) [2023-12-27 15:55:39,845][20347] Updated weights for policy 0, policy_version 1483875 (0.0007) [2023-12-27 15:55:39,903][20347] Updated weights for policy 0, policy_version 1483885 (0.0006) [2023-12-27 15:55:39,938][20381] Updated weights for policy 1, policy_version 1482401 (0.0006) [2023-12-27 15:55:39,968][20347] Updated weights for policy 0, policy_version 1483895 (0.0007) [2023-12-27 15:55:40,002][20381] Updated weights for policy 1, policy_version 1482411 (0.0008) [2023-12-27 15:55:40,057][20381] Updated weights for policy 1, policy_version 1482421 (0.0008) [2023-12-27 15:55:40,120][20381] Updated weights for policy 1, policy_version 1482431 (0.0007) [2023-12-27 15:55:40,653][20347] Updated weights for policy 0, policy_version 1483905 (0.0006) [2023-12-27 15:55:40,713][20347] Updated weights for policy 0, policy_version 1483915 (0.0005) [2023-12-27 15:55:40,766][20347] Updated weights for policy 0, policy_version 1483925 (0.0006) [2023-12-27 15:55:40,770][20381] Updated weights for policy 1, policy_version 1482441 (0.0006) [2023-12-27 15:55:40,830][20381] Updated weights for policy 1, policy_version 1482451 (0.0005) [2023-12-27 15:55:40,889][20381] Updated weights for policy 1, policy_version 1482461 (0.0007) [2023-12-27 15:55:41,446][20347] Updated weights for policy 0, policy_version 1483935 (0.0006) [2023-12-27 15:55:41,505][20347] Updated weights for policy 0, policy_version 1483945 (0.0006) [2023-12-27 15:55:41,565][20347] Updated weights for policy 0, policy_version 1483955 (0.0008) [2023-12-27 15:55:41,605][20381] Updated weights for policy 1, policy_version 1482471 (0.0007) [2023-12-27 15:55:41,670][20381] Updated weights for policy 1, policy_version 1482481 (0.0007) [2023-12-27 15:55:41,737][20381] Updated weights for policy 1, policy_version 1482491 (0.0008) [2023-12-27 15:55:42,227][20347] Updated weights for policy 0, policy_version 1483965 (0.0007) [2023-12-27 15:55:42,290][20347] Updated weights for policy 0, policy_version 1483975 (0.0007) [2023-12-27 15:55:42,359][20347] Updated weights for policy 0, policy_version 1483985 (0.0008) [2023-12-27 15:55:42,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20521.7). Total num frames: 759521280. Throughput: 0: 10249.5, 1: 10349.8. Samples: 759518116. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:42,363][19170] Avg episode reward: [(0, '951.230'), (1, '921.970')] [2023-12-27 15:55:42,439][20381] Updated weights for policy 1, policy_version 1482501 (0.0008) [2023-12-27 15:55:42,500][20381] Updated weights for policy 1, policy_version 1482511 (0.0006) [2023-12-27 15:55:42,561][20381] Updated weights for policy 1, policy_version 1482521 (0.0008) [2023-12-27 15:55:43,041][20347] Updated weights for policy 0, policy_version 1483995 (0.0008) [2023-12-27 15:55:43,092][20347] Updated weights for policy 0, policy_version 1484005 (0.0005) [2023-12-27 15:55:43,145][20347] Updated weights for policy 0, policy_version 1484015 (0.0007) [2023-12-27 15:55:43,256][20381] Updated weights for policy 1, policy_version 1482531 (0.0007) [2023-12-27 15:55:43,305][20381] Updated weights for policy 1, policy_version 1482541 (0.0009) [2023-12-27 15:55:43,359][20381] Updated weights for policy 1, policy_version 1482551 (0.0008) [2023-12-27 15:55:43,732][20347] Updated weights for policy 0, policy_version 1484025 (0.0006) [2023-12-27 15:55:43,787][20347] Updated weights for policy 0, policy_version 1484035 (0.0007) [2023-12-27 15:55:43,836][20347] Updated weights for policy 0, policy_version 1484045 (0.0007) [2023-12-27 15:55:43,884][20347] Updated weights for policy 0, policy_version 1484055 (0.0009) [2023-12-27 15:55:44,055][20381] Updated weights for policy 1, policy_version 1482561 (0.0006) [2023-12-27 15:55:44,108][20381] Updated weights for policy 1, policy_version 1482571 (0.0007) [2023-12-27 15:55:44,158][20381] Updated weights for policy 1, policy_version 1482581 (0.0008) [2023-12-27 15:55:44,211][20381] Updated weights for policy 1, policy_version 1482591 (0.0010) [2023-12-27 15:55:44,543][20347] Updated weights for policy 0, policy_version 1484065 (0.0005) [2023-12-27 15:55:44,598][20347] Updated weights for policy 0, policy_version 1484075 (0.0005) [2023-12-27 15:55:44,650][20347] Updated weights for policy 0, policy_version 1484085 (0.0008) [2023-12-27 15:55:44,971][20381] Updated weights for policy 1, policy_version 1482601 (0.0008) [2023-12-27 15:55:45,025][20381] Updated weights for policy 1, policy_version 1482611 (0.0008) [2023-12-27 15:55:45,086][20381] Updated weights for policy 1, policy_version 1482621 (0.0008) [2023-12-27 15:55:45,292][20347] Updated weights for policy 0, policy_version 1484095 (0.0007) [2023-12-27 15:55:45,349][20347] Updated weights for policy 0, policy_version 1484105 (0.0008) [2023-12-27 15:55:45,395][20347] Updated weights for policy 0, policy_version 1484115 (0.0007) [2023-12-27 15:55:45,778][20381] Updated weights for policy 1, policy_version 1482631 (0.0007) [2023-12-27 15:55:45,843][20381] Updated weights for policy 1, policy_version 1482641 (0.0007) [2023-12-27 15:55:45,894][20381] Updated weights for policy 1, policy_version 1482652 (0.0009) [2023-12-27 15:55:46,032][20347] Updated weights for policy 0, policy_version 1484125 (0.0006) [2023-12-27 15:55:46,080][20347] Updated weights for policy 0, policy_version 1484135 (0.0006) [2023-12-27 15:55:46,130][20347] Updated weights for policy 0, policy_version 1484145 (0.0008) [2023-12-27 15:55:46,509][20381] Updated weights for policy 1, policy_version 1482662 (0.0005) [2023-12-27 15:55:46,562][20381] Updated weights for policy 1, policy_version 1482672 (0.0005) [2023-12-27 15:55:46,610][20381] Updated weights for policy 1, policy_version 1482682 (0.0005) [2023-12-27 15:55:46,757][20347] Updated weights for policy 0, policy_version 1484155 (0.0007) [2023-12-27 15:55:46,808][20347] Updated weights for policy 0, policy_version 1484165 (0.0005) [2023-12-27 15:55:46,857][20347] Updated weights for policy 0, policy_version 1484175 (0.0005) [2023-12-27 15:55:47,256][20381] Updated weights for policy 1, policy_version 1482692 (0.0009) [2023-12-27 15:55:47,304][20381] Updated weights for policy 1, policy_version 1482702 (0.0006) [2023-12-27 15:55:47,338][20292] KL-divergence is very high: 102.8144 [2023-12-27 15:55:47,361][20381] Updated weights for policy 1, policy_version 1482712 (0.0005) [2023-12-27 15:55:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 759627776. Throughput: 0: 10333.3, 1: 10400.8. Samples: 759643016. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:47,363][19170] Avg episode reward: [(0, '961.090'), (1, '951.200')] [2023-12-27 15:55:47,383][20292] KL-divergence is very high: 114.6908 [2023-12-27 15:55:47,556][20347] Updated weights for policy 0, policy_version 1484185 (0.0006) [2023-12-27 15:55:47,616][20347] Updated weights for policy 0, policy_version 1484195 (0.0005) [2023-12-27 15:55:47,672][20347] Updated weights for policy 0, policy_version 1484205 (0.0006) [2023-12-27 15:55:47,729][20347] Updated weights for policy 0, policy_version 1484215 (0.0008) [2023-12-27 15:55:47,990][20381] Updated weights for policy 1, policy_version 1482722 (0.0006) [2023-12-27 15:55:48,041][20381] Updated weights for policy 1, policy_version 1482732 (0.0007) [2023-12-27 15:55:48,085][20381] Updated weights for policy 1, policy_version 1482742 (0.0008) [2023-12-27 15:55:48,130][20381] Updated weights for policy 1, policy_version 1482752 (0.0007) [2023-12-27 15:55:48,398][20347] Updated weights for policy 0, policy_version 1484225 (0.0008) [2023-12-27 15:55:48,455][20347] Updated weights for policy 0, policy_version 1484235 (0.0008) [2023-12-27 15:55:48,511][20347] Updated weights for policy 0, policy_version 1484245 (0.0007) [2023-12-27 15:55:48,865][20381] Updated weights for policy 1, policy_version 1482762 (0.0008) [2023-12-27 15:55:48,926][20381] Updated weights for policy 1, policy_version 1482772 (0.0008) [2023-12-27 15:55:48,984][20381] Updated weights for policy 1, policy_version 1482782 (0.0007) [2023-12-27 15:55:49,180][20347] Updated weights for policy 0, policy_version 1484255 (0.0007) [2023-12-27 15:55:49,241][20347] Updated weights for policy 0, policy_version 1484265 (0.0008) [2023-12-27 15:55:49,299][20347] Updated weights for policy 0, policy_version 1484275 (0.0008) [2023-12-27 15:55:49,709][20381] Updated weights for policy 1, policy_version 1482792 (0.0007) [2023-12-27 15:55:49,771][20381] Updated weights for policy 1, policy_version 1482802 (0.0007) [2023-12-27 15:55:49,841][20381] Updated weights for policy 1, policy_version 1482812 (0.0008) [2023-12-27 15:55:50,028][20347] Updated weights for policy 0, policy_version 1484285 (0.0008) [2023-12-27 15:55:50,079][20347] Updated weights for policy 0, policy_version 1484295 (0.0008) [2023-12-27 15:55:50,137][20347] Updated weights for policy 0, policy_version 1484305 (0.0008) [2023-12-27 15:55:50,473][20381] Updated weights for policy 1, policy_version 1482822 (0.0008) [2023-12-27 15:55:50,527][20381] Updated weights for policy 1, policy_version 1482832 (0.0007) [2023-12-27 15:55:50,580][20381] Updated weights for policy 1, policy_version 1482842 (0.0007) [2023-12-27 15:55:50,835][20347] Updated weights for policy 0, policy_version 1484315 (0.0007) [2023-12-27 15:55:50,886][20291] KL-divergence is very high: 227.5484 [2023-12-27 15:55:50,892][20347] Updated weights for policy 0, policy_version 1484325 (0.0007) [2023-12-27 15:55:50,929][20291] KL-divergence is very high: 190.5611 [2023-12-27 15:55:50,946][20347] Updated weights for policy 0, policy_version 1484335 (0.0006) [2023-12-27 15:55:51,274][20381] Updated weights for policy 1, policy_version 1482852 (0.0006) [2023-12-27 15:55:51,325][20381] Updated weights for policy 1, policy_version 1482862 (0.0006) [2023-12-27 15:55:51,390][20381] Updated weights for policy 1, policy_version 1482872 (0.0007) [2023-12-27 15:55:51,666][20347] Updated weights for policy 0, policy_version 1484345 (0.0006) [2023-12-27 15:55:51,736][20347] Updated weights for policy 0, policy_version 1484355 (0.0008) [2023-12-27 15:55:51,802][20347] Updated weights for policy 0, policy_version 1484365 (0.0009) [2023-12-27 15:55:51,858][20347] Updated weights for policy 0, policy_version 1484375 (0.0007) [2023-12-27 15:55:52,111][20381] Updated weights for policy 1, policy_version 1482882 (0.0007) [2023-12-27 15:55:52,163][20381] Updated weights for policy 1, policy_version 1482892 (0.0010) [2023-12-27 15:55:52,217][20381] Updated weights for policy 1, policy_version 1482902 (0.0009) [2023-12-27 15:55:52,274][20381] Updated weights for policy 1, policy_version 1482912 (0.0006) [2023-12-27 15:55:52,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 759734272. Throughput: 0: 10297.4, 1: 10373.3. Samples: 759703336. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:52,363][19170] Avg episode reward: [(0, '921.940'), (1, '960.930')] [2023-12-27 15:55:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001484376_380076032.pth... [2023-12-27 15:55:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001482912_379658240.pth... [2023-12-27 15:55:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001481696_379346944.pth [2023-12-27 15:55:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001483160_379764736.pth [2023-12-27 15:55:52,541][20347] Updated weights for policy 0, policy_version 1484385 (0.0007) [2023-12-27 15:55:52,594][20347] Updated weights for policy 0, policy_version 1484395 (0.0006) [2023-12-27 15:55:52,657][20347] Updated weights for policy 0, policy_version 1484405 (0.0007) [2023-12-27 15:55:52,948][20381] Updated weights for policy 1, policy_version 1482922 (0.0008) [2023-12-27 15:55:53,007][20381] Updated weights for policy 1, policy_version 1482932 (0.0007) [2023-12-27 15:55:53,063][20381] Updated weights for policy 1, policy_version 1482942 (0.0008) [2023-12-27 15:55:53,340][20347] Updated weights for policy 0, policy_version 1484415 (0.0006) [2023-12-27 15:55:53,392][20347] Updated weights for policy 0, policy_version 1484425 (0.0005) [2023-12-27 15:55:53,440][20347] Updated weights for policy 0, policy_version 1484435 (0.0005) [2023-12-27 15:55:53,698][20381] Updated weights for policy 1, policy_version 1482952 (0.0006) [2023-12-27 15:55:53,755][20381] Updated weights for policy 1, policy_version 1482962 (0.0008) [2023-12-27 15:55:53,808][20381] Updated weights for policy 1, policy_version 1482972 (0.0008) [2023-12-27 15:55:54,091][20347] Updated weights for policy 0, policy_version 1484445 (0.0006) [2023-12-27 15:55:54,163][20347] Updated weights for policy 0, policy_version 1484455 (0.0005) [2023-12-27 15:55:54,229][20347] Updated weights for policy 0, policy_version 1484465 (0.0007) [2023-12-27 15:55:54,492][20381] Updated weights for policy 1, policy_version 1482982 (0.0009) [2023-12-27 15:55:54,541][20381] Updated weights for policy 1, policy_version 1482992 (0.0011) [2023-12-27 15:55:54,593][20381] Updated weights for policy 1, policy_version 1483002 (0.0011) [2023-12-27 15:55:54,806][20347] Updated weights for policy 0, policy_version 1484475 (0.0007) [2023-12-27 15:55:54,852][20347] Updated weights for policy 0, policy_version 1484485 (0.0007) [2023-12-27 15:55:54,898][20347] Updated weights for policy 0, policy_version 1484495 (0.0007) [2023-12-27 15:55:55,285][20381] Updated weights for policy 1, policy_version 1483012 (0.0008) [2023-12-27 15:55:55,341][20381] Updated weights for policy 1, policy_version 1483022 (0.0005) [2023-12-27 15:55:55,388][20381] Updated weights for policy 1, policy_version 1483032 (0.0007) [2023-12-27 15:55:55,512][20347] Updated weights for policy 0, policy_version 1484505 (0.0007) [2023-12-27 15:55:55,573][20347] Updated weights for policy 0, policy_version 1484515 (0.0005) [2023-12-27 15:55:55,632][20347] Updated weights for policy 0, policy_version 1484525 (0.0005) [2023-12-27 15:55:55,667][20291] KL-divergence is very high: 109.5578 [2023-12-27 15:55:55,685][20347] Updated weights for policy 0, policy_version 1484535 (0.0005) [2023-12-27 15:55:56,043][20381] Updated weights for policy 1, policy_version 1483042 (0.0008) [2023-12-27 15:55:56,102][20381] Updated weights for policy 1, policy_version 1483052 (0.0007) [2023-12-27 15:55:56,165][20381] Updated weights for policy 1, policy_version 1483062 (0.0011) [2023-12-27 15:55:56,212][20381] Updated weights for policy 1, policy_version 1483072 (0.0006) [2023-12-27 15:55:56,310][20347] Updated weights for policy 0, policy_version 1484545 (0.0008) [2023-12-27 15:55:56,363][20347] Updated weights for policy 0, policy_version 1484555 (0.0007) [2023-12-27 15:55:56,406][20347] Updated weights for policy 0, policy_version 1484565 (0.0005) [2023-12-27 15:55:56,775][20381] Updated weights for policy 1, policy_version 1483082 (0.0005) [2023-12-27 15:55:56,821][20381] Updated weights for policy 1, policy_version 1483092 (0.0005) [2023-12-27 15:55:56,864][20381] Updated weights for policy 1, policy_version 1483102 (0.0005) [2023-12-27 15:55:56,983][20347] Updated weights for policy 0, policy_version 1484575 (0.0006) [2023-12-27 15:55:57,040][20347] Updated weights for policy 0, policy_version 1484585 (0.0006) [2023-12-27 15:55:57,106][20347] Updated weights for policy 0, policy_version 1484595 (0.0006) [2023-12-27 15:55:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 759840768. Throughput: 0: 10331.8, 1: 10405.1. Samples: 759828092. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:55:57,363][19170] Avg episode reward: [(0, '883.010'), (1, '941.440')] [2023-12-27 15:55:57,530][20381] Updated weights for policy 1, policy_version 1483112 (0.0005) [2023-12-27 15:55:57,585][20381] Updated weights for policy 1, policy_version 1483122 (0.0005) [2023-12-27 15:55:57,637][20381] Updated weights for policy 1, policy_version 1483132 (0.0007) [2023-12-27 15:55:57,663][20347] Updated weights for policy 0, policy_version 1484605 (0.0006) [2023-12-27 15:55:57,710][20347] Updated weights for policy 0, policy_version 1484615 (0.0007) [2023-12-27 15:55:57,754][20347] Updated weights for policy 0, policy_version 1484625 (0.0007) [2023-12-27 15:55:58,249][20381] Updated weights for policy 1, policy_version 1483142 (0.0008) [2023-12-27 15:55:58,308][20381] Updated weights for policy 1, policy_version 1483152 (0.0008) [2023-12-27 15:55:58,380][20381] Updated weights for policy 1, policy_version 1483162 (0.0011) [2023-12-27 15:55:58,488][20347] Updated weights for policy 0, policy_version 1484635 (0.0008) [2023-12-27 15:55:58,550][20347] Updated weights for policy 0, policy_version 1484645 (0.0007) [2023-12-27 15:55:58,616][20347] Updated weights for policy 0, policy_version 1484655 (0.0008) [2023-12-27 15:55:59,117][20381] Updated weights for policy 1, policy_version 1483172 (0.0009) [2023-12-27 15:55:59,177][20381] Updated weights for policy 1, policy_version 1483182 (0.0008) [2023-12-27 15:55:59,243][20381] Updated weights for policy 1, policy_version 1483192 (0.0007) [2023-12-27 15:55:59,292][20347] Updated weights for policy 0, policy_version 1484665 (0.0007) [2023-12-27 15:55:59,369][20347] Updated weights for policy 0, policy_version 1484675 (0.0008) [2023-12-27 15:55:59,426][20347] Updated weights for policy 0, policy_version 1484685 (0.0008) [2023-12-27 15:55:59,480][20347] Updated weights for policy 0, policy_version 1484695 (0.0009) [2023-12-27 15:55:59,887][20381] Updated weights for policy 1, policy_version 1483202 (0.0007) [2023-12-27 15:55:59,955][20381] Updated weights for policy 1, policy_version 1483212 (0.0008) [2023-12-27 15:56:00,010][20381] Updated weights for policy 1, policy_version 1483223 (0.0008) [2023-12-27 15:56:00,218][20347] Updated weights for policy 0, policy_version 1484705 (0.0008) [2023-12-27 15:56:00,266][20347] Updated weights for policy 0, policy_version 1484715 (0.0008) [2023-12-27 15:56:00,322][20347] Updated weights for policy 0, policy_version 1484725 (0.0007) [2023-12-27 15:56:00,633][20381] Updated weights for policy 1, policy_version 1483233 (0.0006) [2023-12-27 15:56:00,680][20381] Updated weights for policy 1, policy_version 1483243 (0.0007) [2023-12-27 15:56:00,734][20381] Updated weights for policy 1, policy_version 1483253 (0.0007) [2023-12-27 15:56:00,780][20381] Updated weights for policy 1, policy_version 1483263 (0.0008) [2023-12-27 15:56:01,031][20347] Updated weights for policy 0, policy_version 1484735 (0.0009) [2023-12-27 15:56:01,094][20347] Updated weights for policy 0, policy_version 1484745 (0.0008) [2023-12-27 15:56:01,158][20347] Updated weights for policy 0, policy_version 1484755 (0.0007) [2023-12-27 15:56:01,481][20381] Updated weights for policy 1, policy_version 1483273 (0.0011) [2023-12-27 15:56:01,542][20381] Updated weights for policy 1, policy_version 1483283 (0.0011) [2023-12-27 15:56:01,607][20381] Updated weights for policy 1, policy_version 1483293 (0.0011) [2023-12-27 15:56:01,790][20347] Updated weights for policy 0, policy_version 1484765 (0.0008) [2023-12-27 15:56:01,848][20347] Updated weights for policy 0, policy_version 1484775 (0.0007) [2023-12-27 15:56:01,904][20347] Updated weights for policy 0, policy_version 1484785 (0.0008) [2023-12-27 15:56:02,260][20381] Updated weights for policy 1, policy_version 1483303 (0.0009) [2023-12-27 15:56:02,322][20381] Updated weights for policy 1, policy_version 1483313 (0.0011) [2023-12-27 15:56:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 759939072. Throughput: 0: 10326.2, 1: 10463.2. Samples: 759953556. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:56:02,363][19170] Avg episode reward: [(0, '912.310'), (1, '922.400')] [2023-12-27 15:56:02,396][20381] Updated weights for policy 1, policy_version 1483323 (0.0008) [2023-12-27 15:56:02,634][20347] Updated weights for policy 0, policy_version 1484795 (0.0007) [2023-12-27 15:56:02,696][20347] Updated weights for policy 0, policy_version 1484805 (0.0007) [2023-12-27 15:56:02,756][20347] Updated weights for policy 0, policy_version 1484815 (0.0009) [2023-12-27 15:56:03,046][20381] Updated weights for policy 1, policy_version 1483333 (0.0006) [2023-12-27 15:56:03,109][20381] Updated weights for policy 1, policy_version 1483343 (0.0008) [2023-12-27 15:56:03,166][20381] Updated weights for policy 1, policy_version 1483353 (0.0010) [2023-12-27 15:56:03,351][20347] Updated weights for policy 0, policy_version 1484825 (0.0006) [2023-12-27 15:56:03,407][20347] Updated weights for policy 0, policy_version 1484835 (0.0006) [2023-12-27 15:56:03,474][20347] Updated weights for policy 0, policy_version 1484845 (0.0007) [2023-12-27 15:56:03,532][20347] Updated weights for policy 0, policy_version 1484855 (0.0007) [2023-12-27 15:56:03,778][20381] Updated weights for policy 1, policy_version 1483363 (0.0009) [2023-12-27 15:56:03,846][20381] Updated weights for policy 1, policy_version 1483376 (0.0006) [2023-12-27 15:56:03,906][20381] Updated weights for policy 1, policy_version 1483386 (0.0008) [2023-12-27 15:56:04,203][20347] Updated weights for policy 0, policy_version 1484865 (0.0007) [2023-12-27 15:56:04,264][20347] Updated weights for policy 0, policy_version 1484875 (0.0008) [2023-12-27 15:56:04,323][20347] Updated weights for policy 0, policy_version 1484885 (0.0007) [2023-12-27 15:56:04,633][20381] Updated weights for policy 1, policy_version 1483396 (0.0009) [2023-12-27 15:56:04,687][20381] Updated weights for policy 1, policy_version 1483406 (0.0011) [2023-12-27 15:56:04,743][20381] Updated weights for policy 1, policy_version 1483416 (0.0011) [2023-12-27 15:56:04,983][20347] Updated weights for policy 0, policy_version 1484895 (0.0008) [2023-12-27 15:56:05,039][20347] Updated weights for policy 0, policy_version 1484905 (0.0007) [2023-12-27 15:56:05,095][20347] Updated weights for policy 0, policy_version 1484915 (0.0007) [2023-12-27 15:56:05,447][20381] Updated weights for policy 1, policy_version 1483426 (0.0011) [2023-12-27 15:56:05,492][20381] Updated weights for policy 1, policy_version 1483436 (0.0010) [2023-12-27 15:56:05,538][20381] Updated weights for policy 1, policy_version 1483446 (0.0010) [2023-12-27 15:56:05,590][20381] Updated weights for policy 1, policy_version 1483456 (0.0010) [2023-12-27 15:56:05,666][20347] Updated weights for policy 0, policy_version 1484925 (0.0006) [2023-12-27 15:56:05,715][20347] Updated weights for policy 0, policy_version 1484935 (0.0007) [2023-12-27 15:56:05,765][20347] Updated weights for policy 0, policy_version 1484945 (0.0007) [2023-12-27 15:56:06,312][20381] Updated weights for policy 1, policy_version 1483466 (0.0008) [2023-12-27 15:56:06,372][20381] Updated weights for policy 1, policy_version 1483476 (0.0008) [2023-12-27 15:56:06,434][20381] Updated weights for policy 1, policy_version 1483486 (0.0008) [2023-12-27 15:56:06,467][20347] Updated weights for policy 0, policy_version 1484955 (0.0007) [2023-12-27 15:56:06,524][20347] Updated weights for policy 0, policy_version 1484965 (0.0007) [2023-12-27 15:56:06,582][20347] Updated weights for policy 0, policy_version 1484975 (0.0007) [2023-12-27 15:56:07,174][20381] Updated weights for policy 1, policy_version 1483496 (0.0008) [2023-12-27 15:56:07,236][20381] Updated weights for policy 1, policy_version 1483506 (0.0006) [2023-12-27 15:56:07,272][20347] Updated weights for policy 0, policy_version 1484985 (0.0008) [2023-12-27 15:56:07,303][20381] Updated weights for policy 1, policy_version 1483516 (0.0007) [2023-12-27 15:56:07,331][20347] Updated weights for policy 0, policy_version 1484995 (0.0008) [2023-12-27 15:56:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 760045568. Throughput: 0: 10367.4, 1: 10425.3. Samples: 760015264. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-12-27 15:56:07,363][19170] Avg episode reward: [(0, '951.480'), (1, '912.150')] [2023-12-27 15:56:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001483520_379813888.pth... [2023-12-27 15:56:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001482304_379502592.pth [2023-12-27 15:56:07,387][20347] Updated weights for policy 0, policy_version 1485005 (0.0008) [2023-12-27 15:56:07,444][20347] Updated weights for policy 0, policy_version 1485015 (0.0007) [2023-12-27 15:56:07,448][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001485016_380239872.pth... [2023-12-27 15:56:07,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001483768_379920384.pth [2023-12-27 15:56:07,944][20381] Updated weights for policy 1, policy_version 1483526 (0.0009) [2023-12-27 15:56:07,990][20381] Updated weights for policy 1, policy_version 1483536 (0.0010) [2023-12-27 15:56:08,050][20381] Updated weights for policy 1, policy_version 1483546 (0.0007) [2023-12-27 15:56:08,083][20347] Updated weights for policy 0, policy_version 1485025 (0.0006) [2023-12-27 15:56:08,136][20347] Updated weights for policy 0, policy_version 1485035 (0.0009) [2023-12-27 15:56:08,193][20347] Updated weights for policy 0, policy_version 1485045 (0.0006) [2023-12-27 15:56:08,718][20381] Updated weights for policy 1, policy_version 1483556 (0.0007) [2023-12-27 15:56:08,777][20381] Updated weights for policy 1, policy_version 1483566 (0.0008) [2023-12-27 15:56:08,829][20381] Updated weights for policy 1, policy_version 1483576 (0.0007) [2023-12-27 15:56:08,877][20347] Updated weights for policy 0, policy_version 1485055 (0.0006) [2023-12-27 15:56:08,932][20347] Updated weights for policy 0, policy_version 1485065 (0.0007) [2023-12-27 15:56:08,987][20347] Updated weights for policy 0, policy_version 1485075 (0.0007) [2023-12-27 15:56:09,498][20381] Updated weights for policy 1, policy_version 1483586 (0.0007) [2023-12-27 15:56:09,559][20381] Updated weights for policy 1, policy_version 1483596 (0.0008) [2023-12-27 15:56:09,618][20381] Updated weights for policy 1, policy_version 1483606 (0.0006) [2023-12-27 15:56:09,681][20381] Updated weights for policy 1, policy_version 1483616 (0.0007) [2023-12-27 15:56:09,697][20347] Updated weights for policy 0, policy_version 1485085 (0.0008) [2023-12-27 15:56:09,747][20347] Updated weights for policy 0, policy_version 1485095 (0.0009) [2023-12-27 15:56:09,798][20347] Updated weights for policy 0, policy_version 1485105 (0.0008) [2023-12-27 15:56:10,316][20381] Updated weights for policy 1, policy_version 1483626 (0.0005) [2023-12-27 15:56:10,376][20381] Updated weights for policy 1, policy_version 1483636 (0.0011) [2023-12-27 15:56:10,434][20381] Updated weights for policy 1, policy_version 1483646 (0.0009) [2023-12-27 15:56:10,519][20347] Updated weights for policy 0, policy_version 1485115 (0.0008) [2023-12-27 15:56:10,574][20347] Updated weights for policy 0, policy_version 1485125 (0.0007) [2023-12-27 15:56:10,631][20347] Updated weights for policy 0, policy_version 1485135 (0.0008) [2023-12-27 15:56:11,070][20381] Updated weights for policy 1, policy_version 1483656 (0.0008) [2023-12-27 15:56:11,126][20381] Updated weights for policy 1, policy_version 1483666 (0.0009) [2023-12-27 15:56:11,192][20381] Updated weights for policy 1, policy_version 1483676 (0.0008) [2023-12-27 15:56:11,314][20347] Updated weights for policy 0, policy_version 1485145 (0.0007) [2023-12-27 15:56:11,385][20347] Updated weights for policy 0, policy_version 1485155 (0.0009) [2023-12-27 15:56:11,444][20347] Updated weights for policy 0, policy_version 1485165 (0.0010) [2023-12-27 15:56:11,505][20347] Updated weights for policy 0, policy_version 1485175 (0.0011) [2023-12-27 15:56:11,910][20381] Updated weights for policy 1, policy_version 1483686 (0.0008) [2023-12-27 15:56:11,967][20381] Updated weights for policy 1, policy_version 1483696 (0.0007) [2023-12-27 15:56:12,031][20381] Updated weights for policy 1, policy_version 1483706 (0.0007) [2023-12-27 15:56:12,362][20347] Updated weights for policy 0, policy_version 1485185 (0.0008) [2023-12-27 15:56:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 760143872. Throughput: 0: 10432.7, 1: 10371.9. Samples: 760137316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:12,363][19170] Avg episode reward: [(0, '961.150'), (1, '872.980')] [2023-12-27 15:56:12,425][20347] Updated weights for policy 0, policy_version 1485195 (0.0008) [2023-12-27 15:56:12,486][20347] Updated weights for policy 0, policy_version 1485205 (0.0008) [2023-12-27 15:56:12,628][20381] Updated weights for policy 1, policy_version 1483716 (0.0006) [2023-12-27 15:56:12,685][20381] Updated weights for policy 1, policy_version 1483726 (0.0007) [2023-12-27 15:56:12,744][20381] Updated weights for policy 1, policy_version 1483736 (0.0007) [2023-12-27 15:56:13,208][20347] Updated weights for policy 0, policy_version 1485215 (0.0008) [2023-12-27 15:56:13,269][20347] Updated weights for policy 0, policy_version 1485225 (0.0009) [2023-12-27 15:56:13,330][20347] Updated weights for policy 0, policy_version 1485235 (0.0007) [2023-12-27 15:56:13,376][20381] Updated weights for policy 1, policy_version 1483746 (0.0007) [2023-12-27 15:56:13,442][20381] Updated weights for policy 1, policy_version 1483756 (0.0007) [2023-12-27 15:56:13,495][20381] Updated weights for policy 1, policy_version 1483766 (0.0007) [2023-12-27 15:56:13,564][20381] Updated weights for policy 1, policy_version 1483776 (0.0006) [2023-12-27 15:56:14,080][20381] Updated weights for policy 1, policy_version 1483786 (0.0006) [2023-12-27 15:56:14,104][20347] Updated weights for policy 0, policy_version 1485245 (0.0008) [2023-12-27 15:56:14,141][20381] Updated weights for policy 1, policy_version 1483796 (0.0005) [2023-12-27 15:56:14,155][20347] Updated weights for policy 0, policy_version 1485255 (0.0005) [2023-12-27 15:56:14,195][20381] Updated weights for policy 1, policy_version 1483806 (0.0005) [2023-12-27 15:56:14,209][20347] Updated weights for policy 0, policy_version 1485265 (0.0006) [2023-12-27 15:56:14,786][20381] Updated weights for policy 1, policy_version 1483816 (0.0008) [2023-12-27 15:56:14,821][20347] Updated weights for policy 0, policy_version 1485275 (0.0010) [2023-12-27 15:56:14,852][20381] Updated weights for policy 1, policy_version 1483826 (0.0008) [2023-12-27 15:56:14,882][20347] Updated weights for policy 0, policy_version 1485285 (0.0009) [2023-12-27 15:56:14,915][20381] Updated weights for policy 1, policy_version 1483836 (0.0007) [2023-12-27 15:56:14,945][20347] Updated weights for policy 0, policy_version 1485295 (0.0006) [2023-12-27 15:56:15,599][20381] Updated weights for policy 1, policy_version 1483846 (0.0009) [2023-12-27 15:56:15,660][20347] Updated weights for policy 0, policy_version 1485305 (0.0007) [2023-12-27 15:56:15,665][20381] Updated weights for policy 1, policy_version 1483856 (0.0007) [2023-12-27 15:56:15,719][20347] Updated weights for policy 0, policy_version 1485315 (0.0009) [2023-12-27 15:56:15,724][20381] Updated weights for policy 1, policy_version 1483866 (0.0005) [2023-12-27 15:56:15,777][20347] Updated weights for policy 0, policy_version 1485325 (0.0007) [2023-12-27 15:56:15,835][20347] Updated weights for policy 0, policy_version 1485335 (0.0006) [2023-12-27 15:56:16,299][20381] Updated weights for policy 1, policy_version 1483876 (0.0006) [2023-12-27 15:56:16,352][20381] Updated weights for policy 1, policy_version 1483886 (0.0005) [2023-12-27 15:56:16,407][20381] Updated weights for policy 1, policy_version 1483896 (0.0005) [2023-12-27 15:56:16,521][20347] Updated weights for policy 0, policy_version 1485345 (0.0008) [2023-12-27 15:56:16,569][20347] Updated weights for policy 0, policy_version 1485355 (0.0009) [2023-12-27 15:56:16,613][20347] Updated weights for policy 0, policy_version 1485365 (0.0007) [2023-12-27 15:56:16,975][20381] Updated weights for policy 1, policy_version 1483906 (0.0005) [2023-12-27 15:56:17,022][20381] Updated weights for policy 1, policy_version 1483916 (0.0007) [2023-12-27 15:56:17,071][20381] Updated weights for policy 1, policy_version 1483926 (0.0007) [2023-12-27 15:56:17,118][20381] Updated weights for policy 1, policy_version 1483936 (0.0008) [2023-12-27 15:56:17,252][20347] Updated weights for policy 0, policy_version 1485375 (0.0007) [2023-12-27 15:56:17,306][20347] Updated weights for policy 0, policy_version 1485385 (0.0007) [2023-12-27 15:56:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 760250368. Throughput: 0: 10270.0, 1: 10456.7. Samples: 760261928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:17,363][19170] Avg episode reward: [(0, '961.050'), (1, '902.290')] [2023-12-27 15:56:17,365][20347] Updated weights for policy 0, policy_version 1485395 (0.0008) [2023-12-27 15:56:17,833][20381] Updated weights for policy 1, policy_version 1483946 (0.0007) [2023-12-27 15:56:17,888][20381] Updated weights for policy 1, policy_version 1483956 (0.0008) [2023-12-27 15:56:17,930][20381] Updated weights for policy 1, policy_version 1483966 (0.0007) [2023-12-27 15:56:18,008][20347] Updated weights for policy 0, policy_version 1485405 (0.0006) [2023-12-27 15:56:18,063][20347] Updated weights for policy 0, policy_version 1485415 (0.0007) [2023-12-27 15:56:18,126][20347] Updated weights for policy 0, policy_version 1485425 (0.0008) [2023-12-27 15:56:18,683][20381] Updated weights for policy 1, policy_version 1483976 (0.0008) [2023-12-27 15:56:18,738][20381] Updated weights for policy 1, policy_version 1483986 (0.0008) [2023-12-27 15:56:18,796][20381] Updated weights for policy 1, policy_version 1483996 (0.0008) [2023-12-27 15:56:18,813][20347] Updated weights for policy 0, policy_version 1485435 (0.0008) [2023-12-27 15:56:18,874][20347] Updated weights for policy 0, policy_version 1485445 (0.0008) [2023-12-27 15:56:18,934][20347] Updated weights for policy 0, policy_version 1485455 (0.0008) [2023-12-27 15:56:19,488][20381] Updated weights for policy 1, policy_version 1484006 (0.0007) [2023-12-27 15:56:19,551][20381] Updated weights for policy 1, policy_version 1484016 (0.0008) [2023-12-27 15:56:19,610][20381] Updated weights for policy 1, policy_version 1484026 (0.0008) [2023-12-27 15:56:19,627][20347] Updated weights for policy 0, policy_version 1485465 (0.0007) [2023-12-27 15:56:19,689][20347] Updated weights for policy 0, policy_version 1485475 (0.0009) [2023-12-27 15:56:19,749][20347] Updated weights for policy 0, policy_version 1485485 (0.0008) [2023-12-27 15:56:19,809][20347] Updated weights for policy 0, policy_version 1485495 (0.0008) [2023-12-27 15:56:20,341][20381] Updated weights for policy 1, policy_version 1484036 (0.0008) [2023-12-27 15:56:20,400][20381] Updated weights for policy 1, policy_version 1484046 (0.0008) [2023-12-27 15:56:20,457][20381] Updated weights for policy 1, policy_version 1484056 (0.0008) [2023-12-27 15:56:20,531][20347] Updated weights for policy 0, policy_version 1485505 (0.0007) [2023-12-27 15:56:20,593][20347] Updated weights for policy 0, policy_version 1485515 (0.0007) [2023-12-27 15:56:20,652][20347] Updated weights for policy 0, policy_version 1485525 (0.0009) [2023-12-27 15:56:21,134][20381] Updated weights for policy 1, policy_version 1484066 (0.0009) [2023-12-27 15:56:21,189][20381] Updated weights for policy 1, policy_version 1484076 (0.0008) [2023-12-27 15:56:21,241][20381] Updated weights for policy 1, policy_version 1484086 (0.0008) [2023-12-27 15:56:21,297][20381] Updated weights for policy 1, policy_version 1484096 (0.0008) [2023-12-27 15:56:21,467][20347] Updated weights for policy 0, policy_version 1485535 (0.0007) [2023-12-27 15:56:21,527][20347] Updated weights for policy 0, policy_version 1485545 (0.0007) [2023-12-27 15:56:21,587][20347] Updated weights for policy 0, policy_version 1485555 (0.0008) [2023-12-27 15:56:22,045][20381] Updated weights for policy 1, policy_version 1484106 (0.0007) [2023-12-27 15:56:22,106][20381] Updated weights for policy 1, policy_version 1484116 (0.0008) [2023-12-27 15:56:22,166][20381] Updated weights for policy 1, policy_version 1484126 (0.0008) [2023-12-27 15:56:22,278][20347] Updated weights for policy 0, policy_version 1485565 (0.0008) [2023-12-27 15:56:22,342][20347] Updated weights for policy 0, policy_version 1485575 (0.0008) [2023-12-27 15:56:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 760348672. Throughput: 0: 10266.0, 1: 10368.9. Samples: 760321408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:22,363][19170] Avg episode reward: [(0, '980.580'), (1, '961.000')] [2023-12-27 15:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001484128_379969536.pth... [2023-12-27 15:56:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001482912_379658240.pth [2023-12-27 15:56:22,405][20347] Updated weights for policy 0, policy_version 1485585 (0.0008) [2023-12-27 15:56:22,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001485592_380387328.pth... [2023-12-27 15:56:22,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001484376_380076032.pth [2023-12-27 15:56:22,807][20381] Updated weights for policy 1, policy_version 1484136 (0.0007) [2023-12-27 15:56:22,865][20381] Updated weights for policy 1, policy_version 1484146 (0.0007) [2023-12-27 15:56:22,921][20381] Updated weights for policy 1, policy_version 1484156 (0.0007) [2023-12-27 15:56:23,120][20347] Updated weights for policy 0, policy_version 1485595 (0.0008) [2023-12-27 15:56:23,170][20347] Updated weights for policy 0, policy_version 1485605 (0.0008) [2023-12-27 15:56:23,217][20347] Updated weights for policy 0, policy_version 1485615 (0.0005) [2023-12-27 15:56:23,591][20381] Updated weights for policy 1, policy_version 1484166 (0.0006) [2023-12-27 15:56:23,636][20381] Updated weights for policy 1, policy_version 1484176 (0.0005) [2023-12-27 15:56:23,684][20381] Updated weights for policy 1, policy_version 1484186 (0.0005) [2023-12-27 15:56:23,807][20347] Updated weights for policy 0, policy_version 1485625 (0.0006) [2023-12-27 15:56:23,868][20347] Updated weights for policy 0, policy_version 1485635 (0.0010) [2023-12-27 15:56:23,917][20347] Updated weights for policy 0, policy_version 1485645 (0.0010) [2023-12-27 15:56:23,971][20347] Updated weights for policy 0, policy_version 1485655 (0.0010) [2023-12-27 15:56:24,253][20381] Updated weights for policy 1, policy_version 1484196 (0.0006) [2023-12-27 15:56:24,315][20381] Updated weights for policy 1, policy_version 1484206 (0.0008) [2023-12-27 15:56:24,368][20381] Updated weights for policy 1, policy_version 1484216 (0.0009) [2023-12-27 15:56:24,619][20347] Updated weights for policy 0, policy_version 1485665 (0.0006) [2023-12-27 15:56:24,673][20347] Updated weights for policy 0, policy_version 1485675 (0.0005) [2023-12-27 15:56:24,731][20347] Updated weights for policy 0, policy_version 1485685 (0.0005) [2023-12-27 15:56:25,088][20381] Updated weights for policy 1, policy_version 1484226 (0.0009) [2023-12-27 15:56:25,144][20381] Updated weights for policy 1, policy_version 1484236 (0.0007) [2023-12-27 15:56:25,200][20381] Updated weights for policy 1, policy_version 1484246 (0.0006) [2023-12-27 15:56:25,257][20381] Updated weights for policy 1, policy_version 1484256 (0.0005) [2023-12-27 15:56:25,291][20347] Updated weights for policy 0, policy_version 1485695 (0.0007) [2023-12-27 15:56:25,341][20347] Updated weights for policy 0, policy_version 1485705 (0.0010) [2023-12-27 15:56:25,390][20347] Updated weights for policy 0, policy_version 1485715 (0.0010) [2023-12-27 15:56:25,892][20381] Updated weights for policy 1, policy_version 1484266 (0.0005) [2023-12-27 15:56:25,938][20381] Updated weights for policy 1, policy_version 1484276 (0.0005) [2023-12-27 15:56:25,970][20347] Updated weights for policy 0, policy_version 1485725 (0.0010) [2023-12-27 15:56:25,993][20381] Updated weights for policy 1, policy_version 1484286 (0.0005) [2023-12-27 15:56:26,022][20347] Updated weights for policy 0, policy_version 1485735 (0.0010) [2023-12-27 15:56:26,080][20347] Updated weights for policy 0, policy_version 1485745 (0.0010) [2023-12-27 15:56:26,650][20381] Updated weights for policy 1, policy_version 1484296 (0.0007) [2023-12-27 15:56:26,687][20347] Updated weights for policy 0, policy_version 1485755 (0.0006) [2023-12-27 15:56:26,710][20381] Updated weights for policy 1, policy_version 1484306 (0.0006) [2023-12-27 15:56:26,745][20347] Updated weights for policy 0, policy_version 1485765 (0.0005) [2023-12-27 15:56:26,767][20381] Updated weights for policy 1, policy_version 1484316 (0.0006) [2023-12-27 15:56:26,798][20347] Updated weights for policy 0, policy_version 1485775 (0.0008) [2023-12-27 15:56:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 760455168. Throughput: 0: 10297.1, 1: 10345.0. Samples: 760447008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '961.080')] [2023-12-27 15:56:27,401][20381] Updated weights for policy 1, policy_version 1484326 (0.0006) [2023-12-27 15:56:27,443][20347] Updated weights for policy 0, policy_version 1485785 (0.0010) [2023-12-27 15:56:27,451][20381] Updated weights for policy 1, policy_version 1484336 (0.0005) [2023-12-27 15:56:27,497][20347] Updated weights for policy 0, policy_version 1485795 (0.0010) [2023-12-27 15:56:27,506][20381] Updated weights for policy 1, policy_version 1484346 (0.0005) [2023-12-27 15:56:27,546][20347] Updated weights for policy 0, policy_version 1485805 (0.0010) [2023-12-27 15:56:27,597][20347] Updated weights for policy 0, policy_version 1485815 (0.0010) [2023-12-27 15:56:28,061][20381] Updated weights for policy 1, policy_version 1484356 (0.0006) [2023-12-27 15:56:28,138][20381] Updated weights for policy 1, policy_version 1484366 (0.0005) [2023-12-27 15:56:28,186][20381] Updated weights for policy 1, policy_version 1484376 (0.0005) [2023-12-27 15:56:28,283][20347] Updated weights for policy 0, policy_version 1485825 (0.0010) [2023-12-27 15:56:28,336][20347] Updated weights for policy 0, policy_version 1485835 (0.0010) [2023-12-27 15:56:28,400][20347] Updated weights for policy 0, policy_version 1485845 (0.0011) [2023-12-27 15:56:28,754][20381] Updated weights for policy 1, policy_version 1484386 (0.0005) [2023-12-27 15:56:28,809][20381] Updated weights for policy 1, policy_version 1484396 (0.0005) [2023-12-27 15:56:28,868][20381] Updated weights for policy 1, policy_version 1484406 (0.0006) [2023-12-27 15:56:28,925][20381] Updated weights for policy 1, policy_version 1484416 (0.0005) [2023-12-27 15:56:29,176][20347] Updated weights for policy 0, policy_version 1485855 (0.0008) [2023-12-27 15:56:29,226][20347] Updated weights for policy 0, policy_version 1485865 (0.0007) [2023-12-27 15:56:29,286][20347] Updated weights for policy 0, policy_version 1485875 (0.0008) [2023-12-27 15:56:29,518][20381] Updated weights for policy 1, policy_version 1484426 (0.0005) [2023-12-27 15:56:29,575][20381] Updated weights for policy 1, policy_version 1484436 (0.0005) [2023-12-27 15:56:29,629][20381] Updated weights for policy 1, policy_version 1484446 (0.0005) [2023-12-27 15:56:29,986][20347] Updated weights for policy 0, policy_version 1485885 (0.0007) [2023-12-27 15:56:30,045][20347] Updated weights for policy 0, policy_version 1485895 (0.0008) [2023-12-27 15:56:30,103][20347] Updated weights for policy 0, policy_version 1485905 (0.0008) [2023-12-27 15:56:30,238][20381] Updated weights for policy 1, policy_version 1484456 (0.0007) [2023-12-27 15:56:30,298][20381] Updated weights for policy 1, policy_version 1484466 (0.0006) [2023-12-27 15:56:30,355][20381] Updated weights for policy 1, policy_version 1484476 (0.0006) [2023-12-27 15:56:30,805][20347] Updated weights for policy 0, policy_version 1485915 (0.0007) [2023-12-27 15:56:30,862][20347] Updated weights for policy 0, policy_version 1485925 (0.0006) [2023-12-27 15:56:30,918][20347] Updated weights for policy 0, policy_version 1485935 (0.0010) [2023-12-27 15:56:30,931][20381] Updated weights for policy 1, policy_version 1484486 (0.0006) [2023-12-27 15:56:30,980][20381] Updated weights for policy 1, policy_version 1484496 (0.0006) [2023-12-27 15:56:31,030][20381] Updated weights for policy 1, policy_version 1484506 (0.0008) [2023-12-27 15:56:31,615][20347] Updated weights for policy 0, policy_version 1485945 (0.0010) [2023-12-27 15:56:31,679][20347] Updated weights for policy 0, policy_version 1485955 (0.0008) [2023-12-27 15:56:31,747][20347] Updated weights for policy 0, policy_version 1485965 (0.0006) [2023-12-27 15:56:31,789][20381] Updated weights for policy 1, policy_version 1484516 (0.0008) [2023-12-27 15:56:31,802][20347] Updated weights for policy 0, policy_version 1485975 (0.0006) [2023-12-27 15:56:31,854][20381] Updated weights for policy 1, policy_version 1484526 (0.0007) [2023-12-27 15:56:31,913][20381] Updated weights for policy 1, policy_version 1484536 (0.0006) [2023-12-27 15:56:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 760561664. Throughput: 0: 10198.9, 1: 10483.4. Samples: 760573720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:32,363][19170] Avg episode reward: [(0, '990.290'), (1, '970.810')] [2023-12-27 15:56:32,498][20347] Updated weights for policy 0, policy_version 1485985 (0.0007) [2023-12-27 15:56:32,550][20347] Updated weights for policy 0, policy_version 1485995 (0.0007) [2023-12-27 15:56:32,578][20381] Updated weights for policy 1, policy_version 1484546 (0.0007) [2023-12-27 15:56:32,610][20347] Updated weights for policy 0, policy_version 1486005 (0.0007) [2023-12-27 15:56:32,637][20381] Updated weights for policy 1, policy_version 1484556 (0.0007) [2023-12-27 15:56:32,694][20381] Updated weights for policy 1, policy_version 1484566 (0.0007) [2023-12-27 15:56:32,750][20381] Updated weights for policy 1, policy_version 1484576 (0.0007) [2023-12-27 15:56:33,240][20347] Updated weights for policy 0, policy_version 1486015 (0.0006) [2023-12-27 15:56:33,291][20347] Updated weights for policy 0, policy_version 1486025 (0.0006) [2023-12-27 15:56:33,342][20347] Updated weights for policy 0, policy_version 1486035 (0.0007) [2023-12-27 15:56:33,382][20381] Updated weights for policy 1, policy_version 1484586 (0.0008) [2023-12-27 15:56:33,427][20381] Updated weights for policy 1, policy_version 1484596 (0.0008) [2023-12-27 15:56:33,478][20381] Updated weights for policy 1, policy_version 1484606 (0.0009) [2023-12-27 15:56:33,941][20347] Updated weights for policy 0, policy_version 1486045 (0.0007) [2023-12-27 15:56:33,994][20347] Updated weights for policy 0, policy_version 1486055 (0.0008) [2023-12-27 15:56:34,051][20347] Updated weights for policy 0, policy_version 1486065 (0.0008) [2023-12-27 15:56:34,174][20381] Updated weights for policy 1, policy_version 1484616 (0.0009) [2023-12-27 15:56:34,230][20381] Updated weights for policy 1, policy_version 1484626 (0.0008) [2023-12-27 15:56:34,286][20381] Updated weights for policy 1, policy_version 1484636 (0.0006) [2023-12-27 15:56:34,758][20347] Updated weights for policy 0, policy_version 1486075 (0.0008) [2023-12-27 15:56:34,820][20347] Updated weights for policy 0, policy_version 1486085 (0.0005) [2023-12-27 15:56:34,880][20347] Updated weights for policy 0, policy_version 1486095 (0.0005) [2023-12-27 15:56:35,018][20381] Updated weights for policy 1, policy_version 1484646 (0.0007) [2023-12-27 15:56:35,077][20381] Updated weights for policy 1, policy_version 1484656 (0.0008) [2023-12-27 15:56:35,129][20381] Updated weights for policy 1, policy_version 1484666 (0.0008) [2023-12-27 15:56:35,428][20347] Updated weights for policy 0, policy_version 1486105 (0.0006) [2023-12-27 15:56:35,474][20347] Updated weights for policy 0, policy_version 1486115 (0.0005) [2023-12-27 15:56:35,519][20347] Updated weights for policy 0, policy_version 1486125 (0.0005) [2023-12-27 15:56:35,568][20347] Updated weights for policy 0, policy_version 1486135 (0.0006) [2023-12-27 15:56:35,854][20381] Updated weights for policy 1, policy_version 1484676 (0.0008) [2023-12-27 15:56:35,910][20381] Updated weights for policy 1, policy_version 1484686 (0.0007) [2023-12-27 15:56:35,969][20381] Updated weights for policy 1, policy_version 1484696 (0.0008) [2023-12-27 15:56:35,982][20292] KL-divergence is very high: 123.2829 [2023-12-27 15:56:36,228][20347] Updated weights for policy 0, policy_version 1486145 (0.0005) [2023-12-27 15:56:36,289][20347] Updated weights for policy 0, policy_version 1486155 (0.0006) [2023-12-27 15:56:36,348][20347] Updated weights for policy 0, policy_version 1486165 (0.0007) [2023-12-27 15:56:36,706][20381] Updated weights for policy 1, policy_version 1484706 (0.0007) [2023-12-27 15:56:36,756][20381] Updated weights for policy 1, policy_version 1484716 (0.0006) [2023-12-27 15:56:36,804][20381] Updated weights for policy 1, policy_version 1484726 (0.0007) [2023-12-27 15:56:36,857][20381] Updated weights for policy 1, policy_version 1484736 (0.0011) [2023-12-27 15:56:36,995][20347] Updated weights for policy 0, policy_version 1486175 (0.0008) [2023-12-27 15:56:37,048][20347] Updated weights for policy 0, policy_version 1486185 (0.0008) [2023-12-27 15:56:37,104][20347] Updated weights for policy 0, policy_version 1486195 (0.0009) [2023-12-27 15:56:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 760668160. Throughput: 0: 10251.3, 1: 10475.7. Samples: 760636052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:37,363][19170] Avg episode reward: [(0, '942.360'), (1, '951.280')] [2023-12-27 15:56:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001486200_380542976.pth... [2023-12-27 15:56:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001485016_380239872.pth [2023-12-27 15:56:37,419][20381] Updated weights for policy 1, policy_version 1484746 (0.0010) [2023-12-27 15:56:37,472][20381] Updated weights for policy 1, policy_version 1484756 (0.0007) [2023-12-27 15:56:37,528][20381] Updated weights for policy 1, policy_version 1484766 (0.0006) [2023-12-27 15:56:37,536][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001484768_380133376.pth... [2023-12-27 15:56:37,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001483520_379813888.pth [2023-12-27 15:56:37,849][20347] Updated weights for policy 0, policy_version 1486205 (0.0009) [2023-12-27 15:56:37,903][20347] Updated weights for policy 0, policy_version 1486215 (0.0006) [2023-12-27 15:56:37,956][20347] Updated weights for policy 0, policy_version 1486225 (0.0005) [2023-12-27 15:56:38,182][20381] Updated weights for policy 1, policy_version 1484776 (0.0011) [2023-12-27 15:56:38,240][20381] Updated weights for policy 1, policy_version 1484786 (0.0010) [2023-12-27 15:56:38,296][20381] Updated weights for policy 1, policy_version 1484796 (0.0006) [2023-12-27 15:56:38,623][20347] Updated weights for policy 0, policy_version 1486235 (0.0006) [2023-12-27 15:56:38,677][20347] Updated weights for policy 0, policy_version 1486245 (0.0007) [2023-12-27 15:56:38,729][20347] Updated weights for policy 0, policy_version 1486255 (0.0008) [2023-12-27 15:56:38,897][20381] Updated weights for policy 1, policy_version 1484806 (0.0006) [2023-12-27 15:56:38,955][20381] Updated weights for policy 1, policy_version 1484816 (0.0008) [2023-12-27 15:56:39,009][20381] Updated weights for policy 1, policy_version 1484826 (0.0010) [2023-12-27 15:56:39,466][20347] Updated weights for policy 0, policy_version 1486265 (0.0008) [2023-12-27 15:56:39,530][20347] Updated weights for policy 0, policy_version 1486275 (0.0005) [2023-12-27 15:56:39,596][20347] Updated weights for policy 0, policy_version 1486285 (0.0007) [2023-12-27 15:56:39,661][20347] Updated weights for policy 0, policy_version 1486295 (0.0009) [2023-12-27 15:56:39,736][20381] Updated weights for policy 1, policy_version 1484836 (0.0011) [2023-12-27 15:56:39,799][20381] Updated weights for policy 1, policy_version 1484846 (0.0010) [2023-12-27 15:56:39,860][20381] Updated weights for policy 1, policy_version 1484856 (0.0010) [2023-12-27 15:56:40,326][20347] Updated weights for policy 0, policy_version 1486305 (0.0010) [2023-12-27 15:56:40,389][20347] Updated weights for policy 0, policy_version 1486315 (0.0011) [2023-12-27 15:56:40,443][20347] Updated weights for policy 0, policy_version 1486325 (0.0011) [2023-12-27 15:56:40,565][20381] Updated weights for policy 1, policy_version 1484866 (0.0010) [2023-12-27 15:56:40,624][20381] Updated weights for policy 1, policy_version 1484876 (0.0007) [2023-12-27 15:56:40,684][20381] Updated weights for policy 1, policy_version 1484886 (0.0010) [2023-12-27 15:56:40,740][20381] Updated weights for policy 1, policy_version 1484896 (0.0011) [2023-12-27 15:56:41,088][20347] Updated weights for policy 0, policy_version 1486335 (0.0010) [2023-12-27 15:56:41,153][20347] Updated weights for policy 0, policy_version 1486345 (0.0009) [2023-12-27 15:56:41,229][20347] Updated weights for policy 0, policy_version 1486355 (0.0008) [2023-12-27 15:56:41,430][20381] Updated weights for policy 1, policy_version 1484906 (0.0009) [2023-12-27 15:56:41,487][20381] Updated weights for policy 1, policy_version 1484916 (0.0008) [2023-12-27 15:56:41,543][20381] Updated weights for policy 1, policy_version 1484926 (0.0007) [2023-12-27 15:56:41,948][20347] Updated weights for policy 0, policy_version 1486365 (0.0009) [2023-12-27 15:56:42,007][20347] Updated weights for policy 0, policy_version 1486375 (0.0007) [2023-12-27 15:56:42,067][20347] Updated weights for policy 0, policy_version 1486385 (0.0008) [2023-12-27 15:56:42,250][20381] Updated weights for policy 1, policy_version 1484936 (0.0006) [2023-12-27 15:56:42,318][20381] Updated weights for policy 1, policy_version 1484946 (0.0006) [2023-12-27 15:56:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 760766464. Throughput: 0: 10236.3, 1: 10483.7. Samples: 760760492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:42,363][19170] Avg episode reward: [(0, '952.020'), (1, '970.770')] [2023-12-27 15:56:42,384][20381] Updated weights for policy 1, policy_version 1484956 (0.0008) [2023-12-27 15:56:42,792][20347] Updated weights for policy 0, policy_version 1486395 (0.0008) [2023-12-27 15:56:42,849][20347] Updated weights for policy 0, policy_version 1486405 (0.0007) [2023-12-27 15:56:42,912][20347] Updated weights for policy 0, policy_version 1486415 (0.0008) [2023-12-27 15:56:43,029][20381] Updated weights for policy 1, policy_version 1484966 (0.0007) [2023-12-27 15:56:43,074][20381] Updated weights for policy 1, policy_version 1484976 (0.0009) [2023-12-27 15:56:43,133][20381] Updated weights for policy 1, policy_version 1484986 (0.0008) [2023-12-27 15:56:43,542][20347] Updated weights for policy 0, policy_version 1486425 (0.0008) [2023-12-27 15:56:43,606][20347] Updated weights for policy 0, policy_version 1486435 (0.0010) [2023-12-27 15:56:43,659][20347] Updated weights for policy 0, policy_version 1486445 (0.0010) [2023-12-27 15:56:43,713][20347] Updated weights for policy 0, policy_version 1486455 (0.0010) [2023-12-27 15:56:43,830][20381] Updated weights for policy 1, policy_version 1484996 (0.0009) [2023-12-27 15:56:43,884][20381] Updated weights for policy 1, policy_version 1485006 (0.0010) [2023-12-27 15:56:43,938][20381] Updated weights for policy 1, policy_version 1485016 (0.0010) [2023-12-27 15:56:44,380][20347] Updated weights for policy 0, policy_version 1486465 (0.0010) [2023-12-27 15:56:44,432][20347] Updated weights for policy 0, policy_version 1486475 (0.0010) [2023-12-27 15:56:44,483][20347] Updated weights for policy 0, policy_version 1486485 (0.0010) [2023-12-27 15:56:44,605][20381] Updated weights for policy 1, policy_version 1485026 (0.0010) [2023-12-27 15:56:44,665][20381] Updated weights for policy 1, policy_version 1485036 (0.0010) [2023-12-27 15:56:44,725][20381] Updated weights for policy 1, policy_version 1485046 (0.0010) [2023-12-27 15:56:44,784][20381] Updated weights for policy 1, policy_version 1485056 (0.0010) [2023-12-27 15:56:45,135][20347] Updated weights for policy 0, policy_version 1486495 (0.0007) [2023-12-27 15:56:45,204][20347] Updated weights for policy 0, policy_version 1486505 (0.0007) [2023-12-27 15:56:45,267][20347] Updated weights for policy 0, policy_version 1486515 (0.0008) [2023-12-27 15:56:45,425][20381] Updated weights for policy 1, policy_version 1485066 (0.0011) [2023-12-27 15:56:45,481][20381] Updated weights for policy 1, policy_version 1485076 (0.0011) [2023-12-27 15:56:45,536][20381] Updated weights for policy 1, policy_version 1485086 (0.0011) [2023-12-27 15:56:45,937][20347] Updated weights for policy 0, policy_version 1486525 (0.0007) [2023-12-27 15:56:45,997][20347] Updated weights for policy 0, policy_version 1486535 (0.0006) [2023-12-27 15:56:46,060][20347] Updated weights for policy 0, policy_version 1486545 (0.0007) [2023-12-27 15:56:46,218][20381] Updated weights for policy 1, policy_version 1485096 (0.0011) [2023-12-27 15:56:46,275][20381] Updated weights for policy 1, policy_version 1485106 (0.0010) [2023-12-27 15:56:46,340][20381] Updated weights for policy 1, policy_version 1485116 (0.0011) [2023-12-27 15:56:46,741][20347] Updated weights for policy 0, policy_version 1486555 (0.0008) [2023-12-27 15:56:46,786][20347] Updated weights for policy 0, policy_version 1486565 (0.0007) [2023-12-27 15:56:46,840][20347] Updated weights for policy 0, policy_version 1486575 (0.0007) [2023-12-27 15:56:46,977][20381] Updated weights for policy 1, policy_version 1485126 (0.0008) [2023-12-27 15:56:47,031][20381] Updated weights for policy 1, policy_version 1485136 (0.0007) [2023-12-27 15:56:47,086][20381] Updated weights for policy 1, policy_version 1485146 (0.0008) [2023-12-27 15:56:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 760872960. Throughput: 0: 10211.9, 1: 10427.2. Samples: 760882320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:47,364][19170] Avg episode reward: [(0, '951.270'), (1, '980.500')] [2023-12-27 15:56:47,549][20347] Updated weights for policy 0, policy_version 1486585 (0.0008) [2023-12-27 15:56:47,605][20347] Updated weights for policy 0, policy_version 1486595 (0.0008) [2023-12-27 15:56:47,664][20347] Updated weights for policy 0, policy_version 1486605 (0.0008) [2023-12-27 15:56:47,721][20381] Updated weights for policy 1, policy_version 1485156 (0.0007) [2023-12-27 15:56:47,722][20347] Updated weights for policy 0, policy_version 1486615 (0.0007) [2023-12-27 15:56:47,777][20381] Updated weights for policy 1, policy_version 1485166 (0.0005) [2023-12-27 15:56:47,837][20381] Updated weights for policy 1, policy_version 1485176 (0.0005) [2023-12-27 15:56:48,385][20381] Updated weights for policy 1, policy_version 1485186 (0.0005) [2023-12-27 15:56:48,446][20381] Updated weights for policy 1, policy_version 1485196 (0.0006) [2023-12-27 15:56:48,474][20347] Updated weights for policy 0, policy_version 1486625 (0.0007) [2023-12-27 15:56:48,506][20381] Updated weights for policy 1, policy_version 1485206 (0.0005) [2023-12-27 15:56:48,538][20347] Updated weights for policy 0, policy_version 1486635 (0.0006) [2023-12-27 15:56:48,568][20381] Updated weights for policy 1, policy_version 1485216 (0.0006) [2023-12-27 15:56:48,597][20347] Updated weights for policy 0, policy_version 1486645 (0.0006) [2023-12-27 15:56:49,150][20381] Updated weights for policy 1, policy_version 1485226 (0.0010) [2023-12-27 15:56:49,183][20347] Updated weights for policy 0, policy_version 1486655 (0.0005) [2023-12-27 15:56:49,196][20381] Updated weights for policy 1, policy_version 1485236 (0.0007) [2023-12-27 15:56:49,250][20347] Updated weights for policy 0, policy_version 1486665 (0.0007) [2023-12-27 15:56:49,259][20381] Updated weights for policy 1, policy_version 1485246 (0.0008) [2023-12-27 15:56:49,308][20347] Updated weights for policy 0, policy_version 1486675 (0.0007) [2023-12-27 15:56:49,933][20347] Updated weights for policy 0, policy_version 1486685 (0.0007) [2023-12-27 15:56:50,000][20347] Updated weights for policy 0, policy_version 1486695 (0.0008) [2023-12-27 15:56:50,010][20381] Updated weights for policy 1, policy_version 1485256 (0.0008) [2023-12-27 15:56:50,061][20381] Updated weights for policy 1, policy_version 1485266 (0.0007) [2023-12-27 15:56:50,062][20347] Updated weights for policy 0, policy_version 1486705 (0.0006) [2023-12-27 15:56:50,126][20381] Updated weights for policy 1, policy_version 1485276 (0.0008) [2023-12-27 15:56:50,709][20347] Updated weights for policy 0, policy_version 1486715 (0.0006) [2023-12-27 15:56:50,761][20347] Updated weights for policy 0, policy_version 1486725 (0.0006) [2023-12-27 15:56:50,790][20381] Updated weights for policy 1, policy_version 1485286 (0.0006) [2023-12-27 15:56:50,808][20347] Updated weights for policy 0, policy_version 1486735 (0.0006) [2023-12-27 15:56:50,844][20381] Updated weights for policy 1, policy_version 1485296 (0.0007) [2023-12-27 15:56:50,893][20381] Updated weights for policy 1, policy_version 1485306 (0.0008) [2023-12-27 15:56:51,525][20347] Updated weights for policy 0, policy_version 1486745 (0.0007) [2023-12-27 15:56:51,561][20381] Updated weights for policy 1, policy_version 1485316 (0.0007) [2023-12-27 15:56:51,594][20347] Updated weights for policy 0, policy_version 1486755 (0.0008) [2023-12-27 15:56:51,626][20381] Updated weights for policy 1, policy_version 1485326 (0.0007) [2023-12-27 15:56:51,654][20347] Updated weights for policy 0, policy_version 1486765 (0.0008) [2023-12-27 15:56:51,695][20381] Updated weights for policy 1, policy_version 1485336 (0.0008) [2023-12-27 15:56:51,720][20347] Updated weights for policy 0, policy_version 1486775 (0.0008) [2023-12-27 15:56:52,331][20381] Updated weights for policy 1, policy_version 1485346 (0.0007) [2023-12-27 15:56:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 760971264. Throughput: 0: 10198.9, 1: 10467.7. Samples: 760945260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:52,363][19170] Avg episode reward: [(0, '931.710'), (1, '980.500')] [2023-12-27 15:56:52,398][20381] Updated weights for policy 1, policy_version 1485356 (0.0009) [2023-12-27 15:56:52,404][20347] Updated weights for policy 0, policy_version 1486785 (0.0008) [2023-12-27 15:56:52,457][20381] Updated weights for policy 1, policy_version 1485366 (0.0008) [2023-12-27 15:56:52,466][20347] Updated weights for policy 0, policy_version 1486795 (0.0008) [2023-12-27 15:56:52,515][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001485376_380289024.pth... [2023-12-27 15:56:52,518][20381] Updated weights for policy 1, policy_version 1485376 (0.0006) [2023-12-27 15:56:52,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001484128_379969536.pth [2023-12-27 15:56:52,524][20347] Updated weights for policy 0, policy_version 1486805 (0.0006) [2023-12-27 15:56:52,540][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001486808_380698624.pth... [2023-12-27 15:56:52,544][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001485592_380387328.pth [2023-12-27 15:56:53,136][20381] Updated weights for policy 1, policy_version 1485386 (0.0005) [2023-12-27 15:56:53,183][20381] Updated weights for policy 1, policy_version 1485396 (0.0005) [2023-12-27 15:56:53,225][20347] Updated weights for policy 0, policy_version 1486815 (0.0006) [2023-12-27 15:56:53,240][20381] Updated weights for policy 1, policy_version 1485406 (0.0006) [2023-12-27 15:56:53,276][20347] Updated weights for policy 0, policy_version 1486825 (0.0007) [2023-12-27 15:56:53,332][20347] Updated weights for policy 0, policy_version 1486835 (0.0008) [2023-12-27 15:56:53,889][20381] Updated weights for policy 1, policy_version 1485416 (0.0006) [2023-12-27 15:56:53,943][20381] Updated weights for policy 1, policy_version 1485426 (0.0007) [2023-12-27 15:56:53,999][20381] Updated weights for policy 1, policy_version 1485436 (0.0007) [2023-12-27 15:56:54,038][20347] Updated weights for policy 0, policy_version 1486845 (0.0010) [2023-12-27 15:56:54,092][20347] Updated weights for policy 0, policy_version 1486855 (0.0010) [2023-12-27 15:56:54,153][20347] Updated weights for policy 0, policy_version 1486865 (0.0010) [2023-12-27 15:56:54,650][20381] Updated weights for policy 1, policy_version 1485446 (0.0009) [2023-12-27 15:56:54,701][20381] Updated weights for policy 1, policy_version 1485456 (0.0010) [2023-12-27 15:56:54,755][20381] Updated weights for policy 1, policy_version 1485466 (0.0010) [2023-12-27 15:56:54,817][20347] Updated weights for policy 0, policy_version 1486875 (0.0009) [2023-12-27 15:56:54,880][20347] Updated weights for policy 0, policy_version 1486885 (0.0008) [2023-12-27 15:56:54,937][20347] Updated weights for policy 0, policy_version 1486895 (0.0009) [2023-12-27 15:56:55,468][20381] Updated weights for policy 1, policy_version 1485476 (0.0009) [2023-12-27 15:56:55,524][20381] Updated weights for policy 1, policy_version 1485486 (0.0010) [2023-12-27 15:56:55,577][20381] Updated weights for policy 1, policy_version 1485496 (0.0006) [2023-12-27 15:56:55,591][20347] Updated weights for policy 0, policy_version 1486905 (0.0011) [2023-12-27 15:56:55,645][20347] Updated weights for policy 0, policy_version 1486915 (0.0008) [2023-12-27 15:56:55,690][20347] Updated weights for policy 0, policy_version 1486925 (0.0005) [2023-12-27 15:56:55,733][20347] Updated weights for policy 0, policy_version 1486935 (0.0005) [2023-12-27 15:56:56,191][20381] Updated weights for policy 1, policy_version 1485506 (0.0006) [2023-12-27 15:56:56,246][20381] Updated weights for policy 1, policy_version 1485516 (0.0005) [2023-12-27 15:56:56,309][20381] Updated weights for policy 1, policy_version 1485526 (0.0007) [2023-12-27 15:56:56,333][20347] Updated weights for policy 0, policy_version 1486945 (0.0010) [2023-12-27 15:56:56,360][20381] Updated weights for policy 1, policy_version 1485536 (0.0010) [2023-12-27 15:56:56,377][20347] Updated weights for policy 0, policy_version 1486955 (0.0010) [2023-12-27 15:56:56,379][20291] KL-divergence is very high: 110.0661 [2023-12-27 15:56:56,429][20347] Updated weights for policy 0, policy_version 1486965 (0.0010) [2023-12-27 15:56:56,964][20381] Updated weights for policy 1, policy_version 1485546 (0.0009) [2023-12-27 15:56:57,019][20381] Updated weights for policy 1, policy_version 1485556 (0.0007) [2023-12-27 15:56:57,033][20347] Updated weights for policy 0, policy_version 1486975 (0.0007) [2023-12-27 15:56:57,075][20381] Updated weights for policy 1, policy_version 1485566 (0.0007) [2023-12-27 15:56:57,083][20347] Updated weights for policy 0, policy_version 1486985 (0.0005) [2023-12-27 15:56:57,138][20347] Updated weights for policy 0, policy_version 1486995 (0.0005) [2023-12-27 15:56:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 761085952. Throughput: 0: 10192.9, 1: 10539.4. Samples: 761070268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:56:57,363][19170] Avg episode reward: [(0, '951.220'), (1, '980.520')] [2023-12-27 15:56:57,623][20347] Updated weights for policy 0, policy_version 1487005 (0.0006) [2023-12-27 15:56:57,685][20347] Updated weights for policy 0, policy_version 1487015 (0.0008) [2023-12-27 15:56:57,715][20381] Updated weights for policy 1, policy_version 1485576 (0.0005) [2023-12-27 15:56:57,746][20347] Updated weights for policy 0, policy_version 1487025 (0.0008) [2023-12-27 15:56:57,774][20381] Updated weights for policy 1, policy_version 1485586 (0.0006) [2023-12-27 15:56:57,829][20381] Updated weights for policy 1, policy_version 1485596 (0.0007) [2023-12-27 15:56:58,457][20347] Updated weights for policy 0, policy_version 1487035 (0.0007) [2023-12-27 15:56:58,518][20347] Updated weights for policy 0, policy_version 1487045 (0.0007) [2023-12-27 15:56:58,527][20381] Updated weights for policy 1, policy_version 1485606 (0.0007) [2023-12-27 15:56:58,574][20347] Updated weights for policy 0, policy_version 1487055 (0.0006) [2023-12-27 15:56:58,585][20381] Updated weights for policy 1, policy_version 1485616 (0.0008) [2023-12-27 15:56:58,648][20381] Updated weights for policy 1, policy_version 1485626 (0.0006) [2023-12-27 15:56:59,303][20347] Updated weights for policy 0, policy_version 1487065 (0.0008) [2023-12-27 15:56:59,368][20347] Updated weights for policy 0, policy_version 1487075 (0.0010) [2023-12-27 15:56:59,429][20347] Updated weights for policy 0, policy_version 1487085 (0.0006) [2023-12-27 15:56:59,477][20381] Updated weights for policy 1, policy_version 1485636 (0.0007) [2023-12-27 15:56:59,493][20347] Updated weights for policy 0, policy_version 1487095 (0.0009) [2023-12-27 15:56:59,534][20381] Updated weights for policy 1, policy_version 1485646 (0.0007) [2023-12-27 15:56:59,593][20381] Updated weights for policy 1, policy_version 1485656 (0.0007) [2023-12-27 15:57:00,156][20347] Updated weights for policy 0, policy_version 1487105 (0.0010) [2023-12-27 15:57:00,209][20347] Updated weights for policy 0, policy_version 1487115 (0.0006) [2023-12-27 15:57:00,268][20347] Updated weights for policy 0, policy_version 1487125 (0.0006) [2023-12-27 15:57:00,339][20381] Updated weights for policy 1, policy_version 1485666 (0.0007) [2023-12-27 15:57:00,395][20381] Updated weights for policy 1, policy_version 1485676 (0.0008) [2023-12-27 15:57:00,444][20381] Updated weights for policy 1, policy_version 1485686 (0.0008) [2023-12-27 15:57:00,490][20381] Updated weights for policy 1, policy_version 1485696 (0.0008) [2023-12-27 15:57:00,903][20347] Updated weights for policy 0, policy_version 1487135 (0.0008) [2023-12-27 15:57:00,958][20347] Updated weights for policy 0, policy_version 1487146 (0.0007) [2023-12-27 15:57:01,014][20347] Updated weights for policy 0, policy_version 1487156 (0.0005) [2023-12-27 15:57:01,212][20381] Updated weights for policy 1, policy_version 1485706 (0.0008) [2023-12-27 15:57:01,271][20381] Updated weights for policy 1, policy_version 1485716 (0.0007) [2023-12-27 15:57:01,323][20381] Updated weights for policy 1, policy_version 1485726 (0.0008) [2023-12-27 15:57:01,684][20347] Updated weights for policy 0, policy_version 1487166 (0.0007) [2023-12-27 15:57:01,750][20347] Updated weights for policy 0, policy_version 1487176 (0.0007) [2023-12-27 15:57:01,805][20347] Updated weights for policy 0, policy_version 1487186 (0.0009) [2023-12-27 15:57:01,983][20381] Updated weights for policy 1, policy_version 1485736 (0.0007) [2023-12-27 15:57:02,046][20381] Updated weights for policy 1, policy_version 1485746 (0.0009) [2023-12-27 15:57:02,108][20381] Updated weights for policy 1, policy_version 1485756 (0.0008) [2023-12-27 15:57:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 761184256. Throughput: 0: 10318.8, 1: 10395.4. Samples: 761194064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:02,363][19170] Avg episode reward: [(0, '980.500'), (1, '970.780')] [2023-12-27 15:57:02,459][20347] Updated weights for policy 0, policy_version 1487196 (0.0010) [2023-12-27 15:57:02,517][20347] Updated weights for policy 0, policy_version 1487206 (0.0007) [2023-12-27 15:57:02,580][20347] Updated weights for policy 0, policy_version 1487216 (0.0008) [2023-12-27 15:57:02,786][20381] Updated weights for policy 1, policy_version 1485766 (0.0007) [2023-12-27 15:57:02,842][20381] Updated weights for policy 1, policy_version 1485776 (0.0006) [2023-12-27 15:57:02,907][20381] Updated weights for policy 1, policy_version 1485786 (0.0009) [2023-12-27 15:57:03,129][20347] Updated weights for policy 0, policy_version 1487226 (0.0007) [2023-12-27 15:57:03,184][20347] Updated weights for policy 0, policy_version 1487236 (0.0005) [2023-12-27 15:57:03,240][20347] Updated weights for policy 0, policy_version 1487246 (0.0005) [2023-12-27 15:57:03,293][20347] Updated weights for policy 0, policy_version 1487256 (0.0005) [2023-12-27 15:57:03,609][20381] Updated weights for policy 1, policy_version 1485796 (0.0008) [2023-12-27 15:57:03,671][20381] Updated weights for policy 1, policy_version 1485806 (0.0007) [2023-12-27 15:57:03,724][20381] Updated weights for policy 1, policy_version 1485816 (0.0008) [2023-12-27 15:57:03,787][20347] Updated weights for policy 0, policy_version 1487266 (0.0005) [2023-12-27 15:57:03,844][20347] Updated weights for policy 0, policy_version 1487276 (0.0007) [2023-12-27 15:57:03,904][20347] Updated weights for policy 0, policy_version 1487286 (0.0007) [2023-12-27 15:57:04,426][20381] Updated weights for policy 1, policy_version 1485826 (0.0007) [2023-12-27 15:57:04,488][20381] Updated weights for policy 1, policy_version 1485836 (0.0007) [2023-12-27 15:57:04,540][20347] Updated weights for policy 0, policy_version 1487296 (0.0009) [2023-12-27 15:57:04,549][20381] Updated weights for policy 1, policy_version 1485846 (0.0007) [2023-12-27 15:57:04,592][20347] Updated weights for policy 0, policy_version 1487306 (0.0010) [2023-12-27 15:57:04,602][20381] Updated weights for policy 1, policy_version 1485856 (0.0006) [2023-12-27 15:57:04,657][20347] Updated weights for policy 0, policy_version 1487316 (0.0006) [2023-12-27 15:57:05,245][20347] Updated weights for policy 0, policy_version 1487326 (0.0007) [2023-12-27 15:57:05,300][20347] Updated weights for policy 0, policy_version 1487336 (0.0005) [2023-12-27 15:57:05,309][20381] Updated weights for policy 1, policy_version 1485866 (0.0007) [2023-12-27 15:57:05,352][20347] Updated weights for policy 0, policy_version 1487346 (0.0008) [2023-12-27 15:57:05,363][20381] Updated weights for policy 1, policy_version 1485876 (0.0005) [2023-12-27 15:57:05,421][20381] Updated weights for policy 1, policy_version 1485886 (0.0007) [2023-12-27 15:57:05,932][20347] Updated weights for policy 0, policy_version 1487356 (0.0007) [2023-12-27 15:57:05,982][20347] Updated weights for policy 0, policy_version 1487366 (0.0005) [2023-12-27 15:57:06,033][20347] Updated weights for policy 0, policy_version 1487376 (0.0006) [2023-12-27 15:57:06,106][20381] Updated weights for policy 1, policy_version 1485896 (0.0008) [2023-12-27 15:57:06,173][20381] Updated weights for policy 1, policy_version 1485906 (0.0008) [2023-12-27 15:57:06,235][20381] Updated weights for policy 1, policy_version 1485916 (0.0007) [2023-12-27 15:57:06,642][20347] Updated weights for policy 0, policy_version 1487386 (0.0007) [2023-12-27 15:57:06,703][20347] Updated weights for policy 0, policy_version 1487396 (0.0007) [2023-12-27 15:57:06,769][20347] Updated weights for policy 0, policy_version 1487406 (0.0008) [2023-12-27 15:57:06,828][20347] Updated weights for policy 0, policy_version 1487416 (0.0007) [2023-12-27 15:57:06,975][20381] Updated weights for policy 1, policy_version 1485926 (0.0009) [2023-12-27 15:57:07,039][20381] Updated weights for policy 1, policy_version 1485936 (0.0010) [2023-12-27 15:57:07,102][20381] Updated weights for policy 1, policy_version 1485946 (0.0011) [2023-12-27 15:57:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 761290752. Throughput: 0: 10425.2, 1: 10410.7. Samples: 761259028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:07,363][19170] Avg episode reward: [(0, '970.920'), (1, '970.780')] [2023-12-27 15:57:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001485952_380436480.pth... [2023-12-27 15:57:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001487416_380854272.pth... [2023-12-27 15:57:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001484768_380133376.pth [2023-12-27 15:57:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001486200_380542976.pth [2023-12-27 15:57:07,477][20291] KL-divergence is very high: 143.9882 [2023-12-27 15:57:07,483][20347] Updated weights for policy 0, policy_version 1487426 (0.0008) [2023-12-27 15:57:07,514][20291] KL-divergence is very high: 120.1723 [2023-12-27 15:57:07,532][20347] Updated weights for policy 0, policy_version 1487436 (0.0009) [2023-12-27 15:57:07,578][20347] Updated weights for policy 0, policy_version 1487446 (0.0007) [2023-12-27 15:57:07,747][20381] Updated weights for policy 1, policy_version 1485956 (0.0008) [2023-12-27 15:57:07,803][20381] Updated weights for policy 1, policy_version 1485966 (0.0005) [2023-12-27 15:57:07,853][20381] Updated weights for policy 1, policy_version 1485976 (0.0008) [2023-12-27 15:57:08,233][20347] Updated weights for policy 0, policy_version 1487456 (0.0008) [2023-12-27 15:57:08,286][20347] Updated weights for policy 0, policy_version 1487466 (0.0008) [2023-12-27 15:57:08,359][20347] Updated weights for policy 0, policy_version 1487476 (0.0008) [2023-12-27 15:57:08,561][20381] Updated weights for policy 1, policy_version 1485986 (0.0009) [2023-12-27 15:57:08,621][20381] Updated weights for policy 1, policy_version 1485996 (0.0008) [2023-12-27 15:57:08,676][20381] Updated weights for policy 1, policy_version 1486006 (0.0007) [2023-12-27 15:57:08,737][20381] Updated weights for policy 1, policy_version 1486016 (0.0007) [2023-12-27 15:57:09,000][20347] Updated weights for policy 0, policy_version 1487486 (0.0008) [2023-12-27 15:57:09,050][20347] Updated weights for policy 0, policy_version 1487496 (0.0008) [2023-12-27 15:57:09,108][20347] Updated weights for policy 0, policy_version 1487506 (0.0007) [2023-12-27 15:57:09,407][20381] Updated weights for policy 1, policy_version 1486026 (0.0011) [2023-12-27 15:57:09,464][20381] Updated weights for policy 1, policy_version 1486036 (0.0008) [2023-12-27 15:57:09,518][20381] Updated weights for policy 1, policy_version 1486046 (0.0008) [2023-12-27 15:57:09,885][20347] Updated weights for policy 0, policy_version 1487516 (0.0008) [2023-12-27 15:57:09,947][20347] Updated weights for policy 0, policy_version 1487526 (0.0008) [2023-12-27 15:57:10,010][20347] Updated weights for policy 0, policy_version 1487536 (0.0008) [2023-12-27 15:57:10,189][20381] Updated weights for policy 1, policy_version 1486056 (0.0011) [2023-12-27 15:57:10,250][20381] Updated weights for policy 1, policy_version 1486066 (0.0010) [2023-12-27 15:57:10,308][20381] Updated weights for policy 1, policy_version 1486076 (0.0008) [2023-12-27 15:57:10,753][20347] Updated weights for policy 0, policy_version 1487546 (0.0008) [2023-12-27 15:57:10,799][20347] Updated weights for policy 0, policy_version 1487556 (0.0007) [2023-12-27 15:57:10,852][20347] Updated weights for policy 0, policy_version 1487566 (0.0005) [2023-12-27 15:57:10,911][20347] Updated weights for policy 0, policy_version 1487576 (0.0007) [2023-12-27 15:57:10,924][20381] Updated weights for policy 1, policy_version 1486086 (0.0008) [2023-12-27 15:57:10,978][20381] Updated weights for policy 1, policy_version 1486096 (0.0005) [2023-12-27 15:57:11,042][20381] Updated weights for policy 1, policy_version 1486106 (0.0008) [2023-12-27 15:57:11,707][20347] Updated weights for policy 0, policy_version 1487586 (0.0008) [2023-12-27 15:57:11,775][20347] Updated weights for policy 0, policy_version 1487596 (0.0007) [2023-12-27 15:57:11,805][20381] Updated weights for policy 1, policy_version 1486116 (0.0007) [2023-12-27 15:57:11,830][20347] Updated weights for policy 0, policy_version 1487606 (0.0006) [2023-12-27 15:57:11,861][20381] Updated weights for policy 1, policy_version 1486126 (0.0007) [2023-12-27 15:57:11,918][20381] Updated weights for policy 1, policy_version 1486136 (0.0008) [2023-12-27 15:57:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 761389056. Throughput: 0: 10407.4, 1: 10367.9. Samples: 761381900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:12,363][19170] Avg episode reward: [(0, '961.230'), (1, '980.520')] [2023-12-27 15:57:12,508][20347] Updated weights for policy 0, policy_version 1487616 (0.0007) [2023-12-27 15:57:12,568][20347] Updated weights for policy 0, policy_version 1487626 (0.0008) [2023-12-27 15:57:12,618][20347] Updated weights for policy 0, policy_version 1487636 (0.0008) [2023-12-27 15:57:12,653][20381] Updated weights for policy 1, policy_version 1486146 (0.0007) [2023-12-27 15:57:12,707][20381] Updated weights for policy 1, policy_version 1486156 (0.0008) [2023-12-27 15:57:12,757][20381] Updated weights for policy 1, policy_version 1486166 (0.0008) [2023-12-27 15:57:12,806][20381] Updated weights for policy 1, policy_version 1486176 (0.0009) [2023-12-27 15:57:13,217][20347] Updated weights for policy 0, policy_version 1487646 (0.0008) [2023-12-27 15:57:13,274][20347] Updated weights for policy 0, policy_version 1487656 (0.0008) [2023-12-27 15:57:13,330][20347] Updated weights for policy 0, policy_version 1487666 (0.0007) [2023-12-27 15:57:13,465][20381] Updated weights for policy 1, policy_version 1486186 (0.0008) [2023-12-27 15:57:13,515][20381] Updated weights for policy 1, policy_version 1486196 (0.0007) [2023-12-27 15:57:13,571][20381] Updated weights for policy 1, policy_version 1486206 (0.0008) [2023-12-27 15:57:13,995][20347] Updated weights for policy 0, policy_version 1487676 (0.0007) [2023-12-27 15:57:14,051][20347] Updated weights for policy 0, policy_version 1487686 (0.0007) [2023-12-27 15:57:14,107][20347] Updated weights for policy 0, policy_version 1487696 (0.0009) [2023-12-27 15:57:14,253][20381] Updated weights for policy 1, policy_version 1486216 (0.0008) [2023-12-27 15:57:14,308][20381] Updated weights for policy 1, policy_version 1486226 (0.0005) [2023-12-27 15:57:14,361][20381] Updated weights for policy 1, policy_version 1486236 (0.0006) [2023-12-27 15:57:14,807][20347] Updated weights for policy 0, policy_version 1487706 (0.0007) [2023-12-27 15:57:14,863][20347] Updated weights for policy 0, policy_version 1487716 (0.0007) [2023-12-27 15:57:14,928][20347] Updated weights for policy 0, policy_version 1487726 (0.0008) [2023-12-27 15:57:14,985][20347] Updated weights for policy 0, policy_version 1487736 (0.0007) [2023-12-27 15:57:15,080][20381] Updated weights for policy 1, policy_version 1486246 (0.0008) [2023-12-27 15:57:15,140][20381] Updated weights for policy 1, policy_version 1486256 (0.0007) [2023-12-27 15:57:15,193][20381] Updated weights for policy 1, policy_version 1486266 (0.0008) [2023-12-27 15:57:15,686][20347] Updated weights for policy 0, policy_version 1487746 (0.0007) [2023-12-27 15:57:15,743][20347] Updated weights for policy 0, policy_version 1487756 (0.0005) [2023-12-27 15:57:15,800][20347] Updated weights for policy 0, policy_version 1487766 (0.0006) [2023-12-27 15:57:15,875][20381] Updated weights for policy 1, policy_version 1486276 (0.0008) [2023-12-27 15:57:15,934][20381] Updated weights for policy 1, policy_version 1486286 (0.0008) [2023-12-27 15:57:15,983][20381] Updated weights for policy 1, policy_version 1486296 (0.0007) [2023-12-27 15:57:16,425][20347] Updated weights for policy 0, policy_version 1487776 (0.0008) [2023-12-27 15:57:16,480][20347] Updated weights for policy 0, policy_version 1487786 (0.0007) [2023-12-27 15:57:16,536][20347] Updated weights for policy 0, policy_version 1487796 (0.0008) [2023-12-27 15:57:16,684][20381] Updated weights for policy 1, policy_version 1486306 (0.0008) [2023-12-27 15:57:16,732][20381] Updated weights for policy 1, policy_version 1486316 (0.0007) [2023-12-27 15:57:16,783][20381] Updated weights for policy 1, policy_version 1486326 (0.0008) [2023-12-27 15:57:16,831][20381] Updated weights for policy 1, policy_version 1486336 (0.0008) [2023-12-27 15:57:17,232][20347] Updated weights for policy 0, policy_version 1487806 (0.0008) [2023-12-27 15:57:17,288][20347] Updated weights for policy 0, policy_version 1487816 (0.0008) [2023-12-27 15:57:17,337][20347] Updated weights for policy 0, policy_version 1487826 (0.0007) [2023-12-27 15:57:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 761487360. Throughput: 0: 10424.0, 1: 10232.9. Samples: 761503280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:17,363][19170] Avg episode reward: [(0, '980.630'), (1, '990.260')] [2023-12-27 15:57:17,540][20381] Updated weights for policy 1, policy_version 1486346 (0.0007) [2023-12-27 15:57:17,589][20381] Updated weights for policy 1, policy_version 1486356 (0.0008) [2023-12-27 15:57:17,649][20381] Updated weights for policy 1, policy_version 1486366 (0.0007) [2023-12-27 15:57:18,015][20347] Updated weights for policy 0, policy_version 1487836 (0.0007) [2023-12-27 15:57:18,070][20347] Updated weights for policy 0, policy_version 1487846 (0.0007) [2023-12-27 15:57:18,129][20347] Updated weights for policy 0, policy_version 1487856 (0.0008) [2023-12-27 15:57:18,357][20381] Updated weights for policy 1, policy_version 1486376 (0.0007) [2023-12-27 15:57:18,410][20381] Updated weights for policy 1, policy_version 1486386 (0.0008) [2023-12-27 15:57:18,470][20381] Updated weights for policy 1, policy_version 1486396 (0.0008) [2023-12-27 15:57:18,830][20347] Updated weights for policy 0, policy_version 1487866 (0.0007) [2023-12-27 15:57:18,886][20347] Updated weights for policy 0, policy_version 1487876 (0.0007) [2023-12-27 15:57:18,944][20347] Updated weights for policy 0, policy_version 1487886 (0.0007) [2023-12-27 15:57:18,995][20347] Updated weights for policy 0, policy_version 1487896 (0.0007) [2023-12-27 15:57:19,167][20381] Updated weights for policy 1, policy_version 1486406 (0.0008) [2023-12-27 15:57:19,229][20381] Updated weights for policy 1, policy_version 1486416 (0.0008) [2023-12-27 15:57:19,300][20381] Updated weights for policy 1, policy_version 1486426 (0.0008) [2023-12-27 15:57:19,717][20347] Updated weights for policy 0, policy_version 1487906 (0.0008) [2023-12-27 15:57:19,781][20347] Updated weights for policy 0, policy_version 1487916 (0.0007) [2023-12-27 15:57:19,845][20347] Updated weights for policy 0, policy_version 1487926 (0.0007) [2023-12-27 15:57:20,035][20381] Updated weights for policy 1, policy_version 1486436 (0.0007) [2023-12-27 15:57:20,096][20381] Updated weights for policy 1, policy_version 1486446 (0.0008) [2023-12-27 15:57:20,153][20381] Updated weights for policy 1, policy_version 1486456 (0.0008) [2023-12-27 15:57:20,553][20347] Updated weights for policy 0, policy_version 1487936 (0.0007) [2023-12-27 15:57:20,622][20347] Updated weights for policy 0, policy_version 1487946 (0.0009) [2023-12-27 15:57:20,684][20347] Updated weights for policy 0, policy_version 1487956 (0.0008) [2023-12-27 15:57:20,857][20381] Updated weights for policy 1, policy_version 1486466 (0.0008) [2023-12-27 15:57:20,924][20381] Updated weights for policy 1, policy_version 1486476 (0.0008) [2023-12-27 15:57:20,986][20381] Updated weights for policy 1, policy_version 1486486 (0.0008) [2023-12-27 15:57:21,054][20381] Updated weights for policy 1, policy_version 1486496 (0.0008) [2023-12-27 15:57:21,397][20347] Updated weights for policy 0, policy_version 1487966 (0.0007) [2023-12-27 15:57:21,452][20347] Updated weights for policy 0, policy_version 1487976 (0.0007) [2023-12-27 15:57:21,504][20347] Updated weights for policy 0, policy_version 1487986 (0.0005) [2023-12-27 15:57:21,860][20381] Updated weights for policy 1, policy_version 1486506 (0.0009) [2023-12-27 15:57:21,915][20381] Updated weights for policy 1, policy_version 1486516 (0.0009) [2023-12-27 15:57:21,970][20381] Updated weights for policy 1, policy_version 1486526 (0.0009) [2023-12-27 15:57:22,256][20347] Updated weights for policy 0, policy_version 1487996 (0.0007) [2023-12-27 15:57:22,322][20347] Updated weights for policy 0, policy_version 1488006 (0.0006) [2023-12-27 15:57:22,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 761585664. Throughput: 0: 10361.8, 1: 10202.9. Samples: 761561464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:22,363][19170] Avg episode reward: [(0, '970.770'), (1, '990.280')] [2023-12-27 15:57:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001486528_380583936.pth... [2023-12-27 15:57:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001485376_380289024.pth [2023-12-27 15:57:22,386][20347] Updated weights for policy 0, policy_version 1488016 (0.0008) [2023-12-27 15:57:22,429][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001488024_381009920.pth... [2023-12-27 15:57:22,432][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001486808_380698624.pth [2023-12-27 15:57:22,705][20381] Updated weights for policy 1, policy_version 1486536 (0.0008) [2023-12-27 15:57:22,768][20381] Updated weights for policy 1, policy_version 1486546 (0.0006) [2023-12-27 15:57:22,832][20381] Updated weights for policy 1, policy_version 1486556 (0.0006) [2023-12-27 15:57:23,095][20347] Updated weights for policy 0, policy_version 1488026 (0.0007) [2023-12-27 15:57:23,162][20347] Updated weights for policy 0, policy_version 1488036 (0.0008) [2023-12-27 15:57:23,221][20347] Updated weights for policy 0, policy_version 1488046 (0.0007) [2023-12-27 15:57:23,269][20347] Updated weights for policy 0, policy_version 1488056 (0.0008) [2023-12-27 15:57:23,402][20381] Updated weights for policy 1, policy_version 1486566 (0.0007) [2023-12-27 15:57:23,454][20381] Updated weights for policy 1, policy_version 1486576 (0.0007) [2023-12-27 15:57:23,502][20381] Updated weights for policy 1, policy_version 1486586 (0.0008) [2023-12-27 15:57:23,897][20347] Updated weights for policy 0, policy_version 1488066 (0.0008) [2023-12-27 15:57:23,957][20347] Updated weights for policy 0, policy_version 1488076 (0.0007) [2023-12-27 15:57:24,010][20347] Updated weights for policy 0, policy_version 1488086 (0.0008) [2023-12-27 15:57:24,202][20381] Updated weights for policy 1, policy_version 1486596 (0.0008) [2023-12-27 15:57:24,251][20381] Updated weights for policy 1, policy_version 1486606 (0.0008) [2023-12-27 15:57:24,303][20381] Updated weights for policy 1, policy_version 1486616 (0.0008) [2023-12-27 15:57:24,691][20347] Updated weights for policy 0, policy_version 1488096 (0.0006) [2023-12-27 15:57:24,737][20347] Updated weights for policy 0, policy_version 1488106 (0.0009) [2023-12-27 15:57:24,779][20347] Updated weights for policy 0, policy_version 1488116 (0.0009) [2023-12-27 15:57:25,033][20381] Updated weights for policy 1, policy_version 1486626 (0.0007) [2023-12-27 15:57:25,080][20381] Updated weights for policy 1, policy_version 1486636 (0.0007) [2023-12-27 15:57:25,124][20381] Updated weights for policy 1, policy_version 1486646 (0.0007) [2023-12-27 15:57:25,168][20381] Updated weights for policy 1, policy_version 1486656 (0.0007) [2023-12-27 15:57:25,405][20347] Updated weights for policy 0, policy_version 1488126 (0.0008) [2023-12-27 15:57:25,455][20347] Updated weights for policy 0, policy_version 1488136 (0.0010) [2023-12-27 15:57:25,506][20347] Updated weights for policy 0, policy_version 1488146 (0.0010) [2023-12-27 15:57:25,873][20381] Updated weights for policy 1, policy_version 1486666 (0.0007) [2023-12-27 15:57:25,918][20381] Updated weights for policy 1, policy_version 1486676 (0.0008) [2023-12-27 15:57:25,962][20381] Updated weights for policy 1, policy_version 1486686 (0.0009) [2023-12-27 15:57:26,137][20347] Updated weights for policy 0, policy_version 1488156 (0.0010) [2023-12-27 15:57:26,198][20347] Updated weights for policy 0, policy_version 1488166 (0.0006) [2023-12-27 15:57:26,255][20347] Updated weights for policy 0, policy_version 1488176 (0.0005) [2023-12-27 15:57:26,690][20381] Updated weights for policy 1, policy_version 1486696 (0.0008) [2023-12-27 15:57:26,737][20381] Updated weights for policy 1, policy_version 1486706 (0.0007) [2023-12-27 15:57:26,793][20381] Updated weights for policy 1, policy_version 1486716 (0.0007) [2023-12-27 15:57:26,797][20347] Updated weights for policy 0, policy_version 1488186 (0.0005) [2023-12-27 15:57:26,847][20347] Updated weights for policy 0, policy_version 1488196 (0.0007) [2023-12-27 15:57:26,910][20347] Updated weights for policy 0, policy_version 1488206 (0.0006) [2023-12-27 15:57:26,967][20347] Updated weights for policy 0, policy_version 1488216 (0.0007) [2023-12-27 15:57:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 761692160. Throughput: 0: 10343.6, 1: 10133.4. Samples: 761681956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:27,363][19170] Avg episode reward: [(0, '970.730'), (1, '990.280')] [2023-12-27 15:57:27,468][20381] Updated weights for policy 1, policy_version 1486726 (0.0007) [2023-12-27 15:57:27,523][20381] Updated weights for policy 1, policy_version 1486736 (0.0005) [2023-12-27 15:57:27,578][20381] Updated weights for policy 1, policy_version 1486746 (0.0006) [2023-12-27 15:57:27,602][20347] Updated weights for policy 0, policy_version 1488226 (0.0005) [2023-12-27 15:57:27,646][20347] Updated weights for policy 0, policy_version 1488236 (0.0007) [2023-12-27 15:57:27,703][20347] Updated weights for policy 0, policy_version 1488246 (0.0008) [2023-12-27 15:57:28,193][20381] Updated weights for policy 1, policy_version 1486756 (0.0006) [2023-12-27 15:57:28,244][20381] Updated weights for policy 1, policy_version 1486766 (0.0008) [2023-12-27 15:57:28,299][20381] Updated weights for policy 1, policy_version 1486776 (0.0008) [2023-12-27 15:57:28,397][20347] Updated weights for policy 0, policy_version 1488256 (0.0008) [2023-12-27 15:57:28,457][20347] Updated weights for policy 0, policy_version 1488266 (0.0007) [2023-12-27 15:57:28,514][20347] Updated weights for policy 0, policy_version 1488276 (0.0007) [2023-12-27 15:57:29,024][20381] Updated weights for policy 1, policy_version 1486786 (0.0008) [2023-12-27 15:57:29,078][20381] Updated weights for policy 1, policy_version 1486796 (0.0010) [2023-12-27 15:57:29,126][20347] Updated weights for policy 0, policy_version 1488286 (0.0008) [2023-12-27 15:57:29,136][20381] Updated weights for policy 1, policy_version 1486806 (0.0010) [2023-12-27 15:57:29,179][20347] Updated weights for policy 0, policy_version 1488296 (0.0006) [2023-12-27 15:57:29,192][20381] Updated weights for policy 1, policy_version 1486816 (0.0011) [2023-12-27 15:57:29,238][20347] Updated weights for policy 0, policy_version 1488306 (0.0008) [2023-12-27 15:57:29,926][20347] Updated weights for policy 0, policy_version 1488316 (0.0008) [2023-12-27 15:57:29,975][20381] Updated weights for policy 1, policy_version 1486826 (0.0009) [2023-12-27 15:57:29,987][20347] Updated weights for policy 0, policy_version 1488326 (0.0007) [2023-12-27 15:57:30,023][20381] Updated weights for policy 1, policy_version 1486836 (0.0007) [2023-12-27 15:57:30,051][20347] Updated weights for policy 0, policy_version 1488336 (0.0006) [2023-12-27 15:57:30,076][20381] Updated weights for policy 1, policy_version 1486846 (0.0006) [2023-12-27 15:57:30,674][20347] Updated weights for policy 0, policy_version 1488346 (0.0008) [2023-12-27 15:57:30,731][20347] Updated weights for policy 0, policy_version 1488356 (0.0007) [2023-12-27 15:57:30,776][20347] Updated weights for policy 0, policy_version 1488366 (0.0008) [2023-12-27 15:57:30,812][20381] Updated weights for policy 1, policy_version 1486856 (0.0007) [2023-12-27 15:57:30,826][20347] Updated weights for policy 0, policy_version 1488376 (0.0006) [2023-12-27 15:57:30,867][20381] Updated weights for policy 1, policy_version 1486866 (0.0007) [2023-12-27 15:57:30,924][20381] Updated weights for policy 1, policy_version 1486876 (0.0007) [2023-12-27 15:57:31,488][20347] Updated weights for policy 0, policy_version 1488386 (0.0007) [2023-12-27 15:57:31,544][20347] Updated weights for policy 0, policy_version 1488396 (0.0007) [2023-12-27 15:57:31,597][20347] Updated weights for policy 0, policy_version 1488406 (0.0009) [2023-12-27 15:57:31,648][20381] Updated weights for policy 1, policy_version 1486886 (0.0007) [2023-12-27 15:57:31,710][20381] Updated weights for policy 1, policy_version 1486896 (0.0008) [2023-12-27 15:57:31,767][20381] Updated weights for policy 1, policy_version 1486906 (0.0008) [2023-12-27 15:57:32,316][20347] Updated weights for policy 0, policy_version 1488416 (0.0008) [2023-12-27 15:57:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 761790464. Throughput: 0: 10425.7, 1: 10087.1. Samples: 761805392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:32,363][19170] Avg episode reward: [(0, '961.040'), (1, '1000.000')] [2023-12-27 15:57:32,383][20347] Updated weights for policy 0, policy_version 1488426 (0.0007) [2023-12-27 15:57:32,445][20347] Updated weights for policy 0, policy_version 1488436 (0.0009) [2023-12-27 15:57:32,467][20381] Updated weights for policy 1, policy_version 1486916 (0.0007) [2023-12-27 15:57:32,523][20381] Updated weights for policy 1, policy_version 1486926 (0.0007) [2023-12-27 15:57:32,588][20381] Updated weights for policy 1, policy_version 1486936 (0.0008) [2023-12-27 15:57:33,159][20381] Updated weights for policy 1, policy_version 1486946 (0.0008) [2023-12-27 15:57:33,209][20381] Updated weights for policy 1, policy_version 1486956 (0.0007) [2023-12-27 15:57:33,220][20347] Updated weights for policy 0, policy_version 1488446 (0.0008) [2023-12-27 15:57:33,258][20381] Updated weights for policy 1, policy_version 1486966 (0.0005) [2023-12-27 15:57:33,270][20347] Updated weights for policy 0, policy_version 1488456 (0.0007) [2023-12-27 15:57:33,301][20381] Updated weights for policy 1, policy_version 1486976 (0.0005) [2023-12-27 15:57:33,319][20347] Updated weights for policy 0, policy_version 1488466 (0.0008) [2023-12-27 15:57:33,961][20381] Updated weights for policy 1, policy_version 1486986 (0.0008) [2023-12-27 15:57:34,015][20381] Updated weights for policy 1, policy_version 1486996 (0.0006) [2023-12-27 15:57:34,025][20347] Updated weights for policy 0, policy_version 1488476 (0.0007) [2023-12-27 15:57:34,070][20381] Updated weights for policy 1, policy_version 1487006 (0.0006) [2023-12-27 15:57:34,079][20347] Updated weights for policy 0, policy_version 1488486 (0.0005) [2023-12-27 15:57:34,136][20347] Updated weights for policy 0, policy_version 1488496 (0.0007) [2023-12-27 15:57:34,745][20381] Updated weights for policy 1, policy_version 1487016 (0.0010) [2023-12-27 15:57:34,790][20381] Updated weights for policy 1, policy_version 1487026 (0.0010) [2023-12-27 15:57:34,840][20381] Updated weights for policy 1, policy_version 1487036 (0.0010) [2023-12-27 15:57:34,872][20347] Updated weights for policy 0, policy_version 1488506 (0.0006) [2023-12-27 15:57:34,933][20347] Updated weights for policy 0, policy_version 1488516 (0.0006) [2023-12-27 15:57:34,978][20347] Updated weights for policy 0, policy_version 1488526 (0.0007) [2023-12-27 15:57:35,036][20347] Updated weights for policy 0, policy_version 1488536 (0.0007) [2023-12-27 15:57:35,511][20381] Updated weights for policy 1, policy_version 1487046 (0.0010) [2023-12-27 15:57:35,572][20381] Updated weights for policy 1, policy_version 1487056 (0.0010) [2023-12-27 15:57:35,629][20381] Updated weights for policy 1, policy_version 1487066 (0.0010) [2023-12-27 15:57:35,666][20291] KL-divergence is very high: 111.1951 [2023-12-27 15:57:35,672][20347] Updated weights for policy 0, policy_version 1488546 (0.0005) [2023-12-27 15:57:35,710][20291] KL-divergence is very high: 117.6108 [2023-12-27 15:57:35,727][20347] Updated weights for policy 0, policy_version 1488556 (0.0007) [2023-12-27 15:57:35,774][20347] Updated weights for policy 0, policy_version 1488566 (0.0007) [2023-12-27 15:57:36,325][20381] Updated weights for policy 1, policy_version 1487076 (0.0010) [2023-12-27 15:57:36,386][20381] Updated weights for policy 1, policy_version 1487086 (0.0009) [2023-12-27 15:57:36,448][20381] Updated weights for policy 1, policy_version 1487096 (0.0009) [2023-12-27 15:57:36,526][20347] Updated weights for policy 0, policy_version 1488576 (0.0009) [2023-12-27 15:57:36,587][20347] Updated weights for policy 0, policy_version 1488586 (0.0008) [2023-12-27 15:57:36,643][20347] Updated weights for policy 0, policy_version 1488596 (0.0006) [2023-12-27 15:57:37,207][20381] Updated weights for policy 1, policy_version 1487106 (0.0009) [2023-12-27 15:57:37,266][20381] Updated weights for policy 1, policy_version 1487116 (0.0006) [2023-12-27 15:57:37,315][20347] Updated weights for policy 0, policy_version 1488606 (0.0006) [2023-12-27 15:57:37,326][20381] Updated weights for policy 1, policy_version 1487126 (0.0007) [2023-12-27 15:57:37,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 761888768. Throughput: 0: 10389.1, 1: 10087.7. Samples: 761866716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:37,363][19170] Avg episode reward: [(0, '951.270'), (1, '1000.000')] [2023-12-27 15:57:37,375][20347] Updated weights for policy 0, policy_version 1488616 (0.0008) [2023-12-27 15:57:37,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001487136_380739584.pth... [2023-12-27 15:57:37,387][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001485952_380436480.pth [2023-12-27 15:57:37,387][20381] Updated weights for policy 1, policy_version 1487136 (0.0007) [2023-12-27 15:57:37,434][20347] Updated weights for policy 0, policy_version 1488626 (0.0006) [2023-12-27 15:57:37,471][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001488632_381165568.pth... [2023-12-27 15:57:37,474][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001487416_380854272.pth [2023-12-27 15:57:38,012][20381] Updated weights for policy 1, policy_version 1487146 (0.0007) [2023-12-27 15:57:38,066][20381] Updated weights for policy 1, policy_version 1487156 (0.0006) [2023-12-27 15:57:38,090][20347] Updated weights for policy 0, policy_version 1488636 (0.0008) [2023-12-27 15:57:38,122][20381] Updated weights for policy 1, policy_version 1487166 (0.0005) [2023-12-27 15:57:38,148][20347] Updated weights for policy 0, policy_version 1488646 (0.0007) [2023-12-27 15:57:38,203][20347] Updated weights for policy 0, policy_version 1488656 (0.0010) [2023-12-27 15:57:38,690][20381] Updated weights for policy 1, policy_version 1487176 (0.0006) [2023-12-27 15:57:38,748][20381] Updated weights for policy 1, policy_version 1487186 (0.0011) [2023-12-27 15:57:38,804][20381] Updated weights for policy 1, policy_version 1487196 (0.0010) [2023-12-27 15:57:38,876][20347] Updated weights for policy 0, policy_version 1488666 (0.0010) [2023-12-27 15:57:38,937][20347] Updated weights for policy 0, policy_version 1488676 (0.0010) [2023-12-27 15:57:38,994][20347] Updated weights for policy 0, policy_version 1488686 (0.0010) [2023-12-27 15:57:39,042][20347] Updated weights for policy 0, policy_version 1488696 (0.0010) [2023-12-27 15:57:39,525][20381] Updated weights for policy 1, policy_version 1487206 (0.0009) [2023-12-27 15:57:39,577][20381] Updated weights for policy 1, policy_version 1487216 (0.0008) [2023-12-27 15:57:39,629][20381] Updated weights for policy 1, policy_version 1487226 (0.0008) [2023-12-27 15:57:39,755][20347] Updated weights for policy 0, policy_version 1488706 (0.0007) [2023-12-27 15:57:39,815][20347] Updated weights for policy 0, policy_version 1488716 (0.0008) [2023-12-27 15:57:39,884][20347] Updated weights for policy 0, policy_version 1488726 (0.0008) [2023-12-27 15:57:40,405][20381] Updated weights for policy 1, policy_version 1487236 (0.0011) [2023-12-27 15:57:40,461][20381] Updated weights for policy 1, policy_version 1487246 (0.0011) [2023-12-27 15:57:40,524][20381] Updated weights for policy 1, policy_version 1487256 (0.0011) [2023-12-27 15:57:40,592][20347] Updated weights for policy 0, policy_version 1488736 (0.0008) [2023-12-27 15:57:40,649][20347] Updated weights for policy 0, policy_version 1488746 (0.0007) [2023-12-27 15:57:40,705][20347] Updated weights for policy 0, policy_version 1488756 (0.0007) [2023-12-27 15:57:41,169][20381] Updated weights for policy 1, policy_version 1487266 (0.0010) [2023-12-27 15:57:41,226][20381] Updated weights for policy 1, policy_version 1487276 (0.0008) [2023-12-27 15:57:41,287][20381] Updated weights for policy 1, policy_version 1487286 (0.0007) [2023-12-27 15:57:41,346][20381] Updated weights for policy 1, policy_version 1487296 (0.0008) [2023-12-27 15:57:41,421][20347] Updated weights for policy 0, policy_version 1488766 (0.0008) [2023-12-27 15:57:41,483][20347] Updated weights for policy 0, policy_version 1488776 (0.0007) [2023-12-27 15:57:41,541][20347] Updated weights for policy 0, policy_version 1488786 (0.0006) [2023-12-27 15:57:42,071][20381] Updated weights for policy 1, policy_version 1487306 (0.0005) [2023-12-27 15:57:42,130][20381] Updated weights for policy 1, policy_version 1487316 (0.0007) [2023-12-27 15:57:42,194][20381] Updated weights for policy 1, policy_version 1487326 (0.0008) [2023-12-27 15:57:42,261][20347] Updated weights for policy 0, policy_version 1488796 (0.0008) [2023-12-27 15:57:42,321][20347] Updated weights for policy 0, policy_version 1488806 (0.0009) [2023-12-27 15:57:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 761995264. Throughput: 0: 10376.9, 1: 10019.1. Samples: 761988088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:42,363][19170] Avg episode reward: [(0, '980.510'), (1, '990.290')] [2023-12-27 15:57:42,389][20347] Updated weights for policy 0, policy_version 1488816 (0.0009) [2023-12-27 15:57:42,834][20381] Updated weights for policy 1, policy_version 1487336 (0.0007) [2023-12-27 15:57:42,889][20381] Updated weights for policy 1, policy_version 1487346 (0.0006) [2023-12-27 15:57:42,945][20381] Updated weights for policy 1, policy_version 1487356 (0.0007) [2023-12-27 15:57:43,057][20347] Updated weights for policy 0, policy_version 1488826 (0.0009) [2023-12-27 15:57:43,112][20347] Updated weights for policy 0, policy_version 1488836 (0.0005) [2023-12-27 15:57:43,160][20347] Updated weights for policy 0, policy_version 1488846 (0.0005) [2023-12-27 15:57:43,221][20347] Updated weights for policy 0, policy_version 1488856 (0.0005) [2023-12-27 15:57:43,601][20381] Updated weights for policy 1, policy_version 1487366 (0.0005) [2023-12-27 15:57:43,661][20381] Updated weights for policy 1, policy_version 1487376 (0.0006) [2023-12-27 15:57:43,721][20381] Updated weights for policy 1, policy_version 1487386 (0.0005) [2023-12-27 15:57:43,794][20347] Updated weights for policy 0, policy_version 1488866 (0.0005) [2023-12-27 15:57:43,844][20347] Updated weights for policy 0, policy_version 1488876 (0.0005) [2023-12-27 15:57:43,900][20347] Updated weights for policy 0, policy_version 1488886 (0.0005) [2023-12-27 15:57:44,263][20381] Updated weights for policy 1, policy_version 1487396 (0.0007) [2023-12-27 15:57:44,323][20381] Updated weights for policy 1, policy_version 1487406 (0.0006) [2023-12-27 15:57:44,383][20381] Updated weights for policy 1, policy_version 1487416 (0.0008) [2023-12-27 15:57:44,491][20347] Updated weights for policy 0, policy_version 1488896 (0.0006) [2023-12-27 15:57:44,552][20347] Updated weights for policy 0, policy_version 1488906 (0.0008) [2023-12-27 15:57:44,612][20347] Updated weights for policy 0, policy_version 1488916 (0.0007) [2023-12-27 15:57:45,092][20381] Updated weights for policy 1, policy_version 1487426 (0.0008) [2023-12-27 15:57:45,156][20381] Updated weights for policy 1, policy_version 1487436 (0.0008) [2023-12-27 15:57:45,224][20381] Updated weights for policy 1, policy_version 1487446 (0.0007) [2023-12-27 15:57:45,246][20347] Updated weights for policy 0, policy_version 1488926 (0.0006) [2023-12-27 15:57:45,286][20381] Updated weights for policy 1, policy_version 1487456 (0.0008) [2023-12-27 15:57:45,303][20347] Updated weights for policy 0, policy_version 1488936 (0.0006) [2023-12-27 15:57:45,362][20347] Updated weights for policy 0, policy_version 1488946 (0.0006) [2023-12-27 15:57:45,967][20347] Updated weights for policy 0, policy_version 1488956 (0.0007) [2023-12-27 15:57:46,015][20347] Updated weights for policy 0, policy_version 1488966 (0.0007) [2023-12-27 15:57:46,022][20381] Updated weights for policy 1, policy_version 1487466 (0.0005) [2023-12-27 15:57:46,066][20347] Updated weights for policy 0, policy_version 1488976 (0.0007) [2023-12-27 15:57:46,081][20381] Updated weights for policy 1, policy_version 1487476 (0.0005) [2023-12-27 15:57:46,138][20381] Updated weights for policy 1, policy_version 1487486 (0.0005) [2023-12-27 15:57:46,705][20381] Updated weights for policy 1, policy_version 1487496 (0.0007) [2023-12-27 15:57:46,750][20347] Updated weights for policy 0, policy_version 1488986 (0.0007) [2023-12-27 15:57:46,759][20381] Updated weights for policy 1, policy_version 1487506 (0.0008) [2023-12-27 15:57:46,812][20347] Updated weights for policy 0, policy_version 1488996 (0.0007) [2023-12-27 15:57:46,815][20381] Updated weights for policy 1, policy_version 1487516 (0.0005) [2023-12-27 15:57:46,865][20347] Updated weights for policy 0, policy_version 1489006 (0.0007) [2023-12-27 15:57:46,913][20347] Updated weights for policy 0, policy_version 1489016 (0.0007) [2023-12-27 15:57:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 762101760. Throughput: 0: 10359.1, 1: 10082.3. Samples: 762113928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:47,364][19170] Avg episode reward: [(0, '970.710'), (1, '990.290')] [2023-12-27 15:57:47,477][20381] Updated weights for policy 1, policy_version 1487526 (0.0007) [2023-12-27 15:57:47,540][20381] Updated weights for policy 1, policy_version 1487536 (0.0008) [2023-12-27 15:57:47,596][20347] Updated weights for policy 0, policy_version 1489026 (0.0007) [2023-12-27 15:57:47,599][20381] Updated weights for policy 1, policy_version 1487546 (0.0005) [2023-12-27 15:57:47,654][20347] Updated weights for policy 0, policy_version 1489036 (0.0006) [2023-12-27 15:57:47,716][20347] Updated weights for policy 0, policy_version 1489046 (0.0007) [2023-12-27 15:57:48,260][20381] Updated weights for policy 1, policy_version 1487556 (0.0007) [2023-12-27 15:57:48,309][20381] Updated weights for policy 1, policy_version 1487566 (0.0009) [2023-12-27 15:57:48,371][20381] Updated weights for policy 1, policy_version 1487576 (0.0007) [2023-12-27 15:57:48,388][20347] Updated weights for policy 0, policy_version 1489056 (0.0007) [2023-12-27 15:57:48,454][20347] Updated weights for policy 0, policy_version 1489066 (0.0007) [2023-12-27 15:57:48,512][20347] Updated weights for policy 0, policy_version 1489076 (0.0008) [2023-12-27 15:57:49,133][20381] Updated weights for policy 1, policy_version 1487586 (0.0008) [2023-12-27 15:57:49,154][20347] Updated weights for policy 0, policy_version 1489086 (0.0006) [2023-12-27 15:57:49,186][20381] Updated weights for policy 1, policy_version 1487596 (0.0006) [2023-12-27 15:57:49,203][20347] Updated weights for policy 0, policy_version 1489096 (0.0005) [2023-12-27 15:57:49,249][20381] Updated weights for policy 1, policy_version 1487606 (0.0007) [2023-12-27 15:57:49,273][20347] Updated weights for policy 0, policy_version 1489106 (0.0007) [2023-12-27 15:57:49,306][20381] Updated weights for policy 1, policy_version 1487616 (0.0006) [2023-12-27 15:57:50,003][20347] Updated weights for policy 0, policy_version 1489116 (0.0007) [2023-12-27 15:57:50,027][20381] Updated weights for policy 1, policy_version 1487626 (0.0007) [2023-12-27 15:57:50,062][20347] Updated weights for policy 0, policy_version 1489126 (0.0007) [2023-12-27 15:57:50,084][20381] Updated weights for policy 1, policy_version 1487636 (0.0006) [2023-12-27 15:57:50,115][20347] Updated weights for policy 0, policy_version 1489136 (0.0007) [2023-12-27 15:57:50,144][20381] Updated weights for policy 1, policy_version 1487646 (0.0007) [2023-12-27 15:57:50,817][20381] Updated weights for policy 1, policy_version 1487656 (0.0008) [2023-12-27 15:57:50,850][20347] Updated weights for policy 0, policy_version 1489146 (0.0007) [2023-12-27 15:57:50,868][20381] Updated weights for policy 1, policy_version 1487666 (0.0006) [2023-12-27 15:57:50,907][20347] Updated weights for policy 0, policy_version 1489156 (0.0006) [2023-12-27 15:57:50,934][20381] Updated weights for policy 1, policy_version 1487676 (0.0006) [2023-12-27 15:57:50,965][20347] Updated weights for policy 0, policy_version 1489166 (0.0006) [2023-12-27 15:57:51,026][20347] Updated weights for policy 0, policy_version 1489176 (0.0008) [2023-12-27 15:57:51,715][20381] Updated weights for policy 1, policy_version 1487686 (0.0008) [2023-12-27 15:57:51,762][20347] Updated weights for policy 0, policy_version 1489186 (0.0009) [2023-12-27 15:57:51,782][20381] Updated weights for policy 1, policy_version 1487696 (0.0008) [2023-12-27 15:57:51,821][20347] Updated weights for policy 0, policy_version 1489196 (0.0008) [2023-12-27 15:57:51,838][20381] Updated weights for policy 1, policy_version 1487706 (0.0006) [2023-12-27 15:57:51,885][20347] Updated weights for policy 0, policy_version 1489206 (0.0006) [2023-12-27 15:57:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 762200064. Throughput: 0: 10262.5, 1: 10067.3. Samples: 762173868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:52,363][19170] Avg episode reward: [(0, '970.710'), (1, '1000.000')] [2023-12-27 15:57:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001487712_380887040.pth... [2023-12-27 15:57:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001489208_381313024.pth... [2023-12-27 15:57:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001488024_381009920.pth [2023-12-27 15:57:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001486528_380583936.pth [2023-12-27 15:57:52,545][20347] Updated weights for policy 0, policy_version 1489216 (0.0008) [2023-12-27 15:57:52,567][20381] Updated weights for policy 1, policy_version 1487716 (0.0006) [2023-12-27 15:57:52,603][20347] Updated weights for policy 0, policy_version 1489226 (0.0008) [2023-12-27 15:57:52,612][20381] Updated weights for policy 1, policy_version 1487726 (0.0006) [2023-12-27 15:57:52,661][20347] Updated weights for policy 0, policy_version 1489236 (0.0006) [2023-12-27 15:57:52,663][20381] Updated weights for policy 1, policy_version 1487736 (0.0007) [2023-12-27 15:57:53,267][20347] Updated weights for policy 0, policy_version 1489246 (0.0007) [2023-12-27 15:57:53,315][20347] Updated weights for policy 0, policy_version 1489256 (0.0008) [2023-12-27 15:57:53,368][20347] Updated weights for policy 0, policy_version 1489266 (0.0008) [2023-12-27 15:57:53,397][20381] Updated weights for policy 1, policy_version 1487746 (0.0007) [2023-12-27 15:57:53,455][20381] Updated weights for policy 1, policy_version 1487756 (0.0008) [2023-12-27 15:57:53,511][20381] Updated weights for policy 1, policy_version 1487766 (0.0008) [2023-12-27 15:57:53,563][20381] Updated weights for policy 1, policy_version 1487776 (0.0007) [2023-12-27 15:57:53,997][20347] Updated weights for policy 0, policy_version 1489276 (0.0006) [2023-12-27 15:57:54,052][20347] Updated weights for policy 0, policy_version 1489286 (0.0007) [2023-12-27 15:57:54,104][20347] Updated weights for policy 0, policy_version 1489296 (0.0010) [2023-12-27 15:57:54,263][20381] Updated weights for policy 1, policy_version 1487786 (0.0007) [2023-12-27 15:57:54,320][20381] Updated weights for policy 1, policy_version 1487796 (0.0007) [2023-12-27 15:57:54,381][20381] Updated weights for policy 1, policy_version 1487806 (0.0007) [2023-12-27 15:57:54,701][20347] Updated weights for policy 0, policy_version 1489306 (0.0009) [2023-12-27 15:57:54,754][20347] Updated weights for policy 0, policy_version 1489316 (0.0005) [2023-12-27 15:57:54,815][20347] Updated weights for policy 0, policy_version 1489326 (0.0005) [2023-12-27 15:57:54,870][20347] Updated weights for policy 0, policy_version 1489336 (0.0005) [2023-12-27 15:57:55,070][20381] Updated weights for policy 1, policy_version 1487816 (0.0008) [2023-12-27 15:57:55,124][20381] Updated weights for policy 1, policy_version 1487826 (0.0006) [2023-12-27 15:57:55,179][20381] Updated weights for policy 1, policy_version 1487836 (0.0008) [2023-12-27 15:57:55,431][20347] Updated weights for policy 0, policy_version 1489346 (0.0010) [2023-12-27 15:57:55,485][20347] Updated weights for policy 0, policy_version 1489356 (0.0010) [2023-12-27 15:57:55,543][20347] Updated weights for policy 0, policy_version 1489366 (0.0010) [2023-12-27 15:57:55,872][20381] Updated weights for policy 1, policy_version 1487846 (0.0009) [2023-12-27 15:57:55,919][20381] Updated weights for policy 1, policy_version 1487856 (0.0010) [2023-12-27 15:57:55,983][20381] Updated weights for policy 1, policy_version 1487866 (0.0010) [2023-12-27 15:57:56,167][20347] Updated weights for policy 0, policy_version 1489376 (0.0009) [2023-12-27 15:57:56,218][20347] Updated weights for policy 0, policy_version 1489386 (0.0007) [2023-12-27 15:57:56,281][20347] Updated weights for policy 0, policy_version 1489396 (0.0005) [2023-12-27 15:57:56,614][20381] Updated weights for policy 1, policy_version 1487876 (0.0011) [2023-12-27 15:57:56,675][20381] Updated weights for policy 1, policy_version 1487886 (0.0010) [2023-12-27 15:57:56,736][20381] Updated weights for policy 1, policy_version 1487896 (0.0010) [2023-12-27 15:57:56,860][20347] Updated weights for policy 0, policy_version 1489406 (0.0007) [2023-12-27 15:57:56,910][20347] Updated weights for policy 0, policy_version 1489416 (0.0005) [2023-12-27 15:57:56,964][20347] Updated weights for policy 0, policy_version 1489426 (0.0005) [2023-12-27 15:57:57,344][20381] Updated weights for policy 1, policy_version 1487906 (0.0009) [2023-12-27 15:57:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 762306560. Throughput: 0: 10292.4, 1: 10017.9. Samples: 762295860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:57:57,363][19170] Avg episode reward: [(0, '951.350'), (1, '990.260')] [2023-12-27 15:57:57,395][20381] Updated weights for policy 1, policy_version 1487916 (0.0010) [2023-12-27 15:57:57,457][20381] Updated weights for policy 1, policy_version 1487926 (0.0006) [2023-12-27 15:57:57,499][20347] Updated weights for policy 0, policy_version 1489436 (0.0007) [2023-12-27 15:57:57,510][20381] Updated weights for policy 1, policy_version 1487936 (0.0005) [2023-12-27 15:57:57,554][20347] Updated weights for policy 0, policy_version 1489446 (0.0010) [2023-12-27 15:57:57,612][20347] Updated weights for policy 0, policy_version 1489456 (0.0005) [2023-12-27 15:57:58,135][20381] Updated weights for policy 1, policy_version 1487946 (0.0010) [2023-12-27 15:57:58,161][20347] Updated weights for policy 0, policy_version 1489466 (0.0006) [2023-12-27 15:57:58,184][20381] Updated weights for policy 1, policy_version 1487956 (0.0010) [2023-12-27 15:57:58,220][20347] Updated weights for policy 0, policy_version 1489476 (0.0008) [2023-12-27 15:57:58,247][20381] Updated weights for policy 1, policy_version 1487966 (0.0009) [2023-12-27 15:57:58,279][20347] Updated weights for policy 0, policy_version 1489486 (0.0010) [2023-12-27 15:57:58,338][20347] Updated weights for policy 0, policy_version 1489496 (0.0009) [2023-12-27 15:57:59,056][20381] Updated weights for policy 1, policy_version 1487976 (0.0008) [2023-12-27 15:57:59,119][20381] Updated weights for policy 1, policy_version 1487986 (0.0006) [2023-12-27 15:57:59,127][20347] Updated weights for policy 0, policy_version 1489506 (0.0008) [2023-12-27 15:57:59,176][20381] Updated weights for policy 1, policy_version 1487996 (0.0008) [2023-12-27 15:57:59,181][20347] Updated weights for policy 0, policy_version 1489516 (0.0007) [2023-12-27 15:57:59,250][20347] Updated weights for policy 0, policy_version 1489526 (0.0008) [2023-12-27 15:57:59,813][20381] Updated weights for policy 1, policy_version 1488006 (0.0008) [2023-12-27 15:57:59,880][20381] Updated weights for policy 1, policy_version 1488016 (0.0007) [2023-12-27 15:57:59,939][20347] Updated weights for policy 0, policy_version 1489536 (0.0007) [2023-12-27 15:57:59,945][20381] Updated weights for policy 1, policy_version 1488026 (0.0007) [2023-12-27 15:57:59,994][20347] Updated weights for policy 0, policy_version 1489546 (0.0008) [2023-12-27 15:58:00,045][20347] Updated weights for policy 0, policy_version 1489556 (0.0008) [2023-12-27 15:58:00,593][20381] Updated weights for policy 1, policy_version 1488036 (0.0006) [2023-12-27 15:58:00,642][20381] Updated weights for policy 1, policy_version 1488046 (0.0008) [2023-12-27 15:58:00,691][20381] Updated weights for policy 1, policy_version 1488056 (0.0009) [2023-12-27 15:58:00,768][20347] Updated weights for policy 0, policy_version 1489566 (0.0007) [2023-12-27 15:58:00,821][20347] Updated weights for policy 0, policy_version 1489576 (0.0008) [2023-12-27 15:58:00,872][20347] Updated weights for policy 0, policy_version 1489586 (0.0006) [2023-12-27 15:58:01,428][20381] Updated weights for policy 1, policy_version 1488066 (0.0009) [2023-12-27 15:58:01,482][20381] Updated weights for policy 1, policy_version 1488076 (0.0007) [2023-12-27 15:58:01,539][20381] Updated weights for policy 1, policy_version 1488086 (0.0006) [2023-12-27 15:58:01,560][20347] Updated weights for policy 0, policy_version 1489596 (0.0007) [2023-12-27 15:58:01,596][20381] Updated weights for policy 1, policy_version 1488096 (0.0007) [2023-12-27 15:58:01,622][20347] Updated weights for policy 0, policy_version 1489606 (0.0010) [2023-12-27 15:58:01,681][20347] Updated weights for policy 0, policy_version 1489616 (0.0010) [2023-12-27 15:58:02,304][20381] Updated weights for policy 1, policy_version 1488106 (0.0007) [2023-12-27 15:58:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 762404864. Throughput: 0: 10346.4, 1: 10057.1. Samples: 762421436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:58:02,363][19170] Avg episode reward: [(0, '951.330'), (1, '990.290')] [2023-12-27 15:58:02,366][20381] Updated weights for policy 1, policy_version 1488116 (0.0007) [2023-12-27 15:58:02,374][20347] Updated weights for policy 0, policy_version 1489626 (0.0009) [2023-12-27 15:58:02,425][20381] Updated weights for policy 1, policy_version 1488126 (0.0005) [2023-12-27 15:58:02,431][20347] Updated weights for policy 0, policy_version 1489636 (0.0008) [2023-12-27 15:58:02,493][20347] Updated weights for policy 0, policy_version 1489646 (0.0007) [2023-12-27 15:58:02,551][20347] Updated weights for policy 0, policy_version 1489656 (0.0010) [2023-12-27 15:58:03,093][20381] Updated weights for policy 1, policy_version 1488136 (0.0007) [2023-12-27 15:58:03,140][20381] Updated weights for policy 1, policy_version 1488146 (0.0007) [2023-12-27 15:58:03,186][20381] Updated weights for policy 1, policy_version 1488156 (0.0007) [2023-12-27 15:58:03,240][20347] Updated weights for policy 0, policy_version 1489666 (0.0010) [2023-12-27 15:58:03,294][20347] Updated weights for policy 0, policy_version 1489676 (0.0010) [2023-12-27 15:58:03,339][20347] Updated weights for policy 0, policy_version 1489686 (0.0010) [2023-12-27 15:58:03,868][20381] Updated weights for policy 1, policy_version 1488166 (0.0010) [2023-12-27 15:58:03,922][20381] Updated weights for policy 1, policy_version 1488176 (0.0007) [2023-12-27 15:58:03,972][20347] Updated weights for policy 0, policy_version 1489696 (0.0009) [2023-12-27 15:58:03,976][20381] Updated weights for policy 1, policy_version 1488186 (0.0011) [2023-12-27 15:58:04,032][20347] Updated weights for policy 0, policy_version 1489706 (0.0010) [2023-12-27 15:58:04,090][20347] Updated weights for policy 0, policy_version 1489716 (0.0010) [2023-12-27 15:58:04,661][20381] Updated weights for policy 1, policy_version 1488196 (0.0008) [2023-12-27 15:58:04,718][20381] Updated weights for policy 1, policy_version 1488206 (0.0005) [2023-12-27 15:58:04,766][20381] Updated weights for policy 1, policy_version 1488216 (0.0007) [2023-12-27 15:58:04,773][20347] Updated weights for policy 0, policy_version 1489726 (0.0010) [2023-12-27 15:58:04,827][20347] Updated weights for policy 0, policy_version 1489736 (0.0010) [2023-12-27 15:58:04,881][20347] Updated weights for policy 0, policy_version 1489746 (0.0010) [2023-12-27 15:58:05,346][20381] Updated weights for policy 1, policy_version 1488226 (0.0005) [2023-12-27 15:58:05,403][20381] Updated weights for policy 1, policy_version 1488236 (0.0007) [2023-12-27 15:58:05,452][20381] Updated weights for policy 1, policy_version 1488246 (0.0010) [2023-12-27 15:58:05,497][20347] Updated weights for policy 0, policy_version 1489756 (0.0010) [2023-12-27 15:58:05,502][20381] Updated weights for policy 1, policy_version 1488256 (0.0008) [2023-12-27 15:58:05,550][20347] Updated weights for policy 0, policy_version 1489766 (0.0010) [2023-12-27 15:58:05,593][20347] Updated weights for policy 0, policy_version 1489776 (0.0005) [2023-12-27 15:58:06,158][20381] Updated weights for policy 1, policy_version 1488266 (0.0007) [2023-12-27 15:58:06,222][20381] Updated weights for policy 1, policy_version 1488276 (0.0007) [2023-12-27 15:58:06,222][20347] Updated weights for policy 0, policy_version 1489786 (0.0006) [2023-12-27 15:58:06,284][20381] Updated weights for policy 1, policy_version 1488286 (0.0007) [2023-12-27 15:58:06,287][20347] Updated weights for policy 0, policy_version 1489796 (0.0006) [2023-12-27 15:58:06,348][20347] Updated weights for policy 0, policy_version 1489806 (0.0008) [2023-12-27 15:58:06,409][20347] Updated weights for policy 0, policy_version 1489816 (0.0008) [2023-12-27 15:58:06,883][20381] Updated weights for policy 1, policy_version 1488296 (0.0010) [2023-12-27 15:58:06,941][20381] Updated weights for policy 1, policy_version 1488306 (0.0010) [2023-12-27 15:58:07,001][20381] Updated weights for policy 1, policy_version 1488316 (0.0011) [2023-12-27 15:58:07,104][20347] Updated weights for policy 0, policy_version 1489826 (0.0007) [2023-12-27 15:58:07,166][20347] Updated weights for policy 0, policy_version 1489836 (0.0007) [2023-12-27 15:58:07,226][20347] Updated weights for policy 0, policy_version 1489846 (0.0008) [2023-12-27 15:58:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 762519552. Throughput: 0: 10376.8, 1: 10118.9. Samples: 762483772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 15:58:07,363][19170] Avg episode reward: [(0, '970.750'), (1, '990.290')] [2023-12-27 15:58:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001488320_381042688.pth... [2023-12-27 15:58:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001489848_381476864.pth... [2023-12-27 15:58:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001487136_380739584.pth [2023-12-27 15:58:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001488632_381165568.pth [2023-12-27 15:58:07,697][20381] Updated weights for policy 1, policy_version 1488326 (0.0010) [2023-12-27 15:58:07,757][20381] Updated weights for policy 1, policy_version 1488336 (0.0009) [2023-12-27 15:58:07,815][20381] Updated weights for policy 1, policy_version 1488346 (0.0008) [2023-12-27 15:58:07,908][20347] Updated weights for policy 0, policy_version 1489856 (0.0007) [2023-12-27 15:58:07,972][20347] Updated weights for policy 0, policy_version 1489866 (0.0008) [2023-12-27 15:58:08,024][20347] Updated weights for policy 0, policy_version 1489876 (0.0009) [2023-12-27 15:58:08,466][20381] Updated weights for policy 1, policy_version 1488356 (0.0007) [2023-12-27 15:58:08,531][20381] Updated weights for policy 1, policy_version 1488366 (0.0009) [2023-12-27 15:58:08,600][20381] Updated weights for policy 1, policy_version 1488376 (0.0008) [2023-12-27 15:58:08,771][20347] Updated weights for policy 0, policy_version 1489886 (0.0007) [2023-12-27 15:58:08,831][20347] Updated weights for policy 0, policy_version 1489896 (0.0008) [2023-12-27 15:58:08,887][20347] Updated weights for policy 0, policy_version 1489906 (0.0007) [2023-12-27 15:58:09,266][20381] Updated weights for policy 1, policy_version 1488386 (0.0008) [2023-12-27 15:58:09,335][20381] Updated weights for policy 1, policy_version 1488396 (0.0007) [2023-12-27 15:58:09,398][20381] Updated weights for policy 1, policy_version 1488406 (0.0008) [2023-12-27 15:58:09,459][20381] Updated weights for policy 1, policy_version 1488416 (0.0007) [2023-12-27 15:58:09,671][20347] Updated weights for policy 0, policy_version 1489916 (0.0008) [2023-12-27 15:58:09,719][20347] Updated weights for policy 0, policy_version 1489926 (0.0008) [2023-12-27 15:58:09,775][20347] Updated weights for policy 0, policy_version 1489936 (0.0007) [2023-12-27 15:58:10,182][20381] Updated weights for policy 1, policy_version 1488426 (0.0007) [2023-12-27 15:58:10,239][20381] Updated weights for policy 1, policy_version 1488436 (0.0007) [2023-12-27 15:58:10,299][20381] Updated weights for policy 1, policy_version 1488446 (0.0008) [2023-12-27 15:58:10,448][20347] Updated weights for policy 0, policy_version 1489946 (0.0007) [2023-12-27 15:58:10,511][20347] Updated weights for policy 0, policy_version 1489956 (0.0005) [2023-12-27 15:58:10,568][20347] Updated weights for policy 0, policy_version 1489966 (0.0005) [2023-12-27 15:58:10,632][20347] Updated weights for policy 0, policy_version 1489976 (0.0006) [2023-12-27 15:58:10,908][20381] Updated weights for policy 1, policy_version 1488456 (0.0007) [2023-12-27 15:58:10,963][20381] Updated weights for policy 1, policy_version 1488466 (0.0008) [2023-12-27 15:58:11,019][20381] Updated weights for policy 1, policy_version 1488476 (0.0007) [2023-12-27 15:58:11,334][20347] Updated weights for policy 0, policy_version 1489986 (0.0008) [2023-12-27 15:58:11,403][20347] Updated weights for policy 0, policy_version 1489996 (0.0010) [2023-12-27 15:58:11,469][20347] Updated weights for policy 0, policy_version 1490006 (0.0009) [2023-12-27 15:58:11,808][20381] Updated weights for policy 1, policy_version 1488486 (0.0008) [2023-12-27 15:58:11,877][20381] Updated weights for policy 1, policy_version 1488496 (0.0008) [2023-12-27 15:58:11,938][20381] Updated weights for policy 1, policy_version 1488506 (0.0011) [2023-12-27 15:58:12,179][20347] Updated weights for policy 0, policy_version 1490016 (0.0007) [2023-12-27 15:58:12,231][20347] Updated weights for policy 0, policy_version 1490026 (0.0005) [2023-12-27 15:58:12,292][20347] Updated weights for policy 0, policy_version 1490036 (0.0007) [2023-12-27 15:58:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 762617856. Throughput: 0: 10359.7, 1: 10188.3. Samples: 762606620. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:12,363][19170] Avg episode reward: [(0, '980.560'), (1, '1000.000')] [2023-12-27 15:58:12,568][20381] Updated weights for policy 1, policy_version 1488516 (0.0008) [2023-12-27 15:58:12,627][20381] Updated weights for policy 1, policy_version 1488526 (0.0011) [2023-12-27 15:58:12,692][20381] Updated weights for policy 1, policy_version 1488536 (0.0010) [2023-12-27 15:58:12,905][20347] Updated weights for policy 0, policy_version 1490046 (0.0007) [2023-12-27 15:58:12,955][20347] Updated weights for policy 0, policy_version 1490056 (0.0007) [2023-12-27 15:58:13,009][20347] Updated weights for policy 0, policy_version 1490066 (0.0007) [2023-12-27 15:58:13,352][20381] Updated weights for policy 1, policy_version 1488546 (0.0011) [2023-12-27 15:58:13,401][20381] Updated weights for policy 1, policy_version 1488556 (0.0011) [2023-12-27 15:58:13,446][20381] Updated weights for policy 1, policy_version 1488566 (0.0010) [2023-12-27 15:58:13,498][20381] Updated weights for policy 1, policy_version 1488576 (0.0010) [2023-12-27 15:58:13,669][20347] Updated weights for policy 0, policy_version 1490076 (0.0007) [2023-12-27 15:58:13,729][20347] Updated weights for policy 0, policy_version 1490086 (0.0007) [2023-12-27 15:58:13,777][20347] Updated weights for policy 0, policy_version 1490096 (0.0007) [2023-12-27 15:58:14,193][20381] Updated weights for policy 1, policy_version 1488586 (0.0009) [2023-12-27 15:58:14,245][20381] Updated weights for policy 1, policy_version 1488596 (0.0010) [2023-12-27 15:58:14,302][20381] Updated weights for policy 1, policy_version 1488606 (0.0010) [2023-12-27 15:58:14,408][20347] Updated weights for policy 0, policy_version 1490106 (0.0007) [2023-12-27 15:58:14,461][20347] Updated weights for policy 0, policy_version 1490116 (0.0007) [2023-12-27 15:58:14,512][20347] Updated weights for policy 0, policy_version 1490126 (0.0007) [2023-12-27 15:58:14,560][20347] Updated weights for policy 0, policy_version 1490136 (0.0007) [2023-12-27 15:58:14,960][20381] Updated weights for policy 1, policy_version 1488616 (0.0011) [2023-12-27 15:58:15,024][20381] Updated weights for policy 1, policy_version 1488626 (0.0011) [2023-12-27 15:58:15,083][20381] Updated weights for policy 1, policy_version 1488636 (0.0007) [2023-12-27 15:58:15,290][20347] Updated weights for policy 0, policy_version 1490146 (0.0007) [2023-12-27 15:58:15,342][20347] Updated weights for policy 0, policy_version 1490156 (0.0008) [2023-12-27 15:58:15,394][20347] Updated weights for policy 0, policy_version 1490166 (0.0008) [2023-12-27 15:58:15,754][20381] Updated weights for policy 1, policy_version 1488646 (0.0008) [2023-12-27 15:58:15,817][20381] Updated weights for policy 1, policy_version 1488656 (0.0008) [2023-12-27 15:58:15,891][20381] Updated weights for policy 1, policy_version 1488666 (0.0008) [2023-12-27 15:58:16,102][20347] Updated weights for policy 0, policy_version 1490176 (0.0007) [2023-12-27 15:58:16,151][20347] Updated weights for policy 0, policy_version 1490186 (0.0007) [2023-12-27 15:58:16,195][20347] Updated weights for policy 0, policy_version 1490196 (0.0007) [2023-12-27 15:58:16,505][20381] Updated weights for policy 1, policy_version 1488676 (0.0007) [2023-12-27 15:58:16,565][20381] Updated weights for policy 1, policy_version 1488686 (0.0007) [2023-12-27 15:58:16,614][20381] Updated weights for policy 1, policy_version 1488696 (0.0010) [2023-12-27 15:58:16,799][20347] Updated weights for policy 0, policy_version 1490206 (0.0007) [2023-12-27 15:58:16,854][20347] Updated weights for policy 0, policy_version 1490216 (0.0008) [2023-12-27 15:58:16,917][20347] Updated weights for policy 0, policy_version 1490226 (0.0007) [2023-12-27 15:58:17,285][20381] Updated weights for policy 1, policy_version 1488706 (0.0010) [2023-12-27 15:58:17,330][20381] Updated weights for policy 1, policy_version 1488716 (0.0010) [2023-12-27 15:58:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 762716160. Throughput: 0: 10310.8, 1: 10247.2. Samples: 762730504. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:17,364][19170] Avg episode reward: [(0, '970.880'), (1, '1000.000')] [2023-12-27 15:58:17,375][20381] Updated weights for policy 1, policy_version 1488726 (0.0010) [2023-12-27 15:58:17,420][20381] Updated weights for policy 1, policy_version 1488736 (0.0010) [2023-12-27 15:58:17,575][20347] Updated weights for policy 0, policy_version 1490236 (0.0007) [2023-12-27 15:58:17,632][20347] Updated weights for policy 0, policy_version 1490246 (0.0008) [2023-12-27 15:58:17,690][20347] Updated weights for policy 0, policy_version 1490256 (0.0007) [2023-12-27 15:58:18,066][20381] Updated weights for policy 1, policy_version 1488746 (0.0007) [2023-12-27 15:58:18,122][20381] Updated weights for policy 1, policy_version 1488756 (0.0007) [2023-12-27 15:58:18,180][20381] Updated weights for policy 1, policy_version 1488766 (0.0007) [2023-12-27 15:58:18,391][20347] Updated weights for policy 0, policy_version 1490266 (0.0007) [2023-12-27 15:58:18,450][20347] Updated weights for policy 0, policy_version 1490276 (0.0008) [2023-12-27 15:58:18,509][20347] Updated weights for policy 0, policy_version 1490286 (0.0008) [2023-12-27 15:58:18,568][20347] Updated weights for policy 0, policy_version 1490296 (0.0007) [2023-12-27 15:58:18,839][20381] Updated weights for policy 1, policy_version 1488776 (0.0007) [2023-12-27 15:58:18,899][20381] Updated weights for policy 1, policy_version 1488786 (0.0008) [2023-12-27 15:58:18,959][20381] Updated weights for policy 1, policy_version 1488796 (0.0008) [2023-12-27 15:58:19,280][20347] Updated weights for policy 0, policy_version 1490306 (0.0007) [2023-12-27 15:58:19,339][20347] Updated weights for policy 0, policy_version 1490316 (0.0007) [2023-12-27 15:58:19,402][20347] Updated weights for policy 0, policy_version 1490326 (0.0007) [2023-12-27 15:58:19,704][20381] Updated weights for policy 1, policy_version 1488806 (0.0008) [2023-12-27 15:58:19,765][20381] Updated weights for policy 1, policy_version 1488816 (0.0009) [2023-12-27 15:58:19,827][20381] Updated weights for policy 1, policy_version 1488826 (0.0008) [2023-12-27 15:58:20,049][20347] Updated weights for policy 0, policy_version 1490336 (0.0007) [2023-12-27 15:58:20,105][20347] Updated weights for policy 0, policy_version 1490346 (0.0007) [2023-12-27 15:58:20,155][20347] Updated weights for policy 0, policy_version 1490356 (0.0007) [2023-12-27 15:58:20,487][20381] Updated weights for policy 1, policy_version 1488836 (0.0009) [2023-12-27 15:58:20,538][20381] Updated weights for policy 1, policy_version 1488846 (0.0008) [2023-12-27 15:58:20,603][20381] Updated weights for policy 1, policy_version 1488856 (0.0007) [2023-12-27 15:58:20,878][20347] Updated weights for policy 0, policy_version 1490366 (0.0009) [2023-12-27 15:58:20,938][20347] Updated weights for policy 0, policy_version 1490376 (0.0011) [2023-12-27 15:58:20,997][20347] Updated weights for policy 0, policy_version 1490386 (0.0011) [2023-12-27 15:58:21,349][20381] Updated weights for policy 1, policy_version 1488866 (0.0007) [2023-12-27 15:58:21,416][20381] Updated weights for policy 1, policy_version 1488876 (0.0011) [2023-12-27 15:58:21,465][20381] Updated weights for policy 1, policy_version 1488886 (0.0009) [2023-12-27 15:58:21,517][20381] Updated weights for policy 1, policy_version 1488896 (0.0008) [2023-12-27 15:58:21,768][20347] Updated weights for policy 0, policy_version 1490396 (0.0009) [2023-12-27 15:58:21,827][20347] Updated weights for policy 0, policy_version 1490406 (0.0010) [2023-12-27 15:58:21,892][20347] Updated weights for policy 0, policy_version 1490416 (0.0011) [2023-12-27 15:58:22,283][20381] Updated weights for policy 1, policy_version 1488906 (0.0008) [2023-12-27 15:58:22,341][20381] Updated weights for policy 1, policy_version 1488916 (0.0008) [2023-12-27 15:58:22,364][19170] Fps is (10 sec: 19656.9, 60 sec: 20479.3, 300 sec: 20521.5). Total num frames: 762814464. Throughput: 0: 10351.6, 1: 10203.6. Samples: 762791740. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:22,365][19170] Avg episode reward: [(0, '931.770'), (1, '1000.000')] [2023-12-27 15:58:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001490424_381624320.pth... [2023-12-27 15:58:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001489208_381313024.pth [2023-12-27 15:58:22,411][20381] Updated weights for policy 1, policy_version 1488926 (0.0008) [2023-12-27 15:58:22,421][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001488928_381198336.pth... [2023-12-27 15:58:22,425][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001487712_380887040.pth [2023-12-27 15:58:22,648][20347] Updated weights for policy 0, policy_version 1490426 (0.0010) [2023-12-27 15:58:22,715][20347] Updated weights for policy 0, policy_version 1490436 (0.0008) [2023-12-27 15:58:22,775][20347] Updated weights for policy 0, policy_version 1490446 (0.0008) [2023-12-27 15:58:22,837][20347] Updated weights for policy 0, policy_version 1490456 (0.0007) [2023-12-27 15:58:23,053][20381] Updated weights for policy 1, policy_version 1488936 (0.0007) [2023-12-27 15:58:23,112][20381] Updated weights for policy 1, policy_version 1488946 (0.0006) [2023-12-27 15:58:23,175][20381] Updated weights for policy 1, policy_version 1488956 (0.0006) [2023-12-27 15:58:23,489][20347] Updated weights for policy 0, policy_version 1490466 (0.0005) [2023-12-27 15:58:23,532][20347] Updated weights for policy 0, policy_version 1490476 (0.0005) [2023-12-27 15:58:23,581][20347] Updated weights for policy 0, policy_version 1490486 (0.0005) [2023-12-27 15:58:23,814][20381] Updated weights for policy 1, policy_version 1488966 (0.0009) [2023-12-27 15:58:23,862][20381] Updated weights for policy 1, policy_version 1488976 (0.0008) [2023-12-27 15:58:23,922][20381] Updated weights for policy 1, policy_version 1488986 (0.0005) [2023-12-27 15:58:24,202][20347] Updated weights for policy 0, policy_version 1490496 (0.0005) [2023-12-27 15:58:24,253][20347] Updated weights for policy 0, policy_version 1490506 (0.0006) [2023-12-27 15:58:24,310][20347] Updated weights for policy 0, policy_version 1490516 (0.0007) [2023-12-27 15:58:24,552][20381] Updated weights for policy 1, policy_version 1488996 (0.0009) [2023-12-27 15:58:24,604][20381] Updated weights for policy 1, policy_version 1489006 (0.0010) [2023-12-27 15:58:24,653][20381] Updated weights for policy 1, policy_version 1489016 (0.0011) [2023-12-27 15:58:24,956][20347] Updated weights for policy 0, policy_version 1490526 (0.0010) [2023-12-27 15:58:25,005][20347] Updated weights for policy 0, policy_version 1490536 (0.0007) [2023-12-27 15:58:25,049][20347] Updated weights for policy 0, policy_version 1490546 (0.0009) [2023-12-27 15:58:25,223][20381] Updated weights for policy 1, policy_version 1489026 (0.0011) [2023-12-27 15:58:25,278][20381] Updated weights for policy 1, policy_version 1489036 (0.0011) [2023-12-27 15:58:25,329][20381] Updated weights for policy 1, policy_version 1489046 (0.0010) [2023-12-27 15:58:25,383][20381] Updated weights for policy 1, policy_version 1489056 (0.0010) [2023-12-27 15:58:25,713][20347] Updated weights for policy 0, policy_version 1490556 (0.0008) [2023-12-27 15:58:25,760][20347] Updated weights for policy 0, policy_version 1490566 (0.0006) [2023-12-27 15:58:25,820][20347] Updated weights for policy 0, policy_version 1490576 (0.0005) [2023-12-27 15:58:26,083][20381] Updated weights for policy 1, policy_version 1489066 (0.0011) [2023-12-27 15:58:26,135][20381] Updated weights for policy 1, policy_version 1489076 (0.0010) [2023-12-27 15:58:26,180][20381] Updated weights for policy 1, policy_version 1489086 (0.0011) [2023-12-27 15:58:26,430][20347] Updated weights for policy 0, policy_version 1490586 (0.0005) [2023-12-27 15:58:26,475][20347] Updated weights for policy 0, policy_version 1490596 (0.0005) [2023-12-27 15:58:26,520][20347] Updated weights for policy 0, policy_version 1490606 (0.0005) [2023-12-27 15:58:26,573][20347] Updated weights for policy 0, policy_version 1490616 (0.0005) [2023-12-27 15:58:26,865][20381] Updated weights for policy 1, policy_version 1489096 (0.0010) [2023-12-27 15:58:26,922][20381] Updated weights for policy 1, policy_version 1489106 (0.0010) [2023-12-27 15:58:26,981][20381] Updated weights for policy 1, policy_version 1489116 (0.0011) [2023-12-27 15:58:27,117][20347] Updated weights for policy 0, policy_version 1490626 (0.0007) [2023-12-27 15:58:27,170][20347] Updated weights for policy 0, policy_version 1490636 (0.0007) [2023-12-27 15:58:27,222][20347] Updated weights for policy 0, policy_version 1490646 (0.0007) [2023-12-27 15:58:27,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 762929152. Throughput: 0: 10370.7, 1: 10226.7. Samples: 762914976. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:27,363][19170] Avg episode reward: [(0, '941.550'), (1, '1000.000')] [2023-12-27 15:58:27,633][20381] Updated weights for policy 1, policy_version 1489126 (0.0010) [2023-12-27 15:58:27,678][20381] Updated weights for policy 1, policy_version 1489136 (0.0010) [2023-12-27 15:58:27,729][20381] Updated weights for policy 1, policy_version 1489146 (0.0010) [2023-12-27 15:58:27,875][20347] Updated weights for policy 0, policy_version 1490656 (0.0007) [2023-12-27 15:58:27,935][20347] Updated weights for policy 0, policy_version 1490666 (0.0007) [2023-12-27 15:58:27,983][20347] Updated weights for policy 0, policy_version 1490676 (0.0008) [2023-12-27 15:58:28,375][20381] Updated weights for policy 1, policy_version 1489156 (0.0010) [2023-12-27 15:58:28,420][20381] Updated weights for policy 1, policy_version 1489166 (0.0010) [2023-12-27 15:58:28,475][20381] Updated weights for policy 1, policy_version 1489176 (0.0011) [2023-12-27 15:58:28,622][20347] Updated weights for policy 0, policy_version 1490686 (0.0007) [2023-12-27 15:58:28,673][20347] Updated weights for policy 0, policy_version 1490696 (0.0007) [2023-12-27 15:58:28,737][20347] Updated weights for policy 0, policy_version 1490706 (0.0007) [2023-12-27 15:58:29,172][20381] Updated weights for policy 1, policy_version 1489186 (0.0010) [2023-12-27 15:58:29,225][20381] Updated weights for policy 1, policy_version 1489196 (0.0009) [2023-12-27 15:58:29,287][20381] Updated weights for policy 1, policy_version 1489206 (0.0008) [2023-12-27 15:58:29,345][20381] Updated weights for policy 1, policy_version 1489216 (0.0008) [2023-12-27 15:58:29,439][20347] Updated weights for policy 0, policy_version 1490716 (0.0006) [2023-12-27 15:58:29,488][20347] Updated weights for policy 0, policy_version 1490726 (0.0008) [2023-12-27 15:58:29,536][20347] Updated weights for policy 0, policy_version 1490736 (0.0007) [2023-12-27 15:58:30,122][20381] Updated weights for policy 1, policy_version 1489226 (0.0006) [2023-12-27 15:58:30,176][20381] Updated weights for policy 1, policy_version 1489236 (0.0005) [2023-12-27 15:58:30,240][20381] Updated weights for policy 1, policy_version 1489246 (0.0008) [2023-12-27 15:58:30,243][20347] Updated weights for policy 0, policy_version 1490746 (0.0006) [2023-12-27 15:58:30,298][20347] Updated weights for policy 0, policy_version 1490756 (0.0007) [2023-12-27 15:58:30,346][20347] Updated weights for policy 0, policy_version 1490766 (0.0007) [2023-12-27 15:58:30,397][20347] Updated weights for policy 0, policy_version 1490776 (0.0007) [2023-12-27 15:58:30,829][20381] Updated weights for policy 1, policy_version 1489256 (0.0010) [2023-12-27 15:58:30,889][20381] Updated weights for policy 1, policy_version 1489266 (0.0010) [2023-12-27 15:58:30,947][20381] Updated weights for policy 1, policy_version 1489276 (0.0011) [2023-12-27 15:58:31,149][20347] Updated weights for policy 0, policy_version 1490786 (0.0008) [2023-12-27 15:58:31,209][20347] Updated weights for policy 0, policy_version 1490796 (0.0007) [2023-12-27 15:58:31,277][20347] Updated weights for policy 0, policy_version 1490806 (0.0008) [2023-12-27 15:58:31,677][20381] Updated weights for policy 1, policy_version 1489286 (0.0009) [2023-12-27 15:58:31,742][20381] Updated weights for policy 1, policy_version 1489296 (0.0008) [2023-12-27 15:58:31,803][20381] Updated weights for policy 1, policy_version 1489306 (0.0008) [2023-12-27 15:58:31,973][20347] Updated weights for policy 0, policy_version 1490816 (0.0008) [2023-12-27 15:58:32,034][20347] Updated weights for policy 0, policy_version 1490826 (0.0007) [2023-12-27 15:58:32,091][20347] Updated weights for policy 0, policy_version 1490836 (0.0008) [2023-12-27 15:58:32,362][19170] Fps is (10 sec: 21303.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 763027456. Throughput: 0: 10344.8, 1: 10216.0. Samples: 763039160. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:32,363][19170] Avg episode reward: [(0, '961.010'), (1, '1000.000')] [2023-12-27 15:58:32,418][20381] Updated weights for policy 1, policy_version 1489316 (0.0009) [2023-12-27 15:58:32,476][20381] Updated weights for policy 1, policy_version 1489326 (0.0007) [2023-12-27 15:58:32,529][20381] Updated weights for policy 1, policy_version 1489336 (0.0007) [2023-12-27 15:58:32,783][20347] Updated weights for policy 0, policy_version 1490846 (0.0006) [2023-12-27 15:58:32,839][20347] Updated weights for policy 0, policy_version 1490856 (0.0006) [2023-12-27 15:58:32,887][20347] Updated weights for policy 0, policy_version 1490866 (0.0005) [2023-12-27 15:58:33,253][20381] Updated weights for policy 1, policy_version 1489346 (0.0007) [2023-12-27 15:58:33,307][20381] Updated weights for policy 1, policy_version 1489356 (0.0005) [2023-12-27 15:58:33,354][20381] Updated weights for policy 1, policy_version 1489366 (0.0007) [2023-12-27 15:58:33,403][20381] Updated weights for policy 1, policy_version 1489376 (0.0006) [2023-12-27 15:58:33,455][20347] Updated weights for policy 0, policy_version 1490876 (0.0007) [2023-12-27 15:58:33,506][20347] Updated weights for policy 0, policy_version 1490886 (0.0010) [2023-12-27 15:58:33,560][20347] Updated weights for policy 0, policy_version 1490896 (0.0010) [2023-12-27 15:58:34,031][20381] Updated weights for policy 1, policy_version 1489386 (0.0006) [2023-12-27 15:58:34,088][20381] Updated weights for policy 1, policy_version 1489396 (0.0006) [2023-12-27 15:58:34,147][20381] Updated weights for policy 1, policy_version 1489406 (0.0007) [2023-12-27 15:58:34,149][20347] Updated weights for policy 0, policy_version 1490906 (0.0007) [2023-12-27 15:58:34,211][20347] Updated weights for policy 0, policy_version 1490916 (0.0011) [2023-12-27 15:58:34,271][20347] Updated weights for policy 0, policy_version 1490926 (0.0010) [2023-12-27 15:58:34,337][20347] Updated weights for policy 0, policy_version 1490936 (0.0011) [2023-12-27 15:58:34,855][20381] Updated weights for policy 1, policy_version 1489416 (0.0008) [2023-12-27 15:58:34,917][20381] Updated weights for policy 1, policy_version 1489426 (0.0008) [2023-12-27 15:58:34,966][20381] Updated weights for policy 1, policy_version 1489436 (0.0006) [2023-12-27 15:58:34,982][20347] Updated weights for policy 0, policy_version 1490946 (0.0010) [2023-12-27 15:58:35,038][20347] Updated weights for policy 0, policy_version 1490956 (0.0007) [2023-12-27 15:58:35,100][20347] Updated weights for policy 0, policy_version 1490966 (0.0006) [2023-12-27 15:58:35,601][20381] Updated weights for policy 1, policy_version 1489446 (0.0006) [2023-12-27 15:58:35,655][20347] Updated weights for policy 0, policy_version 1490976 (0.0009) [2023-12-27 15:58:35,658][20381] Updated weights for policy 1, policy_version 1489456 (0.0008) [2023-12-27 15:58:35,706][20347] Updated weights for policy 0, policy_version 1490986 (0.0010) [2023-12-27 15:58:35,719][20381] Updated weights for policy 1, policy_version 1489466 (0.0010) [2023-12-27 15:58:35,758][20347] Updated weights for policy 0, policy_version 1490996 (0.0007) [2023-12-27 15:58:36,413][20381] Updated weights for policy 1, policy_version 1489476 (0.0009) [2023-12-27 15:58:36,458][20347] Updated weights for policy 0, policy_version 1491006 (0.0006) [2023-12-27 15:58:36,472][20381] Updated weights for policy 1, policy_version 1489486 (0.0006) [2023-12-27 15:58:36,516][20347] Updated weights for policy 0, policy_version 1491016 (0.0006) [2023-12-27 15:58:36,533][20381] Updated weights for policy 1, policy_version 1489496 (0.0007) [2023-12-27 15:58:36,573][20347] Updated weights for policy 0, policy_version 1491026 (0.0006) [2023-12-27 15:58:37,215][20347] Updated weights for policy 0, policy_version 1491036 (0.0008) [2023-12-27 15:58:37,246][20381] Updated weights for policy 1, policy_version 1489506 (0.0007) [2023-12-27 15:58:37,272][20347] Updated weights for policy 0, policy_version 1491046 (0.0006) [2023-12-27 15:58:37,296][20381] Updated weights for policy 1, policy_version 1489516 (0.0006) [2023-12-27 15:58:37,321][20347] Updated weights for policy 0, policy_version 1491056 (0.0005) [2023-12-27 15:58:37,353][20381] Updated weights for policy 1, policy_version 1489526 (0.0007) [2023-12-27 15:58:37,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 763133952. Throughput: 0: 10394.8, 1: 10242.4. Samples: 763102540. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:37,363][19170] Avg episode reward: [(0, '960.840'), (1, '1000.000')] [2023-12-27 15:58:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001491064_381788160.pth... [2023-12-27 15:58:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001489848_381476864.pth [2023-12-27 15:58:37,409][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001489536_381353984.pth... [2023-12-27 15:58:37,411][20381] Updated weights for policy 1, policy_version 1489536 (0.0007) [2023-12-27 15:58:37,413][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001488320_381042688.pth [2023-12-27 15:58:38,075][20347] Updated weights for policy 0, policy_version 1491066 (0.0006) [2023-12-27 15:58:38,092][20381] Updated weights for policy 1, policy_version 1489546 (0.0007) [2023-12-27 15:58:38,136][20347] Updated weights for policy 0, policy_version 1491076 (0.0006) [2023-12-27 15:58:38,149][20381] Updated weights for policy 1, policy_version 1489556 (0.0006) [2023-12-27 15:58:38,191][20347] Updated weights for policy 0, policy_version 1491086 (0.0006) [2023-12-27 15:58:38,204][20381] Updated weights for policy 1, policy_version 1489566 (0.0006) [2023-12-27 15:58:38,241][20347] Updated weights for policy 0, policy_version 1491096 (0.0006) [2023-12-27 15:58:38,930][20347] Updated weights for policy 0, policy_version 1491106 (0.0005) [2023-12-27 15:58:38,936][20381] Updated weights for policy 1, policy_version 1489576 (0.0007) [2023-12-27 15:58:38,975][20347] Updated weights for policy 0, policy_version 1491116 (0.0008) [2023-12-27 15:58:38,991][20381] Updated weights for policy 1, policy_version 1489586 (0.0007) [2023-12-27 15:58:39,020][20347] Updated weights for policy 0, policy_version 1491126 (0.0005) [2023-12-27 15:58:39,037][20381] Updated weights for policy 1, policy_version 1489596 (0.0006) [2023-12-27 15:58:39,706][20347] Updated weights for policy 0, policy_version 1491136 (0.0007) [2023-12-27 15:58:39,766][20347] Updated weights for policy 0, policy_version 1491146 (0.0008) [2023-12-27 15:58:39,794][20381] Updated weights for policy 1, policy_version 1489606 (0.0007) [2023-12-27 15:58:39,827][20347] Updated weights for policy 0, policy_version 1491156 (0.0008) [2023-12-27 15:58:39,859][20381] Updated weights for policy 1, policy_version 1489616 (0.0009) [2023-12-27 15:58:39,925][20381] Updated weights for policy 1, policy_version 1489626 (0.0009) [2023-12-27 15:58:40,542][20347] Updated weights for policy 0, policy_version 1491166 (0.0007) [2023-12-27 15:58:40,568][20381] Updated weights for policy 1, policy_version 1489636 (0.0010) [2023-12-27 15:58:40,596][20347] Updated weights for policy 0, policy_version 1491176 (0.0006) [2023-12-27 15:58:40,624][20381] Updated weights for policy 1, policy_version 1489646 (0.0010) [2023-12-27 15:58:40,649][20347] Updated weights for policy 0, policy_version 1491186 (0.0005) [2023-12-27 15:58:40,681][20381] Updated weights for policy 1, policy_version 1489656 (0.0011) [2023-12-27 15:58:41,385][20347] Updated weights for policy 0, policy_version 1491196 (0.0007) [2023-12-27 15:58:41,433][20381] Updated weights for policy 1, policy_version 1489666 (0.0011) [2023-12-27 15:58:41,450][20347] Updated weights for policy 0, policy_version 1491206 (0.0008) [2023-12-27 15:58:41,482][20381] Updated weights for policy 1, policy_version 1489676 (0.0007) [2023-12-27 15:58:41,507][20347] Updated weights for policy 0, policy_version 1491216 (0.0007) [2023-12-27 15:58:41,532][20381] Updated weights for policy 1, policy_version 1489686 (0.0008) [2023-12-27 15:58:41,582][20381] Updated weights for policy 1, policy_version 1489696 (0.0008) [2023-12-27 15:58:42,259][20347] Updated weights for policy 0, policy_version 1491226 (0.0006) [2023-12-27 15:58:42,305][20381] Updated weights for policy 1, policy_version 1489706 (0.0007) [2023-12-27 15:58:42,319][20347] Updated weights for policy 0, policy_version 1491236 (0.0006) [2023-12-27 15:58:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 763224064. Throughput: 0: 10353.1, 1: 10274.0. Samples: 763224084. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:42,363][19170] Avg episode reward: [(0, '912.340'), (1, '1000.000')] [2023-12-27 15:58:42,367][20381] Updated weights for policy 1, policy_version 1489716 (0.0008) [2023-12-27 15:58:42,387][20347] Updated weights for policy 0, policy_version 1491246 (0.0007) [2023-12-27 15:58:42,426][20381] Updated weights for policy 1, policy_version 1489726 (0.0009) [2023-12-27 15:58:42,446][20347] Updated weights for policy 0, policy_version 1491256 (0.0007) [2023-12-27 15:58:43,093][20347] Updated weights for policy 0, policy_version 1491266 (0.0006) [2023-12-27 15:58:43,096][20381] Updated weights for policy 1, policy_version 1489736 (0.0007) [2023-12-27 15:58:43,147][20347] Updated weights for policy 0, policy_version 1491276 (0.0006) [2023-12-27 15:58:43,161][20381] Updated weights for policy 1, policy_version 1489746 (0.0006) [2023-12-27 15:58:43,201][20347] Updated weights for policy 0, policy_version 1491286 (0.0008) [2023-12-27 15:58:43,220][20381] Updated weights for policy 1, policy_version 1489756 (0.0006) [2023-12-27 15:58:43,774][20347] Updated weights for policy 0, policy_version 1491296 (0.0006) [2023-12-27 15:58:43,835][20347] Updated weights for policy 0, policy_version 1491306 (0.0005) [2023-12-27 15:58:43,882][20381] Updated weights for policy 1, policy_version 1489766 (0.0006) [2023-12-27 15:58:43,896][20347] Updated weights for policy 0, policy_version 1491316 (0.0007) [2023-12-27 15:58:43,939][20381] Updated weights for policy 1, policy_version 1489776 (0.0007) [2023-12-27 15:58:43,995][20381] Updated weights for policy 1, policy_version 1489786 (0.0008) [2023-12-27 15:58:44,413][20347] Updated weights for policy 0, policy_version 1491326 (0.0007) [2023-12-27 15:58:44,469][20347] Updated weights for policy 0, policy_version 1491336 (0.0005) [2023-12-27 15:58:44,525][20347] Updated weights for policy 0, policy_version 1491346 (0.0005) [2023-12-27 15:58:44,738][20381] Updated weights for policy 1, policy_version 1489796 (0.0008) [2023-12-27 15:58:44,800][20381] Updated weights for policy 1, policy_version 1489806 (0.0008) [2023-12-27 15:58:44,865][20381] Updated weights for policy 1, policy_version 1489816 (0.0007) [2023-12-27 15:58:45,164][20347] Updated weights for policy 0, policy_version 1491356 (0.0005) [2023-12-27 15:58:45,223][20347] Updated weights for policy 0, policy_version 1491366 (0.0009) [2023-12-27 15:58:45,269][20347] Updated weights for policy 0, policy_version 1491376 (0.0010) [2023-12-27 15:58:45,556][20381] Updated weights for policy 1, policy_version 1489826 (0.0006) [2023-12-27 15:58:45,611][20381] Updated weights for policy 1, policy_version 1489836 (0.0005) [2023-12-27 15:58:45,664][20381] Updated weights for policy 1, policy_version 1489846 (0.0005) [2023-12-27 15:58:45,720][20381] Updated weights for policy 1, policy_version 1489856 (0.0007) [2023-12-27 15:58:45,884][20347] Updated weights for policy 0, policy_version 1491386 (0.0009) [2023-12-27 15:58:45,942][20347] Updated weights for policy 0, policy_version 1491396 (0.0005) [2023-12-27 15:58:45,996][20347] Updated weights for policy 0, policy_version 1491406 (0.0010) [2023-12-27 15:58:46,053][20347] Updated weights for policy 0, policy_version 1491416 (0.0010) [2023-12-27 15:58:46,352][20381] Updated weights for policy 1, policy_version 1489866 (0.0007) [2023-12-27 15:58:46,406][20381] Updated weights for policy 1, policy_version 1489876 (0.0007) [2023-12-27 15:58:46,462][20381] Updated weights for policy 1, policy_version 1489886 (0.0007) [2023-12-27 15:58:46,705][20347] Updated weights for policy 0, policy_version 1491426 (0.0010) [2023-12-27 15:58:46,762][20347] Updated weights for policy 0, policy_version 1491436 (0.0010) [2023-12-27 15:58:46,814][20347] Updated weights for policy 0, policy_version 1491446 (0.0010) [2023-12-27 15:58:47,203][20381] Updated weights for policy 1, policy_version 1489896 (0.0008) [2023-12-27 15:58:47,260][20381] Updated weights for policy 1, policy_version 1489906 (0.0007) [2023-12-27 15:58:47,307][20381] Updated weights for policy 1, policy_version 1489916 (0.0007) [2023-12-27 15:58:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 763338752. Throughput: 0: 10370.9, 1: 10224.3. Samples: 763348220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:47,363][19170] Avg episode reward: [(0, '902.580'), (1, '1000.000')] [2023-12-27 15:58:47,427][20347] Updated weights for policy 0, policy_version 1491456 (0.0010) [2023-12-27 15:58:47,476][20347] Updated weights for policy 0, policy_version 1491466 (0.0010) [2023-12-27 15:58:47,520][20347] Updated weights for policy 0, policy_version 1491476 (0.0010) [2023-12-27 15:58:48,007][20381] Updated weights for policy 1, policy_version 1489926 (0.0006) [2023-12-27 15:58:48,064][20381] Updated weights for policy 1, policy_version 1489936 (0.0007) [2023-12-27 15:58:48,111][20347] Updated weights for policy 0, policy_version 1491486 (0.0005) [2023-12-27 15:58:48,121][20381] Updated weights for policy 1, policy_version 1489946 (0.0008) [2023-12-27 15:58:48,165][20347] Updated weights for policy 0, policy_version 1491496 (0.0005) [2023-12-27 15:58:48,225][20347] Updated weights for policy 0, policy_version 1491506 (0.0009) [2023-12-27 15:58:48,829][20381] Updated weights for policy 1, policy_version 1489956 (0.0007) [2023-12-27 15:58:48,837][20347] Updated weights for policy 0, policy_version 1491516 (0.0009) [2023-12-27 15:58:48,884][20381] Updated weights for policy 1, policy_version 1489966 (0.0007) [2023-12-27 15:58:48,900][20347] Updated weights for policy 0, policy_version 1491526 (0.0009) [2023-12-27 15:58:48,937][20381] Updated weights for policy 1, policy_version 1489976 (0.0005) [2023-12-27 15:58:48,957][20347] Updated weights for policy 0, policy_version 1491536 (0.0009) [2023-12-27 15:58:49,607][20381] Updated weights for policy 1, policy_version 1489986 (0.0007) [2023-12-27 15:58:49,627][20347] Updated weights for policy 0, policy_version 1491546 (0.0006) [2023-12-27 15:58:49,657][20381] Updated weights for policy 1, policy_version 1489996 (0.0011) [2023-12-27 15:58:49,680][20347] Updated weights for policy 0, policy_version 1491556 (0.0007) [2023-12-27 15:58:49,704][20381] Updated weights for policy 1, policy_version 1490006 (0.0011) [2023-12-27 15:58:49,738][20347] Updated weights for policy 0, policy_version 1491566 (0.0007) [2023-12-27 15:58:49,759][20381] Updated weights for policy 1, policy_version 1490016 (0.0011) [2023-12-27 15:58:49,797][20347] Updated weights for policy 0, policy_version 1491576 (0.0005) [2023-12-27 15:58:50,451][20381] Updated weights for policy 1, policy_version 1490026 (0.0011) [2023-12-27 15:58:50,475][20347] Updated weights for policy 0, policy_version 1491586 (0.0006) [2023-12-27 15:58:50,510][20381] Updated weights for policy 1, policy_version 1490036 (0.0011) [2023-12-27 15:58:50,532][20347] Updated weights for policy 0, policy_version 1491596 (0.0005) [2023-12-27 15:58:50,565][20381] Updated weights for policy 1, policy_version 1490046 (0.0010) [2023-12-27 15:58:50,600][20347] Updated weights for policy 0, policy_version 1491606 (0.0006) [2023-12-27 15:58:51,246][20381] Updated weights for policy 1, policy_version 1490056 (0.0007) [2023-12-27 15:58:51,276][20347] Updated weights for policy 0, policy_version 1491616 (0.0007) [2023-12-27 15:58:51,310][20381] Updated weights for policy 1, policy_version 1490066 (0.0008) [2023-12-27 15:58:51,335][20347] Updated weights for policy 0, policy_version 1491626 (0.0006) [2023-12-27 15:58:51,371][20381] Updated weights for policy 1, policy_version 1490076 (0.0007) [2023-12-27 15:58:51,400][20347] Updated weights for policy 0, policy_version 1491636 (0.0007) [2023-12-27 15:58:52,040][20347] Updated weights for policy 0, policy_version 1491646 (0.0007) [2023-12-27 15:58:52,102][20347] Updated weights for policy 0, policy_version 1491656 (0.0007) [2023-12-27 15:58:52,131][20381] Updated weights for policy 1, policy_version 1490086 (0.0007) [2023-12-27 15:58:52,162][20347] Updated weights for policy 0, policy_version 1491666 (0.0005) [2023-12-27 15:58:52,190][20381] Updated weights for policy 1, policy_version 1490096 (0.0006) [2023-12-27 15:58:52,251][20381] Updated weights for policy 1, policy_version 1490106 (0.0008) [2023-12-27 15:58:52,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 763445248. Throughput: 0: 10410.1, 1: 10186.4. Samples: 763410616. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:52,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 15:58:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001490112_381501440.pth... [2023-12-27 15:58:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001491672_381943808.pth... [2023-12-27 15:58:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001490424_381624320.pth [2023-12-27 15:58:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001488928_381198336.pth [2023-12-27 15:58:52,848][20347] Updated weights for policy 0, policy_version 1491676 (0.0006) [2023-12-27 15:58:52,899][20347] Updated weights for policy 0, policy_version 1491686 (0.0005) [2023-12-27 15:58:52,955][20347] Updated weights for policy 0, policy_version 1491696 (0.0005) [2023-12-27 15:58:52,994][20381] Updated weights for policy 1, policy_version 1490116 (0.0008) [2023-12-27 15:58:53,054][20381] Updated weights for policy 1, policy_version 1490126 (0.0009) [2023-12-27 15:58:53,118][20381] Updated weights for policy 1, policy_version 1490136 (0.0008) [2023-12-27 15:58:53,622][20347] Updated weights for policy 0, policy_version 1491706 (0.0009) [2023-12-27 15:58:53,671][20347] Updated weights for policy 0, policy_version 1491716 (0.0010) [2023-12-27 15:58:53,725][20347] Updated weights for policy 0, policy_version 1491726 (0.0010) [2023-12-27 15:58:53,741][20381] Updated weights for policy 1, policy_version 1490146 (0.0007) [2023-12-27 15:58:53,786][20347] Updated weights for policy 0, policy_version 1491736 (0.0007) [2023-12-27 15:58:53,799][20381] Updated weights for policy 1, policy_version 1490156 (0.0008) [2023-12-27 15:58:53,852][20381] Updated weights for policy 1, policy_version 1490166 (0.0008) [2023-12-27 15:58:53,903][20381] Updated weights for policy 1, policy_version 1490176 (0.0007) [2023-12-27 15:58:54,333][20347] Updated weights for policy 0, policy_version 1491746 (0.0005) [2023-12-27 15:58:54,385][20347] Updated weights for policy 0, policy_version 1491756 (0.0005) [2023-12-27 15:58:54,445][20347] Updated weights for policy 0, policy_version 1491766 (0.0005) [2023-12-27 15:58:54,622][20381] Updated weights for policy 1, policy_version 1490186 (0.0007) [2023-12-27 15:58:54,680][20381] Updated weights for policy 1, policy_version 1490196 (0.0005) [2023-12-27 15:58:54,742][20381] Updated weights for policy 1, policy_version 1490206 (0.0007) [2023-12-27 15:58:55,045][20347] Updated weights for policy 0, policy_version 1491776 (0.0006) [2023-12-27 15:58:55,096][20347] Updated weights for policy 0, policy_version 1491786 (0.0008) [2023-12-27 15:58:55,146][20347] Updated weights for policy 0, policy_version 1491796 (0.0010) [2023-12-27 15:58:55,359][20381] Updated weights for policy 1, policy_version 1490216 (0.0006) [2023-12-27 15:58:55,418][20381] Updated weights for policy 1, policy_version 1490226 (0.0007) [2023-12-27 15:58:55,467][20381] Updated weights for policy 1, policy_version 1490236 (0.0007) [2023-12-27 15:58:55,813][20347] Updated weights for policy 0, policy_version 1491806 (0.0010) [2023-12-27 15:58:55,870][20347] Updated weights for policy 0, policy_version 1491816 (0.0010) [2023-12-27 15:58:55,927][20347] Updated weights for policy 0, policy_version 1491826 (0.0010) [2023-12-27 15:58:56,080][20381] Updated weights for policy 1, policy_version 1490246 (0.0007) [2023-12-27 15:58:56,135][20381] Updated weights for policy 1, policy_version 1490256 (0.0008) [2023-12-27 15:58:56,193][20381] Updated weights for policy 1, policy_version 1490266 (0.0007) [2023-12-27 15:58:56,579][20347] Updated weights for policy 0, policy_version 1491836 (0.0010) [2023-12-27 15:58:56,630][20347] Updated weights for policy 0, policy_version 1491846 (0.0010) [2023-12-27 15:58:56,680][20347] Updated weights for policy 0, policy_version 1491856 (0.0010) [2023-12-27 15:58:56,860][20381] Updated weights for policy 1, policy_version 1490276 (0.0007) [2023-12-27 15:58:56,913][20381] Updated weights for policy 1, policy_version 1490286 (0.0007) [2023-12-27 15:58:56,963][20381] Updated weights for policy 1, policy_version 1490296 (0.0007) [2023-12-27 15:58:57,361][20347] Updated weights for policy 0, policy_version 1491866 (0.0010) [2023-12-27 15:58:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 763543552. Throughput: 0: 10480.7, 1: 10161.4. Samples: 763535516. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:58:57,363][19170] Avg episode reward: [(0, '980.500'), (1, '980.520')] [2023-12-27 15:58:57,421][20347] Updated weights for policy 0, policy_version 1491876 (0.0010) [2023-12-27 15:58:57,478][20347] Updated weights for policy 0, policy_version 1491886 (0.0010) [2023-12-27 15:58:57,537][20347] Updated weights for policy 0, policy_version 1491896 (0.0010) [2023-12-27 15:58:57,648][20381] Updated weights for policy 1, policy_version 1490306 (0.0008) [2023-12-27 15:58:57,697][20381] Updated weights for policy 1, policy_version 1490316 (0.0010) [2023-12-27 15:58:57,777][20292] KL-divergence is very high: 222.9003 [2023-12-27 15:58:57,778][20381] Updated weights for policy 1, policy_version 1490326 (0.0008) [2023-12-27 15:58:57,820][20292] KL-divergence is very high: 121.7149 [2023-12-27 15:58:57,833][20381] Updated weights for policy 1, policy_version 1490336 (0.0008) [2023-12-27 15:58:58,147][20347] Updated weights for policy 0, policy_version 1491906 (0.0010) [2023-12-27 15:58:58,208][20347] Updated weights for policy 0, policy_version 1491916 (0.0006) [2023-12-27 15:58:58,267][20347] Updated weights for policy 0, policy_version 1491926 (0.0008) [2023-12-27 15:58:58,562][20381] Updated weights for policy 1, policy_version 1490346 (0.0008) [2023-12-27 15:58:58,622][20381] Updated weights for policy 1, policy_version 1490356 (0.0007) [2023-12-27 15:58:58,682][20381] Updated weights for policy 1, policy_version 1490366 (0.0008) [2023-12-27 15:58:59,091][20347] Updated weights for policy 0, policy_version 1491936 (0.0008) [2023-12-27 15:58:59,145][20347] Updated weights for policy 0, policy_version 1491946 (0.0008) [2023-12-27 15:58:59,199][20347] Updated weights for policy 0, policy_version 1491956 (0.0008) [2023-12-27 15:58:59,488][20381] Updated weights for policy 1, policy_version 1490376 (0.0007) [2023-12-27 15:58:59,540][20381] Updated weights for policy 1, policy_version 1490386 (0.0008) [2023-12-27 15:58:59,594][20381] Updated weights for policy 1, policy_version 1490396 (0.0009) [2023-12-27 15:58:59,913][20347] Updated weights for policy 0, policy_version 1491966 (0.0007) [2023-12-27 15:58:59,978][20347] Updated weights for policy 0, policy_version 1491976 (0.0007) [2023-12-27 15:59:00,044][20347] Updated weights for policy 0, policy_version 1491986 (0.0007) [2023-12-27 15:59:00,263][20381] Updated weights for policy 1, policy_version 1490406 (0.0007) [2023-12-27 15:59:00,307][20381] Updated weights for policy 1, policy_version 1490416 (0.0007) [2023-12-27 15:59:00,364][20381] Updated weights for policy 1, policy_version 1490426 (0.0008) [2023-12-27 15:59:00,693][20347] Updated weights for policy 0, policy_version 1491996 (0.0007) [2023-12-27 15:59:00,751][20347] Updated weights for policy 0, policy_version 1492006 (0.0009) [2023-12-27 15:59:00,805][20347] Updated weights for policy 0, policy_version 1492016 (0.0008) [2023-12-27 15:59:00,927][20381] Updated weights for policy 1, policy_version 1490436 (0.0007) [2023-12-27 15:59:00,974][20381] Updated weights for policy 1, policy_version 1490446 (0.0007) [2023-12-27 15:59:01,021][20381] Updated weights for policy 1, policy_version 1490456 (0.0007) [2023-12-27 15:59:01,530][20347] Updated weights for policy 0, policy_version 1492026 (0.0007) [2023-12-27 15:59:01,586][20347] Updated weights for policy 0, policy_version 1492036 (0.0011) [2023-12-27 15:59:01,650][20347] Updated weights for policy 0, policy_version 1492046 (0.0010) [2023-12-27 15:59:01,705][20347] Updated weights for policy 0, policy_version 1492056 (0.0010) [2023-12-27 15:59:01,722][20381] Updated weights for policy 1, policy_version 1490466 (0.0008) [2023-12-27 15:59:01,788][20381] Updated weights for policy 1, policy_version 1490476 (0.0008) [2023-12-27 15:59:01,843][20381] Updated weights for policy 1, policy_version 1490486 (0.0007) [2023-12-27 15:59:01,901][20381] Updated weights for policy 1, policy_version 1490496 (0.0007) [2023-12-27 15:59:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 763641856. Throughput: 0: 10440.1, 1: 10122.7. Samples: 763655828. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '980.520')] [2023-12-27 15:59:02,406][20347] Updated weights for policy 0, policy_version 1492066 (0.0009) [2023-12-27 15:59:02,466][20347] Updated weights for policy 0, policy_version 1492076 (0.0011) [2023-12-27 15:59:02,526][20347] Updated weights for policy 0, policy_version 1492086 (0.0011) [2023-12-27 15:59:02,611][20381] Updated weights for policy 1, policy_version 1490506 (0.0008) [2023-12-27 15:59:02,668][20381] Updated weights for policy 1, policy_version 1490516 (0.0005) [2023-12-27 15:59:02,729][20381] Updated weights for policy 1, policy_version 1490526 (0.0007) [2023-12-27 15:59:03,125][20347] Updated weights for policy 0, policy_version 1492096 (0.0006) [2023-12-27 15:59:03,179][20347] Updated weights for policy 0, policy_version 1492106 (0.0005) [2023-12-27 15:59:03,226][20347] Updated weights for policy 0, policy_version 1492116 (0.0005) [2023-12-27 15:59:03,400][20381] Updated weights for policy 1, policy_version 1490536 (0.0007) [2023-12-27 15:59:03,449][20381] Updated weights for policy 1, policy_version 1490546 (0.0010) [2023-12-27 15:59:03,505][20381] Updated weights for policy 1, policy_version 1490556 (0.0010) [2023-12-27 15:59:03,872][20347] Updated weights for policy 0, policy_version 1492126 (0.0006) [2023-12-27 15:59:03,922][20347] Updated weights for policy 0, policy_version 1492136 (0.0008) [2023-12-27 15:59:03,977][20347] Updated weights for policy 0, policy_version 1492146 (0.0007) [2023-12-27 15:59:04,218][20381] Updated weights for policy 1, policy_version 1490566 (0.0011) [2023-12-27 15:59:04,280][20381] Updated weights for policy 1, policy_version 1490576 (0.0010) [2023-12-27 15:59:04,343][20381] Updated weights for policy 1, policy_version 1490586 (0.0008) [2023-12-27 15:59:04,660][20347] Updated weights for policy 0, policy_version 1492156 (0.0007) [2023-12-27 15:59:04,716][20347] Updated weights for policy 0, policy_version 1492166 (0.0007) [2023-12-27 15:59:04,772][20347] Updated weights for policy 0, policy_version 1492176 (0.0007) [2023-12-27 15:59:05,031][20381] Updated weights for policy 1, policy_version 1490596 (0.0007) [2023-12-27 15:59:05,088][20381] Updated weights for policy 1, policy_version 1490606 (0.0008) [2023-12-27 15:59:05,147][20381] Updated weights for policy 1, policy_version 1490616 (0.0006) [2023-12-27 15:59:05,407][20347] Updated weights for policy 0, policy_version 1492186 (0.0007) [2023-12-27 15:59:05,459][20347] Updated weights for policy 0, policy_version 1492196 (0.0007) [2023-12-27 15:59:05,515][20347] Updated weights for policy 0, policy_version 1492206 (0.0007) [2023-12-27 15:59:05,574][20347] Updated weights for policy 0, policy_version 1492216 (0.0008) [2023-12-27 15:59:05,840][20381] Updated weights for policy 1, policy_version 1490626 (0.0008) [2023-12-27 15:59:05,899][20381] Updated weights for policy 1, policy_version 1490636 (0.0007) [2023-12-27 15:59:05,958][20381] Updated weights for policy 1, policy_version 1490646 (0.0008) [2023-12-27 15:59:06,026][20381] Updated weights for policy 1, policy_version 1490656 (0.0009) [2023-12-27 15:59:06,181][20347] Updated weights for policy 0, policy_version 1492226 (0.0008) [2023-12-27 15:59:06,243][20347] Updated weights for policy 0, policy_version 1492236 (0.0008) [2023-12-27 15:59:06,299][20347] Updated weights for policy 0, policy_version 1492246 (0.0008) [2023-12-27 15:59:06,760][20381] Updated weights for policy 1, policy_version 1490666 (0.0008) [2023-12-27 15:59:06,826][20381] Updated weights for policy 1, policy_version 1490676 (0.0008) [2023-12-27 15:59:06,879][20381] Updated weights for policy 1, policy_version 1490686 (0.0008) [2023-12-27 15:59:07,014][20347] Updated weights for policy 0, policy_version 1492256 (0.0006) [2023-12-27 15:59:07,073][20347] Updated weights for policy 0, policy_version 1492266 (0.0008) [2023-12-27 15:59:07,119][20347] Updated weights for policy 0, policy_version 1492276 (0.0006) [2023-12-27 15:59:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 763748352. Throughput: 0: 10458.3, 1: 10105.9. Samples: 763717088. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:07,363][19170] Avg episode reward: [(0, '970.860'), (1, '1000.000')] [2023-12-27 15:59:07,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001492280_382099456.pth... [2023-12-27 15:59:07,366][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001490688_381648896.pth... [2023-12-27 15:59:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001491064_381788160.pth [2023-12-27 15:59:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001489536_381353984.pth [2023-12-27 15:59:07,584][20381] Updated weights for policy 1, policy_version 1490696 (0.0007) [2023-12-27 15:59:07,641][20381] Updated weights for policy 1, policy_version 1490706 (0.0007) [2023-12-27 15:59:07,696][20381] Updated weights for policy 1, policy_version 1490716 (0.0006) [2023-12-27 15:59:07,758][20347] Updated weights for policy 0, policy_version 1492286 (0.0006) [2023-12-27 15:59:07,822][20347] Updated weights for policy 0, policy_version 1492296 (0.0011) [2023-12-27 15:59:07,879][20347] Updated weights for policy 0, policy_version 1492306 (0.0010) [2023-12-27 15:59:08,412][20381] Updated weights for policy 1, policy_version 1490726 (0.0007) [2023-12-27 15:59:08,472][20381] Updated weights for policy 1, policy_version 1490736 (0.0005) [2023-12-27 15:59:08,526][20381] Updated weights for policy 1, policy_version 1490746 (0.0006) [2023-12-27 15:59:08,553][20347] Updated weights for policy 0, policy_version 1492316 (0.0011) [2023-12-27 15:59:08,612][20347] Updated weights for policy 0, policy_version 1492326 (0.0010) [2023-12-27 15:59:08,673][20347] Updated weights for policy 0, policy_version 1492336 (0.0005) [2023-12-27 15:59:09,115][20381] Updated weights for policy 1, policy_version 1490756 (0.0007) [2023-12-27 15:59:09,167][20381] Updated weights for policy 1, policy_version 1490766 (0.0005) [2023-12-27 15:59:09,220][20381] Updated weights for policy 1, policy_version 1490776 (0.0008) [2023-12-27 15:59:09,354][20347] Updated weights for policy 0, policy_version 1492346 (0.0007) [2023-12-27 15:59:09,420][20347] Updated weights for policy 0, policy_version 1492356 (0.0008) [2023-12-27 15:59:09,485][20347] Updated weights for policy 0, policy_version 1492366 (0.0007) [2023-12-27 15:59:09,553][20347] Updated weights for policy 0, policy_version 1492376 (0.0007) [2023-12-27 15:59:09,933][20381] Updated weights for policy 1, policy_version 1490786 (0.0008) [2023-12-27 15:59:09,993][20381] Updated weights for policy 1, policy_version 1490796 (0.0008) [2023-12-27 15:59:10,057][20381] Updated weights for policy 1, policy_version 1490806 (0.0009) [2023-12-27 15:59:10,118][20381] Updated weights for policy 1, policy_version 1490816 (0.0008) [2023-12-27 15:59:10,196][20347] Updated weights for policy 0, policy_version 1492386 (0.0008) [2023-12-27 15:59:10,259][20347] Updated weights for policy 0, policy_version 1492396 (0.0008) [2023-12-27 15:59:10,317][20347] Updated weights for policy 0, policy_version 1492406 (0.0008) [2023-12-27 15:59:10,803][20381] Updated weights for policy 1, policy_version 1490826 (0.0010) [2023-12-27 15:59:10,856][20381] Updated weights for policy 1, policy_version 1490836 (0.0011) [2023-12-27 15:59:10,914][20381] Updated weights for policy 1, policy_version 1490846 (0.0006) [2023-12-27 15:59:10,959][20347] Updated weights for policy 0, policy_version 1492416 (0.0009) [2023-12-27 15:59:11,014][20347] Updated weights for policy 0, policy_version 1492426 (0.0008) [2023-12-27 15:59:11,078][20347] Updated weights for policy 0, policy_version 1492436 (0.0008) [2023-12-27 15:59:11,561][20381] Updated weights for policy 1, policy_version 1490856 (0.0006) [2023-12-27 15:59:11,630][20381] Updated weights for policy 1, policy_version 1490866 (0.0007) [2023-12-27 15:59:11,683][20381] Updated weights for policy 1, policy_version 1490876 (0.0007) [2023-12-27 15:59:11,856][20347] Updated weights for policy 0, policy_version 1492446 (0.0008) [2023-12-27 15:59:11,919][20347] Updated weights for policy 0, policy_version 1492456 (0.0007) [2023-12-27 15:59:11,971][20347] Updated weights for policy 0, policy_version 1492466 (0.0007) [2023-12-27 15:59:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 763846656. Throughput: 0: 10478.4, 1: 10081.0. Samples: 763840148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:12,363][19170] Avg episode reward: [(0, '941.600'), (1, '1000.000')] [2023-12-27 15:59:12,457][20381] Updated weights for policy 1, policy_version 1490886 (0.0008) [2023-12-27 15:59:12,518][20381] Updated weights for policy 1, policy_version 1490896 (0.0006) [2023-12-27 15:59:12,586][20381] Updated weights for policy 1, policy_version 1490906 (0.0009) [2023-12-27 15:59:12,699][20347] Updated weights for policy 0, policy_version 1492476 (0.0008) [2023-12-27 15:59:12,757][20347] Updated weights for policy 0, policy_version 1492486 (0.0005) [2023-12-27 15:59:12,811][20347] Updated weights for policy 0, policy_version 1492496 (0.0005) [2023-12-27 15:59:13,219][20381] Updated weights for policy 1, policy_version 1490916 (0.0009) [2023-12-27 15:59:13,270][20381] Updated weights for policy 1, policy_version 1490926 (0.0010) [2023-12-27 15:59:13,321][20381] Updated weights for policy 1, policy_version 1490936 (0.0009) [2023-12-27 15:59:13,380][20347] Updated weights for policy 0, policy_version 1492506 (0.0005) [2023-12-27 15:59:13,436][20347] Updated weights for policy 0, policy_version 1492516 (0.0005) [2023-12-27 15:59:13,489][20347] Updated weights for policy 0, policy_version 1492526 (0.0005) [2023-12-27 15:59:13,538][20347] Updated weights for policy 0, policy_version 1492536 (0.0006) [2023-12-27 15:59:13,936][20381] Updated weights for policy 1, policy_version 1490946 (0.0006) [2023-12-27 15:59:13,984][20381] Updated weights for policy 1, policy_version 1490956 (0.0005) [2023-12-27 15:59:14,042][20381] Updated weights for policy 1, policy_version 1490966 (0.0005) [2023-12-27 15:59:14,103][20381] Updated weights for policy 1, policy_version 1490976 (0.0008) [2023-12-27 15:59:14,149][20347] Updated weights for policy 0, policy_version 1492546 (0.0010) [2023-12-27 15:59:14,203][20347] Updated weights for policy 0, policy_version 1492556 (0.0010) [2023-12-27 15:59:14,266][20347] Updated weights for policy 0, policy_version 1492566 (0.0008) [2023-12-27 15:59:14,690][20381] Updated weights for policy 1, policy_version 1490986 (0.0010) [2023-12-27 15:59:14,745][20381] Updated weights for policy 1, policy_version 1490996 (0.0010) [2023-12-27 15:59:14,808][20381] Updated weights for policy 1, policy_version 1491006 (0.0008) [2023-12-27 15:59:14,884][20347] Updated weights for policy 0, policy_version 1492576 (0.0008) [2023-12-27 15:59:14,949][20347] Updated weights for policy 0, policy_version 1492586 (0.0008) [2023-12-27 15:59:15,012][20347] Updated weights for policy 0, policy_version 1492596 (0.0007) [2023-12-27 15:59:15,441][20381] Updated weights for policy 1, policy_version 1491016 (0.0006) [2023-12-27 15:59:15,495][20381] Updated weights for policy 1, policy_version 1491026 (0.0005) [2023-12-27 15:59:15,543][20381] Updated weights for policy 1, policy_version 1491036 (0.0010) [2023-12-27 15:59:15,690][20347] Updated weights for policy 0, policy_version 1492606 (0.0006) [2023-12-27 15:59:15,739][20347] Updated weights for policy 0, policy_version 1492616 (0.0007) [2023-12-27 15:59:15,793][20347] Updated weights for policy 0, policy_version 1492626 (0.0007) [2023-12-27 15:59:16,162][20381] Updated weights for policy 1, policy_version 1491046 (0.0007) [2023-12-27 15:59:16,219][20381] Updated weights for policy 1, policy_version 1491056 (0.0005) [2023-12-27 15:59:16,277][20381] Updated weights for policy 1, policy_version 1491066 (0.0005) [2023-12-27 15:59:16,474][20347] Updated weights for policy 0, policy_version 1492636 (0.0008) [2023-12-27 15:59:16,535][20347] Updated weights for policy 0, policy_version 1492646 (0.0008) [2023-12-27 15:59:16,595][20347] Updated weights for policy 0, policy_version 1492656 (0.0007) [2023-12-27 15:59:16,869][20381] Updated weights for policy 1, policy_version 1491076 (0.0007) [2023-12-27 15:59:16,920][20381] Updated weights for policy 1, policy_version 1491086 (0.0007) [2023-12-27 15:59:16,969][20381] Updated weights for policy 1, policy_version 1491096 (0.0007) [2023-12-27 15:59:17,300][20347] Updated weights for policy 0, policy_version 1492666 (0.0008) [2023-12-27 15:59:17,348][20347] Updated weights for policy 0, policy_version 1492676 (0.0007) [2023-12-27 15:59:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 763953152. Throughput: 0: 10473.6, 1: 10126.8. Samples: 763966184. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:17,364][19170] Avg episode reward: [(0, '951.290'), (1, '1000.000')] [2023-12-27 15:59:17,403][20347] Updated weights for policy 0, policy_version 1492686 (0.0007) [2023-12-27 15:59:17,460][20347] Updated weights for policy 0, policy_version 1492696 (0.0008) [2023-12-27 15:59:17,664][20381] Updated weights for policy 1, policy_version 1491106 (0.0007) [2023-12-27 15:59:17,726][20381] Updated weights for policy 1, policy_version 1491116 (0.0007) [2023-12-27 15:59:17,785][20381] Updated weights for policy 1, policy_version 1491126 (0.0008) [2023-12-27 15:59:17,851][20381] Updated weights for policy 1, policy_version 1491136 (0.0008) [2023-12-27 15:59:18,130][20347] Updated weights for policy 0, policy_version 1492706 (0.0010) [2023-12-27 15:59:18,194][20347] Updated weights for policy 0, policy_version 1492716 (0.0010) [2023-12-27 15:59:18,259][20347] Updated weights for policy 0, policy_version 1492726 (0.0010) [2023-12-27 15:59:18,458][20381] Updated weights for policy 1, policy_version 1491146 (0.0008) [2023-12-27 15:59:18,507][20381] Updated weights for policy 1, policy_version 1491156 (0.0007) [2023-12-27 15:59:18,569][20381] Updated weights for policy 1, policy_version 1491166 (0.0008) [2023-12-27 15:59:18,947][20347] Updated weights for policy 0, policy_version 1492736 (0.0007) [2023-12-27 15:59:19,005][20347] Updated weights for policy 0, policy_version 1492746 (0.0010) [2023-12-27 15:59:19,066][20347] Updated weights for policy 0, policy_version 1492756 (0.0010) [2023-12-27 15:59:19,218][20381] Updated weights for policy 1, policy_version 1491176 (0.0008) [2023-12-27 15:59:19,287][20381] Updated weights for policy 1, policy_version 1491186 (0.0007) [2023-12-27 15:59:19,348][20381] Updated weights for policy 1, policy_version 1491196 (0.0007) [2023-12-27 15:59:19,731][20347] Updated weights for policy 0, policy_version 1492766 (0.0007) [2023-12-27 15:59:19,789][20347] Updated weights for policy 0, policy_version 1492776 (0.0007) [2023-12-27 15:59:19,857][20347] Updated weights for policy 0, policy_version 1492786 (0.0009) [2023-12-27 15:59:19,995][20381] Updated weights for policy 1, policy_version 1491206 (0.0007) [2023-12-27 15:59:20,056][20381] Updated weights for policy 1, policy_version 1491216 (0.0007) [2023-12-27 15:59:20,119][20381] Updated weights for policy 1, policy_version 1491226 (0.0006) [2023-12-27 15:59:20,563][20347] Updated weights for policy 0, policy_version 1492796 (0.0007) [2023-12-27 15:59:20,627][20347] Updated weights for policy 0, policy_version 1492806 (0.0007) [2023-12-27 15:59:20,689][20347] Updated weights for policy 0, policy_version 1492816 (0.0007) [2023-12-27 15:59:20,835][20381] Updated weights for policy 1, policy_version 1491236 (0.0007) [2023-12-27 15:59:20,906][20381] Updated weights for policy 1, policy_version 1491246 (0.0007) [2023-12-27 15:59:20,972][20381] Updated weights for policy 1, policy_version 1491256 (0.0007) [2023-12-27 15:59:21,443][20347] Updated weights for policy 0, policy_version 1492826 (0.0007) [2023-12-27 15:59:21,508][20347] Updated weights for policy 0, policy_version 1492836 (0.0008) [2023-12-27 15:59:21,557][20347] Updated weights for policy 0, policy_version 1492846 (0.0007) [2023-12-27 15:59:21,607][20347] Updated weights for policy 0, policy_version 1492856 (0.0008) [2023-12-27 15:59:21,701][20381] Updated weights for policy 1, policy_version 1491266 (0.0007) [2023-12-27 15:59:21,770][20381] Updated weights for policy 1, policy_version 1491276 (0.0008) [2023-12-27 15:59:21,830][20381] Updated weights for policy 1, policy_version 1491286 (0.0007) [2023-12-27 15:59:21,887][20381] Updated weights for policy 1, policy_version 1491296 (0.0007) [2023-12-27 15:59:22,262][20347] Updated weights for policy 0, policy_version 1492866 (0.0008) [2023-12-27 15:59:22,317][20347] Updated weights for policy 0, policy_version 1492876 (0.0006) [2023-12-27 15:59:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20617.2, 300 sec: 20549.4). Total num frames: 764051456. Throughput: 0: 10416.9, 1: 10147.1. Samples: 764027924. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:22,363][19170] Avg episode reward: [(0, '941.400'), (1, '1000.000')] [2023-12-27 15:59:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001491296_381804544.pth... [2023-12-27 15:59:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001490112_381501440.pth [2023-12-27 15:59:22,383][20347] Updated weights for policy 0, policy_version 1492886 (0.0008) [2023-12-27 15:59:22,397][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001492888_382255104.pth... [2023-12-27 15:59:22,402][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001491672_381943808.pth [2023-12-27 15:59:22,634][20381] Updated weights for policy 1, policy_version 1491306 (0.0011) [2023-12-27 15:59:22,694][20381] Updated weights for policy 1, policy_version 1491316 (0.0009) [2023-12-27 15:59:22,745][20381] Updated weights for policy 1, policy_version 1491326 (0.0008) [2023-12-27 15:59:23,124][20347] Updated weights for policy 0, policy_version 1492896 (0.0007) [2023-12-27 15:59:23,183][20347] Updated weights for policy 0, policy_version 1492906 (0.0007) [2023-12-27 15:59:23,237][20347] Updated weights for policy 0, policy_version 1492916 (0.0007) [2023-12-27 15:59:23,471][20381] Updated weights for policy 1, policy_version 1491336 (0.0010) [2023-12-27 15:59:23,520][20381] Updated weights for policy 1, policy_version 1491346 (0.0010) [2023-12-27 15:59:23,574][20381] Updated weights for policy 1, policy_version 1491356 (0.0010) [2023-12-27 15:59:23,941][20347] Updated weights for policy 0, policy_version 1492926 (0.0006) [2023-12-27 15:59:23,992][20347] Updated weights for policy 0, policy_version 1492936 (0.0005) [2023-12-27 15:59:24,047][20347] Updated weights for policy 0, policy_version 1492946 (0.0005) [2023-12-27 15:59:24,277][20381] Updated weights for policy 1, policy_version 1491366 (0.0010) [2023-12-27 15:59:24,326][20381] Updated weights for policy 1, policy_version 1491376 (0.0005) [2023-12-27 15:59:24,371][20381] Updated weights for policy 1, policy_version 1491386 (0.0005) [2023-12-27 15:59:24,725][20347] Updated weights for policy 0, policy_version 1492956 (0.0006) [2023-12-27 15:59:24,790][20347] Updated weights for policy 0, policy_version 1492966 (0.0008) [2023-12-27 15:59:24,846][20347] Updated weights for policy 0, policy_version 1492976 (0.0008) [2023-12-27 15:59:24,908][20381] Updated weights for policy 1, policy_version 1491396 (0.0005) [2023-12-27 15:59:24,969][20381] Updated weights for policy 1, policy_version 1491406 (0.0005) [2023-12-27 15:59:25,030][20381] Updated weights for policy 1, policy_version 1491416 (0.0008) [2023-12-27 15:59:25,566][20347] Updated weights for policy 0, policy_version 1492986 (0.0008) [2023-12-27 15:59:25,619][20347] Updated weights for policy 0, policy_version 1492996 (0.0007) [2023-12-27 15:59:25,666][20381] Updated weights for policy 1, policy_version 1491426 (0.0010) [2023-12-27 15:59:25,674][20347] Updated weights for policy 0, policy_version 1493006 (0.0008) [2023-12-27 15:59:25,719][20381] Updated weights for policy 1, policy_version 1491436 (0.0005) [2023-12-27 15:59:25,739][20347] Updated weights for policy 0, policy_version 1493016 (0.0008) [2023-12-27 15:59:25,767][20381] Updated weights for policy 1, policy_version 1491446 (0.0008) [2023-12-27 15:59:25,822][20381] Updated weights for policy 1, policy_version 1491456 (0.0010) [2023-12-27 15:59:26,382][20381] Updated weights for policy 1, policy_version 1491466 (0.0006) [2023-12-27 15:59:26,443][20381] Updated weights for policy 1, policy_version 1491476 (0.0006) [2023-12-27 15:59:26,452][20347] Updated weights for policy 0, policy_version 1493026 (0.0007) [2023-12-27 15:59:26,503][20381] Updated weights for policy 1, policy_version 1491486 (0.0006) [2023-12-27 15:59:26,505][20347] Updated weights for policy 0, policy_version 1493036 (0.0006) [2023-12-27 15:59:26,552][20347] Updated weights for policy 0, policy_version 1493046 (0.0007) [2023-12-27 15:59:27,099][20381] Updated weights for policy 1, policy_version 1491496 (0.0005) [2023-12-27 15:59:27,159][20381] Updated weights for policy 1, policy_version 1491506 (0.0008) [2023-12-27 15:59:27,219][20381] Updated weights for policy 1, policy_version 1491516 (0.0007) [2023-12-27 15:59:27,249][20347] Updated weights for policy 0, policy_version 1493056 (0.0007) [2023-12-27 15:59:27,305][20347] Updated weights for policy 0, policy_version 1493066 (0.0008) [2023-12-27 15:59:27,362][20347] Updated weights for policy 0, policy_version 1493076 (0.0007) [2023-12-27 15:59:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 764157952. Throughput: 0: 10364.3, 1: 10188.2. Samples: 764148952. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:27,363][19170] Avg episode reward: [(0, '951.460'), (1, '990.270')] [2023-12-27 15:59:27,805][20381] Updated weights for policy 1, policy_version 1491526 (0.0007) [2023-12-27 15:59:27,865][20381] Updated weights for policy 1, policy_version 1491536 (0.0008) [2023-12-27 15:59:27,913][20381] Updated weights for policy 1, policy_version 1491546 (0.0007) [2023-12-27 15:59:28,006][20347] Updated weights for policy 0, policy_version 1493086 (0.0008) [2023-12-27 15:59:28,053][20347] Updated weights for policy 0, policy_version 1493096 (0.0007) [2023-12-27 15:59:28,080][20291] KL-divergence is very high: 111.2637 [2023-12-27 15:59:28,110][20347] Updated weights for policy 0, policy_version 1493106 (0.0005) [2023-12-27 15:59:28,654][20381] Updated weights for policy 1, policy_version 1491556 (0.0007) [2023-12-27 15:59:28,712][20381] Updated weights for policy 1, policy_version 1491566 (0.0007) [2023-12-27 15:59:28,758][20381] Updated weights for policy 1, policy_version 1491576 (0.0008) [2023-12-27 15:59:28,778][20347] Updated weights for policy 0, policy_version 1493116 (0.0008) [2023-12-27 15:59:28,839][20347] Updated weights for policy 0, policy_version 1493126 (0.0010) [2023-12-27 15:59:28,906][20347] Updated weights for policy 0, policy_version 1493136 (0.0011) [2023-12-27 15:59:29,484][20381] Updated weights for policy 1, policy_version 1491586 (0.0007) [2023-12-27 15:59:29,522][20347] Updated weights for policy 0, policy_version 1493146 (0.0009) [2023-12-27 15:59:29,550][20381] Updated weights for policy 1, policy_version 1491596 (0.0009) [2023-12-27 15:59:29,586][20347] Updated weights for policy 0, policy_version 1493156 (0.0008) [2023-12-27 15:59:29,607][20381] Updated weights for policy 1, policy_version 1491606 (0.0006) [2023-12-27 15:59:29,640][20347] Updated weights for policy 0, policy_version 1493166 (0.0010) [2023-12-27 15:59:29,661][20381] Updated weights for policy 1, policy_version 1491616 (0.0006) [2023-12-27 15:59:29,692][20347] Updated weights for policy 0, policy_version 1493176 (0.0010) [2023-12-27 15:59:30,343][20347] Updated weights for policy 0, policy_version 1493186 (0.0006) [2023-12-27 15:59:30,374][20381] Updated weights for policy 1, policy_version 1491626 (0.0006) [2023-12-27 15:59:30,402][20347] Updated weights for policy 0, policy_version 1493196 (0.0005) [2023-12-27 15:59:30,437][20381] Updated weights for policy 1, policy_version 1491636 (0.0007) [2023-12-27 15:59:30,472][20347] Updated weights for policy 0, policy_version 1493206 (0.0010) [2023-12-27 15:59:30,492][20381] Updated weights for policy 1, policy_version 1491646 (0.0007) [2023-12-27 15:59:31,099][20347] Updated weights for policy 0, policy_version 1493216 (0.0007) [2023-12-27 15:59:31,102][20381] Updated weights for policy 1, policy_version 1491656 (0.0010) [2023-12-27 15:59:31,160][20347] Updated weights for policy 0, policy_version 1493226 (0.0009) [2023-12-27 15:59:31,168][20381] Updated weights for policy 1, policy_version 1491666 (0.0011) [2023-12-27 15:59:31,215][20347] Updated weights for policy 0, policy_version 1493236 (0.0008) [2023-12-27 15:59:31,225][20381] Updated weights for policy 1, policy_version 1491676 (0.0009) [2023-12-27 15:59:31,928][20381] Updated weights for policy 1, policy_version 1491686 (0.0011) [2023-12-27 15:59:31,939][20347] Updated weights for policy 0, policy_version 1493246 (0.0007) [2023-12-27 15:59:31,997][20381] Updated weights for policy 1, policy_version 1491696 (0.0011) [2023-12-27 15:59:32,010][20347] Updated weights for policy 0, policy_version 1493256 (0.0010) [2023-12-27 15:59:32,058][20381] Updated weights for policy 1, policy_version 1491706 (0.0007) [2023-12-27 15:59:32,069][20347] Updated weights for policy 0, policy_version 1493266 (0.0010) [2023-12-27 15:59:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 764264448. Throughput: 0: 10311.0, 1: 10261.3. Samples: 764273968. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:32,363][19170] Avg episode reward: [(0, '941.780'), (1, '980.520')] [2023-12-27 15:59:32,704][20381] Updated weights for policy 1, policy_version 1491716 (0.0007) [2023-12-27 15:59:32,749][20347] Updated weights for policy 0, policy_version 1493276 (0.0009) [2023-12-27 15:59:32,761][20381] Updated weights for policy 1, policy_version 1491726 (0.0008) [2023-12-27 15:59:32,812][20347] Updated weights for policy 0, policy_version 1493286 (0.0008) [2023-12-27 15:59:32,825][20381] Updated weights for policy 1, policy_version 1491736 (0.0006) [2023-12-27 15:59:32,872][20347] Updated weights for policy 0, policy_version 1493296 (0.0009) [2023-12-27 15:59:33,505][20381] Updated weights for policy 1, policy_version 1491746 (0.0008) [2023-12-27 15:59:33,542][20347] Updated weights for policy 0, policy_version 1493306 (0.0007) [2023-12-27 15:59:33,558][20381] Updated weights for policy 1, policy_version 1491756 (0.0007) [2023-12-27 15:59:33,598][20347] Updated weights for policy 0, policy_version 1493316 (0.0008) [2023-12-27 15:59:33,614][20381] Updated weights for policy 1, policy_version 1491766 (0.0006) [2023-12-27 15:59:33,654][20347] Updated weights for policy 0, policy_version 1493326 (0.0008) [2023-12-27 15:59:33,670][20381] Updated weights for policy 1, policy_version 1491776 (0.0006) [2023-12-27 15:59:33,710][20347] Updated weights for policy 0, policy_version 1493336 (0.0006) [2023-12-27 15:59:34,301][20347] Updated weights for policy 0, policy_version 1493346 (0.0008) [2023-12-27 15:59:34,351][20347] Updated weights for policy 0, policy_version 1493356 (0.0008) [2023-12-27 15:59:34,383][20381] Updated weights for policy 1, policy_version 1491786 (0.0006) [2023-12-27 15:59:34,404][20347] Updated weights for policy 0, policy_version 1493366 (0.0006) [2023-12-27 15:59:34,441][20381] Updated weights for policy 1, policy_version 1491796 (0.0007) [2023-12-27 15:59:34,503][20381] Updated weights for policy 1, policy_version 1491806 (0.0007) [2023-12-27 15:59:35,125][20381] Updated weights for policy 1, policy_version 1491816 (0.0009) [2023-12-27 15:59:35,129][20347] Updated weights for policy 0, policy_version 1493376 (0.0006) [2023-12-27 15:59:35,169][20381] Updated weights for policy 1, policy_version 1491826 (0.0010) [2023-12-27 15:59:35,179][20347] Updated weights for policy 0, policy_version 1493386 (0.0005) [2023-12-27 15:59:35,227][20381] Updated weights for policy 1, policy_version 1491836 (0.0010) [2023-12-27 15:59:35,235][20347] Updated weights for policy 0, policy_version 1493396 (0.0005) [2023-12-27 15:59:35,839][20381] Updated weights for policy 1, policy_version 1491846 (0.0010) [2023-12-27 15:59:35,894][20381] Updated weights for policy 1, policy_version 1491856 (0.0010) [2023-12-27 15:59:35,918][20347] Updated weights for policy 0, policy_version 1493406 (0.0007) [2023-12-27 15:59:35,954][20381] Updated weights for policy 1, policy_version 1491866 (0.0007) [2023-12-27 15:59:35,974][20347] Updated weights for policy 0, policy_version 1493416 (0.0008) [2023-12-27 15:59:36,022][20347] Updated weights for policy 0, policy_version 1493426 (0.0008) [2023-12-27 15:59:36,576][20381] Updated weights for policy 1, policy_version 1491876 (0.0008) [2023-12-27 15:59:36,637][20381] Updated weights for policy 1, policy_version 1491886 (0.0009) [2023-12-27 15:59:36,696][20381] Updated weights for policy 1, policy_version 1491896 (0.0011) [2023-12-27 15:59:36,743][20347] Updated weights for policy 0, policy_version 1493436 (0.0007) [2023-12-27 15:59:36,787][20347] Updated weights for policy 0, policy_version 1493446 (0.0007) [2023-12-27 15:59:36,844][20347] Updated weights for policy 0, policy_version 1493456 (0.0007) [2023-12-27 15:59:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 764362752. Throughput: 0: 10276.9, 1: 10273.0. Samples: 764335360. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:37,363][19170] Avg episode reward: [(0, '941.560'), (1, '980.660')] [2023-12-27 15:59:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001493464_382402560.pth... [2023-12-27 15:59:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001491904_381960192.pth... [2023-12-27 15:59:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001492280_382099456.pth [2023-12-27 15:59:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001490688_381648896.pth [2023-12-27 15:59:37,396][20381] Updated weights for policy 1, policy_version 1491906 (0.0010) [2023-12-27 15:59:37,441][20381] Updated weights for policy 1, policy_version 1491916 (0.0010) [2023-12-27 15:59:37,490][20381] Updated weights for policy 1, policy_version 1491926 (0.0010) [2023-12-27 15:59:37,499][20347] Updated weights for policy 0, policy_version 1493466 (0.0007) [2023-12-27 15:59:37,539][20381] Updated weights for policy 1, policy_version 1491936 (0.0010) [2023-12-27 15:59:37,553][20347] Updated weights for policy 0, policy_version 1493476 (0.0006) [2023-12-27 15:59:37,602][20347] Updated weights for policy 0, policy_version 1493486 (0.0008) [2023-12-27 15:59:37,659][20347] Updated weights for policy 0, policy_version 1493496 (0.0007) [2023-12-27 15:59:38,243][20381] Updated weights for policy 1, policy_version 1491946 (0.0010) [2023-12-27 15:59:38,302][20381] Updated weights for policy 1, policy_version 1491956 (0.0011) [2023-12-27 15:59:38,366][20381] Updated weights for policy 1, policy_version 1491966 (0.0010) [2023-12-27 15:59:38,398][20347] Updated weights for policy 0, policy_version 1493506 (0.0008) [2023-12-27 15:59:38,459][20347] Updated weights for policy 0, policy_version 1493516 (0.0009) [2023-12-27 15:59:38,521][20347] Updated weights for policy 0, policy_version 1493526 (0.0007) [2023-12-27 15:59:39,061][20381] Updated weights for policy 1, policy_version 1491976 (0.0005) [2023-12-27 15:59:39,120][20381] Updated weights for policy 1, policy_version 1491986 (0.0005) [2023-12-27 15:59:39,180][20381] Updated weights for policy 1, policy_version 1491996 (0.0007) [2023-12-27 15:59:39,232][20347] Updated weights for policy 0, policy_version 1493536 (0.0007) [2023-12-27 15:59:39,296][20347] Updated weights for policy 0, policy_version 1493546 (0.0008) [2023-12-27 15:59:39,363][20347] Updated weights for policy 0, policy_version 1493556 (0.0007) [2023-12-27 15:59:39,800][20381] Updated weights for policy 1, policy_version 1492006 (0.0009) [2023-12-27 15:59:39,865][20381] Updated weights for policy 1, policy_version 1492016 (0.0011) [2023-12-27 15:59:39,925][20381] Updated weights for policy 1, policy_version 1492026 (0.0010) [2023-12-27 15:59:40,127][20347] Updated weights for policy 0, policy_version 1493566 (0.0007) [2023-12-27 15:59:40,182][20347] Updated weights for policy 0, policy_version 1493576 (0.0006) [2023-12-27 15:59:40,245][20347] Updated weights for policy 0, policy_version 1493586 (0.0008) [2023-12-27 15:59:40,555][20381] Updated weights for policy 1, policy_version 1492036 (0.0009) [2023-12-27 15:59:40,608][20381] Updated weights for policy 1, policy_version 1492046 (0.0008) [2023-12-27 15:59:40,658][20381] Updated weights for policy 1, policy_version 1492056 (0.0008) [2023-12-27 15:59:40,952][20347] Updated weights for policy 0, policy_version 1493596 (0.0008) [2023-12-27 15:59:41,015][20347] Updated weights for policy 0, policy_version 1493606 (0.0008) [2023-12-27 15:59:41,080][20347] Updated weights for policy 0, policy_version 1493616 (0.0008) [2023-12-27 15:59:41,371][20381] Updated weights for policy 1, policy_version 1492066 (0.0008) [2023-12-27 15:59:41,441][20381] Updated weights for policy 1, policy_version 1492076 (0.0007) [2023-12-27 15:59:41,496][20381] Updated weights for policy 1, policy_version 1492086 (0.0007) [2023-12-27 15:59:41,560][20381] Updated weights for policy 1, policy_version 1492096 (0.0008) [2023-12-27 15:59:41,880][20347] Updated weights for policy 0, policy_version 1493626 (0.0009) [2023-12-27 15:59:41,953][20347] Updated weights for policy 0, policy_version 1493636 (0.0008) [2023-12-27 15:59:42,019][20347] Updated weights for policy 0, policy_version 1493646 (0.0005) [2023-12-27 15:59:42,073][20347] Updated weights for policy 0, policy_version 1493656 (0.0006) [2023-12-27 15:59:42,241][20381] Updated weights for policy 1, policy_version 1492106 (0.0009) [2023-12-27 15:59:42,301][20381] Updated weights for policy 1, policy_version 1492116 (0.0009) [2023-12-27 15:59:42,357][20381] Updated weights for policy 1, policy_version 1492126 (0.0007) [2023-12-27 15:59:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 764461056. Throughput: 0: 10155.8, 1: 10302.8. Samples: 764456152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:42,363][19170] Avg episode reward: [(0, '951.250'), (1, '990.410')] [2023-12-27 15:59:42,759][20347] Updated weights for policy 0, policy_version 1493666 (0.0007) [2023-12-27 15:59:42,816][20347] Updated weights for policy 0, policy_version 1493676 (0.0007) [2023-12-27 15:59:42,872][20347] Updated weights for policy 0, policy_version 1493686 (0.0007) [2023-12-27 15:59:43,062][20381] Updated weights for policy 1, policy_version 1492136 (0.0008) [2023-12-27 15:59:43,114][20381] Updated weights for policy 1, policy_version 1492146 (0.0008) [2023-12-27 15:59:43,167][20381] Updated weights for policy 1, policy_version 1492156 (0.0007) [2023-12-27 15:59:43,588][20347] Updated weights for policy 0, policy_version 1493696 (0.0007) [2023-12-27 15:59:43,640][20347] Updated weights for policy 0, policy_version 1493706 (0.0007) [2023-12-27 15:59:43,698][20347] Updated weights for policy 0, policy_version 1493716 (0.0008) [2023-12-27 15:59:43,835][20381] Updated weights for policy 1, policy_version 1492166 (0.0008) [2023-12-27 15:59:43,889][20381] Updated weights for policy 1, policy_version 1492176 (0.0007) [2023-12-27 15:59:43,937][20381] Updated weights for policy 1, policy_version 1492186 (0.0007) [2023-12-27 15:59:44,385][20347] Updated weights for policy 0, policy_version 1493726 (0.0008) [2023-12-27 15:59:44,445][20347] Updated weights for policy 0, policy_version 1493736 (0.0007) [2023-12-27 15:59:44,504][20347] Updated weights for policy 0, policy_version 1493746 (0.0006) [2023-12-27 15:59:44,592][20381] Updated weights for policy 1, policy_version 1492196 (0.0008) [2023-12-27 15:59:44,646][20381] Updated weights for policy 1, policy_version 1492206 (0.0007) [2023-12-27 15:59:44,694][20381] Updated weights for policy 1, policy_version 1492216 (0.0008) [2023-12-27 15:59:45,176][20347] Updated weights for policy 0, policy_version 1493756 (0.0006) [2023-12-27 15:59:45,230][20347] Updated weights for policy 0, policy_version 1493766 (0.0007) [2023-12-27 15:59:45,285][20347] Updated weights for policy 0, policy_version 1493776 (0.0008) [2023-12-27 15:59:45,412][20381] Updated weights for policy 1, policy_version 1492226 (0.0008) [2023-12-27 15:59:45,470][20381] Updated weights for policy 1, policy_version 1492236 (0.0008) [2023-12-27 15:59:45,519][20381] Updated weights for policy 1, policy_version 1492246 (0.0007) [2023-12-27 15:59:45,560][20381] Updated weights for policy 1, policy_version 1492256 (0.0007) [2023-12-27 15:59:45,930][20347] Updated weights for policy 0, policy_version 1493786 (0.0008) [2023-12-27 15:59:45,982][20347] Updated weights for policy 0, policy_version 1493796 (0.0009) [2023-12-27 15:59:46,028][20347] Updated weights for policy 0, policy_version 1493806 (0.0010) [2023-12-27 15:59:46,082][20347] Updated weights for policy 0, policy_version 1493816 (0.0010) [2023-12-27 15:59:46,299][20381] Updated weights for policy 1, policy_version 1492266 (0.0008) [2023-12-27 15:59:46,356][20381] Updated weights for policy 1, policy_version 1492276 (0.0008) [2023-12-27 15:59:46,413][20381] Updated weights for policy 1, policy_version 1492286 (0.0008) [2023-12-27 15:59:46,774][20347] Updated weights for policy 0, policy_version 1493826 (0.0007) [2023-12-27 15:59:46,829][20347] Updated weights for policy 0, policy_version 1493836 (0.0010) [2023-12-27 15:59:46,889][20347] Updated weights for policy 0, policy_version 1493846 (0.0010) [2023-12-27 15:59:47,107][20381] Updated weights for policy 1, policy_version 1492296 (0.0007) [2023-12-27 15:59:47,165][20381] Updated weights for policy 1, policy_version 1492306 (0.0009) [2023-12-27 15:59:47,219][20381] Updated weights for policy 1, policy_version 1492318 (0.0009) [2023-12-27 15:59:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 764567552. Throughput: 0: 10153.8, 1: 10285.7. Samples: 764575604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:47,363][19170] Avg episode reward: [(0, '960.980'), (1, '990.410')] [2023-12-27 15:59:47,555][20347] Updated weights for policy 0, policy_version 1493856 (0.0008) [2023-12-27 15:59:47,612][20347] Updated weights for policy 0, policy_version 1493866 (0.0005) [2023-12-27 15:59:47,676][20347] Updated weights for policy 0, policy_version 1493876 (0.0005) [2023-12-27 15:59:47,900][20381] Updated weights for policy 1, policy_version 1492328 (0.0006) [2023-12-27 15:59:47,951][20381] Updated weights for policy 1, policy_version 1492338 (0.0010) [2023-12-27 15:59:48,004][20381] Updated weights for policy 1, policy_version 1492348 (0.0010) [2023-12-27 15:59:48,363][20347] Updated weights for policy 0, policy_version 1493886 (0.0007) [2023-12-27 15:59:48,430][20347] Updated weights for policy 0, policy_version 1493896 (0.0007) [2023-12-27 15:59:48,488][20347] Updated weights for policy 0, policy_version 1493906 (0.0008) [2023-12-27 15:59:48,618][20381] Updated weights for policy 1, policy_version 1492358 (0.0009) [2023-12-27 15:59:48,675][20381] Updated weights for policy 1, policy_version 1492368 (0.0007) [2023-12-27 15:59:48,737][20381] Updated weights for policy 1, policy_version 1492378 (0.0006) [2023-12-27 15:59:49,179][20347] Updated weights for policy 0, policy_version 1493916 (0.0008) [2023-12-27 15:59:49,236][20347] Updated weights for policy 0, policy_version 1493926 (0.0010) [2023-12-27 15:59:49,295][20347] Updated weights for policy 0, policy_version 1493936 (0.0007) [2023-12-27 15:59:49,370][20381] Updated weights for policy 1, policy_version 1492388 (0.0010) [2023-12-27 15:59:49,427][20381] Updated weights for policy 1, policy_version 1492398 (0.0007) [2023-12-27 15:59:49,485][20381] Updated weights for policy 1, policy_version 1492408 (0.0005) [2023-12-27 15:59:49,986][20347] Updated weights for policy 0, policy_version 1493946 (0.0009) [2023-12-27 15:59:50,043][20347] Updated weights for policy 0, policy_version 1493956 (0.0008) [2023-12-27 15:59:50,127][20347] Updated weights for policy 0, policy_version 1493966 (0.0008) [2023-12-27 15:59:50,175][20347] Updated weights for policy 0, policy_version 1493976 (0.0008) [2023-12-27 15:59:50,207][20381] Updated weights for policy 1, policy_version 1492418 (0.0006) [2023-12-27 15:59:50,260][20381] Updated weights for policy 1, policy_version 1492428 (0.0010) [2023-12-27 15:59:50,310][20381] Updated weights for policy 1, policy_version 1492438 (0.0008) [2023-12-27 15:59:50,365][20381] Updated weights for policy 1, policy_version 1492448 (0.0011) [2023-12-27 15:59:50,794][20347] Updated weights for policy 0, policy_version 1493986 (0.0007) [2023-12-27 15:59:50,842][20347] Updated weights for policy 0, policy_version 1493996 (0.0007) [2023-12-27 15:59:50,894][20347] Updated weights for policy 0, policy_version 1494006 (0.0007) [2023-12-27 15:59:51,087][20381] Updated weights for policy 1, policy_version 1492459 (0.0007) [2023-12-27 15:59:51,145][20381] Updated weights for policy 1, policy_version 1492469 (0.0007) [2023-12-27 15:59:51,201][20381] Updated weights for policy 1, policy_version 1492479 (0.0006) [2023-12-27 15:59:51,675][20347] Updated weights for policy 0, policy_version 1494016 (0.0008) [2023-12-27 15:59:51,737][20347] Updated weights for policy 0, policy_version 1494026 (0.0008) [2023-12-27 15:59:51,794][20347] Updated weights for policy 0, policy_version 1494036 (0.0008) [2023-12-27 15:59:51,882][20381] Updated weights for policy 1, policy_version 1492489 (0.0008) [2023-12-27 15:59:51,942][20381] Updated weights for policy 1, policy_version 1492499 (0.0008) [2023-12-27 15:59:52,005][20381] Updated weights for policy 1, policy_version 1492509 (0.0006) [2023-12-27 15:59:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 764665856. Throughput: 0: 10117.6, 1: 10338.3. Samples: 764637604. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:52,363][19170] Avg episode reward: [(0, '961.070'), (1, '1000.000')] [2023-12-27 15:59:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001492512_382115840.pth... [2023-12-27 15:59:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001494040_382550016.pth... [2023-12-27 15:59:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001491296_381804544.pth [2023-12-27 15:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001492512_382115840.pth [2023-12-27 15:59:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001492888_382255104.pth [2023-12-27 15:59:52,377][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001494040_382550016.pth [2023-12-27 15:59:52,440][20347] Updated weights for policy 0, policy_version 1494046 (0.0008) [2023-12-27 15:59:52,510][20347] Updated weights for policy 0, policy_version 1494056 (0.0006) [2023-12-27 15:59:52,567][20347] Updated weights for policy 0, policy_version 1494066 (0.0006) [2023-12-27 15:59:52,639][20381] Updated weights for policy 1, policy_version 1492519 (0.0008) [2023-12-27 15:59:52,702][20381] Updated weights for policy 1, policy_version 1492529 (0.0008) [2023-12-27 15:59:52,760][20381] Updated weights for policy 1, policy_version 1492539 (0.0009) [2023-12-27 15:59:53,157][20347] Updated weights for policy 0, policy_version 1494076 (0.0007) [2023-12-27 15:59:53,223][20347] Updated weights for policy 0, policy_version 1494086 (0.0011) [2023-12-27 15:59:53,283][20347] Updated weights for policy 0, policy_version 1494096 (0.0011) [2023-12-27 15:59:53,498][20381] Updated weights for policy 1, policy_version 1492549 (0.0009) [2023-12-27 15:59:53,562][20381] Updated weights for policy 1, policy_version 1492559 (0.0011) [2023-12-27 15:59:53,620][20381] Updated weights for policy 1, policy_version 1492569 (0.0010) [2023-12-27 15:59:53,938][20347] Updated weights for policy 0, policy_version 1494106 (0.0011) [2023-12-27 15:59:53,986][20347] Updated weights for policy 0, policy_version 1494116 (0.0010) [2023-12-27 15:59:54,031][20347] Updated weights for policy 0, policy_version 1494126 (0.0010) [2023-12-27 15:59:54,083][20347] Updated weights for policy 0, policy_version 1494136 (0.0010) [2023-12-27 15:59:54,235][20381] Updated weights for policy 1, policy_version 1492579 (0.0010) [2023-12-27 15:59:54,292][20381] Updated weights for policy 1, policy_version 1492589 (0.0007) [2023-12-27 15:59:54,357][20381] Updated weights for policy 1, policy_version 1492599 (0.0006) [2023-12-27 15:59:54,755][20347] Updated weights for policy 0, policy_version 1494146 (0.0010) [2023-12-27 15:59:54,812][20347] Updated weights for policy 0, policy_version 1494156 (0.0010) [2023-12-27 15:59:54,869][20347] Updated weights for policy 0, policy_version 1494166 (0.0010) [2023-12-27 15:59:55,027][20381] Updated weights for policy 1, policy_version 1492609 (0.0008) [2023-12-27 15:59:55,082][20381] Updated weights for policy 1, policy_version 1492619 (0.0008) [2023-12-27 15:59:55,140][20381] Updated weights for policy 1, policy_version 1492629 (0.0007) [2023-12-27 15:59:55,194][20381] Updated weights for policy 1, policy_version 1492639 (0.0007) [2023-12-27 15:59:55,547][20347] Updated weights for policy 0, policy_version 1494176 (0.0010) [2023-12-27 15:59:55,608][20347] Updated weights for policy 0, policy_version 1494186 (0.0010) [2023-12-27 15:59:55,664][20347] Updated weights for policy 0, policy_version 1494196 (0.0010) [2023-12-27 15:59:55,897][20381] Updated weights for policy 1, policy_version 1492649 (0.0008) [2023-12-27 15:59:55,948][20381] Updated weights for policy 1, policy_version 1492659 (0.0007) [2023-12-27 15:59:56,008][20381] Updated weights for policy 1, policy_version 1492669 (0.0008) [2023-12-27 15:59:56,352][20347] Updated weights for policy 0, policy_version 1494206 (0.0007) [2023-12-27 15:59:56,406][20347] Updated weights for policy 0, policy_version 1494216 (0.0007) [2023-12-27 15:59:56,462][20347] Updated weights for policy 0, policy_version 1494226 (0.0006) [2023-12-27 15:59:56,578][20381] Updated weights for policy 1, policy_version 1492679 (0.0008) [2023-12-27 15:59:56,632][20381] Updated weights for policy 1, policy_version 1492689 (0.0008) [2023-12-27 15:59:56,677][20381] Updated weights for policy 1, policy_version 1492699 (0.0007) [2023-12-27 15:59:57,141][20347] Updated weights for policy 0, policy_version 1494236 (0.0007) [2023-12-27 15:59:57,197][20347] Updated weights for policy 0, policy_version 1494246 (0.0007) [2023-12-27 15:59:57,249][20347] Updated weights for policy 0, policy_version 1494256 (0.0007) [2023-12-27 15:59:57,338][20381] Updated weights for policy 1, policy_version 1492709 (0.0009) [2023-12-27 15:59:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 764772352. Throughput: 0: 10105.6, 1: 10345.4. Samples: 764760444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 15:59:57,363][19170] Avg episode reward: [(0, '961.000'), (1, '1000.000')] [2023-12-27 15:59:57,393][20381] Updated weights for policy 1, policy_version 1492719 (0.0010) [2023-12-27 15:59:57,450][20381] Updated weights for policy 1, policy_version 1492729 (0.0010) [2023-12-27 15:59:57,939][20347] Updated weights for policy 0, policy_version 1494266 (0.0007) [2023-12-27 15:59:57,993][20347] Updated weights for policy 0, policy_version 1494276 (0.0005) [2023-12-27 15:59:58,041][20347] Updated weights for policy 0, policy_version 1494286 (0.0005) [2023-12-27 15:59:58,089][20347] Updated weights for policy 0, policy_version 1494296 (0.0005) [2023-12-27 15:59:58,112][20381] Updated weights for policy 1, policy_version 1492739 (0.0010) [2023-12-27 15:59:58,162][20381] Updated weights for policy 1, policy_version 1492749 (0.0009) [2023-12-27 15:59:58,224][20381] Updated weights for policy 1, policy_version 1492759 (0.0007) [2023-12-27 15:59:58,906][20347] Updated weights for policy 0, policy_version 1494306 (0.0006) [2023-12-27 15:59:58,970][20347] Updated weights for policy 0, policy_version 1494316 (0.0008) [2023-12-27 15:59:59,026][20347] Updated weights for policy 0, policy_version 1494326 (0.0008) [2023-12-27 15:59:59,027][20381] Updated weights for policy 1, policy_version 1492769 (0.0007) [2023-12-27 15:59:59,073][20381] Updated weights for policy 1, policy_version 1492779 (0.0007) [2023-12-27 15:59:59,127][20381] Updated weights for policy 1, policy_version 1492789 (0.0008) [2023-12-27 15:59:59,186][20381] Updated weights for policy 1, policy_version 1492799 (0.0007) [2023-12-27 15:59:59,760][20347] Updated weights for policy 0, policy_version 1494336 (0.0007) [2023-12-27 15:59:59,818][20347] Updated weights for policy 0, policy_version 1494346 (0.0008) [2023-12-27 15:59:59,830][20381] Updated weights for policy 1, policy_version 1492809 (0.0007) [2023-12-27 15:59:59,880][20347] Updated weights for policy 0, policy_version 1494356 (0.0008) [2023-12-27 15:59:59,891][20381] Updated weights for policy 1, policy_version 1492819 (0.0007) [2023-12-27 15:59:59,954][20381] Updated weights for policy 1, policy_version 1492829 (0.0008) [2023-12-27 16:00:00,553][20347] Updated weights for policy 0, policy_version 1494366 (0.0008) [2023-12-27 16:00:00,610][20347] Updated weights for policy 0, policy_version 1494376 (0.0010) [2023-12-27 16:00:00,664][20347] Updated weights for policy 0, policy_version 1494386 (0.0010) [2023-12-27 16:00:00,701][20381] Updated weights for policy 1, policy_version 1492839 (0.0006) [2023-12-27 16:00:00,750][20381] Updated weights for policy 1, policy_version 1492849 (0.0007) [2023-12-27 16:00:00,808][20381] Updated weights for policy 1, policy_version 1492859 (0.0008) [2023-12-27 16:00:01,303][20347] Updated weights for policy 0, policy_version 1494396 (0.0009) [2023-12-27 16:00:01,375][20347] Updated weights for policy 0, policy_version 1494406 (0.0008) [2023-12-27 16:00:01,427][20347] Updated weights for policy 0, policy_version 1494416 (0.0008) [2023-12-27 16:00:01,511][20381] Updated weights for policy 1, policy_version 1492869 (0.0007) [2023-12-27 16:00:01,562][20381] Updated weights for policy 1, policy_version 1492879 (0.0007) [2023-12-27 16:00:01,620][20381] Updated weights for policy 1, policy_version 1492889 (0.0008) [2023-12-27 16:00:02,106][20347] Updated weights for policy 0, policy_version 1494426 (0.0009) [2023-12-27 16:00:02,161][20347] Updated weights for policy 0, policy_version 1494436 (0.0008) [2023-12-27 16:00:02,216][20347] Updated weights for policy 0, policy_version 1494446 (0.0008) [2023-12-27 16:00:02,275][20347] Updated weights for policy 0, policy_version 1494456 (0.0007) [2023-12-27 16:00:02,351][20381] Updated weights for policy 1, policy_version 1492899 (0.0008) [2023-12-27 16:00:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 764870656. Throughput: 0: 10059.9, 1: 10267.4. Samples: 764880908. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 16:00:02,363][19170] Avg episode reward: [(0, '941.360'), (1, '1000.000')] [2023-12-27 16:00:02,415][20381] Updated weights for policy 1, policy_version 1492909 (0.0008) [2023-12-27 16:00:02,471][20381] Updated weights for policy 1, policy_version 1492919 (0.0008) [2023-12-27 16:00:02,952][20347] Updated weights for policy 0, policy_version 1494466 (0.0008) [2023-12-27 16:00:02,998][20347] Updated weights for policy 0, policy_version 1494476 (0.0007) [2023-12-27 16:00:03,047][20347] Updated weights for policy 0, policy_version 1494486 (0.0008) [2023-12-27 16:00:03,150][20381] Updated weights for policy 1, policy_version 1492929 (0.0008) [2023-12-27 16:00:03,204][20381] Updated weights for policy 1, policy_version 1492939 (0.0007) [2023-12-27 16:00:03,258][20381] Updated weights for policy 1, policy_version 1492949 (0.0007) [2023-12-27 16:00:03,315][20381] Updated weights for policy 1, policy_version 1492959 (0.0008) [2023-12-27 16:00:03,762][20347] Updated weights for policy 0, policy_version 1494496 (0.0007) [2023-12-27 16:00:03,814][20347] Updated weights for policy 0, policy_version 1494506 (0.0008) [2023-12-27 16:00:03,873][20347] Updated weights for policy 0, policy_version 1494516 (0.0008) [2023-12-27 16:00:04,005][20381] Updated weights for policy 1, policy_version 1492969 (0.0008) [2023-12-27 16:00:04,068][20381] Updated weights for policy 1, policy_version 1492979 (0.0008) [2023-12-27 16:00:04,132][20381] Updated weights for policy 1, policy_version 1492989 (0.0007) [2023-12-27 16:00:04,629][20347] Updated weights for policy 0, policy_version 1494526 (0.0009) [2023-12-27 16:00:04,674][20347] Updated weights for policy 0, policy_version 1494536 (0.0010) [2023-12-27 16:00:04,723][20347] Updated weights for policy 0, policy_version 1494546 (0.0010) [2023-12-27 16:00:04,758][20381] Updated weights for policy 1, policy_version 1492999 (0.0006) [2023-12-27 16:00:04,811][20381] Updated weights for policy 1, policy_version 1493009 (0.0005) [2023-12-27 16:00:04,859][20381] Updated weights for policy 1, policy_version 1493019 (0.0005) [2023-12-27 16:00:05,391][20347] Updated weights for policy 0, policy_version 1494556 (0.0010) [2023-12-27 16:00:05,439][20347] Updated weights for policy 0, policy_version 1494566 (0.0010) [2023-12-27 16:00:05,467][20381] Updated weights for policy 1, policy_version 1493029 (0.0006) [2023-12-27 16:00:05,494][20347] Updated weights for policy 0, policy_version 1494576 (0.0010) [2023-12-27 16:00:05,525][20381] Updated weights for policy 1, policy_version 1493039 (0.0006) [2023-12-27 16:00:05,575][20381] Updated weights for policy 1, policy_version 1493049 (0.0007) [2023-12-27 16:00:06,160][20381] Updated weights for policy 1, policy_version 1493059 (0.0008) [2023-12-27 16:00:06,172][20347] Updated weights for policy 0, policy_version 1494586 (0.0010) [2023-12-27 16:00:06,215][20381] Updated weights for policy 1, policy_version 1493069 (0.0006) [2023-12-27 16:00:06,231][20347] Updated weights for policy 0, policy_version 1494596 (0.0010) [2023-12-27 16:00:06,277][20381] Updated weights for policy 1, policy_version 1493079 (0.0005) [2023-12-27 16:00:06,290][20347] Updated weights for policy 0, policy_version 1494606 (0.0011) [2023-12-27 16:00:06,347][20347] Updated weights for policy 0, policy_version 1494616 (0.0010) [2023-12-27 16:00:06,997][20381] Updated weights for policy 1, policy_version 1493089 (0.0005) [2023-12-27 16:00:07,020][20347] Updated weights for policy 0, policy_version 1494626 (0.0010) [2023-12-27 16:00:07,055][20381] Updated weights for policy 1, policy_version 1493099 (0.0005) [2023-12-27 16:00:07,076][20347] Updated weights for policy 0, policy_version 1494636 (0.0010) [2023-12-27 16:00:07,107][20381] Updated weights for policy 1, policy_version 1493109 (0.0005) [2023-12-27 16:00:07,134][20347] Updated weights for policy 0, policy_version 1494646 (0.0010) [2023-12-27 16:00:07,162][20381] Updated weights for policy 1, policy_version 1493119 (0.0005) [2023-12-27 16:00:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 764977152. Throughput: 0: 10053.9, 1: 10275.4. Samples: 764942744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 16:00:07,363][19170] Avg episode reward: [(0, '921.930'), (1, '990.330')] [2023-12-27 16:00:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001493120_382271488.pth... [2023-12-27 16:00:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001494648_382705664.pth... [2023-12-27 16:00:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001493464_382402560.pth [2023-12-27 16:00:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001491904_381960192.pth [2023-12-27 16:00:07,759][20381] Updated weights for policy 1, policy_version 1493129 (0.0007) [2023-12-27 16:00:07,814][20381] Updated weights for policy 1, policy_version 1493139 (0.0008) [2023-12-27 16:00:07,829][20347] Updated weights for policy 0, policy_version 1494656 (0.0007) [2023-12-27 16:00:07,874][20381] Updated weights for policy 1, policy_version 1493149 (0.0005) [2023-12-27 16:00:07,876][20347] Updated weights for policy 0, policy_version 1494666 (0.0006) [2023-12-27 16:00:07,929][20347] Updated weights for policy 0, policy_version 1494676 (0.0006) [2023-12-27 16:00:08,566][20381] Updated weights for policy 1, policy_version 1493159 (0.0006) [2023-12-27 16:00:08,568][20347] Updated weights for policy 0, policy_version 1494686 (0.0008) [2023-12-27 16:00:08,619][20381] Updated weights for policy 1, policy_version 1493169 (0.0007) [2023-12-27 16:00:08,622][20347] Updated weights for policy 0, policy_version 1494696 (0.0005) [2023-12-27 16:00:08,677][20381] Updated weights for policy 1, policy_version 1493179 (0.0007) [2023-12-27 16:00:08,680][20347] Updated weights for policy 0, policy_version 1494706 (0.0006) [2023-12-27 16:00:09,300][20381] Updated weights for policy 1, policy_version 1493189 (0.0006) [2023-12-27 16:00:09,366][20347] Updated weights for policy 0, policy_version 1494716 (0.0008) [2023-12-27 16:00:09,367][20381] Updated weights for policy 1, policy_version 1493199 (0.0007) [2023-12-27 16:00:09,432][20381] Updated weights for policy 1, policy_version 1493209 (0.0008) [2023-12-27 16:00:09,438][20347] Updated weights for policy 0, policy_version 1494726 (0.0010) [2023-12-27 16:00:09,505][20347] Updated weights for policy 0, policy_version 1494736 (0.0010) [2023-12-27 16:00:10,105][20381] Updated weights for policy 1, policy_version 1493219 (0.0007) [2023-12-27 16:00:10,167][20381] Updated weights for policy 1, policy_version 1493229 (0.0006) [2023-12-27 16:00:10,216][20347] Updated weights for policy 0, policy_version 1494746 (0.0007) [2023-12-27 16:00:10,229][20381] Updated weights for policy 1, policy_version 1493239 (0.0008) [2023-12-27 16:00:10,281][20347] Updated weights for policy 0, policy_version 1494756 (0.0006) [2023-12-27 16:00:10,338][20347] Updated weights for policy 0, policy_version 1494766 (0.0008) [2023-12-27 16:00:10,399][20347] Updated weights for policy 0, policy_version 1494776 (0.0007) [2023-12-27 16:00:10,801][20381] Updated weights for policy 1, policy_version 1493249 (0.0007) [2023-12-27 16:00:10,851][20381] Updated weights for policy 1, policy_version 1493259 (0.0008) [2023-12-27 16:00:10,911][20381] Updated weights for policy 1, policy_version 1493269 (0.0007) [2023-12-27 16:00:10,957][20381] Updated weights for policy 1, policy_version 1493279 (0.0010) [2023-12-27 16:00:11,095][20347] Updated weights for policy 0, policy_version 1494787 (0.0008) [2023-12-27 16:00:11,161][20347] Updated weights for policy 0, policy_version 1494797 (0.0008) [2023-12-27 16:00:11,222][20347] Updated weights for policy 0, policy_version 1494807 (0.0008) [2023-12-27 16:00:11,836][20381] Updated weights for policy 1, policy_version 1493289 (0.0009) [2023-12-27 16:00:11,888][20381] Updated weights for policy 1, policy_version 1493299 (0.0011) [2023-12-27 16:00:11,950][20381] Updated weights for policy 1, policy_version 1493309 (0.0011) [2023-12-27 16:00:11,966][20347] Updated weights for policy 0, policy_version 1494817 (0.0006) [2023-12-27 16:00:12,029][20347] Updated weights for policy 0, policy_version 1494827 (0.0006) [2023-12-27 16:00:12,094][20347] Updated weights for policy 0, policy_version 1494837 (0.0010) [2023-12-27 16:00:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 765075456. Throughput: 0: 10085.0, 1: 10307.1. Samples: 765066596. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:12,363][19170] Avg episode reward: [(0, '931.660'), (1, '990.330')] [2023-12-27 16:00:12,624][20381] Updated weights for policy 1, policy_version 1493319 (0.0009) [2023-12-27 16:00:12,685][20381] Updated weights for policy 1, policy_version 1493329 (0.0008) [2023-12-27 16:00:12,733][20347] Updated weights for policy 0, policy_version 1494847 (0.0010) [2023-12-27 16:00:12,739][20381] Updated weights for policy 1, policy_version 1493339 (0.0007) [2023-12-27 16:00:12,794][20347] Updated weights for policy 0, policy_version 1494857 (0.0009) [2023-12-27 16:00:12,851][20347] Updated weights for policy 0, policy_version 1494867 (0.0008) [2023-12-27 16:00:13,386][20381] Updated weights for policy 1, policy_version 1493349 (0.0008) [2023-12-27 16:00:13,448][20381] Updated weights for policy 1, policy_version 1493359 (0.0008) [2023-12-27 16:00:13,500][20381] Updated weights for policy 1, policy_version 1493369 (0.0007) [2023-12-27 16:00:13,558][20347] Updated weights for policy 0, policy_version 1494877 (0.0009) [2023-12-27 16:00:13,613][20347] Updated weights for policy 0, policy_version 1494887 (0.0009) [2023-12-27 16:00:13,664][20347] Updated weights for policy 0, policy_version 1494897 (0.0006) [2023-12-27 16:00:14,230][20381] Updated weights for policy 1, policy_version 1493379 (0.0007) [2023-12-27 16:00:14,249][20347] Updated weights for policy 0, policy_version 1494907 (0.0007) [2023-12-27 16:00:14,296][20381] Updated weights for policy 1, policy_version 1493389 (0.0006) [2023-12-27 16:00:14,306][20347] Updated weights for policy 0, policy_version 1494917 (0.0007) [2023-12-27 16:00:14,351][20381] Updated weights for policy 1, policy_version 1493399 (0.0006) [2023-12-27 16:00:14,361][20347] Updated weights for policy 0, policy_version 1494927 (0.0006) [2023-12-27 16:00:15,007][20347] Updated weights for policy 0, policy_version 1494937 (0.0006) [2023-12-27 16:00:15,058][20381] Updated weights for policy 1, policy_version 1493409 (0.0007) [2023-12-27 16:00:15,063][20347] Updated weights for policy 0, policy_version 1494947 (0.0008) [2023-12-27 16:00:15,114][20381] Updated weights for policy 1, policy_version 1493419 (0.0006) [2023-12-27 16:00:15,127][20347] Updated weights for policy 0, policy_version 1494957 (0.0008) [2023-12-27 16:00:15,166][20381] Updated weights for policy 1, policy_version 1493429 (0.0006) [2023-12-27 16:00:15,188][20347] Updated weights for policy 0, policy_version 1494967 (0.0008) [2023-12-27 16:00:15,228][20381] Updated weights for policy 1, policy_version 1493439 (0.0006) [2023-12-27 16:00:15,832][20381] Updated weights for policy 1, policy_version 1493449 (0.0007) [2023-12-27 16:00:15,884][20381] Updated weights for policy 1, policy_version 1493459 (0.0007) [2023-12-27 16:00:15,915][20347] Updated weights for policy 0, policy_version 1494977 (0.0008) [2023-12-27 16:00:15,929][20381] Updated weights for policy 1, policy_version 1493469 (0.0006) [2023-12-27 16:00:15,964][20347] Updated weights for policy 0, policy_version 1494987 (0.0006) [2023-12-27 16:00:16,013][20347] Updated weights for policy 0, policy_version 1494997 (0.0005) [2023-12-27 16:00:16,644][20381] Updated weights for policy 1, policy_version 1493479 (0.0007) [2023-12-27 16:00:16,700][20381] Updated weights for policy 1, policy_version 1493489 (0.0007) [2023-12-27 16:00:16,716][20347] Updated weights for policy 0, policy_version 1495007 (0.0008) [2023-12-27 16:00:16,752][20381] Updated weights for policy 1, policy_version 1493499 (0.0006) [2023-12-27 16:00:16,771][20347] Updated weights for policy 0, policy_version 1495017 (0.0005) [2023-12-27 16:00:16,832][20347] Updated weights for policy 0, policy_version 1495027 (0.0006) [2023-12-27 16:00:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 765173760. Throughput: 0: 10059.6, 1: 10249.7. Samples: 765187888. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:17,363][19170] Avg episode reward: [(0, '931.530'), (1, '1000.000')] [2023-12-27 16:00:17,403][20381] Updated weights for policy 1, policy_version 1493509 (0.0006) [2023-12-27 16:00:17,457][20381] Updated weights for policy 1, policy_version 1493519 (0.0005) [2023-12-27 16:00:17,467][20347] Updated weights for policy 0, policy_version 1495037 (0.0006) [2023-12-27 16:00:17,511][20381] Updated weights for policy 1, policy_version 1493529 (0.0005) [2023-12-27 16:00:17,518][20347] Updated weights for policy 0, policy_version 1495047 (0.0005) [2023-12-27 16:00:17,569][20347] Updated weights for policy 0, policy_version 1495057 (0.0005) [2023-12-27 16:00:18,069][20381] Updated weights for policy 1, policy_version 1493539 (0.0006) [2023-12-27 16:00:18,123][20381] Updated weights for policy 1, policy_version 1493549 (0.0007) [2023-12-27 16:00:18,184][20381] Updated weights for policy 1, policy_version 1493559 (0.0008) [2023-12-27 16:00:18,238][20347] Updated weights for policy 0, policy_version 1495067 (0.0007) [2023-12-27 16:00:18,294][20347] Updated weights for policy 0, policy_version 1495077 (0.0008) [2023-12-27 16:00:18,370][20347] Updated weights for policy 0, policy_version 1495087 (0.0009) [2023-12-27 16:00:18,820][20381] Updated weights for policy 1, policy_version 1493569 (0.0011) [2023-12-27 16:00:18,873][20381] Updated weights for policy 1, policy_version 1493579 (0.0010) [2023-12-27 16:00:18,923][20381] Updated weights for policy 1, policy_version 1493589 (0.0010) [2023-12-27 16:00:18,971][20381] Updated weights for policy 1, policy_version 1493599 (0.0010) [2023-12-27 16:00:19,039][20347] Updated weights for policy 0, policy_version 1495097 (0.0007) [2023-12-27 16:00:19,093][20347] Updated weights for policy 0, policy_version 1495107 (0.0005) [2023-12-27 16:00:19,155][20347] Updated weights for policy 0, policy_version 1495117 (0.0008) [2023-12-27 16:00:19,200][20347] Updated weights for policy 0, policy_version 1495127 (0.0007) [2023-12-27 16:00:19,627][20381] Updated weights for policy 1, policy_version 1493609 (0.0008) [2023-12-27 16:00:19,684][20381] Updated weights for policy 1, policy_version 1493619 (0.0008) [2023-12-27 16:00:19,747][20381] Updated weights for policy 1, policy_version 1493629 (0.0008) [2023-12-27 16:00:19,937][20347] Updated weights for policy 0, policy_version 1495137 (0.0008) [2023-12-27 16:00:19,997][20347] Updated weights for policy 0, policy_version 1495147 (0.0008) [2023-12-27 16:00:20,055][20347] Updated weights for policy 0, policy_version 1495157 (0.0007) [2023-12-27 16:00:20,483][20381] Updated weights for policy 1, policy_version 1493639 (0.0008) [2023-12-27 16:00:20,540][20381] Updated weights for policy 1, policy_version 1493649 (0.0008) [2023-12-27 16:00:20,605][20381] Updated weights for policy 1, policy_version 1493659 (0.0007) [2023-12-27 16:00:20,770][20347] Updated weights for policy 0, policy_version 1495167 (0.0010) [2023-12-27 16:00:20,826][20347] Updated weights for policy 0, policy_version 1495177 (0.0011) [2023-12-27 16:00:20,880][20347] Updated weights for policy 0, policy_version 1495187 (0.0011) [2023-12-27 16:00:21,329][20381] Updated weights for policy 1, policy_version 1493669 (0.0009) [2023-12-27 16:00:21,397][20381] Updated weights for policy 1, policy_version 1493679 (0.0011) [2023-12-27 16:00:21,460][20381] Updated weights for policy 1, policy_version 1493689 (0.0007) [2023-12-27 16:00:21,652][20347] Updated weights for policy 0, policy_version 1495197 (0.0009) [2023-12-27 16:00:21,710][20347] Updated weights for policy 0, policy_version 1495207 (0.0007) [2023-12-27 16:00:21,775][20347] Updated weights for policy 0, policy_version 1495217 (0.0008) [2023-12-27 16:00:22,179][20381] Updated weights for policy 1, policy_version 1493699 (0.0008) [2023-12-27 16:00:22,231][20381] Updated weights for policy 1, policy_version 1493709 (0.0009) [2023-12-27 16:00:22,287][20381] Updated weights for policy 1, policy_version 1493719 (0.0008) [2023-12-27 16:00:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 765280256. Throughput: 0: 10047.5, 1: 10296.0. Samples: 765250816. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:22,363][19170] Avg episode reward: [(0, '951.260'), (1, '1000.000')] [2023-12-27 16:00:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001493728_382427136.pth... [2023-12-27 16:00:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001495224_382853120.pth... [2023-12-27 16:00:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001492512_382115840.pth [2023-12-27 16:00:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001494040_382550016.pth [2023-12-27 16:00:22,433][20347] Updated weights for policy 0, policy_version 1495227 (0.0008) [2023-12-27 16:00:22,489][20347] Updated weights for policy 0, policy_version 1495237 (0.0008) [2023-12-27 16:00:22,495][20291] KL-divergence is very high: 105.0701 [2023-12-27 16:00:22,549][20347] Updated weights for policy 0, policy_version 1495247 (0.0008) [2023-12-27 16:00:22,945][20381] Updated weights for policy 1, policy_version 1493729 (0.0006) [2023-12-27 16:00:23,007][20381] Updated weights for policy 1, policy_version 1493739 (0.0007) [2023-12-27 16:00:23,066][20381] Updated weights for policy 1, policy_version 1493749 (0.0007) [2023-12-27 16:00:23,129][20381] Updated weights for policy 1, policy_version 1493759 (0.0007) [2023-12-27 16:00:23,277][20347] Updated weights for policy 0, policy_version 1495257 (0.0008) [2023-12-27 16:00:23,331][20347] Updated weights for policy 0, policy_version 1495267 (0.0010) [2023-12-27 16:00:23,385][20347] Updated weights for policy 0, policy_version 1495277 (0.0008) [2023-12-27 16:00:23,443][20347] Updated weights for policy 0, policy_version 1495287 (0.0008) [2023-12-27 16:00:23,816][20381] Updated weights for policy 1, policy_version 1493769 (0.0008) [2023-12-27 16:00:23,862][20381] Updated weights for policy 1, policy_version 1493779 (0.0008) [2023-12-27 16:00:23,915][20381] Updated weights for policy 1, policy_version 1493789 (0.0008) [2023-12-27 16:00:24,156][20347] Updated weights for policy 0, policy_version 1495297 (0.0010) [2023-12-27 16:00:24,216][20347] Updated weights for policy 0, policy_version 1495307 (0.0010) [2023-12-27 16:00:24,271][20347] Updated weights for policy 0, policy_version 1495317 (0.0010) [2023-12-27 16:00:24,654][20381] Updated weights for policy 1, policy_version 1493799 (0.0008) [2023-12-27 16:00:24,716][20381] Updated weights for policy 1, policy_version 1493809 (0.0007) [2023-12-27 16:00:24,776][20381] Updated weights for policy 1, policy_version 1493819 (0.0008) [2023-12-27 16:00:24,861][20347] Updated weights for policy 0, policy_version 1495327 (0.0007) [2023-12-27 16:00:24,909][20347] Updated weights for policy 0, policy_version 1495337 (0.0005) [2023-12-27 16:00:24,960][20347] Updated weights for policy 0, policy_version 1495347 (0.0005) [2023-12-27 16:00:25,376][20381] Updated weights for policy 1, policy_version 1493829 (0.0009) [2023-12-27 16:00:25,424][20381] Updated weights for policy 1, policy_version 1493839 (0.0007) [2023-12-27 16:00:25,469][20381] Updated weights for policy 1, policy_version 1493849 (0.0006) [2023-12-27 16:00:25,583][20347] Updated weights for policy 0, policy_version 1495357 (0.0006) [2023-12-27 16:00:25,626][20347] Updated weights for policy 0, policy_version 1495367 (0.0008) [2023-12-27 16:00:25,677][20347] Updated weights for policy 0, policy_version 1495377 (0.0007) [2023-12-27 16:00:26,113][20381] Updated weights for policy 1, policy_version 1493859 (0.0005) [2023-12-27 16:00:26,168][20381] Updated weights for policy 1, policy_version 1493869 (0.0005) [2023-12-27 16:00:26,224][20381] Updated weights for policy 1, policy_version 1493879 (0.0007) [2023-12-27 16:00:26,411][20347] Updated weights for policy 0, policy_version 1495387 (0.0008) [2023-12-27 16:00:26,463][20347] Updated weights for policy 0, policy_version 1495397 (0.0007) [2023-12-27 16:00:26,515][20347] Updated weights for policy 0, policy_version 1495407 (0.0008) [2023-12-27 16:00:26,814][20381] Updated weights for policy 1, policy_version 1493889 (0.0006) [2023-12-27 16:00:26,875][20381] Updated weights for policy 1, policy_version 1493899 (0.0008) [2023-12-27 16:00:26,935][20381] Updated weights for policy 1, policy_version 1493909 (0.0008) [2023-12-27 16:00:26,998][20381] Updated weights for policy 1, policy_version 1493919 (0.0008) [2023-12-27 16:00:27,179][20347] Updated weights for policy 0, policy_version 1495417 (0.0008) [2023-12-27 16:00:27,228][20347] Updated weights for policy 0, policy_version 1495427 (0.0007) [2023-12-27 16:00:27,290][20347] Updated weights for policy 0, policy_version 1495437 (0.0007) [2023-12-27 16:00:27,352][20347] Updated weights for policy 0, policy_version 1495447 (0.0007) [2023-12-27 16:00:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 765386752. Throughput: 0: 10102.2, 1: 10257.4. Samples: 765372336. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:27,363][19170] Avg episode reward: [(0, '922.020'), (1, '990.800')] [2023-12-27 16:00:27,618][20381] Updated weights for policy 1, policy_version 1493929 (0.0008) [2023-12-27 16:00:27,673][20381] Updated weights for policy 1, policy_version 1493939 (0.0007) [2023-12-27 16:00:27,728][20381] Updated weights for policy 1, policy_version 1493949 (0.0008) [2023-12-27 16:00:28,038][20347] Updated weights for policy 0, policy_version 1495457 (0.0008) [2023-12-27 16:00:28,089][20347] Updated weights for policy 0, policy_version 1495467 (0.0007) [2023-12-27 16:00:28,143][20347] Updated weights for policy 0, policy_version 1495477 (0.0008) [2023-12-27 16:00:28,391][20381] Updated weights for policy 1, policy_version 1493959 (0.0007) [2023-12-27 16:00:28,455][20381] Updated weights for policy 1, policy_version 1493969 (0.0007) [2023-12-27 16:00:28,505][20381] Updated weights for policy 1, policy_version 1493979 (0.0007) [2023-12-27 16:00:28,900][20347] Updated weights for policy 0, policy_version 1495487 (0.0006) [2023-12-27 16:00:28,959][20347] Updated weights for policy 0, policy_version 1495497 (0.0008) [2023-12-27 16:00:29,024][20347] Updated weights for policy 0, policy_version 1495507 (0.0006) [2023-12-27 16:00:29,113][20381] Updated weights for policy 1, policy_version 1493989 (0.0007) [2023-12-27 16:00:29,172][20381] Updated weights for policy 1, policy_version 1493999 (0.0008) [2023-12-27 16:00:29,224][20381] Updated weights for policy 1, policy_version 1494009 (0.0008) [2023-12-27 16:00:29,709][20347] Updated weights for policy 0, policy_version 1495517 (0.0005) [2023-12-27 16:00:29,759][20347] Updated weights for policy 0, policy_version 1495527 (0.0007) [2023-12-27 16:00:29,809][20347] Updated weights for policy 0, policy_version 1495537 (0.0007) [2023-12-27 16:00:29,898][20381] Updated weights for policy 1, policy_version 1494019 (0.0008) [2023-12-27 16:00:29,963][20381] Updated weights for policy 1, policy_version 1494029 (0.0008) [2023-12-27 16:00:30,022][20381] Updated weights for policy 1, policy_version 1494039 (0.0008) [2023-12-27 16:00:30,484][20347] Updated weights for policy 0, policy_version 1495547 (0.0006) [2023-12-27 16:00:30,539][20347] Updated weights for policy 0, policy_version 1495557 (0.0007) [2023-12-27 16:00:30,599][20347] Updated weights for policy 0, policy_version 1495567 (0.0006) [2023-12-27 16:00:30,724][20381] Updated weights for policy 1, policy_version 1494049 (0.0008) [2023-12-27 16:00:30,773][20381] Updated weights for policy 1, policy_version 1494059 (0.0005) [2023-12-27 16:00:30,818][20381] Updated weights for policy 1, policy_version 1494069 (0.0007) [2023-12-27 16:00:30,872][20381] Updated weights for policy 1, policy_version 1494079 (0.0008) [2023-12-27 16:00:31,200][20347] Updated weights for policy 0, policy_version 1495577 (0.0007) [2023-12-27 16:00:31,262][20347] Updated weights for policy 0, policy_version 1495587 (0.0011) [2023-12-27 16:00:31,323][20347] Updated weights for policy 0, policy_version 1495597 (0.0010) [2023-12-27 16:00:31,387][20347] Updated weights for policy 0, policy_version 1495607 (0.0011) [2023-12-27 16:00:31,581][20381] Updated weights for policy 1, policy_version 1494089 (0.0010) [2023-12-27 16:00:31,640][20381] Updated weights for policy 1, policy_version 1494099 (0.0009) [2023-12-27 16:00:31,693][20381] Updated weights for policy 1, policy_version 1494109 (0.0008) [2023-12-27 16:00:32,126][20347] Updated weights for policy 0, policy_version 1495617 (0.0007) [2023-12-27 16:00:32,148][20291] KL-divergence is very high: 138.4890 [2023-12-27 16:00:32,159][20291] KL-divergence is very high: 207.1276 [2023-12-27 16:00:32,176][20291] KL-divergence is very high: 206.8917 [2023-12-27 16:00:32,182][20347] Updated weights for policy 0, policy_version 1495627 (0.0008) [2023-12-27 16:00:32,192][20291] KL-divergence is very high: 173.5609 [2023-12-27 16:00:32,203][20291] KL-divergence is very high: 134.7569 [2023-12-27 16:00:32,237][20347] Updated weights for policy 0, policy_version 1495637 (0.0007) [2023-12-27 16:00:32,315][20381] Updated weights for policy 1, policy_version 1494119 (0.0007) [2023-12-27 16:00:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 765485056. Throughput: 0: 10115.3, 1: 10331.9. Samples: 765495728. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:32,363][19170] Avg episode reward: [(0, '824.410'), (1, '981.140')] [2023-12-27 16:00:32,381][20381] Updated weights for policy 1, policy_version 1494129 (0.0008) [2023-12-27 16:00:32,444][20381] Updated weights for policy 1, policy_version 1494139 (0.0008) [2023-12-27 16:00:32,881][20347] Updated weights for policy 0, policy_version 1495647 (0.0007) [2023-12-27 16:00:32,933][20347] Updated weights for policy 0, policy_version 1495657 (0.0006) [2023-12-27 16:00:32,989][20347] Updated weights for policy 0, policy_version 1495667 (0.0005) [2023-12-27 16:00:33,184][20381] Updated weights for policy 1, policy_version 1494149 (0.0010) [2023-12-27 16:00:33,232][20381] Updated weights for policy 1, policy_version 1494159 (0.0010) [2023-12-27 16:00:33,287][20381] Updated weights for policy 1, policy_version 1494169 (0.0010) [2023-12-27 16:00:33,627][20347] Updated weights for policy 0, policy_version 1495677 (0.0006) [2023-12-27 16:00:33,682][20347] Updated weights for policy 0, policy_version 1495687 (0.0007) [2023-12-27 16:00:33,737][20347] Updated weights for policy 0, policy_version 1495697 (0.0007) [2023-12-27 16:00:33,975][20381] Updated weights for policy 1, policy_version 1494179 (0.0010) [2023-12-27 16:00:34,024][20381] Updated weights for policy 1, policy_version 1494189 (0.0010) [2023-12-27 16:00:34,077][20381] Updated weights for policy 1, policy_version 1494199 (0.0010) [2023-12-27 16:00:34,466][20347] Updated weights for policy 0, policy_version 1495707 (0.0008) [2023-12-27 16:00:34,535][20347] Updated weights for policy 0, policy_version 1495717 (0.0006) [2023-12-27 16:00:34,606][20347] Updated weights for policy 0, policy_version 1495727 (0.0006) [2023-12-27 16:00:34,848][20381] Updated weights for policy 1, policy_version 1494209 (0.0010) [2023-12-27 16:00:34,916][20381] Updated weights for policy 1, policy_version 1494219 (0.0006) [2023-12-27 16:00:34,977][20381] Updated weights for policy 1, policy_version 1494229 (0.0007) [2023-12-27 16:00:35,035][20381] Updated weights for policy 1, policy_version 1494239 (0.0006) [2023-12-27 16:00:35,305][20347] Updated weights for policy 0, policy_version 1495737 (0.0008) [2023-12-27 16:00:35,360][20347] Updated weights for policy 0, policy_version 1495747 (0.0008) [2023-12-27 16:00:35,418][20347] Updated weights for policy 0, policy_version 1495757 (0.0007) [2023-12-27 16:00:35,475][20347] Updated weights for policy 0, policy_version 1495767 (0.0006) [2023-12-27 16:00:35,632][20381] Updated weights for policy 1, policy_version 1494249 (0.0005) [2023-12-27 16:00:35,680][20381] Updated weights for policy 1, policy_version 1494259 (0.0005) [2023-12-27 16:00:35,726][20381] Updated weights for policy 1, policy_version 1494269 (0.0005) [2023-12-27 16:00:36,166][20347] Updated weights for policy 0, policy_version 1495777 (0.0007) [2023-12-27 16:00:36,220][20347] Updated weights for policy 0, policy_version 1495787 (0.0006) [2023-12-27 16:00:36,271][20347] Updated weights for policy 0, policy_version 1495797 (0.0007) [2023-12-27 16:00:36,275][20291] KL-divergence is very high: 163.4216 [2023-12-27 16:00:36,281][20291] KL-divergence is very high: 179.4041 [2023-12-27 16:00:36,341][20381] Updated weights for policy 1, policy_version 1494279 (0.0008) [2023-12-27 16:00:36,398][20381] Updated weights for policy 1, policy_version 1494289 (0.0008) [2023-12-27 16:00:36,451][20381] Updated weights for policy 1, policy_version 1494299 (0.0008) [2023-12-27 16:00:36,925][20291] KL-divergence is very high: 116.2070 [2023-12-27 16:00:36,940][20347] Updated weights for policy 0, policy_version 1495807 (0.0008) [2023-12-27 16:00:36,991][20347] Updated weights for policy 0, policy_version 1495817 (0.0007) [2023-12-27 16:00:37,041][20347] Updated weights for policy 0, policy_version 1495827 (0.0007) [2023-12-27 16:00:37,125][20381] Updated weights for policy 1, policy_version 1494309 (0.0009) [2023-12-27 16:00:37,182][20381] Updated weights for policy 1, policy_version 1494319 (0.0011) [2023-12-27 16:00:37,241][20381] Updated weights for policy 1, policy_version 1494329 (0.0011) [2023-12-27 16:00:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 765591552. Throughput: 0: 10130.0, 1: 10281.2. Samples: 765556104. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:37,363][19170] Avg episode reward: [(0, '824.430'), (1, '980.620')] [2023-12-27 16:00:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001494336_382582784.pth... [2023-12-27 16:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001495832_383008768.pth... [2023-12-27 16:00:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001493120_382271488.pth [2023-12-27 16:00:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001494648_382705664.pth [2023-12-27 16:00:37,753][20347] Updated weights for policy 0, policy_version 1495837 (0.0007) [2023-12-27 16:00:37,815][20347] Updated weights for policy 0, policy_version 1495847 (0.0006) [2023-12-27 16:00:37,868][20347] Updated weights for policy 0, policy_version 1495857 (0.0005) [2023-12-27 16:00:37,930][20381] Updated weights for policy 1, policy_version 1494339 (0.0011) [2023-12-27 16:00:37,982][20381] Updated weights for policy 1, policy_version 1494349 (0.0010) [2023-12-27 16:00:38,035][20381] Updated weights for policy 1, policy_version 1494359 (0.0011) [2023-12-27 16:00:38,556][20347] Updated weights for policy 0, policy_version 1495867 (0.0008) [2023-12-27 16:00:38,621][20347] Updated weights for policy 0, policy_version 1495877 (0.0007) [2023-12-27 16:00:38,687][20347] Updated weights for policy 0, policy_version 1495887 (0.0008) [2023-12-27 16:00:38,779][20381] Updated weights for policy 1, policy_version 1494369 (0.0011) [2023-12-27 16:00:38,840][20381] Updated weights for policy 1, policy_version 1494379 (0.0006) [2023-12-27 16:00:38,902][20381] Updated weights for policy 1, policy_version 1494389 (0.0011) [2023-12-27 16:00:38,964][20381] Updated weights for policy 1, policy_version 1494399 (0.0008) [2023-12-27 16:00:39,288][20347] Updated weights for policy 0, policy_version 1495897 (0.0010) [2023-12-27 16:00:39,359][20347] Updated weights for policy 0, policy_version 1495907 (0.0008) [2023-12-27 16:00:39,432][20347] Updated weights for policy 0, policy_version 1495917 (0.0010) [2023-12-27 16:00:39,504][20347] Updated weights for policy 0, policy_version 1495927 (0.0008) [2023-12-27 16:00:39,680][20381] Updated weights for policy 1, policy_version 1494409 (0.0009) [2023-12-27 16:00:39,751][20381] Updated weights for policy 1, policy_version 1494419 (0.0007) [2023-12-27 16:00:39,817][20381] Updated weights for policy 1, policy_version 1494429 (0.0007) [2023-12-27 16:00:40,186][20347] Updated weights for policy 0, policy_version 1495937 (0.0008) [2023-12-27 16:00:40,246][20347] Updated weights for policy 0, policy_version 1495947 (0.0008) [2023-12-27 16:00:40,307][20347] Updated weights for policy 0, policy_version 1495957 (0.0008) [2023-12-27 16:00:40,511][20381] Updated weights for policy 1, policy_version 1494439 (0.0008) [2023-12-27 16:00:40,561][20381] Updated weights for policy 1, policy_version 1494449 (0.0007) [2023-12-27 16:00:40,608][20381] Updated weights for policy 1, policy_version 1494459 (0.0007) [2023-12-27 16:00:41,019][20347] Updated weights for policy 0, policy_version 1495967 (0.0008) [2023-12-27 16:00:41,081][20347] Updated weights for policy 0, policy_version 1495977 (0.0007) [2023-12-27 16:00:41,145][20347] Updated weights for policy 0, policy_version 1495987 (0.0007) [2023-12-27 16:00:41,335][20381] Updated weights for policy 1, policy_version 1494469 (0.0008) [2023-12-27 16:00:41,398][20381] Updated weights for policy 1, policy_version 1494479 (0.0009) [2023-12-27 16:00:41,463][20381] Updated weights for policy 1, policy_version 1494489 (0.0010) [2023-12-27 16:00:41,962][20347] Updated weights for policy 0, policy_version 1495997 (0.0006) [2023-12-27 16:00:42,024][20347] Updated weights for policy 0, policy_version 1496007 (0.0006) [2023-12-27 16:00:42,090][20347] Updated weights for policy 0, policy_version 1496017 (0.0005) [2023-12-27 16:00:42,255][20381] Updated weights for policy 1, policy_version 1494499 (0.0010) [2023-12-27 16:00:42,318][20381] Updated weights for policy 1, policy_version 1494509 (0.0011) [2023-12-27 16:00:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 765681664. Throughput: 0: 10086.8, 1: 10281.1. Samples: 765677004. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:42,363][19170] Avg episode reward: [(0, '873.250'), (1, '990.280')] [2023-12-27 16:00:42,386][20381] Updated weights for policy 1, policy_version 1494519 (0.0009) [2023-12-27 16:00:42,773][20347] Updated weights for policy 0, policy_version 1496027 (0.0007) [2023-12-27 16:00:42,837][20347] Updated weights for policy 0, policy_version 1496037 (0.0008) [2023-12-27 16:00:42,893][20347] Updated weights for policy 0, policy_version 1496047 (0.0010) [2023-12-27 16:00:43,015][20381] Updated weights for policy 1, policy_version 1494529 (0.0008) [2023-12-27 16:00:43,067][20381] Updated weights for policy 1, policy_version 1494539 (0.0009) [2023-12-27 16:00:43,123][20381] Updated weights for policy 1, policy_version 1494549 (0.0007) [2023-12-27 16:00:43,191][20381] Updated weights for policy 1, policy_version 1494559 (0.0007) [2023-12-27 16:00:43,556][20347] Updated weights for policy 0, policy_version 1496057 (0.0008) [2023-12-27 16:00:43,614][20347] Updated weights for policy 0, policy_version 1496067 (0.0006) [2023-12-27 16:00:43,666][20347] Updated weights for policy 0, policy_version 1496077 (0.0005) [2023-12-27 16:00:43,727][20347] Updated weights for policy 0, policy_version 1496087 (0.0006) [2023-12-27 16:00:43,815][20381] Updated weights for policy 1, policy_version 1494569 (0.0008) [2023-12-27 16:00:43,870][20381] Updated weights for policy 1, policy_version 1494579 (0.0007) [2023-12-27 16:00:43,926][20381] Updated weights for policy 1, policy_version 1494589 (0.0007) [2023-12-27 16:00:44,271][20347] Updated weights for policy 0, policy_version 1496097 (0.0008) [2023-12-27 16:00:44,321][20347] Updated weights for policy 0, policy_version 1496107 (0.0009) [2023-12-27 16:00:44,370][20347] Updated weights for policy 0, policy_version 1496117 (0.0008) [2023-12-27 16:00:44,556][20381] Updated weights for policy 1, policy_version 1494599 (0.0008) [2023-12-27 16:00:44,612][20381] Updated weights for policy 1, policy_version 1494609 (0.0007) [2023-12-27 16:00:44,668][20381] Updated weights for policy 1, policy_version 1494619 (0.0007) [2023-12-27 16:00:45,098][20347] Updated weights for policy 0, policy_version 1496127 (0.0007) [2023-12-27 16:00:45,159][20347] Updated weights for policy 0, policy_version 1496137 (0.0008) [2023-12-27 16:00:45,189][20291] KL-divergence is very high: 167.0332 [2023-12-27 16:00:45,196][20291] KL-divergence is very high: 178.0446 [2023-12-27 16:00:45,219][20347] Updated weights for policy 0, policy_version 1496147 (0.0007) [2023-12-27 16:00:45,220][20291] KL-divergence is very high: 153.6624 [2023-12-27 16:00:45,237][20291] KL-divergence is very high: 163.0675 [2023-12-27 16:00:45,242][20291] KL-divergence is very high: 145.2885 [2023-12-27 16:00:45,403][20381] Updated weights for policy 1, policy_version 1494629 (0.0010) [2023-12-27 16:00:45,463][20381] Updated weights for policy 1, policy_version 1494639 (0.0010) [2023-12-27 16:00:45,518][20381] Updated weights for policy 1, policy_version 1494649 (0.0009) [2023-12-27 16:00:45,962][20347] Updated weights for policy 0, policy_version 1496157 (0.0007) [2023-12-27 16:00:46,022][20347] Updated weights for policy 0, policy_version 1496167 (0.0007) [2023-12-27 16:00:46,077][20347] Updated weights for policy 0, policy_version 1496177 (0.0006) [2023-12-27 16:00:46,101][20381] Updated weights for policy 1, policy_version 1494659 (0.0005) [2023-12-27 16:00:46,166][20381] Updated weights for policy 1, policy_version 1494669 (0.0007) [2023-12-27 16:00:46,231][20381] Updated weights for policy 1, policy_version 1494679 (0.0005) [2023-12-27 16:00:46,737][20347] Updated weights for policy 0, policy_version 1496187 (0.0006) [2023-12-27 16:00:46,785][20347] Updated weights for policy 0, policy_version 1496197 (0.0005) [2023-12-27 16:00:46,836][20347] Updated weights for policy 0, policy_version 1496207 (0.0005) [2023-12-27 16:00:46,872][20381] Updated weights for policy 1, policy_version 1494689 (0.0006) [2023-12-27 16:00:46,930][20381] Updated weights for policy 1, policy_version 1494699 (0.0008) [2023-12-27 16:00:46,979][20381] Updated weights for policy 1, policy_version 1494709 (0.0008) [2023-12-27 16:00:47,040][20381] Updated weights for policy 1, policy_version 1494719 (0.0005) [2023-12-27 16:00:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 765788160. Throughput: 0: 10104.5, 1: 10294.0. Samples: 765798844. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:47,364][19170] Avg episode reward: [(0, '921.920'), (1, '1000.000')] [2023-12-27 16:00:47,467][20347] Updated weights for policy 0, policy_version 1496217 (0.0006) [2023-12-27 16:00:47,519][20347] Updated weights for policy 0, policy_version 1496227 (0.0009) [2023-12-27 16:00:47,570][20347] Updated weights for policy 0, policy_version 1496237 (0.0008) [2023-12-27 16:00:47,580][20381] Updated weights for policy 1, policy_version 1494729 (0.0005) [2023-12-27 16:00:47,624][20347] Updated weights for policy 0, policy_version 1496247 (0.0007) [2023-12-27 16:00:47,632][20381] Updated weights for policy 1, policy_version 1494739 (0.0005) [2023-12-27 16:00:47,686][20381] Updated weights for policy 1, policy_version 1494749 (0.0006) [2023-12-27 16:00:48,314][20347] Updated weights for policy 0, policy_version 1496257 (0.0007) [2023-12-27 16:00:48,346][20381] Updated weights for policy 1, policy_version 1494759 (0.0007) [2023-12-27 16:00:48,385][20347] Updated weights for policy 0, policy_version 1496267 (0.0008) [2023-12-27 16:00:48,407][20381] Updated weights for policy 1, policy_version 1494769 (0.0007) [2023-12-27 16:00:48,447][20347] Updated weights for policy 0, policy_version 1496277 (0.0008) [2023-12-27 16:00:48,468][20381] Updated weights for policy 1, policy_version 1494779 (0.0007) [2023-12-27 16:00:49,092][20347] Updated weights for policy 0, policy_version 1496287 (0.0006) [2023-12-27 16:00:49,133][20381] Updated weights for policy 1, policy_version 1494789 (0.0005) [2023-12-27 16:00:49,142][20347] Updated weights for policy 0, policy_version 1496297 (0.0009) [2023-12-27 16:00:49,188][20381] Updated weights for policy 1, policy_version 1494799 (0.0010) [2023-12-27 16:00:49,190][20347] Updated weights for policy 0, policy_version 1496307 (0.0010) [2023-12-27 16:00:49,250][20381] Updated weights for policy 1, policy_version 1494809 (0.0009) [2023-12-27 16:00:49,890][20347] Updated weights for policy 0, policy_version 1496317 (0.0010) [2023-12-27 16:00:49,908][20381] Updated weights for policy 1, policy_version 1494819 (0.0009) [2023-12-27 16:00:49,956][20347] Updated weights for policy 0, policy_version 1496327 (0.0008) [2023-12-27 16:00:49,974][20381] Updated weights for policy 1, policy_version 1494829 (0.0010) [2023-12-27 16:00:50,023][20347] Updated weights for policy 0, policy_version 1496337 (0.0008) [2023-12-27 16:00:50,034][20381] Updated weights for policy 1, policy_version 1494839 (0.0010) [2023-12-27 16:00:50,714][20347] Updated weights for policy 0, policy_version 1496347 (0.0008) [2023-12-27 16:00:50,765][20347] Updated weights for policy 0, policy_version 1496357 (0.0008) [2023-12-27 16:00:50,795][20381] Updated weights for policy 1, policy_version 1494849 (0.0011) [2023-12-27 16:00:50,820][20347] Updated weights for policy 0, policy_version 1496367 (0.0010) [2023-12-27 16:00:50,854][20381] Updated weights for policy 1, policy_version 1494859 (0.0011) [2023-12-27 16:00:50,911][20381] Updated weights for policy 1, policy_version 1494869 (0.0011) [2023-12-27 16:00:50,961][20381] Updated weights for policy 1, policy_version 1494879 (0.0011) [2023-12-27 16:00:51,517][20347] Updated weights for policy 0, policy_version 1496377 (0.0010) [2023-12-27 16:00:51,574][20347] Updated weights for policy 0, policy_version 1496387 (0.0010) [2023-12-27 16:00:51,633][20347] Updated weights for policy 0, policy_version 1496397 (0.0007) [2023-12-27 16:00:51,695][20381] Updated weights for policy 1, policy_version 1494889 (0.0011) [2023-12-27 16:00:51,697][20347] Updated weights for policy 0, policy_version 1496407 (0.0006) [2023-12-27 16:00:51,755][20381] Updated weights for policy 1, policy_version 1494899 (0.0011) [2023-12-27 16:00:51,814][20381] Updated weights for policy 1, policy_version 1494909 (0.0007) [2023-12-27 16:00:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 765886464. Throughput: 0: 10114.0, 1: 10292.2. Samples: 765861020. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:52,363][19170] Avg episode reward: [(0, '951.490'), (1, '990.540')] [2023-12-27 16:00:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001494912_382730240.pth... [2023-12-27 16:00:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001493728_382427136.pth [2023-12-27 16:00:52,408][20347] Updated weights for policy 0, policy_version 1496417 (0.0011) [2023-12-27 16:00:52,442][20291] KL-divergence is very high: 174.6300 [2023-12-27 16:00:52,470][20347] Updated weights for policy 0, policy_version 1496427 (0.0009) [2023-12-27 16:00:52,492][20291] KL-divergence is very high: 184.0866 [2023-12-27 16:00:52,506][20381] Updated weights for policy 1, policy_version 1494919 (0.0006) [2023-12-27 16:00:52,529][20347] Updated weights for policy 0, policy_version 1496437 (0.0010) [2023-12-27 16:00:52,548][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001496440_383164416.pth... [2023-12-27 16:00:52,551][20292] KL-divergence is very high: 189.6409 [2023-12-27 16:00:52,553][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001495224_382853120.pth [2023-12-27 16:00:52,569][20381] Updated weights for policy 1, policy_version 1494929 (0.0006) [2023-12-27 16:00:52,595][20292] KL-divergence is very high: 185.4949 [2023-12-27 16:00:52,622][20381] Updated weights for policy 1, policy_version 1494939 (0.0006) [2023-12-27 16:00:52,643][20292] KL-divergence is very high: 122.2498 [2023-12-27 16:00:53,204][20347] Updated weights for policy 0, policy_version 1496447 (0.0009) [2023-12-27 16:00:53,257][20347] Updated weights for policy 0, policy_version 1496457 (0.0007) [2023-12-27 16:00:53,257][20381] Updated weights for policy 1, policy_version 1494949 (0.0007) [2023-12-27 16:00:53,306][20347] Updated weights for policy 0, policy_version 1496467 (0.0010) [2023-12-27 16:00:53,312][20381] Updated weights for policy 1, policy_version 1494959 (0.0007) [2023-12-27 16:00:53,369][20381] Updated weights for policy 1, policy_version 1494969 (0.0007) [2023-12-27 16:00:53,981][20347] Updated weights for policy 0, policy_version 1496477 (0.0011) [2023-12-27 16:00:54,045][20347] Updated weights for policy 0, policy_version 1496487 (0.0011) [2023-12-27 16:00:54,098][20381] Updated weights for policy 1, policy_version 1494979 (0.0008) [2023-12-27 16:00:54,105][20347] Updated weights for policy 0, policy_version 1496497 (0.0011) [2023-12-27 16:00:54,158][20381] Updated weights for policy 1, policy_version 1494989 (0.0008) [2023-12-27 16:00:54,220][20381] Updated weights for policy 1, policy_version 1494999 (0.0006) [2023-12-27 16:00:54,721][20347] Updated weights for policy 0, policy_version 1496507 (0.0009) [2023-12-27 16:00:54,781][20347] Updated weights for policy 0, policy_version 1496517 (0.0008) [2023-12-27 16:00:54,839][20347] Updated weights for policy 0, policy_version 1496527 (0.0010) [2023-12-27 16:00:54,879][20381] Updated weights for policy 1, policy_version 1495009 (0.0007) [2023-12-27 16:00:54,939][20381] Updated weights for policy 1, policy_version 1495019 (0.0005) [2023-12-27 16:00:54,998][20381] Updated weights for policy 1, policy_version 1495029 (0.0007) [2023-12-27 16:00:55,063][20381] Updated weights for policy 1, policy_version 1495039 (0.0007) [2023-12-27 16:00:55,493][20347] Updated weights for policy 0, policy_version 1496537 (0.0010) [2023-12-27 16:00:55,551][20347] Updated weights for policy 0, policy_version 1496547 (0.0010) [2023-12-27 16:00:55,608][20347] Updated weights for policy 0, policy_version 1496557 (0.0010) [2023-12-27 16:00:55,656][20347] Updated weights for policy 0, policy_version 1496567 (0.0010) [2023-12-27 16:00:55,712][20381] Updated weights for policy 1, policy_version 1495049 (0.0007) [2023-12-27 16:00:55,762][20381] Updated weights for policy 1, policy_version 1495059 (0.0007) [2023-12-27 16:00:55,806][20381] Updated weights for policy 1, policy_version 1495069 (0.0007) [2023-12-27 16:00:56,347][20347] Updated weights for policy 0, policy_version 1496577 (0.0010) [2023-12-27 16:00:56,407][20347] Updated weights for policy 0, policy_version 1496587 (0.0010) [2023-12-27 16:00:56,461][20347] Updated weights for policy 0, policy_version 1496597 (0.0010) [2023-12-27 16:00:56,508][20381] Updated weights for policy 1, policy_version 1495079 (0.0009) [2023-12-27 16:00:56,574][20381] Updated weights for policy 1, policy_version 1495089 (0.0010) [2023-12-27 16:00:56,623][20381] Updated weights for policy 1, policy_version 1495099 (0.0005) [2023-12-27 16:00:57,002][20347] Updated weights for policy 0, policy_version 1496607 (0.0007) [2023-12-27 16:00:57,049][20347] Updated weights for policy 0, policy_version 1496617 (0.0008) [2023-12-27 16:00:57,107][20347] Updated weights for policy 0, policy_version 1496627 (0.0007) [2023-12-27 16:00:57,161][20381] Updated weights for policy 1, policy_version 1495109 (0.0008) [2023-12-27 16:00:57,212][20381] Updated weights for policy 1, policy_version 1495119 (0.0010) [2023-12-27 16:00:57,255][20381] Updated weights for policy 1, policy_version 1495129 (0.0007) [2023-12-27 16:00:57,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 766001152. Throughput: 0: 10128.1, 1: 10222.8. Samples: 765982384. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:00:57,363][19170] Avg episode reward: [(0, '951.450'), (1, '981.310')] [2023-12-27 16:00:57,698][20347] Updated weights for policy 0, policy_version 1496637 (0.0006) [2023-12-27 16:00:57,759][20347] Updated weights for policy 0, policy_version 1496647 (0.0005) [2023-12-27 16:00:57,806][20347] Updated weights for policy 0, policy_version 1496657 (0.0005) [2023-12-27 16:00:57,858][20381] Updated weights for policy 1, policy_version 1495139 (0.0007) [2023-12-27 16:00:57,925][20381] Updated weights for policy 1, policy_version 1495149 (0.0010) [2023-12-27 16:00:57,982][20381] Updated weights for policy 1, policy_version 1495159 (0.0011) [2023-12-27 16:00:58,350][20347] Updated weights for policy 0, policy_version 1496667 (0.0006) [2023-12-27 16:00:58,413][20347] Updated weights for policy 0, policy_version 1496677 (0.0008) [2023-12-27 16:00:58,483][20347] Updated weights for policy 0, policy_version 1496687 (0.0008) [2023-12-27 16:00:58,654][20381] Updated weights for policy 1, policy_version 1495169 (0.0008) [2023-12-27 16:00:58,714][20381] Updated weights for policy 1, policy_version 1495179 (0.0010) [2023-12-27 16:00:58,776][20381] Updated weights for policy 1, policy_version 1495189 (0.0010) [2023-12-27 16:00:58,841][20381] Updated weights for policy 1, policy_version 1495199 (0.0012) [2023-12-27 16:00:59,244][20347] Updated weights for policy 0, policy_version 1496697 (0.0008) [2023-12-27 16:00:59,305][20347] Updated weights for policy 0, policy_version 1496707 (0.0007) [2023-12-27 16:00:59,374][20347] Updated weights for policy 0, policy_version 1496717 (0.0008) [2023-12-27 16:00:59,435][20347] Updated weights for policy 0, policy_version 1496727 (0.0009) [2023-12-27 16:00:59,606][20381] Updated weights for policy 1, policy_version 1495209 (0.0007) [2023-12-27 16:00:59,664][20381] Updated weights for policy 1, policy_version 1495219 (0.0010) [2023-12-27 16:00:59,716][20381] Updated weights for policy 1, policy_version 1495229 (0.0010) [2023-12-27 16:01:00,146][20347] Updated weights for policy 0, policy_version 1496737 (0.0010) [2023-12-27 16:01:00,201][20347] Updated weights for policy 0, policy_version 1496747 (0.0010) [2023-12-27 16:01:00,260][20347] Updated weights for policy 0, policy_version 1496757 (0.0010) [2023-12-27 16:01:00,370][20381] Updated weights for policy 1, policy_version 1495239 (0.0007) [2023-12-27 16:01:00,417][20381] Updated weights for policy 1, policy_version 1495249 (0.0005) [2023-12-27 16:01:00,473][20381] Updated weights for policy 1, policy_version 1495259 (0.0005) [2023-12-27 16:01:00,885][20347] Updated weights for policy 0, policy_version 1496767 (0.0010) [2023-12-27 16:01:00,937][20347] Updated weights for policy 0, policy_version 1496777 (0.0010) [2023-12-27 16:01:00,990][20347] Updated weights for policy 0, policy_version 1496787 (0.0010) [2023-12-27 16:01:01,128][20381] Updated weights for policy 1, policy_version 1495269 (0.0009) [2023-12-27 16:01:01,189][20381] Updated weights for policy 1, policy_version 1495279 (0.0008) [2023-12-27 16:01:01,252][20381] Updated weights for policy 1, policy_version 1495289 (0.0010) [2023-12-27 16:01:01,685][20347] Updated weights for policy 0, policy_version 1496797 (0.0009) [2023-12-27 16:01:01,741][20347] Updated weights for policy 0, policy_version 1496807 (0.0008) [2023-12-27 16:01:01,805][20347] Updated weights for policy 0, policy_version 1496817 (0.0007) [2023-12-27 16:01:01,923][20381] Updated weights for policy 1, policy_version 1495299 (0.0009) [2023-12-27 16:01:01,977][20381] Updated weights for policy 1, policy_version 1495309 (0.0008) [2023-12-27 16:01:02,029][20381] Updated weights for policy 1, policy_version 1495319 (0.0007) [2023-12-27 16:01:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 766099456. Throughput: 0: 10191.1, 1: 10281.3. Samples: 766109148. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:02,363][19170] Avg episode reward: [(0, '961.060'), (1, '990.300')] [2023-12-27 16:01:02,458][20347] Updated weights for policy 0, policy_version 1496827 (0.0005) [2023-12-27 16:01:02,523][20347] Updated weights for policy 0, policy_version 1496837 (0.0007) [2023-12-27 16:01:02,576][20347] Updated weights for policy 0, policy_version 1496847 (0.0008) [2023-12-27 16:01:02,757][20381] Updated weights for policy 1, policy_version 1495329 (0.0008) [2023-12-27 16:01:02,817][20381] Updated weights for policy 1, policy_version 1495339 (0.0008) [2023-12-27 16:01:02,872][20381] Updated weights for policy 1, policy_version 1495349 (0.0009) [2023-12-27 16:01:02,920][20381] Updated weights for policy 1, policy_version 1495359 (0.0007) [2023-12-27 16:01:02,925][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 16:01:03,184][20347] Updated weights for policy 0, policy_version 1496857 (0.0005) [2023-12-27 16:01:03,242][20347] Updated weights for policy 0, policy_version 1496867 (0.0007) [2023-12-27 16:01:03,301][20347] Updated weights for policy 0, policy_version 1496877 (0.0007) [2023-12-27 16:01:03,354][20347] Updated weights for policy 0, policy_version 1496887 (0.0006) [2023-12-27 16:01:03,604][20381] Updated weights for policy 1, policy_version 1495369 (0.0008) [2023-12-27 16:01:03,657][20381] Updated weights for policy 1, policy_version 1495379 (0.0005) [2023-12-27 16:01:03,713][20381] Updated weights for policy 1, policy_version 1495389 (0.0008) [2023-12-27 16:01:04,017][20347] Updated weights for policy 0, policy_version 1496897 (0.0006) [2023-12-27 16:01:04,055][20291] KL-divergence is very high: 100.6382 [2023-12-27 16:01:04,077][20347] Updated weights for policy 0, policy_version 1496907 (0.0006) [2023-12-27 16:01:04,141][20347] Updated weights for policy 0, policy_version 1496917 (0.0011) [2023-12-27 16:01:04,367][20381] Updated weights for policy 1, policy_version 1495399 (0.0007) [2023-12-27 16:01:04,429][20381] Updated weights for policy 1, policy_version 1495409 (0.0007) [2023-12-27 16:01:04,492][20381] Updated weights for policy 1, policy_version 1495419 (0.0007) [2023-12-27 16:01:04,799][20347] Updated weights for policy 0, policy_version 1496927 (0.0007) [2023-12-27 16:01:04,858][20347] Updated weights for policy 0, policy_version 1496937 (0.0005) [2023-12-27 16:01:04,918][20347] Updated weights for policy 0, policy_version 1496947 (0.0005) [2023-12-27 16:01:05,080][20381] Updated weights for policy 1, policy_version 1495429 (0.0006) [2023-12-27 16:01:05,125][20381] Updated weights for policy 1, policy_version 1495439 (0.0005) [2023-12-27 16:01:05,171][20381] Updated weights for policy 1, policy_version 1495449 (0.0005) [2023-12-27 16:01:05,506][20347] Updated weights for policy 0, policy_version 1496957 (0.0008) [2023-12-27 16:01:05,558][20347] Updated weights for policy 0, policy_version 1496967 (0.0010) [2023-12-27 16:01:05,619][20347] Updated weights for policy 0, policy_version 1496977 (0.0010) [2023-12-27 16:01:05,716][20381] Updated weights for policy 1, policy_version 1495459 (0.0005) [2023-12-27 16:01:05,769][20381] Updated weights for policy 1, policy_version 1495469 (0.0007) [2023-12-27 16:01:05,825][20381] Updated weights for policy 1, policy_version 1495479 (0.0011) [2023-12-27 16:01:06,266][20347] Updated weights for policy 0, policy_version 1496987 (0.0010) [2023-12-27 16:01:06,336][20347] Updated weights for policy 0, policy_version 1496997 (0.0009) [2023-12-27 16:01:06,400][20347] Updated weights for policy 0, policy_version 1497007 (0.0011) [2023-12-27 16:01:06,538][20381] Updated weights for policy 1, policy_version 1495489 (0.0011) [2023-12-27 16:01:06,602][20381] Updated weights for policy 1, policy_version 1495499 (0.0011) [2023-12-27 16:01:06,664][20381] Updated weights for policy 1, policy_version 1495509 (0.0011) [2023-12-27 16:01:06,717][20381] Updated weights for policy 1, policy_version 1495519 (0.0011) [2023-12-27 16:01:07,070][20347] Updated weights for policy 0, policy_version 1497017 (0.0009) [2023-12-27 16:01:07,125][20347] Updated weights for policy 0, policy_version 1497027 (0.0007) [2023-12-27 16:01:07,187][20347] Updated weights for policy 0, policy_version 1497037 (0.0005) [2023-12-27 16:01:07,248][20347] Updated weights for policy 0, policy_version 1497047 (0.0005) [2023-12-27 16:01:07,333][20381] Updated weights for policy 1, policy_version 1495529 (0.0011) [2023-12-27 16:01:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 766205952. Throughput: 0: 10230.7, 1: 10270.2. Samples: 766173356. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:07,363][19170] Avg episode reward: [(0, '970.830'), (1, '990.300')] [2023-12-27 16:01:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001497048_383320064.pth... [2023-12-27 16:01:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001495832_383008768.pth [2023-12-27 16:01:07,395][20381] Updated weights for policy 1, policy_version 1495539 (0.0011) [2023-12-27 16:01:07,449][20381] Updated weights for policy 1, policy_version 1495549 (0.0011) [2023-12-27 16:01:07,467][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001495552_382894080.pth... [2023-12-27 16:01:07,471][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001494336_382582784.pth [2023-12-27 16:01:07,884][20347] Updated weights for policy 0, policy_version 1497057 (0.0010) [2023-12-27 16:01:07,937][20347] Updated weights for policy 0, policy_version 1497067 (0.0010) [2023-12-27 16:01:07,988][20347] Updated weights for policy 0, policy_version 1497077 (0.0010) [2023-12-27 16:01:08,090][20381] Updated weights for policy 1, policy_version 1495559 (0.0008) [2023-12-27 16:01:08,149][20381] Updated weights for policy 1, policy_version 1495569 (0.0005) [2023-12-27 16:01:08,206][20381] Updated weights for policy 1, policy_version 1495579 (0.0010) [2023-12-27 16:01:08,638][20347] Updated weights for policy 0, policy_version 1497087 (0.0009) [2023-12-27 16:01:08,693][20347] Updated weights for policy 0, policy_version 1497097 (0.0005) [2023-12-27 16:01:08,747][20347] Updated weights for policy 0, policy_version 1497107 (0.0005) [2023-12-27 16:01:08,881][20381] Updated weights for policy 1, policy_version 1495589 (0.0010) [2023-12-27 16:01:08,943][20381] Updated weights for policy 1, policy_version 1495599 (0.0008) [2023-12-27 16:01:09,004][20381] Updated weights for policy 1, policy_version 1495609 (0.0006) [2023-12-27 16:01:09,428][20347] Updated weights for policy 0, policy_version 1497117 (0.0007) [2023-12-27 16:01:09,484][20347] Updated weights for policy 0, policy_version 1497127 (0.0010) [2023-12-27 16:01:09,539][20347] Updated weights for policy 0, policy_version 1497137 (0.0009) [2023-12-27 16:01:09,727][20381] Updated weights for policy 1, policy_version 1495619 (0.0007) [2023-12-27 16:01:09,795][20381] Updated weights for policy 1, policy_version 1495629 (0.0009) [2023-12-27 16:01:09,852][20381] Updated weights for policy 1, policy_version 1495639 (0.0011) [2023-12-27 16:01:10,219][20347] Updated weights for policy 0, policy_version 1497147 (0.0010) [2023-12-27 16:01:10,268][20347] Updated weights for policy 0, policy_version 1497157 (0.0010) [2023-12-27 16:01:10,315][20347] Updated weights for policy 0, policy_version 1497167 (0.0010) [2023-12-27 16:01:10,566][20381] Updated weights for policy 1, policy_version 1495649 (0.0010) [2023-12-27 16:01:10,620][20381] Updated weights for policy 1, policy_version 1495659 (0.0007) [2023-12-27 16:01:10,672][20381] Updated weights for policy 1, policy_version 1495669 (0.0008) [2023-12-27 16:01:10,724][20381] Updated weights for policy 1, policy_version 1495679 (0.0007) [2023-12-27 16:01:10,975][20347] Updated weights for policy 0, policy_version 1497177 (0.0010) [2023-12-27 16:01:11,042][20347] Updated weights for policy 0, policy_version 1497187 (0.0007) [2023-12-27 16:01:11,110][20347] Updated weights for policy 0, policy_version 1497197 (0.0008) [2023-12-27 16:01:11,179][20347] Updated weights for policy 0, policy_version 1497207 (0.0008) [2023-12-27 16:01:11,519][20381] Updated weights for policy 1, policy_version 1495689 (0.0008) [2023-12-27 16:01:11,580][20381] Updated weights for policy 1, policy_version 1495699 (0.0008) [2023-12-27 16:01:11,655][20381] Updated weights for policy 1, policy_version 1495709 (0.0009) [2023-12-27 16:01:11,940][20347] Updated weights for policy 0, policy_version 1497217 (0.0007) [2023-12-27 16:01:12,006][20347] Updated weights for policy 0, policy_version 1497227 (0.0010) [2023-12-27 16:01:12,068][20347] Updated weights for policy 0, policy_version 1497237 (0.0010) [2023-12-27 16:01:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 766304256. Throughput: 0: 10274.7, 1: 10280.4. Samples: 766297320. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:12,363][19170] Avg episode reward: [(0, '951.260'), (1, '1000.000')] [2023-12-27 16:01:12,401][20381] Updated weights for policy 1, policy_version 1495719 (0.0007) [2023-12-27 16:01:12,453][20381] Updated weights for policy 1, policy_version 1495729 (0.0008) [2023-12-27 16:01:12,509][20381] Updated weights for policy 1, policy_version 1495739 (0.0007) [2023-12-27 16:01:12,744][20347] Updated weights for policy 0, policy_version 1497247 (0.0010) [2023-12-27 16:01:12,800][20347] Updated weights for policy 0, policy_version 1497257 (0.0010) [2023-12-27 16:01:12,861][20347] Updated weights for policy 0, policy_version 1497267 (0.0010) [2023-12-27 16:01:13,171][20381] Updated weights for policy 1, policy_version 1495749 (0.0008) [2023-12-27 16:01:13,228][20381] Updated weights for policy 1, policy_version 1495759 (0.0007) [2023-12-27 16:01:13,288][20381] Updated weights for policy 1, policy_version 1495769 (0.0007) [2023-12-27 16:01:13,528][20347] Updated weights for policy 0, policy_version 1497277 (0.0008) [2023-12-27 16:01:13,573][20347] Updated weights for policy 0, policy_version 1497287 (0.0007) [2023-12-27 16:01:13,619][20347] Updated weights for policy 0, policy_version 1497297 (0.0007) [2023-12-27 16:01:13,960][20381] Updated weights for policy 1, policy_version 1495779 (0.0007) [2023-12-27 16:01:14,023][20381] Updated weights for policy 1, policy_version 1495789 (0.0008) [2023-12-27 16:01:14,081][20381] Updated weights for policy 1, policy_version 1495799 (0.0008) [2023-12-27 16:01:14,284][20347] Updated weights for policy 0, policy_version 1497307 (0.0007) [2023-12-27 16:01:14,334][20347] Updated weights for policy 0, policy_version 1497317 (0.0005) [2023-12-27 16:01:14,382][20347] Updated weights for policy 0, policy_version 1497327 (0.0005) [2023-12-27 16:01:14,700][20381] Updated weights for policy 1, policy_version 1495809 (0.0009) [2023-12-27 16:01:14,776][20381] Updated weights for policy 1, policy_version 1495819 (0.0008) [2023-12-27 16:01:14,841][20381] Updated weights for policy 1, policy_version 1495829 (0.0006) [2023-12-27 16:01:14,900][20381] Updated weights for policy 1, policy_version 1495839 (0.0007) [2023-12-27 16:01:14,994][20347] Updated weights for policy 0, policy_version 1497337 (0.0006) [2023-12-27 16:01:15,056][20347] Updated weights for policy 0, policy_version 1497347 (0.0009) [2023-12-27 16:01:15,121][20347] Updated weights for policy 0, policy_version 1497357 (0.0008) [2023-12-27 16:01:15,178][20347] Updated weights for policy 0, policy_version 1497367 (0.0009) [2023-12-27 16:01:15,525][20381] Updated weights for policy 1, policy_version 1495849 (0.0006) [2023-12-27 16:01:15,581][20381] Updated weights for policy 1, policy_version 1495859 (0.0007) [2023-12-27 16:01:15,639][20381] Updated weights for policy 1, policy_version 1495869 (0.0007) [2023-12-27 16:01:15,890][20347] Updated weights for policy 0, policy_version 1497377 (0.0007) [2023-12-27 16:01:15,943][20347] Updated weights for policy 0, policy_version 1497387 (0.0007) [2023-12-27 16:01:15,991][20347] Updated weights for policy 0, policy_version 1497397 (0.0008) [2023-12-27 16:01:16,240][20381] Updated weights for policy 1, policy_version 1495879 (0.0006) [2023-12-27 16:01:16,286][20381] Updated weights for policy 1, policy_version 1495889 (0.0005) [2023-12-27 16:01:16,340][20381] Updated weights for policy 1, policy_version 1495899 (0.0005) [2023-12-27 16:01:16,661][20347] Updated weights for policy 0, policy_version 1497407 (0.0007) [2023-12-27 16:01:16,719][20347] Updated weights for policy 0, policy_version 1497417 (0.0009) [2023-12-27 16:01:16,774][20347] Updated weights for policy 0, policy_version 1497427 (0.0008) [2023-12-27 16:01:16,922][20381] Updated weights for policy 1, policy_version 1495909 (0.0006) [2023-12-27 16:01:16,970][20381] Updated weights for policy 1, policy_version 1495919 (0.0010) [2023-12-27 16:01:17,024][20381] Updated weights for policy 1, policy_version 1495929 (0.0009) [2023-12-27 16:01:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 766410752. Throughput: 0: 10282.1, 1: 10270.8. Samples: 766420608. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:17,363][19170] Avg episode reward: [(0, '921.940'), (1, '1000.000')] [2023-12-27 16:01:17,383][20347] Updated weights for policy 0, policy_version 1497437 (0.0008) [2023-12-27 16:01:17,434][20347] Updated weights for policy 0, policy_version 1497447 (0.0008) [2023-12-27 16:01:17,493][20347] Updated weights for policy 0, policy_version 1497457 (0.0008) [2023-12-27 16:01:17,760][20381] Updated weights for policy 1, policy_version 1495939 (0.0008) [2023-12-27 16:01:17,818][20381] Updated weights for policy 1, policy_version 1495949 (0.0007) [2023-12-27 16:01:17,871][20381] Updated weights for policy 1, policy_version 1495959 (0.0008) [2023-12-27 16:01:18,172][20347] Updated weights for policy 0, policy_version 1497467 (0.0006) [2023-12-27 16:01:18,208][20291] KL-divergence is very high: 164.2631 [2023-12-27 16:01:18,228][20347] Updated weights for policy 0, policy_version 1497477 (0.0008) [2023-12-27 16:01:18,252][20291] KL-divergence is very high: 242.1197 [2023-12-27 16:01:18,287][20347] Updated weights for policy 0, policy_version 1497487 (0.0007) [2023-12-27 16:01:18,298][20291] KL-divergence is very high: 170.0428 [2023-12-27 16:01:18,582][20381] Updated weights for policy 1, policy_version 1495969 (0.0007) [2023-12-27 16:01:18,646][20381] Updated weights for policy 1, policy_version 1495979 (0.0008) [2023-12-27 16:01:18,707][20381] Updated weights for policy 1, policy_version 1495989 (0.0006) [2023-12-27 16:01:18,768][20381] Updated weights for policy 1, policy_version 1495999 (0.0007) [2023-12-27 16:01:18,982][20347] Updated weights for policy 0, policy_version 1497497 (0.0008) [2023-12-27 16:01:19,045][20347] Updated weights for policy 0, policy_version 1497507 (0.0006) [2023-12-27 16:01:19,106][20347] Updated weights for policy 0, policy_version 1497517 (0.0006) [2023-12-27 16:01:19,166][20347] Updated weights for policy 0, policy_version 1497527 (0.0008) [2023-12-27 16:01:19,490][20381] Updated weights for policy 1, policy_version 1496009 (0.0008) [2023-12-27 16:01:19,543][20381] Updated weights for policy 1, policy_version 1496019 (0.0008) [2023-12-27 16:01:19,597][20381] Updated weights for policy 1, policy_version 1496029 (0.0008) [2023-12-27 16:01:19,811][20347] Updated weights for policy 0, policy_version 1497537 (0.0008) [2023-12-27 16:01:19,876][20347] Updated weights for policy 0, policy_version 1497547 (0.0008) [2023-12-27 16:01:19,942][20347] Updated weights for policy 0, policy_version 1497557 (0.0008) [2023-12-27 16:01:20,363][20381] Updated weights for policy 1, policy_version 1496039 (0.0008) [2023-12-27 16:01:20,419][20381] Updated weights for policy 1, policy_version 1496049 (0.0008) [2023-12-27 16:01:20,475][20381] Updated weights for policy 1, policy_version 1496059 (0.0008) [2023-12-27 16:01:20,628][20347] Updated weights for policy 0, policy_version 1497567 (0.0008) [2023-12-27 16:01:20,686][20347] Updated weights for policy 0, policy_version 1497577 (0.0008) [2023-12-27 16:01:20,739][20347] Updated weights for policy 0, policy_version 1497587 (0.0008) [2023-12-27 16:01:21,210][20381] Updated weights for policy 1, policy_version 1496069 (0.0007) [2023-12-27 16:01:21,273][20381] Updated weights for policy 1, policy_version 1496079 (0.0008) [2023-12-27 16:01:21,340][20381] Updated weights for policy 1, policy_version 1496089 (0.0008) [2023-12-27 16:01:21,518][20347] Updated weights for policy 0, policy_version 1497597 (0.0009) [2023-12-27 16:01:21,578][20347] Updated weights for policy 0, policy_version 1497607 (0.0008) [2023-12-27 16:01:21,646][20347] Updated weights for policy 0, policy_version 1497617 (0.0008) [2023-12-27 16:01:22,060][20381] Updated weights for policy 1, policy_version 1496099 (0.0009) [2023-12-27 16:01:22,116][20381] Updated weights for policy 1, policy_version 1496109 (0.0008) [2023-12-27 16:01:22,170][20381] Updated weights for policy 1, policy_version 1496119 (0.0009) [2023-12-27 16:01:22,325][20347] Updated weights for policy 0, policy_version 1497627 (0.0008) [2023-12-27 16:01:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 766509056. Throughput: 0: 10282.7, 1: 10254.8. Samples: 766480292. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:22,363][19170] Avg episode reward: [(0, '931.760'), (1, '1000.000')] [2023-12-27 16:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001496128_383041536.pth... [2023-12-27 16:01:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001494912_382730240.pth [2023-12-27 16:01:22,394][20347] Updated weights for policy 0, policy_version 1497637 (0.0008) [2023-12-27 16:01:22,453][20347] Updated weights for policy 0, policy_version 1497647 (0.0008) [2023-12-27 16:01:22,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001497656_383475712.pth... [2023-12-27 16:01:22,506][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001496440_383164416.pth [2023-12-27 16:01:22,886][20381] Updated weights for policy 1, policy_version 1496129 (0.0009) [2023-12-27 16:01:22,936][20381] Updated weights for policy 1, policy_version 1496139 (0.0011) [2023-12-27 16:01:22,992][20381] Updated weights for policy 1, policy_version 1496149 (0.0010) [2023-12-27 16:01:23,049][20381] Updated weights for policy 1, policy_version 1496159 (0.0011) [2023-12-27 16:01:23,165][20347] Updated weights for policy 0, policy_version 1497657 (0.0008) [2023-12-27 16:01:23,219][20347] Updated weights for policy 0, policy_version 1497667 (0.0007) [2023-12-27 16:01:23,272][20347] Updated weights for policy 0, policy_version 1497677 (0.0008) [2023-12-27 16:01:23,323][20347] Updated weights for policy 0, policy_version 1497687 (0.0007) [2023-12-27 16:01:23,752][20381] Updated weights for policy 1, policy_version 1496169 (0.0009) [2023-12-27 16:01:23,806][20381] Updated weights for policy 1, policy_version 1496179 (0.0008) [2023-12-27 16:01:23,864][20381] Updated weights for policy 1, policy_version 1496189 (0.0008) [2023-12-27 16:01:24,039][20347] Updated weights for policy 0, policy_version 1497697 (0.0007) [2023-12-27 16:01:24,099][20347] Updated weights for policy 0, policy_version 1497707 (0.0008) [2023-12-27 16:01:24,155][20347] Updated weights for policy 0, policy_version 1497717 (0.0008) [2023-12-27 16:01:24,540][20381] Updated weights for policy 1, policy_version 1496199 (0.0007) [2023-12-27 16:01:24,598][20381] Updated weights for policy 1, policy_version 1496209 (0.0007) [2023-12-27 16:01:24,656][20381] Updated weights for policy 1, policy_version 1496219 (0.0007) [2023-12-27 16:01:24,858][20347] Updated weights for policy 0, policy_version 1497727 (0.0008) [2023-12-27 16:01:24,919][20347] Updated weights for policy 0, policy_version 1497737 (0.0007) [2023-12-27 16:01:24,968][20347] Updated weights for policy 0, policy_version 1497747 (0.0008) [2023-12-27 16:01:25,338][20381] Updated weights for policy 1, policy_version 1496229 (0.0008) [2023-12-27 16:01:25,390][20381] Updated weights for policy 1, policy_version 1496239 (0.0008) [2023-12-27 16:01:25,449][20381] Updated weights for policy 1, policy_version 1496249 (0.0007) [2023-12-27 16:01:25,597][20347] Updated weights for policy 0, policy_version 1497757 (0.0006) [2023-12-27 16:01:25,654][20347] Updated weights for policy 0, policy_version 1497767 (0.0005) [2023-12-27 16:01:25,708][20347] Updated weights for policy 0, policy_version 1497777 (0.0007) [2023-12-27 16:01:26,070][20381] Updated weights for policy 1, policy_version 1496259 (0.0008) [2023-12-27 16:01:26,122][20381] Updated weights for policy 1, policy_version 1496269 (0.0007) [2023-12-27 16:01:26,180][20381] Updated weights for policy 1, policy_version 1496279 (0.0008) [2023-12-27 16:01:26,308][20347] Updated weights for policy 0, policy_version 1497787 (0.0007) [2023-12-27 16:01:26,359][20347] Updated weights for policy 0, policy_version 1497797 (0.0005) [2023-12-27 16:01:26,405][20347] Updated weights for policy 0, policy_version 1497807 (0.0005) [2023-12-27 16:01:26,832][20381] Updated weights for policy 1, policy_version 1496289 (0.0008) [2023-12-27 16:01:26,888][20381] Updated weights for policy 1, policy_version 1496299 (0.0008) [2023-12-27 16:01:26,934][20381] Updated weights for policy 1, policy_version 1496309 (0.0007) [2023-12-27 16:01:26,980][20381] Updated weights for policy 1, policy_version 1496319 (0.0007) [2023-12-27 16:01:26,994][20347] Updated weights for policy 0, policy_version 1497817 (0.0008) [2023-12-27 16:01:27,046][20347] Updated weights for policy 0, policy_version 1497827 (0.0007) [2023-12-27 16:01:27,095][20347] Updated weights for policy 0, policy_version 1497837 (0.0008) [2023-12-27 16:01:27,146][20347] Updated weights for policy 0, policy_version 1497847 (0.0007) [2023-12-27 16:01:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 766615552. Throughput: 0: 10281.1, 1: 10236.6. Samples: 766600296. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:27,363][19170] Avg episode reward: [(0, '961.030'), (1, '1000.000')] [2023-12-27 16:01:27,617][20381] Updated weights for policy 1, policy_version 1496329 (0.0008) [2023-12-27 16:01:27,673][20381] Updated weights for policy 1, policy_version 1496339 (0.0008) [2023-12-27 16:01:27,722][20381] Updated weights for policy 1, policy_version 1496349 (0.0007) [2023-12-27 16:01:27,879][20347] Updated weights for policy 0, policy_version 1497857 (0.0006) [2023-12-27 16:01:27,935][20347] Updated weights for policy 0, policy_version 1497867 (0.0008) [2023-12-27 16:01:27,983][20347] Updated weights for policy 0, policy_version 1497877 (0.0007) [2023-12-27 16:01:28,465][20381] Updated weights for policy 1, policy_version 1496359 (0.0008) [2023-12-27 16:01:28,523][20381] Updated weights for policy 1, policy_version 1496369 (0.0008) [2023-12-27 16:01:28,568][20347] Updated weights for policy 0, policy_version 1497887 (0.0008) [2023-12-27 16:01:28,582][20381] Updated weights for policy 1, policy_version 1496379 (0.0007) [2023-12-27 16:01:28,626][20347] Updated weights for policy 0, policy_version 1497897 (0.0008) [2023-12-27 16:01:28,675][20347] Updated weights for policy 0, policy_version 1497907 (0.0008) [2023-12-27 16:01:29,198][20381] Updated weights for policy 1, policy_version 1496389 (0.0006) [2023-12-27 16:01:29,255][20381] Updated weights for policy 1, policy_version 1496399 (0.0008) [2023-12-27 16:01:29,309][20381] Updated weights for policy 1, policy_version 1496409 (0.0008) [2023-12-27 16:01:29,410][20347] Updated weights for policy 0, policy_version 1497917 (0.0008) [2023-12-27 16:01:29,468][20347] Updated weights for policy 0, policy_version 1497927 (0.0007) [2023-12-27 16:01:29,525][20347] Updated weights for policy 0, policy_version 1497937 (0.0007) [2023-12-27 16:01:30,055][20381] Updated weights for policy 1, policy_version 1496419 (0.0007) [2023-12-27 16:01:30,129][20381] Updated weights for policy 1, policy_version 1496429 (0.0005) [2023-12-27 16:01:30,173][20347] Updated weights for policy 0, policy_version 1497947 (0.0008) [2023-12-27 16:01:30,188][20381] Updated weights for policy 1, policy_version 1496439 (0.0006) [2023-12-27 16:01:30,240][20347] Updated weights for policy 0, policy_version 1497957 (0.0007) [2023-12-27 16:01:30,301][20347] Updated weights for policy 0, policy_version 1497967 (0.0007) [2023-12-27 16:01:30,798][20381] Updated weights for policy 1, policy_version 1496449 (0.0007) [2023-12-27 16:01:30,850][20381] Updated weights for policy 1, policy_version 1496459 (0.0008) [2023-12-27 16:01:30,907][20381] Updated weights for policy 1, policy_version 1496469 (0.0008) [2023-12-27 16:01:30,953][20347] Updated weights for policy 0, policy_version 1497977 (0.0007) [2023-12-27 16:01:30,963][20381] Updated weights for policy 1, policy_version 1496479 (0.0008) [2023-12-27 16:01:31,003][20347] Updated weights for policy 0, policy_version 1497987 (0.0006) [2023-12-27 16:01:31,062][20347] Updated weights for policy 0, policy_version 1497997 (0.0007) [2023-12-27 16:01:31,111][20347] Updated weights for policy 0, policy_version 1498007 (0.0008) [2023-12-27 16:01:31,743][20381] Updated weights for policy 1, policy_version 1496489 (0.0009) [2023-12-27 16:01:31,801][20381] Updated weights for policy 1, policy_version 1496499 (0.0006) [2023-12-27 16:01:31,807][20347] Updated weights for policy 0, policy_version 1498017 (0.0006) [2023-12-27 16:01:31,862][20347] Updated weights for policy 0, policy_version 1498027 (0.0007) [2023-12-27 16:01:31,863][20381] Updated weights for policy 1, policy_version 1496509 (0.0005) [2023-12-27 16:01:31,919][20347] Updated weights for policy 0, policy_version 1498037 (0.0006) [2023-12-27 16:01:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 766713856. Throughput: 0: 10339.7, 1: 10239.6. Samples: 766724908. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:32,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 16:01:32,499][20381] Updated weights for policy 1, policy_version 1496519 (0.0008) [2023-12-27 16:01:32,557][20347] Updated weights for policy 0, policy_version 1498047 (0.0006) [2023-12-27 16:01:32,563][20381] Updated weights for policy 1, policy_version 1496529 (0.0008) [2023-12-27 16:01:32,614][20347] Updated weights for policy 0, policy_version 1498057 (0.0008) [2023-12-27 16:01:32,620][20381] Updated weights for policy 1, policy_version 1496539 (0.0007) [2023-12-27 16:01:32,666][20347] Updated weights for policy 0, policy_version 1498067 (0.0006) [2023-12-27 16:01:33,245][20347] Updated weights for policy 0, policy_version 1498077 (0.0008) [2023-12-27 16:01:33,254][20381] Updated weights for policy 1, policy_version 1496549 (0.0008) [2023-12-27 16:01:33,304][20347] Updated weights for policy 0, policy_version 1498087 (0.0007) [2023-12-27 16:01:33,305][20381] Updated weights for policy 1, policy_version 1496559 (0.0010) [2023-12-27 16:01:33,356][20347] Updated weights for policy 0, policy_version 1498098 (0.0006) [2023-12-27 16:01:33,356][20381] Updated weights for policy 1, policy_version 1496569 (0.0010) [2023-12-27 16:01:33,926][20347] Updated weights for policy 0, policy_version 1498108 (0.0006) [2023-12-27 16:01:33,974][20381] Updated weights for policy 1, policy_version 1496579 (0.0008) [2023-12-27 16:01:33,981][20347] Updated weights for policy 0, policy_version 1498118 (0.0009) [2023-12-27 16:01:34,036][20347] Updated weights for policy 0, policy_version 1498128 (0.0010) [2023-12-27 16:01:34,037][20381] Updated weights for policy 1, policy_version 1496589 (0.0006) [2023-12-27 16:01:34,098][20381] Updated weights for policy 1, policy_version 1496599 (0.0006) [2023-12-27 16:01:34,717][20347] Updated weights for policy 0, policy_version 1498138 (0.0011) [2023-12-27 16:01:34,771][20347] Updated weights for policy 0, policy_version 1498148 (0.0010) [2023-12-27 16:01:34,801][20381] Updated weights for policy 1, policy_version 1496609 (0.0008) [2023-12-27 16:01:34,819][20347] Updated weights for policy 0, policy_version 1498158 (0.0006) [2023-12-27 16:01:34,856][20381] Updated weights for policy 1, policy_version 1496619 (0.0010) [2023-12-27 16:01:34,875][20347] Updated weights for policy 0, policy_version 1498168 (0.0006) [2023-12-27 16:01:34,904][20381] Updated weights for policy 1, policy_version 1496629 (0.0010) [2023-12-27 16:01:34,952][20381] Updated weights for policy 1, policy_version 1496639 (0.0010) [2023-12-27 16:01:35,455][20347] Updated weights for policy 0, policy_version 1498178 (0.0010) [2023-12-27 16:01:35,512][20347] Updated weights for policy 0, policy_version 1498188 (0.0010) [2023-12-27 16:01:35,565][20347] Updated weights for policy 0, policy_version 1498198 (0.0010) [2023-12-27 16:01:35,659][20381] Updated weights for policy 1, policy_version 1496649 (0.0011) [2023-12-27 16:01:35,704][20381] Updated weights for policy 1, policy_version 1496659 (0.0010) [2023-12-27 16:01:35,762][20381] Updated weights for policy 1, policy_version 1496669 (0.0010) [2023-12-27 16:01:36,244][20347] Updated weights for policy 0, policy_version 1498208 (0.0008) [2023-12-27 16:01:36,304][20347] Updated weights for policy 0, policy_version 1498218 (0.0006) [2023-12-27 16:01:36,364][20347] Updated weights for policy 0, policy_version 1498228 (0.0006) [2023-12-27 16:01:36,473][20381] Updated weights for policy 1, policy_version 1496679 (0.0008) [2023-12-27 16:01:36,524][20381] Updated weights for policy 1, policy_version 1496689 (0.0009) [2023-12-27 16:01:36,580][20381] Updated weights for policy 1, policy_version 1496699 (0.0008) [2023-12-27 16:01:36,921][20347] Updated weights for policy 0, policy_version 1498238 (0.0009) [2023-12-27 16:01:36,969][20347] Updated weights for policy 0, policy_version 1498248 (0.0010) [2023-12-27 16:01:37,022][20347] Updated weights for policy 0, policy_version 1498258 (0.0010) [2023-12-27 16:01:37,328][20381] Updated weights for policy 1, policy_version 1496709 (0.0008) [2023-12-27 16:01:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 766820352. Throughput: 0: 10412.3, 1: 10231.6. Samples: 766789996. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:37,363][19170] Avg episode reward: [(0, '980.620'), (1, '1000.000')] [2023-12-27 16:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001498264_383631360.pth... [2023-12-27 16:01:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001497048_383320064.pth [2023-12-27 16:01:37,387][20381] Updated weights for policy 1, policy_version 1496719 (0.0009) [2023-12-27 16:01:37,446][20381] Updated weights for policy 1, policy_version 1496729 (0.0008) [2023-12-27 16:01:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001496736_383197184.pth... [2023-12-27 16:01:37,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001495552_382894080.pth [2023-12-27 16:01:37,665][20347] Updated weights for policy 0, policy_version 1498268 (0.0010) [2023-12-27 16:01:37,727][20347] Updated weights for policy 0, policy_version 1498278 (0.0011) [2023-12-27 16:01:37,789][20347] Updated weights for policy 0, policy_version 1498288 (0.0010) [2023-12-27 16:01:38,119][20381] Updated weights for policy 1, policy_version 1496739 (0.0007) [2023-12-27 16:01:38,176][20381] Updated weights for policy 1, policy_version 1496749 (0.0009) [2023-12-27 16:01:38,233][20381] Updated weights for policy 1, policy_version 1496759 (0.0009) [2023-12-27 16:01:38,503][20347] Updated weights for policy 0, policy_version 1498298 (0.0009) [2023-12-27 16:01:38,567][20347] Updated weights for policy 0, policy_version 1498308 (0.0006) [2023-12-27 16:01:38,629][20347] Updated weights for policy 0, policy_version 1498318 (0.0010) [2023-12-27 16:01:38,692][20347] Updated weights for policy 0, policy_version 1498328 (0.0011) [2023-12-27 16:01:38,848][20381] Updated weights for policy 1, policy_version 1496769 (0.0005) [2023-12-27 16:01:38,909][20381] Updated weights for policy 1, policy_version 1496779 (0.0006) [2023-12-27 16:01:38,955][20381] Updated weights for policy 1, policy_version 1496789 (0.0007) [2023-12-27 16:01:39,008][20381] Updated weights for policy 1, policy_version 1496799 (0.0007) [2023-12-27 16:01:39,373][20347] Updated weights for policy 0, policy_version 1498338 (0.0010) [2023-12-27 16:01:39,440][20347] Updated weights for policy 0, policy_version 1498348 (0.0009) [2023-12-27 16:01:39,462][20291] KL-divergence is very high: 131.0787 [2023-12-27 16:01:39,494][20347] Updated weights for policy 0, policy_version 1498358 (0.0007) [2023-12-27 16:01:39,687][20381] Updated weights for policy 1, policy_version 1496809 (0.0007) [2023-12-27 16:01:39,750][20381] Updated weights for policy 1, policy_version 1496819 (0.0007) [2023-12-27 16:01:39,810][20381] Updated weights for policy 1, policy_version 1496829 (0.0008) [2023-12-27 16:01:40,209][20347] Updated weights for policy 0, policy_version 1498368 (0.0010) [2023-12-27 16:01:40,270][20347] Updated weights for policy 0, policy_version 1498378 (0.0010) [2023-12-27 16:01:40,327][20347] Updated weights for policy 0, policy_version 1498388 (0.0007) [2023-12-27 16:01:40,532][20381] Updated weights for policy 1, policy_version 1496839 (0.0007) [2023-12-27 16:01:40,595][20381] Updated weights for policy 1, policy_version 1496849 (0.0006) [2023-12-27 16:01:40,656][20381] Updated weights for policy 1, policy_version 1496859 (0.0005) [2023-12-27 16:01:40,951][20347] Updated weights for policy 0, policy_version 1498398 (0.0008) [2023-12-27 16:01:41,010][20347] Updated weights for policy 0, policy_version 1498408 (0.0007) [2023-12-27 16:01:41,075][20347] Updated weights for policy 0, policy_version 1498418 (0.0009) [2023-12-27 16:01:41,358][20381] Updated weights for policy 1, policy_version 1496869 (0.0008) [2023-12-27 16:01:41,418][20381] Updated weights for policy 1, policy_version 1496879 (0.0009) [2023-12-27 16:01:41,478][20381] Updated weights for policy 1, policy_version 1496889 (0.0011) [2023-12-27 16:01:41,845][20347] Updated weights for policy 0, policy_version 1498428 (0.0008) [2023-12-27 16:01:41,901][20347] Updated weights for policy 0, policy_version 1498438 (0.0009) [2023-12-27 16:01:41,955][20347] Updated weights for policy 0, policy_version 1498448 (0.0010) [2023-12-27 16:01:42,211][20381] Updated weights for policy 1, policy_version 1496899 (0.0011) [2023-12-27 16:01:42,273][20381] Updated weights for policy 1, policy_version 1496909 (0.0010) [2023-12-27 16:01:42,327][20381] Updated weights for policy 1, policy_version 1496919 (0.0008) [2023-12-27 16:01:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 766918656. Throughput: 0: 10447.1, 1: 10237.1. Samples: 766913176. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:42,363][19170] Avg episode reward: [(0, '970.830'), (1, '1000.000')] [2023-12-27 16:01:42,714][20347] Updated weights for policy 0, policy_version 1498458 (0.0009) [2023-12-27 16:01:42,773][20347] Updated weights for policy 0, policy_version 1498468 (0.0007) [2023-12-27 16:01:42,826][20347] Updated weights for policy 0, policy_version 1498478 (0.0008) [2023-12-27 16:01:42,887][20347] Updated weights for policy 0, policy_version 1498488 (0.0008) [2023-12-27 16:01:43,009][20381] Updated weights for policy 1, policy_version 1496929 (0.0009) [2023-12-27 16:01:43,061][20381] Updated weights for policy 1, policy_version 1496939 (0.0007) [2023-12-27 16:01:43,113][20381] Updated weights for policy 1, policy_version 1496949 (0.0008) [2023-12-27 16:01:43,168][20381] Updated weights for policy 1, policy_version 1496959 (0.0006) [2023-12-27 16:01:43,601][20347] Updated weights for policy 0, policy_version 1498498 (0.0007) [2023-12-27 16:01:43,650][20347] Updated weights for policy 0, policy_version 1498508 (0.0008) [2023-12-27 16:01:43,699][20347] Updated weights for policy 0, policy_version 1498519 (0.0009) [2023-12-27 16:01:43,762][20381] Updated weights for policy 1, policy_version 1496969 (0.0005) [2023-12-27 16:01:43,821][20381] Updated weights for policy 1, policy_version 1496979 (0.0006) [2023-12-27 16:01:43,875][20381] Updated weights for policy 1, policy_version 1496989 (0.0008) [2023-12-27 16:01:44,436][20347] Updated weights for policy 0, policy_version 1498529 (0.0010) [2023-12-27 16:01:44,464][20381] Updated weights for policy 1, policy_version 1496999 (0.0006) [2023-12-27 16:01:44,485][20347] Updated weights for policy 0, policy_version 1498539 (0.0010) [2023-12-27 16:01:44,517][20381] Updated weights for policy 1, policy_version 1497009 (0.0005) [2023-12-27 16:01:44,534][20347] Updated weights for policy 0, policy_version 1498549 (0.0010) [2023-12-27 16:01:44,573][20381] Updated weights for policy 1, policy_version 1497019 (0.0006) [2023-12-27 16:01:45,281][20347] Updated weights for policy 0, policy_version 1498559 (0.0007) [2023-12-27 16:01:45,326][20381] Updated weights for policy 1, policy_version 1497029 (0.0007) [2023-12-27 16:01:45,335][20347] Updated weights for policy 0, policy_version 1498569 (0.0007) [2023-12-27 16:01:45,381][20381] Updated weights for policy 1, policy_version 1497039 (0.0007) [2023-12-27 16:01:45,391][20347] Updated weights for policy 0, policy_version 1498579 (0.0005) [2023-12-27 16:01:45,431][20381] Updated weights for policy 1, policy_version 1497049 (0.0008) [2023-12-27 16:01:45,989][20347] Updated weights for policy 0, policy_version 1498589 (0.0005) [2023-12-27 16:01:46,044][20347] Updated weights for policy 0, policy_version 1498599 (0.0005) [2023-12-27 16:01:46,102][20347] Updated weights for policy 0, policy_version 1498609 (0.0005) [2023-12-27 16:01:46,159][20381] Updated weights for policy 1, policy_version 1497059 (0.0007) [2023-12-27 16:01:46,212][20381] Updated weights for policy 1, policy_version 1497069 (0.0006) [2023-12-27 16:01:46,265][20381] Updated weights for policy 1, policy_version 1497079 (0.0005) [2023-12-27 16:01:46,726][20347] Updated weights for policy 0, policy_version 1498619 (0.0006) [2023-12-27 16:01:46,779][20347] Updated weights for policy 0, policy_version 1498629 (0.0005) [2023-12-27 16:01:46,839][20347] Updated weights for policy 0, policy_version 1498639 (0.0005) [2023-12-27 16:01:46,846][20381] Updated weights for policy 1, policy_version 1497089 (0.0006) [2023-12-27 16:01:46,903][20381] Updated weights for policy 1, policy_version 1497099 (0.0008) [2023-12-27 16:01:46,961][20381] Updated weights for policy 1, policy_version 1497109 (0.0011) [2023-12-27 16:01:47,016][20381] Updated weights for policy 1, policy_version 1497119 (0.0011) [2023-12-27 16:01:47,360][20347] Updated weights for policy 0, policy_version 1498649 (0.0005) [2023-12-27 16:01:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 767025152. Throughput: 0: 10373.8, 1: 10212.2. Samples: 767035516. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:47,363][19170] Avg episode reward: [(0, '980.550'), (1, '1000.000')] [2023-12-27 16:01:47,411][20347] Updated weights for policy 0, policy_version 1498659 (0.0005) [2023-12-27 16:01:47,459][20347] Updated weights for policy 0, policy_version 1498669 (0.0005) [2023-12-27 16:01:47,515][20347] Updated weights for policy 0, policy_version 1498679 (0.0005) [2023-12-27 16:01:47,712][20381] Updated weights for policy 1, policy_version 1497129 (0.0006) [2023-12-27 16:01:47,772][20381] Updated weights for policy 1, policy_version 1497139 (0.0005) [2023-12-27 16:01:47,824][20381] Updated weights for policy 1, policy_version 1497149 (0.0005) [2023-12-27 16:01:48,047][20347] Updated weights for policy 0, policy_version 1498689 (0.0009) [2023-12-27 16:01:48,118][20347] Updated weights for policy 0, policy_version 1498699 (0.0006) [2023-12-27 16:01:48,173][20347] Updated weights for policy 0, policy_version 1498709 (0.0008) [2023-12-27 16:01:48,508][20381] Updated weights for policy 1, policy_version 1497159 (0.0007) [2023-12-27 16:01:48,569][20381] Updated weights for policy 1, policy_version 1497169 (0.0009) [2023-12-27 16:01:48,626][20381] Updated weights for policy 1, policy_version 1497179 (0.0008) [2023-12-27 16:01:48,796][20347] Updated weights for policy 0, policy_version 1498719 (0.0007) [2023-12-27 16:01:48,853][20347] Updated weights for policy 0, policy_version 1498729 (0.0007) [2023-12-27 16:01:48,906][20347] Updated weights for policy 0, policy_version 1498739 (0.0007) [2023-12-27 16:01:49,334][20381] Updated weights for policy 1, policy_version 1497189 (0.0008) [2023-12-27 16:01:49,397][20381] Updated weights for policy 1, policy_version 1497199 (0.0008) [2023-12-27 16:01:49,459][20381] Updated weights for policy 1, policy_version 1497209 (0.0008) [2023-12-27 16:01:49,622][20347] Updated weights for policy 0, policy_version 1498749 (0.0007) [2023-12-27 16:01:49,683][20347] Updated weights for policy 0, policy_version 1498759 (0.0008) [2023-12-27 16:01:49,741][20347] Updated weights for policy 0, policy_version 1498769 (0.0008) [2023-12-27 16:01:50,128][20381] Updated weights for policy 1, policy_version 1497219 (0.0008) [2023-12-27 16:01:50,182][20381] Updated weights for policy 1, policy_version 1497229 (0.0009) [2023-12-27 16:01:50,234][20381] Updated weights for policy 1, policy_version 1497239 (0.0010) [2023-12-27 16:01:50,435][20347] Updated weights for policy 0, policy_version 1498779 (0.0008) [2023-12-27 16:01:50,493][20347] Updated weights for policy 0, policy_version 1498789 (0.0007) [2023-12-27 16:01:50,544][20347] Updated weights for policy 0, policy_version 1498799 (0.0007) [2023-12-27 16:01:50,934][20381] Updated weights for policy 1, policy_version 1497249 (0.0010) [2023-12-27 16:01:50,995][20381] Updated weights for policy 1, policy_version 1497259 (0.0008) [2023-12-27 16:01:51,062][20381] Updated weights for policy 1, policy_version 1497269 (0.0007) [2023-12-27 16:01:51,131][20381] Updated weights for policy 1, policy_version 1497279 (0.0008) [2023-12-27 16:01:51,279][20347] Updated weights for policy 0, policy_version 1498809 (0.0008) [2023-12-27 16:01:51,345][20347] Updated weights for policy 0, policy_version 1498819 (0.0007) [2023-12-27 16:01:51,409][20347] Updated weights for policy 0, policy_version 1498829 (0.0008) [2023-12-27 16:01:51,471][20347] Updated weights for policy 0, policy_version 1498839 (0.0008) [2023-12-27 16:01:51,823][20381] Updated weights for policy 1, policy_version 1497289 (0.0006) [2023-12-27 16:01:51,896][20381] Updated weights for policy 1, policy_version 1497299 (0.0008) [2023-12-27 16:01:51,966][20381] Updated weights for policy 1, policy_version 1497309 (0.0008) [2023-12-27 16:01:52,182][20347] Updated weights for policy 0, policy_version 1498849 (0.0008) [2023-12-27 16:01:52,247][20347] Updated weights for policy 0, policy_version 1498859 (0.0008) [2023-12-27 16:01:52,314][20347] Updated weights for policy 0, policy_version 1498869 (0.0009) [2023-12-27 16:01:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 767131648. Throughput: 0: 10377.4, 1: 10165.4. Samples: 767097784. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:52,363][19170] Avg episode reward: [(0, '980.570'), (1, '1000.000')] [2023-12-27 16:01:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001497312_383344640.pth... [2023-12-27 16:01:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001498872_383787008.pth... [2023-12-27 16:01:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001497656_383475712.pth [2023-12-27 16:01:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001496128_383041536.pth [2023-12-27 16:01:52,641][20381] Updated weights for policy 1, policy_version 1497319 (0.0008) [2023-12-27 16:01:52,694][20381] Updated weights for policy 1, policy_version 1497329 (0.0008) [2023-12-27 16:01:52,754][20381] Updated weights for policy 1, policy_version 1497339 (0.0008) [2023-12-27 16:01:53,021][20347] Updated weights for policy 0, policy_version 1498879 (0.0008) [2023-12-27 16:01:53,081][20347] Updated weights for policy 0, policy_version 1498889 (0.0008) [2023-12-27 16:01:53,140][20347] Updated weights for policy 0, policy_version 1498899 (0.0007) [2023-12-27 16:01:53,481][20381] Updated weights for policy 1, policy_version 1497349 (0.0008) [2023-12-27 16:01:53,542][20381] Updated weights for policy 1, policy_version 1497359 (0.0009) [2023-12-27 16:01:53,606][20381] Updated weights for policy 1, policy_version 1497369 (0.0009) [2023-12-27 16:01:53,768][20347] Updated weights for policy 0, policy_version 1498909 (0.0008) [2023-12-27 16:01:53,826][20347] Updated weights for policy 0, policy_version 1498919 (0.0008) [2023-12-27 16:01:53,885][20347] Updated weights for policy 0, policy_version 1498929 (0.0009) [2023-12-27 16:01:54,293][20381] Updated weights for policy 1, policy_version 1497379 (0.0007) [2023-12-27 16:01:54,343][20381] Updated weights for policy 1, policy_version 1497389 (0.0007) [2023-12-27 16:01:54,407][20381] Updated weights for policy 1, policy_version 1497399 (0.0007) [2023-12-27 16:01:54,628][20347] Updated weights for policy 0, policy_version 1498939 (0.0008) [2023-12-27 16:01:54,689][20347] Updated weights for policy 0, policy_version 1498949 (0.0008) [2023-12-27 16:01:54,736][20291] KL-divergence is very high: 100.0266 [2023-12-27 16:01:54,755][20347] Updated weights for policy 0, policy_version 1498959 (0.0007) [2023-12-27 16:01:55,095][20381] Updated weights for policy 1, policy_version 1497409 (0.0008) [2023-12-27 16:01:55,153][20381] Updated weights for policy 1, policy_version 1497419 (0.0006) [2023-12-27 16:01:55,220][20381] Updated weights for policy 1, policy_version 1497429 (0.0006) [2023-12-27 16:01:55,273][20381] Updated weights for policy 1, policy_version 1497439 (0.0009) [2023-12-27 16:01:55,414][20347] Updated weights for policy 0, policy_version 1498969 (0.0007) [2023-12-27 16:01:55,468][20347] Updated weights for policy 0, policy_version 1498979 (0.0007) [2023-12-27 16:01:55,517][20347] Updated weights for policy 0, policy_version 1498989 (0.0008) [2023-12-27 16:01:55,569][20347] Updated weights for policy 0, policy_version 1498999 (0.0008) [2023-12-27 16:01:55,867][20381] Updated weights for policy 1, policy_version 1497449 (0.0010) [2023-12-27 16:01:55,930][20381] Updated weights for policy 1, policy_version 1497459 (0.0010) [2023-12-27 16:01:55,988][20381] Updated weights for policy 1, policy_version 1497469 (0.0010) [2023-12-27 16:01:56,330][20347] Updated weights for policy 0, policy_version 1499009 (0.0008) [2023-12-27 16:01:56,377][20347] Updated weights for policy 0, policy_version 1499019 (0.0009) [2023-12-27 16:01:56,436][20347] Updated weights for policy 0, policy_version 1499029 (0.0007) [2023-12-27 16:01:56,558][20381] Updated weights for policy 1, policy_version 1497479 (0.0007) [2023-12-27 16:01:56,613][20381] Updated weights for policy 1, policy_version 1497489 (0.0005) [2023-12-27 16:01:56,668][20381] Updated weights for policy 1, policy_version 1497499 (0.0007) [2023-12-27 16:01:57,111][20347] Updated weights for policy 0, policy_version 1499039 (0.0007) [2023-12-27 16:01:57,164][20347] Updated weights for policy 0, policy_version 1499049 (0.0007) [2023-12-27 16:01:57,212][20347] Updated weights for policy 0, policy_version 1499059 (0.0007) [2023-12-27 16:01:57,267][20381] Updated weights for policy 1, policy_version 1497509 (0.0008) [2023-12-27 16:01:57,324][20381] Updated weights for policy 1, policy_version 1497519 (0.0010) [2023-12-27 16:01:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 767229952. Throughput: 0: 10304.6, 1: 10172.7. Samples: 767218800. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:01:57,363][19170] Avg episode reward: [(0, '970.810'), (1, '980.690')] [2023-12-27 16:01:57,380][20381] Updated weights for policy 1, policy_version 1497529 (0.0010) [2023-12-27 16:01:57,957][20381] Updated weights for policy 1, policy_version 1497539 (0.0009) [2023-12-27 16:01:57,958][20347] Updated weights for policy 0, policy_version 1499069 (0.0007) [2023-12-27 16:01:58,004][20381] Updated weights for policy 1, policy_version 1497549 (0.0010) [2023-12-27 16:01:58,015][20347] Updated weights for policy 0, policy_version 1499079 (0.0006) [2023-12-27 16:01:58,061][20381] Updated weights for policy 1, policy_version 1497559 (0.0010) [2023-12-27 16:01:58,070][20347] Updated weights for policy 0, policy_version 1499089 (0.0005) [2023-12-27 16:01:58,799][20381] Updated weights for policy 1, policy_version 1497569 (0.0008) [2023-12-27 16:01:58,858][20347] Updated weights for policy 0, policy_version 1499099 (0.0007) [2023-12-27 16:01:58,871][20381] Updated weights for policy 1, policy_version 1497579 (0.0008) [2023-12-27 16:01:58,924][20347] Updated weights for policy 0, policy_version 1499109 (0.0006) [2023-12-27 16:01:58,930][20381] Updated weights for policy 1, policy_version 1497589 (0.0008) [2023-12-27 16:01:58,987][20347] Updated weights for policy 0, policy_version 1499119 (0.0008) [2023-12-27 16:01:58,989][20381] Updated weights for policy 1, policy_version 1497599 (0.0008) [2023-12-27 16:01:59,712][20381] Updated weights for policy 1, policy_version 1497609 (0.0011) [2023-12-27 16:01:59,763][20381] Updated weights for policy 1, policy_version 1497619 (0.0010) [2023-12-27 16:01:59,776][20347] Updated weights for policy 0, policy_version 1499129 (0.0007) [2023-12-27 16:01:59,816][20381] Updated weights for policy 1, policy_version 1497629 (0.0010) [2023-12-27 16:01:59,835][20347] Updated weights for policy 0, policy_version 1499139 (0.0006) [2023-12-27 16:01:59,892][20347] Updated weights for policy 0, policy_version 1499149 (0.0008) [2023-12-27 16:01:59,952][20347] Updated weights for policy 0, policy_version 1499159 (0.0008) [2023-12-27 16:02:00,510][20381] Updated weights for policy 1, policy_version 1497639 (0.0010) [2023-12-27 16:02:00,558][20381] Updated weights for policy 1, policy_version 1497649 (0.0011) [2023-12-27 16:02:00,607][20381] Updated weights for policy 1, policy_version 1497659 (0.0010) [2023-12-27 16:02:00,669][20347] Updated weights for policy 0, policy_version 1499169 (0.0007) [2023-12-27 16:02:00,724][20347] Updated weights for policy 0, policy_version 1499179 (0.0007) [2023-12-27 16:02:00,777][20347] Updated weights for policy 0, policy_version 1499189 (0.0007) [2023-12-27 16:02:01,310][20381] Updated weights for policy 1, policy_version 1497669 (0.0009) [2023-12-27 16:02:01,375][20381] Updated weights for policy 1, policy_version 1497679 (0.0011) [2023-12-27 16:02:01,434][20381] Updated weights for policy 1, policy_version 1497689 (0.0007) [2023-12-27 16:02:01,552][20347] Updated weights for policy 0, policy_version 1499199 (0.0008) [2023-12-27 16:02:01,610][20347] Updated weights for policy 0, policy_version 1499209 (0.0007) [2023-12-27 16:02:01,674][20347] Updated weights for policy 0, policy_version 1499219 (0.0009) [2023-12-27 16:02:02,087][20381] Updated weights for policy 1, policy_version 1497699 (0.0006) [2023-12-27 16:02:02,148][20381] Updated weights for policy 1, policy_version 1497709 (0.0005) [2023-12-27 16:02:02,204][20381] Updated weights for policy 1, policy_version 1497719 (0.0010) [2023-12-27 16:02:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 767328256. Throughput: 0: 10224.3, 1: 10176.9. Samples: 767338660. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:02:02,363][19170] Avg episode reward: [(0, '970.830'), (1, '980.690')] [2023-12-27 16:02:02,368][20347] Updated weights for policy 0, policy_version 1499229 (0.0010) [2023-12-27 16:02:02,417][20347] Updated weights for policy 0, policy_version 1499239 (0.0008) [2023-12-27 16:02:02,472][20347] Updated weights for policy 0, policy_version 1499249 (0.0010) [2023-12-27 16:02:02,884][20381] Updated weights for policy 1, policy_version 1497729 (0.0008) [2023-12-27 16:02:02,934][20381] Updated weights for policy 1, policy_version 1497739 (0.0005) [2023-12-27 16:02:02,996][20381] Updated weights for policy 1, policy_version 1497749 (0.0005) [2023-12-27 16:02:03,046][20381] Updated weights for policy 1, policy_version 1497759 (0.0005) [2023-12-27 16:02:03,092][20347] Updated weights for policy 0, policy_version 1499259 (0.0009) [2023-12-27 16:02:03,133][20347] Updated weights for policy 0, policy_version 1499269 (0.0005) [2023-12-27 16:02:03,177][20347] Updated weights for policy 0, policy_version 1499279 (0.0005) [2023-12-27 16:02:03,666][20381] Updated weights for policy 1, policy_version 1497769 (0.0007) [2023-12-27 16:02:03,720][20381] Updated weights for policy 1, policy_version 1497779 (0.0007) [2023-12-27 16:02:03,751][20347] Updated weights for policy 0, policy_version 1499289 (0.0010) [2023-12-27 16:02:03,774][20381] Updated weights for policy 1, policy_version 1497789 (0.0008) [2023-12-27 16:02:03,804][20347] Updated weights for policy 0, policy_version 1499299 (0.0005) [2023-12-27 16:02:03,866][20347] Updated weights for policy 0, policy_version 1499309 (0.0007) [2023-12-27 16:02:03,872][20291] KL-divergence is very high: 112.6839 [2023-12-27 16:02:03,909][20291] KL-divergence is very high: 184.0055 [2023-12-27 16:02:03,913][20347] Updated weights for policy 0, policy_version 1499319 (0.0006) [2023-12-27 16:02:04,519][20381] Updated weights for policy 1, policy_version 1497799 (0.0007) [2023-12-27 16:02:04,529][20347] Updated weights for policy 0, policy_version 1499329 (0.0005) [2023-12-27 16:02:04,579][20381] Updated weights for policy 1, policy_version 1497809 (0.0007) [2023-12-27 16:02:04,580][20347] Updated weights for policy 0, policy_version 1499339 (0.0005) [2023-12-27 16:02:04,639][20347] Updated weights for policy 0, policy_version 1499349 (0.0005) [2023-12-27 16:02:04,641][20381] Updated weights for policy 1, policy_version 1497819 (0.0008) [2023-12-27 16:02:05,244][20347] Updated weights for policy 0, policy_version 1499359 (0.0009) [2023-12-27 16:02:05,289][20347] Updated weights for policy 0, policy_version 1499369 (0.0010) [2023-12-27 16:02:05,343][20347] Updated weights for policy 0, policy_version 1499379 (0.0010) [2023-12-27 16:02:05,345][20381] Updated weights for policy 1, policy_version 1497829 (0.0006) [2023-12-27 16:02:05,400][20381] Updated weights for policy 1, policy_version 1497839 (0.0008) [2023-12-27 16:02:05,457][20381] Updated weights for policy 1, policy_version 1497849 (0.0007) [2023-12-27 16:02:06,026][20347] Updated weights for policy 0, policy_version 1499389 (0.0007) [2023-12-27 16:02:06,080][20347] Updated weights for policy 0, policy_version 1499399 (0.0010) [2023-12-27 16:02:06,136][20347] Updated weights for policy 0, policy_version 1499409 (0.0010) [2023-12-27 16:02:06,187][20381] Updated weights for policy 1, policy_version 1497859 (0.0007) [2023-12-27 16:02:06,247][20381] Updated weights for policy 1, policy_version 1497869 (0.0008) [2023-12-27 16:02:06,306][20381] Updated weights for policy 1, policy_version 1497879 (0.0007) [2023-12-27 16:02:06,804][20347] Updated weights for policy 0, policy_version 1499419 (0.0010) [2023-12-27 16:02:06,862][20347] Updated weights for policy 0, policy_version 1499429 (0.0010) [2023-12-27 16:02:06,918][20347] Updated weights for policy 0, policy_version 1499439 (0.0010) [2023-12-27 16:02:06,981][20381] Updated weights for policy 1, policy_version 1497889 (0.0008) [2023-12-27 16:02:07,030][20381] Updated weights for policy 1, policy_version 1497899 (0.0005) [2023-12-27 16:02:07,074][20381] Updated weights for policy 1, policy_version 1497909 (0.0005) [2023-12-27 16:02:07,122][20381] Updated weights for policy 1, policy_version 1497919 (0.0007) [2023-12-27 16:02:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 767434752. Throughput: 0: 10301.7, 1: 10197.8. Samples: 767402768. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-12-27 16:02:07,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 16:02:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001499448_383934464.pth... [2023-12-27 16:02:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001497920_383500288.pth... [2023-12-27 16:02:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001498264_383631360.pth [2023-12-27 16:02:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001496736_383197184.pth [2023-12-27 16:02:07,565][20347] Updated weights for policy 0, policy_version 1499449 (0.0010) [2023-12-27 16:02:07,620][20347] Updated weights for policy 0, policy_version 1499459 (0.0010) [2023-12-27 16:02:07,674][20347] Updated weights for policy 0, policy_version 1499469 (0.0010) [2023-12-27 16:02:07,735][20347] Updated weights for policy 0, policy_version 1499479 (0.0010) [2023-12-27 16:02:07,810][20381] Updated weights for policy 1, policy_version 1497929 (0.0007) [2023-12-27 16:02:07,866][20381] Updated weights for policy 1, policy_version 1497939 (0.0009) [2023-12-27 16:02:07,909][20381] Updated weights for policy 1, policy_version 1497949 (0.0007) [2023-12-27 16:02:08,331][20347] Updated weights for policy 0, policy_version 1499489 (0.0007) [2023-12-27 16:02:08,399][20347] Updated weights for policy 0, policy_version 1499499 (0.0010) [2023-12-27 16:02:08,459][20347] Updated weights for policy 0, policy_version 1499509 (0.0010) [2023-12-27 16:02:08,565][20381] Updated weights for policy 1, policy_version 1497959 (0.0009) [2023-12-27 16:02:08,622][20381] Updated weights for policy 1, policy_version 1497969 (0.0010) [2023-12-27 16:02:08,676][20381] Updated weights for policy 1, policy_version 1497979 (0.0007) [2023-12-27 16:02:09,115][20347] Updated weights for policy 0, policy_version 1499519 (0.0010) [2023-12-27 16:02:09,168][20347] Updated weights for policy 0, policy_version 1499529 (0.0007) [2023-12-27 16:02:09,233][20347] Updated weights for policy 0, policy_version 1499539 (0.0007) [2023-12-27 16:02:09,294][20381] Updated weights for policy 1, policy_version 1497989 (0.0006) [2023-12-27 16:02:09,369][20381] Updated weights for policy 1, policy_version 1497999 (0.0010) [2023-12-27 16:02:09,437][20381] Updated weights for policy 1, policy_version 1498009 (0.0008) [2023-12-27 16:02:09,944][20347] Updated weights for policy 0, policy_version 1499549 (0.0007) [2023-12-27 16:02:09,994][20347] Updated weights for policy 0, policy_version 1499559 (0.0008) [2023-12-27 16:02:10,048][20347] Updated weights for policy 0, policy_version 1499569 (0.0007) [2023-12-27 16:02:10,166][20381] Updated weights for policy 1, policy_version 1498019 (0.0008) [2023-12-27 16:02:10,217][20381] Updated weights for policy 1, policy_version 1498029 (0.0007) [2023-12-27 16:02:10,265][20381] Updated weights for policy 1, policy_version 1498039 (0.0008) [2023-12-27 16:02:10,728][20347] Updated weights for policy 0, policy_version 1499579 (0.0007) [2023-12-27 16:02:10,786][20347] Updated weights for policy 0, policy_version 1499589 (0.0007) [2023-12-27 16:02:10,851][20347] Updated weights for policy 0, policy_version 1499599 (0.0006) [2023-12-27 16:02:11,004][20381] Updated weights for policy 1, policy_version 1498049 (0.0008) [2023-12-27 16:02:11,061][20381] Updated weights for policy 1, policy_version 1498059 (0.0008) [2023-12-27 16:02:11,122][20381] Updated weights for policy 1, policy_version 1498069 (0.0008) [2023-12-27 16:02:11,187][20381] Updated weights for policy 1, policy_version 1498079 (0.0008) [2023-12-27 16:02:11,561][20347] Updated weights for policy 0, policy_version 1499609 (0.0005) [2023-12-27 16:02:11,619][20347] Updated weights for policy 0, policy_version 1499619 (0.0008) [2023-12-27 16:02:11,677][20347] Updated weights for policy 0, policy_version 1499629 (0.0009) [2023-12-27 16:02:11,731][20347] Updated weights for policy 0, policy_version 1499639 (0.0009) [2023-12-27 16:02:11,943][20381] Updated weights for policy 1, policy_version 1498089 (0.0006) [2023-12-27 16:02:11,994][20381] Updated weights for policy 1, policy_version 1498099 (0.0007) [2023-12-27 16:02:12,055][20381] Updated weights for policy 1, policy_version 1498109 (0.0009) [2023-12-27 16:02:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 767533056. Throughput: 0: 10362.7, 1: 10206.6. Samples: 767525912. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:12,363][19170] Avg episode reward: [(0, '970.670'), (1, '990.270')] [2023-12-27 16:02:12,410][20347] Updated weights for policy 0, policy_version 1499649 (0.0006) [2023-12-27 16:02:12,475][20347] Updated weights for policy 0, policy_version 1499659 (0.0005) [2023-12-27 16:02:12,536][20347] Updated weights for policy 0, policy_version 1499669 (0.0008) [2023-12-27 16:02:12,743][20381] Updated weights for policy 1, policy_version 1498119 (0.0007) [2023-12-27 16:02:12,800][20381] Updated weights for policy 1, policy_version 1498129 (0.0008) [2023-12-27 16:02:12,858][20381] Updated weights for policy 1, policy_version 1498139 (0.0005) [2023-12-27 16:02:13,124][20347] Updated weights for policy 0, policy_version 1499679 (0.0009) [2023-12-27 16:02:13,174][20347] Updated weights for policy 0, policy_version 1499689 (0.0008) [2023-12-27 16:02:13,222][20347] Updated weights for policy 0, policy_version 1499699 (0.0007) [2023-12-27 16:02:13,461][20381] Updated weights for policy 1, policy_version 1498149 (0.0007) [2023-12-27 16:02:13,510][20381] Updated weights for policy 1, policy_version 1498159 (0.0006) [2023-12-27 16:02:13,559][20381] Updated weights for policy 1, policy_version 1498169 (0.0010) [2023-12-27 16:02:13,887][20347] Updated weights for policy 0, policy_version 1499709 (0.0007) [2023-12-27 16:02:13,933][20347] Updated weights for policy 0, policy_version 1499719 (0.0006) [2023-12-27 16:02:13,976][20347] Updated weights for policy 0, policy_version 1499729 (0.0005) [2023-12-27 16:02:14,258][20381] Updated weights for policy 1, policy_version 1498179 (0.0008) [2023-12-27 16:02:14,310][20381] Updated weights for policy 1, policy_version 1498189 (0.0008) [2023-12-27 16:02:14,366][20381] Updated weights for policy 1, policy_version 1498199 (0.0008) [2023-12-27 16:02:14,638][20347] Updated weights for policy 0, policy_version 1499739 (0.0006) [2023-12-27 16:02:14,692][20347] Updated weights for policy 0, policy_version 1499749 (0.0007) [2023-12-27 16:02:14,743][20347] Updated weights for policy 0, policy_version 1499759 (0.0008) [2023-12-27 16:02:15,029][20381] Updated weights for policy 1, policy_version 1498209 (0.0008) [2023-12-27 16:02:15,070][20292] KL-divergence is very high: 114.6952 [2023-12-27 16:02:15,086][20381] Updated weights for policy 1, policy_version 1498219 (0.0008) [2023-12-27 16:02:15,145][20381] Updated weights for policy 1, policy_version 1498229 (0.0011) [2023-12-27 16:02:15,204][20381] Updated weights for policy 1, policy_version 1498239 (0.0010) [2023-12-27 16:02:15,462][20347] Updated weights for policy 0, policy_version 1499769 (0.0011) [2023-12-27 16:02:15,522][20347] Updated weights for policy 0, policy_version 1499779 (0.0010) [2023-12-27 16:02:15,570][20347] Updated weights for policy 0, policy_version 1499789 (0.0010) [2023-12-27 16:02:15,618][20347] Updated weights for policy 0, policy_version 1499799 (0.0010) [2023-12-27 16:02:15,883][20381] Updated weights for policy 1, policy_version 1498249 (0.0008) [2023-12-27 16:02:15,934][20381] Updated weights for policy 1, policy_version 1498259 (0.0007) [2023-12-27 16:02:15,997][20381] Updated weights for policy 1, policy_version 1498269 (0.0008) [2023-12-27 16:02:16,301][20347] Updated weights for policy 0, policy_version 1499809 (0.0010) [2023-12-27 16:02:16,365][20347] Updated weights for policy 0, policy_version 1499819 (0.0010) [2023-12-27 16:02:16,436][20347] Updated weights for policy 0, policy_version 1499829 (0.0007) [2023-12-27 16:02:16,588][20381] Updated weights for policy 1, policy_version 1498279 (0.0007) [2023-12-27 16:02:16,639][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-27 16:02:17,023][20347] Updated weights for policy 0, policy_version 1499839 (0.0007) [2023-12-27 16:02:17,074][20347] Updated weights for policy 0, policy_version 1499849 (0.0007) [2023-12-27 16:02:17,136][20347] Updated weights for policy 0, policy_version 1499859 (0.0008) [2023-12-27 16:02:17,314][20381] Updated weights for policy 1, policy_version 1498289 (0.0009) [2023-12-27 16:02:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 767639552. Throughput: 0: 10347.6, 1: 10228.6. Samples: 767650840. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:17,363][19170] Avg episode reward: [(0, '970.680'), (1, '970.780')] [2023-12-27 16:02:17,379][20381] Updated weights for policy 1, policy_version 1498299 (0.0008) [2023-12-27 16:02:17,437][20381] Updated weights for policy 1, policy_version 1498309 (0.0007) [2023-12-27 16:02:17,496][20381] Updated weights for policy 1, policy_version 1498319 (0.0008) [2023-12-27 16:02:17,771][20347] Updated weights for policy 0, policy_version 1499869 (0.0007) [2023-12-27 16:02:17,824][20347] Updated weights for policy 0, policy_version 1499879 (0.0007) [2023-12-27 16:02:17,873][20347] Updated weights for policy 0, policy_version 1499889 (0.0008) [2023-12-27 16:02:18,113][20381] Updated weights for policy 1, policy_version 1498329 (0.0006) [2023-12-27 16:02:18,167][20381] Updated weights for policy 1, policy_version 1498339 (0.0006) [2023-12-27 16:02:18,221][20381] Updated weights for policy 1, policy_version 1498349 (0.0007) [2023-12-27 16:02:18,637][20347] Updated weights for policy 0, policy_version 1499899 (0.0008) [2023-12-27 16:02:18,694][20347] Updated weights for policy 0, policy_version 1499909 (0.0007) [2023-12-27 16:02:18,748][20347] Updated weights for policy 0, policy_version 1499919 (0.0006) [2023-12-27 16:02:18,837][20381] Updated weights for policy 1, policy_version 1498359 (0.0007) [2023-12-27 16:02:18,897][20381] Updated weights for policy 1, policy_version 1498369 (0.0009) [2023-12-27 16:02:18,953][20381] Updated weights for policy 1, policy_version 1498379 (0.0009) [2023-12-27 16:02:19,449][20347] Updated weights for policy 0, policy_version 1499929 (0.0008) [2023-12-27 16:02:19,519][20347] Updated weights for policy 0, policy_version 1499939 (0.0008) [2023-12-27 16:02:19,575][20347] Updated weights for policy 0, policy_version 1499949 (0.0008) [2023-12-27 16:02:19,632][20347] Updated weights for policy 0, policy_version 1499959 (0.0007) [2023-12-27 16:02:19,729][20381] Updated weights for policy 1, policy_version 1498389 (0.0008) [2023-12-27 16:02:19,792][20381] Updated weights for policy 1, policy_version 1498399 (0.0008) [2023-12-27 16:02:19,861][20381] Updated weights for policy 1, policy_version 1498409 (0.0008) [2023-12-27 16:02:20,374][20347] Updated weights for policy 0, policy_version 1499969 (0.0008) [2023-12-27 16:02:20,436][20347] Updated weights for policy 0, policy_version 1499979 (0.0007) [2023-12-27 16:02:20,494][20347] Updated weights for policy 0, policy_version 1499989 (0.0008) [2023-12-27 16:02:20,632][20381] Updated weights for policy 1, policy_version 1498419 (0.0008) [2023-12-27 16:02:20,686][20381] Updated weights for policy 1, policy_version 1498429 (0.0007) [2023-12-27 16:02:20,743][20381] Updated weights for policy 1, policy_version 1498439 (0.0007) [2023-12-27 16:02:21,225][20347] Updated weights for policy 0, policy_version 1499999 (0.0008) [2023-12-27 16:02:21,294][20347] Updated weights for policy 0, policy_version 1500009 (0.0008) [2023-12-27 16:02:21,356][20347] Updated weights for policy 0, policy_version 1500019 (0.0008) [2023-12-27 16:02:21,464][20381] Updated weights for policy 1, policy_version 1498449 (0.0008) [2023-12-27 16:02:21,523][20381] Updated weights for policy 1, policy_version 1498459 (0.0008) [2023-12-27 16:02:21,587][20381] Updated weights for policy 1, policy_version 1498469 (0.0009) [2023-12-27 16:02:21,652][20381] Updated weights for policy 1, policy_version 1498479 (0.0008) [2023-12-27 16:02:22,089][20347] Updated weights for policy 0, policy_version 1500029 (0.0008) [2023-12-27 16:02:22,148][20347] Updated weights for policy 0, policy_version 1500039 (0.0008) [2023-12-27 16:02:22,204][20347] Updated weights for policy 0, policy_version 1500049 (0.0008) [2023-12-27 16:02:22,347][20381] Updated weights for policy 1, policy_version 1498489 (0.0008) [2023-12-27 16:02:22,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 767737856. Throughput: 0: 10250.0, 1: 10211.9. Samples: 767710784. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:22,363][19170] Avg episode reward: [(0, '980.460'), (1, '980.510')] [2023-12-27 16:02:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001500056_384090112.pth... [2023-12-27 16:02:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001498872_383787008.pth [2023-12-27 16:02:22,413][20381] Updated weights for policy 1, policy_version 1498499 (0.0008) [2023-12-27 16:02:22,476][20381] Updated weights for policy 1, policy_version 1498509 (0.0007) [2023-12-27 16:02:22,494][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001498512_383655936.pth... [2023-12-27 16:02:22,499][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001497312_383344640.pth [2023-12-27 16:02:22,925][20347] Updated weights for policy 0, policy_version 1500059 (0.0007) [2023-12-27 16:02:22,985][20347] Updated weights for policy 0, policy_version 1500069 (0.0008) [2023-12-27 16:02:23,040][20347] Updated weights for policy 0, policy_version 1500079 (0.0008) [2023-12-27 16:02:23,179][20381] Updated weights for policy 1, policy_version 1498519 (0.0008) [2023-12-27 16:02:23,233][20381] Updated weights for policy 1, policy_version 1498529 (0.0008) [2023-12-27 16:02:23,279][20381] Updated weights for policy 1, policy_version 1498539 (0.0007) [2023-12-27 16:02:23,698][20347] Updated weights for policy 0, policy_version 1500089 (0.0008) [2023-12-27 16:02:23,750][20347] Updated weights for policy 0, policy_version 1500099 (0.0008) [2023-12-27 16:02:23,803][20347] Updated weights for policy 0, policy_version 1500109 (0.0007) [2023-12-27 16:02:23,859][20347] Updated weights for policy 0, policy_version 1500119 (0.0008) [2023-12-27 16:02:23,976][20381] Updated weights for policy 1, policy_version 1498549 (0.0007) [2023-12-27 16:02:24,031][20381] Updated weights for policy 1, policy_version 1498559 (0.0009) [2023-12-27 16:02:24,084][20381] Updated weights for policy 1, policy_version 1498569 (0.0007) [2023-12-27 16:02:24,513][20347] Updated weights for policy 0, policy_version 1500129 (0.0007) [2023-12-27 16:02:24,579][20347] Updated weights for policy 0, policy_version 1500139 (0.0008) [2023-12-27 16:02:24,631][20347] Updated weights for policy 0, policy_version 1500149 (0.0008) [2023-12-27 16:02:24,794][20381] Updated weights for policy 1, policy_version 1498579 (0.0007) [2023-12-27 16:02:24,851][20381] Updated weights for policy 1, policy_version 1498589 (0.0008) [2023-12-27 16:02:24,903][20381] Updated weights for policy 1, policy_version 1498599 (0.0006) [2023-12-27 16:02:25,337][20347] Updated weights for policy 0, policy_version 1500159 (0.0007) [2023-12-27 16:02:25,393][20347] Updated weights for policy 0, policy_version 1500169 (0.0007) [2023-12-27 16:02:25,452][20347] Updated weights for policy 0, policy_version 1500179 (0.0008) [2023-12-27 16:02:25,531][20381] Updated weights for policy 1, policy_version 1498609 (0.0006) [2023-12-27 16:02:25,586][20381] Updated weights for policy 1, policy_version 1498619 (0.0009) [2023-12-27 16:02:25,631][20381] Updated weights for policy 1, policy_version 1498629 (0.0005) [2023-12-27 16:02:25,675][20381] Updated weights for policy 1, policy_version 1498639 (0.0005) [2023-12-27 16:02:26,158][20347] Updated weights for policy 0, policy_version 1500189 (0.0009) [2023-12-27 16:02:26,220][20347] Updated weights for policy 0, policy_version 1500199 (0.0011) [2023-12-27 16:02:26,282][20347] Updated weights for policy 0, policy_version 1500209 (0.0010) [2023-12-27 16:02:26,313][20381] Updated weights for policy 1, policy_version 1498649 (0.0005) [2023-12-27 16:02:26,379][20381] Updated weights for policy 1, policy_version 1498659 (0.0006) [2023-12-27 16:02:26,439][20381] Updated weights for policy 1, policy_version 1498669 (0.0005) [2023-12-27 16:02:26,892][20347] Updated weights for policy 0, policy_version 1500219 (0.0010) [2023-12-27 16:02:26,943][20347] Updated weights for policy 0, policy_version 1500229 (0.0007) [2023-12-27 16:02:26,993][20347] Updated weights for policy 0, policy_version 1500239 (0.0007) [2023-12-27 16:02:27,050][20381] Updated weights for policy 1, policy_version 1498679 (0.0006) [2023-12-27 16:02:27,101][20381] Updated weights for policy 1, policy_version 1498689 (0.0005) [2023-12-27 16:02:27,146][20381] Updated weights for policy 1, policy_version 1498699 (0.0005) [2023-12-27 16:02:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 767844352. Throughput: 0: 10158.9, 1: 10211.0. Samples: 767829824. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 16:02:27,600][20347] Updated weights for policy 0, policy_version 1500249 (0.0006) [2023-12-27 16:02:27,654][20347] Updated weights for policy 0, policy_version 1500259 (0.0005) [2023-12-27 16:02:27,697][20347] Updated weights for policy 0, policy_version 1500269 (0.0005) [2023-12-27 16:02:27,746][20347] Updated weights for policy 0, policy_version 1500279 (0.0007) [2023-12-27 16:02:27,855][20381] Updated weights for policy 1, policy_version 1498709 (0.0007) [2023-12-27 16:02:27,913][20381] Updated weights for policy 1, policy_version 1498719 (0.0008) [2023-12-27 16:02:27,961][20381] Updated weights for policy 1, policy_version 1498729 (0.0008) [2023-12-27 16:02:28,390][20347] Updated weights for policy 0, policy_version 1500289 (0.0008) [2023-12-27 16:02:28,449][20347] Updated weights for policy 0, policy_version 1500299 (0.0008) [2023-12-27 16:02:28,500][20347] Updated weights for policy 0, policy_version 1500309 (0.0007) [2023-12-27 16:02:28,626][20381] Updated weights for policy 1, policy_version 1498739 (0.0008) [2023-12-27 16:02:28,675][20381] Updated weights for policy 1, policy_version 1498749 (0.0008) [2023-12-27 16:02:28,724][20381] Updated weights for policy 1, policy_version 1498759 (0.0008) [2023-12-27 16:02:29,201][20347] Updated weights for policy 0, policy_version 1500319 (0.0006) [2023-12-27 16:02:29,263][20347] Updated weights for policy 0, policy_version 1500329 (0.0007) [2023-12-27 16:02:29,319][20347] Updated weights for policy 0, policy_version 1500339 (0.0008) [2023-12-27 16:02:29,431][20381] Updated weights for policy 1, policy_version 1498769 (0.0007) [2023-12-27 16:02:29,491][20381] Updated weights for policy 1, policy_version 1498779 (0.0008) [2023-12-27 16:02:29,543][20381] Updated weights for policy 1, policy_version 1498789 (0.0008) [2023-12-27 16:02:29,596][20381] Updated weights for policy 1, policy_version 1498799 (0.0007) [2023-12-27 16:02:30,018][20347] Updated weights for policy 0, policy_version 1500349 (0.0008) [2023-12-27 16:02:30,075][20347] Updated weights for policy 0, policy_version 1500359 (0.0008) [2023-12-27 16:02:30,134][20347] Updated weights for policy 0, policy_version 1500369 (0.0008) [2023-12-27 16:02:30,307][20381] Updated weights for policy 1, policy_version 1498809 (0.0008) [2023-12-27 16:02:30,363][20381] Updated weights for policy 1, policy_version 1498819 (0.0007) [2023-12-27 16:02:30,416][20381] Updated weights for policy 1, policy_version 1498829 (0.0007) [2023-12-27 16:02:30,741][20347] Updated weights for policy 0, policy_version 1500379 (0.0007) [2023-12-27 16:02:30,800][20347] Updated weights for policy 0, policy_version 1500389 (0.0005) [2023-12-27 16:02:30,857][20347] Updated weights for policy 0, policy_version 1500399 (0.0007) [2023-12-27 16:02:31,008][20381] Updated weights for policy 1, policy_version 1498839 (0.0007) [2023-12-27 16:02:31,070][20381] Updated weights for policy 1, policy_version 1498849 (0.0009) [2023-12-27 16:02:31,123][20381] Updated weights for policy 1, policy_version 1498859 (0.0008) [2023-12-27 16:02:31,545][20347] Updated weights for policy 0, policy_version 1500409 (0.0006) [2023-12-27 16:02:31,590][20347] Updated weights for policy 0, policy_version 1500419 (0.0010) [2023-12-27 16:02:31,655][20347] Updated weights for policy 0, policy_version 1500429 (0.0010) [2023-12-27 16:02:31,710][20347] Updated weights for policy 0, policy_version 1500439 (0.0008) [2023-12-27 16:02:31,836][20381] Updated weights for policy 1, policy_version 1498869 (0.0007) [2023-12-27 16:02:31,893][20381] Updated weights for policy 1, policy_version 1498879 (0.0008) [2023-12-27 16:02:31,949][20381] Updated weights for policy 1, policy_version 1498889 (0.0008) [2023-12-27 16:02:32,356][20347] Updated weights for policy 0, policy_version 1500449 (0.0008) [2023-12-27 16:02:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 767942656. Throughput: 0: 10221.0, 1: 10214.8. Samples: 767955132. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:32,363][19170] Avg episode reward: [(0, '990.260'), (1, '980.520')] [2023-12-27 16:02:32,412][20347] Updated weights for policy 0, policy_version 1500459 (0.0006) [2023-12-27 16:02:32,469][20347] Updated weights for policy 0, policy_version 1500469 (0.0007) [2023-12-27 16:02:32,580][20381] Updated weights for policy 1, policy_version 1498899 (0.0007) [2023-12-27 16:02:32,645][20381] Updated weights for policy 1, policy_version 1498909 (0.0007) [2023-12-27 16:02:32,706][20381] Updated weights for policy 1, policy_version 1498919 (0.0006) [2023-12-27 16:02:33,073][20347] Updated weights for policy 0, policy_version 1500479 (0.0006) [2023-12-27 16:02:33,131][20347] Updated weights for policy 0, policy_version 1500489 (0.0006) [2023-12-27 16:02:33,183][20347] Updated weights for policy 0, policy_version 1500499 (0.0005) [2023-12-27 16:02:33,250][20381] Updated weights for policy 1, policy_version 1498929 (0.0006) [2023-12-27 16:02:33,310][20381] Updated weights for policy 1, policy_version 1498939 (0.0006) [2023-12-27 16:02:33,362][20381] Updated weights for policy 1, policy_version 1498949 (0.0005) [2023-12-27 16:02:33,410][20381] Updated weights for policy 1, policy_version 1498959 (0.0008) [2023-12-27 16:02:33,728][20347] Updated weights for policy 0, policy_version 1500509 (0.0005) [2023-12-27 16:02:33,778][20347] Updated weights for policy 0, policy_version 1500519 (0.0005) [2023-12-27 16:02:33,834][20347] Updated weights for policy 0, policy_version 1500529 (0.0007) [2023-12-27 16:02:33,968][20381] Updated weights for policy 1, policy_version 1498969 (0.0010) [2023-12-27 16:02:34,022][20381] Updated weights for policy 1, policy_version 1498979 (0.0010) [2023-12-27 16:02:34,070][20381] Updated weights for policy 1, policy_version 1498989 (0.0010) [2023-12-27 16:02:34,510][20347] Updated weights for policy 0, policy_version 1500539 (0.0009) [2023-12-27 16:02:34,574][20347] Updated weights for policy 0, policy_version 1500549 (0.0006) [2023-12-27 16:02:34,637][20347] Updated weights for policy 0, policy_version 1500559 (0.0005) [2023-12-27 16:02:34,803][20381] Updated weights for policy 1, policy_version 1498999 (0.0009) [2023-12-27 16:02:34,862][20381] Updated weights for policy 1, policy_version 1499009 (0.0008) [2023-12-27 16:02:34,915][20381] Updated weights for policy 1, policy_version 1499019 (0.0006) [2023-12-27 16:02:35,187][20347] Updated weights for policy 0, policy_version 1500569 (0.0006) [2023-12-27 16:02:35,246][20347] Updated weights for policy 0, policy_version 1500579 (0.0010) [2023-12-27 16:02:35,304][20347] Updated weights for policy 0, policy_version 1500589 (0.0010) [2023-12-27 16:02:35,351][20347] Updated weights for policy 0, policy_version 1500599 (0.0008) [2023-12-27 16:02:35,515][20381] Updated weights for policy 1, policy_version 1499029 (0.0008) [2023-12-27 16:02:35,573][20381] Updated weights for policy 1, policy_version 1499039 (0.0010) [2023-12-27 16:02:35,627][20381] Updated weights for policy 1, policy_version 1499049 (0.0010) [2023-12-27 16:02:36,045][20347] Updated weights for policy 0, policy_version 1500609 (0.0010) [2023-12-27 16:02:36,106][20347] Updated weights for policy 0, policy_version 1500619 (0.0010) [2023-12-27 16:02:36,173][20347] Updated weights for policy 0, policy_version 1500629 (0.0010) [2023-12-27 16:02:36,300][20381] Updated weights for policy 1, policy_version 1499059 (0.0010) [2023-12-27 16:02:36,349][20381] Updated weights for policy 1, policy_version 1499069 (0.0011) [2023-12-27 16:02:36,399][20381] Updated weights for policy 1, policy_version 1499079 (0.0010) [2023-12-27 16:02:36,807][20347] Updated weights for policy 0, policy_version 1500639 (0.0007) [2023-12-27 16:02:36,864][20347] Updated weights for policy 0, policy_version 1500649 (0.0005) [2023-12-27 16:02:36,919][20347] Updated weights for policy 0, policy_version 1500659 (0.0005) [2023-12-27 16:02:37,120][20381] Updated weights for policy 1, policy_version 1499089 (0.0010) [2023-12-27 16:02:37,177][20381] Updated weights for policy 1, policy_version 1499099 (0.0007) [2023-12-27 16:02:37,239][20381] Updated weights for policy 1, policy_version 1499109 (0.0008) [2023-12-27 16:02:37,297][20381] Updated weights for policy 1, policy_version 1499119 (0.0007) [2023-12-27 16:02:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 768057344. Throughput: 0: 10263.5, 1: 10285.4. Samples: 768022484. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:37,363][19170] Avg episode reward: [(0, '990.260'), (1, '961.090')] [2023-12-27 16:02:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001499120_383811584.pth... [2023-12-27 16:02:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001500664_384245760.pth... [2023-12-27 16:02:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001499448_383934464.pth [2023-12-27 16:02:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001497920_383500288.pth [2023-12-27 16:02:37,508][20347] Updated weights for policy 0, policy_version 1500669 (0.0008) [2023-12-27 16:02:37,547][20291] KL-divergence is very high: 329.7641 [2023-12-27 16:02:37,557][20347] Updated weights for policy 0, policy_version 1500679 (0.0010) [2023-12-27 16:02:37,590][20291] KL-divergence is very high: 328.5577 [2023-12-27 16:02:37,613][20347] Updated weights for policy 0, policy_version 1500689 (0.0010) [2023-12-27 16:02:37,632][20291] KL-divergence is very high: 200.2620 [2023-12-27 16:02:37,996][20381] Updated weights for policy 1, policy_version 1499129 (0.0006) [2023-12-27 16:02:38,045][20381] Updated weights for policy 1, policy_version 1499139 (0.0010) [2023-12-27 16:02:38,103][20381] Updated weights for policy 1, policy_version 1499149 (0.0010) [2023-12-27 16:02:38,243][20347] Updated weights for policy 0, policy_version 1500699 (0.0010) [2023-12-27 16:02:38,303][20347] Updated weights for policy 0, policy_version 1500709 (0.0010) [2023-12-27 16:02:38,363][20347] Updated weights for policy 0, policy_version 1500719 (0.0010) [2023-12-27 16:02:38,767][20381] Updated weights for policy 1, policy_version 1499159 (0.0010) [2023-12-27 16:02:38,821][20381] Updated weights for policy 1, policy_version 1499169 (0.0010) [2023-12-27 16:02:38,874][20381] Updated weights for policy 1, policy_version 1499179 (0.0007) [2023-12-27 16:02:39,039][20347] Updated weights for policy 0, policy_version 1500729 (0.0010) [2023-12-27 16:02:39,087][20347] Updated weights for policy 0, policy_version 1500739 (0.0010) [2023-12-27 16:02:39,139][20347] Updated weights for policy 0, policy_version 1500749 (0.0006) [2023-12-27 16:02:39,184][20347] Updated weights for policy 0, policy_version 1500759 (0.0010) [2023-12-27 16:02:39,600][20381] Updated weights for policy 1, policy_version 1499189 (0.0007) [2023-12-27 16:02:39,663][20381] Updated weights for policy 1, policy_version 1499199 (0.0009) [2023-12-27 16:02:39,722][20381] Updated weights for policy 1, policy_version 1499209 (0.0008) [2023-12-27 16:02:39,963][20347] Updated weights for policy 0, policy_version 1500769 (0.0008) [2023-12-27 16:02:40,022][20347] Updated weights for policy 0, policy_version 1500779 (0.0007) [2023-12-27 16:02:40,079][20347] Updated weights for policy 0, policy_version 1500789 (0.0008) [2023-12-27 16:02:40,443][20381] Updated weights for policy 1, policy_version 1499219 (0.0008) [2023-12-27 16:02:40,499][20381] Updated weights for policy 1, policy_version 1499229 (0.0006) [2023-12-27 16:02:40,568][20381] Updated weights for policy 1, policy_version 1499239 (0.0005) [2023-12-27 16:02:40,645][20347] Updated weights for policy 0, policy_version 1500799 (0.0006) [2023-12-27 16:02:40,691][20347] Updated weights for policy 0, policy_version 1500809 (0.0005) [2023-12-27 16:02:40,738][20347] Updated weights for policy 0, policy_version 1500819 (0.0006) [2023-12-27 16:02:41,258][20381] Updated weights for policy 1, policy_version 1499249 (0.0008) [2023-12-27 16:02:41,315][20381] Updated weights for policy 1, policy_version 1499259 (0.0008) [2023-12-27 16:02:41,382][20381] Updated weights for policy 1, policy_version 1499269 (0.0008) [2023-12-27 16:02:41,447][20381] Updated weights for policy 1, policy_version 1499279 (0.0007) [2023-12-27 16:02:41,472][20347] Updated weights for policy 0, policy_version 1500829 (0.0009) [2023-12-27 16:02:41,536][20347] Updated weights for policy 0, policy_version 1500839 (0.0009) [2023-12-27 16:02:41,593][20347] Updated weights for policy 0, policy_version 1500849 (0.0008) [2023-12-27 16:02:42,108][20381] Updated weights for policy 1, policy_version 1499289 (0.0006) [2023-12-27 16:02:42,170][20381] Updated weights for policy 1, policy_version 1499299 (0.0007) [2023-12-27 16:02:42,229][20381] Updated weights for policy 1, policy_version 1499309 (0.0006) [2023-12-27 16:02:42,289][20347] Updated weights for policy 0, policy_version 1500859 (0.0008) [2023-12-27 16:02:42,355][20347] Updated weights for policy 0, policy_version 1500869 (0.0009) [2023-12-27 16:02:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 768155648. Throughput: 0: 10352.4, 1: 10274.0. Samples: 768146992. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:42,363][19170] Avg episode reward: [(0, '990.260'), (1, '961.080')] [2023-12-27 16:02:42,418][20347] Updated weights for policy 0, policy_version 1500879 (0.0006) [2023-12-27 16:02:42,955][20381] Updated weights for policy 1, policy_version 1499319 (0.0006) [2023-12-27 16:02:43,006][20381] Updated weights for policy 1, policy_version 1499329 (0.0006) [2023-12-27 16:02:43,014][20347] Updated weights for policy 0, policy_version 1500889 (0.0005) [2023-12-27 16:02:43,053][20381] Updated weights for policy 1, policy_version 1499339 (0.0006) [2023-12-27 16:02:43,065][20347] Updated weights for policy 0, policy_version 1500899 (0.0006) [2023-12-27 16:02:43,115][20347] Updated weights for policy 0, policy_version 1500909 (0.0005) [2023-12-27 16:02:43,165][20347] Updated weights for policy 0, policy_version 1500919 (0.0007) [2023-12-27 16:02:43,690][20381] Updated weights for policy 1, policy_version 1499349 (0.0007) [2023-12-27 16:02:43,741][20381] Updated weights for policy 1, policy_version 1499359 (0.0007) [2023-12-27 16:02:43,789][20381] Updated weights for policy 1, policy_version 1499369 (0.0006) [2023-12-27 16:02:43,813][20347] Updated weights for policy 0, policy_version 1500929 (0.0010) [2023-12-27 16:02:43,866][20347] Updated weights for policy 0, policy_version 1500939 (0.0009) [2023-12-27 16:02:43,911][20347] Updated weights for policy 0, policy_version 1500949 (0.0007) [2023-12-27 16:02:44,379][20381] Updated weights for policy 1, policy_version 1499379 (0.0005) [2023-12-27 16:02:44,439][20381] Updated weights for policy 1, policy_version 1499389 (0.0005) [2023-12-27 16:02:44,494][20381] Updated weights for policy 1, policy_version 1499399 (0.0005) [2023-12-27 16:02:44,651][20347] Updated weights for policy 0, policy_version 1500959 (0.0007) [2023-12-27 16:02:44,701][20347] Updated weights for policy 0, policy_version 1500969 (0.0007) [2023-12-27 16:02:44,760][20347] Updated weights for policy 0, policy_version 1500979 (0.0007) [2023-12-27 16:02:45,171][20381] Updated weights for policy 1, policy_version 1499409 (0.0009) [2023-12-27 16:02:45,229][20381] Updated weights for policy 1, policy_version 1499419 (0.0008) [2023-12-27 16:02:45,288][20381] Updated weights for policy 1, policy_version 1499429 (0.0007) [2023-12-27 16:02:45,339][20381] Updated weights for policy 1, policy_version 1499439 (0.0005) [2023-12-27 16:02:45,482][20347] Updated weights for policy 0, policy_version 1500989 (0.0006) [2023-12-27 16:02:45,538][20347] Updated weights for policy 0, policy_version 1500999 (0.0005) [2023-12-27 16:02:45,591][20347] Updated weights for policy 0, policy_version 1501009 (0.0005) [2023-12-27 16:02:45,926][20381] Updated weights for policy 1, policy_version 1499449 (0.0009) [2023-12-27 16:02:45,982][20381] Updated weights for policy 1, policy_version 1499459 (0.0005) [2023-12-27 16:02:46,036][20381] Updated weights for policy 1, policy_version 1499469 (0.0005) [2023-12-27 16:02:46,237][20347] Updated weights for policy 0, policy_version 1501019 (0.0006) [2023-12-27 16:02:46,286][20347] Updated weights for policy 0, policy_version 1501029 (0.0008) [2023-12-27 16:02:46,339][20347] Updated weights for policy 0, policy_version 1501039 (0.0007) [2023-12-27 16:02:46,662][20381] Updated weights for policy 1, policy_version 1499479 (0.0007) [2023-12-27 16:02:46,721][20381] Updated weights for policy 1, policy_version 1499489 (0.0007) [2023-12-27 16:02:46,773][20381] Updated weights for policy 1, policy_version 1499499 (0.0008) [2023-12-27 16:02:46,978][20347] Updated weights for policy 0, policy_version 1501049 (0.0007) [2023-12-27 16:02:47,031][20347] Updated weights for policy 0, policy_version 1501059 (0.0008) [2023-12-27 16:02:47,090][20347] Updated weights for policy 0, policy_version 1501069 (0.0005) [2023-12-27 16:02:47,151][20347] Updated weights for policy 0, policy_version 1501079 (0.0005) [2023-12-27 16:02:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 768262144. Throughput: 0: 10451.6, 1: 10298.9. Samples: 768272436. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:47,364][19170] Avg episode reward: [(0, '970.870'), (1, '951.400')] [2023-12-27 16:02:47,405][20381] Updated weights for policy 1, policy_version 1499509 (0.0006) [2023-12-27 16:02:47,457][20381] Updated weights for policy 1, policy_version 1499519 (0.0005) [2023-12-27 16:02:47,519][20381] Updated weights for policy 1, policy_version 1499529 (0.0006) [2023-12-27 16:02:47,776][20347] Updated weights for policy 0, policy_version 1501089 (0.0006) [2023-12-27 16:02:47,834][20347] Updated weights for policy 0, policy_version 1501099 (0.0006) [2023-12-27 16:02:47,893][20347] Updated weights for policy 0, policy_version 1501109 (0.0006) [2023-12-27 16:02:48,134][20381] Updated weights for policy 1, policy_version 1499539 (0.0008) [2023-12-27 16:02:48,187][20381] Updated weights for policy 1, policy_version 1499549 (0.0009) [2023-12-27 16:02:48,238][20381] Updated weights for policy 1, policy_version 1499559 (0.0009) [2023-12-27 16:02:48,476][20347] Updated weights for policy 0, policy_version 1501119 (0.0009) [2023-12-27 16:02:48,534][20347] Updated weights for policy 0, policy_version 1501129 (0.0010) [2023-12-27 16:02:48,592][20347] Updated weights for policy 0, policy_version 1501139 (0.0010) [2023-12-27 16:02:48,930][20381] Updated weights for policy 1, policy_version 1499569 (0.0007) [2023-12-27 16:02:48,983][20381] Updated weights for policy 1, policy_version 1499579 (0.0008) [2023-12-27 16:02:49,032][20381] Updated weights for policy 1, policy_version 1499589 (0.0008) [2023-12-27 16:02:49,095][20381] Updated weights for policy 1, policy_version 1499599 (0.0008) [2023-12-27 16:02:49,198][20347] Updated weights for policy 0, policy_version 1501149 (0.0008) [2023-12-27 16:02:49,258][20347] Updated weights for policy 0, policy_version 1501159 (0.0008) [2023-12-27 16:02:49,317][20347] Updated weights for policy 0, policy_version 1501169 (0.0007) [2023-12-27 16:02:49,856][20381] Updated weights for policy 1, policy_version 1499609 (0.0008) [2023-12-27 16:02:49,915][20381] Updated weights for policy 1, policy_version 1499619 (0.0008) [2023-12-27 16:02:49,975][20347] Updated weights for policy 0, policy_version 1501179 (0.0008) [2023-12-27 16:02:49,982][20381] Updated weights for policy 1, policy_version 1499629 (0.0008) [2023-12-27 16:02:50,033][20347] Updated weights for policy 0, policy_version 1501189 (0.0008) [2023-12-27 16:02:50,079][20347] Updated weights for policy 0, policy_version 1501199 (0.0008) [2023-12-27 16:02:50,565][20381] Updated weights for policy 1, policy_version 1499639 (0.0008) [2023-12-27 16:02:50,624][20381] Updated weights for policy 1, policy_version 1499649 (0.0006) [2023-12-27 16:02:50,679][20381] Updated weights for policy 1, policy_version 1499659 (0.0007) [2023-12-27 16:02:50,844][20347] Updated weights for policy 0, policy_version 1501209 (0.0008) [2023-12-27 16:02:50,910][20347] Updated weights for policy 0, policy_version 1501219 (0.0008) [2023-12-27 16:02:50,963][20347] Updated weights for policy 0, policy_version 1501229 (0.0008) [2023-12-27 16:02:51,024][20347] Updated weights for policy 0, policy_version 1501239 (0.0008) [2023-12-27 16:02:51,409][20381] Updated weights for policy 1, policy_version 1499669 (0.0007) [2023-12-27 16:02:51,467][20381] Updated weights for policy 1, policy_version 1499679 (0.0007) [2023-12-27 16:02:51,531][20381] Updated weights for policy 1, policy_version 1499689 (0.0007) [2023-12-27 16:02:51,791][20347] Updated weights for policy 0, policy_version 1501249 (0.0007) [2023-12-27 16:02:51,848][20347] Updated weights for policy 0, policy_version 1501259 (0.0005) [2023-12-27 16:02:51,909][20347] Updated weights for policy 0, policy_version 1501269 (0.0006) [2023-12-27 16:02:52,251][20381] Updated weights for policy 1, policy_version 1499699 (0.0007) [2023-12-27 16:02:52,312][20381] Updated weights for policy 1, policy_version 1499709 (0.0008) [2023-12-27 16:02:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 768360448. Throughput: 0: 10424.4, 1: 10314.0. Samples: 768336000. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:52,363][19170] Avg episode reward: [(0, '961.140'), (1, '912.430')] [2023-12-27 16:02:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001501272_384401408.pth... [2023-12-27 16:02:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001500056_384090112.pth [2023-12-27 16:02:52,379][20381] Updated weights for policy 1, policy_version 1499719 (0.0008) [2023-12-27 16:02:52,422][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001499728_383967232.pth... [2023-12-27 16:02:52,425][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001498512_383655936.pth [2023-12-27 16:02:52,538][20347] Updated weights for policy 0, policy_version 1501279 (0.0008) [2023-12-27 16:02:52,601][20347] Updated weights for policy 0, policy_version 1501289 (0.0008) [2023-12-27 16:02:52,662][20347] Updated weights for policy 0, policy_version 1501299 (0.0007) [2023-12-27 16:02:53,085][20381] Updated weights for policy 1, policy_version 1499729 (0.0008) [2023-12-27 16:02:53,145][20381] Updated weights for policy 1, policy_version 1499739 (0.0008) [2023-12-27 16:02:53,197][20381] Updated weights for policy 1, policy_version 1499749 (0.0007) [2023-12-27 16:02:53,254][20381] Updated weights for policy 1, policy_version 1499759 (0.0008) [2023-12-27 16:02:53,302][20347] Updated weights for policy 0, policy_version 1501309 (0.0008) [2023-12-27 16:02:53,349][20347] Updated weights for policy 0, policy_version 1501319 (0.0007) [2023-12-27 16:02:53,398][20347] Updated weights for policy 0, policy_version 1501329 (0.0008) [2023-12-27 16:02:53,953][20381] Updated weights for policy 1, policy_version 1499769 (0.0006) [2023-12-27 16:02:54,005][20347] Updated weights for policy 0, policy_version 1501339 (0.0006) [2023-12-27 16:02:54,009][20381] Updated weights for policy 1, policy_version 1499779 (0.0005) [2023-12-27 16:02:54,057][20347] Updated weights for policy 0, policy_version 1501349 (0.0006) [2023-12-27 16:02:54,069][20381] Updated weights for policy 1, policy_version 1499789 (0.0006) [2023-12-27 16:02:54,113][20347] Updated weights for policy 0, policy_version 1501359 (0.0006) [2023-12-27 16:02:54,691][20381] Updated weights for policy 1, policy_version 1499799 (0.0006) [2023-12-27 16:02:54,715][20347] Updated weights for policy 0, policy_version 1501369 (0.0008) [2023-12-27 16:02:54,744][20381] Updated weights for policy 1, policy_version 1499809 (0.0005) [2023-12-27 16:02:54,772][20347] Updated weights for policy 0, policy_version 1501379 (0.0006) [2023-12-27 16:02:54,790][20381] Updated weights for policy 1, policy_version 1499819 (0.0005) [2023-12-27 16:02:54,832][20347] Updated weights for policy 0, policy_version 1501389 (0.0008) [2023-12-27 16:02:54,891][20347] Updated weights for policy 0, policy_version 1501399 (0.0008) [2023-12-27 16:02:55,471][20381] Updated weights for policy 1, policy_version 1499829 (0.0009) [2023-12-27 16:02:55,521][20381] Updated weights for policy 1, policy_version 1499839 (0.0010) [2023-12-27 16:02:55,573][20381] Updated weights for policy 1, policy_version 1499849 (0.0011) [2023-12-27 16:02:55,595][20347] Updated weights for policy 0, policy_version 1501409 (0.0007) [2023-12-27 16:02:55,642][20347] Updated weights for policy 0, policy_version 1501419 (0.0008) [2023-12-27 16:02:55,693][20347] Updated weights for policy 0, policy_version 1501429 (0.0007) [2023-12-27 16:02:56,265][20381] Updated weights for policy 1, policy_version 1499859 (0.0010) [2023-12-27 16:02:56,320][20381] Updated weights for policy 1, policy_version 1499869 (0.0008) [2023-12-27 16:02:56,369][20381] Updated weights for policy 1, policy_version 1499879 (0.0007) [2023-12-27 16:02:56,409][20347] Updated weights for policy 0, policy_version 1501439 (0.0006) [2023-12-27 16:02:56,463][20347] Updated weights for policy 0, policy_version 1501449 (0.0007) [2023-12-27 16:02:56,518][20347] Updated weights for policy 0, policy_version 1501459 (0.0008) [2023-12-27 16:02:57,080][20381] Updated weights for policy 1, policy_version 1499889 (0.0006) [2023-12-27 16:02:57,128][20381] Updated weights for policy 1, policy_version 1499899 (0.0007) [2023-12-27 16:02:57,162][20347] Updated weights for policy 0, policy_version 1501469 (0.0008) [2023-12-27 16:02:57,176][20381] Updated weights for policy 1, policy_version 1499909 (0.0005) [2023-12-27 16:02:57,208][20347] Updated weights for policy 0, policy_version 1501479 (0.0007) [2023-12-27 16:02:57,228][20381] Updated weights for policy 1, policy_version 1499919 (0.0006) [2023-12-27 16:02:57,264][20347] Updated weights for policy 0, policy_version 1501489 (0.0007) [2023-12-27 16:02:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 768475136. Throughput: 0: 10398.7, 1: 10320.8. Samples: 768458288. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:02:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '921.980')] [2023-12-27 16:02:57,874][20347] Updated weights for policy 0, policy_version 1501499 (0.0007) [2023-12-27 16:02:57,927][20347] Updated weights for policy 0, policy_version 1501509 (0.0010) [2023-12-27 16:02:57,927][20381] Updated weights for policy 1, policy_version 1499929 (0.0010) [2023-12-27 16:02:57,977][20347] Updated weights for policy 0, policy_version 1501519 (0.0010) [2023-12-27 16:02:57,981][20381] Updated weights for policy 1, policy_version 1499939 (0.0006) [2023-12-27 16:02:58,039][20381] Updated weights for policy 1, policy_version 1499949 (0.0007) [2023-12-27 16:02:58,692][20347] Updated weights for policy 0, policy_version 1501529 (0.0010) [2023-12-27 16:02:58,709][20381] Updated weights for policy 1, policy_version 1499959 (0.0010) [2023-12-27 16:02:58,749][20347] Updated weights for policy 0, policy_version 1501539 (0.0008) [2023-12-27 16:02:58,771][20381] Updated weights for policy 1, policy_version 1499969 (0.0008) [2023-12-27 16:02:58,815][20347] Updated weights for policy 0, policy_version 1501549 (0.0008) [2023-12-27 16:02:58,833][20381] Updated weights for policy 1, policy_version 1499979 (0.0010) [2023-12-27 16:02:58,885][20347] Updated weights for policy 0, policy_version 1501559 (0.0008) [2023-12-27 16:02:59,643][20381] Updated weights for policy 1, policy_version 1499989 (0.0008) [2023-12-27 16:02:59,673][20347] Updated weights for policy 0, policy_version 1501569 (0.0008) [2023-12-27 16:02:59,699][20381] Updated weights for policy 1, policy_version 1499999 (0.0007) [2023-12-27 16:02:59,717][20347] Updated weights for policy 0, policy_version 1501579 (0.0006) [2023-12-27 16:02:59,755][20381] Updated weights for policy 1, policy_version 1500009 (0.0007) [2023-12-27 16:02:59,775][20347] Updated weights for policy 0, policy_version 1501589 (0.0006) [2023-12-27 16:03:00,358][20381] Updated weights for policy 1, policy_version 1500019 (0.0007) [2023-12-27 16:03:00,411][20381] Updated weights for policy 1, policy_version 1500029 (0.0009) [2023-12-27 16:03:00,463][20381] Updated weights for policy 1, policy_version 1500039 (0.0008) [2023-12-27 16:03:00,494][20347] Updated weights for policy 0, policy_version 1501599 (0.0005) [2023-12-27 16:03:00,552][20347] Updated weights for policy 0, policy_version 1501609 (0.0006) [2023-12-27 16:03:00,603][20347] Updated weights for policy 0, policy_version 1501619 (0.0007) [2023-12-27 16:03:01,126][20381] Updated weights for policy 1, policy_version 1500049 (0.0008) [2023-12-27 16:03:01,185][20381] Updated weights for policy 1, policy_version 1500059 (0.0007) [2023-12-27 16:03:01,240][20381] Updated weights for policy 1, policy_version 1500069 (0.0007) [2023-12-27 16:03:01,290][20347] Updated weights for policy 0, policy_version 1501629 (0.0007) [2023-12-27 16:03:01,303][20381] Updated weights for policy 1, policy_version 1500079 (0.0007) [2023-12-27 16:03:01,348][20347] Updated weights for policy 0, policy_version 1501639 (0.0007) [2023-12-27 16:03:01,418][20347] Updated weights for policy 0, policy_version 1501649 (0.0008) [2023-12-27 16:03:02,009][20381] Updated weights for policy 1, policy_version 1500089 (0.0008) [2023-12-27 16:03:02,062][20381] Updated weights for policy 1, policy_version 1500099 (0.0008) [2023-12-27 16:03:02,095][20347] Updated weights for policy 0, policy_version 1501659 (0.0008) [2023-12-27 16:03:02,126][20381] Updated weights for policy 1, policy_version 1500109 (0.0008) [2023-12-27 16:03:02,156][20347] Updated weights for policy 0, policy_version 1501669 (0.0008) [2023-12-27 16:03:02,208][20347] Updated weights for policy 0, policy_version 1501679 (0.0008) [2023-12-27 16:03:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 768573440. Throughput: 0: 10364.0, 1: 10260.8. Samples: 768578956. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:02,363][19170] Avg episode reward: [(0, '990.340'), (1, '970.740')] [2023-12-27 16:03:02,857][20347] Updated weights for policy 0, policy_version 1501689 (0.0008) [2023-12-27 16:03:02,869][20381] Updated weights for policy 1, policy_version 1500119 (0.0006) [2023-12-27 16:03:02,912][20347] Updated weights for policy 0, policy_version 1501699 (0.0010) [2023-12-27 16:03:02,919][20381] Updated weights for policy 1, policy_version 1500129 (0.0005) [2023-12-27 16:03:02,968][20381] Updated weights for policy 1, policy_version 1500139 (0.0005) [2023-12-27 16:03:02,969][20347] Updated weights for policy 0, policy_version 1501709 (0.0010) [2023-12-27 16:03:03,025][20347] Updated weights for policy 0, policy_version 1501719 (0.0010) [2023-12-27 16:03:03,580][20381] Updated weights for policy 1, policy_version 1500149 (0.0005) [2023-12-27 16:03:03,596][20347] Updated weights for policy 0, policy_version 1501729 (0.0008) [2023-12-27 16:03:03,628][20381] Updated weights for policy 1, policy_version 1500159 (0.0005) [2023-12-27 16:03:03,655][20347] Updated weights for policy 0, policy_version 1501739 (0.0010) [2023-12-27 16:03:03,682][20381] Updated weights for policy 1, policy_version 1500169 (0.0005) [2023-12-27 16:03:03,713][20347] Updated weights for policy 0, policy_version 1501749 (0.0010) [2023-12-27 16:03:04,331][20347] Updated weights for policy 0, policy_version 1501759 (0.0006) [2023-12-27 16:03:04,391][20347] Updated weights for policy 0, policy_version 1501769 (0.0005) [2023-12-27 16:03:04,405][20381] Updated weights for policy 1, policy_version 1500179 (0.0006) [2023-12-27 16:03:04,459][20347] Updated weights for policy 0, policy_version 1501779 (0.0010) [2023-12-27 16:03:04,464][20381] Updated weights for policy 1, policy_version 1500189 (0.0006) [2023-12-27 16:03:04,523][20381] Updated weights for policy 1, policy_version 1500199 (0.0006) [2023-12-27 16:03:05,100][20347] Updated weights for policy 0, policy_version 1501789 (0.0009) [2023-12-27 16:03:05,151][20347] Updated weights for policy 0, policy_version 1501799 (0.0007) [2023-12-27 16:03:05,180][20381] Updated weights for policy 1, policy_version 1500209 (0.0006) [2023-12-27 16:03:05,205][20347] Updated weights for policy 0, policy_version 1501809 (0.0007) [2023-12-27 16:03:05,227][20381] Updated weights for policy 1, policy_version 1500219 (0.0006) [2023-12-27 16:03:05,278][20381] Updated weights for policy 1, policy_version 1500229 (0.0007) [2023-12-27 16:03:05,330][20381] Updated weights for policy 1, policy_version 1500239 (0.0007) [2023-12-27 16:03:05,847][20347] Updated weights for policy 0, policy_version 1501819 (0.0007) [2023-12-27 16:03:05,901][20347] Updated weights for policy 0, policy_version 1501829 (0.0010) [2023-12-27 16:03:05,955][20347] Updated weights for policy 0, policy_version 1501839 (0.0010) [2023-12-27 16:03:06,062][20381] Updated weights for policy 1, policy_version 1500249 (0.0009) [2023-12-27 16:03:06,134][20381] Updated weights for policy 1, policy_version 1500260 (0.0007) [2023-12-27 16:03:06,199][20381] Updated weights for policy 1, policy_version 1500270 (0.0007) [2023-12-27 16:03:06,615][20347] Updated weights for policy 0, policy_version 1501849 (0.0007) [2023-12-27 16:03:06,675][20347] Updated weights for policy 0, policy_version 1501859 (0.0009) [2023-12-27 16:03:06,740][20347] Updated weights for policy 0, policy_version 1501869 (0.0010) [2023-12-27 16:03:06,803][20347] Updated weights for policy 0, policy_version 1501879 (0.0011) [2023-12-27 16:03:06,899][20381] Updated weights for policy 1, policy_version 1500280 (0.0007) [2023-12-27 16:03:06,953][20381] Updated weights for policy 1, policy_version 1500290 (0.0007) [2023-12-27 16:03:07,009][20381] Updated weights for policy 1, policy_version 1500300 (0.0007) [2023-12-27 16:03:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 768671744. Throughput: 0: 10439.5, 1: 10267.6. Samples: 768642600. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:07,363][19170] Avg episode reward: [(0, '990.340'), (1, '980.530')] [2023-12-27 16:03:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001500304_384114688.pth... [2023-12-27 16:03:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001499120_383811584.pth [2023-12-27 16:03:07,409][20347] Updated weights for policy 0, policy_version 1501889 (0.0010) [2023-12-27 16:03:07,464][20347] Updated weights for policy 0, policy_version 1501899 (0.0010) [2023-12-27 16:03:07,521][20347] Updated weights for policy 0, policy_version 1501909 (0.0007) [2023-12-27 16:03:07,538][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001501912_384565248.pth... [2023-12-27 16:03:07,542][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001500664_384245760.pth [2023-12-27 16:03:07,757][20381] Updated weights for policy 1, policy_version 1500310 (0.0007) [2023-12-27 16:03:07,813][20381] Updated weights for policy 1, policy_version 1500320 (0.0009) [2023-12-27 16:03:07,866][20381] Updated weights for policy 1, policy_version 1500330 (0.0009) [2023-12-27 16:03:08,091][20347] Updated weights for policy 0, policy_version 1501919 (0.0007) [2023-12-27 16:03:08,139][20347] Updated weights for policy 0, policy_version 1501929 (0.0005) [2023-12-27 16:03:08,198][20347] Updated weights for policy 0, policy_version 1501939 (0.0009) [2023-12-27 16:03:08,551][20381] Updated weights for policy 1, policy_version 1500340 (0.0008) [2023-12-27 16:03:08,607][20381] Updated weights for policy 1, policy_version 1500350 (0.0011) [2023-12-27 16:03:08,670][20381] Updated weights for policy 1, policy_version 1500360 (0.0009) [2023-12-27 16:03:08,840][20347] Updated weights for policy 0, policy_version 1501949 (0.0007) [2023-12-27 16:03:08,899][20347] Updated weights for policy 0, policy_version 1501959 (0.0009) [2023-12-27 16:03:08,954][20347] Updated weights for policy 0, policy_version 1501969 (0.0008) [2023-12-27 16:03:09,383][20381] Updated weights for policy 1, policy_version 1500370 (0.0007) [2023-12-27 16:03:09,452][20381] Updated weights for policy 1, policy_version 1500380 (0.0006) [2023-12-27 16:03:09,514][20381] Updated weights for policy 1, policy_version 1500390 (0.0008) [2023-12-27 16:03:09,576][20347] Updated weights for policy 0, policy_version 1501979 (0.0006) [2023-12-27 16:03:09,578][20381] Updated weights for policy 1, policy_version 1500400 (0.0007) [2023-12-27 16:03:09,636][20347] Updated weights for policy 0, policy_version 1501989 (0.0007) [2023-12-27 16:03:09,696][20347] Updated weights for policy 0, policy_version 1501999 (0.0008) [2023-12-27 16:03:10,336][20381] Updated weights for policy 1, policy_version 1500410 (0.0007) [2023-12-27 16:03:10,379][20347] Updated weights for policy 0, policy_version 1502009 (0.0008) [2023-12-27 16:03:10,398][20381] Updated weights for policy 1, policy_version 1500420 (0.0008) [2023-12-27 16:03:10,439][20347] Updated weights for policy 0, policy_version 1502019 (0.0008) [2023-12-27 16:03:10,456][20381] Updated weights for policy 1, policy_version 1500430 (0.0007) [2023-12-27 16:03:10,495][20347] Updated weights for policy 0, policy_version 1502029 (0.0007) [2023-12-27 16:03:10,555][20347] Updated weights for policy 0, policy_version 1502039 (0.0007) [2023-12-27 16:03:11,118][20381] Updated weights for policy 1, policy_version 1500440 (0.0008) [2023-12-27 16:03:11,191][20381] Updated weights for policy 1, policy_version 1500450 (0.0008) [2023-12-27 16:03:11,243][20381] Updated weights for policy 1, policy_version 1500460 (0.0009) [2023-12-27 16:03:11,263][20347] Updated weights for policy 0, policy_version 1502049 (0.0008) [2023-12-27 16:03:11,327][20347] Updated weights for policy 0, policy_version 1502059 (0.0006) [2023-12-27 16:03:11,396][20347] Updated weights for policy 0, policy_version 1502069 (0.0008) [2023-12-27 16:03:11,942][20381] Updated weights for policy 1, policy_version 1500470 (0.0010) [2023-12-27 16:03:11,998][20381] Updated weights for policy 1, policy_version 1500480 (0.0010) [2023-12-27 16:03:12,062][20381] Updated weights for policy 1, policy_version 1500490 (0.0011) [2023-12-27 16:03:12,107][20347] Updated weights for policy 0, policy_version 1502079 (0.0008) [2023-12-27 16:03:12,163][20347] Updated weights for policy 0, policy_version 1502089 (0.0008) [2023-12-27 16:03:12,224][20347] Updated weights for policy 0, policy_version 1502099 (0.0008) [2023-12-27 16:03:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 768778240. Throughput: 0: 10569.5, 1: 10218.9. Samples: 768765304. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.850')] [2023-12-27 16:03:12,711][20381] Updated weights for policy 1, policy_version 1500500 (0.0010) [2023-12-27 16:03:12,771][20381] Updated weights for policy 1, policy_version 1500510 (0.0011) [2023-12-27 16:03:12,835][20381] Updated weights for policy 1, policy_version 1500520 (0.0009) [2023-12-27 16:03:12,977][20347] Updated weights for policy 0, policy_version 1502109 (0.0006) [2023-12-27 16:03:13,035][20347] Updated weights for policy 0, policy_version 1502119 (0.0007) [2023-12-27 16:03:13,092][20347] Updated weights for policy 0, policy_version 1502129 (0.0007) [2023-12-27 16:03:13,481][20381] Updated weights for policy 1, policy_version 1500530 (0.0006) [2023-12-27 16:03:13,534][20381] Updated weights for policy 1, policy_version 1500540 (0.0011) [2023-12-27 16:03:13,579][20381] Updated weights for policy 1, policy_version 1500550 (0.0010) [2023-12-27 16:03:13,624][20381] Updated weights for policy 1, policy_version 1500560 (0.0010) [2023-12-27 16:03:13,737][20347] Updated weights for policy 0, policy_version 1502139 (0.0006) [2023-12-27 16:03:13,786][20347] Updated weights for policy 0, policy_version 1502149 (0.0007) [2023-12-27 16:03:13,838][20347] Updated weights for policy 0, policy_version 1502159 (0.0005) [2023-12-27 16:03:14,323][20381] Updated weights for policy 1, policy_version 1500570 (0.0010) [2023-12-27 16:03:14,372][20381] Updated weights for policy 1, policy_version 1500580 (0.0010) [2023-12-27 16:03:14,398][20347] Updated weights for policy 0, policy_version 1502169 (0.0005) [2023-12-27 16:03:14,426][20381] Updated weights for policy 1, policy_version 1500590 (0.0010) [2023-12-27 16:03:14,451][20347] Updated weights for policy 0, policy_version 1502179 (0.0005) [2023-12-27 16:03:14,501][20347] Updated weights for policy 0, policy_version 1502189 (0.0005) [2023-12-27 16:03:14,554][20347] Updated weights for policy 0, policy_version 1502199 (0.0005) [2023-12-27 16:03:15,127][20347] Updated weights for policy 0, policy_version 1502209 (0.0009) [2023-12-27 16:03:15,139][20381] Updated weights for policy 1, policy_version 1500600 (0.0008) [2023-12-27 16:03:15,179][20347] Updated weights for policy 0, policy_version 1502219 (0.0010) [2023-12-27 16:03:15,196][20381] Updated weights for policy 1, policy_version 1500610 (0.0006) [2023-12-27 16:03:15,232][20347] Updated weights for policy 0, policy_version 1502229 (0.0010) [2023-12-27 16:03:15,253][20381] Updated weights for policy 1, policy_version 1500620 (0.0010) [2023-12-27 16:03:15,865][20347] Updated weights for policy 0, policy_version 1502239 (0.0008) [2023-12-27 16:03:15,908][20381] Updated weights for policy 1, policy_version 1500630 (0.0010) [2023-12-27 16:03:15,929][20347] Updated weights for policy 0, policy_version 1502249 (0.0007) [2023-12-27 16:03:15,963][20381] Updated weights for policy 1, policy_version 1500640 (0.0011) [2023-12-27 16:03:15,991][20347] Updated weights for policy 0, policy_version 1502259 (0.0009) [2023-12-27 16:03:16,018][20381] Updated weights for policy 1, policy_version 1500650 (0.0011) [2023-12-27 16:03:16,626][20347] Updated weights for policy 0, policy_version 1502269 (0.0010) [2023-12-27 16:03:16,626][20381] Updated weights for policy 1, policy_version 1500660 (0.0008) [2023-12-27 16:03:16,670][20381] Updated weights for policy 1, policy_version 1500670 (0.0005) [2023-12-27 16:03:16,672][20347] Updated weights for policy 0, policy_version 1502279 (0.0010) [2023-12-27 16:03:16,717][20381] Updated weights for policy 1, policy_version 1500680 (0.0008) [2023-12-27 16:03:16,720][20347] Updated weights for policy 0, policy_version 1502289 (0.0010) [2023-12-27 16:03:17,317][20381] Updated weights for policy 1, policy_version 1500690 (0.0008) [2023-12-27 16:03:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.6). Total num frames: 768876544. Throughput: 0: 10563.3, 1: 10248.1. Samples: 768891644. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.540')] [2023-12-27 16:03:17,367][20381] Updated weights for policy 1, policy_version 1500700 (0.0007) [2023-12-27 16:03:17,419][20381] Updated weights for policy 1, policy_version 1500710 (0.0010) [2023-12-27 16:03:17,424][20347] Updated weights for policy 0, policy_version 1502299 (0.0009) [2023-12-27 16:03:17,469][20381] Updated weights for policy 1, policy_version 1500720 (0.0010) [2023-12-27 16:03:17,474][20347] Updated weights for policy 0, policy_version 1502309 (0.0006) [2023-12-27 16:03:17,525][20347] Updated weights for policy 0, policy_version 1502319 (0.0007) [2023-12-27 16:03:18,165][20381] Updated weights for policy 1, policy_version 1500730 (0.0005) [2023-12-27 16:03:18,192][20347] Updated weights for policy 0, policy_version 1502329 (0.0008) [2023-12-27 16:03:18,219][20381] Updated weights for policy 1, policy_version 1500740 (0.0005) [2023-12-27 16:03:18,239][20347] Updated weights for policy 0, policy_version 1502339 (0.0007) [2023-12-27 16:03:18,275][20381] Updated weights for policy 1, policy_version 1500750 (0.0005) [2023-12-27 16:03:18,289][20347] Updated weights for policy 0, policy_version 1502349 (0.0008) [2023-12-27 16:03:18,350][20347] Updated weights for policy 0, policy_version 1502359 (0.0008) [2023-12-27 16:03:18,955][20381] Updated weights for policy 1, policy_version 1500760 (0.0008) [2023-12-27 16:03:18,992][20347] Updated weights for policy 0, policy_version 1502369 (0.0006) [2023-12-27 16:03:19,017][20381] Updated weights for policy 1, policy_version 1500770 (0.0007) [2023-12-27 16:03:19,043][20347] Updated weights for policy 0, policy_version 1502379 (0.0005) [2023-12-27 16:03:19,075][20381] Updated weights for policy 1, policy_version 1500780 (0.0007) [2023-12-27 16:03:19,094][20347] Updated weights for policy 0, policy_version 1502389 (0.0006) [2023-12-27 16:03:19,693][20347] Updated weights for policy 0, policy_version 1502399 (0.0005) [2023-12-27 16:03:19,753][20347] Updated weights for policy 0, policy_version 1502409 (0.0005) [2023-12-27 16:03:19,813][20347] Updated weights for policy 0, policy_version 1502419 (0.0006) [2023-12-27 16:03:19,907][20381] Updated weights for policy 1, policy_version 1500790 (0.0007) [2023-12-27 16:03:19,968][20381] Updated weights for policy 1, policy_version 1500800 (0.0008) [2023-12-27 16:03:20,024][20381] Updated weights for policy 1, policy_version 1500810 (0.0009) [2023-12-27 16:03:20,369][20347] Updated weights for policy 0, policy_version 1502429 (0.0008) [2023-12-27 16:03:20,430][20347] Updated weights for policy 0, policy_version 1502439 (0.0006) [2023-12-27 16:03:20,487][20347] Updated weights for policy 0, policy_version 1502449 (0.0007) [2023-12-27 16:03:20,780][20381] Updated weights for policy 1, policy_version 1500820 (0.0007) [2023-12-27 16:03:20,837][20381] Updated weights for policy 1, policy_version 1500830 (0.0007) [2023-12-27 16:03:20,892][20381] Updated weights for policy 1, policy_version 1500840 (0.0008) [2023-12-27 16:03:21,186][20347] Updated weights for policy 0, policy_version 1502459 (0.0006) [2023-12-27 16:03:21,240][20347] Updated weights for policy 0, policy_version 1502469 (0.0008) [2023-12-27 16:03:21,298][20347] Updated weights for policy 0, policy_version 1502479 (0.0008) [2023-12-27 16:03:21,613][20381] Updated weights for policy 1, policy_version 1500850 (0.0008) [2023-12-27 16:03:21,678][20381] Updated weights for policy 1, policy_version 1500860 (0.0007) [2023-12-27 16:03:21,746][20381] Updated weights for policy 1, policy_version 1500870 (0.0006) [2023-12-27 16:03:21,810][20381] Updated weights for policy 1, policy_version 1500880 (0.0008) [2023-12-27 16:03:22,043][20347] Updated weights for policy 0, policy_version 1502489 (0.0007) [2023-12-27 16:03:22,106][20347] Updated weights for policy 0, policy_version 1502499 (0.0009) [2023-12-27 16:03:22,162][20347] Updated weights for policy 0, policy_version 1502509 (0.0008) [2023-12-27 16:03:22,220][20347] Updated weights for policy 0, policy_version 1502519 (0.0008) [2023-12-27 16:03:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 768983040. Throughput: 0: 10528.5, 1: 10154.8. Samples: 768953236. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.520')] [2023-12-27 16:03:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001500880_384262144.pth... [2023-12-27 16:03:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001502520_384720896.pth... [2023-12-27 16:03:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001499728_383967232.pth [2023-12-27 16:03:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001501272_384401408.pth [2023-12-27 16:03:22,504][20381] Updated weights for policy 1, policy_version 1500890 (0.0008) [2023-12-27 16:03:22,566][20381] Updated weights for policy 1, policy_version 1500900 (0.0008) [2023-12-27 16:03:22,626][20381] Updated weights for policy 1, policy_version 1500910 (0.0008) [2023-12-27 16:03:22,943][20347] Updated weights for policy 0, policy_version 1502529 (0.0006) [2023-12-27 16:03:22,997][20347] Updated weights for policy 0, policy_version 1502539 (0.0005) [2023-12-27 16:03:23,048][20347] Updated weights for policy 0, policy_version 1502549 (0.0005) [2023-12-27 16:03:23,373][20381] Updated weights for policy 1, policy_version 1500920 (0.0008) [2023-12-27 16:03:23,391][20292] KL-divergence is very high: 117.4603 [2023-12-27 16:03:23,400][20292] KL-divergence is very high: 132.1421 [2023-12-27 16:03:23,421][20381] Updated weights for policy 1, policy_version 1500930 (0.0010) [2023-12-27 16:03:23,433][20292] KL-divergence is very high: 109.3004 [2023-12-27 16:03:23,447][20292] KL-divergence is very high: 106.7545 [2023-12-27 16:03:23,491][20381] Updated weights for policy 1, policy_version 1500940 (0.0008) [2023-12-27 16:03:23,635][20347] Updated weights for policy 0, policy_version 1502559 (0.0007) [2023-12-27 16:03:23,688][20347] Updated weights for policy 0, policy_version 1502569 (0.0007) [2023-12-27 16:03:23,746][20347] Updated weights for policy 0, policy_version 1502579 (0.0007) [2023-12-27 16:03:24,224][20381] Updated weights for policy 1, policy_version 1500950 (0.0008) [2023-12-27 16:03:24,276][20381] Updated weights for policy 1, policy_version 1500960 (0.0007) [2023-12-27 16:03:24,335][20381] Updated weights for policy 1, policy_version 1500970 (0.0008) [2023-12-27 16:03:24,368][20347] Updated weights for policy 0, policy_version 1502589 (0.0007) [2023-12-27 16:03:24,431][20347] Updated weights for policy 0, policy_version 1502599 (0.0008) [2023-12-27 16:03:24,481][20347] Updated weights for policy 0, policy_version 1502609 (0.0008) [2023-12-27 16:03:25,030][20381] Updated weights for policy 1, policy_version 1500980 (0.0007) [2023-12-27 16:03:25,074][20381] Updated weights for policy 1, policy_version 1500990 (0.0007) [2023-12-27 16:03:25,122][20381] Updated weights for policy 1, policy_version 1501000 (0.0007) [2023-12-27 16:03:25,172][20347] Updated weights for policy 0, policy_version 1502619 (0.0008) [2023-12-27 16:03:25,234][20347] Updated weights for policy 0, policy_version 1502629 (0.0008) [2023-12-27 16:03:25,288][20347] Updated weights for policy 0, policy_version 1502639 (0.0006) [2023-12-27 16:03:25,869][20381] Updated weights for policy 1, policy_version 1501010 (0.0008) [2023-12-27 16:03:25,934][20381] Updated weights for policy 1, policy_version 1501020 (0.0011) [2023-12-27 16:03:25,975][20347] Updated weights for policy 0, policy_version 1502649 (0.0005) [2023-12-27 16:03:25,982][20381] Updated weights for policy 1, policy_version 1501030 (0.0011) [2023-12-27 16:03:26,031][20347] Updated weights for policy 0, policy_version 1502659 (0.0006) [2023-12-27 16:03:26,038][20381] Updated weights for policy 1, policy_version 1501040 (0.0009) [2023-12-27 16:03:26,094][20347] Updated weights for policy 0, policy_version 1502669 (0.0007) [2023-12-27 16:03:26,155][20347] Updated weights for policy 0, policy_version 1502679 (0.0007) [2023-12-27 16:03:26,658][20381] Updated weights for policy 1, policy_version 1501050 (0.0008) [2023-12-27 16:03:26,707][20381] Updated weights for policy 1, policy_version 1501060 (0.0007) [2023-12-27 16:03:26,755][20381] Updated weights for policy 1, policy_version 1501070 (0.0008) [2023-12-27 16:03:26,867][20347] Updated weights for policy 0, policy_version 1502689 (0.0006) [2023-12-27 16:03:26,925][20347] Updated weights for policy 0, policy_version 1502699 (0.0005) [2023-12-27 16:03:26,983][20347] Updated weights for policy 0, policy_version 1502709 (0.0005) [2023-12-27 16:03:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 769081344. Throughput: 0: 10490.9, 1: 10098.9. Samples: 769073536. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.740')] [2023-12-27 16:03:27,401][20381] Updated weights for policy 1, policy_version 1501080 (0.0006) [2023-12-27 16:03:27,453][20381] Updated weights for policy 1, policy_version 1501090 (0.0005) [2023-12-27 16:03:27,505][20381] Updated weights for policy 1, policy_version 1501100 (0.0005) [2023-12-27 16:03:27,643][20347] Updated weights for policy 0, policy_version 1502719 (0.0007) [2023-12-27 16:03:27,695][20347] Updated weights for policy 0, policy_version 1502729 (0.0007) [2023-12-27 16:03:27,750][20347] Updated weights for policy 0, policy_version 1502739 (0.0007) [2023-12-27 16:03:28,127][20381] Updated weights for policy 1, policy_version 1501110 (0.0008) [2023-12-27 16:03:28,186][20381] Updated weights for policy 1, policy_version 1501120 (0.0005) [2023-12-27 16:03:28,237][20381] Updated weights for policy 1, policy_version 1501130 (0.0005) [2023-12-27 16:03:28,325][20347] Updated weights for policy 0, policy_version 1502749 (0.0007) [2023-12-27 16:03:28,389][20347] Updated weights for policy 0, policy_version 1502759 (0.0007) [2023-12-27 16:03:28,452][20347] Updated weights for policy 0, policy_version 1502769 (0.0006) [2023-12-27 16:03:28,896][20381] Updated weights for policy 1, policy_version 1501140 (0.0005) [2023-12-27 16:03:28,949][20381] Updated weights for policy 1, policy_version 1501150 (0.0010) [2023-12-27 16:03:28,994][20381] Updated weights for policy 1, policy_version 1501160 (0.0010) [2023-12-27 16:03:29,084][20347] Updated weights for policy 0, policy_version 1502779 (0.0007) [2023-12-27 16:03:29,134][20347] Updated weights for policy 0, policy_version 1502789 (0.0008) [2023-12-27 16:03:29,188][20347] Updated weights for policy 0, policy_version 1502799 (0.0007) [2023-12-27 16:03:29,693][20381] Updated weights for policy 1, policy_version 1501170 (0.0009) [2023-12-27 16:03:29,741][20381] Updated weights for policy 1, policy_version 1501180 (0.0007) [2023-12-27 16:03:29,793][20381] Updated weights for policy 1, policy_version 1501190 (0.0007) [2023-12-27 16:03:29,851][20381] Updated weights for policy 1, policy_version 1501200 (0.0011) [2023-12-27 16:03:29,863][20347] Updated weights for policy 0, policy_version 1502809 (0.0007) [2023-12-27 16:03:29,926][20347] Updated weights for policy 0, policy_version 1502819 (0.0008) [2023-12-27 16:03:29,989][20347] Updated weights for policy 0, policy_version 1502829 (0.0008) [2023-12-27 16:03:30,044][20347] Updated weights for policy 0, policy_version 1502839 (0.0007) [2023-12-27 16:03:30,534][20381] Updated weights for policy 1, policy_version 1501210 (0.0010) [2023-12-27 16:03:30,584][20381] Updated weights for policy 1, policy_version 1501220 (0.0010) [2023-12-27 16:03:30,636][20381] Updated weights for policy 1, policy_version 1501230 (0.0010) [2023-12-27 16:03:30,689][20347] Updated weights for policy 0, policy_version 1502849 (0.0008) [2023-12-27 16:03:30,754][20347] Updated weights for policy 0, policy_version 1502859 (0.0006) [2023-12-27 16:03:30,807][20347] Updated weights for policy 0, policy_version 1502869 (0.0008) [2023-12-27 16:03:31,290][20381] Updated weights for policy 1, policy_version 1501240 (0.0011) [2023-12-27 16:03:31,350][20381] Updated weights for policy 1, policy_version 1501250 (0.0011) [2023-12-27 16:03:31,409][20381] Updated weights for policy 1, policy_version 1501260 (0.0011) [2023-12-27 16:03:31,492][20347] Updated weights for policy 0, policy_version 1502879 (0.0008) [2023-12-27 16:03:31,540][20347] Updated weights for policy 0, policy_version 1502889 (0.0010) [2023-12-27 16:03:31,589][20347] Updated weights for policy 0, policy_version 1502899 (0.0010) [2023-12-27 16:03:32,106][20381] Updated weights for policy 1, policy_version 1501270 (0.0007) [2023-12-27 16:03:32,170][20381] Updated weights for policy 1, policy_version 1501280 (0.0008) [2023-12-27 16:03:32,228][20381] Updated weights for policy 1, policy_version 1501290 (0.0008) [2023-12-27 16:03:32,279][20347] Updated weights for policy 0, policy_version 1502909 (0.0010) [2023-12-27 16:03:32,338][20347] Updated weights for policy 0, policy_version 1502919 (0.0011) [2023-12-27 16:03:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 769187840. Throughput: 0: 10505.6, 1: 10093.2. Samples: 769199380. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.240')] [2023-12-27 16:03:32,404][20347] Updated weights for policy 0, policy_version 1502929 (0.0009) [2023-12-27 16:03:32,909][20381] Updated weights for policy 1, policy_version 1501300 (0.0007) [2023-12-27 16:03:32,953][20381] Updated weights for policy 1, policy_version 1501310 (0.0005) [2023-12-27 16:03:32,999][20381] Updated weights for policy 1, policy_version 1501320 (0.0007) [2023-12-27 16:03:33,070][20347] Updated weights for policy 0, policy_version 1502939 (0.0008) [2023-12-27 16:03:33,124][20347] Updated weights for policy 0, policy_version 1502949 (0.0009) [2023-12-27 16:03:33,184][20347] Updated weights for policy 0, policy_version 1502959 (0.0007) [2023-12-27 16:03:33,624][20381] Updated weights for policy 1, policy_version 1501330 (0.0009) [2023-12-27 16:03:33,672][20381] Updated weights for policy 1, policy_version 1501340 (0.0005) [2023-12-27 16:03:33,722][20381] Updated weights for policy 1, policy_version 1501350 (0.0006) [2023-12-27 16:03:33,751][20347] Updated weights for policy 0, policy_version 1502969 (0.0006) [2023-12-27 16:03:33,770][20381] Updated weights for policy 1, policy_version 1501360 (0.0008) [2023-12-27 16:03:33,803][20347] Updated weights for policy 0, policy_version 1502979 (0.0008) [2023-12-27 16:03:33,850][20347] Updated weights for policy 0, policy_version 1502989 (0.0007) [2023-12-27 16:03:33,897][20347] Updated weights for policy 0, policy_version 1502999 (0.0007) [2023-12-27 16:03:34,418][20381] Updated weights for policy 1, policy_version 1501370 (0.0011) [2023-12-27 16:03:34,474][20381] Updated weights for policy 1, policy_version 1501380 (0.0011) [2023-12-27 16:03:34,536][20381] Updated weights for policy 1, policy_version 1501390 (0.0010) [2023-12-27 16:03:34,666][20347] Updated weights for policy 0, policy_version 1503009 (0.0007) [2023-12-27 16:03:34,725][20347] Updated weights for policy 0, policy_version 1503019 (0.0010) [2023-12-27 16:03:34,779][20347] Updated weights for policy 0, policy_version 1503029 (0.0010) [2023-12-27 16:03:35,187][20381] Updated weights for policy 1, policy_version 1501400 (0.0010) [2023-12-27 16:03:35,243][20381] Updated weights for policy 1, policy_version 1501410 (0.0010) [2023-12-27 16:03:35,300][20381] Updated weights for policy 1, policy_version 1501420 (0.0010) [2023-12-27 16:03:35,423][20347] Updated weights for policy 0, policy_version 1503039 (0.0010) [2023-12-27 16:03:35,474][20347] Updated weights for policy 0, policy_version 1503049 (0.0010) [2023-12-27 16:03:35,531][20347] Updated weights for policy 0, policy_version 1503059 (0.0010) [2023-12-27 16:03:35,952][20381] Updated weights for policy 1, policy_version 1501430 (0.0011) [2023-12-27 16:03:36,008][20381] Updated weights for policy 1, policy_version 1501440 (0.0010) [2023-12-27 16:03:36,059][20381] Updated weights for policy 1, policy_version 1501450 (0.0010) [2023-12-27 16:03:36,214][20347] Updated weights for policy 0, policy_version 1503069 (0.0010) [2023-12-27 16:03:36,276][20347] Updated weights for policy 0, policy_version 1503079 (0.0009) [2023-12-27 16:03:36,333][20347] Updated weights for policy 0, policy_version 1503089 (0.0011) [2023-12-27 16:03:36,737][20381] Updated weights for policy 1, policy_version 1501460 (0.0010) [2023-12-27 16:03:36,785][20381] Updated weights for policy 1, policy_version 1501470 (0.0006) [2023-12-27 16:03:36,844][20381] Updated weights for policy 1, policy_version 1501480 (0.0006) [2023-12-27 16:03:36,999][20347] Updated weights for policy 0, policy_version 1503099 (0.0009) [2023-12-27 16:03:37,050][20347] Updated weights for policy 0, policy_version 1503109 (0.0005) [2023-12-27 16:03:37,109][20347] Updated weights for policy 0, policy_version 1503119 (0.0006) [2023-12-27 16:03:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 769294336. Throughput: 0: 10470.0, 1: 10123.8. Samples: 769262716. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.790')] [2023-12-27 16:03:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001501488_384417792.pth... [2023-12-27 16:03:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001503128_384876544.pth... [2023-12-27 16:03:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001500304_384114688.pth [2023-12-27 16:03:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001501912_384565248.pth [2023-12-27 16:03:37,524][20381] Updated weights for policy 1, policy_version 1501490 (0.0007) [2023-12-27 16:03:37,579][20381] Updated weights for policy 1, policy_version 1501500 (0.0006) [2023-12-27 16:03:37,629][20381] Updated weights for policy 1, policy_version 1501510 (0.0007) [2023-12-27 16:03:37,684][20381] Updated weights for policy 1, policy_version 1501520 (0.0007) [2023-12-27 16:03:37,760][20347] Updated weights for policy 0, policy_version 1503129 (0.0007) [2023-12-27 16:03:37,811][20347] Updated weights for policy 0, policy_version 1503139 (0.0005) [2023-12-27 16:03:37,864][20347] Updated weights for policy 0, policy_version 1503149 (0.0008) [2023-12-27 16:03:37,909][20347] Updated weights for policy 0, policy_version 1503159 (0.0010) [2023-12-27 16:03:38,418][20381] Updated weights for policy 1, policy_version 1501530 (0.0007) [2023-12-27 16:03:38,479][20381] Updated weights for policy 1, policy_version 1501540 (0.0008) [2023-12-27 16:03:38,541][20381] Updated weights for policy 1, policy_version 1501550 (0.0008) [2023-12-27 16:03:38,569][20347] Updated weights for policy 0, policy_version 1503169 (0.0006) [2023-12-27 16:03:38,629][20347] Updated weights for policy 0, policy_version 1503179 (0.0007) [2023-12-27 16:03:38,680][20347] Updated weights for policy 0, policy_version 1503189 (0.0008) [2023-12-27 16:03:39,269][20381] Updated weights for policy 1, policy_version 1501560 (0.0008) [2023-12-27 16:03:39,332][20381] Updated weights for policy 1, policy_version 1501570 (0.0009) [2023-12-27 16:03:39,395][20347] Updated weights for policy 0, policy_version 1503199 (0.0009) [2023-12-27 16:03:39,396][20381] Updated weights for policy 1, policy_version 1501580 (0.0009) [2023-12-27 16:03:39,460][20347] Updated weights for policy 0, policy_version 1503209 (0.0006) [2023-12-27 16:03:39,517][20347] Updated weights for policy 0, policy_version 1503219 (0.0005) [2023-12-27 16:03:40,149][20381] Updated weights for policy 1, policy_version 1501590 (0.0008) [2023-12-27 16:03:40,171][20347] Updated weights for policy 0, policy_version 1503229 (0.0007) [2023-12-27 16:03:40,207][20381] Updated weights for policy 1, policy_version 1501600 (0.0007) [2023-12-27 16:03:40,239][20347] Updated weights for policy 0, policy_version 1503239 (0.0006) [2023-12-27 16:03:40,260][20381] Updated weights for policy 1, policy_version 1501610 (0.0005) [2023-12-27 16:03:40,298][20347] Updated weights for policy 0, policy_version 1503249 (0.0006) [2023-12-27 16:03:40,957][20381] Updated weights for policy 1, policy_version 1501620 (0.0007) [2023-12-27 16:03:40,990][20347] Updated weights for policy 0, policy_version 1503259 (0.0008) [2023-12-27 16:03:41,008][20381] Updated weights for policy 1, policy_version 1501630 (0.0007) [2023-12-27 16:03:41,047][20347] Updated weights for policy 0, policy_version 1503269 (0.0008) [2023-12-27 16:03:41,073][20381] Updated weights for policy 1, policy_version 1501640 (0.0006) [2023-12-27 16:03:41,106][20347] Updated weights for policy 0, policy_version 1503279 (0.0007) [2023-12-27 16:03:41,888][20381] Updated weights for policy 1, policy_version 1501650 (0.0008) [2023-12-27 16:03:41,900][20347] Updated weights for policy 0, policy_version 1503289 (0.0007) [2023-12-27 16:03:41,943][20381] Updated weights for policy 1, policy_version 1501660 (0.0008) [2023-12-27 16:03:41,962][20347] Updated weights for policy 0, policy_version 1503299 (0.0005) [2023-12-27 16:03:41,992][20381] Updated weights for policy 1, policy_version 1501670 (0.0008) [2023-12-27 16:03:42,017][20347] Updated weights for policy 0, policy_version 1503309 (0.0006) [2023-12-27 16:03:42,046][20381] Updated weights for policy 1, policy_version 1501680 (0.0007) [2023-12-27 16:03:42,068][20347] Updated weights for policy 0, policy_version 1503319 (0.0006) [2023-12-27 16:03:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 769392640. Throughput: 0: 10478.1, 1: 10076.9. Samples: 769383264. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.330')] [2023-12-27 16:03:42,760][20381] Updated weights for policy 1, policy_version 1501690 (0.0008) [2023-12-27 16:03:42,769][20347] Updated weights for policy 0, policy_version 1503329 (0.0007) [2023-12-27 16:03:42,815][20347] Updated weights for policy 0, policy_version 1503339 (0.0006) [2023-12-27 16:03:42,829][20381] Updated weights for policy 1, policy_version 1501700 (0.0006) [2023-12-27 16:03:42,871][20347] Updated weights for policy 0, policy_version 1503349 (0.0008) [2023-12-27 16:03:42,889][20381] Updated weights for policy 1, policy_version 1501710 (0.0007) [2023-12-27 16:03:43,556][20347] Updated weights for policy 0, policy_version 1503359 (0.0007) [2023-12-27 16:03:43,571][20381] Updated weights for policy 1, policy_version 1501720 (0.0007) [2023-12-27 16:03:43,615][20347] Updated weights for policy 0, policy_version 1503369 (0.0005) [2023-12-27 16:03:43,622][20381] Updated weights for policy 1, policy_version 1501730 (0.0007) [2023-12-27 16:03:43,670][20381] Updated weights for policy 1, policy_version 1501740 (0.0005) [2023-12-27 16:03:43,672][20347] Updated weights for policy 0, policy_version 1503379 (0.0006) [2023-12-27 16:03:44,301][20381] Updated weights for policy 1, policy_version 1501750 (0.0009) [2023-12-27 16:03:44,317][20347] Updated weights for policy 0, policy_version 1503389 (0.0006) [2023-12-27 16:03:44,362][20381] Updated weights for policy 1, policy_version 1501760 (0.0010) [2023-12-27 16:03:44,374][20347] Updated weights for policy 0, policy_version 1503399 (0.0005) [2023-12-27 16:03:44,422][20381] Updated weights for policy 1, policy_version 1501770 (0.0011) [2023-12-27 16:03:44,426][20347] Updated weights for policy 0, policy_version 1503409 (0.0006) [2023-12-27 16:03:45,047][20381] Updated weights for policy 1, policy_version 1501780 (0.0008) [2023-12-27 16:03:45,096][20347] Updated weights for policy 0, policy_version 1503419 (0.0009) [2023-12-27 16:03:45,100][20381] Updated weights for policy 1, policy_version 1501790 (0.0006) [2023-12-27 16:03:45,151][20347] Updated weights for policy 0, policy_version 1503429 (0.0010) [2023-12-27 16:03:45,160][20381] Updated weights for policy 1, policy_version 1501800 (0.0006) [2023-12-27 16:03:45,222][20347] Updated weights for policy 0, policy_version 1503439 (0.0010) [2023-12-27 16:03:45,750][20381] Updated weights for policy 1, policy_version 1501810 (0.0006) [2023-12-27 16:03:45,809][20381] Updated weights for policy 1, policy_version 1501820 (0.0008) [2023-12-27 16:03:45,861][20381] Updated weights for policy 1, policy_version 1501830 (0.0010) [2023-12-27 16:03:45,894][20347] Updated weights for policy 0, policy_version 1503449 (0.0010) [2023-12-27 16:03:45,920][20381] Updated weights for policy 1, policy_version 1501840 (0.0011) [2023-12-27 16:03:45,951][20347] Updated weights for policy 0, policy_version 1503459 (0.0010) [2023-12-27 16:03:46,000][20347] Updated weights for policy 0, policy_version 1503469 (0.0010) [2023-12-27 16:03:46,044][20347] Updated weights for policy 0, policy_version 1503479 (0.0010) [2023-12-27 16:03:46,574][20381] Updated weights for policy 1, policy_version 1501850 (0.0005) [2023-12-27 16:03:46,633][20381] Updated weights for policy 1, policy_version 1501860 (0.0005) [2023-12-27 16:03:46,643][20347] Updated weights for policy 0, policy_version 1503489 (0.0008) [2023-12-27 16:03:46,691][20381] Updated weights for policy 1, policy_version 1501870 (0.0006) [2023-12-27 16:03:46,693][20347] Updated weights for policy 0, policy_version 1503499 (0.0007) [2023-12-27 16:03:46,754][20347] Updated weights for policy 0, policy_version 1503509 (0.0007) [2023-12-27 16:03:47,279][20381] Updated weights for policy 1, policy_version 1501880 (0.0006) [2023-12-27 16:03:47,332][20381] Updated weights for policy 1, policy_version 1501890 (0.0006) [2023-12-27 16:03:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 769490944. Throughput: 0: 10479.5, 1: 10156.3. Samples: 769507568. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.590')] [2023-12-27 16:03:47,390][20381] Updated weights for policy 1, policy_version 1501900 (0.0010) [2023-12-27 16:03:47,444][20347] Updated weights for policy 0, policy_version 1503519 (0.0007) [2023-12-27 16:03:47,499][20347] Updated weights for policy 0, policy_version 1503529 (0.0007) [2023-12-27 16:03:47,553][20347] Updated weights for policy 0, policy_version 1503539 (0.0008) [2023-12-27 16:03:48,016][20381] Updated weights for policy 1, policy_version 1501910 (0.0007) [2023-12-27 16:03:48,064][20381] Updated weights for policy 1, policy_version 1501920 (0.0007) [2023-12-27 16:03:48,116][20381] Updated weights for policy 1, policy_version 1501930 (0.0008) [2023-12-27 16:03:48,145][20347] Updated weights for policy 0, policy_version 1503549 (0.0006) [2023-12-27 16:03:48,201][20347] Updated weights for policy 0, policy_version 1503559 (0.0007) [2023-12-27 16:03:48,250][20347] Updated weights for policy 0, policy_version 1503569 (0.0005) [2023-12-27 16:03:48,817][20381] Updated weights for policy 1, policy_version 1501940 (0.0008) [2023-12-27 16:03:48,876][20381] Updated weights for policy 1, policy_version 1501950 (0.0008) [2023-12-27 16:03:48,936][20381] Updated weights for policy 1, policy_version 1501960 (0.0008) [2023-12-27 16:03:48,980][20347] Updated weights for policy 0, policy_version 1503579 (0.0006) [2023-12-27 16:03:49,040][20347] Updated weights for policy 0, policy_version 1503589 (0.0008) [2023-12-27 16:03:49,092][20347] Updated weights for policy 0, policy_version 1503599 (0.0009) [2023-12-27 16:03:49,618][20381] Updated weights for policy 1, policy_version 1501970 (0.0009) [2023-12-27 16:03:49,678][20381] Updated weights for policy 1, policy_version 1501980 (0.0011) [2023-12-27 16:03:49,742][20381] Updated weights for policy 1, policy_version 1501990 (0.0011) [2023-12-27 16:03:49,795][20381] Updated weights for policy 1, policy_version 1502000 (0.0009) [2023-12-27 16:03:49,815][20347] Updated weights for policy 0, policy_version 1503609 (0.0008) [2023-12-27 16:03:49,876][20347] Updated weights for policy 0, policy_version 1503619 (0.0007) [2023-12-27 16:03:49,942][20347] Updated weights for policy 0, policy_version 1503629 (0.0007) [2023-12-27 16:03:49,990][20347] Updated weights for policy 0, policy_version 1503639 (0.0007) [2023-12-27 16:03:50,488][20381] Updated weights for policy 1, policy_version 1502010 (0.0008) [2023-12-27 16:03:50,544][20381] Updated weights for policy 1, policy_version 1502020 (0.0008) [2023-12-27 16:03:50,603][20381] Updated weights for policy 1, policy_version 1502030 (0.0007) [2023-12-27 16:03:50,691][20347] Updated weights for policy 0, policy_version 1503649 (0.0010) [2023-12-27 16:03:50,745][20347] Updated weights for policy 0, policy_version 1503659 (0.0009) [2023-12-27 16:03:50,800][20347] Updated weights for policy 0, policy_version 1503669 (0.0006) [2023-12-27 16:03:51,317][20381] Updated weights for policy 1, policy_version 1502040 (0.0008) [2023-12-27 16:03:51,384][20381] Updated weights for policy 1, policy_version 1502050 (0.0008) [2023-12-27 16:03:51,446][20381] Updated weights for policy 1, policy_version 1502060 (0.0007) [2023-12-27 16:03:51,491][20347] Updated weights for policy 0, policy_version 1503679 (0.0008) [2023-12-27 16:03:51,539][20347] Updated weights for policy 0, policy_version 1503689 (0.0008) [2023-12-27 16:03:51,596][20347] Updated weights for policy 0, policy_version 1503699 (0.0006) [2023-12-27 16:03:52,158][20381] Updated weights for policy 1, policy_version 1502070 (0.0008) [2023-12-27 16:03:52,211][20381] Updated weights for policy 1, policy_version 1502080 (0.0008) [2023-12-27 16:03:52,275][20381] Updated weights for policy 1, policy_version 1502090 (0.0009) [2023-12-27 16:03:52,329][20347] Updated weights for policy 0, policy_version 1503709 (0.0008) [2023-12-27 16:03:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 769597440. Throughput: 0: 10428.8, 1: 10157.2. Samples: 769568972. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.550')] [2023-12-27 16:03:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001502096_384573440.pth... [2023-12-27 16:03:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001500880_384262144.pth [2023-12-27 16:03:52,395][20347] Updated weights for policy 0, policy_version 1503719 (0.0007) [2023-12-27 16:03:52,461][20347] Updated weights for policy 0, policy_version 1503729 (0.0008) [2023-12-27 16:03:52,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001503736_385032192.pth... [2023-12-27 16:03:52,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001502520_384720896.pth [2023-12-27 16:03:52,979][20381] Updated weights for policy 1, policy_version 1502100 (0.0007) [2023-12-27 16:03:53,034][20381] Updated weights for policy 1, policy_version 1502110 (0.0007) [2023-12-27 16:03:53,088][20381] Updated weights for policy 1, policy_version 1502120 (0.0008) [2023-12-27 16:03:53,139][20347] Updated weights for policy 0, policy_version 1503739 (0.0009) [2023-12-27 16:03:53,198][20347] Updated weights for policy 0, policy_version 1503749 (0.0009) [2023-12-27 16:03:53,269][20347] Updated weights for policy 0, policy_version 1503759 (0.0008) [2023-12-27 16:03:53,663][20381] Updated weights for policy 1, policy_version 1502130 (0.0007) [2023-12-27 16:03:53,712][20381] Updated weights for policy 1, policy_version 1502140 (0.0007) [2023-12-27 16:03:53,760][20381] Updated weights for policy 1, policy_version 1502150 (0.0008) [2023-12-27 16:03:53,809][20381] Updated weights for policy 1, policy_version 1502160 (0.0007) [2023-12-27 16:03:53,921][20347] Updated weights for policy 0, policy_version 1503769 (0.0006) [2023-12-27 16:03:53,969][20347] Updated weights for policy 0, policy_version 1503779 (0.0010) [2023-12-27 16:03:54,021][20347] Updated weights for policy 0, policy_version 1503789 (0.0010) [2023-12-27 16:03:54,082][20347] Updated weights for policy 0, policy_version 1503799 (0.0010) [2023-12-27 16:03:54,514][20381] Updated weights for policy 1, policy_version 1502170 (0.0007) [2023-12-27 16:03:54,569][20381] Updated weights for policy 1, policy_version 1502180 (0.0008) [2023-12-27 16:03:54,616][20381] Updated weights for policy 1, policy_version 1502190 (0.0007) [2023-12-27 16:03:54,760][20347] Updated weights for policy 0, policy_version 1503809 (0.0010) [2023-12-27 16:03:54,820][20347] Updated weights for policy 0, policy_version 1503819 (0.0010) [2023-12-27 16:03:54,880][20347] Updated weights for policy 0, policy_version 1503829 (0.0010) [2023-12-27 16:03:55,349][20381] Updated weights for policy 1, policy_version 1502200 (0.0005) [2023-12-27 16:03:55,418][20381] Updated weights for policy 1, policy_version 1502210 (0.0005) [2023-12-27 16:03:55,478][20381] Updated weights for policy 1, policy_version 1502220 (0.0008) [2023-12-27 16:03:55,492][20347] Updated weights for policy 0, policy_version 1503839 (0.0007) [2023-12-27 16:03:55,558][20347] Updated weights for policy 0, policy_version 1503849 (0.0008) [2023-12-27 16:03:55,615][20347] Updated weights for policy 0, policy_version 1503859 (0.0007) [2023-12-27 16:03:56,208][20381] Updated weights for policy 1, policy_version 1502230 (0.0007) [2023-12-27 16:03:56,256][20347] Updated weights for policy 0, policy_version 1503869 (0.0006) [2023-12-27 16:03:56,265][20381] Updated weights for policy 1, policy_version 1502240 (0.0007) [2023-12-27 16:03:56,302][20347] Updated weights for policy 0, policy_version 1503879 (0.0005) [2023-12-27 16:03:56,337][20381] Updated weights for policy 1, policy_version 1502250 (0.0007) [2023-12-27 16:03:56,349][20347] Updated weights for policy 0, policy_version 1503889 (0.0008) [2023-12-27 16:03:56,985][20381] Updated weights for policy 1, policy_version 1502260 (0.0006) [2023-12-27 16:03:57,028][20347] Updated weights for policy 0, policy_version 1503899 (0.0006) [2023-12-27 16:03:57,035][20381] Updated weights for policy 1, policy_version 1502270 (0.0007) [2023-12-27 16:03:57,074][20347] Updated weights for policy 0, policy_version 1503909 (0.0006) [2023-12-27 16:03:57,089][20381] Updated weights for policy 1, policy_version 1502280 (0.0007) [2023-12-27 16:03:57,135][20347] Updated weights for policy 0, policy_version 1503919 (0.0007) [2023-12-27 16:03:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 769703936. Throughput: 0: 10355.7, 1: 10209.0. Samples: 769690716. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:03:57,363][19170] Avg episode reward: [(0, '990.250'), (1, '951.300')] [2023-12-27 16:03:57,774][20381] Updated weights for policy 1, policy_version 1502290 (0.0006) [2023-12-27 16:03:57,825][20347] Updated weights for policy 0, policy_version 1503929 (0.0008) [2023-12-27 16:03:57,827][20381] Updated weights for policy 1, policy_version 1502300 (0.0005) [2023-12-27 16:03:57,880][20381] Updated weights for policy 1, policy_version 1502310 (0.0006) [2023-12-27 16:03:57,883][20347] Updated weights for policy 0, policy_version 1503939 (0.0007) [2023-12-27 16:03:57,936][20381] Updated weights for policy 1, policy_version 1502320 (0.0005) [2023-12-27 16:03:57,938][20347] Updated weights for policy 0, policy_version 1503949 (0.0007) [2023-12-27 16:03:57,993][20347] Updated weights for policy 0, policy_version 1503959 (0.0007) [2023-12-27 16:03:58,651][20381] Updated weights for policy 1, policy_version 1502330 (0.0008) [2023-12-27 16:03:58,710][20381] Updated weights for policy 1, policy_version 1502340 (0.0009) [2023-12-27 16:03:58,737][20347] Updated weights for policy 0, policy_version 1503969 (0.0007) [2023-12-27 16:03:58,778][20381] Updated weights for policy 1, policy_version 1502350 (0.0007) [2023-12-27 16:03:58,800][20347] Updated weights for policy 0, policy_version 1503979 (0.0008) [2023-12-27 16:03:58,862][20347] Updated weights for policy 0, policy_version 1503989 (0.0009) [2023-12-27 16:03:59,565][20381] Updated weights for policy 1, policy_version 1502360 (0.0008) [2023-12-27 16:03:59,617][20381] Updated weights for policy 1, policy_version 1502370 (0.0007) [2023-12-27 16:03:59,669][20381] Updated weights for policy 1, policy_version 1502380 (0.0007) [2023-12-27 16:03:59,686][20347] Updated weights for policy 0, policy_version 1503999 (0.0010) [2023-12-27 16:03:59,737][20347] Updated weights for policy 0, policy_version 1504009 (0.0010) [2023-12-27 16:03:59,792][20347] Updated weights for policy 0, policy_version 1504019 (0.0010) [2023-12-27 16:04:00,368][20381] Updated weights for policy 1, policy_version 1502390 (0.0007) [2023-12-27 16:04:00,421][20381] Updated weights for policy 1, policy_version 1502400 (0.0008) [2023-12-27 16:04:00,466][20347] Updated weights for policy 0, policy_version 1504029 (0.0008) [2023-12-27 16:04:00,471][20381] Updated weights for policy 1, policy_version 1502410 (0.0007) [2023-12-27 16:04:00,515][20347] Updated weights for policy 0, policy_version 1504039 (0.0006) [2023-12-27 16:04:00,567][20347] Updated weights for policy 0, policy_version 1504049 (0.0006) [2023-12-27 16:04:01,199][20381] Updated weights for policy 1, policy_version 1502420 (0.0008) [2023-12-27 16:04:01,204][20347] Updated weights for policy 0, policy_version 1504059 (0.0006) [2023-12-27 16:04:01,261][20347] Updated weights for policy 0, policy_version 1504069 (0.0006) [2023-12-27 16:04:01,262][20381] Updated weights for policy 1, policy_version 1502430 (0.0007) [2023-12-27 16:04:01,314][20381] Updated weights for policy 1, policy_version 1502440 (0.0007) [2023-12-27 16:04:01,317][20347] Updated weights for policy 0, policy_version 1504079 (0.0005) [2023-12-27 16:04:01,999][20347] Updated weights for policy 0, policy_version 1504089 (0.0008) [2023-12-27 16:04:02,048][20381] Updated weights for policy 1, policy_version 1502450 (0.0007) [2023-12-27 16:04:02,056][20347] Updated weights for policy 0, policy_version 1504099 (0.0008) [2023-12-27 16:04:02,106][20381] Updated weights for policy 1, policy_version 1502460 (0.0009) [2023-12-27 16:04:02,114][20347] Updated weights for policy 0, policy_version 1504109 (0.0009) [2023-12-27 16:04:02,163][20381] Updated weights for policy 1, policy_version 1502470 (0.0006) [2023-12-27 16:04:02,176][20347] Updated weights for policy 0, policy_version 1504119 (0.0005) [2023-12-27 16:04:02,219][20381] Updated weights for policy 1, policy_version 1502480 (0.0007) [2023-12-27 16:04:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 769802240. Throughput: 0: 10290.8, 1: 10085.0. Samples: 769808556. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:04:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '961.010')] [2023-12-27 16:04:02,858][20347] Updated weights for policy 0, policy_version 1504129 (0.0007) [2023-12-27 16:04:02,895][20381] Updated weights for policy 1, policy_version 1502490 (0.0007) [2023-12-27 16:04:02,911][20347] Updated weights for policy 0, policy_version 1504139 (0.0005) [2023-12-27 16:04:02,950][20381] Updated weights for policy 1, policy_version 1502500 (0.0005) [2023-12-27 16:04:02,966][20347] Updated weights for policy 0, policy_version 1504149 (0.0006) [2023-12-27 16:04:03,002][20381] Updated weights for policy 1, policy_version 1502510 (0.0006) [2023-12-27 16:04:03,560][20347] Updated weights for policy 0, policy_version 1504159 (0.0005) [2023-12-27 16:04:03,614][20347] Updated weights for policy 0, policy_version 1504169 (0.0005) [2023-12-27 16:04:03,672][20347] Updated weights for policy 0, policy_version 1504179 (0.0005) [2023-12-27 16:04:03,708][20381] Updated weights for policy 1, policy_version 1502520 (0.0008) [2023-12-27 16:04:03,768][20381] Updated weights for policy 1, policy_version 1502530 (0.0007) [2023-12-27 16:04:03,831][20381] Updated weights for policy 1, policy_version 1502540 (0.0006) [2023-12-27 16:04:04,280][20347] Updated weights for policy 0, policy_version 1504189 (0.0006) [2023-12-27 16:04:04,336][20347] Updated weights for policy 0, policy_version 1504199 (0.0007) [2023-12-27 16:04:04,388][20347] Updated weights for policy 0, policy_version 1504209 (0.0005) [2023-12-27 16:04:04,523][20381] Updated weights for policy 1, policy_version 1502550 (0.0010) [2023-12-27 16:04:04,580][20381] Updated weights for policy 1, policy_version 1502560 (0.0007) [2023-12-27 16:04:04,638][20381] Updated weights for policy 1, policy_version 1502570 (0.0011) [2023-12-27 16:04:05,018][20347] Updated weights for policy 0, policy_version 1504219 (0.0006) [2023-12-27 16:04:05,064][20347] Updated weights for policy 0, policy_version 1504229 (0.0005) [2023-12-27 16:04:05,110][20347] Updated weights for policy 0, policy_version 1504239 (0.0005) [2023-12-27 16:04:05,268][20381] Updated weights for policy 1, policy_version 1502580 (0.0011) [2023-12-27 16:04:05,321][20381] Updated weights for policy 1, policy_version 1502590 (0.0011) [2023-12-27 16:04:05,379][20381] Updated weights for policy 1, policy_version 1502600 (0.0011) [2023-12-27 16:04:05,762][20347] Updated weights for policy 0, policy_version 1504249 (0.0006) [2023-12-27 16:04:05,826][20347] Updated weights for policy 0, policy_version 1504259 (0.0006) [2023-12-27 16:04:05,878][20347] Updated weights for policy 0, policy_version 1504269 (0.0006) [2023-12-27 16:04:05,942][20347] Updated weights for policy 0, policy_version 1504279 (0.0008) [2023-12-27 16:04:06,067][20381] Updated weights for policy 1, policy_version 1502610 (0.0009) [2023-12-27 16:04:06,128][20381] Updated weights for policy 1, policy_version 1502620 (0.0006) [2023-12-27 16:04:06,191][20381] Updated weights for policy 1, policy_version 1502630 (0.0007) [2023-12-27 16:04:06,248][20381] Updated weights for policy 1, policy_version 1502640 (0.0009) [2023-12-27 16:04:06,588][20347] Updated weights for policy 0, policy_version 1504289 (0.0008) [2023-12-27 16:04:06,646][20347] Updated weights for policy 0, policy_version 1504299 (0.0007) [2023-12-27 16:04:06,708][20347] Updated weights for policy 0, policy_version 1504309 (0.0008) [2023-12-27 16:04:06,894][20381] Updated weights for policy 1, policy_version 1502650 (0.0006) [2023-12-27 16:04:06,959][20381] Updated weights for policy 1, policy_version 1502660 (0.0006) [2023-12-27 16:04:07,020][20381] Updated weights for policy 1, policy_version 1502670 (0.0006) [2023-12-27 16:04:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 769900544. Throughput: 0: 10289.0, 1: 10130.8. Samples: 769872132. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:04:07,364][19170] Avg episode reward: [(0, '990.250'), (1, '980.470')] [2023-12-27 16:04:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001502672_384720896.pth... [2023-12-27 16:04:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001501488_384417792.pth [2023-12-27 16:04:07,402][20347] Updated weights for policy 0, policy_version 1504319 (0.0006) [2023-12-27 16:04:07,453][20347] Updated weights for policy 0, policy_version 1504329 (0.0005) [2023-12-27 16:04:07,500][20347] Updated weights for policy 0, policy_version 1504339 (0.0006) [2023-12-27 16:04:07,524][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001504344_385187840.pth... [2023-12-27 16:04:07,527][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001503128_384876544.pth [2023-12-27 16:04:07,583][20381] Updated weights for policy 1, policy_version 1502680 (0.0007) [2023-12-27 16:04:07,639][20381] Updated weights for policy 1, policy_version 1502690 (0.0009) [2023-12-27 16:04:07,693][20381] Updated weights for policy 1, policy_version 1502700 (0.0010) [2023-12-27 16:04:08,204][20347] Updated weights for policy 0, policy_version 1504349 (0.0007) [2023-12-27 16:04:08,266][20347] Updated weights for policy 0, policy_version 1504359 (0.0007) [2023-12-27 16:04:08,293][20381] Updated weights for policy 1, policy_version 1502710 (0.0009) [2023-12-27 16:04:08,322][20347] Updated weights for policy 0, policy_version 1504369 (0.0007) [2023-12-27 16:04:08,349][20381] Updated weights for policy 1, policy_version 1502720 (0.0009) [2023-12-27 16:04:08,412][20381] Updated weights for policy 1, policy_version 1502730 (0.0011) [2023-12-27 16:04:09,047][20347] Updated weights for policy 0, policy_version 1504379 (0.0009) [2023-12-27 16:04:09,102][20347] Updated weights for policy 0, policy_version 1504389 (0.0010) [2023-12-27 16:04:09,130][20381] Updated weights for policy 1, policy_version 1502740 (0.0008) [2023-12-27 16:04:09,158][20347] Updated weights for policy 0, policy_version 1504399 (0.0010) [2023-12-27 16:04:09,184][20381] Updated weights for policy 1, policy_version 1502750 (0.0007) [2023-12-27 16:04:09,247][20381] Updated weights for policy 1, policy_version 1502760 (0.0008) [2023-12-27 16:04:09,814][20347] Updated weights for policy 0, policy_version 1504409 (0.0010) [2023-12-27 16:04:09,874][20347] Updated weights for policy 0, policy_version 1504419 (0.0008) [2023-12-27 16:04:09,939][20347] Updated weights for policy 0, policy_version 1504429 (0.0007) [2023-12-27 16:04:09,994][20381] Updated weights for policy 1, policy_version 1502770 (0.0007) [2023-12-27 16:04:09,995][20347] Updated weights for policy 0, policy_version 1504439 (0.0006) [2023-12-27 16:04:10,054][20381] Updated weights for policy 1, policy_version 1502780 (0.0008) [2023-12-27 16:04:10,115][20381] Updated weights for policy 1, policy_version 1502790 (0.0008) [2023-12-27 16:04:10,182][20381] Updated weights for policy 1, policy_version 1502800 (0.0008) [2023-12-27 16:04:10,651][20347] Updated weights for policy 0, policy_version 1504449 (0.0009) [2023-12-27 16:04:10,711][20347] Updated weights for policy 0, policy_version 1504459 (0.0008) [2023-12-27 16:04:10,769][20347] Updated weights for policy 0, policy_version 1504469 (0.0008) [2023-12-27 16:04:10,922][20381] Updated weights for policy 1, policy_version 1502810 (0.0007) [2023-12-27 16:04:10,986][20381] Updated weights for policy 1, policy_version 1502820 (0.0008) [2023-12-27 16:04:11,047][20381] Updated weights for policy 1, policy_version 1502830 (0.0008) [2023-12-27 16:04:11,522][20347] Updated weights for policy 0, policy_version 1504479 (0.0007) [2023-12-27 16:04:11,577][20347] Updated weights for policy 0, policy_version 1504489 (0.0008) [2023-12-27 16:04:11,639][20347] Updated weights for policy 0, policy_version 1504499 (0.0008) [2023-12-27 16:04:11,759][20381] Updated weights for policy 1, policy_version 1502840 (0.0008) [2023-12-27 16:04:11,814][20381] Updated weights for policy 1, policy_version 1502850 (0.0008) [2023-12-27 16:04:11,870][20381] Updated weights for policy 1, policy_version 1502860 (0.0007) [2023-12-27 16:04:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 769998848. Throughput: 0: 10254.9, 1: 10191.3. Samples: 769993616. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-12-27 16:04:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.710')] [2023-12-27 16:04:12,380][20347] Updated weights for policy 0, policy_version 1504509 (0.0009) [2023-12-27 16:04:12,441][20347] Updated weights for policy 0, policy_version 1504519 (0.0007) [2023-12-27 16:04:12,503][20347] Updated weights for policy 0, policy_version 1504529 (0.0008) [2023-12-27 16:04:12,592][20381] Updated weights for policy 1, policy_version 1502870 (0.0007) [2023-12-27 16:04:12,654][20381] Updated weights for policy 1, policy_version 1502880 (0.0008) [2023-12-27 16:04:12,714][20381] Updated weights for policy 1, policy_version 1502890 (0.0008) [2023-12-27 16:04:13,192][20347] Updated weights for policy 0, policy_version 1504539 (0.0008) [2023-12-27 16:04:13,244][20347] Updated weights for policy 0, policy_version 1504549 (0.0008) [2023-12-27 16:04:13,295][20347] Updated weights for policy 0, policy_version 1504559 (0.0007) [2023-12-27 16:04:13,314][20381] Updated weights for policy 1, policy_version 1502900 (0.0007) [2023-12-27 16:04:13,369][20381] Updated weights for policy 1, policy_version 1502910 (0.0005) [2023-12-27 16:04:13,428][20381] Updated weights for policy 1, policy_version 1502920 (0.0008) [2023-12-27 16:04:13,925][20347] Updated weights for policy 0, policy_version 1504569 (0.0008) [2023-12-27 16:04:13,977][20347] Updated weights for policy 0, policy_version 1504579 (0.0008) [2023-12-27 16:04:14,033][20347] Updated weights for policy 0, policy_version 1504589 (0.0007) [2023-12-27 16:04:14,084][20347] Updated weights for policy 0, policy_version 1504599 (0.0007) [2023-12-27 16:04:14,099][20381] Updated weights for policy 1, policy_version 1502930 (0.0006) [2023-12-27 16:04:14,154][20381] Updated weights for policy 1, policy_version 1502940 (0.0008) [2023-12-27 16:04:14,214][20381] Updated weights for policy 1, policy_version 1502950 (0.0008) [2023-12-27 16:04:14,273][20381] Updated weights for policy 1, policy_version 1502960 (0.0008) [2023-12-27 16:04:14,823][20347] Updated weights for policy 0, policy_version 1504609 (0.0008) [2023-12-27 16:04:14,850][20381] Updated weights for policy 1, policy_version 1502970 (0.0007) [2023-12-27 16:04:14,882][20347] Updated weights for policy 0, policy_version 1504619 (0.0008) [2023-12-27 16:04:14,913][20381] Updated weights for policy 1, policy_version 1502980 (0.0006) [2023-12-27 16:04:14,942][20347] Updated weights for policy 0, policy_version 1504629 (0.0008) [2023-12-27 16:04:14,969][20381] Updated weights for policy 1, policy_version 1502990 (0.0006) [2023-12-27 16:04:15,615][20381] Updated weights for policy 1, policy_version 1503000 (0.0008) [2023-12-27 16:04:15,650][20347] Updated weights for policy 0, policy_version 1504639 (0.0006) [2023-12-27 16:04:15,676][20381] Updated weights for policy 1, policy_version 1503010 (0.0008) [2023-12-27 16:04:15,711][20347] Updated weights for policy 0, policy_version 1504649 (0.0006) [2023-12-27 16:04:15,733][20381] Updated weights for policy 1, policy_version 1503020 (0.0008) [2023-12-27 16:04:15,767][20347] Updated weights for policy 0, policy_version 1504659 (0.0007) [2023-12-27 16:04:16,328][20381] Updated weights for policy 1, policy_version 1503030 (0.0006) [2023-12-27 16:04:16,389][20381] Updated weights for policy 1, policy_version 1503040 (0.0007) [2023-12-27 16:04:16,453][20381] Updated weights for policy 1, policy_version 1503050 (0.0007) [2023-12-27 16:04:16,470][20347] Updated weights for policy 0, policy_version 1504669 (0.0008) [2023-12-27 16:04:16,523][20347] Updated weights for policy 0, policy_version 1504679 (0.0007) [2023-12-27 16:04:16,574][20347] Updated weights for policy 0, policy_version 1504689 (0.0007) [2023-12-27 16:04:17,066][20381] Updated weights for policy 1, policy_version 1503060 (0.0006) [2023-12-27 16:04:17,131][20381] Updated weights for policy 1, policy_version 1503070 (0.0006) [2023-12-27 16:04:17,187][20381] Updated weights for policy 1, policy_version 1503080 (0.0006) [2023-12-27 16:04:17,227][20347] Updated weights for policy 0, policy_version 1504699 (0.0008) [2023-12-27 16:04:17,292][20347] Updated weights for policy 0, policy_version 1504709 (0.0005) [2023-12-27 16:04:17,340][20347] Updated weights for policy 0, policy_version 1504719 (0.0005) [2023-12-27 16:04:17,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 770105344. Throughput: 0: 10184.3, 1: 10203.1. Samples: 770116812. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.510')] [2023-12-27 16:04:17,814][20381] Updated weights for policy 1, policy_version 1503090 (0.0007) [2023-12-27 16:04:17,870][20381] Updated weights for policy 1, policy_version 1503100 (0.0005) [2023-12-27 16:04:17,927][20381] Updated weights for policy 1, policy_version 1503110 (0.0005) [2023-12-27 16:04:17,973][20381] Updated weights for policy 1, policy_version 1503120 (0.0007) [2023-12-27 16:04:17,998][20347] Updated weights for policy 0, policy_version 1504729 (0.0005) [2023-12-27 16:04:18,048][20347] Updated weights for policy 0, policy_version 1504739 (0.0007) [2023-12-27 16:04:18,103][20347] Updated weights for policy 0, policy_version 1504749 (0.0008) [2023-12-27 16:04:18,155][20347] Updated weights for policy 0, policy_version 1504759 (0.0008) [2023-12-27 16:04:18,621][20381] Updated weights for policy 1, policy_version 1503130 (0.0005) [2023-12-27 16:04:18,683][20381] Updated weights for policy 1, policy_version 1503140 (0.0006) [2023-12-27 16:04:18,749][20381] Updated weights for policy 1, policy_version 1503150 (0.0006) [2023-12-27 16:04:18,799][20347] Updated weights for policy 0, policy_version 1504769 (0.0007) [2023-12-27 16:04:18,849][20347] Updated weights for policy 0, policy_version 1504779 (0.0010) [2023-12-27 16:04:18,911][20347] Updated weights for policy 0, policy_version 1504789 (0.0007) [2023-12-27 16:04:19,382][20381] Updated weights for policy 1, policy_version 1503160 (0.0008) [2023-12-27 16:04:19,442][20381] Updated weights for policy 1, policy_version 1503170 (0.0008) [2023-12-27 16:04:19,502][20381] Updated weights for policy 1, policy_version 1503180 (0.0008) [2023-12-27 16:04:19,643][20347] Updated weights for policy 0, policy_version 1504799 (0.0008) [2023-12-27 16:04:19,706][20347] Updated weights for policy 0, policy_version 1504809 (0.0007) [2023-12-27 16:04:19,766][20347] Updated weights for policy 0, policy_version 1504819 (0.0008) [2023-12-27 16:04:20,222][20381] Updated weights for policy 1, policy_version 1503190 (0.0009) [2023-12-27 16:04:20,277][20381] Updated weights for policy 1, policy_version 1503200 (0.0009) [2023-12-27 16:04:20,336][20381] Updated weights for policy 1, policy_version 1503210 (0.0008) [2023-12-27 16:04:20,408][20347] Updated weights for policy 0, policy_version 1504829 (0.0007) [2023-12-27 16:04:20,469][20347] Updated weights for policy 0, policy_version 1504839 (0.0009) [2023-12-27 16:04:20,528][20347] Updated weights for policy 0, policy_version 1504849 (0.0008) [2023-12-27 16:04:21,009][20381] Updated weights for policy 1, policy_version 1503220 (0.0007) [2023-12-27 16:04:21,077][20381] Updated weights for policy 1, policy_version 1503230 (0.0008) [2023-12-27 16:04:21,146][20381] Updated weights for policy 1, policy_version 1503240 (0.0007) [2023-12-27 16:04:21,247][20347] Updated weights for policy 0, policy_version 1504859 (0.0007) [2023-12-27 16:04:21,315][20347] Updated weights for policy 0, policy_version 1504869 (0.0006) [2023-12-27 16:04:21,387][20347] Updated weights for policy 0, policy_version 1504879 (0.0008) [2023-12-27 16:04:21,809][20381] Updated weights for policy 1, policy_version 1503250 (0.0007) [2023-12-27 16:04:21,878][20381] Updated weights for policy 1, policy_version 1503260 (0.0005) [2023-12-27 16:04:21,934][20381] Updated weights for policy 1, policy_version 1503270 (0.0008) [2023-12-27 16:04:21,992][20381] Updated weights for policy 1, policy_version 1503280 (0.0006) [2023-12-27 16:04:22,148][20347] Updated weights for policy 0, policy_version 1504889 (0.0008) [2023-12-27 16:04:22,207][20347] Updated weights for policy 0, policy_version 1504899 (0.0007) [2023-12-27 16:04:22,270][20347] Updated weights for policy 0, policy_version 1504909 (0.0008) [2023-12-27 16:04:22,323][20347] Updated weights for policy 0, policy_version 1504919 (0.0007) [2023-12-27 16:04:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 770211840. Throughput: 0: 10177.0, 1: 10189.0. Samples: 770179188. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.490')] [2023-12-27 16:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001503280_384876544.pth... [2023-12-27 16:04:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001504920_385335296.pth... [2023-12-27 16:04:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001502096_384573440.pth [2023-12-27 16:04:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001503736_385032192.pth [2023-12-27 16:04:22,697][20381] Updated weights for policy 1, policy_version 1503290 (0.0008) [2023-12-27 16:04:22,762][20381] Updated weights for policy 1, policy_version 1503300 (0.0007) [2023-12-27 16:04:22,823][20381] Updated weights for policy 1, policy_version 1503310 (0.0007) [2023-12-27 16:04:23,038][20347] Updated weights for policy 0, policy_version 1504929 (0.0008) [2023-12-27 16:04:23,090][20347] Updated weights for policy 0, policy_version 1504939 (0.0008) [2023-12-27 16:04:23,140][20347] Updated weights for policy 0, policy_version 1504949 (0.0007) [2023-12-27 16:04:23,432][20381] Updated weights for policy 1, policy_version 1503320 (0.0007) [2023-12-27 16:04:23,479][20381] Updated weights for policy 1, policy_version 1503330 (0.0008) [2023-12-27 16:04:23,522][20381] Updated weights for policy 1, policy_version 1503340 (0.0008) [2023-12-27 16:04:23,846][20347] Updated weights for policy 0, policy_version 1504959 (0.0007) [2023-12-27 16:04:23,900][20347] Updated weights for policy 0, policy_version 1504969 (0.0008) [2023-12-27 16:04:23,952][20347] Updated weights for policy 0, policy_version 1504979 (0.0008) [2023-12-27 16:04:24,241][20381] Updated weights for policy 1, policy_version 1503350 (0.0008) [2023-12-27 16:04:24,290][20381] Updated weights for policy 1, policy_version 1503360 (0.0008) [2023-12-27 16:04:24,336][20381] Updated weights for policy 1, policy_version 1503370 (0.0008) [2023-12-27 16:04:24,605][20347] Updated weights for policy 0, policy_version 1504989 (0.0006) [2023-12-27 16:04:24,660][20347] Updated weights for policy 0, policy_version 1504999 (0.0008) [2023-12-27 16:04:24,704][20347] Updated weights for policy 0, policy_version 1505009 (0.0009) [2023-12-27 16:04:24,958][20381] Updated weights for policy 1, policy_version 1503380 (0.0008) [2023-12-27 16:04:25,004][20381] Updated weights for policy 1, policy_version 1503390 (0.0010) [2023-12-27 16:04:25,053][20381] Updated weights for policy 1, policy_version 1503400 (0.0009) [2023-12-27 16:04:25,417][20347] Updated weights for policy 0, policy_version 1505019 (0.0009) [2023-12-27 16:04:25,480][20347] Updated weights for policy 0, policy_version 1505029 (0.0010) [2023-12-27 16:04:25,541][20347] Updated weights for policy 0, policy_version 1505039 (0.0010) [2023-12-27 16:04:25,708][20381] Updated weights for policy 1, policy_version 1503410 (0.0006) [2023-12-27 16:04:25,763][20381] Updated weights for policy 1, policy_version 1503420 (0.0010) [2023-12-27 16:04:25,823][20381] Updated weights for policy 1, policy_version 1503430 (0.0009) [2023-12-27 16:04:25,883][20381] Updated weights for policy 1, policy_version 1503440 (0.0011) [2023-12-27 16:04:26,163][20347] Updated weights for policy 0, policy_version 1505049 (0.0008) [2023-12-27 16:04:26,222][20347] Updated weights for policy 0, policy_version 1505059 (0.0010) [2023-12-27 16:04:26,277][20347] Updated weights for policy 0, policy_version 1505069 (0.0010) [2023-12-27 16:04:26,335][20347] Updated weights for policy 0, policy_version 1505079 (0.0011) [2023-12-27 16:04:26,580][20381] Updated weights for policy 1, policy_version 1503450 (0.0009) [2023-12-27 16:04:26,624][20381] Updated weights for policy 1, policy_version 1503460 (0.0009) [2023-12-27 16:04:26,672][20381] Updated weights for policy 1, policy_version 1503470 (0.0008) [2023-12-27 16:04:26,946][20347] Updated weights for policy 0, policy_version 1505089 (0.0007) [2023-12-27 16:04:27,002][20347] Updated weights for policy 0, policy_version 1505099 (0.0005) [2023-12-27 16:04:27,055][20347] Updated weights for policy 0, policy_version 1505109 (0.0005) [2023-12-27 16:04:27,300][20381] Updated weights for policy 1, policy_version 1503480 (0.0008) [2023-12-27 16:04:27,361][20381] Updated weights for policy 1, policy_version 1503490 (0.0008) [2023-12-27 16:04:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 770310144. Throughput: 0: 10147.6, 1: 10253.9. Samples: 770301328. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.470')] [2023-12-27 16:04:27,418][20381] Updated weights for policy 1, policy_version 1503500 (0.0009) [2023-12-27 16:04:27,614][20347] Updated weights for policy 0, policy_version 1505119 (0.0007) [2023-12-27 16:04:27,674][20347] Updated weights for policy 0, policy_version 1505129 (0.0010) [2023-12-27 16:04:27,729][20347] Updated weights for policy 0, policy_version 1505139 (0.0010) [2023-12-27 16:04:28,071][20381] Updated weights for policy 1, policy_version 1503510 (0.0007) [2023-12-27 16:04:28,130][20381] Updated weights for policy 1, policy_version 1503520 (0.0006) [2023-12-27 16:04:28,176][20381] Updated weights for policy 1, policy_version 1503530 (0.0009) [2023-12-27 16:04:28,330][20347] Updated weights for policy 0, policy_version 1505149 (0.0008) [2023-12-27 16:04:28,390][20347] Updated weights for policy 0, policy_version 1505159 (0.0008) [2023-12-27 16:04:28,446][20347] Updated weights for policy 0, policy_version 1505169 (0.0007) [2023-12-27 16:04:28,885][20381] Updated weights for policy 1, policy_version 1503540 (0.0007) [2023-12-27 16:04:28,940][20381] Updated weights for policy 1, policy_version 1503550 (0.0007) [2023-12-27 16:04:28,991][20381] Updated weights for policy 1, policy_version 1503560 (0.0007) [2023-12-27 16:04:29,080][20347] Updated weights for policy 0, policy_version 1505179 (0.0005) [2023-12-27 16:04:29,126][20347] Updated weights for policy 0, policy_version 1505189 (0.0005) [2023-12-27 16:04:29,174][20347] Updated weights for policy 0, policy_version 1505199 (0.0005) [2023-12-27 16:04:29,767][20347] Updated weights for policy 0, policy_version 1505209 (0.0007) [2023-12-27 16:04:29,804][20381] Updated weights for policy 1, policy_version 1503570 (0.0008) [2023-12-27 16:04:29,827][20347] Updated weights for policy 0, policy_version 1505219 (0.0008) [2023-12-27 16:04:29,868][20381] Updated weights for policy 1, policy_version 1503580 (0.0007) [2023-12-27 16:04:29,886][20347] Updated weights for policy 0, policy_version 1505229 (0.0007) [2023-12-27 16:04:29,926][20381] Updated weights for policy 1, policy_version 1503590 (0.0006) [2023-12-27 16:04:29,946][20347] Updated weights for policy 0, policy_version 1505239 (0.0007) [2023-12-27 16:04:29,989][20381] Updated weights for policy 1, policy_version 1503600 (0.0008) [2023-12-27 16:04:30,565][20347] Updated weights for policy 0, policy_version 1505249 (0.0007) [2023-12-27 16:04:30,623][20347] Updated weights for policy 0, policy_version 1505259 (0.0005) [2023-12-27 16:04:30,683][20347] Updated weights for policy 0, policy_version 1505269 (0.0005) [2023-12-27 16:04:30,699][20381] Updated weights for policy 1, policy_version 1503610 (0.0007) [2023-12-27 16:04:30,744][20381] Updated weights for policy 1, policy_version 1503620 (0.0008) [2023-12-27 16:04:30,803][20381] Updated weights for policy 1, policy_version 1503630 (0.0007) [2023-12-27 16:04:31,309][20347] Updated weights for policy 0, policy_version 1505279 (0.0006) [2023-12-27 16:04:31,375][20347] Updated weights for policy 0, policy_version 1505289 (0.0008) [2023-12-27 16:04:31,437][20347] Updated weights for policy 0, policy_version 1505299 (0.0008) [2023-12-27 16:04:31,572][20381] Updated weights for policy 1, policy_version 1503640 (0.0006) [2023-12-27 16:04:31,635][20381] Updated weights for policy 1, policy_version 1503650 (0.0006) [2023-12-27 16:04:31,696][20381] Updated weights for policy 1, policy_version 1503660 (0.0006) [2023-12-27 16:04:32,123][20347] Updated weights for policy 0, policy_version 1505309 (0.0008) [2023-12-27 16:04:32,175][20347] Updated weights for policy 0, policy_version 1505319 (0.0007) [2023-12-27 16:04:32,221][20347] Updated weights for policy 0, policy_version 1505329 (0.0007) [2023-12-27 16:04:32,342][20381] Updated weights for policy 1, policy_version 1503670 (0.0006) [2023-12-27 16:04:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 770416640. Throughput: 0: 10258.7, 1: 10175.4. Samples: 770427100. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.010')] [2023-12-27 16:04:32,407][20381] Updated weights for policy 1, policy_version 1503680 (0.0008) [2023-12-27 16:04:32,471][20381] Updated weights for policy 1, policy_version 1503690 (0.0008) [2023-12-27 16:04:32,907][20347] Updated weights for policy 0, policy_version 1505339 (0.0007) [2023-12-27 16:04:32,956][20347] Updated weights for policy 0, policy_version 1505349 (0.0009) [2023-12-27 16:04:33,015][20347] Updated weights for policy 0, policy_version 1505359 (0.0008) [2023-12-27 16:04:33,120][20381] Updated weights for policy 1, policy_version 1503700 (0.0007) [2023-12-27 16:04:33,171][20381] Updated weights for policy 1, policy_version 1503710 (0.0007) [2023-12-27 16:04:33,220][20381] Updated weights for policy 1, policy_version 1503720 (0.0008) [2023-12-27 16:04:33,715][20347] Updated weights for policy 0, policy_version 1505369 (0.0007) [2023-12-27 16:04:33,765][20347] Updated weights for policy 0, policy_version 1505379 (0.0005) [2023-12-27 16:04:33,813][20347] Updated weights for policy 0, policy_version 1505389 (0.0006) [2023-12-27 16:04:33,860][20347] Updated weights for policy 0, policy_version 1505399 (0.0010) [2023-12-27 16:04:33,875][20381] Updated weights for policy 1, policy_version 1503730 (0.0007) [2023-12-27 16:04:33,926][20381] Updated weights for policy 1, policy_version 1503740 (0.0006) [2023-12-27 16:04:33,978][20381] Updated weights for policy 1, policy_version 1503750 (0.0006) [2023-12-27 16:04:34,025][20381] Updated weights for policy 1, policy_version 1503760 (0.0008) [2023-12-27 16:04:34,525][20347] Updated weights for policy 0, policy_version 1505409 (0.0010) [2023-12-27 16:04:34,584][20347] Updated weights for policy 0, policy_version 1505419 (0.0010) [2023-12-27 16:04:34,639][20347] Updated weights for policy 0, policy_version 1505429 (0.0007) [2023-12-27 16:04:34,728][20381] Updated weights for policy 1, policy_version 1503770 (0.0008) [2023-12-27 16:04:34,773][20381] Updated weights for policy 1, policy_version 1503780 (0.0007) [2023-12-27 16:04:34,828][20381] Updated weights for policy 1, policy_version 1503790 (0.0008) [2023-12-27 16:04:35,305][20347] Updated weights for policy 0, policy_version 1505439 (0.0006) [2023-12-27 16:04:35,366][20347] Updated weights for policy 0, policy_version 1505449 (0.0007) [2023-12-27 16:04:35,419][20347] Updated weights for policy 0, policy_version 1505459 (0.0009) [2023-12-27 16:04:35,510][20381] Updated weights for policy 1, policy_version 1503800 (0.0009) [2023-12-27 16:04:35,556][20381] Updated weights for policy 1, policy_version 1503810 (0.0008) [2023-12-27 16:04:35,604][20381] Updated weights for policy 1, policy_version 1503820 (0.0008) [2023-12-27 16:04:35,976][20347] Updated weights for policy 0, policy_version 1505469 (0.0010) [2023-12-27 16:04:36,028][20347] Updated weights for policy 0, policy_version 1505479 (0.0010) [2023-12-27 16:04:36,082][20347] Updated weights for policy 0, policy_version 1505489 (0.0010) [2023-12-27 16:04:36,302][20381] Updated weights for policy 1, policy_version 1503830 (0.0007) [2023-12-27 16:04:36,354][20381] Updated weights for policy 1, policy_version 1503840 (0.0005) [2023-12-27 16:04:36,407][20381] Updated weights for policy 1, policy_version 1503850 (0.0008) [2023-12-27 16:04:36,720][20347] Updated weights for policy 0, policy_version 1505499 (0.0008) [2023-12-27 16:04:36,781][20347] Updated weights for policy 0, policy_version 1505509 (0.0006) [2023-12-27 16:04:36,835][20347] Updated weights for policy 0, policy_version 1505519 (0.0005) [2023-12-27 16:04:37,080][20381] Updated weights for policy 1, policy_version 1503860 (0.0008) [2023-12-27 16:04:37,130][20381] Updated weights for policy 1, policy_version 1503870 (0.0006) [2023-12-27 16:04:37,177][20381] Updated weights for policy 1, policy_version 1503880 (0.0009) [2023-12-27 16:04:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 770523136. Throughput: 0: 10256.9, 1: 10168.8. Samples: 770488132. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '931.760')] [2023-12-27 16:04:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001505528_385490944.pth... [2023-12-27 16:04:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001503888_385032192.pth... [2023-12-27 16:04:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001504344_385187840.pth [2023-12-27 16:04:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001502672_384720896.pth [2023-12-27 16:04:37,443][20347] Updated weights for policy 0, policy_version 1505529 (0.0008) [2023-12-27 16:04:37,510][20347] Updated weights for policy 0, policy_version 1505539 (0.0010) [2023-12-27 16:04:37,575][20347] Updated weights for policy 0, policy_version 1505549 (0.0010) [2023-12-27 16:04:37,633][20347] Updated weights for policy 0, policy_version 1505559 (0.0010) [2023-12-27 16:04:37,771][20381] Updated weights for policy 1, policy_version 1503890 (0.0010) [2023-12-27 16:04:37,831][20381] Updated weights for policy 1, policy_version 1503900 (0.0006) [2023-12-27 16:04:37,890][20381] Updated weights for policy 1, policy_version 1503910 (0.0005) [2023-12-27 16:04:37,935][20381] Updated weights for policy 1, policy_version 1503920 (0.0005) [2023-12-27 16:04:38,287][20347] Updated weights for policy 0, policy_version 1505569 (0.0010) [2023-12-27 16:04:38,347][20347] Updated weights for policy 0, policy_version 1505579 (0.0009) [2023-12-27 16:04:38,404][20347] Updated weights for policy 0, policy_version 1505589 (0.0006) [2023-12-27 16:04:38,532][20381] Updated weights for policy 1, policy_version 1503930 (0.0007) [2023-12-27 16:04:38,590][20381] Updated weights for policy 1, policy_version 1503940 (0.0007) [2023-12-27 16:04:38,648][20381] Updated weights for policy 1, policy_version 1503950 (0.0007) [2023-12-27 16:04:39,066][20347] Updated weights for policy 0, policy_version 1505599 (0.0006) [2023-12-27 16:04:39,117][20347] Updated weights for policy 0, policy_version 1505609 (0.0005) [2023-12-27 16:04:39,165][20347] Updated weights for policy 0, policy_version 1505619 (0.0005) [2023-12-27 16:04:39,359][20381] Updated weights for policy 1, policy_version 1503960 (0.0009) [2023-12-27 16:04:39,424][20381] Updated weights for policy 1, policy_version 1503970 (0.0008) [2023-12-27 16:04:39,490][20381] Updated weights for policy 1, policy_version 1503980 (0.0007) [2023-12-27 16:04:39,902][20347] Updated weights for policy 0, policy_version 1505629 (0.0007) [2023-12-27 16:04:39,968][20347] Updated weights for policy 0, policy_version 1505639 (0.0007) [2023-12-27 16:04:40,026][20347] Updated weights for policy 0, policy_version 1505649 (0.0008) [2023-12-27 16:04:40,217][20381] Updated weights for policy 1, policy_version 1503990 (0.0007) [2023-12-27 16:04:40,276][20381] Updated weights for policy 1, policy_version 1504000 (0.0008) [2023-12-27 16:04:40,336][20381] Updated weights for policy 1, policy_version 1504010 (0.0008) [2023-12-27 16:04:40,732][20347] Updated weights for policy 0, policy_version 1505659 (0.0008) [2023-12-27 16:04:40,776][20347] Updated weights for policy 0, policy_version 1505669 (0.0008) [2023-12-27 16:04:40,823][20347] Updated weights for policy 0, policy_version 1505679 (0.0007) [2023-12-27 16:04:41,004][20381] Updated weights for policy 1, policy_version 1504020 (0.0008) [2023-12-27 16:04:41,071][20381] Updated weights for policy 1, policy_version 1504030 (0.0006) [2023-12-27 16:04:41,138][20381] Updated weights for policy 1, policy_version 1504040 (0.0007) [2023-12-27 16:04:41,574][20347] Updated weights for policy 0, policy_version 1505689 (0.0005) [2023-12-27 16:04:41,628][20347] Updated weights for policy 0, policy_version 1505699 (0.0008) [2023-12-27 16:04:41,688][20347] Updated weights for policy 0, policy_version 1505709 (0.0007) [2023-12-27 16:04:41,748][20347] Updated weights for policy 0, policy_version 1505719 (0.0008) [2023-12-27 16:04:41,905][20381] Updated weights for policy 1, policy_version 1504050 (0.0009) [2023-12-27 16:04:41,953][20381] Updated weights for policy 1, policy_version 1504060 (0.0008) [2023-12-27 16:04:42,000][20381] Updated weights for policy 1, policy_version 1504070 (0.0008) [2023-12-27 16:04:42,048][20381] Updated weights for policy 1, policy_version 1504080 (0.0008) [2023-12-27 16:04:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 770621440. Throughput: 0: 10299.2, 1: 10217.4. Samples: 770613964. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.280')] [2023-12-27 16:04:42,430][20347] Updated weights for policy 0, policy_version 1505729 (0.0008) [2023-12-27 16:04:42,479][20347] Updated weights for policy 0, policy_version 1505739 (0.0008) [2023-12-27 16:04:42,535][20347] Updated weights for policy 0, policy_version 1505749 (0.0008) [2023-12-27 16:04:42,816][20381] Updated weights for policy 1, policy_version 1504090 (0.0007) [2023-12-27 16:04:42,877][20381] Updated weights for policy 1, policy_version 1504100 (0.0008) [2023-12-27 16:04:42,931][20381] Updated weights for policy 1, policy_version 1504110 (0.0008) [2023-12-27 16:04:43,234][20347] Updated weights for policy 0, policy_version 1505759 (0.0007) [2023-12-27 16:04:43,296][20347] Updated weights for policy 0, policy_version 1505769 (0.0005) [2023-12-27 16:04:43,354][20347] Updated weights for policy 0, policy_version 1505779 (0.0005) [2023-12-27 16:04:43,623][20381] Updated weights for policy 1, policy_version 1504120 (0.0007) [2023-12-27 16:04:43,671][20381] Updated weights for policy 1, policy_version 1504130 (0.0007) [2023-12-27 16:04:43,715][20381] Updated weights for policy 1, policy_version 1504140 (0.0007) [2023-12-27 16:04:43,982][20347] Updated weights for policy 0, policy_version 1505789 (0.0008) [2023-12-27 16:04:44,037][20347] Updated weights for policy 0, policy_version 1505799 (0.0010) [2023-12-27 16:04:44,085][20347] Updated weights for policy 0, policy_version 1505809 (0.0010) [2023-12-27 16:04:44,363][20381] Updated weights for policy 1, policy_version 1504150 (0.0008) [2023-12-27 16:04:44,421][20381] Updated weights for policy 1, policy_version 1504160 (0.0007) [2023-12-27 16:04:44,472][20381] Updated weights for policy 1, policy_version 1504170 (0.0007) [2023-12-27 16:04:44,767][20347] Updated weights for policy 0, policy_version 1505819 (0.0009) [2023-12-27 16:04:44,830][20347] Updated weights for policy 0, policy_version 1505829 (0.0007) [2023-12-27 16:04:44,892][20347] Updated weights for policy 0, policy_version 1505839 (0.0008) [2023-12-27 16:04:45,185][20381] Updated weights for policy 1, policy_version 1504180 (0.0007) [2023-12-27 16:04:45,237][20381] Updated weights for policy 1, policy_version 1504190 (0.0007) [2023-12-27 16:04:45,291][20381] Updated weights for policy 1, policy_version 1504200 (0.0007) [2023-12-27 16:04:45,580][20347] Updated weights for policy 0, policy_version 1505849 (0.0010) [2023-12-27 16:04:45,628][20347] Updated weights for policy 0, policy_version 1505859 (0.0010) [2023-12-27 16:04:45,674][20347] Updated weights for policy 0, policy_version 1505869 (0.0010) [2023-12-27 16:04:45,720][20347] Updated weights for policy 0, policy_version 1505879 (0.0010) [2023-12-27 16:04:45,894][20381] Updated weights for policy 1, policy_version 1504210 (0.0006) [2023-12-27 16:04:45,950][20381] Updated weights for policy 1, policy_version 1504220 (0.0005) [2023-12-27 16:04:46,005][20381] Updated weights for policy 1, policy_version 1504230 (0.0005) [2023-12-27 16:04:46,057][20381] Updated weights for policy 1, policy_version 1504240 (0.0005) [2023-12-27 16:04:46,432][20347] Updated weights for policy 0, policy_version 1505889 (0.0006) [2023-12-27 16:04:46,491][20347] Updated weights for policy 0, policy_version 1505899 (0.0008) [2023-12-27 16:04:46,547][20347] Updated weights for policy 0, policy_version 1505909 (0.0007) [2023-12-27 16:04:46,643][20381] Updated weights for policy 1, policy_version 1504250 (0.0008) [2023-12-27 16:04:46,703][20381] Updated weights for policy 1, policy_version 1504260 (0.0008) [2023-12-27 16:04:46,747][20381] Updated weights for policy 1, policy_version 1504270 (0.0008) [2023-12-27 16:04:47,069][20347] Updated weights for policy 0, policy_version 1505919 (0.0006) [2023-12-27 16:04:47,121][20347] Updated weights for policy 0, policy_version 1505929 (0.0005) [2023-12-27 16:04:47,188][20347] Updated weights for policy 0, policy_version 1505939 (0.0008) [2023-12-27 16:04:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 770727936. Throughput: 0: 10353.9, 1: 10289.3. Samples: 770737500. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.920')] [2023-12-27 16:04:47,498][20381] Updated weights for policy 1, policy_version 1504280 (0.0008) [2023-12-27 16:04:47,557][20381] Updated weights for policy 1, policy_version 1504290 (0.0007) [2023-12-27 16:04:47,615][20381] Updated weights for policy 1, policy_version 1504300 (0.0008) [2023-12-27 16:04:47,828][20347] Updated weights for policy 0, policy_version 1505949 (0.0010) [2023-12-27 16:04:47,873][20347] Updated weights for policy 0, policy_version 1505959 (0.0010) [2023-12-27 16:04:47,918][20347] Updated weights for policy 0, policy_version 1505969 (0.0010) [2023-12-27 16:04:48,282][20381] Updated weights for policy 1, policy_version 1504310 (0.0006) [2023-12-27 16:04:48,346][20381] Updated weights for policy 1, policy_version 1504320 (0.0007) [2023-12-27 16:04:48,407][20381] Updated weights for policy 1, policy_version 1504330 (0.0011) [2023-12-27 16:04:48,643][20347] Updated weights for policy 0, policy_version 1505979 (0.0010) [2023-12-27 16:04:48,701][20347] Updated weights for policy 0, policy_version 1505989 (0.0010) [2023-12-27 16:04:48,753][20347] Updated weights for policy 0, policy_version 1505999 (0.0010) [2023-12-27 16:04:49,067][20381] Updated weights for policy 1, policy_version 1504340 (0.0009) [2023-12-27 16:04:49,116][20381] Updated weights for policy 1, policy_version 1504350 (0.0005) [2023-12-27 16:04:49,172][20381] Updated weights for policy 1, policy_version 1504360 (0.0005) [2023-12-27 16:04:49,466][20347] Updated weights for policy 0, policy_version 1506009 (0.0010) [2023-12-27 16:04:49,524][20347] Updated weights for policy 0, policy_version 1506019 (0.0009) [2023-12-27 16:04:49,581][20347] Updated weights for policy 0, policy_version 1506029 (0.0008) [2023-12-27 16:04:49,635][20347] Updated weights for policy 0, policy_version 1506039 (0.0009) [2023-12-27 16:04:49,787][20381] Updated weights for policy 1, policy_version 1504370 (0.0006) [2023-12-27 16:04:49,850][20381] Updated weights for policy 1, policy_version 1504380 (0.0007) [2023-12-27 16:04:49,911][20381] Updated weights for policy 1, policy_version 1504390 (0.0007) [2023-12-27 16:04:49,973][20381] Updated weights for policy 1, policy_version 1504400 (0.0007) [2023-12-27 16:04:50,352][20347] Updated weights for policy 0, policy_version 1506049 (0.0007) [2023-12-27 16:04:50,420][20347] Updated weights for policy 0, policy_version 1506059 (0.0008) [2023-12-27 16:04:50,479][20347] Updated weights for policy 0, policy_version 1506069 (0.0008) [2023-12-27 16:04:50,628][20381] Updated weights for policy 1, policy_version 1504410 (0.0007) [2023-12-27 16:04:50,680][20381] Updated weights for policy 1, policy_version 1504420 (0.0006) [2023-12-27 16:04:50,733][20381] Updated weights for policy 1, policy_version 1504430 (0.0008) [2023-12-27 16:04:51,253][20347] Updated weights for policy 0, policy_version 1506079 (0.0007) [2023-12-27 16:04:51,311][20347] Updated weights for policy 0, policy_version 1506089 (0.0005) [2023-12-27 16:04:51,378][20347] Updated weights for policy 0, policy_version 1506099 (0.0007) [2023-12-27 16:04:51,460][20381] Updated weights for policy 1, policy_version 1504440 (0.0007) [2023-12-27 16:04:51,519][20381] Updated weights for policy 1, policy_version 1504450 (0.0006) [2023-12-27 16:04:51,579][20381] Updated weights for policy 1, policy_version 1504460 (0.0006) [2023-12-27 16:04:52,029][20347] Updated weights for policy 0, policy_version 1506109 (0.0007) [2023-12-27 16:04:52,094][20347] Updated weights for policy 0, policy_version 1506119 (0.0007) [2023-12-27 16:04:52,150][20347] Updated weights for policy 0, policy_version 1506129 (0.0008) [2023-12-27 16:04:52,264][20381] Updated weights for policy 1, policy_version 1504470 (0.0008) [2023-12-27 16:04:52,325][20381] Updated weights for policy 1, policy_version 1504480 (0.0009) [2023-12-27 16:04:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 770826240. Throughput: 0: 10295.4, 1: 10293.9. Samples: 770798640. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '971.160')] [2023-12-27 16:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001506136_385646592.pth... [2023-12-27 16:04:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001504920_385335296.pth [2023-12-27 16:04:52,389][20381] Updated weights for policy 1, policy_version 1504490 (0.0007) [2023-12-27 16:04:52,424][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001504496_385187840.pth... [2023-12-27 16:04:52,428][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001503280_384876544.pth [2023-12-27 16:04:52,874][20347] Updated weights for policy 0, policy_version 1506139 (0.0007) [2023-12-27 16:04:52,928][20347] Updated weights for policy 0, policy_version 1506149 (0.0008) [2023-12-27 16:04:52,987][20347] Updated weights for policy 0, policy_version 1506159 (0.0008) [2023-12-27 16:04:53,079][20381] Updated weights for policy 1, policy_version 1504500 (0.0007) [2023-12-27 16:04:53,137][20381] Updated weights for policy 1, policy_version 1504510 (0.0008) [2023-12-27 16:04:53,192][20381] Updated weights for policy 1, policy_version 1504520 (0.0008) [2023-12-27 16:04:53,682][20347] Updated weights for policy 0, policy_version 1506169 (0.0008) [2023-12-27 16:04:53,736][20347] Updated weights for policy 0, policy_version 1506179 (0.0007) [2023-12-27 16:04:53,789][20347] Updated weights for policy 0, policy_version 1506189 (0.0007) [2023-12-27 16:04:53,839][20381] Updated weights for policy 1, policy_version 1504530 (0.0008) [2023-12-27 16:04:53,852][20347] Updated weights for policy 0, policy_version 1506199 (0.0007) [2023-12-27 16:04:53,900][20381] Updated weights for policy 1, policy_version 1504540 (0.0006) [2023-12-27 16:04:53,955][20381] Updated weights for policy 1, policy_version 1504550 (0.0005) [2023-12-27 16:04:54,012][20381] Updated weights for policy 1, policy_version 1504560 (0.0009) [2023-12-27 16:04:54,552][20381] Updated weights for policy 1, policy_version 1504570 (0.0010) [2023-12-27 16:04:54,590][20347] Updated weights for policy 0, policy_version 1506209 (0.0006) [2023-12-27 16:04:54,605][20381] Updated weights for policy 1, policy_version 1504580 (0.0010) [2023-12-27 16:04:54,636][20347] Updated weights for policy 0, policy_version 1506219 (0.0007) [2023-12-27 16:04:54,659][20381] Updated weights for policy 1, policy_version 1504590 (0.0011) [2023-12-27 16:04:54,683][20347] Updated weights for policy 0, policy_version 1506229 (0.0007) [2023-12-27 16:04:55,350][20381] Updated weights for policy 1, policy_version 1504600 (0.0010) [2023-12-27 16:04:55,398][20347] Updated weights for policy 0, policy_version 1506239 (0.0006) [2023-12-27 16:04:55,407][20381] Updated weights for policy 1, policy_version 1504610 (0.0010) [2023-12-27 16:04:55,448][20347] Updated weights for policy 0, policy_version 1506249 (0.0006) [2023-12-27 16:04:55,464][20381] Updated weights for policy 1, policy_version 1504620 (0.0010) [2023-12-27 16:04:55,494][20347] Updated weights for policy 0, policy_version 1506259 (0.0006) [2023-12-27 16:04:56,080][20381] Updated weights for policy 1, policy_version 1504630 (0.0009) [2023-12-27 16:04:56,136][20381] Updated weights for policy 1, policy_version 1504640 (0.0008) [2023-12-27 16:04:56,193][20381] Updated weights for policy 1, policy_version 1504650 (0.0008) [2023-12-27 16:04:56,222][20347] Updated weights for policy 0, policy_version 1506269 (0.0007) [2023-12-27 16:04:56,272][20347] Updated weights for policy 0, policy_version 1506279 (0.0007) [2023-12-27 16:04:56,316][20347] Updated weights for policy 0, policy_version 1506289 (0.0008) [2023-12-27 16:04:56,816][20381] Updated weights for policy 1, policy_version 1504660 (0.0006) [2023-12-27 16:04:56,864][20381] Updated weights for policy 1, policy_version 1504670 (0.0005) [2023-12-27 16:04:56,919][20381] Updated weights for policy 1, policy_version 1504680 (0.0007) [2023-12-27 16:04:57,076][20347] Updated weights for policy 0, policy_version 1506299 (0.0007) [2023-12-27 16:04:57,135][20347] Updated weights for policy 0, policy_version 1506309 (0.0007) [2023-12-27 16:04:57,187][20347] Updated weights for policy 0, policy_version 1506319 (0.0007) [2023-12-27 16:04:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 770932736. Throughput: 0: 10252.2, 1: 10335.1. Samples: 770920044. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:04:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.280')] [2023-12-27 16:04:57,546][20381] Updated weights for policy 1, policy_version 1504690 (0.0007) [2023-12-27 16:04:57,611][20381] Updated weights for policy 1, policy_version 1504700 (0.0005) [2023-12-27 16:04:57,664][20381] Updated weights for policy 1, policy_version 1504710 (0.0006) [2023-12-27 16:04:57,725][20381] Updated weights for policy 1, policy_version 1504720 (0.0011) [2023-12-27 16:04:57,872][20347] Updated weights for policy 0, policy_version 1506329 (0.0007) [2023-12-27 16:04:57,935][20347] Updated weights for policy 0, policy_version 1506339 (0.0010) [2023-12-27 16:04:57,995][20347] Updated weights for policy 0, policy_version 1506349 (0.0010) [2023-12-27 16:04:58,055][20347] Updated weights for policy 0, policy_version 1506359 (0.0010) [2023-12-27 16:04:58,306][20381] Updated weights for policy 1, policy_version 1504730 (0.0008) [2023-12-27 16:04:58,376][20381] Updated weights for policy 1, policy_version 1504740 (0.0015) [2023-12-27 16:04:58,439][20381] Updated weights for policy 1, policy_version 1504750 (0.0008) [2023-12-27 16:04:58,799][20347] Updated weights for policy 0, policy_version 1506369 (0.0009) [2023-12-27 16:04:58,864][20347] Updated weights for policy 0, policy_version 1506379 (0.0009) [2023-12-27 16:04:58,937][20347] Updated weights for policy 0, policy_version 1506389 (0.0008) [2023-12-27 16:04:59,317][20381] Updated weights for policy 1, policy_version 1504760 (0.0008) [2023-12-27 16:04:59,385][20381] Updated weights for policy 1, policy_version 1504770 (0.0009) [2023-12-27 16:04:59,447][20381] Updated weights for policy 1, policy_version 1504780 (0.0009) [2023-12-27 16:04:59,741][20347] Updated weights for policy 0, policy_version 1506399 (0.0007) [2023-12-27 16:04:59,799][20347] Updated weights for policy 0, policy_version 1506409 (0.0008) [2023-12-27 16:04:59,884][20347] Updated weights for policy 0, policy_version 1506419 (0.0008) [2023-12-27 16:05:00,108][20381] Updated weights for policy 1, policy_version 1504790 (0.0008) [2023-12-27 16:05:00,171][20381] Updated weights for policy 1, policy_version 1504800 (0.0006) [2023-12-27 16:05:00,234][20381] Updated weights for policy 1, policy_version 1504810 (0.0007) [2023-12-27 16:05:00,578][20347] Updated weights for policy 0, policy_version 1506429 (0.0009) [2023-12-27 16:05:00,631][20347] Updated weights for policy 0, policy_version 1506439 (0.0007) [2023-12-27 16:05:00,679][20347] Updated weights for policy 0, policy_version 1506449 (0.0007) [2023-12-27 16:05:00,808][20381] Updated weights for policy 1, policy_version 1504820 (0.0009) [2023-12-27 16:05:00,861][20381] Updated weights for policy 1, policy_version 1504830 (0.0009) [2023-12-27 16:05:00,906][20381] Updated weights for policy 1, policy_version 1504840 (0.0008) [2023-12-27 16:05:01,376][20347] Updated weights for policy 0, policy_version 1506459 (0.0007) [2023-12-27 16:05:01,431][20347] Updated weights for policy 0, policy_version 1506469 (0.0008) [2023-12-27 16:05:01,490][20347] Updated weights for policy 0, policy_version 1506479 (0.0008) [2023-12-27 16:05:01,581][20381] Updated weights for policy 1, policy_version 1504850 (0.0009) [2023-12-27 16:05:01,645][20381] Updated weights for policy 1, policy_version 1504860 (0.0007) [2023-12-27 16:05:01,707][20381] Updated weights for policy 1, policy_version 1504870 (0.0008) [2023-12-27 16:05:01,764][20381] Updated weights for policy 1, policy_version 1504880 (0.0008) [2023-12-27 16:05:02,205][20347] Updated weights for policy 0, policy_version 1506489 (0.0008) [2023-12-27 16:05:02,259][20347] Updated weights for policy 0, policy_version 1506499 (0.0007) [2023-12-27 16:05:02,317][20347] Updated weights for policy 0, policy_version 1506509 (0.0007) [2023-12-27 16:05:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 771022848. Throughput: 0: 10206.0, 1: 10294.0. Samples: 771039312. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.250')] [2023-12-27 16:05:02,379][20347] Updated weights for policy 0, policy_version 1506519 (0.0007) [2023-12-27 16:05:02,400][20381] Updated weights for policy 1, policy_version 1504890 (0.0008) [2023-12-27 16:05:02,457][20381] Updated weights for policy 1, policy_version 1504900 (0.0007) [2023-12-27 16:05:02,515][20381] Updated weights for policy 1, policy_version 1504910 (0.0008) [2023-12-27 16:05:02,993][20347] Updated weights for policy 0, policy_version 1506529 (0.0008) [2023-12-27 16:05:03,048][20347] Updated weights for policy 0, policy_version 1506539 (0.0007) [2023-12-27 16:05:03,108][20347] Updated weights for policy 0, policy_version 1506549 (0.0007) [2023-12-27 16:05:03,120][20381] Updated weights for policy 1, policy_version 1504920 (0.0006) [2023-12-27 16:05:03,166][20381] Updated weights for policy 1, policy_version 1504930 (0.0005) [2023-12-27 16:05:03,212][20381] Updated weights for policy 1, policy_version 1504940 (0.0006) [2023-12-27 16:05:03,817][20347] Updated weights for policy 0, policy_version 1506559 (0.0008) [2023-12-27 16:05:03,877][20347] Updated weights for policy 0, policy_version 1506569 (0.0008) [2023-12-27 16:05:03,882][20381] Updated weights for policy 1, policy_version 1504950 (0.0007) [2023-12-27 16:05:03,934][20347] Updated weights for policy 0, policy_version 1506579 (0.0006) [2023-12-27 16:05:03,943][20381] Updated weights for policy 1, policy_version 1504960 (0.0006) [2023-12-27 16:05:03,997][20381] Updated weights for policy 1, policy_version 1504970 (0.0005) [2023-12-27 16:05:04,596][20381] Updated weights for policy 1, policy_version 1504980 (0.0005) [2023-12-27 16:05:04,649][20381] Updated weights for policy 1, policy_version 1504990 (0.0005) [2023-12-27 16:05:04,679][20347] Updated weights for policy 0, policy_version 1506589 (0.0008) [2023-12-27 16:05:04,702][20381] Updated weights for policy 1, policy_version 1505000 (0.0005) [2023-12-27 16:05:04,745][20347] Updated weights for policy 0, policy_version 1506599 (0.0008) [2023-12-27 16:05:04,805][20347] Updated weights for policy 0, policy_version 1506609 (0.0009) [2023-12-27 16:05:05,279][20381] Updated weights for policy 1, policy_version 1505010 (0.0006) [2023-12-27 16:05:05,330][20381] Updated weights for policy 1, policy_version 1505020 (0.0010) [2023-12-27 16:05:05,385][20381] Updated weights for policy 1, policy_version 1505030 (0.0007) [2023-12-27 16:05:05,436][20381] Updated weights for policy 1, policy_version 1505040 (0.0008) [2023-12-27 16:05:05,440][20347] Updated weights for policy 0, policy_version 1506619 (0.0009) [2023-12-27 16:05:05,497][20347] Updated weights for policy 0, policy_version 1506629 (0.0007) [2023-12-27 16:05:05,547][20347] Updated weights for policy 0, policy_version 1506639 (0.0009) [2023-12-27 16:05:06,040][20381] Updated weights for policy 1, policy_version 1505050 (0.0007) [2023-12-27 16:05:06,100][20381] Updated weights for policy 1, policy_version 1505060 (0.0006) [2023-12-27 16:05:06,168][20381] Updated weights for policy 1, policy_version 1505070 (0.0007) [2023-12-27 16:05:06,188][20347] Updated weights for policy 0, policy_version 1506649 (0.0008) [2023-12-27 16:05:06,245][20347] Updated weights for policy 0, policy_version 1506659 (0.0007) [2023-12-27 16:05:06,305][20347] Updated weights for policy 0, policy_version 1506669 (0.0007) [2023-12-27 16:05:06,360][20347] Updated weights for policy 0, policy_version 1506679 (0.0007) [2023-12-27 16:05:06,825][20381] Updated weights for policy 1, policy_version 1505080 (0.0010) [2023-12-27 16:05:06,873][20381] Updated weights for policy 1, policy_version 1505090 (0.0010) [2023-12-27 16:05:06,931][20381] Updated weights for policy 1, policy_version 1505100 (0.0007) [2023-12-27 16:05:07,089][20347] Updated weights for policy 0, policy_version 1506689 (0.0008) [2023-12-27 16:05:07,148][20347] Updated weights for policy 0, policy_version 1506699 (0.0007) [2023-12-27 16:05:07,201][20347] Updated weights for policy 0, policy_version 1506709 (0.0008) [2023-12-27 16:05:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 771137536. Throughput: 0: 10189.5, 1: 10345.2. Samples: 771103252. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '980.530')] [2023-12-27 16:05:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001505104_385343488.pth... [2023-12-27 16:05:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001506712_385794048.pth... [2023-12-27 16:05:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001503888_385032192.pth [2023-12-27 16:05:07,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001505528_385490944.pth [2023-12-27 16:05:07,619][20381] Updated weights for policy 1, policy_version 1505110 (0.0010) [2023-12-27 16:05:07,665][20381] Updated weights for policy 1, policy_version 1505120 (0.0011) [2023-12-27 16:05:07,714][20381] Updated weights for policy 1, policy_version 1505130 (0.0010) [2023-12-27 16:05:07,880][20347] Updated weights for policy 0, policy_version 1506719 (0.0007) [2023-12-27 16:05:07,935][20347] Updated weights for policy 0, policy_version 1506729 (0.0007) [2023-12-27 16:05:07,982][20347] Updated weights for policy 0, policy_version 1506739 (0.0007) [2023-12-27 16:05:08,459][20381] Updated weights for policy 1, policy_version 1505140 (0.0010) [2023-12-27 16:05:08,523][20381] Updated weights for policy 1, policy_version 1505150 (0.0008) [2023-12-27 16:05:08,582][20381] Updated weights for policy 1, policy_version 1505160 (0.0011) [2023-12-27 16:05:08,688][20347] Updated weights for policy 0, policy_version 1506749 (0.0008) [2023-12-27 16:05:08,742][20347] Updated weights for policy 0, policy_version 1506759 (0.0007) [2023-12-27 16:05:08,802][20347] Updated weights for policy 0, policy_version 1506769 (0.0007) [2023-12-27 16:05:09,233][20381] Updated weights for policy 1, policy_version 1505170 (0.0010) [2023-12-27 16:05:09,293][20381] Updated weights for policy 1, policy_version 1505180 (0.0008) [2023-12-27 16:05:09,356][20381] Updated weights for policy 1, policy_version 1505190 (0.0008) [2023-12-27 16:05:09,424][20381] Updated weights for policy 1, policy_version 1505200 (0.0007) [2023-12-27 16:05:09,549][20347] Updated weights for policy 0, policy_version 1506779 (0.0008) [2023-12-27 16:05:09,609][20347] Updated weights for policy 0, policy_version 1506789 (0.0007) [2023-12-27 16:05:09,666][20347] Updated weights for policy 0, policy_version 1506799 (0.0008) [2023-12-27 16:05:10,136][20381] Updated weights for policy 1, policy_version 1505210 (0.0007) [2023-12-27 16:05:10,201][20381] Updated weights for policy 1, policy_version 1505220 (0.0007) [2023-12-27 16:05:10,261][20381] Updated weights for policy 1, policy_version 1505230 (0.0007) [2023-12-27 16:05:10,404][20347] Updated weights for policy 0, policy_version 1506809 (0.0008) [2023-12-27 16:05:10,465][20347] Updated weights for policy 0, policy_version 1506819 (0.0007) [2023-12-27 16:05:10,520][20347] Updated weights for policy 0, policy_version 1506829 (0.0006) [2023-12-27 16:05:10,570][20347] Updated weights for policy 0, policy_version 1506839 (0.0006) [2023-12-27 16:05:10,957][20381] Updated weights for policy 1, policy_version 1505240 (0.0005) [2023-12-27 16:05:11,021][20381] Updated weights for policy 1, policy_version 1505250 (0.0005) [2023-12-27 16:05:11,088][20381] Updated weights for policy 1, policy_version 1505260 (0.0007) [2023-12-27 16:05:11,226][20347] Updated weights for policy 0, policy_version 1506849 (0.0007) [2023-12-27 16:05:11,295][20347] Updated weights for policy 0, policy_version 1506859 (0.0008) [2023-12-27 16:05:11,358][20347] Updated weights for policy 0, policy_version 1506869 (0.0008) [2023-12-27 16:05:11,788][20381] Updated weights for policy 1, policy_version 1505270 (0.0008) [2023-12-27 16:05:11,853][20381] Updated weights for policy 1, policy_version 1505280 (0.0008) [2023-12-27 16:05:11,919][20381] Updated weights for policy 1, policy_version 1505290 (0.0007) [2023-12-27 16:05:12,078][20347] Updated weights for policy 0, policy_version 1506879 (0.0007) [2023-12-27 16:05:12,127][20347] Updated weights for policy 0, policy_version 1506889 (0.0009) [2023-12-27 16:05:12,186][20347] Updated weights for policy 0, policy_version 1506899 (0.0008) [2023-12-27 16:05:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 771235840. Throughput: 0: 10189.8, 1: 10336.6. Samples: 771225016. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.280')] [2023-12-27 16:05:12,559][20381] Updated weights for policy 1, policy_version 1505300 (0.0008) [2023-12-27 16:05:12,626][20381] Updated weights for policy 1, policy_version 1505310 (0.0008) [2023-12-27 16:05:12,683][20381] Updated weights for policy 1, policy_version 1505320 (0.0007) [2023-12-27 16:05:12,834][20347] Updated weights for policy 0, policy_version 1506909 (0.0008) [2023-12-27 16:05:12,883][20347] Updated weights for policy 0, policy_version 1506919 (0.0007) [2023-12-27 16:05:12,932][20347] Updated weights for policy 0, policy_version 1506929 (0.0007) [2023-12-27 16:05:13,277][20381] Updated weights for policy 1, policy_version 1505330 (0.0008) [2023-12-27 16:05:13,333][20381] Updated weights for policy 1, policy_version 1505340 (0.0007) [2023-12-27 16:05:13,384][20381] Updated weights for policy 1, policy_version 1505350 (0.0006) [2023-12-27 16:05:13,434][20381] Updated weights for policy 1, policy_version 1505360 (0.0005) [2023-12-27 16:05:13,681][20347] Updated weights for policy 0, policy_version 1506939 (0.0009) [2023-12-27 16:05:13,742][20347] Updated weights for policy 0, policy_version 1506949 (0.0006) [2023-12-27 16:05:13,803][20347] Updated weights for policy 0, policy_version 1506959 (0.0007) [2023-12-27 16:05:14,066][20381] Updated weights for policy 1, policy_version 1505370 (0.0008) [2023-12-27 16:05:14,127][20381] Updated weights for policy 1, policy_version 1505380 (0.0008) [2023-12-27 16:05:14,180][20381] Updated weights for policy 1, policy_version 1505390 (0.0008) [2023-12-27 16:05:14,477][20347] Updated weights for policy 0, policy_version 1506969 (0.0007) [2023-12-27 16:05:14,536][20347] Updated weights for policy 0, policy_version 1506979 (0.0008) [2023-12-27 16:05:14,563][20291] KL-divergence is very high: 177.1299 [2023-12-27 16:05:14,592][20347] Updated weights for policy 0, policy_version 1506989 (0.0007) [2023-12-27 16:05:14,608][20291] KL-divergence is very high: 100.6263 [2023-12-27 16:05:14,644][20347] Updated weights for policy 0, policy_version 1506999 (0.0007) [2023-12-27 16:05:14,938][20381] Updated weights for policy 1, policy_version 1505400 (0.0008) [2023-12-27 16:05:15,008][20381] Updated weights for policy 1, policy_version 1505410 (0.0007) [2023-12-27 16:05:15,078][20381] Updated weights for policy 1, policy_version 1505420 (0.0009) [2023-12-27 16:05:15,321][20347] Updated weights for policy 0, policy_version 1507009 (0.0007) [2023-12-27 16:05:15,382][20347] Updated weights for policy 0, policy_version 1507019 (0.0007) [2023-12-27 16:05:15,441][20347] Updated weights for policy 0, policy_version 1507029 (0.0007) [2023-12-27 16:05:15,727][20381] Updated weights for policy 1, policy_version 1505430 (0.0007) [2023-12-27 16:05:15,777][20381] Updated weights for policy 1, policy_version 1505440 (0.0006) [2023-12-27 16:05:15,826][20381] Updated weights for policy 1, policy_version 1505450 (0.0005) [2023-12-27 16:05:16,155][20347] Updated weights for policy 0, policy_version 1507039 (0.0008) [2023-12-27 16:05:16,214][20347] Updated weights for policy 0, policy_version 1507049 (0.0006) [2023-12-27 16:05:16,262][20347] Updated weights for policy 0, policy_version 1507059 (0.0005) [2023-12-27 16:05:16,429][20381] Updated weights for policy 1, policy_version 1505460 (0.0006) [2023-12-27 16:05:16,478][20381] Updated weights for policy 1, policy_version 1505470 (0.0007) [2023-12-27 16:05:16,523][20381] Updated weights for policy 1, policy_version 1505480 (0.0009) [2023-12-27 16:05:16,876][20347] Updated weights for policy 0, policy_version 1507069 (0.0006) [2023-12-27 16:05:16,930][20347] Updated weights for policy 0, policy_version 1507079 (0.0006) [2023-12-27 16:05:16,985][20347] Updated weights for policy 0, policy_version 1507089 (0.0006) [2023-12-27 16:05:17,183][20381] Updated weights for policy 1, policy_version 1505490 (0.0008) [2023-12-27 16:05:17,230][20381] Updated weights for policy 1, policy_version 1505500 (0.0008) [2023-12-27 16:05:17,275][20381] Updated weights for policy 1, policy_version 1505510 (0.0007) [2023-12-27 16:05:17,323][20381] Updated weights for policy 1, policy_version 1505520 (0.0010) [2023-12-27 16:05:17,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 771342336. Throughput: 0: 10076.4, 1: 10409.6. Samples: 771348968. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:17,363][19170] Avg episode reward: [(0, '980.490'), (1, '1000.000')] [2023-12-27 16:05:17,512][20347] Updated weights for policy 0, policy_version 1507099 (0.0005) [2023-12-27 16:05:17,570][20347] Updated weights for policy 0, policy_version 1507109 (0.0005) [2023-12-27 16:05:17,631][20347] Updated weights for policy 0, policy_version 1507119 (0.0007) [2023-12-27 16:05:18,055][20381] Updated weights for policy 1, policy_version 1505530 (0.0007) [2023-12-27 16:05:18,108][20381] Updated weights for policy 1, policy_version 1505540 (0.0007) [2023-12-27 16:05:18,156][20381] Updated weights for policy 1, policy_version 1505550 (0.0006) [2023-12-27 16:05:18,268][20347] Updated weights for policy 0, policy_version 1507129 (0.0008) [2023-12-27 16:05:18,329][20347] Updated weights for policy 0, policy_version 1507139 (0.0007) [2023-12-27 16:05:18,396][20347] Updated weights for policy 0, policy_version 1507149 (0.0007) [2023-12-27 16:05:18,463][20347] Updated weights for policy 0, policy_version 1507159 (0.0007) [2023-12-27 16:05:18,890][20381] Updated weights for policy 1, policy_version 1505560 (0.0007) [2023-12-27 16:05:18,963][20381] Updated weights for policy 1, policy_version 1505570 (0.0007) [2023-12-27 16:05:19,020][20381] Updated weights for policy 1, policy_version 1505580 (0.0007) [2023-12-27 16:05:19,095][20347] Updated weights for policy 0, policy_version 1507169 (0.0007) [2023-12-27 16:05:19,150][20347] Updated weights for policy 0, policy_version 1507179 (0.0008) [2023-12-27 16:05:19,202][20347] Updated weights for policy 0, policy_version 1507189 (0.0008) [2023-12-27 16:05:19,714][20381] Updated weights for policy 1, policy_version 1505590 (0.0007) [2023-12-27 16:05:19,770][20381] Updated weights for policy 1, policy_version 1505600 (0.0007) [2023-12-27 16:05:19,821][20381] Updated weights for policy 1, policy_version 1505610 (0.0008) [2023-12-27 16:05:19,958][20347] Updated weights for policy 0, policy_version 1507199 (0.0008) [2023-12-27 16:05:20,016][20347] Updated weights for policy 0, policy_version 1507209 (0.0006) [2023-12-27 16:05:20,066][20347] Updated weights for policy 0, policy_version 1507219 (0.0006) [2023-12-27 16:05:20,551][20381] Updated weights for policy 1, policy_version 1505620 (0.0008) [2023-12-27 16:05:20,615][20381] Updated weights for policy 1, policy_version 1505630 (0.0007) [2023-12-27 16:05:20,676][20381] Updated weights for policy 1, policy_version 1505640 (0.0008) [2023-12-27 16:05:20,787][20347] Updated weights for policy 0, policy_version 1507229 (0.0007) [2023-12-27 16:05:20,843][20347] Updated weights for policy 0, policy_version 1507239 (0.0008) [2023-12-27 16:05:20,908][20347] Updated weights for policy 0, policy_version 1507249 (0.0007) [2023-12-27 16:05:21,360][20381] Updated weights for policy 1, policy_version 1505650 (0.0006) [2023-12-27 16:05:21,428][20381] Updated weights for policy 1, policy_version 1505660 (0.0008) [2023-12-27 16:05:21,497][20381] Updated weights for policy 1, policy_version 1505670 (0.0006) [2023-12-27 16:05:21,562][20381] Updated weights for policy 1, policy_version 1505680 (0.0005) [2023-12-27 16:05:21,612][20347] Updated weights for policy 0, policy_version 1507259 (0.0008) [2023-12-27 16:05:21,675][20347] Updated weights for policy 0, policy_version 1507269 (0.0008) [2023-12-27 16:05:21,743][20347] Updated weights for policy 0, policy_version 1507279 (0.0008) [2023-12-27 16:05:22,173][20381] Updated weights for policy 1, policy_version 1505690 (0.0006) [2023-12-27 16:05:22,236][20381] Updated weights for policy 1, policy_version 1505700 (0.0006) [2023-12-27 16:05:22,291][20381] Updated weights for policy 1, policy_version 1505710 (0.0008) [2023-12-27 16:05:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 771440640. Throughput: 0: 10102.4, 1: 10371.2. Samples: 771409444. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:22,363][19170] Avg episode reward: [(0, '961.100'), (1, '1000.000')] [2023-12-27 16:05:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001507288_385941504.pth... [2023-12-27 16:05:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001505712_385499136.pth... [2023-12-27 16:05:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001504496_385187840.pth [2023-12-27 16:05:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001506136_385646592.pth [2023-12-27 16:05:22,490][20347] Updated weights for policy 0, policy_version 1507289 (0.0007) [2023-12-27 16:05:22,543][20347] Updated weights for policy 0, policy_version 1507299 (0.0008) [2023-12-27 16:05:22,603][20347] Updated weights for policy 0, policy_version 1507309 (0.0007) [2023-12-27 16:05:22,662][20347] Updated weights for policy 0, policy_version 1507319 (0.0008) [2023-12-27 16:05:22,991][20381] Updated weights for policy 1, policy_version 1505720 (0.0005) [2023-12-27 16:05:23,061][20381] Updated weights for policy 1, policy_version 1505730 (0.0006) [2023-12-27 16:05:23,114][20381] Updated weights for policy 1, policy_version 1505740 (0.0008) [2023-12-27 16:05:23,385][20347] Updated weights for policy 0, policy_version 1507329 (0.0006) [2023-12-27 16:05:23,452][20347] Updated weights for policy 0, policy_version 1507339 (0.0006) [2023-12-27 16:05:23,514][20347] Updated weights for policy 0, policy_version 1507349 (0.0008) [2023-12-27 16:05:23,761][20381] Updated weights for policy 1, policy_version 1505750 (0.0009) [2023-12-27 16:05:23,812][20381] Updated weights for policy 1, policy_version 1505760 (0.0008) [2023-12-27 16:05:23,864][20381] Updated weights for policy 1, policy_version 1505770 (0.0008) [2023-12-27 16:05:24,093][20347] Updated weights for policy 0, policy_version 1507359 (0.0006) [2023-12-27 16:05:24,155][20347] Updated weights for policy 0, policy_version 1507369 (0.0007) [2023-12-27 16:05:24,216][20347] Updated weights for policy 0, policy_version 1507379 (0.0006) [2023-12-27 16:05:24,552][20381] Updated weights for policy 1, policy_version 1505780 (0.0009) [2023-12-27 16:05:24,612][20381] Updated weights for policy 1, policy_version 1505790 (0.0008) [2023-12-27 16:05:24,663][20381] Updated weights for policy 1, policy_version 1505800 (0.0008) [2023-12-27 16:05:24,810][20347] Updated weights for policy 0, policy_version 1507389 (0.0007) [2023-12-27 16:05:24,874][20347] Updated weights for policy 0, policy_version 1507399 (0.0005) [2023-12-27 16:05:24,922][20347] Updated weights for policy 0, policy_version 1507409 (0.0005) [2023-12-27 16:05:25,347][20381] Updated weights for policy 1, policy_version 1505810 (0.0009) [2023-12-27 16:05:25,392][20381] Updated weights for policy 1, policy_version 1505820 (0.0010) [2023-12-27 16:05:25,441][20381] Updated weights for policy 1, policy_version 1505830 (0.0010) [2023-12-27 16:05:25,486][20381] Updated weights for policy 1, policy_version 1505840 (0.0010) [2023-12-27 16:05:25,569][20347] Updated weights for policy 0, policy_version 1507419 (0.0006) [2023-12-27 16:05:25,627][20347] Updated weights for policy 0, policy_version 1507429 (0.0007) [2023-12-27 16:05:25,686][20347] Updated weights for policy 0, policy_version 1507439 (0.0008) [2023-12-27 16:05:26,113][20381] Updated weights for policy 1, policy_version 1505850 (0.0007) [2023-12-27 16:05:26,158][20381] Updated weights for policy 1, policy_version 1505860 (0.0009) [2023-12-27 16:05:26,207][20381] Updated weights for policy 1, policy_version 1505870 (0.0008) [2023-12-27 16:05:26,318][20347] Updated weights for policy 0, policy_version 1507449 (0.0007) [2023-12-27 16:05:26,380][20347] Updated weights for policy 0, policy_version 1507459 (0.0006) [2023-12-27 16:05:26,431][20347] Updated weights for policy 0, policy_version 1507469 (0.0009) [2023-12-27 16:05:26,489][20347] Updated weights for policy 0, policy_version 1507479 (0.0007) [2023-12-27 16:05:26,916][20381] Updated weights for policy 1, policy_version 1505880 (0.0008) [2023-12-27 16:05:26,968][20381] Updated weights for policy 1, policy_version 1505890 (0.0008) [2023-12-27 16:05:27,027][20381] Updated weights for policy 1, policy_version 1505900 (0.0008) [2023-12-27 16:05:27,175][20347] Updated weights for policy 0, policy_version 1507489 (0.0008) [2023-12-27 16:05:27,231][20347] Updated weights for policy 0, policy_version 1507499 (0.0007) [2023-12-27 16:05:27,279][20347] Updated weights for policy 0, policy_version 1507509 (0.0005) [2023-12-27 16:05:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 771547136. Throughput: 0: 10038.2, 1: 10355.8. Samples: 771531688. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:27,363][19170] Avg episode reward: [(0, '951.340'), (1, '1000.000')] [2023-12-27 16:05:27,665][20381] Updated weights for policy 1, policy_version 1505910 (0.0009) [2023-12-27 16:05:27,725][20381] Updated weights for policy 1, policy_version 1505920 (0.0009) [2023-12-27 16:05:27,786][20381] Updated weights for policy 1, policy_version 1505930 (0.0011) [2023-12-27 16:05:27,961][20347] Updated weights for policy 0, policy_version 1507519 (0.0008) [2023-12-27 16:05:28,018][20347] Updated weights for policy 0, policy_version 1507530 (0.0008) [2023-12-27 16:05:28,069][20347] Updated weights for policy 0, policy_version 1507540 (0.0009) [2023-12-27 16:05:28,366][20381] Updated weights for policy 1, policy_version 1505940 (0.0008) [2023-12-27 16:05:28,429][20381] Updated weights for policy 1, policy_version 1505950 (0.0007) [2023-12-27 16:05:28,485][20381] Updated weights for policy 1, policy_version 1505960 (0.0005) [2023-12-27 16:05:28,751][20347] Updated weights for policy 0, policy_version 1507550 (0.0009) [2023-12-27 16:05:28,798][20347] Updated weights for policy 0, policy_version 1507560 (0.0007) [2023-12-27 16:05:28,851][20347] Updated weights for policy 0, policy_version 1507570 (0.0007) [2023-12-27 16:05:29,058][20381] Updated weights for policy 1, policy_version 1505970 (0.0006) [2023-12-27 16:05:29,122][20381] Updated weights for policy 1, policy_version 1505980 (0.0008) [2023-12-27 16:05:29,169][20381] Updated weights for policy 1, policy_version 1505990 (0.0008) [2023-12-27 16:05:29,223][20381] Updated weights for policy 1, policy_version 1506000 (0.0007) [2023-12-27 16:05:29,612][20347] Updated weights for policy 0, policy_version 1507580 (0.0007) [2023-12-27 16:05:29,664][20347] Updated weights for policy 0, policy_version 1507590 (0.0007) [2023-12-27 16:05:29,716][20347] Updated weights for policy 0, policy_version 1507600 (0.0007) [2023-12-27 16:05:29,932][20381] Updated weights for policy 1, policy_version 1506010 (0.0006) [2023-12-27 16:05:29,990][20381] Updated weights for policy 1, policy_version 1506020 (0.0008) [2023-12-27 16:05:30,050][20381] Updated weights for policy 1, policy_version 1506030 (0.0008) [2023-12-27 16:05:30,529][20347] Updated weights for policy 0, policy_version 1507610 (0.0008) [2023-12-27 16:05:30,588][20347] Updated weights for policy 0, policy_version 1507620 (0.0008) [2023-12-27 16:05:30,647][20347] Updated weights for policy 0, policy_version 1507630 (0.0006) [2023-12-27 16:05:30,696][20347] Updated weights for policy 0, policy_version 1507640 (0.0007) [2023-12-27 16:05:30,749][20381] Updated weights for policy 1, policy_version 1506040 (0.0007) [2023-12-27 16:05:30,794][20381] Updated weights for policy 1, policy_version 1506050 (0.0008) [2023-12-27 16:05:30,847][20381] Updated weights for policy 1, policy_version 1506060 (0.0007) [2023-12-27 16:05:31,400][20347] Updated weights for policy 0, policy_version 1507650 (0.0007) [2023-12-27 16:05:31,452][20347] Updated weights for policy 0, policy_version 1507660 (0.0008) [2023-12-27 16:05:31,512][20347] Updated weights for policy 0, policy_version 1507670 (0.0008) [2023-12-27 16:05:31,570][20381] Updated weights for policy 1, policy_version 1506070 (0.0007) [2023-12-27 16:05:31,626][20381] Updated weights for policy 1, policy_version 1506080 (0.0008) [2023-12-27 16:05:31,690][20381] Updated weights for policy 1, policy_version 1506090 (0.0006) [2023-12-27 16:05:32,174][20347] Updated weights for policy 0, policy_version 1507680 (0.0006) [2023-12-27 16:05:32,238][20347] Updated weights for policy 0, policy_version 1507690 (0.0008) [2023-12-27 16:05:32,308][20347] Updated weights for policy 0, policy_version 1507700 (0.0008) [2023-12-27 16:05:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 771645440. Throughput: 0: 9979.8, 1: 10403.7. Samples: 771654756. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:32,363][19170] Avg episode reward: [(0, '970.730'), (1, '1000.000')] [2023-12-27 16:05:32,364][20381] Updated weights for policy 1, policy_version 1506100 (0.0009) [2023-12-27 16:05:32,426][20381] Updated weights for policy 1, policy_version 1506110 (0.0008) [2023-12-27 16:05:32,476][20381] Updated weights for policy 1, policy_version 1506120 (0.0007) [2023-12-27 16:05:32,906][20347] Updated weights for policy 0, policy_version 1507710 (0.0008) [2023-12-27 16:05:32,963][20347] Updated weights for policy 0, policy_version 1507720 (0.0010) [2023-12-27 16:05:33,013][20347] Updated weights for policy 0, policy_version 1507730 (0.0007) [2023-12-27 16:05:33,093][20381] Updated weights for policy 1, policy_version 1506130 (0.0008) [2023-12-27 16:05:33,142][20381] Updated weights for policy 1, policy_version 1506140 (0.0007) [2023-12-27 16:05:33,200][20381] Updated weights for policy 1, policy_version 1506150 (0.0005) [2023-12-27 16:05:33,250][20381] Updated weights for policy 1, policy_version 1506160 (0.0007) [2023-12-27 16:05:33,595][20347] Updated weights for policy 0, policy_version 1507740 (0.0005) [2023-12-27 16:05:33,643][20347] Updated weights for policy 0, policy_version 1507750 (0.0005) [2023-12-27 16:05:33,692][20347] Updated weights for policy 0, policy_version 1507760 (0.0005) [2023-12-27 16:05:33,911][20381] Updated weights for policy 1, policy_version 1506170 (0.0008) [2023-12-27 16:05:33,966][20381] Updated weights for policy 1, policy_version 1506180 (0.0008) [2023-12-27 16:05:34,022][20381] Updated weights for policy 1, policy_version 1506190 (0.0007) [2023-12-27 16:05:34,304][20347] Updated weights for policy 0, policy_version 1507770 (0.0007) [2023-12-27 16:05:34,367][20347] Updated weights for policy 0, policy_version 1507780 (0.0008) [2023-12-27 16:05:34,425][20347] Updated weights for policy 0, policy_version 1507790 (0.0008) [2023-12-27 16:05:34,480][20347] Updated weights for policy 0, policy_version 1507800 (0.0008) [2023-12-27 16:05:34,756][20381] Updated weights for policy 1, policy_version 1506200 (0.0008) [2023-12-27 16:05:34,819][20381] Updated weights for policy 1, policy_version 1506210 (0.0008) [2023-12-27 16:05:34,874][20381] Updated weights for policy 1, policy_version 1506220 (0.0008) [2023-12-27 16:05:35,100][20347] Updated weights for policy 0, policy_version 1507810 (0.0007) [2023-12-27 16:05:35,144][20347] Updated weights for policy 0, policy_version 1507820 (0.0008) [2023-12-27 16:05:35,195][20347] Updated weights for policy 0, policy_version 1507830 (0.0006) [2023-12-27 16:05:35,548][20381] Updated weights for policy 1, policy_version 1506230 (0.0006) [2023-12-27 16:05:35,595][20381] Updated weights for policy 1, policy_version 1506240 (0.0006) [2023-12-27 16:05:35,646][20381] Updated weights for policy 1, policy_version 1506250 (0.0007) [2023-12-27 16:05:35,818][20347] Updated weights for policy 0, policy_version 1507840 (0.0008) [2023-12-27 16:05:35,880][20347] Updated weights for policy 0, policy_version 1507850 (0.0006) [2023-12-27 16:05:35,940][20347] Updated weights for policy 0, policy_version 1507860 (0.0006) [2023-12-27 16:05:36,280][20381] Updated weights for policy 1, policy_version 1506260 (0.0009) [2023-12-27 16:05:36,337][20381] Updated weights for policy 1, policy_version 1506270 (0.0011) [2023-12-27 16:05:36,396][20381] Updated weights for policy 1, policy_version 1506280 (0.0011) [2023-12-27 16:05:36,602][20347] Updated weights for policy 0, policy_version 1507870 (0.0009) [2023-12-27 16:05:36,665][20347] Updated weights for policy 0, policy_version 1507880 (0.0011) [2023-12-27 16:05:36,711][20347] Updated weights for policy 0, policy_version 1507890 (0.0010) [2023-12-27 16:05:37,104][20381] Updated weights for policy 1, policy_version 1506290 (0.0010) [2023-12-27 16:05:37,161][20381] Updated weights for policy 1, policy_version 1506300 (0.0007) [2023-12-27 16:05:37,220][20381] Updated weights for policy 1, policy_version 1506310 (0.0007) [2023-12-27 16:05:37,270][20381] Updated weights for policy 1, policy_version 1506320 (0.0008) [2023-12-27 16:05:37,338][20347] Updated weights for policy 0, policy_version 1507900 (0.0008) [2023-12-27 16:05:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 771751936. Throughput: 0: 10068.7, 1: 10395.6. Samples: 771719532. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:37,363][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 16:05:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001506320_385654784.pth... [2023-12-27 16:05:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001505104_385343488.pth [2023-12-27 16:05:37,398][20347] Updated weights for policy 0, policy_version 1507910 (0.0008) [2023-12-27 16:05:37,457][20347] Updated weights for policy 0, policy_version 1507920 (0.0007) [2023-12-27 16:05:37,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001507928_386105344.pth... [2023-12-27 16:05:37,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001506712_385794048.pth [2023-12-27 16:05:37,934][20381] Updated weights for policy 1, policy_version 1506330 (0.0005) [2023-12-27 16:05:37,984][20381] Updated weights for policy 1, policy_version 1506340 (0.0005) [2023-12-27 16:05:38,043][20381] Updated weights for policy 1, policy_version 1506350 (0.0006) [2023-12-27 16:05:38,140][20347] Updated weights for policy 0, policy_version 1507930 (0.0007) [2023-12-27 16:05:38,196][20347] Updated weights for policy 0, policy_version 1507940 (0.0005) [2023-12-27 16:05:38,247][20347] Updated weights for policy 0, policy_version 1507950 (0.0005) [2023-12-27 16:05:38,303][20347] Updated weights for policy 0, policy_version 1507960 (0.0005) [2023-12-27 16:05:38,687][20381] Updated weights for policy 1, policy_version 1506360 (0.0006) [2023-12-27 16:05:38,739][20381] Updated weights for policy 1, policy_version 1506370 (0.0005) [2023-12-27 16:05:38,801][20381] Updated weights for policy 1, policy_version 1506380 (0.0005) [2023-12-27 16:05:38,915][20347] Updated weights for policy 0, policy_version 1507970 (0.0005) [2023-12-27 16:05:38,976][20347] Updated weights for policy 0, policy_version 1507980 (0.0005) [2023-12-27 16:05:39,030][20347] Updated weights for policy 0, policy_version 1507990 (0.0005) [2023-12-27 16:05:39,456][20381] Updated weights for policy 1, policy_version 1506390 (0.0007) [2023-12-27 16:05:39,508][20381] Updated weights for policy 1, policy_version 1506400 (0.0008) [2023-12-27 16:05:39,561][20381] Updated weights for policy 1, policy_version 1506410 (0.0008) [2023-12-27 16:05:39,626][20347] Updated weights for policy 0, policy_version 1508000 (0.0008) [2023-12-27 16:05:39,681][20347] Updated weights for policy 0, policy_version 1508010 (0.0008) [2023-12-27 16:05:39,745][20347] Updated weights for policy 0, policy_version 1508020 (0.0007) [2023-12-27 16:05:40,255][20381] Updated weights for policy 1, policy_version 1506420 (0.0008) [2023-12-27 16:05:40,324][20381] Updated weights for policy 1, policy_version 1506430 (0.0007) [2023-12-27 16:05:40,390][20381] Updated weights for policy 1, policy_version 1506440 (0.0008) [2023-12-27 16:05:40,457][20347] Updated weights for policy 0, policy_version 1508030 (0.0007) [2023-12-27 16:05:40,518][20347] Updated weights for policy 0, policy_version 1508040 (0.0010) [2023-12-27 16:05:40,572][20347] Updated weights for policy 0, policy_version 1508050 (0.0011) [2023-12-27 16:05:41,024][20381] Updated weights for policy 1, policy_version 1506450 (0.0007) [2023-12-27 16:05:41,080][20381] Updated weights for policy 1, policy_version 1506460 (0.0007) [2023-12-27 16:05:41,147][20381] Updated weights for policy 1, policy_version 1506470 (0.0007) [2023-12-27 16:05:41,206][20381] Updated weights for policy 1, policy_version 1506480 (0.0007) [2023-12-27 16:05:41,277][20347] Updated weights for policy 0, policy_version 1508060 (0.0009) [2023-12-27 16:05:41,332][20347] Updated weights for policy 0, policy_version 1508070 (0.0010) [2023-12-27 16:05:41,402][20347] Updated weights for policy 0, policy_version 1508080 (0.0010) [2023-12-27 16:05:41,919][20381] Updated weights for policy 1, policy_version 1506490 (0.0007) [2023-12-27 16:05:41,980][20381] Updated weights for policy 1, policy_version 1506500 (0.0007) [2023-12-27 16:05:42,033][20381] Updated weights for policy 1, policy_version 1506510 (0.0008) [2023-12-27 16:05:42,131][20347] Updated weights for policy 0, policy_version 1508090 (0.0007) [2023-12-27 16:05:42,191][20347] Updated weights for policy 0, policy_version 1508100 (0.0008) [2023-12-27 16:05:42,246][20347] Updated weights for policy 0, policy_version 1508110 (0.0008) [2023-12-27 16:05:42,304][20347] Updated weights for policy 0, policy_version 1508120 (0.0009) [2023-12-27 16:05:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 771858432. Throughput: 0: 10173.1, 1: 10373.6. Samples: 771844648. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:42,363][19170] Avg episode reward: [(0, '990.250'), (1, '990.280')] [2023-12-27 16:05:42,702][20381] Updated weights for policy 1, policy_version 1506520 (0.0008) [2023-12-27 16:05:42,755][20381] Updated weights for policy 1, policy_version 1506530 (0.0008) [2023-12-27 16:05:42,814][20381] Updated weights for policy 1, policy_version 1506540 (0.0008) [2023-12-27 16:05:42,976][20347] Updated weights for policy 0, policy_version 1508130 (0.0006) [2023-12-27 16:05:43,023][20347] Updated weights for policy 0, policy_version 1508140 (0.0008) [2023-12-27 16:05:43,078][20347] Updated weights for policy 0, policy_version 1508150 (0.0008) [2023-12-27 16:05:43,525][20381] Updated weights for policy 1, policy_version 1506550 (0.0008) [2023-12-27 16:05:43,576][20381] Updated weights for policy 1, policy_version 1506560 (0.0010) [2023-12-27 16:05:43,641][20381] Updated weights for policy 1, policy_version 1506570 (0.0011) [2023-12-27 16:05:43,755][20347] Updated weights for policy 0, policy_version 1508160 (0.0007) [2023-12-27 16:05:43,812][20347] Updated weights for policy 0, policy_version 1508170 (0.0007) [2023-12-27 16:05:43,863][20347] Updated weights for policy 0, policy_version 1508180 (0.0007) [2023-12-27 16:05:44,265][20381] Updated weights for policy 1, policy_version 1506580 (0.0009) [2023-12-27 16:05:44,316][20381] Updated weights for policy 1, policy_version 1506590 (0.0007) [2023-12-27 16:05:44,366][20381] Updated weights for policy 1, policy_version 1506600 (0.0007) [2023-12-27 16:05:44,507][20347] Updated weights for policy 0, policy_version 1508190 (0.0006) [2023-12-27 16:05:44,562][20347] Updated weights for policy 0, policy_version 1508200 (0.0007) [2023-12-27 16:05:44,614][20347] Updated weights for policy 0, policy_version 1508210 (0.0006) [2023-12-27 16:05:45,088][20381] Updated weights for policy 1, policy_version 1506610 (0.0005) [2023-12-27 16:05:45,143][20381] Updated weights for policy 1, policy_version 1506620 (0.0007) [2023-12-27 16:05:45,195][20381] Updated weights for policy 1, policy_version 1506630 (0.0008) [2023-12-27 16:05:45,250][20381] Updated weights for policy 1, policy_version 1506640 (0.0008) [2023-12-27 16:05:45,336][20347] Updated weights for policy 0, policy_version 1508220 (0.0006) [2023-12-27 16:05:45,397][20347] Updated weights for policy 0, policy_version 1508230 (0.0007) [2023-12-27 16:05:45,459][20347] Updated weights for policy 0, policy_version 1508240 (0.0009) [2023-12-27 16:05:45,961][20381] Updated weights for policy 1, policy_version 1506650 (0.0008) [2023-12-27 16:05:46,006][20381] Updated weights for policy 1, policy_version 1506660 (0.0007) [2023-12-27 16:05:46,057][20347] Updated weights for policy 0, policy_version 1508250 (0.0008) [2023-12-27 16:05:46,063][20381] Updated weights for policy 1, policy_version 1506670 (0.0007) [2023-12-27 16:05:46,107][20347] Updated weights for policy 0, policy_version 1508260 (0.0008) [2023-12-27 16:05:46,158][20347] Updated weights for policy 0, policy_version 1508270 (0.0007) [2023-12-27 16:05:46,207][20347] Updated weights for policy 0, policy_version 1508280 (0.0008) [2023-12-27 16:05:46,807][20381] Updated weights for policy 1, policy_version 1506680 (0.0008) [2023-12-27 16:05:46,840][20347] Updated weights for policy 0, policy_version 1508290 (0.0009) [2023-12-27 16:05:46,861][20381] Updated weights for policy 1, policy_version 1506690 (0.0008) [2023-12-27 16:05:46,889][20347] Updated weights for policy 0, policy_version 1508300 (0.0006) [2023-12-27 16:05:46,910][20381] Updated weights for policy 1, policy_version 1506700 (0.0005) [2023-12-27 16:05:46,937][20347] Updated weights for policy 0, policy_version 1508310 (0.0006) [2023-12-27 16:05:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 771956736. Throughput: 0: 10275.1, 1: 10336.2. Samples: 771966820. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:47,363][19170] Avg episode reward: [(0, '990.260'), (1, '990.280')] [2023-12-27 16:05:47,562][20347] Updated weights for policy 0, policy_version 1508320 (0.0006) [2023-12-27 16:05:47,607][20381] Updated weights for policy 1, policy_version 1506710 (0.0007) [2023-12-27 16:05:47,620][20347] Updated weights for policy 0, policy_version 1508330 (0.0006) [2023-12-27 16:05:47,655][20381] Updated weights for policy 1, policy_version 1506720 (0.0006) [2023-12-27 16:05:47,678][20347] Updated weights for policy 0, policy_version 1508340 (0.0007) [2023-12-27 16:05:47,707][20381] Updated weights for policy 1, policy_version 1506730 (0.0006) [2023-12-27 16:05:48,359][20381] Updated weights for policy 1, policy_version 1506740 (0.0009) [2023-12-27 16:05:48,382][20347] Updated weights for policy 0, policy_version 1508350 (0.0007) [2023-12-27 16:05:48,418][20381] Updated weights for policy 1, policy_version 1506750 (0.0007) [2023-12-27 16:05:48,431][20347] Updated weights for policy 0, policy_version 1508360 (0.0006) [2023-12-27 16:05:48,474][20381] Updated weights for policy 1, policy_version 1506760 (0.0008) [2023-12-27 16:05:48,491][20347] Updated weights for policy 0, policy_version 1508370 (0.0006) [2023-12-27 16:05:49,162][20381] Updated weights for policy 1, policy_version 1506770 (0.0007) [2023-12-27 16:05:49,174][20347] Updated weights for policy 0, policy_version 1508380 (0.0007) [2023-12-27 16:05:49,213][20381] Updated weights for policy 1, policy_version 1506780 (0.0005) [2023-12-27 16:05:49,228][20347] Updated weights for policy 0, policy_version 1508390 (0.0006) [2023-12-27 16:05:49,276][20381] Updated weights for policy 1, policy_version 1506790 (0.0008) [2023-12-27 16:05:49,289][20347] Updated weights for policy 0, policy_version 1508400 (0.0007) [2023-12-27 16:05:49,334][20381] Updated weights for policy 1, policy_version 1506800 (0.0008) [2023-12-27 16:05:50,017][20381] Updated weights for policy 1, policy_version 1506810 (0.0006) [2023-12-27 16:05:50,028][20347] Updated weights for policy 0, policy_version 1508410 (0.0007) [2023-12-27 16:05:50,079][20381] Updated weights for policy 1, policy_version 1506820 (0.0005) [2023-12-27 16:05:50,085][20347] Updated weights for policy 0, policy_version 1508420 (0.0007) [2023-12-27 16:05:50,132][20381] Updated weights for policy 1, policy_version 1506830 (0.0006) [2023-12-27 16:05:50,145][20347] Updated weights for policy 0, policy_version 1508430 (0.0007) [2023-12-27 16:05:50,205][20347] Updated weights for policy 0, policy_version 1508440 (0.0008) [2023-12-27 16:05:50,760][20381] Updated weights for policy 1, policy_version 1506840 (0.0007) [2023-12-27 16:05:50,816][20381] Updated weights for policy 1, policy_version 1506850 (0.0007) [2023-12-27 16:05:50,872][20381] Updated weights for policy 1, policy_version 1506860 (0.0008) [2023-12-27 16:05:50,906][20347] Updated weights for policy 0, policy_version 1508450 (0.0007) [2023-12-27 16:05:50,959][20347] Updated weights for policy 0, policy_version 1508460 (0.0005) [2023-12-27 16:05:51,020][20347] Updated weights for policy 0, policy_version 1508470 (0.0006) [2023-12-27 16:05:51,613][20381] Updated weights for policy 1, policy_version 1506870 (0.0008) [2023-12-27 16:05:51,683][20381] Updated weights for policy 1, policy_version 1506880 (0.0008) [2023-12-27 16:05:51,745][20381] Updated weights for policy 1, policy_version 1506890 (0.0008) [2023-12-27 16:05:51,796][20347] Updated weights for policy 0, policy_version 1508480 (0.0007) [2023-12-27 16:05:51,858][20347] Updated weights for policy 0, policy_version 1508490 (0.0007) [2023-12-27 16:05:51,921][20347] Updated weights for policy 0, policy_version 1508500 (0.0008) [2023-12-27 16:05:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 772055040. Throughput: 0: 10279.0, 1: 10265.1. Samples: 772027732. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:52,363][19170] Avg episode reward: [(0, '990.260'), (1, '990.260')] [2023-12-27 16:05:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001506896_385802240.pth... [2023-12-27 16:05:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001508504_386252800.pth... [2023-12-27 16:05:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001507288_385941504.pth [2023-12-27 16:05:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001505712_385499136.pth [2023-12-27 16:05:52,499][20381] Updated weights for policy 1, policy_version 1506900 (0.0008) [2023-12-27 16:05:52,556][20381] Updated weights for policy 1, policy_version 1506910 (0.0008) [2023-12-27 16:05:52,601][20347] Updated weights for policy 0, policy_version 1508510 (0.0006) [2023-12-27 16:05:52,611][20381] Updated weights for policy 1, policy_version 1506920 (0.0006) [2023-12-27 16:05:52,659][20347] Updated weights for policy 0, policy_version 1508520 (0.0007) [2023-12-27 16:05:52,716][20347] Updated weights for policy 0, policy_version 1508530 (0.0008) [2023-12-27 16:05:53,308][20381] Updated weights for policy 1, policy_version 1506930 (0.0006) [2023-12-27 16:05:53,362][20381] Updated weights for policy 1, policy_version 1506940 (0.0007) [2023-12-27 16:05:53,414][20347] Updated weights for policy 0, policy_version 1508540 (0.0007) [2023-12-27 16:05:53,416][20381] Updated weights for policy 1, policy_version 1506950 (0.0008) [2023-12-27 16:05:53,467][20381] Updated weights for policy 1, policy_version 1506960 (0.0007) [2023-12-27 16:05:53,469][20347] Updated weights for policy 0, policy_version 1508550 (0.0005) [2023-12-27 16:05:53,525][20347] Updated weights for policy 0, policy_version 1508560 (0.0008) [2023-12-27 16:05:54,126][20381] Updated weights for policy 1, policy_version 1506970 (0.0007) [2023-12-27 16:05:54,191][20381] Updated weights for policy 1, policy_version 1506980 (0.0005) [2023-12-27 16:05:54,222][20347] Updated weights for policy 0, policy_version 1508570 (0.0007) [2023-12-27 16:05:54,248][20381] Updated weights for policy 1, policy_version 1506990 (0.0010) [2023-12-27 16:05:54,278][20347] Updated weights for policy 0, policy_version 1508580 (0.0008) [2023-12-27 16:05:54,336][20347] Updated weights for policy 0, policy_version 1508590 (0.0007) [2023-12-27 16:05:54,388][20347] Updated weights for policy 0, policy_version 1508600 (0.0008) [2023-12-27 16:05:54,860][20381] Updated weights for policy 1, policy_version 1507000 (0.0008) [2023-12-27 16:05:54,909][20381] Updated weights for policy 1, policy_version 1507010 (0.0010) [2023-12-27 16:05:54,958][20381] Updated weights for policy 1, policy_version 1507020 (0.0010) [2023-12-27 16:05:54,999][20347] Updated weights for policy 0, policy_version 1508610 (0.0006) [2023-12-27 16:05:55,048][20347] Updated weights for policy 0, policy_version 1508620 (0.0009) [2023-12-27 16:05:55,095][20347] Updated weights for policy 0, policy_version 1508630 (0.0009) [2023-12-27 16:05:55,567][20381] Updated weights for policy 1, policy_version 1507030 (0.0011) [2023-12-27 16:05:55,624][20381] Updated weights for policy 1, policy_version 1507040 (0.0010) [2023-12-27 16:05:55,683][20381] Updated weights for policy 1, policy_version 1507050 (0.0011) [2023-12-27 16:05:55,734][20347] Updated weights for policy 0, policy_version 1508640 (0.0006) [2023-12-27 16:05:55,795][20347] Updated weights for policy 0, policy_version 1508650 (0.0007) [2023-12-27 16:05:55,849][20347] Updated weights for policy 0, policy_version 1508660 (0.0010) [2023-12-27 16:05:56,354][20381] Updated weights for policy 1, policy_version 1507060 (0.0010) [2023-12-27 16:05:56,410][20381] Updated weights for policy 1, policy_version 1507070 (0.0010) [2023-12-27 16:05:56,470][20381] Updated weights for policy 1, policy_version 1507080 (0.0010) [2023-12-27 16:05:56,542][20347] Updated weights for policy 0, policy_version 1508670 (0.0008) [2023-12-27 16:05:56,600][20347] Updated weights for policy 0, policy_version 1508680 (0.0007) [2023-12-27 16:05:56,658][20347] Updated weights for policy 0, policy_version 1508690 (0.0008) [2023-12-27 16:05:57,141][20381] Updated weights for policy 1, policy_version 1507090 (0.0007) [2023-12-27 16:05:57,195][20381] Updated weights for policy 1, policy_version 1507100 (0.0010) [2023-12-27 16:05:57,249][20381] Updated weights for policy 1, policy_version 1507110 (0.0010) [2023-12-27 16:05:57,307][20381] Updated weights for policy 1, policy_version 1507120 (0.0010) [2023-12-27 16:05:57,343][20347] Updated weights for policy 0, policy_version 1508700 (0.0009) [2023-12-27 16:05:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 772161536. Throughput: 0: 10276.5, 1: 10287.5. Samples: 772150396. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:05:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.260')] [2023-12-27 16:05:57,395][20347] Updated weights for policy 0, policy_version 1508710 (0.0010) [2023-12-27 16:05:57,442][20347] Updated weights for policy 0, policy_version 1508720 (0.0008) [2023-12-27 16:05:57,972][20381] Updated weights for policy 1, policy_version 1507130 (0.0010) [2023-12-27 16:05:58,029][20381] Updated weights for policy 1, policy_version 1507140 (0.0010) [2023-12-27 16:05:58,080][20381] Updated weights for policy 1, policy_version 1507150 (0.0010) [2023-12-27 16:05:58,133][20347] Updated weights for policy 0, policy_version 1508730 (0.0010) [2023-12-27 16:05:58,185][20347] Updated weights for policy 0, policy_version 1508740 (0.0010) [2023-12-27 16:05:58,250][20347] Updated weights for policy 0, policy_version 1508750 (0.0010) [2023-12-27 16:05:58,307][20347] Updated weights for policy 0, policy_version 1508760 (0.0010) [2023-12-27 16:05:58,806][20381] Updated weights for policy 1, policy_version 1507160 (0.0008) [2023-12-27 16:05:58,873][20381] Updated weights for policy 1, policy_version 1507170 (0.0007) [2023-12-27 16:05:58,934][20381] Updated weights for policy 1, policy_version 1507180 (0.0007) [2023-12-27 16:05:59,169][20347] Updated weights for policy 0, policy_version 1508770 (0.0006) [2023-12-27 16:05:59,232][20347] Updated weights for policy 0, policy_version 1508780 (0.0007) [2023-12-27 16:05:59,296][20347] Updated weights for policy 0, policy_version 1508790 (0.0007) [2023-12-27 16:05:59,727][20381] Updated weights for policy 1, policy_version 1507190 (0.0008) [2023-12-27 16:05:59,787][20381] Updated weights for policy 1, policy_version 1507200 (0.0007) [2023-12-27 16:05:59,851][20381] Updated weights for policy 1, policy_version 1507210 (0.0008) [2023-12-27 16:05:59,867][20347] Updated weights for policy 0, policy_version 1508800 (0.0008) [2023-12-27 16:05:59,924][20347] Updated weights for policy 0, policy_version 1508810 (0.0006) [2023-12-27 16:05:59,984][20347] Updated weights for policy 0, policy_version 1508820 (0.0007) [2023-12-27 16:06:00,525][20381] Updated weights for policy 1, policy_version 1507220 (0.0008) [2023-12-27 16:06:00,583][20381] Updated weights for policy 1, policy_version 1507230 (0.0011) [2023-12-27 16:06:00,625][20347] Updated weights for policy 0, policy_version 1508830 (0.0008) [2023-12-27 16:06:00,631][20381] Updated weights for policy 1, policy_version 1507240 (0.0010) [2023-12-27 16:06:00,671][20347] Updated weights for policy 0, policy_version 1508840 (0.0005) [2023-12-27 16:06:00,731][20347] Updated weights for policy 0, policy_version 1508850 (0.0007) [2023-12-27 16:06:01,404][20381] Updated weights for policy 1, policy_version 1507250 (0.0010) [2023-12-27 16:06:01,462][20381] Updated weights for policy 1, policy_version 1507260 (0.0008) [2023-12-27 16:06:01,490][20347] Updated weights for policy 0, policy_version 1508860 (0.0009) [2023-12-27 16:06:01,526][20381] Updated weights for policy 1, policy_version 1507270 (0.0008) [2023-12-27 16:06:01,547][20347] Updated weights for policy 0, policy_version 1508870 (0.0008) [2023-12-27 16:06:01,587][20381] Updated weights for policy 1, policy_version 1507280 (0.0006) [2023-12-27 16:06:01,603][20347] Updated weights for policy 0, policy_version 1508880 (0.0009) [2023-12-27 16:06:02,299][20381] Updated weights for policy 1, policy_version 1507290 (0.0008) [2023-12-27 16:06:02,329][20347] Updated weights for policy 0, policy_version 1508890 (0.0009) [2023-12-27 16:06:02,360][20381] Updated weights for policy 1, policy_version 1507300 (0.0008) [2023-12-27 16:06:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 772251648. Throughput: 0: 10230.6, 1: 10186.8. Samples: 772267756. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:06:02,363][19170] Avg episode reward: [(0, '980.450'), (1, '1000.000')] [2023-12-27 16:06:02,394][20347] Updated weights for policy 0, policy_version 1508900 (0.0008) [2023-12-27 16:06:02,422][20381] Updated weights for policy 1, policy_version 1507310 (0.0008) [2023-12-27 16:06:02,448][20347] Updated weights for policy 0, policy_version 1508910 (0.0006) [2023-12-27 16:06:02,505][20347] Updated weights for policy 0, policy_version 1508920 (0.0007) [2023-12-27 16:06:03,100][20381] Updated weights for policy 1, policy_version 1507320 (0.0010) [2023-12-27 16:06:03,132][20347] Updated weights for policy 0, policy_version 1508930 (0.0005) [2023-12-27 16:06:03,155][20381] Updated weights for policy 1, policy_version 1507330 (0.0010) [2023-12-27 16:06:03,189][20347] Updated weights for policy 0, policy_version 1508940 (0.0007) [2023-12-27 16:06:03,211][20381] Updated weights for policy 1, policy_version 1507340 (0.0011) [2023-12-27 16:06:03,238][20347] Updated weights for policy 0, policy_version 1508950 (0.0006) [2023-12-27 16:06:03,809][20381] Updated weights for policy 1, policy_version 1507350 (0.0008) [2023-12-27 16:06:03,877][20381] Updated weights for policy 1, policy_version 1507360 (0.0007) [2023-12-27 16:06:03,903][20347] Updated weights for policy 0, policy_version 1508960 (0.0010) [2023-12-27 16:06:03,941][20381] Updated weights for policy 1, policy_version 1507370 (0.0008) [2023-12-27 16:06:03,954][20347] Updated weights for policy 0, policy_version 1508970 (0.0010) [2023-12-27 16:06:04,010][20347] Updated weights for policy 0, policy_version 1508980 (0.0010) [2023-12-27 16:06:04,570][20381] Updated weights for policy 1, policy_version 1507380 (0.0009) [2023-12-27 16:06:04,626][20381] Updated weights for policy 1, policy_version 1507390 (0.0007) [2023-12-27 16:06:04,675][20381] Updated weights for policy 1, policy_version 1507400 (0.0007) [2023-12-27 16:06:04,714][20347] Updated weights for policy 0, policy_version 1508990 (0.0010) [2023-12-27 16:06:04,770][20347] Updated weights for policy 0, policy_version 1509000 (0.0008) [2023-12-27 16:06:04,823][20347] Updated weights for policy 0, policy_version 1509010 (0.0005) [2023-12-27 16:06:05,342][20381] Updated weights for policy 1, policy_version 1507410 (0.0007) [2023-12-27 16:06:05,394][20381] Updated weights for policy 1, policy_version 1507420 (0.0006) [2023-12-27 16:06:05,404][20347] Updated weights for policy 0, policy_version 1509020 (0.0006) [2023-12-27 16:06:05,445][20381] Updated weights for policy 1, policy_version 1507430 (0.0005) [2023-12-27 16:06:05,458][20347] Updated weights for policy 0, policy_version 1509030 (0.0010) [2023-12-27 16:06:05,491][20381] Updated weights for policy 1, policy_version 1507440 (0.0005) [2023-12-27 16:06:05,504][20347] Updated weights for policy 0, policy_version 1509040 (0.0010) [2023-12-27 16:06:06,033][20381] Updated weights for policy 1, policy_version 1507450 (0.0007) [2023-12-27 16:06:06,093][20381] Updated weights for policy 1, policy_version 1507460 (0.0005) [2023-12-27 16:06:06,153][20381] Updated weights for policy 1, policy_version 1507470 (0.0008) [2023-12-27 16:06:06,210][20347] Updated weights for policy 0, policy_version 1509050 (0.0009) [2023-12-27 16:06:06,260][20347] Updated weights for policy 0, policy_version 1509060 (0.0005) [2023-12-27 16:06:06,310][20347] Updated weights for policy 0, policy_version 1509070 (0.0007) [2023-12-27 16:06:06,357][20347] Updated weights for policy 0, policy_version 1509080 (0.0010) [2023-12-27 16:06:06,781][20381] Updated weights for policy 1, policy_version 1507480 (0.0007) [2023-12-27 16:06:06,834][20381] Updated weights for policy 1, policy_version 1507490 (0.0008) [2023-12-27 16:06:06,879][20381] Updated weights for policy 1, policy_version 1507500 (0.0007) [2023-12-27 16:06:07,006][20347] Updated weights for policy 0, policy_version 1509090 (0.0011) [2023-12-27 16:06:07,058][20347] Updated weights for policy 0, policy_version 1509100 (0.0010) [2023-12-27 16:06:07,119][20347] Updated weights for policy 0, policy_version 1509110 (0.0009) [2023-12-27 16:06:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 772366336. Throughput: 0: 10241.5, 1: 10249.6. Samples: 772331540. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:06:07,363][19170] Avg episode reward: [(0, '980.450'), (1, '1000.000')] [2023-12-27 16:06:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001507504_385957888.pth... [2023-12-27 16:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001509112_386408448.pth... [2023-12-27 16:06:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001507928_386105344.pth [2023-12-27 16:06:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001506320_385654784.pth [2023-12-27 16:06:07,587][20381] Updated weights for policy 1, policy_version 1507510 (0.0008) [2023-12-27 16:06:07,640][20381] Updated weights for policy 1, policy_version 1507520 (0.0011) [2023-12-27 16:06:07,691][20381] Updated weights for policy 1, policy_version 1507530 (0.0010) [2023-12-27 16:06:07,781][20347] Updated weights for policy 0, policy_version 1509120 (0.0010) [2023-12-27 16:06:07,827][20347] Updated weights for policy 0, policy_version 1509130 (0.0010) [2023-12-27 16:06:07,888][20347] Updated weights for policy 0, policy_version 1509140 (0.0010) [2023-12-27 16:06:08,391][20381] Updated weights for policy 1, policy_version 1507540 (0.0009) [2023-12-27 16:06:08,451][20381] Updated weights for policy 1, policy_version 1507550 (0.0008) [2023-12-27 16:06:08,513][20381] Updated weights for policy 1, policy_version 1507560 (0.0007) [2023-12-27 16:06:08,594][20347] Updated weights for policy 0, policy_version 1509150 (0.0010) [2023-12-27 16:06:08,656][20347] Updated weights for policy 0, policy_version 1509160 (0.0011) [2023-12-27 16:06:08,718][20347] Updated weights for policy 0, policy_version 1509170 (0.0010) [2023-12-27 16:06:09,204][20381] Updated weights for policy 1, policy_version 1507570 (0.0007) [2023-12-27 16:06:09,269][20381] Updated weights for policy 1, policy_version 1507580 (0.0009) [2023-12-27 16:06:09,329][20381] Updated weights for policy 1, policy_version 1507590 (0.0007) [2023-12-27 16:06:09,389][20347] Updated weights for policy 0, policy_version 1509180 (0.0009) [2023-12-27 16:06:09,394][20381] Updated weights for policy 1, policy_version 1507600 (0.0008) [2023-12-27 16:06:09,441][20347] Updated weights for policy 0, policy_version 1509190 (0.0007) [2023-12-27 16:06:09,495][20347] Updated weights for policy 0, policy_version 1509200 (0.0007) [2023-12-27 16:06:10,077][20381] Updated weights for policy 1, policy_version 1507610 (0.0007) [2023-12-27 16:06:10,140][20381] Updated weights for policy 1, policy_version 1507620 (0.0008) [2023-12-27 16:06:10,197][20381] Updated weights for policy 1, policy_version 1507630 (0.0007) [2023-12-27 16:06:10,233][20347] Updated weights for policy 0, policy_version 1509210 (0.0009) [2023-12-27 16:06:10,280][20347] Updated weights for policy 0, policy_version 1509220 (0.0010) [2023-12-27 16:06:10,332][20347] Updated weights for policy 0, policy_version 1509230 (0.0010) [2023-12-27 16:06:10,392][20347] Updated weights for policy 0, policy_version 1509240 (0.0010) [2023-12-27 16:06:10,864][20381] Updated weights for policy 1, policy_version 1507640 (0.0008) [2023-12-27 16:06:10,923][20381] Updated weights for policy 1, policy_version 1507650 (0.0007) [2023-12-27 16:06:10,983][20381] Updated weights for policy 1, policy_version 1507660 (0.0005) [2023-12-27 16:06:11,071][20347] Updated weights for policy 0, policy_version 1509250 (0.0008) [2023-12-27 16:06:11,141][20347] Updated weights for policy 0, policy_version 1509260 (0.0008) [2023-12-27 16:06:11,197][20347] Updated weights for policy 0, policy_version 1509270 (0.0008) [2023-12-27 16:06:11,772][20381] Updated weights for policy 1, policy_version 1507670 (0.0007) [2023-12-27 16:06:11,828][20381] Updated weights for policy 1, policy_version 1507680 (0.0008) [2023-12-27 16:06:11,880][20381] Updated weights for policy 1, policy_version 1507690 (0.0008) [2023-12-27 16:06:11,938][20347] Updated weights for policy 0, policy_version 1509280 (0.0007) [2023-12-27 16:06:11,987][20347] Updated weights for policy 0, policy_version 1509290 (0.0006) [2023-12-27 16:06:12,036][20347] Updated weights for policy 0, policy_version 1509300 (0.0008) [2023-12-27 16:06:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 772464640. Throughput: 0: 10270.8, 1: 10249.8. Samples: 772455116. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) [2023-12-27 16:06:12,363][19170] Avg episode reward: [(0, '990.250'), (1, '990.290')] [2023-12-27 16:06:12,605][20381] Updated weights for policy 1, policy_version 1507700 (0.0008) [2023-12-27 16:06:12,661][20381] Updated weights for policy 1, policy_version 1507710 (0.0007) [2023-12-27 16:06:12,713][20381] Updated weights for policy 1, policy_version 1507720 (0.0008) [2023-12-27 16:06:12,765][20347] Updated weights for policy 0, policy_version 1509310 (0.0006) [2023-12-27 16:06:12,827][20347] Updated weights for policy 0, policy_version 1509320 (0.0006) [2023-12-27 16:06:12,882][20347] Updated weights for policy 0, policy_version 1509330 (0.0005) [2023-12-27 16:06:13,447][20347] Updated weights for policy 0, policy_version 1509340 (0.0005) [2023-12-27 16:06:13,452][20381] Updated weights for policy 1, policy_version 1507730 (0.0007) [2023-12-27 16:06:13,501][20347] Updated weights for policy 0, policy_version 1509350 (0.0005) [2023-12-27 16:06:13,511][20381] Updated weights for policy 1, policy_version 1507740 (0.0007) [2023-12-27 16:06:13,549][20347] Updated weights for policy 0, policy_version 1509360 (0.0005) [2023-12-27 16:06:13,565][20381] Updated weights for policy 1, policy_version 1507750 (0.0006) [2023-12-27 16:06:13,617][20381] Updated weights for policy 1, policy_version 1507760 (0.0005) [2023-12-27 16:06:14,061][20347] Updated weights for policy 0, policy_version 1509370 (0.0006) [2023-12-27 16:06:14,110][20347] Updated weights for policy 0, policy_version 1509380 (0.0009) [2023-12-27 16:06:14,161][20347] Updated weights for policy 0, policy_version 1509390 (0.0009) [2023-12-27 16:06:14,212][20347] Updated weights for policy 0, policy_version 1509400 (0.0007) [2023-12-27 16:06:14,247][20381] Updated weights for policy 1, policy_version 1507770 (0.0007) [2023-12-27 16:06:14,303][20381] Updated weights for policy 1, policy_version 1507780 (0.0008) [2023-12-27 16:06:14,364][20381] Updated weights for policy 1, policy_version 1507790 (0.0008) [2023-12-27 16:06:14,863][20347] Updated weights for policy 0, policy_version 1509410 (0.0007) [2023-12-27 16:06:14,925][20347] Updated weights for policy 0, policy_version 1509420 (0.0007) [2023-12-27 16:06:14,976][20347] Updated weights for policy 0, policy_version 1509430 (0.0008) [2023-12-27 16:06:15,088][20381] Updated weights for policy 1, policy_version 1507800 (0.0007) [2023-12-27 16:06:15,141][20381] Updated weights for policy 1, policy_version 1507810 (0.0008) [2023-12-27 16:06:15,190][20381] Updated weights for policy 1, policy_version 1507820 (0.0008) [2023-12-27 16:06:15,623][20347] Updated weights for policy 0, policy_version 1509440 (0.0008) [2023-12-27 16:06:15,671][20347] Updated weights for policy 0, policy_version 1509450 (0.0008) [2023-12-27 16:06:15,726][20347] Updated weights for policy 0, policy_version 1509460 (0.0008) [2023-12-27 16:06:15,872][20381] Updated weights for policy 1, policy_version 1507830 (0.0008) [2023-12-27 16:06:15,919][20381] Updated weights for policy 1, policy_version 1507840 (0.0007) [2023-12-27 16:06:15,971][20381] Updated weights for policy 1, policy_version 1507850 (0.0007) [2023-12-27 16:06:16,436][20347] Updated weights for policy 0, policy_version 1509470 (0.0007) [2023-12-27 16:06:16,486][20347] Updated weights for policy 0, policy_version 1509480 (0.0007) [2023-12-27 16:06:16,537][20347] Updated weights for policy 0, policy_version 1509490 (0.0008) [2023-12-27 16:06:16,564][20381] Updated weights for policy 1, policy_version 1507860 (0.0009) [2023-12-27 16:06:16,621][20381] Updated weights for policy 1, policy_version 1507870 (0.0005) [2023-12-27 16:06:16,672][20381] Updated weights for policy 1, policy_version 1507880 (0.0010) [2023-12-27 16:06:17,158][20347] Updated weights for policy 0, policy_version 1509500 (0.0005) [2023-12-27 16:06:17,206][20347] Updated weights for policy 0, policy_version 1509510 (0.0005) [2023-12-27 16:06:17,208][20291] KL-divergence is very high: 484.5980 [2023-12-27 16:06:17,247][20291] KL-divergence is very high: 442.6562 [2023-12-27 16:06:17,258][20347] Updated weights for policy 0, policy_version 1509520 (0.0007) [2023-12-27 16:06:17,290][20291] KL-divergence is very high: 182.2365 [2023-12-27 16:06:17,316][20381] Updated weights for policy 1, policy_version 1507890 (0.0007) [2023-12-27 16:06:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 772571136. Throughput: 0: 10359.5, 1: 10202.9. Samples: 772580064. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:17,363][19170] Avg episode reward: [(0, '980.420'), (1, '980.540')] [2023-12-27 16:06:17,372][20381] Updated weights for policy 1, policy_version 1507900 (0.0007) [2023-12-27 16:06:17,427][20381] Updated weights for policy 1, policy_version 1507910 (0.0008) [2023-12-27 16:06:17,481][20381] Updated weights for policy 1, policy_version 1507920 (0.0008) [2023-12-27 16:06:17,851][20347] Updated weights for policy 0, policy_version 1509530 (0.0007) [2023-12-27 16:06:17,906][20347] Updated weights for policy 0, policy_version 1509540 (0.0006) [2023-12-27 16:06:17,966][20347] Updated weights for policy 0, policy_version 1509550 (0.0005) [2023-12-27 16:06:18,027][20347] Updated weights for policy 0, policy_version 1509560 (0.0006) [2023-12-27 16:06:18,150][20381] Updated weights for policy 1, policy_version 1507930 (0.0010) [2023-12-27 16:06:18,198][20381] Updated weights for policy 1, policy_version 1507940 (0.0010) [2023-12-27 16:06:18,253][20381] Updated weights for policy 1, policy_version 1507950 (0.0010) [2023-12-27 16:06:18,632][20347] Updated weights for policy 0, policy_version 1509570 (0.0008) [2023-12-27 16:06:18,688][20347] Updated weights for policy 0, policy_version 1509580 (0.0008) [2023-12-27 16:06:18,744][20347] Updated weights for policy 0, policy_version 1509590 (0.0008) [2023-12-27 16:06:18,906][20381] Updated weights for policy 1, policy_version 1507960 (0.0007) [2023-12-27 16:06:18,964][20381] Updated weights for policy 1, policy_version 1507970 (0.0008) [2023-12-27 16:06:19,016][20381] Updated weights for policy 1, policy_version 1507980 (0.0008) [2023-12-27 16:06:19,504][20347] Updated weights for policy 0, policy_version 1509600 (0.0007) [2023-12-27 16:06:19,566][20347] Updated weights for policy 0, policy_version 1509610 (0.0007) [2023-12-27 16:06:19,629][20347] Updated weights for policy 0, policy_version 1509620 (0.0007) [2023-12-27 16:06:19,695][20381] Updated weights for policy 1, policy_version 1507990 (0.0007) [2023-12-27 16:06:19,755][20381] Updated weights for policy 1, policy_version 1508000 (0.0008) [2023-12-27 16:06:19,815][20381] Updated weights for policy 1, policy_version 1508010 (0.0008) [2023-12-27 16:06:20,363][20347] Updated weights for policy 0, policy_version 1509630 (0.0008) [2023-12-27 16:06:20,423][20347] Updated weights for policy 0, policy_version 1509640 (0.0007) [2023-12-27 16:06:20,475][20347] Updated weights for policy 0, policy_version 1509650 (0.0006) [2023-12-27 16:06:20,482][20381] Updated weights for policy 1, policy_version 1508020 (0.0007) [2023-12-27 16:06:20,541][20381] Updated weights for policy 1, policy_version 1508030 (0.0005) [2023-12-27 16:06:20,606][20381] Updated weights for policy 1, policy_version 1508040 (0.0008) [2023-12-27 16:06:21,187][20381] Updated weights for policy 1, policy_version 1508050 (0.0006) [2023-12-27 16:06:21,187][20347] Updated weights for policy 0, policy_version 1509660 (0.0007) [2023-12-27 16:06:21,242][20347] Updated weights for policy 0, policy_version 1509670 (0.0008) [2023-12-27 16:06:21,243][20381] Updated weights for policy 1, policy_version 1508060 (0.0007) [2023-12-27 16:06:21,301][20381] Updated weights for policy 1, policy_version 1508070 (0.0006) [2023-12-27 16:06:21,303][20347] Updated weights for policy 0, policy_version 1509680 (0.0008) [2023-12-27 16:06:21,358][20381] Updated weights for policy 1, policy_version 1508080 (0.0006) [2023-12-27 16:06:21,974][20347] Updated weights for policy 0, policy_version 1509690 (0.0010) [2023-12-27 16:06:22,028][20347] Updated weights for policy 0, policy_version 1509700 (0.0008) [2023-12-27 16:06:22,091][20347] Updated weights for policy 0, policy_version 1509710 (0.0010) [2023-12-27 16:06:22,096][20381] Updated weights for policy 1, policy_version 1508090 (0.0006) [2023-12-27 16:06:22,147][20347] Updated weights for policy 0, policy_version 1509720 (0.0010) [2023-12-27 16:06:22,157][20381] Updated weights for policy 1, policy_version 1508100 (0.0006) [2023-12-27 16:06:22,215][20381] Updated weights for policy 1, policy_version 1508110 (0.0007) [2023-12-27 16:06:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 772677632. Throughput: 0: 10296.3, 1: 10198.3. Samples: 772641792. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:22,363][19170] Avg episode reward: [(0, '960.850'), (1, '980.590')] [2023-12-27 16:06:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001508112_386113536.pth... [2023-12-27 16:06:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001509720_386564096.pth... [2023-12-27 16:06:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001506896_385802240.pth [2023-12-27 16:06:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001508504_386252800.pth [2023-12-27 16:06:22,887][20347] Updated weights for policy 0, policy_version 1509730 (0.0006) [2023-12-27 16:06:22,912][20381] Updated weights for policy 1, policy_version 1508120 (0.0005) [2023-12-27 16:06:22,943][20347] Updated weights for policy 0, policy_version 1509740 (0.0006) [2023-12-27 16:06:22,968][20381] Updated weights for policy 1, policy_version 1508130 (0.0007) [2023-12-27 16:06:23,000][20347] Updated weights for policy 0, policy_version 1509750 (0.0007) [2023-12-27 16:06:23,025][20381] Updated weights for policy 1, policy_version 1508140 (0.0006) [2023-12-27 16:06:23,603][20347] Updated weights for policy 0, policy_version 1509760 (0.0005) [2023-12-27 16:06:23,657][20347] Updated weights for policy 0, policy_version 1509770 (0.0005) [2023-12-27 16:06:23,678][20381] Updated weights for policy 1, policy_version 1508150 (0.0006) [2023-12-27 16:06:23,714][20347] Updated weights for policy 0, policy_version 1509780 (0.0005) [2023-12-27 16:06:23,736][20381] Updated weights for policy 1, policy_version 1508160 (0.0007) [2023-12-27 16:06:23,795][20381] Updated weights for policy 1, policy_version 1508170 (0.0009) [2023-12-27 16:06:24,328][20347] Updated weights for policy 0, policy_version 1509790 (0.0005) [2023-12-27 16:06:24,382][20347] Updated weights for policy 0, policy_version 1509800 (0.0005) [2023-12-27 16:06:24,434][20347] Updated weights for policy 0, policy_version 1509810 (0.0006) [2023-12-27 16:06:24,488][20381] Updated weights for policy 1, policy_version 1508180 (0.0007) [2023-12-27 16:06:24,542][20381] Updated weights for policy 1, policy_version 1508190 (0.0007) [2023-12-27 16:06:24,597][20381] Updated weights for policy 1, policy_version 1508200 (0.0006) [2023-12-27 16:06:25,079][20347] Updated weights for policy 0, policy_version 1509820 (0.0007) [2023-12-27 16:06:25,127][20347] Updated weights for policy 0, policy_version 1509830 (0.0010) [2023-12-27 16:06:25,171][20347] Updated weights for policy 0, policy_version 1509840 (0.0010) [2023-12-27 16:06:25,214][20381] Updated weights for policy 1, policy_version 1508210 (0.0006) [2023-12-27 16:06:25,273][20381] Updated weights for policy 1, policy_version 1508220 (0.0008) [2023-12-27 16:06:25,332][20381] Updated weights for policy 1, policy_version 1508230 (0.0008) [2023-12-27 16:06:25,379][20381] Updated weights for policy 1, policy_version 1508240 (0.0008) [2023-12-27 16:06:25,734][20347] Updated weights for policy 0, policy_version 1509850 (0.0006) [2023-12-27 16:06:25,781][20347] Updated weights for policy 0, policy_version 1509860 (0.0007) [2023-12-27 16:06:25,829][20347] Updated weights for policy 0, policy_version 1509870 (0.0007) [2023-12-27 16:06:25,878][20347] Updated weights for policy 0, policy_version 1509880 (0.0007) [2023-12-27 16:06:26,073][20381] Updated weights for policy 1, policy_version 1508250 (0.0005) [2023-12-27 16:06:26,135][20381] Updated weights for policy 1, policy_version 1508260 (0.0006) [2023-12-27 16:06:26,188][20381] Updated weights for policy 1, policy_version 1508270 (0.0007) [2023-12-27 16:06:26,553][20347] Updated weights for policy 0, policy_version 1509890 (0.0007) [2023-12-27 16:06:26,599][20347] Updated weights for policy 0, policy_version 1509900 (0.0009) [2023-12-27 16:06:26,650][20347] Updated weights for policy 0, policy_version 1509910 (0.0007) [2023-12-27 16:06:26,801][20381] Updated weights for policy 1, policy_version 1508280 (0.0006) [2023-12-27 16:06:26,868][20381] Updated weights for policy 1, policy_version 1508290 (0.0010) [2023-12-27 16:06:26,923][20381] Updated weights for policy 1, policy_version 1508300 (0.0008) [2023-12-27 16:06:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 772775936. Throughput: 0: 10305.3, 1: 10211.8. Samples: 772767920. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:27,363][19170] Avg episode reward: [(0, '951.090'), (1, '980.600')] [2023-12-27 16:06:27,383][20347] Updated weights for policy 0, policy_version 1509920 (0.0008) [2023-12-27 16:06:27,436][20347] Updated weights for policy 0, policy_version 1509930 (0.0007) [2023-12-27 16:06:27,491][20347] Updated weights for policy 0, policy_version 1509940 (0.0007) [2023-12-27 16:06:27,569][20381] Updated weights for policy 1, policy_version 1508310 (0.0007) [2023-12-27 16:06:27,612][20381] Updated weights for policy 1, policy_version 1508320 (0.0005) [2023-12-27 16:06:27,664][20381] Updated weights for policy 1, policy_version 1508330 (0.0005) [2023-12-27 16:06:28,149][20347] Updated weights for policy 0, policy_version 1509950 (0.0009) [2023-12-27 16:06:28,207][20347] Updated weights for policy 0, policy_version 1509960 (0.0010) [2023-12-27 16:06:28,265][20347] Updated weights for policy 0, policy_version 1509970 (0.0010) [2023-12-27 16:06:28,275][20381] Updated weights for policy 1, policy_version 1508340 (0.0007) [2023-12-27 16:06:28,328][20381] Updated weights for policy 1, policy_version 1508350 (0.0010) [2023-12-27 16:06:28,396][20381] Updated weights for policy 1, policy_version 1508360 (0.0006) [2023-12-27 16:06:28,958][20347] Updated weights for policy 0, policy_version 1509980 (0.0010) [2023-12-27 16:06:29,005][20381] Updated weights for policy 1, policy_version 1508370 (0.0007) [2023-12-27 16:06:29,019][20347] Updated weights for policy 0, policy_version 1509990 (0.0010) [2023-12-27 16:06:29,060][20381] Updated weights for policy 1, policy_version 1508380 (0.0010) [2023-12-27 16:06:29,076][20347] Updated weights for policy 0, policy_version 1510000 (0.0009) [2023-12-27 16:06:29,107][20381] Updated weights for policy 1, policy_version 1508390 (0.0010) [2023-12-27 16:06:29,163][20381] Updated weights for policy 1, policy_version 1508400 (0.0010) [2023-12-27 16:06:29,720][20347] Updated weights for policy 0, policy_version 1510010 (0.0010) [2023-12-27 16:06:29,773][20347] Updated weights for policy 0, policy_version 1510020 (0.0007) [2023-12-27 16:06:29,840][20347] Updated weights for policy 0, policy_version 1510030 (0.0008) [2023-12-27 16:06:29,842][20381] Updated weights for policy 1, policy_version 1508410 (0.0007) [2023-12-27 16:06:29,897][20347] Updated weights for policy 0, policy_version 1510040 (0.0007) [2023-12-27 16:06:29,903][20381] Updated weights for policy 1, policy_version 1508420 (0.0008) [2023-12-27 16:06:29,969][20381] Updated weights for policy 1, policy_version 1508430 (0.0008) [2023-12-27 16:06:30,613][20347] Updated weights for policy 0, policy_version 1510050 (0.0009) [2023-12-27 16:06:30,621][20381] Updated weights for policy 1, policy_version 1508440 (0.0006) [2023-12-27 16:06:30,663][20347] Updated weights for policy 0, policy_version 1510060 (0.0009) [2023-12-27 16:06:30,670][20381] Updated weights for policy 1, policy_version 1508450 (0.0005) [2023-12-27 16:06:30,721][20347] Updated weights for policy 0, policy_version 1510070 (0.0010) [2023-12-27 16:06:30,726][20381] Updated weights for policy 1, policy_version 1508460 (0.0005) [2023-12-27 16:06:31,307][20381] Updated weights for policy 1, policy_version 1508470 (0.0007) [2023-12-27 16:06:31,377][20381] Updated weights for policy 1, policy_version 1508480 (0.0007) [2023-12-27 16:06:31,377][20347] Updated weights for policy 0, policy_version 1510080 (0.0010) [2023-12-27 16:06:31,437][20347] Updated weights for policy 0, policy_version 1510090 (0.0008) [2023-12-27 16:06:31,443][20381] Updated weights for policy 1, policy_version 1508490 (0.0006) [2023-12-27 16:06:31,499][20347] Updated weights for policy 0, policy_version 1510100 (0.0009) [2023-12-27 16:06:32,171][20381] Updated weights for policy 1, policy_version 1508500 (0.0007) [2023-12-27 16:06:32,202][20347] Updated weights for policy 0, policy_version 1510110 (0.0006) [2023-12-27 16:06:32,231][20381] Updated weights for policy 1, policy_version 1508510 (0.0008) [2023-12-27 16:06:32,267][20347] Updated weights for policy 0, policy_version 1510120 (0.0007) [2023-12-27 16:06:32,299][20381] Updated weights for policy 1, policy_version 1508520 (0.0008) [2023-12-27 16:06:32,324][20347] Updated weights for policy 0, policy_version 1510130 (0.0007) [2023-12-27 16:06:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 772882432. Throughput: 0: 10286.3, 1: 10277.0. Samples: 772892168. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:32,363][19170] Avg episode reward: [(0, '970.660'), (1, '990.260')] [2023-12-27 16:06:32,972][20347] Updated weights for policy 0, policy_version 1510140 (0.0008) [2023-12-27 16:06:33,011][20381] Updated weights for policy 1, policy_version 1508530 (0.0007) [2023-12-27 16:06:33,023][20347] Updated weights for policy 0, policy_version 1510150 (0.0009) [2023-12-27 16:06:33,070][20381] Updated weights for policy 1, policy_version 1508540 (0.0006) [2023-12-27 16:06:33,078][20347] Updated weights for policy 0, policy_version 1510160 (0.0007) [2023-12-27 16:06:33,128][20381] Updated weights for policy 1, policy_version 1508550 (0.0009) [2023-12-27 16:06:33,183][20381] Updated weights for policy 1, policy_version 1508560 (0.0007) [2023-12-27 16:06:33,706][20347] Updated weights for policy 0, policy_version 1510170 (0.0005) [2023-12-27 16:06:33,766][20347] Updated weights for policy 0, policy_version 1510180 (0.0005) [2023-12-27 16:06:33,779][20381] Updated weights for policy 1, policy_version 1508570 (0.0006) [2023-12-27 16:06:33,822][20347] Updated weights for policy 0, policy_version 1510190 (0.0005) [2023-12-27 16:06:33,830][20381] Updated weights for policy 1, policy_version 1508580 (0.0007) [2023-12-27 16:06:33,872][20347] Updated weights for policy 0, policy_version 1510200 (0.0005) [2023-12-27 16:06:33,881][20381] Updated weights for policy 1, policy_version 1508590 (0.0008) [2023-12-27 16:06:34,488][20347] Updated weights for policy 0, policy_version 1510210 (0.0010) [2023-12-27 16:06:34,549][20347] Updated weights for policy 0, policy_version 1510220 (0.0011) [2023-12-27 16:06:34,556][20381] Updated weights for policy 1, policy_version 1508600 (0.0006) [2023-12-27 16:06:34,610][20347] Updated weights for policy 0, policy_version 1510230 (0.0008) [2023-12-27 16:06:34,618][20381] Updated weights for policy 1, policy_version 1508610 (0.0008) [2023-12-27 16:06:34,673][20381] Updated weights for policy 1, policy_version 1508620 (0.0006) [2023-12-27 16:06:35,205][20381] Updated weights for policy 1, policy_version 1508630 (0.0006) [2023-12-27 16:06:35,258][20381] Updated weights for policy 1, policy_version 1508640 (0.0005) [2023-12-27 16:06:35,303][20347] Updated weights for policy 0, policy_version 1510240 (0.0010) [2023-12-27 16:06:35,316][20381] Updated weights for policy 1, policy_version 1508650 (0.0006) [2023-12-27 16:06:35,359][20347] Updated weights for policy 0, policy_version 1510250 (0.0009) [2023-12-27 16:06:35,416][20347] Updated weights for policy 0, policy_version 1510260 (0.0005) [2023-12-27 16:06:35,952][20381] Updated weights for policy 1, policy_version 1508660 (0.0005) [2023-12-27 16:06:36,001][20347] Updated weights for policy 0, policy_version 1510270 (0.0005) [2023-12-27 16:06:36,011][20381] Updated weights for policy 1, policy_version 1508670 (0.0007) [2023-12-27 16:06:36,054][20347] Updated weights for policy 0, policy_version 1510280 (0.0009) [2023-12-27 16:06:36,067][20381] Updated weights for policy 1, policy_version 1508680 (0.0006) [2023-12-27 16:06:36,110][20347] Updated weights for policy 0, policy_version 1510290 (0.0010) [2023-12-27 16:06:36,743][20381] Updated weights for policy 1, policy_version 1508690 (0.0008) [2023-12-27 16:06:36,788][20347] Updated weights for policy 0, policy_version 1510300 (0.0010) [2023-12-27 16:06:36,799][20381] Updated weights for policy 1, policy_version 1508700 (0.0008) [2023-12-27 16:06:36,844][20347] Updated weights for policy 0, policy_version 1510310 (0.0010) [2023-12-27 16:06:36,854][20381] Updated weights for policy 1, policy_version 1508710 (0.0008) [2023-12-27 16:06:36,900][20347] Updated weights for policy 0, policy_version 1510320 (0.0010) [2023-12-27 16:06:36,910][20381] Updated weights for policy 1, policy_version 1508720 (0.0005) [2023-12-27 16:06:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 772988928. Throughput: 0: 10326.6, 1: 10321.4. Samples: 772956888. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.820')] [2023-12-27 16:06:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001508720_386269184.pth... [2023-12-27 16:06:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001510328_386719744.pth... [2023-12-27 16:06:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001507504_385957888.pth [2023-12-27 16:06:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001509112_386408448.pth [2023-12-27 16:06:37,574][20347] Updated weights for policy 0, policy_version 1510330 (0.0010) [2023-12-27 16:06:37,602][20381] Updated weights for policy 1, policy_version 1508730 (0.0006) [2023-12-27 16:06:37,636][20347] Updated weights for policy 0, policy_version 1510340 (0.0010) [2023-12-27 16:06:37,658][20381] Updated weights for policy 1, policy_version 1508740 (0.0007) [2023-12-27 16:06:37,691][20347] Updated weights for policy 0, policy_version 1510350 (0.0010) [2023-12-27 16:06:37,718][20381] Updated weights for policy 1, policy_version 1508750 (0.0009) [2023-12-27 16:06:37,746][20347] Updated weights for policy 0, policy_version 1510360 (0.0010) [2023-12-27 16:06:38,350][20381] Updated weights for policy 1, policy_version 1508760 (0.0007) [2023-12-27 16:06:38,410][20381] Updated weights for policy 1, policy_version 1508770 (0.0006) [2023-12-27 16:06:38,433][20347] Updated weights for policy 0, policy_version 1510370 (0.0011) [2023-12-27 16:06:38,459][20291] KL-divergence is very high: 117.4507 [2023-12-27 16:06:38,465][20381] Updated weights for policy 1, policy_version 1508780 (0.0006) [2023-12-27 16:06:38,496][20347] Updated weights for policy 0, policy_version 1510380 (0.0010) [2023-12-27 16:06:38,510][20291] KL-divergence is very high: 110.1865 [2023-12-27 16:06:38,563][20347] Updated weights for policy 0, policy_version 1510390 (0.0011) [2023-12-27 16:06:39,183][20381] Updated weights for policy 1, policy_version 1508790 (0.0008) [2023-12-27 16:06:39,217][20347] Updated weights for policy 0, policy_version 1510400 (0.0011) [2023-12-27 16:06:39,248][20381] Updated weights for policy 1, policy_version 1508800 (0.0008) [2023-12-27 16:06:39,282][20347] Updated weights for policy 0, policy_version 1510410 (0.0010) [2023-12-27 16:06:39,311][20381] Updated weights for policy 1, policy_version 1508810 (0.0009) [2023-12-27 16:06:39,342][20347] Updated weights for policy 0, policy_version 1510420 (0.0009) [2023-12-27 16:06:40,094][20381] Updated weights for policy 1, policy_version 1508820 (0.0008) [2023-12-27 16:06:40,112][20347] Updated weights for policy 0, policy_version 1510430 (0.0007) [2023-12-27 16:06:40,150][20381] Updated weights for policy 1, policy_version 1508830 (0.0007) [2023-12-27 16:06:40,172][20347] Updated weights for policy 0, policy_version 1510440 (0.0006) [2023-12-27 16:06:40,204][20381] Updated weights for policy 1, policy_version 1508840 (0.0006) [2023-12-27 16:06:40,228][20347] Updated weights for policy 0, policy_version 1510450 (0.0007) [2023-12-27 16:06:40,874][20347] Updated weights for policy 0, policy_version 1510460 (0.0006) [2023-12-27 16:06:40,897][20381] Updated weights for policy 1, policy_version 1508850 (0.0005) [2023-12-27 16:06:40,926][20347] Updated weights for policy 0, policy_version 1510470 (0.0007) [2023-12-27 16:06:40,953][20381] Updated weights for policy 1, policy_version 1508860 (0.0006) [2023-12-27 16:06:40,986][20347] Updated weights for policy 0, policy_version 1510480 (0.0008) [2023-12-27 16:06:40,997][20381] Updated weights for policy 1, policy_version 1508870 (0.0005) [2023-12-27 16:06:41,058][20381] Updated weights for policy 1, policy_version 1508880 (0.0007) [2023-12-27 16:06:41,679][20347] Updated weights for policy 0, policy_version 1510490 (0.0008) [2023-12-27 16:06:41,752][20347] Updated weights for policy 0, policy_version 1510500 (0.0007) [2023-12-27 16:06:41,804][20347] Updated weights for policy 0, policy_version 1510510 (0.0008) [2023-12-27 16:06:41,805][20381] Updated weights for policy 1, policy_version 1508890 (0.0008) [2023-12-27 16:06:41,860][20381] Updated weights for policy 1, policy_version 1508900 (0.0008) [2023-12-27 16:06:41,861][20347] Updated weights for policy 0, policy_version 1510520 (0.0008) [2023-12-27 16:06:41,917][20381] Updated weights for policy 1, policy_version 1508910 (0.0008) [2023-12-27 16:06:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 773087232. Throughput: 0: 10343.2, 1: 10279.5. Samples: 773078416. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:42,363][19170] Avg episode reward: [(0, '980.440'), (1, '970.820')] [2023-12-27 16:06:42,587][20347] Updated weights for policy 0, policy_version 1510530 (0.0010) [2023-12-27 16:06:42,646][20347] Updated weights for policy 0, policy_version 1510540 (0.0010) [2023-12-27 16:06:42,648][20381] Updated weights for policy 1, policy_version 1508920 (0.0008) [2023-12-27 16:06:42,700][20381] Updated weights for policy 1, policy_version 1508930 (0.0006) [2023-12-27 16:06:42,701][20347] Updated weights for policy 0, policy_version 1510550 (0.0010) [2023-12-27 16:06:42,748][20381] Updated weights for policy 1, policy_version 1508940 (0.0007) [2023-12-27 16:06:43,333][20347] Updated weights for policy 0, policy_version 1510560 (0.0008) [2023-12-27 16:06:43,385][20347] Updated weights for policy 0, policy_version 1510570 (0.0010) [2023-12-27 16:06:43,403][20381] Updated weights for policy 1, policy_version 1508950 (0.0006) [2023-12-27 16:06:43,433][20347] Updated weights for policy 0, policy_version 1510580 (0.0007) [2023-12-27 16:06:43,457][20381] Updated weights for policy 1, policy_version 1508960 (0.0007) [2023-12-27 16:06:43,517][20381] Updated weights for policy 1, policy_version 1508970 (0.0008) [2023-12-27 16:06:44,041][20347] Updated weights for policy 0, policy_version 1510590 (0.0008) [2023-12-27 16:06:44,105][20347] Updated weights for policy 0, policy_version 1510600 (0.0010) [2023-12-27 16:06:44,150][20347] Updated weights for policy 0, policy_version 1510610 (0.0010) [2023-12-27 16:06:44,223][20381] Updated weights for policy 1, policy_version 1508980 (0.0008) [2023-12-27 16:06:44,279][20381] Updated weights for policy 1, policy_version 1508990 (0.0009) [2023-12-27 16:06:44,327][20381] Updated weights for policy 1, policy_version 1509000 (0.0007) [2023-12-27 16:06:44,737][20347] Updated weights for policy 0, policy_version 1510620 (0.0008) [2023-12-27 16:06:44,799][20347] Updated weights for policy 0, policy_version 1510630 (0.0007) [2023-12-27 16:06:44,854][20347] Updated weights for policy 0, policy_version 1510640 (0.0007) [2023-12-27 16:06:45,022][20381] Updated weights for policy 1, policy_version 1509010 (0.0006) [2023-12-27 16:06:45,076][20381] Updated weights for policy 1, policy_version 1509020 (0.0007) [2023-12-27 16:06:45,137][20381] Updated weights for policy 1, policy_version 1509030 (0.0007) [2023-12-27 16:06:45,201][20381] Updated weights for policy 1, policy_version 1509040 (0.0007) [2023-12-27 16:06:45,535][20347] Updated weights for policy 0, policy_version 1510650 (0.0008) [2023-12-27 16:06:45,598][20347] Updated weights for policy 0, policy_version 1510660 (0.0008) [2023-12-27 16:06:45,655][20347] Updated weights for policy 0, policy_version 1510670 (0.0008) [2023-12-27 16:06:45,702][20347] Updated weights for policy 0, policy_version 1510680 (0.0008) [2023-12-27 16:06:45,858][20381] Updated weights for policy 1, policy_version 1509050 (0.0007) [2023-12-27 16:06:45,919][20381] Updated weights for policy 1, policy_version 1509060 (0.0008) [2023-12-27 16:06:45,979][20381] Updated weights for policy 1, policy_version 1509070 (0.0008) [2023-12-27 16:06:46,344][20347] Updated weights for policy 0, policy_version 1510690 (0.0008) [2023-12-27 16:06:46,404][20347] Updated weights for policy 0, policy_version 1510700 (0.0007) [2023-12-27 16:06:46,453][20347] Updated weights for policy 0, policy_version 1510710 (0.0007) [2023-12-27 16:06:46,637][20381] Updated weights for policy 1, policy_version 1509080 (0.0006) [2023-12-27 16:06:46,695][20381] Updated weights for policy 1, policy_version 1509090 (0.0005) [2023-12-27 16:06:46,761][20381] Updated weights for policy 1, policy_version 1509100 (0.0008) [2023-12-27 16:06:47,133][20347] Updated weights for policy 0, policy_version 1510720 (0.0007) [2023-12-27 16:06:47,192][20347] Updated weights for policy 0, policy_version 1510730 (0.0008) [2023-12-27 16:06:47,250][20347] Updated weights for policy 0, policy_version 1510740 (0.0008) [2023-12-27 16:06:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 773193728. Throughput: 0: 10447.2, 1: 10323.7. Samples: 773202448. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:47,363][19170] Avg episode reward: [(0, '951.340'), (1, '980.510')] [2023-12-27 16:06:47,385][20381] Updated weights for policy 1, policy_version 1509110 (0.0008) [2023-12-27 16:06:47,446][20381] Updated weights for policy 1, policy_version 1509120 (0.0008) [2023-12-27 16:06:47,504][20381] Updated weights for policy 1, policy_version 1509130 (0.0008) [2023-12-27 16:06:47,923][20347] Updated weights for policy 0, policy_version 1510750 (0.0007) [2023-12-27 16:06:47,982][20347] Updated weights for policy 0, policy_version 1510760 (0.0006) [2023-12-27 16:06:48,046][20347] Updated weights for policy 0, policy_version 1510770 (0.0008) [2023-12-27 16:06:48,211][20381] Updated weights for policy 1, policy_version 1509140 (0.0008) [2023-12-27 16:06:48,256][20381] Updated weights for policy 1, policy_version 1509150 (0.0008) [2023-12-27 16:06:48,305][20381] Updated weights for policy 1, policy_version 1509160 (0.0007) [2023-12-27 16:06:48,692][20347] Updated weights for policy 0, policy_version 1510780 (0.0008) [2023-12-27 16:06:48,748][20347] Updated weights for policy 0, policy_version 1510790 (0.0008) [2023-12-27 16:06:48,798][20347] Updated weights for policy 0, policy_version 1510800 (0.0006) [2023-12-27 16:06:49,093][20381] Updated weights for policy 1, policy_version 1509170 (0.0009) [2023-12-27 16:06:49,153][20381] Updated weights for policy 1, policy_version 1509180 (0.0009) [2023-12-27 16:06:49,230][20381] Updated weights for policy 1, policy_version 1509190 (0.0007) [2023-12-27 16:06:49,290][20381] Updated weights for policy 1, policy_version 1509200 (0.0007) [2023-12-27 16:06:49,427][20347] Updated weights for policy 0, policy_version 1510810 (0.0005) [2023-12-27 16:06:49,486][20347] Updated weights for policy 0, policy_version 1510820 (0.0007) [2023-12-27 16:06:49,538][20347] Updated weights for policy 0, policy_version 1510830 (0.0008) [2023-12-27 16:06:49,590][20347] Updated weights for policy 0, policy_version 1510840 (0.0007) [2023-12-27 16:06:49,987][20381] Updated weights for policy 1, policy_version 1509210 (0.0007) [2023-12-27 16:06:50,047][20381] Updated weights for policy 1, policy_version 1509220 (0.0008) [2023-12-27 16:06:50,100][20381] Updated weights for policy 1, policy_version 1509230 (0.0010) [2023-12-27 16:06:50,328][20347] Updated weights for policy 0, policy_version 1510850 (0.0008) [2023-12-27 16:06:50,352][20291] KL-divergence is very high: 114.2071 [2023-12-27 16:06:50,375][20291] KL-divergence is very high: 154.3882 [2023-12-27 16:06:50,380][20347] Updated weights for policy 0, policy_version 1510860 (0.0008) [2023-12-27 16:06:50,396][20291] KL-divergence is very high: 100.6579 [2023-12-27 16:06:50,433][20347] Updated weights for policy 0, policy_version 1510870 (0.0008) [2023-12-27 16:06:50,747][20381] Updated weights for policy 1, policy_version 1509240 (0.0011) [2023-12-27 16:06:50,798][20381] Updated weights for policy 1, policy_version 1509250 (0.0010) [2023-12-27 16:06:50,856][20381] Updated weights for policy 1, policy_version 1509260 (0.0009) [2023-12-27 16:06:51,195][20347] Updated weights for policy 0, policy_version 1510880 (0.0008) [2023-12-27 16:06:51,258][20347] Updated weights for policy 0, policy_version 1510890 (0.0008) [2023-12-27 16:06:51,310][20347] Updated weights for policy 0, policy_version 1510900 (0.0007) [2023-12-27 16:06:51,621][20381] Updated weights for policy 1, policy_version 1509270 (0.0008) [2023-12-27 16:06:51,688][20381] Updated weights for policy 1, policy_version 1509280 (0.0011) [2023-12-27 16:06:51,755][20381] Updated weights for policy 1, policy_version 1509290 (0.0011) [2023-12-27 16:06:51,989][20347] Updated weights for policy 0, policy_version 1510910 (0.0007) [2023-12-27 16:06:52,049][20347] Updated weights for policy 0, policy_version 1510920 (0.0008) [2023-12-27 16:06:52,103][20347] Updated weights for policy 0, policy_version 1510930 (0.0008) [2023-12-27 16:06:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 773292032. Throughput: 0: 10428.0, 1: 10264.5. Samples: 773262704. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:52,363][19170] Avg episode reward: [(0, '931.780'), (1, '980.510')] [2023-12-27 16:06:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001510936_386875392.pth... [2023-12-27 16:06:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001509720_386564096.pth [2023-12-27 16:06:52,379][20381] Updated weights for policy 1, policy_version 1509300 (0.0008) [2023-12-27 16:06:52,439][20381] Updated weights for policy 1, policy_version 1509310 (0.0009) [2023-12-27 16:06:52,495][20381] Updated weights for policy 1, policy_version 1509320 (0.0008) [2023-12-27 16:06:52,543][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001509328_386424832.pth... [2023-12-27 16:06:52,547][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001508112_386113536.pth [2023-12-27 16:06:52,830][20347] Updated weights for policy 0, policy_version 1510940 (0.0007) [2023-12-27 16:06:52,886][20347] Updated weights for policy 0, policy_version 1510950 (0.0007) [2023-12-27 16:06:52,944][20347] Updated weights for policy 0, policy_version 1510960 (0.0007) [2023-12-27 16:06:53,188][20381] Updated weights for policy 1, policy_version 1509330 (0.0006) [2023-12-27 16:06:53,244][20381] Updated weights for policy 1, policy_version 1509340 (0.0005) [2023-12-27 16:06:53,289][20381] Updated weights for policy 1, policy_version 1509350 (0.0005) [2023-12-27 16:06:53,335][20381] Updated weights for policy 1, policy_version 1509360 (0.0006) [2023-12-27 16:06:53,539][20347] Updated weights for policy 0, policy_version 1510970 (0.0005) [2023-12-27 16:06:53,590][20347] Updated weights for policy 0, policy_version 1510980 (0.0006) [2023-12-27 16:06:53,634][20347] Updated weights for policy 0, policy_version 1510990 (0.0005) [2023-12-27 16:06:53,680][20347] Updated weights for policy 0, policy_version 1511000 (0.0005) [2023-12-27 16:06:53,968][20381] Updated weights for policy 1, policy_version 1509370 (0.0007) [2023-12-27 16:06:54,034][20381] Updated weights for policy 1, policy_version 1509380 (0.0005) [2023-12-27 16:06:54,080][20381] Updated weights for policy 1, policy_version 1509390 (0.0005) [2023-12-27 16:06:54,234][20347] Updated weights for policy 0, policy_version 1511010 (0.0009) [2023-12-27 16:06:54,286][20347] Updated weights for policy 0, policy_version 1511020 (0.0005) [2023-12-27 16:06:54,345][20347] Updated weights for policy 0, policy_version 1511030 (0.0010) [2023-12-27 16:06:54,651][20381] Updated weights for policy 1, policy_version 1509400 (0.0008) [2023-12-27 16:06:54,707][20381] Updated weights for policy 1, policy_version 1509410 (0.0005) [2023-12-27 16:06:54,769][20381] Updated weights for policy 1, policy_version 1509420 (0.0005) [2023-12-27 16:06:55,023][20347] Updated weights for policy 0, policy_version 1511040 (0.0007) [2023-12-27 16:06:55,078][20347] Updated weights for policy 0, policy_version 1511050 (0.0005) [2023-12-27 16:06:55,135][20347] Updated weights for policy 0, policy_version 1511060 (0.0005) [2023-12-27 16:06:55,373][20381] Updated weights for policy 1, policy_version 1509430 (0.0008) [2023-12-27 16:06:55,419][20381] Updated weights for policy 1, policy_version 1509440 (0.0010) [2023-12-27 16:06:55,464][20381] Updated weights for policy 1, policy_version 1509450 (0.0010) [2023-12-27 16:06:55,745][20347] Updated weights for policy 0, policy_version 1511070 (0.0006) [2023-12-27 16:06:55,800][20347] Updated weights for policy 0, policy_version 1511080 (0.0008) [2023-12-27 16:06:55,810][20291] KL-divergence is very high: 108.7607 [2023-12-27 16:06:55,817][20291] KL-divergence is very high: 111.8427 [2023-12-27 16:06:55,858][20347] Updated weights for policy 0, policy_version 1511090 (0.0007) [2023-12-27 16:06:56,125][20381] Updated weights for policy 1, policy_version 1509460 (0.0011) [2023-12-27 16:06:56,170][20381] Updated weights for policy 1, policy_version 1509470 (0.0011) [2023-12-27 16:06:56,215][20381] Updated weights for policy 1, policy_version 1509480 (0.0010) [2023-12-27 16:06:56,460][20347] Updated weights for policy 0, policy_version 1511100 (0.0008) [2023-12-27 16:06:56,512][20347] Updated weights for policy 0, policy_version 1511110 (0.0007) [2023-12-27 16:06:56,566][20347] Updated weights for policy 0, policy_version 1511120 (0.0008) [2023-12-27 16:06:56,900][20381] Updated weights for policy 1, policy_version 1509490 (0.0010) [2023-12-27 16:06:56,957][20381] Updated weights for policy 1, policy_version 1509500 (0.0007) [2023-12-27 16:06:57,014][20381] Updated weights for policy 1, policy_version 1509510 (0.0007) [2023-12-27 16:06:57,073][20381] Updated weights for policy 1, policy_version 1509520 (0.0007) [2023-12-27 16:06:57,266][20347] Updated weights for policy 0, policy_version 1511130 (0.0007) [2023-12-27 16:06:57,322][20347] Updated weights for policy 0, policy_version 1511140 (0.0007) [2023-12-27 16:06:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 773398528. Throughput: 0: 10458.5, 1: 10308.5. Samples: 773389628. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:06:57,363][19170] Avg episode reward: [(0, '863.270'), (1, '980.500')] [2023-12-27 16:06:57,381][20347] Updated weights for policy 0, policy_version 1511150 (0.0007) [2023-12-27 16:06:57,448][20347] Updated weights for policy 0, policy_version 1511160 (0.0009) [2023-12-27 16:06:57,674][20381] Updated weights for policy 1, policy_version 1509530 (0.0008) [2023-12-27 16:06:57,723][20381] Updated weights for policy 1, policy_version 1509540 (0.0008) [2023-12-27 16:06:57,785][20381] Updated weights for policy 1, policy_version 1509550 (0.0008) [2023-12-27 16:06:58,132][20347] Updated weights for policy 0, policy_version 1511170 (0.0007) [2023-12-27 16:06:58,146][20291] KL-divergence is very high: 113.2806 [2023-12-27 16:06:58,188][20347] Updated weights for policy 0, policy_version 1511180 (0.0007) [2023-12-27 16:06:58,249][20347] Updated weights for policy 0, policy_version 1511190 (0.0008) [2023-12-27 16:06:58,486][20381] Updated weights for policy 1, policy_version 1509560 (0.0008) [2023-12-27 16:06:58,556][20381] Updated weights for policy 1, policy_version 1509570 (0.0008) [2023-12-27 16:06:58,621][20381] Updated weights for policy 1, policy_version 1509580 (0.0008) [2023-12-27 16:06:59,080][20347] Updated weights for policy 0, policy_version 1511200 (0.0008) [2023-12-27 16:06:59,139][20347] Updated weights for policy 0, policy_version 1511210 (0.0007) [2023-12-27 16:06:59,210][20347] Updated weights for policy 0, policy_version 1511220 (0.0007) [2023-12-27 16:06:59,418][20381] Updated weights for policy 1, policy_version 1509590 (0.0011) [2023-12-27 16:06:59,469][20381] Updated weights for policy 1, policy_version 1509600 (0.0007) [2023-12-27 16:06:59,520][20381] Updated weights for policy 1, policy_version 1509610 (0.0008) [2023-12-27 16:06:59,859][20347] Updated weights for policy 0, policy_version 1511230 (0.0007) [2023-12-27 16:06:59,913][20347] Updated weights for policy 0, policy_version 1511240 (0.0007) [2023-12-27 16:06:59,976][20347] Updated weights for policy 0, policy_version 1511250 (0.0008) [2023-12-27 16:07:00,275][20381] Updated weights for policy 1, policy_version 1509620 (0.0008) [2023-12-27 16:07:00,324][20381] Updated weights for policy 1, policy_version 1509630 (0.0008) [2023-12-27 16:07:00,382][20381] Updated weights for policy 1, policy_version 1509640 (0.0008) [2023-12-27 16:07:00,621][20347] Updated weights for policy 0, policy_version 1511260 (0.0006) [2023-12-27 16:07:00,675][20347] Updated weights for policy 0, policy_version 1511270 (0.0007) [2023-12-27 16:07:00,726][20347] Updated weights for policy 0, policy_version 1511280 (0.0007) [2023-12-27 16:07:01,020][20381] Updated weights for policy 1, policy_version 1509650 (0.0008) [2023-12-27 16:07:01,085][20381] Updated weights for policy 1, policy_version 1509660 (0.0007) [2023-12-27 16:07:01,153][20381] Updated weights for policy 1, policy_version 1509670 (0.0008) [2023-12-27 16:07:01,197][20381] Updated weights for policy 1, policy_version 1509680 (0.0008) [2023-12-27 16:07:01,435][20347] Updated weights for policy 0, policy_version 1511290 (0.0007) [2023-12-27 16:07:01,490][20347] Updated weights for policy 0, policy_version 1511300 (0.0007) [2023-12-27 16:07:01,548][20347] Updated weights for policy 0, policy_version 1511310 (0.0008) [2023-12-27 16:07:01,606][20347] Updated weights for policy 0, policy_version 1511320 (0.0007) [2023-12-27 16:07:01,901][20381] Updated weights for policy 1, policy_version 1509690 (0.0006) [2023-12-27 16:07:01,963][20381] Updated weights for policy 1, policy_version 1509700 (0.0007) [2023-12-27 16:07:02,026][20381] Updated weights for policy 1, policy_version 1509710 (0.0006) [2023-12-27 16:07:02,298][20347] Updated weights for policy 0, policy_version 1511330 (0.0009) [2023-12-27 16:07:02,350][20347] Updated weights for policy 0, policy_version 1511340 (0.0009) [2023-12-27 16:07:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 773496832. Throughput: 0: 10368.4, 1: 10286.4. Samples: 773509532. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:02,363][19170] Avg episode reward: [(0, '902.230'), (1, '980.500')] [2023-12-27 16:07:02,411][20347] Updated weights for policy 0, policy_version 1511350 (0.0009) [2023-12-27 16:07:02,719][20381] Updated weights for policy 1, policy_version 1509720 (0.0007) [2023-12-27 16:07:02,774][20381] Updated weights for policy 1, policy_version 1509730 (0.0007) [2023-12-27 16:07:02,828][20381] Updated weights for policy 1, policy_version 1509740 (0.0007) [2023-12-27 16:07:03,152][20347] Updated weights for policy 0, policy_version 1511360 (0.0007) [2023-12-27 16:07:03,210][20347] Updated weights for policy 0, policy_version 1511370 (0.0007) [2023-12-27 16:07:03,252][20347] Updated weights for policy 0, policy_version 1511380 (0.0005) [2023-12-27 16:07:03,511][20381] Updated weights for policy 1, policy_version 1509750 (0.0008) [2023-12-27 16:07:03,571][20381] Updated weights for policy 1, policy_version 1509760 (0.0007) [2023-12-27 16:07:03,629][20381] Updated weights for policy 1, policy_version 1509770 (0.0007) [2023-12-27 16:07:03,844][20347] Updated weights for policy 0, policy_version 1511390 (0.0007) [2023-12-27 16:07:03,902][20347] Updated weights for policy 0, policy_version 1511400 (0.0008) [2023-12-27 16:07:03,955][20347] Updated weights for policy 0, policy_version 1511410 (0.0008) [2023-12-27 16:07:04,312][20381] Updated weights for policy 1, policy_version 1509780 (0.0008) [2023-12-27 16:07:04,373][20381] Updated weights for policy 1, policy_version 1509790 (0.0008) [2023-12-27 16:07:04,425][20381] Updated weights for policy 1, policy_version 1509800 (0.0008) [2023-12-27 16:07:04,668][20347] Updated weights for policy 0, policy_version 1511420 (0.0007) [2023-12-27 16:07:04,720][20347] Updated weights for policy 0, policy_version 1511430 (0.0007) [2023-12-27 16:07:04,766][20347] Updated weights for policy 0, policy_version 1511440 (0.0005) [2023-12-27 16:07:05,166][20381] Updated weights for policy 1, policy_version 1509810 (0.0007) [2023-12-27 16:07:05,222][20381] Updated weights for policy 1, policy_version 1509820 (0.0008) [2023-12-27 16:07:05,267][20381] Updated weights for policy 1, policy_version 1509830 (0.0007) [2023-12-27 16:07:05,309][20381] Updated weights for policy 1, policy_version 1509840 (0.0008) [2023-12-27 16:07:05,418][20347] Updated weights for policy 0, policy_version 1511450 (0.0006) [2023-12-27 16:07:05,467][20347] Updated weights for policy 0, policy_version 1511460 (0.0007) [2023-12-27 16:07:05,516][20347] Updated weights for policy 0, policy_version 1511470 (0.0008) [2023-12-27 16:07:05,567][20347] Updated weights for policy 0, policy_version 1511480 (0.0006) [2023-12-27 16:07:06,024][20381] Updated weights for policy 1, policy_version 1509850 (0.0008) [2023-12-27 16:07:06,081][20381] Updated weights for policy 1, policy_version 1509860 (0.0008) [2023-12-27 16:07:06,146][20381] Updated weights for policy 1, policy_version 1509870 (0.0009) [2023-12-27 16:07:06,183][20347] Updated weights for policy 0, policy_version 1511490 (0.0008) [2023-12-27 16:07:06,241][20347] Updated weights for policy 0, policy_version 1511500 (0.0008) [2023-12-27 16:07:06,300][20347] Updated weights for policy 0, policy_version 1511510 (0.0008) [2023-12-27 16:07:06,878][20381] Updated weights for policy 1, policy_version 1509880 (0.0007) [2023-12-27 16:07:06,929][20381] Updated weights for policy 1, policy_version 1509890 (0.0008) [2023-12-27 16:07:06,949][20347] Updated weights for policy 0, policy_version 1511520 (0.0009) [2023-12-27 16:07:06,981][20381] Updated weights for policy 1, policy_version 1509900 (0.0006) [2023-12-27 16:07:06,995][20347] Updated weights for policy 0, policy_version 1511530 (0.0005) [2023-12-27 16:07:07,047][20347] Updated weights for policy 0, policy_version 1511540 (0.0006) [2023-12-27 16:07:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 773603328. Throughput: 0: 10362.5, 1: 10260.1. Samples: 773569816. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:07,364][19170] Avg episode reward: [(0, '931.490'), (1, '990.270')] [2023-12-27 16:07:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001509904_386572288.pth... [2023-12-27 16:07:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001511544_387031040.pth... [2023-12-27 16:07:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001508720_386269184.pth [2023-12-27 16:07:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001510328_386719744.pth [2023-12-27 16:07:07,580][20381] Updated weights for policy 1, policy_version 1509910 (0.0006) [2023-12-27 16:07:07,638][20381] Updated weights for policy 1, policy_version 1509920 (0.0006) [2023-12-27 16:07:07,695][20381] Updated weights for policy 1, policy_version 1509930 (0.0006) [2023-12-27 16:07:07,796][20347] Updated weights for policy 0, policy_version 1511550 (0.0008) [2023-12-27 16:07:07,856][20347] Updated weights for policy 0, policy_version 1511560 (0.0009) [2023-12-27 16:07:07,908][20347] Updated weights for policy 0, policy_version 1511570 (0.0008) [2023-12-27 16:07:08,237][20381] Updated weights for policy 1, policy_version 1509940 (0.0007) [2023-12-27 16:07:08,286][20381] Updated weights for policy 1, policy_version 1509950 (0.0008) [2023-12-27 16:07:08,349][20381] Updated weights for policy 1, policy_version 1509960 (0.0009) [2023-12-27 16:07:08,638][20347] Updated weights for policy 0, policy_version 1511580 (0.0007) [2023-12-27 16:07:08,697][20347] Updated weights for policy 0, policy_version 1511590 (0.0008) [2023-12-27 16:07:08,755][20347] Updated weights for policy 0, policy_version 1511600 (0.0008) [2023-12-27 16:07:09,069][20381] Updated weights for policy 1, policy_version 1509970 (0.0008) [2023-12-27 16:07:09,116][20381] Updated weights for policy 1, policy_version 1509980 (0.0010) [2023-12-27 16:07:09,164][20381] Updated weights for policy 1, policy_version 1509990 (0.0010) [2023-12-27 16:07:09,219][20381] Updated weights for policy 1, policy_version 1510000 (0.0009) [2023-12-27 16:07:09,530][20347] Updated weights for policy 0, policy_version 1511610 (0.0007) [2023-12-27 16:07:09,588][20347] Updated weights for policy 0, policy_version 1511620 (0.0007) [2023-12-27 16:07:09,650][20347] Updated weights for policy 0, policy_version 1511630 (0.0008) [2023-12-27 16:07:09,705][20347] Updated weights for policy 0, policy_version 1511640 (0.0008) [2023-12-27 16:07:09,990][20381] Updated weights for policy 1, policy_version 1510010 (0.0008) [2023-12-27 16:07:10,049][20381] Updated weights for policy 1, policy_version 1510020 (0.0008) [2023-12-27 16:07:10,113][20381] Updated weights for policy 1, policy_version 1510030 (0.0007) [2023-12-27 16:07:10,486][20347] Updated weights for policy 0, policy_version 1511650 (0.0008) [2023-12-27 16:07:10,546][20347] Updated weights for policy 0, policy_version 1511660 (0.0008) [2023-12-27 16:07:10,604][20347] Updated weights for policy 0, policy_version 1511670 (0.0007) [2023-12-27 16:07:10,789][20381] Updated weights for policy 1, policy_version 1510040 (0.0006) [2023-12-27 16:07:10,839][20381] Updated weights for policy 1, policy_version 1510050 (0.0005) [2023-12-27 16:07:10,888][20381] Updated weights for policy 1, policy_version 1510060 (0.0006) [2023-12-27 16:07:11,322][20347] Updated weights for policy 0, policy_version 1511680 (0.0007) [2023-12-27 16:07:11,390][20347] Updated weights for policy 0, policy_version 1511690 (0.0008) [2023-12-27 16:07:11,444][20347] Updated weights for policy 0, policy_version 1511700 (0.0006) [2023-12-27 16:07:11,595][20381] Updated weights for policy 1, policy_version 1510070 (0.0009) [2023-12-27 16:07:11,662][20381] Updated weights for policy 1, policy_version 1510080 (0.0008) [2023-12-27 16:07:11,721][20381] Updated weights for policy 1, policy_version 1510090 (0.0008) [2023-12-27 16:07:12,175][20347] Updated weights for policy 0, policy_version 1511710 (0.0007) [2023-12-27 16:07:12,231][20347] Updated weights for policy 0, policy_version 1511720 (0.0008) [2023-12-27 16:07:12,291][20347] Updated weights for policy 0, policy_version 1511730 (0.0008) [2023-12-27 16:07:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 773701632. Throughput: 0: 10261.4, 1: 10239.3. Samples: 773690452. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:12,363][19170] Avg episode reward: [(0, '951.040'), (1, '980.540')] [2023-12-27 16:07:12,441][20381] Updated weights for policy 1, policy_version 1510100 (0.0008) [2023-12-27 16:07:12,486][20381] Updated weights for policy 1, policy_version 1510110 (0.0005) [2023-12-27 16:07:12,541][20381] Updated weights for policy 1, policy_version 1510120 (0.0005) [2023-12-27 16:07:12,970][20347] Updated weights for policy 0, policy_version 1511740 (0.0008) [2023-12-27 16:07:13,015][20347] Updated weights for policy 0, policy_version 1511750 (0.0007) [2023-12-27 16:07:13,063][20347] Updated weights for policy 0, policy_version 1511760 (0.0007) [2023-12-27 16:07:13,139][20381] Updated weights for policy 1, policy_version 1510130 (0.0007) [2023-12-27 16:07:13,186][20381] Updated weights for policy 1, policy_version 1510140 (0.0006) [2023-12-27 16:07:13,233][20381] Updated weights for policy 1, policy_version 1510150 (0.0008) [2023-12-27 16:07:13,275][20381] Updated weights for policy 1, policy_version 1510160 (0.0008) [2023-12-27 16:07:13,740][20347] Updated weights for policy 0, policy_version 1511770 (0.0007) [2023-12-27 16:07:13,788][20347] Updated weights for policy 0, policy_version 1511780 (0.0007) [2023-12-27 16:07:13,841][20347] Updated weights for policy 0, policy_version 1511790 (0.0008) [2023-12-27 16:07:13,898][20347] Updated weights for policy 0, policy_version 1511800 (0.0007) [2023-12-27 16:07:13,993][20381] Updated weights for policy 1, policy_version 1510170 (0.0008) [2023-12-27 16:07:14,051][20381] Updated weights for policy 1, policy_version 1510180 (0.0007) [2023-12-27 16:07:14,106][20381] Updated weights for policy 1, policy_version 1510190 (0.0008) [2023-12-27 16:07:14,530][20347] Updated weights for policy 0, policy_version 1511810 (0.0007) [2023-12-27 16:07:14,585][20347] Updated weights for policy 0, policy_version 1511820 (0.0008) [2023-12-27 16:07:14,639][20347] Updated weights for policy 0, policy_version 1511830 (0.0007) [2023-12-27 16:07:14,821][20381] Updated weights for policy 1, policy_version 1510200 (0.0008) [2023-12-27 16:07:14,881][20381] Updated weights for policy 1, policy_version 1510210 (0.0008) [2023-12-27 16:07:14,943][20381] Updated weights for policy 1, policy_version 1510220 (0.0007) [2023-12-27 16:07:15,335][20347] Updated weights for policy 0, policy_version 1511840 (0.0006) [2023-12-27 16:07:15,393][20347] Updated weights for policy 0, policy_version 1511850 (0.0006) [2023-12-27 16:07:15,478][20347] Updated weights for policy 0, policy_version 1511860 (0.0007) [2023-12-27 16:07:15,643][20381] Updated weights for policy 1, policy_version 1510230 (0.0008) [2023-12-27 16:07:15,700][20381] Updated weights for policy 1, policy_version 1510240 (0.0007) [2023-12-27 16:07:15,753][20381] Updated weights for policy 1, policy_version 1510250 (0.0008) [2023-12-27 16:07:16,111][20347] Updated weights for policy 0, policy_version 1511870 (0.0007) [2023-12-27 16:07:16,159][20347] Updated weights for policy 0, policy_version 1511880 (0.0007) [2023-12-27 16:07:16,211][20347] Updated weights for policy 0, policy_version 1511890 (0.0007) [2023-12-27 16:07:16,474][20381] Updated weights for policy 1, policy_version 1510260 (0.0008) [2023-12-27 16:07:16,522][20381] Updated weights for policy 1, policy_version 1510270 (0.0008) [2023-12-27 16:07:16,581][20381] Updated weights for policy 1, policy_version 1510280 (0.0008) [2023-12-27 16:07:16,834][20347] Updated weights for policy 0, policy_version 1511900 (0.0007) [2023-12-27 16:07:16,882][20347] Updated weights for policy 0, policy_version 1511910 (0.0008) [2023-12-27 16:07:16,936][20347] Updated weights for policy 0, policy_version 1511920 (0.0007) [2023-12-27 16:07:17,251][20381] Updated weights for policy 1, policy_version 1510290 (0.0007) [2023-12-27 16:07:17,298][20381] Updated weights for policy 1, policy_version 1510300 (0.0005) [2023-12-27 16:07:17,343][20381] Updated weights for policy 1, policy_version 1510310 (0.0005) [2023-12-27 16:07:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 773799936. Throughput: 0: 10280.4, 1: 10170.0. Samples: 773812432. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:17,363][19170] Avg episode reward: [(0, '931.520'), (1, '980.630')] [2023-12-27 16:07:17,400][20381] Updated weights for policy 1, policy_version 1510320 (0.0010) [2023-12-27 16:07:17,671][20347] Updated weights for policy 0, policy_version 1511930 (0.0007) [2023-12-27 16:07:17,723][20347] Updated weights for policy 0, policy_version 1511940 (0.0008) [2023-12-27 16:07:17,775][20347] Updated weights for policy 0, policy_version 1511950 (0.0008) [2023-12-27 16:07:17,824][20347] Updated weights for policy 0, policy_version 1511960 (0.0007) [2023-12-27 16:07:17,995][20381] Updated weights for policy 1, policy_version 1510330 (0.0007) [2023-12-27 16:07:18,053][20381] Updated weights for policy 1, policy_version 1510340 (0.0008) [2023-12-27 16:07:18,113][20381] Updated weights for policy 1, policy_version 1510350 (0.0007) [2023-12-27 16:07:18,536][20347] Updated weights for policy 0, policy_version 1511970 (0.0008) [2023-12-27 16:07:18,600][20347] Updated weights for policy 0, policy_version 1511980 (0.0007) [2023-12-27 16:07:18,657][20347] Updated weights for policy 0, policy_version 1511990 (0.0007) [2023-12-27 16:07:18,822][20381] Updated weights for policy 1, policy_version 1510360 (0.0008) [2023-12-27 16:07:18,877][20381] Updated weights for policy 1, policy_version 1510370 (0.0009) [2023-12-27 16:07:18,938][20381] Updated weights for policy 1, policy_version 1510380 (0.0008) [2023-12-27 16:07:19,378][20347] Updated weights for policy 0, policy_version 1512000 (0.0008) [2023-12-27 16:07:19,426][20347] Updated weights for policy 0, policy_version 1512010 (0.0008) [2023-12-27 16:07:19,488][20347] Updated weights for policy 0, policy_version 1512020 (0.0007) [2023-12-27 16:07:19,645][20381] Updated weights for policy 1, policy_version 1510390 (0.0008) [2023-12-27 16:07:19,699][20381] Updated weights for policy 1, policy_version 1510400 (0.0008) [2023-12-27 16:07:19,756][20381] Updated weights for policy 1, policy_version 1510410 (0.0008) [2023-12-27 16:07:20,228][20347] Updated weights for policy 0, policy_version 1512030 (0.0010) [2023-12-27 16:07:20,283][20347] Updated weights for policy 0, policy_version 1512040 (0.0010) [2023-12-27 16:07:20,339][20347] Updated weights for policy 0, policy_version 1512050 (0.0010) [2023-12-27 16:07:20,517][20381] Updated weights for policy 1, policy_version 1510420 (0.0009) [2023-12-27 16:07:20,581][20381] Updated weights for policy 1, policy_version 1510430 (0.0007) [2023-12-27 16:07:20,644][20381] Updated weights for policy 1, policy_version 1510440 (0.0011) [2023-12-27 16:07:21,037][20347] Updated weights for policy 0, policy_version 1512060 (0.0010) [2023-12-27 16:07:21,103][20347] Updated weights for policy 0, policy_version 1512070 (0.0010) [2023-12-27 16:07:21,160][20347] Updated weights for policy 0, policy_version 1512080 (0.0010) [2023-12-27 16:07:21,414][20381] Updated weights for policy 1, policy_version 1510450 (0.0009) [2023-12-27 16:07:21,473][20381] Updated weights for policy 1, policy_version 1510460 (0.0011) [2023-12-27 16:07:21,531][20381] Updated weights for policy 1, policy_version 1510470 (0.0007) [2023-12-27 16:07:21,589][20381] Updated weights for policy 1, policy_version 1510480 (0.0008) [2023-12-27 16:07:21,920][20347] Updated weights for policy 0, policy_version 1512090 (0.0007) [2023-12-27 16:07:21,978][20347] Updated weights for policy 0, policy_version 1512100 (0.0009) [2023-12-27 16:07:22,035][20347] Updated weights for policy 0, policy_version 1512110 (0.0006) [2023-12-27 16:07:22,090][20347] Updated weights for policy 0, policy_version 1512120 (0.0006) [2023-12-27 16:07:22,336][20381] Updated weights for policy 1, policy_version 1510490 (0.0011) [2023-12-27 16:07:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 773898240. Throughput: 0: 10231.2, 1: 10119.7. Samples: 773872680. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:22,363][19170] Avg episode reward: [(0, '882.760'), (1, '990.360')] [2023-12-27 16:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001512120_387178496.pth... [2023-12-27 16:07:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001510936_386875392.pth [2023-12-27 16:07:22,396][20381] Updated weights for policy 1, policy_version 1510500 (0.0011) [2023-12-27 16:07:22,454][20381] Updated weights for policy 1, policy_version 1510511 (0.0010) [2023-12-27 16:07:22,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001510512_386727936.pth... [2023-12-27 16:07:22,461][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001509328_386424832.pth [2023-12-27 16:07:22,739][20347] Updated weights for policy 0, policy_version 1512130 (0.0008) [2023-12-27 16:07:22,798][20347] Updated weights for policy 0, policy_version 1512140 (0.0007) [2023-12-27 16:07:22,859][20347] Updated weights for policy 0, policy_version 1512150 (0.0008) [2023-12-27 16:07:23,212][20381] Updated weights for policy 1, policy_version 1510521 (0.0008) [2023-12-27 16:07:23,258][20381] Updated weights for policy 1, policy_version 1510531 (0.0005) [2023-12-27 16:07:23,315][20381] Updated weights for policy 1, policy_version 1510541 (0.0005) [2023-12-27 16:07:23,458][20347] Updated weights for policy 0, policy_version 1512160 (0.0009) [2023-12-27 16:07:23,523][20347] Updated weights for policy 0, policy_version 1512170 (0.0005) [2023-12-27 16:07:23,584][20347] Updated weights for policy 0, policy_version 1512180 (0.0005) [2023-12-27 16:07:23,877][20381] Updated weights for policy 1, policy_version 1510551 (0.0005) [2023-12-27 16:07:23,933][20381] Updated weights for policy 1, policy_version 1510561 (0.0008) [2023-12-27 16:07:23,983][20381] Updated weights for policy 1, policy_version 1510571 (0.0008) [2023-12-27 16:07:24,101][20347] Updated weights for policy 0, policy_version 1512190 (0.0008) [2023-12-27 16:07:24,162][20347] Updated weights for policy 0, policy_version 1512200 (0.0010) [2023-12-27 16:07:24,226][20347] Updated weights for policy 0, policy_version 1512210 (0.0011) [2023-12-27 16:07:24,547][20381] Updated weights for policy 1, policy_version 1510581 (0.0008) [2023-12-27 16:07:24,603][20381] Updated weights for policy 1, policy_version 1510591 (0.0007) [2023-12-27 16:07:24,650][20381] Updated weights for policy 1, policy_version 1510601 (0.0006) [2023-12-27 16:07:24,912][20347] Updated weights for policy 0, policy_version 1512220 (0.0011) [2023-12-27 16:07:24,970][20347] Updated weights for policy 0, policy_version 1512230 (0.0007) [2023-12-27 16:07:25,026][20347] Updated weights for policy 0, policy_version 1512240 (0.0007) [2023-12-27 16:07:25,216][20381] Updated weights for policy 1, policy_version 1510611 (0.0005) [2023-12-27 16:07:25,266][20381] Updated weights for policy 1, policy_version 1510621 (0.0007) [2023-12-27 16:07:25,322][20381] Updated weights for policy 1, policy_version 1510631 (0.0007) [2023-12-27 16:07:25,675][20347] Updated weights for policy 0, policy_version 1512250 (0.0010) [2023-12-27 16:07:25,729][20347] Updated weights for policy 0, policy_version 1512260 (0.0010) [2023-12-27 16:07:25,784][20347] Updated weights for policy 0, policy_version 1512270 (0.0010) [2023-12-27 16:07:25,834][20347] Updated weights for policy 0, policy_version 1512280 (0.0007) [2023-12-27 16:07:25,908][20381] Updated weights for policy 1, policy_version 1510641 (0.0007) [2023-12-27 16:07:25,973][20381] Updated weights for policy 1, policy_version 1510651 (0.0008) [2023-12-27 16:07:26,034][20381] Updated weights for policy 1, policy_version 1510661 (0.0011) [2023-12-27 16:07:26,106][20381] Updated weights for policy 1, policy_version 1510671 (0.0006) [2023-12-27 16:07:26,477][20347] Updated weights for policy 0, policy_version 1512290 (0.0005) [2023-12-27 16:07:26,539][20347] Updated weights for policy 0, policy_version 1512300 (0.0005) [2023-12-27 16:07:26,596][20347] Updated weights for policy 0, policy_version 1512310 (0.0007) [2023-12-27 16:07:26,615][20381] Updated weights for policy 1, policy_version 1510681 (0.0008) [2023-12-27 16:07:26,662][20381] Updated weights for policy 1, policy_version 1510691 (0.0007) [2023-12-27 16:07:26,708][20381] Updated weights for policy 1, policy_version 1510701 (0.0007) [2023-12-27 16:07:27,214][20347] Updated weights for policy 0, policy_version 1512320 (0.0010) [2023-12-27 16:07:27,281][20347] Updated weights for policy 0, policy_version 1512330 (0.0009) [2023-12-27 16:07:27,343][20347] Updated weights for policy 0, policy_version 1512340 (0.0009) [2023-12-27 16:07:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 774004736. Throughput: 0: 10265.8, 1: 10206.9. Samples: 773999692. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:27,364][19170] Avg episode reward: [(0, '911.950'), (1, '970.820')] [2023-12-27 16:07:27,408][20381] Updated weights for policy 1, policy_version 1510711 (0.0007) [2023-12-27 16:07:27,477][20381] Updated weights for policy 1, policy_version 1510721 (0.0007) [2023-12-27 16:07:27,525][20381] Updated weights for policy 1, policy_version 1510731 (0.0007) [2023-12-27 16:07:27,939][20347] Updated weights for policy 0, policy_version 1512350 (0.0006) [2023-12-27 16:07:27,996][20347] Updated weights for policy 0, policy_version 1512360 (0.0005) [2023-12-27 16:07:28,052][20347] Updated weights for policy 0, policy_version 1512370 (0.0010) [2023-12-27 16:07:28,226][20381] Updated weights for policy 1, policy_version 1510741 (0.0008) [2023-12-27 16:07:28,276][20381] Updated weights for policy 1, policy_version 1510751 (0.0008) [2023-12-27 16:07:28,332][20381] Updated weights for policy 1, policy_version 1510761 (0.0008) [2023-12-27 16:07:28,647][20347] Updated weights for policy 0, policy_version 1512380 (0.0008) [2023-12-27 16:07:28,710][20347] Updated weights for policy 0, policy_version 1512390 (0.0008) [2023-12-27 16:07:28,761][20347] Updated weights for policy 0, policy_version 1512400 (0.0007) [2023-12-27 16:07:29,018][20381] Updated weights for policy 1, policy_version 1510771 (0.0008) [2023-12-27 16:07:29,065][20381] Updated weights for policy 1, policy_version 1510781 (0.0007) [2023-12-27 16:07:29,120][20381] Updated weights for policy 1, policy_version 1510791 (0.0007) [2023-12-27 16:07:29,470][20347] Updated weights for policy 0, policy_version 1512410 (0.0007) [2023-12-27 16:07:29,518][20347] Updated weights for policy 0, policy_version 1512420 (0.0007) [2023-12-27 16:07:29,571][20347] Updated weights for policy 0, policy_version 1512430 (0.0007) [2023-12-27 16:07:29,618][20347] Updated weights for policy 0, policy_version 1512440 (0.0009) [2023-12-27 16:07:29,804][20381] Updated weights for policy 1, policy_version 1510801 (0.0007) [2023-12-27 16:07:29,866][20381] Updated weights for policy 1, policy_version 1510811 (0.0009) [2023-12-27 16:07:29,929][20381] Updated weights for policy 1, policy_version 1510821 (0.0010) [2023-12-27 16:07:29,990][20381] Updated weights for policy 1, policy_version 1510831 (0.0007) [2023-12-27 16:07:30,244][20347] Updated weights for policy 0, policy_version 1512450 (0.0008) [2023-12-27 16:07:30,306][20347] Updated weights for policy 0, policy_version 1512460 (0.0008) [2023-12-27 16:07:30,359][20347] Updated weights for policy 0, policy_version 1512470 (0.0007) [2023-12-27 16:07:30,656][20381] Updated weights for policy 1, policy_version 1510841 (0.0006) [2023-12-27 16:07:30,717][20381] Updated weights for policy 1, policy_version 1510851 (0.0005) [2023-12-27 16:07:30,770][20381] Updated weights for policy 1, policy_version 1510861 (0.0005) [2023-12-27 16:07:30,961][20347] Updated weights for policy 0, policy_version 1512480 (0.0006) [2023-12-27 16:07:30,991][20291] KL-divergence is very high: 137.7940 [2023-12-27 16:07:30,996][20291] KL-divergence is very high: 138.5121 [2023-12-27 16:07:31,005][20291] KL-divergence is very high: 160.2622 [2023-12-27 16:07:31,015][20347] Updated weights for policy 0, policy_version 1512490 (0.0005) [2023-12-27 16:07:31,080][20347] Updated weights for policy 0, policy_version 1512500 (0.0007) [2023-12-27 16:07:31,425][20381] Updated weights for policy 1, policy_version 1510871 (0.0008) [2023-12-27 16:07:31,475][20381] Updated weights for policy 1, policy_version 1510881 (0.0010) [2023-12-27 16:07:31,530][20381] Updated weights for policy 1, policy_version 1510891 (0.0010) [2023-12-27 16:07:31,799][20347] Updated weights for policy 0, policy_version 1512510 (0.0007) [2023-12-27 16:07:31,849][20291] KL-divergence is very high: 139.5563 [2023-12-27 16:07:31,862][20347] Updated weights for policy 0, policy_version 1512520 (0.0008) [2023-12-27 16:07:31,926][20347] Updated weights for policy 0, policy_version 1512530 (0.0006) [2023-12-27 16:07:32,210][20381] Updated weights for policy 1, policy_version 1510901 (0.0010) [2023-12-27 16:07:32,270][20381] Updated weights for policy 1, policy_version 1510911 (0.0010) [2023-12-27 16:07:32,327][20381] Updated weights for policy 1, policy_version 1510921 (0.0011) [2023-12-27 16:07:32,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 774111232. Throughput: 0: 10285.7, 1: 10248.9. Samples: 774126504. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:32,363][19170] Avg episode reward: [(0, '921.670'), (1, '951.280')] [2023-12-27 16:07:32,589][20347] Updated weights for policy 0, policy_version 1512540 (0.0006) [2023-12-27 16:07:32,647][20347] Updated weights for policy 0, policy_version 1512550 (0.0008) [2023-12-27 16:07:32,702][20347] Updated weights for policy 0, policy_version 1512560 (0.0007) [2023-12-27 16:07:33,013][20381] Updated weights for policy 1, policy_version 1510931 (0.0008) [2023-12-27 16:07:33,068][20381] Updated weights for policy 1, policy_version 1510941 (0.0011) [2023-12-27 16:07:33,120][20381] Updated weights for policy 1, policy_version 1510951 (0.0010) [2023-12-27 16:07:33,400][20347] Updated weights for policy 0, policy_version 1512570 (0.0008) [2023-12-27 16:07:33,456][20347] Updated weights for policy 0, policy_version 1512580 (0.0008) [2023-12-27 16:07:33,505][20347] Updated weights for policy 0, policy_version 1512590 (0.0007) [2023-12-27 16:07:33,552][20347] Updated weights for policy 0, policy_version 1512600 (0.0008) [2023-12-27 16:07:33,739][20381] Updated weights for policy 1, policy_version 1510961 (0.0010) [2023-12-27 16:07:33,802][20381] Updated weights for policy 1, policy_version 1510971 (0.0006) [2023-12-27 16:07:33,852][20381] Updated weights for policy 1, policy_version 1510981 (0.0008) [2023-12-27 16:07:33,906][20381] Updated weights for policy 1, policy_version 1510991 (0.0010) [2023-12-27 16:07:34,255][20347] Updated weights for policy 0, policy_version 1512610 (0.0008) [2023-12-27 16:07:34,311][20347] Updated weights for policy 0, policy_version 1512620 (0.0007) [2023-12-27 16:07:34,370][20347] Updated weights for policy 0, policy_version 1512630 (0.0007) [2023-12-27 16:07:34,535][20381] Updated weights for policy 1, policy_version 1511001 (0.0008) [2023-12-27 16:07:34,599][20381] Updated weights for policy 1, policy_version 1511011 (0.0009) [2023-12-27 16:07:34,664][20381] Updated weights for policy 1, policy_version 1511021 (0.0009) [2023-12-27 16:07:35,026][20347] Updated weights for policy 0, policy_version 1512640 (0.0005) [2023-12-27 16:07:35,083][20347] Updated weights for policy 0, policy_version 1512650 (0.0005) [2023-12-27 16:07:35,138][20347] Updated weights for policy 0, policy_version 1512660 (0.0006) [2023-12-27 16:07:35,324][20381] Updated weights for policy 1, policy_version 1511031 (0.0009) [2023-12-27 16:07:35,388][20381] Updated weights for policy 1, policy_version 1511041 (0.0010) [2023-12-27 16:07:35,446][20381] Updated weights for policy 1, policy_version 1511051 (0.0010) [2023-12-27 16:07:35,749][20347] Updated weights for policy 0, policy_version 1512670 (0.0008) [2023-12-27 16:07:35,797][20347] Updated weights for policy 0, policy_version 1512680 (0.0010) [2023-12-27 16:07:35,849][20347] Updated weights for policy 0, policy_version 1512690 (0.0010) [2023-12-27 16:07:36,019][20381] Updated weights for policy 1, policy_version 1511061 (0.0008) [2023-12-27 16:07:36,078][20381] Updated weights for policy 1, policy_version 1511071 (0.0005) [2023-12-27 16:07:36,143][20381] Updated weights for policy 1, policy_version 1511081 (0.0007) [2023-12-27 16:07:36,512][20347] Updated weights for policy 0, policy_version 1512700 (0.0008) [2023-12-27 16:07:36,578][20347] Updated weights for policy 0, policy_version 1512710 (0.0011) [2023-12-27 16:07:36,638][20347] Updated weights for policy 0, policy_version 1512720 (0.0011) [2023-12-27 16:07:36,806][20381] Updated weights for policy 1, policy_version 1511091 (0.0007) [2023-12-27 16:07:36,868][20381] Updated weights for policy 1, policy_version 1511101 (0.0007) [2023-12-27 16:07:36,930][20381] Updated weights for policy 1, policy_version 1511111 (0.0010) [2023-12-27 16:07:37,242][20347] Updated weights for policy 0, policy_version 1512730 (0.0010) [2023-12-27 16:07:37,301][20347] Updated weights for policy 0, policy_version 1512740 (0.0006) [2023-12-27 16:07:37,352][20347] Updated weights for policy 0, policy_version 1512750 (0.0007) [2023-12-27 16:07:37,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 774217728. Throughput: 0: 10279.5, 1: 10305.4. Samples: 774189024. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:37,363][19170] Avg episode reward: [(0, '902.200'), (1, '961.040')] [2023-12-27 16:07:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001511120_386883584.pth... [2023-12-27 16:07:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001509904_386572288.pth [2023-12-27 16:07:37,412][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001512760_387342336.pth... [2023-12-27 16:07:37,415][20347] Updated weights for policy 0, policy_version 1512760 (0.0009) [2023-12-27 16:07:37,417][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001511544_387031040.pth [2023-12-27 16:07:37,605][20381] Updated weights for policy 1, policy_version 1511121 (0.0008) [2023-12-27 16:07:37,669][20381] Updated weights for policy 1, policy_version 1511131 (0.0007) [2023-12-27 16:07:37,729][20381] Updated weights for policy 1, policy_version 1511141 (0.0006) [2023-12-27 16:07:37,781][20381] Updated weights for policy 1, policy_version 1511151 (0.0005) [2023-12-27 16:07:38,066][20347] Updated weights for policy 0, policy_version 1512770 (0.0008) [2023-12-27 16:07:38,119][20347] Updated weights for policy 0, policy_version 1512780 (0.0005) [2023-12-27 16:07:38,188][20347] Updated weights for policy 0, policy_version 1512790 (0.0005) [2023-12-27 16:07:38,370][20381] Updated weights for policy 1, policy_version 1511161 (0.0009) [2023-12-27 16:07:38,430][20381] Updated weights for policy 1, policy_version 1511171 (0.0008) [2023-12-27 16:07:38,491][20381] Updated weights for policy 1, policy_version 1511181 (0.0010) [2023-12-27 16:07:38,814][20347] Updated weights for policy 0, policy_version 1512800 (0.0009) [2023-12-27 16:07:38,872][20347] Updated weights for policy 0, policy_version 1512810 (0.0010) [2023-12-27 16:07:38,933][20347] Updated weights for policy 0, policy_version 1512820 (0.0010) [2023-12-27 16:07:39,176][20381] Updated weights for policy 1, policy_version 1511191 (0.0008) [2023-12-27 16:07:39,241][20381] Updated weights for policy 1, policy_version 1511201 (0.0008) [2023-12-27 16:07:39,307][20381] Updated weights for policy 1, policy_version 1511211 (0.0009) [2023-12-27 16:07:39,632][20347] Updated weights for policy 0, policy_version 1512830 (0.0010) [2023-12-27 16:07:39,694][20347] Updated weights for policy 0, policy_version 1512840 (0.0010) [2023-12-27 16:07:39,750][20347] Updated weights for policy 0, policy_version 1512850 (0.0011) [2023-12-27 16:07:40,065][20381] Updated weights for policy 1, policy_version 1511221 (0.0009) [2023-12-27 16:07:40,123][20381] Updated weights for policy 1, policy_version 1511231 (0.0008) [2023-12-27 16:07:40,174][20381] Updated weights for policy 1, policy_version 1511241 (0.0007) [2023-12-27 16:07:40,450][20347] Updated weights for policy 0, policy_version 1512860 (0.0010) [2023-12-27 16:07:40,511][20347] Updated weights for policy 0, policy_version 1512870 (0.0011) [2023-12-27 16:07:40,568][20347] Updated weights for policy 0, policy_version 1512880 (0.0010) [2023-12-27 16:07:40,828][20381] Updated weights for policy 1, policy_version 1511251 (0.0007) [2023-12-27 16:07:40,878][20381] Updated weights for policy 1, policy_version 1511261 (0.0005) [2023-12-27 16:07:40,930][20381] Updated weights for policy 1, policy_version 1511271 (0.0005) [2023-12-27 16:07:41,221][20347] Updated weights for policy 0, policy_version 1512890 (0.0011) [2023-12-27 16:07:41,288][20347] Updated weights for policy 0, policy_version 1512900 (0.0009) [2023-12-27 16:07:41,350][20347] Updated weights for policy 0, policy_version 1512910 (0.0008) [2023-12-27 16:07:41,421][20347] Updated weights for policy 0, policy_version 1512920 (0.0008) [2023-12-27 16:07:41,645][20381] Updated weights for policy 1, policy_version 1511281 (0.0008) [2023-12-27 16:07:41,702][20381] Updated weights for policy 1, policy_version 1511291 (0.0008) [2023-12-27 16:07:41,770][20381] Updated weights for policy 1, policy_version 1511301 (0.0009) [2023-12-27 16:07:41,833][20381] Updated weights for policy 1, policy_version 1511311 (0.0009) [2023-12-27 16:07:42,277][20347] Updated weights for policy 0, policy_version 1512930 (0.0010) [2023-12-27 16:07:42,341][20347] Updated weights for policy 0, policy_version 1512940 (0.0009) [2023-12-27 16:07:42,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 774316032. Throughput: 0: 10280.0, 1: 10271.6. Samples: 774314456. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:42,363][19170] Avg episode reward: [(0, '951.230'), (1, '961.070')] [2023-12-27 16:07:42,409][20347] Updated weights for policy 0, policy_version 1512950 (0.0009) [2023-12-27 16:07:42,605][20381] Updated weights for policy 1, policy_version 1511321 (0.0006) [2023-12-27 16:07:42,667][20381] Updated weights for policy 1, policy_version 1511331 (0.0007) [2023-12-27 16:07:42,717][20381] Updated weights for policy 1, policy_version 1511341 (0.0008) [2023-12-27 16:07:43,114][20347] Updated weights for policy 0, policy_version 1512960 (0.0010) [2023-12-27 16:07:43,166][20347] Updated weights for policy 0, policy_version 1512970 (0.0010) [2023-12-27 16:07:43,220][20347] Updated weights for policy 0, policy_version 1512980 (0.0010) [2023-12-27 16:07:43,397][20381] Updated weights for policy 1, policy_version 1511351 (0.0006) [2023-12-27 16:07:43,451][20381] Updated weights for policy 1, policy_version 1511361 (0.0005) [2023-12-27 16:07:43,505][20381] Updated weights for policy 1, policy_version 1511371 (0.0006) [2023-12-27 16:07:43,829][20347] Updated weights for policy 0, policy_version 1512990 (0.0008) [2023-12-27 16:07:43,876][20347] Updated weights for policy 0, policy_version 1513000 (0.0007) [2023-12-27 16:07:43,923][20347] Updated weights for policy 0, policy_version 1513010 (0.0007) [2023-12-27 16:07:44,098][20381] Updated weights for policy 1, policy_version 1511381 (0.0008) [2023-12-27 16:07:44,154][20381] Updated weights for policy 1, policy_version 1511391 (0.0011) [2023-12-27 16:07:44,212][20381] Updated weights for policy 1, policy_version 1511401 (0.0010) [2023-12-27 16:07:44,611][20347] Updated weights for policy 0, policy_version 1513020 (0.0007) [2023-12-27 16:07:44,675][20347] Updated weights for policy 0, policy_version 1513030 (0.0008) [2023-12-27 16:07:44,729][20347] Updated weights for policy 0, policy_version 1513040 (0.0007) [2023-12-27 16:07:44,892][20381] Updated weights for policy 1, policy_version 1511411 (0.0010) [2023-12-27 16:07:44,952][20381] Updated weights for policy 1, policy_version 1511421 (0.0011) [2023-12-27 16:07:44,997][20381] Updated weights for policy 1, policy_version 1511431 (0.0010) [2023-12-27 16:07:45,413][20347] Updated weights for policy 0, policy_version 1513050 (0.0007) [2023-12-27 16:07:45,465][20347] Updated weights for policy 0, policy_version 1513060 (0.0009) [2023-12-27 16:07:45,513][20347] Updated weights for policy 0, policy_version 1513070 (0.0010) [2023-12-27 16:07:45,564][20347] Updated weights for policy 0, policy_version 1513080 (0.0007) [2023-12-27 16:07:45,628][20381] Updated weights for policy 1, policy_version 1511441 (0.0010) [2023-12-27 16:07:45,676][20381] Updated weights for policy 1, policy_version 1511451 (0.0010) [2023-12-27 16:07:45,730][20381] Updated weights for policy 1, policy_version 1511461 (0.0005) [2023-12-27 16:07:45,785][20381] Updated weights for policy 1, policy_version 1511471 (0.0005) [2023-12-27 16:07:46,134][20347] Updated weights for policy 0, policy_version 1513090 (0.0005) [2023-12-27 16:07:46,185][20347] Updated weights for policy 0, policy_version 1513100 (0.0005) [2023-12-27 16:07:46,232][20347] Updated weights for policy 0, policy_version 1513110 (0.0005) [2023-12-27 16:07:46,367][20381] Updated weights for policy 1, policy_version 1511481 (0.0009) [2023-12-27 16:07:46,419][20381] Updated weights for policy 1, policy_version 1511491 (0.0010) [2023-12-27 16:07:46,480][20381] Updated weights for policy 1, policy_version 1511501 (0.0008) [2023-12-27 16:07:46,822][20347] Updated weights for policy 0, policy_version 1513120 (0.0005) [2023-12-27 16:07:46,879][20347] Updated weights for policy 0, policy_version 1513130 (0.0005) [2023-12-27 16:07:46,935][20347] Updated weights for policy 0, policy_version 1513140 (0.0008) [2023-12-27 16:07:47,076][20381] Updated weights for policy 1, policy_version 1511511 (0.0008) [2023-12-27 16:07:47,132][20381] Updated weights for policy 1, policy_version 1511521 (0.0006) [2023-12-27 16:07:47,195][20381] Updated weights for policy 1, policy_version 1511531 (0.0008) [2023-12-27 16:07:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 774430720. Throughput: 0: 10329.3, 1: 10334.8. Samples: 774439412. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:47,363][19170] Avg episode reward: [(0, '960.980'), (1, '970.900')] [2023-12-27 16:07:47,485][20347] Updated weights for policy 0, policy_version 1513150 (0.0006) [2023-12-27 16:07:47,533][20347] Updated weights for policy 0, policy_version 1513160 (0.0008) [2023-12-27 16:07:47,585][20347] Updated weights for policy 0, policy_version 1513170 (0.0006) [2023-12-27 16:07:47,878][20381] Updated weights for policy 1, policy_version 1511542 (0.0010) [2023-12-27 16:07:47,933][20381] Updated weights for policy 1, policy_version 1511552 (0.0011) [2023-12-27 16:07:47,988][20381] Updated weights for policy 1, policy_version 1511562 (0.0010) [2023-12-27 16:07:48,220][20347] Updated weights for policy 0, policy_version 1513180 (0.0007) [2023-12-27 16:07:48,265][20347] Updated weights for policy 0, policy_version 1513190 (0.0010) [2023-12-27 16:07:48,313][20347] Updated weights for policy 0, policy_version 1513200 (0.0010) [2023-12-27 16:07:48,657][20381] Updated weights for policy 1, policy_version 1511572 (0.0009) [2023-12-27 16:07:48,713][20381] Updated weights for policy 1, policy_version 1511582 (0.0006) [2023-12-27 16:07:48,772][20381] Updated weights for policy 1, policy_version 1511592 (0.0007) [2023-12-27 16:07:49,022][20347] Updated weights for policy 0, policy_version 1513210 (0.0010) [2023-12-27 16:07:49,079][20347] Updated weights for policy 0, policy_version 1513220 (0.0010) [2023-12-27 16:07:49,130][20347] Updated weights for policy 0, policy_version 1513230 (0.0010) [2023-12-27 16:07:49,179][20347] Updated weights for policy 0, policy_version 1513240 (0.0005) [2023-12-27 16:07:49,470][20381] Updated weights for policy 1, policy_version 1511602 (0.0007) [2023-12-27 16:07:49,526][20381] Updated weights for policy 1, policy_version 1511612 (0.0007) [2023-12-27 16:07:49,579][20381] Updated weights for policy 1, policy_version 1511622 (0.0007) [2023-12-27 16:07:49,636][20381] Updated weights for policy 1, policy_version 1511632 (0.0007) [2023-12-27 16:07:49,961][20347] Updated weights for policy 0, policy_version 1513250 (0.0008) [2023-12-27 16:07:50,022][20347] Updated weights for policy 0, policy_version 1513260 (0.0008) [2023-12-27 16:07:50,083][20347] Updated weights for policy 0, policy_version 1513270 (0.0007) [2023-12-27 16:07:50,295][20381] Updated weights for policy 1, policy_version 1511642 (0.0007) [2023-12-27 16:07:50,344][20381] Updated weights for policy 1, policy_version 1511652 (0.0006) [2023-12-27 16:07:50,400][20381] Updated weights for policy 1, policy_version 1511662 (0.0006) [2023-12-27 16:07:50,811][20347] Updated weights for policy 0, policy_version 1513280 (0.0007) [2023-12-27 16:07:50,872][20347] Updated weights for policy 0, policy_version 1513290 (0.0008) [2023-12-27 16:07:50,934][20347] Updated weights for policy 0, policy_version 1513300 (0.0008) [2023-12-27 16:07:51,017][20381] Updated weights for policy 1, policy_version 1511672 (0.0006) [2023-12-27 16:07:51,082][20381] Updated weights for policy 1, policy_version 1511682 (0.0009) [2023-12-27 16:07:51,141][20381] Updated weights for policy 1, policy_version 1511692 (0.0006) [2023-12-27 16:07:51,630][20347] Updated weights for policy 0, policy_version 1513310 (0.0010) [2023-12-27 16:07:51,692][20347] Updated weights for policy 0, policy_version 1513320 (0.0010) [2023-12-27 16:07:51,767][20347] Updated weights for policy 0, policy_version 1513330 (0.0011) [2023-12-27 16:07:51,906][20381] Updated weights for policy 1, policy_version 1511702 (0.0007) [2023-12-27 16:07:51,960][20381] Updated weights for policy 1, policy_version 1511712 (0.0005) [2023-12-27 16:07:52,024][20381] Updated weights for policy 1, policy_version 1511722 (0.0008) [2023-12-27 16:07:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 774529024. Throughput: 0: 10331.8, 1: 10379.4. Samples: 774501816. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:52,363][19170] Avg episode reward: [(0, '951.150'), (1, '970.950')] [2023-12-27 16:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001511728_387039232.pth... [2023-12-27 16:07:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001513336_387489792.pth... [2023-12-27 16:07:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001510512_386727936.pth [2023-12-27 16:07:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001512120_387178496.pth [2023-12-27 16:07:52,442][20347] Updated weights for policy 0, policy_version 1513340 (0.0007) [2023-12-27 16:07:52,502][20347] Updated weights for policy 0, policy_version 1513350 (0.0008) [2023-12-27 16:07:52,558][20347] Updated weights for policy 0, policy_version 1513360 (0.0006) [2023-12-27 16:07:52,742][20381] Updated weights for policy 1, policy_version 1511732 (0.0007) [2023-12-27 16:07:52,803][20381] Updated weights for policy 1, policy_version 1511742 (0.0008) [2023-12-27 16:07:52,861][20381] Updated weights for policy 1, policy_version 1511752 (0.0008) [2023-12-27 16:07:53,118][20347] Updated weights for policy 0, policy_version 1513370 (0.0006) [2023-12-27 16:07:53,174][20347] Updated weights for policy 0, policy_version 1513380 (0.0005) [2023-12-27 16:07:53,232][20347] Updated weights for policy 0, policy_version 1513390 (0.0005) [2023-12-27 16:07:53,297][20347] Updated weights for policy 0, policy_version 1513400 (0.0005) [2023-12-27 16:07:53,515][20381] Updated weights for policy 1, policy_version 1511762 (0.0008) [2023-12-27 16:07:53,576][20381] Updated weights for policy 1, policy_version 1511772 (0.0005) [2023-12-27 16:07:53,638][20381] Updated weights for policy 1, policy_version 1511782 (0.0005) [2023-12-27 16:07:53,697][20381] Updated weights for policy 1, policy_version 1511792 (0.0006) [2023-12-27 16:07:53,940][20347] Updated weights for policy 0, policy_version 1513410 (0.0005) [2023-12-27 16:07:53,996][20347] Updated weights for policy 0, policy_version 1513420 (0.0005) [2023-12-27 16:07:54,055][20347] Updated weights for policy 0, policy_version 1513430 (0.0005) [2023-12-27 16:07:54,243][20381] Updated weights for policy 1, policy_version 1511802 (0.0008) [2023-12-27 16:07:54,293][20381] Updated weights for policy 1, policy_version 1511812 (0.0010) [2023-12-27 16:07:54,342][20381] Updated weights for policy 1, policy_version 1511822 (0.0009) [2023-12-27 16:07:54,579][20347] Updated weights for policy 0, policy_version 1513440 (0.0005) [2023-12-27 16:07:54,632][20347] Updated weights for policy 0, policy_version 1513450 (0.0005) [2023-12-27 16:07:54,685][20347] Updated weights for policy 0, policy_version 1513460 (0.0005) [2023-12-27 16:07:55,056][20381] Updated weights for policy 1, policy_version 1511832 (0.0007) [2023-12-27 16:07:55,111][20381] Updated weights for policy 1, policy_version 1511842 (0.0008) [2023-12-27 16:07:55,157][20381] Updated weights for policy 1, policy_version 1511852 (0.0007) [2023-12-27 16:07:55,304][20347] Updated weights for policy 0, policy_version 1513470 (0.0005) [2023-12-27 16:07:55,364][20347] Updated weights for policy 0, policy_version 1513480 (0.0005) [2023-12-27 16:07:55,424][20347] Updated weights for policy 0, policy_version 1513490 (0.0005) [2023-12-27 16:07:55,870][20381] Updated weights for policy 1, policy_version 1511862 (0.0008) [2023-12-27 16:07:55,923][20381] Updated weights for policy 1, policy_version 1511872 (0.0007) [2023-12-27 16:07:55,971][20381] Updated weights for policy 1, policy_version 1511882 (0.0005) [2023-12-27 16:07:55,993][20347] Updated weights for policy 0, policy_version 1513500 (0.0005) [2023-12-27 16:07:56,048][20347] Updated weights for policy 0, policy_version 1513510 (0.0005) [2023-12-27 16:07:56,105][20347] Updated weights for policy 0, policy_version 1513520 (0.0005) [2023-12-27 16:07:56,615][20381] Updated weights for policy 1, policy_version 1511892 (0.0005) [2023-12-27 16:07:56,657][20347] Updated weights for policy 0, policy_version 1513530 (0.0006) [2023-12-27 16:07:56,673][20381] Updated weights for policy 1, policy_version 1511902 (0.0005) [2023-12-27 16:07:56,712][20347] Updated weights for policy 0, policy_version 1513540 (0.0008) [2023-12-27 16:07:56,728][20381] Updated weights for policy 1, policy_version 1511912 (0.0005) [2023-12-27 16:07:56,765][20347] Updated weights for policy 0, policy_version 1513550 (0.0008) [2023-12-27 16:07:56,816][20347] Updated weights for policy 0, policy_version 1513560 (0.0007) [2023-12-27 16:07:57,246][20381] Updated weights for policy 1, policy_version 1511922 (0.0006) [2023-12-27 16:07:57,293][20381] Updated weights for policy 1, policy_version 1511932 (0.0009) [2023-12-27 16:07:57,340][20381] Updated weights for policy 1, policy_version 1511942 (0.0009) [2023-12-27 16:07:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 774635520. Throughput: 0: 10471.1, 1: 10394.0. Samples: 774629384. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:07:57,363][19170] Avg episode reward: [(0, '941.290'), (1, '990.310')] [2023-12-27 16:07:57,388][20381] Updated weights for policy 1, policy_version 1511952 (0.0010) [2023-12-27 16:07:57,486][20347] Updated weights for policy 0, policy_version 1513570 (0.0007) [2023-12-27 16:07:57,540][20347] Updated weights for policy 0, policy_version 1513580 (0.0007) [2023-12-27 16:07:57,593][20347] Updated weights for policy 0, policy_version 1513590 (0.0007) [2023-12-27 16:07:58,102][20381] Updated weights for policy 1, policy_version 1511962 (0.0006) [2023-12-27 16:07:58,147][20381] Updated weights for policy 1, policy_version 1511972 (0.0006) [2023-12-27 16:07:58,208][20381] Updated weights for policy 1, policy_version 1511982 (0.0006) [2023-12-27 16:07:58,331][20347] Updated weights for policy 0, policy_version 1513600 (0.0009) [2023-12-27 16:07:58,397][20347] Updated weights for policy 0, policy_version 1513610 (0.0009) [2023-12-27 16:07:58,455][20347] Updated weights for policy 0, policy_version 1513620 (0.0009) [2023-12-27 16:07:58,971][20381] Updated weights for policy 1, policy_version 1511992 (0.0008) [2023-12-27 16:07:59,034][20381] Updated weights for policy 1, policy_version 1512002 (0.0008) [2023-12-27 16:07:59,095][20381] Updated weights for policy 1, policy_version 1512012 (0.0007) [2023-12-27 16:07:59,256][20347] Updated weights for policy 0, policy_version 1513630 (0.0009) [2023-12-27 16:07:59,322][20347] Updated weights for policy 0, policy_version 1513640 (0.0008) [2023-12-27 16:07:59,382][20347] Updated weights for policy 0, policy_version 1513650 (0.0008) [2023-12-27 16:07:59,793][20381] Updated weights for policy 1, policy_version 1512022 (0.0007) [2023-12-27 16:07:59,852][20381] Updated weights for policy 1, policy_version 1512032 (0.0008) [2023-12-27 16:07:59,904][20381] Updated weights for policy 1, policy_version 1512042 (0.0006) [2023-12-27 16:08:00,083][20347] Updated weights for policy 0, policy_version 1513660 (0.0010) [2023-12-27 16:08:00,149][20347] Updated weights for policy 0, policy_version 1513670 (0.0011) [2023-12-27 16:08:00,212][20347] Updated weights for policy 0, policy_version 1513680 (0.0010) [2023-12-27 16:08:00,583][20381] Updated weights for policy 1, policy_version 1512052 (0.0007) [2023-12-27 16:08:00,630][20381] Updated weights for policy 1, policy_version 1512062 (0.0007) [2023-12-27 16:08:00,680][20381] Updated weights for policy 1, policy_version 1512072 (0.0007) [2023-12-27 16:08:00,863][20347] Updated weights for policy 0, policy_version 1513690 (0.0010) [2023-12-27 16:08:00,927][20347] Updated weights for policy 0, policy_version 1513700 (0.0010) [2023-12-27 16:08:00,987][20347] Updated weights for policy 0, policy_version 1513710 (0.0010) [2023-12-27 16:08:01,049][20347] Updated weights for policy 0, policy_version 1513720 (0.0009) [2023-12-27 16:08:01,360][20381] Updated weights for policy 1, policy_version 1512082 (0.0007) [2023-12-27 16:08:01,425][20381] Updated weights for policy 1, policy_version 1512092 (0.0008) [2023-12-27 16:08:01,483][20381] Updated weights for policy 1, policy_version 1512102 (0.0007) [2023-12-27 16:08:01,537][20381] Updated weights for policy 1, policy_version 1512112 (0.0007) [2023-12-27 16:08:01,776][20347] Updated weights for policy 0, policy_version 1513730 (0.0007) [2023-12-27 16:08:01,835][20347] Updated weights for policy 0, policy_version 1513740 (0.0007) [2023-12-27 16:08:01,895][20347] Updated weights for policy 0, policy_version 1513750 (0.0007) [2023-12-27 16:08:02,235][20381] Updated weights for policy 1, policy_version 1512122 (0.0010) [2023-12-27 16:08:02,300][20381] Updated weights for policy 1, policy_version 1512132 (0.0011) [2023-12-27 16:08:02,361][20381] Updated weights for policy 1, policy_version 1512142 (0.0007) [2023-12-27 16:08:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 774733824. Throughput: 0: 10417.8, 1: 10415.2. Samples: 774749916. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:08:02,363][19170] Avg episode reward: [(0, '951.220'), (1, '990.300')] [2023-12-27 16:08:02,555][20347] Updated weights for policy 0, policy_version 1513760 (0.0007) [2023-12-27 16:08:02,616][20347] Updated weights for policy 0, policy_version 1513770 (0.0007) [2023-12-27 16:08:02,675][20347] Updated weights for policy 0, policy_version 1513780 (0.0008) [2023-12-27 16:08:03,005][20381] Updated weights for policy 1, policy_version 1512152 (0.0005) [2023-12-27 16:08:03,057][20381] Updated weights for policy 1, policy_version 1512162 (0.0007) [2023-12-27 16:08:03,101][20381] Updated weights for policy 1, policy_version 1512172 (0.0008) [2023-12-27 16:08:03,339][20347] Updated weights for policy 0, policy_version 1513790 (0.0007) [2023-12-27 16:08:03,396][20347] Updated weights for policy 0, policy_version 1513800 (0.0008) [2023-12-27 16:08:03,452][20347] Updated weights for policy 0, policy_version 1513810 (0.0007) [2023-12-27 16:08:03,801][20381] Updated weights for policy 1, policy_version 1512182 (0.0006) [2023-12-27 16:08:03,859][20381] Updated weights for policy 1, policy_version 1512192 (0.0006) [2023-12-27 16:08:03,916][20381] Updated weights for policy 1, policy_version 1512202 (0.0007) [2023-12-27 16:08:04,077][20347] Updated weights for policy 0, policy_version 1513820 (0.0008) [2023-12-27 16:08:04,132][20347] Updated weights for policy 0, policy_version 1513830 (0.0007) [2023-12-27 16:08:04,192][20347] Updated weights for policy 0, policy_version 1513840 (0.0008) [2023-12-27 16:08:04,606][20381] Updated weights for policy 1, policy_version 1512212 (0.0008) [2023-12-27 16:08:04,664][20381] Updated weights for policy 1, policy_version 1512222 (0.0007) [2023-12-27 16:08:04,712][20381] Updated weights for policy 1, policy_version 1512232 (0.0006) [2023-12-27 16:08:04,919][20347] Updated weights for policy 0, policy_version 1513850 (0.0008) [2023-12-27 16:08:04,972][20347] Updated weights for policy 0, policy_version 1513860 (0.0009) [2023-12-27 16:08:05,020][20347] Updated weights for policy 0, policy_version 1513870 (0.0008) [2023-12-27 16:08:05,070][20347] Updated weights for policy 0, policy_version 1513880 (0.0008) [2023-12-27 16:08:05,315][20381] Updated weights for policy 1, policy_version 1512242 (0.0006) [2023-12-27 16:08:05,367][20381] Updated weights for policy 1, policy_version 1512252 (0.0009) [2023-12-27 16:08:05,427][20381] Updated weights for policy 1, policy_version 1512262 (0.0005) [2023-12-27 16:08:05,487][20381] Updated weights for policy 1, policy_version 1512272 (0.0006) [2023-12-27 16:08:05,805][20347] Updated weights for policy 0, policy_version 1513890 (0.0007) [2023-12-27 16:08:05,858][20347] Updated weights for policy 0, policy_version 1513900 (0.0007) [2023-12-27 16:08:05,907][20347] Updated weights for policy 0, policy_version 1513910 (0.0007) [2023-12-27 16:08:06,101][20381] Updated weights for policy 1, policy_version 1512282 (0.0011) [2023-12-27 16:08:06,169][20381] Updated weights for policy 1, policy_version 1512292 (0.0011) [2023-12-27 16:08:06,232][20381] Updated weights for policy 1, policy_version 1512302 (0.0011) [2023-12-27 16:08:06,603][20347] Updated weights for policy 0, policy_version 1513920 (0.0007) [2023-12-27 16:08:06,673][20347] Updated weights for policy 0, policy_version 1513930 (0.0006) [2023-12-27 16:08:06,740][20347] Updated weights for policy 0, policy_version 1513940 (0.0009) [2023-12-27 16:08:06,960][20381] Updated weights for policy 1, policy_version 1512312 (0.0011) [2023-12-27 16:08:07,022][20381] Updated weights for policy 1, policy_version 1512322 (0.0011) [2023-12-27 16:08:07,083][20381] Updated weights for policy 1, policy_version 1512332 (0.0011) [2023-12-27 16:08:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 774840320. Throughput: 0: 10441.9, 1: 10431.9. Samples: 774811996. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:08:07,363][19170] Avg episode reward: [(0, '970.790'), (1, '990.300')] [2023-12-27 16:08:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001512336_387194880.pth... [2023-12-27 16:08:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001513944_387645440.pth... [2023-12-27 16:08:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001511120_386883584.pth [2023-12-27 16:08:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001512760_387342336.pth [2023-12-27 16:08:07,467][20347] Updated weights for policy 0, policy_version 1513950 (0.0008) [2023-12-27 16:08:07,514][20347] Updated weights for policy 0, policy_version 1513960 (0.0007) [2023-12-27 16:08:07,559][20347] Updated weights for policy 0, policy_version 1513970 (0.0007) [2023-12-27 16:08:07,778][20381] Updated weights for policy 1, policy_version 1512342 (0.0009) [2023-12-27 16:08:07,829][20381] Updated weights for policy 1, policy_version 1512352 (0.0008) [2023-12-27 16:08:07,881][20381] Updated weights for policy 1, policy_version 1512362 (0.0007) [2023-12-27 16:08:08,205][20347] Updated weights for policy 0, policy_version 1513980 (0.0007) [2023-12-27 16:08:08,249][20347] Updated weights for policy 0, policy_version 1513990 (0.0008) [2023-12-27 16:08:08,298][20347] Updated weights for policy 0, policy_version 1514000 (0.0010) [2023-12-27 16:08:08,612][20381] Updated weights for policy 1, policy_version 1512372 (0.0007) [2023-12-27 16:08:08,665][20381] Updated weights for policy 1, policy_version 1512382 (0.0006) [2023-12-27 16:08:08,719][20381] Updated weights for policy 1, policy_version 1512392 (0.0007) [2023-12-27 16:08:08,985][20347] Updated weights for policy 0, policy_version 1514010 (0.0008) [2023-12-27 16:08:09,040][20347] Updated weights for policy 0, policy_version 1514020 (0.0006) [2023-12-27 16:08:09,102][20347] Updated weights for policy 0, policy_version 1514030 (0.0005) [2023-12-27 16:08:09,164][20347] Updated weights for policy 0, policy_version 1514040 (0.0008) [2023-12-27 16:08:09,380][20381] Updated weights for policy 1, policy_version 1512402 (0.0005) [2023-12-27 16:08:09,445][20381] Updated weights for policy 1, policy_version 1512412 (0.0008) [2023-12-27 16:08:09,509][20381] Updated weights for policy 1, policy_version 1512422 (0.0007) [2023-12-27 16:08:09,566][20381] Updated weights for policy 1, policy_version 1512432 (0.0008) [2023-12-27 16:08:09,866][20347] Updated weights for policy 0, policy_version 1514050 (0.0008) [2023-12-27 16:08:09,932][20347] Updated weights for policy 0, policy_version 1514060 (0.0008) [2023-12-27 16:08:09,994][20347] Updated weights for policy 0, policy_version 1514070 (0.0008) [2023-12-27 16:08:10,310][20381] Updated weights for policy 1, policy_version 1512442 (0.0008) [2023-12-27 16:08:10,375][20381] Updated weights for policy 1, policy_version 1512452 (0.0008) [2023-12-27 16:08:10,437][20381] Updated weights for policy 1, policy_version 1512462 (0.0008) [2023-12-27 16:08:10,682][20347] Updated weights for policy 0, policy_version 1514080 (0.0008) [2023-12-27 16:08:10,731][20347] Updated weights for policy 0, policy_version 1514090 (0.0008) [2023-12-27 16:08:10,780][20347] Updated weights for policy 0, policy_version 1514100 (0.0007) [2023-12-27 16:08:10,995][20381] Updated weights for policy 1, policy_version 1512472 (0.0006) [2023-12-27 16:08:11,058][20381] Updated weights for policy 1, policy_version 1512482 (0.0007) [2023-12-27 16:08:11,116][20381] Updated weights for policy 1, policy_version 1512492 (0.0008) [2023-12-27 16:08:11,533][20347] Updated weights for policy 0, policy_version 1514110 (0.0008) [2023-12-27 16:08:11,587][20347] Updated weights for policy 0, policy_version 1514120 (0.0008) [2023-12-27 16:08:11,654][20347] Updated weights for policy 0, policy_version 1514130 (0.0007) [2023-12-27 16:08:11,859][20381] Updated weights for policy 1, policy_version 1512502 (0.0008) [2023-12-27 16:08:11,919][20381] Updated weights for policy 1, policy_version 1512512 (0.0008) [2023-12-27 16:08:11,981][20381] Updated weights for policy 1, policy_version 1512522 (0.0008) [2023-12-27 16:08:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 774938624. Throughput: 0: 10371.2, 1: 10362.7. Samples: 774932712. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:08:12,363][19170] Avg episode reward: [(0, '951.040'), (1, '1000.000')] [2023-12-27 16:08:12,376][20347] Updated weights for policy 0, policy_version 1514140 (0.0008) [2023-12-27 16:08:12,437][20347] Updated weights for policy 0, policy_version 1514150 (0.0008) [2023-12-27 16:08:12,491][20347] Updated weights for policy 0, policy_version 1514160 (0.0008) [2023-12-27 16:08:12,698][20381] Updated weights for policy 1, policy_version 1512532 (0.0006) [2023-12-27 16:08:12,763][20381] Updated weights for policy 1, policy_version 1512542 (0.0009) [2023-12-27 16:08:12,823][20381] Updated weights for policy 1, policy_version 1512552 (0.0011) [2023-12-27 16:08:13,177][20347] Updated weights for policy 0, policy_version 1514170 (0.0006) [2023-12-27 16:08:13,242][20347] Updated weights for policy 0, policy_version 1514180 (0.0008) [2023-12-27 16:08:13,302][20347] Updated weights for policy 0, policy_version 1514190 (0.0007) [2023-12-27 16:08:13,368][20347] Updated weights for policy 0, policy_version 1514200 (0.0007) [2023-12-27 16:08:13,478][20381] Updated weights for policy 1, policy_version 1512562 (0.0011) [2023-12-27 16:08:13,538][20381] Updated weights for policy 1, policy_version 1512572 (0.0011) [2023-12-27 16:08:13,596][20381] Updated weights for policy 1, policy_version 1512582 (0.0010) [2023-12-27 16:08:13,652][20381] Updated weights for policy 1, policy_version 1512592 (0.0010) [2023-12-27 16:08:14,019][20347] Updated weights for policy 0, policy_version 1514210 (0.0008) [2023-12-27 16:08:14,066][20347] Updated weights for policy 0, policy_version 1514220 (0.0007) [2023-12-27 16:08:14,112][20347] Updated weights for policy 0, policy_version 1514230 (0.0007) [2023-12-27 16:08:14,317][20381] Updated weights for policy 1, policy_version 1512602 (0.0007) [2023-12-27 16:08:14,375][20381] Updated weights for policy 1, policy_version 1512612 (0.0010) [2023-12-27 16:08:14,433][20381] Updated weights for policy 1, policy_version 1512622 (0.0010) [2023-12-27 16:08:14,742][20347] Updated weights for policy 0, policy_version 1514240 (0.0006) [2023-12-27 16:08:14,805][20347] Updated weights for policy 0, policy_version 1514250 (0.0008) [2023-12-27 16:08:14,853][20347] Updated weights for policy 0, policy_version 1514260 (0.0008) [2023-12-27 16:08:15,099][20381] Updated weights for policy 1, policy_version 1512632 (0.0008) [2023-12-27 16:08:15,162][20381] Updated weights for policy 1, policy_version 1512642 (0.0010) [2023-12-27 16:08:15,224][20381] Updated weights for policy 1, policy_version 1512652 (0.0010) [2023-12-27 16:08:15,528][20347] Updated weights for policy 0, policy_version 1514270 (0.0010) [2023-12-27 16:08:15,587][20347] Updated weights for policy 0, policy_version 1514280 (0.0010) [2023-12-27 16:08:15,645][20347] Updated weights for policy 0, policy_version 1514290 (0.0010) [2023-12-27 16:08:15,861][20381] Updated weights for policy 1, policy_version 1512662 (0.0007) [2023-12-27 16:08:15,911][20381] Updated weights for policy 1, policy_version 1512672 (0.0008) [2023-12-27 16:08:15,969][20381] Updated weights for policy 1, policy_version 1512682 (0.0010) [2023-12-27 16:08:16,272][20347] Updated weights for policy 0, policy_version 1514300 (0.0010) [2023-12-27 16:08:16,329][20347] Updated weights for policy 0, policy_version 1514310 (0.0010) [2023-12-27 16:08:16,385][20347] Updated weights for policy 0, policy_version 1514320 (0.0010) [2023-12-27 16:08:16,539][20381] Updated weights for policy 1, policy_version 1512692 (0.0008) [2023-12-27 16:08:16,593][20381] Updated weights for policy 1, policy_version 1512702 (0.0005) [2023-12-27 16:08:16,644][20381] Updated weights for policy 1, policy_version 1512712 (0.0005) [2023-12-27 16:08:16,993][20347] Updated weights for policy 0, policy_version 1514330 (0.0009) [2023-12-27 16:08:17,050][20347] Updated weights for policy 0, policy_version 1514340 (0.0005) [2023-12-27 16:08:17,111][20347] Updated weights for policy 0, policy_version 1514350 (0.0005) [2023-12-27 16:08:17,165][20347] Updated weights for policy 0, policy_version 1514360 (0.0007) [2023-12-27 16:08:17,201][20381] Updated weights for policy 1, policy_version 1512722 (0.0005) [2023-12-27 16:08:17,260][20381] Updated weights for policy 1, policy_version 1512732 (0.0007) [2023-12-27 16:08:17,308][20381] Updated weights for policy 1, policy_version 1512742 (0.0007) [2023-12-27 16:08:17,355][20381] Updated weights for policy 1, policy_version 1512752 (0.0007) [2023-12-27 16:08:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 775053312. Throughput: 0: 10319.6, 1: 10385.3. Samples: 775058228. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:17,363][19170] Avg episode reward: [(0, '941.260'), (1, '990.240')] [2023-12-27 16:08:17,775][20347] Updated weights for policy 0, policy_version 1514370 (0.0010) [2023-12-27 16:08:17,824][20347] Updated weights for policy 0, policy_version 1514380 (0.0010) [2023-12-27 16:08:17,880][20347] Updated weights for policy 0, policy_version 1514390 (0.0010) [2023-12-27 16:08:18,027][20381] Updated weights for policy 1, policy_version 1512762 (0.0008) [2023-12-27 16:08:18,074][20381] Updated weights for policy 1, policy_version 1512772 (0.0008) [2023-12-27 16:08:18,131][20381] Updated weights for policy 1, policy_version 1512782 (0.0007) [2023-12-27 16:08:18,587][20347] Updated weights for policy 0, policy_version 1514400 (0.0007) [2023-12-27 16:08:18,638][20347] Updated weights for policy 0, policy_version 1514410 (0.0007) [2023-12-27 16:08:18,690][20347] Updated weights for policy 0, policy_version 1514420 (0.0007) [2023-12-27 16:08:18,810][20381] Updated weights for policy 1, policy_version 1512792 (0.0008) [2023-12-27 16:08:18,865][20381] Updated weights for policy 1, policy_version 1512802 (0.0006) [2023-12-27 16:08:18,916][20381] Updated weights for policy 1, policy_version 1512812 (0.0006) [2023-12-27 16:08:19,316][20347] Updated weights for policy 0, policy_version 1514430 (0.0007) [2023-12-27 16:08:19,378][20347] Updated weights for policy 0, policy_version 1514440 (0.0006) [2023-12-27 16:08:19,441][20347] Updated weights for policy 0, policy_version 1514450 (0.0007) [2023-12-27 16:08:19,577][20381] Updated weights for policy 1, policy_version 1512822 (0.0006) [2023-12-27 16:08:19,627][20381] Updated weights for policy 1, policy_version 1512832 (0.0011) [2023-12-27 16:08:19,690][20381] Updated weights for policy 1, policy_version 1512842 (0.0011) [2023-12-27 16:08:20,098][20347] Updated weights for policy 0, policy_version 1514460 (0.0007) [2023-12-27 16:08:20,157][20347] Updated weights for policy 0, policy_version 1514470 (0.0007) [2023-12-27 16:08:20,215][20347] Updated weights for policy 0, policy_version 1514480 (0.0007) [2023-12-27 16:08:20,384][20381] Updated weights for policy 1, policy_version 1512852 (0.0008) [2023-12-27 16:08:20,442][20381] Updated weights for policy 1, policy_version 1512862 (0.0007) [2023-12-27 16:08:20,508][20381] Updated weights for policy 1, policy_version 1512872 (0.0005) [2023-12-27 16:08:20,937][20347] Updated weights for policy 0, policy_version 1514490 (0.0007) [2023-12-27 16:08:20,999][20347] Updated weights for policy 0, policy_version 1514500 (0.0007) [2023-12-27 16:08:21,073][20347] Updated weights for policy 0, policy_version 1514510 (0.0008) [2023-12-27 16:08:21,132][20347] Updated weights for policy 0, policy_version 1514520 (0.0008) [2023-12-27 16:08:21,217][20381] Updated weights for policy 1, policy_version 1512882 (0.0008) [2023-12-27 16:08:21,277][20381] Updated weights for policy 1, policy_version 1512892 (0.0009) [2023-12-27 16:08:21,329][20381] Updated weights for policy 1, policy_version 1512902 (0.0008) [2023-12-27 16:08:21,390][20381] Updated weights for policy 1, policy_version 1512912 (0.0008) [2023-12-27 16:08:21,831][20347] Updated weights for policy 0, policy_version 1514530 (0.0009) [2023-12-27 16:08:21,892][20347] Updated weights for policy 0, policy_version 1514540 (0.0009) [2023-12-27 16:08:21,950][20347] Updated weights for policy 0, policy_version 1514550 (0.0009) [2023-12-27 16:08:22,167][20381] Updated weights for policy 1, policy_version 1512922 (0.0007) [2023-12-27 16:08:22,215][20381] Updated weights for policy 1, policy_version 1512932 (0.0007) [2023-12-27 16:08:22,271][20381] Updated weights for policy 1, policy_version 1512942 (0.0008) [2023-12-27 16:08:22,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 775151616. Throughput: 0: 10337.8, 1: 10366.1. Samples: 775120700. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:22,363][19170] Avg episode reward: [(0, '941.600'), (1, '970.950')] [2023-12-27 16:08:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001512944_387350528.pth... [2023-12-27 16:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001514552_387801088.pth... [2023-12-27 16:08:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001511728_387039232.pth [2023-12-27 16:08:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001513336_387489792.pth [2023-12-27 16:08:22,722][20347] Updated weights for policy 0, policy_version 1514560 (0.0010) [2023-12-27 16:08:22,777][20347] Updated weights for policy 0, policy_version 1514570 (0.0009) [2023-12-27 16:08:22,834][20347] Updated weights for policy 0, policy_version 1514580 (0.0010) [2023-12-27 16:08:23,021][20381] Updated weights for policy 1, policy_version 1512952 (0.0006) [2023-12-27 16:08:23,080][20381] Updated weights for policy 1, policy_version 1512962 (0.0005) [2023-12-27 16:08:23,138][20381] Updated weights for policy 1, policy_version 1512972 (0.0011) [2023-12-27 16:08:23,459][20347] Updated weights for policy 0, policy_version 1514590 (0.0009) [2023-12-27 16:08:23,508][20347] Updated weights for policy 0, policy_version 1514600 (0.0006) [2023-12-27 16:08:23,570][20347] Updated weights for policy 0, policy_version 1514610 (0.0006) [2023-12-27 16:08:23,686][20381] Updated weights for policy 1, policy_version 1512982 (0.0009) [2023-12-27 16:08:23,743][20381] Updated weights for policy 1, policy_version 1512992 (0.0010) [2023-12-27 16:08:23,788][20381] Updated weights for policy 1, policy_version 1513002 (0.0010) [2023-12-27 16:08:24,261][20347] Updated weights for policy 0, policy_version 1514620 (0.0008) [2023-12-27 16:08:24,325][20347] Updated weights for policy 0, policy_version 1514630 (0.0008) [2023-12-27 16:08:24,390][20347] Updated weights for policy 0, policy_version 1514640 (0.0007) [2023-12-27 16:08:24,415][20381] Updated weights for policy 1, policy_version 1513012 (0.0008) [2023-12-27 16:08:24,470][20381] Updated weights for policy 1, policy_version 1513022 (0.0005) [2023-12-27 16:08:24,518][20381] Updated weights for policy 1, policy_version 1513032 (0.0005) [2023-12-27 16:08:25,043][20347] Updated weights for policy 0, policy_version 1514650 (0.0008) [2023-12-27 16:08:25,087][20347] Updated weights for policy 0, policy_version 1514660 (0.0007) [2023-12-27 16:08:25,098][20381] Updated weights for policy 1, policy_version 1513042 (0.0006) [2023-12-27 16:08:25,143][20347] Updated weights for policy 0, policy_version 1514670 (0.0007) [2023-12-27 16:08:25,150][20381] Updated weights for policy 1, policy_version 1513052 (0.0007) [2023-12-27 16:08:25,201][20347] Updated weights for policy 0, policy_version 1514680 (0.0008) [2023-12-27 16:08:25,204][20381] Updated weights for policy 1, policy_version 1513062 (0.0006) [2023-12-27 16:08:25,258][20381] Updated weights for policy 1, policy_version 1513072 (0.0007) [2023-12-27 16:08:25,891][20347] Updated weights for policy 0, policy_version 1514690 (0.0006) [2023-12-27 16:08:25,897][20381] Updated weights for policy 1, policy_version 1513082 (0.0008) [2023-12-27 16:08:25,950][20347] Updated weights for policy 0, policy_version 1514700 (0.0006) [2023-12-27 16:08:25,952][20381] Updated weights for policy 1, policy_version 1513092 (0.0007) [2023-12-27 16:08:25,999][20347] Updated weights for policy 0, policy_version 1514710 (0.0006) [2023-12-27 16:08:26,008][20381] Updated weights for policy 1, policy_version 1513102 (0.0005) [2023-12-27 16:08:26,578][20381] Updated weights for policy 1, policy_version 1513112 (0.0007) [2023-12-27 16:08:26,609][20347] Updated weights for policy 0, policy_version 1514720 (0.0006) [2023-12-27 16:08:26,633][20381] Updated weights for policy 1, policy_version 1513122 (0.0006) [2023-12-27 16:08:26,663][20347] Updated weights for policy 0, policy_version 1514730 (0.0006) [2023-12-27 16:08:26,694][20381] Updated weights for policy 1, policy_version 1513132 (0.0006) [2023-12-27 16:08:26,708][20347] Updated weights for policy 0, policy_version 1514740 (0.0005) [2023-12-27 16:08:27,333][20381] Updated weights for policy 1, policy_version 1513142 (0.0007) [2023-12-27 16:08:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 775249920. Throughput: 0: 10259.3, 1: 10384.5. Samples: 775243424. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:27,363][19170] Avg episode reward: [(0, '941.420'), (1, '980.570')] [2023-12-27 16:08:27,382][20381] Updated weights for policy 1, policy_version 1513152 (0.0007) [2023-12-27 16:08:27,414][20347] Updated weights for policy 0, policy_version 1514750 (0.0006) [2023-12-27 16:08:27,427][20381] Updated weights for policy 1, policy_version 1513162 (0.0006) [2023-12-27 16:08:27,467][20347] Updated weights for policy 0, policy_version 1514760 (0.0006) [2023-12-27 16:08:27,518][20347] Updated weights for policy 0, policy_version 1514770 (0.0008) [2023-12-27 16:08:28,123][20381] Updated weights for policy 1, policy_version 1513172 (0.0008) [2023-12-27 16:08:28,179][20381] Updated weights for policy 1, policy_version 1513182 (0.0008) [2023-12-27 16:08:28,217][20347] Updated weights for policy 0, policy_version 1514780 (0.0006) [2023-12-27 16:08:28,232][20381] Updated weights for policy 1, policy_version 1513192 (0.0008) [2023-12-27 16:08:28,273][20347] Updated weights for policy 0, policy_version 1514790 (0.0005) [2023-12-27 16:08:28,337][20347] Updated weights for policy 0, policy_version 1514800 (0.0008) [2023-12-27 16:08:28,953][20381] Updated weights for policy 1, policy_version 1513202 (0.0007) [2023-12-27 16:08:28,988][20347] Updated weights for policy 0, policy_version 1514810 (0.0007) [2023-12-27 16:08:29,015][20381] Updated weights for policy 1, policy_version 1513212 (0.0006) [2023-12-27 16:08:29,042][20347] Updated weights for policy 0, policy_version 1514820 (0.0008) [2023-12-27 16:08:29,073][20381] Updated weights for policy 1, policy_version 1513222 (0.0006) [2023-12-27 16:08:29,093][20347] Updated weights for policy 0, policy_version 1514830 (0.0005) [2023-12-27 16:08:29,128][20381] Updated weights for policy 1, policy_version 1513232 (0.0005) [2023-12-27 16:08:29,146][20347] Updated weights for policy 0, policy_version 1514840 (0.0006) [2023-12-27 16:08:29,804][20381] Updated weights for policy 1, policy_version 1513242 (0.0009) [2023-12-27 16:08:29,862][20347] Updated weights for policy 0, policy_version 1514850 (0.0007) [2023-12-27 16:08:29,864][20381] Updated weights for policy 1, policy_version 1513252 (0.0008) [2023-12-27 16:08:29,918][20381] Updated weights for policy 1, policy_version 1513262 (0.0007) [2023-12-27 16:08:29,925][20347] Updated weights for policy 0, policy_version 1514860 (0.0008) [2023-12-27 16:08:29,986][20347] Updated weights for policy 0, policy_version 1514870 (0.0008) [2023-12-27 16:08:30,504][20381] Updated weights for policy 1, policy_version 1513272 (0.0007) [2023-12-27 16:08:30,560][20381] Updated weights for policy 1, policy_version 1513282 (0.0009) [2023-12-27 16:08:30,609][20381] Updated weights for policy 1, policy_version 1513292 (0.0007) [2023-12-27 16:08:30,651][20347] Updated weights for policy 0, policy_version 1514880 (0.0008) [2023-12-27 16:08:30,698][20347] Updated weights for policy 0, policy_version 1514890 (0.0007) [2023-12-27 16:08:30,749][20347] Updated weights for policy 0, policy_version 1514900 (0.0008) [2023-12-27 16:08:31,311][20381] Updated weights for policy 1, policy_version 1513302 (0.0007) [2023-12-27 16:08:31,376][20381] Updated weights for policy 1, policy_version 1513312 (0.0008) [2023-12-27 16:08:31,429][20381] Updated weights for policy 1, policy_version 1513322 (0.0007) [2023-12-27 16:08:31,468][20347] Updated weights for policy 0, policy_version 1514910 (0.0008) [2023-12-27 16:08:31,516][20347] Updated weights for policy 0, policy_version 1514920 (0.0008) [2023-12-27 16:08:31,570][20347] Updated weights for policy 0, policy_version 1514930 (0.0008) [2023-12-27 16:08:32,057][20381] Updated weights for policy 1, policy_version 1513332 (0.0007) [2023-12-27 16:08:32,109][20381] Updated weights for policy 1, policy_version 1513342 (0.0007) [2023-12-27 16:08:32,166][20381] Updated weights for policy 1, policy_version 1513352 (0.0008) [2023-12-27 16:08:32,341][20347] Updated weights for policy 0, policy_version 1514940 (0.0006) [2023-12-27 16:08:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 775356416. Throughput: 0: 10236.9, 1: 10379.5. Samples: 775367152. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:32,363][19170] Avg episode reward: [(0, '970.640'), (1, '990.270')] [2023-12-27 16:08:32,403][20347] Updated weights for policy 0, policy_version 1514950 (0.0009) [2023-12-27 16:08:32,457][20347] Updated weights for policy 0, policy_version 1514960 (0.0007) [2023-12-27 16:08:32,832][20381] Updated weights for policy 1, policy_version 1513362 (0.0007) [2023-12-27 16:08:32,880][20381] Updated weights for policy 1, policy_version 1513372 (0.0005) [2023-12-27 16:08:32,940][20381] Updated weights for policy 1, policy_version 1513382 (0.0005) [2023-12-27 16:08:32,996][20381] Updated weights for policy 1, policy_version 1513392 (0.0005) [2023-12-27 16:08:33,105][20347] Updated weights for policy 0, policy_version 1514970 (0.0007) [2023-12-27 16:08:33,162][20347] Updated weights for policy 0, policy_version 1514980 (0.0005) [2023-12-27 16:08:33,214][20347] Updated weights for policy 0, policy_version 1514990 (0.0005) [2023-12-27 16:08:33,260][20347] Updated weights for policy 0, policy_version 1515000 (0.0005) [2023-12-27 16:08:33,599][20381] Updated weights for policy 1, policy_version 1513402 (0.0009) [2023-12-27 16:08:33,644][20381] Updated weights for policy 1, policy_version 1513412 (0.0009) [2023-12-27 16:08:33,688][20381] Updated weights for policy 1, policy_version 1513422 (0.0008) [2023-12-27 16:08:33,793][20347] Updated weights for policy 0, policy_version 1515010 (0.0010) [2023-12-27 16:08:33,847][20347] Updated weights for policy 0, policy_version 1515020 (0.0010) [2023-12-27 16:08:33,893][20347] Updated weights for policy 0, policy_version 1515030 (0.0010) [2023-12-27 16:08:34,468][20381] Updated weights for policy 1, policy_version 1513432 (0.0007) [2023-12-27 16:08:34,533][20381] Updated weights for policy 1, policy_version 1513442 (0.0009) [2023-12-27 16:08:34,541][20347] Updated weights for policy 0, policy_version 1515040 (0.0008) [2023-12-27 16:08:34,597][20381] Updated weights for policy 1, policy_version 1513452 (0.0007) [2023-12-27 16:08:34,599][20347] Updated weights for policy 0, policy_version 1515050 (0.0007) [2023-12-27 16:08:34,658][20347] Updated weights for policy 0, policy_version 1515060 (0.0007) [2023-12-27 16:08:35,256][20347] Updated weights for policy 0, policy_version 1515070 (0.0006) [2023-12-27 16:08:35,317][20381] Updated weights for policy 1, policy_version 1513462 (0.0007) [2023-12-27 16:08:35,318][20347] Updated weights for policy 0, policy_version 1515080 (0.0005) [2023-12-27 16:08:35,367][20381] Updated weights for policy 1, policy_version 1513472 (0.0007) [2023-12-27 16:08:35,379][20347] Updated weights for policy 0, policy_version 1515090 (0.0007) [2023-12-27 16:08:35,424][20381] Updated weights for policy 1, policy_version 1513482 (0.0007) [2023-12-27 16:08:35,983][20347] Updated weights for policy 0, policy_version 1515100 (0.0005) [2023-12-27 16:08:36,030][20347] Updated weights for policy 0, policy_version 1515110 (0.0005) [2023-12-27 16:08:36,086][20347] Updated weights for policy 0, policy_version 1515120 (0.0007) [2023-12-27 16:08:36,100][20381] Updated weights for policy 1, policy_version 1513492 (0.0007) [2023-12-27 16:08:36,165][20381] Updated weights for policy 1, policy_version 1513502 (0.0007) [2023-12-27 16:08:36,238][20381] Updated weights for policy 1, policy_version 1513512 (0.0009) [2023-12-27 16:08:36,792][20347] Updated weights for policy 0, policy_version 1515130 (0.0008) [2023-12-27 16:08:36,843][20347] Updated weights for policy 0, policy_version 1515140 (0.0008) [2023-12-27 16:08:36,897][20347] Updated weights for policy 0, policy_version 1515150 (0.0008) [2023-12-27 16:08:36,938][20381] Updated weights for policy 1, policy_version 1513522 (0.0008) [2023-12-27 16:08:36,948][20347] Updated weights for policy 0, policy_version 1515160 (0.0007) [2023-12-27 16:08:36,990][20381] Updated weights for policy 1, policy_version 1513532 (0.0007) [2023-12-27 16:08:37,040][20381] Updated weights for policy 1, policy_version 1513542 (0.0008) [2023-12-27 16:08:37,097][20381] Updated weights for policy 1, policy_version 1513552 (0.0008) [2023-12-27 16:08:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 775462912. Throughput: 0: 10291.9, 1: 10361.7. Samples: 775431228. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:37,363][19170] Avg episode reward: [(0, '970.590'), (1, '1000.000')] [2023-12-27 16:08:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001513552_387506176.pth... [2023-12-27 16:08:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001515160_387956736.pth... [2023-12-27 16:08:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001512336_387194880.pth [2023-12-27 16:08:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001513944_387645440.pth [2023-12-27 16:08:37,629][20347] Updated weights for policy 0, policy_version 1515170 (0.0007) [2023-12-27 16:08:37,679][20347] Updated weights for policy 0, policy_version 1515180 (0.0008) [2023-12-27 16:08:37,724][20347] Updated weights for policy 0, policy_version 1515190 (0.0007) [2023-12-27 16:08:37,793][20381] Updated weights for policy 1, policy_version 1513562 (0.0010) [2023-12-27 16:08:37,857][20381] Updated weights for policy 1, policy_version 1513572 (0.0010) [2023-12-27 16:08:37,926][20381] Updated weights for policy 1, policy_version 1513582 (0.0008) [2023-12-27 16:08:38,410][20347] Updated weights for policy 0, policy_version 1515200 (0.0007) [2023-12-27 16:08:38,469][20347] Updated weights for policy 0, policy_version 1515210 (0.0007) [2023-12-27 16:08:38,527][20347] Updated weights for policy 0, policy_version 1515220 (0.0007) [2023-12-27 16:08:38,630][20381] Updated weights for policy 1, policy_version 1513592 (0.0008) [2023-12-27 16:08:38,692][20381] Updated weights for policy 1, policy_version 1513602 (0.0010) [2023-12-27 16:08:38,759][20381] Updated weights for policy 1, policy_version 1513612 (0.0008) [2023-12-27 16:08:39,164][20347] Updated weights for policy 0, policy_version 1515230 (0.0008) [2023-12-27 16:08:39,221][20347] Updated weights for policy 0, policy_version 1515240 (0.0008) [2023-12-27 16:08:39,286][20347] Updated weights for policy 0, policy_version 1515250 (0.0008) [2023-12-27 16:08:39,457][20381] Updated weights for policy 1, policy_version 1513622 (0.0007) [2023-12-27 16:08:39,513][20381] Updated weights for policy 1, policy_version 1513632 (0.0008) [2023-12-27 16:08:39,566][20381] Updated weights for policy 1, policy_version 1513642 (0.0008) [2023-12-27 16:08:39,968][20347] Updated weights for policy 0, policy_version 1515260 (0.0008) [2023-12-27 16:08:40,024][20347] Updated weights for policy 0, policy_version 1515270 (0.0008) [2023-12-27 16:08:40,078][20347] Updated weights for policy 0, policy_version 1515280 (0.0008) [2023-12-27 16:08:40,396][20381] Updated weights for policy 1, policy_version 1513652 (0.0007) [2023-12-27 16:08:40,465][20381] Updated weights for policy 1, policy_version 1513662 (0.0009) [2023-12-27 16:08:40,524][20381] Updated weights for policy 1, policy_version 1513672 (0.0010) [2023-12-27 16:08:40,774][20347] Updated weights for policy 0, policy_version 1515290 (0.0008) [2023-12-27 16:08:40,826][20347] Updated weights for policy 0, policy_version 1515300 (0.0008) [2023-12-27 16:08:40,878][20347] Updated weights for policy 0, policy_version 1515310 (0.0007) [2023-12-27 16:08:40,940][20347] Updated weights for policy 0, policy_version 1515320 (0.0008) [2023-12-27 16:08:41,187][20381] Updated weights for policy 1, policy_version 1513682 (0.0010) [2023-12-27 16:08:41,255][20381] Updated weights for policy 1, policy_version 1513692 (0.0007) [2023-12-27 16:08:41,322][20381] Updated weights for policy 1, policy_version 1513702 (0.0011) [2023-12-27 16:08:41,393][20381] Updated weights for policy 1, policy_version 1513712 (0.0009) [2023-12-27 16:08:41,732][20347] Updated weights for policy 0, policy_version 1515330 (0.0008) [2023-12-27 16:08:41,800][20347] Updated weights for policy 0, policy_version 1515340 (0.0007) [2023-12-27 16:08:41,866][20347] Updated weights for policy 0, policy_version 1515350 (0.0006) [2023-12-27 16:08:42,078][20381] Updated weights for policy 1, policy_version 1513722 (0.0007) [2023-12-27 16:08:42,142][20381] Updated weights for policy 1, policy_version 1513732 (0.0006) [2023-12-27 16:08:42,199][20381] Updated weights for policy 1, policy_version 1513742 (0.0008) [2023-12-27 16:08:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 775561216. Throughput: 0: 10214.7, 1: 10293.5. Samples: 775552252. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:42,363][19170] Avg episode reward: [(0, '950.990'), (1, '980.580')] [2023-12-27 16:08:42,549][20347] Updated weights for policy 0, policy_version 1515360 (0.0008) [2023-12-27 16:08:42,606][20347] Updated weights for policy 0, policy_version 1515370 (0.0008) [2023-12-27 16:08:42,662][20347] Updated weights for policy 0, policy_version 1515380 (0.0008) [2023-12-27 16:08:42,849][20381] Updated weights for policy 1, policy_version 1513752 (0.0008) [2023-12-27 16:08:42,911][20381] Updated weights for policy 1, policy_version 1513762 (0.0009) [2023-12-27 16:08:42,971][20381] Updated weights for policy 1, policy_version 1513772 (0.0005) [2023-12-27 16:08:43,323][20347] Updated weights for policy 0, policy_version 1515390 (0.0008) [2023-12-27 16:08:43,381][20347] Updated weights for policy 0, policy_version 1515400 (0.0007) [2023-12-27 16:08:43,439][20347] Updated weights for policy 0, policy_version 1515410 (0.0008) [2023-12-27 16:08:43,681][20381] Updated weights for policy 1, policy_version 1513782 (0.0007) [2023-12-27 16:08:43,735][20381] Updated weights for policy 1, policy_version 1513792 (0.0009) [2023-12-27 16:08:43,783][20381] Updated weights for policy 1, policy_version 1513802 (0.0009) [2023-12-27 16:08:44,135][20347] Updated weights for policy 0, policy_version 1515420 (0.0007) [2023-12-27 16:08:44,192][20347] Updated weights for policy 0, policy_version 1515430 (0.0007) [2023-12-27 16:08:44,249][20347] Updated weights for policy 0, policy_version 1515440 (0.0008) [2023-12-27 16:08:44,492][20381] Updated weights for policy 1, policy_version 1513812 (0.0010) [2023-12-27 16:08:44,552][20381] Updated weights for policy 1, policy_version 1513822 (0.0010) [2023-12-27 16:08:44,607][20381] Updated weights for policy 1, policy_version 1513832 (0.0007) [2023-12-27 16:08:44,824][20347] Updated weights for policy 0, policy_version 1515450 (0.0005) [2023-12-27 16:08:44,881][20347] Updated weights for policy 0, policy_version 1515460 (0.0007) [2023-12-27 16:08:44,941][20347] Updated weights for policy 0, policy_version 1515470 (0.0008) [2023-12-27 16:08:44,994][20347] Updated weights for policy 0, policy_version 1515480 (0.0008) [2023-12-27 16:08:45,268][20381] Updated weights for policy 1, policy_version 1513842 (0.0006) [2023-12-27 16:08:45,331][20381] Updated weights for policy 1, policy_version 1513852 (0.0007) [2023-12-27 16:08:45,388][20381] Updated weights for policy 1, policy_version 1513862 (0.0008) [2023-12-27 16:08:45,434][20381] Updated weights for policy 1, policy_version 1513872 (0.0008) [2023-12-27 16:08:45,702][20347] Updated weights for policy 0, policy_version 1515490 (0.0008) [2023-12-27 16:08:45,752][20347] Updated weights for policy 0, policy_version 1515500 (0.0008) [2023-12-27 16:08:45,799][20347] Updated weights for policy 0, policy_version 1515510 (0.0008) [2023-12-27 16:08:46,100][20381] Updated weights for policy 1, policy_version 1513882 (0.0005) [2023-12-27 16:08:46,163][20381] Updated weights for policy 1, policy_version 1513892 (0.0005) [2023-12-27 16:08:46,215][20381] Updated weights for policy 1, policy_version 1513902 (0.0005) [2023-12-27 16:08:46,537][20347] Updated weights for policy 0, policy_version 1515520 (0.0006) [2023-12-27 16:08:46,588][20347] Updated weights for policy 0, policy_version 1515530 (0.0005) [2023-12-27 16:08:46,642][20347] Updated weights for policy 0, policy_version 1515540 (0.0005) [2023-12-27 16:08:46,861][20381] Updated weights for policy 1, policy_version 1513912 (0.0007) [2023-12-27 16:08:46,913][20381] Updated weights for policy 1, policy_version 1513922 (0.0007) [2023-12-27 16:08:46,958][20381] Updated weights for policy 1, policy_version 1513932 (0.0005) [2023-12-27 16:08:47,282][20347] Updated weights for policy 0, policy_version 1515550 (0.0007) [2023-12-27 16:08:47,330][20347] Updated weights for policy 0, policy_version 1515560 (0.0008) [2023-12-27 16:08:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 775659520. Throughput: 0: 10259.0, 1: 10292.0. Samples: 775674712. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:47,364][19170] Avg episode reward: [(0, '921.740'), (1, '980.580')] [2023-12-27 16:08:47,388][20347] Updated weights for policy 0, policy_version 1515570 (0.0008) [2023-12-27 16:08:47,582][20381] Updated weights for policy 1, policy_version 1513942 (0.0006) [2023-12-27 16:08:47,641][20381] Updated weights for policy 1, policy_version 1513952 (0.0007) [2023-12-27 16:08:47,696][20381] Updated weights for policy 1, policy_version 1513962 (0.0008) [2023-12-27 16:08:48,038][20347] Updated weights for policy 0, policy_version 1515580 (0.0007) [2023-12-27 16:08:48,090][20347] Updated weights for policy 0, policy_version 1515590 (0.0006) [2023-12-27 16:08:48,138][20347] Updated weights for policy 0, policy_version 1515600 (0.0008) [2023-12-27 16:08:48,352][20381] Updated weights for policy 1, policy_version 1513972 (0.0009) [2023-12-27 16:08:48,412][20381] Updated weights for policy 1, policy_version 1513982 (0.0005) [2023-12-27 16:08:48,472][20381] Updated weights for policy 1, policy_version 1513992 (0.0006) [2023-12-27 16:08:48,777][20347] Updated weights for policy 0, policy_version 1515610 (0.0006) [2023-12-27 16:08:48,830][20347] Updated weights for policy 0, policy_version 1515620 (0.0010) [2023-12-27 16:08:48,883][20347] Updated weights for policy 0, policy_version 1515630 (0.0010) [2023-12-27 16:08:48,941][20347] Updated weights for policy 0, policy_version 1515640 (0.0010) [2023-12-27 16:08:49,030][20381] Updated weights for policy 1, policy_version 1514002 (0.0006) [2023-12-27 16:08:49,080][20381] Updated weights for policy 1, policy_version 1514012 (0.0007) [2023-12-27 16:08:49,141][20381] Updated weights for policy 1, policy_version 1514022 (0.0008) [2023-12-27 16:08:49,189][20381] Updated weights for policy 1, policy_version 1514032 (0.0007) [2023-12-27 16:08:49,634][20347] Updated weights for policy 0, policy_version 1515650 (0.0010) [2023-12-27 16:08:49,688][20347] Updated weights for policy 0, policy_version 1515660 (0.0011) [2023-12-27 16:08:49,740][20347] Updated weights for policy 0, policy_version 1515670 (0.0010) [2023-12-27 16:08:49,840][20381] Updated weights for policy 1, policy_version 1514042 (0.0007) [2023-12-27 16:08:49,898][20381] Updated weights for policy 1, policy_version 1514052 (0.0008) [2023-12-27 16:08:49,961][20381] Updated weights for policy 1, policy_version 1514062 (0.0008) [2023-12-27 16:08:50,430][20347] Updated weights for policy 0, policy_version 1515680 (0.0010) [2023-12-27 16:08:50,479][20347] Updated weights for policy 0, policy_version 1515690 (0.0010) [2023-12-27 16:08:50,539][20347] Updated weights for policy 0, policy_version 1515700 (0.0006) [2023-12-27 16:08:50,660][20381] Updated weights for policy 1, policy_version 1514072 (0.0008) [2023-12-27 16:08:50,712][20381] Updated weights for policy 1, policy_version 1514082 (0.0008) [2023-12-27 16:08:50,759][20381] Updated weights for policy 1, policy_version 1514092 (0.0008) [2023-12-27 16:08:51,281][20347] Updated weights for policy 0, policy_version 1515710 (0.0008) [2023-12-27 16:08:51,343][20347] Updated weights for policy 0, policy_version 1515720 (0.0007) [2023-12-27 16:08:51,408][20347] Updated weights for policy 0, policy_version 1515730 (0.0008) [2023-12-27 16:08:51,540][20381] Updated weights for policy 1, policy_version 1514102 (0.0007) [2023-12-27 16:08:51,590][20381] Updated weights for policy 1, policy_version 1514112 (0.0008) [2023-12-27 16:08:51,651][20381] Updated weights for policy 1, policy_version 1514122 (0.0008) [2023-12-27 16:08:52,166][20347] Updated weights for policy 0, policy_version 1515740 (0.0008) [2023-12-27 16:08:52,217][20347] Updated weights for policy 0, policy_version 1515750 (0.0008) [2023-12-27 16:08:52,277][20347] Updated weights for policy 0, policy_version 1515760 (0.0007) [2023-12-27 16:08:52,303][20381] Updated weights for policy 1, policy_version 1514132 (0.0010) [2023-12-27 16:08:52,361][20381] Updated weights for policy 1, policy_version 1514142 (0.0007) [2023-12-27 16:08:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 775766016. Throughput: 0: 10256.2, 1: 10310.8. Samples: 775737512. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:52,363][19170] Avg episode reward: [(0, '951.140'), (1, '980.550')] [2023-12-27 16:08:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001515768_388112384.pth... [2023-12-27 16:08:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001514552_387801088.pth [2023-12-27 16:08:52,424][20381] Updated weights for policy 1, policy_version 1514152 (0.0008) [2023-12-27 16:08:52,459][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001514160_387661824.pth... [2023-12-27 16:08:52,462][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001512944_387350528.pth [2023-12-27 16:08:52,985][20347] Updated weights for policy 0, policy_version 1515770 (0.0006) [2023-12-27 16:08:53,036][20347] Updated weights for policy 0, policy_version 1515780 (0.0007) [2023-12-27 16:08:53,084][20347] Updated weights for policy 0, policy_version 1515790 (0.0007) [2023-12-27 16:08:53,132][20381] Updated weights for policy 1, policy_version 1514162 (0.0008) [2023-12-27 16:08:53,134][20347] Updated weights for policy 0, policy_version 1515800 (0.0008) [2023-12-27 16:08:53,195][20381] Updated weights for policy 1, policy_version 1514172 (0.0005) [2023-12-27 16:08:53,244][20381] Updated weights for policy 1, policy_version 1514182 (0.0007) [2023-12-27 16:08:53,287][20381] Updated weights for policy 1, policy_version 1514192 (0.0007) [2023-12-27 16:08:53,852][20347] Updated weights for policy 0, policy_version 1515810 (0.0008) [2023-12-27 16:08:53,911][20347] Updated weights for policy 0, policy_version 1515820 (0.0008) [2023-12-27 16:08:53,920][20381] Updated weights for policy 1, policy_version 1514202 (0.0007) [2023-12-27 16:08:53,967][20347] Updated weights for policy 0, policy_version 1515830 (0.0006) [2023-12-27 16:08:53,969][20381] Updated weights for policy 1, policy_version 1514212 (0.0007) [2023-12-27 16:08:54,014][20381] Updated weights for policy 1, policy_version 1514222 (0.0005) [2023-12-27 16:08:54,635][20381] Updated weights for policy 1, policy_version 1514232 (0.0009) [2023-12-27 16:08:54,662][20347] Updated weights for policy 0, policy_version 1515840 (0.0006) [2023-12-27 16:08:54,690][20381] Updated weights for policy 1, policy_version 1514242 (0.0011) [2023-12-27 16:08:54,717][20347] Updated weights for policy 0, policy_version 1515850 (0.0005) [2023-12-27 16:08:54,746][20381] Updated weights for policy 1, policy_version 1514252 (0.0011) [2023-12-27 16:08:54,763][20347] Updated weights for policy 0, policy_version 1515860 (0.0005) [2023-12-27 16:08:55,367][20347] Updated weights for policy 0, policy_version 1515870 (0.0008) [2023-12-27 16:08:55,422][20347] Updated weights for policy 0, policy_version 1515880 (0.0010) [2023-12-27 16:08:55,428][20381] Updated weights for policy 1, policy_version 1514262 (0.0008) [2023-12-27 16:08:55,476][20381] Updated weights for policy 1, policy_version 1514272 (0.0006) [2023-12-27 16:08:55,476][20347] Updated weights for policy 0, policy_version 1515890 (0.0010) [2023-12-27 16:08:55,525][20381] Updated weights for policy 1, policy_version 1514282 (0.0007) [2023-12-27 16:08:56,124][20381] Updated weights for policy 1, policy_version 1514292 (0.0008) [2023-12-27 16:08:56,166][20347] Updated weights for policy 0, policy_version 1515900 (0.0010) [2023-12-27 16:08:56,185][20381] Updated weights for policy 1, policy_version 1514302 (0.0006) [2023-12-27 16:08:56,223][20347] Updated weights for policy 0, policy_version 1515910 (0.0007) [2023-12-27 16:08:56,238][20381] Updated weights for policy 1, policy_version 1514312 (0.0006) [2023-12-27 16:08:56,268][20347] Updated weights for policy 0, policy_version 1515920 (0.0005) [2023-12-27 16:08:56,801][20381] Updated weights for policy 1, policy_version 1514322 (0.0006) [2023-12-27 16:08:56,848][20381] Updated weights for policy 1, policy_version 1514332 (0.0005) [2023-12-27 16:08:56,905][20381] Updated weights for policy 1, policy_version 1514342 (0.0005) [2023-12-27 16:08:56,956][20381] Updated weights for policy 1, policy_version 1514352 (0.0005) [2023-12-27 16:08:57,001][20347] Updated weights for policy 0, policy_version 1515930 (0.0007) [2023-12-27 16:08:57,060][20347] Updated weights for policy 0, policy_version 1515940 (0.0008) [2023-12-27 16:08:57,115][20347] Updated weights for policy 0, policy_version 1515950 (0.0008) [2023-12-27 16:08:57,168][20347] Updated weights for policy 0, policy_version 1515960 (0.0005) [2023-12-27 16:08:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 775872512. Throughput: 0: 10270.8, 1: 10360.0. Samples: 775861096. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:08:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '980.550')] [2023-12-27 16:08:57,534][20381] Updated weights for policy 1, policy_version 1514362 (0.0005) [2023-12-27 16:08:57,593][20381] Updated weights for policy 1, policy_version 1514372 (0.0005) [2023-12-27 16:08:57,644][20381] Updated weights for policy 1, policy_version 1514382 (0.0005) [2023-12-27 16:08:57,881][20347] Updated weights for policy 0, policy_version 1515970 (0.0007) [2023-12-27 16:08:57,944][20347] Updated weights for policy 0, policy_version 1515980 (0.0008) [2023-12-27 16:08:58,019][20347] Updated weights for policy 0, policy_version 1515990 (0.0009) [2023-12-27 16:08:58,207][20381] Updated weights for policy 1, policy_version 1514392 (0.0008) [2023-12-27 16:08:58,270][20381] Updated weights for policy 1, policy_version 1514402 (0.0008) [2023-12-27 16:08:58,328][20381] Updated weights for policy 1, policy_version 1514412 (0.0008) [2023-12-27 16:08:58,736][20347] Updated weights for policy 0, policy_version 1516000 (0.0007) [2023-12-27 16:08:58,800][20347] Updated weights for policy 0, policy_version 1516010 (0.0007) [2023-12-27 16:08:58,866][20347] Updated weights for policy 0, policy_version 1516020 (0.0008) [2023-12-27 16:08:59,037][20381] Updated weights for policy 1, policy_version 1514422 (0.0008) [2023-12-27 16:08:59,089][20381] Updated weights for policy 1, policy_version 1514432 (0.0008) [2023-12-27 16:08:59,154][20381] Updated weights for policy 1, policy_version 1514442 (0.0008) [2023-12-27 16:08:59,564][20347] Updated weights for policy 0, policy_version 1516030 (0.0007) [2023-12-27 16:08:59,629][20347] Updated weights for policy 0, policy_version 1516040 (0.0007) [2023-12-27 16:08:59,682][20347] Updated weights for policy 0, policy_version 1516050 (0.0007) [2023-12-27 16:08:59,797][20381] Updated weights for policy 1, policy_version 1514452 (0.0007) [2023-12-27 16:08:59,859][20381] Updated weights for policy 1, policy_version 1514462 (0.0007) [2023-12-27 16:08:59,916][20381] Updated weights for policy 1, policy_version 1514472 (0.0005) [2023-12-27 16:09:00,352][20347] Updated weights for policy 0, policy_version 1516060 (0.0007) [2023-12-27 16:09:00,406][20347] Updated weights for policy 0, policy_version 1516070 (0.0010) [2023-12-27 16:09:00,466][20347] Updated weights for policy 0, policy_version 1516080 (0.0009) [2023-12-27 16:09:00,585][20381] Updated weights for policy 1, policy_version 1514482 (0.0008) [2023-12-27 16:09:00,639][20381] Updated weights for policy 1, policy_version 1514492 (0.0006) [2023-12-27 16:09:00,696][20381] Updated weights for policy 1, policy_version 1514502 (0.0005) [2023-12-27 16:09:00,749][20381] Updated weights for policy 1, policy_version 1514512 (0.0007) [2023-12-27 16:09:01,136][20347] Updated weights for policy 0, policy_version 1516090 (0.0009) [2023-12-27 16:09:01,199][20347] Updated weights for policy 0, policy_version 1516100 (0.0009) [2023-12-27 16:09:01,261][20347] Updated weights for policy 0, policy_version 1516110 (0.0008) [2023-12-27 16:09:01,318][20347] Updated weights for policy 0, policy_version 1516120 (0.0007) [2023-12-27 16:09:01,460][20381] Updated weights for policy 1, policy_version 1514522 (0.0011) [2023-12-27 16:09:01,513][20381] Updated weights for policy 1, policy_version 1514532 (0.0010) [2023-12-27 16:09:01,562][20381] Updated weights for policy 1, policy_version 1514542 (0.0010) [2023-12-27 16:09:02,115][20347] Updated weights for policy 0, policy_version 1516130 (0.0008) [2023-12-27 16:09:02,168][20347] Updated weights for policy 0, policy_version 1516140 (0.0006) [2023-12-27 16:09:02,174][20381] Updated weights for policy 1, policy_version 1514552 (0.0009) [2023-12-27 16:09:02,223][20347] Updated weights for policy 0, policy_version 1516150 (0.0006) [2023-12-27 16:09:02,225][20381] Updated weights for policy 1, policy_version 1514562 (0.0006) [2023-12-27 16:09:02,285][20381] Updated weights for policy 1, policy_version 1514572 (0.0008) [2023-12-27 16:09:02,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 775979008. Throughput: 0: 10177.3, 1: 10377.6. Samples: 775983200. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '990.240')] [2023-12-27 16:09:02,978][20381] Updated weights for policy 1, policy_version 1514582 (0.0006) [2023-12-27 16:09:02,985][20347] Updated weights for policy 0, policy_version 1516160 (0.0007) [2023-12-27 16:09:03,036][20347] Updated weights for policy 0, policy_version 1516170 (0.0007) [2023-12-27 16:09:03,038][20381] Updated weights for policy 1, policy_version 1514592 (0.0005) [2023-12-27 16:09:03,068][20292] KL-divergence is very high: 168.2760 [2023-12-27 16:09:03,081][20292] KL-divergence is very high: 117.1681 [2023-12-27 16:09:03,088][20347] Updated weights for policy 0, policy_version 1516180 (0.0007) [2023-12-27 16:09:03,096][20381] Updated weights for policy 1, policy_version 1514602 (0.0005) [2023-12-27 16:09:03,593][20381] Updated weights for policy 1, policy_version 1514612 (0.0005) [2023-12-27 16:09:03,642][20381] Updated weights for policy 1, policy_version 1514622 (0.0007) [2023-12-27 16:09:03,694][20381] Updated weights for policy 1, policy_version 1514632 (0.0007) [2023-12-27 16:09:03,763][20347] Updated weights for policy 0, policy_version 1516190 (0.0009) [2023-12-27 16:09:03,822][20347] Updated weights for policy 0, policy_version 1516200 (0.0010) [2023-12-27 16:09:03,881][20347] Updated weights for policy 0, policy_version 1516210 (0.0010) [2023-12-27 16:09:04,351][20381] Updated weights for policy 1, policy_version 1514642 (0.0010) [2023-12-27 16:09:04,407][20381] Updated weights for policy 1, policy_version 1514652 (0.0011) [2023-12-27 16:09:04,462][20381] Updated weights for policy 1, policy_version 1514662 (0.0011) [2023-12-27 16:09:04,525][20381] Updated weights for policy 1, policy_version 1514672 (0.0011) [2023-12-27 16:09:04,613][20347] Updated weights for policy 0, policy_version 1516220 (0.0009) [2023-12-27 16:09:04,672][20347] Updated weights for policy 0, policy_version 1516230 (0.0006) [2023-12-27 16:09:04,729][20347] Updated weights for policy 0, policy_version 1516240 (0.0007) [2023-12-27 16:09:05,197][20381] Updated weights for policy 1, policy_version 1514682 (0.0006) [2023-12-27 16:09:05,244][20381] Updated weights for policy 1, policy_version 1514692 (0.0006) [2023-12-27 16:09:05,304][20381] Updated weights for policy 1, policy_version 1514702 (0.0010) [2023-12-27 16:09:05,446][20347] Updated weights for policy 0, policy_version 1516250 (0.0007) [2023-12-27 16:09:05,500][20347] Updated weights for policy 0, policy_version 1516260 (0.0007) [2023-12-27 16:09:05,548][20347] Updated weights for policy 0, policy_version 1516270 (0.0007) [2023-12-27 16:09:05,595][20347] Updated weights for policy 0, policy_version 1516280 (0.0007) [2023-12-27 16:09:05,888][20381] Updated weights for policy 1, policy_version 1514712 (0.0010) [2023-12-27 16:09:05,890][20292] KL-divergence is very high: 105.2554 [2023-12-27 16:09:05,931][20292] KL-divergence is very high: 123.5954 [2023-12-27 16:09:05,943][20381] Updated weights for policy 1, policy_version 1514722 (0.0008) [2023-12-27 16:09:05,992][20381] Updated weights for policy 1, policy_version 1514732 (0.0008) [2023-12-27 16:09:06,340][20347] Updated weights for policy 0, policy_version 1516290 (0.0008) [2023-12-27 16:09:06,407][20347] Updated weights for policy 0, policy_version 1516300 (0.0007) [2023-12-27 16:09:06,470][20347] Updated weights for policy 0, policy_version 1516310 (0.0007) [2023-12-27 16:09:06,705][20381] Updated weights for policy 1, policy_version 1514742 (0.0006) [2023-12-27 16:09:06,762][20381] Updated weights for policy 1, policy_version 1514752 (0.0007) [2023-12-27 16:09:06,819][20381] Updated weights for policy 1, policy_version 1514762 (0.0007) [2023-12-27 16:09:07,145][20347] Updated weights for policy 0, policy_version 1516320 (0.0006) [2023-12-27 16:09:07,208][20347] Updated weights for policy 0, policy_version 1516330 (0.0005) [2023-12-27 16:09:07,263][20347] Updated weights for policy 0, policy_version 1516340 (0.0005) [2023-12-27 16:09:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 776077312. Throughput: 0: 10120.4, 1: 10424.8. Samples: 776045236. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:07,364][19170] Avg episode reward: [(0, '990.230'), (1, '953.120')] [2023-12-27 16:09:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001514768_387817472.pth... [2023-12-27 16:09:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001516344_388259840.pth... [2023-12-27 16:09:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001513552_387506176.pth [2023-12-27 16:09:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001515160_387956736.pth [2023-12-27 16:09:07,455][20381] Updated weights for policy 1, policy_version 1514772 (0.0007) [2023-12-27 16:09:07,519][20381] Updated weights for policy 1, policy_version 1514782 (0.0010) [2023-12-27 16:09:07,578][20381] Updated weights for policy 1, policy_version 1514792 (0.0011) [2023-12-27 16:09:07,856][20347] Updated weights for policy 0, policy_version 1516350 (0.0007) [2023-12-27 16:09:07,916][20347] Updated weights for policy 0, policy_version 1516360 (0.0008) [2023-12-27 16:09:07,973][20347] Updated weights for policy 0, policy_version 1516370 (0.0008) [2023-12-27 16:09:08,153][20381] Updated weights for policy 1, policy_version 1514802 (0.0009) [2023-12-27 16:09:08,210][20381] Updated weights for policy 1, policy_version 1514812 (0.0007) [2023-12-27 16:09:08,263][20381] Updated weights for policy 1, policy_version 1514822 (0.0007) [2023-12-27 16:09:08,311][20381] Updated weights for policy 1, policy_version 1514832 (0.0007) [2023-12-27 16:09:08,696][20347] Updated weights for policy 0, policy_version 1516380 (0.0009) [2023-12-27 16:09:08,756][20347] Updated weights for policy 0, policy_version 1516390 (0.0008) [2023-12-27 16:09:08,816][20347] Updated weights for policy 0, policy_version 1516400 (0.0008) [2023-12-27 16:09:08,972][20381] Updated weights for policy 1, policy_version 1514842 (0.0008) [2023-12-27 16:09:09,020][20381] Updated weights for policy 1, policy_version 1514852 (0.0007) [2023-12-27 16:09:09,079][20381] Updated weights for policy 1, policy_version 1514862 (0.0008) [2023-12-27 16:09:09,566][20347] Updated weights for policy 0, policy_version 1516410 (0.0008) [2023-12-27 16:09:09,631][20347] Updated weights for policy 0, policy_version 1516420 (0.0009) [2023-12-27 16:09:09,693][20347] Updated weights for policy 0, policy_version 1516430 (0.0008) [2023-12-27 16:09:09,752][20381] Updated weights for policy 1, policy_version 1514872 (0.0006) [2023-12-27 16:09:09,756][20347] Updated weights for policy 0, policy_version 1516440 (0.0007) [2023-12-27 16:09:09,815][20381] Updated weights for policy 1, policy_version 1514882 (0.0006) [2023-12-27 16:09:09,884][20381] Updated weights for policy 1, policy_version 1514892 (0.0007) [2023-12-27 16:09:10,522][20347] Updated weights for policy 0, policy_version 1516450 (0.0008) [2023-12-27 16:09:10,570][20381] Updated weights for policy 1, policy_version 1514902 (0.0010) [2023-12-27 16:09:10,584][20347] Updated weights for policy 0, policy_version 1516460 (0.0006) [2023-12-27 16:09:10,624][20381] Updated weights for policy 1, policy_version 1514912 (0.0010) [2023-12-27 16:09:10,638][20347] Updated weights for policy 0, policy_version 1516470 (0.0006) [2023-12-27 16:09:10,674][20381] Updated weights for policy 1, policy_version 1514922 (0.0010) [2023-12-27 16:09:11,389][20381] Updated weights for policy 1, policy_version 1514932 (0.0010) [2023-12-27 16:09:11,408][20347] Updated weights for policy 0, policy_version 1516480 (0.0008) [2023-12-27 16:09:11,450][20381] Updated weights for policy 1, policy_version 1514942 (0.0006) [2023-12-27 16:09:11,467][20347] Updated weights for policy 0, policy_version 1516490 (0.0011) [2023-12-27 16:09:11,503][20381] Updated weights for policy 1, policy_version 1514952 (0.0007) [2023-12-27 16:09:11,530][20347] Updated weights for policy 0, policy_version 1516500 (0.0009) [2023-12-27 16:09:12,208][20381] Updated weights for policy 1, policy_version 1514962 (0.0009) [2023-12-27 16:09:12,266][20381] Updated weights for policy 1, policy_version 1514972 (0.0006) [2023-12-27 16:09:12,326][20381] Updated weights for policy 1, policy_version 1514982 (0.0007) [2023-12-27 16:09:12,327][20347] Updated weights for policy 0, policy_version 1516510 (0.0009) [2023-12-27 16:09:12,362][19170] Fps is (10 sec: 18841.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776167424. Throughput: 0: 10096.1, 1: 10421.9. Samples: 776166736. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:12,364][19170] Avg episode reward: [(0, '990.220'), (1, '980.490')] [2023-12-27 16:09:12,392][20381] Updated weights for policy 1, policy_version 1514992 (0.0008) [2023-12-27 16:09:12,393][20347] Updated weights for policy 0, policy_version 1516520 (0.0007) [2023-12-27 16:09:12,457][20347] Updated weights for policy 0, policy_version 1516530 (0.0008) [2023-12-27 16:09:13,002][20381] Updated weights for policy 1, policy_version 1515002 (0.0008) [2023-12-27 16:09:13,057][20381] Updated weights for policy 1, policy_version 1515012 (0.0008) [2023-12-27 16:09:13,123][20381] Updated weights for policy 1, policy_version 1515022 (0.0007) [2023-12-27 16:09:13,165][20347] Updated weights for policy 0, policy_version 1516540 (0.0008) [2023-12-27 16:09:13,227][20347] Updated weights for policy 0, policy_version 1516550 (0.0007) [2023-12-27 16:09:13,284][20347] Updated weights for policy 0, policy_version 1516560 (0.0008) [2023-12-27 16:09:13,785][20381] Updated weights for policy 1, policy_version 1515032 (0.0005) [2023-12-27 16:09:13,831][20381] Updated weights for policy 1, policy_version 1515042 (0.0005) [2023-12-27 16:09:13,880][20381] Updated weights for policy 1, policy_version 1515052 (0.0005) [2023-12-27 16:09:13,964][20347] Updated weights for policy 0, policy_version 1516570 (0.0008) [2023-12-27 16:09:14,018][20347] Updated weights for policy 0, policy_version 1516580 (0.0005) [2023-12-27 16:09:14,068][20347] Updated weights for policy 0, policy_version 1516590 (0.0005) [2023-12-27 16:09:14,121][20347] Updated weights for policy 0, policy_version 1516600 (0.0005) [2023-12-27 16:09:14,546][20381] Updated weights for policy 1, policy_version 1515062 (0.0005) [2023-12-27 16:09:14,609][20381] Updated weights for policy 1, policy_version 1515072 (0.0006) [2023-12-27 16:09:14,669][20381] Updated weights for policy 1, policy_version 1515082 (0.0009) [2023-12-27 16:09:14,683][20347] Updated weights for policy 0, policy_version 1516610 (0.0010) [2023-12-27 16:09:14,729][20347] Updated weights for policy 0, policy_version 1516620 (0.0005) [2023-12-27 16:09:14,789][20347] Updated weights for policy 0, policy_version 1516630 (0.0007) [2023-12-27 16:09:15,327][20381] Updated weights for policy 1, policy_version 1515092 (0.0008) [2023-12-27 16:09:15,386][20381] Updated weights for policy 1, policy_version 1515102 (0.0008) [2023-12-27 16:09:15,434][20381] Updated weights for policy 1, policy_version 1515112 (0.0007) [2023-12-27 16:09:15,483][20347] Updated weights for policy 0, policy_version 1516640 (0.0009) [2023-12-27 16:09:15,534][20347] Updated weights for policy 0, policy_version 1516650 (0.0010) [2023-12-27 16:09:15,588][20347] Updated weights for policy 0, policy_version 1516660 (0.0010) [2023-12-27 16:09:16,118][20381] Updated weights for policy 1, policy_version 1515122 (0.0006) [2023-12-27 16:09:16,168][20381] Updated weights for policy 1, policy_version 1515132 (0.0005) [2023-12-27 16:09:16,220][20381] Updated weights for policy 1, policy_version 1515142 (0.0005) [2023-12-27 16:09:16,278][20347] Updated weights for policy 0, policy_version 1516670 (0.0010) [2023-12-27 16:09:16,279][20381] Updated weights for policy 1, policy_version 1515152 (0.0008) [2023-12-27 16:09:16,337][20347] Updated weights for policy 0, policy_version 1516680 (0.0010) [2023-12-27 16:09:16,398][20347] Updated weights for policy 0, policy_version 1516690 (0.0007) [2023-12-27 16:09:16,879][20381] Updated weights for policy 1, policy_version 1515162 (0.0005) [2023-12-27 16:09:16,929][20381] Updated weights for policy 1, policy_version 1515172 (0.0006) [2023-12-27 16:09:16,989][20381] Updated weights for policy 1, policy_version 1515182 (0.0008) [2023-12-27 16:09:17,019][20347] Updated weights for policy 0, policy_version 1516700 (0.0010) [2023-12-27 16:09:17,079][20347] Updated weights for policy 0, policy_version 1516710 (0.0010) [2023-12-27 16:09:17,131][20347] Updated weights for policy 0, policy_version 1516720 (0.0010) [2023-12-27 16:09:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 776282112. Throughput: 0: 10089.0, 1: 10424.8. Samples: 776290272. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:17,364][19170] Avg episode reward: [(0, '990.220'), (1, '980.580')] [2023-12-27 16:09:17,627][20381] Updated weights for policy 1, policy_version 1515192 (0.0011) [2023-12-27 16:09:17,675][20381] Updated weights for policy 1, policy_version 1515202 (0.0010) [2023-12-27 16:09:17,727][20381] Updated weights for policy 1, policy_version 1515212 (0.0011) [2023-12-27 16:09:17,799][20347] Updated weights for policy 0, policy_version 1516730 (0.0009) [2023-12-27 16:09:17,859][20347] Updated weights for policy 0, policy_version 1516740 (0.0008) [2023-12-27 16:09:17,911][20347] Updated weights for policy 0, policy_version 1516750 (0.0008) [2023-12-27 16:09:17,962][20347] Updated weights for policy 0, policy_version 1516760 (0.0007) [2023-12-27 16:09:18,398][20381] Updated weights for policy 1, policy_version 1515222 (0.0008) [2023-12-27 16:09:18,460][20381] Updated weights for policy 1, policy_version 1515232 (0.0010) [2023-12-27 16:09:18,522][20381] Updated weights for policy 1, policy_version 1515242 (0.0010) [2023-12-27 16:09:18,592][20347] Updated weights for policy 0, policy_version 1516770 (0.0007) [2023-12-27 16:09:18,640][20347] Updated weights for policy 0, policy_version 1516780 (0.0009) [2023-12-27 16:09:18,694][20347] Updated weights for policy 0, policy_version 1516790 (0.0008) [2023-12-27 16:09:19,189][20381] Updated weights for policy 1, policy_version 1515252 (0.0010) [2023-12-27 16:09:19,246][20381] Updated weights for policy 1, policy_version 1515262 (0.0008) [2023-12-27 16:09:19,297][20381] Updated weights for policy 1, policy_version 1515272 (0.0006) [2023-12-27 16:09:19,460][20347] Updated weights for policy 0, policy_version 1516800 (0.0010) [2023-12-27 16:09:19,525][20347] Updated weights for policy 0, policy_version 1516810 (0.0010) [2023-12-27 16:09:19,585][20347] Updated weights for policy 0, policy_version 1516820 (0.0007) [2023-12-27 16:09:20,019][20381] Updated weights for policy 1, policy_version 1515282 (0.0008) [2023-12-27 16:09:20,076][20381] Updated weights for policy 1, policy_version 1515292 (0.0007) [2023-12-27 16:09:20,132][20381] Updated weights for policy 1, policy_version 1515302 (0.0007) [2023-12-27 16:09:20,191][20381] Updated weights for policy 1, policy_version 1515312 (0.0008) [2023-12-27 16:09:20,278][20347] Updated weights for policy 0, policy_version 1516830 (0.0007) [2023-12-27 16:09:20,339][20347] Updated weights for policy 0, policy_version 1516840 (0.0008) [2023-12-27 16:09:20,397][20347] Updated weights for policy 0, policy_version 1516850 (0.0008) [2023-12-27 16:09:20,867][20381] Updated weights for policy 1, policy_version 1515322 (0.0007) [2023-12-27 16:09:20,922][20381] Updated weights for policy 1, policy_version 1515332 (0.0008) [2023-12-27 16:09:20,975][20381] Updated weights for policy 1, policy_version 1515342 (0.0007) [2023-12-27 16:09:21,161][20347] Updated weights for policy 0, policy_version 1516860 (0.0008) [2023-12-27 16:09:21,222][20347] Updated weights for policy 0, policy_version 1516870 (0.0008) [2023-12-27 16:09:21,287][20347] Updated weights for policy 0, policy_version 1516880 (0.0008) [2023-12-27 16:09:21,771][20381] Updated weights for policy 1, policy_version 1515352 (0.0008) [2023-12-27 16:09:21,831][20381] Updated weights for policy 1, policy_version 1515362 (0.0008) [2023-12-27 16:09:21,884][20381] Updated weights for policy 1, policy_version 1515372 (0.0007) [2023-12-27 16:09:22,005][20347] Updated weights for policy 0, policy_version 1516890 (0.0007) [2023-12-27 16:09:22,058][20347] Updated weights for policy 0, policy_version 1516900 (0.0008) [2023-12-27 16:09:22,115][20347] Updated weights for policy 0, policy_version 1516910 (0.0005) [2023-12-27 16:09:22,176][20347] Updated weights for policy 0, policy_version 1516920 (0.0007) [2023-12-27 16:09:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 776380416. Throughput: 0: 10018.8, 1: 10411.9. Samples: 776350608. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:22,363][19170] Avg episode reward: [(0, '990.330'), (1, '980.580')] [2023-12-27 16:09:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001515376_387973120.pth... [2023-12-27 16:09:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001516920_388407296.pth... [2023-12-27 16:09:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001515768_388112384.pth [2023-12-27 16:09:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001514160_387661824.pth [2023-12-27 16:09:22,608][20381] Updated weights for policy 1, policy_version 1515382 (0.0008) [2023-12-27 16:09:22,674][20381] Updated weights for policy 1, policy_version 1515392 (0.0007) [2023-12-27 16:09:22,730][20381] Updated weights for policy 1, policy_version 1515402 (0.0007) [2023-12-27 16:09:22,867][20347] Updated weights for policy 0, policy_version 1516930 (0.0011) [2023-12-27 16:09:22,920][20347] Updated weights for policy 0, policy_version 1516940 (0.0010) [2023-12-27 16:09:22,971][20347] Updated weights for policy 0, policy_version 1516950 (0.0010) [2023-12-27 16:09:23,390][20381] Updated weights for policy 1, policy_version 1515412 (0.0007) [2023-12-27 16:09:23,437][20381] Updated weights for policy 1, policy_version 1515422 (0.0008) [2023-12-27 16:09:23,481][20381] Updated weights for policy 1, policy_version 1515432 (0.0007) [2023-12-27 16:09:23,631][20347] Updated weights for policy 0, policy_version 1516960 (0.0006) [2023-12-27 16:09:23,676][20347] Updated weights for policy 0, policy_version 1516970 (0.0005) [2023-12-27 16:09:23,731][20347] Updated weights for policy 0, policy_version 1516980 (0.0008) [2023-12-27 16:09:24,264][20381] Updated weights for policy 1, policy_version 1515442 (0.0008) [2023-12-27 16:09:24,282][20347] Updated weights for policy 0, policy_version 1516990 (0.0007) [2023-12-27 16:09:24,324][20381] Updated weights for policy 1, policy_version 1515452 (0.0009) [2023-12-27 16:09:24,345][20347] Updated weights for policy 0, policy_version 1517000 (0.0010) [2023-12-27 16:09:24,376][20381] Updated weights for policy 1, policy_version 1515462 (0.0005) [2023-12-27 16:09:24,407][20347] Updated weights for policy 0, policy_version 1517010 (0.0010) [2023-12-27 16:09:24,423][20381] Updated weights for policy 1, policy_version 1515472 (0.0006) [2023-12-27 16:09:24,966][20347] Updated weights for policy 0, policy_version 1517020 (0.0010) [2023-12-27 16:09:25,025][20347] Updated weights for policy 0, policy_version 1517030 (0.0010) [2023-12-27 16:09:25,057][20381] Updated weights for policy 1, policy_version 1515482 (0.0011) [2023-12-27 16:09:25,077][20347] Updated weights for policy 0, policy_version 1517040 (0.0010) [2023-12-27 16:09:25,114][20381] Updated weights for policy 1, policy_version 1515492 (0.0010) [2023-12-27 16:09:25,171][20381] Updated weights for policy 1, policy_version 1515502 (0.0010) [2023-12-27 16:09:25,720][20347] Updated weights for policy 0, policy_version 1517050 (0.0009) [2023-12-27 16:09:25,769][20347] Updated weights for policy 0, policy_version 1517060 (0.0005) [2023-12-27 16:09:25,819][20347] Updated weights for policy 0, policy_version 1517070 (0.0006) [2023-12-27 16:09:25,844][20381] Updated weights for policy 1, policy_version 1515512 (0.0010) [2023-12-27 16:09:25,866][20347] Updated weights for policy 0, policy_version 1517080 (0.0005) [2023-12-27 16:09:25,901][20381] Updated weights for policy 1, policy_version 1515522 (0.0005) [2023-12-27 16:09:25,966][20381] Updated weights for policy 1, policy_version 1515532 (0.0005) [2023-12-27 16:09:26,466][20347] Updated weights for policy 0, policy_version 1517090 (0.0010) [2023-12-27 16:09:26,511][20347] Updated weights for policy 0, policy_version 1517100 (0.0010) [2023-12-27 16:09:26,556][20347] Updated weights for policy 0, policy_version 1517110 (0.0010) [2023-12-27 16:09:26,591][20381] Updated weights for policy 1, policy_version 1515542 (0.0006) [2023-12-27 16:09:26,669][20381] Updated weights for policy 1, policy_version 1515552 (0.0007) [2023-12-27 16:09:26,719][20381] Updated weights for policy 1, policy_version 1515562 (0.0007) [2023-12-27 16:09:27,247][20347] Updated weights for policy 0, policy_version 1517120 (0.0010) [2023-12-27 16:09:27,304][20347] Updated weights for policy 0, policy_version 1517130 (0.0010) [2023-12-27 16:09:27,359][20347] Updated weights for policy 0, policy_version 1517140 (0.0010) [2023-12-27 16:09:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776478720. Throughput: 0: 10055.3, 1: 10445.9. Samples: 776474804. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:27,363][19170] Avg episode reward: [(0, '961.230'), (1, '990.270')] [2023-12-27 16:09:27,389][20381] Updated weights for policy 1, policy_version 1515572 (0.0007) [2023-12-27 16:09:27,439][20381] Updated weights for policy 1, policy_version 1515582 (0.0007) [2023-12-27 16:09:27,493][20381] Updated weights for policy 1, policy_version 1515592 (0.0007) [2023-12-27 16:09:28,039][20347] Updated weights for policy 0, policy_version 1517150 (0.0009) [2023-12-27 16:09:28,092][20347] Updated weights for policy 0, policy_version 1517160 (0.0006) [2023-12-27 16:09:28,149][20347] Updated weights for policy 0, policy_version 1517170 (0.0007) [2023-12-27 16:09:28,192][20381] Updated weights for policy 1, policy_version 1515602 (0.0008) [2023-12-27 16:09:28,236][20381] Updated weights for policy 1, policy_version 1515612 (0.0010) [2023-12-27 16:09:28,295][20381] Updated weights for policy 1, policy_version 1515622 (0.0009) [2023-12-27 16:09:28,346][20381] Updated weights for policy 1, policy_version 1515632 (0.0007) [2023-12-27 16:09:28,791][20347] Updated weights for policy 0, policy_version 1517180 (0.0006) [2023-12-27 16:09:28,849][20347] Updated weights for policy 0, policy_version 1517190 (0.0005) [2023-12-27 16:09:28,904][20347] Updated weights for policy 0, policy_version 1517200 (0.0005) [2023-12-27 16:09:29,044][20381] Updated weights for policy 1, policy_version 1515642 (0.0010) [2023-12-27 16:09:29,089][20381] Updated weights for policy 1, policy_version 1515652 (0.0010) [2023-12-27 16:09:29,138][20381] Updated weights for policy 1, policy_version 1515662 (0.0010) [2023-12-27 16:09:29,457][20347] Updated weights for policy 0, policy_version 1517210 (0.0006) [2023-12-27 16:09:29,518][20347] Updated weights for policy 0, policy_version 1517220 (0.0010) [2023-12-27 16:09:29,569][20347] Updated weights for policy 0, policy_version 1517230 (0.0010) [2023-12-27 16:09:29,623][20347] Updated weights for policy 0, policy_version 1517240 (0.0010) [2023-12-27 16:09:29,882][20381] Updated weights for policy 1, policy_version 1515672 (0.0008) [2023-12-27 16:09:29,940][20381] Updated weights for policy 1, policy_version 1515682 (0.0008) [2023-12-27 16:09:29,995][20381] Updated weights for policy 1, policy_version 1515692 (0.0010) [2023-12-27 16:09:30,285][20347] Updated weights for policy 0, policy_version 1517250 (0.0008) [2023-12-27 16:09:30,349][20347] Updated weights for policy 0, policy_version 1517260 (0.0006) [2023-12-27 16:09:30,404][20347] Updated weights for policy 0, policy_version 1517270 (0.0007) [2023-12-27 16:09:30,705][20381] Updated weights for policy 1, policy_version 1515702 (0.0009) [2023-12-27 16:09:30,762][20381] Updated weights for policy 1, policy_version 1515712 (0.0010) [2023-12-27 16:09:30,821][20381] Updated weights for policy 1, policy_version 1515722 (0.0008) [2023-12-27 16:09:31,046][20347] Updated weights for policy 0, policy_version 1517280 (0.0008) [2023-12-27 16:09:31,110][20347] Updated weights for policy 0, policy_version 1517290 (0.0008) [2023-12-27 16:09:31,174][20347] Updated weights for policy 0, policy_version 1517300 (0.0008) [2023-12-27 16:09:31,536][20381] Updated weights for policy 1, policy_version 1515732 (0.0009) [2023-12-27 16:09:31,600][20381] Updated weights for policy 1, policy_version 1515742 (0.0007) [2023-12-27 16:09:31,669][20381] Updated weights for policy 1, policy_version 1515752 (0.0008) [2023-12-27 16:09:31,886][20347] Updated weights for policy 0, policy_version 1517310 (0.0007) [2023-12-27 16:09:31,946][20347] Updated weights for policy 0, policy_version 1517320 (0.0008) [2023-12-27 16:09:32,001][20347] Updated weights for policy 0, policy_version 1517330 (0.0008) [2023-12-27 16:09:32,335][20381] Updated weights for policy 1, policy_version 1515762 (0.0008) [2023-12-27 16:09:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776585216. Throughput: 0: 10094.2, 1: 10416.1. Samples: 776597672. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:32,363][19170] Avg episode reward: [(0, '961.130'), (1, '951.420')] [2023-12-27 16:09:32,401][20381] Updated weights for policy 1, policy_version 1515772 (0.0008) [2023-12-27 16:09:32,455][20381] Updated weights for policy 1, policy_version 1515782 (0.0007) [2023-12-27 16:09:32,510][20381] Updated weights for policy 1, policy_version 1515792 (0.0008) [2023-12-27 16:09:32,695][20347] Updated weights for policy 0, policy_version 1517340 (0.0007) [2023-12-27 16:09:32,749][20347] Updated weights for policy 0, policy_version 1517350 (0.0007) [2023-12-27 16:09:32,803][20347] Updated weights for policy 0, policy_version 1517360 (0.0007) [2023-12-27 16:09:33,211][20381] Updated weights for policy 1, policy_version 1515802 (0.0011) [2023-12-27 16:09:33,265][20381] Updated weights for policy 1, policy_version 1515812 (0.0010) [2023-12-27 16:09:33,314][20381] Updated weights for policy 1, policy_version 1515822 (0.0010) [2023-12-27 16:09:33,478][20347] Updated weights for policy 0, policy_version 1517370 (0.0007) [2023-12-27 16:09:33,527][20347] Updated weights for policy 0, policy_version 1517380 (0.0005) [2023-12-27 16:09:33,581][20347] Updated weights for policy 0, policy_version 1517390 (0.0005) [2023-12-27 16:09:33,640][20347] Updated weights for policy 0, policy_version 1517400 (0.0007) [2023-12-27 16:09:34,004][20381] Updated weights for policy 1, policy_version 1515832 (0.0007) [2023-12-27 16:09:34,053][20381] Updated weights for policy 1, policy_version 1515842 (0.0008) [2023-12-27 16:09:34,117][20381] Updated weights for policy 1, policy_version 1515852 (0.0008) [2023-12-27 16:09:34,279][20347] Updated weights for policy 0, policy_version 1517410 (0.0008) [2023-12-27 16:09:34,336][20347] Updated weights for policy 0, policy_version 1517420 (0.0007) [2023-12-27 16:09:34,396][20347] Updated weights for policy 0, policy_version 1517430 (0.0008) [2023-12-27 16:09:34,789][20381] Updated weights for policy 1, policy_version 1515862 (0.0007) [2023-12-27 16:09:34,848][20381] Updated weights for policy 1, policy_version 1515872 (0.0005) [2023-12-27 16:09:34,902][20381] Updated weights for policy 1, policy_version 1515882 (0.0007) [2023-12-27 16:09:35,137][20347] Updated weights for policy 0, policy_version 1517440 (0.0008) [2023-12-27 16:09:35,189][20347] Updated weights for policy 0, policy_version 1517450 (0.0008) [2023-12-27 16:09:35,234][20347] Updated weights for policy 0, policy_version 1517460 (0.0008) [2023-12-27 16:09:35,483][20381] Updated weights for policy 1, policy_version 1515892 (0.0006) [2023-12-27 16:09:35,535][20381] Updated weights for policy 1, policy_version 1515902 (0.0006) [2023-12-27 16:09:35,590][20381] Updated weights for policy 1, policy_version 1515912 (0.0006) [2023-12-27 16:09:35,992][20347] Updated weights for policy 0, policy_version 1517470 (0.0009) [2023-12-27 16:09:36,040][20347] Updated weights for policy 0, policy_version 1517480 (0.0010) [2023-12-27 16:09:36,109][20347] Updated weights for policy 0, policy_version 1517492 (0.0010) [2023-12-27 16:09:36,185][20381] Updated weights for policy 1, policy_version 1515922 (0.0008) [2023-12-27 16:09:36,235][20381] Updated weights for policy 1, policy_version 1515932 (0.0007) [2023-12-27 16:09:36,286][20381] Updated weights for policy 1, policy_version 1515942 (0.0008) [2023-12-27 16:09:36,336][20381] Updated weights for policy 1, policy_version 1515952 (0.0007) [2023-12-27 16:09:36,838][20347] Updated weights for policy 0, policy_version 1517502 (0.0009) [2023-12-27 16:09:36,885][20347] Updated weights for policy 0, policy_version 1517512 (0.0008) [2023-12-27 16:09:36,942][20347] Updated weights for policy 0, policy_version 1517522 (0.0006) [2023-12-27 16:09:37,093][20381] Updated weights for policy 1, policy_version 1515962 (0.0007) [2023-12-27 16:09:37,149][20381] Updated weights for policy 1, policy_version 1515972 (0.0008) [2023-12-27 16:09:37,206][20381] Updated weights for policy 1, policy_version 1515982 (0.0007) [2023-12-27 16:09:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 776691712. Throughput: 0: 10087.4, 1: 10395.9. Samples: 776659260. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:37,363][19170] Avg episode reward: [(0, '980.570'), (1, '902.570')] [2023-12-27 16:09:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001515984_388128768.pth... [2023-12-27 16:09:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001517528_388562944.pth... [2023-12-27 16:09:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001514768_387817472.pth [2023-12-27 16:09:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001516344_388259840.pth [2023-12-27 16:09:37,593][20347] Updated weights for policy 0, policy_version 1517532 (0.0007) [2023-12-27 16:09:37,652][20347] Updated weights for policy 0, policy_version 1517542 (0.0006) [2023-12-27 16:09:37,696][20347] Updated weights for policy 0, policy_version 1517552 (0.0005) [2023-12-27 16:09:37,858][20381] Updated weights for policy 1, policy_version 1515992 (0.0008) [2023-12-27 16:09:37,913][20381] Updated weights for policy 1, policy_version 1516002 (0.0008) [2023-12-27 16:09:37,970][20381] Updated weights for policy 1, policy_version 1516012 (0.0007) [2023-12-27 16:09:38,289][20347] Updated weights for policy 0, policy_version 1517562 (0.0006) [2023-12-27 16:09:38,351][20347] Updated weights for policy 0, policy_version 1517572 (0.0008) [2023-12-27 16:09:38,401][20347] Updated weights for policy 0, policy_version 1517582 (0.0008) [2023-12-27 16:09:38,458][20347] Updated weights for policy 0, policy_version 1517592 (0.0008) [2023-12-27 16:09:38,713][20381] Updated weights for policy 1, policy_version 1516022 (0.0007) [2023-12-27 16:09:38,764][20381] Updated weights for policy 1, policy_version 1516032 (0.0007) [2023-12-27 16:09:38,814][20381] Updated weights for policy 1, policy_version 1516042 (0.0008) [2023-12-27 16:09:39,171][20347] Updated weights for policy 0, policy_version 1517602 (0.0007) [2023-12-27 16:09:39,228][20347] Updated weights for policy 0, policy_version 1517612 (0.0008) [2023-12-27 16:09:39,287][20347] Updated weights for policy 0, policy_version 1517622 (0.0008) [2023-12-27 16:09:39,566][20381] Updated weights for policy 1, policy_version 1516052 (0.0008) [2023-12-27 16:09:39,632][20381] Updated weights for policy 1, policy_version 1516062 (0.0007) [2023-12-27 16:09:39,702][20381] Updated weights for policy 1, policy_version 1516072 (0.0008) [2023-12-27 16:09:40,030][20347] Updated weights for policy 0, policy_version 1517632 (0.0008) [2023-12-27 16:09:40,084][20347] Updated weights for policy 0, policy_version 1517642 (0.0008) [2023-12-27 16:09:40,128][20347] Updated weights for policy 0, policy_version 1517652 (0.0008) [2023-12-27 16:09:40,414][20381] Updated weights for policy 1, policy_version 1516082 (0.0008) [2023-12-27 16:09:40,464][20381] Updated weights for policy 1, policy_version 1516092 (0.0008) [2023-12-27 16:09:40,514][20381] Updated weights for policy 1, policy_version 1516102 (0.0008) [2023-12-27 16:09:40,570][20381] Updated weights for policy 1, policy_version 1516112 (0.0007) [2023-12-27 16:09:40,864][20347] Updated weights for policy 0, policy_version 1517662 (0.0008) [2023-12-27 16:09:40,913][20347] Updated weights for policy 0, policy_version 1517672 (0.0008) [2023-12-27 16:09:40,963][20347] Updated weights for policy 0, policy_version 1517682 (0.0008) [2023-12-27 16:09:41,271][20381] Updated weights for policy 1, policy_version 1516122 (0.0008) [2023-12-27 16:09:41,327][20381] Updated weights for policy 1, policy_version 1516132 (0.0008) [2023-12-27 16:09:41,399][20381] Updated weights for policy 1, policy_version 1516142 (0.0008) [2023-12-27 16:09:41,864][20347] Updated weights for policy 0, policy_version 1517692 (0.0009) [2023-12-27 16:09:41,916][20347] Updated weights for policy 0, policy_version 1517702 (0.0008) [2023-12-27 16:09:41,970][20347] Updated weights for policy 0, policy_version 1517712 (0.0008) [2023-12-27 16:09:42,108][20381] Updated weights for policy 1, policy_version 1516152 (0.0007) [2023-12-27 16:09:42,167][20381] Updated weights for policy 1, policy_version 1516162 (0.0007) [2023-12-27 16:09:42,220][20381] Updated weights for policy 1, policy_version 1516172 (0.0008) [2023-12-27 16:09:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776790016. Throughput: 0: 10070.0, 1: 10333.3. Samples: 776779248. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:42,363][19170] Avg episode reward: [(0, '961.100'), (1, '941.530')] [2023-12-27 16:09:42,651][20347] Updated weights for policy 0, policy_version 1517722 (0.0008) [2023-12-27 16:09:42,709][20347] Updated weights for policy 0, policy_version 1517732 (0.0008) [2023-12-27 16:09:42,760][20347] Updated weights for policy 0, policy_version 1517742 (0.0008) [2023-12-27 16:09:42,797][20291] KL-divergence is very high: 105.3297 [2023-12-27 16:09:42,819][20347] Updated weights for policy 0, policy_version 1517752 (0.0007) [2023-12-27 16:09:42,938][20381] Updated weights for policy 1, policy_version 1516182 (0.0007) [2023-12-27 16:09:42,983][20381] Updated weights for policy 1, policy_version 1516192 (0.0009) [2023-12-27 16:09:43,032][20381] Updated weights for policy 1, policy_version 1516202 (0.0009) [2023-12-27 16:09:43,425][20347] Updated weights for policy 0, policy_version 1517762 (0.0010) [2023-12-27 16:09:43,490][20347] Updated weights for policy 0, policy_version 1517772 (0.0006) [2023-12-27 16:09:43,550][20347] Updated weights for policy 0, policy_version 1517782 (0.0006) [2023-12-27 16:09:43,796][20381] Updated weights for policy 1, policy_version 1516212 (0.0009) [2023-12-27 16:09:43,852][20381] Updated weights for policy 1, policy_version 1516222 (0.0008) [2023-12-27 16:09:43,911][20381] Updated weights for policy 1, policy_version 1516232 (0.0007) [2023-12-27 16:09:44,222][20347] Updated weights for policy 0, policy_version 1517792 (0.0007) [2023-12-27 16:09:44,290][20347] Updated weights for policy 0, policy_version 1517802 (0.0005) [2023-12-27 16:09:44,357][20347] Updated weights for policy 0, policy_version 1517812 (0.0006) [2023-12-27 16:09:44,530][20381] Updated weights for policy 1, policy_version 1516242 (0.0007) [2023-12-27 16:09:44,587][20381] Updated weights for policy 1, policy_version 1516252 (0.0005) [2023-12-27 16:09:44,636][20381] Updated weights for policy 1, policy_version 1516262 (0.0005) [2023-12-27 16:09:44,694][20381] Updated weights for policy 1, policy_version 1516272 (0.0005) [2023-12-27 16:09:44,965][20347] Updated weights for policy 0, policy_version 1517822 (0.0009) [2023-12-27 16:09:45,034][20347] Updated weights for policy 0, policy_version 1517832 (0.0011) [2023-12-27 16:09:45,096][20347] Updated weights for policy 0, policy_version 1517842 (0.0011) [2023-12-27 16:09:45,343][20381] Updated weights for policy 1, policy_version 1516282 (0.0008) [2023-12-27 16:09:45,391][20381] Updated weights for policy 1, policy_version 1516292 (0.0008) [2023-12-27 16:09:45,434][20381] Updated weights for policy 1, policy_version 1516302 (0.0007) [2023-12-27 16:09:45,781][20347] Updated weights for policy 0, policy_version 1517852 (0.0007) [2023-12-27 16:09:45,840][20347] Updated weights for policy 0, policy_version 1517862 (0.0008) [2023-12-27 16:09:45,899][20347] Updated weights for policy 0, policy_version 1517872 (0.0005) [2023-12-27 16:09:46,170][20381] Updated weights for policy 1, policy_version 1516312 (0.0007) [2023-12-27 16:09:46,215][20292] KL-divergence is very high: 107.2195 [2023-12-27 16:09:46,221][20381] Updated weights for policy 1, policy_version 1516322 (0.0008) [2023-12-27 16:09:46,265][20381] Updated weights for policy 1, policy_version 1516332 (0.0007) [2023-12-27 16:09:46,537][20347] Updated weights for policy 0, policy_version 1517882 (0.0006) [2023-12-27 16:09:46,591][20347] Updated weights for policy 0, policy_version 1517892 (0.0006) [2023-12-27 16:09:46,647][20347] Updated weights for policy 0, policy_version 1517902 (0.0009) [2023-12-27 16:09:46,701][20347] Updated weights for policy 0, policy_version 1517912 (0.0008) [2023-12-27 16:09:46,953][20381] Updated weights for policy 1, policy_version 1516342 (0.0009) [2023-12-27 16:09:46,999][20381] Updated weights for policy 1, policy_version 1516352 (0.0010) [2023-12-27 16:09:47,044][20381] Updated weights for policy 1, policy_version 1516362 (0.0010) [2023-12-27 16:09:47,316][20347] Updated weights for policy 0, policy_version 1517922 (0.0005) [2023-12-27 16:09:47,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776888320. Throughput: 0: 10144.4, 1: 10238.2. Samples: 776900416. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:47,363][19170] Avg episode reward: [(0, '961.070'), (1, '922.180')] [2023-12-27 16:09:47,375][20347] Updated weights for policy 0, policy_version 1517932 (0.0005) [2023-12-27 16:09:47,425][20347] Updated weights for policy 0, policy_version 1517942 (0.0006) [2023-12-27 16:09:47,729][20381] Updated weights for policy 1, policy_version 1516372 (0.0011) [2023-12-27 16:09:47,774][20381] Updated weights for policy 1, policy_version 1516382 (0.0007) [2023-12-27 16:09:47,819][20381] Updated weights for policy 1, policy_version 1516392 (0.0007) [2023-12-27 16:09:48,033][20347] Updated weights for policy 0, policy_version 1517952 (0.0009) [2023-12-27 16:09:48,080][20347] Updated weights for policy 0, policy_version 1517962 (0.0010) [2023-12-27 16:09:48,136][20347] Updated weights for policy 0, policy_version 1517972 (0.0010) [2023-12-27 16:09:48,535][20381] Updated weights for policy 1, policy_version 1516402 (0.0008) [2023-12-27 16:09:48,583][20381] Updated weights for policy 1, policy_version 1516412 (0.0007) [2023-12-27 16:09:48,634][20381] Updated weights for policy 1, policy_version 1516422 (0.0006) [2023-12-27 16:09:48,690][20381] Updated weights for policy 1, policy_version 1516432 (0.0008) [2023-12-27 16:09:48,857][20347] Updated weights for policy 0, policy_version 1517982 (0.0008) [2023-12-27 16:09:48,916][20347] Updated weights for policy 0, policy_version 1517992 (0.0005) [2023-12-27 16:09:48,985][20347] Updated weights for policy 0, policy_version 1518002 (0.0006) [2023-12-27 16:09:49,340][20381] Updated weights for policy 1, policy_version 1516442 (0.0008) [2023-12-27 16:09:49,397][20381] Updated weights for policy 1, policy_version 1516452 (0.0008) [2023-12-27 16:09:49,451][20381] Updated weights for policy 1, policy_version 1516462 (0.0008) [2023-12-27 16:09:49,581][20347] Updated weights for policy 0, policy_version 1518012 (0.0007) [2023-12-27 16:09:49,639][20347] Updated weights for policy 0, policy_version 1518022 (0.0008) [2023-12-27 16:09:49,696][20347] Updated weights for policy 0, policy_version 1518032 (0.0008) [2023-12-27 16:09:50,237][20381] Updated weights for policy 1, policy_version 1516472 (0.0008) [2023-12-27 16:09:50,294][20381] Updated weights for policy 1, policy_version 1516482 (0.0007) [2023-12-27 16:09:50,353][20381] Updated weights for policy 1, policy_version 1516492 (0.0008) [2023-12-27 16:09:50,405][20347] Updated weights for policy 0, policy_version 1518042 (0.0006) [2023-12-27 16:09:50,462][20347] Updated weights for policy 0, policy_version 1518052 (0.0010) [2023-12-27 16:09:50,519][20347] Updated weights for policy 0, policy_version 1518062 (0.0010) [2023-12-27 16:09:50,577][20347] Updated weights for policy 0, policy_version 1518072 (0.0010) [2023-12-27 16:09:51,065][20381] Updated weights for policy 1, policy_version 1516502 (0.0008) [2023-12-27 16:09:51,127][20381] Updated weights for policy 1, policy_version 1516512 (0.0008) [2023-12-27 16:09:51,186][20381] Updated weights for policy 1, policy_version 1516522 (0.0007) [2023-12-27 16:09:51,304][20347] Updated weights for policy 0, policy_version 1518082 (0.0009) [2023-12-27 16:09:51,357][20347] Updated weights for policy 0, policy_version 1518092 (0.0011) [2023-12-27 16:09:51,425][20347] Updated weights for policy 0, policy_version 1518102 (0.0009) [2023-12-27 16:09:51,953][20381] Updated weights for policy 1, policy_version 1516532 (0.0008) [2023-12-27 16:09:52,002][20381] Updated weights for policy 1, policy_version 1516542 (0.0008) [2023-12-27 16:09:52,046][20381] Updated weights for policy 1, policy_version 1516552 (0.0006) [2023-12-27 16:09:52,123][20347] Updated weights for policy 0, policy_version 1518112 (0.0008) [2023-12-27 16:09:52,172][20347] Updated weights for policy 0, policy_version 1518122 (0.0007) [2023-12-27 16:09:52,227][20347] Updated weights for policy 0, policy_version 1518132 (0.0008) [2023-12-27 16:09:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 776994816. Throughput: 0: 10209.4, 1: 10178.7. Samples: 776962696. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:52,363][19170] Avg episode reward: [(0, '970.780'), (1, '883.090')] [2023-12-27 16:09:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001516560_388276224.pth... [2023-12-27 16:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001518136_388718592.pth... [2023-12-27 16:09:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001515376_387973120.pth [2023-12-27 16:09:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001516920_388407296.pth [2023-12-27 16:09:52,789][20381] Updated weights for policy 1, policy_version 1516562 (0.0007) [2023-12-27 16:09:52,833][20292] KL-divergence is very high: 116.9864 [2023-12-27 16:09:52,838][20292] KL-divergence is very high: 121.3973 [2023-12-27 16:09:52,843][20381] Updated weights for policy 1, policy_version 1516572 (0.0008) [2023-12-27 16:09:52,860][20292] KL-divergence is very high: 221.4505 [2023-12-27 16:09:52,875][20292] KL-divergence is very high: 214.0804 [2023-12-27 16:09:52,881][20292] KL-divergence is very high: 164.6004 [2023-12-27 16:09:52,897][20381] Updated weights for policy 1, policy_version 1516582 (0.0008) [2023-12-27 16:09:52,901][20292] KL-divergence is very high: 111.3096 [2023-12-27 16:09:52,921][20347] Updated weights for policy 0, policy_version 1518142 (0.0007) [2023-12-27 16:09:52,948][20381] Updated weights for policy 1, policy_version 1516592 (0.0005) [2023-12-27 16:09:52,982][20347] Updated weights for policy 0, policy_version 1518152 (0.0010) [2023-12-27 16:09:53,043][20347] Updated weights for policy 0, policy_version 1518162 (0.0010) [2023-12-27 16:09:53,494][20292] KL-divergence is very high: 109.1525 [2023-12-27 16:09:53,534][20292] KL-divergence is very high: 166.0196 [2023-12-27 16:09:53,543][20381] Updated weights for policy 1, policy_version 1516602 (0.0007) [2023-12-27 16:09:53,590][20381] Updated weights for policy 1, policy_version 1516612 (0.0010) [2023-12-27 16:09:53,633][20347] Updated weights for policy 0, policy_version 1518172 (0.0008) [2023-12-27 16:09:53,640][20381] Updated weights for policy 1, policy_version 1516622 (0.0007) [2023-12-27 16:09:53,687][20347] Updated weights for policy 0, policy_version 1518182 (0.0006) [2023-12-27 16:09:53,745][20347] Updated weights for policy 0, policy_version 1518192 (0.0006) [2023-12-27 16:09:54,315][20381] Updated weights for policy 1, policy_version 1516632 (0.0006) [2023-12-27 16:09:54,366][20347] Updated weights for policy 0, policy_version 1518202 (0.0007) [2023-12-27 16:09:54,373][20381] Updated weights for policy 1, policy_version 1516642 (0.0008) [2023-12-27 16:09:54,422][20347] Updated weights for policy 0, policy_version 1518212 (0.0006) [2023-12-27 16:09:54,431][20381] Updated weights for policy 1, policy_version 1516652 (0.0007) [2023-12-27 16:09:54,478][20347] Updated weights for policy 0, policy_version 1518222 (0.0005) [2023-12-27 16:09:54,536][20347] Updated weights for policy 0, policy_version 1518232 (0.0007) [2023-12-27 16:09:54,993][20381] Updated weights for policy 1, policy_version 1516662 (0.0009) [2023-12-27 16:09:55,046][20381] Updated weights for policy 1, policy_version 1516672 (0.0011) [2023-12-27 16:09:55,101][20381] Updated weights for policy 1, policy_version 1516682 (0.0010) [2023-12-27 16:09:55,178][20347] Updated weights for policy 0, policy_version 1518242 (0.0006) [2023-12-27 16:09:55,234][20347] Updated weights for policy 0, policy_version 1518252 (0.0007) [2023-12-27 16:09:55,288][20347] Updated weights for policy 0, policy_version 1518262 (0.0008) [2023-12-27 16:09:55,772][20381] Updated weights for policy 1, policy_version 1516692 (0.0009) [2023-12-27 16:09:55,830][20381] Updated weights for policy 1, policy_version 1516702 (0.0008) [2023-12-27 16:09:55,881][20381] Updated weights for policy 1, policy_version 1516712 (0.0008) [2023-12-27 16:09:55,998][20347] Updated weights for policy 0, policy_version 1518272 (0.0007) [2023-12-27 16:09:56,052][20347] Updated weights for policy 0, policy_version 1518282 (0.0007) [2023-12-27 16:09:56,112][20347] Updated weights for policy 0, policy_version 1518292 (0.0008) [2023-12-27 16:09:56,500][20381] Updated weights for policy 1, policy_version 1516722 (0.0008) [2023-12-27 16:09:56,554][20381] Updated weights for policy 1, policy_version 1516732 (0.0008) [2023-12-27 16:09:56,602][20381] Updated weights for policy 1, policy_version 1516742 (0.0007) [2023-12-27 16:09:56,656][20381] Updated weights for policy 1, policy_version 1516752 (0.0007) [2023-12-27 16:09:56,742][20347] Updated weights for policy 0, policy_version 1518302 (0.0007) [2023-12-27 16:09:56,793][20347] Updated weights for policy 0, policy_version 1518312 (0.0008) [2023-12-27 16:09:56,838][20347] Updated weights for policy 0, policy_version 1518322 (0.0008) [2023-12-27 16:09:57,276][20381] Updated weights for policy 1, policy_version 1516762 (0.0008) [2023-12-27 16:09:57,332][20381] Updated weights for policy 1, policy_version 1516772 (0.0008) [2023-12-27 16:09:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 777093120. Throughput: 0: 10279.5, 1: 10167.6. Samples: 777086852. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:09:57,363][19170] Avg episode reward: [(0, '980.490'), (1, '902.510')] [2023-12-27 16:09:57,381][20381] Updated weights for policy 1, policy_version 1516782 (0.0008) [2023-12-27 16:09:57,553][20347] Updated weights for policy 0, policy_version 1518332 (0.0007) [2023-12-27 16:09:57,605][20347] Updated weights for policy 0, policy_version 1518342 (0.0005) [2023-12-27 16:09:57,658][20347] Updated weights for policy 0, policy_version 1518352 (0.0006) [2023-12-27 16:09:58,085][20381] Updated weights for policy 1, policy_version 1516793 (0.0009) [2023-12-27 16:09:58,135][20381] Updated weights for policy 1, policy_version 1516803 (0.0006) [2023-12-27 16:09:58,180][20381] Updated weights for policy 1, policy_version 1516813 (0.0006) [2023-12-27 16:09:58,201][20347] Updated weights for policy 0, policy_version 1518362 (0.0006) [2023-12-27 16:09:58,263][20347] Updated weights for policy 0, policy_version 1518372 (0.0010) [2023-12-27 16:09:58,315][20347] Updated weights for policy 0, policy_version 1518382 (0.0010) [2023-12-27 16:09:59,011][20381] Updated weights for policy 1, policy_version 1516823 (0.0007) [2023-12-27 16:09:59,062][20381] Updated weights for policy 1, policy_version 1516833 (0.0007) [2023-12-27 16:09:59,114][20381] Updated weights for policy 1, policy_version 1516843 (0.0007) [2023-12-27 16:09:59,128][20347] Updated weights for policy 0, policy_version 1518393 (0.0008) [2023-12-27 16:09:59,182][20347] Updated weights for policy 0, policy_version 1518403 (0.0008) [2023-12-27 16:09:59,253][20347] Updated weights for policy 0, policy_version 1518413 (0.0008) [2023-12-27 16:09:59,316][20347] Updated weights for policy 0, policy_version 1518423 (0.0008) [2023-12-27 16:09:59,875][20381] Updated weights for policy 1, policy_version 1516853 (0.0008) [2023-12-27 16:09:59,936][20381] Updated weights for policy 1, policy_version 1516863 (0.0008) [2023-12-27 16:09:59,989][20381] Updated weights for policy 1, policy_version 1516873 (0.0011) [2023-12-27 16:10:00,069][20347] Updated weights for policy 0, policy_version 1518433 (0.0008) [2023-12-27 16:10:00,127][20347] Updated weights for policy 0, policy_version 1518443 (0.0008) [2023-12-27 16:10:00,176][20347] Updated weights for policy 0, policy_version 1518453 (0.0006) [2023-12-27 16:10:00,632][20381] Updated weights for policy 1, policy_version 1516883 (0.0010) [2023-12-27 16:10:00,684][20381] Updated weights for policy 1, policy_version 1516893 (0.0010) [2023-12-27 16:10:00,742][20381] Updated weights for policy 1, policy_version 1516903 (0.0011) [2023-12-27 16:10:00,838][20347] Updated weights for policy 0, policy_version 1518463 (0.0006) [2023-12-27 16:10:00,898][20347] Updated weights for policy 0, policy_version 1518473 (0.0007) [2023-12-27 16:10:00,952][20347] Updated weights for policy 0, policy_version 1518483 (0.0007) [2023-12-27 16:10:01,458][20381] Updated weights for policy 1, policy_version 1516913 (0.0010) [2023-12-27 16:10:01,518][20381] Updated weights for policy 1, policy_version 1516923 (0.0008) [2023-12-27 16:10:01,578][20381] Updated weights for policy 1, policy_version 1516933 (0.0008) [2023-12-27 16:10:01,648][20381] Updated weights for policy 1, policy_version 1516943 (0.0008) [2023-12-27 16:10:01,677][20347] Updated weights for policy 0, policy_version 1518493 (0.0007) [2023-12-27 16:10:01,746][20347] Updated weights for policy 0, policy_version 1518503 (0.0006) [2023-12-27 16:10:01,753][20291] KL-divergence is very high: 137.6496 [2023-12-27 16:10:01,797][20347] Updated weights for policy 0, policy_version 1518513 (0.0007) [2023-12-27 16:10:02,303][20381] Updated weights for policy 1, policy_version 1516953 (0.0007) [2023-12-27 16:10:02,359][20381] Updated weights for policy 1, policy_version 1516963 (0.0007) [2023-12-27 16:10:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 777191424. Throughput: 0: 10273.5, 1: 10110.3. Samples: 777207544. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:10:02,363][19170] Avg episode reward: [(0, '980.510'), (1, '902.420')] [2023-12-27 16:10:02,418][20381] Updated weights for policy 1, policy_version 1516973 (0.0007) [2023-12-27 16:10:02,456][20347] Updated weights for policy 0, policy_version 1518523 (0.0006) [2023-12-27 16:10:02,519][20347] Updated weights for policy 0, policy_version 1518533 (0.0008) [2023-12-27 16:10:02,574][20347] Updated weights for policy 0, policy_version 1518543 (0.0008) [2023-12-27 16:10:03,070][20381] Updated weights for policy 1, policy_version 1516983 (0.0006) [2023-12-27 16:10:03,124][20381] Updated weights for policy 1, policy_version 1516993 (0.0005) [2023-12-27 16:10:03,182][20381] Updated weights for policy 1, policy_version 1517003 (0.0005) [2023-12-27 16:10:03,239][20347] Updated weights for policy 0, policy_version 1518553 (0.0007) [2023-12-27 16:10:03,297][20347] Updated weights for policy 0, policy_version 1518563 (0.0007) [2023-12-27 16:10:03,346][20347] Updated weights for policy 0, policy_version 1518573 (0.0008) [2023-12-27 16:10:03,395][20347] Updated weights for policy 0, policy_version 1518583 (0.0008) [2023-12-27 16:10:03,786][20381] Updated weights for policy 1, policy_version 1517013 (0.0007) [2023-12-27 16:10:03,849][20381] Updated weights for policy 1, policy_version 1517023 (0.0008) [2023-12-27 16:10:03,908][20381] Updated weights for policy 1, policy_version 1517033 (0.0008) [2023-12-27 16:10:04,058][20347] Updated weights for policy 0, policy_version 1518593 (0.0009) [2023-12-27 16:10:04,112][20347] Updated weights for policy 0, policy_version 1518603 (0.0009) [2023-12-27 16:10:04,161][20347] Updated weights for policy 0, policy_version 1518613 (0.0007) [2023-12-27 16:10:04,497][20381] Updated weights for policy 1, policy_version 1517043 (0.0008) [2023-12-27 16:10:04,559][20381] Updated weights for policy 1, policy_version 1517053 (0.0009) [2023-12-27 16:10:04,628][20381] Updated weights for policy 1, policy_version 1517063 (0.0009) [2023-12-27 16:10:04,853][20347] Updated weights for policy 0, policy_version 1518623 (0.0007) [2023-12-27 16:10:04,902][20347] Updated weights for policy 0, policy_version 1518633 (0.0009) [2023-12-27 16:10:04,957][20347] Updated weights for policy 0, policy_version 1518643 (0.0008) [2023-12-27 16:10:05,268][20381] Updated weights for policy 1, policy_version 1517073 (0.0007) [2023-12-27 16:10:05,322][20381] Updated weights for policy 1, policy_version 1517083 (0.0005) [2023-12-27 16:10:05,371][20381] Updated weights for policy 1, policy_version 1517093 (0.0006) [2023-12-27 16:10:05,423][20381] Updated weights for policy 1, policy_version 1517103 (0.0006) [2023-12-27 16:10:05,595][20347] Updated weights for policy 0, policy_version 1518653 (0.0008) [2023-12-27 16:10:05,646][20347] Updated weights for policy 0, policy_version 1518663 (0.0007) [2023-12-27 16:10:05,695][20347] Updated weights for policy 0, policy_version 1518673 (0.0007) [2023-12-27 16:10:06,052][20381] Updated weights for policy 1, policy_version 1517113 (0.0006) [2023-12-27 16:10:06,132][20381] Updated weights for policy 1, policy_version 1517123 (0.0008) [2023-12-27 16:10:06,200][20381] Updated weights for policy 1, policy_version 1517133 (0.0008) [2023-12-27 16:10:06,389][20347] Updated weights for policy 0, policy_version 1518683 (0.0008) [2023-12-27 16:10:06,447][20347] Updated weights for policy 0, policy_version 1518693 (0.0007) [2023-12-27 16:10:06,505][20347] Updated weights for policy 0, policy_version 1518703 (0.0008) [2023-12-27 16:10:06,867][20381] Updated weights for policy 1, policy_version 1517143 (0.0007) [2023-12-27 16:10:06,923][20381] Updated weights for policy 1, policy_version 1517153 (0.0008) [2023-12-27 16:10:06,978][20381] Updated weights for policy 1, policy_version 1517163 (0.0008) [2023-12-27 16:10:07,196][20347] Updated weights for policy 0, policy_version 1518713 (0.0007) [2023-12-27 16:10:07,253][20347] Updated weights for policy 0, policy_version 1518723 (0.0008) [2023-12-27 16:10:07,309][20347] Updated weights for policy 0, policy_version 1518733 (0.0008) [2023-12-27 16:10:07,357][20347] Updated weights for policy 0, policy_version 1518743 (0.0007) [2023-12-27 16:10:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 777306112. Throughput: 0: 10275.8, 1: 10158.3. Samples: 777270144. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:10:07,363][19170] Avg episode reward: [(0, '980.510'), (1, '921.990')] [2023-12-27 16:10:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001518744_388874240.pth... [2023-12-27 16:10:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001517168_388431872.pth... [2023-12-27 16:10:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001517528_388562944.pth [2023-12-27 16:10:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001515984_388128768.pth [2023-12-27 16:10:07,689][20381] Updated weights for policy 1, policy_version 1517173 (0.0008) [2023-12-27 16:10:07,747][20381] Updated weights for policy 1, policy_version 1517183 (0.0007) [2023-12-27 16:10:07,804][20381] Updated weights for policy 1, policy_version 1517193 (0.0007) [2023-12-27 16:10:07,965][20347] Updated weights for policy 0, policy_version 1518753 (0.0007) [2023-12-27 16:10:08,017][20347] Updated weights for policy 0, policy_version 1518763 (0.0005) [2023-12-27 16:10:08,062][20347] Updated weights for policy 0, policy_version 1518773 (0.0005) [2023-12-27 16:10:08,522][20381] Updated weights for policy 1, policy_version 1517203 (0.0007) [2023-12-27 16:10:08,577][20381] Updated weights for policy 1, policy_version 1517213 (0.0007) [2023-12-27 16:10:08,630][20381] Updated weights for policy 1, policy_version 1517223 (0.0007) [2023-12-27 16:10:08,698][20347] Updated weights for policy 0, policy_version 1518783 (0.0007) [2023-12-27 16:10:08,757][20347] Updated weights for policy 0, policy_version 1518793 (0.0008) [2023-12-27 16:10:08,812][20347] Updated weights for policy 0, policy_version 1518803 (0.0007) [2023-12-27 16:10:09,343][20381] Updated weights for policy 1, policy_version 1517233 (0.0007) [2023-12-27 16:10:09,410][20381] Updated weights for policy 1, policy_version 1517243 (0.0011) [2023-12-27 16:10:09,475][20381] Updated weights for policy 1, policy_version 1517253 (0.0009) [2023-12-27 16:10:09,536][20381] Updated weights for policy 1, policy_version 1517263 (0.0008) [2023-12-27 16:10:09,572][20347] Updated weights for policy 0, policy_version 1518813 (0.0007) [2023-12-27 16:10:09,640][20347] Updated weights for policy 0, policy_version 1518823 (0.0006) [2023-12-27 16:10:09,698][20347] Updated weights for policy 0, policy_version 1518833 (0.0009) [2023-12-27 16:10:10,225][20381] Updated weights for policy 1, policy_version 1517273 (0.0006) [2023-12-27 16:10:10,285][20381] Updated weights for policy 1, policy_version 1517283 (0.0007) [2023-12-27 16:10:10,353][20381] Updated weights for policy 1, policy_version 1517293 (0.0008) [2023-12-27 16:10:10,391][20347] Updated weights for policy 0, policy_version 1518843 (0.0007) [2023-12-27 16:10:10,446][20347] Updated weights for policy 0, policy_version 1518853 (0.0007) [2023-12-27 16:10:10,500][20347] Updated weights for policy 0, policy_version 1518863 (0.0007) [2023-12-27 16:10:10,998][20381] Updated weights for policy 1, policy_version 1517303 (0.0006) [2023-12-27 16:10:11,053][20381] Updated weights for policy 1, policy_version 1517313 (0.0007) [2023-12-27 16:10:11,114][20381] Updated weights for policy 1, policy_version 1517323 (0.0008) [2023-12-27 16:10:11,181][20347] Updated weights for policy 0, policy_version 1518873 (0.0007) [2023-12-27 16:10:11,241][20347] Updated weights for policy 0, policy_version 1518883 (0.0008) [2023-12-27 16:10:11,306][20347] Updated weights for policy 0, policy_version 1518893 (0.0010) [2023-12-27 16:10:11,371][20347] Updated weights for policy 0, policy_version 1518903 (0.0010) [2023-12-27 16:10:11,888][20381] Updated weights for policy 1, policy_version 1517333 (0.0007) [2023-12-27 16:10:11,947][20381] Updated weights for policy 1, policy_version 1517343 (0.0005) [2023-12-27 16:10:12,008][20381] Updated weights for policy 1, policy_version 1517353 (0.0006) [2023-12-27 16:10:12,065][20347] Updated weights for policy 0, policy_version 1518913 (0.0006) [2023-12-27 16:10:12,119][20347] Updated weights for policy 0, policy_version 1518923 (0.0010) [2023-12-27 16:10:12,172][20347] Updated weights for policy 0, policy_version 1518933 (0.0010) [2023-12-27 16:10:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 777404416. Throughput: 0: 10269.4, 1: 10141.8. Samples: 777393308. Policy #0 lag: (min: 3.0, avg: 8.6, max: 35.0) [2023-12-27 16:10:12,363][19170] Avg episode reward: [(0, '990.230'), (1, '960.930')] [2023-12-27 16:10:12,621][20381] Updated weights for policy 1, policy_version 1517363 (0.0006) [2023-12-27 16:10:12,677][20381] Updated weights for policy 1, policy_version 1517373 (0.0008) [2023-12-27 16:10:12,728][20381] Updated weights for policy 1, policy_version 1517383 (0.0008) [2023-12-27 16:10:12,872][20347] Updated weights for policy 0, policy_version 1518943 (0.0011) [2023-12-27 16:10:12,929][20347] Updated weights for policy 0, policy_version 1518953 (0.0006) [2023-12-27 16:10:12,980][20347] Updated weights for policy 0, policy_version 1518963 (0.0008) [2023-12-27 16:10:13,419][20381] Updated weights for policy 1, policy_version 1517393 (0.0008) [2023-12-27 16:10:13,468][20381] Updated weights for policy 1, policy_version 1517403 (0.0011) [2023-12-27 16:10:13,526][20381] Updated weights for policy 1, policy_version 1517413 (0.0011) [2023-12-27 16:10:13,579][20381] Updated weights for policy 1, policy_version 1517423 (0.0007) [2023-12-27 16:10:13,587][20347] Updated weights for policy 0, policy_version 1518973 (0.0007) [2023-12-27 16:10:13,642][20347] Updated weights for policy 0, policy_version 1518983 (0.0006) [2023-12-27 16:10:13,706][20347] Updated weights for policy 0, policy_version 1518993 (0.0007) [2023-12-27 16:10:14,266][20381] Updated weights for policy 1, policy_version 1517433 (0.0010) [2023-12-27 16:10:14,317][20347] Updated weights for policy 0, policy_version 1519003 (0.0006) [2023-12-27 16:10:14,324][20381] Updated weights for policy 1, policy_version 1517443 (0.0009) [2023-12-27 16:10:14,373][20347] Updated weights for policy 0, policy_version 1519013 (0.0006) [2023-12-27 16:10:14,382][20381] Updated weights for policy 1, policy_version 1517453 (0.0008) [2023-12-27 16:10:14,424][20347] Updated weights for policy 0, policy_version 1519023 (0.0006) [2023-12-27 16:10:15,078][20347] Updated weights for policy 0, policy_version 1519033 (0.0007) [2023-12-27 16:10:15,097][20381] Updated weights for policy 1, policy_version 1517463 (0.0006) [2023-12-27 16:10:15,137][20347] Updated weights for policy 0, policy_version 1519043 (0.0007) [2023-12-27 16:10:15,155][20381] Updated weights for policy 1, policy_version 1517473 (0.0006) [2023-12-27 16:10:15,200][20347] Updated weights for policy 0, policy_version 1519053 (0.0006) [2023-12-27 16:10:15,214][20381] Updated weights for policy 1, policy_version 1517483 (0.0006) [2023-12-27 16:10:15,257][20347] Updated weights for policy 0, policy_version 1519063 (0.0007) [2023-12-27 16:10:15,860][20381] Updated weights for policy 1, policy_version 1517493 (0.0006) [2023-12-27 16:10:15,913][20381] Updated weights for policy 1, policy_version 1517503 (0.0007) [2023-12-27 16:10:15,956][20347] Updated weights for policy 0, policy_version 1519073 (0.0005) [2023-12-27 16:10:15,964][20381] Updated weights for policy 1, policy_version 1517513 (0.0006) [2023-12-27 16:10:16,010][20347] Updated weights for policy 0, policy_version 1519083 (0.0005) [2023-12-27 16:10:16,063][20347] Updated weights for policy 0, policy_version 1519093 (0.0005) [2023-12-27 16:10:16,640][20347] Updated weights for policy 0, policy_version 1519103 (0.0006) [2023-12-27 16:10:16,689][20347] Updated weights for policy 0, policy_version 1519113 (0.0007) [2023-12-27 16:10:16,693][20381] Updated weights for policy 1, policy_version 1517523 (0.0008) [2023-12-27 16:10:16,738][20347] Updated weights for policy 0, policy_version 1519123 (0.0007) [2023-12-27 16:10:16,755][20381] Updated weights for policy 1, policy_version 1517533 (0.0011) [2023-12-27 16:10:16,819][20381] Updated weights for policy 1, policy_version 1517543 (0.0010) [2023-12-27 16:10:17,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 777502720. Throughput: 0: 10273.7, 1: 10154.3. Samples: 777516932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '941.510')] [2023-12-27 16:10:17,393][20347] Updated weights for policy 0, policy_version 1519133 (0.0008) [2023-12-27 16:10:17,444][20381] Updated weights for policy 1, policy_version 1517553 (0.0011) [2023-12-27 16:10:17,454][20347] Updated weights for policy 0, policy_version 1519143 (0.0010) [2023-12-27 16:10:17,502][20381] Updated weights for policy 1, policy_version 1517563 (0.0007) [2023-12-27 16:10:17,517][20347] Updated weights for policy 0, policy_version 1519153 (0.0010) [2023-12-27 16:10:17,553][20381] Updated weights for policy 1, policy_version 1517573 (0.0011) [2023-12-27 16:10:17,607][20381] Updated weights for policy 1, policy_version 1517583 (0.0010) [2023-12-27 16:10:18,124][20347] Updated weights for policy 0, policy_version 1519163 (0.0010) [2023-12-27 16:10:18,182][20347] Updated weights for policy 0, policy_version 1519173 (0.0011) [2023-12-27 16:10:18,240][20347] Updated weights for policy 0, policy_version 1519183 (0.0010) [2023-12-27 16:10:18,252][20381] Updated weights for policy 1, policy_version 1517593 (0.0005) [2023-12-27 16:10:18,310][20381] Updated weights for policy 1, policy_version 1517603 (0.0005) [2023-12-27 16:10:18,379][20381] Updated weights for policy 1, policy_version 1517613 (0.0007) [2023-12-27 16:10:18,921][20347] Updated weights for policy 0, policy_version 1519193 (0.0010) [2023-12-27 16:10:18,932][20381] Updated weights for policy 1, policy_version 1517623 (0.0008) [2023-12-27 16:10:18,976][20347] Updated weights for policy 0, policy_version 1519203 (0.0010) [2023-12-27 16:10:18,982][20381] Updated weights for policy 1, policy_version 1517633 (0.0006) [2023-12-27 16:10:19,028][20347] Updated weights for policy 0, policy_version 1519213 (0.0010) [2023-12-27 16:10:19,037][20381] Updated weights for policy 1, policy_version 1517643 (0.0005) [2023-12-27 16:10:19,082][20347] Updated weights for policy 0, policy_version 1519223 (0.0010) [2023-12-27 16:10:19,749][20381] Updated weights for policy 1, policy_version 1517653 (0.0007) [2023-12-27 16:10:19,807][20347] Updated weights for policy 0, policy_version 1519233 (0.0010) [2023-12-27 16:10:19,814][20381] Updated weights for policy 1, policy_version 1517663 (0.0008) [2023-12-27 16:10:19,867][20347] Updated weights for policy 0, policy_version 1519243 (0.0010) [2023-12-27 16:10:19,877][20381] Updated weights for policy 1, policy_version 1517673 (0.0008) [2023-12-27 16:10:19,930][20347] Updated weights for policy 0, policy_version 1519253 (0.0006) [2023-12-27 16:10:20,606][20381] Updated weights for policy 1, policy_version 1517683 (0.0008) [2023-12-27 16:10:20,658][20381] Updated weights for policy 1, policy_version 1517693 (0.0007) [2023-12-27 16:10:20,659][20347] Updated weights for policy 0, policy_version 1519263 (0.0010) [2023-12-27 16:10:20,714][20381] Updated weights for policy 1, policy_version 1517703 (0.0006) [2023-12-27 16:10:20,723][20347] Updated weights for policy 0, policy_version 1519273 (0.0011) [2023-12-27 16:10:20,782][20347] Updated weights for policy 0, policy_version 1519283 (0.0010) [2023-12-27 16:10:21,324][20381] Updated weights for policy 1, policy_version 1517713 (0.0007) [2023-12-27 16:10:21,394][20381] Updated weights for policy 1, policy_version 1517723 (0.0009) [2023-12-27 16:10:21,453][20381] Updated weights for policy 1, policy_version 1517733 (0.0008) [2023-12-27 16:10:21,514][20347] Updated weights for policy 0, policy_version 1519293 (0.0009) [2023-12-27 16:10:21,516][20381] Updated weights for policy 1, policy_version 1517743 (0.0007) [2023-12-27 16:10:21,571][20347] Updated weights for policy 0, policy_version 1519303 (0.0007) [2023-12-27 16:10:21,638][20347] Updated weights for policy 0, policy_version 1519313 (0.0008) [2023-12-27 16:10:22,241][20381] Updated weights for policy 1, policy_version 1517753 (0.0007) [2023-12-27 16:10:22,298][20381] Updated weights for policy 1, policy_version 1517763 (0.0007) [2023-12-27 16:10:22,351][20347] Updated weights for policy 0, policy_version 1519323 (0.0006) [2023-12-27 16:10:22,356][20381] Updated weights for policy 1, policy_version 1517773 (0.0008) [2023-12-27 16:10:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 777601024. Throughput: 0: 10275.5, 1: 10163.5. Samples: 777579012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:22,363][19170] Avg episode reward: [(0, '961.060'), (1, '902.660')] [2023-12-27 16:10:22,376][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001517776_388587520.pth... [2023-12-27 16:10:22,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001516560_388276224.pth [2023-12-27 16:10:22,418][20347] Updated weights for policy 0, policy_version 1519333 (0.0008) [2023-12-27 16:10:22,470][20347] Updated weights for policy 0, policy_version 1519343 (0.0007) [2023-12-27 16:10:22,517][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001519352_389029888.pth... [2023-12-27 16:10:22,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001518136_388718592.pth [2023-12-27 16:10:23,026][20381] Updated weights for policy 1, policy_version 1517783 (0.0008) [2023-12-27 16:10:23,074][20381] Updated weights for policy 1, policy_version 1517793 (0.0007) [2023-12-27 16:10:23,122][20381] Updated weights for policy 1, policy_version 1517803 (0.0007) [2023-12-27 16:10:23,176][20347] Updated weights for policy 0, policy_version 1519353 (0.0007) [2023-12-27 16:10:23,227][20347] Updated weights for policy 0, policy_version 1519363 (0.0009) [2023-12-27 16:10:23,272][20347] Updated weights for policy 0, policy_version 1519373 (0.0010) [2023-12-27 16:10:23,319][20347] Updated weights for policy 0, policy_version 1519383 (0.0010) [2023-12-27 16:10:23,848][20381] Updated weights for policy 1, policy_version 1517813 (0.0007) [2023-12-27 16:10:23,899][20381] Updated weights for policy 1, policy_version 1517823 (0.0006) [2023-12-27 16:10:23,940][20347] Updated weights for policy 0, policy_version 1519393 (0.0008) [2023-12-27 16:10:23,953][20381] Updated weights for policy 1, policy_version 1517833 (0.0007) [2023-12-27 16:10:23,998][20347] Updated weights for policy 0, policy_version 1519403 (0.0007) [2023-12-27 16:10:24,057][20347] Updated weights for policy 0, policy_version 1519413 (0.0008) [2023-12-27 16:10:24,614][20381] Updated weights for policy 1, policy_version 1517843 (0.0006) [2023-12-27 16:10:24,669][20381] Updated weights for policy 1, policy_version 1517853 (0.0008) [2023-12-27 16:10:24,727][20381] Updated weights for policy 1, policy_version 1517863 (0.0008) [2023-12-27 16:10:24,746][20347] Updated weights for policy 0, policy_version 1519423 (0.0006) [2023-12-27 16:10:24,799][20347] Updated weights for policy 0, policy_version 1519433 (0.0007) [2023-12-27 16:10:24,848][20347] Updated weights for policy 0, policy_version 1519443 (0.0006) [2023-12-27 16:10:25,444][20381] Updated weights for policy 1, policy_version 1517873 (0.0008) [2023-12-27 16:10:25,491][20347] Updated weights for policy 0, policy_version 1519453 (0.0008) [2023-12-27 16:10:25,505][20381] Updated weights for policy 1, policy_version 1517883 (0.0006) [2023-12-27 16:10:25,545][20347] Updated weights for policy 0, policy_version 1519463 (0.0005) [2023-12-27 16:10:25,561][20381] Updated weights for policy 1, policy_version 1517893 (0.0007) [2023-12-27 16:10:25,598][20347] Updated weights for policy 0, policy_version 1519473 (0.0006) [2023-12-27 16:10:25,618][20381] Updated weights for policy 1, policy_version 1517903 (0.0006) [2023-12-27 16:10:26,298][20381] Updated weights for policy 1, policy_version 1517913 (0.0006) [2023-12-27 16:10:26,306][20347] Updated weights for policy 0, policy_version 1519483 (0.0007) [2023-12-27 16:10:26,354][20381] Updated weights for policy 1, policy_version 1517923 (0.0007) [2023-12-27 16:10:26,361][20347] Updated weights for policy 0, policy_version 1519493 (0.0005) [2023-12-27 16:10:26,406][20381] Updated weights for policy 1, policy_version 1517933 (0.0007) [2023-12-27 16:10:26,417][20347] Updated weights for policy 0, policy_version 1519503 (0.0005) [2023-12-27 16:10:27,064][20347] Updated weights for policy 0, policy_version 1519513 (0.0007) [2023-12-27 16:10:27,100][20381] Updated weights for policy 1, policy_version 1517943 (0.0006) [2023-12-27 16:10:27,113][20347] Updated weights for policy 0, policy_version 1519523 (0.0006) [2023-12-27 16:10:27,152][20381] Updated weights for policy 1, policy_version 1517953 (0.0005) [2023-12-27 16:10:27,165][20347] Updated weights for policy 0, policy_version 1519533 (0.0006) [2023-12-27 16:10:27,204][20381] Updated weights for policy 1, policy_version 1517963 (0.0006) [2023-12-27 16:10:27,217][20347] Updated weights for policy 0, policy_version 1519543 (0.0006) [2023-12-27 16:10:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 777715712. Throughput: 0: 10299.5, 1: 10172.9. Samples: 777700508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:27,364][19170] Avg episode reward: [(0, '980.590'), (1, '892.810')] [2023-12-27 16:10:27,839][20347] Updated weights for policy 0, policy_version 1519553 (0.0006) [2023-12-27 16:10:27,889][20347] Updated weights for policy 0, policy_version 1519563 (0.0005) [2023-12-27 16:10:27,906][20381] Updated weights for policy 1, policy_version 1517973 (0.0006) [2023-12-27 16:10:27,949][20347] Updated weights for policy 0, policy_version 1519573 (0.0006) [2023-12-27 16:10:27,966][20381] Updated weights for policy 1, policy_version 1517983 (0.0007) [2023-12-27 16:10:28,025][20381] Updated weights for policy 1, policy_version 1517993 (0.0008) [2023-12-27 16:10:28,602][20347] Updated weights for policy 0, policy_version 1519583 (0.0007) [2023-12-27 16:10:28,662][20347] Updated weights for policy 0, policy_version 1519593 (0.0008) [2023-12-27 16:10:28,716][20347] Updated weights for policy 0, policy_version 1519603 (0.0007) [2023-12-27 16:10:28,720][20381] Updated weights for policy 1, policy_version 1518003 (0.0007) [2023-12-27 16:10:28,770][20381] Updated weights for policy 1, policy_version 1518013 (0.0006) [2023-12-27 16:10:28,829][20381] Updated weights for policy 1, policy_version 1518023 (0.0007) [2023-12-27 16:10:29,354][20347] Updated weights for policy 0, policy_version 1519613 (0.0007) [2023-12-27 16:10:29,411][20347] Updated weights for policy 0, policy_version 1519623 (0.0008) [2023-12-27 16:10:29,466][20347] Updated weights for policy 0, policy_version 1519633 (0.0007) [2023-12-27 16:10:29,540][20381] Updated weights for policy 1, policy_version 1518033 (0.0007) [2023-12-27 16:10:29,601][20381] Updated weights for policy 1, policy_version 1518043 (0.0008) [2023-12-27 16:10:29,652][20381] Updated weights for policy 1, policy_version 1518053 (0.0008) [2023-12-27 16:10:29,697][20381] Updated weights for policy 1, policy_version 1518063 (0.0007) [2023-12-27 16:10:30,167][20347] Updated weights for policy 0, policy_version 1519643 (0.0008) [2023-12-27 16:10:30,223][20347] Updated weights for policy 0, policy_version 1519653 (0.0007) [2023-12-27 16:10:30,279][20347] Updated weights for policy 0, policy_version 1519663 (0.0007) [2023-12-27 16:10:30,406][20381] Updated weights for policy 1, policy_version 1518073 (0.0007) [2023-12-27 16:10:30,458][20381] Updated weights for policy 1, policy_version 1518083 (0.0007) [2023-12-27 16:10:30,513][20381] Updated weights for policy 1, policy_version 1518093 (0.0008) [2023-12-27 16:10:30,909][20347] Updated weights for policy 0, policy_version 1519673 (0.0008) [2023-12-27 16:10:30,968][20347] Updated weights for policy 0, policy_version 1519683 (0.0007) [2023-12-27 16:10:31,017][20347] Updated weights for policy 0, policy_version 1519693 (0.0008) [2023-12-27 16:10:31,086][20347] Updated weights for policy 0, policy_version 1519703 (0.0008) [2023-12-27 16:10:31,220][20381] Updated weights for policy 1, policy_version 1518103 (0.0007) [2023-12-27 16:10:31,281][20381] Updated weights for policy 1, policy_version 1518113 (0.0007) [2023-12-27 16:10:31,340][20381] Updated weights for policy 1, policy_version 1518123 (0.0008) [2023-12-27 16:10:31,769][20347] Updated weights for policy 0, policy_version 1519713 (0.0007) [2023-12-27 16:10:31,831][20347] Updated weights for policy 0, policy_version 1519723 (0.0009) [2023-12-27 16:10:31,885][20347] Updated weights for policy 0, policy_version 1519733 (0.0008) [2023-12-27 16:10:31,964][20381] Updated weights for policy 1, policy_version 1518133 (0.0007) [2023-12-27 16:10:32,022][20381] Updated weights for policy 1, policy_version 1518143 (0.0007) [2023-12-27 16:10:32,079][20381] Updated weights for policy 1, policy_version 1518153 (0.0008) [2023-12-27 16:10:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 777814016. Throughput: 0: 10330.9, 1: 10170.3. Samples: 777822972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:32,363][19170] Avg episode reward: [(0, '980.660'), (1, '873.380')] [2023-12-27 16:10:32,606][20347] Updated weights for policy 0, policy_version 1519743 (0.0008) [2023-12-27 16:10:32,669][20347] Updated weights for policy 0, policy_version 1519753 (0.0007) [2023-12-27 16:10:32,725][20347] Updated weights for policy 0, policy_version 1519763 (0.0006) [2023-12-27 16:10:32,734][20381] Updated weights for policy 1, policy_version 1518163 (0.0007) [2023-12-27 16:10:32,793][20381] Updated weights for policy 1, policy_version 1518173 (0.0007) [2023-12-27 16:10:32,810][20292] KL-divergence is very high: 175.8433 [2023-12-27 16:10:32,822][20292] KL-divergence is very high: 214.2164 [2023-12-27 16:10:32,848][20381] Updated weights for policy 1, policy_version 1518183 (0.0008) [2023-12-27 16:10:32,854][20292] KL-divergence is very high: 181.6752 [2023-12-27 16:10:32,866][20292] KL-divergence is very high: 134.6553 [2023-12-27 16:10:33,395][20347] Updated weights for policy 0, policy_version 1519773 (0.0007) [2023-12-27 16:10:33,444][20347] Updated weights for policy 0, policy_version 1519783 (0.0009) [2023-12-27 16:10:33,469][20381] Updated weights for policy 1, policy_version 1518193 (0.0007) [2023-12-27 16:10:33,497][20347] Updated weights for policy 0, policy_version 1519793 (0.0007) [2023-12-27 16:10:33,522][20381] Updated weights for policy 1, policy_version 1518203 (0.0005) [2023-12-27 16:10:33,572][20381] Updated weights for policy 1, policy_version 1518213 (0.0005) [2023-12-27 16:10:33,633][20381] Updated weights for policy 1, policy_version 1518223 (0.0007) [2023-12-27 16:10:34,230][20347] Updated weights for policy 0, policy_version 1519803 (0.0007) [2023-12-27 16:10:34,270][20381] Updated weights for policy 1, policy_version 1518233 (0.0007) [2023-12-27 16:10:34,287][20347] Updated weights for policy 0, policy_version 1519813 (0.0005) [2023-12-27 16:10:34,329][20381] Updated weights for policy 1, policy_version 1518243 (0.0007) [2023-12-27 16:10:34,349][20347] Updated weights for policy 0, policy_version 1519823 (0.0007) [2023-12-27 16:10:34,388][20381] Updated weights for policy 1, policy_version 1518253 (0.0007) [2023-12-27 16:10:35,048][20347] Updated weights for policy 0, policy_version 1519833 (0.0006) [2023-12-27 16:10:35,052][20381] Updated weights for policy 1, policy_version 1518263 (0.0007) [2023-12-27 16:10:35,100][20347] Updated weights for policy 0, policy_version 1519843 (0.0010) [2023-12-27 16:10:35,105][20381] Updated weights for policy 1, policy_version 1518273 (0.0011) [2023-12-27 16:10:35,148][20347] Updated weights for policy 0, policy_version 1519853 (0.0010) [2023-12-27 16:10:35,157][20381] Updated weights for policy 1, policy_version 1518283 (0.0007) [2023-12-27 16:10:35,193][20347] Updated weights for policy 0, policy_version 1519863 (0.0010) [2023-12-27 16:10:35,771][20381] Updated weights for policy 1, policy_version 1518293 (0.0007) [2023-12-27 16:10:35,820][20381] Updated weights for policy 1, policy_version 1518303 (0.0008) [2023-12-27 16:10:35,840][20347] Updated weights for policy 0, policy_version 1519873 (0.0007) [2023-12-27 16:10:35,874][20381] Updated weights for policy 1, policy_version 1518313 (0.0005) [2023-12-27 16:10:35,892][20347] Updated weights for policy 0, policy_version 1519883 (0.0010) [2023-12-27 16:10:35,945][20347] Updated weights for policy 0, policy_version 1519893 (0.0010) [2023-12-27 16:10:36,517][20381] Updated weights for policy 1, policy_version 1518323 (0.0007) [2023-12-27 16:10:36,573][20381] Updated weights for policy 1, policy_version 1518333 (0.0010) [2023-12-27 16:10:36,633][20381] Updated weights for policy 1, policy_version 1518343 (0.0011) [2023-12-27 16:10:36,638][20347] Updated weights for policy 0, policy_version 1519903 (0.0009) [2023-12-27 16:10:36,695][20347] Updated weights for policy 0, policy_version 1519913 (0.0008) [2023-12-27 16:10:36,748][20347] Updated weights for policy 0, policy_version 1519923 (0.0008) [2023-12-27 16:10:37,301][20381] Updated weights for policy 1, policy_version 1518353 (0.0008) [2023-12-27 16:10:37,360][20381] Updated weights for policy 1, policy_version 1518363 (0.0008) [2023-12-27 16:10:37,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 777912320. Throughput: 0: 10277.5, 1: 10210.6. Samples: 777884656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:37,363][19170] Avg episode reward: [(0, '980.660'), (1, '892.850')] [2023-12-27 16:10:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001519928_389177344.pth... [2023-12-27 16:10:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001518744_388874240.pth [2023-12-27 16:10:37,412][20381] Updated weights for policy 1, policy_version 1518373 (0.0006) [2023-12-27 16:10:37,473][20347] Updated weights for policy 0, policy_version 1519933 (0.0007) [2023-12-27 16:10:37,473][20381] Updated weights for policy 1, policy_version 1518383 (0.0006) [2023-12-27 16:10:37,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001518384_388743168.pth... [2023-12-27 16:10:37,482][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001517168_388431872.pth [2023-12-27 16:10:37,529][20347] Updated weights for policy 0, policy_version 1519943 (0.0007) [2023-12-27 16:10:37,586][20347] Updated weights for policy 0, policy_version 1519953 (0.0009) [2023-12-27 16:10:38,057][20381] Updated weights for policy 1, policy_version 1518393 (0.0008) [2023-12-27 16:10:38,115][20381] Updated weights for policy 1, policy_version 1518403 (0.0007) [2023-12-27 16:10:38,175][20381] Updated weights for policy 1, policy_version 1518413 (0.0008) [2023-12-27 16:10:38,315][20347] Updated weights for policy 0, policy_version 1519963 (0.0008) [2023-12-27 16:10:38,378][20347] Updated weights for policy 0, policy_version 1519973 (0.0008) [2023-12-27 16:10:38,435][20347] Updated weights for policy 0, policy_version 1519983 (0.0006) [2023-12-27 16:10:38,828][20381] Updated weights for policy 1, policy_version 1518423 (0.0007) [2023-12-27 16:10:38,888][20381] Updated weights for policy 1, policy_version 1518433 (0.0008) [2023-12-27 16:10:38,943][20381] Updated weights for policy 1, policy_version 1518443 (0.0007) [2023-12-27 16:10:39,088][20347] Updated weights for policy 0, policy_version 1519993 (0.0006) [2023-12-27 16:10:39,143][20347] Updated weights for policy 0, policy_version 1520003 (0.0008) [2023-12-27 16:10:39,198][20347] Updated weights for policy 0, policy_version 1520013 (0.0008) [2023-12-27 16:10:39,272][20347] Updated weights for policy 0, policy_version 1520023 (0.0008) [2023-12-27 16:10:39,635][20381] Updated weights for policy 1, policy_version 1518453 (0.0005) [2023-12-27 16:10:39,694][20381] Updated weights for policy 1, policy_version 1518463 (0.0008) [2023-12-27 16:10:39,745][20381] Updated weights for policy 1, policy_version 1518473 (0.0008) [2023-12-27 16:10:40,000][20347] Updated weights for policy 0, policy_version 1520033 (0.0007) [2023-12-27 16:10:40,050][20347] Updated weights for policy 0, policy_version 1520043 (0.0008) [2023-12-27 16:10:40,103][20347] Updated weights for policy 0, policy_version 1520053 (0.0007) [2023-12-27 16:10:40,494][20381] Updated weights for policy 1, policy_version 1518483 (0.0009) [2023-12-27 16:10:40,550][20381] Updated weights for policy 1, policy_version 1518493 (0.0008) [2023-12-27 16:10:40,599][20381] Updated weights for policy 1, policy_version 1518503 (0.0008) [2023-12-27 16:10:40,830][20347] Updated weights for policy 0, policy_version 1520063 (0.0010) [2023-12-27 16:10:40,886][20347] Updated weights for policy 0, policy_version 1520073 (0.0010) [2023-12-27 16:10:40,949][20347] Updated weights for policy 0, policy_version 1520083 (0.0010) [2023-12-27 16:10:41,349][20381] Updated weights for policy 1, policy_version 1518513 (0.0007) [2023-12-27 16:10:41,417][20381] Updated weights for policy 1, policy_version 1518523 (0.0012) [2023-12-27 16:10:41,477][20381] Updated weights for policy 1, policy_version 1518533 (0.0008) [2023-12-27 16:10:41,538][20381] Updated weights for policy 1, policy_version 1518543 (0.0008) [2023-12-27 16:10:41,586][20347] Updated weights for policy 0, policy_version 1520093 (0.0009) [2023-12-27 16:10:41,654][20347] Updated weights for policy 0, policy_version 1520103 (0.0008) [2023-12-27 16:10:41,722][20347] Updated weights for policy 0, policy_version 1520113 (0.0007) [2023-12-27 16:10:42,316][20381] Updated weights for policy 1, policy_version 1518553 (0.0007) [2023-12-27 16:10:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 778010624. Throughput: 0: 10222.3, 1: 10219.0. Samples: 778006712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:42,363][19170] Avg episode reward: [(0, '970.760'), (1, '902.420')] [2023-12-27 16:10:42,385][20381] Updated weights for policy 1, policy_version 1518563 (0.0006) [2023-12-27 16:10:42,442][20381] Updated weights for policy 1, policy_version 1518573 (0.0006) [2023-12-27 16:10:42,509][20347] Updated weights for policy 0, policy_version 1520123 (0.0008) [2023-12-27 16:10:42,561][20347] Updated weights for policy 0, policy_version 1520133 (0.0008) [2023-12-27 16:10:42,608][20347] Updated weights for policy 0, policy_version 1520143 (0.0008) [2023-12-27 16:10:43,108][20381] Updated weights for policy 1, policy_version 1518583 (0.0007) [2023-12-27 16:10:43,155][20381] Updated weights for policy 1, policy_version 1518593 (0.0007) [2023-12-27 16:10:43,205][20381] Updated weights for policy 1, policy_version 1518603 (0.0007) [2023-12-27 16:10:43,232][20347] Updated weights for policy 0, policy_version 1520153 (0.0007) [2023-12-27 16:10:43,292][20347] Updated weights for policy 0, policy_version 1520163 (0.0005) [2023-12-27 16:10:43,348][20347] Updated weights for policy 0, policy_version 1520173 (0.0005) [2023-12-27 16:10:43,397][20347] Updated weights for policy 0, policy_version 1520183 (0.0006) [2023-12-27 16:10:43,911][20381] Updated weights for policy 1, policy_version 1518613 (0.0007) [2023-12-27 16:10:43,977][20381] Updated weights for policy 1, policy_version 1518623 (0.0008) [2023-12-27 16:10:44,037][20381] Updated weights for policy 1, policy_version 1518633 (0.0008) [2023-12-27 16:10:44,067][20347] Updated weights for policy 0, policy_version 1520193 (0.0007) [2023-12-27 16:10:44,123][20347] Updated weights for policy 0, policy_version 1520203 (0.0008) [2023-12-27 16:10:44,183][20347] Updated weights for policy 0, policy_version 1520213 (0.0008) [2023-12-27 16:10:44,727][20381] Updated weights for policy 1, policy_version 1518643 (0.0008) [2023-12-27 16:10:44,791][20381] Updated weights for policy 1, policy_version 1518653 (0.0007) [2023-12-27 16:10:44,855][20381] Updated weights for policy 1, policy_version 1518663 (0.0007) [2023-12-27 16:10:44,909][20347] Updated weights for policy 0, policy_version 1520223 (0.0009) [2023-12-27 16:10:44,969][20347] Updated weights for policy 0, policy_version 1520233 (0.0009) [2023-12-27 16:10:45,034][20347] Updated weights for policy 0, policy_version 1520243 (0.0009) [2023-12-27 16:10:45,510][20381] Updated weights for policy 1, policy_version 1518673 (0.0007) [2023-12-27 16:10:45,565][20381] Updated weights for policy 1, policy_version 1518683 (0.0008) [2023-12-27 16:10:45,631][20381] Updated weights for policy 1, policy_version 1518693 (0.0008) [2023-12-27 16:10:45,693][20381] Updated weights for policy 1, policy_version 1518703 (0.0007) [2023-12-27 16:10:45,767][20347] Updated weights for policy 0, policy_version 1520253 (0.0009) [2023-12-27 16:10:45,819][20347] Updated weights for policy 0, policy_version 1520263 (0.0007) [2023-12-27 16:10:45,872][20347] Updated weights for policy 0, policy_version 1520273 (0.0006) [2023-12-27 16:10:46,297][20381] Updated weights for policy 1, policy_version 1518713 (0.0006) [2023-12-27 16:10:46,351][20381] Updated weights for policy 1, policy_version 1518723 (0.0005) [2023-12-27 16:10:46,402][20381] Updated weights for policy 1, policy_version 1518733 (0.0005) [2023-12-27 16:10:46,568][20347] Updated weights for policy 0, policy_version 1520283 (0.0006) [2023-12-27 16:10:46,613][20347] Updated weights for policy 0, policy_version 1520293 (0.0008) [2023-12-27 16:10:46,678][20347] Updated weights for policy 0, policy_version 1520304 (0.0008) [2023-12-27 16:10:46,961][20381] Updated weights for policy 1, policy_version 1518743 (0.0008) [2023-12-27 16:10:47,016][20381] Updated weights for policy 1, policy_version 1518753 (0.0007) [2023-12-27 16:10:47,065][20381] Updated weights for policy 1, policy_version 1518763 (0.0008) [2023-12-27 16:10:47,331][20347] Updated weights for policy 0, policy_version 1520314 (0.0005) [2023-12-27 16:10:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 778117120. Throughput: 0: 10218.8, 1: 10235.8. Samples: 778127996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:47,363][19170] Avg episode reward: [(0, '951.220'), (1, '863.620')] [2023-12-27 16:10:47,387][20347] Updated weights for policy 0, policy_version 1520324 (0.0005) [2023-12-27 16:10:47,452][20347] Updated weights for policy 0, policy_version 1520334 (0.0005) [2023-12-27 16:10:47,510][20347] Updated weights for policy 0, policy_version 1520344 (0.0005) [2023-12-27 16:10:47,773][20381] Updated weights for policy 1, policy_version 1518773 (0.0007) [2023-12-27 16:10:47,821][20381] Updated weights for policy 1, policy_version 1518783 (0.0008) [2023-12-27 16:10:47,873][20381] Updated weights for policy 1, policy_version 1518793 (0.0010) [2023-12-27 16:10:48,036][20347] Updated weights for policy 0, policy_version 1520354 (0.0005) [2023-12-27 16:10:48,094][20347] Updated weights for policy 0, policy_version 1520364 (0.0005) [2023-12-27 16:10:48,153][20347] Updated weights for policy 0, policy_version 1520374 (0.0005) [2023-12-27 16:10:48,627][20381] Updated weights for policy 1, policy_version 1518803 (0.0008) [2023-12-27 16:10:48,682][20381] Updated weights for policy 1, policy_version 1518813 (0.0008) [2023-12-27 16:10:48,684][20347] Updated weights for policy 0, policy_version 1520384 (0.0006) [2023-12-27 16:10:48,739][20381] Updated weights for policy 1, policy_version 1518823 (0.0006) [2023-12-27 16:10:48,740][20347] Updated weights for policy 0, policy_version 1520394 (0.0005) [2023-12-27 16:10:48,798][20347] Updated weights for policy 0, policy_version 1520404 (0.0005) [2023-12-27 16:10:49,374][20381] Updated weights for policy 1, policy_version 1518833 (0.0006) [2023-12-27 16:10:49,432][20381] Updated weights for policy 1, policy_version 1518843 (0.0006) [2023-12-27 16:10:49,490][20381] Updated weights for policy 1, policy_version 1518853 (0.0006) [2023-12-27 16:10:49,499][20347] Updated weights for policy 0, policy_version 1520414 (0.0006) [2023-12-27 16:10:49,543][20381] Updated weights for policy 1, policy_version 1518863 (0.0005) [2023-12-27 16:10:49,559][20347] Updated weights for policy 0, policy_version 1520424 (0.0006) [2023-12-27 16:10:49,617][20347] Updated weights for policy 0, policy_version 1520434 (0.0007) [2023-12-27 16:10:50,217][20381] Updated weights for policy 1, policy_version 1518873 (0.0008) [2023-12-27 16:10:50,273][20381] Updated weights for policy 1, policy_version 1518883 (0.0007) [2023-12-27 16:10:50,325][20381] Updated weights for policy 1, policy_version 1518893 (0.0007) [2023-12-27 16:10:50,332][20347] Updated weights for policy 0, policy_version 1520444 (0.0007) [2023-12-27 16:10:50,388][20347] Updated weights for policy 0, policy_version 1520454 (0.0009) [2023-12-27 16:10:50,440][20347] Updated weights for policy 0, policy_version 1520464 (0.0010) [2023-12-27 16:10:51,096][20381] Updated weights for policy 1, policy_version 1518903 (0.0008) [2023-12-27 16:10:51,110][20347] Updated weights for policy 0, policy_version 1520474 (0.0009) [2023-12-27 16:10:51,157][20381] Updated weights for policy 1, policy_version 1518913 (0.0008) [2023-12-27 16:10:51,177][20347] Updated weights for policy 0, policy_version 1520484 (0.0009) [2023-12-27 16:10:51,212][20381] Updated weights for policy 1, policy_version 1518923 (0.0008) [2023-12-27 16:10:51,230][20347] Updated weights for policy 0, policy_version 1520494 (0.0007) [2023-12-27 16:10:51,279][20347] Updated weights for policy 0, policy_version 1520504 (0.0008) [2023-12-27 16:10:51,846][20381] Updated weights for policy 1, policy_version 1518933 (0.0008) [2023-12-27 16:10:51,901][20381] Updated weights for policy 1, policy_version 1518943 (0.0007) [2023-12-27 16:10:51,957][20381] Updated weights for policy 1, policy_version 1518953 (0.0008) [2023-12-27 16:10:52,054][20347] Updated weights for policy 0, policy_version 1520514 (0.0009) [2023-12-27 16:10:52,116][20347] Updated weights for policy 0, policy_version 1520524 (0.0007) [2023-12-27 16:10:52,174][20347] Updated weights for policy 0, policy_version 1520534 (0.0007) [2023-12-27 16:10:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 778223616. Throughput: 0: 10271.0, 1: 10191.7. Samples: 778190968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:52,363][19170] Avg episode reward: [(0, '970.690'), (1, '844.160')] [2023-12-27 16:10:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001518960_388890624.pth... [2023-12-27 16:10:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001520536_389332992.pth... [2023-12-27 16:10:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001519352_389029888.pth [2023-12-27 16:10:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001517776_388587520.pth [2023-12-27 16:10:52,693][20381] Updated weights for policy 1, policy_version 1518963 (0.0007) [2023-12-27 16:10:52,752][20381] Updated weights for policy 1, policy_version 1518973 (0.0005) [2023-12-27 16:10:52,808][20381] Updated weights for policy 1, policy_version 1518983 (0.0006) [2023-12-27 16:10:52,808][20347] Updated weights for policy 0, policy_version 1520544 (0.0006) [2023-12-27 16:10:52,864][20347] Updated weights for policy 0, policy_version 1520554 (0.0010) [2023-12-27 16:10:52,923][20347] Updated weights for policy 0, policy_version 1520564 (0.0009) [2023-12-27 16:10:53,416][20381] Updated weights for policy 1, policy_version 1518993 (0.0005) [2023-12-27 16:10:53,479][20381] Updated weights for policy 1, policy_version 1519003 (0.0008) [2023-12-27 16:10:53,539][20381] Updated weights for policy 1, policy_version 1519013 (0.0007) [2023-12-27 16:10:53,579][20347] Updated weights for policy 0, policy_version 1520574 (0.0008) [2023-12-27 16:10:53,592][20381] Updated weights for policy 1, policy_version 1519023 (0.0006) [2023-12-27 16:10:53,631][20347] Updated weights for policy 0, policy_version 1520584 (0.0007) [2023-12-27 16:10:53,682][20347] Updated weights for policy 0, policy_version 1520594 (0.0007) [2023-12-27 16:10:54,252][20381] Updated weights for policy 1, policy_version 1519033 (0.0008) [2023-12-27 16:10:54,283][20347] Updated weights for policy 0, policy_version 1520604 (0.0005) [2023-12-27 16:10:54,310][20381] Updated weights for policy 1, policy_version 1519043 (0.0008) [2023-12-27 16:10:54,336][20347] Updated weights for policy 0, policy_version 1520614 (0.0006) [2023-12-27 16:10:54,363][20381] Updated weights for policy 1, policy_version 1519053 (0.0008) [2023-12-27 16:10:54,392][20347] Updated weights for policy 0, policy_version 1520624 (0.0008) [2023-12-27 16:10:55,040][20381] Updated weights for policy 1, policy_version 1519063 (0.0007) [2023-12-27 16:10:55,082][20347] Updated weights for policy 0, policy_version 1520634 (0.0008) [2023-12-27 16:10:55,091][20381] Updated weights for policy 1, policy_version 1519073 (0.0007) [2023-12-27 16:10:55,139][20347] Updated weights for policy 0, policy_version 1520644 (0.0005) [2023-12-27 16:10:55,144][20381] Updated weights for policy 1, policy_version 1519083 (0.0006) [2023-12-27 16:10:55,189][20347] Updated weights for policy 0, policy_version 1520654 (0.0007) [2023-12-27 16:10:55,241][20347] Updated weights for policy 0, policy_version 1520664 (0.0008) [2023-12-27 16:10:55,857][20381] Updated weights for policy 1, policy_version 1519093 (0.0007) [2023-12-27 16:10:55,917][20381] Updated weights for policy 1, policy_version 1519103 (0.0008) [2023-12-27 16:10:55,924][20347] Updated weights for policy 0, policy_version 1520674 (0.0006) [2023-12-27 16:10:55,973][20381] Updated weights for policy 1, policy_version 1519113 (0.0008) [2023-12-27 16:10:55,980][20347] Updated weights for policy 0, policy_version 1520684 (0.0006) [2023-12-27 16:10:56,036][20347] Updated weights for policy 0, policy_version 1520694 (0.0006) [2023-12-27 16:10:56,603][20347] Updated weights for policy 0, policy_version 1520704 (0.0006) [2023-12-27 16:10:56,654][20347] Updated weights for policy 0, policy_version 1520714 (0.0005) [2023-12-27 16:10:56,700][20347] Updated weights for policy 0, policy_version 1520724 (0.0005) [2023-12-27 16:10:56,703][20381] Updated weights for policy 1, policy_version 1519123 (0.0007) [2023-12-27 16:10:56,760][20381] Updated weights for policy 1, policy_version 1519133 (0.0007) [2023-12-27 16:10:56,808][20381] Updated weights for policy 1, policy_version 1519143 (0.0007) [2023-12-27 16:10:57,284][20347] Updated weights for policy 0, policy_version 1520734 (0.0007) [2023-12-27 16:10:57,340][20347] Updated weights for policy 0, policy_version 1520744 (0.0009) [2023-12-27 16:10:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 778321920. Throughput: 0: 10249.4, 1: 10220.4. Samples: 778314452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:10:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '873.050')] [2023-12-27 16:10:57,396][20347] Updated weights for policy 0, policy_version 1520754 (0.0007) [2023-12-27 16:10:57,451][20381] Updated weights for policy 1, policy_version 1519153 (0.0007) [2023-12-27 16:10:57,510][20381] Updated weights for policy 1, policy_version 1519163 (0.0007) [2023-12-27 16:10:57,560][20381] Updated weights for policy 1, policy_version 1519173 (0.0005) [2023-12-27 16:10:57,607][20381] Updated weights for policy 1, policy_version 1519183 (0.0005) [2023-12-27 16:10:58,073][20347] Updated weights for policy 0, policy_version 1520764 (0.0007) [2023-12-27 16:10:58,118][20347] Updated weights for policy 0, policy_version 1520774 (0.0007) [2023-12-27 16:10:58,173][20347] Updated weights for policy 0, policy_version 1520784 (0.0006) [2023-12-27 16:10:58,189][20381] Updated weights for policy 1, policy_version 1519193 (0.0006) [2023-12-27 16:10:58,252][20381] Updated weights for policy 1, policy_version 1519203 (0.0006) [2023-12-27 16:10:58,310][20381] Updated weights for policy 1, policy_version 1519213 (0.0007) [2023-12-27 16:10:58,942][20347] Updated weights for policy 0, policy_version 1520794 (0.0008) [2023-12-27 16:10:59,004][20347] Updated weights for policy 0, policy_version 1520804 (0.0008) [2023-12-27 16:10:59,059][20347] Updated weights for policy 0, policy_version 1520814 (0.0007) [2023-12-27 16:10:59,075][20381] Updated weights for policy 1, policy_version 1519223 (0.0008) [2023-12-27 16:10:59,125][20347] Updated weights for policy 0, policy_version 1520824 (0.0008) [2023-12-27 16:10:59,147][20381] Updated weights for policy 1, policy_version 1519233 (0.0008) [2023-12-27 16:10:59,205][20381] Updated weights for policy 1, policy_version 1519243 (0.0008) [2023-12-27 16:10:59,851][20347] Updated weights for policy 0, policy_version 1520834 (0.0009) [2023-12-27 16:10:59,889][20381] Updated weights for policy 1, policy_version 1519253 (0.0008) [2023-12-27 16:10:59,907][20347] Updated weights for policy 0, policy_version 1520844 (0.0006) [2023-12-27 16:10:59,952][20381] Updated weights for policy 1, policy_version 1519263 (0.0006) [2023-12-27 16:10:59,971][20347] Updated weights for policy 0, policy_version 1520854 (0.0007) [2023-12-27 16:11:00,013][20381] Updated weights for policy 1, policy_version 1519273 (0.0007) [2023-12-27 16:11:00,572][20381] Updated weights for policy 1, policy_version 1519283 (0.0008) [2023-12-27 16:11:00,592][20347] Updated weights for policy 0, policy_version 1520864 (0.0009) [2023-12-27 16:11:00,616][20381] Updated weights for policy 1, policy_version 1519293 (0.0005) [2023-12-27 16:11:00,640][20347] Updated weights for policy 0, policy_version 1520874 (0.0010) [2023-12-27 16:11:00,666][20381] Updated weights for policy 1, policy_version 1519303 (0.0006) [2023-12-27 16:11:00,683][20347] Updated weights for policy 0, policy_version 1520884 (0.0006) [2023-12-27 16:11:01,376][20381] Updated weights for policy 1, policy_version 1519313 (0.0006) [2023-12-27 16:11:01,386][20347] Updated weights for policy 0, policy_version 1520894 (0.0010) [2023-12-27 16:11:01,431][20381] Updated weights for policy 1, policy_version 1519323 (0.0008) [2023-12-27 16:11:01,436][20347] Updated weights for policy 0, policy_version 1520904 (0.0010) [2023-12-27 16:11:01,486][20381] Updated weights for policy 1, policy_version 1519333 (0.0008) [2023-12-27 16:11:01,489][20347] Updated weights for policy 0, policy_version 1520914 (0.0010) [2023-12-27 16:11:01,542][20381] Updated weights for policy 1, policy_version 1519343 (0.0006) [2023-12-27 16:11:02,143][20347] Updated weights for policy 0, policy_version 1520924 (0.0010) [2023-12-27 16:11:02,196][20381] Updated weights for policy 1, policy_version 1519353 (0.0007) [2023-12-27 16:11:02,211][20347] Updated weights for policy 0, policy_version 1520934 (0.0010) [2023-12-27 16:11:02,256][20381] Updated weights for policy 1, policy_version 1519363 (0.0006) [2023-12-27 16:11:02,274][20347] Updated weights for policy 0, policy_version 1520944 (0.0010) [2023-12-27 16:11:02,323][20381] Updated weights for policy 1, policy_version 1519373 (0.0008) [2023-12-27 16:11:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 778436608. Throughput: 0: 10237.5, 1: 10252.9. Samples: 778439000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:02,364][19170] Avg episode reward: [(0, '1000.000'), (1, '863.400')] [2023-12-27 16:11:02,921][20347] Updated weights for policy 0, policy_version 1520954 (0.0011) [2023-12-27 16:11:02,971][20347] Updated weights for policy 0, policy_version 1520964 (0.0007) [2023-12-27 16:11:03,025][20347] Updated weights for policy 0, policy_version 1520974 (0.0010) [2023-12-27 16:11:03,041][20381] Updated weights for policy 1, policy_version 1519383 (0.0010) [2023-12-27 16:11:03,078][20347] Updated weights for policy 0, policy_version 1520984 (0.0010) [2023-12-27 16:11:03,103][20381] Updated weights for policy 1, policy_version 1519393 (0.0011) [2023-12-27 16:11:03,156][20381] Updated weights for policy 1, policy_version 1519403 (0.0009) [2023-12-27 16:11:03,736][20347] Updated weights for policy 0, policy_version 1520994 (0.0006) [2023-12-27 16:11:03,770][20381] Updated weights for policy 1, policy_version 1519413 (0.0005) [2023-12-27 16:11:03,787][20347] Updated weights for policy 0, policy_version 1521004 (0.0010) [2023-12-27 16:11:03,824][20381] Updated weights for policy 1, policy_version 1519423 (0.0005) [2023-12-27 16:11:03,836][20347] Updated weights for policy 0, policy_version 1521014 (0.0010) [2023-12-27 16:11:03,884][20381] Updated weights for policy 1, policy_version 1519433 (0.0007) [2023-12-27 16:11:04,549][20381] Updated weights for policy 1, policy_version 1519443 (0.0007) [2023-12-27 16:11:04,559][20347] Updated weights for policy 0, policy_version 1521024 (0.0010) [2023-12-27 16:11:04,601][20381] Updated weights for policy 1, policy_version 1519453 (0.0010) [2023-12-27 16:11:04,605][20347] Updated weights for policy 0, policy_version 1521034 (0.0010) [2023-12-27 16:11:04,657][20347] Updated weights for policy 0, policy_version 1521044 (0.0010) [2023-12-27 16:11:04,657][20381] Updated weights for policy 1, policy_version 1519463 (0.0010) [2023-12-27 16:11:05,278][20347] Updated weights for policy 0, policy_version 1521054 (0.0008) [2023-12-27 16:11:05,288][20381] Updated weights for policy 1, policy_version 1519473 (0.0011) [2023-12-27 16:11:05,334][20347] Updated weights for policy 0, policy_version 1521064 (0.0005) [2023-12-27 16:11:05,342][20381] Updated weights for policy 1, policy_version 1519483 (0.0010) [2023-12-27 16:11:05,390][20347] Updated weights for policy 0, policy_version 1521074 (0.0009) [2023-12-27 16:11:05,397][20381] Updated weights for policy 1, policy_version 1519493 (0.0010) [2023-12-27 16:11:05,448][20381] Updated weights for policy 1, policy_version 1519503 (0.0010) [2023-12-27 16:11:06,051][20347] Updated weights for policy 0, policy_version 1521084 (0.0009) [2023-12-27 16:11:06,112][20347] Updated weights for policy 0, policy_version 1521094 (0.0007) [2023-12-27 16:11:06,138][20381] Updated weights for policy 1, policy_version 1519513 (0.0009) [2023-12-27 16:11:06,174][20347] Updated weights for policy 0, policy_version 1521104 (0.0009) [2023-12-27 16:11:06,198][20381] Updated weights for policy 1, policy_version 1519523 (0.0008) [2023-12-27 16:11:06,266][20381] Updated weights for policy 1, policy_version 1519533 (0.0008) [2023-12-27 16:11:06,881][20347] Updated weights for policy 0, policy_version 1521114 (0.0008) [2023-12-27 16:11:06,934][20381] Updated weights for policy 1, policy_version 1519543 (0.0007) [2023-12-27 16:11:06,940][20347] Updated weights for policy 0, policy_version 1521124 (0.0008) [2023-12-27 16:11:06,997][20381] Updated weights for policy 1, policy_version 1519553 (0.0006) [2023-12-27 16:11:07,006][20347] Updated weights for policy 0, policy_version 1521134 (0.0006) [2023-12-27 16:11:07,052][20381] Updated weights for policy 1, policy_version 1519563 (0.0007) [2023-12-27 16:11:07,073][20347] Updated weights for policy 0, policy_version 1521144 (0.0005) [2023-12-27 16:11:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 778534912. Throughput: 0: 10238.4, 1: 10252.1. Samples: 778501084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.940')] [2023-12-27 16:11:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001519568_389046272.pth... [2023-12-27 16:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001521144_389488640.pth... [2023-12-27 16:11:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001518384_388743168.pth [2023-12-27 16:11:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001519928_389177344.pth [2023-12-27 16:11:07,671][20347] Updated weights for policy 0, policy_version 1521154 (0.0008) [2023-12-27 16:11:07,712][20381] Updated weights for policy 1, policy_version 1519573 (0.0007) [2023-12-27 16:11:07,727][20347] Updated weights for policy 0, policy_version 1521164 (0.0007) [2023-12-27 16:11:07,770][20381] Updated weights for policy 1, policy_version 1519583 (0.0005) [2023-12-27 16:11:07,787][20347] Updated weights for policy 0, policy_version 1521174 (0.0008) [2023-12-27 16:11:07,833][20381] Updated weights for policy 1, policy_version 1519593 (0.0005) [2023-12-27 16:11:08,464][20381] Updated weights for policy 1, policy_version 1519603 (0.0010) [2023-12-27 16:11:08,481][20347] Updated weights for policy 0, policy_version 1521184 (0.0010) [2023-12-27 16:11:08,523][20381] Updated weights for policy 1, policy_version 1519613 (0.0011) [2023-12-27 16:11:08,531][20347] Updated weights for policy 0, policy_version 1521194 (0.0010) [2023-12-27 16:11:08,579][20381] Updated weights for policy 1, policy_version 1519623 (0.0010) [2023-12-27 16:11:08,580][20347] Updated weights for policy 0, policy_version 1521204 (0.0010) [2023-12-27 16:11:09,185][20347] Updated weights for policy 0, policy_version 1521214 (0.0007) [2023-12-27 16:11:09,234][20381] Updated weights for policy 1, policy_version 1519633 (0.0007) [2023-12-27 16:11:09,262][20347] Updated weights for policy 0, policy_version 1521224 (0.0012) [2023-12-27 16:11:09,295][20381] Updated weights for policy 1, policy_version 1519643 (0.0010) [2023-12-27 16:11:09,322][20347] Updated weights for policy 0, policy_version 1521234 (0.0006) [2023-12-27 16:11:09,353][20381] Updated weights for policy 1, policy_version 1519653 (0.0010) [2023-12-27 16:11:09,415][20381] Updated weights for policy 1, policy_version 1519663 (0.0011) [2023-12-27 16:11:10,061][20347] Updated weights for policy 0, policy_version 1521244 (0.0008) [2023-12-27 16:11:10,113][20347] Updated weights for policy 0, policy_version 1521254 (0.0006) [2023-12-27 16:11:10,118][20381] Updated weights for policy 1, policy_version 1519673 (0.0011) [2023-12-27 16:11:10,175][20347] Updated weights for policy 0, policy_version 1521264 (0.0006) [2023-12-27 16:11:10,177][20381] Updated weights for policy 1, policy_version 1519683 (0.0010) [2023-12-27 16:11:10,238][20381] Updated weights for policy 1, policy_version 1519693 (0.0007) [2023-12-27 16:11:10,833][20347] Updated weights for policy 0, policy_version 1521274 (0.0008) [2023-12-27 16:11:10,897][20347] Updated weights for policy 0, policy_version 1521284 (0.0007) [2023-12-27 16:11:10,911][20381] Updated weights for policy 1, policy_version 1519703 (0.0006) [2023-12-27 16:11:10,951][20347] Updated weights for policy 0, policy_version 1521294 (0.0010) [2023-12-27 16:11:10,970][20381] Updated weights for policy 1, policy_version 1519713 (0.0005) [2023-12-27 16:11:11,003][20347] Updated weights for policy 0, policy_version 1521304 (0.0010) [2023-12-27 16:11:11,054][20381] Updated weights for policy 1, policy_version 1519723 (0.0006) [2023-12-27 16:11:11,827][20347] Updated weights for policy 0, policy_version 1521314 (0.0007) [2023-12-27 16:11:11,875][20381] Updated weights for policy 1, policy_version 1519733 (0.0007) [2023-12-27 16:11:11,885][20347] Updated weights for policy 0, policy_version 1521324 (0.0006) [2023-12-27 16:11:11,932][20381] Updated weights for policy 1, policy_version 1519743 (0.0007) [2023-12-27 16:11:11,946][20347] Updated weights for policy 0, policy_version 1521334 (0.0006) [2023-12-27 16:11:11,990][20381] Updated weights for policy 1, policy_version 1519753 (0.0008) [2023-12-27 16:11:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 778633216. Throughput: 0: 10281.3, 1: 10287.1. Samples: 778626084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:12,363][19170] Avg episode reward: [(0, '980.470'), (1, '882.890')] [2023-12-27 16:11:12,644][20347] Updated weights for policy 0, policy_version 1521344 (0.0007) [2023-12-27 16:11:12,699][20347] Updated weights for policy 0, policy_version 1521354 (0.0007) [2023-12-27 16:11:12,708][20381] Updated weights for policy 1, policy_version 1519763 (0.0008) [2023-12-27 16:11:12,756][20347] Updated weights for policy 0, policy_version 1521364 (0.0005) [2023-12-27 16:11:12,762][20381] Updated weights for policy 1, policy_version 1519773 (0.0005) [2023-12-27 16:11:12,813][20381] Updated weights for policy 1, policy_version 1519783 (0.0007) [2023-12-27 16:11:13,480][20347] Updated weights for policy 0, policy_version 1521374 (0.0008) [2023-12-27 16:11:13,493][20381] Updated weights for policy 1, policy_version 1519793 (0.0007) [2023-12-27 16:11:13,539][20347] Updated weights for policy 0, policy_version 1521384 (0.0005) [2023-12-27 16:11:13,552][20381] Updated weights for policy 1, policy_version 1519803 (0.0011) [2023-12-27 16:11:13,593][20347] Updated weights for policy 0, policy_version 1521394 (0.0005) [2023-12-27 16:11:13,607][20381] Updated weights for policy 1, policy_version 1519813 (0.0010) [2023-12-27 16:11:13,662][20381] Updated weights for policy 1, policy_version 1519823 (0.0011) [2023-12-27 16:11:14,262][20381] Updated weights for policy 1, policy_version 1519833 (0.0010) [2023-12-27 16:11:14,267][20347] Updated weights for policy 0, policy_version 1521404 (0.0005) [2023-12-27 16:11:14,319][20381] Updated weights for policy 1, policy_version 1519843 (0.0010) [2023-12-27 16:11:14,329][20347] Updated weights for policy 0, policy_version 1521414 (0.0008) [2023-12-27 16:11:14,381][20381] Updated weights for policy 1, policy_version 1519853 (0.0008) [2023-12-27 16:11:14,383][20347] Updated weights for policy 0, policy_version 1521424 (0.0006) [2023-12-27 16:11:15,101][20347] Updated weights for policy 0, policy_version 1521434 (0.0006) [2023-12-27 16:11:15,103][20381] Updated weights for policy 1, policy_version 1519863 (0.0007) [2023-12-27 16:11:15,152][20381] Updated weights for policy 1, policy_version 1519873 (0.0006) [2023-12-27 16:11:15,156][20347] Updated weights for policy 0, policy_version 1521444 (0.0007) [2023-12-27 16:11:15,209][20381] Updated weights for policy 1, policy_version 1519883 (0.0007) [2023-12-27 16:11:15,211][20347] Updated weights for policy 0, policy_version 1521454 (0.0007) [2023-12-27 16:11:15,273][20347] Updated weights for policy 0, policy_version 1521464 (0.0008) [2023-12-27 16:11:15,763][20381] Updated weights for policy 1, policy_version 1519893 (0.0006) [2023-12-27 16:11:15,822][20381] Updated weights for policy 1, policy_version 1519903 (0.0007) [2023-12-27 16:11:15,881][20381] Updated weights for policy 1, policy_version 1519913 (0.0005) [2023-12-27 16:11:15,962][20347] Updated weights for policy 0, policy_version 1521474 (0.0008) [2023-12-27 16:11:16,017][20347] Updated weights for policy 0, policy_version 1521484 (0.0007) [2023-12-27 16:11:16,075][20347] Updated weights for policy 0, policy_version 1521494 (0.0009) [2023-12-27 16:11:16,474][20381] Updated weights for policy 1, policy_version 1519923 (0.0005) [2023-12-27 16:11:16,525][20381] Updated weights for policy 1, policy_version 1519933 (0.0005) [2023-12-27 16:11:16,578][20381] Updated weights for policy 1, policy_version 1519943 (0.0006) [2023-12-27 16:11:16,764][20347] Updated weights for policy 0, policy_version 1521504 (0.0010) [2023-12-27 16:11:16,825][20347] Updated weights for policy 0, policy_version 1521514 (0.0010) [2023-12-27 16:11:16,878][20347] Updated weights for policy 0, policy_version 1521524 (0.0010) [2023-12-27 16:11:17,244][20381] Updated weights for policy 1, policy_version 1519953 (0.0007) [2023-12-27 16:11:17,296][20381] Updated weights for policy 1, policy_version 1519963 (0.0008) [2023-12-27 16:11:17,351][20381] Updated weights for policy 1, policy_version 1519973 (0.0006) [2023-12-27 16:11:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 778731520. Throughput: 0: 10192.9, 1: 10353.1. Samples: 778747540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:17,363][19170] Avg episode reward: [(0, '980.470'), (1, '844.090')] [2023-12-27 16:11:17,399][20381] Updated weights for policy 1, policy_version 1519983 (0.0008) [2023-12-27 16:11:17,450][20347] Updated weights for policy 0, policy_version 1521534 (0.0009) [2023-12-27 16:11:17,499][20347] Updated weights for policy 0, policy_version 1521544 (0.0010) [2023-12-27 16:11:17,556][20347] Updated weights for policy 0, policy_version 1521554 (0.0009) [2023-12-27 16:11:18,076][20381] Updated weights for policy 1, policy_version 1519993 (0.0007) [2023-12-27 16:11:18,126][20381] Updated weights for policy 1, policy_version 1520003 (0.0008) [2023-12-27 16:11:18,176][20347] Updated weights for policy 0, policy_version 1521564 (0.0010) [2023-12-27 16:11:18,182][20381] Updated weights for policy 1, policy_version 1520013 (0.0007) [2023-12-27 16:11:18,230][20347] Updated weights for policy 0, policy_version 1521574 (0.0008) [2023-12-27 16:11:18,288][20347] Updated weights for policy 0, policy_version 1521584 (0.0007) [2023-12-27 16:11:18,899][20381] Updated weights for policy 1, policy_version 1520023 (0.0007) [2023-12-27 16:11:18,956][20381] Updated weights for policy 1, policy_version 1520033 (0.0008) [2023-12-27 16:11:19,004][20347] Updated weights for policy 0, policy_version 1521594 (0.0007) [2023-12-27 16:11:19,010][20381] Updated weights for policy 1, policy_version 1520043 (0.0007) [2023-12-27 16:11:19,055][20347] Updated weights for policy 0, policy_version 1521604 (0.0006) [2023-12-27 16:11:19,101][20347] Updated weights for policy 0, policy_version 1521614 (0.0007) [2023-12-27 16:11:19,157][20347] Updated weights for policy 0, policy_version 1521624 (0.0008) [2023-12-27 16:11:19,663][20381] Updated weights for policy 1, policy_version 1520053 (0.0007) [2023-12-27 16:11:19,719][20381] Updated weights for policy 1, policy_version 1520063 (0.0007) [2023-12-27 16:11:19,776][20381] Updated weights for policy 1, policy_version 1520073 (0.0008) [2023-12-27 16:11:19,929][20347] Updated weights for policy 0, policy_version 1521634 (0.0008) [2023-12-27 16:11:20,001][20347] Updated weights for policy 0, policy_version 1521644 (0.0009) [2023-12-27 16:11:20,064][20347] Updated weights for policy 0, policy_version 1521654 (0.0007) [2023-12-27 16:11:20,492][20381] Updated weights for policy 1, policy_version 1520083 (0.0009) [2023-12-27 16:11:20,547][20381] Updated weights for policy 1, policy_version 1520093 (0.0011) [2023-12-27 16:11:20,610][20381] Updated weights for policy 1, policy_version 1520103 (0.0010) [2023-12-27 16:11:20,765][20347] Updated weights for policy 0, policy_version 1521664 (0.0007) [2023-12-27 16:11:20,828][20347] Updated weights for policy 0, policy_version 1521674 (0.0007) [2023-12-27 16:11:20,881][20347] Updated weights for policy 0, policy_version 1521684 (0.0008) [2023-12-27 16:11:21,322][20381] Updated weights for policy 1, policy_version 1520113 (0.0011) [2023-12-27 16:11:21,390][20381] Updated weights for policy 1, policy_version 1520123 (0.0011) [2023-12-27 16:11:21,456][20381] Updated weights for policy 1, policy_version 1520133 (0.0011) [2023-12-27 16:11:21,515][20381] Updated weights for policy 1, policy_version 1520143 (0.0010) [2023-12-27 16:11:21,598][20347] Updated weights for policy 0, policy_version 1521694 (0.0006) [2023-12-27 16:11:21,663][20347] Updated weights for policy 0, policy_version 1521704 (0.0006) [2023-12-27 16:11:21,723][20347] Updated weights for policy 0, policy_version 1521714 (0.0008) [2023-12-27 16:11:22,191][20381] Updated weights for policy 1, policy_version 1520153 (0.0008) [2023-12-27 16:11:22,252][20381] Updated weights for policy 1, policy_version 1520163 (0.0008) [2023-12-27 16:11:22,312][20381] Updated weights for policy 1, policy_version 1520173 (0.0008) [2023-12-27 16:11:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 778838016. Throughput: 0: 10214.6, 1: 10302.6. Samples: 778807932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '834.470')] [2023-12-27 16:11:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001520176_389201920.pth... [2023-12-27 16:11:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001518960_388890624.pth [2023-12-27 16:11:22,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001521720_389636096.pth... [2023-12-27 16:11:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001520536_389332992.pth [2023-12-27 16:11:22,509][20347] Updated weights for policy 0, policy_version 1521724 (0.0007) [2023-12-27 16:11:22,574][20347] Updated weights for policy 0, policy_version 1521734 (0.0008) [2023-12-27 16:11:22,632][20347] Updated weights for policy 0, policy_version 1521744 (0.0007) [2023-12-27 16:11:23,013][20381] Updated weights for policy 1, policy_version 1520183 (0.0007) [2023-12-27 16:11:23,065][20381] Updated weights for policy 1, policy_version 1520193 (0.0008) [2023-12-27 16:11:23,123][20381] Updated weights for policy 1, policy_version 1520203 (0.0007) [2023-12-27 16:11:23,304][20347] Updated weights for policy 0, policy_version 1521754 (0.0008) [2023-12-27 16:11:23,351][20347] Updated weights for policy 0, policy_version 1521764 (0.0006) [2023-12-27 16:11:23,398][20347] Updated weights for policy 0, policy_version 1521774 (0.0005) [2023-12-27 16:11:23,443][20347] Updated weights for policy 0, policy_version 1521784 (0.0005) [2023-12-27 16:11:23,802][20381] Updated weights for policy 1, policy_version 1520213 (0.0009) [2023-12-27 16:11:23,858][20381] Updated weights for policy 1, policy_version 1520223 (0.0009) [2023-12-27 16:11:23,915][20381] Updated weights for policy 1, policy_version 1520233 (0.0008) [2023-12-27 16:11:24,096][20347] Updated weights for policy 0, policy_version 1521794 (0.0010) [2023-12-27 16:11:24,154][20347] Updated weights for policy 0, policy_version 1521804 (0.0010) [2023-12-27 16:11:24,219][20347] Updated weights for policy 0, policy_version 1521814 (0.0010) [2023-12-27 16:11:24,575][20381] Updated weights for policy 1, policy_version 1520243 (0.0008) [2023-12-27 16:11:24,627][20381] Updated weights for policy 1, policy_version 1520253 (0.0010) [2023-12-27 16:11:24,681][20381] Updated weights for policy 1, policy_version 1520263 (0.0010) [2023-12-27 16:11:24,804][20347] Updated weights for policy 0, policy_version 1521824 (0.0006) [2023-12-27 16:11:24,852][20347] Updated weights for policy 0, policy_version 1521834 (0.0005) [2023-12-27 16:11:24,906][20347] Updated weights for policy 0, policy_version 1521844 (0.0005) [2023-12-27 16:11:25,327][20381] Updated weights for policy 1, policy_version 1520273 (0.0010) [2023-12-27 16:11:25,383][20381] Updated weights for policy 1, policy_version 1520283 (0.0005) [2023-12-27 16:11:25,439][20381] Updated weights for policy 1, policy_version 1520293 (0.0005) [2023-12-27 16:11:25,481][20347] Updated weights for policy 0, policy_version 1521854 (0.0007) [2023-12-27 16:11:25,490][20381] Updated weights for policy 1, policy_version 1520303 (0.0005) [2023-12-27 16:11:25,537][20347] Updated weights for policy 0, policy_version 1521864 (0.0007) [2023-12-27 16:11:25,589][20347] Updated weights for policy 0, policy_version 1521874 (0.0005) [2023-12-27 16:11:26,116][20381] Updated weights for policy 1, policy_version 1520313 (0.0009) [2023-12-27 16:11:26,166][20381] Updated weights for policy 1, policy_version 1520323 (0.0008) [2023-12-27 16:11:26,216][20381] Updated weights for policy 1, policy_version 1520333 (0.0008) [2023-12-27 16:11:26,230][20347] Updated weights for policy 0, policy_version 1521884 (0.0005) [2023-12-27 16:11:26,279][20347] Updated weights for policy 0, policy_version 1521894 (0.0005) [2023-12-27 16:11:26,334][20347] Updated weights for policy 0, policy_version 1521904 (0.0005) [2023-12-27 16:11:26,874][20347] Updated weights for policy 0, policy_version 1521914 (0.0006) [2023-12-27 16:11:26,928][20347] Updated weights for policy 0, policy_version 1521924 (0.0007) [2023-12-27 16:11:26,964][20381] Updated weights for policy 1, policy_version 1520343 (0.0006) [2023-12-27 16:11:26,980][20347] Updated weights for policy 0, policy_version 1521934 (0.0005) [2023-12-27 16:11:27,016][20381] Updated weights for policy 1, policy_version 1520353 (0.0006) [2023-12-27 16:11:27,037][20347] Updated weights for policy 0, policy_version 1521944 (0.0007) [2023-12-27 16:11:27,067][20381] Updated weights for policy 1, policy_version 1520363 (0.0007) [2023-12-27 16:11:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 778944512. Throughput: 0: 10287.4, 1: 10261.5. Samples: 778931408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:27,363][19170] Avg episode reward: [(0, '990.250'), (1, '843.910')] [2023-12-27 16:11:27,696][20381] Updated weights for policy 1, policy_version 1520373 (0.0007) [2023-12-27 16:11:27,725][20347] Updated weights for policy 0, policy_version 1521954 (0.0005) [2023-12-27 16:11:27,752][20381] Updated weights for policy 1, policy_version 1520383 (0.0006) [2023-12-27 16:11:27,770][20347] Updated weights for policy 0, policy_version 1521964 (0.0005) [2023-12-27 16:11:27,809][20381] Updated weights for policy 1, policy_version 1520393 (0.0007) [2023-12-27 16:11:27,814][20347] Updated weights for policy 0, policy_version 1521974 (0.0008) [2023-12-27 16:11:28,439][20347] Updated weights for policy 0, policy_version 1521984 (0.0007) [2023-12-27 16:11:28,497][20347] Updated weights for policy 0, policy_version 1521994 (0.0007) [2023-12-27 16:11:28,522][20381] Updated weights for policy 1, policy_version 1520403 (0.0007) [2023-12-27 16:11:28,557][20347] Updated weights for policy 0, policy_version 1522004 (0.0007) [2023-12-27 16:11:28,578][20381] Updated weights for policy 1, policy_version 1520413 (0.0006) [2023-12-27 16:11:28,637][20381] Updated weights for policy 1, policy_version 1520423 (0.0007) [2023-12-27 16:11:29,210][20347] Updated weights for policy 0, policy_version 1522014 (0.0006) [2023-12-27 16:11:29,270][20347] Updated weights for policy 0, policy_version 1522024 (0.0008) [2023-12-27 16:11:29,322][20347] Updated weights for policy 0, policy_version 1522034 (0.0008) [2023-12-27 16:11:29,383][20381] Updated weights for policy 1, policy_version 1520433 (0.0007) [2023-12-27 16:11:29,437][20381] Updated weights for policy 1, policy_version 1520443 (0.0007) [2023-12-27 16:11:29,491][20381] Updated weights for policy 1, policy_version 1520453 (0.0008) [2023-12-27 16:11:29,552][20381] Updated weights for policy 1, policy_version 1520463 (0.0008) [2023-12-27 16:11:30,061][20347] Updated weights for policy 0, policy_version 1522044 (0.0008) [2023-12-27 16:11:30,121][20347] Updated weights for policy 0, policy_version 1522054 (0.0008) [2023-12-27 16:11:30,182][20347] Updated weights for policy 0, policy_version 1522064 (0.0007) [2023-12-27 16:11:30,236][20292] KL-divergence is very high: 305.6913 [2023-12-27 16:11:30,269][20381] Updated weights for policy 1, policy_version 1520473 (0.0007) [2023-12-27 16:11:30,274][20292] KL-divergence is very high: 377.6865 [2023-12-27 16:11:30,314][20292] KL-divergence is very high: 308.8629 [2023-12-27 16:11:30,318][20381] Updated weights for policy 1, policy_version 1520483 (0.0008) [2023-12-27 16:11:30,352][20292] KL-divergence is very high: 221.5989 [2023-12-27 16:11:30,370][20381] Updated weights for policy 1, policy_version 1520493 (0.0007) [2023-12-27 16:11:30,806][20347] Updated weights for policy 0, policy_version 1522074 (0.0007) [2023-12-27 16:11:30,866][20347] Updated weights for policy 0, policy_version 1522084 (0.0009) [2023-12-27 16:11:30,919][20347] Updated weights for policy 0, policy_version 1522094 (0.0010) [2023-12-27 16:11:30,966][20381] Updated weights for policy 1, policy_version 1520503 (0.0006) [2023-12-27 16:11:30,986][20347] Updated weights for policy 0, policy_version 1522104 (0.0010) [2023-12-27 16:11:31,021][20381] Updated weights for policy 1, policy_version 1520513 (0.0008) [2023-12-27 16:11:31,083][20381] Updated weights for policy 1, policy_version 1520523 (0.0010) [2023-12-27 16:11:31,683][20347] Updated weights for policy 0, policy_version 1522114 (0.0007) [2023-12-27 16:11:31,752][20347] Updated weights for policy 0, policy_version 1522124 (0.0008) [2023-12-27 16:11:31,752][20381] Updated weights for policy 1, policy_version 1520533 (0.0007) [2023-12-27 16:11:31,806][20381] Updated weights for policy 1, policy_version 1520543 (0.0007) [2023-12-27 16:11:31,807][20347] Updated weights for policy 0, policy_version 1522134 (0.0008) [2023-12-27 16:11:31,857][20381] Updated weights for policy 1, policy_version 1520553 (0.0005) [2023-12-27 16:11:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 779042816. Throughput: 0: 10341.2, 1: 10282.4. Samples: 779056060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:32,363][19170] Avg episode reward: [(0, '980.510'), (1, '804.760')] [2023-12-27 16:11:32,450][20292] KL-divergence is very high: 260.1048 [2023-12-27 16:11:32,469][20381] Updated weights for policy 1, policy_version 1520563 (0.0006) [2023-12-27 16:11:32,529][20381] Updated weights for policy 1, policy_version 1520573 (0.0008) [2023-12-27 16:11:32,538][20292] KL-divergence is very high: 477.0331 [2023-12-27 16:11:32,553][20292] KL-divergence is very high: 169.6378 [2023-12-27 16:11:32,553][20347] Updated weights for policy 0, policy_version 1522144 (0.0006) [2023-12-27 16:11:32,578][20381] Updated weights for policy 1, policy_version 1520583 (0.0006) [2023-12-27 16:11:32,579][20292] KL-divergence is very high: 211.0004 [2023-12-27 16:11:32,612][20347] Updated weights for policy 0, policy_version 1522154 (0.0007) [2023-12-27 16:11:32,662][20347] Updated weights for policy 0, policy_version 1522164 (0.0007) [2023-12-27 16:11:33,270][20381] Updated weights for policy 1, policy_version 1520593 (0.0007) [2023-12-27 16:11:33,320][20381] Updated weights for policy 1, policy_version 1520603 (0.0005) [2023-12-27 16:11:33,340][20347] Updated weights for policy 0, policy_version 1522174 (0.0008) [2023-12-27 16:11:33,374][20381] Updated weights for policy 1, policy_version 1520613 (0.0007) [2023-12-27 16:11:33,398][20347] Updated weights for policy 0, policy_version 1522184 (0.0006) [2023-12-27 16:11:33,419][20381] Updated weights for policy 1, policy_version 1520623 (0.0005) [2023-12-27 16:11:33,444][20347] Updated weights for policy 0, policy_version 1522194 (0.0006) [2023-12-27 16:11:34,029][20381] Updated weights for policy 1, policy_version 1520633 (0.0005) [2023-12-27 16:11:34,043][20292] KL-divergence is very high: 153.8287 [2023-12-27 16:11:34,076][20381] Updated weights for policy 1, policy_version 1520643 (0.0005) [2023-12-27 16:11:34,126][20381] Updated weights for policy 1, policy_version 1520653 (0.0005) [2023-12-27 16:11:34,138][20347] Updated weights for policy 0, policy_version 1522204 (0.0008) [2023-12-27 16:11:34,203][20347] Updated weights for policy 0, policy_version 1522214 (0.0007) [2023-12-27 16:11:34,252][20291] KL-divergence is very high: 184.1483 [2023-12-27 16:11:34,267][20347] Updated weights for policy 0, policy_version 1522224 (0.0008) [2023-12-27 16:11:34,302][20291] KL-divergence is very high: 161.5532 [2023-12-27 16:11:34,739][20292] KL-divergence is very high: 199.7050 [2023-12-27 16:11:34,759][20292] KL-divergence is very high: 562.3804 [2023-12-27 16:11:34,766][20292] KL-divergence is very high: 514.1750 [2023-12-27 16:11:34,772][20292] KL-divergence is very high: 299.5140 [2023-12-27 16:11:34,777][20381] Updated weights for policy 1, policy_version 1520663 (0.0006) [2023-12-27 16:11:34,785][20292] KL-divergence is very high: 643.1035 [2023-12-27 16:11:34,794][20292] KL-divergence is very high: 836.4063 [2023-12-27 16:11:34,814][20292] KL-divergence is very high: 568.2065 [2023-12-27 16:11:34,820][20292] KL-divergence is very high: 370.3419 [2023-12-27 16:11:34,825][20292] KL-divergence is very high: 118.7883 [2023-12-27 16:11:34,835][20292] KL-divergence is very high: 183.4692 [2023-12-27 16:11:34,839][20381] Updated weights for policy 1, policy_version 1520673 (0.0006) [2023-12-27 16:11:34,840][20292] KL-divergence is very high: 243.4867 [2023-12-27 16:11:34,875][20292] KL-divergence is very high: 133.6977 [2023-12-27 16:11:34,886][20381] Updated weights for policy 1, policy_version 1520683 (0.0007) [2023-12-27 16:11:35,025][20347] Updated weights for policy 0, policy_version 1522234 (0.0008) [2023-12-27 16:11:35,072][20347] Updated weights for policy 0, policy_version 1522244 (0.0008) [2023-12-27 16:11:35,124][20347] Updated weights for policy 0, policy_version 1522254 (0.0007) [2023-12-27 16:11:35,171][20347] Updated weights for policy 0, policy_version 1522264 (0.0008) [2023-12-27 16:11:35,468][20381] Updated weights for policy 1, policy_version 1520693 (0.0007) [2023-12-27 16:11:35,520][20381] Updated weights for policy 1, policy_version 1520703 (0.0006) [2023-12-27 16:11:35,576][20381] Updated weights for policy 1, policy_version 1520713 (0.0005) [2023-12-27 16:11:35,963][20347] Updated weights for policy 0, policy_version 1522274 (0.0008) [2023-12-27 16:11:36,016][20347] Updated weights for policy 0, policy_version 1522284 (0.0007) [2023-12-27 16:11:36,071][20347] Updated weights for policy 0, policy_version 1522294 (0.0007) [2023-12-27 16:11:36,186][20381] Updated weights for policy 1, policy_version 1520723 (0.0006) [2023-12-27 16:11:36,239][20381] Updated weights for policy 1, policy_version 1520733 (0.0008) [2023-12-27 16:11:36,298][20381] Updated weights for policy 1, policy_version 1520743 (0.0008) [2023-12-27 16:11:36,777][20347] Updated weights for policy 0, policy_version 1522304 (0.0007) [2023-12-27 16:11:36,829][20347] Updated weights for policy 0, policy_version 1522314 (0.0006) [2023-12-27 16:11:36,877][20347] Updated weights for policy 0, policy_version 1522324 (0.0007) [2023-12-27 16:11:36,985][20381] Updated weights for policy 1, policy_version 1520753 (0.0008) [2023-12-27 16:11:37,041][20381] Updated weights for policy 1, policy_version 1520763 (0.0008) [2023-12-27 16:11:37,102][20381] Updated weights for policy 1, policy_version 1520773 (0.0008) [2023-12-27 16:11:37,166][20381] Updated weights for policy 1, policy_version 1520783 (0.0008) [2023-12-27 16:11:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 779149312. Throughput: 0: 10270.6, 1: 10329.4. Samples: 779117968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:37,363][19170] Avg episode reward: [(0, '980.500'), (1, '823.640')] [2023-12-27 16:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001520784_389357568.pth... [2023-12-27 16:11:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001522328_389791744.pth... [2023-12-27 16:11:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001519568_389046272.pth [2023-12-27 16:11:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001521144_389488640.pth [2023-12-27 16:11:37,623][20347] Updated weights for policy 0, policy_version 1522334 (0.0008) [2023-12-27 16:11:37,681][20347] Updated weights for policy 0, policy_version 1522344 (0.0008) [2023-12-27 16:11:37,754][20347] Updated weights for policy 0, policy_version 1522354 (0.0007) [2023-12-27 16:11:37,783][20381] Updated weights for policy 1, policy_version 1520793 (0.0007) [2023-12-27 16:11:37,837][20381] Updated weights for policy 1, policy_version 1520803 (0.0007) [2023-12-27 16:11:37,882][20381] Updated weights for policy 1, policy_version 1520813 (0.0008) [2023-12-27 16:11:38,480][20347] Updated weights for policy 0, policy_version 1522364 (0.0007) [2023-12-27 16:11:38,492][20381] Updated weights for policy 1, policy_version 1520823 (0.0007) [2023-12-27 16:11:38,539][20347] Updated weights for policy 0, policy_version 1522374 (0.0006) [2023-12-27 16:11:38,552][20381] Updated weights for policy 1, policy_version 1520833 (0.0006) [2023-12-27 16:11:38,593][20347] Updated weights for policy 0, policy_version 1522384 (0.0007) [2023-12-27 16:11:38,605][20381] Updated weights for policy 1, policy_version 1520843 (0.0008) [2023-12-27 16:11:39,182][20381] Updated weights for policy 1, policy_version 1520853 (0.0006) [2023-12-27 16:11:39,234][20381] Updated weights for policy 1, policy_version 1520863 (0.0007) [2023-12-27 16:11:39,271][20347] Updated weights for policy 0, policy_version 1522394 (0.0008) [2023-12-27 16:11:39,299][20381] Updated weights for policy 1, policy_version 1520873 (0.0007) [2023-12-27 16:11:39,327][20347] Updated weights for policy 0, policy_version 1522404 (0.0008) [2023-12-27 16:11:39,397][20347] Updated weights for policy 0, policy_version 1522414 (0.0011) [2023-12-27 16:11:39,459][20347] Updated weights for policy 0, policy_version 1522424 (0.0010) [2023-12-27 16:11:40,040][20381] Updated weights for policy 1, policy_version 1520883 (0.0005) [2023-12-27 16:11:40,097][20381] Updated weights for policy 1, policy_version 1520893 (0.0006) [2023-12-27 16:11:40,159][20381] Updated weights for policy 1, policy_version 1520903 (0.0007) [2023-12-27 16:11:40,186][20347] Updated weights for policy 0, policy_version 1522434 (0.0008) [2023-12-27 16:11:40,247][20347] Updated weights for policy 0, policy_version 1522444 (0.0007) [2023-12-27 16:11:40,311][20347] Updated weights for policy 0, policy_version 1522454 (0.0009) [2023-12-27 16:11:40,807][20381] Updated weights for policy 1, policy_version 1520913 (0.0007) [2023-12-27 16:11:40,871][20381] Updated weights for policy 1, policy_version 1520923 (0.0007) [2023-12-27 16:11:40,927][20381] Updated weights for policy 1, policy_version 1520933 (0.0008) [2023-12-27 16:11:40,960][20347] Updated weights for policy 0, policy_version 1522464 (0.0007) [2023-12-27 16:11:40,984][20381] Updated weights for policy 1, policy_version 1520943 (0.0007) [2023-12-27 16:11:41,015][20347] Updated weights for policy 0, policy_version 1522474 (0.0010) [2023-12-27 16:11:41,082][20347] Updated weights for policy 0, policy_version 1522484 (0.0009) [2023-12-27 16:11:41,792][20381] Updated weights for policy 1, policy_version 1520953 (0.0008) [2023-12-27 16:11:41,795][20347] Updated weights for policy 0, policy_version 1522494 (0.0008) [2023-12-27 16:11:41,850][20347] Updated weights for policy 0, policy_version 1522504 (0.0007) [2023-12-27 16:11:41,856][20381] Updated weights for policy 1, policy_version 1520963 (0.0009) [2023-12-27 16:11:41,909][20347] Updated weights for policy 0, policy_version 1522514 (0.0007) [2023-12-27 16:11:41,914][20381] Updated weights for policy 1, policy_version 1520973 (0.0007) [2023-12-27 16:11:42,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 779247616. Throughput: 0: 10174.7, 1: 10390.9. Samples: 779239908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:42,363][19170] Avg episode reward: [(0, '970.760'), (1, '789.510')] [2023-12-27 16:11:42,590][20381] Updated weights for policy 1, policy_version 1520983 (0.0007) [2023-12-27 16:11:42,652][20347] Updated weights for policy 0, policy_version 1522524 (0.0009) [2023-12-27 16:11:42,655][20381] Updated weights for policy 1, policy_version 1520993 (0.0007) [2023-12-27 16:11:42,718][20347] Updated weights for policy 0, policy_version 1522534 (0.0011) [2023-12-27 16:11:42,722][20381] Updated weights for policy 1, policy_version 1521003 (0.0007) [2023-12-27 16:11:42,779][20347] Updated weights for policy 0, policy_version 1522544 (0.0010) [2023-12-27 16:11:42,826][20291] KL-divergence is very high: 676.3860 [2023-12-27 16:11:43,373][20347] Updated weights for policy 0, policy_version 1522554 (0.0009) [2023-12-27 16:11:43,408][20381] Updated weights for policy 1, policy_version 1521013 (0.0007) [2023-12-27 16:11:43,422][20347] Updated weights for policy 0, policy_version 1522564 (0.0006) [2023-12-27 16:11:43,468][20381] Updated weights for policy 1, policy_version 1521023 (0.0007) [2023-12-27 16:11:43,470][20347] Updated weights for policy 0, policy_version 1522574 (0.0006) [2023-12-27 16:11:43,520][20347] Updated weights for policy 0, policy_version 1522584 (0.0007) [2023-12-27 16:11:43,520][20381] Updated weights for policy 1, policy_version 1521033 (0.0009) [2023-12-27 16:11:44,198][20347] Updated weights for policy 0, policy_version 1522594 (0.0007) [2023-12-27 16:11:44,204][20381] Updated weights for policy 1, policy_version 1521043 (0.0007) [2023-12-27 16:11:44,252][20381] Updated weights for policy 1, policy_version 1521053 (0.0006) [2023-12-27 16:11:44,253][20347] Updated weights for policy 0, policy_version 1522604 (0.0007) [2023-12-27 16:11:44,308][20381] Updated weights for policy 1, policy_version 1521063 (0.0005) [2023-12-27 16:11:44,309][20347] Updated weights for policy 0, policy_version 1522614 (0.0008) [2023-12-27 16:11:44,981][20381] Updated weights for policy 1, policy_version 1521073 (0.0006) [2023-12-27 16:11:45,011][20347] Updated weights for policy 0, policy_version 1522624 (0.0008) [2023-12-27 16:11:45,040][20381] Updated weights for policy 1, policy_version 1521083 (0.0006) [2023-12-27 16:11:45,072][20347] Updated weights for policy 0, policy_version 1522634 (0.0008) [2023-12-27 16:11:45,095][20381] Updated weights for policy 1, policy_version 1521093 (0.0007) [2023-12-27 16:11:45,136][20347] Updated weights for policy 0, policy_version 1522644 (0.0008) [2023-12-27 16:11:45,152][20381] Updated weights for policy 1, policy_version 1521103 (0.0007) [2023-12-27 16:11:45,807][20381] Updated weights for policy 1, policy_version 1521113 (0.0006) [2023-12-27 16:11:45,816][20347] Updated weights for policy 0, policy_version 1522654 (0.0007) [2023-12-27 16:11:45,862][20381] Updated weights for policy 1, policy_version 1521123 (0.0007) [2023-12-27 16:11:45,865][20347] Updated weights for policy 0, policy_version 1522664 (0.0005) [2023-12-27 16:11:45,912][20347] Updated weights for policy 0, policy_version 1522674 (0.0010) [2023-12-27 16:11:45,915][20381] Updated weights for policy 1, policy_version 1521133 (0.0007) [2023-12-27 16:11:46,552][20381] Updated weights for policy 1, policy_version 1521143 (0.0009) [2023-12-27 16:11:46,594][20347] Updated weights for policy 0, policy_version 1522684 (0.0009) [2023-12-27 16:11:46,606][20381] Updated weights for policy 1, policy_version 1521153 (0.0008) [2023-12-27 16:11:46,651][20347] Updated weights for policy 0, policy_version 1522694 (0.0005) [2023-12-27 16:11:46,668][20381] Updated weights for policy 1, policy_version 1521163 (0.0008) [2023-12-27 16:11:46,708][20347] Updated weights for policy 0, policy_version 1522704 (0.0005) [2023-12-27 16:11:47,258][20347] Updated weights for policy 0, policy_version 1522714 (0.0006) [2023-12-27 16:11:47,315][20347] Updated weights for policy 0, policy_version 1522724 (0.0007) [2023-12-27 16:11:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 779345920. Throughput: 0: 10155.4, 1: 10363.7. Samples: 779362360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:47,363][19170] Avg episode reward: [(0, '980.520'), (1, '794.400')] [2023-12-27 16:11:47,370][20347] Updated weights for policy 0, policy_version 1522734 (0.0005) [2023-12-27 16:11:47,395][20381] Updated weights for policy 1, policy_version 1521173 (0.0008) [2023-12-27 16:11:47,427][20347] Updated weights for policy 0, policy_version 1522744 (0.0005) [2023-12-27 16:11:47,457][20381] Updated weights for policy 1, policy_version 1521183 (0.0008) [2023-12-27 16:11:47,489][20292] KL-divergence is very high: 120.9482 [2023-12-27 16:11:47,518][20381] Updated weights for policy 1, policy_version 1521193 (0.0007) [2023-12-27 16:11:47,538][20292] KL-divergence is very high: 129.0032 [2023-12-27 16:11:48,025][20347] Updated weights for policy 0, policy_version 1522754 (0.0010) [2023-12-27 16:11:48,089][20347] Updated weights for policy 0, policy_version 1522764 (0.0010) [2023-12-27 16:11:48,146][20347] Updated weights for policy 0, policy_version 1522774 (0.0009) [2023-12-27 16:11:48,245][20381] Updated weights for policy 1, policy_version 1521203 (0.0008) [2023-12-27 16:11:48,310][20381] Updated weights for policy 1, policy_version 1521213 (0.0009) [2023-12-27 16:11:48,377][20381] Updated weights for policy 1, policy_version 1521223 (0.0007) [2023-12-27 16:11:48,810][20347] Updated weights for policy 0, policy_version 1522784 (0.0009) [2023-12-27 16:11:48,869][20347] Updated weights for policy 0, policy_version 1522794 (0.0010) [2023-12-27 16:11:48,932][20347] Updated weights for policy 0, policy_version 1522804 (0.0010) [2023-12-27 16:11:49,014][20381] Updated weights for policy 1, policy_version 1521233 (0.0007) [2023-12-27 16:11:49,065][20381] Updated weights for policy 1, policy_version 1521243 (0.0005) [2023-12-27 16:11:49,120][20381] Updated weights for policy 1, policy_version 1521253 (0.0005) [2023-12-27 16:11:49,174][20381] Updated weights for policy 1, policy_version 1521263 (0.0008) [2023-12-27 16:11:49,527][20347] Updated weights for policy 0, policy_version 1522814 (0.0010) [2023-12-27 16:11:49,589][20347] Updated weights for policy 0, policy_version 1522824 (0.0011) [2023-12-27 16:11:49,644][20347] Updated weights for policy 0, policy_version 1522834 (0.0010) [2023-12-27 16:11:49,812][20381] Updated weights for policy 1, policy_version 1521273 (0.0008) [2023-12-27 16:11:49,873][20381] Updated weights for policy 1, policy_version 1521283 (0.0009) [2023-12-27 16:11:49,933][20381] Updated weights for policy 1, policy_version 1521293 (0.0008) [2023-12-27 16:11:50,328][20347] Updated weights for policy 0, policy_version 1522844 (0.0009) [2023-12-27 16:11:50,391][20347] Updated weights for policy 0, policy_version 1522854 (0.0008) [2023-12-27 16:11:50,448][20347] Updated weights for policy 0, policy_version 1522864 (0.0008) [2023-12-27 16:11:50,702][20381] Updated weights for policy 1, policy_version 1521305 (0.0008) [2023-12-27 16:11:50,767][20381] Updated weights for policy 1, policy_version 1521315 (0.0009) [2023-12-27 16:11:50,825][20381] Updated weights for policy 1, policy_version 1521325 (0.0009) [2023-12-27 16:11:51,117][20347] Updated weights for policy 0, policy_version 1522874 (0.0009) [2023-12-27 16:11:51,188][20347] Updated weights for policy 0, policy_version 1522884 (0.0006) [2023-12-27 16:11:51,253][20347] Updated weights for policy 0, policy_version 1522894 (0.0006) [2023-12-27 16:11:51,317][20347] Updated weights for policy 0, policy_version 1522904 (0.0006) [2023-12-27 16:11:51,575][20381] Updated weights for policy 1, policy_version 1521335 (0.0007) [2023-12-27 16:11:51,637][20381] Updated weights for policy 1, policy_version 1521345 (0.0008) [2023-12-27 16:11:51,696][20381] Updated weights for policy 1, policy_version 1521355 (0.0007) [2023-12-27 16:11:51,991][20347] Updated weights for policy 0, policy_version 1522914 (0.0005) [2023-12-27 16:11:52,043][20347] Updated weights for policy 0, policy_version 1522924 (0.0006) [2023-12-27 16:11:52,102][20347] Updated weights for policy 0, policy_version 1522934 (0.0006) [2023-12-27 16:11:52,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 779452416. Throughput: 0: 10186.4, 1: 10323.5. Samples: 779424028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:52,363][19170] Avg episode reward: [(0, '990.280'), (1, '779.230')] [2023-12-27 16:11:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001522936_389947392.pth... [2023-12-27 16:11:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001521720_389636096.pth [2023-12-27 16:11:52,395][20381] Updated weights for policy 1, policy_version 1521365 (0.0007) [2023-12-27 16:11:52,445][20381] Updated weights for policy 1, policy_version 1521375 (0.0010) [2023-12-27 16:11:52,494][20381] Updated weights for policy 1, policy_version 1521385 (0.0008) [2023-12-27 16:11:52,531][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001521392_389513216.pth... [2023-12-27 16:11:52,535][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001520176_389201920.pth [2023-12-27 16:11:52,709][20347] Updated weights for policy 0, policy_version 1522944 (0.0008) [2023-12-27 16:11:52,772][20347] Updated weights for policy 0, policy_version 1522954 (0.0006) [2023-12-27 16:11:52,832][20347] Updated weights for policy 0, policy_version 1522964 (0.0008) [2023-12-27 16:11:53,230][20381] Updated weights for policy 1, policy_version 1521395 (0.0008) [2023-12-27 16:11:53,279][20381] Updated weights for policy 1, policy_version 1521405 (0.0007) [2023-12-27 16:11:53,327][20381] Updated weights for policy 1, policy_version 1521415 (0.0007) [2023-12-27 16:11:53,457][20347] Updated weights for policy 0, policy_version 1522974 (0.0009) [2023-12-27 16:11:53,511][20347] Updated weights for policy 0, policy_version 1522984 (0.0010) [2023-12-27 16:11:53,565][20347] Updated weights for policy 0, policy_version 1522994 (0.0010) [2023-12-27 16:11:54,063][20381] Updated weights for policy 1, policy_version 1521425 (0.0007) [2023-12-27 16:11:54,120][20381] Updated weights for policy 1, policy_version 1521435 (0.0005) [2023-12-27 16:11:54,167][20381] Updated weights for policy 1, policy_version 1521445 (0.0005) [2023-12-27 16:11:54,175][20347] Updated weights for policy 0, policy_version 1523004 (0.0007) [2023-12-27 16:11:54,219][20381] Updated weights for policy 1, policy_version 1521455 (0.0005) [2023-12-27 16:11:54,231][20347] Updated weights for policy 0, policy_version 1523014 (0.0010) [2023-12-27 16:11:54,276][20347] Updated weights for policy 0, policy_version 1523024 (0.0010) [2023-12-27 16:11:54,842][20381] Updated weights for policy 1, policy_version 1521465 (0.0007) [2023-12-27 16:11:54,895][20381] Updated weights for policy 1, policy_version 1521475 (0.0007) [2023-12-27 16:11:54,916][20347] Updated weights for policy 0, policy_version 1523034 (0.0010) [2023-12-27 16:11:54,950][20381] Updated weights for policy 1, policy_version 1521485 (0.0006) [2023-12-27 16:11:54,971][20347] Updated weights for policy 0, policy_version 1523044 (0.0010) [2023-12-27 16:11:55,029][20347] Updated weights for policy 0, policy_version 1523054 (0.0010) [2023-12-27 16:11:55,087][20347] Updated weights for policy 0, policy_version 1523064 (0.0010) [2023-12-27 16:11:55,629][20381] Updated weights for policy 1, policy_version 1521495 (0.0009) [2023-12-27 16:11:55,687][20381] Updated weights for policy 1, policy_version 1521505 (0.0011) [2023-12-27 16:11:55,738][20381] Updated weights for policy 1, policy_version 1521515 (0.0005) [2023-12-27 16:11:55,743][20347] Updated weights for policy 0, policy_version 1523074 (0.0010) [2023-12-27 16:11:55,799][20347] Updated weights for policy 0, policy_version 1523084 (0.0010) [2023-12-27 16:11:55,852][20347] Updated weights for policy 0, policy_version 1523094 (0.0009) [2023-12-27 16:11:56,374][20381] Updated weights for policy 1, policy_version 1521525 (0.0006) [2023-12-27 16:11:56,429][20381] Updated weights for policy 1, policy_version 1521535 (0.0006) [2023-12-27 16:11:56,442][20347] Updated weights for policy 0, policy_version 1523104 (0.0006) [2023-12-27 16:11:56,477][20381] Updated weights for policy 1, policy_version 1521545 (0.0008) [2023-12-27 16:11:56,495][20347] Updated weights for policy 0, policy_version 1523114 (0.0005) [2023-12-27 16:11:56,553][20347] Updated weights for policy 0, policy_version 1523124 (0.0006) [2023-12-27 16:11:57,089][20347] Updated weights for policy 0, policy_version 1523134 (0.0006) [2023-12-27 16:11:57,146][20347] Updated weights for policy 0, policy_version 1523144 (0.0008) [2023-12-27 16:11:57,166][20381] Updated weights for policy 1, policy_version 1521555 (0.0008) [2023-12-27 16:11:57,200][20347] Updated weights for policy 0, policy_version 1523154 (0.0010) [2023-12-27 16:11:57,224][20381] Updated weights for policy 1, policy_version 1521565 (0.0011) [2023-12-27 16:11:57,268][20381] Updated weights for policy 1, policy_version 1521575 (0.0010) [2023-12-27 16:11:57,362][19170] Fps is (10 sec: 22118.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 779567104. Throughput: 0: 10224.7, 1: 10265.6. Samples: 779548148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:11:57,363][19170] Avg episode reward: [(0, '990.280'), (1, '805.220')] [2023-12-27 16:11:57,812][20347] Updated weights for policy 0, policy_version 1523164 (0.0010) [2023-12-27 16:11:57,857][20347] Updated weights for policy 0, policy_version 1523174 (0.0010) [2023-12-27 16:11:57,877][20381] Updated weights for policy 1, policy_version 1521585 (0.0010) [2023-12-27 16:11:57,906][20347] Updated weights for policy 0, policy_version 1523184 (0.0010) [2023-12-27 16:11:57,930][20381] Updated weights for policy 1, policy_version 1521595 (0.0006) [2023-12-27 16:11:57,982][20381] Updated weights for policy 1, policy_version 1521605 (0.0007) [2023-12-27 16:11:58,037][20381] Updated weights for policy 1, policy_version 1521615 (0.0007) [2023-12-27 16:11:58,657][20347] Updated weights for policy 0, policy_version 1523194 (0.0010) [2023-12-27 16:11:58,716][20347] Updated weights for policy 0, policy_version 1523204 (0.0009) [2023-12-27 16:11:58,733][20381] Updated weights for policy 1, policy_version 1521625 (0.0009) [2023-12-27 16:11:58,778][20347] Updated weights for policy 0, policy_version 1523214 (0.0008) [2023-12-27 16:11:58,797][20381] Updated weights for policy 1, policy_version 1521635 (0.0009) [2023-12-27 16:11:58,840][20347] Updated weights for policy 0, policy_version 1523224 (0.0008) [2023-12-27 16:11:58,864][20381] Updated weights for policy 1, policy_version 1521645 (0.0010) [2023-12-27 16:11:59,600][20347] Updated weights for policy 0, policy_version 1523234 (0.0006) [2023-12-27 16:11:59,613][20381] Updated weights for policy 1, policy_version 1521655 (0.0008) [2023-12-27 16:11:59,651][20347] Updated weights for policy 0, policy_version 1523244 (0.0009) [2023-12-27 16:11:59,669][20381] Updated weights for policy 1, policy_version 1521665 (0.0008) [2023-12-27 16:11:59,699][20347] Updated weights for policy 0, policy_version 1523254 (0.0005) [2023-12-27 16:11:59,728][20381] Updated weights for policy 1, policy_version 1521675 (0.0010) [2023-12-27 16:12:00,361][20347] Updated weights for policy 0, policy_version 1523264 (0.0006) [2023-12-27 16:12:00,408][20381] Updated weights for policy 1, policy_version 1521685 (0.0007) [2023-12-27 16:12:00,418][20347] Updated weights for policy 0, policy_version 1523274 (0.0007) [2023-12-27 16:12:00,462][20381] Updated weights for policy 1, policy_version 1521695 (0.0006) [2023-12-27 16:12:00,475][20347] Updated weights for policy 0, policy_version 1523284 (0.0005) [2023-12-27 16:12:00,511][20381] Updated weights for policy 1, policy_version 1521705 (0.0005) [2023-12-27 16:12:01,070][20347] Updated weights for policy 0, policy_version 1523294 (0.0008) [2023-12-27 16:12:01,091][20381] Updated weights for policy 1, policy_version 1521715 (0.0005) [2023-12-27 16:12:01,133][20347] Updated weights for policy 0, policy_version 1523304 (0.0007) [2023-12-27 16:12:01,154][20381] Updated weights for policy 1, policy_version 1521725 (0.0007) [2023-12-27 16:12:01,190][20347] Updated weights for policy 0, policy_version 1523314 (0.0006) [2023-12-27 16:12:01,216][20381] Updated weights for policy 1, policy_version 1521735 (0.0007) [2023-12-27 16:12:01,864][20381] Updated weights for policy 1, policy_version 1521745 (0.0007) [2023-12-27 16:12:01,912][20347] Updated weights for policy 0, policy_version 1523324 (0.0006) [2023-12-27 16:12:01,925][20381] Updated weights for policy 1, policy_version 1521755 (0.0007) [2023-12-27 16:12:01,970][20347] Updated weights for policy 0, policy_version 1523334 (0.0006) [2023-12-27 16:12:01,989][20381] Updated weights for policy 1, policy_version 1521765 (0.0008) [2023-12-27 16:12:02,029][20347] Updated weights for policy 0, policy_version 1523344 (0.0006) [2023-12-27 16:12:02,050][20381] Updated weights for policy 1, policy_version 1521775 (0.0006) [2023-12-27 16:12:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 779665408. Throughput: 0: 10331.4, 1: 10252.2. Samples: 779673804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:12:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '814.770')] [2023-12-27 16:12:02,656][20347] Updated weights for policy 0, policy_version 1523354 (0.0006) [2023-12-27 16:12:02,720][20347] Updated weights for policy 0, policy_version 1523364 (0.0006) [2023-12-27 16:12:02,744][20381] Updated weights for policy 1, policy_version 1521785 (0.0006) [2023-12-27 16:12:02,780][20347] Updated weights for policy 0, policy_version 1523374 (0.0006) [2023-12-27 16:12:02,798][20381] Updated weights for policy 1, policy_version 1521795 (0.0006) [2023-12-27 16:12:02,833][20347] Updated weights for policy 0, policy_version 1523384 (0.0005) [2023-12-27 16:12:02,853][20381] Updated weights for policy 1, policy_version 1521805 (0.0008) [2023-12-27 16:12:03,397][20347] Updated weights for policy 0, policy_version 1523394 (0.0008) [2023-12-27 16:12:03,446][20347] Updated weights for policy 0, policy_version 1523404 (0.0007) [2023-12-27 16:12:03,501][20347] Updated weights for policy 0, policy_version 1523414 (0.0008) [2023-12-27 16:12:03,555][20381] Updated weights for policy 1, policy_version 1521815 (0.0008) [2023-12-27 16:12:03,610][20381] Updated weights for policy 1, policy_version 1521825 (0.0007) [2023-12-27 16:12:03,665][20381] Updated weights for policy 1, policy_version 1521835 (0.0008) [2023-12-27 16:12:04,226][20347] Updated weights for policy 0, policy_version 1523424 (0.0007) [2023-12-27 16:12:04,293][20347] Updated weights for policy 0, policy_version 1523434 (0.0007) [2023-12-27 16:12:04,319][20381] Updated weights for policy 1, policy_version 1521845 (0.0007) [2023-12-27 16:12:04,352][20347] Updated weights for policy 0, policy_version 1523444 (0.0009) [2023-12-27 16:12:04,373][20381] Updated weights for policy 1, policy_version 1521855 (0.0007) [2023-12-27 16:12:04,430][20381] Updated weights for policy 1, policy_version 1521865 (0.0007) [2023-12-27 16:12:05,007][20381] Updated weights for policy 1, policy_version 1521875 (0.0007) [2023-12-27 16:12:05,057][20347] Updated weights for policy 0, policy_version 1523454 (0.0007) [2023-12-27 16:12:05,061][20381] Updated weights for policy 1, policy_version 1521885 (0.0007) [2023-12-27 16:12:05,107][20347] Updated weights for policy 0, policy_version 1523464 (0.0006) [2023-12-27 16:12:05,110][20381] Updated weights for policy 1, policy_version 1521895 (0.0006) [2023-12-27 16:12:05,170][20347] Updated weights for policy 0, policy_version 1523474 (0.0006) [2023-12-27 16:12:05,781][20347] Updated weights for policy 0, policy_version 1523484 (0.0006) [2023-12-27 16:12:05,833][20347] Updated weights for policy 0, policy_version 1523494 (0.0005) [2023-12-27 16:12:05,833][20381] Updated weights for policy 1, policy_version 1521905 (0.0007) [2023-12-27 16:12:05,887][20347] Updated weights for policy 0, policy_version 1523504 (0.0007) [2023-12-27 16:12:05,892][20381] Updated weights for policy 1, policy_version 1521915 (0.0006) [2023-12-27 16:12:05,948][20381] Updated weights for policy 1, policy_version 1521925 (0.0006) [2023-12-27 16:12:06,012][20381] Updated weights for policy 1, policy_version 1521935 (0.0008) [2023-12-27 16:12:06,587][20347] Updated weights for policy 0, policy_version 1523514 (0.0006) [2023-12-27 16:12:06,642][20347] Updated weights for policy 0, policy_version 1523524 (0.0007) [2023-12-27 16:12:06,691][20347] Updated weights for policy 0, policy_version 1523534 (0.0007) [2023-12-27 16:12:06,697][20381] Updated weights for policy 1, policy_version 1521945 (0.0005) [2023-12-27 16:12:06,749][20347] Updated weights for policy 0, policy_version 1523544 (0.0007) [2023-12-27 16:12:06,761][20381] Updated weights for policy 1, policy_version 1521955 (0.0006) [2023-12-27 16:12:06,826][20381] Updated weights for policy 1, policy_version 1521965 (0.0008) [2023-12-27 16:12:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 779763712. Throughput: 0: 10346.0, 1: 10290.0. Samples: 779736548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:12:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '795.390')] [2023-12-27 16:12:07,384][20347] Updated weights for policy 0, policy_version 1523554 (0.0006) [2023-12-27 16:12:07,407][20381] Updated weights for policy 1, policy_version 1521975 (0.0006) [2023-12-27 16:12:07,431][20347] Updated weights for policy 0, policy_version 1523564 (0.0007) [2023-12-27 16:12:07,457][20381] Updated weights for policy 1, policy_version 1521985 (0.0005) [2023-12-27 16:12:07,479][20347] Updated weights for policy 0, policy_version 1523574 (0.0007) [2023-12-27 16:12:07,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001523576_390111232.pth... [2023-12-27 16:12:07,493][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001522328_389791744.pth [2023-12-27 16:12:07,505][20381] Updated weights for policy 1, policy_version 1521995 (0.0007) [2023-12-27 16:12:07,528][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001522000_389668864.pth... [2023-12-27 16:12:07,531][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001520784_389357568.pth [2023-12-27 16:12:08,152][20381] Updated weights for policy 1, policy_version 1522005 (0.0008) [2023-12-27 16:12:08,182][20347] Updated weights for policy 0, policy_version 1523584 (0.0006) [2023-12-27 16:12:08,203][20381] Updated weights for policy 1, policy_version 1522015 (0.0006) [2023-12-27 16:12:08,244][20347] Updated weights for policy 0, policy_version 1523594 (0.0006) [2023-12-27 16:12:08,260][20381] Updated weights for policy 1, policy_version 1522025 (0.0007) [2023-12-27 16:12:08,306][20347] Updated weights for policy 0, policy_version 1523604 (0.0007) [2023-12-27 16:12:08,936][20347] Updated weights for policy 0, policy_version 1523614 (0.0007) [2023-12-27 16:12:08,988][20347] Updated weights for policy 0, policy_version 1523624 (0.0008) [2023-12-27 16:12:09,010][20381] Updated weights for policy 1, policy_version 1522035 (0.0007) [2023-12-27 16:12:09,037][20347] Updated weights for policy 0, policy_version 1523634 (0.0008) [2023-12-27 16:12:09,065][20381] Updated weights for policy 1, policy_version 1522045 (0.0006) [2023-12-27 16:12:09,126][20381] Updated weights for policy 1, policy_version 1522055 (0.0007) [2023-12-27 16:12:09,786][20347] Updated weights for policy 0, policy_version 1523644 (0.0005) [2023-12-27 16:12:09,850][20347] Updated weights for policy 0, policy_version 1523654 (0.0008) [2023-12-27 16:12:09,854][20381] Updated weights for policy 1, policy_version 1522065 (0.0007) [2023-12-27 16:12:09,911][20347] Updated weights for policy 0, policy_version 1523664 (0.0007) [2023-12-27 16:12:09,917][20381] Updated weights for policy 1, policy_version 1522075 (0.0010) [2023-12-27 16:12:09,982][20381] Updated weights for policy 1, policy_version 1522085 (0.0011) [2023-12-27 16:12:10,040][20381] Updated weights for policy 1, policy_version 1522095 (0.0008) [2023-12-27 16:12:10,656][20347] Updated weights for policy 0, policy_version 1523674 (0.0007) [2023-12-27 16:12:10,722][20347] Updated weights for policy 0, policy_version 1523684 (0.0005) [2023-12-27 16:12:10,780][20347] Updated weights for policy 0, policy_version 1523694 (0.0006) [2023-12-27 16:12:10,799][20381] Updated weights for policy 1, policy_version 1522105 (0.0007) [2023-12-27 16:12:10,839][20347] Updated weights for policy 0, policy_version 1523704 (0.0006) [2023-12-27 16:12:10,855][20381] Updated weights for policy 1, policy_version 1522115 (0.0010) [2023-12-27 16:12:10,912][20381] Updated weights for policy 1, policy_version 1522125 (0.0010) [2023-12-27 16:12:11,548][20347] Updated weights for policy 0, policy_version 1523714 (0.0010) [2023-12-27 16:12:11,609][20347] Updated weights for policy 0, policy_version 1523724 (0.0011) [2023-12-27 16:12:11,646][20381] Updated weights for policy 1, policy_version 1522135 (0.0008) [2023-12-27 16:12:11,671][20347] Updated weights for policy 0, policy_version 1523734 (0.0008) [2023-12-27 16:12:11,720][20381] Updated weights for policy 1, policy_version 1522145 (0.0008) [2023-12-27 16:12:11,792][20381] Updated weights for policy 1, policy_version 1522155 (0.0009) [2023-12-27 16:12:12,338][20347] Updated weights for policy 0, policy_version 1523744 (0.0007) [2023-12-27 16:12:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 779862016. Throughput: 0: 10319.6, 1: 10270.8. Samples: 779857980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:12:12,363][19170] Avg episode reward: [(0, '970.860'), (1, '756.580')] [2023-12-27 16:12:12,404][20347] Updated weights for policy 0, policy_version 1523754 (0.0009) [2023-12-27 16:12:12,462][20347] Updated weights for policy 0, policy_version 1523764 (0.0007) [2023-12-27 16:12:12,467][20381] Updated weights for policy 1, policy_version 1522165 (0.0006) [2023-12-27 16:12:12,525][20381] Updated weights for policy 1, policy_version 1522175 (0.0005) [2023-12-27 16:12:12,589][20381] Updated weights for policy 1, policy_version 1522185 (0.0005) [2023-12-27 16:12:13,171][20347] Updated weights for policy 0, policy_version 1523774 (0.0009) [2023-12-27 16:12:13,213][20381] Updated weights for policy 1, policy_version 1522195 (0.0007) [2023-12-27 16:12:13,226][20347] Updated weights for policy 0, policy_version 1523784 (0.0010) [2023-12-27 16:12:13,265][20381] Updated weights for policy 1, policy_version 1522205 (0.0011) [2023-12-27 16:12:13,283][20347] Updated weights for policy 0, policy_version 1523794 (0.0008) [2023-12-27 16:12:13,311][20381] Updated weights for policy 1, policy_version 1522215 (0.0010) [2023-12-27 16:12:13,865][20347] Updated weights for policy 0, policy_version 1523804 (0.0006) [2023-12-27 16:12:13,910][20347] Updated weights for policy 0, policy_version 1523814 (0.0005) [2023-12-27 16:12:13,956][20347] Updated weights for policy 0, policy_version 1523824 (0.0005) [2023-12-27 16:12:13,992][20381] Updated weights for policy 1, policy_version 1522225 (0.0010) [2023-12-27 16:12:14,058][20381] Updated weights for policy 1, policy_version 1522235 (0.0005) [2023-12-27 16:12:14,118][20381] Updated weights for policy 1, policy_version 1522245 (0.0005) [2023-12-27 16:12:14,171][20381] Updated weights for policy 1, policy_version 1522255 (0.0005) [2023-12-27 16:12:14,565][20347] Updated weights for policy 0, policy_version 1523834 (0.0005) [2023-12-27 16:12:14,615][20347] Updated weights for policy 0, policy_version 1523844 (0.0005) [2023-12-27 16:12:14,669][20347] Updated weights for policy 0, policy_version 1523854 (0.0005) [2023-12-27 16:12:14,691][20381] Updated weights for policy 1, policy_version 1522265 (0.0007) [2023-12-27 16:12:14,725][20347] Updated weights for policy 0, policy_version 1523864 (0.0006) [2023-12-27 16:12:14,754][20381] Updated weights for policy 1, policy_version 1522275 (0.0007) [2023-12-27 16:12:14,827][20381] Updated weights for policy 1, policy_version 1522285 (0.0008) [2023-12-27 16:12:15,400][20347] Updated weights for policy 0, policy_version 1523874 (0.0008) [2023-12-27 16:12:15,455][20347] Updated weights for policy 0, policy_version 1523884 (0.0007) [2023-12-27 16:12:15,495][20381] Updated weights for policy 1, policy_version 1522295 (0.0007) [2023-12-27 16:12:15,512][20347] Updated weights for policy 0, policy_version 1523894 (0.0006) [2023-12-27 16:12:15,556][20381] Updated weights for policy 1, policy_version 1522305 (0.0007) [2023-12-27 16:12:15,606][20381] Updated weights for policy 1, policy_version 1522315 (0.0009) [2023-12-27 16:12:16,118][20347] Updated weights for policy 0, policy_version 1523904 (0.0008) [2023-12-27 16:12:16,166][20347] Updated weights for policy 0, policy_version 1523914 (0.0010) [2023-12-27 16:12:16,224][20347] Updated weights for policy 0, policy_version 1523924 (0.0010) [2023-12-27 16:12:16,242][20381] Updated weights for policy 1, policy_version 1522325 (0.0007) [2023-12-27 16:12:16,298][20381] Updated weights for policy 1, policy_version 1522335 (0.0007) [2023-12-27 16:12:16,357][20381] Updated weights for policy 1, policy_version 1522345 (0.0008) [2023-12-27 16:12:16,820][20347] Updated weights for policy 0, policy_version 1523934 (0.0010) [2023-12-27 16:12:16,873][20347] Updated weights for policy 0, policy_version 1523944 (0.0010) [2023-12-27 16:12:16,928][20347] Updated weights for policy 0, policy_version 1523954 (0.0010) [2023-12-27 16:12:16,964][20381] Updated weights for policy 1, policy_version 1522355 (0.0007) [2023-12-27 16:12:17,010][20381] Updated weights for policy 1, policy_version 1522365 (0.0007) [2023-12-27 16:12:17,058][20381] Updated weights for policy 1, policy_version 1522375 (0.0006) [2023-12-27 16:12:17,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 779976704. Throughput: 0: 10342.8, 1: 10298.0. Samples: 779984900. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:17,364][19170] Avg episode reward: [(0, '980.570'), (1, '824.450')] [2023-12-27 16:12:17,584][20347] Updated weights for policy 0, policy_version 1523964 (0.0009) [2023-12-27 16:12:17,636][20347] Updated weights for policy 0, policy_version 1523974 (0.0010) [2023-12-27 16:12:17,683][20347] Updated weights for policy 0, policy_version 1523984 (0.0007) [2023-12-27 16:12:17,733][20381] Updated weights for policy 1, policy_version 1522385 (0.0008) [2023-12-27 16:12:17,788][20381] Updated weights for policy 1, policy_version 1522395 (0.0010) [2023-12-27 16:12:17,837][20381] Updated weights for policy 1, policy_version 1522405 (0.0010) [2023-12-27 16:12:17,889][20381] Updated weights for policy 1, policy_version 1522415 (0.0010) [2023-12-27 16:12:18,400][20347] Updated weights for policy 0, policy_version 1523994 (0.0008) [2023-12-27 16:12:18,463][20347] Updated weights for policy 0, policy_version 1524004 (0.0008) [2023-12-27 16:12:18,523][20347] Updated weights for policy 0, policy_version 1524014 (0.0008) [2023-12-27 16:12:18,542][20381] Updated weights for policy 1, policy_version 1522425 (0.0007) [2023-12-27 16:12:18,586][20347] Updated weights for policy 0, policy_version 1524024 (0.0007) [2023-12-27 16:12:18,602][20381] Updated weights for policy 1, policy_version 1522435 (0.0008) [2023-12-27 16:12:18,656][20381] Updated weights for policy 1, policy_version 1522445 (0.0008) [2023-12-27 16:12:19,222][20347] Updated weights for policy 0, policy_version 1524034 (0.0010) [2023-12-27 16:12:19,285][20347] Updated weights for policy 0, policy_version 1524044 (0.0008) [2023-12-27 16:12:19,350][20347] Updated weights for policy 0, policy_version 1524054 (0.0009) [2023-12-27 16:12:19,399][20381] Updated weights for policy 1, policy_version 1522455 (0.0008) [2023-12-27 16:12:19,455][20381] Updated weights for policy 1, policy_version 1522465 (0.0007) [2023-12-27 16:12:19,523][20381] Updated weights for policy 1, policy_version 1522475 (0.0007) [2023-12-27 16:12:20,119][20347] Updated weights for policy 0, policy_version 1524064 (0.0008) [2023-12-27 16:12:20,178][20347] Updated weights for policy 0, policy_version 1524074 (0.0008) [2023-12-27 16:12:20,185][20381] Updated weights for policy 1, policy_version 1522485 (0.0006) [2023-12-27 16:12:20,229][20347] Updated weights for policy 0, policy_version 1524084 (0.0007) [2023-12-27 16:12:20,239][20381] Updated weights for policy 1, policy_version 1522495 (0.0007) [2023-12-27 16:12:20,297][20381] Updated weights for policy 1, policy_version 1522505 (0.0006) [2023-12-27 16:12:20,912][20347] Updated weights for policy 0, policy_version 1524094 (0.0007) [2023-12-27 16:12:20,979][20347] Updated weights for policy 0, policy_version 1524104 (0.0008) [2023-12-27 16:12:20,981][20381] Updated weights for policy 1, policy_version 1522515 (0.0008) [2023-12-27 16:12:21,032][20347] Updated weights for policy 0, policy_version 1524114 (0.0007) [2023-12-27 16:12:21,050][20381] Updated weights for policy 1, policy_version 1522525 (0.0010) [2023-12-27 16:12:21,111][20381] Updated weights for policy 1, policy_version 1522535 (0.0011) [2023-12-27 16:12:21,765][20347] Updated weights for policy 0, policy_version 1524124 (0.0008) [2023-12-27 16:12:21,816][20347] Updated weights for policy 0, policy_version 1524134 (0.0008) [2023-12-27 16:12:21,865][20347] Updated weights for policy 0, policy_version 1524144 (0.0007) [2023-12-27 16:12:21,908][20381] Updated weights for policy 1, policy_version 1522545 (0.0009) [2023-12-27 16:12:21,971][20381] Updated weights for policy 1, policy_version 1522555 (0.0011) [2023-12-27 16:12:22,028][20381] Updated weights for policy 1, policy_version 1522565 (0.0007) [2023-12-27 16:12:22,082][20381] Updated weights for policy 1, policy_version 1522575 (0.0008) [2023-12-27 16:12:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 780075008. Throughput: 0: 10354.4, 1: 10266.8. Samples: 780045920. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:22,363][19170] Avg episode reward: [(0, '980.590'), (1, '833.910')] [2023-12-27 16:12:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001522576_389816320.pth... [2023-12-27 16:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001524152_390258688.pth... [2023-12-27 16:12:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001521392_389513216.pth [2023-12-27 16:12:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001522936_389947392.pth [2023-12-27 16:12:22,622][20347] Updated weights for policy 0, policy_version 1524154 (0.0007) [2023-12-27 16:12:22,681][20347] Updated weights for policy 0, policy_version 1524164 (0.0008) [2023-12-27 16:12:22,740][20347] Updated weights for policy 0, policy_version 1524174 (0.0006) [2023-12-27 16:12:22,765][20381] Updated weights for policy 1, policy_version 1522585 (0.0007) [2023-12-27 16:12:22,797][20347] Updated weights for policy 0, policy_version 1524184 (0.0006) [2023-12-27 16:12:22,824][20381] Updated weights for policy 1, policy_version 1522595 (0.0006) [2023-12-27 16:12:22,889][20381] Updated weights for policy 1, policy_version 1522605 (0.0008) [2023-12-27 16:12:23,367][20347] Updated weights for policy 0, policy_version 1524194 (0.0008) [2023-12-27 16:12:23,422][20347] Updated weights for policy 0, policy_version 1524204 (0.0008) [2023-12-27 16:12:23,474][20347] Updated weights for policy 0, policy_version 1524214 (0.0007) [2023-12-27 16:12:23,606][20381] Updated weights for policy 1, policy_version 1522615 (0.0007) [2023-12-27 16:12:23,659][20381] Updated weights for policy 1, policy_version 1522625 (0.0005) [2023-12-27 16:12:23,716][20381] Updated weights for policy 1, policy_version 1522635 (0.0005) [2023-12-27 16:12:24,176][20347] Updated weights for policy 0, policy_version 1524224 (0.0007) [2023-12-27 16:12:24,228][20347] Updated weights for policy 0, policy_version 1524234 (0.0007) [2023-12-27 16:12:24,281][20347] Updated weights for policy 0, policy_version 1524244 (0.0007) [2023-12-27 16:12:24,344][20381] Updated weights for policy 1, policy_version 1522645 (0.0007) [2023-12-27 16:12:24,404][20381] Updated weights for policy 1, policy_version 1522655 (0.0007) [2023-12-27 16:12:24,455][20381] Updated weights for policy 1, policy_version 1522665 (0.0007) [2023-12-27 16:12:24,998][20347] Updated weights for policy 0, policy_version 1524254 (0.0008) [2023-12-27 16:12:25,059][20347] Updated weights for policy 0, policy_version 1524264 (0.0008) [2023-12-27 16:12:25,068][20381] Updated weights for policy 1, policy_version 1522675 (0.0007) [2023-12-27 16:12:25,115][20381] Updated weights for policy 1, policy_version 1522685 (0.0005) [2023-12-27 16:12:25,120][20347] Updated weights for policy 0, policy_version 1524274 (0.0007) [2023-12-27 16:12:25,171][20381] Updated weights for policy 1, policy_version 1522695 (0.0006) [2023-12-27 16:12:25,731][20381] Updated weights for policy 1, policy_version 1522705 (0.0008) [2023-12-27 16:12:25,788][20381] Updated weights for policy 1, policy_version 1522715 (0.0008) [2023-12-27 16:12:25,831][20347] Updated weights for policy 0, policy_version 1524284 (0.0007) [2023-12-27 16:12:25,844][20381] Updated weights for policy 1, policy_version 1522725 (0.0007) [2023-12-27 16:12:25,883][20347] Updated weights for policy 0, policy_version 1524294 (0.0006) [2023-12-27 16:12:25,899][20381] Updated weights for policy 1, policy_version 1522735 (0.0007) [2023-12-27 16:12:25,940][20347] Updated weights for policy 0, policy_version 1524304 (0.0008) [2023-12-27 16:12:26,564][20381] Updated weights for policy 1, policy_version 1522745 (0.0008) [2023-12-27 16:12:26,584][20347] Updated weights for policy 0, policy_version 1524314 (0.0007) [2023-12-27 16:12:26,622][20381] Updated weights for policy 1, policy_version 1522755 (0.0006) [2023-12-27 16:12:26,638][20347] Updated weights for policy 0, policy_version 1524324 (0.0007) [2023-12-27 16:12:26,682][20381] Updated weights for policy 1, policy_version 1522765 (0.0005) [2023-12-27 16:12:26,699][20347] Updated weights for policy 0, policy_version 1524334 (0.0008) [2023-12-27 16:12:26,750][20347] Updated weights for policy 0, policy_version 1524344 (0.0006) [2023-12-27 16:12:27,225][20381] Updated weights for policy 1, policy_version 1522775 (0.0006) [2023-12-27 16:12:27,281][20381] Updated weights for policy 1, policy_version 1522785 (0.0006) [2023-12-27 16:12:27,339][20381] Updated weights for policy 1, policy_version 1522795 (0.0006) [2023-12-27 16:12:27,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 780173312. Throughput: 0: 10394.4, 1: 10214.3. Samples: 780167296. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:27,363][19170] Avg episode reward: [(0, '970.880'), (1, '853.710')] [2023-12-27 16:12:27,467][20347] Updated weights for policy 0, policy_version 1524354 (0.0008) [2023-12-27 16:12:27,514][20347] Updated weights for policy 0, policy_version 1524364 (0.0007) [2023-12-27 16:12:27,571][20347] Updated weights for policy 0, policy_version 1524374 (0.0007) [2023-12-27 16:12:27,893][20381] Updated weights for policy 1, policy_version 1522805 (0.0005) [2023-12-27 16:12:27,952][20381] Updated weights for policy 1, policy_version 1522815 (0.0005) [2023-12-27 16:12:28,007][20381] Updated weights for policy 1, policy_version 1522825 (0.0007) [2023-12-27 16:12:28,283][20347] Updated weights for policy 0, policy_version 1524384 (0.0006) [2023-12-27 16:12:28,335][20347] Updated weights for policy 0, policy_version 1524394 (0.0006) [2023-12-27 16:12:28,403][20347] Updated weights for policy 0, policy_version 1524404 (0.0007) [2023-12-27 16:12:28,635][20381] Updated weights for policy 1, policy_version 1522835 (0.0008) [2023-12-27 16:12:28,687][20381] Updated weights for policy 1, policy_version 1522845 (0.0008) [2023-12-27 16:12:28,739][20381] Updated weights for policy 1, policy_version 1522855 (0.0008) [2023-12-27 16:12:29,113][20347] Updated weights for policy 0, policy_version 1524414 (0.0007) [2023-12-27 16:12:29,173][20347] Updated weights for policy 0, policy_version 1524424 (0.0009) [2023-12-27 16:12:29,236][20347] Updated weights for policy 0, policy_version 1524434 (0.0008) [2023-12-27 16:12:29,465][20381] Updated weights for policy 1, policy_version 1522865 (0.0007) [2023-12-27 16:12:29,525][20381] Updated weights for policy 1, policy_version 1522875 (0.0008) [2023-12-27 16:12:29,584][20381] Updated weights for policy 1, policy_version 1522885 (0.0008) [2023-12-27 16:12:29,640][20381] Updated weights for policy 1, policy_version 1522895 (0.0008) [2023-12-27 16:12:29,898][20347] Updated weights for policy 0, policy_version 1524444 (0.0007) [2023-12-27 16:12:29,962][20347] Updated weights for policy 0, policy_version 1524454 (0.0008) [2023-12-27 16:12:30,023][20347] Updated weights for policy 0, policy_version 1524464 (0.0007) [2023-12-27 16:12:30,368][20381] Updated weights for policy 1, policy_version 1522905 (0.0007) [2023-12-27 16:12:30,417][20381] Updated weights for policy 1, policy_version 1522915 (0.0005) [2023-12-27 16:12:30,469][20381] Updated weights for policy 1, policy_version 1522925 (0.0005) [2023-12-27 16:12:30,736][20347] Updated weights for policy 0, policy_version 1524474 (0.0007) [2023-12-27 16:12:30,783][20347] Updated weights for policy 0, policy_version 1524484 (0.0007) [2023-12-27 16:12:30,831][20347] Updated weights for policy 0, policy_version 1524494 (0.0007) [2023-12-27 16:12:30,884][20347] Updated weights for policy 0, policy_version 1524504 (0.0007) [2023-12-27 16:12:31,102][20381] Updated weights for policy 1, policy_version 1522935 (0.0006) [2023-12-27 16:12:31,168][20381] Updated weights for policy 1, policy_version 1522945 (0.0010) [2023-12-27 16:12:31,216][20381] Updated weights for policy 1, policy_version 1522955 (0.0006) [2023-12-27 16:12:31,604][20347] Updated weights for policy 0, policy_version 1524514 (0.0008) [2023-12-27 16:12:31,663][20347] Updated weights for policy 0, policy_version 1524524 (0.0007) [2023-12-27 16:12:31,717][20347] Updated weights for policy 0, policy_version 1524534 (0.0007) [2023-12-27 16:12:31,896][20381] Updated weights for policy 1, policy_version 1522965 (0.0007) [2023-12-27 16:12:31,949][20381] Updated weights for policy 1, policy_version 1522975 (0.0007) [2023-12-27 16:12:32,005][20381] Updated weights for policy 1, policy_version 1522985 (0.0007) [2023-12-27 16:12:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 780279808. Throughput: 0: 10367.7, 1: 10299.8. Samples: 780292396. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:32,363][19170] Avg episode reward: [(0, '980.520'), (1, '873.560')] [2023-12-27 16:12:32,404][20347] Updated weights for policy 0, policy_version 1524544 (0.0010) [2023-12-27 16:12:32,467][20347] Updated weights for policy 0, policy_version 1524554 (0.0008) [2023-12-27 16:12:32,531][20347] Updated weights for policy 0, policy_version 1524564 (0.0010) [2023-12-27 16:12:32,682][20381] Updated weights for policy 1, policy_version 1522995 (0.0008) [2023-12-27 16:12:32,746][20381] Updated weights for policy 1, policy_version 1523005 (0.0007) [2023-12-27 16:12:32,808][20381] Updated weights for policy 1, policy_version 1523015 (0.0008) [2023-12-27 16:12:33,202][20347] Updated weights for policy 0, policy_version 1524574 (0.0010) [2023-12-27 16:12:33,255][20347] Updated weights for policy 0, policy_version 1524584 (0.0010) [2023-12-27 16:12:33,308][20347] Updated weights for policy 0, policy_version 1524594 (0.0009) [2023-12-27 16:12:33,510][20381] Updated weights for policy 1, policy_version 1523025 (0.0008) [2023-12-27 16:12:33,567][20381] Updated weights for policy 1, policy_version 1523035 (0.0007) [2023-12-27 16:12:33,613][20381] Updated weights for policy 1, policy_version 1523045 (0.0007) [2023-12-27 16:12:33,666][20381] Updated weights for policy 1, policy_version 1523055 (0.0008) [2023-12-27 16:12:33,853][20347] Updated weights for policy 0, policy_version 1524604 (0.0007) [2023-12-27 16:12:33,903][20347] Updated weights for policy 0, policy_version 1524614 (0.0005) [2023-12-27 16:12:33,956][20347] Updated weights for policy 0, policy_version 1524624 (0.0005) [2023-12-27 16:12:34,414][20381] Updated weights for policy 1, policy_version 1523065 (0.0008) [2023-12-27 16:12:34,474][20381] Updated weights for policy 1, policy_version 1523075 (0.0009) [2023-12-27 16:12:34,535][20381] Updated weights for policy 1, policy_version 1523085 (0.0006) [2023-12-27 16:12:34,646][20347] Updated weights for policy 0, policy_version 1524634 (0.0006) [2023-12-27 16:12:34,698][20347] Updated weights for policy 0, policy_version 1524644 (0.0006) [2023-12-27 16:12:34,761][20347] Updated weights for policy 0, policy_version 1524654 (0.0006) [2023-12-27 16:12:34,817][20347] Updated weights for policy 0, policy_version 1524664 (0.0006) [2023-12-27 16:12:35,200][20381] Updated weights for policy 1, policy_version 1523095 (0.0010) [2023-12-27 16:12:35,249][20381] Updated weights for policy 1, policy_version 1523105 (0.0010) [2023-12-27 16:12:35,309][20381] Updated weights for policy 1, policy_version 1523115 (0.0009) [2023-12-27 16:12:35,442][20347] Updated weights for policy 0, policy_version 1524674 (0.0005) [2023-12-27 16:12:35,504][20347] Updated weights for policy 0, policy_version 1524684 (0.0006) [2023-12-27 16:12:35,559][20347] Updated weights for policy 0, policy_version 1524694 (0.0008) [2023-12-27 16:12:36,012][20381] Updated weights for policy 1, policy_version 1523125 (0.0009) [2023-12-27 16:12:36,076][20381] Updated weights for policy 1, policy_version 1523135 (0.0008) [2023-12-27 16:12:36,105][20347] Updated weights for policy 0, policy_version 1524704 (0.0007) [2023-12-27 16:12:36,134][20381] Updated weights for policy 1, policy_version 1523145 (0.0006) [2023-12-27 16:12:36,166][20347] Updated weights for policy 0, policy_version 1524714 (0.0007) [2023-12-27 16:12:36,232][20347] Updated weights for policy 0, policy_version 1524724 (0.0006) [2023-12-27 16:12:36,874][20381] Updated weights for policy 1, policy_version 1523155 (0.0006) [2023-12-27 16:12:36,910][20347] Updated weights for policy 0, policy_version 1524734 (0.0006) [2023-12-27 16:12:36,934][20381] Updated weights for policy 1, policy_version 1523165 (0.0007) [2023-12-27 16:12:36,968][20347] Updated weights for policy 0, policy_version 1524744 (0.0006) [2023-12-27 16:12:36,993][20381] Updated weights for policy 1, policy_version 1523175 (0.0005) [2023-12-27 16:12:37,031][20347] Updated weights for policy 0, policy_version 1524754 (0.0006) [2023-12-27 16:12:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 780386304. Throughput: 0: 10366.3, 1: 10302.3. Samples: 780354116. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.800')] [2023-12-27 16:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001523184_389971968.pth... [2023-12-27 16:12:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001524760_390414336.pth... [2023-12-27 16:12:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001522000_389668864.pth [2023-12-27 16:12:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001523576_390111232.pth [2023-12-27 16:12:37,639][20347] Updated weights for policy 0, policy_version 1524764 (0.0006) [2023-12-27 16:12:37,680][20381] Updated weights for policy 1, policy_version 1523185 (0.0006) [2023-12-27 16:12:37,704][20347] Updated weights for policy 0, policy_version 1524774 (0.0009) [2023-12-27 16:12:37,745][20381] Updated weights for policy 1, policy_version 1523195 (0.0006) [2023-12-27 16:12:37,769][20347] Updated weights for policy 0, policy_version 1524784 (0.0010) [2023-12-27 16:12:37,802][20381] Updated weights for policy 1, policy_version 1523205 (0.0006) [2023-12-27 16:12:37,862][20381] Updated weights for policy 1, policy_version 1523215 (0.0008) [2023-12-27 16:12:38,417][20347] Updated weights for policy 0, policy_version 1524794 (0.0010) [2023-12-27 16:12:38,468][20347] Updated weights for policy 0, policy_version 1524804 (0.0007) [2023-12-27 16:12:38,523][20347] Updated weights for policy 0, policy_version 1524814 (0.0008) [2023-12-27 16:12:38,532][20381] Updated weights for policy 1, policy_version 1523225 (0.0010) [2023-12-27 16:12:38,587][20347] Updated weights for policy 0, policy_version 1524824 (0.0009) [2023-12-27 16:12:38,591][20381] Updated weights for policy 1, policy_version 1523235 (0.0011) [2023-12-27 16:12:38,643][20381] Updated weights for policy 1, policy_version 1523245 (0.0011) [2023-12-27 16:12:39,282][20347] Updated weights for policy 0, policy_version 1524834 (0.0006) [2023-12-27 16:12:39,332][20381] Updated weights for policy 1, policy_version 1523255 (0.0010) [2023-12-27 16:12:39,346][20347] Updated weights for policy 0, policy_version 1524844 (0.0008) [2023-12-27 16:12:39,400][20381] Updated weights for policy 1, policy_version 1523265 (0.0009) [2023-12-27 16:12:39,418][20347] Updated weights for policy 0, policy_version 1524854 (0.0006) [2023-12-27 16:12:39,465][20381] Updated weights for policy 1, policy_version 1523275 (0.0008) [2023-12-27 16:12:40,155][20381] Updated weights for policy 1, policy_version 1523285 (0.0007) [2023-12-27 16:12:40,185][20347] Updated weights for policy 0, policy_version 1524864 (0.0008) [2023-12-27 16:12:40,208][20381] Updated weights for policy 1, policy_version 1523295 (0.0007) [2023-12-27 16:12:40,242][20347] Updated weights for policy 0, policy_version 1524874 (0.0008) [2023-12-27 16:12:40,264][20381] Updated weights for policy 1, policy_version 1523305 (0.0006) [2023-12-27 16:12:40,299][20347] Updated weights for policy 0, policy_version 1524884 (0.0007) [2023-12-27 16:12:40,892][20381] Updated weights for policy 1, policy_version 1523315 (0.0006) [2023-12-27 16:12:40,942][20381] Updated weights for policy 1, policy_version 1523325 (0.0005) [2023-12-27 16:12:40,990][20381] Updated weights for policy 1, policy_version 1523335 (0.0006) [2023-12-27 16:12:41,007][20347] Updated weights for policy 0, policy_version 1524894 (0.0008) [2023-12-27 16:12:41,073][20347] Updated weights for policy 0, policy_version 1524904 (0.0007) [2023-12-27 16:12:41,139][20347] Updated weights for policy 0, policy_version 1524914 (0.0009) [2023-12-27 16:12:41,800][20381] Updated weights for policy 1, policy_version 1523345 (0.0007) [2023-12-27 16:12:41,841][20347] Updated weights for policy 0, policy_version 1524925 (0.0007) [2023-12-27 16:12:41,862][20381] Updated weights for policy 1, policy_version 1523355 (0.0008) [2023-12-27 16:12:41,893][20347] Updated weights for policy 0, policy_version 1524935 (0.0007) [2023-12-27 16:12:41,918][20381] Updated weights for policy 1, policy_version 1523365 (0.0006) [2023-12-27 16:12:41,959][20347] Updated weights for policy 0, policy_version 1524945 (0.0007) [2023-12-27 16:12:41,979][20381] Updated weights for policy 1, policy_version 1523375 (0.0009) [2023-12-27 16:12:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 780484608. Throughput: 0: 10310.6, 1: 10296.8. Samples: 780475484. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.890')] [2023-12-27 16:12:42,643][20347] Updated weights for policy 0, policy_version 1524955 (0.0008) [2023-12-27 16:12:42,667][20381] Updated weights for policy 1, policy_version 1523385 (0.0008) [2023-12-27 16:12:42,703][20347] Updated weights for policy 0, policy_version 1524965 (0.0006) [2023-12-27 16:12:42,730][20381] Updated weights for policy 1, policy_version 1523395 (0.0006) [2023-12-27 16:12:42,770][20347] Updated weights for policy 0, policy_version 1524975 (0.0006) [2023-12-27 16:12:42,787][20381] Updated weights for policy 1, policy_version 1523405 (0.0006) [2023-12-27 16:12:43,424][20381] Updated weights for policy 1, policy_version 1523415 (0.0007) [2023-12-27 16:12:43,484][20381] Updated weights for policy 1, policy_version 1523425 (0.0006) [2023-12-27 16:12:43,491][20347] Updated weights for policy 0, policy_version 1524985 (0.0006) [2023-12-27 16:12:43,542][20381] Updated weights for policy 1, policy_version 1523435 (0.0006) [2023-12-27 16:12:43,546][20347] Updated weights for policy 0, policy_version 1524995 (0.0006) [2023-12-27 16:12:43,603][20347] Updated weights for policy 0, policy_version 1525005 (0.0008) [2023-12-27 16:12:43,635][20291] KL-divergence is very high: 107.8451 [2023-12-27 16:12:43,649][20347] Updated weights for policy 0, policy_version 1525015 (0.0005) [2023-12-27 16:12:44,179][20381] Updated weights for policy 1, policy_version 1523445 (0.0008) [2023-12-27 16:12:44,239][20381] Updated weights for policy 1, policy_version 1523455 (0.0007) [2023-12-27 16:12:44,280][20291] KL-divergence is very high: 109.1408 [2023-12-27 16:12:44,290][20381] Updated weights for policy 1, policy_version 1523465 (0.0006) [2023-12-27 16:12:44,318][20347] Updated weights for policy 0, policy_version 1525025 (0.0008) [2023-12-27 16:12:44,369][20291] KL-divergence is very high: 252.7509 [2023-12-27 16:12:44,376][20347] Updated weights for policy 0, policy_version 1525035 (0.0007) [2023-12-27 16:12:44,414][20291] KL-divergence is very high: 184.8195 [2023-12-27 16:12:44,431][20347] Updated weights for policy 0, policy_version 1525045 (0.0007) [2023-12-27 16:12:44,983][20381] Updated weights for policy 1, policy_version 1523475 (0.0008) [2023-12-27 16:12:45,048][20381] Updated weights for policy 1, policy_version 1523485 (0.0008) [2023-12-27 16:12:45,100][20381] Updated weights for policy 1, policy_version 1523495 (0.0007) [2023-12-27 16:12:45,115][20347] Updated weights for policy 0, policy_version 1525055 (0.0006) [2023-12-27 16:12:45,172][20347] Updated weights for policy 0, policy_version 1525065 (0.0007) [2023-12-27 16:12:45,227][20347] Updated weights for policy 0, policy_version 1525075 (0.0006) [2023-12-27 16:12:45,771][20381] Updated weights for policy 1, policy_version 1523505 (0.0008) [2023-12-27 16:12:45,832][20381] Updated weights for policy 1, policy_version 1523515 (0.0005) [2023-12-27 16:12:45,883][20381] Updated weights for policy 1, policy_version 1523525 (0.0005) [2023-12-27 16:12:45,930][20347] Updated weights for policy 0, policy_version 1525085 (0.0007) [2023-12-27 16:12:45,935][20381] Updated weights for policy 1, policy_version 1523535 (0.0005) [2023-12-27 16:12:45,974][20347] Updated weights for policy 0, policy_version 1525095 (0.0008) [2023-12-27 16:12:46,030][20347] Updated weights for policy 0, policy_version 1525105 (0.0007) [2023-12-27 16:12:46,496][20381] Updated weights for policy 1, policy_version 1523545 (0.0007) [2023-12-27 16:12:46,543][20381] Updated weights for policy 1, policy_version 1523555 (0.0007) [2023-12-27 16:12:46,591][20381] Updated weights for policy 1, policy_version 1523565 (0.0007) [2023-12-27 16:12:46,731][20347] Updated weights for policy 0, policy_version 1525115 (0.0008) [2023-12-27 16:12:46,786][20347] Updated weights for policy 0, policy_version 1525125 (0.0008) [2023-12-27 16:12:46,838][20347] Updated weights for policy 0, policy_version 1525135 (0.0007) [2023-12-27 16:12:47,302][20381] Updated weights for policy 1, policy_version 1523575 (0.0007) [2023-12-27 16:12:47,355][20381] Updated weights for policy 1, policy_version 1523585 (0.0008) [2023-12-27 16:12:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 780582912. Throughput: 0: 10230.7, 1: 10298.4. Samples: 780597612. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:47,363][19170] Avg episode reward: [(0, '970.890'), (1, '883.640')] [2023-12-27 16:12:47,408][20381] Updated weights for policy 1, policy_version 1523595 (0.0008) [2023-12-27 16:12:47,531][20347] Updated weights for policy 0, policy_version 1525145 (0.0007) [2023-12-27 16:12:47,594][20347] Updated weights for policy 0, policy_version 1525155 (0.0009) [2023-12-27 16:12:47,648][20347] Updated weights for policy 0, policy_version 1525165 (0.0010) [2023-12-27 16:12:47,706][20347] Updated weights for policy 0, policy_version 1525175 (0.0010) [2023-12-27 16:12:48,096][20381] Updated weights for policy 1, policy_version 1523605 (0.0008) [2023-12-27 16:12:48,149][20381] Updated weights for policy 1, policy_version 1523615 (0.0007) [2023-12-27 16:12:48,212][20381] Updated weights for policy 1, policy_version 1523625 (0.0008) [2023-12-27 16:12:48,383][20347] Updated weights for policy 0, policy_version 1525185 (0.0008) [2023-12-27 16:12:48,452][20347] Updated weights for policy 0, policy_version 1525195 (0.0007) [2023-12-27 16:12:48,512][20347] Updated weights for policy 0, policy_version 1525205 (0.0008) [2023-12-27 16:12:48,840][20381] Updated weights for policy 1, policy_version 1523635 (0.0007) [2023-12-27 16:12:48,893][20381] Updated weights for policy 1, policy_version 1523645 (0.0006) [2023-12-27 16:12:48,943][20381] Updated weights for policy 1, policy_version 1523655 (0.0007) [2023-12-27 16:12:49,132][20347] Updated weights for policy 0, policy_version 1525215 (0.0007) [2023-12-27 16:12:49,190][20347] Updated weights for policy 0, policy_version 1525225 (0.0005) [2023-12-27 16:12:49,257][20347] Updated weights for policy 0, policy_version 1525235 (0.0008) [2023-12-27 16:12:49,711][20381] Updated weights for policy 1, policy_version 1523665 (0.0008) [2023-12-27 16:12:49,771][20381] Updated weights for policy 1, policy_version 1523675 (0.0007) [2023-12-27 16:12:49,840][20381] Updated weights for policy 1, policy_version 1523685 (0.0008) [2023-12-27 16:12:49,900][20381] Updated weights for policy 1, policy_version 1523695 (0.0007) [2023-12-27 16:12:49,935][20347] Updated weights for policy 0, policy_version 1525245 (0.0009) [2023-12-27 16:12:49,945][20291] KL-divergence is very high: 623.7408 [2023-12-27 16:12:49,993][20347] Updated weights for policy 0, policy_version 1525255 (0.0006) [2023-12-27 16:12:49,995][20291] KL-divergence is very high: 730.7765 [2023-12-27 16:12:50,046][20291] KL-divergence is very high: 271.3388 [2023-12-27 16:12:50,059][20347] Updated weights for policy 0, policy_version 1525265 (0.0006) [2023-12-27 16:12:50,654][20381] Updated weights for policy 1, policy_version 1523705 (0.0006) [2023-12-27 16:12:50,656][20347] Updated weights for policy 0, policy_version 1525275 (0.0006) [2023-12-27 16:12:50,709][20381] Updated weights for policy 1, policy_version 1523715 (0.0007) [2023-12-27 16:12:50,709][20347] Updated weights for policy 0, policy_version 1525285 (0.0008) [2023-12-27 16:12:50,760][20347] Updated weights for policy 0, policy_version 1525295 (0.0008) [2023-12-27 16:12:50,763][20381] Updated weights for policy 1, policy_version 1523725 (0.0007) [2023-12-27 16:12:51,444][20381] Updated weights for policy 1, policy_version 1523735 (0.0008) [2023-12-27 16:12:51,506][20381] Updated weights for policy 1, policy_version 1523745 (0.0007) [2023-12-27 16:12:51,530][20347] Updated weights for policy 0, policy_version 1525305 (0.0007) [2023-12-27 16:12:51,571][20381] Updated weights for policy 1, policy_version 1523755 (0.0007) [2023-12-27 16:12:51,582][20347] Updated weights for policy 0, policy_version 1525315 (0.0007) [2023-12-27 16:12:51,643][20347] Updated weights for policy 0, policy_version 1525325 (0.0008) [2023-12-27 16:12:51,703][20347] Updated weights for policy 0, policy_version 1525335 (0.0006) [2023-12-27 16:12:52,253][20381] Updated weights for policy 1, policy_version 1523765 (0.0007) [2023-12-27 16:12:52,316][20381] Updated weights for policy 1, policy_version 1523775 (0.0008) [2023-12-27 16:12:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 780681216. Throughput: 0: 10225.9, 1: 10260.6. Samples: 780658440. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:52,363][19170] Avg episode reward: [(0, '912.560'), (1, '883.520')] [2023-12-27 16:12:52,377][20381] Updated weights for policy 1, policy_version 1523785 (0.0008) [2023-12-27 16:12:52,391][20347] Updated weights for policy 0, policy_version 1525345 (0.0007) [2023-12-27 16:12:52,422][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001523792_390127616.pth... [2023-12-27 16:12:52,426][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001522576_389816320.pth [2023-12-27 16:12:52,455][20347] Updated weights for policy 0, policy_version 1525355 (0.0007) [2023-12-27 16:12:52,507][20347] Updated weights for policy 0, policy_version 1525365 (0.0007) [2023-12-27 16:12:52,523][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001525368_390569984.pth... [2023-12-27 16:12:52,526][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001524152_390258688.pth [2023-12-27 16:12:52,943][20381] Updated weights for policy 1, policy_version 1523795 (0.0009) [2023-12-27 16:12:52,995][20381] Updated weights for policy 1, policy_version 1523805 (0.0010) [2023-12-27 16:12:53,012][20292] KL-divergence is very high: 364.8567 [2023-12-27 16:12:53,040][20381] Updated weights for policy 1, policy_version 1523815 (0.0010) [2023-12-27 16:12:53,049][20292] KL-divergence is very high: 216.1734 [2023-12-27 16:12:53,239][20347] Updated weights for policy 0, policy_version 1525375 (0.0006) [2023-12-27 16:12:53,306][20347] Updated weights for policy 0, policy_version 1525385 (0.0006) [2023-12-27 16:12:53,365][20347] Updated weights for policy 0, policy_version 1525395 (0.0007) [2023-12-27 16:12:53,714][20381] Updated weights for policy 1, policy_version 1523825 (0.0010) [2023-12-27 16:12:53,759][20381] Updated weights for policy 1, policy_version 1523835 (0.0010) [2023-12-27 16:12:53,807][20381] Updated weights for policy 1, policy_version 1523845 (0.0010) [2023-12-27 16:12:53,862][20381] Updated weights for policy 1, policy_version 1523855 (0.0010) [2023-12-27 16:12:53,998][20347] Updated weights for policy 0, policy_version 1525405 (0.0007) [2023-12-27 16:12:54,052][20347] Updated weights for policy 0, policy_version 1525415 (0.0008) [2023-12-27 16:12:54,110][20347] Updated weights for policy 0, policy_version 1525425 (0.0007) [2023-12-27 16:12:54,576][20381] Updated weights for policy 1, policy_version 1523865 (0.0010) [2023-12-27 16:12:54,627][20381] Updated weights for policy 1, policy_version 1523875 (0.0009) [2023-12-27 16:12:54,691][20381] Updated weights for policy 1, policy_version 1523885 (0.0008) [2023-12-27 16:12:54,792][20347] Updated weights for policy 0, policy_version 1525435 (0.0007) [2023-12-27 16:12:54,846][20347] Updated weights for policy 0, policy_version 1525445 (0.0007) [2023-12-27 16:12:54,899][20347] Updated weights for policy 0, policy_version 1525455 (0.0007) [2023-12-27 16:12:55,279][20381] Updated weights for policy 1, policy_version 1523895 (0.0009) [2023-12-27 16:12:55,340][20381] Updated weights for policy 1, policy_version 1523905 (0.0007) [2023-12-27 16:12:55,393][20381] Updated weights for policy 1, policy_version 1523915 (0.0005) [2023-12-27 16:12:55,650][20347] Updated weights for policy 0, policy_version 1525465 (0.0007) [2023-12-27 16:12:55,697][20347] Updated weights for policy 0, policy_version 1525475 (0.0008) [2023-12-27 16:12:55,749][20347] Updated weights for policy 0, policy_version 1525485 (0.0009) [2023-12-27 16:12:55,797][20347] Updated weights for policy 0, policy_version 1525495 (0.0008) [2023-12-27 16:12:55,910][20381] Updated weights for policy 1, policy_version 1523925 (0.0005) [2023-12-27 16:12:55,962][20381] Updated weights for policy 1, policy_version 1523935 (0.0005) [2023-12-27 16:12:56,019][20381] Updated weights for policy 1, policy_version 1523945 (0.0005) [2023-12-27 16:12:56,519][20347] Updated weights for policy 0, policy_version 1525505 (0.0007) [2023-12-27 16:12:56,572][20347] Updated weights for policy 0, policy_version 1525515 (0.0007) [2023-12-27 16:12:56,612][20381] Updated weights for policy 1, policy_version 1523955 (0.0007) [2023-12-27 16:12:56,624][20347] Updated weights for policy 0, policy_version 1525525 (0.0006) [2023-12-27 16:12:56,663][20381] Updated weights for policy 1, policy_version 1523965 (0.0011) [2023-12-27 16:12:56,714][20381] Updated weights for policy 1, policy_version 1523975 (0.0010) [2023-12-27 16:12:57,286][20347] Updated weights for policy 0, policy_version 1525535 (0.0007) [2023-12-27 16:12:57,293][20381] Updated weights for policy 1, policy_version 1523985 (0.0010) [2023-12-27 16:12:57,344][20347] Updated weights for policy 0, policy_version 1525545 (0.0006) [2023-12-27 16:12:57,351][20381] Updated weights for policy 1, policy_version 1523995 (0.0006) [2023-12-27 16:12:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 780787712. Throughput: 0: 10187.8, 1: 10359.0. Samples: 780782580. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:12:57,363][19170] Avg episode reward: [(0, '922.150'), (1, '903.180')] [2023-12-27 16:12:57,392][20347] Updated weights for policy 0, policy_version 1525555 (0.0008) [2023-12-27 16:12:57,401][20381] Updated weights for policy 1, policy_version 1524005 (0.0007) [2023-12-27 16:12:57,449][20381] Updated weights for policy 1, policy_version 1524015 (0.0010) [2023-12-27 16:12:58,082][20381] Updated weights for policy 1, policy_version 1524025 (0.0006) [2023-12-27 16:12:58,101][20347] Updated weights for policy 0, policy_version 1525565 (0.0006) [2023-12-27 16:12:58,135][20381] Updated weights for policy 1, policy_version 1524035 (0.0006) [2023-12-27 16:12:58,157][20347] Updated weights for policy 0, policy_version 1525575 (0.0007) [2023-12-27 16:12:58,193][20381] Updated weights for policy 1, policy_version 1524045 (0.0006) [2023-12-27 16:12:58,214][20347] Updated weights for policy 0, policy_version 1525585 (0.0006) [2023-12-27 16:12:58,951][20381] Updated weights for policy 1, policy_version 1524055 (0.0008) [2023-12-27 16:12:59,017][20381] Updated weights for policy 1, policy_version 1524065 (0.0007) [2023-12-27 16:12:59,026][20347] Updated weights for policy 0, policy_version 1525595 (0.0007) [2023-12-27 16:12:59,076][20381] Updated weights for policy 1, policy_version 1524075 (0.0007) [2023-12-27 16:12:59,090][20347] Updated weights for policy 0, policy_version 1525605 (0.0008) [2023-12-27 16:12:59,157][20347] Updated weights for policy 0, policy_version 1525615 (0.0008) [2023-12-27 16:12:59,776][20381] Updated weights for policy 1, policy_version 1524085 (0.0007) [2023-12-27 16:12:59,838][20381] Updated weights for policy 1, policy_version 1524095 (0.0008) [2023-12-27 16:12:59,896][20347] Updated weights for policy 0, policy_version 1525625 (0.0006) [2023-12-27 16:12:59,897][20381] Updated weights for policy 1, policy_version 1524105 (0.0009) [2023-12-27 16:12:59,961][20347] Updated weights for policy 0, policy_version 1525635 (0.0007) [2023-12-27 16:13:00,024][20347] Updated weights for policy 0, policy_version 1525645 (0.0006) [2023-12-27 16:13:00,085][20347] Updated weights for policy 0, policy_version 1525655 (0.0005) [2023-12-27 16:13:00,575][20381] Updated weights for policy 1, policy_version 1524115 (0.0006) [2023-12-27 16:13:00,622][20381] Updated weights for policy 1, policy_version 1524125 (0.0007) [2023-12-27 16:13:00,675][20381] Updated weights for policy 1, policy_version 1524135 (0.0007) [2023-12-27 16:13:00,692][20347] Updated weights for policy 0, policy_version 1525665 (0.0007) [2023-12-27 16:13:00,749][20347] Updated weights for policy 0, policy_version 1525675 (0.0005) [2023-12-27 16:13:00,812][20347] Updated weights for policy 0, policy_version 1525685 (0.0005) [2023-12-27 16:13:01,312][20381] Updated weights for policy 1, policy_version 1524145 (0.0007) [2023-12-27 16:13:01,379][20381] Updated weights for policy 1, policy_version 1524155 (0.0011) [2023-12-27 16:13:01,440][20347] Updated weights for policy 0, policy_version 1525695 (0.0007) [2023-12-27 16:13:01,442][20381] Updated weights for policy 1, policy_version 1524165 (0.0008) [2023-12-27 16:13:01,488][20347] Updated weights for policy 0, policy_version 1525705 (0.0006) [2023-12-27 16:13:01,493][20381] Updated weights for policy 1, policy_version 1524175 (0.0008) [2023-12-27 16:13:01,542][20347] Updated weights for policy 0, policy_version 1525715 (0.0006) [2023-12-27 16:13:02,179][20381] Updated weights for policy 1, policy_version 1524185 (0.0006) [2023-12-27 16:13:02,242][20381] Updated weights for policy 1, policy_version 1524195 (0.0009) [2023-12-27 16:13:02,261][20347] Updated weights for policy 0, policy_version 1525725 (0.0007) [2023-12-27 16:13:02,303][20381] Updated weights for policy 1, policy_version 1524205 (0.0007) [2023-12-27 16:13:02,319][20347] Updated weights for policy 0, policy_version 1525735 (0.0008) [2023-12-27 16:13:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 780894208. Throughput: 0: 10114.6, 1: 10331.2. Samples: 780904956. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:02,363][19170] Avg episode reward: [(0, '961.010'), (1, '912.490')] [2023-12-27 16:13:02,383][20347] Updated weights for policy 0, policy_version 1525745 (0.0008) [2023-12-27 16:13:02,979][20347] Updated weights for policy 0, policy_version 1525755 (0.0007) [2023-12-27 16:13:02,987][20381] Updated weights for policy 1, policy_version 1524215 (0.0007) [2023-12-27 16:13:03,025][20347] Updated weights for policy 0, policy_version 1525765 (0.0005) [2023-12-27 16:13:03,039][20381] Updated weights for policy 1, policy_version 1524225 (0.0007) [2023-12-27 16:13:03,047][20291] KL-divergence is very high: 242.8962 [2023-12-27 16:13:03,069][20347] Updated weights for policy 0, policy_version 1525775 (0.0005) [2023-12-27 16:13:03,086][20291] KL-divergence is very high: 188.1737 [2023-12-27 16:13:03,103][20381] Updated weights for policy 1, policy_version 1524235 (0.0007) [2023-12-27 16:13:03,668][20347] Updated weights for policy 0, policy_version 1525785 (0.0005) [2023-12-27 16:13:03,711][20381] Updated weights for policy 1, policy_version 1524245 (0.0007) [2023-12-27 16:13:03,726][20347] Updated weights for policy 0, policy_version 1525795 (0.0007) [2023-12-27 16:13:03,772][20381] Updated weights for policy 1, policy_version 1524255 (0.0006) [2023-12-27 16:13:03,783][20347] Updated weights for policy 0, policy_version 1525805 (0.0006) [2023-12-27 16:13:03,839][20381] Updated weights for policy 1, policy_version 1524265 (0.0006) [2023-12-27 16:13:03,849][20347] Updated weights for policy 0, policy_version 1525815 (0.0006) [2023-12-27 16:13:04,517][20381] Updated weights for policy 1, policy_version 1524275 (0.0007) [2023-12-27 16:13:04,543][20347] Updated weights for policy 0, policy_version 1525825 (0.0007) [2023-12-27 16:13:04,575][20381] Updated weights for policy 1, policy_version 1524285 (0.0005) [2023-12-27 16:13:04,603][20347] Updated weights for policy 0, policy_version 1525835 (0.0008) [2023-12-27 16:13:04,635][20381] Updated weights for policy 1, policy_version 1524295 (0.0006) [2023-12-27 16:13:04,660][20347] Updated weights for policy 0, policy_version 1525845 (0.0007) [2023-12-27 16:13:05,205][20347] Updated weights for policy 0, policy_version 1525855 (0.0005) [2023-12-27 16:13:05,252][20347] Updated weights for policy 0, policy_version 1525865 (0.0005) [2023-12-27 16:13:05,303][20347] Updated weights for policy 0, policy_version 1525875 (0.0005) [2023-12-27 16:13:05,340][20381] Updated weights for policy 1, policy_version 1524305 (0.0008) [2023-12-27 16:13:05,397][20381] Updated weights for policy 1, policy_version 1524315 (0.0007) [2023-12-27 16:13:05,458][20381] Updated weights for policy 1, policy_version 1524325 (0.0008) [2023-12-27 16:13:05,511][20381] Updated weights for policy 1, policy_version 1524335 (0.0007) [2023-12-27 16:13:05,906][20347] Updated weights for policy 0, policy_version 1525885 (0.0007) [2023-12-27 16:13:05,964][20347] Updated weights for policy 0, policy_version 1525895 (0.0008) [2023-12-27 16:13:06,019][20347] Updated weights for policy 0, policy_version 1525905 (0.0007) [2023-12-27 16:13:06,175][20381] Updated weights for policy 1, policy_version 1524345 (0.0009) [2023-12-27 16:13:06,236][20381] Updated weights for policy 1, policy_version 1524355 (0.0008) [2023-12-27 16:13:06,300][20381] Updated weights for policy 1, policy_version 1524365 (0.0007) [2023-12-27 16:13:06,661][20347] Updated weights for policy 0, policy_version 1525915 (0.0006) [2023-12-27 16:13:06,730][20347] Updated weights for policy 0, policy_version 1525925 (0.0007) [2023-12-27 16:13:06,791][20347] Updated weights for policy 0, policy_version 1525935 (0.0008) [2023-12-27 16:13:06,887][20381] Updated weights for policy 1, policy_version 1524375 (0.0007) [2023-12-27 16:13:06,950][20381] Updated weights for policy 1, policy_version 1524385 (0.0005) [2023-12-27 16:13:07,014][20381] Updated weights for policy 1, policy_version 1524395 (0.0005) [2023-12-27 16:13:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 781000704. Throughput: 0: 10182.4, 1: 10330.3. Samples: 780968992. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:07,363][19170] Avg episode reward: [(0, '961.000'), (1, '853.410')] [2023-12-27 16:13:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001524400_390283264.pth... [2023-12-27 16:13:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001525944_390717440.pth... [2023-12-27 16:13:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001523184_389971968.pth [2023-12-27 16:13:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001524760_390414336.pth [2023-12-27 16:13:07,446][20347] Updated weights for policy 0, policy_version 1525945 (0.0008) [2023-12-27 16:13:07,501][20347] Updated weights for policy 0, policy_version 1525955 (0.0008) [2023-12-27 16:13:07,553][20347] Updated weights for policy 0, policy_version 1525965 (0.0008) [2023-12-27 16:13:07,602][20347] Updated weights for policy 0, policy_version 1525975 (0.0008) [2023-12-27 16:13:07,631][20381] Updated weights for policy 1, policy_version 1524405 (0.0005) [2023-12-27 16:13:07,693][20381] Updated weights for policy 1, policy_version 1524415 (0.0005) [2023-12-27 16:13:07,747][20381] Updated weights for policy 1, policy_version 1524425 (0.0005) [2023-12-27 16:13:08,320][20381] Updated weights for policy 1, policy_version 1524435 (0.0006) [2023-12-27 16:13:08,349][20347] Updated weights for policy 0, policy_version 1525985 (0.0010) [2023-12-27 16:13:08,388][20381] Updated weights for policy 1, policy_version 1524445 (0.0007) [2023-12-27 16:13:08,409][20347] Updated weights for policy 0, policy_version 1525995 (0.0008) [2023-12-27 16:13:08,455][20381] Updated weights for policy 1, policy_version 1524455 (0.0007) [2023-12-27 16:13:08,461][20347] Updated weights for policy 0, policy_version 1526005 (0.0007) [2023-12-27 16:13:09,047][20381] Updated weights for policy 1, policy_version 1524465 (0.0007) [2023-12-27 16:13:09,105][20381] Updated weights for policy 1, policy_version 1524475 (0.0011) [2023-12-27 16:13:09,137][20347] Updated weights for policy 0, policy_version 1526015 (0.0006) [2023-12-27 16:13:09,165][20381] Updated weights for policy 1, policy_version 1524485 (0.0011) [2023-12-27 16:13:09,203][20347] Updated weights for policy 0, policy_version 1526025 (0.0011) [2023-12-27 16:13:09,224][20381] Updated weights for policy 1, policy_version 1524495 (0.0009) [2023-12-27 16:13:09,272][20347] Updated weights for policy 0, policy_version 1526035 (0.0010) [2023-12-27 16:13:09,969][20347] Updated weights for policy 0, policy_version 1526045 (0.0010) [2023-12-27 16:13:10,008][20381] Updated weights for policy 1, policy_version 1524505 (0.0008) [2023-12-27 16:13:10,024][20347] Updated weights for policy 0, policy_version 1526055 (0.0011) [2023-12-27 16:13:10,068][20381] Updated weights for policy 1, policy_version 1524515 (0.0009) [2023-12-27 16:13:10,080][20347] Updated weights for policy 0, policy_version 1526065 (0.0011) [2023-12-27 16:13:10,124][20381] Updated weights for policy 1, policy_version 1524525 (0.0006) [2023-12-27 16:13:10,761][20347] Updated weights for policy 0, policy_version 1526075 (0.0009) [2023-12-27 16:13:10,806][20347] Updated weights for policy 0, policy_version 1526085 (0.0005) [2023-12-27 16:13:10,811][20381] Updated weights for policy 1, policy_version 1524535 (0.0008) [2023-12-27 16:13:10,864][20381] Updated weights for policy 1, policy_version 1524545 (0.0007) [2023-12-27 16:13:10,865][20347] Updated weights for policy 0, policy_version 1526095 (0.0006) [2023-12-27 16:13:10,922][20381] Updated weights for policy 1, policy_version 1524555 (0.0005) [2023-12-27 16:13:11,598][20347] Updated weights for policy 0, policy_version 1526105 (0.0007) [2023-12-27 16:13:11,624][20381] Updated weights for policy 1, policy_version 1524565 (0.0007) [2023-12-27 16:13:11,664][20347] Updated weights for policy 0, policy_version 1526115 (0.0010) [2023-12-27 16:13:11,685][20381] Updated weights for policy 1, policy_version 1524575 (0.0007) [2023-12-27 16:13:11,721][20347] Updated weights for policy 0, policy_version 1526125 (0.0008) [2023-12-27 16:13:11,749][20381] Updated weights for policy 1, policy_version 1524585 (0.0008) [2023-12-27 16:13:11,786][20347] Updated weights for policy 0, policy_version 1526135 (0.0007) [2023-12-27 16:13:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 781099008. Throughput: 0: 10216.9, 1: 10370.7. Samples: 781093740. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:12,363][19170] Avg episode reward: [(0, '990.220'), (1, '863.590')] [2023-12-27 16:13:12,449][20347] Updated weights for policy 0, policy_version 1526145 (0.0006) [2023-12-27 16:13:12,477][20381] Updated weights for policy 1, policy_version 1524595 (0.0008) [2023-12-27 16:13:12,506][20347] Updated weights for policy 0, policy_version 1526155 (0.0007) [2023-12-27 16:13:12,540][20381] Updated weights for policy 1, policy_version 1524605 (0.0008) [2023-12-27 16:13:12,563][20347] Updated weights for policy 0, policy_version 1526165 (0.0007) [2023-12-27 16:13:12,602][20381] Updated weights for policy 1, policy_version 1524615 (0.0007) [2023-12-27 16:13:13,190][20347] Updated weights for policy 0, policy_version 1526175 (0.0007) [2023-12-27 16:13:13,224][20381] Updated weights for policy 1, policy_version 1524625 (0.0007) [2023-12-27 16:13:13,247][20347] Updated weights for policy 0, policy_version 1526185 (0.0010) [2023-12-27 16:13:13,280][20381] Updated weights for policy 1, policy_version 1524635 (0.0005) [2023-12-27 16:13:13,299][20347] Updated weights for policy 0, policy_version 1526195 (0.0010) [2023-12-27 16:13:13,334][20381] Updated weights for policy 1, policy_version 1524645 (0.0005) [2023-12-27 16:13:13,391][20381] Updated weights for policy 1, policy_version 1524655 (0.0005) [2023-12-27 16:13:13,914][20347] Updated weights for policy 0, policy_version 1526205 (0.0008) [2023-12-27 16:13:13,969][20347] Updated weights for policy 0, policy_version 1526215 (0.0009) [2023-12-27 16:13:13,991][20381] Updated weights for policy 1, policy_version 1524665 (0.0006) [2023-12-27 16:13:14,021][20347] Updated weights for policy 0, policy_version 1526225 (0.0010) [2023-12-27 16:13:14,046][20381] Updated weights for policy 1, policy_version 1524675 (0.0005) [2023-12-27 16:13:14,100][20381] Updated weights for policy 1, policy_version 1524685 (0.0005) [2023-12-27 16:13:14,652][20347] Updated weights for policy 0, policy_version 1526235 (0.0010) [2023-12-27 16:13:14,675][20381] Updated weights for policy 1, policy_version 1524695 (0.0006) [2023-12-27 16:13:14,702][20347] Updated weights for policy 0, policy_version 1526245 (0.0010) [2023-12-27 16:13:14,723][20381] Updated weights for policy 1, policy_version 1524705 (0.0008) [2023-12-27 16:13:14,751][20347] Updated weights for policy 0, policy_version 1526255 (0.0010) [2023-12-27 16:13:14,781][20381] Updated weights for policy 1, policy_version 1524715 (0.0010) [2023-12-27 16:13:15,411][20381] Updated weights for policy 1, policy_version 1524725 (0.0008) [2023-12-27 16:13:15,473][20381] Updated weights for policy 1, policy_version 1524735 (0.0011) [2023-12-27 16:13:15,507][20347] Updated weights for policy 0, policy_version 1526265 (0.0011) [2023-12-27 16:13:15,535][20381] Updated weights for policy 1, policy_version 1524745 (0.0010) [2023-12-27 16:13:15,571][20347] Updated weights for policy 0, policy_version 1526275 (0.0011) [2023-12-27 16:13:15,631][20347] Updated weights for policy 0, policy_version 1526285 (0.0011) [2023-12-27 16:13:15,691][20347] Updated weights for policy 0, policy_version 1526295 (0.0010) [2023-12-27 16:13:16,126][20381] Updated weights for policy 1, policy_version 1524755 (0.0009) [2023-12-27 16:13:16,184][20381] Updated weights for policy 1, policy_version 1524765 (0.0006) [2023-12-27 16:13:16,229][20381] Updated weights for policy 1, policy_version 1524775 (0.0005) [2023-12-27 16:13:16,322][20347] Updated weights for policy 0, policy_version 1526305 (0.0006) [2023-12-27 16:13:16,375][20347] Updated weights for policy 0, policy_version 1526315 (0.0005) [2023-12-27 16:13:16,436][20347] Updated weights for policy 0, policy_version 1526325 (0.0005) [2023-12-27 16:13:16,841][20381] Updated weights for policy 1, policy_version 1524785 (0.0006) [2023-12-27 16:13:16,892][20381] Updated weights for policy 1, policy_version 1524795 (0.0010) [2023-12-27 16:13:16,953][20381] Updated weights for policy 1, policy_version 1524805 (0.0011) [2023-12-27 16:13:16,981][20347] Updated weights for policy 0, policy_version 1526335 (0.0005) [2023-12-27 16:13:17,009][20381] Updated weights for policy 1, policy_version 1524815 (0.0011) [2023-12-27 16:13:17,035][20347] Updated weights for policy 0, policy_version 1526345 (0.0006) [2023-12-27 16:13:17,081][20347] Updated weights for policy 0, policy_version 1526355 (0.0007) [2023-12-27 16:13:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 781213696. Throughput: 0: 10282.7, 1: 10364.4. Samples: 781221512. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:17,363][19170] Avg episode reward: [(0, '980.530'), (1, '864.280')] [2023-12-27 16:13:17,701][20381] Updated weights for policy 1, policy_version 1524825 (0.0011) [2023-12-27 16:13:17,738][20292] KL-divergence is very high: 146.1425 [2023-12-27 16:13:17,753][20381] Updated weights for policy 1, policy_version 1524835 (0.0010) [2023-12-27 16:13:17,773][20347] Updated weights for policy 0, policy_version 1526365 (0.0008) [2023-12-27 16:13:17,778][20292] KL-divergence is very high: 182.1435 [2023-12-27 16:13:17,811][20381] Updated weights for policy 1, policy_version 1524845 (0.0011) [2023-12-27 16:13:17,828][20347] Updated weights for policy 0, policy_version 1526375 (0.0008) [2023-12-27 16:13:17,882][20347] Updated weights for policy 0, policy_version 1526385 (0.0010) [2023-12-27 16:13:18,463][20347] Updated weights for policy 0, policy_version 1526395 (0.0010) [2023-12-27 16:13:18,465][20292] KL-divergence is very high: 155.0381 [2023-12-27 16:13:18,483][20381] Updated weights for policy 1, policy_version 1524855 (0.0011) [2023-12-27 16:13:18,511][20292] KL-divergence is very high: 146.9169 [2023-12-27 16:13:18,512][20347] Updated weights for policy 0, policy_version 1526405 (0.0008) [2023-12-27 16:13:18,538][20381] Updated weights for policy 1, policy_version 1524865 (0.0011) [2023-12-27 16:13:18,567][20347] Updated weights for policy 0, policy_version 1526415 (0.0006) [2023-12-27 16:13:18,588][20381] Updated weights for policy 1, policy_version 1524875 (0.0011) [2023-12-27 16:13:19,260][20381] Updated weights for policy 1, policy_version 1524885 (0.0011) [2023-12-27 16:13:19,311][20347] Updated weights for policy 0, policy_version 1526425 (0.0006) [2023-12-27 16:13:19,316][20381] Updated weights for policy 1, policy_version 1524895 (0.0009) [2023-12-27 16:13:19,375][20347] Updated weights for policy 0, policy_version 1526435 (0.0007) [2023-12-27 16:13:19,380][20381] Updated weights for policy 1, policy_version 1524905 (0.0008) [2023-12-27 16:13:19,437][20347] Updated weights for policy 0, policy_version 1526445 (0.0007) [2023-12-27 16:13:19,502][20347] Updated weights for policy 0, policy_version 1526455 (0.0007) [2023-12-27 16:13:20,087][20381] Updated weights for policy 1, policy_version 1524915 (0.0007) [2023-12-27 16:13:20,149][20381] Updated weights for policy 1, policy_version 1524925 (0.0007) [2023-12-27 16:13:20,218][20381] Updated weights for policy 1, policy_version 1524935 (0.0006) [2023-12-27 16:13:20,257][20347] Updated weights for policy 0, policy_version 1526465 (0.0006) [2023-12-27 16:13:20,328][20347] Updated weights for policy 0, policy_version 1526475 (0.0008) [2023-12-27 16:13:20,386][20347] Updated weights for policy 0, policy_version 1526485 (0.0007) [2023-12-27 16:13:20,920][20381] Updated weights for policy 1, policy_version 1524945 (0.0007) [2023-12-27 16:13:20,981][20381] Updated weights for policy 1, policy_version 1524955 (0.0008) [2023-12-27 16:13:21,046][20381] Updated weights for policy 1, policy_version 1524965 (0.0007) [2023-12-27 16:13:21,048][20347] Updated weights for policy 0, policy_version 1526495 (0.0007) [2023-12-27 16:13:21,110][20381] Updated weights for policy 1, policy_version 1524975 (0.0006) [2023-12-27 16:13:21,113][20347] Updated weights for policy 0, policy_version 1526505 (0.0007) [2023-12-27 16:13:21,157][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000000 [2023-12-27 16:13:21,832][20381] Updated weights for policy 1, policy_version 1524985 (0.0008) [2023-12-27 16:13:21,864][20292] KL-divergence is very high: 155.1610 [2023-12-27 16:13:21,888][20381] Updated weights for policy 1, policy_version 1524995 (0.0008) [2023-12-27 16:13:21,909][20347] Updated weights for policy 0, policy_version 1526515 (0.0008) [2023-12-27 16:13:21,912][20292] KL-divergence is very high: 195.7229 [2023-12-27 16:13:21,948][20381] Updated weights for policy 1, policy_version 1525005 (0.0008) [2023-12-27 16:13:21,961][20292] KL-divergence is very high: 117.7059 [2023-12-27 16:13:21,968][20347] Updated weights for policy 0, policy_version 1526525 (0.0007) [2023-12-27 16:13:22,033][20347] Updated weights for policy 0, policy_version 1526535 (0.0006) [2023-12-27 16:13:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 781312000. Throughput: 0: 10240.7, 1: 10383.9. Samples: 781282220. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:22,363][19170] Avg episode reward: [(0, '922.070'), (1, '853.930')] [2023-12-27 16:13:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001525008_390438912.pth... [2023-12-27 16:13:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001526544_390873088.pth... [2023-12-27 16:13:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001523792_390127616.pth [2023-12-27 16:13:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001525368_390569984.pth [2023-12-27 16:13:22,685][20381] Updated weights for policy 1, policy_version 1525015 (0.0010) [2023-12-27 16:13:22,742][20381] Updated weights for policy 1, policy_version 1525025 (0.0011) [2023-12-27 16:13:22,748][20347] Updated weights for policy 0, policy_version 1526545 (0.0007) [2023-12-27 16:13:22,805][20347] Updated weights for policy 0, policy_version 1526555 (0.0011) [2023-12-27 16:13:22,810][20381] Updated weights for policy 1, policy_version 1525035 (0.0011) [2023-12-27 16:13:22,865][20347] Updated weights for policy 0, policy_version 1526565 (0.0010) [2023-12-27 16:13:22,930][20347] Updated weights for policy 0, policy_version 1526575 (0.0007) [2023-12-27 16:13:23,481][20381] Updated weights for policy 1, policy_version 1525045 (0.0008) [2023-12-27 16:13:23,537][20381] Updated weights for policy 1, policy_version 1525055 (0.0005) [2023-12-27 16:13:23,568][20347] Updated weights for policy 0, policy_version 1526585 (0.0008) [2023-12-27 16:13:23,570][20292] KL-divergence is very high: 183.6218 [2023-12-27 16:13:23,584][20381] Updated weights for policy 1, policy_version 1525065 (0.0006) [2023-12-27 16:13:23,590][20292] KL-divergence is very high: 162.6471 [2023-12-27 16:13:23,610][20292] KL-divergence is very high: 189.9688 [2023-12-27 16:13:23,625][20347] Updated weights for policy 0, policy_version 1526595 (0.0005) [2023-12-27 16:13:23,673][20347] Updated weights for policy 0, policy_version 1526605 (0.0005) [2023-12-27 16:13:24,272][20381] Updated weights for policy 1, policy_version 1525075 (0.0008) [2023-12-27 16:13:24,300][20347] Updated weights for policy 0, policy_version 1526615 (0.0009) [2023-12-27 16:13:24,325][20381] Updated weights for policy 1, policy_version 1525085 (0.0005) [2023-12-27 16:13:24,349][20347] Updated weights for policy 0, policy_version 1526625 (0.0010) [2023-12-27 16:13:24,377][20381] Updated weights for policy 1, policy_version 1525095 (0.0005) [2023-12-27 16:13:24,405][20347] Updated weights for policy 0, policy_version 1526635 (0.0010) [2023-12-27 16:13:25,072][20381] Updated weights for policy 1, policy_version 1525105 (0.0006) [2023-12-27 16:13:25,103][20347] Updated weights for policy 0, policy_version 1526645 (0.0010) [2023-12-27 16:13:25,124][20381] Updated weights for policy 1, policy_version 1525115 (0.0005) [2023-12-27 16:13:25,155][20347] Updated weights for policy 0, policy_version 1526655 (0.0010) [2023-12-27 16:13:25,175][20381] Updated weights for policy 1, policy_version 1525125 (0.0005) [2023-12-27 16:13:25,204][20347] Updated weights for policy 0, policy_version 1526665 (0.0010) [2023-12-27 16:13:25,229][20381] Updated weights for policy 1, policy_version 1525135 (0.0006) [2023-12-27 16:13:25,788][20347] Updated weights for policy 0, policy_version 1526675 (0.0010) [2023-12-27 16:13:25,842][20347] Updated weights for policy 0, policy_version 1526685 (0.0010) [2023-12-27 16:13:25,890][20347] Updated weights for policy 0, policy_version 1526695 (0.0010) [2023-12-27 16:13:25,908][20381] Updated weights for policy 1, policy_version 1525145 (0.0005) [2023-12-27 16:13:25,963][20381] Updated weights for policy 1, policy_version 1525155 (0.0007) [2023-12-27 16:13:26,014][20381] Updated weights for policy 1, policy_version 1525165 (0.0007) [2023-12-27 16:13:26,567][20347] Updated weights for policy 0, policy_version 1526705 (0.0010) [2023-12-27 16:13:26,618][20347] Updated weights for policy 0, policy_version 1526715 (0.0010) [2023-12-27 16:13:26,664][20381] Updated weights for policy 1, policy_version 1525175 (0.0006) [2023-12-27 16:13:26,666][20347] Updated weights for policy 0, policy_version 1526725 (0.0010) [2023-12-27 16:13:26,709][20381] Updated weights for policy 1, policy_version 1525185 (0.0005) [2023-12-27 16:13:26,711][20347] Updated weights for policy 0, policy_version 1526735 (0.0010) [2023-12-27 16:13:26,762][20381] Updated weights for policy 1, policy_version 1525195 (0.0006) [2023-12-27 16:13:27,312][20347] Updated weights for policy 0, policy_version 1526745 (0.0008) [2023-12-27 16:13:27,336][20381] Updated weights for policy 1, policy_version 1525205 (0.0005) [2023-12-27 16:13:27,360][20347] Updated weights for policy 0, policy_version 1526755 (0.0010) [2023-12-27 16:13:27,363][19170] Fps is (10 sec: 19660.0, 60 sec: 20616.4, 300 sec: 20521.6). Total num frames: 781410304. Throughput: 0: 10233.4, 1: 10397.0. Samples: 781403860. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:27,364][19170] Avg episode reward: [(0, '941.680'), (1, '775.770')] [2023-12-27 16:13:27,395][20381] Updated weights for policy 1, policy_version 1525215 (0.0006) [2023-12-27 16:13:27,411][20347] Updated weights for policy 0, policy_version 1526765 (0.0010) [2023-12-27 16:13:27,446][20381] Updated weights for policy 1, policy_version 1525225 (0.0006) [2023-12-27 16:13:28,090][20347] Updated weights for policy 0, policy_version 1526775 (0.0007) [2023-12-27 16:13:28,139][20347] Updated weights for policy 0, policy_version 1526785 (0.0006) [2023-12-27 16:13:28,141][20381] Updated weights for policy 1, policy_version 1525235 (0.0007) [2023-12-27 16:13:28,196][20381] Updated weights for policy 1, policy_version 1525245 (0.0005) [2023-12-27 16:13:28,200][20347] Updated weights for policy 0, policy_version 1526795 (0.0008) [2023-12-27 16:13:28,243][20381] Updated weights for policy 1, policy_version 1525255 (0.0007) [2023-12-27 16:13:28,838][20381] Updated weights for policy 1, policy_version 1525265 (0.0007) [2023-12-27 16:13:28,893][20381] Updated weights for policy 1, policy_version 1525275 (0.0006) [2023-12-27 16:13:28,906][20347] Updated weights for policy 0, policy_version 1526805 (0.0008) [2023-12-27 16:13:28,944][20381] Updated weights for policy 1, policy_version 1525285 (0.0005) [2023-12-27 16:13:28,961][20292] KL-divergence is very high: 100.9235 [2023-12-27 16:13:28,963][20347] Updated weights for policy 0, policy_version 1526815 (0.0010) [2023-12-27 16:13:28,973][20292] KL-divergence is very high: 116.7177 [2023-12-27 16:13:28,978][20292] KL-divergence is very high: 120.1411 [2023-12-27 16:13:28,998][20381] Updated weights for policy 1, policy_version 1525295 (0.0005) [2023-12-27 16:13:29,021][20347] Updated weights for policy 0, policy_version 1526825 (0.0010) [2023-12-27 16:13:29,570][20381] Updated weights for policy 1, policy_version 1525305 (0.0006) [2023-12-27 16:13:29,619][20381] Updated weights for policy 1, policy_version 1525315 (0.0007) [2023-12-27 16:13:29,664][20381] Updated weights for policy 1, policy_version 1525325 (0.0007) [2023-12-27 16:13:29,683][20347] Updated weights for policy 0, policy_version 1526835 (0.0008) [2023-12-27 16:13:29,727][20347] Updated weights for policy 0, policy_version 1526845 (0.0005) [2023-12-27 16:13:29,782][20347] Updated weights for policy 0, policy_version 1526855 (0.0006) [2023-12-27 16:13:30,400][20381] Updated weights for policy 1, policy_version 1525335 (0.0008) [2023-12-27 16:13:30,459][20381] Updated weights for policy 1, policy_version 1525345 (0.0007) [2023-12-27 16:13:30,470][20347] Updated weights for policy 0, policy_version 1526865 (0.0007) [2023-12-27 16:13:30,521][20381] Updated weights for policy 1, policy_version 1525355 (0.0007) [2023-12-27 16:13:30,531][20347] Updated weights for policy 0, policy_version 1526875 (0.0006) [2023-12-27 16:13:30,597][20347] Updated weights for policy 0, policy_version 1526885 (0.0006) [2023-12-27 16:13:30,653][20347] Updated weights for policy 0, policy_version 1526895 (0.0008) [2023-12-27 16:13:31,215][20381] Updated weights for policy 1, policy_version 1525365 (0.0007) [2023-12-27 16:13:31,276][20381] Updated weights for policy 1, policy_version 1525375 (0.0008) [2023-12-27 16:13:31,334][20381] Updated weights for policy 1, policy_version 1525385 (0.0008) [2023-12-27 16:13:31,375][20347] Updated weights for policy 0, policy_version 1526905 (0.0008) [2023-12-27 16:13:31,433][20347] Updated weights for policy 0, policy_version 1526915 (0.0008) [2023-12-27 16:13:31,496][20347] Updated weights for policy 0, policy_version 1526925 (0.0007) [2023-12-27 16:13:32,014][20381] Updated weights for policy 1, policy_version 1525395 (0.0008) [2023-12-27 16:13:32,071][20381] Updated weights for policy 1, policy_version 1525405 (0.0006) [2023-12-27 16:13:32,135][20381] Updated weights for policy 1, policy_version 1525415 (0.0006) [2023-12-27 16:13:32,198][20347] Updated weights for policy 0, policy_version 1526935 (0.0007) [2023-12-27 16:13:32,250][20347] Updated weights for policy 0, policy_version 1526945 (0.0008) [2023-12-27 16:13:32,305][20347] Updated weights for policy 0, policy_version 1526955 (0.0009) [2023-12-27 16:13:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 781524992. Throughput: 0: 10273.9, 1: 10434.0. Samples: 781529468. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:32,363][19170] Avg episode reward: [(0, '970.900'), (1, '834.360')] [2023-12-27 16:13:32,755][20381] Updated weights for policy 1, policy_version 1525425 (0.0006) [2023-12-27 16:13:32,812][20381] Updated weights for policy 1, policy_version 1525435 (0.0005) [2023-12-27 16:13:32,874][20381] Updated weights for policy 1, policy_version 1525445 (0.0007) [2023-12-27 16:13:32,932][20381] Updated weights for policy 1, policy_version 1525455 (0.0008) [2023-12-27 16:13:33,016][20347] Updated weights for policy 0, policy_version 1526965 (0.0008) [2023-12-27 16:13:33,076][20347] Updated weights for policy 0, policy_version 1526975 (0.0008) [2023-12-27 16:13:33,124][20347] Updated weights for policy 0, policy_version 1526985 (0.0007) [2023-12-27 16:13:33,621][20381] Updated weights for policy 1, policy_version 1525465 (0.0007) [2023-12-27 16:13:33,675][20381] Updated weights for policy 1, policy_version 1525475 (0.0007) [2023-12-27 16:13:33,717][20347] Updated weights for policy 0, policy_version 1526995 (0.0008) [2023-12-27 16:13:33,723][20381] Updated weights for policy 1, policy_version 1525485 (0.0006) [2023-12-27 16:13:33,770][20347] Updated weights for policy 0, policy_version 1527005 (0.0010) [2023-12-27 16:13:33,824][20347] Updated weights for policy 0, policy_version 1527015 (0.0010) [2023-12-27 16:13:34,377][20347] Updated weights for policy 0, policy_version 1527025 (0.0009) [2023-12-27 16:13:34,437][20347] Updated weights for policy 0, policy_version 1527035 (0.0009) [2023-12-27 16:13:34,447][20381] Updated weights for policy 1, policy_version 1525495 (0.0006) [2023-12-27 16:13:34,494][20347] Updated weights for policy 0, policy_version 1527045 (0.0010) [2023-12-27 16:13:34,503][20381] Updated weights for policy 1, policy_version 1525505 (0.0005) [2023-12-27 16:13:34,546][20347] Updated weights for policy 0, policy_version 1527055 (0.0009) [2023-12-27 16:13:34,565][20381] Updated weights for policy 1, policy_version 1525515 (0.0006) [2023-12-27 16:13:35,111][20347] Updated weights for policy 0, policy_version 1527065 (0.0006) [2023-12-27 16:13:35,164][20347] Updated weights for policy 0, policy_version 1527075 (0.0005) [2023-12-27 16:13:35,220][20347] Updated weights for policy 0, policy_version 1527085 (0.0005) [2023-12-27 16:13:35,243][20381] Updated weights for policy 1, policy_version 1525525 (0.0009) [2023-12-27 16:13:35,289][20381] Updated weights for policy 1, policy_version 1525535 (0.0010) [2023-12-27 16:13:35,339][20381] Updated weights for policy 1, policy_version 1525545 (0.0009) [2023-12-27 16:13:35,847][20347] Updated weights for policy 0, policy_version 1527095 (0.0008) [2023-12-27 16:13:35,912][20347] Updated weights for policy 0, policy_version 1527105 (0.0007) [2023-12-27 16:13:35,970][20381] Updated weights for policy 1, policy_version 1525555 (0.0007) [2023-12-27 16:13:35,973][20347] Updated weights for policy 0, policy_version 1527115 (0.0008) [2023-12-27 16:13:36,027][20381] Updated weights for policy 1, policy_version 1525565 (0.0010) [2023-12-27 16:13:36,088][20381] Updated weights for policy 1, policy_version 1525575 (0.0010) [2023-12-27 16:13:36,563][20347] Updated weights for policy 0, policy_version 1527125 (0.0006) [2023-12-27 16:13:36,626][20347] Updated weights for policy 0, policy_version 1527135 (0.0010) [2023-12-27 16:13:36,691][20347] Updated weights for policy 0, policy_version 1527145 (0.0010) [2023-12-27 16:13:36,786][20381] Updated weights for policy 1, policy_version 1525585 (0.0009) [2023-12-27 16:13:36,837][20381] Updated weights for policy 1, policy_version 1525595 (0.0008) [2023-12-27 16:13:36,897][20381] Updated weights for policy 1, policy_version 1525605 (0.0010) [2023-12-27 16:13:36,949][20381] Updated weights for policy 1, policy_version 1525615 (0.0010) [2023-12-27 16:13:37,251][20347] Updated weights for policy 0, policy_version 1527155 (0.0009) [2023-12-27 16:13:37,307][20347] Updated weights for policy 0, policy_version 1527165 (0.0006) [2023-12-27 16:13:37,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 781623296. Throughput: 0: 10329.5, 1: 10454.0. Samples: 781593700. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:37,363][19170] Avg episode reward: [(0, '970.970'), (1, '922.150')] [2023-12-27 16:13:37,364][20347] Updated weights for policy 0, policy_version 1527175 (0.0008) [2023-12-27 16:13:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001525616_390594560.pth... [2023-12-27 16:13:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001524400_390283264.pth [2023-12-27 16:13:37,414][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001527184_391036928.pth... [2023-12-27 16:13:37,418][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001525944_390717440.pth [2023-12-27 16:13:37,647][20381] Updated weights for policy 1, policy_version 1525625 (0.0010) [2023-12-27 16:13:37,699][20381] Updated weights for policy 1, policy_version 1525635 (0.0010) [2023-12-27 16:13:37,757][20381] Updated weights for policy 1, policy_version 1525645 (0.0010) [2023-12-27 16:13:38,010][20347] Updated weights for policy 0, policy_version 1527185 (0.0010) [2023-12-27 16:13:38,062][20347] Updated weights for policy 0, policy_version 1527195 (0.0007) [2023-12-27 16:13:38,117][20347] Updated weights for policy 0, policy_version 1527205 (0.0007) [2023-12-27 16:13:38,165][20347] Updated weights for policy 0, policy_version 1527215 (0.0007) [2023-12-27 16:13:38,464][20381] Updated weights for policy 1, policy_version 1525655 (0.0009) [2023-12-27 16:13:38,526][20381] Updated weights for policy 1, policy_version 1525665 (0.0010) [2023-12-27 16:13:38,578][20381] Updated weights for policy 1, policy_version 1525675 (0.0010) [2023-12-27 16:13:38,864][20347] Updated weights for policy 0, policy_version 1527225 (0.0007) [2023-12-27 16:13:38,930][20347] Updated weights for policy 0, policy_version 1527235 (0.0010) [2023-12-27 16:13:38,997][20347] Updated weights for policy 0, policy_version 1527245 (0.0008) [2023-12-27 16:13:39,260][20381] Updated weights for policy 1, policy_version 1525685 (0.0009) [2023-12-27 16:13:39,324][20381] Updated weights for policy 1, policy_version 1525695 (0.0007) [2023-12-27 16:13:39,393][20381] Updated weights for policy 1, policy_version 1525705 (0.0009) [2023-12-27 16:13:39,681][20347] Updated weights for policy 0, policy_version 1527255 (0.0009) [2023-12-27 16:13:39,699][20291] KL-divergence is very high: 231.8676 [2023-12-27 16:13:39,743][20347] Updated weights for policy 0, policy_version 1527265 (0.0008) [2023-12-27 16:13:39,747][20291] KL-divergence is very high: 260.8402 [2023-12-27 16:13:39,792][20291] KL-divergence is very high: 203.0742 [2023-12-27 16:13:39,800][20347] Updated weights for policy 0, policy_version 1527275 (0.0008) [2023-12-27 16:13:40,115][20381] Updated weights for policy 1, policy_version 1525715 (0.0008) [2023-12-27 16:13:40,171][20381] Updated weights for policy 1, policy_version 1525725 (0.0007) [2023-12-27 16:13:40,230][20381] Updated weights for policy 1, policy_version 1525735 (0.0006) [2023-12-27 16:13:40,487][20347] Updated weights for policy 0, policy_version 1527285 (0.0007) [2023-12-27 16:13:40,546][20347] Updated weights for policy 0, policy_version 1527295 (0.0008) [2023-12-27 16:13:40,602][20347] Updated weights for policy 0, policy_version 1527305 (0.0006) [2023-12-27 16:13:40,860][20381] Updated weights for policy 1, policy_version 1525745 (0.0008) [2023-12-27 16:13:40,913][20381] Updated weights for policy 1, policy_version 1525755 (0.0005) [2023-12-27 16:13:40,962][20381] Updated weights for policy 1, policy_version 1525765 (0.0005) [2023-12-27 16:13:41,013][20381] Updated weights for policy 1, policy_version 1525775 (0.0006) [2023-12-27 16:13:41,274][20347] Updated weights for policy 0, policy_version 1527315 (0.0007) [2023-12-27 16:13:41,335][20347] Updated weights for policy 0, policy_version 1527325 (0.0011) [2023-12-27 16:13:41,426][20347] Updated weights for policy 0, policy_version 1527335 (0.0008) [2023-12-27 16:13:41,690][20381] Updated weights for policy 1, policy_version 1525785 (0.0008) [2023-12-27 16:13:41,763][20381] Updated weights for policy 1, policy_version 1525795 (0.0008) [2023-12-27 16:13:41,823][20381] Updated weights for policy 1, policy_version 1525805 (0.0009) [2023-12-27 16:13:42,115][20347] Updated weights for policy 0, policy_version 1527345 (0.0010) [2023-12-27 16:13:42,179][20347] Updated weights for policy 0, policy_version 1527355 (0.0010) [2023-12-27 16:13:42,241][20347] Updated weights for policy 0, policy_version 1527365 (0.0010) [2023-12-27 16:13:42,308][20347] Updated weights for policy 0, policy_version 1527375 (0.0010) [2023-12-27 16:13:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 781729792. Throughput: 0: 10440.5, 1: 10374.2. Samples: 781719244. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:42,363][19170] Avg episode reward: [(0, '941.670'), (1, '843.970')] [2023-12-27 16:13:42,525][20381] Updated weights for policy 1, policy_version 1525815 (0.0006) [2023-12-27 16:13:42,584][20381] Updated weights for policy 1, policy_version 1525825 (0.0010) [2023-12-27 16:13:42,642][20381] Updated weights for policy 1, policy_version 1525835 (0.0010) [2023-12-27 16:13:42,933][20347] Updated weights for policy 0, policy_version 1527385 (0.0006) [2023-12-27 16:13:42,985][20347] Updated weights for policy 0, policy_version 1527395 (0.0005) [2023-12-27 16:13:43,034][20347] Updated weights for policy 0, policy_version 1527405 (0.0007) [2023-12-27 16:13:43,334][20381] Updated weights for policy 1, policy_version 1525845 (0.0009) [2023-12-27 16:13:43,394][20381] Updated weights for policy 1, policy_version 1525855 (0.0007) [2023-12-27 16:13:43,455][20381] Updated weights for policy 1, policy_version 1525865 (0.0008) [2023-12-27 16:13:43,629][20347] Updated weights for policy 0, policy_version 1527415 (0.0010) [2023-12-27 16:13:43,688][20347] Updated weights for policy 0, policy_version 1527425 (0.0009) [2023-12-27 16:13:43,745][20347] Updated weights for policy 0, policy_version 1527435 (0.0010) [2023-12-27 16:13:44,091][20381] Updated weights for policy 1, policy_version 1525875 (0.0006) [2023-12-27 16:13:44,142][20381] Updated weights for policy 1, policy_version 1525885 (0.0005) [2023-12-27 16:13:44,202][20381] Updated weights for policy 1, policy_version 1525895 (0.0005) [2023-12-27 16:13:44,355][20347] Updated weights for policy 0, policy_version 1527445 (0.0010) [2023-12-27 16:13:44,410][20347] Updated weights for policy 0, policy_version 1527455 (0.0010) [2023-12-27 16:13:44,461][20347] Updated weights for policy 0, policy_version 1527465 (0.0010) [2023-12-27 16:13:44,812][20381] Updated weights for policy 1, policy_version 1525905 (0.0007) [2023-12-27 16:13:44,873][20381] Updated weights for policy 1, policy_version 1525915 (0.0007) [2023-12-27 16:13:44,925][20381] Updated weights for policy 1, policy_version 1525925 (0.0007) [2023-12-27 16:13:44,975][20381] Updated weights for policy 1, policy_version 1525935 (0.0008) [2023-12-27 16:13:45,201][20347] Updated weights for policy 0, policy_version 1527475 (0.0009) [2023-12-27 16:13:45,256][20347] Updated weights for policy 0, policy_version 1527485 (0.0008) [2023-12-27 16:13:45,310][20347] Updated weights for policy 0, policy_version 1527495 (0.0008) [2023-12-27 16:13:45,579][20381] Updated weights for policy 1, policy_version 1525945 (0.0006) [2023-12-27 16:13:45,639][20381] Updated weights for policy 1, policy_version 1525955 (0.0006) [2023-12-27 16:13:45,696][20381] Updated weights for policy 1, policy_version 1525965 (0.0005) [2023-12-27 16:13:46,052][20347] Updated weights for policy 0, policy_version 1527506 (0.0010) [2023-12-27 16:13:46,111][20347] Updated weights for policy 0, policy_version 1527516 (0.0008) [2023-12-27 16:13:46,169][20347] Updated weights for policy 0, policy_version 1527526 (0.0008) [2023-12-27 16:13:46,228][20347] Updated weights for policy 0, policy_version 1527536 (0.0008) [2023-12-27 16:13:46,299][20381] Updated weights for policy 1, policy_version 1525975 (0.0006) [2023-12-27 16:13:46,356][20381] Updated weights for policy 1, policy_version 1525985 (0.0007) [2023-12-27 16:13:46,418][20381] Updated weights for policy 1, policy_version 1525995 (0.0007) [2023-12-27 16:13:46,988][20347] Updated weights for policy 0, policy_version 1527546 (0.0008) [2023-12-27 16:13:47,047][20347] Updated weights for policy 0, policy_version 1527556 (0.0008) [2023-12-27 16:13:47,049][20381] Updated weights for policy 1, policy_version 1526005 (0.0006) [2023-12-27 16:13:47,102][20381] Updated weights for policy 1, policy_version 1526015 (0.0005) [2023-12-27 16:13:47,110][20347] Updated weights for policy 0, policy_version 1527566 (0.0008) [2023-12-27 16:13:47,159][20381] Updated weights for policy 1, policy_version 1526025 (0.0007) [2023-12-27 16:13:47,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 781836288. Throughput: 0: 10441.9, 1: 10409.8. Samples: 781843288. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:47,363][19170] Avg episode reward: [(0, '951.200'), (1, '883.020')] [2023-12-27 16:13:47,800][20347] Updated weights for policy 0, policy_version 1527576 (0.0007) [2023-12-27 16:13:47,819][20381] Updated weights for policy 1, policy_version 1526035 (0.0008) [2023-12-27 16:13:47,856][20347] Updated weights for policy 0, policy_version 1527586 (0.0005) [2023-12-27 16:13:47,874][20381] Updated weights for policy 1, policy_version 1526045 (0.0008) [2023-12-27 16:13:47,915][20347] Updated weights for policy 0, policy_version 1527596 (0.0005) [2023-12-27 16:13:47,932][20381] Updated weights for policy 1, policy_version 1526055 (0.0005) [2023-12-27 16:13:48,530][20381] Updated weights for policy 1, policy_version 1526065 (0.0008) [2023-12-27 16:13:48,564][20347] Updated weights for policy 0, policy_version 1527606 (0.0007) [2023-12-27 16:13:48,590][20381] Updated weights for policy 1, policy_version 1526075 (0.0009) [2023-12-27 16:13:48,620][20347] Updated weights for policy 0, policy_version 1527616 (0.0008) [2023-12-27 16:13:48,646][20381] Updated weights for policy 1, policy_version 1526085 (0.0011) [2023-12-27 16:13:48,675][20347] Updated weights for policy 0, policy_version 1527626 (0.0008) [2023-12-27 16:13:48,704][20381] Updated weights for policy 1, policy_version 1526095 (0.0010) [2023-12-27 16:13:49,302][20381] Updated weights for policy 1, policy_version 1526105 (0.0011) [2023-12-27 16:13:49,370][20381] Updated weights for policy 1, policy_version 1526115 (0.0010) [2023-12-27 16:13:49,391][20347] Updated weights for policy 0, policy_version 1527636 (0.0011) [2023-12-27 16:13:49,421][20381] Updated weights for policy 1, policy_version 1526125 (0.0009) [2023-12-27 16:13:49,451][20347] Updated weights for policy 0, policy_version 1527646 (0.0009) [2023-12-27 16:13:49,510][20347] Updated weights for policy 0, policy_version 1527656 (0.0008) [2023-12-27 16:13:50,099][20381] Updated weights for policy 1, policy_version 1526135 (0.0008) [2023-12-27 16:13:50,159][20381] Updated weights for policy 1, policy_version 1526145 (0.0007) [2023-12-27 16:13:50,201][20347] Updated weights for policy 0, policy_version 1527666 (0.0006) [2023-12-27 16:13:50,216][20381] Updated weights for policy 1, policy_version 1526155 (0.0008) [2023-12-27 16:13:50,256][20347] Updated weights for policy 0, policy_version 1527676 (0.0010) [2023-12-27 16:13:50,318][20347] Updated weights for policy 0, policy_version 1527686 (0.0010) [2023-12-27 16:13:50,379][20347] Updated weights for policy 0, policy_version 1527696 (0.0010) [2023-12-27 16:13:50,952][20381] Updated weights for policy 1, policy_version 1526165 (0.0007) [2023-12-27 16:13:51,009][20347] Updated weights for policy 0, policy_version 1527706 (0.0007) [2023-12-27 16:13:51,016][20381] Updated weights for policy 1, policy_version 1526175 (0.0006) [2023-12-27 16:13:51,065][20347] Updated weights for policy 0, policy_version 1527716 (0.0007) [2023-12-27 16:13:51,077][20381] Updated weights for policy 1, policy_version 1526185 (0.0007) [2023-12-27 16:13:51,130][20347] Updated weights for policy 0, policy_version 1527726 (0.0007) [2023-12-27 16:13:51,842][20381] Updated weights for policy 1, policy_version 1526195 (0.0007) [2023-12-27 16:13:51,868][20347] Updated weights for policy 0, policy_version 1527736 (0.0006) [2023-12-27 16:13:51,901][20381] Updated weights for policy 1, policy_version 1526205 (0.0007) [2023-12-27 16:13:51,929][20347] Updated weights for policy 0, policy_version 1527746 (0.0006) [2023-12-27 16:13:51,962][20381] Updated weights for policy 1, policy_version 1526215 (0.0008) [2023-12-27 16:13:51,986][20347] Updated weights for policy 0, policy_version 1527756 (0.0006) [2023-12-27 16:13:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 781934592. Throughput: 0: 10372.9, 1: 10432.5. Samples: 781905236. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:52,363][19170] Avg episode reward: [(0, '980.480'), (1, '922.000')] [2023-12-27 16:13:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001526224_390750208.pth... [2023-12-27 16:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001527760_391184384.pth... [2023-12-27 16:13:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001526544_390873088.pth [2023-12-27 16:13:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001525008_390438912.pth [2023-12-27 16:13:52,583][20347] Updated weights for policy 0, policy_version 1527766 (0.0007) [2023-12-27 16:13:52,645][20347] Updated weights for policy 0, policy_version 1527776 (0.0008) [2023-12-27 16:13:52,707][20347] Updated weights for policy 0, policy_version 1527786 (0.0007) [2023-12-27 16:13:52,709][20381] Updated weights for policy 1, policy_version 1526225 (0.0006) [2023-12-27 16:13:52,764][20381] Updated weights for policy 1, policy_version 1526235 (0.0007) [2023-12-27 16:13:52,824][20381] Updated weights for policy 1, policy_version 1526245 (0.0008) [2023-12-27 16:13:52,880][20381] Updated weights for policy 1, policy_version 1526255 (0.0008) [2023-12-27 16:13:53,365][20347] Updated weights for policy 0, policy_version 1527796 (0.0007) [2023-12-27 16:13:53,414][20347] Updated weights for policy 0, policy_version 1527806 (0.0005) [2023-12-27 16:13:53,468][20347] Updated weights for policy 0, policy_version 1527816 (0.0005) [2023-12-27 16:13:53,554][20381] Updated weights for policy 1, policy_version 1526265 (0.0008) [2023-12-27 16:13:53,616][20381] Updated weights for policy 1, policy_version 1526275 (0.0008) [2023-12-27 16:13:53,672][20381] Updated weights for policy 1, policy_version 1526285 (0.0008) [2023-12-27 16:13:54,094][20347] Updated weights for policy 0, policy_version 1527826 (0.0006) [2023-12-27 16:13:54,156][20347] Updated weights for policy 0, policy_version 1527836 (0.0005) [2023-12-27 16:13:54,212][20347] Updated weights for policy 0, policy_version 1527846 (0.0006) [2023-12-27 16:13:54,269][20347] Updated weights for policy 0, policy_version 1527856 (0.0008) [2023-12-27 16:13:54,399][20381] Updated weights for policy 1, policy_version 1526295 (0.0007) [2023-12-27 16:13:54,463][20381] Updated weights for policy 1, policy_version 1526305 (0.0007) [2023-12-27 16:13:54,527][20381] Updated weights for policy 1, policy_version 1526315 (0.0007) [2023-12-27 16:13:54,924][20347] Updated weights for policy 0, policy_version 1527866 (0.0010) [2023-12-27 16:13:54,977][20347] Updated weights for policy 0, policy_version 1527876 (0.0010) [2023-12-27 16:13:55,040][20347] Updated weights for policy 0, policy_version 1527886 (0.0011) [2023-12-27 16:13:55,202][20381] Updated weights for policy 1, policy_version 1526325 (0.0006) [2023-12-27 16:13:55,263][20381] Updated weights for policy 1, policy_version 1526335 (0.0007) [2023-12-27 16:13:55,320][20381] Updated weights for policy 1, policy_version 1526345 (0.0007) [2023-12-27 16:13:55,677][20347] Updated weights for policy 0, policy_version 1527896 (0.0009) [2023-12-27 16:13:55,723][20347] Updated weights for policy 0, policy_version 1527906 (0.0010) [2023-12-27 16:13:55,771][20347] Updated weights for policy 0, policy_version 1527916 (0.0010) [2023-12-27 16:13:55,907][20381] Updated weights for policy 1, policy_version 1526355 (0.0006) [2023-12-27 16:13:55,954][20381] Updated weights for policy 1, policy_version 1526365 (0.0005) [2023-12-27 16:13:56,004][20381] Updated weights for policy 1, policy_version 1526375 (0.0005) [2023-12-27 16:13:56,362][20347] Updated weights for policy 0, policy_version 1527926 (0.0007) [2023-12-27 16:13:56,423][20347] Updated weights for policy 0, policy_version 1527936 (0.0007) [2023-12-27 16:13:56,474][20347] Updated weights for policy 0, policy_version 1527946 (0.0010) [2023-12-27 16:13:56,591][20381] Updated weights for policy 1, policy_version 1526385 (0.0010) [2023-12-27 16:13:56,644][20381] Updated weights for policy 1, policy_version 1526395 (0.0005) [2023-12-27 16:13:56,692][20381] Updated weights for policy 1, policy_version 1526405 (0.0005) [2023-12-27 16:13:56,752][20381] Updated weights for policy 1, policy_version 1526415 (0.0006) [2023-12-27 16:13:57,132][20347] Updated weights for policy 0, policy_version 1527956 (0.0010) [2023-12-27 16:13:57,193][20347] Updated weights for policy 0, policy_version 1527966 (0.0010) [2023-12-27 16:13:57,254][20347] Updated weights for policy 0, policy_version 1527976 (0.0010) [2023-12-27 16:13:57,274][20381] Updated weights for policy 1, policy_version 1526425 (0.0005) [2023-12-27 16:13:57,328][20381] Updated weights for policy 1, policy_version 1526435 (0.0006) [2023-12-27 16:13:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 782041088. Throughput: 0: 10413.1, 1: 10347.4. Samples: 782027960. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:13:57,363][19170] Avg episode reward: [(0, '961.050'), (1, '912.310')] [2023-12-27 16:13:57,376][20381] Updated weights for policy 1, policy_version 1526445 (0.0007) [2023-12-27 16:13:57,872][20347] Updated weights for policy 0, policy_version 1527986 (0.0009) [2023-12-27 16:13:57,927][20347] Updated weights for policy 0, policy_version 1527996 (0.0007) [2023-12-27 16:13:57,988][20347] Updated weights for policy 0, policy_version 1528006 (0.0010) [2023-12-27 16:13:58,047][20347] Updated weights for policy 0, policy_version 1528016 (0.0009) [2023-12-27 16:13:58,065][20381] Updated weights for policy 1, policy_version 1526455 (0.0006) [2023-12-27 16:13:58,118][20381] Updated weights for policy 1, policy_version 1526465 (0.0005) [2023-12-27 16:13:58,166][20381] Updated weights for policy 1, policy_version 1526475 (0.0005) [2023-12-27 16:13:58,751][20347] Updated weights for policy 0, policy_version 1528026 (0.0008) [2023-12-27 16:13:58,828][20347] Updated weights for policy 0, policy_version 1528036 (0.0008) [2023-12-27 16:13:58,891][20347] Updated weights for policy 0, policy_version 1528046 (0.0008) [2023-12-27 16:13:58,945][20381] Updated weights for policy 1, policy_version 1526485 (0.0007) [2023-12-27 16:13:58,995][20381] Updated weights for policy 1, policy_version 1526495 (0.0008) [2023-12-27 16:13:59,058][20381] Updated weights for policy 1, policy_version 1526505 (0.0008) [2023-12-27 16:13:59,631][20347] Updated weights for policy 0, policy_version 1528056 (0.0009) [2023-12-27 16:13:59,689][20347] Updated weights for policy 0, policy_version 1528066 (0.0010) [2023-12-27 16:13:59,708][20381] Updated weights for policy 1, policy_version 1526515 (0.0008) [2023-12-27 16:13:59,741][20347] Updated weights for policy 0, policy_version 1528076 (0.0010) [2023-12-27 16:13:59,757][20381] Updated weights for policy 1, policy_version 1526525 (0.0005) [2023-12-27 16:13:59,812][20381] Updated weights for policy 1, policy_version 1526535 (0.0007) [2023-12-27 16:14:00,402][20347] Updated weights for policy 0, policy_version 1528086 (0.0010) [2023-12-27 16:14:00,458][20347] Updated weights for policy 0, policy_version 1528096 (0.0010) [2023-12-27 16:14:00,505][20347] Updated weights for policy 0, policy_version 1528106 (0.0008) [2023-12-27 16:14:00,561][20381] Updated weights for policy 1, policy_version 1526545 (0.0008) [2023-12-27 16:14:00,615][20381] Updated weights for policy 1, policy_version 1526555 (0.0007) [2023-12-27 16:14:00,679][20381] Updated weights for policy 1, policy_version 1526565 (0.0007) [2023-12-27 16:14:00,738][20381] Updated weights for policy 1, policy_version 1526575 (0.0006) [2023-12-27 16:14:01,177][20347] Updated weights for policy 0, policy_version 1528116 (0.0007) [2023-12-27 16:14:01,243][20347] Updated weights for policy 0, policy_version 1528126 (0.0008) [2023-12-27 16:14:01,299][20347] Updated weights for policy 0, policy_version 1528136 (0.0006) [2023-12-27 16:14:01,402][20381] Updated weights for policy 1, policy_version 1526585 (0.0008) [2023-12-27 16:14:01,454][20381] Updated weights for policy 1, policy_version 1526595 (0.0005) [2023-12-27 16:14:01,500][20381] Updated weights for policy 1, policy_version 1526605 (0.0005) [2023-12-27 16:14:01,986][20347] Updated weights for policy 0, policy_version 1528146 (0.0006) [2023-12-27 16:14:02,051][20347] Updated weights for policy 0, policy_version 1528156 (0.0006) [2023-12-27 16:14:02,101][20381] Updated weights for policy 1, policy_version 1526615 (0.0007) [2023-12-27 16:14:02,118][20347] Updated weights for policy 0, policy_version 1528166 (0.0006) [2023-12-27 16:14:02,154][20381] Updated weights for policy 1, policy_version 1526625 (0.0007) [2023-12-27 16:14:02,183][20347] Updated weights for policy 0, policy_version 1528176 (0.0006) [2023-12-27 16:14:02,215][20381] Updated weights for policy 1, policy_version 1526635 (0.0007) [2023-12-27 16:14:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 782147584. Throughput: 0: 10377.0, 1: 10333.1. Samples: 782153464. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:14:02,363][19170] Avg episode reward: [(0, '970.780'), (1, '970.790')] [2023-12-27 16:14:02,778][20347] Updated weights for policy 0, policy_version 1528186 (0.0005) [2023-12-27 16:14:02,837][20347] Updated weights for policy 0, policy_version 1528196 (0.0007) [2023-12-27 16:14:02,901][20347] Updated weights for policy 0, policy_version 1528206 (0.0007) [2023-12-27 16:14:02,929][20381] Updated weights for policy 1, policy_version 1526645 (0.0008) [2023-12-27 16:14:02,991][20381] Updated weights for policy 1, policy_version 1526655 (0.0008) [2023-12-27 16:14:03,045][20381] Updated weights for policy 1, policy_version 1526665 (0.0007) [2023-12-27 16:14:03,568][20347] Updated weights for policy 0, policy_version 1528216 (0.0009) [2023-12-27 16:14:03,619][20347] Updated weights for policy 0, policy_version 1528226 (0.0010) [2023-12-27 16:14:03,671][20347] Updated weights for policy 0, policy_version 1528236 (0.0010) [2023-12-27 16:14:03,703][20381] Updated weights for policy 1, policy_version 1526675 (0.0008) [2023-12-27 16:14:03,757][20381] Updated weights for policy 1, policy_version 1526685 (0.0007) [2023-12-27 16:14:03,812][20381] Updated weights for policy 1, policy_version 1526695 (0.0007) [2023-12-27 16:14:03,815][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 16:14:04,368][20347] Updated weights for policy 0, policy_version 1528246 (0.0009) [2023-12-27 16:14:04,418][20347] Updated weights for policy 0, policy_version 1528256 (0.0008) [2023-12-27 16:14:04,467][20347] Updated weights for policy 0, policy_version 1528266 (0.0008) [2023-12-27 16:14:04,565][20381] Updated weights for policy 1, policy_version 1526705 (0.0007) [2023-12-27 16:14:04,621][20381] Updated weights for policy 1, policy_version 1526715 (0.0007) [2023-12-27 16:14:04,663][20381] Updated weights for policy 1, policy_version 1526725 (0.0005) [2023-12-27 16:14:05,134][20347] Updated weights for policy 0, policy_version 1528276 (0.0010) [2023-12-27 16:14:05,197][20347] Updated weights for policy 0, policy_version 1528286 (0.0010) [2023-12-27 16:14:05,252][20347] Updated weights for policy 0, policy_version 1528296 (0.0010) [2023-12-27 16:14:05,345][20381] Updated weights for policy 1, policy_version 1526735 (0.0010) [2023-12-27 16:14:05,393][20381] Updated weights for policy 1, policy_version 1526745 (0.0010) [2023-12-27 16:14:05,444][20381] Updated weights for policy 1, policy_version 1526755 (0.0010) [2023-12-27 16:14:05,929][20347] Updated weights for policy 0, policy_version 1528306 (0.0010) [2023-12-27 16:14:05,980][20347] Updated weights for policy 0, policy_version 1528316 (0.0010) [2023-12-27 16:14:06,031][20347] Updated weights for policy 0, policy_version 1528326 (0.0010) [2023-12-27 16:14:06,087][20347] Updated weights for policy 0, policy_version 1528336 (0.0009) [2023-12-27 16:14:06,098][20381] Updated weights for policy 1, policy_version 1526765 (0.0008) [2023-12-27 16:14:06,164][20381] Updated weights for policy 1, policy_version 1526775 (0.0008) [2023-12-27 16:14:06,214][20381] Updated weights for policy 1, policy_version 1526785 (0.0011) [2023-12-27 16:14:06,768][20347] Updated weights for policy 0, policy_version 1528346 (0.0007) [2023-12-27 16:14:06,817][20347] Updated weights for policy 0, policy_version 1528356 (0.0005) [2023-12-27 16:14:06,839][20381] Updated weights for policy 1, policy_version 1526795 (0.0009) [2023-12-27 16:14:06,875][20347] Updated weights for policy 0, policy_version 1528366 (0.0005) [2023-12-27 16:14:06,897][20381] Updated weights for policy 1, policy_version 1526805 (0.0006) [2023-12-27 16:14:06,961][20381] Updated weights for policy 1, policy_version 1526815 (0.0005) [2023-12-27 16:14:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 782245888. Throughput: 0: 10394.1, 1: 10330.5. Samples: 782214828. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:14:07,363][19170] Avg episode reward: [(0, '951.210'), (1, '941.510')] [2023-12-27 16:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001528368_391340032.pth... [2023-12-27 16:14:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001526824_390905856.pth... [2023-12-27 16:14:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001527184_391036928.pth [2023-12-27 16:14:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001525616_390594560.pth [2023-12-27 16:14:07,479][20347] Updated weights for policy 0, policy_version 1528376 (0.0007) [2023-12-27 16:14:07,538][20347] Updated weights for policy 0, policy_version 1528386 (0.0008) [2023-12-27 16:14:07,544][20381] Updated weights for policy 1, policy_version 1526825 (0.0006) [2023-12-27 16:14:07,587][20347] Updated weights for policy 0, policy_version 1528396 (0.0005) [2023-12-27 16:14:07,599][20381] Updated weights for policy 1, policy_version 1526835 (0.0011) [2023-12-27 16:14:07,657][20381] Updated weights for policy 1, policy_version 1526845 (0.0010) [2023-12-27 16:14:07,719][20381] Updated weights for policy 1, policy_version 1526855 (0.0011) [2023-12-27 16:14:08,269][20347] Updated weights for policy 0, policy_version 1528406 (0.0008) [2023-12-27 16:14:08,318][20347] Updated weights for policy 0, policy_version 1528416 (0.0008) [2023-12-27 16:14:08,376][20381] Updated weights for policy 1, policy_version 1526865 (0.0008) [2023-12-27 16:14:08,383][20347] Updated weights for policy 0, policy_version 1528426 (0.0011) [2023-12-27 16:14:08,428][20381] Updated weights for policy 1, policy_version 1526875 (0.0006) [2023-12-27 16:14:08,486][20381] Updated weights for policy 1, policy_version 1526885 (0.0006) [2023-12-27 16:14:09,046][20347] Updated weights for policy 0, policy_version 1528436 (0.0009) [2023-12-27 16:14:09,094][20381] Updated weights for policy 1, policy_version 1526895 (0.0009) [2023-12-27 16:14:09,102][20347] Updated weights for policy 0, policy_version 1528446 (0.0010) [2023-12-27 16:14:09,149][20381] Updated weights for policy 1, policy_version 1526905 (0.0007) [2023-12-27 16:14:09,158][20347] Updated weights for policy 0, policy_version 1528456 (0.0010) [2023-12-27 16:14:09,199][20381] Updated weights for policy 1, policy_version 1526915 (0.0006) [2023-12-27 16:14:09,938][20347] Updated weights for policy 0, policy_version 1528466 (0.0010) [2023-12-27 16:14:09,939][20381] Updated weights for policy 1, policy_version 1526925 (0.0009) [2023-12-27 16:14:09,993][20381] Updated weights for policy 1, policy_version 1526935 (0.0008) [2023-12-27 16:14:09,998][20347] Updated weights for policy 0, policy_version 1528476 (0.0011) [2023-12-27 16:14:10,051][20381] Updated weights for policy 1, policy_version 1526945 (0.0010) [2023-12-27 16:14:10,060][20347] Updated weights for policy 0, policy_version 1528486 (0.0009) [2023-12-27 16:14:10,114][20347] Updated weights for policy 0, policy_version 1528496 (0.0006) [2023-12-27 16:14:10,734][20347] Updated weights for policy 0, policy_version 1528506 (0.0005) [2023-12-27 16:14:10,783][20347] Updated weights for policy 0, policy_version 1528516 (0.0005) [2023-12-27 16:14:10,814][20381] Updated weights for policy 1, policy_version 1526955 (0.0010) [2023-12-27 16:14:10,838][20347] Updated weights for policy 0, policy_version 1528526 (0.0005) [2023-12-27 16:14:10,870][20381] Updated weights for policy 1, policy_version 1526965 (0.0008) [2023-12-27 16:14:10,920][20381] Updated weights for policy 1, policy_version 1526975 (0.0007) [2023-12-27 16:14:11,485][20347] Updated weights for policy 0, policy_version 1528536 (0.0007) [2023-12-27 16:14:11,550][20347] Updated weights for policy 0, policy_version 1528546 (0.0008) [2023-12-27 16:14:11,618][20347] Updated weights for policy 0, policy_version 1528556 (0.0007) [2023-12-27 16:14:11,670][20381] Updated weights for policy 1, policy_version 1526985 (0.0007) [2023-12-27 16:14:11,741][20381] Updated weights for policy 1, policy_version 1526995 (0.0008) [2023-12-27 16:14:11,798][20381] Updated weights for policy 1, policy_version 1527005 (0.0008) [2023-12-27 16:14:11,852][20381] Updated weights for policy 1, policy_version 1527015 (0.0008) [2023-12-27 16:14:12,321][20347] Updated weights for policy 0, policy_version 1528566 (0.0007) [2023-12-27 16:14:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 782344192. Throughput: 0: 10417.5, 1: 10369.3. Samples: 782339260. Policy #0 lag: (min: 31.0, avg: 33.0, max: 63.0) [2023-12-27 16:14:12,363][19170] Avg episode reward: [(0, '931.800'), (1, '912.260')] [2023-12-27 16:14:12,383][20347] Updated weights for policy 0, policy_version 1528576 (0.0008) [2023-12-27 16:14:12,445][20347] Updated weights for policy 0, policy_version 1528586 (0.0008) [2023-12-27 16:14:12,576][20381] Updated weights for policy 1, policy_version 1527025 (0.0007) [2023-12-27 16:14:12,633][20381] Updated weights for policy 1, policy_version 1527035 (0.0008) [2023-12-27 16:14:12,686][20381] Updated weights for policy 1, policy_version 1527045 (0.0008) [2023-12-27 16:14:13,172][20347] Updated weights for policy 0, policy_version 1528596 (0.0007) [2023-12-27 16:14:13,226][20347] Updated weights for policy 0, policy_version 1528606 (0.0007) [2023-12-27 16:14:13,277][20347] Updated weights for policy 0, policy_version 1528616 (0.0007) [2023-12-27 16:14:13,319][20381] Updated weights for policy 1, policy_version 1527055 (0.0006) [2023-12-27 16:14:13,381][20381] Updated weights for policy 1, policy_version 1527065 (0.0006) [2023-12-27 16:14:13,435][20381] Updated weights for policy 1, policy_version 1527075 (0.0007) [2023-12-27 16:14:13,986][20381] Updated weights for policy 1, policy_version 1527085 (0.0007) [2023-12-27 16:14:13,991][20347] Updated weights for policy 0, policy_version 1528626 (0.0008) [2023-12-27 16:14:14,044][20381] Updated weights for policy 1, policy_version 1527095 (0.0006) [2023-12-27 16:14:14,046][20347] Updated weights for policy 0, policy_version 1528636 (0.0007) [2023-12-27 16:14:14,103][20381] Updated weights for policy 1, policy_version 1527105 (0.0005) [2023-12-27 16:14:14,110][20347] Updated weights for policy 0, policy_version 1528646 (0.0007) [2023-12-27 16:14:14,167][20347] Updated weights for policy 0, policy_version 1528656 (0.0007) [2023-12-27 16:14:14,763][20381] Updated weights for policy 1, policy_version 1527115 (0.0007) [2023-12-27 16:14:14,807][20347] Updated weights for policy 0, policy_version 1528666 (0.0007) [2023-12-27 16:14:14,831][20381] Updated weights for policy 1, policy_version 1527125 (0.0007) [2023-12-27 16:14:14,869][20347] Updated weights for policy 0, policy_version 1528676 (0.0009) [2023-12-27 16:14:14,893][20381] Updated weights for policy 1, policy_version 1527135 (0.0006) [2023-12-27 16:14:14,926][20347] Updated weights for policy 0, policy_version 1528686 (0.0008) [2023-12-27 16:14:15,562][20347] Updated weights for policy 0, policy_version 1528696 (0.0005) [2023-12-27 16:14:15,614][20347] Updated weights for policy 0, policy_version 1528706 (0.0006) [2023-12-27 16:14:15,616][20381] Updated weights for policy 1, policy_version 1527145 (0.0007) [2023-12-27 16:14:15,668][20381] Updated weights for policy 1, policy_version 1527155 (0.0005) [2023-12-27 16:14:15,677][20347] Updated weights for policy 0, policy_version 1528716 (0.0007) [2023-12-27 16:14:15,717][20381] Updated weights for policy 1, policy_version 1527165 (0.0006) [2023-12-27 16:14:15,770][20381] Updated weights for policy 1, policy_version 1527175 (0.0008) [2023-12-27 16:14:16,341][20347] Updated weights for policy 0, policy_version 1528726 (0.0009) [2023-12-27 16:14:16,397][20347] Updated weights for policy 0, policy_version 1528736 (0.0010) [2023-12-27 16:14:16,459][20347] Updated weights for policy 0, policy_version 1528746 (0.0010) [2023-12-27 16:14:16,479][20381] Updated weights for policy 1, policy_version 1527185 (0.0006) [2023-12-27 16:14:16,533][20381] Updated weights for policy 1, policy_version 1527195 (0.0006) [2023-12-27 16:14:16,591][20381] Updated weights for policy 1, policy_version 1527205 (0.0005) [2023-12-27 16:14:17,135][20347] Updated weights for policy 0, policy_version 1528756 (0.0009) [2023-12-27 16:14:17,162][20381] Updated weights for policy 1, policy_version 1527215 (0.0005) [2023-12-27 16:14:17,179][20347] Updated weights for policy 0, policy_version 1528766 (0.0007) [2023-12-27 16:14:17,222][20381] Updated weights for policy 1, policy_version 1527225 (0.0006) [2023-12-27 16:14:17,234][20347] Updated weights for policy 0, policy_version 1528776 (0.0007) [2023-12-27 16:14:17,272][20381] Updated weights for policy 1, policy_version 1527235 (0.0007) [2023-12-27 16:14:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 782458880. Throughput: 0: 10395.3, 1: 10343.9. Samples: 782462732. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:17,363][19170] Avg episode reward: [(0, '931.700'), (1, '922.410')] [2023-12-27 16:14:17,901][20347] Updated weights for policy 0, policy_version 1528786 (0.0007) [2023-12-27 16:14:17,958][20347] Updated weights for policy 0, policy_version 1528796 (0.0005) [2023-12-27 16:14:17,995][20381] Updated weights for policy 1, policy_version 1527245 (0.0007) [2023-12-27 16:14:18,026][20347] Updated weights for policy 0, policy_version 1528806 (0.0006) [2023-12-27 16:14:18,048][20381] Updated weights for policy 1, policy_version 1527255 (0.0006) [2023-12-27 16:14:18,080][20347] Updated weights for policy 0, policy_version 1528816 (0.0007) [2023-12-27 16:14:18,103][20381] Updated weights for policy 1, policy_version 1527265 (0.0006) [2023-12-27 16:14:18,743][20347] Updated weights for policy 0, policy_version 1528826 (0.0008) [2023-12-27 16:14:18,811][20347] Updated weights for policy 0, policy_version 1528836 (0.0008) [2023-12-27 16:14:18,813][20381] Updated weights for policy 1, policy_version 1527275 (0.0007) [2023-12-27 16:14:18,862][20381] Updated weights for policy 1, policy_version 1527285 (0.0006) [2023-12-27 16:14:18,870][20347] Updated weights for policy 0, policy_version 1528846 (0.0010) [2023-12-27 16:14:18,920][20381] Updated weights for policy 1, policy_version 1527295 (0.0009) [2023-12-27 16:14:19,528][20347] Updated weights for policy 0, policy_version 1528856 (0.0010) [2023-12-27 16:14:19,589][20347] Updated weights for policy 0, policy_version 1528866 (0.0010) [2023-12-27 16:14:19,632][20381] Updated weights for policy 1, policy_version 1527305 (0.0009) [2023-12-27 16:14:19,639][20347] Updated weights for policy 0, policy_version 1528876 (0.0008) [2023-12-27 16:14:19,685][20381] Updated weights for policy 1, policy_version 1527315 (0.0006) [2023-12-27 16:14:19,743][20381] Updated weights for policy 1, policy_version 1527325 (0.0005) [2023-12-27 16:14:19,798][20381] Updated weights for policy 1, policy_version 1527335 (0.0006) [2023-12-27 16:14:19,803][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 16:14:20,307][20347] Updated weights for policy 0, policy_version 1528886 (0.0007) [2023-12-27 16:14:20,371][20347] Updated weights for policy 0, policy_version 1528896 (0.0006) [2023-12-27 16:14:20,432][20347] Updated weights for policy 0, policy_version 1528906 (0.0010) [2023-12-27 16:14:20,541][20381] Updated weights for policy 1, policy_version 1527345 (0.0008) [2023-12-27 16:14:20,606][20381] Updated weights for policy 1, policy_version 1527355 (0.0010) [2023-12-27 16:14:20,657][20381] Updated weights for policy 1, policy_version 1527365 (0.0007) [2023-12-27 16:14:21,198][20347] Updated weights for policy 0, policy_version 1528916 (0.0010) [2023-12-27 16:14:21,264][20381] Updated weights for policy 1, policy_version 1527375 (0.0007) [2023-12-27 16:14:21,268][20347] Updated weights for policy 0, policy_version 1528926 (0.0008) [2023-12-27 16:14:21,324][20381] Updated weights for policy 1, policy_version 1527385 (0.0007) [2023-12-27 16:14:21,335][20347] Updated weights for policy 0, policy_version 1528936 (0.0006) [2023-12-27 16:14:21,384][20381] Updated weights for policy 1, policy_version 1527395 (0.0007) [2023-12-27 16:14:22,036][20347] Updated weights for policy 0, policy_version 1528946 (0.0007) [2023-12-27 16:14:22,081][20381] Updated weights for policy 1, policy_version 1527405 (0.0006) [2023-12-27 16:14:22,087][20347] Updated weights for policy 0, policy_version 1528956 (0.0007) [2023-12-27 16:14:22,136][20381] Updated weights for policy 1, policy_version 1527415 (0.0006) [2023-12-27 16:14:22,143][20347] Updated weights for policy 0, policy_version 1528966 (0.0008) [2023-12-27 16:14:22,195][20347] Updated weights for policy 0, policy_version 1528976 (0.0007) [2023-12-27 16:14:22,216][20381] Updated weights for policy 1, policy_version 1527425 (0.0006) [2023-12-27 16:14:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 782557184. Throughput: 0: 10326.1, 1: 10308.3. Samples: 782522252. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:22,363][19170] Avg episode reward: [(0, '951.220'), (1, '951.730')] [2023-12-27 16:14:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001527432_391061504.pth... [2023-12-27 16:14:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001528976_391495680.pth... [2023-12-27 16:14:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001526224_390750208.pth [2023-12-27 16:14:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001527760_391184384.pth [2023-12-27 16:14:22,911][20381] Updated weights for policy 1, policy_version 1527435 (0.0007) [2023-12-27 16:14:22,968][20381] Updated weights for policy 1, policy_version 1527445 (0.0007) [2023-12-27 16:14:22,976][20347] Updated weights for policy 0, policy_version 1528986 (0.0006) [2023-12-27 16:14:23,021][20381] Updated weights for policy 1, policy_version 1527455 (0.0005) [2023-12-27 16:14:23,035][20347] Updated weights for policy 0, policy_version 1528996 (0.0007) [2023-12-27 16:14:23,085][20347] Updated weights for policy 0, policy_version 1529006 (0.0006) [2023-12-27 16:14:23,717][20381] Updated weights for policy 1, policy_version 1527465 (0.0008) [2023-12-27 16:14:23,765][20381] Updated weights for policy 1, policy_version 1527475 (0.0005) [2023-12-27 16:14:23,776][20347] Updated weights for policy 0, policy_version 1529016 (0.0007) [2023-12-27 16:14:23,813][20381] Updated weights for policy 1, policy_version 1527485 (0.0005) [2023-12-27 16:14:23,828][20347] Updated weights for policy 0, policy_version 1529026 (0.0008) [2023-12-27 16:14:23,862][20381] Updated weights for policy 1, policy_version 1527495 (0.0005) [2023-12-27 16:14:23,878][20347] Updated weights for policy 0, policy_version 1529036 (0.0008) [2023-12-27 16:14:24,435][20381] Updated weights for policy 1, policy_version 1527505 (0.0007) [2023-12-27 16:14:24,486][20381] Updated weights for policy 1, policy_version 1527515 (0.0007) [2023-12-27 16:14:24,541][20381] Updated weights for policy 1, policy_version 1527525 (0.0007) [2023-12-27 16:14:24,663][20347] Updated weights for policy 0, policy_version 1529046 (0.0009) [2023-12-27 16:14:24,682][20291] KL-divergence is very high: 457.8980 [2023-12-27 16:14:24,706][20291] KL-divergence is very high: 1007.6552 [2023-12-27 16:14:24,725][20347] Updated weights for policy 0, policy_version 1529056 (0.0011) [2023-12-27 16:14:24,731][20291] KL-divergence is very high: 1428.6179 [2023-12-27 16:14:24,755][20291] KL-divergence is very high: 1145.2069 [2023-12-27 16:14:24,780][20291] KL-divergence is very high: 954.8022 [2023-12-27 16:14:24,787][20347] Updated weights for policy 0, policy_version 1529066 (0.0011) [2023-12-27 16:14:24,803][20291] KL-divergence is very high: 443.6768 [2023-12-27 16:14:25,192][20381] Updated weights for policy 1, policy_version 1527535 (0.0006) [2023-12-27 16:14:25,250][20381] Updated weights for policy 1, policy_version 1527545 (0.0008) [2023-12-27 16:14:25,306][20381] Updated weights for policy 1, policy_version 1527555 (0.0008) [2023-12-27 16:14:25,395][20347] Updated weights for policy 0, policy_version 1529076 (0.0010) [2023-12-27 16:14:25,453][20347] Updated weights for policy 0, policy_version 1529086 (0.0011) [2023-12-27 16:14:25,502][20347] Updated weights for policy 0, policy_version 1529096 (0.0010) [2023-12-27 16:14:25,913][20381] Updated weights for policy 1, policy_version 1527565 (0.0008) [2023-12-27 16:14:25,960][20381] Updated weights for policy 1, policy_version 1527576 (0.0009) [2023-12-27 16:14:26,013][20381] Updated weights for policy 1, policy_version 1527586 (0.0005) [2023-12-27 16:14:26,134][20347] Updated weights for policy 0, policy_version 1529106 (0.0010) [2023-12-27 16:14:26,183][20347] Updated weights for policy 0, policy_version 1529116 (0.0010) [2023-12-27 16:14:26,236][20347] Updated weights for policy 0, policy_version 1529126 (0.0010) [2023-12-27 16:14:26,285][20347] Updated weights for policy 0, policy_version 1529136 (0.0010) [2023-12-27 16:14:26,683][20381] Updated weights for policy 1, policy_version 1527596 (0.0007) [2023-12-27 16:14:26,735][20381] Updated weights for policy 1, policy_version 1527606 (0.0008) [2023-12-27 16:14:26,783][20381] Updated weights for policy 1, policy_version 1527616 (0.0010) [2023-12-27 16:14:26,879][20347] Updated weights for policy 0, policy_version 1529146 (0.0007) [2023-12-27 16:14:26,930][20347] Updated weights for policy 0, policy_version 1529156 (0.0011) [2023-12-27 16:14:26,981][20347] Updated weights for policy 0, policy_version 1529166 (0.0010) [2023-12-27 16:14:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.2, 300 sec: 20577.2). Total num frames: 782655488. Throughput: 0: 10223.7, 1: 10351.9. Samples: 782645144. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:27,363][19170] Avg episode reward: [(0, '941.520'), (1, '970.760')] [2023-12-27 16:14:27,433][20381] Updated weights for policy 1, policy_version 1527626 (0.0010) [2023-12-27 16:14:27,484][20381] Updated weights for policy 1, policy_version 1527636 (0.0010) [2023-12-27 16:14:27,540][20381] Updated weights for policy 1, policy_version 1527646 (0.0010) [2023-12-27 16:14:27,590][20381] Updated weights for policy 1, policy_version 1527656 (0.0010) [2023-12-27 16:14:27,669][20347] Updated weights for policy 0, policy_version 1529176 (0.0006) [2023-12-27 16:14:27,727][20347] Updated weights for policy 0, policy_version 1529186 (0.0005) [2023-12-27 16:14:27,772][20347] Updated weights for policy 0, policy_version 1529196 (0.0006) [2023-12-27 16:14:28,273][20381] Updated weights for policy 1, policy_version 1527666 (0.0010) [2023-12-27 16:14:28,318][20381] Updated weights for policy 1, policy_version 1527676 (0.0010) [2023-12-27 16:14:28,380][20381] Updated weights for policy 1, policy_version 1527686 (0.0010) [2023-12-27 16:14:28,411][20347] Updated weights for policy 0, policy_version 1529206 (0.0009) [2023-12-27 16:14:28,473][20347] Updated weights for policy 0, policy_version 1529216 (0.0010) [2023-12-27 16:14:28,535][20347] Updated weights for policy 0, policy_version 1529226 (0.0010) [2023-12-27 16:14:29,092][20381] Updated weights for policy 1, policy_version 1527696 (0.0011) [2023-12-27 16:14:29,152][20381] Updated weights for policy 1, policy_version 1527706 (0.0010) [2023-12-27 16:14:29,173][20347] Updated weights for policy 0, policy_version 1529236 (0.0009) [2023-12-27 16:14:29,210][20381] Updated weights for policy 1, policy_version 1527716 (0.0010) [2023-12-27 16:14:29,241][20347] Updated weights for policy 0, policy_version 1529246 (0.0007) [2023-12-27 16:14:29,288][20347] Updated weights for policy 0, policy_version 1529256 (0.0008) [2023-12-27 16:14:29,946][20347] Updated weights for policy 0, policy_version 1529266 (0.0008) [2023-12-27 16:14:29,999][20381] Updated weights for policy 1, policy_version 1527726 (0.0009) [2023-12-27 16:14:30,003][20347] Updated weights for policy 0, policy_version 1529276 (0.0007) [2023-12-27 16:14:30,056][20347] Updated weights for policy 0, policy_version 1529286 (0.0006) [2023-12-27 16:14:30,060][20381] Updated weights for policy 1, policy_version 1527736 (0.0007) [2023-12-27 16:14:30,110][20347] Updated weights for policy 0, policy_version 1529296 (0.0006) [2023-12-27 16:14:30,122][20381] Updated weights for policy 1, policy_version 1527746 (0.0005) [2023-12-27 16:14:30,690][20381] Updated weights for policy 1, policy_version 1527756 (0.0007) [2023-12-27 16:14:30,747][20381] Updated weights for policy 1, policy_version 1527766 (0.0010) [2023-12-27 16:14:30,773][20347] Updated weights for policy 0, policy_version 1529306 (0.0005) [2023-12-27 16:14:30,798][20381] Updated weights for policy 1, policy_version 1527776 (0.0010) [2023-12-27 16:14:30,821][20347] Updated weights for policy 0, policy_version 1529316 (0.0007) [2023-12-27 16:14:30,874][20347] Updated weights for policy 0, policy_version 1529326 (0.0007) [2023-12-27 16:14:31,451][20381] Updated weights for policy 1, policy_version 1527786 (0.0010) [2023-12-27 16:14:31,503][20381] Updated weights for policy 1, policy_version 1527796 (0.0007) [2023-12-27 16:14:31,555][20381] Updated weights for policy 1, policy_version 1527806 (0.0009) [2023-12-27 16:14:31,577][20347] Updated weights for policy 0, policy_version 1529336 (0.0006) [2023-12-27 16:14:31,578][20291] KL-divergence is very high: 114.0705 [2023-12-27 16:14:31,609][20381] Updated weights for policy 1, policy_version 1527816 (0.0008) [2023-12-27 16:14:31,637][20291] KL-divergence is very high: 133.7016 [2023-12-27 16:14:31,648][20347] Updated weights for policy 0, policy_version 1529346 (0.0006) [2023-12-27 16:14:31,711][20347] Updated weights for policy 0, policy_version 1529356 (0.0007) [2023-12-27 16:14:32,294][20381] Updated weights for policy 1, policy_version 1527826 (0.0007) [2023-12-27 16:14:32,301][20347] Updated weights for policy 0, policy_version 1529366 (0.0008) [2023-12-27 16:14:32,329][20291] KL-divergence is very high: 119.0500 [2023-12-27 16:14:32,348][20291] KL-divergence is very high: 131.0334 [2023-12-27 16:14:32,353][20381] Updated weights for policy 1, policy_version 1527836 (0.0007) [2023-12-27 16:14:32,354][20291] KL-divergence is very high: 112.8921 [2023-12-27 16:14:32,358][20347] Updated weights for policy 0, policy_version 1529376 (0.0005) [2023-12-27 16:14:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 782753792. Throughput: 0: 10283.8, 1: 10324.0. Samples: 782770640. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:32,363][19170] Avg episode reward: [(0, '882.600'), (1, '980.510')] [2023-12-27 16:14:32,420][20347] Updated weights for policy 0, policy_version 1529386 (0.0007) [2023-12-27 16:14:32,422][20381] Updated weights for policy 1, policy_version 1527846 (0.0008) [2023-12-27 16:14:33,011][20381] Updated weights for policy 1, policy_version 1527856 (0.0006) [2023-12-27 16:14:33,067][20381] Updated weights for policy 1, policy_version 1527866 (0.0010) [2023-12-27 16:14:33,113][20381] Updated weights for policy 1, policy_version 1527876 (0.0010) [2023-12-27 16:14:33,132][20347] Updated weights for policy 0, policy_version 1529396 (0.0006) [2023-12-27 16:14:33,183][20347] Updated weights for policy 0, policy_version 1529406 (0.0005) [2023-12-27 16:14:33,236][20347] Updated weights for policy 0, policy_version 1529416 (0.0005) [2023-12-27 16:14:33,752][20381] Updated weights for policy 1, policy_version 1527886 (0.0010) [2023-12-27 16:14:33,798][20347] Updated weights for policy 0, policy_version 1529426 (0.0005) [2023-12-27 16:14:33,807][20381] Updated weights for policy 1, policy_version 1527896 (0.0010) [2023-12-27 16:14:33,846][20347] Updated weights for policy 0, policy_version 1529436 (0.0005) [2023-12-27 16:14:33,861][20381] Updated weights for policy 1, policy_version 1527906 (0.0010) [2023-12-27 16:14:33,897][20347] Updated weights for policy 0, policy_version 1529446 (0.0008) [2023-12-27 16:14:33,942][20347] Updated weights for policy 0, policy_version 1529456 (0.0008) [2023-12-27 16:14:34,489][20381] Updated weights for policy 1, policy_version 1527916 (0.0008) [2023-12-27 16:14:34,546][20381] Updated weights for policy 1, policy_version 1527926 (0.0010) [2023-12-27 16:14:34,606][20381] Updated weights for policy 1, policy_version 1527936 (0.0011) [2023-12-27 16:14:34,664][20347] Updated weights for policy 0, policy_version 1529466 (0.0006) [2023-12-27 16:14:34,723][20347] Updated weights for policy 0, policy_version 1529476 (0.0007) [2023-12-27 16:14:34,781][20347] Updated weights for policy 0, policy_version 1529486 (0.0007) [2023-12-27 16:14:35,261][20381] Updated weights for policy 1, policy_version 1527946 (0.0010) [2023-12-27 16:14:35,314][20381] Updated weights for policy 1, policy_version 1527956 (0.0010) [2023-12-27 16:14:35,362][20381] Updated weights for policy 1, policy_version 1527966 (0.0010) [2023-12-27 16:14:35,416][20381] Updated weights for policy 1, policy_version 1527976 (0.0010) [2023-12-27 16:14:35,495][20347] Updated weights for policy 0, policy_version 1529496 (0.0008) [2023-12-27 16:14:35,549][20347] Updated weights for policy 0, policy_version 1529506 (0.0007) [2023-12-27 16:14:35,603][20347] Updated weights for policy 0, policy_version 1529516 (0.0007) [2023-12-27 16:14:36,080][20381] Updated weights for policy 1, policy_version 1527986 (0.0005) [2023-12-27 16:14:36,140][20381] Updated weights for policy 1, policy_version 1527996 (0.0008) [2023-12-27 16:14:36,189][20381] Updated weights for policy 1, policy_version 1528006 (0.0009) [2023-12-27 16:14:36,341][20347] Updated weights for policy 0, policy_version 1529526 (0.0010) [2023-12-27 16:14:36,359][20291] KL-divergence is very high: 131.0735 [2023-12-27 16:14:36,377][20291] KL-divergence is very high: 117.7330 [2023-12-27 16:14:36,399][20347] Updated weights for policy 0, policy_version 1529536 (0.0007) [2023-12-27 16:14:36,461][20347] Updated weights for policy 0, policy_version 1529546 (0.0007) [2023-12-27 16:14:36,945][20381] Updated weights for policy 1, policy_version 1528016 (0.0006) [2023-12-27 16:14:37,008][20381] Updated weights for policy 1, policy_version 1528026 (0.0005) [2023-12-27 16:14:37,070][20381] Updated weights for policy 1, policy_version 1528036 (0.0005) [2023-12-27 16:14:37,096][20347] Updated weights for policy 0, policy_version 1529556 (0.0008) [2023-12-27 16:14:37,144][20347] Updated weights for policy 0, policy_version 1529566 (0.0010) [2023-12-27 16:14:37,193][20347] Updated weights for policy 0, policy_version 1529576 (0.0010) [2023-12-27 16:14:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 782868480. Throughput: 0: 10317.9, 1: 10338.0. Samples: 782834756. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:37,364][19170] Avg episode reward: [(0, '764.900'), (1, '970.710')] [2023-12-27 16:14:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001528040_391217152.pth... [2023-12-27 16:14:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001529584_391651328.pth... [2023-12-27 16:14:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001526824_390905856.pth [2023-12-27 16:14:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001528368_391340032.pth [2023-12-27 16:14:37,673][20381] Updated weights for policy 1, policy_version 1528046 (0.0009) [2023-12-27 16:14:37,734][20381] Updated weights for policy 1, policy_version 1528056 (0.0008) [2023-12-27 16:14:37,798][20381] Updated weights for policy 1, policy_version 1528066 (0.0008) [2023-12-27 16:14:37,799][20347] Updated weights for policy 0, policy_version 1529586 (0.0010) [2023-12-27 16:14:37,848][20347] Updated weights for policy 0, policy_version 1529596 (0.0005) [2023-12-27 16:14:37,909][20347] Updated weights for policy 0, policy_version 1529606 (0.0005) [2023-12-27 16:14:37,964][20347] Updated weights for policy 0, policy_version 1529616 (0.0005) [2023-12-27 16:14:38,555][20381] Updated weights for policy 1, policy_version 1528076 (0.0009) [2023-12-27 16:14:38,576][20347] Updated weights for policy 0, policy_version 1529626 (0.0007) [2023-12-27 16:14:38,615][20381] Updated weights for policy 1, policy_version 1528086 (0.0011) [2023-12-27 16:14:38,632][20347] Updated weights for policy 0, policy_version 1529636 (0.0005) [2023-12-27 16:14:38,671][20381] Updated weights for policy 1, policy_version 1528096 (0.0011) [2023-12-27 16:14:38,686][20347] Updated weights for policy 0, policy_version 1529646 (0.0005) [2023-12-27 16:14:39,268][20347] Updated weights for policy 0, policy_version 1529656 (0.0007) [2023-12-27 16:14:39,333][20347] Updated weights for policy 0, policy_version 1529666 (0.0007) [2023-12-27 16:14:39,375][20381] Updated weights for policy 1, policy_version 1528106 (0.0010) [2023-12-27 16:14:39,401][20347] Updated weights for policy 0, policy_version 1529676 (0.0007) [2023-12-27 16:14:39,448][20381] Updated weights for policy 1, policy_version 1528116 (0.0008) [2023-12-27 16:14:39,513][20381] Updated weights for policy 1, policy_version 1528126 (0.0008) [2023-12-27 16:14:39,570][20381] Updated weights for policy 1, policy_version 1528136 (0.0008) [2023-12-27 16:14:40,117][20347] Updated weights for policy 0, policy_version 1529686 (0.0007) [2023-12-27 16:14:40,131][20291] KL-divergence is very high: 119.4454 [2023-12-27 16:14:40,137][20291] KL-divergence is very high: 200.0889 [2023-12-27 16:14:40,148][20291] KL-divergence is very high: 119.1712 [2023-12-27 16:14:40,178][20347] Updated weights for policy 0, policy_version 1529696 (0.0006) [2023-12-27 16:14:40,184][20291] KL-divergence is very high: 101.8428 [2023-12-27 16:14:40,239][20347] Updated weights for policy 0, policy_version 1529706 (0.0009) [2023-12-27 16:14:40,335][20381] Updated weights for policy 1, policy_version 1528146 (0.0006) [2023-12-27 16:14:40,402][20381] Updated weights for policy 1, policy_version 1528156 (0.0006) [2023-12-27 16:14:40,462][20381] Updated weights for policy 1, policy_version 1528166 (0.0007) [2023-12-27 16:14:40,911][20347] Updated weights for policy 0, policy_version 1529716 (0.0008) [2023-12-27 16:14:40,971][20347] Updated weights for policy 0, policy_version 1529726 (0.0008) [2023-12-27 16:14:41,023][20347] Updated weights for policy 0, policy_version 1529736 (0.0008) [2023-12-27 16:14:41,160][20381] Updated weights for policy 1, policy_version 1528176 (0.0008) [2023-12-27 16:14:41,228][20381] Updated weights for policy 1, policy_version 1528186 (0.0008) [2023-12-27 16:14:41,292][20381] Updated weights for policy 1, policy_version 1528196 (0.0008) [2023-12-27 16:14:41,770][20347] Updated weights for policy 0, policy_version 1529746 (0.0009) [2023-12-27 16:14:41,810][20291] KL-divergence is very high: 123.9096 [2023-12-27 16:14:41,817][20291] KL-divergence is very high: 134.2228 [2023-12-27 16:14:41,826][20347] Updated weights for policy 0, policy_version 1529756 (0.0008) [2023-12-27 16:14:41,833][20291] KL-divergence is very high: 117.3514 [2023-12-27 16:14:41,840][20291] KL-divergence is very high: 108.2557 [2023-12-27 16:14:41,880][20347] Updated weights for policy 0, policy_version 1529766 (0.0008) [2023-12-27 16:14:41,927][20347] Updated weights for policy 0, policy_version 1529776 (0.0008) [2023-12-27 16:14:42,008][20381] Updated weights for policy 1, policy_version 1528206 (0.0007) [2023-12-27 16:14:42,059][20381] Updated weights for policy 1, policy_version 1528216 (0.0009) [2023-12-27 16:14:42,116][20381] Updated weights for policy 1, policy_version 1528226 (0.0007) [2023-12-27 16:14:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 782966784. Throughput: 0: 10308.2, 1: 10324.5. Samples: 782956432. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:42,363][19170] Avg episode reward: [(0, '716.790'), (1, '912.270')] [2023-12-27 16:14:42,615][20347] Updated weights for policy 0, policy_version 1529786 (0.0007) [2023-12-27 16:14:42,672][20347] Updated weights for policy 0, policy_version 1529796 (0.0008) [2023-12-27 16:14:42,730][20347] Updated weights for policy 0, policy_version 1529806 (0.0009) [2023-12-27 16:14:42,806][20381] Updated weights for policy 1, policy_version 1528236 (0.0008) [2023-12-27 16:14:42,874][20381] Updated weights for policy 1, policy_version 1528246 (0.0008) [2023-12-27 16:14:42,938][20381] Updated weights for policy 1, policy_version 1528256 (0.0008) [2023-12-27 16:14:43,297][20347] Updated weights for policy 0, policy_version 1529816 (0.0007) [2023-12-27 16:14:43,350][20347] Updated weights for policy 0, policy_version 1529826 (0.0007) [2023-12-27 16:14:43,402][20347] Updated weights for policy 0, policy_version 1529836 (0.0007) [2023-12-27 16:14:43,594][20381] Updated weights for policy 1, policy_version 1528266 (0.0008) [2023-12-27 16:14:43,643][20381] Updated weights for policy 1, policy_version 1528276 (0.0008) [2023-12-27 16:14:43,697][20381] Updated weights for policy 1, policy_version 1528286 (0.0007) [2023-12-27 16:14:43,745][20381] Updated weights for policy 1, policy_version 1528296 (0.0007) [2023-12-27 16:14:44,074][20347] Updated weights for policy 0, policy_version 1529846 (0.0006) [2023-12-27 16:14:44,128][20347] Updated weights for policy 0, policy_version 1529856 (0.0007) [2023-12-27 16:14:44,178][20347] Updated weights for policy 0, policy_version 1529866 (0.0007) [2023-12-27 16:14:44,390][20381] Updated weights for policy 1, policy_version 1528306 (0.0008) [2023-12-27 16:14:44,448][20381] Updated weights for policy 1, policy_version 1528316 (0.0010) [2023-12-27 16:14:44,500][20381] Updated weights for policy 1, policy_version 1528326 (0.0010) [2023-12-27 16:14:44,848][20347] Updated weights for policy 0, policy_version 1529876 (0.0007) [2023-12-27 16:14:44,907][20347] Updated weights for policy 0, policy_version 1529886 (0.0008) [2023-12-27 16:14:44,961][20347] Updated weights for policy 0, policy_version 1529896 (0.0008) [2023-12-27 16:14:45,192][20381] Updated weights for policy 1, policy_version 1528336 (0.0008) [2023-12-27 16:14:45,255][20381] Updated weights for policy 1, policy_version 1528346 (0.0008) [2023-12-27 16:14:45,312][20381] Updated weights for policy 1, policy_version 1528356 (0.0006) [2023-12-27 16:14:45,647][20347] Updated weights for policy 0, policy_version 1529906 (0.0008) [2023-12-27 16:14:45,705][20347] Updated weights for policy 0, policy_version 1529916 (0.0007) [2023-12-27 16:14:45,750][20347] Updated weights for policy 0, policy_version 1529926 (0.0007) [2023-12-27 16:14:45,810][20347] Updated weights for policy 0, policy_version 1529936 (0.0008) [2023-12-27 16:14:45,962][20381] Updated weights for policy 1, policy_version 1528366 (0.0006) [2023-12-27 16:14:46,027][20381] Updated weights for policy 1, policy_version 1528376 (0.0007) [2023-12-27 16:14:46,083][20381] Updated weights for policy 1, policy_version 1528386 (0.0008) [2023-12-27 16:14:46,483][20347] Updated weights for policy 0, policy_version 1529946 (0.0008) [2023-12-27 16:14:46,540][20347] Updated weights for policy 0, policy_version 1529956 (0.0009) [2023-12-27 16:14:46,605][20347] Updated weights for policy 0, policy_version 1529966 (0.0008) [2023-12-27 16:14:46,730][20381] Updated weights for policy 1, policy_version 1528396 (0.0008) [2023-12-27 16:14:46,793][20381] Updated weights for policy 1, policy_version 1528406 (0.0008) [2023-12-27 16:14:46,866][20381] Updated weights for policy 1, policy_version 1528416 (0.0009) [2023-12-27 16:14:47,232][20347] Updated weights for policy 0, policy_version 1529976 (0.0009) [2023-12-27 16:14:47,284][20347] Updated weights for policy 0, policy_version 1529986 (0.0006) [2023-12-27 16:14:47,342][20347] Updated weights for policy 0, policy_version 1529996 (0.0006) [2023-12-27 16:14:47,363][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 783065088. Throughput: 0: 10320.1, 1: 10298.8. Samples: 783081316. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:47,364][19170] Avg episode reward: [(0, '794.620'), (1, '931.870')] [2023-12-27 16:14:47,454][20381] Updated weights for policy 1, policy_version 1528426 (0.0006) [2023-12-27 16:14:47,510][20381] Updated weights for policy 1, policy_version 1528436 (0.0008) [2023-12-27 16:14:47,563][20381] Updated weights for policy 1, policy_version 1528446 (0.0008) [2023-12-27 16:14:47,619][20381] Updated weights for policy 1, policy_version 1528456 (0.0008) [2023-12-27 16:14:47,983][20347] Updated weights for policy 0, policy_version 1530006 (0.0006) [2023-12-27 16:14:48,045][20347] Updated weights for policy 0, policy_version 1530016 (0.0007) [2023-12-27 16:14:48,102][20347] Updated weights for policy 0, policy_version 1530026 (0.0007) [2023-12-27 16:14:48,292][20381] Updated weights for policy 1, policy_version 1528466 (0.0007) [2023-12-27 16:14:48,346][20381] Updated weights for policy 1, policy_version 1528476 (0.0008) [2023-12-27 16:14:48,408][20381] Updated weights for policy 1, policy_version 1528486 (0.0006) [2023-12-27 16:14:48,849][20347] Updated weights for policy 0, policy_version 1530036 (0.0006) [2023-12-27 16:14:48,908][20347] Updated weights for policy 0, policy_version 1530046 (0.0007) [2023-12-27 16:14:48,955][20347] Updated weights for policy 0, policy_version 1530056 (0.0007) [2023-12-27 16:14:49,077][20381] Updated weights for policy 1, policy_version 1528496 (0.0010) [2023-12-27 16:14:49,137][20381] Updated weights for policy 1, policy_version 1528506 (0.0010) [2023-12-27 16:14:49,201][20381] Updated weights for policy 1, policy_version 1528516 (0.0005) [2023-12-27 16:14:49,697][20347] Updated weights for policy 0, policy_version 1530066 (0.0007) [2023-12-27 16:14:49,759][20347] Updated weights for policy 0, policy_version 1530076 (0.0008) [2023-12-27 16:14:49,818][20347] Updated weights for policy 0, policy_version 1530086 (0.0008) [2023-12-27 16:14:49,848][20381] Updated weights for policy 1, policy_version 1528526 (0.0008) [2023-12-27 16:14:49,888][20347] Updated weights for policy 0, policy_version 1530096 (0.0007) [2023-12-27 16:14:49,912][20381] Updated weights for policy 1, policy_version 1528536 (0.0008) [2023-12-27 16:14:49,979][20381] Updated weights for policy 1, policy_version 1528546 (0.0009) [2023-12-27 16:14:50,634][20347] Updated weights for policy 0, policy_version 1530106 (0.0007) [2023-12-27 16:14:50,691][20381] Updated weights for policy 1, policy_version 1528556 (0.0011) [2023-12-27 16:14:50,697][20347] Updated weights for policy 0, policy_version 1530116 (0.0007) [2023-12-27 16:14:50,747][20381] Updated weights for policy 1, policy_version 1528566 (0.0006) [2023-12-27 16:14:50,762][20347] Updated weights for policy 0, policy_version 1530126 (0.0007) [2023-12-27 16:14:50,806][20381] Updated weights for policy 1, policy_version 1528576 (0.0007) [2023-12-27 16:14:50,828][20292] KL-divergence is very high: 157.2334 [2023-12-27 16:14:51,427][20347] Updated weights for policy 0, policy_version 1530136 (0.0006) [2023-12-27 16:14:51,484][20347] Updated weights for policy 0, policy_version 1530146 (0.0006) [2023-12-27 16:14:51,525][20381] Updated weights for policy 1, policy_version 1528586 (0.0007) [2023-12-27 16:14:51,546][20347] Updated weights for policy 0, policy_version 1530156 (0.0006) [2023-12-27 16:14:51,579][20381] Updated weights for policy 1, policy_version 1528596 (0.0007) [2023-12-27 16:14:51,643][20381] Updated weights for policy 1, policy_version 1528606 (0.0007) [2023-12-27 16:14:51,705][20381] Updated weights for policy 1, policy_version 1528616 (0.0007) [2023-12-27 16:14:52,271][20347] Updated weights for policy 0, policy_version 1530166 (0.0007) [2023-12-27 16:14:52,330][20347] Updated weights for policy 0, policy_version 1530176 (0.0008) [2023-12-27 16:14:52,337][20381] Updated weights for policy 1, policy_version 1528626 (0.0006) [2023-12-27 16:14:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 783163392. Throughput: 0: 10286.6, 1: 10304.2. Samples: 783141416. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:52,363][19170] Avg episode reward: [(0, '804.320'), (1, '971.010')] [2023-12-27 16:14:52,393][20347] Updated weights for policy 0, policy_version 1530186 (0.0008) [2023-12-27 16:14:52,398][20381] Updated weights for policy 1, policy_version 1528636 (0.0007) [2023-12-27 16:14:52,425][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001530192_391806976.pth... [2023-12-27 16:14:52,428][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001528976_391495680.pth [2023-12-27 16:14:52,454][20381] Updated weights for policy 1, policy_version 1528646 (0.0008) [2023-12-27 16:14:52,464][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001528648_391372800.pth... [2023-12-27 16:14:52,468][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001527432_391061504.pth [2023-12-27 16:14:53,070][20381] Updated weights for policy 1, policy_version 1528656 (0.0008) [2023-12-27 16:14:53,123][20381] Updated weights for policy 1, policy_version 1528666 (0.0006) [2023-12-27 16:14:53,139][20347] Updated weights for policy 0, policy_version 1530196 (0.0006) [2023-12-27 16:14:53,178][20381] Updated weights for policy 1, policy_version 1528676 (0.0006) [2023-12-27 16:14:53,195][20347] Updated weights for policy 0, policy_version 1530206 (0.0006) [2023-12-27 16:14:53,248][20347] Updated weights for policy 0, policy_version 1530216 (0.0007) [2023-12-27 16:14:53,813][20347] Updated weights for policy 0, policy_version 1530226 (0.0007) [2023-12-27 16:14:53,872][20347] Updated weights for policy 0, policy_version 1530236 (0.0005) [2023-12-27 16:14:53,891][20381] Updated weights for policy 1, policy_version 1528686 (0.0006) [2023-12-27 16:14:53,930][20347] Updated weights for policy 0, policy_version 1530246 (0.0005) [2023-12-27 16:14:53,952][20381] Updated weights for policy 1, policy_version 1528696 (0.0008) [2023-12-27 16:14:53,989][20347] Updated weights for policy 0, policy_version 1530256 (0.0006) [2023-12-27 16:14:54,004][20381] Updated weights for policy 1, policy_version 1528706 (0.0006) [2023-12-27 16:14:54,597][20381] Updated weights for policy 1, policy_version 1528716 (0.0008) [2023-12-27 16:14:54,615][20347] Updated weights for policy 0, policy_version 1530266 (0.0005) [2023-12-27 16:14:54,653][20381] Updated weights for policy 1, policy_version 1528726 (0.0010) [2023-12-27 16:14:54,670][20347] Updated weights for policy 0, policy_version 1530276 (0.0005) [2023-12-27 16:14:54,711][20381] Updated weights for policy 1, policy_version 1528736 (0.0010) [2023-12-27 16:14:54,727][20347] Updated weights for policy 0, policy_version 1530286 (0.0005) [2023-12-27 16:14:55,352][20381] Updated weights for policy 1, policy_version 1528746 (0.0010) [2023-12-27 16:14:55,358][20347] Updated weights for policy 0, policy_version 1530296 (0.0007) [2023-12-27 16:14:55,406][20381] Updated weights for policy 1, policy_version 1528756 (0.0010) [2023-12-27 16:14:55,418][20347] Updated weights for policy 0, policy_version 1530306 (0.0008) [2023-12-27 16:14:55,460][20381] Updated weights for policy 1, policy_version 1528766 (0.0010) [2023-12-27 16:14:55,482][20347] Updated weights for policy 0, policy_version 1530316 (0.0008) [2023-12-27 16:14:55,515][20381] Updated weights for policy 1, policy_version 1528776 (0.0010) [2023-12-27 16:14:56,091][20347] Updated weights for policy 0, policy_version 1530326 (0.0006) [2023-12-27 16:14:56,141][20347] Updated weights for policy 0, policy_version 1530336 (0.0007) [2023-12-27 16:14:56,194][20347] Updated weights for policy 0, policy_version 1530346 (0.0006) [2023-12-27 16:14:56,204][20381] Updated weights for policy 1, policy_version 1528786 (0.0009) [2023-12-27 16:14:56,254][20381] Updated weights for policy 1, policy_version 1528796 (0.0010) [2023-12-27 16:14:56,308][20381] Updated weights for policy 1, policy_version 1528806 (0.0011) [2023-12-27 16:14:56,858][20347] Updated weights for policy 0, policy_version 1530356 (0.0007) [2023-12-27 16:14:56,906][20347] Updated weights for policy 0, policy_version 1530366 (0.0007) [2023-12-27 16:14:56,960][20381] Updated weights for policy 1, policy_version 1528816 (0.0011) [2023-12-27 16:14:56,961][20347] Updated weights for policy 0, policy_version 1530376 (0.0006) [2023-12-27 16:14:57,015][20381] Updated weights for policy 1, policy_version 1528826 (0.0010) [2023-12-27 16:14:57,075][20381] Updated weights for policy 1, policy_version 1528836 (0.0010) [2023-12-27 16:14:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 783278080. Throughput: 0: 10287.5, 1: 10321.3. Samples: 783266656. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:14:57,363][19170] Avg episode reward: [(0, '862.840'), (1, '970.980')] [2023-12-27 16:14:57,676][20347] Updated weights for policy 0, policy_version 1530386 (0.0006) [2023-12-27 16:14:57,712][20381] Updated weights for policy 1, policy_version 1528846 (0.0008) [2023-12-27 16:14:57,732][20347] Updated weights for policy 0, policy_version 1530396 (0.0007) [2023-12-27 16:14:57,758][20381] Updated weights for policy 1, policy_version 1528856 (0.0006) [2023-12-27 16:14:57,785][20347] Updated weights for policy 0, policy_version 1530406 (0.0006) [2023-12-27 16:14:57,806][20381] Updated weights for policy 1, policy_version 1528866 (0.0006) [2023-12-27 16:14:57,843][20347] Updated weights for policy 0, policy_version 1530416 (0.0007) [2023-12-27 16:14:58,417][20381] Updated weights for policy 1, policy_version 1528876 (0.0006) [2023-12-27 16:14:58,483][20381] Updated weights for policy 1, policy_version 1528886 (0.0008) [2023-12-27 16:14:58,543][20381] Updated weights for policy 1, policy_version 1528896 (0.0007) [2023-12-27 16:14:58,593][20347] Updated weights for policy 0, policy_version 1530426 (0.0008) [2023-12-27 16:14:58,653][20347] Updated weights for policy 0, policy_version 1530436 (0.0008) [2023-12-27 16:14:58,716][20347] Updated weights for policy 0, policy_version 1530446 (0.0007) [2023-12-27 16:14:59,319][20381] Updated weights for policy 1, policy_version 1528906 (0.0007) [2023-12-27 16:14:59,385][20381] Updated weights for policy 1, policy_version 1528916 (0.0010) [2023-12-27 16:14:59,443][20381] Updated weights for policy 1, policy_version 1528926 (0.0008) [2023-12-27 16:14:59,489][20347] Updated weights for policy 0, policy_version 1530456 (0.0006) [2023-12-27 16:14:59,498][20381] Updated weights for policy 1, policy_version 1528936 (0.0008) [2023-12-27 16:14:59,537][20347] Updated weights for policy 0, policy_version 1530466 (0.0005) [2023-12-27 16:14:59,587][20347] Updated weights for policy 0, policy_version 1530476 (0.0005) [2023-12-27 16:15:00,209][20347] Updated weights for policy 0, policy_version 1530486 (0.0006) [2023-12-27 16:15:00,264][20381] Updated weights for policy 1, policy_version 1528946 (0.0008) [2023-12-27 16:15:00,275][20347] Updated weights for policy 0, policy_version 1530496 (0.0007) [2023-12-27 16:15:00,318][20381] Updated weights for policy 1, policy_version 1528956 (0.0008) [2023-12-27 16:15:00,331][20347] Updated weights for policy 0, policy_version 1530506 (0.0007) [2023-12-27 16:15:00,380][20381] Updated weights for policy 1, policy_version 1528966 (0.0008) [2023-12-27 16:15:00,884][20347] Updated weights for policy 0, policy_version 1530516 (0.0008) [2023-12-27 16:15:00,938][20347] Updated weights for policy 0, policy_version 1530526 (0.0010) [2023-12-27 16:15:00,965][20381] Updated weights for policy 1, policy_version 1528976 (0.0007) [2023-12-27 16:15:00,988][20347] Updated weights for policy 0, policy_version 1530536 (0.0010) [2023-12-27 16:15:01,014][20381] Updated weights for policy 1, policy_version 1528986 (0.0006) [2023-12-27 16:15:01,075][20381] Updated weights for policy 1, policy_version 1528996 (0.0007) [2023-12-27 16:15:01,626][20347] Updated weights for policy 0, policy_version 1530546 (0.0008) [2023-12-27 16:15:01,673][20381] Updated weights for policy 1, policy_version 1529006 (0.0007) [2023-12-27 16:15:01,687][20347] Updated weights for policy 0, policy_version 1530556 (0.0008) [2023-12-27 16:15:01,731][20381] Updated weights for policy 1, policy_version 1529016 (0.0007) [2023-12-27 16:15:01,755][20347] Updated weights for policy 0, policy_version 1530566 (0.0010) [2023-12-27 16:15:01,796][20381] Updated weights for policy 1, policy_version 1529026 (0.0007) [2023-12-27 16:15:01,815][20347] Updated weights for policy 0, policy_version 1530576 (0.0006) [2023-12-27 16:15:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 783376384. Throughput: 0: 10306.2, 1: 10304.8. Samples: 783390228. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:02,363][19170] Avg episode reward: [(0, '853.100'), (1, '980.560')] [2023-12-27 16:15:02,444][20347] Updated weights for policy 0, policy_version 1530586 (0.0010) [2023-12-27 16:15:02,477][20381] Updated weights for policy 1, policy_version 1529036 (0.0008) [2023-12-27 16:15:02,499][20347] Updated weights for policy 0, policy_version 1530596 (0.0010) [2023-12-27 16:15:02,530][20381] Updated weights for policy 1, policy_version 1529046 (0.0006) [2023-12-27 16:15:02,550][20347] Updated weights for policy 0, policy_version 1530606 (0.0010) [2023-12-27 16:15:02,588][20381] Updated weights for policy 1, policy_version 1529056 (0.0005) [2023-12-27 16:15:03,170][20381] Updated weights for policy 1, policy_version 1529066 (0.0006) [2023-12-27 16:15:03,219][20381] Updated weights for policy 1, policy_version 1529076 (0.0006) [2023-12-27 16:15:03,242][20347] Updated weights for policy 0, policy_version 1530616 (0.0010) [2023-12-27 16:15:03,267][20381] Updated weights for policy 1, policy_version 1529086 (0.0005) [2023-12-27 16:15:03,287][20347] Updated weights for policy 0, policy_version 1530626 (0.0010) [2023-12-27 16:15:03,313][20381] Updated weights for policy 1, policy_version 1529096 (0.0005) [2023-12-27 16:15:03,332][20347] Updated weights for policy 0, policy_version 1530636 (0.0010) [2023-12-27 16:15:03,893][20381] Updated weights for policy 1, policy_version 1529106 (0.0009) [2023-12-27 16:15:03,945][20347] Updated weights for policy 0, policy_version 1530646 (0.0010) [2023-12-27 16:15:03,956][20381] Updated weights for policy 1, policy_version 1529116 (0.0011) [2023-12-27 16:15:04,006][20347] Updated weights for policy 0, policy_version 1530656 (0.0006) [2023-12-27 16:15:04,016][20381] Updated weights for policy 1, policy_version 1529126 (0.0011) [2023-12-27 16:15:04,066][20347] Updated weights for policy 0, policy_version 1530666 (0.0010) [2023-12-27 16:15:04,704][20347] Updated weights for policy 0, policy_version 1530676 (0.0008) [2023-12-27 16:15:04,712][20381] Updated weights for policy 1, policy_version 1529136 (0.0008) [2023-12-27 16:15:04,765][20381] Updated weights for policy 1, policy_version 1529146 (0.0007) [2023-12-27 16:15:04,765][20347] Updated weights for policy 0, policy_version 1530686 (0.0005) [2023-12-27 16:15:04,811][20381] Updated weights for policy 1, policy_version 1529156 (0.0007) [2023-12-27 16:15:04,819][20347] Updated weights for policy 0, policy_version 1530696 (0.0005) [2023-12-27 16:15:05,362][20347] Updated weights for policy 0, policy_version 1530706 (0.0005) [2023-12-27 16:15:05,414][20347] Updated weights for policy 0, policy_version 1530716 (0.0005) [2023-12-27 16:15:05,464][20347] Updated weights for policy 0, policy_version 1530726 (0.0005) [2023-12-27 16:15:05,494][20381] Updated weights for policy 1, policy_version 1529166 (0.0007) [2023-12-27 16:15:05,517][20347] Updated weights for policy 0, policy_version 1530736 (0.0005) [2023-12-27 16:15:05,553][20381] Updated weights for policy 1, policy_version 1529176 (0.0008) [2023-12-27 16:15:05,619][20381] Updated weights for policy 1, policy_version 1529186 (0.0007) [2023-12-27 16:15:06,153][20347] Updated weights for policy 0, policy_version 1530746 (0.0010) [2023-12-27 16:15:06,173][20291] KL-divergence is very high: 109.3714 [2023-12-27 16:15:06,186][20291] KL-divergence is very high: 114.8872 [2023-12-27 16:15:06,212][20291] KL-divergence is very high: 103.0763 [2023-12-27 16:15:06,216][20347] Updated weights for policy 0, policy_version 1530756 (0.0006) [2023-12-27 16:15:06,259][20381] Updated weights for policy 1, policy_version 1529196 (0.0008) [2023-12-27 16:15:06,275][20347] Updated weights for policy 0, policy_version 1530766 (0.0007) [2023-12-27 16:15:06,316][20381] Updated weights for policy 1, policy_version 1529206 (0.0008) [2023-12-27 16:15:06,375][20381] Updated weights for policy 1, policy_version 1529216 (0.0007) [2023-12-27 16:15:06,969][20347] Updated weights for policy 0, policy_version 1530776 (0.0007) [2023-12-27 16:15:07,023][20347] Updated weights for policy 0, policy_version 1530786 (0.0007) [2023-12-27 16:15:07,073][20381] Updated weights for policy 1, policy_version 1529226 (0.0006) [2023-12-27 16:15:07,081][20347] Updated weights for policy 0, policy_version 1530796 (0.0008) [2023-12-27 16:15:07,126][20381] Updated weights for policy 1, policy_version 1529236 (0.0007) [2023-12-27 16:15:07,174][20381] Updated weights for policy 1, policy_version 1529246 (0.0008) [2023-12-27 16:15:07,227][20381] Updated weights for policy 1, policy_version 1529256 (0.0007) [2023-12-27 16:15:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 783491072. Throughput: 0: 10358.9, 1: 10394.2. Samples: 783456140. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:07,363][19170] Avg episode reward: [(0, '843.330'), (1, '990.280')] [2023-12-27 16:15:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001529256_391528448.pth... [2023-12-27 16:15:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001530800_391962624.pth... [2023-12-27 16:15:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001528040_391217152.pth [2023-12-27 16:15:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001529584_391651328.pth [2023-12-27 16:15:07,780][20347] Updated weights for policy 0, policy_version 1530806 (0.0007) [2023-12-27 16:15:07,841][20347] Updated weights for policy 0, policy_version 1530816 (0.0009) [2023-12-27 16:15:07,873][20381] Updated weights for policy 1, policy_version 1529266 (0.0011) [2023-12-27 16:15:07,892][20347] Updated weights for policy 0, policy_version 1530826 (0.0007) [2023-12-27 16:15:07,935][20381] Updated weights for policy 1, policy_version 1529276 (0.0011) [2023-12-27 16:15:07,991][20381] Updated weights for policy 1, policy_version 1529286 (0.0010) [2023-12-27 16:15:08,548][20347] Updated weights for policy 0, policy_version 1530836 (0.0009) [2023-12-27 16:15:08,605][20347] Updated weights for policy 0, policy_version 1530846 (0.0010) [2023-12-27 16:15:08,661][20347] Updated weights for policy 0, policy_version 1530856 (0.0008) [2023-12-27 16:15:08,689][20381] Updated weights for policy 1, policy_version 1529296 (0.0011) [2023-12-27 16:15:08,744][20381] Updated weights for policy 1, policy_version 1529306 (0.0011) [2023-12-27 16:15:08,805][20381] Updated weights for policy 1, policy_version 1529316 (0.0007) [2023-12-27 16:15:09,348][20347] Updated weights for policy 0, policy_version 1530866 (0.0006) [2023-12-27 16:15:09,417][20347] Updated weights for policy 0, policy_version 1530876 (0.0008) [2023-12-27 16:15:09,462][20381] Updated weights for policy 1, policy_version 1529326 (0.0010) [2023-12-27 16:15:09,476][20347] Updated weights for policy 0, policy_version 1530886 (0.0006) [2023-12-27 16:15:09,516][20381] Updated weights for policy 1, policy_version 1529336 (0.0011) [2023-12-27 16:15:09,531][20347] Updated weights for policy 0, policy_version 1530896 (0.0008) [2023-12-27 16:15:09,566][20381] Updated weights for policy 1, policy_version 1529346 (0.0011) [2023-12-27 16:15:10,237][20347] Updated weights for policy 0, policy_version 1530906 (0.0008) [2023-12-27 16:15:10,249][20381] Updated weights for policy 1, policy_version 1529356 (0.0009) [2023-12-27 16:15:10,295][20347] Updated weights for policy 0, policy_version 1530916 (0.0006) [2023-12-27 16:15:10,312][20381] Updated weights for policy 1, policy_version 1529366 (0.0011) [2023-12-27 16:15:10,348][20347] Updated weights for policy 0, policy_version 1530926 (0.0006) [2023-12-27 16:15:10,369][20381] Updated weights for policy 1, policy_version 1529376 (0.0011) [2023-12-27 16:15:11,021][20381] Updated weights for policy 1, policy_version 1529386 (0.0010) [2023-12-27 16:15:11,062][20347] Updated weights for policy 0, policy_version 1530936 (0.0007) [2023-12-27 16:15:11,086][20381] Updated weights for policy 1, policy_version 1529396 (0.0008) [2023-12-27 16:15:11,121][20347] Updated weights for policy 0, policy_version 1530946 (0.0007) [2023-12-27 16:15:11,148][20381] Updated weights for policy 1, policy_version 1529406 (0.0008) [2023-12-27 16:15:11,188][20347] Updated weights for policy 0, policy_version 1530956 (0.0009) [2023-12-27 16:15:11,213][20381] Updated weights for policy 1, policy_version 1529416 (0.0006) [2023-12-27 16:15:11,970][20381] Updated weights for policy 1, policy_version 1529426 (0.0008) [2023-12-27 16:15:12,002][20347] Updated weights for policy 0, policy_version 1530966 (0.0006) [2023-12-27 16:15:12,031][20381] Updated weights for policy 1, policy_version 1529436 (0.0006) [2023-12-27 16:15:12,064][20347] Updated weights for policy 0, policy_version 1530976 (0.0006) [2023-12-27 16:15:12,097][20381] Updated weights for policy 1, policy_version 1529446 (0.0005) [2023-12-27 16:15:12,123][20347] Updated weights for policy 0, policy_version 1530986 (0.0006) [2023-12-27 16:15:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 783589376. Throughput: 0: 10393.2, 1: 10362.3. Samples: 783579144. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:12,363][19170] Avg episode reward: [(0, '872.840'), (1, '970.780')] [2023-12-27 16:15:12,771][20347] Updated weights for policy 0, policy_version 1530996 (0.0006) [2023-12-27 16:15:12,791][20381] Updated weights for policy 1, policy_version 1529456 (0.0007) [2023-12-27 16:15:12,832][20347] Updated weights for policy 0, policy_version 1531006 (0.0006) [2023-12-27 16:15:12,842][20381] Updated weights for policy 1, policy_version 1529466 (0.0006) [2023-12-27 16:15:12,892][20347] Updated weights for policy 0, policy_version 1531016 (0.0006) [2023-12-27 16:15:12,894][20381] Updated weights for policy 1, policy_version 1529476 (0.0005) [2023-12-27 16:15:13,511][20381] Updated weights for policy 1, policy_version 1529486 (0.0008) [2023-12-27 16:15:13,546][20347] Updated weights for policy 0, policy_version 1531026 (0.0008) [2023-12-27 16:15:13,561][20381] Updated weights for policy 1, policy_version 1529496 (0.0011) [2023-12-27 16:15:13,597][20347] Updated weights for policy 0, policy_version 1531036 (0.0010) [2023-12-27 16:15:13,609][20381] Updated weights for policy 1, policy_version 1529506 (0.0005) [2023-12-27 16:15:13,647][20347] Updated weights for policy 0, policy_version 1531046 (0.0008) [2023-12-27 16:15:13,692][20347] Updated weights for policy 0, policy_version 1531056 (0.0005) [2023-12-27 16:15:14,220][20381] Updated weights for policy 1, policy_version 1529516 (0.0007) [2023-12-27 16:15:14,265][20381] Updated weights for policy 1, policy_version 1529526 (0.0010) [2023-12-27 16:15:14,273][20347] Updated weights for policy 0, policy_version 1531066 (0.0005) [2023-12-27 16:15:14,310][20381] Updated weights for policy 1, policy_version 1529536 (0.0010) [2023-12-27 16:15:14,324][20347] Updated weights for policy 0, policy_version 1531076 (0.0005) [2023-12-27 16:15:14,378][20347] Updated weights for policy 0, policy_version 1531086 (0.0005) [2023-12-27 16:15:14,968][20347] Updated weights for policy 0, policy_version 1531096 (0.0010) [2023-12-27 16:15:15,024][20347] Updated weights for policy 0, policy_version 1531106 (0.0010) [2023-12-27 16:15:15,071][20381] Updated weights for policy 1, policy_version 1529546 (0.0010) [2023-12-27 16:15:15,078][20347] Updated weights for policy 0, policy_version 1531116 (0.0011) [2023-12-27 16:15:15,130][20381] Updated weights for policy 1, policy_version 1529556 (0.0010) [2023-12-27 16:15:15,193][20381] Updated weights for policy 1, policy_version 1529566 (0.0011) [2023-12-27 16:15:15,253][20381] Updated weights for policy 1, policy_version 1529576 (0.0010) [2023-12-27 16:15:15,767][20347] Updated weights for policy 0, policy_version 1531126 (0.0010) [2023-12-27 16:15:15,824][20347] Updated weights for policy 0, policy_version 1531136 (0.0010) [2023-12-27 16:15:15,889][20347] Updated weights for policy 0, policy_version 1531146 (0.0010) [2023-12-27 16:15:15,900][20381] Updated weights for policy 1, policy_version 1529586 (0.0010) [2023-12-27 16:15:15,942][20381] Updated weights for policy 1, policy_version 1529596 (0.0010) [2023-12-27 16:15:15,995][20381] Updated weights for policy 1, policy_version 1529606 (0.0010) [2023-12-27 16:15:16,565][20347] Updated weights for policy 0, policy_version 1531156 (0.0009) [2023-12-27 16:15:16,620][20347] Updated weights for policy 0, policy_version 1531166 (0.0008) [2023-12-27 16:15:16,635][20381] Updated weights for policy 1, policy_version 1529616 (0.0008) [2023-12-27 16:15:16,673][20291] KL-divergence is very high: 100.7149 [2023-12-27 16:15:16,678][20347] Updated weights for policy 0, policy_version 1531176 (0.0008) [2023-12-27 16:15:16,693][20381] Updated weights for policy 1, policy_version 1529626 (0.0006) [2023-12-27 16:15:16,748][20381] Updated weights for policy 1, policy_version 1529636 (0.0007) [2023-12-27 16:15:17,306][20347] Updated weights for policy 0, policy_version 1531186 (0.0007) [2023-12-27 16:15:17,361][20347] Updated weights for policy 0, policy_version 1531196 (0.0010) [2023-12-27 16:15:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 783687680. Throughput: 0: 10371.9, 1: 10371.7. Samples: 783704104. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:17,363][19170] Avg episode reward: [(0, '911.990'), (1, '941.610')] [2023-12-27 16:15:17,376][20381] Updated weights for policy 1, policy_version 1529646 (0.0007) [2023-12-27 16:15:17,403][20291] KL-divergence is very high: 118.4049 [2023-12-27 16:15:17,409][20291] KL-divergence is very high: 112.8514 [2023-12-27 16:15:17,420][20347] Updated weights for policy 0, policy_version 1531206 (0.0010) [2023-12-27 16:15:17,436][20381] Updated weights for policy 1, policy_version 1529656 (0.0005) [2023-12-27 16:15:17,478][20347] Updated weights for policy 0, policy_version 1531216 (0.0010) [2023-12-27 16:15:17,485][20381] Updated weights for policy 1, policy_version 1529666 (0.0005) [2023-12-27 16:15:18,071][20347] Updated weights for policy 0, policy_version 1531226 (0.0005) [2023-12-27 16:15:18,105][20381] Updated weights for policy 1, policy_version 1529676 (0.0006) [2023-12-27 16:15:18,125][20347] Updated weights for policy 0, policy_version 1531236 (0.0006) [2023-12-27 16:15:18,164][20381] Updated weights for policy 1, policy_version 1529686 (0.0005) [2023-12-27 16:15:18,177][20347] Updated weights for policy 0, policy_version 1531246 (0.0007) [2023-12-27 16:15:18,220][20381] Updated weights for policy 1, policy_version 1529696 (0.0005) [2023-12-27 16:15:18,838][20381] Updated weights for policy 1, policy_version 1529706 (0.0006) [2023-12-27 16:15:18,846][20347] Updated weights for policy 0, policy_version 1531256 (0.0008) [2023-12-27 16:15:18,896][20381] Updated weights for policy 1, policy_version 1529716 (0.0006) [2023-12-27 16:15:18,904][20347] Updated weights for policy 0, policy_version 1531266 (0.0007) [2023-12-27 16:15:18,955][20347] Updated weights for policy 0, policy_version 1531276 (0.0008) [2023-12-27 16:15:18,958][20381] Updated weights for policy 1, policy_version 1529726 (0.0006) [2023-12-27 16:15:19,019][20381] Updated weights for policy 1, policy_version 1529736 (0.0009) [2023-12-27 16:15:19,637][20347] Updated weights for policy 0, policy_version 1531286 (0.0008) [2023-12-27 16:15:19,692][20347] Updated weights for policy 0, policy_version 1531296 (0.0007) [2023-12-27 16:15:19,724][20381] Updated weights for policy 1, policy_version 1529746 (0.0009) [2023-12-27 16:15:19,754][20347] Updated weights for policy 0, policy_version 1531306 (0.0008) [2023-12-27 16:15:19,780][20381] Updated weights for policy 1, policy_version 1529756 (0.0006) [2023-12-27 16:15:19,853][20381] Updated weights for policy 1, policy_version 1529767 (0.0007) [2023-12-27 16:15:20,498][20347] Updated weights for policy 0, policy_version 1531316 (0.0009) [2023-12-27 16:15:20,557][20347] Updated weights for policy 0, policy_version 1531326 (0.0010) [2023-12-27 16:15:20,588][20381] Updated weights for policy 1, policy_version 1529777 (0.0007) [2023-12-27 16:15:20,622][20347] Updated weights for policy 0, policy_version 1531336 (0.0007) [2023-12-27 16:15:20,651][20381] Updated weights for policy 1, policy_version 1529787 (0.0008) [2023-12-27 16:15:20,705][20381] Updated weights for policy 1, policy_version 1529797 (0.0008) [2023-12-27 16:15:21,310][20347] Updated weights for policy 0, policy_version 1531346 (0.0008) [2023-12-27 16:15:21,376][20347] Updated weights for policy 0, policy_version 1531356 (0.0010) [2023-12-27 16:15:21,440][20347] Updated weights for policy 0, policy_version 1531366 (0.0008) [2023-12-27 16:15:21,491][20381] Updated weights for policy 1, policy_version 1529807 (0.0007) [2023-12-27 16:15:21,504][20347] Updated weights for policy 0, policy_version 1531376 (0.0009) [2023-12-27 16:15:21,550][20381] Updated weights for policy 1, policy_version 1529817 (0.0007) [2023-12-27 16:15:21,609][20381] Updated weights for policy 1, policy_version 1529827 (0.0008) [2023-12-27 16:15:22,192][20347] Updated weights for policy 0, policy_version 1531386 (0.0007) [2023-12-27 16:15:22,245][20347] Updated weights for policy 0, policy_version 1531396 (0.0007) [2023-12-27 16:15:22,306][20347] Updated weights for policy 0, policy_version 1531406 (0.0007) [2023-12-27 16:15:22,348][20381] Updated weights for policy 1, policy_version 1529837 (0.0006) [2023-12-27 16:15:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 783794176. Throughput: 0: 10367.4, 1: 10335.2. Samples: 783766364. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:22,363][19170] Avg episode reward: [(0, '882.450'), (1, '951.340')] [2023-12-27 16:15:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001531408_392118272.pth... [2023-12-27 16:15:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001530192_391806976.pth [2023-12-27 16:15:22,415][20381] Updated weights for policy 1, policy_version 1529847 (0.0007) [2023-12-27 16:15:22,473][20381] Updated weights for policy 1, policy_version 1529857 (0.0006) [2023-12-27 16:15:22,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001529864_391684096.pth... [2023-12-27 16:15:22,517][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001528648_391372800.pth [2023-12-27 16:15:23,010][20381] Updated weights for policy 1, policy_version 1529867 (0.0006) [2023-12-27 16:15:23,067][20381] Updated weights for policy 1, policy_version 1529877 (0.0007) [2023-12-27 16:15:23,076][20347] Updated weights for policy 0, policy_version 1531416 (0.0008) [2023-12-27 16:15:23,115][20381] Updated weights for policy 1, policy_version 1529887 (0.0007) [2023-12-27 16:15:23,126][20347] Updated weights for policy 0, policy_version 1531426 (0.0006) [2023-12-27 16:15:23,184][20347] Updated weights for policy 0, policy_version 1531436 (0.0008) [2023-12-27 16:15:23,841][20381] Updated weights for policy 1, policy_version 1529897 (0.0008) [2023-12-27 16:15:23,847][20347] Updated weights for policy 0, policy_version 1531446 (0.0007) [2023-12-27 16:15:23,888][20381] Updated weights for policy 1, policy_version 1529907 (0.0006) [2023-12-27 16:15:23,902][20347] Updated weights for policy 0, policy_version 1531456 (0.0006) [2023-12-27 16:15:23,942][20381] Updated weights for policy 1, policy_version 1529917 (0.0007) [2023-12-27 16:15:23,959][20347] Updated weights for policy 0, policy_version 1531466 (0.0007) [2023-12-27 16:15:23,996][20381] Updated weights for policy 1, policy_version 1529927 (0.0005) [2023-12-27 16:15:24,607][20347] Updated weights for policy 0, policy_version 1531476 (0.0007) [2023-12-27 16:15:24,652][20381] Updated weights for policy 1, policy_version 1529937 (0.0006) [2023-12-27 16:15:24,659][20347] Updated weights for policy 0, policy_version 1531486 (0.0007) [2023-12-27 16:15:24,704][20347] Updated weights for policy 0, policy_version 1531496 (0.0007) [2023-12-27 16:15:24,715][20381] Updated weights for policy 1, policy_version 1529947 (0.0005) [2023-12-27 16:15:24,777][20381] Updated weights for policy 1, policy_version 1529957 (0.0005) [2023-12-27 16:15:25,347][20381] Updated weights for policy 1, policy_version 1529967 (0.0007) [2023-12-27 16:15:25,375][20347] Updated weights for policy 0, policy_version 1531506 (0.0007) [2023-12-27 16:15:25,400][20381] Updated weights for policy 1, policy_version 1529977 (0.0006) [2023-12-27 16:15:25,420][20347] Updated weights for policy 0, policy_version 1531516 (0.0005) [2023-12-27 16:15:25,455][20381] Updated weights for policy 1, policy_version 1529987 (0.0008) [2023-12-27 16:15:25,469][20347] Updated weights for policy 0, policy_version 1531526 (0.0006) [2023-12-27 16:15:25,522][20347] Updated weights for policy 0, policy_version 1531536 (0.0007) [2023-12-27 16:15:26,127][20291] KL-divergence is very high: 139.3212 [2023-12-27 16:15:26,127][20381] Updated weights for policy 1, policy_version 1529997 (0.0006) [2023-12-27 16:15:26,137][20347] Updated weights for policy 0, policy_version 1531546 (0.0008) [2023-12-27 16:15:26,165][20291] KL-divergence is very high: 166.7074 [2023-12-27 16:15:26,174][20381] Updated weights for policy 1, policy_version 1530007 (0.0006) [2023-12-27 16:15:26,186][20347] Updated weights for policy 0, policy_version 1531556 (0.0006) [2023-12-27 16:15:26,207][20291] KL-divergence is very high: 147.9493 [2023-12-27 16:15:26,224][20381] Updated weights for policy 1, policy_version 1530017 (0.0007) [2023-12-27 16:15:26,238][20347] Updated weights for policy 0, policy_version 1531566 (0.0006) [2023-12-27 16:15:26,915][20381] Updated weights for policy 1, policy_version 1530027 (0.0006) [2023-12-27 16:15:26,937][20347] Updated weights for policy 0, policy_version 1531576 (0.0005) [2023-12-27 16:15:26,963][20381] Updated weights for policy 1, policy_version 1530037 (0.0007) [2023-12-27 16:15:26,971][20292] KL-divergence is very high: 128.5613 [2023-12-27 16:15:26,976][20292] KL-divergence is very high: 102.8810 [2023-12-27 16:15:26,986][20347] Updated weights for policy 0, policy_version 1531586 (0.0005) [2023-12-27 16:15:26,998][20291] KL-divergence is very high: 115.1924 [2023-12-27 16:15:27,010][20292] KL-divergence is very high: 132.6854 [2023-12-27 16:15:27,012][20381] Updated weights for policy 1, policy_version 1530047 (0.0007) [2023-12-27 16:15:27,031][20347] Updated weights for policy 0, policy_version 1531596 (0.0005) [2023-12-27 16:15:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 783900672. Throughput: 0: 10334.3, 1: 10384.7. Samples: 783888788. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:27,363][19170] Avg episode reward: [(0, '765.280'), (1, '961.140')] [2023-12-27 16:15:27,702][20347] Updated weights for policy 0, policy_version 1531606 (0.0007) [2023-12-27 16:15:27,723][20381] Updated weights for policy 1, policy_version 1530057 (0.0007) [2023-12-27 16:15:27,749][20347] Updated weights for policy 0, policy_version 1531616 (0.0008) [2023-12-27 16:15:27,778][20381] Updated weights for policy 1, policy_version 1530067 (0.0006) [2023-12-27 16:15:27,795][20347] Updated weights for policy 0, policy_version 1531626 (0.0005) [2023-12-27 16:15:27,824][20381] Updated weights for policy 1, policy_version 1530077 (0.0005) [2023-12-27 16:15:27,872][20381] Updated weights for policy 1, policy_version 1530087 (0.0007) [2023-12-27 16:15:28,518][20381] Updated weights for policy 1, policy_version 1530097 (0.0006) [2023-12-27 16:15:28,531][20347] Updated weights for policy 0, policy_version 1531636 (0.0006) [2023-12-27 16:15:28,576][20381] Updated weights for policy 1, policy_version 1530107 (0.0006) [2023-12-27 16:15:28,581][20347] Updated weights for policy 0, policy_version 1531646 (0.0006) [2023-12-27 16:15:28,630][20347] Updated weights for policy 0, policy_version 1531656 (0.0008) [2023-12-27 16:15:28,637][20381] Updated weights for policy 1, policy_version 1530117 (0.0006) [2023-12-27 16:15:29,292][20381] Updated weights for policy 1, policy_version 1530127 (0.0007) [2023-12-27 16:15:29,351][20381] Updated weights for policy 1, policy_version 1530137 (0.0008) [2023-12-27 16:15:29,381][20347] Updated weights for policy 0, policy_version 1531666 (0.0008) [2023-12-27 16:15:29,412][20381] Updated weights for policy 1, policy_version 1530147 (0.0007) [2023-12-27 16:15:29,436][20347] Updated weights for policy 0, policy_version 1531676 (0.0006) [2023-12-27 16:15:29,493][20347] Updated weights for policy 0, policy_version 1531686 (0.0007) [2023-12-27 16:15:29,552][20347] Updated weights for policy 0, policy_version 1531696 (0.0008) [2023-12-27 16:15:30,116][20381] Updated weights for policy 1, policy_version 1530157 (0.0007) [2023-12-27 16:15:30,170][20381] Updated weights for policy 1, policy_version 1530167 (0.0008) [2023-12-27 16:15:30,227][20381] Updated weights for policy 1, policy_version 1530177 (0.0007) [2023-12-27 16:15:30,252][20347] Updated weights for policy 0, policy_version 1531706 (0.0005) [2023-12-27 16:15:30,308][20347] Updated weights for policy 0, policy_version 1531716 (0.0007) [2023-12-27 16:15:30,368][20347] Updated weights for policy 0, policy_version 1531726 (0.0007) [2023-12-27 16:15:30,898][20381] Updated weights for policy 1, policy_version 1530187 (0.0007) [2023-12-27 16:15:30,956][20381] Updated weights for policy 1, policy_version 1530197 (0.0006) [2023-12-27 16:15:30,967][20347] Updated weights for policy 0, policy_version 1531736 (0.0006) [2023-12-27 16:15:31,012][20381] Updated weights for policy 1, policy_version 1530207 (0.0006) [2023-12-27 16:15:31,025][20347] Updated weights for policy 0, policy_version 1531746 (0.0006) [2023-12-27 16:15:31,081][20347] Updated weights for policy 0, policy_version 1531756 (0.0009) [2023-12-27 16:15:31,739][20381] Updated weights for policy 1, policy_version 1530217 (0.0007) [2023-12-27 16:15:31,769][20347] Updated weights for policy 0, policy_version 1531766 (0.0008) [2023-12-27 16:15:31,796][20381] Updated weights for policy 1, policy_version 1530227 (0.0007) [2023-12-27 16:15:31,817][20347] Updated weights for policy 0, policy_version 1531776 (0.0008) [2023-12-27 16:15:31,849][20381] Updated weights for policy 1, policy_version 1530237 (0.0006) [2023-12-27 16:15:31,867][20347] Updated weights for policy 0, policy_version 1531786 (0.0006) [2023-12-27 16:15:31,905][20381] Updated weights for policy 1, policy_version 1530247 (0.0006) [2023-12-27 16:15:32,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 783998976. Throughput: 0: 10292.8, 1: 10344.1. Samples: 784009976. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:32,363][19170] Avg episode reward: [(0, '755.300'), (1, '970.890')] [2023-12-27 16:15:32,611][20381] Updated weights for policy 1, policy_version 1530257 (0.0008) [2023-12-27 16:15:32,622][20347] Updated weights for policy 0, policy_version 1531796 (0.0009) [2023-12-27 16:15:32,663][20381] Updated weights for policy 1, policy_version 1530267 (0.0009) [2023-12-27 16:15:32,672][20347] Updated weights for policy 0, policy_version 1531806 (0.0010) [2023-12-27 16:15:32,712][20381] Updated weights for policy 1, policy_version 1530277 (0.0005) [2023-12-27 16:15:32,724][20347] Updated weights for policy 0, policy_version 1531816 (0.0010) [2023-12-27 16:15:33,356][20347] Updated weights for policy 0, policy_version 1531826 (0.0010) [2023-12-27 16:15:33,404][20347] Updated weights for policy 0, policy_version 1531836 (0.0007) [2023-12-27 16:15:33,443][20381] Updated weights for policy 1, policy_version 1530287 (0.0006) [2023-12-27 16:15:33,457][20347] Updated weights for policy 0, policy_version 1531846 (0.0006) [2023-12-27 16:15:33,491][20381] Updated weights for policy 1, policy_version 1530297 (0.0008) [2023-12-27 16:15:33,514][20347] Updated weights for policy 0, policy_version 1531856 (0.0005) [2023-12-27 16:15:33,537][20381] Updated weights for policy 1, policy_version 1530307 (0.0007) [2023-12-27 16:15:34,048][20347] Updated weights for policy 0, policy_version 1531866 (0.0005) [2023-12-27 16:15:34,103][20347] Updated weights for policy 0, policy_version 1531876 (0.0005) [2023-12-27 16:15:34,170][20347] Updated weights for policy 0, policy_version 1531886 (0.0006) [2023-12-27 16:15:34,295][20381] Updated weights for policy 1, policy_version 1530317 (0.0007) [2023-12-27 16:15:34,352][20381] Updated weights for policy 1, policy_version 1530327 (0.0007) [2023-12-27 16:15:34,415][20381] Updated weights for policy 1, policy_version 1530337 (0.0010) [2023-12-27 16:15:34,835][20347] Updated weights for policy 0, policy_version 1531896 (0.0007) [2023-12-27 16:15:34,884][20347] Updated weights for policy 0, policy_version 1531906 (0.0007) [2023-12-27 16:15:34,944][20347] Updated weights for policy 0, policy_version 1531916 (0.0007) [2023-12-27 16:15:35,099][20381] Updated weights for policy 1, policy_version 1530347 (0.0010) [2023-12-27 16:15:35,170][20381] Updated weights for policy 1, policy_version 1530357 (0.0008) [2023-12-27 16:15:35,227][20381] Updated weights for policy 1, policy_version 1530367 (0.0007) [2023-12-27 16:15:35,555][20347] Updated weights for policy 0, policy_version 1531926 (0.0007) [2023-12-27 16:15:35,612][20347] Updated weights for policy 0, policy_version 1531936 (0.0007) [2023-12-27 16:15:35,667][20347] Updated weights for policy 0, policy_version 1531946 (0.0007) [2023-12-27 16:15:35,917][20381] Updated weights for policy 1, policy_version 1530377 (0.0010) [2023-12-27 16:15:35,982][20381] Updated weights for policy 1, policy_version 1530387 (0.0008) [2023-12-27 16:15:36,037][20381] Updated weights for policy 1, policy_version 1530397 (0.0008) [2023-12-27 16:15:36,098][20381] Updated weights for policy 1, policy_version 1530407 (0.0007) [2023-12-27 16:15:36,282][20347] Updated weights for policy 0, policy_version 1531956 (0.0007) [2023-12-27 16:15:36,339][20347] Updated weights for policy 0, policy_version 1531966 (0.0008) [2023-12-27 16:15:36,396][20347] Updated weights for policy 0, policy_version 1531976 (0.0009) [2023-12-27 16:15:36,783][20381] Updated weights for policy 1, policy_version 1530417 (0.0008) [2023-12-27 16:15:36,833][20381] Updated weights for policy 1, policy_version 1530427 (0.0008) [2023-12-27 16:15:36,880][20381] Updated weights for policy 1, policy_version 1530437 (0.0007) [2023-12-27 16:15:37,094][20347] Updated weights for policy 0, policy_version 1531986 (0.0007) [2023-12-27 16:15:37,157][20347] Updated weights for policy 0, policy_version 1531996 (0.0008) [2023-12-27 16:15:37,219][20347] Updated weights for policy 0, policy_version 1532006 (0.0008) [2023-12-27 16:15:37,279][20347] Updated weights for policy 0, policy_version 1532016 (0.0006) [2023-12-27 16:15:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 784105472. Throughput: 0: 10367.1, 1: 10314.5. Samples: 784072084. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:37,363][19170] Avg episode reward: [(0, '774.990'), (1, '990.340')] [2023-12-27 16:15:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001530440_391831552.pth... [2023-12-27 16:15:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001532016_392273920.pth... [2023-12-27 16:15:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001529256_391528448.pth [2023-12-27 16:15:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001530800_391962624.pth [2023-12-27 16:15:37,587][20381] Updated weights for policy 1, policy_version 1530447 (0.0009) [2023-12-27 16:15:37,646][20381] Updated weights for policy 1, policy_version 1530457 (0.0008) [2023-12-27 16:15:37,710][20381] Updated weights for policy 1, policy_version 1530467 (0.0006) [2023-12-27 16:15:37,913][20347] Updated weights for policy 0, policy_version 1532026 (0.0007) [2023-12-27 16:15:37,965][20347] Updated weights for policy 0, policy_version 1532036 (0.0008) [2023-12-27 16:15:38,022][20347] Updated weights for policy 0, policy_version 1532046 (0.0007) [2023-12-27 16:15:38,400][20381] Updated weights for policy 1, policy_version 1530477 (0.0007) [2023-12-27 16:15:38,455][20381] Updated weights for policy 1, policy_version 1530487 (0.0007) [2023-12-27 16:15:38,512][20381] Updated weights for policy 1, policy_version 1530497 (0.0008) [2023-12-27 16:15:38,733][20347] Updated weights for policy 0, policy_version 1532056 (0.0008) [2023-12-27 16:15:38,789][20347] Updated weights for policy 0, policy_version 1532066 (0.0008) [2023-12-27 16:15:38,836][20347] Updated weights for policy 0, policy_version 1532076 (0.0007) [2023-12-27 16:15:39,227][20381] Updated weights for policy 1, policy_version 1530507 (0.0008) [2023-12-27 16:15:39,291][20381] Updated weights for policy 1, policy_version 1530517 (0.0009) [2023-12-27 16:15:39,363][20381] Updated weights for policy 1, policy_version 1530527 (0.0008) [2023-12-27 16:15:39,602][20347] Updated weights for policy 0, policy_version 1532086 (0.0008) [2023-12-27 16:15:39,658][20347] Updated weights for policy 0, policy_version 1532096 (0.0008) [2023-12-27 16:15:39,715][20347] Updated weights for policy 0, policy_version 1532106 (0.0008) [2023-12-27 16:15:40,098][20381] Updated weights for policy 1, policy_version 1530537 (0.0009) [2023-12-27 16:15:40,153][20381] Updated weights for policy 1, policy_version 1530547 (0.0007) [2023-12-27 16:15:40,218][20381] Updated weights for policy 1, policy_version 1530557 (0.0007) [2023-12-27 16:15:40,282][20381] Updated weights for policy 1, policy_version 1530567 (0.0007) [2023-12-27 16:15:40,519][20347] Updated weights for policy 0, policy_version 1532116 (0.0008) [2023-12-27 16:15:40,583][20347] Updated weights for policy 0, policy_version 1532126 (0.0008) [2023-12-27 16:15:40,640][20347] Updated weights for policy 0, policy_version 1532136 (0.0008) [2023-12-27 16:15:40,941][20381] Updated weights for policy 1, policy_version 1530577 (0.0007) [2023-12-27 16:15:40,994][20381] Updated weights for policy 1, policy_version 1530587 (0.0008) [2023-12-27 16:15:41,063][20381] Updated weights for policy 1, policy_version 1530597 (0.0008) [2023-12-27 16:15:41,452][20347] Updated weights for policy 0, policy_version 1532146 (0.0007) [2023-12-27 16:15:41,515][20347] Updated weights for policy 0, policy_version 1532156 (0.0008) [2023-12-27 16:15:41,583][20347] Updated weights for policy 0, policy_version 1532166 (0.0008) [2023-12-27 16:15:41,647][20347] Updated weights for policy 0, policy_version 1532176 (0.0009) [2023-12-27 16:15:41,861][20381] Updated weights for policy 1, policy_version 1530607 (0.0008) [2023-12-27 16:15:41,928][20381] Updated weights for policy 1, policy_version 1530617 (0.0009) [2023-12-27 16:15:41,997][20381] Updated weights for policy 1, policy_version 1530627 (0.0009) [2023-12-27 16:15:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 784195584. Throughput: 0: 10302.9, 1: 10222.7. Samples: 784190308. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:42,363][19170] Avg episode reward: [(0, '814.170'), (1, '980.640')] [2023-12-27 16:15:42,499][20347] Updated weights for policy 0, policy_version 1532186 (0.0008) [2023-12-27 16:15:42,564][20347] Updated weights for policy 0, policy_version 1532196 (0.0007) [2023-12-27 16:15:42,624][20347] Updated weights for policy 0, policy_version 1532206 (0.0009) [2023-12-27 16:15:42,624][20291] KL-divergence is very high: 122.7941 [2023-12-27 16:15:42,631][20291] KL-divergence is very high: 141.0100 [2023-12-27 16:15:42,772][20381] Updated weights for policy 1, policy_version 1530637 (0.0008) [2023-12-27 16:15:42,828][20381] Updated weights for policy 1, policy_version 1530647 (0.0006) [2023-12-27 16:15:42,890][20381] Updated weights for policy 1, policy_version 1530657 (0.0005) [2023-12-27 16:15:43,294][20291] KL-divergence is very high: 131.9439 [2023-12-27 16:15:43,333][20347] Updated weights for policy 0, policy_version 1532216 (0.0007) [2023-12-27 16:15:43,394][20347] Updated weights for policy 0, policy_version 1532226 (0.0008) [2023-12-27 16:15:43,452][20347] Updated weights for policy 0, policy_version 1532236 (0.0007) [2023-12-27 16:15:43,595][20381] Updated weights for policy 1, policy_version 1530667 (0.0008) [2023-12-27 16:15:43,646][20381] Updated weights for policy 1, policy_version 1530677 (0.0007) [2023-12-27 16:15:43,697][20381] Updated weights for policy 1, policy_version 1530687 (0.0007) [2023-12-27 16:15:44,155][20347] Updated weights for policy 0, policy_version 1532246 (0.0007) [2023-12-27 16:15:44,219][20347] Updated weights for policy 0, policy_version 1532256 (0.0007) [2023-12-27 16:15:44,286][20347] Updated weights for policy 0, policy_version 1532266 (0.0007) [2023-12-27 16:15:44,372][20381] Updated weights for policy 1, policy_version 1530697 (0.0007) [2023-12-27 16:15:44,425][20381] Updated weights for policy 1, policy_version 1530707 (0.0007) [2023-12-27 16:15:44,483][20381] Updated weights for policy 1, policy_version 1530717 (0.0008) [2023-12-27 16:15:44,541][20381] Updated weights for policy 1, policy_version 1530727 (0.0007) [2023-12-27 16:15:44,985][20347] Updated weights for policy 0, policy_version 1532276 (0.0007) [2023-12-27 16:15:45,047][20347] Updated weights for policy 0, policy_version 1532286 (0.0006) [2023-12-27 16:15:45,111][20347] Updated weights for policy 0, policy_version 1532296 (0.0007) [2023-12-27 16:15:45,246][20381] Updated weights for policy 1, policy_version 1530737 (0.0007) [2023-12-27 16:15:45,311][20381] Updated weights for policy 1, policy_version 1530747 (0.0007) [2023-12-27 16:15:45,373][20381] Updated weights for policy 1, policy_version 1530757 (0.0008) [2023-12-27 16:15:45,729][20347] Updated weights for policy 0, policy_version 1532306 (0.0011) [2023-12-27 16:15:45,784][20347] Updated weights for policy 0, policy_version 1532316 (0.0010) [2023-12-27 16:15:45,841][20347] Updated weights for policy 0, policy_version 1532326 (0.0010) [2023-12-27 16:15:45,890][20347] Updated weights for policy 0, policy_version 1532336 (0.0010) [2023-12-27 16:15:46,023][20381] Updated weights for policy 1, policy_version 1530767 (0.0008) [2023-12-27 16:15:46,075][20381] Updated weights for policy 1, policy_version 1530777 (0.0008) [2023-12-27 16:15:46,122][20381] Updated weights for policy 1, policy_version 1530787 (0.0007) [2023-12-27 16:15:46,556][20347] Updated weights for policy 0, policy_version 1532346 (0.0006) [2023-12-27 16:15:46,605][20347] Updated weights for policy 0, policy_version 1532356 (0.0005) [2023-12-27 16:15:46,659][20347] Updated weights for policy 0, policy_version 1532366 (0.0006) [2023-12-27 16:15:46,837][20381] Updated weights for policy 1, policy_version 1530797 (0.0008) [2023-12-27 16:15:46,900][20381] Updated weights for policy 1, policy_version 1530807 (0.0009) [2023-12-27 16:15:46,963][20381] Updated weights for policy 1, policy_version 1530817 (0.0008) [2023-12-27 16:15:47,345][20347] Updated weights for policy 0, policy_version 1532376 (0.0008) [2023-12-27 16:15:47,362][19170] Fps is (10 sec: 18841.7, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 784293888. Throughput: 0: 10221.2, 1: 10147.2. Samples: 784306804. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:47,363][19170] Avg episode reward: [(0, '813.840'), (1, '970.820')] [2023-12-27 16:15:47,397][20347] Updated weights for policy 0, policy_version 1532386 (0.0008) [2023-12-27 16:15:47,443][20347] Updated weights for policy 0, policy_version 1532396 (0.0007) [2023-12-27 16:15:47,602][20381] Updated weights for policy 1, policy_version 1530827 (0.0007) [2023-12-27 16:15:47,651][20381] Updated weights for policy 1, policy_version 1530837 (0.0005) [2023-12-27 16:15:47,705][20381] Updated weights for policy 1, policy_version 1530847 (0.0005) [2023-12-27 16:15:48,151][20347] Updated weights for policy 0, policy_version 1532406 (0.0007) [2023-12-27 16:15:48,200][20347] Updated weights for policy 0, policy_version 1532416 (0.0005) [2023-12-27 16:15:48,247][20381] Updated weights for policy 1, policy_version 1530857 (0.0005) [2023-12-27 16:15:48,251][20347] Updated weights for policy 0, policy_version 1532426 (0.0006) [2023-12-27 16:15:48,298][20381] Updated weights for policy 1, policy_version 1530867 (0.0005) [2023-12-27 16:15:48,357][20381] Updated weights for policy 1, policy_version 1530877 (0.0007) [2023-12-27 16:15:48,414][20381] Updated weights for policy 1, policy_version 1530887 (0.0008) [2023-12-27 16:15:48,947][20347] Updated weights for policy 0, policy_version 1532436 (0.0006) [2023-12-27 16:15:49,005][20347] Updated weights for policy 0, policy_version 1532446 (0.0007) [2023-12-27 16:15:49,056][20347] Updated weights for policy 0, policy_version 1532456 (0.0007) [2023-12-27 16:15:49,124][20381] Updated weights for policy 1, policy_version 1530897 (0.0010) [2023-12-27 16:15:49,184][20381] Updated weights for policy 1, policy_version 1530907 (0.0007) [2023-12-27 16:15:49,246][20381] Updated weights for policy 1, policy_version 1530917 (0.0008) [2023-12-27 16:15:49,692][20347] Updated weights for policy 0, policy_version 1532466 (0.0007) [2023-12-27 16:15:49,754][20347] Updated weights for policy 0, policy_version 1532476 (0.0007) [2023-12-27 16:15:49,809][20347] Updated weights for policy 0, policy_version 1532486 (0.0007) [2023-12-27 16:15:49,875][20347] Updated weights for policy 0, policy_version 1532496 (0.0008) [2023-12-27 16:15:49,998][20381] Updated weights for policy 1, policy_version 1530927 (0.0008) [2023-12-27 16:15:50,055][20381] Updated weights for policy 1, policy_version 1530937 (0.0008) [2023-12-27 16:15:50,114][20381] Updated weights for policy 1, policy_version 1530947 (0.0008) [2023-12-27 16:15:50,613][20347] Updated weights for policy 0, policy_version 1532506 (0.0009) [2023-12-27 16:15:50,660][20347] Updated weights for policy 0, policy_version 1532516 (0.0008) [2023-12-27 16:15:50,716][20347] Updated weights for policy 0, policy_version 1532526 (0.0008) [2023-12-27 16:15:50,789][20381] Updated weights for policy 1, policy_version 1530957 (0.0007) [2023-12-27 16:15:50,845][20381] Updated weights for policy 1, policy_version 1530967 (0.0006) [2023-12-27 16:15:50,903][20381] Updated weights for policy 1, policy_version 1530977 (0.0007) [2023-12-27 16:15:51,439][20347] Updated weights for policy 0, policy_version 1532536 (0.0008) [2023-12-27 16:15:51,501][20347] Updated weights for policy 0, policy_version 1532546 (0.0007) [2023-12-27 16:15:51,561][20347] Updated weights for policy 0, policy_version 1532556 (0.0005) [2023-12-27 16:15:51,604][20381] Updated weights for policy 1, policy_version 1530987 (0.0008) [2023-12-27 16:15:51,670][20381] Updated weights for policy 1, policy_version 1530997 (0.0008) [2023-12-27 16:15:51,731][20381] Updated weights for policy 1, policy_version 1531007 (0.0008) [2023-12-27 16:15:52,269][20347] Updated weights for policy 0, policy_version 1532566 (0.0008) [2023-12-27 16:15:52,336][20347] Updated weights for policy 0, policy_version 1532576 (0.0010) [2023-12-27 16:15:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 784392192. Throughput: 0: 10164.2, 1: 10114.9. Samples: 784368704. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:52,363][19170] Avg episode reward: [(0, '872.750'), (1, '980.520')] [2023-12-27 16:15:52,368][20381] Updated weights for policy 1, policy_version 1531017 (0.0008) [2023-12-27 16:15:52,405][20347] Updated weights for policy 0, policy_version 1532586 (0.0009) [2023-12-27 16:15:52,430][20381] Updated weights for policy 1, policy_version 1531027 (0.0007) [2023-12-27 16:15:52,436][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001532592_392421376.pth... [2023-12-27 16:15:52,440][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001531408_392118272.pth [2023-12-27 16:15:52,495][20381] Updated weights for policy 1, policy_version 1531037 (0.0006) [2023-12-27 16:15:52,556][20381] Updated weights for policy 1, policy_version 1531047 (0.0009) [2023-12-27 16:15:52,559][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001531048_391987200.pth... [2023-12-27 16:15:52,562][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001529864_391684096.pth [2023-12-27 16:15:53,084][20347] Updated weights for policy 0, policy_version 1532596 (0.0008) [2023-12-27 16:15:53,146][20347] Updated weights for policy 0, policy_version 1532606 (0.0006) [2023-12-27 16:15:53,197][20347] Updated weights for policy 0, policy_version 1532616 (0.0007) [2023-12-27 16:15:53,242][20381] Updated weights for policy 1, policy_version 1531057 (0.0007) [2023-12-27 16:15:53,290][20381] Updated weights for policy 1, policy_version 1531067 (0.0008) [2023-12-27 16:15:53,345][20381] Updated weights for policy 1, policy_version 1531077 (0.0007) [2023-12-27 16:15:53,879][20347] Updated weights for policy 0, policy_version 1532626 (0.0007) [2023-12-27 16:15:53,927][20347] Updated weights for policy 0, policy_version 1532636 (0.0007) [2023-12-27 16:15:53,974][20347] Updated weights for policy 0, policy_version 1532646 (0.0007) [2023-12-27 16:15:54,022][20347] Updated weights for policy 0, policy_version 1532656 (0.0007) [2023-12-27 16:15:54,058][20381] Updated weights for policy 1, policy_version 1531087 (0.0009) [2023-12-27 16:15:54,109][20381] Updated weights for policy 1, policy_version 1531097 (0.0011) [2023-12-27 16:15:54,156][20381] Updated weights for policy 1, policy_version 1531107 (0.0011) [2023-12-27 16:15:54,694][20347] Updated weights for policy 0, policy_version 1532666 (0.0007) [2023-12-27 16:15:54,756][20347] Updated weights for policy 0, policy_version 1532676 (0.0006) [2023-12-27 16:15:54,804][20347] Updated weights for policy 0, policy_version 1532686 (0.0006) [2023-12-27 16:15:54,865][20381] Updated weights for policy 1, policy_version 1531117 (0.0011) [2023-12-27 16:15:54,917][20381] Updated weights for policy 1, policy_version 1531127 (0.0010) [2023-12-27 16:15:54,970][20381] Updated weights for policy 1, policy_version 1531137 (0.0011) [2023-12-27 16:15:55,401][20347] Updated weights for policy 0, policy_version 1532696 (0.0005) [2023-12-27 16:15:55,470][20347] Updated weights for policy 0, policy_version 1532706 (0.0005) [2023-12-27 16:15:55,527][20347] Updated weights for policy 0, policy_version 1532716 (0.0008) [2023-12-27 16:15:55,668][20381] Updated weights for policy 1, policy_version 1531147 (0.0010) [2023-12-27 16:15:55,720][20381] Updated weights for policy 1, policy_version 1531157 (0.0008) [2023-12-27 16:15:55,781][20381] Updated weights for policy 1, policy_version 1531167 (0.0007) [2023-12-27 16:15:56,130][20347] Updated weights for policy 0, policy_version 1532726 (0.0007) [2023-12-27 16:15:56,182][20347] Updated weights for policy 0, policy_version 1532736 (0.0009) [2023-12-27 16:15:56,236][20347] Updated weights for policy 0, policy_version 1532746 (0.0005) [2023-12-27 16:15:56,387][20381] Updated weights for policy 1, policy_version 1531177 (0.0006) [2023-12-27 16:15:56,438][20381] Updated weights for policy 1, policy_version 1531187 (0.0005) [2023-12-27 16:15:56,491][20381] Updated weights for policy 1, policy_version 1531197 (0.0006) [2023-12-27 16:15:56,541][20381] Updated weights for policy 1, policy_version 1531207 (0.0007) [2023-12-27 16:15:56,893][20347] Updated weights for policy 0, policy_version 1532756 (0.0007) [2023-12-27 16:15:56,947][20347] Updated weights for policy 0, policy_version 1532766 (0.0007) [2023-12-27 16:15:57,004][20347] Updated weights for policy 0, policy_version 1532776 (0.0008) [2023-12-27 16:15:57,213][20381] Updated weights for policy 1, policy_version 1531217 (0.0007) [2023-12-27 16:15:57,269][20381] Updated weights for policy 1, policy_version 1531227 (0.0007) [2023-12-27 16:15:57,326][20381] Updated weights for policy 1, policy_version 1531237 (0.0007) [2023-12-27 16:15:57,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 784506880. Throughput: 0: 10172.2, 1: 10095.8. Samples: 784491204. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:15:57,364][19170] Avg episode reward: [(0, '872.670'), (1, '990.300')] [2023-12-27 16:15:57,618][20347] Updated weights for policy 0, policy_version 1532786 (0.0010) [2023-12-27 16:15:57,672][20347] Updated weights for policy 0, policy_version 1532796 (0.0010) [2023-12-27 16:15:57,726][20347] Updated weights for policy 0, policy_version 1532806 (0.0010) [2023-12-27 16:15:57,771][20347] Updated weights for policy 0, policy_version 1532816 (0.0010) [2023-12-27 16:15:58,032][20381] Updated weights for policy 1, policy_version 1531247 (0.0007) [2023-12-27 16:15:58,085][20381] Updated weights for policy 1, policy_version 1531257 (0.0007) [2023-12-27 16:15:58,140][20381] Updated weights for policy 1, policy_version 1531267 (0.0007) [2023-12-27 16:15:58,490][20291] KL-divergence is very high: 539.4122 [2023-12-27 16:15:58,494][20347] Updated weights for policy 0, policy_version 1532826 (0.0008) [2023-12-27 16:15:58,540][20291] KL-divergence is very high: 760.1682 [2023-12-27 16:15:58,561][20347] Updated weights for policy 0, policy_version 1532836 (0.0008) [2023-12-27 16:15:58,593][20291] KL-divergence is very high: 513.2610 [2023-12-27 16:15:58,622][20347] Updated weights for policy 0, policy_version 1532846 (0.0011) [2023-12-27 16:15:58,960][20381] Updated weights for policy 1, policy_version 1531277 (0.0007) [2023-12-27 16:15:59,017][20381] Updated weights for policy 1, policy_version 1531287 (0.0007) [2023-12-27 16:15:59,072][20381] Updated weights for policy 1, policy_version 1531297 (0.0008) [2023-12-27 16:15:59,313][20291] KL-divergence is very high: 244.3265 [2023-12-27 16:15:59,351][20347] Updated weights for policy 0, policy_version 1532856 (0.0009) [2023-12-27 16:15:59,363][20291] KL-divergence is very high: 206.7384 [2023-12-27 16:15:59,414][20347] Updated weights for policy 0, policy_version 1532866 (0.0008) [2023-12-27 16:15:59,416][20291] KL-divergence is very high: 236.9356 [2023-12-27 16:15:59,466][20291] KL-divergence is very high: 213.6703 [2023-12-27 16:15:59,479][20347] Updated weights for policy 0, policy_version 1532876 (0.0008) [2023-12-27 16:15:59,844][20381] Updated weights for policy 1, policy_version 1531307 (0.0007) [2023-12-27 16:15:59,904][20381] Updated weights for policy 1, policy_version 1531317 (0.0007) [2023-12-27 16:15:59,971][20381] Updated weights for policy 1, policy_version 1531327 (0.0007) [2023-12-27 16:16:00,123][20347] Updated weights for policy 0, policy_version 1532886 (0.0007) [2023-12-27 16:16:00,180][20347] Updated weights for policy 0, policy_version 1532896 (0.0008) [2023-12-27 16:16:00,234][20347] Updated weights for policy 0, policy_version 1532906 (0.0008) [2023-12-27 16:16:00,662][20381] Updated weights for policy 1, policy_version 1531337 (0.0007) [2023-12-27 16:16:00,709][20381] Updated weights for policy 1, policy_version 1531347 (0.0008) [2023-12-27 16:16:00,757][20381] Updated weights for policy 1, policy_version 1531357 (0.0007) [2023-12-27 16:16:00,799][20381] Updated weights for policy 1, policy_version 1531367 (0.0007) [2023-12-27 16:16:00,910][20347] Updated weights for policy 0, policy_version 1532916 (0.0008) [2023-12-27 16:16:00,958][20347] Updated weights for policy 0, policy_version 1532926 (0.0008) [2023-12-27 16:16:01,011][20347] Updated weights for policy 0, policy_version 1532936 (0.0008) [2023-12-27 16:16:01,562][20381] Updated weights for policy 1, policy_version 1531377 (0.0007) [2023-12-27 16:16:01,624][20381] Updated weights for policy 1, policy_version 1531387 (0.0009) [2023-12-27 16:16:01,681][20381] Updated weights for policy 1, policy_version 1531397 (0.0010) [2023-12-27 16:16:01,779][20347] Updated weights for policy 0, policy_version 1532946 (0.0009) [2023-12-27 16:16:01,845][20347] Updated weights for policy 0, policy_version 1532956 (0.0008) [2023-12-27 16:16:01,908][20347] Updated weights for policy 0, policy_version 1532966 (0.0006) [2023-12-27 16:16:01,957][20347] Updated weights for policy 0, policy_version 1532976 (0.0009) [2023-12-27 16:16:02,302][20381] Updated weights for policy 1, policy_version 1531407 (0.0011) [2023-12-27 16:16:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 784596992. Throughput: 0: 10115.1, 1: 10018.7. Samples: 784610124. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:16:02,363][19170] Avg episode reward: [(0, '872.600'), (1, '970.830')] [2023-12-27 16:16:02,367][20381] Updated weights for policy 1, policy_version 1531417 (0.0009) [2023-12-27 16:16:02,433][20381] Updated weights for policy 1, policy_version 1531427 (0.0005) [2023-12-27 16:16:02,647][20347] Updated weights for policy 0, policy_version 1532986 (0.0006) [2023-12-27 16:16:02,692][20347] Updated weights for policy 0, policy_version 1532996 (0.0007) [2023-12-27 16:16:02,743][20347] Updated weights for policy 0, policy_version 1533006 (0.0007) [2023-12-27 16:16:03,083][20381] Updated weights for policy 1, policy_version 1531437 (0.0010) [2023-12-27 16:16:03,144][20381] Updated weights for policy 1, policy_version 1531447 (0.0010) [2023-12-27 16:16:03,208][20381] Updated weights for policy 1, policy_version 1531457 (0.0011) [2023-12-27 16:16:03,354][20347] Updated weights for policy 0, policy_version 1533016 (0.0006) [2023-12-27 16:16:03,411][20347] Updated weights for policy 0, policy_version 1533026 (0.0007) [2023-12-27 16:16:03,465][20347] Updated weights for policy 0, policy_version 1533036 (0.0005) [2023-12-27 16:16:03,891][20381] Updated weights for policy 1, policy_version 1531467 (0.0011) [2023-12-27 16:16:03,951][20381] Updated weights for policy 1, policy_version 1531477 (0.0010) [2023-12-27 16:16:04,004][20381] Updated weights for policy 1, policy_version 1531487 (0.0009) [2023-12-27 16:16:04,055][20347] Updated weights for policy 0, policy_version 1533046 (0.0006) [2023-12-27 16:16:04,116][20347] Updated weights for policy 0, policy_version 1533056 (0.0006) [2023-12-27 16:16:04,177][20347] Updated weights for policy 0, policy_version 1533066 (0.0007) [2023-12-27 16:16:04,702][20381] Updated weights for policy 1, policy_version 1531497 (0.0007) [2023-12-27 16:16:04,756][20381] Updated weights for policy 1, policy_version 1531507 (0.0005) [2023-12-27 16:16:04,809][20381] Updated weights for policy 1, policy_version 1531517 (0.0005) [2023-12-27 16:16:04,836][20347] Updated weights for policy 0, policy_version 1533076 (0.0007) [2023-12-27 16:16:04,861][20381] Updated weights for policy 1, policy_version 1531527 (0.0005) [2023-12-27 16:16:04,890][20347] Updated weights for policy 0, policy_version 1533086 (0.0005) [2023-12-27 16:16:04,943][20347] Updated weights for policy 0, policy_version 1533096 (0.0006) [2023-12-27 16:16:05,502][20381] Updated weights for policy 1, policy_version 1531537 (0.0007) [2023-12-27 16:16:05,547][20347] Updated weights for policy 0, policy_version 1533106 (0.0005) [2023-12-27 16:16:05,548][20381] Updated weights for policy 1, policy_version 1531547 (0.0007) [2023-12-27 16:16:05,602][20347] Updated weights for policy 0, policy_version 1533116 (0.0005) [2023-12-27 16:16:05,605][20381] Updated weights for policy 1, policy_version 1531557 (0.0006) [2023-12-27 16:16:05,662][20347] Updated weights for policy 0, policy_version 1533126 (0.0006) [2023-12-27 16:16:05,709][20347] Updated weights for policy 0, policy_version 1533136 (0.0008) [2023-12-27 16:16:06,238][20381] Updated weights for policy 1, policy_version 1531567 (0.0008) [2023-12-27 16:16:06,292][20381] Updated weights for policy 1, policy_version 1531577 (0.0008) [2023-12-27 16:16:06,340][20381] Updated weights for policy 1, policy_version 1531587 (0.0008) [2023-12-27 16:16:06,403][20347] Updated weights for policy 0, policy_version 1533146 (0.0008) [2023-12-27 16:16:06,463][20347] Updated weights for policy 0, policy_version 1533156 (0.0008) [2023-12-27 16:16:06,516][20347] Updated weights for policy 0, policy_version 1533166 (0.0008) [2023-12-27 16:16:07,071][20381] Updated weights for policy 1, policy_version 1531597 (0.0009) [2023-12-27 16:16:07,123][20381] Updated weights for policy 1, policy_version 1531607 (0.0010) [2023-12-27 16:16:07,156][20347] Updated weights for policy 0, policy_version 1533176 (0.0005) [2023-12-27 16:16:07,171][20381] Updated weights for policy 1, policy_version 1531617 (0.0010) [2023-12-27 16:16:07,215][20347] Updated weights for policy 0, policy_version 1533186 (0.0006) [2023-12-27 16:16:07,272][20347] Updated weights for policy 0, policy_version 1533196 (0.0007) [2023-12-27 16:16:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20604.9). Total num frames: 784711680. Throughput: 0: 10146.7, 1: 10016.9. Samples: 784673732. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:16:07,364][19170] Avg episode reward: [(0, '872.810'), (1, '922.910')] [2023-12-27 16:16:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001531624_392134656.pth... [2023-12-27 16:16:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001533200_392577024.pth... [2023-12-27 16:16:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001530440_391831552.pth [2023-12-27 16:16:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001532016_392273920.pth [2023-12-27 16:16:07,805][20381] Updated weights for policy 1, policy_version 1531627 (0.0011) [2023-12-27 16:16:07,863][20381] Updated weights for policy 1, policy_version 1531637 (0.0010) [2023-12-27 16:16:07,911][20381] Updated weights for policy 1, policy_version 1531647 (0.0010) [2023-12-27 16:16:07,969][20347] Updated weights for policy 0, policy_version 1533206 (0.0009) [2023-12-27 16:16:08,021][20347] Updated weights for policy 0, policy_version 1533216 (0.0008) [2023-12-27 16:16:08,074][20347] Updated weights for policy 0, policy_version 1533226 (0.0008) [2023-12-27 16:16:08,609][20381] Updated weights for policy 1, policy_version 1531657 (0.0010) [2023-12-27 16:16:08,658][20381] Updated weights for policy 1, policy_version 1531667 (0.0008) [2023-12-27 16:16:08,710][20381] Updated weights for policy 1, policy_version 1531677 (0.0007) [2023-12-27 16:16:08,756][20347] Updated weights for policy 0, policy_version 1533236 (0.0007) [2023-12-27 16:16:08,770][20381] Updated weights for policy 1, policy_version 1531687 (0.0008) [2023-12-27 16:16:08,817][20347] Updated weights for policy 0, policy_version 1533246 (0.0007) [2023-12-27 16:16:08,878][20347] Updated weights for policy 0, policy_version 1533256 (0.0008) [2023-12-27 16:16:09,488][20381] Updated weights for policy 1, policy_version 1531697 (0.0008) [2023-12-27 16:16:09,548][20381] Updated weights for policy 1, policy_version 1531707 (0.0008) [2023-12-27 16:16:09,576][20347] Updated weights for policy 0, policy_version 1533266 (0.0008) [2023-12-27 16:16:09,605][20381] Updated weights for policy 1, policy_version 1531717 (0.0008) [2023-12-27 16:16:09,636][20347] Updated weights for policy 0, policy_version 1533276 (0.0006) [2023-12-27 16:16:09,698][20347] Updated weights for policy 0, policy_version 1533286 (0.0007) [2023-12-27 16:16:09,785][20347] Updated weights for policy 0, policy_version 1533296 (0.0008) [2023-12-27 16:16:10,354][20381] Updated weights for policy 1, policy_version 1531727 (0.0007) [2023-12-27 16:16:10,417][20381] Updated weights for policy 1, policy_version 1531737 (0.0008) [2023-12-27 16:16:10,438][20347] Updated weights for policy 0, policy_version 1533306 (0.0007) [2023-12-27 16:16:10,473][20381] Updated weights for policy 1, policy_version 1531747 (0.0007) [2023-12-27 16:16:10,496][20347] Updated weights for policy 0, policy_version 1533316 (0.0006) [2023-12-27 16:16:10,564][20347] Updated weights for policy 0, policy_version 1533326 (0.0005) [2023-12-27 16:16:11,138][20347] Updated weights for policy 0, policy_version 1533336 (0.0007) [2023-12-27 16:16:11,203][20347] Updated weights for policy 0, policy_version 1533346 (0.0008) [2023-12-27 16:16:11,242][20381] Updated weights for policy 1, policy_version 1531757 (0.0006) [2023-12-27 16:16:11,268][20347] Updated weights for policy 0, policy_version 1533356 (0.0006) [2023-12-27 16:16:11,305][20381] Updated weights for policy 1, policy_version 1531767 (0.0008) [2023-12-27 16:16:11,369][20381] Updated weights for policy 1, policy_version 1531777 (0.0008) [2023-12-27 16:16:12,037][20347] Updated weights for policy 0, policy_version 1533366 (0.0007) [2023-12-27 16:16:12,090][20347] Updated weights for policy 0, policy_version 1533376 (0.0008) [2023-12-27 16:16:12,124][20381] Updated weights for policy 1, policy_version 1531787 (0.0007) [2023-12-27 16:16:12,145][20347] Updated weights for policy 0, policy_version 1533386 (0.0006) [2023-12-27 16:16:12,178][20381] Updated weights for policy 1, policy_version 1531797 (0.0006) [2023-12-27 16:16:12,238][20381] Updated weights for policy 1, policy_version 1531807 (0.0008) [2023-12-27 16:16:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 784809984. Throughput: 0: 10178.5, 1: 9994.0. Samples: 784796552. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-12-27 16:16:12,363][19170] Avg episode reward: [(0, '804.180'), (1, '941.550')] [2023-12-27 16:16:12,810][20347] Updated weights for policy 0, policy_version 1533396 (0.0006) [2023-12-27 16:16:12,863][20347] Updated weights for policy 0, policy_version 1533406 (0.0007) [2023-12-27 16:16:12,918][20347] Updated weights for policy 0, policy_version 1533416 (0.0007) [2023-12-27 16:16:12,974][20381] Updated weights for policy 1, policy_version 1531817 (0.0008) [2023-12-27 16:16:13,023][20381] Updated weights for policy 1, policy_version 1531827 (0.0011) [2023-12-27 16:16:13,074][20381] Updated weights for policy 1, policy_version 1531837 (0.0010) [2023-12-27 16:16:13,126][20381] Updated weights for policy 1, policy_version 1531847 (0.0010) [2023-12-27 16:16:13,588][20347] Updated weights for policy 0, policy_version 1533426 (0.0007) [2023-12-27 16:16:13,644][20347] Updated weights for policy 0, policy_version 1533436 (0.0007) [2023-12-27 16:16:13,702][20347] Updated weights for policy 0, policy_version 1533446 (0.0005) [2023-12-27 16:16:13,762][20347] Updated weights for policy 0, policy_version 1533456 (0.0007) [2023-12-27 16:16:13,841][20381] Updated weights for policy 1, policy_version 1531857 (0.0011) [2023-12-27 16:16:13,896][20381] Updated weights for policy 1, policy_version 1531867 (0.0010) [2023-12-27 16:16:13,944][20381] Updated weights for policy 1, policy_version 1531877 (0.0006) [2023-12-27 16:16:14,393][20347] Updated weights for policy 0, policy_version 1533466 (0.0007) [2023-12-27 16:16:14,443][20347] Updated weights for policy 0, policy_version 1533476 (0.0007) [2023-12-27 16:16:14,494][20347] Updated weights for policy 0, policy_version 1533486 (0.0007) [2023-12-27 16:16:14,611][20381] Updated weights for policy 1, policy_version 1531887 (0.0009) [2023-12-27 16:16:14,668][20381] Updated weights for policy 1, policy_version 1531897 (0.0007) [2023-12-27 16:16:14,723][20381] Updated weights for policy 1, policy_version 1531907 (0.0008) [2023-12-27 16:16:15,185][20347] Updated weights for policy 0, policy_version 1533496 (0.0006) [2023-12-27 16:16:15,246][20347] Updated weights for policy 0, policy_version 1533506 (0.0008) [2023-12-27 16:16:15,307][20347] Updated weights for policy 0, policy_version 1533516 (0.0008) [2023-12-27 16:16:15,466][20381] Updated weights for policy 1, policy_version 1531917 (0.0009) [2023-12-27 16:16:15,524][20381] Updated weights for policy 1, policy_version 1531927 (0.0010) [2023-12-27 16:16:15,578][20381] Updated weights for policy 1, policy_version 1531937 (0.0010) [2023-12-27 16:16:15,984][20347] Updated weights for policy 0, policy_version 1533526 (0.0009) [2023-12-27 16:16:16,035][20347] Updated weights for policy 0, policy_version 1533536 (0.0010) [2023-12-27 16:16:16,080][20347] Updated weights for policy 0, policy_version 1533546 (0.0010) [2023-12-27 16:16:16,234][20381] Updated weights for policy 1, policy_version 1531947 (0.0010) [2023-12-27 16:16:16,290][20381] Updated weights for policy 1, policy_version 1531957 (0.0011) [2023-12-27 16:16:16,346][20381] Updated weights for policy 1, policy_version 1531967 (0.0011) [2023-12-27 16:16:16,745][20347] Updated weights for policy 0, policy_version 1533556 (0.0010) [2023-12-27 16:16:16,806][20347] Updated weights for policy 0, policy_version 1533566 (0.0010) [2023-12-27 16:16:16,864][20347] Updated weights for policy 0, policy_version 1533576 (0.0010) [2023-12-27 16:16:17,011][20381] Updated weights for policy 1, policy_version 1531977 (0.0010) [2023-12-27 16:16:17,063][20381] Updated weights for policy 1, policy_version 1531987 (0.0010) [2023-12-27 16:16:17,111][20381] Updated weights for policy 1, policy_version 1531997 (0.0011) [2023-12-27 16:16:17,161][20381] Updated weights for policy 1, policy_version 1532007 (0.0009) [2023-12-27 16:16:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 784908288. Throughput: 0: 10201.6, 1: 9951.9. Samples: 784916884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:17,363][19170] Avg episode reward: [(0, '803.940'), (1, '1000.000')] [2023-12-27 16:16:17,521][20347] Updated weights for policy 0, policy_version 1533586 (0.0010) [2023-12-27 16:16:17,582][20347] Updated weights for policy 0, policy_version 1533596 (0.0010) [2023-12-27 16:16:17,643][20347] Updated weights for policy 0, policy_version 1533606 (0.0010) [2023-12-27 16:16:17,695][20347] Updated weights for policy 0, policy_version 1533616 (0.0010) [2023-12-27 16:16:17,906][20381] Updated weights for policy 1, policy_version 1532017 (0.0007) [2023-12-27 16:16:17,957][20381] Updated weights for policy 1, policy_version 1532027 (0.0007) [2023-12-27 16:16:18,012][20381] Updated weights for policy 1, policy_version 1532037 (0.0007) [2023-12-27 16:16:18,383][20347] Updated weights for policy 0, policy_version 1533626 (0.0010) [2023-12-27 16:16:18,442][20347] Updated weights for policy 0, policy_version 1533636 (0.0010) [2023-12-27 16:16:18,509][20347] Updated weights for policy 0, policy_version 1533646 (0.0010) [2023-12-27 16:16:18,708][20381] Updated weights for policy 1, policy_version 1532047 (0.0007) [2023-12-27 16:16:18,766][20381] Updated weights for policy 1, policy_version 1532057 (0.0007) [2023-12-27 16:16:18,832][20381] Updated weights for policy 1, policy_version 1532067 (0.0008) [2023-12-27 16:16:19,152][20347] Updated weights for policy 0, policy_version 1533656 (0.0007) [2023-12-27 16:16:19,210][20347] Updated weights for policy 0, policy_version 1533666 (0.0006) [2023-12-27 16:16:19,280][20347] Updated weights for policy 0, policy_version 1533676 (0.0011) [2023-12-27 16:16:19,493][20381] Updated weights for policy 1, policy_version 1532077 (0.0008) [2023-12-27 16:16:19,551][20381] Updated weights for policy 1, policy_version 1532087 (0.0008) [2023-12-27 16:16:19,616][20381] Updated weights for policy 1, policy_version 1532097 (0.0006) [2023-12-27 16:16:20,103][20347] Updated weights for policy 0, policy_version 1533686 (0.0008) [2023-12-27 16:16:20,162][20347] Updated weights for policy 0, policy_version 1533696 (0.0007) [2023-12-27 16:16:20,215][20347] Updated weights for policy 0, policy_version 1533706 (0.0008) [2023-12-27 16:16:20,279][20381] Updated weights for policy 1, policy_version 1532107 (0.0006) [2023-12-27 16:16:20,342][20381] Updated weights for policy 1, policy_version 1532117 (0.0006) [2023-12-27 16:16:20,404][20381] Updated weights for policy 1, policy_version 1532127 (0.0007) [2023-12-27 16:16:20,947][20347] Updated weights for policy 0, policy_version 1533716 (0.0008) [2023-12-27 16:16:21,003][20347] Updated weights for policy 0, policy_version 1533726 (0.0007) [2023-12-27 16:16:21,073][20347] Updated weights for policy 0, policy_version 1533736 (0.0009) [2023-12-27 16:16:21,113][20381] Updated weights for policy 1, policy_version 1532137 (0.0007) [2023-12-27 16:16:21,175][20381] Updated weights for policy 1, policy_version 1532147 (0.0011) [2023-12-27 16:16:21,239][20381] Updated weights for policy 1, policy_version 1532157 (0.0011) [2023-12-27 16:16:21,306][20381] Updated weights for policy 1, policy_version 1532167 (0.0011) [2023-12-27 16:16:21,845][20347] Updated weights for policy 0, policy_version 1533746 (0.0007) [2023-12-27 16:16:21,900][20347] Updated weights for policy 0, policy_version 1533756 (0.0007) [2023-12-27 16:16:21,948][20347] Updated weights for policy 0, policy_version 1533766 (0.0007) [2023-12-27 16:16:22,002][20347] Updated weights for policy 0, policy_version 1533776 (0.0007) [2023-12-27 16:16:22,041][20381] Updated weights for policy 1, policy_version 1532177 (0.0011) [2023-12-27 16:16:22,098][20381] Updated weights for policy 1, policy_version 1532187 (0.0011) [2023-12-27 16:16:22,165][20381] Updated weights for policy 1, policy_version 1532197 (0.0011) [2023-12-27 16:16:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 785006592. Throughput: 0: 10138.3, 1: 9980.6. Samples: 784977436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:22,363][19170] Avg episode reward: [(0, '862.730'), (1, '1000.000')] [2023-12-27 16:16:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001532200_392282112.pth... [2023-12-27 16:16:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001533776_392724480.pth... [2023-12-27 16:16:22,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001531048_391987200.pth [2023-12-27 16:16:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001532592_392421376.pth [2023-12-27 16:16:22,709][20347] Updated weights for policy 0, policy_version 1533786 (0.0007) [2023-12-27 16:16:22,765][20347] Updated weights for policy 0, policy_version 1533796 (0.0008) [2023-12-27 16:16:22,822][20347] Updated weights for policy 0, policy_version 1533806 (0.0009) [2023-12-27 16:16:22,829][20381] Updated weights for policy 1, policy_version 1532207 (0.0009) [2023-12-27 16:16:22,892][20381] Updated weights for policy 1, policy_version 1532217 (0.0008) [2023-12-27 16:16:22,949][20381] Updated weights for policy 1, policy_version 1532227 (0.0006) [2023-12-27 16:16:23,515][20347] Updated weights for policy 0, policy_version 1533816 (0.0008) [2023-12-27 16:16:23,569][20347] Updated weights for policy 0, policy_version 1533826 (0.0008) [2023-12-27 16:16:23,574][20381] Updated weights for policy 1, policy_version 1532237 (0.0009) [2023-12-27 16:16:23,622][20347] Updated weights for policy 0, policy_version 1533836 (0.0006) [2023-12-27 16:16:23,630][20381] Updated weights for policy 1, policy_version 1532247 (0.0010) [2023-12-27 16:16:23,682][20381] Updated weights for policy 1, policy_version 1532257 (0.0010) [2023-12-27 16:16:24,252][20347] Updated weights for policy 0, policy_version 1533846 (0.0009) [2023-12-27 16:16:24,272][20381] Updated weights for policy 1, policy_version 1532267 (0.0006) [2023-12-27 16:16:24,312][20347] Updated weights for policy 0, policy_version 1533856 (0.0010) [2023-12-27 16:16:24,326][20381] Updated weights for policy 1, policy_version 1532277 (0.0011) [2023-12-27 16:16:24,371][20347] Updated weights for policy 0, policy_version 1533866 (0.0010) [2023-12-27 16:16:24,381][20381] Updated weights for policy 1, policy_version 1532287 (0.0011) [2023-12-27 16:16:25,020][20347] Updated weights for policy 0, policy_version 1533876 (0.0007) [2023-12-27 16:16:25,051][20381] Updated weights for policy 1, policy_version 1532297 (0.0011) [2023-12-27 16:16:25,076][20347] Updated weights for policy 0, policy_version 1533886 (0.0006) [2023-12-27 16:16:25,111][20381] Updated weights for policy 1, policy_version 1532307 (0.0006) [2023-12-27 16:16:25,130][20347] Updated weights for policy 0, policy_version 1533896 (0.0006) [2023-12-27 16:16:25,166][20381] Updated weights for policy 1, policy_version 1532317 (0.0006) [2023-12-27 16:16:25,220][20381] Updated weights for policy 1, policy_version 1532327 (0.0008) [2023-12-27 16:16:25,753][20347] Updated weights for policy 0, policy_version 1533906 (0.0008) [2023-12-27 16:16:25,808][20347] Updated weights for policy 0, policy_version 1533916 (0.0007) [2023-12-27 16:16:25,864][20347] Updated weights for policy 0, policy_version 1533926 (0.0006) [2023-12-27 16:16:25,920][20347] Updated weights for policy 0, policy_version 1533936 (0.0006) [2023-12-27 16:16:25,929][20381] Updated weights for policy 1, policy_version 1532337 (0.0006) [2023-12-27 16:16:25,987][20381] Updated weights for policy 1, policy_version 1532347 (0.0007) [2023-12-27 16:16:26,049][20381] Updated weights for policy 1, policy_version 1532357 (0.0008) [2023-12-27 16:16:26,612][20381] Updated weights for policy 1, policy_version 1532367 (0.0006) [2023-12-27 16:16:26,640][20347] Updated weights for policy 0, policy_version 1533946 (0.0008) [2023-12-27 16:16:26,667][20381] Updated weights for policy 1, policy_version 1532377 (0.0005) [2023-12-27 16:16:26,688][20347] Updated weights for policy 0, policy_version 1533956 (0.0010) [2023-12-27 16:16:26,721][20381] Updated weights for policy 1, policy_version 1532387 (0.0006) [2023-12-27 16:16:26,733][20347] Updated weights for policy 0, policy_version 1533966 (0.0008) [2023-12-27 16:16:27,355][20381] Updated weights for policy 1, policy_version 1532397 (0.0007) [2023-12-27 16:16:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20070.4, 300 sec: 20549.4). Total num frames: 785104896. Throughput: 0: 10162.8, 1: 10034.3. Samples: 785099176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:27,363][19170] Avg episode reward: [(0, '862.770'), (1, '990.300')] [2023-12-27 16:16:27,405][20381] Updated weights for policy 1, policy_version 1532407 (0.0005) [2023-12-27 16:16:27,425][20347] Updated weights for policy 0, policy_version 1533976 (0.0009) [2023-12-27 16:16:27,462][20381] Updated weights for policy 1, policy_version 1532417 (0.0005) [2023-12-27 16:16:27,478][20347] Updated weights for policy 0, policy_version 1533986 (0.0010) [2023-12-27 16:16:27,529][20347] Updated weights for policy 0, policy_version 1533996 (0.0010) [2023-12-27 16:16:28,151][20381] Updated weights for policy 1, policy_version 1532427 (0.0006) [2023-12-27 16:16:28,172][20347] Updated weights for policy 0, policy_version 1534006 (0.0010) [2023-12-27 16:16:28,210][20381] Updated weights for policy 1, policy_version 1532437 (0.0007) [2023-12-27 16:16:28,229][20347] Updated weights for policy 0, policy_version 1534016 (0.0010) [2023-12-27 16:16:28,260][20381] Updated weights for policy 1, policy_version 1532447 (0.0006) [2023-12-27 16:16:28,280][20347] Updated weights for policy 0, policy_version 1534026 (0.0010) [2023-12-27 16:16:28,945][20347] Updated weights for policy 0, policy_version 1534036 (0.0009) [2023-12-27 16:16:28,951][20381] Updated weights for policy 1, policy_version 1532457 (0.0006) [2023-12-27 16:16:28,997][20347] Updated weights for policy 0, policy_version 1534046 (0.0007) [2023-12-27 16:16:29,003][20381] Updated weights for policy 1, policy_version 1532467 (0.0005) [2023-12-27 16:16:29,046][20347] Updated weights for policy 0, policy_version 1534056 (0.0008) [2023-12-27 16:16:29,061][20381] Updated weights for policy 1, policy_version 1532477 (0.0005) [2023-12-27 16:16:29,117][20381] Updated weights for policy 1, policy_version 1532487 (0.0006) [2023-12-27 16:16:29,763][20381] Updated weights for policy 1, policy_version 1532497 (0.0008) [2023-12-27 16:16:29,770][20347] Updated weights for policy 0, policy_version 1534066 (0.0007) [2023-12-27 16:16:29,818][20381] Updated weights for policy 1, policy_version 1532507 (0.0007) [2023-12-27 16:16:29,828][20347] Updated weights for policy 0, policy_version 1534076 (0.0006) [2023-12-27 16:16:29,884][20381] Updated weights for policy 1, policy_version 1532517 (0.0007) [2023-12-27 16:16:29,887][20347] Updated weights for policy 0, policy_version 1534086 (0.0008) [2023-12-27 16:16:29,950][20347] Updated weights for policy 0, policy_version 1534096 (0.0009) [2023-12-27 16:16:30,561][20347] Updated weights for policy 0, policy_version 1534106 (0.0008) [2023-12-27 16:16:30,566][20381] Updated weights for policy 1, policy_version 1532527 (0.0007) [2023-12-27 16:16:30,621][20347] Updated weights for policy 0, policy_version 1534116 (0.0008) [2023-12-27 16:16:30,625][20381] Updated weights for policy 1, policy_version 1532537 (0.0005) [2023-12-27 16:16:30,682][20347] Updated weights for policy 0, policy_version 1534126 (0.0008) [2023-12-27 16:16:30,682][20381] Updated weights for policy 1, policy_version 1532547 (0.0005) [2023-12-27 16:16:31,239][20381] Updated weights for policy 1, policy_version 1532557 (0.0006) [2023-12-27 16:16:31,297][20381] Updated weights for policy 1, policy_version 1532567 (0.0007) [2023-12-27 16:16:31,356][20381] Updated weights for policy 1, policy_version 1532577 (0.0008) [2023-12-27 16:16:31,423][20347] Updated weights for policy 0, policy_version 1534136 (0.0008) [2023-12-27 16:16:31,478][20347] Updated weights for policy 0, policy_version 1534146 (0.0008) [2023-12-27 16:16:31,541][20347] Updated weights for policy 0, policy_version 1534156 (0.0006) [2023-12-27 16:16:32,051][20381] Updated weights for policy 1, policy_version 1532587 (0.0009) [2023-12-27 16:16:32,114][20381] Updated weights for policy 1, policy_version 1532597 (0.0011) [2023-12-27 16:16:32,178][20381] Updated weights for policy 1, policy_version 1532607 (0.0010) [2023-12-27 16:16:32,207][20347] Updated weights for policy 0, policy_version 1534166 (0.0008) [2023-12-27 16:16:32,267][20347] Updated weights for policy 0, policy_version 1534176 (0.0008) [2023-12-27 16:16:32,329][20347] Updated weights for policy 0, policy_version 1534186 (0.0008) [2023-12-27 16:16:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 785211392. Throughput: 0: 10232.0, 1: 10157.5. Samples: 785224332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:32,363][19170] Avg episode reward: [(0, '872.640'), (1, '990.300')] [2023-12-27 16:16:32,817][20381] Updated weights for policy 1, policy_version 1532617 (0.0005) [2023-12-27 16:16:32,881][20381] Updated weights for policy 1, policy_version 1532627 (0.0008) [2023-12-27 16:16:32,943][20381] Updated weights for policy 1, policy_version 1532637 (0.0007) [2023-12-27 16:16:32,954][20347] Updated weights for policy 0, policy_version 1534196 (0.0007) [2023-12-27 16:16:32,999][20381] Updated weights for policy 1, policy_version 1532647 (0.0008) [2023-12-27 16:16:33,014][20347] Updated weights for policy 0, policy_version 1534206 (0.0006) [2023-12-27 16:16:33,075][20347] Updated weights for policy 0, policy_version 1534216 (0.0007) [2023-12-27 16:16:33,582][20381] Updated weights for policy 1, policy_version 1532657 (0.0007) [2023-12-27 16:16:33,640][20381] Updated weights for policy 1, policy_version 1532667 (0.0008) [2023-12-27 16:16:33,682][20347] Updated weights for policy 0, policy_version 1534226 (0.0007) [2023-12-27 16:16:33,694][20381] Updated weights for policy 1, policy_version 1532677 (0.0007) [2023-12-27 16:16:33,728][20347] Updated weights for policy 0, policy_version 1534236 (0.0007) [2023-12-27 16:16:33,782][20347] Updated weights for policy 0, policy_version 1534246 (0.0008) [2023-12-27 16:16:33,832][20347] Updated weights for policy 0, policy_version 1534256 (0.0007) [2023-12-27 16:16:34,307][20381] Updated weights for policy 1, policy_version 1532687 (0.0007) [2023-12-27 16:16:34,366][20381] Updated weights for policy 1, policy_version 1532697 (0.0007) [2023-12-27 16:16:34,427][20381] Updated weights for policy 1, policy_version 1532707 (0.0008) [2023-12-27 16:16:34,521][20347] Updated weights for policy 0, policy_version 1534266 (0.0008) [2023-12-27 16:16:34,584][20347] Updated weights for policy 0, policy_version 1534276 (0.0008) [2023-12-27 16:16:34,641][20347] Updated weights for policy 0, policy_version 1534286 (0.0007) [2023-12-27 16:16:35,089][20381] Updated weights for policy 1, policy_version 1532717 (0.0009) [2023-12-27 16:16:35,144][20381] Updated weights for policy 1, policy_version 1532727 (0.0008) [2023-12-27 16:16:35,195][20381] Updated weights for policy 1, policy_version 1532737 (0.0007) [2023-12-27 16:16:35,341][20347] Updated weights for policy 0, policy_version 1534296 (0.0007) [2023-12-27 16:16:35,400][20347] Updated weights for policy 0, policy_version 1534306 (0.0007) [2023-12-27 16:16:35,445][20347] Updated weights for policy 0, policy_version 1534316 (0.0008) [2023-12-27 16:16:35,870][20381] Updated weights for policy 1, policy_version 1532747 (0.0006) [2023-12-27 16:16:35,929][20381] Updated weights for policy 1, policy_version 1532757 (0.0008) [2023-12-27 16:16:35,987][20381] Updated weights for policy 1, policy_version 1532767 (0.0009) [2023-12-27 16:16:36,156][20347] Updated weights for policy 0, policy_version 1534326 (0.0006) [2023-12-27 16:16:36,209][20347] Updated weights for policy 0, policy_version 1534336 (0.0005) [2023-12-27 16:16:36,260][20347] Updated weights for policy 0, policy_version 1534346 (0.0006) [2023-12-27 16:16:36,700][20381] Updated weights for policy 1, policy_version 1532777 (0.0006) [2023-12-27 16:16:36,761][20381] Updated weights for policy 1, policy_version 1532787 (0.0005) [2023-12-27 16:16:36,821][20381] Updated weights for policy 1, policy_version 1532797 (0.0005) [2023-12-27 16:16:36,878][20381] Updated weights for policy 1, policy_version 1532807 (0.0006) [2023-12-27 16:16:36,984][20347] Updated weights for policy 0, policy_version 1534356 (0.0008) [2023-12-27 16:16:37,044][20347] Updated weights for policy 0, policy_version 1534366 (0.0008) [2023-12-27 16:16:37,107][20347] Updated weights for policy 0, policy_version 1534376 (0.0008) [2023-12-27 16:16:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 785317888. Throughput: 0: 10253.1, 1: 10178.2. Samples: 785288108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:37,363][19170] Avg episode reward: [(0, '882.450'), (1, '980.530')] [2023-12-27 16:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001532808_392437760.pth... [2023-12-27 16:16:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001534384_392880128.pth... [2023-12-27 16:16:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001531624_392134656.pth [2023-12-27 16:16:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001533200_392577024.pth [2023-12-27 16:16:37,514][20381] Updated weights for policy 1, policy_version 1532817 (0.0008) [2023-12-27 16:16:37,560][20381] Updated weights for policy 1, policy_version 1532827 (0.0008) [2023-12-27 16:16:37,611][20381] Updated weights for policy 1, policy_version 1532837 (0.0007) [2023-12-27 16:16:37,811][20347] Updated weights for policy 0, policy_version 1534386 (0.0008) [2023-12-27 16:16:37,864][20347] Updated weights for policy 0, policy_version 1534396 (0.0007) [2023-12-27 16:16:37,923][20347] Updated weights for policy 0, policy_version 1534406 (0.0007) [2023-12-27 16:16:37,973][20347] Updated weights for policy 0, policy_version 1534416 (0.0009) [2023-12-27 16:16:38,234][20381] Updated weights for policy 1, policy_version 1532847 (0.0007) [2023-12-27 16:16:38,285][20381] Updated weights for policy 1, policy_version 1532857 (0.0007) [2023-12-27 16:16:38,341][20381] Updated weights for policy 1, policy_version 1532867 (0.0007) [2023-12-27 16:16:38,667][20347] Updated weights for policy 0, policy_version 1534426 (0.0008) [2023-12-27 16:16:38,723][20347] Updated weights for policy 0, policy_version 1534436 (0.0006) [2023-12-27 16:16:38,788][20347] Updated weights for policy 0, policy_version 1534446 (0.0006) [2023-12-27 16:16:39,033][20381] Updated weights for policy 1, policy_version 1532877 (0.0008) [2023-12-27 16:16:39,093][20381] Updated weights for policy 1, policy_version 1532887 (0.0008) [2023-12-27 16:16:39,155][20381] Updated weights for policy 1, policy_version 1532897 (0.0008) [2023-12-27 16:16:39,434][20347] Updated weights for policy 0, policy_version 1534456 (0.0006) [2023-12-27 16:16:39,488][20347] Updated weights for policy 0, policy_version 1534466 (0.0006) [2023-12-27 16:16:39,548][20347] Updated weights for policy 0, policy_version 1534476 (0.0006) [2023-12-27 16:16:39,915][20381] Updated weights for policy 1, policy_version 1532907 (0.0008) [2023-12-27 16:16:39,970][20381] Updated weights for policy 1, policy_version 1532917 (0.0010) [2023-12-27 16:16:40,023][20381] Updated weights for policy 1, policy_version 1532927 (0.0011) [2023-12-27 16:16:40,142][20347] Updated weights for policy 0, policy_version 1534486 (0.0007) [2023-12-27 16:16:40,198][20347] Updated weights for policy 0, policy_version 1534496 (0.0008) [2023-12-27 16:16:40,247][20347] Updated weights for policy 0, policy_version 1534506 (0.0007) [2023-12-27 16:16:40,730][20381] Updated weights for policy 1, policy_version 1532937 (0.0010) [2023-12-27 16:16:40,783][20381] Updated weights for policy 1, policy_version 1532947 (0.0011) [2023-12-27 16:16:40,833][20381] Updated weights for policy 1, policy_version 1532957 (0.0010) [2023-12-27 16:16:40,884][20381] Updated weights for policy 1, policy_version 1532967 (0.0010) [2023-12-27 16:16:40,951][20347] Updated weights for policy 0, policy_version 1534516 (0.0008) [2023-12-27 16:16:40,999][20347] Updated weights for policy 0, policy_version 1534526 (0.0007) [2023-12-27 16:16:41,058][20347] Updated weights for policy 0, policy_version 1534536 (0.0008) [2023-12-27 16:16:41,631][20381] Updated weights for policy 1, policy_version 1532977 (0.0009) [2023-12-27 16:16:41,702][20381] Updated weights for policy 1, policy_version 1532987 (0.0010) [2023-12-27 16:16:41,767][20381] Updated weights for policy 1, policy_version 1532997 (0.0010) [2023-12-27 16:16:41,833][20347] Updated weights for policy 0, policy_version 1534546 (0.0007) [2023-12-27 16:16:41,899][20347] Updated weights for policy 0, policy_version 1534556 (0.0006) [2023-12-27 16:16:41,966][20347] Updated weights for policy 0, policy_version 1534566 (0.0007) [2023-12-27 16:16:42,035][20347] Updated weights for policy 0, policy_version 1534576 (0.0007) [2023-12-27 16:16:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 785416192. Throughput: 0: 10230.7, 1: 10182.2. Samples: 785409780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:42,363][19170] Avg episode reward: [(0, '921.620'), (1, '980.530')] [2023-12-27 16:16:42,416][20381] Updated weights for policy 1, policy_version 1533007 (0.0009) [2023-12-27 16:16:42,479][20381] Updated weights for policy 1, policy_version 1533017 (0.0008) [2023-12-27 16:16:42,539][20381] Updated weights for policy 1, policy_version 1533027 (0.0007) [2023-12-27 16:16:42,707][20347] Updated weights for policy 0, policy_version 1534586 (0.0011) [2023-12-27 16:16:42,762][20347] Updated weights for policy 0, policy_version 1534596 (0.0011) [2023-12-27 16:16:42,814][20347] Updated weights for policy 0, policy_version 1534606 (0.0010) [2023-12-27 16:16:43,208][20381] Updated weights for policy 1, policy_version 1533037 (0.0006) [2023-12-27 16:16:43,267][20381] Updated weights for policy 1, policy_version 1533047 (0.0007) [2023-12-27 16:16:43,322][20381] Updated weights for policy 1, policy_version 1533057 (0.0007) [2023-12-27 16:16:43,505][20347] Updated weights for policy 0, policy_version 1534616 (0.0010) [2023-12-27 16:16:43,562][20347] Updated weights for policy 0, policy_version 1534626 (0.0010) [2023-12-27 16:16:43,617][20347] Updated weights for policy 0, policy_version 1534636 (0.0010) [2023-12-27 16:16:44,027][20381] Updated weights for policy 1, policy_version 1533067 (0.0008) [2023-12-27 16:16:44,074][20381] Updated weights for policy 1, policy_version 1533077 (0.0007) [2023-12-27 16:16:44,132][20381] Updated weights for policy 1, policy_version 1533087 (0.0007) [2023-12-27 16:16:44,291][20347] Updated weights for policy 0, policy_version 1534646 (0.0010) [2023-12-27 16:16:44,337][20347] Updated weights for policy 0, policy_version 1534656 (0.0010) [2023-12-27 16:16:44,382][20347] Updated weights for policy 0, policy_version 1534666 (0.0010) [2023-12-27 16:16:44,841][20381] Updated weights for policy 1, policy_version 1533097 (0.0008) [2023-12-27 16:16:44,896][20381] Updated weights for policy 1, policy_version 1533107 (0.0008) [2023-12-27 16:16:44,959][20381] Updated weights for policy 1, policy_version 1533117 (0.0008) [2023-12-27 16:16:45,020][20381] Updated weights for policy 1, policy_version 1533127 (0.0008) [2023-12-27 16:16:45,060][20347] Updated weights for policy 0, policy_version 1534676 (0.0011) [2023-12-27 16:16:45,120][20347] Updated weights for policy 0, policy_version 1534686 (0.0010) [2023-12-27 16:16:45,177][20347] Updated weights for policy 0, policy_version 1534696 (0.0011) [2023-12-27 16:16:45,755][20381] Updated weights for policy 1, policy_version 1533137 (0.0007) [2023-12-27 16:16:45,808][20381] Updated weights for policy 1, policy_version 1533147 (0.0007) [2023-12-27 16:16:45,861][20381] Updated weights for policy 1, policy_version 1533157 (0.0008) [2023-12-27 16:16:45,862][20347] Updated weights for policy 0, policy_version 1534706 (0.0011) [2023-12-27 16:16:45,907][20347] Updated weights for policy 0, policy_version 1534716 (0.0010) [2023-12-27 16:16:45,956][20347] Updated weights for policy 0, policy_version 1534726 (0.0010) [2023-12-27 16:16:46,005][20347] Updated weights for policy 0, policy_version 1534736 (0.0010) [2023-12-27 16:16:46,550][20381] Updated weights for policy 1, policy_version 1533167 (0.0008) [2023-12-27 16:16:46,610][20381] Updated weights for policy 1, policy_version 1533177 (0.0007) [2023-12-27 16:16:46,664][20381] Updated weights for policy 1, policy_version 1533187 (0.0007) [2023-12-27 16:16:46,718][20347] Updated weights for policy 0, policy_version 1534746 (0.0010) [2023-12-27 16:16:46,770][20347] Updated weights for policy 0, policy_version 1534756 (0.0010) [2023-12-27 16:16:46,824][20347] Updated weights for policy 0, policy_version 1534766 (0.0008) [2023-12-27 16:16:47,290][20381] Updated weights for policy 1, policy_version 1533197 (0.0008) [2023-12-27 16:16:47,355][20381] Updated weights for policy 1, policy_version 1533207 (0.0007) [2023-12-27 16:16:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 785514496. Throughput: 0: 10236.2, 1: 10202.8. Samples: 785529884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:47,363][19170] Avg episode reward: [(0, '862.780'), (1, '1000.000')] [2023-12-27 16:16:47,421][20381] Updated weights for policy 1, policy_version 1533217 (0.0006) [2023-12-27 16:16:47,517][20347] Updated weights for policy 0, policy_version 1534776 (0.0009) [2023-12-27 16:16:47,579][20347] Updated weights for policy 0, policy_version 1534786 (0.0010) [2023-12-27 16:16:47,637][20347] Updated weights for policy 0, policy_version 1534796 (0.0007) [2023-12-27 16:16:48,025][20381] Updated weights for policy 1, policy_version 1533227 (0.0007) [2023-12-27 16:16:48,085][20381] Updated weights for policy 1, policy_version 1533237 (0.0007) [2023-12-27 16:16:48,137][20381] Updated weights for policy 1, policy_version 1533247 (0.0008) [2023-12-27 16:16:48,299][20347] Updated weights for policy 0, policy_version 1534806 (0.0010) [2023-12-27 16:16:48,367][20347] Updated weights for policy 0, policy_version 1534816 (0.0009) [2023-12-27 16:16:48,429][20347] Updated weights for policy 0, policy_version 1534826 (0.0008) [2023-12-27 16:16:48,864][20381] Updated weights for policy 1, policy_version 1533257 (0.0007) [2023-12-27 16:16:48,913][20381] Updated weights for policy 1, policy_version 1533267 (0.0007) [2023-12-27 16:16:48,962][20381] Updated weights for policy 1, policy_version 1533277 (0.0008) [2023-12-27 16:16:49,013][20381] Updated weights for policy 1, policy_version 1533287 (0.0008) [2023-12-27 16:16:49,090][20347] Updated weights for policy 0, policy_version 1534836 (0.0009) [2023-12-27 16:16:49,090][20291] KL-divergence is very high: 107.5893 [2023-12-27 16:16:49,109][20291] KL-divergence is very high: 146.0726 [2023-12-27 16:16:49,128][20291] KL-divergence is very high: 134.1028 [2023-12-27 16:16:49,138][20347] Updated weights for policy 0, policy_version 1534846 (0.0010) [2023-12-27 16:16:49,186][20347] Updated weights for policy 0, policy_version 1534856 (0.0010) [2023-12-27 16:16:49,757][20381] Updated weights for policy 1, policy_version 1533297 (0.0007) [2023-12-27 16:16:49,824][20381] Updated weights for policy 1, policy_version 1533307 (0.0007) [2023-12-27 16:16:49,889][20381] Updated weights for policy 1, policy_version 1533317 (0.0008) [2023-12-27 16:16:49,907][20347] Updated weights for policy 0, policy_version 1534866 (0.0010) [2023-12-27 16:16:49,965][20347] Updated weights for policy 0, policy_version 1534876 (0.0008) [2023-12-27 16:16:50,028][20347] Updated weights for policy 0, policy_version 1534886 (0.0006) [2023-12-27 16:16:50,084][20347] Updated weights for policy 0, policy_version 1534896 (0.0006) [2023-12-27 16:16:50,599][20381] Updated weights for policy 1, policy_version 1533327 (0.0007) [2023-12-27 16:16:50,660][20381] Updated weights for policy 1, policy_version 1533337 (0.0008) [2023-12-27 16:16:50,719][20381] Updated weights for policy 1, policy_version 1533347 (0.0008) [2023-12-27 16:16:50,733][20347] Updated weights for policy 0, policy_version 1534906 (0.0011) [2023-12-27 16:16:50,790][20347] Updated weights for policy 0, policy_version 1534916 (0.0010) [2023-12-27 16:16:50,845][20347] Updated weights for policy 0, policy_version 1534926 (0.0010) [2023-12-27 16:16:51,490][20381] Updated weights for policy 1, policy_version 1533357 (0.0007) [2023-12-27 16:16:51,533][20347] Updated weights for policy 0, policy_version 1534936 (0.0008) [2023-12-27 16:16:51,547][20381] Updated weights for policy 1, policy_version 1533367 (0.0005) [2023-12-27 16:16:51,593][20347] Updated weights for policy 0, policy_version 1534946 (0.0007) [2023-12-27 16:16:51,633][20381] Updated weights for policy 1, policy_version 1533377 (0.0006) [2023-12-27 16:16:51,656][20347] Updated weights for policy 0, policy_version 1534956 (0.0008) [2023-12-27 16:16:52,257][20381] Updated weights for policy 1, policy_version 1533387 (0.0008) [2023-12-27 16:16:52,317][20381] Updated weights for policy 1, policy_version 1533397 (0.0007) [2023-12-27 16:16:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 785612800. Throughput: 0: 10174.8, 1: 10180.7. Samples: 785589728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:52,365][19170] Avg episode reward: [(0, '794.350'), (1, '990.320')] [2023-12-27 16:16:52,382][20381] Updated weights for policy 1, policy_version 1533407 (0.0007) [2023-12-27 16:16:52,386][20347] Updated weights for policy 0, policy_version 1534966 (0.0009) [2023-12-27 16:16:52,436][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001533416_392593408.pth... [2023-12-27 16:16:52,440][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001532200_392282112.pth [2023-12-27 16:16:52,445][20347] Updated weights for policy 0, policy_version 1534976 (0.0010) [2023-12-27 16:16:52,501][20347] Updated weights for policy 0, policy_version 1534986 (0.0010) [2023-12-27 16:16:52,532][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001534992_393035776.pth... [2023-12-27 16:16:52,535][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001533776_392724480.pth [2023-12-27 16:16:53,041][20381] Updated weights for policy 1, policy_version 1533417 (0.0009) [2023-12-27 16:16:53,101][20381] Updated weights for policy 1, policy_version 1533427 (0.0009) [2023-12-27 16:16:53,162][20381] Updated weights for policy 1, policy_version 1533437 (0.0006) [2023-12-27 16:16:53,207][20347] Updated weights for policy 0, policy_version 1534996 (0.0008) [2023-12-27 16:16:53,223][20381] Updated weights for policy 1, policy_version 1533447 (0.0008) [2023-12-27 16:16:53,259][20347] Updated weights for policy 0, policy_version 1535006 (0.0006) [2023-12-27 16:16:53,313][20347] Updated weights for policy 0, policy_version 1535016 (0.0007) [2023-12-27 16:16:53,771][20381] Updated weights for policy 1, policy_version 1533457 (0.0010) [2023-12-27 16:16:53,826][20381] Updated weights for policy 1, policy_version 1533467 (0.0011) [2023-12-27 16:16:53,887][20381] Updated weights for policy 1, policy_version 1533477 (0.0010) [2023-12-27 16:16:53,986][20347] Updated weights for policy 0, policy_version 1535026 (0.0008) [2023-12-27 16:16:54,041][20347] Updated weights for policy 0, policy_version 1535036 (0.0010) [2023-12-27 16:16:54,092][20347] Updated weights for policy 0, policy_version 1535046 (0.0010) [2023-12-27 16:16:54,149][20347] Updated weights for policy 0, policy_version 1535056 (0.0008) [2023-12-27 16:16:54,565][20381] Updated weights for policy 1, policy_version 1533487 (0.0011) [2023-12-27 16:16:54,622][20381] Updated weights for policy 1, policy_version 1533497 (0.0010) [2023-12-27 16:16:54,679][20381] Updated weights for policy 1, policy_version 1533507 (0.0010) [2023-12-27 16:16:54,764][20347] Updated weights for policy 0, policy_version 1535066 (0.0011) [2023-12-27 16:16:54,791][20291] KL-divergence is very high: 169.2892 [2023-12-27 16:16:54,822][20347] Updated weights for policy 0, policy_version 1535076 (0.0010) [2023-12-27 16:16:54,831][20291] KL-divergence is very high: 223.8173 [2023-12-27 16:16:54,870][20291] KL-divergence is very high: 118.7493 [2023-12-27 16:16:54,870][20347] Updated weights for policy 0, policy_version 1535086 (0.0008) [2023-12-27 16:16:55,390][20381] Updated weights for policy 1, policy_version 1533517 (0.0009) [2023-12-27 16:16:55,443][20381] Updated weights for policy 1, policy_version 1533527 (0.0008) [2023-12-27 16:16:55,496][20381] Updated weights for policy 1, policy_version 1533537 (0.0006) [2023-12-27 16:16:55,506][20347] Updated weights for policy 0, policy_version 1535096 (0.0007) [2023-12-27 16:16:55,559][20347] Updated weights for policy 0, policy_version 1535106 (0.0005) [2023-12-27 16:16:55,612][20347] Updated weights for policy 0, policy_version 1535116 (0.0005) [2023-12-27 16:16:56,208][20381] Updated weights for policy 1, policy_version 1533547 (0.0007) [2023-12-27 16:16:56,247][20347] Updated weights for policy 0, policy_version 1535126 (0.0005) [2023-12-27 16:16:56,253][20381] Updated weights for policy 1, policy_version 1533557 (0.0006) [2023-12-27 16:16:56,295][20347] Updated weights for policy 0, policy_version 1535136 (0.0006) [2023-12-27 16:16:56,310][20381] Updated weights for policy 1, policy_version 1533567 (0.0007) [2023-12-27 16:16:56,343][20347] Updated weights for policy 0, policy_version 1535146 (0.0007) [2023-12-27 16:16:56,921][20347] Updated weights for policy 0, policy_version 1535156 (0.0008) [2023-12-27 16:16:56,978][20347] Updated weights for policy 0, policy_version 1535166 (0.0010) [2023-12-27 16:16:57,002][20381] Updated weights for policy 1, policy_version 1533577 (0.0007) [2023-12-27 16:16:57,038][20347] Updated weights for policy 0, policy_version 1535176 (0.0010) [2023-12-27 16:16:57,050][20381] Updated weights for policy 1, policy_version 1533587 (0.0011) [2023-12-27 16:16:57,107][20381] Updated weights for policy 1, policy_version 1533597 (0.0010) [2023-12-27 16:16:57,165][20381] Updated weights for policy 1, policy_version 1533607 (0.0011) [2023-12-27 16:16:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 785727488. Throughput: 0: 10172.3, 1: 10201.9. Samples: 785713392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:16:57,363][19170] Avg episode reward: [(0, '843.470'), (1, '990.320')] [2023-12-27 16:16:57,644][20347] Updated weights for policy 0, policy_version 1535186 (0.0008) [2023-12-27 16:16:57,689][20347] Updated weights for policy 0, policy_version 1535196 (0.0007) [2023-12-27 16:16:57,739][20347] Updated weights for policy 0, policy_version 1535206 (0.0006) [2023-12-27 16:16:57,756][20381] Updated weights for policy 1, policy_version 1533617 (0.0006) [2023-12-27 16:16:57,790][20347] Updated weights for policy 0, policy_version 1535216 (0.0007) [2023-12-27 16:16:57,811][20381] Updated weights for policy 1, policy_version 1533627 (0.0005) [2023-12-27 16:16:57,869][20381] Updated weights for policy 1, policy_version 1533637 (0.0005) [2023-12-27 16:16:58,493][20381] Updated weights for policy 1, policy_version 1533647 (0.0007) [2023-12-27 16:16:58,532][20347] Updated weights for policy 0, policy_version 1535226 (0.0008) [2023-12-27 16:16:58,563][20381] Updated weights for policy 1, policy_version 1533657 (0.0008) [2023-12-27 16:16:58,605][20347] Updated weights for policy 0, policy_version 1535236 (0.0010) [2023-12-27 16:16:58,628][20381] Updated weights for policy 1, policy_version 1533667 (0.0008) [2023-12-27 16:16:58,663][20347] Updated weights for policy 0, policy_version 1535246 (0.0007) [2023-12-27 16:16:59,369][20381] Updated weights for policy 1, policy_version 1533677 (0.0008) [2023-12-27 16:16:59,393][20347] Updated weights for policy 0, policy_version 1535256 (0.0007) [2023-12-27 16:16:59,425][20381] Updated weights for policy 1, policy_version 1533687 (0.0010) [2023-12-27 16:16:59,443][20347] Updated weights for policy 0, policy_version 1535266 (0.0005) [2023-12-27 16:16:59,475][20381] Updated weights for policy 1, policy_version 1533697 (0.0011) [2023-12-27 16:16:59,488][20347] Updated weights for policy 0, policy_version 1535276 (0.0005) [2023-12-27 16:17:00,169][20381] Updated weights for policy 1, policy_version 1533707 (0.0011) [2023-12-27 16:17:00,229][20381] Updated weights for policy 1, policy_version 1533717 (0.0006) [2023-12-27 16:17:00,236][20347] Updated weights for policy 0, policy_version 1535286 (0.0007) [2023-12-27 16:17:00,290][20381] Updated weights for policy 1, policy_version 1533727 (0.0009) [2023-12-27 16:17:00,298][20347] Updated weights for policy 0, policy_version 1535296 (0.0008) [2023-12-27 16:17:00,355][20347] Updated weights for policy 0, policy_version 1535306 (0.0006) [2023-12-27 16:17:00,949][20381] Updated weights for policy 1, policy_version 1533737 (0.0011) [2023-12-27 16:17:00,994][20347] Updated weights for policy 0, policy_version 1535316 (0.0006) [2023-12-27 16:17:00,997][20381] Updated weights for policy 1, policy_version 1533747 (0.0010) [2023-12-27 16:17:01,049][20347] Updated weights for policy 0, policy_version 1535326 (0.0005) [2023-12-27 16:17:01,050][20291] KL-divergence is very high: 220.0314 [2023-12-27 16:17:01,054][20381] Updated weights for policy 1, policy_version 1533757 (0.0009) [2023-12-27 16:17:01,062][20291] KL-divergence is very high: 144.7871 [2023-12-27 16:17:01,096][20291] KL-divergence is very high: 181.3470 [2023-12-27 16:17:01,107][20347] Updated weights for policy 0, policy_version 1535336 (0.0006) [2023-12-27 16:17:01,110][20381] Updated weights for policy 1, policy_version 1533767 (0.0010) [2023-12-27 16:17:01,771][20347] Updated weights for policy 0, policy_version 1535346 (0.0007) [2023-12-27 16:17:01,832][20347] Updated weights for policy 0, policy_version 1535356 (0.0007) [2023-12-27 16:17:01,864][20381] Updated weights for policy 1, policy_version 1533777 (0.0011) [2023-12-27 16:17:01,893][20347] Updated weights for policy 0, policy_version 1535366 (0.0006) [2023-12-27 16:17:01,921][20381] Updated weights for policy 1, policy_version 1533787 (0.0008) [2023-12-27 16:17:01,951][20347] Updated weights for policy 0, policy_version 1535376 (0.0009) [2023-12-27 16:17:01,978][20381] Updated weights for policy 1, policy_version 1533797 (0.0011) [2023-12-27 16:17:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 785825792. Throughput: 0: 10177.2, 1: 10250.6. Samples: 785836140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:02,363][19170] Avg episode reward: [(0, '804.020'), (1, '1000.000')] [2023-12-27 16:17:02,665][20347] Updated weights for policy 0, policy_version 1535386 (0.0008) [2023-12-27 16:17:02,705][20381] Updated weights for policy 1, policy_version 1533807 (0.0009) [2023-12-27 16:17:02,717][20347] Updated weights for policy 0, policy_version 1535396 (0.0007) [2023-12-27 16:17:02,765][20381] Updated weights for policy 1, policy_version 1533817 (0.0006) [2023-12-27 16:17:02,771][20347] Updated weights for policy 0, policy_version 1535406 (0.0007) [2023-12-27 16:17:02,825][20381] Updated weights for policy 1, policy_version 1533827 (0.0011) [2023-12-27 16:17:03,470][20347] Updated weights for policy 0, policy_version 1535416 (0.0007) [2023-12-27 16:17:03,493][20381] Updated weights for policy 1, policy_version 1533837 (0.0008) [2023-12-27 16:17:03,525][20347] Updated weights for policy 0, policy_version 1535426 (0.0005) [2023-12-27 16:17:03,544][20381] Updated weights for policy 1, policy_version 1533847 (0.0006) [2023-12-27 16:17:03,579][20347] Updated weights for policy 0, policy_version 1535436 (0.0005) [2023-12-27 16:17:03,594][20381] Updated weights for policy 1, policy_version 1533857 (0.0006) [2023-12-27 16:17:04,208][20347] Updated weights for policy 0, policy_version 1535446 (0.0007) [2023-12-27 16:17:04,265][20347] Updated weights for policy 0, policy_version 1535456 (0.0008) [2023-12-27 16:17:04,300][20381] Updated weights for policy 1, policy_version 1533867 (0.0008) [2023-12-27 16:17:04,328][20347] Updated weights for policy 0, policy_version 1535466 (0.0006) [2023-12-27 16:17:04,354][20381] Updated weights for policy 1, policy_version 1533877 (0.0007) [2023-12-27 16:17:04,406][20381] Updated weights for policy 1, policy_version 1533887 (0.0007) [2023-12-27 16:17:05,036][20347] Updated weights for policy 0, policy_version 1535476 (0.0006) [2023-12-27 16:17:05,092][20347] Updated weights for policy 0, policy_version 1535486 (0.0009) [2023-12-27 16:17:05,105][20381] Updated weights for policy 1, policy_version 1533897 (0.0008) [2023-12-27 16:17:05,141][20347] Updated weights for policy 0, policy_version 1535496 (0.0010) [2023-12-27 16:17:05,154][20381] Updated weights for policy 1, policy_version 1533907 (0.0010) [2023-12-27 16:17:05,200][20381] Updated weights for policy 1, policy_version 1533917 (0.0010) [2023-12-27 16:17:05,260][20381] Updated weights for policy 1, policy_version 1533927 (0.0007) [2023-12-27 16:17:05,766][20347] Updated weights for policy 0, policy_version 1535506 (0.0010) [2023-12-27 16:17:05,823][20347] Updated weights for policy 0, policy_version 1535516 (0.0010) [2023-12-27 16:17:05,875][20347] Updated weights for policy 0, policy_version 1535526 (0.0009) [2023-12-27 16:17:05,902][20381] Updated weights for policy 1, policy_version 1533937 (0.0010) [2023-12-27 16:17:05,923][20347] Updated weights for policy 0, policy_version 1535536 (0.0010) [2023-12-27 16:17:05,956][20381] Updated weights for policy 1, policy_version 1533947 (0.0010) [2023-12-27 16:17:06,009][20381] Updated weights for policy 1, policy_version 1533957 (0.0010) [2023-12-27 16:17:06,595][20347] Updated weights for policy 0, policy_version 1535546 (0.0010) [2023-12-27 16:17:06,652][20347] Updated weights for policy 0, policy_version 1535556 (0.0010) [2023-12-27 16:17:06,708][20347] Updated weights for policy 0, policy_version 1535566 (0.0010) [2023-12-27 16:17:06,720][20381] Updated weights for policy 1, policy_version 1533967 (0.0009) [2023-12-27 16:17:06,776][20381] Updated weights for policy 1, policy_version 1533977 (0.0008) [2023-12-27 16:17:06,828][20381] Updated weights for policy 1, policy_version 1533987 (0.0007) [2023-12-27 16:17:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 785924096. Throughput: 0: 10189.4, 1: 10239.8. Samples: 785896744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:07,363][19170] Avg episode reward: [(0, '833.250'), (1, '1000.000')] [2023-12-27 16:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001533992_392740864.pth... [2023-12-27 16:17:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001532808_392437760.pth [2023-12-27 16:17:07,404][20347] Updated weights for policy 0, policy_version 1535576 (0.0008) [2023-12-27 16:17:07,456][20347] Updated weights for policy 0, policy_version 1535586 (0.0008) [2023-12-27 16:17:07,507][20347] Updated weights for policy 0, policy_version 1535596 (0.0007) [2023-12-27 16:17:07,528][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001535600_393191424.pth... [2023-12-27 16:17:07,532][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001534384_392880128.pth [2023-12-27 16:17:07,540][20381] Updated weights for policy 1, policy_version 1533997 (0.0007) [2023-12-27 16:17:07,589][20381] Updated weights for policy 1, policy_version 1534007 (0.0008) [2023-12-27 16:17:07,641][20381] Updated weights for policy 1, policy_version 1534017 (0.0006) [2023-12-27 16:17:08,206][20347] Updated weights for policy 0, policy_version 1535606 (0.0006) [2023-12-27 16:17:08,266][20347] Updated weights for policy 0, policy_version 1535616 (0.0005) [2023-12-27 16:17:08,271][20381] Updated weights for policy 1, policy_version 1534027 (0.0008) [2023-12-27 16:17:08,325][20381] Updated weights for policy 1, policy_version 1534037 (0.0006) [2023-12-27 16:17:08,336][20347] Updated weights for policy 0, policy_version 1535626 (0.0007) [2023-12-27 16:17:08,389][20381] Updated weights for policy 1, policy_version 1534048 (0.0012) [2023-12-27 16:17:08,974][20347] Updated weights for policy 0, policy_version 1535636 (0.0010) [2023-12-27 16:17:09,010][20381] Updated weights for policy 1, policy_version 1534058 (0.0007) [2023-12-27 16:17:09,030][20347] Updated weights for policy 0, policy_version 1535646 (0.0006) [2023-12-27 16:17:09,066][20381] Updated weights for policy 1, policy_version 1534068 (0.0005) [2023-12-27 16:17:09,085][20347] Updated weights for policy 0, policy_version 1535656 (0.0005) [2023-12-27 16:17:09,125][20381] Updated weights for policy 1, policy_version 1534078 (0.0005) [2023-12-27 16:17:09,173][20381] Updated weights for policy 1, policy_version 1534088 (0.0005) [2023-12-27 16:17:09,754][20347] Updated weights for policy 0, policy_version 1535666 (0.0005) [2023-12-27 16:17:09,808][20347] Updated weights for policy 0, policy_version 1535676 (0.0008) [2023-12-27 16:17:09,877][20347] Updated weights for policy 0, policy_version 1535686 (0.0007) [2023-12-27 16:17:09,878][20381] Updated weights for policy 1, policy_version 1534098 (0.0011) [2023-12-27 16:17:09,941][20381] Updated weights for policy 1, policy_version 1534108 (0.0010) [2023-12-27 16:17:09,947][20347] Updated weights for policy 0, policy_version 1535696 (0.0007) [2023-12-27 16:17:10,003][20381] Updated weights for policy 1, policy_version 1534118 (0.0010) [2023-12-27 16:17:10,663][20381] Updated weights for policy 1, policy_version 1534128 (0.0007) [2023-12-27 16:17:10,676][20347] Updated weights for policy 0, policy_version 1535706 (0.0006) [2023-12-27 16:17:10,722][20381] Updated weights for policy 1, policy_version 1534138 (0.0009) [2023-12-27 16:17:10,731][20347] Updated weights for policy 0, policy_version 1535716 (0.0007) [2023-12-27 16:17:10,780][20381] Updated weights for policy 1, policy_version 1534148 (0.0005) [2023-12-27 16:17:10,788][20347] Updated weights for policy 0, policy_version 1535726 (0.0007) [2023-12-27 16:17:11,481][20381] Updated weights for policy 1, policy_version 1534158 (0.0010) [2023-12-27 16:17:11,510][20292] KL-divergence is very high: 256.5000 [2023-12-27 16:17:11,544][20347] Updated weights for policy 0, policy_version 1535736 (0.0006) [2023-12-27 16:17:11,544][20381] Updated weights for policy 1, policy_version 1534168 (0.0007) [2023-12-27 16:17:11,601][20347] Updated weights for policy 0, policy_version 1535746 (0.0006) [2023-12-27 16:17:11,604][20381] Updated weights for policy 1, policy_version 1534178 (0.0008) [2023-12-27 16:17:11,667][20347] Updated weights for policy 0, policy_version 1535756 (0.0009) [2023-12-27 16:17:12,331][20381] Updated weights for policy 1, policy_version 1534188 (0.0007) [2023-12-27 16:17:12,336][20347] Updated weights for policy 0, policy_version 1535766 (0.0007) [2023-12-27 16:17:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 786022400. Throughput: 0: 10200.4, 1: 10269.2. Samples: 786020304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:12,363][19170] Avg episode reward: [(0, '843.200'), (1, '990.260')] [2023-12-27 16:17:12,402][20381] Updated weights for policy 1, policy_version 1534198 (0.0008) [2023-12-27 16:17:12,405][20347] Updated weights for policy 0, policy_version 1535776 (0.0007) [2023-12-27 16:17:12,464][20381] Updated weights for policy 1, policy_version 1534208 (0.0007) [2023-12-27 16:17:12,469][20347] Updated weights for policy 0, policy_version 1535786 (0.0006) [2023-12-27 16:17:13,026][20347] Updated weights for policy 0, policy_version 1535796 (0.0005) [2023-12-27 16:17:13,088][20347] Updated weights for policy 0, policy_version 1535806 (0.0006) [2023-12-27 16:17:13,139][20381] Updated weights for policy 1, policy_version 1534218 (0.0008) [2023-12-27 16:17:13,147][20347] Updated weights for policy 0, policy_version 1535816 (0.0006) [2023-12-27 16:17:13,188][20381] Updated weights for policy 1, policy_version 1534228 (0.0008) [2023-12-27 16:17:13,235][20381] Updated weights for policy 1, policy_version 1534238 (0.0008) [2023-12-27 16:17:13,283][20381] Updated weights for policy 1, policy_version 1534248 (0.0007) [2023-12-27 16:17:13,741][20291] KL-divergence is very high: 112.6253 [2023-12-27 16:17:13,747][20347] Updated weights for policy 0, policy_version 1535826 (0.0005) [2023-12-27 16:17:13,762][20291] KL-divergence is very high: 107.6369 [2023-12-27 16:17:13,806][20347] Updated weights for policy 0, policy_version 1535836 (0.0009) [2023-12-27 16:17:13,861][20347] Updated weights for policy 0, policy_version 1535846 (0.0008) [2023-12-27 16:17:13,930][20347] Updated weights for policy 0, policy_version 1535856 (0.0007) [2023-12-27 16:17:14,006][20381] Updated weights for policy 1, policy_version 1534258 (0.0010) [2023-12-27 16:17:14,068][20381] Updated weights for policy 1, policy_version 1534268 (0.0010) [2023-12-27 16:17:14,136][20381] Updated weights for policy 1, policy_version 1534278 (0.0008) [2023-12-27 16:17:14,593][20347] Updated weights for policy 0, policy_version 1535866 (0.0005) [2023-12-27 16:17:14,649][20347] Updated weights for policy 0, policy_version 1535876 (0.0005) [2023-12-27 16:17:14,705][20347] Updated weights for policy 0, policy_version 1535886 (0.0006) [2023-12-27 16:17:14,730][20381] Updated weights for policy 1, policy_version 1534288 (0.0010) [2023-12-27 16:17:14,792][20381] Updated weights for policy 1, policy_version 1534298 (0.0011) [2023-12-27 16:17:14,854][20381] Updated weights for policy 1, policy_version 1534308 (0.0011) [2023-12-27 16:17:15,318][20347] Updated weights for policy 0, policy_version 1535896 (0.0006) [2023-12-27 16:17:15,385][20347] Updated weights for policy 0, policy_version 1535906 (0.0005) [2023-12-27 16:17:15,450][20347] Updated weights for policy 0, policy_version 1535916 (0.0005) [2023-12-27 16:17:15,497][20381] Updated weights for policy 1, policy_version 1534318 (0.0007) [2023-12-27 16:17:15,548][20381] Updated weights for policy 1, policy_version 1534328 (0.0005) [2023-12-27 16:17:15,601][20381] Updated weights for policy 1, policy_version 1534338 (0.0005) [2023-12-27 16:17:16,051][20347] Updated weights for policy 0, policy_version 1535926 (0.0005) [2023-12-27 16:17:16,104][20347] Updated weights for policy 0, policy_version 1535936 (0.0005) [2023-12-27 16:17:16,162][20347] Updated weights for policy 0, policy_version 1535946 (0.0008) [2023-12-27 16:17:16,258][20381] Updated weights for policy 1, policy_version 1534348 (0.0008) [2023-12-27 16:17:16,307][20381] Updated weights for policy 1, policy_version 1534358 (0.0010) [2023-12-27 16:17:16,355][20381] Updated weights for policy 1, policy_version 1534368 (0.0010) [2023-12-27 16:17:16,781][20347] Updated weights for policy 0, policy_version 1535956 (0.0010) [2023-12-27 16:17:16,829][20347] Updated weights for policy 0, policy_version 1535966 (0.0010) [2023-12-27 16:17:16,884][20347] Updated weights for policy 0, policy_version 1535976 (0.0010) [2023-12-27 16:17:16,964][20381] Updated weights for policy 1, policy_version 1534378 (0.0009) [2023-12-27 16:17:17,010][20381] Updated weights for policy 1, policy_version 1534388 (0.0005) [2023-12-27 16:17:17,067][20381] Updated weights for policy 1, policy_version 1534398 (0.0007) [2023-12-27 16:17:17,111][20381] Updated weights for policy 1, policy_version 1534408 (0.0007) [2023-12-27 16:17:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 786137088. Throughput: 0: 10270.1, 1: 10221.6. Samples: 786146460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:17,364][19170] Avg episode reward: [(0, '784.540'), (1, '980.520')] [2023-12-27 16:17:17,569][20347] Updated weights for policy 0, policy_version 1535986 (0.0010) [2023-12-27 16:17:17,630][20347] Updated weights for policy 0, policy_version 1535996 (0.0008) [2023-12-27 16:17:17,688][20347] Updated weights for policy 0, policy_version 1536006 (0.0006) [2023-12-27 16:17:17,751][20347] Updated weights for policy 0, policy_version 1536016 (0.0008) [2023-12-27 16:17:17,805][20381] Updated weights for policy 1, policy_version 1534418 (0.0006) [2023-12-27 16:17:17,866][20381] Updated weights for policy 1, policy_version 1534428 (0.0007) [2023-12-27 16:17:17,925][20381] Updated weights for policy 1, policy_version 1534438 (0.0008) [2023-12-27 16:17:18,456][20347] Updated weights for policy 0, policy_version 1536026 (0.0005) [2023-12-27 16:17:18,521][20347] Updated weights for policy 0, policy_version 1536036 (0.0005) [2023-12-27 16:17:18,572][20381] Updated weights for policy 1, policy_version 1534448 (0.0008) [2023-12-27 16:17:18,580][20347] Updated weights for policy 0, policy_version 1536046 (0.0007) [2023-12-27 16:17:18,631][20381] Updated weights for policy 1, policy_version 1534458 (0.0008) [2023-12-27 16:17:18,690][20381] Updated weights for policy 1, policy_version 1534468 (0.0009) [2023-12-27 16:17:19,185][20347] Updated weights for policy 0, policy_version 1536056 (0.0006) [2023-12-27 16:17:19,249][20347] Updated weights for policy 0, policy_version 1536066 (0.0007) [2023-12-27 16:17:19,316][20347] Updated weights for policy 0, policy_version 1536076 (0.0008) [2023-12-27 16:17:19,392][20381] Updated weights for policy 1, policy_version 1534478 (0.0008) [2023-12-27 16:17:19,446][20381] Updated weights for policy 1, policy_version 1534488 (0.0008) [2023-12-27 16:17:19,507][20381] Updated weights for policy 1, policy_version 1534498 (0.0007) [2023-12-27 16:17:19,969][20347] Updated weights for policy 0, policy_version 1536086 (0.0010) [2023-12-27 16:17:20,029][20347] Updated weights for policy 0, policy_version 1536096 (0.0010) [2023-12-27 16:17:20,086][20347] Updated weights for policy 0, policy_version 1536106 (0.0010) [2023-12-27 16:17:20,239][20381] Updated weights for policy 1, policy_version 1534508 (0.0007) [2023-12-27 16:17:20,289][20381] Updated weights for policy 1, policy_version 1534518 (0.0008) [2023-12-27 16:17:20,342][20381] Updated weights for policy 1, policy_version 1534528 (0.0007) [2023-12-27 16:17:20,790][20347] Updated weights for policy 0, policy_version 1536116 (0.0009) [2023-12-27 16:17:20,856][20347] Updated weights for policy 0, policy_version 1536126 (0.0008) [2023-12-27 16:17:20,869][20291] KL-divergence is very high: 102.0606 [2023-12-27 16:17:20,917][20347] Updated weights for policy 0, policy_version 1536136 (0.0009) [2023-12-27 16:17:21,088][20381] Updated weights for policy 1, policy_version 1534538 (0.0009) [2023-12-27 16:17:21,158][20381] Updated weights for policy 1, policy_version 1534548 (0.0009) [2023-12-27 16:17:21,228][20381] Updated weights for policy 1, policy_version 1534558 (0.0009) [2023-12-27 16:17:21,294][20381] Updated weights for policy 1, policy_version 1534568 (0.0008) [2023-12-27 16:17:21,667][20347] Updated weights for policy 0, policy_version 1536146 (0.0009) [2023-12-27 16:17:21,740][20347] Updated weights for policy 0, policy_version 1536156 (0.0008) [2023-12-27 16:17:21,799][20347] Updated weights for policy 0, policy_version 1536166 (0.0009) [2023-12-27 16:17:21,853][20347] Updated weights for policy 0, policy_version 1536176 (0.0007) [2023-12-27 16:17:21,990][20381] Updated weights for policy 1, policy_version 1534578 (0.0006) [2023-12-27 16:17:22,048][20381] Updated weights for policy 1, policy_version 1534588 (0.0006) [2023-12-27 16:17:22,109][20381] Updated weights for policy 1, policy_version 1534598 (0.0006) [2023-12-27 16:17:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 786235392. Throughput: 0: 10262.3, 1: 10164.9. Samples: 786207332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:22,363][19170] Avg episode reward: [(0, '853.070'), (1, '990.260')] [2023-12-27 16:17:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001534600_392896512.pth... [2023-12-27 16:17:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001536176_393338880.pth... [2023-12-27 16:17:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001533416_392593408.pth [2023-12-27 16:17:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001534992_393035776.pth [2023-12-27 16:17:22,569][20347] Updated weights for policy 0, policy_version 1536186 (0.0010) [2023-12-27 16:17:22,622][20347] Updated weights for policy 0, policy_version 1536196 (0.0011) [2023-12-27 16:17:22,684][20347] Updated weights for policy 0, policy_version 1536206 (0.0010) [2023-12-27 16:17:22,763][20381] Updated weights for policy 1, policy_version 1534608 (0.0006) [2023-12-27 16:17:22,816][20381] Updated weights for policy 1, policy_version 1534618 (0.0008) [2023-12-27 16:17:22,880][20381] Updated weights for policy 1, policy_version 1534628 (0.0008) [2023-12-27 16:17:23,378][20347] Updated weights for policy 0, policy_version 1536216 (0.0007) [2023-12-27 16:17:23,430][20347] Updated weights for policy 0, policy_version 1536226 (0.0005) [2023-12-27 16:17:23,484][20347] Updated weights for policy 0, policy_version 1536236 (0.0005) [2023-12-27 16:17:23,530][20381] Updated weights for policy 1, policy_version 1534638 (0.0008) [2023-12-27 16:17:23,595][20381] Updated weights for policy 1, policy_version 1534648 (0.0009) [2023-12-27 16:17:23,646][20381] Updated weights for policy 1, policy_version 1534658 (0.0007) [2023-12-27 16:17:24,089][20347] Updated weights for policy 0, policy_version 1536246 (0.0008) [2023-12-27 16:17:24,147][20347] Updated weights for policy 0, policy_version 1536256 (0.0010) [2023-12-27 16:17:24,196][20347] Updated weights for policy 0, policy_version 1536266 (0.0010) [2023-12-27 16:17:24,273][20381] Updated weights for policy 1, policy_version 1534668 (0.0007) [2023-12-27 16:17:24,329][20381] Updated weights for policy 1, policy_version 1534678 (0.0005) [2023-12-27 16:17:24,382][20381] Updated weights for policy 1, policy_version 1534688 (0.0008) [2023-12-27 16:17:24,767][20347] Updated weights for policy 0, policy_version 1536276 (0.0010) [2023-12-27 16:17:24,824][20347] Updated weights for policy 0, policy_version 1536286 (0.0009) [2023-12-27 16:17:24,881][20347] Updated weights for policy 0, policy_version 1536296 (0.0010) [2023-12-27 16:17:25,039][20381] Updated weights for policy 1, policy_version 1534698 (0.0007) [2023-12-27 16:17:25,093][20381] Updated weights for policy 1, policy_version 1534708 (0.0005) [2023-12-27 16:17:25,146][20381] Updated weights for policy 1, policy_version 1534718 (0.0006) [2023-12-27 16:17:25,209][20381] Updated weights for policy 1, policy_version 1534728 (0.0006) [2023-12-27 16:17:25,499][20347] Updated weights for policy 0, policy_version 1536306 (0.0010) [2023-12-27 16:17:25,543][20347] Updated weights for policy 0, policy_version 1536316 (0.0010) [2023-12-27 16:17:25,591][20347] Updated weights for policy 0, policy_version 1536326 (0.0010) [2023-12-27 16:17:25,638][20347] Updated weights for policy 0, policy_version 1536336 (0.0010) [2023-12-27 16:17:25,847][20381] Updated weights for policy 1, policy_version 1534738 (0.0007) [2023-12-27 16:17:25,900][20381] Updated weights for policy 1, policy_version 1534748 (0.0008) [2023-12-27 16:17:25,955][20381] Updated weights for policy 1, policy_version 1534758 (0.0005) [2023-12-27 16:17:26,318][20347] Updated weights for policy 0, policy_version 1536346 (0.0010) [2023-12-27 16:17:26,368][20347] Updated weights for policy 0, policy_version 1536356 (0.0010) [2023-12-27 16:17:26,423][20347] Updated weights for policy 0, policy_version 1536366 (0.0010) [2023-12-27 16:17:26,560][20381] Updated weights for policy 1, policy_version 1534768 (0.0009) [2023-12-27 16:17:26,605][20381] Updated weights for policy 1, policy_version 1534778 (0.0010) [2023-12-27 16:17:26,653][20381] Updated weights for policy 1, policy_version 1534788 (0.0010) [2023-12-27 16:17:27,043][20347] Updated weights for policy 0, policy_version 1536376 (0.0007) [2023-12-27 16:17:27,103][20347] Updated weights for policy 0, policy_version 1536386 (0.0007) [2023-12-27 16:17:27,156][20347] Updated weights for policy 0, policy_version 1536396 (0.0007) [2023-12-27 16:17:27,321][20381] Updated weights for policy 1, policy_version 1534798 (0.0009) [2023-12-27 16:17:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 786341888. Throughput: 0: 10299.4, 1: 10181.6. Samples: 786331424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:27,363][19170] Avg episode reward: [(0, '853.200'), (1, '1000.000')] [2023-12-27 16:17:27,371][20381] Updated weights for policy 1, policy_version 1534808 (0.0010) [2023-12-27 16:17:27,418][20381] Updated weights for policy 1, policy_version 1534818 (0.0010) [2023-12-27 16:17:27,772][20347] Updated weights for policy 0, policy_version 1536406 (0.0007) [2023-12-27 16:17:27,823][20347] Updated weights for policy 0, policy_version 1536416 (0.0007) [2023-12-27 16:17:27,877][20347] Updated weights for policy 0, policy_version 1536426 (0.0008) [2023-12-27 16:17:27,981][20381] Updated weights for policy 1, policy_version 1534828 (0.0008) [2023-12-27 16:17:28,029][20381] Updated weights for policy 1, policy_version 1534838 (0.0009) [2023-12-27 16:17:28,078][20381] Updated weights for policy 1, policy_version 1534848 (0.0009) [2023-12-27 16:17:28,492][20347] Updated weights for policy 0, policy_version 1536436 (0.0006) [2023-12-27 16:17:28,547][20347] Updated weights for policy 0, policy_version 1536446 (0.0008) [2023-12-27 16:17:28,603][20347] Updated weights for policy 0, policy_version 1536456 (0.0008) [2023-12-27 16:17:28,673][20381] Updated weights for policy 1, policy_version 1534858 (0.0008) [2023-12-27 16:17:28,731][20381] Updated weights for policy 1, policy_version 1534868 (0.0008) [2023-12-27 16:17:28,783][20381] Updated weights for policy 1, policy_version 1534878 (0.0006) [2023-12-27 16:17:28,838][20381] Updated weights for policy 1, policy_version 1534888 (0.0009) [2023-12-27 16:17:29,247][20347] Updated weights for policy 0, policy_version 1536466 (0.0007) [2023-12-27 16:17:29,313][20347] Updated weights for policy 0, policy_version 1536476 (0.0007) [2023-12-27 16:17:29,381][20347] Updated weights for policy 0, policy_version 1536486 (0.0008) [2023-12-27 16:17:29,438][20347] Updated weights for policy 0, policy_version 1536496 (0.0007) [2023-12-27 16:17:29,524][20381] Updated weights for policy 1, policy_version 1534898 (0.0007) [2023-12-27 16:17:29,575][20381] Updated weights for policy 1, policy_version 1534908 (0.0005) [2023-12-27 16:17:29,641][20381] Updated weights for policy 1, policy_version 1534918 (0.0007) [2023-12-27 16:17:30,061][20347] Updated weights for policy 0, policy_version 1536506 (0.0007) [2023-12-27 16:17:30,123][20347] Updated weights for policy 0, policy_version 1536516 (0.0008) [2023-12-27 16:17:30,181][20347] Updated weights for policy 0, policy_version 1536526 (0.0007) [2023-12-27 16:17:30,359][20381] Updated weights for policy 1, policy_version 1534928 (0.0008) [2023-12-27 16:17:30,409][20381] Updated weights for policy 1, policy_version 1534938 (0.0008) [2023-12-27 16:17:30,462][20381] Updated weights for policy 1, policy_version 1534948 (0.0008) [2023-12-27 16:17:30,851][20347] Updated weights for policy 0, policy_version 1536536 (0.0007) [2023-12-27 16:17:30,903][20347] Updated weights for policy 0, policy_version 1536546 (0.0007) [2023-12-27 16:17:30,951][20347] Updated weights for policy 0, policy_version 1536556 (0.0005) [2023-12-27 16:17:31,132][20381] Updated weights for policy 1, policy_version 1534958 (0.0007) [2023-12-27 16:17:31,199][20381] Updated weights for policy 1, policy_version 1534968 (0.0008) [2023-12-27 16:17:31,262][20381] Updated weights for policy 1, policy_version 1534978 (0.0007) [2023-12-27 16:17:31,696][20347] Updated weights for policy 0, policy_version 1536566 (0.0007) [2023-12-27 16:17:31,752][20347] Updated weights for policy 0, policy_version 1536576 (0.0008) [2023-12-27 16:17:31,796][20347] Updated weights for policy 0, policy_version 1536586 (0.0008) [2023-12-27 16:17:31,984][20381] Updated weights for policy 1, policy_version 1534988 (0.0008) [2023-12-27 16:17:32,037][20381] Updated weights for policy 1, policy_version 1534998 (0.0008) [2023-12-27 16:17:32,089][20381] Updated weights for policy 1, policy_version 1535008 (0.0007) [2023-12-27 16:17:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 786448384. Throughput: 0: 10379.7, 1: 10253.2. Samples: 786458364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:32,363][19170] Avg episode reward: [(0, '813.790'), (1, '1000.000')] [2023-12-27 16:17:32,425][20347] Updated weights for policy 0, policy_version 1536596 (0.0008) [2023-12-27 16:17:32,476][20347] Updated weights for policy 0, policy_version 1536606 (0.0007) [2023-12-27 16:17:32,488][20291] KL-divergence is very high: 104.5722 [2023-12-27 16:17:32,528][20347] Updated weights for policy 0, policy_version 1536616 (0.0007) [2023-12-27 16:17:32,759][20381] Updated weights for policy 1, policy_version 1535018 (0.0008) [2023-12-27 16:17:32,811][20381] Updated weights for policy 1, policy_version 1535028 (0.0005) [2023-12-27 16:17:32,876][20381] Updated weights for policy 1, policy_version 1535038 (0.0009) [2023-12-27 16:17:32,929][20381] Updated weights for policy 1, policy_version 1535048 (0.0011) [2023-12-27 16:17:33,224][20347] Updated weights for policy 0, policy_version 1536626 (0.0007) [2023-12-27 16:17:33,274][20291] KL-divergence is very high: 124.7615 [2023-12-27 16:17:33,280][20347] Updated weights for policy 0, policy_version 1536636 (0.0008) [2023-12-27 16:17:33,293][20291] KL-divergence is very high: 159.7644 [2023-12-27 16:17:33,298][20291] KL-divergence is very high: 104.6885 [2023-12-27 16:17:33,302][20291] KL-divergence is very high: 152.8557 [2023-12-27 16:17:33,312][20291] KL-divergence is very high: 138.7365 [2023-12-27 16:17:33,330][20347] Updated weights for policy 0, policy_version 1536646 (0.0006) [2023-12-27 16:17:33,389][20347] Updated weights for policy 0, policy_version 1536656 (0.0011) [2023-12-27 16:17:33,561][20381] Updated weights for policy 1, policy_version 1535058 (0.0011) [2023-12-27 16:17:33,610][20381] Updated weights for policy 1, policy_version 1535068 (0.0010) [2023-12-27 16:17:33,655][20381] Updated weights for policy 1, policy_version 1535078 (0.0010) [2023-12-27 16:17:33,984][20347] Updated weights for policy 0, policy_version 1536666 (0.0005) [2023-12-27 16:17:34,042][20347] Updated weights for policy 0, policy_version 1536676 (0.0005) [2023-12-27 16:17:34,100][20347] Updated weights for policy 0, policy_version 1536686 (0.0005) [2023-12-27 16:17:34,301][20381] Updated weights for policy 1, policy_version 1535088 (0.0011) [2023-12-27 16:17:34,355][20381] Updated weights for policy 1, policy_version 1535098 (0.0011) [2023-12-27 16:17:34,415][20381] Updated weights for policy 1, policy_version 1535108 (0.0011) [2023-12-27 16:17:34,729][20347] Updated weights for policy 0, policy_version 1536696 (0.0007) [2023-12-27 16:17:34,788][20347] Updated weights for policy 0, policy_version 1536706 (0.0007) [2023-12-27 16:17:34,844][20347] Updated weights for policy 0, policy_version 1536716 (0.0008) [2023-12-27 16:17:35,051][20381] Updated weights for policy 1, policy_version 1535118 (0.0008) [2023-12-27 16:17:35,102][20381] Updated weights for policy 1, policy_version 1535128 (0.0007) [2023-12-27 16:17:35,156][20381] Updated weights for policy 1, policy_version 1535138 (0.0007) [2023-12-27 16:17:35,438][20347] Updated weights for policy 0, policy_version 1536726 (0.0005) [2023-12-27 16:17:35,492][20347] Updated weights for policy 0, policy_version 1536736 (0.0008) [2023-12-27 16:17:35,545][20347] Updated weights for policy 0, policy_version 1536746 (0.0007) [2023-12-27 16:17:35,777][20381] Updated weights for policy 1, policy_version 1535148 (0.0009) [2023-12-27 16:17:35,832][20381] Updated weights for policy 1, policy_version 1535158 (0.0010) [2023-12-27 16:17:35,889][20381] Updated weights for policy 1, policy_version 1535168 (0.0010) [2023-12-27 16:17:36,219][20347] Updated weights for policy 0, policy_version 1536756 (0.0008) [2023-12-27 16:17:36,277][20347] Updated weights for policy 0, policy_version 1536766 (0.0010) [2023-12-27 16:17:36,334][20347] Updated weights for policy 0, policy_version 1536776 (0.0010) [2023-12-27 16:17:36,346][20291] KL-divergence is very high: 111.4350 [2023-12-27 16:17:36,364][20291] KL-divergence is very high: 148.9282 [2023-12-27 16:17:36,371][20291] KL-divergence is very high: 150.7896 [2023-12-27 16:17:36,377][20291] KL-divergence is very high: 145.8703 [2023-12-27 16:17:36,598][20381] Updated weights for policy 1, policy_version 1535178 (0.0009) [2023-12-27 16:17:36,660][20381] Updated weights for policy 1, policy_version 1535188 (0.0008) [2023-12-27 16:17:36,711][20381] Updated weights for policy 1, policy_version 1535198 (0.0007) [2023-12-27 16:17:36,763][20381] Updated weights for policy 1, policy_version 1535208 (0.0006) [2023-12-27 16:17:37,005][20347] Updated weights for policy 0, policy_version 1536786 (0.0009) [2023-12-27 16:17:37,056][20347] Updated weights for policy 0, policy_version 1536796 (0.0010) [2023-12-27 16:17:37,116][20347] Updated weights for policy 0, policy_version 1536806 (0.0010) [2023-12-27 16:17:37,172][20347] Updated weights for policy 0, policy_version 1536816 (0.0006) [2023-12-27 16:17:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 786554880. Throughput: 0: 10431.7, 1: 10299.4. Samples: 786522624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:37,363][19170] Avg episode reward: [(0, '803.790'), (1, '1000.000')] [2023-12-27 16:17:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001536816_393502720.pth... [2023-12-27 16:17:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001535600_393191424.pth [2023-12-27 16:17:37,402][20381] Updated weights for policy 1, policy_version 1535218 (0.0011) [2023-12-27 16:17:37,457][20381] Updated weights for policy 1, policy_version 1535228 (0.0010) [2023-12-27 16:17:37,505][20381] Updated weights for policy 1, policy_version 1535238 (0.0010) [2023-12-27 16:17:37,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001535240_393060352.pth... [2023-12-27 16:17:37,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001533992_392740864.pth [2023-12-27 16:17:37,816][20347] Updated weights for policy 0, policy_version 1536826 (0.0007) [2023-12-27 16:17:37,868][20347] Updated weights for policy 0, policy_version 1536836 (0.0006) [2023-12-27 16:17:37,919][20347] Updated weights for policy 0, policy_version 1536846 (0.0005) [2023-12-27 16:17:38,183][20381] Updated weights for policy 1, policy_version 1535248 (0.0010) [2023-12-27 16:17:38,232][20381] Updated weights for policy 1, policy_version 1535258 (0.0010) [2023-12-27 16:17:38,281][20381] Updated weights for policy 1, policy_version 1535268 (0.0010) [2023-12-27 16:17:38,528][20347] Updated weights for policy 0, policy_version 1536856 (0.0007) [2023-12-27 16:17:38,587][20347] Updated weights for policy 0, policy_version 1536866 (0.0007) [2023-12-27 16:17:38,643][20347] Updated weights for policy 0, policy_version 1536876 (0.0008) [2023-12-27 16:17:39,012][20381] Updated weights for policy 1, policy_version 1535278 (0.0010) [2023-12-27 16:17:39,073][20381] Updated weights for policy 1, policy_version 1535288 (0.0011) [2023-12-27 16:17:39,124][20381] Updated weights for policy 1, policy_version 1535298 (0.0010) [2023-12-27 16:17:39,353][20347] Updated weights for policy 0, policy_version 1536886 (0.0008) [2023-12-27 16:17:39,427][20347] Updated weights for policy 0, policy_version 1536896 (0.0008) [2023-12-27 16:17:39,481][20347] Updated weights for policy 0, policy_version 1536906 (0.0008) [2023-12-27 16:17:39,840][20381] Updated weights for policy 1, policy_version 1535308 (0.0010) [2023-12-27 16:17:39,902][20381] Updated weights for policy 1, policy_version 1535318 (0.0011) [2023-12-27 16:17:39,965][20381] Updated weights for policy 1, policy_version 1535328 (0.0009) [2023-12-27 16:17:40,227][20347] Updated weights for policy 0, policy_version 1536916 (0.0008) [2023-12-27 16:17:40,287][20347] Updated weights for policy 0, policy_version 1536926 (0.0007) [2023-12-27 16:17:40,348][20347] Updated weights for policy 0, policy_version 1536936 (0.0007) [2023-12-27 16:17:40,671][20381] Updated weights for policy 1, policy_version 1535338 (0.0010) [2023-12-27 16:17:40,730][20381] Updated weights for policy 1, policy_version 1535348 (0.0009) [2023-12-27 16:17:40,788][20381] Updated weights for policy 1, policy_version 1535358 (0.0008) [2023-12-27 16:17:40,843][20381] Updated weights for policy 1, policy_version 1535368 (0.0007) [2023-12-27 16:17:40,944][20347] Updated weights for policy 0, policy_version 1536946 (0.0007) [2023-12-27 16:17:40,994][20347] Updated weights for policy 0, policy_version 1536956 (0.0007) [2023-12-27 16:17:41,050][20347] Updated weights for policy 0, policy_version 1536966 (0.0007) [2023-12-27 16:17:41,102][20347] Updated weights for policy 0, policy_version 1536976 (0.0008) [2023-12-27 16:17:41,535][20381] Updated weights for policy 1, policy_version 1535378 (0.0007) [2023-12-27 16:17:41,598][20381] Updated weights for policy 1, policy_version 1535388 (0.0007) [2023-12-27 16:17:41,666][20381] Updated weights for policy 1, policy_version 1535398 (0.0008) [2023-12-27 16:17:41,866][20347] Updated weights for policy 0, policy_version 1536986 (0.0007) [2023-12-27 16:17:41,925][20347] Updated weights for policy 0, policy_version 1536996 (0.0008) [2023-12-27 16:17:41,979][20347] Updated weights for policy 0, policy_version 1537006 (0.0008) [2023-12-27 16:17:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 786653184. Throughput: 0: 10438.0, 1: 10290.1. Samples: 786646160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:42,363][19170] Avg episode reward: [(0, '774.610'), (1, '990.250')] [2023-12-27 16:17:42,409][20381] Updated weights for policy 1, policy_version 1535408 (0.0007) [2023-12-27 16:17:42,464][20381] Updated weights for policy 1, policy_version 1535418 (0.0008) [2023-12-27 16:17:42,521][20381] Updated weights for policy 1, policy_version 1535428 (0.0006) [2023-12-27 16:17:42,685][20347] Updated weights for policy 0, policy_version 1537016 (0.0006) [2023-12-27 16:17:42,743][20347] Updated weights for policy 0, policy_version 1537026 (0.0008) [2023-12-27 16:17:42,796][20347] Updated weights for policy 0, policy_version 1537036 (0.0007) [2023-12-27 16:17:43,162][20381] Updated weights for policy 1, policy_version 1535438 (0.0006) [2023-12-27 16:17:43,227][20381] Updated weights for policy 1, policy_version 1535448 (0.0005) [2023-12-27 16:17:43,292][20381] Updated weights for policy 1, policy_version 1535458 (0.0005) [2023-12-27 16:17:43,420][20347] Updated weights for policy 0, policy_version 1537046 (0.0007) [2023-12-27 16:17:43,469][20347] Updated weights for policy 0, policy_version 1537056 (0.0008) [2023-12-27 16:17:43,517][20347] Updated weights for policy 0, policy_version 1537066 (0.0007) [2023-12-27 16:17:43,889][20381] Updated weights for policy 1, policy_version 1535468 (0.0007) [2023-12-27 16:17:43,949][20381] Updated weights for policy 1, policy_version 1535478 (0.0006) [2023-12-27 16:17:44,005][20381] Updated weights for policy 1, policy_version 1535488 (0.0006) [2023-12-27 16:17:44,262][20347] Updated weights for policy 0, policy_version 1537076 (0.0008) [2023-12-27 16:17:44,311][20347] Updated weights for policy 0, policy_version 1537086 (0.0007) [2023-12-27 16:17:44,366][20347] Updated weights for policy 0, policy_version 1537096 (0.0008) [2023-12-27 16:17:44,578][20381] Updated weights for policy 1, policy_version 1535498 (0.0007) [2023-12-27 16:17:44,632][20381] Updated weights for policy 1, policy_version 1535508 (0.0005) [2023-12-27 16:17:44,689][20381] Updated weights for policy 1, policy_version 1535518 (0.0006) [2023-12-27 16:17:44,743][20381] Updated weights for policy 1, policy_version 1535528 (0.0008) [2023-12-27 16:17:45,098][20347] Updated weights for policy 0, policy_version 1537106 (0.0008) [2023-12-27 16:17:45,156][20347] Updated weights for policy 0, policy_version 1537116 (0.0007) [2023-12-27 16:17:45,202][20347] Updated weights for policy 0, policy_version 1537126 (0.0008) [2023-12-27 16:17:45,254][20347] Updated weights for policy 0, policy_version 1537136 (0.0006) [2023-12-27 16:17:45,415][20381] Updated weights for policy 1, policy_version 1535538 (0.0008) [2023-12-27 16:17:45,468][20381] Updated weights for policy 1, policy_version 1535548 (0.0008) [2023-12-27 16:17:45,528][20381] Updated weights for policy 1, policy_version 1535558 (0.0008) [2023-12-27 16:17:45,909][20347] Updated weights for policy 0, policy_version 1537146 (0.0007) [2023-12-27 16:17:45,966][20347] Updated weights for policy 0, policy_version 1537156 (0.0007) [2023-12-27 16:17:46,014][20347] Updated weights for policy 0, policy_version 1537166 (0.0007) [2023-12-27 16:17:46,227][20381] Updated weights for policy 1, policy_version 1535568 (0.0006) [2023-12-27 16:17:46,279][20381] Updated weights for policy 1, policy_version 1535578 (0.0005) [2023-12-27 16:17:46,331][20381] Updated weights for policy 1, policy_version 1535588 (0.0006) [2023-12-27 16:17:46,748][20347] Updated weights for policy 0, policy_version 1537176 (0.0007) [2023-12-27 16:17:46,796][20347] Updated weights for policy 0, policy_version 1537186 (0.0007) [2023-12-27 16:17:46,842][20347] Updated weights for policy 0, policy_version 1537196 (0.0007) [2023-12-27 16:17:46,846][20381] Updated weights for policy 1, policy_version 1535598 (0.0005) [2023-12-27 16:17:46,897][20381] Updated weights for policy 1, policy_version 1535608 (0.0005) [2023-12-27 16:17:46,953][20381] Updated weights for policy 1, policy_version 1535618 (0.0007) [2023-12-27 16:17:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 786759680. Throughput: 0: 10383.0, 1: 10358.6. Samples: 786769512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:47,363][19170] Avg episode reward: [(0, '814.160'), (1, '980.620')] [2023-12-27 16:17:47,598][20347] Updated weights for policy 0, policy_version 1537206 (0.0010) [2023-12-27 16:17:47,605][20381] Updated weights for policy 1, policy_version 1535628 (0.0008) [2023-12-27 16:17:47,655][20381] Updated weights for policy 1, policy_version 1535638 (0.0007) [2023-12-27 16:17:47,656][20347] Updated weights for policy 0, policy_version 1537216 (0.0010) [2023-12-27 16:17:47,722][20381] Updated weights for policy 1, policy_version 1535648 (0.0006) [2023-12-27 16:17:47,725][20347] Updated weights for policy 0, policy_version 1537226 (0.0010) [2023-12-27 16:17:48,359][20381] Updated weights for policy 1, policy_version 1535658 (0.0009) [2023-12-27 16:17:48,417][20381] Updated weights for policy 1, policy_version 1535668 (0.0007) [2023-12-27 16:17:48,423][20347] Updated weights for policy 0, policy_version 1537236 (0.0010) [2023-12-27 16:17:48,470][20292] KL-divergence is very high: 269.9985 [2023-12-27 16:17:48,472][20381] Updated weights for policy 1, policy_version 1535678 (0.0008) [2023-12-27 16:17:48,478][20347] Updated weights for policy 0, policy_version 1537246 (0.0009) [2023-12-27 16:17:48,518][20381] Updated weights for policy 1, policy_version 1535688 (0.0005) [2023-12-27 16:17:48,533][20347] Updated weights for policy 0, policy_version 1537256 (0.0010) [2023-12-27 16:17:49,217][20347] Updated weights for policy 0, policy_version 1537266 (0.0010) [2023-12-27 16:17:49,224][20381] Updated weights for policy 1, policy_version 1535698 (0.0007) [2023-12-27 16:17:49,285][20347] Updated weights for policy 0, policy_version 1537276 (0.0011) [2023-12-27 16:17:49,293][20381] Updated weights for policy 1, policy_version 1535708 (0.0011) [2023-12-27 16:17:49,351][20347] Updated weights for policy 0, policy_version 1537286 (0.0010) [2023-12-27 16:17:49,365][20381] Updated weights for policy 1, policy_version 1535718 (0.0010) [2023-12-27 16:17:49,410][20347] Updated weights for policy 0, policy_version 1537296 (0.0008) [2023-12-27 16:17:50,012][20381] Updated weights for policy 1, policy_version 1535728 (0.0007) [2023-12-27 16:17:50,065][20381] Updated weights for policy 1, policy_version 1535738 (0.0007) [2023-12-27 16:17:50,121][20347] Updated weights for policy 0, policy_version 1537306 (0.0008) [2023-12-27 16:17:50,122][20381] Updated weights for policy 1, policy_version 1535748 (0.0008) [2023-12-27 16:17:50,178][20347] Updated weights for policy 0, policy_version 1537316 (0.0008) [2023-12-27 16:17:50,235][20347] Updated weights for policy 0, policy_version 1537326 (0.0008) [2023-12-27 16:17:50,791][20381] Updated weights for policy 1, policy_version 1535758 (0.0006) [2023-12-27 16:17:50,854][20381] Updated weights for policy 1, policy_version 1535768 (0.0005) [2023-12-27 16:17:50,915][20381] Updated weights for policy 1, policy_version 1535778 (0.0007) [2023-12-27 16:17:50,949][20347] Updated weights for policy 0, policy_version 1537336 (0.0010) [2023-12-27 16:17:51,012][20347] Updated weights for policy 0, policy_version 1537346 (0.0007) [2023-12-27 16:17:51,074][20347] Updated weights for policy 0, policy_version 1537356 (0.0011) [2023-12-27 16:17:51,593][20381] Updated weights for policy 1, policy_version 1535788 (0.0007) [2023-12-27 16:17:51,654][20381] Updated weights for policy 1, policy_version 1535798 (0.0009) [2023-12-27 16:17:51,714][20381] Updated weights for policy 1, policy_version 1535808 (0.0008) [2023-12-27 16:17:51,805][20347] Updated weights for policy 0, policy_version 1537366 (0.0007) [2023-12-27 16:17:51,862][20347] Updated weights for policy 0, policy_version 1537376 (0.0006) [2023-12-27 16:17:51,917][20347] Updated weights for policy 0, policy_version 1537386 (0.0008) [2023-12-27 16:17:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 786857984. Throughput: 0: 10358.6, 1: 10374.1. Samples: 786829720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:52,363][19170] Avg episode reward: [(0, '862.890'), (1, '970.880')] [2023-12-27 16:17:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001537392_393650176.pth... [2023-12-27 16:17:52,373][20381] Updated weights for policy 1, policy_version 1535818 (0.0007) [2023-12-27 16:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001536176_393338880.pth [2023-12-27 16:17:52,432][20381] Updated weights for policy 1, policy_version 1535828 (0.0006) [2023-12-27 16:17:52,486][20381] Updated weights for policy 1, policy_version 1535838 (0.0008) [2023-12-27 16:17:52,541][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001535848_393216000.pth... [2023-12-27 16:17:52,542][20381] Updated weights for policy 1, policy_version 1535848 (0.0006) [2023-12-27 16:17:52,544][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001534600_392896512.pth [2023-12-27 16:17:52,574][20347] Updated weights for policy 0, policy_version 1537396 (0.0009) [2023-12-27 16:17:52,633][20347] Updated weights for policy 0, policy_version 1537406 (0.0010) [2023-12-27 16:17:52,695][20347] Updated weights for policy 0, policy_version 1537416 (0.0009) [2023-12-27 16:17:53,126][20381] Updated weights for policy 1, policy_version 1535858 (0.0006) [2023-12-27 16:17:53,181][20381] Updated weights for policy 1, policy_version 1535868 (0.0005) [2023-12-27 16:17:53,237][20381] Updated weights for policy 1, policy_version 1535878 (0.0009) [2023-12-27 16:17:53,318][20347] Updated weights for policy 0, policy_version 1537426 (0.0007) [2023-12-27 16:17:53,373][20347] Updated weights for policy 0, policy_version 1537436 (0.0007) [2023-12-27 16:17:53,425][20347] Updated weights for policy 0, policy_version 1537446 (0.0005) [2023-12-27 16:17:53,474][20347] Updated weights for policy 0, policy_version 1537456 (0.0005) [2023-12-27 16:17:53,826][20381] Updated weights for policy 1, policy_version 1535888 (0.0010) [2023-12-27 16:17:53,883][20381] Updated weights for policy 1, policy_version 1535898 (0.0010) [2023-12-27 16:17:53,956][20381] Updated weights for policy 1, policy_version 1535908 (0.0010) [2023-12-27 16:17:54,059][20347] Updated weights for policy 0, policy_version 1537466 (0.0007) [2023-12-27 16:17:54,115][20347] Updated weights for policy 0, policy_version 1537476 (0.0007) [2023-12-27 16:17:54,171][20347] Updated weights for policy 0, policy_version 1537486 (0.0008) [2023-12-27 16:17:54,638][20381] Updated weights for policy 1, policy_version 1535918 (0.0006) [2023-12-27 16:17:54,684][20381] Updated weights for policy 1, policy_version 1535928 (0.0010) [2023-12-27 16:17:54,737][20381] Updated weights for policy 1, policy_version 1535938 (0.0007) [2023-12-27 16:17:54,831][20347] Updated weights for policy 0, policy_version 1537496 (0.0009) [2023-12-27 16:17:54,884][20347] Updated weights for policy 0, policy_version 1537506 (0.0008) [2023-12-27 16:17:54,931][20347] Updated weights for policy 0, policy_version 1537516 (0.0006) [2023-12-27 16:17:55,420][20381] Updated weights for policy 1, policy_version 1535948 (0.0006) [2023-12-27 16:17:55,472][20381] Updated weights for policy 1, policy_version 1535958 (0.0007) [2023-12-27 16:17:55,520][20381] Updated weights for policy 1, policy_version 1535968 (0.0007) [2023-12-27 16:17:55,621][20347] Updated weights for policy 0, policy_version 1537526 (0.0005) [2023-12-27 16:17:55,667][20347] Updated weights for policy 0, policy_version 1537536 (0.0005) [2023-12-27 16:17:55,726][20347] Updated weights for policy 0, policy_version 1537546 (0.0005) [2023-12-27 16:17:56,186][20381] Updated weights for policy 1, policy_version 1535978 (0.0007) [2023-12-27 16:17:56,240][20381] Updated weights for policy 1, policy_version 1535988 (0.0006) [2023-12-27 16:17:56,246][20347] Updated weights for policy 0, policy_version 1537556 (0.0006) [2023-12-27 16:17:56,293][20381] Updated weights for policy 1, policy_version 1535998 (0.0006) [2023-12-27 16:17:56,297][20347] Updated weights for policy 0, policy_version 1537566 (0.0006) [2023-12-27 16:17:56,346][20381] Updated weights for policy 1, policy_version 1536008 (0.0006) [2023-12-27 16:17:56,351][20347] Updated weights for policy 0, policy_version 1537576 (0.0006) [2023-12-27 16:17:56,917][20347] Updated weights for policy 0, policy_version 1537586 (0.0006) [2023-12-27 16:17:56,970][20347] Updated weights for policy 0, policy_version 1537596 (0.0007) [2023-12-27 16:17:56,978][20381] Updated weights for policy 1, policy_version 1536018 (0.0010) [2023-12-27 16:17:57,028][20347] Updated weights for policy 0, policy_version 1537606 (0.0007) [2023-12-27 16:17:57,029][20381] Updated weights for policy 1, policy_version 1536028 (0.0010) [2023-12-27 16:17:57,081][20347] Updated weights for policy 0, policy_version 1537616 (0.0005) [2023-12-27 16:17:57,083][20381] Updated weights for policy 1, policy_version 1536038 (0.0010) [2023-12-27 16:17:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 786972672. Throughput: 0: 10414.5, 1: 10409.9. Samples: 786957404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:17:57,363][19170] Avg episode reward: [(0, '824.170'), (1, '961.790')] [2023-12-27 16:17:57,727][20347] Updated weights for policy 0, policy_version 1537626 (0.0007) [2023-12-27 16:17:57,757][20381] Updated weights for policy 1, policy_version 1536048 (0.0006) [2023-12-27 16:17:57,770][20347] Updated weights for policy 0, policy_version 1537636 (0.0007) [2023-12-27 16:17:57,809][20381] Updated weights for policy 1, policy_version 1536058 (0.0006) [2023-12-27 16:17:57,833][20347] Updated weights for policy 0, policy_version 1537646 (0.0008) [2023-12-27 16:17:57,869][20381] Updated weights for policy 1, policy_version 1536068 (0.0005) [2023-12-27 16:17:58,504][20381] Updated weights for policy 1, policy_version 1536078 (0.0009) [2023-12-27 16:17:58,511][20347] Updated weights for policy 0, policy_version 1537656 (0.0008) [2023-12-27 16:17:58,568][20381] Updated weights for policy 1, policy_version 1536088 (0.0011) [2023-12-27 16:17:58,574][20347] Updated weights for policy 0, policy_version 1537666 (0.0008) [2023-12-27 16:17:58,633][20381] Updated weights for policy 1, policy_version 1536098 (0.0007) [2023-12-27 16:17:58,636][20347] Updated weights for policy 0, policy_version 1537676 (0.0007) [2023-12-27 16:17:59,346][20347] Updated weights for policy 0, policy_version 1537686 (0.0008) [2023-12-27 16:17:59,366][20381] Updated weights for policy 1, policy_version 1536108 (0.0009) [2023-12-27 16:17:59,424][20347] Updated weights for policy 0, policy_version 1537697 (0.0008) [2023-12-27 16:17:59,427][20381] Updated weights for policy 1, policy_version 1536118 (0.0008) [2023-12-27 16:17:59,481][20347] Updated weights for policy 0, policy_version 1537707 (0.0007) [2023-12-27 16:17:59,487][20381] Updated weights for policy 1, policy_version 1536128 (0.0008) [2023-12-27 16:18:00,101][20381] Updated weights for policy 1, policy_version 1536138 (0.0006) [2023-12-27 16:18:00,151][20381] Updated weights for policy 1, policy_version 1536148 (0.0007) [2023-12-27 16:18:00,179][20347] Updated weights for policy 0, policy_version 1537717 (0.0007) [2023-12-27 16:18:00,203][20381] Updated weights for policy 1, policy_version 1536158 (0.0010) [2023-12-27 16:18:00,232][20347] Updated weights for policy 0, policy_version 1537727 (0.0005) [2023-12-27 16:18:00,259][20381] Updated weights for policy 1, policy_version 1536168 (0.0011) [2023-12-27 16:18:00,293][20347] Updated weights for policy 0, policy_version 1537737 (0.0006) [2023-12-27 16:18:00,901][20347] Updated weights for policy 0, policy_version 1537747 (0.0008) [2023-12-27 16:18:00,920][20381] Updated weights for policy 1, policy_version 1536178 (0.0010) [2023-12-27 16:18:00,955][20347] Updated weights for policy 0, policy_version 1537757 (0.0007) [2023-12-27 16:18:00,970][20381] Updated weights for policy 1, policy_version 1536188 (0.0010) [2023-12-27 16:18:01,008][20347] Updated weights for policy 0, policy_version 1537767 (0.0007) [2023-12-27 16:18:01,023][20381] Updated weights for policy 1, policy_version 1536198 (0.0009) [2023-12-27 16:18:01,746][20381] Updated weights for policy 1, policy_version 1536208 (0.0010) [2023-12-27 16:18:01,769][20347] Updated weights for policy 0, policy_version 1537777 (0.0007) [2023-12-27 16:18:01,799][20381] Updated weights for policy 1, policy_version 1536218 (0.0009) [2023-12-27 16:18:01,824][20347] Updated weights for policy 0, policy_version 1537787 (0.0007) [2023-12-27 16:18:01,862][20381] Updated weights for policy 1, policy_version 1536228 (0.0005) [2023-12-27 16:18:01,880][20347] Updated weights for policy 0, policy_version 1537797 (0.0007) [2023-12-27 16:18:01,937][20347] Updated weights for policy 0, policy_version 1537807 (0.0008) [2023-12-27 16:18:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 787070976. Throughput: 0: 10354.7, 1: 10433.7. Samples: 787081936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:18:02,363][19170] Avg episode reward: [(0, '804.190'), (1, '971.570')] [2023-12-27 16:18:02,450][20381] Updated weights for policy 1, policy_version 1536238 (0.0008) [2023-12-27 16:18:02,504][20381] Updated weights for policy 1, policy_version 1536248 (0.0010) [2023-12-27 16:18:02,557][20381] Updated weights for policy 1, policy_version 1536258 (0.0010) [2023-12-27 16:18:02,659][20347] Updated weights for policy 0, policy_version 1537817 (0.0010) [2023-12-27 16:18:02,718][20347] Updated weights for policy 0, policy_version 1537827 (0.0009) [2023-12-27 16:18:02,770][20347] Updated weights for policy 0, policy_version 1537837 (0.0007) [2023-12-27 16:18:03,269][20381] Updated weights for policy 1, policy_version 1536268 (0.0009) [2023-12-27 16:18:03,328][20381] Updated weights for policy 1, policy_version 1536278 (0.0010) [2023-12-27 16:18:03,387][20381] Updated weights for policy 1, policy_version 1536288 (0.0010) [2023-12-27 16:18:03,467][20347] Updated weights for policy 0, policy_version 1537847 (0.0008) [2023-12-27 16:18:03,518][20347] Updated weights for policy 0, policy_version 1537857 (0.0010) [2023-12-27 16:18:03,571][20347] Updated weights for policy 0, policy_version 1537867 (0.0008) [2023-12-27 16:18:04,053][20381] Updated weights for policy 1, policy_version 1536298 (0.0010) [2023-12-27 16:18:04,108][20381] Updated weights for policy 1, policy_version 1536308 (0.0007) [2023-12-27 16:18:04,171][20381] Updated weights for policy 1, policy_version 1536318 (0.0008) [2023-12-27 16:18:04,229][20381] Updated weights for policy 1, policy_version 1536328 (0.0007) [2023-12-27 16:18:04,235][20347] Updated weights for policy 0, policy_version 1537877 (0.0008) [2023-12-27 16:18:04,300][20347] Updated weights for policy 0, policy_version 1537887 (0.0010) [2023-12-27 16:18:04,363][20347] Updated weights for policy 0, policy_version 1537897 (0.0009) [2023-12-27 16:18:04,907][20381] Updated weights for policy 1, policy_version 1536338 (0.0007) [2023-12-27 16:18:04,968][20381] Updated weights for policy 1, policy_version 1536348 (0.0007) [2023-12-27 16:18:05,017][20347] Updated weights for policy 0, policy_version 1537907 (0.0010) [2023-12-27 16:18:05,023][20381] Updated weights for policy 1, policy_version 1536358 (0.0008) [2023-12-27 16:18:05,086][20347] Updated weights for policy 0, policy_version 1537917 (0.0010) [2023-12-27 16:18:05,151][20347] Updated weights for policy 0, policy_version 1537927 (0.0011) [2023-12-27 16:18:05,691][20381] Updated weights for policy 1, policy_version 1536368 (0.0007) [2023-12-27 16:18:05,743][20381] Updated weights for policy 1, policy_version 1536378 (0.0008) [2023-12-27 16:18:05,762][20347] Updated weights for policy 0, policy_version 1537937 (0.0010) [2023-12-27 16:18:05,802][20381] Updated weights for policy 1, policy_version 1536388 (0.0007) [2023-12-27 16:18:05,822][20347] Updated weights for policy 0, policy_version 1537947 (0.0008) [2023-12-27 16:18:05,873][20347] Updated weights for policy 0, policy_version 1537957 (0.0005) [2023-12-27 16:18:05,934][20347] Updated weights for policy 0, policy_version 1537967 (0.0005) [2023-12-27 16:18:06,528][20347] Updated weights for policy 0, policy_version 1537977 (0.0008) [2023-12-27 16:18:06,543][20381] Updated weights for policy 1, policy_version 1536398 (0.0005) [2023-12-27 16:18:06,586][20347] Updated weights for policy 0, policy_version 1537987 (0.0006) [2023-12-27 16:18:06,600][20381] Updated weights for policy 1, policy_version 1536408 (0.0009) [2023-12-27 16:18:06,646][20347] Updated weights for policy 0, policy_version 1537997 (0.0006) [2023-12-27 16:18:06,661][20381] Updated weights for policy 1, policy_version 1536418 (0.0011) [2023-12-27 16:18:07,251][20347] Updated weights for policy 0, policy_version 1538007 (0.0008) [2023-12-27 16:18:07,298][20347] Updated weights for policy 0, policy_version 1538017 (0.0006) [2023-12-27 16:18:07,345][20381] Updated weights for policy 1, policy_version 1536428 (0.0009) [2023-12-27 16:18:07,347][20347] Updated weights for policy 0, policy_version 1538027 (0.0006) [2023-12-27 16:18:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 787169280. Throughput: 0: 10356.0, 1: 10447.3. Samples: 787143480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:18:07,363][19170] Avg episode reward: [(0, '794.090'), (1, '952.060')] [2023-12-27 16:18:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001538032_393814016.pth... [2023-12-27 16:18:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001536816_393502720.pth [2023-12-27 16:18:07,401][20381] Updated weights for policy 1, policy_version 1536438 (0.0008) [2023-12-27 16:18:07,457][20381] Updated weights for policy 1, policy_version 1536448 (0.0007) [2023-12-27 16:18:07,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001536456_393371648.pth... [2023-12-27 16:18:07,500][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001535240_393060352.pth [2023-12-27 16:18:07,941][20347] Updated weights for policy 0, policy_version 1538037 (0.0007) [2023-12-27 16:18:07,996][20347] Updated weights for policy 0, policy_version 1538047 (0.0007) [2023-12-27 16:18:08,052][20347] Updated weights for policy 0, policy_version 1538057 (0.0008) [2023-12-27 16:18:08,124][20381] Updated weights for policy 1, policy_version 1536458 (0.0007) [2023-12-27 16:18:08,176][20381] Updated weights for policy 1, policy_version 1536468 (0.0007) [2023-12-27 16:18:08,228][20381] Updated weights for policy 1, policy_version 1536479 (0.0007) [2023-12-27 16:18:08,726][20347] Updated weights for policy 0, policy_version 1538067 (0.0008) [2023-12-27 16:18:08,790][20347] Updated weights for policy 0, policy_version 1538077 (0.0009) [2023-12-27 16:18:08,846][20347] Updated weights for policy 0, policy_version 1538087 (0.0008) [2023-12-27 16:18:08,921][20381] Updated weights for policy 1, policy_version 1536489 (0.0005) [2023-12-27 16:18:08,977][20381] Updated weights for policy 1, policy_version 1536499 (0.0008) [2023-12-27 16:18:09,032][20381] Updated weights for policy 1, policy_version 1536509 (0.0007) [2023-12-27 16:18:09,089][20381] Updated weights for policy 1, policy_version 1536519 (0.0006) [2023-12-27 16:18:09,588][20347] Updated weights for policy 0, policy_version 1538097 (0.0008) [2023-12-27 16:18:09,640][20347] Updated weights for policy 0, policy_version 1538107 (0.0008) [2023-12-27 16:18:09,687][20347] Updated weights for policy 0, policy_version 1538117 (0.0008) [2023-12-27 16:18:09,752][20347] Updated weights for policy 0, policy_version 1538127 (0.0007) [2023-12-27 16:18:09,769][20381] Updated weights for policy 1, policy_version 1536529 (0.0007) [2023-12-27 16:18:09,830][20381] Updated weights for policy 1, policy_version 1536539 (0.0008) [2023-12-27 16:18:09,893][20381] Updated weights for policy 1, policy_version 1536549 (0.0009) [2023-12-27 16:18:10,482][20347] Updated weights for policy 0, policy_version 1538137 (0.0009) [2023-12-27 16:18:10,544][20347] Updated weights for policy 0, policy_version 1538147 (0.0009) [2023-12-27 16:18:10,609][20347] Updated weights for policy 0, policy_version 1538157 (0.0007) [2023-12-27 16:18:10,637][20381] Updated weights for policy 1, policy_version 1536559 (0.0007) [2023-12-27 16:18:10,690][20381] Updated weights for policy 1, policy_version 1536569 (0.0008) [2023-12-27 16:18:10,746][20381] Updated weights for policy 1, policy_version 1536579 (0.0007) [2023-12-27 16:18:11,342][20347] Updated weights for policy 0, policy_version 1538167 (0.0008) [2023-12-27 16:18:11,414][20347] Updated weights for policy 0, policy_version 1538177 (0.0007) [2023-12-27 16:18:11,475][20347] Updated weights for policy 0, policy_version 1538187 (0.0007) [2023-12-27 16:18:11,482][20381] Updated weights for policy 1, policy_version 1536589 (0.0006) [2023-12-27 16:18:11,540][20381] Updated weights for policy 1, policy_version 1536599 (0.0006) [2023-12-27 16:18:11,599][20381] Updated weights for policy 1, policy_version 1536609 (0.0007) [2023-12-27 16:18:12,158][20347] Updated weights for policy 0, policy_version 1538197 (0.0006) [2023-12-27 16:18:12,217][20347] Updated weights for policy 0, policy_version 1538207 (0.0007) [2023-12-27 16:18:12,251][20381] Updated weights for policy 1, policy_version 1536619 (0.0008) [2023-12-27 16:18:12,275][20347] Updated weights for policy 0, policy_version 1538217 (0.0007) [2023-12-27 16:18:12,312][20381] Updated weights for policy 1, policy_version 1536629 (0.0006) [2023-12-27 16:18:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 787275776. Throughput: 0: 10369.8, 1: 10409.0. Samples: 787266476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:18:12,363][19170] Avg episode reward: [(0, '804.090'), (1, '943.010')] [2023-12-27 16:18:12,372][20381] Updated weights for policy 1, policy_version 1536639 (0.0008) [2023-12-27 16:18:12,977][20381] Updated weights for policy 1, policy_version 1536649 (0.0007) [2023-12-27 16:18:13,014][20347] Updated weights for policy 0, policy_version 1538227 (0.0007) [2023-12-27 16:18:13,027][20381] Updated weights for policy 1, policy_version 1536659 (0.0006) [2023-12-27 16:18:13,049][20291] KL-divergence is very high: 169.1694 [2023-12-27 16:18:13,065][20347] Updated weights for policy 0, policy_version 1538237 (0.0006) [2023-12-27 16:18:13,079][20381] Updated weights for policy 1, policy_version 1536669 (0.0005) [2023-12-27 16:18:13,089][20291] KL-divergence is very high: 172.7557 [2023-12-27 16:18:13,117][20347] Updated weights for policy 0, policy_version 1538247 (0.0006) [2023-12-27 16:18:13,133][20381] Updated weights for policy 1, policy_version 1536679 (0.0005) [2023-12-27 16:18:13,667][20381] Updated weights for policy 1, policy_version 1536689 (0.0005) [2023-12-27 16:18:13,716][20381] Updated weights for policy 1, policy_version 1536699 (0.0007) [2023-12-27 16:18:13,762][20381] Updated weights for policy 1, policy_version 1536709 (0.0007) [2023-12-27 16:18:13,862][20347] Updated weights for policy 0, policy_version 1538257 (0.0007) [2023-12-27 16:18:13,915][20347] Updated weights for policy 0, policy_version 1538267 (0.0008) [2023-12-27 16:18:13,969][20347] Updated weights for policy 0, policy_version 1538277 (0.0010) [2023-12-27 16:18:13,996][20291] KL-divergence is very high: 271.6507 [2023-12-27 16:18:14,025][20347] Updated weights for policy 0, policy_version 1538287 (0.0008) [2023-12-27 16:18:14,331][20381] Updated weights for policy 1, policy_version 1536719 (0.0007) [2023-12-27 16:18:14,375][20381] Updated weights for policy 1, policy_version 1536729 (0.0007) [2023-12-27 16:18:14,428][20381] Updated weights for policy 1, policy_version 1536739 (0.0006) [2023-12-27 16:18:14,691][20347] Updated weights for policy 0, policy_version 1538297 (0.0007) [2023-12-27 16:18:14,751][20291] KL-divergence is very high: 111.6097 [2023-12-27 16:18:14,763][20291] KL-divergence is very high: 104.0016 [2023-12-27 16:18:14,772][20291] KL-divergence is very high: 203.9178 [2023-12-27 16:18:14,776][20347] Updated weights for policy 0, policy_version 1538307 (0.0007) [2023-12-27 16:18:14,822][20291] KL-divergence is very high: 134.8725 [2023-12-27 16:18:14,839][20347] Updated weights for policy 0, policy_version 1538317 (0.0007) [2023-12-27 16:18:15,133][20381] Updated weights for policy 1, policy_version 1536749 (0.0007) [2023-12-27 16:18:15,195][20381] Updated weights for policy 1, policy_version 1536759 (0.0007) [2023-12-27 16:18:15,260][20381] Updated weights for policy 1, policy_version 1536769 (0.0007) [2023-12-27 16:18:15,542][20347] Updated weights for policy 0, policy_version 1538327 (0.0008) [2023-12-27 16:18:15,609][20347] Updated weights for policy 0, policy_version 1538337 (0.0008) [2023-12-27 16:18:15,670][20347] Updated weights for policy 0, policy_version 1538347 (0.0007) [2023-12-27 16:18:15,892][20381] Updated weights for policy 1, policy_version 1536779 (0.0007) [2023-12-27 16:18:15,945][20381] Updated weights for policy 1, policy_version 1536789 (0.0005) [2023-12-27 16:18:15,994][20381] Updated weights for policy 1, policy_version 1536799 (0.0007) [2023-12-27 16:18:16,257][20347] Updated weights for policy 0, policy_version 1538357 (0.0007) [2023-12-27 16:18:16,314][20347] Updated weights for policy 0, policy_version 1538367 (0.0007) [2023-12-27 16:18:16,369][20347] Updated weights for policy 0, policy_version 1538377 (0.0007) [2023-12-27 16:18:16,668][20381] Updated weights for policy 1, policy_version 1536809 (0.0008) [2023-12-27 16:18:16,732][20381] Updated weights for policy 1, policy_version 1536819 (0.0007) [2023-12-27 16:18:16,788][20381] Updated weights for policy 1, policy_version 1536829 (0.0007) [2023-12-27 16:18:16,844][20381] Updated weights for policy 1, policy_version 1536839 (0.0008) [2023-12-27 16:18:17,070][20347] Updated weights for policy 0, policy_version 1538387 (0.0007) [2023-12-27 16:18:17,128][20347] Updated weights for policy 0, policy_version 1538397 (0.0008) [2023-12-27 16:18:17,180][20347] Updated weights for policy 0, policy_version 1538407 (0.0008) [2023-12-27 16:18:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 787382272. Throughput: 0: 10277.7, 1: 10449.2. Samples: 787391076. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:17,363][19170] Avg episode reward: [(0, '676.460'), (1, '971.470')] [2023-12-27 16:18:17,507][20381] Updated weights for policy 1, policy_version 1536849 (0.0007) [2023-12-27 16:18:17,554][20381] Updated weights for policy 1, policy_version 1536859 (0.0008) [2023-12-27 16:18:17,606][20381] Updated weights for policy 1, policy_version 1536869 (0.0007) [2023-12-27 16:18:17,849][20347] Updated weights for policy 0, policy_version 1538417 (0.0008) [2023-12-27 16:18:17,907][20347] Updated weights for policy 0, policy_version 1538427 (0.0007) [2023-12-27 16:18:17,959][20347] Updated weights for policy 0, policy_version 1538437 (0.0009) [2023-12-27 16:18:18,024][20347] Updated weights for policy 0, policy_version 1538447 (0.0008) [2023-12-27 16:18:18,310][20381] Updated weights for policy 1, policy_version 1536879 (0.0007) [2023-12-27 16:18:18,377][20381] Updated weights for policy 1, policy_version 1536889 (0.0008) [2023-12-27 16:18:18,434][20381] Updated weights for policy 1, policy_version 1536899 (0.0008) [2023-12-27 16:18:18,704][20347] Updated weights for policy 0, policy_version 1538457 (0.0008) [2023-12-27 16:18:18,760][20347] Updated weights for policy 0, policy_version 1538467 (0.0008) [2023-12-27 16:18:18,814][20347] Updated weights for policy 0, policy_version 1538477 (0.0007) [2023-12-27 16:18:19,071][20381] Updated weights for policy 1, policy_version 1536909 (0.0007) [2023-12-27 16:18:19,117][20381] Updated weights for policy 1, policy_version 1536919 (0.0005) [2023-12-27 16:18:19,172][20381] Updated weights for policy 1, policy_version 1536929 (0.0005) [2023-12-27 16:18:19,586][20347] Updated weights for policy 0, policy_version 1538487 (0.0008) [2023-12-27 16:18:19,636][20347] Updated weights for policy 0, policy_version 1538497 (0.0008) [2023-12-27 16:18:19,698][20347] Updated weights for policy 0, policy_version 1538507 (0.0008) [2023-12-27 16:18:19,837][20381] Updated weights for policy 1, policy_version 1536939 (0.0008) [2023-12-27 16:18:19,902][20381] Updated weights for policy 1, policy_version 1536949 (0.0007) [2023-12-27 16:18:19,970][20381] Updated weights for policy 1, policy_version 1536959 (0.0008) [2023-12-27 16:18:20,371][20347] Updated weights for policy 0, policy_version 1538517 (0.0007) [2023-12-27 16:18:20,422][20347] Updated weights for policy 0, policy_version 1538527 (0.0007) [2023-12-27 16:18:20,481][20347] Updated weights for policy 0, policy_version 1538537 (0.0008) [2023-12-27 16:18:20,692][20381] Updated weights for policy 1, policy_version 1536969 (0.0008) [2023-12-27 16:18:20,751][20381] Updated weights for policy 1, policy_version 1536979 (0.0008) [2023-12-27 16:18:20,815][20381] Updated weights for policy 1, policy_version 1536989 (0.0008) [2023-12-27 16:18:20,870][20381] Updated weights for policy 1, policy_version 1536999 (0.0008) [2023-12-27 16:18:21,124][20347] Updated weights for policy 0, policy_version 1538547 (0.0007) [2023-12-27 16:18:21,186][20347] Updated weights for policy 0, policy_version 1538557 (0.0006) [2023-12-27 16:18:21,245][20347] Updated weights for policy 0, policy_version 1538567 (0.0007) [2023-12-27 16:18:21,610][20381] Updated weights for policy 1, policy_version 1537009 (0.0007) [2023-12-27 16:18:21,671][20381] Updated weights for policy 1, policy_version 1537019 (0.0008) [2023-12-27 16:18:21,749][20381] Updated weights for policy 1, policy_version 1537029 (0.0007) [2023-12-27 16:18:21,980][20347] Updated weights for policy 0, policy_version 1538577 (0.0007) [2023-12-27 16:18:22,043][20347] Updated weights for policy 0, policy_version 1538587 (0.0006) [2023-12-27 16:18:22,110][20347] Updated weights for policy 0, policy_version 1538597 (0.0008) [2023-12-27 16:18:22,167][20347] Updated weights for policy 0, policy_version 1538607 (0.0008) [2023-12-27 16:18:22,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 787480576. Throughput: 0: 10206.6, 1: 10420.4. Samples: 787450844. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:22,364][19170] Avg episode reward: [(0, '678.550'), (1, '961.900')] [2023-12-27 16:18:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001537032_393519104.pth... [2023-12-27 16:18:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001538608_393961472.pth... [2023-12-27 16:18:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001537392_393650176.pth [2023-12-27 16:18:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001535848_393216000.pth [2023-12-27 16:18:22,465][20381] Updated weights for policy 1, policy_version 1537039 (0.0007) [2023-12-27 16:18:22,526][20381] Updated weights for policy 1, policy_version 1537049 (0.0008) [2023-12-27 16:18:22,591][20381] Updated weights for policy 1, policy_version 1537059 (0.0008) [2023-12-27 16:18:22,898][20347] Updated weights for policy 0, policy_version 1538617 (0.0008) [2023-12-27 16:18:22,959][20347] Updated weights for policy 0, policy_version 1538627 (0.0006) [2023-12-27 16:18:23,024][20347] Updated weights for policy 0, policy_version 1538637 (0.0007) [2023-12-27 16:18:23,255][20381] Updated weights for policy 1, policy_version 1537069 (0.0006) [2023-12-27 16:18:23,304][20381] Updated weights for policy 1, policy_version 1537079 (0.0005) [2023-12-27 16:18:23,354][20381] Updated weights for policy 1, policy_version 1537089 (0.0007) [2023-12-27 16:18:23,668][20347] Updated weights for policy 0, policy_version 1538647 (0.0006) [2023-12-27 16:18:23,732][20347] Updated weights for policy 0, policy_version 1538657 (0.0007) [2023-12-27 16:18:23,785][20347] Updated weights for policy 0, policy_version 1538667 (0.0007) [2023-12-27 16:18:24,015][20381] Updated weights for policy 1, policy_version 1537099 (0.0007) [2023-12-27 16:18:24,063][20381] Updated weights for policy 1, policy_version 1537109 (0.0007) [2023-12-27 16:18:24,111][20381] Updated weights for policy 1, policy_version 1537119 (0.0007) [2023-12-27 16:18:24,335][20347] Updated weights for policy 0, policy_version 1538677 (0.0007) [2023-12-27 16:18:24,382][20347] Updated weights for policy 0, policy_version 1538687 (0.0007) [2023-12-27 16:18:24,437][20347] Updated weights for policy 0, policy_version 1538697 (0.0007) [2023-12-27 16:18:24,799][20381] Updated weights for policy 1, policy_version 1537129 (0.0008) [2023-12-27 16:18:24,856][20381] Updated weights for policy 1, policy_version 1537139 (0.0010) [2023-12-27 16:18:24,915][20381] Updated weights for policy 1, policy_version 1537149 (0.0011) [2023-12-27 16:18:24,970][20381] Updated weights for policy 1, policy_version 1537159 (0.0011) [2023-12-27 16:18:25,096][20347] Updated weights for policy 0, policy_version 1538707 (0.0008) [2023-12-27 16:18:25,148][20347] Updated weights for policy 0, policy_version 1538717 (0.0005) [2023-12-27 16:18:25,215][20347] Updated weights for policy 0, policy_version 1538727 (0.0005) [2023-12-27 16:18:25,614][20381] Updated weights for policy 1, policy_version 1537169 (0.0008) [2023-12-27 16:18:25,664][20381] Updated weights for policy 1, policy_version 1537179 (0.0007) [2023-12-27 16:18:25,712][20381] Updated weights for policy 1, policy_version 1537189 (0.0007) [2023-12-27 16:18:25,823][20347] Updated weights for policy 0, policy_version 1538737 (0.0005) [2023-12-27 16:18:25,874][20347] Updated weights for policy 0, policy_version 1538747 (0.0006) [2023-12-27 16:18:25,928][20347] Updated weights for policy 0, policy_version 1538757 (0.0007) [2023-12-27 16:18:25,984][20347] Updated weights for policy 0, policy_version 1538767 (0.0006) [2023-12-27 16:18:26,435][20381] Updated weights for policy 1, policy_version 1537199 (0.0007) [2023-12-27 16:18:26,489][20381] Updated weights for policy 1, policy_version 1537209 (0.0007) [2023-12-27 16:18:26,542][20381] Updated weights for policy 1, policy_version 1537220 (0.0007) [2023-12-27 16:18:26,673][20347] Updated weights for policy 0, policy_version 1538777 (0.0005) [2023-12-27 16:18:26,691][20291] KL-divergence is very high: 199.0231 [2023-12-27 16:18:26,701][20291] KL-divergence is very high: 220.8638 [2023-12-27 16:18:26,706][20291] KL-divergence is very high: 208.6113 [2023-12-27 16:18:26,711][20291] KL-divergence is very high: 173.2623 [2023-12-27 16:18:26,724][20347] Updated weights for policy 0, policy_version 1538787 (0.0005) [2023-12-27 16:18:26,729][20291] KL-divergence is very high: 116.1948 [2023-12-27 16:18:26,772][20347] Updated weights for policy 0, policy_version 1538797 (0.0006) [2023-12-27 16:18:27,154][20381] Updated weights for policy 1, policy_version 1537230 (0.0008) [2023-12-27 16:18:27,209][20381] Updated weights for policy 1, policy_version 1537240 (0.0006) [2023-12-27 16:18:27,263][20381] Updated weights for policy 1, policy_version 1537250 (0.0005) [2023-12-27 16:18:27,286][20291] KL-divergence is very high: 158.2181 [2023-12-27 16:18:27,313][20347] Updated weights for policy 0, policy_version 1538807 (0.0007) [2023-12-27 16:18:27,331][20291] KL-divergence is very high: 239.2202 [2023-12-27 16:18:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 787587072. Throughput: 0: 10218.9, 1: 10403.5. Samples: 787574168. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:27,363][19170] Avg episode reward: [(0, '719.600'), (1, '942.400')] [2023-12-27 16:18:27,368][20347] Updated weights for policy 0, policy_version 1538817 (0.0009) [2023-12-27 16:18:27,371][20291] KL-divergence is very high: 202.7383 [2023-12-27 16:18:27,413][20291] KL-divergence is very high: 239.7237 [2023-12-27 16:18:27,417][20347] Updated weights for policy 0, policy_version 1538827 (0.0008) [2023-12-27 16:18:27,822][20381] Updated weights for policy 1, policy_version 1537260 (0.0007) [2023-12-27 16:18:27,873][20381] Updated weights for policy 1, policy_version 1537270 (0.0005) [2023-12-27 16:18:27,928][20381] Updated weights for policy 1, policy_version 1537280 (0.0005) [2023-12-27 16:18:27,953][20347] Updated weights for policy 0, policy_version 1538837 (0.0008) [2023-12-27 16:18:28,010][20347] Updated weights for policy 0, policy_version 1538847 (0.0010) [2023-12-27 16:18:28,057][20347] Updated weights for policy 0, policy_version 1538857 (0.0009) [2023-12-27 16:18:28,554][20381] Updated weights for policy 1, policy_version 1537290 (0.0006) [2023-12-27 16:18:28,611][20381] Updated weights for policy 1, policy_version 1537300 (0.0011) [2023-12-27 16:18:28,670][20381] Updated weights for policy 1, policy_version 1537310 (0.0008) [2023-12-27 16:18:28,706][20347] Updated weights for policy 0, policy_version 1538867 (0.0010) [2023-12-27 16:18:28,730][20381] Updated weights for policy 1, policy_version 1537320 (0.0005) [2023-12-27 16:18:28,761][20347] Updated weights for policy 0, policy_version 1538877 (0.0009) [2023-12-27 16:18:28,816][20347] Updated weights for policy 0, policy_version 1538887 (0.0010) [2023-12-27 16:18:29,437][20347] Updated weights for policy 0, policy_version 1538897 (0.0008) [2023-12-27 16:18:29,440][20381] Updated weights for policy 1, policy_version 1537330 (0.0005) [2023-12-27 16:18:29,486][20347] Updated weights for policy 0, policy_version 1538907 (0.0010) [2023-12-27 16:18:29,493][20381] Updated weights for policy 1, policy_version 1537340 (0.0005) [2023-12-27 16:18:29,531][20347] Updated weights for policy 0, policy_version 1538917 (0.0010) [2023-12-27 16:18:29,549][20381] Updated weights for policy 1, policy_version 1537350 (0.0005) [2023-12-27 16:18:29,580][20347] Updated weights for policy 0, policy_version 1538927 (0.0010) [2023-12-27 16:18:30,203][20381] Updated weights for policy 1, policy_version 1537360 (0.0007) [2023-12-27 16:18:30,210][20292] KL-divergence is very high: 206.1503 [2023-12-27 16:18:30,261][20292] KL-divergence is very high: 197.6139 [2023-12-27 16:18:30,266][20381] Updated weights for policy 1, policy_version 1537370 (0.0005) [2023-12-27 16:18:30,293][20347] Updated weights for policy 0, policy_version 1538937 (0.0006) [2023-12-27 16:18:30,328][20381] Updated weights for policy 1, policy_version 1537380 (0.0005) [2023-12-27 16:18:30,345][20347] Updated weights for policy 0, policy_version 1538947 (0.0006) [2023-12-27 16:18:30,397][20347] Updated weights for policy 0, policy_version 1538957 (0.0007) [2023-12-27 16:18:30,965][20381] Updated weights for policy 1, policy_version 1537390 (0.0009) [2023-12-27 16:18:30,996][20347] Updated weights for policy 0, policy_version 1538967 (0.0006) [2023-12-27 16:18:31,013][20381] Updated weights for policy 1, policy_version 1537400 (0.0010) [2023-12-27 16:18:31,048][20347] Updated weights for policy 0, policy_version 1538977 (0.0007) [2023-12-27 16:18:31,081][20381] Updated weights for policy 1, policy_version 1537410 (0.0011) [2023-12-27 16:18:31,105][20347] Updated weights for policy 0, policy_version 1538987 (0.0006) [2023-12-27 16:18:31,759][20347] Updated weights for policy 0, policy_version 1538997 (0.0008) [2023-12-27 16:18:31,799][20381] Updated weights for policy 1, policy_version 1537420 (0.0009) [2023-12-27 16:18:31,815][20347] Updated weights for policy 0, policy_version 1539007 (0.0008) [2023-12-27 16:18:31,862][20381] Updated weights for policy 1, policy_version 1537430 (0.0008) [2023-12-27 16:18:31,869][20347] Updated weights for policy 0, policy_version 1539017 (0.0008) [2023-12-27 16:18:31,915][20381] Updated weights for policy 1, policy_version 1537440 (0.0005) [2023-12-27 16:18:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 787693568. Throughput: 0: 10370.3, 1: 10411.6. Samples: 787704700. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:32,363][19170] Avg episode reward: [(0, '718.620'), (1, '951.280')] [2023-12-27 16:18:32,507][20381] Updated weights for policy 1, policy_version 1537450 (0.0005) [2023-12-27 16:18:32,559][20381] Updated weights for policy 1, policy_version 1537460 (0.0006) [2023-12-27 16:18:32,611][20381] Updated weights for policy 1, policy_version 1537470 (0.0007) [2023-12-27 16:18:32,640][20347] Updated weights for policy 0, policy_version 1539027 (0.0007) [2023-12-27 16:18:32,671][20381] Updated weights for policy 1, policy_version 1537480 (0.0007) [2023-12-27 16:18:32,700][20347] Updated weights for policy 0, policy_version 1539037 (0.0006) [2023-12-27 16:18:32,753][20347] Updated weights for policy 0, policy_version 1539047 (0.0007) [2023-12-27 16:18:33,354][20381] Updated weights for policy 1, policy_version 1537490 (0.0010) [2023-12-27 16:18:33,388][20347] Updated weights for policy 0, policy_version 1539057 (0.0007) [2023-12-27 16:18:33,408][20381] Updated weights for policy 1, policy_version 1537500 (0.0010) [2023-12-27 16:18:33,440][20347] Updated weights for policy 0, policy_version 1539067 (0.0005) [2023-12-27 16:18:33,464][20381] Updated weights for policy 1, policy_version 1537510 (0.0010) [2023-12-27 16:18:33,500][20347] Updated weights for policy 0, policy_version 1539077 (0.0006) [2023-12-27 16:18:33,568][20347] Updated weights for policy 0, policy_version 1539087 (0.0008) [2023-12-27 16:18:34,159][20381] Updated weights for policy 1, policy_version 1537520 (0.0010) [2023-12-27 16:18:34,224][20381] Updated weights for policy 1, policy_version 1537530 (0.0007) [2023-12-27 16:18:34,263][20347] Updated weights for policy 0, policy_version 1539097 (0.0006) [2023-12-27 16:18:34,289][20381] Updated weights for policy 1, policy_version 1537540 (0.0007) [2023-12-27 16:18:34,326][20347] Updated weights for policy 0, policy_version 1539107 (0.0007) [2023-12-27 16:18:34,388][20347] Updated weights for policy 0, policy_version 1539117 (0.0007) [2023-12-27 16:18:34,985][20381] Updated weights for policy 1, policy_version 1537550 (0.0009) [2023-12-27 16:18:35,047][20381] Updated weights for policy 1, policy_version 1537560 (0.0010) [2023-12-27 16:18:35,088][20347] Updated weights for policy 0, policy_version 1539127 (0.0007) [2023-12-27 16:18:35,108][20381] Updated weights for policy 1, policy_version 1537570 (0.0010) [2023-12-27 16:18:35,132][20347] Updated weights for policy 0, policy_version 1539137 (0.0005) [2023-12-27 16:18:35,176][20347] Updated weights for policy 0, policy_version 1539147 (0.0007) [2023-12-27 16:18:35,718][20381] Updated weights for policy 1, policy_version 1537580 (0.0011) [2023-12-27 16:18:35,779][20381] Updated weights for policy 1, policy_version 1537590 (0.0010) [2023-12-27 16:18:35,803][20347] Updated weights for policy 0, policy_version 1539157 (0.0009) [2023-12-27 16:18:35,823][20291] KL-divergence is very high: 129.8613 [2023-12-27 16:18:35,828][20291] KL-divergence is very high: 152.7918 [2023-12-27 16:18:35,830][20381] Updated weights for policy 1, policy_version 1537600 (0.0009) [2023-12-27 16:18:35,839][20291] KL-divergence is very high: 202.1673 [2023-12-27 16:18:35,852][20347] Updated weights for policy 0, policy_version 1539167 (0.0010) [2023-12-27 16:18:35,857][20291] KL-divergence is very high: 244.6033 [2023-12-27 16:18:35,862][20291] KL-divergence is very high: 256.2621 [2023-12-27 16:18:35,867][20291] KL-divergence is very high: 233.1049 [2023-12-27 16:18:35,877][20291] KL-divergence is very high: 207.8806 [2023-12-27 16:18:35,896][20291] KL-divergence is very high: 146.0181 [2023-12-27 16:18:35,900][20347] Updated weights for policy 0, policy_version 1539177 (0.0010) [2023-12-27 16:18:35,901][20291] KL-divergence is very high: 139.4757 [2023-12-27 16:18:35,906][20291] KL-divergence is very high: 116.9111 [2023-12-27 16:18:36,493][20381] Updated weights for policy 1, policy_version 1537610 (0.0011) [2023-12-27 16:18:36,553][20347] Updated weights for policy 0, policy_version 1539187 (0.0010) [2023-12-27 16:18:36,556][20381] Updated weights for policy 1, policy_version 1537620 (0.0011) [2023-12-27 16:18:36,592][20291] KL-divergence is very high: 138.4044 [2023-12-27 16:18:36,614][20381] Updated weights for policy 1, policy_version 1537630 (0.0012) [2023-12-27 16:18:36,616][20347] Updated weights for policy 0, policy_version 1539197 (0.0007) [2023-12-27 16:18:36,639][20291] KL-divergence is very high: 137.3915 [2023-12-27 16:18:36,675][20381] Updated weights for policy 1, policy_version 1537640 (0.0011) [2023-12-27 16:18:36,676][20347] Updated weights for policy 0, policy_version 1539207 (0.0008) [2023-12-27 16:18:36,689][20291] KL-divergence is very high: 108.3558 [2023-12-27 16:18:37,310][20347] Updated weights for policy 0, policy_version 1539217 (0.0007) [2023-12-27 16:18:37,352][20381] Updated weights for policy 1, policy_version 1537650 (0.0011) [2023-12-27 16:18:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 787791872. Throughput: 0: 10386.1, 1: 10404.6. Samples: 787765296. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:37,363][19170] Avg episode reward: [(0, '736.860'), (1, '961.040')] [2023-12-27 16:18:37,365][20347] Updated weights for policy 0, policy_version 1539227 (0.0006) [2023-12-27 16:18:37,405][20381] Updated weights for policy 1, policy_version 1537660 (0.0010) [2023-12-27 16:18:37,422][20347] Updated weights for policy 0, policy_version 1539237 (0.0005) [2023-12-27 16:18:37,460][20381] Updated weights for policy 1, policy_version 1537670 (0.0011) [2023-12-27 16:18:37,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001537672_393682944.pth... [2023-12-27 16:18:37,475][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001536456_393371648.pth [2023-12-27 16:18:37,480][20347] Updated weights for policy 0, policy_version 1539247 (0.0006) [2023-12-27 16:18:37,485][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001539248_394125312.pth... [2023-12-27 16:18:37,488][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001538032_393814016.pth [2023-12-27 16:18:38,030][20347] Updated weights for policy 0, policy_version 1539257 (0.0009) [2023-12-27 16:18:38,088][20347] Updated weights for policy 0, policy_version 1539267 (0.0010) [2023-12-27 16:18:38,144][20347] Updated weights for policy 0, policy_version 1539277 (0.0010) [2023-12-27 16:18:38,147][20381] Updated weights for policy 1, policy_version 1537680 (0.0011) [2023-12-27 16:18:38,195][20381] Updated weights for policy 1, policy_version 1537690 (0.0010) [2023-12-27 16:18:38,251][20381] Updated weights for policy 1, policy_version 1537700 (0.0011) [2023-12-27 16:18:38,825][20347] Updated weights for policy 0, policy_version 1539287 (0.0010) [2023-12-27 16:18:38,886][20347] Updated weights for policy 0, policy_version 1539297 (0.0010) [2023-12-27 16:18:38,945][20347] Updated weights for policy 0, policy_version 1539307 (0.0010) [2023-12-27 16:18:38,955][20381] Updated weights for policy 1, policy_version 1537710 (0.0011) [2023-12-27 16:18:39,003][20381] Updated weights for policy 1, policy_version 1537720 (0.0010) [2023-12-27 16:18:39,051][20381] Updated weights for policy 1, policy_version 1537730 (0.0010) [2023-12-27 16:18:39,649][20347] Updated weights for policy 0, policy_version 1539317 (0.0009) [2023-12-27 16:18:39,711][20347] Updated weights for policy 0, policy_version 1539327 (0.0009) [2023-12-27 16:18:39,753][20381] Updated weights for policy 1, policy_version 1537740 (0.0009) [2023-12-27 16:18:39,770][20347] Updated weights for policy 0, policy_version 1539337 (0.0010) [2023-12-27 16:18:39,804][20381] Updated weights for policy 1, policy_version 1537750 (0.0006) [2023-12-27 16:18:39,869][20381] Updated weights for policy 1, policy_version 1537760 (0.0008) [2023-12-27 16:18:40,477][20347] Updated weights for policy 0, policy_version 1539347 (0.0010) [2023-12-27 16:18:40,530][20347] Updated weights for policy 0, policy_version 1539357 (0.0008) [2023-12-27 16:18:40,569][20381] Updated weights for policy 1, policy_version 1537770 (0.0008) [2023-12-27 16:18:40,589][20347] Updated weights for policy 0, policy_version 1539367 (0.0006) [2023-12-27 16:18:40,628][20381] Updated weights for policy 1, policy_version 1537780 (0.0007) [2023-12-27 16:18:40,684][20381] Updated weights for policy 1, policy_version 1537790 (0.0008) [2023-12-27 16:18:40,736][20381] Updated weights for policy 1, policy_version 1537800 (0.0008) [2023-12-27 16:18:41,353][20347] Updated weights for policy 0, policy_version 1539377 (0.0006) [2023-12-27 16:18:41,417][20347] Updated weights for policy 0, policy_version 1539387 (0.0007) [2023-12-27 16:18:41,452][20381] Updated weights for policy 1, policy_version 1537810 (0.0009) [2023-12-27 16:18:41,481][20347] Updated weights for policy 0, policy_version 1539397 (0.0007) [2023-12-27 16:18:41,512][20381] Updated weights for policy 1, policy_version 1537820 (0.0009) [2023-12-27 16:18:41,546][20347] Updated weights for policy 0, policy_version 1539407 (0.0008) [2023-12-27 16:18:41,569][20381] Updated weights for policy 1, policy_version 1537830 (0.0008) [2023-12-27 16:18:42,275][20347] Updated weights for policy 0, policy_version 1539417 (0.0006) [2023-12-27 16:18:42,280][20381] Updated weights for policy 1, policy_version 1537840 (0.0006) [2023-12-27 16:18:42,337][20347] Updated weights for policy 0, policy_version 1539427 (0.0007) [2023-12-27 16:18:42,347][20381] Updated weights for policy 1, policy_version 1537850 (0.0007) [2023-12-27 16:18:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 787890176. Throughput: 0: 10360.8, 1: 10335.6. Samples: 787888740. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:42,363][19170] Avg episode reward: [(0, '775.190'), (1, '980.590')] [2023-12-27 16:18:42,400][20347] Updated weights for policy 0, policy_version 1539437 (0.0007) [2023-12-27 16:18:42,409][20381] Updated weights for policy 1, policy_version 1537860 (0.0006) [2023-12-27 16:18:43,050][20381] Updated weights for policy 1, policy_version 1537870 (0.0006) [2023-12-27 16:18:43,086][20347] Updated weights for policy 0, policy_version 1539447 (0.0007) [2023-12-27 16:18:43,101][20381] Updated weights for policy 1, policy_version 1537880 (0.0005) [2023-12-27 16:18:43,140][20347] Updated weights for policy 0, policy_version 1539457 (0.0009) [2023-12-27 16:18:43,162][20381] Updated weights for policy 1, policy_version 1537890 (0.0005) [2023-12-27 16:18:43,192][20347] Updated weights for policy 0, policy_version 1539467 (0.0010) [2023-12-27 16:18:43,778][20381] Updated weights for policy 1, policy_version 1537900 (0.0008) [2023-12-27 16:18:43,833][20381] Updated weights for policy 1, policy_version 1537910 (0.0010) [2023-12-27 16:18:43,839][20347] Updated weights for policy 0, policy_version 1539477 (0.0008) [2023-12-27 16:18:43,885][20347] Updated weights for policy 0, policy_version 1539487 (0.0005) [2023-12-27 16:18:43,891][20381] Updated weights for policy 1, policy_version 1537920 (0.0010) [2023-12-27 16:18:43,933][20347] Updated weights for policy 0, policy_version 1539497 (0.0005) [2023-12-27 16:18:44,557][20347] Updated weights for policy 0, policy_version 1539507 (0.0006) [2023-12-27 16:18:44,603][20381] Updated weights for policy 1, policy_version 1537930 (0.0010) [2023-12-27 16:18:44,614][20347] Updated weights for policy 0, policy_version 1539517 (0.0007) [2023-12-27 16:18:44,659][20381] Updated weights for policy 1, policy_version 1537940 (0.0008) [2023-12-27 16:18:44,671][20347] Updated weights for policy 0, policy_version 1539527 (0.0007) [2023-12-27 16:18:44,714][20381] Updated weights for policy 1, policy_version 1537950 (0.0008) [2023-12-27 16:18:44,771][20381] Updated weights for policy 1, policy_version 1537960 (0.0008) [2023-12-27 16:18:45,350][20347] Updated weights for policy 0, policy_version 1539537 (0.0006) [2023-12-27 16:18:45,405][20347] Updated weights for policy 0, policy_version 1539547 (0.0010) [2023-12-27 16:18:45,463][20347] Updated weights for policy 0, policy_version 1539557 (0.0010) [2023-12-27 16:18:45,490][20381] Updated weights for policy 1, policy_version 1537970 (0.0005) [2023-12-27 16:18:45,518][20347] Updated weights for policy 0, policy_version 1539567 (0.0006) [2023-12-27 16:18:45,533][20381] Updated weights for policy 1, policy_version 1537980 (0.0006) [2023-12-27 16:18:45,579][20381] Updated weights for policy 1, policy_version 1537990 (0.0008) [2023-12-27 16:18:46,174][20347] Updated weights for policy 0, policy_version 1539577 (0.0010) [2023-12-27 16:18:46,216][20381] Updated weights for policy 1, policy_version 1538000 (0.0006) [2023-12-27 16:18:46,230][20347] Updated weights for policy 0, policy_version 1539587 (0.0010) [2023-12-27 16:18:46,263][20381] Updated weights for policy 1, policy_version 1538010 (0.0005) [2023-12-27 16:18:46,278][20347] Updated weights for policy 0, policy_version 1539597 (0.0010) [2023-12-27 16:18:46,310][20381] Updated weights for policy 1, policy_version 1538020 (0.0005) [2023-12-27 16:18:46,864][20381] Updated weights for policy 1, policy_version 1538030 (0.0008) [2023-12-27 16:18:46,920][20381] Updated weights for policy 1, policy_version 1538040 (0.0010) [2023-12-27 16:18:46,956][20291] KL-divergence is very high: 170.0659 [2023-12-27 16:18:46,966][20347] Updated weights for policy 0, policy_version 1539607 (0.0009) [2023-12-27 16:18:46,972][20291] KL-divergence is very high: 315.4336 [2023-12-27 16:18:46,976][20291] KL-divergence is very high: 338.0995 [2023-12-27 16:18:46,979][20381] Updated weights for policy 1, policy_version 1538050 (0.0005) [2023-12-27 16:18:46,981][20291] KL-divergence is very high: 286.2904 [2023-12-27 16:18:46,987][20291] KL-divergence is very high: 310.4072 [2023-12-27 16:18:46,993][20291] KL-divergence is very high: 252.7009 [2023-12-27 16:18:46,998][20291] KL-divergence is very high: 184.1418 [2023-12-27 16:18:47,018][20347] Updated weights for policy 0, policy_version 1539617 (0.0007) [2023-12-27 16:18:47,045][20291] KL-divergence is very high: 120.5844 [2023-12-27 16:18:47,069][20347] Updated weights for policy 0, policy_version 1539627 (0.0009) [2023-12-27 16:18:47,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 788004864. Throughput: 0: 10360.3, 1: 10338.8. Samples: 788013400. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:47,364][19170] Avg episode reward: [(0, '752.500'), (1, '970.870')] [2023-12-27 16:18:47,604][20381] Updated weights for policy 1, policy_version 1538060 (0.0009) [2023-12-27 16:18:47,662][20381] Updated weights for policy 1, policy_version 1538070 (0.0010) [2023-12-27 16:18:47,719][20381] Updated weights for policy 1, policy_version 1538080 (0.0009) [2023-12-27 16:18:47,728][20347] Updated weights for policy 0, policy_version 1539637 (0.0007) [2023-12-27 16:18:47,730][20291] KL-divergence is very high: 189.4691 [2023-12-27 16:18:47,774][20291] KL-divergence is very high: 231.8917 [2023-12-27 16:18:47,785][20347] Updated weights for policy 0, policy_version 1539647 (0.0006) [2023-12-27 16:18:47,818][20291] KL-divergence is very high: 183.5927 [2023-12-27 16:18:47,841][20347] Updated weights for policy 0, policy_version 1539657 (0.0006) [2023-12-27 16:18:47,858][20291] KL-divergence is very high: 156.1310 [2023-12-27 16:18:48,406][20381] Updated weights for policy 1, policy_version 1538090 (0.0006) [2023-12-27 16:18:48,464][20381] Updated weights for policy 1, policy_version 1538100 (0.0008) [2023-12-27 16:18:48,529][20381] Updated weights for policy 1, policy_version 1538110 (0.0008) [2023-12-27 16:18:48,551][20347] Updated weights for policy 0, policy_version 1539667 (0.0005) [2023-12-27 16:18:48,576][20291] KL-divergence is very high: 115.4569 [2023-12-27 16:18:48,580][20381] Updated weights for policy 1, policy_version 1538120 (0.0008) [2023-12-27 16:18:48,603][20347] Updated weights for policy 0, policy_version 1539677 (0.0007) [2023-12-27 16:18:48,657][20347] Updated weights for policy 0, policy_version 1539687 (0.0007) [2023-12-27 16:18:49,300][20381] Updated weights for policy 1, policy_version 1538130 (0.0009) [2023-12-27 16:18:49,363][20381] Updated weights for policy 1, policy_version 1538140 (0.0007) [2023-12-27 16:18:49,405][20347] Updated weights for policy 0, policy_version 1539697 (0.0007) [2023-12-27 16:18:49,426][20381] Updated weights for policy 1, policy_version 1538150 (0.0008) [2023-12-27 16:18:49,468][20347] Updated weights for policy 0, policy_version 1539707 (0.0008) [2023-12-27 16:18:49,524][20347] Updated weights for policy 0, policy_version 1539717 (0.0006) [2023-12-27 16:18:49,583][20347] Updated weights for policy 0, policy_version 1539727 (0.0007) [2023-12-27 16:18:50,117][20381] Updated weights for policy 1, policy_version 1538160 (0.0008) [2023-12-27 16:18:50,179][20381] Updated weights for policy 1, policy_version 1538170 (0.0008) [2023-12-27 16:18:50,204][20291] KL-divergence is very high: 158.4544 [2023-12-27 16:18:50,234][20381] Updated weights for policy 1, policy_version 1538180 (0.0007) [2023-12-27 16:18:50,244][20347] Updated weights for policy 0, policy_version 1539737 (0.0006) [2023-12-27 16:18:50,251][20291] KL-divergence is very high: 193.5975 [2023-12-27 16:18:50,296][20291] KL-divergence is very high: 216.9751 [2023-12-27 16:18:50,304][20347] Updated weights for policy 0, policy_version 1539747 (0.0009) [2023-12-27 16:18:50,339][20291] KL-divergence is very high: 172.9131 [2023-12-27 16:18:50,356][20347] Updated weights for policy 0, policy_version 1539757 (0.0009) [2023-12-27 16:18:50,847][20381] Updated weights for policy 1, policy_version 1538190 (0.0006) [2023-12-27 16:18:50,906][20381] Updated weights for policy 1, policy_version 1538200 (0.0006) [2023-12-27 16:18:50,964][20381] Updated weights for policy 1, policy_version 1538210 (0.0007) [2023-12-27 16:18:51,107][20347] Updated weights for policy 0, policy_version 1539767 (0.0008) [2023-12-27 16:18:51,172][20347] Updated weights for policy 0, policy_version 1539777 (0.0007) [2023-12-27 16:18:51,219][20347] Updated weights for policy 0, policy_version 1539787 (0.0009) [2023-12-27 16:18:51,675][20381] Updated weights for policy 1, policy_version 1538220 (0.0008) [2023-12-27 16:18:51,736][20381] Updated weights for policy 1, policy_version 1538230 (0.0008) [2023-12-27 16:18:51,791][20381] Updated weights for policy 1, policy_version 1538240 (0.0007) [2023-12-27 16:18:51,875][20347] Updated weights for policy 0, policy_version 1539797 (0.0008) [2023-12-27 16:18:51,933][20347] Updated weights for policy 0, policy_version 1539807 (0.0009) [2023-12-27 16:18:51,990][20347] Updated weights for policy 0, policy_version 1539817 (0.0011) [2023-12-27 16:18:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 788103168. Throughput: 0: 10353.2, 1: 10323.3. Samples: 788073924. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:52,363][19170] Avg episode reward: [(0, '636.440'), (1, '990.280')] [2023-12-27 16:18:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001538248_393830400.pth... [2023-12-27 16:18:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001539824_394272768.pth... [2023-12-27 16:18:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001537032_393519104.pth [2023-12-27 16:18:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001538608_393961472.pth [2023-12-27 16:18:52,516][20381] Updated weights for policy 1, policy_version 1538250 (0.0006) [2023-12-27 16:18:52,570][20381] Updated weights for policy 1, policy_version 1538260 (0.0008) [2023-12-27 16:18:52,629][20381] Updated weights for policy 1, policy_version 1538270 (0.0006) [2023-12-27 16:18:52,666][20347] Updated weights for policy 0, policy_version 1539827 (0.0007) [2023-12-27 16:18:52,686][20381] Updated weights for policy 1, policy_version 1538280 (0.0007) [2023-12-27 16:18:52,729][20347] Updated weights for policy 0, policy_version 1539837 (0.0007) [2023-12-27 16:18:52,799][20347] Updated weights for policy 0, policy_version 1539847 (0.0008) [2023-12-27 16:18:53,388][20381] Updated weights for policy 1, policy_version 1538290 (0.0007) [2023-12-27 16:18:53,439][20381] Updated weights for policy 1, policy_version 1538300 (0.0007) [2023-12-27 16:18:53,462][20347] Updated weights for policy 0, policy_version 1539857 (0.0008) [2023-12-27 16:18:53,495][20381] Updated weights for policy 1, policy_version 1538310 (0.0008) [2023-12-27 16:18:53,520][20347] Updated weights for policy 0, policy_version 1539867 (0.0008) [2023-12-27 16:18:53,573][20347] Updated weights for policy 0, policy_version 1539877 (0.0007) [2023-12-27 16:18:53,633][20347] Updated weights for policy 0, policy_version 1539887 (0.0007) [2023-12-27 16:18:54,163][20381] Updated weights for policy 1, policy_version 1538320 (0.0007) [2023-12-27 16:18:54,218][20381] Updated weights for policy 1, policy_version 1538330 (0.0008) [2023-12-27 16:18:54,275][20381] Updated weights for policy 1, policy_version 1538340 (0.0007) [2023-12-27 16:18:54,334][20347] Updated weights for policy 0, policy_version 1539897 (0.0005) [2023-12-27 16:18:54,395][20347] Updated weights for policy 0, policy_version 1539907 (0.0006) [2023-12-27 16:18:54,456][20347] Updated weights for policy 0, policy_version 1539917 (0.0006) [2023-12-27 16:18:54,960][20381] Updated weights for policy 1, policy_version 1538350 (0.0007) [2023-12-27 16:18:55,012][20347] Updated weights for policy 0, policy_version 1539927 (0.0008) [2023-12-27 16:18:55,019][20381] Updated weights for policy 1, policy_version 1538360 (0.0006) [2023-12-27 16:18:55,063][20347] Updated weights for policy 0, policy_version 1539937 (0.0005) [2023-12-27 16:18:55,074][20381] Updated weights for policy 1, policy_version 1538370 (0.0007) [2023-12-27 16:18:55,116][20347] Updated weights for policy 0, policy_version 1539947 (0.0005) [2023-12-27 16:18:55,668][20347] Updated weights for policy 0, policy_version 1539957 (0.0005) [2023-12-27 16:18:55,729][20347] Updated weights for policy 0, policy_version 1539967 (0.0005) [2023-12-27 16:18:55,783][20347] Updated weights for policy 0, policy_version 1539977 (0.0005) [2023-12-27 16:18:55,791][20381] Updated weights for policy 1, policy_version 1538380 (0.0007) [2023-12-27 16:18:55,846][20381] Updated weights for policy 1, policy_version 1538390 (0.0005) [2023-12-27 16:18:55,901][20381] Updated weights for policy 1, policy_version 1538400 (0.0007) [2023-12-27 16:18:56,408][20347] Updated weights for policy 0, policy_version 1539987 (0.0007) [2023-12-27 16:18:56,461][20347] Updated weights for policy 0, policy_version 1539997 (0.0007) [2023-12-27 16:18:56,512][20347] Updated weights for policy 0, policy_version 1540007 (0.0007) [2023-12-27 16:18:56,571][20381] Updated weights for policy 1, policy_version 1538410 (0.0007) [2023-12-27 16:18:56,625][20381] Updated weights for policy 1, policy_version 1538420 (0.0005) [2023-12-27 16:18:56,697][20381] Updated weights for policy 1, policy_version 1538430 (0.0008) [2023-12-27 16:18:56,758][20381] Updated weights for policy 1, policy_version 1538440 (0.0007) [2023-12-27 16:18:57,192][20347] Updated weights for policy 0, policy_version 1540017 (0.0008) [2023-12-27 16:18:57,237][20347] Updated weights for policy 0, policy_version 1540027 (0.0010) [2023-12-27 16:18:57,278][20347] Updated weights for policy 0, policy_version 1540037 (0.0010) [2023-12-27 16:18:57,330][20347] Updated weights for policy 0, policy_version 1540047 (0.0010) [2023-12-27 16:18:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 788209664. Throughput: 0: 10346.2, 1: 10339.8. Samples: 788197340. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:18:57,363][19170] Avg episode reward: [(0, '715.480'), (1, '990.290')] [2023-12-27 16:18:57,407][20381] Updated weights for policy 1, policy_version 1538450 (0.0007) [2023-12-27 16:18:57,466][20381] Updated weights for policy 1, policy_version 1538460 (0.0007) [2023-12-27 16:18:57,525][20381] Updated weights for policy 1, policy_version 1538470 (0.0007) [2023-12-27 16:18:58,037][20347] Updated weights for policy 0, policy_version 1540057 (0.0008) [2023-12-27 16:18:58,090][20347] Updated weights for policy 0, policy_version 1540067 (0.0008) [2023-12-27 16:18:58,138][20347] Updated weights for policy 0, policy_version 1540077 (0.0007) [2023-12-27 16:18:58,208][20381] Updated weights for policy 1, policy_version 1538480 (0.0008) [2023-12-27 16:18:58,256][20381] Updated weights for policy 1, policy_version 1538490 (0.0007) [2023-12-27 16:18:58,302][20381] Updated weights for policy 1, policy_version 1538500 (0.0008) [2023-12-27 16:18:58,989][20347] Updated weights for policy 0, policy_version 1540087 (0.0008) [2023-12-27 16:18:59,032][20381] Updated weights for policy 1, policy_version 1538510 (0.0008) [2023-12-27 16:18:59,050][20347] Updated weights for policy 0, policy_version 1540097 (0.0006) [2023-12-27 16:18:59,095][20381] Updated weights for policy 1, policy_version 1538520 (0.0007) [2023-12-27 16:18:59,109][20347] Updated weights for policy 0, policy_version 1540107 (0.0007) [2023-12-27 16:18:59,156][20381] Updated weights for policy 1, policy_version 1538530 (0.0008) [2023-12-27 16:18:59,804][20347] Updated weights for policy 0, policy_version 1540117 (0.0006) [2023-12-27 16:18:59,840][20291] KL-divergence is very high: 112.0581 [2023-12-27 16:18:59,851][20291] KL-divergence is very high: 153.0623 [2023-12-27 16:18:59,858][20291] KL-divergence is very high: 160.8345 [2023-12-27 16:18:59,862][20347] Updated weights for policy 0, policy_version 1540127 (0.0008) [2023-12-27 16:18:59,864][20291] KL-divergence is very high: 156.1642 [2023-12-27 16:18:59,869][20291] KL-divergence is very high: 115.7713 [2023-12-27 16:18:59,884][20291] KL-divergence is very high: 100.3373 [2023-12-27 16:18:59,899][20381] Updated weights for policy 1, policy_version 1538540 (0.0007) [2023-12-27 16:18:59,917][20347] Updated weights for policy 0, policy_version 1540137 (0.0008) [2023-12-27 16:18:59,961][20381] Updated weights for policy 1, policy_version 1538550 (0.0008) [2023-12-27 16:19:00,018][20381] Updated weights for policy 1, policy_version 1538560 (0.0007) [2023-12-27 16:19:00,628][20347] Updated weights for policy 0, policy_version 1540147 (0.0006) [2023-12-27 16:19:00,654][20291] KL-divergence is very high: 144.4156 [2023-12-27 16:19:00,676][20347] Updated weights for policy 0, policy_version 1540157 (0.0007) [2023-12-27 16:19:00,690][20291] KL-divergence is very high: 101.7798 [2023-12-27 16:19:00,719][20381] Updated weights for policy 1, policy_version 1538570 (0.0007) [2023-12-27 16:19:00,731][20347] Updated weights for policy 0, policy_version 1540167 (0.0010) [2023-12-27 16:19:00,768][20381] Updated weights for policy 1, policy_version 1538580 (0.0009) [2023-12-27 16:19:00,821][20381] Updated weights for policy 1, policy_version 1538590 (0.0007) [2023-12-27 16:19:00,870][20381] Updated weights for policy 1, policy_version 1538600 (0.0009) [2023-12-27 16:19:01,409][20347] Updated weights for policy 0, policy_version 1540177 (0.0010) [2023-12-27 16:19:01,461][20347] Updated weights for policy 0, policy_version 1540187 (0.0010) [2023-12-27 16:19:01,516][20347] Updated weights for policy 0, policy_version 1540197 (0.0010) [2023-12-27 16:19:01,568][20347] Updated weights for policy 0, policy_version 1540207 (0.0010) [2023-12-27 16:19:01,614][20381] Updated weights for policy 1, policy_version 1538610 (0.0007) [2023-12-27 16:19:01,679][20381] Updated weights for policy 1, policy_version 1538620 (0.0008) [2023-12-27 16:19:01,737][20381] Updated weights for policy 1, policy_version 1538630 (0.0006) [2023-12-27 16:19:02,282][20347] Updated weights for policy 0, policy_version 1540217 (0.0011) [2023-12-27 16:19:02,341][20347] Updated weights for policy 0, policy_version 1540227 (0.0010) [2023-12-27 16:19:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 788299776. Throughput: 0: 10348.7, 1: 10213.3. Samples: 788316364. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:02,363][19170] Avg episode reward: [(0, '649.090'), (1, '990.290')] [2023-12-27 16:19:02,402][20347] Updated weights for policy 0, policy_version 1540237 (0.0008) [2023-12-27 16:19:02,412][20381] Updated weights for policy 1, policy_version 1538640 (0.0007) [2023-12-27 16:19:02,459][20381] Updated weights for policy 1, policy_version 1538650 (0.0007) [2023-12-27 16:19:02,506][20381] Updated weights for policy 1, policy_version 1538660 (0.0007) [2023-12-27 16:19:03,149][20347] Updated weights for policy 0, policy_version 1540247 (0.0007) [2023-12-27 16:19:03,197][20347] Updated weights for policy 0, policy_version 1540257 (0.0008) [2023-12-27 16:19:03,230][20381] Updated weights for policy 1, policy_version 1538670 (0.0006) [2023-12-27 16:19:03,251][20347] Updated weights for policy 0, policy_version 1540267 (0.0007) [2023-12-27 16:19:03,287][20381] Updated weights for policy 1, policy_version 1538680 (0.0007) [2023-12-27 16:19:03,338][20381] Updated weights for policy 1, policy_version 1538690 (0.0008) [2023-12-27 16:19:03,983][20347] Updated weights for policy 0, policy_version 1540277 (0.0008) [2023-12-27 16:19:04,005][20381] Updated weights for policy 1, policy_version 1538700 (0.0008) [2023-12-27 16:19:04,038][20347] Updated weights for policy 0, policy_version 1540287 (0.0007) [2023-12-27 16:19:04,055][20381] Updated weights for policy 1, policy_version 1538710 (0.0006) [2023-12-27 16:19:04,095][20347] Updated weights for policy 0, policy_version 1540297 (0.0006) [2023-12-27 16:19:04,108][20381] Updated weights for policy 1, policy_version 1538720 (0.0006) [2023-12-27 16:19:04,719][20381] Updated weights for policy 1, policy_version 1538730 (0.0006) [2023-12-27 16:19:04,780][20381] Updated weights for policy 1, policy_version 1538740 (0.0008) [2023-12-27 16:19:04,840][20381] Updated weights for policy 1, policy_version 1538750 (0.0007) [2023-12-27 16:19:04,855][20347] Updated weights for policy 0, policy_version 1540307 (0.0007) [2023-12-27 16:19:04,903][20381] Updated weights for policy 1, policy_version 1538760 (0.0008) [2023-12-27 16:19:04,906][20347] Updated weights for policy 0, policy_version 1540317 (0.0007) [2023-12-27 16:19:04,959][20347] Updated weights for policy 0, policy_version 1540327 (0.0008) [2023-12-27 16:19:05,541][20347] Updated weights for policy 0, policy_version 1540337 (0.0005) [2023-12-27 16:19:05,582][20291] KL-divergence is very high: 115.6172 [2023-12-27 16:19:05,586][20291] KL-divergence is very high: 196.7473 [2023-12-27 16:19:05,590][20381] Updated weights for policy 1, policy_version 1538770 (0.0007) [2023-12-27 16:19:05,591][20347] Updated weights for policy 0, policy_version 1540347 (0.0006) [2023-12-27 16:19:05,592][20291] KL-divergence is very high: 237.8184 [2023-12-27 16:19:05,597][20291] KL-divergence is very high: 303.4229 [2023-12-27 16:19:05,603][20291] KL-divergence is very high: 228.6937 [2023-12-27 16:19:05,613][20291] KL-divergence is very high: 167.3496 [2023-12-27 16:19:05,644][20347] Updated weights for policy 0, policy_version 1540357 (0.0006) [2023-12-27 16:19:05,645][20381] Updated weights for policy 1, policy_version 1538780 (0.0005) [2023-12-27 16:19:05,691][20347] Updated weights for policy 0, policy_version 1540367 (0.0006) [2023-12-27 16:19:05,697][20381] Updated weights for policy 1, policy_version 1538790 (0.0005) [2023-12-27 16:19:06,243][20291] KL-divergence is very high: 291.3736 [2023-12-27 16:19:06,289][20347] Updated weights for policy 0, policy_version 1540377 (0.0009) [2023-12-27 16:19:06,290][20291] KL-divergence is very high: 262.0343 [2023-12-27 16:19:06,321][20381] Updated weights for policy 1, policy_version 1538800 (0.0007) [2023-12-27 16:19:06,337][20291] KL-divergence is very high: 153.6630 [2023-12-27 16:19:06,349][20347] Updated weights for policy 0, policy_version 1540387 (0.0006) [2023-12-27 16:19:06,378][20381] Updated weights for policy 1, policy_version 1538810 (0.0008) [2023-12-27 16:19:06,388][20291] KL-divergence is very high: 106.7953 [2023-12-27 16:19:06,410][20347] Updated weights for policy 0, policy_version 1540397 (0.0010) [2023-12-27 16:19:06,435][20381] Updated weights for policy 1, policy_version 1538820 (0.0008) [2023-12-27 16:19:07,017][20381] Updated weights for policy 1, policy_version 1538830 (0.0007) [2023-12-27 16:19:07,065][20347] Updated weights for policy 0, policy_version 1540407 (0.0010) [2023-12-27 16:19:07,075][20381] Updated weights for policy 1, policy_version 1538840 (0.0007) [2023-12-27 16:19:07,118][20347] Updated weights for policy 0, policy_version 1540417 (0.0008) [2023-12-27 16:19:07,130][20381] Updated weights for policy 1, policy_version 1538850 (0.0006) [2023-12-27 16:19:07,177][20347] Updated weights for policy 0, policy_version 1540427 (0.0005) [2023-12-27 16:19:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 788414464. Throughput: 0: 10338.2, 1: 10222.6. Samples: 788376080. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:07,363][19170] Avg episode reward: [(0, '605.650'), (1, '990.270')] [2023-12-27 16:19:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001538856_393986048.pth... [2023-12-27 16:19:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001540432_394428416.pth... [2023-12-27 16:19:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001539248_394125312.pth [2023-12-27 16:19:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001537672_393682944.pth [2023-12-27 16:19:07,765][20347] Updated weights for policy 0, policy_version 1540437 (0.0009) [2023-12-27 16:19:07,769][20381] Updated weights for policy 1, policy_version 1538860 (0.0007) [2023-12-27 16:19:07,814][20347] Updated weights for policy 0, policy_version 1540447 (0.0010) [2023-12-27 16:19:07,829][20381] Updated weights for policy 1, policy_version 1538870 (0.0007) [2023-12-27 16:19:07,864][20347] Updated weights for policy 0, policy_version 1540457 (0.0010) [2023-12-27 16:19:07,877][20381] Updated weights for policy 1, policy_version 1538880 (0.0007) [2023-12-27 16:19:08,590][20381] Updated weights for policy 1, policy_version 1538890 (0.0008) [2023-12-27 16:19:08,612][20347] Updated weights for policy 0, policy_version 1540467 (0.0010) [2023-12-27 16:19:08,649][20381] Updated weights for policy 1, policy_version 1538900 (0.0010) [2023-12-27 16:19:08,671][20347] Updated weights for policy 0, policy_version 1540477 (0.0008) [2023-12-27 16:19:08,709][20381] Updated weights for policy 1, policy_version 1538910 (0.0011) [2023-12-27 16:19:08,732][20347] Updated weights for policy 0, policy_version 1540487 (0.0007) [2023-12-27 16:19:08,763][20381] Updated weights for policy 1, policy_version 1538920 (0.0009) [2023-12-27 16:19:09,444][20347] Updated weights for policy 0, policy_version 1540497 (0.0008) [2023-12-27 16:19:09,472][20381] Updated weights for policy 1, policy_version 1538930 (0.0008) [2023-12-27 16:19:09,494][20347] Updated weights for policy 0, policy_version 1540507 (0.0007) [2023-12-27 16:19:09,538][20381] Updated weights for policy 1, policy_version 1538940 (0.0008) [2023-12-27 16:19:09,550][20347] Updated weights for policy 0, policy_version 1540517 (0.0007) [2023-12-27 16:19:09,568][20292] KL-divergence is very high: 671.7693 [2023-12-27 16:19:09,609][20347] Updated weights for policy 0, policy_version 1540527 (0.0010) [2023-12-27 16:19:09,609][20381] Updated weights for policy 1, policy_version 1538950 (0.0008) [2023-12-27 16:19:10,309][20381] Updated weights for policy 1, policy_version 1538960 (0.0006) [2023-12-27 16:19:10,331][20347] Updated weights for policy 0, policy_version 1540537 (0.0007) [2023-12-27 16:19:10,365][20381] Updated weights for policy 1, policy_version 1538970 (0.0008) [2023-12-27 16:19:10,387][20347] Updated weights for policy 0, policy_version 1540547 (0.0007) [2023-12-27 16:19:10,422][20381] Updated weights for policy 1, policy_version 1538980 (0.0008) [2023-12-27 16:19:10,447][20347] Updated weights for policy 0, policy_version 1540557 (0.0005) [2023-12-27 16:19:11,130][20381] Updated weights for policy 1, policy_version 1538990 (0.0007) [2023-12-27 16:19:11,146][20347] Updated weights for policy 0, policy_version 1540567 (0.0008) [2023-12-27 16:19:11,195][20381] Updated weights for policy 1, policy_version 1539000 (0.0009) [2023-12-27 16:19:11,205][20347] Updated weights for policy 0, policy_version 1540577 (0.0006) [2023-12-27 16:19:11,259][20381] Updated weights for policy 1, policy_version 1539010 (0.0008) [2023-12-27 16:19:11,268][20347] Updated weights for policy 0, policy_version 1540587 (0.0007) [2023-12-27 16:19:11,943][20347] Updated weights for policy 0, policy_version 1540597 (0.0006) [2023-12-27 16:19:11,993][20381] Updated weights for policy 1, policy_version 1539020 (0.0009) [2023-12-27 16:19:11,997][20347] Updated weights for policy 0, policy_version 1540607 (0.0008) [2023-12-27 16:19:12,050][20381] Updated weights for policy 1, policy_version 1539030 (0.0006) [2023-12-27 16:19:12,055][20347] Updated weights for policy 0, policy_version 1540617 (0.0007) [2023-12-27 16:19:12,115][20381] Updated weights for policy 1, policy_version 1539040 (0.0008) [2023-12-27 16:19:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 788512768. Throughput: 0: 10347.3, 1: 10257.1. Samples: 788501368. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:12,363][19170] Avg episode reward: [(0, '641.830'), (1, '980.530')] [2023-12-27 16:19:12,768][20347] Updated weights for policy 0, policy_version 1540627 (0.0006) [2023-12-27 16:19:12,833][20381] Updated weights for policy 1, policy_version 1539050 (0.0008) [2023-12-27 16:19:12,837][20347] Updated weights for policy 0, policy_version 1540637 (0.0005) [2023-12-27 16:19:12,890][20381] Updated weights for policy 1, policy_version 1539060 (0.0008) [2023-12-27 16:19:12,893][20347] Updated weights for policy 0, policy_version 1540647 (0.0006) [2023-12-27 16:19:12,944][20381] Updated weights for policy 1, policy_version 1539070 (0.0006) [2023-12-27 16:19:13,000][20381] Updated weights for policy 1, policy_version 1539080 (0.0007) [2023-12-27 16:19:13,440][20347] Updated weights for policy 0, policy_version 1540657 (0.0006) [2023-12-27 16:19:13,499][20347] Updated weights for policy 0, policy_version 1540667 (0.0008) [2023-12-27 16:19:13,548][20347] Updated weights for policy 0, policy_version 1540677 (0.0007) [2023-12-27 16:19:13,600][20347] Updated weights for policy 0, policy_version 1540687 (0.0007) [2023-12-27 16:19:13,636][20381] Updated weights for policy 1, policy_version 1539090 (0.0006) [2023-12-27 16:19:13,699][20381] Updated weights for policy 1, policy_version 1539100 (0.0007) [2023-12-27 16:19:13,765][20381] Updated weights for policy 1, policy_version 1539110 (0.0007) [2023-12-27 16:19:14,228][20347] Updated weights for policy 0, policy_version 1540697 (0.0007) [2023-12-27 16:19:14,284][20347] Updated weights for policy 0, policy_version 1540707 (0.0008) [2023-12-27 16:19:14,336][20347] Updated weights for policy 0, policy_version 1540717 (0.0008) [2023-12-27 16:19:14,450][20381] Updated weights for policy 1, policy_version 1539120 (0.0006) [2023-12-27 16:19:14,509][20381] Updated weights for policy 1, policy_version 1539130 (0.0008) [2023-12-27 16:19:14,565][20381] Updated weights for policy 1, policy_version 1539140 (0.0008) [2023-12-27 16:19:14,987][20347] Updated weights for policy 0, policy_version 1540727 (0.0007) [2023-12-27 16:19:15,044][20347] Updated weights for policy 0, policy_version 1540737 (0.0008) [2023-12-27 16:19:15,097][20347] Updated weights for policy 0, policy_version 1540747 (0.0008) [2023-12-27 16:19:15,285][20381] Updated weights for policy 1, policy_version 1539150 (0.0007) [2023-12-27 16:19:15,341][20381] Updated weights for policy 1, policy_version 1539160 (0.0008) [2023-12-27 16:19:15,392][20381] Updated weights for policy 1, policy_version 1539170 (0.0010) [2023-12-27 16:19:15,750][20347] Updated weights for policy 0, policy_version 1540757 (0.0007) [2023-12-27 16:19:15,815][20347] Updated weights for policy 0, policy_version 1540767 (0.0008) [2023-12-27 16:19:15,873][20347] Updated weights for policy 0, policy_version 1540777 (0.0005) [2023-12-27 16:19:16,027][20381] Updated weights for policy 1, policy_version 1539180 (0.0009) [2023-12-27 16:19:16,092][20381] Updated weights for policy 1, policy_version 1539190 (0.0008) [2023-12-27 16:19:16,159][20381] Updated weights for policy 1, policy_version 1539200 (0.0008) [2023-12-27 16:19:16,451][20347] Updated weights for policy 0, policy_version 1540787 (0.0005) [2023-12-27 16:19:16,500][20347] Updated weights for policy 0, policy_version 1540797 (0.0005) [2023-12-27 16:19:16,549][20347] Updated weights for policy 0, policy_version 1540807 (0.0006) [2023-12-27 16:19:16,825][20381] Updated weights for policy 1, policy_version 1539210 (0.0008) [2023-12-27 16:19:16,890][20381] Updated weights for policy 1, policy_version 1539220 (0.0006) [2023-12-27 16:19:16,943][20381] Updated weights for policy 1, policy_version 1539230 (0.0006) [2023-12-27 16:19:17,003][20381] Updated weights for policy 1, policy_version 1539240 (0.0006) [2023-12-27 16:19:17,100][20347] Updated weights for policy 0, policy_version 1540817 (0.0007) [2023-12-27 16:19:17,164][20347] Updated weights for policy 0, policy_version 1540827 (0.0005) [2023-12-27 16:19:17,223][20347] Updated weights for policy 0, policy_version 1540837 (0.0006) [2023-12-27 16:19:17,271][20347] Updated weights for policy 0, policy_version 1540847 (0.0008) [2023-12-27 16:19:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 788619264. Throughput: 0: 10315.7, 1: 10177.8. Samples: 788626912. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:17,364][19170] Avg episode reward: [(0, '585.940'), (1, '951.180')] [2023-12-27 16:19:17,580][20381] Updated weights for policy 1, policy_version 1539250 (0.0005) [2023-12-27 16:19:17,628][20381] Updated weights for policy 1, policy_version 1539260 (0.0005) [2023-12-27 16:19:17,679][20381] Updated weights for policy 1, policy_version 1539270 (0.0006) [2023-12-27 16:19:17,864][20347] Updated weights for policy 0, policy_version 1540857 (0.0005) [2023-12-27 16:19:17,915][20347] Updated weights for policy 0, policy_version 1540867 (0.0005) [2023-12-27 16:19:17,967][20347] Updated weights for policy 0, policy_version 1540877 (0.0005) [2023-12-27 16:19:18,218][20381] Updated weights for policy 1, policy_version 1539280 (0.0009) [2023-12-27 16:19:18,267][20381] Updated weights for policy 1, policy_version 1539290 (0.0010) [2023-12-27 16:19:18,315][20381] Updated weights for policy 1, policy_version 1539300 (0.0010) [2023-12-27 16:19:18,569][20347] Updated weights for policy 0, policy_version 1540887 (0.0007) [2023-12-27 16:19:18,627][20347] Updated weights for policy 0, policy_version 1540897 (0.0007) [2023-12-27 16:19:18,690][20347] Updated weights for policy 0, policy_version 1540907 (0.0007) [2023-12-27 16:19:19,048][20381] Updated weights for policy 1, policy_version 1539310 (0.0010) [2023-12-27 16:19:19,102][20381] Updated weights for policy 1, policy_version 1539320 (0.0010) [2023-12-27 16:19:19,153][20381] Updated weights for policy 1, policy_version 1539330 (0.0010) [2023-12-27 16:19:19,462][20347] Updated weights for policy 0, policy_version 1540917 (0.0008) [2023-12-27 16:19:19,531][20347] Updated weights for policy 0, policy_version 1540927 (0.0009) [2023-12-27 16:19:19,594][20347] Updated weights for policy 0, policy_version 1540937 (0.0008) [2023-12-27 16:19:19,922][20381] Updated weights for policy 1, policy_version 1539340 (0.0010) [2023-12-27 16:19:19,993][20381] Updated weights for policy 1, policy_version 1539350 (0.0009) [2023-12-27 16:19:20,055][20381] Updated weights for policy 1, policy_version 1539360 (0.0011) [2023-12-27 16:19:20,358][20347] Updated weights for policy 0, policy_version 1540947 (0.0008) [2023-12-27 16:19:20,415][20347] Updated weights for policy 0, policy_version 1540957 (0.0008) [2023-12-27 16:19:20,479][20347] Updated weights for policy 0, policy_version 1540967 (0.0008) [2023-12-27 16:19:20,731][20381] Updated weights for policy 1, policy_version 1539370 (0.0011) [2023-12-27 16:19:20,793][20381] Updated weights for policy 1, policy_version 1539380 (0.0011) [2023-12-27 16:19:20,853][20381] Updated weights for policy 1, policy_version 1539390 (0.0007) [2023-12-27 16:19:20,914][20381] Updated weights for policy 1, policy_version 1539400 (0.0005) [2023-12-27 16:19:21,243][20347] Updated weights for policy 0, policy_version 1540977 (0.0008) [2023-12-27 16:19:21,313][20347] Updated weights for policy 0, policy_version 1540987 (0.0008) [2023-12-27 16:19:21,377][20347] Updated weights for policy 0, policy_version 1540997 (0.0008) [2023-12-27 16:19:21,444][20347] Updated weights for policy 0, policy_version 1541007 (0.0008) [2023-12-27 16:19:21,584][20381] Updated weights for policy 1, policy_version 1539410 (0.0008) [2023-12-27 16:19:21,594][20292] KL-divergence is very high: 114.8687 [2023-12-27 16:19:21,619][20292] KL-divergence is very high: 117.9380 [2023-12-27 16:19:21,643][20292] KL-divergence is very high: 146.2195 [2023-12-27 16:19:21,649][20381] Updated weights for policy 1, policy_version 1539421 (0.0012) [2023-12-27 16:19:21,711][20381] Updated weights for policy 1, policy_version 1539431 (0.0011) [2023-12-27 16:19:22,159][20347] Updated weights for policy 0, policy_version 1541017 (0.0008) [2023-12-27 16:19:22,209][20347] Updated weights for policy 0, policy_version 1541027 (0.0007) [2023-12-27 16:19:22,267][20347] Updated weights for policy 0, policy_version 1541037 (0.0008) [2023-12-27 16:19:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 788717568. Throughput: 0: 10336.9, 1: 10189.4. Samples: 788688980. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:22,363][19170] Avg episode reward: [(0, '641.540'), (1, '912.170')] [2023-12-27 16:19:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001539432_394133504.pth... [2023-12-27 16:19:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001541040_394584064.pth... [2023-12-27 16:19:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001539824_394272768.pth [2023-12-27 16:19:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001538248_393830400.pth [2023-12-27 16:19:22,478][20381] Updated weights for policy 1, policy_version 1539441 (0.0010) [2023-12-27 16:19:22,548][20381] Updated weights for policy 1, policy_version 1539451 (0.0010) [2023-12-27 16:19:22,600][20381] Updated weights for policy 1, policy_version 1539461 (0.0006) [2023-12-27 16:19:22,979][20347] Updated weights for policy 0, policy_version 1541047 (0.0006) [2023-12-27 16:19:23,034][20347] Updated weights for policy 0, policy_version 1541057 (0.0005) [2023-12-27 16:19:23,088][20347] Updated weights for policy 0, policy_version 1541067 (0.0006) [2023-12-27 16:19:23,293][20381] Updated weights for policy 1, policy_version 1539471 (0.0009) [2023-12-27 16:19:23,342][20381] Updated weights for policy 1, policy_version 1539481 (0.0010) [2023-12-27 16:19:23,386][20381] Updated weights for policy 1, policy_version 1539491 (0.0010) [2023-12-27 16:19:23,692][20347] Updated weights for policy 0, policy_version 1541077 (0.0006) [2023-12-27 16:19:23,740][20347] Updated weights for policy 0, policy_version 1541087 (0.0005) [2023-12-27 16:19:23,801][20347] Updated weights for policy 0, policy_version 1541097 (0.0010) [2023-12-27 16:19:24,044][20381] Updated weights for policy 1, policy_version 1539501 (0.0009) [2023-12-27 16:19:24,100][20381] Updated weights for policy 1, policy_version 1539511 (0.0011) [2023-12-27 16:19:24,156][20381] Updated weights for policy 1, policy_version 1539521 (0.0011) [2023-12-27 16:19:24,424][20347] Updated weights for policy 0, policy_version 1541107 (0.0010) [2023-12-27 16:19:24,472][20347] Updated weights for policy 0, policy_version 1541117 (0.0009) [2023-12-27 16:19:24,526][20347] Updated weights for policy 0, policy_version 1541127 (0.0006) [2023-12-27 16:19:24,807][20381] Updated weights for policy 1, policy_version 1539531 (0.0011) [2023-12-27 16:19:24,863][20381] Updated weights for policy 1, policy_version 1539541 (0.0011) [2023-12-27 16:19:24,909][20381] Updated weights for policy 1, policy_version 1539551 (0.0011) [2023-12-27 16:19:25,084][20347] Updated weights for policy 0, policy_version 1541137 (0.0005) [2023-12-27 16:19:25,139][20347] Updated weights for policy 0, policy_version 1541147 (0.0007) [2023-12-27 16:19:25,188][20347] Updated weights for policy 0, policy_version 1541157 (0.0010) [2023-12-27 16:19:25,243][20347] Updated weights for policy 0, policy_version 1541167 (0.0010) [2023-12-27 16:19:25,490][20381] Updated weights for policy 1, policy_version 1539561 (0.0010) [2023-12-27 16:19:25,552][20381] Updated weights for policy 1, policy_version 1539571 (0.0005) [2023-12-27 16:19:25,603][20381] Updated weights for policy 1, policy_version 1539581 (0.0007) [2023-12-27 16:19:25,649][20381] Updated weights for policy 1, policy_version 1539591 (0.0011) [2023-12-27 16:19:25,852][20347] Updated weights for policy 0, policy_version 1541177 (0.0010) [2023-12-27 16:19:25,900][20347] Updated weights for policy 0, policy_version 1541187 (0.0010) [2023-12-27 16:19:25,942][20347] Updated weights for policy 0, policy_version 1541197 (0.0010) [2023-12-27 16:19:26,252][20381] Updated weights for policy 1, policy_version 1539601 (0.0006) [2023-12-27 16:19:26,297][20381] Updated weights for policy 1, policy_version 1539611 (0.0007) [2023-12-27 16:19:26,341][20381] Updated weights for policy 1, policy_version 1539621 (0.0007) [2023-12-27 16:19:26,643][20347] Updated weights for policy 0, policy_version 1541207 (0.0009) [2023-12-27 16:19:26,695][20347] Updated weights for policy 0, policy_version 1541217 (0.0008) [2023-12-27 16:19:26,747][20347] Updated weights for policy 0, policy_version 1541227 (0.0007) [2023-12-27 16:19:27,003][20381] Updated weights for policy 1, policy_version 1539631 (0.0009) [2023-12-27 16:19:27,050][20381] Updated weights for policy 1, policy_version 1539641 (0.0008) [2023-12-27 16:19:27,106][20381] Updated weights for policy 1, policy_version 1539651 (0.0007) [2023-12-27 16:19:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 788824064. Throughput: 0: 10333.3, 1: 10239.4. Samples: 788814512. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:27,363][19170] Avg episode reward: [(0, '654.890'), (1, '902.400')] [2023-12-27 16:19:27,393][20347] Updated weights for policy 0, policy_version 1541237 (0.0008) [2023-12-27 16:19:27,438][20347] Updated weights for policy 0, policy_version 1541247 (0.0008) [2023-12-27 16:19:27,487][20347] Updated weights for policy 0, policy_version 1541257 (0.0009) [2023-12-27 16:19:27,744][20381] Updated weights for policy 1, policy_version 1539661 (0.0008) [2023-12-27 16:19:27,792][20381] Updated weights for policy 1, policy_version 1539671 (0.0010) [2023-12-27 16:19:27,850][20381] Updated weights for policy 1, policy_version 1539681 (0.0010) [2023-12-27 16:19:28,228][20347] Updated weights for policy 0, policy_version 1541267 (0.0008) [2023-12-27 16:19:28,275][20347] Updated weights for policy 0, policy_version 1541277 (0.0007) [2023-12-27 16:19:28,328][20347] Updated weights for policy 0, policy_version 1541287 (0.0007) [2023-12-27 16:19:28,502][20381] Updated weights for policy 1, policy_version 1539691 (0.0010) [2023-12-27 16:19:28,560][20381] Updated weights for policy 1, policy_version 1539701 (0.0006) [2023-12-27 16:19:28,615][20381] Updated weights for policy 1, policy_version 1539711 (0.0008) [2023-12-27 16:19:29,012][20347] Updated weights for policy 0, policy_version 1541297 (0.0007) [2023-12-27 16:19:29,066][20347] Updated weights for policy 0, policy_version 1541307 (0.0007) [2023-12-27 16:19:29,119][20347] Updated weights for policy 0, policy_version 1541317 (0.0007) [2023-12-27 16:19:29,171][20347] Updated weights for policy 0, policy_version 1541327 (0.0007) [2023-12-27 16:19:29,317][20381] Updated weights for policy 1, policy_version 1539721 (0.0008) [2023-12-27 16:19:29,383][20381] Updated weights for policy 1, policy_version 1539731 (0.0009) [2023-12-27 16:19:29,449][20381] Updated weights for policy 1, policy_version 1539741 (0.0006) [2023-12-27 16:19:29,508][20381] Updated weights for policy 1, policy_version 1539751 (0.0006) [2023-12-27 16:19:29,919][20347] Updated weights for policy 0, policy_version 1541337 (0.0007) [2023-12-27 16:19:29,981][20347] Updated weights for policy 0, policy_version 1541347 (0.0008) [2023-12-27 16:19:30,037][20347] Updated weights for policy 0, policy_version 1541357 (0.0008) [2023-12-27 16:19:30,126][20381] Updated weights for policy 1, policy_version 1539761 (0.0008) [2023-12-27 16:19:30,182][20381] Updated weights for policy 1, policy_version 1539771 (0.0007) [2023-12-27 16:19:30,243][20381] Updated weights for policy 1, policy_version 1539781 (0.0008) [2023-12-27 16:19:30,702][20347] Updated weights for policy 0, policy_version 1541367 (0.0007) [2023-12-27 16:19:30,758][20347] Updated weights for policy 0, policy_version 1541377 (0.0007) [2023-12-27 16:19:30,819][20347] Updated weights for policy 0, policy_version 1541387 (0.0008) [2023-12-27 16:19:30,934][20381] Updated weights for policy 1, policy_version 1539791 (0.0008) [2023-12-27 16:19:30,992][20381] Updated weights for policy 1, policy_version 1539801 (0.0007) [2023-12-27 16:19:31,049][20381] Updated weights for policy 1, policy_version 1539811 (0.0008) [2023-12-27 16:19:31,453][20347] Updated weights for policy 0, policy_version 1541397 (0.0006) [2023-12-27 16:19:31,506][20347] Updated weights for policy 0, policy_version 1541407 (0.0006) [2023-12-27 16:19:31,556][20347] Updated weights for policy 0, policy_version 1541417 (0.0005) [2023-12-27 16:19:31,692][20381] Updated weights for policy 1, policy_version 1539821 (0.0007) [2023-12-27 16:19:31,752][20381] Updated weights for policy 1, policy_version 1539831 (0.0007) [2023-12-27 16:19:31,814][20381] Updated weights for policy 1, policy_version 1539841 (0.0006) [2023-12-27 16:19:32,255][20347] Updated weights for policy 0, policy_version 1541427 (0.0007) [2023-12-27 16:19:32,317][20347] Updated weights for policy 0, policy_version 1541437 (0.0008) [2023-12-27 16:19:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 788922368. Throughput: 0: 10330.8, 1: 10230.4. Samples: 788938648. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:32,363][19170] Avg episode reward: [(0, '731.890'), (1, '892.620')] [2023-12-27 16:19:32,380][20347] Updated weights for policy 0, policy_version 1541447 (0.0007) [2023-12-27 16:19:32,468][20381] Updated weights for policy 1, policy_version 1539851 (0.0005) [2023-12-27 16:19:32,535][20381] Updated weights for policy 1, policy_version 1539861 (0.0005) [2023-12-27 16:19:32,602][20381] Updated weights for policy 1, policy_version 1539871 (0.0006) [2023-12-27 16:19:33,030][20347] Updated weights for policy 0, policy_version 1541457 (0.0009) [2023-12-27 16:19:33,089][20347] Updated weights for policy 0, policy_version 1541467 (0.0009) [2023-12-27 16:19:33,148][20347] Updated weights for policy 0, policy_version 1541477 (0.0010) [2023-12-27 16:19:33,203][20347] Updated weights for policy 0, policy_version 1541487 (0.0010) [2023-12-27 16:19:33,247][20381] Updated weights for policy 1, policy_version 1539881 (0.0008) [2023-12-27 16:19:33,300][20381] Updated weights for policy 1, policy_version 1539891 (0.0009) [2023-12-27 16:19:33,342][20381] Updated weights for policy 1, policy_version 1539901 (0.0010) [2023-12-27 16:19:33,387][20381] Updated weights for policy 1, policy_version 1539911 (0.0008) [2023-12-27 16:19:33,766][20347] Updated weights for policy 0, policy_version 1541497 (0.0010) [2023-12-27 16:19:33,815][20347] Updated weights for policy 0, policy_version 1541507 (0.0010) [2023-12-27 16:19:33,867][20347] Updated weights for policy 0, policy_version 1541517 (0.0010) [2023-12-27 16:19:34,082][20381] Updated weights for policy 1, policy_version 1539921 (0.0010) [2023-12-27 16:19:34,141][20381] Updated weights for policy 1, policy_version 1539931 (0.0011) [2023-12-27 16:19:34,206][20381] Updated weights for policy 1, policy_version 1539941 (0.0009) [2023-12-27 16:19:34,565][20347] Updated weights for policy 0, policy_version 1541527 (0.0011) [2023-12-27 16:19:34,638][20347] Updated weights for policy 0, policy_version 1541537 (0.0011) [2023-12-27 16:19:34,708][20347] Updated weights for policy 0, policy_version 1541547 (0.0010) [2023-12-27 16:19:34,908][20381] Updated weights for policy 1, policy_version 1539951 (0.0010) [2023-12-27 16:19:34,966][20381] Updated weights for policy 1, policy_version 1539961 (0.0010) [2023-12-27 16:19:35,025][20381] Updated weights for policy 1, policy_version 1539971 (0.0011) [2023-12-27 16:19:35,361][20347] Updated weights for policy 0, policy_version 1541557 (0.0008) [2023-12-27 16:19:35,412][20347] Updated weights for policy 0, policy_version 1541567 (0.0010) [2023-12-27 16:19:35,470][20347] Updated weights for policy 0, policy_version 1541577 (0.0010) [2023-12-27 16:19:35,650][20381] Updated weights for policy 1, policy_version 1539981 (0.0011) [2023-12-27 16:19:35,716][20381] Updated weights for policy 1, policy_version 1539991 (0.0011) [2023-12-27 16:19:35,781][20381] Updated weights for policy 1, policy_version 1540001 (0.0011) [2023-12-27 16:19:36,086][20347] Updated weights for policy 0, policy_version 1541587 (0.0010) [2023-12-27 16:19:36,152][20347] Updated weights for policy 0, policy_version 1541597 (0.0011) [2023-12-27 16:19:36,209][20347] Updated weights for policy 0, policy_version 1541607 (0.0008) [2023-12-27 16:19:36,463][20381] Updated weights for policy 1, policy_version 1540011 (0.0011) [2023-12-27 16:19:36,519][20381] Updated weights for policy 1, policy_version 1540021 (0.0008) [2023-12-27 16:19:36,576][20381] Updated weights for policy 1, policy_version 1540031 (0.0007) [2023-12-27 16:19:36,905][20347] Updated weights for policy 0, policy_version 1541617 (0.0010) [2023-12-27 16:19:36,959][20347] Updated weights for policy 0, policy_version 1541627 (0.0007) [2023-12-27 16:19:37,019][20347] Updated weights for policy 0, policy_version 1541637 (0.0007) [2023-12-27 16:19:37,074][20347] Updated weights for policy 0, policy_version 1541647 (0.0007) [2023-12-27 16:19:37,249][20381] Updated weights for policy 1, policy_version 1540041 (0.0009) [2023-12-27 16:19:37,300][20381] Updated weights for policy 1, policy_version 1540051 (0.0008) [2023-12-27 16:19:37,350][20381] Updated weights for policy 1, policy_version 1540061 (0.0011) [2023-12-27 16:19:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 789028864. Throughput: 0: 10347.7, 1: 10252.0. Samples: 789000908. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:37,363][19170] Avg episode reward: [(0, '699.310'), (1, '854.310')] [2023-12-27 16:19:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001541648_394739712.pth... [2023-12-27 16:19:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001540432_394428416.pth [2023-12-27 16:19:37,399][20381] Updated weights for policy 1, policy_version 1540071 (0.0011) [2023-12-27 16:19:37,403][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001540072_394297344.pth... [2023-12-27 16:19:37,406][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001538856_393986048.pth [2023-12-27 16:19:37,728][20347] Updated weights for policy 0, policy_version 1541657 (0.0010) [2023-12-27 16:19:37,784][20347] Updated weights for policy 0, policy_version 1541667 (0.0010) [2023-12-27 16:19:37,843][20347] Updated weights for policy 0, policy_version 1541677 (0.0010) [2023-12-27 16:19:38,037][20381] Updated weights for policy 1, policy_version 1540081 (0.0006) [2023-12-27 16:19:38,095][20381] Updated weights for policy 1, policy_version 1540091 (0.0007) [2023-12-27 16:19:38,150][20381] Updated weights for policy 1, policy_version 1540101 (0.0008) [2023-12-27 16:19:38,526][20347] Updated weights for policy 0, policy_version 1541687 (0.0007) [2023-12-27 16:19:38,579][20347] Updated weights for policy 0, policy_version 1541697 (0.0006) [2023-12-27 16:19:38,643][20347] Updated weights for policy 0, policy_version 1541707 (0.0008) [2023-12-27 16:19:38,868][20381] Updated weights for policy 1, policy_version 1540111 (0.0009) [2023-12-27 16:19:38,930][20381] Updated weights for policy 1, policy_version 1540121 (0.0005) [2023-12-27 16:19:38,991][20381] Updated weights for policy 1, policy_version 1540131 (0.0008) [2023-12-27 16:19:39,338][20347] Updated weights for policy 0, policy_version 1541717 (0.0008) [2023-12-27 16:19:39,414][20347] Updated weights for policy 0, policy_version 1541727 (0.0008) [2023-12-27 16:19:39,468][20347] Updated weights for policy 0, policy_version 1541737 (0.0006) [2023-12-27 16:19:39,736][20381] Updated weights for policy 1, policy_version 1540141 (0.0008) [2023-12-27 16:19:39,798][20381] Updated weights for policy 1, policy_version 1540151 (0.0007) [2023-12-27 16:19:39,875][20381] Updated weights for policy 1, policy_version 1540161 (0.0008) [2023-12-27 16:19:40,174][20347] Updated weights for policy 0, policy_version 1541747 (0.0006) [2023-12-27 16:19:40,227][20347] Updated weights for policy 0, policy_version 1541757 (0.0008) [2023-12-27 16:19:40,290][20347] Updated weights for policy 0, policy_version 1541767 (0.0007) [2023-12-27 16:19:40,569][20381] Updated weights for policy 1, policy_version 1540171 (0.0008) [2023-12-27 16:19:40,614][20381] Updated weights for policy 1, policy_version 1540181 (0.0007) [2023-12-27 16:19:40,660][20381] Updated weights for policy 1, policy_version 1540191 (0.0008) [2023-12-27 16:19:40,956][20347] Updated weights for policy 0, policy_version 1541777 (0.0007) [2023-12-27 16:19:41,027][20347] Updated weights for policy 0, policy_version 1541787 (0.0008) [2023-12-27 16:19:41,094][20347] Updated weights for policy 0, policy_version 1541797 (0.0008) [2023-12-27 16:19:41,168][20347] Updated weights for policy 0, policy_version 1541807 (0.0008) [2023-12-27 16:19:41,342][20381] Updated weights for policy 1, policy_version 1540201 (0.0008) [2023-12-27 16:19:41,403][20381] Updated weights for policy 1, policy_version 1540211 (0.0010) [2023-12-27 16:19:41,464][20381] Updated weights for policy 1, policy_version 1540221 (0.0009) [2023-12-27 16:19:41,520][20381] Updated weights for policy 1, policy_version 1540231 (0.0009) [2023-12-27 16:19:41,901][20347] Updated weights for policy 0, policy_version 1541817 (0.0006) [2023-12-27 16:19:41,964][20347] Updated weights for policy 0, policy_version 1541827 (0.0006) [2023-12-27 16:19:42,014][20347] Updated weights for policy 0, policy_version 1541837 (0.0008) [2023-12-27 16:19:42,216][20381] Updated weights for policy 1, policy_version 1540241 (0.0008) [2023-12-27 16:19:42,270][20381] Updated weights for policy 1, policy_version 1540251 (0.0009) [2023-12-27 16:19:42,327][20381] Updated weights for policy 1, policy_version 1540261 (0.0008) [2023-12-27 16:19:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 789135360. Throughput: 0: 10321.8, 1: 10254.1. Samples: 789123256. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:42,363][19170] Avg episode reward: [(0, '614.290'), (1, '844.660')] [2023-12-27 16:19:42,657][20347] Updated weights for policy 0, policy_version 1541847 (0.0005) [2023-12-27 16:19:42,718][20347] Updated weights for policy 0, policy_version 1541857 (0.0006) [2023-12-27 16:19:42,780][20347] Updated weights for policy 0, policy_version 1541867 (0.0006) [2023-12-27 16:19:43,095][20381] Updated weights for policy 1, policy_version 1540271 (0.0008) [2023-12-27 16:19:43,150][20381] Updated weights for policy 1, policy_version 1540281 (0.0008) [2023-12-27 16:19:43,203][20381] Updated weights for policy 1, policy_version 1540291 (0.0006) [2023-12-27 16:19:43,320][20347] Updated weights for policy 0, policy_version 1541877 (0.0005) [2023-12-27 16:19:43,376][20347] Updated weights for policy 0, policy_version 1541887 (0.0006) [2023-12-27 16:19:43,432][20347] Updated weights for policy 0, policy_version 1541897 (0.0007) [2023-12-27 16:19:43,897][20381] Updated weights for policy 1, policy_version 1540301 (0.0008) [2023-12-27 16:19:43,961][20381] Updated weights for policy 1, policy_version 1540311 (0.0010) [2023-12-27 16:19:44,025][20381] Updated weights for policy 1, policy_version 1540321 (0.0011) [2023-12-27 16:19:44,085][20347] Updated weights for policy 0, policy_version 1541907 (0.0008) [2023-12-27 16:19:44,139][20347] Updated weights for policy 0, policy_version 1541917 (0.0005) [2023-12-27 16:19:44,192][20347] Updated weights for policy 0, policy_version 1541927 (0.0009) [2023-12-27 16:19:44,653][20381] Updated weights for policy 1, policy_version 1540331 (0.0011) [2023-12-27 16:19:44,714][20381] Updated weights for policy 1, policy_version 1540341 (0.0010) [2023-12-27 16:19:44,764][20381] Updated weights for policy 1, policy_version 1540351 (0.0010) [2023-12-27 16:19:44,770][20347] Updated weights for policy 0, policy_version 1541937 (0.0010) [2023-12-27 16:19:44,831][20347] Updated weights for policy 0, policy_version 1541947 (0.0008) [2023-12-27 16:19:44,888][20347] Updated weights for policy 0, policy_version 1541957 (0.0011) [2023-12-27 16:19:44,953][20347] Updated weights for policy 0, policy_version 1541967 (0.0007) [2023-12-27 16:19:45,459][20381] Updated weights for policy 1, policy_version 1540361 (0.0010) [2023-12-27 16:19:45,520][20381] Updated weights for policy 1, policy_version 1540371 (0.0008) [2023-12-27 16:19:45,581][20381] Updated weights for policy 1, policy_version 1540381 (0.0008) [2023-12-27 16:19:45,590][20347] Updated weights for policy 0, policy_version 1541977 (0.0005) [2023-12-27 16:19:45,632][20381] Updated weights for policy 1, policy_version 1540391 (0.0008) [2023-12-27 16:19:45,638][20347] Updated weights for policy 0, policy_version 1541987 (0.0005) [2023-12-27 16:19:45,688][20347] Updated weights for policy 0, policy_version 1541997 (0.0005) [2023-12-27 16:19:46,300][20381] Updated weights for policy 1, policy_version 1540401 (0.0005) [2023-12-27 16:19:46,356][20381] Updated weights for policy 1, policy_version 1540411 (0.0006) [2023-12-27 16:19:46,357][20347] Updated weights for policy 0, policy_version 1542007 (0.0007) [2023-12-27 16:19:46,406][20381] Updated weights for policy 1, policy_version 1540421 (0.0006) [2023-12-27 16:19:46,414][20347] Updated weights for policy 0, policy_version 1542017 (0.0006) [2023-12-27 16:19:46,462][20347] Updated weights for policy 0, policy_version 1542027 (0.0007) [2023-12-27 16:19:47,075][20381] Updated weights for policy 1, policy_version 1540431 (0.0005) [2023-12-27 16:19:47,092][20347] Updated weights for policy 0, policy_version 1542037 (0.0006) [2023-12-27 16:19:47,130][20381] Updated weights for policy 1, policy_version 1540441 (0.0006) [2023-12-27 16:19:47,143][20347] Updated weights for policy 0, policy_version 1542047 (0.0007) [2023-12-27 16:19:47,180][20381] Updated weights for policy 1, policy_version 1540451 (0.0006) [2023-12-27 16:19:47,204][20347] Updated weights for policy 0, policy_version 1542057 (0.0006) [2023-12-27 16:19:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 789241856. Throughput: 0: 10431.4, 1: 10257.8. Samples: 789247380. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:47,363][19170] Avg episode reward: [(0, '699.630'), (1, '922.120')] [2023-12-27 16:19:47,832][20381] Updated weights for policy 1, policy_version 1540461 (0.0008) [2023-12-27 16:19:47,834][20347] Updated weights for policy 0, policy_version 1542067 (0.0008) [2023-12-27 16:19:47,892][20381] Updated weights for policy 1, policy_version 1540471 (0.0011) [2023-12-27 16:19:47,893][20347] Updated weights for policy 0, policy_version 1542077 (0.0010) [2023-12-27 16:19:47,943][20347] Updated weights for policy 0, policy_version 1542087 (0.0010) [2023-12-27 16:19:47,948][20381] Updated weights for policy 1, policy_version 1540481 (0.0011) [2023-12-27 16:19:48,587][20347] Updated weights for policy 0, policy_version 1542097 (0.0010) [2023-12-27 16:19:48,622][20381] Updated weights for policy 1, policy_version 1540491 (0.0009) [2023-12-27 16:19:48,645][20347] Updated weights for policy 0, policy_version 1542107 (0.0007) [2023-12-27 16:19:48,690][20381] Updated weights for policy 1, policy_version 1540501 (0.0009) [2023-12-27 16:19:48,697][20347] Updated weights for policy 0, policy_version 1542117 (0.0007) [2023-12-27 16:19:48,745][20381] Updated weights for policy 1, policy_version 1540511 (0.0011) [2023-12-27 16:19:48,750][20347] Updated weights for policy 0, policy_version 1542127 (0.0008) [2023-12-27 16:19:49,386][20347] Updated weights for policy 0, policy_version 1542137 (0.0007) [2023-12-27 16:19:49,400][20381] Updated weights for policy 1, policy_version 1540521 (0.0010) [2023-12-27 16:19:49,447][20347] Updated weights for policy 0, policy_version 1542147 (0.0006) [2023-12-27 16:19:49,463][20381] Updated weights for policy 1, policy_version 1540531 (0.0009) [2023-12-27 16:19:49,506][20347] Updated weights for policy 0, policy_version 1542157 (0.0007) [2023-12-27 16:19:49,524][20381] Updated weights for policy 1, policy_version 1540541 (0.0008) [2023-12-27 16:19:49,583][20381] Updated weights for policy 1, policy_version 1540551 (0.0009) [2023-12-27 16:19:50,202][20347] Updated weights for policy 0, policy_version 1542167 (0.0006) [2023-12-27 16:19:50,265][20347] Updated weights for policy 0, policy_version 1542177 (0.0006) [2023-12-27 16:19:50,290][20381] Updated weights for policy 1, policy_version 1540561 (0.0010) [2023-12-27 16:19:50,319][20347] Updated weights for policy 0, policy_version 1542187 (0.0006) [2023-12-27 16:19:50,350][20381] Updated weights for policy 1, policy_version 1540571 (0.0008) [2023-12-27 16:19:50,421][20381] Updated weights for policy 1, policy_version 1540581 (0.0008) [2023-12-27 16:19:51,037][20347] Updated weights for policy 0, policy_version 1542197 (0.0008) [2023-12-27 16:19:51,039][20381] Updated weights for policy 1, policy_version 1540591 (0.0006) [2023-12-27 16:19:51,098][20381] Updated weights for policy 1, policy_version 1540601 (0.0008) [2023-12-27 16:19:51,098][20347] Updated weights for policy 0, policy_version 1542207 (0.0007) [2023-12-27 16:19:51,160][20347] Updated weights for policy 0, policy_version 1542217 (0.0007) [2023-12-27 16:19:51,164][20381] Updated weights for policy 1, policy_version 1540611 (0.0007) [2023-12-27 16:19:51,885][20381] Updated weights for policy 1, policy_version 1540621 (0.0007) [2023-12-27 16:19:51,949][20381] Updated weights for policy 1, policy_version 1540631 (0.0006) [2023-12-27 16:19:51,959][20347] Updated weights for policy 0, policy_version 1542227 (0.0008) [2023-12-27 16:19:52,005][20381] Updated weights for policy 1, policy_version 1540641 (0.0008) [2023-12-27 16:19:52,021][20347] Updated weights for policy 0, policy_version 1542237 (0.0008) [2023-12-27 16:19:52,076][20347] Updated weights for policy 0, policy_version 1542247 (0.0006) [2023-12-27 16:19:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 789340160. Throughput: 0: 10497.9, 1: 10251.0. Samples: 789309784. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:52,363][19170] Avg episode reward: [(0, '668.560'), (1, '882.870')] [2023-12-27 16:19:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001540648_394444800.pth... [2023-12-27 16:19:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001542256_394895360.pth... [2023-12-27 16:19:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001539432_394133504.pth [2023-12-27 16:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001541040_394584064.pth [2023-12-27 16:19:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001540648_394444800.pth [2023-12-27 16:19:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001542256_394895360.pth [2023-12-27 16:19:52,640][20381] Updated weights for policy 1, policy_version 1540651 (0.0006) [2023-12-27 16:19:52,694][20381] Updated weights for policy 1, policy_version 1540661 (0.0007) [2023-12-27 16:19:52,749][20381] Updated weights for policy 1, policy_version 1540671 (0.0008) [2023-12-27 16:19:52,785][20292] KL-divergence is very high: 108.5227 [2023-12-27 16:19:52,787][20347] Updated weights for policy 0, policy_version 1542257 (0.0007) [2023-12-27 16:19:52,845][20347] Updated weights for policy 0, policy_version 1542267 (0.0010) [2023-12-27 16:19:52,894][20347] Updated weights for policy 0, policy_version 1542277 (0.0010) [2023-12-27 16:19:52,943][20347] Updated weights for policy 0, policy_version 1542287 (0.0010) [2023-12-27 16:19:53,459][20381] Updated weights for policy 1, policy_version 1540681 (0.0007) [2023-12-27 16:19:53,511][20381] Updated weights for policy 1, policy_version 1540691 (0.0008) [2023-12-27 16:19:53,559][20381] Updated weights for policy 1, policy_version 1540701 (0.0008) [2023-12-27 16:19:53,603][20347] Updated weights for policy 0, policy_version 1542297 (0.0010) [2023-12-27 16:19:53,613][20381] Updated weights for policy 1, policy_version 1540711 (0.0006) [2023-12-27 16:19:53,660][20347] Updated weights for policy 0, policy_version 1542307 (0.0010) [2023-12-27 16:19:53,720][20347] Updated weights for policy 0, policy_version 1542317 (0.0010) [2023-12-27 16:19:54,278][20381] Updated weights for policy 1, policy_version 1540721 (0.0008) [2023-12-27 16:19:54,337][20381] Updated weights for policy 1, policy_version 1540731 (0.0008) [2023-12-27 16:19:54,396][20381] Updated weights for policy 1, policy_version 1540741 (0.0008) [2023-12-27 16:19:54,400][20347] Updated weights for policy 0, policy_version 1542327 (0.0010) [2023-12-27 16:19:54,452][20347] Updated weights for policy 0, policy_version 1542337 (0.0010) [2023-12-27 16:19:54,506][20347] Updated weights for policy 0, policy_version 1542347 (0.0010) [2023-12-27 16:19:55,094][20381] Updated weights for policy 1, policy_version 1540751 (0.0008) [2023-12-27 16:19:55,129][20347] Updated weights for policy 0, policy_version 1542357 (0.0010) [2023-12-27 16:19:55,153][20381] Updated weights for policy 1, policy_version 1540761 (0.0006) [2023-12-27 16:19:55,185][20347] Updated weights for policy 0, policy_version 1542367 (0.0008) [2023-12-27 16:19:55,203][20381] Updated weights for policy 1, policy_version 1540771 (0.0006) [2023-12-27 16:19:55,244][20347] Updated weights for policy 0, policy_version 1542377 (0.0007) [2023-12-27 16:19:55,762][20381] Updated weights for policy 1, policy_version 1540781 (0.0008) [2023-12-27 16:19:55,821][20381] Updated weights for policy 1, policy_version 1540791 (0.0011) [2023-12-27 16:19:55,882][20381] Updated weights for policy 1, policy_version 1540801 (0.0011) [2023-12-27 16:19:55,965][20347] Updated weights for policy 0, policy_version 1542387 (0.0008) [2023-12-27 16:19:56,019][20347] Updated weights for policy 0, policy_version 1542397 (0.0010) [2023-12-27 16:19:56,075][20347] Updated weights for policy 0, policy_version 1542407 (0.0010) [2023-12-27 16:19:56,557][20381] Updated weights for policy 1, policy_version 1540811 (0.0010) [2023-12-27 16:19:56,614][20381] Updated weights for policy 1, policy_version 1540821 (0.0005) [2023-12-27 16:19:56,676][20381] Updated weights for policy 1, policy_version 1540831 (0.0010) [2023-12-27 16:19:56,755][20347] Updated weights for policy 0, policy_version 1542417 (0.0010) [2023-12-27 16:19:56,805][20347] Updated weights for policy 0, policy_version 1542427 (0.0008) [2023-12-27 16:19:56,850][20347] Updated weights for policy 0, policy_version 1542437 (0.0005) [2023-12-27 16:19:56,902][20347] Updated weights for policy 0, policy_version 1542447 (0.0008) [2023-12-27 16:19:57,203][20381] Updated weights for policy 1, policy_version 1540841 (0.0010) [2023-12-27 16:19:57,255][20381] Updated weights for policy 1, policy_version 1540851 (0.0010) [2023-12-27 16:19:57,308][20381] Updated weights for policy 1, policy_version 1540861 (0.0008) [2023-12-27 16:19:57,360][20381] Updated weights for policy 1, policy_version 1540871 (0.0006) [2023-12-27 16:19:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 789438464. Throughput: 0: 10413.2, 1: 10273.9. Samples: 789432288. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:19:57,363][19170] Avg episode reward: [(0, '718.220'), (1, '807.350')] [2023-12-27 16:19:57,565][20347] Updated weights for policy 0, policy_version 1542457 (0.0006) [2023-12-27 16:19:57,610][20347] Updated weights for policy 0, policy_version 1542467 (0.0005) [2023-12-27 16:19:57,664][20347] Updated weights for policy 0, policy_version 1542477 (0.0005) [2023-12-27 16:19:57,908][20381] Updated weights for policy 1, policy_version 1540881 (0.0005) [2023-12-27 16:19:57,968][20381] Updated weights for policy 1, policy_version 1540891 (0.0005) [2023-12-27 16:19:58,031][20381] Updated weights for policy 1, policy_version 1540901 (0.0007) [2023-12-27 16:19:58,207][20347] Updated weights for policy 0, policy_version 1542487 (0.0006) [2023-12-27 16:19:58,270][20347] Updated weights for policy 0, policy_version 1542497 (0.0008) [2023-12-27 16:19:58,331][20347] Updated weights for policy 0, policy_version 1542507 (0.0008) [2023-12-27 16:19:58,683][20381] Updated weights for policy 1, policy_version 1540911 (0.0010) [2023-12-27 16:19:58,751][20381] Updated weights for policy 1, policy_version 1540921 (0.0009) [2023-12-27 16:19:58,822][20381] Updated weights for policy 1, policy_version 1540931 (0.0010) [2023-12-27 16:19:59,050][20347] Updated weights for policy 0, policy_version 1542517 (0.0008) [2023-12-27 16:19:59,116][20347] Updated weights for policy 0, policy_version 1542527 (0.0008) [2023-12-27 16:19:59,169][20347] Updated weights for policy 0, policy_version 1542537 (0.0007) [2023-12-27 16:19:59,538][20381] Updated weights for policy 1, policy_version 1540941 (0.0008) [2023-12-27 16:19:59,590][20381] Updated weights for policy 1, policy_version 1540951 (0.0008) [2023-12-27 16:19:59,640][20381] Updated weights for policy 1, policy_version 1540961 (0.0008) [2023-12-27 16:19:59,926][20347] Updated weights for policy 0, policy_version 1542547 (0.0009) [2023-12-27 16:19:59,988][20347] Updated weights for policy 0, policy_version 1542557 (0.0011) [2023-12-27 16:20:00,047][20347] Updated weights for policy 0, policy_version 1542567 (0.0011) [2023-12-27 16:20:00,313][20381] Updated weights for policy 1, policy_version 1540971 (0.0007) [2023-12-27 16:20:00,371][20381] Updated weights for policy 1, policy_version 1540981 (0.0008) [2023-12-27 16:20:00,422][20381] Updated weights for policy 1, policy_version 1540991 (0.0007) [2023-12-27 16:20:00,682][20347] Updated weights for policy 0, policy_version 1542577 (0.0010) [2023-12-27 16:20:00,737][20347] Updated weights for policy 0, policy_version 1542587 (0.0005) [2023-12-27 16:20:00,790][20347] Updated weights for policy 0, policy_version 1542597 (0.0010) [2023-12-27 16:20:00,850][20347] Updated weights for policy 0, policy_version 1542607 (0.0010) [2023-12-27 16:20:01,053][20381] Updated weights for policy 1, policy_version 1541001 (0.0008) [2023-12-27 16:20:01,110][20381] Updated weights for policy 1, policy_version 1541011 (0.0007) [2023-12-27 16:20:01,175][20381] Updated weights for policy 1, policy_version 1541021 (0.0008) [2023-12-27 16:20:01,234][20381] Updated weights for policy 1, policy_version 1541031 (0.0008) [2023-12-27 16:20:01,576][20347] Updated weights for policy 0, policy_version 1542617 (0.0010) [2023-12-27 16:20:01,638][20347] Updated weights for policy 0, policy_version 1542627 (0.0010) [2023-12-27 16:20:01,699][20347] Updated weights for policy 0, policy_version 1542637 (0.0009) [2023-12-27 16:20:02,001][20381] Updated weights for policy 1, policy_version 1541041 (0.0009) [2023-12-27 16:20:02,064][20381] Updated weights for policy 1, policy_version 1541051 (0.0008) [2023-12-27 16:20:02,129][20381] Updated weights for policy 1, policy_version 1541061 (0.0010) [2023-12-27 16:20:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 789544960. Throughput: 0: 10346.3, 1: 10333.4. Samples: 789557496. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:20:02,363][19170] Avg episode reward: [(0, '668.780'), (1, '699.760')] [2023-12-27 16:20:02,371][20347] Updated weights for policy 0, policy_version 1542647 (0.0007) [2023-12-27 16:20:02,427][20347] Updated weights for policy 0, policy_version 1542657 (0.0006) [2023-12-27 16:20:02,483][20347] Updated weights for policy 0, policy_version 1542667 (0.0006) [2023-12-27 16:20:02,756][20381] Updated weights for policy 1, policy_version 1541071 (0.0007) [2023-12-27 16:20:02,819][20381] Updated weights for policy 1, policy_version 1541081 (0.0005) [2023-12-27 16:20:02,876][20381] Updated weights for policy 1, policy_version 1541091 (0.0005) [2023-12-27 16:20:03,147][20347] Updated weights for policy 0, policy_version 1542677 (0.0009) [2023-12-27 16:20:03,193][20347] Updated weights for policy 0, policy_version 1542687 (0.0008) [2023-12-27 16:20:03,248][20347] Updated weights for policy 0, policy_version 1542697 (0.0005) [2023-12-27 16:20:03,504][20381] Updated weights for policy 1, policy_version 1541101 (0.0007) [2023-12-27 16:20:03,566][20381] Updated weights for policy 1, policy_version 1541111 (0.0007) [2023-12-27 16:20:03,626][20381] Updated weights for policy 1, policy_version 1541121 (0.0007) [2023-12-27 16:20:03,834][20347] Updated weights for policy 0, policy_version 1542707 (0.0005) [2023-12-27 16:20:03,893][20347] Updated weights for policy 0, policy_version 1542717 (0.0009) [2023-12-27 16:20:03,962][20347] Updated weights for policy 0, policy_version 1542727 (0.0009) [2023-12-27 16:20:04,377][20381] Updated weights for policy 1, policy_version 1541131 (0.0009) [2023-12-27 16:20:04,433][20381] Updated weights for policy 1, policy_version 1541141 (0.0006) [2023-12-27 16:20:04,492][20381] Updated weights for policy 1, policy_version 1541151 (0.0011) [2023-12-27 16:20:04,590][20347] Updated weights for policy 0, policy_version 1542737 (0.0009) [2023-12-27 16:20:04,651][20347] Updated weights for policy 0, policy_version 1542747 (0.0005) [2023-12-27 16:20:04,652][20291] KL-divergence is very high: 149.0771 [2023-12-27 16:20:04,702][20291] KL-divergence is very high: 107.4995 [2023-12-27 16:20:04,715][20347] Updated weights for policy 0, policy_version 1542757 (0.0005) [2023-12-27 16:20:04,772][20347] Updated weights for policy 0, policy_version 1542767 (0.0005) [2023-12-27 16:20:05,176][20381] Updated weights for policy 1, policy_version 1541161 (0.0010) [2023-12-27 16:20:05,240][20381] Updated weights for policy 1, policy_version 1541171 (0.0005) [2023-12-27 16:20:05,297][20381] Updated weights for policy 1, policy_version 1541181 (0.0006) [2023-12-27 16:20:05,352][20381] Updated weights for policy 1, policy_version 1541191 (0.0010) [2023-12-27 16:20:05,365][20347] Updated weights for policy 0, policy_version 1542777 (0.0010) [2023-12-27 16:20:05,383][20291] KL-divergence is very high: 107.1796 [2023-12-27 16:20:05,406][20291] KL-divergence is very high: 122.3323 [2023-12-27 16:20:05,423][20347] Updated weights for policy 0, policy_version 1542787 (0.0010) [2023-12-27 16:20:05,429][20291] KL-divergence is very high: 167.1447 [2023-12-27 16:20:05,451][20291] KL-divergence is very high: 107.3641 [2023-12-27 16:20:05,475][20291] KL-divergence is very high: 114.2548 [2023-12-27 16:20:05,481][20347] Updated weights for policy 0, policy_version 1542797 (0.0010) [2023-12-27 16:20:05,910][20381] Updated weights for policy 1, policy_version 1541201 (0.0010) [2023-12-27 16:20:05,968][20381] Updated weights for policy 1, policy_version 1541211 (0.0007) [2023-12-27 16:20:06,023][20381] Updated weights for policy 1, policy_version 1541221 (0.0007) [2023-12-27 16:20:06,130][20347] Updated weights for policy 0, policy_version 1542807 (0.0008) [2023-12-27 16:20:06,187][20347] Updated weights for policy 0, policy_version 1542817 (0.0006) [2023-12-27 16:20:06,244][20347] Updated weights for policy 0, policy_version 1542827 (0.0007) [2023-12-27 16:20:06,759][20381] Updated weights for policy 1, policy_version 1541231 (0.0006) [2023-12-27 16:20:06,824][20381] Updated weights for policy 1, policy_version 1541241 (0.0006) [2023-12-27 16:20:06,887][20381] Updated weights for policy 1, policy_version 1541251 (0.0005) [2023-12-27 16:20:06,936][20347] Updated weights for policy 0, policy_version 1542837 (0.0007) [2023-12-27 16:20:06,992][20347] Updated weights for policy 0, policy_version 1542847 (0.0008) [2023-12-27 16:20:07,054][20347] Updated weights for policy 0, policy_version 1542857 (0.0007) [2023-12-27 16:20:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 789651456. Throughput: 0: 10384.4, 1: 10326.6. Samples: 789620972. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:20:07,363][19170] Avg episode reward: [(0, '698.490'), (1, '720.370')] [2023-12-27 16:20:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001542864_395051008.pth... [2023-12-27 16:20:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001541256_394600448.pth... [2023-12-27 16:20:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001540072_394297344.pth [2023-12-27 16:20:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001541648_394739712.pth [2023-12-27 16:20:07,429][20381] Updated weights for policy 1, policy_version 1541261 (0.0006) [2023-12-27 16:20:07,481][20381] Updated weights for policy 1, policy_version 1541271 (0.0008) [2023-12-27 16:20:07,540][20381] Updated weights for policy 1, policy_version 1541281 (0.0007) [2023-12-27 16:20:07,764][20347] Updated weights for policy 0, policy_version 1542867 (0.0009) [2023-12-27 16:20:07,824][20347] Updated weights for policy 0, policy_version 1542877 (0.0007) [2023-12-27 16:20:07,884][20347] Updated weights for policy 0, policy_version 1542887 (0.0007) [2023-12-27 16:20:08,173][20381] Updated weights for policy 1, policy_version 1541291 (0.0007) [2023-12-27 16:20:08,230][20381] Updated weights for policy 1, policy_version 1541301 (0.0005) [2023-12-27 16:20:08,285][20381] Updated weights for policy 1, policy_version 1541311 (0.0005) [2023-12-27 16:20:08,586][20347] Updated weights for policy 0, policy_version 1542897 (0.0007) [2023-12-27 16:20:08,645][20347] Updated weights for policy 0, policy_version 1542907 (0.0005) [2023-12-27 16:20:08,695][20347] Updated weights for policy 0, policy_version 1542917 (0.0006) [2023-12-27 16:20:08,756][20347] Updated weights for policy 0, policy_version 1542927 (0.0007) [2023-12-27 16:20:08,929][20381] Updated weights for policy 1, policy_version 1541321 (0.0006) [2023-12-27 16:20:08,987][20381] Updated weights for policy 1, policy_version 1541331 (0.0005) [2023-12-27 16:20:09,036][20381] Updated weights for policy 1, policy_version 1541341 (0.0005) [2023-12-27 16:20:09,088][20381] Updated weights for policy 1, policy_version 1541351 (0.0005) [2023-12-27 16:20:09,439][20347] Updated weights for policy 0, policy_version 1542937 (0.0007) [2023-12-27 16:20:09,493][20347] Updated weights for policy 0, policy_version 1542947 (0.0005) [2023-12-27 16:20:09,550][20347] Updated weights for policy 0, policy_version 1542957 (0.0005) [2023-12-27 16:20:09,817][20381] Updated weights for policy 1, policy_version 1541361 (0.0007) [2023-12-27 16:20:09,882][20381] Updated weights for policy 1, policy_version 1541371 (0.0008) [2023-12-27 16:20:09,943][20381] Updated weights for policy 1, policy_version 1541381 (0.0008) [2023-12-27 16:20:10,195][20347] Updated weights for policy 0, policy_version 1542967 (0.0008) [2023-12-27 16:20:10,257][20347] Updated weights for policy 0, policy_version 1542977 (0.0007) [2023-12-27 16:20:10,323][20347] Updated weights for policy 0, policy_version 1542987 (0.0006) [2023-12-27 16:20:10,526][20381] Updated weights for policy 1, policy_version 1541391 (0.0007) [2023-12-27 16:20:10,579][20381] Updated weights for policy 1, policy_version 1541401 (0.0008) [2023-12-27 16:20:10,631][20381] Updated weights for policy 1, policy_version 1541411 (0.0008) [2023-12-27 16:20:10,896][20347] Updated weights for policy 0, policy_version 1542997 (0.0009) [2023-12-27 16:20:10,951][20347] Updated weights for policy 0, policy_version 1543007 (0.0007) [2023-12-27 16:20:11,007][20347] Updated weights for policy 0, policy_version 1543017 (0.0007) [2023-12-27 16:20:11,384][20381] Updated weights for policy 1, policy_version 1541421 (0.0007) [2023-12-27 16:20:11,447][20381] Updated weights for policy 1, policy_version 1541431 (0.0007) [2023-12-27 16:20:11,508][20381] Updated weights for policy 1, policy_version 1541441 (0.0007) [2023-12-27 16:20:11,781][20347] Updated weights for policy 0, policy_version 1543027 (0.0008) [2023-12-27 16:20:11,839][20347] Updated weights for policy 0, policy_version 1543037 (0.0008) [2023-12-27 16:20:11,889][20347] Updated weights for policy 0, policy_version 1543047 (0.0008) [2023-12-27 16:20:12,154][20381] Updated weights for policy 1, policy_version 1541451 (0.0006) [2023-12-27 16:20:12,208][20381] Updated weights for policy 1, policy_version 1541461 (0.0007) [2023-12-27 16:20:12,267][20381] Updated weights for policy 1, policy_version 1541471 (0.0007) [2023-12-27 16:20:12,366][19170] Fps is (10 sec: 21291.9, 60 sec: 20751.9, 300 sec: 20577.0). Total num frames: 789757952. Throughput: 0: 10358.5, 1: 10337.9. Samples: 789745920. Policy #0 lag: (min: 31.0, avg: 42.4, max: 63.0) [2023-12-27 16:20:12,367][19170] Avg episode reward: [(0, '640.630'), (1, '781.530')] [2023-12-27 16:20:12,615][20347] Updated weights for policy 0, policy_version 1543057 (0.0008) [2023-12-27 16:20:12,676][20347] Updated weights for policy 0, policy_version 1543067 (0.0006) [2023-12-27 16:20:12,734][20347] Updated weights for policy 0, policy_version 1543077 (0.0007) [2023-12-27 16:20:12,794][20347] Updated weights for policy 0, policy_version 1543087 (0.0010) [2023-12-27 16:20:12,976][20381] Updated weights for policy 1, policy_version 1541481 (0.0007) [2023-12-27 16:20:13,033][20381] Updated weights for policy 1, policy_version 1541491 (0.0007) [2023-12-27 16:20:13,093][20381] Updated weights for policy 1, policy_version 1541501 (0.0007) [2023-12-27 16:20:13,152][20381] Updated weights for policy 1, policy_version 1541511 (0.0008) [2023-12-27 16:20:13,374][20347] Updated weights for policy 0, policy_version 1543097 (0.0006) [2023-12-27 16:20:13,434][20347] Updated weights for policy 0, policy_version 1543107 (0.0006) [2023-12-27 16:20:13,499][20347] Updated weights for policy 0, policy_version 1543117 (0.0007) [2023-12-27 16:20:13,833][20381] Updated weights for policy 1, policy_version 1541521 (0.0008) [2023-12-27 16:20:13,885][20381] Updated weights for policy 1, policy_version 1541531 (0.0007) [2023-12-27 16:20:13,943][20381] Updated weights for policy 1, policy_version 1541541 (0.0006) [2023-12-27 16:20:14,133][20347] Updated weights for policy 0, policy_version 1543127 (0.0008) [2023-12-27 16:20:14,179][20347] Updated weights for policy 0, policy_version 1543137 (0.0008) [2023-12-27 16:20:14,233][20347] Updated weights for policy 0, policy_version 1543147 (0.0007) [2023-12-27 16:20:14,616][20381] Updated weights for policy 1, policy_version 1541551 (0.0005) [2023-12-27 16:20:14,671][20381] Updated weights for policy 1, policy_version 1541561 (0.0007) [2023-12-27 16:20:14,731][20381] Updated weights for policy 1, policy_version 1541571 (0.0006) [2023-12-27 16:20:14,986][20347] Updated weights for policy 0, policy_version 1543157 (0.0007) [2023-12-27 16:20:15,044][20347] Updated weights for policy 0, policy_version 1543167 (0.0007) [2023-12-27 16:20:15,106][20347] Updated weights for policy 0, policy_version 1543177 (0.0008) [2023-12-27 16:20:15,380][20381] Updated weights for policy 1, policy_version 1541581 (0.0008) [2023-12-27 16:20:15,459][20381] Updated weights for policy 1, policy_version 1541591 (0.0010) [2023-12-27 16:20:15,516][20381] Updated weights for policy 1, policy_version 1541601 (0.0008) [2023-12-27 16:20:15,836][20347] Updated weights for policy 0, policy_version 1543187 (0.0008) [2023-12-27 16:20:15,893][20347] Updated weights for policy 0, policy_version 1543197 (0.0007) [2023-12-27 16:20:15,947][20347] Updated weights for policy 0, policy_version 1543207 (0.0007) [2023-12-27 16:20:16,139][20381] Updated weights for policy 1, policy_version 1541611 (0.0007) [2023-12-27 16:20:16,184][20381] Updated weights for policy 1, policy_version 1541621 (0.0010) [2023-12-27 16:20:16,237][20381] Updated weights for policy 1, policy_version 1541631 (0.0010) [2023-12-27 16:20:16,544][20347] Updated weights for policy 0, policy_version 1543217 (0.0006) [2023-12-27 16:20:16,611][20347] Updated weights for policy 0, policy_version 1543227 (0.0005) [2023-12-27 16:20:16,663][20347] Updated weights for policy 0, policy_version 1543237 (0.0005) [2023-12-27 16:20:16,711][20347] Updated weights for policy 0, policy_version 1543247 (0.0005) [2023-12-27 16:20:16,927][20381] Updated weights for policy 1, policy_version 1541641 (0.0010) [2023-12-27 16:20:16,980][20381] Updated weights for policy 1, policy_version 1541651 (0.0011) [2023-12-27 16:20:17,036][20381] Updated weights for policy 1, policy_version 1541661 (0.0011) [2023-12-27 16:20:17,091][20381] Updated weights for policy 1, policy_version 1541671 (0.0011) [2023-12-27 16:20:17,264][20347] Updated weights for policy 0, policy_version 1543257 (0.0007) [2023-12-27 16:20:17,315][20347] Updated weights for policy 0, policy_version 1543267 (0.0008) [2023-12-27 16:20:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 789856256. Throughput: 0: 10379.8, 1: 10302.8. Samples: 789869364. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:17,363][19170] Avg episode reward: [(0, '726.050'), (1, '793.770')] [2023-12-27 16:20:17,367][20347] Updated weights for policy 0, policy_version 1543277 (0.0007) [2023-12-27 16:20:17,787][20381] Updated weights for policy 1, policy_version 1541681 (0.0011) [2023-12-27 16:20:17,839][20381] Updated weights for policy 1, policy_version 1541691 (0.0010) [2023-12-27 16:20:17,893][20381] Updated weights for policy 1, policy_version 1541701 (0.0007) [2023-12-27 16:20:17,960][20347] Updated weights for policy 0, policy_version 1543287 (0.0008) [2023-12-27 16:20:18,014][20347] Updated weights for policy 0, policy_version 1543297 (0.0007) [2023-12-27 16:20:18,072][20347] Updated weights for policy 0, policy_version 1543307 (0.0007) [2023-12-27 16:20:18,590][20381] Updated weights for policy 1, policy_version 1541711 (0.0010) [2023-12-27 16:20:18,643][20381] Updated weights for policy 1, policy_version 1541721 (0.0011) [2023-12-27 16:20:18,702][20381] Updated weights for policy 1, policy_version 1541731 (0.0011) [2023-12-27 16:20:18,795][20347] Updated weights for policy 0, policy_version 1543317 (0.0008) [2023-12-27 16:20:18,851][20347] Updated weights for policy 0, policy_version 1543327 (0.0007) [2023-12-27 16:20:18,909][20347] Updated weights for policy 0, policy_version 1543337 (0.0007) [2023-12-27 16:20:19,433][20381] Updated weights for policy 1, policy_version 1541741 (0.0008) [2023-12-27 16:20:19,486][20381] Updated weights for policy 1, policy_version 1541751 (0.0008) [2023-12-27 16:20:19,547][20381] Updated weights for policy 1, policy_version 1541761 (0.0008) [2023-12-27 16:20:19,641][20347] Updated weights for policy 0, policy_version 1543347 (0.0008) [2023-12-27 16:20:19,698][20347] Updated weights for policy 0, policy_version 1543357 (0.0007) [2023-12-27 16:20:19,751][20347] Updated weights for policy 0, policy_version 1543367 (0.0006) [2023-12-27 16:20:20,280][20381] Updated weights for policy 1, policy_version 1541771 (0.0006) [2023-12-27 16:20:20,333][20381] Updated weights for policy 1, policy_version 1541781 (0.0008) [2023-12-27 16:20:20,385][20381] Updated weights for policy 1, policy_version 1541791 (0.0007) [2023-12-27 16:20:20,511][20347] Updated weights for policy 0, policy_version 1543377 (0.0007) [2023-12-27 16:20:20,563][20347] Updated weights for policy 0, policy_version 1543387 (0.0006) [2023-12-27 16:20:20,625][20347] Updated weights for policy 0, policy_version 1543397 (0.0006) [2023-12-27 16:20:20,687][20347] Updated weights for policy 0, policy_version 1543407 (0.0005) [2023-12-27 16:20:21,142][20381] Updated weights for policy 1, policy_version 1541801 (0.0007) [2023-12-27 16:20:21,199][20381] Updated weights for policy 1, policy_version 1541811 (0.0011) [2023-12-27 16:20:21,260][20381] Updated weights for policy 1, policy_version 1541821 (0.0011) [2023-12-27 16:20:21,297][20347] Updated weights for policy 0, policy_version 1543417 (0.0009) [2023-12-27 16:20:21,315][20381] Updated weights for policy 1, policy_version 1541831 (0.0011) [2023-12-27 16:20:21,368][20347] Updated weights for policy 0, policy_version 1543427 (0.0008) [2023-12-27 16:20:21,428][20347] Updated weights for policy 0, policy_version 1543437 (0.0008) [2023-12-27 16:20:22,058][20381] Updated weights for policy 1, policy_version 1541841 (0.0011) [2023-12-27 16:20:22,112][20381] Updated weights for policy 1, policy_version 1541851 (0.0011) [2023-12-27 16:20:22,155][20347] Updated weights for policy 0, policy_version 1543447 (0.0007) [2023-12-27 16:20:22,173][20381] Updated weights for policy 1, policy_version 1541861 (0.0006) [2023-12-27 16:20:22,215][20347] Updated weights for policy 0, policy_version 1543457 (0.0007) [2023-12-27 16:20:22,280][20347] Updated weights for policy 0, policy_version 1543467 (0.0010) [2023-12-27 16:20:22,362][19170] Fps is (10 sec: 20486.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 789962752. Throughput: 0: 10346.2, 1: 10273.6. Samples: 789928800. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:22,365][19170] Avg episode reward: [(0, '716.390'), (1, '750.410')] [2023-12-27 16:20:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001541864_394756096.pth... [2023-12-27 16:20:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001543472_395206656.pth... [2023-12-27 16:20:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001540648_394444800.pth [2023-12-27 16:20:22,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001542256_394895360.pth [2023-12-27 16:20:22,902][20381] Updated weights for policy 1, policy_version 1541871 (0.0008) [2023-12-27 16:20:22,953][20347] Updated weights for policy 0, policy_version 1543477 (0.0009) [2023-12-27 16:20:22,964][20381] Updated weights for policy 1, policy_version 1541881 (0.0007) [2023-12-27 16:20:23,012][20347] Updated weights for policy 0, policy_version 1543487 (0.0007) [2023-12-27 16:20:23,014][20381] Updated weights for policy 1, policy_version 1541891 (0.0011) [2023-12-27 16:20:23,067][20347] Updated weights for policy 0, policy_version 1543497 (0.0006) [2023-12-27 16:20:23,634][20381] Updated weights for policy 1, policy_version 1541901 (0.0008) [2023-12-27 16:20:23,684][20381] Updated weights for policy 1, policy_version 1541911 (0.0008) [2023-12-27 16:20:23,733][20381] Updated weights for policy 1, policy_version 1541921 (0.0010) [2023-12-27 16:20:23,752][20347] Updated weights for policy 0, policy_version 1543507 (0.0007) [2023-12-27 16:20:23,810][20347] Updated weights for policy 0, policy_version 1543517 (0.0008) [2023-12-27 16:20:23,867][20347] Updated weights for policy 0, policy_version 1543527 (0.0005) [2023-12-27 16:20:24,324][20381] Updated weights for policy 1, policy_version 1541931 (0.0006) [2023-12-27 16:20:24,373][20381] Updated weights for policy 1, policy_version 1541941 (0.0010) [2023-12-27 16:20:24,432][20381] Updated weights for policy 1, policy_version 1541951 (0.0006) [2023-12-27 16:20:24,498][20347] Updated weights for policy 0, policy_version 1543537 (0.0008) [2023-12-27 16:20:24,551][20347] Updated weights for policy 0, policy_version 1543547 (0.0010) [2023-12-27 16:20:24,614][20347] Updated weights for policy 0, policy_version 1543557 (0.0006) [2023-12-27 16:20:24,637][20291] KL-divergence is very high: 360.0819 [2023-12-27 16:20:24,644][20291] KL-divergence is very high: 385.7635 [2023-12-27 16:20:24,672][20347] Updated weights for policy 0, policy_version 1543567 (0.0009) [2023-12-27 16:20:25,062][20381] Updated weights for policy 1, policy_version 1541961 (0.0007) [2023-12-27 16:20:25,109][20381] Updated weights for policy 1, policy_version 1541971 (0.0007) [2023-12-27 16:20:25,156][20381] Updated weights for policy 1, policy_version 1541981 (0.0006) [2023-12-27 16:20:25,209][20381] Updated weights for policy 1, policy_version 1541991 (0.0005) [2023-12-27 16:20:25,246][20291] KL-divergence is very high: 364.3062 [2023-12-27 16:20:25,291][20291] KL-divergence is very high: 341.1181 [2023-12-27 16:20:25,297][20347] Updated weights for policy 0, policy_version 1543577 (0.0010) [2023-12-27 16:20:25,334][20291] KL-divergence is very high: 318.6785 [2023-12-27 16:20:25,348][20347] Updated weights for policy 0, policy_version 1543587 (0.0010) [2023-12-27 16:20:25,374][20291] KL-divergence is very high: 300.4306 [2023-12-27 16:20:25,398][20347] Updated weights for policy 0, policy_version 1543597 (0.0010) [2023-12-27 16:20:25,850][20381] Updated weights for policy 1, policy_version 1542001 (0.0008) [2023-12-27 16:20:25,914][20381] Updated weights for policy 1, policy_version 1542011 (0.0007) [2023-12-27 16:20:25,962][20381] Updated weights for policy 1, policy_version 1542021 (0.0007) [2023-12-27 16:20:26,075][20347] Updated weights for policy 0, policy_version 1543607 (0.0010) [2023-12-27 16:20:26,127][20347] Updated weights for policy 0, policy_version 1543617 (0.0010) [2023-12-27 16:20:26,178][20347] Updated weights for policy 0, policy_version 1543627 (0.0010) [2023-12-27 16:20:26,588][20381] Updated weights for policy 1, policy_version 1542031 (0.0006) [2023-12-27 16:20:26,638][20381] Updated weights for policy 1, policy_version 1542041 (0.0005) [2023-12-27 16:20:26,686][20381] Updated weights for policy 1, policy_version 1542051 (0.0010) [2023-12-27 16:20:26,814][20347] Updated weights for policy 0, policy_version 1543637 (0.0010) [2023-12-27 16:20:26,866][20347] Updated weights for policy 0, policy_version 1543647 (0.0010) [2023-12-27 16:20:26,920][20347] Updated weights for policy 0, policy_version 1543657 (0.0010) [2023-12-27 16:20:27,218][20381] Updated weights for policy 1, policy_version 1542061 (0.0006) [2023-12-27 16:20:27,260][20381] Updated weights for policy 1, policy_version 1542071 (0.0005) [2023-12-27 16:20:27,306][20381] Updated weights for policy 1, policy_version 1542081 (0.0008) [2023-12-27 16:20:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 790069248. Throughput: 0: 10345.6, 1: 10297.3. Samples: 790052184. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:27,363][19170] Avg episode reward: [(0, '774.400'), (1, '617.960')] [2023-12-27 16:20:27,613][20347] Updated weights for policy 0, policy_version 1543667 (0.0009) [2023-12-27 16:20:27,664][20347] Updated weights for policy 0, policy_version 1543677 (0.0005) [2023-12-27 16:20:27,730][20347] Updated weights for policy 0, policy_version 1543687 (0.0011) [2023-12-27 16:20:27,952][20381] Updated weights for policy 1, policy_version 1542091 (0.0009) [2023-12-27 16:20:28,007][20381] Updated weights for policy 1, policy_version 1542101 (0.0007) [2023-12-27 16:20:28,058][20381] Updated weights for policy 1, policy_version 1542111 (0.0007) [2023-12-27 16:20:28,350][20347] Updated weights for policy 0, policy_version 1543697 (0.0010) [2023-12-27 16:20:28,411][20347] Updated weights for policy 0, policy_version 1543707 (0.0011) [2023-12-27 16:20:28,472][20347] Updated weights for policy 0, policy_version 1543717 (0.0011) [2023-12-27 16:20:28,524][20347] Updated weights for policy 0, policy_version 1543727 (0.0010) [2023-12-27 16:20:28,753][20381] Updated weights for policy 1, policy_version 1542121 (0.0007) [2023-12-27 16:20:28,806][20381] Updated weights for policy 1, policy_version 1542131 (0.0009) [2023-12-27 16:20:28,859][20381] Updated weights for policy 1, policy_version 1542141 (0.0009) [2023-12-27 16:20:28,907][20381] Updated weights for policy 1, policy_version 1542152 (0.0009) [2023-12-27 16:20:29,277][20347] Updated weights for policy 0, policy_version 1543737 (0.0009) [2023-12-27 16:20:29,331][20347] Updated weights for policy 0, policy_version 1543747 (0.0007) [2023-12-27 16:20:29,396][20347] Updated weights for policy 0, policy_version 1543757 (0.0006) [2023-12-27 16:20:29,574][20381] Updated weights for policy 1, policy_version 1542162 (0.0005) [2023-12-27 16:20:29,629][20381] Updated weights for policy 1, policy_version 1542172 (0.0005) [2023-12-27 16:20:29,683][20381] Updated weights for policy 1, policy_version 1542182 (0.0005) [2023-12-27 16:20:30,077][20347] Updated weights for policy 0, policy_version 1543767 (0.0008) [2023-12-27 16:20:30,133][20347] Updated weights for policy 0, policy_version 1543777 (0.0009) [2023-12-27 16:20:30,182][20347] Updated weights for policy 0, policy_version 1543787 (0.0005) [2023-12-27 16:20:30,300][20381] Updated weights for policy 1, policy_version 1542192 (0.0008) [2023-12-27 16:20:30,360][20381] Updated weights for policy 1, policy_version 1542202 (0.0009) [2023-12-27 16:20:30,433][20381] Updated weights for policy 1, policy_version 1542212 (0.0008) [2023-12-27 16:20:30,793][20347] Updated weights for policy 0, policy_version 1543797 (0.0007) [2023-12-27 16:20:30,838][20291] KL-divergence is very high: 124.1266 [2023-12-27 16:20:30,841][20347] Updated weights for policy 0, policy_version 1543807 (0.0005) [2023-12-27 16:20:30,843][20291] KL-divergence is very high: 121.8240 [2023-12-27 16:20:30,895][20347] Updated weights for policy 0, policy_version 1543817 (0.0005) [2023-12-27 16:20:31,116][20381] Updated weights for policy 1, policy_version 1542222 (0.0009) [2023-12-27 16:20:31,181][20381] Updated weights for policy 1, policy_version 1542232 (0.0011) [2023-12-27 16:20:31,240][20381] Updated weights for policy 1, policy_version 1542242 (0.0010) [2023-12-27 16:20:31,523][20347] Updated weights for policy 0, policy_version 1543827 (0.0005) [2023-12-27 16:20:31,577][20291] KL-divergence is very high: 152.4591 [2023-12-27 16:20:31,581][20347] Updated weights for policy 0, policy_version 1543837 (0.0005) [2023-12-27 16:20:31,627][20291] KL-divergence is very high: 124.2492 [2023-12-27 16:20:31,647][20347] Updated weights for policy 0, policy_version 1543847 (0.0008) [2023-12-27 16:20:31,671][20291] KL-divergence is very high: 108.8807 [2023-12-27 16:20:31,966][20381] Updated weights for policy 1, policy_version 1542252 (0.0009) [2023-12-27 16:20:32,021][20381] Updated weights for policy 1, policy_version 1542262 (0.0007) [2023-12-27 16:20:32,070][20381] Updated weights for policy 1, policy_version 1542272 (0.0008) [2023-12-27 16:20:32,298][20347] Updated weights for policy 0, policy_version 1543857 (0.0006) [2023-12-27 16:20:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 790167552. Throughput: 0: 10308.8, 1: 10383.9. Samples: 790178552. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:32,363][19170] Avg episode reward: [(0, '719.730'), (1, '711.150')] [2023-12-27 16:20:32,367][20347] Updated weights for policy 0, policy_version 1543867 (0.0006) [2023-12-27 16:20:32,427][20347] Updated weights for policy 0, policy_version 1543877 (0.0005) [2023-12-27 16:20:32,486][20347] Updated weights for policy 0, policy_version 1543887 (0.0006) [2023-12-27 16:20:32,769][20381] Updated weights for policy 1, policy_version 1542282 (0.0007) [2023-12-27 16:20:32,821][20381] Updated weights for policy 1, policy_version 1542292 (0.0005) [2023-12-27 16:20:32,872][20381] Updated weights for policy 1, policy_version 1542302 (0.0005) [2023-12-27 16:20:32,929][20381] Updated weights for policy 1, policy_version 1542312 (0.0005) [2023-12-27 16:20:33,157][20347] Updated weights for policy 0, policy_version 1543897 (0.0005) [2023-12-27 16:20:33,212][20347] Updated weights for policy 0, policy_version 1543907 (0.0009) [2023-12-27 16:20:33,256][20347] Updated weights for policy 0, policy_version 1543917 (0.0010) [2023-12-27 16:20:33,473][20381] Updated weights for policy 1, policy_version 1542322 (0.0007) [2023-12-27 16:20:33,525][20381] Updated weights for policy 1, policy_version 1542332 (0.0009) [2023-12-27 16:20:33,575][20381] Updated weights for policy 1, policy_version 1542342 (0.0008) [2023-12-27 16:20:33,812][20347] Updated weights for policy 0, policy_version 1543927 (0.0007) [2023-12-27 16:20:33,862][20347] Updated weights for policy 0, policy_version 1543937 (0.0005) [2023-12-27 16:20:33,909][20347] Updated weights for policy 0, policy_version 1543947 (0.0005) [2023-12-27 16:20:34,173][20381] Updated weights for policy 1, policy_version 1542352 (0.0007) [2023-12-27 16:20:34,234][20381] Updated weights for policy 1, policy_version 1542362 (0.0007) [2023-12-27 16:20:34,287][20381] Updated weights for policy 1, policy_version 1542372 (0.0007) [2023-12-27 16:20:34,575][20347] Updated weights for policy 0, policy_version 1543957 (0.0007) [2023-12-27 16:20:34,633][20347] Updated weights for policy 0, policy_version 1543967 (0.0008) [2023-12-27 16:20:34,690][20347] Updated weights for policy 0, policy_version 1543977 (0.0010) [2023-12-27 16:20:35,022][20381] Updated weights for policy 1, policy_version 1542382 (0.0008) [2023-12-27 16:20:35,074][20381] Updated weights for policy 1, policy_version 1542392 (0.0007) [2023-12-27 16:20:35,118][20381] Updated weights for policy 1, policy_version 1542402 (0.0008) [2023-12-27 16:20:35,321][20347] Updated weights for policy 0, policy_version 1543987 (0.0010) [2023-12-27 16:20:35,378][20347] Updated weights for policy 0, policy_version 1543997 (0.0010) [2023-12-27 16:20:35,379][20291] KL-divergence is very high: 100.9076 [2023-12-27 16:20:35,424][20291] KL-divergence is very high: 148.7443 [2023-12-27 16:20:35,436][20347] Updated weights for policy 0, policy_version 1544007 (0.0010) [2023-12-27 16:20:35,469][20291] KL-divergence is very high: 149.1175 [2023-12-27 16:20:35,766][20381] Updated weights for policy 1, policy_version 1542412 (0.0007) [2023-12-27 16:20:35,821][20381] Updated weights for policy 1, policy_version 1542422 (0.0006) [2023-12-27 16:20:35,871][20381] Updated weights for policy 1, policy_version 1542432 (0.0009) [2023-12-27 16:20:35,999][20347] Updated weights for policy 0, policy_version 1544017 (0.0010) [2023-12-27 16:20:36,048][20347] Updated weights for policy 0, policy_version 1544027 (0.0005) [2023-12-27 16:20:36,076][20291] KL-divergence is very high: 350.7871 [2023-12-27 16:20:36,083][20291] KL-divergence is very high: 123.7998 [2023-12-27 16:20:36,091][20291] KL-divergence is very high: 355.2567 [2023-12-27 16:20:36,110][20347] Updated weights for policy 0, policy_version 1544037 (0.0009) [2023-12-27 16:20:36,130][20291] KL-divergence is very high: 273.8011 [2023-12-27 16:20:36,142][20291] KL-divergence is very high: 200.6379 [2023-12-27 16:20:36,167][20347] Updated weights for policy 0, policy_version 1544047 (0.0009) [2023-12-27 16:20:36,502][20381] Updated weights for policy 1, policy_version 1542442 (0.0006) [2023-12-27 16:20:36,555][20381] Updated weights for policy 1, policy_version 1542452 (0.0005) [2023-12-27 16:20:36,611][20381] Updated weights for policy 1, policy_version 1542462 (0.0006) [2023-12-27 16:20:36,663][20381] Updated weights for policy 1, policy_version 1542472 (0.0007) [2023-12-27 16:20:36,889][20347] Updated weights for policy 0, policy_version 1544057 (0.0008) [2023-12-27 16:20:36,946][20347] Updated weights for policy 0, policy_version 1544067 (0.0008) [2023-12-27 16:20:37,000][20347] Updated weights for policy 0, policy_version 1544077 (0.0007) [2023-12-27 16:20:37,268][20381] Updated weights for policy 1, policy_version 1542482 (0.0005) [2023-12-27 16:20:37,322][20381] Updated weights for policy 1, policy_version 1542492 (0.0005) [2023-12-27 16:20:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 790274048. Throughput: 0: 10333.9, 1: 10422.8. Samples: 790243832. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:37,363][19170] Avg episode reward: [(0, '604.750'), (1, '634.440')] [2023-12-27 16:20:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001544080_395362304.pth... [2023-12-27 16:20:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001542864_395051008.pth [2023-12-27 16:20:37,381][20381] Updated weights for policy 1, policy_version 1542502 (0.0005) [2023-12-27 16:20:37,390][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001542504_394919936.pth... [2023-12-27 16:20:37,393][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001541256_394600448.pth [2023-12-27 16:20:37,669][20347] Updated weights for policy 0, policy_version 1544087 (0.0010) [2023-12-27 16:20:37,729][20347] Updated weights for policy 0, policy_version 1544097 (0.0011) [2023-12-27 16:20:37,792][20347] Updated weights for policy 0, policy_version 1544107 (0.0010) [2023-12-27 16:20:37,935][20381] Updated weights for policy 1, policy_version 1542512 (0.0005) [2023-12-27 16:20:37,986][20381] Updated weights for policy 1, policy_version 1542522 (0.0007) [2023-12-27 16:20:38,038][20381] Updated weights for policy 1, policy_version 1542532 (0.0008) [2023-12-27 16:20:38,499][20347] Updated weights for policy 0, policy_version 1544117 (0.0008) [2023-12-27 16:20:38,556][20347] Updated weights for policy 0, policy_version 1544127 (0.0008) [2023-12-27 16:20:38,614][20347] Updated weights for policy 0, policy_version 1544137 (0.0009) [2023-12-27 16:20:38,677][20381] Updated weights for policy 1, policy_version 1542542 (0.0006) [2023-12-27 16:20:38,733][20381] Updated weights for policy 1, policy_version 1542552 (0.0005) [2023-12-27 16:20:38,796][20381] Updated weights for policy 1, policy_version 1542562 (0.0006) [2023-12-27 16:20:39,295][20347] Updated weights for policy 0, policy_version 1544147 (0.0011) [2023-12-27 16:20:39,380][20381] Updated weights for policy 1, policy_version 1542572 (0.0007) [2023-12-27 16:20:39,387][20347] Updated weights for policy 0, policy_version 1544157 (0.0009) [2023-12-27 16:20:39,443][20381] Updated weights for policy 1, policy_version 1542582 (0.0009) [2023-12-27 16:20:39,458][20347] Updated weights for policy 0, policy_version 1544167 (0.0008) [2023-12-27 16:20:39,500][20381] Updated weights for policy 1, policy_version 1542592 (0.0006) [2023-12-27 16:20:40,117][20347] Updated weights for policy 0, policy_version 1544177 (0.0008) [2023-12-27 16:20:40,182][20347] Updated weights for policy 0, policy_version 1544187 (0.0008) [2023-12-27 16:20:40,207][20381] Updated weights for policy 1, policy_version 1542602 (0.0007) [2023-12-27 16:20:40,240][20347] Updated weights for policy 0, policy_version 1544197 (0.0006) [2023-12-27 16:20:40,265][20381] Updated weights for policy 1, policy_version 1542612 (0.0007) [2023-12-27 16:20:40,294][20347] Updated weights for policy 0, policy_version 1544207 (0.0006) [2023-12-27 16:20:40,322][20381] Updated weights for policy 1, policy_version 1542622 (0.0007) [2023-12-27 16:20:40,383][20381] Updated weights for policy 1, policy_version 1542632 (0.0008) [2023-12-27 16:20:41,021][20347] Updated weights for policy 0, policy_version 1544217 (0.0010) [2023-12-27 16:20:41,052][20381] Updated weights for policy 1, policy_version 1542642 (0.0006) [2023-12-27 16:20:41,083][20347] Updated weights for policy 0, policy_version 1544227 (0.0009) [2023-12-27 16:20:41,114][20381] Updated weights for policy 1, policy_version 1542652 (0.0007) [2023-12-27 16:20:41,146][20347] Updated weights for policy 0, policy_version 1544237 (0.0008) [2023-12-27 16:20:41,178][20381] Updated weights for policy 1, policy_version 1542662 (0.0007) [2023-12-27 16:20:41,886][20347] Updated weights for policy 0, policy_version 1544247 (0.0007) [2023-12-27 16:20:41,933][20381] Updated weights for policy 1, policy_version 1542672 (0.0009) [2023-12-27 16:20:41,947][20347] Updated weights for policy 0, policy_version 1544257 (0.0006) [2023-12-27 16:20:41,993][20381] Updated weights for policy 1, policy_version 1542682 (0.0011) [2023-12-27 16:20:42,032][20347] Updated weights for policy 0, policy_version 1544267 (0.0005) [2023-12-27 16:20:42,055][20381] Updated weights for policy 1, policy_version 1542692 (0.0010) [2023-12-27 16:20:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 790380544. Throughput: 0: 10345.7, 1: 10481.4. Samples: 790369508. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:42,363][19170] Avg episode reward: [(0, '588.050'), (1, '644.580')] [2023-12-27 16:20:42,674][20347] Updated weights for policy 0, policy_version 1544277 (0.0006) [2023-12-27 16:20:42,733][20347] Updated weights for policy 0, policy_version 1544287 (0.0006) [2023-12-27 16:20:42,791][20347] Updated weights for policy 0, policy_version 1544297 (0.0007) [2023-12-27 16:20:42,813][20381] Updated weights for policy 1, policy_version 1542702 (0.0008) [2023-12-27 16:20:42,863][20381] Updated weights for policy 1, policy_version 1542712 (0.0007) [2023-12-27 16:20:42,913][20381] Updated weights for policy 1, policy_version 1542722 (0.0009) [2023-12-27 16:20:43,448][20347] Updated weights for policy 0, policy_version 1544307 (0.0006) [2023-12-27 16:20:43,501][20347] Updated weights for policy 0, policy_version 1544317 (0.0007) [2023-12-27 16:20:43,556][20347] Updated weights for policy 0, policy_version 1544327 (0.0007) [2023-12-27 16:20:43,641][20381] Updated weights for policy 1, policy_version 1542732 (0.0008) [2023-12-27 16:20:43,692][20381] Updated weights for policy 1, policy_version 1542742 (0.0008) [2023-12-27 16:20:43,745][20381] Updated weights for policy 1, policy_version 1542752 (0.0008) [2023-12-27 16:20:44,207][20347] Updated weights for policy 0, policy_version 1544337 (0.0007) [2023-12-27 16:20:44,263][20347] Updated weights for policy 0, policy_version 1544347 (0.0006) [2023-12-27 16:20:44,319][20347] Updated weights for policy 0, policy_version 1544357 (0.0005) [2023-12-27 16:20:44,365][20347] Updated weights for policy 0, policy_version 1544367 (0.0005) [2023-12-27 16:20:44,457][20381] Updated weights for policy 1, policy_version 1542762 (0.0007) [2023-12-27 16:20:44,509][20381] Updated weights for policy 1, policy_version 1542772 (0.0007) [2023-12-27 16:20:44,568][20381] Updated weights for policy 1, policy_version 1542782 (0.0007) [2023-12-27 16:20:44,614][20381] Updated weights for policy 1, policy_version 1542792 (0.0007) [2023-12-27 16:20:45,039][20347] Updated weights for policy 0, policy_version 1544377 (0.0007) [2023-12-27 16:20:45,088][20347] Updated weights for policy 0, policy_version 1544387 (0.0008) [2023-12-27 16:20:45,140][20347] Updated weights for policy 0, policy_version 1544397 (0.0008) [2023-12-27 16:20:45,377][20381] Updated weights for policy 1, policy_version 1542802 (0.0009) [2023-12-27 16:20:45,437][20381] Updated weights for policy 1, policy_version 1542812 (0.0007) [2023-12-27 16:20:45,498][20381] Updated weights for policy 1, policy_version 1542822 (0.0008) [2023-12-27 16:20:45,804][20347] Updated weights for policy 0, policy_version 1544407 (0.0007) [2023-12-27 16:20:45,855][20347] Updated weights for policy 0, policy_version 1544417 (0.0007) [2023-12-27 16:20:45,908][20347] Updated weights for policy 0, policy_version 1544427 (0.0007) [2023-12-27 16:20:46,163][20381] Updated weights for policy 1, policy_version 1542832 (0.0006) [2023-12-27 16:20:46,227][20381] Updated weights for policy 1, policy_version 1542842 (0.0005) [2023-12-27 16:20:46,285][20381] Updated weights for policy 1, policy_version 1542852 (0.0005) [2023-12-27 16:20:46,628][20347] Updated weights for policy 0, policy_version 1544437 (0.0007) [2023-12-27 16:20:46,673][20347] Updated weights for policy 0, policy_version 1544447 (0.0009) [2023-12-27 16:20:46,728][20347] Updated weights for policy 0, policy_version 1544457 (0.0008) [2023-12-27 16:20:46,876][20381] Updated weights for policy 1, policy_version 1542862 (0.0008) [2023-12-27 16:20:46,934][20381] Updated weights for policy 1, policy_version 1542872 (0.0010) [2023-12-27 16:20:46,988][20381] Updated weights for policy 1, policy_version 1542883 (0.0008) [2023-12-27 16:20:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 790478848. Throughput: 0: 10331.1, 1: 10404.4. Samples: 790490592. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:47,363][19170] Avg episode reward: [(0, '473.950'), (1, '702.490')] [2023-12-27 16:20:47,460][20347] Updated weights for policy 0, policy_version 1544467 (0.0008) [2023-12-27 16:20:47,522][20347] Updated weights for policy 0, policy_version 1544477 (0.0008) [2023-12-27 16:20:47,579][20347] Updated weights for policy 0, policy_version 1544487 (0.0007) [2023-12-27 16:20:47,618][20381] Updated weights for policy 1, policy_version 1542893 (0.0010) [2023-12-27 16:20:47,669][20381] Updated weights for policy 1, policy_version 1542903 (0.0010) [2023-12-27 16:20:47,735][20381] Updated weights for policy 1, policy_version 1542913 (0.0010) [2023-12-27 16:20:48,256][20347] Updated weights for policy 0, policy_version 1544497 (0.0006) [2023-12-27 16:20:48,313][20347] Updated weights for policy 0, policy_version 1544507 (0.0006) [2023-12-27 16:20:48,379][20347] Updated weights for policy 0, policy_version 1544517 (0.0007) [2023-12-27 16:20:48,384][20381] Updated weights for policy 1, policy_version 1542923 (0.0007) [2023-12-27 16:20:48,432][20381] Updated weights for policy 1, policy_version 1542933 (0.0006) [2023-12-27 16:20:48,436][20347] Updated weights for policy 0, policy_version 1544527 (0.0008) [2023-12-27 16:20:48,483][20381] Updated weights for policy 1, policy_version 1542943 (0.0007) [2023-12-27 16:20:49,124][20347] Updated weights for policy 0, policy_version 1544537 (0.0008) [2023-12-27 16:20:49,185][20347] Updated weights for policy 0, policy_version 1544547 (0.0009) [2023-12-27 16:20:49,211][20381] Updated weights for policy 1, policy_version 1542953 (0.0008) [2023-12-27 16:20:49,252][20347] Updated weights for policy 0, policy_version 1544557 (0.0009) [2023-12-27 16:20:49,278][20381] Updated weights for policy 1, policy_version 1542963 (0.0009) [2023-12-27 16:20:49,347][20381] Updated weights for policy 1, policy_version 1542973 (0.0009) [2023-12-27 16:20:49,413][20381] Updated weights for policy 1, policy_version 1542983 (0.0008) [2023-12-27 16:20:49,906][20347] Updated weights for policy 0, policy_version 1544567 (0.0008) [2023-12-27 16:20:49,983][20347] Updated weights for policy 0, policy_version 1544580 (0.0008) [2023-12-27 16:20:50,045][20347] Updated weights for policy 0, policy_version 1544590 (0.0009) [2023-12-27 16:20:50,133][20381] Updated weights for policy 1, policy_version 1542993 (0.0007) [2023-12-27 16:20:50,190][20381] Updated weights for policy 1, policy_version 1543003 (0.0008) [2023-12-27 16:20:50,247][20381] Updated weights for policy 1, policy_version 1543013 (0.0007) [2023-12-27 16:20:50,686][20347] Updated weights for policy 0, policy_version 1544600 (0.0006) [2023-12-27 16:20:50,741][20347] Updated weights for policy 0, policy_version 1544610 (0.0006) [2023-12-27 16:20:50,796][20347] Updated weights for policy 0, policy_version 1544620 (0.0007) [2023-12-27 16:20:51,007][20381] Updated weights for policy 1, policy_version 1543023 (0.0008) [2023-12-27 16:20:51,071][20381] Updated weights for policy 1, policy_version 1543033 (0.0009) [2023-12-27 16:20:51,134][20381] Updated weights for policy 1, policy_version 1543043 (0.0009) [2023-12-27 16:20:51,494][20347] Updated weights for policy 0, policy_version 1544630 (0.0007) [2023-12-27 16:20:51,552][20347] Updated weights for policy 0, policy_version 1544640 (0.0007) [2023-12-27 16:20:51,612][20347] Updated weights for policy 0, policy_version 1544650 (0.0008) [2023-12-27 16:20:51,827][20381] Updated weights for policy 1, policy_version 1543053 (0.0008) [2023-12-27 16:20:51,884][20381] Updated weights for policy 1, policy_version 1543063 (0.0005) [2023-12-27 16:20:51,948][20381] Updated weights for policy 1, policy_version 1543073 (0.0005) [2023-12-27 16:20:52,337][20347] Updated weights for policy 0, policy_version 1544660 (0.0008) [2023-12-27 16:20:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 790577152. Throughput: 0: 10262.1, 1: 10382.2. Samples: 790549968. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:52,363][19170] Avg episode reward: [(0, '625.730'), (1, '750.220')] [2023-12-27 16:20:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001543080_395067392.pth... [2023-12-27 16:20:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001541864_394756096.pth [2023-12-27 16:20:52,404][20347] Updated weights for policy 0, policy_version 1544670 (0.0008) [2023-12-27 16:20:52,459][20347] Updated weights for policy 0, policy_version 1544680 (0.0007) [2023-12-27 16:20:52,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001544688_395517952.pth... [2023-12-27 16:20:52,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001543472_395206656.pth [2023-12-27 16:20:52,608][20381] Updated weights for policy 1, policy_version 1543083 (0.0006) [2023-12-27 16:20:52,660][20381] Updated weights for policy 1, policy_version 1543093 (0.0007) [2023-12-27 16:20:52,712][20381] Updated weights for policy 1, policy_version 1543103 (0.0007) [2023-12-27 16:20:53,123][20347] Updated weights for policy 0, policy_version 1544690 (0.0007) [2023-12-27 16:20:53,171][20347] Updated weights for policy 0, policy_version 1544700 (0.0007) [2023-12-27 16:20:53,221][20347] Updated weights for policy 0, policy_version 1544710 (0.0005) [2023-12-27 16:20:53,268][20347] Updated weights for policy 0, policy_version 1544720 (0.0005) [2023-12-27 16:20:53,423][20381] Updated weights for policy 1, policy_version 1543113 (0.0007) [2023-12-27 16:20:53,471][20381] Updated weights for policy 1, policy_version 1543123 (0.0007) [2023-12-27 16:20:53,525][20381] Updated weights for policy 1, policy_version 1543133 (0.0007) [2023-12-27 16:20:53,580][20381] Updated weights for policy 1, policy_version 1543143 (0.0008) [2023-12-27 16:20:53,896][20347] Updated weights for policy 0, policy_version 1544730 (0.0009) [2023-12-27 16:20:53,951][20347] Updated weights for policy 0, policy_version 1544740 (0.0005) [2023-12-27 16:20:54,012][20347] Updated weights for policy 0, policy_version 1544750 (0.0005) [2023-12-27 16:20:54,333][20381] Updated weights for policy 1, policy_version 1543153 (0.0007) [2023-12-27 16:20:54,389][20381] Updated weights for policy 1, policy_version 1543163 (0.0006) [2023-12-27 16:20:54,455][20381] Updated weights for policy 1, policy_version 1543173 (0.0005) [2023-12-27 16:20:54,554][20347] Updated weights for policy 0, policy_version 1544760 (0.0007) [2023-12-27 16:20:54,609][20347] Updated weights for policy 0, policy_version 1544770 (0.0010) [2023-12-27 16:20:54,659][20347] Updated weights for policy 0, policy_version 1544780 (0.0007) [2023-12-27 16:20:55,049][20381] Updated weights for policy 1, policy_version 1543183 (0.0007) [2023-12-27 16:20:55,110][20381] Updated weights for policy 1, policy_version 1543193 (0.0005) [2023-12-27 16:20:55,169][20381] Updated weights for policy 1, policy_version 1543203 (0.0005) [2023-12-27 16:20:55,198][20347] Updated weights for policy 0, policy_version 1544790 (0.0007) [2023-12-27 16:20:55,247][20347] Updated weights for policy 0, policy_version 1544800 (0.0009) [2023-12-27 16:20:55,304][20347] Updated weights for policy 0, policy_version 1544810 (0.0005) [2023-12-27 16:20:55,753][20381] Updated weights for policy 1, policy_version 1543213 (0.0009) [2023-12-27 16:20:55,808][20381] Updated weights for policy 1, policy_version 1543223 (0.0011) [2023-12-27 16:20:55,866][20381] Updated weights for policy 1, policy_version 1543233 (0.0011) [2023-12-27 16:20:55,958][20347] Updated weights for policy 0, policy_version 1544820 (0.0007) [2023-12-27 16:20:56,013][20347] Updated weights for policy 0, policy_version 1544830 (0.0010) [2023-12-27 16:20:56,069][20347] Updated weights for policy 0, policy_version 1544840 (0.0010) [2023-12-27 16:20:56,489][20381] Updated weights for policy 1, policy_version 1543243 (0.0010) [2023-12-27 16:20:56,547][20381] Updated weights for policy 1, policy_version 1543253 (0.0008) [2023-12-27 16:20:56,608][20381] Updated weights for policy 1, policy_version 1543263 (0.0007) [2023-12-27 16:20:56,719][20347] Updated weights for policy 0, policy_version 1544850 (0.0009) [2023-12-27 16:20:56,773][20347] Updated weights for policy 0, policy_version 1544860 (0.0005) [2023-12-27 16:20:56,823][20347] Updated weights for policy 0, policy_version 1544870 (0.0005) [2023-12-27 16:20:56,871][20347] Updated weights for policy 0, policy_version 1544880 (0.0005) [2023-12-27 16:20:57,254][20381] Updated weights for policy 1, policy_version 1543273 (0.0010) [2023-12-27 16:20:57,298][20381] Updated weights for policy 1, policy_version 1543283 (0.0008) [2023-12-27 16:20:57,356][20381] Updated weights for policy 1, policy_version 1543293 (0.0006) [2023-12-27 16:20:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 790683648. Throughput: 0: 10358.4, 1: 10322.0. Samples: 790676464. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:20:57,363][19170] Avg episode reward: [(0, '572.830'), (1, '639.170')] [2023-12-27 16:20:57,406][20381] Updated weights for policy 1, policy_version 1543303 (0.0005) [2023-12-27 16:20:57,484][20347] Updated weights for policy 0, policy_version 1544890 (0.0008) [2023-12-27 16:20:57,537][20347] Updated weights for policy 0, policy_version 1544900 (0.0007) [2023-12-27 16:20:57,590][20347] Updated weights for policy 0, policy_version 1544910 (0.0008) [2023-12-27 16:20:58,003][20381] Updated weights for policy 1, policy_version 1543313 (0.0008) [2023-12-27 16:20:58,065][20381] Updated weights for policy 1, policy_version 1543323 (0.0008) [2023-12-27 16:20:58,127][20381] Updated weights for policy 1, policy_version 1543333 (0.0006) [2023-12-27 16:20:58,316][20347] Updated weights for policy 0, policy_version 1544920 (0.0008) [2023-12-27 16:20:58,383][20347] Updated weights for policy 0, policy_version 1544930 (0.0008) [2023-12-27 16:20:58,446][20347] Updated weights for policy 0, policy_version 1544940 (0.0008) [2023-12-27 16:20:58,882][20381] Updated weights for policy 1, policy_version 1543343 (0.0007) [2023-12-27 16:20:58,944][20381] Updated weights for policy 1, policy_version 1543353 (0.0008) [2023-12-27 16:20:59,009][20381] Updated weights for policy 1, policy_version 1543363 (0.0009) [2023-12-27 16:20:59,203][20347] Updated weights for policy 0, policy_version 1544950 (0.0009) [2023-12-27 16:20:59,278][20347] Updated weights for policy 0, policy_version 1544960 (0.0010) [2023-12-27 16:20:59,337][20347] Updated weights for policy 0, policy_version 1544970 (0.0009) [2023-12-27 16:20:59,772][20381] Updated weights for policy 1, policy_version 1543373 (0.0007) [2023-12-27 16:20:59,831][20381] Updated weights for policy 1, policy_version 1543383 (0.0007) [2023-12-27 16:20:59,880][20381] Updated weights for policy 1, policy_version 1543393 (0.0007) [2023-12-27 16:21:00,013][20347] Updated weights for policy 0, policy_version 1544980 (0.0009) [2023-12-27 16:21:00,069][20347] Updated weights for policy 0, policy_version 1544990 (0.0010) [2023-12-27 16:21:00,129][20347] Updated weights for policy 0, policy_version 1545000 (0.0008) [2023-12-27 16:21:00,530][20381] Updated weights for policy 1, policy_version 1543403 (0.0006) [2023-12-27 16:21:00,587][20381] Updated weights for policy 1, policy_version 1543413 (0.0005) [2023-12-27 16:21:00,636][20381] Updated weights for policy 1, policy_version 1543423 (0.0005) [2023-12-27 16:21:00,827][20347] Updated weights for policy 0, policy_version 1545010 (0.0006) [2023-12-27 16:21:00,878][20347] Updated weights for policy 0, policy_version 1545020 (0.0005) [2023-12-27 16:21:00,938][20347] Updated weights for policy 0, policy_version 1545030 (0.0005) [2023-12-27 16:21:00,995][20347] Updated weights for policy 0, policy_version 1545040 (0.0006) [2023-12-27 16:21:01,193][20381] Updated weights for policy 1, policy_version 1543433 (0.0006) [2023-12-27 16:21:01,258][20381] Updated weights for policy 1, policy_version 1543443 (0.0008) [2023-12-27 16:21:01,328][20381] Updated weights for policy 1, policy_version 1543453 (0.0007) [2023-12-27 16:21:01,389][20381] Updated weights for policy 1, policy_version 1543463 (0.0008) [2023-12-27 16:21:01,618][20347] Updated weights for policy 0, policy_version 1545050 (0.0007) [2023-12-27 16:21:01,685][20347] Updated weights for policy 0, policy_version 1545060 (0.0008) [2023-12-27 16:21:01,749][20347] Updated weights for policy 0, policy_version 1545070 (0.0007) [2023-12-27 16:21:02,095][20381] Updated weights for policy 1, policy_version 1543473 (0.0008) [2023-12-27 16:21:02,141][20381] Updated weights for policy 1, policy_version 1543483 (0.0007) [2023-12-27 16:21:02,196][20381] Updated weights for policy 1, policy_version 1543493 (0.0008) [2023-12-27 16:21:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 790790144. Throughput: 0: 10333.9, 1: 10341.1. Samples: 790799736. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:02,363][19170] Avg episode reward: [(0, '688.730'), (1, '548.690')] [2023-12-27 16:21:02,413][20347] Updated weights for policy 0, policy_version 1545080 (0.0007) [2023-12-27 16:21:02,472][20347] Updated weights for policy 0, policy_version 1545090 (0.0007) [2023-12-27 16:21:02,534][20347] Updated weights for policy 0, policy_version 1545100 (0.0007) [2023-12-27 16:21:02,879][20381] Updated weights for policy 1, policy_version 1543503 (0.0006) [2023-12-27 16:21:02,935][20381] Updated weights for policy 1, policy_version 1543513 (0.0005) [2023-12-27 16:21:02,995][20381] Updated weights for policy 1, policy_version 1543523 (0.0006) [2023-12-27 16:21:03,192][20347] Updated weights for policy 0, policy_version 1545110 (0.0009) [2023-12-27 16:21:03,249][20347] Updated weights for policy 0, policy_version 1545120 (0.0010) [2023-12-27 16:21:03,315][20347] Updated weights for policy 0, policy_version 1545130 (0.0010) [2023-12-27 16:21:03,589][20381] Updated weights for policy 1, policy_version 1543533 (0.0010) [2023-12-27 16:21:03,638][20381] Updated weights for policy 1, policy_version 1543543 (0.0011) [2023-12-27 16:21:03,694][20381] Updated weights for policy 1, policy_version 1543553 (0.0011) [2023-12-27 16:21:03,950][20347] Updated weights for policy 0, policy_version 1545140 (0.0007) [2023-12-27 16:21:04,004][20347] Updated weights for policy 0, policy_version 1545150 (0.0005) [2023-12-27 16:21:04,061][20347] Updated weights for policy 0, policy_version 1545160 (0.0006) [2023-12-27 16:21:04,404][20381] Updated weights for policy 1, policy_version 1543563 (0.0008) [2023-12-27 16:21:04,463][20381] Updated weights for policy 1, policy_version 1543573 (0.0008) [2023-12-27 16:21:04,524][20381] Updated weights for policy 1, policy_version 1543583 (0.0008) [2023-12-27 16:21:04,803][20347] Updated weights for policy 0, policy_version 1545170 (0.0007) [2023-12-27 16:21:04,862][20347] Updated weights for policy 0, policy_version 1545180 (0.0007) [2023-12-27 16:21:04,919][20347] Updated weights for policy 0, policy_version 1545190 (0.0007) [2023-12-27 16:21:04,981][20347] Updated weights for policy 0, policy_version 1545200 (0.0008) [2023-12-27 16:21:05,171][20381] Updated weights for policy 1, policy_version 1543593 (0.0008) [2023-12-27 16:21:05,221][20381] Updated weights for policy 1, policy_version 1543603 (0.0010) [2023-12-27 16:21:05,272][20381] Updated weights for policy 1, policy_version 1543613 (0.0010) [2023-12-27 16:21:05,323][20381] Updated weights for policy 1, policy_version 1543623 (0.0010) [2023-12-27 16:21:05,638][20347] Updated weights for policy 0, policy_version 1545210 (0.0007) [2023-12-27 16:21:05,690][20347] Updated weights for policy 0, policy_version 1545220 (0.0010) [2023-12-27 16:21:05,738][20347] Updated weights for policy 0, policy_version 1545230 (0.0010) [2023-12-27 16:21:05,979][20381] Updated weights for policy 1, policy_version 1543633 (0.0006) [2023-12-27 16:21:06,023][20381] Updated weights for policy 1, policy_version 1543643 (0.0005) [2023-12-27 16:21:06,084][20381] Updated weights for policy 1, policy_version 1543653 (0.0006) [2023-12-27 16:21:06,409][20347] Updated weights for policy 0, policy_version 1545240 (0.0010) [2023-12-27 16:21:06,469][20347] Updated weights for policy 0, policy_version 1545250 (0.0009) [2023-12-27 16:21:06,526][20347] Updated weights for policy 0, policy_version 1545260 (0.0008) [2023-12-27 16:21:06,822][20381] Updated weights for policy 1, policy_version 1543663 (0.0007) [2023-12-27 16:21:06,880][20381] Updated weights for policy 1, policy_version 1543673 (0.0008) [2023-12-27 16:21:06,941][20381] Updated weights for policy 1, policy_version 1543683 (0.0008) [2023-12-27 16:21:07,111][20347] Updated weights for policy 0, policy_version 1545270 (0.0005) [2023-12-27 16:21:07,165][20347] Updated weights for policy 0, policy_version 1545280 (0.0005) [2023-12-27 16:21:07,221][20347] Updated weights for policy 0, policy_version 1545290 (0.0005) [2023-12-27 16:21:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 790896640. Throughput: 0: 10339.7, 1: 10391.7. Samples: 790861708. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:07,363][19170] Avg episode reward: [(0, '695.980'), (1, '606.080')] [2023-12-27 16:21:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001543688_395223040.pth... [2023-12-27 16:21:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001545296_395673600.pth... [2023-12-27 16:21:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001542504_394919936.pth [2023-12-27 16:21:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001544080_395362304.pth [2023-12-27 16:21:07,682][20381] Updated weights for policy 1, policy_version 1543693 (0.0007) [2023-12-27 16:21:07,737][20381] Updated weights for policy 1, policy_version 1543703 (0.0006) [2023-12-27 16:21:07,782][20347] Updated weights for policy 0, policy_version 1545300 (0.0005) [2023-12-27 16:21:07,800][20381] Updated weights for policy 1, policy_version 1543713 (0.0008) [2023-12-27 16:21:07,834][20347] Updated weights for policy 0, policy_version 1545310 (0.0005) [2023-12-27 16:21:07,885][20347] Updated weights for policy 0, policy_version 1545320 (0.0005) [2023-12-27 16:21:08,457][20347] Updated weights for policy 0, policy_version 1545330 (0.0006) [2023-12-27 16:21:08,500][20381] Updated weights for policy 1, policy_version 1543723 (0.0008) [2023-12-27 16:21:08,515][20347] Updated weights for policy 0, policy_version 1545340 (0.0005) [2023-12-27 16:21:08,559][20381] Updated weights for policy 1, policy_version 1543733 (0.0006) [2023-12-27 16:21:08,572][20347] Updated weights for policy 0, policy_version 1545350 (0.0007) [2023-12-27 16:21:08,618][20381] Updated weights for policy 1, policy_version 1543743 (0.0007) [2023-12-27 16:21:08,624][20347] Updated weights for policy 0, policy_version 1545360 (0.0006) [2023-12-27 16:21:09,230][20347] Updated weights for policy 0, policy_version 1545370 (0.0007) [2023-12-27 16:21:09,299][20347] Updated weights for policy 0, policy_version 1545380 (0.0007) [2023-12-27 16:21:09,322][20381] Updated weights for policy 1, policy_version 1543753 (0.0008) [2023-12-27 16:21:09,362][20347] Updated weights for policy 0, policy_version 1545390 (0.0008) [2023-12-27 16:21:09,379][20381] Updated weights for policy 1, policy_version 1543763 (0.0007) [2023-12-27 16:21:09,440][20381] Updated weights for policy 1, policy_version 1543773 (0.0008) [2023-12-27 16:21:09,500][20381] Updated weights for policy 1, policy_version 1543783 (0.0009) [2023-12-27 16:21:10,039][20347] Updated weights for policy 0, policy_version 1545400 (0.0006) [2023-12-27 16:21:10,103][20347] Updated weights for policy 0, policy_version 1545410 (0.0008) [2023-12-27 16:21:10,156][20347] Updated weights for policy 0, policy_version 1545420 (0.0005) [2023-12-27 16:21:10,297][20381] Updated weights for policy 1, policy_version 1543793 (0.0008) [2023-12-27 16:21:10,357][20381] Updated weights for policy 1, policy_version 1543803 (0.0008) [2023-12-27 16:21:10,417][20381] Updated weights for policy 1, policy_version 1543813 (0.0008) [2023-12-27 16:21:10,775][20347] Updated weights for policy 0, policy_version 1545430 (0.0007) [2023-12-27 16:21:10,823][20347] Updated weights for policy 0, policy_version 1545440 (0.0007) [2023-12-27 16:21:10,874][20347] Updated weights for policy 0, policy_version 1545450 (0.0007) [2023-12-27 16:21:11,155][20381] Updated weights for policy 1, policy_version 1543823 (0.0007) [2023-12-27 16:21:11,218][20381] Updated weights for policy 1, policy_version 1543833 (0.0007) [2023-12-27 16:21:11,287][20381] Updated weights for policy 1, policy_version 1543843 (0.0007) [2023-12-27 16:21:11,643][20347] Updated weights for policy 0, policy_version 1545460 (0.0008) [2023-12-27 16:21:11,730][20347] Updated weights for policy 0, policy_version 1545470 (0.0008) [2023-12-27 16:21:11,800][20347] Updated weights for policy 0, policy_version 1545480 (0.0008) [2023-12-27 16:21:12,023][20381] Updated weights for policy 1, policy_version 1543853 (0.0007) [2023-12-27 16:21:12,088][20381] Updated weights for policy 1, policy_version 1543863 (0.0005) [2023-12-27 16:21:12,146][20381] Updated weights for policy 1, policy_version 1543873 (0.0008) [2023-12-27 16:21:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20617.7, 300 sec: 20632.7). Total num frames: 790994944. Throughput: 0: 10442.6, 1: 10297.7. Samples: 790985500. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:12,363][19170] Avg episode reward: [(0, '572.670'), (1, '642.580')] [2023-12-27 16:21:12,515][20347] Updated weights for policy 0, policy_version 1545490 (0.0008) [2023-12-27 16:21:12,538][20291] KL-divergence is very high: 124.2906 [2023-12-27 16:21:12,544][20291] KL-divergence is very high: 140.0762 [2023-12-27 16:21:12,550][20291] KL-divergence is very high: 256.3123 [2023-12-27 16:21:12,556][20291] KL-divergence is very high: 238.0056 [2023-12-27 16:21:12,572][20347] Updated weights for policy 0, policy_version 1545500 (0.0011) [2023-12-27 16:21:12,591][20291] KL-divergence is very high: 100.9143 [2023-12-27 16:21:12,620][20347] Updated weights for policy 0, policy_version 1545510 (0.0010) [2023-12-27 16:21:12,686][20347] Updated weights for policy 0, policy_version 1545520 (0.0010) [2023-12-27 16:21:12,743][20381] Updated weights for policy 1, policy_version 1543883 (0.0010) [2023-12-27 16:21:12,793][20381] Updated weights for policy 1, policy_version 1543893 (0.0011) [2023-12-27 16:21:12,843][20381] Updated weights for policy 1, policy_version 1543903 (0.0011) [2023-12-27 16:21:13,336][20291] KL-divergence is very high: 282.4739 [2023-12-27 16:21:13,366][20347] Updated weights for policy 0, policy_version 1545530 (0.0010) [2023-12-27 16:21:13,376][20291] KL-divergence is very high: 281.9089 [2023-12-27 16:21:13,413][20291] KL-divergence is very high: 194.3428 [2023-12-27 16:21:13,415][20347] Updated weights for policy 0, policy_version 1545540 (0.0010) [2023-12-27 16:21:13,449][20291] KL-divergence is very high: 129.5190 [2023-12-27 16:21:13,460][20347] Updated weights for policy 0, policy_version 1545550 (0.0010) [2023-12-27 16:21:13,502][20381] Updated weights for policy 1, policy_version 1543913 (0.0011) [2023-12-27 16:21:13,555][20381] Updated weights for policy 1, policy_version 1543923 (0.0011) [2023-12-27 16:21:13,601][20381] Updated weights for policy 1, policy_version 1543933 (0.0011) [2023-12-27 16:21:13,650][20381] Updated weights for policy 1, policy_version 1543943 (0.0010) [2023-12-27 16:21:14,164][20347] Updated weights for policy 0, policy_version 1545560 (0.0010) [2023-12-27 16:21:14,223][20347] Updated weights for policy 0, policy_version 1545570 (0.0010) [2023-12-27 16:21:14,284][20347] Updated weights for policy 0, policy_version 1545580 (0.0010) [2023-12-27 16:21:14,356][20381] Updated weights for policy 1, policy_version 1543953 (0.0010) [2023-12-27 16:21:14,409][20381] Updated weights for policy 1, policy_version 1543963 (0.0008) [2023-12-27 16:21:14,458][20381] Updated weights for policy 1, policy_version 1543973 (0.0008) [2023-12-27 16:21:14,958][20347] Updated weights for policy 0, policy_version 1545590 (0.0007) [2023-12-27 16:21:14,960][20291] KL-divergence is very high: 166.9043 [2023-12-27 16:21:15,013][20291] KL-divergence is very high: 160.8069 [2023-12-27 16:21:15,026][20347] Updated weights for policy 0, policy_version 1545600 (0.0006) [2023-12-27 16:21:15,090][20347] Updated weights for policy 0, policy_version 1545610 (0.0006) [2023-12-27 16:21:15,158][20381] Updated weights for policy 1, policy_version 1543983 (0.0010) [2023-12-27 16:21:15,225][20381] Updated weights for policy 1, policy_version 1543993 (0.0010) [2023-12-27 16:21:15,295][20381] Updated weights for policy 1, policy_version 1544003 (0.0009) [2023-12-27 16:21:15,709][20347] Updated weights for policy 0, policy_version 1545620 (0.0006) [2023-12-27 16:21:15,766][20347] Updated weights for policy 0, policy_version 1545630 (0.0006) [2023-12-27 16:21:15,819][20347] Updated weights for policy 0, policy_version 1545640 (0.0006) [2023-12-27 16:21:15,924][20381] Updated weights for policy 1, policy_version 1544013 (0.0008) [2023-12-27 16:21:15,980][20381] Updated weights for policy 1, policy_version 1544023 (0.0005) [2023-12-27 16:21:16,029][20381] Updated weights for policy 1, policy_version 1544033 (0.0006) [2023-12-27 16:21:16,465][20347] Updated weights for policy 0, policy_version 1545650 (0.0008) [2023-12-27 16:21:16,525][20347] Updated weights for policy 0, policy_version 1545660 (0.0008) [2023-12-27 16:21:16,582][20347] Updated weights for policy 0, policy_version 1545670 (0.0006) [2023-12-27 16:21:16,634][20347] Updated weights for policy 0, policy_version 1545680 (0.0005) [2023-12-27 16:21:16,707][20381] Updated weights for policy 1, policy_version 1544043 (0.0007) [2023-12-27 16:21:16,761][20381] Updated weights for policy 1, policy_version 1544053 (0.0009) [2023-12-27 16:21:16,814][20381] Updated weights for policy 1, policy_version 1544063 (0.0007) [2023-12-27 16:21:17,284][20347] Updated weights for policy 0, policy_version 1545690 (0.0008) [2023-12-27 16:21:17,335][20347] Updated weights for policy 0, policy_version 1545700 (0.0008) [2023-12-27 16:21:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 791093248. Throughput: 0: 10408.2, 1: 10265.4. Samples: 791108860. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:17,363][19170] Avg episode reward: [(0, '662.890'), (1, '524.380')] [2023-12-27 16:21:17,382][20347] Updated weights for policy 0, policy_version 1545710 (0.0007) [2023-12-27 16:21:17,420][20381] Updated weights for policy 1, policy_version 1544073 (0.0006) [2023-12-27 16:21:17,474][20381] Updated weights for policy 1, policy_version 1544083 (0.0006) [2023-12-27 16:21:17,526][20381] Updated weights for policy 1, policy_version 1544093 (0.0008) [2023-12-27 16:21:17,572][20381] Updated weights for policy 1, policy_version 1544103 (0.0008) [2023-12-27 16:21:18,097][20347] Updated weights for policy 0, policy_version 1545720 (0.0008) [2023-12-27 16:21:18,152][20347] Updated weights for policy 0, policy_version 1545730 (0.0007) [2023-12-27 16:21:18,218][20347] Updated weights for policy 0, policy_version 1545740 (0.0008) [2023-12-27 16:21:18,266][20381] Updated weights for policy 1, policy_version 1544113 (0.0007) [2023-12-27 16:21:18,322][20381] Updated weights for policy 1, policy_version 1544123 (0.0008) [2023-12-27 16:21:18,384][20381] Updated weights for policy 1, policy_version 1544133 (0.0008) [2023-12-27 16:21:18,828][20347] Updated weights for policy 0, policy_version 1545750 (0.0007) [2023-12-27 16:21:18,880][20347] Updated weights for policy 0, policy_version 1545760 (0.0007) [2023-12-27 16:21:18,942][20347] Updated weights for policy 0, policy_version 1545770 (0.0008) [2023-12-27 16:21:19,102][20381] Updated weights for policy 1, policy_version 1544143 (0.0008) [2023-12-27 16:21:19,151][20381] Updated weights for policy 1, policy_version 1544153 (0.0008) [2023-12-27 16:21:19,194][20381] Updated weights for policy 1, policy_version 1544163 (0.0008) [2023-12-27 16:21:19,678][20347] Updated weights for policy 0, policy_version 1545780 (0.0008) [2023-12-27 16:21:19,738][20347] Updated weights for policy 0, policy_version 1545790 (0.0008) [2023-12-27 16:21:19,801][20347] Updated weights for policy 0, policy_version 1545800 (0.0007) [2023-12-27 16:21:20,076][20381] Updated weights for policy 1, policy_version 1544173 (0.0008) [2023-12-27 16:21:20,131][20381] Updated weights for policy 1, policy_version 1544183 (0.0009) [2023-12-27 16:21:20,191][20381] Updated weights for policy 1, policy_version 1544193 (0.0009) [2023-12-27 16:21:20,497][20347] Updated weights for policy 0, policy_version 1545810 (0.0008) [2023-12-27 16:21:20,556][20347] Updated weights for policy 0, policy_version 1545820 (0.0008) [2023-12-27 16:21:20,556][20291] KL-divergence is very high: 163.3710 [2023-12-27 16:21:20,581][20291] KL-divergence is very high: 206.9306 [2023-12-27 16:21:20,595][20291] KL-divergence is very high: 142.5756 [2023-12-27 16:21:20,613][20291] KL-divergence is very high: 149.9510 [2023-12-27 16:21:20,622][20347] Updated weights for policy 0, policy_version 1545830 (0.0007) [2023-12-27 16:21:20,630][20291] KL-divergence is very high: 127.0314 [2023-12-27 16:21:20,679][20347] Updated weights for policy 0, policy_version 1545840 (0.0006) [2023-12-27 16:21:20,973][20381] Updated weights for policy 1, policy_version 1544203 (0.0008) [2023-12-27 16:21:21,037][20381] Updated weights for policy 1, policy_version 1544213 (0.0011) [2023-12-27 16:21:21,096][20381] Updated weights for policy 1, policy_version 1544223 (0.0010) [2023-12-27 16:21:21,325][20347] Updated weights for policy 0, policy_version 1545850 (0.0011) [2023-12-27 16:21:21,376][20291] KL-divergence is very high: 104.5015 [2023-12-27 16:21:21,402][20347] Updated weights for policy 0, policy_version 1545860 (0.0011) [2023-12-27 16:21:21,429][20291] KL-divergence is very high: 104.5775 [2023-12-27 16:21:21,458][20347] Updated weights for policy 0, policy_version 1545870 (0.0006) [2023-12-27 16:21:21,838][20381] Updated weights for policy 1, policy_version 1544233 (0.0011) [2023-12-27 16:21:21,898][20381] Updated weights for policy 1, policy_version 1544243 (0.0008) [2023-12-27 16:21:21,958][20381] Updated weights for policy 1, policy_version 1544253 (0.0009) [2023-12-27 16:21:22,017][20381] Updated weights for policy 1, policy_version 1544263 (0.0008) [2023-12-27 16:21:22,165][20347] Updated weights for policy 0, policy_version 1545880 (0.0009) [2023-12-27 16:21:22,217][20347] Updated weights for policy 0, policy_version 1545890 (0.0010) [2023-12-27 16:21:22,278][20347] Updated weights for policy 0, policy_version 1545900 (0.0011) [2023-12-27 16:21:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 791199744. Throughput: 0: 10340.6, 1: 10183.9. Samples: 791167436. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:22,363][19170] Avg episode reward: [(0, '632.150'), (1, '570.660')] [2023-12-27 16:21:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001544264_395370496.pth... [2023-12-27 16:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001545904_395829248.pth... [2023-12-27 16:21:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001543080_395067392.pth [2023-12-27 16:21:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001544688_395517952.pth [2023-12-27 16:21:22,701][20381] Updated weights for policy 1, policy_version 1544273 (0.0008) [2023-12-27 16:21:22,754][20381] Updated weights for policy 1, policy_version 1544283 (0.0007) [2023-12-27 16:21:22,809][20381] Updated weights for policy 1, policy_version 1544293 (0.0007) [2023-12-27 16:21:22,946][20347] Updated weights for policy 0, policy_version 1545910 (0.0011) [2023-12-27 16:21:23,002][20347] Updated weights for policy 0, policy_version 1545920 (0.0007) [2023-12-27 16:21:23,057][20347] Updated weights for policy 0, policy_version 1545930 (0.0010) [2023-12-27 16:21:23,561][20381] Updated weights for policy 1, policy_version 1544303 (0.0006) [2023-12-27 16:21:23,619][20381] Updated weights for policy 1, policy_version 1544313 (0.0006) [2023-12-27 16:21:23,665][20381] Updated weights for policy 1, policy_version 1544323 (0.0007) [2023-12-27 16:21:23,689][20347] Updated weights for policy 0, policy_version 1545940 (0.0010) [2023-12-27 16:21:23,744][20347] Updated weights for policy 0, policy_version 1545950 (0.0010) [2023-12-27 16:21:23,801][20347] Updated weights for policy 0, policy_version 1545960 (0.0010) [2023-12-27 16:21:24,253][20381] Updated weights for policy 1, policy_version 1544333 (0.0007) [2023-12-27 16:21:24,311][20381] Updated weights for policy 1, policy_version 1544343 (0.0007) [2023-12-27 16:21:24,369][20381] Updated weights for policy 1, policy_version 1544353 (0.0008) [2023-12-27 16:21:24,480][20347] Updated weights for policy 0, policy_version 1545970 (0.0010) [2023-12-27 16:21:24,525][20347] Updated weights for policy 0, policy_version 1545980 (0.0010) [2023-12-27 16:21:24,570][20347] Updated weights for policy 0, policy_version 1545990 (0.0010) [2023-12-27 16:21:24,618][20347] Updated weights for policy 0, policy_version 1546000 (0.0010) [2023-12-27 16:21:25,015][20381] Updated weights for policy 1, policy_version 1544363 (0.0006) [2023-12-27 16:21:25,073][20381] Updated weights for policy 1, policy_version 1544373 (0.0008) [2023-12-27 16:21:25,124][20381] Updated weights for policy 1, policy_version 1544383 (0.0008) [2023-12-27 16:21:25,254][20347] Updated weights for policy 0, policy_version 1546010 (0.0010) [2023-12-27 16:21:25,299][20347] Updated weights for policy 0, policy_version 1546020 (0.0010) [2023-12-27 16:21:25,348][20347] Updated weights for policy 0, policy_version 1546030 (0.0010) [2023-12-27 16:21:25,694][20381] Updated weights for policy 1, policy_version 1544393 (0.0005) [2023-12-27 16:21:25,754][20381] Updated weights for policy 1, policy_version 1544403 (0.0006) [2023-12-27 16:21:25,807][20381] Updated weights for policy 1, policy_version 1544413 (0.0005) [2023-12-27 16:21:25,869][20381] Updated weights for policy 1, policy_version 1544423 (0.0005) [2023-12-27 16:21:26,055][20347] Updated weights for policy 0, policy_version 1546040 (0.0010) [2023-12-27 16:21:26,107][20347] Updated weights for policy 0, policy_version 1546050 (0.0010) [2023-12-27 16:21:26,165][20347] Updated weights for policy 0, policy_version 1546060 (0.0010) [2023-12-27 16:21:26,467][20381] Updated weights for policy 1, policy_version 1544433 (0.0006) [2023-12-27 16:21:26,517][20381] Updated weights for policy 1, policy_version 1544443 (0.0009) [2023-12-27 16:21:26,567][20381] Updated weights for policy 1, policy_version 1544453 (0.0010) [2023-12-27 16:21:26,818][20347] Updated weights for policy 0, policy_version 1546070 (0.0010) [2023-12-27 16:21:26,870][20347] Updated weights for policy 0, policy_version 1546080 (0.0010) [2023-12-27 16:21:26,921][20347] Updated weights for policy 0, policy_version 1546090 (0.0010) [2023-12-27 16:21:27,154][20381] Updated weights for policy 1, policy_version 1544463 (0.0008) [2023-12-27 16:21:27,213][20381] Updated weights for policy 1, policy_version 1544473 (0.0009) [2023-12-27 16:21:27,265][20381] Updated weights for policy 1, policy_version 1544483 (0.0006) [2023-12-27 16:21:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 791306240. Throughput: 0: 10378.8, 1: 10114.8. Samples: 791291720. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:27,363][19170] Avg episode reward: [(0, '669.700'), (1, '682.290')] [2023-12-27 16:21:27,612][20347] Updated weights for policy 0, policy_version 1546100 (0.0010) [2023-12-27 16:21:27,675][20347] Updated weights for policy 0, policy_version 1546110 (0.0010) [2023-12-27 16:21:27,735][20347] Updated weights for policy 0, policy_version 1546120 (0.0010) [2023-12-27 16:21:27,913][20381] Updated weights for policy 1, policy_version 1544493 (0.0007) [2023-12-27 16:21:27,958][20381] Updated weights for policy 1, policy_version 1544503 (0.0009) [2023-12-27 16:21:28,012][20381] Updated weights for policy 1, policy_version 1544513 (0.0010) [2023-12-27 16:21:28,389][20347] Updated weights for policy 0, policy_version 1546130 (0.0010) [2023-12-27 16:21:28,448][20347] Updated weights for policy 0, policy_version 1546140 (0.0010) [2023-12-27 16:21:28,500][20347] Updated weights for policy 0, policy_version 1546150 (0.0010) [2023-12-27 16:21:28,557][20347] Updated weights for policy 0, policy_version 1546160 (0.0010) [2023-12-27 16:21:28,623][20381] Updated weights for policy 1, policy_version 1544523 (0.0005) [2023-12-27 16:21:28,682][20381] Updated weights for policy 1, policy_version 1544533 (0.0005) [2023-12-27 16:21:28,743][20381] Updated weights for policy 1, policy_version 1544543 (0.0005) [2023-12-27 16:21:29,244][20347] Updated weights for policy 0, policy_version 1546170 (0.0009) [2023-12-27 16:21:29,305][20347] Updated weights for policy 0, policy_version 1546180 (0.0010) [2023-12-27 16:21:29,341][20381] Updated weights for policy 1, policy_version 1544553 (0.0006) [2023-12-27 16:21:29,371][20347] Updated weights for policy 0, policy_version 1546190 (0.0009) [2023-12-27 16:21:29,405][20381] Updated weights for policy 1, policy_version 1544563 (0.0008) [2023-12-27 16:21:29,457][20381] Updated weights for policy 1, policy_version 1544573 (0.0010) [2023-12-27 16:21:29,503][20381] Updated weights for policy 1, policy_version 1544583 (0.0010) [2023-12-27 16:21:30,106][20347] Updated weights for policy 0, policy_version 1546200 (0.0010) [2023-12-27 16:21:30,161][20347] Updated weights for policy 0, policy_version 1546210 (0.0010) [2023-12-27 16:21:30,204][20381] Updated weights for policy 1, policy_version 1544593 (0.0010) [2023-12-27 16:21:30,219][20347] Updated weights for policy 0, policy_version 1546220 (0.0008) [2023-12-27 16:21:30,257][20381] Updated weights for policy 1, policy_version 1544603 (0.0011) [2023-12-27 16:21:30,309][20381] Updated weights for policy 1, policy_version 1544613 (0.0010) [2023-12-27 16:21:30,930][20347] Updated weights for policy 0, policy_version 1546230 (0.0006) [2023-12-27 16:21:30,943][20381] Updated weights for policy 1, policy_version 1544623 (0.0011) [2023-12-27 16:21:30,986][20347] Updated weights for policy 0, policy_version 1546240 (0.0006) [2023-12-27 16:21:30,993][20381] Updated weights for policy 1, policy_version 1544633 (0.0011) [2023-12-27 16:21:31,038][20347] Updated weights for policy 0, policy_version 1546250 (0.0006) [2023-12-27 16:21:31,051][20381] Updated weights for policy 1, policy_version 1544643 (0.0011) [2023-12-27 16:21:31,720][20347] Updated weights for policy 0, policy_version 1546260 (0.0007) [2023-12-27 16:21:31,777][20347] Updated weights for policy 0, policy_version 1546270 (0.0008) [2023-12-27 16:21:31,808][20381] Updated weights for policy 1, policy_version 1544653 (0.0008) [2023-12-27 16:21:31,833][20347] Updated weights for policy 0, policy_version 1546280 (0.0007) [2023-12-27 16:21:31,861][20381] Updated weights for policy 1, policy_version 1544663 (0.0005) [2023-12-27 16:21:31,913][20381] Updated weights for policy 1, policy_version 1544673 (0.0007) [2023-12-27 16:21:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 791404544. Throughput: 0: 10363.4, 1: 10198.6. Samples: 791415880. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:32,363][19170] Avg episode reward: [(0, '782.690'), (1, '496.910')] [2023-12-27 16:21:32,529][20347] Updated weights for policy 0, policy_version 1546290 (0.0007) [2023-12-27 16:21:32,577][20347] Updated weights for policy 0, policy_version 1546300 (0.0008) [2023-12-27 16:21:32,597][20381] Updated weights for policy 1, policy_version 1544683 (0.0009) [2023-12-27 16:21:32,626][20347] Updated weights for policy 0, policy_version 1546310 (0.0006) [2023-12-27 16:21:32,654][20381] Updated weights for policy 1, policy_version 1544693 (0.0006) [2023-12-27 16:21:32,674][20347] Updated weights for policy 0, policy_version 1546320 (0.0005) [2023-12-27 16:21:32,708][20381] Updated weights for policy 1, policy_version 1544703 (0.0008) [2023-12-27 16:21:33,308][20347] Updated weights for policy 0, policy_version 1546330 (0.0008) [2023-12-27 16:21:33,361][20347] Updated weights for policy 0, policy_version 1546340 (0.0007) [2023-12-27 16:21:33,419][20381] Updated weights for policy 1, policy_version 1544713 (0.0007) [2023-12-27 16:21:33,424][20347] Updated weights for policy 0, policy_version 1546350 (0.0008) [2023-12-27 16:21:33,467][20381] Updated weights for policy 1, policy_version 1544723 (0.0007) [2023-12-27 16:21:33,513][20381] Updated weights for policy 1, policy_version 1544733 (0.0008) [2023-12-27 16:21:33,568][20381] Updated weights for policy 1, policy_version 1544743 (0.0007) [2023-12-27 16:21:34,094][20347] Updated weights for policy 0, policy_version 1546360 (0.0006) [2023-12-27 16:21:34,141][20347] Updated weights for policy 0, policy_version 1546370 (0.0006) [2023-12-27 16:21:34,198][20347] Updated weights for policy 0, policy_version 1546380 (0.0008) [2023-12-27 16:21:34,203][20381] Updated weights for policy 1, policy_version 1544753 (0.0007) [2023-12-27 16:21:34,263][20381] Updated weights for policy 1, policy_version 1544763 (0.0007) [2023-12-27 16:21:34,320][20381] Updated weights for policy 1, policy_version 1544773 (0.0008) [2023-12-27 16:21:34,859][20347] Updated weights for policy 0, policy_version 1546390 (0.0007) [2023-12-27 16:21:34,918][20347] Updated weights for policy 0, policy_version 1546400 (0.0008) [2023-12-27 16:21:34,973][20381] Updated weights for policy 1, policy_version 1544783 (0.0007) [2023-12-27 16:21:34,978][20347] Updated weights for policy 0, policy_version 1546410 (0.0009) [2023-12-27 16:21:35,026][20381] Updated weights for policy 1, policy_version 1544793 (0.0008) [2023-12-27 16:21:35,087][20381] Updated weights for policy 1, policy_version 1544803 (0.0009) [2023-12-27 16:21:35,609][20347] Updated weights for policy 0, policy_version 1546420 (0.0008) [2023-12-27 16:21:35,669][20347] Updated weights for policy 0, policy_version 1546430 (0.0007) [2023-12-27 16:21:35,719][20347] Updated weights for policy 0, policy_version 1546440 (0.0007) [2023-12-27 16:21:35,781][20381] Updated weights for policy 1, policy_version 1544813 (0.0008) [2023-12-27 16:21:35,841][20381] Updated weights for policy 1, policy_version 1544823 (0.0006) [2023-12-27 16:21:35,896][20381] Updated weights for policy 1, policy_version 1544833 (0.0005) [2023-12-27 16:21:36,448][20347] Updated weights for policy 0, policy_version 1546450 (0.0008) [2023-12-27 16:21:36,511][20347] Updated weights for policy 0, policy_version 1546460 (0.0010) [2023-12-27 16:21:36,533][20381] Updated weights for policy 1, policy_version 1544843 (0.0005) [2023-12-27 16:21:36,572][20347] Updated weights for policy 0, policy_version 1546470 (0.0010) [2023-12-27 16:21:36,591][20381] Updated weights for policy 1, policy_version 1544853 (0.0008) [2023-12-27 16:21:36,631][20347] Updated weights for policy 0, policy_version 1546480 (0.0011) [2023-12-27 16:21:36,645][20381] Updated weights for policy 1, policy_version 1544863 (0.0007) [2023-12-27 16:21:37,218][20347] Updated weights for policy 0, policy_version 1546490 (0.0005) [2023-12-27 16:21:37,253][20291] KL-divergence is very high: 114.8639 [2023-12-27 16:21:37,270][20347] Updated weights for policy 0, policy_version 1546500 (0.0005) [2023-12-27 16:21:37,332][20347] Updated weights for policy 0, policy_version 1546510 (0.0005) [2023-12-27 16:21:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 791511040. Throughput: 0: 10406.1, 1: 10228.3. Samples: 791478516. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:37,363][20381] Updated weights for policy 1, policy_version 1544873 (0.0008) [2023-12-27 16:21:37,363][19170] Avg episode reward: [(0, '693.520'), (1, '451.050')] [2023-12-27 16:21:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001546512_395984896.pth... [2023-12-27 16:21:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001545296_395673600.pth [2023-12-27 16:21:37,422][20381] Updated weights for policy 1, policy_version 1544883 (0.0007) [2023-12-27 16:21:37,478][20381] Updated weights for policy 1, policy_version 1544893 (0.0006) [2023-12-27 16:21:37,532][20381] Updated weights for policy 1, policy_version 1544903 (0.0006) [2023-12-27 16:21:37,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001544904_395534336.pth... [2023-12-27 16:21:37,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001543688_395223040.pth [2023-12-27 16:21:37,924][20347] Updated weights for policy 0, policy_version 1546520 (0.0005) [2023-12-27 16:21:37,987][20347] Updated weights for policy 0, policy_version 1546530 (0.0008) [2023-12-27 16:21:38,041][20347] Updated weights for policy 0, policy_version 1546540 (0.0008) [2023-12-27 16:21:38,217][20381] Updated weights for policy 1, policy_version 1544913 (0.0007) [2023-12-27 16:21:38,277][20381] Updated weights for policy 1, policy_version 1544923 (0.0008) [2023-12-27 16:21:38,342][20381] Updated weights for policy 1, policy_version 1544933 (0.0008) [2023-12-27 16:21:38,729][20347] Updated weights for policy 0, policy_version 1546550 (0.0007) [2023-12-27 16:21:38,783][20347] Updated weights for policy 0, policy_version 1546560 (0.0008) [2023-12-27 16:21:38,837][20347] Updated weights for policy 0, policy_version 1546570 (0.0008) [2023-12-27 16:21:39,013][20381] Updated weights for policy 1, policy_version 1544943 (0.0008) [2023-12-27 16:21:39,076][20381] Updated weights for policy 1, policy_version 1544953 (0.0008) [2023-12-27 16:21:39,129][20381] Updated weights for policy 1, policy_version 1544963 (0.0007) [2023-12-27 16:21:39,577][20347] Updated weights for policy 0, policy_version 1546580 (0.0008) [2023-12-27 16:21:39,632][20347] Updated weights for policy 0, policy_version 1546590 (0.0008) [2023-12-27 16:21:39,694][20347] Updated weights for policy 0, policy_version 1546600 (0.0006) [2023-12-27 16:21:39,883][20381] Updated weights for policy 1, policy_version 1544973 (0.0007) [2023-12-27 16:21:39,948][20381] Updated weights for policy 1, policy_version 1544983 (0.0007) [2023-12-27 16:21:40,008][20381] Updated weights for policy 1, policy_version 1544993 (0.0008) [2023-12-27 16:21:40,333][20347] Updated weights for policy 0, policy_version 1546610 (0.0009) [2023-12-27 16:21:40,400][20347] Updated weights for policy 0, policy_version 1546620 (0.0006) [2023-12-27 16:21:40,464][20347] Updated weights for policy 0, policy_version 1546630 (0.0006) [2023-12-27 16:21:40,511][20347] Updated weights for policy 0, policy_version 1546640 (0.0006) [2023-12-27 16:21:40,611][20381] Updated weights for policy 1, policy_version 1545003 (0.0008) [2023-12-27 16:21:40,663][20381] Updated weights for policy 1, policy_version 1545013 (0.0010) [2023-12-27 16:21:40,710][20381] Updated weights for policy 1, policy_version 1545023 (0.0010) [2023-12-27 16:21:41,155][20347] Updated weights for policy 0, policy_version 1546650 (0.0007) [2023-12-27 16:21:41,217][20347] Updated weights for policy 0, policy_version 1546660 (0.0008) [2023-12-27 16:21:41,288][20347] Updated weights for policy 0, policy_version 1546670 (0.0007) [2023-12-27 16:21:41,440][20381] Updated weights for policy 1, policy_version 1545033 (0.0009) [2023-12-27 16:21:41,503][20381] Updated weights for policy 1, policy_version 1545043 (0.0009) [2023-12-27 16:21:41,555][20381] Updated weights for policy 1, policy_version 1545053 (0.0009) [2023-12-27 16:21:41,617][20381] Updated weights for policy 1, policy_version 1545063 (0.0009) [2023-12-27 16:21:42,045][20347] Updated weights for policy 0, policy_version 1546680 (0.0007) [2023-12-27 16:21:42,095][20347] Updated weights for policy 0, policy_version 1546690 (0.0007) [2023-12-27 16:21:42,150][20347] Updated weights for policy 0, policy_version 1546700 (0.0008) [2023-12-27 16:21:42,356][20381] Updated weights for policy 1, policy_version 1545073 (0.0008) [2023-12-27 16:21:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 791609344. Throughput: 0: 10346.8, 1: 10216.8. Samples: 791601828. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:42,363][19170] Avg episode reward: [(0, '529.180'), (1, '441.940')] [2023-12-27 16:21:42,421][20381] Updated weights for policy 1, policy_version 1545083 (0.0008) [2023-12-27 16:21:42,468][20381] Updated weights for policy 1, policy_version 1545093 (0.0007) [2023-12-27 16:21:42,797][20347] Updated weights for policy 0, policy_version 1546710 (0.0009) [2023-12-27 16:21:42,842][20291] KL-divergence is very high: 107.1917 [2023-12-27 16:21:42,843][20347] Updated weights for policy 0, policy_version 1546720 (0.0008) [2023-12-27 16:21:42,888][20347] Updated weights for policy 0, policy_version 1546730 (0.0007) [2023-12-27 16:21:43,150][20381] Updated weights for policy 1, policy_version 1545103 (0.0008) [2023-12-27 16:21:43,199][20381] Updated weights for policy 1, policy_version 1545113 (0.0008) [2023-12-27 16:21:43,245][20381] Updated weights for policy 1, policy_version 1545123 (0.0007) [2023-12-27 16:21:43,610][20347] Updated weights for policy 0, policy_version 1546740 (0.0007) [2023-12-27 16:21:43,674][20347] Updated weights for policy 0, policy_version 1546750 (0.0005) [2023-12-27 16:21:43,728][20347] Updated weights for policy 0, policy_version 1546760 (0.0005) [2023-12-27 16:21:43,979][20381] Updated weights for policy 1, policy_version 1545133 (0.0007) [2023-12-27 16:21:44,039][20381] Updated weights for policy 1, policy_version 1545143 (0.0008) [2023-12-27 16:21:44,091][20381] Updated weights for policy 1, policy_version 1545153 (0.0008) [2023-12-27 16:21:44,264][20347] Updated weights for policy 0, policy_version 1546770 (0.0006) [2023-12-27 16:21:44,316][20347] Updated weights for policy 0, policy_version 1546780 (0.0008) [2023-12-27 16:21:44,374][20347] Updated weights for policy 0, policy_version 1546790 (0.0008) [2023-12-27 16:21:44,423][20347] Updated weights for policy 0, policy_version 1546800 (0.0007) [2023-12-27 16:21:44,817][20381] Updated weights for policy 1, policy_version 1545163 (0.0008) [2023-12-27 16:21:44,879][20381] Updated weights for policy 1, policy_version 1545173 (0.0007) [2023-12-27 16:21:44,947][20381] Updated weights for policy 1, policy_version 1545183 (0.0006) [2023-12-27 16:21:45,169][20347] Updated weights for policy 0, policy_version 1546810 (0.0007) [2023-12-27 16:21:45,195][20291] KL-divergence is very high: 139.9687 [2023-12-27 16:21:45,215][20347] Updated weights for policy 0, policy_version 1546820 (0.0008) [2023-12-27 16:21:45,233][20291] KL-divergence is very high: 107.2591 [2023-12-27 16:21:45,264][20347] Updated weights for policy 0, policy_version 1546830 (0.0007) [2023-12-27 16:21:45,586][20381] Updated weights for policy 1, policy_version 1545193 (0.0008) [2023-12-27 16:21:45,649][20381] Updated weights for policy 1, policy_version 1545203 (0.0008) [2023-12-27 16:21:45,698][20381] Updated weights for policy 1, policy_version 1545213 (0.0008) [2023-12-27 16:21:45,753][20381] Updated weights for policy 1, policy_version 1545223 (0.0008) [2023-12-27 16:21:45,955][20347] Updated weights for policy 0, policy_version 1546840 (0.0006) [2023-12-27 16:21:46,005][20347] Updated weights for policy 0, policy_version 1546850 (0.0007) [2023-12-27 16:21:46,053][20347] Updated weights for policy 0, policy_version 1546860 (0.0007) [2023-12-27 16:21:46,476][20381] Updated weights for policy 1, policy_version 1545233 (0.0008) [2023-12-27 16:21:46,522][20381] Updated weights for policy 1, policy_version 1545243 (0.0008) [2023-12-27 16:21:46,569][20381] Updated weights for policy 1, policy_version 1545253 (0.0006) [2023-12-27 16:21:46,746][20347] Updated weights for policy 0, policy_version 1546870 (0.0008) [2023-12-27 16:21:46,803][20347] Updated weights for policy 0, policy_version 1546880 (0.0005) [2023-12-27 16:21:46,851][20347] Updated weights for policy 0, policy_version 1546890 (0.0005) [2023-12-27 16:21:47,214][20381] Updated weights for policy 1, policy_version 1545263 (0.0008) [2023-12-27 16:21:47,263][20381] Updated weights for policy 1, policy_version 1545273 (0.0008) [2023-12-27 16:21:47,308][20381] Updated weights for policy 1, policy_version 1545283 (0.0010) [2023-12-27 16:21:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20688.3). Total num frames: 791715840. Throughput: 0: 10343.4, 1: 10167.4. Samples: 791722720. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:47,364][19170] Avg episode reward: [(0, '567.630'), (1, '402.240')] [2023-12-27 16:21:47,484][20347] Updated weights for policy 0, policy_version 1546900 (0.0006) [2023-12-27 16:21:47,535][20347] Updated weights for policy 0, policy_version 1546910 (0.0005) [2023-12-27 16:21:47,586][20347] Updated weights for policy 0, policy_version 1546920 (0.0007) [2023-12-27 16:21:47,978][20381] Updated weights for policy 1, policy_version 1545293 (0.0009) [2023-12-27 16:21:48,034][20381] Updated weights for policy 1, policy_version 1545303 (0.0007) [2023-12-27 16:21:48,085][20381] Updated weights for policy 1, policy_version 1545313 (0.0007) [2023-12-27 16:21:48,189][20347] Updated weights for policy 0, policy_version 1546930 (0.0008) [2023-12-27 16:21:48,247][20347] Updated weights for policy 0, policy_version 1546940 (0.0010) [2023-12-27 16:21:48,294][20347] Updated weights for policy 0, policy_version 1546950 (0.0009) [2023-12-27 16:21:48,352][20347] Updated weights for policy 0, policy_version 1546960 (0.0006) [2023-12-27 16:21:48,722][20381] Updated weights for policy 1, policy_version 1545323 (0.0007) [2023-12-27 16:21:48,786][20381] Updated weights for policy 1, policy_version 1545333 (0.0007) [2023-12-27 16:21:48,854][20381] Updated weights for policy 1, policy_version 1545343 (0.0006) [2023-12-27 16:21:49,038][20347] Updated weights for policy 0, policy_version 1546970 (0.0006) [2023-12-27 16:21:49,087][20347] Updated weights for policy 0, policy_version 1546980 (0.0010) [2023-12-27 16:21:49,138][20347] Updated weights for policy 0, policy_version 1546990 (0.0009) [2023-12-27 16:21:49,439][20381] Updated weights for policy 1, policy_version 1545353 (0.0006) [2023-12-27 16:21:49,498][20381] Updated weights for policy 1, policy_version 1545363 (0.0010) [2023-12-27 16:21:49,557][20381] Updated weights for policy 1, policy_version 1545373 (0.0010) [2023-12-27 16:21:49,613][20381] Updated weights for policy 1, policy_version 1545383 (0.0011) [2023-12-27 16:21:49,769][20347] Updated weights for policy 0, policy_version 1547000 (0.0006) [2023-12-27 16:21:49,843][20347] Updated weights for policy 0, policy_version 1547011 (0.0008) [2023-12-27 16:21:49,911][20347] Updated weights for policy 0, policy_version 1547021 (0.0008) [2023-12-27 16:21:50,340][20381] Updated weights for policy 1, policy_version 1545393 (0.0011) [2023-12-27 16:21:50,398][20381] Updated weights for policy 1, policy_version 1545403 (0.0011) [2023-12-27 16:21:50,455][20381] Updated weights for policy 1, policy_version 1545413 (0.0011) [2023-12-27 16:21:50,600][20347] Updated weights for policy 0, policy_version 1547031 (0.0008) [2023-12-27 16:21:50,649][20347] Updated weights for policy 0, policy_version 1547041 (0.0005) [2023-12-27 16:21:50,710][20347] Updated weights for policy 0, policy_version 1547051 (0.0007) [2023-12-27 16:21:51,163][20381] Updated weights for policy 1, policy_version 1545423 (0.0009) [2023-12-27 16:21:51,219][20381] Updated weights for policy 1, policy_version 1545433 (0.0007) [2023-12-27 16:21:51,275][20381] Updated weights for policy 1, policy_version 1545443 (0.0009) [2023-12-27 16:21:51,383][20347] Updated weights for policy 0, policy_version 1547061 (0.0007) [2023-12-27 16:21:51,441][20347] Updated weights for policy 0, policy_version 1547071 (0.0006) [2023-12-27 16:21:51,495][20347] Updated weights for policy 0, policy_version 1547081 (0.0006) [2023-12-27 16:21:52,040][20381] Updated weights for policy 1, policy_version 1545453 (0.0008) [2023-12-27 16:21:52,091][20381] Updated weights for policy 1, policy_version 1545463 (0.0007) [2023-12-27 16:21:52,154][20381] Updated weights for policy 1, policy_version 1545473 (0.0007) [2023-12-27 16:21:52,222][20347] Updated weights for policy 0, policy_version 1547091 (0.0008) [2023-12-27 16:21:52,283][20347] Updated weights for policy 0, policy_version 1547101 (0.0007) [2023-12-27 16:21:52,342][20347] Updated weights for policy 0, policy_version 1547111 (0.0008) [2023-12-27 16:21:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 791814144. Throughput: 0: 10394.5, 1: 10165.2. Samples: 791786896. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:52,363][19170] Avg episode reward: [(0, '535.310'), (1, '441.350')] [2023-12-27 16:21:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001545480_395681792.pth... [2023-12-27 16:21:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001544264_395370496.pth [2023-12-27 16:21:52,398][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001547120_396140544.pth... [2023-12-27 16:21:52,402][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001545904_395829248.pth [2023-12-27 16:21:52,854][20381] Updated weights for policy 1, policy_version 1545483 (0.0007) [2023-12-27 16:21:52,909][20381] Updated weights for policy 1, policy_version 1545493 (0.0008) [2023-12-27 16:21:52,965][20381] Updated weights for policy 1, policy_version 1545503 (0.0007) [2023-12-27 16:21:53,048][20347] Updated weights for policy 0, policy_version 1547121 (0.0008) [2023-12-27 16:21:53,096][20347] Updated weights for policy 0, policy_version 1547131 (0.0010) [2023-12-27 16:21:53,153][20347] Updated weights for policy 0, policy_version 1547141 (0.0010) [2023-12-27 16:21:53,210][20347] Updated weights for policy 0, policy_version 1547151 (0.0010) [2023-12-27 16:21:53,533][20381] Updated weights for policy 1, policy_version 1545513 (0.0007) [2023-12-27 16:21:53,596][20381] Updated weights for policy 1, policy_version 1545523 (0.0006) [2023-12-27 16:21:53,647][20381] Updated weights for policy 1, policy_version 1545533 (0.0005) [2023-12-27 16:21:53,698][20381] Updated weights for policy 1, policy_version 1545543 (0.0005) [2023-12-27 16:21:53,834][20347] Updated weights for policy 0, policy_version 1547161 (0.0007) [2023-12-27 16:21:53,891][20347] Updated weights for policy 0, policy_version 1547171 (0.0005) [2023-12-27 16:21:53,953][20347] Updated weights for policy 0, policy_version 1547181 (0.0005) [2023-12-27 16:21:54,256][20381] Updated weights for policy 1, policy_version 1545553 (0.0007) [2023-12-27 16:21:54,311][20381] Updated weights for policy 1, policy_version 1545563 (0.0007) [2023-12-27 16:21:54,363][20381] Updated weights for policy 1, policy_version 1545573 (0.0007) [2023-12-27 16:21:54,561][20347] Updated weights for policy 0, policy_version 1547191 (0.0007) [2023-12-27 16:21:54,621][20347] Updated weights for policy 0, policy_version 1547201 (0.0008) [2023-12-27 16:21:54,669][20347] Updated weights for policy 0, policy_version 1547211 (0.0007) [2023-12-27 16:21:55,062][20381] Updated weights for policy 1, policy_version 1545583 (0.0008) [2023-12-27 16:21:55,121][20381] Updated weights for policy 1, policy_version 1545593 (0.0008) [2023-12-27 16:21:55,179][20381] Updated weights for policy 1, policy_version 1545603 (0.0008) [2023-12-27 16:21:55,336][20347] Updated weights for policy 0, policy_version 1547221 (0.0008) [2023-12-27 16:21:55,385][20347] Updated weights for policy 0, policy_version 1547231 (0.0008) [2023-12-27 16:21:55,438][20347] Updated weights for policy 0, policy_version 1547241 (0.0008) [2023-12-27 16:21:55,762][20381] Updated weights for policy 1, policy_version 1545613 (0.0007) [2023-12-27 16:21:55,814][20381] Updated weights for policy 1, policy_version 1545623 (0.0007) [2023-12-27 16:21:55,872][20381] Updated weights for policy 1, policy_version 1545633 (0.0008) [2023-12-27 16:21:56,129][20347] Updated weights for policy 0, policy_version 1547251 (0.0007) [2023-12-27 16:21:56,187][20347] Updated weights for policy 0, policy_version 1547261 (0.0008) [2023-12-27 16:21:56,247][20347] Updated weights for policy 0, policy_version 1547271 (0.0007) [2023-12-27 16:21:56,571][20381] Updated weights for policy 1, policy_version 1545643 (0.0008) [2023-12-27 16:21:56,629][20381] Updated weights for policy 1, policy_version 1545653 (0.0007) [2023-12-27 16:21:56,687][20381] Updated weights for policy 1, policy_version 1545663 (0.0008) [2023-12-27 16:21:56,941][20347] Updated weights for policy 0, policy_version 1547281 (0.0008) [2023-12-27 16:21:56,995][20347] Updated weights for policy 0, policy_version 1547291 (0.0007) [2023-12-27 16:21:57,046][20347] Updated weights for policy 0, policy_version 1547301 (0.0007) [2023-12-27 16:21:57,100][20347] Updated weights for policy 0, policy_version 1547311 (0.0008) [2023-12-27 16:21:57,323][20381] Updated weights for policy 1, policy_version 1545673 (0.0008) [2023-12-27 16:21:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 791920640. Throughput: 0: 10306.8, 1: 10289.4. Samples: 791912328. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:21:57,363][19170] Avg episode reward: [(0, '440.870'), (1, '329.410')] [2023-12-27 16:21:57,375][20381] Updated weights for policy 1, policy_version 1545683 (0.0007) [2023-12-27 16:21:57,428][20381] Updated weights for policy 1, policy_version 1545693 (0.0008) [2023-12-27 16:21:57,487][20381] Updated weights for policy 1, policy_version 1545703 (0.0008) [2023-12-27 16:21:57,767][20347] Updated weights for policy 0, policy_version 1547321 (0.0006) [2023-12-27 16:21:57,825][20347] Updated weights for policy 0, policy_version 1547331 (0.0005) [2023-12-27 16:21:57,875][20347] Updated weights for policy 0, policy_version 1547341 (0.0005) [2023-12-27 16:21:58,242][20381] Updated weights for policy 1, policy_version 1545713 (0.0010) [2023-12-27 16:21:58,298][20381] Updated weights for policy 1, policy_version 1545723 (0.0007) [2023-12-27 16:21:58,356][20381] Updated weights for policy 1, policy_version 1545733 (0.0008) [2023-12-27 16:21:58,471][20347] Updated weights for policy 0, policy_version 1547351 (0.0007) [2023-12-27 16:21:58,525][20347] Updated weights for policy 0, policy_version 1547361 (0.0008) [2023-12-27 16:21:58,584][20347] Updated weights for policy 0, policy_version 1547371 (0.0008) [2023-12-27 16:21:59,166][20381] Updated weights for policy 1, policy_version 1545743 (0.0010) [2023-12-27 16:21:59,226][20381] Updated weights for policy 1, policy_version 1545753 (0.0008) [2023-12-27 16:21:59,291][20381] Updated weights for policy 1, policy_version 1545763 (0.0007) [2023-12-27 16:21:59,332][20347] Updated weights for policy 0, policy_version 1547381 (0.0008) [2023-12-27 16:21:59,394][20347] Updated weights for policy 0, policy_version 1547391 (0.0007) [2023-12-27 16:21:59,451][20347] Updated weights for policy 0, policy_version 1547401 (0.0008) [2023-12-27 16:22:00,004][20381] Updated weights for policy 1, policy_version 1545773 (0.0006) [2023-12-27 16:22:00,057][20381] Updated weights for policy 1, policy_version 1545783 (0.0005) [2023-12-27 16:22:00,111][20381] Updated weights for policy 1, policy_version 1545793 (0.0007) [2023-12-27 16:22:00,158][20347] Updated weights for policy 0, policy_version 1547411 (0.0008) [2023-12-27 16:22:00,215][20347] Updated weights for policy 0, policy_version 1547421 (0.0008) [2023-12-27 16:22:00,264][20347] Updated weights for policy 0, policy_version 1547431 (0.0008) [2023-12-27 16:22:00,752][20381] Updated weights for policy 1, policy_version 1545803 (0.0007) [2023-12-27 16:22:00,799][20381] Updated weights for policy 1, policy_version 1545813 (0.0010) [2023-12-27 16:22:00,847][20381] Updated weights for policy 1, policy_version 1545823 (0.0010) [2023-12-27 16:22:00,893][20347] Updated weights for policy 0, policy_version 1547441 (0.0006) [2023-12-27 16:22:00,943][20347] Updated weights for policy 0, policy_version 1547451 (0.0005) [2023-12-27 16:22:00,991][20347] Updated weights for policy 0, policy_version 1547461 (0.0005) [2023-12-27 16:22:01,058][20347] Updated weights for policy 0, policy_version 1547471 (0.0007) [2023-12-27 16:22:01,567][20381] Updated weights for policy 1, policy_version 1545833 (0.0010) [2023-12-27 16:22:01,629][20381] Updated weights for policy 1, policy_version 1545843 (0.0010) [2023-12-27 16:22:01,682][20381] Updated weights for policy 1, policy_version 1545853 (0.0007) [2023-12-27 16:22:01,740][20381] Updated weights for policy 1, policy_version 1545863 (0.0008) [2023-12-27 16:22:01,767][20347] Updated weights for policy 0, policy_version 1547481 (0.0008) [2023-12-27 16:22:01,822][20347] Updated weights for policy 0, policy_version 1547491 (0.0008) [2023-12-27 16:22:01,869][20347] Updated weights for policy 0, policy_version 1547501 (0.0008) [2023-12-27 16:22:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20660.5). Total num frames: 792018944. Throughput: 0: 10296.5, 1: 10238.2. Samples: 792032924. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:22:02,363][19170] Avg episode reward: [(0, '387.620'), (1, '163.700')] [2023-12-27 16:22:02,425][20381] Updated weights for policy 1, policy_version 1545873 (0.0008) [2023-12-27 16:22:02,473][20381] Updated weights for policy 1, policy_version 1545883 (0.0008) [2023-12-27 16:22:02,520][20381] Updated weights for policy 1, policy_version 1545893 (0.0008) [2023-12-27 16:22:02,573][20347] Updated weights for policy 0, policy_version 1547511 (0.0007) [2023-12-27 16:22:02,632][20347] Updated weights for policy 0, policy_version 1547521 (0.0008) [2023-12-27 16:22:02,693][20347] Updated weights for policy 0, policy_version 1547531 (0.0008) [2023-12-27 16:22:03,198][20381] Updated weights for policy 1, policy_version 1545903 (0.0007) [2023-12-27 16:22:03,245][20381] Updated weights for policy 1, policy_version 1545913 (0.0005) [2023-12-27 16:22:03,292][20381] Updated weights for policy 1, policy_version 1545923 (0.0007) [2023-12-27 16:22:03,394][20347] Updated weights for policy 0, policy_version 1547541 (0.0007) [2023-12-27 16:22:03,445][20347] Updated weights for policy 0, policy_version 1547551 (0.0008) [2023-12-27 16:22:03,496][20347] Updated weights for policy 0, policy_version 1547561 (0.0007) [2023-12-27 16:22:03,950][20381] Updated weights for policy 1, policy_version 1545933 (0.0007) [2023-12-27 16:22:04,006][20381] Updated weights for policy 1, policy_version 1545943 (0.0008) [2023-12-27 16:22:04,072][20381] Updated weights for policy 1, policy_version 1545953 (0.0008) [2023-12-27 16:22:04,220][20347] Updated weights for policy 0, policy_version 1547571 (0.0006) [2023-12-27 16:22:04,288][20347] Updated weights for policy 0, policy_version 1547581 (0.0007) [2023-12-27 16:22:04,353][20347] Updated weights for policy 0, policy_version 1547591 (0.0008) [2023-12-27 16:22:04,709][20381] Updated weights for policy 1, policy_version 1545963 (0.0006) [2023-12-27 16:22:04,757][20381] Updated weights for policy 1, policy_version 1545973 (0.0008) [2023-12-27 16:22:04,810][20381] Updated weights for policy 1, policy_version 1545983 (0.0007) [2023-12-27 16:22:05,015][20347] Updated weights for policy 0, policy_version 1547601 (0.0009) [2023-12-27 16:22:05,083][20347] Updated weights for policy 0, policy_version 1547611 (0.0009) [2023-12-27 16:22:05,151][20347] Updated weights for policy 0, policy_version 1547621 (0.0007) [2023-12-27 16:22:05,217][20347] Updated weights for policy 0, policy_version 1547631 (0.0010) [2023-12-27 16:22:05,462][20381] Updated weights for policy 1, policy_version 1545993 (0.0008) [2023-12-27 16:22:05,517][20381] Updated weights for policy 1, policy_version 1546003 (0.0007) [2023-12-27 16:22:05,572][20381] Updated weights for policy 1, policy_version 1546013 (0.0008) [2023-12-27 16:22:05,627][20381] Updated weights for policy 1, policy_version 1546023 (0.0008) [2023-12-27 16:22:05,863][20347] Updated weights for policy 0, policy_version 1547641 (0.0010) [2023-12-27 16:22:05,923][20347] Updated weights for policy 0, policy_version 1547651 (0.0007) [2023-12-27 16:22:05,980][20347] Updated weights for policy 0, policy_version 1547661 (0.0008) [2023-12-27 16:22:06,307][20381] Updated weights for policy 1, policy_version 1546033 (0.0008) [2023-12-27 16:22:06,361][20381] Updated weights for policy 1, policy_version 1546043 (0.0008) [2023-12-27 16:22:06,413][20381] Updated weights for policy 1, policy_version 1546053 (0.0007) [2023-12-27 16:22:06,631][20347] Updated weights for policy 0, policy_version 1547671 (0.0010) [2023-12-27 16:22:06,691][20347] Updated weights for policy 0, policy_version 1547681 (0.0010) [2023-12-27 16:22:06,749][20347] Updated weights for policy 0, policy_version 1547691 (0.0010) [2023-12-27 16:22:07,125][20381] Updated weights for policy 1, policy_version 1546063 (0.0010) [2023-12-27 16:22:07,178][20381] Updated weights for policy 1, policy_version 1546073 (0.0011) [2023-12-27 16:22:07,237][20381] Updated weights for policy 1, policy_version 1546083 (0.0009) [2023-12-27 16:22:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20688.3). Total num frames: 792125440. Throughput: 0: 10303.9, 1: 10287.0. Samples: 792094024. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:22:07,363][19170] Avg episode reward: [(0, '356.360'), (1, '430.240')] [2023-12-27 16:22:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001547696_396288000.pth... [2023-12-27 16:22:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001546088_395837440.pth... [2023-12-27 16:22:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001546512_395984896.pth [2023-12-27 16:22:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001544904_395534336.pth [2023-12-27 16:22:07,439][20347] Updated weights for policy 0, policy_version 1547701 (0.0010) [2023-12-27 16:22:07,500][20347] Updated weights for policy 0, policy_version 1547711 (0.0010) [2023-12-27 16:22:07,544][20347] Updated weights for policy 0, policy_version 1547721 (0.0010) [2023-12-27 16:22:07,861][20381] Updated weights for policy 1, policy_version 1546093 (0.0008) [2023-12-27 16:22:07,910][20381] Updated weights for policy 1, policy_version 1546103 (0.0006) [2023-12-27 16:22:07,967][20381] Updated weights for policy 1, policy_version 1546113 (0.0007) [2023-12-27 16:22:08,226][20347] Updated weights for policy 0, policy_version 1547731 (0.0010) [2023-12-27 16:22:08,287][20347] Updated weights for policy 0, policy_version 1547741 (0.0010) [2023-12-27 16:22:08,351][20347] Updated weights for policy 0, policy_version 1547751 (0.0010) [2023-12-27 16:22:08,621][20381] Updated weights for policy 1, policy_version 1546123 (0.0007) [2023-12-27 16:22:08,677][20381] Updated weights for policy 1, policy_version 1546133 (0.0005) [2023-12-27 16:22:08,737][20381] Updated weights for policy 1, policy_version 1546143 (0.0006) [2023-12-27 16:22:09,029][20347] Updated weights for policy 0, policy_version 1547761 (0.0010) [2023-12-27 16:22:09,087][20347] Updated weights for policy 0, policy_version 1547771 (0.0010) [2023-12-27 16:22:09,141][20347] Updated weights for policy 0, policy_version 1547781 (0.0010) [2023-12-27 16:22:09,191][20347] Updated weights for policy 0, policy_version 1547791 (0.0010) [2023-12-27 16:22:09,364][20381] Updated weights for policy 1, policy_version 1546153 (0.0007) [2023-12-27 16:22:09,433][20381] Updated weights for policy 1, policy_version 1546163 (0.0008) [2023-12-27 16:22:09,493][20381] Updated weights for policy 1, policy_version 1546173 (0.0009) [2023-12-27 16:22:09,544][20381] Updated weights for policy 1, policy_version 1546183 (0.0007) [2023-12-27 16:22:09,970][20347] Updated weights for policy 0, policy_version 1547801 (0.0011) [2023-12-27 16:22:10,036][20347] Updated weights for policy 0, policy_version 1547811 (0.0011) [2023-12-27 16:22:10,098][20347] Updated weights for policy 0, policy_version 1547821 (0.0010) [2023-12-27 16:22:10,302][20381] Updated weights for policy 1, policy_version 1546193 (0.0007) [2023-12-27 16:22:10,363][20381] Updated weights for policy 1, policy_version 1546203 (0.0007) [2023-12-27 16:22:10,426][20381] Updated weights for policy 1, policy_version 1546213 (0.0008) [2023-12-27 16:22:10,769][20347] Updated weights for policy 0, policy_version 1547831 (0.0010) [2023-12-27 16:22:10,820][20347] Updated weights for policy 0, policy_version 1547841 (0.0010) [2023-12-27 16:22:10,868][20347] Updated weights for policy 0, policy_version 1547851 (0.0010) [2023-12-27 16:22:11,042][20381] Updated weights for policy 1, policy_version 1546223 (0.0007) [2023-12-27 16:22:11,105][20381] Updated weights for policy 1, policy_version 1546233 (0.0009) [2023-12-27 16:22:11,178][20381] Updated weights for policy 1, policy_version 1546243 (0.0007) [2023-12-27 16:22:11,583][20347] Updated weights for policy 0, policy_version 1547861 (0.0009) [2023-12-27 16:22:11,652][20347] Updated weights for policy 0, policy_version 1547871 (0.0007) [2023-12-27 16:22:11,714][20347] Updated weights for policy 0, policy_version 1547881 (0.0007) [2023-12-27 16:22:11,914][20381] Updated weights for policy 1, policy_version 1546253 (0.0008) [2023-12-27 16:22:11,963][20381] Updated weights for policy 1, policy_version 1546263 (0.0010) [2023-12-27 16:22:12,019][20381] Updated weights for policy 1, policy_version 1546273 (0.0011) [2023-12-27 16:22:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 792223744. Throughput: 0: 10262.7, 1: 10275.9. Samples: 792215956. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-12-27 16:22:12,363][19170] Avg episode reward: [(0, '378.330'), (1, '349.500')] [2023-12-27 16:22:12,392][20347] Updated weights for policy 0, policy_version 1547891 (0.0008) [2023-12-27 16:22:12,446][20347] Updated weights for policy 0, policy_version 1547901 (0.0005) [2023-12-27 16:22:12,507][20347] Updated weights for policy 0, policy_version 1547911 (0.0005) [2023-12-27 16:22:12,732][20381] Updated weights for policy 1, policy_version 1546283 (0.0011) [2023-12-27 16:22:12,791][20381] Updated weights for policy 1, policy_version 1546293 (0.0010) [2023-12-27 16:22:12,847][20381] Updated weights for policy 1, policy_version 1546303 (0.0011) [2023-12-27 16:22:13,124][20347] Updated weights for policy 0, policy_version 1547921 (0.0006) [2023-12-27 16:22:13,181][20347] Updated weights for policy 0, policy_version 1547931 (0.0008) [2023-12-27 16:22:13,236][20347] Updated weights for policy 0, policy_version 1547941 (0.0008) [2023-12-27 16:22:13,293][20347] Updated weights for policy 0, policy_version 1547951 (0.0006) [2023-12-27 16:22:13,529][20381] Updated weights for policy 1, policy_version 1546313 (0.0010) [2023-12-27 16:22:13,587][20381] Updated weights for policy 1, policy_version 1546323 (0.0008) [2023-12-27 16:22:13,641][20381] Updated weights for policy 1, policy_version 1546333 (0.0010) [2023-12-27 16:22:13,689][20381] Updated weights for policy 1, policy_version 1546343 (0.0010) [2023-12-27 16:22:13,894][20347] Updated weights for policy 0, policy_version 1547961 (0.0005) [2023-12-27 16:22:13,950][20347] Updated weights for policy 0, policy_version 1547971 (0.0005) [2023-12-27 16:22:14,004][20347] Updated weights for policy 0, policy_version 1547981 (0.0005) [2023-12-27 16:22:14,306][20381] Updated weights for policy 1, policy_version 1546353 (0.0006) [2023-12-27 16:22:14,368][20381] Updated weights for policy 1, policy_version 1546363 (0.0006) [2023-12-27 16:22:14,423][20381] Updated weights for policy 1, policy_version 1546373 (0.0005) [2023-12-27 16:22:14,626][20347] Updated weights for policy 0, policy_version 1547991 (0.0007) [2023-12-27 16:22:14,680][20347] Updated weights for policy 0, policy_version 1548001 (0.0010) [2023-12-27 16:22:14,736][20347] Updated weights for policy 0, policy_version 1548011 (0.0007) [2023-12-27 16:22:15,103][20381] Updated weights for policy 1, policy_version 1546383 (0.0011) [2023-12-27 16:22:15,164][20381] Updated weights for policy 1, policy_version 1546393 (0.0011) [2023-12-27 16:22:15,220][20381] Updated weights for policy 1, policy_version 1546403 (0.0011) [2023-12-27 16:22:15,312][20347] Updated weights for policy 0, policy_version 1548021 (0.0006) [2023-12-27 16:22:15,366][20347] Updated weights for policy 0, policy_version 1548031 (0.0009) [2023-12-27 16:22:15,422][20347] Updated weights for policy 0, policy_version 1548041 (0.0010) [2023-12-27 16:22:15,910][20381] Updated weights for policy 1, policy_version 1546413 (0.0011) [2023-12-27 16:22:15,959][20381] Updated weights for policy 1, policy_version 1546423 (0.0010) [2023-12-27 16:22:16,007][20381] Updated weights for policy 1, policy_version 1546433 (0.0010) [2023-12-27 16:22:16,052][20347] Updated weights for policy 0, policy_version 1548051 (0.0009) [2023-12-27 16:22:16,107][20347] Updated weights for policy 0, policy_version 1548061 (0.0007) [2023-12-27 16:22:16,160][20347] Updated weights for policy 0, policy_version 1548071 (0.0007) [2023-12-27 16:22:16,717][20381] Updated weights for policy 1, policy_version 1546443 (0.0010) [2023-12-27 16:22:16,766][20381] Updated weights for policy 1, policy_version 1546453 (0.0009) [2023-12-27 16:22:16,776][20347] Updated weights for policy 0, policy_version 1548081 (0.0007) [2023-12-27 16:22:16,824][20381] Updated weights for policy 1, policy_version 1546463 (0.0005) [2023-12-27 16:22:16,841][20347] Updated weights for policy 0, policy_version 1548091 (0.0006) [2023-12-27 16:22:16,908][20347] Updated weights for policy 0, policy_version 1548101 (0.0007) [2023-12-27 16:22:16,975][20347] Updated weights for policy 0, policy_version 1548111 (0.0006) [2023-12-27 16:22:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 792330240. Throughput: 0: 10358.1, 1: 10220.4. Samples: 792341912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:17,363][19170] Avg episode reward: [(0, '461.300'), (1, '312.490')] [2023-12-27 16:22:17,472][20381] Updated weights for policy 1, policy_version 1546473 (0.0005) [2023-12-27 16:22:17,534][20381] Updated weights for policy 1, policy_version 1546483 (0.0005) [2023-12-27 16:22:17,547][20347] Updated weights for policy 0, policy_version 1548121 (0.0005) [2023-12-27 16:22:17,593][20381] Updated weights for policy 1, policy_version 1546493 (0.0007) [2023-12-27 16:22:17,602][20347] Updated weights for policy 0, policy_version 1548131 (0.0006) [2023-12-27 16:22:17,639][20381] Updated weights for policy 1, policy_version 1546503 (0.0005) [2023-12-27 16:22:17,648][20347] Updated weights for policy 0, policy_version 1548141 (0.0005) [2023-12-27 16:22:18,270][20381] Updated weights for policy 1, policy_version 1546513 (0.0005) [2023-12-27 16:22:18,319][20381] Updated weights for policy 1, policy_version 1546523 (0.0005) [2023-12-27 16:22:18,333][20347] Updated weights for policy 0, policy_version 1548151 (0.0008) [2023-12-27 16:22:18,375][20381] Updated weights for policy 1, policy_version 1546533 (0.0007) [2023-12-27 16:22:18,396][20347] Updated weights for policy 0, policy_version 1548161 (0.0007) [2023-12-27 16:22:18,455][20347] Updated weights for policy 0, policy_version 1548171 (0.0007) [2023-12-27 16:22:19,094][20381] Updated weights for policy 1, policy_version 1546543 (0.0005) [2023-12-27 16:22:19,117][20347] Updated weights for policy 0, policy_version 1548181 (0.0007) [2023-12-27 16:22:19,148][20381] Updated weights for policy 1, policy_version 1546553 (0.0005) [2023-12-27 16:22:19,173][20347] Updated weights for policy 0, policy_version 1548191 (0.0007) [2023-12-27 16:22:19,203][20381] Updated weights for policy 1, policy_version 1546563 (0.0006) [2023-12-27 16:22:19,233][20347] Updated weights for policy 0, policy_version 1548201 (0.0008) [2023-12-27 16:22:19,814][20381] Updated weights for policy 1, policy_version 1546573 (0.0007) [2023-12-27 16:22:19,886][20381] Updated weights for policy 1, policy_version 1546583 (0.0010) [2023-12-27 16:22:19,948][20381] Updated weights for policy 1, policy_version 1546593 (0.0009) [2023-12-27 16:22:20,032][20347] Updated weights for policy 0, policy_version 1548211 (0.0008) [2023-12-27 16:22:20,092][20347] Updated weights for policy 0, policy_version 1548221 (0.0009) [2023-12-27 16:22:20,150][20347] Updated weights for policy 0, policy_version 1548231 (0.0009) [2023-12-27 16:22:20,794][20381] Updated weights for policy 1, policy_version 1546603 (0.0009) [2023-12-27 16:22:20,850][20381] Updated weights for policy 1, policy_version 1546613 (0.0011) [2023-12-27 16:22:20,889][20347] Updated weights for policy 0, policy_version 1548241 (0.0008) [2023-12-27 16:22:20,911][20381] Updated weights for policy 1, policy_version 1546623 (0.0010) [2023-12-27 16:22:20,952][20347] Updated weights for policy 0, policy_version 1548251 (0.0007) [2023-12-27 16:22:21,014][20347] Updated weights for policy 0, policy_version 1548261 (0.0008) [2023-12-27 16:22:21,084][20347] Updated weights for policy 0, policy_version 1548271 (0.0008) [2023-12-27 16:22:21,649][20381] Updated weights for policy 1, policy_version 1546633 (0.0010) [2023-12-27 16:22:21,712][20381] Updated weights for policy 1, policy_version 1546643 (0.0007) [2023-12-27 16:22:21,783][20381] Updated weights for policy 1, policy_version 1546653 (0.0012) [2023-12-27 16:22:21,840][20381] Updated weights for policy 1, policy_version 1546663 (0.0010) [2023-12-27 16:22:21,856][20347] Updated weights for policy 0, policy_version 1548281 (0.0006) [2023-12-27 16:22:21,922][20347] Updated weights for policy 0, policy_version 1548291 (0.0006) [2023-12-27 16:22:21,986][20347] Updated weights for policy 0, policy_version 1548301 (0.0006) [2023-12-27 16:22:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 792428544. Throughput: 0: 10330.2, 1: 10215.6. Samples: 792403072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:22,363][19170] Avg episode reward: [(0, '397.740'), (1, '478.880')] [2023-12-27 16:22:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001546664_395984896.pth... [2023-12-27 16:22:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001548304_396443648.pth... [2023-12-27 16:22:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001545480_395681792.pth [2023-12-27 16:22:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001547120_396140544.pth [2023-12-27 16:22:22,546][20381] Updated weights for policy 1, policy_version 1546673 (0.0011) [2023-12-27 16:22:22,574][20347] Updated weights for policy 0, policy_version 1548311 (0.0006) [2023-12-27 16:22:22,600][20381] Updated weights for policy 1, policy_version 1546683 (0.0011) [2023-12-27 16:22:22,635][20347] Updated weights for policy 0, policy_version 1548321 (0.0008) [2023-12-27 16:22:22,661][20381] Updated weights for policy 1, policy_version 1546693 (0.0011) [2023-12-27 16:22:22,686][20347] Updated weights for policy 0, policy_version 1548331 (0.0006) [2023-12-27 16:22:23,339][20381] Updated weights for policy 1, policy_version 1546703 (0.0010) [2023-12-27 16:22:23,371][20347] Updated weights for policy 0, policy_version 1548341 (0.0008) [2023-12-27 16:22:23,392][20381] Updated weights for policy 1, policy_version 1546713 (0.0010) [2023-12-27 16:22:23,426][20347] Updated weights for policy 0, policy_version 1548351 (0.0010) [2023-12-27 16:22:23,441][20381] Updated weights for policy 1, policy_version 1546723 (0.0010) [2023-12-27 16:22:23,482][20347] Updated weights for policy 0, policy_version 1548361 (0.0008) [2023-12-27 16:22:24,139][20347] Updated weights for policy 0, policy_version 1548371 (0.0007) [2023-12-27 16:22:24,141][20381] Updated weights for policy 1, policy_version 1546733 (0.0010) [2023-12-27 16:22:24,194][20381] Updated weights for policy 1, policy_version 1546743 (0.0010) [2023-12-27 16:22:24,195][20347] Updated weights for policy 0, policy_version 1548381 (0.0008) [2023-12-27 16:22:24,243][20381] Updated weights for policy 1, policy_version 1546753 (0.0010) [2023-12-27 16:22:24,249][20347] Updated weights for policy 0, policy_version 1548391 (0.0006) [2023-12-27 16:22:24,804][20347] Updated weights for policy 0, policy_version 1548401 (0.0006) [2023-12-27 16:22:24,850][20381] Updated weights for policy 1, policy_version 1546763 (0.0009) [2023-12-27 16:22:24,857][20347] Updated weights for policy 0, policy_version 1548411 (0.0007) [2023-12-27 16:22:24,907][20347] Updated weights for policy 0, policy_version 1548421 (0.0006) [2023-12-27 16:22:24,910][20381] Updated weights for policy 1, policy_version 1546773 (0.0005) [2023-12-27 16:22:24,960][20381] Updated weights for policy 1, policy_version 1546783 (0.0006) [2023-12-27 16:22:24,965][20347] Updated weights for policy 0, policy_version 1548431 (0.0007) [2023-12-27 16:22:25,508][20381] Updated weights for policy 1, policy_version 1546793 (0.0010) [2023-12-27 16:22:25,560][20381] Updated weights for policy 1, policy_version 1546803 (0.0005) [2023-12-27 16:22:25,590][20347] Updated weights for policy 0, policy_version 1548441 (0.0008) [2023-12-27 16:22:25,617][20381] Updated weights for policy 1, policy_version 1546813 (0.0009) [2023-12-27 16:22:25,658][20347] Updated weights for policy 0, policy_version 1548451 (0.0008) [2023-12-27 16:22:25,680][20381] Updated weights for policy 1, policy_version 1546823 (0.0011) [2023-12-27 16:22:25,722][20347] Updated weights for policy 0, policy_version 1548461 (0.0007) [2023-12-27 16:22:26,296][20347] Updated weights for policy 0, policy_version 1548471 (0.0007) [2023-12-27 16:22:26,342][20381] Updated weights for policy 1, policy_version 1546833 (0.0011) [2023-12-27 16:22:26,353][20347] Updated weights for policy 0, policy_version 1548481 (0.0005) [2023-12-27 16:22:26,395][20381] Updated weights for policy 1, policy_version 1546843 (0.0011) [2023-12-27 16:22:26,418][20347] Updated weights for policy 0, policy_version 1548491 (0.0005) [2023-12-27 16:22:26,457][20381] Updated weights for policy 1, policy_version 1546853 (0.0008) [2023-12-27 16:22:27,036][20347] Updated weights for policy 0, policy_version 1548501 (0.0006) [2023-12-27 16:22:27,083][20347] Updated weights for policy 0, policy_version 1548511 (0.0007) [2023-12-27 16:22:27,131][20347] Updated weights for policy 0, policy_version 1548521 (0.0006) [2023-12-27 16:22:27,132][20381] Updated weights for policy 1, policy_version 1546863 (0.0010) [2023-12-27 16:22:27,187][20381] Updated weights for policy 1, policy_version 1546873 (0.0010) [2023-12-27 16:22:27,242][20381] Updated weights for policy 1, policy_version 1546883 (0.0010) [2023-12-27 16:22:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 792543232. Throughput: 0: 10318.9, 1: 10243.6. Samples: 792527140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:27,363][19170] Avg episode reward: [(0, '420.680'), (1, '460.170')] [2023-12-27 16:22:27,835][20347] Updated weights for policy 0, policy_version 1548531 (0.0006) [2023-12-27 16:22:27,895][20347] Updated weights for policy 0, policy_version 1548541 (0.0007) [2023-12-27 16:22:27,926][20381] Updated weights for policy 1, policy_version 1546893 (0.0010) [2023-12-27 16:22:27,946][20347] Updated weights for policy 0, policy_version 1548551 (0.0006) [2023-12-27 16:22:27,973][20381] Updated weights for policy 1, policy_version 1546903 (0.0010) [2023-12-27 16:22:28,024][20381] Updated weights for policy 1, policy_version 1546913 (0.0010) [2023-12-27 16:22:28,660][20347] Updated weights for policy 0, policy_version 1548561 (0.0006) [2023-12-27 16:22:28,705][20381] Updated weights for policy 1, policy_version 1546923 (0.0010) [2023-12-27 16:22:28,713][20347] Updated weights for policy 0, policy_version 1548571 (0.0006) [2023-12-27 16:22:28,756][20381] Updated weights for policy 1, policy_version 1546933 (0.0010) [2023-12-27 16:22:28,765][20347] Updated weights for policy 0, policy_version 1548581 (0.0005) [2023-12-27 16:22:28,810][20381] Updated weights for policy 1, policy_version 1546943 (0.0010) [2023-12-27 16:22:28,816][20347] Updated weights for policy 0, policy_version 1548591 (0.0006) [2023-12-27 16:22:29,519][20347] Updated weights for policy 0, policy_version 1548601 (0.0007) [2023-12-27 16:22:29,520][20381] Updated weights for policy 1, policy_version 1546953 (0.0010) [2023-12-27 16:22:29,575][20347] Updated weights for policy 0, policy_version 1548611 (0.0005) [2023-12-27 16:22:29,580][20381] Updated weights for policy 1, policy_version 1546963 (0.0006) [2023-12-27 16:22:29,630][20347] Updated weights for policy 0, policy_version 1548621 (0.0007) [2023-12-27 16:22:29,632][20381] Updated weights for policy 1, policy_version 1546973 (0.0006) [2023-12-27 16:22:29,688][20381] Updated weights for policy 1, policy_version 1546983 (0.0007) [2023-12-27 16:22:30,279][20347] Updated weights for policy 0, policy_version 1548631 (0.0006) [2023-12-27 16:22:30,341][20347] Updated weights for policy 0, policy_version 1548641 (0.0011) [2023-12-27 16:22:30,397][20347] Updated weights for policy 0, policy_version 1548651 (0.0010) [2023-12-27 16:22:30,398][20381] Updated weights for policy 1, policy_version 1546993 (0.0008) [2023-12-27 16:22:30,461][20381] Updated weights for policy 1, policy_version 1547003 (0.0008) [2023-12-27 16:22:30,518][20381] Updated weights for policy 1, policy_version 1547013 (0.0007) [2023-12-27 16:22:30,999][20347] Updated weights for policy 0, policy_version 1548661 (0.0007) [2023-12-27 16:22:31,052][20347] Updated weights for policy 0, policy_version 1548671 (0.0008) [2023-12-27 16:22:31,106][20347] Updated weights for policy 0, policy_version 1548681 (0.0008) [2023-12-27 16:22:31,191][20381] Updated weights for policy 1, policy_version 1547023 (0.0008) [2023-12-27 16:22:31,243][20381] Updated weights for policy 1, policy_version 1547033 (0.0009) [2023-12-27 16:22:31,301][20381] Updated weights for policy 1, policy_version 1547043 (0.0009) [2023-12-27 16:22:31,913][20347] Updated weights for policy 0, policy_version 1548691 (0.0008) [2023-12-27 16:22:31,970][20347] Updated weights for policy 0, policy_version 1548701 (0.0010) [2023-12-27 16:22:32,030][20347] Updated weights for policy 0, policy_version 1548711 (0.0008) [2023-12-27 16:22:32,102][20381] Updated weights for policy 1, policy_version 1547053 (0.0008) [2023-12-27 16:22:32,165][20381] Updated weights for policy 1, policy_version 1547063 (0.0005) [2023-12-27 16:22:32,227][20381] Updated weights for policy 1, policy_version 1547073 (0.0006) [2023-12-27 16:22:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 792641536. Throughput: 0: 10334.7, 1: 10240.3. Samples: 792648592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:32,363][19170] Avg episode reward: [(0, '339.680'), (1, '509.420')] [2023-12-27 16:22:32,689][20347] Updated weights for policy 0, policy_version 1548721 (0.0007) [2023-12-27 16:22:32,748][20347] Updated weights for policy 0, policy_version 1548731 (0.0009) [2023-12-27 16:22:32,799][20381] Updated weights for policy 1, policy_version 1547083 (0.0007) [2023-12-27 16:22:32,799][20347] Updated weights for policy 0, policy_version 1548741 (0.0010) [2023-12-27 16:22:32,858][20381] Updated weights for policy 1, policy_version 1547093 (0.0007) [2023-12-27 16:22:32,859][20347] Updated weights for policy 0, policy_version 1548752 (0.0008) [2023-12-27 16:22:32,911][20381] Updated weights for policy 1, policy_version 1547103 (0.0006) [2023-12-27 16:22:33,449][20347] Updated weights for policy 0, policy_version 1548762 (0.0005) [2023-12-27 16:22:33,495][20347] Updated weights for policy 0, policy_version 1548772 (0.0005) [2023-12-27 16:22:33,548][20347] Updated weights for policy 0, policy_version 1548782 (0.0005) [2023-12-27 16:22:33,557][20381] Updated weights for policy 1, policy_version 1547113 (0.0008) [2023-12-27 16:22:33,607][20381] Updated weights for policy 1, policy_version 1547123 (0.0005) [2023-12-27 16:22:33,658][20381] Updated weights for policy 1, policy_version 1547133 (0.0006) [2023-12-27 16:22:33,709][20381] Updated weights for policy 1, policy_version 1547143 (0.0007) [2023-12-27 16:22:34,140][20347] Updated weights for policy 0, policy_version 1548792 (0.0005) [2023-12-27 16:22:34,204][20347] Updated weights for policy 0, policy_version 1548802 (0.0007) [2023-12-27 16:22:34,251][20347] Updated weights for policy 0, policy_version 1548812 (0.0008) [2023-12-27 16:22:34,346][20381] Updated weights for policy 1, policy_version 1547153 (0.0008) [2023-12-27 16:22:34,414][20381] Updated weights for policy 1, policy_version 1547163 (0.0008) [2023-12-27 16:22:34,467][20381] Updated weights for policy 1, policy_version 1547173 (0.0007) [2023-12-27 16:22:34,966][20347] Updated weights for policy 0, policy_version 1548822 (0.0008) [2023-12-27 16:22:35,015][20347] Updated weights for policy 0, policy_version 1548832 (0.0008) [2023-12-27 16:22:35,071][20347] Updated weights for policy 0, policy_version 1548842 (0.0008) [2023-12-27 16:22:35,090][20381] Updated weights for policy 1, policy_version 1547183 (0.0006) [2023-12-27 16:22:35,146][20381] Updated weights for policy 1, policy_version 1547193 (0.0005) [2023-12-27 16:22:35,202][20381] Updated weights for policy 1, policy_version 1547203 (0.0006) [2023-12-27 16:22:35,806][20347] Updated weights for policy 0, policy_version 1548852 (0.0009) [2023-12-27 16:22:35,855][20381] Updated weights for policy 1, policy_version 1547213 (0.0009) [2023-12-27 16:22:35,861][20347] Updated weights for policy 0, policy_version 1548862 (0.0008) [2023-12-27 16:22:35,904][20381] Updated weights for policy 1, policy_version 1547223 (0.0005) [2023-12-27 16:22:35,913][20347] Updated weights for policy 0, policy_version 1548872 (0.0006) [2023-12-27 16:22:35,951][20381] Updated weights for policy 1, policy_version 1547233 (0.0006) [2023-12-27 16:22:36,599][20381] Updated weights for policy 1, policy_version 1547243 (0.0007) [2023-12-27 16:22:36,639][20347] Updated weights for policy 0, policy_version 1548882 (0.0006) [2023-12-27 16:22:36,662][20381] Updated weights for policy 1, policy_version 1547253 (0.0005) [2023-12-27 16:22:36,702][20347] Updated weights for policy 0, policy_version 1548892 (0.0007) [2023-12-27 16:22:36,722][20381] Updated weights for policy 1, policy_version 1547263 (0.0005) [2023-12-27 16:22:36,759][20347] Updated weights for policy 0, policy_version 1548902 (0.0007) [2023-12-27 16:22:36,822][20347] Updated weights for policy 0, policy_version 1548912 (0.0008) [2023-12-27 16:22:37,359][20381] Updated weights for policy 1, policy_version 1547273 (0.0008) [2023-12-27 16:22:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 792739840. Throughput: 0: 10326.3, 1: 10268.0. Samples: 792713640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:37,363][19170] Avg episode reward: [(0, '362.380'), (1, '499.990')] [2023-12-27 16:22:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001548912_396599296.pth... [2023-12-27 16:22:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001547696_396288000.pth [2023-12-27 16:22:37,412][20381] Updated weights for policy 1, policy_version 1547283 (0.0007) [2023-12-27 16:22:37,465][20381] Updated weights for policy 1, policy_version 1547293 (0.0008) [2023-12-27 16:22:37,484][20347] Updated weights for policy 0, policy_version 1548922 (0.0005) [2023-12-27 16:22:37,518][20381] Updated weights for policy 1, policy_version 1547303 (0.0007) [2023-12-27 16:22:37,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001547304_396148736.pth... [2023-12-27 16:22:37,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001546088_395837440.pth [2023-12-27 16:22:37,539][20347] Updated weights for policy 0, policy_version 1548932 (0.0005) [2023-12-27 16:22:37,594][20347] Updated weights for policy 0, policy_version 1548942 (0.0007) [2023-12-27 16:22:38,191][20347] Updated weights for policy 0, policy_version 1548952 (0.0006) [2023-12-27 16:22:38,239][20381] Updated weights for policy 1, policy_version 1547313 (0.0006) [2023-12-27 16:22:38,241][20347] Updated weights for policy 0, policy_version 1548962 (0.0007) [2023-12-27 16:22:38,290][20347] Updated weights for policy 0, policy_version 1548972 (0.0007) [2023-12-27 16:22:38,294][20381] Updated weights for policy 1, policy_version 1547323 (0.0005) [2023-12-27 16:22:38,360][20381] Updated weights for policy 1, policy_version 1547333 (0.0007) [2023-12-27 16:22:38,947][20381] Updated weights for policy 1, policy_version 1547343 (0.0006) [2023-12-27 16:22:38,984][20347] Updated weights for policy 0, policy_version 1548982 (0.0006) [2023-12-27 16:22:39,002][20381] Updated weights for policy 1, policy_version 1547353 (0.0006) [2023-12-27 16:22:39,040][20347] Updated weights for policy 0, policy_version 1548992 (0.0005) [2023-12-27 16:22:39,053][20381] Updated weights for policy 1, policy_version 1547363 (0.0007) [2023-12-27 16:22:39,106][20347] Updated weights for policy 0, policy_version 1549002 (0.0008) [2023-12-27 16:22:39,766][20381] Updated weights for policy 1, policy_version 1547373 (0.0006) [2023-12-27 16:22:39,794][20347] Updated weights for policy 0, policy_version 1549012 (0.0007) [2023-12-27 16:22:39,836][20381] Updated weights for policy 1, policy_version 1547383 (0.0007) [2023-12-27 16:22:39,856][20347] Updated weights for policy 0, policy_version 1549022 (0.0009) [2023-12-27 16:22:39,903][20381] Updated weights for policy 1, policy_version 1547393 (0.0007) [2023-12-27 16:22:39,920][20347] Updated weights for policy 0, policy_version 1549032 (0.0006) [2023-12-27 16:22:40,617][20381] Updated weights for policy 1, policy_version 1547403 (0.0006) [2023-12-27 16:22:40,628][20347] Updated weights for policy 0, policy_version 1549042 (0.0007) [2023-12-27 16:22:40,672][20381] Updated weights for policy 1, policy_version 1547413 (0.0007) [2023-12-27 16:22:40,681][20347] Updated weights for policy 0, policy_version 1549052 (0.0008) [2023-12-27 16:22:40,731][20381] Updated weights for policy 1, policy_version 1547423 (0.0006) [2023-12-27 16:22:40,738][20347] Updated weights for policy 0, policy_version 1549062 (0.0006) [2023-12-27 16:22:40,793][20347] Updated weights for policy 0, policy_version 1549072 (0.0007) [2023-12-27 16:22:41,455][20347] Updated weights for policy 0, policy_version 1549082 (0.0008) [2023-12-27 16:22:41,475][20381] Updated weights for policy 1, policy_version 1547433 (0.0008) [2023-12-27 16:22:41,514][20347] Updated weights for policy 0, policy_version 1549092 (0.0006) [2023-12-27 16:22:41,544][20381] Updated weights for policy 1, policy_version 1547443 (0.0008) [2023-12-27 16:22:41,583][20347] Updated weights for policy 0, policy_version 1549102 (0.0007) [2023-12-27 16:22:41,611][20381] Updated weights for policy 1, policy_version 1547453 (0.0008) [2023-12-27 16:22:41,678][20381] Updated weights for policy 1, policy_version 1547463 (0.0008) [2023-12-27 16:22:42,347][20347] Updated weights for policy 0, policy_version 1549112 (0.0010) [2023-12-27 16:22:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 792838144. Throughput: 0: 10290.3, 1: 10223.8. Samples: 792835460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:42,363][19170] Avg episode reward: [(0, '344.990'), (1, '278.340')] [2023-12-27 16:22:42,406][20347] Updated weights for policy 0, policy_version 1549122 (0.0009) [2023-12-27 16:22:42,413][20381] Updated weights for policy 1, policy_version 1547473 (0.0011) [2023-12-27 16:22:42,465][20347] Updated weights for policy 0, policy_version 1549132 (0.0008) [2023-12-27 16:22:42,470][20381] Updated weights for policy 1, policy_version 1547483 (0.0011) [2023-12-27 16:22:42,529][20381] Updated weights for policy 1, policy_version 1547493 (0.0011) [2023-12-27 16:22:43,128][20347] Updated weights for policy 0, policy_version 1549142 (0.0009) [2023-12-27 16:22:43,180][20347] Updated weights for policy 0, policy_version 1549152 (0.0010) [2023-12-27 16:22:43,189][20381] Updated weights for policy 1, policy_version 1547503 (0.0011) [2023-12-27 16:22:43,229][20347] Updated weights for policy 0, policy_version 1549162 (0.0010) [2023-12-27 16:22:43,235][20381] Updated weights for policy 1, policy_version 1547513 (0.0010) [2023-12-27 16:22:43,286][20381] Updated weights for policy 1, policy_version 1547523 (0.0010) [2023-12-27 16:22:43,842][20347] Updated weights for policy 0, policy_version 1549172 (0.0008) [2023-12-27 16:22:43,897][20347] Updated weights for policy 0, policy_version 1549182 (0.0005) [2023-12-27 16:22:43,954][20347] Updated weights for policy 0, policy_version 1549192 (0.0005) [2023-12-27 16:22:44,000][20381] Updated weights for policy 1, policy_version 1547533 (0.0011) [2023-12-27 16:22:44,052][20381] Updated weights for policy 1, policy_version 1547543 (0.0010) [2023-12-27 16:22:44,134][20381] Updated weights for policy 1, policy_version 1547553 (0.0010) [2023-12-27 16:22:44,545][20347] Updated weights for policy 0, policy_version 1549202 (0.0006) [2023-12-27 16:22:44,593][20347] Updated weights for policy 0, policy_version 1549212 (0.0010) [2023-12-27 16:22:44,644][20347] Updated weights for policy 0, policy_version 1549222 (0.0010) [2023-12-27 16:22:44,692][20347] Updated weights for policy 0, policy_version 1549232 (0.0010) [2023-12-27 16:22:44,747][20381] Updated weights for policy 1, policy_version 1547563 (0.0007) [2023-12-27 16:22:44,805][20381] Updated weights for policy 1, policy_version 1547573 (0.0007) [2023-12-27 16:22:44,867][20381] Updated weights for policy 1, policy_version 1547583 (0.0009) [2023-12-27 16:22:45,361][20347] Updated weights for policy 0, policy_version 1549242 (0.0007) [2023-12-27 16:22:45,405][20347] Updated weights for policy 0, policy_version 1549252 (0.0007) [2023-12-27 16:22:45,453][20347] Updated weights for policy 0, policy_version 1549262 (0.0007) [2023-12-27 16:22:45,600][20381] Updated weights for policy 1, policy_version 1547593 (0.0009) [2023-12-27 16:22:45,654][20381] Updated weights for policy 1, policy_version 1547603 (0.0010) [2023-12-27 16:22:45,711][20381] Updated weights for policy 1, policy_version 1547613 (0.0010) [2023-12-27 16:22:45,764][20381] Updated weights for policy 1, policy_version 1547623 (0.0010) [2023-12-27 16:22:46,216][20347] Updated weights for policy 0, policy_version 1549272 (0.0007) [2023-12-27 16:22:46,272][20347] Updated weights for policy 0, policy_version 1549282 (0.0008) [2023-12-27 16:22:46,326][20347] Updated weights for policy 0, policy_version 1549292 (0.0007) [2023-12-27 16:22:46,345][20381] Updated weights for policy 1, policy_version 1547633 (0.0008) [2023-12-27 16:22:46,405][20381] Updated weights for policy 1, policy_version 1547643 (0.0008) [2023-12-27 16:22:46,459][20381] Updated weights for policy 1, policy_version 1547653 (0.0008) [2023-12-27 16:22:47,036][20347] Updated weights for policy 0, policy_version 1549302 (0.0008) [2023-12-27 16:22:47,085][20347] Updated weights for policy 0, policy_version 1549312 (0.0008) [2023-12-27 16:22:47,113][20381] Updated weights for policy 1, policy_version 1547663 (0.0007) [2023-12-27 16:22:47,141][20347] Updated weights for policy 0, policy_version 1549322 (0.0006) [2023-12-27 16:22:47,168][20381] Updated weights for policy 1, policy_version 1547673 (0.0006) [2023-12-27 16:22:47,221][20381] Updated weights for policy 1, policy_version 1547683 (0.0008) [2023-12-27 16:22:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 792952832. Throughput: 0: 10309.9, 1: 10250.8. Samples: 792958152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:47,363][19170] Avg episode reward: [(0, '295.700'), (1, '362.100')] [2023-12-27 16:22:47,835][20381] Updated weights for policy 1, policy_version 1547693 (0.0008) [2023-12-27 16:22:47,857][20347] Updated weights for policy 0, policy_version 1549332 (0.0005) [2023-12-27 16:22:47,890][20381] Updated weights for policy 1, policy_version 1547703 (0.0006) [2023-12-27 16:22:47,908][20347] Updated weights for policy 0, policy_version 1549342 (0.0005) [2023-12-27 16:22:47,941][20381] Updated weights for policy 1, policy_version 1547713 (0.0006) [2023-12-27 16:22:47,955][20347] Updated weights for policy 0, policy_version 1549352 (0.0005) [2023-12-27 16:22:48,634][20347] Updated weights for policy 0, policy_version 1549362 (0.0008) [2023-12-27 16:22:48,661][20381] Updated weights for policy 1, policy_version 1547723 (0.0006) [2023-12-27 16:22:48,700][20347] Updated weights for policy 0, policy_version 1549372 (0.0007) [2023-12-27 16:22:48,731][20381] Updated weights for policy 1, policy_version 1547733 (0.0006) [2023-12-27 16:22:48,763][20347] Updated weights for policy 0, policy_version 1549382 (0.0006) [2023-12-27 16:22:48,801][20381] Updated weights for policy 1, policy_version 1547743 (0.0007) [2023-12-27 16:22:48,824][20347] Updated weights for policy 0, policy_version 1549392 (0.0009) [2023-12-27 16:22:49,480][20381] Updated weights for policy 1, policy_version 1547753 (0.0011) [2023-12-27 16:22:49,536][20381] Updated weights for policy 1, policy_version 1547763 (0.0010) [2023-12-27 16:22:49,539][20347] Updated weights for policy 0, policy_version 1549402 (0.0009) [2023-12-27 16:22:49,596][20381] Updated weights for policy 1, policy_version 1547773 (0.0011) [2023-12-27 16:22:49,599][20347] Updated weights for policy 0, policy_version 1549412 (0.0010) [2023-12-27 16:22:49,659][20381] Updated weights for policy 1, policy_version 1547783 (0.0011) [2023-12-27 16:22:49,662][20347] Updated weights for policy 0, policy_version 1549422 (0.0010) [2023-12-27 16:22:50,390][20347] Updated weights for policy 0, policy_version 1549432 (0.0010) [2023-12-27 16:22:50,403][20381] Updated weights for policy 1, policy_version 1547793 (0.0008) [2023-12-27 16:22:50,439][20347] Updated weights for policy 0, policy_version 1549442 (0.0008) [2023-12-27 16:22:50,460][20381] Updated weights for policy 1, policy_version 1547803 (0.0008) [2023-12-27 16:22:50,494][20347] Updated weights for policy 0, policy_version 1549452 (0.0008) [2023-12-27 16:22:50,519][20381] Updated weights for policy 1, policy_version 1547813 (0.0008) [2023-12-27 16:22:51,141][20347] Updated weights for policy 0, policy_version 1549462 (0.0007) [2023-12-27 16:22:51,207][20347] Updated weights for policy 0, policy_version 1549472 (0.0007) [2023-12-27 16:22:51,270][20347] Updated weights for policy 0, policy_version 1549482 (0.0008) [2023-12-27 16:22:51,316][20381] Updated weights for policy 1, policy_version 1547823 (0.0006) [2023-12-27 16:22:51,383][20381] Updated weights for policy 1, policy_version 1547833 (0.0008) [2023-12-27 16:22:51,440][20381] Updated weights for policy 1, policy_version 1547843 (0.0008) [2023-12-27 16:22:51,943][20347] Updated weights for policy 0, policy_version 1549492 (0.0007) [2023-12-27 16:22:52,002][20347] Updated weights for policy 0, policy_version 1549502 (0.0008) [2023-12-27 16:22:52,060][20347] Updated weights for policy 0, policy_version 1549512 (0.0007) [2023-12-27 16:22:52,165][20381] Updated weights for policy 1, policy_version 1547853 (0.0006) [2023-12-27 16:22:52,224][20381] Updated weights for policy 1, policy_version 1547863 (0.0005) [2023-12-27 16:22:52,289][20381] Updated weights for policy 1, policy_version 1547873 (0.0009) [2023-12-27 16:22:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 793051136. Throughput: 0: 10294.7, 1: 10238.7. Samples: 793018024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:52,363][19170] Avg episode reward: [(0, '185.890'), (1, '492.630')] [2023-12-27 16:22:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001547880_396296192.pth... [2023-12-27 16:22:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001549520_396754944.pth... [2023-12-27 16:22:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001546664_395984896.pth [2023-12-27 16:22:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001548304_396443648.pth [2023-12-27 16:22:52,747][20347] Updated weights for policy 0, policy_version 1549522 (0.0007) [2023-12-27 16:22:52,813][20347] Updated weights for policy 0, policy_version 1549532 (0.0007) [2023-12-27 16:22:52,863][20347] Updated weights for policy 0, policy_version 1549542 (0.0007) [2023-12-27 16:22:52,918][20347] Updated weights for policy 0, policy_version 1549552 (0.0007) [2023-12-27 16:22:53,010][20381] Updated weights for policy 1, policy_version 1547883 (0.0007) [2023-12-27 16:22:53,075][20381] Updated weights for policy 1, policy_version 1547893 (0.0005) [2023-12-27 16:22:53,136][20381] Updated weights for policy 1, policy_version 1547903 (0.0005) [2023-12-27 16:22:53,583][20347] Updated weights for policy 0, policy_version 1549562 (0.0007) [2023-12-27 16:22:53,649][20347] Updated weights for policy 0, policy_version 1549572 (0.0005) [2023-12-27 16:22:53,702][20381] Updated weights for policy 1, policy_version 1547913 (0.0006) [2023-12-27 16:22:53,708][20347] Updated weights for policy 0, policy_version 1549582 (0.0005) [2023-12-27 16:22:53,760][20381] Updated weights for policy 1, policy_version 1547923 (0.0010) [2023-12-27 16:22:53,819][20381] Updated weights for policy 1, policy_version 1547933 (0.0010) [2023-12-27 16:22:53,873][20381] Updated weights for policy 1, policy_version 1547943 (0.0011) [2023-12-27 16:22:54,243][20347] Updated weights for policy 0, policy_version 1549592 (0.0005) [2023-12-27 16:22:54,302][20347] Updated weights for policy 0, policy_version 1549602 (0.0005) [2023-12-27 16:22:54,359][20347] Updated weights for policy 0, policy_version 1549612 (0.0005) [2023-12-27 16:22:54,538][20381] Updated weights for policy 1, policy_version 1547953 (0.0007) [2023-12-27 16:22:54,597][20381] Updated weights for policy 1, policy_version 1547963 (0.0011) [2023-12-27 16:22:54,648][20381] Updated weights for policy 1, policy_version 1547973 (0.0011) [2023-12-27 16:22:54,952][20347] Updated weights for policy 0, policy_version 1549622 (0.0008) [2023-12-27 16:22:55,012][20347] Updated weights for policy 0, policy_version 1549632 (0.0007) [2023-12-27 16:22:55,070][20347] Updated weights for policy 0, policy_version 1549642 (0.0007) [2023-12-27 16:22:55,311][20381] Updated weights for policy 1, policy_version 1547983 (0.0010) [2023-12-27 16:22:55,362][20381] Updated weights for policy 1, policy_version 1547993 (0.0010) [2023-12-27 16:22:55,410][20381] Updated weights for policy 1, policy_version 1548003 (0.0010) [2023-12-27 16:22:55,776][20347] Updated weights for policy 0, policy_version 1549652 (0.0007) [2023-12-27 16:22:55,823][20347] Updated weights for policy 0, policy_version 1549662 (0.0007) [2023-12-27 16:22:55,875][20347] Updated weights for policy 0, policy_version 1549672 (0.0007) [2023-12-27 16:22:56,114][20381] Updated weights for policy 1, policy_version 1548013 (0.0010) [2023-12-27 16:22:56,165][20381] Updated weights for policy 1, policy_version 1548023 (0.0010) [2023-12-27 16:22:56,226][20381] Updated weights for policy 1, policy_version 1548033 (0.0010) [2023-12-27 16:22:56,585][20347] Updated weights for policy 0, policy_version 1549682 (0.0007) [2023-12-27 16:22:56,632][20347] Updated weights for policy 0, policy_version 1549692 (0.0007) [2023-12-27 16:22:56,682][20347] Updated weights for policy 0, policy_version 1549702 (0.0007) [2023-12-27 16:22:56,726][20347] Updated weights for policy 0, policy_version 1549712 (0.0007) [2023-12-27 16:22:56,900][20381] Updated weights for policy 1, policy_version 1548043 (0.0010) [2023-12-27 16:22:56,957][20381] Updated weights for policy 1, policy_version 1548053 (0.0010) [2023-12-27 16:22:57,018][20381] Updated weights for policy 1, policy_version 1548063 (0.0011) [2023-12-27 16:22:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793149440. Throughput: 0: 10346.7, 1: 10200.8. Samples: 793140596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:22:57,363][19170] Avg episode reward: [(0, '191.920'), (1, '550.940')] [2023-12-27 16:22:57,386][20347] Updated weights for policy 0, policy_version 1549722 (0.0007) [2023-12-27 16:22:57,452][20347] Updated weights for policy 0, policy_version 1549732 (0.0008) [2023-12-27 16:22:57,514][20347] Updated weights for policy 0, policy_version 1549742 (0.0008) [2023-12-27 16:22:57,655][20381] Updated weights for policy 1, policy_version 1548073 (0.0010) [2023-12-27 16:22:57,715][20381] Updated weights for policy 1, policy_version 1548083 (0.0006) [2023-12-27 16:22:57,774][20381] Updated weights for policy 1, policy_version 1548093 (0.0005) [2023-12-27 16:22:57,833][20381] Updated weights for policy 1, policy_version 1548103 (0.0005) [2023-12-27 16:22:58,255][20347] Updated weights for policy 0, policy_version 1549752 (0.0008) [2023-12-27 16:22:58,309][20347] Updated weights for policy 0, policy_version 1549762 (0.0007) [2023-12-27 16:22:58,388][20381] Updated weights for policy 1, policy_version 1548113 (0.0010) [2023-12-27 16:22:58,400][20347] Updated weights for policy 0, policy_version 1549773 (0.0009) [2023-12-27 16:22:58,453][20381] Updated weights for policy 1, policy_version 1548123 (0.0011) [2023-12-27 16:22:58,517][20381] Updated weights for policy 1, policy_version 1548133 (0.0011) [2023-12-27 16:22:59,230][20381] Updated weights for policy 1, policy_version 1548143 (0.0011) [2023-12-27 16:22:59,260][20347] Updated weights for policy 0, policy_version 1549783 (0.0008) [2023-12-27 16:22:59,298][20381] Updated weights for policy 1, policy_version 1548153 (0.0008) [2023-12-27 16:22:59,321][20347] Updated weights for policy 0, policy_version 1549793 (0.0007) [2023-12-27 16:22:59,360][20381] Updated weights for policy 1, policy_version 1548163 (0.0007) [2023-12-27 16:22:59,381][20347] Updated weights for policy 0, policy_version 1549803 (0.0008) [2023-12-27 16:22:59,988][20381] Updated weights for policy 1, policy_version 1548173 (0.0006) [2023-12-27 16:23:00,027][20347] Updated weights for policy 0, policy_version 1549813 (0.0007) [2023-12-27 16:23:00,049][20381] Updated weights for policy 1, policy_version 1548183 (0.0006) [2023-12-27 16:23:00,090][20347] Updated weights for policy 0, policy_version 1549823 (0.0009) [2023-12-27 16:23:00,108][20381] Updated weights for policy 1, policy_version 1548193 (0.0006) [2023-12-27 16:23:00,146][20347] Updated weights for policy 0, policy_version 1549833 (0.0011) [2023-12-27 16:23:00,699][20381] Updated weights for policy 1, policy_version 1548203 (0.0006) [2023-12-27 16:23:00,745][20381] Updated weights for policy 1, policy_version 1548213 (0.0008) [2023-12-27 16:23:00,798][20381] Updated weights for policy 1, policy_version 1548223 (0.0007) [2023-12-27 16:23:00,804][20347] Updated weights for policy 0, policy_version 1549843 (0.0009) [2023-12-27 16:23:00,858][20347] Updated weights for policy 0, policy_version 1549853 (0.0006) [2023-12-27 16:23:00,914][20347] Updated weights for policy 0, policy_version 1549863 (0.0010) [2023-12-27 16:23:01,484][20381] Updated weights for policy 1, policy_version 1548233 (0.0006) [2023-12-27 16:23:01,537][20381] Updated weights for policy 1, policy_version 1548243 (0.0008) [2023-12-27 16:23:01,594][20381] Updated weights for policy 1, policy_version 1548253 (0.0008) [2023-12-27 16:23:01,658][20381] Updated weights for policy 1, policy_version 1548263 (0.0009) [2023-12-27 16:23:01,693][20347] Updated weights for policy 0, policy_version 1549873 (0.0010) [2023-12-27 16:23:01,759][20347] Updated weights for policy 0, policy_version 1549883 (0.0008) [2023-12-27 16:23:01,815][20347] Updated weights for policy 0, policy_version 1549893 (0.0007) [2023-12-27 16:23:01,871][20347] Updated weights for policy 0, policy_version 1549903 (0.0007) [2023-12-27 16:23:02,357][20381] Updated weights for policy 1, policy_version 1548273 (0.0007) [2023-12-27 16:23:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793247744. Throughput: 0: 10207.1, 1: 10236.4. Samples: 793261868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:02,363][19170] Avg episode reward: [(0, '242.200'), (1, '655.280')] [2023-12-27 16:23:02,411][20381] Updated weights for policy 1, policy_version 1548283 (0.0007) [2023-12-27 16:23:02,461][20381] Updated weights for policy 1, policy_version 1548293 (0.0008) [2023-12-27 16:23:02,575][20347] Updated weights for policy 0, policy_version 1549913 (0.0008) [2023-12-27 16:23:02,621][20347] Updated weights for policy 0, policy_version 1549923 (0.0007) [2023-12-27 16:23:02,674][20347] Updated weights for policy 0, policy_version 1549933 (0.0008) [2023-12-27 16:23:03,129][20381] Updated weights for policy 1, policy_version 1548303 (0.0009) [2023-12-27 16:23:03,184][20381] Updated weights for policy 1, policy_version 1548313 (0.0010) [2023-12-27 16:23:03,232][20381] Updated weights for policy 1, policy_version 1548323 (0.0010) [2023-12-27 16:23:03,410][20347] Updated weights for policy 0, policy_version 1549943 (0.0007) [2023-12-27 16:23:03,460][20347] Updated weights for policy 0, policy_version 1549953 (0.0007) [2023-12-27 16:23:03,505][20347] Updated weights for policy 0, policy_version 1549963 (0.0007) [2023-12-27 16:23:03,879][20381] Updated weights for policy 1, policy_version 1548333 (0.0009) [2023-12-27 16:23:03,932][20381] Updated weights for policy 1, policy_version 1548343 (0.0007) [2023-12-27 16:23:03,986][20381] Updated weights for policy 1, policy_version 1548353 (0.0007) [2023-12-27 16:23:04,181][20347] Updated weights for policy 0, policy_version 1549973 (0.0009) [2023-12-27 16:23:04,237][20347] Updated weights for policy 0, policy_version 1549983 (0.0010) [2023-12-27 16:23:04,292][20347] Updated weights for policy 0, policy_version 1549993 (0.0010) [2023-12-27 16:23:04,709][20381] Updated weights for policy 1, policy_version 1548363 (0.0007) [2023-12-27 16:23:04,755][20381] Updated weights for policy 1, policy_version 1548373 (0.0008) [2023-12-27 16:23:04,812][20381] Updated weights for policy 1, policy_version 1548383 (0.0006) [2023-12-27 16:23:04,908][20347] Updated weights for policy 0, policy_version 1550003 (0.0009) [2023-12-27 16:23:04,958][20347] Updated weights for policy 0, policy_version 1550013 (0.0009) [2023-12-27 16:23:05,008][20347] Updated weights for policy 0, policy_version 1550023 (0.0009) [2023-12-27 16:23:05,445][20381] Updated weights for policy 1, policy_version 1548393 (0.0006) [2023-12-27 16:23:05,503][20381] Updated weights for policy 1, policy_version 1548403 (0.0007) [2023-12-27 16:23:05,555][20381] Updated weights for policy 1, policy_version 1548413 (0.0005) [2023-12-27 16:23:05,613][20381] Updated weights for policy 1, policy_version 1548423 (0.0010) [2023-12-27 16:23:05,658][20347] Updated weights for policy 0, policy_version 1550033 (0.0010) [2023-12-27 16:23:05,713][20347] Updated weights for policy 0, policy_version 1550043 (0.0007) [2023-12-27 16:23:05,765][20347] Updated weights for policy 0, policy_version 1550053 (0.0007) [2023-12-27 16:23:05,823][20347] Updated weights for policy 0, policy_version 1550063 (0.0008) [2023-12-27 16:23:06,291][20381] Updated weights for policy 1, policy_version 1548433 (0.0011) [2023-12-27 16:23:06,348][20381] Updated weights for policy 1, policy_version 1548443 (0.0011) [2023-12-27 16:23:06,407][20381] Updated weights for policy 1, policy_version 1548453 (0.0011) [2023-12-27 16:23:06,575][20347] Updated weights for policy 0, policy_version 1550073 (0.0007) [2023-12-27 16:23:06,630][20347] Updated weights for policy 0, policy_version 1550083 (0.0006) [2023-12-27 16:23:06,686][20347] Updated weights for policy 0, policy_version 1550093 (0.0007) [2023-12-27 16:23:07,107][20381] Updated weights for policy 1, policy_version 1548463 (0.0011) [2023-12-27 16:23:07,166][20381] Updated weights for policy 1, policy_version 1548473 (0.0011) [2023-12-27 16:23:07,223][20381] Updated weights for policy 1, policy_version 1548483 (0.0011) [2023-12-27 16:23:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793354240. Throughput: 0: 10214.1, 1: 10246.9. Samples: 793323820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:07,363][19170] Avg episode reward: [(0, '270.630'), (1, '690.140')] [2023-12-27 16:23:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001548488_396451840.pth... [2023-12-27 16:23:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001547304_396148736.pth [2023-12-27 16:23:07,378][20347] Updated weights for policy 0, policy_version 1550103 (0.0007) [2023-12-27 16:23:07,437][20347] Updated weights for policy 0, policy_version 1550113 (0.0005) [2023-12-27 16:23:07,496][20347] Updated weights for policy 0, policy_version 1550123 (0.0010) [2023-12-27 16:23:07,517][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001550128_396910592.pth... [2023-12-27 16:23:07,520][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001548912_396599296.pth [2023-12-27 16:23:07,909][20381] Updated weights for policy 1, policy_version 1548493 (0.0008) [2023-12-27 16:23:07,966][20381] Updated weights for policy 1, policy_version 1548503 (0.0007) [2023-12-27 16:23:08,020][20381] Updated weights for policy 1, policy_version 1548513 (0.0007) [2023-12-27 16:23:08,105][20347] Updated weights for policy 0, policy_version 1550133 (0.0010) [2023-12-27 16:23:08,152][20347] Updated weights for policy 0, policy_version 1550143 (0.0009) [2023-12-27 16:23:08,200][20347] Updated weights for policy 0, policy_version 1550153 (0.0010) [2023-12-27 16:23:08,676][20381] Updated weights for policy 1, policy_version 1548523 (0.0007) [2023-12-27 16:23:08,731][20381] Updated weights for policy 1, policy_version 1548533 (0.0007) [2023-12-27 16:23:08,789][20381] Updated weights for policy 1, policy_version 1548543 (0.0007) [2023-12-27 16:23:08,867][20347] Updated weights for policy 0, policy_version 1550163 (0.0009) [2023-12-27 16:23:08,930][20347] Updated weights for policy 0, policy_version 1550173 (0.0007) [2023-12-27 16:23:08,977][20347] Updated weights for policy 0, policy_version 1550183 (0.0010) [2023-12-27 16:23:09,534][20381] Updated weights for policy 1, policy_version 1548553 (0.0007) [2023-12-27 16:23:09,597][20381] Updated weights for policy 1, policy_version 1548563 (0.0008) [2023-12-27 16:23:09,618][20347] Updated weights for policy 0, policy_version 1550193 (0.0007) [2023-12-27 16:23:09,658][20381] Updated weights for policy 1, policy_version 1548573 (0.0006) [2023-12-27 16:23:09,678][20347] Updated weights for policy 0, policy_version 1550203 (0.0011) [2023-12-27 16:23:09,714][20381] Updated weights for policy 1, policy_version 1548583 (0.0005) [2023-12-27 16:23:09,734][20347] Updated weights for policy 0, policy_version 1550213 (0.0010) [2023-12-27 16:23:09,793][20347] Updated weights for policy 0, policy_version 1550223 (0.0011) [2023-12-27 16:23:10,404][20381] Updated weights for policy 1, policy_version 1548593 (0.0006) [2023-12-27 16:23:10,462][20381] Updated weights for policy 1, policy_version 1548603 (0.0006) [2023-12-27 16:23:10,503][20347] Updated weights for policy 0, policy_version 1550233 (0.0010) [2023-12-27 16:23:10,521][20381] Updated weights for policy 1, policy_version 1548613 (0.0008) [2023-12-27 16:23:10,559][20347] Updated weights for policy 0, policy_version 1550243 (0.0010) [2023-12-27 16:23:10,620][20347] Updated weights for policy 0, policy_version 1550253 (0.0010) [2023-12-27 16:23:11,144][20381] Updated weights for policy 1, policy_version 1548623 (0.0007) [2023-12-27 16:23:11,198][20381] Updated weights for policy 1, policy_version 1548633 (0.0008) [2023-12-27 16:23:11,257][20381] Updated weights for policy 1, policy_version 1548643 (0.0007) [2023-12-27 16:23:11,294][20347] Updated weights for policy 0, policy_version 1550263 (0.0008) [2023-12-27 16:23:11,350][20347] Updated weights for policy 0, policy_version 1550273 (0.0008) [2023-12-27 16:23:11,424][20347] Updated weights for policy 0, policy_version 1550283 (0.0011) [2023-12-27 16:23:12,064][20381] Updated weights for policy 1, policy_version 1548653 (0.0008) [2023-12-27 16:23:12,089][20347] Updated weights for policy 0, policy_version 1550293 (0.0007) [2023-12-27 16:23:12,124][20381] Updated weights for policy 1, policy_version 1548663 (0.0007) [2023-12-27 16:23:12,145][20347] Updated weights for policy 0, policy_version 1550303 (0.0006) [2023-12-27 16:23:12,177][20381] Updated weights for policy 1, policy_version 1548673 (0.0009) [2023-12-27 16:23:12,211][20347] Updated weights for policy 0, policy_version 1550313 (0.0006) [2023-12-27 16:23:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 793460736. Throughput: 0: 10197.2, 1: 10230.6. Samples: 793446396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:12,363][19170] Avg episode reward: [(0, '262.950'), (1, '625.170')] [2023-12-27 16:23:12,932][20381] Updated weights for policy 1, policy_version 1548683 (0.0008) [2023-12-27 16:23:12,968][20347] Updated weights for policy 0, policy_version 1550323 (0.0008) [2023-12-27 16:23:12,994][20381] Updated weights for policy 1, policy_version 1548693 (0.0007) [2023-12-27 16:23:13,021][20347] Updated weights for policy 0, policy_version 1550333 (0.0007) [2023-12-27 16:23:13,055][20381] Updated weights for policy 1, policy_version 1548703 (0.0006) [2023-12-27 16:23:13,071][20347] Updated weights for policy 0, policy_version 1550343 (0.0009) [2023-12-27 16:23:13,681][20381] Updated weights for policy 1, policy_version 1548713 (0.0008) [2023-12-27 16:23:13,732][20381] Updated weights for policy 1, policy_version 1548723 (0.0007) [2023-12-27 16:23:13,789][20381] Updated weights for policy 1, policy_version 1548733 (0.0007) [2023-12-27 16:23:13,792][20347] Updated weights for policy 0, policy_version 1550353 (0.0007) [2023-12-27 16:23:13,842][20347] Updated weights for policy 0, policy_version 1550363 (0.0007) [2023-12-27 16:23:13,848][20381] Updated weights for policy 1, policy_version 1548743 (0.0006) [2023-12-27 16:23:13,889][20347] Updated weights for policy 0, policy_version 1550373 (0.0007) [2023-12-27 16:23:13,936][20347] Updated weights for policy 0, policy_version 1550383 (0.0007) [2023-12-27 16:23:14,509][20381] Updated weights for policy 1, policy_version 1548753 (0.0009) [2023-12-27 16:23:14,562][20381] Updated weights for policy 1, policy_version 1548763 (0.0006) [2023-12-27 16:23:14,586][20347] Updated weights for policy 0, policy_version 1550393 (0.0007) [2023-12-27 16:23:14,618][20381] Updated weights for policy 1, policy_version 1548773 (0.0007) [2023-12-27 16:23:14,635][20347] Updated weights for policy 0, policy_version 1550403 (0.0005) [2023-12-27 16:23:14,681][20347] Updated weights for policy 0, policy_version 1550413 (0.0007) [2023-12-27 16:23:15,340][20381] Updated weights for policy 1, policy_version 1548783 (0.0008) [2023-12-27 16:23:15,354][20347] Updated weights for policy 0, policy_version 1550423 (0.0005) [2023-12-27 16:23:15,391][20381] Updated weights for policy 1, policy_version 1548793 (0.0007) [2023-12-27 16:23:15,406][20347] Updated weights for policy 0, policy_version 1550433 (0.0005) [2023-12-27 16:23:15,442][20381] Updated weights for policy 1, policy_version 1548803 (0.0008) [2023-12-27 16:23:15,459][20347] Updated weights for policy 0, policy_version 1550443 (0.0006) [2023-12-27 16:23:16,124][20381] Updated weights for policy 1, policy_version 1548813 (0.0007) [2023-12-27 16:23:16,129][20347] Updated weights for policy 0, policy_version 1550453 (0.0007) [2023-12-27 16:23:16,174][20381] Updated weights for policy 1, policy_version 1548823 (0.0010) [2023-12-27 16:23:16,182][20347] Updated weights for policy 0, policy_version 1550463 (0.0008) [2023-12-27 16:23:16,227][20381] Updated weights for policy 1, policy_version 1548833 (0.0010) [2023-12-27 16:23:16,237][20347] Updated weights for policy 0, policy_version 1550473 (0.0006) [2023-12-27 16:23:16,821][20347] Updated weights for policy 0, policy_version 1550483 (0.0006) [2023-12-27 16:23:16,875][20347] Updated weights for policy 0, policy_version 1550493 (0.0007) [2023-12-27 16:23:16,908][20381] Updated weights for policy 1, policy_version 1548843 (0.0010) [2023-12-27 16:23:16,928][20347] Updated weights for policy 0, policy_version 1550503 (0.0006) [2023-12-27 16:23:16,964][20381] Updated weights for policy 1, policy_version 1548853 (0.0010) [2023-12-27 16:23:17,016][20381] Updated weights for policy 1, policy_version 1548863 (0.0010) [2023-12-27 16:23:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793559040. Throughput: 0: 10192.1, 1: 10233.4. Samples: 793567736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:17,363][19170] Avg episode reward: [(0, '277.900'), (1, '673.990')] [2023-12-27 16:23:17,537][20347] Updated weights for policy 0, policy_version 1550513 (0.0006) [2023-12-27 16:23:17,588][20347] Updated weights for policy 0, policy_version 1550523 (0.0006) [2023-12-27 16:23:17,618][20381] Updated weights for policy 1, policy_version 1548873 (0.0010) [2023-12-27 16:23:17,640][20347] Updated weights for policy 0, policy_version 1550533 (0.0010) [2023-12-27 16:23:17,675][20381] Updated weights for policy 1, policy_version 1548883 (0.0010) [2023-12-27 16:23:17,698][20347] Updated weights for policy 0, policy_version 1550543 (0.0010) [2023-12-27 16:23:17,726][20381] Updated weights for policy 1, policy_version 1548893 (0.0010) [2023-12-27 16:23:17,784][20381] Updated weights for policy 1, policy_version 1548903 (0.0007) [2023-12-27 16:23:18,298][20347] Updated weights for policy 0, policy_version 1550553 (0.0007) [2023-12-27 16:23:18,357][20381] Updated weights for policy 1, policy_version 1548913 (0.0010) [2023-12-27 16:23:18,364][20347] Updated weights for policy 0, policy_version 1550563 (0.0006) [2023-12-27 16:23:18,420][20381] Updated weights for policy 1, policy_version 1548923 (0.0011) [2023-12-27 16:23:18,430][20347] Updated weights for policy 0, policy_version 1550573 (0.0006) [2023-12-27 16:23:18,476][20381] Updated weights for policy 1, policy_version 1548933 (0.0011) [2023-12-27 16:23:19,064][20347] Updated weights for policy 0, policy_version 1550583 (0.0006) [2023-12-27 16:23:19,107][20381] Updated weights for policy 1, policy_version 1548943 (0.0007) [2023-12-27 16:23:19,120][20347] Updated weights for policy 0, policy_version 1550593 (0.0007) [2023-12-27 16:23:19,159][20381] Updated weights for policy 1, policy_version 1548953 (0.0005) [2023-12-27 16:23:19,175][20347] Updated weights for policy 0, policy_version 1550603 (0.0007) [2023-12-27 16:23:19,208][20381] Updated weights for policy 1, policy_version 1548963 (0.0006) [2023-12-27 16:23:19,940][20381] Updated weights for policy 1, policy_version 1548973 (0.0008) [2023-12-27 16:23:20,002][20381] Updated weights for policy 1, policy_version 1548983 (0.0008) [2023-12-27 16:23:20,019][20347] Updated weights for policy 0, policy_version 1550613 (0.0008) [2023-12-27 16:23:20,059][20381] Updated weights for policy 1, policy_version 1548993 (0.0008) [2023-12-27 16:23:20,073][20347] Updated weights for policy 0, policy_version 1550623 (0.0008) [2023-12-27 16:23:20,133][20347] Updated weights for policy 0, policy_version 1550633 (0.0008) [2023-12-27 16:23:20,851][20381] Updated weights for policy 1, policy_version 1549003 (0.0008) [2023-12-27 16:23:20,872][20347] Updated weights for policy 0, policy_version 1550643 (0.0009) [2023-12-27 16:23:20,909][20381] Updated weights for policy 1, policy_version 1549013 (0.0006) [2023-12-27 16:23:20,933][20347] Updated weights for policy 0, policy_version 1550653 (0.0010) [2023-12-27 16:23:20,972][20381] Updated weights for policy 1, policy_version 1549023 (0.0007) [2023-12-27 16:23:20,999][20347] Updated weights for policy 0, policy_version 1550663 (0.0010) [2023-12-27 16:23:21,627][20347] Updated weights for policy 0, policy_version 1550673 (0.0009) [2023-12-27 16:23:21,698][20347] Updated weights for policy 0, policy_version 1550683 (0.0008) [2023-12-27 16:23:21,728][20381] Updated weights for policy 1, policy_version 1549033 (0.0006) [2023-12-27 16:23:21,770][20347] Updated weights for policy 0, policy_version 1550693 (0.0008) [2023-12-27 16:23:21,807][20381] Updated weights for policy 1, policy_version 1549043 (0.0006) [2023-12-27 16:23:21,830][20347] Updated weights for policy 0, policy_version 1550703 (0.0008) [2023-12-27 16:23:21,876][20381] Updated weights for policy 1, policy_version 1549053 (0.0006) [2023-12-27 16:23:21,944][20381] Updated weights for policy 1, policy_version 1549063 (0.0008) [2023-12-27 16:23:22,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 793657344. Throughput: 0: 10175.9, 1: 10210.4. Samples: 793631028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:22,364][19170] Avg episode reward: [(0, '241.550'), (1, '673.400')] [2023-12-27 16:23:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001549064_396599296.pth... [2023-12-27 16:23:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001550704_397058048.pth... [2023-12-27 16:23:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001547880_396296192.pth [2023-12-27 16:23:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001549520_396754944.pth [2023-12-27 16:23:22,532][20347] Updated weights for policy 0, policy_version 1550713 (0.0006) [2023-12-27 16:23:22,592][20347] Updated weights for policy 0, policy_version 1550723 (0.0007) [2023-12-27 16:23:22,638][20381] Updated weights for policy 1, policy_version 1549073 (0.0007) [2023-12-27 16:23:22,655][20347] Updated weights for policy 0, policy_version 1550733 (0.0006) [2023-12-27 16:23:22,696][20381] Updated weights for policy 1, policy_version 1549083 (0.0006) [2023-12-27 16:23:22,746][20381] Updated weights for policy 1, policy_version 1549093 (0.0008) [2023-12-27 16:23:23,344][20347] Updated weights for policy 0, policy_version 1550743 (0.0008) [2023-12-27 16:23:23,408][20347] Updated weights for policy 0, policy_version 1550753 (0.0006) [2023-12-27 16:23:23,446][20381] Updated weights for policy 1, policy_version 1549103 (0.0010) [2023-12-27 16:23:23,456][20347] Updated weights for policy 0, policy_version 1550763 (0.0010) [2023-12-27 16:23:23,503][20381] Updated weights for policy 1, policy_version 1549113 (0.0010) [2023-12-27 16:23:23,554][20381] Updated weights for policy 1, policy_version 1549123 (0.0010) [2023-12-27 16:23:24,119][20347] Updated weights for policy 0, policy_version 1550773 (0.0010) [2023-12-27 16:23:24,176][20347] Updated weights for policy 0, policy_version 1550783 (0.0006) [2023-12-27 16:23:24,184][20381] Updated weights for policy 1, policy_version 1549133 (0.0011) [2023-12-27 16:23:24,237][20347] Updated weights for policy 0, policy_version 1550793 (0.0005) [2023-12-27 16:23:24,244][20381] Updated weights for policy 1, policy_version 1549143 (0.0011) [2023-12-27 16:23:24,302][20381] Updated weights for policy 1, policy_version 1549153 (0.0011) [2023-12-27 16:23:24,891][20347] Updated weights for policy 0, policy_version 1550803 (0.0008) [2023-12-27 16:23:24,893][20381] Updated weights for policy 1, policy_version 1549163 (0.0007) [2023-12-27 16:23:24,944][20381] Updated weights for policy 1, policy_version 1549173 (0.0005) [2023-12-27 16:23:24,947][20347] Updated weights for policy 0, policy_version 1550813 (0.0007) [2023-12-27 16:23:24,993][20381] Updated weights for policy 1, policy_version 1549183 (0.0005) [2023-12-27 16:23:25,007][20347] Updated weights for policy 0, policy_version 1550823 (0.0005) [2023-12-27 16:23:25,614][20381] Updated weights for policy 1, policy_version 1549193 (0.0007) [2023-12-27 16:23:25,615][20347] Updated weights for policy 0, policy_version 1550833 (0.0007) [2023-12-27 16:23:25,667][20381] Updated weights for policy 1, policy_version 1549203 (0.0005) [2023-12-27 16:23:25,671][20347] Updated weights for policy 0, policy_version 1550843 (0.0007) [2023-12-27 16:23:25,721][20381] Updated weights for policy 1, policy_version 1549213 (0.0006) [2023-12-27 16:23:25,728][20347] Updated weights for policy 0, policy_version 1550853 (0.0007) [2023-12-27 16:23:25,778][20381] Updated weights for policy 1, policy_version 1549223 (0.0010) [2023-12-27 16:23:25,781][20347] Updated weights for policy 0, policy_version 1550863 (0.0007) [2023-12-27 16:23:26,367][20381] Updated weights for policy 1, policy_version 1549233 (0.0010) [2023-12-27 16:23:26,408][20347] Updated weights for policy 0, policy_version 1550873 (0.0006) [2023-12-27 16:23:26,420][20381] Updated weights for policy 1, policy_version 1549243 (0.0010) [2023-12-27 16:23:26,452][20347] Updated weights for policy 0, policy_version 1550883 (0.0007) [2023-12-27 16:23:26,472][20381] Updated weights for policy 1, policy_version 1549253 (0.0005) [2023-12-27 16:23:26,496][20347] Updated weights for policy 0, policy_version 1550893 (0.0006) [2023-12-27 16:23:27,064][20381] Updated weights for policy 1, policy_version 1549263 (0.0009) [2023-12-27 16:23:27,085][20347] Updated weights for policy 0, policy_version 1550903 (0.0009) [2023-12-27 16:23:27,119][20381] Updated weights for policy 1, policy_version 1549273 (0.0009) [2023-12-27 16:23:27,141][20347] Updated weights for policy 0, policy_version 1550913 (0.0007) [2023-12-27 16:23:27,177][20381] Updated weights for policy 1, policy_version 1549283 (0.0009) [2023-12-27 16:23:27,197][20347] Updated weights for policy 0, policy_version 1550923 (0.0006) [2023-12-27 16:23:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793772032. Throughput: 0: 10187.5, 1: 10228.6. Samples: 793754188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:27,363][19170] Avg episode reward: [(0, '289.180'), (1, '604.810')] [2023-12-27 16:23:27,752][20381] Updated weights for policy 1, policy_version 1549293 (0.0010) [2023-12-27 16:23:27,818][20381] Updated weights for policy 1, policy_version 1549303 (0.0005) [2023-12-27 16:23:27,862][20347] Updated weights for policy 0, policy_version 1550933 (0.0006) [2023-12-27 16:23:27,872][20381] Updated weights for policy 1, policy_version 1549313 (0.0006) [2023-12-27 16:23:27,917][20347] Updated weights for policy 0, policy_version 1550943 (0.0005) [2023-12-27 16:23:27,972][20347] Updated weights for policy 0, policy_version 1550953 (0.0005) [2023-12-27 16:23:28,561][20381] Updated weights for policy 1, policy_version 1549323 (0.0006) [2023-12-27 16:23:28,624][20347] Updated weights for policy 0, policy_version 1550963 (0.0006) [2023-12-27 16:23:28,628][20381] Updated weights for policy 1, policy_version 1549333 (0.0005) [2023-12-27 16:23:28,684][20347] Updated weights for policy 0, policy_version 1550973 (0.0006) [2023-12-27 16:23:28,689][20381] Updated weights for policy 1, policy_version 1549343 (0.0005) [2023-12-27 16:23:28,741][20347] Updated weights for policy 0, policy_version 1550983 (0.0008) [2023-12-27 16:23:29,221][20381] Updated weights for policy 1, policy_version 1549353 (0.0006) [2023-12-27 16:23:29,284][20381] Updated weights for policy 1, policy_version 1549363 (0.0008) [2023-12-27 16:23:29,346][20381] Updated weights for policy 1, policy_version 1549373 (0.0008) [2023-12-27 16:23:29,399][20381] Updated weights for policy 1, policy_version 1549383 (0.0009) [2023-12-27 16:23:29,481][20347] Updated weights for policy 0, policy_version 1550993 (0.0008) [2023-12-27 16:23:29,541][20347] Updated weights for policy 0, policy_version 1551003 (0.0005) [2023-12-27 16:23:29,594][20347] Updated weights for policy 0, policy_version 1551013 (0.0006) [2023-12-27 16:23:29,644][20347] Updated weights for policy 0, policy_version 1551023 (0.0007) [2023-12-27 16:23:30,097][20381] Updated weights for policy 1, policy_version 1549393 (0.0006) [2023-12-27 16:23:30,159][20381] Updated weights for policy 1, policy_version 1549403 (0.0009) [2023-12-27 16:23:30,209][20381] Updated weights for policy 1, policy_version 1549413 (0.0007) [2023-12-27 16:23:30,341][20347] Updated weights for policy 0, policy_version 1551033 (0.0006) [2023-12-27 16:23:30,403][20347] Updated weights for policy 0, policy_version 1551043 (0.0006) [2023-12-27 16:23:30,466][20347] Updated weights for policy 0, policy_version 1551053 (0.0007) [2023-12-27 16:23:30,784][20381] Updated weights for policy 1, policy_version 1549423 (0.0006) [2023-12-27 16:23:30,843][20381] Updated weights for policy 1, policy_version 1549433 (0.0008) [2023-12-27 16:23:30,894][20381] Updated weights for policy 1, policy_version 1549443 (0.0010) [2023-12-27 16:23:31,083][20347] Updated weights for policy 0, policy_version 1551063 (0.0010) [2023-12-27 16:23:31,147][20347] Updated weights for policy 0, policy_version 1551073 (0.0009) [2023-12-27 16:23:31,205][20347] Updated weights for policy 0, policy_version 1551083 (0.0008) [2023-12-27 16:23:31,626][20381] Updated weights for policy 1, policy_version 1549453 (0.0010) [2023-12-27 16:23:31,694][20381] Updated weights for policy 1, policy_version 1549463 (0.0006) [2023-12-27 16:23:31,765][20381] Updated weights for policy 1, policy_version 1549473 (0.0008) [2023-12-27 16:23:31,921][20347] Updated weights for policy 0, policy_version 1551093 (0.0007) [2023-12-27 16:23:31,977][20347] Updated weights for policy 0, policy_version 1551103 (0.0006) [2023-12-27 16:23:32,032][20347] Updated weights for policy 0, policy_version 1551113 (0.0009) [2023-12-27 16:23:32,357][20381] Updated weights for policy 1, policy_version 1549483 (0.0008) [2023-12-27 16:23:32,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 793870336. Throughput: 0: 10214.0, 1: 10323.6. Samples: 793882340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:32,363][19170] Avg episode reward: [(0, '271.340'), (1, '566.840')] [2023-12-27 16:23:32,422][20381] Updated weights for policy 1, policy_version 1549493 (0.0008) [2023-12-27 16:23:32,482][20381] Updated weights for policy 1, policy_version 1549503 (0.0008) [2023-12-27 16:23:32,670][20347] Updated weights for policy 0, policy_version 1551123 (0.0009) [2023-12-27 16:23:32,727][20347] Updated weights for policy 0, policy_version 1551133 (0.0007) [2023-12-27 16:23:32,786][20347] Updated weights for policy 0, policy_version 1551143 (0.0010) [2023-12-27 16:23:33,176][20381] Updated weights for policy 1, policy_version 1549513 (0.0007) [2023-12-27 16:23:33,234][20381] Updated weights for policy 1, policy_version 1549523 (0.0005) [2023-12-27 16:23:33,289][20381] Updated weights for policy 1, policy_version 1549533 (0.0007) [2023-12-27 16:23:33,343][20381] Updated weights for policy 1, policy_version 1549543 (0.0005) [2023-12-27 16:23:33,375][20347] Updated weights for policy 0, policy_version 1551153 (0.0007) [2023-12-27 16:23:33,429][20347] Updated weights for policy 0, policy_version 1551163 (0.0005) [2023-12-27 16:23:33,492][20347] Updated weights for policy 0, policy_version 1551173 (0.0005) [2023-12-27 16:23:33,560][20347] Updated weights for policy 0, policy_version 1551183 (0.0006) [2023-12-27 16:23:33,862][20381] Updated weights for policy 1, policy_version 1549553 (0.0005) [2023-12-27 16:23:33,909][20381] Updated weights for policy 1, policy_version 1549563 (0.0005) [2023-12-27 16:23:33,958][20381] Updated weights for policy 1, policy_version 1549573 (0.0005) [2023-12-27 16:23:34,198][20347] Updated weights for policy 0, policy_version 1551193 (0.0008) [2023-12-27 16:23:34,258][20347] Updated weights for policy 0, policy_version 1551203 (0.0007) [2023-12-27 16:23:34,319][20347] Updated weights for policy 0, policy_version 1551213 (0.0009) [2023-12-27 16:23:34,584][20381] Updated weights for policy 1, policy_version 1549583 (0.0007) [2023-12-27 16:23:34,640][20381] Updated weights for policy 1, policy_version 1549593 (0.0007) [2023-12-27 16:23:34,698][20381] Updated weights for policy 1, policy_version 1549603 (0.0008) [2023-12-27 16:23:35,025][20347] Updated weights for policy 0, policy_version 1551223 (0.0006) [2023-12-27 16:23:35,075][20347] Updated weights for policy 0, policy_version 1551233 (0.0008) [2023-12-27 16:23:35,127][20347] Updated weights for policy 0, policy_version 1551243 (0.0008) [2023-12-27 16:23:35,320][20381] Updated weights for policy 1, policy_version 1549613 (0.0006) [2023-12-27 16:23:35,370][20381] Updated weights for policy 1, policy_version 1549623 (0.0005) [2023-12-27 16:23:35,429][20381] Updated weights for policy 1, policy_version 1549633 (0.0006) [2023-12-27 16:23:35,835][20347] Updated weights for policy 0, policy_version 1551253 (0.0006) [2023-12-27 16:23:35,892][20347] Updated weights for policy 0, policy_version 1551263 (0.0009) [2023-12-27 16:23:35,942][20347] Updated weights for policy 0, policy_version 1551273 (0.0009) [2023-12-27 16:23:36,021][20381] Updated weights for policy 1, policy_version 1549643 (0.0006) [2023-12-27 16:23:36,070][20381] Updated weights for policy 1, policy_version 1549653 (0.0008) [2023-12-27 16:23:36,130][20381] Updated weights for policy 1, policy_version 1549663 (0.0008) [2023-12-27 16:23:36,652][20347] Updated weights for policy 0, policy_version 1551283 (0.0008) [2023-12-27 16:23:36,709][20347] Updated weights for policy 0, policy_version 1551293 (0.0008) [2023-12-27 16:23:36,766][20347] Updated weights for policy 0, policy_version 1551303 (0.0006) [2023-12-27 16:23:36,862][20381] Updated weights for policy 1, policy_version 1549673 (0.0009) [2023-12-27 16:23:36,907][20381] Updated weights for policy 1, policy_version 1549683 (0.0008) [2023-12-27 16:23:36,969][20381] Updated weights for policy 1, policy_version 1549693 (0.0008) [2023-12-27 16:23:37,024][20381] Updated weights for policy 1, policy_version 1549703 (0.0008) [2023-12-27 16:23:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 793976832. Throughput: 0: 10264.2, 1: 10383.2. Samples: 793947156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:37,363][19170] Avg episode reward: [(0, '346.240'), (1, '644.200')] [2023-12-27 16:23:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001549704_396763136.pth... [2023-12-27 16:23:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001551312_397213696.pth... [2023-12-27 16:23:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001548488_396451840.pth [2023-12-27 16:23:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001550128_396910592.pth [2023-12-27 16:23:37,418][20347] Updated weights for policy 0, policy_version 1551313 (0.0006) [2023-12-27 16:23:37,474][20347] Updated weights for policy 0, policy_version 1551323 (0.0008) [2023-12-27 16:23:37,525][20347] Updated weights for policy 0, policy_version 1551333 (0.0010) [2023-12-27 16:23:37,573][20347] Updated weights for policy 0, policy_version 1551343 (0.0008) [2023-12-27 16:23:37,755][20381] Updated weights for policy 1, policy_version 1549713 (0.0008) [2023-12-27 16:23:37,806][20381] Updated weights for policy 1, policy_version 1549723 (0.0009) [2023-12-27 16:23:37,811][20292] KL-divergence is very high: 156.5830 [2023-12-27 16:23:37,821][20292] KL-divergence is very high: 450.9783 [2023-12-27 16:23:37,831][20292] KL-divergence is very high: 515.6506 [2023-12-27 16:23:37,849][20292] KL-divergence is very high: 139.5260 [2023-12-27 16:23:37,855][20381] Updated weights for policy 1, policy_version 1549733 (0.0007) [2023-12-27 16:23:37,859][20292] KL-divergence is very high: 292.6747 [2023-12-27 16:23:38,225][20347] Updated weights for policy 0, policy_version 1551353 (0.0010) [2023-12-27 16:23:38,283][20347] Updated weights for policy 0, policy_version 1551363 (0.0010) [2023-12-27 16:23:38,353][20347] Updated weights for policy 0, policy_version 1551373 (0.0010) [2023-12-27 16:23:38,528][20292] KL-divergence is very high: 227.1800 [2023-12-27 16:23:38,567][20381] Updated weights for policy 1, policy_version 1549743 (0.0007) [2023-12-27 16:23:38,570][20292] KL-divergence is very high: 119.2995 [2023-12-27 16:23:38,616][20381] Updated weights for policy 1, policy_version 1549753 (0.0007) [2023-12-27 16:23:38,650][20292] KL-divergence is very high: 101.3809 [2023-12-27 16:23:38,663][20381] Updated weights for policy 1, policy_version 1549763 (0.0007) [2023-12-27 16:23:39,036][20347] Updated weights for policy 0, policy_version 1551383 (0.0009) [2023-12-27 16:23:39,096][20347] Updated weights for policy 0, policy_version 1551393 (0.0007) [2023-12-27 16:23:39,154][20347] Updated weights for policy 0, policy_version 1551403 (0.0005) [2023-12-27 16:23:39,407][20381] Updated weights for policy 1, policy_version 1549773 (0.0008) [2023-12-27 16:23:39,463][20381] Updated weights for policy 1, policy_version 1549783 (0.0008) [2023-12-27 16:23:39,515][20381] Updated weights for policy 1, policy_version 1549793 (0.0008) [2023-12-27 16:23:39,872][20347] Updated weights for policy 0, policy_version 1551413 (0.0006) [2023-12-27 16:23:39,933][20347] Updated weights for policy 0, policy_version 1551423 (0.0008) [2023-12-27 16:23:39,993][20347] Updated weights for policy 0, policy_version 1551433 (0.0007) [2023-12-27 16:23:40,229][20381] Updated weights for policy 1, policy_version 1549803 (0.0007) [2023-12-27 16:23:40,294][20381] Updated weights for policy 1, policy_version 1549813 (0.0009) [2023-12-27 16:23:40,358][20381] Updated weights for policy 1, policy_version 1549823 (0.0010) [2023-12-27 16:23:40,664][20347] Updated weights for policy 0, policy_version 1551443 (0.0007) [2023-12-27 16:23:40,712][20347] Updated weights for policy 0, policy_version 1551453 (0.0010) [2023-12-27 16:23:40,757][20347] Updated weights for policy 0, policy_version 1551463 (0.0010) [2023-12-27 16:23:40,982][20381] Updated weights for policy 1, policy_version 1549833 (0.0010) [2023-12-27 16:23:41,045][20381] Updated weights for policy 1, policy_version 1549843 (0.0008) [2023-12-27 16:23:41,111][20381] Updated weights for policy 1, policy_version 1549853 (0.0011) [2023-12-27 16:23:41,189][20381] Updated weights for policy 1, policy_version 1549863 (0.0011) [2023-12-27 16:23:41,494][20347] Updated weights for policy 0, policy_version 1551473 (0.0010) [2023-12-27 16:23:41,550][20347] Updated weights for policy 0, policy_version 1551483 (0.0008) [2023-12-27 16:23:41,620][20347] Updated weights for policy 0, policy_version 1551493 (0.0008) [2023-12-27 16:23:41,688][20347] Updated weights for policy 0, policy_version 1551503 (0.0007) [2023-12-27 16:23:41,879][20381] Updated weights for policy 1, policy_version 1549873 (0.0008) [2023-12-27 16:23:41,937][20381] Updated weights for policy 1, policy_version 1549883 (0.0008) [2023-12-27 16:23:42,002][20381] Updated weights for policy 1, policy_version 1549893 (0.0008) [2023-12-27 16:23:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 794075136. Throughput: 0: 10226.2, 1: 10384.6. Samples: 794068076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:42,363][19170] Avg episode reward: [(0, '341.940'), (1, '617.480')] [2023-12-27 16:23:42,409][20347] Updated weights for policy 0, policy_version 1551513 (0.0007) [2023-12-27 16:23:42,465][20347] Updated weights for policy 0, policy_version 1551523 (0.0008) [2023-12-27 16:23:42,518][20347] Updated weights for policy 0, policy_version 1551533 (0.0007) [2023-12-27 16:23:42,661][20381] Updated weights for policy 1, policy_version 1549903 (0.0010) [2023-12-27 16:23:42,723][20381] Updated weights for policy 1, policy_version 1549913 (0.0010) [2023-12-27 16:23:42,786][20381] Updated weights for policy 1, policy_version 1549923 (0.0011) [2023-12-27 16:23:43,183][20347] Updated weights for policy 0, policy_version 1551543 (0.0008) [2023-12-27 16:23:43,228][20347] Updated weights for policy 0, policy_version 1551553 (0.0010) [2023-12-27 16:23:43,276][20347] Updated weights for policy 0, policy_version 1551563 (0.0007) [2023-12-27 16:23:43,348][20381] Updated weights for policy 1, policy_version 1549933 (0.0006) [2023-12-27 16:23:43,403][20381] Updated weights for policy 1, policy_version 1549943 (0.0005) [2023-12-27 16:23:43,459][20381] Updated weights for policy 1, policy_version 1549953 (0.0005) [2023-12-27 16:23:43,998][20347] Updated weights for policy 0, policy_version 1551573 (0.0006) [2023-12-27 16:23:44,063][20347] Updated weights for policy 0, policy_version 1551583 (0.0008) [2023-12-27 16:23:44,092][20381] Updated weights for policy 1, policy_version 1549963 (0.0006) [2023-12-27 16:23:44,116][20347] Updated weights for policy 0, policy_version 1551593 (0.0007) [2023-12-27 16:23:44,149][20381] Updated weights for policy 1, policy_version 1549973 (0.0008) [2023-12-27 16:23:44,201][20381] Updated weights for policy 1, policy_version 1549983 (0.0008) [2023-12-27 16:23:44,785][20347] Updated weights for policy 0, policy_version 1551603 (0.0006) [2023-12-27 16:23:44,845][20347] Updated weights for policy 0, policy_version 1551613 (0.0008) [2023-12-27 16:23:44,880][20381] Updated weights for policy 1, policy_version 1549993 (0.0008) [2023-12-27 16:23:44,910][20347] Updated weights for policy 0, policy_version 1551623 (0.0007) [2023-12-27 16:23:44,940][20381] Updated weights for policy 1, policy_version 1550003 (0.0005) [2023-12-27 16:23:45,009][20381] Updated weights for policy 1, policy_version 1550013 (0.0005) [2023-12-27 16:23:45,067][20381] Updated weights for policy 1, policy_version 1550023 (0.0006) [2023-12-27 16:23:45,618][20347] Updated weights for policy 0, policy_version 1551633 (0.0008) [2023-12-27 16:23:45,676][20381] Updated weights for policy 1, policy_version 1550033 (0.0005) [2023-12-27 16:23:45,684][20347] Updated weights for policy 0, policy_version 1551643 (0.0008) [2023-12-27 16:23:45,691][20292] KL-divergence is very high: 100.8774 [2023-12-27 16:23:45,710][20292] KL-divergence is very high: 198.4423 [2023-12-27 16:23:45,727][20381] Updated weights for policy 1, policy_version 1550043 (0.0005) [2023-12-27 16:23:45,731][20292] KL-divergence is very high: 286.4860 [2023-12-27 16:23:45,747][20347] Updated weights for policy 0, policy_version 1551653 (0.0007) [2023-12-27 16:23:45,750][20292] KL-divergence is very high: 213.8185 [2023-12-27 16:23:45,779][20292] KL-divergence is very high: 183.4126 [2023-12-27 16:23:45,785][20381] Updated weights for policy 1, policy_version 1550053 (0.0005) [2023-12-27 16:23:45,806][20347] Updated weights for policy 0, policy_version 1551663 (0.0006) [2023-12-27 16:23:46,342][20381] Updated weights for policy 1, policy_version 1550063 (0.0009) [2023-12-27 16:23:46,399][20292] KL-divergence is very high: 153.2561 [2023-12-27 16:23:46,405][20381] Updated weights for policy 1, policy_version 1550073 (0.0011) [2023-12-27 16:23:46,447][20292] KL-divergence is very high: 154.3119 [2023-12-27 16:23:46,466][20381] Updated weights for policy 1, policy_version 1550083 (0.0010) [2023-12-27 16:23:46,505][20347] Updated weights for policy 0, policy_version 1551673 (0.0006) [2023-12-27 16:23:46,562][20347] Updated weights for policy 0, policy_version 1551683 (0.0007) [2023-12-27 16:23:46,612][20347] Updated weights for policy 0, policy_version 1551693 (0.0007) [2023-12-27 16:23:47,057][20381] Updated weights for policy 1, policy_version 1550093 (0.0008) [2023-12-27 16:23:47,106][20381] Updated weights for policy 1, policy_version 1550103 (0.0005) [2023-12-27 16:23:47,163][20381] Updated weights for policy 1, policy_version 1550113 (0.0005) [2023-12-27 16:23:47,346][20347] Updated weights for policy 0, policy_version 1551703 (0.0008) [2023-12-27 16:23:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 794181632. Throughput: 0: 10233.2, 1: 10443.8. Samples: 794192332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:47,363][19170] Avg episode reward: [(0, '259.060'), (1, '614.110')] [2023-12-27 16:23:47,401][20347] Updated weights for policy 0, policy_version 1551713 (0.0006) [2023-12-27 16:23:47,456][20347] Updated weights for policy 0, policy_version 1551723 (0.0007) [2023-12-27 16:23:47,813][20381] Updated weights for policy 1, policy_version 1550123 (0.0008) [2023-12-27 16:23:47,864][20381] Updated weights for policy 1, policy_version 1550133 (0.0006) [2023-12-27 16:23:47,919][20381] Updated weights for policy 1, policy_version 1550143 (0.0005) [2023-12-27 16:23:48,155][20347] Updated weights for policy 0, policy_version 1551733 (0.0008) [2023-12-27 16:23:48,209][20347] Updated weights for policy 0, policy_version 1551743 (0.0007) [2023-12-27 16:23:48,268][20347] Updated weights for policy 0, policy_version 1551753 (0.0007) [2023-12-27 16:23:48,595][20381] Updated weights for policy 1, policy_version 1550153 (0.0008) [2023-12-27 16:23:48,653][20381] Updated weights for policy 1, policy_version 1550163 (0.0008) [2023-12-27 16:23:48,713][20381] Updated weights for policy 1, policy_version 1550173 (0.0008) [2023-12-27 16:23:48,770][20381] Updated weights for policy 1, policy_version 1550183 (0.0008) [2023-12-27 16:23:48,980][20347] Updated weights for policy 0, policy_version 1551763 (0.0006) [2023-12-27 16:23:49,057][20347] Updated weights for policy 0, policy_version 1551773 (0.0007) [2023-12-27 16:23:49,122][20347] Updated weights for policy 0, policy_version 1551783 (0.0007) [2023-12-27 16:23:49,543][20381] Updated weights for policy 1, policy_version 1550193 (0.0008) [2023-12-27 16:23:49,597][20381] Updated weights for policy 1, policy_version 1550203 (0.0008) [2023-12-27 16:23:49,660][20381] Updated weights for policy 1, policy_version 1550213 (0.0006) [2023-12-27 16:23:49,824][20347] Updated weights for policy 0, policy_version 1551793 (0.0006) [2023-12-27 16:23:49,889][20347] Updated weights for policy 0, policy_version 1551803 (0.0009) [2023-12-27 16:23:49,954][20347] Updated weights for policy 0, policy_version 1551813 (0.0009) [2023-12-27 16:23:50,013][20347] Updated weights for policy 0, policy_version 1551823 (0.0007) [2023-12-27 16:23:50,389][20381] Updated weights for policy 1, policy_version 1550223 (0.0007) [2023-12-27 16:23:50,442][20381] Updated weights for policy 1, policy_version 1550233 (0.0008) [2023-12-27 16:23:50,492][20381] Updated weights for policy 1, policy_version 1550243 (0.0008) [2023-12-27 16:23:50,734][20347] Updated weights for policy 0, policy_version 1551833 (0.0006) [2023-12-27 16:23:50,790][20347] Updated weights for policy 0, policy_version 1551843 (0.0009) [2023-12-27 16:23:50,850][20347] Updated weights for policy 0, policy_version 1551853 (0.0008) [2023-12-27 16:23:51,222][20381] Updated weights for policy 1, policy_version 1550253 (0.0006) [2023-12-27 16:23:51,279][20381] Updated weights for policy 1, policy_version 1550263 (0.0008) [2023-12-27 16:23:51,337][20381] Updated weights for policy 1, policy_version 1550273 (0.0007) [2023-12-27 16:23:51,597][20347] Updated weights for policy 0, policy_version 1551863 (0.0008) [2023-12-27 16:23:51,667][20347] Updated weights for policy 0, policy_version 1551873 (0.0007) [2023-12-27 16:23:51,736][20347] Updated weights for policy 0, policy_version 1551883 (0.0007) [2023-12-27 16:23:52,093][20381] Updated weights for policy 1, policy_version 1550283 (0.0007) [2023-12-27 16:23:52,140][20381] Updated weights for policy 1, policy_version 1550293 (0.0008) [2023-12-27 16:23:52,182][20381] Updated weights for policy 1, policy_version 1550303 (0.0008) [2023-12-27 16:23:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 794279936. Throughput: 0: 10197.1, 1: 10409.5. Samples: 794251116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:52,363][19170] Avg episode reward: [(0, '272.740'), (1, '614.570')] [2023-12-27 16:23:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001550312_396918784.pth... [2023-12-27 16:23:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001551888_397361152.pth... [2023-12-27 16:23:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001550704_397058048.pth [2023-12-27 16:23:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001549064_396599296.pth [2023-12-27 16:23:52,414][20347] Updated weights for policy 0, policy_version 1551893 (0.0008) [2023-12-27 16:23:52,470][20347] Updated weights for policy 0, policy_version 1551903 (0.0008) [2023-12-27 16:23:52,528][20347] Updated weights for policy 0, policy_version 1551913 (0.0007) [2023-12-27 16:23:52,940][20381] Updated weights for policy 1, policy_version 1550313 (0.0008) [2023-12-27 16:23:52,996][20381] Updated weights for policy 1, policy_version 1550323 (0.0007) [2023-12-27 16:23:53,046][20381] Updated weights for policy 1, policy_version 1550333 (0.0008) [2023-12-27 16:23:53,091][20381] Updated weights for policy 1, policy_version 1550343 (0.0009) [2023-12-27 16:23:53,157][20347] Updated weights for policy 0, policy_version 1551923 (0.0006) [2023-12-27 16:23:53,219][20347] Updated weights for policy 0, policy_version 1551933 (0.0005) [2023-12-27 16:23:53,277][20347] Updated weights for policy 0, policy_version 1551943 (0.0007) [2023-12-27 16:23:53,815][20381] Updated weights for policy 1, policy_version 1550353 (0.0007) [2023-12-27 16:23:53,867][20381] Updated weights for policy 1, policy_version 1550363 (0.0009) [2023-12-27 16:23:53,906][20347] Updated weights for policy 0, policy_version 1551953 (0.0007) [2023-12-27 16:23:53,918][20381] Updated weights for policy 1, policy_version 1550373 (0.0009) [2023-12-27 16:23:53,970][20347] Updated weights for policy 0, policy_version 1551963 (0.0005) [2023-12-27 16:23:54,029][20347] Updated weights for policy 0, policy_version 1551973 (0.0005) [2023-12-27 16:23:54,090][20347] Updated weights for policy 0, policy_version 1551983 (0.0009) [2023-12-27 16:23:54,597][20381] Updated weights for policy 1, policy_version 1550383 (0.0010) [2023-12-27 16:23:54,643][20381] Updated weights for policy 1, policy_version 1550393 (0.0010) [2023-12-27 16:23:54,699][20381] Updated weights for policy 1, policy_version 1550403 (0.0008) [2023-12-27 16:23:54,699][20347] Updated weights for policy 0, policy_version 1551993 (0.0010) [2023-12-27 16:23:54,747][20347] Updated weights for policy 0, policy_version 1552003 (0.0010) [2023-12-27 16:23:54,802][20347] Updated weights for policy 0, policy_version 1552013 (0.0010) [2023-12-27 16:23:55,395][20381] Updated weights for policy 1, policy_version 1550413 (0.0009) [2023-12-27 16:23:55,452][20381] Updated weights for policy 1, policy_version 1550423 (0.0008) [2023-12-27 16:23:55,452][20347] Updated weights for policy 0, policy_version 1552023 (0.0011) [2023-12-27 16:23:55,503][20381] Updated weights for policy 1, policy_version 1550433 (0.0008) [2023-12-27 16:23:55,510][20347] Updated weights for policy 0, policy_version 1552033 (0.0009) [2023-12-27 16:23:55,568][20347] Updated weights for policy 0, policy_version 1552043 (0.0007) [2023-12-27 16:23:56,114][20381] Updated weights for policy 1, policy_version 1550443 (0.0009) [2023-12-27 16:23:56,168][20381] Updated weights for policy 1, policy_version 1550453 (0.0011) [2023-12-27 16:23:56,191][20347] Updated weights for policy 0, policy_version 1552053 (0.0008) [2023-12-27 16:23:56,217][20381] Updated weights for policy 1, policy_version 1550463 (0.0010) [2023-12-27 16:23:56,241][20347] Updated weights for policy 0, policy_version 1552063 (0.0010) [2023-12-27 16:23:56,286][20347] Updated weights for policy 0, policy_version 1552073 (0.0010) [2023-12-27 16:23:56,860][20381] Updated weights for policy 1, policy_version 1550473 (0.0010) [2023-12-27 16:23:56,922][20381] Updated weights for policy 1, policy_version 1550483 (0.0006) [2023-12-27 16:23:56,925][20347] Updated weights for policy 0, policy_version 1552083 (0.0010) [2023-12-27 16:23:56,976][20381] Updated weights for policy 1, policy_version 1550493 (0.0007) [2023-12-27 16:23:56,978][20347] Updated weights for policy 0, policy_version 1552093 (0.0007) [2023-12-27 16:23:57,025][20381] Updated weights for policy 1, policy_version 1550503 (0.0007) [2023-12-27 16:23:57,026][20347] Updated weights for policy 0, policy_version 1552103 (0.0006) [2023-12-27 16:23:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 794386432. Throughput: 0: 10213.9, 1: 10379.9. Samples: 794373116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:23:57,363][19170] Avg episode reward: [(0, '321.470'), (1, '662.850')] [2023-12-27 16:23:57,638][20347] Updated weights for policy 0, policy_version 1552113 (0.0005) [2023-12-27 16:23:57,658][20381] Updated weights for policy 1, policy_version 1550513 (0.0006) [2023-12-27 16:23:57,686][20347] Updated weights for policy 0, policy_version 1552123 (0.0008) [2023-12-27 16:23:57,713][20381] Updated weights for policy 1, policy_version 1550523 (0.0005) [2023-12-27 16:23:57,738][20347] Updated weights for policy 0, policy_version 1552133 (0.0007) [2023-12-27 16:23:57,774][20381] Updated weights for policy 1, policy_version 1550533 (0.0005) [2023-12-27 16:23:57,794][20347] Updated weights for policy 0, policy_version 1552143 (0.0007) [2023-12-27 16:23:58,405][20381] Updated weights for policy 1, policy_version 1550543 (0.0007) [2023-12-27 16:23:58,467][20381] Updated weights for policy 1, policy_version 1550553 (0.0007) [2023-12-27 16:23:58,514][20347] Updated weights for policy 0, policy_version 1552153 (0.0008) [2023-12-27 16:23:58,521][20381] Updated weights for policy 1, policy_version 1550563 (0.0011) [2023-12-27 16:23:58,574][20347] Updated weights for policy 0, policy_version 1552163 (0.0008) [2023-12-27 16:23:58,636][20347] Updated weights for policy 0, policy_version 1552173 (0.0007) [2023-12-27 16:23:59,354][20381] Updated weights for policy 1, policy_version 1550573 (0.0010) [2023-12-27 16:23:59,416][20381] Updated weights for policy 1, policy_version 1550583 (0.0008) [2023-12-27 16:23:59,463][20381] Updated weights for policy 1, policy_version 1550593 (0.0007) [2023-12-27 16:23:59,536][20347] Updated weights for policy 0, policy_version 1552183 (0.0008) [2023-12-27 16:23:59,593][20347] Updated weights for policy 0, policy_version 1552193 (0.0006) [2023-12-27 16:23:59,659][20347] Updated weights for policy 0, policy_version 1552203 (0.0009) [2023-12-27 16:24:00,104][20381] Updated weights for policy 1, policy_version 1550603 (0.0006) [2023-12-27 16:24:00,153][20381] Updated weights for policy 1, policy_version 1550613 (0.0005) [2023-12-27 16:24:00,212][20381] Updated weights for policy 1, policy_version 1550623 (0.0007) [2023-12-27 16:24:00,330][20347] Updated weights for policy 0, policy_version 1552213 (0.0007) [2023-12-27 16:24:00,382][20347] Updated weights for policy 0, policy_version 1552223 (0.0007) [2023-12-27 16:24:00,439][20347] Updated weights for policy 0, policy_version 1552233 (0.0008) [2023-12-27 16:24:00,805][20381] Updated weights for policy 1, policy_version 1550633 (0.0008) [2023-12-27 16:24:00,851][20381] Updated weights for policy 1, policy_version 1550643 (0.0009) [2023-12-27 16:24:00,907][20381] Updated weights for policy 1, policy_version 1550653 (0.0008) [2023-12-27 16:24:00,962][20381] Updated weights for policy 1, policy_version 1550663 (0.0008) [2023-12-27 16:24:01,002][20347] Updated weights for policy 0, policy_version 1552243 (0.0008) [2023-12-27 16:24:01,058][20347] Updated weights for policy 0, policy_version 1552253 (0.0006) [2023-12-27 16:24:01,107][20347] Updated weights for policy 0, policy_version 1552263 (0.0008) [2023-12-27 16:24:01,713][20381] Updated weights for policy 1, policy_version 1550673 (0.0008) [2023-12-27 16:24:01,772][20381] Updated weights for policy 1, policy_version 1550683 (0.0008) [2023-12-27 16:24:01,827][20381] Updated weights for policy 1, policy_version 1550693 (0.0007) [2023-12-27 16:24:01,841][20347] Updated weights for policy 0, policy_version 1552273 (0.0007) [2023-12-27 16:24:01,899][20347] Updated weights for policy 0, policy_version 1552283 (0.0010) [2023-12-27 16:24:01,963][20347] Updated weights for policy 0, policy_version 1552293 (0.0010) [2023-12-27 16:24:02,022][20347] Updated weights for policy 0, policy_version 1552303 (0.0010) [2023-12-27 16:24:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 794484736. Throughput: 0: 10188.3, 1: 10421.8. Samples: 794495188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:24:02,363][19170] Avg episode reward: [(0, '298.090'), (1, '673.120')] [2023-12-27 16:24:02,519][20381] Updated weights for policy 1, policy_version 1550703 (0.0007) [2023-12-27 16:24:02,577][20381] Updated weights for policy 1, policy_version 1550713 (0.0007) [2023-12-27 16:24:02,631][20381] Updated weights for policy 1, policy_version 1550723 (0.0007) [2023-12-27 16:24:02,674][20347] Updated weights for policy 0, policy_version 1552313 (0.0010) [2023-12-27 16:24:02,728][20347] Updated weights for policy 0, policy_version 1552323 (0.0010) [2023-12-27 16:24:02,789][20347] Updated weights for policy 0, policy_version 1552333 (0.0010) [2023-12-27 16:24:03,292][20381] Updated weights for policy 1, policy_version 1550733 (0.0006) [2023-12-27 16:24:03,346][20381] Updated weights for policy 1, policy_version 1550743 (0.0005) [2023-12-27 16:24:03,402][20381] Updated weights for policy 1, policy_version 1550753 (0.0005) [2023-12-27 16:24:03,418][20347] Updated weights for policy 0, policy_version 1552343 (0.0010) [2023-12-27 16:24:03,474][20347] Updated weights for policy 0, policy_version 1552353 (0.0010) [2023-12-27 16:24:03,528][20347] Updated weights for policy 0, policy_version 1552363 (0.0010) [2023-12-27 16:24:04,001][20381] Updated weights for policy 1, policy_version 1550763 (0.0006) [2023-12-27 16:24:04,062][20381] Updated weights for policy 1, policy_version 1550773 (0.0008) [2023-12-27 16:24:04,124][20381] Updated weights for policy 1, policy_version 1550783 (0.0008) [2023-12-27 16:24:04,248][20347] Updated weights for policy 0, policy_version 1552373 (0.0010) [2023-12-27 16:24:04,309][20347] Updated weights for policy 0, policy_version 1552383 (0.0009) [2023-12-27 16:24:04,365][20347] Updated weights for policy 0, policy_version 1552393 (0.0008) [2023-12-27 16:24:04,752][20381] Updated weights for policy 1, policy_version 1550793 (0.0007) [2023-12-27 16:24:04,810][20381] Updated weights for policy 1, policy_version 1550803 (0.0005) [2023-12-27 16:24:04,872][20381] Updated weights for policy 1, policy_version 1550813 (0.0005) [2023-12-27 16:24:04,933][20381] Updated weights for policy 1, policy_version 1550823 (0.0005) [2023-12-27 16:24:04,989][20347] Updated weights for policy 0, policy_version 1552403 (0.0007) [2023-12-27 16:24:05,046][20347] Updated weights for policy 0, policy_version 1552413 (0.0008) [2023-12-27 16:24:05,100][20347] Updated weights for policy 0, policy_version 1552423 (0.0008) [2023-12-27 16:24:05,473][20381] Updated weights for policy 1, policy_version 1550833 (0.0007) [2023-12-27 16:24:05,521][20381] Updated weights for policy 1, policy_version 1550843 (0.0009) [2023-12-27 16:24:05,565][20381] Updated weights for policy 1, policy_version 1550853 (0.0008) [2023-12-27 16:24:05,783][20347] Updated weights for policy 0, policy_version 1552433 (0.0006) [2023-12-27 16:24:05,838][20347] Updated weights for policy 0, policy_version 1552443 (0.0007) [2023-12-27 16:24:05,906][20347] Updated weights for policy 0, policy_version 1552453 (0.0008) [2023-12-27 16:24:05,969][20347] Updated weights for policy 0, policy_version 1552463 (0.0007) [2023-12-27 16:24:06,256][20381] Updated weights for policy 1, policy_version 1550863 (0.0007) [2023-12-27 16:24:06,311][20381] Updated weights for policy 1, policy_version 1550873 (0.0007) [2023-12-27 16:24:06,371][20381] Updated weights for policy 1, policy_version 1550883 (0.0007) [2023-12-27 16:24:06,718][20347] Updated weights for policy 0, policy_version 1552473 (0.0011) [2023-12-27 16:24:06,780][20347] Updated weights for policy 0, policy_version 1552483 (0.0010) [2023-12-27 16:24:06,840][20347] Updated weights for policy 0, policy_version 1552493 (0.0011) [2023-12-27 16:24:07,076][20381] Updated weights for policy 1, policy_version 1550893 (0.0009) [2023-12-27 16:24:07,128][20381] Updated weights for policy 1, policy_version 1550903 (0.0011) [2023-12-27 16:24:07,178][20381] Updated weights for policy 1, policy_version 1550913 (0.0009) [2023-12-27 16:24:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 794591232. Throughput: 0: 10188.9, 1: 10447.9. Samples: 794559676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:24:07,363][19170] Avg episode reward: [(0, '334.850'), (1, '673.260')] [2023-12-27 16:24:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001550920_397074432.pth... [2023-12-27 16:24:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001552496_397516800.pth... [2023-12-27 16:24:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001549704_396763136.pth [2023-12-27 16:24:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001551312_397213696.pth [2023-12-27 16:24:07,550][20347] Updated weights for policy 0, policy_version 1552503 (0.0008) [2023-12-27 16:24:07,602][20347] Updated weights for policy 0, policy_version 1552513 (0.0007) [2023-12-27 16:24:07,651][20347] Updated weights for policy 0, policy_version 1552523 (0.0007) [2023-12-27 16:24:07,728][20381] Updated weights for policy 1, policy_version 1550923 (0.0006) [2023-12-27 16:24:07,782][20381] Updated weights for policy 1, policy_version 1550933 (0.0007) [2023-12-27 16:24:07,829][20381] Updated weights for policy 1, policy_version 1550943 (0.0008) [2023-12-27 16:24:08,321][20347] Updated weights for policy 0, policy_version 1552533 (0.0008) [2023-12-27 16:24:08,386][20347] Updated weights for policy 0, policy_version 1552543 (0.0009) [2023-12-27 16:24:08,447][20347] Updated weights for policy 0, policy_version 1552553 (0.0006) [2023-12-27 16:24:08,496][20381] Updated weights for policy 1, policy_version 1550953 (0.0009) [2023-12-27 16:24:08,553][20381] Updated weights for policy 1, policy_version 1550963 (0.0009) [2023-12-27 16:24:08,618][20381] Updated weights for policy 1, policy_version 1550973 (0.0005) [2023-12-27 16:24:08,679][20381] Updated weights for policy 1, policy_version 1550983 (0.0005) [2023-12-27 16:24:09,092][20347] Updated weights for policy 0, policy_version 1552563 (0.0007) [2023-12-27 16:24:09,138][20347] Updated weights for policy 0, policy_version 1552573 (0.0005) [2023-12-27 16:24:09,189][20347] Updated weights for policy 0, policy_version 1552583 (0.0005) [2023-12-27 16:24:09,285][20381] Updated weights for policy 1, policy_version 1550993 (0.0010) [2023-12-27 16:24:09,346][20381] Updated weights for policy 1, policy_version 1551003 (0.0011) [2023-12-27 16:24:09,412][20381] Updated weights for policy 1, policy_version 1551013 (0.0008) [2023-12-27 16:24:09,967][20347] Updated weights for policy 0, policy_version 1552593 (0.0007) [2023-12-27 16:24:10,019][20347] Updated weights for policy 0, policy_version 1552603 (0.0008) [2023-12-27 16:24:10,071][20347] Updated weights for policy 0, policy_version 1552613 (0.0007) [2023-12-27 16:24:10,117][20381] Updated weights for policy 1, policy_version 1551023 (0.0010) [2023-12-27 16:24:10,130][20347] Updated weights for policy 0, policy_version 1552623 (0.0006) [2023-12-27 16:24:10,170][20381] Updated weights for policy 1, policy_version 1551033 (0.0008) [2023-12-27 16:24:10,226][20381] Updated weights for policy 1, policy_version 1551043 (0.0006) [2023-12-27 16:24:10,859][20381] Updated weights for policy 1, policy_version 1551053 (0.0008) [2023-12-27 16:24:10,868][20347] Updated weights for policy 0, policy_version 1552633 (0.0008) [2023-12-27 16:24:10,906][20381] Updated weights for policy 1, policy_version 1551063 (0.0007) [2023-12-27 16:24:10,926][20347] Updated weights for policy 0, policy_version 1552643 (0.0007) [2023-12-27 16:24:10,953][20381] Updated weights for policy 1, policy_version 1551073 (0.0005) [2023-12-27 16:24:10,986][20347] Updated weights for policy 0, policy_version 1552653 (0.0008) [2023-12-27 16:24:11,646][20381] Updated weights for policy 1, policy_version 1551083 (0.0006) [2023-12-27 16:24:11,716][20381] Updated weights for policy 1, policy_version 1551093 (0.0008) [2023-12-27 16:24:11,775][20381] Updated weights for policy 1, policy_version 1551103 (0.0008) [2023-12-27 16:24:11,798][20347] Updated weights for policy 0, policy_version 1552663 (0.0007) [2023-12-27 16:24:11,856][20347] Updated weights for policy 0, policy_version 1552673 (0.0008) [2023-12-27 16:24:11,917][20347] Updated weights for policy 0, policy_version 1552683 (0.0008) [2023-12-27 16:24:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 794689536. Throughput: 0: 10139.7, 1: 10481.5. Samples: 794682144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:24:12,363][19170] Avg episode reward: [(0, '384.230'), (1, '624.670')] [2023-12-27 16:24:12,521][20381] Updated weights for policy 1, policy_version 1551113 (0.0007) [2023-12-27 16:24:12,538][20347] Updated weights for policy 0, policy_version 1552693 (0.0008) [2023-12-27 16:24:12,586][20381] Updated weights for policy 1, policy_version 1551123 (0.0005) [2023-12-27 16:24:12,598][20347] Updated weights for policy 0, policy_version 1552703 (0.0006) [2023-12-27 16:24:12,647][20381] Updated weights for policy 1, policy_version 1551133 (0.0005) [2023-12-27 16:24:12,654][20347] Updated weights for policy 0, policy_version 1552713 (0.0007) [2023-12-27 16:24:12,709][20381] Updated weights for policy 1, policy_version 1551143 (0.0006) [2023-12-27 16:24:13,305][20347] Updated weights for policy 0, policy_version 1552723 (0.0007) [2023-12-27 16:24:13,348][20381] Updated weights for policy 1, policy_version 1551153 (0.0006) [2023-12-27 16:24:13,362][20347] Updated weights for policy 0, policy_version 1552733 (0.0007) [2023-12-27 16:24:13,404][20381] Updated weights for policy 1, policy_version 1551163 (0.0007) [2023-12-27 16:24:13,411][20347] Updated weights for policy 0, policy_version 1552743 (0.0005) [2023-12-27 16:24:13,460][20381] Updated weights for policy 1, policy_version 1551173 (0.0007) [2023-12-27 16:24:13,988][20347] Updated weights for policy 0, policy_version 1552753 (0.0005) [2023-12-27 16:24:14,051][20347] Updated weights for policy 0, policy_version 1552763 (0.0005) [2023-12-27 16:24:14,108][20347] Updated weights for policy 0, policy_version 1552773 (0.0005) [2023-12-27 16:24:14,169][20347] Updated weights for policy 0, policy_version 1552783 (0.0005) [2023-12-27 16:24:14,198][20381] Updated weights for policy 1, policy_version 1551183 (0.0010) [2023-12-27 16:24:14,256][20381] Updated weights for policy 1, policy_version 1551193 (0.0009) [2023-12-27 16:24:14,309][20381] Updated weights for policy 1, policy_version 1551203 (0.0007) [2023-12-27 16:24:14,764][20347] Updated weights for policy 0, policy_version 1552793 (0.0007) [2023-12-27 16:24:14,827][20347] Updated weights for policy 0, policy_version 1552803 (0.0008) [2023-12-27 16:24:14,892][20347] Updated weights for policy 0, policy_version 1552813 (0.0008) [2023-12-27 16:24:15,004][20381] Updated weights for policy 1, policy_version 1551213 (0.0007) [2023-12-27 16:24:15,065][20381] Updated weights for policy 1, policy_version 1551223 (0.0008) [2023-12-27 16:24:15,124][20381] Updated weights for policy 1, policy_version 1551233 (0.0007) [2023-12-27 16:24:15,651][20347] Updated weights for policy 0, policy_version 1552823 (0.0007) [2023-12-27 16:24:15,711][20347] Updated weights for policy 0, policy_version 1552833 (0.0006) [2023-12-27 16:24:15,724][20381] Updated weights for policy 1, policy_version 1551243 (0.0007) [2023-12-27 16:24:15,768][20347] Updated weights for policy 0, policy_version 1552843 (0.0006) [2023-12-27 16:24:15,788][20381] Updated weights for policy 1, policy_version 1551253 (0.0008) [2023-12-27 16:24:15,850][20381] Updated weights for policy 1, policy_version 1551263 (0.0009) [2023-12-27 16:24:16,423][20347] Updated weights for policy 0, policy_version 1552853 (0.0006) [2023-12-27 16:24:16,486][20347] Updated weights for policy 0, policy_version 1552863 (0.0006) [2023-12-27 16:24:16,515][20381] Updated weights for policy 1, policy_version 1551273 (0.0007) [2023-12-27 16:24:16,540][20347] Updated weights for policy 0, policy_version 1552873 (0.0006) [2023-12-27 16:24:16,571][20381] Updated weights for policy 1, policy_version 1551283 (0.0007) [2023-12-27 16:24:16,619][20381] Updated weights for policy 1, policy_version 1551293 (0.0007) [2023-12-27 16:24:16,661][20381] Updated weights for policy 1, policy_version 1551303 (0.0008) [2023-12-27 16:24:17,202][20347] Updated weights for policy 0, policy_version 1552883 (0.0008) [2023-12-27 16:24:17,248][20347] Updated weights for policy 0, policy_version 1552893 (0.0008) [2023-12-27 16:24:17,295][20347] Updated weights for policy 0, policy_version 1552903 (0.0008) [2023-12-27 16:24:17,329][20381] Updated weights for policy 1, policy_version 1551313 (0.0007) [2023-12-27 16:24:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 794796032. Throughput: 0: 10122.0, 1: 10386.1. Samples: 794805208. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:17,363][19170] Avg episode reward: [(0, '344.440'), (1, '624.010')] [2023-12-27 16:24:17,384][20381] Updated weights for policy 1, policy_version 1551323 (0.0007) [2023-12-27 16:24:17,432][20381] Updated weights for policy 1, policy_version 1551333 (0.0007) [2023-12-27 16:24:18,010][20347] Updated weights for policy 0, policy_version 1552913 (0.0009) [2023-12-27 16:24:18,065][20347] Updated weights for policy 0, policy_version 1552923 (0.0010) [2023-12-27 16:24:18,119][20381] Updated weights for policy 1, policy_version 1551343 (0.0006) [2023-12-27 16:24:18,124][20347] Updated weights for policy 0, policy_version 1552933 (0.0011) [2023-12-27 16:24:18,167][20381] Updated weights for policy 1, policy_version 1551353 (0.0005) [2023-12-27 16:24:18,176][20347] Updated weights for policy 0, policy_version 1552943 (0.0010) [2023-12-27 16:24:18,217][20381] Updated weights for policy 1, policy_version 1551363 (0.0007) [2023-12-27 16:24:18,873][20381] Updated weights for policy 1, policy_version 1551373 (0.0007) [2023-12-27 16:24:18,874][20347] Updated weights for policy 0, policy_version 1552953 (0.0011) [2023-12-27 16:24:18,933][20381] Updated weights for policy 1, policy_version 1551383 (0.0007) [2023-12-27 16:24:18,934][20347] Updated weights for policy 0, policy_version 1552963 (0.0011) [2023-12-27 16:24:18,984][20381] Updated weights for policy 1, policy_version 1551393 (0.0006) [2023-12-27 16:24:18,986][20347] Updated weights for policy 0, policy_version 1552973 (0.0010) [2023-12-27 16:24:19,616][20381] Updated weights for policy 1, policy_version 1551403 (0.0006) [2023-12-27 16:24:19,678][20381] Updated weights for policy 1, policy_version 1551413 (0.0007) [2023-12-27 16:24:19,695][20347] Updated weights for policy 0, policy_version 1552983 (0.0011) [2023-12-27 16:24:19,735][20381] Updated weights for policy 1, policy_version 1551423 (0.0006) [2023-12-27 16:24:19,756][20347] Updated weights for policy 0, policy_version 1552993 (0.0011) [2023-12-27 16:24:19,816][20347] Updated weights for policy 0, policy_version 1553003 (0.0010) [2023-12-27 16:24:20,469][20381] Updated weights for policy 1, policy_version 1551433 (0.0006) [2023-12-27 16:24:20,519][20381] Updated weights for policy 1, policy_version 1551443 (0.0008) [2023-12-27 16:24:20,563][20347] Updated weights for policy 0, policy_version 1553013 (0.0009) [2023-12-27 16:24:20,571][20381] Updated weights for policy 1, policy_version 1551453 (0.0007) [2023-12-27 16:24:20,622][20347] Updated weights for policy 0, policy_version 1553023 (0.0008) [2023-12-27 16:24:20,629][20381] Updated weights for policy 1, policy_version 1551463 (0.0007) [2023-12-27 16:24:20,682][20347] Updated weights for policy 0, policy_version 1553033 (0.0008) [2023-12-27 16:24:21,398][20381] Updated weights for policy 1, policy_version 1551473 (0.0011) [2023-12-27 16:24:21,444][20347] Updated weights for policy 0, policy_version 1553043 (0.0007) [2023-12-27 16:24:21,458][20381] Updated weights for policy 1, policy_version 1551483 (0.0011) [2023-12-27 16:24:21,504][20347] Updated weights for policy 0, policy_version 1553053 (0.0006) [2023-12-27 16:24:21,517][20381] Updated weights for policy 1, policy_version 1551493 (0.0011) [2023-12-27 16:24:21,565][20347] Updated weights for policy 0, policy_version 1553063 (0.0006) [2023-12-27 16:24:22,164][20381] Updated weights for policy 1, policy_version 1551503 (0.0008) [2023-12-27 16:24:22,217][20381] Updated weights for policy 1, policy_version 1551513 (0.0008) [2023-12-27 16:24:22,241][20347] Updated weights for policy 0, policy_version 1553073 (0.0008) [2023-12-27 16:24:22,284][20381] Updated weights for policy 1, policy_version 1551523 (0.0007) [2023-12-27 16:24:22,305][20347] Updated weights for policy 0, policy_version 1553083 (0.0006) [2023-12-27 16:24:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 794894336. Throughput: 0: 10086.0, 1: 10331.3. Samples: 794865936. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:22,363][19170] Avg episode reward: [(0, '363.020'), (1, '700.350')] [2023-12-27 16:24:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001551528_397230080.pth... [2023-12-27 16:24:22,370][20347] Updated weights for policy 0, policy_version 1553093 (0.0008) [2023-12-27 16:24:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001550312_396918784.pth [2023-12-27 16:24:22,427][20347] Updated weights for policy 0, policy_version 1553103 (0.0008) [2023-12-27 16:24:22,432][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001553104_397672448.pth... [2023-12-27 16:24:22,435][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001551888_397361152.pth [2023-12-27 16:24:23,032][20381] Updated weights for policy 1, policy_version 1551533 (0.0009) [2023-12-27 16:24:23,087][20381] Updated weights for policy 1, policy_version 1551543 (0.0009) [2023-12-27 16:24:23,137][20381] Updated weights for policy 1, policy_version 1551553 (0.0011) [2023-12-27 16:24:23,158][20347] Updated weights for policy 0, policy_version 1553113 (0.0007) [2023-12-27 16:24:23,211][20347] Updated weights for policy 0, policy_version 1553123 (0.0007) [2023-12-27 16:24:23,269][20347] Updated weights for policy 0, policy_version 1553133 (0.0008) [2023-12-27 16:24:23,782][20381] Updated weights for policy 1, policy_version 1551563 (0.0009) [2023-12-27 16:24:23,845][20381] Updated weights for policy 1, policy_version 1551573 (0.0006) [2023-12-27 16:24:23,896][20381] Updated weights for policy 1, policy_version 1551583 (0.0007) [2023-12-27 16:24:24,006][20347] Updated weights for policy 0, policy_version 1553143 (0.0008) [2023-12-27 16:24:24,066][20347] Updated weights for policy 0, policy_version 1553153 (0.0008) [2023-12-27 16:24:24,127][20347] Updated weights for policy 0, policy_version 1553163 (0.0008) [2023-12-27 16:24:24,480][20381] Updated weights for policy 1, policy_version 1551593 (0.0006) [2023-12-27 16:24:24,529][20381] Updated weights for policy 1, policy_version 1551603 (0.0006) [2023-12-27 16:24:24,573][20381] Updated weights for policy 1, policy_version 1551613 (0.0005) [2023-12-27 16:24:24,618][20381] Updated weights for policy 1, policy_version 1551623 (0.0006) [2023-12-27 16:24:24,866][20347] Updated weights for policy 0, policy_version 1553173 (0.0007) [2023-12-27 16:24:24,924][20347] Updated weights for policy 0, policy_version 1553183 (0.0008) [2023-12-27 16:24:24,982][20347] Updated weights for policy 0, policy_version 1553193 (0.0008) [2023-12-27 16:24:25,196][20381] Updated weights for policy 1, policy_version 1551633 (0.0006) [2023-12-27 16:24:25,249][20381] Updated weights for policy 1, policy_version 1551643 (0.0007) [2023-12-27 16:24:25,299][20381] Updated weights for policy 1, policy_version 1551653 (0.0008) [2023-12-27 16:24:25,654][20347] Updated weights for policy 0, policy_version 1553203 (0.0006) [2023-12-27 16:24:25,705][20347] Updated weights for policy 0, policy_version 1553213 (0.0005) [2023-12-27 16:24:25,755][20347] Updated weights for policy 0, policy_version 1553223 (0.0005) [2023-12-27 16:24:25,915][20381] Updated weights for policy 1, policy_version 1551663 (0.0007) [2023-12-27 16:24:25,967][20381] Updated weights for policy 1, policy_version 1551673 (0.0007) [2023-12-27 16:24:26,018][20381] Updated weights for policy 1, policy_version 1551683 (0.0007) [2023-12-27 16:24:26,345][20347] Updated weights for policy 0, policy_version 1553233 (0.0006) [2023-12-27 16:24:26,403][20347] Updated weights for policy 0, policy_version 1553243 (0.0005) [2023-12-27 16:24:26,462][20347] Updated weights for policy 0, policy_version 1553253 (0.0008) [2023-12-27 16:24:26,524][20347] Updated weights for policy 0, policy_version 1553263 (0.0008) [2023-12-27 16:24:26,632][20381] Updated weights for policy 1, policy_version 1551693 (0.0007) [2023-12-27 16:24:26,689][20381] Updated weights for policy 1, policy_version 1551703 (0.0010) [2023-12-27 16:24:26,749][20381] Updated weights for policy 1, policy_version 1551713 (0.0010) [2023-12-27 16:24:27,198][20347] Updated weights for policy 0, policy_version 1553273 (0.0008) [2023-12-27 16:24:27,242][20347] Updated weights for policy 0, policy_version 1553283 (0.0005) [2023-12-27 16:24:27,295][20347] Updated weights for policy 0, policy_version 1553293 (0.0007) [2023-12-27 16:24:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 795000832. Throughput: 0: 10032.3, 1: 10417.7. Samples: 794988324. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:27,363][19170] Avg episode reward: [(0, '395.040'), (1, '651.130')] [2023-12-27 16:24:27,369][20381] Updated weights for policy 1, policy_version 1551723 (0.0010) [2023-12-27 16:24:27,430][20381] Updated weights for policy 1, policy_version 1551733 (0.0009) [2023-12-27 16:24:27,484][20381] Updated weights for policy 1, policy_version 1551743 (0.0009) [2023-12-27 16:24:27,945][20347] Updated weights for policy 0, policy_version 1553303 (0.0007) [2023-12-27 16:24:28,000][20347] Updated weights for policy 0, policy_version 1553313 (0.0008) [2023-12-27 16:24:28,058][20347] Updated weights for policy 0, policy_version 1553323 (0.0007) [2023-12-27 16:24:28,097][20381] Updated weights for policy 1, policy_version 1551753 (0.0006) [2023-12-27 16:24:28,148][20381] Updated weights for policy 1, policy_version 1551763 (0.0007) [2023-12-27 16:24:28,201][20381] Updated weights for policy 1, policy_version 1551773 (0.0008) [2023-12-27 16:24:28,250][20381] Updated weights for policy 1, policy_version 1551783 (0.0008) [2023-12-27 16:24:28,771][20347] Updated weights for policy 0, policy_version 1553333 (0.0007) [2023-12-27 16:24:28,816][20347] Updated weights for policy 0, policy_version 1553343 (0.0007) [2023-12-27 16:24:28,868][20347] Updated weights for policy 0, policy_version 1553353 (0.0007) [2023-12-27 16:24:28,935][20381] Updated weights for policy 1, policy_version 1551793 (0.0005) [2023-12-27 16:24:28,993][20381] Updated weights for policy 1, policy_version 1551803 (0.0005) [2023-12-27 16:24:29,051][20381] Updated weights for policy 1, policy_version 1551813 (0.0005) [2023-12-27 16:24:29,640][20381] Updated weights for policy 1, policy_version 1551823 (0.0007) [2023-12-27 16:24:29,656][20347] Updated weights for policy 0, policy_version 1553363 (0.0008) [2023-12-27 16:24:29,694][20381] Updated weights for policy 1, policy_version 1551833 (0.0008) [2023-12-27 16:24:29,708][20347] Updated weights for policy 0, policy_version 1553373 (0.0005) [2023-12-27 16:24:29,748][20381] Updated weights for policy 1, policy_version 1551843 (0.0006) [2023-12-27 16:24:29,768][20347] Updated weights for policy 0, policy_version 1553383 (0.0007) [2023-12-27 16:24:30,387][20381] Updated weights for policy 1, policy_version 1551853 (0.0007) [2023-12-27 16:24:30,436][20381] Updated weights for policy 1, policy_version 1551863 (0.0005) [2023-12-27 16:24:30,490][20381] Updated weights for policy 1, policy_version 1551873 (0.0007) [2023-12-27 16:24:30,490][20347] Updated weights for policy 0, policy_version 1553393 (0.0007) [2023-12-27 16:24:30,539][20347] Updated weights for policy 0, policy_version 1553403 (0.0005) [2023-12-27 16:24:30,589][20347] Updated weights for policy 0, policy_version 1553413 (0.0005) [2023-12-27 16:24:30,638][20347] Updated weights for policy 0, policy_version 1553423 (0.0008) [2023-12-27 16:24:31,064][20381] Updated weights for policy 1, policy_version 1551883 (0.0007) [2023-12-27 16:24:31,119][20381] Updated weights for policy 1, policy_version 1551893 (0.0008) [2023-12-27 16:24:31,189][20381] Updated weights for policy 1, policy_version 1551903 (0.0008) [2023-12-27 16:24:31,360][20347] Updated weights for policy 0, policy_version 1553433 (0.0007) [2023-12-27 16:24:31,418][20347] Updated weights for policy 0, policy_version 1553443 (0.0007) [2023-12-27 16:24:31,471][20347] Updated weights for policy 0, policy_version 1553453 (0.0007) [2023-12-27 16:24:31,946][20381] Updated weights for policy 1, policy_version 1551913 (0.0007) [2023-12-27 16:24:32,001][20381] Updated weights for policy 1, policy_version 1551923 (0.0010) [2023-12-27 16:24:32,050][20381] Updated weights for policy 1, policy_version 1551933 (0.0010) [2023-12-27 16:24:32,109][20381] Updated weights for policy 1, policy_version 1551943 (0.0010) [2023-12-27 16:24:32,175][20347] Updated weights for policy 0, policy_version 1553463 (0.0007) [2023-12-27 16:24:32,223][20347] Updated weights for policy 0, policy_version 1553473 (0.0007) [2023-12-27 16:24:32,277][20347] Updated weights for policy 0, policy_version 1553483 (0.0008) [2023-12-27 16:24:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 795107328. Throughput: 0: 10050.6, 1: 10398.8. Samples: 795112552. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:32,363][19170] Avg episode reward: [(0, '334.640'), (1, '718.280')] [2023-12-27 16:24:32,807][20381] Updated weights for policy 1, policy_version 1551953 (0.0008) [2023-12-27 16:24:32,867][20381] Updated weights for policy 1, policy_version 1551963 (0.0008) [2023-12-27 16:24:32,923][20381] Updated weights for policy 1, policy_version 1551973 (0.0008) [2023-12-27 16:24:32,990][20347] Updated weights for policy 0, policy_version 1553493 (0.0007) [2023-12-27 16:24:33,042][20347] Updated weights for policy 0, policy_version 1553503 (0.0007) [2023-12-27 16:24:33,095][20347] Updated weights for policy 0, policy_version 1553513 (0.0008) [2023-12-27 16:24:33,557][20381] Updated weights for policy 1, policy_version 1551983 (0.0007) [2023-12-27 16:24:33,615][20381] Updated weights for policy 1, policy_version 1551993 (0.0008) [2023-12-27 16:24:33,671][20381] Updated weights for policy 1, policy_version 1552003 (0.0005) [2023-12-27 16:24:33,822][20347] Updated weights for policy 0, policy_version 1553523 (0.0008) [2023-12-27 16:24:33,875][20347] Updated weights for policy 0, policy_version 1553533 (0.0007) [2023-12-27 16:24:33,922][20347] Updated weights for policy 0, policy_version 1553543 (0.0008) [2023-12-27 16:24:34,267][20381] Updated weights for policy 1, policy_version 1552013 (0.0006) [2023-12-27 16:24:34,326][20381] Updated weights for policy 1, policy_version 1552023 (0.0008) [2023-12-27 16:24:34,384][20381] Updated weights for policy 1, policy_version 1552033 (0.0008) [2023-12-27 16:24:34,608][20347] Updated weights for policy 0, policy_version 1553553 (0.0009) [2023-12-27 16:24:34,670][20347] Updated weights for policy 0, policy_version 1553563 (0.0008) [2023-12-27 16:24:34,719][20347] Updated weights for policy 0, policy_version 1553573 (0.0008) [2023-12-27 16:24:34,772][20347] Updated weights for policy 0, policy_version 1553583 (0.0007) [2023-12-27 16:24:35,088][20381] Updated weights for policy 1, policy_version 1552043 (0.0007) [2023-12-27 16:24:35,143][20381] Updated weights for policy 1, policy_version 1552053 (0.0007) [2023-12-27 16:24:35,196][20381] Updated weights for policy 1, policy_version 1552063 (0.0008) [2023-12-27 16:24:35,479][20347] Updated weights for policy 0, policy_version 1553593 (0.0007) [2023-12-27 16:24:35,525][20347] Updated weights for policy 0, policy_version 1553603 (0.0007) [2023-12-27 16:24:35,583][20347] Updated weights for policy 0, policy_version 1553613 (0.0008) [2023-12-27 16:24:35,869][20381] Updated weights for policy 1, policy_version 1552073 (0.0007) [2023-12-27 16:24:35,919][20381] Updated weights for policy 1, policy_version 1552083 (0.0007) [2023-12-27 16:24:35,965][20381] Updated weights for policy 1, policy_version 1552093 (0.0008) [2023-12-27 16:24:36,015][20381] Updated weights for policy 1, policy_version 1552103 (0.0008) [2023-12-27 16:24:36,282][20347] Updated weights for policy 0, policy_version 1553623 (0.0008) [2023-12-27 16:24:36,342][20347] Updated weights for policy 0, policy_version 1553633 (0.0006) [2023-12-27 16:24:36,404][20347] Updated weights for policy 0, policy_version 1553643 (0.0006) [2023-12-27 16:24:36,741][20381] Updated weights for policy 1, policy_version 1552113 (0.0007) [2023-12-27 16:24:36,794][20381] Updated weights for policy 1, policy_version 1552123 (0.0010) [2023-12-27 16:24:36,850][20381] Updated weights for policy 1, policy_version 1552133 (0.0009) [2023-12-27 16:24:37,013][20347] Updated weights for policy 0, policy_version 1553653 (0.0005) [2023-12-27 16:24:37,074][20347] Updated weights for policy 0, policy_version 1553663 (0.0009) [2023-12-27 16:24:37,126][20347] Updated weights for policy 0, policy_version 1553673 (0.0010) [2023-12-27 16:24:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 795205632. Throughput: 0: 10063.1, 1: 10444.2. Samples: 795173948. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:37,363][19170] Avg episode reward: [(0, '318.370'), (1, '621.160')] [2023-12-27 16:24:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001552136_397385728.pth... [2023-12-27 16:24:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001553680_397819904.pth... [2023-12-27 16:24:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001550920_397074432.pth [2023-12-27 16:24:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001552496_397516800.pth [2023-12-27 16:24:37,598][20381] Updated weights for policy 1, policy_version 1552143 (0.0008) [2023-12-27 16:24:37,650][20381] Updated weights for policy 1, policy_version 1552153 (0.0008) [2023-12-27 16:24:37,696][20381] Updated weights for policy 1, policy_version 1552163 (0.0008) [2023-12-27 16:24:37,773][20347] Updated weights for policy 0, policy_version 1553683 (0.0009) [2023-12-27 16:24:37,821][20347] Updated weights for policy 0, policy_version 1553693 (0.0009) [2023-12-27 16:24:37,866][20347] Updated weights for policy 0, policy_version 1553703 (0.0010) [2023-12-27 16:24:38,385][20381] Updated weights for policy 1, policy_version 1552173 (0.0006) [2023-12-27 16:24:38,440][20381] Updated weights for policy 1, policy_version 1552183 (0.0009) [2023-12-27 16:24:38,492][20381] Updated weights for policy 1, policy_version 1552193 (0.0010) [2023-12-27 16:24:38,561][20347] Updated weights for policy 0, policy_version 1553713 (0.0010) [2023-12-27 16:24:38,622][20347] Updated weights for policy 0, policy_version 1553723 (0.0008) [2023-12-27 16:24:38,675][20347] Updated weights for policy 0, policy_version 1553733 (0.0007) [2023-12-27 16:24:38,728][20347] Updated weights for policy 0, policy_version 1553743 (0.0006) [2023-12-27 16:24:39,187][20381] Updated weights for policy 1, policy_version 1552203 (0.0010) [2023-12-27 16:24:39,251][20381] Updated weights for policy 1, policy_version 1552213 (0.0009) [2023-12-27 16:24:39,304][20381] Updated weights for policy 1, policy_version 1552223 (0.0008) [2023-12-27 16:24:39,384][20347] Updated weights for policy 0, policy_version 1553753 (0.0008) [2023-12-27 16:24:39,452][20347] Updated weights for policy 0, policy_version 1553763 (0.0008) [2023-12-27 16:24:39,507][20347] Updated weights for policy 0, policy_version 1553773 (0.0008) [2023-12-27 16:24:40,001][20381] Updated weights for policy 1, policy_version 1552233 (0.0008) [2023-12-27 16:24:40,068][20381] Updated weights for policy 1, policy_version 1552243 (0.0009) [2023-12-27 16:24:40,127][20381] Updated weights for policy 1, policy_version 1552253 (0.0011) [2023-12-27 16:24:40,191][20381] Updated weights for policy 1, policy_version 1552263 (0.0011) [2023-12-27 16:24:40,222][20347] Updated weights for policy 0, policy_version 1553783 (0.0009) [2023-12-27 16:24:40,278][20347] Updated weights for policy 0, policy_version 1553793 (0.0011) [2023-12-27 16:24:40,338][20347] Updated weights for policy 0, policy_version 1553803 (0.0010) [2023-12-27 16:24:40,844][20381] Updated weights for policy 1, policy_version 1552273 (0.0010) [2023-12-27 16:24:40,899][20381] Updated weights for policy 1, policy_version 1552283 (0.0010) [2023-12-27 16:24:40,958][20381] Updated weights for policy 1, policy_version 1552293 (0.0011) [2023-12-27 16:24:41,021][20347] Updated weights for policy 0, policy_version 1553813 (0.0010) [2023-12-27 16:24:41,088][20347] Updated weights for policy 0, policy_version 1553823 (0.0009) [2023-12-27 16:24:41,152][20347] Updated weights for policy 0, policy_version 1553833 (0.0009) [2023-12-27 16:24:41,720][20381] Updated weights for policy 1, policy_version 1552303 (0.0008) [2023-12-27 16:24:41,787][20381] Updated weights for policy 1, policy_version 1552313 (0.0008) [2023-12-27 16:24:41,847][20381] Updated weights for policy 1, policy_version 1552323 (0.0007) [2023-12-27 16:24:41,960][20347] Updated weights for policy 0, policy_version 1553843 (0.0009) [2023-12-27 16:24:42,020][20347] Updated weights for policy 0, policy_version 1553853 (0.0011) [2023-12-27 16:24:42,068][20347] Updated weights for policy 0, policy_version 1553863 (0.0007) [2023-12-27 16:24:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 795303936. Throughput: 0: 10044.6, 1: 10444.0. Samples: 795295104. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:42,363][19170] Avg episode reward: [(0, '365.830'), (1, '700.350')] [2023-12-27 16:24:42,501][20381] Updated weights for policy 1, policy_version 1552333 (0.0007) [2023-12-27 16:24:42,566][20381] Updated weights for policy 1, policy_version 1552343 (0.0007) [2023-12-27 16:24:42,630][20381] Updated weights for policy 1, policy_version 1552353 (0.0008) [2023-12-27 16:24:42,700][20347] Updated weights for policy 0, policy_version 1553873 (0.0006) [2023-12-27 16:24:42,760][20347] Updated weights for policy 0, policy_version 1553883 (0.0007) [2023-12-27 16:24:42,816][20347] Updated weights for policy 0, policy_version 1553893 (0.0010) [2023-12-27 16:24:42,878][20347] Updated weights for policy 0, policy_version 1553903 (0.0010) [2023-12-27 16:24:43,283][20381] Updated weights for policy 1, policy_version 1552363 (0.0008) [2023-12-27 16:24:43,334][20381] Updated weights for policy 1, policy_version 1552373 (0.0007) [2023-12-27 16:24:43,387][20381] Updated weights for policy 1, policy_version 1552383 (0.0007) [2023-12-27 16:24:43,515][20347] Updated weights for policy 0, policy_version 1553913 (0.0007) [2023-12-27 16:24:43,572][20347] Updated weights for policy 0, policy_version 1553923 (0.0009) [2023-12-27 16:24:43,629][20347] Updated weights for policy 0, policy_version 1553933 (0.0006) [2023-12-27 16:24:44,090][20381] Updated weights for policy 1, policy_version 1552393 (0.0007) [2023-12-27 16:24:44,134][20381] Updated weights for policy 1, policy_version 1552403 (0.0007) [2023-12-27 16:24:44,182][20347] Updated weights for policy 0, policy_version 1553943 (0.0008) [2023-12-27 16:24:44,187][20381] Updated weights for policy 1, policy_version 1552413 (0.0006) [2023-12-27 16:24:44,237][20347] Updated weights for policy 0, policy_version 1553953 (0.0006) [2023-12-27 16:24:44,239][20381] Updated weights for policy 1, policy_version 1552423 (0.0007) [2023-12-27 16:24:44,300][20347] Updated weights for policy 0, policy_version 1553963 (0.0008) [2023-12-27 16:24:44,939][20347] Updated weights for policy 0, policy_version 1553973 (0.0009) [2023-12-27 16:24:44,963][20381] Updated weights for policy 1, policy_version 1552433 (0.0006) [2023-12-27 16:24:44,992][20347] Updated weights for policy 0, policy_version 1553983 (0.0008) [2023-12-27 16:24:45,019][20381] Updated weights for policy 1, policy_version 1552443 (0.0006) [2023-12-27 16:24:45,041][20347] Updated weights for policy 0, policy_version 1553993 (0.0008) [2023-12-27 16:24:45,069][20381] Updated weights for policy 1, policy_version 1552453 (0.0007) [2023-12-27 16:24:45,729][20381] Updated weights for policy 1, policy_version 1552463 (0.0008) [2023-12-27 16:24:45,735][20347] Updated weights for policy 0, policy_version 1554003 (0.0007) [2023-12-27 16:24:45,786][20347] Updated weights for policy 0, policy_version 1554013 (0.0005) [2023-12-27 16:24:45,793][20381] Updated weights for policy 1, policy_version 1552473 (0.0009) [2023-12-27 16:24:45,841][20347] Updated weights for policy 0, policy_version 1554023 (0.0005) [2023-12-27 16:24:45,851][20381] Updated weights for policy 1, policy_version 1552483 (0.0008) [2023-12-27 16:24:46,480][20347] Updated weights for policy 0, policy_version 1554033 (0.0006) [2023-12-27 16:24:46,506][20381] Updated weights for policy 1, policy_version 1552493 (0.0009) [2023-12-27 16:24:46,530][20347] Updated weights for policy 0, policy_version 1554043 (0.0005) [2023-12-27 16:24:46,558][20381] Updated weights for policy 1, policy_version 1552503 (0.0010) [2023-12-27 16:24:46,578][20347] Updated weights for policy 0, policy_version 1554053 (0.0005) [2023-12-27 16:24:46,613][20381] Updated weights for policy 1, policy_version 1552513 (0.0010) [2023-12-27 16:24:46,626][20347] Updated weights for policy 0, policy_version 1554063 (0.0005) [2023-12-27 16:24:47,294][20347] Updated weights for policy 0, policy_version 1554073 (0.0007) [2023-12-27 16:24:47,297][20381] Updated weights for policy 1, policy_version 1552523 (0.0011) [2023-12-27 16:24:47,349][20347] Updated weights for policy 0, policy_version 1554083 (0.0006) [2023-12-27 16:24:47,354][20381] Updated weights for policy 1, policy_version 1552533 (0.0009) [2023-12-27 16:24:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 795402240. Throughput: 0: 10106.2, 1: 10429.5. Samples: 795419296. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:47,363][19170] Avg episode reward: [(0, '387.030'), (1, '719.380')] [2023-12-27 16:24:47,400][20381] Updated weights for policy 1, policy_version 1552543 (0.0011) [2023-12-27 16:24:47,403][20347] Updated weights for policy 0, policy_version 1554093 (0.0006) [2023-12-27 16:24:47,995][20381] Updated weights for policy 1, policy_version 1552553 (0.0009) [2023-12-27 16:24:48,046][20381] Updated weights for policy 1, policy_version 1552563 (0.0008) [2023-12-27 16:24:48,097][20381] Updated weights for policy 1, policy_version 1552573 (0.0007) [2023-12-27 16:24:48,117][20347] Updated weights for policy 0, policy_version 1554103 (0.0008) [2023-12-27 16:24:48,142][20381] Updated weights for policy 1, policy_version 1552583 (0.0007) [2023-12-27 16:24:48,166][20347] Updated weights for policy 0, policy_version 1554113 (0.0007) [2023-12-27 16:24:48,231][20347] Updated weights for policy 0, policy_version 1554123 (0.0008) [2023-12-27 16:24:48,871][20381] Updated weights for policy 1, policy_version 1552593 (0.0007) [2023-12-27 16:24:48,921][20381] Updated weights for policy 1, policy_version 1552603 (0.0006) [2023-12-27 16:24:48,927][20347] Updated weights for policy 0, policy_version 1554133 (0.0007) [2023-12-27 16:24:48,979][20381] Updated weights for policy 1, policy_version 1552613 (0.0005) [2023-12-27 16:24:48,984][20347] Updated weights for policy 0, policy_version 1554143 (0.0007) [2023-12-27 16:24:49,039][20347] Updated weights for policy 0, policy_version 1554153 (0.0007) [2023-12-27 16:24:49,617][20292] KL-divergence is very high: 113.3928 [2023-12-27 16:24:49,619][20381] Updated weights for policy 1, policy_version 1552623 (0.0007) [2023-12-27 16:24:49,667][20381] Updated weights for policy 1, policy_version 1552633 (0.0008) [2023-12-27 16:24:49,710][20347] Updated weights for policy 0, policy_version 1554163 (0.0007) [2023-12-27 16:24:49,727][20381] Updated weights for policy 1, policy_version 1552643 (0.0006) [2023-12-27 16:24:49,762][20347] Updated weights for policy 0, policy_version 1554173 (0.0006) [2023-12-27 16:24:49,809][20347] Updated weights for policy 0, policy_version 1554183 (0.0007) [2023-12-27 16:24:50,458][20347] Updated weights for policy 0, policy_version 1554193 (0.0008) [2023-12-27 16:24:50,480][20381] Updated weights for policy 1, policy_version 1552653 (0.0008) [2023-12-27 16:24:50,516][20347] Updated weights for policy 0, policy_version 1554203 (0.0005) [2023-12-27 16:24:50,534][20381] Updated weights for policy 1, policy_version 1552663 (0.0008) [2023-12-27 16:24:50,574][20347] Updated weights for policy 0, policy_version 1554213 (0.0006) [2023-12-27 16:24:50,601][20381] Updated weights for policy 1, policy_version 1552673 (0.0008) [2023-12-27 16:24:50,639][20347] Updated weights for policy 0, policy_version 1554223 (0.0006) [2023-12-27 16:24:51,301][20381] Updated weights for policy 1, policy_version 1552683 (0.0007) [2023-12-27 16:24:51,381][20381] Updated weights for policy 1, policy_version 1552693 (0.0008) [2023-12-27 16:24:51,435][20347] Updated weights for policy 0, policy_version 1554233 (0.0009) [2023-12-27 16:24:51,442][20381] Updated weights for policy 1, policy_version 1552703 (0.0006) [2023-12-27 16:24:51,496][20347] Updated weights for policy 0, policy_version 1554243 (0.0008) [2023-12-27 16:24:51,561][20347] Updated weights for policy 0, policy_version 1554253 (0.0007) [2023-12-27 16:24:52,153][20381] Updated weights for policy 1, policy_version 1552713 (0.0008) [2023-12-27 16:24:52,204][20381] Updated weights for policy 1, policy_version 1552723 (0.0007) [2023-12-27 16:24:52,261][20381] Updated weights for policy 1, policy_version 1552733 (0.0008) [2023-12-27 16:24:52,301][20347] Updated weights for policy 0, policy_version 1554263 (0.0007) [2023-12-27 16:24:52,324][20381] Updated weights for policy 1, policy_version 1552743 (0.0008) [2023-12-27 16:24:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 795508736. Throughput: 0: 10069.4, 1: 10382.4. Samples: 795480008. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:52,363][19170] Avg episode reward: [(0, '454.670'), (1, '680.300')] [2023-12-27 16:24:52,369][20347] Updated weights for policy 0, policy_version 1554273 (0.0011) [2023-12-27 16:24:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001552744_397541376.pth... [2023-12-27 16:24:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001551528_397230080.pth [2023-12-27 16:24:52,425][20347] Updated weights for policy 0, policy_version 1554283 (0.0009) [2023-12-27 16:24:52,445][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001554288_397975552.pth... [2023-12-27 16:24:52,448][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001553104_397672448.pth [2023-12-27 16:24:53,010][20381] Updated weights for policy 1, policy_version 1552753 (0.0008) [2023-12-27 16:24:53,073][20381] Updated weights for policy 1, policy_version 1552763 (0.0008) [2023-12-27 16:24:53,089][20347] Updated weights for policy 0, policy_version 1554293 (0.0010) [2023-12-27 16:24:53,129][20381] Updated weights for policy 1, policy_version 1552773 (0.0007) [2023-12-27 16:24:53,138][20347] Updated weights for policy 0, policy_version 1554303 (0.0010) [2023-12-27 16:24:53,186][20347] Updated weights for policy 0, policy_version 1554313 (0.0010) [2023-12-27 16:24:53,815][20347] Updated weights for policy 0, policy_version 1554323 (0.0010) [2023-12-27 16:24:53,826][20381] Updated weights for policy 1, policy_version 1552783 (0.0007) [2023-12-27 16:24:53,875][20347] Updated weights for policy 0, policy_version 1554333 (0.0008) [2023-12-27 16:24:53,889][20381] Updated weights for policy 1, policy_version 1552793 (0.0007) [2023-12-27 16:24:53,935][20347] Updated weights for policy 0, policy_version 1554343 (0.0011) [2023-12-27 16:24:53,954][20381] Updated weights for policy 1, policy_version 1552803 (0.0006) [2023-12-27 16:24:54,588][20347] Updated weights for policy 0, policy_version 1554353 (0.0010) [2023-12-27 16:24:54,647][20347] Updated weights for policy 0, policy_version 1554363 (0.0005) [2023-12-27 16:24:54,666][20381] Updated weights for policy 1, policy_version 1552813 (0.0006) [2023-12-27 16:24:54,696][20347] Updated weights for policy 0, policy_version 1554373 (0.0005) [2023-12-27 16:24:54,720][20381] Updated weights for policy 1, policy_version 1552823 (0.0006) [2023-12-27 16:24:54,752][20347] Updated weights for policy 0, policy_version 1554383 (0.0006) [2023-12-27 16:24:54,783][20381] Updated weights for policy 1, policy_version 1552833 (0.0005) [2023-12-27 16:24:55,361][20381] Updated weights for policy 1, policy_version 1552843 (0.0006) [2023-12-27 16:24:55,415][20381] Updated weights for policy 1, policy_version 1552853 (0.0006) [2023-12-27 16:24:55,447][20347] Updated weights for policy 0, policy_version 1554393 (0.0008) [2023-12-27 16:24:55,467][20381] Updated weights for policy 1, policy_version 1552863 (0.0007) [2023-12-27 16:24:55,492][20347] Updated weights for policy 0, policy_version 1554403 (0.0010) [2023-12-27 16:24:55,537][20347] Updated weights for policy 0, policy_version 1554413 (0.0010) [2023-12-27 16:24:56,088][20381] Updated weights for policy 1, policy_version 1552873 (0.0006) [2023-12-27 16:24:56,147][20381] Updated weights for policy 1, policy_version 1552883 (0.0008) [2023-12-27 16:24:56,205][20381] Updated weights for policy 1, policy_version 1552893 (0.0007) [2023-12-27 16:24:56,232][20347] Updated weights for policy 0, policy_version 1554423 (0.0006) [2023-12-27 16:24:56,261][20381] Updated weights for policy 1, policy_version 1552903 (0.0007) [2023-12-27 16:24:56,283][20347] Updated weights for policy 0, policy_version 1554433 (0.0006) [2023-12-27 16:24:56,341][20347] Updated weights for policy 0, policy_version 1554443 (0.0008) [2023-12-27 16:24:56,870][20381] Updated weights for policy 1, policy_version 1552913 (0.0009) [2023-12-27 16:24:56,919][20381] Updated weights for policy 1, policy_version 1552923 (0.0009) [2023-12-27 16:24:56,966][20381] Updated weights for policy 1, policy_version 1552933 (0.0009) [2023-12-27 16:24:56,973][20347] Updated weights for policy 0, policy_version 1554453 (0.0010) [2023-12-27 16:24:57,028][20347] Updated weights for policy 0, policy_version 1554463 (0.0010) [2023-12-27 16:24:57,075][20347] Updated weights for policy 0, policy_version 1554473 (0.0010) [2023-12-27 16:24:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 795615232. Throughput: 0: 10117.3, 1: 10322.9. Samples: 795601956. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:24:57,363][19170] Avg episode reward: [(0, '397.730'), (1, '689.320')] [2023-12-27 16:24:57,593][20381] Updated weights for policy 1, policy_version 1552943 (0.0006) [2023-12-27 16:24:57,641][20381] Updated weights for policy 1, policy_version 1552953 (0.0006) [2023-12-27 16:24:57,701][20381] Updated weights for policy 1, policy_version 1552963 (0.0006) [2023-12-27 16:24:57,777][20347] Updated weights for policy 0, policy_version 1554483 (0.0010) [2023-12-27 16:24:57,831][20347] Updated weights for policy 0, policy_version 1554493 (0.0010) [2023-12-27 16:24:57,884][20347] Updated weights for policy 0, policy_version 1554503 (0.0010) [2023-12-27 16:24:58,286][20381] Updated weights for policy 1, policy_version 1552973 (0.0008) [2023-12-27 16:24:58,352][20381] Updated weights for policy 1, policy_version 1552983 (0.0010) [2023-12-27 16:24:58,409][20381] Updated weights for policy 1, policy_version 1552993 (0.0008) [2023-12-27 16:24:58,567][20347] Updated weights for policy 0, policy_version 1554513 (0.0010) [2023-12-27 16:24:58,633][20347] Updated weights for policy 0, policy_version 1554523 (0.0009) [2023-12-27 16:24:58,700][20347] Updated weights for policy 0, policy_version 1554533 (0.0009) [2023-12-27 16:24:58,768][20347] Updated weights for policy 0, policy_version 1554543 (0.0008) [2023-12-27 16:24:59,245][20381] Updated weights for policy 1, policy_version 1553003 (0.0009) [2023-12-27 16:24:59,304][20381] Updated weights for policy 1, policy_version 1553013 (0.0008) [2023-12-27 16:24:59,369][20381] Updated weights for policy 1, policy_version 1553023 (0.0011) [2023-12-27 16:24:59,530][20347] Updated weights for policy 0, policy_version 1554553 (0.0010) [2023-12-27 16:24:59,575][20347] Updated weights for policy 0, policy_version 1554563 (0.0010) [2023-12-27 16:24:59,627][20347] Updated weights for policy 0, policy_version 1554573 (0.0010) [2023-12-27 16:25:00,114][20381] Updated weights for policy 1, policy_version 1553033 (0.0011) [2023-12-27 16:25:00,176][20381] Updated weights for policy 1, policy_version 1553043 (0.0008) [2023-12-27 16:25:00,231][20381] Updated weights for policy 1, policy_version 1553053 (0.0007) [2023-12-27 16:25:00,288][20381] Updated weights for policy 1, policy_version 1553063 (0.0007) [2023-12-27 16:25:00,340][20347] Updated weights for policy 0, policy_version 1554583 (0.0007) [2023-12-27 16:25:00,397][20347] Updated weights for policy 0, policy_version 1554593 (0.0005) [2023-12-27 16:25:00,446][20347] Updated weights for policy 0, policy_version 1554603 (0.0006) [2023-12-27 16:25:01,000][20347] Updated weights for policy 0, policy_version 1554613 (0.0008) [2023-12-27 16:25:01,015][20381] Updated weights for policy 1, policy_version 1553073 (0.0006) [2023-12-27 16:25:01,065][20347] Updated weights for policy 0, policy_version 1554623 (0.0008) [2023-12-27 16:25:01,078][20381] Updated weights for policy 1, policy_version 1553083 (0.0008) [2023-12-27 16:25:01,119][20347] Updated weights for policy 0, policy_version 1554633 (0.0010) [2023-12-27 16:25:01,136][20381] Updated weights for policy 1, policy_version 1553093 (0.0006) [2023-12-27 16:25:01,836][20381] Updated weights for policy 1, policy_version 1553103 (0.0007) [2023-12-27 16:25:01,851][20347] Updated weights for policy 0, policy_version 1554643 (0.0009) [2023-12-27 16:25:01,898][20381] Updated weights for policy 1, policy_version 1553113 (0.0006) [2023-12-27 16:25:01,907][20347] Updated weights for policy 0, policy_version 1554653 (0.0007) [2023-12-27 16:25:01,960][20381] Updated weights for policy 1, policy_version 1553123 (0.0006) [2023-12-27 16:25:01,967][20347] Updated weights for policy 0, policy_version 1554663 (0.0006) [2023-12-27 16:25:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 795713536. Throughput: 0: 10076.4, 1: 10332.6. Samples: 795723616. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:02,363][19170] Avg episode reward: [(0, '466.410'), (1, '728.560')] [2023-12-27 16:25:02,583][20381] Updated weights for policy 1, policy_version 1553133 (0.0005) [2023-12-27 16:25:02,649][20381] Updated weights for policy 1, policy_version 1553143 (0.0006) [2023-12-27 16:25:02,667][20347] Updated weights for policy 0, policy_version 1554673 (0.0008) [2023-12-27 16:25:02,706][20381] Updated weights for policy 1, policy_version 1553153 (0.0008) [2023-12-27 16:25:02,723][20347] Updated weights for policy 0, policy_version 1554683 (0.0006) [2023-12-27 16:25:02,781][20347] Updated weights for policy 0, policy_version 1554693 (0.0007) [2023-12-27 16:25:02,836][20347] Updated weights for policy 0, policy_version 1554703 (0.0007) [2023-12-27 16:25:03,282][20381] Updated weights for policy 1, policy_version 1553163 (0.0007) [2023-12-27 16:25:03,325][20381] Updated weights for policy 1, policy_version 1553173 (0.0005) [2023-12-27 16:25:03,378][20381] Updated weights for policy 1, policy_version 1553183 (0.0005) [2023-12-27 16:25:03,573][20347] Updated weights for policy 0, policy_version 1554713 (0.0007) [2023-12-27 16:25:03,632][20347] Updated weights for policy 0, policy_version 1554723 (0.0008) [2023-12-27 16:25:03,690][20347] Updated weights for policy 0, policy_version 1554733 (0.0008) [2023-12-27 16:25:04,035][20381] Updated weights for policy 1, policy_version 1553193 (0.0005) [2023-12-27 16:25:04,097][20381] Updated weights for policy 1, policy_version 1553203 (0.0006) [2023-12-27 16:25:04,160][20381] Updated weights for policy 1, policy_version 1553213 (0.0008) [2023-12-27 16:25:04,224][20381] Updated weights for policy 1, policy_version 1553223 (0.0007) [2023-12-27 16:25:04,315][20347] Updated weights for policy 0, policy_version 1554743 (0.0009) [2023-12-27 16:25:04,373][20347] Updated weights for policy 0, policy_version 1554753 (0.0008) [2023-12-27 16:25:04,433][20347] Updated weights for policy 0, policy_version 1554763 (0.0008) [2023-12-27 16:25:04,913][20381] Updated weights for policy 1, policy_version 1553233 (0.0008) [2023-12-27 16:25:04,948][20292] KL-divergence is very high: 144.3115 [2023-12-27 16:25:04,976][20381] Updated weights for policy 1, policy_version 1553243 (0.0008) [2023-12-27 16:25:04,996][20292] KL-divergence is very high: 101.6734 [2023-12-27 16:25:05,002][20292] KL-divergence is very high: 135.4966 [2023-12-27 16:25:05,032][20381] Updated weights for policy 1, policy_version 1553253 (0.0009) [2023-12-27 16:25:05,083][20347] Updated weights for policy 0, policy_version 1554773 (0.0008) [2023-12-27 16:25:05,138][20347] Updated weights for policy 0, policy_version 1554783 (0.0007) [2023-12-27 16:25:05,194][20347] Updated weights for policy 0, policy_version 1554793 (0.0006) [2023-12-27 16:25:05,679][20381] Updated weights for policy 1, policy_version 1553263 (0.0007) [2023-12-27 16:25:05,727][20381] Updated weights for policy 1, policy_version 1553273 (0.0008) [2023-12-27 16:25:05,777][20381] Updated weights for policy 1, policy_version 1553283 (0.0007) [2023-12-27 16:25:05,785][20347] Updated weights for policy 0, policy_version 1554803 (0.0008) [2023-12-27 16:25:05,840][20347] Updated weights for policy 0, policy_version 1554813 (0.0007) [2023-12-27 16:25:05,900][20347] Updated weights for policy 0, policy_version 1554823 (0.0006) [2023-12-27 16:25:06,509][20381] Updated weights for policy 1, policy_version 1553293 (0.0007) [2023-12-27 16:25:06,572][20381] Updated weights for policy 1, policy_version 1553303 (0.0007) [2023-12-27 16:25:06,590][20347] Updated weights for policy 0, policy_version 1554833 (0.0007) [2023-12-27 16:25:06,634][20381] Updated weights for policy 1, policy_version 1553313 (0.0008) [2023-12-27 16:25:06,648][20347] Updated weights for policy 0, policy_version 1554843 (0.0006) [2023-12-27 16:25:06,703][20347] Updated weights for policy 0, policy_version 1554853 (0.0006) [2023-12-27 16:25:06,757][20347] Updated weights for policy 0, policy_version 1554863 (0.0007) [2023-12-27 16:25:07,305][20381] Updated weights for policy 1, policy_version 1553323 (0.0006) [2023-12-27 16:25:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20521.9). Total num frames: 795811840. Throughput: 0: 10091.7, 1: 10347.5. Samples: 795785696. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:07,363][19170] Avg episode reward: [(0, '549.000'), (1, '737.510')] [2023-12-27 16:25:07,364][20381] Updated weights for policy 1, policy_version 1553333 (0.0008) [2023-12-27 16:25:07,381][20347] Updated weights for policy 0, policy_version 1554873 (0.0005) [2023-12-27 16:25:07,423][20381] Updated weights for policy 1, policy_version 1553343 (0.0010) [2023-12-27 16:25:07,439][20347] Updated weights for policy 0, policy_version 1554883 (0.0005) [2023-12-27 16:25:07,470][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001553352_397697024.pth... [2023-12-27 16:25:07,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001552136_397385728.pth [2023-12-27 16:25:07,491][20347] Updated weights for policy 0, policy_version 1554893 (0.0010) [2023-12-27 16:25:07,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001554896_398131200.pth... [2023-12-27 16:25:07,508][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001553680_397819904.pth [2023-12-27 16:25:08,074][20347] Updated weights for policy 0, policy_version 1554903 (0.0007) [2023-12-27 16:25:08,118][20381] Updated weights for policy 1, policy_version 1553353 (0.0010) [2023-12-27 16:25:08,130][20347] Updated weights for policy 0, policy_version 1554913 (0.0005) [2023-12-27 16:25:08,174][20381] Updated weights for policy 1, policy_version 1553363 (0.0005) [2023-12-27 16:25:08,193][20347] Updated weights for policy 0, policy_version 1554923 (0.0006) [2023-12-27 16:25:08,234][20381] Updated weights for policy 1, policy_version 1553373 (0.0005) [2023-12-27 16:25:08,295][20381] Updated weights for policy 1, policy_version 1553383 (0.0007) [2023-12-27 16:25:08,817][20347] Updated weights for policy 0, policy_version 1554933 (0.0011) [2023-12-27 16:25:08,871][20347] Updated weights for policy 0, policy_version 1554943 (0.0008) [2023-12-27 16:25:08,931][20381] Updated weights for policy 1, policy_version 1553393 (0.0006) [2023-12-27 16:25:08,933][20347] Updated weights for policy 0, policy_version 1554953 (0.0011) [2023-12-27 16:25:08,998][20381] Updated weights for policy 1, policy_version 1553403 (0.0008) [2023-12-27 16:25:09,062][20381] Updated weights for policy 1, policy_version 1553413 (0.0009) [2023-12-27 16:25:09,585][20347] Updated weights for policy 0, policy_version 1554963 (0.0006) [2023-12-27 16:25:09,643][20347] Updated weights for policy 0, policy_version 1554973 (0.0007) [2023-12-27 16:25:09,696][20347] Updated weights for policy 0, policy_version 1554983 (0.0007) [2023-12-27 16:25:09,807][20381] Updated weights for policy 1, policy_version 1553423 (0.0008) [2023-12-27 16:25:09,874][20381] Updated weights for policy 1, policy_version 1553433 (0.0008) [2023-12-27 16:25:09,939][20381] Updated weights for policy 1, policy_version 1553443 (0.0008) [2023-12-27 16:25:10,354][20347] Updated weights for policy 0, policy_version 1554993 (0.0007) [2023-12-27 16:25:10,414][20347] Updated weights for policy 0, policy_version 1555003 (0.0008) [2023-12-27 16:25:10,473][20347] Updated weights for policy 0, policy_version 1555013 (0.0008) [2023-12-27 16:25:10,523][20347] Updated weights for policy 0, policy_version 1555023 (0.0008) [2023-12-27 16:25:10,659][20381] Updated weights for policy 1, policy_version 1553453 (0.0008) [2023-12-27 16:25:10,731][20381] Updated weights for policy 1, policy_version 1553463 (0.0008) [2023-12-27 16:25:10,780][20381] Updated weights for policy 1, policy_version 1553473 (0.0008) [2023-12-27 16:25:11,307][20347] Updated weights for policy 0, policy_version 1555033 (0.0010) [2023-12-27 16:25:11,379][20347] Updated weights for policy 0, policy_version 1555043 (0.0010) [2023-12-27 16:25:11,438][20347] Updated weights for policy 0, policy_version 1555053 (0.0007) [2023-12-27 16:25:11,549][20381] Updated weights for policy 1, policy_version 1553483 (0.0007) [2023-12-27 16:25:11,617][20381] Updated weights for policy 1, policy_version 1553493 (0.0007) [2023-12-27 16:25:11,689][20381] Updated weights for policy 1, policy_version 1553503 (0.0008) [2023-12-27 16:25:12,115][20347] Updated weights for policy 0, policy_version 1555063 (0.0007) [2023-12-27 16:25:12,170][20347] Updated weights for policy 0, policy_version 1555073 (0.0006) [2023-12-27 16:25:12,225][20347] Updated weights for policy 0, policy_version 1555083 (0.0008) [2023-12-27 16:25:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 795918336. Throughput: 0: 10218.3, 1: 10245.0. Samples: 795909176. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:12,363][19170] Avg episode reward: [(0, '351.400'), (1, '757.140')] [2023-12-27 16:25:12,383][20381] Updated weights for policy 1, policy_version 1553513 (0.0008) [2023-12-27 16:25:12,442][20381] Updated weights for policy 1, policy_version 1553523 (0.0007) [2023-12-27 16:25:12,504][20381] Updated weights for policy 1, policy_version 1553533 (0.0006) [2023-12-27 16:25:12,557][20381] Updated weights for policy 1, policy_version 1553543 (0.0007) [2023-12-27 16:25:12,930][20347] Updated weights for policy 0, policy_version 1555093 (0.0007) [2023-12-27 16:25:12,988][20347] Updated weights for policy 0, policy_version 1555103 (0.0006) [2023-12-27 16:25:13,042][20347] Updated weights for policy 0, policy_version 1555113 (0.0007) [2023-12-27 16:25:13,266][20381] Updated weights for policy 1, policy_version 1553553 (0.0006) [2023-12-27 16:25:13,326][20381] Updated weights for policy 1, policy_version 1553563 (0.0006) [2023-12-27 16:25:13,379][20381] Updated weights for policy 1, policy_version 1553573 (0.0005) [2023-12-27 16:25:13,756][20347] Updated weights for policy 0, policy_version 1555123 (0.0007) [2023-12-27 16:25:13,802][20347] Updated weights for policy 0, policy_version 1555133 (0.0009) [2023-12-27 16:25:13,848][20347] Updated weights for policy 0, policy_version 1555143 (0.0008) [2023-12-27 16:25:13,895][20381] Updated weights for policy 1, policy_version 1553583 (0.0005) [2023-12-27 16:25:13,948][20381] Updated weights for policy 1, policy_version 1553593 (0.0005) [2023-12-27 16:25:14,003][20381] Updated weights for policy 1, policy_version 1553603 (0.0005) [2023-12-27 16:25:14,567][20381] Updated weights for policy 1, policy_version 1553613 (0.0005) [2023-12-27 16:25:14,603][20347] Updated weights for policy 0, policy_version 1555153 (0.0007) [2023-12-27 16:25:14,619][20381] Updated weights for policy 1, policy_version 1553623 (0.0008) [2023-12-27 16:25:14,646][20347] Updated weights for policy 0, policy_version 1555163 (0.0007) [2023-12-27 16:25:14,669][20381] Updated weights for policy 1, policy_version 1553633 (0.0005) [2023-12-27 16:25:14,697][20347] Updated weights for policy 0, policy_version 1555173 (0.0007) [2023-12-27 16:25:14,746][20347] Updated weights for policy 0, policy_version 1555183 (0.0008) [2023-12-27 16:25:15,328][20381] Updated weights for policy 1, policy_version 1553643 (0.0008) [2023-12-27 16:25:15,376][20381] Updated weights for policy 1, policy_version 1553653 (0.0005) [2023-12-27 16:25:15,422][20381] Updated weights for policy 1, policy_version 1553663 (0.0006) [2023-12-27 16:25:15,492][20347] Updated weights for policy 0, policy_version 1555193 (0.0005) [2023-12-27 16:25:15,541][20347] Updated weights for policy 0, policy_version 1555203 (0.0005) [2023-12-27 16:25:15,591][20347] Updated weights for policy 0, policy_version 1555213 (0.0005) [2023-12-27 16:25:16,089][20381] Updated weights for policy 1, policy_version 1553673 (0.0006) [2023-12-27 16:25:16,148][20347] Updated weights for policy 0, policy_version 1555223 (0.0007) [2023-12-27 16:25:16,160][20381] Updated weights for policy 1, policy_version 1553683 (0.0010) [2023-12-27 16:25:16,192][20347] Updated weights for policy 0, policy_version 1555233 (0.0006) [2023-12-27 16:25:16,212][20381] Updated weights for policy 1, policy_version 1553693 (0.0007) [2023-12-27 16:25:16,237][20347] Updated weights for policy 0, policy_version 1555243 (0.0006) [2023-12-27 16:25:16,264][20381] Updated weights for policy 1, policy_version 1553703 (0.0005) [2023-12-27 16:25:16,803][20347] Updated weights for policy 0, policy_version 1555253 (0.0006) [2023-12-27 16:25:16,815][20381] Updated weights for policy 1, policy_version 1553713 (0.0008) [2023-12-27 16:25:16,856][20347] Updated weights for policy 0, policy_version 1555263 (0.0006) [2023-12-27 16:25:16,871][20381] Updated weights for policy 1, policy_version 1553723 (0.0007) [2023-12-27 16:25:16,915][20347] Updated weights for policy 0, policy_version 1555273 (0.0006) [2023-12-27 16:25:16,923][20381] Updated weights for policy 1, policy_version 1553733 (0.0007) [2023-12-27 16:25:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 796024832. Throughput: 0: 10273.5, 1: 10241.9. Samples: 796035744. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:17,363][19170] Avg episode reward: [(0, '446.440'), (1, '749.740')] [2023-12-27 16:25:17,499][20347] Updated weights for policy 0, policy_version 1555283 (0.0006) [2023-12-27 16:25:17,563][20347] Updated weights for policy 0, policy_version 1555293 (0.0010) [2023-12-27 16:25:17,582][20381] Updated weights for policy 1, policy_version 1553743 (0.0010) [2023-12-27 16:25:17,611][20347] Updated weights for policy 0, policy_version 1555303 (0.0010) [2023-12-27 16:25:17,640][20381] Updated weights for policy 1, policy_version 1553753 (0.0010) [2023-12-27 16:25:17,691][20381] Updated weights for policy 1, policy_version 1553763 (0.0010) [2023-12-27 16:25:18,282][20347] Updated weights for policy 0, policy_version 1555313 (0.0010) [2023-12-27 16:25:18,345][20347] Updated weights for policy 0, policy_version 1555323 (0.0010) [2023-12-27 16:25:18,375][20381] Updated weights for policy 1, policy_version 1553773 (0.0010) [2023-12-27 16:25:18,405][20347] Updated weights for policy 0, policy_version 1555333 (0.0009) [2023-12-27 16:25:18,430][20381] Updated weights for policy 1, policy_version 1553783 (0.0011) [2023-12-27 16:25:18,456][20347] Updated weights for policy 0, policy_version 1555343 (0.0010) [2023-12-27 16:25:18,486][20381] Updated weights for policy 1, policy_version 1553793 (0.0010) [2023-12-27 16:25:19,155][20347] Updated weights for policy 0, policy_version 1555353 (0.0010) [2023-12-27 16:25:19,168][20381] Updated weights for policy 1, policy_version 1553803 (0.0011) [2023-12-27 16:25:19,209][20347] Updated weights for policy 0, policy_version 1555363 (0.0010) [2023-12-27 16:25:19,232][20381] Updated weights for policy 1, policy_version 1553814 (0.0009) [2023-12-27 16:25:19,272][20347] Updated weights for policy 0, policy_version 1555373 (0.0011) [2023-12-27 16:25:19,286][20381] Updated weights for policy 1, policy_version 1553824 (0.0008) [2023-12-27 16:25:20,006][20347] Updated weights for policy 0, policy_version 1555383 (0.0008) [2023-12-27 16:25:20,050][20381] Updated weights for policy 1, policy_version 1553834 (0.0008) [2023-12-27 16:25:20,067][20347] Updated weights for policy 0, policy_version 1555393 (0.0007) [2023-12-27 16:25:20,112][20381] Updated weights for policy 1, policy_version 1553844 (0.0006) [2023-12-27 16:25:20,122][20347] Updated weights for policy 0, policy_version 1555403 (0.0007) [2023-12-27 16:25:20,170][20381] Updated weights for policy 1, policy_version 1553854 (0.0008) [2023-12-27 16:25:20,232][20381] Updated weights for policy 1, policy_version 1553864 (0.0006) [2023-12-27 16:25:20,866][20381] Updated weights for policy 1, policy_version 1553874 (0.0007) [2023-12-27 16:25:20,907][20347] Updated weights for policy 0, policy_version 1555413 (0.0006) [2023-12-27 16:25:20,924][20381] Updated weights for policy 1, policy_version 1553884 (0.0007) [2023-12-27 16:25:20,969][20347] Updated weights for policy 0, policy_version 1555423 (0.0006) [2023-12-27 16:25:20,985][20381] Updated weights for policy 1, policy_version 1553894 (0.0006) [2023-12-27 16:25:21,030][20347] Updated weights for policy 0, policy_version 1555433 (0.0007) [2023-12-27 16:25:21,732][20381] Updated weights for policy 1, policy_version 1553904 (0.0008) [2023-12-27 16:25:21,780][20347] Updated weights for policy 0, policy_version 1555443 (0.0008) [2023-12-27 16:25:21,791][20381] Updated weights for policy 1, policy_version 1553914 (0.0007) [2023-12-27 16:25:21,839][20347] Updated weights for policy 0, policy_version 1555453 (0.0008) [2023-12-27 16:25:21,848][20381] Updated weights for policy 1, policy_version 1553924 (0.0007) [2023-12-27 16:25:21,897][20347] Updated weights for policy 0, policy_version 1555463 (0.0006) [2023-12-27 16:25:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 796123136. Throughput: 0: 10292.7, 1: 10200.7. Samples: 796096148. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:22,363][19170] Avg episode reward: [(0, '400.160'), (1, '753.520')] [2023-12-27 16:25:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001553928_397844480.pth... [2023-12-27 16:25:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001555472_398278656.pth... [2023-12-27 16:25:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001552744_397541376.pth [2023-12-27 16:25:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001554288_397975552.pth [2023-12-27 16:25:22,588][20381] Updated weights for policy 1, policy_version 1553934 (0.0007) [2023-12-27 16:25:22,600][20347] Updated weights for policy 0, policy_version 1555473 (0.0006) [2023-12-27 16:25:22,651][20381] Updated weights for policy 1, policy_version 1553944 (0.0007) [2023-12-27 16:25:22,657][20347] Updated weights for policy 0, policy_version 1555483 (0.0006) [2023-12-27 16:25:22,714][20347] Updated weights for policy 0, policy_version 1555493 (0.0005) [2023-12-27 16:25:22,716][20381] Updated weights for policy 1, policy_version 1553954 (0.0007) [2023-12-27 16:25:22,771][20347] Updated weights for policy 0, policy_version 1555503 (0.0005) [2023-12-27 16:25:23,389][20347] Updated weights for policy 0, policy_version 1555513 (0.0005) [2023-12-27 16:25:23,435][20381] Updated weights for policy 1, policy_version 1553964 (0.0007) [2023-12-27 16:25:23,449][20347] Updated weights for policy 0, policy_version 1555523 (0.0005) [2023-12-27 16:25:23,491][20381] Updated weights for policy 1, policy_version 1553974 (0.0005) [2023-12-27 16:25:23,501][20347] Updated weights for policy 0, policy_version 1555533 (0.0006) [2023-12-27 16:25:23,541][20381] Updated weights for policy 1, policy_version 1553984 (0.0005) [2023-12-27 16:25:24,098][20347] Updated weights for policy 0, policy_version 1555543 (0.0007) [2023-12-27 16:25:24,150][20381] Updated weights for policy 1, policy_version 1553994 (0.0007) [2023-12-27 16:25:24,157][20347] Updated weights for policy 0, policy_version 1555553 (0.0010) [2023-12-27 16:25:24,204][20381] Updated weights for policy 1, policy_version 1554004 (0.0007) [2023-12-27 16:25:24,206][20347] Updated weights for policy 0, policy_version 1555563 (0.0006) [2023-12-27 16:25:24,261][20381] Updated weights for policy 1, policy_version 1554014 (0.0007) [2023-12-27 16:25:24,325][20381] Updated weights for policy 1, policy_version 1554024 (0.0007) [2023-12-27 16:25:24,817][20347] Updated weights for policy 0, policy_version 1555573 (0.0008) [2023-12-27 16:25:24,876][20347] Updated weights for policy 0, policy_version 1555583 (0.0010) [2023-12-27 16:25:24,928][20347] Updated weights for policy 0, policy_version 1555593 (0.0010) [2023-12-27 16:25:25,018][20381] Updated weights for policy 1, policy_version 1554034 (0.0008) [2023-12-27 16:25:25,077][20381] Updated weights for policy 1, policy_version 1554044 (0.0007) [2023-12-27 16:25:25,133][20381] Updated weights for policy 1, policy_version 1554054 (0.0007) [2023-12-27 16:25:25,571][20347] Updated weights for policy 0, policy_version 1555603 (0.0009) [2023-12-27 16:25:25,619][20347] Updated weights for policy 0, policy_version 1555613 (0.0008) [2023-12-27 16:25:25,677][20347] Updated weights for policy 0, policy_version 1555623 (0.0008) [2023-12-27 16:25:25,710][20381] Updated weights for policy 1, policy_version 1554064 (0.0006) [2023-12-27 16:25:25,768][20381] Updated weights for policy 1, policy_version 1554074 (0.0007) [2023-12-27 16:25:25,817][20381] Updated weights for policy 1, policy_version 1554084 (0.0008) [2023-12-27 16:25:26,378][20347] Updated weights for policy 0, policy_version 1555633 (0.0006) [2023-12-27 16:25:26,425][20347] Updated weights for policy 0, policy_version 1555643 (0.0005) [2023-12-27 16:25:26,475][20347] Updated weights for policy 0, policy_version 1555653 (0.0006) [2023-12-27 16:25:26,531][20347] Updated weights for policy 0, policy_version 1555663 (0.0006) [2023-12-27 16:25:26,543][20381] Updated weights for policy 1, policy_version 1554094 (0.0008) [2023-12-27 16:25:26,600][20381] Updated weights for policy 1, policy_version 1554104 (0.0006) [2023-12-27 16:25:26,650][20381] Updated weights for policy 1, policy_version 1554114 (0.0008) [2023-12-27 16:25:27,201][20347] Updated weights for policy 0, policy_version 1555673 (0.0007) [2023-12-27 16:25:27,258][20347] Updated weights for policy 0, policy_version 1555683 (0.0007) [2023-12-27 16:25:27,290][20381] Updated weights for policy 1, policy_version 1554124 (0.0005) [2023-12-27 16:25:27,311][20347] Updated weights for policy 0, policy_version 1555693 (0.0006) [2023-12-27 16:25:27,345][20381] Updated weights for policy 1, policy_version 1554134 (0.0005) [2023-12-27 16:25:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 796229632. Throughput: 0: 10299.5, 1: 10229.3. Samples: 796218896. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:27,363][19170] Avg episode reward: [(0, '371.440'), (1, '712.820')] [2023-12-27 16:25:27,393][20381] Updated weights for policy 1, policy_version 1554144 (0.0007) [2023-12-27 16:25:27,906][20347] Updated weights for policy 0, policy_version 1555703 (0.0006) [2023-12-27 16:25:27,965][20347] Updated weights for policy 0, policy_version 1555713 (0.0006) [2023-12-27 16:25:28,020][20347] Updated weights for policy 0, policy_version 1555723 (0.0006) [2023-12-27 16:25:28,070][20381] Updated weights for policy 1, policy_version 1554154 (0.0008) [2023-12-27 16:25:28,122][20381] Updated weights for policy 1, policy_version 1554164 (0.0010) [2023-12-27 16:25:28,169][20381] Updated weights for policy 1, policy_version 1554174 (0.0010) [2023-12-27 16:25:28,217][20381] Updated weights for policy 1, policy_version 1554184 (0.0009) [2023-12-27 16:25:28,640][20347] Updated weights for policy 0, policy_version 1555733 (0.0005) [2023-12-27 16:25:28,699][20347] Updated weights for policy 0, policy_version 1555743 (0.0006) [2023-12-27 16:25:28,757][20347] Updated weights for policy 0, policy_version 1555753 (0.0008) [2023-12-27 16:25:28,914][20381] Updated weights for policy 1, policy_version 1554194 (0.0009) [2023-12-27 16:25:28,966][20381] Updated weights for policy 1, policy_version 1554204 (0.0007) [2023-12-27 16:25:29,010][20381] Updated weights for policy 1, policy_version 1554214 (0.0005) [2023-12-27 16:25:29,401][20347] Updated weights for policy 0, policy_version 1555763 (0.0006) [2023-12-27 16:25:29,452][20347] Updated weights for policy 0, policy_version 1555773 (0.0005) [2023-12-27 16:25:29,504][20347] Updated weights for policy 0, policy_version 1555783 (0.0005) [2023-12-27 16:25:29,791][20381] Updated weights for policy 1, policy_version 1554224 (0.0007) [2023-12-27 16:25:29,856][20381] Updated weights for policy 1, policy_version 1554234 (0.0007) [2023-12-27 16:25:29,921][20381] Updated weights for policy 1, policy_version 1554244 (0.0008) [2023-12-27 16:25:30,125][20347] Updated weights for policy 0, policy_version 1555793 (0.0006) [2023-12-27 16:25:30,173][20347] Updated weights for policy 0, policy_version 1555803 (0.0010) [2023-12-27 16:25:30,225][20347] Updated weights for policy 0, policy_version 1555813 (0.0010) [2023-12-27 16:25:30,284][20347] Updated weights for policy 0, policy_version 1555823 (0.0010) [2023-12-27 16:25:30,608][20381] Updated weights for policy 1, policy_version 1554254 (0.0007) [2023-12-27 16:25:30,609][20292] KL-divergence is very high: 104.9725 [2023-12-27 16:25:30,619][20292] KL-divergence is very high: 128.6274 [2023-12-27 16:25:30,629][20292] KL-divergence is very high: 146.2117 [2023-12-27 16:25:30,635][20292] KL-divergence is very high: 137.1160 [2023-12-27 16:25:30,649][20292] KL-divergence is very high: 142.6735 [2023-12-27 16:25:30,658][20292] KL-divergence is very high: 105.3866 [2023-12-27 16:25:30,658][20381] Updated weights for policy 1, policy_version 1554264 (0.0007) [2023-12-27 16:25:30,705][20381] Updated weights for policy 1, policy_version 1554274 (0.0006) [2023-12-27 16:25:30,909][20347] Updated weights for policy 0, policy_version 1555833 (0.0008) [2023-12-27 16:25:30,960][20347] Updated weights for policy 0, policy_version 1555843 (0.0010) [2023-12-27 16:25:31,016][20347] Updated weights for policy 0, policy_version 1555853 (0.0007) [2023-12-27 16:25:31,304][20292] KL-divergence is very high: 113.8928 [2023-12-27 16:25:31,323][20381] Updated weights for policy 1, policy_version 1554284 (0.0006) [2023-12-27 16:25:31,392][20381] Updated weights for policy 1, policy_version 1554294 (0.0007) [2023-12-27 16:25:31,457][20381] Updated weights for policy 1, policy_version 1554304 (0.0007) [2023-12-27 16:25:31,681][20347] Updated weights for policy 0, policy_version 1555863 (0.0008) [2023-12-27 16:25:31,738][20347] Updated weights for policy 0, policy_version 1555873 (0.0007) [2023-12-27 16:25:31,796][20347] Updated weights for policy 0, policy_version 1555883 (0.0008) [2023-12-27 16:25:32,184][20381] Updated weights for policy 1, policy_version 1554314 (0.0008) [2023-12-27 16:25:32,233][20381] Updated weights for policy 1, policy_version 1554324 (0.0007) [2023-12-27 16:25:32,302][20381] Updated weights for policy 1, policy_version 1554334 (0.0009) [2023-12-27 16:25:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 796327936. Throughput: 0: 10337.3, 1: 10236.8. Samples: 796345132. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:32,363][19170] Avg episode reward: [(0, '323.450'), (1, '701.100')] [2023-12-27 16:25:32,364][20381] Updated weights for policy 1, policy_version 1554344 (0.0008) [2023-12-27 16:25:32,491][20347] Updated weights for policy 0, policy_version 1555893 (0.0007) [2023-12-27 16:25:32,546][20347] Updated weights for policy 0, policy_version 1555903 (0.0008) [2023-12-27 16:25:32,603][20347] Updated weights for policy 0, policy_version 1555913 (0.0008) [2023-12-27 16:25:33,038][20381] Updated weights for policy 1, policy_version 1554354 (0.0008) [2023-12-27 16:25:33,084][20381] Updated weights for policy 1, policy_version 1554364 (0.0008) [2023-12-27 16:25:33,140][20381] Updated weights for policy 1, policy_version 1554374 (0.0007) [2023-12-27 16:25:33,269][20347] Updated weights for policy 0, policy_version 1555923 (0.0007) [2023-12-27 16:25:33,330][20347] Updated weights for policy 0, policy_version 1555933 (0.0007) [2023-12-27 16:25:33,382][20347] Updated weights for policy 0, policy_version 1555943 (0.0007) [2023-12-27 16:25:33,761][20381] Updated weights for policy 1, policy_version 1554384 (0.0008) [2023-12-27 16:25:33,814][20381] Updated weights for policy 1, policy_version 1554394 (0.0008) [2023-12-27 16:25:33,860][20381] Updated weights for policy 1, policy_version 1554404 (0.0010) [2023-12-27 16:25:33,968][20347] Updated weights for policy 0, policy_version 1555953 (0.0006) [2023-12-27 16:25:34,015][20347] Updated weights for policy 0, policy_version 1555963 (0.0007) [2023-12-27 16:25:34,063][20347] Updated weights for policy 0, policy_version 1555973 (0.0007) [2023-12-27 16:25:34,120][20347] Updated weights for policy 0, policy_version 1555983 (0.0008) [2023-12-27 16:25:34,580][20381] Updated weights for policy 1, policy_version 1554414 (0.0009) [2023-12-27 16:25:34,639][20381] Updated weights for policy 1, policy_version 1554424 (0.0007) [2023-12-27 16:25:34,698][20381] Updated weights for policy 1, policy_version 1554434 (0.0007) [2023-12-27 16:25:34,878][20347] Updated weights for policy 0, policy_version 1555993 (0.0008) [2023-12-27 16:25:34,929][20347] Updated weights for policy 0, policy_version 1556003 (0.0007) [2023-12-27 16:25:34,988][20347] Updated weights for policy 0, policy_version 1556013 (0.0008) [2023-12-27 16:25:35,387][20381] Updated weights for policy 1, policy_version 1554444 (0.0008) [2023-12-27 16:25:35,437][20381] Updated weights for policy 1, policy_version 1554454 (0.0007) [2023-12-27 16:25:35,488][20381] Updated weights for policy 1, policy_version 1554464 (0.0007) [2023-12-27 16:25:35,672][20347] Updated weights for policy 0, policy_version 1556023 (0.0009) [2023-12-27 16:25:35,729][20347] Updated weights for policy 0, policy_version 1556033 (0.0010) [2023-12-27 16:25:35,784][20347] Updated weights for policy 0, policy_version 1556043 (0.0010) [2023-12-27 16:25:36,149][20381] Updated weights for policy 1, policy_version 1554474 (0.0008) [2023-12-27 16:25:36,197][20381] Updated weights for policy 1, policy_version 1554484 (0.0008) [2023-12-27 16:25:36,259][20381] Updated weights for policy 1, policy_version 1554494 (0.0008) [2023-12-27 16:25:36,322][20381] Updated weights for policy 1, policy_version 1554504 (0.0008) [2023-12-27 16:25:36,461][20347] Updated weights for policy 0, policy_version 1556053 (0.0009) [2023-12-27 16:25:36,526][20347] Updated weights for policy 0, policy_version 1556063 (0.0007) [2023-12-27 16:25:36,587][20347] Updated weights for policy 0, policy_version 1556073 (0.0008) [2023-12-27 16:25:37,069][20381] Updated weights for policy 1, policy_version 1554514 (0.0008) [2023-12-27 16:25:37,121][20381] Updated weights for policy 1, policy_version 1554524 (0.0008) [2023-12-27 16:25:37,178][20381] Updated weights for policy 1, policy_version 1554534 (0.0008) [2023-12-27 16:25:37,234][20347] Updated weights for policy 0, policy_version 1556083 (0.0007) [2023-12-27 16:25:37,287][20347] Updated weights for policy 0, policy_version 1556093 (0.0008) [2023-12-27 16:25:37,346][20347] Updated weights for policy 0, policy_version 1556103 (0.0008) [2023-12-27 16:25:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 796434432. Throughput: 0: 10362.2, 1: 10230.5. Samples: 796406680. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:37,363][19170] Avg episode reward: [(0, '450.040'), (1, '714.870')] [2023-12-27 16:25:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001554536_398000128.pth... [2023-12-27 16:25:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001553352_397697024.pth [2023-12-27 16:25:37,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001556112_398442496.pth... [2023-12-27 16:25:37,405][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001554896_398131200.pth [2023-12-27 16:25:37,805][20381] Updated weights for policy 1, policy_version 1554544 (0.0007) [2023-12-27 16:25:37,872][20381] Updated weights for policy 1, policy_version 1554554 (0.0008) [2023-12-27 16:25:37,930][20381] Updated weights for policy 1, policy_version 1554564 (0.0010) [2023-12-27 16:25:38,021][20347] Updated weights for policy 0, policy_version 1556113 (0.0007) [2023-12-27 16:25:38,068][20347] Updated weights for policy 0, policy_version 1556123 (0.0010) [2023-12-27 16:25:38,119][20347] Updated weights for policy 0, policy_version 1556133 (0.0010) [2023-12-27 16:25:38,173][20347] Updated weights for policy 0, policy_version 1556143 (0.0010) [2023-12-27 16:25:38,572][20381] Updated weights for policy 1, policy_version 1554574 (0.0007) [2023-12-27 16:25:38,633][20381] Updated weights for policy 1, policy_version 1554584 (0.0005) [2023-12-27 16:25:38,692][20381] Updated weights for policy 1, policy_version 1554594 (0.0005) [2023-12-27 16:25:38,866][20347] Updated weights for policy 0, policy_version 1556153 (0.0006) [2023-12-27 16:25:38,927][20347] Updated weights for policy 0, policy_version 1556163 (0.0006) [2023-12-27 16:25:38,981][20347] Updated weights for policy 0, policy_version 1556173 (0.0007) [2023-12-27 16:25:39,325][20381] Updated weights for policy 1, policy_version 1554604 (0.0007) [2023-12-27 16:25:39,397][20381] Updated weights for policy 1, policy_version 1554614 (0.0008) [2023-12-27 16:25:39,460][20381] Updated weights for policy 1, policy_version 1554624 (0.0006) [2023-12-27 16:25:39,667][20347] Updated weights for policy 0, policy_version 1556183 (0.0008) [2023-12-27 16:25:39,748][20347] Updated weights for policy 0, policy_version 1556193 (0.0008) [2023-12-27 16:25:39,800][20347] Updated weights for policy 0, policy_version 1556203 (0.0007) [2023-12-27 16:25:40,165][20381] Updated weights for policy 1, policy_version 1554634 (0.0006) [2023-12-27 16:25:40,224][20381] Updated weights for policy 1, policy_version 1554644 (0.0007) [2023-12-27 16:25:40,279][20381] Updated weights for policy 1, policy_version 1554654 (0.0008) [2023-12-27 16:25:40,340][20381] Updated weights for policy 1, policy_version 1554664 (0.0007) [2023-12-27 16:25:40,507][20347] Updated weights for policy 0, policy_version 1556213 (0.0007) [2023-12-27 16:25:40,569][20347] Updated weights for policy 0, policy_version 1556223 (0.0006) [2023-12-27 16:25:40,627][20347] Updated weights for policy 0, policy_version 1556233 (0.0006) [2023-12-27 16:25:41,056][20381] Updated weights for policy 1, policy_version 1554674 (0.0008) [2023-12-27 16:25:41,121][20381] Updated weights for policy 1, policy_version 1554684 (0.0008) [2023-12-27 16:25:41,187][20381] Updated weights for policy 1, policy_version 1554694 (0.0008) [2023-12-27 16:25:41,247][20347] Updated weights for policy 0, policy_version 1556243 (0.0006) [2023-12-27 16:25:41,316][20347] Updated weights for policy 0, policy_version 1556253 (0.0006) [2023-12-27 16:25:41,382][20347] Updated weights for policy 0, policy_version 1556263 (0.0007) [2023-12-27 16:25:41,961][20381] Updated weights for policy 1, policy_version 1554704 (0.0007) [2023-12-27 16:25:42,012][20381] Updated weights for policy 1, policy_version 1554714 (0.0008) [2023-12-27 16:25:42,069][20381] Updated weights for policy 1, policy_version 1554724 (0.0008) [2023-12-27 16:25:42,121][20347] Updated weights for policy 0, policy_version 1556273 (0.0008) [2023-12-27 16:25:42,176][20347] Updated weights for policy 0, policy_version 1556283 (0.0005) [2023-12-27 16:25:42,225][20347] Updated weights for policy 0, policy_version 1556293 (0.0005) [2023-12-27 16:25:42,287][20347] Updated weights for policy 0, policy_version 1556303 (0.0007) [2023-12-27 16:25:42,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 796540928. Throughput: 0: 10368.8, 1: 10221.3. Samples: 796528512. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:42,363][19170] Avg episode reward: [(0, '664.000'), (1, '714.750')] [2023-12-27 16:25:42,808][20381] Updated weights for policy 1, policy_version 1554734 (0.0009) [2023-12-27 16:25:42,864][20381] Updated weights for policy 1, policy_version 1554744 (0.0011) [2023-12-27 16:25:42,926][20381] Updated weights for policy 1, policy_version 1554754 (0.0011) [2023-12-27 16:25:42,936][20347] Updated weights for policy 0, policy_version 1556313 (0.0005) [2023-12-27 16:25:42,984][20347] Updated weights for policy 0, policy_version 1556323 (0.0006) [2023-12-27 16:25:43,038][20347] Updated weights for policy 0, policy_version 1556333 (0.0007) [2023-12-27 16:25:43,589][20381] Updated weights for policy 1, policy_version 1554764 (0.0010) [2023-12-27 16:25:43,647][20381] Updated weights for policy 1, policy_version 1554774 (0.0010) [2023-12-27 16:25:43,700][20381] Updated weights for policy 1, policy_version 1554784 (0.0010) [2023-12-27 16:25:43,747][20347] Updated weights for policy 0, policy_version 1556343 (0.0006) [2023-12-27 16:25:43,802][20347] Updated weights for policy 0, policy_version 1556353 (0.0005) [2023-12-27 16:25:43,849][20347] Updated weights for policy 0, policy_version 1556363 (0.0007) [2023-12-27 16:25:44,382][20381] Updated weights for policy 1, policy_version 1554794 (0.0010) [2023-12-27 16:25:44,437][20381] Updated weights for policy 1, policy_version 1554804 (0.0010) [2023-12-27 16:25:44,454][20347] Updated weights for policy 0, policy_version 1556373 (0.0006) [2023-12-27 16:25:44,496][20381] Updated weights for policy 1, policy_version 1554814 (0.0010) [2023-12-27 16:25:44,513][20347] Updated weights for policy 0, policy_version 1556383 (0.0005) [2023-12-27 16:25:44,554][20381] Updated weights for policy 1, policy_version 1554824 (0.0010) [2023-12-27 16:25:44,572][20347] Updated weights for policy 0, policy_version 1556393 (0.0006) [2023-12-27 16:25:45,187][20381] Updated weights for policy 1, policy_version 1554834 (0.0011) [2023-12-27 16:25:45,239][20347] Updated weights for policy 0, policy_version 1556403 (0.0008) [2023-12-27 16:25:45,249][20381] Updated weights for policy 1, policy_version 1554844 (0.0011) [2023-12-27 16:25:45,296][20347] Updated weights for policy 0, policy_version 1556413 (0.0005) [2023-12-27 16:25:45,308][20381] Updated weights for policy 1, policy_version 1554854 (0.0010) [2023-12-27 16:25:45,353][20347] Updated weights for policy 0, policy_version 1556423 (0.0007) [2023-12-27 16:25:45,932][20381] Updated weights for policy 1, policy_version 1554864 (0.0011) [2023-12-27 16:25:45,977][20381] Updated weights for policy 1, policy_version 1554874 (0.0010) [2023-12-27 16:25:46,000][20347] Updated weights for policy 0, policy_version 1556433 (0.0005) [2023-12-27 16:25:46,022][20381] Updated weights for policy 1, policy_version 1554884 (0.0010) [2023-12-27 16:25:46,062][20347] Updated weights for policy 0, policy_version 1556443 (0.0007) [2023-12-27 16:25:46,109][20347] Updated weights for policy 0, policy_version 1556453 (0.0006) [2023-12-27 16:25:46,161][20347] Updated weights for policy 0, policy_version 1556463 (0.0008) [2023-12-27 16:25:46,678][20381] Updated weights for policy 1, policy_version 1554894 (0.0008) [2023-12-27 16:25:46,732][20381] Updated weights for policy 1, policy_version 1554904 (0.0010) [2023-12-27 16:25:46,764][20347] Updated weights for policy 0, policy_version 1556473 (0.0006) [2023-12-27 16:25:46,784][20381] Updated weights for policy 1, policy_version 1554914 (0.0010) [2023-12-27 16:25:46,816][20347] Updated weights for policy 0, policy_version 1556483 (0.0005) [2023-12-27 16:25:46,862][20347] Updated weights for policy 0, policy_version 1556493 (0.0007) [2023-12-27 16:25:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 796639232. Throughput: 0: 10420.0, 1: 10220.0. Samples: 796652416. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:47,364][19170] Avg episode reward: [(0, '756.100'), (1, '758.560')] [2023-12-27 16:25:47,487][20381] Updated weights for policy 1, policy_version 1554924 (0.0010) [2023-12-27 16:25:47,538][20381] Updated weights for policy 1, policy_version 1554934 (0.0008) [2023-12-27 16:25:47,559][20347] Updated weights for policy 0, policy_version 1556503 (0.0006) [2023-12-27 16:25:47,598][20381] Updated weights for policy 1, policy_version 1554944 (0.0006) [2023-12-27 16:25:47,605][20347] Updated weights for policy 0, policy_version 1556513 (0.0007) [2023-12-27 16:25:47,655][20347] Updated weights for policy 0, policy_version 1556523 (0.0008) [2023-12-27 16:25:48,287][20381] Updated weights for policy 1, policy_version 1554954 (0.0006) [2023-12-27 16:25:48,327][20347] Updated weights for policy 0, policy_version 1556533 (0.0007) [2023-12-27 16:25:48,344][20381] Updated weights for policy 1, policy_version 1554964 (0.0007) [2023-12-27 16:25:48,392][20347] Updated weights for policy 0, policy_version 1556543 (0.0007) [2023-12-27 16:25:48,405][20381] Updated weights for policy 1, policy_version 1554974 (0.0007) [2023-12-27 16:25:48,454][20347] Updated weights for policy 0, policy_version 1556553 (0.0006) [2023-12-27 16:25:48,460][20381] Updated weights for policy 1, policy_version 1554984 (0.0006) [2023-12-27 16:25:49,112][20381] Updated weights for policy 1, policy_version 1554994 (0.0007) [2023-12-27 16:25:49,140][20347] Updated weights for policy 0, policy_version 1556563 (0.0007) [2023-12-27 16:25:49,167][20381] Updated weights for policy 1, policy_version 1555004 (0.0006) [2023-12-27 16:25:49,195][20347] Updated weights for policy 0, policy_version 1556573 (0.0005) [2023-12-27 16:25:49,228][20381] Updated weights for policy 1, policy_version 1555014 (0.0007) [2023-12-27 16:25:49,265][20347] Updated weights for policy 0, policy_version 1556583 (0.0008) [2023-12-27 16:25:49,924][20347] Updated weights for policy 0, policy_version 1556593 (0.0008) [2023-12-27 16:25:49,986][20347] Updated weights for policy 0, policy_version 1556603 (0.0005) [2023-12-27 16:25:49,993][20381] Updated weights for policy 1, policy_version 1555024 (0.0008) [2023-12-27 16:25:50,047][20381] Updated weights for policy 1, policy_version 1555034 (0.0008) [2023-12-27 16:25:50,050][20347] Updated weights for policy 0, policy_version 1556613 (0.0007) [2023-12-27 16:25:50,104][20381] Updated weights for policy 1, policy_version 1555044 (0.0006) [2023-12-27 16:25:50,111][20347] Updated weights for policy 0, policy_version 1556623 (0.0006) [2023-12-27 16:25:50,739][20347] Updated weights for policy 0, policy_version 1556633 (0.0005) [2023-12-27 16:25:50,792][20347] Updated weights for policy 0, policy_version 1556643 (0.0007) [2023-12-27 16:25:50,841][20381] Updated weights for policy 1, policy_version 1555054 (0.0006) [2023-12-27 16:25:50,851][20347] Updated weights for policy 0, policy_version 1556653 (0.0006) [2023-12-27 16:25:50,898][20381] Updated weights for policy 1, policy_version 1555064 (0.0007) [2023-12-27 16:25:50,968][20381] Updated weights for policy 1, policy_version 1555074 (0.0008) [2023-12-27 16:25:51,559][20347] Updated weights for policy 0, policy_version 1556663 (0.0008) [2023-12-27 16:25:51,619][20347] Updated weights for policy 0, policy_version 1556673 (0.0008) [2023-12-27 16:25:51,682][20347] Updated weights for policy 0, policy_version 1556683 (0.0008) [2023-12-27 16:25:51,712][20381] Updated weights for policy 1, policy_version 1555084 (0.0008) [2023-12-27 16:25:51,778][20381] Updated weights for policy 1, policy_version 1555094 (0.0007) [2023-12-27 16:25:51,847][20381] Updated weights for policy 1, policy_version 1555104 (0.0008) [2023-12-27 16:25:52,363][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 796737536. Throughput: 0: 10417.5, 1: 10176.9. Samples: 796712444. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:52,363][19170] Avg episode reward: [(0, '747.660'), (1, '815.040')] [2023-12-27 16:25:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001556688_398589952.pth... [2023-12-27 16:25:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001555112_398147584.pth... [2023-12-27 16:25:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001555472_398278656.pth [2023-12-27 16:25:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001553928_397844480.pth [2023-12-27 16:25:52,414][20347] Updated weights for policy 0, policy_version 1556693 (0.0007) [2023-12-27 16:25:52,471][20347] Updated weights for policy 0, policy_version 1556703 (0.0006) [2023-12-27 16:25:52,527][20347] Updated weights for policy 0, policy_version 1556713 (0.0008) [2023-12-27 16:25:52,570][20381] Updated weights for policy 1, policy_version 1555114 (0.0008) [2023-12-27 16:25:52,634][20381] Updated weights for policy 1, policy_version 1555124 (0.0008) [2023-12-27 16:25:52,695][20381] Updated weights for policy 1, policy_version 1555134 (0.0008) [2023-12-27 16:25:52,752][20381] Updated weights for policy 1, policy_version 1555144 (0.0008) [2023-12-27 16:25:53,200][20347] Updated weights for policy 0, policy_version 1556723 (0.0008) [2023-12-27 16:25:53,251][20347] Updated weights for policy 0, policy_version 1556733 (0.0008) [2023-12-27 16:25:53,309][20347] Updated weights for policy 0, policy_version 1556743 (0.0008) [2023-12-27 16:25:53,410][20381] Updated weights for policy 1, policy_version 1555154 (0.0008) [2023-12-27 16:25:53,457][20381] Updated weights for policy 1, policy_version 1555164 (0.0010) [2023-12-27 16:25:53,512][20381] Updated weights for policy 1, policy_version 1555174 (0.0008) [2023-12-27 16:25:53,970][20347] Updated weights for policy 0, policy_version 1556753 (0.0007) [2023-12-27 16:25:54,026][20347] Updated weights for policy 0, policy_version 1556763 (0.0006) [2023-12-27 16:25:54,081][20347] Updated weights for policy 0, policy_version 1556773 (0.0007) [2023-12-27 16:25:54,126][20381] Updated weights for policy 1, policy_version 1555184 (0.0006) [2023-12-27 16:25:54,135][20347] Updated weights for policy 0, policy_version 1556783 (0.0006) [2023-12-27 16:25:54,173][20381] Updated weights for policy 1, policy_version 1555194 (0.0007) [2023-12-27 16:25:54,230][20381] Updated weights for policy 1, policy_version 1555204 (0.0008) [2023-12-27 16:25:54,851][20347] Updated weights for policy 0, policy_version 1556793 (0.0008) [2023-12-27 16:25:54,875][20381] Updated weights for policy 1, policy_version 1555214 (0.0007) [2023-12-27 16:25:54,908][20347] Updated weights for policy 0, policy_version 1556803 (0.0005) [2023-12-27 16:25:54,929][20381] Updated weights for policy 1, policy_version 1555224 (0.0006) [2023-12-27 16:25:54,959][20347] Updated weights for policy 0, policy_version 1556813 (0.0005) [2023-12-27 16:25:54,987][20381] Updated weights for policy 1, policy_version 1555234 (0.0007) [2023-12-27 16:25:55,640][20381] Updated weights for policy 1, policy_version 1555244 (0.0008) [2023-12-27 16:25:55,693][20347] Updated weights for policy 0, policy_version 1556823 (0.0007) [2023-12-27 16:25:55,697][20381] Updated weights for policy 1, policy_version 1555254 (0.0008) [2023-12-27 16:25:55,754][20347] Updated weights for policy 0, policy_version 1556833 (0.0006) [2023-12-27 16:25:55,763][20381] Updated weights for policy 1, policy_version 1555264 (0.0008) [2023-12-27 16:25:55,805][20347] Updated weights for policy 0, policy_version 1556843 (0.0007) [2023-12-27 16:25:56,311][20381] Updated weights for policy 1, policy_version 1555274 (0.0006) [2023-12-27 16:25:56,359][20381] Updated weights for policy 1, policy_version 1555284 (0.0010) [2023-12-27 16:25:56,407][20381] Updated weights for policy 1, policy_version 1555294 (0.0010) [2023-12-27 16:25:56,441][20347] Updated weights for policy 0, policy_version 1556853 (0.0006) [2023-12-27 16:25:56,455][20381] Updated weights for policy 1, policy_version 1555304 (0.0010) [2023-12-27 16:25:56,493][20347] Updated weights for policy 0, policy_version 1556863 (0.0007) [2023-12-27 16:25:56,544][20347] Updated weights for policy 0, policy_version 1556873 (0.0009) [2023-12-27 16:25:57,060][20381] Updated weights for policy 1, policy_version 1555314 (0.0009) [2023-12-27 16:25:57,112][20381] Updated weights for policy 1, policy_version 1555324 (0.0010) [2023-12-27 16:25:57,167][20381] Updated weights for policy 1, policy_version 1555334 (0.0010) [2023-12-27 16:25:57,179][20347] Updated weights for policy 0, policy_version 1556883 (0.0009) [2023-12-27 16:25:57,238][20347] Updated weights for policy 0, policy_version 1556893 (0.0010) [2023-12-27 16:25:57,301][20347] Updated weights for policy 0, policy_version 1556903 (0.0009) [2023-12-27 16:25:57,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 796852224. Throughput: 0: 10315.8, 1: 10241.4. Samples: 796834248. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:25:57,363][19170] Avg episode reward: [(0, '650.040'), (1, '835.530')] [2023-12-27 16:25:57,787][20381] Updated weights for policy 1, policy_version 1555344 (0.0010) [2023-12-27 16:25:57,836][20381] Updated weights for policy 1, policy_version 1555354 (0.0009) [2023-12-27 16:25:57,893][20381] Updated weights for policy 1, policy_version 1555364 (0.0005) [2023-12-27 16:25:57,932][20347] Updated weights for policy 0, policy_version 1556913 (0.0009) [2023-12-27 16:25:57,997][20347] Updated weights for policy 0, policy_version 1556923 (0.0005) [2023-12-27 16:25:58,063][20347] Updated weights for policy 0, policy_version 1556933 (0.0005) [2023-12-27 16:25:58,126][20347] Updated weights for policy 0, policy_version 1556943 (0.0007) [2023-12-27 16:25:58,541][20381] Updated weights for policy 1, policy_version 1555374 (0.0008) [2023-12-27 16:25:58,600][20381] Updated weights for policy 1, policy_version 1555384 (0.0010) [2023-12-27 16:25:58,662][20381] Updated weights for policy 1, policy_version 1555394 (0.0012) [2023-12-27 16:25:58,822][20347] Updated weights for policy 0, policy_version 1556953 (0.0008) [2023-12-27 16:25:58,883][20347] Updated weights for policy 0, policy_version 1556963 (0.0007) [2023-12-27 16:25:58,949][20347] Updated weights for policy 0, policy_version 1556973 (0.0006) [2023-12-27 16:25:59,455][20381] Updated weights for policy 1, policy_version 1555404 (0.0008) [2023-12-27 16:25:59,509][20381] Updated weights for policy 1, policy_version 1555414 (0.0005) [2023-12-27 16:25:59,567][20381] Updated weights for policy 1, policy_version 1555424 (0.0007) [2023-12-27 16:25:59,681][20347] Updated weights for policy 0, policy_version 1556983 (0.0006) [2023-12-27 16:25:59,740][20347] Updated weights for policy 0, policy_version 1556993 (0.0006) [2023-12-27 16:25:59,802][20347] Updated weights for policy 0, policy_version 1557003 (0.0007) [2023-12-27 16:26:00,212][20381] Updated weights for policy 1, policy_version 1555434 (0.0008) [2023-12-27 16:26:00,268][20381] Updated weights for policy 1, policy_version 1555444 (0.0010) [2023-12-27 16:26:00,327][20381] Updated weights for policy 1, policy_version 1555454 (0.0011) [2023-12-27 16:26:00,391][20381] Updated weights for policy 1, policy_version 1555464 (0.0011) [2023-12-27 16:26:00,486][20347] Updated weights for policy 0, policy_version 1557013 (0.0006) [2023-12-27 16:26:00,542][20347] Updated weights for policy 0, policy_version 1557023 (0.0005) [2023-12-27 16:26:00,586][20347] Updated weights for policy 0, policy_version 1557033 (0.0005) [2023-12-27 16:26:00,949][20381] Updated weights for policy 1, policy_version 1555474 (0.0008) [2023-12-27 16:26:01,002][20381] Updated weights for policy 1, policy_version 1555484 (0.0010) [2023-12-27 16:26:01,063][20381] Updated weights for policy 1, policy_version 1555494 (0.0007) [2023-12-27 16:26:01,159][20347] Updated weights for policy 0, policy_version 1557043 (0.0006) [2023-12-27 16:26:01,219][20347] Updated weights for policy 0, policy_version 1557053 (0.0007) [2023-12-27 16:26:01,273][20347] Updated weights for policy 0, policy_version 1557063 (0.0007) [2023-12-27 16:26:01,774][20381] Updated weights for policy 1, policy_version 1555504 (0.0008) [2023-12-27 16:26:01,834][20381] Updated weights for policy 1, policy_version 1555514 (0.0008) [2023-12-27 16:26:01,892][20381] Updated weights for policy 1, policy_version 1555524 (0.0009) [2023-12-27 16:26:01,977][20347] Updated weights for policy 0, policy_version 1557073 (0.0008) [2023-12-27 16:26:02,034][20347] Updated weights for policy 0, policy_version 1557083 (0.0008) [2023-12-27 16:26:02,094][20347] Updated weights for policy 0, policy_version 1557093 (0.0007) [2023-12-27 16:26:02,147][20347] Updated weights for policy 0, policy_version 1557103 (0.0007) [2023-12-27 16:26:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 796950528. Throughput: 0: 10322.5, 1: 10206.0. Samples: 796959524. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:26:02,363][19170] Avg episode reward: [(0, '698.260'), (1, '798.380')] [2023-12-27 16:26:02,577][20381] Updated weights for policy 1, policy_version 1555534 (0.0010) [2023-12-27 16:26:02,635][20381] Updated weights for policy 1, policy_version 1555544 (0.0010) [2023-12-27 16:26:02,693][20381] Updated weights for policy 1, policy_version 1555554 (0.0010) [2023-12-27 16:26:02,822][20347] Updated weights for policy 0, policy_version 1557113 (0.0010) [2023-12-27 16:26:02,881][20347] Updated weights for policy 0, policy_version 1557123 (0.0010) [2023-12-27 16:26:02,936][20347] Updated weights for policy 0, policy_version 1557133 (0.0010) [2023-12-27 16:26:03,344][20381] Updated weights for policy 1, policy_version 1555564 (0.0009) [2023-12-27 16:26:03,398][20381] Updated weights for policy 1, policy_version 1555574 (0.0008) [2023-12-27 16:26:03,447][20381] Updated weights for policy 1, policy_version 1555584 (0.0008) [2023-12-27 16:26:03,537][20347] Updated weights for policy 0, policy_version 1557143 (0.0007) [2023-12-27 16:26:03,595][20347] Updated weights for policy 0, policy_version 1557153 (0.0005) [2023-12-27 16:26:03,655][20347] Updated weights for policy 0, policy_version 1557163 (0.0006) [2023-12-27 16:26:04,219][20381] Updated weights for policy 1, policy_version 1555594 (0.0008) [2023-12-27 16:26:04,222][20347] Updated weights for policy 0, policy_version 1557173 (0.0009) [2023-12-27 16:26:04,281][20347] Updated weights for policy 0, policy_version 1557183 (0.0007) [2023-12-27 16:26:04,283][20381] Updated weights for policy 1, policy_version 1555604 (0.0007) [2023-12-27 16:26:04,337][20347] Updated weights for policy 0, policy_version 1557193 (0.0009) [2023-12-27 16:26:04,346][20381] Updated weights for policy 1, policy_version 1555614 (0.0006) [2023-12-27 16:26:04,404][20381] Updated weights for policy 1, policy_version 1555624 (0.0007) [2023-12-27 16:26:04,938][20347] Updated weights for policy 0, policy_version 1557203 (0.0009) [2023-12-27 16:26:04,993][20347] Updated weights for policy 0, policy_version 1557213 (0.0006) [2023-12-27 16:26:05,051][20347] Updated weights for policy 0, policy_version 1557223 (0.0006) [2023-12-27 16:26:05,136][20381] Updated weights for policy 1, policy_version 1555634 (0.0008) [2023-12-27 16:26:05,195][20381] Updated weights for policy 1, policy_version 1555644 (0.0008) [2023-12-27 16:26:05,256][20381] Updated weights for policy 1, policy_version 1555654 (0.0006) [2023-12-27 16:26:05,620][20347] Updated weights for policy 0, policy_version 1557233 (0.0007) [2023-12-27 16:26:05,673][20347] Updated weights for policy 0, policy_version 1557243 (0.0008) [2023-12-27 16:26:05,731][20347] Updated weights for policy 0, policy_version 1557253 (0.0005) [2023-12-27 16:26:05,784][20347] Updated weights for policy 0, policy_version 1557263 (0.0005) [2023-12-27 16:26:05,845][20381] Updated weights for policy 1, policy_version 1555664 (0.0009) [2023-12-27 16:26:05,896][20381] Updated weights for policy 1, policy_version 1555674 (0.0010) [2023-12-27 16:26:05,945][20381] Updated weights for policy 1, policy_version 1555684 (0.0010) [2023-12-27 16:26:06,417][20347] Updated weights for policy 0, policy_version 1557273 (0.0005) [2023-12-27 16:26:06,471][20347] Updated weights for policy 0, policy_version 1557283 (0.0006) [2023-12-27 16:26:06,534][20347] Updated weights for policy 0, policy_version 1557293 (0.0006) [2023-12-27 16:26:06,596][20381] Updated weights for policy 1, policy_version 1555694 (0.0009) [2023-12-27 16:26:06,656][20381] Updated weights for policy 1, policy_version 1555704 (0.0011) [2023-12-27 16:26:06,713][20381] Updated weights for policy 1, policy_version 1555714 (0.0011) [2023-12-27 16:26:07,146][20347] Updated weights for policy 0, policy_version 1557303 (0.0005) [2023-12-27 16:26:07,213][20347] Updated weights for policy 0, policy_version 1557313 (0.0007) [2023-12-27 16:26:07,276][20347] Updated weights for policy 0, policy_version 1557323 (0.0008) [2023-12-27 16:26:07,361][20381] Updated weights for policy 1, policy_version 1555724 (0.0008) [2023-12-27 16:26:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 797057024. Throughput: 0: 10386.4, 1: 10187.4. Samples: 797021968. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:26:07,363][19170] Avg episode reward: [(0, '713.620'), (1, '729.890')] [2023-12-27 16:26:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001557328_398753792.pth... [2023-12-27 16:26:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001556112_398442496.pth [2023-12-27 16:26:07,415][20381] Updated weights for policy 1, policy_version 1555734 (0.0011) [2023-12-27 16:26:07,461][20381] Updated weights for policy 1, policy_version 1555744 (0.0011) [2023-12-27 16:26:07,500][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001555752_398311424.pth... [2023-12-27 16:26:07,505][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001554536_398000128.pth [2023-12-27 16:26:07,897][20347] Updated weights for policy 0, policy_version 1557333 (0.0007) [2023-12-27 16:26:07,959][20347] Updated weights for policy 0, policy_version 1557343 (0.0005) [2023-12-27 16:26:08,014][20347] Updated weights for policy 0, policy_version 1557353 (0.0008) [2023-12-27 16:26:08,085][20381] Updated weights for policy 1, policy_version 1555754 (0.0010) [2023-12-27 16:26:08,151][20381] Updated weights for policy 1, policy_version 1555764 (0.0008) [2023-12-27 16:26:08,214][20381] Updated weights for policy 1, policy_version 1555774 (0.0008) [2023-12-27 16:26:08,268][20381] Updated weights for policy 1, policy_version 1555784 (0.0007) [2023-12-27 16:26:08,608][20347] Updated weights for policy 0, policy_version 1557363 (0.0007) [2023-12-27 16:26:08,666][20347] Updated weights for policy 0, policy_version 1557373 (0.0010) [2023-12-27 16:26:08,727][20347] Updated weights for policy 0, policy_version 1557383 (0.0010) [2023-12-27 16:26:08,953][20381] Updated weights for policy 1, policy_version 1555794 (0.0007) [2023-12-27 16:26:09,011][20381] Updated weights for policy 1, policy_version 1555804 (0.0007) [2023-12-27 16:26:09,064][20381] Updated weights for policy 1, policy_version 1555814 (0.0008) [2023-12-27 16:26:09,399][20347] Updated weights for policy 0, policy_version 1557393 (0.0010) [2023-12-27 16:26:09,458][20347] Updated weights for policy 0, policy_version 1557403 (0.0008) [2023-12-27 16:26:09,515][20347] Updated weights for policy 0, policy_version 1557413 (0.0008) [2023-12-27 16:26:09,567][20347] Updated weights for policy 0, policy_version 1557423 (0.0005) [2023-12-27 16:26:09,858][20381] Updated weights for policy 1, policy_version 1555824 (0.0007) [2023-12-27 16:26:09,927][20381] Updated weights for policy 1, policy_version 1555834 (0.0007) [2023-12-27 16:26:09,986][20381] Updated weights for policy 1, policy_version 1555844 (0.0006) [2023-12-27 16:26:10,282][20347] Updated weights for policy 0, policy_version 1557433 (0.0006) [2023-12-27 16:26:10,334][20347] Updated weights for policy 0, policy_version 1557443 (0.0006) [2023-12-27 16:26:10,397][20347] Updated weights for policy 0, policy_version 1557453 (0.0007) [2023-12-27 16:26:10,606][20381] Updated weights for policy 1, policy_version 1555854 (0.0007) [2023-12-27 16:26:10,652][20381] Updated weights for policy 1, policy_version 1555864 (0.0007) [2023-12-27 16:26:10,697][20381] Updated weights for policy 1, policy_version 1555874 (0.0007) [2023-12-27 16:26:11,045][20347] Updated weights for policy 0, policy_version 1557463 (0.0008) [2023-12-27 16:26:11,100][20347] Updated weights for policy 0, policy_version 1557473 (0.0008) [2023-12-27 16:26:11,168][20347] Updated weights for policy 0, policy_version 1557483 (0.0008) [2023-12-27 16:26:11,472][20381] Updated weights for policy 1, policy_version 1555884 (0.0007) [2023-12-27 16:26:11,531][20381] Updated weights for policy 1, policy_version 1555894 (0.0006) [2023-12-27 16:26:11,591][20381] Updated weights for policy 1, policy_version 1555904 (0.0006) [2023-12-27 16:26:11,981][20347] Updated weights for policy 0, policy_version 1557493 (0.0007) [2023-12-27 16:26:12,038][20347] Updated weights for policy 0, policy_version 1557503 (0.0007) [2023-12-27 16:26:12,092][20347] Updated weights for policy 0, policy_version 1557513 (0.0008) [2023-12-27 16:26:12,293][20381] Updated weights for policy 1, policy_version 1555914 (0.0007) [2023-12-27 16:26:12,352][20381] Updated weights for policy 1, policy_version 1555924 (0.0006) [2023-12-27 16:26:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 797155328. Throughput: 0: 10451.7, 1: 10210.5. Samples: 797148696. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:26:12,363][19170] Avg episode reward: [(0, '651.390'), (1, '712.970')] [2023-12-27 16:26:12,418][20381] Updated weights for policy 1, policy_version 1555934 (0.0007) [2023-12-27 16:26:12,474][20381] Updated weights for policy 1, policy_version 1555944 (0.0008) [2023-12-27 16:26:12,789][20347] Updated weights for policy 0, policy_version 1557523 (0.0006) [2023-12-27 16:26:12,839][20347] Updated weights for policy 0, policy_version 1557533 (0.0008) [2023-12-27 16:26:12,890][20347] Updated weights for policy 0, policy_version 1557543 (0.0008) [2023-12-27 16:26:13,115][20381] Updated weights for policy 1, policy_version 1555954 (0.0006) [2023-12-27 16:26:13,162][20381] Updated weights for policy 1, policy_version 1555964 (0.0005) [2023-12-27 16:26:13,208][20381] Updated weights for policy 1, policy_version 1555974 (0.0005) [2023-12-27 16:26:13,485][20347] Updated weights for policy 0, policy_version 1557553 (0.0007) [2023-12-27 16:26:13,541][20347] Updated weights for policy 0, policy_version 1557563 (0.0005) [2023-12-27 16:26:13,595][20347] Updated weights for policy 0, policy_version 1557573 (0.0006) [2023-12-27 16:26:13,650][20347] Updated weights for policy 0, policy_version 1557583 (0.0008) [2023-12-27 16:26:13,788][20381] Updated weights for policy 1, policy_version 1555984 (0.0006) [2023-12-27 16:26:13,841][20381] Updated weights for policy 1, policy_version 1555994 (0.0008) [2023-12-27 16:26:13,892][20381] Updated weights for policy 1, policy_version 1556004 (0.0008) [2023-12-27 16:26:14,253][20347] Updated weights for policy 0, policy_version 1557593 (0.0006) [2023-12-27 16:26:14,311][20347] Updated weights for policy 0, policy_version 1557603 (0.0007) [2023-12-27 16:26:14,373][20347] Updated weights for policy 0, policy_version 1557613 (0.0008) [2023-12-27 16:26:14,534][20381] Updated weights for policy 1, policy_version 1556014 (0.0007) [2023-12-27 16:26:14,586][20381] Updated weights for policy 1, policy_version 1556024 (0.0007) [2023-12-27 16:26:14,638][20381] Updated weights for policy 1, policy_version 1556034 (0.0005) [2023-12-27 16:26:15,024][20347] Updated weights for policy 0, policy_version 1557623 (0.0011) [2023-12-27 16:26:15,077][20347] Updated weights for policy 0, policy_version 1557633 (0.0010) [2023-12-27 16:26:15,130][20347] Updated weights for policy 0, policy_version 1557643 (0.0011) [2023-12-27 16:26:15,297][20381] Updated weights for policy 1, policy_version 1556044 (0.0007) [2023-12-27 16:26:15,360][20381] Updated weights for policy 1, policy_version 1556054 (0.0011) [2023-12-27 16:26:15,415][20381] Updated weights for policy 1, policy_version 1556064 (0.0010) [2023-12-27 16:26:15,782][20347] Updated weights for policy 0, policy_version 1557653 (0.0010) [2023-12-27 16:26:15,844][20347] Updated weights for policy 0, policy_version 1557663 (0.0010) [2023-12-27 16:26:15,893][20347] Updated weights for policy 0, policy_version 1557673 (0.0010) [2023-12-27 16:26:16,008][20381] Updated weights for policy 1, policy_version 1556074 (0.0009) [2023-12-27 16:26:16,070][20381] Updated weights for policy 1, policy_version 1556084 (0.0008) [2023-12-27 16:26:16,133][20381] Updated weights for policy 1, policy_version 1556094 (0.0008) [2023-12-27 16:26:16,194][20381] Updated weights for policy 1, policy_version 1556104 (0.0007) [2023-12-27 16:26:16,531][20347] Updated weights for policy 0, policy_version 1557683 (0.0010) [2023-12-27 16:26:16,583][20347] Updated weights for policy 0, policy_version 1557693 (0.0010) [2023-12-27 16:26:16,635][20347] Updated weights for policy 0, policy_version 1557703 (0.0010) [2023-12-27 16:26:16,856][20381] Updated weights for policy 1, policy_version 1556114 (0.0006) [2023-12-27 16:26:16,915][20381] Updated weights for policy 1, policy_version 1556124 (0.0006) [2023-12-27 16:26:16,961][20381] Updated weights for policy 1, policy_version 1556134 (0.0005) [2023-12-27 16:26:17,326][20347] Updated weights for policy 0, policy_version 1557713 (0.0010) [2023-12-27 16:26:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 797261824. Throughput: 0: 10400.2, 1: 10275.9. Samples: 797275556. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) [2023-12-27 16:26:17,363][19170] Avg episode reward: [(0, '576.440'), (1, '788.660')] [2023-12-27 16:26:17,386][20347] Updated weights for policy 0, policy_version 1557723 (0.0010) [2023-12-27 16:26:17,440][20347] Updated weights for policy 0, policy_version 1557733 (0.0010) [2023-12-27 16:26:17,488][20347] Updated weights for policy 0, policy_version 1557743 (0.0010) [2023-12-27 16:26:17,600][20381] Updated weights for policy 1, policy_version 1556144 (0.0005) [2023-12-27 16:26:17,657][20381] Updated weights for policy 1, policy_version 1556154 (0.0008) [2023-12-27 16:26:17,705][20381] Updated weights for policy 1, policy_version 1556164 (0.0008) [2023-12-27 16:26:18,153][20347] Updated weights for policy 0, policy_version 1557753 (0.0006) [2023-12-27 16:26:18,209][20347] Updated weights for policy 0, policy_version 1557763 (0.0005) [2023-12-27 16:26:18,262][20347] Updated weights for policy 0, policy_version 1557773 (0.0009) [2023-12-27 16:26:18,282][20381] Updated weights for policy 1, policy_version 1556174 (0.0006) [2023-12-27 16:26:18,341][20381] Updated weights for policy 1, policy_version 1556184 (0.0007) [2023-12-27 16:26:18,400][20381] Updated weights for policy 1, policy_version 1556194 (0.0007) [2023-12-27 16:26:18,865][20347] Updated weights for policy 0, policy_version 1557783 (0.0010) [2023-12-27 16:26:18,925][20347] Updated weights for policy 0, policy_version 1557793 (0.0011) [2023-12-27 16:26:18,980][20347] Updated weights for policy 0, policy_version 1557803 (0.0006) [2023-12-27 16:26:19,151][20381] Updated weights for policy 1, policy_version 1556204 (0.0008) [2023-12-27 16:26:19,212][20381] Updated weights for policy 1, policy_version 1556214 (0.0009) [2023-12-27 16:26:19,281][20381] Updated weights for policy 1, policy_version 1556224 (0.0008) [2023-12-27 16:26:19,603][20347] Updated weights for policy 0, policy_version 1557813 (0.0007) [2023-12-27 16:26:19,670][20347] Updated weights for policy 0, policy_version 1557823 (0.0009) [2023-12-27 16:26:19,735][20347] Updated weights for policy 0, policy_version 1557833 (0.0008) [2023-12-27 16:26:19,947][20381] Updated weights for policy 1, policy_version 1556234 (0.0008) [2023-12-27 16:26:20,008][20381] Updated weights for policy 1, policy_version 1556244 (0.0006) [2023-12-27 16:26:20,071][20381] Updated weights for policy 1, policy_version 1556254 (0.0008) [2023-12-27 16:26:20,132][20381] Updated weights for policy 1, policy_version 1556264 (0.0008) [2023-12-27 16:26:20,456][20347] Updated weights for policy 0, policy_version 1557843 (0.0009) [2023-12-27 16:26:20,511][20347] Updated weights for policy 0, policy_version 1557853 (0.0010) [2023-12-27 16:26:20,575][20347] Updated weights for policy 0, policy_version 1557863 (0.0011) [2023-12-27 16:26:20,870][20381] Updated weights for policy 1, policy_version 1556274 (0.0008) [2023-12-27 16:26:20,930][20381] Updated weights for policy 1, policy_version 1556284 (0.0008) [2023-12-27 16:26:20,991][20381] Updated weights for policy 1, policy_version 1556294 (0.0007) [2023-12-27 16:26:21,297][20347] Updated weights for policy 0, policy_version 1557873 (0.0011) [2023-12-27 16:26:21,360][20347] Updated weights for policy 0, policy_version 1557883 (0.0010) [2023-12-27 16:26:21,430][20347] Updated weights for policy 0, policy_version 1557893 (0.0008) [2023-12-27 16:26:21,492][20347] Updated weights for policy 0, policy_version 1557903 (0.0008) [2023-12-27 16:26:21,768][20381] Updated weights for policy 1, policy_version 1556304 (0.0007) [2023-12-27 16:26:21,831][20381] Updated weights for policy 1, policy_version 1556314 (0.0008) [2023-12-27 16:26:21,893][20381] Updated weights for policy 1, policy_version 1556324 (0.0008) [2023-12-27 16:26:22,225][20347] Updated weights for policy 0, policy_version 1557913 (0.0008) [2023-12-27 16:26:22,286][20347] Updated weights for policy 0, policy_version 1557923 (0.0008) [2023-12-27 16:26:22,338][20347] Updated weights for policy 0, policy_version 1557933 (0.0007) [2023-12-27 16:26:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 797368320. Throughput: 0: 10414.9, 1: 10271.8. Samples: 797337580. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:22,363][19170] Avg episode reward: [(0, '558.290'), (1, '741.230')] [2023-12-27 16:26:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001556328_398458880.pth... [2023-12-27 16:26:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001557936_398909440.pth... [2023-12-27 16:26:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001555112_398147584.pth [2023-12-27 16:26:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001556688_398589952.pth [2023-12-27 16:26:22,606][20381] Updated weights for policy 1, policy_version 1556334 (0.0008) [2023-12-27 16:26:22,660][20381] Updated weights for policy 1, policy_version 1556344 (0.0008) [2023-12-27 16:26:22,709][20381] Updated weights for policy 1, policy_version 1556354 (0.0007) [2023-12-27 16:26:22,988][20347] Updated weights for policy 0, policy_version 1557943 (0.0006) [2023-12-27 16:26:23,045][20347] Updated weights for policy 0, policy_version 1557953 (0.0010) [2023-12-27 16:26:23,107][20347] Updated weights for policy 0, policy_version 1557963 (0.0010) [2023-12-27 16:26:23,394][20381] Updated weights for policy 1, policy_version 1556364 (0.0007) [2023-12-27 16:26:23,449][20381] Updated weights for policy 1, policy_version 1556374 (0.0008) [2023-12-27 16:26:23,505][20381] Updated weights for policy 1, policy_version 1556384 (0.0007) [2023-12-27 16:26:23,746][20347] Updated weights for policy 0, policy_version 1557973 (0.0008) [2023-12-27 16:26:23,798][20347] Updated weights for policy 0, policy_version 1557983 (0.0010) [2023-12-27 16:26:23,843][20347] Updated weights for policy 0, policy_version 1557993 (0.0010) [2023-12-27 16:26:24,197][20381] Updated weights for policy 1, policy_version 1556394 (0.0007) [2023-12-27 16:26:24,251][20381] Updated weights for policy 1, policy_version 1556404 (0.0007) [2023-12-27 16:26:24,301][20381] Updated weights for policy 1, policy_version 1556414 (0.0007) [2023-12-27 16:26:24,357][20381] Updated weights for policy 1, policy_version 1556424 (0.0007) [2023-12-27 16:26:24,461][20347] Updated weights for policy 0, policy_version 1558003 (0.0009) [2023-12-27 16:26:24,517][20347] Updated weights for policy 0, policy_version 1558013 (0.0005) [2023-12-27 16:26:24,571][20347] Updated weights for policy 0, policy_version 1558023 (0.0005) [2023-12-27 16:26:24,976][20381] Updated weights for policy 1, policy_version 1556434 (0.0005) [2023-12-27 16:26:25,032][20381] Updated weights for policy 1, policy_version 1556444 (0.0005) [2023-12-27 16:26:25,087][20381] Updated weights for policy 1, policy_version 1556454 (0.0005) [2023-12-27 16:26:25,181][20347] Updated weights for policy 0, policy_version 1558033 (0.0006) [2023-12-27 16:26:25,232][20347] Updated weights for policy 0, policy_version 1558043 (0.0010) [2023-12-27 16:26:25,284][20347] Updated weights for policy 0, policy_version 1558053 (0.0010) [2023-12-27 16:26:25,335][20347] Updated weights for policy 0, policy_version 1558063 (0.0010) [2023-12-27 16:26:25,692][20381] Updated weights for policy 1, policy_version 1556464 (0.0007) [2023-12-27 16:26:25,757][20381] Updated weights for policy 1, policy_version 1556474 (0.0007) [2023-12-27 16:26:25,816][20381] Updated weights for policy 1, policy_version 1556484 (0.0008) [2023-12-27 16:26:26,002][20347] Updated weights for policy 0, policy_version 1558073 (0.0009) [2023-12-27 16:26:26,053][20347] Updated weights for policy 0, policy_version 1558083 (0.0005) [2023-12-27 16:26:26,108][20347] Updated weights for policy 0, policy_version 1558093 (0.0005) [2023-12-27 16:26:26,470][20381] Updated weights for policy 1, policy_version 1556494 (0.0008) [2023-12-27 16:26:26,518][20381] Updated weights for policy 1, policy_version 1556504 (0.0007) [2023-12-27 16:26:26,572][20381] Updated weights for policy 1, policy_version 1556514 (0.0006) [2023-12-27 16:26:26,743][20347] Updated weights for policy 0, policy_version 1558103 (0.0009) [2023-12-27 16:26:26,792][20347] Updated weights for policy 0, policy_version 1558113 (0.0007) [2023-12-27 16:26:26,840][20347] Updated weights for policy 0, policy_version 1558123 (0.0010) [2023-12-27 16:26:27,307][20381] Updated weights for policy 1, policy_version 1556524 (0.0007) [2023-12-27 16:26:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 797466624. Throughput: 0: 10433.2, 1: 10265.2. Samples: 797459936. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:27,363][19170] Avg episode reward: [(0, '596.000'), (1, '771.400')] [2023-12-27 16:26:27,365][20381] Updated weights for policy 1, policy_version 1556534 (0.0007) [2023-12-27 16:26:27,410][20381] Updated weights for policy 1, policy_version 1556544 (0.0007) [2023-12-27 16:26:27,470][20347] Updated weights for policy 0, policy_version 1558133 (0.0008) [2023-12-27 16:26:27,527][20347] Updated weights for policy 0, policy_version 1558143 (0.0005) [2023-12-27 16:26:27,577][20347] Updated weights for policy 0, policy_version 1558153 (0.0006) [2023-12-27 16:26:28,149][20381] Updated weights for policy 1, policy_version 1556554 (0.0009) [2023-12-27 16:26:28,162][20347] Updated weights for policy 0, policy_version 1558163 (0.0010) [2023-12-27 16:26:28,197][20381] Updated weights for policy 1, policy_version 1556564 (0.0005) [2023-12-27 16:26:28,217][20347] Updated weights for policy 0, policy_version 1558173 (0.0010) [2023-12-27 16:26:28,252][20381] Updated weights for policy 1, policy_version 1556574 (0.0005) [2023-12-27 16:26:28,271][20347] Updated weights for policy 0, policy_version 1558183 (0.0010) [2023-12-27 16:26:28,300][20381] Updated weights for policy 1, policy_version 1556584 (0.0005) [2023-12-27 16:26:28,942][20381] Updated weights for policy 1, policy_version 1556594 (0.0006) [2023-12-27 16:26:28,958][20347] Updated weights for policy 0, policy_version 1558193 (0.0010) [2023-12-27 16:26:29,001][20381] Updated weights for policy 1, policy_version 1556604 (0.0005) [2023-12-27 16:26:29,010][20347] Updated weights for policy 0, policy_version 1558203 (0.0010) [2023-12-27 16:26:29,050][20381] Updated weights for policy 1, policy_version 1556614 (0.0005) [2023-12-27 16:26:29,057][20347] Updated weights for policy 0, policy_version 1558213 (0.0010) [2023-12-27 16:26:29,105][20347] Updated weights for policy 0, policy_version 1558223 (0.0010) [2023-12-27 16:26:29,678][20381] Updated weights for policy 1, policy_version 1556624 (0.0009) [2023-12-27 16:26:29,736][20381] Updated weights for policy 1, policy_version 1556634 (0.0008) [2023-12-27 16:26:29,796][20381] Updated weights for policy 1, policy_version 1556644 (0.0006) [2023-12-27 16:26:29,797][20347] Updated weights for policy 0, policy_version 1558233 (0.0007) [2023-12-27 16:26:29,851][20347] Updated weights for policy 0, policy_version 1558243 (0.0008) [2023-12-27 16:26:29,907][20347] Updated weights for policy 0, policy_version 1558253 (0.0007) [2023-12-27 16:26:30,457][20381] Updated weights for policy 1, policy_version 1556654 (0.0007) [2023-12-27 16:26:30,514][20381] Updated weights for policy 1, policy_version 1556664 (0.0007) [2023-12-27 16:26:30,569][20381] Updated weights for policy 1, policy_version 1556674 (0.0007) [2023-12-27 16:26:30,637][20347] Updated weights for policy 0, policy_version 1558263 (0.0005) [2023-12-27 16:26:30,693][20347] Updated weights for policy 0, policy_version 1558273 (0.0007) [2023-12-27 16:26:30,745][20347] Updated weights for policy 0, policy_version 1558283 (0.0008) [2023-12-27 16:26:31,232][20381] Updated weights for policy 1, policy_version 1556684 (0.0008) [2023-12-27 16:26:31,295][20381] Updated weights for policy 1, policy_version 1556694 (0.0007) [2023-12-27 16:26:31,352][20381] Updated weights for policy 1, policy_version 1556704 (0.0006) [2023-12-27 16:26:31,492][20347] Updated weights for policy 0, policy_version 1558293 (0.0007) [2023-12-27 16:26:31,551][20347] Updated weights for policy 0, policy_version 1558303 (0.0008) [2023-12-27 16:26:31,610][20347] Updated weights for policy 0, policy_version 1558313 (0.0008) [2023-12-27 16:26:32,047][20381] Updated weights for policy 1, policy_version 1556714 (0.0006) [2023-12-27 16:26:32,107][20381] Updated weights for policy 1, policy_version 1556724 (0.0007) [2023-12-27 16:26:32,160][20381] Updated weights for policy 1, policy_version 1556734 (0.0008) [2023-12-27 16:26:32,212][20381] Updated weights for policy 1, policy_version 1556744 (0.0009) [2023-12-27 16:26:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 797573120. Throughput: 0: 10426.6, 1: 10268.4. Samples: 797583688. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:32,363][19170] Avg episode reward: [(0, '679.660'), (1, '837.010')] [2023-12-27 16:26:32,369][20347] Updated weights for policy 0, policy_version 1558323 (0.0007) [2023-12-27 16:26:32,427][20347] Updated weights for policy 0, policy_version 1558333 (0.0006) [2023-12-27 16:26:32,482][20347] Updated weights for policy 0, policy_version 1558343 (0.0007) [2023-12-27 16:26:32,957][20381] Updated weights for policy 1, policy_version 1556754 (0.0011) [2023-12-27 16:26:33,002][20381] Updated weights for policy 1, policy_version 1556764 (0.0010) [2023-12-27 16:26:33,048][20381] Updated weights for policy 1, policy_version 1556774 (0.0010) [2023-12-27 16:26:33,172][20347] Updated weights for policy 0, policy_version 1558353 (0.0007) [2023-12-27 16:26:33,228][20347] Updated weights for policy 0, policy_version 1558363 (0.0007) [2023-12-27 16:26:33,282][20347] Updated weights for policy 0, policy_version 1558373 (0.0007) [2023-12-27 16:26:33,333][20347] Updated weights for policy 0, policy_version 1558383 (0.0007) [2023-12-27 16:26:33,657][20381] Updated weights for policy 1, policy_version 1556784 (0.0010) [2023-12-27 16:26:33,711][20381] Updated weights for policy 1, policy_version 1556794 (0.0010) [2023-12-27 16:26:33,760][20381] Updated weights for policy 1, policy_version 1556804 (0.0006) [2023-12-27 16:26:34,078][20347] Updated weights for policy 0, policy_version 1558393 (0.0007) [2023-12-27 16:26:34,132][20347] Updated weights for policy 0, policy_version 1558403 (0.0007) [2023-12-27 16:26:34,195][20347] Updated weights for policy 0, policy_version 1558413 (0.0009) [2023-12-27 16:26:34,332][20381] Updated weights for policy 1, policy_version 1556814 (0.0005) [2023-12-27 16:26:34,391][20381] Updated weights for policy 1, policy_version 1556824 (0.0005) [2023-12-27 16:26:34,457][20381] Updated weights for policy 1, policy_version 1556834 (0.0007) [2023-12-27 16:26:34,900][20347] Updated weights for policy 0, policy_version 1558423 (0.0006) [2023-12-27 16:26:34,959][20347] Updated weights for policy 0, policy_version 1558433 (0.0010) [2023-12-27 16:26:35,008][20347] Updated weights for policy 0, policy_version 1558443 (0.0007) [2023-12-27 16:26:35,083][20381] Updated weights for policy 1, policy_version 1556844 (0.0007) [2023-12-27 16:26:35,141][20381] Updated weights for policy 1, policy_version 1556854 (0.0005) [2023-12-27 16:26:35,198][20381] Updated weights for policy 1, policy_version 1556864 (0.0005) [2023-12-27 16:26:35,617][20347] Updated weights for policy 0, policy_version 1558453 (0.0008) [2023-12-27 16:26:35,674][20347] Updated weights for policy 0, policy_version 1558463 (0.0007) [2023-12-27 16:26:35,727][20347] Updated weights for policy 0, policy_version 1558473 (0.0005) [2023-12-27 16:26:35,783][20381] Updated weights for policy 1, policy_version 1556874 (0.0006) [2023-12-27 16:26:35,839][20381] Updated weights for policy 1, policy_version 1556884 (0.0008) [2023-12-27 16:26:35,891][20381] Updated weights for policy 1, policy_version 1556894 (0.0007) [2023-12-27 16:26:35,937][20381] Updated weights for policy 1, policy_version 1556904 (0.0008) [2023-12-27 16:26:36,413][20347] Updated weights for policy 0, policy_version 1558483 (0.0008) [2023-12-27 16:26:36,469][20347] Updated weights for policy 0, policy_version 1558493 (0.0007) [2023-12-27 16:26:36,520][20347] Updated weights for policy 0, policy_version 1558503 (0.0007) [2023-12-27 16:26:36,601][20381] Updated weights for policy 1, policy_version 1556914 (0.0006) [2023-12-27 16:26:36,651][20381] Updated weights for policy 1, policy_version 1556924 (0.0006) [2023-12-27 16:26:36,703][20381] Updated weights for policy 1, policy_version 1556934 (0.0006) [2023-12-27 16:26:37,185][20347] Updated weights for policy 0, policy_version 1558513 (0.0008) [2023-12-27 16:26:37,250][20347] Updated weights for policy 0, policy_version 1558523 (0.0007) [2023-12-27 16:26:37,322][20347] Updated weights for policy 0, policy_version 1558533 (0.0007) [2023-12-27 16:26:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 797671424. Throughput: 0: 10402.1, 1: 10351.0. Samples: 797646336. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:37,363][19170] Avg episode reward: [(0, '622.430'), (1, '815.290')] [2023-12-27 16:26:37,388][20381] Updated weights for policy 1, policy_version 1556944 (0.0007) [2023-12-27 16:26:37,390][20347] Updated weights for policy 0, policy_version 1558543 (0.0007) [2023-12-27 16:26:37,394][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001558544_399065088.pth... [2023-12-27 16:26:37,399][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001557328_398753792.pth [2023-12-27 16:26:37,441][20381] Updated weights for policy 1, policy_version 1556954 (0.0009) [2023-12-27 16:26:37,497][20381] Updated weights for policy 1, policy_version 1556964 (0.0008) [2023-12-27 16:26:37,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001556968_398622720.pth... [2023-12-27 16:26:37,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001555752_398311424.pth [2023-12-27 16:26:38,029][20347] Updated weights for policy 0, policy_version 1558553 (0.0006) [2023-12-27 16:26:38,080][20347] Updated weights for policy 0, policy_version 1558563 (0.0005) [2023-12-27 16:26:38,128][20347] Updated weights for policy 0, policy_version 1558573 (0.0005) [2023-12-27 16:26:38,212][20381] Updated weights for policy 1, policy_version 1556974 (0.0008) [2023-12-27 16:26:38,275][20381] Updated weights for policy 1, policy_version 1556984 (0.0008) [2023-12-27 16:26:38,337][20381] Updated weights for policy 1, policy_version 1556994 (0.0008) [2023-12-27 16:26:38,803][20347] Updated weights for policy 0, policy_version 1558583 (0.0007) [2023-12-27 16:26:38,862][20347] Updated weights for policy 0, policy_version 1558593 (0.0008) [2023-12-27 16:26:38,922][20347] Updated weights for policy 0, policy_version 1558603 (0.0008) [2023-12-27 16:26:38,971][20381] Updated weights for policy 1, policy_version 1557004 (0.0007) [2023-12-27 16:26:39,026][20381] Updated weights for policy 1, policy_version 1557014 (0.0007) [2023-12-27 16:26:39,082][20381] Updated weights for policy 1, policy_version 1557024 (0.0008) [2023-12-27 16:26:39,647][20347] Updated weights for policy 0, policy_version 1558613 (0.0009) [2023-12-27 16:26:39,703][20347] Updated weights for policy 0, policy_version 1558623 (0.0010) [2023-12-27 16:26:39,761][20347] Updated weights for policy 0, policy_version 1558633 (0.0010) [2023-12-27 16:26:39,810][20381] Updated weights for policy 1, policy_version 1557034 (0.0008) [2023-12-27 16:26:39,873][20381] Updated weights for policy 1, policy_version 1557044 (0.0008) [2023-12-27 16:26:39,935][20381] Updated weights for policy 1, policy_version 1557054 (0.0007) [2023-12-27 16:26:39,991][20381] Updated weights for policy 1, policy_version 1557064 (0.0007) [2023-12-27 16:26:40,458][20347] Updated weights for policy 0, policy_version 1558643 (0.0009) [2023-12-27 16:26:40,521][20347] Updated weights for policy 0, policy_version 1558653 (0.0005) [2023-12-27 16:26:40,573][20347] Updated weights for policy 0, policy_version 1558663 (0.0005) [2023-12-27 16:26:40,696][20381] Updated weights for policy 1, policy_version 1557074 (0.0008) [2023-12-27 16:26:40,744][20381] Updated weights for policy 1, policy_version 1557084 (0.0007) [2023-12-27 16:26:40,796][20381] Updated weights for policy 1, policy_version 1557095 (0.0009) [2023-12-27 16:26:41,125][20347] Updated weights for policy 0, policy_version 1558673 (0.0006) [2023-12-27 16:26:41,194][20347] Updated weights for policy 0, policy_version 1558683 (0.0011) [2023-12-27 16:26:41,259][20347] Updated weights for policy 0, policy_version 1558693 (0.0011) [2023-12-27 16:26:41,323][20347] Updated weights for policy 0, policy_version 1558703 (0.0010) [2023-12-27 16:26:41,540][20381] Updated weights for policy 1, policy_version 1557105 (0.0007) [2023-12-27 16:26:41,597][20381] Updated weights for policy 1, policy_version 1557115 (0.0008) [2023-12-27 16:26:41,666][20381] Updated weights for policy 1, policy_version 1557125 (0.0008) [2023-12-27 16:26:42,043][20347] Updated weights for policy 0, policy_version 1558713 (0.0007) [2023-12-27 16:26:42,098][20347] Updated weights for policy 0, policy_version 1558723 (0.0010) [2023-12-27 16:26:42,159][20347] Updated weights for policy 0, policy_version 1558733 (0.0010) [2023-12-27 16:26:42,356][20381] Updated weights for policy 1, policy_version 1557135 (0.0008) [2023-12-27 16:26:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 797777920. Throughput: 0: 10466.6, 1: 10316.3. Samples: 797769480. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:42,363][19170] Avg episode reward: [(0, '597.110'), (1, '805.260')] [2023-12-27 16:26:42,421][20381] Updated weights for policy 1, policy_version 1557145 (0.0007) [2023-12-27 16:26:42,485][20381] Updated weights for policy 1, policy_version 1557155 (0.0008) [2023-12-27 16:26:42,862][20347] Updated weights for policy 0, policy_version 1558743 (0.0008) [2023-12-27 16:26:42,914][20347] Updated weights for policy 0, policy_version 1558753 (0.0007) [2023-12-27 16:26:42,962][20347] Updated weights for policy 0, policy_version 1558763 (0.0007) [2023-12-27 16:26:43,192][20381] Updated weights for policy 1, policy_version 1557165 (0.0007) [2023-12-27 16:26:43,248][20381] Updated weights for policy 1, policy_version 1557175 (0.0008) [2023-12-27 16:26:43,307][20381] Updated weights for policy 1, policy_version 1557185 (0.0008) [2023-12-27 16:26:43,655][20347] Updated weights for policy 0, policy_version 1558773 (0.0007) [2023-12-27 16:26:43,710][20347] Updated weights for policy 0, policy_version 1558783 (0.0005) [2023-12-27 16:26:43,760][20347] Updated weights for policy 0, policy_version 1558793 (0.0007) [2023-12-27 16:26:44,022][20381] Updated weights for policy 1, policy_version 1557195 (0.0008) [2023-12-27 16:26:44,073][20381] Updated weights for policy 1, policy_version 1557205 (0.0007) [2023-12-27 16:26:44,125][20381] Updated weights for policy 1, policy_version 1557215 (0.0007) [2023-12-27 16:26:44,369][20347] Updated weights for policy 0, policy_version 1558803 (0.0006) [2023-12-27 16:26:44,419][20347] Updated weights for policy 0, policy_version 1558813 (0.0008) [2023-12-27 16:26:44,466][20347] Updated weights for policy 0, policy_version 1558823 (0.0007) [2023-12-27 16:26:44,803][20381] Updated weights for policy 1, policy_version 1557225 (0.0007) [2023-12-27 16:26:44,866][20381] Updated weights for policy 1, policy_version 1557235 (0.0007) [2023-12-27 16:26:44,933][20381] Updated weights for policy 1, policy_version 1557245 (0.0006) [2023-12-27 16:26:44,997][20381] Updated weights for policy 1, policy_version 1557255 (0.0006) [2023-12-27 16:26:45,180][20347] Updated weights for policy 0, policy_version 1558833 (0.0008) [2023-12-27 16:26:45,238][20347] Updated weights for policy 0, policy_version 1558843 (0.0008) [2023-12-27 16:26:45,285][20347] Updated weights for policy 0, policy_version 1558853 (0.0008) [2023-12-27 16:26:45,340][20347] Updated weights for policy 0, policy_version 1558863 (0.0008) [2023-12-27 16:26:45,614][20381] Updated weights for policy 1, policy_version 1557265 (0.0007) [2023-12-27 16:26:45,658][20381] Updated weights for policy 1, policy_version 1557275 (0.0007) [2023-12-27 16:26:45,709][20381] Updated weights for policy 1, policy_version 1557285 (0.0007) [2023-12-27 16:26:46,044][20347] Updated weights for policy 0, policy_version 1558873 (0.0008) [2023-12-27 16:26:46,093][20347] Updated weights for policy 0, policy_version 1558883 (0.0007) [2023-12-27 16:26:46,144][20347] Updated weights for policy 0, policy_version 1558893 (0.0008) [2023-12-27 16:26:46,374][20381] Updated weights for policy 1, policy_version 1557295 (0.0007) [2023-12-27 16:26:46,420][20381] Updated weights for policy 1, policy_version 1557305 (0.0008) [2023-12-27 16:26:46,470][20381] Updated weights for policy 1, policy_version 1557315 (0.0007) [2023-12-27 16:26:46,848][20347] Updated weights for policy 0, policy_version 1558903 (0.0008) [2023-12-27 16:26:46,902][20347] Updated weights for policy 0, policy_version 1558913 (0.0007) [2023-12-27 16:26:46,961][20347] Updated weights for policy 0, policy_version 1558923 (0.0008) [2023-12-27 16:26:47,172][20381] Updated weights for policy 1, policy_version 1557325 (0.0008) [2023-12-27 16:26:47,222][20381] Updated weights for policy 1, policy_version 1557335 (0.0007) [2023-12-27 16:26:47,276][20381] Updated weights for policy 1, policy_version 1557345 (0.0007) [2023-12-27 16:26:47,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 797884416. Throughput: 0: 10412.2, 1: 10265.0. Samples: 797890004. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:47,364][19170] Avg episode reward: [(0, '526.850'), (1, '873.750')] [2023-12-27 16:26:47,649][20347] Updated weights for policy 0, policy_version 1558933 (0.0008) [2023-12-27 16:26:47,705][20347] Updated weights for policy 0, policy_version 1558943 (0.0008) [2023-12-27 16:26:47,764][20347] Updated weights for policy 0, policy_version 1558953 (0.0007) [2023-12-27 16:26:47,957][20381] Updated weights for policy 1, policy_version 1557355 (0.0007) [2023-12-27 16:26:48,008][20381] Updated weights for policy 1, policy_version 1557365 (0.0005) [2023-12-27 16:26:48,058][20381] Updated weights for policy 1, policy_version 1557375 (0.0008) [2023-12-27 16:26:48,479][20347] Updated weights for policy 0, policy_version 1558963 (0.0008) [2023-12-27 16:26:48,538][20347] Updated weights for policy 0, policy_version 1558973 (0.0008) [2023-12-27 16:26:48,603][20347] Updated weights for policy 0, policy_version 1558983 (0.0007) [2023-12-27 16:26:48,686][20381] Updated weights for policy 1, policy_version 1557385 (0.0008) [2023-12-27 16:26:48,745][20381] Updated weights for policy 1, policy_version 1557395 (0.0007) [2023-12-27 16:26:48,799][20381] Updated weights for policy 1, policy_version 1557405 (0.0008) [2023-12-27 16:26:48,863][20381] Updated weights for policy 1, policy_version 1557415 (0.0008) [2023-12-27 16:26:49,275][20347] Updated weights for policy 0, policy_version 1558993 (0.0008) [2023-12-27 16:26:49,337][20347] Updated weights for policy 0, policy_version 1559003 (0.0007) [2023-12-27 16:26:49,393][20347] Updated weights for policy 0, policy_version 1559013 (0.0006) [2023-12-27 16:26:49,451][20347] Updated weights for policy 0, policy_version 1559023 (0.0005) [2023-12-27 16:26:49,607][20381] Updated weights for policy 1, policy_version 1557425 (0.0008) [2023-12-27 16:26:49,661][20381] Updated weights for policy 1, policy_version 1557435 (0.0007) [2023-12-27 16:26:49,710][20381] Updated weights for policy 1, policy_version 1557445 (0.0008) [2023-12-27 16:26:50,117][20347] Updated weights for policy 0, policy_version 1559033 (0.0007) [2023-12-27 16:26:50,174][20347] Updated weights for policy 0, policy_version 1559043 (0.0008) [2023-12-27 16:26:50,223][20347] Updated weights for policy 0, policy_version 1559053 (0.0007) [2023-12-27 16:26:50,403][20381] Updated weights for policy 1, policy_version 1557455 (0.0007) [2023-12-27 16:26:50,459][20381] Updated weights for policy 1, policy_version 1557465 (0.0007) [2023-12-27 16:26:50,513][20381] Updated weights for policy 1, policy_version 1557475 (0.0008) [2023-12-27 16:26:50,979][20347] Updated weights for policy 0, policy_version 1559063 (0.0008) [2023-12-27 16:26:51,039][20347] Updated weights for policy 0, policy_version 1559073 (0.0008) [2023-12-27 16:26:51,108][20347] Updated weights for policy 0, policy_version 1559083 (0.0007) [2023-12-27 16:26:51,194][20381] Updated weights for policy 1, policy_version 1557485 (0.0008) [2023-12-27 16:26:51,256][20381] Updated weights for policy 1, policy_version 1557495 (0.0008) [2023-12-27 16:26:51,324][20381] Updated weights for policy 1, policy_version 1557505 (0.0007) [2023-12-27 16:26:51,743][20347] Updated weights for policy 0, policy_version 1559093 (0.0007) [2023-12-27 16:26:51,790][20347] Updated weights for policy 0, policy_version 1559103 (0.0005) [2023-12-27 16:26:51,836][20347] Updated weights for policy 0, policy_version 1559113 (0.0005) [2023-12-27 16:26:52,048][20381] Updated weights for policy 1, policy_version 1557515 (0.0008) [2023-12-27 16:26:52,104][20381] Updated weights for policy 1, policy_version 1557525 (0.0007) [2023-12-27 16:26:52,159][20381] Updated weights for policy 1, policy_version 1557535 (0.0007) [2023-12-27 16:26:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 797982720. Throughput: 0: 10355.4, 1: 10290.0. Samples: 797951008. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:52,363][19170] Avg episode reward: [(0, '430.670'), (1, '873.660')] [2023-12-27 16:26:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001559120_399212544.pth... [2023-12-27 16:26:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001557544_398770176.pth... [2023-12-27 16:26:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001556328_398458880.pth [2023-12-27 16:26:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001557936_398909440.pth [2023-12-27 16:26:52,513][20347] Updated weights for policy 0, policy_version 1559123 (0.0006) [2023-12-27 16:26:52,569][20347] Updated weights for policy 0, policy_version 1559133 (0.0007) [2023-12-27 16:26:52,627][20347] Updated weights for policy 0, policy_version 1559143 (0.0007) [2023-12-27 16:26:52,845][20381] Updated weights for policy 1, policy_version 1557545 (0.0007) [2023-12-27 16:26:52,903][20381] Updated weights for policy 1, policy_version 1557555 (0.0005) [2023-12-27 16:26:52,957][20381] Updated weights for policy 1, policy_version 1557565 (0.0007) [2023-12-27 16:26:53,011][20381] Updated weights for policy 1, policy_version 1557575 (0.0008) [2023-12-27 16:26:53,349][20347] Updated weights for policy 0, policy_version 1559153 (0.0007) [2023-12-27 16:26:53,398][20347] Updated weights for policy 0, policy_version 1559163 (0.0007) [2023-12-27 16:26:53,445][20347] Updated weights for policy 0, policy_version 1559173 (0.0007) [2023-12-27 16:26:53,489][20347] Updated weights for policy 0, policy_version 1559183 (0.0005) [2023-12-27 16:26:53,638][20381] Updated weights for policy 1, policy_version 1557585 (0.0006) [2023-12-27 16:26:53,689][20381] Updated weights for policy 1, policy_version 1557595 (0.0006) [2023-12-27 16:26:53,739][20381] Updated weights for policy 1, policy_version 1557605 (0.0006) [2023-12-27 16:26:54,212][20347] Updated weights for policy 0, policy_version 1559193 (0.0008) [2023-12-27 16:26:54,266][20347] Updated weights for policy 0, policy_version 1559203 (0.0007) [2023-12-27 16:26:54,325][20347] Updated weights for policy 0, policy_version 1559213 (0.0006) [2023-12-27 16:26:54,334][20381] Updated weights for policy 1, policy_version 1557615 (0.0006) [2023-12-27 16:26:54,393][20381] Updated weights for policy 1, policy_version 1557625 (0.0007) [2023-12-27 16:26:54,455][20381] Updated weights for policy 1, policy_version 1557635 (0.0007) [2023-12-27 16:26:55,016][20347] Updated weights for policy 0, policy_version 1559223 (0.0009) [2023-12-27 16:26:55,040][20381] Updated weights for policy 1, policy_version 1557645 (0.0006) [2023-12-27 16:26:55,071][20347] Updated weights for policy 0, policy_version 1559233 (0.0010) [2023-12-27 16:26:55,095][20381] Updated weights for policy 1, policy_version 1557655 (0.0006) [2023-12-27 16:26:55,123][20347] Updated weights for policy 0, policy_version 1559243 (0.0010) [2023-12-27 16:26:55,156][20381] Updated weights for policy 1, policy_version 1557665 (0.0008) [2023-12-27 16:26:55,796][20347] Updated weights for policy 0, policy_version 1559253 (0.0008) [2023-12-27 16:26:55,796][20381] Updated weights for policy 1, policy_version 1557675 (0.0006) [2023-12-27 16:26:55,848][20381] Updated weights for policy 1, policy_version 1557685 (0.0007) [2023-12-27 16:26:55,857][20347] Updated weights for policy 0, policy_version 1559263 (0.0006) [2023-12-27 16:26:55,896][20381] Updated weights for policy 1, policy_version 1557695 (0.0005) [2023-12-27 16:26:55,908][20347] Updated weights for policy 0, policy_version 1559273 (0.0010) [2023-12-27 16:26:56,532][20347] Updated weights for policy 0, policy_version 1559283 (0.0007) [2023-12-27 16:26:56,580][20347] Updated weights for policy 0, policy_version 1559293 (0.0007) [2023-12-27 16:26:56,595][20381] Updated weights for policy 1, policy_version 1557705 (0.0007) [2023-12-27 16:26:56,630][20347] Updated weights for policy 0, policy_version 1559303 (0.0007) [2023-12-27 16:26:56,653][20381] Updated weights for policy 1, policy_version 1557715 (0.0009) [2023-12-27 16:26:56,708][20381] Updated weights for policy 1, policy_version 1557725 (0.0010) [2023-12-27 16:26:56,762][20381] Updated weights for policy 1, policy_version 1557735 (0.0010) [2023-12-27 16:26:57,293][20347] Updated weights for policy 0, policy_version 1559313 (0.0006) [2023-12-27 16:26:57,328][20381] Updated weights for policy 1, policy_version 1557745 (0.0007) [2023-12-27 16:26:57,349][20347] Updated weights for policy 0, policy_version 1559323 (0.0006) [2023-12-27 16:26:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 798081024. Throughput: 0: 10259.7, 1: 10324.3. Samples: 798074972. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:26:57,363][19170] Avg episode reward: [(0, '438.880'), (1, '854.950')] [2023-12-27 16:26:57,384][20381] Updated weights for policy 1, policy_version 1557755 (0.0006) [2023-12-27 16:26:57,394][20347] Updated weights for policy 0, policy_version 1559333 (0.0005) [2023-12-27 16:26:57,442][20347] Updated weights for policy 0, policy_version 1559343 (0.0006) [2023-12-27 16:26:57,444][20381] Updated weights for policy 1, policy_version 1557765 (0.0006) [2023-12-27 16:26:58,056][20347] Updated weights for policy 0, policy_version 1559353 (0.0007) [2023-12-27 16:26:58,091][20381] Updated weights for policy 1, policy_version 1557775 (0.0006) [2023-12-27 16:26:58,118][20347] Updated weights for policy 0, policy_version 1559363 (0.0010) [2023-12-27 16:26:58,145][20381] Updated weights for policy 1, policy_version 1557785 (0.0005) [2023-12-27 16:26:58,177][20347] Updated weights for policy 0, policy_version 1559373 (0.0010) [2023-12-27 16:26:58,208][20381] Updated weights for policy 1, policy_version 1557795 (0.0006) [2023-12-27 16:26:58,968][20347] Updated weights for policy 0, policy_version 1559383 (0.0007) [2023-12-27 16:26:58,996][20381] Updated weights for policy 1, policy_version 1557805 (0.0007) [2023-12-27 16:26:59,027][20347] Updated weights for policy 0, policy_version 1559393 (0.0008) [2023-12-27 16:26:59,057][20381] Updated weights for policy 1, policy_version 1557815 (0.0008) [2023-12-27 16:26:59,079][20347] Updated weights for policy 0, policy_version 1559403 (0.0008) [2023-12-27 16:26:59,120][20381] Updated weights for policy 1, policy_version 1557825 (0.0008) [2023-12-27 16:26:59,781][20347] Updated weights for policy 0, policy_version 1559413 (0.0007) [2023-12-27 16:26:59,810][20381] Updated weights for policy 1, policy_version 1557835 (0.0007) [2023-12-27 16:26:59,843][20347] Updated weights for policy 0, policy_version 1559423 (0.0008) [2023-12-27 16:26:59,874][20381] Updated weights for policy 1, policy_version 1557845 (0.0008) [2023-12-27 16:26:59,907][20347] Updated weights for policy 0, policy_version 1559433 (0.0014) [2023-12-27 16:26:59,934][20381] Updated weights for policy 1, policy_version 1557855 (0.0007) [2023-12-27 16:27:00,535][20347] Updated weights for policy 0, policy_version 1559443 (0.0009) [2023-12-27 16:27:00,594][20347] Updated weights for policy 0, policy_version 1559453 (0.0005) [2023-12-27 16:27:00,649][20347] Updated weights for policy 0, policy_version 1559463 (0.0005) [2023-12-27 16:27:00,706][20381] Updated weights for policy 1, policy_version 1557865 (0.0007) [2023-12-27 16:27:00,765][20381] Updated weights for policy 1, policy_version 1557875 (0.0008) [2023-12-27 16:27:00,817][20381] Updated weights for policy 1, policy_version 1557885 (0.0009) [2023-12-27 16:27:00,865][20381] Updated weights for policy 1, policy_version 1557895 (0.0009) [2023-12-27 16:27:01,207][20347] Updated weights for policy 0, policy_version 1559473 (0.0006) [2023-12-27 16:27:01,268][20347] Updated weights for policy 0, policy_version 1559483 (0.0007) [2023-12-27 16:27:01,324][20347] Updated weights for policy 0, policy_version 1559493 (0.0007) [2023-12-27 16:27:01,389][20347] Updated weights for policy 0, policy_version 1559503 (0.0008) [2023-12-27 16:27:01,623][20381] Updated weights for policy 1, policy_version 1557905 (0.0008) [2023-12-27 16:27:01,687][20381] Updated weights for policy 1, policy_version 1557915 (0.0009) [2023-12-27 16:27:01,748][20381] Updated weights for policy 1, policy_version 1557925 (0.0008) [2023-12-27 16:27:02,091][20347] Updated weights for policy 0, policy_version 1559513 (0.0008) [2023-12-27 16:27:02,150][20347] Updated weights for policy 0, policy_version 1559523 (0.0006) [2023-12-27 16:27:02,212][20347] Updated weights for policy 0, policy_version 1559533 (0.0005) [2023-12-27 16:27:02,335][20381] Updated weights for policy 1, policy_version 1557935 (0.0008) [2023-12-27 16:27:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 798187520. Throughput: 0: 10259.8, 1: 10248.3. Samples: 798198420. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:02,363][19170] Avg episode reward: [(0, '466.040'), (1, '795.050')] [2023-12-27 16:27:02,396][20381] Updated weights for policy 1, policy_version 1557945 (0.0009) [2023-12-27 16:27:02,449][20381] Updated weights for policy 1, policy_version 1557955 (0.0010) [2023-12-27 16:27:02,870][20347] Updated weights for policy 0, policy_version 1559543 (0.0007) [2023-12-27 16:27:02,915][20347] Updated weights for policy 0, policy_version 1559553 (0.0005) [2023-12-27 16:27:02,969][20347] Updated weights for policy 0, policy_version 1559563 (0.0005) [2023-12-27 16:27:03,126][20381] Updated weights for policy 1, policy_version 1557965 (0.0010) [2023-12-27 16:27:03,182][20381] Updated weights for policy 1, policy_version 1557975 (0.0010) [2023-12-27 16:27:03,240][20381] Updated weights for policy 1, policy_version 1557985 (0.0010) [2023-12-27 16:27:03,566][20347] Updated weights for policy 0, policy_version 1559573 (0.0006) [2023-12-27 16:27:03,621][20347] Updated weights for policy 0, policy_version 1559583 (0.0007) [2023-12-27 16:27:03,673][20347] Updated weights for policy 0, policy_version 1559593 (0.0008) [2023-12-27 16:27:03,914][20381] Updated weights for policy 1, policy_version 1557995 (0.0010) [2023-12-27 16:27:03,975][20381] Updated weights for policy 1, policy_version 1558005 (0.0006) [2023-12-27 16:27:04,033][20381] Updated weights for policy 1, policy_version 1558015 (0.0011) [2023-12-27 16:27:04,386][20347] Updated weights for policy 0, policy_version 1559603 (0.0007) [2023-12-27 16:27:04,442][20347] Updated weights for policy 0, policy_version 1559613 (0.0009) [2023-12-27 16:27:04,501][20347] Updated weights for policy 0, policy_version 1559623 (0.0007) [2023-12-27 16:27:04,711][20381] Updated weights for policy 1, policy_version 1558025 (0.0010) [2023-12-27 16:27:04,759][20381] Updated weights for policy 1, policy_version 1558035 (0.0007) [2023-12-27 16:27:04,813][20381] Updated weights for policy 1, policy_version 1558045 (0.0007) [2023-12-27 16:27:04,875][20381] Updated weights for policy 1, policy_version 1558055 (0.0007) [2023-12-27 16:27:05,161][20347] Updated weights for policy 0, policy_version 1559633 (0.0008) [2023-12-27 16:27:05,209][20347] Updated weights for policy 0, policy_version 1559643 (0.0010) [2023-12-27 16:27:05,266][20347] Updated weights for policy 0, policy_version 1559653 (0.0010) [2023-12-27 16:27:05,328][20347] Updated weights for policy 0, policy_version 1559663 (0.0010) [2023-12-27 16:27:05,514][20381] Updated weights for policy 1, policy_version 1558065 (0.0006) [2023-12-27 16:27:05,562][20381] Updated weights for policy 1, policy_version 1558075 (0.0005) [2023-12-27 16:27:05,621][20381] Updated weights for policy 1, policy_version 1558085 (0.0005) [2023-12-27 16:27:06,004][20347] Updated weights for policy 0, policy_version 1559673 (0.0010) [2023-12-27 16:27:06,066][20347] Updated weights for policy 0, policy_version 1559683 (0.0010) [2023-12-27 16:27:06,129][20347] Updated weights for policy 0, policy_version 1559693 (0.0010) [2023-12-27 16:27:06,197][20381] Updated weights for policy 1, policy_version 1558095 (0.0006) [2023-12-27 16:27:06,260][20381] Updated weights for policy 1, policy_version 1558105 (0.0008) [2023-12-27 16:27:06,266][20292] KL-divergence is very high: 100.4726 [2023-12-27 16:27:06,321][20381] Updated weights for policy 1, policy_version 1558115 (0.0008) [2023-12-27 16:27:06,827][20347] Updated weights for policy 0, policy_version 1559703 (0.0010) [2023-12-27 16:27:06,892][20347] Updated weights for policy 0, policy_version 1559713 (0.0011) [2023-12-27 16:27:06,947][20347] Updated weights for policy 0, policy_version 1559723 (0.0010) [2023-12-27 16:27:07,014][20381] Updated weights for policy 1, policy_version 1558125 (0.0008) [2023-12-27 16:27:07,071][20381] Updated weights for policy 1, policy_version 1558135 (0.0007) [2023-12-27 16:27:07,126][20381] Updated weights for policy 1, policy_version 1558145 (0.0007) [2023-12-27 16:27:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 798294016. Throughput: 0: 10256.9, 1: 10244.2. Samples: 798260132. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:07,363][19170] Avg episode reward: [(0, '458.300'), (1, '765.640')] [2023-12-27 16:27:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001558152_398925824.pth... [2023-12-27 16:27:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001559728_399368192.pth... [2023-12-27 16:27:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001556968_398622720.pth [2023-12-27 16:27:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001558544_399065088.pth [2023-12-27 16:27:07,627][20347] Updated weights for policy 0, policy_version 1559733 (0.0011) [2023-12-27 16:27:07,679][20347] Updated weights for policy 0, policy_version 1559743 (0.0010) [2023-12-27 16:27:07,742][20347] Updated weights for policy 0, policy_version 1559753 (0.0010) [2023-12-27 16:27:07,820][20381] Updated weights for policy 1, policy_version 1558155 (0.0008) [2023-12-27 16:27:07,870][20381] Updated weights for policy 1, policy_version 1558165 (0.0011) [2023-12-27 16:27:07,923][20381] Updated weights for policy 1, policy_version 1558175 (0.0011) [2023-12-27 16:27:08,413][20347] Updated weights for policy 0, policy_version 1559763 (0.0009) [2023-12-27 16:27:08,478][20347] Updated weights for policy 0, policy_version 1559773 (0.0008) [2023-12-27 16:27:08,546][20347] Updated weights for policy 0, policy_version 1559783 (0.0011) [2023-12-27 16:27:08,573][20381] Updated weights for policy 1, policy_version 1558185 (0.0010) [2023-12-27 16:27:08,624][20381] Updated weights for policy 1, policy_version 1558195 (0.0008) [2023-12-27 16:27:08,674][20381] Updated weights for policy 1, policy_version 1558205 (0.0008) [2023-12-27 16:27:08,723][20381] Updated weights for policy 1, policy_version 1558215 (0.0007) [2023-12-27 16:27:09,192][20347] Updated weights for policy 0, policy_version 1559793 (0.0009) [2023-12-27 16:27:09,255][20347] Updated weights for policy 0, policy_version 1559803 (0.0009) [2023-12-27 16:27:09,320][20347] Updated weights for policy 0, policy_version 1559813 (0.0008) [2023-12-27 16:27:09,379][20347] Updated weights for policy 0, policy_version 1559823 (0.0008) [2023-12-27 16:27:09,476][20381] Updated weights for policy 1, policy_version 1558225 (0.0007) [2023-12-27 16:27:09,535][20381] Updated weights for policy 1, policy_version 1558235 (0.0009) [2023-12-27 16:27:09,590][20381] Updated weights for policy 1, policy_version 1558245 (0.0007) [2023-12-27 16:27:10,075][20347] Updated weights for policy 0, policy_version 1559833 (0.0008) [2023-12-27 16:27:10,138][20347] Updated weights for policy 0, policy_version 1559843 (0.0009) [2023-12-27 16:27:10,198][20347] Updated weights for policy 0, policy_version 1559853 (0.0008) [2023-12-27 16:27:10,349][20381] Updated weights for policy 1, policy_version 1558255 (0.0010) [2023-12-27 16:27:10,398][20381] Updated weights for policy 1, policy_version 1558265 (0.0011) [2023-12-27 16:27:10,451][20381] Updated weights for policy 1, policy_version 1558275 (0.0011) [2023-12-27 16:27:10,812][20347] Updated weights for policy 0, policy_version 1559863 (0.0006) [2023-12-27 16:27:10,871][20347] Updated weights for policy 0, policy_version 1559873 (0.0007) [2023-12-27 16:27:10,936][20347] Updated weights for policy 0, policy_version 1559883 (0.0007) [2023-12-27 16:27:11,167][20381] Updated weights for policy 1, policy_version 1558285 (0.0009) [2023-12-27 16:27:11,233][20381] Updated weights for policy 1, policy_version 1558295 (0.0007) [2023-12-27 16:27:11,298][20381] Updated weights for policy 1, policy_version 1558305 (0.0008) [2023-12-27 16:27:11,664][20347] Updated weights for policy 0, policy_version 1559893 (0.0007) [2023-12-27 16:27:11,722][20347] Updated weights for policy 0, policy_version 1559903 (0.0006) [2023-12-27 16:27:11,789][20347] Updated weights for policy 0, policy_version 1559913 (0.0008) [2023-12-27 16:27:11,977][20381] Updated weights for policy 1, policy_version 1558315 (0.0007) [2023-12-27 16:27:12,037][20381] Updated weights for policy 1, policy_version 1558325 (0.0008) [2023-12-27 16:27:12,098][20381] Updated weights for policy 1, policy_version 1558335 (0.0008) [2023-12-27 16:27:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 798392320. Throughput: 0: 10227.7, 1: 10277.9. Samples: 798382684. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:12,363][19170] Avg episode reward: [(0, '525.840'), (1, '794.850')] [2023-12-27 16:27:12,515][20347] Updated weights for policy 0, policy_version 1559923 (0.0008) [2023-12-27 16:27:12,580][20347] Updated weights for policy 0, policy_version 1559933 (0.0008) [2023-12-27 16:27:12,640][20347] Updated weights for policy 0, policy_version 1559943 (0.0008) [2023-12-27 16:27:12,776][20381] Updated weights for policy 1, policy_version 1558345 (0.0007) [2023-12-27 16:27:12,830][20381] Updated weights for policy 1, policy_version 1558355 (0.0008) [2023-12-27 16:27:12,889][20381] Updated weights for policy 1, policy_version 1558365 (0.0008) [2023-12-27 16:27:12,944][20381] Updated weights for policy 1, policy_version 1558375 (0.0008) [2023-12-27 16:27:13,311][20347] Updated weights for policy 0, policy_version 1559953 (0.0008) [2023-12-27 16:27:13,360][20347] Updated weights for policy 0, policy_version 1559963 (0.0007) [2023-12-27 16:27:13,409][20347] Updated weights for policy 0, policy_version 1559973 (0.0007) [2023-12-27 16:27:13,466][20347] Updated weights for policy 0, policy_version 1559983 (0.0008) [2023-12-27 16:27:13,682][20381] Updated weights for policy 1, policy_version 1558385 (0.0007) [2023-12-27 16:27:13,750][20381] Updated weights for policy 1, policy_version 1558395 (0.0008) [2023-12-27 16:27:13,818][20381] Updated weights for policy 1, policy_version 1558405 (0.0008) [2023-12-27 16:27:14,079][20347] Updated weights for policy 0, policy_version 1559993 (0.0008) [2023-12-27 16:27:14,129][20347] Updated weights for policy 0, policy_version 1560003 (0.0005) [2023-12-27 16:27:14,186][20347] Updated weights for policy 0, policy_version 1560013 (0.0005) [2023-12-27 16:27:14,526][20381] Updated weights for policy 1, policy_version 1558415 (0.0008) [2023-12-27 16:27:14,574][20381] Updated weights for policy 1, policy_version 1558425 (0.0007) [2023-12-27 16:27:14,619][20381] Updated weights for policy 1, policy_version 1558435 (0.0007) [2023-12-27 16:27:14,812][20347] Updated weights for policy 0, policy_version 1560023 (0.0007) [2023-12-27 16:27:14,875][20347] Updated weights for policy 0, policy_version 1560033 (0.0006) [2023-12-27 16:27:14,940][20347] Updated weights for policy 0, policy_version 1560043 (0.0006) [2023-12-27 16:27:15,312][20381] Updated weights for policy 1, policy_version 1558445 (0.0006) [2023-12-27 16:27:15,373][20381] Updated weights for policy 1, policy_version 1558455 (0.0005) [2023-12-27 16:27:15,426][20381] Updated weights for policy 1, policy_version 1558465 (0.0005) [2023-12-27 16:27:15,517][20347] Updated weights for policy 0, policy_version 1560053 (0.0005) [2023-12-27 16:27:15,573][20347] Updated weights for policy 0, policy_version 1560063 (0.0005) [2023-12-27 16:27:15,628][20347] Updated weights for policy 0, policy_version 1560073 (0.0005) [2023-12-27 16:27:16,078][20381] Updated weights for policy 1, policy_version 1558475 (0.0009) [2023-12-27 16:27:16,130][20381] Updated weights for policy 1, policy_version 1558485 (0.0008) [2023-12-27 16:27:16,133][20347] Updated weights for policy 0, policy_version 1560083 (0.0005) [2023-12-27 16:27:16,184][20381] Updated weights for policy 1, policy_version 1558495 (0.0007) [2023-12-27 16:27:16,187][20347] Updated weights for policy 0, policy_version 1560093 (0.0005) [2023-12-27 16:27:16,240][20347] Updated weights for policy 0, policy_version 1560103 (0.0008) [2023-12-27 16:27:16,761][20347] Updated weights for policy 0, policy_version 1560113 (0.0006) [2023-12-27 16:27:16,824][20347] Updated weights for policy 0, policy_version 1560123 (0.0010) [2023-12-27 16:27:16,873][20381] Updated weights for policy 1, policy_version 1558505 (0.0008) [2023-12-27 16:27:16,883][20347] Updated weights for policy 0, policy_version 1560133 (0.0010) [2023-12-27 16:27:16,938][20381] Updated weights for policy 1, policy_version 1558515 (0.0010) [2023-12-27 16:27:16,938][20347] Updated weights for policy 0, policy_version 1560143 (0.0010) [2023-12-27 16:27:17,001][20381] Updated weights for policy 1, policy_version 1558525 (0.0010) [2023-12-27 16:27:17,056][20381] Updated weights for policy 1, policy_version 1558535 (0.0010) [2023-12-27 16:27:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 798498816. Throughput: 0: 10312.1, 1: 10226.9. Samples: 798507944. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:17,363][19170] Avg episode reward: [(0, '505.590'), (1, '834.590')] [2023-12-27 16:27:17,587][20347] Updated weights for policy 0, policy_version 1560153 (0.0007) [2023-12-27 16:27:17,641][20347] Updated weights for policy 0, policy_version 1560163 (0.0010) [2023-12-27 16:27:17,686][20347] Updated weights for policy 0, policy_version 1560173 (0.0010) [2023-12-27 16:27:17,723][20381] Updated weights for policy 1, policy_version 1558545 (0.0010) [2023-12-27 16:27:17,780][20381] Updated weights for policy 1, policy_version 1558555 (0.0010) [2023-12-27 16:27:17,830][20381] Updated weights for policy 1, policy_version 1558565 (0.0010) [2023-12-27 16:27:18,360][20347] Updated weights for policy 0, policy_version 1560183 (0.0009) [2023-12-27 16:27:18,421][20347] Updated weights for policy 0, policy_version 1560193 (0.0008) [2023-12-27 16:27:18,455][20381] Updated weights for policy 1, policy_version 1558575 (0.0009) [2023-12-27 16:27:18,477][20347] Updated weights for policy 0, policy_version 1560203 (0.0008) [2023-12-27 16:27:18,514][20381] Updated weights for policy 1, policy_version 1558585 (0.0007) [2023-12-27 16:27:18,574][20381] Updated weights for policy 1, policy_version 1558595 (0.0007) [2023-12-27 16:27:19,112][20347] Updated weights for policy 0, policy_version 1560213 (0.0006) [2023-12-27 16:27:19,169][20347] Updated weights for policy 0, policy_version 1560223 (0.0005) [2023-12-27 16:27:19,192][20381] Updated weights for policy 1, policy_version 1558605 (0.0007) [2023-12-27 16:27:19,222][20347] Updated weights for policy 0, policy_version 1560233 (0.0009) [2023-12-27 16:27:19,252][20381] Updated weights for policy 1, policy_version 1558615 (0.0007) [2023-12-27 16:27:19,309][20381] Updated weights for policy 1, policy_version 1558625 (0.0008) [2023-12-27 16:27:19,927][20347] Updated weights for policy 0, policy_version 1560243 (0.0008) [2023-12-27 16:27:19,980][20347] Updated weights for policy 0, policy_version 1560253 (0.0005) [2023-12-27 16:27:20,035][20347] Updated weights for policy 0, policy_version 1560263 (0.0006) [2023-12-27 16:27:20,043][20381] Updated weights for policy 1, policy_version 1558635 (0.0008) [2023-12-27 16:27:20,099][20381] Updated weights for policy 1, policy_version 1558645 (0.0008) [2023-12-27 16:27:20,151][20381] Updated weights for policy 1, policy_version 1558655 (0.0008) [2023-12-27 16:27:20,710][20347] Updated weights for policy 0, policy_version 1560273 (0.0008) [2023-12-27 16:27:20,763][20347] Updated weights for policy 0, policy_version 1560283 (0.0010) [2023-12-27 16:27:20,824][20347] Updated weights for policy 0, policy_version 1560293 (0.0007) [2023-12-27 16:27:20,887][20347] Updated weights for policy 0, policy_version 1560303 (0.0008) [2023-12-27 16:27:20,960][20381] Updated weights for policy 1, policy_version 1558665 (0.0008) [2023-12-27 16:27:21,028][20381] Updated weights for policy 1, policy_version 1558675 (0.0006) [2023-12-27 16:27:21,089][20381] Updated weights for policy 1, policy_version 1558685 (0.0008) [2023-12-27 16:27:21,150][20381] Updated weights for policy 1, policy_version 1558695 (0.0008) [2023-12-27 16:27:21,609][20347] Updated weights for policy 0, policy_version 1560313 (0.0008) [2023-12-27 16:27:21,676][20347] Updated weights for policy 0, policy_version 1560323 (0.0008) [2023-12-27 16:27:21,735][20347] Updated weights for policy 0, policy_version 1560333 (0.0009) [2023-12-27 16:27:21,884][20381] Updated weights for policy 1, policy_version 1558705 (0.0008) [2023-12-27 16:27:21,943][20381] Updated weights for policy 1, policy_version 1558715 (0.0008) [2023-12-27 16:27:22,002][20381] Updated weights for policy 1, policy_version 1558725 (0.0007) [2023-12-27 16:27:22,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 798597120. Throughput: 0: 10346.4, 1: 10182.5. Samples: 798570136. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:22,363][19170] Avg episode reward: [(0, '479.670'), (1, '854.170')] [2023-12-27 16:27:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001558728_399073280.pth... [2023-12-27 16:27:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001560336_399523840.pth... [2023-12-27 16:27:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001559120_399212544.pth [2023-12-27 16:27:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001557544_398770176.pth [2023-12-27 16:27:22,426][20347] Updated weights for policy 0, policy_version 1560343 (0.0010) [2023-12-27 16:27:22,483][20347] Updated weights for policy 0, policy_version 1560353 (0.0006) [2023-12-27 16:27:22,558][20347] Updated weights for policy 0, policy_version 1560364 (0.0007) [2023-12-27 16:27:22,692][20381] Updated weights for policy 1, policy_version 1558735 (0.0007) [2023-12-27 16:27:22,751][20381] Updated weights for policy 1, policy_version 1558745 (0.0007) [2023-12-27 16:27:22,812][20381] Updated weights for policy 1, policy_version 1558755 (0.0007) [2023-12-27 16:27:23,214][20347] Updated weights for policy 0, policy_version 1560374 (0.0008) [2023-12-27 16:27:23,267][20347] Updated weights for policy 0, policy_version 1560384 (0.0010) [2023-12-27 16:27:23,322][20347] Updated weights for policy 0, policy_version 1560394 (0.0010) [2023-12-27 16:27:23,451][20381] Updated weights for policy 1, policy_version 1558765 (0.0007) [2023-12-27 16:27:23,505][20381] Updated weights for policy 1, policy_version 1558775 (0.0007) [2023-12-27 16:27:23,563][20381] Updated weights for policy 1, policy_version 1558785 (0.0008) [2023-12-27 16:27:23,935][20347] Updated weights for policy 0, policy_version 1560404 (0.0008) [2023-12-27 16:27:23,987][20347] Updated weights for policy 0, policy_version 1560414 (0.0010) [2023-12-27 16:27:24,044][20347] Updated weights for policy 0, policy_version 1560424 (0.0010) [2023-12-27 16:27:24,206][20381] Updated weights for policy 1, policy_version 1558795 (0.0008) [2023-12-27 16:27:24,265][20381] Updated weights for policy 1, policy_version 1558805 (0.0007) [2023-12-27 16:27:24,321][20381] Updated weights for policy 1, policy_version 1558815 (0.0008) [2023-12-27 16:27:24,692][20347] Updated weights for policy 0, policy_version 1560434 (0.0010) [2023-12-27 16:27:24,751][20347] Updated weights for policy 0, policy_version 1560444 (0.0008) [2023-12-27 16:27:24,800][20347] Updated weights for policy 0, policy_version 1560454 (0.0008) [2023-12-27 16:27:24,856][20347] Updated weights for policy 0, policy_version 1560464 (0.0007) [2023-12-27 16:27:24,940][20381] Updated weights for policy 1, policy_version 1558825 (0.0007) [2023-12-27 16:27:24,995][20381] Updated weights for policy 1, policy_version 1558835 (0.0008) [2023-12-27 16:27:25,046][20381] Updated weights for policy 1, policy_version 1558845 (0.0007) [2023-12-27 16:27:25,095][20381] Updated weights for policy 1, policy_version 1558855 (0.0008) [2023-12-27 16:27:25,508][20347] Updated weights for policy 0, policy_version 1560474 (0.0007) [2023-12-27 16:27:25,557][20347] Updated weights for policy 0, policy_version 1560484 (0.0008) [2023-12-27 16:27:25,605][20347] Updated weights for policy 0, policy_version 1560494 (0.0007) [2023-12-27 16:27:25,814][20381] Updated weights for policy 1, policy_version 1558865 (0.0007) [2023-12-27 16:27:25,875][20381] Updated weights for policy 1, policy_version 1558875 (0.0008) [2023-12-27 16:27:25,932][20381] Updated weights for policy 1, policy_version 1558885 (0.0008) [2023-12-27 16:27:26,324][20347] Updated weights for policy 0, policy_version 1560504 (0.0008) [2023-12-27 16:27:26,378][20347] Updated weights for policy 0, policy_version 1560514 (0.0008) [2023-12-27 16:27:26,435][20347] Updated weights for policy 0, policy_version 1560524 (0.0007) [2023-12-27 16:27:26,598][20381] Updated weights for policy 1, policy_version 1558895 (0.0009) [2023-12-27 16:27:26,659][20381] Updated weights for policy 1, policy_version 1558905 (0.0006) [2023-12-27 16:27:26,718][20381] Updated weights for policy 1, policy_version 1558915 (0.0008) [2023-12-27 16:27:27,040][20347] Updated weights for policy 0, policy_version 1560534 (0.0006) [2023-12-27 16:27:27,087][20347] Updated weights for policy 0, policy_version 1560544 (0.0005) [2023-12-27 16:27:27,136][20347] Updated weights for policy 0, policy_version 1560554 (0.0005) [2023-12-27 16:27:27,334][20381] Updated weights for policy 1, policy_version 1558925 (0.0007) [2023-12-27 16:27:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 798703616. Throughput: 0: 10348.4, 1: 10189.5. Samples: 798693684. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:27,363][19170] Avg episode reward: [(0, '612.950'), (1, '873.060')] [2023-12-27 16:27:27,396][20381] Updated weights for policy 1, policy_version 1558935 (0.0007) [2023-12-27 16:27:27,450][20381] Updated weights for policy 1, policy_version 1558945 (0.0007) [2023-12-27 16:27:27,783][20347] Updated weights for policy 0, policy_version 1560564 (0.0010) [2023-12-27 16:27:27,833][20347] Updated weights for policy 0, policy_version 1560574 (0.0007) [2023-12-27 16:27:27,893][20347] Updated weights for policy 0, policy_version 1560584 (0.0008) [2023-12-27 16:27:28,151][20381] Updated weights for policy 1, policy_version 1558955 (0.0007) [2023-12-27 16:27:28,206][20381] Updated weights for policy 1, policy_version 1558965 (0.0008) [2023-12-27 16:27:28,268][20381] Updated weights for policy 1, policy_version 1558975 (0.0007) [2023-12-27 16:27:28,460][20347] Updated weights for policy 0, policy_version 1560594 (0.0007) [2023-12-27 16:27:28,523][20347] Updated weights for policy 0, policy_version 1560604 (0.0010) [2023-12-27 16:27:28,578][20347] Updated weights for policy 0, policy_version 1560614 (0.0010) [2023-12-27 16:27:28,634][20347] Updated weights for policy 0, policy_version 1560624 (0.0011) [2023-12-27 16:27:28,972][20381] Updated weights for policy 1, policy_version 1558985 (0.0008) [2023-12-27 16:27:29,018][20381] Updated weights for policy 1, policy_version 1558995 (0.0008) [2023-12-27 16:27:29,070][20381] Updated weights for policy 1, policy_version 1559005 (0.0008) [2023-12-27 16:27:29,123][20381] Updated weights for policy 1, policy_version 1559015 (0.0008) [2023-12-27 16:27:29,292][20347] Updated weights for policy 0, policy_version 1560634 (0.0010) [2023-12-27 16:27:29,356][20347] Updated weights for policy 0, policy_version 1560644 (0.0010) [2023-12-27 16:27:29,419][20347] Updated weights for policy 0, policy_version 1560654 (0.0009) [2023-12-27 16:27:29,827][20381] Updated weights for policy 1, policy_version 1559025 (0.0007) [2023-12-27 16:27:29,889][20381] Updated weights for policy 1, policy_version 1559035 (0.0007) [2023-12-27 16:27:29,957][20381] Updated weights for policy 1, policy_version 1559045 (0.0010) [2023-12-27 16:27:30,094][20347] Updated weights for policy 0, policy_version 1560664 (0.0009) [2023-12-27 16:27:30,155][20347] Updated weights for policy 0, policy_version 1560674 (0.0008) [2023-12-27 16:27:30,214][20347] Updated weights for policy 0, policy_version 1560684 (0.0011) [2023-12-27 16:27:30,588][20381] Updated weights for policy 1, policy_version 1559055 (0.0008) [2023-12-27 16:27:30,639][20381] Updated weights for policy 1, policy_version 1559065 (0.0009) [2023-12-27 16:27:30,695][20381] Updated weights for policy 1, policy_version 1559075 (0.0008) [2023-12-27 16:27:30,806][20347] Updated weights for policy 0, policy_version 1560694 (0.0007) [2023-12-27 16:27:30,862][20347] Updated weights for policy 0, policy_version 1560704 (0.0005) [2023-12-27 16:27:30,916][20347] Updated weights for policy 0, policy_version 1560714 (0.0009) [2023-12-27 16:27:31,424][20381] Updated weights for policy 1, policy_version 1559085 (0.0007) [2023-12-27 16:27:31,490][20381] Updated weights for policy 1, policy_version 1559095 (0.0008) [2023-12-27 16:27:31,549][20381] Updated weights for policy 1, policy_version 1559105 (0.0008) [2023-12-27 16:27:31,580][20347] Updated weights for policy 0, policy_version 1560724 (0.0008) [2023-12-27 16:27:31,645][20347] Updated weights for policy 0, policy_version 1560734 (0.0007) [2023-12-27 16:27:31,710][20347] Updated weights for policy 0, policy_version 1560744 (0.0007) [2023-12-27 16:27:32,222][20381] Updated weights for policy 1, policy_version 1559115 (0.0008) [2023-12-27 16:27:32,286][20381] Updated weights for policy 1, policy_version 1559125 (0.0011) [2023-12-27 16:27:32,340][20381] Updated weights for policy 1, policy_version 1559135 (0.0011) [2023-12-27 16:27:32,346][20347] Updated weights for policy 0, policy_version 1560754 (0.0008) [2023-12-27 16:27:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 798801920. Throughput: 0: 10450.6, 1: 10200.0. Samples: 798819280. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:32,363][19170] Avg episode reward: [(0, '632.310'), (1, '873.840')] [2023-12-27 16:27:32,412][20347] Updated weights for policy 0, policy_version 1560764 (0.0009) [2023-12-27 16:27:32,467][20347] Updated weights for policy 0, policy_version 1560774 (0.0008) [2023-12-27 16:27:32,514][20347] Updated weights for policy 0, policy_version 1560784 (0.0007) [2023-12-27 16:27:32,992][20381] Updated weights for policy 1, policy_version 1559145 (0.0010) [2023-12-27 16:27:33,046][20381] Updated weights for policy 1, policy_version 1559155 (0.0010) [2023-12-27 16:27:33,101][20381] Updated weights for policy 1, policy_version 1559165 (0.0010) [2023-12-27 16:27:33,157][20381] Updated weights for policy 1, policy_version 1559175 (0.0010) [2023-12-27 16:27:33,172][20347] Updated weights for policy 0, policy_version 1560794 (0.0008) [2023-12-27 16:27:33,224][20347] Updated weights for policy 0, policy_version 1560804 (0.0007) [2023-12-27 16:27:33,283][20347] Updated weights for policy 0, policy_version 1560814 (0.0007) [2023-12-27 16:27:33,824][20381] Updated weights for policy 1, policy_version 1559185 (0.0010) [2023-12-27 16:27:33,871][20381] Updated weights for policy 1, policy_version 1559195 (0.0010) [2023-12-27 16:27:33,913][20381] Updated weights for policy 1, policy_version 1559205 (0.0006) [2023-12-27 16:27:33,958][20347] Updated weights for policy 0, policy_version 1560824 (0.0007) [2023-12-27 16:27:34,012][20347] Updated weights for policy 0, policy_version 1560834 (0.0007) [2023-12-27 16:27:34,066][20347] Updated weights for policy 0, policy_version 1560844 (0.0008) [2023-12-27 16:27:34,567][20381] Updated weights for policy 1, policy_version 1559215 (0.0009) [2023-12-27 16:27:34,630][20381] Updated weights for policy 1, policy_version 1559225 (0.0008) [2023-12-27 16:27:34,687][20381] Updated weights for policy 1, policy_version 1559235 (0.0007) [2023-12-27 16:27:34,800][20347] Updated weights for policy 0, policy_version 1560854 (0.0008) [2023-12-27 16:27:34,867][20347] Updated weights for policy 0, policy_version 1560864 (0.0008) [2023-12-27 16:27:34,922][20347] Updated weights for policy 0, policy_version 1560874 (0.0009) [2023-12-27 16:27:35,397][20381] Updated weights for policy 1, policy_version 1559245 (0.0008) [2023-12-27 16:27:35,446][20381] Updated weights for policy 1, policy_version 1559255 (0.0008) [2023-12-27 16:27:35,496][20381] Updated weights for policy 1, policy_version 1559265 (0.0008) [2023-12-27 16:27:35,544][20347] Updated weights for policy 0, policy_version 1560884 (0.0007) [2023-12-27 16:27:35,598][20347] Updated weights for policy 0, policy_version 1560894 (0.0008) [2023-12-27 16:27:35,648][20347] Updated weights for policy 0, policy_version 1560904 (0.0007) [2023-12-27 16:27:36,122][20381] Updated weights for policy 1, policy_version 1559275 (0.0007) [2023-12-27 16:27:36,183][20381] Updated weights for policy 1, policy_version 1559285 (0.0008) [2023-12-27 16:27:36,242][20381] Updated weights for policy 1, policy_version 1559295 (0.0008) [2023-12-27 16:27:36,389][20347] Updated weights for policy 0, policy_version 1560914 (0.0007) [2023-12-27 16:27:36,453][20347] Updated weights for policy 0, policy_version 1560924 (0.0008) [2023-12-27 16:27:36,512][20347] Updated weights for policy 0, policy_version 1560934 (0.0007) [2023-12-27 16:27:36,568][20347] Updated weights for policy 0, policy_version 1560944 (0.0007) [2023-12-27 16:27:36,924][20381] Updated weights for policy 1, policy_version 1559305 (0.0007) [2023-12-27 16:27:36,981][20381] Updated weights for policy 1, policy_version 1559315 (0.0005) [2023-12-27 16:27:37,036][20381] Updated weights for policy 1, policy_version 1559325 (0.0005) [2023-12-27 16:27:37,092][20381] Updated weights for policy 1, policy_version 1559335 (0.0005) [2023-12-27 16:27:37,310][20347] Updated weights for policy 0, policy_version 1560954 (0.0008) [2023-12-27 16:27:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 798908416. Throughput: 0: 10443.3, 1: 10218.8. Samples: 798880808. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:37,363][19170] Avg episode reward: [(0, '584.000'), (1, '903.330')] [2023-12-27 16:27:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001559336_399228928.pth... [2023-12-27 16:27:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001558152_398925824.pth [2023-12-27 16:27:37,377][20347] Updated weights for policy 0, policy_version 1560964 (0.0006) [2023-12-27 16:27:37,435][20347] Updated weights for policy 0, policy_version 1560974 (0.0005) [2023-12-27 16:27:37,447][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001560976_399687680.pth... [2023-12-27 16:27:37,451][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001559728_399368192.pth [2023-12-27 16:27:37,655][20381] Updated weights for policy 1, policy_version 1559345 (0.0006) [2023-12-27 16:27:37,712][20381] Updated weights for policy 1, policy_version 1559355 (0.0006) [2023-12-27 16:27:37,773][20381] Updated weights for policy 1, policy_version 1559365 (0.0006) [2023-12-27 16:27:38,061][20347] Updated weights for policy 0, policy_version 1560984 (0.0008) [2023-12-27 16:27:38,121][20347] Updated weights for policy 0, policy_version 1560994 (0.0008) [2023-12-27 16:27:38,172][20347] Updated weights for policy 0, policy_version 1561004 (0.0008) [2023-12-27 16:27:38,428][20381] Updated weights for policy 1, policy_version 1559375 (0.0008) [2023-12-27 16:27:38,481][20381] Updated weights for policy 1, policy_version 1559385 (0.0008) [2023-12-27 16:27:38,534][20381] Updated weights for policy 1, policy_version 1559395 (0.0006) [2023-12-27 16:27:38,883][20347] Updated weights for policy 0, policy_version 1561014 (0.0008) [2023-12-27 16:27:38,939][20347] Updated weights for policy 0, policy_version 1561024 (0.0007) [2023-12-27 16:27:38,996][20347] Updated weights for policy 0, policy_version 1561034 (0.0008) [2023-12-27 16:27:39,206][20381] Updated weights for policy 1, policy_version 1559405 (0.0008) [2023-12-27 16:27:39,274][20381] Updated weights for policy 1, policy_version 1559415 (0.0008) [2023-12-27 16:27:39,333][20381] Updated weights for policy 1, policy_version 1559425 (0.0007) [2023-12-27 16:27:39,698][20347] Updated weights for policy 0, policy_version 1561044 (0.0006) [2023-12-27 16:27:39,762][20347] Updated weights for policy 0, policy_version 1561054 (0.0007) [2023-12-27 16:27:39,774][20291] KL-divergence is very high: 103.4192 [2023-12-27 16:27:39,823][20347] Updated weights for policy 0, policy_version 1561064 (0.0008) [2023-12-27 16:27:40,065][20381] Updated weights for policy 1, policy_version 1559435 (0.0007) [2023-12-27 16:27:40,127][20381] Updated weights for policy 1, policy_version 1559445 (0.0006) [2023-12-27 16:27:40,188][20381] Updated weights for policy 1, policy_version 1559455 (0.0006) [2023-12-27 16:27:40,580][20347] Updated weights for policy 0, policy_version 1561074 (0.0007) [2023-12-27 16:27:40,636][20347] Updated weights for policy 0, policy_version 1561084 (0.0008) [2023-12-27 16:27:40,685][20347] Updated weights for policy 0, policy_version 1561094 (0.0006) [2023-12-27 16:27:40,740][20347] Updated weights for policy 0, policy_version 1561104 (0.0006) [2023-12-27 16:27:40,841][20381] Updated weights for policy 1, policy_version 1559465 (0.0006) [2023-12-27 16:27:40,921][20381] Updated weights for policy 1, policy_version 1559475 (0.0008) [2023-12-27 16:27:40,972][20381] Updated weights for policy 1, policy_version 1559485 (0.0007) [2023-12-27 16:27:41,020][20381] Updated weights for policy 1, policy_version 1559495 (0.0009) [2023-12-27 16:27:41,401][20347] Updated weights for policy 0, policy_version 1561114 (0.0009) [2023-12-27 16:27:41,458][20347] Updated weights for policy 0, policy_version 1561124 (0.0008) [2023-12-27 16:27:41,519][20347] Updated weights for policy 0, policy_version 1561134 (0.0007) [2023-12-27 16:27:41,787][20381] Updated weights for policy 1, policy_version 1559505 (0.0008) [2023-12-27 16:27:41,849][20381] Updated weights for policy 1, policy_version 1559515 (0.0009) [2023-12-27 16:27:41,909][20381] Updated weights for policy 1, policy_version 1559525 (0.0008) [2023-12-27 16:27:42,255][20347] Updated weights for policy 0, policy_version 1561144 (0.0007) [2023-12-27 16:27:42,322][20347] Updated weights for policy 0, policy_version 1561154 (0.0009) [2023-12-27 16:27:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 799006720. Throughput: 0: 10433.2, 1: 10185.6. Samples: 799002820. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:42,363][19170] Avg episode reward: [(0, '680.600'), (1, '893.260')] [2023-12-27 16:27:42,370][20347] Updated weights for policy 0, policy_version 1561164 (0.0007) [2023-12-27 16:27:42,660][20381] Updated weights for policy 1, policy_version 1559535 (0.0010) [2023-12-27 16:27:42,720][20381] Updated weights for policy 1, policy_version 1559545 (0.0009) [2023-12-27 16:27:42,778][20381] Updated weights for policy 1, policy_version 1559555 (0.0011) [2023-12-27 16:27:43,059][20347] Updated weights for policy 0, policy_version 1561174 (0.0007) [2023-12-27 16:27:43,109][20347] Updated weights for policy 0, policy_version 1561184 (0.0007) [2023-12-27 16:27:43,167][20347] Updated weights for policy 0, policy_version 1561194 (0.0008) [2023-12-27 16:27:43,435][20381] Updated weights for policy 1, policy_version 1559565 (0.0011) [2023-12-27 16:27:43,488][20381] Updated weights for policy 1, policy_version 1559575 (0.0011) [2023-12-27 16:27:43,536][20381] Updated weights for policy 1, policy_version 1559585 (0.0010) [2023-12-27 16:27:43,835][20347] Updated weights for policy 0, policy_version 1561204 (0.0007) [2023-12-27 16:27:43,889][20347] Updated weights for policy 0, policy_version 1561214 (0.0005) [2023-12-27 16:27:43,951][20347] Updated weights for policy 0, policy_version 1561224 (0.0006) [2023-12-27 16:27:44,235][20381] Updated weights for policy 1, policy_version 1559595 (0.0010) [2023-12-27 16:27:44,294][20381] Updated weights for policy 1, policy_version 1559605 (0.0011) [2023-12-27 16:27:44,349][20381] Updated weights for policy 1, policy_version 1559615 (0.0011) [2023-12-27 16:27:44,525][20347] Updated weights for policy 0, policy_version 1561234 (0.0010) [2023-12-27 16:27:44,585][20347] Updated weights for policy 0, policy_version 1561244 (0.0010) [2023-12-27 16:27:44,650][20347] Updated weights for policy 0, policy_version 1561254 (0.0010) [2023-12-27 16:27:44,703][20347] Updated weights for policy 0, policy_version 1561264 (0.0006) [2023-12-27 16:27:45,019][20381] Updated weights for policy 1, policy_version 1559625 (0.0010) [2023-12-27 16:27:45,072][20381] Updated weights for policy 1, policy_version 1559635 (0.0007) [2023-12-27 16:27:45,127][20381] Updated weights for policy 1, policy_version 1559645 (0.0007) [2023-12-27 16:27:45,189][20381] Updated weights for policy 1, policy_version 1559655 (0.0007) [2023-12-27 16:27:45,392][20347] Updated weights for policy 0, policy_version 1561274 (0.0009) [2023-12-27 16:27:45,448][20347] Updated weights for policy 0, policy_version 1561284 (0.0006) [2023-12-27 16:27:45,506][20347] Updated weights for policy 0, policy_version 1561294 (0.0010) [2023-12-27 16:27:45,880][20381] Updated weights for policy 1, policy_version 1559665 (0.0006) [2023-12-27 16:27:45,936][20381] Updated weights for policy 1, policy_version 1559675 (0.0005) [2023-12-27 16:27:45,988][20381] Updated weights for policy 1, policy_version 1559685 (0.0005) [2023-12-27 16:27:46,061][20347] Updated weights for policy 0, policy_version 1561304 (0.0006) [2023-12-27 16:27:46,110][20347] Updated weights for policy 0, policy_version 1561314 (0.0005) [2023-12-27 16:27:46,170][20347] Updated weights for policy 0, policy_version 1561324 (0.0005) [2023-12-27 16:27:46,672][20381] Updated weights for policy 1, policy_version 1559695 (0.0008) [2023-12-27 16:27:46,718][20381] Updated weights for policy 1, policy_version 1559705 (0.0007) [2023-12-27 16:27:46,731][20347] Updated weights for policy 0, policy_version 1561334 (0.0008) [2023-12-27 16:27:46,763][20381] Updated weights for policy 1, policy_version 1559715 (0.0005) [2023-12-27 16:27:46,783][20347] Updated weights for policy 0, policy_version 1561344 (0.0010) [2023-12-27 16:27:46,846][20347] Updated weights for policy 0, policy_version 1561354 (0.0010) [2023-12-27 16:27:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 799113216. Throughput: 0: 10457.2, 1: 10167.8. Samples: 799126548. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:47,363][19170] Avg episode reward: [(0, '590.950'), (1, '853.680')] [2023-12-27 16:27:47,482][20381] Updated weights for policy 1, policy_version 1559725 (0.0007) [2023-12-27 16:27:47,527][20347] Updated weights for policy 0, policy_version 1561364 (0.0010) [2023-12-27 16:27:47,534][20381] Updated weights for policy 1, policy_version 1559735 (0.0007) [2023-12-27 16:27:47,575][20347] Updated weights for policy 0, policy_version 1561374 (0.0010) [2023-12-27 16:27:47,584][20381] Updated weights for policy 1, policy_version 1559745 (0.0007) [2023-12-27 16:27:47,627][20347] Updated weights for policy 0, policy_version 1561384 (0.0010) [2023-12-27 16:27:48,297][20381] Updated weights for policy 1, policy_version 1559755 (0.0005) [2023-12-27 16:27:48,308][20347] Updated weights for policy 0, policy_version 1561394 (0.0010) [2023-12-27 16:27:48,357][20381] Updated weights for policy 1, policy_version 1559765 (0.0008) [2023-12-27 16:27:48,367][20347] Updated weights for policy 0, policy_version 1561404 (0.0008) [2023-12-27 16:27:48,415][20381] Updated weights for policy 1, policy_version 1559775 (0.0008) [2023-12-27 16:27:48,422][20347] Updated weights for policy 0, policy_version 1561414 (0.0008) [2023-12-27 16:27:48,477][20347] Updated weights for policy 0, policy_version 1561424 (0.0010) [2023-12-27 16:27:49,125][20347] Updated weights for policy 0, policy_version 1561434 (0.0007) [2023-12-27 16:27:49,155][20381] Updated weights for policy 1, policy_version 1559785 (0.0005) [2023-12-27 16:27:49,186][20347] Updated weights for policy 0, policy_version 1561444 (0.0010) [2023-12-27 16:27:49,204][20381] Updated weights for policy 1, policy_version 1559795 (0.0006) [2023-12-27 16:27:49,249][20347] Updated weights for policy 0, policy_version 1561454 (0.0010) [2023-12-27 16:27:49,267][20381] Updated weights for policy 1, policy_version 1559805 (0.0008) [2023-12-27 16:27:49,331][20381] Updated weights for policy 1, policy_version 1559815 (0.0007) [2023-12-27 16:27:49,943][20347] Updated weights for policy 0, policy_version 1561464 (0.0010) [2023-12-27 16:27:49,985][20381] Updated weights for policy 1, policy_version 1559825 (0.0007) [2023-12-27 16:27:50,005][20347] Updated weights for policy 0, policy_version 1561474 (0.0010) [2023-12-27 16:27:50,042][20381] Updated weights for policy 1, policy_version 1559835 (0.0006) [2023-12-27 16:27:50,062][20347] Updated weights for policy 0, policy_version 1561484 (0.0010) [2023-12-27 16:27:50,096][20381] Updated weights for policy 1, policy_version 1559845 (0.0006) [2023-12-27 16:27:50,668][20381] Updated weights for policy 1, policy_version 1559855 (0.0005) [2023-12-27 16:27:50,712][20292] KL-divergence is very high: 127.8839 [2023-12-27 16:27:50,721][20381] Updated weights for policy 1, policy_version 1559865 (0.0006) [2023-12-27 16:27:50,753][20292] KL-divergence is very high: 146.4855 [2023-12-27 16:27:50,754][20347] Updated weights for policy 0, policy_version 1561494 (0.0009) [2023-12-27 16:27:50,775][20381] Updated weights for policy 1, policy_version 1559875 (0.0006) [2023-12-27 16:27:50,818][20347] Updated weights for policy 0, policy_version 1561504 (0.0008) [2023-12-27 16:27:50,882][20347] Updated weights for policy 0, policy_version 1561514 (0.0008) [2023-12-27 16:27:51,548][20381] Updated weights for policy 1, policy_version 1559885 (0.0007) [2023-12-27 16:27:51,596][20347] Updated weights for policy 0, policy_version 1561524 (0.0009) [2023-12-27 16:27:51,606][20381] Updated weights for policy 1, policy_version 1559895 (0.0007) [2023-12-27 16:27:51,667][20347] Updated weights for policy 0, policy_version 1561534 (0.0008) [2023-12-27 16:27:51,669][20381] Updated weights for policy 1, policy_version 1559905 (0.0007) [2023-12-27 16:27:51,726][20347] Updated weights for policy 0, policy_version 1561544 (0.0007) [2023-12-27 16:27:52,354][20347] Updated weights for policy 0, policy_version 1561554 (0.0009) [2023-12-27 16:27:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 799211520. Throughput: 0: 10444.2, 1: 10156.7. Samples: 799187172. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:52,363][19170] Avg episode reward: [(0, '692.410'), (1, '844.560')] [2023-12-27 16:27:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001559912_399376384.pth... [2023-12-27 16:27:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001558728_399073280.pth [2023-12-27 16:27:52,413][20381] Updated weights for policy 1, policy_version 1559915 (0.0006) [2023-12-27 16:27:52,420][20347] Updated weights for policy 0, policy_version 1561564 (0.0008) [2023-12-27 16:27:52,478][20381] Updated weights for policy 1, policy_version 1559925 (0.0007) [2023-12-27 16:27:52,487][20347] Updated weights for policy 0, policy_version 1561574 (0.0008) [2023-12-27 16:27:52,531][20381] Updated weights for policy 1, policy_version 1559935 (0.0008) [2023-12-27 16:27:52,547][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001561584_399843328.pth... [2023-12-27 16:27:52,550][20347] Updated weights for policy 0, policy_version 1561584 (0.0008) [2023-12-27 16:27:52,552][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001560336_399523840.pth [2023-12-27 16:27:53,200][20347] Updated weights for policy 0, policy_version 1561594 (0.0007) [2023-12-27 16:27:53,213][20381] Updated weights for policy 1, policy_version 1559945 (0.0007) [2023-12-27 16:27:53,256][20347] Updated weights for policy 0, policy_version 1561604 (0.0007) [2023-12-27 16:27:53,271][20381] Updated weights for policy 1, policy_version 1559955 (0.0006) [2023-12-27 16:27:53,305][20347] Updated weights for policy 0, policy_version 1561614 (0.0008) [2023-12-27 16:27:53,321][20381] Updated weights for policy 1, policy_version 1559965 (0.0005) [2023-12-27 16:27:53,367][20381] Updated weights for policy 1, policy_version 1559975 (0.0005) [2023-12-27 16:27:53,995][20381] Updated weights for policy 1, policy_version 1559985 (0.0005) [2023-12-27 16:27:54,004][20347] Updated weights for policy 0, policy_version 1561624 (0.0008) [2023-12-27 16:27:54,044][20381] Updated weights for policy 1, policy_version 1559995 (0.0005) [2023-12-27 16:27:54,054][20347] Updated weights for policy 0, policy_version 1561634 (0.0008) [2023-12-27 16:27:54,090][20381] Updated weights for policy 1, policy_version 1560005 (0.0005) [2023-12-27 16:27:54,106][20347] Updated weights for policy 0, policy_version 1561644 (0.0008) [2023-12-27 16:27:54,728][20381] Updated weights for policy 1, policy_version 1560015 (0.0007) [2023-12-27 16:27:54,789][20381] Updated weights for policy 1, policy_version 1560025 (0.0007) [2023-12-27 16:27:54,810][20347] Updated weights for policy 0, policy_version 1561654 (0.0006) [2023-12-27 16:27:54,845][20381] Updated weights for policy 1, policy_version 1560035 (0.0006) [2023-12-27 16:27:54,863][20347] Updated weights for policy 0, policy_version 1561664 (0.0005) [2023-12-27 16:27:54,911][20347] Updated weights for policy 0, policy_version 1561674 (0.0008) [2023-12-27 16:27:55,549][20381] Updated weights for policy 1, policy_version 1560045 (0.0007) [2023-12-27 16:27:55,554][20347] Updated weights for policy 0, policy_version 1561684 (0.0006) [2023-12-27 16:27:55,597][20381] Updated weights for policy 1, policy_version 1560055 (0.0005) [2023-12-27 16:27:55,607][20347] Updated weights for policy 0, policy_version 1561694 (0.0005) [2023-12-27 16:27:55,646][20381] Updated weights for policy 1, policy_version 1560065 (0.0005) [2023-12-27 16:27:55,667][20347] Updated weights for policy 0, policy_version 1561704 (0.0007) [2023-12-27 16:27:56,301][20381] Updated weights for policy 1, policy_version 1560075 (0.0006) [2023-12-27 16:27:56,326][20347] Updated weights for policy 0, policy_version 1561714 (0.0009) [2023-12-27 16:27:56,355][20381] Updated weights for policy 1, policy_version 1560085 (0.0006) [2023-12-27 16:27:56,383][20347] Updated weights for policy 0, policy_version 1561724 (0.0007) [2023-12-27 16:27:56,417][20381] Updated weights for policy 1, policy_version 1560095 (0.0005) [2023-12-27 16:27:56,437][20347] Updated weights for policy 0, policy_version 1561734 (0.0007) [2023-12-27 16:27:56,490][20347] Updated weights for policy 0, policy_version 1561744 (0.0007) [2023-12-27 16:27:57,083][20381] Updated weights for policy 1, policy_version 1560105 (0.0008) [2023-12-27 16:27:57,135][20381] Updated weights for policy 1, policy_version 1560115 (0.0006) [2023-12-27 16:27:57,155][20347] Updated weights for policy 0, policy_version 1561754 (0.0005) [2023-12-27 16:27:57,197][20381] Updated weights for policy 1, policy_version 1560125 (0.0007) [2023-12-27 16:27:57,214][20347] Updated weights for policy 0, policy_version 1561764 (0.0006) [2023-12-27 16:27:57,242][20381] Updated weights for policy 1, policy_version 1560135 (0.0006) [2023-12-27 16:27:57,267][20347] Updated weights for policy 0, policy_version 1561774 (0.0006) [2023-12-27 16:27:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 799326208. Throughput: 0: 10437.3, 1: 10163.5. Samples: 799309724. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:27:57,363][19170] Avg episode reward: [(0, '758.390'), (1, '796.540')] [2023-12-27 16:27:57,850][20347] Updated weights for policy 0, policy_version 1561784 (0.0008) [2023-12-27 16:27:57,904][20347] Updated weights for policy 0, policy_version 1561794 (0.0008) [2023-12-27 16:27:57,917][20381] Updated weights for policy 1, policy_version 1560145 (0.0005) [2023-12-27 16:27:57,950][20347] Updated weights for policy 0, policy_version 1561804 (0.0007) [2023-12-27 16:27:57,965][20381] Updated weights for policy 1, policy_version 1560155 (0.0005) [2023-12-27 16:27:58,019][20381] Updated weights for policy 1, policy_version 1560165 (0.0005) [2023-12-27 16:27:58,642][20381] Updated weights for policy 1, policy_version 1560175 (0.0006) [2023-12-27 16:27:58,710][20381] Updated weights for policy 1, policy_version 1560185 (0.0007) [2023-12-27 16:27:58,729][20347] Updated weights for policy 0, policy_version 1561814 (0.0009) [2023-12-27 16:27:58,773][20381] Updated weights for policy 1, policy_version 1560195 (0.0007) [2023-12-27 16:27:58,799][20347] Updated weights for policy 0, policy_version 1561824 (0.0013) [2023-12-27 16:27:58,863][20347] Updated weights for policy 0, policy_version 1561834 (0.0008) [2023-12-27 16:27:59,504][20381] Updated weights for policy 1, policy_version 1560205 (0.0008) [2023-12-27 16:27:59,564][20381] Updated weights for policy 1, policy_version 1560215 (0.0005) [2023-12-27 16:27:59,621][20381] Updated weights for policy 1, policy_version 1560225 (0.0008) [2023-12-27 16:27:59,637][20347] Updated weights for policy 0, policy_version 1561844 (0.0008) [2023-12-27 16:27:59,694][20347] Updated weights for policy 0, policy_version 1561854 (0.0006) [2023-12-27 16:27:59,738][20347] Updated weights for policy 0, policy_version 1561864 (0.0007) [2023-12-27 16:28:00,332][20381] Updated weights for policy 1, policy_version 1560235 (0.0010) [2023-12-27 16:28:00,388][20381] Updated weights for policy 1, policy_version 1560245 (0.0008) [2023-12-27 16:28:00,444][20381] Updated weights for policy 1, policy_version 1560255 (0.0008) [2023-12-27 16:28:00,483][20347] Updated weights for policy 0, policy_version 1561874 (0.0007) [2023-12-27 16:28:00,538][20347] Updated weights for policy 0, policy_version 1561884 (0.0007) [2023-12-27 16:28:00,595][20347] Updated weights for policy 0, policy_version 1561894 (0.0007) [2023-12-27 16:28:00,657][20347] Updated weights for policy 0, policy_version 1561904 (0.0007) [2023-12-27 16:28:01,092][20381] Updated weights for policy 1, policy_version 1560265 (0.0007) [2023-12-27 16:28:01,156][20381] Updated weights for policy 1, policy_version 1560275 (0.0008) [2023-12-27 16:28:01,222][20381] Updated weights for policy 1, policy_version 1560285 (0.0006) [2023-12-27 16:28:01,283][20381] Updated weights for policy 1, policy_version 1560295 (0.0008) [2023-12-27 16:28:01,313][20347] Updated weights for policy 0, policy_version 1561914 (0.0006) [2023-12-27 16:28:01,384][20347] Updated weights for policy 0, policy_version 1561924 (0.0007) [2023-12-27 16:28:01,434][20347] Updated weights for policy 0, policy_version 1561934 (0.0008) [2023-12-27 16:28:01,961][20381] Updated weights for policy 1, policy_version 1560305 (0.0008) [2023-12-27 16:28:02,021][20381] Updated weights for policy 1, policy_version 1560315 (0.0007) [2023-12-27 16:28:02,073][20381] Updated weights for policy 1, policy_version 1560325 (0.0007) [2023-12-27 16:28:02,134][20347] Updated weights for policy 0, policy_version 1561944 (0.0008) [2023-12-27 16:28:02,189][20347] Updated weights for policy 0, policy_version 1561954 (0.0008) [2023-12-27 16:28:02,245][20347] Updated weights for policy 0, policy_version 1561964 (0.0008) [2023-12-27 16:28:02,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 799424512. Throughput: 0: 10307.2, 1: 10195.1. Samples: 799430556. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:28:02,363][19170] Avg episode reward: [(0, '737.320'), (1, '834.910')] [2023-12-27 16:28:02,771][20381] Updated weights for policy 1, policy_version 1560335 (0.0008) [2023-12-27 16:28:02,829][20381] Updated weights for policy 1, policy_version 1560345 (0.0008) [2023-12-27 16:28:02,889][20381] Updated weights for policy 1, policy_version 1560355 (0.0008) [2023-12-27 16:28:02,955][20347] Updated weights for policy 0, policy_version 1561974 (0.0008) [2023-12-27 16:28:03,014][20347] Updated weights for policy 0, policy_version 1561984 (0.0008) [2023-12-27 16:28:03,068][20347] Updated weights for policy 0, policy_version 1561994 (0.0005) [2023-12-27 16:28:03,621][20381] Updated weights for policy 1, policy_version 1560365 (0.0007) [2023-12-27 16:28:03,643][20347] Updated weights for policy 0, policy_version 1562004 (0.0007) [2023-12-27 16:28:03,682][20381] Updated weights for policy 1, policy_version 1560375 (0.0007) [2023-12-27 16:28:03,692][20347] Updated weights for policy 0, policy_version 1562014 (0.0010) [2023-12-27 16:28:03,734][20381] Updated weights for policy 1, policy_version 1560385 (0.0007) [2023-12-27 16:28:03,741][20347] Updated weights for policy 0, policy_version 1562024 (0.0010) [2023-12-27 16:28:04,391][20381] Updated weights for policy 1, policy_version 1560395 (0.0006) [2023-12-27 16:28:04,454][20347] Updated weights for policy 0, policy_version 1562034 (0.0010) [2023-12-27 16:28:04,457][20381] Updated weights for policy 1, policy_version 1560405 (0.0008) [2023-12-27 16:28:04,513][20347] Updated weights for policy 0, policy_version 1562044 (0.0010) [2023-12-27 16:28:04,520][20381] Updated weights for policy 1, policy_version 1560415 (0.0008) [2023-12-27 16:28:04,572][20347] Updated weights for policy 0, policy_version 1562054 (0.0009) [2023-12-27 16:28:04,635][20347] Updated weights for policy 0, policy_version 1562064 (0.0010) [2023-12-27 16:28:05,099][20381] Updated weights for policy 1, policy_version 1560425 (0.0007) [2023-12-27 16:28:05,168][20381] Updated weights for policy 1, policy_version 1560435 (0.0007) [2023-12-27 16:28:05,223][20381] Updated weights for policy 1, policy_version 1560445 (0.0006) [2023-12-27 16:28:05,273][20381] Updated weights for policy 1, policy_version 1560455 (0.0006) [2023-12-27 16:28:05,317][20347] Updated weights for policy 0, policy_version 1562074 (0.0010) [2023-12-27 16:28:05,379][20347] Updated weights for policy 0, policy_version 1562084 (0.0010) [2023-12-27 16:28:05,439][20347] Updated weights for policy 0, policy_version 1562094 (0.0010) [2023-12-27 16:28:05,901][20381] Updated weights for policy 1, policy_version 1560465 (0.0007) [2023-12-27 16:28:05,953][20381] Updated weights for policy 1, policy_version 1560475 (0.0007) [2023-12-27 16:28:06,010][20381] Updated weights for policy 1, policy_version 1560485 (0.0007) [2023-12-27 16:28:06,098][20347] Updated weights for policy 0, policy_version 1562104 (0.0010) [2023-12-27 16:28:06,162][20347] Updated weights for policy 0, policy_version 1562114 (0.0010) [2023-12-27 16:28:06,224][20347] Updated weights for policy 0, policy_version 1562124 (0.0008) [2023-12-27 16:28:06,703][20381] Updated weights for policy 1, policy_version 1560495 (0.0008) [2023-12-27 16:28:06,751][20381] Updated weights for policy 1, policy_version 1560505 (0.0006) [2023-12-27 16:28:06,800][20381] Updated weights for policy 1, policy_version 1560515 (0.0006) [2023-12-27 16:28:06,948][20347] Updated weights for policy 0, policy_version 1562134 (0.0009) [2023-12-27 16:28:07,013][20347] Updated weights for policy 0, policy_version 1562144 (0.0010) [2023-12-27 16:28:07,102][20347] Updated weights for policy 0, policy_version 1562154 (0.0010) [2023-12-27 16:28:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 799522816. Throughput: 0: 10301.1, 1: 10192.3. Samples: 799492336. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:28:07,363][19170] Avg episode reward: [(0, '727.330'), (1, '854.610')] [2023-12-27 16:28:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001562160_399990784.pth... [2023-12-27 16:28:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001560520_399532032.pth... [2023-12-27 16:28:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001560976_399687680.pth [2023-12-27 16:28:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001559336_399228928.pth [2023-12-27 16:28:07,481][20381] Updated weights for policy 1, policy_version 1560525 (0.0008) [2023-12-27 16:28:07,531][20381] Updated weights for policy 1, policy_version 1560535 (0.0008) [2023-12-27 16:28:07,578][20381] Updated weights for policy 1, policy_version 1560545 (0.0010) [2023-12-27 16:28:07,677][20347] Updated weights for policy 0, policy_version 1562164 (0.0008) [2023-12-27 16:28:07,735][20347] Updated weights for policy 0, policy_version 1562174 (0.0006) [2023-12-27 16:28:07,786][20347] Updated weights for policy 0, policy_version 1562184 (0.0005) [2023-12-27 16:28:08,240][20381] Updated weights for policy 1, policy_version 1560555 (0.0009) [2023-12-27 16:28:08,300][20381] Updated weights for policy 1, policy_version 1560565 (0.0005) [2023-12-27 16:28:08,367][20347] Updated weights for policy 0, policy_version 1562194 (0.0006) [2023-12-27 16:28:08,370][20381] Updated weights for policy 1, policy_version 1560575 (0.0011) [2023-12-27 16:28:08,427][20347] Updated weights for policy 0, policy_version 1562204 (0.0008) [2023-12-27 16:28:08,481][20347] Updated weights for policy 0, policy_version 1562214 (0.0006) [2023-12-27 16:28:08,533][20347] Updated weights for policy 0, policy_version 1562224 (0.0010) [2023-12-27 16:28:09,035][20381] Updated weights for policy 1, policy_version 1560585 (0.0011) [2023-12-27 16:28:09,086][20381] Updated weights for policy 1, policy_version 1560595 (0.0010) [2023-12-27 16:28:09,138][20381] Updated weights for policy 1, policy_version 1560605 (0.0006) [2023-12-27 16:28:09,166][20347] Updated weights for policy 0, policy_version 1562234 (0.0005) [2023-12-27 16:28:09,191][20381] Updated weights for policy 1, policy_version 1560615 (0.0006) [2023-12-27 16:28:09,226][20347] Updated weights for policy 0, policy_version 1562244 (0.0008) [2023-12-27 16:28:09,280][20347] Updated weights for policy 0, policy_version 1562254 (0.0008) [2023-12-27 16:28:09,881][20381] Updated weights for policy 1, policy_version 1560625 (0.0008) [2023-12-27 16:28:09,940][20381] Updated weights for policy 1, policy_version 1560635 (0.0008) [2023-12-27 16:28:09,964][20347] Updated weights for policy 0, policy_version 1562264 (0.0011) [2023-12-27 16:28:10,005][20381] Updated weights for policy 1, policy_version 1560645 (0.0008) [2023-12-27 16:28:10,015][20347] Updated weights for policy 0, policy_version 1562274 (0.0010) [2023-12-27 16:28:10,071][20347] Updated weights for policy 0, policy_version 1562284 (0.0009) [2023-12-27 16:28:10,679][20347] Updated weights for policy 0, policy_version 1562294 (0.0005) [2023-12-27 16:28:10,728][20347] Updated weights for policy 0, policy_version 1562304 (0.0009) [2023-12-27 16:28:10,788][20347] Updated weights for policy 0, policy_version 1562314 (0.0005) [2023-12-27 16:28:10,793][20381] Updated weights for policy 1, policy_version 1560655 (0.0007) [2023-12-27 16:28:10,851][20381] Updated weights for policy 1, policy_version 1560665 (0.0008) [2023-12-27 16:28:10,909][20381] Updated weights for policy 1, policy_version 1560675 (0.0009) [2023-12-27 16:28:11,520][20347] Updated weights for policy 0, policy_version 1562324 (0.0008) [2023-12-27 16:28:11,586][20347] Updated weights for policy 0, policy_version 1562334 (0.0005) [2023-12-27 16:28:11,648][20347] Updated weights for policy 0, policy_version 1562344 (0.0009) [2023-12-27 16:28:11,671][20381] Updated weights for policy 1, policy_version 1560685 (0.0007) [2023-12-27 16:28:11,734][20381] Updated weights for policy 1, policy_version 1560695 (0.0008) [2023-12-27 16:28:11,799][20381] Updated weights for policy 1, policy_version 1560705 (0.0007) [2023-12-27 16:28:12,339][20347] Updated weights for policy 0, policy_version 1562354 (0.0008) [2023-12-27 16:28:12,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 799621120. Throughput: 0: 10330.6, 1: 10172.4. Samples: 799616316. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:28:12,363][19170] Avg episode reward: [(0, '785.520'), (1, '834.970')] [2023-12-27 16:28:12,406][20347] Updated weights for policy 0, policy_version 1562364 (0.0008) [2023-12-27 16:28:12,461][20347] Updated weights for policy 0, policy_version 1562374 (0.0008) [2023-12-27 16:28:12,519][20347] Updated weights for policy 0, policy_version 1562384 (0.0008) [2023-12-27 16:28:12,556][20381] Updated weights for policy 1, policy_version 1560715 (0.0007) [2023-12-27 16:28:12,619][20381] Updated weights for policy 1, policy_version 1560725 (0.0006) [2023-12-27 16:28:12,685][20381] Updated weights for policy 1, policy_version 1560735 (0.0007) [2023-12-27 16:28:13,240][20347] Updated weights for policy 0, policy_version 1562394 (0.0005) [2023-12-27 16:28:13,299][20347] Updated weights for policy 0, policy_version 1562404 (0.0007) [2023-12-27 16:28:13,327][20381] Updated weights for policy 1, policy_version 1560745 (0.0011) [2023-12-27 16:28:13,352][20347] Updated weights for policy 0, policy_version 1562414 (0.0007) [2023-12-27 16:28:13,384][20381] Updated weights for policy 1, policy_version 1560755 (0.0007) [2023-12-27 16:28:13,445][20381] Updated weights for policy 1, policy_version 1560765 (0.0010) [2023-12-27 16:28:13,489][20381] Updated weights for policy 1, policy_version 1560775 (0.0006) [2023-12-27 16:28:13,971][20347] Updated weights for policy 0, policy_version 1562424 (0.0008) [2023-12-27 16:28:14,030][20347] Updated weights for policy 0, policy_version 1562434 (0.0008) [2023-12-27 16:28:14,077][20347] Updated weights for policy 0, policy_version 1562444 (0.0007) [2023-12-27 16:28:14,112][20381] Updated weights for policy 1, policy_version 1560785 (0.0006) [2023-12-27 16:28:14,171][20381] Updated weights for policy 1, policy_version 1560795 (0.0008) [2023-12-27 16:28:14,218][20381] Updated weights for policy 1, policy_version 1560805 (0.0007) [2023-12-27 16:28:14,748][20347] Updated weights for policy 0, policy_version 1562454 (0.0005) [2023-12-27 16:28:14,810][20347] Updated weights for policy 0, policy_version 1562464 (0.0007) [2023-12-27 16:28:14,866][20347] Updated weights for policy 0, policy_version 1562474 (0.0006) [2023-12-27 16:28:14,876][20381] Updated weights for policy 1, policy_version 1560815 (0.0008) [2023-12-27 16:28:14,930][20381] Updated weights for policy 1, policy_version 1560825 (0.0007) [2023-12-27 16:28:14,987][20381] Updated weights for policy 1, policy_version 1560835 (0.0008) [2023-12-27 16:28:15,542][20347] Updated weights for policy 0, policy_version 1562484 (0.0007) [2023-12-27 16:28:15,596][20347] Updated weights for policy 0, policy_version 1562494 (0.0005) [2023-12-27 16:28:15,652][20347] Updated weights for policy 0, policy_version 1562504 (0.0005) [2023-12-27 16:28:15,754][20381] Updated weights for policy 1, policy_version 1560845 (0.0008) [2023-12-27 16:28:15,808][20381] Updated weights for policy 1, policy_version 1560855 (0.0007) [2023-12-27 16:28:15,867][20381] Updated weights for policy 1, policy_version 1560865 (0.0008) [2023-12-27 16:28:16,189][20347] Updated weights for policy 0, policy_version 1562514 (0.0005) [2023-12-27 16:28:16,241][20347] Updated weights for policy 0, policy_version 1562524 (0.0005) [2023-12-27 16:28:16,297][20347] Updated weights for policy 0, policy_version 1562534 (0.0005) [2023-12-27 16:28:16,352][20347] Updated weights for policy 0, policy_version 1562544 (0.0007) [2023-12-27 16:28:16,617][20381] Updated weights for policy 1, policy_version 1560875 (0.0006) [2023-12-27 16:28:16,665][20381] Updated weights for policy 1, policy_version 1560885 (0.0007) [2023-12-27 16:28:16,715][20381] Updated weights for policy 1, policy_version 1560895 (0.0008) [2023-12-27 16:28:16,884][20347] Updated weights for policy 0, policy_version 1562554 (0.0005) [2023-12-27 16:28:16,940][20347] Updated weights for policy 0, policy_version 1562564 (0.0009) [2023-12-27 16:28:17,001][20347] Updated weights for policy 0, policy_version 1562574 (0.0010) [2023-12-27 16:28:17,362][20381] Updated weights for policy 1, policy_version 1560905 (0.0008) [2023-12-27 16:28:17,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 799727616. Throughput: 0: 10311.8, 1: 10156.1. Samples: 799740336. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-12-27 16:28:17,364][19170] Avg episode reward: [(0, '708.550'), (1, '883.000')] [2023-12-27 16:28:17,421][20381] Updated weights for policy 1, policy_version 1560915 (0.0008) [2023-12-27 16:28:17,474][20381] Updated weights for policy 1, policy_version 1560925 (0.0005) [2023-12-27 16:28:17,530][20381] Updated weights for policy 1, policy_version 1560935 (0.0006) [2023-12-27 16:28:17,634][20347] Updated weights for policy 0, policy_version 1562584 (0.0010) [2023-12-27 16:28:17,682][20347] Updated weights for policy 0, policy_version 1562594 (0.0010) [2023-12-27 16:28:17,730][20347] Updated weights for policy 0, policy_version 1562604 (0.0008) [2023-12-27 16:28:18,226][20381] Updated weights for policy 1, policy_version 1560945 (0.0008) [2023-12-27 16:28:18,272][20381] Updated weights for policy 1, policy_version 1560955 (0.0008) [2023-12-27 16:28:18,323][20381] Updated weights for policy 1, policy_version 1560965 (0.0007) [2023-12-27 16:28:18,351][20347] Updated weights for policy 0, policy_version 1562614 (0.0008) [2023-12-27 16:28:18,417][20347] Updated weights for policy 0, policy_version 1562624 (0.0010) [2023-12-27 16:28:18,485][20347] Updated weights for policy 0, policy_version 1562634 (0.0011) [2023-12-27 16:28:19,060][20381] Updated weights for policy 1, policy_version 1560975 (0.0008) [2023-12-27 16:28:19,111][20381] Updated weights for policy 1, policy_version 1560985 (0.0007) [2023-12-27 16:28:19,146][20347] Updated weights for policy 0, policy_version 1562644 (0.0010) [2023-12-27 16:28:19,170][20381] Updated weights for policy 1, policy_version 1560995 (0.0006) [2023-12-27 16:28:19,195][20347] Updated weights for policy 0, policy_version 1562654 (0.0010) [2023-12-27 16:28:19,257][20347] Updated weights for policy 0, policy_version 1562664 (0.0011) [2023-12-27 16:28:19,900][20381] Updated weights for policy 1, policy_version 1561005 (0.0006) [2023-12-27 16:28:19,961][20381] Updated weights for policy 1, policy_version 1561015 (0.0007) [2023-12-27 16:28:19,990][20347] Updated weights for policy 0, policy_version 1562674 (0.0010) [2023-12-27 16:28:20,018][20381] Updated weights for policy 1, policy_version 1561025 (0.0006) [2023-12-27 16:28:20,048][20347] Updated weights for policy 0, policy_version 1562684 (0.0007) [2023-12-27 16:28:20,102][20347] Updated weights for policy 0, policy_version 1562694 (0.0008) [2023-12-27 16:28:20,162][20347] Updated weights for policy 0, policy_version 1562704 (0.0008) [2023-12-27 16:28:20,672][20381] Updated weights for policy 1, policy_version 1561035 (0.0006) [2023-12-27 16:28:20,726][20381] Updated weights for policy 1, policy_version 1561045 (0.0006) [2023-12-27 16:28:20,785][20381] Updated weights for policy 1, policy_version 1561055 (0.0005) [2023-12-27 16:28:20,932][20347] Updated weights for policy 0, policy_version 1562714 (0.0011) [2023-12-27 16:28:20,995][20347] Updated weights for policy 0, policy_version 1562724 (0.0011) [2023-12-27 16:28:21,065][20347] Updated weights for policy 0, policy_version 1562734 (0.0010) [2023-12-27 16:28:21,521][20381] Updated weights for policy 1, policy_version 1561065 (0.0007) [2023-12-27 16:28:21,580][20381] Updated weights for policy 1, policy_version 1561075 (0.0007) [2023-12-27 16:28:21,619][20292] KL-divergence is very high: 103.3130 [2023-12-27 16:28:21,644][20381] Updated weights for policy 1, policy_version 1561085 (0.0008) [2023-12-27 16:28:21,701][20381] Updated weights for policy 1, policy_version 1561095 (0.0008) [2023-12-27 16:28:21,865][20347] Updated weights for policy 0, policy_version 1562744 (0.0011) [2023-12-27 16:28:21,923][20347] Updated weights for policy 0, policy_version 1562754 (0.0011) [2023-12-27 16:28:21,985][20347] Updated weights for policy 0, policy_version 1562764 (0.0011) [2023-12-27 16:28:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 799825920. Throughput: 0: 10321.4, 1: 10131.6. Samples: 799801196. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:22,363][19170] Avg episode reward: [(0, '641.970'), (1, '883.150')] [2023-12-27 16:28:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001562768_400146432.pth... [2023-12-27 16:28:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001561584_399843328.pth [2023-12-27 16:28:22,389][20381] Updated weights for policy 1, policy_version 1561105 (0.0009) [2023-12-27 16:28:22,458][20381] Updated weights for policy 1, policy_version 1561115 (0.0010) [2023-12-27 16:28:22,514][20381] Updated weights for policy 1, policy_version 1561125 (0.0011) [2023-12-27 16:28:22,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001561128_399687680.pth... [2023-12-27 16:28:22,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001559912_399376384.pth [2023-12-27 16:28:22,684][20347] Updated weights for policy 0, policy_version 1562774 (0.0011) [2023-12-27 16:28:22,741][20347] Updated weights for policy 0, policy_version 1562784 (0.0010) [2023-12-27 16:28:22,798][20347] Updated weights for policy 0, policy_version 1562794 (0.0010) [2023-12-27 16:28:23,215][20381] Updated weights for policy 1, policy_version 1561135 (0.0008) [2023-12-27 16:28:23,271][20381] Updated weights for policy 1, policy_version 1561145 (0.0010) [2023-12-27 16:28:23,326][20381] Updated weights for policy 1, policy_version 1561155 (0.0009) [2023-12-27 16:28:23,487][20347] Updated weights for policy 0, policy_version 1562804 (0.0008) [2023-12-27 16:28:23,552][20347] Updated weights for policy 0, policy_version 1562814 (0.0006) [2023-12-27 16:28:23,617][20347] Updated weights for policy 0, policy_version 1562824 (0.0006) [2023-12-27 16:28:23,970][20381] Updated weights for policy 1, policy_version 1561165 (0.0011) [2023-12-27 16:28:24,028][20381] Updated weights for policy 1, policy_version 1561175 (0.0011) [2023-12-27 16:28:24,082][20381] Updated weights for policy 1, policy_version 1561185 (0.0010) [2023-12-27 16:28:24,241][20347] Updated weights for policy 0, policy_version 1562834 (0.0007) [2023-12-27 16:28:24,292][20347] Updated weights for policy 0, policy_version 1562844 (0.0007) [2023-12-27 16:28:24,344][20347] Updated weights for policy 0, policy_version 1562854 (0.0007) [2023-12-27 16:28:24,407][20347] Updated weights for policy 0, policy_version 1562864 (0.0007) [2023-12-27 16:28:24,767][20381] Updated weights for policy 1, policy_version 1561195 (0.0011) [2023-12-27 16:28:24,825][20381] Updated weights for policy 1, policy_version 1561205 (0.0011) [2023-12-27 16:28:24,876][20381] Updated weights for policy 1, policy_version 1561215 (0.0010) [2023-12-27 16:28:25,094][20347] Updated weights for policy 0, policy_version 1562874 (0.0007) [2023-12-27 16:28:25,143][20347] Updated weights for policy 0, policy_version 1562884 (0.0007) [2023-12-27 16:28:25,200][20347] Updated weights for policy 0, policy_version 1562894 (0.0007) [2023-12-27 16:28:25,566][20381] Updated weights for policy 1, policy_version 1561225 (0.0010) [2023-12-27 16:28:25,625][20381] Updated weights for policy 1, policy_version 1561235 (0.0011) [2023-12-27 16:28:25,686][20381] Updated weights for policy 1, policy_version 1561245 (0.0011) [2023-12-27 16:28:25,744][20381] Updated weights for policy 1, policy_version 1561255 (0.0010) [2023-12-27 16:28:25,852][20347] Updated weights for policy 0, policy_version 1562904 (0.0007) [2023-12-27 16:28:25,899][20347] Updated weights for policy 0, policy_version 1562914 (0.0007) [2023-12-27 16:28:25,953][20347] Updated weights for policy 0, policy_version 1562924 (0.0008) [2023-12-27 16:28:26,348][20381] Updated weights for policy 1, policy_version 1561265 (0.0011) [2023-12-27 16:28:26,409][20381] Updated weights for policy 1, policy_version 1561275 (0.0010) [2023-12-27 16:28:26,472][20381] Updated weights for policy 1, policy_version 1561285 (0.0010) [2023-12-27 16:28:26,539][20347] Updated weights for policy 0, policy_version 1562934 (0.0006) [2023-12-27 16:28:26,593][20347] Updated weights for policy 0, policy_version 1562944 (0.0007) [2023-12-27 16:28:26,646][20347] Updated weights for policy 0, policy_version 1562954 (0.0006) [2023-12-27 16:28:27,149][20381] Updated weights for policy 1, policy_version 1561295 (0.0011) [2023-12-27 16:28:27,194][20381] Updated weights for policy 1, policy_version 1561305 (0.0010) [2023-12-27 16:28:27,213][20347] Updated weights for policy 0, policy_version 1562964 (0.0005) [2023-12-27 16:28:27,247][20381] Updated weights for policy 1, policy_version 1561315 (0.0010) [2023-12-27 16:28:27,264][20347] Updated weights for policy 0, policy_version 1562974 (0.0008) [2023-12-27 16:28:27,315][20347] Updated weights for policy 0, policy_version 1562984 (0.0007) [2023-12-27 16:28:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 799940608. Throughput: 0: 10317.7, 1: 10105.6. Samples: 799921868. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:27,363][19170] Avg episode reward: [(0, '765.670'), (1, '795.240')] [2023-12-27 16:28:27,894][20347] Updated weights for policy 0, policy_version 1562994 (0.0007) [2023-12-27 16:28:27,915][20381] Updated weights for policy 1, policy_version 1561325 (0.0011) [2023-12-27 16:28:27,953][20347] Updated weights for policy 0, policy_version 1563004 (0.0008) [2023-12-27 16:28:27,974][20381] Updated weights for policy 1, policy_version 1561335 (0.0011) [2023-12-27 16:28:28,006][20347] Updated weights for policy 0, policy_version 1563014 (0.0005) [2023-12-27 16:28:28,031][20381] Updated weights for policy 1, policy_version 1561345 (0.0010) [2023-12-27 16:28:28,063][20347] Updated weights for policy 0, policy_version 1563024 (0.0007) [2023-12-27 16:28:28,747][20381] Updated weights for policy 1, policy_version 1561355 (0.0009) [2023-12-27 16:28:28,806][20381] Updated weights for policy 1, policy_version 1561365 (0.0006) [2023-12-27 16:28:28,814][20347] Updated weights for policy 0, policy_version 1563034 (0.0009) [2023-12-27 16:28:28,871][20347] Updated weights for policy 0, policy_version 1563044 (0.0010) [2023-12-27 16:28:28,872][20381] Updated weights for policy 1, policy_version 1561375 (0.0006) [2023-12-27 16:28:28,927][20347] Updated weights for policy 0, policy_version 1563054 (0.0010) [2023-12-27 16:28:29,428][20381] Updated weights for policy 1, policy_version 1561385 (0.0006) [2023-12-27 16:28:29,480][20381] Updated weights for policy 1, policy_version 1561395 (0.0010) [2023-12-27 16:28:29,533][20381] Updated weights for policy 1, policy_version 1561405 (0.0011) [2023-12-27 16:28:29,591][20381] Updated weights for policy 1, policy_version 1561415 (0.0010) [2023-12-27 16:28:29,642][20347] Updated weights for policy 0, policy_version 1563064 (0.0008) [2023-12-27 16:28:29,703][20347] Updated weights for policy 0, policy_version 1563074 (0.0007) [2023-12-27 16:28:29,762][20347] Updated weights for policy 0, policy_version 1563084 (0.0006) [2023-12-27 16:28:30,263][20381] Updated weights for policy 1, policy_version 1561425 (0.0006) [2023-12-27 16:28:30,324][20381] Updated weights for policy 1, policy_version 1561435 (0.0005) [2023-12-27 16:28:30,385][20381] Updated weights for policy 1, policy_version 1561445 (0.0007) [2023-12-27 16:28:30,457][20347] Updated weights for policy 0, policy_version 1563094 (0.0009) [2023-12-27 16:28:30,507][20347] Updated weights for policy 0, policy_version 1563104 (0.0007) [2023-12-27 16:28:30,559][20347] Updated weights for policy 0, policy_version 1563114 (0.0008) [2023-12-27 16:28:30,975][20381] Updated weights for policy 1, policy_version 1561455 (0.0008) [2023-12-27 16:28:31,030][20381] Updated weights for policy 1, policy_version 1561465 (0.0008) [2023-12-27 16:28:31,090][20381] Updated weights for policy 1, policy_version 1561475 (0.0009) [2023-12-27 16:28:31,244][20347] Updated weights for policy 0, policy_version 1563124 (0.0008) [2023-12-27 16:28:31,306][20347] Updated weights for policy 0, policy_version 1563134 (0.0008) [2023-12-27 16:28:31,388][20347] Updated weights for policy 0, policy_version 1563144 (0.0007) [2023-12-27 16:28:31,755][20381] Updated weights for policy 1, policy_version 1561485 (0.0009) [2023-12-27 16:28:31,813][20381] Updated weights for policy 1, policy_version 1561495 (0.0008) [2023-12-27 16:28:31,876][20381] Updated weights for policy 1, policy_version 1561505 (0.0006) [2023-12-27 16:28:32,065][20347] Updated weights for policy 0, policy_version 1563154 (0.0008) [2023-12-27 16:28:32,124][20347] Updated weights for policy 0, policy_version 1563164 (0.0009) [2023-12-27 16:28:32,187][20347] Updated weights for policy 0, policy_version 1563174 (0.0007) [2023-12-27 16:28:32,246][20347] Updated weights for policy 0, policy_version 1563184 (0.0007) [2023-12-27 16:28:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 800038912. Throughput: 0: 10316.8, 1: 10170.0. Samples: 800048456. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:32,363][19170] Avg episode reward: [(0, '794.130'), (1, '824.480')] [2023-12-27 16:28:32,566][20381] Updated weights for policy 1, policy_version 1561515 (0.0006) [2023-12-27 16:28:32,624][20381] Updated weights for policy 1, policy_version 1561525 (0.0007) [2023-12-27 16:28:32,674][20381] Updated weights for policy 1, policy_version 1561535 (0.0007) [2023-12-27 16:28:32,931][20347] Updated weights for policy 0, policy_version 1563194 (0.0005) [2023-12-27 16:28:32,989][20347] Updated weights for policy 0, policy_version 1563204 (0.0006) [2023-12-27 16:28:33,036][20347] Updated weights for policy 0, policy_version 1563214 (0.0007) [2023-12-27 16:28:33,324][20381] Updated weights for policy 1, policy_version 1561545 (0.0008) [2023-12-27 16:28:33,380][20381] Updated weights for policy 1, policy_version 1561555 (0.0008) [2023-12-27 16:28:33,435][20381] Updated weights for policy 1, policy_version 1561565 (0.0007) [2023-12-27 16:28:33,490][20381] Updated weights for policy 1, policy_version 1561575 (0.0007) [2023-12-27 16:28:33,694][20347] Updated weights for policy 0, policy_version 1563224 (0.0006) [2023-12-27 16:28:33,753][20347] Updated weights for policy 0, policy_version 1563234 (0.0005) [2023-12-27 16:28:33,812][20347] Updated weights for policy 0, policy_version 1563244 (0.0006) [2023-12-27 16:28:34,201][20381] Updated weights for policy 1, policy_version 1561591 (0.0007) [2023-12-27 16:28:34,263][20381] Updated weights for policy 1, policy_version 1561601 (0.0010) [2023-12-27 16:28:34,437][20347] Updated weights for policy 0, policy_version 1563254 (0.0007) [2023-12-27 16:28:34,499][20347] Updated weights for policy 0, policy_version 1563264 (0.0007) [2023-12-27 16:28:34,564][20347] Updated weights for policy 0, policy_version 1563274 (0.0007) [2023-12-27 16:28:34,969][20381] Updated weights for policy 1, policy_version 1561611 (0.0011) [2023-12-27 16:28:35,027][20381] Updated weights for policy 1, policy_version 1561621 (0.0010) [2023-12-27 16:28:35,081][20381] Updated weights for policy 1, policy_version 1561631 (0.0010) [2023-12-27 16:28:35,281][20347] Updated weights for policy 0, policy_version 1563284 (0.0008) [2023-12-27 16:28:35,331][20347] Updated weights for policy 0, policy_version 1563294 (0.0009) [2023-12-27 16:28:35,386][20347] Updated weights for policy 0, policy_version 1563304 (0.0007) [2023-12-27 16:28:35,703][20381] Updated weights for policy 1, policy_version 1561641 (0.0010) [2023-12-27 16:28:35,755][20381] Updated weights for policy 1, policy_version 1561651 (0.0007) [2023-12-27 16:28:35,802][20381] Updated weights for policy 1, policy_version 1561661 (0.0005) [2023-12-27 16:28:35,859][20381] Updated weights for policy 1, policy_version 1561671 (0.0005) [2023-12-27 16:28:36,010][20347] Updated weights for policy 0, policy_version 1563314 (0.0007) [2023-12-27 16:28:36,059][20347] Updated weights for policy 0, policy_version 1563324 (0.0009) [2023-12-27 16:28:36,116][20347] Updated weights for policy 0, policy_version 1563334 (0.0007) [2023-12-27 16:28:36,164][20347] Updated weights for policy 0, policy_version 1563344 (0.0007) [2023-12-27 16:28:36,441][20381] Updated weights for policy 1, policy_version 1561681 (0.0007) [2023-12-27 16:28:36,500][20381] Updated weights for policy 1, policy_version 1561691 (0.0007) [2023-12-27 16:28:36,564][20381] Updated weights for policy 1, policy_version 1561701 (0.0006) [2023-12-27 16:28:36,873][20347] Updated weights for policy 0, policy_version 1563354 (0.0007) [2023-12-27 16:28:36,921][20347] Updated weights for policy 0, policy_version 1563364 (0.0007) [2023-12-27 16:28:36,969][20347] Updated weights for policy 0, policy_version 1563374 (0.0007) [2023-12-27 16:28:37,182][20381] Updated weights for policy 1, policy_version 1561711 (0.0009) [2023-12-27 16:28:37,241][20381] Updated weights for policy 1, policy_version 1561721 (0.0011) [2023-12-27 16:28:37,303][20381] Updated weights for policy 1, policy_version 1561731 (0.0011) [2023-12-27 16:28:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 800145408. Throughput: 0: 10309.7, 1: 10185.2. Samples: 800109448. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:37,364][19170] Avg episode reward: [(0, '784.520'), (1, '873.240')] [2023-12-27 16:28:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001561736_399843328.pth... [2023-12-27 16:28:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001563376_400302080.pth... [2023-12-27 16:28:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001560520_399532032.pth [2023-12-27 16:28:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001562160_399990784.pth [2023-12-27 16:28:37,687][20347] Updated weights for policy 0, policy_version 1563384 (0.0009) [2023-12-27 16:28:37,745][20347] Updated weights for policy 0, policy_version 1563394 (0.0007) [2023-12-27 16:28:37,800][20347] Updated weights for policy 0, policy_version 1563404 (0.0008) [2023-12-27 16:28:37,907][20381] Updated weights for policy 1, policy_version 1561741 (0.0009) [2023-12-27 16:28:37,953][20381] Updated weights for policy 1, policy_version 1561751 (0.0006) [2023-12-27 16:28:37,998][20381] Updated weights for policy 1, policy_version 1561761 (0.0006) [2023-12-27 16:28:38,539][20347] Updated weights for policy 0, policy_version 1563414 (0.0008) [2023-12-27 16:28:38,597][20347] Updated weights for policy 0, policy_version 1563424 (0.0007) [2023-12-27 16:28:38,661][20347] Updated weights for policy 0, policy_version 1563434 (0.0008) [2023-12-27 16:28:38,674][20381] Updated weights for policy 1, policy_version 1561771 (0.0007) [2023-12-27 16:28:38,734][20381] Updated weights for policy 1, policy_version 1561781 (0.0007) [2023-12-27 16:28:38,791][20381] Updated weights for policy 1, policy_version 1561791 (0.0008) [2023-12-27 16:28:39,350][20347] Updated weights for policy 0, policy_version 1563444 (0.0007) [2023-12-27 16:28:39,414][20347] Updated weights for policy 0, policy_version 1563454 (0.0009) [2023-12-27 16:28:39,476][20347] Updated weights for policy 0, policy_version 1563464 (0.0009) [2023-12-27 16:28:39,478][20381] Updated weights for policy 1, policy_version 1561801 (0.0008) [2023-12-27 16:28:39,538][20381] Updated weights for policy 1, policy_version 1561811 (0.0011) [2023-12-27 16:28:39,601][20381] Updated weights for policy 1, policy_version 1561821 (0.0011) [2023-12-27 16:28:39,664][20381] Updated weights for policy 1, policy_version 1561831 (0.0011) [2023-12-27 16:28:40,151][20347] Updated weights for policy 0, policy_version 1563474 (0.0008) [2023-12-27 16:28:40,209][20347] Updated weights for policy 0, policy_version 1563484 (0.0007) [2023-12-27 16:28:40,268][20347] Updated weights for policy 0, policy_version 1563494 (0.0007) [2023-12-27 16:28:40,328][20347] Updated weights for policy 0, policy_version 1563504 (0.0009) [2023-12-27 16:28:40,346][20381] Updated weights for policy 1, policy_version 1561841 (0.0007) [2023-12-27 16:28:40,401][20381] Updated weights for policy 1, policy_version 1561851 (0.0007) [2023-12-27 16:28:40,453][20381] Updated weights for policy 1, policy_version 1561861 (0.0008) [2023-12-27 16:28:41,003][20347] Updated weights for policy 0, policy_version 1563514 (0.0008) [2023-12-27 16:28:41,066][20347] Updated weights for policy 0, policy_version 1563524 (0.0007) [2023-12-27 16:28:41,122][20347] Updated weights for policy 0, policy_version 1563534 (0.0008) [2023-12-27 16:28:41,149][20381] Updated weights for policy 1, policy_version 1561871 (0.0008) [2023-12-27 16:28:41,212][20381] Updated weights for policy 1, policy_version 1561881 (0.0008) [2023-12-27 16:28:41,272][20381] Updated weights for policy 1, policy_version 1561891 (0.0008) [2023-12-27 16:28:41,906][20347] Updated weights for policy 0, policy_version 1563544 (0.0007) [2023-12-27 16:28:41,974][20347] Updated weights for policy 0, policy_version 1563554 (0.0008) [2023-12-27 16:28:41,981][20381] Updated weights for policy 1, policy_version 1561901 (0.0007) [2023-12-27 16:28:42,033][20347] Updated weights for policy 0, policy_version 1563564 (0.0006) [2023-12-27 16:28:42,038][20381] Updated weights for policy 1, policy_version 1561911 (0.0006) [2023-12-27 16:28:42,100][20381] Updated weights for policy 1, policy_version 1561921 (0.0006) [2023-12-27 16:28:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 800243712. Throughput: 0: 10310.7, 1: 10227.6. Samples: 800233948. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:42,363][19170] Avg episode reward: [(0, '814.450'), (1, '824.760')] [2023-12-27 16:28:42,722][20347] Updated weights for policy 0, policy_version 1563574 (0.0009) [2023-12-27 16:28:42,754][20381] Updated weights for policy 1, policy_version 1561931 (0.0007) [2023-12-27 16:28:42,782][20347] Updated weights for policy 0, policy_version 1563584 (0.0011) [2023-12-27 16:28:42,812][20381] Updated weights for policy 1, policy_version 1561941 (0.0007) [2023-12-27 16:28:42,839][20347] Updated weights for policy 0, policy_version 1563594 (0.0010) [2023-12-27 16:28:42,867][20292] KL-divergence is very high: 236.5500 [2023-12-27 16:28:42,872][20381] Updated weights for policy 1, policy_version 1561951 (0.0005) [2023-12-27 16:28:42,879][20292] KL-divergence is very high: 429.5944 [2023-12-27 16:28:42,885][20292] KL-divergence is very high: 446.9272 [2023-12-27 16:28:42,891][20292] KL-divergence is very high: 326.6774 [2023-12-27 16:28:42,919][20292] KL-divergence is very high: 224.5242 [2023-12-27 16:28:43,472][20381] Updated weights for policy 1, policy_version 1561961 (0.0006) [2023-12-27 16:28:43,490][20292] KL-divergence is very high: 135.0523 [2023-12-27 16:28:43,531][20381] Updated weights for policy 1, policy_version 1561971 (0.0006) [2023-12-27 16:28:43,544][20347] Updated weights for policy 0, policy_version 1563604 (0.0008) [2023-12-27 16:28:43,585][20381] Updated weights for policy 1, policy_version 1561981 (0.0006) [2023-12-27 16:28:43,600][20347] Updated weights for policy 0, policy_version 1563614 (0.0008) [2023-12-27 16:28:43,626][20292] KL-divergence is very high: 107.0759 [2023-12-27 16:28:43,643][20381] Updated weights for policy 1, policy_version 1561991 (0.0006) [2023-12-27 16:28:43,658][20347] Updated weights for policy 0, policy_version 1563624 (0.0008) [2023-12-27 16:28:44,260][20381] Updated weights for policy 1, policy_version 1562001 (0.0010) [2023-12-27 16:28:44,303][20347] Updated weights for policy 0, policy_version 1563634 (0.0007) [2023-12-27 16:28:44,317][20381] Updated weights for policy 1, policy_version 1562011 (0.0010) [2023-12-27 16:28:44,365][20347] Updated weights for policy 0, policy_version 1563644 (0.0005) [2023-12-27 16:28:44,376][20381] Updated weights for policy 1, policy_version 1562021 (0.0009) [2023-12-27 16:28:44,419][20347] Updated weights for policy 0, policy_version 1563654 (0.0006) [2023-12-27 16:28:44,467][20347] Updated weights for policy 0, policy_version 1563664 (0.0008) [2023-12-27 16:28:45,070][20381] Updated weights for policy 1, policy_version 1562031 (0.0009) [2023-12-27 16:28:45,084][20347] Updated weights for policy 0, policy_version 1563674 (0.0008) [2023-12-27 16:28:45,123][20381] Updated weights for policy 1, policy_version 1562041 (0.0010) [2023-12-27 16:28:45,145][20347] Updated weights for policy 0, policy_version 1563684 (0.0008) [2023-12-27 16:28:45,176][20381] Updated weights for policy 1, policy_version 1562051 (0.0011) [2023-12-27 16:28:45,211][20347] Updated weights for policy 0, policy_version 1563694 (0.0008) [2023-12-27 16:28:45,893][20381] Updated weights for policy 1, policy_version 1562061 (0.0007) [2023-12-27 16:28:45,920][20347] Updated weights for policy 0, policy_version 1563704 (0.0005) [2023-12-27 16:28:45,944][20381] Updated weights for policy 1, policy_version 1562071 (0.0008) [2023-12-27 16:28:45,968][20347] Updated weights for policy 0, policy_version 1563714 (0.0010) [2023-12-27 16:28:45,992][20381] Updated weights for policy 1, policy_version 1562081 (0.0010) [2023-12-27 16:28:46,013][20347] Updated weights for policy 0, policy_version 1563724 (0.0010) [2023-12-27 16:28:46,572][20381] Updated weights for policy 1, policy_version 1562091 (0.0010) [2023-12-27 16:28:46,623][20381] Updated weights for policy 1, policy_version 1562101 (0.0010) [2023-12-27 16:28:46,657][20347] Updated weights for policy 0, policy_version 1563734 (0.0010) [2023-12-27 16:28:46,669][20381] Updated weights for policy 1, policy_version 1562111 (0.0010) [2023-12-27 16:28:46,715][20347] Updated weights for policy 0, policy_version 1563744 (0.0010) [2023-12-27 16:28:46,770][20347] Updated weights for policy 0, policy_version 1563754 (0.0010) [2023-12-27 16:28:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 800342016. Throughput: 0: 10321.8, 1: 10275.7. Samples: 800357436. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:47,363][19170] Avg episode reward: [(0, '794.260'), (1, '767.500')] [2023-12-27 16:28:47,364][20381] Updated weights for policy 1, policy_version 1562121 (0.0010) [2023-12-27 16:28:47,412][20381] Updated weights for policy 1, policy_version 1562131 (0.0010) [2023-12-27 16:28:47,429][20347] Updated weights for policy 0, policy_version 1563764 (0.0007) [2023-12-27 16:28:47,457][20381] Updated weights for policy 1, policy_version 1562141 (0.0010) [2023-12-27 16:28:47,478][20347] Updated weights for policy 0, policy_version 1563774 (0.0005) [2023-12-27 16:28:47,504][20381] Updated weights for policy 1, policy_version 1562151 (0.0008) [2023-12-27 16:28:47,525][20347] Updated weights for policy 0, policy_version 1563784 (0.0007) [2023-12-27 16:28:48,135][20381] Updated weights for policy 1, policy_version 1562161 (0.0007) [2023-12-27 16:28:48,176][20381] Updated weights for policy 1, policy_version 1562171 (0.0007) [2023-12-27 16:28:48,228][20381] Updated weights for policy 1, policy_version 1562181 (0.0008) [2023-12-27 16:28:48,250][20347] Updated weights for policy 0, policy_version 1563794 (0.0007) [2023-12-27 16:28:48,305][20347] Updated weights for policy 0, policy_version 1563804 (0.0008) [2023-12-27 16:28:48,361][20347] Updated weights for policy 0, policy_version 1563814 (0.0008) [2023-12-27 16:28:48,417][20347] Updated weights for policy 0, policy_version 1563824 (0.0006) [2023-12-27 16:28:48,884][20381] Updated weights for policy 1, policy_version 1562191 (0.0008) [2023-12-27 16:28:48,938][20381] Updated weights for policy 1, policy_version 1562201 (0.0006) [2023-12-27 16:28:48,992][20381] Updated weights for policy 1, policy_version 1562211 (0.0006) [2023-12-27 16:28:49,105][20347] Updated weights for policy 0, policy_version 1563834 (0.0010) [2023-12-27 16:28:49,157][20347] Updated weights for policy 0, policy_version 1563844 (0.0010) [2023-12-27 16:28:49,222][20347] Updated weights for policy 0, policy_version 1563854 (0.0013) [2023-12-27 16:28:49,615][20381] Updated weights for policy 1, policy_version 1562221 (0.0008) [2023-12-27 16:28:49,672][20381] Updated weights for policy 1, policy_version 1562231 (0.0008) [2023-12-27 16:28:49,726][20381] Updated weights for policy 1, policy_version 1562241 (0.0008) [2023-12-27 16:28:49,847][20347] Updated weights for policy 0, policy_version 1563864 (0.0008) [2023-12-27 16:28:49,912][20347] Updated weights for policy 0, policy_version 1563874 (0.0007) [2023-12-27 16:28:49,978][20347] Updated weights for policy 0, policy_version 1563884 (0.0006) [2023-12-27 16:28:50,502][20381] Updated weights for policy 1, policy_version 1562251 (0.0008) [2023-12-27 16:28:50,551][20381] Updated weights for policy 1, policy_version 1562261 (0.0005) [2023-12-27 16:28:50,620][20381] Updated weights for policy 1, policy_version 1562271 (0.0008) [2023-12-27 16:28:50,635][20347] Updated weights for policy 0, policy_version 1563894 (0.0007) [2023-12-27 16:28:50,700][20347] Updated weights for policy 0, policy_version 1563904 (0.0007) [2023-12-27 16:28:50,764][20347] Updated weights for policy 0, policy_version 1563914 (0.0008) [2023-12-27 16:28:51,297][20381] Updated weights for policy 1, policy_version 1562281 (0.0006) [2023-12-27 16:28:51,357][20381] Updated weights for policy 1, policy_version 1562291 (0.0008) [2023-12-27 16:28:51,423][20381] Updated weights for policy 1, policy_version 1562301 (0.0008) [2023-12-27 16:28:51,436][20347] Updated weights for policy 0, policy_version 1563924 (0.0007) [2023-12-27 16:28:51,486][20381] Updated weights for policy 1, policy_version 1562311 (0.0007) [2023-12-27 16:28:51,488][20347] Updated weights for policy 0, policy_version 1563934 (0.0006) [2023-12-27 16:28:51,544][20347] Updated weights for policy 0, policy_version 1563944 (0.0007) [2023-12-27 16:28:52,222][20381] Updated weights for policy 1, policy_version 1562321 (0.0010) [2023-12-27 16:28:52,242][20347] Updated weights for policy 0, policy_version 1563954 (0.0007) [2023-12-27 16:28:52,282][20381] Updated weights for policy 1, policy_version 1562331 (0.0011) [2023-12-27 16:28:52,308][20347] Updated weights for policy 0, policy_version 1563964 (0.0007) [2023-12-27 16:28:52,340][20381] Updated weights for policy 1, policy_version 1562341 (0.0010) [2023-12-27 16:28:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 800448512. Throughput: 0: 10318.8, 1: 10277.4. Samples: 800419164. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:52,363][19170] Avg episode reward: [(0, '794.180'), (1, '805.990')] [2023-12-27 16:28:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001562344_399998976.pth... [2023-12-27 16:28:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001561128_399687680.pth [2023-12-27 16:28:52,374][20347] Updated weights for policy 0, policy_version 1563974 (0.0008) [2023-12-27 16:28:52,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001563984_400457728.pth... [2023-12-27 16:28:52,432][20347] Updated weights for policy 0, policy_version 1563984 (0.0008) [2023-12-27 16:28:52,433][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001562768_400146432.pth [2023-12-27 16:28:52,996][20381] Updated weights for policy 1, policy_version 1562351 (0.0010) [2023-12-27 16:28:53,049][20381] Updated weights for policy 1, policy_version 1562361 (0.0009) [2023-12-27 16:28:53,050][20347] Updated weights for policy 0, policy_version 1563994 (0.0010) [2023-12-27 16:28:53,101][20381] Updated weights for policy 1, policy_version 1562371 (0.0005) [2023-12-27 16:28:53,108][20347] Updated weights for policy 0, policy_version 1564004 (0.0010) [2023-12-27 16:28:53,174][20347] Updated weights for policy 0, policy_version 1564014 (0.0010) [2023-12-27 16:28:53,742][20381] Updated weights for policy 1, policy_version 1562381 (0.0006) [2023-12-27 16:28:53,801][20381] Updated weights for policy 1, policy_version 1562391 (0.0007) [2023-12-27 16:28:53,822][20347] Updated weights for policy 0, policy_version 1564024 (0.0007) [2023-12-27 16:28:53,861][20381] Updated weights for policy 1, policy_version 1562401 (0.0006) [2023-12-27 16:28:53,875][20347] Updated weights for policy 0, policy_version 1564034 (0.0007) [2023-12-27 16:28:53,929][20347] Updated weights for policy 0, policy_version 1564044 (0.0006) [2023-12-27 16:28:54,470][20381] Updated weights for policy 1, policy_version 1562411 (0.0007) [2023-12-27 16:28:54,523][20381] Updated weights for policy 1, policy_version 1562421 (0.0005) [2023-12-27 16:28:54,575][20381] Updated weights for policy 1, policy_version 1562431 (0.0008) [2023-12-27 16:28:54,649][20347] Updated weights for policy 0, policy_version 1564054 (0.0008) [2023-12-27 16:28:54,697][20347] Updated weights for policy 0, policy_version 1564064 (0.0005) [2023-12-27 16:28:54,750][20347] Updated weights for policy 0, policy_version 1564074 (0.0005) [2023-12-27 16:28:55,279][20381] Updated weights for policy 1, policy_version 1562441 (0.0007) [2023-12-27 16:28:55,332][20381] Updated weights for policy 1, policy_version 1562451 (0.0008) [2023-12-27 16:28:55,344][20292] KL-divergence is very high: 122.5199 [2023-12-27 16:28:55,355][20292] KL-divergence is very high: 163.4477 [2023-12-27 16:28:55,366][20292] KL-divergence is very high: 168.1827 [2023-12-27 16:28:55,370][20347] Updated weights for policy 0, policy_version 1564084 (0.0005) [2023-12-27 16:28:55,377][20292] KL-divergence is very high: 174.4763 [2023-12-27 16:28:55,386][20381] Updated weights for policy 1, policy_version 1562461 (0.0010) [2023-12-27 16:28:55,387][20292] KL-divergence is very high: 153.6089 [2023-12-27 16:28:55,398][20292] KL-divergence is very high: 130.3266 [2023-12-27 16:28:55,421][20347] Updated weights for policy 0, policy_version 1564094 (0.0005) [2023-12-27 16:28:55,441][20381] Updated weights for policy 1, policy_version 1562471 (0.0010) [2023-12-27 16:28:55,475][20347] Updated weights for policy 0, policy_version 1564104 (0.0006) [2023-12-27 16:28:56,150][20381] Updated weights for policy 1, policy_version 1562481 (0.0008) [2023-12-27 16:28:56,171][20347] Updated weights for policy 0, policy_version 1564114 (0.0007) [2023-12-27 16:28:56,204][20381] Updated weights for policy 1, policy_version 1562491 (0.0006) [2023-12-27 16:28:56,228][20347] Updated weights for policy 0, policy_version 1564124 (0.0007) [2023-12-27 16:28:56,262][20381] Updated weights for policy 1, policy_version 1562501 (0.0008) [2023-12-27 16:28:56,291][20347] Updated weights for policy 0, policy_version 1564134 (0.0008) [2023-12-27 16:28:56,340][20347] Updated weights for policy 0, policy_version 1564144 (0.0008) [2023-12-27 16:28:56,861][20381] Updated weights for policy 1, policy_version 1562511 (0.0006) [2023-12-27 16:28:56,896][20347] Updated weights for policy 0, policy_version 1564154 (0.0010) [2023-12-27 16:28:56,907][20381] Updated weights for policy 1, policy_version 1562521 (0.0005) [2023-12-27 16:28:56,943][20347] Updated weights for policy 0, policy_version 1564164 (0.0007) [2023-12-27 16:28:56,952][20381] Updated weights for policy 1, policy_version 1562531 (0.0005) [2023-12-27 16:28:56,996][20347] Updated weights for policy 0, policy_version 1564174 (0.0008) [2023-12-27 16:28:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 800555008. Throughput: 0: 10268.4, 1: 10292.6. Samples: 800541560. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:28:57,363][19170] Avg episode reward: [(0, '843.370'), (1, '760.240')] [2023-12-27 16:28:57,475][20381] Updated weights for policy 1, policy_version 1562541 (0.0008) [2023-12-27 16:28:57,521][20381] Updated weights for policy 1, policy_version 1562551 (0.0010) [2023-12-27 16:28:57,568][20381] Updated weights for policy 1, policy_version 1562561 (0.0008) [2023-12-27 16:28:57,584][20347] Updated weights for policy 0, policy_version 1564184 (0.0007) [2023-12-27 16:28:57,636][20347] Updated weights for policy 0, policy_version 1564194 (0.0008) [2023-12-27 16:28:57,696][20347] Updated weights for policy 0, policy_version 1564204 (0.0007) [2023-12-27 16:28:58,158][20381] Updated weights for policy 1, policy_version 1562571 (0.0007) [2023-12-27 16:28:58,220][20381] Updated weights for policy 1, policy_version 1562581 (0.0006) [2023-12-27 16:28:58,276][20381] Updated weights for policy 1, policy_version 1562591 (0.0008) [2023-12-27 16:28:58,319][20347] Updated weights for policy 0, policy_version 1564214 (0.0006) [2023-12-27 16:28:58,382][20347] Updated weights for policy 0, policy_version 1564224 (0.0008) [2023-12-27 16:28:58,445][20347] Updated weights for policy 0, policy_version 1564234 (0.0007) [2023-12-27 16:28:59,034][20381] Updated weights for policy 1, policy_version 1562601 (0.0008) [2023-12-27 16:28:59,101][20381] Updated weights for policy 1, policy_version 1562611 (0.0009) [2023-12-27 16:28:59,160][20381] Updated weights for policy 1, policy_version 1562621 (0.0008) [2023-12-27 16:28:59,219][20381] Updated weights for policy 1, policy_version 1562631 (0.0008) [2023-12-27 16:28:59,222][20347] Updated weights for policy 0, policy_version 1564244 (0.0008) [2023-12-27 16:28:59,287][20347] Updated weights for policy 0, policy_version 1564254 (0.0008) [2023-12-27 16:28:59,353][20347] Updated weights for policy 0, policy_version 1564264 (0.0008) [2023-12-27 16:28:59,958][20381] Updated weights for policy 1, policy_version 1562641 (0.0008) [2023-12-27 16:29:00,009][20381] Updated weights for policy 1, policy_version 1562651 (0.0007) [2023-12-27 16:29:00,059][20381] Updated weights for policy 1, policy_version 1562661 (0.0008) [2023-12-27 16:29:00,084][20347] Updated weights for policy 0, policy_version 1564274 (0.0007) [2023-12-27 16:29:00,141][20347] Updated weights for policy 0, policy_version 1564284 (0.0007) [2023-12-27 16:29:00,200][20347] Updated weights for policy 0, policy_version 1564294 (0.0008) [2023-12-27 16:29:00,255][20347] Updated weights for policy 0, policy_version 1564304 (0.0008) [2023-12-27 16:29:00,777][20381] Updated weights for policy 1, policy_version 1562671 (0.0009) [2023-12-27 16:29:00,834][20381] Updated weights for policy 1, policy_version 1562681 (0.0008) [2023-12-27 16:29:00,885][20347] Updated weights for policy 0, policy_version 1564314 (0.0005) [2023-12-27 16:29:00,887][20381] Updated weights for policy 1, policy_version 1562691 (0.0007) [2023-12-27 16:29:00,896][20292] KL-divergence is very high: 116.0703 [2023-12-27 16:29:00,902][20292] KL-divergence is very high: 124.6096 [2023-12-27 16:29:00,938][20347] Updated weights for policy 0, policy_version 1564324 (0.0007) [2023-12-27 16:29:00,986][20347] Updated weights for policy 0, policy_version 1564334 (0.0007) [2023-12-27 16:29:01,643][20381] Updated weights for policy 1, policy_version 1562701 (0.0008) [2023-12-27 16:29:01,656][20347] Updated weights for policy 0, policy_version 1564344 (0.0008) [2023-12-27 16:29:01,694][20381] Updated weights for policy 1, policy_version 1562711 (0.0005) [2023-12-27 16:29:01,708][20347] Updated weights for policy 0, policy_version 1564354 (0.0008) [2023-12-27 16:29:01,757][20381] Updated weights for policy 1, policy_version 1562721 (0.0007) [2023-12-27 16:29:01,772][20347] Updated weights for policy 0, policy_version 1564364 (0.0008) [2023-12-27 16:29:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 800653312. Throughput: 0: 10236.0, 1: 10327.0. Samples: 800665672. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:02,363][19170] Avg episode reward: [(0, '794.090'), (1, '748.090')] [2023-12-27 16:29:02,456][20347] Updated weights for policy 0, policy_version 1564374 (0.0007) [2023-12-27 16:29:02,498][20381] Updated weights for policy 1, policy_version 1562731 (0.0008) [2023-12-27 16:29:02,518][20347] Updated weights for policy 0, policy_version 1564384 (0.0007) [2023-12-27 16:29:02,558][20381] Updated weights for policy 1, policy_version 1562741 (0.0006) [2023-12-27 16:29:02,564][20347] Updated weights for policy 0, policy_version 1564394 (0.0010) [2023-12-27 16:29:02,618][20381] Updated weights for policy 1, policy_version 1562751 (0.0008) [2023-12-27 16:29:03,137][20347] Updated weights for policy 0, policy_version 1564404 (0.0008) [2023-12-27 16:29:03,187][20347] Updated weights for policy 0, policy_version 1564414 (0.0007) [2023-12-27 16:29:03,237][20381] Updated weights for policy 1, policy_version 1562761 (0.0007) [2023-12-27 16:29:03,238][20347] Updated weights for policy 0, policy_version 1564424 (0.0010) [2023-12-27 16:29:03,285][20381] Updated weights for policy 1, policy_version 1562771 (0.0010) [2023-12-27 16:29:03,330][20381] Updated weights for policy 1, policy_version 1562781 (0.0010) [2023-12-27 16:29:03,385][20381] Updated weights for policy 1, policy_version 1562791 (0.0010) [2023-12-27 16:29:03,794][20347] Updated weights for policy 0, policy_version 1564434 (0.0010) [2023-12-27 16:29:03,852][20347] Updated weights for policy 0, policy_version 1564444 (0.0010) [2023-12-27 16:29:03,909][20347] Updated weights for policy 0, policy_version 1564454 (0.0007) [2023-12-27 16:29:03,967][20347] Updated weights for policy 0, policy_version 1564464 (0.0005) [2023-12-27 16:29:04,072][20381] Updated weights for policy 1, policy_version 1562801 (0.0011) [2023-12-27 16:29:04,134][20381] Updated weights for policy 1, policy_version 1562811 (0.0011) [2023-12-27 16:29:04,195][20381] Updated weights for policy 1, policy_version 1562821 (0.0011) [2023-12-27 16:29:04,582][20347] Updated weights for policy 0, policy_version 1564474 (0.0008) [2023-12-27 16:29:04,641][20347] Updated weights for policy 0, policy_version 1564484 (0.0005) [2023-12-27 16:29:04,695][20347] Updated weights for policy 0, policy_version 1564494 (0.0005) [2023-12-27 16:29:04,866][20381] Updated weights for policy 1, policy_version 1562831 (0.0011) [2023-12-27 16:29:04,925][20381] Updated weights for policy 1, policy_version 1562841 (0.0010) [2023-12-27 16:29:04,977][20381] Updated weights for policy 1, policy_version 1562851 (0.0010) [2023-12-27 16:29:05,300][20347] Updated weights for policy 0, policy_version 1564504 (0.0007) [2023-12-27 16:29:05,354][20347] Updated weights for policy 0, policy_version 1564514 (0.0007) [2023-12-27 16:29:05,412][20347] Updated weights for policy 0, policy_version 1564524 (0.0007) [2023-12-27 16:29:05,632][20381] Updated weights for policy 1, policy_version 1562861 (0.0011) [2023-12-27 16:29:05,677][20381] Updated weights for policy 1, policy_version 1562871 (0.0010) [2023-12-27 16:29:05,721][20381] Updated weights for policy 1, policy_version 1562881 (0.0010) [2023-12-27 16:29:06,055][20347] Updated weights for policy 0, policy_version 1564534 (0.0007) [2023-12-27 16:29:06,116][20347] Updated weights for policy 0, policy_version 1564544 (0.0008) [2023-12-27 16:29:06,185][20347] Updated weights for policy 0, policy_version 1564554 (0.0009) [2023-12-27 16:29:06,416][20381] Updated weights for policy 1, policy_version 1562891 (0.0010) [2023-12-27 16:29:06,476][20381] Updated weights for policy 1, policy_version 1562901 (0.0008) [2023-12-27 16:29:06,538][20381] Updated weights for policy 1, policy_version 1562911 (0.0009) [2023-12-27 16:29:06,897][20347] Updated weights for policy 0, policy_version 1564564 (0.0008) [2023-12-27 16:29:06,954][20347] Updated weights for policy 0, policy_version 1564574 (0.0007) [2023-12-27 16:29:07,005][20347] Updated weights for policy 0, policy_version 1564584 (0.0007) [2023-12-27 16:29:07,212][20381] Updated weights for policy 1, policy_version 1562921 (0.0009) [2023-12-27 16:29:07,271][20381] Updated weights for policy 1, policy_version 1562931 (0.0005) [2023-12-27 16:29:07,326][20381] Updated weights for policy 1, policy_version 1562941 (0.0005) [2023-12-27 16:29:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 800759808. Throughput: 0: 10311.7, 1: 10346.7. Samples: 800730820. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:07,363][19170] Avg episode reward: [(0, '804.060'), (1, '873.030')] [2023-12-27 16:29:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001564592_400613376.pth... [2023-12-27 16:29:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001563376_400302080.pth [2023-12-27 16:29:07,383][20381] Updated weights for policy 1, policy_version 1562951 (0.0006) [2023-12-27 16:29:07,386][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001562952_400154624.pth... [2023-12-27 16:29:07,390][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001561736_399843328.pth [2023-12-27 16:29:07,734][20347] Updated weights for policy 0, policy_version 1564594 (0.0007) [2023-12-27 16:29:07,789][20347] Updated weights for policy 0, policy_version 1564604 (0.0007) [2023-12-27 16:29:07,848][20347] Updated weights for policy 0, policy_version 1564614 (0.0007) [2023-12-27 16:29:07,905][20347] Updated weights for policy 0, policy_version 1564624 (0.0007) [2023-12-27 16:29:07,993][20381] Updated weights for policy 1, policy_version 1562961 (0.0010) [2023-12-27 16:29:08,051][20381] Updated weights for policy 1, policy_version 1562971 (0.0011) [2023-12-27 16:29:08,113][20381] Updated weights for policy 1, policy_version 1562981 (0.0011) [2023-12-27 16:29:08,566][20347] Updated weights for policy 0, policy_version 1564634 (0.0006) [2023-12-27 16:29:08,620][20347] Updated weights for policy 0, policy_version 1564644 (0.0007) [2023-12-27 16:29:08,672][20347] Updated weights for policy 0, policy_version 1564654 (0.0010) [2023-12-27 16:29:08,864][20381] Updated weights for policy 1, policy_version 1562991 (0.0010) [2023-12-27 16:29:08,926][20381] Updated weights for policy 1, policy_version 1563001 (0.0006) [2023-12-27 16:29:08,985][20381] Updated weights for policy 1, policy_version 1563011 (0.0007) [2023-12-27 16:29:09,284][20347] Updated weights for policy 0, policy_version 1564664 (0.0009) [2023-12-27 16:29:09,339][20291] KL-divergence is very high: 123.0979 [2023-12-27 16:29:09,345][20347] Updated weights for policy 0, policy_version 1564674 (0.0009) [2023-12-27 16:29:09,398][20291] KL-divergence is very high: 122.0156 [2023-12-27 16:29:09,419][20347] Updated weights for policy 0, policy_version 1564684 (0.0008) [2023-12-27 16:29:09,662][20381] Updated weights for policy 1, policy_version 1563021 (0.0008) [2023-12-27 16:29:09,715][20381] Updated weights for policy 1, policy_version 1563031 (0.0006) [2023-12-27 16:29:09,782][20381] Updated weights for policy 1, policy_version 1563041 (0.0008) [2023-12-27 16:29:10,150][20291] KL-divergence is very high: 103.2888 [2023-12-27 16:29:10,187][20347] Updated weights for policy 0, policy_version 1564694 (0.0009) [2023-12-27 16:29:10,247][20347] Updated weights for policy 0, policy_version 1564704 (0.0008) [2023-12-27 16:29:10,306][20347] Updated weights for policy 0, policy_version 1564714 (0.0008) [2023-12-27 16:29:10,483][20381] Updated weights for policy 1, policy_version 1563051 (0.0007) [2023-12-27 16:29:10,539][20381] Updated weights for policy 1, policy_version 1563061 (0.0005) [2023-12-27 16:29:10,586][20381] Updated weights for policy 1, policy_version 1563071 (0.0005) [2023-12-27 16:29:10,952][20347] Updated weights for policy 0, policy_version 1564724 (0.0007) [2023-12-27 16:29:11,007][20347] Updated weights for policy 0, policy_version 1564734 (0.0005) [2023-12-27 16:29:11,071][20347] Updated weights for policy 0, policy_version 1564744 (0.0008) [2023-12-27 16:29:11,255][20381] Updated weights for policy 1, policy_version 1563081 (0.0006) [2023-12-27 16:29:11,315][20381] Updated weights for policy 1, policy_version 1563091 (0.0008) [2023-12-27 16:29:11,370][20381] Updated weights for policy 1, policy_version 1563101 (0.0008) [2023-12-27 16:29:11,443][20381] Updated weights for policy 1, policy_version 1563111 (0.0008) [2023-12-27 16:29:11,747][20347] Updated weights for policy 0, policy_version 1564754 (0.0008) [2023-12-27 16:29:11,810][20347] Updated weights for policy 0, policy_version 1564764 (0.0007) [2023-12-27 16:29:11,870][20347] Updated weights for policy 0, policy_version 1564774 (0.0008) [2023-12-27 16:29:11,934][20347] Updated weights for policy 0, policy_version 1564784 (0.0007) [2023-12-27 16:29:12,171][20381] Updated weights for policy 1, policy_version 1563121 (0.0005) [2023-12-27 16:29:12,230][20381] Updated weights for policy 1, policy_version 1563131 (0.0008) [2023-12-27 16:29:12,296][20381] Updated weights for policy 1, policy_version 1563141 (0.0008) [2023-12-27 16:29:12,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 800866304. Throughput: 0: 10340.4, 1: 10375.3. Samples: 800854072. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:12,363][19170] Avg episode reward: [(0, '833.500'), (1, '843.740')] [2023-12-27 16:29:12,633][20347] Updated weights for policy 0, policy_version 1564794 (0.0008) [2023-12-27 16:29:12,693][20347] Updated weights for policy 0, policy_version 1564804 (0.0008) [2023-12-27 16:29:12,743][20347] Updated weights for policy 0, policy_version 1564814 (0.0007) [2023-12-27 16:29:12,976][20381] Updated weights for policy 1, policy_version 1563151 (0.0007) [2023-12-27 16:29:13,031][20381] Updated weights for policy 1, policy_version 1563161 (0.0008) [2023-12-27 16:29:13,074][20381] Updated weights for policy 1, policy_version 1563171 (0.0007) [2023-12-27 16:29:13,446][20347] Updated weights for policy 0, policy_version 1564824 (0.0008) [2023-12-27 16:29:13,502][20347] Updated weights for policy 0, policy_version 1564834 (0.0007) [2023-12-27 16:29:13,552][20347] Updated weights for policy 0, policy_version 1564844 (0.0008) [2023-12-27 16:29:13,736][20381] Updated weights for policy 1, policy_version 1563181 (0.0007) [2023-12-27 16:29:13,780][20381] Updated weights for policy 1, policy_version 1563191 (0.0005) [2023-12-27 16:29:13,824][20381] Updated weights for policy 1, policy_version 1563201 (0.0005) [2023-12-27 16:29:14,274][20347] Updated weights for policy 0, policy_version 1564854 (0.0007) [2023-12-27 16:29:14,334][20347] Updated weights for policy 0, policy_version 1564864 (0.0007) [2023-12-27 16:29:14,390][20347] Updated weights for policy 0, policy_version 1564874 (0.0006) [2023-12-27 16:29:14,467][20381] Updated weights for policy 1, policy_version 1563211 (0.0007) [2023-12-27 16:29:14,512][20381] Updated weights for policy 1, policy_version 1563221 (0.0010) [2023-12-27 16:29:14,561][20381] Updated weights for policy 1, policy_version 1563231 (0.0010) [2023-12-27 16:29:14,992][20347] Updated weights for policy 0, policy_version 1564884 (0.0005) [2023-12-27 16:29:15,050][20347] Updated weights for policy 0, policy_version 1564894 (0.0005) [2023-12-27 16:29:15,108][20347] Updated weights for policy 0, policy_version 1564904 (0.0007) [2023-12-27 16:29:15,292][20381] Updated weights for policy 1, policy_version 1563241 (0.0010) [2023-12-27 16:29:15,355][20381] Updated weights for policy 1, policy_version 1563251 (0.0009) [2023-12-27 16:29:15,404][20381] Updated weights for policy 1, policy_version 1563261 (0.0007) [2023-12-27 16:29:15,460][20381] Updated weights for policy 1, policy_version 1563271 (0.0009) [2023-12-27 16:29:15,775][20347] Updated weights for policy 0, policy_version 1564914 (0.0007) [2023-12-27 16:29:15,836][20347] Updated weights for policy 0, policy_version 1564924 (0.0008) [2023-12-27 16:29:15,898][20347] Updated weights for policy 0, policy_version 1564934 (0.0008) [2023-12-27 16:29:15,959][20347] Updated weights for policy 0, policy_version 1564944 (0.0007) [2023-12-27 16:29:16,139][20381] Updated weights for policy 1, policy_version 1563281 (0.0011) [2023-12-27 16:29:16,201][20381] Updated weights for policy 1, policy_version 1563291 (0.0011) [2023-12-27 16:29:16,250][20381] Updated weights for policy 1, policy_version 1563301 (0.0010) [2023-12-27 16:29:16,644][20347] Updated weights for policy 0, policy_version 1564954 (0.0007) [2023-12-27 16:29:16,698][20347] Updated weights for policy 0, policy_version 1564964 (0.0006) [2023-12-27 16:29:16,747][20347] Updated weights for policy 0, policy_version 1564974 (0.0005) [2023-12-27 16:29:16,917][20381] Updated weights for policy 1, policy_version 1563311 (0.0008) [2023-12-27 16:29:16,972][20381] Updated weights for policy 1, policy_version 1563321 (0.0009) [2023-12-27 16:29:17,020][20381] Updated weights for policy 1, policy_version 1563331 (0.0008) [2023-12-27 16:29:17,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 800964608. Throughput: 0: 10294.8, 1: 10358.9. Samples: 800977872. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:17,363][19170] Avg episode reward: [(0, '882.800'), (1, '853.490')] [2023-12-27 16:29:17,371][20347] Updated weights for policy 0, policy_version 1564984 (0.0007) [2023-12-27 16:29:17,424][20347] Updated weights for policy 0, policy_version 1564994 (0.0007) [2023-12-27 16:29:17,472][20347] Updated weights for policy 0, policy_version 1565004 (0.0008) [2023-12-27 16:29:17,644][20381] Updated weights for policy 1, policy_version 1563341 (0.0007) [2023-12-27 16:29:17,700][20381] Updated weights for policy 1, policy_version 1563351 (0.0007) [2023-12-27 16:29:17,764][20381] Updated weights for policy 1, policy_version 1563361 (0.0007) [2023-12-27 16:29:18,162][20347] Updated weights for policy 0, policy_version 1565014 (0.0008) [2023-12-27 16:29:18,218][20347] Updated weights for policy 0, policy_version 1565024 (0.0008) [2023-12-27 16:29:18,285][20347] Updated weights for policy 0, policy_version 1565034 (0.0006) [2023-12-27 16:29:18,399][20381] Updated weights for policy 1, policy_version 1563371 (0.0007) [2023-12-27 16:29:18,465][20381] Updated weights for policy 1, policy_version 1563381 (0.0006) [2023-12-27 16:29:18,529][20381] Updated weights for policy 1, policy_version 1563391 (0.0005) [2023-12-27 16:29:19,006][20347] Updated weights for policy 0, policy_version 1565044 (0.0007) [2023-12-27 16:29:19,060][20381] Updated weights for policy 1, policy_version 1563401 (0.0006) [2023-12-27 16:29:19,060][20347] Updated weights for policy 0, policy_version 1565054 (0.0008) [2023-12-27 16:29:19,113][20381] Updated weights for policy 1, policy_version 1563411 (0.0011) [2023-12-27 16:29:19,114][20347] Updated weights for policy 0, policy_version 1565064 (0.0005) [2023-12-27 16:29:19,167][20381] Updated weights for policy 1, policy_version 1563421 (0.0010) [2023-12-27 16:29:19,226][20381] Updated weights for policy 1, policy_version 1563431 (0.0011) [2023-12-27 16:29:19,850][20347] Updated weights for policy 0, policy_version 1565074 (0.0006) [2023-12-27 16:29:19,906][20347] Updated weights for policy 0, policy_version 1565084 (0.0006) [2023-12-27 16:29:19,973][20347] Updated weights for policy 0, policy_version 1565094 (0.0007) [2023-12-27 16:29:19,975][20381] Updated weights for policy 1, policy_version 1563441 (0.0008) [2023-12-27 16:29:20,036][20381] Updated weights for policy 1, policy_version 1563451 (0.0007) [2023-12-27 16:29:20,037][20347] Updated weights for policy 0, policy_version 1565104 (0.0006) [2023-12-27 16:29:20,091][20381] Updated weights for policy 1, policy_version 1563461 (0.0008) [2023-12-27 16:29:20,661][20347] Updated weights for policy 0, policy_version 1565114 (0.0011) [2023-12-27 16:29:20,720][20347] Updated weights for policy 0, policy_version 1565124 (0.0010) [2023-12-27 16:29:20,781][20347] Updated weights for policy 0, policy_version 1565134 (0.0010) [2023-12-27 16:29:20,832][20381] Updated weights for policy 1, policy_version 1563471 (0.0009) [2023-12-27 16:29:20,895][20381] Updated weights for policy 1, policy_version 1563481 (0.0011) [2023-12-27 16:29:20,958][20381] Updated weights for policy 1, policy_version 1563491 (0.0011) [2023-12-27 16:29:21,509][20347] Updated weights for policy 0, policy_version 1565144 (0.0006) [2023-12-27 16:29:21,557][20347] Updated weights for policy 0, policy_version 1565154 (0.0006) [2023-12-27 16:29:21,606][20347] Updated weights for policy 0, policy_version 1565164 (0.0006) [2023-12-27 16:29:21,673][20381] Updated weights for policy 1, policy_version 1563501 (0.0009) [2023-12-27 16:29:21,736][20381] Updated weights for policy 1, policy_version 1563511 (0.0008) [2023-12-27 16:29:21,802][20381] Updated weights for policy 1, policy_version 1563521 (0.0008) [2023-12-27 16:29:22,314][20347] Updated weights for policy 0, policy_version 1565174 (0.0007) [2023-12-27 16:29:22,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 801062912. Throughput: 0: 10260.8, 1: 10382.5. Samples: 801038388. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:22,363][19170] Avg episode reward: [(0, '863.360'), (1, '765.450')] [2023-12-27 16:29:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001563528_400302080.pth... [2023-12-27 16:29:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001562344_399998976.pth [2023-12-27 16:29:22,394][20347] Updated weights for policy 0, policy_version 1565184 (0.0007) [2023-12-27 16:29:22,455][20347] Updated weights for policy 0, policy_version 1565194 (0.0007) [2023-12-27 16:29:22,488][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001565200_400769024.pth... [2023-12-27 16:29:22,492][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001563984_400457728.pth [2023-12-27 16:29:22,570][20381] Updated weights for policy 1, policy_version 1563531 (0.0008) [2023-12-27 16:29:22,630][20381] Updated weights for policy 1, policy_version 1563541 (0.0006) [2023-12-27 16:29:22,691][20381] Updated weights for policy 1, policy_version 1563551 (0.0008) [2023-12-27 16:29:23,115][20347] Updated weights for policy 0, policy_version 1565204 (0.0008) [2023-12-27 16:29:23,173][20347] Updated weights for policy 0, policy_version 1565214 (0.0008) [2023-12-27 16:29:23,232][20347] Updated weights for policy 0, policy_version 1565224 (0.0008) [2023-12-27 16:29:23,317][20381] Updated weights for policy 1, policy_version 1563561 (0.0008) [2023-12-27 16:29:23,368][20381] Updated weights for policy 1, policy_version 1563571 (0.0007) [2023-12-27 16:29:23,420][20381] Updated weights for policy 1, policy_version 1563581 (0.0008) [2023-12-27 16:29:23,470][20381] Updated weights for policy 1, policy_version 1563591 (0.0008) [2023-12-27 16:29:23,896][20347] Updated weights for policy 0, policy_version 1565234 (0.0008) [2023-12-27 16:29:23,945][20347] Updated weights for policy 0, policy_version 1565244 (0.0007) [2023-12-27 16:29:23,994][20347] Updated weights for policy 0, policy_version 1565254 (0.0007) [2023-12-27 16:29:24,052][20347] Updated weights for policy 0, policy_version 1565264 (0.0008) [2023-12-27 16:29:24,140][20381] Updated weights for policy 1, policy_version 1563601 (0.0007) [2023-12-27 16:29:24,194][20381] Updated weights for policy 1, policy_version 1563611 (0.0008) [2023-12-27 16:29:24,249][20381] Updated weights for policy 1, policy_version 1563621 (0.0009) [2023-12-27 16:29:24,670][20347] Updated weights for policy 0, policy_version 1565274 (0.0006) [2023-12-27 16:29:24,722][20347] Updated weights for policy 0, policy_version 1565284 (0.0006) [2023-12-27 16:29:24,773][20347] Updated weights for policy 0, policy_version 1565294 (0.0005) [2023-12-27 16:29:24,969][20381] Updated weights for policy 1, policy_version 1563631 (0.0008) [2023-12-27 16:29:25,024][20381] Updated weights for policy 1, policy_version 1563641 (0.0008) [2023-12-27 16:29:25,081][20381] Updated weights for policy 1, policy_version 1563651 (0.0008) [2023-12-27 16:29:25,332][20347] Updated weights for policy 0, policy_version 1565304 (0.0006) [2023-12-27 16:29:25,387][20347] Updated weights for policy 0, policy_version 1565314 (0.0007) [2023-12-27 16:29:25,441][20347] Updated weights for policy 0, policy_version 1565324 (0.0007) [2023-12-27 16:29:25,775][20381] Updated weights for policy 1, policy_version 1563661 (0.0007) [2023-12-27 16:29:25,832][20381] Updated weights for policy 1, policy_version 1563671 (0.0005) [2023-12-27 16:29:25,885][20381] Updated weights for policy 1, policy_version 1563681 (0.0007) [2023-12-27 16:29:26,135][20347] Updated weights for policy 0, policy_version 1565334 (0.0007) [2023-12-27 16:29:26,183][20347] Updated weights for policy 0, policy_version 1565344 (0.0008) [2023-12-27 16:29:26,241][20347] Updated weights for policy 0, policy_version 1565354 (0.0007) [2023-12-27 16:29:26,494][20381] Updated weights for policy 1, policy_version 1563691 (0.0008) [2023-12-27 16:29:26,555][20381] Updated weights for policy 1, policy_version 1563701 (0.0008) [2023-12-27 16:29:26,618][20381] Updated weights for policy 1, policy_version 1563711 (0.0008) [2023-12-27 16:29:26,894][20347] Updated weights for policy 0, policy_version 1565364 (0.0007) [2023-12-27 16:29:26,950][20347] Updated weights for policy 0, policy_version 1565374 (0.0007) [2023-12-27 16:29:27,005][20347] Updated weights for policy 0, policy_version 1565384 (0.0006) [2023-12-27 16:29:27,313][20381] Updated weights for policy 1, policy_version 1563721 (0.0008) [2023-12-27 16:29:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 801169408. Throughput: 0: 10299.6, 1: 10284.2. Samples: 801160220. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:27,363][19170] Avg episode reward: [(0, '794.450'), (1, '873.040')] [2023-12-27 16:29:27,376][20381] Updated weights for policy 1, policy_version 1563731 (0.0008) [2023-12-27 16:29:27,427][20381] Updated weights for policy 1, policy_version 1563741 (0.0007) [2023-12-27 16:29:27,480][20381] Updated weights for policy 1, policy_version 1563751 (0.0008) [2023-12-27 16:29:27,630][20347] Updated weights for policy 0, policy_version 1565394 (0.0008) [2023-12-27 16:29:27,681][20347] Updated weights for policy 0, policy_version 1565404 (0.0005) [2023-12-27 16:29:27,734][20347] Updated weights for policy 0, policy_version 1565414 (0.0008) [2023-12-27 16:29:27,778][20347] Updated weights for policy 0, policy_version 1565424 (0.0010) [2023-12-27 16:29:28,158][20381] Updated weights for policy 1, policy_version 1563761 (0.0006) [2023-12-27 16:29:28,223][20381] Updated weights for policy 1, policy_version 1563771 (0.0006) [2023-12-27 16:29:28,277][20381] Updated weights for policy 1, policy_version 1563781 (0.0007) [2023-12-27 16:29:28,474][20347] Updated weights for policy 0, policy_version 1565434 (0.0009) [2023-12-27 16:29:28,528][20347] Updated weights for policy 0, policy_version 1565444 (0.0009) [2023-12-27 16:29:28,585][20347] Updated weights for policy 0, policy_version 1565454 (0.0009) [2023-12-27 16:29:28,912][20381] Updated weights for policy 1, policy_version 1563791 (0.0007) [2023-12-27 16:29:28,966][20381] Updated weights for policy 1, policy_version 1563801 (0.0007) [2023-12-27 16:29:29,027][20381] Updated weights for policy 1, policy_version 1563811 (0.0008) [2023-12-27 16:29:29,278][20347] Updated weights for policy 0, policy_version 1565464 (0.0007) [2023-12-27 16:29:29,328][20347] Updated weights for policy 0, policy_version 1565474 (0.0006) [2023-12-27 16:29:29,401][20347] Updated weights for policy 0, policy_version 1565484 (0.0008) [2023-12-27 16:29:29,697][20381] Updated weights for policy 1, policy_version 1563821 (0.0007) [2023-12-27 16:29:29,750][20381] Updated weights for policy 1, policy_version 1563831 (0.0007) [2023-12-27 16:29:29,803][20381] Updated weights for policy 1, policy_version 1563841 (0.0008) [2023-12-27 16:29:30,072][20347] Updated weights for policy 0, policy_version 1565494 (0.0007) [2023-12-27 16:29:30,135][20347] Updated weights for policy 0, policy_version 1565504 (0.0006) [2023-12-27 16:29:30,194][20347] Updated weights for policy 0, policy_version 1565514 (0.0007) [2023-12-27 16:29:30,539][20381] Updated weights for policy 1, policy_version 1563851 (0.0009) [2023-12-27 16:29:30,589][20381] Updated weights for policy 1, policy_version 1563861 (0.0008) [2023-12-27 16:29:30,641][20381] Updated weights for policy 1, policy_version 1563871 (0.0007) [2023-12-27 16:29:30,782][20347] Updated weights for policy 0, policy_version 1565524 (0.0007) [2023-12-27 16:29:30,832][20347] Updated weights for policy 0, policy_version 1565534 (0.0008) [2023-12-27 16:29:30,886][20347] Updated weights for policy 0, policy_version 1565544 (0.0008) [2023-12-27 16:29:30,887][20291] KL-divergence is very high: 126.4615 [2023-12-27 16:29:30,916][20291] KL-divergence is very high: 119.3434 [2023-12-27 16:29:31,361][20381] Updated weights for policy 1, policy_version 1563881 (0.0008) [2023-12-27 16:29:31,428][20381] Updated weights for policy 1, policy_version 1563891 (0.0008) [2023-12-27 16:29:31,490][20381] Updated weights for policy 1, policy_version 1563901 (0.0008) [2023-12-27 16:29:31,538][20292] KL-divergence is very high: 103.5623 [2023-12-27 16:29:31,544][20381] Updated weights for policy 1, policy_version 1563911 (0.0008) [2023-12-27 16:29:31,644][20347] Updated weights for policy 0, policy_version 1565554 (0.0007) [2023-12-27 16:29:31,697][20347] Updated weights for policy 0, policy_version 1565564 (0.0006) [2023-12-27 16:29:31,761][20347] Updated weights for policy 0, policy_version 1565574 (0.0008) [2023-12-27 16:29:31,811][20347] Updated weights for policy 0, policy_version 1565584 (0.0006) [2023-12-27 16:29:32,322][20381] Updated weights for policy 1, policy_version 1563921 (0.0007) [2023-12-27 16:29:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 801267712. Throughput: 0: 10356.8, 1: 10211.7. Samples: 801283020. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:32,363][19170] Avg episode reward: [(0, '794.290'), (1, '854.400')] [2023-12-27 16:29:32,386][20381] Updated weights for policy 1, policy_version 1563931 (0.0008) [2023-12-27 16:29:32,450][20381] Updated weights for policy 1, policy_version 1563941 (0.0007) [2023-12-27 16:29:32,475][20347] Updated weights for policy 0, policy_version 1565594 (0.0005) [2023-12-27 16:29:32,521][20347] Updated weights for policy 0, policy_version 1565604 (0.0007) [2023-12-27 16:29:32,570][20347] Updated weights for policy 0, policy_version 1565614 (0.0007) [2023-12-27 16:29:33,124][20381] Updated weights for policy 1, policy_version 1563951 (0.0007) [2023-12-27 16:29:33,182][20381] Updated weights for policy 1, policy_version 1563961 (0.0007) [2023-12-27 16:29:33,237][20381] Updated weights for policy 1, policy_version 1563971 (0.0007) [2023-12-27 16:29:33,248][20347] Updated weights for policy 0, policy_version 1565624 (0.0010) [2023-12-27 16:29:33,304][20347] Updated weights for policy 0, policy_version 1565634 (0.0008) [2023-12-27 16:29:33,355][20347] Updated weights for policy 0, policy_version 1565644 (0.0005) [2023-12-27 16:29:33,928][20347] Updated weights for policy 0, policy_version 1565654 (0.0005) [2023-12-27 16:29:33,928][20381] Updated weights for policy 1, policy_version 1563981 (0.0007) [2023-12-27 16:29:33,975][20381] Updated weights for policy 1, policy_version 1563991 (0.0007) [2023-12-27 16:29:33,985][20347] Updated weights for policy 0, policy_version 1565664 (0.0007) [2023-12-27 16:29:34,019][20381] Updated weights for policy 1, policy_version 1564001 (0.0005) [2023-12-27 16:29:34,029][20347] Updated weights for policy 0, policy_version 1565674 (0.0010) [2023-12-27 16:29:34,706][20347] Updated weights for policy 0, policy_version 1565684 (0.0006) [2023-12-27 16:29:34,763][20381] Updated weights for policy 1, policy_version 1564011 (0.0006) [2023-12-27 16:29:34,764][20347] Updated weights for policy 0, policy_version 1565694 (0.0008) [2023-12-27 16:29:34,819][20381] Updated weights for policy 1, policy_version 1564021 (0.0006) [2023-12-27 16:29:34,819][20347] Updated weights for policy 0, policy_version 1565704 (0.0006) [2023-12-27 16:29:34,875][20381] Updated weights for policy 1, policy_version 1564031 (0.0008) [2023-12-27 16:29:35,462][20347] Updated weights for policy 0, policy_version 1565714 (0.0006) [2023-12-27 16:29:35,518][20347] Updated weights for policy 0, policy_version 1565724 (0.0007) [2023-12-27 16:29:35,574][20347] Updated weights for policy 0, policy_version 1565734 (0.0008) [2023-12-27 16:29:35,582][20381] Updated weights for policy 1, policy_version 1564041 (0.0008) [2023-12-27 16:29:35,621][20347] Updated weights for policy 0, policy_version 1565744 (0.0007) [2023-12-27 16:29:35,635][20381] Updated weights for policy 1, policy_version 1564051 (0.0006) [2023-12-27 16:29:35,683][20381] Updated weights for policy 1, policy_version 1564061 (0.0006) [2023-12-27 16:29:35,741][20381] Updated weights for policy 1, policy_version 1564071 (0.0005) [2023-12-27 16:29:36,314][20347] Updated weights for policy 0, policy_version 1565754 (0.0008) [2023-12-27 16:29:36,362][20347] Updated weights for policy 0, policy_version 1565764 (0.0007) [2023-12-27 16:29:36,414][20347] Updated weights for policy 0, policy_version 1565774 (0.0006) [2023-12-27 16:29:36,435][20381] Updated weights for policy 1, policy_version 1564081 (0.0006) [2023-12-27 16:29:36,502][20381] Updated weights for policy 1, policy_version 1564091 (0.0008) [2023-12-27 16:29:36,564][20381] Updated weights for policy 1, policy_version 1564101 (0.0007) [2023-12-27 16:29:37,107][20381] Updated weights for policy 1, policy_version 1564111 (0.0005) [2023-12-27 16:29:37,122][20347] Updated weights for policy 0, policy_version 1565784 (0.0005) [2023-12-27 16:29:37,159][20381] Updated weights for policy 1, policy_version 1564121 (0.0007) [2023-12-27 16:29:37,173][20347] Updated weights for policy 0, policy_version 1565794 (0.0005) [2023-12-27 16:29:37,184][20291] KL-divergence is very high: 224.5320 [2023-12-27 16:29:37,190][20291] KL-divergence is very high: 118.3535 [2023-12-27 16:29:37,209][20381] Updated weights for policy 1, policy_version 1564131 (0.0005) [2023-12-27 16:29:37,224][20291] KL-divergence is very high: 282.8564 [2023-12-27 16:29:37,226][20347] Updated weights for policy 0, policy_version 1565804 (0.0006) [2023-12-27 16:29:37,230][20291] KL-divergence is very high: 127.7248 [2023-12-27 16:29:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 801382400. Throughput: 0: 10389.0, 1: 10175.3. Samples: 801344556. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:37,363][19170] Avg episode reward: [(0, '853.010'), (1, '766.180')] [2023-12-27 16:29:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001564136_400457728.pth... [2023-12-27 16:29:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001565808_400924672.pth... [2023-12-27 16:29:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001562952_400154624.pth [2023-12-27 16:29:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001564592_400613376.pth [2023-12-27 16:29:37,879][20347] Updated weights for policy 0, policy_version 1565814 (0.0007) [2023-12-27 16:29:37,895][20381] Updated weights for policy 1, policy_version 1564141 (0.0006) [2023-12-27 16:29:37,937][20347] Updated weights for policy 0, policy_version 1565824 (0.0006) [2023-12-27 16:29:37,958][20381] Updated weights for policy 1, policy_version 1564151 (0.0009) [2023-12-27 16:29:37,990][20347] Updated weights for policy 0, policy_version 1565834 (0.0006) [2023-12-27 16:29:38,015][20381] Updated weights for policy 1, policy_version 1564161 (0.0010) [2023-12-27 16:29:38,674][20381] Updated weights for policy 1, policy_version 1564171 (0.0011) [2023-12-27 16:29:38,731][20347] Updated weights for policy 0, policy_version 1565844 (0.0008) [2023-12-27 16:29:38,734][20381] Updated weights for policy 1, policy_version 1564181 (0.0011) [2023-12-27 16:29:38,790][20381] Updated weights for policy 1, policy_version 1564191 (0.0010) [2023-12-27 16:29:38,795][20347] Updated weights for policy 0, policy_version 1565854 (0.0006) [2023-12-27 16:29:38,856][20347] Updated weights for policy 0, policy_version 1565864 (0.0006) [2023-12-27 16:29:39,557][20347] Updated weights for policy 0, policy_version 1565874 (0.0006) [2023-12-27 16:29:39,557][20381] Updated weights for policy 1, policy_version 1564201 (0.0008) [2023-12-27 16:29:39,613][20381] Updated weights for policy 1, policy_version 1564211 (0.0006) [2023-12-27 16:29:39,617][20347] Updated weights for policy 0, policy_version 1565884 (0.0007) [2023-12-27 16:29:39,664][20381] Updated weights for policy 1, policy_version 1564221 (0.0006) [2023-12-27 16:29:39,682][20347] Updated weights for policy 0, policy_version 1565894 (0.0006) [2023-12-27 16:29:39,723][20381] Updated weights for policy 1, policy_version 1564231 (0.0007) [2023-12-27 16:29:39,743][20347] Updated weights for policy 0, policy_version 1565904 (0.0007) [2023-12-27 16:29:40,386][20347] Updated weights for policy 0, policy_version 1565914 (0.0008) [2023-12-27 16:29:40,419][20381] Updated weights for policy 1, policy_version 1564241 (0.0006) [2023-12-27 16:29:40,441][20347] Updated weights for policy 0, policy_version 1565924 (0.0006) [2023-12-27 16:29:40,478][20381] Updated weights for policy 1, policy_version 1564251 (0.0008) [2023-12-27 16:29:40,500][20347] Updated weights for policy 0, policy_version 1565934 (0.0008) [2023-12-27 16:29:40,535][20381] Updated weights for policy 1, policy_version 1564261 (0.0008) [2023-12-27 16:29:41,155][20347] Updated weights for policy 0, policy_version 1565944 (0.0007) [2023-12-27 16:29:41,187][20381] Updated weights for policy 1, policy_version 1564271 (0.0009) [2023-12-27 16:29:41,217][20347] Updated weights for policy 0, policy_version 1565954 (0.0006) [2023-12-27 16:29:41,247][20381] Updated weights for policy 1, policy_version 1564281 (0.0009) [2023-12-27 16:29:41,280][20347] Updated weights for policy 0, policy_version 1565964 (0.0007) [2023-12-27 16:29:41,301][20381] Updated weights for policy 1, policy_version 1564291 (0.0007) [2023-12-27 16:29:42,012][20347] Updated weights for policy 0, policy_version 1565974 (0.0007) [2023-12-27 16:29:42,064][20381] Updated weights for policy 1, policy_version 1564301 (0.0007) [2023-12-27 16:29:42,072][20347] Updated weights for policy 0, policy_version 1565984 (0.0007) [2023-12-27 16:29:42,129][20381] Updated weights for policy 1, policy_version 1564311 (0.0006) [2023-12-27 16:29:42,135][20347] Updated weights for policy 0, policy_version 1565994 (0.0007) [2023-12-27 16:29:42,191][20381] Updated weights for policy 1, policy_version 1564321 (0.0006) [2023-12-27 16:29:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 801480704. Throughput: 0: 10371.9, 1: 10187.5. Samples: 801466736. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:42,363][19170] Avg episode reward: [(0, '833.510'), (1, '707.490')] [2023-12-27 16:29:42,827][20347] Updated weights for policy 0, policy_version 1566004 (0.0006) [2023-12-27 16:29:42,873][20381] Updated weights for policy 1, policy_version 1564331 (0.0007) [2023-12-27 16:29:42,886][20347] Updated weights for policy 0, policy_version 1566014 (0.0007) [2023-12-27 16:29:42,927][20381] Updated weights for policy 1, policy_version 1564341 (0.0005) [2023-12-27 16:29:42,936][20347] Updated weights for policy 0, policy_version 1566024 (0.0008) [2023-12-27 16:29:42,973][20381] Updated weights for policy 1, policy_version 1564351 (0.0005) [2023-12-27 16:29:43,598][20381] Updated weights for policy 1, policy_version 1564361 (0.0006) [2023-12-27 16:29:43,605][20347] Updated weights for policy 0, policy_version 1566034 (0.0007) [2023-12-27 16:29:43,655][20381] Updated weights for policy 1, policy_version 1564371 (0.0006) [2023-12-27 16:29:43,656][20347] Updated weights for policy 0, policy_version 1566044 (0.0005) [2023-12-27 16:29:43,686][20292] KL-divergence is very high: 106.5062 [2023-12-27 16:29:43,708][20381] Updated weights for policy 1, policy_version 1564381 (0.0005) [2023-12-27 16:29:43,710][20347] Updated weights for policy 0, policy_version 1566054 (0.0005) [2023-12-27 16:29:43,716][20291] KL-divergence is very high: 156.3539 [2023-12-27 16:29:43,721][20291] KL-divergence is very high: 168.3904 [2023-12-27 16:29:43,762][20291] KL-divergence is very high: 117.0860 [2023-12-27 16:29:43,763][20381] Updated weights for policy 1, policy_version 1564391 (0.0005) [2023-12-27 16:29:43,768][20347] Updated weights for policy 0, policy_version 1566064 (0.0005) [2023-12-27 16:29:44,322][20381] Updated weights for policy 1, policy_version 1564401 (0.0007) [2023-12-27 16:29:44,381][20381] Updated weights for policy 1, policy_version 1564411 (0.0007) [2023-12-27 16:29:44,422][20347] Updated weights for policy 0, policy_version 1566074 (0.0006) [2023-12-27 16:29:44,439][20381] Updated weights for policy 1, policy_version 1564421 (0.0005) [2023-12-27 16:29:44,487][20347] Updated weights for policy 0, policy_version 1566084 (0.0007) [2023-12-27 16:29:44,547][20347] Updated weights for policy 0, policy_version 1566094 (0.0007) [2023-12-27 16:29:45,089][20381] Updated weights for policy 1, policy_version 1564431 (0.0005) [2023-12-27 16:29:45,155][20381] Updated weights for policy 1, policy_version 1564441 (0.0006) [2023-12-27 16:29:45,216][20381] Updated weights for policy 1, policy_version 1564451 (0.0006) [2023-12-27 16:29:45,269][20347] Updated weights for policy 0, policy_version 1566104 (0.0009) [2023-12-27 16:29:45,329][20347] Updated weights for policy 0, policy_version 1566114 (0.0008) [2023-12-27 16:29:45,389][20347] Updated weights for policy 0, policy_version 1566124 (0.0008) [2023-12-27 16:29:45,743][20381] Updated weights for policy 1, policy_version 1564461 (0.0006) [2023-12-27 16:29:45,799][20381] Updated weights for policy 1, policy_version 1564471 (0.0007) [2023-12-27 16:29:45,851][20381] Updated weights for policy 1, policy_version 1564481 (0.0008) [2023-12-27 16:29:46,121][20347] Updated weights for policy 0, policy_version 1566134 (0.0007) [2023-12-27 16:29:46,178][20347] Updated weights for policy 0, policy_version 1566144 (0.0007) [2023-12-27 16:29:46,235][20347] Updated weights for policy 0, policy_version 1566154 (0.0007) [2023-12-27 16:29:46,525][20381] Updated weights for policy 1, policy_version 1564491 (0.0008) [2023-12-27 16:29:46,573][20381] Updated weights for policy 1, policy_version 1564501 (0.0008) [2023-12-27 16:29:46,622][20381] Updated weights for policy 1, policy_version 1564511 (0.0008) [2023-12-27 16:29:46,815][20347] Updated weights for policy 0, policy_version 1566164 (0.0006) [2023-12-27 16:29:46,873][20347] Updated weights for policy 0, policy_version 1566174 (0.0005) [2023-12-27 16:29:46,921][20347] Updated weights for policy 0, policy_version 1566184 (0.0005) [2023-12-27 16:29:47,347][20381] Updated weights for policy 1, policy_version 1564521 (0.0008) [2023-12-27 16:29:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 801579008. Throughput: 0: 10346.5, 1: 10219.9. Samples: 801591160. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:47,363][19170] Avg episode reward: [(0, '882.710'), (1, '795.530')] [2023-12-27 16:29:47,410][20381] Updated weights for policy 1, policy_version 1564531 (0.0007) [2023-12-27 16:29:47,463][20347] Updated weights for policy 0, policy_version 1566194 (0.0005) [2023-12-27 16:29:47,466][20381] Updated weights for policy 1, policy_version 1564541 (0.0007) [2023-12-27 16:29:47,507][20347] Updated weights for policy 0, policy_version 1566204 (0.0007) [2023-12-27 16:29:47,517][20381] Updated weights for policy 1, policy_version 1564551 (0.0005) [2023-12-27 16:29:47,572][20347] Updated weights for policy 0, policy_version 1566214 (0.0010) [2023-12-27 16:29:47,619][20347] Updated weights for policy 0, policy_version 1566224 (0.0006) [2023-12-27 16:29:48,132][20381] Updated weights for policy 1, policy_version 1564561 (0.0007) [2023-12-27 16:29:48,191][20381] Updated weights for policy 1, policy_version 1564571 (0.0006) [2023-12-27 16:29:48,225][20347] Updated weights for policy 0, policy_version 1566234 (0.0008) [2023-12-27 16:29:48,250][20381] Updated weights for policy 1, policy_version 1564581 (0.0006) [2023-12-27 16:29:48,279][20347] Updated weights for policy 0, policy_version 1566244 (0.0005) [2023-12-27 16:29:48,334][20347] Updated weights for policy 0, policy_version 1566254 (0.0006) [2023-12-27 16:29:48,994][20381] Updated weights for policy 1, policy_version 1564591 (0.0006) [2023-12-27 16:29:49,026][20347] Updated weights for policy 0, policy_version 1566264 (0.0007) [2023-12-27 16:29:49,057][20381] Updated weights for policy 1, policy_version 1564601 (0.0006) [2023-12-27 16:29:49,079][20347] Updated weights for policy 0, policy_version 1566274 (0.0005) [2023-12-27 16:29:49,116][20381] Updated weights for policy 1, policy_version 1564611 (0.0007) [2023-12-27 16:29:49,133][20347] Updated weights for policy 0, policy_version 1566284 (0.0005) [2023-12-27 16:29:49,822][20381] Updated weights for policy 1, policy_version 1564621 (0.0007) [2023-12-27 16:29:49,825][20347] Updated weights for policy 0, policy_version 1566294 (0.0006) [2023-12-27 16:29:49,883][20381] Updated weights for policy 1, policy_version 1564631 (0.0007) [2023-12-27 16:29:49,885][20347] Updated weights for policy 0, policy_version 1566304 (0.0006) [2023-12-27 16:29:49,943][20381] Updated weights for policy 1, policy_version 1564641 (0.0006) [2023-12-27 16:29:49,946][20347] Updated weights for policy 0, policy_version 1566314 (0.0007) [2023-12-27 16:29:50,646][20381] Updated weights for policy 1, policy_version 1564651 (0.0007) [2023-12-27 16:29:50,676][20347] Updated weights for policy 0, policy_version 1566324 (0.0007) [2023-12-27 16:29:50,701][20381] Updated weights for policy 1, policy_version 1564661 (0.0006) [2023-12-27 16:29:50,737][20347] Updated weights for policy 0, policy_version 1566334 (0.0006) [2023-12-27 16:29:50,760][20381] Updated weights for policy 1, policy_version 1564671 (0.0009) [2023-12-27 16:29:50,795][20347] Updated weights for policy 0, policy_version 1566344 (0.0009) [2023-12-27 16:29:51,432][20381] Updated weights for policy 1, policy_version 1564681 (0.0007) [2023-12-27 16:29:51,481][20381] Updated weights for policy 1, policy_version 1564691 (0.0007) [2023-12-27 16:29:51,530][20381] Updated weights for policy 1, policy_version 1564701 (0.0007) [2023-12-27 16:29:51,585][20381] Updated weights for policy 1, policy_version 1564711 (0.0007) [2023-12-27 16:29:51,593][20347] Updated weights for policy 0, policy_version 1566354 (0.0008) [2023-12-27 16:29:51,657][20347] Updated weights for policy 0, policy_version 1566364 (0.0007) [2023-12-27 16:29:51,725][20347] Updated weights for policy 0, policy_version 1566374 (0.0008) [2023-12-27 16:29:51,791][20347] Updated weights for policy 0, policy_version 1566384 (0.0008) [2023-12-27 16:29:52,258][20381] Updated weights for policy 1, policy_version 1564721 (0.0007) [2023-12-27 16:29:52,312][20381] Updated weights for policy 1, policy_version 1564731 (0.0008) [2023-12-27 16:29:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 801677312. Throughput: 0: 10290.7, 1: 10206.0. Samples: 801653176. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:52,365][20381] Updated weights for policy 1, policy_version 1564741 (0.0007) [2023-12-27 16:29:52,366][19170] Avg episode reward: [(0, '862.900'), (1, '824.340')] [2023-12-27 16:29:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001566384_401072128.pth... [2023-12-27 16:29:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001565200_400769024.pth [2023-12-27 16:29:52,387][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001564744_400613376.pth... [2023-12-27 16:29:52,391][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001563528_400302080.pth [2023-12-27 16:29:52,513][20347] Updated weights for policy 0, policy_version 1566394 (0.0010) [2023-12-27 16:29:52,568][20347] Updated weights for policy 0, policy_version 1566404 (0.0010) [2023-12-27 16:29:52,629][20347] Updated weights for policy 0, policy_version 1566414 (0.0011) [2023-12-27 16:29:53,072][20381] Updated weights for policy 1, policy_version 1564751 (0.0007) [2023-12-27 16:29:53,132][20381] Updated weights for policy 1, policy_version 1564761 (0.0009) [2023-12-27 16:29:53,193][20381] Updated weights for policy 1, policy_version 1564771 (0.0011) [2023-12-27 16:29:53,258][20347] Updated weights for policy 0, policy_version 1566424 (0.0006) [2023-12-27 16:29:53,311][20347] Updated weights for policy 0, policy_version 1566434 (0.0005) [2023-12-27 16:29:53,370][20347] Updated weights for policy 0, policy_version 1566444 (0.0007) [2023-12-27 16:29:53,810][20381] Updated weights for policy 1, policy_version 1564784 (0.0008) [2023-12-27 16:29:53,868][20381] Updated weights for policy 1, policy_version 1564794 (0.0007) [2023-12-27 16:29:53,922][20381] Updated weights for policy 1, policy_version 1564804 (0.0006) [2023-12-27 16:29:53,923][20347] Updated weights for policy 0, policy_version 1566454 (0.0008) [2023-12-27 16:29:53,978][20347] Updated weights for policy 0, policy_version 1566464 (0.0007) [2023-12-27 16:29:54,036][20347] Updated weights for policy 0, policy_version 1566474 (0.0007) [2023-12-27 16:29:54,613][20381] Updated weights for policy 1, policy_version 1564814 (0.0005) [2023-12-27 16:29:54,667][20381] Updated weights for policy 1, policy_version 1564824 (0.0007) [2023-12-27 16:29:54,718][20381] Updated weights for policy 1, policy_version 1564834 (0.0007) [2023-12-27 16:29:54,723][20347] Updated weights for policy 0, policy_version 1566484 (0.0006) [2023-12-27 16:29:54,782][20347] Updated weights for policy 0, policy_version 1566494 (0.0006) [2023-12-27 16:29:54,843][20347] Updated weights for policy 0, policy_version 1566504 (0.0006) [2023-12-27 16:29:55,376][20381] Updated weights for policy 1, policy_version 1564844 (0.0007) [2023-12-27 16:29:55,424][20381] Updated weights for policy 1, policy_version 1564854 (0.0008) [2023-12-27 16:29:55,472][20381] Updated weights for policy 1, policy_version 1564864 (0.0007) [2023-12-27 16:29:55,506][20347] Updated weights for policy 0, policy_version 1566514 (0.0007) [2023-12-27 16:29:55,560][20347] Updated weights for policy 0, policy_version 1566524 (0.0009) [2023-12-27 16:29:55,604][20347] Updated weights for policy 0, policy_version 1566534 (0.0007) [2023-12-27 16:29:55,648][20347] Updated weights for policy 0, policy_version 1566544 (0.0007) [2023-12-27 16:29:56,115][20381] Updated weights for policy 1, policy_version 1564874 (0.0007) [2023-12-27 16:29:56,169][20381] Updated weights for policy 1, policy_version 1564884 (0.0008) [2023-12-27 16:29:56,229][20381] Updated weights for policy 1, policy_version 1564894 (0.0008) [2023-12-27 16:29:56,274][20381] Updated weights for policy 1, policy_version 1564904 (0.0008) [2023-12-27 16:29:56,335][20347] Updated weights for policy 0, policy_version 1566554 (0.0005) [2023-12-27 16:29:56,387][20347] Updated weights for policy 0, policy_version 1566564 (0.0005) [2023-12-27 16:29:56,444][20347] Updated weights for policy 0, policy_version 1566574 (0.0007) [2023-12-27 16:29:56,861][20381] Updated weights for policy 1, policy_version 1564914 (0.0005) [2023-12-27 16:29:56,913][20381] Updated weights for policy 1, policy_version 1564924 (0.0005) [2023-12-27 16:29:56,968][20381] Updated weights for policy 1, policy_version 1564934 (0.0007) [2023-12-27 16:29:57,115][20347] Updated weights for policy 0, policy_version 1566584 (0.0007) [2023-12-27 16:29:57,164][20347] Updated weights for policy 0, policy_version 1566594 (0.0008) [2023-12-27 16:29:57,209][20347] Updated weights for policy 0, policy_version 1566604 (0.0007) [2023-12-27 16:29:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 801792000. Throughput: 0: 10282.0, 1: 10216.2. Samples: 801776496. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:29:57,363][19170] Avg episode reward: [(0, '853.020'), (1, '873.050')] [2023-12-27 16:29:57,628][20381] Updated weights for policy 1, policy_version 1564944 (0.0008) [2023-12-27 16:29:57,678][20381] Updated weights for policy 1, policy_version 1564954 (0.0007) [2023-12-27 16:29:57,729][20381] Updated weights for policy 1, policy_version 1564964 (0.0008) [2023-12-27 16:29:57,860][20347] Updated weights for policy 0, policy_version 1566614 (0.0006) [2023-12-27 16:29:57,912][20347] Updated weights for policy 0, policy_version 1566624 (0.0005) [2023-12-27 16:29:57,959][20347] Updated weights for policy 0, policy_version 1566634 (0.0005) [2023-12-27 16:29:58,395][20381] Updated weights for policy 1, policy_version 1564974 (0.0008) [2023-12-27 16:29:58,459][20381] Updated weights for policy 1, policy_version 1564984 (0.0009) [2023-12-27 16:29:58,505][20381] Updated weights for policy 1, policy_version 1564994 (0.0008) [2023-12-27 16:29:58,607][20347] Updated weights for policy 0, policy_version 1566644 (0.0006) [2023-12-27 16:29:58,670][20347] Updated weights for policy 0, policy_version 1566654 (0.0008) [2023-12-27 16:29:58,733][20347] Updated weights for policy 0, policy_version 1566664 (0.0007) [2023-12-27 16:29:59,327][20381] Updated weights for policy 1, policy_version 1565004 (0.0008) [2023-12-27 16:29:59,391][20381] Updated weights for policy 1, policy_version 1565014 (0.0008) [2023-12-27 16:29:59,450][20381] Updated weights for policy 1, policy_version 1565024 (0.0006) [2023-12-27 16:29:59,545][20347] Updated weights for policy 0, policy_version 1566674 (0.0007) [2023-12-27 16:29:59,592][20347] Updated weights for policy 0, policy_version 1566684 (0.0007) [2023-12-27 16:29:59,644][20347] Updated weights for policy 0, policy_version 1566694 (0.0008) [2023-12-27 16:29:59,704][20347] Updated weights for policy 0, policy_version 1566704 (0.0007) [2023-12-27 16:30:00,134][20381] Updated weights for policy 1, policy_version 1565034 (0.0007) [2023-12-27 16:30:00,193][20381] Updated weights for policy 1, policy_version 1565044 (0.0005) [2023-12-27 16:30:00,256][20381] Updated weights for policy 1, policy_version 1565054 (0.0005) [2023-12-27 16:30:00,325][20381] Updated weights for policy 1, policy_version 1565064 (0.0005) [2023-12-27 16:30:00,467][20347] Updated weights for policy 0, policy_version 1566714 (0.0008) [2023-12-27 16:30:00,520][20347] Updated weights for policy 0, policy_version 1566724 (0.0006) [2023-12-27 16:30:00,574][20347] Updated weights for policy 0, policy_version 1566734 (0.0005) [2023-12-27 16:30:00,920][20381] Updated weights for policy 1, policy_version 1565074 (0.0007) [2023-12-27 16:30:00,977][20381] Updated weights for policy 1, policy_version 1565084 (0.0010) [2023-12-27 16:30:01,026][20381] Updated weights for policy 1, policy_version 1565094 (0.0010) [2023-12-27 16:30:01,217][20347] Updated weights for policy 0, policy_version 1566744 (0.0006) [2023-12-27 16:30:01,280][20347] Updated weights for policy 0, policy_version 1566754 (0.0007) [2023-12-27 16:30:01,340][20347] Updated weights for policy 0, policy_version 1566764 (0.0007) [2023-12-27 16:30:01,750][20381] Updated weights for policy 1, policy_version 1565104 (0.0009) [2023-12-27 16:30:01,802][20381] Updated weights for policy 1, policy_version 1565114 (0.0008) [2023-12-27 16:30:01,859][20381] Updated weights for policy 1, policy_version 1565124 (0.0008) [2023-12-27 16:30:02,065][20347] Updated weights for policy 0, policy_version 1566774 (0.0007) [2023-12-27 16:30:02,123][20347] Updated weights for policy 0, policy_version 1566784 (0.0006) [2023-12-27 16:30:02,187][20347] Updated weights for policy 0, policy_version 1566794 (0.0006) [2023-12-27 16:30:02,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 801890304. Throughput: 0: 10271.7, 1: 10212.1. Samples: 801899640. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:30:02,363][19170] Avg episode reward: [(0, '882.410'), (1, '891.630')] [2023-12-27 16:30:02,522][20381] Updated weights for policy 1, policy_version 1565134 (0.0008) [2023-12-27 16:30:02,577][20381] Updated weights for policy 1, policy_version 1565144 (0.0008) [2023-12-27 16:30:02,636][20381] Updated weights for policy 1, policy_version 1565154 (0.0005) [2023-12-27 16:30:02,833][20347] Updated weights for policy 0, policy_version 1566804 (0.0006) [2023-12-27 16:30:02,893][20347] Updated weights for policy 0, policy_version 1566814 (0.0008) [2023-12-27 16:30:02,953][20347] Updated weights for policy 0, policy_version 1566824 (0.0007) [2023-12-27 16:30:03,295][20381] Updated weights for policy 1, policy_version 1565164 (0.0008) [2023-12-27 16:30:03,341][20381] Updated weights for policy 1, policy_version 1565174 (0.0008) [2023-12-27 16:30:03,387][20381] Updated weights for policy 1, policy_version 1565184 (0.0007) [2023-12-27 16:30:03,554][20347] Updated weights for policy 0, policy_version 1566834 (0.0008) [2023-12-27 16:30:03,611][20347] Updated weights for policy 0, policy_version 1566844 (0.0007) [2023-12-27 16:30:03,658][20347] Updated weights for policy 0, policy_version 1566854 (0.0007) [2023-12-27 16:30:03,711][20347] Updated weights for policy 0, policy_version 1566864 (0.0006) [2023-12-27 16:30:04,085][20381] Updated weights for policy 1, policy_version 1565194 (0.0008) [2023-12-27 16:30:04,150][20381] Updated weights for policy 1, policy_version 1565204 (0.0010) [2023-12-27 16:30:04,210][20381] Updated weights for policy 1, policy_version 1565214 (0.0011) [2023-12-27 16:30:04,281][20381] Updated weights for policy 1, policy_version 1565224 (0.0011) [2023-12-27 16:30:04,321][20347] Updated weights for policy 0, policy_version 1566874 (0.0008) [2023-12-27 16:30:04,366][20347] Updated weights for policy 0, policy_version 1566884 (0.0007) [2023-12-27 16:30:04,412][20347] Updated weights for policy 0, policy_version 1566894 (0.0007) [2023-12-27 16:30:04,932][20381] Updated weights for policy 1, policy_version 1565234 (0.0011) [2023-12-27 16:30:04,981][20381] Updated weights for policy 1, policy_version 1565244 (0.0010) [2023-12-27 16:30:05,036][20381] Updated weights for policy 1, policy_version 1565254 (0.0011) [2023-12-27 16:30:05,053][20347] Updated weights for policy 0, policy_version 1566904 (0.0006) [2023-12-27 16:30:05,107][20347] Updated weights for policy 0, policy_version 1566914 (0.0006) [2023-12-27 16:30:05,163][20347] Updated weights for policy 0, policy_version 1566924 (0.0006) [2023-12-27 16:30:05,729][20381] Updated weights for policy 1, policy_version 1565264 (0.0010) [2023-12-27 16:30:05,784][20381] Updated weights for policy 1, policy_version 1565274 (0.0008) [2023-12-27 16:30:05,786][20347] Updated weights for policy 0, policy_version 1566934 (0.0007) [2023-12-27 16:30:05,842][20381] Updated weights for policy 1, policy_version 1565284 (0.0006) [2023-12-27 16:30:05,851][20347] Updated weights for policy 0, policy_version 1566944 (0.0006) [2023-12-27 16:30:05,916][20347] Updated weights for policy 0, policy_version 1566954 (0.0008) [2023-12-27 16:30:06,496][20381] Updated weights for policy 1, policy_version 1565294 (0.0007) [2023-12-27 16:30:06,557][20381] Updated weights for policy 1, policy_version 1565304 (0.0008) [2023-12-27 16:30:06,610][20381] Updated weights for policy 1, policy_version 1565314 (0.0007) [2023-12-27 16:30:06,634][20347] Updated weights for policy 0, policy_version 1566964 (0.0009) [2023-12-27 16:30:06,694][20347] Updated weights for policy 0, policy_version 1566974 (0.0008) [2023-12-27 16:30:06,744][20347] Updated weights for policy 0, policy_version 1566984 (0.0006) [2023-12-27 16:30:07,191][20381] Updated weights for policy 1, policy_version 1565324 (0.0008) [2023-12-27 16:30:07,246][20381] Updated weights for policy 1, policy_version 1565334 (0.0005) [2023-12-27 16:30:07,302][20381] Updated weights for policy 1, policy_version 1565344 (0.0005) [2023-12-27 16:30:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 801996800. Throughput: 0: 10359.5, 1: 10197.3. Samples: 801963444. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:30:07,363][19170] Avg episode reward: [(0, '843.470'), (1, '793.530')] [2023-12-27 16:30:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001565352_400769024.pth... [2023-12-27 16:30:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001566992_401227776.pth... [2023-12-27 16:30:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001565808_400924672.pth [2023-12-27 16:30:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001564136_400457728.pth [2023-12-27 16:30:07,471][20347] Updated weights for policy 0, policy_version 1566994 (0.0005) [2023-12-27 16:30:07,522][20347] Updated weights for policy 0, policy_version 1567004 (0.0005) [2023-12-27 16:30:07,566][20347] Updated weights for policy 0, policy_version 1567014 (0.0006) [2023-12-27 16:30:07,615][20347] Updated weights for policy 0, policy_version 1567024 (0.0009) [2023-12-27 16:30:07,814][20381] Updated weights for policy 1, policy_version 1565354 (0.0006) [2023-12-27 16:30:07,869][20381] Updated weights for policy 1, policy_version 1565364 (0.0007) [2023-12-27 16:30:07,929][20381] Updated weights for policy 1, policy_version 1565374 (0.0006) [2023-12-27 16:30:07,980][20381] Updated weights for policy 1, policy_version 1565384 (0.0007) [2023-12-27 16:30:08,285][20347] Updated weights for policy 0, policy_version 1567034 (0.0007) [2023-12-27 16:30:08,338][20347] Updated weights for policy 0, policy_version 1567044 (0.0008) [2023-12-27 16:30:08,404][20347] Updated weights for policy 0, policy_version 1567054 (0.0008) [2023-12-27 16:30:08,634][20381] Updated weights for policy 1, policy_version 1565394 (0.0008) [2023-12-27 16:30:08,694][20381] Updated weights for policy 1, policy_version 1565404 (0.0008) [2023-12-27 16:30:08,752][20381] Updated weights for policy 1, policy_version 1565414 (0.0008) [2023-12-27 16:30:09,134][20347] Updated weights for policy 0, policy_version 1567064 (0.0009) [2023-12-27 16:30:09,194][20347] Updated weights for policy 0, policy_version 1567074 (0.0010) [2023-12-27 16:30:09,262][20347] Updated weights for policy 0, policy_version 1567084 (0.0009) [2023-12-27 16:30:09,507][20381] Updated weights for policy 1, policy_version 1565424 (0.0008) [2023-12-27 16:30:09,594][20381] Updated weights for policy 1, policy_version 1565434 (0.0008) [2023-12-27 16:30:09,664][20381] Updated weights for policy 1, policy_version 1565444 (0.0008) [2023-12-27 16:30:10,002][20347] Updated weights for policy 0, policy_version 1567094 (0.0007) [2023-12-27 16:30:10,063][20347] Updated weights for policy 0, policy_version 1567104 (0.0007) [2023-12-27 16:30:10,128][20347] Updated weights for policy 0, policy_version 1567114 (0.0007) [2023-12-27 16:30:10,439][20381] Updated weights for policy 1, policy_version 1565454 (0.0007) [2023-12-27 16:30:10,496][20381] Updated weights for policy 1, policy_version 1565464 (0.0007) [2023-12-27 16:30:10,558][20381] Updated weights for policy 1, policy_version 1565474 (0.0011) [2023-12-27 16:30:10,749][20347] Updated weights for policy 0, policy_version 1567124 (0.0007) [2023-12-27 16:30:10,798][20347] Updated weights for policy 0, policy_version 1567134 (0.0010) [2023-12-27 16:30:10,854][20347] Updated weights for policy 0, policy_version 1567144 (0.0010) [2023-12-27 16:30:11,261][20381] Updated weights for policy 1, policy_version 1565484 (0.0010) [2023-12-27 16:30:11,321][20381] Updated weights for policy 1, policy_version 1565494 (0.0008) [2023-12-27 16:30:11,385][20381] Updated weights for policy 1, policy_version 1565504 (0.0008) [2023-12-27 16:30:11,583][20347] Updated weights for policy 0, policy_version 1567154 (0.0009) [2023-12-27 16:30:11,661][20347] Updated weights for policy 0, policy_version 1567164 (0.0008) [2023-12-27 16:30:11,725][20347] Updated weights for policy 0, policy_version 1567174 (0.0009) [2023-12-27 16:30:11,786][20347] Updated weights for policy 0, policy_version 1567184 (0.0011) [2023-12-27 16:30:12,178][20381] Updated weights for policy 1, policy_version 1565514 (0.0010) [2023-12-27 16:30:12,234][20381] Updated weights for policy 1, policy_version 1565524 (0.0008) [2023-12-27 16:30:12,296][20381] Updated weights for policy 1, policy_version 1565534 (0.0008) [2023-12-27 16:30:12,358][20381] Updated weights for policy 1, policy_version 1565544 (0.0008) [2023-12-27 16:30:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 802095104. Throughput: 0: 10307.9, 1: 10252.0. Samples: 802085416. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:30:12,363][19170] Avg episode reward: [(0, '872.800'), (1, '851.710')] [2023-12-27 16:30:12,515][20347] Updated weights for policy 0, policy_version 1567194 (0.0010) [2023-12-27 16:30:12,571][20347] Updated weights for policy 0, policy_version 1567204 (0.0010) [2023-12-27 16:30:12,631][20347] Updated weights for policy 0, policy_version 1567214 (0.0011) [2023-12-27 16:30:13,037][20381] Updated weights for policy 1, policy_version 1565554 (0.0008) [2023-12-27 16:30:13,098][20381] Updated weights for policy 1, policy_version 1565564 (0.0007) [2023-12-27 16:30:13,158][20381] Updated weights for policy 1, policy_version 1565574 (0.0008) [2023-12-27 16:30:13,252][20347] Updated weights for policy 0, policy_version 1567224 (0.0006) [2023-12-27 16:30:13,306][20347] Updated weights for policy 0, policy_version 1567234 (0.0006) [2023-12-27 16:30:13,364][20347] Updated weights for policy 0, policy_version 1567244 (0.0010) [2023-12-27 16:30:13,843][20381] Updated weights for policy 1, policy_version 1565584 (0.0010) [2023-12-27 16:30:13,894][20381] Updated weights for policy 1, policy_version 1565594 (0.0010) [2023-12-27 16:30:13,945][20347] Updated weights for policy 0, policy_version 1567254 (0.0010) [2023-12-27 16:30:13,948][20381] Updated weights for policy 1, policy_version 1565604 (0.0006) [2023-12-27 16:30:14,001][20347] Updated weights for policy 0, policy_version 1567264 (0.0010) [2023-12-27 16:30:14,063][20347] Updated weights for policy 0, policy_version 1567274 (0.0010) [2023-12-27 16:30:14,616][20381] Updated weights for policy 1, policy_version 1565614 (0.0009) [2023-12-27 16:30:14,677][20381] Updated weights for policy 1, policy_version 1565624 (0.0007) [2023-12-27 16:30:14,731][20381] Updated weights for policy 1, policy_version 1565634 (0.0008) [2023-12-27 16:30:14,744][20347] Updated weights for policy 0, policy_version 1567284 (0.0008) [2023-12-27 16:30:14,803][20347] Updated weights for policy 0, policy_version 1567294 (0.0007) [2023-12-27 16:30:14,863][20347] Updated weights for policy 0, policy_version 1567304 (0.0005) [2023-12-27 16:30:15,403][20381] Updated weights for policy 1, policy_version 1565644 (0.0009) [2023-12-27 16:30:15,449][20381] Updated weights for policy 1, policy_version 1565654 (0.0010) [2023-12-27 16:30:15,502][20381] Updated weights for policy 1, policy_version 1565664 (0.0010) [2023-12-27 16:30:15,547][20347] Updated weights for policy 0, policy_version 1567314 (0.0008) [2023-12-27 16:30:15,599][20347] Updated weights for policy 0, policy_version 1567324 (0.0011) [2023-12-27 16:30:15,661][20347] Updated weights for policy 0, policy_version 1567334 (0.0010) [2023-12-27 16:30:15,715][20347] Updated weights for policy 0, policy_version 1567344 (0.0010) [2023-12-27 16:30:16,164][20381] Updated weights for policy 1, policy_version 1565674 (0.0009) [2023-12-27 16:30:16,228][20381] Updated weights for policy 1, policy_version 1565684 (0.0007) [2023-12-27 16:30:16,280][20381] Updated weights for policy 1, policy_version 1565694 (0.0009) [2023-12-27 16:30:16,332][20381] Updated weights for policy 1, policy_version 1565704 (0.0009) [2023-12-27 16:30:16,367][20347] Updated weights for policy 0, policy_version 1567354 (0.0005) [2023-12-27 16:30:16,423][20347] Updated weights for policy 0, policy_version 1567364 (0.0005) [2023-12-27 16:30:16,480][20347] Updated weights for policy 0, policy_version 1567374 (0.0007) [2023-12-27 16:30:17,009][20381] Updated weights for policy 1, policy_version 1565714 (0.0010) [2023-12-27 16:30:17,054][20381] Updated weights for policy 1, policy_version 1565724 (0.0009) [2023-12-27 16:30:17,094][20347] Updated weights for policy 0, policy_version 1567384 (0.0006) [2023-12-27 16:30:17,110][20381] Updated weights for policy 1, policy_version 1565734 (0.0007) [2023-12-27 16:30:17,140][20347] Updated weights for policy 0, policy_version 1567394 (0.0005) [2023-12-27 16:30:17,184][20347] Updated weights for policy 0, policy_version 1567404 (0.0005) [2023-12-27 16:30:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 802201600. Throughput: 0: 10286.6, 1: 10253.3. Samples: 802207312. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-12-27 16:30:17,363][19170] Avg episode reward: [(0, '931.710'), (1, '867.420')] [2023-12-27 16:30:17,776][20347] Updated weights for policy 0, policy_version 1567414 (0.0007) [2023-12-27 16:30:17,837][20347] Updated weights for policy 0, policy_version 1567424 (0.0007) [2023-12-27 16:30:17,857][20381] Updated weights for policy 1, policy_version 1565744 (0.0006) [2023-12-27 16:30:17,886][20347] Updated weights for policy 0, policy_version 1567434 (0.0008) [2023-12-27 16:30:17,906][20381] Updated weights for policy 1, policy_version 1565754 (0.0005) [2023-12-27 16:30:17,965][20381] Updated weights for policy 1, policy_version 1565764 (0.0005) [2023-12-27 16:30:18,541][20381] Updated weights for policy 1, policy_version 1565774 (0.0007) [2023-12-27 16:30:18,602][20381] Updated weights for policy 1, policy_version 1565784 (0.0007) [2023-12-27 16:30:18,603][20347] Updated weights for policy 0, policy_version 1567444 (0.0007) [2023-12-27 16:30:18,656][20381] Updated weights for policy 1, policy_version 1565794 (0.0006) [2023-12-27 16:30:18,665][20347] Updated weights for policy 0, policy_version 1567454 (0.0007) [2023-12-27 16:30:18,722][20347] Updated weights for policy 0, policy_version 1567464 (0.0008) [2023-12-27 16:30:19,373][20381] Updated weights for policy 1, policy_version 1565804 (0.0009) [2023-12-27 16:30:19,424][20347] Updated weights for policy 0, policy_version 1567474 (0.0007) [2023-12-27 16:30:19,434][20381] Updated weights for policy 1, policy_version 1565814 (0.0011) [2023-12-27 16:30:19,481][20347] Updated weights for policy 0, policy_version 1567484 (0.0007) [2023-12-27 16:30:19,498][20381] Updated weights for policy 1, policy_version 1565824 (0.0010) [2023-12-27 16:30:19,548][20347] Updated weights for policy 0, policy_version 1567494 (0.0007) [2023-12-27 16:30:19,607][20347] Updated weights for policy 0, policy_version 1567504 (0.0008) [2023-12-27 16:30:20,214][20381] Updated weights for policy 1, policy_version 1565834 (0.0007) [2023-12-27 16:30:20,276][20381] Updated weights for policy 1, policy_version 1565844 (0.0006) [2023-12-27 16:30:20,336][20381] Updated weights for policy 1, policy_version 1565854 (0.0006) [2023-12-27 16:30:20,375][20347] Updated weights for policy 0, policy_version 1567514 (0.0007) [2023-12-27 16:30:20,390][20381] Updated weights for policy 1, policy_version 1565864 (0.0007) [2023-12-27 16:30:20,431][20347] Updated weights for policy 0, policy_version 1567524 (0.0008) [2023-12-27 16:30:20,490][20347] Updated weights for policy 0, policy_version 1567534 (0.0008) [2023-12-27 16:30:21,015][20381] Updated weights for policy 1, policy_version 1565874 (0.0008) [2023-12-27 16:30:21,079][20381] Updated weights for policy 1, policy_version 1565884 (0.0008) [2023-12-27 16:30:21,151][20381] Updated weights for policy 1, policy_version 1565894 (0.0008) [2023-12-27 16:30:21,252][20347] Updated weights for policy 0, policy_version 1567545 (0.0007) [2023-12-27 16:30:21,316][20347] Updated weights for policy 0, policy_version 1567555 (0.0008) [2023-12-27 16:30:21,381][20347] Updated weights for policy 0, policy_version 1567565 (0.0008) [2023-12-27 16:30:21,951][20381] Updated weights for policy 1, policy_version 1565904 (0.0008) [2023-12-27 16:30:22,001][20381] Updated weights for policy 1, policy_version 1565914 (0.0009) [2023-12-27 16:30:22,056][20381] Updated weights for policy 1, policy_version 1565924 (0.0009) [2023-12-27 16:30:22,180][20347] Updated weights for policy 0, policy_version 1567575 (0.0009) [2023-12-27 16:30:22,236][20347] Updated weights for policy 0, policy_version 1567585 (0.0009) [2023-12-27 16:30:22,305][20347] Updated weights for policy 0, policy_version 1567595 (0.0009) [2023-12-27 16:30:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 802299904. Throughput: 0: 10244.3, 1: 10282.4. Samples: 802268260. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:22,363][19170] Avg episode reward: [(0, '902.200'), (1, '852.270')] [2023-12-27 16:30:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001565928_400916480.pth... [2023-12-27 16:30:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001567600_401383424.pth... [2023-12-27 16:30:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001564744_400613376.pth [2023-12-27 16:30:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001566384_401072128.pth [2023-12-27 16:30:22,825][20381] Updated weights for policy 1, policy_version 1565934 (0.0008) [2023-12-27 16:30:22,887][20381] Updated weights for policy 1, policy_version 1565944 (0.0009) [2023-12-27 16:30:22,948][20381] Updated weights for policy 1, policy_version 1565954 (0.0007) [2023-12-27 16:30:23,050][20347] Updated weights for policy 0, policy_version 1567605 (0.0010) [2023-12-27 16:30:23,108][20347] Updated weights for policy 0, policy_version 1567615 (0.0010) [2023-12-27 16:30:23,167][20347] Updated weights for policy 0, policy_version 1567625 (0.0010) [2023-12-27 16:30:23,587][20381] Updated weights for policy 1, policy_version 1565964 (0.0009) [2023-12-27 16:30:23,638][20381] Updated weights for policy 1, policy_version 1565974 (0.0008) [2023-12-27 16:30:23,692][20381] Updated weights for policy 1, policy_version 1565984 (0.0007) [2023-12-27 16:30:23,811][20347] Updated weights for policy 0, policy_version 1567635 (0.0009) [2023-12-27 16:30:23,863][20347] Updated weights for policy 0, policy_version 1567645 (0.0005) [2023-12-27 16:30:23,892][20291] KL-divergence is very high: 133.9076 [2023-12-27 16:30:23,916][20347] Updated weights for policy 0, policy_version 1567655 (0.0006) [2023-12-27 16:30:23,938][20291] KL-divergence is very high: 132.7937 [2023-12-27 16:30:24,368][20381] Updated weights for policy 1, policy_version 1565994 (0.0006) [2023-12-27 16:30:24,414][20381] Updated weights for policy 1, policy_version 1566004 (0.0007) [2023-12-27 16:30:24,462][20381] Updated weights for policy 1, policy_version 1566014 (0.0007) [2023-12-27 16:30:24,507][20381] Updated weights for policy 1, policy_version 1566024 (0.0008) [2023-12-27 16:30:24,583][20347] Updated weights for policy 0, policy_version 1567665 (0.0005) [2023-12-27 16:30:24,649][20347] Updated weights for policy 0, policy_version 1567675 (0.0007) [2023-12-27 16:30:24,714][20347] Updated weights for policy 0, policy_version 1567685 (0.0008) [2023-12-27 16:30:24,772][20347] Updated weights for policy 0, policy_version 1567695 (0.0009) [2023-12-27 16:30:25,223][20381] Updated weights for policy 1, policy_version 1566034 (0.0008) [2023-12-27 16:30:25,279][20381] Updated weights for policy 1, policy_version 1566044 (0.0008) [2023-12-27 16:30:25,343][20381] Updated weights for policy 1, policy_version 1566054 (0.0008) [2023-12-27 16:30:25,432][20347] Updated weights for policy 0, policy_version 1567705 (0.0011) [2023-12-27 16:30:25,477][20347] Updated weights for policy 0, policy_version 1567715 (0.0010) [2023-12-27 16:30:25,525][20347] Updated weights for policy 0, policy_version 1567725 (0.0010) [2023-12-27 16:30:25,949][20381] Updated weights for policy 1, policy_version 1566064 (0.0010) [2023-12-27 16:30:25,997][20381] Updated weights for policy 1, policy_version 1566074 (0.0010) [2023-12-27 16:30:26,052][20381] Updated weights for policy 1, policy_version 1566084 (0.0011) [2023-12-27 16:30:26,204][20347] Updated weights for policy 0, policy_version 1567735 (0.0008) [2023-12-27 16:30:26,264][20347] Updated weights for policy 0, policy_version 1567745 (0.0006) [2023-12-27 16:30:26,318][20347] Updated weights for policy 0, policy_version 1567755 (0.0007) [2023-12-27 16:30:26,721][20381] Updated weights for policy 1, policy_version 1566094 (0.0007) [2023-12-27 16:30:26,782][20381] Updated weights for policy 1, policy_version 1566104 (0.0005) [2023-12-27 16:30:26,835][20381] Updated weights for policy 1, policy_version 1566114 (0.0006) [2023-12-27 16:30:26,894][20347] Updated weights for policy 0, policy_version 1567765 (0.0007) [2023-12-27 16:30:26,958][20347] Updated weights for policy 0, policy_version 1567775 (0.0006) [2023-12-27 16:30:27,007][20347] Updated weights for policy 0, policy_version 1567785 (0.0005) [2023-12-27 16:30:27,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 802398208. Throughput: 0: 10196.2, 1: 10261.6. Samples: 802387340. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:27,363][19170] Avg episode reward: [(0, '902.020'), (1, '818.460')] [2023-12-27 16:30:27,410][20381] Updated weights for policy 1, policy_version 1566124 (0.0005) [2023-12-27 16:30:27,476][20381] Updated weights for policy 1, policy_version 1566134 (0.0007) [2023-12-27 16:30:27,530][20381] Updated weights for policy 1, policy_version 1566144 (0.0007) [2023-12-27 16:30:27,596][20347] Updated weights for policy 0, policy_version 1567795 (0.0005) [2023-12-27 16:30:27,647][20347] Updated weights for policy 0, policy_version 1567805 (0.0008) [2023-12-27 16:30:27,699][20347] Updated weights for policy 0, policy_version 1567815 (0.0008) [2023-12-27 16:30:28,153][20381] Updated weights for policy 1, policy_version 1566154 (0.0010) [2023-12-27 16:30:28,203][20381] Updated weights for policy 1, policy_version 1566164 (0.0006) [2023-12-27 16:30:28,256][20381] Updated weights for policy 1, policy_version 1566174 (0.0005) [2023-12-27 16:30:28,306][20381] Updated weights for policy 1, policy_version 1566184 (0.0005) [2023-12-27 16:30:28,432][20347] Updated weights for policy 0, policy_version 1567825 (0.0007) [2023-12-27 16:30:28,483][20347] Updated weights for policy 0, policy_version 1567835 (0.0008) [2023-12-27 16:30:28,534][20347] Updated weights for policy 0, policy_version 1567845 (0.0007) [2023-12-27 16:30:28,583][20347] Updated weights for policy 0, policy_version 1567855 (0.0008) [2023-12-27 16:30:28,960][20381] Updated weights for policy 1, policy_version 1566194 (0.0008) [2023-12-27 16:30:29,014][20381] Updated weights for policy 1, policy_version 1566204 (0.0005) [2023-12-27 16:30:29,070][20381] Updated weights for policy 1, policy_version 1566214 (0.0006) [2023-12-27 16:30:29,274][20347] Updated weights for policy 0, policy_version 1567865 (0.0007) [2023-12-27 16:30:29,335][20347] Updated weights for policy 0, policy_version 1567875 (0.0007) [2023-12-27 16:30:29,407][20347] Updated weights for policy 0, policy_version 1567885 (0.0008) [2023-12-27 16:30:29,716][20381] Updated weights for policy 1, policy_version 1566224 (0.0006) [2023-12-27 16:30:29,776][20381] Updated weights for policy 1, policy_version 1566234 (0.0007) [2023-12-27 16:30:29,830][20381] Updated weights for policy 1, policy_version 1566244 (0.0008) [2023-12-27 16:30:30,124][20347] Updated weights for policy 0, policy_version 1567895 (0.0007) [2023-12-27 16:30:30,175][20347] Updated weights for policy 0, policy_version 1567905 (0.0007) [2023-12-27 16:30:30,238][20347] Updated weights for policy 0, policy_version 1567915 (0.0007) [2023-12-27 16:30:30,519][20381] Updated weights for policy 1, policy_version 1566254 (0.0006) [2023-12-27 16:30:30,576][20381] Updated weights for policy 1, policy_version 1566264 (0.0006) [2023-12-27 16:30:30,629][20381] Updated weights for policy 1, policy_version 1566274 (0.0006) [2023-12-27 16:30:30,930][20347] Updated weights for policy 0, policy_version 1567925 (0.0008) [2023-12-27 16:30:30,993][20347] Updated weights for policy 0, policy_version 1567935 (0.0008) [2023-12-27 16:30:31,051][20347] Updated weights for policy 0, policy_version 1567945 (0.0007) [2023-12-27 16:30:31,231][20381] Updated weights for policy 1, policy_version 1566284 (0.0006) [2023-12-27 16:30:31,295][20381] Updated weights for policy 1, policy_version 1566294 (0.0008) [2023-12-27 16:30:31,355][20381] Updated weights for policy 1, policy_version 1566304 (0.0007) [2023-12-27 16:30:31,766][20347] Updated weights for policy 0, policy_version 1567955 (0.0007) [2023-12-27 16:30:31,822][20347] Updated weights for policy 0, policy_version 1567965 (0.0008) [2023-12-27 16:30:31,879][20347] Updated weights for policy 0, policy_version 1567975 (0.0008) [2023-12-27 16:30:32,052][20381] Updated weights for policy 1, policy_version 1566314 (0.0008) [2023-12-27 16:30:32,107][20381] Updated weights for policy 1, policy_version 1566324 (0.0007) [2023-12-27 16:30:32,159][20381] Updated weights for policy 1, policy_version 1566334 (0.0005) [2023-12-27 16:30:32,206][20381] Updated weights for policy 1, policy_version 1566344 (0.0005) [2023-12-27 16:30:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 802504704. Throughput: 0: 10195.5, 1: 10275.5. Samples: 802512352. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:32,363][19170] Avg episode reward: [(0, '882.670'), (1, '873.460')] [2023-12-27 16:30:32,596][20347] Updated weights for policy 0, policy_version 1567985 (0.0007) [2023-12-27 16:30:32,655][20347] Updated weights for policy 0, policy_version 1567995 (0.0008) [2023-12-27 16:30:32,714][20347] Updated weights for policy 0, policy_version 1568005 (0.0008) [2023-12-27 16:30:32,774][20347] Updated weights for policy 0, policy_version 1568015 (0.0008) [2023-12-27 16:30:32,846][20381] Updated weights for policy 1, policy_version 1566354 (0.0006) [2023-12-27 16:30:32,894][20381] Updated weights for policy 1, policy_version 1566364 (0.0008) [2023-12-27 16:30:32,945][20381] Updated weights for policy 1, policy_version 1566374 (0.0007) [2023-12-27 16:30:33,453][20347] Updated weights for policy 0, policy_version 1568025 (0.0006) [2023-12-27 16:30:33,501][20347] Updated weights for policy 0, policy_version 1568035 (0.0005) [2023-12-27 16:30:33,547][20347] Updated weights for policy 0, policy_version 1568045 (0.0005) [2023-12-27 16:30:33,579][20381] Updated weights for policy 1, policy_version 1566384 (0.0007) [2023-12-27 16:30:33,626][20381] Updated weights for policy 1, policy_version 1566394 (0.0008) [2023-12-27 16:30:33,677][20381] Updated weights for policy 1, policy_version 1566404 (0.0008) [2023-12-27 16:30:34,201][20347] Updated weights for policy 0, policy_version 1568055 (0.0007) [2023-12-27 16:30:34,257][20347] Updated weights for policy 0, policy_version 1568065 (0.0007) [2023-12-27 16:30:34,315][20381] Updated weights for policy 1, policy_version 1566414 (0.0007) [2023-12-27 16:30:34,317][20347] Updated weights for policy 0, policy_version 1568075 (0.0007) [2023-12-27 16:30:34,376][20381] Updated weights for policy 1, policy_version 1566424 (0.0006) [2023-12-27 16:30:34,430][20381] Updated weights for policy 1, policy_version 1566434 (0.0008) [2023-12-27 16:30:34,998][20347] Updated weights for policy 0, policy_version 1568085 (0.0007) [2023-12-27 16:30:35,054][20347] Updated weights for policy 0, policy_version 1568095 (0.0010) [2023-12-27 16:30:35,108][20347] Updated weights for policy 0, policy_version 1568105 (0.0007) [2023-12-27 16:30:35,118][20381] Updated weights for policy 1, policy_version 1566444 (0.0007) [2023-12-27 16:30:35,170][20381] Updated weights for policy 1, policy_version 1566454 (0.0006) [2023-12-27 16:30:35,213][20381] Updated weights for policy 1, policy_version 1566464 (0.0008) [2023-12-27 16:30:35,708][20347] Updated weights for policy 0, policy_version 1568115 (0.0006) [2023-12-27 16:30:35,793][20347] Updated weights for policy 0, policy_version 1568125 (0.0010) [2023-12-27 16:30:35,851][20347] Updated weights for policy 0, policy_version 1568135 (0.0007) [2023-12-27 16:30:35,950][20381] Updated weights for policy 1, policy_version 1566474 (0.0009) [2023-12-27 16:30:36,001][20381] Updated weights for policy 1, policy_version 1566484 (0.0009) [2023-12-27 16:30:36,045][20381] Updated weights for policy 1, policy_version 1566494 (0.0008) [2023-12-27 16:30:36,102][20381] Updated weights for policy 1, policy_version 1566504 (0.0007) [2023-12-27 16:30:36,415][20347] Updated weights for policy 0, policy_version 1568145 (0.0005) [2023-12-27 16:30:36,478][20347] Updated weights for policy 0, policy_version 1568155 (0.0007) [2023-12-27 16:30:36,537][20347] Updated weights for policy 0, policy_version 1568165 (0.0011) [2023-12-27 16:30:36,601][20347] Updated weights for policy 0, policy_version 1568175 (0.0011) [2023-12-27 16:30:36,857][20381] Updated weights for policy 1, policy_version 1566514 (0.0008) [2023-12-27 16:30:36,907][20381] Updated weights for policy 1, policy_version 1566524 (0.0006) [2023-12-27 16:30:36,962][20381] Updated weights for policy 1, policy_version 1566534 (0.0008) [2023-12-27 16:30:37,263][20347] Updated weights for policy 0, policy_version 1568185 (0.0010) [2023-12-27 16:30:37,323][20347] Updated weights for policy 0, policy_version 1568195 (0.0008) [2023-12-27 16:30:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 802603008. Throughput: 0: 10175.2, 1: 10317.5. Samples: 802575348. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:37,365][19170] Avg episode reward: [(0, '902.250'), (1, '912.010')] [2023-12-27 16:30:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001566536_401072128.pth... [2023-12-27 16:30:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001565352_400769024.pth [2023-12-27 16:30:37,385][20347] Updated weights for policy 0, policy_version 1568205 (0.0008) [2023-12-27 16:30:37,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001568208_401539072.pth... [2023-12-27 16:30:37,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001566992_401227776.pth [2023-12-27 16:30:37,679][20381] Updated weights for policy 1, policy_version 1566544 (0.0007) [2023-12-27 16:30:37,731][20381] Updated weights for policy 1, policy_version 1566554 (0.0007) [2023-12-27 16:30:37,783][20381] Updated weights for policy 1, policy_version 1566564 (0.0008) [2023-12-27 16:30:38,094][20347] Updated weights for policy 0, policy_version 1568215 (0.0006) [2023-12-27 16:30:38,146][20347] Updated weights for policy 0, policy_version 1568225 (0.0007) [2023-12-27 16:30:38,203][20347] Updated weights for policy 0, policy_version 1568235 (0.0006) [2023-12-27 16:30:38,443][20381] Updated weights for policy 1, policy_version 1566574 (0.0009) [2023-12-27 16:30:38,500][20381] Updated weights for policy 1, policy_version 1566584 (0.0011) [2023-12-27 16:30:38,553][20381] Updated weights for policy 1, policy_version 1566594 (0.0011) [2023-12-27 16:30:38,872][20347] Updated weights for policy 0, policy_version 1568245 (0.0007) [2023-12-27 16:30:38,932][20347] Updated weights for policy 0, policy_version 1568255 (0.0008) [2023-12-27 16:30:38,989][20347] Updated weights for policy 0, policy_version 1568265 (0.0006) [2023-12-27 16:30:39,247][20381] Updated weights for policy 1, policy_version 1566604 (0.0010) [2023-12-27 16:30:39,315][20381] Updated weights for policy 1, policy_version 1566614 (0.0007) [2023-12-27 16:30:39,385][20381] Updated weights for policy 1, policy_version 1566624 (0.0010) [2023-12-27 16:30:39,713][20347] Updated weights for policy 0, policy_version 1568275 (0.0006) [2023-12-27 16:30:39,777][20347] Updated weights for policy 0, policy_version 1568285 (0.0008) [2023-12-27 16:30:39,842][20347] Updated weights for policy 0, policy_version 1568295 (0.0008) [2023-12-27 16:30:40,145][20381] Updated weights for policy 1, policy_version 1566634 (0.0009) [2023-12-27 16:30:40,209][20381] Updated weights for policy 1, policy_version 1566644 (0.0007) [2023-12-27 16:30:40,271][20381] Updated weights for policy 1, policy_version 1566654 (0.0008) [2023-12-27 16:30:40,334][20381] Updated weights for policy 1, policy_version 1566664 (0.0008) [2023-12-27 16:30:40,564][20347] Updated weights for policy 0, policy_version 1568305 (0.0007) [2023-12-27 16:30:40,613][20347] Updated weights for policy 0, policy_version 1568315 (0.0006) [2023-12-27 16:30:40,670][20347] Updated weights for policy 0, policy_version 1568325 (0.0009) [2023-12-27 16:30:40,730][20347] Updated weights for policy 0, policy_version 1568335 (0.0009) [2023-12-27 16:30:41,089][20381] Updated weights for policy 1, policy_version 1566674 (0.0013) [2023-12-27 16:30:41,171][20381] Updated weights for policy 1, policy_version 1566685 (0.0013) [2023-12-27 16:30:41,233][20381] Updated weights for policy 1, policy_version 1566695 (0.0008) [2023-12-27 16:30:41,515][20347] Updated weights for policy 0, policy_version 1568345 (0.0008) [2023-12-27 16:30:41,585][20347] Updated weights for policy 0, policy_version 1568355 (0.0007) [2023-12-27 16:30:41,653][20347] Updated weights for policy 0, policy_version 1568365 (0.0008) [2023-12-27 16:30:41,995][20381] Updated weights for policy 1, policy_version 1566705 (0.0011) [2023-12-27 16:30:42,060][20381] Updated weights for policy 1, policy_version 1566715 (0.0006) [2023-12-27 16:30:42,126][20381] Updated weights for policy 1, policy_version 1566725 (0.0008) [2023-12-27 16:30:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 802701312. Throughput: 0: 10199.8, 1: 10204.1. Samples: 802694672. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:42,363][19170] Avg episode reward: [(0, '911.910'), (1, '892.880')] [2023-12-27 16:30:42,401][20347] Updated weights for policy 0, policy_version 1568375 (0.0008) [2023-12-27 16:30:42,460][20347] Updated weights for policy 0, policy_version 1568385 (0.0008) [2023-12-27 16:30:42,520][20347] Updated weights for policy 0, policy_version 1568395 (0.0008) [2023-12-27 16:30:42,789][20381] Updated weights for policy 1, policy_version 1566735 (0.0006) [2023-12-27 16:30:42,856][20381] Updated weights for policy 1, policy_version 1566745 (0.0009) [2023-12-27 16:30:42,919][20381] Updated weights for policy 1, policy_version 1566755 (0.0011) [2023-12-27 16:30:43,234][20347] Updated weights for policy 0, policy_version 1568405 (0.0006) [2023-12-27 16:30:43,289][20347] Updated weights for policy 0, policy_version 1568415 (0.0008) [2023-12-27 16:30:43,346][20347] Updated weights for policy 0, policy_version 1568425 (0.0006) [2023-12-27 16:30:43,595][20381] Updated weights for policy 1, policy_version 1566765 (0.0011) [2023-12-27 16:30:43,647][20381] Updated weights for policy 1, policy_version 1566775 (0.0011) [2023-12-27 16:30:43,705][20381] Updated weights for policy 1, policy_version 1566785 (0.0011) [2023-12-27 16:30:43,964][20347] Updated weights for policy 0, policy_version 1568435 (0.0008) [2023-12-27 16:30:44,011][20347] Updated weights for policy 0, policy_version 1568445 (0.0007) [2023-12-27 16:30:44,068][20347] Updated weights for policy 0, policy_version 1568455 (0.0006) [2023-12-27 16:30:44,360][20381] Updated weights for policy 1, policy_version 1566795 (0.0009) [2023-12-27 16:30:44,417][20381] Updated weights for policy 1, policy_version 1566805 (0.0007) [2023-12-27 16:30:44,481][20381] Updated weights for policy 1, policy_version 1566815 (0.0008) [2023-12-27 16:30:44,761][20347] Updated weights for policy 0, policy_version 1568465 (0.0008) [2023-12-27 16:30:44,829][20347] Updated weights for policy 0, policy_version 1568475 (0.0009) [2023-12-27 16:30:44,889][20347] Updated weights for policy 0, policy_version 1568485 (0.0007) [2023-12-27 16:30:44,953][20347] Updated weights for policy 0, policy_version 1568495 (0.0007) [2023-12-27 16:30:45,202][20381] Updated weights for policy 1, policy_version 1566825 (0.0006) [2023-12-27 16:30:45,262][20381] Updated weights for policy 1, policy_version 1566835 (0.0008) [2023-12-27 16:30:45,324][20381] Updated weights for policy 1, policy_version 1566845 (0.0007) [2023-12-27 16:30:45,380][20381] Updated weights for policy 1, policy_version 1566855 (0.0006) [2023-12-27 16:30:45,676][20347] Updated weights for policy 0, policy_version 1568505 (0.0010) [2023-12-27 16:30:45,734][20347] Updated weights for policy 0, policy_version 1568515 (0.0010) [2023-12-27 16:30:45,790][20347] Updated weights for policy 0, policy_version 1568525 (0.0010) [2023-12-27 16:30:46,039][20381] Updated weights for policy 1, policy_version 1566865 (0.0007) [2023-12-27 16:30:46,096][20381] Updated weights for policy 1, policy_version 1566875 (0.0008) [2023-12-27 16:30:46,153][20381] Updated weights for policy 1, policy_version 1566885 (0.0008) [2023-12-27 16:30:46,472][20347] Updated weights for policy 0, policy_version 1568535 (0.0007) [2023-12-27 16:30:46,522][20347] Updated weights for policy 0, policy_version 1568545 (0.0007) [2023-12-27 16:30:46,577][20347] Updated weights for policy 0, policy_version 1568555 (0.0010) [2023-12-27 16:30:46,872][20381] Updated weights for policy 1, policy_version 1566895 (0.0007) [2023-12-27 16:30:46,927][20381] Updated weights for policy 1, policy_version 1566905 (0.0007) [2023-12-27 16:30:46,988][20381] Updated weights for policy 1, policy_version 1566915 (0.0006) [2023-12-27 16:30:47,216][20347] Updated weights for policy 0, policy_version 1568565 (0.0009) [2023-12-27 16:30:47,269][20347] Updated weights for policy 0, policy_version 1568575 (0.0010) [2023-12-27 16:30:47,318][20347] Updated weights for policy 0, policy_version 1568585 (0.0009) [2023-12-27 16:30:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 802807808. Throughput: 0: 10154.0, 1: 10152.4. Samples: 802813424. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:47,363][19170] Avg episode reward: [(0, '911.930'), (1, '843.820')] [2023-12-27 16:30:47,692][20381] Updated weights for policy 1, policy_version 1566925 (0.0007) [2023-12-27 16:30:47,750][20381] Updated weights for policy 1, policy_version 1566935 (0.0007) [2023-12-27 16:30:47,804][20381] Updated weights for policy 1, policy_version 1566945 (0.0007) [2023-12-27 16:30:47,998][20347] Updated weights for policy 0, policy_version 1568595 (0.0007) [2023-12-27 16:30:48,054][20347] Updated weights for policy 0, policy_version 1568605 (0.0010) [2023-12-27 16:30:48,117][20347] Updated weights for policy 0, policy_version 1568615 (0.0010) [2023-12-27 16:30:48,489][20381] Updated weights for policy 1, policy_version 1566955 (0.0007) [2023-12-27 16:30:48,550][20381] Updated weights for policy 1, policy_version 1566965 (0.0008) [2023-12-27 16:30:48,607][20381] Updated weights for policy 1, policy_version 1566975 (0.0008) [2023-12-27 16:30:48,809][20347] Updated weights for policy 0, policy_version 1568625 (0.0010) [2023-12-27 16:30:48,871][20347] Updated weights for policy 0, policy_version 1568635 (0.0010) [2023-12-27 16:30:48,927][20347] Updated weights for policy 0, policy_version 1568645 (0.0008) [2023-12-27 16:30:48,978][20347] Updated weights for policy 0, policy_version 1568655 (0.0005) [2023-12-27 16:30:49,166][20381] Updated weights for policy 1, policy_version 1566985 (0.0008) [2023-12-27 16:30:49,231][20381] Updated weights for policy 1, policy_version 1566995 (0.0008) [2023-12-27 16:30:49,288][20381] Updated weights for policy 1, policy_version 1567005 (0.0010) [2023-12-27 16:30:49,352][20381] Updated weights for policy 1, policy_version 1567015 (0.0011) [2023-12-27 16:30:49,606][20347] Updated weights for policy 0, policy_version 1568665 (0.0006) [2023-12-27 16:30:49,663][20347] Updated weights for policy 0, policy_version 1568675 (0.0005) [2023-12-27 16:30:49,716][20347] Updated weights for policy 0, policy_version 1568685 (0.0006) [2023-12-27 16:30:50,018][20381] Updated weights for policy 1, policy_version 1567025 (0.0008) [2023-12-27 16:30:50,079][20381] Updated weights for policy 1, policy_version 1567035 (0.0007) [2023-12-27 16:30:50,142][20381] Updated weights for policy 1, policy_version 1567045 (0.0007) [2023-12-27 16:30:50,395][20347] Updated weights for policy 0, policy_version 1568695 (0.0007) [2023-12-27 16:30:50,455][20347] Updated weights for policy 0, policy_version 1568705 (0.0008) [2023-12-27 16:30:50,514][20347] Updated weights for policy 0, policy_version 1568715 (0.0008) [2023-12-27 16:30:50,863][20381] Updated weights for policy 1, policy_version 1567055 (0.0008) [2023-12-27 16:30:50,928][20381] Updated weights for policy 1, policy_version 1567065 (0.0007) [2023-12-27 16:30:50,989][20381] Updated weights for policy 1, policy_version 1567075 (0.0008) [2023-12-27 16:30:51,175][20347] Updated weights for policy 0, policy_version 1568725 (0.0007) [2023-12-27 16:30:51,235][20347] Updated weights for policy 0, policy_version 1568735 (0.0007) [2023-12-27 16:30:51,295][20347] Updated weights for policy 0, policy_version 1568745 (0.0007) [2023-12-27 16:30:51,759][20381] Updated weights for policy 1, policy_version 1567085 (0.0008) [2023-12-27 16:30:51,805][20381] Updated weights for policy 1, policy_version 1567095 (0.0008) [2023-12-27 16:30:51,868][20381] Updated weights for policy 1, policy_version 1567105 (0.0009) [2023-12-27 16:30:51,982][20347] Updated weights for policy 0, policy_version 1568755 (0.0007) [2023-12-27 16:30:52,047][20347] Updated weights for policy 0, policy_version 1568765 (0.0009) [2023-12-27 16:30:52,095][20347] Updated weights for policy 0, policy_version 1568775 (0.0009) [2023-12-27 16:30:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.1, 300 sec: 20521.6). Total num frames: 802906112. Throughput: 0: 10109.2, 1: 10165.8. Samples: 802875820. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:52,363][19170] Avg episode reward: [(0, '931.580'), (1, '892.680')] [2023-12-27 16:30:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001567112_401219584.pth... [2023-12-27 16:30:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001568784_401686528.pth... [2023-12-27 16:30:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001565928_400916480.pth [2023-12-27 16:30:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001567600_401383424.pth [2023-12-27 16:30:52,675][20381] Updated weights for policy 1, policy_version 1567115 (0.0007) [2023-12-27 16:30:52,733][20381] Updated weights for policy 1, policy_version 1567125 (0.0005) [2023-12-27 16:30:52,782][20347] Updated weights for policy 0, policy_version 1568785 (0.0008) [2023-12-27 16:30:52,794][20381] Updated weights for policy 1, policy_version 1567135 (0.0009) [2023-12-27 16:30:52,841][20347] Updated weights for policy 0, policy_version 1568795 (0.0006) [2023-12-27 16:30:52,905][20347] Updated weights for policy 0, policy_version 1568805 (0.0006) [2023-12-27 16:30:52,970][20347] Updated weights for policy 0, policy_version 1568815 (0.0005) [2023-12-27 16:30:53,436][20381] Updated weights for policy 1, policy_version 1567145 (0.0006) [2023-12-27 16:30:53,499][20381] Updated weights for policy 1, policy_version 1567155 (0.0007) [2023-12-27 16:30:53,522][20347] Updated weights for policy 0, policy_version 1568825 (0.0007) [2023-12-27 16:30:53,560][20381] Updated weights for policy 1, policy_version 1567165 (0.0006) [2023-12-27 16:30:53,576][20347] Updated weights for policy 0, policy_version 1568835 (0.0008) [2023-12-27 16:30:53,611][20381] Updated weights for policy 1, policy_version 1567175 (0.0005) [2023-12-27 16:30:53,631][20347] Updated weights for policy 0, policy_version 1568845 (0.0007) [2023-12-27 16:30:54,213][20381] Updated weights for policy 1, policy_version 1567185 (0.0006) [2023-12-27 16:30:54,259][20381] Updated weights for policy 1, policy_version 1567195 (0.0005) [2023-12-27 16:30:54,302][20347] Updated weights for policy 0, policy_version 1568855 (0.0009) [2023-12-27 16:30:54,307][20381] Updated weights for policy 1, policy_version 1567205 (0.0005) [2023-12-27 16:30:54,353][20347] Updated weights for policy 0, policy_version 1568865 (0.0010) [2023-12-27 16:30:54,416][20347] Updated weights for policy 0, policy_version 1568875 (0.0008) [2023-12-27 16:30:55,030][20381] Updated weights for policy 1, policy_version 1567215 (0.0008) [2023-12-27 16:30:55,090][20381] Updated weights for policy 1, policy_version 1567225 (0.0007) [2023-12-27 16:30:55,104][20347] Updated weights for policy 0, policy_version 1568885 (0.0007) [2023-12-27 16:30:55,147][20381] Updated weights for policy 1, policy_version 1567235 (0.0008) [2023-12-27 16:30:55,162][20347] Updated weights for policy 0, policy_version 1568895 (0.0009) [2023-12-27 16:30:55,217][20347] Updated weights for policy 0, policy_version 1568905 (0.0010) [2023-12-27 16:30:55,807][20381] Updated weights for policy 1, policy_version 1567245 (0.0008) [2023-12-27 16:30:55,855][20381] Updated weights for policy 1, policy_version 1567255 (0.0007) [2023-12-27 16:30:55,887][20347] Updated weights for policy 0, policy_version 1568915 (0.0010) [2023-12-27 16:30:55,897][20381] Updated weights for policy 1, policy_version 1567265 (0.0008) [2023-12-27 16:30:55,933][20347] Updated weights for policy 0, policy_version 1568925 (0.0010) [2023-12-27 16:30:55,990][20347] Updated weights for policy 0, policy_version 1568935 (0.0007) [2023-12-27 16:30:56,536][20381] Updated weights for policy 1, policy_version 1567275 (0.0006) [2023-12-27 16:30:56,584][20381] Updated weights for policy 1, policy_version 1567285 (0.0005) [2023-12-27 16:30:56,584][20347] Updated weights for policy 0, policy_version 1568945 (0.0007) [2023-12-27 16:30:56,634][20347] Updated weights for policy 0, policy_version 1568955 (0.0006) [2023-12-27 16:30:56,638][20381] Updated weights for policy 1, policy_version 1567295 (0.0006) [2023-12-27 16:30:56,683][20347] Updated weights for policy 0, policy_version 1568965 (0.0006) [2023-12-27 16:30:56,738][20347] Updated weights for policy 0, policy_version 1568975 (0.0006) [2023-12-27 16:30:57,234][20381] Updated weights for policy 1, policy_version 1567305 (0.0010) [2023-12-27 16:30:57,302][20381] Updated weights for policy 1, policy_version 1567315 (0.0009) [2023-12-27 16:30:57,362][20381] Updated weights for policy 1, policy_version 1567325 (0.0009) [2023-12-27 16:30:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 803004416. Throughput: 0: 10149.6, 1: 10132.6. Samples: 802998116. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:30:57,363][19170] Avg episode reward: [(0, '941.460'), (1, '749.030')] [2023-12-27 16:30:57,399][20347] Updated weights for policy 0, policy_version 1568985 (0.0010) [2023-12-27 16:30:57,426][20381] Updated weights for policy 1, policy_version 1567335 (0.0008) [2023-12-27 16:30:57,442][20347] Updated weights for policy 0, policy_version 1568995 (0.0009) [2023-12-27 16:30:57,492][20347] Updated weights for policy 0, policy_version 1569005 (0.0009) [2023-12-27 16:30:58,029][20381] Updated weights for policy 1, policy_version 1567345 (0.0010) [2023-12-27 16:30:58,079][20381] Updated weights for policy 1, policy_version 1567355 (0.0010) [2023-12-27 16:30:58,128][20381] Updated weights for policy 1, policy_version 1567365 (0.0011) [2023-12-27 16:30:58,139][20347] Updated weights for policy 0, policy_version 1569015 (0.0005) [2023-12-27 16:30:58,203][20347] Updated weights for policy 0, policy_version 1569025 (0.0007) [2023-12-27 16:30:58,267][20347] Updated weights for policy 0, policy_version 1569035 (0.0011) [2023-12-27 16:30:58,908][20381] Updated weights for policy 1, policy_version 1567375 (0.0012) [2023-12-27 16:30:58,963][20381] Updated weights for policy 1, policy_version 1567385 (0.0010) [2023-12-27 16:30:59,019][20381] Updated weights for policy 1, policy_version 1567395 (0.0007) [2023-12-27 16:30:59,054][20347] Updated weights for policy 0, policy_version 1569045 (0.0009) [2023-12-27 16:30:59,114][20347] Updated weights for policy 0, policy_version 1569055 (0.0010) [2023-12-27 16:30:59,169][20347] Updated weights for policy 0, policy_version 1569065 (0.0010) [2023-12-27 16:30:59,717][20381] Updated weights for policy 1, policy_version 1567405 (0.0007) [2023-12-27 16:30:59,770][20381] Updated weights for policy 1, policy_version 1567415 (0.0005) [2023-12-27 16:30:59,834][20381] Updated weights for policy 1, policy_version 1567425 (0.0010) [2023-12-27 16:30:59,882][20347] Updated weights for policy 0, policy_version 1569075 (0.0010) [2023-12-27 16:30:59,946][20347] Updated weights for policy 0, policy_version 1569085 (0.0008) [2023-12-27 16:31:00,004][20347] Updated weights for policy 0, policy_version 1569095 (0.0007) [2023-12-27 16:31:00,452][20381] Updated weights for policy 1, policy_version 1567435 (0.0009) [2023-12-27 16:31:00,501][20381] Updated weights for policy 1, policy_version 1567445 (0.0008) [2023-12-27 16:31:00,553][20381] Updated weights for policy 1, policy_version 1567455 (0.0007) [2023-12-27 16:31:00,660][20347] Updated weights for policy 0, policy_version 1569105 (0.0008) [2023-12-27 16:31:00,714][20347] Updated weights for policy 0, policy_version 1569115 (0.0010) [2023-12-27 16:31:00,776][20347] Updated weights for policy 0, policy_version 1569125 (0.0010) [2023-12-27 16:31:00,836][20347] Updated weights for policy 0, policy_version 1569135 (0.0010) [2023-12-27 16:31:01,216][20381] Updated weights for policy 1, policy_version 1567465 (0.0007) [2023-12-27 16:31:01,277][20381] Updated weights for policy 1, policy_version 1567475 (0.0008) [2023-12-27 16:31:01,329][20381] Updated weights for policy 1, policy_version 1567485 (0.0008) [2023-12-27 16:31:01,397][20381] Updated weights for policy 1, policy_version 1567495 (0.0008) [2023-12-27 16:31:01,517][20347] Updated weights for policy 0, policy_version 1569145 (0.0008) [2023-12-27 16:31:01,577][20347] Updated weights for policy 0, policy_version 1569155 (0.0010) [2023-12-27 16:31:01,639][20347] Updated weights for policy 0, policy_version 1569165 (0.0010) [2023-12-27 16:31:02,228][20381] Updated weights for policy 1, policy_version 1567505 (0.0006) [2023-12-27 16:31:02,288][20381] Updated weights for policy 1, policy_version 1567515 (0.0007) [2023-12-27 16:31:02,347][20381] Updated weights for policy 1, policy_version 1567525 (0.0008) [2023-12-27 16:31:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 803102720. Throughput: 0: 10120.7, 1: 10166.3. Samples: 803120228. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:02,363][19170] Avg episode reward: [(0, '931.560'), (1, '807.410')] [2023-12-27 16:31:02,374][20347] Updated weights for policy 0, policy_version 1569175 (0.0011) [2023-12-27 16:31:02,420][20347] Updated weights for policy 0, policy_version 1569185 (0.0006) [2023-12-27 16:31:02,469][20347] Updated weights for policy 0, policy_version 1569195 (0.0006) [2023-12-27 16:31:03,038][20347] Updated weights for policy 0, policy_version 1569205 (0.0009) [2023-12-27 16:31:03,061][20381] Updated weights for policy 1, policy_version 1567535 (0.0009) [2023-12-27 16:31:03,090][20347] Updated weights for policy 0, policy_version 1569215 (0.0009) [2023-12-27 16:31:03,114][20381] Updated weights for policy 1, policy_version 1567545 (0.0006) [2023-12-27 16:31:03,142][20347] Updated weights for policy 0, policy_version 1569225 (0.0010) [2023-12-27 16:31:03,166][20381] Updated weights for policy 1, policy_version 1567555 (0.0005) [2023-12-27 16:31:03,761][20347] Updated weights for policy 0, policy_version 1569235 (0.0009) [2023-12-27 16:31:03,792][20381] Updated weights for policy 1, policy_version 1567565 (0.0006) [2023-12-27 16:31:03,813][20347] Updated weights for policy 0, policy_version 1569245 (0.0007) [2023-12-27 16:31:03,853][20381] Updated weights for policy 1, policy_version 1567575 (0.0008) [2023-12-27 16:31:03,877][20347] Updated weights for policy 0, policy_version 1569255 (0.0010) [2023-12-27 16:31:03,910][20381] Updated weights for policy 1, policy_version 1567585 (0.0007) [2023-12-27 16:31:04,529][20347] Updated weights for policy 0, policy_version 1569265 (0.0010) [2023-12-27 16:31:04,564][20381] Updated weights for policy 1, policy_version 1567595 (0.0007) [2023-12-27 16:31:04,584][20347] Updated weights for policy 0, policy_version 1569275 (0.0010) [2023-12-27 16:31:04,615][20381] Updated weights for policy 1, policy_version 1567605 (0.0005) [2023-12-27 16:31:04,629][20347] Updated weights for policy 0, policy_version 1569285 (0.0010) [2023-12-27 16:31:04,668][20381] Updated weights for policy 1, policy_version 1567615 (0.0008) [2023-12-27 16:31:04,679][20347] Updated weights for policy 0, policy_version 1569295 (0.0010) [2023-12-27 16:31:05,314][20347] Updated weights for policy 0, policy_version 1569305 (0.0006) [2023-12-27 16:31:05,333][20381] Updated weights for policy 1, policy_version 1567625 (0.0008) [2023-12-27 16:31:05,365][20347] Updated weights for policy 0, policy_version 1569315 (0.0005) [2023-12-27 16:31:05,382][20381] Updated weights for policy 1, policy_version 1567635 (0.0008) [2023-12-27 16:31:05,419][20347] Updated weights for policy 0, policy_version 1569325 (0.0005) [2023-12-27 16:31:05,434][20381] Updated weights for policy 1, policy_version 1567645 (0.0008) [2023-12-27 16:31:05,492][20381] Updated weights for policy 1, policy_version 1567655 (0.0009) [2023-12-27 16:31:06,040][20347] Updated weights for policy 0, policy_version 1569335 (0.0009) [2023-12-27 16:31:06,089][20347] Updated weights for policy 0, policy_version 1569345 (0.0010) [2023-12-27 16:31:06,136][20381] Updated weights for policy 1, policy_version 1567665 (0.0010) [2023-12-27 16:31:06,155][20347] Updated weights for policy 0, policy_version 1569355 (0.0008) [2023-12-27 16:31:06,198][20381] Updated weights for policy 1, policy_version 1567675 (0.0011) [2023-12-27 16:31:06,259][20381] Updated weights for policy 1, policy_version 1567685 (0.0011) [2023-12-27 16:31:06,878][20347] Updated weights for policy 0, policy_version 1569365 (0.0008) [2023-12-27 16:31:06,929][20347] Updated weights for policy 0, policy_version 1569375 (0.0005) [2023-12-27 16:31:06,953][20381] Updated weights for policy 1, policy_version 1567695 (0.0011) [2023-12-27 16:31:06,979][20347] Updated weights for policy 0, policy_version 1569385 (0.0005) [2023-12-27 16:31:07,016][20381] Updated weights for policy 1, policy_version 1567705 (0.0010) [2023-12-27 16:31:07,061][20381] Updated weights for policy 1, policy_version 1567715 (0.0010) [2023-12-27 16:31:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 803217408. Throughput: 0: 10193.5, 1: 10178.2. Samples: 803184988. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:07,363][19170] Avg episode reward: [(0, '951.100'), (1, '894.400')] [2023-12-27 16:31:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001567720_401375232.pth... [2023-12-27 16:31:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001569392_401842176.pth... [2023-12-27 16:31:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001566536_401072128.pth [2023-12-27 16:31:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001568208_401539072.pth [2023-12-27 16:31:07,644][20347] Updated weights for policy 0, policy_version 1569395 (0.0008) [2023-12-27 16:31:07,695][20347] Updated weights for policy 0, policy_version 1569405 (0.0010) [2023-12-27 16:31:07,715][20381] Updated weights for policy 1, policy_version 1567725 (0.0011) [2023-12-27 16:31:07,751][20347] Updated weights for policy 0, policy_version 1569415 (0.0010) [2023-12-27 16:31:07,774][20381] Updated weights for policy 1, policy_version 1567735 (0.0011) [2023-12-27 16:31:07,833][20381] Updated weights for policy 1, policy_version 1567745 (0.0010) [2023-12-27 16:31:08,437][20347] Updated weights for policy 0, policy_version 1569425 (0.0010) [2023-12-27 16:31:08,485][20347] Updated weights for policy 0, policy_version 1569435 (0.0008) [2023-12-27 16:31:08,513][20381] Updated weights for policy 1, policy_version 1567755 (0.0010) [2023-12-27 16:31:08,540][20347] Updated weights for policy 0, policy_version 1569445 (0.0006) [2023-12-27 16:31:08,571][20381] Updated weights for policy 1, policy_version 1567765 (0.0011) [2023-12-27 16:31:08,599][20347] Updated weights for policy 0, policy_version 1569455 (0.0006) [2023-12-27 16:31:08,630][20381] Updated weights for policy 1, policy_version 1567775 (0.0011) [2023-12-27 16:31:09,318][20347] Updated weights for policy 0, policy_version 1569465 (0.0008) [2023-12-27 16:31:09,335][20381] Updated weights for policy 1, policy_version 1567785 (0.0010) [2023-12-27 16:31:09,382][20347] Updated weights for policy 0, policy_version 1569475 (0.0007) [2023-12-27 16:31:09,405][20381] Updated weights for policy 1, policy_version 1567795 (0.0009) [2023-12-27 16:31:09,444][20347] Updated weights for policy 0, policy_version 1569485 (0.0007) [2023-12-27 16:31:09,468][20381] Updated weights for policy 1, policy_version 1567805 (0.0011) [2023-12-27 16:31:09,533][20381] Updated weights for policy 1, policy_version 1567815 (0.0010) [2023-12-27 16:31:10,131][20347] Updated weights for policy 0, policy_version 1569495 (0.0007) [2023-12-27 16:31:10,189][20347] Updated weights for policy 0, policy_version 1569505 (0.0008) [2023-12-27 16:31:10,253][20347] Updated weights for policy 0, policy_version 1569515 (0.0008) [2023-12-27 16:31:10,307][20381] Updated weights for policy 1, policy_version 1567825 (0.0007) [2023-12-27 16:31:10,371][20381] Updated weights for policy 1, policy_version 1567835 (0.0008) [2023-12-27 16:31:10,433][20381] Updated weights for policy 1, policy_version 1567845 (0.0010) [2023-12-27 16:31:10,885][20347] Updated weights for policy 0, policy_version 1569525 (0.0007) [2023-12-27 16:31:10,948][20347] Updated weights for policy 0, policy_version 1569535 (0.0007) [2023-12-27 16:31:11,010][20347] Updated weights for policy 0, policy_version 1569545 (0.0008) [2023-12-27 16:31:11,111][20381] Updated weights for policy 1, policy_version 1567855 (0.0009) [2023-12-27 16:31:11,175][20381] Updated weights for policy 1, policy_version 1567865 (0.0008) [2023-12-27 16:31:11,245][20381] Updated weights for policy 1, policy_version 1567875 (0.0006) [2023-12-27 16:31:11,787][20347] Updated weights for policy 0, policy_version 1569555 (0.0008) [2023-12-27 16:31:11,839][20347] Updated weights for policy 0, policy_version 1569565 (0.0009) [2023-12-27 16:31:11,894][20347] Updated weights for policy 0, policy_version 1569575 (0.0009) [2023-12-27 16:31:11,971][20381] Updated weights for policy 1, policy_version 1567885 (0.0008) [2023-12-27 16:31:12,029][20381] Updated weights for policy 1, policy_version 1567895 (0.0006) [2023-12-27 16:31:12,088][20381] Updated weights for policy 1, policy_version 1567905 (0.0007) [2023-12-27 16:31:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 803315712. Throughput: 0: 10258.1, 1: 10167.9. Samples: 803306504. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:12,363][19170] Avg episode reward: [(0, '941.270'), (1, '912.820')] [2023-12-27 16:31:12,677][20347] Updated weights for policy 0, policy_version 1569585 (0.0009) [2023-12-27 16:31:12,732][20347] Updated weights for policy 0, policy_version 1569595 (0.0006) [2023-12-27 16:31:12,758][20381] Updated weights for policy 1, policy_version 1567915 (0.0007) [2023-12-27 16:31:12,793][20347] Updated weights for policy 0, policy_version 1569605 (0.0007) [2023-12-27 16:31:12,810][20381] Updated weights for policy 1, policy_version 1567925 (0.0008) [2023-12-27 16:31:12,852][20347] Updated weights for policy 0, policy_version 1569615 (0.0007) [2023-12-27 16:31:12,870][20381] Updated weights for policy 1, policy_version 1567935 (0.0008) [2023-12-27 16:31:13,488][20381] Updated weights for policy 1, policy_version 1567945 (0.0007) [2023-12-27 16:31:13,541][20381] Updated weights for policy 1, policy_version 1567955 (0.0005) [2023-12-27 16:31:13,552][20347] Updated weights for policy 0, policy_version 1569625 (0.0008) [2023-12-27 16:31:13,599][20381] Updated weights for policy 1, policy_version 1567965 (0.0006) [2023-12-27 16:31:13,612][20347] Updated weights for policy 0, policy_version 1569635 (0.0007) [2023-12-27 16:31:13,653][20381] Updated weights for policy 1, policy_version 1567975 (0.0005) [2023-12-27 16:31:13,664][20347] Updated weights for policy 0, policy_version 1569645 (0.0007) [2023-12-27 16:31:14,298][20381] Updated weights for policy 1, policy_version 1567985 (0.0008) [2023-12-27 16:31:14,355][20381] Updated weights for policy 1, policy_version 1567995 (0.0007) [2023-12-27 16:31:14,368][20347] Updated weights for policy 0, policy_version 1569655 (0.0006) [2023-12-27 16:31:14,404][20381] Updated weights for policy 1, policy_version 1568005 (0.0006) [2023-12-27 16:31:14,421][20347] Updated weights for policy 0, policy_version 1569665 (0.0006) [2023-12-27 16:31:14,482][20347] Updated weights for policy 0, policy_version 1569675 (0.0007) [2023-12-27 16:31:15,091][20347] Updated weights for policy 0, policy_version 1569685 (0.0007) [2023-12-27 16:31:15,099][20381] Updated weights for policy 1, policy_version 1568015 (0.0006) [2023-12-27 16:31:15,155][20347] Updated weights for policy 0, policy_version 1569695 (0.0006) [2023-12-27 16:31:15,157][20381] Updated weights for policy 1, policy_version 1568025 (0.0006) [2023-12-27 16:31:15,216][20381] Updated weights for policy 1, policy_version 1568035 (0.0006) [2023-12-27 16:31:15,217][20347] Updated weights for policy 0, policy_version 1569705 (0.0007) [2023-12-27 16:31:15,790][20347] Updated weights for policy 0, policy_version 1569715 (0.0006) [2023-12-27 16:31:15,838][20381] Updated weights for policy 1, policy_version 1568045 (0.0005) [2023-12-27 16:31:15,844][20347] Updated weights for policy 0, policy_version 1569725 (0.0005) [2023-12-27 16:31:15,890][20381] Updated weights for policy 1, policy_version 1568055 (0.0005) [2023-12-27 16:31:15,894][20347] Updated weights for policy 0, policy_version 1569735 (0.0007) [2023-12-27 16:31:15,946][20381] Updated weights for policy 1, policy_version 1568065 (0.0005) [2023-12-27 16:31:16,500][20381] Updated weights for policy 1, policy_version 1568075 (0.0005) [2023-12-27 16:31:16,535][20347] Updated weights for policy 0, policy_version 1569745 (0.0008) [2023-12-27 16:31:16,553][20381] Updated weights for policy 1, policy_version 1568085 (0.0006) [2023-12-27 16:31:16,568][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 16:31:16,592][20347] Updated weights for policy 0, policy_version 1569755 (0.0010) [2023-12-27 16:31:16,655][20347] Updated weights for policy 0, policy_version 1569765 (0.0008) [2023-12-27 16:31:16,716][20347] Updated weights for policy 0, policy_version 1569775 (0.0006) [2023-12-27 16:31:17,207][20381] Updated weights for policy 1, policy_version 1568095 (0.0006) [2023-12-27 16:31:17,259][20347] Updated weights for policy 0, policy_version 1569785 (0.0007) [2023-12-27 16:31:17,260][20381] Updated weights for policy 1, policy_version 1568105 (0.0009) [2023-12-27 16:31:17,313][20347] Updated weights for policy 0, policy_version 1569795 (0.0006) [2023-12-27 16:31:17,322][20381] Updated weights for policy 1, policy_version 1568115 (0.0006) [2023-12-27 16:31:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 803422208. Throughput: 0: 10266.0, 1: 10172.5. Samples: 803432088. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:17,363][19170] Avg episode reward: [(0, '911.810'), (1, '844.650')] [2023-12-27 16:31:17,367][20347] Updated weights for policy 0, policy_version 1569805 (0.0006) [2023-12-27 16:31:17,952][20381] Updated weights for policy 1, policy_version 1568125 (0.0007) [2023-12-27 16:31:17,989][20347] Updated weights for policy 0, policy_version 1569815 (0.0006) [2023-12-27 16:31:18,004][20381] Updated weights for policy 1, policy_version 1568135 (0.0006) [2023-12-27 16:31:18,036][20347] Updated weights for policy 0, policy_version 1569825 (0.0006) [2023-12-27 16:31:18,063][20381] Updated weights for policy 1, policy_version 1568145 (0.0006) [2023-12-27 16:31:18,089][20347] Updated weights for policy 0, policy_version 1569835 (0.0006) [2023-12-27 16:31:18,723][20347] Updated weights for policy 0, policy_version 1569845 (0.0006) [2023-12-27 16:31:18,789][20347] Updated weights for policy 0, policy_version 1569855 (0.0006) [2023-12-27 16:31:18,805][20381] Updated weights for policy 1, policy_version 1568155 (0.0007) [2023-12-27 16:31:18,852][20347] Updated weights for policy 0, policy_version 1569865 (0.0006) [2023-12-27 16:31:18,863][20381] Updated weights for policy 1, policy_version 1568165 (0.0007) [2023-12-27 16:31:18,924][20381] Updated weights for policy 1, policy_version 1568175 (0.0007) [2023-12-27 16:31:19,453][20347] Updated weights for policy 0, policy_version 1569875 (0.0008) [2023-12-27 16:31:19,517][20347] Updated weights for policy 0, policy_version 1569885 (0.0008) [2023-12-27 16:31:19,582][20347] Updated weights for policy 0, policy_version 1569895 (0.0008) [2023-12-27 16:31:19,617][20381] Updated weights for policy 1, policy_version 1568185 (0.0008) [2023-12-27 16:31:19,668][20381] Updated weights for policy 1, policy_version 1568195 (0.0007) [2023-12-27 16:31:19,722][20381] Updated weights for policy 1, policy_version 1568205 (0.0008) [2023-12-27 16:31:19,782][20381] Updated weights for policy 1, policy_version 1568215 (0.0007) [2023-12-27 16:31:20,335][20347] Updated weights for policy 0, policy_version 1569905 (0.0007) [2023-12-27 16:31:20,395][20347] Updated weights for policy 0, policy_version 1569915 (0.0011) [2023-12-27 16:31:20,456][20347] Updated weights for policy 0, policy_version 1569925 (0.0007) [2023-12-27 16:31:20,513][20381] Updated weights for policy 1, policy_version 1568225 (0.0006) [2023-12-27 16:31:20,514][20347] Updated weights for policy 0, policy_version 1569935 (0.0008) [2023-12-27 16:31:20,571][20381] Updated weights for policy 1, policy_version 1568235 (0.0006) [2023-12-27 16:31:20,628][20381] Updated weights for policy 1, policy_version 1568245 (0.0008) [2023-12-27 16:31:21,251][20347] Updated weights for policy 0, policy_version 1569945 (0.0008) [2023-12-27 16:31:21,314][20347] Updated weights for policy 0, policy_version 1569955 (0.0007) [2023-12-27 16:31:21,316][20381] Updated weights for policy 1, policy_version 1568255 (0.0007) [2023-12-27 16:31:21,383][20347] Updated weights for policy 0, policy_version 1569965 (0.0007) [2023-12-27 16:31:21,392][20381] Updated weights for policy 1, policy_version 1568265 (0.0008) [2023-12-27 16:31:21,456][20381] Updated weights for policy 1, policy_version 1568275 (0.0008) [2023-12-27 16:31:22,086][20347] Updated weights for policy 0, policy_version 1569975 (0.0007) [2023-12-27 16:31:22,142][20347] Updated weights for policy 0, policy_version 1569985 (0.0007) [2023-12-27 16:31:22,194][20381] Updated weights for policy 1, policy_version 1568285 (0.0008) [2023-12-27 16:31:22,200][20347] Updated weights for policy 0, policy_version 1569995 (0.0005) [2023-12-27 16:31:22,254][20381] Updated weights for policy 1, policy_version 1568295 (0.0007) [2023-12-27 16:31:22,316][20381] Updated weights for policy 1, policy_version 1568305 (0.0007) [2023-12-27 16:31:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 803528704. Throughput: 0: 10296.0, 1: 10118.1. Samples: 803493980. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:22,363][19170] Avg episode reward: [(0, '911.950'), (1, '825.970')] [2023-12-27 16:31:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001568312_401530880.pth... [2023-12-27 16:31:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001570000_401997824.pth... [2023-12-27 16:31:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001567112_401219584.pth [2023-12-27 16:31:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001568784_401686528.pth [2023-12-27 16:31:22,879][20347] Updated weights for policy 0, policy_version 1570005 (0.0007) [2023-12-27 16:31:22,929][20347] Updated weights for policy 0, policy_version 1570015 (0.0008) [2023-12-27 16:31:22,990][20347] Updated weights for policy 0, policy_version 1570025 (0.0008) [2023-12-27 16:31:23,005][20381] Updated weights for policy 1, policy_version 1568315 (0.0007) [2023-12-27 16:31:23,056][20292] KL-divergence is very high: 100.7106 [2023-12-27 16:31:23,062][20381] Updated weights for policy 1, policy_version 1568325 (0.0007) [2023-12-27 16:31:23,117][20381] Updated weights for policy 1, policy_version 1568335 (0.0007) [2023-12-27 16:31:23,684][20347] Updated weights for policy 0, policy_version 1570035 (0.0007) [2023-12-27 16:31:23,746][20347] Updated weights for policy 0, policy_version 1570045 (0.0007) [2023-12-27 16:31:23,798][20347] Updated weights for policy 0, policy_version 1570055 (0.0008) [2023-12-27 16:31:23,800][20381] Updated weights for policy 1, policy_version 1568345 (0.0007) [2023-12-27 16:31:23,855][20381] Updated weights for policy 1, policy_version 1568355 (0.0008) [2023-12-27 16:31:23,914][20381] Updated weights for policy 1, policy_version 1568365 (0.0008) [2023-12-27 16:31:23,974][20381] Updated weights for policy 1, policy_version 1568375 (0.0009) [2023-12-27 16:31:24,414][20347] Updated weights for policy 0, policy_version 1570065 (0.0005) [2023-12-27 16:31:24,468][20347] Updated weights for policy 0, policy_version 1570075 (0.0008) [2023-12-27 16:31:24,523][20347] Updated weights for policy 0, policy_version 1570085 (0.0007) [2023-12-27 16:31:24,591][20347] Updated weights for policy 0, policy_version 1570095 (0.0008) [2023-12-27 16:31:24,678][20381] Updated weights for policy 1, policy_version 1568385 (0.0006) [2023-12-27 16:31:24,736][20381] Updated weights for policy 1, policy_version 1568395 (0.0009) [2023-12-27 16:31:24,798][20381] Updated weights for policy 1, policy_version 1568405 (0.0008) [2023-12-27 16:31:25,247][20347] Updated weights for policy 0, policy_version 1570105 (0.0007) [2023-12-27 16:31:25,297][20347] Updated weights for policy 0, policy_version 1570115 (0.0007) [2023-12-27 16:31:25,345][20347] Updated weights for policy 0, policy_version 1570125 (0.0005) [2023-12-27 16:31:25,516][20381] Updated weights for policy 1, policy_version 1568415 (0.0007) [2023-12-27 16:31:25,570][20381] Updated weights for policy 1, policy_version 1568425 (0.0009) [2023-12-27 16:31:25,619][20381] Updated weights for policy 1, policy_version 1568435 (0.0008) [2023-12-27 16:31:25,944][20347] Updated weights for policy 0, policy_version 1570135 (0.0007) [2023-12-27 16:31:25,995][20347] Updated weights for policy 0, policy_version 1570145 (0.0008) [2023-12-27 16:31:26,052][20347] Updated weights for policy 0, policy_version 1570155 (0.0008) [2023-12-27 16:31:26,344][20381] Updated weights for policy 1, policy_version 1568445 (0.0008) [2023-12-27 16:31:26,397][20381] Updated weights for policy 1, policy_version 1568455 (0.0009) [2023-12-27 16:31:26,442][20381] Updated weights for policy 1, policy_version 1568465 (0.0008) [2023-12-27 16:31:26,647][20347] Updated weights for policy 0, policy_version 1570165 (0.0006) [2023-12-27 16:31:26,701][20347] Updated weights for policy 0, policy_version 1570175 (0.0005) [2023-12-27 16:31:26,746][20347] Updated weights for policy 0, policy_version 1570185 (0.0005) [2023-12-27 16:31:27,107][20381] Updated weights for policy 1, policy_version 1568475 (0.0006) [2023-12-27 16:31:27,170][20381] Updated weights for policy 1, policy_version 1568485 (0.0008) [2023-12-27 16:31:27,226][20381] Updated weights for policy 1, policy_version 1568495 (0.0007) [2023-12-27 16:31:27,248][20347] Updated weights for policy 0, policy_version 1570195 (0.0005) [2023-12-27 16:31:27,305][20347] Updated weights for policy 0, policy_version 1570205 (0.0006) [2023-12-27 16:31:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 803627008. Throughput: 0: 10288.4, 1: 10142.1. Samples: 803614048. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:27,363][19170] Avg episode reward: [(0, '941.360'), (1, '806.310')] [2023-12-27 16:31:27,367][20347] Updated weights for policy 0, policy_version 1570215 (0.0007) [2023-12-27 16:31:27,932][20347] Updated weights for policy 0, policy_version 1570225 (0.0007) [2023-12-27 16:31:27,952][20381] Updated weights for policy 1, policy_version 1568505 (0.0008) [2023-12-27 16:31:27,978][20347] Updated weights for policy 0, policy_version 1570235 (0.0006) [2023-12-27 16:31:28,002][20381] Updated weights for policy 1, policy_version 1568515 (0.0007) [2023-12-27 16:31:28,038][20347] Updated weights for policy 0, policy_version 1570245 (0.0006) [2023-12-27 16:31:28,047][20381] Updated weights for policy 1, policy_version 1568525 (0.0007) [2023-12-27 16:31:28,090][20347] Updated weights for policy 0, policy_version 1570255 (0.0006) [2023-12-27 16:31:28,097][20381] Updated weights for policy 1, policy_version 1568535 (0.0007) [2023-12-27 16:31:28,671][20347] Updated weights for policy 0, policy_version 1570265 (0.0006) [2023-12-27 16:31:28,732][20347] Updated weights for policy 0, policy_version 1570275 (0.0006) [2023-12-27 16:31:28,792][20347] Updated weights for policy 0, policy_version 1570285 (0.0007) [2023-12-27 16:31:28,852][20381] Updated weights for policy 1, policy_version 1568545 (0.0010) [2023-12-27 16:31:28,909][20381] Updated weights for policy 1, policy_version 1568555 (0.0011) [2023-12-27 16:31:28,967][20381] Updated weights for policy 1, policy_version 1568565 (0.0010) [2023-12-27 16:31:29,350][20347] Updated weights for policy 0, policy_version 1570295 (0.0007) [2023-12-27 16:31:29,412][20347] Updated weights for policy 0, policy_version 1570305 (0.0008) [2023-12-27 16:31:29,469][20347] Updated weights for policy 0, policy_version 1570315 (0.0006) [2023-12-27 16:31:29,696][20381] Updated weights for policy 1, policy_version 1568575 (0.0008) [2023-12-27 16:31:29,752][20381] Updated weights for policy 1, policy_version 1568585 (0.0008) [2023-12-27 16:31:29,809][20381] Updated weights for policy 1, policy_version 1568595 (0.0007) [2023-12-27 16:31:30,164][20347] Updated weights for policy 0, policy_version 1570325 (0.0006) [2023-12-27 16:31:30,213][20347] Updated weights for policy 0, policy_version 1570335 (0.0007) [2023-12-27 16:31:30,228][20291] KL-divergence is very high: 314.7295 [2023-12-27 16:31:30,264][20347] Updated weights for policy 0, policy_version 1570345 (0.0007) [2023-12-27 16:31:30,269][20291] KL-divergence is very high: 381.5807 [2023-12-27 16:31:30,491][20381] Updated weights for policy 1, policy_version 1568605 (0.0007) [2023-12-27 16:31:30,545][20381] Updated weights for policy 1, policy_version 1568615 (0.0009) [2023-12-27 16:31:30,594][20381] Updated weights for policy 1, policy_version 1568625 (0.0009) [2023-12-27 16:31:30,875][20347] Updated weights for policy 0, policy_version 1570355 (0.0007) [2023-12-27 16:31:30,937][20347] Updated weights for policy 0, policy_version 1570365 (0.0006) [2023-12-27 16:31:31,000][20347] Updated weights for policy 0, policy_version 1570375 (0.0005) [2023-12-27 16:31:31,282][20381] Updated weights for policy 1, policy_version 1568635 (0.0008) [2023-12-27 16:31:31,341][20381] Updated weights for policy 1, policy_version 1568645 (0.0008) [2023-12-27 16:31:31,408][20381] Updated weights for policy 1, policy_version 1568655 (0.0008) [2023-12-27 16:31:31,582][20347] Updated weights for policy 0, policy_version 1570385 (0.0006) [2023-12-27 16:31:31,640][20347] Updated weights for policy 0, policy_version 1570395 (0.0009) [2023-12-27 16:31:31,702][20347] Updated weights for policy 0, policy_version 1570405 (0.0008) [2023-12-27 16:31:31,768][20347] Updated weights for policy 0, policy_version 1570415 (0.0009) [2023-12-27 16:31:32,111][20381] Updated weights for policy 1, policy_version 1568665 (0.0008) [2023-12-27 16:31:32,180][20381] Updated weights for policy 1, policy_version 1568675 (0.0008) [2023-12-27 16:31:32,241][20381] Updated weights for policy 1, policy_version 1568685 (0.0007) [2023-12-27 16:31:32,294][20381] Updated weights for policy 1, policy_version 1568695 (0.0006) [2023-12-27 16:31:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 803733504. Throughput: 0: 10492.3, 1: 10134.0. Samples: 803741608. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:32,363][19170] Avg episode reward: [(0, '951.070'), (1, '825.080')] [2023-12-27 16:31:32,515][20347] Updated weights for policy 0, policy_version 1570425 (0.0008) [2023-12-27 16:31:32,572][20347] Updated weights for policy 0, policy_version 1570435 (0.0008) [2023-12-27 16:31:32,629][20347] Updated weights for policy 0, policy_version 1570445 (0.0008) [2023-12-27 16:31:32,927][20381] Updated weights for policy 1, policy_version 1568705 (0.0008) [2023-12-27 16:31:32,968][20381] Updated weights for policy 1, policy_version 1568715 (0.0007) [2023-12-27 16:31:33,013][20381] Updated weights for policy 1, policy_version 1568725 (0.0007) [2023-12-27 16:31:33,374][20347] Updated weights for policy 0, policy_version 1570455 (0.0007) [2023-12-27 16:31:33,422][20347] Updated weights for policy 0, policy_version 1570465 (0.0008) [2023-12-27 16:31:33,477][20347] Updated weights for policy 0, policy_version 1570475 (0.0008) [2023-12-27 16:31:33,626][20381] Updated weights for policy 1, policy_version 1568735 (0.0006) [2023-12-27 16:31:33,687][20381] Updated weights for policy 1, policy_version 1568745 (0.0006) [2023-12-27 16:31:33,750][20381] Updated weights for policy 1, policy_version 1568755 (0.0005) [2023-12-27 16:31:34,173][20347] Updated weights for policy 0, policy_version 1570485 (0.0007) [2023-12-27 16:31:34,234][20347] Updated weights for policy 0, policy_version 1570495 (0.0008) [2023-12-27 16:31:34,293][20347] Updated weights for policy 0, policy_version 1570505 (0.0007) [2023-12-27 16:31:34,338][20381] Updated weights for policy 1, policy_version 1568765 (0.0006) [2023-12-27 16:31:34,395][20381] Updated weights for policy 1, policy_version 1568775 (0.0007) [2023-12-27 16:31:34,460][20381] Updated weights for policy 1, policy_version 1568785 (0.0008) [2023-12-27 16:31:34,952][20347] Updated weights for policy 0, policy_version 1570515 (0.0007) [2023-12-27 16:31:35,013][20347] Updated weights for policy 0, policy_version 1570525 (0.0006) [2023-12-27 16:31:35,044][20381] Updated weights for policy 1, policy_version 1568795 (0.0007) [2023-12-27 16:31:35,071][20347] Updated weights for policy 0, policy_version 1570535 (0.0008) [2023-12-27 16:31:35,098][20381] Updated weights for policy 1, policy_version 1568805 (0.0007) [2023-12-27 16:31:35,156][20381] Updated weights for policy 1, policy_version 1568815 (0.0007) [2023-12-27 16:31:35,669][20347] Updated weights for policy 0, policy_version 1570545 (0.0008) [2023-12-27 16:31:35,715][20347] Updated weights for policy 0, policy_version 1570555 (0.0005) [2023-12-27 16:31:35,760][20347] Updated weights for policy 0, policy_version 1570565 (0.0006) [2023-12-27 16:31:35,814][20347] Updated weights for policy 0, policy_version 1570575 (0.0007) [2023-12-27 16:31:35,873][20381] Updated weights for policy 1, policy_version 1568825 (0.0008) [2023-12-27 16:31:35,931][20381] Updated weights for policy 1, policy_version 1568835 (0.0011) [2023-12-27 16:31:35,982][20381] Updated weights for policy 1, policy_version 1568845 (0.0010) [2023-12-27 16:31:36,029][20381] Updated weights for policy 1, policy_version 1568855 (0.0006) [2023-12-27 16:31:36,522][20347] Updated weights for policy 0, policy_version 1570585 (0.0008) [2023-12-27 16:31:36,575][20347] Updated weights for policy 0, policy_version 1570595 (0.0008) [2023-12-27 16:31:36,635][20347] Updated weights for policy 0, policy_version 1570605 (0.0007) [2023-12-27 16:31:36,698][20381] Updated weights for policy 1, policy_version 1568865 (0.0007) [2023-12-27 16:31:36,752][20381] Updated weights for policy 1, policy_version 1568875 (0.0008) [2023-12-27 16:31:36,800][20381] Updated weights for policy 1, policy_version 1568885 (0.0010) [2023-12-27 16:31:37,265][20347] Updated weights for policy 0, policy_version 1570615 (0.0008) [2023-12-27 16:31:37,325][20347] Updated weights for policy 0, policy_version 1570625 (0.0007) [2023-12-27 16:31:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 803831808. Throughput: 0: 10472.3, 1: 10169.6. Samples: 803804704. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:37,363][19170] Avg episode reward: [(0, '970.650'), (1, '834.320')] [2023-12-27 16:31:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001568888_401678336.pth... [2023-12-27 16:31:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001567720_401375232.pth [2023-12-27 16:31:37,383][20347] Updated weights for policy 0, policy_version 1570635 (0.0007) [2023-12-27 16:31:37,412][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001570640_402161664.pth... [2023-12-27 16:31:37,416][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001569392_401842176.pth [2023-12-27 16:31:37,488][20381] Updated weights for policy 1, policy_version 1568895 (0.0010) [2023-12-27 16:31:37,554][20381] Updated weights for policy 1, policy_version 1568905 (0.0011) [2023-12-27 16:31:37,617][20381] Updated weights for policy 1, policy_version 1568915 (0.0011) [2023-12-27 16:31:38,100][20347] Updated weights for policy 0, policy_version 1570645 (0.0008) [2023-12-27 16:31:38,156][20347] Updated weights for policy 0, policy_version 1570655 (0.0008) [2023-12-27 16:31:38,213][20381] Updated weights for policy 1, policy_version 1568925 (0.0010) [2023-12-27 16:31:38,214][20347] Updated weights for policy 0, policy_version 1570665 (0.0007) [2023-12-27 16:31:38,264][20381] Updated weights for policy 1, policy_version 1568935 (0.0008) [2023-12-27 16:31:38,319][20381] Updated weights for policy 1, policy_version 1568945 (0.0005) [2023-12-27 16:31:38,857][20347] Updated weights for policy 0, policy_version 1570675 (0.0005) [2023-12-27 16:31:38,908][20347] Updated weights for policy 0, policy_version 1570685 (0.0005) [2023-12-27 16:31:38,964][20347] Updated weights for policy 0, policy_version 1570695 (0.0007) [2023-12-27 16:31:39,003][20381] Updated weights for policy 1, policy_version 1568955 (0.0008) [2023-12-27 16:31:39,058][20381] Updated weights for policy 1, policy_version 1568965 (0.0010) [2023-12-27 16:31:39,110][20381] Updated weights for policy 1, policy_version 1568975 (0.0011) [2023-12-27 16:31:39,662][20347] Updated weights for policy 0, policy_version 1570705 (0.0010) [2023-12-27 16:31:39,726][20347] Updated weights for policy 0, policy_version 1570715 (0.0008) [2023-12-27 16:31:39,783][20347] Updated weights for policy 0, policy_version 1570725 (0.0008) [2023-12-27 16:31:39,843][20347] Updated weights for policy 0, policy_version 1570735 (0.0008) [2023-12-27 16:31:39,889][20381] Updated weights for policy 1, policy_version 1568985 (0.0010) [2023-12-27 16:31:39,951][20381] Updated weights for policy 1, policy_version 1568995 (0.0007) [2023-12-27 16:31:40,007][20381] Updated weights for policy 1, policy_version 1569005 (0.0008) [2023-12-27 16:31:40,060][20381] Updated weights for policy 1, policy_version 1569015 (0.0007) [2023-12-27 16:31:40,577][20347] Updated weights for policy 0, policy_version 1570745 (0.0008) [2023-12-27 16:31:40,625][20347] Updated weights for policy 0, policy_version 1570755 (0.0007) [2023-12-27 16:31:40,669][20347] Updated weights for policy 0, policy_version 1570765 (0.0007) [2023-12-27 16:31:40,747][20381] Updated weights for policy 1, policy_version 1569025 (0.0008) [2023-12-27 16:31:40,811][20381] Updated weights for policy 1, policy_version 1569035 (0.0008) [2023-12-27 16:31:40,872][20381] Updated weights for policy 1, policy_version 1569045 (0.0011) [2023-12-27 16:31:41,339][20347] Updated weights for policy 0, policy_version 1570775 (0.0008) [2023-12-27 16:31:41,417][20347] Updated weights for policy 0, policy_version 1570785 (0.0008) [2023-12-27 16:31:41,470][20347] Updated weights for policy 0, policy_version 1570795 (0.0007) [2023-12-27 16:31:41,586][20381] Updated weights for policy 1, policy_version 1569055 (0.0007) [2023-12-27 16:31:41,589][20292] KL-divergence is very high: 150.3525 [2023-12-27 16:31:41,596][20292] KL-divergence is very high: 170.2394 [2023-12-27 16:31:41,646][20292] KL-divergence is very high: 220.6927 [2023-12-27 16:31:41,652][20292] KL-divergence is very high: 218.5449 [2023-12-27 16:31:41,658][20381] Updated weights for policy 1, policy_version 1569065 (0.0007) [2023-12-27 16:31:41,693][20292] KL-divergence is very high: 146.0884 [2023-12-27 16:31:41,700][20292] KL-divergence is very high: 138.5859 [2023-12-27 16:31:41,720][20381] Updated weights for policy 1, policy_version 1569075 (0.0008) [2023-12-27 16:31:42,205][20347] Updated weights for policy 0, policy_version 1570805 (0.0007) [2023-12-27 16:31:42,267][20347] Updated weights for policy 0, policy_version 1570815 (0.0008) [2023-12-27 16:31:42,326][20347] Updated weights for policy 0, policy_version 1570825 (0.0008) [2023-12-27 16:31:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 803930112. Throughput: 0: 10461.5, 1: 10181.7. Samples: 803927060. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:42,363][19170] Avg episode reward: [(0, '941.410'), (1, '873.400')] [2023-12-27 16:31:42,386][20381] Updated weights for policy 1, policy_version 1569085 (0.0010) [2023-12-27 16:31:42,441][20381] Updated weights for policy 1, policy_version 1569095 (0.0008) [2023-12-27 16:31:42,493][20381] Updated weights for policy 1, policy_version 1569105 (0.0008) [2023-12-27 16:31:43,052][20347] Updated weights for policy 0, policy_version 1570835 (0.0008) [2023-12-27 16:31:43,100][20347] Updated weights for policy 0, policy_version 1570845 (0.0008) [2023-12-27 16:31:43,145][20381] Updated weights for policy 1, policy_version 1569115 (0.0008) [2023-12-27 16:31:43,151][20347] Updated weights for policy 0, policy_version 1570855 (0.0006) [2023-12-27 16:31:43,195][20381] Updated weights for policy 1, policy_version 1569125 (0.0006) [2023-12-27 16:31:43,244][20381] Updated weights for policy 1, policy_version 1569135 (0.0008) [2023-12-27 16:31:43,878][20347] Updated weights for policy 0, policy_version 1570865 (0.0007) [2023-12-27 16:31:43,900][20381] Updated weights for policy 1, policy_version 1569145 (0.0008) [2023-12-27 16:31:43,938][20347] Updated weights for policy 0, policy_version 1570875 (0.0006) [2023-12-27 16:31:43,955][20381] Updated weights for policy 1, policy_version 1569155 (0.0007) [2023-12-27 16:31:44,008][20347] Updated weights for policy 0, policy_version 1570885 (0.0008) [2023-12-27 16:31:44,011][20381] Updated weights for policy 1, policy_version 1569165 (0.0006) [2023-12-27 16:31:44,065][20347] Updated weights for policy 0, policy_version 1570895 (0.0007) [2023-12-27 16:31:44,074][20381] Updated weights for policy 1, policy_version 1569175 (0.0007) [2023-12-27 16:31:44,727][20347] Updated weights for policy 0, policy_version 1570905 (0.0009) [2023-12-27 16:31:44,753][20381] Updated weights for policy 1, policy_version 1569185 (0.0006) [2023-12-27 16:31:44,792][20347] Updated weights for policy 0, policy_version 1570915 (0.0007) [2023-12-27 16:31:44,813][20381] Updated weights for policy 1, policy_version 1569195 (0.0008) [2023-12-27 16:31:44,857][20347] Updated weights for policy 0, policy_version 1570925 (0.0007) [2023-12-27 16:31:44,876][20381] Updated weights for policy 1, policy_version 1569205 (0.0008) [2023-12-27 16:31:45,501][20347] Updated weights for policy 0, policy_version 1570935 (0.0007) [2023-12-27 16:31:45,562][20347] Updated weights for policy 0, policy_version 1570945 (0.0008) [2023-12-27 16:31:45,595][20381] Updated weights for policy 1, policy_version 1569215 (0.0008) [2023-12-27 16:31:45,618][20347] Updated weights for policy 0, policy_version 1570955 (0.0006) [2023-12-27 16:31:45,645][20381] Updated weights for policy 1, policy_version 1569225 (0.0009) [2023-12-27 16:31:45,696][20381] Updated weights for policy 1, policy_version 1569235 (0.0006) [2023-12-27 16:31:46,295][20347] Updated weights for policy 0, policy_version 1570965 (0.0006) [2023-12-27 16:31:46,305][20381] Updated weights for policy 1, policy_version 1569245 (0.0006) [2023-12-27 16:31:46,345][20347] Updated weights for policy 0, policy_version 1570975 (0.0007) [2023-12-27 16:31:46,364][20381] Updated weights for policy 1, policy_version 1569255 (0.0008) [2023-12-27 16:31:46,397][20347] Updated weights for policy 0, policy_version 1570985 (0.0007) [2023-12-27 16:31:46,429][20381] Updated weights for policy 1, policy_version 1569265 (0.0008) [2023-12-27 16:31:47,030][20381] Updated weights for policy 1, policy_version 1569275 (0.0006) [2023-12-27 16:31:47,046][20347] Updated weights for policy 0, policy_version 1570995 (0.0006) [2023-12-27 16:31:47,081][20381] Updated weights for policy 1, policy_version 1569285 (0.0006) [2023-12-27 16:31:47,095][20347] Updated weights for policy 0, policy_version 1571005 (0.0005) [2023-12-27 16:31:47,131][20381] Updated weights for policy 1, policy_version 1569295 (0.0005) [2023-12-27 16:31:47,145][20347] Updated weights for policy 0, policy_version 1571015 (0.0006) [2023-12-27 16:31:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 804044800. Throughput: 0: 10443.0, 1: 10225.8. Samples: 804050320. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:47,363][19170] Avg episode reward: [(0, '941.460'), (1, '892.800')] [2023-12-27 16:31:47,734][20347] Updated weights for policy 0, policy_version 1571025 (0.0008) [2023-12-27 16:31:47,786][20381] Updated weights for policy 1, policy_version 1569305 (0.0006) [2023-12-27 16:31:47,800][20347] Updated weights for policy 0, policy_version 1571035 (0.0007) [2023-12-27 16:31:47,837][20381] Updated weights for policy 1, policy_version 1569315 (0.0006) [2023-12-27 16:31:47,861][20347] Updated weights for policy 0, policy_version 1571045 (0.0010) [2023-12-27 16:31:47,890][20381] Updated weights for policy 1, policy_version 1569325 (0.0006) [2023-12-27 16:31:47,919][20347] Updated weights for policy 0, policy_version 1571055 (0.0010) [2023-12-27 16:31:47,941][20381] Updated weights for policy 1, policy_version 1569335 (0.0006) [2023-12-27 16:31:48,552][20347] Updated weights for policy 0, policy_version 1571065 (0.0008) [2023-12-27 16:31:48,611][20347] Updated weights for policy 0, policy_version 1571075 (0.0008) [2023-12-27 16:31:48,628][20381] Updated weights for policy 1, policy_version 1569345 (0.0006) [2023-12-27 16:31:48,666][20347] Updated weights for policy 0, policy_version 1571085 (0.0006) [2023-12-27 16:31:48,686][20381] Updated weights for policy 1, policy_version 1569355 (0.0008) [2023-12-27 16:31:48,732][20381] Updated weights for policy 1, policy_version 1569365 (0.0008) [2023-12-27 16:31:49,351][20347] Updated weights for policy 0, policy_version 1571095 (0.0007) [2023-12-27 16:31:49,413][20347] Updated weights for policy 0, policy_version 1571105 (0.0008) [2023-12-27 16:31:49,460][20381] Updated weights for policy 1, policy_version 1569375 (0.0010) [2023-12-27 16:31:49,466][20347] Updated weights for policy 0, policy_version 1571115 (0.0006) [2023-12-27 16:31:49,520][20381] Updated weights for policy 1, policy_version 1569385 (0.0008) [2023-12-27 16:31:49,581][20381] Updated weights for policy 1, policy_version 1569395 (0.0008) [2023-12-27 16:31:50,182][20347] Updated weights for policy 0, policy_version 1571125 (0.0005) [2023-12-27 16:31:50,246][20347] Updated weights for policy 0, policy_version 1571135 (0.0006) [2023-12-27 16:31:50,288][20381] Updated weights for policy 1, policy_version 1569405 (0.0008) [2023-12-27 16:31:50,312][20347] Updated weights for policy 0, policy_version 1571145 (0.0007) [2023-12-27 16:31:50,341][20381] Updated weights for policy 1, policy_version 1569415 (0.0005) [2023-12-27 16:31:50,399][20381] Updated weights for policy 1, policy_version 1569425 (0.0008) [2023-12-27 16:31:51,008][20347] Updated weights for policy 0, policy_version 1571155 (0.0007) [2023-12-27 16:31:51,053][20381] Updated weights for policy 1, policy_version 1569435 (0.0007) [2023-12-27 16:31:51,073][20347] Updated weights for policy 0, policy_version 1571165 (0.0008) [2023-12-27 16:31:51,110][20381] Updated weights for policy 1, policy_version 1569445 (0.0007) [2023-12-27 16:31:51,137][20347] Updated weights for policy 0, policy_version 1571175 (0.0007) [2023-12-27 16:31:51,167][20381] Updated weights for policy 1, policy_version 1569455 (0.0008) [2023-12-27 16:31:51,841][20381] Updated weights for policy 1, policy_version 1569465 (0.0006) [2023-12-27 16:31:51,886][20347] Updated weights for policy 0, policy_version 1571185 (0.0006) [2023-12-27 16:31:51,902][20381] Updated weights for policy 1, policy_version 1569475 (0.0009) [2023-12-27 16:31:51,920][20292] KL-divergence is very high: 143.3608 [2023-12-27 16:31:51,933][20292] KL-divergence is very high: 128.4202 [2023-12-27 16:31:51,941][20347] Updated weights for policy 0, policy_version 1571195 (0.0007) [2023-12-27 16:31:51,961][20381] Updated weights for policy 1, policy_version 1569485 (0.0008) [2023-12-27 16:31:51,969][20292] KL-divergence is very high: 110.8776 [2023-12-27 16:31:51,997][20347] Updated weights for policy 0, policy_version 1571205 (0.0006) [2023-12-27 16:31:52,022][20381] Updated weights for policy 1, policy_version 1569495 (0.0008) [2023-12-27 16:31:52,057][20347] Updated weights for policy 0, policy_version 1571215 (0.0007) [2023-12-27 16:31:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 804143104. Throughput: 0: 10406.9, 1: 10196.6. Samples: 804112148. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:52,363][19170] Avg episode reward: [(0, '931.550'), (1, '892.630')] [2023-12-27 16:31:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001569496_401833984.pth... [2023-12-27 16:31:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001571216_402309120.pth... [2023-12-27 16:31:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001568312_401530880.pth [2023-12-27 16:31:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001570000_401997824.pth [2023-12-27 16:31:52,657][20381] Updated weights for policy 1, policy_version 1569505 (0.0006) [2023-12-27 16:31:52,716][20381] Updated weights for policy 1, policy_version 1569515 (0.0007) [2023-12-27 16:31:52,768][20347] Updated weights for policy 0, policy_version 1571225 (0.0006) [2023-12-27 16:31:52,774][20381] Updated weights for policy 1, policy_version 1569525 (0.0006) [2023-12-27 16:31:52,824][20347] Updated weights for policy 0, policy_version 1571235 (0.0007) [2023-12-27 16:31:52,881][20347] Updated weights for policy 0, policy_version 1571245 (0.0008) [2023-12-27 16:31:53,353][20381] Updated weights for policy 1, policy_version 1569535 (0.0009) [2023-12-27 16:31:53,397][20381] Updated weights for policy 1, policy_version 1569545 (0.0010) [2023-12-27 16:31:53,448][20381] Updated weights for policy 1, policy_version 1569555 (0.0010) [2023-12-27 16:31:53,585][20347] Updated weights for policy 0, policy_version 1571255 (0.0008) [2023-12-27 16:31:53,635][20347] Updated weights for policy 0, policy_version 1571265 (0.0008) [2023-12-27 16:31:53,683][20347] Updated weights for policy 0, policy_version 1571275 (0.0007) [2023-12-27 16:31:54,093][20381] Updated weights for policy 1, policy_version 1569565 (0.0008) [2023-12-27 16:31:54,138][20381] Updated weights for policy 1, policy_version 1569575 (0.0006) [2023-12-27 16:31:54,186][20381] Updated weights for policy 1, policy_version 1569585 (0.0007) [2023-12-27 16:31:54,421][20347] Updated weights for policy 0, policy_version 1571285 (0.0007) [2023-12-27 16:31:54,478][20347] Updated weights for policy 0, policy_version 1571295 (0.0008) [2023-12-27 16:31:54,534][20347] Updated weights for policy 0, policy_version 1571305 (0.0008) [2023-12-27 16:31:54,830][20381] Updated weights for policy 1, policy_version 1569595 (0.0008) [2023-12-27 16:31:54,887][20381] Updated weights for policy 1, policy_version 1569605 (0.0006) [2023-12-27 16:31:54,953][20381] Updated weights for policy 1, policy_version 1569615 (0.0008) [2023-12-27 16:31:55,216][20347] Updated weights for policy 0, policy_version 1571315 (0.0007) [2023-12-27 16:31:55,276][20347] Updated weights for policy 0, policy_version 1571325 (0.0006) [2023-12-27 16:31:55,337][20347] Updated weights for policy 0, policy_version 1571335 (0.0005) [2023-12-27 16:31:55,660][20381] Updated weights for policy 1, policy_version 1569625 (0.0008) [2023-12-27 16:31:55,718][20381] Updated weights for policy 1, policy_version 1569635 (0.0010) [2023-12-27 16:31:55,768][20381] Updated weights for policy 1, policy_version 1569645 (0.0010) [2023-12-27 16:31:55,820][20381] Updated weights for policy 1, policy_version 1569655 (0.0010) [2023-12-27 16:31:55,975][20347] Updated weights for policy 0, policy_version 1571345 (0.0006) [2023-12-27 16:31:56,026][20347] Updated weights for policy 0, policy_version 1571355 (0.0007) [2023-12-27 16:31:56,081][20347] Updated weights for policy 0, policy_version 1571365 (0.0007) [2023-12-27 16:31:56,136][20347] Updated weights for policy 0, policy_version 1571375 (0.0007) [2023-12-27 16:31:56,503][20381] Updated weights for policy 1, policy_version 1569665 (0.0010) [2023-12-27 16:31:56,554][20381] Updated weights for policy 1, policy_version 1569675 (0.0010) [2023-12-27 16:31:56,608][20381] Updated weights for policy 1, policy_version 1569685 (0.0010) [2023-12-27 16:31:56,826][20347] Updated weights for policy 0, policy_version 1571385 (0.0005) [2023-12-27 16:31:56,876][20347] Updated weights for policy 0, policy_version 1571395 (0.0005) [2023-12-27 16:31:56,930][20347] Updated weights for policy 0, policy_version 1571405 (0.0006) [2023-12-27 16:31:57,260][20381] Updated weights for policy 1, policy_version 1569695 (0.0009) [2023-12-27 16:31:57,315][20381] Updated weights for policy 1, policy_version 1569705 (0.0007) [2023-12-27 16:31:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 804241408. Throughput: 0: 10349.9, 1: 10278.3. Samples: 804234772. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:31:57,363][20381] Updated weights for policy 1, policy_version 1569715 (0.0007) [2023-12-27 16:31:57,363][19170] Avg episode reward: [(0, '950.980'), (1, '853.600')] [2023-12-27 16:31:57,558][20347] Updated weights for policy 0, policy_version 1571415 (0.0008) [2023-12-27 16:31:57,603][20347] Updated weights for policy 0, policy_version 1571425 (0.0008) [2023-12-27 16:31:57,649][20347] Updated weights for policy 0, policy_version 1571435 (0.0009) [2023-12-27 16:31:57,925][20381] Updated weights for policy 1, policy_version 1569725 (0.0006) [2023-12-27 16:31:57,981][20381] Updated weights for policy 1, policy_version 1569735 (0.0010) [2023-12-27 16:31:58,033][20381] Updated weights for policy 1, policy_version 1569745 (0.0010) [2023-12-27 16:31:58,400][20347] Updated weights for policy 0, policy_version 1571445 (0.0008) [2023-12-27 16:31:58,457][20347] Updated weights for policy 0, policy_version 1571455 (0.0007) [2023-12-27 16:31:58,519][20347] Updated weights for policy 0, policy_version 1571465 (0.0008) [2023-12-27 16:31:58,737][20381] Updated weights for policy 1, policy_version 1569755 (0.0009) [2023-12-27 16:31:58,810][20381] Updated weights for policy 1, policy_version 1569765 (0.0008) [2023-12-27 16:31:58,881][20381] Updated weights for policy 1, policy_version 1569775 (0.0007) [2023-12-27 16:31:59,280][20347] Updated weights for policy 0, policy_version 1571475 (0.0008) [2023-12-27 16:31:59,344][20347] Updated weights for policy 0, policy_version 1571485 (0.0008) [2023-12-27 16:31:59,408][20347] Updated weights for policy 0, policy_version 1571495 (0.0011) [2023-12-27 16:31:59,663][20381] Updated weights for policy 1, policy_version 1569785 (0.0008) [2023-12-27 16:31:59,712][20381] Updated weights for policy 1, policy_version 1569795 (0.0008) [2023-12-27 16:31:59,757][20381] Updated weights for policy 1, policy_version 1569805 (0.0008) [2023-12-27 16:31:59,804][20381] Updated weights for policy 1, policy_version 1569815 (0.0007) [2023-12-27 16:32:00,090][20347] Updated weights for policy 0, policy_version 1571505 (0.0006) [2023-12-27 16:32:00,152][20347] Updated weights for policy 0, policy_version 1571515 (0.0008) [2023-12-27 16:32:00,211][20347] Updated weights for policy 0, policy_version 1571525 (0.0008) [2023-12-27 16:32:00,271][20347] Updated weights for policy 0, policy_version 1571535 (0.0008) [2023-12-27 16:32:00,562][20381] Updated weights for policy 1, policy_version 1569825 (0.0008) [2023-12-27 16:32:00,614][20381] Updated weights for policy 1, policy_version 1569835 (0.0008) [2023-12-27 16:32:00,660][20381] Updated weights for policy 1, policy_version 1569845 (0.0006) [2023-12-27 16:32:00,901][20347] Updated weights for policy 0, policy_version 1571545 (0.0007) [2023-12-27 16:32:00,954][20347] Updated weights for policy 0, policy_version 1571555 (0.0010) [2023-12-27 16:32:01,005][20347] Updated weights for policy 0, policy_version 1571565 (0.0010) [2023-12-27 16:32:01,304][20381] Updated weights for policy 1, policy_version 1569855 (0.0008) [2023-12-27 16:32:01,362][20381] Updated weights for policy 1, policy_version 1569865 (0.0007) [2023-12-27 16:32:01,421][20381] Updated weights for policy 1, policy_version 1569875 (0.0008) [2023-12-27 16:32:01,722][20347] Updated weights for policy 0, policy_version 1571575 (0.0010) [2023-12-27 16:32:01,776][20347] Updated weights for policy 0, policy_version 1571585 (0.0009) [2023-12-27 16:32:01,834][20347] Updated weights for policy 0, policy_version 1571595 (0.0010) [2023-12-27 16:32:02,138][20381] Updated weights for policy 1, policy_version 1569885 (0.0007) [2023-12-27 16:32:02,200][20381] Updated weights for policy 1, policy_version 1569895 (0.0007) [2023-12-27 16:32:02,250][20381] Updated weights for policy 1, policy_version 1569905 (0.0007) [2023-12-27 16:32:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 804347904. Throughput: 0: 10315.9, 1: 10198.6. Samples: 804355240. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:32:02,363][19170] Avg episode reward: [(0, '980.400'), (1, '844.350')] [2023-12-27 16:32:02,495][20347] Updated weights for policy 0, policy_version 1571605 (0.0009) [2023-12-27 16:32:02,547][20347] Updated weights for policy 0, policy_version 1571615 (0.0008) [2023-12-27 16:32:02,604][20347] Updated weights for policy 0, policy_version 1571625 (0.0008) [2023-12-27 16:32:02,960][20381] Updated weights for policy 1, policy_version 1569915 (0.0008) [2023-12-27 16:32:03,019][20381] Updated weights for policy 1, policy_version 1569925 (0.0007) [2023-12-27 16:32:03,075][20381] Updated weights for policy 1, policy_version 1569935 (0.0008) [2023-12-27 16:32:03,258][20347] Updated weights for policy 0, policy_version 1571635 (0.0007) [2023-12-27 16:32:03,304][20347] Updated weights for policy 0, policy_version 1571645 (0.0008) [2023-12-27 16:32:03,351][20347] Updated weights for policy 0, policy_version 1571655 (0.0007) [2023-12-27 16:32:03,714][20381] Updated weights for policy 1, policy_version 1569945 (0.0008) [2023-12-27 16:32:03,764][20381] Updated weights for policy 1, policy_version 1569955 (0.0008) [2023-12-27 16:32:03,796][20292] KL-divergence is very high: 320.7000 [2023-12-27 16:32:03,819][20292] KL-divergence is very high: 643.1674 [2023-12-27 16:32:03,826][20381] Updated weights for policy 1, policy_version 1569965 (0.0007) [2023-12-27 16:32:03,844][20292] KL-divergence is very high: 715.6483 [2023-12-27 16:32:03,867][20292] KL-divergence is very high: 559.3801 [2023-12-27 16:32:03,886][20381] Updated weights for policy 1, policy_version 1569975 (0.0008) [2023-12-27 16:32:04,036][20347] Updated weights for policy 0, policy_version 1571665 (0.0007) [2023-12-27 16:32:04,096][20347] Updated weights for policy 0, policy_version 1571675 (0.0008) [2023-12-27 16:32:04,159][20347] Updated weights for policy 0, policy_version 1571685 (0.0007) [2023-12-27 16:32:04,217][20347] Updated weights for policy 0, policy_version 1571695 (0.0008) [2023-12-27 16:32:04,573][20381] Updated weights for policy 1, policy_version 1569985 (0.0007) [2023-12-27 16:32:04,631][20381] Updated weights for policy 1, policy_version 1569995 (0.0008) [2023-12-27 16:32:04,689][20381] Updated weights for policy 1, policy_version 1570005 (0.0008) [2023-12-27 16:32:04,897][20347] Updated weights for policy 0, policy_version 1571705 (0.0008) [2023-12-27 16:32:04,942][20347] Updated weights for policy 0, policy_version 1571715 (0.0007) [2023-12-27 16:32:04,990][20347] Updated weights for policy 0, policy_version 1571725 (0.0007) [2023-12-27 16:32:05,342][20381] Updated weights for policy 1, policy_version 1570015 (0.0007) [2023-12-27 16:32:05,403][20381] Updated weights for policy 1, policy_version 1570025 (0.0008) [2023-12-27 16:32:05,462][20381] Updated weights for policy 1, policy_version 1570035 (0.0008) [2023-12-27 16:32:05,677][20347] Updated weights for policy 0, policy_version 1571735 (0.0006) [2023-12-27 16:32:05,731][20347] Updated weights for policy 0, policy_version 1571745 (0.0005) [2023-12-27 16:32:05,792][20347] Updated weights for policy 0, policy_version 1571755 (0.0005) [2023-12-27 16:32:06,190][20381] Updated weights for policy 1, policy_version 1570045 (0.0008) [2023-12-27 16:32:06,255][20381] Updated weights for policy 1, policy_version 1570055 (0.0008) [2023-12-27 16:32:06,313][20381] Updated weights for policy 1, policy_version 1570065 (0.0008) [2023-12-27 16:32:06,419][20347] Updated weights for policy 0, policy_version 1571765 (0.0006) [2023-12-27 16:32:06,478][20347] Updated weights for policy 0, policy_version 1571775 (0.0007) [2023-12-27 16:32:06,538][20347] Updated weights for policy 0, policy_version 1571785 (0.0007) [2023-12-27 16:32:07,035][20381] Updated weights for policy 1, policy_version 1570075 (0.0008) [2023-12-27 16:32:07,088][20381] Updated weights for policy 1, policy_version 1570085 (0.0007) [2023-12-27 16:32:07,144][20381] Updated weights for policy 1, policy_version 1570095 (0.0008) [2023-12-27 16:32:07,161][20347] Updated weights for policy 0, policy_version 1571795 (0.0005) [2023-12-27 16:32:07,211][20347] Updated weights for policy 0, policy_version 1571805 (0.0005) [2023-12-27 16:32:07,269][20347] Updated weights for policy 0, policy_version 1571815 (0.0005) [2023-12-27 16:32:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 804454400. Throughput: 0: 10290.3, 1: 10218.8. Samples: 804416896. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:32:07,364][19170] Avg episode reward: [(0, '970.640'), (1, '795.480')] [2023-12-27 16:32:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001570104_401989632.pth... [2023-12-27 16:32:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001571824_402464768.pth... [2023-12-27 16:32:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001568888_401678336.pth [2023-12-27 16:32:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001570640_402161664.pth [2023-12-27 16:32:07,815][20381] Updated weights for policy 1, policy_version 1570105 (0.0007) [2023-12-27 16:32:07,822][20347] Updated weights for policy 0, policy_version 1571825 (0.0005) [2023-12-27 16:32:07,869][20381] Updated weights for policy 1, policy_version 1570115 (0.0007) [2023-12-27 16:32:07,877][20347] Updated weights for policy 0, policy_version 1571835 (0.0006) [2023-12-27 16:32:07,919][20381] Updated weights for policy 1, policy_version 1570125 (0.0006) [2023-12-27 16:32:07,929][20347] Updated weights for policy 0, policy_version 1571845 (0.0010) [2023-12-27 16:32:07,978][20381] Updated weights for policy 1, policy_version 1570135 (0.0005) [2023-12-27 16:32:07,984][20347] Updated weights for policy 0, policy_version 1571855 (0.0010) [2023-12-27 16:32:08,598][20347] Updated weights for policy 0, policy_version 1571865 (0.0010) [2023-12-27 16:32:08,661][20347] Updated weights for policy 0, policy_version 1571875 (0.0007) [2023-12-27 16:32:08,678][20381] Updated weights for policy 1, policy_version 1570145 (0.0006) [2023-12-27 16:32:08,720][20347] Updated weights for policy 0, policy_version 1571885 (0.0008) [2023-12-27 16:32:08,736][20381] Updated weights for policy 1, policy_version 1570155 (0.0008) [2023-12-27 16:32:08,795][20381] Updated weights for policy 1, policy_version 1570165 (0.0008) [2023-12-27 16:32:09,392][20347] Updated weights for policy 0, policy_version 1571895 (0.0008) [2023-12-27 16:32:09,464][20347] Updated weights for policy 0, policy_version 1571905 (0.0013) [2023-12-27 16:32:09,487][20381] Updated weights for policy 1, policy_version 1570175 (0.0006) [2023-12-27 16:32:09,531][20347] Updated weights for policy 0, policy_version 1571915 (0.0011) [2023-12-27 16:32:09,546][20381] Updated weights for policy 1, policy_version 1570185 (0.0006) [2023-12-27 16:32:09,606][20381] Updated weights for policy 1, policy_version 1570195 (0.0006) [2023-12-27 16:32:10,234][20347] Updated weights for policy 0, policy_version 1571925 (0.0011) [2023-12-27 16:32:10,254][20381] Updated weights for policy 1, policy_version 1570205 (0.0006) [2023-12-27 16:32:10,283][20347] Updated weights for policy 0, policy_version 1571935 (0.0011) [2023-12-27 16:32:10,311][20381] Updated weights for policy 1, policy_version 1570215 (0.0006) [2023-12-27 16:32:10,343][20347] Updated weights for policy 0, policy_version 1571945 (0.0011) [2023-12-27 16:32:10,370][20381] Updated weights for policy 1, policy_version 1570225 (0.0006) [2023-12-27 16:32:11,031][20347] Updated weights for policy 0, policy_version 1571955 (0.0010) [2023-12-27 16:32:11,051][20381] Updated weights for policy 1, policy_version 1570235 (0.0008) [2023-12-27 16:32:11,100][20347] Updated weights for policy 0, policy_version 1571965 (0.0010) [2023-12-27 16:32:11,115][20381] Updated weights for policy 1, policy_version 1570245 (0.0008) [2023-12-27 16:32:11,172][20347] Updated weights for policy 0, policy_version 1571976 (0.0009) [2023-12-27 16:32:11,179][20381] Updated weights for policy 1, policy_version 1570255 (0.0007) [2023-12-27 16:32:11,910][20347] Updated weights for policy 0, policy_version 1571986 (0.0009) [2023-12-27 16:32:11,921][20381] Updated weights for policy 1, policy_version 1570265 (0.0008) [2023-12-27 16:32:11,970][20347] Updated weights for policy 0, policy_version 1571996 (0.0011) [2023-12-27 16:32:11,978][20381] Updated weights for policy 1, policy_version 1570275 (0.0010) [2023-12-27 16:32:12,027][20347] Updated weights for policy 0, policy_version 1572006 (0.0011) [2023-12-27 16:32:12,037][20381] Updated weights for policy 1, policy_version 1570285 (0.0007) [2023-12-27 16:32:12,085][20347] Updated weights for policy 0, policy_version 1572016 (0.0011) [2023-12-27 16:32:12,094][20381] Updated weights for policy 1, policy_version 1570295 (0.0006) [2023-12-27 16:32:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 804552704. Throughput: 0: 10351.1, 1: 10255.2. Samples: 804541332. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:32:12,363][19170] Avg episode reward: [(0, '941.370'), (1, '863.220')] [2023-12-27 16:32:12,792][20381] Updated weights for policy 1, policy_version 1570305 (0.0006) [2023-12-27 16:32:12,797][20347] Updated weights for policy 0, policy_version 1572026 (0.0007) [2023-12-27 16:32:12,848][20381] Updated weights for policy 1, policy_version 1570315 (0.0006) [2023-12-27 16:32:12,853][20347] Updated weights for policy 0, policy_version 1572036 (0.0008) [2023-12-27 16:32:12,901][20381] Updated weights for policy 1, policy_version 1570325 (0.0005) [2023-12-27 16:32:12,911][20347] Updated weights for policy 0, policy_version 1572046 (0.0007) [2023-12-27 16:32:13,506][20381] Updated weights for policy 1, policy_version 1570335 (0.0005) [2023-12-27 16:32:13,560][20381] Updated weights for policy 1, policy_version 1570345 (0.0005) [2023-12-27 16:32:13,581][20347] Updated weights for policy 0, policy_version 1572056 (0.0007) [2023-12-27 16:32:13,617][20381] Updated weights for policy 1, policy_version 1570355 (0.0005) [2023-12-27 16:32:13,634][20347] Updated weights for policy 0, policy_version 1572066 (0.0008) [2023-12-27 16:32:13,691][20347] Updated weights for policy 0, policy_version 1572076 (0.0008) [2023-12-27 16:32:14,244][20381] Updated weights for policy 1, policy_version 1570365 (0.0007) [2023-12-27 16:32:14,299][20381] Updated weights for policy 1, policy_version 1570375 (0.0006) [2023-12-27 16:32:14,351][20381] Updated weights for policy 1, policy_version 1570385 (0.0005) [2023-12-27 16:32:14,397][20347] Updated weights for policy 0, policy_version 1572086 (0.0006) [2023-12-27 16:32:14,450][20347] Updated weights for policy 0, policy_version 1572096 (0.0007) [2023-12-27 16:32:14,504][20347] Updated weights for policy 0, policy_version 1572106 (0.0007) [2023-12-27 16:32:15,061][20381] Updated weights for policy 1, policy_version 1570395 (0.0007) [2023-12-27 16:32:15,131][20381] Updated weights for policy 1, policy_version 1570405 (0.0008) [2023-12-27 16:32:15,143][20347] Updated weights for policy 0, policy_version 1572116 (0.0007) [2023-12-27 16:32:15,189][20381] Updated weights for policy 1, policy_version 1570415 (0.0006) [2023-12-27 16:32:15,202][20347] Updated weights for policy 0, policy_version 1572126 (0.0007) [2023-12-27 16:32:15,264][20347] Updated weights for policy 0, policy_version 1572136 (0.0006) [2023-12-27 16:32:15,904][20347] Updated weights for policy 0, policy_version 1572146 (0.0007) [2023-12-27 16:32:15,925][20381] Updated weights for policy 1, policy_version 1570425 (0.0007) [2023-12-27 16:32:15,954][20347] Updated weights for policy 0, policy_version 1572156 (0.0006) [2023-12-27 16:32:15,981][20381] Updated weights for policy 1, policy_version 1570435 (0.0011) [2023-12-27 16:32:16,011][20347] Updated weights for policy 0, policy_version 1572166 (0.0007) [2023-12-27 16:32:16,038][20381] Updated weights for policy 1, policy_version 1570445 (0.0011) [2023-12-27 16:32:16,073][20347] Updated weights for policy 0, policy_version 1572176 (0.0008) [2023-12-27 16:32:16,090][20381] Updated weights for policy 1, policy_version 1570455 (0.0010) [2023-12-27 16:32:16,703][20347] Updated weights for policy 0, policy_version 1572186 (0.0007) [2023-12-27 16:32:16,753][20347] Updated weights for policy 0, policy_version 1572196 (0.0009) [2023-12-27 16:32:16,760][20381] Updated weights for policy 1, policy_version 1570465 (0.0011) [2023-12-27 16:32:16,808][20347] Updated weights for policy 0, policy_version 1572206 (0.0006) [2023-12-27 16:32:16,821][20381] Updated weights for policy 1, policy_version 1570475 (0.0011) [2023-12-27 16:32:16,879][20381] Updated weights for policy 1, policy_version 1570485 (0.0010) [2023-12-27 16:32:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 804651008. Throughput: 0: 10187.4, 1: 10301.7. Samples: 804663620. Policy #0 lag: (min: 13.0, avg: 17.5, max: 45.0) [2023-12-27 16:32:17,363][19170] Avg episode reward: [(0, '941.350'), (1, '853.840')] [2023-12-27 16:32:17,387][20347] Updated weights for policy 0, policy_version 1572216 (0.0006) [2023-12-27 16:32:17,441][20347] Updated weights for policy 0, policy_version 1572226 (0.0007) [2023-12-27 16:32:17,497][20347] Updated weights for policy 0, policy_version 1572236 (0.0006) [2023-12-27 16:32:17,507][20381] Updated weights for policy 1, policy_version 1570495 (0.0011) [2023-12-27 16:32:17,564][20381] Updated weights for policy 1, policy_version 1570505 (0.0007) [2023-12-27 16:32:17,627][20381] Updated weights for policy 1, policy_version 1570515 (0.0008) [2023-12-27 16:32:18,101][20347] Updated weights for policy 0, policy_version 1572246 (0.0008) [2023-12-27 16:32:18,148][20347] Updated weights for policy 0, policy_version 1572256 (0.0008) [2023-12-27 16:32:18,198][20381] Updated weights for policy 1, policy_version 1570525 (0.0008) [2023-12-27 16:32:18,203][20347] Updated weights for policy 0, policy_version 1572266 (0.0010) [2023-12-27 16:32:18,251][20381] Updated weights for policy 1, policy_version 1570535 (0.0005) [2023-12-27 16:32:18,311][20381] Updated weights for policy 1, policy_version 1570545 (0.0006) [2023-12-27 16:32:18,912][20347] Updated weights for policy 0, policy_version 1572276 (0.0009) [2023-12-27 16:32:18,933][20381] Updated weights for policy 1, policy_version 1570555 (0.0008) [2023-12-27 16:32:18,967][20347] Updated weights for policy 0, policy_version 1572286 (0.0006) [2023-12-27 16:32:18,991][20381] Updated weights for policy 1, policy_version 1570565 (0.0005) [2023-12-27 16:32:19,024][20347] Updated weights for policy 0, policy_version 1572296 (0.0007) [2023-12-27 16:32:19,041][20381] Updated weights for policy 1, policy_version 1570575 (0.0007) [2023-12-27 16:32:19,694][20347] Updated weights for policy 0, policy_version 1572306 (0.0005) [2023-12-27 16:32:19,759][20347] Updated weights for policy 0, policy_version 1572316 (0.0005) [2023-12-27 16:32:19,808][20381] Updated weights for policy 1, policy_version 1570585 (0.0007) [2023-12-27 16:32:19,822][20347] Updated weights for policy 0, policy_version 1572326 (0.0008) [2023-12-27 16:32:19,873][20381] Updated weights for policy 1, policy_version 1570595 (0.0009) [2023-12-27 16:32:19,887][20347] Updated weights for policy 0, policy_version 1572336 (0.0007) [2023-12-27 16:32:19,932][20381] Updated weights for policy 1, policy_version 1570605 (0.0007) [2023-12-27 16:32:19,998][20381] Updated weights for policy 1, policy_version 1570615 (0.0009) [2023-12-27 16:32:20,539][20347] Updated weights for policy 0, policy_version 1572346 (0.0008) [2023-12-27 16:32:20,607][20347] Updated weights for policy 0, policy_version 1572356 (0.0007) [2023-12-27 16:32:20,659][20347] Updated weights for policy 0, policy_version 1572366 (0.0008) [2023-12-27 16:32:20,746][20381] Updated weights for policy 1, policy_version 1570625 (0.0009) [2023-12-27 16:32:20,807][20381] Updated weights for policy 1, policy_version 1570635 (0.0007) [2023-12-27 16:32:20,860][20381] Updated weights for policy 1, policy_version 1570645 (0.0008) [2023-12-27 16:32:21,414][20347] Updated weights for policy 0, policy_version 1572376 (0.0008) [2023-12-27 16:32:21,468][20347] Updated weights for policy 0, policy_version 1572386 (0.0007) [2023-12-27 16:32:21,526][20347] Updated weights for policy 0, policy_version 1572396 (0.0007) [2023-12-27 16:32:21,651][20381] Updated weights for policy 1, policy_version 1570655 (0.0009) [2023-12-27 16:32:21,715][20381] Updated weights for policy 1, policy_version 1570665 (0.0007) [2023-12-27 16:32:21,780][20381] Updated weights for policy 1, policy_version 1570675 (0.0007) [2023-12-27 16:32:22,261][20347] Updated weights for policy 0, policy_version 1572406 (0.0007) [2023-12-27 16:32:22,315][20347] Updated weights for policy 0, policy_version 1572416 (0.0006) [2023-12-27 16:32:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 804749312. Throughput: 0: 10239.0, 1: 10247.9. Samples: 804726616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:22,363][19170] Avg episode reward: [(0, '921.670'), (1, '824.530')] [2023-12-27 16:32:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001570680_402137088.pth... [2023-12-27 16:32:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001569496_401833984.pth [2023-12-27 16:32:22,378][20347] Updated weights for policy 0, policy_version 1572426 (0.0008) [2023-12-27 16:32:22,419][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001572432_402620416.pth... [2023-12-27 16:32:22,423][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001571216_402309120.pth [2023-12-27 16:32:22,489][20381] Updated weights for policy 1, policy_version 1570685 (0.0007) [2023-12-27 16:32:22,547][20381] Updated weights for policy 1, policy_version 1570695 (0.0008) [2023-12-27 16:32:22,610][20381] Updated weights for policy 1, policy_version 1570705 (0.0008) [2023-12-27 16:32:23,045][20347] Updated weights for policy 0, policy_version 1572436 (0.0007) [2023-12-27 16:32:23,101][20347] Updated weights for policy 0, policy_version 1572446 (0.0007) [2023-12-27 16:32:23,160][20347] Updated weights for policy 0, policy_version 1572456 (0.0008) [2023-12-27 16:32:23,276][20381] Updated weights for policy 1, policy_version 1570715 (0.0008) [2023-12-27 16:32:23,330][20381] Updated weights for policy 1, policy_version 1570725 (0.0008) [2023-12-27 16:32:23,376][20381] Updated weights for policy 1, policy_version 1570735 (0.0008) [2023-12-27 16:32:23,780][20347] Updated weights for policy 0, policy_version 1572466 (0.0007) [2023-12-27 16:32:23,839][20347] Updated weights for policy 0, policy_version 1572476 (0.0006) [2023-12-27 16:32:23,902][20347] Updated weights for policy 0, policy_version 1572486 (0.0005) [2023-12-27 16:32:23,960][20347] Updated weights for policy 0, policy_version 1572496 (0.0005) [2023-12-27 16:32:24,056][20381] Updated weights for policy 1, policy_version 1570745 (0.0010) [2023-12-27 16:32:24,115][20381] Updated weights for policy 1, policy_version 1570755 (0.0005) [2023-12-27 16:32:24,180][20381] Updated weights for policy 1, policy_version 1570765 (0.0005) [2023-12-27 16:32:24,235][20381] Updated weights for policy 1, policy_version 1570775 (0.0005) [2023-12-27 16:32:24,589][20347] Updated weights for policy 0, policy_version 1572506 (0.0010) [2023-12-27 16:32:24,638][20347] Updated weights for policy 0, policy_version 1572516 (0.0010) [2023-12-27 16:32:24,704][20347] Updated weights for policy 0, policy_version 1572526 (0.0010) [2023-12-27 16:32:24,805][20381] Updated weights for policy 1, policy_version 1570785 (0.0009) [2023-12-27 16:32:24,873][20381] Updated weights for policy 1, policy_version 1570795 (0.0011) [2023-12-27 16:32:24,937][20381] Updated weights for policy 1, policy_version 1570805 (0.0011) [2023-12-27 16:32:25,325][20347] Updated weights for policy 0, policy_version 1572536 (0.0007) [2023-12-27 16:32:25,379][20347] Updated weights for policy 0, policy_version 1572546 (0.0008) [2023-12-27 16:32:25,433][20347] Updated weights for policy 0, policy_version 1572556 (0.0006) [2023-12-27 16:32:25,603][20381] Updated weights for policy 1, policy_version 1570815 (0.0009) [2023-12-27 16:32:25,650][20381] Updated weights for policy 1, policy_version 1570825 (0.0007) [2023-12-27 16:32:25,696][20381] Updated weights for policy 1, policy_version 1570835 (0.0010) [2023-12-27 16:32:26,174][20347] Updated weights for policy 0, policy_version 1572566 (0.0008) [2023-12-27 16:32:26,226][20347] Updated weights for policy 0, policy_version 1572576 (0.0006) [2023-12-27 16:32:26,287][20347] Updated weights for policy 0, policy_version 1572586 (0.0005) [2023-12-27 16:32:26,331][20381] Updated weights for policy 1, policy_version 1570845 (0.0006) [2023-12-27 16:32:26,382][20381] Updated weights for policy 1, policy_version 1570855 (0.0006) [2023-12-27 16:32:26,427][20381] Updated weights for policy 1, policy_version 1570865 (0.0010) [2023-12-27 16:32:26,922][20347] Updated weights for policy 0, policy_version 1572596 (0.0009) [2023-12-27 16:32:26,967][20347] Updated weights for policy 0, policy_version 1572606 (0.0010) [2023-12-27 16:32:27,010][20381] Updated weights for policy 1, policy_version 1570875 (0.0009) [2023-12-27 16:32:27,012][20347] Updated weights for policy 0, policy_version 1572616 (0.0010) [2023-12-27 16:32:27,060][20381] Updated weights for policy 1, policy_version 1570885 (0.0009) [2023-12-27 16:32:27,116][20381] Updated weights for policy 1, policy_version 1570895 (0.0005) [2023-12-27 16:32:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 804864000. Throughput: 0: 10224.1, 1: 10256.1. Samples: 804848668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:27,363][19170] Avg episode reward: [(0, '931.430'), (1, '863.380')] [2023-12-27 16:32:27,669][20381] Updated weights for policy 1, policy_version 1570905 (0.0005) [2023-12-27 16:32:27,696][20347] Updated weights for policy 0, policy_version 1572626 (0.0010) [2023-12-27 16:32:27,726][20381] Updated weights for policy 1, policy_version 1570915 (0.0005) [2023-12-27 16:32:27,750][20347] Updated weights for policy 0, policy_version 1572636 (0.0010) [2023-12-27 16:32:27,784][20381] Updated weights for policy 1, policy_version 1570925 (0.0005) [2023-12-27 16:32:27,804][20347] Updated weights for policy 0, policy_version 1572646 (0.0010) [2023-12-27 16:32:27,836][20381] Updated weights for policy 1, policy_version 1570935 (0.0005) [2023-12-27 16:32:27,848][20347] Updated weights for policy 0, policy_version 1572656 (0.0010) [2023-12-27 16:32:28,443][20381] Updated weights for policy 1, policy_version 1570945 (0.0007) [2023-12-27 16:32:28,490][20381] Updated weights for policy 1, policy_version 1570955 (0.0007) [2023-12-27 16:32:28,518][20347] Updated weights for policy 0, policy_version 1572666 (0.0010) [2023-12-27 16:32:28,532][20381] Updated weights for policy 1, policy_version 1570965 (0.0006) [2023-12-27 16:32:28,563][20347] Updated weights for policy 0, policy_version 1572676 (0.0010) [2023-12-27 16:32:28,615][20347] Updated weights for policy 0, policy_version 1572686 (0.0010) [2023-12-27 16:32:29,265][20381] Updated weights for policy 1, policy_version 1570975 (0.0008) [2023-12-27 16:32:29,307][20347] Updated weights for policy 0, policy_version 1572696 (0.0010) [2023-12-27 16:32:29,319][20381] Updated weights for policy 1, policy_version 1570985 (0.0007) [2023-12-27 16:32:29,369][20347] Updated weights for policy 0, policy_version 1572706 (0.0008) [2023-12-27 16:32:29,382][20381] Updated weights for policy 1, policy_version 1570995 (0.0008) [2023-12-27 16:32:29,434][20347] Updated weights for policy 0, policy_version 1572716 (0.0008) [2023-12-27 16:32:30,074][20381] Updated weights for policy 1, policy_version 1571005 (0.0007) [2023-12-27 16:32:30,107][20347] Updated weights for policy 0, policy_version 1572726 (0.0008) [2023-12-27 16:32:30,140][20381] Updated weights for policy 1, policy_version 1571015 (0.0007) [2023-12-27 16:32:30,164][20347] Updated weights for policy 0, policy_version 1572736 (0.0007) [2023-12-27 16:32:30,197][20381] Updated weights for policy 1, policy_version 1571025 (0.0005) [2023-12-27 16:32:30,221][20347] Updated weights for policy 0, policy_version 1572746 (0.0006) [2023-12-27 16:32:30,849][20381] Updated weights for policy 1, policy_version 1571035 (0.0005) [2023-12-27 16:32:30,906][20381] Updated weights for policy 1, policy_version 1571045 (0.0005) [2023-12-27 16:32:30,951][20347] Updated weights for policy 0, policy_version 1572756 (0.0007) [2023-12-27 16:32:30,952][20381] Updated weights for policy 1, policy_version 1571055 (0.0005) [2023-12-27 16:32:31,004][20347] Updated weights for policy 0, policy_version 1572766 (0.0009) [2023-12-27 16:32:31,070][20347] Updated weights for policy 0, policy_version 1572776 (0.0008) [2023-12-27 16:32:31,646][20381] Updated weights for policy 1, policy_version 1571065 (0.0006) [2023-12-27 16:32:31,690][20292] KL-divergence is very high: 176.0150 [2023-12-27 16:32:31,711][20381] Updated weights for policy 1, policy_version 1571075 (0.0007) [2023-12-27 16:32:31,742][20292] KL-divergence is very high: 242.4647 [2023-12-27 16:32:31,772][20381] Updated weights for policy 1, policy_version 1571085 (0.0007) [2023-12-27 16:32:31,789][20292] KL-divergence is very high: 180.0380 [2023-12-27 16:32:31,822][20347] Updated weights for policy 0, policy_version 1572786 (0.0006) [2023-12-27 16:32:31,830][20381] Updated weights for policy 1, policy_version 1571095 (0.0005) [2023-12-27 16:32:31,871][20347] Updated weights for policy 0, policy_version 1572796 (0.0008) [2023-12-27 16:32:31,928][20347] Updated weights for policy 0, policy_version 1572806 (0.0008) [2023-12-27 16:32:31,994][20347] Updated weights for policy 0, policy_version 1572816 (0.0008) [2023-12-27 16:32:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 804962304. Throughput: 0: 10218.4, 1: 10289.3. Samples: 804973168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:32,363][19170] Avg episode reward: [(0, '951.110'), (1, '853.590')] [2023-12-27 16:32:32,430][20381] Updated weights for policy 1, policy_version 1571105 (0.0009) [2023-12-27 16:32:32,482][20381] Updated weights for policy 1, policy_version 1571115 (0.0005) [2023-12-27 16:32:32,545][20381] Updated weights for policy 1, policy_version 1571125 (0.0005) [2023-12-27 16:32:32,670][20347] Updated weights for policy 0, policy_version 1572826 (0.0005) [2023-12-27 16:32:32,726][20347] Updated weights for policy 0, policy_version 1572836 (0.0005) [2023-12-27 16:32:32,786][20347] Updated weights for policy 0, policy_version 1572846 (0.0005) [2023-12-27 16:32:33,173][20381] Updated weights for policy 1, policy_version 1571135 (0.0008) [2023-12-27 16:32:33,232][20381] Updated weights for policy 1, policy_version 1571145 (0.0005) [2023-12-27 16:32:33,287][20381] Updated weights for policy 1, policy_version 1571155 (0.0008) [2023-12-27 16:32:33,367][20347] Updated weights for policy 0, policy_version 1572856 (0.0007) [2023-12-27 16:32:33,424][20347] Updated weights for policy 0, policy_version 1572866 (0.0007) [2023-12-27 16:32:33,476][20347] Updated weights for policy 0, policy_version 1572876 (0.0007) [2023-12-27 16:32:33,940][20381] Updated weights for policy 1, policy_version 1571165 (0.0010) [2023-12-27 16:32:33,997][20381] Updated weights for policy 1, policy_version 1571175 (0.0010) [2023-12-27 16:32:34,060][20381] Updated weights for policy 1, policy_version 1571185 (0.0010) [2023-12-27 16:32:34,123][20347] Updated weights for policy 0, policy_version 1572886 (0.0009) [2023-12-27 16:32:34,185][20347] Updated weights for policy 0, policy_version 1572896 (0.0009) [2023-12-27 16:32:34,239][20347] Updated weights for policy 0, policy_version 1572906 (0.0010) [2023-12-27 16:32:34,698][20381] Updated weights for policy 1, policy_version 1571195 (0.0010) [2023-12-27 16:32:34,753][20381] Updated weights for policy 1, policy_version 1571205 (0.0007) [2023-12-27 16:32:34,807][20381] Updated weights for policy 1, policy_version 1571215 (0.0007) [2023-12-27 16:32:34,968][20347] Updated weights for policy 0, policy_version 1572916 (0.0008) [2023-12-27 16:32:35,029][20347] Updated weights for policy 0, policy_version 1572926 (0.0005) [2023-12-27 16:32:35,087][20347] Updated weights for policy 0, policy_version 1572936 (0.0006) [2023-12-27 16:32:35,518][20381] Updated weights for policy 1, policy_version 1571225 (0.0008) [2023-12-27 16:32:35,579][20381] Updated weights for policy 1, policy_version 1571235 (0.0008) [2023-12-27 16:32:35,634][20381] Updated weights for policy 1, policy_version 1571245 (0.0008) [2023-12-27 16:32:35,695][20347] Updated weights for policy 0, policy_version 1572946 (0.0008) [2023-12-27 16:32:35,697][20381] Updated weights for policy 1, policy_version 1571255 (0.0007) [2023-12-27 16:32:35,752][20347] Updated weights for policy 0, policy_version 1572956 (0.0007) [2023-12-27 16:32:35,816][20347] Updated weights for policy 0, policy_version 1572966 (0.0008) [2023-12-27 16:32:35,871][20347] Updated weights for policy 0, policy_version 1572976 (0.0010) [2023-12-27 16:32:36,351][20381] Updated weights for policy 1, policy_version 1571265 (0.0008) [2023-12-27 16:32:36,412][20381] Updated weights for policy 1, policy_version 1571275 (0.0009) [2023-12-27 16:32:36,469][20381] Updated weights for policy 1, policy_version 1571285 (0.0008) [2023-12-27 16:32:36,538][20347] Updated weights for policy 0, policy_version 1572986 (0.0008) [2023-12-27 16:32:36,599][20347] Updated weights for policy 0, policy_version 1572996 (0.0008) [2023-12-27 16:32:36,658][20347] Updated weights for policy 0, policy_version 1573006 (0.0008) [2023-12-27 16:32:37,182][20381] Updated weights for policy 1, policy_version 1571295 (0.0007) [2023-12-27 16:32:37,241][20381] Updated weights for policy 1, policy_version 1571305 (0.0008) [2023-12-27 16:32:37,293][20381] Updated weights for policy 1, policy_version 1571315 (0.0007) [2023-12-27 16:32:37,353][20347] Updated weights for policy 0, policy_version 1573016 (0.0010) [2023-12-27 16:32:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 805068800. Throughput: 0: 10225.8, 1: 10327.2. Samples: 805037036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:37,364][19170] Avg episode reward: [(0, '980.520'), (1, '853.540')] [2023-12-27 16:32:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001571320_402300928.pth... [2023-12-27 16:32:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001570104_401989632.pth [2023-12-27 16:32:37,405][20347] Updated weights for policy 0, policy_version 1573026 (0.0008) [2023-12-27 16:32:37,462][20347] Updated weights for policy 0, policy_version 1573036 (0.0007) [2023-12-27 16:32:37,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001573040_402776064.pth... [2023-12-27 16:32:37,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001571824_402464768.pth [2023-12-27 16:32:37,964][20381] Updated weights for policy 1, policy_version 1571325 (0.0007) [2023-12-27 16:32:38,019][20381] Updated weights for policy 1, policy_version 1571335 (0.0008) [2023-12-27 16:32:38,044][20347] Updated weights for policy 0, policy_version 1573046 (0.0006) [2023-12-27 16:32:38,070][20381] Updated weights for policy 1, policy_version 1571345 (0.0007) [2023-12-27 16:32:38,098][20347] Updated weights for policy 0, policy_version 1573056 (0.0006) [2023-12-27 16:32:38,151][20347] Updated weights for policy 0, policy_version 1573066 (0.0007) [2023-12-27 16:32:38,782][20381] Updated weights for policy 1, policy_version 1571355 (0.0006) [2023-12-27 16:32:38,827][20381] Updated weights for policy 1, policy_version 1571365 (0.0006) [2023-12-27 16:32:38,860][20347] Updated weights for policy 0, policy_version 1573076 (0.0009) [2023-12-27 16:32:38,881][20381] Updated weights for policy 1, policy_version 1571375 (0.0006) [2023-12-27 16:32:38,920][20347] Updated weights for policy 0, policy_version 1573086 (0.0010) [2023-12-27 16:32:38,977][20347] Updated weights for policy 0, policy_version 1573096 (0.0010) [2023-12-27 16:32:39,587][20381] Updated weights for policy 1, policy_version 1571385 (0.0005) [2023-12-27 16:32:39,674][20381] Updated weights for policy 1, policy_version 1571395 (0.0006) [2023-12-27 16:32:39,729][20347] Updated weights for policy 0, policy_version 1573106 (0.0010) [2023-12-27 16:32:39,733][20381] Updated weights for policy 1, policy_version 1571405 (0.0007) [2023-12-27 16:32:39,791][20347] Updated weights for policy 0, policy_version 1573116 (0.0007) [2023-12-27 16:32:39,793][20381] Updated weights for policy 1, policy_version 1571415 (0.0006) [2023-12-27 16:32:39,857][20347] Updated weights for policy 0, policy_version 1573126 (0.0008) [2023-12-27 16:32:39,924][20347] Updated weights for policy 0, policy_version 1573136 (0.0008) [2023-12-27 16:32:40,364][20381] Updated weights for policy 1, policy_version 1571425 (0.0007) [2023-12-27 16:32:40,424][20381] Updated weights for policy 1, policy_version 1571435 (0.0007) [2023-12-27 16:32:40,480][20381] Updated weights for policy 1, policy_version 1571445 (0.0007) [2023-12-27 16:32:40,658][20347] Updated weights for policy 0, policy_version 1573146 (0.0005) [2023-12-27 16:32:40,718][20347] Updated weights for policy 0, policy_version 1573156 (0.0005) [2023-12-27 16:32:40,778][20347] Updated weights for policy 0, policy_version 1573166 (0.0008) [2023-12-27 16:32:41,162][20381] Updated weights for policy 1, policy_version 1571455 (0.0008) [2023-12-27 16:32:41,220][20381] Updated weights for policy 1, policy_version 1571465 (0.0007) [2023-12-27 16:32:41,284][20381] Updated weights for policy 1, policy_version 1571475 (0.0010) [2023-12-27 16:32:41,482][20347] Updated weights for policy 0, policy_version 1573176 (0.0009) [2023-12-27 16:32:41,543][20347] Updated weights for policy 0, policy_version 1573186 (0.0008) [2023-12-27 16:32:41,603][20347] Updated weights for policy 0, policy_version 1573196 (0.0008) [2023-12-27 16:32:42,013][20381] Updated weights for policy 1, policy_version 1571485 (0.0008) [2023-12-27 16:32:42,064][20381] Updated weights for policy 1, policy_version 1571495 (0.0006) [2023-12-27 16:32:42,120][20381] Updated weights for policy 1, policy_version 1571505 (0.0006) [2023-12-27 16:32:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 805167104. Throughput: 0: 10267.6, 1: 10273.0. Samples: 805159096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:42,363][19170] Avg episode reward: [(0, '980.490'), (1, '883.040')] [2023-12-27 16:32:42,440][20347] Updated weights for policy 0, policy_version 1573206 (0.0008) [2023-12-27 16:32:42,492][20347] Updated weights for policy 0, policy_version 1573216 (0.0007) [2023-12-27 16:32:42,540][20347] Updated weights for policy 0, policy_version 1573226 (0.0007) [2023-12-27 16:32:42,791][20381] Updated weights for policy 1, policy_version 1571515 (0.0008) [2023-12-27 16:32:42,844][20381] Updated weights for policy 1, policy_version 1571525 (0.0008) [2023-12-27 16:32:42,891][20381] Updated weights for policy 1, policy_version 1571535 (0.0007) [2023-12-27 16:32:43,226][20347] Updated weights for policy 0, policy_version 1573236 (0.0007) [2023-12-27 16:32:43,277][20347] Updated weights for policy 0, policy_version 1573246 (0.0010) [2023-12-27 16:32:43,331][20347] Updated weights for policy 0, policy_version 1573256 (0.0010) [2023-12-27 16:32:43,571][20381] Updated weights for policy 1, policy_version 1571545 (0.0009) [2023-12-27 16:32:43,626][20381] Updated weights for policy 1, policy_version 1571555 (0.0006) [2023-12-27 16:32:43,686][20381] Updated weights for policy 1, policy_version 1571565 (0.0006) [2023-12-27 16:32:43,741][20381] Updated weights for policy 1, policy_version 1571575 (0.0006) [2023-12-27 16:32:43,975][20347] Updated weights for policy 0, policy_version 1573266 (0.0010) [2023-12-27 16:32:44,030][20347] Updated weights for policy 0, policy_version 1573276 (0.0010) [2023-12-27 16:32:44,087][20347] Updated weights for policy 0, policy_version 1573286 (0.0010) [2023-12-27 16:32:44,150][20347] Updated weights for policy 0, policy_version 1573296 (0.0009) [2023-12-27 16:32:44,303][20381] Updated weights for policy 1, policy_version 1571585 (0.0006) [2023-12-27 16:32:44,359][20381] Updated weights for policy 1, policy_version 1571595 (0.0005) [2023-12-27 16:32:44,413][20381] Updated weights for policy 1, policy_version 1571605 (0.0011) [2023-12-27 16:32:44,809][20347] Updated weights for policy 0, policy_version 1573306 (0.0008) [2023-12-27 16:32:44,865][20347] Updated weights for policy 0, policy_version 1573316 (0.0007) [2023-12-27 16:32:44,913][20347] Updated weights for policy 0, policy_version 1573326 (0.0008) [2023-12-27 16:32:45,110][20381] Updated weights for policy 1, policy_version 1571615 (0.0011) [2023-12-27 16:32:45,164][20381] Updated weights for policy 1, policy_version 1571625 (0.0011) [2023-12-27 16:32:45,221][20381] Updated weights for policy 1, policy_version 1571635 (0.0011) [2023-12-27 16:32:45,629][20347] Updated weights for policy 0, policy_version 1573336 (0.0008) [2023-12-27 16:32:45,679][20347] Updated weights for policy 0, policy_version 1573346 (0.0008) [2023-12-27 16:32:45,738][20347] Updated weights for policy 0, policy_version 1573356 (0.0010) [2023-12-27 16:32:45,855][20381] Updated weights for policy 1, policy_version 1571645 (0.0010) [2023-12-27 16:32:45,904][20381] Updated weights for policy 1, policy_version 1571655 (0.0011) [2023-12-27 16:32:45,953][20381] Updated weights for policy 1, policy_version 1571665 (0.0010) [2023-12-27 16:32:46,306][20347] Updated weights for policy 0, policy_version 1573366 (0.0008) [2023-12-27 16:32:46,362][20347] Updated weights for policy 0, policy_version 1573376 (0.0006) [2023-12-27 16:32:46,421][20347] Updated weights for policy 0, policy_version 1573386 (0.0008) [2023-12-27 16:32:46,624][20381] Updated weights for policy 1, policy_version 1571675 (0.0011) [2023-12-27 16:32:46,684][20381] Updated weights for policy 1, policy_version 1571685 (0.0011) [2023-12-27 16:32:46,737][20381] Updated weights for policy 1, policy_version 1571695 (0.0010) [2023-12-27 16:32:47,081][20347] Updated weights for policy 0, policy_version 1573396 (0.0007) [2023-12-27 16:32:47,139][20347] Updated weights for policy 0, policy_version 1573406 (0.0010) [2023-12-27 16:32:47,197][20347] Updated weights for policy 0, policy_version 1573416 (0.0010) [2023-12-27 16:32:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 805273600. Throughput: 0: 10295.0, 1: 10311.2. Samples: 805282524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:47,364][19170] Avg episode reward: [(0, '980.410'), (1, '892.600')] [2023-12-27 16:32:47,433][20381] Updated weights for policy 1, policy_version 1571705 (0.0010) [2023-12-27 16:32:47,485][20381] Updated weights for policy 1, policy_version 1571715 (0.0005) [2023-12-27 16:32:47,536][20381] Updated weights for policy 1, policy_version 1571725 (0.0005) [2023-12-27 16:32:47,588][20381] Updated weights for policy 1, policy_version 1571735 (0.0005) [2023-12-27 16:32:47,819][20347] Updated weights for policy 0, policy_version 1573426 (0.0010) [2023-12-27 16:32:47,876][20347] Updated weights for policy 0, policy_version 1573436 (0.0010) [2023-12-27 16:32:47,933][20347] Updated weights for policy 0, policy_version 1573446 (0.0010) [2023-12-27 16:32:47,985][20347] Updated weights for policy 0, policy_version 1573456 (0.0010) [2023-12-27 16:32:48,175][20381] Updated weights for policy 1, policy_version 1571745 (0.0009) [2023-12-27 16:32:48,231][20381] Updated weights for policy 1, policy_version 1571755 (0.0011) [2023-12-27 16:32:48,286][20381] Updated weights for policy 1, policy_version 1571765 (0.0011) [2023-12-27 16:32:48,697][20347] Updated weights for policy 0, policy_version 1573466 (0.0007) [2023-12-27 16:32:48,752][20347] Updated weights for policy 0, policy_version 1573476 (0.0006) [2023-12-27 16:32:48,801][20347] Updated weights for policy 0, policy_version 1573486 (0.0009) [2023-12-27 16:32:49,024][20381] Updated weights for policy 1, policy_version 1571775 (0.0011) [2023-12-27 16:32:49,091][20381] Updated weights for policy 1, policy_version 1571785 (0.0011) [2023-12-27 16:32:49,146][20381] Updated weights for policy 1, policy_version 1571795 (0.0011) [2023-12-27 16:32:49,465][20347] Updated weights for policy 0, policy_version 1573496 (0.0010) [2023-12-27 16:32:49,529][20347] Updated weights for policy 0, policy_version 1573506 (0.0007) [2023-12-27 16:32:49,584][20347] Updated weights for policy 0, policy_version 1573516 (0.0010) [2023-12-27 16:32:49,836][20381] Updated weights for policy 1, policy_version 1571805 (0.0010) [2023-12-27 16:32:49,895][20381] Updated weights for policy 1, policy_version 1571815 (0.0008) [2023-12-27 16:32:49,961][20381] Updated weights for policy 1, policy_version 1571825 (0.0009) [2023-12-27 16:32:50,272][20347] Updated weights for policy 0, policy_version 1573526 (0.0010) [2023-12-27 16:32:50,333][20347] Updated weights for policy 0, policy_version 1573536 (0.0010) [2023-12-27 16:32:50,388][20347] Updated weights for policy 0, policy_version 1573546 (0.0010) [2023-12-27 16:32:50,638][20381] Updated weights for policy 1, policy_version 1571835 (0.0007) [2023-12-27 16:32:50,691][20381] Updated weights for policy 1, policy_version 1571845 (0.0005) [2023-12-27 16:32:50,741][20381] Updated weights for policy 1, policy_version 1571855 (0.0005) [2023-12-27 16:32:51,035][20347] Updated weights for policy 0, policy_version 1573556 (0.0009) [2023-12-27 16:32:51,091][20347] Updated weights for policy 0, policy_version 1573566 (0.0008) [2023-12-27 16:32:51,157][20347] Updated weights for policy 0, policy_version 1573576 (0.0009) [2023-12-27 16:32:51,439][20381] Updated weights for policy 1, policy_version 1571865 (0.0006) [2023-12-27 16:32:51,497][20381] Updated weights for policy 1, policy_version 1571875 (0.0008) [2023-12-27 16:32:51,553][20381] Updated weights for policy 1, policy_version 1571885 (0.0008) [2023-12-27 16:32:51,613][20381] Updated weights for policy 1, policy_version 1571895 (0.0008) [2023-12-27 16:32:51,886][20347] Updated weights for policy 0, policy_version 1573586 (0.0007) [2023-12-27 16:32:51,933][20347] Updated weights for policy 0, policy_version 1573596 (0.0008) [2023-12-27 16:32:51,992][20347] Updated weights for policy 0, policy_version 1573606 (0.0007) [2023-12-27 16:32:52,053][20347] Updated weights for policy 0, policy_version 1573616 (0.0008) [2023-12-27 16:32:52,359][20381] Updated weights for policy 1, policy_version 1571905 (0.0006) [2023-12-27 16:32:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 805371904. Throughput: 0: 10290.7, 1: 10311.9. Samples: 805344008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:52,363][19170] Avg episode reward: [(0, '980.450'), (1, '775.080')] [2023-12-27 16:32:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001573616_402923520.pth... [2023-12-27 16:32:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001572432_402620416.pth [2023-12-27 16:32:52,425][20381] Updated weights for policy 1, policy_version 1571915 (0.0007) [2023-12-27 16:32:52,483][20381] Updated weights for policy 1, policy_version 1571925 (0.0006) [2023-12-27 16:32:52,498][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001571928_402456576.pth... [2023-12-27 16:32:52,503][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001570680_402137088.pth [2023-12-27 16:32:52,788][20347] Updated weights for policy 0, policy_version 1573626 (0.0008) [2023-12-27 16:32:52,849][20347] Updated weights for policy 0, policy_version 1573636 (0.0005) [2023-12-27 16:32:52,907][20347] Updated weights for policy 0, policy_version 1573646 (0.0005) [2023-12-27 16:32:53,121][20381] Updated weights for policy 1, policy_version 1571935 (0.0007) [2023-12-27 16:32:53,178][20381] Updated weights for policy 1, policy_version 1571945 (0.0007) [2023-12-27 16:32:53,233][20381] Updated weights for policy 1, policy_version 1571955 (0.0007) [2023-12-27 16:32:53,554][20347] Updated weights for policy 0, policy_version 1573656 (0.0007) [2023-12-27 16:32:53,607][20347] Updated weights for policy 0, policy_version 1573666 (0.0008) [2023-12-27 16:32:53,662][20347] Updated weights for policy 0, policy_version 1573677 (0.0006) [2023-12-27 16:32:53,859][20381] Updated weights for policy 1, policy_version 1571965 (0.0006) [2023-12-27 16:32:53,920][20381] Updated weights for policy 1, policy_version 1571975 (0.0005) [2023-12-27 16:32:53,983][20381] Updated weights for policy 1, policy_version 1571985 (0.0005) [2023-12-27 16:32:54,314][20347] Updated weights for policy 0, policy_version 1573687 (0.0009) [2023-12-27 16:32:54,368][20347] Updated weights for policy 0, policy_version 1573697 (0.0010) [2023-12-27 16:32:54,426][20347] Updated weights for policy 0, policy_version 1573707 (0.0010) [2023-12-27 16:32:54,579][20381] Updated weights for policy 1, policy_version 1571995 (0.0007) [2023-12-27 16:32:54,627][20381] Updated weights for policy 1, policy_version 1572005 (0.0010) [2023-12-27 16:32:54,675][20381] Updated weights for policy 1, policy_version 1572015 (0.0010) [2023-12-27 16:32:55,011][20347] Updated weights for policy 0, policy_version 1573717 (0.0008) [2023-12-27 16:32:55,075][20347] Updated weights for policy 0, policy_version 1573727 (0.0006) [2023-12-27 16:32:55,137][20347] Updated weights for policy 0, policy_version 1573737 (0.0005) [2023-12-27 16:32:55,317][20381] Updated weights for policy 1, policy_version 1572025 (0.0007) [2023-12-27 16:32:55,377][20381] Updated weights for policy 1, policy_version 1572035 (0.0008) [2023-12-27 16:32:55,436][20381] Updated weights for policy 1, policy_version 1572045 (0.0008) [2023-12-27 16:32:55,498][20381] Updated weights for policy 1, policy_version 1572055 (0.0007) [2023-12-27 16:32:55,774][20347] Updated weights for policy 0, policy_version 1573747 (0.0008) [2023-12-27 16:32:55,831][20347] Updated weights for policy 0, policy_version 1573757 (0.0008) [2023-12-27 16:32:55,859][20291] KL-divergence is very high: 197.8450 [2023-12-27 16:32:55,888][20347] Updated weights for policy 0, policy_version 1573767 (0.0007) [2023-12-27 16:32:55,906][20291] KL-divergence is very high: 191.6286 [2023-12-27 16:32:56,178][20381] Updated weights for policy 1, policy_version 1572065 (0.0009) [2023-12-27 16:32:56,228][20381] Updated weights for policy 1, policy_version 1572075 (0.0010) [2023-12-27 16:32:56,274][20381] Updated weights for policy 1, policy_version 1572085 (0.0010) [2023-12-27 16:32:56,518][20347] Updated weights for policy 0, policy_version 1573777 (0.0007) [2023-12-27 16:32:56,571][20347] Updated weights for policy 0, policy_version 1573787 (0.0010) [2023-12-27 16:32:56,628][20347] Updated weights for policy 0, policy_version 1573797 (0.0010) [2023-12-27 16:32:56,689][20347] Updated weights for policy 0, policy_version 1573807 (0.0006) [2023-12-27 16:32:56,927][20381] Updated weights for policy 1, policy_version 1572095 (0.0010) [2023-12-27 16:32:56,978][20381] Updated weights for policy 1, policy_version 1572105 (0.0010) [2023-12-27 16:32:57,027][20381] Updated weights for policy 1, policy_version 1572115 (0.0011) [2023-12-27 16:32:57,344][20347] Updated weights for policy 0, policy_version 1573817 (0.0005) [2023-12-27 16:32:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 805478400. Throughput: 0: 10239.5, 1: 10373.8. Samples: 805468928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:32:57,363][19170] Avg episode reward: [(0, '931.590'), (1, '804.490')] [2023-12-27 16:32:57,387][20347] Updated weights for policy 0, policy_version 1573827 (0.0006) [2023-12-27 16:32:57,439][20347] Updated weights for policy 0, policy_version 1573837 (0.0010) [2023-12-27 16:32:57,668][20381] Updated weights for policy 1, policy_version 1572125 (0.0008) [2023-12-27 16:32:57,729][20381] Updated weights for policy 1, policy_version 1572135 (0.0006) [2023-12-27 16:32:57,791][20381] Updated weights for policy 1, policy_version 1572145 (0.0010) [2023-12-27 16:32:58,117][20347] Updated weights for policy 0, policy_version 1573847 (0.0008) [2023-12-27 16:32:58,164][20347] Updated weights for policy 0, policy_version 1573857 (0.0007) [2023-12-27 16:32:58,218][20347] Updated weights for policy 0, policy_version 1573867 (0.0007) [2023-12-27 16:32:58,356][20381] Updated weights for policy 1, policy_version 1572155 (0.0011) [2023-12-27 16:32:58,418][20381] Updated weights for policy 1, policy_version 1572165 (0.0007) [2023-12-27 16:32:58,479][20381] Updated weights for policy 1, policy_version 1572175 (0.0008) [2023-12-27 16:32:59,061][20347] Updated weights for policy 0, policy_version 1573877 (0.0007) [2023-12-27 16:32:59,115][20347] Updated weights for policy 0, policy_version 1573887 (0.0009) [2023-12-27 16:32:59,179][20347] Updated weights for policy 0, policy_version 1573897 (0.0008) [2023-12-27 16:32:59,246][20381] Updated weights for policy 1, policy_version 1572185 (0.0008) [2023-12-27 16:32:59,310][20381] Updated weights for policy 1, policy_version 1572195 (0.0008) [2023-12-27 16:32:59,373][20381] Updated weights for policy 1, policy_version 1572205 (0.0008) [2023-12-27 16:32:59,433][20381] Updated weights for policy 1, policy_version 1572215 (0.0009) [2023-12-27 16:32:59,929][20347] Updated weights for policy 0, policy_version 1573907 (0.0008) [2023-12-27 16:32:59,989][20347] Updated weights for policy 0, policy_version 1573917 (0.0005) [2023-12-27 16:33:00,049][20347] Updated weights for policy 0, policy_version 1573927 (0.0006) [2023-12-27 16:33:00,127][20381] Updated weights for policy 1, policy_version 1572225 (0.0007) [2023-12-27 16:33:00,185][20381] Updated weights for policy 1, policy_version 1572235 (0.0008) [2023-12-27 16:33:00,243][20381] Updated weights for policy 1, policy_version 1572245 (0.0009) [2023-12-27 16:33:00,634][20347] Updated weights for policy 0, policy_version 1573937 (0.0007) [2023-12-27 16:33:00,689][20347] Updated weights for policy 0, policy_version 1573947 (0.0008) [2023-12-27 16:33:00,737][20347] Updated weights for policy 0, policy_version 1573957 (0.0007) [2023-12-27 16:33:00,792][20347] Updated weights for policy 0, policy_version 1573967 (0.0008) [2023-12-27 16:33:00,888][20381] Updated weights for policy 1, policy_version 1572255 (0.0009) [2023-12-27 16:33:00,952][20381] Updated weights for policy 1, policy_version 1572265 (0.0008) [2023-12-27 16:33:01,000][20381] Updated weights for policy 1, policy_version 1572275 (0.0008) [2023-12-27 16:33:01,501][20347] Updated weights for policy 0, policy_version 1573977 (0.0005) [2023-12-27 16:33:01,561][20347] Updated weights for policy 0, policy_version 1573987 (0.0006) [2023-12-27 16:33:01,620][20347] Updated weights for policy 0, policy_version 1573997 (0.0007) [2023-12-27 16:33:01,760][20381] Updated weights for policy 1, policy_version 1572285 (0.0007) [2023-12-27 16:33:01,811][20381] Updated weights for policy 1, policy_version 1572295 (0.0005) [2023-12-27 16:33:01,858][20381] Updated weights for policy 1, policy_version 1572305 (0.0006) [2023-12-27 16:33:02,301][20347] Updated weights for policy 0, policy_version 1574007 (0.0010) [2023-12-27 16:33:02,357][20347] Updated weights for policy 0, policy_version 1574017 (0.0010) [2023-12-27 16:33:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 805576704. Throughput: 0: 10246.9, 1: 10380.1. Samples: 805591836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:02,363][19170] Avg episode reward: [(0, '921.650'), (1, '814.200')] [2023-12-27 16:33:02,419][20347] Updated weights for policy 0, policy_version 1574027 (0.0010) [2023-12-27 16:33:02,547][20381] Updated weights for policy 1, policy_version 1572315 (0.0007) [2023-12-27 16:33:02,608][20381] Updated weights for policy 1, policy_version 1572325 (0.0007) [2023-12-27 16:33:02,663][20381] Updated weights for policy 1, policy_version 1572335 (0.0007) [2023-12-27 16:33:03,090][20347] Updated weights for policy 0, policy_version 1574037 (0.0010) [2023-12-27 16:33:03,141][20347] Updated weights for policy 0, policy_version 1574047 (0.0010) [2023-12-27 16:33:03,191][20347] Updated weights for policy 0, policy_version 1574057 (0.0010) [2023-12-27 16:33:03,288][20381] Updated weights for policy 1, policy_version 1572345 (0.0008) [2023-12-27 16:33:03,340][20381] Updated weights for policy 1, policy_version 1572355 (0.0005) [2023-12-27 16:33:03,388][20381] Updated weights for policy 1, policy_version 1572365 (0.0005) [2023-12-27 16:33:03,436][20381] Updated weights for policy 1, policy_version 1572375 (0.0005) [2023-12-27 16:33:03,888][20347] Updated weights for policy 0, policy_version 1574067 (0.0010) [2023-12-27 16:33:03,946][20347] Updated weights for policy 0, policy_version 1574077 (0.0007) [2023-12-27 16:33:04,038][20347] Updated weights for policy 0, policy_version 1574087 (0.0007) [2023-12-27 16:33:04,085][20381] Updated weights for policy 1, policy_version 1572385 (0.0006) [2023-12-27 16:33:04,150][20381] Updated weights for policy 1, policy_version 1572395 (0.0006) [2023-12-27 16:33:04,212][20381] Updated weights for policy 1, policy_version 1572405 (0.0006) [2023-12-27 16:33:04,713][20347] Updated weights for policy 0, policy_version 1574097 (0.0007) [2023-12-27 16:33:04,765][20381] Updated weights for policy 1, policy_version 1572415 (0.0005) [2023-12-27 16:33:04,766][20347] Updated weights for policy 0, policy_version 1574107 (0.0008) [2023-12-27 16:33:04,817][20381] Updated weights for policy 1, policy_version 1572425 (0.0005) [2023-12-27 16:33:04,818][20347] Updated weights for policy 0, policy_version 1574117 (0.0007) [2023-12-27 16:33:04,868][20347] Updated weights for policy 0, policy_version 1574127 (0.0007) [2023-12-27 16:33:04,870][20381] Updated weights for policy 1, policy_version 1572435 (0.0008) [2023-12-27 16:33:05,511][20347] Updated weights for policy 0, policy_version 1574137 (0.0005) [2023-12-27 16:33:05,519][20381] Updated weights for policy 1, policy_version 1572445 (0.0009) [2023-12-27 16:33:05,569][20347] Updated weights for policy 0, policy_version 1574147 (0.0009) [2023-12-27 16:33:05,571][20381] Updated weights for policy 1, policy_version 1572455 (0.0006) [2023-12-27 16:33:05,616][20381] Updated weights for policy 1, policy_version 1572465 (0.0007) [2023-12-27 16:33:05,621][20347] Updated weights for policy 0, policy_version 1574157 (0.0010) [2023-12-27 16:33:06,223][20381] Updated weights for policy 1, policy_version 1572475 (0.0010) [2023-12-27 16:33:06,286][20381] Updated weights for policy 1, policy_version 1572485 (0.0007) [2023-12-27 16:33:06,288][20347] Updated weights for policy 0, policy_version 1574167 (0.0008) [2023-12-27 16:33:06,343][20381] Updated weights for policy 1, policy_version 1572495 (0.0006) [2023-12-27 16:33:06,345][20347] Updated weights for policy 0, policy_version 1574177 (0.0008) [2023-12-27 16:33:06,401][20347] Updated weights for policy 0, policy_version 1574187 (0.0006) [2023-12-27 16:33:07,031][20347] Updated weights for policy 0, policy_version 1574197 (0.0007) [2023-12-27 16:33:07,075][20381] Updated weights for policy 1, policy_version 1572505 (0.0006) [2023-12-27 16:33:07,092][20347] Updated weights for policy 0, policy_version 1574207 (0.0007) [2023-12-27 16:33:07,122][20381] Updated weights for policy 1, policy_version 1572515 (0.0007) [2023-12-27 16:33:07,144][20347] Updated weights for policy 0, policy_version 1574217 (0.0007) [2023-12-27 16:33:07,169][20381] Updated weights for policy 1, policy_version 1572525 (0.0006) [2023-12-27 16:33:07,227][20381] Updated weights for policy 1, policy_version 1572535 (0.0007) [2023-12-27 16:33:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 805691392. Throughput: 0: 10195.1, 1: 10424.7. Samples: 805654508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:07,363][19170] Avg episode reward: [(0, '911.890'), (1, '912.010')] [2023-12-27 16:33:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001572536_402612224.pth... [2023-12-27 16:33:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001574224_403079168.pth... [2023-12-27 16:33:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001571320_402300928.pth [2023-12-27 16:33:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001573040_402776064.pth [2023-12-27 16:33:07,852][20347] Updated weights for policy 0, policy_version 1574227 (0.0007) [2023-12-27 16:33:07,906][20347] Updated weights for policy 0, policy_version 1574237 (0.0008) [2023-12-27 16:33:07,923][20381] Updated weights for policy 1, policy_version 1572545 (0.0006) [2023-12-27 16:33:07,954][20347] Updated weights for policy 0, policy_version 1574247 (0.0007) [2023-12-27 16:33:07,982][20381] Updated weights for policy 1, policy_version 1572555 (0.0005) [2023-12-27 16:33:08,042][20381] Updated weights for policy 1, policy_version 1572565 (0.0007) [2023-12-27 16:33:08,619][20381] Updated weights for policy 1, policy_version 1572575 (0.0008) [2023-12-27 16:33:08,629][20347] Updated weights for policy 0, policy_version 1574257 (0.0008) [2023-12-27 16:33:08,673][20381] Updated weights for policy 1, policy_version 1572585 (0.0006) [2023-12-27 16:33:08,699][20347] Updated weights for policy 0, policy_version 1574267 (0.0009) [2023-12-27 16:33:08,723][20381] Updated weights for policy 1, policy_version 1572595 (0.0006) [2023-12-27 16:33:08,759][20347] Updated weights for policy 0, policy_version 1574277 (0.0006) [2023-12-27 16:33:08,815][20347] Updated weights for policy 0, policy_version 1574287 (0.0010) [2023-12-27 16:33:09,441][20347] Updated weights for policy 0, policy_version 1574297 (0.0008) [2023-12-27 16:33:09,493][20381] Updated weights for policy 1, policy_version 1572605 (0.0008) [2023-12-27 16:33:09,500][20347] Updated weights for policy 0, policy_version 1574307 (0.0008) [2023-12-27 16:33:09,558][20381] Updated weights for policy 1, policy_version 1572615 (0.0010) [2023-12-27 16:33:09,565][20347] Updated weights for policy 0, policy_version 1574317 (0.0007) [2023-12-27 16:33:09,628][20381] Updated weights for policy 1, policy_version 1572625 (0.0011) [2023-12-27 16:33:10,256][20347] Updated weights for policy 0, policy_version 1574327 (0.0010) [2023-12-27 16:33:10,314][20381] Updated weights for policy 1, policy_version 1572635 (0.0010) [2023-12-27 16:33:10,318][20347] Updated weights for policy 0, policy_version 1574337 (0.0010) [2023-12-27 16:33:10,374][20381] Updated weights for policy 1, policy_version 1572645 (0.0010) [2023-12-27 16:33:10,375][20347] Updated weights for policy 0, policy_version 1574347 (0.0010) [2023-12-27 16:33:10,431][20381] Updated weights for policy 1, policy_version 1572655 (0.0011) [2023-12-27 16:33:11,105][20347] Updated weights for policy 0, policy_version 1574357 (0.0009) [2023-12-27 16:33:11,139][20381] Updated weights for policy 1, policy_version 1572665 (0.0008) [2023-12-27 16:33:11,172][20347] Updated weights for policy 0, policy_version 1574367 (0.0007) [2023-12-27 16:33:11,201][20381] Updated weights for policy 1, policy_version 1572675 (0.0010) [2023-12-27 16:33:11,232][20347] Updated weights for policy 0, policy_version 1574377 (0.0006) [2023-12-27 16:33:11,258][20291] KL-divergence is very high: 100.2883 [2023-12-27 16:33:11,263][20381] Updated weights for policy 1, policy_version 1572685 (0.0008) [2023-12-27 16:33:11,327][20381] Updated weights for policy 1, policy_version 1572695 (0.0009) [2023-12-27 16:33:11,989][20347] Updated weights for policy 0, policy_version 1574387 (0.0007) [2023-12-27 16:33:12,023][20381] Updated weights for policy 1, policy_version 1572705 (0.0007) [2023-12-27 16:33:12,036][20347] Updated weights for policy 0, policy_version 1574397 (0.0005) [2023-12-27 16:33:12,079][20381] Updated weights for policy 1, policy_version 1572715 (0.0006) [2023-12-27 16:33:12,098][20347] Updated weights for policy 0, policy_version 1574407 (0.0007) [2023-12-27 16:33:12,130][20381] Updated weights for policy 1, policy_version 1572725 (0.0006) [2023-12-27 16:33:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 805789696. Throughput: 0: 10219.3, 1: 10416.2. Samples: 805777264. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:12,363][19170] Avg episode reward: [(0, '931.670'), (1, '921.880')] [2023-12-27 16:33:12,776][20347] Updated weights for policy 0, policy_version 1574417 (0.0006) [2023-12-27 16:33:12,833][20347] Updated weights for policy 0, policy_version 1574427 (0.0011) [2023-12-27 16:33:12,834][20381] Updated weights for policy 1, policy_version 1572735 (0.0009) [2023-12-27 16:33:12,893][20347] Updated weights for policy 0, policy_version 1574437 (0.0010) [2023-12-27 16:33:12,894][20381] Updated weights for policy 1, policy_version 1572745 (0.0011) [2023-12-27 16:33:12,945][20347] Updated weights for policy 0, policy_version 1574447 (0.0010) [2023-12-27 16:33:12,950][20381] Updated weights for policy 1, policy_version 1572755 (0.0011) [2023-12-27 16:33:13,624][20381] Updated weights for policy 1, policy_version 1572765 (0.0011) [2023-12-27 16:33:13,627][20347] Updated weights for policy 0, policy_version 1574457 (0.0010) [2023-12-27 16:33:13,676][20347] Updated weights for policy 0, policy_version 1574467 (0.0010) [2023-12-27 16:33:13,677][20381] Updated weights for policy 1, policy_version 1572775 (0.0011) [2023-12-27 16:33:13,723][20381] Updated weights for policy 1, policy_version 1572785 (0.0010) [2023-12-27 16:33:13,724][20347] Updated weights for policy 0, policy_version 1574477 (0.0010) [2023-12-27 16:33:14,304][20381] Updated weights for policy 1, policy_version 1572795 (0.0009) [2023-12-27 16:33:14,357][20381] Updated weights for policy 1, policy_version 1572805 (0.0005) [2023-12-27 16:33:14,410][20381] Updated weights for policy 1, policy_version 1572815 (0.0005) [2023-12-27 16:33:14,419][20347] Updated weights for policy 0, policy_version 1574487 (0.0010) [2023-12-27 16:33:14,477][20347] Updated weights for policy 0, policy_version 1574497 (0.0010) [2023-12-27 16:33:14,533][20347] Updated weights for policy 0, policy_version 1574507 (0.0010) [2023-12-27 16:33:15,098][20381] Updated weights for policy 1, policy_version 1572825 (0.0008) [2023-12-27 16:33:15,161][20381] Updated weights for policy 1, policy_version 1572835 (0.0011) [2023-12-27 16:33:15,180][20347] Updated weights for policy 0, policy_version 1574517 (0.0009) [2023-12-27 16:33:15,218][20381] Updated weights for policy 1, policy_version 1572845 (0.0011) [2023-12-27 16:33:15,246][20347] Updated weights for policy 0, policy_version 1574527 (0.0006) [2023-12-27 16:33:15,272][20381] Updated weights for policy 1, policy_version 1572855 (0.0011) [2023-12-27 16:33:15,308][20347] Updated weights for policy 0, policy_version 1574537 (0.0005) [2023-12-27 16:33:15,885][20381] Updated weights for policy 1, policy_version 1572865 (0.0006) [2023-12-27 16:33:15,912][20347] Updated weights for policy 0, policy_version 1574547 (0.0006) [2023-12-27 16:33:15,944][20381] Updated weights for policy 1, policy_version 1572875 (0.0005) [2023-12-27 16:33:15,978][20347] Updated weights for policy 0, policy_version 1574557 (0.0008) [2023-12-27 16:33:15,993][20381] Updated weights for policy 1, policy_version 1572885 (0.0005) [2023-12-27 16:33:16,026][20347] Updated weights for policy 0, policy_version 1574567 (0.0007) [2023-12-27 16:33:16,597][20381] Updated weights for policy 1, policy_version 1572895 (0.0005) [2023-12-27 16:33:16,651][20381] Updated weights for policy 1, policy_version 1572905 (0.0005) [2023-12-27 16:33:16,704][20381] Updated weights for policy 1, policy_version 1572915 (0.0006) [2023-12-27 16:33:16,716][20347] Updated weights for policy 0, policy_version 1574577 (0.0008) [2023-12-27 16:33:16,780][20347] Updated weights for policy 0, policy_version 1574587 (0.0007) [2023-12-27 16:33:16,835][20347] Updated weights for policy 0, policy_version 1574597 (0.0006) [2023-12-27 16:33:16,887][20347] Updated weights for policy 0, policy_version 1574607 (0.0006) [2023-12-27 16:33:17,266][20381] Updated weights for policy 1, policy_version 1572925 (0.0006) [2023-12-27 16:33:17,322][20381] Updated weights for policy 1, policy_version 1572935 (0.0005) [2023-12-27 16:33:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 805888000. Throughput: 0: 10246.5, 1: 10414.3. Samples: 805902904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:17,363][19170] Avg episode reward: [(0, '970.780'), (1, '824.240')] [2023-12-27 16:33:17,370][20381] Updated weights for policy 1, policy_version 1572945 (0.0005) [2023-12-27 16:33:17,483][20347] Updated weights for policy 0, policy_version 1574617 (0.0006) [2023-12-27 16:33:17,531][20347] Updated weights for policy 0, policy_version 1574627 (0.0005) [2023-12-27 16:33:17,591][20347] Updated weights for policy 0, policy_version 1574637 (0.0005) [2023-12-27 16:33:17,954][20381] Updated weights for policy 1, policy_version 1572955 (0.0009) [2023-12-27 16:33:18,015][20381] Updated weights for policy 1, policy_version 1572965 (0.0009) [2023-12-27 16:33:18,075][20381] Updated weights for policy 1, policy_version 1572975 (0.0011) [2023-12-27 16:33:18,239][20347] Updated weights for policy 0, policy_version 1574647 (0.0005) [2023-12-27 16:33:18,296][20347] Updated weights for policy 0, policy_version 1574657 (0.0006) [2023-12-27 16:33:18,365][20347] Updated weights for policy 0, policy_version 1574667 (0.0011) [2023-12-27 16:33:18,795][20381] Updated weights for policy 1, policy_version 1572985 (0.0010) [2023-12-27 16:33:18,862][20381] Updated weights for policy 1, policy_version 1572995 (0.0007) [2023-12-27 16:33:18,932][20381] Updated weights for policy 1, policy_version 1573005 (0.0006) [2023-12-27 16:33:18,995][20381] Updated weights for policy 1, policy_version 1573015 (0.0007) [2023-12-27 16:33:19,020][20347] Updated weights for policy 0, policy_version 1574677 (0.0007) [2023-12-27 16:33:19,079][20347] Updated weights for policy 0, policy_version 1574687 (0.0006) [2023-12-27 16:33:19,133][20347] Updated weights for policy 0, policy_version 1574697 (0.0008) [2023-12-27 16:33:19,697][20381] Updated weights for policy 1, policy_version 1573025 (0.0007) [2023-12-27 16:33:19,765][20381] Updated weights for policy 1, policy_version 1573035 (0.0007) [2023-12-27 16:33:19,785][20347] Updated weights for policy 0, policy_version 1574707 (0.0008) [2023-12-27 16:33:19,827][20381] Updated weights for policy 1, policy_version 1573045 (0.0007) [2023-12-27 16:33:19,851][20347] Updated weights for policy 0, policy_version 1574717 (0.0008) [2023-12-27 16:33:19,915][20347] Updated weights for policy 0, policy_version 1574727 (0.0011) [2023-12-27 16:33:20,587][20381] Updated weights for policy 1, policy_version 1573055 (0.0009) [2023-12-27 16:33:20,648][20381] Updated weights for policy 1, policy_version 1573065 (0.0009) [2023-12-27 16:33:20,706][20381] Updated weights for policy 1, policy_version 1573075 (0.0009) [2023-12-27 16:33:20,724][20347] Updated weights for policy 0, policy_version 1574737 (0.0008) [2023-12-27 16:33:20,778][20347] Updated weights for policy 0, policy_version 1574747 (0.0009) [2023-12-27 16:33:20,828][20347] Updated weights for policy 0, policy_version 1574757 (0.0009) [2023-12-27 16:33:20,885][20347] Updated weights for policy 0, policy_version 1574767 (0.0009) [2023-12-27 16:33:21,476][20381] Updated weights for policy 1, policy_version 1573085 (0.0007) [2023-12-27 16:33:21,541][20381] Updated weights for policy 1, policy_version 1573095 (0.0006) [2023-12-27 16:33:21,597][20381] Updated weights for policy 1, policy_version 1573105 (0.0006) [2023-12-27 16:33:21,686][20347] Updated weights for policy 0, policy_version 1574777 (0.0009) [2023-12-27 16:33:21,747][20347] Updated weights for policy 0, policy_version 1574787 (0.0008) [2023-12-27 16:33:21,807][20347] Updated weights for policy 0, policy_version 1574797 (0.0007) [2023-12-27 16:33:22,218][20381] Updated weights for policy 1, policy_version 1573115 (0.0008) [2023-12-27 16:33:22,290][20381] Updated weights for policy 1, policy_version 1573125 (0.0008) [2023-12-27 16:33:22,352][20381] Updated weights for policy 1, policy_version 1573135 (0.0008) [2023-12-27 16:33:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 805986304. Throughput: 0: 10241.1, 1: 10386.9. Samples: 805965292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:22,363][19170] Avg episode reward: [(0, '970.750'), (1, '853.580')] [2023-12-27 16:33:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001574800_403226624.pth... [2023-12-27 16:33:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001573616_402923520.pth [2023-12-27 16:33:22,407][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001573144_402767872.pth... [2023-12-27 16:33:22,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001571928_402456576.pth [2023-12-27 16:33:22,486][20347] Updated weights for policy 0, policy_version 1574807 (0.0009) [2023-12-27 16:33:22,545][20347] Updated weights for policy 0, policy_version 1574817 (0.0008) [2023-12-27 16:33:22,608][20347] Updated weights for policy 0, policy_version 1574827 (0.0005) [2023-12-27 16:33:23,025][20381] Updated weights for policy 1, policy_version 1573145 (0.0007) [2023-12-27 16:33:23,081][20381] Updated weights for policy 1, policy_version 1573155 (0.0009) [2023-12-27 16:33:23,130][20381] Updated weights for policy 1, policy_version 1573165 (0.0007) [2023-12-27 16:33:23,162][20347] Updated weights for policy 0, policy_version 1574837 (0.0006) [2023-12-27 16:33:23,190][20381] Updated weights for policy 1, policy_version 1573175 (0.0008) [2023-12-27 16:33:23,220][20347] Updated weights for policy 0, policy_version 1574847 (0.0006) [2023-12-27 16:33:23,278][20347] Updated weights for policy 0, policy_version 1574857 (0.0006) [2023-12-27 16:33:23,825][20381] Updated weights for policy 1, policy_version 1573185 (0.0005) [2023-12-27 16:33:23,877][20381] Updated weights for policy 1, policy_version 1573195 (0.0005) [2023-12-27 16:33:23,920][20381] Updated weights for policy 1, policy_version 1573205 (0.0005) [2023-12-27 16:33:23,935][20347] Updated weights for policy 0, policy_version 1574867 (0.0005) [2023-12-27 16:33:23,997][20347] Updated weights for policy 0, policy_version 1574877 (0.0006) [2023-12-27 16:33:24,049][20347] Updated weights for policy 0, policy_version 1574887 (0.0008) [2023-12-27 16:33:24,585][20381] Updated weights for policy 1, policy_version 1573215 (0.0005) [2023-12-27 16:33:24,632][20381] Updated weights for policy 1, policy_version 1573225 (0.0005) [2023-12-27 16:33:24,696][20381] Updated weights for policy 1, policy_version 1573235 (0.0008) [2023-12-27 16:33:24,720][20347] Updated weights for policy 0, policy_version 1574897 (0.0007) [2023-12-27 16:33:24,782][20347] Updated weights for policy 0, policy_version 1574907 (0.0005) [2023-12-27 16:33:24,843][20347] Updated weights for policy 0, policy_version 1574917 (0.0006) [2023-12-27 16:33:24,894][20347] Updated weights for policy 0, policy_version 1574927 (0.0005) [2023-12-27 16:33:25,303][20381] Updated weights for policy 1, policy_version 1573245 (0.0007) [2023-12-27 16:33:25,363][20381] Updated weights for policy 1, policy_version 1573255 (0.0007) [2023-12-27 16:33:25,412][20347] Updated weights for policy 0, policy_version 1574937 (0.0005) [2023-12-27 16:33:25,423][20381] Updated weights for policy 1, policy_version 1573265 (0.0008) [2023-12-27 16:33:25,467][20347] Updated weights for policy 0, policy_version 1574947 (0.0005) [2023-12-27 16:33:25,518][20347] Updated weights for policy 0, policy_version 1574957 (0.0007) [2023-12-27 16:33:26,100][20381] Updated weights for policy 1, policy_version 1573275 (0.0007) [2023-12-27 16:33:26,162][20381] Updated weights for policy 1, policy_version 1573285 (0.0006) [2023-12-27 16:33:26,162][20347] Updated weights for policy 0, policy_version 1574967 (0.0006) [2023-12-27 16:33:26,222][20381] Updated weights for policy 1, policy_version 1573295 (0.0007) [2023-12-27 16:33:26,227][20347] Updated weights for policy 0, policy_version 1574977 (0.0006) [2023-12-27 16:33:26,287][20347] Updated weights for policy 0, policy_version 1574987 (0.0006) [2023-12-27 16:33:26,811][20381] Updated weights for policy 1, policy_version 1573305 (0.0007) [2023-12-27 16:33:26,868][20381] Updated weights for policy 1, policy_version 1573315 (0.0005) [2023-12-27 16:33:26,898][20347] Updated weights for policy 0, policy_version 1574997 (0.0005) [2023-12-27 16:33:26,931][20381] Updated weights for policy 1, policy_version 1573325 (0.0005) [2023-12-27 16:33:26,954][20347] Updated weights for policy 0, policy_version 1575007 (0.0005) [2023-12-27 16:33:26,991][20381] Updated weights for policy 1, policy_version 1573335 (0.0005) [2023-12-27 16:33:27,006][20347] Updated weights for policy 0, policy_version 1575017 (0.0006) [2023-12-27 16:33:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 806100992. Throughput: 0: 10265.3, 1: 10382.2. Samples: 806088232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:27,363][19170] Avg episode reward: [(0, '960.980'), (1, '814.480')] [2023-12-27 16:33:27,521][20381] Updated weights for policy 1, policy_version 1573345 (0.0005) [2023-12-27 16:33:27,577][20381] Updated weights for policy 1, policy_version 1573355 (0.0008) [2023-12-27 16:33:27,579][20347] Updated weights for policy 0, policy_version 1575027 (0.0009) [2023-12-27 16:33:27,627][20381] Updated weights for policy 1, policy_version 1573365 (0.0008) [2023-12-27 16:33:27,635][20347] Updated weights for policy 0, policy_version 1575037 (0.0005) [2023-12-27 16:33:27,690][20347] Updated weights for policy 0, policy_version 1575047 (0.0005) [2023-12-27 16:33:28,256][20347] Updated weights for policy 0, policy_version 1575057 (0.0005) [2023-12-27 16:33:28,308][20347] Updated weights for policy 0, policy_version 1575067 (0.0006) [2023-12-27 16:33:28,321][20381] Updated weights for policy 1, policy_version 1573375 (0.0008) [2023-12-27 16:33:28,371][20347] Updated weights for policy 0, policy_version 1575077 (0.0007) [2023-12-27 16:33:28,384][20381] Updated weights for policy 1, policy_version 1573385 (0.0007) [2023-12-27 16:33:28,432][20347] Updated weights for policy 0, policy_version 1575087 (0.0006) [2023-12-27 16:33:28,446][20381] Updated weights for policy 1, policy_version 1573395 (0.0006) [2023-12-27 16:33:29,029][20381] Updated weights for policy 1, policy_version 1573405 (0.0006) [2023-12-27 16:33:29,085][20381] Updated weights for policy 1, policy_version 1573415 (0.0005) [2023-12-27 16:33:29,135][20347] Updated weights for policy 0, policy_version 1575097 (0.0006) [2023-12-27 16:33:29,145][20381] Updated weights for policy 1, policy_version 1573425 (0.0005) [2023-12-27 16:33:29,183][20347] Updated weights for policy 0, policy_version 1575107 (0.0007) [2023-12-27 16:33:29,244][20347] Updated weights for policy 0, policy_version 1575117 (0.0009) [2023-12-27 16:33:29,787][20381] Updated weights for policy 1, policy_version 1573435 (0.0006) [2023-12-27 16:33:29,840][20381] Updated weights for policy 1, policy_version 1573445 (0.0008) [2023-12-27 16:33:29,882][20347] Updated weights for policy 0, policy_version 1575127 (0.0009) [2023-12-27 16:33:29,901][20381] Updated weights for policy 1, policy_version 1573455 (0.0007) [2023-12-27 16:33:29,940][20347] Updated weights for policy 0, policy_version 1575137 (0.0010) [2023-12-27 16:33:29,996][20347] Updated weights for policy 0, policy_version 1575147 (0.0010) [2023-12-27 16:33:30,591][20381] Updated weights for policy 1, policy_version 1573465 (0.0007) [2023-12-27 16:33:30,633][20347] Updated weights for policy 0, policy_version 1575157 (0.0011) [2023-12-27 16:33:30,637][20381] Updated weights for policy 1, policy_version 1573475 (0.0005) [2023-12-27 16:33:30,684][20381] Updated weights for policy 1, policy_version 1573485 (0.0005) [2023-12-27 16:33:30,685][20347] Updated weights for policy 0, policy_version 1575167 (0.0010) [2023-12-27 16:33:30,741][20381] Updated weights for policy 1, policy_version 1573495 (0.0005) [2023-12-27 16:33:30,742][20347] Updated weights for policy 0, policy_version 1575177 (0.0010) [2023-12-27 16:33:31,361][20381] Updated weights for policy 1, policy_version 1573505 (0.0008) [2023-12-27 16:33:31,426][20381] Updated weights for policy 1, policy_version 1573515 (0.0007) [2023-12-27 16:33:31,474][20347] Updated weights for policy 0, policy_version 1575187 (0.0009) [2023-12-27 16:33:31,486][20381] Updated weights for policy 1, policy_version 1573525 (0.0006) [2023-12-27 16:33:31,529][20347] Updated weights for policy 0, policy_version 1575197 (0.0006) [2023-12-27 16:33:31,590][20347] Updated weights for policy 0, policy_version 1575207 (0.0006) [2023-12-27 16:33:32,115][20381] Updated weights for policy 1, policy_version 1573535 (0.0007) [2023-12-27 16:33:32,169][20381] Updated weights for policy 1, policy_version 1573545 (0.0008) [2023-12-27 16:33:32,207][20347] Updated weights for policy 0, policy_version 1575217 (0.0007) [2023-12-27 16:33:32,230][20381] Updated weights for policy 1, policy_version 1573555 (0.0008) [2023-12-27 16:33:32,266][20347] Updated weights for policy 0, policy_version 1575227 (0.0006) [2023-12-27 16:33:32,320][20347] Updated weights for policy 0, policy_version 1575237 (0.0005) [2023-12-27 16:33:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 806207488. Throughput: 0: 10342.4, 1: 10460.2. Samples: 806218640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:32,363][19170] Avg episode reward: [(0, '990.230'), (1, '882.930')] [2023-12-27 16:33:32,390][20347] Updated weights for policy 0, policy_version 1575247 (0.0008) [2023-12-27 16:33:32,940][20381] Updated weights for policy 1, policy_version 1573565 (0.0009) [2023-12-27 16:33:32,986][20347] Updated weights for policy 0, policy_version 1575257 (0.0005) [2023-12-27 16:33:32,993][20381] Updated weights for policy 1, policy_version 1573575 (0.0009) [2023-12-27 16:33:33,043][20347] Updated weights for policy 0, policy_version 1575267 (0.0008) [2023-12-27 16:33:33,051][20381] Updated weights for policy 1, policy_version 1573585 (0.0007) [2023-12-27 16:33:33,096][20347] Updated weights for policy 0, policy_version 1575277 (0.0007) [2023-12-27 16:33:33,666][20381] Updated weights for policy 1, policy_version 1573595 (0.0007) [2023-12-27 16:33:33,706][20347] Updated weights for policy 0, policy_version 1575287 (0.0006) [2023-12-27 16:33:33,716][20381] Updated weights for policy 1, policy_version 1573605 (0.0007) [2023-12-27 16:33:33,762][20347] Updated weights for policy 0, policy_version 1575297 (0.0008) [2023-12-27 16:33:33,765][20381] Updated weights for policy 1, policy_version 1573615 (0.0005) [2023-12-27 16:33:33,819][20347] Updated weights for policy 0, policy_version 1575307 (0.0007) [2023-12-27 16:33:34,455][20381] Updated weights for policy 1, policy_version 1573625 (0.0007) [2023-12-27 16:33:34,518][20381] Updated weights for policy 1, policy_version 1573635 (0.0008) [2023-12-27 16:33:34,535][20347] Updated weights for policy 0, policy_version 1575317 (0.0008) [2023-12-27 16:33:34,581][20381] Updated weights for policy 1, policy_version 1573645 (0.0009) [2023-12-27 16:33:34,594][20347] Updated weights for policy 0, policy_version 1575327 (0.0008) [2023-12-27 16:33:34,645][20381] Updated weights for policy 1, policy_version 1573655 (0.0008) [2023-12-27 16:33:34,654][20347] Updated weights for policy 0, policy_version 1575337 (0.0007) [2023-12-27 16:33:35,334][20381] Updated weights for policy 1, policy_version 1573665 (0.0008) [2023-12-27 16:33:35,343][20347] Updated weights for policy 0, policy_version 1575347 (0.0007) [2023-12-27 16:33:35,386][20381] Updated weights for policy 1, policy_version 1573675 (0.0007) [2023-12-27 16:33:35,391][20347] Updated weights for policy 0, policy_version 1575357 (0.0006) [2023-12-27 16:33:35,437][20381] Updated weights for policy 1, policy_version 1573685 (0.0007) [2023-12-27 16:33:35,440][20347] Updated weights for policy 0, policy_version 1575367 (0.0005) [2023-12-27 16:33:36,039][20347] Updated weights for policy 0, policy_version 1575377 (0.0006) [2023-12-27 16:33:36,093][20347] Updated weights for policy 0, policy_version 1575387 (0.0006) [2023-12-27 16:33:36,154][20381] Updated weights for policy 1, policy_version 1573695 (0.0009) [2023-12-27 16:33:36,155][20347] Updated weights for policy 0, policy_version 1575397 (0.0007) [2023-12-27 16:33:36,214][20381] Updated weights for policy 1, policy_version 1573705 (0.0011) [2023-12-27 16:33:36,219][20347] Updated weights for policy 0, policy_version 1575407 (0.0006) [2023-12-27 16:33:36,278][20381] Updated weights for policy 1, policy_version 1573715 (0.0011) [2023-12-27 16:33:36,832][20347] Updated weights for policy 0, policy_version 1575417 (0.0006) [2023-12-27 16:33:36,893][20347] Updated weights for policy 0, policy_version 1575427 (0.0007) [2023-12-27 16:33:36,945][20381] Updated weights for policy 1, policy_version 1573725 (0.0008) [2023-12-27 16:33:36,948][20347] Updated weights for policy 0, policy_version 1575437 (0.0007) [2023-12-27 16:33:37,009][20381] Updated weights for policy 1, policy_version 1573735 (0.0008) [2023-12-27 16:33:37,067][20381] Updated weights for policy 1, policy_version 1573745 (0.0011) [2023-12-27 16:33:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.2, 300 sec: 20577.2). Total num frames: 806313984. Throughput: 0: 10362.4, 1: 10472.9. Samples: 806281596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:37,363][19170] Avg episode reward: [(0, '980.470'), (1, '902.360')] [2023-12-27 16:33:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001573752_402923520.pth... [2023-12-27 16:33:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001575440_403390464.pth... [2023-12-27 16:33:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001574224_403079168.pth [2023-12-27 16:33:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001572536_402612224.pth [2023-12-27 16:33:37,471][20347] Updated weights for policy 0, policy_version 1575447 (0.0008) [2023-12-27 16:33:37,540][20347] Updated weights for policy 0, policy_version 1575457 (0.0008) [2023-12-27 16:33:37,604][20347] Updated weights for policy 0, policy_version 1575467 (0.0008) [2023-12-27 16:33:37,655][20381] Updated weights for policy 1, policy_version 1573755 (0.0009) [2023-12-27 16:33:37,712][20381] Updated weights for policy 1, policy_version 1573765 (0.0006) [2023-12-27 16:33:37,768][20381] Updated weights for policy 1, policy_version 1573775 (0.0005) [2023-12-27 16:33:38,187][20347] Updated weights for policy 0, policy_version 1575477 (0.0007) [2023-12-27 16:33:38,243][20347] Updated weights for policy 0, policy_version 1575487 (0.0006) [2023-12-27 16:33:38,292][20347] Updated weights for policy 0, policy_version 1575497 (0.0005) [2023-12-27 16:33:38,365][20381] Updated weights for policy 1, policy_version 1573785 (0.0007) [2023-12-27 16:33:38,414][20381] Updated weights for policy 1, policy_version 1573795 (0.0011) [2023-12-27 16:33:38,477][20381] Updated weights for policy 1, policy_version 1573805 (0.0011) [2023-12-27 16:33:38,542][20381] Updated weights for policy 1, policy_version 1573815 (0.0011) [2023-12-27 16:33:38,938][20347] Updated weights for policy 0, policy_version 1575507 (0.0008) [2023-12-27 16:33:38,991][20347] Updated weights for policy 0, policy_version 1575517 (0.0010) [2023-12-27 16:33:39,038][20347] Updated weights for policy 0, policy_version 1575527 (0.0010) [2023-12-27 16:33:39,228][20381] Updated weights for policy 1, policy_version 1573825 (0.0009) [2023-12-27 16:33:39,289][20381] Updated weights for policy 1, policy_version 1573835 (0.0006) [2023-12-27 16:33:39,349][20381] Updated weights for policy 1, policy_version 1573845 (0.0008) [2023-12-27 16:33:39,789][20347] Updated weights for policy 0, policy_version 1575537 (0.0010) [2023-12-27 16:33:39,847][20347] Updated weights for policy 0, policy_version 1575547 (0.0009) [2023-12-27 16:33:39,909][20347] Updated weights for policy 0, policy_version 1575557 (0.0008) [2023-12-27 16:33:39,974][20347] Updated weights for policy 0, policy_version 1575567 (0.0009) [2023-12-27 16:33:40,111][20381] Updated weights for policy 1, policy_version 1573855 (0.0010) [2023-12-27 16:33:40,166][20381] Updated weights for policy 1, policy_version 1573865 (0.0010) [2023-12-27 16:33:40,219][20381] Updated weights for policy 1, policy_version 1573875 (0.0011) [2023-12-27 16:33:40,680][20347] Updated weights for policy 0, policy_version 1575577 (0.0007) [2023-12-27 16:33:40,739][20347] Updated weights for policy 0, policy_version 1575587 (0.0007) [2023-12-27 16:33:40,791][20347] Updated weights for policy 0, policy_version 1575597 (0.0009) [2023-12-27 16:33:40,925][20381] Updated weights for policy 1, policy_version 1573885 (0.0010) [2023-12-27 16:33:40,987][20381] Updated weights for policy 1, policy_version 1573895 (0.0009) [2023-12-27 16:33:41,046][20381] Updated weights for policy 1, policy_version 1573905 (0.0008) [2023-12-27 16:33:41,551][20347] Updated weights for policy 0, policy_version 1575607 (0.0007) [2023-12-27 16:33:41,613][20347] Updated weights for policy 0, policy_version 1575617 (0.0008) [2023-12-27 16:33:41,671][20347] Updated weights for policy 0, policy_version 1575627 (0.0009) [2023-12-27 16:33:41,855][20381] Updated weights for policy 1, policy_version 1573915 (0.0009) [2023-12-27 16:33:41,911][20381] Updated weights for policy 1, policy_version 1573925 (0.0008) [2023-12-27 16:33:41,966][20381] Updated weights for policy 1, policy_version 1573935 (0.0008) [2023-12-27 16:33:42,354][20347] Updated weights for policy 0, policy_version 1575637 (0.0009) [2023-12-27 16:33:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 806412288. Throughput: 0: 10419.0, 1: 10417.7. Samples: 806406580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:42,363][19170] Avg episode reward: [(0, '960.890'), (1, '882.990')] [2023-12-27 16:33:42,418][20347] Updated weights for policy 0, policy_version 1575647 (0.0007) [2023-12-27 16:33:42,477][20347] Updated weights for policy 0, policy_version 1575657 (0.0008) [2023-12-27 16:33:42,688][20381] Updated weights for policy 1, policy_version 1573945 (0.0008) [2023-12-27 16:33:42,748][20381] Updated weights for policy 1, policy_version 1573955 (0.0008) [2023-12-27 16:33:42,804][20381] Updated weights for policy 1, policy_version 1573965 (0.0008) [2023-12-27 16:33:42,863][20381] Updated weights for policy 1, policy_version 1573975 (0.0008) [2023-12-27 16:33:43,154][20347] Updated weights for policy 0, policy_version 1575667 (0.0007) [2023-12-27 16:33:43,207][20347] Updated weights for policy 0, policy_version 1575677 (0.0005) [2023-12-27 16:33:43,255][20347] Updated weights for policy 0, policy_version 1575687 (0.0005) [2023-12-27 16:33:43,499][20381] Updated weights for policy 1, policy_version 1573985 (0.0005) [2023-12-27 16:33:43,550][20381] Updated weights for policy 1, policy_version 1573995 (0.0008) [2023-12-27 16:33:43,596][20381] Updated weights for policy 1, policy_version 1574005 (0.0009) [2023-12-27 16:33:43,782][20347] Updated weights for policy 0, policy_version 1575697 (0.0006) [2023-12-27 16:33:43,838][20347] Updated weights for policy 0, policy_version 1575707 (0.0008) [2023-12-27 16:33:43,890][20347] Updated weights for policy 0, policy_version 1575717 (0.0007) [2023-12-27 16:33:43,936][20347] Updated weights for policy 0, policy_version 1575727 (0.0008) [2023-12-27 16:33:44,227][20381] Updated weights for policy 1, policy_version 1574015 (0.0009) [2023-12-27 16:33:44,285][20381] Updated weights for policy 1, policy_version 1574025 (0.0010) [2023-12-27 16:33:44,346][20381] Updated weights for policy 1, policy_version 1574035 (0.0011) [2023-12-27 16:33:44,610][20347] Updated weights for policy 0, policy_version 1575737 (0.0006) [2023-12-27 16:33:44,670][20347] Updated weights for policy 0, policy_version 1575747 (0.0006) [2023-12-27 16:33:44,727][20347] Updated weights for policy 0, policy_version 1575757 (0.0008) [2023-12-27 16:33:44,934][20381] Updated weights for policy 1, policy_version 1574045 (0.0006) [2023-12-27 16:33:44,990][20381] Updated weights for policy 1, policy_version 1574055 (0.0006) [2023-12-27 16:33:45,043][20381] Updated weights for policy 1, policy_version 1574065 (0.0007) [2023-12-27 16:33:45,438][20347] Updated weights for policy 0, policy_version 1575767 (0.0008) [2023-12-27 16:33:45,484][20347] Updated weights for policy 0, policy_version 1575777 (0.0008) [2023-12-27 16:33:45,529][20347] Updated weights for policy 0, policy_version 1575787 (0.0007) [2023-12-27 16:33:45,704][20381] Updated weights for policy 1, policy_version 1574075 (0.0007) [2023-12-27 16:33:45,751][20381] Updated weights for policy 1, policy_version 1574085 (0.0006) [2023-12-27 16:33:45,803][20381] Updated weights for policy 1, policy_version 1574095 (0.0007) [2023-12-27 16:33:46,253][20347] Updated weights for policy 0, policy_version 1575797 (0.0008) [2023-12-27 16:33:46,301][20347] Updated weights for policy 0, policy_version 1575807 (0.0007) [2023-12-27 16:33:46,356][20347] Updated weights for policy 0, policy_version 1575817 (0.0007) [2023-12-27 16:33:46,491][20381] Updated weights for policy 1, policy_version 1574105 (0.0008) [2023-12-27 16:33:46,546][20381] Updated weights for policy 1, policy_version 1574115 (0.0008) [2023-12-27 16:33:46,597][20381] Updated weights for policy 1, policy_version 1574125 (0.0007) [2023-12-27 16:33:46,649][20381] Updated weights for policy 1, policy_version 1574135 (0.0008) [2023-12-27 16:33:46,988][20347] Updated weights for policy 0, policy_version 1575827 (0.0006) [2023-12-27 16:33:47,049][20347] Updated weights for policy 0, policy_version 1575837 (0.0007) [2023-12-27 16:33:47,107][20347] Updated weights for policy 0, policy_version 1575847 (0.0007) [2023-12-27 16:33:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 806518784. Throughput: 0: 10452.5, 1: 10427.4. Samples: 806531432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:47,363][19170] Avg episode reward: [(0, '970.620'), (1, '853.490')] [2023-12-27 16:33:47,365][20381] Updated weights for policy 1, policy_version 1574145 (0.0008) [2023-12-27 16:33:47,413][20381] Updated weights for policy 1, policy_version 1574155 (0.0007) [2023-12-27 16:33:47,459][20381] Updated weights for policy 1, policy_version 1574165 (0.0008) [2023-12-27 16:33:47,730][20347] Updated weights for policy 0, policy_version 1575857 (0.0007) [2023-12-27 16:33:47,778][20347] Updated weights for policy 0, policy_version 1575867 (0.0007) [2023-12-27 16:33:47,834][20347] Updated weights for policy 0, policy_version 1575877 (0.0007) [2023-12-27 16:33:47,886][20347] Updated weights for policy 0, policy_version 1575887 (0.0008) [2023-12-27 16:33:48,131][20381] Updated weights for policy 1, policy_version 1574175 (0.0006) [2023-12-27 16:33:48,186][20381] Updated weights for policy 1, policy_version 1574185 (0.0005) [2023-12-27 16:33:48,240][20381] Updated weights for policy 1, policy_version 1574195 (0.0007) [2023-12-27 16:33:48,596][20347] Updated weights for policy 0, policy_version 1575897 (0.0007) [2023-12-27 16:33:48,655][20347] Updated weights for policy 0, policy_version 1575907 (0.0007) [2023-12-27 16:33:48,720][20347] Updated weights for policy 0, policy_version 1575917 (0.0006) [2023-12-27 16:33:48,904][20381] Updated weights for policy 1, policy_version 1574205 (0.0007) [2023-12-27 16:33:48,954][20381] Updated weights for policy 1, policy_version 1574215 (0.0008) [2023-12-27 16:33:49,009][20381] Updated weights for policy 1, policy_version 1574225 (0.0007) [2023-12-27 16:33:49,383][20347] Updated weights for policy 0, policy_version 1575927 (0.0007) [2023-12-27 16:33:49,444][20347] Updated weights for policy 0, policy_version 1575937 (0.0009) [2023-12-27 16:33:49,502][20347] Updated weights for policy 0, policy_version 1575947 (0.0008) [2023-12-27 16:33:49,716][20381] Updated weights for policy 1, policy_version 1574235 (0.0007) [2023-12-27 16:33:49,776][20381] Updated weights for policy 1, policy_version 1574245 (0.0008) [2023-12-27 16:33:49,841][20381] Updated weights for policy 1, policy_version 1574255 (0.0007) [2023-12-27 16:33:50,221][20347] Updated weights for policy 0, policy_version 1575957 (0.0007) [2023-12-27 16:33:50,275][20347] Updated weights for policy 0, policy_version 1575967 (0.0007) [2023-12-27 16:33:50,324][20347] Updated weights for policy 0, policy_version 1575977 (0.0007) [2023-12-27 16:33:50,536][20381] Updated weights for policy 1, policy_version 1574265 (0.0007) [2023-12-27 16:33:50,591][20381] Updated weights for policy 1, policy_version 1574275 (0.0009) [2023-12-27 16:33:50,638][20381] Updated weights for policy 1, policy_version 1574285 (0.0006) [2023-12-27 16:33:50,695][20381] Updated weights for policy 1, policy_version 1574295 (0.0008) [2023-12-27 16:33:50,941][20347] Updated weights for policy 0, policy_version 1575987 (0.0007) [2023-12-27 16:33:50,987][20347] Updated weights for policy 0, policy_version 1575997 (0.0005) [2023-12-27 16:33:51,051][20347] Updated weights for policy 0, policy_version 1576007 (0.0006) [2023-12-27 16:33:51,363][20381] Updated weights for policy 1, policy_version 1574305 (0.0008) [2023-12-27 16:33:51,428][20381] Updated weights for policy 1, policy_version 1574315 (0.0007) [2023-12-27 16:33:51,487][20381] Updated weights for policy 1, policy_version 1574325 (0.0008) [2023-12-27 16:33:51,740][20347] Updated weights for policy 0, policy_version 1576017 (0.0007) [2023-12-27 16:33:51,800][20347] Updated weights for policy 0, policy_version 1576027 (0.0007) [2023-12-27 16:33:51,860][20347] Updated weights for policy 0, policy_version 1576037 (0.0008) [2023-12-27 16:33:51,913][20347] Updated weights for policy 0, policy_version 1576047 (0.0007) [2023-12-27 16:33:52,212][20381] Updated weights for policy 1, policy_version 1574335 (0.0007) [2023-12-27 16:33:52,278][20381] Updated weights for policy 1, policy_version 1574345 (0.0008) [2023-12-27 16:33:52,342][20381] Updated weights for policy 1, policy_version 1574355 (0.0008) [2023-12-27 16:33:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 806617088. Throughput: 0: 10464.3, 1: 10366.9. Samples: 806591912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:52,363][19170] Avg episode reward: [(0, '980.470'), (1, '863.110')] [2023-12-27 16:33:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001576048_403546112.pth... [2023-12-27 16:33:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001574800_403226624.pth [2023-12-27 16:33:52,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001574360_403079168.pth... [2023-12-27 16:33:52,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001573144_402767872.pth [2023-12-27 16:33:52,639][20347] Updated weights for policy 0, policy_version 1576057 (0.0010) [2023-12-27 16:33:52,713][20347] Updated weights for policy 0, policy_version 1576067 (0.0011) [2023-12-27 16:33:52,775][20347] Updated weights for policy 0, policy_version 1576077 (0.0011) [2023-12-27 16:33:52,933][20381] Updated weights for policy 1, policy_version 1574365 (0.0009) [2023-12-27 16:33:52,991][20381] Updated weights for policy 1, policy_version 1574375 (0.0010) [2023-12-27 16:33:53,042][20381] Updated weights for policy 1, policy_version 1574385 (0.0010) [2023-12-27 16:33:53,409][20347] Updated weights for policy 0, policy_version 1576087 (0.0010) [2023-12-27 16:33:53,465][20347] Updated weights for policy 0, policy_version 1576097 (0.0010) [2023-12-27 16:33:53,522][20347] Updated weights for policy 0, policy_version 1576107 (0.0010) [2023-12-27 16:33:53,715][20381] Updated weights for policy 1, policy_version 1574395 (0.0010) [2023-12-27 16:33:53,762][20381] Updated weights for policy 1, policy_version 1574405 (0.0007) [2023-12-27 16:33:53,819][20381] Updated weights for policy 1, policy_version 1574415 (0.0007) [2023-12-27 16:33:54,153][20347] Updated weights for policy 0, policy_version 1576117 (0.0008) [2023-12-27 16:33:54,208][20347] Updated weights for policy 0, policy_version 1576127 (0.0010) [2023-12-27 16:33:54,264][20347] Updated weights for policy 0, policy_version 1576137 (0.0011) [2023-12-27 16:33:54,471][20381] Updated weights for policy 1, policy_version 1574425 (0.0008) [2023-12-27 16:33:54,531][20381] Updated weights for policy 1, policy_version 1574435 (0.0007) [2023-12-27 16:33:54,582][20381] Updated weights for policy 1, policy_version 1574445 (0.0008) [2023-12-27 16:33:54,638][20381] Updated weights for policy 1, policy_version 1574455 (0.0007) [2023-12-27 16:33:54,900][20347] Updated weights for policy 0, policy_version 1576147 (0.0009) [2023-12-27 16:33:54,955][20347] Updated weights for policy 0, policy_version 1576157 (0.0008) [2023-12-27 16:33:55,014][20347] Updated weights for policy 0, policy_version 1576167 (0.0006) [2023-12-27 16:33:55,210][20381] Updated weights for policy 1, policy_version 1574465 (0.0010) [2023-12-27 16:33:55,267][20381] Updated weights for policy 1, policy_version 1574475 (0.0006) [2023-12-27 16:33:55,317][20381] Updated weights for policy 1, policy_version 1574485 (0.0005) [2023-12-27 16:33:55,678][20347] Updated weights for policy 0, policy_version 1576177 (0.0006) [2023-12-27 16:33:55,743][20347] Updated weights for policy 0, policy_version 1576187 (0.0011) [2023-12-27 16:33:55,805][20347] Updated weights for policy 0, policy_version 1576197 (0.0007) [2023-12-27 16:33:55,865][20381] Updated weights for policy 1, policy_version 1574495 (0.0009) [2023-12-27 16:33:55,870][20347] Updated weights for policy 0, policy_version 1576207 (0.0007) [2023-12-27 16:33:55,920][20381] Updated weights for policy 1, policy_version 1574505 (0.0011) [2023-12-27 16:33:55,983][20381] Updated weights for policy 1, policy_version 1574515 (0.0010) [2023-12-27 16:33:56,571][20347] Updated weights for policy 0, policy_version 1576217 (0.0007) [2023-12-27 16:33:56,615][20347] Updated weights for policy 0, policy_version 1576227 (0.0007) [2023-12-27 16:33:56,658][20347] Updated weights for policy 0, policy_version 1576237 (0.0007) [2023-12-27 16:33:56,704][20381] Updated weights for policy 1, policy_version 1574525 (0.0011) [2023-12-27 16:33:56,757][20381] Updated weights for policy 1, policy_version 1574535 (0.0010) [2023-12-27 16:33:56,806][20381] Updated weights for policy 1, policy_version 1574545 (0.0008) [2023-12-27 16:33:57,343][20347] Updated weights for policy 0, policy_version 1576247 (0.0007) [2023-12-27 16:33:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 806723584. Throughput: 0: 10477.3, 1: 10442.5. Samples: 806718652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:33:57,363][19170] Avg episode reward: [(0, '990.270'), (1, '863.330')] [2023-12-27 16:33:57,392][20347] Updated weights for policy 0, policy_version 1576257 (0.0007) [2023-12-27 16:33:57,447][20347] Updated weights for policy 0, policy_version 1576267 (0.0006) [2023-12-27 16:33:57,488][20381] Updated weights for policy 1, policy_version 1574555 (0.0008) [2023-12-27 16:33:57,535][20381] Updated weights for policy 1, policy_version 1574565 (0.0005) [2023-12-27 16:33:57,583][20381] Updated weights for policy 1, policy_version 1574575 (0.0005) [2023-12-27 16:33:58,060][20347] Updated weights for policy 0, policy_version 1576277 (0.0008) [2023-12-27 16:33:58,117][20347] Updated weights for policy 0, policy_version 1576287 (0.0010) [2023-12-27 16:33:58,167][20347] Updated weights for policy 0, policy_version 1576297 (0.0010) [2023-12-27 16:33:58,168][20381] Updated weights for policy 1, policy_version 1574585 (0.0005) [2023-12-27 16:33:58,229][20381] Updated weights for policy 1, policy_version 1574595 (0.0007) [2023-12-27 16:33:58,285][20381] Updated weights for policy 1, policy_version 1574605 (0.0008) [2023-12-27 16:33:58,351][20381] Updated weights for policy 1, policy_version 1574615 (0.0008) [2023-12-27 16:33:58,995][20347] Updated weights for policy 0, policy_version 1576307 (0.0009) [2023-12-27 16:33:59,056][20347] Updated weights for policy 0, policy_version 1576317 (0.0008) [2023-12-27 16:33:59,089][20381] Updated weights for policy 1, policy_version 1574625 (0.0007) [2023-12-27 16:33:59,114][20347] Updated weights for policy 0, policy_version 1576327 (0.0006) [2023-12-27 16:33:59,144][20381] Updated weights for policy 1, policy_version 1574635 (0.0006) [2023-12-27 16:33:59,203][20381] Updated weights for policy 1, policy_version 1574645 (0.0006) [2023-12-27 16:33:59,868][20381] Updated weights for policy 1, policy_version 1574655 (0.0008) [2023-12-27 16:33:59,873][20347] Updated weights for policy 0, policy_version 1576337 (0.0007) [2023-12-27 16:33:59,913][20381] Updated weights for policy 1, policy_version 1574665 (0.0007) [2023-12-27 16:33:59,940][20347] Updated weights for policy 0, policy_version 1576347 (0.0007) [2023-12-27 16:33:59,982][20381] Updated weights for policy 1, policy_version 1574675 (0.0008) [2023-12-27 16:34:00,007][20347] Updated weights for policy 0, policy_version 1576357 (0.0007) [2023-12-27 16:34:00,065][20347] Updated weights for policy 0, policy_version 1576367 (0.0008) [2023-12-27 16:34:00,678][20381] Updated weights for policy 1, policy_version 1574685 (0.0007) [2023-12-27 16:34:00,729][20347] Updated weights for policy 0, policy_version 1576377 (0.0006) [2023-12-27 16:34:00,731][20381] Updated weights for policy 1, policy_version 1574695 (0.0007) [2023-12-27 16:34:00,775][20381] Updated weights for policy 1, policy_version 1574705 (0.0008) [2023-12-27 16:34:00,782][20347] Updated weights for policy 0, policy_version 1576387 (0.0008) [2023-12-27 16:34:00,830][20347] Updated weights for policy 0, policy_version 1576397 (0.0010) [2023-12-27 16:34:01,463][20381] Updated weights for policy 1, policy_version 1574715 (0.0006) [2023-12-27 16:34:01,502][20347] Updated weights for policy 0, policy_version 1576407 (0.0009) [2023-12-27 16:34:01,520][20381] Updated weights for policy 1, policy_version 1574725 (0.0006) [2023-12-27 16:34:01,564][20347] Updated weights for policy 0, policy_version 1576417 (0.0009) [2023-12-27 16:34:01,581][20381] Updated weights for policy 1, policy_version 1574735 (0.0006) [2023-12-27 16:34:01,630][20347] Updated weights for policy 0, policy_version 1576427 (0.0008) [2023-12-27 16:34:02,272][20381] Updated weights for policy 1, policy_version 1574745 (0.0006) [2023-12-27 16:34:02,276][20347] Updated weights for policy 0, policy_version 1576437 (0.0007) [2023-12-27 16:34:02,331][20381] Updated weights for policy 1, policy_version 1574755 (0.0005) [2023-12-27 16:34:02,335][20347] Updated weights for policy 0, policy_version 1576447 (0.0006) [2023-12-27 16:34:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 806821888. Throughput: 0: 10444.4, 1: 10381.4. Samples: 806840068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:34:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '834.210')] [2023-12-27 16:34:02,394][20381] Updated weights for policy 1, policy_version 1574765 (0.0007) [2023-12-27 16:34:02,397][20347] Updated weights for policy 0, policy_version 1576457 (0.0007) [2023-12-27 16:34:02,457][20381] Updated weights for policy 1, policy_version 1574775 (0.0006) [2023-12-27 16:34:03,000][20347] Updated weights for policy 0, policy_version 1576467 (0.0008) [2023-12-27 16:34:03,052][20347] Updated weights for policy 0, policy_version 1576477 (0.0010) [2023-12-27 16:34:03,072][20381] Updated weights for policy 1, policy_version 1574785 (0.0009) [2023-12-27 16:34:03,104][20347] Updated weights for policy 0, policy_version 1576487 (0.0010) [2023-12-27 16:34:03,124][20381] Updated weights for policy 1, policy_version 1574795 (0.0008) [2023-12-27 16:34:03,187][20381] Updated weights for policy 1, policy_version 1574805 (0.0006) [2023-12-27 16:34:03,774][20347] Updated weights for policy 0, policy_version 1576497 (0.0010) [2023-12-27 16:34:03,786][20381] Updated weights for policy 1, policy_version 1574815 (0.0007) [2023-12-27 16:34:03,834][20347] Updated weights for policy 0, policy_version 1576507 (0.0005) [2023-12-27 16:34:03,835][20381] Updated weights for policy 1, policy_version 1574825 (0.0007) [2023-12-27 16:34:03,894][20381] Updated weights for policy 1, policy_version 1574835 (0.0007) [2023-12-27 16:34:03,895][20347] Updated weights for policy 0, policy_version 1576517 (0.0007) [2023-12-27 16:34:03,950][20347] Updated weights for policy 0, policy_version 1576527 (0.0006) [2023-12-27 16:34:04,608][20347] Updated weights for policy 0, policy_version 1576537 (0.0007) [2023-12-27 16:34:04,650][20381] Updated weights for policy 1, policy_version 1574845 (0.0009) [2023-12-27 16:34:04,658][20347] Updated weights for policy 0, policy_version 1576547 (0.0006) [2023-12-27 16:34:04,695][20381] Updated weights for policy 1, policy_version 1574855 (0.0010) [2023-12-27 16:34:04,702][20347] Updated weights for policy 0, policy_version 1576557 (0.0005) [2023-12-27 16:34:04,741][20381] Updated weights for policy 1, policy_version 1574865 (0.0010) [2023-12-27 16:34:05,279][20347] Updated weights for policy 0, policy_version 1576567 (0.0008) [2023-12-27 16:34:05,334][20347] Updated weights for policy 0, policy_version 1576577 (0.0008) [2023-12-27 16:34:05,388][20347] Updated weights for policy 0, policy_version 1576587 (0.0008) [2023-12-27 16:34:05,456][20381] Updated weights for policy 1, policy_version 1574875 (0.0009) [2023-12-27 16:34:05,511][20381] Updated weights for policy 1, policy_version 1574885 (0.0005) [2023-12-27 16:34:05,567][20381] Updated weights for policy 1, policy_version 1574895 (0.0005) [2023-12-27 16:34:06,076][20381] Updated weights for policy 1, policy_version 1574905 (0.0005) [2023-12-27 16:34:06,085][20347] Updated weights for policy 0, policy_version 1576597 (0.0008) [2023-12-27 16:34:06,150][20381] Updated weights for policy 1, policy_version 1574915 (0.0008) [2023-12-27 16:34:06,152][20347] Updated weights for policy 0, policy_version 1576607 (0.0008) [2023-12-27 16:34:06,210][20381] Updated weights for policy 1, policy_version 1574925 (0.0007) [2023-12-27 16:34:06,212][20347] Updated weights for policy 0, policy_version 1576617 (0.0007) [2023-12-27 16:34:06,271][20381] Updated weights for policy 1, policy_version 1574935 (0.0006) [2023-12-27 16:34:06,837][20347] Updated weights for policy 0, policy_version 1576627 (0.0006) [2023-12-27 16:34:06,896][20347] Updated weights for policy 0, policy_version 1576637 (0.0005) [2023-12-27 16:34:06,952][20381] Updated weights for policy 1, policy_version 1574945 (0.0006) [2023-12-27 16:34:06,959][20347] Updated weights for policy 0, policy_version 1576647 (0.0005) [2023-12-27 16:34:07,005][20381] Updated weights for policy 1, policy_version 1574955 (0.0008) [2023-12-27 16:34:07,063][20381] Updated weights for policy 1, policy_version 1574965 (0.0006) [2023-12-27 16:34:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 806936576. Throughput: 0: 10457.6, 1: 10391.7. Samples: 806903512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:34:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '833.920')] [2023-12-27 16:34:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001574968_403234816.pth... [2023-12-27 16:34:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001576656_403701760.pth... [2023-12-27 16:34:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001573752_402923520.pth [2023-12-27 16:34:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001575440_403390464.pth [2023-12-27 16:34:07,570][20347] Updated weights for policy 0, policy_version 1576657 (0.0006) [2023-12-27 16:34:07,634][20347] Updated weights for policy 0, policy_version 1576667 (0.0010) [2023-12-27 16:34:07,695][20347] Updated weights for policy 0, policy_version 1576677 (0.0010) [2023-12-27 16:34:07,715][20381] Updated weights for policy 1, policy_version 1574975 (0.0007) [2023-12-27 16:34:07,752][20347] Updated weights for policy 0, policy_version 1576687 (0.0006) [2023-12-27 16:34:07,773][20381] Updated weights for policy 1, policy_version 1574985 (0.0009) [2023-12-27 16:34:07,834][20381] Updated weights for policy 1, policy_version 1574995 (0.0010) [2023-12-27 16:34:08,306][20347] Updated weights for policy 0, policy_version 1576697 (0.0010) [2023-12-27 16:34:08,371][20347] Updated weights for policy 0, policy_version 1576707 (0.0010) [2023-12-27 16:34:08,436][20347] Updated weights for policy 0, policy_version 1576717 (0.0011) [2023-12-27 16:34:08,480][20381] Updated weights for policy 1, policy_version 1575005 (0.0009) [2023-12-27 16:34:08,546][20381] Updated weights for policy 1, policy_version 1575015 (0.0011) [2023-12-27 16:34:08,612][20381] Updated weights for policy 1, policy_version 1575025 (0.0011) [2023-12-27 16:34:09,099][20347] Updated weights for policy 0, policy_version 1576727 (0.0010) [2023-12-27 16:34:09,163][20347] Updated weights for policy 0, policy_version 1576737 (0.0011) [2023-12-27 16:34:09,230][20347] Updated weights for policy 0, policy_version 1576747 (0.0011) [2023-12-27 16:34:09,310][20381] Updated weights for policy 1, policy_version 1575035 (0.0010) [2023-12-27 16:34:09,372][20381] Updated weights for policy 1, policy_version 1575045 (0.0008) [2023-12-27 16:34:09,439][20381] Updated weights for policy 1, policy_version 1575055 (0.0008) [2023-12-27 16:34:09,941][20347] Updated weights for policy 0, policy_version 1576757 (0.0009) [2023-12-27 16:34:09,998][20347] Updated weights for policy 0, policy_version 1576767 (0.0008) [2023-12-27 16:34:10,061][20347] Updated weights for policy 0, policy_version 1576777 (0.0007) [2023-12-27 16:34:10,164][20381] Updated weights for policy 1, policy_version 1575065 (0.0008) [2023-12-27 16:34:10,217][20381] Updated weights for policy 1, policy_version 1575075 (0.0005) [2023-12-27 16:34:10,277][20381] Updated weights for policy 1, policy_version 1575085 (0.0008) [2023-12-27 16:34:10,338][20381] Updated weights for policy 1, policy_version 1575095 (0.0007) [2023-12-27 16:34:10,782][20347] Updated weights for policy 0, policy_version 1576787 (0.0007) [2023-12-27 16:34:10,838][20347] Updated weights for policy 0, policy_version 1576797 (0.0007) [2023-12-27 16:34:10,885][20347] Updated weights for policy 0, policy_version 1576807 (0.0009) [2023-12-27 16:34:10,980][20381] Updated weights for policy 1, policy_version 1575105 (0.0006) [2023-12-27 16:34:11,049][20381] Updated weights for policy 1, policy_version 1575115 (0.0007) [2023-12-27 16:34:11,117][20381] Updated weights for policy 1, policy_version 1575125 (0.0011) [2023-12-27 16:34:11,603][20347] Updated weights for policy 0, policy_version 1576817 (0.0006) [2023-12-27 16:34:11,669][20347] Updated weights for policy 0, policy_version 1576827 (0.0009) [2023-12-27 16:34:11,735][20347] Updated weights for policy 0, policy_version 1576837 (0.0011) [2023-12-27 16:34:11,794][20347] Updated weights for policy 0, policy_version 1576847 (0.0008) [2023-12-27 16:34:11,852][20381] Updated weights for policy 1, policy_version 1575135 (0.0010) [2023-12-27 16:34:11,908][20381] Updated weights for policy 1, policy_version 1575145 (0.0011) [2023-12-27 16:34:11,963][20381] Updated weights for policy 1, policy_version 1575155 (0.0007) [2023-12-27 16:34:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 807034880. Throughput: 0: 10480.8, 1: 10425.1. Samples: 807029000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:34:12,363][19170] Avg episode reward: [(0, '980.440'), (1, '853.330')] [2023-12-27 16:34:12,569][20347] Updated weights for policy 0, policy_version 1576857 (0.0010) [2023-12-27 16:34:12,608][20381] Updated weights for policy 1, policy_version 1575165 (0.0009) [2023-12-27 16:34:12,622][20347] Updated weights for policy 0, policy_version 1576867 (0.0010) [2023-12-27 16:34:12,668][20381] Updated weights for policy 1, policy_version 1575175 (0.0007) [2023-12-27 16:34:12,680][20347] Updated weights for policy 0, policy_version 1576877 (0.0010) [2023-12-27 16:34:12,732][20381] Updated weights for policy 1, policy_version 1575185 (0.0007) [2023-12-27 16:34:13,348][20347] Updated weights for policy 0, policy_version 1576887 (0.0007) [2023-12-27 16:34:13,402][20347] Updated weights for policy 0, policy_version 1576897 (0.0007) [2023-12-27 16:34:13,405][20381] Updated weights for policy 1, policy_version 1575195 (0.0010) [2023-12-27 16:34:13,451][20347] Updated weights for policy 0, policy_version 1576907 (0.0007) [2023-12-27 16:34:13,459][20381] Updated weights for policy 1, policy_version 1575205 (0.0008) [2023-12-27 16:34:13,516][20381] Updated weights for policy 1, policy_version 1575215 (0.0007) [2023-12-27 16:34:14,020][20347] Updated weights for policy 0, policy_version 1576917 (0.0006) [2023-12-27 16:34:14,068][20347] Updated weights for policy 0, policy_version 1576927 (0.0006) [2023-12-27 16:34:14,119][20347] Updated weights for policy 0, policy_version 1576937 (0.0008) [2023-12-27 16:34:14,195][20381] Updated weights for policy 1, policy_version 1575225 (0.0007) [2023-12-27 16:34:14,248][20381] Updated weights for policy 1, policy_version 1575235 (0.0007) [2023-12-27 16:34:14,297][20381] Updated weights for policy 1, policy_version 1575245 (0.0007) [2023-12-27 16:34:14,344][20381] Updated weights for policy 1, policy_version 1575255 (0.0007) [2023-12-27 16:34:14,785][20347] Updated weights for policy 0, policy_version 1576947 (0.0010) [2023-12-27 16:34:14,848][20347] Updated weights for policy 0, policy_version 1576957 (0.0010) [2023-12-27 16:34:14,911][20347] Updated weights for policy 0, policy_version 1576967 (0.0007) [2023-12-27 16:34:15,090][20381] Updated weights for policy 1, policy_version 1575265 (0.0008) [2023-12-27 16:34:15,143][20381] Updated weights for policy 1, policy_version 1575275 (0.0008) [2023-12-27 16:34:15,199][20381] Updated weights for policy 1, policy_version 1575285 (0.0007) [2023-12-27 16:34:15,591][20347] Updated weights for policy 0, policy_version 1576977 (0.0010) [2023-12-27 16:34:15,648][20347] Updated weights for policy 0, policy_version 1576987 (0.0010) [2023-12-27 16:34:15,696][20347] Updated weights for policy 0, policy_version 1576997 (0.0010) [2023-12-27 16:34:15,745][20347] Updated weights for policy 0, policy_version 1577007 (0.0010) [2023-12-27 16:34:15,867][20381] Updated weights for policy 1, policy_version 1575295 (0.0008) [2023-12-27 16:34:15,932][20381] Updated weights for policy 1, policy_version 1575305 (0.0006) [2023-12-27 16:34:15,989][20381] Updated weights for policy 1, policy_version 1575315 (0.0005) [2023-12-27 16:34:16,448][20347] Updated weights for policy 0, policy_version 1577017 (0.0010) [2023-12-27 16:34:16,496][20347] Updated weights for policy 0, policy_version 1577027 (0.0010) [2023-12-27 16:34:16,545][20347] Updated weights for policy 0, policy_version 1577037 (0.0010) [2023-12-27 16:34:16,587][20381] Updated weights for policy 1, policy_version 1575325 (0.0006) [2023-12-27 16:34:16,645][20381] Updated weights for policy 1, policy_version 1575335 (0.0005) [2023-12-27 16:34:16,695][20381] Updated weights for policy 1, policy_version 1575345 (0.0006) [2023-12-27 16:34:17,243][20347] Updated weights for policy 0, policy_version 1577047 (0.0008) [2023-12-27 16:34:17,288][20347] Updated weights for policy 0, policy_version 1577057 (0.0008) [2023-12-27 16:34:17,315][20381] Updated weights for policy 1, policy_version 1575355 (0.0007) [2023-12-27 16:34:17,342][20347] Updated weights for policy 0, policy_version 1577067 (0.0009) [2023-12-27 16:34:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 807133184. Throughput: 0: 10402.9, 1: 10336.4. Samples: 807151912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:34:17,363][19170] Avg episode reward: [(0, '990.220'), (1, '873.920')] [2023-12-27 16:34:17,367][20381] Updated weights for policy 1, policy_version 1575365 (0.0006) [2023-12-27 16:34:17,417][20381] Updated weights for policy 1, policy_version 1575375 (0.0007) [2023-12-27 16:34:18,011][20347] Updated weights for policy 0, policy_version 1577077 (0.0006) [2023-12-27 16:34:18,064][20381] Updated weights for policy 1, policy_version 1575385 (0.0008) [2023-12-27 16:34:18,071][20347] Updated weights for policy 0, policy_version 1577087 (0.0005) [2023-12-27 16:34:18,118][20381] Updated weights for policy 1, policy_version 1575395 (0.0006) [2023-12-27 16:34:18,132][20347] Updated weights for policy 0, policy_version 1577097 (0.0005) [2023-12-27 16:34:18,175][20381] Updated weights for policy 1, policy_version 1575405 (0.0006) [2023-12-27 16:34:18,226][20381] Updated weights for policy 1, policy_version 1575415 (0.0005) [2023-12-27 16:34:18,700][20347] Updated weights for policy 0, policy_version 1577107 (0.0009) [2023-12-27 16:34:18,762][20347] Updated weights for policy 0, policy_version 1577117 (0.0011) [2023-12-27 16:34:18,818][20347] Updated weights for policy 0, policy_version 1577127 (0.0011) [2023-12-27 16:34:18,851][20381] Updated weights for policy 1, policy_version 1575425 (0.0010) [2023-12-27 16:34:18,905][20381] Updated weights for policy 1, policy_version 1575435 (0.0008) [2023-12-27 16:34:18,960][20381] Updated weights for policy 1, policy_version 1575445 (0.0007) [2023-12-27 16:34:19,495][20347] Updated weights for policy 0, policy_version 1577137 (0.0010) [2023-12-27 16:34:19,559][20347] Updated weights for policy 0, policy_version 1577147 (0.0008) [2023-12-27 16:34:19,620][20347] Updated weights for policy 0, policy_version 1577157 (0.0008) [2023-12-27 16:34:19,676][20347] Updated weights for policy 0, policy_version 1577167 (0.0011) [2023-12-27 16:34:19,682][20381] Updated weights for policy 1, policy_version 1575455 (0.0007) [2023-12-27 16:34:19,750][20381] Updated weights for policy 1, policy_version 1575465 (0.0006) [2023-12-27 16:34:19,814][20381] Updated weights for policy 1, policy_version 1575475 (0.0006) [2023-12-27 16:34:20,355][20381] Updated weights for policy 1, policy_version 1575485 (0.0007) [2023-12-27 16:34:20,413][20381] Updated weights for policy 1, policy_version 1575495 (0.0006) [2023-12-27 16:34:20,466][20347] Updated weights for policy 0, policy_version 1577177 (0.0007) [2023-12-27 16:34:20,472][20381] Updated weights for policy 1, policy_version 1575505 (0.0006) [2023-12-27 16:34:20,521][20347] Updated weights for policy 0, policy_version 1577187 (0.0007) [2023-12-27 16:34:20,584][20347] Updated weights for policy 0, policy_version 1577197 (0.0007) [2023-12-27 16:34:21,137][20381] Updated weights for policy 1, policy_version 1575515 (0.0007) [2023-12-27 16:34:21,195][20381] Updated weights for policy 1, policy_version 1575525 (0.0008) [2023-12-27 16:34:21,258][20381] Updated weights for policy 1, policy_version 1575535 (0.0008) [2023-12-27 16:34:21,341][20347] Updated weights for policy 0, policy_version 1577207 (0.0009) [2023-12-27 16:34:21,421][20347] Updated weights for policy 0, policy_version 1577217 (0.0007) [2023-12-27 16:34:21,487][20347] Updated weights for policy 0, policy_version 1577227 (0.0009) [2023-12-27 16:34:21,989][20381] Updated weights for policy 1, policy_version 1575545 (0.0007) [2023-12-27 16:34:22,053][20381] Updated weights for policy 1, policy_version 1575555 (0.0011) [2023-12-27 16:34:22,117][20381] Updated weights for policy 1, policy_version 1575565 (0.0011) [2023-12-27 16:34:22,167][20381] Updated weights for policy 1, policy_version 1575575 (0.0010) [2023-12-27 16:34:22,224][20347] Updated weights for policy 0, policy_version 1577237 (0.0009) [2023-12-27 16:34:22,288][20347] Updated weights for policy 0, policy_version 1577247 (0.0009) [2023-12-27 16:34:22,339][20347] Updated weights for policy 0, policy_version 1577257 (0.0010) [2023-12-27 16:34:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 807239680. Throughput: 0: 10381.9, 1: 10346.8. Samples: 807214392. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.160')] [2023-12-27 16:34:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001575576_403390464.pth... [2023-12-27 16:34:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001574360_403079168.pth [2023-12-27 16:34:22,384][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001577264_403857408.pth... [2023-12-27 16:34:22,388][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001576048_403546112.pth [2023-12-27 16:34:22,863][20381] Updated weights for policy 1, policy_version 1575585 (0.0011) [2023-12-27 16:34:22,918][20381] Updated weights for policy 1, policy_version 1575595 (0.0010) [2023-12-27 16:34:22,967][20381] Updated weights for policy 1, policy_version 1575605 (0.0010) [2023-12-27 16:34:23,024][20347] Updated weights for policy 0, policy_version 1577267 (0.0008) [2023-12-27 16:34:23,088][20347] Updated weights for policy 0, policy_version 1577277 (0.0007) [2023-12-27 16:34:23,143][20347] Updated weights for policy 0, policy_version 1577287 (0.0007) [2023-12-27 16:34:23,634][20381] Updated weights for policy 1, policy_version 1575615 (0.0007) [2023-12-27 16:34:23,689][20381] Updated weights for policy 1, policy_version 1575625 (0.0005) [2023-12-27 16:34:23,733][20381] Updated weights for policy 1, policy_version 1575635 (0.0005) [2023-12-27 16:34:23,782][20347] Updated weights for policy 0, policy_version 1577297 (0.0009) [2023-12-27 16:34:23,843][20347] Updated weights for policy 0, policy_version 1577307 (0.0009) [2023-12-27 16:34:23,902][20347] Updated weights for policy 0, policy_version 1577317 (0.0010) [2023-12-27 16:34:23,954][20347] Updated weights for policy 0, policy_version 1577327 (0.0010) [2023-12-27 16:34:24,347][20381] Updated weights for policy 1, policy_version 1575645 (0.0006) [2023-12-27 16:34:24,405][20381] Updated weights for policy 1, policy_version 1575655 (0.0008) [2023-12-27 16:34:24,463][20381] Updated weights for policy 1, policy_version 1575665 (0.0007) [2023-12-27 16:34:24,626][20347] Updated weights for policy 0, policy_version 1577337 (0.0010) [2023-12-27 16:34:24,685][20347] Updated weights for policy 0, policy_version 1577347 (0.0007) [2023-12-27 16:34:24,737][20347] Updated weights for policy 0, policy_version 1577357 (0.0005) [2023-12-27 16:34:25,082][20381] Updated weights for policy 1, policy_version 1575675 (0.0007) [2023-12-27 16:34:25,145][20381] Updated weights for policy 1, policy_version 1575685 (0.0006) [2023-12-27 16:34:25,207][20381] Updated weights for policy 1, policy_version 1575695 (0.0008) [2023-12-27 16:34:25,341][20347] Updated weights for policy 0, policy_version 1577367 (0.0006) [2023-12-27 16:34:25,394][20347] Updated weights for policy 0, policy_version 1577377 (0.0008) [2023-12-27 16:34:25,445][20347] Updated weights for policy 0, policy_version 1577387 (0.0007) [2023-12-27 16:34:25,799][20381] Updated weights for policy 1, policy_version 1575705 (0.0007) [2023-12-27 16:34:25,848][20381] Updated weights for policy 1, policy_version 1575715 (0.0005) [2023-12-27 16:34:25,901][20381] Updated weights for policy 1, policy_version 1575725 (0.0005) [2023-12-27 16:34:25,946][20381] Updated weights for policy 1, policy_version 1575735 (0.0005) [2023-12-27 16:34:26,131][20347] Updated weights for policy 0, policy_version 1577397 (0.0007) [2023-12-27 16:34:26,199][20347] Updated weights for policy 0, policy_version 1577407 (0.0008) [2023-12-27 16:34:26,261][20347] Updated weights for policy 0, policy_version 1577417 (0.0006) [2023-12-27 16:34:26,555][20381] Updated weights for policy 1, policy_version 1575745 (0.0007) [2023-12-27 16:34:26,615][20381] Updated weights for policy 1, policy_version 1575755 (0.0007) [2023-12-27 16:34:26,671][20381] Updated weights for policy 1, policy_version 1575765 (0.0008) [2023-12-27 16:34:26,936][20347] Updated weights for policy 0, policy_version 1577427 (0.0006) [2023-12-27 16:34:26,983][20347] Updated weights for policy 0, policy_version 1577437 (0.0007) [2023-12-27 16:34:27,039][20347] Updated weights for policy 0, policy_version 1577447 (0.0005) [2023-12-27 16:34:27,333][20381] Updated weights for policy 1, policy_version 1575775 (0.0006) [2023-12-27 16:34:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 807346176. Throughput: 0: 10286.3, 1: 10422.1. Samples: 807338460. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.230')] [2023-12-27 16:34:27,388][20381] Updated weights for policy 1, policy_version 1575785 (0.0005) [2023-12-27 16:34:27,443][20381] Updated weights for policy 1, policy_version 1575795 (0.0005) [2023-12-27 16:34:27,654][20347] Updated weights for policy 0, policy_version 1577457 (0.0008) [2023-12-27 16:34:27,703][20347] Updated weights for policy 0, policy_version 1577467 (0.0007) [2023-12-27 16:34:27,755][20347] Updated weights for policy 0, policy_version 1577477 (0.0007) [2023-12-27 16:34:27,813][20347] Updated weights for policy 0, policy_version 1577487 (0.0007) [2023-12-27 16:34:28,043][20381] Updated weights for policy 1, policy_version 1575805 (0.0006) [2023-12-27 16:34:28,093][20381] Updated weights for policy 1, policy_version 1575815 (0.0008) [2023-12-27 16:34:28,145][20381] Updated weights for policy 1, policy_version 1575825 (0.0008) [2023-12-27 16:34:28,496][20347] Updated weights for policy 0, policy_version 1577497 (0.0006) [2023-12-27 16:34:28,552][20347] Updated weights for policy 0, policy_version 1577507 (0.0005) [2023-12-27 16:34:28,612][20347] Updated weights for policy 0, policy_version 1577517 (0.0005) [2023-12-27 16:34:28,847][20381] Updated weights for policy 1, policy_version 1575835 (0.0007) [2023-12-27 16:34:28,908][20381] Updated weights for policy 1, policy_version 1575845 (0.0008) [2023-12-27 16:34:28,967][20381] Updated weights for policy 1, policy_version 1575855 (0.0007) [2023-12-27 16:34:29,216][20347] Updated weights for policy 0, policy_version 1577527 (0.0009) [2023-12-27 16:34:29,277][20347] Updated weights for policy 0, policy_version 1577537 (0.0010) [2023-12-27 16:34:29,346][20347] Updated weights for policy 0, policy_version 1577547 (0.0010) [2023-12-27 16:34:29,594][20381] Updated weights for policy 1, policy_version 1575865 (0.0005) [2023-12-27 16:34:29,643][20381] Updated weights for policy 1, policy_version 1575875 (0.0005) [2023-12-27 16:34:29,696][20381] Updated weights for policy 1, policy_version 1575885 (0.0007) [2023-12-27 16:34:29,749][20381] Updated weights for policy 1, policy_version 1575895 (0.0011) [2023-12-27 16:34:30,046][20347] Updated weights for policy 0, policy_version 1577557 (0.0007) [2023-12-27 16:34:30,103][20347] Updated weights for policy 0, policy_version 1577567 (0.0005) [2023-12-27 16:34:30,170][20347] Updated weights for policy 0, policy_version 1577577 (0.0005) [2023-12-27 16:34:30,450][20381] Updated weights for policy 1, policy_version 1575905 (0.0006) [2023-12-27 16:34:30,470][20292] KL-divergence is very high: 155.0386 [2023-12-27 16:34:30,489][20292] KL-divergence is very high: 212.4772 [2023-12-27 16:34:30,500][20381] Updated weights for policy 1, policy_version 1575915 (0.0006) [2023-12-27 16:34:30,512][20292] KL-divergence is very high: 174.7657 [2023-12-27 16:34:30,533][20292] KL-divergence is very high: 135.6107 [2023-12-27 16:34:30,551][20381] Updated weights for policy 1, policy_version 1575925 (0.0010) [2023-12-27 16:34:30,785][20347] Updated weights for policy 0, policy_version 1577587 (0.0006) [2023-12-27 16:34:30,833][20347] Updated weights for policy 0, policy_version 1577597 (0.0008) [2023-12-27 16:34:30,885][20347] Updated weights for policy 0, policy_version 1577607 (0.0009) [2023-12-27 16:34:31,179][20381] Updated weights for policy 1, policy_version 1575935 (0.0009) [2023-12-27 16:34:31,241][20381] Updated weights for policy 1, policy_version 1575945 (0.0008) [2023-12-27 16:34:31,299][20381] Updated weights for policy 1, policy_version 1575955 (0.0008) [2023-12-27 16:34:31,606][20347] Updated weights for policy 0, policy_version 1577617 (0.0006) [2023-12-27 16:34:31,677][20347] Updated weights for policy 0, policy_version 1577627 (0.0008) [2023-12-27 16:34:31,733][20347] Updated weights for policy 0, policy_version 1577637 (0.0008) [2023-12-27 16:34:31,800][20347] Updated weights for policy 0, policy_version 1577647 (0.0008) [2023-12-27 16:34:31,944][20381] Updated weights for policy 1, policy_version 1575965 (0.0008) [2023-12-27 16:34:32,007][20381] Updated weights for policy 1, policy_version 1575975 (0.0007) [2023-12-27 16:34:32,067][20381] Updated weights for policy 1, policy_version 1575985 (0.0009) [2023-12-27 16:34:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 807452672. Throughput: 0: 10268.5, 1: 10450.7. Samples: 807463796. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:32,363][19170] Avg episode reward: [(0, '980.440'), (1, '826.530')] [2023-12-27 16:34:32,520][20347] Updated weights for policy 0, policy_version 1577657 (0.0007) [2023-12-27 16:34:32,584][20347] Updated weights for policy 0, policy_version 1577667 (0.0006) [2023-12-27 16:34:32,647][20347] Updated weights for policy 0, policy_version 1577677 (0.0006) [2023-12-27 16:34:32,753][20381] Updated weights for policy 1, policy_version 1575995 (0.0009) [2023-12-27 16:34:32,812][20381] Updated weights for policy 1, policy_version 1576005 (0.0009) [2023-12-27 16:34:32,859][20381] Updated weights for policy 1, policy_version 1576015 (0.0005) [2023-12-27 16:34:33,273][20347] Updated weights for policy 0, policy_version 1577687 (0.0007) [2023-12-27 16:34:33,324][20347] Updated weights for policy 0, policy_version 1577697 (0.0005) [2023-12-27 16:34:33,373][20347] Updated weights for policy 0, policy_version 1577707 (0.0005) [2023-12-27 16:34:33,511][20381] Updated weights for policy 1, policy_version 1576025 (0.0006) [2023-12-27 16:34:33,576][20381] Updated weights for policy 1, policy_version 1576035 (0.0008) [2023-12-27 16:34:33,638][20381] Updated weights for policy 1, policy_version 1576045 (0.0007) [2023-12-27 16:34:33,691][20381] Updated weights for policy 1, policy_version 1576055 (0.0005) [2023-12-27 16:34:33,956][20347] Updated weights for policy 0, policy_version 1577717 (0.0006) [2023-12-27 16:34:34,004][20347] Updated weights for policy 0, policy_version 1577727 (0.0007) [2023-12-27 16:34:34,049][20347] Updated weights for policy 0, policy_version 1577737 (0.0008) [2023-12-27 16:34:34,353][20381] Updated weights for policy 1, policy_version 1576065 (0.0008) [2023-12-27 16:34:34,409][20381] Updated weights for policy 1, policy_version 1576075 (0.0008) [2023-12-27 16:34:34,464][20381] Updated weights for policy 1, policy_version 1576085 (0.0008) [2023-12-27 16:34:34,740][20347] Updated weights for policy 0, policy_version 1577747 (0.0007) [2023-12-27 16:34:34,804][20347] Updated weights for policy 0, policy_version 1577757 (0.0006) [2023-12-27 16:34:34,863][20347] Updated weights for policy 0, policy_version 1577767 (0.0007) [2023-12-27 16:34:35,214][20381] Updated weights for policy 1, policy_version 1576095 (0.0009) [2023-12-27 16:34:35,264][20381] Updated weights for policy 1, policy_version 1576105 (0.0008) [2023-12-27 16:34:35,313][20381] Updated weights for policy 1, policy_version 1576115 (0.0008) [2023-12-27 16:34:35,462][20347] Updated weights for policy 0, policy_version 1577777 (0.0007) [2023-12-27 16:34:35,523][20347] Updated weights for policy 0, policy_version 1577787 (0.0008) [2023-12-27 16:34:35,578][20347] Updated weights for policy 0, policy_version 1577797 (0.0008) [2023-12-27 16:34:35,632][20347] Updated weights for policy 0, policy_version 1577807 (0.0006) [2023-12-27 16:34:36,087][20381] Updated weights for policy 1, policy_version 1576125 (0.0008) [2023-12-27 16:34:36,154][20381] Updated weights for policy 1, policy_version 1576135 (0.0007) [2023-12-27 16:34:36,208][20381] Updated weights for policy 1, policy_version 1576145 (0.0006) [2023-12-27 16:34:36,213][20347] Updated weights for policy 0, policy_version 1577817 (0.0008) [2023-12-27 16:34:36,278][20347] Updated weights for policy 0, policy_version 1577827 (0.0007) [2023-12-27 16:34:36,342][20347] Updated weights for policy 0, policy_version 1577837 (0.0009) [2023-12-27 16:34:36,885][20381] Updated weights for policy 1, policy_version 1576155 (0.0006) [2023-12-27 16:34:36,934][20381] Updated weights for policy 1, policy_version 1576165 (0.0009) [2023-12-27 16:34:36,986][20347] Updated weights for policy 0, policy_version 1577847 (0.0008) [2023-12-27 16:34:36,991][20381] Updated weights for policy 1, policy_version 1576175 (0.0007) [2023-12-27 16:34:37,036][20347] Updated weights for policy 0, policy_version 1577857 (0.0011) [2023-12-27 16:34:37,085][20347] Updated weights for policy 0, policy_version 1577867 (0.0010) [2023-12-27 16:34:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 807559168. Throughput: 0: 10301.5, 1: 10459.1. Samples: 807526140. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:37,363][19170] Avg episode reward: [(0, '960.980'), (1, '709.820')] [2023-12-27 16:34:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001576184_403546112.pth... [2023-12-27 16:34:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001577872_404013056.pth... [2023-12-27 16:34:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001574968_403234816.pth [2023-12-27 16:34:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001576656_403701760.pth [2023-12-27 16:34:37,647][20347] Updated weights for policy 0, policy_version 1577877 (0.0008) [2023-12-27 16:34:37,704][20347] Updated weights for policy 0, policy_version 1577887 (0.0006) [2023-12-27 16:34:37,752][20381] Updated weights for policy 1, policy_version 1576185 (0.0006) [2023-12-27 16:34:37,756][20347] Updated weights for policy 0, policy_version 1577897 (0.0006) [2023-12-27 16:34:37,814][20381] Updated weights for policy 1, policy_version 1576195 (0.0006) [2023-12-27 16:34:37,863][20381] Updated weights for policy 1, policy_version 1576205 (0.0007) [2023-12-27 16:34:37,909][20381] Updated weights for policy 1, policy_version 1576215 (0.0007) [2023-12-27 16:34:38,389][20347] Updated weights for policy 0, policy_version 1577907 (0.0006) [2023-12-27 16:34:38,447][20347] Updated weights for policy 0, policy_version 1577917 (0.0007) [2023-12-27 16:34:38,510][20347] Updated weights for policy 0, policy_version 1577927 (0.0010) [2023-12-27 16:34:38,617][20381] Updated weights for policy 1, policy_version 1576225 (0.0006) [2023-12-27 16:34:38,673][20381] Updated weights for policy 1, policy_version 1576235 (0.0007) [2023-12-27 16:34:38,725][20381] Updated weights for policy 1, policy_version 1576245 (0.0007) [2023-12-27 16:34:39,189][20347] Updated weights for policy 0, policy_version 1577937 (0.0010) [2023-12-27 16:34:39,249][20347] Updated weights for policy 0, policy_version 1577947 (0.0009) [2023-12-27 16:34:39,309][20347] Updated weights for policy 0, policy_version 1577957 (0.0007) [2023-12-27 16:34:39,370][20347] Updated weights for policy 0, policy_version 1577967 (0.0011) [2023-12-27 16:34:39,493][20381] Updated weights for policy 1, policy_version 1576255 (0.0008) [2023-12-27 16:34:39,547][20292] KL-divergence is very high: 133.6719 [2023-12-27 16:34:39,554][20381] Updated weights for policy 1, policy_version 1576265 (0.0008) [2023-12-27 16:34:39,597][20292] KL-divergence is very high: 104.0400 [2023-12-27 16:34:39,615][20381] Updated weights for policy 1, policy_version 1576275 (0.0008) [2023-12-27 16:34:40,081][20347] Updated weights for policy 0, policy_version 1577977 (0.0007) [2023-12-27 16:34:40,140][20347] Updated weights for policy 0, policy_version 1577987 (0.0007) [2023-12-27 16:34:40,199][20347] Updated weights for policy 0, policy_version 1577997 (0.0007) [2023-12-27 16:34:40,371][20381] Updated weights for policy 1, policy_version 1576285 (0.0007) [2023-12-27 16:34:40,428][20381] Updated weights for policy 1, policy_version 1576295 (0.0010) [2023-12-27 16:34:40,479][20381] Updated weights for policy 1, policy_version 1576305 (0.0010) [2023-12-27 16:34:40,922][20347] Updated weights for policy 0, policy_version 1578007 (0.0008) [2023-12-27 16:34:40,986][20347] Updated weights for policy 0, policy_version 1578017 (0.0008) [2023-12-27 16:34:41,047][20347] Updated weights for policy 0, policy_version 1578027 (0.0009) [2023-12-27 16:34:41,125][20381] Updated weights for policy 1, policy_version 1576315 (0.0008) [2023-12-27 16:34:41,184][20381] Updated weights for policy 1, policy_version 1576325 (0.0009) [2023-12-27 16:34:41,235][20381] Updated weights for policy 1, policy_version 1576335 (0.0009) [2023-12-27 16:34:41,829][20347] Updated weights for policy 0, policy_version 1578037 (0.0006) [2023-12-27 16:34:41,892][20347] Updated weights for policy 0, policy_version 1578047 (0.0005) [2023-12-27 16:34:41,957][20347] Updated weights for policy 0, policy_version 1578057 (0.0006) [2023-12-27 16:34:42,012][20381] Updated weights for policy 1, policy_version 1576345 (0.0008) [2023-12-27 16:34:42,078][20381] Updated weights for policy 1, policy_version 1576355 (0.0010) [2023-12-27 16:34:42,137][20381] Updated weights for policy 1, policy_version 1576365 (0.0010) [2023-12-27 16:34:42,201][20381] Updated weights for policy 1, policy_version 1576375 (0.0010) [2023-12-27 16:34:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 807657472. Throughput: 0: 10324.3, 1: 10321.1. Samples: 807647700. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:42,363][19170] Avg episode reward: [(0, '970.740'), (1, '718.550')] [2023-12-27 16:34:42,562][20347] Updated weights for policy 0, policy_version 1578067 (0.0006) [2023-12-27 16:34:42,624][20347] Updated weights for policy 0, policy_version 1578077 (0.0007) [2023-12-27 16:34:42,679][20347] Updated weights for policy 0, policy_version 1578087 (0.0007) [2023-12-27 16:34:42,923][20381] Updated weights for policy 1, policy_version 1576385 (0.0006) [2023-12-27 16:34:42,974][20381] Updated weights for policy 1, policy_version 1576395 (0.0008) [2023-12-27 16:34:43,020][20381] Updated weights for policy 1, policy_version 1576405 (0.0006) [2023-12-27 16:34:43,321][20347] Updated weights for policy 0, policy_version 1578097 (0.0008) [2023-12-27 16:34:43,376][20347] Updated weights for policy 0, policy_version 1578107 (0.0006) [2023-12-27 16:34:43,440][20347] Updated weights for policy 0, policy_version 1578117 (0.0008) [2023-12-27 16:34:43,493][20347] Updated weights for policy 0, policy_version 1578127 (0.0007) [2023-12-27 16:34:43,589][20381] Updated weights for policy 1, policy_version 1576415 (0.0005) [2023-12-27 16:34:43,644][20381] Updated weights for policy 1, policy_version 1576425 (0.0005) [2023-12-27 16:34:43,711][20381] Updated weights for policy 1, policy_version 1576435 (0.0005) [2023-12-27 16:34:44,090][20347] Updated weights for policy 0, policy_version 1578137 (0.0005) [2023-12-27 16:34:44,155][20347] Updated weights for policy 0, policy_version 1578147 (0.0007) [2023-12-27 16:34:44,213][20347] Updated weights for policy 0, policy_version 1578157 (0.0007) [2023-12-27 16:34:44,298][20381] Updated weights for policy 1, policy_version 1576445 (0.0007) [2023-12-27 16:34:44,357][20381] Updated weights for policy 1, policy_version 1576455 (0.0005) [2023-12-27 16:34:44,415][20381] Updated weights for policy 1, policy_version 1576465 (0.0005) [2023-12-27 16:34:44,876][20347] Updated weights for policy 0, policy_version 1578167 (0.0008) [2023-12-27 16:34:44,936][20347] Updated weights for policy 0, policy_version 1578177 (0.0008) [2023-12-27 16:34:44,993][20347] Updated weights for policy 0, policy_version 1578187 (0.0008) [2023-12-27 16:34:45,027][20381] Updated weights for policy 1, policy_version 1576475 (0.0006) [2023-12-27 16:34:45,089][20381] Updated weights for policy 1, policy_version 1576485 (0.0008) [2023-12-27 16:34:45,147][20381] Updated weights for policy 1, policy_version 1576495 (0.0008) [2023-12-27 16:34:45,691][20347] Updated weights for policy 0, policy_version 1578197 (0.0008) [2023-12-27 16:34:45,752][20347] Updated weights for policy 0, policy_version 1578207 (0.0008) [2023-12-27 16:34:45,809][20347] Updated weights for policy 0, policy_version 1578217 (0.0008) [2023-12-27 16:34:45,850][20381] Updated weights for policy 1, policy_version 1576505 (0.0008) [2023-12-27 16:34:45,903][20381] Updated weights for policy 1, policy_version 1576515 (0.0005) [2023-12-27 16:34:45,961][20381] Updated weights for policy 1, policy_version 1576525 (0.0005) [2023-12-27 16:34:46,019][20381] Updated weights for policy 1, policy_version 1576535 (0.0007) [2023-12-27 16:34:46,520][20347] Updated weights for policy 0, policy_version 1578227 (0.0007) [2023-12-27 16:34:46,571][20347] Updated weights for policy 0, policy_version 1578237 (0.0007) [2023-12-27 16:34:46,623][20347] Updated weights for policy 0, policy_version 1578247 (0.0006) [2023-12-27 16:34:46,647][20381] Updated weights for policy 1, policy_version 1576545 (0.0010) [2023-12-27 16:34:46,702][20381] Updated weights for policy 1, policy_version 1576555 (0.0010) [2023-12-27 16:34:46,758][20381] Updated weights for policy 1, policy_version 1576565 (0.0011) [2023-12-27 16:34:47,331][20347] Updated weights for policy 0, policy_version 1578257 (0.0005) [2023-12-27 16:34:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 807755776. Throughput: 0: 10371.9, 1: 10347.1. Samples: 807772424. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:47,363][19170] Avg episode reward: [(0, '980.430'), (1, '747.460')] [2023-12-27 16:34:47,378][20347] Updated weights for policy 0, policy_version 1578267 (0.0006) [2023-12-27 16:34:47,395][20381] Updated weights for policy 1, policy_version 1576575 (0.0011) [2023-12-27 16:34:47,423][20347] Updated weights for policy 0, policy_version 1578277 (0.0005) [2023-12-27 16:34:47,440][20381] Updated weights for policy 1, policy_version 1576585 (0.0010) [2023-12-27 16:34:47,478][20347] Updated weights for policy 0, policy_version 1578287 (0.0007) [2023-12-27 16:34:47,493][20381] Updated weights for policy 1, policy_version 1576595 (0.0010) [2023-12-27 16:34:48,195][20381] Updated weights for policy 1, policy_version 1576605 (0.0008) [2023-12-27 16:34:48,212][20347] Updated weights for policy 0, policy_version 1578297 (0.0008) [2023-12-27 16:34:48,244][20381] Updated weights for policy 1, policy_version 1576615 (0.0010) [2023-12-27 16:34:48,268][20347] Updated weights for policy 0, policy_version 1578307 (0.0006) [2023-12-27 16:34:48,293][20381] Updated weights for policy 1, policy_version 1576625 (0.0011) [2023-12-27 16:34:48,324][20347] Updated weights for policy 0, policy_version 1578317 (0.0007) [2023-12-27 16:34:49,033][20381] Updated weights for policy 1, policy_version 1576635 (0.0010) [2023-12-27 16:34:49,066][20347] Updated weights for policy 0, policy_version 1578327 (0.0009) [2023-12-27 16:34:49,087][20381] Updated weights for policy 1, policy_version 1576645 (0.0010) [2023-12-27 16:34:49,114][20347] Updated weights for policy 0, policy_version 1578337 (0.0005) [2023-12-27 16:34:49,135][20381] Updated weights for policy 1, policy_version 1576655 (0.0010) [2023-12-27 16:34:49,156][20347] Updated weights for policy 0, policy_version 1578347 (0.0005) [2023-12-27 16:34:49,828][20381] Updated weights for policy 1, policy_version 1576665 (0.0010) [2023-12-27 16:34:49,833][20347] Updated weights for policy 0, policy_version 1578357 (0.0007) [2023-12-27 16:34:49,889][20381] Updated weights for policy 1, policy_version 1576675 (0.0007) [2023-12-27 16:34:49,895][20347] Updated weights for policy 0, policy_version 1578367 (0.0007) [2023-12-27 16:34:49,955][20381] Updated weights for policy 1, policy_version 1576685 (0.0008) [2023-12-27 16:34:49,968][20347] Updated weights for policy 0, policy_version 1578377 (0.0007) [2023-12-27 16:34:50,012][20381] Updated weights for policy 1, policy_version 1576695 (0.0007) [2023-12-27 16:34:50,576][20347] Updated weights for policy 0, policy_version 1578387 (0.0007) [2023-12-27 16:34:50,635][20347] Updated weights for policy 0, policy_version 1578397 (0.0006) [2023-12-27 16:34:50,701][20347] Updated weights for policy 0, policy_version 1578407 (0.0007) [2023-12-27 16:34:50,720][20381] Updated weights for policy 1, policy_version 1576705 (0.0008) [2023-12-27 16:34:50,776][20381] Updated weights for policy 1, policy_version 1576715 (0.0007) [2023-12-27 16:34:50,833][20381] Updated weights for policy 1, policy_version 1576725 (0.0008) [2023-12-27 16:34:51,410][20347] Updated weights for policy 0, policy_version 1578417 (0.0007) [2023-12-27 16:34:51,459][20347] Updated weights for policy 0, policy_version 1578427 (0.0009) [2023-12-27 16:34:51,510][20347] Updated weights for policy 0, policy_version 1578437 (0.0008) [2023-12-27 16:34:51,565][20347] Updated weights for policy 0, policy_version 1578447 (0.0006) [2023-12-27 16:34:51,596][20381] Updated weights for policy 1, policy_version 1576735 (0.0008) [2023-12-27 16:34:51,658][20381] Updated weights for policy 1, policy_version 1576745 (0.0008) [2023-12-27 16:34:51,720][20381] Updated weights for policy 1, policy_version 1576755 (0.0007) [2023-12-27 16:34:52,327][20347] Updated weights for policy 0, policy_version 1578457 (0.0007) [2023-12-27 16:34:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 807854080. Throughput: 0: 10303.6, 1: 10321.7. Samples: 807831656. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:52,363][19170] Avg episode reward: [(0, '970.650'), (1, '815.200')] [2023-12-27 16:34:52,396][20347] Updated weights for policy 0, policy_version 1578467 (0.0007) [2023-12-27 16:34:52,398][20381] Updated weights for policy 1, policy_version 1576765 (0.0010) [2023-12-27 16:34:52,456][20347] Updated weights for policy 0, policy_version 1578477 (0.0006) [2023-12-27 16:34:52,461][20381] Updated weights for policy 1, policy_version 1576775 (0.0011) [2023-12-27 16:34:52,474][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001578480_404168704.pth... [2023-12-27 16:34:52,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001577264_403857408.pth [2023-12-27 16:34:52,526][20381] Updated weights for policy 1, policy_version 1576785 (0.0008) [2023-12-27 16:34:52,570][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001576792_403701760.pth... [2023-12-27 16:34:52,574][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001575576_403390464.pth [2023-12-27 16:34:53,119][20347] Updated weights for policy 0, policy_version 1578487 (0.0007) [2023-12-27 16:34:53,123][20381] Updated weights for policy 1, policy_version 1576795 (0.0006) [2023-12-27 16:34:53,163][20347] Updated weights for policy 0, policy_version 1578497 (0.0007) [2023-12-27 16:34:53,182][20381] Updated weights for policy 1, policy_version 1576805 (0.0009) [2023-12-27 16:34:53,217][20347] Updated weights for policy 0, policy_version 1578507 (0.0006) [2023-12-27 16:34:53,240][20381] Updated weights for policy 1, policy_version 1576815 (0.0010) [2023-12-27 16:34:53,894][20381] Updated weights for policy 1, policy_version 1576825 (0.0009) [2023-12-27 16:34:53,900][20347] Updated weights for policy 0, policy_version 1578517 (0.0007) [2023-12-27 16:34:53,943][20381] Updated weights for policy 1, policy_version 1576835 (0.0010) [2023-12-27 16:34:53,959][20347] Updated weights for policy 0, policy_version 1578527 (0.0005) [2023-12-27 16:34:53,998][20381] Updated weights for policy 1, policy_version 1576845 (0.0010) [2023-12-27 16:34:54,004][20347] Updated weights for policy 0, policy_version 1578537 (0.0005) [2023-12-27 16:34:54,053][20381] Updated weights for policy 1, policy_version 1576855 (0.0010) [2023-12-27 16:34:54,704][20347] Updated weights for policy 0, policy_version 1578547 (0.0006) [2023-12-27 16:34:54,748][20381] Updated weights for policy 1, policy_version 1576865 (0.0010) [2023-12-27 16:34:54,753][20347] Updated weights for policy 0, policy_version 1578557 (0.0009) [2023-12-27 16:34:54,802][20347] Updated weights for policy 0, policy_version 1578567 (0.0005) [2023-12-27 16:34:54,803][20381] Updated weights for policy 1, policy_version 1576875 (0.0010) [2023-12-27 16:34:54,855][20381] Updated weights for policy 1, policy_version 1576885 (0.0010) [2023-12-27 16:34:55,507][20347] Updated weights for policy 0, policy_version 1578577 (0.0005) [2023-12-27 16:34:55,520][20381] Updated weights for policy 1, policy_version 1576895 (0.0011) [2023-12-27 16:34:55,565][20347] Updated weights for policy 0, policy_version 1578587 (0.0005) [2023-12-27 16:34:55,578][20381] Updated weights for policy 1, policy_version 1576905 (0.0010) [2023-12-27 16:34:55,624][20347] Updated weights for policy 0, policy_version 1578597 (0.0006) [2023-12-27 16:34:55,630][20381] Updated weights for policy 1, policy_version 1576915 (0.0010) [2023-12-27 16:34:55,670][20347] Updated weights for policy 0, policy_version 1578607 (0.0006) [2023-12-27 16:34:56,254][20381] Updated weights for policy 1, policy_version 1576925 (0.0011) [2023-12-27 16:34:56,268][20347] Updated weights for policy 0, policy_version 1578617 (0.0007) [2023-12-27 16:34:56,312][20381] Updated weights for policy 1, policy_version 1576935 (0.0010) [2023-12-27 16:34:56,320][20347] Updated weights for policy 0, policy_version 1578627 (0.0007) [2023-12-27 16:34:56,373][20381] Updated weights for policy 1, policy_version 1576945 (0.0010) [2023-12-27 16:34:56,375][20347] Updated weights for policy 0, policy_version 1578637 (0.0008) [2023-12-27 16:34:57,052][20381] Updated weights for policy 1, policy_version 1576955 (0.0010) [2023-12-27 16:34:57,078][20347] Updated weights for policy 0, policy_version 1578647 (0.0007) [2023-12-27 16:34:57,105][20381] Updated weights for policy 1, policy_version 1576965 (0.0010) [2023-12-27 16:34:57,133][20347] Updated weights for policy 0, policy_version 1578657 (0.0005) [2023-12-27 16:34:57,163][20381] Updated weights for policy 1, policy_version 1576975 (0.0010) [2023-12-27 16:34:57,189][20347] Updated weights for policy 0, policy_version 1578667 (0.0009) [2023-12-27 16:34:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 807968768. Throughput: 0: 10265.8, 1: 10299.0. Samples: 807954420. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:34:57,364][19170] Avg episode reward: [(0, '990.220'), (1, '844.410')] [2023-12-27 16:34:57,834][20381] Updated weights for policy 1, policy_version 1576985 (0.0010) [2023-12-27 16:34:57,888][20381] Updated weights for policy 1, policy_version 1576995 (0.0010) [2023-12-27 16:34:57,888][20347] Updated weights for policy 0, policy_version 1578677 (0.0008) [2023-12-27 16:34:57,934][20347] Updated weights for policy 0, policy_version 1578687 (0.0007) [2023-12-27 16:34:57,945][20381] Updated weights for policy 1, policy_version 1577005 (0.0010) [2023-12-27 16:34:57,983][20347] Updated weights for policy 0, policy_version 1578697 (0.0006) [2023-12-27 16:34:57,992][20381] Updated weights for policy 1, policy_version 1577015 (0.0010) [2023-12-27 16:34:58,768][20347] Updated weights for policy 0, policy_version 1578707 (0.0008) [2023-12-27 16:34:58,788][20381] Updated weights for policy 1, policy_version 1577025 (0.0008) [2023-12-27 16:34:58,840][20347] Updated weights for policy 0, policy_version 1578717 (0.0009) [2023-12-27 16:34:58,852][20381] Updated weights for policy 1, policy_version 1577035 (0.0007) [2023-12-27 16:34:58,914][20347] Updated weights for policy 0, policy_version 1578727 (0.0009) [2023-12-27 16:34:58,924][20381] Updated weights for policy 1, policy_version 1577045 (0.0007) [2023-12-27 16:34:59,621][20347] Updated weights for policy 0, policy_version 1578737 (0.0011) [2023-12-27 16:34:59,690][20347] Updated weights for policy 0, policy_version 1578747 (0.0008) [2023-12-27 16:34:59,696][20381] Updated weights for policy 1, policy_version 1577055 (0.0006) [2023-12-27 16:34:59,739][20381] Updated weights for policy 1, policy_version 1577065 (0.0007) [2023-12-27 16:34:59,748][20347] Updated weights for policy 0, policy_version 1578757 (0.0010) [2023-12-27 16:34:59,786][20381] Updated weights for policy 1, policy_version 1577075 (0.0005) [2023-12-27 16:34:59,809][20347] Updated weights for policy 0, policy_version 1578767 (0.0010) [2023-12-27 16:35:00,445][20381] Updated weights for policy 1, policy_version 1577085 (0.0007) [2023-12-27 16:35:00,457][20347] Updated weights for policy 0, policy_version 1578777 (0.0010) [2023-12-27 16:35:00,496][20381] Updated weights for policy 1, policy_version 1577095 (0.0007) [2023-12-27 16:35:00,519][20347] Updated weights for policy 0, policy_version 1578787 (0.0005) [2023-12-27 16:35:00,553][20381] Updated weights for policy 1, policy_version 1577105 (0.0007) [2023-12-27 16:35:00,574][20347] Updated weights for policy 0, policy_version 1578797 (0.0006) [2023-12-27 16:35:01,183][20347] Updated weights for policy 0, policy_version 1578807 (0.0007) [2023-12-27 16:35:01,238][20381] Updated weights for policy 1, policy_version 1577115 (0.0007) [2023-12-27 16:35:01,244][20347] Updated weights for policy 0, policy_version 1578817 (0.0007) [2023-12-27 16:35:01,297][20381] Updated weights for policy 1, policy_version 1577125 (0.0008) [2023-12-27 16:35:01,311][20347] Updated weights for policy 0, policy_version 1578827 (0.0007) [2023-12-27 16:35:01,359][20381] Updated weights for policy 1, policy_version 1577135 (0.0007) [2023-12-27 16:35:02,022][20347] Updated weights for policy 0, policy_version 1578837 (0.0006) [2023-12-27 16:35:02,041][20381] Updated weights for policy 1, policy_version 1577145 (0.0007) [2023-12-27 16:35:02,081][20347] Updated weights for policy 0, policy_version 1578847 (0.0006) [2023-12-27 16:35:02,096][20381] Updated weights for policy 1, policy_version 1577155 (0.0006) [2023-12-27 16:35:02,139][20347] Updated weights for policy 0, policy_version 1578857 (0.0008) [2023-12-27 16:35:02,152][20381] Updated weights for policy 1, policy_version 1577165 (0.0007) [2023-12-27 16:35:02,212][20381] Updated weights for policy 1, policy_version 1577175 (0.0006) [2023-12-27 16:35:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 808067072. Throughput: 0: 10245.1, 1: 10252.6. Samples: 808074312. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '814.140')] [2023-12-27 16:35:02,841][20347] Updated weights for policy 0, policy_version 1578867 (0.0007) [2023-12-27 16:35:02,850][20381] Updated weights for policy 1, policy_version 1577185 (0.0007) [2023-12-27 16:35:02,893][20347] Updated weights for policy 0, policy_version 1578877 (0.0006) [2023-12-27 16:35:02,903][20381] Updated weights for policy 1, policy_version 1577195 (0.0005) [2023-12-27 16:35:02,945][20347] Updated weights for policy 0, policy_version 1578887 (0.0006) [2023-12-27 16:35:02,958][20381] Updated weights for policy 1, policy_version 1577205 (0.0005) [2023-12-27 16:35:03,644][20347] Updated weights for policy 0, policy_version 1578897 (0.0007) [2023-12-27 16:35:03,665][20381] Updated weights for policy 1, policy_version 1577215 (0.0006) [2023-12-27 16:35:03,703][20347] Updated weights for policy 0, policy_version 1578907 (0.0007) [2023-12-27 16:35:03,720][20381] Updated weights for policy 1, policy_version 1577225 (0.0005) [2023-12-27 16:35:03,758][20347] Updated weights for policy 0, policy_version 1578917 (0.0007) [2023-12-27 16:35:03,769][20381] Updated weights for policy 1, policy_version 1577235 (0.0006) [2023-12-27 16:35:03,811][20347] Updated weights for policy 0, policy_version 1578927 (0.0008) [2023-12-27 16:35:04,432][20381] Updated weights for policy 1, policy_version 1577245 (0.0008) [2023-12-27 16:35:04,489][20347] Updated weights for policy 0, policy_version 1578937 (0.0006) [2023-12-27 16:35:04,489][20381] Updated weights for policy 1, policy_version 1577255 (0.0008) [2023-12-27 16:35:04,545][20347] Updated weights for policy 0, policy_version 1578947 (0.0005) [2023-12-27 16:35:04,547][20381] Updated weights for policy 1, policy_version 1577265 (0.0008) [2023-12-27 16:35:04,596][20347] Updated weights for policy 0, policy_version 1578957 (0.0006) [2023-12-27 16:35:05,212][20381] Updated weights for policy 1, policy_version 1577275 (0.0006) [2023-12-27 16:35:05,259][20347] Updated weights for policy 0, policy_version 1578967 (0.0007) [2023-12-27 16:35:05,265][20381] Updated weights for policy 1, policy_version 1577285 (0.0006) [2023-12-27 16:35:05,314][20381] Updated weights for policy 1, policy_version 1577295 (0.0006) [2023-12-27 16:35:05,315][20347] Updated weights for policy 0, policy_version 1578977 (0.0007) [2023-12-27 16:35:05,363][20347] Updated weights for policy 0, policy_version 1578987 (0.0008) [2023-12-27 16:35:05,926][20381] Updated weights for policy 1, policy_version 1577305 (0.0006) [2023-12-27 16:35:05,987][20381] Updated weights for policy 1, policy_version 1577315 (0.0008) [2023-12-27 16:35:05,997][20347] Updated weights for policy 0, policy_version 1578997 (0.0007) [2023-12-27 16:35:06,035][20381] Updated weights for policy 1, policy_version 1577325 (0.0005) [2023-12-27 16:35:06,052][20347] Updated weights for policy 0, policy_version 1579007 (0.0006) [2023-12-27 16:35:06,080][20381] Updated weights for policy 1, policy_version 1577335 (0.0005) [2023-12-27 16:35:06,106][20347] Updated weights for policy 0, policy_version 1579017 (0.0008) [2023-12-27 16:35:06,789][20347] Updated weights for policy 0, policy_version 1579027 (0.0008) [2023-12-27 16:35:06,807][20381] Updated weights for policy 1, policy_version 1577345 (0.0010) [2023-12-27 16:35:06,847][20347] Updated weights for policy 0, policy_version 1579037 (0.0008) [2023-12-27 16:35:06,891][20381] Updated weights for policy 1, policy_version 1577355 (0.0011) [2023-12-27 16:35:06,908][20347] Updated weights for policy 0, policy_version 1579047 (0.0006) [2023-12-27 16:35:06,947][20381] Updated weights for policy 1, policy_version 1577365 (0.0011) [2023-12-27 16:35:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 808165376. Throughput: 0: 10234.3, 1: 10240.4. Samples: 808135752. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:07,363][19170] Avg episode reward: [(0, '990.250'), (1, '823.860')] [2023-12-27 16:35:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001577368_403849216.pth... [2023-12-27 16:35:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001579056_404316160.pth... [2023-12-27 16:35:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001577872_404013056.pth [2023-12-27 16:35:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001576184_403546112.pth [2023-12-27 16:35:07,603][20347] Updated weights for policy 0, policy_version 1579057 (0.0005) [2023-12-27 16:35:07,630][20381] Updated weights for policy 1, policy_version 1577375 (0.0010) [2023-12-27 16:35:07,657][20347] Updated weights for policy 0, policy_version 1579067 (0.0005) [2023-12-27 16:35:07,692][20381] Updated weights for policy 1, policy_version 1577385 (0.0011) [2023-12-27 16:35:07,714][20347] Updated weights for policy 0, policy_version 1579077 (0.0006) [2023-12-27 16:35:07,743][20381] Updated weights for policy 1, policy_version 1577395 (0.0011) [2023-12-27 16:35:07,766][20347] Updated weights for policy 0, policy_version 1579087 (0.0007) [2023-12-27 16:35:08,358][20381] Updated weights for policy 1, policy_version 1577405 (0.0009) [2023-12-27 16:35:08,392][20347] Updated weights for policy 0, policy_version 1579097 (0.0007) [2023-12-27 16:35:08,413][20381] Updated weights for policy 1, policy_version 1577415 (0.0007) [2023-12-27 16:35:08,445][20347] Updated weights for policy 0, policy_version 1579107 (0.0007) [2023-12-27 16:35:08,466][20381] Updated weights for policy 1, policy_version 1577425 (0.0007) [2023-12-27 16:35:08,498][20347] Updated weights for policy 0, policy_version 1579117 (0.0007) [2023-12-27 16:35:09,149][20347] Updated weights for policy 0, policy_version 1579127 (0.0007) [2023-12-27 16:35:09,159][20381] Updated weights for policy 1, policy_version 1577435 (0.0006) [2023-12-27 16:35:09,197][20347] Updated weights for policy 0, policy_version 1579137 (0.0010) [2023-12-27 16:35:09,211][20381] Updated weights for policy 1, policy_version 1577445 (0.0007) [2023-12-27 16:35:09,271][20347] Updated weights for policy 0, policy_version 1579147 (0.0009) [2023-12-27 16:35:09,282][20381] Updated weights for policy 1, policy_version 1577456 (0.0009) [2023-12-27 16:35:09,979][20347] Updated weights for policy 0, policy_version 1579157 (0.0008) [2023-12-27 16:35:10,019][20381] Updated weights for policy 1, policy_version 1577466 (0.0009) [2023-12-27 16:35:10,037][20347] Updated weights for policy 0, policy_version 1579167 (0.0007) [2023-12-27 16:35:10,073][20381] Updated weights for policy 1, policy_version 1577476 (0.0011) [2023-12-27 16:35:10,089][20347] Updated weights for policy 0, policy_version 1579177 (0.0007) [2023-12-27 16:35:10,130][20381] Updated weights for policy 1, policy_version 1577486 (0.0011) [2023-12-27 16:35:10,190][20381] Updated weights for policy 1, policy_version 1577496 (0.0011) [2023-12-27 16:35:10,791][20347] Updated weights for policy 0, policy_version 1579187 (0.0007) [2023-12-27 16:35:10,844][20347] Updated weights for policy 0, policy_version 1579197 (0.0006) [2023-12-27 16:35:10,864][20381] Updated weights for policy 1, policy_version 1577506 (0.0006) [2023-12-27 16:35:10,899][20347] Updated weights for policy 0, policy_version 1579207 (0.0007) [2023-12-27 16:35:10,924][20381] Updated weights for policy 1, policy_version 1577516 (0.0006) [2023-12-27 16:35:10,976][20381] Updated weights for policy 1, policy_version 1577526 (0.0007) [2023-12-27 16:35:11,559][20347] Updated weights for policy 0, policy_version 1579217 (0.0006) [2023-12-27 16:35:11,623][20347] Updated weights for policy 0, policy_version 1579227 (0.0007) [2023-12-27 16:35:11,674][20381] Updated weights for policy 1, policy_version 1577536 (0.0009) [2023-12-27 16:35:11,698][20347] Updated weights for policy 0, policy_version 1579237 (0.0009) [2023-12-27 16:35:11,737][20381] Updated weights for policy 1, policy_version 1577546 (0.0008) [2023-12-27 16:35:11,761][20347] Updated weights for policy 0, policy_version 1579247 (0.0008) [2023-12-27 16:35:11,786][20381] Updated weights for policy 1, policy_version 1577556 (0.0007) [2023-12-27 16:35:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 808263680. Throughput: 0: 10274.7, 1: 10174.9. Samples: 808258696. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:12,363][19170] Avg episode reward: [(0, '980.430'), (1, '775.140')] [2023-12-27 16:35:12,439][20381] Updated weights for policy 1, policy_version 1577566 (0.0007) [2023-12-27 16:35:12,457][20347] Updated weights for policy 0, policy_version 1579257 (0.0007) [2023-12-27 16:35:12,496][20381] Updated weights for policy 1, policy_version 1577576 (0.0006) [2023-12-27 16:35:12,526][20347] Updated weights for policy 0, policy_version 1579267 (0.0005) [2023-12-27 16:35:12,563][20381] Updated weights for policy 1, policy_version 1577586 (0.0006) [2023-12-27 16:35:12,592][20347] Updated weights for policy 0, policy_version 1579277 (0.0006) [2023-12-27 16:35:13,191][20347] Updated weights for policy 0, policy_version 1579287 (0.0006) [2023-12-27 16:35:13,200][20381] Updated weights for policy 1, policy_version 1577596 (0.0006) [2023-12-27 16:35:13,229][20291] KL-divergence is very high: 399.1595 [2023-12-27 16:35:13,240][20347] Updated weights for policy 0, policy_version 1579297 (0.0005) [2023-12-27 16:35:13,246][20381] Updated weights for policy 1, policy_version 1577606 (0.0006) [2023-12-27 16:35:13,269][20291] KL-divergence is very high: 454.7189 [2023-12-27 16:35:13,288][20347] Updated weights for policy 0, policy_version 1579307 (0.0005) [2023-12-27 16:35:13,294][20381] Updated weights for policy 1, policy_version 1577616 (0.0005) [2023-12-27 16:35:13,309][20291] KL-divergence is very high: 233.2328 [2023-12-27 16:35:13,979][20347] Updated weights for policy 0, policy_version 1579317 (0.0006) [2023-12-27 16:35:13,999][20381] Updated weights for policy 1, policy_version 1577626 (0.0007) [2023-12-27 16:35:14,035][20347] Updated weights for policy 0, policy_version 1579327 (0.0005) [2023-12-27 16:35:14,051][20381] Updated weights for policy 1, policy_version 1577636 (0.0007) [2023-12-27 16:35:14,084][20347] Updated weights for policy 0, policy_version 1579337 (0.0005) [2023-12-27 16:35:14,105][20381] Updated weights for policy 1, policy_version 1577646 (0.0006) [2023-12-27 16:35:14,162][20381] Updated weights for policy 1, policy_version 1577656 (0.0006) [2023-12-27 16:35:14,740][20347] Updated weights for policy 0, policy_version 1579347 (0.0007) [2023-12-27 16:35:14,823][20347] Updated weights for policy 0, policy_version 1579357 (0.0009) [2023-12-27 16:35:14,838][20381] Updated weights for policy 1, policy_version 1577666 (0.0007) [2023-12-27 16:35:14,883][20347] Updated weights for policy 0, policy_version 1579367 (0.0010) [2023-12-27 16:35:14,900][20381] Updated weights for policy 1, policy_version 1577676 (0.0008) [2023-12-27 16:35:14,958][20381] Updated weights for policy 1, policy_version 1577686 (0.0007) [2023-12-27 16:35:15,520][20347] Updated weights for policy 0, policy_version 1579377 (0.0009) [2023-12-27 16:35:15,577][20347] Updated weights for policy 0, policy_version 1579387 (0.0008) [2023-12-27 16:35:15,631][20347] Updated weights for policy 0, policy_version 1579397 (0.0005) [2023-12-27 16:35:15,642][20381] Updated weights for policy 1, policy_version 1577696 (0.0007) [2023-12-27 16:35:15,679][20347] Updated weights for policy 0, policy_version 1579407 (0.0005) [2023-12-27 16:35:15,691][20381] Updated weights for policy 1, policy_version 1577706 (0.0007) [2023-12-27 16:35:15,744][20381] Updated weights for policy 1, policy_version 1577716 (0.0009) [2023-12-27 16:35:16,250][20347] Updated weights for policy 0, policy_version 1579417 (0.0005) [2023-12-27 16:35:16,307][20347] Updated weights for policy 0, policy_version 1579427 (0.0005) [2023-12-27 16:35:16,364][20347] Updated weights for policy 0, policy_version 1579437 (0.0005) [2023-12-27 16:35:16,452][20381] Updated weights for policy 1, policy_version 1577726 (0.0007) [2023-12-27 16:35:16,506][20381] Updated weights for policy 1, policy_version 1577736 (0.0007) [2023-12-27 16:35:16,558][20381] Updated weights for policy 1, policy_version 1577746 (0.0007) [2023-12-27 16:35:16,919][20347] Updated weights for policy 0, policy_version 1579447 (0.0005) [2023-12-27 16:35:16,978][20347] Updated weights for policy 0, policy_version 1579457 (0.0007) [2023-12-27 16:35:17,034][20347] Updated weights for policy 0, policy_version 1579467 (0.0008) [2023-12-27 16:35:17,260][20381] Updated weights for policy 1, policy_version 1577756 (0.0007) [2023-12-27 16:35:17,320][20381] Updated weights for policy 1, policy_version 1577766 (0.0005) [2023-12-27 16:35:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 808370176. Throughput: 0: 10319.2, 1: 10116.4. Samples: 808383396. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:17,363][19170] Avg episode reward: [(0, '970.710'), (1, '747.740')] [2023-12-27 16:35:17,380][20381] Updated weights for policy 1, policy_version 1577776 (0.0005) [2023-12-27 16:35:17,737][20347] Updated weights for policy 0, policy_version 1579477 (0.0009) [2023-12-27 16:35:17,797][20347] Updated weights for policy 0, policy_version 1579487 (0.0008) [2023-12-27 16:35:17,857][20347] Updated weights for policy 0, policy_version 1579497 (0.0008) [2023-12-27 16:35:17,873][20381] Updated weights for policy 1, policy_version 1577786 (0.0005) [2023-12-27 16:35:17,926][20381] Updated weights for policy 1, policy_version 1577796 (0.0007) [2023-12-27 16:35:17,980][20381] Updated weights for policy 1, policy_version 1577806 (0.0008) [2023-12-27 16:35:18,036][20381] Updated weights for policy 1, policy_version 1577816 (0.0008) [2023-12-27 16:35:18,592][20347] Updated weights for policy 0, policy_version 1579507 (0.0007) [2023-12-27 16:35:18,642][20381] Updated weights for policy 1, policy_version 1577826 (0.0011) [2023-12-27 16:35:18,644][20347] Updated weights for policy 0, policy_version 1579517 (0.0005) [2023-12-27 16:35:18,700][20347] Updated weights for policy 0, policy_version 1579527 (0.0005) [2023-12-27 16:35:18,702][20381] Updated weights for policy 1, policy_version 1577836 (0.0011) [2023-12-27 16:35:18,757][20381] Updated weights for policy 1, policy_version 1577846 (0.0006) [2023-12-27 16:35:19,434][20347] Updated weights for policy 0, policy_version 1579537 (0.0007) [2023-12-27 16:35:19,457][20381] Updated weights for policy 1, policy_version 1577856 (0.0008) [2023-12-27 16:35:19,497][20347] Updated weights for policy 0, policy_version 1579547 (0.0008) [2023-12-27 16:35:19,520][20381] Updated weights for policy 1, policy_version 1577866 (0.0008) [2023-12-27 16:35:19,559][20347] Updated weights for policy 0, policy_version 1579557 (0.0006) [2023-12-27 16:35:19,584][20381] Updated weights for policy 1, policy_version 1577876 (0.0006) [2023-12-27 16:35:19,619][20347] Updated weights for policy 0, policy_version 1579567 (0.0007) [2023-12-27 16:35:20,286][20381] Updated weights for policy 1, policy_version 1577886 (0.0007) [2023-12-27 16:35:20,348][20381] Updated weights for policy 1, policy_version 1577896 (0.0006) [2023-12-27 16:35:20,406][20381] Updated weights for policy 1, policy_version 1577906 (0.0005) [2023-12-27 16:35:20,410][20347] Updated weights for policy 0, policy_version 1579577 (0.0008) [2023-12-27 16:35:20,459][20347] Updated weights for policy 0, policy_version 1579587 (0.0008) [2023-12-27 16:35:20,516][20347] Updated weights for policy 0, policy_version 1579597 (0.0008) [2023-12-27 16:35:21,088][20381] Updated weights for policy 1, policy_version 1577916 (0.0006) [2023-12-27 16:35:21,154][20381] Updated weights for policy 1, policy_version 1577926 (0.0008) [2023-12-27 16:35:21,216][20381] Updated weights for policy 1, policy_version 1577936 (0.0008) [2023-12-27 16:35:21,246][20347] Updated weights for policy 0, policy_version 1579607 (0.0006) [2023-12-27 16:35:21,306][20347] Updated weights for policy 0, policy_version 1579617 (0.0008) [2023-12-27 16:35:21,372][20347] Updated weights for policy 0, policy_version 1579627 (0.0008) [2023-12-27 16:35:21,982][20381] Updated weights for policy 1, policy_version 1577946 (0.0008) [2023-12-27 16:35:22,048][20381] Updated weights for policy 1, policy_version 1577956 (0.0006) [2023-12-27 16:35:22,110][20381] Updated weights for policy 1, policy_version 1577966 (0.0007) [2023-12-27 16:35:22,140][20347] Updated weights for policy 0, policy_version 1579637 (0.0010) [2023-12-27 16:35:22,165][20381] Updated weights for policy 1, policy_version 1577976 (0.0006) [2023-12-27 16:35:22,195][20347] Updated weights for policy 0, policy_version 1579647 (0.0010) [2023-12-27 16:35:22,254][20347] Updated weights for policy 0, policy_version 1579657 (0.0010) [2023-12-27 16:35:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 808476672. Throughput: 0: 10247.9, 1: 10166.3. Samples: 808444780. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:22,363][19170] Avg episode reward: [(0, '980.510'), (1, '794.880')] [2023-12-27 16:35:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001577976_404004864.pth... [2023-12-27 16:35:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001579664_404471808.pth... [2023-12-27 16:35:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001576792_403701760.pth [2023-12-27 16:35:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001578480_404168704.pth [2023-12-27 16:35:22,827][20381] Updated weights for policy 1, policy_version 1577986 (0.0007) [2023-12-27 16:35:22,888][20381] Updated weights for policy 1, policy_version 1577996 (0.0005) [2023-12-27 16:35:22,940][20381] Updated weights for policy 1, policy_version 1578006 (0.0005) [2023-12-27 16:35:22,963][20347] Updated weights for policy 0, policy_version 1579667 (0.0009) [2023-12-27 16:35:23,026][20347] Updated weights for policy 0, policy_version 1579677 (0.0010) [2023-12-27 16:35:23,085][20347] Updated weights for policy 0, policy_version 1579687 (0.0007) [2023-12-27 16:35:23,538][20381] Updated weights for policy 1, policy_version 1578016 (0.0010) [2023-12-27 16:35:23,600][20381] Updated weights for policy 1, policy_version 1578026 (0.0011) [2023-12-27 16:35:23,661][20381] Updated weights for policy 1, policy_version 1578036 (0.0011) [2023-12-27 16:35:23,677][20347] Updated weights for policy 0, policy_version 1579697 (0.0006) [2023-12-27 16:35:23,735][20347] Updated weights for policy 0, policy_version 1579707 (0.0005) [2023-12-27 16:35:23,789][20347] Updated weights for policy 0, policy_version 1579717 (0.0007) [2023-12-27 16:35:23,843][20347] Updated weights for policy 0, policy_version 1579727 (0.0007) [2023-12-27 16:35:24,351][20381] Updated weights for policy 1, policy_version 1578046 (0.0011) [2023-12-27 16:35:24,406][20381] Updated weights for policy 1, policy_version 1578056 (0.0011) [2023-12-27 16:35:24,452][20381] Updated weights for policy 1, policy_version 1578066 (0.0010) [2023-12-27 16:35:24,464][20347] Updated weights for policy 0, policy_version 1579737 (0.0006) [2023-12-27 16:35:24,516][20347] Updated weights for policy 0, policy_version 1579747 (0.0006) [2023-12-27 16:35:24,573][20347] Updated weights for policy 0, policy_version 1579757 (0.0006) [2023-12-27 16:35:25,152][20381] Updated weights for policy 1, policy_version 1578076 (0.0011) [2023-12-27 16:35:25,204][20381] Updated weights for policy 1, policy_version 1578086 (0.0011) [2023-12-27 16:35:25,240][20347] Updated weights for policy 0, policy_version 1579767 (0.0007) [2023-12-27 16:35:25,253][20381] Updated weights for policy 1, policy_version 1578096 (0.0007) [2023-12-27 16:35:25,293][20347] Updated weights for policy 0, policy_version 1579777 (0.0006) [2023-12-27 16:35:25,342][20347] Updated weights for policy 0, policy_version 1579787 (0.0007) [2023-12-27 16:35:25,900][20381] Updated weights for policy 1, policy_version 1578106 (0.0008) [2023-12-27 16:35:25,949][20381] Updated weights for policy 1, policy_version 1578116 (0.0009) [2023-12-27 16:35:25,998][20381] Updated weights for policy 1, policy_version 1578126 (0.0008) [2023-12-27 16:35:26,015][20347] Updated weights for policy 0, policy_version 1579797 (0.0007) [2023-12-27 16:35:26,056][20381] Updated weights for policy 1, policy_version 1578136 (0.0007) [2023-12-27 16:35:26,073][20347] Updated weights for policy 0, policy_version 1579807 (0.0007) [2023-12-27 16:35:26,128][20347] Updated weights for policy 0, policy_version 1579817 (0.0010) [2023-12-27 16:35:26,629][20381] Updated weights for policy 1, policy_version 1578146 (0.0006) [2023-12-27 16:35:26,697][20381] Updated weights for policy 1, policy_version 1578156 (0.0007) [2023-12-27 16:35:26,753][20381] Updated weights for policy 1, policy_version 1578166 (0.0008) [2023-12-27 16:35:26,798][20347] Updated weights for policy 0, policy_version 1579827 (0.0010) [2023-12-27 16:35:26,863][20347] Updated weights for policy 0, policy_version 1579837 (0.0009) [2023-12-27 16:35:26,920][20347] Updated weights for policy 0, policy_version 1579847 (0.0007) [2023-12-27 16:35:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 808574976. Throughput: 0: 10182.2, 1: 10245.4. Samples: 808566936. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:27,363][19170] Avg episode reward: [(0, '961.080'), (1, '862.930')] [2023-12-27 16:35:27,436][20381] Updated weights for policy 1, policy_version 1578176 (0.0006) [2023-12-27 16:35:27,491][20381] Updated weights for policy 1, policy_version 1578186 (0.0005) [2023-12-27 16:35:27,543][20381] Updated weights for policy 1, policy_version 1578196 (0.0005) [2023-12-27 16:35:27,568][20347] Updated weights for policy 0, policy_version 1579857 (0.0007) [2023-12-27 16:35:27,622][20347] Updated weights for policy 0, policy_version 1579867 (0.0010) [2023-12-27 16:35:27,673][20347] Updated weights for policy 0, policy_version 1579877 (0.0010) [2023-12-27 16:35:27,722][20347] Updated weights for policy 0, policy_version 1579887 (0.0006) [2023-12-27 16:35:28,171][20381] Updated weights for policy 1, policy_version 1578206 (0.0007) [2023-12-27 16:35:28,218][20381] Updated weights for policy 1, policy_version 1578216 (0.0007) [2023-12-27 16:35:28,271][20381] Updated weights for policy 1, policy_version 1578226 (0.0009) [2023-12-27 16:35:28,283][20347] Updated weights for policy 0, policy_version 1579897 (0.0005) [2023-12-27 16:35:28,336][20347] Updated weights for policy 0, policy_version 1579907 (0.0007) [2023-12-27 16:35:28,405][20347] Updated weights for policy 0, policy_version 1579917 (0.0005) [2023-12-27 16:35:29,010][20347] Updated weights for policy 0, policy_version 1579927 (0.0006) [2023-12-27 16:35:29,019][20381] Updated weights for policy 1, policy_version 1578236 (0.0008) [2023-12-27 16:35:29,061][20381] Updated weights for policy 1, policy_version 1578246 (0.0005) [2023-12-27 16:35:29,063][20347] Updated weights for policy 0, policy_version 1579937 (0.0006) [2023-12-27 16:35:29,112][20347] Updated weights for policy 0, policy_version 1579947 (0.0007) [2023-12-27 16:35:29,114][20381] Updated weights for policy 1, policy_version 1578256 (0.0005) [2023-12-27 16:35:29,714][20381] Updated weights for policy 1, policy_version 1578266 (0.0005) [2023-12-27 16:35:29,775][20381] Updated weights for policy 1, policy_version 1578276 (0.0006) [2023-12-27 16:35:29,844][20381] Updated weights for policy 1, policy_version 1578286 (0.0007) [2023-12-27 16:35:29,899][20347] Updated weights for policy 0, policy_version 1579957 (0.0007) [2023-12-27 16:35:29,903][20381] Updated weights for policy 1, policy_version 1578296 (0.0008) [2023-12-27 16:35:29,964][20347] Updated weights for policy 0, policy_version 1579967 (0.0008) [2023-12-27 16:35:30,021][20347] Updated weights for policy 0, policy_version 1579977 (0.0008) [2023-12-27 16:35:30,576][20292] KL-divergence is very high: 135.2068 [2023-12-27 16:35:30,588][20381] Updated weights for policy 1, policy_version 1578306 (0.0007) [2023-12-27 16:35:30,625][20292] KL-divergence is very high: 125.2991 [2023-12-27 16:35:30,654][20381] Updated weights for policy 1, policy_version 1578316 (0.0006) [2023-12-27 16:35:30,674][20347] Updated weights for policy 0, policy_version 1579987 (0.0006) [2023-12-27 16:35:30,706][20381] Updated weights for policy 1, policy_version 1578326 (0.0006) [2023-12-27 16:35:30,729][20347] Updated weights for policy 0, policy_version 1579997 (0.0005) [2023-12-27 16:35:30,783][20347] Updated weights for policy 0, policy_version 1580007 (0.0005) [2023-12-27 16:35:31,413][20381] Updated weights for policy 1, policy_version 1578336 (0.0007) [2023-12-27 16:35:31,424][20347] Updated weights for policy 0, policy_version 1580017 (0.0007) [2023-12-27 16:35:31,466][20381] Updated weights for policy 1, policy_version 1578346 (0.0007) [2023-12-27 16:35:31,482][20347] Updated weights for policy 0, policy_version 1580027 (0.0006) [2023-12-27 16:35:31,523][20381] Updated weights for policy 1, policy_version 1578356 (0.0007) [2023-12-27 16:35:31,539][20347] Updated weights for policy 0, policy_version 1580037 (0.0007) [2023-12-27 16:35:31,628][20347] Updated weights for policy 0, policy_version 1580047 (0.0007) [2023-12-27 16:35:32,198][20381] Updated weights for policy 1, policy_version 1578366 (0.0006) [2023-12-27 16:35:32,261][20381] Updated weights for policy 1, policy_version 1578376 (0.0008) [2023-12-27 16:35:32,312][20347] Updated weights for policy 0, policy_version 1580057 (0.0008) [2023-12-27 16:35:32,324][20381] Updated weights for policy 1, policy_version 1578386 (0.0007) [2023-12-27 16:35:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 808673280. Throughput: 0: 10210.3, 1: 10238.2. Samples: 808692608. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:32,363][19170] Avg episode reward: [(0, '941.270'), (1, '872.770')] [2023-12-27 16:35:32,371][20347] Updated weights for policy 0, policy_version 1580067 (0.0008) [2023-12-27 16:35:32,423][20347] Updated weights for policy 0, policy_version 1580077 (0.0008) [2023-12-27 16:35:33,044][20381] Updated weights for policy 1, policy_version 1578396 (0.0007) [2023-12-27 16:35:33,094][20381] Updated weights for policy 1, policy_version 1578406 (0.0007) [2023-12-27 16:35:33,103][20347] Updated weights for policy 0, policy_version 1580087 (0.0006) [2023-12-27 16:35:33,139][20381] Updated weights for policy 1, policy_version 1578416 (0.0007) [2023-12-27 16:35:33,152][20347] Updated weights for policy 0, policy_version 1580097 (0.0005) [2023-12-27 16:35:33,205][20347] Updated weights for policy 0, policy_version 1580107 (0.0008) [2023-12-27 16:35:33,812][20381] Updated weights for policy 1, policy_version 1578426 (0.0007) [2023-12-27 16:35:33,842][20347] Updated weights for policy 0, policy_version 1580117 (0.0010) [2023-12-27 16:35:33,867][20381] Updated weights for policy 1, policy_version 1578436 (0.0005) [2023-12-27 16:35:33,887][20347] Updated weights for policy 0, policy_version 1580127 (0.0010) [2023-12-27 16:35:33,929][20381] Updated weights for policy 1, policy_version 1578446 (0.0005) [2023-12-27 16:35:33,935][20347] Updated weights for policy 0, policy_version 1580137 (0.0008) [2023-12-27 16:35:33,984][20381] Updated weights for policy 1, policy_version 1578456 (0.0005) [2023-12-27 16:35:34,628][20381] Updated weights for policy 1, policy_version 1578466 (0.0007) [2023-12-27 16:35:34,651][20347] Updated weights for policy 0, policy_version 1580147 (0.0010) [2023-12-27 16:35:34,682][20381] Updated weights for policy 1, policy_version 1578476 (0.0006) [2023-12-27 16:35:34,713][20347] Updated weights for policy 0, policy_version 1580157 (0.0010) [2023-12-27 16:35:34,742][20381] Updated weights for policy 1, policy_version 1578486 (0.0008) [2023-12-27 16:35:34,772][20347] Updated weights for policy 0, policy_version 1580167 (0.0010) [2023-12-27 16:35:35,353][20347] Updated weights for policy 0, policy_version 1580177 (0.0010) [2023-12-27 16:35:35,414][20347] Updated weights for policy 0, policy_version 1580187 (0.0010) [2023-12-27 16:35:35,467][20347] Updated weights for policy 0, policy_version 1580197 (0.0010) [2023-12-27 16:35:35,472][20381] Updated weights for policy 1, policy_version 1578496 (0.0007) [2023-12-27 16:35:35,518][20347] Updated weights for policy 0, policy_version 1580207 (0.0010) [2023-12-27 16:35:35,532][20381] Updated weights for policy 1, policy_version 1578506 (0.0007) [2023-12-27 16:35:35,588][20381] Updated weights for policy 1, policy_version 1578516 (0.0008) [2023-12-27 16:35:36,180][20347] Updated weights for policy 0, policy_version 1580217 (0.0008) [2023-12-27 16:35:36,231][20347] Updated weights for policy 0, policy_version 1580227 (0.0008) [2023-12-27 16:35:36,274][20381] Updated weights for policy 1, policy_version 1578526 (0.0008) [2023-12-27 16:35:36,290][20347] Updated weights for policy 0, policy_version 1580237 (0.0009) [2023-12-27 16:35:36,325][20381] Updated weights for policy 1, policy_version 1578536 (0.0008) [2023-12-27 16:35:36,385][20381] Updated weights for policy 1, policy_version 1578546 (0.0007) [2023-12-27 16:35:36,980][20347] Updated weights for policy 0, policy_version 1580247 (0.0007) [2023-12-27 16:35:37,039][20347] Updated weights for policy 0, policy_version 1580257 (0.0007) [2023-12-27 16:35:37,101][20347] Updated weights for policy 0, policy_version 1580267 (0.0008) [2023-12-27 16:35:37,126][20381] Updated weights for policy 1, policy_version 1578556 (0.0007) [2023-12-27 16:35:37,182][20381] Updated weights for policy 1, policy_version 1578566 (0.0009) [2023-12-27 16:35:37,239][20381] Updated weights for policy 1, policy_version 1578576 (0.0008) [2023-12-27 16:35:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 808787968. Throughput: 0: 10240.1, 1: 10245.4. Samples: 808753500. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:37,363][19170] Avg episode reward: [(0, '941.260'), (1, '785.740')] [2023-12-27 16:35:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001578584_404160512.pth... [2023-12-27 16:35:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001580272_404627456.pth... [2023-12-27 16:35:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001577368_403849216.pth [2023-12-27 16:35:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001579056_404316160.pth [2023-12-27 16:35:37,633][20347] Updated weights for policy 0, policy_version 1580277 (0.0009) [2023-12-27 16:35:37,697][20347] Updated weights for policy 0, policy_version 1580287 (0.0010) [2023-12-27 16:35:37,761][20347] Updated weights for policy 0, policy_version 1580297 (0.0008) [2023-12-27 16:35:37,994][20381] Updated weights for policy 1, policy_version 1578586 (0.0008) [2023-12-27 16:35:38,055][20381] Updated weights for policy 1, policy_version 1578596 (0.0005) [2023-12-27 16:35:38,109][20381] Updated weights for policy 1, policy_version 1578606 (0.0005) [2023-12-27 16:35:38,161][20381] Updated weights for policy 1, policy_version 1578616 (0.0006) [2023-12-27 16:35:38,434][20347] Updated weights for policy 0, policy_version 1580307 (0.0009) [2023-12-27 16:35:38,495][20347] Updated weights for policy 0, policy_version 1580317 (0.0008) [2023-12-27 16:35:38,553][20347] Updated weights for policy 0, policy_version 1580327 (0.0007) [2023-12-27 16:35:38,824][20292] KL-divergence is very high: 129.5737 [2023-12-27 16:35:38,830][20292] KL-divergence is very high: 106.8597 [2023-12-27 16:35:38,835][20381] Updated weights for policy 1, policy_version 1578626 (0.0008) [2023-12-27 16:35:38,843][20292] KL-divergence is very high: 139.5401 [2023-12-27 16:35:38,869][20292] KL-divergence is very high: 104.1834 [2023-12-27 16:35:38,874][20292] KL-divergence is very high: 125.9073 [2023-12-27 16:35:38,896][20381] Updated weights for policy 1, policy_version 1578636 (0.0006) [2023-12-27 16:35:38,952][20381] Updated weights for policy 1, policy_version 1578646 (0.0008) [2023-12-27 16:35:39,130][20347] Updated weights for policy 0, policy_version 1580337 (0.0008) [2023-12-27 16:35:39,179][20347] Updated weights for policy 0, policy_version 1580347 (0.0006) [2023-12-27 16:35:39,240][20347] Updated weights for policy 0, policy_version 1580357 (0.0008) [2023-12-27 16:35:39,303][20347] Updated weights for policy 0, policy_version 1580367 (0.0006) [2023-12-27 16:35:39,642][20381] Updated weights for policy 1, policy_version 1578656 (0.0009) [2023-12-27 16:35:39,695][20381] Updated weights for policy 1, policy_version 1578666 (0.0006) [2023-12-27 16:35:39,753][20381] Updated weights for policy 1, policy_version 1578676 (0.0007) [2023-12-27 16:35:40,033][20347] Updated weights for policy 0, policy_version 1580377 (0.0008) [2023-12-27 16:35:40,090][20347] Updated weights for policy 0, policy_version 1580387 (0.0007) [2023-12-27 16:35:40,137][20347] Updated weights for policy 0, policy_version 1580397 (0.0009) [2023-12-27 16:35:40,400][20381] Updated weights for policy 1, policy_version 1578686 (0.0008) [2023-12-27 16:35:40,455][20381] Updated weights for policy 1, policy_version 1578696 (0.0008) [2023-12-27 16:35:40,513][20381] Updated weights for policy 1, policy_version 1578706 (0.0007) [2023-12-27 16:35:40,814][20347] Updated weights for policy 0, policy_version 1580407 (0.0010) [2023-12-27 16:35:40,864][20347] Updated weights for policy 0, policy_version 1580417 (0.0010) [2023-12-27 16:35:40,915][20347] Updated weights for policy 0, policy_version 1580427 (0.0007) [2023-12-27 16:35:41,234][20381] Updated weights for policy 1, policy_version 1578716 (0.0010) [2023-12-27 16:35:41,295][20381] Updated weights for policy 1, policy_version 1578726 (0.0010) [2023-12-27 16:35:41,355][20381] Updated weights for policy 1, policy_version 1578736 (0.0010) [2023-12-27 16:35:41,668][20347] Updated weights for policy 0, policy_version 1580437 (0.0007) [2023-12-27 16:35:41,729][20347] Updated weights for policy 0, policy_version 1580447 (0.0011) [2023-12-27 16:35:41,790][20347] Updated weights for policy 0, policy_version 1580457 (0.0010) [2023-12-27 16:35:42,095][20381] Updated weights for policy 1, policy_version 1578746 (0.0008) [2023-12-27 16:35:42,147][20381] Updated weights for policy 1, policy_version 1578756 (0.0007) [2023-12-27 16:35:42,195][20381] Updated weights for policy 1, policy_version 1578766 (0.0007) [2023-12-27 16:35:42,247][20381] Updated weights for policy 1, policy_version 1578776 (0.0007) [2023-12-27 16:35:42,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 808886272. Throughput: 0: 10296.8, 1: 10204.9. Samples: 808876996. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:42,363][19170] Avg episode reward: [(0, '970.610'), (1, '758.010')] [2023-12-27 16:35:42,497][20347] Updated weights for policy 0, policy_version 1580467 (0.0009) [2023-12-27 16:35:42,557][20347] Updated weights for policy 0, policy_version 1580477 (0.0006) [2023-12-27 16:35:42,616][20347] Updated weights for policy 0, policy_version 1580487 (0.0007) [2023-12-27 16:35:42,966][20381] Updated weights for policy 1, policy_version 1578786 (0.0008) [2023-12-27 16:35:43,017][20381] Updated weights for policy 1, policy_version 1578796 (0.0007) [2023-12-27 16:35:43,072][20381] Updated weights for policy 1, policy_version 1578806 (0.0007) [2023-12-27 16:35:43,282][20347] Updated weights for policy 0, policy_version 1580497 (0.0007) [2023-12-27 16:35:43,330][20347] Updated weights for policy 0, policy_version 1580507 (0.0007) [2023-12-27 16:35:43,385][20347] Updated weights for policy 0, policy_version 1580517 (0.0007) [2023-12-27 16:35:43,444][20347] Updated weights for policy 0, policy_version 1580527 (0.0008) [2023-12-27 16:35:43,747][20381] Updated weights for policy 1, policy_version 1578816 (0.0007) [2023-12-27 16:35:43,801][20381] Updated weights for policy 1, policy_version 1578826 (0.0007) [2023-12-27 16:35:43,856][20381] Updated weights for policy 1, policy_version 1578836 (0.0007) [2023-12-27 16:35:44,145][20347] Updated weights for policy 0, policy_version 1580537 (0.0007) [2023-12-27 16:35:44,194][20347] Updated weights for policy 0, policy_version 1580547 (0.0008) [2023-12-27 16:35:44,243][20347] Updated weights for policy 0, policy_version 1580557 (0.0008) [2023-12-27 16:35:44,546][20381] Updated weights for policy 1, policy_version 1578846 (0.0008) [2023-12-27 16:35:44,597][20381] Updated weights for policy 1, policy_version 1578856 (0.0007) [2023-12-27 16:35:44,648][20381] Updated weights for policy 1, policy_version 1578866 (0.0007) [2023-12-27 16:35:44,962][20347] Updated weights for policy 0, policy_version 1580567 (0.0006) [2023-12-27 16:35:45,026][20347] Updated weights for policy 0, policy_version 1580577 (0.0007) [2023-12-27 16:35:45,088][20347] Updated weights for policy 0, policy_version 1580587 (0.0007) [2023-12-27 16:35:45,382][20381] Updated weights for policy 1, policy_version 1578876 (0.0009) [2023-12-27 16:35:45,430][20381] Updated weights for policy 1, policy_version 1578886 (0.0010) [2023-12-27 16:35:45,476][20381] Updated weights for policy 1, policy_version 1578896 (0.0010) [2023-12-27 16:35:45,704][20347] Updated weights for policy 0, policy_version 1580597 (0.0007) [2023-12-27 16:35:45,760][20347] Updated weights for policy 0, policy_version 1580607 (0.0007) [2023-12-27 16:35:45,820][20347] Updated weights for policy 0, policy_version 1580617 (0.0005) [2023-12-27 16:35:46,119][20381] Updated weights for policy 1, policy_version 1578906 (0.0009) [2023-12-27 16:35:46,178][20381] Updated weights for policy 1, policy_version 1578916 (0.0005) [2023-12-27 16:35:46,233][20381] Updated weights for policy 1, policy_version 1578926 (0.0005) [2023-12-27 16:35:46,288][20381] Updated weights for policy 1, policy_version 1578936 (0.0006) [2023-12-27 16:35:46,487][20347] Updated weights for policy 0, policy_version 1580627 (0.0007) [2023-12-27 16:35:46,543][20347] Updated weights for policy 0, policy_version 1580637 (0.0008) [2023-12-27 16:35:46,589][20347] Updated weights for policy 0, policy_version 1580647 (0.0010) [2023-12-27 16:35:46,836][20381] Updated weights for policy 1, policy_version 1578946 (0.0005) [2023-12-27 16:35:46,894][20381] Updated weights for policy 1, policy_version 1578956 (0.0007) [2023-12-27 16:35:46,943][20381] Updated weights for policy 1, policy_version 1578966 (0.0008) [2023-12-27 16:35:47,334][20347] Updated weights for policy 0, policy_version 1580657 (0.0010) [2023-12-27 16:35:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 808984576. Throughput: 0: 10301.4, 1: 10236.6. Samples: 808998516. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:47,363][19170] Avg episode reward: [(0, '970.620'), (1, '749.800')] [2023-12-27 16:35:47,391][20347] Updated weights for policy 0, policy_version 1580667 (0.0008) [2023-12-27 16:35:47,448][20347] Updated weights for policy 0, policy_version 1580677 (0.0008) [2023-12-27 16:35:47,503][20347] Updated weights for policy 0, policy_version 1580687 (0.0007) [2023-12-27 16:35:47,560][20381] Updated weights for policy 1, policy_version 1578976 (0.0006) [2023-12-27 16:35:47,622][20381] Updated weights for policy 1, policy_version 1578986 (0.0005) [2023-12-27 16:35:47,676][20381] Updated weights for policy 1, policy_version 1578996 (0.0007) [2023-12-27 16:35:48,207][20347] Updated weights for policy 0, policy_version 1580697 (0.0007) [2023-12-27 16:35:48,269][20347] Updated weights for policy 0, policy_version 1580707 (0.0007) [2023-12-27 16:35:48,327][20347] Updated weights for policy 0, policy_version 1580717 (0.0007) [2023-12-27 16:35:48,344][20381] Updated weights for policy 1, policy_version 1579006 (0.0008) [2023-12-27 16:35:48,412][20381] Updated weights for policy 1, policy_version 1579016 (0.0007) [2023-12-27 16:35:48,479][20381] Updated weights for policy 1, policy_version 1579026 (0.0007) [2023-12-27 16:35:49,028][20347] Updated weights for policy 0, policy_version 1580727 (0.0008) [2023-12-27 16:35:49,043][20381] Updated weights for policy 1, policy_version 1579036 (0.0008) [2023-12-27 16:35:49,081][20347] Updated weights for policy 0, policy_version 1580737 (0.0008) [2023-12-27 16:35:49,098][20381] Updated weights for policy 1, policy_version 1579046 (0.0006) [2023-12-27 16:35:49,131][20347] Updated weights for policy 0, policy_version 1580747 (0.0005) [2023-12-27 16:35:49,147][20381] Updated weights for policy 1, policy_version 1579056 (0.0005) [2023-12-27 16:35:49,854][20347] Updated weights for policy 0, policy_version 1580757 (0.0006) [2023-12-27 16:35:49,919][20347] Updated weights for policy 0, policy_version 1580767 (0.0007) [2023-12-27 16:35:49,920][20381] Updated weights for policy 1, policy_version 1579066 (0.0007) [2023-12-27 16:35:49,987][20347] Updated weights for policy 0, policy_version 1580777 (0.0007) [2023-12-27 16:35:49,988][20381] Updated weights for policy 1, policy_version 1579076 (0.0007) [2023-12-27 16:35:50,050][20381] Updated weights for policy 1, policy_version 1579086 (0.0008) [2023-12-27 16:35:50,103][20381] Updated weights for policy 1, policy_version 1579096 (0.0009) [2023-12-27 16:35:50,579][20347] Updated weights for policy 0, policy_version 1580787 (0.0007) [2023-12-27 16:35:50,616][20291] KL-divergence is very high: 120.7449 [2023-12-27 16:35:50,630][20347] Updated weights for policy 0, policy_version 1580797 (0.0007) [2023-12-27 16:35:50,655][20291] KL-divergence is very high: 166.9839 [2023-12-27 16:35:50,682][20347] Updated weights for policy 0, policy_version 1580807 (0.0008) [2023-12-27 16:35:50,696][20291] KL-divergence is very high: 125.2170 [2023-12-27 16:35:50,823][20381] Updated weights for policy 1, policy_version 1579106 (0.0007) [2023-12-27 16:35:50,867][20381] Updated weights for policy 1, policy_version 1579116 (0.0009) [2023-12-27 16:35:50,916][20381] Updated weights for policy 1, policy_version 1579126 (0.0007) [2023-12-27 16:35:51,468][20347] Updated weights for policy 0, policy_version 1580817 (0.0008) [2023-12-27 16:35:51,524][20347] Updated weights for policy 0, policy_version 1580827 (0.0008) [2023-12-27 16:35:51,574][20347] Updated weights for policy 0, policy_version 1580837 (0.0008) [2023-12-27 16:35:51,635][20347] Updated weights for policy 0, policy_version 1580847 (0.0008) [2023-12-27 16:35:51,681][20381] Updated weights for policy 1, policy_version 1579136 (0.0008) [2023-12-27 16:35:51,749][20381] Updated weights for policy 1, policy_version 1579146 (0.0014) [2023-12-27 16:35:51,806][20381] Updated weights for policy 1, policy_version 1579156 (0.0007) [2023-12-27 16:35:52,332][20347] Updated weights for policy 0, policy_version 1580857 (0.0010) [2023-12-27 16:35:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 809082880. Throughput: 0: 10275.1, 1: 10227.5. Samples: 809058368. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:52,363][19170] Avg episode reward: [(0, '951.270'), (1, '738.370')] [2023-12-27 16:35:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001579160_404307968.pth... [2023-12-27 16:35:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001577976_404004864.pth [2023-12-27 16:35:52,397][20347] Updated weights for policy 0, policy_version 1580867 (0.0010) [2023-12-27 16:35:52,455][20347] Updated weights for policy 0, policy_version 1580877 (0.0009) [2023-12-27 16:35:52,468][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001580880_404783104.pth... [2023-12-27 16:35:52,472][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001579664_404471808.pth [2023-12-27 16:35:52,542][20381] Updated weights for policy 1, policy_version 1579166 (0.0009) [2023-12-27 16:35:52,603][20381] Updated weights for policy 1, policy_version 1579176 (0.0010) [2023-12-27 16:35:52,664][20381] Updated weights for policy 1, policy_version 1579186 (0.0010) [2023-12-27 16:35:53,144][20347] Updated weights for policy 0, policy_version 1580887 (0.0010) [2023-12-27 16:35:53,202][20347] Updated weights for policy 0, policy_version 1580897 (0.0007) [2023-12-27 16:35:53,257][20347] Updated weights for policy 0, policy_version 1580907 (0.0008) [2023-12-27 16:35:53,320][20381] Updated weights for policy 1, policy_version 1579196 (0.0009) [2023-12-27 16:35:53,373][20381] Updated weights for policy 1, policy_version 1579206 (0.0005) [2023-12-27 16:35:53,428][20381] Updated weights for policy 1, policy_version 1579216 (0.0005) [2023-12-27 16:35:53,873][20347] Updated weights for policy 0, policy_version 1580917 (0.0008) [2023-12-27 16:35:53,933][20347] Updated weights for policy 0, policy_version 1580927 (0.0007) [2023-12-27 16:35:54,000][20347] Updated weights for policy 0, policy_version 1580937 (0.0006) [2023-12-27 16:35:54,051][20381] Updated weights for policy 1, policy_version 1579226 (0.0005) [2023-12-27 16:35:54,107][20381] Updated weights for policy 1, policy_version 1579236 (0.0005) [2023-12-27 16:35:54,165][20381] Updated weights for policy 1, policy_version 1579246 (0.0005) [2023-12-27 16:35:54,217][20381] Updated weights for policy 1, policy_version 1579256 (0.0006) [2023-12-27 16:35:54,599][20347] Updated weights for policy 0, policy_version 1580947 (0.0006) [2023-12-27 16:35:54,668][20347] Updated weights for policy 0, policy_version 1580957 (0.0007) [2023-12-27 16:35:54,733][20347] Updated weights for policy 0, policy_version 1580967 (0.0005) [2023-12-27 16:35:54,783][20381] Updated weights for policy 1, policy_version 1579266 (0.0005) [2023-12-27 16:35:54,830][20381] Updated weights for policy 1, policy_version 1579276 (0.0005) [2023-12-27 16:35:54,878][20381] Updated weights for policy 1, policy_version 1579286 (0.0005) [2023-12-27 16:35:55,321][20347] Updated weights for policy 0, policy_version 1580977 (0.0005) [2023-12-27 16:35:55,379][20347] Updated weights for policy 0, policy_version 1580987 (0.0007) [2023-12-27 16:35:55,433][20347] Updated weights for policy 0, policy_version 1580997 (0.0007) [2023-12-27 16:35:55,489][20347] Updated weights for policy 0, policy_version 1581007 (0.0008) [2023-12-27 16:35:55,510][20381] Updated weights for policy 1, policy_version 1579296 (0.0008) [2023-12-27 16:35:55,567][20381] Updated weights for policy 1, policy_version 1579306 (0.0007) [2023-12-27 16:35:55,626][20381] Updated weights for policy 1, policy_version 1579316 (0.0005) [2023-12-27 16:35:56,146][20347] Updated weights for policy 0, policy_version 1581017 (0.0006) [2023-12-27 16:35:56,203][20347] Updated weights for policy 0, policy_version 1581027 (0.0005) [2023-12-27 16:35:56,259][20347] Updated weights for policy 0, policy_version 1581037 (0.0005) [2023-12-27 16:35:56,326][20381] Updated weights for policy 1, policy_version 1579326 (0.0007) [2023-12-27 16:35:56,383][20381] Updated weights for policy 1, policy_version 1579336 (0.0007) [2023-12-27 16:35:56,437][20381] Updated weights for policy 1, policy_version 1579346 (0.0008) [2023-12-27 16:35:56,821][20347] Updated weights for policy 0, policy_version 1581047 (0.0005) [2023-12-27 16:35:56,878][20347] Updated weights for policy 0, policy_version 1581057 (0.0006) [2023-12-27 16:35:56,926][20347] Updated weights for policy 0, policy_version 1581067 (0.0007) [2023-12-27 16:35:57,055][20381] Updated weights for policy 1, policy_version 1579356 (0.0008) [2023-12-27 16:35:57,113][20381] Updated weights for policy 1, policy_version 1579366 (0.0008) [2023-12-27 16:35:57,171][20381] Updated weights for policy 1, policy_version 1579376 (0.0008) [2023-12-27 16:35:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20660.5). Total num frames: 809197568. Throughput: 0: 10289.2, 1: 10260.8. Samples: 809183444. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:35:57,363][19170] Avg episode reward: [(0, '912.200'), (1, '834.850')] [2023-12-27 16:35:57,579][20347] Updated weights for policy 0, policy_version 1581077 (0.0007) [2023-12-27 16:35:57,634][20347] Updated weights for policy 0, policy_version 1581087 (0.0008) [2023-12-27 16:35:57,685][20347] Updated weights for policy 0, policy_version 1581097 (0.0007) [2023-12-27 16:35:57,778][20381] Updated weights for policy 1, policy_version 1579386 (0.0008) [2023-12-27 16:35:57,824][20381] Updated weights for policy 1, policy_version 1579396 (0.0007) [2023-12-27 16:35:57,883][20381] Updated weights for policy 1, policy_version 1579406 (0.0007) [2023-12-27 16:35:57,933][20381] Updated weights for policy 1, policy_version 1579416 (0.0008) [2023-12-27 16:35:58,359][20347] Updated weights for policy 0, policy_version 1581107 (0.0007) [2023-12-27 16:35:58,426][20347] Updated weights for policy 0, policy_version 1581117 (0.0010) [2023-12-27 16:35:58,492][20347] Updated weights for policy 0, policy_version 1581127 (0.0010) [2023-12-27 16:35:58,696][20381] Updated weights for policy 1, policy_version 1579426 (0.0011) [2023-12-27 16:35:58,756][20381] Updated weights for policy 1, policy_version 1579436 (0.0009) [2023-12-27 16:35:58,820][20381] Updated weights for policy 1, policy_version 1579446 (0.0009) [2023-12-27 16:35:59,264][20347] Updated weights for policy 0, policy_version 1581137 (0.0009) [2023-12-27 16:35:59,326][20347] Updated weights for policy 0, policy_version 1581147 (0.0008) [2023-12-27 16:35:59,390][20347] Updated weights for policy 0, policy_version 1581157 (0.0008) [2023-12-27 16:35:59,444][20347] Updated weights for policy 0, policy_version 1581167 (0.0008) [2023-12-27 16:35:59,604][20381] Updated weights for policy 1, policy_version 1579456 (0.0007) [2023-12-27 16:35:59,665][20381] Updated weights for policy 1, policy_version 1579466 (0.0008) [2023-12-27 16:35:59,716][20381] Updated weights for policy 1, policy_version 1579476 (0.0010) [2023-12-27 16:36:00,172][20347] Updated weights for policy 0, policy_version 1581177 (0.0007) [2023-12-27 16:36:00,232][20347] Updated weights for policy 0, policy_version 1581187 (0.0007) [2023-12-27 16:36:00,293][20347] Updated weights for policy 0, policy_version 1581197 (0.0007) [2023-12-27 16:36:00,419][20381] Updated weights for policy 1, policy_version 1579486 (0.0010) [2023-12-27 16:36:00,467][20381] Updated weights for policy 1, policy_version 1579496 (0.0010) [2023-12-27 16:36:00,521][20381] Updated weights for policy 1, policy_version 1579506 (0.0010) [2023-12-27 16:36:00,976][20347] Updated weights for policy 0, policy_version 1581207 (0.0007) [2023-12-27 16:36:01,032][20347] Updated weights for policy 0, policy_version 1581217 (0.0009) [2023-12-27 16:36:01,101][20347] Updated weights for policy 0, policy_version 1581227 (0.0008) [2023-12-27 16:36:01,220][20381] Updated weights for policy 1, policy_version 1579516 (0.0009) [2023-12-27 16:36:01,283][20381] Updated weights for policy 1, policy_version 1579526 (0.0007) [2023-12-27 16:36:01,336][20381] Updated weights for policy 1, policy_version 1579536 (0.0006) [2023-12-27 16:36:01,836][20347] Updated weights for policy 0, policy_version 1581237 (0.0008) [2023-12-27 16:36:01,900][20347] Updated weights for policy 0, policy_version 1581247 (0.0007) [2023-12-27 16:36:01,954][20347] Updated weights for policy 0, policy_version 1581257 (0.0008) [2023-12-27 16:36:02,014][20381] Updated weights for policy 1, policy_version 1579546 (0.0008) [2023-12-27 16:36:02,071][20381] Updated weights for policy 1, policy_version 1579556 (0.0006) [2023-12-27 16:36:02,128][20381] Updated weights for policy 1, policy_version 1579566 (0.0007) [2023-12-27 16:36:02,188][20381] Updated weights for policy 1, policy_version 1579576 (0.0007) [2023-12-27 16:36:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 809295872. Throughput: 0: 10201.5, 1: 10246.1. Samples: 809303540. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:36:02,363][19170] Avg episode reward: [(0, '921.820'), (1, '775.570')] [2023-12-27 16:36:02,701][20347] Updated weights for policy 0, policy_version 1581267 (0.0009) [2023-12-27 16:36:02,756][20347] Updated weights for policy 0, policy_version 1581277 (0.0008) [2023-12-27 16:36:02,819][20347] Updated weights for policy 0, policy_version 1581287 (0.0009) [2023-12-27 16:36:02,829][20381] Updated weights for policy 1, policy_version 1579586 (0.0008) [2023-12-27 16:36:02,890][20381] Updated weights for policy 1, policy_version 1579596 (0.0009) [2023-12-27 16:36:02,942][20381] Updated weights for policy 1, policy_version 1579606 (0.0008) [2023-12-27 16:36:03,525][20381] Updated weights for policy 1, policy_version 1579616 (0.0006) [2023-12-27 16:36:03,562][20347] Updated weights for policy 0, policy_version 1581297 (0.0009) [2023-12-27 16:36:03,575][20381] Updated weights for policy 1, policy_version 1579626 (0.0006) [2023-12-27 16:36:03,612][20291] KL-divergence is very high: 156.3383 [2023-12-27 16:36:03,613][20347] Updated weights for policy 0, policy_version 1581307 (0.0006) [2023-12-27 16:36:03,619][20381] Updated weights for policy 1, policy_version 1579636 (0.0005) [2023-12-27 16:36:03,656][20291] KL-divergence is very high: 206.8827 [2023-12-27 16:36:03,666][20347] Updated weights for policy 0, policy_version 1581317 (0.0007) [2023-12-27 16:36:03,701][20291] KL-divergence is very high: 151.3506 [2023-12-27 16:36:03,722][20347] Updated weights for policy 0, policy_version 1581327 (0.0008) [2023-12-27 16:36:04,288][20381] Updated weights for policy 1, policy_version 1579646 (0.0008) [2023-12-27 16:36:04,346][20381] Updated weights for policy 1, policy_version 1579656 (0.0008) [2023-12-27 16:36:04,406][20381] Updated weights for policy 1, policy_version 1579666 (0.0008) [2023-12-27 16:36:04,451][20347] Updated weights for policy 0, policy_version 1581337 (0.0006) [2023-12-27 16:36:04,502][20347] Updated weights for policy 0, policy_version 1581347 (0.0007) [2023-12-27 16:36:04,565][20347] Updated weights for policy 0, policy_version 1581357 (0.0006) [2023-12-27 16:36:05,070][20381] Updated weights for policy 1, policy_version 1579676 (0.0007) [2023-12-27 16:36:05,137][20381] Updated weights for policy 1, policy_version 1579686 (0.0008) [2023-12-27 16:36:05,196][20381] Updated weights for policy 1, policy_version 1579696 (0.0008) [2023-12-27 16:36:05,255][20347] Updated weights for policy 0, policy_version 1581367 (0.0007) [2023-12-27 16:36:05,316][20347] Updated weights for policy 0, policy_version 1581377 (0.0008) [2023-12-27 16:36:05,375][20347] Updated weights for policy 0, policy_version 1581387 (0.0008) [2023-12-27 16:36:05,886][20381] Updated weights for policy 1, policy_version 1579706 (0.0007) [2023-12-27 16:36:05,940][20381] Updated weights for policy 1, policy_version 1579716 (0.0008) [2023-12-27 16:36:05,990][20347] Updated weights for policy 0, policy_version 1581397 (0.0007) [2023-12-27 16:36:05,995][20381] Updated weights for policy 1, policy_version 1579726 (0.0006) [2023-12-27 16:36:06,049][20347] Updated weights for policy 0, policy_version 1581407 (0.0005) [2023-12-27 16:36:06,054][20381] Updated weights for policy 1, policy_version 1579736 (0.0006) [2023-12-27 16:36:06,110][20347] Updated weights for policy 0, policy_version 1581417 (0.0008) [2023-12-27 16:36:06,763][20381] Updated weights for policy 1, policy_version 1579746 (0.0006) [2023-12-27 16:36:06,824][20381] Updated weights for policy 1, policy_version 1579756 (0.0007) [2023-12-27 16:36:06,825][20347] Updated weights for policy 0, policy_version 1581427 (0.0009) [2023-12-27 16:36:06,881][20347] Updated weights for policy 0, policy_version 1581437 (0.0011) [2023-12-27 16:36:06,883][20381] Updated weights for policy 1, policy_version 1579766 (0.0006) [2023-12-27 16:36:06,941][20347] Updated weights for policy 0, policy_version 1581447 (0.0011) [2023-12-27 16:36:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 809394176. Throughput: 0: 10202.8, 1: 10240.5. Samples: 809364732. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:36:07,363][19170] Avg episode reward: [(0, '921.940'), (1, '794.750')] [2023-12-27 16:36:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001581456_404930560.pth... [2023-12-27 16:36:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001579768_404463616.pth... [2023-12-27 16:36:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001580272_404627456.pth [2023-12-27 16:36:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001578584_404160512.pth [2023-12-27 16:36:07,477][20381] Updated weights for policy 1, policy_version 1579776 (0.0006) [2023-12-27 16:36:07,538][20381] Updated weights for policy 1, policy_version 1579786 (0.0008) [2023-12-27 16:36:07,596][20381] Updated weights for policy 1, policy_version 1579796 (0.0010) [2023-12-27 16:36:07,611][20347] Updated weights for policy 0, policy_version 1581457 (0.0010) [2023-12-27 16:36:07,665][20347] Updated weights for policy 0, policy_version 1581467 (0.0005) [2023-12-27 16:36:07,718][20347] Updated weights for policy 0, policy_version 1581477 (0.0005) [2023-12-27 16:36:07,771][20347] Updated weights for policy 0, policy_version 1581487 (0.0005) [2023-12-27 16:36:08,191][20381] Updated weights for policy 1, policy_version 1579806 (0.0008) [2023-12-27 16:36:08,247][20381] Updated weights for policy 1, policy_version 1579816 (0.0009) [2023-12-27 16:36:08,313][20381] Updated weights for policy 1, policy_version 1579826 (0.0010) [2023-12-27 16:36:08,377][20347] Updated weights for policy 0, policy_version 1581497 (0.0009) [2023-12-27 16:36:08,443][20347] Updated weights for policy 0, policy_version 1581507 (0.0008) [2023-12-27 16:36:08,509][20347] Updated weights for policy 0, policy_version 1581517 (0.0007) [2023-12-27 16:36:08,969][20381] Updated weights for policy 1, policy_version 1579836 (0.0008) [2023-12-27 16:36:09,032][20381] Updated weights for policy 1, policy_version 1579846 (0.0006) [2023-12-27 16:36:09,089][20381] Updated weights for policy 1, policy_version 1579856 (0.0007) [2023-12-27 16:36:09,123][20347] Updated weights for policy 0, policy_version 1581527 (0.0007) [2023-12-27 16:36:09,174][20347] Updated weights for policy 0, policy_version 1581537 (0.0005) [2023-12-27 16:36:09,232][20347] Updated weights for policy 0, policy_version 1581547 (0.0008) [2023-12-27 16:36:09,772][20381] Updated weights for policy 1, policy_version 1579866 (0.0009) [2023-12-27 16:36:09,833][20381] Updated weights for policy 1, policy_version 1579876 (0.0011) [2023-12-27 16:36:09,893][20381] Updated weights for policy 1, policy_version 1579886 (0.0009) [2023-12-27 16:36:09,897][20347] Updated weights for policy 0, policy_version 1581557 (0.0009) [2023-12-27 16:36:09,955][20381] Updated weights for policy 1, policy_version 1579896 (0.0008) [2023-12-27 16:36:09,960][20347] Updated weights for policy 0, policy_version 1581567 (0.0006) [2023-12-27 16:36:10,024][20347] Updated weights for policy 0, policy_version 1581577 (0.0007) [2023-12-27 16:36:10,655][20347] Updated weights for policy 0, policy_version 1581587 (0.0006) [2023-12-27 16:36:10,682][20381] Updated weights for policy 1, policy_version 1579906 (0.0011) [2023-12-27 16:36:10,708][20347] Updated weights for policy 0, policy_version 1581597 (0.0005) [2023-12-27 16:36:10,746][20381] Updated weights for policy 1, policy_version 1579916 (0.0006) [2023-12-27 16:36:10,760][20347] Updated weights for policy 0, policy_version 1581607 (0.0006) [2023-12-27 16:36:10,808][20381] Updated weights for policy 1, policy_version 1579926 (0.0005) [2023-12-27 16:36:11,445][20347] Updated weights for policy 0, policy_version 1581617 (0.0007) [2023-12-27 16:36:11,460][20381] Updated weights for policy 1, policy_version 1579936 (0.0008) [2023-12-27 16:36:11,504][20347] Updated weights for policy 0, policy_version 1581627 (0.0006) [2023-12-27 16:36:11,518][20381] Updated weights for policy 1, policy_version 1579946 (0.0006) [2023-12-27 16:36:11,567][20347] Updated weights for policy 0, policy_version 1581637 (0.0006) [2023-12-27 16:36:11,576][20381] Updated weights for policy 1, policy_version 1579956 (0.0006) [2023-12-27 16:36:11,645][20347] Updated weights for policy 0, policy_version 1581647 (0.0007) [2023-12-27 16:36:12,305][20347] Updated weights for policy 0, policy_version 1581657 (0.0007) [2023-12-27 16:36:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 809492480. Throughput: 0: 10265.1, 1: 10240.6. Samples: 809489692. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:36:12,363][19170] Avg episode reward: [(0, '961.040'), (1, '745.720')] [2023-12-27 16:36:12,372][20347] Updated weights for policy 0, policy_version 1581667 (0.0007) [2023-12-27 16:36:12,415][20381] Updated weights for policy 1, policy_version 1579966 (0.0007) [2023-12-27 16:36:12,441][20347] Updated weights for policy 0, policy_version 1581677 (0.0007) [2023-12-27 16:36:12,478][20381] Updated weights for policy 1, policy_version 1579976 (0.0006) [2023-12-27 16:36:12,536][20381] Updated weights for policy 1, policy_version 1579986 (0.0008) [2023-12-27 16:36:13,141][20347] Updated weights for policy 0, policy_version 1581687 (0.0007) [2023-12-27 16:36:13,143][20381] Updated weights for policy 1, policy_version 1579996 (0.0007) [2023-12-27 16:36:13,191][20381] Updated weights for policy 1, policy_version 1580006 (0.0006) [2023-12-27 16:36:13,199][20347] Updated weights for policy 0, policy_version 1581697 (0.0008) [2023-12-27 16:36:13,248][20381] Updated weights for policy 1, policy_version 1580016 (0.0007) [2023-12-27 16:36:13,259][20347] Updated weights for policy 0, policy_version 1581707 (0.0007) [2023-12-27 16:36:13,910][20381] Updated weights for policy 1, policy_version 1580026 (0.0006) [2023-12-27 16:36:13,925][20347] Updated weights for policy 0, policy_version 1581717 (0.0006) [2023-12-27 16:36:13,966][20381] Updated weights for policy 1, policy_version 1580036 (0.0008) [2023-12-27 16:36:13,986][20347] Updated weights for policy 0, policy_version 1581727 (0.0007) [2023-12-27 16:36:14,012][20381] Updated weights for policy 1, policy_version 1580046 (0.0009) [2023-12-27 16:36:14,042][20347] Updated weights for policy 0, policy_version 1581737 (0.0010) [2023-12-27 16:36:14,066][20381] Updated weights for policy 1, policy_version 1580056 (0.0008) [2023-12-27 16:36:14,616][20347] Updated weights for policy 0, policy_version 1581747 (0.0010) [2023-12-27 16:36:14,676][20347] Updated weights for policy 0, policy_version 1581757 (0.0010) [2023-12-27 16:36:14,729][20347] Updated weights for policy 0, policy_version 1581767 (0.0007) [2023-12-27 16:36:14,767][20381] Updated weights for policy 1, policy_version 1580066 (0.0007) [2023-12-27 16:36:14,831][20381] Updated weights for policy 1, policy_version 1580076 (0.0006) [2023-12-27 16:36:14,897][20381] Updated weights for policy 1, policy_version 1580086 (0.0006) [2023-12-27 16:36:15,394][20347] Updated weights for policy 0, policy_version 1581777 (0.0009) [2023-12-27 16:36:15,455][20347] Updated weights for policy 0, policy_version 1581787 (0.0005) [2023-12-27 16:36:15,519][20347] Updated weights for policy 0, policy_version 1581797 (0.0005) [2023-12-27 16:36:15,550][20381] Updated weights for policy 1, policy_version 1580096 (0.0008) [2023-12-27 16:36:15,572][20347] Updated weights for policy 0, policy_version 1581807 (0.0006) [2023-12-27 16:36:15,600][20381] Updated weights for policy 1, policy_version 1580106 (0.0007) [2023-12-27 16:36:15,647][20381] Updated weights for policy 1, policy_version 1580116 (0.0007) [2023-12-27 16:36:16,171][20347] Updated weights for policy 0, policy_version 1581817 (0.0010) [2023-12-27 16:36:16,235][20347] Updated weights for policy 0, policy_version 1581827 (0.0010) [2023-12-27 16:36:16,286][20347] Updated weights for policy 0, policy_version 1581837 (0.0010) [2023-12-27 16:36:16,343][20381] Updated weights for policy 1, policy_version 1580126 (0.0007) [2023-12-27 16:36:16,398][20381] Updated weights for policy 1, policy_version 1580136 (0.0008) [2023-12-27 16:36:16,454][20381] Updated weights for policy 1, policy_version 1580146 (0.0008) [2023-12-27 16:36:16,871][20347] Updated weights for policy 0, policy_version 1581847 (0.0009) [2023-12-27 16:36:16,919][20347] Updated weights for policy 0, policy_version 1581857 (0.0005) [2023-12-27 16:36:16,970][20347] Updated weights for policy 0, policy_version 1581867 (0.0007) [2023-12-27 16:36:17,172][20381] Updated weights for policy 1, policy_version 1580156 (0.0007) [2023-12-27 16:36:17,221][20381] Updated weights for policy 1, policy_version 1580166 (0.0005) [2023-12-27 16:36:17,281][20381] Updated weights for policy 1, policy_version 1580176 (0.0006) [2023-12-27 16:36:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 809607168. Throughput: 0: 10276.0, 1: 10184.0. Samples: 809613308. Policy #0 lag: (min: 9.0, avg: 36.4, max: 41.0) [2023-12-27 16:36:17,363][19170] Avg episode reward: [(0, '990.290'), (1, '824.170')] [2023-12-27 16:36:17,627][20347] Updated weights for policy 0, policy_version 1581877 (0.0009) [2023-12-27 16:36:17,682][20347] Updated weights for policy 0, policy_version 1581887 (0.0010) [2023-12-27 16:36:17,734][20347] Updated weights for policy 0, policy_version 1581897 (0.0009) [2023-12-27 16:36:17,944][20381] Updated weights for policy 1, policy_version 1580186 (0.0007) [2023-12-27 16:36:17,997][20381] Updated weights for policy 1, policy_version 1580196 (0.0007) [2023-12-27 16:36:18,051][20381] Updated weights for policy 1, policy_version 1580206 (0.0006) [2023-12-27 16:36:18,101][20381] Updated weights for policy 1, policy_version 1580216 (0.0008) [2023-12-27 16:36:18,384][20347] Updated weights for policy 0, policy_version 1581907 (0.0009) [2023-12-27 16:36:18,444][20347] Updated weights for policy 0, policy_version 1581917 (0.0006) [2023-12-27 16:36:18,504][20347] Updated weights for policy 0, policy_version 1581927 (0.0005) [2023-12-27 16:36:18,765][20381] Updated weights for policy 1, policy_version 1580226 (0.0009) [2023-12-27 16:36:18,822][20381] Updated weights for policy 1, policy_version 1580236 (0.0007) [2023-12-27 16:36:18,885][20381] Updated weights for policy 1, policy_version 1580246 (0.0007) [2023-12-27 16:36:19,136][20347] Updated weights for policy 0, policy_version 1581937 (0.0010) [2023-12-27 16:36:19,188][20347] Updated weights for policy 0, policy_version 1581947 (0.0009) [2023-12-27 16:36:19,253][20347] Updated weights for policy 0, policy_version 1581957 (0.0010) [2023-12-27 16:36:19,311][20347] Updated weights for policy 0, policy_version 1581967 (0.0006) [2023-12-27 16:36:19,547][20381] Updated weights for policy 1, policy_version 1580256 (0.0009) [2023-12-27 16:36:19,608][20381] Updated weights for policy 1, policy_version 1580266 (0.0008) [2023-12-27 16:36:19,669][20381] Updated weights for policy 1, policy_version 1580276 (0.0006) [2023-12-27 16:36:20,087][20347] Updated weights for policy 0, policy_version 1581977 (0.0009) [2023-12-27 16:36:20,151][20347] Updated weights for policy 0, policy_version 1581987 (0.0008) [2023-12-27 16:36:20,206][20347] Updated weights for policy 0, policy_version 1581997 (0.0008) [2023-12-27 16:36:20,345][20381] Updated weights for policy 1, policy_version 1580286 (0.0007) [2023-12-27 16:36:20,403][20381] Updated weights for policy 1, policy_version 1580296 (0.0008) [2023-12-27 16:36:20,463][20381] Updated weights for policy 1, policy_version 1580306 (0.0008) [2023-12-27 16:36:20,911][20347] Updated weights for policy 0, policy_version 1582007 (0.0010) [2023-12-27 16:36:20,964][20347] Updated weights for policy 0, policy_version 1582017 (0.0011) [2023-12-27 16:36:21,019][20347] Updated weights for policy 0, policy_version 1582027 (0.0010) [2023-12-27 16:36:21,204][20381] Updated weights for policy 1, policy_version 1580316 (0.0008) [2023-12-27 16:36:21,267][20381] Updated weights for policy 1, policy_version 1580326 (0.0008) [2023-12-27 16:36:21,327][20381] Updated weights for policy 1, policy_version 1580336 (0.0008) [2023-12-27 16:36:21,732][20347] Updated weights for policy 0, policy_version 1582037 (0.0010) [2023-12-27 16:36:21,794][20347] Updated weights for policy 0, policy_version 1582047 (0.0011) [2023-12-27 16:36:21,852][20347] Updated weights for policy 0, policy_version 1582057 (0.0010) [2023-12-27 16:36:22,101][20381] Updated weights for policy 1, policy_version 1580346 (0.0009) [2023-12-27 16:36:22,162][20381] Updated weights for policy 1, policy_version 1580356 (0.0008) [2023-12-27 16:36:22,215][20381] Updated weights for policy 1, policy_version 1580366 (0.0007) [2023-12-27 16:36:22,277][20381] Updated weights for policy 1, policy_version 1580376 (0.0008) [2023-12-27 16:36:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 809705472. Throughput: 0: 10287.2, 1: 10209.1. Samples: 809675836. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:22,363][19170] Avg episode reward: [(0, '961.190'), (1, '756.650')] [2023-12-27 16:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001580376_404619264.pth... [2023-12-27 16:36:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001582064_405086208.pth... [2023-12-27 16:36:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001579160_404307968.pth [2023-12-27 16:36:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001580880_404783104.pth [2023-12-27 16:36:22,565][20347] Updated weights for policy 0, policy_version 1582067 (0.0009) [2023-12-27 16:36:22,633][20347] Updated weights for policy 0, policy_version 1582077 (0.0006) [2023-12-27 16:36:22,694][20347] Updated weights for policy 0, policy_version 1582087 (0.0011) [2023-12-27 16:36:22,996][20381] Updated weights for policy 1, policy_version 1580386 (0.0006) [2023-12-27 16:36:23,054][20381] Updated weights for policy 1, policy_version 1580396 (0.0011) [2023-12-27 16:36:23,114][20381] Updated weights for policy 1, policy_version 1580406 (0.0008) [2023-12-27 16:36:23,346][20347] Updated weights for policy 0, policy_version 1582097 (0.0011) [2023-12-27 16:36:23,394][20347] Updated weights for policy 0, policy_version 1582107 (0.0010) [2023-12-27 16:36:23,445][20347] Updated weights for policy 0, policy_version 1582117 (0.0010) [2023-12-27 16:36:23,489][20347] Updated weights for policy 0, policy_version 1582127 (0.0010) [2023-12-27 16:36:23,749][20381] Updated weights for policy 1, policy_version 1580416 (0.0009) [2023-12-27 16:36:23,803][20381] Updated weights for policy 1, policy_version 1580426 (0.0007) [2023-12-27 16:36:23,854][20381] Updated weights for policy 1, policy_version 1580436 (0.0005) [2023-12-27 16:36:24,187][20347] Updated weights for policy 0, policy_version 1582137 (0.0010) [2023-12-27 16:36:24,240][20347] Updated weights for policy 0, policy_version 1582147 (0.0006) [2023-12-27 16:36:24,291][20347] Updated weights for policy 0, policy_version 1582157 (0.0005) [2023-12-27 16:36:24,460][20381] Updated weights for policy 1, policy_version 1580446 (0.0008) [2023-12-27 16:36:24,511][20381] Updated weights for policy 1, policy_version 1580456 (0.0010) [2023-12-27 16:36:24,564][20381] Updated weights for policy 1, policy_version 1580466 (0.0010) [2023-12-27 16:36:24,887][20347] Updated weights for policy 0, policy_version 1582167 (0.0006) [2023-12-27 16:36:24,943][20347] Updated weights for policy 0, policy_version 1582177 (0.0005) [2023-12-27 16:36:25,005][20347] Updated weights for policy 0, policy_version 1582187 (0.0005) [2023-12-27 16:36:25,186][20381] Updated weights for policy 1, policy_version 1580476 (0.0010) [2023-12-27 16:36:25,237][20381] Updated weights for policy 1, policy_version 1580486 (0.0010) [2023-12-27 16:36:25,290][20381] Updated weights for policy 1, policy_version 1580496 (0.0010) [2023-12-27 16:36:25,564][20347] Updated weights for policy 0, policy_version 1582197 (0.0007) [2023-12-27 16:36:25,624][20347] Updated weights for policy 0, policy_version 1582207 (0.0007) [2023-12-27 16:36:25,676][20347] Updated weights for policy 0, policy_version 1582217 (0.0005) [2023-12-27 16:36:25,975][20381] Updated weights for policy 1, policy_version 1580506 (0.0008) [2023-12-27 16:36:26,023][20381] Updated weights for policy 1, policy_version 1580516 (0.0008) [2023-12-27 16:36:26,073][20381] Updated weights for policy 1, policy_version 1580526 (0.0008) [2023-12-27 16:36:26,125][20381] Updated weights for policy 1, policy_version 1580536 (0.0008) [2023-12-27 16:36:26,260][20347] Updated weights for policy 0, policy_version 1582227 (0.0006) [2023-12-27 16:36:26,321][20347] Updated weights for policy 0, policy_version 1582237 (0.0007) [2023-12-27 16:36:26,370][20347] Updated weights for policy 0, policy_version 1582247 (0.0006) [2023-12-27 16:36:26,748][20381] Updated weights for policy 1, policy_version 1580546 (0.0005) [2023-12-27 16:36:26,809][20381] Updated weights for policy 1, policy_version 1580556 (0.0005) [2023-12-27 16:36:26,867][20381] Updated weights for policy 1, policy_version 1580566 (0.0005) [2023-12-27 16:36:27,102][20347] Updated weights for policy 0, policy_version 1582257 (0.0008) [2023-12-27 16:36:27,166][20347] Updated weights for policy 0, policy_version 1582267 (0.0007) [2023-12-27 16:36:27,222][20347] Updated weights for policy 0, policy_version 1582277 (0.0009) [2023-12-27 16:36:27,272][20347] Updated weights for policy 0, policy_version 1582287 (0.0008) [2023-12-27 16:36:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 809811968. Throughput: 0: 10257.2, 1: 10235.0. Samples: 809799144. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:27,363][19170] Avg episode reward: [(0, '941.670'), (1, '746.570')] [2023-12-27 16:36:27,462][20381] Updated weights for policy 1, policy_version 1580576 (0.0007) [2023-12-27 16:36:27,523][20381] Updated weights for policy 1, policy_version 1580586 (0.0005) [2023-12-27 16:36:27,580][20381] Updated weights for policy 1, policy_version 1580596 (0.0006) [2023-12-27 16:36:28,002][20347] Updated weights for policy 0, policy_version 1582297 (0.0006) [2023-12-27 16:36:28,048][20347] Updated weights for policy 0, policy_version 1582307 (0.0005) [2023-12-27 16:36:28,114][20347] Updated weights for policy 0, policy_version 1582317 (0.0008) [2023-12-27 16:36:28,239][20381] Updated weights for policy 1, policy_version 1580606 (0.0005) [2023-12-27 16:36:28,296][20381] Updated weights for policy 1, policy_version 1580616 (0.0005) [2023-12-27 16:36:28,353][20381] Updated weights for policy 1, policy_version 1580626 (0.0007) [2023-12-27 16:36:28,674][20347] Updated weights for policy 0, policy_version 1582327 (0.0009) [2023-12-27 16:36:28,728][20347] Updated weights for policy 0, policy_version 1582337 (0.0005) [2023-12-27 16:36:28,782][20347] Updated weights for policy 0, policy_version 1582347 (0.0009) [2023-12-27 16:36:29,014][20381] Updated weights for policy 1, policy_version 1580636 (0.0007) [2023-12-27 16:36:29,067][20381] Updated weights for policy 1, policy_version 1580646 (0.0009) [2023-12-27 16:36:29,118][20381] Updated weights for policy 1, policy_version 1580656 (0.0010) [2023-12-27 16:36:29,452][20347] Updated weights for policy 0, policy_version 1582357 (0.0008) [2023-12-27 16:36:29,500][20347] Updated weights for policy 0, policy_version 1582367 (0.0005) [2023-12-27 16:36:29,559][20347] Updated weights for policy 0, policy_version 1582377 (0.0005) [2023-12-27 16:36:29,809][20381] Updated weights for policy 1, policy_version 1580666 (0.0009) [2023-12-27 16:36:29,869][20381] Updated weights for policy 1, policy_version 1580676 (0.0007) [2023-12-27 16:36:29,936][20381] Updated weights for policy 1, policy_version 1580686 (0.0009) [2023-12-27 16:36:29,989][20381] Updated weights for policy 1, policy_version 1580696 (0.0008) [2023-12-27 16:36:30,222][20347] Updated weights for policy 0, policy_version 1582387 (0.0006) [2023-12-27 16:36:30,282][20347] Updated weights for policy 0, policy_version 1582397 (0.0007) [2023-12-27 16:36:30,343][20347] Updated weights for policy 0, policy_version 1582407 (0.0008) [2023-12-27 16:36:30,690][20381] Updated weights for policy 1, policy_version 1580706 (0.0008) [2023-12-27 16:36:30,745][20381] Updated weights for policy 1, policy_version 1580716 (0.0007) [2023-12-27 16:36:30,800][20381] Updated weights for policy 1, policy_version 1580726 (0.0007) [2023-12-27 16:36:30,959][20347] Updated weights for policy 0, policy_version 1582417 (0.0007) [2023-12-27 16:36:31,012][20347] Updated weights for policy 0, policy_version 1582427 (0.0009) [2023-12-27 16:36:31,079][20347] Updated weights for policy 0, policy_version 1582437 (0.0008) [2023-12-27 16:36:31,153][20347] Updated weights for policy 0, policy_version 1582447 (0.0007) [2023-12-27 16:36:31,487][20381] Updated weights for policy 1, policy_version 1580736 (0.0008) [2023-12-27 16:36:31,548][20381] Updated weights for policy 1, policy_version 1580746 (0.0010) [2023-12-27 16:36:31,606][20381] Updated weights for policy 1, policy_version 1580756 (0.0011) [2023-12-27 16:36:31,874][20347] Updated weights for policy 0, policy_version 1582457 (0.0010) [2023-12-27 16:36:31,926][20347] Updated weights for policy 0, policy_version 1582467 (0.0010) [2023-12-27 16:36:31,984][20347] Updated weights for policy 0, policy_version 1582477 (0.0010) [2023-12-27 16:36:32,333][20381] Updated weights for policy 1, policy_version 1580766 (0.0009) [2023-12-27 16:36:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 809910272. Throughput: 0: 10284.4, 1: 10263.0. Samples: 809923148. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:32,363][19170] Avg episode reward: [(0, '951.250'), (1, '774.920')] [2023-12-27 16:36:32,391][20381] Updated weights for policy 1, policy_version 1580776 (0.0008) [2023-12-27 16:36:32,444][20381] Updated weights for policy 1, policy_version 1580786 (0.0008) [2023-12-27 16:36:32,669][20347] Updated weights for policy 0, policy_version 1582487 (0.0008) [2023-12-27 16:36:32,728][20347] Updated weights for policy 0, policy_version 1582497 (0.0008) [2023-12-27 16:36:32,786][20347] Updated weights for policy 0, policy_version 1582507 (0.0007) [2023-12-27 16:36:33,173][20381] Updated weights for policy 1, policy_version 1580796 (0.0007) [2023-12-27 16:36:33,216][20381] Updated weights for policy 1, policy_version 1580806 (0.0007) [2023-12-27 16:36:33,265][20381] Updated weights for policy 1, policy_version 1580816 (0.0007) [2023-12-27 16:36:33,362][20347] Updated weights for policy 0, policy_version 1582517 (0.0007) [2023-12-27 16:36:33,413][20347] Updated weights for policy 0, policy_version 1582527 (0.0007) [2023-12-27 16:36:33,468][20347] Updated weights for policy 0, policy_version 1582537 (0.0008) [2023-12-27 16:36:34,001][20381] Updated weights for policy 1, policy_version 1580826 (0.0009) [2023-12-27 16:36:34,040][20347] Updated weights for policy 0, policy_version 1582547 (0.0007) [2023-12-27 16:36:34,056][20381] Updated weights for policy 1, policy_version 1580836 (0.0007) [2023-12-27 16:36:34,094][20347] Updated weights for policy 0, policy_version 1582557 (0.0006) [2023-12-27 16:36:34,110][20381] Updated weights for policy 1, policy_version 1580846 (0.0009) [2023-12-27 16:36:34,150][20347] Updated weights for policy 0, policy_version 1582567 (0.0006) [2023-12-27 16:36:34,173][20381] Updated weights for policy 1, policy_version 1580856 (0.0007) [2023-12-27 16:36:34,818][20347] Updated weights for policy 0, policy_version 1582577 (0.0007) [2023-12-27 16:36:34,843][20381] Updated weights for policy 1, policy_version 1580866 (0.0008) [2023-12-27 16:36:34,864][20347] Updated weights for policy 0, policy_version 1582587 (0.0005) [2023-12-27 16:36:34,898][20381] Updated weights for policy 1, policy_version 1580876 (0.0007) [2023-12-27 16:36:34,916][20347] Updated weights for policy 0, policy_version 1582597 (0.0005) [2023-12-27 16:36:34,957][20381] Updated weights for policy 1, policy_version 1580886 (0.0007) [2023-12-27 16:36:34,967][20347] Updated weights for policy 0, policy_version 1582607 (0.0005) [2023-12-27 16:36:35,537][20381] Updated weights for policy 1, policy_version 1580896 (0.0007) [2023-12-27 16:36:35,597][20381] Updated weights for policy 1, policy_version 1580906 (0.0008) [2023-12-27 16:36:35,656][20381] Updated weights for policy 1, policy_version 1580916 (0.0007) [2023-12-27 16:36:35,669][20347] Updated weights for policy 0, policy_version 1582617 (0.0006) [2023-12-27 16:36:35,726][20347] Updated weights for policy 0, policy_version 1582627 (0.0007) [2023-12-27 16:36:35,785][20347] Updated weights for policy 0, policy_version 1582637 (0.0008) [2023-12-27 16:36:36,355][20381] Updated weights for policy 1, policy_version 1580926 (0.0006) [2023-12-27 16:36:36,417][20381] Updated weights for policy 1, policy_version 1580936 (0.0007) [2023-12-27 16:36:36,478][20381] Updated weights for policy 1, policy_version 1580946 (0.0007) [2023-12-27 16:36:36,491][20347] Updated weights for policy 0, policy_version 1582647 (0.0008) [2023-12-27 16:36:36,547][20347] Updated weights for policy 0, policy_version 1582657 (0.0008) [2023-12-27 16:36:36,602][20347] Updated weights for policy 0, policy_version 1582667 (0.0008) [2023-12-27 16:36:37,179][20381] Updated weights for policy 1, policy_version 1580956 (0.0006) [2023-12-27 16:36:37,241][20381] Updated weights for policy 1, policy_version 1580966 (0.0007) [2023-12-27 16:36:37,279][20347] Updated weights for policy 0, policy_version 1582677 (0.0007) [2023-12-27 16:36:37,296][20381] Updated weights for policy 1, policy_version 1580976 (0.0007) [2023-12-27 16:36:37,337][20347] Updated weights for policy 0, policy_version 1582687 (0.0006) [2023-12-27 16:36:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 810016768. Throughput: 0: 10380.8, 1: 10249.0. Samples: 809986708. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:37,363][19170] Avg episode reward: [(0, '970.820'), (1, '785.740')] [2023-12-27 16:36:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001580984_404774912.pth... [2023-12-27 16:36:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001579768_404463616.pth [2023-12-27 16:36:37,396][20347] Updated weights for policy 0, policy_version 1582697 (0.0007) [2023-12-27 16:36:37,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001582704_405250048.pth... [2023-12-27 16:36:37,439][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001581456_404930560.pth [2023-12-27 16:36:37,874][20381] Updated weights for policy 1, policy_version 1580986 (0.0007) [2023-12-27 16:36:37,925][20381] Updated weights for policy 1, policy_version 1580996 (0.0005) [2023-12-27 16:36:37,929][20347] Updated weights for policy 0, policy_version 1582707 (0.0006) [2023-12-27 16:36:37,979][20381] Updated weights for policy 1, policy_version 1581006 (0.0005) [2023-12-27 16:36:37,984][20347] Updated weights for policy 0, policy_version 1582717 (0.0007) [2023-12-27 16:36:38,036][20381] Updated weights for policy 1, policy_version 1581016 (0.0005) [2023-12-27 16:36:38,041][20347] Updated weights for policy 0, policy_version 1582727 (0.0006) [2023-12-27 16:36:38,648][20381] Updated weights for policy 1, policy_version 1581026 (0.0007) [2023-12-27 16:36:38,700][20381] Updated weights for policy 1, policy_version 1581036 (0.0007) [2023-12-27 16:36:38,733][20347] Updated weights for policy 0, policy_version 1582737 (0.0005) [2023-12-27 16:36:38,758][20381] Updated weights for policy 1, policy_version 1581046 (0.0006) [2023-12-27 16:36:38,793][20347] Updated weights for policy 0, policy_version 1582747 (0.0007) [2023-12-27 16:36:38,856][20347] Updated weights for policy 0, policy_version 1582757 (0.0007) [2023-12-27 16:36:38,912][20347] Updated weights for policy 0, policy_version 1582767 (0.0008) [2023-12-27 16:36:39,528][20381] Updated weights for policy 1, policy_version 1581056 (0.0008) [2023-12-27 16:36:39,591][20381] Updated weights for policy 1, policy_version 1581066 (0.0008) [2023-12-27 16:36:39,652][20381] Updated weights for policy 1, policy_version 1581076 (0.0008) [2023-12-27 16:36:39,680][20347] Updated weights for policy 0, policy_version 1582777 (0.0008) [2023-12-27 16:36:39,742][20347] Updated weights for policy 0, policy_version 1582787 (0.0008) [2023-12-27 16:36:39,800][20347] Updated weights for policy 0, policy_version 1582797 (0.0007) [2023-12-27 16:36:40,338][20381] Updated weights for policy 1, policy_version 1581086 (0.0007) [2023-12-27 16:36:40,401][20381] Updated weights for policy 1, policy_version 1581096 (0.0008) [2023-12-27 16:36:40,457][20381] Updated weights for policy 1, policy_version 1581106 (0.0007) [2023-12-27 16:36:40,562][20347] Updated weights for policy 0, policy_version 1582807 (0.0008) [2023-12-27 16:36:40,618][20347] Updated weights for policy 0, policy_version 1582817 (0.0007) [2023-12-27 16:36:40,677][20347] Updated weights for policy 0, policy_version 1582827 (0.0008) [2023-12-27 16:36:41,075][20381] Updated weights for policy 1, policy_version 1581116 (0.0008) [2023-12-27 16:36:41,144][20381] Updated weights for policy 1, policy_version 1581126 (0.0008) [2023-12-27 16:36:41,208][20381] Updated weights for policy 1, policy_version 1581136 (0.0008) [2023-12-27 16:36:41,424][20347] Updated weights for policy 0, policy_version 1582837 (0.0006) [2023-12-27 16:36:41,480][20347] Updated weights for policy 0, policy_version 1582847 (0.0007) [2023-12-27 16:36:41,533][20347] Updated weights for policy 0, policy_version 1582857 (0.0008) [2023-12-27 16:36:41,937][20381] Updated weights for policy 1, policy_version 1581146 (0.0008) [2023-12-27 16:36:42,000][20381] Updated weights for policy 1, policy_version 1581156 (0.0008) [2023-12-27 16:36:42,058][20381] Updated weights for policy 1, policy_version 1581166 (0.0008) [2023-12-27 16:36:42,108][20381] Updated weights for policy 1, policy_version 1581176 (0.0007) [2023-12-27 16:36:42,245][20347] Updated weights for policy 0, policy_version 1582867 (0.0006) [2023-12-27 16:36:42,312][20347] Updated weights for policy 0, policy_version 1582877 (0.0008) [2023-12-27 16:36:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 810115072. Throughput: 0: 10314.1, 1: 10239.6. Samples: 810108364. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:42,363][19170] Avg episode reward: [(0, '960.950'), (1, '796.270')] [2023-12-27 16:36:42,383][20347] Updated weights for policy 0, policy_version 1582887 (0.0007) [2023-12-27 16:36:42,776][20381] Updated weights for policy 1, policy_version 1581186 (0.0008) [2023-12-27 16:36:42,832][20381] Updated weights for policy 1, policy_version 1581196 (0.0008) [2023-12-27 16:36:42,891][20381] Updated weights for policy 1, policy_version 1581206 (0.0007) [2023-12-27 16:36:43,049][20347] Updated weights for policy 0, policy_version 1582897 (0.0008) [2023-12-27 16:36:43,097][20347] Updated weights for policy 0, policy_version 1582907 (0.0005) [2023-12-27 16:36:43,158][20347] Updated weights for policy 0, policy_version 1582917 (0.0007) [2023-12-27 16:36:43,219][20347] Updated weights for policy 0, policy_version 1582927 (0.0007) [2023-12-27 16:36:43,568][20381] Updated weights for policy 1, policy_version 1581216 (0.0006) [2023-12-27 16:36:43,616][20381] Updated weights for policy 1, policy_version 1581226 (0.0005) [2023-12-27 16:36:43,673][20381] Updated weights for policy 1, policy_version 1581236 (0.0006) [2023-12-27 16:36:43,901][20347] Updated weights for policy 0, policy_version 1582937 (0.0008) [2023-12-27 16:36:43,953][20347] Updated weights for policy 0, policy_version 1582947 (0.0007) [2023-12-27 16:36:44,010][20347] Updated weights for policy 0, policy_version 1582957 (0.0007) [2023-12-27 16:36:44,269][20381] Updated weights for policy 1, policy_version 1581246 (0.0007) [2023-12-27 16:36:44,322][20381] Updated weights for policy 1, policy_version 1581256 (0.0007) [2023-12-27 16:36:44,365][20381] Updated weights for policy 1, policy_version 1581266 (0.0006) [2023-12-27 16:36:44,757][20347] Updated weights for policy 0, policy_version 1582967 (0.0008) [2023-12-27 16:36:44,824][20347] Updated weights for policy 0, policy_version 1582977 (0.0008) [2023-12-27 16:36:44,887][20347] Updated weights for policy 0, policy_version 1582987 (0.0007) [2023-12-27 16:36:44,977][20381] Updated weights for policy 1, policy_version 1581276 (0.0007) [2023-12-27 16:36:45,039][20381] Updated weights for policy 1, policy_version 1581286 (0.0008) [2023-12-27 16:36:45,103][20381] Updated weights for policy 1, policy_version 1581296 (0.0005) [2023-12-27 16:36:45,611][20347] Updated weights for policy 0, policy_version 1582997 (0.0007) [2023-12-27 16:36:45,662][20347] Updated weights for policy 0, policy_version 1583007 (0.0008) [2023-12-27 16:36:45,673][20381] Updated weights for policy 1, policy_version 1581306 (0.0005) [2023-12-27 16:36:45,716][20347] Updated weights for policy 0, policy_version 1583017 (0.0007) [2023-12-27 16:36:45,737][20381] Updated weights for policy 1, policy_version 1581316 (0.0008) [2023-12-27 16:36:45,801][20381] Updated weights for policy 1, policy_version 1581326 (0.0008) [2023-12-27 16:36:45,867][20381] Updated weights for policy 1, policy_version 1581336 (0.0008) [2023-12-27 16:36:46,431][20381] Updated weights for policy 1, policy_version 1581346 (0.0006) [2023-12-27 16:36:46,444][20347] Updated weights for policy 0, policy_version 1583027 (0.0007) [2023-12-27 16:36:46,477][20381] Updated weights for policy 1, policy_version 1581356 (0.0005) [2023-12-27 16:36:46,493][20347] Updated weights for policy 0, policy_version 1583037 (0.0008) [2023-12-27 16:36:46,523][20381] Updated weights for policy 1, policy_version 1581366 (0.0005) [2023-12-27 16:36:46,542][20347] Updated weights for policy 0, policy_version 1583047 (0.0008) [2023-12-27 16:36:47,081][20381] Updated weights for policy 1, policy_version 1581376 (0.0005) [2023-12-27 16:36:47,141][20381] Updated weights for policy 1, policy_version 1581386 (0.0005) [2023-12-27 16:36:47,198][20381] Updated weights for policy 1, policy_version 1581396 (0.0005) [2023-12-27 16:36:47,233][20347] Updated weights for policy 0, policy_version 1583057 (0.0008) [2023-12-27 16:36:47,287][20347] Updated weights for policy 0, policy_version 1583067 (0.0010) [2023-12-27 16:36:47,347][20347] Updated weights for policy 0, policy_version 1583077 (0.0009) [2023-12-27 16:36:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 810221568. Throughput: 0: 10291.8, 1: 10364.6. Samples: 810233080. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:47,363][19170] Avg episode reward: [(0, '921.920'), (1, '755.820')] [2023-12-27 16:36:47,399][20347] Updated weights for policy 0, policy_version 1583087 (0.0005) [2023-12-27 16:36:47,786][20381] Updated weights for policy 1, policy_version 1581406 (0.0005) [2023-12-27 16:36:47,838][20381] Updated weights for policy 1, policy_version 1581416 (0.0007) [2023-12-27 16:36:47,897][20381] Updated weights for policy 1, policy_version 1581426 (0.0009) [2023-12-27 16:36:48,035][20347] Updated weights for policy 0, policy_version 1583097 (0.0006) [2023-12-27 16:36:48,087][20347] Updated weights for policy 0, policy_version 1583107 (0.0005) [2023-12-27 16:36:48,142][20347] Updated weights for policy 0, policy_version 1583117 (0.0005) [2023-12-27 16:36:48,544][20381] Updated weights for policy 1, policy_version 1581436 (0.0007) [2023-12-27 16:36:48,591][20381] Updated weights for policy 1, policy_version 1581446 (0.0007) [2023-12-27 16:36:48,645][20381] Updated weights for policy 1, policy_version 1581456 (0.0007) [2023-12-27 16:36:48,811][20347] Updated weights for policy 0, policy_version 1583127 (0.0007) [2023-12-27 16:36:48,863][20347] Updated weights for policy 0, policy_version 1583137 (0.0007) [2023-12-27 16:36:48,912][20347] Updated weights for policy 0, policy_version 1583147 (0.0005) [2023-12-27 16:36:49,383][20381] Updated weights for policy 1, policy_version 1581466 (0.0007) [2023-12-27 16:36:49,440][20381] Updated weights for policy 1, policy_version 1581476 (0.0008) [2023-12-27 16:36:49,494][20381] Updated weights for policy 1, policy_version 1581486 (0.0008) [2023-12-27 16:36:49,549][20381] Updated weights for policy 1, policy_version 1581496 (0.0008) [2023-12-27 16:36:49,613][20347] Updated weights for policy 0, policy_version 1583157 (0.0006) [2023-12-27 16:36:49,668][20347] Updated weights for policy 0, policy_version 1583167 (0.0009) [2023-12-27 16:36:49,717][20347] Updated weights for policy 0, policy_version 1583177 (0.0010) [2023-12-27 16:36:50,242][20381] Updated weights for policy 1, policy_version 1581506 (0.0006) [2023-12-27 16:36:50,297][20381] Updated weights for policy 1, policy_version 1581516 (0.0006) [2023-12-27 16:36:50,356][20381] Updated weights for policy 1, policy_version 1581526 (0.0007) [2023-12-27 16:36:50,442][20347] Updated weights for policy 0, policy_version 1583187 (0.0009) [2023-12-27 16:36:50,496][20347] Updated weights for policy 0, policy_version 1583197 (0.0007) [2023-12-27 16:36:50,549][20347] Updated weights for policy 0, policy_version 1583207 (0.0007) [2023-12-27 16:36:50,944][20381] Updated weights for policy 1, policy_version 1581536 (0.0007) [2023-12-27 16:36:50,997][20381] Updated weights for policy 1, policy_version 1581546 (0.0008) [2023-12-27 16:36:51,062][20381] Updated weights for policy 1, policy_version 1581556 (0.0007) [2023-12-27 16:36:51,194][20347] Updated weights for policy 0, policy_version 1583217 (0.0008) [2023-12-27 16:36:51,250][20347] Updated weights for policy 0, policy_version 1583227 (0.0006) [2023-12-27 16:36:51,313][20347] Updated weights for policy 0, policy_version 1583237 (0.0006) [2023-12-27 16:36:51,337][20291] KL-divergence is very high: 111.0154 [2023-12-27 16:36:51,378][20347] Updated weights for policy 0, policy_version 1583247 (0.0008) [2023-12-27 16:36:51,784][20381] Updated weights for policy 1, policy_version 1581566 (0.0007) [2023-12-27 16:36:51,843][20381] Updated weights for policy 1, policy_version 1581576 (0.0006) [2023-12-27 16:36:51,906][20381] Updated weights for policy 1, policy_version 1581586 (0.0008) [2023-12-27 16:36:52,017][20347] Updated weights for policy 0, policy_version 1583257 (0.0008) [2023-12-27 16:36:52,085][20347] Updated weights for policy 0, policy_version 1583267 (0.0009) [2023-12-27 16:36:52,139][20347] Updated weights for policy 0, policy_version 1583277 (0.0006) [2023-12-27 16:36:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 810328064. Throughput: 0: 10329.1, 1: 10336.4. Samples: 810294684. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:52,363][19170] Avg episode reward: [(0, '961.040'), (1, '736.320')] [2023-12-27 16:36:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001583280_405397504.pth... [2023-12-27 16:36:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001581592_404930560.pth... [2023-12-27 16:36:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001582064_405086208.pth [2023-12-27 16:36:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001580376_404619264.pth [2023-12-27 16:36:52,583][20381] Updated weights for policy 1, policy_version 1581596 (0.0007) [2023-12-27 16:36:52,639][20381] Updated weights for policy 1, policy_version 1581606 (0.0007) [2023-12-27 16:36:52,693][20381] Updated weights for policy 1, policy_version 1581616 (0.0007) [2023-12-27 16:36:52,856][20347] Updated weights for policy 0, policy_version 1583287 (0.0007) [2023-12-27 16:36:52,908][20347] Updated weights for policy 0, policy_version 1583297 (0.0007) [2023-12-27 16:36:52,960][20347] Updated weights for policy 0, policy_version 1583307 (0.0007) [2023-12-27 16:36:53,356][20381] Updated weights for policy 1, policy_version 1581626 (0.0007) [2023-12-27 16:36:53,401][20381] Updated weights for policy 1, policy_version 1581636 (0.0006) [2023-12-27 16:36:53,453][20381] Updated weights for policy 1, policy_version 1581646 (0.0008) [2023-12-27 16:36:53,508][20381] Updated weights for policy 1, policy_version 1581656 (0.0007) [2023-12-27 16:36:53,642][20347] Updated weights for policy 0, policy_version 1583317 (0.0007) [2023-12-27 16:36:53,694][20347] Updated weights for policy 0, policy_version 1583327 (0.0008) [2023-12-27 16:36:53,743][20347] Updated weights for policy 0, policy_version 1583337 (0.0009) [2023-12-27 16:36:54,102][20381] Updated weights for policy 1, policy_version 1581666 (0.0008) [2023-12-27 16:36:54,153][20381] Updated weights for policy 1, policy_version 1581676 (0.0009) [2023-12-27 16:36:54,210][20381] Updated weights for policy 1, policy_version 1581686 (0.0008) [2023-12-27 16:36:54,466][20347] Updated weights for policy 0, policy_version 1583347 (0.0007) [2023-12-27 16:36:54,526][20347] Updated weights for policy 0, policy_version 1583357 (0.0007) [2023-12-27 16:36:54,580][20347] Updated weights for policy 0, policy_version 1583367 (0.0007) [2023-12-27 16:36:54,924][20381] Updated weights for policy 1, policy_version 1581696 (0.0008) [2023-12-27 16:36:54,983][20381] Updated weights for policy 1, policy_version 1581706 (0.0009) [2023-12-27 16:36:55,045][20381] Updated weights for policy 1, policy_version 1581716 (0.0008) [2023-12-27 16:36:55,281][20347] Updated weights for policy 0, policy_version 1583377 (0.0007) [2023-12-27 16:36:55,332][20347] Updated weights for policy 0, policy_version 1583387 (0.0008) [2023-12-27 16:36:55,390][20347] Updated weights for policy 0, policy_version 1583397 (0.0008) [2023-12-27 16:36:55,437][20347] Updated weights for policy 0, policy_version 1583407 (0.0007) [2023-12-27 16:36:55,705][20381] Updated weights for policy 1, policy_version 1581726 (0.0007) [2023-12-27 16:36:55,759][20381] Updated weights for policy 1, policy_version 1581736 (0.0008) [2023-12-27 16:36:55,808][20381] Updated weights for policy 1, policy_version 1581746 (0.0010) [2023-12-27 16:36:56,116][20347] Updated weights for policy 0, policy_version 1583417 (0.0008) [2023-12-27 16:36:56,173][20347] Updated weights for policy 0, policy_version 1583427 (0.0010) [2023-12-27 16:36:56,230][20347] Updated weights for policy 0, policy_version 1583437 (0.0010) [2023-12-27 16:36:56,471][20381] Updated weights for policy 1, policy_version 1581756 (0.0010) [2023-12-27 16:36:56,526][20381] Updated weights for policy 1, policy_version 1581766 (0.0005) [2023-12-27 16:36:56,574][20381] Updated weights for policy 1, policy_version 1581776 (0.0005) [2023-12-27 16:36:56,849][20347] Updated weights for policy 0, policy_version 1583447 (0.0010) [2023-12-27 16:36:56,906][20347] Updated weights for policy 0, policy_version 1583457 (0.0010) [2023-12-27 16:36:56,960][20347] Updated weights for policy 0, policy_version 1583467 (0.0007) [2023-12-27 16:36:57,098][20381] Updated weights for policy 1, policy_version 1581786 (0.0005) [2023-12-27 16:36:57,150][20381] Updated weights for policy 1, policy_version 1581796 (0.0009) [2023-12-27 16:36:57,198][20381] Updated weights for policy 1, policy_version 1581806 (0.0010) [2023-12-27 16:36:57,250][20381] Updated weights for policy 1, policy_version 1581816 (0.0011) [2023-12-27 16:36:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 810434560. Throughput: 0: 10279.1, 1: 10366.1. Samples: 810418724. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:36:57,363][19170] Avg episode reward: [(0, '980.520'), (1, '775.950')] [2023-12-27 16:36:57,536][20347] Updated weights for policy 0, policy_version 1583477 (0.0005) [2023-12-27 16:36:57,599][20347] Updated weights for policy 0, policy_version 1583487 (0.0007) [2023-12-27 16:36:57,653][20347] Updated weights for policy 0, policy_version 1583497 (0.0010) [2023-12-27 16:36:57,917][20381] Updated weights for policy 1, policy_version 1581826 (0.0008) [2023-12-27 16:36:57,963][20381] Updated weights for policy 1, policy_version 1581836 (0.0009) [2023-12-27 16:36:58,011][20381] Updated weights for policy 1, policy_version 1581846 (0.0007) [2023-12-27 16:36:58,325][20347] Updated weights for policy 0, policy_version 1583507 (0.0010) [2023-12-27 16:36:58,397][20347] Updated weights for policy 0, policy_version 1583517 (0.0007) [2023-12-27 16:36:58,465][20347] Updated weights for policy 0, policy_version 1583527 (0.0007) [2023-12-27 16:36:58,768][20381] Updated weights for policy 1, policy_version 1581856 (0.0008) [2023-12-27 16:36:58,830][20381] Updated weights for policy 1, policy_version 1581866 (0.0008) [2023-12-27 16:36:58,898][20381] Updated weights for policy 1, policy_version 1581876 (0.0010) [2023-12-27 16:36:59,245][20347] Updated weights for policy 0, policy_version 1583537 (0.0008) [2023-12-27 16:36:59,310][20347] Updated weights for policy 0, policy_version 1583547 (0.0009) [2023-12-27 16:36:59,376][20347] Updated weights for policy 0, policy_version 1583557 (0.0007) [2023-12-27 16:36:59,437][20347] Updated weights for policy 0, policy_version 1583567 (0.0006) [2023-12-27 16:36:59,667][20381] Updated weights for policy 1, policy_version 1581886 (0.0007) [2023-12-27 16:36:59,724][20381] Updated weights for policy 1, policy_version 1581896 (0.0008) [2023-12-27 16:36:59,773][20381] Updated weights for policy 1, policy_version 1581906 (0.0007) [2023-12-27 16:37:00,101][20347] Updated weights for policy 0, policy_version 1583577 (0.0009) [2023-12-27 16:37:00,163][20347] Updated weights for policy 0, policy_version 1583587 (0.0008) [2023-12-27 16:37:00,221][20347] Updated weights for policy 0, policy_version 1583597 (0.0007) [2023-12-27 16:37:00,474][20381] Updated weights for policy 1, policy_version 1581916 (0.0007) [2023-12-27 16:37:00,537][20381] Updated weights for policy 1, policy_version 1581926 (0.0006) [2023-12-27 16:37:00,592][20381] Updated weights for policy 1, policy_version 1581936 (0.0009) [2023-12-27 16:37:00,913][20347] Updated weights for policy 0, policy_version 1583607 (0.0006) [2023-12-27 16:37:00,961][20347] Updated weights for policy 0, policy_version 1583617 (0.0005) [2023-12-27 16:37:01,018][20347] Updated weights for policy 0, policy_version 1583627 (0.0005) [2023-12-27 16:37:01,229][20381] Updated weights for policy 1, policy_version 1581946 (0.0008) [2023-12-27 16:37:01,294][20381] Updated weights for policy 1, policy_version 1581956 (0.0008) [2023-12-27 16:37:01,338][20381] Updated weights for policy 1, policy_version 1581966 (0.0008) [2023-12-27 16:37:01,405][20381] Updated weights for policy 1, policy_version 1581976 (0.0008) [2023-12-27 16:37:01,660][20347] Updated weights for policy 0, policy_version 1583637 (0.0007) [2023-12-27 16:37:01,724][20347] Updated weights for policy 0, policy_version 1583647 (0.0006) [2023-12-27 16:37:01,787][20347] Updated weights for policy 0, policy_version 1583657 (0.0008) [2023-12-27 16:37:02,118][20381] Updated weights for policy 1, policy_version 1581986 (0.0007) [2023-12-27 16:37:02,171][20381] Updated weights for policy 1, policy_version 1581996 (0.0007) [2023-12-27 16:37:02,223][20381] Updated weights for policy 1, policy_version 1582006 (0.0008) [2023-12-27 16:37:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 810532864. Throughput: 0: 10233.8, 1: 10383.8. Samples: 810541104. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:02,363][19170] Avg episode reward: [(0, '990.200'), (1, '708.450')] [2023-12-27 16:37:02,494][20347] Updated weights for policy 0, policy_version 1583667 (0.0007) [2023-12-27 16:37:02,544][20347] Updated weights for policy 0, policy_version 1583677 (0.0007) [2023-12-27 16:37:02,600][20347] Updated weights for policy 0, policy_version 1583687 (0.0007) [2023-12-27 16:37:02,864][20381] Updated weights for policy 1, policy_version 1582016 (0.0006) [2023-12-27 16:37:02,912][20381] Updated weights for policy 1, policy_version 1582026 (0.0005) [2023-12-27 16:37:02,964][20381] Updated weights for policy 1, policy_version 1582036 (0.0007) [2023-12-27 16:37:03,321][20347] Updated weights for policy 0, policy_version 1583697 (0.0008) [2023-12-27 16:37:03,377][20347] Updated weights for policy 0, policy_version 1583707 (0.0008) [2023-12-27 16:37:03,440][20347] Updated weights for policy 0, policy_version 1583717 (0.0008) [2023-12-27 16:37:03,496][20347] Updated weights for policy 0, policy_version 1583727 (0.0007) [2023-12-27 16:37:03,589][20381] Updated weights for policy 1, policy_version 1582046 (0.0008) [2023-12-27 16:37:03,643][20381] Updated weights for policy 1, policy_version 1582056 (0.0007) [2023-12-27 16:37:03,698][20381] Updated weights for policy 1, policy_version 1582066 (0.0005) [2023-12-27 16:37:04,155][20347] Updated weights for policy 0, policy_version 1583737 (0.0008) [2023-12-27 16:37:04,205][20347] Updated weights for policy 0, policy_version 1583747 (0.0007) [2023-12-27 16:37:04,262][20347] Updated weights for policy 0, policy_version 1583757 (0.0006) [2023-12-27 16:37:04,366][20381] Updated weights for policy 1, policy_version 1582076 (0.0006) [2023-12-27 16:37:04,422][20381] Updated weights for policy 1, policy_version 1582086 (0.0008) [2023-12-27 16:37:04,476][20381] Updated weights for policy 1, policy_version 1582096 (0.0008) [2023-12-27 16:37:04,925][20347] Updated weights for policy 0, policy_version 1583767 (0.0007) [2023-12-27 16:37:04,972][20347] Updated weights for policy 0, policy_version 1583777 (0.0007) [2023-12-27 16:37:05,031][20347] Updated weights for policy 0, policy_version 1583787 (0.0007) [2023-12-27 16:37:05,143][20381] Updated weights for policy 1, policy_version 1582106 (0.0010) [2023-12-27 16:37:05,188][20381] Updated weights for policy 1, policy_version 1582116 (0.0010) [2023-12-27 16:37:05,238][20381] Updated weights for policy 1, policy_version 1582126 (0.0010) [2023-12-27 16:37:05,286][20381] Updated weights for policy 1, policy_version 1582136 (0.0010) [2023-12-27 16:37:05,727][20347] Updated weights for policy 0, policy_version 1583797 (0.0009) [2023-12-27 16:37:05,784][20347] Updated weights for policy 0, policy_version 1583807 (0.0010) [2023-12-27 16:37:05,844][20347] Updated weights for policy 0, policy_version 1583817 (0.0008) [2023-12-27 16:37:05,909][20381] Updated weights for policy 1, policy_version 1582146 (0.0007) [2023-12-27 16:37:05,961][20381] Updated weights for policy 1, policy_version 1582156 (0.0010) [2023-12-27 16:37:06,006][20381] Updated weights for policy 1, policy_version 1582166 (0.0007) [2023-12-27 16:37:06,584][20347] Updated weights for policy 0, policy_version 1583827 (0.0008) [2023-12-27 16:37:06,646][20347] Updated weights for policy 0, policy_version 1583837 (0.0007) [2023-12-27 16:37:06,669][20381] Updated weights for policy 1, policy_version 1582176 (0.0009) [2023-12-27 16:37:06,697][20347] Updated weights for policy 0, policy_version 1583847 (0.0007) [2023-12-27 16:37:06,733][20381] Updated weights for policy 1, policy_version 1582186 (0.0007) [2023-12-27 16:37:06,799][20381] Updated weights for policy 1, policy_version 1582196 (0.0011) [2023-12-27 16:37:07,308][20347] Updated weights for policy 0, policy_version 1583857 (0.0008) [2023-12-27 16:37:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 810631168. Throughput: 0: 10210.3, 1: 10403.0. Samples: 810603436. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:07,363][20347] Updated weights for policy 0, policy_version 1583867 (0.0010) [2023-12-27 16:37:07,363][19170] Avg episode reward: [(0, '980.420'), (1, '692.070')] [2023-12-27 16:37:07,390][20381] Updated weights for policy 1, policy_version 1582206 (0.0007) [2023-12-27 16:37:07,418][20347] Updated weights for policy 0, policy_version 1583877 (0.0010) [2023-12-27 16:37:07,452][20381] Updated weights for policy 1, policy_version 1582216 (0.0005) [2023-12-27 16:37:07,470][20347] Updated weights for policy 0, policy_version 1583887 (0.0010) [2023-12-27 16:37:07,473][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001583888_405553152.pth... [2023-12-27 16:37:07,477][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001582704_405250048.pth [2023-12-27 16:37:07,505][20381] Updated weights for policy 1, policy_version 1582226 (0.0010) [2023-12-27 16:37:07,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001582232_405094400.pth... [2023-12-27 16:37:07,536][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001580984_404774912.pth [2023-12-27 16:37:08,100][20347] Updated weights for policy 0, policy_version 1583897 (0.0009) [2023-12-27 16:37:08,125][20381] Updated weights for policy 1, policy_version 1582236 (0.0008) [2023-12-27 16:37:08,147][20347] Updated weights for policy 0, policy_version 1583907 (0.0006) [2023-12-27 16:37:08,180][20381] Updated weights for policy 1, policy_version 1582246 (0.0009) [2023-12-27 16:37:08,202][20347] Updated weights for policy 0, policy_version 1583917 (0.0006) [2023-12-27 16:37:08,233][20381] Updated weights for policy 1, policy_version 1582256 (0.0009) [2023-12-27 16:37:08,884][20347] Updated weights for policy 0, policy_version 1583927 (0.0007) [2023-12-27 16:37:08,921][20381] Updated weights for policy 1, policy_version 1582266 (0.0006) [2023-12-27 16:37:08,937][20347] Updated weights for policy 0, policy_version 1583937 (0.0006) [2023-12-27 16:37:08,976][20381] Updated weights for policy 1, policy_version 1582276 (0.0008) [2023-12-27 16:37:08,988][20347] Updated weights for policy 0, policy_version 1583947 (0.0006) [2023-12-27 16:37:09,032][20381] Updated weights for policy 1, policy_version 1582286 (0.0008) [2023-12-27 16:37:09,095][20381] Updated weights for policy 1, policy_version 1582296 (0.0008) [2023-12-27 16:37:09,677][20347] Updated weights for policy 0, policy_version 1583957 (0.0008) [2023-12-27 16:37:09,734][20347] Updated weights for policy 0, policy_version 1583967 (0.0007) [2023-12-27 16:37:09,789][20347] Updated weights for policy 0, policy_version 1583977 (0.0008) [2023-12-27 16:37:09,883][20381] Updated weights for policy 1, policy_version 1582306 (0.0007) [2023-12-27 16:37:09,951][20381] Updated weights for policy 1, policy_version 1582316 (0.0008) [2023-12-27 16:37:10,018][20381] Updated weights for policy 1, policy_version 1582326 (0.0008) [2023-12-27 16:37:10,512][20347] Updated weights for policy 0, policy_version 1583987 (0.0009) [2023-12-27 16:37:10,569][20347] Updated weights for policy 0, policy_version 1583997 (0.0008) [2023-12-27 16:37:10,626][20347] Updated weights for policy 0, policy_version 1584007 (0.0008) [2023-12-27 16:37:10,714][20381] Updated weights for policy 1, policy_version 1582336 (0.0006) [2023-12-27 16:37:10,768][20381] Updated weights for policy 1, policy_version 1582346 (0.0010) [2023-12-27 16:37:10,829][20381] Updated weights for policy 1, policy_version 1582356 (0.0008) [2023-12-27 16:37:11,405][20347] Updated weights for policy 0, policy_version 1584017 (0.0007) [2023-12-27 16:37:11,466][20347] Updated weights for policy 0, policy_version 1584027 (0.0008) [2023-12-27 16:37:11,523][20347] Updated weights for policy 0, policy_version 1584037 (0.0008) [2023-12-27 16:37:11,580][20347] Updated weights for policy 0, policy_version 1584047 (0.0008) [2023-12-27 16:37:11,605][20381] Updated weights for policy 1, policy_version 1582366 (0.0009) [2023-12-27 16:37:11,666][20381] Updated weights for policy 1, policy_version 1582376 (0.0007) [2023-12-27 16:37:11,732][20381] Updated weights for policy 1, policy_version 1582386 (0.0007) [2023-12-27 16:37:12,351][20347] Updated weights for policy 0, policy_version 1584057 (0.0007) [2023-12-27 16:37:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 810729472. Throughput: 0: 10176.9, 1: 10428.6. Samples: 810726392. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:12,363][19170] Avg episode reward: [(0, '970.610'), (1, '702.210')] [2023-12-27 16:37:12,415][20347] Updated weights for policy 0, policy_version 1584067 (0.0006) [2023-12-27 16:37:12,456][20381] Updated weights for policy 1, policy_version 1582396 (0.0007) [2023-12-27 16:37:12,473][20347] Updated weights for policy 0, policy_version 1584077 (0.0007) [2023-12-27 16:37:12,514][20381] Updated weights for policy 1, policy_version 1582406 (0.0007) [2023-12-27 16:37:12,571][20381] Updated weights for policy 1, policy_version 1582416 (0.0009) [2023-12-27 16:37:13,123][20347] Updated weights for policy 0, policy_version 1584087 (0.0007) [2023-12-27 16:37:13,177][20347] Updated weights for policy 0, policy_version 1584097 (0.0008) [2023-12-27 16:37:13,231][20347] Updated weights for policy 0, policy_version 1584107 (0.0007) [2023-12-27 16:37:13,329][20381] Updated weights for policy 1, policy_version 1582426 (0.0008) [2023-12-27 16:37:13,385][20381] Updated weights for policy 1, policy_version 1582436 (0.0007) [2023-12-27 16:37:13,451][20381] Updated weights for policy 1, policy_version 1582446 (0.0008) [2023-12-27 16:37:13,506][20381] Updated weights for policy 1, policy_version 1582456 (0.0007) [2023-12-27 16:37:13,924][20347] Updated weights for policy 0, policy_version 1584117 (0.0007) [2023-12-27 16:37:13,975][20347] Updated weights for policy 0, policy_version 1584127 (0.0007) [2023-12-27 16:37:14,024][20347] Updated weights for policy 0, policy_version 1584137 (0.0008) [2023-12-27 16:37:14,184][20381] Updated weights for policy 1, policy_version 1582466 (0.0007) [2023-12-27 16:37:14,241][20381] Updated weights for policy 1, policy_version 1582476 (0.0007) [2023-12-27 16:37:14,300][20381] Updated weights for policy 1, policy_version 1582486 (0.0008) [2023-12-27 16:37:14,661][20347] Updated weights for policy 0, policy_version 1584147 (0.0007) [2023-12-27 16:37:14,716][20347] Updated weights for policy 0, policy_version 1584157 (0.0005) [2023-12-27 16:37:14,778][20347] Updated weights for policy 0, policy_version 1584167 (0.0009) [2023-12-27 16:37:15,027][20381] Updated weights for policy 1, policy_version 1582496 (0.0007) [2023-12-27 16:37:15,093][20381] Updated weights for policy 1, policy_version 1582506 (0.0005) [2023-12-27 16:37:15,159][20381] Updated weights for policy 1, policy_version 1582516 (0.0005) [2023-12-27 16:37:15,485][20347] Updated weights for policy 0, policy_version 1584177 (0.0007) [2023-12-27 16:37:15,536][20347] Updated weights for policy 0, policy_version 1584187 (0.0007) [2023-12-27 16:37:15,588][20347] Updated weights for policy 0, policy_version 1584197 (0.0007) [2023-12-27 16:37:15,639][20347] Updated weights for policy 0, policy_version 1584207 (0.0008) [2023-12-27 16:37:15,726][20381] Updated weights for policy 1, policy_version 1582526 (0.0007) [2023-12-27 16:37:15,778][20381] Updated weights for policy 1, policy_version 1582536 (0.0008) [2023-12-27 16:37:15,830][20381] Updated weights for policy 1, policy_version 1582546 (0.0008) [2023-12-27 16:37:16,345][20347] Updated weights for policy 0, policy_version 1584217 (0.0007) [2023-12-27 16:37:16,399][20347] Updated weights for policy 0, policy_version 1584227 (0.0007) [2023-12-27 16:37:16,460][20347] Updated weights for policy 0, policy_version 1584237 (0.0007) [2023-12-27 16:37:16,514][20381] Updated weights for policy 1, policy_version 1582556 (0.0009) [2023-12-27 16:37:16,569][20381] Updated weights for policy 1, policy_version 1582566 (0.0010) [2023-12-27 16:37:16,620][20381] Updated weights for policy 1, policy_version 1582576 (0.0010) [2023-12-27 16:37:17,056][20347] Updated weights for policy 0, policy_version 1584247 (0.0007) [2023-12-27 16:37:17,110][20347] Updated weights for policy 0, policy_version 1584257 (0.0007) [2023-12-27 16:37:17,156][20347] Updated weights for policy 0, policy_version 1584267 (0.0005) [2023-12-27 16:37:17,274][20381] Updated weights for policy 1, policy_version 1582586 (0.0010) [2023-12-27 16:37:17,332][20381] Updated weights for policy 1, policy_version 1582596 (0.0010) [2023-12-27 16:37:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 810835968. Throughput: 0: 10159.1, 1: 10387.0. Samples: 810847724. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:17,363][19170] Avg episode reward: [(0, '970.590'), (1, '701.240')] [2023-12-27 16:37:17,387][20381] Updated weights for policy 1, policy_version 1582606 (0.0010) [2023-12-27 16:37:17,442][20381] Updated weights for policy 1, policy_version 1582616 (0.0010) [2023-12-27 16:37:17,815][20347] Updated weights for policy 0, policy_version 1584277 (0.0006) [2023-12-27 16:37:17,868][20347] Updated weights for policy 0, policy_version 1584287 (0.0007) [2023-12-27 16:37:17,918][20347] Updated weights for policy 0, policy_version 1584297 (0.0007) [2023-12-27 16:37:18,121][20381] Updated weights for policy 1, policy_version 1582626 (0.0005) [2023-12-27 16:37:18,170][20381] Updated weights for policy 1, policy_version 1582636 (0.0009) [2023-12-27 16:37:18,224][20381] Updated weights for policy 1, policy_version 1582646 (0.0010) [2023-12-27 16:37:18,599][20347] Updated weights for policy 0, policy_version 1584307 (0.0007) [2023-12-27 16:37:18,663][20347] Updated weights for policy 0, policy_version 1584317 (0.0007) [2023-12-27 16:37:18,725][20347] Updated weights for policy 0, policy_version 1584327 (0.0006) [2023-12-27 16:37:18,898][20381] Updated weights for policy 1, policy_version 1582656 (0.0011) [2023-12-27 16:37:18,954][20381] Updated weights for policy 1, policy_version 1582666 (0.0011) [2023-12-27 16:37:19,016][20381] Updated weights for policy 1, policy_version 1582676 (0.0011) [2023-12-27 16:37:19,404][20347] Updated weights for policy 0, policy_version 1584337 (0.0006) [2023-12-27 16:37:19,468][20347] Updated weights for policy 0, policy_version 1584347 (0.0008) [2023-12-27 16:37:19,533][20347] Updated weights for policy 0, policy_version 1584357 (0.0006) [2023-12-27 16:37:19,599][20347] Updated weights for policy 0, policy_version 1584367 (0.0008) [2023-12-27 16:37:19,709][20381] Updated weights for policy 1, policy_version 1582686 (0.0009) [2023-12-27 16:37:19,772][20381] Updated weights for policy 1, policy_version 1582696 (0.0008) [2023-12-27 16:37:19,839][20381] Updated weights for policy 1, policy_version 1582706 (0.0008) [2023-12-27 16:37:20,337][20347] Updated weights for policy 0, policy_version 1584377 (0.0010) [2023-12-27 16:37:20,395][20347] Updated weights for policy 0, policy_version 1584387 (0.0010) [2023-12-27 16:37:20,456][20347] Updated weights for policy 0, policy_version 1584397 (0.0010) [2023-12-27 16:37:20,520][20381] Updated weights for policy 1, policy_version 1582716 (0.0007) [2023-12-27 16:37:20,589][20381] Updated weights for policy 1, policy_version 1582726 (0.0006) [2023-12-27 16:37:20,653][20381] Updated weights for policy 1, policy_version 1582736 (0.0007) [2023-12-27 16:37:21,133][20347] Updated weights for policy 0, policy_version 1584407 (0.0007) [2023-12-27 16:37:21,202][20347] Updated weights for policy 0, policy_version 1584417 (0.0008) [2023-12-27 16:37:21,268][20347] Updated weights for policy 0, policy_version 1584427 (0.0008) [2023-12-27 16:37:21,360][20381] Updated weights for policy 1, policy_version 1582746 (0.0011) [2023-12-27 16:37:21,427][20381] Updated weights for policy 1, policy_version 1582756 (0.0009) [2023-12-27 16:37:21,476][20381] Updated weights for policy 1, policy_version 1582766 (0.0007) [2023-12-27 16:37:21,525][20381] Updated weights for policy 1, policy_version 1582776 (0.0007) [2023-12-27 16:37:21,957][20347] Updated weights for policy 0, policy_version 1584437 (0.0007) [2023-12-27 16:37:22,017][20347] Updated weights for policy 0, policy_version 1584447 (0.0008) [2023-12-27 16:37:22,072][20347] Updated weights for policy 0, policy_version 1584457 (0.0007) [2023-12-27 16:37:22,292][20381] Updated weights for policy 1, policy_version 1582786 (0.0009) [2023-12-27 16:37:22,358][20381] Updated weights for policy 1, policy_version 1582796 (0.0010) [2023-12-27 16:37:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 810934272. Throughput: 0: 10086.9, 1: 10379.7. Samples: 810907708. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:22,363][19170] Avg episode reward: [(0, '990.200'), (1, '738.080')] [2023-12-27 16:37:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001584464_405700608.pth... [2023-12-27 16:37:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001583280_405397504.pth [2023-12-27 16:37:22,422][20381] Updated weights for policy 1, policy_version 1582806 (0.0007) [2023-12-27 16:37:22,431][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001582808_405241856.pth... [2023-12-27 16:37:22,434][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001581592_404930560.pth [2023-12-27 16:37:22,789][20347] Updated weights for policy 0, policy_version 1584467 (0.0008) [2023-12-27 16:37:22,840][20347] Updated weights for policy 0, policy_version 1584477 (0.0007) [2023-12-27 16:37:22,895][20347] Updated weights for policy 0, policy_version 1584487 (0.0006) [2023-12-27 16:37:23,007][20381] Updated weights for policy 1, policy_version 1582816 (0.0006) [2023-12-27 16:37:23,053][20381] Updated weights for policy 1, policy_version 1582826 (0.0010) [2023-12-27 16:37:23,099][20381] Updated weights for policy 1, policy_version 1582836 (0.0010) [2023-12-27 16:37:23,541][20347] Updated weights for policy 0, policy_version 1584497 (0.0006) [2023-12-27 16:37:23,605][20347] Updated weights for policy 0, policy_version 1584507 (0.0007) [2023-12-27 16:37:23,660][20347] Updated weights for policy 0, policy_version 1584517 (0.0008) [2023-12-27 16:37:23,721][20347] Updated weights for policy 0, policy_version 1584527 (0.0007) [2023-12-27 16:37:23,781][20381] Updated weights for policy 1, policy_version 1582846 (0.0009) [2023-12-27 16:37:23,826][20381] Updated weights for policy 1, policy_version 1582856 (0.0008) [2023-12-27 16:37:23,878][20381] Updated weights for policy 1, policy_version 1582866 (0.0007) [2023-12-27 16:37:24,351][20347] Updated weights for policy 0, policy_version 1584537 (0.0007) [2023-12-27 16:37:24,408][20347] Updated weights for policy 0, policy_version 1584547 (0.0008) [2023-12-27 16:37:24,457][20347] Updated weights for policy 0, policy_version 1584557 (0.0007) [2023-12-27 16:37:24,631][20381] Updated weights for policy 1, policy_version 1582876 (0.0008) [2023-12-27 16:37:24,695][20381] Updated weights for policy 1, policy_version 1582886 (0.0010) [2023-12-27 16:37:24,753][20381] Updated weights for policy 1, policy_version 1582896 (0.0008) [2023-12-27 16:37:25,086][20347] Updated weights for policy 0, policy_version 1584567 (0.0007) [2023-12-27 16:37:25,139][20347] Updated weights for policy 0, policy_version 1584577 (0.0007) [2023-12-27 16:37:25,190][20347] Updated weights for policy 0, policy_version 1584587 (0.0005) [2023-12-27 16:37:25,390][20381] Updated weights for policy 1, policy_version 1582906 (0.0006) [2023-12-27 16:37:25,445][20381] Updated weights for policy 1, policy_version 1582916 (0.0011) [2023-12-27 16:37:25,493][20381] Updated weights for policy 1, policy_version 1582926 (0.0010) [2023-12-27 16:37:25,551][20381] Updated weights for policy 1, policy_version 1582936 (0.0011) [2023-12-27 16:37:25,784][20347] Updated weights for policy 0, policy_version 1584597 (0.0007) [2023-12-27 16:37:25,845][20347] Updated weights for policy 0, policy_version 1584607 (0.0007) [2023-12-27 16:37:25,905][20347] Updated weights for policy 0, policy_version 1584617 (0.0010) [2023-12-27 16:37:26,248][20381] Updated weights for policy 1, policy_version 1582946 (0.0008) [2023-12-27 16:37:26,305][20381] Updated weights for policy 1, policy_version 1582956 (0.0010) [2023-12-27 16:37:26,363][20381] Updated weights for policy 1, policy_version 1582966 (0.0010) [2023-12-27 16:37:26,546][20347] Updated weights for policy 0, policy_version 1584627 (0.0010) [2023-12-27 16:37:26,594][20347] Updated weights for policy 0, policy_version 1584637 (0.0010) [2023-12-27 16:37:26,634][20347] Updated weights for policy 0, policy_version 1584647 (0.0010) [2023-12-27 16:37:27,036][20381] Updated weights for policy 1, policy_version 1582976 (0.0010) [2023-12-27 16:37:27,083][20381] Updated weights for policy 1, policy_version 1582986 (0.0010) [2023-12-27 16:37:27,148][20381] Updated weights for policy 1, policy_version 1582996 (0.0005) [2023-12-27 16:37:27,327][20347] Updated weights for policy 0, policy_version 1584657 (0.0009) [2023-12-27 16:37:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 811040768. Throughput: 0: 10152.9, 1: 10357.3. Samples: 811031320. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:27,363][19170] Avg episode reward: [(0, '980.430'), (1, '805.550')] [2023-12-27 16:37:27,382][20347] Updated weights for policy 0, policy_version 1584667 (0.0008) [2023-12-27 16:37:27,442][20347] Updated weights for policy 0, policy_version 1584677 (0.0010) [2023-12-27 16:37:27,497][20347] Updated weights for policy 0, policy_version 1584687 (0.0010) [2023-12-27 16:37:27,766][20381] Updated weights for policy 1, policy_version 1583006 (0.0006) [2023-12-27 16:37:27,823][20381] Updated weights for policy 1, policy_version 1583016 (0.0005) [2023-12-27 16:37:27,875][20381] Updated weights for policy 1, policy_version 1583026 (0.0007) [2023-12-27 16:37:28,126][20347] Updated weights for policy 0, policy_version 1584697 (0.0010) [2023-12-27 16:37:28,175][20347] Updated weights for policy 0, policy_version 1584707 (0.0007) [2023-12-27 16:37:28,231][20347] Updated weights for policy 0, policy_version 1584717 (0.0007) [2023-12-27 16:37:28,534][20381] Updated weights for policy 1, policy_version 1583036 (0.0007) [2023-12-27 16:37:28,589][20381] Updated weights for policy 1, policy_version 1583046 (0.0007) [2023-12-27 16:37:28,650][20381] Updated weights for policy 1, policy_version 1583056 (0.0007) [2023-12-27 16:37:28,914][20347] Updated weights for policy 0, policy_version 1584727 (0.0007) [2023-12-27 16:37:28,968][20347] Updated weights for policy 0, policy_version 1584737 (0.0008) [2023-12-27 16:37:29,016][20347] Updated weights for policy 0, policy_version 1584747 (0.0008) [2023-12-27 16:37:29,217][20381] Updated weights for policy 1, policy_version 1583066 (0.0007) [2023-12-27 16:37:29,280][20381] Updated weights for policy 1, policy_version 1583076 (0.0008) [2023-12-27 16:37:29,341][20381] Updated weights for policy 1, policy_version 1583086 (0.0008) [2023-12-27 16:37:29,404][20381] Updated weights for policy 1, policy_version 1583096 (0.0009) [2023-12-27 16:37:29,703][20347] Updated weights for policy 0, policy_version 1584757 (0.0006) [2023-12-27 16:37:29,754][20347] Updated weights for policy 0, policy_version 1584767 (0.0006) [2023-12-27 16:37:29,805][20347] Updated weights for policy 0, policy_version 1584777 (0.0007) [2023-12-27 16:37:30,124][20381] Updated weights for policy 1, policy_version 1583106 (0.0011) [2023-12-27 16:37:30,177][20381] Updated weights for policy 1, policy_version 1583116 (0.0011) [2023-12-27 16:37:30,231][20381] Updated weights for policy 1, policy_version 1583126 (0.0010) [2023-12-27 16:37:30,504][20347] Updated weights for policy 0, policy_version 1584787 (0.0008) [2023-12-27 16:37:30,551][20347] Updated weights for policy 0, policy_version 1584797 (0.0007) [2023-12-27 16:37:30,608][20347] Updated weights for policy 0, policy_version 1584807 (0.0006) [2023-12-27 16:37:30,877][20381] Updated weights for policy 1, policy_version 1583136 (0.0006) [2023-12-27 16:37:30,927][20381] Updated weights for policy 1, policy_version 1583146 (0.0005) [2023-12-27 16:37:30,988][20381] Updated weights for policy 1, policy_version 1583156 (0.0007) [2023-12-27 16:37:31,327][20347] Updated weights for policy 0, policy_version 1584817 (0.0006) [2023-12-27 16:37:31,400][20347] Updated weights for policy 0, policy_version 1584827 (0.0008) [2023-12-27 16:37:31,455][20347] Updated weights for policy 0, policy_version 1584837 (0.0008) [2023-12-27 16:37:31,511][20347] Updated weights for policy 0, policy_version 1584847 (0.0008) [2023-12-27 16:37:31,657][20381] Updated weights for policy 1, policy_version 1583166 (0.0008) [2023-12-27 16:37:31,711][20381] Updated weights for policy 1, policy_version 1583176 (0.0008) [2023-12-27 16:37:31,775][20381] Updated weights for policy 1, policy_version 1583186 (0.0006) [2023-12-27 16:37:32,278][20347] Updated weights for policy 0, policy_version 1584857 (0.0009) [2023-12-27 16:37:32,330][20347] Updated weights for policy 0, policy_version 1584867 (0.0008) [2023-12-27 16:37:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 811139072. Throughput: 0: 10202.7, 1: 10291.2. Samples: 811155304. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:32,363][19170] Avg episode reward: [(0, '960.790'), (1, '804.680')] [2023-12-27 16:37:32,393][20347] Updated weights for policy 0, policy_version 1584877 (0.0008) [2023-12-27 16:37:32,474][20381] Updated weights for policy 1, policy_version 1583196 (0.0006) [2023-12-27 16:37:32,533][20381] Updated weights for policy 1, policy_version 1583206 (0.0005) [2023-12-27 16:37:32,593][20381] Updated weights for policy 1, policy_version 1583216 (0.0006) [2023-12-27 16:37:33,035][20347] Updated weights for policy 0, policy_version 1584887 (0.0008) [2023-12-27 16:37:33,095][20347] Updated weights for policy 0, policy_version 1584897 (0.0010) [2023-12-27 16:37:33,127][20381] Updated weights for policy 1, policy_version 1583226 (0.0006) [2023-12-27 16:37:33,149][20347] Updated weights for policy 0, policy_version 1584907 (0.0010) [2023-12-27 16:37:33,182][20381] Updated weights for policy 1, policy_version 1583236 (0.0005) [2023-12-27 16:37:33,233][20381] Updated weights for policy 1, policy_version 1583246 (0.0006) [2023-12-27 16:37:33,286][20381] Updated weights for policy 1, policy_version 1583256 (0.0010) [2023-12-27 16:37:33,728][20347] Updated weights for policy 0, policy_version 1584917 (0.0008) [2023-12-27 16:37:33,779][20347] Updated weights for policy 0, policy_version 1584927 (0.0005) [2023-12-27 16:37:33,832][20347] Updated weights for policy 0, policy_version 1584937 (0.0005) [2023-12-27 16:37:33,911][20381] Updated weights for policy 1, policy_version 1583266 (0.0009) [2023-12-27 16:37:33,964][20381] Updated weights for policy 1, policy_version 1583276 (0.0008) [2023-12-27 16:37:34,020][20381] Updated weights for policy 1, policy_version 1583286 (0.0007) [2023-12-27 16:37:34,466][20347] Updated weights for policy 0, policy_version 1584947 (0.0006) [2023-12-27 16:37:34,526][20347] Updated weights for policy 0, policy_version 1584957 (0.0008) [2023-12-27 16:37:34,587][20347] Updated weights for policy 0, policy_version 1584967 (0.0006) [2023-12-27 16:37:34,723][20381] Updated weights for policy 1, policy_version 1583296 (0.0007) [2023-12-27 16:37:34,783][20381] Updated weights for policy 1, policy_version 1583306 (0.0007) [2023-12-27 16:37:34,838][20381] Updated weights for policy 1, policy_version 1583316 (0.0007) [2023-12-27 16:37:35,273][20347] Updated weights for policy 0, policy_version 1584977 (0.0006) [2023-12-27 16:37:35,332][20347] Updated weights for policy 0, policy_version 1584987 (0.0008) [2023-12-27 16:37:35,376][20347] Updated weights for policy 0, policy_version 1584997 (0.0007) [2023-12-27 16:37:35,427][20347] Updated weights for policy 0, policy_version 1585007 (0.0008) [2023-12-27 16:37:35,463][20381] Updated weights for policy 1, policy_version 1583326 (0.0008) [2023-12-27 16:37:35,516][20381] Updated weights for policy 1, policy_version 1583336 (0.0008) [2023-12-27 16:37:35,572][20381] Updated weights for policy 1, policy_version 1583346 (0.0008) [2023-12-27 16:37:36,103][20347] Updated weights for policy 0, policy_version 1585017 (0.0006) [2023-12-27 16:37:36,156][20347] Updated weights for policy 0, policy_version 1585027 (0.0006) [2023-12-27 16:37:36,213][20347] Updated weights for policy 0, policy_version 1585037 (0.0008) [2023-12-27 16:37:36,247][20381] Updated weights for policy 1, policy_version 1583356 (0.0008) [2023-12-27 16:37:36,295][20381] Updated weights for policy 1, policy_version 1583366 (0.0008) [2023-12-27 16:37:36,354][20381] Updated weights for policy 1, policy_version 1583376 (0.0009) [2023-12-27 16:37:36,899][20347] Updated weights for policy 0, policy_version 1585048 (0.0007) [2023-12-27 16:37:36,952][20347] Updated weights for policy 0, policy_version 1585058 (0.0008) [2023-12-27 16:37:36,998][20347] Updated weights for policy 0, policy_version 1585068 (0.0009) [2023-12-27 16:37:37,080][20381] Updated weights for policy 1, policy_version 1583386 (0.0007) [2023-12-27 16:37:37,139][20381] Updated weights for policy 1, policy_version 1583396 (0.0007) [2023-12-27 16:37:37,202][20381] Updated weights for policy 1, policy_version 1583406 (0.0008) [2023-12-27 16:37:37,252][20381] Updated weights for policy 1, policy_version 1583416 (0.0007) [2023-12-27 16:37:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 811253760. Throughput: 0: 10238.1, 1: 10325.3. Samples: 811220040. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:37,363][19170] Avg episode reward: [(0, '951.040'), (1, '757.420')] [2023-12-27 16:37:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001583416_405397504.pth... [2023-12-27 16:37:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001585072_405856256.pth... [2023-12-27 16:37:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001582232_405094400.pth [2023-12-27 16:37:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001583888_405553152.pth [2023-12-27 16:37:37,737][20347] Updated weights for policy 0, policy_version 1585078 (0.0006) [2023-12-27 16:37:37,794][20347] Updated weights for policy 0, policy_version 1585088 (0.0005) [2023-12-27 16:37:37,843][20347] Updated weights for policy 0, policy_version 1585098 (0.0005) [2023-12-27 16:37:37,922][20381] Updated weights for policy 1, policy_version 1583426 (0.0009) [2023-12-27 16:37:37,969][20381] Updated weights for policy 1, policy_version 1583436 (0.0007) [2023-12-27 16:37:38,024][20381] Updated weights for policy 1, policy_version 1583446 (0.0006) [2023-12-27 16:37:38,463][20347] Updated weights for policy 0, policy_version 1585108 (0.0005) [2023-12-27 16:37:38,519][20347] Updated weights for policy 0, policy_version 1585118 (0.0006) [2023-12-27 16:37:38,575][20347] Updated weights for policy 0, policy_version 1585128 (0.0005) [2023-12-27 16:37:38,753][20381] Updated weights for policy 1, policy_version 1583456 (0.0007) [2023-12-27 16:37:38,822][20381] Updated weights for policy 1, policy_version 1583466 (0.0008) [2023-12-27 16:37:38,879][20381] Updated weights for policy 1, policy_version 1583476 (0.0008) [2023-12-27 16:37:39,179][20347] Updated weights for policy 0, policy_version 1585138 (0.0009) [2023-12-27 16:37:39,243][20347] Updated weights for policy 0, policy_version 1585148 (0.0009) [2023-12-27 16:37:39,305][20347] Updated weights for policy 0, policy_version 1585158 (0.0009) [2023-12-27 16:37:39,385][20347] Updated weights for policy 0, policy_version 1585168 (0.0010) [2023-12-27 16:37:39,616][20381] Updated weights for policy 1, policy_version 1583486 (0.0008) [2023-12-27 16:37:39,669][20381] Updated weights for policy 1, policy_version 1583496 (0.0009) [2023-12-27 16:37:39,733][20381] Updated weights for policy 1, policy_version 1583506 (0.0007) [2023-12-27 16:37:40,091][20347] Updated weights for policy 0, policy_version 1585178 (0.0008) [2023-12-27 16:37:40,150][20347] Updated weights for policy 0, policy_version 1585188 (0.0010) [2023-12-27 16:37:40,209][20347] Updated weights for policy 0, policy_version 1585198 (0.0011) [2023-12-27 16:37:40,434][20381] Updated weights for policy 1, policy_version 1583516 (0.0006) [2023-12-27 16:37:40,496][20381] Updated weights for policy 1, policy_version 1583526 (0.0007) [2023-12-27 16:37:40,557][20381] Updated weights for policy 1, policy_version 1583536 (0.0007) [2023-12-27 16:37:40,912][20347] Updated weights for policy 0, policy_version 1585208 (0.0009) [2023-12-27 16:37:40,962][20347] Updated weights for policy 0, policy_version 1585218 (0.0010) [2023-12-27 16:37:41,016][20347] Updated weights for policy 0, policy_version 1585228 (0.0011) [2023-12-27 16:37:41,242][20381] Updated weights for policy 1, policy_version 1583546 (0.0006) [2023-12-27 16:37:41,311][20381] Updated weights for policy 1, policy_version 1583556 (0.0009) [2023-12-27 16:37:41,379][20381] Updated weights for policy 1, policy_version 1583566 (0.0008) [2023-12-27 16:37:41,440][20381] Updated weights for policy 1, policy_version 1583576 (0.0009) [2023-12-27 16:37:41,788][20347] Updated weights for policy 0, policy_version 1585238 (0.0008) [2023-12-27 16:37:41,850][20347] Updated weights for policy 0, policy_version 1585248 (0.0006) [2023-12-27 16:37:41,910][20347] Updated weights for policy 0, policy_version 1585258 (0.0008) [2023-12-27 16:37:42,179][20381] Updated weights for policy 1, policy_version 1583586 (0.0007) [2023-12-27 16:37:42,234][20381] Updated weights for policy 1, policy_version 1583596 (0.0007) [2023-12-27 16:37:42,289][20381] Updated weights for policy 1, policy_version 1583606 (0.0007) [2023-12-27 16:37:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 811352064. Throughput: 0: 10242.4, 1: 10261.8. Samples: 811341412. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:42,363][19170] Avg episode reward: [(0, '951.090'), (1, '787.030')] [2023-12-27 16:37:42,603][20347] Updated weights for policy 0, policy_version 1585268 (0.0010) [2023-12-27 16:37:42,662][20347] Updated weights for policy 0, policy_version 1585278 (0.0011) [2023-12-27 16:37:42,721][20347] Updated weights for policy 0, policy_version 1585288 (0.0010) [2023-12-27 16:37:43,037][20381] Updated weights for policy 1, policy_version 1583616 (0.0007) [2023-12-27 16:37:43,093][20381] Updated weights for policy 1, policy_version 1583626 (0.0008) [2023-12-27 16:37:43,141][20381] Updated weights for policy 1, policy_version 1583636 (0.0008) [2023-12-27 16:37:43,406][20347] Updated weights for policy 0, policy_version 1585298 (0.0010) [2023-12-27 16:37:43,463][20347] Updated weights for policy 0, policy_version 1585308 (0.0006) [2023-12-27 16:37:43,517][20347] Updated weights for policy 0, policy_version 1585318 (0.0005) [2023-12-27 16:37:43,564][20347] Updated weights for policy 0, policy_version 1585328 (0.0007) [2023-12-27 16:37:43,821][20381] Updated weights for policy 1, policy_version 1583646 (0.0010) [2023-12-27 16:37:43,866][20381] Updated weights for policy 1, policy_version 1583656 (0.0010) [2023-12-27 16:37:43,915][20381] Updated weights for policy 1, policy_version 1583666 (0.0010) [2023-12-27 16:37:44,215][20347] Updated weights for policy 0, policy_version 1585338 (0.0006) [2023-12-27 16:37:44,262][20347] Updated weights for policy 0, policy_version 1585348 (0.0005) [2023-12-27 16:37:44,312][20347] Updated weights for policy 0, policy_version 1585358 (0.0006) [2023-12-27 16:37:44,608][20381] Updated weights for policy 1, policy_version 1583676 (0.0010) [2023-12-27 16:37:44,613][20292] KL-divergence is very high: 109.2871 [2023-12-27 16:37:44,642][20292] KL-divergence is very high: 112.2211 [2023-12-27 16:37:44,662][20381] Updated weights for policy 1, policy_version 1583686 (0.0011) [2023-12-27 16:37:44,711][20381] Updated weights for policy 1, policy_version 1583696 (0.0010) [2023-12-27 16:37:44,925][20347] Updated weights for policy 0, policy_version 1585368 (0.0010) [2023-12-27 16:37:44,988][20347] Updated weights for policy 0, policy_version 1585378 (0.0010) [2023-12-27 16:37:45,052][20347] Updated weights for policy 0, policy_version 1585388 (0.0010) [2023-12-27 16:37:45,424][20381] Updated weights for policy 1, policy_version 1583706 (0.0006) [2023-12-27 16:37:45,476][20381] Updated weights for policy 1, policy_version 1583716 (0.0008) [2023-12-27 16:37:45,521][20381] Updated weights for policy 1, policy_version 1583726 (0.0008) [2023-12-27 16:37:45,575][20381] Updated weights for policy 1, policy_version 1583736 (0.0006) [2023-12-27 16:37:45,733][20347] Updated weights for policy 0, policy_version 1585398 (0.0007) [2023-12-27 16:37:45,778][20347] Updated weights for policy 0, policy_version 1585408 (0.0010) [2023-12-27 16:37:45,823][20347] Updated weights for policy 0, policy_version 1585418 (0.0010) [2023-12-27 16:37:46,309][20381] Updated weights for policy 1, policy_version 1583746 (0.0006) [2023-12-27 16:37:46,356][20381] Updated weights for policy 1, policy_version 1583756 (0.0006) [2023-12-27 16:37:46,408][20381] Updated weights for policy 1, policy_version 1583766 (0.0007) [2023-12-27 16:37:46,490][20347] Updated weights for policy 0, policy_version 1585428 (0.0008) [2023-12-27 16:37:46,537][20347] Updated weights for policy 0, policy_version 1585438 (0.0005) [2023-12-27 16:37:46,581][20347] Updated weights for policy 0, policy_version 1585448 (0.0005) [2023-12-27 16:37:47,016][20381] Updated weights for policy 1, policy_version 1583776 (0.0010) [2023-12-27 16:37:47,069][20381] Updated weights for policy 1, policy_version 1583786 (0.0010) [2023-12-27 16:37:47,115][20381] Updated weights for policy 1, policy_version 1583796 (0.0010) [2023-12-27 16:37:47,201][20347] Updated weights for policy 0, policy_version 1585458 (0.0007) [2023-12-27 16:37:47,257][20347] Updated weights for policy 0, policy_version 1585468 (0.0010) [2023-12-27 16:37:47,311][20347] Updated weights for policy 0, policy_version 1585478 (0.0010) [2023-12-27 16:37:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 811450368. Throughput: 0: 10258.5, 1: 10236.2. Samples: 811463368. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:47,364][19170] Avg episode reward: [(0, '980.730'), (1, '747.710')] [2023-12-27 16:37:47,367][20347] Updated weights for policy 0, policy_version 1585488 (0.0005) [2023-12-27 16:37:47,763][20381] Updated weights for policy 1, policy_version 1583806 (0.0007) [2023-12-27 16:37:47,819][20381] Updated weights for policy 1, policy_version 1583816 (0.0005) [2023-12-27 16:37:47,870][20381] Updated weights for policy 1, policy_version 1583826 (0.0005) [2023-12-27 16:37:47,993][20347] Updated weights for policy 0, policy_version 1585498 (0.0006) [2023-12-27 16:37:48,048][20347] Updated weights for policy 0, policy_version 1585508 (0.0010) [2023-12-27 16:37:48,100][20347] Updated weights for policy 0, policy_version 1585518 (0.0010) [2023-12-27 16:37:48,426][20381] Updated weights for policy 1, policy_version 1583836 (0.0006) [2023-12-27 16:37:48,486][20381] Updated weights for policy 1, policy_version 1583846 (0.0008) [2023-12-27 16:37:48,540][20381] Updated weights for policy 1, policy_version 1583856 (0.0007) [2023-12-27 16:37:48,787][20347] Updated weights for policy 0, policy_version 1585528 (0.0006) [2023-12-27 16:37:48,844][20347] Updated weights for policy 0, policy_version 1585538 (0.0007) [2023-12-27 16:37:48,907][20347] Updated weights for policy 0, policy_version 1585548 (0.0006) [2023-12-27 16:37:49,271][20381] Updated weights for policy 1, policy_version 1583866 (0.0008) [2023-12-27 16:37:49,340][20381] Updated weights for policy 1, policy_version 1583876 (0.0010) [2023-12-27 16:37:49,406][20381] Updated weights for policy 1, policy_version 1583886 (0.0010) [2023-12-27 16:37:49,472][20381] Updated weights for policy 1, policy_version 1583896 (0.0011) [2023-12-27 16:37:49,630][20347] Updated weights for policy 0, policy_version 1585558 (0.0007) [2023-12-27 16:37:49,689][20347] Updated weights for policy 0, policy_version 1585568 (0.0009) [2023-12-27 16:37:49,741][20347] Updated weights for policy 0, policy_version 1585578 (0.0008) [2023-12-27 16:37:50,149][20381] Updated weights for policy 1, policy_version 1583906 (0.0008) [2023-12-27 16:37:50,211][20381] Updated weights for policy 1, policy_version 1583916 (0.0007) [2023-12-27 16:37:50,270][20381] Updated weights for policy 1, policy_version 1583926 (0.0006) [2023-12-27 16:37:50,475][20347] Updated weights for policy 0, policy_version 1585588 (0.0007) [2023-12-27 16:37:50,535][20347] Updated weights for policy 0, policy_version 1585598 (0.0007) [2023-12-27 16:37:50,596][20347] Updated weights for policy 0, policy_version 1585608 (0.0007) [2023-12-27 16:37:50,864][20381] Updated weights for policy 1, policy_version 1583936 (0.0006) [2023-12-27 16:37:50,924][20381] Updated weights for policy 1, policy_version 1583946 (0.0006) [2023-12-27 16:37:50,983][20381] Updated weights for policy 1, policy_version 1583956 (0.0007) [2023-12-27 16:37:51,289][20347] Updated weights for policy 0, policy_version 1585618 (0.0006) [2023-12-27 16:37:51,354][20347] Updated weights for policy 0, policy_version 1585628 (0.0007) [2023-12-27 16:37:51,417][20347] Updated weights for policy 0, policy_version 1585638 (0.0008) [2023-12-27 16:37:51,461][20347] Updated weights for policy 0, policy_version 1585648 (0.0008) [2023-12-27 16:37:51,666][20381] Updated weights for policy 1, policy_version 1583966 (0.0008) [2023-12-27 16:37:51,727][20381] Updated weights for policy 1, policy_version 1583976 (0.0007) [2023-12-27 16:37:51,792][20381] Updated weights for policy 1, policy_version 1583986 (0.0006) [2023-12-27 16:37:52,094][20347] Updated weights for policy 0, policy_version 1585658 (0.0010) [2023-12-27 16:37:52,144][20347] Updated weights for policy 0, policy_version 1585668 (0.0005) [2023-12-27 16:37:52,205][20347] Updated weights for policy 0, policy_version 1585678 (0.0005) [2023-12-27 16:37:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 811556864. Throughput: 0: 10262.4, 1: 10232.5. Samples: 811525708. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:52,363][19170] Avg episode reward: [(0, '980.720'), (1, '758.780')] [2023-12-27 16:37:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001583992_405544960.pth... [2023-12-27 16:37:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001585680_406011904.pth... [2023-12-27 16:37:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001582808_405241856.pth [2023-12-27 16:37:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001584464_405700608.pth [2023-12-27 16:37:52,437][20381] Updated weights for policy 1, policy_version 1583996 (0.0006) [2023-12-27 16:37:52,497][20381] Updated weights for policy 1, policy_version 1584006 (0.0006) [2023-12-27 16:37:52,559][20381] Updated weights for policy 1, policy_version 1584016 (0.0006) [2023-12-27 16:37:52,915][20347] Updated weights for policy 0, policy_version 1585688 (0.0010) [2023-12-27 16:37:52,967][20347] Updated weights for policy 0, policy_version 1585698 (0.0010) [2023-12-27 16:37:53,025][20347] Updated weights for policy 0, policy_version 1585708 (0.0010) [2023-12-27 16:37:53,154][20381] Updated weights for policy 1, policy_version 1584026 (0.0007) [2023-12-27 16:37:53,208][20381] Updated weights for policy 1, policy_version 1584036 (0.0006) [2023-12-27 16:37:53,266][20381] Updated weights for policy 1, policy_version 1584046 (0.0007) [2023-12-27 16:37:53,324][20381] Updated weights for policy 1, policy_version 1584056 (0.0008) [2023-12-27 16:37:53,660][20347] Updated weights for policy 0, policy_version 1585718 (0.0011) [2023-12-27 16:37:53,712][20347] Updated weights for policy 0, policy_version 1585728 (0.0010) [2023-12-27 16:37:53,760][20347] Updated weights for policy 0, policy_version 1585738 (0.0010) [2023-12-27 16:37:53,945][20381] Updated weights for policy 1, policy_version 1584066 (0.0005) [2023-12-27 16:37:53,998][20381] Updated weights for policy 1, policy_version 1584076 (0.0005) [2023-12-27 16:37:54,049][20381] Updated weights for policy 1, policy_version 1584086 (0.0008) [2023-12-27 16:37:54,454][20347] Updated weights for policy 0, policy_version 1585748 (0.0009) [2023-12-27 16:37:54,508][20347] Updated weights for policy 0, policy_version 1585758 (0.0007) [2023-12-27 16:37:54,566][20347] Updated weights for policy 0, policy_version 1585768 (0.0007) [2023-12-27 16:37:54,686][20381] Updated weights for policy 1, policy_version 1584096 (0.0007) [2023-12-27 16:37:54,734][20381] Updated weights for policy 1, policy_version 1584106 (0.0008) [2023-12-27 16:37:54,780][20381] Updated weights for policy 1, policy_version 1584116 (0.0007) [2023-12-27 16:37:55,184][20347] Updated weights for policy 0, policy_version 1585778 (0.0007) [2023-12-27 16:37:55,240][20347] Updated weights for policy 0, policy_version 1585788 (0.0005) [2023-12-27 16:37:55,291][20347] Updated weights for policy 0, policy_version 1585798 (0.0005) [2023-12-27 16:37:55,341][20347] Updated weights for policy 0, policy_version 1585808 (0.0008) [2023-12-27 16:37:55,494][20381] Updated weights for policy 1, policy_version 1584126 (0.0006) [2023-12-27 16:37:55,549][20381] Updated weights for policy 1, policy_version 1584136 (0.0005) [2023-12-27 16:37:55,607][20381] Updated weights for policy 1, policy_version 1584146 (0.0008) [2023-12-27 16:37:55,964][20347] Updated weights for policy 0, policy_version 1585818 (0.0009) [2023-12-27 16:37:56,022][20347] Updated weights for policy 0, policy_version 1585828 (0.0010) [2023-12-27 16:37:56,074][20347] Updated weights for policy 0, policy_version 1585838 (0.0010) [2023-12-27 16:37:56,183][20381] Updated weights for policy 1, policy_version 1584156 (0.0009) [2023-12-27 16:37:56,238][20381] Updated weights for policy 1, policy_version 1584166 (0.0007) [2023-12-27 16:37:56,289][20381] Updated weights for policy 1, policy_version 1584176 (0.0007) [2023-12-27 16:37:56,740][20347] Updated weights for policy 0, policy_version 1585848 (0.0010) [2023-12-27 16:37:56,791][20347] Updated weights for policy 0, policy_version 1585858 (0.0010) [2023-12-27 16:37:56,838][20347] Updated weights for policy 0, policy_version 1585868 (0.0010) [2023-12-27 16:37:56,974][20381] Updated weights for policy 1, policy_version 1584186 (0.0007) [2023-12-27 16:37:57,033][20381] Updated weights for policy 1, policy_version 1584196 (0.0008) [2023-12-27 16:37:57,088][20381] Updated weights for policy 1, policy_version 1584206 (0.0008) [2023-12-27 16:37:57,146][20381] Updated weights for policy 1, policy_version 1584216 (0.0007) [2023-12-27 16:37:57,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20479.9, 300 sec: 20632.7). Total num frames: 811663360. Throughput: 0: 10308.0, 1: 10293.4. Samples: 811653456. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:37:57,364][19170] Avg episode reward: [(0, '970.660'), (1, '705.870')] [2023-12-27 16:37:57,519][20347] Updated weights for policy 0, policy_version 1585878 (0.0010) [2023-12-27 16:37:57,567][20347] Updated weights for policy 0, policy_version 1585888 (0.0010) [2023-12-27 16:37:57,627][20347] Updated weights for policy 0, policy_version 1585898 (0.0011) [2023-12-27 16:37:57,781][20381] Updated weights for policy 1, policy_version 1584226 (0.0007) [2023-12-27 16:37:57,833][20381] Updated weights for policy 1, policy_version 1584236 (0.0007) [2023-12-27 16:37:57,886][20381] Updated weights for policy 1, policy_version 1584246 (0.0009) [2023-12-27 16:37:58,297][20347] Updated weights for policy 0, policy_version 1585908 (0.0010) [2023-12-27 16:37:58,364][20347] Updated weights for policy 0, policy_version 1585918 (0.0010) [2023-12-27 16:37:58,431][20347] Updated weights for policy 0, policy_version 1585928 (0.0008) [2023-12-27 16:37:58,634][20381] Updated weights for policy 1, policy_version 1584256 (0.0008) [2023-12-27 16:37:58,693][20381] Updated weights for policy 1, policy_version 1584266 (0.0008) [2023-12-27 16:37:58,749][20381] Updated weights for policy 1, policy_version 1584276 (0.0007) [2023-12-27 16:37:59,157][20347] Updated weights for policy 0, policy_version 1585938 (0.0009) [2023-12-27 16:37:59,209][20347] Updated weights for policy 0, policy_version 1585948 (0.0007) [2023-12-27 16:37:59,279][20347] Updated weights for policy 0, policy_version 1585958 (0.0008) [2023-12-27 16:37:59,338][20347] Updated weights for policy 0, policy_version 1585968 (0.0008) [2023-12-27 16:37:59,501][20381] Updated weights for policy 1, policy_version 1584286 (0.0008) [2023-12-27 16:37:59,561][20381] Updated weights for policy 1, policy_version 1584296 (0.0008) [2023-12-27 16:37:59,626][20381] Updated weights for policy 1, policy_version 1584306 (0.0011) [2023-12-27 16:38:00,054][20347] Updated weights for policy 0, policy_version 1585978 (0.0009) [2023-12-27 16:38:00,107][20347] Updated weights for policy 0, policy_version 1585988 (0.0010) [2023-12-27 16:38:00,156][20347] Updated weights for policy 0, policy_version 1585998 (0.0010) [2023-12-27 16:38:00,261][20381] Updated weights for policy 1, policy_version 1584316 (0.0008) [2023-12-27 16:38:00,318][20381] Updated weights for policy 1, policy_version 1584326 (0.0008) [2023-12-27 16:38:00,368][20381] Updated weights for policy 1, policy_version 1584336 (0.0006) [2023-12-27 16:38:00,833][20347] Updated weights for policy 0, policy_version 1586008 (0.0010) [2023-12-27 16:38:00,909][20347] Updated weights for policy 0, policy_version 1586018 (0.0010) [2023-12-27 16:38:00,916][20381] Updated weights for policy 1, policy_version 1584346 (0.0006) [2023-12-27 16:38:00,961][20381] Updated weights for policy 1, policy_version 1584356 (0.0009) [2023-12-27 16:38:00,962][20347] Updated weights for policy 0, policy_version 1586028 (0.0010) [2023-12-27 16:38:01,005][20381] Updated weights for policy 1, policy_version 1584366 (0.0005) [2023-12-27 16:38:01,064][20381] Updated weights for policy 1, policy_version 1584376 (0.0007) [2023-12-27 16:38:01,732][20347] Updated weights for policy 0, policy_version 1586038 (0.0009) [2023-12-27 16:38:01,783][20347] Updated weights for policy 0, policy_version 1586048 (0.0006) [2023-12-27 16:38:01,795][20381] Updated weights for policy 1, policy_version 1584386 (0.0011) [2023-12-27 16:38:01,839][20347] Updated weights for policy 0, policy_version 1586058 (0.0006) [2023-12-27 16:38:01,861][20381] Updated weights for policy 1, policy_version 1584396 (0.0011) [2023-12-27 16:38:01,923][20381] Updated weights for policy 1, policy_version 1584406 (0.0010) [2023-12-27 16:38:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 811761664. Throughput: 0: 10281.3, 1: 10312.1. Samples: 811774428. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:38:02,363][19170] Avg episode reward: [(0, '980.420'), (1, '647.900')] [2023-12-27 16:38:02,473][20347] Updated weights for policy 0, policy_version 1586068 (0.0007) [2023-12-27 16:38:02,521][20347] Updated weights for policy 0, policy_version 1586078 (0.0007) [2023-12-27 16:38:02,570][20347] Updated weights for policy 0, policy_version 1586088 (0.0008) [2023-12-27 16:38:02,616][20381] Updated weights for policy 1, policy_version 1584416 (0.0007) [2023-12-27 16:38:02,668][20381] Updated weights for policy 1, policy_version 1584426 (0.0008) [2023-12-27 16:38:02,733][20381] Updated weights for policy 1, policy_version 1584436 (0.0006) [2023-12-27 16:38:03,312][20347] Updated weights for policy 0, policy_version 1586098 (0.0007) [2023-12-27 16:38:03,365][20347] Updated weights for policy 0, policy_version 1586108 (0.0007) [2023-12-27 16:38:03,383][20381] Updated weights for policy 1, policy_version 1584446 (0.0006) [2023-12-27 16:38:03,423][20347] Updated weights for policy 0, policy_version 1586118 (0.0007) [2023-12-27 16:38:03,446][20381] Updated weights for policy 1, policy_version 1584456 (0.0007) [2023-12-27 16:38:03,478][20347] Updated weights for policy 0, policy_version 1586128 (0.0009) [2023-12-27 16:38:03,497][20381] Updated weights for policy 1, policy_version 1584466 (0.0010) [2023-12-27 16:38:04,099][20381] Updated weights for policy 1, policy_version 1584476 (0.0009) [2023-12-27 16:38:04,155][20381] Updated weights for policy 1, policy_version 1584486 (0.0005) [2023-12-27 16:38:04,210][20381] Updated weights for policy 1, policy_version 1584496 (0.0009) [2023-12-27 16:38:04,227][20347] Updated weights for policy 0, policy_version 1586138 (0.0006) [2023-12-27 16:38:04,286][20347] Updated weights for policy 0, policy_version 1586148 (0.0007) [2023-12-27 16:38:04,347][20347] Updated weights for policy 0, policy_version 1586158 (0.0008) [2023-12-27 16:38:04,804][20381] Updated weights for policy 1, policy_version 1584506 (0.0010) [2023-12-27 16:38:04,863][20381] Updated weights for policy 1, policy_version 1584516 (0.0007) [2023-12-27 16:38:04,920][20381] Updated weights for policy 1, policy_version 1584526 (0.0006) [2023-12-27 16:38:04,980][20381] Updated weights for policy 1, policy_version 1584536 (0.0005) [2023-12-27 16:38:05,059][20347] Updated weights for policy 0, policy_version 1586168 (0.0009) [2023-12-27 16:38:05,115][20347] Updated weights for policy 0, policy_version 1586178 (0.0010) [2023-12-27 16:38:05,175][20347] Updated weights for policy 0, policy_version 1586188 (0.0010) [2023-12-27 16:38:05,504][20381] Updated weights for policy 1, policy_version 1584546 (0.0007) [2023-12-27 16:38:05,553][20381] Updated weights for policy 1, policy_version 1584556 (0.0008) [2023-12-27 16:38:05,601][20381] Updated weights for policy 1, policy_version 1584566 (0.0007) [2023-12-27 16:38:05,853][20347] Updated weights for policy 0, policy_version 1586198 (0.0010) [2023-12-27 16:38:05,905][20347] Updated weights for policy 0, policy_version 1586208 (0.0010) [2023-12-27 16:38:05,959][20347] Updated weights for policy 0, policy_version 1586218 (0.0010) [2023-12-27 16:38:06,259][20381] Updated weights for policy 1, policy_version 1584576 (0.0007) [2023-12-27 16:38:06,313][20381] Updated weights for policy 1, policy_version 1584586 (0.0007) [2023-12-27 16:38:06,365][20381] Updated weights for policy 1, policy_version 1584596 (0.0007) [2023-12-27 16:38:06,625][20347] Updated weights for policy 0, policy_version 1586228 (0.0008) [2023-12-27 16:38:06,684][20347] Updated weights for policy 0, policy_version 1586238 (0.0006) [2023-12-27 16:38:06,746][20347] Updated weights for policy 0, policy_version 1586248 (0.0006) [2023-12-27 16:38:07,021][20381] Updated weights for policy 1, policy_version 1584606 (0.0007) [2023-12-27 16:38:07,072][20381] Updated weights for policy 1, policy_version 1584616 (0.0005) [2023-12-27 16:38:07,130][20381] Updated weights for policy 1, policy_version 1584626 (0.0005) [2023-12-27 16:38:07,287][20347] Updated weights for policy 0, policy_version 1586258 (0.0006) [2023-12-27 16:38:07,335][20347] Updated weights for policy 0, policy_version 1586268 (0.0010) [2023-12-27 16:38:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 811868160. Throughput: 0: 10265.0, 1: 10387.9. Samples: 811837096. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:38:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '583.330')] [2023-12-27 16:38:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001584632_405708800.pth... [2023-12-27 16:38:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001583416_405397504.pth [2023-12-27 16:38:07,380][20347] Updated weights for policy 0, policy_version 1586278 (0.0009) [2023-12-27 16:38:07,430][20347] Updated weights for policy 0, policy_version 1586288 (0.0007) [2023-12-27 16:38:07,430][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001586288_406167552.pth... [2023-12-27 16:38:07,434][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001585072_405856256.pth [2023-12-27 16:38:07,772][20381] Updated weights for policy 1, policy_version 1584636 (0.0006) [2023-12-27 16:38:07,823][20381] Updated weights for policy 1, policy_version 1584646 (0.0007) [2023-12-27 16:38:07,881][20381] Updated weights for policy 1, policy_version 1584656 (0.0008) [2023-12-27 16:38:08,081][20347] Updated weights for policy 0, policy_version 1586298 (0.0010) [2023-12-27 16:38:08,136][20347] Updated weights for policy 0, policy_version 1586308 (0.0010) [2023-12-27 16:38:08,185][20347] Updated weights for policy 0, policy_version 1586318 (0.0010) [2023-12-27 16:38:08,499][20381] Updated weights for policy 1, policy_version 1584666 (0.0007) [2023-12-27 16:38:08,553][20381] Updated weights for policy 1, policy_version 1584676 (0.0007) [2023-12-27 16:38:08,605][20381] Updated weights for policy 1, policy_version 1584686 (0.0007) [2023-12-27 16:38:08,665][20381] Updated weights for policy 1, policy_version 1584696 (0.0005) [2023-12-27 16:38:08,926][20347] Updated weights for policy 0, policy_version 1586328 (0.0008) [2023-12-27 16:38:08,988][20347] Updated weights for policy 0, policy_version 1586338 (0.0008) [2023-12-27 16:38:09,049][20347] Updated weights for policy 0, policy_version 1586348 (0.0007) [2023-12-27 16:38:09,274][20381] Updated weights for policy 1, policy_version 1584706 (0.0006) [2023-12-27 16:38:09,335][20381] Updated weights for policy 1, policy_version 1584716 (0.0007) [2023-12-27 16:38:09,405][20381] Updated weights for policy 1, policy_version 1584726 (0.0008) [2023-12-27 16:38:09,705][20347] Updated weights for policy 0, policy_version 1586358 (0.0007) [2023-12-27 16:38:09,766][20347] Updated weights for policy 0, policy_version 1586368 (0.0007) [2023-12-27 16:38:09,832][20347] Updated weights for policy 0, policy_version 1586378 (0.0007) [2023-12-27 16:38:10,147][20381] Updated weights for policy 1, policy_version 1584736 (0.0008) [2023-12-27 16:38:10,211][20381] Updated weights for policy 1, policy_version 1584746 (0.0006) [2023-12-27 16:38:10,274][20381] Updated weights for policy 1, policy_version 1584756 (0.0005) [2023-12-27 16:38:10,518][20347] Updated weights for policy 0, policy_version 1586388 (0.0008) [2023-12-27 16:38:10,567][20347] Updated weights for policy 0, policy_version 1586398 (0.0007) [2023-12-27 16:38:10,617][20347] Updated weights for policy 0, policy_version 1586408 (0.0006) [2023-12-27 16:38:10,960][20381] Updated weights for policy 1, policy_version 1584766 (0.0006) [2023-12-27 16:38:11,014][20381] Updated weights for policy 1, policy_version 1584776 (0.0007) [2023-12-27 16:38:11,080][20381] Updated weights for policy 1, policy_version 1584786 (0.0009) [2023-12-27 16:38:11,318][20347] Updated weights for policy 0, policy_version 1586418 (0.0008) [2023-12-27 16:38:11,387][20347] Updated weights for policy 0, policy_version 1586428 (0.0008) [2023-12-27 16:38:11,451][20347] Updated weights for policy 0, policy_version 1586438 (0.0008) [2023-12-27 16:38:11,512][20347] Updated weights for policy 0, policy_version 1586448 (0.0008) [2023-12-27 16:38:11,808][20381] Updated weights for policy 1, policy_version 1584796 (0.0006) [2023-12-27 16:38:11,869][20381] Updated weights for policy 1, policy_version 1584806 (0.0008) [2023-12-27 16:38:11,930][20381] Updated weights for policy 1, policy_version 1584816 (0.0008) [2023-12-27 16:38:12,240][20347] Updated weights for policy 0, policy_version 1586458 (0.0008) [2023-12-27 16:38:12,303][20347] Updated weights for policy 0, policy_version 1586468 (0.0007) [2023-12-27 16:38:12,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 811966464. Throughput: 0: 10267.3, 1: 10430.2. Samples: 811962712. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:38:12,363][19170] Avg episode reward: [(0, '980.410'), (1, '569.530')] [2023-12-27 16:38:12,375][20347] Updated weights for policy 0, policy_version 1586478 (0.0008) [2023-12-27 16:38:12,650][20381] Updated weights for policy 1, policy_version 1584826 (0.0008) [2023-12-27 16:38:12,710][20381] Updated weights for policy 1, policy_version 1584836 (0.0007) [2023-12-27 16:38:12,769][20381] Updated weights for policy 1, policy_version 1584846 (0.0008) [2023-12-27 16:38:12,829][20381] Updated weights for policy 1, policy_version 1584856 (0.0008) [2023-12-27 16:38:12,974][20347] Updated weights for policy 0, policy_version 1586488 (0.0007) [2023-12-27 16:38:13,037][20347] Updated weights for policy 0, policy_version 1586498 (0.0008) [2023-12-27 16:38:13,096][20347] Updated weights for policy 0, policy_version 1586508 (0.0008) [2023-12-27 16:38:13,476][20381] Updated weights for policy 1, policy_version 1584866 (0.0008) [2023-12-27 16:38:13,537][20381] Updated weights for policy 1, policy_version 1584876 (0.0008) [2023-12-27 16:38:13,593][20381] Updated weights for policy 1, policy_version 1584886 (0.0007) [2023-12-27 16:38:13,767][20347] Updated weights for policy 0, policy_version 1586518 (0.0008) [2023-12-27 16:38:13,823][20347] Updated weights for policy 0, policy_version 1586528 (0.0008) [2023-12-27 16:38:13,882][20347] Updated weights for policy 0, policy_version 1586538 (0.0008) [2023-12-27 16:38:14,244][20381] Updated weights for policy 1, policy_version 1584896 (0.0006) [2023-12-27 16:38:14,305][20381] Updated weights for policy 1, policy_version 1584906 (0.0006) [2023-12-27 16:38:14,355][20381] Updated weights for policy 1, policy_version 1584916 (0.0006) [2023-12-27 16:38:14,615][20347] Updated weights for policy 0, policy_version 1586548 (0.0008) [2023-12-27 16:38:14,674][20347] Updated weights for policy 0, policy_version 1586558 (0.0007) [2023-12-27 16:38:14,728][20347] Updated weights for policy 0, policy_version 1586568 (0.0008) [2023-12-27 16:38:14,983][20381] Updated weights for policy 1, policy_version 1584926 (0.0006) [2023-12-27 16:38:15,051][20381] Updated weights for policy 1, policy_version 1584936 (0.0007) [2023-12-27 16:38:15,116][20381] Updated weights for policy 1, policy_version 1584946 (0.0007) [2023-12-27 16:38:15,430][20347] Updated weights for policy 0, policy_version 1586578 (0.0008) [2023-12-27 16:38:15,489][20347] Updated weights for policy 0, policy_version 1586588 (0.0008) [2023-12-27 16:38:15,549][20347] Updated weights for policy 0, policy_version 1586598 (0.0006) [2023-12-27 16:38:15,608][20347] Updated weights for policy 0, policy_version 1586608 (0.0008) [2023-12-27 16:38:15,774][20381] Updated weights for policy 1, policy_version 1584956 (0.0007) [2023-12-27 16:38:15,829][20381] Updated weights for policy 1, policy_version 1584966 (0.0007) [2023-12-27 16:38:15,880][20381] Updated weights for policy 1, policy_version 1584976 (0.0007) [2023-12-27 16:38:16,262][20347] Updated weights for policy 0, policy_version 1586618 (0.0007) [2023-12-27 16:38:16,310][20347] Updated weights for policy 0, policy_version 1586628 (0.0007) [2023-12-27 16:38:16,369][20347] Updated weights for policy 0, policy_version 1586638 (0.0008) [2023-12-27 16:38:16,555][20381] Updated weights for policy 1, policy_version 1584986 (0.0008) [2023-12-27 16:38:16,604][20381] Updated weights for policy 1, policy_version 1584996 (0.0009) [2023-12-27 16:38:16,656][20381] Updated weights for policy 1, policy_version 1585006 (0.0006) [2023-12-27 16:38:16,712][20381] Updated weights for policy 1, policy_version 1585016 (0.0005) [2023-12-27 16:38:17,073][20347] Updated weights for policy 0, policy_version 1586648 (0.0008) [2023-12-27 16:38:17,126][20347] Updated weights for policy 0, policy_version 1586659 (0.0009) [2023-12-27 16:38:17,185][20347] Updated weights for policy 0, policy_version 1586669 (0.0008) [2023-12-27 16:38:17,255][20381] Updated weights for policy 1, policy_version 1585026 (0.0008) [2023-12-27 16:38:17,318][20381] Updated weights for policy 1, policy_version 1585036 (0.0007) [2023-12-27 16:38:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 812072960. Throughput: 0: 10225.1, 1: 10437.5. Samples: 812085124. Policy #0 lag: (min: 27.0, avg: 39.2, max: 59.0) [2023-12-27 16:38:17,363][19170] Avg episode reward: [(0, '951.030'), (1, '609.640')] [2023-12-27 16:38:17,374][20381] Updated weights for policy 1, policy_version 1585046 (0.0009) [2023-12-27 16:38:17,863][20347] Updated weights for policy 0, policy_version 1586679 (0.0006) [2023-12-27 16:38:17,919][20347] Updated weights for policy 0, policy_version 1586689 (0.0005) [2023-12-27 16:38:17,974][20347] Updated weights for policy 0, policy_version 1586699 (0.0007) [2023-12-27 16:38:18,002][20381] Updated weights for policy 1, policy_version 1585056 (0.0007) [2023-12-27 16:38:18,054][20381] Updated weights for policy 1, policy_version 1585066 (0.0007) [2023-12-27 16:38:18,103][20381] Updated weights for policy 1, policy_version 1585076 (0.0007) [2023-12-27 16:38:18,630][20347] Updated weights for policy 0, policy_version 1586709 (0.0007) [2023-12-27 16:38:18,677][20347] Updated weights for policy 0, policy_version 1586719 (0.0008) [2023-12-27 16:38:18,733][20347] Updated weights for policy 0, policy_version 1586729 (0.0007) [2023-12-27 16:38:18,779][20381] Updated weights for policy 1, policy_version 1585086 (0.0007) [2023-12-27 16:38:18,834][20381] Updated weights for policy 1, policy_version 1585096 (0.0007) [2023-12-27 16:38:18,891][20381] Updated weights for policy 1, policy_version 1585106 (0.0008) [2023-12-27 16:38:19,529][20347] Updated weights for policy 0, policy_version 1586739 (0.0007) [2023-12-27 16:38:19,565][20381] Updated weights for policy 1, policy_version 1585116 (0.0008) [2023-12-27 16:38:19,583][20347] Updated weights for policy 0, policy_version 1586749 (0.0006) [2023-12-27 16:38:19,625][20381] Updated weights for policy 1, policy_version 1585126 (0.0006) [2023-12-27 16:38:19,642][20347] Updated weights for policy 0, policy_version 1586759 (0.0006) [2023-12-27 16:38:19,685][20381] Updated weights for policy 1, policy_version 1585136 (0.0006) [2023-12-27 16:38:20,294][20347] Updated weights for policy 0, policy_version 1586769 (0.0007) [2023-12-27 16:38:20,358][20347] Updated weights for policy 0, policy_version 1586779 (0.0006) [2023-12-27 16:38:20,384][20381] Updated weights for policy 1, policy_version 1585146 (0.0007) [2023-12-27 16:38:20,421][20347] Updated weights for policy 0, policy_version 1586789 (0.0006) [2023-12-27 16:38:20,439][20381] Updated weights for policy 1, policy_version 1585156 (0.0006) [2023-12-27 16:38:20,480][20347] Updated weights for policy 0, policy_version 1586799 (0.0005) [2023-12-27 16:38:20,493][20381] Updated weights for policy 1, policy_version 1585166 (0.0008) [2023-12-27 16:38:20,546][20381] Updated weights for policy 1, policy_version 1585176 (0.0008) [2023-12-27 16:38:21,152][20347] Updated weights for policy 0, policy_version 1586809 (0.0007) [2023-12-27 16:38:21,219][20347] Updated weights for policy 0, policy_version 1586819 (0.0008) [2023-12-27 16:38:21,284][20347] Updated weights for policy 0, policy_version 1586829 (0.0007) [2023-12-27 16:38:21,301][20381] Updated weights for policy 1, policy_version 1585186 (0.0010) [2023-12-27 16:38:21,361][20381] Updated weights for policy 1, policy_version 1585196 (0.0010) [2023-12-27 16:38:21,421][20381] Updated weights for policy 1, policy_version 1585206 (0.0008) [2023-12-27 16:38:22,015][20347] Updated weights for policy 0, policy_version 1586839 (0.0006) [2023-12-27 16:38:22,074][20347] Updated weights for policy 0, policy_version 1586849 (0.0008) [2023-12-27 16:38:22,136][20347] Updated weights for policy 0, policy_version 1586859 (0.0008) [2023-12-27 16:38:22,229][20381] Updated weights for policy 1, policy_version 1585216 (0.0007) [2023-12-27 16:38:22,297][20381] Updated weights for policy 1, policy_version 1585226 (0.0008) [2023-12-27 16:38:22,358][20381] Updated weights for policy 1, policy_version 1585236 (0.0008) [2023-12-27 16:38:22,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 812171264. Throughput: 0: 10178.9, 1: 10400.3. Samples: 812146108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:22,363][19170] Avg episode reward: [(0, '960.830'), (1, '444.720')] [2023-12-27 16:38:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001586864_406315008.pth... [2023-12-27 16:38:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001585680_406011904.pth [2023-12-27 16:38:22,381][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001585240_405864448.pth... [2023-12-27 16:38:22,385][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001583992_405544960.pth [2023-12-27 16:38:22,797][20347] Updated weights for policy 0, policy_version 1586869 (0.0006) [2023-12-27 16:38:22,850][20347] Updated weights for policy 0, policy_version 1586879 (0.0006) [2023-12-27 16:38:22,905][20347] Updated weights for policy 0, policy_version 1586889 (0.0005) [2023-12-27 16:38:23,053][20381] Updated weights for policy 1, policy_version 1585246 (0.0007) [2023-12-27 16:38:23,115][20381] Updated weights for policy 1, policy_version 1585256 (0.0007) [2023-12-27 16:38:23,177][20381] Updated weights for policy 1, policy_version 1585266 (0.0005) [2023-12-27 16:38:23,561][20347] Updated weights for policy 0, policy_version 1586899 (0.0007) [2023-12-27 16:38:23,612][20347] Updated weights for policy 0, policy_version 1586909 (0.0007) [2023-12-27 16:38:23,677][20347] Updated weights for policy 0, policy_version 1586919 (0.0008) [2023-12-27 16:38:23,704][20381] Updated weights for policy 1, policy_version 1585276 (0.0006) [2023-12-27 16:38:23,754][20381] Updated weights for policy 1, policy_version 1585286 (0.0006) [2023-12-27 16:38:23,806][20381] Updated weights for policy 1, policy_version 1585296 (0.0007) [2023-12-27 16:38:24,365][20347] Updated weights for policy 0, policy_version 1586929 (0.0006) [2023-12-27 16:38:24,399][20381] Updated weights for policy 1, policy_version 1585306 (0.0007) [2023-12-27 16:38:24,416][20347] Updated weights for policy 0, policy_version 1586939 (0.0005) [2023-12-27 16:38:24,459][20381] Updated weights for policy 1, policy_version 1585316 (0.0011) [2023-12-27 16:38:24,466][20347] Updated weights for policy 0, policy_version 1586949 (0.0005) [2023-12-27 16:38:24,514][20381] Updated weights for policy 1, policy_version 1585326 (0.0010) [2023-12-27 16:38:24,520][20347] Updated weights for policy 0, policy_version 1586959 (0.0005) [2023-12-27 16:38:24,568][20381] Updated weights for policy 1, policy_version 1585336 (0.0010) [2023-12-27 16:38:25,101][20347] Updated weights for policy 0, policy_version 1586969 (0.0007) [2023-12-27 16:38:25,157][20347] Updated weights for policy 0, policy_version 1586979 (0.0007) [2023-12-27 16:38:25,204][20347] Updated weights for policy 0, policy_version 1586989 (0.0007) [2023-12-27 16:38:25,259][20381] Updated weights for policy 1, policy_version 1585346 (0.0011) [2023-12-27 16:38:25,317][20381] Updated weights for policy 1, policy_version 1585356 (0.0009) [2023-12-27 16:38:25,364][20381] Updated weights for policy 1, policy_version 1585366 (0.0010) [2023-12-27 16:38:25,864][20347] Updated weights for policy 0, policy_version 1586999 (0.0007) [2023-12-27 16:38:25,922][20347] Updated weights for policy 0, policy_version 1587009 (0.0006) [2023-12-27 16:38:25,977][20347] Updated weights for policy 0, policy_version 1587019 (0.0006) [2023-12-27 16:38:26,048][20381] Updated weights for policy 1, policy_version 1585376 (0.0010) [2023-12-27 16:38:26,096][20381] Updated weights for policy 1, policy_version 1585386 (0.0010) [2023-12-27 16:38:26,147][20381] Updated weights for policy 1, policy_version 1585396 (0.0010) [2023-12-27 16:38:26,556][20347] Updated weights for policy 0, policy_version 1587029 (0.0007) [2023-12-27 16:38:26,616][20347] Updated weights for policy 0, policy_version 1587039 (0.0005) [2023-12-27 16:38:26,674][20347] Updated weights for policy 0, policy_version 1587049 (0.0005) [2023-12-27 16:38:26,783][20381] Updated weights for policy 1, policy_version 1585406 (0.0010) [2023-12-27 16:38:26,834][20381] Updated weights for policy 1, policy_version 1585416 (0.0010) [2023-12-27 16:38:26,886][20381] Updated weights for policy 1, policy_version 1585426 (0.0008) [2023-12-27 16:38:27,261][20347] Updated weights for policy 0, policy_version 1587059 (0.0006) [2023-12-27 16:38:27,318][20347] Updated weights for policy 0, policy_version 1587069 (0.0005) [2023-12-27 16:38:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 812277760. Throughput: 0: 10198.8, 1: 10410.9. Samples: 812268848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:27,363][19170] Avg episode reward: [(0, '990.210'), (1, '580.700')] [2023-12-27 16:38:27,370][20347] Updated weights for policy 0, policy_version 1587079 (0.0008) [2023-12-27 16:38:27,575][20381] Updated weights for policy 1, policy_version 1585436 (0.0007) [2023-12-27 16:38:27,635][20381] Updated weights for policy 1, policy_version 1585446 (0.0008) [2023-12-27 16:38:27,690][20381] Updated weights for policy 1, policy_version 1585456 (0.0007) [2023-12-27 16:38:27,951][20347] Updated weights for policy 0, policy_version 1587089 (0.0006) [2023-12-27 16:38:27,996][20347] Updated weights for policy 0, policy_version 1587099 (0.0008) [2023-12-27 16:38:28,050][20347] Updated weights for policy 0, policy_version 1587109 (0.0007) [2023-12-27 16:38:28,106][20347] Updated weights for policy 0, policy_version 1587119 (0.0008) [2023-12-27 16:38:28,412][20381] Updated weights for policy 1, policy_version 1585466 (0.0008) [2023-12-27 16:38:28,472][20381] Updated weights for policy 1, policy_version 1585476 (0.0011) [2023-12-27 16:38:28,536][20381] Updated weights for policy 1, policy_version 1585486 (0.0008) [2023-12-27 16:38:28,593][20381] Updated weights for policy 1, policy_version 1585496 (0.0006) [2023-12-27 16:38:28,896][20347] Updated weights for policy 0, policy_version 1587129 (0.0008) [2023-12-27 16:38:28,939][20347] Updated weights for policy 0, policy_version 1587139 (0.0007) [2023-12-27 16:38:28,987][20347] Updated weights for policy 0, policy_version 1587149 (0.0008) [2023-12-27 16:38:29,133][20381] Updated weights for policy 1, policy_version 1585506 (0.0005) [2023-12-27 16:38:29,183][20381] Updated weights for policy 1, policy_version 1585516 (0.0007) [2023-12-27 16:38:29,238][20381] Updated weights for policy 1, policy_version 1585526 (0.0008) [2023-12-27 16:38:29,755][20347] Updated weights for policy 0, policy_version 1587159 (0.0008) [2023-12-27 16:38:29,804][20347] Updated weights for policy 0, policy_version 1587169 (0.0010) [2023-12-27 16:38:29,853][20381] Updated weights for policy 1, policy_version 1585536 (0.0008) [2023-12-27 16:38:29,864][20347] Updated weights for policy 0, policy_version 1587179 (0.0007) [2023-12-27 16:38:29,909][20381] Updated weights for policy 1, policy_version 1585546 (0.0006) [2023-12-27 16:38:29,975][20381] Updated weights for policy 1, policy_version 1585556 (0.0006) [2023-12-27 16:38:30,515][20347] Updated weights for policy 0, policy_version 1587189 (0.0007) [2023-12-27 16:38:30,566][20347] Updated weights for policy 0, policy_version 1587199 (0.0008) [2023-12-27 16:38:30,614][20347] Updated weights for policy 0, policy_version 1587209 (0.0007) [2023-12-27 16:38:30,648][20381] Updated weights for policy 1, policy_version 1585566 (0.0006) [2023-12-27 16:38:30,702][20381] Updated weights for policy 1, policy_version 1585576 (0.0006) [2023-12-27 16:38:30,755][20381] Updated weights for policy 1, policy_version 1585586 (0.0005) [2023-12-27 16:38:31,367][20381] Updated weights for policy 1, policy_version 1585596 (0.0007) [2023-12-27 16:38:31,416][20347] Updated weights for policy 0, policy_version 1587219 (0.0008) [2023-12-27 16:38:31,430][20381] Updated weights for policy 1, policy_version 1585606 (0.0009) [2023-12-27 16:38:31,473][20347] Updated weights for policy 0, policy_version 1587229 (0.0005) [2023-12-27 16:38:31,488][20381] Updated weights for policy 1, policy_version 1585616 (0.0008) [2023-12-27 16:38:31,533][20347] Updated weights for policy 0, policy_version 1587239 (0.0006) [2023-12-27 16:38:32,153][20381] Updated weights for policy 1, policy_version 1585626 (0.0007) [2023-12-27 16:38:32,191][20347] Updated weights for policy 0, policy_version 1587249 (0.0006) [2023-12-27 16:38:32,212][20381] Updated weights for policy 1, policy_version 1585636 (0.0007) [2023-12-27 16:38:32,243][20347] Updated weights for policy 0, policy_version 1587259 (0.0007) [2023-12-27 16:38:32,271][20381] Updated weights for policy 1, policy_version 1585646 (0.0007) [2023-12-27 16:38:32,300][20347] Updated weights for policy 0, policy_version 1587269 (0.0007) [2023-12-27 16:38:32,323][20381] Updated weights for policy 1, policy_version 1585656 (0.0008) [2023-12-27 16:38:32,359][20347] Updated weights for policy 0, policy_version 1587279 (0.0006) [2023-12-27 16:38:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 812384256. Throughput: 0: 10199.4, 1: 10518.0. Samples: 812395648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '645.670')] [2023-12-27 16:38:33,038][20381] Updated weights for policy 1, policy_version 1585666 (0.0011) [2023-12-27 16:38:33,054][20347] Updated weights for policy 0, policy_version 1587289 (0.0005) [2023-12-27 16:38:33,103][20347] Updated weights for policy 0, policy_version 1587299 (0.0007) [2023-12-27 16:38:33,105][20381] Updated weights for policy 1, policy_version 1585676 (0.0010) [2023-12-27 16:38:33,152][20347] Updated weights for policy 0, policy_version 1587309 (0.0005) [2023-12-27 16:38:33,161][20381] Updated weights for policy 1, policy_version 1585686 (0.0008) [2023-12-27 16:38:33,810][20381] Updated weights for policy 1, policy_version 1585696 (0.0006) [2023-12-27 16:38:33,847][20347] Updated weights for policy 0, policy_version 1587319 (0.0009) [2023-12-27 16:38:33,863][20381] Updated weights for policy 1, policy_version 1585706 (0.0006) [2023-12-27 16:38:33,895][20347] Updated weights for policy 0, policy_version 1587329 (0.0010) [2023-12-27 16:38:33,911][20381] Updated weights for policy 1, policy_version 1585716 (0.0010) [2023-12-27 16:38:33,949][20347] Updated weights for policy 0, policy_version 1587339 (0.0010) [2023-12-27 16:38:34,530][20381] Updated weights for policy 1, policy_version 1585726 (0.0008) [2023-12-27 16:38:34,593][20381] Updated weights for policy 1, policy_version 1585736 (0.0007) [2023-12-27 16:38:34,651][20381] Updated weights for policy 1, policy_version 1585746 (0.0007) [2023-12-27 16:38:34,670][20347] Updated weights for policy 0, policy_version 1587349 (0.0009) [2023-12-27 16:38:34,722][20347] Updated weights for policy 0, policy_version 1587359 (0.0008) [2023-12-27 16:38:34,776][20347] Updated weights for policy 0, policy_version 1587369 (0.0007) [2023-12-27 16:38:35,342][20381] Updated weights for policy 1, policy_version 1585756 (0.0006) [2023-12-27 16:38:35,388][20381] Updated weights for policy 1, policy_version 1585766 (0.0005) [2023-12-27 16:38:35,430][20347] Updated weights for policy 0, policy_version 1587379 (0.0006) [2023-12-27 16:38:35,448][20381] Updated weights for policy 1, policy_version 1585776 (0.0005) [2023-12-27 16:38:35,477][20347] Updated weights for policy 0, policy_version 1587389 (0.0007) [2023-12-27 16:38:35,530][20347] Updated weights for policy 0, policy_version 1587399 (0.0008) [2023-12-27 16:38:36,089][20381] Updated weights for policy 1, policy_version 1585786 (0.0006) [2023-12-27 16:38:36,154][20381] Updated weights for policy 1, policy_version 1585796 (0.0009) [2023-12-27 16:38:36,207][20381] Updated weights for policy 1, policy_version 1585806 (0.0010) [2023-12-27 16:38:36,234][20347] Updated weights for policy 0, policy_version 1587409 (0.0008) [2023-12-27 16:38:36,266][20381] Updated weights for policy 1, policy_version 1585816 (0.0008) [2023-12-27 16:38:36,286][20347] Updated weights for policy 0, policy_version 1587419 (0.0007) [2023-12-27 16:38:36,343][20347] Updated weights for policy 0, policy_version 1587429 (0.0007) [2023-12-27 16:38:36,398][20347] Updated weights for policy 0, policy_version 1587439 (0.0006) [2023-12-27 16:38:37,007][20347] Updated weights for policy 0, policy_version 1587449 (0.0006) [2023-12-27 16:38:37,020][20381] Updated weights for policy 1, policy_version 1585826 (0.0006) [2023-12-27 16:38:37,062][20347] Updated weights for policy 0, policy_version 1587459 (0.0005) [2023-12-27 16:38:37,070][20381] Updated weights for policy 1, policy_version 1585836 (0.0008) [2023-12-27 16:38:37,117][20347] Updated weights for policy 0, policy_version 1587469 (0.0005) [2023-12-27 16:38:37,127][20381] Updated weights for policy 1, policy_version 1585846 (0.0008) [2023-12-27 16:38:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 812490752. Throughput: 0: 10199.8, 1: 10486.2. Samples: 812456576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '678.010')] [2023-12-27 16:38:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001585848_406020096.pth... [2023-12-27 16:38:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001587472_406470656.pth... [2023-12-27 16:38:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001584632_405708800.pth [2023-12-27 16:38:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001586288_406167552.pth [2023-12-27 16:38:37,760][20347] Updated weights for policy 0, policy_version 1587479 (0.0008) [2023-12-27 16:38:37,783][20381] Updated weights for policy 1, policy_version 1585856 (0.0010) [2023-12-27 16:38:37,807][20347] Updated weights for policy 0, policy_version 1587489 (0.0006) [2023-12-27 16:38:37,846][20381] Updated weights for policy 1, policy_version 1585866 (0.0010) [2023-12-27 16:38:37,858][20347] Updated weights for policy 0, policy_version 1587499 (0.0007) [2023-12-27 16:38:37,898][20381] Updated weights for policy 1, policy_version 1585876 (0.0006) [2023-12-27 16:38:38,538][20381] Updated weights for policy 1, policy_version 1585886 (0.0007) [2023-12-27 16:38:38,595][20347] Updated weights for policy 0, policy_version 1587509 (0.0008) [2023-12-27 16:38:38,596][20381] Updated weights for policy 1, policy_version 1585896 (0.0011) [2023-12-27 16:38:38,648][20347] Updated weights for policy 0, policy_version 1587519 (0.0005) [2023-12-27 16:38:38,653][20381] Updated weights for policy 1, policy_version 1585906 (0.0011) [2023-12-27 16:38:38,704][20347] Updated weights for policy 0, policy_version 1587529 (0.0006) [2023-12-27 16:38:39,363][20347] Updated weights for policy 0, policy_version 1587539 (0.0007) [2023-12-27 16:38:39,366][20381] Updated weights for policy 1, policy_version 1585916 (0.0009) [2023-12-27 16:38:39,438][20381] Updated weights for policy 1, policy_version 1585926 (0.0008) [2023-12-27 16:38:39,439][20347] Updated weights for policy 0, policy_version 1587549 (0.0008) [2023-12-27 16:38:39,498][20381] Updated weights for policy 1, policy_version 1585936 (0.0010) [2023-12-27 16:38:39,503][20347] Updated weights for policy 0, policy_version 1587559 (0.0009) [2023-12-27 16:38:40,221][20381] Updated weights for policy 1, policy_version 1585946 (0.0010) [2023-12-27 16:38:40,227][20347] Updated weights for policy 0, policy_version 1587569 (0.0007) [2023-12-27 16:38:40,272][20381] Updated weights for policy 1, policy_version 1585956 (0.0005) [2023-12-27 16:38:40,277][20347] Updated weights for policy 0, policy_version 1587579 (0.0008) [2023-12-27 16:38:40,324][20381] Updated weights for policy 1, policy_version 1585966 (0.0010) [2023-12-27 16:38:40,326][20347] Updated weights for policy 0, policy_version 1587589 (0.0005) [2023-12-27 16:38:40,379][20381] Updated weights for policy 1, policy_version 1585976 (0.0011) [2023-12-27 16:38:40,385][20347] Updated weights for policy 0, policy_version 1587599 (0.0007) [2023-12-27 16:38:41,026][20347] Updated weights for policy 0, policy_version 1587609 (0.0006) [2023-12-27 16:38:41,082][20347] Updated weights for policy 0, policy_version 1587619 (0.0008) [2023-12-27 16:38:41,130][20381] Updated weights for policy 1, policy_version 1585986 (0.0008) [2023-12-27 16:38:41,145][20347] Updated weights for policy 0, policy_version 1587629 (0.0008) [2023-12-27 16:38:41,195][20381] Updated weights for policy 1, policy_version 1585996 (0.0008) [2023-12-27 16:38:41,262][20381] Updated weights for policy 1, policy_version 1586006 (0.0008) [2023-12-27 16:38:41,893][20347] Updated weights for policy 0, policy_version 1587639 (0.0009) [2023-12-27 16:38:41,947][20347] Updated weights for policy 0, policy_version 1587649 (0.0006) [2023-12-27 16:38:41,960][20381] Updated weights for policy 1, policy_version 1586016 (0.0008) [2023-12-27 16:38:42,009][20347] Updated weights for policy 0, policy_version 1587659 (0.0006) [2023-12-27 16:38:42,018][20381] Updated weights for policy 1, policy_version 1586026 (0.0008) [2023-12-27 16:38:42,077][20381] Updated weights for policy 1, policy_version 1586036 (0.0009) [2023-12-27 16:38:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 812589056. Throughput: 0: 10172.2, 1: 10396.4. Samples: 812579040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '645.200')] [2023-12-27 16:38:42,753][20347] Updated weights for policy 0, policy_version 1587669 (0.0007) [2023-12-27 16:38:42,804][20347] Updated weights for policy 0, policy_version 1587679 (0.0007) [2023-12-27 16:38:42,806][20381] Updated weights for policy 1, policy_version 1586046 (0.0007) [2023-12-27 16:38:42,853][20347] Updated weights for policy 0, policy_version 1587689 (0.0006) [2023-12-27 16:38:42,859][20381] Updated weights for policy 1, policy_version 1586056 (0.0006) [2023-12-27 16:38:42,914][20381] Updated weights for policy 1, policy_version 1586066 (0.0007) [2023-12-27 16:38:43,452][20347] Updated weights for policy 0, policy_version 1587699 (0.0006) [2023-12-27 16:38:43,515][20347] Updated weights for policy 0, policy_version 1587709 (0.0008) [2023-12-27 16:38:43,572][20347] Updated weights for policy 0, policy_version 1587719 (0.0007) [2023-12-27 16:38:43,583][20381] Updated weights for policy 1, policy_version 1586076 (0.0007) [2023-12-27 16:38:43,642][20381] Updated weights for policy 1, policy_version 1586086 (0.0007) [2023-12-27 16:38:43,697][20381] Updated weights for policy 1, policy_version 1586096 (0.0008) [2023-12-27 16:38:44,275][20381] Updated weights for policy 1, policy_version 1586106 (0.0009) [2023-12-27 16:38:44,306][20347] Updated weights for policy 0, policy_version 1587729 (0.0008) [2023-12-27 16:38:44,338][20381] Updated weights for policy 1, policy_version 1586116 (0.0008) [2023-12-27 16:38:44,368][20347] Updated weights for policy 0, policy_version 1587739 (0.0007) [2023-12-27 16:38:44,404][20381] Updated weights for policy 1, policy_version 1586126 (0.0008) [2023-12-27 16:38:44,422][20347] Updated weights for policy 0, policy_version 1587749 (0.0006) [2023-12-27 16:38:44,466][20381] Updated weights for policy 1, policy_version 1586136 (0.0008) [2023-12-27 16:38:44,478][20347] Updated weights for policy 0, policy_version 1587759 (0.0006) [2023-12-27 16:38:45,137][20381] Updated weights for policy 1, policy_version 1586146 (0.0006) [2023-12-27 16:38:45,176][20347] Updated weights for policy 0, policy_version 1587769 (0.0007) [2023-12-27 16:38:45,196][20381] Updated weights for policy 1, policy_version 1586156 (0.0008) [2023-12-27 16:38:45,237][20347] Updated weights for policy 0, policy_version 1587779 (0.0006) [2023-12-27 16:38:45,248][20381] Updated weights for policy 1, policy_version 1586166 (0.0006) [2023-12-27 16:38:45,298][20347] Updated weights for policy 0, policy_version 1587789 (0.0006) [2023-12-27 16:38:45,887][20381] Updated weights for policy 1, policy_version 1586176 (0.0008) [2023-12-27 16:38:45,945][20381] Updated weights for policy 1, policy_version 1586186 (0.0008) [2023-12-27 16:38:45,980][20347] Updated weights for policy 0, policy_version 1587799 (0.0007) [2023-12-27 16:38:45,994][20381] Updated weights for policy 1, policy_version 1586196 (0.0005) [2023-12-27 16:38:46,049][20347] Updated weights for policy 0, policy_version 1587809 (0.0007) [2023-12-27 16:38:46,097][20347] Updated weights for policy 0, policy_version 1587819 (0.0007) [2023-12-27 16:38:46,724][20381] Updated weights for policy 1, policy_version 1586206 (0.0007) [2023-12-27 16:38:46,775][20381] Updated weights for policy 1, policy_version 1586216 (0.0007) [2023-12-27 16:38:46,783][20347] Updated weights for policy 0, policy_version 1587829 (0.0008) [2023-12-27 16:38:46,829][20381] Updated weights for policy 1, policy_version 1586226 (0.0008) [2023-12-27 16:38:46,830][20347] Updated weights for policy 0, policy_version 1587839 (0.0006) [2023-12-27 16:38:46,877][20347] Updated weights for policy 0, policy_version 1587849 (0.0006) [2023-12-27 16:38:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 812687360. Throughput: 0: 10164.6, 1: 10389.7. Samples: 812699372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '652.870')] [2023-12-27 16:38:47,488][20381] Updated weights for policy 1, policy_version 1586236 (0.0005) [2023-12-27 16:38:47,546][20381] Updated weights for policy 1, policy_version 1586246 (0.0005) [2023-12-27 16:38:47,595][20381] Updated weights for policy 1, policy_version 1586256 (0.0005) [2023-12-27 16:38:47,604][20347] Updated weights for policy 0, policy_version 1587859 (0.0008) [2023-12-27 16:38:47,656][20347] Updated weights for policy 0, policy_version 1587869 (0.0007) [2023-12-27 16:38:47,708][20347] Updated weights for policy 0, policy_version 1587879 (0.0008) [2023-12-27 16:38:48,214][20381] Updated weights for policy 1, policy_version 1586266 (0.0005) [2023-12-27 16:38:48,269][20381] Updated weights for policy 1, policy_version 1586276 (0.0008) [2023-12-27 16:38:48,325][20381] Updated weights for policy 1, policy_version 1586286 (0.0008) [2023-12-27 16:38:48,387][20381] Updated weights for policy 1, policy_version 1586296 (0.0008) [2023-12-27 16:38:48,402][20347] Updated weights for policy 0, policy_version 1587889 (0.0008) [2023-12-27 16:38:48,463][20347] Updated weights for policy 0, policy_version 1587899 (0.0007) [2023-12-27 16:38:48,526][20347] Updated weights for policy 0, policy_version 1587909 (0.0007) [2023-12-27 16:38:48,591][20347] Updated weights for policy 0, policy_version 1587919 (0.0007) [2023-12-27 16:38:49,072][20381] Updated weights for policy 1, policy_version 1586306 (0.0007) [2023-12-27 16:38:49,129][20381] Updated weights for policy 1, policy_version 1586316 (0.0008) [2023-12-27 16:38:49,187][20381] Updated weights for policy 1, policy_version 1586326 (0.0008) [2023-12-27 16:38:49,267][20347] Updated weights for policy 0, policy_version 1587929 (0.0007) [2023-12-27 16:38:49,312][20347] Updated weights for policy 0, policy_version 1587939 (0.0008) [2023-12-27 16:38:49,375][20347] Updated weights for policy 0, policy_version 1587949 (0.0008) [2023-12-27 16:38:49,922][20381] Updated weights for policy 1, policy_version 1586336 (0.0009) [2023-12-27 16:38:49,976][20381] Updated weights for policy 1, policy_version 1586346 (0.0009) [2023-12-27 16:38:50,029][20381] Updated weights for policy 1, policy_version 1586356 (0.0009) [2023-12-27 16:38:50,079][20347] Updated weights for policy 0, policy_version 1587959 (0.0008) [2023-12-27 16:38:50,135][20347] Updated weights for policy 0, policy_version 1587969 (0.0007) [2023-12-27 16:38:50,190][20347] Updated weights for policy 0, policy_version 1587979 (0.0008) [2023-12-27 16:38:50,714][20381] Updated weights for policy 1, policy_version 1586366 (0.0009) [2023-12-27 16:38:50,767][20381] Updated weights for policy 1, policy_version 1586376 (0.0007) [2023-12-27 16:38:50,830][20381] Updated weights for policy 1, policy_version 1586386 (0.0005) [2023-12-27 16:38:50,882][20347] Updated weights for policy 0, policy_version 1587989 (0.0007) [2023-12-27 16:38:50,945][20347] Updated weights for policy 0, policy_version 1587999 (0.0007) [2023-12-27 16:38:51,003][20347] Updated weights for policy 0, policy_version 1588009 (0.0008) [2023-12-27 16:38:51,515][20381] Updated weights for policy 1, policy_version 1586396 (0.0006) [2023-12-27 16:38:51,575][20381] Updated weights for policy 1, policy_version 1586406 (0.0008) [2023-12-27 16:38:51,643][20381] Updated weights for policy 1, policy_version 1586416 (0.0008) [2023-12-27 16:38:51,801][20347] Updated weights for policy 0, policy_version 1588019 (0.0007) [2023-12-27 16:38:51,863][20347] Updated weights for policy 0, policy_version 1588029 (0.0006) [2023-12-27 16:38:51,927][20347] Updated weights for policy 0, policy_version 1588039 (0.0009) [2023-12-27 16:38:52,281][20381] Updated weights for policy 1, policy_version 1586426 (0.0009) [2023-12-27 16:38:52,335][20381] Updated weights for policy 1, policy_version 1586436 (0.0008) [2023-12-27 16:38:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 812785664. Throughput: 0: 10174.8, 1: 10321.6. Samples: 812759428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '629.950')] [2023-12-27 16:38:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001588048_406618112.pth... [2023-12-27 16:38:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001586864_406315008.pth [2023-12-27 16:38:52,399][20381] Updated weights for policy 1, policy_version 1586446 (0.0008) [2023-12-27 16:38:52,449][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001586456_406175744.pth... [2023-12-27 16:38:52,451][20381] Updated weights for policy 1, policy_version 1586456 (0.0010) [2023-12-27 16:38:52,453][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001585240_405864448.pth [2023-12-27 16:38:52,575][20347] Updated weights for policy 0, policy_version 1588049 (0.0007) [2023-12-27 16:38:52,635][20347] Updated weights for policy 0, policy_version 1588059 (0.0006) [2023-12-27 16:38:52,689][20347] Updated weights for policy 0, policy_version 1588069 (0.0007) [2023-12-27 16:38:52,738][20347] Updated weights for policy 0, policy_version 1588079 (0.0007) [2023-12-27 16:38:53,092][20381] Updated weights for policy 1, policy_version 1586466 (0.0005) [2023-12-27 16:38:53,143][20381] Updated weights for policy 1, policy_version 1586476 (0.0005) [2023-12-27 16:38:53,201][20381] Updated weights for policy 1, policy_version 1586486 (0.0005) [2023-12-27 16:38:53,463][20347] Updated weights for policy 0, policy_version 1588089 (0.0007) [2023-12-27 16:38:53,513][20347] Updated weights for policy 0, policy_version 1588099 (0.0007) [2023-12-27 16:38:53,564][20347] Updated weights for policy 0, policy_version 1588109 (0.0007) [2023-12-27 16:38:53,787][20381] Updated weights for policy 1, policy_version 1586496 (0.0005) [2023-12-27 16:38:53,846][20381] Updated weights for policy 1, policy_version 1586506 (0.0005) [2023-12-27 16:38:53,901][20381] Updated weights for policy 1, policy_version 1586516 (0.0005) [2023-12-27 16:38:54,299][20347] Updated weights for policy 0, policy_version 1588119 (0.0007) [2023-12-27 16:38:54,358][20347] Updated weights for policy 0, policy_version 1588129 (0.0007) [2023-12-27 16:38:54,412][20347] Updated weights for policy 0, policy_version 1588139 (0.0008) [2023-12-27 16:38:54,483][20381] Updated weights for policy 1, policy_version 1586526 (0.0006) [2023-12-27 16:38:54,530][20381] Updated weights for policy 1, policy_version 1586536 (0.0005) [2023-12-27 16:38:54,582][20381] Updated weights for policy 1, policy_version 1586546 (0.0008) [2023-12-27 16:38:55,109][20347] Updated weights for policy 0, policy_version 1588149 (0.0007) [2023-12-27 16:38:55,155][20347] Updated weights for policy 0, policy_version 1588159 (0.0007) [2023-12-27 16:38:55,206][20347] Updated weights for policy 0, policy_version 1588169 (0.0008) [2023-12-27 16:38:55,283][20381] Updated weights for policy 1, policy_version 1586556 (0.0007) [2023-12-27 16:38:55,334][20381] Updated weights for policy 1, policy_version 1586566 (0.0006) [2023-12-27 16:38:55,389][20381] Updated weights for policy 1, policy_version 1586576 (0.0005) [2023-12-27 16:38:55,913][20347] Updated weights for policy 0, policy_version 1588179 (0.0008) [2023-12-27 16:38:55,969][20347] Updated weights for policy 0, policy_version 1588189 (0.0007) [2023-12-27 16:38:56,020][20381] Updated weights for policy 1, policy_version 1586586 (0.0005) [2023-12-27 16:38:56,030][20347] Updated weights for policy 0, policy_version 1588199 (0.0008) [2023-12-27 16:38:56,076][20381] Updated weights for policy 1, policy_version 1586596 (0.0006) [2023-12-27 16:38:56,129][20381] Updated weights for policy 1, policy_version 1586606 (0.0006) [2023-12-27 16:38:56,176][20381] Updated weights for policy 1, policy_version 1586616 (0.0009) [2023-12-27 16:38:56,715][20347] Updated weights for policy 0, policy_version 1588209 (0.0005) [2023-12-27 16:38:56,760][20347] Updated weights for policy 0, policy_version 1588219 (0.0005) [2023-12-27 16:38:56,803][20381] Updated weights for policy 1, policy_version 1586626 (0.0010) [2023-12-27 16:38:56,818][20347] Updated weights for policy 0, policy_version 1588229 (0.0005) [2023-12-27 16:38:56,850][20381] Updated weights for policy 1, policy_version 1586636 (0.0010) [2023-12-27 16:38:56,877][20347] Updated weights for policy 0, policy_version 1588239 (0.0006) [2023-12-27 16:38:56,898][20381] Updated weights for policy 1, policy_version 1586646 (0.0005) [2023-12-27 16:38:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 812892160. Throughput: 0: 10090.3, 1: 10360.9. Samples: 812883008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:38:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '623.800')] [2023-12-27 16:38:57,509][20347] Updated weights for policy 0, policy_version 1588249 (0.0007) [2023-12-27 16:38:57,568][20347] Updated weights for policy 0, policy_version 1588259 (0.0007) [2023-12-27 16:38:57,604][20381] Updated weights for policy 1, policy_version 1586656 (0.0006) [2023-12-27 16:38:57,620][20347] Updated weights for policy 0, policy_version 1588269 (0.0009) [2023-12-27 16:38:57,654][20381] Updated weights for policy 1, policy_version 1586666 (0.0008) [2023-12-27 16:38:57,702][20381] Updated weights for policy 1, policy_version 1586676 (0.0009) [2023-12-27 16:38:58,207][20347] Updated weights for policy 0, policy_version 1588279 (0.0009) [2023-12-27 16:38:58,267][20347] Updated weights for policy 0, policy_version 1588289 (0.0005) [2023-12-27 16:38:58,330][20347] Updated weights for policy 0, policy_version 1588299 (0.0009) [2023-12-27 16:38:58,442][20381] Updated weights for policy 1, policy_version 1586686 (0.0009) [2023-12-27 16:38:58,506][20381] Updated weights for policy 1, policy_version 1586696 (0.0010) [2023-12-27 16:38:58,573][20381] Updated weights for policy 1, policy_version 1586706 (0.0010) [2023-12-27 16:38:59,114][20347] Updated weights for policy 0, policy_version 1588309 (0.0010) [2023-12-27 16:38:59,175][20347] Updated weights for policy 0, policy_version 1588319 (0.0007) [2023-12-27 16:38:59,254][20347] Updated weights for policy 0, policy_version 1588331 (0.0008) [2023-12-27 16:38:59,430][20381] Updated weights for policy 1, policy_version 1586716 (0.0008) [2023-12-27 16:38:59,486][20381] Updated weights for policy 1, policy_version 1586726 (0.0006) [2023-12-27 16:38:59,549][20381] Updated weights for policy 1, policy_version 1586736 (0.0006) [2023-12-27 16:38:59,966][20347] Updated weights for policy 0, policy_version 1588341 (0.0008) [2023-12-27 16:39:00,025][20347] Updated weights for policy 0, policy_version 1588351 (0.0006) [2023-12-27 16:39:00,075][20347] Updated weights for policy 0, policy_version 1588361 (0.0007) [2023-12-27 16:39:00,222][20381] Updated weights for policy 1, policy_version 1586746 (0.0006) [2023-12-27 16:39:00,278][20381] Updated weights for policy 1, policy_version 1586756 (0.0010) [2023-12-27 16:39:00,327][20381] Updated weights for policy 1, policy_version 1586766 (0.0010) [2023-12-27 16:39:00,386][20381] Updated weights for policy 1, policy_version 1586776 (0.0011) [2023-12-27 16:39:00,740][20347] Updated weights for policy 0, policy_version 1588371 (0.0006) [2023-12-27 16:39:00,785][20347] Updated weights for policy 0, policy_version 1588381 (0.0005) [2023-12-27 16:39:00,835][20347] Updated weights for policy 0, policy_version 1588391 (0.0005) [2023-12-27 16:39:00,975][20381] Updated weights for policy 1, policy_version 1586786 (0.0009) [2023-12-27 16:39:01,030][20381] Updated weights for policy 1, policy_version 1586796 (0.0010) [2023-12-27 16:39:01,085][20381] Updated weights for policy 1, policy_version 1586806 (0.0008) [2023-12-27 16:39:01,554][20347] Updated weights for policy 0, policy_version 1588401 (0.0007) [2023-12-27 16:39:01,611][20347] Updated weights for policy 0, policy_version 1588411 (0.0008) [2023-12-27 16:39:01,674][20347] Updated weights for policy 0, policy_version 1588421 (0.0007) [2023-12-27 16:39:01,731][20347] Updated weights for policy 0, policy_version 1588431 (0.0008) [2023-12-27 16:39:01,804][20381] Updated weights for policy 1, policy_version 1586816 (0.0009) [2023-12-27 16:39:01,851][20381] Updated weights for policy 1, policy_version 1586826 (0.0007) [2023-12-27 16:39:01,901][20381] Updated weights for policy 1, policy_version 1586836 (0.0007) [2023-12-27 16:39:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 812990464. Throughput: 0: 10123.3, 1: 10294.3. Samples: 813003916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '418.960')] [2023-12-27 16:39:02,459][20347] Updated weights for policy 0, policy_version 1588441 (0.0006) [2023-12-27 16:39:02,505][20347] Updated weights for policy 0, policy_version 1588451 (0.0010) [2023-12-27 16:39:02,562][20347] Updated weights for policy 0, policy_version 1588461 (0.0007) [2023-12-27 16:39:02,578][20381] Updated weights for policy 1, policy_version 1586846 (0.0006) [2023-12-27 16:39:02,641][20381] Updated weights for policy 1, policy_version 1586856 (0.0008) [2023-12-27 16:39:02,699][20381] Updated weights for policy 1, policy_version 1586866 (0.0011) [2023-12-27 16:39:03,162][20347] Updated weights for policy 0, policy_version 1588471 (0.0008) [2023-12-27 16:39:03,219][20347] Updated weights for policy 0, policy_version 1588481 (0.0007) [2023-12-27 16:39:03,276][20347] Updated weights for policy 0, policy_version 1588491 (0.0006) [2023-12-27 16:39:03,360][20381] Updated weights for policy 1, policy_version 1586876 (0.0008) [2023-12-27 16:39:03,425][20381] Updated weights for policy 1, policy_version 1586886 (0.0009) [2023-12-27 16:39:03,469][20381] Updated weights for policy 1, policy_version 1586896 (0.0007) [2023-12-27 16:39:03,966][20347] Updated weights for policy 0, policy_version 1588501 (0.0007) [2023-12-27 16:39:04,025][20347] Updated weights for policy 0, policy_version 1588511 (0.0007) [2023-12-27 16:39:04,091][20347] Updated weights for policy 0, policy_version 1588521 (0.0007) [2023-12-27 16:39:04,092][20381] Updated weights for policy 1, policy_version 1586906 (0.0005) [2023-12-27 16:39:04,158][20381] Updated weights for policy 1, policy_version 1586916 (0.0006) [2023-12-27 16:39:04,215][20381] Updated weights for policy 1, policy_version 1586926 (0.0008) [2023-12-27 16:39:04,265][20381] Updated weights for policy 1, policy_version 1586936 (0.0008) [2023-12-27 16:39:04,812][20347] Updated weights for policy 0, policy_version 1588531 (0.0006) [2023-12-27 16:39:04,857][20347] Updated weights for policy 0, policy_version 1588541 (0.0005) [2023-12-27 16:39:04,902][20347] Updated weights for policy 0, policy_version 1588551 (0.0005) [2023-12-27 16:39:04,945][20381] Updated weights for policy 1, policy_version 1586946 (0.0007) [2023-12-27 16:39:05,001][20381] Updated weights for policy 1, policy_version 1586956 (0.0009) [2023-12-27 16:39:05,045][20381] Updated weights for policy 1, policy_version 1586966 (0.0007) [2023-12-27 16:39:05,530][20347] Updated weights for policy 0, policy_version 1588561 (0.0006) [2023-12-27 16:39:05,576][20347] Updated weights for policy 0, policy_version 1588571 (0.0007) [2023-12-27 16:39:05,623][20347] Updated weights for policy 0, policy_version 1588581 (0.0006) [2023-12-27 16:39:05,676][20347] Updated weights for policy 0, policy_version 1588591 (0.0005) [2023-12-27 16:39:05,710][20381] Updated weights for policy 1, policy_version 1586976 (0.0008) [2023-12-27 16:39:05,758][20381] Updated weights for policy 1, policy_version 1586986 (0.0008) [2023-12-27 16:39:05,809][20381] Updated weights for policy 1, policy_version 1586996 (0.0005) [2023-12-27 16:39:06,252][20347] Updated weights for policy 0, policy_version 1588601 (0.0007) [2023-12-27 16:39:06,314][20347] Updated weights for policy 0, policy_version 1588611 (0.0007) [2023-12-27 16:39:06,376][20347] Updated weights for policy 0, policy_version 1588621 (0.0006) [2023-12-27 16:39:06,564][20381] Updated weights for policy 1, policy_version 1587006 (0.0007) [2023-12-27 16:39:06,623][20381] Updated weights for policy 1, policy_version 1587016 (0.0007) [2023-12-27 16:39:06,680][20381] Updated weights for policy 1, policy_version 1587026 (0.0008) [2023-12-27 16:39:07,039][20347] Updated weights for policy 0, policy_version 1588631 (0.0007) [2023-12-27 16:39:07,091][20347] Updated weights for policy 0, policy_version 1588641 (0.0006) [2023-12-27 16:39:07,147][20347] Updated weights for policy 0, policy_version 1588651 (0.0005) [2023-12-27 16:39:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 813096960. Throughput: 0: 10142.4, 1: 10296.1. Samples: 813065840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:07,363][19170] Avg episode reward: [(0, '970.810'), (1, '421.450')] [2023-12-27 16:39:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001588656_406773760.pth... [2023-12-27 16:39:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001587472_406470656.pth [2023-12-27 16:39:07,383][20381] Updated weights for policy 1, policy_version 1587036 (0.0008) [2023-12-27 16:39:07,442][20381] Updated weights for policy 1, policy_version 1587046 (0.0007) [2023-12-27 16:39:07,500][20381] Updated weights for policy 1, policy_version 1587056 (0.0007) [2023-12-27 16:39:07,545][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001587064_406331392.pth... [2023-12-27 16:39:07,549][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001585848_406020096.pth [2023-12-27 16:39:07,808][20347] Updated weights for policy 0, policy_version 1588661 (0.0008) [2023-12-27 16:39:07,863][20347] Updated weights for policy 0, policy_version 1588671 (0.0010) [2023-12-27 16:39:07,922][20347] Updated weights for policy 0, policy_version 1588681 (0.0010) [2023-12-27 16:39:08,195][20381] Updated weights for policy 1, policy_version 1587066 (0.0008) [2023-12-27 16:39:08,245][20381] Updated weights for policy 1, policy_version 1587076 (0.0007) [2023-12-27 16:39:08,299][20381] Updated weights for policy 1, policy_version 1587086 (0.0006) [2023-12-27 16:39:08,353][20381] Updated weights for policy 1, policy_version 1587096 (0.0007) [2023-12-27 16:39:08,623][20347] Updated weights for policy 0, policy_version 1588691 (0.0010) [2023-12-27 16:39:08,686][20347] Updated weights for policy 0, policy_version 1588701 (0.0011) [2023-12-27 16:39:08,748][20347] Updated weights for policy 0, policy_version 1588711 (0.0010) [2023-12-27 16:39:09,078][20381] Updated weights for policy 1, policy_version 1587106 (0.0007) [2023-12-27 16:39:09,131][20381] Updated weights for policy 1, policy_version 1587116 (0.0007) [2023-12-27 16:39:09,186][20381] Updated weights for policy 1, policy_version 1587126 (0.0007) [2023-12-27 16:39:09,445][20347] Updated weights for policy 0, policy_version 1588721 (0.0008) [2023-12-27 16:39:09,502][20347] Updated weights for policy 0, policy_version 1588731 (0.0011) [2023-12-27 16:39:09,558][20347] Updated weights for policy 0, policy_version 1588741 (0.0010) [2023-12-27 16:39:09,612][20347] Updated weights for policy 0, policy_version 1588751 (0.0010) [2023-12-27 16:39:09,970][20381] Updated weights for policy 1, policy_version 1587136 (0.0008) [2023-12-27 16:39:10,026][20381] Updated weights for policy 1, policy_version 1587146 (0.0007) [2023-12-27 16:39:10,085][20381] Updated weights for policy 1, policy_version 1587156 (0.0008) [2023-12-27 16:39:10,340][20347] Updated weights for policy 0, policy_version 1588761 (0.0010) [2023-12-27 16:39:10,397][20347] Updated weights for policy 0, policy_version 1588771 (0.0010) [2023-12-27 16:39:10,459][20347] Updated weights for policy 0, policy_version 1588781 (0.0008) [2023-12-27 16:39:10,769][20381] Updated weights for policy 1, policy_version 1587166 (0.0007) [2023-12-27 16:39:10,823][20381] Updated weights for policy 1, policy_version 1587176 (0.0008) [2023-12-27 16:39:10,882][20381] Updated weights for policy 1, policy_version 1587186 (0.0006) [2023-12-27 16:39:11,151][20347] Updated weights for policy 0, policy_version 1588791 (0.0007) [2023-12-27 16:39:11,210][20347] Updated weights for policy 0, policy_version 1588801 (0.0007) [2023-12-27 16:39:11,273][20347] Updated weights for policy 0, policy_version 1588811 (0.0006) [2023-12-27 16:39:11,582][20381] Updated weights for policy 1, policy_version 1587196 (0.0007) [2023-12-27 16:39:11,644][20381] Updated weights for policy 1, policy_version 1587206 (0.0007) [2023-12-27 16:39:11,707][20381] Updated weights for policy 1, policy_version 1587216 (0.0008) [2023-12-27 16:39:11,986][20347] Updated weights for policy 0, policy_version 1588821 (0.0007) [2023-12-27 16:39:12,041][20347] Updated weights for policy 0, policy_version 1588831 (0.0008) [2023-12-27 16:39:12,093][20347] Updated weights for policy 0, policy_version 1588841 (0.0008) [2023-12-27 16:39:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 813195264. Throughput: 0: 10146.1, 1: 10257.8. Samples: 813187024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:12,364][19170] Avg episode reward: [(0, '970.770'), (1, '557.220')] [2023-12-27 16:39:12,437][20381] Updated weights for policy 1, policy_version 1587226 (0.0008) [2023-12-27 16:39:12,491][20381] Updated weights for policy 1, policy_version 1587236 (0.0008) [2023-12-27 16:39:12,553][20381] Updated weights for policy 1, policy_version 1587246 (0.0010) [2023-12-27 16:39:12,617][20381] Updated weights for policy 1, policy_version 1587256 (0.0009) [2023-12-27 16:39:12,786][20347] Updated weights for policy 0, policy_version 1588851 (0.0007) [2023-12-27 16:39:12,839][20347] Updated weights for policy 0, policy_version 1588861 (0.0008) [2023-12-27 16:39:12,894][20347] Updated weights for policy 0, policy_version 1588871 (0.0007) [2023-12-27 16:39:13,279][20381] Updated weights for policy 1, policy_version 1587266 (0.0005) [2023-12-27 16:39:13,328][20381] Updated weights for policy 1, policy_version 1587276 (0.0005) [2023-12-27 16:39:13,380][20381] Updated weights for policy 1, policy_version 1587286 (0.0005) [2023-12-27 16:39:13,612][20347] Updated weights for policy 0, policy_version 1588881 (0.0006) [2023-12-27 16:39:13,671][20347] Updated weights for policy 0, policy_version 1588891 (0.0009) [2023-12-27 16:39:13,731][20347] Updated weights for policy 0, policy_version 1588901 (0.0007) [2023-12-27 16:39:13,782][20347] Updated weights for policy 0, policy_version 1588911 (0.0006) [2023-12-27 16:39:13,995][20381] Updated weights for policy 1, policy_version 1587296 (0.0006) [2023-12-27 16:39:14,054][20381] Updated weights for policy 1, policy_version 1587306 (0.0006) [2023-12-27 16:39:14,109][20381] Updated weights for policy 1, policy_version 1587316 (0.0008) [2023-12-27 16:39:14,462][20347] Updated weights for policy 0, policy_version 1588921 (0.0007) [2023-12-27 16:39:14,510][20347] Updated weights for policy 0, policy_version 1588931 (0.0009) [2023-12-27 16:39:14,562][20347] Updated weights for policy 0, policy_version 1588941 (0.0010) [2023-12-27 16:39:14,693][20381] Updated weights for policy 1, policy_version 1587326 (0.0008) [2023-12-27 16:39:14,741][20381] Updated weights for policy 1, policy_version 1587336 (0.0007) [2023-12-27 16:39:14,804][20381] Updated weights for policy 1, policy_version 1587346 (0.0007) [2023-12-27 16:39:15,304][20347] Updated weights for policy 0, policy_version 1588951 (0.0011) [2023-12-27 16:39:15,365][20347] Updated weights for policy 0, policy_version 1588961 (0.0010) [2023-12-27 16:39:15,426][20347] Updated weights for policy 0, policy_version 1588971 (0.0009) [2023-12-27 16:39:15,471][20381] Updated weights for policy 1, policy_version 1587356 (0.0008) [2023-12-27 16:39:15,515][20381] Updated weights for policy 1, policy_version 1587366 (0.0008) [2023-12-27 16:39:15,569][20381] Updated weights for policy 1, policy_version 1587376 (0.0005) [2023-12-27 16:39:15,963][20347] Updated weights for policy 0, policy_version 1588981 (0.0006) [2023-12-27 16:39:16,018][20347] Updated weights for policy 0, policy_version 1588991 (0.0005) [2023-12-27 16:39:16,068][20347] Updated weights for policy 0, policy_version 1589001 (0.0005) [2023-12-27 16:39:16,229][20381] Updated weights for policy 1, policy_version 1587386 (0.0006) [2023-12-27 16:39:16,275][20381] Updated weights for policy 1, policy_version 1587396 (0.0007) [2023-12-27 16:39:16,327][20381] Updated weights for policy 1, policy_version 1587406 (0.0007) [2023-12-27 16:39:16,379][20381] Updated weights for policy 1, policy_version 1587416 (0.0008) [2023-12-27 16:39:16,703][20347] Updated weights for policy 0, policy_version 1589011 (0.0006) [2023-12-27 16:39:16,751][20347] Updated weights for policy 0, policy_version 1589021 (0.0009) [2023-12-27 16:39:16,807][20347] Updated weights for policy 0, policy_version 1589031 (0.0007) [2023-12-27 16:39:17,054][20381] Updated weights for policy 1, policy_version 1587426 (0.0007) [2023-12-27 16:39:17,111][20381] Updated weights for policy 1, policy_version 1587436 (0.0008) [2023-12-27 16:39:17,165][20381] Updated weights for policy 1, policy_version 1587446 (0.0007) [2023-12-27 16:39:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 813301760. Throughput: 0: 10126.8, 1: 10218.0. Samples: 813311160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:17,363][19170] Avg episode reward: [(0, '990.240'), (1, '568.820')] [2023-12-27 16:39:17,464][20347] Updated weights for policy 0, policy_version 1589041 (0.0005) [2023-12-27 16:39:17,509][20347] Updated weights for policy 0, policy_version 1589051 (0.0005) [2023-12-27 16:39:17,555][20347] Updated weights for policy 0, policy_version 1589061 (0.0007) [2023-12-27 16:39:17,599][20347] Updated weights for policy 0, policy_version 1589071 (0.0007) [2023-12-27 16:39:17,857][20381] Updated weights for policy 1, policy_version 1587456 (0.0006) [2023-12-27 16:39:17,915][20381] Updated weights for policy 1, policy_version 1587466 (0.0005) [2023-12-27 16:39:17,968][20381] Updated weights for policy 1, policy_version 1587476 (0.0007) [2023-12-27 16:39:18,288][20347] Updated weights for policy 0, policy_version 1589081 (0.0007) [2023-12-27 16:39:18,354][20347] Updated weights for policy 0, policy_version 1589091 (0.0008) [2023-12-27 16:39:18,407][20347] Updated weights for policy 0, policy_version 1589101 (0.0008) [2023-12-27 16:39:18,620][20381] Updated weights for policy 1, policy_version 1587486 (0.0008) [2023-12-27 16:39:18,677][20381] Updated weights for policy 1, policy_version 1587496 (0.0008) [2023-12-27 16:39:18,738][20381] Updated weights for policy 1, policy_version 1587506 (0.0008) [2023-12-27 16:39:19,063][20347] Updated weights for policy 0, policy_version 1589111 (0.0008) [2023-12-27 16:39:19,123][20347] Updated weights for policy 0, policy_version 1589121 (0.0008) [2023-12-27 16:39:19,181][20347] Updated weights for policy 0, policy_version 1589131 (0.0007) [2023-12-27 16:39:19,449][20381] Updated weights for policy 1, policy_version 1587516 (0.0008) [2023-12-27 16:39:19,503][20381] Updated weights for policy 1, policy_version 1587526 (0.0009) [2023-12-27 16:39:19,562][20381] Updated weights for policy 1, policy_version 1587536 (0.0008) [2023-12-27 16:39:19,886][20347] Updated weights for policy 0, policy_version 1589141 (0.0007) [2023-12-27 16:39:19,952][20347] Updated weights for policy 0, policy_version 1589151 (0.0008) [2023-12-27 16:39:20,019][20347] Updated weights for policy 0, policy_version 1589161 (0.0007) [2023-12-27 16:39:20,244][20381] Updated weights for policy 1, policy_version 1587546 (0.0008) [2023-12-27 16:39:20,306][20381] Updated weights for policy 1, policy_version 1587556 (0.0006) [2023-12-27 16:39:20,364][20381] Updated weights for policy 1, policy_version 1587566 (0.0008) [2023-12-27 16:39:20,413][20381] Updated weights for policy 1, policy_version 1587576 (0.0008) [2023-12-27 16:39:20,712][20347] Updated weights for policy 0, policy_version 1589171 (0.0007) [2023-12-27 16:39:20,777][20347] Updated weights for policy 0, policy_version 1589181 (0.0006) [2023-12-27 16:39:20,838][20347] Updated weights for policy 0, policy_version 1589191 (0.0008) [2023-12-27 16:39:21,068][20381] Updated weights for policy 1, policy_version 1587586 (0.0008) [2023-12-27 16:39:21,132][20381] Updated weights for policy 1, policy_version 1587596 (0.0007) [2023-12-27 16:39:21,191][20381] Updated weights for policy 1, policy_version 1587606 (0.0008) [2023-12-27 16:39:21,562][20347] Updated weights for policy 0, policy_version 1589201 (0.0008) [2023-12-27 16:39:21,624][20347] Updated weights for policy 0, policy_version 1589211 (0.0007) [2023-12-27 16:39:21,690][20347] Updated weights for policy 0, policy_version 1589221 (0.0007) [2023-12-27 16:39:21,757][20347] Updated weights for policy 0, policy_version 1589231 (0.0007) [2023-12-27 16:39:21,974][20381] Updated weights for policy 1, policy_version 1587616 (0.0011) [2023-12-27 16:39:22,035][20381] Updated weights for policy 1, policy_version 1587626 (0.0011) [2023-12-27 16:39:22,092][20381] Updated weights for policy 1, policy_version 1587636 (0.0011) [2023-12-27 16:39:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.6). Total num frames: 813400064. Throughput: 0: 10126.7, 1: 10227.3. Samples: 813372504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:22,363][19170] Avg episode reward: [(0, '990.220'), (1, '345.080')] [2023-12-27 16:39:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001587640_406478848.pth... [2023-12-27 16:39:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001589232_406921216.pth... [2023-12-27 16:39:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001586456_406175744.pth [2023-12-27 16:39:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001588048_406618112.pth [2023-12-27 16:39:22,492][20347] Updated weights for policy 0, policy_version 1589241 (0.0008) [2023-12-27 16:39:22,555][20347] Updated weights for policy 0, policy_version 1589251 (0.0009) [2023-12-27 16:39:22,616][20347] Updated weights for policy 0, policy_version 1589261 (0.0007) [2023-12-27 16:39:22,824][20381] Updated weights for policy 1, policy_version 1587646 (0.0011) [2023-12-27 16:39:22,885][20381] Updated weights for policy 1, policy_version 1587656 (0.0011) [2023-12-27 16:39:22,945][20381] Updated weights for policy 1, policy_version 1587666 (0.0011) [2023-12-27 16:39:23,266][20347] Updated weights for policy 0, policy_version 1589271 (0.0006) [2023-12-27 16:39:23,325][20347] Updated weights for policy 0, policy_version 1589281 (0.0007) [2023-12-27 16:39:23,379][20347] Updated weights for policy 0, policy_version 1589291 (0.0007) [2023-12-27 16:39:23,610][20381] Updated weights for policy 1, policy_version 1587676 (0.0009) [2023-12-27 16:39:23,672][20381] Updated weights for policy 1, policy_version 1587686 (0.0005) [2023-12-27 16:39:23,731][20381] Updated weights for policy 1, policy_version 1587696 (0.0005) [2023-12-27 16:39:23,995][20347] Updated weights for policy 0, policy_version 1589301 (0.0007) [2023-12-27 16:39:24,057][20347] Updated weights for policy 0, policy_version 1589311 (0.0008) [2023-12-27 16:39:24,115][20347] Updated weights for policy 0, policy_version 1589321 (0.0007) [2023-12-27 16:39:24,287][20381] Updated weights for policy 1, policy_version 1587706 (0.0005) [2023-12-27 16:39:24,343][20381] Updated weights for policy 1, policy_version 1587716 (0.0005) [2023-12-27 16:39:24,395][20381] Updated weights for policy 1, policy_version 1587726 (0.0005) [2023-12-27 16:39:24,448][20381] Updated weights for policy 1, policy_version 1587736 (0.0005) [2023-12-27 16:39:24,759][20347] Updated weights for policy 0, policy_version 1589331 (0.0007) [2023-12-27 16:39:24,807][20347] Updated weights for policy 0, policy_version 1589341 (0.0005) [2023-12-27 16:39:24,858][20347] Updated weights for policy 0, policy_version 1589351 (0.0005) [2023-12-27 16:39:25,080][20381] Updated weights for policy 1, policy_version 1587746 (0.0006) [2023-12-27 16:39:25,128][20381] Updated weights for policy 1, policy_version 1587756 (0.0005) [2023-12-27 16:39:25,176][20381] Updated weights for policy 1, policy_version 1587766 (0.0005) [2023-12-27 16:39:25,389][20347] Updated weights for policy 0, policy_version 1589361 (0.0005) [2023-12-27 16:39:25,444][20347] Updated weights for policy 0, policy_version 1589371 (0.0005) [2023-12-27 16:39:25,489][20347] Updated weights for policy 0, policy_version 1589381 (0.0005) [2023-12-27 16:39:25,541][20347] Updated weights for policy 0, policy_version 1589391 (0.0006) [2023-12-27 16:39:25,844][20381] Updated weights for policy 1, policy_version 1587776 (0.0008) [2023-12-27 16:39:25,892][20381] Updated weights for policy 1, policy_version 1587786 (0.0008) [2023-12-27 16:39:25,947][20381] Updated weights for policy 1, policy_version 1587796 (0.0008) [2023-12-27 16:39:26,118][20347] Updated weights for policy 0, policy_version 1589401 (0.0008) [2023-12-27 16:39:26,167][20347] Updated weights for policy 0, policy_version 1589411 (0.0007) [2023-12-27 16:39:26,219][20347] Updated weights for policy 0, policy_version 1589421 (0.0008) [2023-12-27 16:39:26,656][20381] Updated weights for policy 1, policy_version 1587806 (0.0007) [2023-12-27 16:39:26,709][20381] Updated weights for policy 1, policy_version 1587816 (0.0008) [2023-12-27 16:39:26,763][20381] Updated weights for policy 1, policy_version 1587826 (0.0007) [2023-12-27 16:39:26,914][20347] Updated weights for policy 0, policy_version 1589431 (0.0007) [2023-12-27 16:39:26,962][20347] Updated weights for policy 0, policy_version 1589441 (0.0008) [2023-12-27 16:39:27,005][20347] Updated weights for policy 0, policy_version 1589451 (0.0007) [2023-12-27 16:39:27,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 813506560. Throughput: 0: 10175.1, 1: 10249.9. Samples: 813498172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:27,364][19170] Avg episode reward: [(0, '980.460'), (1, '449.030')] [2023-12-27 16:39:27,443][20381] Updated weights for policy 1, policy_version 1587836 (0.0008) [2023-12-27 16:39:27,498][20381] Updated weights for policy 1, policy_version 1587846 (0.0007) [2023-12-27 16:39:27,559][20381] Updated weights for policy 1, policy_version 1587856 (0.0007) [2023-12-27 16:39:27,689][20347] Updated weights for policy 0, policy_version 1589461 (0.0006) [2023-12-27 16:39:27,748][20347] Updated weights for policy 0, policy_version 1589471 (0.0005) [2023-12-27 16:39:27,810][20347] Updated weights for policy 0, policy_version 1589481 (0.0005) [2023-12-27 16:39:28,275][20381] Updated weights for policy 1, policy_version 1587866 (0.0009) [2023-12-27 16:39:28,332][20381] Updated weights for policy 1, policy_version 1587876 (0.0008) [2023-12-27 16:39:28,397][20381] Updated weights for policy 1, policy_version 1587886 (0.0007) [2023-12-27 16:39:28,399][20347] Updated weights for policy 0, policy_version 1589491 (0.0006) [2023-12-27 16:39:28,456][20381] Updated weights for policy 1, policy_version 1587896 (0.0008) [2023-12-27 16:39:28,458][20347] Updated weights for policy 0, policy_version 1589501 (0.0007) [2023-12-27 16:39:28,510][20347] Updated weights for policy 0, policy_version 1589511 (0.0005) [2023-12-27 16:39:29,120][20381] Updated weights for policy 1, policy_version 1587906 (0.0008) [2023-12-27 16:39:29,178][20347] Updated weights for policy 0, policy_version 1589521 (0.0007) [2023-12-27 16:39:29,178][20381] Updated weights for policy 1, policy_version 1587916 (0.0008) [2023-12-27 16:39:29,233][20381] Updated weights for policy 1, policy_version 1587926 (0.0007) [2023-12-27 16:39:29,239][20347] Updated weights for policy 0, policy_version 1589531 (0.0007) [2023-12-27 16:39:29,293][20347] Updated weights for policy 0, policy_version 1589541 (0.0008) [2023-12-27 16:39:29,364][20347] Updated weights for policy 0, policy_version 1589551 (0.0008) [2023-12-27 16:39:29,970][20381] Updated weights for policy 1, policy_version 1587936 (0.0008) [2023-12-27 16:39:30,026][20381] Updated weights for policy 1, policy_version 1587946 (0.0006) [2023-12-27 16:39:30,074][20381] Updated weights for policy 1, policy_version 1587956 (0.0007) [2023-12-27 16:39:30,087][20347] Updated weights for policy 0, policy_version 1589561 (0.0007) [2023-12-27 16:39:30,146][20347] Updated weights for policy 0, policy_version 1589571 (0.0007) [2023-12-27 16:39:30,200][20347] Updated weights for policy 0, policy_version 1589581 (0.0007) [2023-12-27 16:39:30,757][20381] Updated weights for policy 1, policy_version 1587966 (0.0008) [2023-12-27 16:39:30,812][20381] Updated weights for policy 1, policy_version 1587976 (0.0007) [2023-12-27 16:39:30,865][20381] Updated weights for policy 1, policy_version 1587986 (0.0007) [2023-12-27 16:39:30,885][20347] Updated weights for policy 0, policy_version 1589591 (0.0008) [2023-12-27 16:39:30,928][20347] Updated weights for policy 0, policy_version 1589601 (0.0007) [2023-12-27 16:39:30,973][20347] Updated weights for policy 0, policy_version 1589611 (0.0007) [2023-12-27 16:39:31,599][20381] Updated weights for policy 1, policy_version 1587996 (0.0007) [2023-12-27 16:39:31,655][20381] Updated weights for policy 1, policy_version 1588006 (0.0008) [2023-12-27 16:39:31,706][20381] Updated weights for policy 1, policy_version 1588016 (0.0008) [2023-12-27 16:39:31,721][20347] Updated weights for policy 0, policy_version 1589621 (0.0006) [2023-12-27 16:39:31,782][20347] Updated weights for policy 0, policy_version 1589631 (0.0007) [2023-12-27 16:39:31,834][20347] Updated weights for policy 0, policy_version 1589641 (0.0007) [2023-12-27 16:39:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 813604864. Throughput: 0: 10214.1, 1: 10224.2. Samples: 813619100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:32,363][19170] Avg episode reward: [(0, '990.240'), (1, '448.660')] [2023-12-27 16:39:32,408][20381] Updated weights for policy 1, policy_version 1588026 (0.0009) [2023-12-27 16:39:32,463][20381] Updated weights for policy 1, policy_version 1588036 (0.0007) [2023-12-27 16:39:32,511][20381] Updated weights for policy 1, policy_version 1588046 (0.0007) [2023-12-27 16:39:32,549][20347] Updated weights for policy 0, policy_version 1589651 (0.0008) [2023-12-27 16:39:32,561][20381] Updated weights for policy 1, policy_version 1588056 (0.0008) [2023-12-27 16:39:32,604][20347] Updated weights for policy 0, policy_version 1589661 (0.0010) [2023-12-27 16:39:32,660][20347] Updated weights for policy 0, policy_version 1589671 (0.0011) [2023-12-27 16:39:33,216][20381] Updated weights for policy 1, policy_version 1588066 (0.0005) [2023-12-27 16:39:33,268][20381] Updated weights for policy 1, policy_version 1588076 (0.0005) [2023-12-27 16:39:33,274][20347] Updated weights for policy 0, policy_version 1589681 (0.0010) [2023-12-27 16:39:33,326][20381] Updated weights for policy 1, policy_version 1588086 (0.0005) [2023-12-27 16:39:33,327][20347] Updated weights for policy 0, policy_version 1589691 (0.0007) [2023-12-27 16:39:33,373][20347] Updated weights for policy 0, policy_version 1589701 (0.0009) [2023-12-27 16:39:33,419][20347] Updated weights for policy 0, policy_version 1589711 (0.0006) [2023-12-27 16:39:33,900][20381] Updated weights for policy 1, policy_version 1588096 (0.0005) [2023-12-27 16:39:33,954][20381] Updated weights for policy 1, policy_version 1588106 (0.0005) [2023-12-27 16:39:34,012][20381] Updated weights for policy 1, policy_version 1588116 (0.0010) [2023-12-27 16:39:34,029][20347] Updated weights for policy 0, policy_version 1589721 (0.0005) [2023-12-27 16:39:34,080][20347] Updated weights for policy 0, policy_version 1589731 (0.0005) [2023-12-27 16:39:34,151][20347] Updated weights for policy 0, policy_version 1589741 (0.0006) [2023-12-27 16:39:34,674][20381] Updated weights for policy 1, policy_version 1588126 (0.0008) [2023-12-27 16:39:34,721][20381] Updated weights for policy 1, policy_version 1588136 (0.0007) [2023-12-27 16:39:34,774][20381] Updated weights for policy 1, policy_version 1588146 (0.0007) [2023-12-27 16:39:34,819][20347] Updated weights for policy 0, policy_version 1589751 (0.0008) [2023-12-27 16:39:34,874][20347] Updated weights for policy 0, policy_version 1589761 (0.0010) [2023-12-27 16:39:34,943][20347] Updated weights for policy 0, policy_version 1589771 (0.0009) [2023-12-27 16:39:35,466][20381] Updated weights for policy 1, policy_version 1588156 (0.0009) [2023-12-27 16:39:35,521][20381] Updated weights for policy 1, policy_version 1588166 (0.0008) [2023-12-27 16:39:35,530][20347] Updated weights for policy 0, policy_version 1589781 (0.0007) [2023-12-27 16:39:35,577][20381] Updated weights for policy 1, policy_version 1588176 (0.0007) [2023-12-27 16:39:35,584][20347] Updated weights for policy 0, policy_version 1589791 (0.0005) [2023-12-27 16:39:35,640][20347] Updated weights for policy 0, policy_version 1589801 (0.0007) [2023-12-27 16:39:36,245][20347] Updated weights for policy 0, policy_version 1589811 (0.0008) [2023-12-27 16:39:36,300][20347] Updated weights for policy 0, policy_version 1589821 (0.0007) [2023-12-27 16:39:36,319][20381] Updated weights for policy 1, policy_version 1588186 (0.0007) [2023-12-27 16:39:36,361][20347] Updated weights for policy 0, policy_version 1589831 (0.0008) [2023-12-27 16:39:36,379][20381] Updated weights for policy 1, policy_version 1588196 (0.0005) [2023-12-27 16:39:36,440][20381] Updated weights for policy 1, policy_version 1588206 (0.0007) [2023-12-27 16:39:36,500][20381] Updated weights for policy 1, policy_version 1588216 (0.0007) [2023-12-27 16:39:37,067][20347] Updated weights for policy 0, policy_version 1589841 (0.0006) [2023-12-27 16:39:37,135][20347] Updated weights for policy 0, policy_version 1589851 (0.0008) [2023-12-27 16:39:37,180][20381] Updated weights for policy 1, policy_version 1588226 (0.0007) [2023-12-27 16:39:37,197][20347] Updated weights for policy 0, policy_version 1589861 (0.0006) [2023-12-27 16:39:37,240][20381] Updated weights for policy 1, policy_version 1588236 (0.0006) [2023-12-27 16:39:37,257][20347] Updated weights for policy 0, policy_version 1589871 (0.0007) [2023-12-27 16:39:37,298][20381] Updated weights for policy 1, policy_version 1588246 (0.0008) [2023-12-27 16:39:37,362][19170] Fps is (10 sec: 21300.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 813719552. Throughput: 0: 10256.0, 1: 10271.6. Samples: 813683168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:37,363][19170] Avg episode reward: [(0, '990.210'), (1, '408.170')] [2023-12-27 16:39:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001588248_406634496.pth... [2023-12-27 16:39:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001589872_407085056.pth... [2023-12-27 16:39:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001587064_406331392.pth [2023-12-27 16:39:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001588656_406773760.pth [2023-12-27 16:39:37,925][20381] Updated weights for policy 1, policy_version 1588256 (0.0007) [2023-12-27 16:39:37,958][20347] Updated weights for policy 0, policy_version 1589881 (0.0007) [2023-12-27 16:39:37,979][20381] Updated weights for policy 1, policy_version 1588266 (0.0006) [2023-12-27 16:39:38,012][20347] Updated weights for policy 0, policy_version 1589891 (0.0007) [2023-12-27 16:39:38,035][20381] Updated weights for policy 1, policy_version 1588276 (0.0007) [2023-12-27 16:39:38,069][20347] Updated weights for policy 0, policy_version 1589901 (0.0007) [2023-12-27 16:39:38,756][20381] Updated weights for policy 1, policy_version 1588286 (0.0009) [2023-12-27 16:39:38,788][20347] Updated weights for policy 0, policy_version 1589911 (0.0007) [2023-12-27 16:39:38,813][20381] Updated weights for policy 1, policy_version 1588296 (0.0007) [2023-12-27 16:39:38,843][20347] Updated weights for policy 0, policy_version 1589921 (0.0007) [2023-12-27 16:39:38,872][20381] Updated weights for policy 1, policy_version 1588306 (0.0008) [2023-12-27 16:39:38,892][20347] Updated weights for policy 0, policy_version 1589931 (0.0007) [2023-12-27 16:39:39,597][20381] Updated weights for policy 1, policy_version 1588316 (0.0007) [2023-12-27 16:39:39,642][20347] Updated weights for policy 0, policy_version 1589941 (0.0007) [2023-12-27 16:39:39,653][20381] Updated weights for policy 1, policy_version 1588326 (0.0008) [2023-12-27 16:39:39,703][20347] Updated weights for policy 0, policy_version 1589951 (0.0007) [2023-12-27 16:39:39,712][20381] Updated weights for policy 1, policy_version 1588336 (0.0007) [2023-12-27 16:39:39,763][20347] Updated weights for policy 0, policy_version 1589961 (0.0007) [2023-12-27 16:39:40,444][20381] Updated weights for policy 1, policy_version 1588346 (0.0006) [2023-12-27 16:39:40,483][20347] Updated weights for policy 0, policy_version 1589971 (0.0007) [2023-12-27 16:39:40,508][20381] Updated weights for policy 1, policy_version 1588356 (0.0007) [2023-12-27 16:39:40,534][20347] Updated weights for policy 0, policy_version 1589981 (0.0007) [2023-12-27 16:39:40,563][20381] Updated weights for policy 1, policy_version 1588366 (0.0007) [2023-12-27 16:39:40,580][20347] Updated weights for policy 0, policy_version 1589991 (0.0006) [2023-12-27 16:39:40,619][20381] Updated weights for policy 1, policy_version 1588376 (0.0005) [2023-12-27 16:39:41,231][20347] Updated weights for policy 0, policy_version 1590001 (0.0006) [2023-12-27 16:39:41,295][20347] Updated weights for policy 0, policy_version 1590011 (0.0008) [2023-12-27 16:39:41,333][20381] Updated weights for policy 1, policy_version 1588386 (0.0008) [2023-12-27 16:39:41,359][20347] Updated weights for policy 0, policy_version 1590021 (0.0008) [2023-12-27 16:39:41,400][20381] Updated weights for policy 1, policy_version 1588396 (0.0007) [2023-12-27 16:39:41,420][20347] Updated weights for policy 0, policy_version 1590031 (0.0008) [2023-12-27 16:39:41,461][20381] Updated weights for policy 1, policy_version 1588406 (0.0006) [2023-12-27 16:39:42,127][20347] Updated weights for policy 0, policy_version 1590041 (0.0008) [2023-12-27 16:39:42,162][20381] Updated weights for policy 1, policy_version 1588416 (0.0007) [2023-12-27 16:39:42,192][20347] Updated weights for policy 0, policy_version 1590051 (0.0007) [2023-12-27 16:39:42,221][20381] Updated weights for policy 1, policy_version 1588426 (0.0007) [2023-12-27 16:39:42,261][20347] Updated weights for policy 0, policy_version 1590061 (0.0007) [2023-12-27 16:39:42,279][20381] Updated weights for policy 1, policy_version 1588436 (0.0008) [2023-12-27 16:39:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 813817856. Throughput: 0: 10301.8, 1: 10150.8. Samples: 813803380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:42,363][19170] Avg episode reward: [(0, '980.470'), (1, '357.100')] [2023-12-27 16:39:42,961][20347] Updated weights for policy 0, policy_version 1590071 (0.0007) [2023-12-27 16:39:42,995][20381] Updated weights for policy 1, policy_version 1588446 (0.0008) [2023-12-27 16:39:43,012][20347] Updated weights for policy 0, policy_version 1590081 (0.0007) [2023-12-27 16:39:43,052][20381] Updated weights for policy 1, policy_version 1588456 (0.0006) [2023-12-27 16:39:43,069][20347] Updated weights for policy 0, policy_version 1590091 (0.0006) [2023-12-27 16:39:43,109][20381] Updated weights for policy 1, policy_version 1588466 (0.0007) [2023-12-27 16:39:43,751][20347] Updated weights for policy 0, policy_version 1590101 (0.0006) [2023-12-27 16:39:43,793][20381] Updated weights for policy 1, policy_version 1588476 (0.0007) [2023-12-27 16:39:43,798][20347] Updated weights for policy 0, policy_version 1590111 (0.0006) [2023-12-27 16:39:43,850][20347] Updated weights for policy 0, policy_version 1590121 (0.0005) [2023-12-27 16:39:43,853][20381] Updated weights for policy 1, policy_version 1588486 (0.0008) [2023-12-27 16:39:43,911][20381] Updated weights for policy 1, policy_version 1588496 (0.0007) [2023-12-27 16:39:44,539][20347] Updated weights for policy 0, policy_version 1590131 (0.0007) [2023-12-27 16:39:44,544][20381] Updated weights for policy 1, policy_version 1588506 (0.0007) [2023-12-27 16:39:44,597][20347] Updated weights for policy 0, policy_version 1590141 (0.0010) [2023-12-27 16:39:44,602][20381] Updated weights for policy 1, policy_version 1588516 (0.0005) [2023-12-27 16:39:44,651][20347] Updated weights for policy 0, policy_version 1590151 (0.0010) [2023-12-27 16:39:44,657][20381] Updated weights for policy 1, policy_version 1588526 (0.0006) [2023-12-27 16:39:44,712][20381] Updated weights for policy 1, policy_version 1588536 (0.0006) [2023-12-27 16:39:45,381][20347] Updated weights for policy 0, policy_version 1590161 (0.0010) [2023-12-27 16:39:45,428][20347] Updated weights for policy 0, policy_version 1590171 (0.0010) [2023-12-27 16:39:45,437][20381] Updated weights for policy 1, policy_version 1588546 (0.0005) [2023-12-27 16:39:45,488][20381] Updated weights for policy 1, policy_version 1588556 (0.0007) [2023-12-27 16:39:45,490][20347] Updated weights for policy 0, policy_version 1590181 (0.0010) [2023-12-27 16:39:45,541][20347] Updated weights for policy 0, policy_version 1590191 (0.0010) [2023-12-27 16:39:45,542][20381] Updated weights for policy 1, policy_version 1588566 (0.0005) [2023-12-27 16:39:46,158][20381] Updated weights for policy 1, policy_version 1588576 (0.0006) [2023-12-27 16:39:46,222][20381] Updated weights for policy 1, policy_version 1588586 (0.0005) [2023-12-27 16:39:46,226][20347] Updated weights for policy 0, policy_version 1590201 (0.0006) [2023-12-27 16:39:46,283][20347] Updated weights for policy 0, policy_version 1590211 (0.0005) [2023-12-27 16:39:46,290][20381] Updated weights for policy 1, policy_version 1588596 (0.0005) [2023-12-27 16:39:46,341][20347] Updated weights for policy 0, policy_version 1590221 (0.0007) [2023-12-27 16:39:46,884][20381] Updated weights for policy 1, policy_version 1588606 (0.0008) [2023-12-27 16:39:46,937][20347] Updated weights for policy 0, policy_version 1590231 (0.0010) [2023-12-27 16:39:46,939][20381] Updated weights for policy 1, policy_version 1588616 (0.0010) [2023-12-27 16:39:46,988][20347] Updated weights for policy 0, policy_version 1590241 (0.0010) [2023-12-27 16:39:46,994][20381] Updated weights for policy 1, policy_version 1588626 (0.0011) [2023-12-27 16:39:47,031][20347] Updated weights for policy 0, policy_version 1590251 (0.0010) [2023-12-27 16:39:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 813916160. Throughput: 0: 10298.1, 1: 10177.5. Samples: 813925320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:47,363][19170] Avg episode reward: [(0, '990.260'), (1, '469.110')] [2023-12-27 16:39:47,646][20381] Updated weights for policy 1, policy_version 1588636 (0.0010) [2023-12-27 16:39:47,671][20347] Updated weights for policy 0, policy_version 1590261 (0.0007) [2023-12-27 16:39:47,701][20381] Updated weights for policy 1, policy_version 1588646 (0.0007) [2023-12-27 16:39:47,720][20347] Updated weights for policy 0, policy_version 1590271 (0.0006) [2023-12-27 16:39:47,759][20381] Updated weights for policy 1, policy_version 1588656 (0.0006) [2023-12-27 16:39:47,781][20347] Updated weights for policy 0, policy_version 1590281 (0.0005) [2023-12-27 16:39:48,396][20381] Updated weights for policy 1, policy_version 1588666 (0.0006) [2023-12-27 16:39:48,418][20347] Updated weights for policy 0, policy_version 1590291 (0.0006) [2023-12-27 16:39:48,459][20381] Updated weights for policy 1, policy_version 1588676 (0.0006) [2023-12-27 16:39:48,481][20347] Updated weights for policy 0, policy_version 1590301 (0.0008) [2023-12-27 16:39:48,515][20381] Updated weights for policy 1, policy_version 1588686 (0.0006) [2023-12-27 16:39:48,537][20347] Updated weights for policy 0, policy_version 1590311 (0.0008) [2023-12-27 16:39:48,571][20381] Updated weights for policy 1, policy_version 1588696 (0.0005) [2023-12-27 16:39:49,119][20381] Updated weights for policy 1, policy_version 1588706 (0.0007) [2023-12-27 16:39:49,173][20381] Updated weights for policy 1, policy_version 1588716 (0.0008) [2023-12-27 16:39:49,235][20381] Updated weights for policy 1, policy_version 1588726 (0.0008) [2023-12-27 16:39:49,325][20347] Updated weights for policy 0, policy_version 1590321 (0.0008) [2023-12-27 16:39:49,394][20347] Updated weights for policy 0, policy_version 1590331 (0.0011) [2023-12-27 16:39:49,458][20347] Updated weights for policy 0, policy_version 1590341 (0.0008) [2023-12-27 16:39:49,522][20347] Updated weights for policy 0, policy_version 1590351 (0.0008) [2023-12-27 16:39:49,925][20381] Updated weights for policy 1, policy_version 1588736 (0.0008) [2023-12-27 16:39:49,988][20381] Updated weights for policy 1, policy_version 1588746 (0.0006) [2023-12-27 16:39:50,048][20381] Updated weights for policy 1, policy_version 1588756 (0.0006) [2023-12-27 16:39:50,225][20347] Updated weights for policy 0, policy_version 1590361 (0.0007) [2023-12-27 16:39:50,295][20347] Updated weights for policy 0, policy_version 1590371 (0.0008) [2023-12-27 16:39:50,360][20347] Updated weights for policy 0, policy_version 1590381 (0.0007) [2023-12-27 16:39:50,678][20381] Updated weights for policy 1, policy_version 1588766 (0.0007) [2023-12-27 16:39:50,738][20381] Updated weights for policy 1, policy_version 1588776 (0.0008) [2023-12-27 16:39:50,796][20381] Updated weights for policy 1, policy_version 1588786 (0.0008) [2023-12-27 16:39:50,973][20347] Updated weights for policy 0, policy_version 1590391 (0.0009) [2023-12-27 16:39:51,041][20347] Updated weights for policy 0, policy_version 1590401 (0.0009) [2023-12-27 16:39:51,104][20347] Updated weights for policy 0, policy_version 1590411 (0.0009) [2023-12-27 16:39:51,549][20381] Updated weights for policy 1, policy_version 1588796 (0.0008) [2023-12-27 16:39:51,611][20381] Updated weights for policy 1, policy_version 1588806 (0.0006) [2023-12-27 16:39:51,674][20381] Updated weights for policy 1, policy_version 1588816 (0.0008) [2023-12-27 16:39:51,826][20347] Updated weights for policy 0, policy_version 1590421 (0.0006) [2023-12-27 16:39:51,892][20347] Updated weights for policy 0, policy_version 1590431 (0.0006) [2023-12-27 16:39:51,945][20347] Updated weights for policy 0, policy_version 1590441 (0.0009) [2023-12-27 16:39:52,255][20381] Updated weights for policy 1, policy_version 1588826 (0.0007) [2023-12-27 16:39:52,316][20381] Updated weights for policy 1, policy_version 1588836 (0.0008) [2023-12-27 16:39:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 814014464. Throughput: 0: 10285.5, 1: 10204.0. Samples: 813987868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '568.410')] [2023-12-27 16:39:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001590448_407232512.pth... [2023-12-27 16:39:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001589232_406921216.pth [2023-12-27 16:39:52,374][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001590448_407232512.pth [2023-12-27 16:39:52,379][20381] Updated weights for policy 1, policy_version 1588846 (0.0008) [2023-12-27 16:39:52,435][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001588856_406790144.pth... [2023-12-27 16:39:52,437][20381] Updated weights for policy 1, policy_version 1588856 (0.0009) [2023-12-27 16:39:52,439][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001587640_406478848.pth [2023-12-27 16:39:52,440][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001588856_406790144.pth [2023-12-27 16:39:52,588][20347] Updated weights for policy 0, policy_version 1590451 (0.0008) [2023-12-27 16:39:52,649][20347] Updated weights for policy 0, policy_version 1590461 (0.0007) [2023-12-27 16:39:52,705][20347] Updated weights for policy 0, policy_version 1590471 (0.0008) [2023-12-27 16:39:53,156][20381] Updated weights for policy 1, policy_version 1588866 (0.0007) [2023-12-27 16:39:53,201][20381] Updated weights for policy 1, policy_version 1588876 (0.0005) [2023-12-27 16:39:53,248][20381] Updated weights for policy 1, policy_version 1588886 (0.0005) [2023-12-27 16:39:53,315][20347] Updated weights for policy 0, policy_version 1590481 (0.0007) [2023-12-27 16:39:53,365][20347] Updated weights for policy 0, policy_version 1590491 (0.0010) [2023-12-27 16:39:53,420][20347] Updated weights for policy 0, policy_version 1590501 (0.0010) [2023-12-27 16:39:53,478][20347] Updated weights for policy 0, policy_version 1590511 (0.0010) [2023-12-27 16:39:53,873][20381] Updated weights for policy 1, policy_version 1588896 (0.0006) [2023-12-27 16:39:53,934][20381] Updated weights for policy 1, policy_version 1588906 (0.0007) [2023-12-27 16:39:53,988][20381] Updated weights for policy 1, policy_version 1588916 (0.0008) [2023-12-27 16:39:54,113][20347] Updated weights for policy 0, policy_version 1590521 (0.0006) [2023-12-27 16:39:54,171][20347] Updated weights for policy 0, policy_version 1590531 (0.0007) [2023-12-27 16:39:54,237][20347] Updated weights for policy 0, policy_version 1590541 (0.0006) [2023-12-27 16:39:54,658][20381] Updated weights for policy 1, policy_version 1588926 (0.0006) [2023-12-27 16:39:54,716][20381] Updated weights for policy 1, policy_version 1588936 (0.0005) [2023-12-27 16:39:54,767][20381] Updated weights for policy 1, policy_version 1588946 (0.0005) [2023-12-27 16:39:54,884][20347] Updated weights for policy 0, policy_version 1590551 (0.0010) [2023-12-27 16:39:54,948][20347] Updated weights for policy 0, policy_version 1590561 (0.0010) [2023-12-27 16:39:55,011][20347] Updated weights for policy 0, policy_version 1590571 (0.0007) [2023-12-27 16:39:55,379][20381] Updated weights for policy 1, policy_version 1588956 (0.0005) [2023-12-27 16:39:55,436][20381] Updated weights for policy 1, policy_version 1588966 (0.0005) [2023-12-27 16:39:55,496][20381] Updated weights for policy 1, policy_version 1588976 (0.0005) [2023-12-27 16:39:55,640][20347] Updated weights for policy 0, policy_version 1590581 (0.0010) [2023-12-27 16:39:55,699][20347] Updated weights for policy 0, policy_version 1590591 (0.0008) [2023-12-27 16:39:55,762][20347] Updated weights for policy 0, policy_version 1590601 (0.0006) [2023-12-27 16:39:56,083][20381] Updated weights for policy 1, policy_version 1588986 (0.0008) [2023-12-27 16:39:56,144][20381] Updated weights for policy 1, policy_version 1588996 (0.0008) [2023-12-27 16:39:56,192][20381] Updated weights for policy 1, policy_version 1589006 (0.0005) [2023-12-27 16:39:56,239][20381] Updated weights for policy 1, policy_version 1589016 (0.0007) [2023-12-27 16:39:56,392][20347] Updated weights for policy 0, policy_version 1590611 (0.0006) [2023-12-27 16:39:56,443][20347] Updated weights for policy 0, policy_version 1590621 (0.0006) [2023-12-27 16:39:56,494][20347] Updated weights for policy 0, policy_version 1590631 (0.0007) [2023-12-27 16:39:56,840][20381] Updated weights for policy 1, policy_version 1589026 (0.0007) [2023-12-27 16:39:56,887][20381] Updated weights for policy 1, policy_version 1589036 (0.0007) [2023-12-27 16:39:56,928][20381] Updated weights for policy 1, policy_version 1589046 (0.0005) [2023-12-27 16:39:57,181][20347] Updated weights for policy 0, policy_version 1590641 (0.0008) [2023-12-27 16:39:57,232][20347] Updated weights for policy 0, policy_version 1590651 (0.0005) [2023-12-27 16:39:57,285][20347] Updated weights for policy 0, policy_version 1590661 (0.0005) [2023-12-27 16:39:57,342][20347] Updated weights for policy 0, policy_version 1590671 (0.0005) [2023-12-27 16:39:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 814129152. Throughput: 0: 10300.1, 1: 10300.3. Samples: 814114040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:39:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '637.770')] [2023-12-27 16:39:57,559][20381] Updated weights for policy 1, policy_version 1589056 (0.0007) [2023-12-27 16:39:57,606][20381] Updated weights for policy 1, policy_version 1589066 (0.0007) [2023-12-27 16:39:57,647][20381] Updated weights for policy 1, policy_version 1589076 (0.0008) [2023-12-27 16:39:57,929][20347] Updated weights for policy 0, policy_version 1590681 (0.0009) [2023-12-27 16:39:57,984][20347] Updated weights for policy 0, policy_version 1590691 (0.0005) [2023-12-27 16:39:58,036][20347] Updated weights for policy 0, policy_version 1590701 (0.0006) [2023-12-27 16:39:58,395][20381] Updated weights for policy 1, policy_version 1589086 (0.0008) [2023-12-27 16:39:58,454][20381] Updated weights for policy 1, policy_version 1589096 (0.0009) [2023-12-27 16:39:58,521][20381] Updated weights for policy 1, policy_version 1589106 (0.0008) [2023-12-27 16:39:58,749][20347] Updated weights for policy 0, policy_version 1590711 (0.0008) [2023-12-27 16:39:58,814][20347] Updated weights for policy 0, policy_version 1590721 (0.0008) [2023-12-27 16:39:58,883][20347] Updated weights for policy 0, policy_version 1590731 (0.0008) [2023-12-27 16:39:59,318][20381] Updated weights for policy 1, policy_version 1589116 (0.0007) [2023-12-27 16:39:59,386][20381] Updated weights for policy 1, policy_version 1589126 (0.0010) [2023-12-27 16:39:59,443][20381] Updated weights for policy 1, policy_version 1589136 (0.0007) [2023-12-27 16:39:59,680][20347] Updated weights for policy 0, policy_version 1590741 (0.0008) [2023-12-27 16:39:59,733][20347] Updated weights for policy 0, policy_version 1590751 (0.0008) [2023-12-27 16:39:59,784][20347] Updated weights for policy 0, policy_version 1590761 (0.0007) [2023-12-27 16:40:00,136][20381] Updated weights for policy 1, policy_version 1589146 (0.0007) [2023-12-27 16:40:00,191][20381] Updated weights for policy 1, policy_version 1589156 (0.0007) [2023-12-27 16:40:00,251][20381] Updated weights for policy 1, policy_version 1589166 (0.0007) [2023-12-27 16:40:00,311][20381] Updated weights for policy 1, policy_version 1589176 (0.0008) [2023-12-27 16:40:00,480][20347] Updated weights for policy 0, policy_version 1590771 (0.0007) [2023-12-27 16:40:00,545][20347] Updated weights for policy 0, policy_version 1590781 (0.0008) [2023-12-27 16:40:00,595][20347] Updated weights for policy 0, policy_version 1590791 (0.0007) [2023-12-27 16:40:01,000][20381] Updated weights for policy 1, policy_version 1589186 (0.0008) [2023-12-27 16:40:01,051][20381] Updated weights for policy 1, policy_version 1589196 (0.0008) [2023-12-27 16:40:01,116][20381] Updated weights for policy 1, policy_version 1589206 (0.0007) [2023-12-27 16:40:01,325][20347] Updated weights for policy 0, policy_version 1590801 (0.0007) [2023-12-27 16:40:01,385][20347] Updated weights for policy 0, policy_version 1590811 (0.0009) [2023-12-27 16:40:01,444][20347] Updated weights for policy 0, policy_version 1590821 (0.0008) [2023-12-27 16:40:01,506][20347] Updated weights for policy 0, policy_version 1590831 (0.0007) [2023-12-27 16:40:01,828][20381] Updated weights for policy 1, policy_version 1589216 (0.0006) [2023-12-27 16:40:01,886][20381] Updated weights for policy 1, policy_version 1589226 (0.0005) [2023-12-27 16:40:01,947][20381] Updated weights for policy 1, policy_version 1589236 (0.0005) [2023-12-27 16:40:02,245][20347] Updated weights for policy 0, policy_version 1590841 (0.0009) [2023-12-27 16:40:02,306][20347] Updated weights for policy 0, policy_version 1590851 (0.0008) [2023-12-27 16:40:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 814219264. Throughput: 0: 10255.4, 1: 10256.1. Samples: 814234176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:40:02,363][19170] Avg episode reward: [(0, '990.310'), (1, '721.160')] [2023-12-27 16:40:02,368][20347] Updated weights for policy 0, policy_version 1590861 (0.0009) [2023-12-27 16:40:02,538][20381] Updated weights for policy 1, policy_version 1589246 (0.0008) [2023-12-27 16:40:02,596][20381] Updated weights for policy 1, policy_version 1589256 (0.0008) [2023-12-27 16:40:02,656][20381] Updated weights for policy 1, policy_version 1589266 (0.0008) [2023-12-27 16:40:03,036][20347] Updated weights for policy 0, policy_version 1590871 (0.0008) [2023-12-27 16:40:03,080][20347] Updated weights for policy 0, policy_version 1590881 (0.0007) [2023-12-27 16:40:03,135][20347] Updated weights for policy 0, policy_version 1590891 (0.0008) [2023-12-27 16:40:03,337][20381] Updated weights for policy 1, policy_version 1589276 (0.0007) [2023-12-27 16:40:03,400][20381] Updated weights for policy 1, policy_version 1589286 (0.0005) [2023-12-27 16:40:03,466][20381] Updated weights for policy 1, policy_version 1589296 (0.0007) [2023-12-27 16:40:03,807][20347] Updated weights for policy 0, policy_version 1590901 (0.0006) [2023-12-27 16:40:03,869][20347] Updated weights for policy 0, policy_version 1590911 (0.0007) [2023-12-27 16:40:03,922][20347] Updated weights for policy 0, policy_version 1590921 (0.0007) [2023-12-27 16:40:04,109][20381] Updated weights for policy 1, policy_version 1589306 (0.0008) [2023-12-27 16:40:04,173][20381] Updated weights for policy 1, policy_version 1589316 (0.0008) [2023-12-27 16:40:04,233][20381] Updated weights for policy 1, policy_version 1589326 (0.0008) [2023-12-27 16:40:04,295][20381] Updated weights for policy 1, policy_version 1589336 (0.0008) [2023-12-27 16:40:04,510][20347] Updated weights for policy 0, policy_version 1590931 (0.0006) [2023-12-27 16:40:04,566][20347] Updated weights for policy 0, policy_version 1590941 (0.0006) [2023-12-27 16:40:04,624][20347] Updated weights for policy 0, policy_version 1590951 (0.0006) [2023-12-27 16:40:04,962][20381] Updated weights for policy 1, policy_version 1589346 (0.0006) [2023-12-27 16:40:05,007][20381] Updated weights for policy 1, policy_version 1589356 (0.0005) [2023-12-27 16:40:05,064][20381] Updated weights for policy 1, policy_version 1589366 (0.0005) [2023-12-27 16:40:05,219][20347] Updated weights for policy 0, policy_version 1590961 (0.0006) [2023-12-27 16:40:05,285][20347] Updated weights for policy 0, policy_version 1590971 (0.0006) [2023-12-27 16:40:05,348][20347] Updated weights for policy 0, policy_version 1590981 (0.0007) [2023-12-27 16:40:05,410][20347] Updated weights for policy 0, policy_version 1590991 (0.0006) [2023-12-27 16:40:05,665][20381] Updated weights for policy 1, policy_version 1589376 (0.0005) [2023-12-27 16:40:05,721][20381] Updated weights for policy 1, policy_version 1589386 (0.0005) [2023-12-27 16:40:05,768][20381] Updated weights for policy 1, policy_version 1589396 (0.0007) [2023-12-27 16:40:06,056][20347] Updated weights for policy 0, policy_version 1591001 (0.0008) [2023-12-27 16:40:06,118][20347] Updated weights for policy 0, policy_version 1591011 (0.0008) [2023-12-27 16:40:06,184][20347] Updated weights for policy 0, policy_version 1591021 (0.0008) [2023-12-27 16:40:06,422][20381] Updated weights for policy 1, policy_version 1589406 (0.0006) [2023-12-27 16:40:06,479][20381] Updated weights for policy 1, policy_version 1589416 (0.0007) [2023-12-27 16:40:06,541][20381] Updated weights for policy 1, policy_version 1589426 (0.0007) [2023-12-27 16:40:06,910][20347] Updated weights for policy 0, policy_version 1591031 (0.0008) [2023-12-27 16:40:06,971][20347] Updated weights for policy 0, policy_version 1591041 (0.0007) [2023-12-27 16:40:07,022][20347] Updated weights for policy 0, policy_version 1591051 (0.0007) [2023-12-27 16:40:07,233][20381] Updated weights for policy 1, policy_version 1589436 (0.0008) [2023-12-27 16:40:07,326][20381] Updated weights for policy 1, policy_version 1589446 (0.0008) [2023-12-27 16:40:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 814325760. Throughput: 0: 10298.0, 1: 10268.5. Samples: 814298000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:40:07,363][19170] Avg episode reward: [(0, '980.520'), (1, '659.200')] [2023-12-27 16:40:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001591056_407388160.pth... [2023-12-27 16:40:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001589872_407085056.pth [2023-12-27 16:40:07,388][20381] Updated weights for policy 1, policy_version 1589456 (0.0010) [2023-12-27 16:40:07,436][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001589464_406945792.pth... [2023-12-27 16:40:07,441][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001588248_406634496.pth [2023-12-27 16:40:07,719][20347] Updated weights for policy 0, policy_version 1591061 (0.0008) [2023-12-27 16:40:07,767][20347] Updated weights for policy 0, policy_version 1591071 (0.0007) [2023-12-27 16:40:07,786][20291] KL-divergence is very high: 121.5307 [2023-12-27 16:40:07,811][20291] KL-divergence is very high: 484.6832 [2023-12-27 16:40:07,816][20291] KL-divergence is very high: 492.5270 [2023-12-27 16:40:07,828][20347] Updated weights for policy 0, policy_version 1591081 (0.0007) [2023-12-27 16:40:07,835][20291] KL-divergence is very high: 278.0659 [2023-12-27 16:40:07,861][20291] KL-divergence is very high: 206.7790 [2023-12-27 16:40:07,868][20291] KL-divergence is very high: 170.1776 [2023-12-27 16:40:07,992][20381] Updated weights for policy 1, policy_version 1589466 (0.0006) [2023-12-27 16:40:08,055][20381] Updated weights for policy 1, policy_version 1589476 (0.0005) [2023-12-27 16:40:08,114][20381] Updated weights for policy 1, policy_version 1589486 (0.0006) [2023-12-27 16:40:08,164][20381] Updated weights for policy 1, policy_version 1589496 (0.0005) [2023-12-27 16:40:08,580][20347] Updated weights for policy 0, policy_version 1591091 (0.0008) [2023-12-27 16:40:08,631][20347] Updated weights for policy 0, policy_version 1591101 (0.0008) [2023-12-27 16:40:08,684][20347] Updated weights for policy 0, policy_version 1591111 (0.0009) [2023-12-27 16:40:08,758][20381] Updated weights for policy 1, policy_version 1589506 (0.0008) [2023-12-27 16:40:08,814][20381] Updated weights for policy 1, policy_version 1589516 (0.0005) [2023-12-27 16:40:08,876][20381] Updated weights for policy 1, policy_version 1589526 (0.0010) [2023-12-27 16:40:09,386][20347] Updated weights for policy 0, policy_version 1591121 (0.0008) [2023-12-27 16:40:09,439][20291] KL-divergence is very high: 108.8257 [2023-12-27 16:40:09,452][20347] Updated weights for policy 0, policy_version 1591131 (0.0008) [2023-12-27 16:40:09,488][20291] KL-divergence is very high: 120.6692 [2023-12-27 16:40:09,511][20347] Updated weights for policy 0, policy_version 1591141 (0.0008) [2023-12-27 16:40:09,571][20347] Updated weights for policy 0, policy_version 1591151 (0.0008) [2023-12-27 16:40:09,593][20381] Updated weights for policy 1, policy_version 1589536 (0.0008) [2023-12-27 16:40:09,657][20381] Updated weights for policy 1, policy_version 1589546 (0.0007) [2023-12-27 16:40:09,718][20381] Updated weights for policy 1, policy_version 1589556 (0.0008) [2023-12-27 16:40:10,390][20347] Updated weights for policy 0, policy_version 1591161 (0.0008) [2023-12-27 16:40:10,444][20381] Updated weights for policy 1, policy_version 1589566 (0.0007) [2023-12-27 16:40:10,450][20347] Updated weights for policy 0, policy_version 1591171 (0.0010) [2023-12-27 16:40:10,500][20381] Updated weights for policy 1, policy_version 1589576 (0.0006) [2023-12-27 16:40:10,502][20347] Updated weights for policy 0, policy_version 1591181 (0.0010) [2023-12-27 16:40:10,563][20381] Updated weights for policy 1, policy_version 1589586 (0.0007) [2023-12-27 16:40:11,126][20347] Updated weights for policy 0, policy_version 1591191 (0.0009) [2023-12-27 16:40:11,197][20347] Updated weights for policy 0, policy_version 1591201 (0.0008) [2023-12-27 16:40:11,262][20347] Updated weights for policy 0, policy_version 1591211 (0.0008) [2023-12-27 16:40:11,284][20381] Updated weights for policy 1, policy_version 1589596 (0.0007) [2023-12-27 16:40:11,350][20381] Updated weights for policy 1, policy_version 1589606 (0.0007) [2023-12-27 16:40:11,419][20381] Updated weights for policy 1, policy_version 1589616 (0.0008) [2023-12-27 16:40:12,066][20347] Updated weights for policy 0, policy_version 1591221 (0.0006) [2023-12-27 16:40:12,120][20347] Updated weights for policy 0, policy_version 1591231 (0.0006) [2023-12-27 16:40:12,128][20381] Updated weights for policy 1, policy_version 1589626 (0.0009) [2023-12-27 16:40:12,169][20347] Updated weights for policy 0, policy_version 1591241 (0.0008) [2023-12-27 16:40:12,191][20381] Updated weights for policy 1, policy_version 1589636 (0.0009) [2023-12-27 16:40:12,246][20381] Updated weights for policy 1, policy_version 1589646 (0.0008) [2023-12-27 16:40:12,312][20381] Updated weights for policy 1, policy_version 1589656 (0.0008) [2023-12-27 16:40:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 814432256. Throughput: 0: 10177.3, 1: 10267.1. Samples: 814418164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:40:12,363][19170] Avg episode reward: [(0, '960.870'), (1, '657.900')] [2023-12-27 16:40:12,848][20347] Updated weights for policy 0, policy_version 1591251 (0.0007) [2023-12-27 16:40:12,909][20347] Updated weights for policy 0, policy_version 1591261 (0.0006) [2023-12-27 16:40:12,950][20381] Updated weights for policy 1, policy_version 1589666 (0.0007) [2023-12-27 16:40:12,972][20347] Updated weights for policy 0, policy_version 1591271 (0.0008) [2023-12-27 16:40:13,003][20381] Updated weights for policy 1, policy_version 1589676 (0.0005) [2023-12-27 16:40:13,061][20381] Updated weights for policy 1, policy_version 1589686 (0.0007) [2023-12-27 16:40:13,619][20347] Updated weights for policy 0, policy_version 1591281 (0.0010) [2023-12-27 16:40:13,663][20381] Updated weights for policy 1, policy_version 1589696 (0.0006) [2023-12-27 16:40:13,676][20347] Updated weights for policy 0, policy_version 1591291 (0.0011) [2023-12-27 16:40:13,719][20381] Updated weights for policy 1, policy_version 1589706 (0.0005) [2023-12-27 16:40:13,732][20347] Updated weights for policy 0, policy_version 1591301 (0.0011) [2023-12-27 16:40:13,779][20381] Updated weights for policy 1, policy_version 1589716 (0.0006) [2023-12-27 16:40:13,788][20347] Updated weights for policy 0, policy_version 1591311 (0.0011) [2023-12-27 16:40:14,483][20347] Updated weights for policy 0, policy_version 1591321 (0.0008) [2023-12-27 16:40:14,487][20381] Updated weights for policy 1, policy_version 1589726 (0.0007) [2023-12-27 16:40:14,542][20347] Updated weights for policy 0, policy_version 1591331 (0.0007) [2023-12-27 16:40:14,548][20381] Updated weights for policy 1, policy_version 1589736 (0.0007) [2023-12-27 16:40:14,600][20347] Updated weights for policy 0, policy_version 1591341 (0.0006) [2023-12-27 16:40:14,608][20381] Updated weights for policy 1, policy_version 1589746 (0.0007) [2023-12-27 16:40:15,272][20347] Updated weights for policy 0, policy_version 1591351 (0.0009) [2023-12-27 16:40:15,299][20381] Updated weights for policy 1, policy_version 1589756 (0.0007) [2023-12-27 16:40:15,333][20347] Updated weights for policy 0, policy_version 1591361 (0.0007) [2023-12-27 16:40:15,352][20381] Updated weights for policy 1, policy_version 1589766 (0.0005) [2023-12-27 16:40:15,387][20347] Updated weights for policy 0, policy_version 1591371 (0.0007) [2023-12-27 16:40:15,409][20381] Updated weights for policy 1, policy_version 1589776 (0.0005) [2023-12-27 16:40:15,926][20381] Updated weights for policy 1, policy_version 1589786 (0.0005) [2023-12-27 16:40:15,978][20381] Updated weights for policy 1, policy_version 1589796 (0.0005) [2023-12-27 16:40:16,031][20381] Updated weights for policy 1, policy_version 1589806 (0.0006) [2023-12-27 16:40:16,083][20381] Updated weights for policy 1, policy_version 1589816 (0.0005) [2023-12-27 16:40:16,145][20347] Updated weights for policy 0, policy_version 1591381 (0.0008) [2023-12-27 16:40:16,212][20347] Updated weights for policy 0, policy_version 1591391 (0.0008) [2023-12-27 16:40:16,267][20347] Updated weights for policy 0, policy_version 1591401 (0.0008) [2023-12-27 16:40:16,715][20381] Updated weights for policy 1, policy_version 1589826 (0.0009) [2023-12-27 16:40:16,762][20381] Updated weights for policy 1, policy_version 1589836 (0.0010) [2023-12-27 16:40:16,816][20381] Updated weights for policy 1, policy_version 1589846 (0.0008) [2023-12-27 16:40:16,871][20347] Updated weights for policy 0, policy_version 1591411 (0.0007) [2023-12-27 16:40:16,926][20347] Updated weights for policy 0, policy_version 1591421 (0.0006) [2023-12-27 16:40:16,987][20347] Updated weights for policy 0, policy_version 1591431 (0.0006) [2023-12-27 16:40:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 814530560. Throughput: 0: 10161.1, 1: 10336.5. Samples: 814541492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 16:40:17,363][19170] Avg episode reward: [(0, '960.860'), (1, '629.820')] [2023-12-27 16:40:17,502][20381] Updated weights for policy 1, policy_version 1589856 (0.0010) [2023-12-27 16:40:17,554][20381] Updated weights for policy 1, policy_version 1589866 (0.0011) [2023-12-27 16:40:17,556][20347] Updated weights for policy 0, policy_version 1591441 (0.0008) [2023-12-27 16:40:17,609][20381] Updated weights for policy 1, policy_version 1589876 (0.0011) [2023-12-27 16:40:17,612][20347] Updated weights for policy 0, policy_version 1591451 (0.0005) [2023-12-27 16:40:17,668][20347] Updated weights for policy 0, policy_version 1591461 (0.0005) [2023-12-27 16:40:17,726][20347] Updated weights for policy 0, policy_version 1591471 (0.0006) [2023-12-27 16:40:18,304][20381] Updated weights for policy 1, policy_version 1589886 (0.0011) [2023-12-27 16:40:18,366][20347] Updated weights for policy 0, policy_version 1591481 (0.0008) [2023-12-27 16:40:18,371][20381] Updated weights for policy 1, policy_version 1589896 (0.0011) [2023-12-27 16:40:18,428][20381] Updated weights for policy 1, policy_version 1589906 (0.0011) [2023-12-27 16:40:18,428][20347] Updated weights for policy 0, policy_version 1591491 (0.0008) [2023-12-27 16:40:18,492][20347] Updated weights for policy 0, policy_version 1591501 (0.0007) [2023-12-27 16:40:19,141][20381] Updated weights for policy 1, policy_version 1589916 (0.0008) [2023-12-27 16:40:19,168][20347] Updated weights for policy 0, policy_version 1591511 (0.0009) [2023-12-27 16:40:19,187][20381] Updated weights for policy 1, policy_version 1589926 (0.0005) [2023-12-27 16:40:19,223][20347] Updated weights for policy 0, policy_version 1591521 (0.0010) [2023-12-27 16:40:19,249][20381] Updated weights for policy 1, policy_version 1589936 (0.0009) [2023-12-27 16:40:19,292][20347] Updated weights for policy 0, policy_version 1591531 (0.0010) [2023-12-27 16:40:19,864][20381] Updated weights for policy 1, policy_version 1589946 (0.0010) [2023-12-27 16:40:19,929][20381] Updated weights for policy 1, policy_version 1589956 (0.0007) [2023-12-27 16:40:19,990][20381] Updated weights for policy 1, policy_version 1589966 (0.0008) [2023-12-27 16:40:20,004][20347] Updated weights for policy 0, policy_version 1591541 (0.0010) [2023-12-27 16:40:20,044][20381] Updated weights for policy 1, policy_version 1589976 (0.0006) [2023-12-27 16:40:20,064][20347] Updated weights for policy 0, policy_version 1591551 (0.0010) [2023-12-27 16:40:20,122][20347] Updated weights for policy 0, policy_version 1591561 (0.0011) [2023-12-27 16:40:20,766][20347] Updated weights for policy 0, policy_version 1591571 (0.0009) [2023-12-27 16:40:20,805][20381] Updated weights for policy 1, policy_version 1589986 (0.0008) [2023-12-27 16:40:20,825][20347] Updated weights for policy 0, policy_version 1591581 (0.0008) [2023-12-27 16:40:20,863][20381] Updated weights for policy 1, policy_version 1589996 (0.0009) [2023-12-27 16:40:20,877][20347] Updated weights for policy 0, policy_version 1591591 (0.0006) [2023-12-27 16:40:20,917][20381] Updated weights for policy 1, policy_version 1590006 (0.0008) [2023-12-27 16:40:21,554][20347] Updated weights for policy 0, policy_version 1591601 (0.0008) [2023-12-27 16:40:21,615][20347] Updated weights for policy 0, policy_version 1591611 (0.0010) [2023-12-27 16:40:21,647][20381] Updated weights for policy 1, policy_version 1590016 (0.0008) [2023-12-27 16:40:21,679][20347] Updated weights for policy 0, policy_version 1591621 (0.0009) [2023-12-27 16:40:21,703][20381] Updated weights for policy 1, policy_version 1590026 (0.0006) [2023-12-27 16:40:21,739][20347] Updated weights for policy 0, policy_version 1591631 (0.0007) [2023-12-27 16:40:21,765][20381] Updated weights for policy 1, policy_version 1590036 (0.0007) [2023-12-27 16:40:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 814628864. Throughput: 0: 10155.0, 1: 10297.3. Samples: 814603524. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:22,363][19170] Avg episode reward: [(0, '980.460'), (1, '548.600')] [2023-12-27 16:40:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001590040_407093248.pth... [2023-12-27 16:40:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001591632_407535616.pth... [2023-12-27 16:40:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001588856_406790144.pth [2023-12-27 16:40:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001590448_407232512.pth [2023-12-27 16:40:22,426][20381] Updated weights for policy 1, policy_version 1590046 (0.0009) [2023-12-27 16:40:22,476][20381] Updated weights for policy 1, policy_version 1590056 (0.0011) [2023-12-27 16:40:22,484][20347] Updated weights for policy 0, policy_version 1591641 (0.0011) [2023-12-27 16:40:22,530][20381] Updated weights for policy 1, policy_version 1590066 (0.0010) [2023-12-27 16:40:22,543][20347] Updated weights for policy 0, policy_version 1591651 (0.0010) [2023-12-27 16:40:22,602][20347] Updated weights for policy 0, policy_version 1591661 (0.0010) [2023-12-27 16:40:23,244][20381] Updated weights for policy 1, policy_version 1590076 (0.0011) [2023-12-27 16:40:23,249][20347] Updated weights for policy 0, policy_version 1591671 (0.0007) [2023-12-27 16:40:23,298][20347] Updated weights for policy 0, policy_version 1591681 (0.0005) [2023-12-27 16:40:23,300][20381] Updated weights for policy 1, policy_version 1590086 (0.0010) [2023-12-27 16:40:23,349][20347] Updated weights for policy 0, policy_version 1591691 (0.0005) [2023-12-27 16:40:23,358][20381] Updated weights for policy 1, policy_version 1590096 (0.0010) [2023-12-27 16:40:23,925][20347] Updated weights for policy 0, policy_version 1591701 (0.0005) [2023-12-27 16:40:23,976][20347] Updated weights for policy 0, policy_version 1591711 (0.0005) [2023-12-27 16:40:24,017][20381] Updated weights for policy 1, policy_version 1590106 (0.0009) [2023-12-27 16:40:24,025][20347] Updated weights for policy 0, policy_version 1591721 (0.0006) [2023-12-27 16:40:24,067][20381] Updated weights for policy 1, policy_version 1590116 (0.0005) [2023-12-27 16:40:24,121][20381] Updated weights for policy 1, policy_version 1590126 (0.0010) [2023-12-27 16:40:24,166][20381] Updated weights for policy 1, policy_version 1590136 (0.0010) [2023-12-27 16:40:24,689][20347] Updated weights for policy 0, policy_version 1591731 (0.0010) [2023-12-27 16:40:24,744][20347] Updated weights for policy 0, policy_version 1591741 (0.0010) [2023-12-27 16:40:24,758][20381] Updated weights for policy 1, policy_version 1590146 (0.0005) [2023-12-27 16:40:24,795][20347] Updated weights for policy 0, policy_version 1591751 (0.0010) [2023-12-27 16:40:24,811][20381] Updated weights for policy 1, policy_version 1590156 (0.0005) [2023-12-27 16:40:24,872][20381] Updated weights for policy 1, policy_version 1590166 (0.0005) [2023-12-27 16:40:25,408][20381] Updated weights for policy 1, policy_version 1590176 (0.0005) [2023-12-27 16:40:25,464][20381] Updated weights for policy 1, policy_version 1590186 (0.0006) [2023-12-27 16:40:25,474][20347] Updated weights for policy 0, policy_version 1591761 (0.0010) [2023-12-27 16:40:25,516][20381] Updated weights for policy 1, policy_version 1590196 (0.0010) [2023-12-27 16:40:25,530][20347] Updated weights for policy 0, policy_version 1591771 (0.0010) [2023-12-27 16:40:25,592][20347] Updated weights for policy 0, policy_version 1591781 (0.0010) [2023-12-27 16:40:25,646][20347] Updated weights for policy 0, policy_version 1591791 (0.0010) [2023-12-27 16:40:26,132][20381] Updated weights for policy 1, policy_version 1590206 (0.0007) [2023-12-27 16:40:26,180][20381] Updated weights for policy 1, policy_version 1590216 (0.0006) [2023-12-27 16:40:26,234][20381] Updated weights for policy 1, policy_version 1590226 (0.0005) [2023-12-27 16:40:26,325][20347] Updated weights for policy 0, policy_version 1591801 (0.0006) [2023-12-27 16:40:26,377][20347] Updated weights for policy 0, policy_version 1591811 (0.0005) [2023-12-27 16:40:26,431][20347] Updated weights for policy 0, policy_version 1591821 (0.0006) [2023-12-27 16:40:26,847][20381] Updated weights for policy 1, policy_version 1590236 (0.0006) [2023-12-27 16:40:26,890][20381] Updated weights for policy 1, policy_version 1590246 (0.0005) [2023-12-27 16:40:26,936][20381] Updated weights for policy 1, policy_version 1590256 (0.0005) [2023-12-27 16:40:27,069][20347] Updated weights for policy 0, policy_version 1591831 (0.0010) [2023-12-27 16:40:27,124][20347] Updated weights for policy 0, policy_version 1591841 (0.0010) [2023-12-27 16:40:27,181][20347] Updated weights for policy 0, policy_version 1591851 (0.0010) [2023-12-27 16:40:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 814743552. Throughput: 0: 10185.8, 1: 10384.7. Samples: 814729052. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:27,363][19170] Avg episode reward: [(0, '980.500'), (1, '443.630')] [2023-12-27 16:40:27,545][20381] Updated weights for policy 1, policy_version 1590266 (0.0006) [2023-12-27 16:40:27,603][20381] Updated weights for policy 1, policy_version 1590276 (0.0007) [2023-12-27 16:40:27,646][20381] Updated weights for policy 1, policy_version 1590286 (0.0007) [2023-12-27 16:40:27,693][20381] Updated weights for policy 1, policy_version 1590296 (0.0007) [2023-12-27 16:40:27,859][20347] Updated weights for policy 0, policy_version 1591861 (0.0010) [2023-12-27 16:40:27,913][20347] Updated weights for policy 0, policy_version 1591871 (0.0010) [2023-12-27 16:40:27,957][20347] Updated weights for policy 0, policy_version 1591881 (0.0010) [2023-12-27 16:40:28,391][20381] Updated weights for policy 1, policy_version 1590306 (0.0008) [2023-12-27 16:40:28,446][20381] Updated weights for policy 1, policy_version 1590316 (0.0007) [2023-12-27 16:40:28,503][20381] Updated weights for policy 1, policy_version 1590326 (0.0005) [2023-12-27 16:40:28,651][20347] Updated weights for policy 0, policy_version 1591891 (0.0010) [2023-12-27 16:40:28,700][20347] Updated weights for policy 0, policy_version 1591901 (0.0010) [2023-12-27 16:40:28,749][20347] Updated weights for policy 0, policy_version 1591911 (0.0010) [2023-12-27 16:40:29,099][20381] Updated weights for policy 1, policy_version 1590336 (0.0009) [2023-12-27 16:40:29,116][20292] KL-divergence is very high: 126.6789 [2023-12-27 16:40:29,122][20292] KL-divergence is very high: 152.6458 [2023-12-27 16:40:29,133][20292] KL-divergence is very high: 185.9697 [2023-12-27 16:40:29,152][20292] KL-divergence is very high: 218.6989 [2023-12-27 16:40:29,158][20381] Updated weights for policy 1, policy_version 1590346 (0.0010) [2023-12-27 16:40:29,164][20292] KL-divergence is very high: 232.3438 [2023-12-27 16:40:29,170][20292] KL-divergence is very high: 232.5822 [2023-12-27 16:40:29,181][20292] KL-divergence is very high: 215.8693 [2023-12-27 16:40:29,197][20292] KL-divergence is very high: 174.6527 [2023-12-27 16:40:29,214][20292] KL-divergence is very high: 141.6007 [2023-12-27 16:40:29,221][20292] KL-divergence is very high: 123.4698 [2023-12-27 16:40:29,221][20381] Updated weights for policy 1, policy_version 1590356 (0.0007) [2023-12-27 16:40:29,463][20347] Updated weights for policy 0, policy_version 1591921 (0.0010) [2023-12-27 16:40:29,509][20347] Updated weights for policy 0, policy_version 1591931 (0.0010) [2023-12-27 16:40:29,557][20347] Updated weights for policy 0, policy_version 1591941 (0.0010) [2023-12-27 16:40:29,606][20347] Updated weights for policy 0, policy_version 1591951 (0.0010) [2023-12-27 16:40:29,913][20381] Updated weights for policy 1, policy_version 1590366 (0.0009) [2023-12-27 16:40:29,938][20292] KL-divergence is very high: 125.3388 [2023-12-27 16:40:29,968][20381] Updated weights for policy 1, policy_version 1590376 (0.0007) [2023-12-27 16:40:29,977][20292] KL-divergence is very high: 118.4211 [2023-12-27 16:40:30,020][20381] Updated weights for policy 1, policy_version 1590386 (0.0007) [2023-12-27 16:40:30,021][20292] KL-divergence is very high: 134.5959 [2023-12-27 16:40:30,341][20347] Updated weights for policy 0, policy_version 1591961 (0.0006) [2023-12-27 16:40:30,398][20347] Updated weights for policy 0, policy_version 1591971 (0.0007) [2023-12-27 16:40:30,480][20347] Updated weights for policy 0, policy_version 1591981 (0.0006) [2023-12-27 16:40:30,647][20381] Updated weights for policy 1, policy_version 1590396 (0.0007) [2023-12-27 16:40:30,706][20381] Updated weights for policy 1, policy_version 1590406 (0.0005) [2023-12-27 16:40:30,769][20381] Updated weights for policy 1, policy_version 1590416 (0.0005) [2023-12-27 16:40:31,154][20347] Updated weights for policy 0, policy_version 1591991 (0.0007) [2023-12-27 16:40:31,217][20347] Updated weights for policy 0, policy_version 1592001 (0.0007) [2023-12-27 16:40:31,275][20347] Updated weights for policy 0, policy_version 1592011 (0.0008) [2023-12-27 16:40:31,408][20381] Updated weights for policy 1, policy_version 1590426 (0.0005) [2023-12-27 16:40:31,461][20381] Updated weights for policy 1, policy_version 1590436 (0.0007) [2023-12-27 16:40:31,514][20381] Updated weights for policy 1, policy_version 1590446 (0.0007) [2023-12-27 16:40:31,560][20381] Updated weights for policy 1, policy_version 1590456 (0.0008) [2023-12-27 16:40:31,995][20347] Updated weights for policy 0, policy_version 1592021 (0.0007) [2023-12-27 16:40:32,053][20347] Updated weights for policy 0, policy_version 1592031 (0.0007) [2023-12-27 16:40:32,104][20347] Updated weights for policy 0, policy_version 1592041 (0.0007) [2023-12-27 16:40:32,265][20381] Updated weights for policy 1, policy_version 1590466 (0.0006) [2023-12-27 16:40:32,320][20381] Updated weights for policy 1, policy_version 1590476 (0.0006) [2023-12-27 16:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 814841856. Throughput: 0: 10179.0, 1: 10461.9. Samples: 814854164. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:32,363][19170] Avg episode reward: [(0, '970.710'), (1, '423.890')] [2023-12-27 16:40:32,379][20381] Updated weights for policy 1, policy_version 1590486 (0.0009) [2023-12-27 16:40:32,839][20347] Updated weights for policy 0, policy_version 1592051 (0.0007) [2023-12-27 16:40:32,883][20347] Updated weights for policy 0, policy_version 1592061 (0.0007) [2023-12-27 16:40:32,936][20347] Updated weights for policy 0, policy_version 1592071 (0.0005) [2023-12-27 16:40:33,045][20381] Updated weights for policy 1, policy_version 1590496 (0.0010) [2023-12-27 16:40:33,102][20381] Updated weights for policy 1, policy_version 1590506 (0.0010) [2023-12-27 16:40:33,159][20381] Updated weights for policy 1, policy_version 1590516 (0.0010) [2023-12-27 16:40:33,574][20347] Updated weights for policy 0, policy_version 1592081 (0.0005) [2023-12-27 16:40:33,627][20347] Updated weights for policy 0, policy_version 1592091 (0.0007) [2023-12-27 16:40:33,684][20347] Updated weights for policy 0, policy_version 1592101 (0.0008) [2023-12-27 16:40:33,741][20347] Updated weights for policy 0, policy_version 1592111 (0.0008) [2023-12-27 16:40:33,767][20381] Updated weights for policy 1, policy_version 1590526 (0.0007) [2023-12-27 16:40:33,827][20381] Updated weights for policy 1, policy_version 1590536 (0.0005) [2023-12-27 16:40:33,886][20381] Updated weights for policy 1, policy_version 1590546 (0.0005) [2023-12-27 16:40:34,462][20381] Updated weights for policy 1, policy_version 1590556 (0.0007) [2023-12-27 16:40:34,511][20347] Updated weights for policy 0, policy_version 1592121 (0.0009) [2023-12-27 16:40:34,519][20381] Updated weights for policy 1, policy_version 1590566 (0.0011) [2023-12-27 16:40:34,568][20347] Updated weights for policy 0, policy_version 1592131 (0.0008) [2023-12-27 16:40:34,582][20381] Updated weights for policy 1, policy_version 1590576 (0.0011) [2023-12-27 16:40:34,627][20347] Updated weights for policy 0, policy_version 1592141 (0.0010) [2023-12-27 16:40:35,246][20381] Updated weights for policy 1, policy_version 1590586 (0.0010) [2023-12-27 16:40:35,275][20347] Updated weights for policy 0, policy_version 1592151 (0.0010) [2023-12-27 16:40:35,306][20381] Updated weights for policy 1, policy_version 1590596 (0.0005) [2023-12-27 16:40:35,326][20347] Updated weights for policy 0, policy_version 1592161 (0.0010) [2023-12-27 16:40:35,367][20381] Updated weights for policy 1, policy_version 1590606 (0.0005) [2023-12-27 16:40:35,384][20347] Updated weights for policy 0, policy_version 1592171 (0.0010) [2023-12-27 16:40:35,423][20381] Updated weights for policy 1, policy_version 1590616 (0.0005) [2023-12-27 16:40:35,952][20381] Updated weights for policy 1, policy_version 1590626 (0.0005) [2023-12-27 16:40:36,017][20381] Updated weights for policy 1, policy_version 1590636 (0.0005) [2023-12-27 16:40:36,049][20347] Updated weights for policy 0, policy_version 1592181 (0.0008) [2023-12-27 16:40:36,085][20381] Updated weights for policy 1, policy_version 1590646 (0.0006) [2023-12-27 16:40:36,099][20347] Updated weights for policy 0, policy_version 1592191 (0.0007) [2023-12-27 16:40:36,160][20347] Updated weights for policy 0, policy_version 1592201 (0.0008) [2023-12-27 16:40:36,718][20381] Updated weights for policy 1, policy_version 1590656 (0.0010) [2023-12-27 16:40:36,776][20381] Updated weights for policy 1, policy_version 1590666 (0.0011) [2023-12-27 16:40:36,836][20381] Updated weights for policy 1, policy_version 1590676 (0.0006) [2023-12-27 16:40:36,910][20347] Updated weights for policy 0, policy_version 1592211 (0.0009) [2023-12-27 16:40:36,964][20347] Updated weights for policy 0, policy_version 1592221 (0.0009) [2023-12-27 16:40:37,028][20347] Updated weights for policy 0, policy_version 1592231 (0.0005) [2023-12-27 16:40:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 814948352. Throughput: 0: 10175.4, 1: 10465.3. Samples: 814916696. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:37,363][19170] Avg episode reward: [(0, '980.470'), (1, '641.830')] [2023-12-27 16:40:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001592240_407691264.pth... [2023-12-27 16:40:37,370][20381] Updated weights for policy 1, policy_version 1590686 (0.0009) [2023-12-27 16:40:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001591056_407388160.pth [2023-12-27 16:40:37,425][20381] Updated weights for policy 1, policy_version 1590696 (0.0011) [2023-12-27 16:40:37,481][20381] Updated weights for policy 1, policy_version 1590706 (0.0010) [2023-12-27 16:40:37,514][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001590712_407265280.pth... [2023-12-27 16:40:37,518][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001589464_406945792.pth [2023-12-27 16:40:37,674][20347] Updated weights for policy 0, policy_version 1592241 (0.0006) [2023-12-27 16:40:37,727][20347] Updated weights for policy 0, policy_version 1592251 (0.0010) [2023-12-27 16:40:37,776][20347] Updated weights for policy 0, policy_version 1592261 (0.0010) [2023-12-27 16:40:37,824][20347] Updated weights for policy 0, policy_version 1592271 (0.0010) [2023-12-27 16:40:38,177][20381] Updated weights for policy 1, policy_version 1590716 (0.0011) [2023-12-27 16:40:38,239][20381] Updated weights for policy 1, policy_version 1590726 (0.0011) [2023-12-27 16:40:38,293][20381] Updated weights for policy 1, policy_version 1590736 (0.0010) [2023-12-27 16:40:38,549][20347] Updated weights for policy 0, policy_version 1592281 (0.0008) [2023-12-27 16:40:38,608][20347] Updated weights for policy 0, policy_version 1592291 (0.0010) [2023-12-27 16:40:38,664][20347] Updated weights for policy 0, policy_version 1592301 (0.0010) [2023-12-27 16:40:38,971][20381] Updated weights for policy 1, policy_version 1590746 (0.0010) [2023-12-27 16:40:39,024][20381] Updated weights for policy 1, policy_version 1590756 (0.0005) [2023-12-27 16:40:39,084][20381] Updated weights for policy 1, policy_version 1590766 (0.0005) [2023-12-27 16:40:39,145][20381] Updated weights for policy 1, policy_version 1590776 (0.0006) [2023-12-27 16:40:39,408][20347] Updated weights for policy 0, policy_version 1592311 (0.0010) [2023-12-27 16:40:39,470][20347] Updated weights for policy 0, policy_version 1592321 (0.0009) [2023-12-27 16:40:39,536][20347] Updated weights for policy 0, policy_version 1592331 (0.0008) [2023-12-27 16:40:39,786][20381] Updated weights for policy 1, policy_version 1590786 (0.0006) [2023-12-27 16:40:39,853][20381] Updated weights for policy 1, policy_version 1590796 (0.0010) [2023-12-27 16:40:39,904][20381] Updated weights for policy 1, policy_version 1590806 (0.0010) [2023-12-27 16:40:40,231][20347] Updated weights for policy 0, policy_version 1592341 (0.0008) [2023-12-27 16:40:40,291][20347] Updated weights for policy 0, policy_version 1592351 (0.0008) [2023-12-27 16:40:40,355][20347] Updated weights for policy 0, policy_version 1592361 (0.0009) [2023-12-27 16:40:40,551][20381] Updated weights for policy 1, policy_version 1590816 (0.0010) [2023-12-27 16:40:40,610][20381] Updated weights for policy 1, policy_version 1590826 (0.0011) [2023-12-27 16:40:40,673][20381] Updated weights for policy 1, policy_version 1590836 (0.0010) [2023-12-27 16:40:41,109][20347] Updated weights for policy 0, policy_version 1592371 (0.0008) [2023-12-27 16:40:41,170][20347] Updated weights for policy 0, policy_version 1592381 (0.0009) [2023-12-27 16:40:41,224][20347] Updated weights for policy 0, policy_version 1592391 (0.0008) [2023-12-27 16:40:41,402][20381] Updated weights for policy 1, policy_version 1590846 (0.0007) [2023-12-27 16:40:41,470][20381] Updated weights for policy 1, policy_version 1590856 (0.0008) [2023-12-27 16:40:41,535][20381] Updated weights for policy 1, policy_version 1590866 (0.0008) [2023-12-27 16:40:42,022][20347] Updated weights for policy 0, policy_version 1592401 (0.0006) [2023-12-27 16:40:42,081][20347] Updated weights for policy 0, policy_version 1592411 (0.0008) [2023-12-27 16:40:42,135][20347] Updated weights for policy 0, policy_version 1592421 (0.0010) [2023-12-27 16:40:42,165][20381] Updated weights for policy 1, policy_version 1590876 (0.0006) [2023-12-27 16:40:42,182][20347] Updated weights for policy 0, policy_version 1592431 (0.0008) [2023-12-27 16:40:42,226][20381] Updated weights for policy 1, policy_version 1590886 (0.0006) [2023-12-27 16:40:42,287][20381] Updated weights for policy 1, policy_version 1590896 (0.0008) [2023-12-27 16:40:42,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20616.3, 300 sec: 20577.2). Total num frames: 815054848. Throughput: 0: 10098.4, 1: 10499.3. Samples: 815040944. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:42,364][19170] Avg episode reward: [(0, '980.610'), (1, '552.030')] [2023-12-27 16:40:42,935][20347] Updated weights for policy 0, policy_version 1592441 (0.0008) [2023-12-27 16:40:42,947][20381] Updated weights for policy 1, policy_version 1590906 (0.0007) [2023-12-27 16:40:42,994][20347] Updated weights for policy 0, policy_version 1592451 (0.0008) [2023-12-27 16:40:42,998][20381] Updated weights for policy 1, policy_version 1590916 (0.0006) [2023-12-27 16:40:43,046][20347] Updated weights for policy 0, policy_version 1592461 (0.0006) [2023-12-27 16:40:43,048][20381] Updated weights for policy 1, policy_version 1590926 (0.0007) [2023-12-27 16:40:43,099][20381] Updated weights for policy 1, policy_version 1590936 (0.0007) [2023-12-27 16:40:43,671][20381] Updated weights for policy 1, policy_version 1590946 (0.0008) [2023-12-27 16:40:43,730][20381] Updated weights for policy 1, policy_version 1590956 (0.0008) [2023-12-27 16:40:43,768][20347] Updated weights for policy 0, policy_version 1592471 (0.0008) [2023-12-27 16:40:43,778][20381] Updated weights for policy 1, policy_version 1590966 (0.0006) [2023-12-27 16:40:43,824][20347] Updated weights for policy 0, policy_version 1592481 (0.0007) [2023-12-27 16:40:43,883][20347] Updated weights for policy 0, policy_version 1592491 (0.0007) [2023-12-27 16:40:44,467][20381] Updated weights for policy 1, policy_version 1590977 (0.0008) [2023-12-27 16:40:44,491][20347] Updated weights for policy 0, policy_version 1592501 (0.0009) [2023-12-27 16:40:44,529][20381] Updated weights for policy 1, policy_version 1590987 (0.0009) [2023-12-27 16:40:44,560][20347] Updated weights for policy 0, policy_version 1592511 (0.0010) [2023-12-27 16:40:44,587][20381] Updated weights for policy 1, policy_version 1590997 (0.0009) [2023-12-27 16:40:44,626][20347] Updated weights for policy 0, policy_version 1592521 (0.0010) [2023-12-27 16:40:45,290][20381] Updated weights for policy 1, policy_version 1591007 (0.0006) [2023-12-27 16:40:45,297][20347] Updated weights for policy 0, policy_version 1592531 (0.0009) [2023-12-27 16:40:45,344][20381] Updated weights for policy 1, policy_version 1591017 (0.0008) [2023-12-27 16:40:45,353][20347] Updated weights for policy 0, policy_version 1592541 (0.0007) [2023-12-27 16:40:45,406][20381] Updated weights for policy 1, policy_version 1591027 (0.0006) [2023-12-27 16:40:45,407][20347] Updated weights for policy 0, policy_version 1592551 (0.0007) [2023-12-27 16:40:45,986][20381] Updated weights for policy 1, policy_version 1591037 (0.0008) [2023-12-27 16:40:46,026][20347] Updated weights for policy 0, policy_version 1592561 (0.0005) [2023-12-27 16:40:46,046][20381] Updated weights for policy 1, policy_version 1591047 (0.0007) [2023-12-27 16:40:46,081][20347] Updated weights for policy 0, policy_version 1592571 (0.0005) [2023-12-27 16:40:46,105][20381] Updated weights for policy 1, policy_version 1591057 (0.0005) [2023-12-27 16:40:46,133][20347] Updated weights for policy 0, policy_version 1592581 (0.0005) [2023-12-27 16:40:46,181][20347] Updated weights for policy 0, policy_version 1592591 (0.0007) [2023-12-27 16:40:46,700][20381] Updated weights for policy 1, policy_version 1591067 (0.0006) [2023-12-27 16:40:46,756][20381] Updated weights for policy 1, policy_version 1591077 (0.0008) [2023-12-27 16:40:46,808][20381] Updated weights for policy 1, policy_version 1591087 (0.0008) [2023-12-27 16:40:46,891][20347] Updated weights for policy 0, policy_version 1592601 (0.0007) [2023-12-27 16:40:46,944][20347] Updated weights for policy 0, policy_version 1592611 (0.0006) [2023-12-27 16:40:47,000][20347] Updated weights for policy 0, policy_version 1592621 (0.0008) [2023-12-27 16:40:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 815153152. Throughput: 0: 10100.2, 1: 10572.2. Samples: 815164440. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:47,364][19170] Avg episode reward: [(0, '990.300'), (1, '347.500')] [2023-12-27 16:40:47,484][20381] Updated weights for policy 1, policy_version 1591097 (0.0007) [2023-12-27 16:40:47,538][20381] Updated weights for policy 1, policy_version 1591107 (0.0008) [2023-12-27 16:40:47,595][20381] Updated weights for policy 1, policy_version 1591117 (0.0007) [2023-12-27 16:40:47,649][20381] Updated weights for policy 1, policy_version 1591127 (0.0008) [2023-12-27 16:40:47,673][20347] Updated weights for policy 0, policy_version 1592631 (0.0006) [2023-12-27 16:40:47,727][20347] Updated weights for policy 0, policy_version 1592641 (0.0006) [2023-12-27 16:40:47,777][20347] Updated weights for policy 0, policy_version 1592651 (0.0008) [2023-12-27 16:40:48,377][20381] Updated weights for policy 1, policy_version 1591137 (0.0008) [2023-12-27 16:40:48,420][20347] Updated weights for policy 0, policy_version 1592661 (0.0007) [2023-12-27 16:40:48,437][20381] Updated weights for policy 1, policy_version 1591147 (0.0006) [2023-12-27 16:40:48,481][20347] Updated weights for policy 0, policy_version 1592671 (0.0006) [2023-12-27 16:40:48,490][20381] Updated weights for policy 1, policy_version 1591157 (0.0006) [2023-12-27 16:40:48,538][20347] Updated weights for policy 0, policy_version 1592681 (0.0006) [2023-12-27 16:40:49,138][20381] Updated weights for policy 1, policy_version 1591167 (0.0006) [2023-12-27 16:40:49,190][20381] Updated weights for policy 1, policy_version 1591177 (0.0008) [2023-12-27 16:40:49,256][20381] Updated weights for policy 1, policy_version 1591187 (0.0008) [2023-12-27 16:40:49,266][20347] Updated weights for policy 0, policy_version 1592691 (0.0007) [2023-12-27 16:40:49,324][20347] Updated weights for policy 0, policy_version 1592701 (0.0007) [2023-12-27 16:40:49,384][20347] Updated weights for policy 0, policy_version 1592711 (0.0007) [2023-12-27 16:40:50,038][20381] Updated weights for policy 1, policy_version 1591197 (0.0007) [2023-12-27 16:40:50,087][20381] Updated weights for policy 1, policy_version 1591207 (0.0008) [2023-12-27 16:40:50,097][20347] Updated weights for policy 0, policy_version 1592721 (0.0007) [2023-12-27 16:40:50,141][20381] Updated weights for policy 1, policy_version 1591217 (0.0010) [2023-12-27 16:40:50,152][20347] Updated weights for policy 0, policy_version 1592731 (0.0006) [2023-12-27 16:40:50,202][20347] Updated weights for policy 0, policy_version 1592741 (0.0008) [2023-12-27 16:40:50,265][20347] Updated weights for policy 0, policy_version 1592751 (0.0010) [2023-12-27 16:40:50,886][20381] Updated weights for policy 1, policy_version 1591227 (0.0010) [2023-12-27 16:40:50,947][20381] Updated weights for policy 1, policy_version 1591237 (0.0006) [2023-12-27 16:40:50,963][20347] Updated weights for policy 0, policy_version 1592761 (0.0011) [2023-12-27 16:40:51,002][20381] Updated weights for policy 1, policy_version 1591247 (0.0005) [2023-12-27 16:40:51,026][20347] Updated weights for policy 0, policy_version 1592771 (0.0011) [2023-12-27 16:40:51,092][20347] Updated weights for policy 0, policy_version 1592781 (0.0011) [2023-12-27 16:40:51,715][20381] Updated weights for policy 1, policy_version 1591257 (0.0007) [2023-12-27 16:40:51,780][20381] Updated weights for policy 1, policy_version 1591267 (0.0007) [2023-12-27 16:40:51,813][20347] Updated weights for policy 0, policy_version 1592791 (0.0011) [2023-12-27 16:40:51,835][20381] Updated weights for policy 1, policy_version 1591277 (0.0008) [2023-12-27 16:40:51,874][20347] Updated weights for policy 0, policy_version 1592801 (0.0011) [2023-12-27 16:40:51,901][20381] Updated weights for policy 1, policy_version 1591287 (0.0006) [2023-12-27 16:40:51,931][20347] Updated weights for policy 0, policy_version 1592811 (0.0010) [2023-12-27 16:40:52,362][19170] Fps is (10 sec: 19661.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 815251456. Throughput: 0: 10051.2, 1: 10526.6. Samples: 815224000. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:52,363][19170] Avg episode reward: [(0, '970.690'), (1, '432.940')] [2023-12-27 16:40:52,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001591288_407412736.pth... [2023-12-27 16:40:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001592816_407838720.pth... [2023-12-27 16:40:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001591632_407535616.pth [2023-12-27 16:40:52,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001590040_407093248.pth [2023-12-27 16:40:52,546][20381] Updated weights for policy 1, policy_version 1591297 (0.0007) [2023-12-27 16:40:52,593][20381] Updated weights for policy 1, policy_version 1591307 (0.0007) [2023-12-27 16:40:52,648][20381] Updated weights for policy 1, policy_version 1591317 (0.0007) [2023-12-27 16:40:52,662][20347] Updated weights for policy 0, policy_version 1592821 (0.0011) [2023-12-27 16:40:52,721][20347] Updated weights for policy 0, policy_version 1592831 (0.0010) [2023-12-27 16:40:52,779][20347] Updated weights for policy 0, policy_version 1592841 (0.0010) [2023-12-27 16:40:53,378][20347] Updated weights for policy 0, policy_version 1592851 (0.0007) [2023-12-27 16:40:53,383][20381] Updated weights for policy 1, policy_version 1591327 (0.0006) [2023-12-27 16:40:53,434][20347] Updated weights for policy 0, policy_version 1592861 (0.0005) [2023-12-27 16:40:53,439][20381] Updated weights for policy 1, policy_version 1591337 (0.0008) [2023-12-27 16:40:53,488][20347] Updated weights for policy 0, policy_version 1592871 (0.0005) [2023-12-27 16:40:53,498][20381] Updated weights for policy 1, policy_version 1591347 (0.0008) [2023-12-27 16:40:54,135][20347] Updated weights for policy 0, policy_version 1592881 (0.0010) [2023-12-27 16:40:54,156][20381] Updated weights for policy 1, policy_version 1591357 (0.0005) [2023-12-27 16:40:54,191][20347] Updated weights for policy 0, policy_version 1592891 (0.0006) [2023-12-27 16:40:54,202][20381] Updated weights for policy 1, policy_version 1591367 (0.0007) [2023-12-27 16:40:54,252][20347] Updated weights for policy 0, policy_version 1592901 (0.0006) [2023-12-27 16:40:54,255][20381] Updated weights for policy 1, policy_version 1591377 (0.0007) [2023-12-27 16:40:54,313][20347] Updated weights for policy 0, policy_version 1592911 (0.0006) [2023-12-27 16:40:54,885][20347] Updated weights for policy 0, policy_version 1592921 (0.0010) [2023-12-27 16:40:54,949][20347] Updated weights for policy 0, policy_version 1592931 (0.0010) [2023-12-27 16:40:54,982][20381] Updated weights for policy 1, policy_version 1591387 (0.0007) [2023-12-27 16:40:55,008][20347] Updated weights for policy 0, policy_version 1592941 (0.0010) [2023-12-27 16:40:55,038][20381] Updated weights for policy 1, policy_version 1591397 (0.0007) [2023-12-27 16:40:55,094][20381] Updated weights for policy 1, policy_version 1591407 (0.0007) [2023-12-27 16:40:55,592][20347] Updated weights for policy 0, policy_version 1592951 (0.0008) [2023-12-27 16:40:55,648][20347] Updated weights for policy 0, policy_version 1592961 (0.0007) [2023-12-27 16:40:55,699][20347] Updated weights for policy 0, policy_version 1592971 (0.0005) [2023-12-27 16:40:55,804][20381] Updated weights for policy 1, policy_version 1591417 (0.0007) [2023-12-27 16:40:55,852][20381] Updated weights for policy 1, policy_version 1591427 (0.0008) [2023-12-27 16:40:55,901][20381] Updated weights for policy 1, policy_version 1591437 (0.0008) [2023-12-27 16:40:55,947][20381] Updated weights for policy 1, policy_version 1591447 (0.0008) [2023-12-27 16:40:56,364][20347] Updated weights for policy 0, policy_version 1592981 (0.0006) [2023-12-27 16:40:56,411][20347] Updated weights for policy 0, policy_version 1592991 (0.0007) [2023-12-27 16:40:56,461][20347] Updated weights for policy 0, policy_version 1593001 (0.0007) [2023-12-27 16:40:56,643][20381] Updated weights for policy 1, policy_version 1591457 (0.0008) [2023-12-27 16:40:56,694][20381] Updated weights for policy 1, policy_version 1591467 (0.0007) [2023-12-27 16:40:56,746][20381] Updated weights for policy 1, policy_version 1591477 (0.0008) [2023-12-27 16:40:57,083][20347] Updated weights for policy 0, policy_version 1593011 (0.0008) [2023-12-27 16:40:57,143][20347] Updated weights for policy 0, policy_version 1593021 (0.0007) [2023-12-27 16:40:57,197][20347] Updated weights for policy 0, policy_version 1593031 (0.0007) [2023-12-27 16:40:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 815357952. Throughput: 0: 10148.5, 1: 10469.4. Samples: 815345972. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:40:57,363][19170] Avg episode reward: [(0, '960.930'), (1, '536.880')] [2023-12-27 16:40:57,478][20381] Updated weights for policy 1, policy_version 1591487 (0.0007) [2023-12-27 16:40:57,531][20381] Updated weights for policy 1, policy_version 1591497 (0.0006) [2023-12-27 16:40:57,577][20381] Updated weights for policy 1, policy_version 1591507 (0.0006) [2023-12-27 16:40:57,811][20347] Updated weights for policy 0, policy_version 1593041 (0.0008) [2023-12-27 16:40:57,867][20347] Updated weights for policy 0, policy_version 1593051 (0.0010) [2023-12-27 16:40:57,915][20347] Updated weights for policy 0, policy_version 1593061 (0.0010) [2023-12-27 16:40:57,971][20347] Updated weights for policy 0, policy_version 1593071 (0.0010) [2023-12-27 16:40:58,146][20381] Updated weights for policy 1, policy_version 1591517 (0.0005) [2023-12-27 16:40:58,211][20381] Updated weights for policy 1, policy_version 1591527 (0.0006) [2023-12-27 16:40:58,264][20381] Updated weights for policy 1, policy_version 1591537 (0.0008) [2023-12-27 16:40:58,675][20347] Updated weights for policy 0, policy_version 1593081 (0.0008) [2023-12-27 16:40:58,741][20347] Updated weights for policy 0, policy_version 1593091 (0.0008) [2023-12-27 16:40:58,807][20347] Updated weights for policy 0, policy_version 1593101 (0.0009) [2023-12-27 16:40:58,965][20381] Updated weights for policy 1, policy_version 1591547 (0.0007) [2023-12-27 16:40:59,028][20381] Updated weights for policy 1, policy_version 1591557 (0.0008) [2023-12-27 16:40:59,087][20381] Updated weights for policy 1, policy_version 1591567 (0.0008) [2023-12-27 16:40:59,556][20347] Updated weights for policy 0, policy_version 1593111 (0.0007) [2023-12-27 16:40:59,617][20347] Updated weights for policy 0, policy_version 1593121 (0.0008) [2023-12-27 16:40:59,677][20347] Updated weights for policy 0, policy_version 1593131 (0.0008) [2023-12-27 16:40:59,839][20381] Updated weights for policy 1, policy_version 1591577 (0.0007) [2023-12-27 16:40:59,903][20381] Updated weights for policy 1, policy_version 1591587 (0.0007) [2023-12-27 16:40:59,972][20381] Updated weights for policy 1, policy_version 1591597 (0.0006) [2023-12-27 16:41:00,029][20381] Updated weights for policy 1, policy_version 1591607 (0.0006) [2023-12-27 16:41:00,419][20347] Updated weights for policy 0, policy_version 1593141 (0.0008) [2023-12-27 16:41:00,473][20347] Updated weights for policy 0, policy_version 1593151 (0.0007) [2023-12-27 16:41:00,531][20347] Updated weights for policy 0, policy_version 1593161 (0.0005) [2023-12-27 16:41:00,624][20381] Updated weights for policy 1, policy_version 1591617 (0.0007) [2023-12-27 16:41:00,678][20381] Updated weights for policy 1, policy_version 1591627 (0.0007) [2023-12-27 16:41:00,734][20381] Updated weights for policy 1, policy_version 1591637 (0.0007) [2023-12-27 16:41:01,189][20347] Updated weights for policy 0, policy_version 1593171 (0.0008) [2023-12-27 16:41:01,254][20347] Updated weights for policy 0, policy_version 1593181 (0.0008) [2023-12-27 16:41:01,313][20347] Updated weights for policy 0, policy_version 1593191 (0.0009) [2023-12-27 16:41:01,465][20381] Updated weights for policy 1, policy_version 1591647 (0.0008) [2023-12-27 16:41:01,529][20381] Updated weights for policy 1, policy_version 1591657 (0.0008) [2023-12-27 16:41:01,587][20381] Updated weights for policy 1, policy_version 1591667 (0.0008) [2023-12-27 16:41:02,002][20347] Updated weights for policy 0, policy_version 1593201 (0.0008) [2023-12-27 16:41:02,061][20347] Updated weights for policy 0, policy_version 1593211 (0.0006) [2023-12-27 16:41:02,119][20347] Updated weights for policy 0, policy_version 1593221 (0.0005) [2023-12-27 16:41:02,175][20347] Updated weights for policy 0, policy_version 1593231 (0.0007) [2023-12-27 16:41:02,305][20381] Updated weights for policy 1, policy_version 1591677 (0.0007) [2023-12-27 16:41:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 815456256. Throughput: 0: 10158.4, 1: 10424.5. Samples: 815467724. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:02,363][19170] Avg episode reward: [(0, '990.240'), (1, '530.110')] [2023-12-27 16:41:02,372][20381] Updated weights for policy 1, policy_version 1591687 (0.0007) [2023-12-27 16:41:02,429][20381] Updated weights for policy 1, policy_version 1591697 (0.0007) [2023-12-27 16:41:02,808][20347] Updated weights for policy 0, policy_version 1593241 (0.0008) [2023-12-27 16:41:02,858][20347] Updated weights for policy 0, policy_version 1593251 (0.0007) [2023-12-27 16:41:02,903][20347] Updated weights for policy 0, policy_version 1593261 (0.0007) [2023-12-27 16:41:03,132][20381] Updated weights for policy 1, policy_version 1591707 (0.0009) [2023-12-27 16:41:03,188][20381] Updated weights for policy 1, policy_version 1591717 (0.0007) [2023-12-27 16:41:03,245][20381] Updated weights for policy 1, policy_version 1591727 (0.0007) [2023-12-27 16:41:03,657][20347] Updated weights for policy 0, policy_version 1593271 (0.0009) [2023-12-27 16:41:03,713][20347] Updated weights for policy 0, policy_version 1593281 (0.0008) [2023-12-27 16:41:03,765][20347] Updated weights for policy 0, policy_version 1593291 (0.0008) [2023-12-27 16:41:03,852][20381] Updated weights for policy 1, policy_version 1591737 (0.0006) [2023-12-27 16:41:03,919][20381] Updated weights for policy 1, policy_version 1591747 (0.0007) [2023-12-27 16:41:03,989][20381] Updated weights for policy 1, policy_version 1591757 (0.0007) [2023-12-27 16:41:04,046][20381] Updated weights for policy 1, policy_version 1591767 (0.0008) [2023-12-27 16:41:04,476][20347] Updated weights for policy 0, policy_version 1593301 (0.0007) [2023-12-27 16:41:04,529][20347] Updated weights for policy 0, policy_version 1593311 (0.0008) [2023-12-27 16:41:04,586][20347] Updated weights for policy 0, policy_version 1593321 (0.0008) [2023-12-27 16:41:04,717][20381] Updated weights for policy 1, policy_version 1591777 (0.0008) [2023-12-27 16:41:04,774][20381] Updated weights for policy 1, policy_version 1591787 (0.0007) [2023-12-27 16:41:04,833][20381] Updated weights for policy 1, policy_version 1591797 (0.0008) [2023-12-27 16:41:05,254][20347] Updated weights for policy 0, policy_version 1593331 (0.0008) [2023-12-27 16:41:05,300][20347] Updated weights for policy 0, policy_version 1593341 (0.0007) [2023-12-27 16:41:05,355][20347] Updated weights for policy 0, policy_version 1593351 (0.0007) [2023-12-27 16:41:05,484][20381] Updated weights for policy 1, policy_version 1591807 (0.0008) [2023-12-27 16:41:05,534][20381] Updated weights for policy 1, policy_version 1591817 (0.0008) [2023-12-27 16:41:05,584][20381] Updated weights for policy 1, policy_version 1591827 (0.0007) [2023-12-27 16:41:06,058][20347] Updated weights for policy 0, policy_version 1593361 (0.0008) [2023-12-27 16:41:06,119][20347] Updated weights for policy 0, policy_version 1593371 (0.0007) [2023-12-27 16:41:06,178][20347] Updated weights for policy 0, policy_version 1593381 (0.0009) [2023-12-27 16:41:06,235][20347] Updated weights for policy 0, policy_version 1593391 (0.0008) [2023-12-27 16:41:06,246][20381] Updated weights for policy 1, policy_version 1591837 (0.0009) [2023-12-27 16:41:06,306][20381] Updated weights for policy 1, policy_version 1591847 (0.0009) [2023-12-27 16:41:06,366][20381] Updated weights for policy 1, policy_version 1591857 (0.0008) [2023-12-27 16:41:06,983][20347] Updated weights for policy 0, policy_version 1593401 (0.0008) [2023-12-27 16:41:07,035][20381] Updated weights for policy 1, policy_version 1591867 (0.0007) [2023-12-27 16:41:07,037][20347] Updated weights for policy 0, policy_version 1593411 (0.0006) [2023-12-27 16:41:07,082][20381] Updated weights for policy 1, policy_version 1591877 (0.0006) [2023-12-27 16:41:07,091][20347] Updated weights for policy 0, policy_version 1593421 (0.0006) [2023-12-27 16:41:07,132][20381] Updated weights for policy 1, policy_version 1591887 (0.0007) [2023-12-27 16:41:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 815562752. Throughput: 0: 10127.3, 1: 10435.2. Samples: 815528836. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '433.300')] [2023-12-27 16:41:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001591896_407568384.pth... [2023-12-27 16:41:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001593424_407994368.pth... [2023-12-27 16:41:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001590712_407265280.pth [2023-12-27 16:41:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001592240_407691264.pth [2023-12-27 16:41:07,733][20347] Updated weights for policy 0, policy_version 1593431 (0.0007) [2023-12-27 16:41:07,795][20347] Updated weights for policy 0, policy_version 1593441 (0.0007) [2023-12-27 16:41:07,830][20381] Updated weights for policy 1, policy_version 1591897 (0.0008) [2023-12-27 16:41:07,855][20347] Updated weights for policy 0, policy_version 1593451 (0.0006) [2023-12-27 16:41:07,884][20381] Updated weights for policy 1, policy_version 1591907 (0.0011) [2023-12-27 16:41:07,942][20381] Updated weights for policy 1, policy_version 1591917 (0.0009) [2023-12-27 16:41:07,998][20381] Updated weights for policy 1, policy_version 1591927 (0.0006) [2023-12-27 16:41:08,568][20381] Updated weights for policy 1, policy_version 1591937 (0.0011) [2023-12-27 16:41:08,574][20347] Updated weights for policy 0, policy_version 1593461 (0.0007) [2023-12-27 16:41:08,621][20347] Updated weights for policy 0, policy_version 1593471 (0.0008) [2023-12-27 16:41:08,624][20381] Updated weights for policy 1, policy_version 1591947 (0.0011) [2023-12-27 16:41:08,673][20347] Updated weights for policy 0, policy_version 1593481 (0.0008) [2023-12-27 16:41:08,683][20381] Updated weights for policy 1, policy_version 1591957 (0.0011) [2023-12-27 16:41:09,376][20347] Updated weights for policy 0, policy_version 1593491 (0.0009) [2023-12-27 16:41:09,383][20381] Updated weights for policy 1, policy_version 1591967 (0.0008) [2023-12-27 16:41:09,448][20347] Updated weights for policy 0, policy_version 1593501 (0.0010) [2023-12-27 16:41:09,449][20381] Updated weights for policy 1, policy_version 1591977 (0.0008) [2023-12-27 16:41:09,505][20347] Updated weights for policy 0, policy_version 1593511 (0.0011) [2023-12-27 16:41:09,510][20381] Updated weights for policy 1, policy_version 1591987 (0.0007) [2023-12-27 16:41:10,180][20347] Updated weights for policy 0, policy_version 1593521 (0.0010) [2023-12-27 16:41:10,228][20381] Updated weights for policy 1, policy_version 1591997 (0.0006) [2023-12-27 16:41:10,237][20347] Updated weights for policy 0, policy_version 1593531 (0.0009) [2023-12-27 16:41:10,288][20381] Updated weights for policy 1, policy_version 1592007 (0.0006) [2023-12-27 16:41:10,300][20347] Updated weights for policy 0, policy_version 1593541 (0.0009) [2023-12-27 16:41:10,348][20381] Updated weights for policy 1, policy_version 1592017 (0.0006) [2023-12-27 16:41:10,361][20347] Updated weights for policy 0, policy_version 1593551 (0.0008) [2023-12-27 16:41:11,000][20381] Updated weights for policy 1, policy_version 1592027 (0.0007) [2023-12-27 16:41:11,062][20347] Updated weights for policy 0, policy_version 1593561 (0.0009) [2023-12-27 16:41:11,065][20381] Updated weights for policy 1, policy_version 1592037 (0.0007) [2023-12-27 16:41:11,125][20381] Updated weights for policy 1, policy_version 1592047 (0.0007) [2023-12-27 16:41:11,139][20347] Updated weights for policy 0, policy_version 1593571 (0.0009) [2023-12-27 16:41:11,210][20347] Updated weights for policy 0, policy_version 1593581 (0.0009) [2023-12-27 16:41:11,866][20381] Updated weights for policy 1, policy_version 1592057 (0.0007) [2023-12-27 16:41:11,922][20381] Updated weights for policy 1, policy_version 1592067 (0.0008) [2023-12-27 16:41:11,972][20381] Updated weights for policy 1, policy_version 1592077 (0.0006) [2023-12-27 16:41:12,030][20381] Updated weights for policy 1, policy_version 1592087 (0.0006) [2023-12-27 16:41:12,051][20347] Updated weights for policy 0, policy_version 1593591 (0.0007) [2023-12-27 16:41:12,066][20291] KL-divergence is very high: 102.6895 [2023-12-27 16:41:12,112][20347] Updated weights for policy 0, policy_version 1593601 (0.0006) [2023-12-27 16:41:12,112][20291] KL-divergence is very high: 120.9061 [2023-12-27 16:41:12,170][20347] Updated weights for policy 0, policy_version 1593611 (0.0005) [2023-12-27 16:41:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 815661056. Throughput: 0: 10075.8, 1: 10400.5. Samples: 815650484. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:12,363][19170] Avg episode reward: [(0, '990.240'), (1, '503.830')] [2023-12-27 16:41:12,649][20381] Updated weights for policy 1, policy_version 1592097 (0.0006) [2023-12-27 16:41:12,701][20381] Updated weights for policy 1, policy_version 1592107 (0.0005) [2023-12-27 16:41:12,753][20381] Updated weights for policy 1, policy_version 1592117 (0.0006) [2023-12-27 16:41:12,856][20347] Updated weights for policy 0, policy_version 1593621 (0.0007) [2023-12-27 16:41:12,905][20347] Updated weights for policy 0, policy_version 1593631 (0.0007) [2023-12-27 16:41:12,960][20347] Updated weights for policy 0, policy_version 1593641 (0.0007) [2023-12-27 16:41:13,370][20381] Updated weights for policy 1, policy_version 1592127 (0.0006) [2023-12-27 16:41:13,428][20381] Updated weights for policy 1, policy_version 1592137 (0.0005) [2023-12-27 16:41:13,486][20381] Updated weights for policy 1, policy_version 1592147 (0.0005) [2023-12-27 16:41:13,718][20347] Updated weights for policy 0, policy_version 1593651 (0.0008) [2023-12-27 16:41:13,770][20347] Updated weights for policy 0, policy_version 1593661 (0.0008) [2023-12-27 16:41:13,832][20347] Updated weights for policy 0, policy_version 1593671 (0.0006) [2023-12-27 16:41:13,990][20381] Updated weights for policy 1, policy_version 1592157 (0.0008) [2023-12-27 16:41:14,046][20381] Updated weights for policy 1, policy_version 1592167 (0.0010) [2023-12-27 16:41:14,094][20381] Updated weights for policy 1, policy_version 1592177 (0.0010) [2023-12-27 16:41:14,410][20347] Updated weights for policy 0, policy_version 1593681 (0.0006) [2023-12-27 16:41:14,473][20347] Updated weights for policy 0, policy_version 1593691 (0.0007) [2023-12-27 16:41:14,533][20347] Updated weights for policy 0, policy_version 1593701 (0.0007) [2023-12-27 16:41:14,598][20347] Updated weights for policy 0, policy_version 1593711 (0.0008) [2023-12-27 16:41:14,785][20381] Updated weights for policy 1, policy_version 1592187 (0.0007) [2023-12-27 16:41:14,847][20381] Updated weights for policy 1, policy_version 1592197 (0.0010) [2023-12-27 16:41:14,913][20381] Updated weights for policy 1, policy_version 1592207 (0.0008) [2023-12-27 16:41:15,295][20347] Updated weights for policy 0, policy_version 1593721 (0.0007) [2023-12-27 16:41:15,357][20347] Updated weights for policy 0, policy_version 1593731 (0.0007) [2023-12-27 16:41:15,416][20347] Updated weights for policy 0, policy_version 1593741 (0.0008) [2023-12-27 16:41:15,637][20381] Updated weights for policy 1, policy_version 1592217 (0.0008) [2023-12-27 16:41:15,701][20381] Updated weights for policy 1, policy_version 1592227 (0.0007) [2023-12-27 16:41:15,752][20381] Updated weights for policy 1, policy_version 1592237 (0.0005) [2023-12-27 16:41:15,800][20381] Updated weights for policy 1, policy_version 1592247 (0.0008) [2023-12-27 16:41:16,020][20347] Updated weights for policy 0, policy_version 1593751 (0.0008) [2023-12-27 16:41:16,072][20347] Updated weights for policy 0, policy_version 1593761 (0.0006) [2023-12-27 16:41:16,131][20347] Updated weights for policy 0, policy_version 1593771 (0.0006) [2023-12-27 16:41:16,491][20381] Updated weights for policy 1, policy_version 1592257 (0.0008) [2023-12-27 16:41:16,541][20381] Updated weights for policy 1, policy_version 1592267 (0.0008) [2023-12-27 16:41:16,591][20381] Updated weights for policy 1, policy_version 1592277 (0.0007) [2023-12-27 16:41:16,737][20347] Updated weights for policy 0, policy_version 1593781 (0.0007) [2023-12-27 16:41:16,790][20347] Updated weights for policy 0, policy_version 1593791 (0.0005) [2023-12-27 16:41:16,849][20347] Updated weights for policy 0, policy_version 1593801 (0.0006) [2023-12-27 16:41:17,342][20381] Updated weights for policy 1, policy_version 1592287 (0.0007) [2023-12-27 16:41:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 815759360. Throughput: 0: 10081.0, 1: 10353.3. Samples: 815773712. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:17,364][19170] Avg episode reward: [(0, '990.240'), (1, '580.710')] [2023-12-27 16:41:17,392][20381] Updated weights for policy 1, policy_version 1592297 (0.0007) [2023-12-27 16:41:17,416][20347] Updated weights for policy 0, policy_version 1593811 (0.0008) [2023-12-27 16:41:17,447][20381] Updated weights for policy 1, policy_version 1592307 (0.0005) [2023-12-27 16:41:17,468][20347] Updated weights for policy 0, policy_version 1593821 (0.0005) [2023-12-27 16:41:17,520][20347] Updated weights for policy 0, policy_version 1593831 (0.0007) [2023-12-27 16:41:18,072][20347] Updated weights for policy 0, policy_version 1593841 (0.0005) [2023-12-27 16:41:18,121][20347] Updated weights for policy 0, policy_version 1593851 (0.0007) [2023-12-27 16:41:18,172][20347] Updated weights for policy 0, policy_version 1593861 (0.0010) [2023-12-27 16:41:18,207][20381] Updated weights for policy 1, policy_version 1592317 (0.0006) [2023-12-27 16:41:18,221][20347] Updated weights for policy 0, policy_version 1593871 (0.0010) [2023-12-27 16:41:18,267][20381] Updated weights for policy 1, policy_version 1592327 (0.0007) [2023-12-27 16:41:18,333][20381] Updated weights for policy 1, policy_version 1592337 (0.0009) [2023-12-27 16:41:18,881][20347] Updated weights for policy 0, policy_version 1593881 (0.0007) [2023-12-27 16:41:18,939][20347] Updated weights for policy 0, policy_version 1593891 (0.0007) [2023-12-27 16:41:18,990][20381] Updated weights for policy 1, policy_version 1592347 (0.0010) [2023-12-27 16:41:18,997][20347] Updated weights for policy 0, policy_version 1593901 (0.0006) [2023-12-27 16:41:19,046][20381] Updated weights for policy 1, policy_version 1592357 (0.0011) [2023-12-27 16:41:19,098][20381] Updated weights for policy 1, policy_version 1592367 (0.0010) [2023-12-27 16:41:19,692][20347] Updated weights for policy 0, policy_version 1593911 (0.0006) [2023-12-27 16:41:19,743][20347] Updated weights for policy 0, policy_version 1593921 (0.0007) [2023-12-27 16:41:19,757][20291] KL-divergence is very high: 138.2592 [2023-12-27 16:41:19,789][20347] Updated weights for policy 0, policy_version 1593931 (0.0008) [2023-12-27 16:41:19,794][20291] KL-divergence is very high: 107.9784 [2023-12-27 16:41:19,852][20381] Updated weights for policy 1, policy_version 1592377 (0.0011) [2023-12-27 16:41:19,913][20381] Updated weights for policy 1, policy_version 1592387 (0.0008) [2023-12-27 16:41:19,978][20381] Updated weights for policy 1, policy_version 1592397 (0.0011) [2023-12-27 16:41:20,038][20381] Updated weights for policy 1, policy_version 1592407 (0.0011) [2023-12-27 16:41:20,514][20347] Updated weights for policy 0, policy_version 1593941 (0.0006) [2023-12-27 16:41:20,580][20347] Updated weights for policy 0, policy_version 1593951 (0.0007) [2023-12-27 16:41:20,636][20347] Updated weights for policy 0, policy_version 1593961 (0.0010) [2023-12-27 16:41:20,751][20381] Updated weights for policy 1, policy_version 1592417 (0.0008) [2023-12-27 16:41:20,820][20381] Updated weights for policy 1, policy_version 1592427 (0.0008) [2023-12-27 16:41:20,880][20381] Updated weights for policy 1, policy_version 1592437 (0.0008) [2023-12-27 16:41:21,271][20347] Updated weights for policy 0, policy_version 1593971 (0.0010) [2023-12-27 16:41:21,325][20347] Updated weights for policy 0, policy_version 1593981 (0.0009) [2023-12-27 16:41:21,392][20347] Updated weights for policy 0, policy_version 1593991 (0.0008) [2023-12-27 16:41:21,593][20381] Updated weights for policy 1, policy_version 1592447 (0.0009) [2023-12-27 16:41:21,660][20381] Updated weights for policy 1, policy_version 1592457 (0.0008) [2023-12-27 16:41:21,719][20381] Updated weights for policy 1, policy_version 1592467 (0.0007) [2023-12-27 16:41:22,140][20347] Updated weights for policy 0, policy_version 1594001 (0.0008) [2023-12-27 16:41:22,200][20347] Updated weights for policy 0, policy_version 1594011 (0.0005) [2023-12-27 16:41:22,262][20347] Updated weights for policy 0, policy_version 1594021 (0.0008) [2023-12-27 16:41:22,317][20347] Updated weights for policy 0, policy_version 1594031 (0.0007) [2023-12-27 16:41:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 815865856. Throughput: 0: 10133.9, 1: 10282.1. Samples: 815835416. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:22,363][19170] Avg episode reward: [(0, '990.300'), (1, '563.630')] [2023-12-27 16:41:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001594032_408150016.pth... [2023-12-27 16:41:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001592472_407715840.pth... [2023-12-27 16:41:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001591288_407412736.pth [2023-12-27 16:41:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001592816_407838720.pth [2023-12-27 16:41:22,416][20381] Updated weights for policy 1, policy_version 1592477 (0.0008) [2023-12-27 16:41:22,468][20381] Updated weights for policy 1, policy_version 1592487 (0.0008) [2023-12-27 16:41:22,524][20381] Updated weights for policy 1, policy_version 1592497 (0.0008) [2023-12-27 16:41:22,972][20347] Updated weights for policy 0, policy_version 1594041 (0.0010) [2023-12-27 16:41:23,031][20347] Updated weights for policy 0, policy_version 1594051 (0.0010) [2023-12-27 16:41:23,087][20347] Updated weights for policy 0, policy_version 1594061 (0.0006) [2023-12-27 16:41:23,302][20381] Updated weights for policy 1, policy_version 1592507 (0.0008) [2023-12-27 16:41:23,353][20381] Updated weights for policy 1, policy_version 1592517 (0.0007) [2023-12-27 16:41:23,397][20381] Updated weights for policy 1, policy_version 1592527 (0.0007) [2023-12-27 16:41:23,715][20347] Updated weights for policy 0, policy_version 1594071 (0.0009) [2023-12-27 16:41:23,766][20347] Updated weights for policy 0, policy_version 1594081 (0.0010) [2023-12-27 16:41:23,817][20347] Updated weights for policy 0, policy_version 1594091 (0.0010) [2023-12-27 16:41:24,108][20381] Updated weights for policy 1, policy_version 1592537 (0.0008) [2023-12-27 16:41:24,160][20381] Updated weights for policy 1, policy_version 1592547 (0.0009) [2023-12-27 16:41:24,215][20381] Updated weights for policy 1, policy_version 1592557 (0.0010) [2023-12-27 16:41:24,263][20381] Updated weights for policy 1, policy_version 1592567 (0.0008) [2023-12-27 16:41:24,534][20347] Updated weights for policy 0, policy_version 1594101 (0.0009) [2023-12-27 16:41:24,595][20347] Updated weights for policy 0, policy_version 1594111 (0.0007) [2023-12-27 16:41:24,649][20347] Updated weights for policy 0, policy_version 1594121 (0.0007) [2023-12-27 16:41:24,917][20381] Updated weights for policy 1, policy_version 1592577 (0.0010) [2023-12-27 16:41:24,979][20381] Updated weights for policy 1, policy_version 1592587 (0.0010) [2023-12-27 16:41:25,036][20381] Updated weights for policy 1, policy_version 1592597 (0.0006) [2023-12-27 16:41:25,208][20347] Updated weights for policy 0, policy_version 1594131 (0.0007) [2023-12-27 16:41:25,260][20347] Updated weights for policy 0, policy_version 1594141 (0.0008) [2023-12-27 16:41:25,309][20347] Updated weights for policy 0, policy_version 1594151 (0.0009) [2023-12-27 16:41:25,559][20381] Updated weights for policy 1, policy_version 1592607 (0.0005) [2023-12-27 16:41:25,604][20381] Updated weights for policy 1, policy_version 1592617 (0.0005) [2023-12-27 16:41:25,650][20381] Updated weights for policy 1, policy_version 1592627 (0.0005) [2023-12-27 16:41:26,072][20347] Updated weights for policy 0, policy_version 1594161 (0.0008) [2023-12-27 16:41:26,129][20347] Updated weights for policy 0, policy_version 1594171 (0.0007) [2023-12-27 16:41:26,187][20347] Updated weights for policy 0, policy_version 1594181 (0.0007) [2023-12-27 16:41:26,214][20381] Updated weights for policy 1, policy_version 1592637 (0.0007) [2023-12-27 16:41:26,243][20347] Updated weights for policy 0, policy_version 1594191 (0.0007) [2023-12-27 16:41:26,266][20381] Updated weights for policy 1, policy_version 1592647 (0.0010) [2023-12-27 16:41:26,313][20381] Updated weights for policy 1, policy_version 1592657 (0.0010) [2023-12-27 16:41:26,910][20347] Updated weights for policy 0, policy_version 1594201 (0.0007) [2023-12-27 16:41:26,937][20381] Updated weights for policy 1, policy_version 1592667 (0.0009) [2023-12-27 16:41:26,960][20347] Updated weights for policy 0, policy_version 1594211 (0.0008) [2023-12-27 16:41:26,993][20381] Updated weights for policy 1, policy_version 1592677 (0.0007) [2023-12-27 16:41:27,020][20347] Updated weights for policy 0, policy_version 1594221 (0.0007) [2023-12-27 16:41:27,048][20381] Updated weights for policy 1, policy_version 1592687 (0.0007) [2023-12-27 16:41:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 815972352. Throughput: 0: 10193.9, 1: 10220.3. Samples: 815959572. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:27,363][19170] Avg episode reward: [(0, '990.300'), (1, '282.160')] [2023-12-27 16:41:27,603][20381] Updated weights for policy 1, policy_version 1592697 (0.0005) [2023-12-27 16:41:27,657][20381] Updated weights for policy 1, policy_version 1592707 (0.0006) [2023-12-27 16:41:27,711][20381] Updated weights for policy 1, policy_version 1592717 (0.0008) [2023-12-27 16:41:27,743][20347] Updated weights for policy 0, policy_version 1594231 (0.0008) [2023-12-27 16:41:27,759][20381] Updated weights for policy 1, policy_version 1592727 (0.0008) [2023-12-27 16:41:27,799][20347] Updated weights for policy 0, policy_version 1594241 (0.0007) [2023-12-27 16:41:27,863][20347] Updated weights for policy 0, policy_version 1594251 (0.0007) [2023-12-27 16:41:28,440][20381] Updated weights for policy 1, policy_version 1592737 (0.0005) [2023-12-27 16:41:28,491][20381] Updated weights for policy 1, policy_version 1592747 (0.0005) [2023-12-27 16:41:28,538][20347] Updated weights for policy 0, policy_version 1594261 (0.0006) [2023-12-27 16:41:28,545][20381] Updated weights for policy 1, policy_version 1592757 (0.0006) [2023-12-27 16:41:28,593][20347] Updated weights for policy 0, policy_version 1594271 (0.0007) [2023-12-27 16:41:28,645][20347] Updated weights for policy 0, policy_version 1594281 (0.0008) [2023-12-27 16:41:29,185][20381] Updated weights for policy 1, policy_version 1592767 (0.0007) [2023-12-27 16:41:29,245][20381] Updated weights for policy 1, policy_version 1592777 (0.0008) [2023-12-27 16:41:29,302][20381] Updated weights for policy 1, policy_version 1592787 (0.0008) [2023-12-27 16:41:29,398][20347] Updated weights for policy 0, policy_version 1594291 (0.0008) [2023-12-27 16:41:29,455][20347] Updated weights for policy 0, policy_version 1594301 (0.0008) [2023-12-27 16:41:29,512][20347] Updated weights for policy 0, policy_version 1594311 (0.0007) [2023-12-27 16:41:30,013][20381] Updated weights for policy 1, policy_version 1592797 (0.0008) [2023-12-27 16:41:30,068][20381] Updated weights for policy 1, policy_version 1592807 (0.0007) [2023-12-27 16:41:30,124][20381] Updated weights for policy 1, policy_version 1592817 (0.0008) [2023-12-27 16:41:30,248][20347] Updated weights for policy 0, policy_version 1594321 (0.0008) [2023-12-27 16:41:30,312][20347] Updated weights for policy 0, policy_version 1594331 (0.0008) [2023-12-27 16:41:30,375][20347] Updated weights for policy 0, policy_version 1594341 (0.0007) [2023-12-27 16:41:30,437][20347] Updated weights for policy 0, policy_version 1594351 (0.0008) [2023-12-27 16:41:30,788][20381] Updated weights for policy 1, policy_version 1592827 (0.0006) [2023-12-27 16:41:30,836][20381] Updated weights for policy 1, policy_version 1592837 (0.0007) [2023-12-27 16:41:30,883][20381] Updated weights for policy 1, policy_version 1592847 (0.0007) [2023-12-27 16:41:31,056][20347] Updated weights for policy 0, policy_version 1594361 (0.0009) [2023-12-27 16:41:31,084][20291] KL-divergence is very high: 293.8191 [2023-12-27 16:41:31,122][20347] Updated weights for policy 0, policy_version 1594371 (0.0007) [2023-12-27 16:41:31,136][20291] KL-divergence is very high: 310.4575 [2023-12-27 16:41:31,191][20347] Updated weights for policy 0, policy_version 1594381 (0.0008) [2023-12-27 16:41:31,191][20291] KL-divergence is very high: 143.4351 [2023-12-27 16:41:31,604][20381] Updated weights for policy 1, policy_version 1592857 (0.0008) [2023-12-27 16:41:31,667][20381] Updated weights for policy 1, policy_version 1592867 (0.0008) [2023-12-27 16:41:31,729][20381] Updated weights for policy 1, policy_version 1592877 (0.0007) [2023-12-27 16:41:31,786][20381] Updated weights for policy 1, policy_version 1592887 (0.0005) [2023-12-27 16:41:31,832][20347] Updated weights for policy 0, policy_version 1594391 (0.0009) [2023-12-27 16:41:31,884][20347] Updated weights for policy 0, policy_version 1594401 (0.0008) [2023-12-27 16:41:31,955][20347] Updated weights for policy 0, policy_version 1594411 (0.0006) [2023-12-27 16:41:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 816070656. Throughput: 0: 10203.1, 1: 10215.8. Samples: 816083288. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:32,363][19170] Avg episode reward: [(0, '990.260'), (1, '231.380')] [2023-12-27 16:41:32,430][20381] Updated weights for policy 1, policy_version 1592897 (0.0007) [2023-12-27 16:41:32,481][20381] Updated weights for policy 1, policy_version 1592907 (0.0008) [2023-12-27 16:41:32,536][20381] Updated weights for policy 1, policy_version 1592917 (0.0007) [2023-12-27 16:41:32,641][20347] Updated weights for policy 0, policy_version 1594421 (0.0006) [2023-12-27 16:41:32,699][20347] Updated weights for policy 0, policy_version 1594431 (0.0007) [2023-12-27 16:41:32,759][20347] Updated weights for policy 0, policy_version 1594441 (0.0009) [2023-12-27 16:41:33,218][20381] Updated weights for policy 1, policy_version 1592927 (0.0006) [2023-12-27 16:41:33,273][20381] Updated weights for policy 1, policy_version 1592937 (0.0005) [2023-12-27 16:41:33,329][20381] Updated weights for policy 1, policy_version 1592947 (0.0005) [2023-12-27 16:41:33,338][20347] Updated weights for policy 0, policy_version 1594451 (0.0006) [2023-12-27 16:41:33,391][20347] Updated weights for policy 0, policy_version 1594461 (0.0006) [2023-12-27 16:41:33,447][20347] Updated weights for policy 0, policy_version 1594471 (0.0005) [2023-12-27 16:41:33,841][20381] Updated weights for policy 1, policy_version 1592957 (0.0005) [2023-12-27 16:41:33,892][20381] Updated weights for policy 1, policy_version 1592967 (0.0005) [2023-12-27 16:41:33,950][20381] Updated weights for policy 1, policy_version 1592977 (0.0007) [2023-12-27 16:41:34,031][20347] Updated weights for policy 0, policy_version 1594481 (0.0005) [2023-12-27 16:41:34,087][20347] Updated weights for policy 0, policy_version 1594491 (0.0007) [2023-12-27 16:41:34,149][20347] Updated weights for policy 0, policy_version 1594501 (0.0008) [2023-12-27 16:41:34,212][20347] Updated weights for policy 0, policy_version 1594511 (0.0007) [2023-12-27 16:41:34,627][20381] Updated weights for policy 1, policy_version 1592987 (0.0007) [2023-12-27 16:41:34,687][20381] Updated weights for policy 1, policy_version 1592997 (0.0008) [2023-12-27 16:41:34,735][20381] Updated weights for policy 1, policy_version 1593007 (0.0006) [2023-12-27 16:41:34,917][20347] Updated weights for policy 0, policy_version 1594521 (0.0008) [2023-12-27 16:41:34,968][20347] Updated weights for policy 0, policy_version 1594531 (0.0007) [2023-12-27 16:41:35,026][20347] Updated weights for policy 0, policy_version 1594541 (0.0007) [2023-12-27 16:41:35,336][20381] Updated weights for policy 1, policy_version 1593017 (0.0006) [2023-12-27 16:41:35,390][20381] Updated weights for policy 1, policy_version 1593027 (0.0005) [2023-12-27 16:41:35,438][20381] Updated weights for policy 1, policy_version 1593037 (0.0005) [2023-12-27 16:41:35,494][20381] Updated weights for policy 1, policy_version 1593047 (0.0005) [2023-12-27 16:41:35,623][20347] Updated weights for policy 0, policy_version 1594551 (0.0006) [2023-12-27 16:41:35,688][20347] Updated weights for policy 0, policy_version 1594561 (0.0005) [2023-12-27 16:41:35,755][20347] Updated weights for policy 0, policy_version 1594571 (0.0008) [2023-12-27 16:41:36,054][20381] Updated weights for policy 1, policy_version 1593057 (0.0007) [2023-12-27 16:41:36,102][20381] Updated weights for policy 1, policy_version 1593067 (0.0008) [2023-12-27 16:41:36,167][20381] Updated weights for policy 1, policy_version 1593077 (0.0008) [2023-12-27 16:41:36,385][20347] Updated weights for policy 0, policy_version 1594581 (0.0011) [2023-12-27 16:41:36,450][20347] Updated weights for policy 0, policy_version 1594591 (0.0010) [2023-12-27 16:41:36,509][20347] Updated weights for policy 0, policy_version 1594601 (0.0011) [2023-12-27 16:41:36,865][20381] Updated weights for policy 1, policy_version 1593087 (0.0007) [2023-12-27 16:41:36,917][20381] Updated weights for policy 1, policy_version 1593097 (0.0007) [2023-12-27 16:41:36,975][20381] Updated weights for policy 1, policy_version 1593107 (0.0008) [2023-12-27 16:41:37,193][20347] Updated weights for policy 0, policy_version 1594611 (0.0011) [2023-12-27 16:41:37,254][20347] Updated weights for policy 0, policy_version 1594621 (0.0010) [2023-12-27 16:41:37,310][20347] Updated weights for policy 0, policy_version 1594631 (0.0010) [2023-12-27 16:41:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 816177152. Throughput: 0: 10236.9, 1: 10288.7. Samples: 816147648. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:37,363][19170] Avg episode reward: [(0, '980.530'), (1, '275.370')] [2023-12-27 16:41:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001593112_407879680.pth... [2023-12-27 16:41:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001594640_408305664.pth... [2023-12-27 16:41:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001591896_407568384.pth [2023-12-27 16:41:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001593424_407994368.pth [2023-12-27 16:41:37,695][20381] Updated weights for policy 1, policy_version 1593117 (0.0008) [2023-12-27 16:41:37,751][20381] Updated weights for policy 1, policy_version 1593127 (0.0007) [2023-12-27 16:41:37,806][20381] Updated weights for policy 1, policy_version 1593137 (0.0007) [2023-12-27 16:41:37,985][20347] Updated weights for policy 0, policy_version 1594641 (0.0011) [2023-12-27 16:41:38,038][20347] Updated weights for policy 0, policy_version 1594651 (0.0010) [2023-12-27 16:41:38,095][20347] Updated weights for policy 0, policy_version 1594661 (0.0010) [2023-12-27 16:41:38,150][20347] Updated weights for policy 0, policy_version 1594671 (0.0010) [2023-12-27 16:41:38,512][20381] Updated weights for policy 1, policy_version 1593147 (0.0008) [2023-12-27 16:41:38,575][20381] Updated weights for policy 1, policy_version 1593157 (0.0011) [2023-12-27 16:41:38,631][20381] Updated weights for policy 1, policy_version 1593167 (0.0009) [2023-12-27 16:41:38,754][20347] Updated weights for policy 0, policy_version 1594681 (0.0010) [2023-12-27 16:41:38,814][20347] Updated weights for policy 0, policy_version 1594691 (0.0010) [2023-12-27 16:41:38,879][20347] Updated weights for policy 0, policy_version 1594701 (0.0010) [2023-12-27 16:41:39,229][20381] Updated weights for policy 1, policy_version 1593177 (0.0009) [2023-12-27 16:41:39,305][20381] Updated weights for policy 1, policy_version 1593187 (0.0007) [2023-12-27 16:41:39,370][20381] Updated weights for policy 1, policy_version 1593197 (0.0009) [2023-12-27 16:41:39,439][20381] Updated weights for policy 1, policy_version 1593207 (0.0008) [2023-12-27 16:41:39,530][20347] Updated weights for policy 0, policy_version 1594711 (0.0009) [2023-12-27 16:41:39,596][20347] Updated weights for policy 0, policy_version 1594721 (0.0008) [2023-12-27 16:41:39,653][20347] Updated weights for policy 0, policy_version 1594731 (0.0008) [2023-12-27 16:41:40,180][20381] Updated weights for policy 1, policy_version 1593217 (0.0010) [2023-12-27 16:41:40,239][20381] Updated weights for policy 1, policy_version 1593227 (0.0010) [2023-12-27 16:41:40,298][20381] Updated weights for policy 1, policy_version 1593237 (0.0010) [2023-12-27 16:41:40,356][20347] Updated weights for policy 0, policy_version 1594741 (0.0008) [2023-12-27 16:41:40,419][20347] Updated weights for policy 0, policy_version 1594751 (0.0008) [2023-12-27 16:41:40,482][20347] Updated weights for policy 0, policy_version 1594761 (0.0008) [2023-12-27 16:41:40,925][20381] Updated weights for policy 1, policy_version 1593247 (0.0007) [2023-12-27 16:41:40,985][20381] Updated weights for policy 1, policy_version 1593257 (0.0007) [2023-12-27 16:41:41,053][20381] Updated weights for policy 1, policy_version 1593267 (0.0008) [2023-12-27 16:41:41,137][20347] Updated weights for policy 0, policy_version 1594771 (0.0008) [2023-12-27 16:41:41,202][20347] Updated weights for policy 0, policy_version 1594781 (0.0007) [2023-12-27 16:41:41,266][20347] Updated weights for policy 0, policy_version 1594791 (0.0007) [2023-12-27 16:41:41,785][20381] Updated weights for policy 1, policy_version 1593277 (0.0007) [2023-12-27 16:41:41,837][20381] Updated weights for policy 1, policy_version 1593287 (0.0008) [2023-12-27 16:41:41,901][20381] Updated weights for policy 1, policy_version 1593297 (0.0008) [2023-12-27 16:41:42,007][20347] Updated weights for policy 0, policy_version 1594801 (0.0008) [2023-12-27 16:41:42,067][20347] Updated weights for policy 0, policy_version 1594811 (0.0006) [2023-12-27 16:41:42,131][20347] Updated weights for policy 0, policy_version 1594821 (0.0007) [2023-12-27 16:41:42,188][20347] Updated weights for policy 0, policy_version 1594831 (0.0007) [2023-12-27 16:41:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.2, 300 sec: 20549.4). Total num frames: 816283648. Throughput: 0: 10236.5, 1: 10357.9. Samples: 816272724. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:42,363][19170] Avg episode reward: [(0, '980.560'), (1, '327.100')] [2023-12-27 16:41:42,628][20381] Updated weights for policy 1, policy_version 1593307 (0.0007) [2023-12-27 16:41:42,686][20381] Updated weights for policy 1, policy_version 1593317 (0.0007) [2023-12-27 16:41:42,739][20381] Updated weights for policy 1, policy_version 1593327 (0.0008) [2023-12-27 16:41:42,896][20347] Updated weights for policy 0, policy_version 1594841 (0.0005) [2023-12-27 16:41:42,951][20347] Updated weights for policy 0, policy_version 1594851 (0.0009) [2023-12-27 16:41:43,002][20347] Updated weights for policy 0, policy_version 1594861 (0.0010) [2023-12-27 16:41:43,408][20381] Updated weights for policy 1, policy_version 1593337 (0.0007) [2023-12-27 16:41:43,456][20381] Updated weights for policy 1, policy_version 1593347 (0.0006) [2023-12-27 16:41:43,503][20381] Updated weights for policy 1, policy_version 1593357 (0.0008) [2023-12-27 16:41:43,550][20381] Updated weights for policy 1, policy_version 1593367 (0.0007) [2023-12-27 16:41:43,668][20347] Updated weights for policy 0, policy_version 1594871 (0.0010) [2023-12-27 16:41:43,713][20347] Updated weights for policy 0, policy_version 1594881 (0.0007) [2023-12-27 16:41:43,759][20347] Updated weights for policy 0, policy_version 1594891 (0.0005) [2023-12-27 16:41:44,229][20381] Updated weights for policy 1, policy_version 1593377 (0.0008) [2023-12-27 16:41:44,283][20381] Updated weights for policy 1, policy_version 1593387 (0.0008) [2023-12-27 16:41:44,338][20381] Updated weights for policy 1, policy_version 1593397 (0.0008) [2023-12-27 16:41:44,365][20347] Updated weights for policy 0, policy_version 1594901 (0.0008) [2023-12-27 16:41:44,414][20347] Updated weights for policy 0, policy_version 1594911 (0.0008) [2023-12-27 16:41:44,473][20347] Updated weights for policy 0, policy_version 1594921 (0.0007) [2023-12-27 16:41:45,111][20381] Updated weights for policy 1, policy_version 1593407 (0.0007) [2023-12-27 16:41:45,146][20347] Updated weights for policy 0, policy_version 1594931 (0.0010) [2023-12-27 16:41:45,170][20381] Updated weights for policy 1, policy_version 1593417 (0.0007) [2023-12-27 16:41:45,204][20347] Updated weights for policy 0, policy_version 1594941 (0.0011) [2023-12-27 16:41:45,229][20381] Updated weights for policy 1, policy_version 1593427 (0.0007) [2023-12-27 16:41:45,262][20347] Updated weights for policy 0, policy_version 1594951 (0.0011) [2023-12-27 16:41:45,873][20347] Updated weights for policy 0, policy_version 1594961 (0.0010) [2023-12-27 16:41:45,890][20381] Updated weights for policy 1, policy_version 1593437 (0.0008) [2023-12-27 16:41:45,918][20347] Updated weights for policy 0, policy_version 1594971 (0.0007) [2023-12-27 16:41:45,951][20381] Updated weights for policy 1, policy_version 1593447 (0.0010) [2023-12-27 16:41:45,969][20347] Updated weights for policy 0, policy_version 1594981 (0.0010) [2023-12-27 16:41:46,007][20381] Updated weights for policy 1, policy_version 1593457 (0.0008) [2023-12-27 16:41:46,025][20347] Updated weights for policy 0, policy_version 1594991 (0.0010) [2023-12-27 16:41:46,616][20381] Updated weights for policy 1, policy_version 1593467 (0.0009) [2023-12-27 16:41:46,679][20381] Updated weights for policy 1, policy_version 1593477 (0.0006) [2023-12-27 16:41:46,731][20381] Updated weights for policy 1, policy_version 1593487 (0.0006) [2023-12-27 16:41:46,736][20347] Updated weights for policy 0, policy_version 1595001 (0.0007) [2023-12-27 16:41:46,787][20347] Updated weights for policy 0, policy_version 1595011 (0.0005) [2023-12-27 16:41:46,836][20347] Updated weights for policy 0, policy_version 1595021 (0.0005) [2023-12-27 16:41:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 816381952. Throughput: 0: 10263.3, 1: 10353.8. Samples: 816395492. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:47,363][19170] Avg episode reward: [(0, '990.290'), (1, '456.250')] [2023-12-27 16:41:47,382][20381] Updated weights for policy 1, policy_version 1593497 (0.0006) [2023-12-27 16:41:47,439][20381] Updated weights for policy 1, policy_version 1593507 (0.0007) [2023-12-27 16:41:47,497][20381] Updated weights for policy 1, policy_version 1593517 (0.0007) [2023-12-27 16:41:47,506][20347] Updated weights for policy 0, policy_version 1595031 (0.0005) [2023-12-27 16:41:47,554][20381] Updated weights for policy 1, policy_version 1593527 (0.0008) [2023-12-27 16:41:47,567][20347] Updated weights for policy 0, policy_version 1595041 (0.0005) [2023-12-27 16:41:47,628][20347] Updated weights for policy 0, policy_version 1595051 (0.0005) [2023-12-27 16:41:48,189][20381] Updated weights for policy 1, policy_version 1593537 (0.0010) [2023-12-27 16:41:48,252][20381] Updated weights for policy 1, policy_version 1593547 (0.0010) [2023-12-27 16:41:48,263][20347] Updated weights for policy 0, policy_version 1595061 (0.0007) [2023-12-27 16:41:48,300][20381] Updated weights for policy 1, policy_version 1593557 (0.0006) [2023-12-27 16:41:48,308][20347] Updated weights for policy 0, policy_version 1595071 (0.0007) [2023-12-27 16:41:48,370][20347] Updated weights for policy 0, policy_version 1595081 (0.0007) [2023-12-27 16:41:48,940][20381] Updated weights for policy 1, policy_version 1593567 (0.0009) [2023-12-27 16:41:48,995][20381] Updated weights for policy 1, policy_version 1593577 (0.0011) [2023-12-27 16:41:49,033][20347] Updated weights for policy 0, policy_version 1595091 (0.0007) [2023-12-27 16:41:49,050][20381] Updated weights for policy 1, policy_version 1593587 (0.0011) [2023-12-27 16:41:49,086][20347] Updated weights for policy 0, policy_version 1595101 (0.0006) [2023-12-27 16:41:49,131][20347] Updated weights for policy 0, policy_version 1595111 (0.0009) [2023-12-27 16:41:49,751][20381] Updated weights for policy 1, policy_version 1593597 (0.0007) [2023-12-27 16:41:49,817][20381] Updated weights for policy 1, policy_version 1593607 (0.0007) [2023-12-27 16:41:49,826][20347] Updated weights for policy 0, policy_version 1595121 (0.0007) [2023-12-27 16:41:49,880][20381] Updated weights for policy 1, policy_version 1593617 (0.0008) [2023-12-27 16:41:49,894][20347] Updated weights for policy 0, policy_version 1595131 (0.0007) [2023-12-27 16:41:49,970][20347] Updated weights for policy 0, policy_version 1595141 (0.0008) [2023-12-27 16:41:50,038][20347] Updated weights for policy 0, policy_version 1595151 (0.0008) [2023-12-27 16:41:50,491][20381] Updated weights for policy 1, policy_version 1593627 (0.0008) [2023-12-27 16:41:50,544][20381] Updated weights for policy 1, policy_version 1593637 (0.0006) [2023-12-27 16:41:50,606][20381] Updated weights for policy 1, policy_version 1593647 (0.0007) [2023-12-27 16:41:50,781][20347] Updated weights for policy 0, policy_version 1595161 (0.0008) [2023-12-27 16:41:50,841][20347] Updated weights for policy 0, policy_version 1595171 (0.0008) [2023-12-27 16:41:50,897][20347] Updated weights for policy 0, policy_version 1595181 (0.0007) [2023-12-27 16:41:51,257][20381] Updated weights for policy 1, policy_version 1593657 (0.0006) [2023-12-27 16:41:51,312][20381] Updated weights for policy 1, policy_version 1593667 (0.0006) [2023-12-27 16:41:51,389][20381] Updated weights for policy 1, policy_version 1593677 (0.0008) [2023-12-27 16:41:51,443][20381] Updated weights for policy 1, policy_version 1593687 (0.0008) [2023-12-27 16:41:51,682][20347] Updated weights for policy 0, policy_version 1595191 (0.0008) [2023-12-27 16:41:51,746][20347] Updated weights for policy 0, policy_version 1595201 (0.0008) [2023-12-27 16:41:51,790][20347] Updated weights for policy 0, policy_version 1595211 (0.0007) [2023-12-27 16:41:52,104][20381] Updated weights for policy 1, policy_version 1593697 (0.0006) [2023-12-27 16:41:52,167][20381] Updated weights for policy 1, policy_version 1593707 (0.0007) [2023-12-27 16:41:52,220][20381] Updated weights for policy 1, policy_version 1593717 (0.0008) [2023-12-27 16:41:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 816488448. Throughput: 0: 10278.0, 1: 10368.9. Samples: 816457944. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '517.040')] [2023-12-27 16:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001593720_408035328.pth... [2023-12-27 16:41:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001592472_407715840.pth [2023-12-27 16:41:52,401][20347] Updated weights for policy 0, policy_version 1595221 (0.0008) [2023-12-27 16:41:52,464][20347] Updated weights for policy 0, policy_version 1595231 (0.0007) [2023-12-27 16:41:52,524][20347] Updated weights for policy 0, policy_version 1595241 (0.0009) [2023-12-27 16:41:52,570][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001595248_408461312.pth... [2023-12-27 16:41:52,575][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001594032_408150016.pth [2023-12-27 16:41:52,882][20381] Updated weights for policy 1, policy_version 1593727 (0.0008) [2023-12-27 16:41:52,932][20381] Updated weights for policy 1, policy_version 1593737 (0.0007) [2023-12-27 16:41:52,980][20381] Updated weights for policy 1, policy_version 1593747 (0.0007) [2023-12-27 16:41:53,201][20347] Updated weights for policy 0, policy_version 1595251 (0.0008) [2023-12-27 16:41:53,248][20347] Updated weights for policy 0, policy_version 1595261 (0.0010) [2023-12-27 16:41:53,297][20347] Updated weights for policy 0, policy_version 1595271 (0.0010) [2023-12-27 16:41:53,651][20381] Updated weights for policy 1, policy_version 1593757 (0.0007) [2023-12-27 16:41:53,714][20381] Updated weights for policy 1, policy_version 1593767 (0.0006) [2023-12-27 16:41:53,773][20381] Updated weights for policy 1, policy_version 1593777 (0.0006) [2023-12-27 16:41:53,945][20347] Updated weights for policy 0, policy_version 1595281 (0.0010) [2023-12-27 16:41:54,006][20347] Updated weights for policy 0, policy_version 1595291 (0.0010) [2023-12-27 16:41:54,068][20347] Updated weights for policy 0, policy_version 1595301 (0.0010) [2023-12-27 16:41:54,120][20347] Updated weights for policy 0, policy_version 1595311 (0.0010) [2023-12-27 16:41:54,398][20381] Updated weights for policy 1, policy_version 1593787 (0.0006) [2023-12-27 16:41:54,450][20381] Updated weights for policy 1, policy_version 1593797 (0.0010) [2023-12-27 16:41:54,496][20381] Updated weights for policy 1, policy_version 1593807 (0.0009) [2023-12-27 16:41:54,681][20347] Updated weights for policy 0, policy_version 1595321 (0.0010) [2023-12-27 16:41:54,743][20347] Updated weights for policy 0, policy_version 1595331 (0.0010) [2023-12-27 16:41:54,799][20347] Updated weights for policy 0, policy_version 1595341 (0.0007) [2023-12-27 16:41:55,195][20381] Updated weights for policy 1, policy_version 1593817 (0.0010) [2023-12-27 16:41:55,248][20381] Updated weights for policy 1, policy_version 1593827 (0.0009) [2023-12-27 16:41:55,307][20381] Updated weights for policy 1, policy_version 1593837 (0.0008) [2023-12-27 16:41:55,367][20381] Updated weights for policy 1, policy_version 1593847 (0.0008) [2023-12-27 16:41:55,424][20347] Updated weights for policy 0, policy_version 1595351 (0.0009) [2023-12-27 16:41:55,475][20347] Updated weights for policy 0, policy_version 1595361 (0.0010) [2023-12-27 16:41:55,528][20347] Updated weights for policy 0, policy_version 1595371 (0.0007) [2023-12-27 16:41:56,010][20381] Updated weights for policy 1, policy_version 1593857 (0.0010) [2023-12-27 16:41:56,057][20381] Updated weights for policy 1, policy_version 1593867 (0.0009) [2023-12-27 16:41:56,108][20381] Updated weights for policy 1, policy_version 1593877 (0.0010) [2023-12-27 16:41:56,197][20347] Updated weights for policy 0, policy_version 1595381 (0.0008) [2023-12-27 16:41:56,258][20347] Updated weights for policy 0, policy_version 1595391 (0.0010) [2023-12-27 16:41:56,316][20347] Updated weights for policy 0, policy_version 1595401 (0.0010) [2023-12-27 16:41:56,785][20381] Updated weights for policy 1, policy_version 1593887 (0.0011) [2023-12-27 16:41:56,841][20381] Updated weights for policy 1, policy_version 1593897 (0.0010) [2023-12-27 16:41:56,889][20381] Updated weights for policy 1, policy_version 1593907 (0.0010) [2023-12-27 16:41:56,925][20347] Updated weights for policy 0, policy_version 1595411 (0.0008) [2023-12-27 16:41:56,970][20347] Updated weights for policy 0, policy_version 1595421 (0.0010) [2023-12-27 16:41:57,014][20347] Updated weights for policy 0, policy_version 1595431 (0.0010) [2023-12-27 16:41:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 816594944. Throughput: 0: 10322.7, 1: 10399.0. Samples: 816582960. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:41:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '320.720')] [2023-12-27 16:41:57,549][20381] Updated weights for policy 1, policy_version 1593917 (0.0008) [2023-12-27 16:41:57,605][20381] Updated weights for policy 1, policy_version 1593927 (0.0008) [2023-12-27 16:41:57,650][20381] Updated weights for policy 1, policy_version 1593937 (0.0007) [2023-12-27 16:41:57,654][20347] Updated weights for policy 0, policy_version 1595441 (0.0010) [2023-12-27 16:41:57,704][20347] Updated weights for policy 0, policy_version 1595451 (0.0005) [2023-12-27 16:41:57,755][20347] Updated weights for policy 0, policy_version 1595461 (0.0009) [2023-12-27 16:41:57,809][20347] Updated weights for policy 0, policy_version 1595471 (0.0010) [2023-12-27 16:41:58,432][20381] Updated weights for policy 1, policy_version 1593947 (0.0007) [2023-12-27 16:41:58,497][20381] Updated weights for policy 1, policy_version 1593957 (0.0007) [2023-12-27 16:41:58,503][20347] Updated weights for policy 0, policy_version 1595481 (0.0011) [2023-12-27 16:41:58,555][20381] Updated weights for policy 1, policy_version 1593967 (0.0008) [2023-12-27 16:41:58,569][20347] Updated weights for policy 0, policy_version 1595491 (0.0011) [2023-12-27 16:41:58,630][20347] Updated weights for policy 0, policy_version 1595501 (0.0010) [2023-12-27 16:41:59,318][20381] Updated weights for policy 1, policy_version 1593977 (0.0006) [2023-12-27 16:41:59,383][20381] Updated weights for policy 1, policy_version 1593987 (0.0008) [2023-12-27 16:41:59,423][20347] Updated weights for policy 0, policy_version 1595511 (0.0008) [2023-12-27 16:41:59,437][20381] Updated weights for policy 1, policy_version 1593997 (0.0011) [2023-12-27 16:41:59,480][20347] Updated weights for policy 0, policy_version 1595521 (0.0007) [2023-12-27 16:41:59,487][20381] Updated weights for policy 1, policy_version 1594007 (0.0010) [2023-12-27 16:41:59,529][20347] Updated weights for policy 0, policy_version 1595531 (0.0008) [2023-12-27 16:42:00,178][20381] Updated weights for policy 1, policy_version 1594017 (0.0007) [2023-12-27 16:42:00,237][20381] Updated weights for policy 1, policy_version 1594027 (0.0006) [2023-12-27 16:42:00,254][20347] Updated weights for policy 0, policy_version 1595541 (0.0007) [2023-12-27 16:42:00,289][20381] Updated weights for policy 1, policy_version 1594037 (0.0006) [2023-12-27 16:42:00,308][20347] Updated weights for policy 0, policy_version 1595551 (0.0006) [2023-12-27 16:42:00,368][20347] Updated weights for policy 0, policy_version 1595561 (0.0008) [2023-12-27 16:42:00,967][20381] Updated weights for policy 1, policy_version 1594047 (0.0006) [2023-12-27 16:42:01,006][20347] Updated weights for policy 0, policy_version 1595571 (0.0009) [2023-12-27 16:42:01,019][20381] Updated weights for policy 1, policy_version 1594057 (0.0006) [2023-12-27 16:42:01,067][20347] Updated weights for policy 0, policy_version 1595581 (0.0007) [2023-12-27 16:42:01,082][20381] Updated weights for policy 1, policy_version 1594067 (0.0006) [2023-12-27 16:42:01,121][20347] Updated weights for policy 0, policy_version 1595591 (0.0008) [2023-12-27 16:42:01,769][20381] Updated weights for policy 1, policy_version 1594077 (0.0008) [2023-12-27 16:42:01,803][20347] Updated weights for policy 0, policy_version 1595601 (0.0011) [2023-12-27 16:42:01,836][20381] Updated weights for policy 1, policy_version 1594087 (0.0007) [2023-12-27 16:42:01,865][20347] Updated weights for policy 0, policy_version 1595611 (0.0006) [2023-12-27 16:42:01,889][20381] Updated weights for policy 1, policy_version 1594097 (0.0006) [2023-12-27 16:42:01,918][20347] Updated weights for policy 0, policy_version 1595621 (0.0007) [2023-12-27 16:42:01,978][20347] Updated weights for policy 0, policy_version 1595631 (0.0008) [2023-12-27 16:42:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 816693248. Throughput: 0: 10343.8, 1: 10346.3. Samples: 816704760. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:42:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '436.470')] [2023-12-27 16:42:02,545][20381] Updated weights for policy 1, policy_version 1594107 (0.0005) [2023-12-27 16:42:02,604][20381] Updated weights for policy 1, policy_version 1594117 (0.0005) [2023-12-27 16:42:02,668][20381] Updated weights for policy 1, policy_version 1594127 (0.0005) [2023-12-27 16:42:02,694][20347] Updated weights for policy 0, policy_version 1595641 (0.0010) [2023-12-27 16:42:02,742][20347] Updated weights for policy 0, policy_version 1595651 (0.0009) [2023-12-27 16:42:02,793][20347] Updated weights for policy 0, policy_version 1595661 (0.0005) [2023-12-27 16:42:03,246][20381] Updated weights for policy 1, policy_version 1594137 (0.0005) [2023-12-27 16:42:03,304][20381] Updated weights for policy 1, policy_version 1594147 (0.0008) [2023-12-27 16:42:03,322][20347] Updated weights for policy 0, policy_version 1595671 (0.0005) [2023-12-27 16:42:03,355][20381] Updated weights for policy 1, policy_version 1594157 (0.0008) [2023-12-27 16:42:03,374][20347] Updated weights for policy 0, policy_version 1595681 (0.0007) [2023-12-27 16:42:03,409][20381] Updated weights for policy 1, policy_version 1594167 (0.0011) [2023-12-27 16:42:03,428][20347] Updated weights for policy 0, policy_version 1595691 (0.0010) [2023-12-27 16:42:04,038][20347] Updated weights for policy 0, policy_version 1595701 (0.0008) [2023-12-27 16:42:04,053][20381] Updated weights for policy 1, policy_version 1594177 (0.0011) [2023-12-27 16:42:04,092][20347] Updated weights for policy 0, policy_version 1595711 (0.0005) [2023-12-27 16:42:04,108][20381] Updated weights for policy 1, policy_version 1594187 (0.0010) [2023-12-27 16:42:04,151][20347] Updated weights for policy 0, policy_version 1595721 (0.0010) [2023-12-27 16:42:04,165][20381] Updated weights for policy 1, policy_version 1594197 (0.0006) [2023-12-27 16:42:04,818][20347] Updated weights for policy 0, policy_version 1595731 (0.0010) [2023-12-27 16:42:04,824][20381] Updated weights for policy 1, policy_version 1594207 (0.0005) [2023-12-27 16:42:04,870][20347] Updated weights for policy 0, policy_version 1595741 (0.0010) [2023-12-27 16:42:04,878][20381] Updated weights for policy 1, policy_version 1594217 (0.0005) [2023-12-27 16:42:04,923][20347] Updated weights for policy 0, policy_version 1595751 (0.0010) [2023-12-27 16:42:04,934][20381] Updated weights for policy 1, policy_version 1594227 (0.0008) [2023-12-27 16:42:05,524][20381] Updated weights for policy 1, policy_version 1594237 (0.0008) [2023-12-27 16:42:05,531][20347] Updated weights for policy 0, policy_version 1595761 (0.0010) [2023-12-27 16:42:05,581][20381] Updated weights for policy 1, policy_version 1594247 (0.0005) [2023-12-27 16:42:05,593][20347] Updated weights for policy 0, policy_version 1595771 (0.0010) [2023-12-27 16:42:05,627][20381] Updated weights for policy 1, policy_version 1594257 (0.0005) [2023-12-27 16:42:05,643][20347] Updated weights for policy 0, policy_version 1595781 (0.0010) [2023-12-27 16:42:05,700][20347] Updated weights for policy 0, policy_version 1595791 (0.0010) [2023-12-27 16:42:06,236][20381] Updated weights for policy 1, policy_version 1594267 (0.0006) [2023-12-27 16:42:06,303][20381] Updated weights for policy 1, policy_version 1594277 (0.0006) [2023-12-27 16:42:06,338][20347] Updated weights for policy 0, policy_version 1595801 (0.0011) [2023-12-27 16:42:06,363][20381] Updated weights for policy 1, policy_version 1594287 (0.0006) [2023-12-27 16:42:06,398][20347] Updated weights for policy 0, policy_version 1595811 (0.0009) [2023-12-27 16:42:06,463][20347] Updated weights for policy 0, policy_version 1595821 (0.0011) [2023-12-27 16:42:06,989][20381] Updated weights for policy 1, policy_version 1594297 (0.0006) [2023-12-27 16:42:07,053][20381] Updated weights for policy 1, policy_version 1594307 (0.0010) [2023-12-27 16:42:07,109][20381] Updated weights for policy 1, policy_version 1594317 (0.0010) [2023-12-27 16:42:07,124][20347] Updated weights for policy 0, policy_version 1595831 (0.0010) [2023-12-27 16:42:07,161][20381] Updated weights for policy 1, policy_version 1594327 (0.0010) [2023-12-27 16:42:07,178][20347] Updated weights for policy 0, policy_version 1595841 (0.0005) [2023-12-27 16:42:07,227][20347] Updated weights for policy 0, policy_version 1595851 (0.0005) [2023-12-27 16:42:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 816807936. Throughput: 0: 10346.1, 1: 10418.3. Samples: 816769820. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:42:07,364][19170] Avg episode reward: [(0, '990.280'), (1, '513.860')] [2023-12-27 16:42:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001594328_408190976.pth... [2023-12-27 16:42:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001595856_408616960.pth... [2023-12-27 16:42:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001593112_407879680.pth [2023-12-27 16:42:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001594640_408305664.pth [2023-12-27 16:42:07,800][20381] Updated weights for policy 1, policy_version 1594337 (0.0011) [2023-12-27 16:42:07,861][20381] Updated weights for policy 1, policy_version 1594347 (0.0010) [2023-12-27 16:42:07,862][20347] Updated weights for policy 0, policy_version 1595861 (0.0007) [2023-12-27 16:42:07,915][20381] Updated weights for policy 1, policy_version 1594357 (0.0010) [2023-12-27 16:42:07,916][20347] Updated weights for policy 0, policy_version 1595871 (0.0010) [2023-12-27 16:42:07,964][20347] Updated weights for policy 0, policy_version 1595881 (0.0007) [2023-12-27 16:42:08,603][20381] Updated weights for policy 1, policy_version 1594367 (0.0007) [2023-12-27 16:42:08,640][20347] Updated weights for policy 0, policy_version 1595891 (0.0010) [2023-12-27 16:42:08,661][20381] Updated weights for policy 1, policy_version 1594377 (0.0006) [2023-12-27 16:42:08,697][20347] Updated weights for policy 0, policy_version 1595901 (0.0009) [2023-12-27 16:42:08,720][20381] Updated weights for policy 1, policy_version 1594387 (0.0008) [2023-12-27 16:42:08,759][20347] Updated weights for policy 0, policy_version 1595911 (0.0008) [2023-12-27 16:42:09,340][20347] Updated weights for policy 0, policy_version 1595921 (0.0010) [2023-12-27 16:42:09,409][20347] Updated weights for policy 0, policy_version 1595931 (0.0011) [2023-12-27 16:42:09,412][20381] Updated weights for policy 1, policy_version 1594397 (0.0007) [2023-12-27 16:42:09,474][20381] Updated weights for policy 1, policy_version 1594407 (0.0006) [2023-12-27 16:42:09,477][20347] Updated weights for policy 0, policy_version 1595941 (0.0008) [2023-12-27 16:42:09,533][20381] Updated weights for policy 1, policy_version 1594417 (0.0007) [2023-12-27 16:42:09,540][20347] Updated weights for policy 0, policy_version 1595951 (0.0008) [2023-12-27 16:42:10,213][20381] Updated weights for policy 1, policy_version 1594427 (0.0006) [2023-12-27 16:42:10,271][20381] Updated weights for policy 1, policy_version 1594437 (0.0009) [2023-12-27 16:42:10,273][20347] Updated weights for policy 0, policy_version 1595961 (0.0006) [2023-12-27 16:42:10,328][20347] Updated weights for policy 0, policy_version 1595971 (0.0007) [2023-12-27 16:42:10,331][20381] Updated weights for policy 1, policy_version 1594447 (0.0006) [2023-12-27 16:42:10,385][20347] Updated weights for policy 0, policy_version 1595981 (0.0006) [2023-12-27 16:42:10,997][20381] Updated weights for policy 1, policy_version 1594457 (0.0007) [2023-12-27 16:42:11,069][20347] Updated weights for policy 0, policy_version 1595991 (0.0007) [2023-12-27 16:42:11,077][20381] Updated weights for policy 1, policy_version 1594467 (0.0008) [2023-12-27 16:42:11,134][20347] Updated weights for policy 0, policy_version 1596001 (0.0008) [2023-12-27 16:42:11,141][20381] Updated weights for policy 1, policy_version 1594477 (0.0007) [2023-12-27 16:42:11,203][20347] Updated weights for policy 0, policy_version 1596011 (0.0008) [2023-12-27 16:42:11,206][20381] Updated weights for policy 1, policy_version 1594487 (0.0007) [2023-12-27 16:42:11,916][20347] Updated weights for policy 0, policy_version 1596021 (0.0008) [2023-12-27 16:42:11,968][20347] Updated weights for policy 0, policy_version 1596031 (0.0008) [2023-12-27 16:42:11,981][20381] Updated weights for policy 1, policy_version 1594497 (0.0007) [2023-12-27 16:42:12,026][20347] Updated weights for policy 0, policy_version 1596041 (0.0009) [2023-12-27 16:42:12,032][20381] Updated weights for policy 1, policy_version 1594507 (0.0007) [2023-12-27 16:42:12,092][20381] Updated weights for policy 1, policy_version 1594517 (0.0006) [2023-12-27 16:42:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 816906240. Throughput: 0: 10375.2, 1: 10426.8. Samples: 816895660. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:42:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '515.710')] [2023-12-27 16:42:12,721][20347] Updated weights for policy 0, policy_version 1596051 (0.0009) [2023-12-27 16:42:12,783][20347] Updated weights for policy 0, policy_version 1596061 (0.0006) [2023-12-27 16:42:12,829][20381] Updated weights for policy 1, policy_version 1594527 (0.0007) [2023-12-27 16:42:12,839][20347] Updated weights for policy 0, policy_version 1596071 (0.0006) [2023-12-27 16:42:12,886][20381] Updated weights for policy 1, policy_version 1594537 (0.0007) [2023-12-27 16:42:12,949][20381] Updated weights for policy 1, policy_version 1594547 (0.0009) [2023-12-27 16:42:13,491][20347] Updated weights for policy 0, policy_version 1596081 (0.0006) [2023-12-27 16:42:13,550][20347] Updated weights for policy 0, policy_version 1596091 (0.0005) [2023-12-27 16:42:13,611][20347] Updated weights for policy 0, policy_version 1596101 (0.0005) [2023-12-27 16:42:13,670][20347] Updated weights for policy 0, policy_version 1596111 (0.0005) [2023-12-27 16:42:13,671][20381] Updated weights for policy 1, policy_version 1594557 (0.0008) [2023-12-27 16:42:13,740][20381] Updated weights for policy 1, policy_version 1594567 (0.0008) [2023-12-27 16:42:13,801][20381] Updated weights for policy 1, policy_version 1594577 (0.0008) [2023-12-27 16:42:14,229][20347] Updated weights for policy 0, policy_version 1596121 (0.0005) [2023-12-27 16:42:14,294][20347] Updated weights for policy 0, policy_version 1596131 (0.0005) [2023-12-27 16:42:14,347][20347] Updated weights for policy 0, policy_version 1596141 (0.0005) [2023-12-27 16:42:14,553][20381] Updated weights for policy 1, policy_version 1594587 (0.0008) [2023-12-27 16:42:14,607][20381] Updated weights for policy 1, policy_version 1594597 (0.0008) [2023-12-27 16:42:14,652][20381] Updated weights for policy 1, policy_version 1594607 (0.0007) [2023-12-27 16:42:14,962][20347] Updated weights for policy 0, policy_version 1596151 (0.0007) [2023-12-27 16:42:15,022][20347] Updated weights for policy 0, policy_version 1596161 (0.0008) [2023-12-27 16:42:15,075][20347] Updated weights for policy 0, policy_version 1596171 (0.0008) [2023-12-27 16:42:15,419][20381] Updated weights for policy 1, policy_version 1594617 (0.0008) [2023-12-27 16:42:15,483][20381] Updated weights for policy 1, policy_version 1594627 (0.0007) [2023-12-27 16:42:15,540][20381] Updated weights for policy 1, policy_version 1594637 (0.0007) [2023-12-27 16:42:15,590][20381] Updated weights for policy 1, policy_version 1594647 (0.0009) [2023-12-27 16:42:15,744][20347] Updated weights for policy 0, policy_version 1596181 (0.0007) [2023-12-27 16:42:15,798][20347] Updated weights for policy 0, policy_version 1596191 (0.0007) [2023-12-27 16:42:15,856][20347] Updated weights for policy 0, policy_version 1596201 (0.0007) [2023-12-27 16:42:16,300][20381] Updated weights for policy 1, policy_version 1594657 (0.0007) [2023-12-27 16:42:16,372][20381] Updated weights for policy 1, policy_version 1594667 (0.0008) [2023-12-27 16:42:16,435][20381] Updated weights for policy 1, policy_version 1594677 (0.0007) [2023-12-27 16:42:16,532][20347] Updated weights for policy 0, policy_version 1596211 (0.0007) [2023-12-27 16:42:16,589][20347] Updated weights for policy 0, policy_version 1596221 (0.0007) [2023-12-27 16:42:16,638][20347] Updated weights for policy 0, policy_version 1596231 (0.0006) [2023-12-27 16:42:17,052][20381] Updated weights for policy 1, policy_version 1594687 (0.0008) [2023-12-27 16:42:17,107][20381] Updated weights for policy 1, policy_version 1594697 (0.0007) [2023-12-27 16:42:17,169][20381] Updated weights for policy 1, policy_version 1594707 (0.0008) [2023-12-27 16:42:17,234][20347] Updated weights for policy 0, policy_version 1596241 (0.0006) [2023-12-27 16:42:17,280][20347] Updated weights for policy 0, policy_version 1596251 (0.0008) [2023-12-27 16:42:17,332][20347] Updated weights for policy 0, policy_version 1596261 (0.0008) [2023-12-27 16:42:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 817004544. Throughput: 0: 10460.8, 1: 10277.9. Samples: 817016528. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-12-27 16:42:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '594.080')] [2023-12-27 16:42:17,384][20347] Updated weights for policy 0, policy_version 1596271 (0.0007) [2023-12-27 16:42:17,842][20381] Updated weights for policy 1, policy_version 1594717 (0.0007) [2023-12-27 16:42:17,896][20381] Updated weights for policy 1, policy_version 1594727 (0.0007) [2023-12-27 16:42:17,948][20381] Updated weights for policy 1, policy_version 1594737 (0.0008) [2023-12-27 16:42:18,076][20347] Updated weights for policy 0, policy_version 1596281 (0.0007) [2023-12-27 16:42:18,130][20347] Updated weights for policy 0, policy_version 1596291 (0.0010) [2023-12-27 16:42:18,178][20347] Updated weights for policy 0, policy_version 1596301 (0.0010) [2023-12-27 16:42:18,693][20381] Updated weights for policy 1, policy_version 1594747 (0.0007) [2023-12-27 16:42:18,744][20381] Updated weights for policy 1, policy_version 1594757 (0.0007) [2023-12-27 16:42:18,805][20381] Updated weights for policy 1, policy_version 1594767 (0.0008) [2023-12-27 16:42:18,811][20347] Updated weights for policy 0, policy_version 1596311 (0.0010) [2023-12-27 16:42:18,866][20347] Updated weights for policy 0, policy_version 1596321 (0.0008) [2023-12-27 16:42:18,917][20347] Updated weights for policy 0, policy_version 1596331 (0.0008) [2023-12-27 16:42:19,533][20381] Updated weights for policy 1, policy_version 1594777 (0.0007) [2023-12-27 16:42:19,587][20381] Updated weights for policy 1, policy_version 1594787 (0.0007) [2023-12-27 16:42:19,596][20347] Updated weights for policy 0, policy_version 1596341 (0.0009) [2023-12-27 16:42:19,645][20381] Updated weights for policy 1, policy_version 1594797 (0.0008) [2023-12-27 16:42:19,659][20347] Updated weights for policy 0, policy_version 1596351 (0.0010) [2023-12-27 16:42:19,702][20381] Updated weights for policy 1, policy_version 1594807 (0.0006) [2023-12-27 16:42:19,711][20347] Updated weights for policy 0, policy_version 1596361 (0.0010) [2023-12-27 16:42:20,403][20347] Updated weights for policy 0, policy_version 1596371 (0.0011) [2023-12-27 16:42:20,424][20381] Updated weights for policy 1, policy_version 1594817 (0.0006) [2023-12-27 16:42:20,458][20347] Updated weights for policy 0, policy_version 1596381 (0.0010) [2023-12-27 16:42:20,475][20381] Updated weights for policy 1, policy_version 1594827 (0.0006) [2023-12-27 16:42:20,504][20347] Updated weights for policy 0, policy_version 1596391 (0.0010) [2023-12-27 16:42:20,533][20381] Updated weights for policy 1, policy_version 1594837 (0.0007) [2023-12-27 16:42:21,222][20347] Updated weights for policy 0, policy_version 1596401 (0.0010) [2023-12-27 16:42:21,255][20381] Updated weights for policy 1, policy_version 1594847 (0.0006) [2023-12-27 16:42:21,287][20347] Updated weights for policy 0, policy_version 1596411 (0.0008) [2023-12-27 16:42:21,319][20381] Updated weights for policy 1, policy_version 1594857 (0.0007) [2023-12-27 16:42:21,343][20347] Updated weights for policy 0, policy_version 1596421 (0.0008) [2023-12-27 16:42:21,384][20381] Updated weights for policy 1, policy_version 1594867 (0.0009) [2023-12-27 16:42:21,416][20347] Updated weights for policy 0, policy_version 1596431 (0.0008) [2023-12-27 16:42:22,126][20381] Updated weights for policy 1, policy_version 1594877 (0.0008) [2023-12-27 16:42:22,183][20381] Updated weights for policy 1, policy_version 1594887 (0.0008) [2023-12-27 16:42:22,201][20347] Updated weights for policy 0, policy_version 1596441 (0.0006) [2023-12-27 16:42:22,230][20381] Updated weights for policy 1, policy_version 1594897 (0.0006) [2023-12-27 16:42:22,258][20347] Updated weights for policy 0, policy_version 1596451 (0.0006) [2023-12-27 16:42:22,322][20347] Updated weights for policy 0, policy_version 1596461 (0.0008) [2023-12-27 16:42:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 817111040. Throughput: 0: 10444.1, 1: 10198.7. Samples: 817076576. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '599.110')] [2023-12-27 16:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001594904_408338432.pth... [2023-12-27 16:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001596464_408772608.pth... [2023-12-27 16:42:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001593720_408035328.pth [2023-12-27 16:42:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001595248_408461312.pth [2023-12-27 16:42:22,932][20381] Updated weights for policy 1, policy_version 1594907 (0.0007) [2023-12-27 16:42:22,984][20347] Updated weights for policy 0, policy_version 1596471 (0.0008) [2023-12-27 16:42:22,995][20381] Updated weights for policy 1, policy_version 1594917 (0.0006) [2023-12-27 16:42:23,040][20347] Updated weights for policy 0, policy_version 1596481 (0.0007) [2023-12-27 16:42:23,054][20381] Updated weights for policy 1, policy_version 1594927 (0.0006) [2023-12-27 16:42:23,097][20347] Updated weights for policy 0, policy_version 1596491 (0.0006) [2023-12-27 16:42:23,664][20381] Updated weights for policy 1, policy_version 1594937 (0.0008) [2023-12-27 16:42:23,713][20381] Updated weights for policy 1, policy_version 1594947 (0.0005) [2023-12-27 16:42:23,742][20347] Updated weights for policy 0, policy_version 1596501 (0.0007) [2023-12-27 16:42:23,768][20381] Updated weights for policy 1, policy_version 1594957 (0.0007) [2023-12-27 16:42:23,801][20347] Updated weights for policy 0, policy_version 1596511 (0.0007) [2023-12-27 16:42:23,812][20381] Updated weights for policy 1, policy_version 1594967 (0.0005) [2023-12-27 16:42:23,863][20347] Updated weights for policy 0, policy_version 1596521 (0.0005) [2023-12-27 16:42:24,431][20381] Updated weights for policy 1, policy_version 1594977 (0.0007) [2023-12-27 16:42:24,488][20347] Updated weights for policy 0, policy_version 1596531 (0.0005) [2023-12-27 16:42:24,494][20381] Updated weights for policy 1, policy_version 1594987 (0.0006) [2023-12-27 16:42:24,537][20347] Updated weights for policy 0, policy_version 1596541 (0.0005) [2023-12-27 16:42:24,557][20381] Updated weights for policy 1, policy_version 1594997 (0.0007) [2023-12-27 16:42:24,587][20347] Updated weights for policy 0, policy_version 1596551 (0.0006) [2023-12-27 16:42:25,190][20381] Updated weights for policy 1, policy_version 1595007 (0.0008) [2023-12-27 16:42:25,241][20381] Updated weights for policy 1, policy_version 1595017 (0.0006) [2023-12-27 16:42:25,258][20347] Updated weights for policy 0, policy_version 1596561 (0.0007) [2023-12-27 16:42:25,291][20381] Updated weights for policy 1, policy_version 1595027 (0.0005) [2023-12-27 16:42:25,312][20347] Updated weights for policy 0, policy_version 1596571 (0.0008) [2023-12-27 16:42:25,372][20347] Updated weights for policy 0, policy_version 1596581 (0.0008) [2023-12-27 16:42:25,429][20347] Updated weights for policy 0, policy_version 1596591 (0.0007) [2023-12-27 16:42:25,984][20381] Updated weights for policy 1, policy_version 1595037 (0.0007) [2023-12-27 16:42:26,027][20347] Updated weights for policy 0, policy_version 1596601 (0.0010) [2023-12-27 16:42:26,036][20381] Updated weights for policy 1, policy_version 1595047 (0.0007) [2023-12-27 16:42:26,081][20347] Updated weights for policy 0, policy_version 1596611 (0.0010) [2023-12-27 16:42:26,087][20381] Updated weights for policy 1, policy_version 1595057 (0.0005) [2023-12-27 16:42:26,135][20347] Updated weights for policy 0, policy_version 1596621 (0.0011) [2023-12-27 16:42:26,740][20381] Updated weights for policy 1, policy_version 1595067 (0.0006) [2023-12-27 16:42:26,753][20347] Updated weights for policy 0, policy_version 1596631 (0.0010) [2023-12-27 16:42:26,791][20381] Updated weights for policy 1, policy_version 1595077 (0.0006) [2023-12-27 16:42:26,805][20347] Updated weights for policy 0, policy_version 1596641 (0.0010) [2023-12-27 16:42:26,850][20381] Updated weights for policy 1, policy_version 1595087 (0.0006) [2023-12-27 16:42:26,873][20347] Updated weights for policy 0, policy_version 1596651 (0.0010) [2023-12-27 16:42:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 817209344. Throughput: 0: 10422.0, 1: 10180.1. Samples: 817199816. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:27,363][19170] Avg episode reward: [(0, '980.450'), (1, '530.640')] [2023-12-27 16:42:27,453][20381] Updated weights for policy 1, policy_version 1595097 (0.0006) [2023-12-27 16:42:27,508][20381] Updated weights for policy 1, policy_version 1595107 (0.0007) [2023-12-27 16:42:27,532][20347] Updated weights for policy 0, policy_version 1596661 (0.0010) [2023-12-27 16:42:27,556][20381] Updated weights for policy 1, policy_version 1595117 (0.0005) [2023-12-27 16:42:27,580][20347] Updated weights for policy 0, policy_version 1596671 (0.0010) [2023-12-27 16:42:27,614][20381] Updated weights for policy 1, policy_version 1595127 (0.0005) [2023-12-27 16:42:27,631][20347] Updated weights for policy 0, policy_version 1596681 (0.0010) [2023-12-27 16:42:28,212][20381] Updated weights for policy 1, policy_version 1595137 (0.0006) [2023-12-27 16:42:28,262][20381] Updated weights for policy 1, policy_version 1595147 (0.0007) [2023-12-27 16:42:28,305][20347] Updated weights for policy 0, policy_version 1596691 (0.0010) [2023-12-27 16:42:28,317][20381] Updated weights for policy 1, policy_version 1595157 (0.0005) [2023-12-27 16:42:28,366][20347] Updated weights for policy 0, policy_version 1596701 (0.0010) [2023-12-27 16:42:28,421][20347] Updated weights for policy 0, policy_version 1596711 (0.0010) [2023-12-27 16:42:28,917][20381] Updated weights for policy 1, policy_version 1595167 (0.0008) [2023-12-27 16:42:28,980][20381] Updated weights for policy 1, policy_version 1595177 (0.0008) [2023-12-27 16:42:29,049][20381] Updated weights for policy 1, policy_version 1595187 (0.0008) [2023-12-27 16:42:29,099][20347] Updated weights for policy 0, policy_version 1596721 (0.0011) [2023-12-27 16:42:29,163][20347] Updated weights for policy 0, policy_version 1596731 (0.0010) [2023-12-27 16:42:29,226][20347] Updated weights for policy 0, policy_version 1596741 (0.0010) [2023-12-27 16:42:29,290][20347] Updated weights for policy 0, policy_version 1596751 (0.0009) [2023-12-27 16:42:29,650][20381] Updated weights for policy 1, policy_version 1595197 (0.0008) [2023-12-27 16:42:29,704][20381] Updated weights for policy 1, policy_version 1595207 (0.0007) [2023-12-27 16:42:29,757][20381] Updated weights for policy 1, policy_version 1595217 (0.0007) [2023-12-27 16:42:29,926][20347] Updated weights for policy 0, policy_version 1596761 (0.0011) [2023-12-27 16:42:29,991][20347] Updated weights for policy 0, policy_version 1596771 (0.0010) [2023-12-27 16:42:30,012][20291] KL-divergence is very high: 110.2172 [2023-12-27 16:42:30,059][20347] Updated weights for policy 0, policy_version 1596781 (0.0010) [2023-12-27 16:42:30,395][20381] Updated weights for policy 1, policy_version 1595227 (0.0007) [2023-12-27 16:42:30,460][20381] Updated weights for policy 1, policy_version 1595237 (0.0006) [2023-12-27 16:42:30,524][20381] Updated weights for policy 1, policy_version 1595247 (0.0006) [2023-12-27 16:42:30,730][20347] Updated weights for policy 0, policy_version 1596791 (0.0008) [2023-12-27 16:42:30,784][20347] Updated weights for policy 0, policy_version 1596801 (0.0005) [2023-12-27 16:42:30,841][20347] Updated weights for policy 0, policy_version 1596811 (0.0007) [2023-12-27 16:42:31,214][20381] Updated weights for policy 1, policy_version 1595257 (0.0008) [2023-12-27 16:42:31,273][20381] Updated weights for policy 1, policy_version 1595267 (0.0008) [2023-12-27 16:42:31,329][20381] Updated weights for policy 1, policy_version 1595277 (0.0008) [2023-12-27 16:42:31,396][20381] Updated weights for policy 1, policy_version 1595287 (0.0009) [2023-12-27 16:42:31,429][20347] Updated weights for policy 0, policy_version 1596821 (0.0008) [2023-12-27 16:42:31,485][20347] Updated weights for policy 0, policy_version 1596831 (0.0010) [2023-12-27 16:42:31,540][20347] Updated weights for policy 0, policy_version 1596841 (0.0010) [2023-12-27 16:42:32,106][20381] Updated weights for policy 1, policy_version 1595297 (0.0011) [2023-12-27 16:42:32,162][20381] Updated weights for policy 1, policy_version 1595307 (0.0010) [2023-12-27 16:42:32,192][20347] Updated weights for policy 0, policy_version 1596851 (0.0010) [2023-12-27 16:42:32,222][20381] Updated weights for policy 1, policy_version 1595317 (0.0007) [2023-12-27 16:42:32,249][20347] Updated weights for policy 0, policy_version 1596861 (0.0007) [2023-12-27 16:42:32,314][20347] Updated weights for policy 0, policy_version 1596871 (0.0006) [2023-12-27 16:42:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 817315840. Throughput: 0: 10446.0, 1: 10265.0. Samples: 817327488. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:32,363][19170] Avg episode reward: [(0, '980.420'), (1, '601.930')] [2023-12-27 16:42:32,934][20347] Updated weights for policy 0, policy_version 1596881 (0.0007) [2023-12-27 16:42:32,963][20381] Updated weights for policy 1, policy_version 1595327 (0.0006) [2023-12-27 16:42:32,979][20347] Updated weights for policy 0, policy_version 1596891 (0.0006) [2023-12-27 16:42:33,016][20381] Updated weights for policy 1, policy_version 1595337 (0.0007) [2023-12-27 16:42:33,025][20347] Updated weights for policy 0, policy_version 1596901 (0.0005) [2023-12-27 16:42:33,069][20381] Updated weights for policy 1, policy_version 1595347 (0.0007) [2023-12-27 16:42:33,078][20347] Updated weights for policy 0, policy_version 1596911 (0.0006) [2023-12-27 16:42:33,714][20347] Updated weights for policy 0, policy_version 1596921 (0.0007) [2023-12-27 16:42:33,761][20347] Updated weights for policy 0, policy_version 1596931 (0.0007) [2023-12-27 16:42:33,777][20381] Updated weights for policy 1, policy_version 1595357 (0.0007) [2023-12-27 16:42:33,818][20347] Updated weights for policy 0, policy_version 1596941 (0.0005) [2023-12-27 16:42:33,830][20381] Updated weights for policy 1, policy_version 1595367 (0.0007) [2023-12-27 16:42:33,880][20381] Updated weights for policy 1, policy_version 1595377 (0.0008) [2023-12-27 16:42:34,527][20347] Updated weights for policy 0, policy_version 1596951 (0.0006) [2023-12-27 16:42:34,545][20381] Updated weights for policy 1, policy_version 1595387 (0.0006) [2023-12-27 16:42:34,589][20347] Updated weights for policy 0, policy_version 1596961 (0.0006) [2023-12-27 16:42:34,604][20381] Updated weights for policy 1, policy_version 1595397 (0.0006) [2023-12-27 16:42:34,648][20347] Updated weights for policy 0, policy_version 1596971 (0.0006) [2023-12-27 16:42:34,691][20381] Updated weights for policy 1, policy_version 1595407 (0.0005) [2023-12-27 16:42:35,260][20381] Updated weights for policy 1, policy_version 1595417 (0.0005) [2023-12-27 16:42:35,315][20381] Updated weights for policy 1, policy_version 1595427 (0.0008) [2023-12-27 16:42:35,320][20347] Updated weights for policy 0, policy_version 1596981 (0.0008) [2023-12-27 16:42:35,366][20381] Updated weights for policy 1, policy_version 1595437 (0.0008) [2023-12-27 16:42:35,375][20347] Updated weights for policy 0, policy_version 1596991 (0.0005) [2023-12-27 16:42:35,421][20347] Updated weights for policy 0, policy_version 1597001 (0.0007) [2023-12-27 16:42:35,423][20381] Updated weights for policy 1, policy_version 1595447 (0.0008) [2023-12-27 16:42:36,096][20347] Updated weights for policy 0, policy_version 1597011 (0.0007) [2023-12-27 16:42:36,123][20381] Updated weights for policy 1, policy_version 1595457 (0.0009) [2023-12-27 16:42:36,161][20347] Updated weights for policy 0, policy_version 1597021 (0.0007) [2023-12-27 16:42:36,185][20381] Updated weights for policy 1, policy_version 1595467 (0.0010) [2023-12-27 16:42:36,217][20347] Updated weights for policy 0, policy_version 1597031 (0.0006) [2023-12-27 16:42:36,252][20381] Updated weights for policy 1, policy_version 1595477 (0.0011) [2023-12-27 16:42:36,850][20347] Updated weights for policy 0, policy_version 1597041 (0.0008) [2023-12-27 16:42:36,915][20347] Updated weights for policy 0, policy_version 1597051 (0.0005) [2023-12-27 16:42:36,947][20381] Updated weights for policy 1, policy_version 1595487 (0.0011) [2023-12-27 16:42:36,970][20347] Updated weights for policy 0, policy_version 1597061 (0.0007) [2023-12-27 16:42:37,003][20381] Updated weights for policy 1, policy_version 1595497 (0.0011) [2023-12-27 16:42:37,023][20347] Updated weights for policy 0, policy_version 1597071 (0.0007) [2023-12-27 16:42:37,067][20381] Updated weights for policy 1, policy_version 1595507 (0.0006) [2023-12-27 16:42:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 817422336. Throughput: 0: 10472.3, 1: 10251.4. Samples: 817390508. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:37,363][19170] Avg episode reward: [(0, '970.730'), (1, '364.560')] [2023-12-27 16:42:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001595512_408494080.pth... [2023-12-27 16:42:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001597072_408928256.pth... [2023-12-27 16:42:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001594328_408190976.pth [2023-12-27 16:42:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001595856_408616960.pth [2023-12-27 16:42:37,644][20347] Updated weights for policy 0, policy_version 1597081 (0.0008) [2023-12-27 16:42:37,699][20347] Updated weights for policy 0, policy_version 1597091 (0.0008) [2023-12-27 16:42:37,703][20381] Updated weights for policy 1, policy_version 1595517 (0.0006) [2023-12-27 16:42:37,760][20347] Updated weights for policy 0, policy_version 1597101 (0.0006) [2023-12-27 16:42:37,760][20381] Updated weights for policy 1, policy_version 1595527 (0.0008) [2023-12-27 16:42:37,816][20381] Updated weights for policy 1, policy_version 1595537 (0.0005) [2023-12-27 16:42:38,370][20347] Updated weights for policy 0, policy_version 1597111 (0.0007) [2023-12-27 16:42:38,419][20347] Updated weights for policy 0, policy_version 1597121 (0.0008) [2023-12-27 16:42:38,457][20381] Updated weights for policy 1, policy_version 1595547 (0.0007) [2023-12-27 16:42:38,475][20347] Updated weights for policy 0, policy_version 1597131 (0.0007) [2023-12-27 16:42:38,514][20381] Updated weights for policy 1, policy_version 1595557 (0.0010) [2023-12-27 16:42:38,578][20381] Updated weights for policy 1, policy_version 1595567 (0.0005) [2023-12-27 16:42:39,141][20347] Updated weights for policy 0, policy_version 1597141 (0.0009) [2023-12-27 16:42:39,187][20347] Updated weights for policy 0, policy_version 1597151 (0.0006) [2023-12-27 16:42:39,221][20381] Updated weights for policy 1, policy_version 1595577 (0.0006) [2023-12-27 16:42:39,246][20347] Updated weights for policy 0, policy_version 1597161 (0.0008) [2023-12-27 16:42:39,283][20381] Updated weights for policy 1, policy_version 1595587 (0.0007) [2023-12-27 16:42:39,349][20381] Updated weights for policy 1, policy_version 1595597 (0.0008) [2023-12-27 16:42:39,411][20381] Updated weights for policy 1, policy_version 1595607 (0.0012) [2023-12-27 16:42:40,006][20347] Updated weights for policy 0, policy_version 1597171 (0.0007) [2023-12-27 16:42:40,065][20347] Updated weights for policy 0, policy_version 1597181 (0.0005) [2023-12-27 16:42:40,096][20381] Updated weights for policy 1, policy_version 1595617 (0.0009) [2023-12-27 16:42:40,124][20347] Updated weights for policy 0, policy_version 1597191 (0.0006) [2023-12-27 16:42:40,150][20381] Updated weights for policy 1, policy_version 1595627 (0.0008) [2023-12-27 16:42:40,198][20381] Updated weights for policy 1, policy_version 1595637 (0.0008) [2023-12-27 16:42:40,709][20347] Updated weights for policy 0, policy_version 1597201 (0.0005) [2023-12-27 16:42:40,766][20347] Updated weights for policy 0, policy_version 1597211 (0.0005) [2023-12-27 16:42:40,815][20347] Updated weights for policy 0, policy_version 1597221 (0.0005) [2023-12-27 16:42:40,866][20347] Updated weights for policy 0, policy_version 1597231 (0.0007) [2023-12-27 16:42:40,892][20381] Updated weights for policy 1, policy_version 1595647 (0.0007) [2023-12-27 16:42:40,943][20381] Updated weights for policy 1, policy_version 1595657 (0.0007) [2023-12-27 16:42:40,999][20381] Updated weights for policy 1, policy_version 1595667 (0.0008) [2023-12-27 16:42:41,558][20347] Updated weights for policy 0, policy_version 1597241 (0.0007) [2023-12-27 16:42:41,626][20347] Updated weights for policy 0, policy_version 1597251 (0.0008) [2023-12-27 16:42:41,690][20347] Updated weights for policy 0, policy_version 1597261 (0.0007) [2023-12-27 16:42:41,693][20381] Updated weights for policy 1, policy_version 1595677 (0.0009) [2023-12-27 16:42:41,765][20381] Updated weights for policy 1, policy_version 1595687 (0.0008) [2023-12-27 16:42:41,820][20381] Updated weights for policy 1, policy_version 1595697 (0.0011) [2023-12-27 16:42:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 817520640. Throughput: 0: 10498.8, 1: 10210.1. Samples: 817514856. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:42,363][19170] Avg episode reward: [(0, '980.530'), (1, '490.710')] [2023-12-27 16:42:42,461][20347] Updated weights for policy 0, policy_version 1597271 (0.0010) [2023-12-27 16:42:42,516][20347] Updated weights for policy 0, policy_version 1597281 (0.0008) [2023-12-27 16:42:42,547][20381] Updated weights for policy 1, policy_version 1595707 (0.0011) [2023-12-27 16:42:42,583][20347] Updated weights for policy 0, policy_version 1597291 (0.0006) [2023-12-27 16:42:42,609][20381] Updated weights for policy 1, policy_version 1595717 (0.0011) [2023-12-27 16:42:42,672][20381] Updated weights for policy 1, policy_version 1595727 (0.0011) [2023-12-27 16:42:43,197][20347] Updated weights for policy 0, policy_version 1597301 (0.0006) [2023-12-27 16:42:43,253][20347] Updated weights for policy 0, policy_version 1597311 (0.0005) [2023-12-27 16:42:43,309][20347] Updated weights for policy 0, policy_version 1597321 (0.0005) [2023-12-27 16:42:43,326][20381] Updated weights for policy 1, policy_version 1595737 (0.0011) [2023-12-27 16:42:43,380][20381] Updated weights for policy 1, policy_version 1595747 (0.0010) [2023-12-27 16:42:43,435][20381] Updated weights for policy 1, policy_version 1595757 (0.0010) [2023-12-27 16:42:43,495][20381] Updated weights for policy 1, policy_version 1595767 (0.0010) [2023-12-27 16:42:43,875][20347] Updated weights for policy 0, policy_version 1597331 (0.0005) [2023-12-27 16:42:43,931][20347] Updated weights for policy 0, policy_version 1597341 (0.0005) [2023-12-27 16:42:43,989][20347] Updated weights for policy 0, policy_version 1597351 (0.0005) [2023-12-27 16:42:44,153][20381] Updated weights for policy 1, policy_version 1595777 (0.0008) [2023-12-27 16:42:44,219][20381] Updated weights for policy 1, policy_version 1595787 (0.0007) [2023-12-27 16:42:44,268][20381] Updated weights for policy 1, policy_version 1595797 (0.0009) [2023-12-27 16:42:44,586][20347] Updated weights for policy 0, policy_version 1597361 (0.0007) [2023-12-27 16:42:44,647][20347] Updated weights for policy 0, policy_version 1597371 (0.0006) [2023-12-27 16:42:44,705][20347] Updated weights for policy 0, policy_version 1597381 (0.0008) [2023-12-27 16:42:44,766][20347] Updated weights for policy 0, policy_version 1597391 (0.0008) [2023-12-27 16:42:45,035][20381] Updated weights for policy 1, policy_version 1595807 (0.0008) [2023-12-27 16:42:45,096][20381] Updated weights for policy 1, policy_version 1595817 (0.0008) [2023-12-27 16:42:45,158][20381] Updated weights for policy 1, policy_version 1595827 (0.0008) [2023-12-27 16:42:45,448][20347] Updated weights for policy 0, policy_version 1597401 (0.0008) [2023-12-27 16:42:45,506][20347] Updated weights for policy 0, policy_version 1597411 (0.0007) [2023-12-27 16:42:45,565][20347] Updated weights for policy 0, policy_version 1597421 (0.0008) [2023-12-27 16:42:45,900][20381] Updated weights for policy 1, policy_version 1595837 (0.0007) [2023-12-27 16:42:45,963][20381] Updated weights for policy 1, policy_version 1595847 (0.0007) [2023-12-27 16:42:46,018][20381] Updated weights for policy 1, policy_version 1595857 (0.0008) [2023-12-27 16:42:46,305][20347] Updated weights for policy 0, policy_version 1597431 (0.0008) [2023-12-27 16:42:46,351][20347] Updated weights for policy 0, policy_version 1597441 (0.0009) [2023-12-27 16:42:46,407][20347] Updated weights for policy 0, policy_version 1597451 (0.0009) [2023-12-27 16:42:46,620][20381] Updated weights for policy 1, policy_version 1595867 (0.0007) [2023-12-27 16:42:46,674][20381] Updated weights for policy 1, policy_version 1595877 (0.0005) [2023-12-27 16:42:46,730][20381] Updated weights for policy 1, policy_version 1595887 (0.0005) [2023-12-27 16:42:47,200][20347] Updated weights for policy 0, policy_version 1597461 (0.0008) [2023-12-27 16:42:47,247][20347] Updated weights for policy 0, policy_version 1597471 (0.0007) [2023-12-27 16:42:47,283][20381] Updated weights for policy 1, policy_version 1595897 (0.0006) [2023-12-27 16:42:47,295][20347] Updated weights for policy 0, policy_version 1597481 (0.0008) [2023-12-27 16:42:47,340][20381] Updated weights for policy 1, policy_version 1595907 (0.0006) [2023-12-27 16:42:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 817627136. Throughput: 0: 10479.9, 1: 10247.3. Samples: 817637484. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:47,363][19170] Avg episode reward: [(0, '990.390'), (1, '422.530')] [2023-12-27 16:42:47,399][20381] Updated weights for policy 1, policy_version 1595917 (0.0008) [2023-12-27 16:42:47,456][20381] Updated weights for policy 1, policy_version 1595927 (0.0007) [2023-12-27 16:42:48,008][20347] Updated weights for policy 0, policy_version 1597491 (0.0006) [2023-12-27 16:42:48,064][20347] Updated weights for policy 0, policy_version 1597501 (0.0007) [2023-12-27 16:42:48,101][20381] Updated weights for policy 1, policy_version 1595937 (0.0007) [2023-12-27 16:42:48,117][20347] Updated weights for policy 0, policy_version 1597511 (0.0005) [2023-12-27 16:42:48,158][20381] Updated weights for policy 1, policy_version 1595947 (0.0007) [2023-12-27 16:42:48,218][20381] Updated weights for policy 1, policy_version 1595957 (0.0008) [2023-12-27 16:42:48,792][20347] Updated weights for policy 0, policy_version 1597521 (0.0006) [2023-12-27 16:42:48,848][20347] Updated weights for policy 0, policy_version 1597531 (0.0008) [2023-12-27 16:42:48,912][20347] Updated weights for policy 0, policy_version 1597541 (0.0007) [2023-12-27 16:42:48,929][20381] Updated weights for policy 1, policy_version 1595967 (0.0008) [2023-12-27 16:42:48,965][20347] Updated weights for policy 0, policy_version 1597551 (0.0005) [2023-12-27 16:42:48,985][20381] Updated weights for policy 1, policy_version 1595977 (0.0007) [2023-12-27 16:42:49,043][20381] Updated weights for policy 1, policy_version 1595987 (0.0007) [2023-12-27 16:42:49,576][20347] Updated weights for policy 0, policy_version 1597561 (0.0008) [2023-12-27 16:42:49,636][20347] Updated weights for policy 0, policy_version 1597571 (0.0007) [2023-12-27 16:42:49,694][20347] Updated weights for policy 0, policy_version 1597581 (0.0009) [2023-12-27 16:42:49,770][20381] Updated weights for policy 1, policy_version 1595997 (0.0006) [2023-12-27 16:42:49,830][20381] Updated weights for policy 1, policy_version 1596007 (0.0006) [2023-12-27 16:42:49,887][20381] Updated weights for policy 1, policy_version 1596017 (0.0008) [2023-12-27 16:42:50,466][20347] Updated weights for policy 0, policy_version 1597591 (0.0008) [2023-12-27 16:42:50,524][20347] Updated weights for policy 0, policy_version 1597601 (0.0007) [2023-12-27 16:42:50,564][20381] Updated weights for policy 1, policy_version 1596027 (0.0007) [2023-12-27 16:42:50,585][20347] Updated weights for policy 0, policy_version 1597611 (0.0007) [2023-12-27 16:42:50,626][20381] Updated weights for policy 1, policy_version 1596037 (0.0008) [2023-12-27 16:42:50,679][20381] Updated weights for policy 1, policy_version 1596047 (0.0008) [2023-12-27 16:42:51,364][20381] Updated weights for policy 1, policy_version 1596057 (0.0007) [2023-12-27 16:42:51,378][20347] Updated weights for policy 0, policy_version 1597621 (0.0008) [2023-12-27 16:42:51,422][20381] Updated weights for policy 1, policy_version 1596067 (0.0007) [2023-12-27 16:42:51,439][20347] Updated weights for policy 0, policy_version 1597631 (0.0009) [2023-12-27 16:42:51,474][20381] Updated weights for policy 1, policy_version 1596077 (0.0007) [2023-12-27 16:42:51,493][20347] Updated weights for policy 0, policy_version 1597641 (0.0009) [2023-12-27 16:42:51,525][20381] Updated weights for policy 1, policy_version 1596087 (0.0007) [2023-12-27 16:42:52,204][20347] Updated weights for policy 0, policy_version 1597651 (0.0009) [2023-12-27 16:42:52,261][20347] Updated weights for policy 0, policy_version 1597661 (0.0008) [2023-12-27 16:42:52,296][20381] Updated weights for policy 1, policy_version 1596097 (0.0007) [2023-12-27 16:42:52,318][20347] Updated weights for policy 0, policy_version 1597671 (0.0006) [2023-12-27 16:42:52,357][20381] Updated weights for policy 1, policy_version 1596107 (0.0008) [2023-12-27 16:42:52,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20521.7). Total num frames: 817717248. Throughput: 0: 10429.5, 1: 10189.2. Samples: 817697660. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:52,363][19170] Avg episode reward: [(0, '990.390'), (1, '309.300')] [2023-12-27 16:42:52,379][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001597680_409083904.pth... [2023-12-27 16:42:52,384][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001596464_408772608.pth [2023-12-27 16:42:52,419][20381] Updated weights for policy 1, policy_version 1596117 (0.0008) [2023-12-27 16:42:52,434][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001596120_408649728.pth... [2023-12-27 16:42:52,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001594904_408338432.pth [2023-12-27 16:42:53,057][20347] Updated weights for policy 0, policy_version 1597681 (0.0008) [2023-12-27 16:42:53,114][20347] Updated weights for policy 0, policy_version 1597691 (0.0007) [2023-12-27 16:42:53,151][20381] Updated weights for policy 1, policy_version 1596127 (0.0006) [2023-12-27 16:42:53,165][20347] Updated weights for policy 0, policy_version 1597701 (0.0008) [2023-12-27 16:42:53,211][20381] Updated weights for policy 1, policy_version 1596137 (0.0005) [2023-12-27 16:42:53,216][20347] Updated weights for policy 0, policy_version 1597711 (0.0008) [2023-12-27 16:42:53,268][20381] Updated weights for policy 1, policy_version 1596147 (0.0006) [2023-12-27 16:42:53,923][20347] Updated weights for policy 0, policy_version 1597721 (0.0006) [2023-12-27 16:42:53,932][20381] Updated weights for policy 1, policy_version 1596157 (0.0006) [2023-12-27 16:42:53,975][20347] Updated weights for policy 0, policy_version 1597731 (0.0006) [2023-12-27 16:42:53,983][20381] Updated weights for policy 1, policy_version 1596167 (0.0005) [2023-12-27 16:42:54,025][20347] Updated weights for policy 0, policy_version 1597741 (0.0006) [2023-12-27 16:42:54,037][20381] Updated weights for policy 1, policy_version 1596177 (0.0006) [2023-12-27 16:42:54,658][20347] Updated weights for policy 0, policy_version 1597751 (0.0009) [2023-12-27 16:42:54,713][20381] Updated weights for policy 1, policy_version 1596187 (0.0008) [2023-12-27 16:42:54,726][20347] Updated weights for policy 0, policy_version 1597761 (0.0011) [2023-12-27 16:42:54,766][20381] Updated weights for policy 1, policy_version 1596197 (0.0005) [2023-12-27 16:42:54,782][20347] Updated weights for policy 0, policy_version 1597771 (0.0010) [2023-12-27 16:42:54,816][20381] Updated weights for policy 1, policy_version 1596207 (0.0006) [2023-12-27 16:42:55,406][20347] Updated weights for policy 0, policy_version 1597781 (0.0008) [2023-12-27 16:42:55,467][20347] Updated weights for policy 0, policy_version 1597791 (0.0005) [2023-12-27 16:42:55,520][20381] Updated weights for policy 1, policy_version 1596217 (0.0009) [2023-12-27 16:42:55,526][20347] Updated weights for policy 0, policy_version 1597801 (0.0008) [2023-12-27 16:42:55,576][20381] Updated weights for policy 1, policy_version 1596227 (0.0006) [2023-12-27 16:42:55,630][20381] Updated weights for policy 1, policy_version 1596237 (0.0007) [2023-12-27 16:42:55,687][20381] Updated weights for policy 1, policy_version 1596247 (0.0008) [2023-12-27 16:42:56,093][20347] Updated weights for policy 0, policy_version 1597811 (0.0008) [2023-12-27 16:42:56,146][20347] Updated weights for policy 0, policy_version 1597821 (0.0006) [2023-12-27 16:42:56,201][20347] Updated weights for policy 0, policy_version 1597831 (0.0010) [2023-12-27 16:42:56,442][20381] Updated weights for policy 1, policy_version 1596257 (0.0007) [2023-12-27 16:42:56,496][20381] Updated weights for policy 1, policy_version 1596267 (0.0009) [2023-12-27 16:42:56,552][20381] Updated weights for policy 1, policy_version 1596277 (0.0008) [2023-12-27 16:42:56,751][20347] Updated weights for policy 0, policy_version 1597841 (0.0009) [2023-12-27 16:42:56,809][20347] Updated weights for policy 0, policy_version 1597851 (0.0007) [2023-12-27 16:42:56,866][20347] Updated weights for policy 0, policy_version 1597861 (0.0007) [2023-12-27 16:42:56,923][20347] Updated weights for policy 0, policy_version 1597871 (0.0007) [2023-12-27 16:42:57,270][20381] Updated weights for policy 1, policy_version 1596287 (0.0008) [2023-12-27 16:42:57,322][20381] Updated weights for policy 1, policy_version 1596297 (0.0005) [2023-12-27 16:42:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 817823744. Throughput: 0: 10370.0, 1: 10119.6. Samples: 817817688. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:42:57,363][19170] Avg episode reward: [(0, '980.490'), (1, '450.990')] [2023-12-27 16:42:57,379][20381] Updated weights for policy 1, policy_version 1596307 (0.0007) [2023-12-27 16:42:57,562][20347] Updated weights for policy 0, policy_version 1597881 (0.0006) [2023-12-27 16:42:57,593][20291] KL-divergence is very high: 170.2233 [2023-12-27 16:42:57,606][20347] Updated weights for policy 0, policy_version 1597891 (0.0009) [2023-12-27 16:42:57,632][20291] KL-divergence is very high: 199.1229 [2023-12-27 16:42:57,660][20347] Updated weights for policy 0, policy_version 1597901 (0.0007) [2023-12-27 16:42:58,066][20381] Updated weights for policy 1, policy_version 1596317 (0.0008) [2023-12-27 16:42:58,114][20381] Updated weights for policy 1, policy_version 1596327 (0.0008) [2023-12-27 16:42:58,166][20381] Updated weights for policy 1, policy_version 1596337 (0.0007) [2023-12-27 16:42:58,276][20347] Updated weights for policy 0, policy_version 1597911 (0.0007) [2023-12-27 16:42:58,340][20347] Updated weights for policy 0, policy_version 1597921 (0.0007) [2023-12-27 16:42:58,404][20347] Updated weights for policy 0, policy_version 1597931 (0.0008) [2023-12-27 16:42:59,031][20381] Updated weights for policy 1, policy_version 1596347 (0.0008) [2023-12-27 16:42:59,095][20381] Updated weights for policy 1, policy_version 1596357 (0.0006) [2023-12-27 16:42:59,114][20347] Updated weights for policy 0, policy_version 1597941 (0.0008) [2023-12-27 16:42:59,157][20381] Updated weights for policy 1, policy_version 1596367 (0.0006) [2023-12-27 16:42:59,167][20347] Updated weights for policy 0, policy_version 1597951 (0.0007) [2023-12-27 16:42:59,235][20347] Updated weights for policy 0, policy_version 1597961 (0.0007) [2023-12-27 16:42:59,825][20381] Updated weights for policy 1, policy_version 1596377 (0.0006) [2023-12-27 16:42:59,887][20381] Updated weights for policy 1, policy_version 1596387 (0.0008) [2023-12-27 16:42:59,928][20347] Updated weights for policy 0, policy_version 1597971 (0.0007) [2023-12-27 16:42:59,952][20381] Updated weights for policy 1, policy_version 1596397 (0.0008) [2023-12-27 16:42:59,984][20347] Updated weights for policy 0, policy_version 1597981 (0.0007) [2023-12-27 16:43:00,003][20381] Updated weights for policy 1, policy_version 1596407 (0.0008) [2023-12-27 16:43:00,038][20347] Updated weights for policy 0, policy_version 1597991 (0.0007) [2023-12-27 16:43:00,638][20347] Updated weights for policy 0, policy_version 1598001 (0.0007) [2023-12-27 16:43:00,700][20347] Updated weights for policy 0, policy_version 1598011 (0.0006) [2023-12-27 16:43:00,730][20381] Updated weights for policy 1, policy_version 1596417 (0.0007) [2023-12-27 16:43:00,761][20347] Updated weights for policy 0, policy_version 1598021 (0.0006) [2023-12-27 16:43:00,777][20381] Updated weights for policy 1, policy_version 1596427 (0.0008) [2023-12-27 16:43:00,816][20347] Updated weights for policy 0, policy_version 1598031 (0.0006) [2023-12-27 16:43:00,830][20381] Updated weights for policy 1, policy_version 1596437 (0.0010) [2023-12-27 16:43:01,514][20347] Updated weights for policy 0, policy_version 1598041 (0.0008) [2023-12-27 16:43:01,536][20381] Updated weights for policy 1, policy_version 1596447 (0.0008) [2023-12-27 16:43:01,574][20347] Updated weights for policy 0, policy_version 1598051 (0.0007) [2023-12-27 16:43:01,582][20381] Updated weights for policy 1, policy_version 1596457 (0.0006) [2023-12-27 16:43:01,634][20381] Updated weights for policy 1, policy_version 1596467 (0.0007) [2023-12-27 16:43:01,637][20347] Updated weights for policy 0, policy_version 1598061 (0.0007) [2023-12-27 16:43:02,288][20347] Updated weights for policy 0, policy_version 1598071 (0.0010) [2023-12-27 16:43:02,329][20381] Updated weights for policy 1, policy_version 1596477 (0.0006) [2023-12-27 16:43:02,346][20347] Updated weights for policy 0, policy_version 1598081 (0.0010) [2023-12-27 16:43:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 817922048. Throughput: 0: 10370.0, 1: 10146.8. Samples: 817939780. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:02,363][19170] Avg episode reward: [(0, '970.700'), (1, '429.850')] [2023-12-27 16:43:02,406][20381] Updated weights for policy 1, policy_version 1596487 (0.0008) [2023-12-27 16:43:02,416][20347] Updated weights for policy 0, policy_version 1598091 (0.0008) [2023-12-27 16:43:02,463][20381] Updated weights for policy 1, policy_version 1596497 (0.0006) [2023-12-27 16:43:03,015][20381] Updated weights for policy 1, policy_version 1596507 (0.0007) [2023-12-27 16:43:03,071][20381] Updated weights for policy 1, policy_version 1596517 (0.0011) [2023-12-27 16:43:03,081][20347] Updated weights for policy 0, policy_version 1598101 (0.0006) [2023-12-27 16:43:03,131][20381] Updated weights for policy 1, policy_version 1596527 (0.0011) [2023-12-27 16:43:03,145][20347] Updated weights for policy 0, policy_version 1598111 (0.0005) [2023-12-27 16:43:03,206][20347] Updated weights for policy 0, policy_version 1598121 (0.0006) [2023-12-27 16:43:03,798][20381] Updated weights for policy 1, policy_version 1596537 (0.0010) [2023-12-27 16:43:03,851][20381] Updated weights for policy 1, policy_version 1596547 (0.0008) [2023-12-27 16:43:03,859][20347] Updated weights for policy 0, policy_version 1598131 (0.0008) [2023-12-27 16:43:03,908][20381] Updated weights for policy 1, policy_version 1596557 (0.0006) [2023-12-27 16:43:03,911][20347] Updated weights for policy 0, policy_version 1598141 (0.0010) [2023-12-27 16:43:03,963][20381] Updated weights for policy 1, policy_version 1596567 (0.0007) [2023-12-27 16:43:03,971][20347] Updated weights for policy 0, policy_version 1598151 (0.0007) [2023-12-27 16:43:04,591][20381] Updated weights for policy 1, policy_version 1596577 (0.0006) [2023-12-27 16:43:04,640][20381] Updated weights for policy 1, policy_version 1596587 (0.0006) [2023-12-27 16:43:04,664][20347] Updated weights for policy 0, policy_version 1598161 (0.0008) [2023-12-27 16:43:04,687][20381] Updated weights for policy 1, policy_version 1596597 (0.0007) [2023-12-27 16:43:04,726][20347] Updated weights for policy 0, policy_version 1598171 (0.0005) [2023-12-27 16:43:04,782][20347] Updated weights for policy 0, policy_version 1598181 (0.0006) [2023-12-27 16:43:04,830][20347] Updated weights for policy 0, policy_version 1598191 (0.0007) [2023-12-27 16:43:05,295][20381] Updated weights for policy 1, policy_version 1596607 (0.0007) [2023-12-27 16:43:05,352][20381] Updated weights for policy 1, policy_version 1596617 (0.0008) [2023-12-27 16:43:05,404][20381] Updated weights for policy 1, policy_version 1596627 (0.0008) [2023-12-27 16:43:05,475][20347] Updated weights for policy 0, policy_version 1598201 (0.0006) [2023-12-27 16:43:05,535][20347] Updated weights for policy 0, policy_version 1598211 (0.0007) [2023-12-27 16:43:05,582][20347] Updated weights for policy 0, policy_version 1598221 (0.0007) [2023-12-27 16:43:06,022][20381] Updated weights for policy 1, policy_version 1596637 (0.0007) [2023-12-27 16:43:06,083][20381] Updated weights for policy 1, policy_version 1596647 (0.0007) [2023-12-27 16:43:06,148][20381] Updated weights for policy 1, policy_version 1596657 (0.0007) [2023-12-27 16:43:06,213][20347] Updated weights for policy 0, policy_version 1598231 (0.0006) [2023-12-27 16:43:06,273][20347] Updated weights for policy 0, policy_version 1598241 (0.0008) [2023-12-27 16:43:06,332][20347] Updated weights for policy 0, policy_version 1598251 (0.0007) [2023-12-27 16:43:06,809][20381] Updated weights for policy 1, policy_version 1596667 (0.0007) [2023-12-27 16:43:06,869][20381] Updated weights for policy 1, policy_version 1596677 (0.0007) [2023-12-27 16:43:06,922][20381] Updated weights for policy 1, policy_version 1596687 (0.0008) [2023-12-27 16:43:06,965][20347] Updated weights for policy 0, policy_version 1598261 (0.0008) [2023-12-27 16:43:07,018][20347] Updated weights for policy 0, policy_version 1598271 (0.0009) [2023-12-27 16:43:07,077][20347] Updated weights for policy 0, policy_version 1598281 (0.0007) [2023-12-27 16:43:07,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 818036736. Throughput: 0: 10366.4, 1: 10244.0. Samples: 818004052. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:07,364][19170] Avg episode reward: [(0, '990.210'), (1, '451.630')] [2023-12-27 16:43:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001598288_409239552.pth... [2023-12-27 16:43:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001596696_408797184.pth... [2023-12-27 16:43:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001597072_408928256.pth [2023-12-27 16:43:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001595512_408494080.pth [2023-12-27 16:43:07,559][20381] Updated weights for policy 1, policy_version 1596697 (0.0006) [2023-12-27 16:43:07,616][20381] Updated weights for policy 1, policy_version 1596707 (0.0008) [2023-12-27 16:43:07,665][20381] Updated weights for policy 1, policy_version 1596717 (0.0008) [2023-12-27 16:43:07,719][20381] Updated weights for policy 1, policy_version 1596727 (0.0006) [2023-12-27 16:43:07,752][20347] Updated weights for policy 0, policy_version 1598291 (0.0007) [2023-12-27 16:43:07,815][20347] Updated weights for policy 0, policy_version 1598301 (0.0006) [2023-12-27 16:43:07,879][20347] Updated weights for policy 0, policy_version 1598311 (0.0007) [2023-12-27 16:43:08,352][20381] Updated weights for policy 1, policy_version 1596737 (0.0008) [2023-12-27 16:43:08,404][20381] Updated weights for policy 1, policy_version 1596747 (0.0008) [2023-12-27 16:43:08,465][20381] Updated weights for policy 1, policy_version 1596757 (0.0007) [2023-12-27 16:43:08,592][20347] Updated weights for policy 0, policy_version 1598321 (0.0007) [2023-12-27 16:43:08,640][20347] Updated weights for policy 0, policy_version 1598331 (0.0008) [2023-12-27 16:43:08,688][20347] Updated weights for policy 0, policy_version 1598341 (0.0008) [2023-12-27 16:43:08,739][20347] Updated weights for policy 0, policy_version 1598351 (0.0008) [2023-12-27 16:43:09,147][20381] Updated weights for policy 1, policy_version 1596767 (0.0006) [2023-12-27 16:43:09,198][20381] Updated weights for policy 1, policy_version 1596777 (0.0007) [2023-12-27 16:43:09,258][20381] Updated weights for policy 1, policy_version 1596787 (0.0008) [2023-12-27 16:43:09,509][20347] Updated weights for policy 0, policy_version 1598361 (0.0006) [2023-12-27 16:43:09,565][20347] Updated weights for policy 0, policy_version 1598371 (0.0007) [2023-12-27 16:43:09,614][20347] Updated weights for policy 0, policy_version 1598381 (0.0007) [2023-12-27 16:43:10,027][20381] Updated weights for policy 1, policy_version 1596797 (0.0008) [2023-12-27 16:43:10,084][20381] Updated weights for policy 1, policy_version 1596807 (0.0008) [2023-12-27 16:43:10,141][20381] Updated weights for policy 1, policy_version 1596817 (0.0008) [2023-12-27 16:43:10,341][20347] Updated weights for policy 0, policy_version 1598391 (0.0008) [2023-12-27 16:43:10,399][20347] Updated weights for policy 0, policy_version 1598401 (0.0008) [2023-12-27 16:43:10,456][20347] Updated weights for policy 0, policy_version 1598411 (0.0007) [2023-12-27 16:43:10,832][20381] Updated weights for policy 1, policy_version 1596827 (0.0008) [2023-12-27 16:43:10,884][20381] Updated weights for policy 1, policy_version 1596837 (0.0006) [2023-12-27 16:43:10,938][20381] Updated weights for policy 1, policy_version 1596847 (0.0008) [2023-12-27 16:43:11,153][20347] Updated weights for policy 0, policy_version 1598421 (0.0008) [2023-12-27 16:43:11,211][20347] Updated weights for policy 0, policy_version 1598431 (0.0008) [2023-12-27 16:43:11,270][20347] Updated weights for policy 0, policy_version 1598441 (0.0008) [2023-12-27 16:43:11,714][20381] Updated weights for policy 1, policy_version 1596857 (0.0010) [2023-12-27 16:43:11,783][20381] Updated weights for policy 1, policy_version 1596867 (0.0009) [2023-12-27 16:43:11,835][20381] Updated weights for policy 1, policy_version 1596877 (0.0007) [2023-12-27 16:43:11,885][20381] Updated weights for policy 1, policy_version 1596887 (0.0008) [2023-12-27 16:43:11,966][20347] Updated weights for policy 0, policy_version 1598451 (0.0007) [2023-12-27 16:43:12,019][20347] Updated weights for policy 0, policy_version 1598461 (0.0008) [2023-12-27 16:43:12,076][20347] Updated weights for policy 0, policy_version 1598471 (0.0008) [2023-12-27 16:43:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 818135040. Throughput: 0: 10342.5, 1: 10241.5. Samples: 818126096. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '343.440')] [2023-12-27 16:43:12,577][20381] Updated weights for policy 1, policy_version 1596897 (0.0008) [2023-12-27 16:43:12,639][20381] Updated weights for policy 1, policy_version 1596907 (0.0007) [2023-12-27 16:43:12,695][20381] Updated weights for policy 1, policy_version 1596917 (0.0008) [2023-12-27 16:43:12,789][20347] Updated weights for policy 0, policy_version 1598481 (0.0007) [2023-12-27 16:43:12,851][20347] Updated weights for policy 0, policy_version 1598491 (0.0010) [2023-12-27 16:43:12,908][20347] Updated weights for policy 0, policy_version 1598501 (0.0009) [2023-12-27 16:43:12,968][20347] Updated weights for policy 0, policy_version 1598511 (0.0008) [2023-12-27 16:43:13,380][20381] Updated weights for policy 1, policy_version 1596927 (0.0008) [2023-12-27 16:43:13,432][20381] Updated weights for policy 1, policy_version 1596937 (0.0007) [2023-12-27 16:43:13,497][20381] Updated weights for policy 1, policy_version 1596947 (0.0008) [2023-12-27 16:43:13,611][20347] Updated weights for policy 0, policy_version 1598521 (0.0007) [2023-12-27 16:43:13,675][20347] Updated weights for policy 0, policy_version 1598531 (0.0007) [2023-12-27 16:43:13,740][20347] Updated weights for policy 0, policy_version 1598541 (0.0007) [2023-12-27 16:43:14,139][20381] Updated weights for policy 1, policy_version 1596957 (0.0007) [2023-12-27 16:43:14,195][20381] Updated weights for policy 1, policy_version 1596967 (0.0005) [2023-12-27 16:43:14,255][20381] Updated weights for policy 1, policy_version 1596977 (0.0008) [2023-12-27 16:43:14,352][20347] Updated weights for policy 0, policy_version 1598551 (0.0010) [2023-12-27 16:43:14,409][20347] Updated weights for policy 0, policy_version 1598561 (0.0010) [2023-12-27 16:43:14,468][20347] Updated weights for policy 0, policy_version 1598571 (0.0007) [2023-12-27 16:43:14,887][20381] Updated weights for policy 1, policy_version 1596987 (0.0007) [2023-12-27 16:43:14,959][20381] Updated weights for policy 1, policy_version 1596997 (0.0007) [2023-12-27 16:43:15,022][20381] Updated weights for policy 1, policy_version 1597007 (0.0008) [2023-12-27 16:43:15,139][20347] Updated weights for policy 0, policy_version 1598581 (0.0010) [2023-12-27 16:43:15,200][20347] Updated weights for policy 0, policy_version 1598591 (0.0011) [2023-12-27 16:43:15,263][20347] Updated weights for policy 0, policy_version 1598601 (0.0011) [2023-12-27 16:43:15,738][20381] Updated weights for policy 1, policy_version 1597017 (0.0007) [2023-12-27 16:43:15,783][20381] Updated weights for policy 1, policy_version 1597027 (0.0007) [2023-12-27 16:43:15,851][20381] Updated weights for policy 1, policy_version 1597037 (0.0007) [2023-12-27 16:43:15,885][20347] Updated weights for policy 0, policy_version 1598611 (0.0008) [2023-12-27 16:43:15,906][20381] Updated weights for policy 1, policy_version 1597047 (0.0009) [2023-12-27 16:43:15,949][20347] Updated weights for policy 0, policy_version 1598621 (0.0007) [2023-12-27 16:43:16,014][20347] Updated weights for policy 0, policy_version 1598631 (0.0006) [2023-12-27 16:43:16,564][20347] Updated weights for policy 0, policy_version 1598641 (0.0005) [2023-12-27 16:43:16,619][20347] Updated weights for policy 0, policy_version 1598651 (0.0005) [2023-12-27 16:43:16,672][20347] Updated weights for policy 0, policy_version 1598661 (0.0006) [2023-12-27 16:43:16,681][20381] Updated weights for policy 1, policy_version 1597057 (0.0006) [2023-12-27 16:43:16,725][20347] Updated weights for policy 0, policy_version 1598671 (0.0006) [2023-12-27 16:43:16,737][20381] Updated weights for policy 1, policy_version 1597067 (0.0007) [2023-12-27 16:43:16,793][20381] Updated weights for policy 1, policy_version 1597077 (0.0008) [2023-12-27 16:43:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 818233344. Throughput: 0: 10337.0, 1: 10117.4. Samples: 818247936. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '512.560')] [2023-12-27 16:43:17,408][20347] Updated weights for policy 0, policy_version 1598681 (0.0007) [2023-12-27 16:43:17,462][20347] Updated weights for policy 0, policy_version 1598691 (0.0007) [2023-12-27 16:43:17,487][20381] Updated weights for policy 1, policy_version 1597087 (0.0007) [2023-12-27 16:43:17,519][20347] Updated weights for policy 0, policy_version 1598701 (0.0005) [2023-12-27 16:43:17,541][20381] Updated weights for policy 1, policy_version 1597097 (0.0008) [2023-12-27 16:43:17,598][20381] Updated weights for policy 1, policy_version 1597108 (0.0009) [2023-12-27 16:43:18,105][20347] Updated weights for policy 0, policy_version 1598711 (0.0009) [2023-12-27 16:43:18,159][20347] Updated weights for policy 0, policy_version 1598721 (0.0010) [2023-12-27 16:43:18,212][20347] Updated weights for policy 0, policy_version 1598731 (0.0011) [2023-12-27 16:43:18,326][20381] Updated weights for policy 1, policy_version 1597118 (0.0008) [2023-12-27 16:43:18,397][20381] Updated weights for policy 1, policy_version 1597128 (0.0008) [2023-12-27 16:43:18,450][20381] Updated weights for policy 1, policy_version 1597138 (0.0007) [2023-12-27 16:43:18,911][20347] Updated weights for policy 0, policy_version 1598741 (0.0010) [2023-12-27 16:43:18,968][20347] Updated weights for policy 0, policy_version 1598751 (0.0010) [2023-12-27 16:43:19,027][20347] Updated weights for policy 0, policy_version 1598761 (0.0010) [2023-12-27 16:43:19,160][20381] Updated weights for policy 1, policy_version 1597148 (0.0008) [2023-12-27 16:43:19,209][20381] Updated weights for policy 1, policy_version 1597158 (0.0008) [2023-12-27 16:43:19,268][20381] Updated weights for policy 1, policy_version 1597168 (0.0009) [2023-12-27 16:43:19,689][20347] Updated weights for policy 0, policy_version 1598771 (0.0009) [2023-12-27 16:43:19,757][20347] Updated weights for policy 0, policy_version 1598781 (0.0008) [2023-12-27 16:43:19,824][20347] Updated weights for policy 0, policy_version 1598791 (0.0010) [2023-12-27 16:43:20,037][20381] Updated weights for policy 1, policy_version 1597178 (0.0008) [2023-12-27 16:43:20,084][20381] Updated weights for policy 1, policy_version 1597188 (0.0007) [2023-12-27 16:43:20,135][20381] Updated weights for policy 1, policy_version 1597198 (0.0008) [2023-12-27 16:43:20,187][20381] Updated weights for policy 1, policy_version 1597208 (0.0007) [2023-12-27 16:43:20,435][20347] Updated weights for policy 0, policy_version 1598801 (0.0011) [2023-12-27 16:43:20,488][20347] Updated weights for policy 0, policy_version 1598811 (0.0006) [2023-12-27 16:43:20,534][20347] Updated weights for policy 0, policy_version 1598821 (0.0009) [2023-12-27 16:43:20,593][20347] Updated weights for policy 0, policy_version 1598831 (0.0008) [2023-12-27 16:43:21,020][20381] Updated weights for policy 1, policy_version 1597218 (0.0008) [2023-12-27 16:43:21,084][20381] Updated weights for policy 1, policy_version 1597228 (0.0009) [2023-12-27 16:43:21,143][20381] Updated weights for policy 1, policy_version 1597238 (0.0009) [2023-12-27 16:43:21,275][20347] Updated weights for policy 0, policy_version 1598841 (0.0009) [2023-12-27 16:43:21,332][20347] Updated weights for policy 0, policy_version 1598851 (0.0009) [2023-12-27 16:43:21,397][20347] Updated weights for policy 0, policy_version 1598861 (0.0011) [2023-12-27 16:43:22,006][20381] Updated weights for policy 1, policy_version 1597248 (0.0007) [2023-12-27 16:43:22,068][20381] Updated weights for policy 1, policy_version 1597258 (0.0007) [2023-12-27 16:43:22,127][20381] Updated weights for policy 1, policy_version 1597268 (0.0007) [2023-12-27 16:43:22,198][20347] Updated weights for policy 0, policy_version 1598871 (0.0010) [2023-12-27 16:43:22,248][20347] Updated weights for policy 0, policy_version 1598881 (0.0010) [2023-12-27 16:43:22,310][20347] Updated weights for policy 0, policy_version 1598891 (0.0011) [2023-12-27 16:43:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 818339840. Throughput: 0: 10308.3, 1: 10056.5. Samples: 818306924. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:22,363][19170] Avg episode reward: [(0, '990.340'), (1, '611.930')] [2023-12-27 16:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001597272_408944640.pth... [2023-12-27 16:43:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001598896_409395200.pth... [2023-12-27 16:43:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001596120_408649728.pth [2023-12-27 16:43:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001597680_409083904.pth [2023-12-27 16:43:22,816][20381] Updated weights for policy 1, policy_version 1597278 (0.0007) [2023-12-27 16:43:22,863][20381] Updated weights for policy 1, policy_version 1597288 (0.0007) [2023-12-27 16:43:22,921][20381] Updated weights for policy 1, policy_version 1597298 (0.0008) [2023-12-27 16:43:23,055][20347] Updated weights for policy 0, policy_version 1598901 (0.0010) [2023-12-27 16:43:23,104][20347] Updated weights for policy 0, policy_version 1598911 (0.0010) [2023-12-27 16:43:23,167][20347] Updated weights for policy 0, policy_version 1598921 (0.0010) [2023-12-27 16:43:23,624][20381] Updated weights for policy 1, policy_version 1597308 (0.0007) [2023-12-27 16:43:23,671][20381] Updated weights for policy 1, policy_version 1597318 (0.0007) [2023-12-27 16:43:23,722][20381] Updated weights for policy 1, policy_version 1597328 (0.0007) [2023-12-27 16:43:23,862][20347] Updated weights for policy 0, policy_version 1598931 (0.0010) [2023-12-27 16:43:23,920][20347] Updated weights for policy 0, policy_version 1598941 (0.0010) [2023-12-27 16:43:23,968][20347] Updated weights for policy 0, policy_version 1598951 (0.0009) [2023-12-27 16:43:24,368][20381] Updated weights for policy 1, policy_version 1597338 (0.0006) [2023-12-27 16:43:24,423][20381] Updated weights for policy 1, policy_version 1597348 (0.0007) [2023-12-27 16:43:24,477][20381] Updated weights for policy 1, policy_version 1597358 (0.0007) [2023-12-27 16:43:24,526][20381] Updated weights for policy 1, policy_version 1597368 (0.0007) [2023-12-27 16:43:24,659][20347] Updated weights for policy 0, policy_version 1598961 (0.0010) [2023-12-27 16:43:24,713][20347] Updated weights for policy 0, policy_version 1598971 (0.0010) [2023-12-27 16:43:24,763][20347] Updated weights for policy 0, policy_version 1598981 (0.0006) [2023-12-27 16:43:24,818][20347] Updated weights for policy 0, policy_version 1598991 (0.0009) [2023-12-27 16:43:25,220][20381] Updated weights for policy 1, policy_version 1597378 (0.0007) [2023-12-27 16:43:25,274][20381] Updated weights for policy 1, policy_version 1597388 (0.0007) [2023-12-27 16:43:25,326][20381] Updated weights for policy 1, policy_version 1597398 (0.0007) [2023-12-27 16:43:25,480][20347] Updated weights for policy 0, policy_version 1599001 (0.0007) [2023-12-27 16:43:25,526][20347] Updated weights for policy 0, policy_version 1599011 (0.0005) [2023-12-27 16:43:25,577][20347] Updated weights for policy 0, policy_version 1599021 (0.0009) [2023-12-27 16:43:26,026][20381] Updated weights for policy 1, policy_version 1597408 (0.0006) [2023-12-27 16:43:26,076][20381] Updated weights for policy 1, policy_version 1597418 (0.0008) [2023-12-27 16:43:26,134][20381] Updated weights for policy 1, policy_version 1597428 (0.0007) [2023-12-27 16:43:26,244][20347] Updated weights for policy 0, policy_version 1599031 (0.0007) [2023-12-27 16:43:26,295][20347] Updated weights for policy 0, policy_version 1599041 (0.0007) [2023-12-27 16:43:26,343][20347] Updated weights for policy 0, policy_version 1599051 (0.0007) [2023-12-27 16:43:26,818][20381] Updated weights for policy 1, policy_version 1597438 (0.0007) [2023-12-27 16:43:26,865][20381] Updated weights for policy 1, policy_version 1597448 (0.0007) [2023-12-27 16:43:26,911][20381] Updated weights for policy 1, policy_version 1597458 (0.0007) [2023-12-27 16:43:27,014][20347] Updated weights for policy 0, policy_version 1599061 (0.0007) [2023-12-27 16:43:27,063][20347] Updated weights for policy 0, policy_version 1599071 (0.0007) [2023-12-27 16:43:27,124][20347] Updated weights for policy 0, policy_version 1599081 (0.0006) [2023-12-27 16:43:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 818438144. Throughput: 0: 10244.4, 1: 9992.3. Samples: 818425512. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:27,364][19170] Avg episode reward: [(0, '980.580'), (1, '623.850')] [2023-12-27 16:43:27,616][20381] Updated weights for policy 1, policy_version 1597468 (0.0008) [2023-12-27 16:43:27,664][20381] Updated weights for policy 1, policy_version 1597478 (0.0008) [2023-12-27 16:43:27,717][20381] Updated weights for policy 1, policy_version 1597488 (0.0007) [2023-12-27 16:43:27,731][20347] Updated weights for policy 0, policy_version 1599091 (0.0007) [2023-12-27 16:43:27,784][20347] Updated weights for policy 0, policy_version 1599101 (0.0005) [2023-12-27 16:43:27,834][20347] Updated weights for policy 0, policy_version 1599111 (0.0005) [2023-12-27 16:43:28,428][20381] Updated weights for policy 1, policy_version 1597498 (0.0007) [2023-12-27 16:43:28,484][20381] Updated weights for policy 1, policy_version 1597508 (0.0005) [2023-12-27 16:43:28,497][20347] Updated weights for policy 0, policy_version 1599121 (0.0006) [2023-12-27 16:43:28,535][20381] Updated weights for policy 1, policy_version 1597518 (0.0005) [2023-12-27 16:43:28,552][20347] Updated weights for policy 0, policy_version 1599131 (0.0007) [2023-12-27 16:43:28,586][20381] Updated weights for policy 1, policy_version 1597528 (0.0005) [2023-12-27 16:43:28,612][20347] Updated weights for policy 0, policy_version 1599141 (0.0007) [2023-12-27 16:43:28,669][20347] Updated weights for policy 0, policy_version 1599151 (0.0007) [2023-12-27 16:43:29,149][20381] Updated weights for policy 1, policy_version 1597538 (0.0005) [2023-12-27 16:43:29,198][20381] Updated weights for policy 1, policy_version 1597548 (0.0006) [2023-12-27 16:43:29,258][20381] Updated weights for policy 1, policy_version 1597558 (0.0008) [2023-12-27 16:43:29,381][20347] Updated weights for policy 0, policy_version 1599161 (0.0008) [2023-12-27 16:43:29,443][20347] Updated weights for policy 0, policy_version 1599171 (0.0008) [2023-12-27 16:43:29,501][20347] Updated weights for policy 0, policy_version 1599181 (0.0007) [2023-12-27 16:43:29,845][20381] Updated weights for policy 1, policy_version 1597568 (0.0007) [2023-12-27 16:43:29,903][20381] Updated weights for policy 1, policy_version 1597578 (0.0006) [2023-12-27 16:43:29,970][20381] Updated weights for policy 1, policy_version 1597588 (0.0007) [2023-12-27 16:43:30,346][20347] Updated weights for policy 0, policy_version 1599191 (0.0009) [2023-12-27 16:43:30,407][20347] Updated weights for policy 0, policy_version 1599201 (0.0007) [2023-12-27 16:43:30,462][20347] Updated weights for policy 0, policy_version 1599211 (0.0008) [2023-12-27 16:43:30,541][20381] Updated weights for policy 1, policy_version 1597598 (0.0007) [2023-12-27 16:43:30,598][20381] Updated weights for policy 1, policy_version 1597608 (0.0007) [2023-12-27 16:43:30,659][20381] Updated weights for policy 1, policy_version 1597618 (0.0007) [2023-12-27 16:43:31,112][20347] Updated weights for policy 0, policy_version 1599221 (0.0007) [2023-12-27 16:43:31,175][20347] Updated weights for policy 0, policy_version 1599231 (0.0008) [2023-12-27 16:43:31,232][20347] Updated weights for policy 0, policy_version 1599241 (0.0008) [2023-12-27 16:43:31,346][20381] Updated weights for policy 1, policy_version 1597628 (0.0009) [2023-12-27 16:43:31,413][20381] Updated weights for policy 1, policy_version 1597638 (0.0007) [2023-12-27 16:43:31,477][20381] Updated weights for policy 1, policy_version 1597648 (0.0005) [2023-12-27 16:43:31,903][20347] Updated weights for policy 0, policy_version 1599251 (0.0007) [2023-12-27 16:43:31,955][20347] Updated weights for policy 0, policy_version 1599261 (0.0007) [2023-12-27 16:43:32,008][20347] Updated weights for policy 0, policy_version 1599271 (0.0007) [2023-12-27 16:43:32,162][20381] Updated weights for policy 1, policy_version 1597658 (0.0008) [2023-12-27 16:43:32,217][20381] Updated weights for policy 1, policy_version 1597668 (0.0008) [2023-12-27 16:43:32,279][20381] Updated weights for policy 1, policy_version 1597678 (0.0007) [2023-12-27 16:43:32,339][20381] Updated weights for policy 1, policy_version 1597688 (0.0007) [2023-12-27 16:43:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 818544640. Throughput: 0: 10244.3, 1: 10031.7. Samples: 818549908. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:32,363][19170] Avg episode reward: [(0, '990.240'), (1, '590.130')] [2023-12-27 16:43:32,675][20347] Updated weights for policy 0, policy_version 1599281 (0.0008) [2023-12-27 16:43:32,734][20347] Updated weights for policy 0, policy_version 1599291 (0.0008) [2023-12-27 16:43:32,794][20347] Updated weights for policy 0, policy_version 1599301 (0.0008) [2023-12-27 16:43:32,850][20347] Updated weights for policy 0, policy_version 1599311 (0.0007) [2023-12-27 16:43:33,051][20381] Updated weights for policy 1, policy_version 1597698 (0.0008) [2023-12-27 16:43:33,108][20381] Updated weights for policy 1, policy_version 1597708 (0.0007) [2023-12-27 16:43:33,163][20381] Updated weights for policy 1, policy_version 1597718 (0.0007) [2023-12-27 16:43:33,519][20347] Updated weights for policy 0, policy_version 1599321 (0.0008) [2023-12-27 16:43:33,566][20347] Updated weights for policy 0, policy_version 1599331 (0.0007) [2023-12-27 16:43:33,622][20347] Updated weights for policy 0, policy_version 1599341 (0.0008) [2023-12-27 16:43:33,851][20381] Updated weights for policy 1, policy_version 1597728 (0.0007) [2023-12-27 16:43:33,893][20381] Updated weights for policy 1, policy_version 1597738 (0.0008) [2023-12-27 16:43:33,945][20381] Updated weights for policy 1, policy_version 1597748 (0.0006) [2023-12-27 16:43:34,318][20347] Updated weights for policy 0, policy_version 1599351 (0.0007) [2023-12-27 16:43:34,375][20347] Updated weights for policy 0, policy_version 1599361 (0.0006) [2023-12-27 16:43:34,439][20347] Updated weights for policy 0, policy_version 1599371 (0.0005) [2023-12-27 16:43:34,684][20381] Updated weights for policy 1, policy_version 1597758 (0.0007) [2023-12-27 16:43:34,730][20381] Updated weights for policy 1, policy_version 1597768 (0.0008) [2023-12-27 16:43:34,780][20381] Updated weights for policy 1, policy_version 1597778 (0.0008) [2023-12-27 16:43:35,093][20347] Updated weights for policy 0, policy_version 1599381 (0.0008) [2023-12-27 16:43:35,142][20347] Updated weights for policy 0, policy_version 1599391 (0.0008) [2023-12-27 16:43:35,195][20347] Updated weights for policy 0, policy_version 1599401 (0.0008) [2023-12-27 16:43:35,401][20381] Updated weights for policy 1, policy_version 1597788 (0.0006) [2023-12-27 16:43:35,461][20381] Updated weights for policy 1, policy_version 1597798 (0.0008) [2023-12-27 16:43:35,526][20381] Updated weights for policy 1, policy_version 1597808 (0.0008) [2023-12-27 16:43:35,866][20347] Updated weights for policy 0, policy_version 1599411 (0.0008) [2023-12-27 16:43:35,927][20347] Updated weights for policy 0, policy_version 1599421 (0.0006) [2023-12-27 16:43:35,977][20347] Updated weights for policy 0, policy_version 1599431 (0.0008) [2023-12-27 16:43:36,161][20381] Updated weights for policy 1, policy_version 1597818 (0.0007) [2023-12-27 16:43:36,225][20381] Updated weights for policy 1, policy_version 1597828 (0.0011) [2023-12-27 16:43:36,285][20381] Updated weights for policy 1, policy_version 1597838 (0.0008) [2023-12-27 16:43:36,351][20381] Updated weights for policy 1, policy_version 1597848 (0.0010) [2023-12-27 16:43:36,735][20347] Updated weights for policy 0, policy_version 1599441 (0.0006) [2023-12-27 16:43:36,794][20347] Updated weights for policy 0, policy_version 1599451 (0.0007) [2023-12-27 16:43:36,847][20347] Updated weights for policy 0, policy_version 1599461 (0.0009) [2023-12-27 16:43:36,910][20347] Updated weights for policy 0, policy_version 1599471 (0.0007) [2023-12-27 16:43:36,956][20381] Updated weights for policy 1, policy_version 1597858 (0.0006) [2023-12-27 16:43:37,010][20381] Updated weights for policy 1, policy_version 1597868 (0.0008) [2023-12-27 16:43:37,054][20381] Updated weights for policy 1, policy_version 1597878 (0.0008) [2023-12-27 16:43:37,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 818642944. Throughput: 0: 10261.3, 1: 10046.9. Samples: 818611528. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:37,363][19170] Avg episode reward: [(0, '990.220'), (1, '663.000')] [2023-12-27 16:43:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001597880_409100288.pth... [2023-12-27 16:43:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001599472_409542656.pth... [2023-12-27 16:43:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001596696_408797184.pth [2023-12-27 16:43:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001598288_409239552.pth [2023-12-27 16:43:37,633][20347] Updated weights for policy 0, policy_version 1599481 (0.0007) [2023-12-27 16:43:37,688][20347] Updated weights for policy 0, policy_version 1599491 (0.0008) [2023-12-27 16:43:37,734][20381] Updated weights for policy 1, policy_version 1597888 (0.0007) [2023-12-27 16:43:37,736][20347] Updated weights for policy 0, policy_version 1599501 (0.0005) [2023-12-27 16:43:37,786][20381] Updated weights for policy 1, policy_version 1597898 (0.0007) [2023-12-27 16:43:37,837][20381] Updated weights for policy 1, policy_version 1597908 (0.0006) [2023-12-27 16:43:38,362][20347] Updated weights for policy 0, policy_version 1599511 (0.0006) [2023-12-27 16:43:38,419][20347] Updated weights for policy 0, policy_version 1599521 (0.0005) [2023-12-27 16:43:38,478][20347] Updated weights for policy 0, policy_version 1599531 (0.0007) [2023-12-27 16:43:38,518][20381] Updated weights for policy 1, policy_version 1597918 (0.0007) [2023-12-27 16:43:38,576][20381] Updated weights for policy 1, policy_version 1597928 (0.0008) [2023-12-27 16:43:38,630][20381] Updated weights for policy 1, policy_version 1597938 (0.0007) [2023-12-27 16:43:39,138][20347] Updated weights for policy 0, policy_version 1599541 (0.0007) [2023-12-27 16:43:39,196][20347] Updated weights for policy 0, policy_version 1599551 (0.0007) [2023-12-27 16:43:39,257][20347] Updated weights for policy 0, policy_version 1599561 (0.0008) [2023-12-27 16:43:39,369][20381] Updated weights for policy 1, policy_version 1597948 (0.0008) [2023-12-27 16:43:39,435][20381] Updated weights for policy 1, policy_version 1597958 (0.0007) [2023-12-27 16:43:39,489][20381] Updated weights for policy 1, policy_version 1597968 (0.0007) [2023-12-27 16:43:40,035][20347] Updated weights for policy 0, policy_version 1599571 (0.0007) [2023-12-27 16:43:40,092][20347] Updated weights for policy 0, policy_version 1599581 (0.0008) [2023-12-27 16:43:40,150][20381] Updated weights for policy 1, policy_version 1597978 (0.0009) [2023-12-27 16:43:40,152][20347] Updated weights for policy 0, policy_version 1599591 (0.0007) [2023-12-27 16:43:40,210][20381] Updated weights for policy 1, policy_version 1597988 (0.0011) [2023-12-27 16:43:40,267][20381] Updated weights for policy 1, policy_version 1597998 (0.0011) [2023-12-27 16:43:40,329][20381] Updated weights for policy 1, policy_version 1598008 (0.0011) [2023-12-27 16:43:40,829][20347] Updated weights for policy 0, policy_version 1599601 (0.0010) [2023-12-27 16:43:40,892][20347] Updated weights for policy 0, policy_version 1599611 (0.0010) [2023-12-27 16:43:40,940][20347] Updated weights for policy 0, policy_version 1599621 (0.0010) [2023-12-27 16:43:40,974][20381] Updated weights for policy 1, policy_version 1598018 (0.0010) [2023-12-27 16:43:41,000][20347] Updated weights for policy 0, policy_version 1599631 (0.0007) [2023-12-27 16:43:41,027][20381] Updated weights for policy 1, policy_version 1598028 (0.0007) [2023-12-27 16:43:41,087][20381] Updated weights for policy 1, policy_version 1598038 (0.0008) [2023-12-27 16:43:41,707][20347] Updated weights for policy 0, policy_version 1599641 (0.0007) [2023-12-27 16:43:41,771][20347] Updated weights for policy 0, policy_version 1599651 (0.0008) [2023-12-27 16:43:41,830][20347] Updated weights for policy 0, policy_version 1599661 (0.0008) [2023-12-27 16:43:41,855][20381] Updated weights for policy 1, policy_version 1598048 (0.0007) [2023-12-27 16:43:41,905][20381] Updated weights for policy 1, policy_version 1598058 (0.0010) [2023-12-27 16:43:41,959][20381] Updated weights for policy 1, policy_version 1598068 (0.0008) [2023-12-27 16:43:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 818741248. Throughput: 0: 10243.6, 1: 10103.5. Samples: 818733312. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:42,363][19170] Avg episode reward: [(0, '980.560'), (1, '616.140')] [2023-12-27 16:43:42,529][20347] Updated weights for policy 0, policy_version 1599671 (0.0007) [2023-12-27 16:43:42,584][20347] Updated weights for policy 0, policy_version 1599681 (0.0008) [2023-12-27 16:43:42,638][20381] Updated weights for policy 1, policy_version 1598078 (0.0006) [2023-12-27 16:43:42,640][20347] Updated weights for policy 0, policy_version 1599691 (0.0007) [2023-12-27 16:43:42,695][20381] Updated weights for policy 1, policy_version 1598088 (0.0007) [2023-12-27 16:43:42,756][20381] Updated weights for policy 1, policy_version 1598098 (0.0007) [2023-12-27 16:43:43,340][20347] Updated weights for policy 0, policy_version 1599701 (0.0007) [2023-12-27 16:43:43,396][20347] Updated weights for policy 0, policy_version 1599711 (0.0007) [2023-12-27 16:43:43,422][20381] Updated weights for policy 1, policy_version 1598108 (0.0008) [2023-12-27 16:43:43,455][20347] Updated weights for policy 0, policy_version 1599721 (0.0006) [2023-12-27 16:43:43,487][20381] Updated weights for policy 1, policy_version 1598118 (0.0006) [2023-12-27 16:43:43,542][20381] Updated weights for policy 1, policy_version 1598128 (0.0007) [2023-12-27 16:43:44,070][20347] Updated weights for policy 0, policy_version 1599731 (0.0006) [2023-12-27 16:43:44,123][20347] Updated weights for policy 0, policy_version 1599741 (0.0007) [2023-12-27 16:43:44,180][20347] Updated weights for policy 0, policy_version 1599751 (0.0009) [2023-12-27 16:43:44,235][20381] Updated weights for policy 1, policy_version 1598138 (0.0007) [2023-12-27 16:43:44,300][20381] Updated weights for policy 1, policy_version 1598148 (0.0008) [2023-12-27 16:43:44,350][20381] Updated weights for policy 1, policy_version 1598158 (0.0008) [2023-12-27 16:43:44,396][20381] Updated weights for policy 1, policy_version 1598168 (0.0007) [2023-12-27 16:43:44,913][20347] Updated weights for policy 0, policy_version 1599761 (0.0007) [2023-12-27 16:43:44,974][20347] Updated weights for policy 0, policy_version 1599771 (0.0008) [2023-12-27 16:43:45,026][20381] Updated weights for policy 1, policy_version 1598178 (0.0006) [2023-12-27 16:43:45,037][20347] Updated weights for policy 0, policy_version 1599781 (0.0008) [2023-12-27 16:43:45,086][20381] Updated weights for policy 1, policy_version 1598188 (0.0006) [2023-12-27 16:43:45,098][20347] Updated weights for policy 0, policy_version 1599791 (0.0007) [2023-12-27 16:43:45,157][20381] Updated weights for policy 1, policy_version 1598198 (0.0006) [2023-12-27 16:43:45,743][20381] Updated weights for policy 1, policy_version 1598208 (0.0010) [2023-12-27 16:43:45,799][20381] Updated weights for policy 1, policy_version 1598218 (0.0009) [2023-12-27 16:43:45,841][20347] Updated weights for policy 0, policy_version 1599801 (0.0008) [2023-12-27 16:43:45,858][20381] Updated weights for policy 1, policy_version 1598228 (0.0006) [2023-12-27 16:43:45,898][20347] Updated weights for policy 0, policy_version 1599811 (0.0009) [2023-12-27 16:43:45,955][20347] Updated weights for policy 0, policy_version 1599821 (0.0007) [2023-12-27 16:43:46,586][20381] Updated weights for policy 1, policy_version 1598238 (0.0007) [2023-12-27 16:43:46,590][20347] Updated weights for policy 0, policy_version 1599831 (0.0007) [2023-12-27 16:43:46,644][20381] Updated weights for policy 1, policy_version 1598248 (0.0008) [2023-12-27 16:43:46,654][20347] Updated weights for policy 0, policy_version 1599841 (0.0006) [2023-12-27 16:43:46,700][20381] Updated weights for policy 1, policy_version 1598258 (0.0008) [2023-12-27 16:43:46,705][20347] Updated weights for policy 0, policy_version 1599851 (0.0005) [2023-12-27 16:43:47,296][20347] Updated weights for policy 0, policy_version 1599861 (0.0007) [2023-12-27 16:43:47,351][20347] Updated weights for policy 0, policy_version 1599871 (0.0008) [2023-12-27 16:43:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20521.6). Total num frames: 818839552. Throughput: 0: 10178.4, 1: 10163.2. Samples: 818855156. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:47,364][19170] Avg episode reward: [(0, '990.340'), (1, '694.650')] [2023-12-27 16:43:47,395][20381] Updated weights for policy 1, policy_version 1598268 (0.0007) [2023-12-27 16:43:47,405][20347] Updated weights for policy 0, policy_version 1599881 (0.0006) [2023-12-27 16:43:47,445][20381] Updated weights for policy 1, policy_version 1598278 (0.0005) [2023-12-27 16:43:47,500][20381] Updated weights for policy 1, policy_version 1598288 (0.0007) [2023-12-27 16:43:48,019][20347] Updated weights for policy 0, policy_version 1599891 (0.0007) [2023-12-27 16:43:48,072][20347] Updated weights for policy 0, policy_version 1599901 (0.0007) [2023-12-27 16:43:48,132][20347] Updated weights for policy 0, policy_version 1599911 (0.0008) [2023-12-27 16:43:48,212][20381] Updated weights for policy 1, policy_version 1598298 (0.0007) [2023-12-27 16:43:48,270][20381] Updated weights for policy 1, policy_version 1598308 (0.0008) [2023-12-27 16:43:48,329][20381] Updated weights for policy 1, policy_version 1598318 (0.0007) [2023-12-27 16:43:48,391][20381] Updated weights for policy 1, policy_version 1598328 (0.0009) [2023-12-27 16:43:48,786][20347] Updated weights for policy 0, policy_version 1599921 (0.0007) [2023-12-27 16:43:48,848][20347] Updated weights for policy 0, policy_version 1599931 (0.0006) [2023-12-27 16:43:48,905][20347] Updated weights for policy 0, policy_version 1599941 (0.0006) [2023-12-27 16:43:48,971][20347] Updated weights for policy 0, policy_version 1599951 (0.0006) [2023-12-27 16:43:49,126][20381] Updated weights for policy 1, policy_version 1598338 (0.0007) [2023-12-27 16:43:49,180][20381] Updated weights for policy 1, policy_version 1598348 (0.0007) [2023-12-27 16:43:49,241][20381] Updated weights for policy 1, policy_version 1598358 (0.0008) [2023-12-27 16:43:49,636][20347] Updated weights for policy 0, policy_version 1599961 (0.0010) [2023-12-27 16:43:49,693][20347] Updated weights for policy 0, policy_version 1599971 (0.0010) [2023-12-27 16:43:49,753][20347] Updated weights for policy 0, policy_version 1599981 (0.0010) [2023-12-27 16:43:49,966][20381] Updated weights for policy 1, policy_version 1598368 (0.0009) [2023-12-27 16:43:50,019][20381] Updated weights for policy 1, policy_version 1598378 (0.0008) [2023-12-27 16:43:50,071][20381] Updated weights for policy 1, policy_version 1598388 (0.0007) [2023-12-27 16:43:50,473][20347] Updated weights for policy 0, policy_version 1599991 (0.0009) [2023-12-27 16:43:50,526][20347] Updated weights for policy 0, policy_version 1600001 (0.0007) [2023-12-27 16:43:50,591][20347] Updated weights for policy 0, policy_version 1600011 (0.0008) [2023-12-27 16:43:50,636][20381] Updated weights for policy 1, policy_version 1598398 (0.0007) [2023-12-27 16:43:50,700][20381] Updated weights for policy 1, policy_version 1598408 (0.0008) [2023-12-27 16:43:50,760][20381] Updated weights for policy 1, policy_version 1598418 (0.0007) [2023-12-27 16:43:51,302][20347] Updated weights for policy 0, policy_version 1600021 (0.0006) [2023-12-27 16:43:51,365][20347] Updated weights for policy 0, policy_version 1600031 (0.0007) [2023-12-27 16:43:51,422][20347] Updated weights for policy 0, policy_version 1600041 (0.0009) [2023-12-27 16:43:51,465][20381] Updated weights for policy 1, policy_version 1598428 (0.0008) [2023-12-27 16:43:51,516][20381] Updated weights for policy 1, policy_version 1598438 (0.0007) [2023-12-27 16:43:51,570][20381] Updated weights for policy 1, policy_version 1598448 (0.0008) [2023-12-27 16:43:52,080][20347] Updated weights for policy 0, policy_version 1600051 (0.0007) [2023-12-27 16:43:52,140][20347] Updated weights for policy 0, policy_version 1600061 (0.0008) [2023-12-27 16:43:52,202][20347] Updated weights for policy 0, policy_version 1600071 (0.0009) [2023-12-27 16:43:52,360][20381] Updated weights for policy 1, policy_version 1598458 (0.0008) [2023-12-27 16:43:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 818946048. Throughput: 0: 10196.2, 1: 10068.4. Samples: 818915956. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.030')] [2023-12-27 16:43:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001600080_409698304.pth... [2023-12-27 16:43:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001598896_409395200.pth [2023-12-27 16:43:52,415][20381] Updated weights for policy 1, policy_version 1598468 (0.0009) [2023-12-27 16:43:52,469][20381] Updated weights for policy 1, policy_version 1598478 (0.0009) [2023-12-27 16:43:52,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001598488_409255936.pth... [2023-12-27 16:43:52,522][20381] Updated weights for policy 1, policy_version 1598488 (0.0008) [2023-12-27 16:43:52,525][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001597272_408944640.pth [2023-12-27 16:43:52,861][20347] Updated weights for policy 0, policy_version 1600081 (0.0010) [2023-12-27 16:43:52,911][20347] Updated weights for policy 0, policy_version 1600091 (0.0006) [2023-12-27 16:43:52,963][20347] Updated weights for policy 0, policy_version 1600101 (0.0008) [2023-12-27 16:43:53,022][20347] Updated weights for policy 0, policy_version 1600111 (0.0005) [2023-12-27 16:43:53,283][20381] Updated weights for policy 1, policy_version 1598498 (0.0007) [2023-12-27 16:43:53,339][20381] Updated weights for policy 1, policy_version 1598508 (0.0007) [2023-12-27 16:43:53,400][20381] Updated weights for policy 1, policy_version 1598518 (0.0008) [2023-12-27 16:43:53,664][20347] Updated weights for policy 0, policy_version 1600121 (0.0010) [2023-12-27 16:43:53,709][20347] Updated weights for policy 0, policy_version 1600131 (0.0007) [2023-12-27 16:43:53,758][20347] Updated weights for policy 0, policy_version 1600141 (0.0009) [2023-12-27 16:43:54,097][20381] Updated weights for policy 1, policy_version 1598528 (0.0009) [2023-12-27 16:43:54,152][20381] Updated weights for policy 1, policy_version 1598538 (0.0007) [2023-12-27 16:43:54,209][20381] Updated weights for policy 1, policy_version 1598548 (0.0007) [2023-12-27 16:43:54,361][20347] Updated weights for policy 0, policy_version 1600151 (0.0010) [2023-12-27 16:43:54,427][20347] Updated weights for policy 0, policy_version 1600161 (0.0010) [2023-12-27 16:43:54,485][20347] Updated weights for policy 0, policy_version 1600171 (0.0011) [2023-12-27 16:43:54,921][20381] Updated weights for policy 1, policy_version 1598558 (0.0007) [2023-12-27 16:43:54,969][20381] Updated weights for policy 1, policy_version 1598568 (0.0008) [2023-12-27 16:43:55,027][20381] Updated weights for policy 1, policy_version 1598578 (0.0007) [2023-12-27 16:43:55,158][20347] Updated weights for policy 0, policy_version 1600181 (0.0010) [2023-12-27 16:43:55,214][20347] Updated weights for policy 0, policy_version 1600191 (0.0010) [2023-12-27 16:43:55,275][20347] Updated weights for policy 0, policy_version 1600201 (0.0010) [2023-12-27 16:43:55,587][20381] Updated weights for policy 1, policy_version 1598588 (0.0007) [2023-12-27 16:43:55,638][20381] Updated weights for policy 1, policy_version 1598598 (0.0007) [2023-12-27 16:43:55,686][20381] Updated weights for policy 1, policy_version 1598608 (0.0010) [2023-12-27 16:43:55,886][20347] Updated weights for policy 0, policy_version 1600211 (0.0009) [2023-12-27 16:43:55,943][20347] Updated weights for policy 0, policy_version 1600221 (0.0006) [2023-12-27 16:43:56,004][20347] Updated weights for policy 0, policy_version 1600231 (0.0010) [2023-12-27 16:43:56,363][20381] Updated weights for policy 1, policy_version 1598618 (0.0011) [2023-12-27 16:43:56,411][20381] Updated weights for policy 1, policy_version 1598628 (0.0010) [2023-12-27 16:43:56,465][20381] Updated weights for policy 1, policy_version 1598638 (0.0010) [2023-12-27 16:43:56,520][20381] Updated weights for policy 1, policy_version 1598648 (0.0010) [2023-12-27 16:43:56,590][20347] Updated weights for policy 0, policy_version 1600241 (0.0010) [2023-12-27 16:43:56,647][20347] Updated weights for policy 0, policy_version 1600251 (0.0009) [2023-12-27 16:43:56,705][20347] Updated weights for policy 0, policy_version 1600261 (0.0010) [2023-12-27 16:43:56,755][20347] Updated weights for policy 0, policy_version 1600271 (0.0010) [2023-12-27 16:43:57,189][20381] Updated weights for policy 1, policy_version 1598658 (0.0010) [2023-12-27 16:43:57,233][20381] Updated weights for policy 1, policy_version 1598668 (0.0010) [2023-12-27 16:43:57,277][20381] Updated weights for policy 1, policy_version 1598678 (0.0010) [2023-12-27 16:43:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 819052544. Throughput: 0: 10233.1, 1: 10069.7. Samples: 819039724. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:43:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '765.530')] [2023-12-27 16:43:57,420][20347] Updated weights for policy 0, policy_version 1600281 (0.0007) [2023-12-27 16:43:57,464][20347] Updated weights for policy 0, policy_version 1600291 (0.0007) [2023-12-27 16:43:57,510][20347] Updated weights for policy 0, policy_version 1600301 (0.0007) [2023-12-27 16:43:57,929][20381] Updated weights for policy 1, policy_version 1598688 (0.0010) [2023-12-27 16:43:57,977][20381] Updated weights for policy 1, policy_version 1598698 (0.0010) [2023-12-27 16:43:58,028][20381] Updated weights for policy 1, policy_version 1598708 (0.0010) [2023-12-27 16:43:58,179][20347] Updated weights for policy 0, policy_version 1600311 (0.0007) [2023-12-27 16:43:58,245][20347] Updated weights for policy 0, policy_version 1600321 (0.0009) [2023-12-27 16:43:58,302][20347] Updated weights for policy 0, policy_version 1600331 (0.0007) [2023-12-27 16:43:58,793][20381] Updated weights for policy 1, policy_version 1598718 (0.0009) [2023-12-27 16:43:58,866][20381] Updated weights for policy 1, policy_version 1598728 (0.0008) [2023-12-27 16:43:58,920][20381] Updated weights for policy 1, policy_version 1598738 (0.0007) [2023-12-27 16:43:59,065][20347] Updated weights for policy 0, policy_version 1600341 (0.0007) [2023-12-27 16:43:59,125][20347] Updated weights for policy 0, policy_version 1600351 (0.0006) [2023-12-27 16:43:59,184][20347] Updated weights for policy 0, policy_version 1600361 (0.0007) [2023-12-27 16:43:59,668][20381] Updated weights for policy 1, policy_version 1598748 (0.0007) [2023-12-27 16:43:59,717][20381] Updated weights for policy 1, policy_version 1598758 (0.0009) [2023-12-27 16:43:59,770][20381] Updated weights for policy 1, policy_version 1598768 (0.0007) [2023-12-27 16:43:59,923][20347] Updated weights for policy 0, policy_version 1600371 (0.0007) [2023-12-27 16:43:59,982][20347] Updated weights for policy 0, policy_version 1600381 (0.0008) [2023-12-27 16:44:00,034][20347] Updated weights for policy 0, policy_version 1600391 (0.0007) [2023-12-27 16:44:00,446][20381] Updated weights for policy 1, policy_version 1598778 (0.0007) [2023-12-27 16:44:00,503][20381] Updated weights for policy 1, policy_version 1598788 (0.0007) [2023-12-27 16:44:00,556][20381] Updated weights for policy 1, policy_version 1598798 (0.0011) [2023-12-27 16:44:00,612][20381] Updated weights for policy 1, policy_version 1598808 (0.0011) [2023-12-27 16:44:00,703][20347] Updated weights for policy 0, policy_version 1600401 (0.0006) [2023-12-27 16:44:00,749][20347] Updated weights for policy 0, policy_version 1600411 (0.0007) [2023-12-27 16:44:00,792][20347] Updated weights for policy 0, policy_version 1600421 (0.0007) [2023-12-27 16:44:00,840][20347] Updated weights for policy 0, policy_version 1600431 (0.0007) [2023-12-27 16:44:01,253][20381] Updated weights for policy 1, policy_version 1598818 (0.0008) [2023-12-27 16:44:01,315][20381] Updated weights for policy 1, policy_version 1598828 (0.0008) [2023-12-27 16:44:01,370][20381] Updated weights for policy 1, policy_version 1598838 (0.0008) [2023-12-27 16:44:01,535][20347] Updated weights for policy 0, policy_version 1600441 (0.0006) [2023-12-27 16:44:01,586][20347] Updated weights for policy 0, policy_version 1600451 (0.0007) [2023-12-27 16:44:01,646][20347] Updated weights for policy 0, policy_version 1600461 (0.0008) [2023-12-27 16:44:02,051][20381] Updated weights for policy 1, policy_version 1598848 (0.0006) [2023-12-27 16:44:02,111][20381] Updated weights for policy 1, policy_version 1598858 (0.0005) [2023-12-27 16:44:02,168][20381] Updated weights for policy 1, policy_version 1598868 (0.0008) [2023-12-27 16:44:02,340][20347] Updated weights for policy 0, policy_version 1600471 (0.0006) [2023-12-27 16:44:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 819150848. Throughput: 0: 10190.1, 1: 10121.0. Samples: 819161936. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:44:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.620')] [2023-12-27 16:44:02,406][20347] Updated weights for policy 0, policy_version 1600481 (0.0009) [2023-12-27 16:44:02,458][20347] Updated weights for policy 0, policy_version 1600491 (0.0007) [2023-12-27 16:44:02,844][20381] Updated weights for policy 1, policy_version 1598878 (0.0007) [2023-12-27 16:44:02,902][20381] Updated weights for policy 1, policy_version 1598888 (0.0007) [2023-12-27 16:44:02,954][20381] Updated weights for policy 1, policy_version 1598898 (0.0005) [2023-12-27 16:44:03,159][20347] Updated weights for policy 0, policy_version 1600501 (0.0007) [2023-12-27 16:44:03,218][20347] Updated weights for policy 0, policy_version 1600511 (0.0008) [2023-12-27 16:44:03,270][20347] Updated weights for policy 0, policy_version 1600521 (0.0008) [2023-12-27 16:44:03,474][20381] Updated weights for policy 1, policy_version 1598908 (0.0005) [2023-12-27 16:44:03,522][20381] Updated weights for policy 1, policy_version 1598918 (0.0005) [2023-12-27 16:44:03,574][20381] Updated weights for policy 1, policy_version 1598928 (0.0005) [2023-12-27 16:44:04,033][20347] Updated weights for policy 0, policy_version 1600531 (0.0008) [2023-12-27 16:44:04,083][20347] Updated weights for policy 0, policy_version 1600541 (0.0007) [2023-12-27 16:44:04,132][20347] Updated weights for policy 0, policy_version 1600551 (0.0007) [2023-12-27 16:44:04,151][20381] Updated weights for policy 1, policy_version 1598938 (0.0005) [2023-12-27 16:44:04,215][20381] Updated weights for policy 1, policy_version 1598948 (0.0006) [2023-12-27 16:44:04,276][20381] Updated weights for policy 1, policy_version 1598958 (0.0008) [2023-12-27 16:44:04,335][20381] Updated weights for policy 1, policy_version 1598968 (0.0007) [2023-12-27 16:44:04,844][20347] Updated weights for policy 0, policy_version 1600561 (0.0007) [2023-12-27 16:44:04,896][20347] Updated weights for policy 0, policy_version 1600571 (0.0007) [2023-12-27 16:44:04,941][20347] Updated weights for policy 0, policy_version 1600581 (0.0006) [2023-12-27 16:44:04,961][20381] Updated weights for policy 1, policy_version 1598978 (0.0008) [2023-12-27 16:44:04,990][20347] Updated weights for policy 0, policy_version 1600591 (0.0008) [2023-12-27 16:44:05,023][20381] Updated weights for policy 1, policy_version 1598988 (0.0007) [2023-12-27 16:44:05,084][20381] Updated weights for policy 1, policy_version 1598998 (0.0008) [2023-12-27 16:44:05,646][20347] Updated weights for policy 0, policy_version 1600601 (0.0005) [2023-12-27 16:44:05,702][20347] Updated weights for policy 0, policy_version 1600611 (0.0005) [2023-12-27 16:44:05,713][20381] Updated weights for policy 1, policy_version 1599008 (0.0008) [2023-12-27 16:44:05,752][20347] Updated weights for policy 0, policy_version 1600621 (0.0006) [2023-12-27 16:44:05,775][20381] Updated weights for policy 1, policy_version 1599018 (0.0007) [2023-12-27 16:44:05,839][20381] Updated weights for policy 1, policy_version 1599028 (0.0007) [2023-12-27 16:44:06,422][20347] Updated weights for policy 0, policy_version 1600631 (0.0010) [2023-12-27 16:44:06,477][20347] Updated weights for policy 0, policy_version 1600641 (0.0010) [2023-12-27 16:44:06,536][20347] Updated weights for policy 0, policy_version 1600651 (0.0010) [2023-12-27 16:44:06,542][20381] Updated weights for policy 1, policy_version 1599038 (0.0009) [2023-12-27 16:44:06,608][20381] Updated weights for policy 1, policy_version 1599048 (0.0011) [2023-12-27 16:44:06,668][20381] Updated weights for policy 1, policy_version 1599058 (0.0011) [2023-12-27 16:44:07,203][20347] Updated weights for policy 0, policy_version 1600661 (0.0008) [2023-12-27 16:44:07,262][20347] Updated weights for policy 0, policy_version 1600671 (0.0010) [2023-12-27 16:44:07,316][20347] Updated weights for policy 0, policy_version 1600681 (0.0010) [2023-12-27 16:44:07,336][20381] Updated weights for policy 1, policy_version 1599068 (0.0008) [2023-12-27 16:44:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 819257344. Throughput: 0: 10159.0, 1: 10232.0. Samples: 819224520. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '672.640')] [2023-12-27 16:44:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001600688_409853952.pth... [2023-12-27 16:44:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001599472_409542656.pth [2023-12-27 16:44:07,394][20381] Updated weights for policy 1, policy_version 1599078 (0.0011) [2023-12-27 16:44:07,454][20381] Updated weights for policy 1, policy_version 1599088 (0.0010) [2023-12-27 16:44:07,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001599096_409411584.pth... [2023-12-27 16:44:07,501][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001597880_409100288.pth [2023-12-27 16:44:07,980][20347] Updated weights for policy 0, policy_version 1600691 (0.0010) [2023-12-27 16:44:08,024][20347] Updated weights for policy 0, policy_version 1600701 (0.0005) [2023-12-27 16:44:08,070][20347] Updated weights for policy 0, policy_version 1600711 (0.0005) [2023-12-27 16:44:08,112][20381] Updated weights for policy 1, policy_version 1599098 (0.0010) [2023-12-27 16:44:08,166][20381] Updated weights for policy 1, policy_version 1599108 (0.0007) [2023-12-27 16:44:08,213][20381] Updated weights for policy 1, policy_version 1599118 (0.0008) [2023-12-27 16:44:08,265][20381] Updated weights for policy 1, policy_version 1599128 (0.0007) [2023-12-27 16:44:08,700][20347] Updated weights for policy 0, policy_version 1600721 (0.0005) [2023-12-27 16:44:08,760][20347] Updated weights for policy 0, policy_version 1600731 (0.0006) [2023-12-27 16:44:08,821][20347] Updated weights for policy 0, policy_version 1600741 (0.0010) [2023-12-27 16:44:08,882][20347] Updated weights for policy 0, policy_version 1600751 (0.0010) [2023-12-27 16:44:09,046][20381] Updated weights for policy 1, policy_version 1599138 (0.0007) [2023-12-27 16:44:09,094][20381] Updated weights for policy 1, policy_version 1599148 (0.0007) [2023-12-27 16:44:09,143][20381] Updated weights for policy 1, policy_version 1599158 (0.0007) [2023-12-27 16:44:09,568][20347] Updated weights for policy 0, policy_version 1600761 (0.0007) [2023-12-27 16:44:09,627][20347] Updated weights for policy 0, policy_version 1600771 (0.0010) [2023-12-27 16:44:09,686][20347] Updated weights for policy 0, policy_version 1600781 (0.0010) [2023-12-27 16:44:09,894][20381] Updated weights for policy 1, policy_version 1599168 (0.0007) [2023-12-27 16:44:09,961][20381] Updated weights for policy 1, policy_version 1599178 (0.0007) [2023-12-27 16:44:10,025][20381] Updated weights for policy 1, policy_version 1599188 (0.0007) [2023-12-27 16:44:10,355][20347] Updated weights for policy 0, policy_version 1600791 (0.0007) [2023-12-27 16:44:10,414][20347] Updated weights for policy 0, policy_version 1600801 (0.0008) [2023-12-27 16:44:10,474][20347] Updated weights for policy 0, policy_version 1600811 (0.0007) [2023-12-27 16:44:10,703][20381] Updated weights for policy 1, policy_version 1599198 (0.0007) [2023-12-27 16:44:10,750][20381] Updated weights for policy 1, policy_version 1599208 (0.0008) [2023-12-27 16:44:10,795][20381] Updated weights for policy 1, policy_version 1599218 (0.0008) [2023-12-27 16:44:11,069][20347] Updated weights for policy 0, policy_version 1600821 (0.0007) [2023-12-27 16:44:11,137][20347] Updated weights for policy 0, policy_version 1600831 (0.0008) [2023-12-27 16:44:11,192][20347] Updated weights for policy 0, policy_version 1600841 (0.0008) [2023-12-27 16:44:11,544][20381] Updated weights for policy 1, policy_version 1599228 (0.0008) [2023-12-27 16:44:11,610][20381] Updated weights for policy 1, policy_version 1599238 (0.0007) [2023-12-27 16:44:11,684][20381] Updated weights for policy 1, policy_version 1599248 (0.0008) [2023-12-27 16:44:11,922][20347] Updated weights for policy 0, policy_version 1600851 (0.0007) [2023-12-27 16:44:11,979][20347] Updated weights for policy 0, policy_version 1600861 (0.0005) [2023-12-27 16:44:12,037][20347] Updated weights for policy 0, policy_version 1600871 (0.0009) [2023-12-27 16:44:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 819355648. Throughput: 0: 10229.9, 1: 10269.5. Samples: 819347980. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:44:12,363][19170] Avg episode reward: [(0, '990.320'), (1, '722.780')] [2023-12-27 16:44:12,398][20381] Updated weights for policy 1, policy_version 1599258 (0.0008) [2023-12-27 16:44:12,452][20381] Updated weights for policy 1, policy_version 1599268 (0.0007) [2023-12-27 16:44:12,513][20381] Updated weights for policy 1, policy_version 1599278 (0.0007) [2023-12-27 16:44:12,571][20381] Updated weights for policy 1, policy_version 1599288 (0.0007) [2023-12-27 16:44:12,740][20347] Updated weights for policy 0, policy_version 1600881 (0.0010) [2023-12-27 16:44:12,802][20347] Updated weights for policy 0, policy_version 1600891 (0.0006) [2023-12-27 16:44:12,854][20347] Updated weights for policy 0, policy_version 1600901 (0.0010) [2023-12-27 16:44:12,911][20347] Updated weights for policy 0, policy_version 1600911 (0.0009) [2023-12-27 16:44:13,283][20381] Updated weights for policy 1, policy_version 1599298 (0.0007) [2023-12-27 16:44:13,340][20381] Updated weights for policy 1, policy_version 1599308 (0.0008) [2023-12-27 16:44:13,400][20381] Updated weights for policy 1, policy_version 1599318 (0.0007) [2023-12-27 16:44:13,540][20347] Updated weights for policy 0, policy_version 1600921 (0.0006) [2023-12-27 16:44:13,595][20347] Updated weights for policy 0, policy_version 1600931 (0.0007) [2023-12-27 16:44:13,645][20347] Updated weights for policy 0, policy_version 1600941 (0.0008) [2023-12-27 16:44:14,096][20381] Updated weights for policy 1, policy_version 1599328 (0.0008) [2023-12-27 16:44:14,148][20381] Updated weights for policy 1, policy_version 1599338 (0.0008) [2023-12-27 16:44:14,200][20347] Updated weights for policy 0, policy_version 1600951 (0.0006) [2023-12-27 16:44:14,209][20381] Updated weights for policy 1, policy_version 1599349 (0.0011) [2023-12-27 16:44:14,244][20347] Updated weights for policy 0, policy_version 1600961 (0.0006) [2023-12-27 16:44:14,291][20347] Updated weights for policy 0, policy_version 1600971 (0.0005) [2023-12-27 16:44:14,894][20347] Updated weights for policy 0, policy_version 1600981 (0.0007) [2023-12-27 16:44:14,947][20381] Updated weights for policy 1, policy_version 1599359 (0.0006) [2023-12-27 16:44:14,949][20347] Updated weights for policy 0, policy_version 1600991 (0.0007) [2023-12-27 16:44:15,003][20347] Updated weights for policy 0, policy_version 1601001 (0.0006) [2023-12-27 16:44:15,005][20381] Updated weights for policy 1, policy_version 1599369 (0.0006) [2023-12-27 16:44:15,064][20381] Updated weights for policy 1, policy_version 1599379 (0.0007) [2023-12-27 16:44:15,647][20381] Updated weights for policy 1, policy_version 1599389 (0.0008) [2023-12-27 16:44:15,698][20381] Updated weights for policy 1, policy_version 1599399 (0.0008) [2023-12-27 16:44:15,711][20347] Updated weights for policy 0, policy_version 1601011 (0.0006) [2023-12-27 16:44:15,751][20381] Updated weights for policy 1, policy_version 1599409 (0.0007) [2023-12-27 16:44:15,773][20347] Updated weights for policy 0, policy_version 1601021 (0.0006) [2023-12-27 16:44:15,829][20347] Updated weights for policy 0, policy_version 1601031 (0.0007) [2023-12-27 16:44:16,359][20381] Updated weights for policy 1, policy_version 1599419 (0.0007) [2023-12-27 16:44:16,420][20381] Updated weights for policy 1, policy_version 1599429 (0.0008) [2023-12-27 16:44:16,483][20381] Updated weights for policy 1, policy_version 1599439 (0.0008) [2023-12-27 16:44:16,536][20347] Updated weights for policy 0, policy_version 1601041 (0.0008) [2023-12-27 16:44:16,599][20347] Updated weights for policy 0, policy_version 1601051 (0.0008) [2023-12-27 16:44:16,650][20347] Updated weights for policy 0, policy_version 1601061 (0.0010) [2023-12-27 16:44:16,708][20347] Updated weights for policy 0, policy_version 1601071 (0.0010) [2023-12-27 16:44:17,194][20381] Updated weights for policy 1, policy_version 1599449 (0.0007) [2023-12-27 16:44:17,245][20381] Updated weights for policy 1, policy_version 1599459 (0.0007) [2023-12-27 16:44:17,295][20381] Updated weights for policy 1, policy_version 1599469 (0.0007) [2023-12-27 16:44:17,340][20347] Updated weights for policy 0, policy_version 1601081 (0.0010) [2023-12-27 16:44:17,349][20381] Updated weights for policy 1, policy_version 1599479 (0.0006) [2023-12-27 16:44:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 819462144. Throughput: 0: 10278.8, 1: 10206.7. Samples: 819471760. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-12-27 16:44:17,364][19170] Avg episode reward: [(0, '990.320'), (1, '652.060')] [2023-12-27 16:44:17,394][20347] Updated weights for policy 0, policy_version 1601091 (0.0010) [2023-12-27 16:44:17,452][20347] Updated weights for policy 0, policy_version 1601101 (0.0010) [2023-12-27 16:44:18,017][20381] Updated weights for policy 1, policy_version 1599489 (0.0007) [2023-12-27 16:44:18,075][20381] Updated weights for policy 1, policy_version 1599499 (0.0006) [2023-12-27 16:44:18,128][20381] Updated weights for policy 1, policy_version 1599509 (0.0006) [2023-12-27 16:44:18,133][20347] Updated weights for policy 0, policy_version 1601111 (0.0010) [2023-12-27 16:44:18,182][20347] Updated weights for policy 0, policy_version 1601121 (0.0010) [2023-12-27 16:44:18,230][20347] Updated weights for policy 0, policy_version 1601131 (0.0010) [2023-12-27 16:44:18,833][20381] Updated weights for policy 1, policy_version 1599519 (0.0007) [2023-12-27 16:44:18,893][20381] Updated weights for policy 1, policy_version 1599529 (0.0007) [2023-12-27 16:44:18,904][20347] Updated weights for policy 0, policy_version 1601141 (0.0009) [2023-12-27 16:44:18,955][20381] Updated weights for policy 1, policy_version 1599539 (0.0007) [2023-12-27 16:44:18,966][20347] Updated weights for policy 0, policy_version 1601151 (0.0008) [2023-12-27 16:44:19,032][20347] Updated weights for policy 0, policy_version 1601161 (0.0008) [2023-12-27 16:44:19,663][20381] Updated weights for policy 1, policy_version 1599549 (0.0006) [2023-12-27 16:44:19,701][20347] Updated weights for policy 0, policy_version 1601171 (0.0008) [2023-12-27 16:44:19,723][20381] Updated weights for policy 1, policy_version 1599559 (0.0007) [2023-12-27 16:44:19,769][20347] Updated weights for policy 0, policy_version 1601181 (0.0007) [2023-12-27 16:44:19,779][20381] Updated weights for policy 1, policy_version 1599569 (0.0008) [2023-12-27 16:44:19,835][20347] Updated weights for policy 0, policy_version 1601191 (0.0008) [2023-12-27 16:44:20,492][20381] Updated weights for policy 1, policy_version 1599579 (0.0008) [2023-12-27 16:44:20,549][20381] Updated weights for policy 1, policy_version 1599589 (0.0007) [2023-12-27 16:44:20,610][20381] Updated weights for policy 1, policy_version 1599599 (0.0009) [2023-12-27 16:44:20,631][20347] Updated weights for policy 0, policy_version 1601201 (0.0008) [2023-12-27 16:44:20,694][20347] Updated weights for policy 0, policy_version 1601211 (0.0008) [2023-12-27 16:44:20,753][20347] Updated weights for policy 0, policy_version 1601221 (0.0008) [2023-12-27 16:44:20,815][20347] Updated weights for policy 0, policy_version 1601231 (0.0008) [2023-12-27 16:44:21,311][20381] Updated weights for policy 1, policy_version 1599609 (0.0007) [2023-12-27 16:44:21,378][20381] Updated weights for policy 1, policy_version 1599619 (0.0011) [2023-12-27 16:44:21,445][20381] Updated weights for policy 1, policy_version 1599629 (0.0008) [2023-12-27 16:44:21,504][20381] Updated weights for policy 1, policy_version 1599639 (0.0007) [2023-12-27 16:44:21,550][20347] Updated weights for policy 0, policy_version 1601241 (0.0007) [2023-12-27 16:44:21,608][20347] Updated weights for policy 0, policy_version 1601251 (0.0006) [2023-12-27 16:44:21,670][20347] Updated weights for policy 0, policy_version 1601261 (0.0009) [2023-12-27 16:44:22,203][20381] Updated weights for policy 1, policy_version 1599649 (0.0008) [2023-12-27 16:44:22,263][20381] Updated weights for policy 1, policy_version 1599659 (0.0008) [2023-12-27 16:44:22,328][20381] Updated weights for policy 1, policy_version 1599669 (0.0007) [2023-12-27 16:44:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 819560448. Throughput: 0: 10252.8, 1: 10178.3. Samples: 819530928. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '736.240')] [2023-12-27 16:44:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001599672_409559040.pth... [2023-12-27 16:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001598488_409255936.pth [2023-12-27 16:44:22,378][20347] Updated weights for policy 0, policy_version 1601271 (0.0008) [2023-12-27 16:44:22,435][20347] Updated weights for policy 0, policy_version 1601281 (0.0008) [2023-12-27 16:44:22,491][20347] Updated weights for policy 0, policy_version 1601291 (0.0008) [2023-12-27 16:44:22,519][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001601296_410009600.pth... [2023-12-27 16:44:22,523][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001600080_409698304.pth [2023-12-27 16:44:23,018][20381] Updated weights for policy 1, policy_version 1599679 (0.0008) [2023-12-27 16:44:23,076][20381] Updated weights for policy 1, policy_version 1599689 (0.0009) [2023-12-27 16:44:23,132][20381] Updated weights for policy 1, policy_version 1599699 (0.0008) [2023-12-27 16:44:23,185][20347] Updated weights for policy 0, policy_version 1601301 (0.0009) [2023-12-27 16:44:23,244][20347] Updated weights for policy 0, policy_version 1601311 (0.0008) [2023-12-27 16:44:23,300][20347] Updated weights for policy 0, policy_version 1601321 (0.0007) [2023-12-27 16:44:23,823][20381] Updated weights for policy 1, policy_version 1599709 (0.0006) [2023-12-27 16:44:23,872][20381] Updated weights for policy 1, policy_version 1599719 (0.0007) [2023-12-27 16:44:23,916][20381] Updated weights for policy 1, policy_version 1599729 (0.0007) [2023-12-27 16:44:23,981][20347] Updated weights for policy 0, policy_version 1601331 (0.0007) [2023-12-27 16:44:24,033][20347] Updated weights for policy 0, policy_version 1601341 (0.0006) [2023-12-27 16:44:24,082][20347] Updated weights for policy 0, policy_version 1601351 (0.0007) [2023-12-27 16:44:24,622][20381] Updated weights for policy 1, policy_version 1599739 (0.0007) [2023-12-27 16:44:24,669][20381] Updated weights for policy 1, policy_version 1599749 (0.0007) [2023-12-27 16:44:24,714][20381] Updated weights for policy 1, policy_version 1599759 (0.0008) [2023-12-27 16:44:24,763][20347] Updated weights for policy 0, policy_version 1601361 (0.0007) [2023-12-27 16:44:24,818][20347] Updated weights for policy 0, policy_version 1601371 (0.0008) [2023-12-27 16:44:24,874][20347] Updated weights for policy 0, policy_version 1601381 (0.0007) [2023-12-27 16:44:24,934][20347] Updated weights for policy 0, policy_version 1601391 (0.0006) [2023-12-27 16:44:25,387][20381] Updated weights for policy 1, policy_version 1599769 (0.0007) [2023-12-27 16:44:25,442][20381] Updated weights for policy 1, policy_version 1599779 (0.0008) [2023-12-27 16:44:25,491][20381] Updated weights for policy 1, policy_version 1599789 (0.0008) [2023-12-27 16:44:25,535][20381] Updated weights for policy 1, policy_version 1599799 (0.0007) [2023-12-27 16:44:25,610][20347] Updated weights for policy 0, policy_version 1601401 (0.0007) [2023-12-27 16:44:25,662][20347] Updated weights for policy 0, policy_version 1601411 (0.0008) [2023-12-27 16:44:25,713][20347] Updated weights for policy 0, policy_version 1601421 (0.0007) [2023-12-27 16:44:26,217][20381] Updated weights for policy 1, policy_version 1599809 (0.0009) [2023-12-27 16:44:26,268][20381] Updated weights for policy 1, policy_version 1599819 (0.0005) [2023-12-27 16:44:26,324][20381] Updated weights for policy 1, policy_version 1599829 (0.0005) [2023-12-27 16:44:26,384][20347] Updated weights for policy 0, policy_version 1601431 (0.0008) [2023-12-27 16:44:26,432][20347] Updated weights for policy 0, policy_version 1601441 (0.0008) [2023-12-27 16:44:26,482][20347] Updated weights for policy 0, policy_version 1601451 (0.0007) [2023-12-27 16:44:26,851][20381] Updated weights for policy 1, policy_version 1599839 (0.0010) [2023-12-27 16:44:26,902][20381] Updated weights for policy 1, policy_version 1599849 (0.0010) [2023-12-27 16:44:26,956][20381] Updated weights for policy 1, policy_version 1599859 (0.0008) [2023-12-27 16:44:27,115][20347] Updated weights for policy 0, policy_version 1601461 (0.0005) [2023-12-27 16:44:27,179][20347] Updated weights for policy 0, policy_version 1601471 (0.0006) [2023-12-27 16:44:27,239][20347] Updated weights for policy 0, policy_version 1601481 (0.0010) [2023-12-27 16:44:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 819666944. Throughput: 0: 10240.4, 1: 10141.0. Samples: 819650476. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '764.530')] [2023-12-27 16:44:27,634][20381] Updated weights for policy 1, policy_version 1599869 (0.0008) [2023-12-27 16:44:27,684][20381] Updated weights for policy 1, policy_version 1599879 (0.0007) [2023-12-27 16:44:27,735][20381] Updated weights for policy 1, policy_version 1599889 (0.0006) [2023-12-27 16:44:27,854][20347] Updated weights for policy 0, policy_version 1601491 (0.0009) [2023-12-27 16:44:27,908][20347] Updated weights for policy 0, policy_version 1601501 (0.0006) [2023-12-27 16:44:27,967][20347] Updated weights for policy 0, policy_version 1601511 (0.0010) [2023-12-27 16:44:28,304][20381] Updated weights for policy 1, policy_version 1599899 (0.0005) [2023-12-27 16:44:28,376][20381] Updated weights for policy 1, policy_version 1599909 (0.0007) [2023-12-27 16:44:28,437][20381] Updated weights for policy 1, policy_version 1599919 (0.0008) [2023-12-27 16:44:28,646][20347] Updated weights for policy 0, policy_version 1601521 (0.0010) [2023-12-27 16:44:28,708][20347] Updated weights for policy 0, policy_version 1601531 (0.0007) [2023-12-27 16:44:28,767][20347] Updated weights for policy 0, policy_version 1601541 (0.0008) [2023-12-27 16:44:28,827][20347] Updated weights for policy 0, policy_version 1601551 (0.0007) [2023-12-27 16:44:29,034][20381] Updated weights for policy 1, policy_version 1599929 (0.0008) [2023-12-27 16:44:29,098][20381] Updated weights for policy 1, policy_version 1599939 (0.0008) [2023-12-27 16:44:29,161][20381] Updated weights for policy 1, policy_version 1599949 (0.0008) [2023-12-27 16:44:29,225][20381] Updated weights for policy 1, policy_version 1599959 (0.0008) [2023-12-27 16:44:29,492][20347] Updated weights for policy 0, policy_version 1601561 (0.0006) [2023-12-27 16:44:29,547][20347] Updated weights for policy 0, policy_version 1601571 (0.0008) [2023-12-27 16:44:29,605][20347] Updated weights for policy 0, policy_version 1601581 (0.0008) [2023-12-27 16:44:29,941][20381] Updated weights for policy 1, policy_version 1599969 (0.0008) [2023-12-27 16:44:29,990][20381] Updated weights for policy 1, policy_version 1599979 (0.0007) [2023-12-27 16:44:30,043][20381] Updated weights for policy 1, policy_version 1599989 (0.0008) [2023-12-27 16:44:30,291][20347] Updated weights for policy 0, policy_version 1601591 (0.0007) [2023-12-27 16:44:30,351][20347] Updated weights for policy 0, policy_version 1601601 (0.0007) [2023-12-27 16:44:30,410][20347] Updated weights for policy 0, policy_version 1601611 (0.0007) [2023-12-27 16:44:30,763][20381] Updated weights for policy 1, policy_version 1599999 (0.0010) [2023-12-27 16:44:30,807][20381] Updated weights for policy 1, policy_version 1600009 (0.0010) [2023-12-27 16:44:30,851][20381] Updated weights for policy 1, policy_version 1600019 (0.0010) [2023-12-27 16:44:31,070][20347] Updated weights for policy 0, policy_version 1601621 (0.0008) [2023-12-27 16:44:31,115][20347] Updated weights for policy 0, policy_version 1601631 (0.0008) [2023-12-27 16:44:31,177][20347] Updated weights for policy 0, policy_version 1601641 (0.0008) [2023-12-27 16:44:31,639][20381] Updated weights for policy 1, policy_version 1600029 (0.0009) [2023-12-27 16:44:31,701][20381] Updated weights for policy 1, policy_version 1600039 (0.0008) [2023-12-27 16:44:31,759][20381] Updated weights for policy 1, policy_version 1600049 (0.0008) [2023-12-27 16:44:31,905][20347] Updated weights for policy 0, policy_version 1601651 (0.0006) [2023-12-27 16:44:31,962][20347] Updated weights for policy 0, policy_version 1601661 (0.0007) [2023-12-27 16:44:32,020][20347] Updated weights for policy 0, policy_version 1601671 (0.0007) [2023-12-27 16:44:32,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 819765248. Throughput: 0: 10288.8, 1: 10189.1. Samples: 819776660. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.390')] [2023-12-27 16:44:32,471][20381] Updated weights for policy 1, policy_version 1600059 (0.0007) [2023-12-27 16:44:32,519][20381] Updated weights for policy 1, policy_version 1600069 (0.0007) [2023-12-27 16:44:32,568][20381] Updated weights for policy 1, policy_version 1600079 (0.0007) [2023-12-27 16:44:32,643][20347] Updated weights for policy 0, policy_version 1601681 (0.0008) [2023-12-27 16:44:32,698][20347] Updated weights for policy 0, policy_version 1601691 (0.0007) [2023-12-27 16:44:32,759][20347] Updated weights for policy 0, policy_version 1601701 (0.0008) [2023-12-27 16:44:32,825][20347] Updated weights for policy 0, policy_version 1601711 (0.0007) [2023-12-27 16:44:33,169][20381] Updated weights for policy 1, policy_version 1600089 (0.0008) [2023-12-27 16:44:33,224][20381] Updated weights for policy 1, policy_version 1600099 (0.0006) [2023-12-27 16:44:33,280][20381] Updated weights for policy 1, policy_version 1600109 (0.0007) [2023-12-27 16:44:33,330][20381] Updated weights for policy 1, policy_version 1600119 (0.0008) [2023-12-27 16:44:33,479][20347] Updated weights for policy 0, policy_version 1601721 (0.0010) [2023-12-27 16:44:33,532][20347] Updated weights for policy 0, policy_version 1601731 (0.0007) [2023-12-27 16:44:33,577][20347] Updated weights for policy 0, policy_version 1601741 (0.0010) [2023-12-27 16:44:34,005][20381] Updated weights for policy 1, policy_version 1600129 (0.0005) [2023-12-27 16:44:34,057][20381] Updated weights for policy 1, policy_version 1600139 (0.0005) [2023-12-27 16:44:34,107][20381] Updated weights for policy 1, policy_version 1600149 (0.0005) [2023-12-27 16:44:34,250][20347] Updated weights for policy 0, policy_version 1601751 (0.0010) [2023-12-27 16:44:34,306][20347] Updated weights for policy 0, policy_version 1601761 (0.0010) [2023-12-27 16:44:34,358][20347] Updated weights for policy 0, policy_version 1601771 (0.0010) [2023-12-27 16:44:34,757][20381] Updated weights for policy 1, policy_version 1600159 (0.0007) [2023-12-27 16:44:34,817][20381] Updated weights for policy 1, policy_version 1600169 (0.0007) [2023-12-27 16:44:34,881][20381] Updated weights for policy 1, policy_version 1600179 (0.0006) [2023-12-27 16:44:35,071][20347] Updated weights for policy 0, policy_version 1601781 (0.0011) [2023-12-27 16:44:35,126][20347] Updated weights for policy 0, policy_version 1601791 (0.0010) [2023-12-27 16:44:35,177][20347] Updated weights for policy 0, policy_version 1601801 (0.0010) [2023-12-27 16:44:35,517][20381] Updated weights for policy 1, policy_version 1600189 (0.0007) [2023-12-27 16:44:35,576][20381] Updated weights for policy 1, policy_version 1600199 (0.0007) [2023-12-27 16:44:35,636][20381] Updated weights for policy 1, policy_version 1600209 (0.0006) [2023-12-27 16:44:35,850][20347] Updated weights for policy 0, policy_version 1601811 (0.0010) [2023-12-27 16:44:35,898][20347] Updated weights for policy 0, policy_version 1601821 (0.0010) [2023-12-27 16:44:35,946][20347] Updated weights for policy 0, policy_version 1601831 (0.0010) [2023-12-27 16:44:36,260][20381] Updated weights for policy 1, policy_version 1600219 (0.0005) [2023-12-27 16:44:36,319][20381] Updated weights for policy 1, policy_version 1600229 (0.0006) [2023-12-27 16:44:36,380][20381] Updated weights for policy 1, policy_version 1600239 (0.0008) [2023-12-27 16:44:36,669][20347] Updated weights for policy 0, policy_version 1601841 (0.0010) [2023-12-27 16:44:36,730][20347] Updated weights for policy 0, policy_version 1601851 (0.0010) [2023-12-27 16:44:36,793][20347] Updated weights for policy 0, policy_version 1601861 (0.0010) [2023-12-27 16:44:36,842][20347] Updated weights for policy 0, policy_version 1601871 (0.0010) [2023-12-27 16:44:36,939][20381] Updated weights for policy 1, policy_version 1600249 (0.0006) [2023-12-27 16:44:36,997][20381] Updated weights for policy 1, policy_version 1600259 (0.0011) [2023-12-27 16:44:37,044][20381] Updated weights for policy 1, policy_version 1600269 (0.0010) [2023-12-27 16:44:37,092][20381] Updated weights for policy 1, policy_version 1600279 (0.0010) [2023-12-27 16:44:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 819871744. Throughput: 0: 10288.3, 1: 10255.3. Samples: 819840416. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '769.990')] [2023-12-27 16:44:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001601872_410157056.pth... [2023-12-27 16:44:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001600280_409714688.pth... [2023-12-27 16:44:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001600688_409853952.pth [2023-12-27 16:44:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001599096_409411584.pth [2023-12-27 16:44:37,471][20347] Updated weights for policy 0, policy_version 1601881 (0.0006) [2023-12-27 16:44:37,528][20347] Updated weights for policy 0, policy_version 1601891 (0.0006) [2023-12-27 16:44:37,583][20347] Updated weights for policy 0, policy_version 1601901 (0.0010) [2023-12-27 16:44:37,759][20381] Updated weights for policy 1, policy_version 1600289 (0.0011) [2023-12-27 16:44:37,812][20381] Updated weights for policy 1, policy_version 1600299 (0.0011) [2023-12-27 16:44:37,860][20381] Updated weights for policy 1, policy_version 1600309 (0.0011) [2023-12-27 16:44:38,209][20347] Updated weights for policy 0, policy_version 1601911 (0.0010) [2023-12-27 16:44:38,267][20347] Updated weights for policy 0, policy_version 1601921 (0.0010) [2023-12-27 16:44:38,329][20347] Updated weights for policy 0, policy_version 1601931 (0.0010) [2023-12-27 16:44:38,511][20381] Updated weights for policy 1, policy_version 1600319 (0.0008) [2023-12-27 16:44:38,574][20381] Updated weights for policy 1, policy_version 1600329 (0.0008) [2023-12-27 16:44:38,634][20381] Updated weights for policy 1, policy_version 1600339 (0.0008) [2023-12-27 16:44:39,002][20347] Updated weights for policy 0, policy_version 1601941 (0.0008) [2023-12-27 16:44:39,054][20347] Updated weights for policy 0, policy_version 1601951 (0.0005) [2023-12-27 16:44:39,114][20347] Updated weights for policy 0, policy_version 1601961 (0.0005) [2023-12-27 16:44:39,363][20381] Updated weights for policy 1, policy_version 1600349 (0.0008) [2023-12-27 16:44:39,435][20381] Updated weights for policy 1, policy_version 1600359 (0.0009) [2023-12-27 16:44:39,491][20381] Updated weights for policy 1, policy_version 1600369 (0.0008) [2023-12-27 16:44:39,815][20347] Updated weights for policy 0, policy_version 1601971 (0.0006) [2023-12-27 16:44:39,883][20347] Updated weights for policy 0, policy_version 1601981 (0.0008) [2023-12-27 16:44:39,938][20347] Updated weights for policy 0, policy_version 1601991 (0.0007) [2023-12-27 16:44:40,274][20381] Updated weights for policy 1, policy_version 1600379 (0.0008) [2023-12-27 16:44:40,329][20381] Updated weights for policy 1, policy_version 1600389 (0.0008) [2023-12-27 16:44:40,385][20381] Updated weights for policy 1, policy_version 1600399 (0.0007) [2023-12-27 16:44:40,609][20347] Updated weights for policy 0, policy_version 1602001 (0.0008) [2023-12-27 16:44:40,662][20347] Updated weights for policy 0, policy_version 1602011 (0.0008) [2023-12-27 16:44:40,724][20347] Updated weights for policy 0, policy_version 1602021 (0.0009) [2023-12-27 16:44:40,789][20347] Updated weights for policy 0, policy_version 1602031 (0.0006) [2023-12-27 16:44:41,072][20381] Updated weights for policy 1, policy_version 1600409 (0.0008) [2023-12-27 16:44:41,130][20381] Updated weights for policy 1, policy_version 1600419 (0.0006) [2023-12-27 16:44:41,208][20381] Updated weights for policy 1, policy_version 1600429 (0.0007) [2023-12-27 16:44:41,278][20381] Updated weights for policy 1, policy_version 1600439 (0.0008) [2023-12-27 16:44:41,558][20347] Updated weights for policy 0, policy_version 1602041 (0.0008) [2023-12-27 16:44:41,612][20347] Updated weights for policy 0, policy_version 1602051 (0.0007) [2023-12-27 16:44:41,678][20347] Updated weights for policy 0, policy_version 1602061 (0.0007) [2023-12-27 16:44:41,990][20381] Updated weights for policy 1, policy_version 1600449 (0.0008) [2023-12-27 16:44:42,048][20381] Updated weights for policy 1, policy_version 1600459 (0.0008) [2023-12-27 16:44:42,102][20381] Updated weights for policy 1, policy_version 1600469 (0.0007) [2023-12-27 16:44:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 819970048. Throughput: 0: 10266.9, 1: 10269.2. Samples: 819963844. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '514.840')] [2023-12-27 16:44:42,402][20347] Updated weights for policy 0, policy_version 1602071 (0.0007) [2023-12-27 16:44:42,462][20347] Updated weights for policy 0, policy_version 1602081 (0.0005) [2023-12-27 16:44:42,525][20347] Updated weights for policy 0, policy_version 1602091 (0.0005) [2023-12-27 16:44:42,803][20381] Updated weights for policy 1, policy_version 1600479 (0.0007) [2023-12-27 16:44:42,852][20381] Updated weights for policy 1, policy_version 1600489 (0.0007) [2023-12-27 16:44:42,904][20381] Updated weights for policy 1, policy_version 1600499 (0.0009) [2023-12-27 16:44:43,154][20347] Updated weights for policy 0, policy_version 1602101 (0.0006) [2023-12-27 16:44:43,210][20347] Updated weights for policy 0, policy_version 1602111 (0.0006) [2023-12-27 16:44:43,268][20347] Updated weights for policy 0, policy_version 1602121 (0.0008) [2023-12-27 16:44:43,631][20381] Updated weights for policy 1, policy_version 1600509 (0.0008) [2023-12-27 16:44:43,684][20381] Updated weights for policy 1, policy_version 1600519 (0.0008) [2023-12-27 16:44:43,733][20381] Updated weights for policy 1, policy_version 1600529 (0.0007) [2023-12-27 16:44:43,934][20347] Updated weights for policy 0, policy_version 1602131 (0.0007) [2023-12-27 16:44:43,983][20347] Updated weights for policy 0, policy_version 1602141 (0.0008) [2023-12-27 16:44:44,035][20347] Updated weights for policy 0, policy_version 1602151 (0.0008) [2023-12-27 16:44:44,451][20381] Updated weights for policy 1, policy_version 1600539 (0.0008) [2023-12-27 16:44:44,503][20381] Updated weights for policy 1, policy_version 1600549 (0.0008) [2023-12-27 16:44:44,557][20381] Updated weights for policy 1, policy_version 1600559 (0.0008) [2023-12-27 16:44:44,770][20347] Updated weights for policy 0, policy_version 1602161 (0.0008) [2023-12-27 16:44:44,839][20347] Updated weights for policy 0, policy_version 1602171 (0.0007) [2023-12-27 16:44:44,903][20347] Updated weights for policy 0, policy_version 1602181 (0.0008) [2023-12-27 16:44:44,967][20347] Updated weights for policy 0, policy_version 1602191 (0.0008) [2023-12-27 16:44:45,259][20381] Updated weights for policy 1, policy_version 1600569 (0.0008) [2023-12-27 16:44:45,317][20381] Updated weights for policy 1, policy_version 1600579 (0.0007) [2023-12-27 16:44:45,376][20381] Updated weights for policy 1, policy_version 1600589 (0.0008) [2023-12-27 16:44:45,436][20381] Updated weights for policy 1, policy_version 1600599 (0.0008) [2023-12-27 16:44:45,552][20347] Updated weights for policy 0, policy_version 1602201 (0.0007) [2023-12-27 16:44:45,610][20347] Updated weights for policy 0, policy_version 1602211 (0.0008) [2023-12-27 16:44:45,668][20347] Updated weights for policy 0, policy_version 1602221 (0.0008) [2023-12-27 16:44:46,114][20381] Updated weights for policy 1, policy_version 1600609 (0.0006) [2023-12-27 16:44:46,171][20381] Updated weights for policy 1, policy_version 1600619 (0.0007) [2023-12-27 16:44:46,231][20381] Updated weights for policy 1, policy_version 1600629 (0.0006) [2023-12-27 16:44:46,329][20347] Updated weights for policy 0, policy_version 1602231 (0.0007) [2023-12-27 16:44:46,378][20347] Updated weights for policy 0, policy_version 1602241 (0.0010) [2023-12-27 16:44:46,436][20347] Updated weights for policy 0, policy_version 1602251 (0.0010) [2023-12-27 16:44:46,930][20381] Updated weights for policy 1, policy_version 1600639 (0.0007) [2023-12-27 16:44:46,986][20381] Updated weights for policy 1, policy_version 1600649 (0.0008) [2023-12-27 16:44:47,045][20381] Updated weights for policy 1, policy_version 1600659 (0.0007) [2023-12-27 16:44:47,159][20347] Updated weights for policy 0, policy_version 1602261 (0.0011) [2023-12-27 16:44:47,211][20347] Updated weights for policy 0, policy_version 1602271 (0.0010) [2023-12-27 16:44:47,270][20347] Updated weights for policy 0, policy_version 1602281 (0.0010) [2023-12-27 16:44:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 820076544. Throughput: 0: 10275.2, 1: 10218.7. Samples: 820084160. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '610.380')] [2023-12-27 16:44:47,743][20381] Updated weights for policy 1, policy_version 1600669 (0.0008) [2023-12-27 16:44:47,803][20381] Updated weights for policy 1, policy_version 1600679 (0.0008) [2023-12-27 16:44:47,860][20381] Updated weights for policy 1, policy_version 1600689 (0.0008) [2023-12-27 16:44:47,920][20347] Updated weights for policy 0, policy_version 1602291 (0.0010) [2023-12-27 16:44:47,976][20347] Updated weights for policy 0, policy_version 1602301 (0.0006) [2023-12-27 16:44:48,037][20347] Updated weights for policy 0, policy_version 1602311 (0.0005) [2023-12-27 16:44:48,462][20381] Updated weights for policy 1, policy_version 1600699 (0.0006) [2023-12-27 16:44:48,513][20381] Updated weights for policy 1, policy_version 1600709 (0.0008) [2023-12-27 16:44:48,564][20381] Updated weights for policy 1, policy_version 1600719 (0.0008) [2023-12-27 16:44:48,652][20347] Updated weights for policy 0, policy_version 1602321 (0.0006) [2023-12-27 16:44:48,718][20347] Updated weights for policy 0, policy_version 1602331 (0.0011) [2023-12-27 16:44:48,790][20347] Updated weights for policy 0, policy_version 1602341 (0.0011) [2023-12-27 16:44:48,856][20347] Updated weights for policy 0, policy_version 1602351 (0.0010) [2023-12-27 16:44:49,224][20381] Updated weights for policy 1, policy_version 1600729 (0.0008) [2023-12-27 16:44:49,287][20381] Updated weights for policy 1, policy_version 1600739 (0.0008) [2023-12-27 16:44:49,340][20381] Updated weights for policy 1, policy_version 1600749 (0.0009) [2023-12-27 16:44:49,399][20381] Updated weights for policy 1, policy_version 1600759 (0.0007) [2023-12-27 16:44:49,554][20347] Updated weights for policy 0, policy_version 1602361 (0.0009) [2023-12-27 16:44:49,613][20347] Updated weights for policy 0, policy_version 1602371 (0.0007) [2023-12-27 16:44:49,672][20347] Updated weights for policy 0, policy_version 1602381 (0.0010) [2023-12-27 16:44:50,104][20381] Updated weights for policy 1, policy_version 1600769 (0.0008) [2023-12-27 16:44:50,159][20381] Updated weights for policy 1, policy_version 1600779 (0.0008) [2023-12-27 16:44:50,207][20381] Updated weights for policy 1, policy_version 1600789 (0.0007) [2023-12-27 16:44:50,358][20347] Updated weights for policy 0, policy_version 1602391 (0.0008) [2023-12-27 16:44:50,419][20347] Updated weights for policy 0, policy_version 1602401 (0.0008) [2023-12-27 16:44:50,478][20347] Updated weights for policy 0, policy_version 1602411 (0.0008) [2023-12-27 16:44:50,984][20381] Updated weights for policy 1, policy_version 1600799 (0.0007) [2023-12-27 16:44:51,045][20381] Updated weights for policy 1, policy_version 1600809 (0.0008) [2023-12-27 16:44:51,107][20381] Updated weights for policy 1, policy_version 1600819 (0.0007) [2023-12-27 16:44:51,176][20347] Updated weights for policy 0, policy_version 1602421 (0.0008) [2023-12-27 16:44:51,235][20347] Updated weights for policy 0, policy_version 1602431 (0.0008) [2023-12-27 16:44:51,296][20347] Updated weights for policy 0, policy_version 1602441 (0.0008) [2023-12-27 16:44:51,850][20381] Updated weights for policy 1, policy_version 1600829 (0.0007) [2023-12-27 16:44:51,908][20381] Updated weights for policy 1, policy_version 1600839 (0.0005) [2023-12-27 16:44:51,965][20381] Updated weights for policy 1, policy_version 1600849 (0.0007) [2023-12-27 16:44:52,029][20347] Updated weights for policy 0, policy_version 1602451 (0.0008) [2023-12-27 16:44:52,082][20347] Updated weights for policy 0, policy_version 1602461 (0.0007) [2023-12-27 16:44:52,136][20347] Updated weights for policy 0, policy_version 1602471 (0.0008) [2023-12-27 16:44:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 820174848. Throughput: 0: 10312.8, 1: 10162.9. Samples: 820145928. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '748.700')] [2023-12-27 16:44:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001602480_410312704.pth... [2023-12-27 16:44:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001600856_409862144.pth... [2023-12-27 16:44:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001599672_409559040.pth [2023-12-27 16:44:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001601296_410009600.pth [2023-12-27 16:44:52,594][20381] Updated weights for policy 1, policy_version 1600859 (0.0007) [2023-12-27 16:44:52,644][20381] Updated weights for policy 1, policy_version 1600869 (0.0006) [2023-12-27 16:44:52,696][20381] Updated weights for policy 1, policy_version 1600879 (0.0007) [2023-12-27 16:44:52,869][20347] Updated weights for policy 0, policy_version 1602481 (0.0008) [2023-12-27 16:44:52,920][20347] Updated weights for policy 0, policy_version 1602491 (0.0007) [2023-12-27 16:44:52,980][20347] Updated weights for policy 0, policy_version 1602501 (0.0008) [2023-12-27 16:44:53,043][20347] Updated weights for policy 0, policy_version 1602511 (0.0007) [2023-12-27 16:44:53,348][20381] Updated weights for policy 1, policy_version 1600889 (0.0007) [2023-12-27 16:44:53,393][20381] Updated weights for policy 1, policy_version 1600899 (0.0007) [2023-12-27 16:44:53,441][20381] Updated weights for policy 1, policy_version 1600909 (0.0008) [2023-12-27 16:44:53,489][20381] Updated weights for policy 1, policy_version 1600919 (0.0007) [2023-12-27 16:44:53,733][20347] Updated weights for policy 0, policy_version 1602521 (0.0007) [2023-12-27 16:44:53,779][20347] Updated weights for policy 0, policy_version 1602531 (0.0007) [2023-12-27 16:44:53,828][20347] Updated weights for policy 0, policy_version 1602541 (0.0007) [2023-12-27 16:44:54,209][20381] Updated weights for policy 1, policy_version 1600929 (0.0008) [2023-12-27 16:44:54,271][20381] Updated weights for policy 1, policy_version 1600939 (0.0009) [2023-12-27 16:44:54,334][20381] Updated weights for policy 1, policy_version 1600949 (0.0008) [2023-12-27 16:44:54,494][20347] Updated weights for policy 0, policy_version 1602551 (0.0009) [2023-12-27 16:44:54,549][20347] Updated weights for policy 0, policy_version 1602561 (0.0005) [2023-12-27 16:44:54,607][20347] Updated weights for policy 0, policy_version 1602571 (0.0007) [2023-12-27 16:44:55,053][20381] Updated weights for policy 1, policy_version 1600959 (0.0008) [2023-12-27 16:44:55,113][20381] Updated weights for policy 1, policy_version 1600969 (0.0009) [2023-12-27 16:44:55,177][20381] Updated weights for policy 1, policy_version 1600979 (0.0010) [2023-12-27 16:44:55,186][20347] Updated weights for policy 0, policy_version 1602581 (0.0006) [2023-12-27 16:44:55,242][20347] Updated weights for policy 0, policy_version 1602591 (0.0006) [2023-12-27 16:44:55,307][20347] Updated weights for policy 0, policy_version 1602601 (0.0006) [2023-12-27 16:44:55,829][20381] Updated weights for policy 1, policy_version 1600989 (0.0008) [2023-12-27 16:44:55,876][20381] Updated weights for policy 1, policy_version 1600999 (0.0007) [2023-12-27 16:44:55,899][20347] Updated weights for policy 0, policy_version 1602611 (0.0008) [2023-12-27 16:44:55,929][20381] Updated weights for policy 1, policy_version 1601009 (0.0009) [2023-12-27 16:44:55,955][20347] Updated weights for policy 0, policy_version 1602621 (0.0010) [2023-12-27 16:44:56,009][20347] Updated weights for policy 0, policy_version 1602631 (0.0010) [2023-12-27 16:44:56,573][20381] Updated weights for policy 1, policy_version 1601019 (0.0006) [2023-12-27 16:44:56,626][20381] Updated weights for policy 1, policy_version 1601029 (0.0007) [2023-12-27 16:44:56,670][20381] Updated weights for policy 1, policy_version 1601039 (0.0007) [2023-12-27 16:44:56,691][20347] Updated weights for policy 0, policy_version 1602641 (0.0010) [2023-12-27 16:44:56,740][20347] Updated weights for policy 0, policy_version 1602651 (0.0010) [2023-12-27 16:44:56,785][20347] Updated weights for policy 0, policy_version 1602661 (0.0010) [2023-12-27 16:44:56,829][20347] Updated weights for policy 0, policy_version 1602671 (0.0010) [2023-12-27 16:44:57,337][20381] Updated weights for policy 1, policy_version 1601049 (0.0006) [2023-12-27 16:44:57,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 820273152. Throughput: 0: 10256.6, 1: 10165.1. Samples: 820266960. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:44:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '759.240')] [2023-12-27 16:44:57,391][20381] Updated weights for policy 1, policy_version 1601059 (0.0007) [2023-12-27 16:44:57,447][20381] Updated weights for policy 1, policy_version 1601069 (0.0007) [2023-12-27 16:44:57,488][20347] Updated weights for policy 0, policy_version 1602681 (0.0010) [2023-12-27 16:44:57,497][20381] Updated weights for policy 1, policy_version 1601079 (0.0006) [2023-12-27 16:44:57,542][20347] Updated weights for policy 0, policy_version 1602691 (0.0010) [2023-12-27 16:44:57,586][20347] Updated weights for policy 0, policy_version 1602701 (0.0007) [2023-12-27 16:44:58,178][20381] Updated weights for policy 1, policy_version 1601089 (0.0006) [2023-12-27 16:44:58,193][20347] Updated weights for policy 0, policy_version 1602711 (0.0009) [2023-12-27 16:44:58,241][20381] Updated weights for policy 1, policy_version 1601099 (0.0007) [2023-12-27 16:44:58,251][20347] Updated weights for policy 0, policy_version 1602721 (0.0006) [2023-12-27 16:44:58,295][20381] Updated weights for policy 1, policy_version 1601109 (0.0006) [2023-12-27 16:44:58,310][20347] Updated weights for policy 0, policy_version 1602731 (0.0007) [2023-12-27 16:44:59,048][20381] Updated weights for policy 1, policy_version 1601119 (0.0008) [2023-12-27 16:44:59,111][20347] Updated weights for policy 0, policy_version 1602741 (0.0007) [2023-12-27 16:44:59,112][20381] Updated weights for policy 1, policy_version 1601129 (0.0008) [2023-12-27 16:44:59,165][20381] Updated weights for policy 1, policy_version 1601139 (0.0008) [2023-12-27 16:44:59,166][20347] Updated weights for policy 0, policy_version 1602751 (0.0006) [2023-12-27 16:44:59,218][20347] Updated weights for policy 0, policy_version 1602761 (0.0007) [2023-12-27 16:44:59,854][20381] Updated weights for policy 1, policy_version 1601149 (0.0008) [2023-12-27 16:44:59,910][20381] Updated weights for policy 1, policy_version 1601159 (0.0008) [2023-12-27 16:44:59,969][20347] Updated weights for policy 0, policy_version 1602771 (0.0008) [2023-12-27 16:44:59,972][20381] Updated weights for policy 1, policy_version 1601169 (0.0008) [2023-12-27 16:45:00,023][20347] Updated weights for policy 0, policy_version 1602781 (0.0007) [2023-12-27 16:45:00,080][20347] Updated weights for policy 0, policy_version 1602791 (0.0007) [2023-12-27 16:45:00,643][20381] Updated weights for policy 1, policy_version 1601179 (0.0007) [2023-12-27 16:45:00,703][20381] Updated weights for policy 1, policy_version 1601189 (0.0006) [2023-12-27 16:45:00,750][20381] Updated weights for policy 1, policy_version 1601199 (0.0005) [2023-12-27 16:45:00,793][20347] Updated weights for policy 0, policy_version 1602801 (0.0008) [2023-12-27 16:45:00,840][20347] Updated weights for policy 0, policy_version 1602811 (0.0010) [2023-12-27 16:45:00,883][20347] Updated weights for policy 0, policy_version 1602821 (0.0010) [2023-12-27 16:45:00,941][20347] Updated weights for policy 0, policy_version 1602831 (0.0008) [2023-12-27 16:45:01,368][20381] Updated weights for policy 1, policy_version 1601209 (0.0006) [2023-12-27 16:45:01,426][20381] Updated weights for policy 1, policy_version 1601219 (0.0009) [2023-12-27 16:45:01,484][20381] Updated weights for policy 1, policy_version 1601229 (0.0007) [2023-12-27 16:45:01,542][20381] Updated weights for policy 1, policy_version 1601239 (0.0009) [2023-12-27 16:45:01,667][20347] Updated weights for policy 0, policy_version 1602841 (0.0007) [2023-12-27 16:45:01,729][20347] Updated weights for policy 0, policy_version 1602851 (0.0008) [2023-12-27 16:45:01,790][20347] Updated weights for policy 0, policy_version 1602861 (0.0006) [2023-12-27 16:45:02,274][20381] Updated weights for policy 1, policy_version 1601249 (0.0008) [2023-12-27 16:45:02,327][20381] Updated weights for policy 1, policy_version 1601259 (0.0008) [2023-12-27 16:45:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 820371456. Throughput: 0: 10196.0, 1: 10172.9. Samples: 820388356. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '767.680')] [2023-12-27 16:45:02,388][20381] Updated weights for policy 1, policy_version 1601269 (0.0008) [2023-12-27 16:45:02,417][20347] Updated weights for policy 0, policy_version 1602871 (0.0006) [2023-12-27 16:45:02,476][20347] Updated weights for policy 0, policy_version 1602881 (0.0007) [2023-12-27 16:45:02,534][20347] Updated weights for policy 0, policy_version 1602891 (0.0007) [2023-12-27 16:45:03,050][20381] Updated weights for policy 1, policy_version 1601279 (0.0008) [2023-12-27 16:45:03,100][20381] Updated weights for policy 1, policy_version 1601289 (0.0007) [2023-12-27 16:45:03,143][20381] Updated weights for policy 1, policy_version 1601299 (0.0008) [2023-12-27 16:45:03,230][20347] Updated weights for policy 0, policy_version 1602901 (0.0009) [2023-12-27 16:45:03,287][20347] Updated weights for policy 0, policy_version 1602911 (0.0010) [2023-12-27 16:45:03,345][20347] Updated weights for policy 0, policy_version 1602921 (0.0010) [2023-12-27 16:45:03,731][20381] Updated weights for policy 1, policy_version 1601309 (0.0006) [2023-12-27 16:45:03,782][20381] Updated weights for policy 1, policy_version 1601319 (0.0005) [2023-12-27 16:45:03,836][20381] Updated weights for policy 1, policy_version 1601329 (0.0006) [2023-12-27 16:45:03,985][20347] Updated weights for policy 0, policy_version 1602931 (0.0010) [2023-12-27 16:45:04,044][20347] Updated weights for policy 0, policy_version 1602941 (0.0010) [2023-12-27 16:45:04,106][20347] Updated weights for policy 0, policy_version 1602951 (0.0010) [2023-12-27 16:45:04,511][20381] Updated weights for policy 1, policy_version 1601339 (0.0008) [2023-12-27 16:45:04,570][20381] Updated weights for policy 1, policy_version 1601349 (0.0008) [2023-12-27 16:45:04,628][20381] Updated weights for policy 1, policy_version 1601359 (0.0007) [2023-12-27 16:45:04,740][20347] Updated weights for policy 0, policy_version 1602961 (0.0010) [2023-12-27 16:45:04,805][20347] Updated weights for policy 0, policy_version 1602971 (0.0006) [2023-12-27 16:45:04,858][20347] Updated weights for policy 0, policy_version 1602981 (0.0006) [2023-12-27 16:45:04,909][20347] Updated weights for policy 0, policy_version 1602991 (0.0006) [2023-12-27 16:45:05,297][20381] Updated weights for policy 1, policy_version 1601369 (0.0007) [2023-12-27 16:45:05,356][20381] Updated weights for policy 1, policy_version 1601379 (0.0010) [2023-12-27 16:45:05,415][20381] Updated weights for policy 1, policy_version 1601389 (0.0009) [2023-12-27 16:45:05,473][20381] Updated weights for policy 1, policy_version 1601399 (0.0010) [2023-12-27 16:45:05,562][20347] Updated weights for policy 0, policy_version 1603001 (0.0010) [2023-12-27 16:45:05,617][20347] Updated weights for policy 0, policy_version 1603011 (0.0010) [2023-12-27 16:45:05,671][20347] Updated weights for policy 0, policy_version 1603021 (0.0010) [2023-12-27 16:45:06,103][20381] Updated weights for policy 1, policy_version 1601409 (0.0010) [2023-12-27 16:45:06,162][20381] Updated weights for policy 1, policy_version 1601419 (0.0011) [2023-12-27 16:45:06,223][20381] Updated weights for policy 1, policy_version 1601429 (0.0010) [2023-12-27 16:45:06,351][20347] Updated weights for policy 0, policy_version 1603031 (0.0008) [2023-12-27 16:45:06,408][20347] Updated weights for policy 0, policy_version 1603041 (0.0007) [2023-12-27 16:45:06,465][20347] Updated weights for policy 0, policy_version 1603051 (0.0008) [2023-12-27 16:45:06,899][20381] Updated weights for policy 1, policy_version 1601439 (0.0007) [2023-12-27 16:45:06,956][20381] Updated weights for policy 1, policy_version 1601449 (0.0005) [2023-12-27 16:45:07,012][20381] Updated weights for policy 1, policy_version 1601459 (0.0005) [2023-12-27 16:45:07,054][20347] Updated weights for policy 0, policy_version 1603061 (0.0006) [2023-12-27 16:45:07,111][20347] Updated weights for policy 0, policy_version 1603071 (0.0005) [2023-12-27 16:45:07,163][20347] Updated weights for policy 0, policy_version 1603081 (0.0005) [2023-12-27 16:45:07,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 820486144. Throughput: 0: 10232.3, 1: 10237.4. Samples: 820452064. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '679.210')] [2023-12-27 16:45:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001601464_410017792.pth... [2023-12-27 16:45:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001603088_410468352.pth... [2023-12-27 16:45:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001600280_409714688.pth [2023-12-27 16:45:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001601872_410157056.pth [2023-12-27 16:45:07,589][20381] Updated weights for policy 1, policy_version 1601469 (0.0005) [2023-12-27 16:45:07,640][20381] Updated weights for policy 1, policy_version 1601479 (0.0007) [2023-12-27 16:45:07,694][20381] Updated weights for policy 1, policy_version 1601489 (0.0008) [2023-12-27 16:45:07,783][20347] Updated weights for policy 0, policy_version 1603091 (0.0005) [2023-12-27 16:45:07,841][20347] Updated weights for policy 0, policy_version 1603101 (0.0010) [2023-12-27 16:45:07,887][20347] Updated weights for policy 0, policy_version 1603111 (0.0007) [2023-12-27 16:45:08,313][20381] Updated weights for policy 1, policy_version 1601499 (0.0008) [2023-12-27 16:45:08,376][20381] Updated weights for policy 1, policy_version 1601509 (0.0007) [2023-12-27 16:45:08,433][20381] Updated weights for policy 1, policy_version 1601519 (0.0009) [2023-12-27 16:45:08,569][20347] Updated weights for policy 0, policy_version 1603121 (0.0007) [2023-12-27 16:45:08,629][20347] Updated weights for policy 0, policy_version 1603131 (0.0010) [2023-12-27 16:45:08,687][20347] Updated weights for policy 0, policy_version 1603141 (0.0010) [2023-12-27 16:45:08,749][20347] Updated weights for policy 0, policy_version 1603151 (0.0010) [2023-12-27 16:45:09,139][20381] Updated weights for policy 1, policy_version 1601529 (0.0008) [2023-12-27 16:45:09,187][20381] Updated weights for policy 1, policy_version 1601539 (0.0007) [2023-12-27 16:45:09,252][20381] Updated weights for policy 1, policy_version 1601549 (0.0008) [2023-12-27 16:45:09,307][20381] Updated weights for policy 1, policy_version 1601559 (0.0009) [2023-12-27 16:45:09,459][20347] Updated weights for policy 0, policy_version 1603161 (0.0009) [2023-12-27 16:45:09,515][20347] Updated weights for policy 0, policy_version 1603171 (0.0009) [2023-12-27 16:45:09,576][20347] Updated weights for policy 0, policy_version 1603181 (0.0007) [2023-12-27 16:45:10,030][20381] Updated weights for policy 1, policy_version 1601569 (0.0006) [2023-12-27 16:45:10,088][20381] Updated weights for policy 1, policy_version 1601579 (0.0008) [2023-12-27 16:45:10,135][20381] Updated weights for policy 1, policy_version 1601589 (0.0009) [2023-12-27 16:45:10,256][20347] Updated weights for policy 0, policy_version 1603191 (0.0007) [2023-12-27 16:45:10,318][20347] Updated weights for policy 0, policy_version 1603201 (0.0008) [2023-12-27 16:45:10,376][20347] Updated weights for policy 0, policy_version 1603211 (0.0007) [2023-12-27 16:45:10,836][20381] Updated weights for policy 1, policy_version 1601599 (0.0007) [2023-12-27 16:45:10,889][20381] Updated weights for policy 1, policy_version 1601609 (0.0008) [2023-12-27 16:45:10,946][20381] Updated weights for policy 1, policy_version 1601619 (0.0005) [2023-12-27 16:45:11,068][20347] Updated weights for policy 0, policy_version 1603221 (0.0008) [2023-12-27 16:45:11,128][20347] Updated weights for policy 0, policy_version 1603231 (0.0007) [2023-12-27 16:45:11,196][20347] Updated weights for policy 0, policy_version 1603241 (0.0007) [2023-12-27 16:45:11,730][20381] Updated weights for policy 1, policy_version 1601629 (0.0008) [2023-12-27 16:45:11,751][20292] KL-divergence is very high: 346.7591 [2023-12-27 16:45:11,797][20381] Updated weights for policy 1, policy_version 1601639 (0.0006) [2023-12-27 16:45:11,805][20292] KL-divergence is very high: 633.7656 [2023-12-27 16:45:11,854][20292] KL-divergence is very high: 718.1096 [2023-12-27 16:45:11,860][20381] Updated weights for policy 1, policy_version 1601649 (0.0006) [2023-12-27 16:45:11,910][20347] Updated weights for policy 0, policy_version 1603251 (0.0008) [2023-12-27 16:45:11,967][20347] Updated weights for policy 0, policy_version 1603261 (0.0008) [2023-12-27 16:45:12,027][20347] Updated weights for policy 0, policy_version 1603271 (0.0008) [2023-12-27 16:45:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 820584448. Throughput: 0: 10293.2, 1: 10277.3. Samples: 820576152. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '737.550')] [2023-12-27 16:45:12,547][20381] Updated weights for policy 1, policy_version 1601659 (0.0006) [2023-12-27 16:45:12,603][20381] Updated weights for policy 1, policy_version 1601669 (0.0008) [2023-12-27 16:45:12,657][20381] Updated weights for policy 1, policy_version 1601679 (0.0008) [2023-12-27 16:45:12,714][20347] Updated weights for policy 0, policy_version 1603281 (0.0008) [2023-12-27 16:45:12,776][20347] Updated weights for policy 0, policy_version 1603291 (0.0007) [2023-12-27 16:45:12,835][20347] Updated weights for policy 0, policy_version 1603301 (0.0009) [2023-12-27 16:45:12,887][20347] Updated weights for policy 0, policy_version 1603311 (0.0006) [2023-12-27 16:45:13,328][20381] Updated weights for policy 1, policy_version 1601689 (0.0007) [2023-12-27 16:45:13,389][20381] Updated weights for policy 1, policy_version 1601699 (0.0008) [2023-12-27 16:45:13,449][20381] Updated weights for policy 1, policy_version 1601709 (0.0007) [2023-12-27 16:45:13,472][20347] Updated weights for policy 0, policy_version 1603321 (0.0009) [2023-12-27 16:45:13,503][20381] Updated weights for policy 1, policy_version 1601719 (0.0006) [2023-12-27 16:45:13,517][20347] Updated weights for policy 0, policy_version 1603331 (0.0010) [2023-12-27 16:45:13,571][20347] Updated weights for policy 0, policy_version 1603341 (0.0010) [2023-12-27 16:45:14,195][20381] Updated weights for policy 1, policy_version 1601729 (0.0007) [2023-12-27 16:45:14,236][20347] Updated weights for policy 0, policy_version 1603351 (0.0010) [2023-12-27 16:45:14,252][20381] Updated weights for policy 1, policy_version 1601739 (0.0007) [2023-12-27 16:45:14,284][20347] Updated weights for policy 0, policy_version 1603361 (0.0010) [2023-12-27 16:45:14,301][20381] Updated weights for policy 1, policy_version 1601749 (0.0005) [2023-12-27 16:45:14,335][20347] Updated weights for policy 0, policy_version 1603371 (0.0010) [2023-12-27 16:45:14,958][20347] Updated weights for policy 0, policy_version 1603381 (0.0006) [2023-12-27 16:45:14,981][20381] Updated weights for policy 1, policy_version 1601759 (0.0006) [2023-12-27 16:45:15,019][20347] Updated weights for policy 0, policy_version 1603391 (0.0011) [2023-12-27 16:45:15,044][20381] Updated weights for policy 1, policy_version 1601769 (0.0006) [2023-12-27 16:45:15,075][20347] Updated weights for policy 0, policy_version 1603401 (0.0010) [2023-12-27 16:45:15,104][20381] Updated weights for policy 1, policy_version 1601779 (0.0005) [2023-12-27 16:45:15,656][20381] Updated weights for policy 1, policy_version 1601789 (0.0006) [2023-12-27 16:45:15,700][20347] Updated weights for policy 0, policy_version 1603411 (0.0009) [2023-12-27 16:45:15,726][20381] Updated weights for policy 1, policy_version 1601799 (0.0007) [2023-12-27 16:45:15,758][20347] Updated weights for policy 0, policy_version 1603421 (0.0009) [2023-12-27 16:45:15,782][20381] Updated weights for policy 1, policy_version 1601809 (0.0005) [2023-12-27 16:45:15,810][20347] Updated weights for policy 0, policy_version 1603431 (0.0010) [2023-12-27 16:45:16,379][20381] Updated weights for policy 1, policy_version 1601819 (0.0006) [2023-12-27 16:45:16,435][20381] Updated weights for policy 1, policy_version 1601829 (0.0005) [2023-12-27 16:45:16,449][20347] Updated weights for policy 0, policy_version 1603441 (0.0010) [2023-12-27 16:45:16,484][20381] Updated weights for policy 1, policy_version 1601839 (0.0005) [2023-12-27 16:45:16,507][20347] Updated weights for policy 0, policy_version 1603451 (0.0010) [2023-12-27 16:45:16,562][20347] Updated weights for policy 0, policy_version 1603461 (0.0010) [2023-12-27 16:45:16,625][20347] Updated weights for policy 0, policy_version 1603471 (0.0010) [2023-12-27 16:45:17,157][20381] Updated weights for policy 1, policy_version 1601849 (0.0008) [2023-12-27 16:45:17,212][20381] Updated weights for policy 1, policy_version 1601859 (0.0006) [2023-12-27 16:45:17,263][20381] Updated weights for policy 1, policy_version 1601869 (0.0006) [2023-12-27 16:45:17,265][20347] Updated weights for policy 0, policy_version 1603481 (0.0010) [2023-12-27 16:45:17,323][20347] Updated weights for policy 0, policy_version 1603491 (0.0009) [2023-12-27 16:45:17,324][20381] Updated weights for policy 1, policy_version 1601879 (0.0008) [2023-12-27 16:45:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 820690944. Throughput: 0: 10294.3, 1: 10261.7. Samples: 820701680. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:17,363][19170] Avg episode reward: [(0, '990.330'), (1, '729.700')] [2023-12-27 16:45:17,381][20347] Updated weights for policy 0, policy_version 1603501 (0.0006) [2023-12-27 16:45:18,001][20381] Updated weights for policy 1, policy_version 1601889 (0.0007) [2023-12-27 16:45:18,059][20381] Updated weights for policy 1, policy_version 1601899 (0.0007) [2023-12-27 16:45:18,068][20347] Updated weights for policy 0, policy_version 1603511 (0.0007) [2023-12-27 16:45:18,107][20381] Updated weights for policy 1, policy_version 1601909 (0.0006) [2023-12-27 16:45:18,120][20347] Updated weights for policy 0, policy_version 1603521 (0.0006) [2023-12-27 16:45:18,167][20347] Updated weights for policy 0, policy_version 1603531 (0.0007) [2023-12-27 16:45:18,820][20381] Updated weights for policy 1, policy_version 1601919 (0.0007) [2023-12-27 16:45:18,881][20381] Updated weights for policy 1, policy_version 1601929 (0.0008) [2023-12-27 16:45:18,917][20347] Updated weights for policy 0, policy_version 1603541 (0.0007) [2023-12-27 16:45:18,938][20381] Updated weights for policy 1, policy_version 1601939 (0.0008) [2023-12-27 16:45:18,973][20347] Updated weights for policy 0, policy_version 1603551 (0.0006) [2023-12-27 16:45:19,025][20347] Updated weights for policy 0, policy_version 1603561 (0.0007) [2023-12-27 16:45:19,678][20381] Updated weights for policy 1, policy_version 1601949 (0.0007) [2023-12-27 16:45:19,709][20347] Updated weights for policy 0, policy_version 1603571 (0.0008) [2023-12-27 16:45:19,738][20381] Updated weights for policy 1, policy_version 1601959 (0.0007) [2023-12-27 16:45:19,771][20347] Updated weights for policy 0, policy_version 1603581 (0.0006) [2023-12-27 16:45:19,793][20381] Updated weights for policy 1, policy_version 1601969 (0.0006) [2023-12-27 16:45:19,834][20347] Updated weights for policy 0, policy_version 1603591 (0.0007) [2023-12-27 16:45:20,513][20347] Updated weights for policy 0, policy_version 1603601 (0.0007) [2023-12-27 16:45:20,554][20381] Updated weights for policy 1, policy_version 1601979 (0.0008) [2023-12-27 16:45:20,577][20347] Updated weights for policy 0, policy_version 1603611 (0.0008) [2023-12-27 16:45:20,616][20381] Updated weights for policy 1, policy_version 1601989 (0.0008) [2023-12-27 16:45:20,640][20347] Updated weights for policy 0, policy_version 1603621 (0.0007) [2023-12-27 16:45:20,683][20381] Updated weights for policy 1, policy_version 1601999 (0.0007) [2023-12-27 16:45:20,701][20347] Updated weights for policy 0, policy_version 1603631 (0.0007) [2023-12-27 16:45:21,455][20381] Updated weights for policy 1, policy_version 1602009 (0.0007) [2023-12-27 16:45:21,490][20347] Updated weights for policy 0, policy_version 1603641 (0.0006) [2023-12-27 16:45:21,523][20381] Updated weights for policy 1, policy_version 1602019 (0.0007) [2023-12-27 16:45:21,551][20347] Updated weights for policy 0, policy_version 1603651 (0.0006) [2023-12-27 16:45:21,586][20381] Updated weights for policy 1, policy_version 1602029 (0.0006) [2023-12-27 16:45:21,609][20347] Updated weights for policy 0, policy_version 1603661 (0.0008) [2023-12-27 16:45:21,652][20381] Updated weights for policy 1, policy_version 1602039 (0.0008) [2023-12-27 16:45:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 820781056. Throughput: 0: 10259.3, 1: 10181.9. Samples: 820760272. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:22,363][19170] Avg episode reward: [(0, '990.330'), (1, '674.440')] [2023-12-27 16:45:22,394][20381] Updated weights for policy 1, policy_version 1602049 (0.0008) [2023-12-27 16:45:22,399][20347] Updated weights for policy 0, policy_version 1603671 (0.0008) [2023-12-27 16:45:22,454][20381] Updated weights for policy 1, policy_version 1602059 (0.0007) [2023-12-27 16:45:22,456][20347] Updated weights for policy 0, policy_version 1603681 (0.0007) [2023-12-27 16:45:22,512][20381] Updated weights for policy 1, policy_version 1602069 (0.0006) [2023-12-27 16:45:22,517][20347] Updated weights for policy 0, policy_version 1603691 (0.0007) [2023-12-27 16:45:22,530][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001602072_410173440.pth... [2023-12-27 16:45:22,534][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001600856_409862144.pth [2023-12-27 16:45:22,546][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001603696_410624000.pth... [2023-12-27 16:45:22,550][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001602480_410312704.pth [2023-12-27 16:45:23,162][20381] Updated weights for policy 1, policy_version 1602079 (0.0007) [2023-12-27 16:45:23,225][20381] Updated weights for policy 1, policy_version 1602089 (0.0009) [2023-12-27 16:45:23,269][20347] Updated weights for policy 0, policy_version 1603701 (0.0007) [2023-12-27 16:45:23,282][20381] Updated weights for policy 1, policy_version 1602099 (0.0006) [2023-12-27 16:45:23,327][20347] Updated weights for policy 0, policy_version 1603711 (0.0007) [2023-12-27 16:45:23,384][20347] Updated weights for policy 0, policy_version 1603721 (0.0007) [2023-12-27 16:45:23,930][20381] Updated weights for policy 1, policy_version 1602109 (0.0007) [2023-12-27 16:45:23,980][20381] Updated weights for policy 1, policy_version 1602119 (0.0007) [2023-12-27 16:45:24,031][20381] Updated weights for policy 1, policy_version 1602129 (0.0005) [2023-12-27 16:45:24,068][20347] Updated weights for policy 0, policy_version 1603731 (0.0007) [2023-12-27 16:45:24,129][20347] Updated weights for policy 0, policy_version 1603741 (0.0005) [2023-12-27 16:45:24,181][20347] Updated weights for policy 0, policy_version 1603751 (0.0005) [2023-12-27 16:45:24,645][20381] Updated weights for policy 1, policy_version 1602139 (0.0006) [2023-12-27 16:45:24,708][20381] Updated weights for policy 1, policy_version 1602149 (0.0007) [2023-12-27 16:45:24,764][20381] Updated weights for policy 1, policy_version 1602159 (0.0007) [2023-12-27 16:45:24,835][20347] Updated weights for policy 0, policy_version 1603761 (0.0007) [2023-12-27 16:45:24,889][20347] Updated weights for policy 0, policy_version 1603771 (0.0010) [2023-12-27 16:45:24,938][20347] Updated weights for policy 0, policy_version 1603781 (0.0010) [2023-12-27 16:45:24,983][20347] Updated weights for policy 0, policy_version 1603791 (0.0010) [2023-12-27 16:45:25,376][20381] Updated weights for policy 1, policy_version 1602169 (0.0007) [2023-12-27 16:45:25,444][20381] Updated weights for policy 1, policy_version 1602179 (0.0007) [2023-12-27 16:45:25,504][20381] Updated weights for policy 1, policy_version 1602189 (0.0005) [2023-12-27 16:45:25,554][20381] Updated weights for policy 1, policy_version 1602199 (0.0005) [2023-12-27 16:45:25,682][20347] Updated weights for policy 0, policy_version 1603801 (0.0010) [2023-12-27 16:45:25,730][20347] Updated weights for policy 0, policy_version 1603811 (0.0010) [2023-12-27 16:45:25,778][20347] Updated weights for policy 0, policy_version 1603821 (0.0010) [2023-12-27 16:45:26,100][20381] Updated weights for policy 1, policy_version 1602209 (0.0005) [2023-12-27 16:45:26,162][20381] Updated weights for policy 1, policy_version 1602219 (0.0005) [2023-12-27 16:45:26,222][20381] Updated weights for policy 1, policy_version 1602229 (0.0005) [2023-12-27 16:45:26,454][20347] Updated weights for policy 0, policy_version 1603831 (0.0008) [2023-12-27 16:45:26,500][20347] Updated weights for policy 0, policy_version 1603841 (0.0008) [2023-12-27 16:45:26,547][20347] Updated weights for policy 0, policy_version 1603851 (0.0010) [2023-12-27 16:45:26,759][20381] Updated weights for policy 1, policy_version 1602239 (0.0005) [2023-12-27 16:45:26,815][20381] Updated weights for policy 1, policy_version 1602249 (0.0005) [2023-12-27 16:45:26,868][20381] Updated weights for policy 1, policy_version 1602259 (0.0005) [2023-12-27 16:45:27,161][20347] Updated weights for policy 0, policy_version 1603861 (0.0010) [2023-12-27 16:45:27,206][20347] Updated weights for policy 0, policy_version 1603871 (0.0010) [2023-12-27 16:45:27,255][20347] Updated weights for policy 0, policy_version 1603881 (0.0010) [2023-12-27 16:45:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 820895744. Throughput: 0: 10184.7, 1: 10204.5. Samples: 820881360. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:27,364][19170] Avg episode reward: [(0, '990.360'), (1, '670.590')] [2023-12-27 16:45:27,473][20381] Updated weights for policy 1, policy_version 1602269 (0.0005) [2023-12-27 16:45:27,519][20381] Updated weights for policy 1, policy_version 1602279 (0.0005) [2023-12-27 16:45:27,564][20381] Updated weights for policy 1, policy_version 1602289 (0.0005) [2023-12-27 16:45:27,925][20347] Updated weights for policy 0, policy_version 1603891 (0.0010) [2023-12-27 16:45:28,013][20347] Updated weights for policy 0, policy_version 1603901 (0.0007) [2023-12-27 16:45:28,072][20347] Updated weights for policy 0, policy_version 1603911 (0.0008) [2023-12-27 16:45:28,094][20381] Updated weights for policy 1, policy_version 1602299 (0.0007) [2023-12-27 16:45:28,145][20381] Updated weights for policy 1, policy_version 1602309 (0.0007) [2023-12-27 16:45:28,189][20381] Updated weights for policy 1, policy_version 1602319 (0.0006) [2023-12-27 16:45:28,716][20347] Updated weights for policy 0, policy_version 1603921 (0.0007) [2023-12-27 16:45:28,778][20347] Updated weights for policy 0, policy_version 1603931 (0.0007) [2023-12-27 16:45:28,826][20381] Updated weights for policy 1, policy_version 1602329 (0.0005) [2023-12-27 16:45:28,838][20347] Updated weights for policy 0, policy_version 1603941 (0.0007) [2023-12-27 16:45:28,881][20381] Updated weights for policy 1, policy_version 1602339 (0.0006) [2023-12-27 16:45:28,901][20347] Updated weights for policy 0, policy_version 1603951 (0.0008) [2023-12-27 16:45:28,938][20381] Updated weights for policy 1, policy_version 1602349 (0.0007) [2023-12-27 16:45:28,994][20381] Updated weights for policy 1, policy_version 1602359 (0.0008) [2023-12-27 16:45:29,574][20347] Updated weights for policy 0, policy_version 1603961 (0.0006) [2023-12-27 16:45:29,625][20347] Updated weights for policy 0, policy_version 1603971 (0.0007) [2023-12-27 16:45:29,674][20347] Updated weights for policy 0, policy_version 1603981 (0.0006) [2023-12-27 16:45:29,702][20381] Updated weights for policy 1, policy_version 1602369 (0.0006) [2023-12-27 16:45:29,760][20381] Updated weights for policy 1, policy_version 1602379 (0.0006) [2023-12-27 16:45:29,814][20381] Updated weights for policy 1, policy_version 1602389 (0.0009) [2023-12-27 16:45:30,309][20347] Updated weights for policy 0, policy_version 1603991 (0.0007) [2023-12-27 16:45:30,368][20347] Updated weights for policy 0, policy_version 1604001 (0.0008) [2023-12-27 16:45:30,419][20347] Updated weights for policy 0, policy_version 1604011 (0.0007) [2023-12-27 16:45:30,490][20381] Updated weights for policy 1, policy_version 1602399 (0.0008) [2023-12-27 16:45:30,542][20381] Updated weights for policy 1, policy_version 1602409 (0.0007) [2023-12-27 16:45:30,602][20381] Updated weights for policy 1, policy_version 1602419 (0.0008) [2023-12-27 16:45:31,050][20347] Updated weights for policy 0, policy_version 1604021 (0.0008) [2023-12-27 16:45:31,108][20347] Updated weights for policy 0, policy_version 1604031 (0.0008) [2023-12-27 16:45:31,171][20347] Updated weights for policy 0, policy_version 1604041 (0.0008) [2023-12-27 16:45:31,360][20381] Updated weights for policy 1, policy_version 1602429 (0.0008) [2023-12-27 16:45:31,421][20381] Updated weights for policy 1, policy_version 1602439 (0.0008) [2023-12-27 16:45:31,472][20381] Updated weights for policy 1, policy_version 1602449 (0.0008) [2023-12-27 16:45:31,880][20347] Updated weights for policy 0, policy_version 1604051 (0.0007) [2023-12-27 16:45:31,938][20347] Updated weights for policy 0, policy_version 1604061 (0.0010) [2023-12-27 16:45:32,002][20347] Updated weights for policy 0, policy_version 1604071 (0.0010) [2023-12-27 16:45:32,240][20381] Updated weights for policy 1, policy_version 1602459 (0.0008) [2023-12-27 16:45:32,307][20381] Updated weights for policy 1, policy_version 1602469 (0.0008) [2023-12-27 16:45:32,361][20381] Updated weights for policy 1, policy_version 1602479 (0.0008) [2023-12-27 16:45:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 820994048. Throughput: 0: 10223.0, 1: 10302.0. Samples: 821007788. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:32,363][19170] Avg episode reward: [(0, '990.360'), (1, '719.900')] [2023-12-27 16:45:32,662][20347] Updated weights for policy 0, policy_version 1604081 (0.0009) [2023-12-27 16:45:32,723][20347] Updated weights for policy 0, policy_version 1604091 (0.0010) [2023-12-27 16:45:32,778][20347] Updated weights for policy 0, policy_version 1604101 (0.0010) [2023-12-27 16:45:32,833][20347] Updated weights for policy 0, policy_version 1604111 (0.0010) [2023-12-27 16:45:33,052][20381] Updated weights for policy 1, policy_version 1602489 (0.0009) [2023-12-27 16:45:33,099][20381] Updated weights for policy 1, policy_version 1602499 (0.0010) [2023-12-27 16:45:33,142][20381] Updated weights for policy 1, policy_version 1602509 (0.0008) [2023-12-27 16:45:33,188][20381] Updated weights for policy 1, policy_version 1602519 (0.0005) [2023-12-27 16:45:33,453][20347] Updated weights for policy 0, policy_version 1604121 (0.0010) [2023-12-27 16:45:33,498][20347] Updated weights for policy 0, policy_version 1604131 (0.0010) [2023-12-27 16:45:33,542][20347] Updated weights for policy 0, policy_version 1604141 (0.0010) [2023-12-27 16:45:33,784][20381] Updated weights for policy 1, policy_version 1602529 (0.0006) [2023-12-27 16:45:33,837][20381] Updated weights for policy 1, policy_version 1602539 (0.0006) [2023-12-27 16:45:33,888][20381] Updated weights for policy 1, policy_version 1602549 (0.0005) [2023-12-27 16:45:34,248][20347] Updated weights for policy 0, policy_version 1604151 (0.0008) [2023-12-27 16:45:34,308][20347] Updated weights for policy 0, policy_version 1604161 (0.0007) [2023-12-27 16:45:34,368][20347] Updated weights for policy 0, policy_version 1604171 (0.0007) [2023-12-27 16:45:34,562][20381] Updated weights for policy 1, policy_version 1602559 (0.0008) [2023-12-27 16:45:34,617][20381] Updated weights for policy 1, policy_version 1602569 (0.0010) [2023-12-27 16:45:34,669][20381] Updated weights for policy 1, policy_version 1602579 (0.0007) [2023-12-27 16:45:34,991][20347] Updated weights for policy 0, policy_version 1604181 (0.0006) [2023-12-27 16:45:35,050][20347] Updated weights for policy 0, policy_version 1604191 (0.0007) [2023-12-27 16:45:35,101][20347] Updated weights for policy 0, policy_version 1604201 (0.0010) [2023-12-27 16:45:35,217][20381] Updated weights for policy 1, policy_version 1602589 (0.0007) [2023-12-27 16:45:35,275][20381] Updated weights for policy 1, policy_version 1602599 (0.0008) [2023-12-27 16:45:35,341][20381] Updated weights for policy 1, policy_version 1602609 (0.0006) [2023-12-27 16:45:35,762][20347] Updated weights for policy 0, policy_version 1604211 (0.0009) [2023-12-27 16:45:35,812][20347] Updated weights for policy 0, policy_version 1604221 (0.0005) [2023-12-27 16:45:35,859][20381] Updated weights for policy 1, policy_version 1602619 (0.0005) [2023-12-27 16:45:35,861][20347] Updated weights for policy 0, policy_version 1604231 (0.0006) [2023-12-27 16:45:35,909][20381] Updated weights for policy 1, policy_version 1602629 (0.0005) [2023-12-27 16:45:35,959][20381] Updated weights for policy 1, policy_version 1602639 (0.0008) [2023-12-27 16:45:36,578][20347] Updated weights for policy 0, policy_version 1604241 (0.0007) [2023-12-27 16:45:36,636][20347] Updated weights for policy 0, policy_version 1604251 (0.0011) [2023-12-27 16:45:36,685][20381] Updated weights for policy 1, policy_version 1602649 (0.0007) [2023-12-27 16:45:36,698][20347] Updated weights for policy 0, policy_version 1604261 (0.0007) [2023-12-27 16:45:36,732][20381] Updated weights for policy 1, policy_version 1602659 (0.0008) [2023-12-27 16:45:36,755][20347] Updated weights for policy 0, policy_version 1604271 (0.0006) [2023-12-27 16:45:36,780][20381] Updated weights for policy 1, policy_version 1602669 (0.0006) [2023-12-27 16:45:36,824][20381] Updated weights for policy 1, policy_version 1602679 (0.0005) [2023-12-27 16:45:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 821100544. Throughput: 0: 10248.7, 1: 10356.4. Samples: 821073156. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '681.020')] [2023-12-27 16:45:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001604272_410771456.pth... [2023-12-27 16:45:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001603088_410468352.pth [2023-12-27 16:45:37,395][20381] Updated weights for policy 1, policy_version 1602689 (0.0005) [2023-12-27 16:45:37,451][20381] Updated weights for policy 1, policy_version 1602699 (0.0006) [2023-12-27 16:45:37,452][20347] Updated weights for policy 0, policy_version 1604281 (0.0011) [2023-12-27 16:45:37,499][20347] Updated weights for policy 0, policy_version 1604291 (0.0010) [2023-12-27 16:45:37,502][20381] Updated weights for policy 1, policy_version 1602709 (0.0005) [2023-12-27 16:45:37,517][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001602712_410337280.pth... [2023-12-27 16:45:37,520][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001601464_410017792.pth [2023-12-27 16:45:37,558][20347] Updated weights for policy 0, policy_version 1604301 (0.0010) [2023-12-27 16:45:38,098][20381] Updated weights for policy 1, policy_version 1602719 (0.0009) [2023-12-27 16:45:38,156][20381] Updated weights for policy 1, policy_version 1602729 (0.0009) [2023-12-27 16:45:38,219][20381] Updated weights for policy 1, policy_version 1602739 (0.0008) [2023-12-27 16:45:38,257][20347] Updated weights for policy 0, policy_version 1604311 (0.0010) [2023-12-27 16:45:38,302][20347] Updated weights for policy 0, policy_version 1604321 (0.0010) [2023-12-27 16:45:38,367][20347] Updated weights for policy 0, policy_version 1604331 (0.0010) [2023-12-27 16:45:38,879][20381] Updated weights for policy 1, policy_version 1602749 (0.0010) [2023-12-27 16:45:38,939][20381] Updated weights for policy 1, policy_version 1602759 (0.0010) [2023-12-27 16:45:38,996][20381] Updated weights for policy 1, policy_version 1602769 (0.0010) [2023-12-27 16:45:39,072][20347] Updated weights for policy 0, policy_version 1604341 (0.0009) [2023-12-27 16:45:39,123][20347] Updated weights for policy 0, policy_version 1604351 (0.0010) [2023-12-27 16:45:39,183][20347] Updated weights for policy 0, policy_version 1604361 (0.0010) [2023-12-27 16:45:39,690][20381] Updated weights for policy 1, policy_version 1602779 (0.0010) [2023-12-27 16:45:39,756][20381] Updated weights for policy 1, policy_version 1602789 (0.0007) [2023-12-27 16:45:39,825][20381] Updated weights for policy 1, policy_version 1602799 (0.0007) [2023-12-27 16:45:39,949][20347] Updated weights for policy 0, policy_version 1604371 (0.0009) [2023-12-27 16:45:40,013][20347] Updated weights for policy 0, policy_version 1604381 (0.0010) [2023-12-27 16:45:40,069][20347] Updated weights for policy 0, policy_version 1604391 (0.0006) [2023-12-27 16:45:40,535][20381] Updated weights for policy 1, policy_version 1602809 (0.0007) [2023-12-27 16:45:40,600][20381] Updated weights for policy 1, policy_version 1602819 (0.0006) [2023-12-27 16:45:40,655][20381] Updated weights for policy 1, policy_version 1602829 (0.0007) [2023-12-27 16:45:40,710][20381] Updated weights for policy 1, policy_version 1602839 (0.0006) [2023-12-27 16:45:40,741][20347] Updated weights for policy 0, policy_version 1604401 (0.0007) [2023-12-27 16:45:40,801][20347] Updated weights for policy 0, policy_version 1604411 (0.0006) [2023-12-27 16:45:40,860][20347] Updated weights for policy 0, policy_version 1604421 (0.0009) [2023-12-27 16:45:40,921][20347] Updated weights for policy 0, policy_version 1604431 (0.0009) [2023-12-27 16:45:41,376][20381] Updated weights for policy 1, policy_version 1602849 (0.0008) [2023-12-27 16:45:41,442][20381] Updated weights for policy 1, policy_version 1602859 (0.0007) [2023-12-27 16:45:41,502][20381] Updated weights for policy 1, policy_version 1602869 (0.0007) [2023-12-27 16:45:41,704][20347] Updated weights for policy 0, policy_version 1604441 (0.0008) [2023-12-27 16:45:41,776][20347] Updated weights for policy 0, policy_version 1604451 (0.0007) [2023-12-27 16:45:41,843][20347] Updated weights for policy 0, policy_version 1604461 (0.0008) [2023-12-27 16:45:42,267][20381] Updated weights for policy 1, policy_version 1602879 (0.0008) [2023-12-27 16:45:42,331][20381] Updated weights for policy 1, policy_version 1602889 (0.0008) [2023-12-27 16:45:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 821198848. Throughput: 0: 10215.7, 1: 10447.8. Samples: 821196812. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '719.400')] [2023-12-27 16:45:42,402][20381] Updated weights for policy 1, policy_version 1602899 (0.0008) [2023-12-27 16:45:42,574][20347] Updated weights for policy 0, policy_version 1604471 (0.0008) [2023-12-27 16:45:42,634][20347] Updated weights for policy 0, policy_version 1604481 (0.0008) [2023-12-27 16:45:42,696][20347] Updated weights for policy 0, policy_version 1604491 (0.0008) [2023-12-27 16:45:43,115][20381] Updated weights for policy 1, policy_version 1602909 (0.0008) [2023-12-27 16:45:43,164][20381] Updated weights for policy 1, policy_version 1602919 (0.0007) [2023-12-27 16:45:43,216][20381] Updated weights for policy 1, policy_version 1602929 (0.0005) [2023-12-27 16:45:43,444][20347] Updated weights for policy 0, policy_version 1604501 (0.0007) [2023-12-27 16:45:43,510][20347] Updated weights for policy 0, policy_version 1604511 (0.0006) [2023-12-27 16:45:43,575][20347] Updated weights for policy 0, policy_version 1604521 (0.0007) [2023-12-27 16:45:43,856][20381] Updated weights for policy 1, policy_version 1602939 (0.0005) [2023-12-27 16:45:43,912][20381] Updated weights for policy 1, policy_version 1602949 (0.0006) [2023-12-27 16:45:43,966][20381] Updated weights for policy 1, policy_version 1602959 (0.0006) [2023-12-27 16:45:44,271][20347] Updated weights for policy 0, policy_version 1604531 (0.0007) [2023-12-27 16:45:44,321][20347] Updated weights for policy 0, policy_version 1604541 (0.0006) [2023-12-27 16:45:44,374][20347] Updated weights for policy 0, policy_version 1604551 (0.0007) [2023-12-27 16:45:44,655][20381] Updated weights for policy 1, policy_version 1602969 (0.0008) [2023-12-27 16:45:44,707][20381] Updated weights for policy 1, policy_version 1602979 (0.0005) [2023-12-27 16:45:44,763][20381] Updated weights for policy 1, policy_version 1602989 (0.0010) [2023-12-27 16:45:44,830][20381] Updated weights for policy 1, policy_version 1602999 (0.0007) [2023-12-27 16:45:45,061][20347] Updated weights for policy 0, policy_version 1604561 (0.0006) [2023-12-27 16:45:45,125][20347] Updated weights for policy 0, policy_version 1604571 (0.0008) [2023-12-27 16:45:45,189][20347] Updated weights for policy 0, policy_version 1604581 (0.0006) [2023-12-27 16:45:45,251][20347] Updated weights for policy 0, policy_version 1604591 (0.0006) [2023-12-27 16:45:45,537][20381] Updated weights for policy 1, policy_version 1603009 (0.0007) [2023-12-27 16:45:45,594][20381] Updated weights for policy 1, policy_version 1603019 (0.0008) [2023-12-27 16:45:45,658][20381] Updated weights for policy 1, policy_version 1603029 (0.0008) [2023-12-27 16:45:45,841][20347] Updated weights for policy 0, policy_version 1604601 (0.0007) [2023-12-27 16:45:45,890][20347] Updated weights for policy 0, policy_version 1604611 (0.0005) [2023-12-27 16:45:45,943][20347] Updated weights for policy 0, policy_version 1604621 (0.0008) [2023-12-27 16:45:46,316][20381] Updated weights for policy 1, policy_version 1603039 (0.0008) [2023-12-27 16:45:46,374][20381] Updated weights for policy 1, policy_version 1603049 (0.0008) [2023-12-27 16:45:46,431][20381] Updated weights for policy 1, policy_version 1603059 (0.0009) [2023-12-27 16:45:46,622][20347] Updated weights for policy 0, policy_version 1604631 (0.0010) [2023-12-27 16:45:46,685][20347] Updated weights for policy 0, policy_version 1604641 (0.0010) [2023-12-27 16:45:46,743][20347] Updated weights for policy 0, policy_version 1604651 (0.0006) [2023-12-27 16:45:47,063][20381] Updated weights for policy 1, policy_version 1603069 (0.0008) [2023-12-27 16:45:47,119][20381] Updated weights for policy 1, policy_version 1603079 (0.0007) [2023-12-27 16:45:47,175][20381] Updated weights for policy 1, policy_version 1603089 (0.0008) [2023-12-27 16:45:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 821305344. Throughput: 0: 10198.0, 1: 10412.2. Samples: 821315812. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '815.900')] [2023-12-27 16:45:47,395][20347] Updated weights for policy 0, policy_version 1604661 (0.0009) [2023-12-27 16:45:47,458][20347] Updated weights for policy 0, policy_version 1604671 (0.0010) [2023-12-27 16:45:47,516][20347] Updated weights for policy 0, policy_version 1604681 (0.0011) [2023-12-27 16:45:47,905][20381] Updated weights for policy 1, policy_version 1603099 (0.0008) [2023-12-27 16:45:47,957][20381] Updated weights for policy 1, policy_version 1603109 (0.0007) [2023-12-27 16:45:48,019][20381] Updated weights for policy 1, policy_version 1603119 (0.0008) [2023-12-27 16:45:48,190][20347] Updated weights for policy 0, policy_version 1604691 (0.0010) [2023-12-27 16:45:48,244][20347] Updated weights for policy 0, policy_version 1604701 (0.0011) [2023-12-27 16:45:48,299][20347] Updated weights for policy 0, policy_version 1604711 (0.0010) [2023-12-27 16:45:48,675][20381] Updated weights for policy 1, policy_version 1603129 (0.0006) [2023-12-27 16:45:48,725][20381] Updated weights for policy 1, policy_version 1603139 (0.0008) [2023-12-27 16:45:48,778][20381] Updated weights for policy 1, policy_version 1603149 (0.0008) [2023-12-27 16:45:48,835][20381] Updated weights for policy 1, policy_version 1603159 (0.0007) [2023-12-27 16:45:48,999][20347] Updated weights for policy 0, policy_version 1604721 (0.0011) [2023-12-27 16:45:49,056][20347] Updated weights for policy 0, policy_version 1604731 (0.0010) [2023-12-27 16:45:49,111][20347] Updated weights for policy 0, policy_version 1604741 (0.0010) [2023-12-27 16:45:49,159][20347] Updated weights for policy 0, policy_version 1604751 (0.0010) [2023-12-27 16:45:49,539][20381] Updated weights for policy 1, policy_version 1603169 (0.0008) [2023-12-27 16:45:49,593][20381] Updated weights for policy 1, policy_version 1603179 (0.0007) [2023-12-27 16:45:49,649][20381] Updated weights for policy 1, policy_version 1603189 (0.0007) [2023-12-27 16:45:49,856][20347] Updated weights for policy 0, policy_version 1604761 (0.0008) [2023-12-27 16:45:49,909][20347] Updated weights for policy 0, policy_version 1604771 (0.0007) [2023-12-27 16:45:49,974][20347] Updated weights for policy 0, policy_version 1604781 (0.0009) [2023-12-27 16:45:50,376][20381] Updated weights for policy 1, policy_version 1603199 (0.0009) [2023-12-27 16:45:50,428][20381] Updated weights for policy 1, policy_version 1603209 (0.0007) [2023-12-27 16:45:50,492][20381] Updated weights for policy 1, policy_version 1603219 (0.0008) [2023-12-27 16:45:50,601][20347] Updated weights for policy 0, policy_version 1604791 (0.0006) [2023-12-27 16:45:50,657][20347] Updated weights for policy 0, policy_version 1604801 (0.0006) [2023-12-27 16:45:50,720][20347] Updated weights for policy 0, policy_version 1604811 (0.0006) [2023-12-27 16:45:51,294][20381] Updated weights for policy 1, policy_version 1603229 (0.0007) [2023-12-27 16:45:51,347][20347] Updated weights for policy 0, policy_version 1604821 (0.0006) [2023-12-27 16:45:51,350][20381] Updated weights for policy 1, policy_version 1603239 (0.0008) [2023-12-27 16:45:51,418][20347] Updated weights for policy 0, policy_version 1604831 (0.0007) [2023-12-27 16:45:51,422][20381] Updated weights for policy 1, policy_version 1603249 (0.0007) [2023-12-27 16:45:51,482][20347] Updated weights for policy 0, policy_version 1604841 (0.0009) [2023-12-27 16:45:52,118][20381] Updated weights for policy 1, policy_version 1603259 (0.0007) [2023-12-27 16:45:52,183][20381] Updated weights for policy 1, policy_version 1603269 (0.0009) [2023-12-27 16:45:52,210][20347] Updated weights for policy 0, policy_version 1604851 (0.0009) [2023-12-27 16:45:52,249][20381] Updated weights for policy 1, policy_version 1603279 (0.0009) [2023-12-27 16:45:52,274][20347] Updated weights for policy 0, policy_version 1604861 (0.0009) [2023-12-27 16:45:52,329][20347] Updated weights for policy 0, policy_version 1604871 (0.0009) [2023-12-27 16:45:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 821403648. Throughput: 0: 10150.7, 1: 10357.6. Samples: 821374940. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '776.930')] [2023-12-27 16:45:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001603288_410484736.pth... [2023-12-27 16:45:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001602072_410173440.pth [2023-12-27 16:45:52,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001604880_410927104.pth... [2023-12-27 16:45:52,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001603696_410624000.pth [2023-12-27 16:45:52,966][20381] Updated weights for policy 1, policy_version 1603289 (0.0009) [2023-12-27 16:45:53,015][20381] Updated weights for policy 1, policy_version 1603299 (0.0008) [2023-12-27 16:45:53,074][20381] Updated weights for policy 1, policy_version 1603309 (0.0008) [2023-12-27 16:45:53,116][20347] Updated weights for policy 0, policy_version 1604881 (0.0009) [2023-12-27 16:45:53,127][20381] Updated weights for policy 1, policy_version 1603319 (0.0007) [2023-12-27 16:45:53,159][20347] Updated weights for policy 0, policy_version 1604891 (0.0007) [2023-12-27 16:45:53,210][20347] Updated weights for policy 0, policy_version 1604901 (0.0007) [2023-12-27 16:45:53,254][20347] Updated weights for policy 0, policy_version 1604911 (0.0007) [2023-12-27 16:45:53,796][20381] Updated weights for policy 1, policy_version 1603329 (0.0007) [2023-12-27 16:45:53,846][20381] Updated weights for policy 1, policy_version 1603339 (0.0006) [2023-12-27 16:45:53,900][20381] Updated weights for policy 1, policy_version 1603349 (0.0007) [2023-12-27 16:45:53,946][20347] Updated weights for policy 0, policy_version 1604921 (0.0007) [2023-12-27 16:45:53,999][20347] Updated weights for policy 0, policy_version 1604931 (0.0006) [2023-12-27 16:45:54,050][20347] Updated weights for policy 0, policy_version 1604941 (0.0010) [2023-12-27 16:45:54,487][20381] Updated weights for policy 1, policy_version 1603359 (0.0006) [2023-12-27 16:45:54,542][20381] Updated weights for policy 1, policy_version 1603369 (0.0007) [2023-12-27 16:45:54,596][20381] Updated weights for policy 1, policy_version 1603379 (0.0008) [2023-12-27 16:45:54,727][20347] Updated weights for policy 0, policy_version 1604951 (0.0010) [2023-12-27 16:45:54,784][20347] Updated weights for policy 0, policy_version 1604961 (0.0010) [2023-12-27 16:45:54,838][20347] Updated weights for policy 0, policy_version 1604971 (0.0010) [2023-12-27 16:45:55,213][20381] Updated weights for policy 1, policy_version 1603389 (0.0007) [2023-12-27 16:45:55,261][20381] Updated weights for policy 1, policy_version 1603399 (0.0006) [2023-12-27 16:45:55,310][20381] Updated weights for policy 1, policy_version 1603409 (0.0005) [2023-12-27 16:45:55,536][20347] Updated weights for policy 0, policy_version 1604981 (0.0011) [2023-12-27 16:45:55,591][20347] Updated weights for policy 0, policy_version 1604991 (0.0011) [2023-12-27 16:45:55,640][20347] Updated weights for policy 0, policy_version 1605001 (0.0010) [2023-12-27 16:45:55,946][20381] Updated weights for policy 1, policy_version 1603419 (0.0007) [2023-12-27 16:45:56,004][20381] Updated weights for policy 1, policy_version 1603429 (0.0005) [2023-12-27 16:45:56,050][20381] Updated weights for policy 1, policy_version 1603439 (0.0005) [2023-12-27 16:45:56,323][20347] Updated weights for policy 0, policy_version 1605011 (0.0009) [2023-12-27 16:45:56,378][20347] Updated weights for policy 0, policy_version 1605021 (0.0005) [2023-12-27 16:45:56,426][20347] Updated weights for policy 0, policy_version 1605031 (0.0005) [2023-12-27 16:45:56,648][20381] Updated weights for policy 1, policy_version 1603449 (0.0007) [2023-12-27 16:45:56,706][20381] Updated weights for policy 1, policy_version 1603459 (0.0005) [2023-12-27 16:45:56,763][20381] Updated weights for policy 1, policy_version 1603469 (0.0009) [2023-12-27 16:45:56,811][20381] Updated weights for policy 1, policy_version 1603479 (0.0010) [2023-12-27 16:45:57,057][20347] Updated weights for policy 0, policy_version 1605041 (0.0008) [2023-12-27 16:45:57,116][20347] Updated weights for policy 0, policy_version 1605051 (0.0010) [2023-12-27 16:45:57,179][20347] Updated weights for policy 0, policy_version 1605061 (0.0010) [2023-12-27 16:45:57,232][20347] Updated weights for policy 0, policy_version 1605071 (0.0010) [2023-12-27 16:45:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 821510144. Throughput: 0: 10117.0, 1: 10363.1. Samples: 821497756. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:45:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '825.000')] [2023-12-27 16:45:57,454][20381] Updated weights for policy 1, policy_version 1603489 (0.0006) [2023-12-27 16:45:57,507][20381] Updated weights for policy 1, policy_version 1603499 (0.0007) [2023-12-27 16:45:57,559][20381] Updated weights for policy 1, policy_version 1603509 (0.0005) [2023-12-27 16:45:57,885][20347] Updated weights for policy 0, policy_version 1605081 (0.0010) [2023-12-27 16:45:57,936][20347] Updated weights for policy 0, policy_version 1605091 (0.0010) [2023-12-27 16:45:57,988][20347] Updated weights for policy 0, policy_version 1605101 (0.0010) [2023-12-27 16:45:58,190][20381] Updated weights for policy 1, policy_version 1603519 (0.0006) [2023-12-27 16:45:58,254][20381] Updated weights for policy 1, policy_version 1603529 (0.0007) [2023-12-27 16:45:58,326][20381] Updated weights for policy 1, policy_version 1603539 (0.0008) [2023-12-27 16:45:58,790][20347] Updated weights for policy 0, policy_version 1605111 (0.0008) [2023-12-27 16:45:58,859][20347] Updated weights for policy 0, policy_version 1605121 (0.0008) [2023-12-27 16:45:58,919][20347] Updated weights for policy 0, policy_version 1605131 (0.0007) [2023-12-27 16:45:59,077][20381] Updated weights for policy 1, policy_version 1603549 (0.0010) [2023-12-27 16:45:59,129][20381] Updated weights for policy 1, policy_version 1603559 (0.0010) [2023-12-27 16:45:59,178][20381] Updated weights for policy 1, policy_version 1603569 (0.0010) [2023-12-27 16:45:59,628][20347] Updated weights for policy 0, policy_version 1605141 (0.0008) [2023-12-27 16:45:59,683][20347] Updated weights for policy 0, policy_version 1605151 (0.0009) [2023-12-27 16:45:59,742][20347] Updated weights for policy 0, policy_version 1605161 (0.0010) [2023-12-27 16:45:59,867][20381] Updated weights for policy 1, policy_version 1603579 (0.0010) [2023-12-27 16:45:59,931][20381] Updated weights for policy 1, policy_version 1603589 (0.0010) [2023-12-27 16:45:59,993][20381] Updated weights for policy 1, policy_version 1603599 (0.0010) [2023-12-27 16:46:00,459][20347] Updated weights for policy 0, policy_version 1605171 (0.0009) [2023-12-27 16:46:00,511][20347] Updated weights for policy 0, policy_version 1605181 (0.0005) [2023-12-27 16:46:00,566][20347] Updated weights for policy 0, policy_version 1605191 (0.0005) [2023-12-27 16:46:00,662][20381] Updated weights for policy 1, policy_version 1603609 (0.0007) [2023-12-27 16:46:00,716][20381] Updated weights for policy 1, policy_version 1603619 (0.0008) [2023-12-27 16:46:00,761][20381] Updated weights for policy 1, policy_version 1603629 (0.0005) [2023-12-27 16:46:00,803][20381] Updated weights for policy 1, policy_version 1603639 (0.0005) [2023-12-27 16:46:01,191][20347] Updated weights for policy 0, policy_version 1605201 (0.0005) [2023-12-27 16:46:01,251][20347] Updated weights for policy 0, policy_version 1605211 (0.0008) [2023-12-27 16:46:01,308][20347] Updated weights for policy 0, policy_version 1605221 (0.0008) [2023-12-27 16:46:01,359][20347] Updated weights for policy 0, policy_version 1605231 (0.0008) [2023-12-27 16:46:01,468][20381] Updated weights for policy 1, policy_version 1603649 (0.0007) [2023-12-27 16:46:01,519][20381] Updated weights for policy 1, policy_version 1603659 (0.0008) [2023-12-27 16:46:01,572][20381] Updated weights for policy 1, policy_version 1603669 (0.0009) [2023-12-27 16:46:02,032][20347] Updated weights for policy 0, policy_version 1605241 (0.0007) [2023-12-27 16:46:02,092][20347] Updated weights for policy 0, policy_version 1605251 (0.0007) [2023-12-27 16:46:02,150][20347] Updated weights for policy 0, policy_version 1605261 (0.0008) [2023-12-27 16:46:02,300][20381] Updated weights for policy 1, policy_version 1603679 (0.0006) [2023-12-27 16:46:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 821608448. Throughput: 0: 10074.1, 1: 10346.0. Samples: 821620580. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:46:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '806.350')] [2023-12-27 16:46:02,364][20381] Updated weights for policy 1, policy_version 1603689 (0.0007) [2023-12-27 16:46:02,420][20381] Updated weights for policy 1, policy_version 1603699 (0.0006) [2023-12-27 16:46:02,792][20347] Updated weights for policy 0, policy_version 1605271 (0.0008) [2023-12-27 16:46:02,847][20347] Updated weights for policy 0, policy_version 1605281 (0.0007) [2023-12-27 16:46:02,898][20347] Updated weights for policy 0, policy_version 1605291 (0.0007) [2023-12-27 16:46:03,008][20381] Updated weights for policy 1, policy_version 1603709 (0.0007) [2023-12-27 16:46:03,060][20381] Updated weights for policy 1, policy_version 1603719 (0.0007) [2023-12-27 16:46:03,121][20381] Updated weights for policy 1, policy_version 1603729 (0.0009) [2023-12-27 16:46:03,577][20347] Updated weights for policy 0, policy_version 1605301 (0.0007) [2023-12-27 16:46:03,632][20347] Updated weights for policy 0, policy_version 1605311 (0.0008) [2023-12-27 16:46:03,685][20347] Updated weights for policy 0, policy_version 1605321 (0.0007) [2023-12-27 16:46:03,770][20381] Updated weights for policy 1, policy_version 1603739 (0.0007) [2023-12-27 16:46:03,815][20381] Updated weights for policy 1, policy_version 1603749 (0.0005) [2023-12-27 16:46:03,879][20381] Updated weights for policy 1, policy_version 1603759 (0.0007) [2023-12-27 16:46:04,330][20347] Updated weights for policy 0, policy_version 1605331 (0.0007) [2023-12-27 16:46:04,395][20347] Updated weights for policy 0, policy_version 1605341 (0.0007) [2023-12-27 16:46:04,459][20347] Updated weights for policy 0, policy_version 1605351 (0.0008) [2023-12-27 16:46:04,561][20381] Updated weights for policy 1, policy_version 1603769 (0.0006) [2023-12-27 16:46:04,609][20381] Updated weights for policy 1, policy_version 1603779 (0.0006) [2023-12-27 16:46:04,661][20381] Updated weights for policy 1, policy_version 1603789 (0.0006) [2023-12-27 16:46:04,716][20381] Updated weights for policy 1, policy_version 1603799 (0.0006) [2023-12-27 16:46:05,090][20347] Updated weights for policy 0, policy_version 1605361 (0.0008) [2023-12-27 16:46:05,143][20347] Updated weights for policy 0, policy_version 1605371 (0.0007) [2023-12-27 16:46:05,195][20347] Updated weights for policy 0, policy_version 1605381 (0.0008) [2023-12-27 16:46:05,256][20347] Updated weights for policy 0, policy_version 1605391 (0.0008) [2023-12-27 16:46:05,297][20381] Updated weights for policy 1, policy_version 1603809 (0.0006) [2023-12-27 16:46:05,354][20381] Updated weights for policy 1, policy_version 1603819 (0.0007) [2023-12-27 16:46:05,400][20381] Updated weights for policy 1, policy_version 1603829 (0.0010) [2023-12-27 16:46:05,815][20347] Updated weights for policy 0, policy_version 1605401 (0.0006) [2023-12-27 16:46:05,867][20347] Updated weights for policy 0, policy_version 1605411 (0.0008) [2023-12-27 16:46:05,922][20347] Updated weights for policy 0, policy_version 1605421 (0.0008) [2023-12-27 16:46:06,101][20381] Updated weights for policy 1, policy_version 1603839 (0.0008) [2023-12-27 16:46:06,163][20381] Updated weights for policy 1, policy_version 1603849 (0.0008) [2023-12-27 16:46:06,224][20381] Updated weights for policy 1, policy_version 1603859 (0.0008) [2023-12-27 16:46:06,643][20347] Updated weights for policy 0, policy_version 1605431 (0.0007) [2023-12-27 16:46:06,695][20347] Updated weights for policy 0, policy_version 1605441 (0.0008) [2023-12-27 16:46:06,745][20347] Updated weights for policy 0, policy_version 1605451 (0.0010) [2023-12-27 16:46:06,961][20381] Updated weights for policy 1, policy_version 1603869 (0.0008) [2023-12-27 16:46:07,022][20381] Updated weights for policy 1, policy_version 1603879 (0.0009) [2023-12-27 16:46:07,078][20381] Updated weights for policy 1, policy_version 1603889 (0.0008) [2023-12-27 16:46:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 821714944. Throughput: 0: 10106.9, 1: 10436.1. Samples: 821684704. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:46:07,363][19170] Avg episode reward: [(0, '990.290'), (1, '766.540')] [2023-12-27 16:46:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001603896_410640384.pth... [2023-12-27 16:46:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001605456_411074560.pth... [2023-12-27 16:46:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001602712_410337280.pth [2023-12-27 16:46:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001604272_410771456.pth [2023-12-27 16:46:07,416][20347] Updated weights for policy 0, policy_version 1605461 (0.0010) [2023-12-27 16:46:07,477][20347] Updated weights for policy 0, policy_version 1605471 (0.0008) [2023-12-27 16:46:07,535][20347] Updated weights for policy 0, policy_version 1605481 (0.0010) [2023-12-27 16:46:07,665][20381] Updated weights for policy 1, policy_version 1603899 (0.0008) [2023-12-27 16:46:07,726][20381] Updated weights for policy 1, policy_version 1603909 (0.0007) [2023-12-27 16:46:07,777][20381] Updated weights for policy 1, policy_version 1603919 (0.0007) [2023-12-27 16:46:08,165][20347] Updated weights for policy 0, policy_version 1605491 (0.0010) [2023-12-27 16:46:08,221][20347] Updated weights for policy 0, policy_version 1605501 (0.0010) [2023-12-27 16:46:08,283][20347] Updated weights for policy 0, policy_version 1605511 (0.0010) [2023-12-27 16:46:08,411][20381] Updated weights for policy 1, policy_version 1603929 (0.0008) [2023-12-27 16:46:08,473][20381] Updated weights for policy 1, policy_version 1603939 (0.0007) [2023-12-27 16:46:08,531][20381] Updated weights for policy 1, policy_version 1603949 (0.0008) [2023-12-27 16:46:08,587][20381] Updated weights for policy 1, policy_version 1603959 (0.0008) [2023-12-27 16:46:08,973][20347] Updated weights for policy 0, policy_version 1605521 (0.0010) [2023-12-27 16:46:09,018][20347] Updated weights for policy 0, policy_version 1605531 (0.0010) [2023-12-27 16:46:09,071][20347] Updated weights for policy 0, policy_version 1605541 (0.0009) [2023-12-27 16:46:09,126][20347] Updated weights for policy 0, policy_version 1605551 (0.0005) [2023-12-27 16:46:09,284][20381] Updated weights for policy 1, policy_version 1603969 (0.0009) [2023-12-27 16:46:09,348][20381] Updated weights for policy 1, policy_version 1603979 (0.0008) [2023-12-27 16:46:09,414][20381] Updated weights for policy 1, policy_version 1603989 (0.0007) [2023-12-27 16:46:09,874][20347] Updated weights for policy 0, policy_version 1605561 (0.0008) [2023-12-27 16:46:09,934][20347] Updated weights for policy 0, policy_version 1605571 (0.0008) [2023-12-27 16:46:09,970][20291] KL-divergence is very high: 106.6651 [2023-12-27 16:46:09,994][20347] Updated weights for policy 0, policy_version 1605581 (0.0007) [2023-12-27 16:46:10,097][20381] Updated weights for policy 1, policy_version 1603999 (0.0009) [2023-12-27 16:46:10,157][20381] Updated weights for policy 1, policy_version 1604009 (0.0009) [2023-12-27 16:46:10,216][20381] Updated weights for policy 1, policy_version 1604019 (0.0008) [2023-12-27 16:46:10,728][20347] Updated weights for policy 0, policy_version 1605591 (0.0009) [2023-12-27 16:46:10,777][20347] Updated weights for policy 0, policy_version 1605601 (0.0007) [2023-12-27 16:46:10,798][20381] Updated weights for policy 1, policy_version 1604029 (0.0005) [2023-12-27 16:46:10,833][20347] Updated weights for policy 0, policy_version 1605611 (0.0007) [2023-12-27 16:46:10,854][20381] Updated weights for policy 1, policy_version 1604039 (0.0005) [2023-12-27 16:46:10,923][20381] Updated weights for policy 1, policy_version 1604049 (0.0008) [2023-12-27 16:46:11,555][20347] Updated weights for policy 0, policy_version 1605621 (0.0007) [2023-12-27 16:46:11,616][20347] Updated weights for policy 0, policy_version 1605631 (0.0008) [2023-12-27 16:46:11,675][20381] Updated weights for policy 1, policy_version 1604059 (0.0008) [2023-12-27 16:46:11,682][20347] Updated weights for policy 0, policy_version 1605641 (0.0006) [2023-12-27 16:46:11,741][20381] Updated weights for policy 1, policy_version 1604069 (0.0008) [2023-12-27 16:46:11,800][20381] Updated weights for policy 1, policy_version 1604079 (0.0008) [2023-12-27 16:46:12,304][20347] Updated weights for policy 0, policy_version 1605651 (0.0007) [2023-12-27 16:46:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 821813248. Throughput: 0: 10171.7, 1: 10414.1. Samples: 821807716. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:46:12,363][19170] Avg episode reward: [(0, '970.910'), (1, '776.180')] [2023-12-27 16:46:12,367][20347] Updated weights for policy 0, policy_version 1605661 (0.0008) [2023-12-27 16:46:12,426][20347] Updated weights for policy 0, policy_version 1605671 (0.0008) [2023-12-27 16:46:12,498][20381] Updated weights for policy 1, policy_version 1604089 (0.0008) [2023-12-27 16:46:12,555][20381] Updated weights for policy 1, policy_version 1604099 (0.0005) [2023-12-27 16:46:12,610][20381] Updated weights for policy 1, policy_version 1604109 (0.0005) [2023-12-27 16:46:12,666][20381] Updated weights for policy 1, policy_version 1604119 (0.0006) [2023-12-27 16:46:13,154][20347] Updated weights for policy 0, policy_version 1605681 (0.0008) [2023-12-27 16:46:13,212][20347] Updated weights for policy 0, policy_version 1605691 (0.0010) [2023-12-27 16:46:13,273][20347] Updated weights for policy 0, policy_version 1605701 (0.0009) [2023-12-27 16:46:13,307][20381] Updated weights for policy 1, policy_version 1604129 (0.0008) [2023-12-27 16:46:13,329][20347] Updated weights for policy 0, policy_version 1605711 (0.0005) [2023-12-27 16:46:13,357][20381] Updated weights for policy 1, policy_version 1604139 (0.0007) [2023-12-27 16:46:13,411][20381] Updated weights for policy 1, policy_version 1604149 (0.0008) [2023-12-27 16:46:13,940][20347] Updated weights for policy 0, policy_version 1605721 (0.0008) [2023-12-27 16:46:13,994][20347] Updated weights for policy 0, policy_version 1605731 (0.0005) [2023-12-27 16:46:14,053][20347] Updated weights for policy 0, policy_version 1605741 (0.0005) [2023-12-27 16:46:14,135][20381] Updated weights for policy 1, policy_version 1604159 (0.0008) [2023-12-27 16:46:14,191][20381] Updated weights for policy 1, policy_version 1604169 (0.0007) [2023-12-27 16:46:14,237][20381] Updated weights for policy 1, policy_version 1604179 (0.0007) [2023-12-27 16:46:14,586][20347] Updated weights for policy 0, policy_version 1605751 (0.0006) [2023-12-27 16:46:14,643][20347] Updated weights for policy 0, policy_version 1605761 (0.0010) [2023-12-27 16:46:14,701][20347] Updated weights for policy 0, policy_version 1605771 (0.0006) [2023-12-27 16:46:15,005][20381] Updated weights for policy 1, policy_version 1604189 (0.0009) [2023-12-27 16:46:15,071][20381] Updated weights for policy 1, policy_version 1604199 (0.0008) [2023-12-27 16:46:15,131][20381] Updated weights for policy 1, policy_version 1604209 (0.0007) [2023-12-27 16:46:15,338][20347] Updated weights for policy 0, policy_version 1605781 (0.0010) [2023-12-27 16:46:15,398][20347] Updated weights for policy 0, policy_version 1605791 (0.0010) [2023-12-27 16:46:15,455][20347] Updated weights for policy 0, policy_version 1605801 (0.0010) [2023-12-27 16:46:15,832][20381] Updated weights for policy 1, policy_version 1604219 (0.0007) [2023-12-27 16:46:15,890][20381] Updated weights for policy 1, policy_version 1604229 (0.0008) [2023-12-27 16:46:15,948][20381] Updated weights for policy 1, policy_version 1604239 (0.0007) [2023-12-27 16:46:16,047][20347] Updated weights for policy 0, policy_version 1605811 (0.0009) [2023-12-27 16:46:16,110][20347] Updated weights for policy 0, policy_version 1605821 (0.0005) [2023-12-27 16:46:16,165][20347] Updated weights for policy 0, policy_version 1605831 (0.0007) [2023-12-27 16:46:16,620][20381] Updated weights for policy 1, policy_version 1604249 (0.0008) [2023-12-27 16:46:16,670][20381] Updated weights for policy 1, policy_version 1604259 (0.0008) [2023-12-27 16:46:16,722][20381] Updated weights for policy 1, policy_version 1604269 (0.0008) [2023-12-27 16:46:16,779][20381] Updated weights for policy 1, policy_version 1604279 (0.0007) [2023-12-27 16:46:16,813][20347] Updated weights for policy 0, policy_version 1605841 (0.0010) [2023-12-27 16:46:16,877][20347] Updated weights for policy 0, policy_version 1605851 (0.0010) [2023-12-27 16:46:16,935][20347] Updated weights for policy 0, policy_version 1605861 (0.0010) [2023-12-27 16:46:16,984][20347] Updated weights for policy 0, policy_version 1605871 (0.0009) [2023-12-27 16:46:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 821919744. Throughput: 0: 10203.4, 1: 10316.1. Samples: 821931164. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-12-27 16:46:17,363][19170] Avg episode reward: [(0, '990.320'), (1, '825.290')] [2023-12-27 16:46:17,470][20381] Updated weights for policy 1, policy_version 1604289 (0.0007) [2023-12-27 16:46:17,529][20381] Updated weights for policy 1, policy_version 1604299 (0.0007) [2023-12-27 16:46:17,591][20381] Updated weights for policy 1, policy_version 1604309 (0.0006) [2023-12-27 16:46:17,665][20347] Updated weights for policy 0, policy_version 1605881 (0.0010) [2023-12-27 16:46:17,727][20347] Updated weights for policy 0, policy_version 1605891 (0.0010) [2023-12-27 16:46:17,790][20347] Updated weights for policy 0, policy_version 1605901 (0.0010) [2023-12-27 16:46:18,230][20381] Updated weights for policy 1, policy_version 1604319 (0.0007) [2023-12-27 16:46:18,286][20381] Updated weights for policy 1, policy_version 1604329 (0.0007) [2023-12-27 16:46:18,340][20381] Updated weights for policy 1, policy_version 1604339 (0.0008) [2023-12-27 16:46:18,472][20347] Updated weights for policy 0, policy_version 1605911 (0.0009) [2023-12-27 16:46:18,525][20347] Updated weights for policy 0, policy_version 1605921 (0.0008) [2023-12-27 16:46:18,580][20347] Updated weights for policy 0, policy_version 1605931 (0.0008) [2023-12-27 16:46:19,040][20381] Updated weights for policy 1, policy_version 1604349 (0.0007) [2023-12-27 16:46:19,099][20381] Updated weights for policy 1, policy_version 1604359 (0.0008) [2023-12-27 16:46:19,157][20381] Updated weights for policy 1, policy_version 1604369 (0.0008) [2023-12-27 16:46:19,298][20347] Updated weights for policy 0, policy_version 1605941 (0.0008) [2023-12-27 16:46:19,358][20347] Updated weights for policy 0, policy_version 1605951 (0.0008) [2023-12-27 16:46:19,423][20347] Updated weights for policy 0, policy_version 1605961 (0.0009) [2023-12-27 16:46:19,865][20381] Updated weights for policy 1, policy_version 1604379 (0.0007) [2023-12-27 16:46:19,935][20381] Updated weights for policy 1, policy_version 1604389 (0.0008) [2023-12-27 16:46:19,995][20381] Updated weights for policy 1, policy_version 1604399 (0.0008) [2023-12-27 16:46:20,150][20347] Updated weights for policy 0, policy_version 1605971 (0.0007) [2023-12-27 16:46:20,202][20347] Updated weights for policy 0, policy_version 1605981 (0.0007) [2023-12-27 16:46:20,263][20347] Updated weights for policy 0, policy_version 1605991 (0.0006) [2023-12-27 16:46:20,708][20381] Updated weights for policy 1, policy_version 1604409 (0.0008) [2023-12-27 16:46:20,772][20381] Updated weights for policy 1, policy_version 1604419 (0.0008) [2023-12-27 16:46:20,830][20381] Updated weights for policy 1, policy_version 1604429 (0.0008) [2023-12-27 16:46:20,863][20347] Updated weights for policy 0, policy_version 1606001 (0.0006) [2023-12-27 16:46:20,885][20381] Updated weights for policy 1, policy_version 1604439 (0.0007) [2023-12-27 16:46:20,924][20347] Updated weights for policy 0, policy_version 1606011 (0.0008) [2023-12-27 16:46:20,981][20347] Updated weights for policy 0, policy_version 1606021 (0.0008) [2023-12-27 16:46:21,042][20347] Updated weights for policy 0, policy_version 1606031 (0.0008) [2023-12-27 16:46:21,669][20381] Updated weights for policy 1, policy_version 1604449 (0.0008) [2023-12-27 16:46:21,736][20381] Updated weights for policy 1, policy_version 1604459 (0.0008) [2023-12-27 16:46:21,794][20381] Updated weights for policy 1, policy_version 1604469 (0.0006) [2023-12-27 16:46:21,808][20347] Updated weights for policy 0, policy_version 1606041 (0.0007) [2023-12-27 16:46:21,878][20347] Updated weights for policy 0, policy_version 1606051 (0.0006) [2023-12-27 16:46:21,948][20347] Updated weights for policy 0, policy_version 1606061 (0.0009) [2023-12-27 16:46:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 822018048. Throughput: 0: 10145.1, 1: 10236.3. Samples: 821990316. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '795.160')] [2023-12-27 16:46:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001604472_410787840.pth... [2023-12-27 16:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001606064_411230208.pth... [2023-12-27 16:46:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001603288_410484736.pth [2023-12-27 16:46:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001604880_410927104.pth [2023-12-27 16:46:22,619][20381] Updated weights for policy 1, policy_version 1604479 (0.0008) [2023-12-27 16:46:22,648][20347] Updated weights for policy 0, policy_version 1606071 (0.0006) [2023-12-27 16:46:22,677][20381] Updated weights for policy 1, policy_version 1604489 (0.0006) [2023-12-27 16:46:22,711][20347] Updated weights for policy 0, policy_version 1606081 (0.0008) [2023-12-27 16:46:22,732][20381] Updated weights for policy 1, policy_version 1604499 (0.0007) [2023-12-27 16:46:22,763][20347] Updated weights for policy 0, policy_version 1606091 (0.0009) [2023-12-27 16:46:23,401][20347] Updated weights for policy 0, policy_version 1606101 (0.0009) [2023-12-27 16:46:23,422][20381] Updated weights for policy 1, policy_version 1604509 (0.0006) [2023-12-27 16:46:23,453][20347] Updated weights for policy 0, policy_version 1606111 (0.0010) [2023-12-27 16:46:23,474][20381] Updated weights for policy 1, policy_version 1604519 (0.0005) [2023-12-27 16:46:23,510][20347] Updated weights for policy 0, policy_version 1606121 (0.0010) [2023-12-27 16:46:23,530][20381] Updated weights for policy 1, policy_version 1604529 (0.0005) [2023-12-27 16:46:24,142][20347] Updated weights for policy 0, policy_version 1606131 (0.0011) [2023-12-27 16:46:24,201][20347] Updated weights for policy 0, policy_version 1606141 (0.0010) [2023-12-27 16:46:24,249][20381] Updated weights for policy 1, policy_version 1604539 (0.0007) [2023-12-27 16:46:24,256][20347] Updated weights for policy 0, policy_version 1606151 (0.0010) [2023-12-27 16:46:24,306][20381] Updated weights for policy 1, policy_version 1604549 (0.0007) [2023-12-27 16:46:24,360][20381] Updated weights for policy 1, policy_version 1604559 (0.0008) [2023-12-27 16:46:24,841][20347] Updated weights for policy 0, policy_version 1606161 (0.0011) [2023-12-27 16:46:24,899][20347] Updated weights for policy 0, policy_version 1606171 (0.0010) [2023-12-27 16:46:24,953][20347] Updated weights for policy 0, policy_version 1606181 (0.0005) [2023-12-27 16:46:25,013][20347] Updated weights for policy 0, policy_version 1606191 (0.0005) [2023-12-27 16:46:25,103][20381] Updated weights for policy 1, policy_version 1604569 (0.0008) [2023-12-27 16:46:25,155][20381] Updated weights for policy 1, policy_version 1604579 (0.0007) [2023-12-27 16:46:25,212][20381] Updated weights for policy 1, policy_version 1604589 (0.0008) [2023-12-27 16:46:25,272][20381] Updated weights for policy 1, policy_version 1604599 (0.0008) [2023-12-27 16:46:25,587][20347] Updated weights for policy 0, policy_version 1606201 (0.0010) [2023-12-27 16:46:25,637][20347] Updated weights for policy 0, policy_version 1606211 (0.0010) [2023-12-27 16:46:25,688][20347] Updated weights for policy 0, policy_version 1606221 (0.0010) [2023-12-27 16:46:25,988][20381] Updated weights for policy 1, policy_version 1604609 (0.0008) [2023-12-27 16:46:26,034][20381] Updated weights for policy 1, policy_version 1604619 (0.0009) [2023-12-27 16:46:26,087][20381] Updated weights for policy 1, policy_version 1604629 (0.0009) [2023-12-27 16:46:26,295][20347] Updated weights for policy 0, policy_version 1606231 (0.0010) [2023-12-27 16:46:26,345][20347] Updated weights for policy 0, policy_version 1606241 (0.0010) [2023-12-27 16:46:26,393][20347] Updated weights for policy 0, policy_version 1606251 (0.0010) [2023-12-27 16:46:26,828][20381] Updated weights for policy 1, policy_version 1604639 (0.0008) [2023-12-27 16:46:26,888][20381] Updated weights for policy 1, policy_version 1604649 (0.0007) [2023-12-27 16:46:26,941][20381] Updated weights for policy 1, policy_version 1604659 (0.0007) [2023-12-27 16:46:27,025][20347] Updated weights for policy 0, policy_version 1606261 (0.0010) [2023-12-27 16:46:27,076][20347] Updated weights for policy 0, policy_version 1606271 (0.0010) [2023-12-27 16:46:27,126][20347] Updated weights for policy 0, policy_version 1606281 (0.0010) [2023-12-27 16:46:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 822124544. Throughput: 0: 10239.9, 1: 10072.0. Samples: 822110848. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '747.570')] [2023-12-27 16:46:27,637][20381] Updated weights for policy 1, policy_version 1604669 (0.0007) [2023-12-27 16:46:27,700][20381] Updated weights for policy 1, policy_version 1604679 (0.0007) [2023-12-27 16:46:27,753][20381] Updated weights for policy 1, policy_version 1604689 (0.0007) [2023-12-27 16:46:27,792][20347] Updated weights for policy 0, policy_version 1606291 (0.0010) [2023-12-27 16:46:27,840][20347] Updated weights for policy 0, policy_version 1606301 (0.0010) [2023-12-27 16:46:27,888][20347] Updated weights for policy 0, policy_version 1606311 (0.0010) [2023-12-27 16:46:28,460][20381] Updated weights for policy 1, policy_version 1604699 (0.0007) [2023-12-27 16:46:28,517][20381] Updated weights for policy 1, policy_version 1604709 (0.0008) [2023-12-27 16:46:28,574][20347] Updated weights for policy 0, policy_version 1606321 (0.0010) [2023-12-27 16:46:28,574][20381] Updated weights for policy 1, policy_version 1604719 (0.0007) [2023-12-27 16:46:28,629][20347] Updated weights for policy 0, policy_version 1606331 (0.0010) [2023-12-27 16:46:28,683][20347] Updated weights for policy 0, policy_version 1606341 (0.0010) [2023-12-27 16:46:28,740][20347] Updated weights for policy 0, policy_version 1606351 (0.0010) [2023-12-27 16:46:29,277][20381] Updated weights for policy 1, policy_version 1604729 (0.0006) [2023-12-27 16:46:29,341][20381] Updated weights for policy 1, policy_version 1604739 (0.0007) [2023-12-27 16:46:29,411][20381] Updated weights for policy 1, policy_version 1604749 (0.0009) [2023-12-27 16:46:29,434][20347] Updated weights for policy 0, policy_version 1606361 (0.0008) [2023-12-27 16:46:29,472][20381] Updated weights for policy 1, policy_version 1604759 (0.0011) [2023-12-27 16:46:29,498][20347] Updated weights for policy 0, policy_version 1606371 (0.0009) [2023-12-27 16:46:29,546][20347] Updated weights for policy 0, policy_version 1606381 (0.0007) [2023-12-27 16:46:30,186][20381] Updated weights for policy 1, policy_version 1604769 (0.0008) [2023-12-27 16:46:30,192][20347] Updated weights for policy 0, policy_version 1606391 (0.0009) [2023-12-27 16:46:30,244][20381] Updated weights for policy 1, policy_version 1604779 (0.0006) [2023-12-27 16:46:30,249][20347] Updated weights for policy 0, policy_version 1606401 (0.0010) [2023-12-27 16:46:30,302][20381] Updated weights for policy 1, policy_version 1604789 (0.0005) [2023-12-27 16:46:30,307][20347] Updated weights for policy 0, policy_version 1606411 (0.0010) [2023-12-27 16:46:30,978][20347] Updated weights for policy 0, policy_version 1606421 (0.0010) [2023-12-27 16:46:30,979][20381] Updated weights for policy 1, policy_version 1604799 (0.0006) [2023-12-27 16:46:31,027][20381] Updated weights for policy 1, policy_version 1604809 (0.0006) [2023-12-27 16:46:31,041][20347] Updated weights for policy 0, policy_version 1606431 (0.0010) [2023-12-27 16:46:31,086][20381] Updated weights for policy 1, policy_version 1604819 (0.0007) [2023-12-27 16:46:31,096][20347] Updated weights for policy 0, policy_version 1606441 (0.0010) [2023-12-27 16:46:31,813][20381] Updated weights for policy 1, policy_version 1604829 (0.0005) [2023-12-27 16:46:31,823][20347] Updated weights for policy 0, policy_version 1606451 (0.0010) [2023-12-27 16:46:31,869][20381] Updated weights for policy 1, policy_version 1604839 (0.0007) [2023-12-27 16:46:31,888][20347] Updated weights for policy 0, policy_version 1606461 (0.0008) [2023-12-27 16:46:31,921][20381] Updated weights for policy 1, policy_version 1604849 (0.0007) [2023-12-27 16:46:31,948][20347] Updated weights for policy 0, policy_version 1606471 (0.0008) [2023-12-27 16:46:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 822222848. Throughput: 0: 10291.9, 1: 10076.7. Samples: 822232400. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '795.590')] [2023-12-27 16:46:32,620][20381] Updated weights for policy 1, policy_version 1604859 (0.0007) [2023-12-27 16:46:32,672][20381] Updated weights for policy 1, policy_version 1604869 (0.0008) [2023-12-27 16:46:32,695][20347] Updated weights for policy 0, policy_version 1606481 (0.0008) [2023-12-27 16:46:32,723][20381] Updated weights for policy 1, policy_version 1604879 (0.0008) [2023-12-27 16:46:32,750][20347] Updated weights for policy 0, policy_version 1606491 (0.0006) [2023-12-27 16:46:32,811][20347] Updated weights for policy 0, policy_version 1606501 (0.0007) [2023-12-27 16:46:32,874][20347] Updated weights for policy 0, policy_version 1606511 (0.0007) [2023-12-27 16:46:33,367][20381] Updated weights for policy 1, policy_version 1604889 (0.0008) [2023-12-27 16:46:33,422][20381] Updated weights for policy 1, policy_version 1604899 (0.0007) [2023-12-27 16:46:33,431][20347] Updated weights for policy 0, policy_version 1606521 (0.0007) [2023-12-27 16:46:33,476][20381] Updated weights for policy 1, policy_version 1604909 (0.0007) [2023-12-27 16:46:33,482][20347] Updated weights for policy 0, policy_version 1606531 (0.0006) [2023-12-27 16:46:33,531][20381] Updated weights for policy 1, policy_version 1604919 (0.0006) [2023-12-27 16:46:33,533][20347] Updated weights for policy 0, policy_version 1606541 (0.0005) [2023-12-27 16:46:34,098][20347] Updated weights for policy 0, policy_version 1606551 (0.0006) [2023-12-27 16:46:34,156][20347] Updated weights for policy 0, policy_version 1606561 (0.0006) [2023-12-27 16:46:34,217][20347] Updated weights for policy 0, policy_version 1606571 (0.0008) [2023-12-27 16:46:34,275][20381] Updated weights for policy 1, policy_version 1604929 (0.0006) [2023-12-27 16:46:34,329][20381] Updated weights for policy 1, policy_version 1604939 (0.0006) [2023-12-27 16:46:34,387][20381] Updated weights for policy 1, policy_version 1604949 (0.0007) [2023-12-27 16:46:34,885][20347] Updated weights for policy 0, policy_version 1606581 (0.0007) [2023-12-27 16:46:34,934][20347] Updated weights for policy 0, policy_version 1606591 (0.0008) [2023-12-27 16:46:34,986][20347] Updated weights for policy 0, policy_version 1606601 (0.0007) [2023-12-27 16:46:35,057][20381] Updated weights for policy 1, policy_version 1604959 (0.0007) [2023-12-27 16:46:35,118][20381] Updated weights for policy 1, policy_version 1604969 (0.0005) [2023-12-27 16:46:35,177][20381] Updated weights for policy 1, policy_version 1604979 (0.0006) [2023-12-27 16:46:35,729][20347] Updated weights for policy 0, policy_version 1606611 (0.0008) [2023-12-27 16:46:35,767][20381] Updated weights for policy 1, policy_version 1604989 (0.0006) [2023-12-27 16:46:35,785][20347] Updated weights for policy 0, policy_version 1606621 (0.0007) [2023-12-27 16:46:35,823][20381] Updated weights for policy 1, policy_version 1604999 (0.0007) [2023-12-27 16:46:35,843][20347] Updated weights for policy 0, policy_version 1606631 (0.0005) [2023-12-27 16:46:35,875][20381] Updated weights for policy 1, policy_version 1605009 (0.0010) [2023-12-27 16:46:36,506][20347] Updated weights for policy 0, policy_version 1606641 (0.0006) [2023-12-27 16:46:36,543][20381] Updated weights for policy 1, policy_version 1605019 (0.0010) [2023-12-27 16:46:36,563][20347] Updated weights for policy 0, policy_version 1606651 (0.0008) [2023-12-27 16:46:36,607][20381] Updated weights for policy 1, policy_version 1605029 (0.0011) [2023-12-27 16:46:36,619][20347] Updated weights for policy 0, policy_version 1606661 (0.0007) [2023-12-27 16:46:36,666][20381] Updated weights for policy 1, policy_version 1605039 (0.0011) [2023-12-27 16:46:36,670][20347] Updated weights for policy 0, policy_version 1606671 (0.0007) [2023-12-27 16:46:37,321][20381] Updated weights for policy 1, policy_version 1605049 (0.0011) [2023-12-27 16:46:37,323][20347] Updated weights for policy 0, policy_version 1606681 (0.0010) [2023-12-27 16:46:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 822321152. Throughput: 0: 10361.1, 1: 10104.1. Samples: 822295876. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '775.580')] [2023-12-27 16:46:37,370][20381] Updated weights for policy 1, policy_version 1605059 (0.0010) [2023-12-27 16:46:37,373][20347] Updated weights for policy 0, policy_version 1606691 (0.0010) [2023-12-27 16:46:37,423][20381] Updated weights for policy 1, policy_version 1605069 (0.0010) [2023-12-27 16:46:37,439][20347] Updated weights for policy 0, policy_version 1606701 (0.0011) [2023-12-27 16:46:37,454][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001606704_411394048.pth... [2023-12-27 16:46:37,458][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001605456_411074560.pth [2023-12-27 16:46:37,480][20381] Updated weights for policy 1, policy_version 1605079 (0.0010) [2023-12-27 16:46:37,484][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001605080_410943488.pth... [2023-12-27 16:46:37,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001603896_410640384.pth [2023-12-27 16:46:38,131][20381] Updated weights for policy 1, policy_version 1605089 (0.0006) [2023-12-27 16:46:38,148][20347] Updated weights for policy 0, policy_version 1606711 (0.0009) [2023-12-27 16:46:38,184][20381] Updated weights for policy 1, policy_version 1605099 (0.0006) [2023-12-27 16:46:38,196][20347] Updated weights for policy 0, policy_version 1606721 (0.0007) [2023-12-27 16:46:38,241][20381] Updated weights for policy 1, policy_version 1605109 (0.0008) [2023-12-27 16:46:38,244][20347] Updated weights for policy 0, policy_version 1606731 (0.0007) [2023-12-27 16:46:38,921][20381] Updated weights for policy 1, policy_version 1605119 (0.0006) [2023-12-27 16:46:38,977][20347] Updated weights for policy 0, policy_version 1606741 (0.0006) [2023-12-27 16:46:38,978][20381] Updated weights for policy 1, policy_version 1605129 (0.0006) [2023-12-27 16:46:39,028][20381] Updated weights for policy 1, policy_version 1605139 (0.0005) [2023-12-27 16:46:39,028][20347] Updated weights for policy 0, policy_version 1606751 (0.0007) [2023-12-27 16:46:39,078][20347] Updated weights for policy 0, policy_version 1606761 (0.0005) [2023-12-27 16:46:39,630][20381] Updated weights for policy 1, policy_version 1605149 (0.0008) [2023-12-27 16:46:39,695][20381] Updated weights for policy 1, policy_version 1605159 (0.0009) [2023-12-27 16:46:39,754][20381] Updated weights for policy 1, policy_version 1605169 (0.0007) [2023-12-27 16:46:39,796][20347] Updated weights for policy 0, policy_version 1606771 (0.0005) [2023-12-27 16:46:39,863][20347] Updated weights for policy 0, policy_version 1606781 (0.0008) [2023-12-27 16:46:39,928][20347] Updated weights for policy 0, policy_version 1606791 (0.0006) [2023-12-27 16:46:40,439][20381] Updated weights for policy 1, policy_version 1605179 (0.0008) [2023-12-27 16:46:40,501][20381] Updated weights for policy 1, policy_version 1605189 (0.0008) [2023-12-27 16:46:40,550][20381] Updated weights for policy 1, policy_version 1605199 (0.0008) [2023-12-27 16:46:40,619][20347] Updated weights for policy 0, policy_version 1606801 (0.0008) [2023-12-27 16:46:40,678][20347] Updated weights for policy 0, policy_version 1606811 (0.0007) [2023-12-27 16:46:40,736][20347] Updated weights for policy 0, policy_version 1606821 (0.0008) [2023-12-27 16:46:40,797][20347] Updated weights for policy 0, policy_version 1606831 (0.0008) [2023-12-27 16:46:41,195][20381] Updated weights for policy 1, policy_version 1605209 (0.0008) [2023-12-27 16:46:41,256][20381] Updated weights for policy 1, policy_version 1605219 (0.0007) [2023-12-27 16:46:41,318][20381] Updated weights for policy 1, policy_version 1605229 (0.0008) [2023-12-27 16:46:41,386][20381] Updated weights for policy 1, policy_version 1605239 (0.0009) [2023-12-27 16:46:41,559][20347] Updated weights for policy 0, policy_version 1606841 (0.0008) [2023-12-27 16:46:41,619][20347] Updated weights for policy 0, policy_version 1606851 (0.0008) [2023-12-27 16:46:41,677][20347] Updated weights for policy 0, policy_version 1606861 (0.0008) [2023-12-27 16:46:42,153][20381] Updated weights for policy 1, policy_version 1605249 (0.0006) [2023-12-27 16:46:42,212][20381] Updated weights for policy 1, policy_version 1605259 (0.0007) [2023-12-27 16:46:42,264][20381] Updated weights for policy 1, policy_version 1605269 (0.0008) [2023-12-27 16:46:42,361][20347] Updated weights for policy 0, policy_version 1606871 (0.0008) [2023-12-27 16:46:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 822427648. Throughput: 0: 10351.2, 1: 10121.0. Samples: 822419004. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '833.890')] [2023-12-27 16:46:42,429][20347] Updated weights for policy 0, policy_version 1606881 (0.0007) [2023-12-27 16:46:42,488][20347] Updated weights for policy 0, policy_version 1606891 (0.0009) [2023-12-27 16:46:42,951][20381] Updated weights for policy 1, policy_version 1605279 (0.0007) [2023-12-27 16:46:43,002][20381] Updated weights for policy 1, policy_version 1605289 (0.0006) [2023-12-27 16:46:43,054][20381] Updated weights for policy 1, policy_version 1605299 (0.0007) [2023-12-27 16:46:43,133][20347] Updated weights for policy 0, policy_version 1606901 (0.0009) [2023-12-27 16:46:43,189][20347] Updated weights for policy 0, policy_version 1606911 (0.0008) [2023-12-27 16:46:43,239][20347] Updated weights for policy 0, policy_version 1606921 (0.0006) [2023-12-27 16:46:43,713][20381] Updated weights for policy 1, policy_version 1605309 (0.0007) [2023-12-27 16:46:43,765][20381] Updated weights for policy 1, policy_version 1605319 (0.0007) [2023-12-27 16:46:43,824][20381] Updated weights for policy 1, policy_version 1605329 (0.0008) [2023-12-27 16:46:43,949][20347] Updated weights for policy 0, policy_version 1606931 (0.0007) [2023-12-27 16:46:43,998][20347] Updated weights for policy 0, policy_version 1606941 (0.0008) [2023-12-27 16:46:44,044][20347] Updated weights for policy 0, policy_version 1606951 (0.0007) [2023-12-27 16:46:44,502][20381] Updated weights for policy 1, policy_version 1605339 (0.0007) [2023-12-27 16:46:44,555][20381] Updated weights for policy 1, policy_version 1605349 (0.0009) [2023-12-27 16:46:44,602][20381] Updated weights for policy 1, policy_version 1605359 (0.0011) [2023-12-27 16:46:44,767][20347] Updated weights for policy 0, policy_version 1606961 (0.0007) [2023-12-27 16:46:44,833][20347] Updated weights for policy 0, policy_version 1606971 (0.0009) [2023-12-27 16:46:44,896][20347] Updated weights for policy 0, policy_version 1606981 (0.0007) [2023-12-27 16:46:44,958][20347] Updated weights for policy 0, policy_version 1606991 (0.0008) [2023-12-27 16:46:45,247][20381] Updated weights for policy 1, policy_version 1605369 (0.0010) [2023-12-27 16:46:45,307][20381] Updated weights for policy 1, policy_version 1605379 (0.0008) [2023-12-27 16:46:45,365][20381] Updated weights for policy 1, policy_version 1605389 (0.0008) [2023-12-27 16:46:45,418][20381] Updated weights for policy 1, policy_version 1605399 (0.0008) [2023-12-27 16:46:45,643][20347] Updated weights for policy 0, policy_version 1607001 (0.0006) [2023-12-27 16:46:45,710][20347] Updated weights for policy 0, policy_version 1607011 (0.0005) [2023-12-27 16:46:45,778][20347] Updated weights for policy 0, policy_version 1607021 (0.0005) [2023-12-27 16:46:46,053][20381] Updated weights for policy 1, policy_version 1605409 (0.0006) [2023-12-27 16:46:46,114][20381] Updated weights for policy 1, policy_version 1605419 (0.0006) [2023-12-27 16:46:46,168][20381] Updated weights for policy 1, policy_version 1605429 (0.0007) [2023-12-27 16:46:46,300][20347] Updated weights for policy 0, policy_version 1607031 (0.0007) [2023-12-27 16:46:46,349][20347] Updated weights for policy 0, policy_version 1607041 (0.0008) [2023-12-27 16:46:46,407][20347] Updated weights for policy 0, policy_version 1607051 (0.0007) [2023-12-27 16:46:46,738][20381] Updated weights for policy 1, policy_version 1605439 (0.0005) [2023-12-27 16:46:46,790][20381] Updated weights for policy 1, policy_version 1605449 (0.0010) [2023-12-27 16:46:46,840][20381] Updated weights for policy 1, policy_version 1605459 (0.0008) [2023-12-27 16:46:47,005][20347] Updated weights for policy 0, policy_version 1607061 (0.0006) [2023-12-27 16:46:47,059][20347] Updated weights for policy 0, policy_version 1607071 (0.0005) [2023-12-27 16:46:47,113][20347] Updated weights for policy 0, policy_version 1607081 (0.0006) [2023-12-27 16:46:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 822534144. Throughput: 0: 10378.9, 1: 10142.9. Samples: 822544056. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.460')] [2023-12-27 16:46:47,510][20381] Updated weights for policy 1, policy_version 1605469 (0.0007) [2023-12-27 16:46:47,571][20381] Updated weights for policy 1, policy_version 1605479 (0.0007) [2023-12-27 16:46:47,628][20381] Updated weights for policy 1, policy_version 1605489 (0.0008) [2023-12-27 16:46:47,797][20347] Updated weights for policy 0, policy_version 1607091 (0.0008) [2023-12-27 16:46:47,859][20347] Updated weights for policy 0, policy_version 1607101 (0.0011) [2023-12-27 16:46:47,924][20347] Updated weights for policy 0, policy_version 1607111 (0.0010) [2023-12-27 16:46:48,385][20381] Updated weights for policy 1, policy_version 1605499 (0.0007) [2023-12-27 16:46:48,436][20381] Updated weights for policy 1, policy_version 1605509 (0.0008) [2023-12-27 16:46:48,493][20381] Updated weights for policy 1, policy_version 1605519 (0.0007) [2023-12-27 16:46:48,519][20347] Updated weights for policy 0, policy_version 1607121 (0.0010) [2023-12-27 16:46:48,578][20347] Updated weights for policy 0, policy_version 1607131 (0.0010) [2023-12-27 16:46:48,639][20347] Updated weights for policy 0, policy_version 1607141 (0.0007) [2023-12-27 16:46:48,704][20347] Updated weights for policy 0, policy_version 1607151 (0.0010) [2023-12-27 16:46:49,154][20381] Updated weights for policy 1, policy_version 1605529 (0.0006) [2023-12-27 16:46:49,202][20381] Updated weights for policy 1, policy_version 1605539 (0.0007) [2023-12-27 16:46:49,267][20381] Updated weights for policy 1, policy_version 1605549 (0.0008) [2023-12-27 16:46:49,329][20381] Updated weights for policy 1, policy_version 1605559 (0.0007) [2023-12-27 16:46:49,358][20347] Updated weights for policy 0, policy_version 1607161 (0.0008) [2023-12-27 16:46:49,428][20347] Updated weights for policy 0, policy_version 1607171 (0.0007) [2023-12-27 16:46:49,486][20347] Updated weights for policy 0, policy_version 1607181 (0.0006) [2023-12-27 16:46:50,094][20381] Updated weights for policy 1, policy_version 1605569 (0.0008) [2023-12-27 16:46:50,156][20381] Updated weights for policy 1, policy_version 1605579 (0.0007) [2023-12-27 16:46:50,213][20381] Updated weights for policy 1, policy_version 1605589 (0.0007) [2023-12-27 16:46:50,251][20347] Updated weights for policy 0, policy_version 1607191 (0.0007) [2023-12-27 16:46:50,310][20347] Updated weights for policy 0, policy_version 1607201 (0.0008) [2023-12-27 16:46:50,371][20347] Updated weights for policy 0, policy_version 1607211 (0.0009) [2023-12-27 16:46:50,923][20381] Updated weights for policy 1, policy_version 1605599 (0.0005) [2023-12-27 16:46:50,984][20381] Updated weights for policy 1, policy_version 1605609 (0.0005) [2023-12-27 16:46:51,046][20381] Updated weights for policy 1, policy_version 1605619 (0.0006) [2023-12-27 16:46:51,081][20347] Updated weights for policy 0, policy_version 1607221 (0.0009) [2023-12-27 16:46:51,144][20347] Updated weights for policy 0, policy_version 1607231 (0.0010) [2023-12-27 16:46:51,202][20347] Updated weights for policy 0, policy_version 1607241 (0.0008) [2023-12-27 16:46:51,714][20381] Updated weights for policy 1, policy_version 1605629 (0.0006) [2023-12-27 16:46:51,770][20381] Updated weights for policy 1, policy_version 1605639 (0.0007) [2023-12-27 16:46:51,825][20381] Updated weights for policy 1, policy_version 1605649 (0.0008) [2023-12-27 16:46:51,921][20347] Updated weights for policy 0, policy_version 1607251 (0.0007) [2023-12-27 16:46:51,975][20347] Updated weights for policy 0, policy_version 1607261 (0.0007) [2023-12-27 16:46:52,024][20347] Updated weights for policy 0, policy_version 1607271 (0.0008) [2023-12-27 16:46:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 822632448. Throughput: 0: 10362.5, 1: 10056.9. Samples: 822603576. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '755.900')] [2023-12-27 16:46:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001605656_411090944.pth... [2023-12-27 16:46:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001607280_411541504.pth... [2023-12-27 16:46:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001604472_410787840.pth [2023-12-27 16:46:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001606064_411230208.pth [2023-12-27 16:46:52,476][20381] Updated weights for policy 1, policy_version 1605659 (0.0006) [2023-12-27 16:46:52,533][20381] Updated weights for policy 1, policy_version 1605669 (0.0007) [2023-12-27 16:46:52,592][20381] Updated weights for policy 1, policy_version 1605679 (0.0008) [2023-12-27 16:46:52,754][20347] Updated weights for policy 0, policy_version 1607281 (0.0007) [2023-12-27 16:46:52,818][20347] Updated weights for policy 0, policy_version 1607291 (0.0008) [2023-12-27 16:46:52,883][20347] Updated weights for policy 0, policy_version 1607301 (0.0008) [2023-12-27 16:46:52,939][20347] Updated weights for policy 0, policy_version 1607311 (0.0008) [2023-12-27 16:46:53,207][20381] Updated weights for policy 1, policy_version 1605689 (0.0007) [2023-12-27 16:46:53,264][20381] Updated weights for policy 1, policy_version 1605699 (0.0010) [2023-12-27 16:46:53,314][20381] Updated weights for policy 1, policy_version 1605709 (0.0007) [2023-12-27 16:46:53,365][20381] Updated weights for policy 1, policy_version 1605719 (0.0010) [2023-12-27 16:46:53,574][20347] Updated weights for policy 0, policy_version 1607321 (0.0007) [2023-12-27 16:46:53,631][20347] Updated weights for policy 0, policy_version 1607331 (0.0007) [2023-12-27 16:46:53,692][20347] Updated weights for policy 0, policy_version 1607341 (0.0005) [2023-12-27 16:46:54,035][20381] Updated weights for policy 1, policy_version 1605729 (0.0011) [2023-12-27 16:46:54,093][20381] Updated weights for policy 1, policy_version 1605739 (0.0011) [2023-12-27 16:46:54,155][20381] Updated weights for policy 1, policy_version 1605749 (0.0010) [2023-12-27 16:46:54,329][20347] Updated weights for policy 0, policy_version 1607351 (0.0007) [2023-12-27 16:46:54,393][20347] Updated weights for policy 0, policy_version 1607361 (0.0007) [2023-12-27 16:46:54,455][20347] Updated weights for policy 0, policy_version 1607371 (0.0008) [2023-12-27 16:46:54,766][20381] Updated weights for policy 1, policy_version 1605759 (0.0007) [2023-12-27 16:46:54,821][20381] Updated weights for policy 1, policy_version 1605769 (0.0005) [2023-12-27 16:46:54,870][20381] Updated weights for policy 1, policy_version 1605779 (0.0007) [2023-12-27 16:46:55,059][20347] Updated weights for policy 0, policy_version 1607381 (0.0007) [2023-12-27 16:46:55,118][20347] Updated weights for policy 0, policy_version 1607391 (0.0005) [2023-12-27 16:46:55,173][20347] Updated weights for policy 0, policy_version 1607401 (0.0006) [2023-12-27 16:46:55,541][20381] Updated weights for policy 1, policy_version 1605789 (0.0008) [2023-12-27 16:46:55,603][20381] Updated weights for policy 1, policy_version 1605799 (0.0008) [2023-12-27 16:46:55,653][20381] Updated weights for policy 1, policy_version 1605809 (0.0007) [2023-12-27 16:46:55,815][20347] Updated weights for policy 0, policy_version 1607411 (0.0006) [2023-12-27 16:46:55,873][20347] Updated weights for policy 0, policy_version 1607421 (0.0006) [2023-12-27 16:46:55,928][20347] Updated weights for policy 0, policy_version 1607431 (0.0006) [2023-12-27 16:46:56,356][20381] Updated weights for policy 1, policy_version 1605819 (0.0006) [2023-12-27 16:46:56,405][20381] Updated weights for policy 1, policy_version 1605829 (0.0008) [2023-12-27 16:46:56,454][20381] Updated weights for policy 1, policy_version 1605839 (0.0008) [2023-12-27 16:46:56,610][20347] Updated weights for policy 0, policy_version 1607441 (0.0007) [2023-12-27 16:46:56,659][20347] Updated weights for policy 0, policy_version 1607451 (0.0007) [2023-12-27 16:46:56,708][20347] Updated weights for policy 0, policy_version 1607461 (0.0008) [2023-12-27 16:46:56,761][20347] Updated weights for policy 0, policy_version 1607471 (0.0008) [2023-12-27 16:46:57,055][20381] Updated weights for policy 1, policy_version 1605849 (0.0007) [2023-12-27 16:46:57,112][20381] Updated weights for policy 1, policy_version 1605859 (0.0005) [2023-12-27 16:46:57,172][20381] Updated weights for policy 1, policy_version 1605869 (0.0005) [2023-12-27 16:46:57,231][20381] Updated weights for policy 1, policy_version 1605879 (0.0005) [2023-12-27 16:46:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 822738944. Throughput: 0: 10369.3, 1: 10075.7. Samples: 822727744. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:46:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '785.350')] [2023-12-27 16:46:57,468][20347] Updated weights for policy 0, policy_version 1607481 (0.0007) [2023-12-27 16:46:57,515][20347] Updated weights for policy 0, policy_version 1607491 (0.0007) [2023-12-27 16:46:57,559][20347] Updated weights for policy 0, policy_version 1607501 (0.0007) [2023-12-27 16:46:57,778][20381] Updated weights for policy 1, policy_version 1605889 (0.0010) [2023-12-27 16:46:57,827][20381] Updated weights for policy 1, policy_version 1605899 (0.0010) [2023-12-27 16:46:57,882][20381] Updated weights for policy 1, policy_version 1605909 (0.0010) [2023-12-27 16:46:58,306][20347] Updated weights for policy 0, policy_version 1607511 (0.0008) [2023-12-27 16:46:58,381][20347] Updated weights for policy 0, policy_version 1607521 (0.0008) [2023-12-27 16:46:58,444][20347] Updated weights for policy 0, policy_version 1607531 (0.0007) [2023-12-27 16:46:58,577][20381] Updated weights for policy 1, policy_version 1605919 (0.0009) [2023-12-27 16:46:58,633][20381] Updated weights for policy 1, policy_version 1605929 (0.0008) [2023-12-27 16:46:58,700][20381] Updated weights for policy 1, policy_version 1605939 (0.0011) [2023-12-27 16:46:59,181][20347] Updated weights for policy 0, policy_version 1607541 (0.0007) [2023-12-27 16:46:59,244][20347] Updated weights for policy 0, policy_version 1607551 (0.0009) [2023-12-27 16:46:59,314][20347] Updated weights for policy 0, policy_version 1607561 (0.0009) [2023-12-27 16:46:59,484][20381] Updated weights for policy 1, policy_version 1605949 (0.0009) [2023-12-27 16:46:59,541][20381] Updated weights for policy 1, policy_version 1605959 (0.0008) [2023-12-27 16:46:59,599][20381] Updated weights for policy 1, policy_version 1605969 (0.0008) [2023-12-27 16:47:00,028][20347] Updated weights for policy 0, policy_version 1607571 (0.0007) [2023-12-27 16:47:00,091][20347] Updated weights for policy 0, policy_version 1607581 (0.0005) [2023-12-27 16:47:00,150][20347] Updated weights for policy 0, policy_version 1607591 (0.0005) [2023-12-27 16:47:00,338][20381] Updated weights for policy 1, policy_version 1605979 (0.0007) [2023-12-27 16:47:00,397][20381] Updated weights for policy 1, policy_version 1605989 (0.0007) [2023-12-27 16:47:00,453][20381] Updated weights for policy 1, policy_version 1605999 (0.0008) [2023-12-27 16:47:00,740][20347] Updated weights for policy 0, policy_version 1607601 (0.0006) [2023-12-27 16:47:00,788][20347] Updated weights for policy 0, policy_version 1607611 (0.0007) [2023-12-27 16:47:00,848][20347] Updated weights for policy 0, policy_version 1607621 (0.0007) [2023-12-27 16:47:00,904][20347] Updated weights for policy 0, policy_version 1607631 (0.0007) [2023-12-27 16:47:01,070][20381] Updated weights for policy 1, policy_version 1606009 (0.0008) [2023-12-27 16:47:01,130][20381] Updated weights for policy 1, policy_version 1606019 (0.0007) [2023-12-27 16:47:01,189][20381] Updated weights for policy 1, policy_version 1606029 (0.0006) [2023-12-27 16:47:01,248][20381] Updated weights for policy 1, policy_version 1606039 (0.0007) [2023-12-27 16:47:01,654][20347] Updated weights for policy 0, policy_version 1607641 (0.0010) [2023-12-27 16:47:01,717][20347] Updated weights for policy 0, policy_version 1607651 (0.0010) [2023-12-27 16:47:01,778][20347] Updated weights for policy 0, policy_version 1607661 (0.0010) [2023-12-27 16:47:01,918][20381] Updated weights for policy 1, policy_version 1606049 (0.0007) [2023-12-27 16:47:01,972][20381] Updated weights for policy 1, policy_version 1606059 (0.0007) [2023-12-27 16:47:02,021][20381] Updated weights for policy 1, policy_version 1606069 (0.0007) [2023-12-27 16:47:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 822837248. Throughput: 0: 10262.3, 1: 10133.0. Samples: 822848952. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '736.140')] [2023-12-27 16:47:02,432][20347] Updated weights for policy 0, policy_version 1607671 (0.0009) [2023-12-27 16:47:02,492][20347] Updated weights for policy 0, policy_version 1607681 (0.0009) [2023-12-27 16:47:02,538][20347] Updated weights for policy 0, policy_version 1607691 (0.0010) [2023-12-27 16:47:02,704][20381] Updated weights for policy 1, policy_version 1606079 (0.0006) [2023-12-27 16:47:02,764][20381] Updated weights for policy 1, policy_version 1606089 (0.0006) [2023-12-27 16:47:02,823][20381] Updated weights for policy 1, policy_version 1606099 (0.0006) [2023-12-27 16:47:03,237][20347] Updated weights for policy 0, policy_version 1607701 (0.0011) [2023-12-27 16:47:03,298][20347] Updated weights for policy 0, policy_version 1607711 (0.0010) [2023-12-27 16:47:03,350][20347] Updated weights for policy 0, policy_version 1607721 (0.0008) [2023-12-27 16:47:03,477][20381] Updated weights for policy 1, policy_version 1606109 (0.0006) [2023-12-27 16:47:03,531][20381] Updated weights for policy 1, policy_version 1606119 (0.0005) [2023-12-27 16:47:03,585][20381] Updated weights for policy 1, policy_version 1606129 (0.0005) [2023-12-27 16:47:03,996][20347] Updated weights for policy 0, policy_version 1607731 (0.0008) [2023-12-27 16:47:04,052][20347] Updated weights for policy 0, policy_version 1607741 (0.0007) [2023-12-27 16:47:04,117][20347] Updated weights for policy 0, policy_version 1607751 (0.0010) [2023-12-27 16:47:04,261][20381] Updated weights for policy 1, policy_version 1606139 (0.0005) [2023-12-27 16:47:04,320][20381] Updated weights for policy 1, policy_version 1606149 (0.0007) [2023-12-27 16:47:04,382][20381] Updated weights for policy 1, policy_version 1606159 (0.0008) [2023-12-27 16:47:04,820][20347] Updated weights for policy 0, policy_version 1607761 (0.0010) [2023-12-27 16:47:04,868][20347] Updated weights for policy 0, policy_version 1607771 (0.0010) [2023-12-27 16:47:04,913][20347] Updated weights for policy 0, policy_version 1607781 (0.0010) [2023-12-27 16:47:04,964][20347] Updated weights for policy 0, policy_version 1607791 (0.0010) [2023-12-27 16:47:04,968][20381] Updated weights for policy 1, policy_version 1606169 (0.0008) [2023-12-27 16:47:05,014][20381] Updated weights for policy 1, policy_version 1606179 (0.0006) [2023-12-27 16:47:05,066][20381] Updated weights for policy 1, policy_version 1606189 (0.0006) [2023-12-27 16:47:05,120][20381] Updated weights for policy 1, policy_version 1606199 (0.0007) [2023-12-27 16:47:05,616][20347] Updated weights for policy 0, policy_version 1607801 (0.0010) [2023-12-27 16:47:05,669][20347] Updated weights for policy 0, policy_version 1607811 (0.0007) [2023-12-27 16:47:05,703][20381] Updated weights for policy 1, policy_version 1606209 (0.0008) [2023-12-27 16:47:05,716][20347] Updated weights for policy 0, policy_version 1607821 (0.0005) [2023-12-27 16:47:05,747][20381] Updated weights for policy 1, policy_version 1606219 (0.0006) [2023-12-27 16:47:05,794][20381] Updated weights for policy 1, policy_version 1606229 (0.0007) [2023-12-27 16:47:06,442][20347] Updated weights for policy 0, policy_version 1607831 (0.0008) [2023-12-27 16:47:06,444][20381] Updated weights for policy 1, policy_version 1606239 (0.0006) [2023-12-27 16:47:06,502][20381] Updated weights for policy 1, policy_version 1606249 (0.0006) [2023-12-27 16:47:06,504][20347] Updated weights for policy 0, policy_version 1607841 (0.0006) [2023-12-27 16:47:06,556][20381] Updated weights for policy 1, policy_version 1606259 (0.0006) [2023-12-27 16:47:06,565][20347] Updated weights for policy 0, policy_version 1607851 (0.0007) [2023-12-27 16:47:07,176][20381] Updated weights for policy 1, policy_version 1606269 (0.0007) [2023-12-27 16:47:07,229][20381] Updated weights for policy 1, policy_version 1606279 (0.0008) [2023-12-27 16:47:07,266][20347] Updated weights for policy 0, policy_version 1607861 (0.0007) [2023-12-27 16:47:07,286][20381] Updated weights for policy 1, policy_version 1606289 (0.0008) [2023-12-27 16:47:07,322][20347] Updated weights for policy 0, policy_version 1607871 (0.0007) [2023-12-27 16:47:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 822943744. Throughput: 0: 10278.8, 1: 10193.5. Samples: 822911572. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '764.940')] [2023-12-27 16:47:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001606296_411254784.pth... [2023-12-27 16:47:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001605080_410943488.pth [2023-12-27 16:47:07,382][20347] Updated weights for policy 0, policy_version 1607881 (0.0008) [2023-12-27 16:47:07,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001607888_411697152.pth... [2023-12-27 16:47:07,424][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001606704_411394048.pth [2023-12-27 16:47:07,896][20381] Updated weights for policy 1, policy_version 1606299 (0.0009) [2023-12-27 16:47:07,944][20381] Updated weights for policy 1, policy_version 1606309 (0.0006) [2023-12-27 16:47:07,995][20381] Updated weights for policy 1, policy_version 1606319 (0.0006) [2023-12-27 16:47:08,031][20347] Updated weights for policy 0, policy_version 1607891 (0.0007) [2023-12-27 16:47:08,094][20347] Updated weights for policy 0, policy_version 1607901 (0.0008) [2023-12-27 16:47:08,153][20347] Updated weights for policy 0, policy_version 1607911 (0.0007) [2023-12-27 16:47:08,674][20381] Updated weights for policy 1, policy_version 1606329 (0.0006) [2023-12-27 16:47:08,728][20381] Updated weights for policy 1, policy_version 1606339 (0.0007) [2023-12-27 16:47:08,788][20381] Updated weights for policy 1, policy_version 1606349 (0.0008) [2023-12-27 16:47:08,849][20381] Updated weights for policy 1, policy_version 1606359 (0.0007) [2023-12-27 16:47:08,874][20347] Updated weights for policy 0, policy_version 1607921 (0.0008) [2023-12-27 16:47:08,930][20347] Updated weights for policy 0, policy_version 1607931 (0.0006) [2023-12-27 16:47:08,978][20347] Updated weights for policy 0, policy_version 1607941 (0.0007) [2023-12-27 16:47:09,027][20347] Updated weights for policy 0, policy_version 1607951 (0.0006) [2023-12-27 16:47:09,561][20381] Updated weights for policy 1, policy_version 1606369 (0.0008) [2023-12-27 16:47:09,613][20381] Updated weights for policy 1, policy_version 1606379 (0.0007) [2023-12-27 16:47:09,670][20381] Updated weights for policy 1, policy_version 1606389 (0.0008) [2023-12-27 16:47:09,748][20347] Updated weights for policy 0, policy_version 1607961 (0.0007) [2023-12-27 16:47:09,808][20347] Updated weights for policy 0, policy_version 1607971 (0.0008) [2023-12-27 16:47:09,879][20347] Updated weights for policy 0, policy_version 1607981 (0.0008) [2023-12-27 16:47:10,344][20381] Updated weights for policy 1, policy_version 1606399 (0.0010) [2023-12-27 16:47:10,404][20381] Updated weights for policy 1, policy_version 1606409 (0.0006) [2023-12-27 16:47:10,461][20381] Updated weights for policy 1, policy_version 1606419 (0.0006) [2023-12-27 16:47:10,622][20347] Updated weights for policy 0, policy_version 1607991 (0.0008) [2023-12-27 16:47:10,672][20347] Updated weights for policy 0, policy_version 1608001 (0.0008) [2023-12-27 16:47:10,734][20347] Updated weights for policy 0, policy_version 1608011 (0.0008) [2023-12-27 16:47:11,026][20381] Updated weights for policy 1, policy_version 1606429 (0.0007) [2023-12-27 16:47:11,090][20381] Updated weights for policy 1, policy_version 1606439 (0.0009) [2023-12-27 16:47:11,153][20381] Updated weights for policy 1, policy_version 1606449 (0.0008) [2023-12-27 16:47:11,501][20347] Updated weights for policy 0, policy_version 1608021 (0.0007) [2023-12-27 16:47:11,564][20347] Updated weights for policy 0, policy_version 1608031 (0.0007) [2023-12-27 16:47:11,625][20347] Updated weights for policy 0, policy_version 1608041 (0.0007) [2023-12-27 16:47:11,910][20381] Updated weights for policy 1, policy_version 1606459 (0.0009) [2023-12-27 16:47:11,970][20381] Updated weights for policy 1, policy_version 1606469 (0.0011) [2023-12-27 16:47:12,030][20381] Updated weights for policy 1, policy_version 1606479 (0.0011) [2023-12-27 16:47:12,347][20347] Updated weights for policy 0, policy_version 1608051 (0.0008) [2023-12-27 16:47:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 823042048. Throughput: 0: 10174.9, 1: 10363.5. Samples: 823035072. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.070')] [2023-12-27 16:47:12,410][20347] Updated weights for policy 0, policy_version 1608061 (0.0008) [2023-12-27 16:47:12,468][20347] Updated weights for policy 0, policy_version 1608071 (0.0008) [2023-12-27 16:47:12,684][20381] Updated weights for policy 1, policy_version 1606489 (0.0009) [2023-12-27 16:47:12,738][20381] Updated weights for policy 1, policy_version 1606499 (0.0007) [2023-12-27 16:47:12,801][20381] Updated weights for policy 1, policy_version 1606509 (0.0009) [2023-12-27 16:47:12,851][20381] Updated weights for policy 1, policy_version 1606519 (0.0008) [2023-12-27 16:47:13,114][20347] Updated weights for policy 0, policy_version 1608081 (0.0008) [2023-12-27 16:47:13,170][20347] Updated weights for policy 0, policy_version 1608091 (0.0008) [2023-12-27 16:47:13,229][20347] Updated weights for policy 0, policy_version 1608101 (0.0008) [2023-12-27 16:47:13,288][20347] Updated weights for policy 0, policy_version 1608111 (0.0008) [2023-12-27 16:47:13,452][20381] Updated weights for policy 1, policy_version 1606529 (0.0008) [2023-12-27 16:47:13,506][20381] Updated weights for policy 1, policy_version 1606539 (0.0007) [2023-12-27 16:47:13,552][20381] Updated weights for policy 1, policy_version 1606549 (0.0007) [2023-12-27 16:47:13,992][20347] Updated weights for policy 0, policy_version 1608121 (0.0007) [2023-12-27 16:47:14,050][20347] Updated weights for policy 0, policy_version 1608131 (0.0007) [2023-12-27 16:47:14,111][20347] Updated weights for policy 0, policy_version 1608141 (0.0006) [2023-12-27 16:47:14,246][20381] Updated weights for policy 1, policy_version 1606559 (0.0008) [2023-12-27 16:47:14,303][20381] Updated weights for policy 1, policy_version 1606569 (0.0005) [2023-12-27 16:47:14,359][20381] Updated weights for policy 1, policy_version 1606579 (0.0005) [2023-12-27 16:47:14,793][20347] Updated weights for policy 0, policy_version 1608151 (0.0007) [2023-12-27 16:47:14,855][20347] Updated weights for policy 0, policy_version 1608161 (0.0009) [2023-12-27 16:47:14,916][20347] Updated weights for policy 0, policy_version 1608171 (0.0008) [2023-12-27 16:47:14,946][20381] Updated weights for policy 1, policy_version 1606589 (0.0006) [2023-12-27 16:47:15,008][20381] Updated weights for policy 1, policy_version 1606599 (0.0006) [2023-12-27 16:47:15,073][20381] Updated weights for policy 1, policy_version 1606609 (0.0007) [2023-12-27 16:47:15,648][20347] Updated weights for policy 0, policy_version 1608181 (0.0008) [2023-12-27 16:47:15,655][20381] Updated weights for policy 1, policy_version 1606619 (0.0007) [2023-12-27 16:47:15,706][20381] Updated weights for policy 1, policy_version 1606629 (0.0006) [2023-12-27 16:47:15,706][20347] Updated weights for policy 0, policy_version 1608191 (0.0008) [2023-12-27 16:47:15,761][20381] Updated weights for policy 1, policy_version 1606639 (0.0005) [2023-12-27 16:47:15,766][20347] Updated weights for policy 0, policy_version 1608201 (0.0007) [2023-12-27 16:47:16,395][20381] Updated weights for policy 1, policy_version 1606649 (0.0006) [2023-12-27 16:47:16,450][20381] Updated weights for policy 1, policy_version 1606659 (0.0007) [2023-12-27 16:47:16,472][20347] Updated weights for policy 0, policy_version 1608211 (0.0008) [2023-12-27 16:47:16,506][20381] Updated weights for policy 1, policy_version 1606669 (0.0007) [2023-12-27 16:47:16,520][20347] Updated weights for policy 0, policy_version 1608221 (0.0008) [2023-12-27 16:47:16,558][20381] Updated weights for policy 1, policy_version 1606679 (0.0005) [2023-12-27 16:47:16,575][20347] Updated weights for policy 0, policy_version 1608231 (0.0008) [2023-12-27 16:47:17,220][20381] Updated weights for policy 1, policy_version 1606689 (0.0008) [2023-12-27 16:47:17,284][20381] Updated weights for policy 1, policy_version 1606699 (0.0007) [2023-12-27 16:47:17,288][20347] Updated weights for policy 0, policy_version 1608241 (0.0007) [2023-12-27 16:47:17,333][20347] Updated weights for policy 0, policy_version 1608251 (0.0006) [2023-12-27 16:47:17,346][20381] Updated weights for policy 1, policy_version 1606709 (0.0006) [2023-12-27 16:47:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 823148544. Throughput: 0: 10120.0, 1: 10460.3. Samples: 823158512. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:17,363][19170] Avg episode reward: [(0, '990.290'), (1, '815.400')] [2023-12-27 16:47:17,380][20347] Updated weights for policy 0, policy_version 1608261 (0.0007) [2023-12-27 16:47:17,425][20347] Updated weights for policy 0, policy_version 1608271 (0.0008) [2023-12-27 16:47:18,056][20381] Updated weights for policy 1, policy_version 1606719 (0.0009) [2023-12-27 16:47:18,115][20381] Updated weights for policy 1, policy_version 1606729 (0.0011) [2023-12-27 16:47:18,160][20347] Updated weights for policy 0, policy_version 1608281 (0.0006) [2023-12-27 16:47:18,181][20381] Updated weights for policy 1, policy_version 1606739 (0.0011) [2023-12-27 16:47:18,214][20347] Updated weights for policy 0, policy_version 1608291 (0.0007) [2023-12-27 16:47:18,262][20347] Updated weights for policy 0, policy_version 1608301 (0.0009) [2023-12-27 16:47:18,881][20381] Updated weights for policy 1, policy_version 1606749 (0.0011) [2023-12-27 16:47:18,930][20381] Updated weights for policy 1, policy_version 1606759 (0.0011) [2023-12-27 16:47:18,950][20347] Updated weights for policy 0, policy_version 1608311 (0.0007) [2023-12-27 16:47:18,983][20381] Updated weights for policy 1, policy_version 1606769 (0.0011) [2023-12-27 16:47:19,007][20347] Updated weights for policy 0, policy_version 1608321 (0.0005) [2023-12-27 16:47:19,056][20347] Updated weights for policy 0, policy_version 1608331 (0.0008) [2023-12-27 16:47:19,724][20381] Updated weights for policy 1, policy_version 1606779 (0.0011) [2023-12-27 16:47:19,756][20347] Updated weights for policy 0, policy_version 1608341 (0.0010) [2023-12-27 16:47:19,781][20381] Updated weights for policy 1, policy_version 1606789 (0.0011) [2023-12-27 16:47:19,816][20347] Updated weights for policy 0, policy_version 1608351 (0.0011) [2023-12-27 16:47:19,842][20381] Updated weights for policy 1, policy_version 1606799 (0.0011) [2023-12-27 16:47:19,879][20347] Updated weights for policy 0, policy_version 1608361 (0.0011) [2023-12-27 16:47:20,579][20347] Updated weights for policy 0, policy_version 1608371 (0.0008) [2023-12-27 16:47:20,581][20381] Updated weights for policy 1, policy_version 1606809 (0.0010) [2023-12-27 16:47:20,642][20347] Updated weights for policy 0, policy_version 1608381 (0.0009) [2023-12-27 16:47:20,642][20381] Updated weights for policy 1, policy_version 1606819 (0.0010) [2023-12-27 16:47:20,699][20347] Updated weights for policy 0, policy_version 1608391 (0.0007) [2023-12-27 16:47:20,703][20381] Updated weights for policy 1, policy_version 1606829 (0.0011) [2023-12-27 16:47:20,760][20381] Updated weights for policy 1, policy_version 1606839 (0.0011) [2023-12-27 16:47:21,465][20347] Updated weights for policy 0, policy_version 1608401 (0.0007) [2023-12-27 16:47:21,491][20381] Updated weights for policy 1, policy_version 1606849 (0.0007) [2023-12-27 16:47:21,522][20347] Updated weights for policy 0, policy_version 1608411 (0.0007) [2023-12-27 16:47:21,545][20381] Updated weights for policy 1, policy_version 1606859 (0.0007) [2023-12-27 16:47:21,583][20347] Updated weights for policy 0, policy_version 1608421 (0.0008) [2023-12-27 16:47:21,600][20381] Updated weights for policy 1, policy_version 1606869 (0.0007) [2023-12-27 16:47:21,642][20347] Updated weights for policy 0, policy_version 1608431 (0.0007) [2023-12-27 16:47:22,359][20381] Updated weights for policy 1, policy_version 1606879 (0.0009) [2023-12-27 16:47:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 823238656. Throughput: 0: 10059.1, 1: 10415.9. Samples: 823217252. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:22,363][19170] Avg episode reward: [(0, '990.290'), (1, '863.500')] [2023-12-27 16:47:22,400][20347] Updated weights for policy 0, policy_version 1608441 (0.0009) [2023-12-27 16:47:22,426][20381] Updated weights for policy 1, policy_version 1606889 (0.0009) [2023-12-27 16:47:22,462][20347] Updated weights for policy 0, policy_version 1608451 (0.0006) [2023-12-27 16:47:22,487][20381] Updated weights for policy 1, policy_version 1606899 (0.0011) [2023-12-27 16:47:22,514][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001606904_411410432.pth... [2023-12-27 16:47:22,517][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001605656_411090944.pth [2023-12-27 16:47:22,523][20347] Updated weights for policy 0, policy_version 1608461 (0.0006) [2023-12-27 16:47:22,538][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001608464_411844608.pth... [2023-12-27 16:47:22,542][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001607280_411541504.pth [2023-12-27 16:47:23,158][20381] Updated weights for policy 1, policy_version 1606909 (0.0008) [2023-12-27 16:47:23,216][20381] Updated weights for policy 1, policy_version 1606919 (0.0007) [2023-12-27 16:47:23,218][20347] Updated weights for policy 0, policy_version 1608471 (0.0007) [2023-12-27 16:47:23,272][20381] Updated weights for policy 1, policy_version 1606929 (0.0007) [2023-12-27 16:47:23,280][20347] Updated weights for policy 0, policy_version 1608481 (0.0007) [2023-12-27 16:47:23,346][20347] Updated weights for policy 0, policy_version 1608491 (0.0008) [2023-12-27 16:47:23,924][20381] Updated weights for policy 1, policy_version 1606939 (0.0006) [2023-12-27 16:47:23,988][20381] Updated weights for policy 1, policy_version 1606949 (0.0008) [2023-12-27 16:47:24,020][20347] Updated weights for policy 0, policy_version 1608501 (0.0006) [2023-12-27 16:47:24,044][20381] Updated weights for policy 1, policy_version 1606959 (0.0008) [2023-12-27 16:47:24,070][20347] Updated weights for policy 0, policy_version 1608511 (0.0005) [2023-12-27 16:47:24,126][20347] Updated weights for policy 0, policy_version 1608521 (0.0006) [2023-12-27 16:47:24,758][20347] Updated weights for policy 0, policy_version 1608531 (0.0007) [2023-12-27 16:47:24,765][20381] Updated weights for policy 1, policy_version 1606969 (0.0008) [2023-12-27 16:47:24,815][20347] Updated weights for policy 0, policy_version 1608541 (0.0007) [2023-12-27 16:47:24,820][20381] Updated weights for policy 1, policy_version 1606979 (0.0009) [2023-12-27 16:47:24,866][20347] Updated weights for policy 0, policy_version 1608551 (0.0005) [2023-12-27 16:47:24,869][20381] Updated weights for policy 1, policy_version 1606989 (0.0007) [2023-12-27 16:47:24,926][20381] Updated weights for policy 1, policy_version 1606999 (0.0008) [2023-12-27 16:47:25,445][20347] Updated weights for policy 0, policy_version 1608561 (0.0007) [2023-12-27 16:47:25,503][20347] Updated weights for policy 0, policy_version 1608571 (0.0008) [2023-12-27 16:47:25,561][20347] Updated weights for policy 0, policy_version 1608581 (0.0005) [2023-12-27 16:47:25,568][20381] Updated weights for policy 1, policy_version 1607009 (0.0007) [2023-12-27 16:47:25,612][20347] Updated weights for policy 0, policy_version 1608591 (0.0006) [2023-12-27 16:47:25,627][20381] Updated weights for policy 1, policy_version 1607019 (0.0008) [2023-12-27 16:47:25,685][20381] Updated weights for policy 1, policy_version 1607029 (0.0008) [2023-12-27 16:47:26,199][20347] Updated weights for policy 0, policy_version 1608601 (0.0008) [2023-12-27 16:47:26,251][20347] Updated weights for policy 0, policy_version 1608611 (0.0007) [2023-12-27 16:47:26,301][20347] Updated weights for policy 0, policy_version 1608621 (0.0007) [2023-12-27 16:47:26,375][20381] Updated weights for policy 1, policy_version 1607039 (0.0007) [2023-12-27 16:47:26,432][20381] Updated weights for policy 1, policy_version 1607049 (0.0005) [2023-12-27 16:47:26,487][20381] Updated weights for policy 1, policy_version 1607059 (0.0005) [2023-12-27 16:47:26,945][20347] Updated weights for policy 0, policy_version 1608631 (0.0010) [2023-12-27 16:47:26,998][20347] Updated weights for policy 0, policy_version 1608641 (0.0008) [2023-12-27 16:47:27,058][20347] Updated weights for policy 0, policy_version 1608651 (0.0006) [2023-12-27 16:47:27,090][20381] Updated weights for policy 1, policy_version 1607069 (0.0007) [2023-12-27 16:47:27,147][20381] Updated weights for policy 1, policy_version 1607079 (0.0007) [2023-12-27 16:47:27,200][20381] Updated weights for policy 1, policy_version 1607089 (0.0008) [2023-12-27 16:47:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 823353344. Throughput: 0: 10075.6, 1: 10337.1. Samples: 823337576. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '825.230')] [2023-12-27 16:47:27,650][20347] Updated weights for policy 0, policy_version 1608661 (0.0007) [2023-12-27 16:47:27,720][20347] Updated weights for policy 0, policy_version 1608671 (0.0007) [2023-12-27 16:47:27,771][20347] Updated weights for policy 0, policy_version 1608681 (0.0007) [2023-12-27 16:47:27,909][20381] Updated weights for policy 1, policy_version 1607099 (0.0008) [2023-12-27 16:47:27,955][20381] Updated weights for policy 1, policy_version 1607109 (0.0009) [2023-12-27 16:47:28,004][20381] Updated weights for policy 1, policy_version 1607119 (0.0008) [2023-12-27 16:47:28,389][20347] Updated weights for policy 0, policy_version 1608691 (0.0007) [2023-12-27 16:47:28,449][20347] Updated weights for policy 0, policy_version 1608701 (0.0007) [2023-12-27 16:47:28,501][20347] Updated weights for policy 0, policy_version 1608711 (0.0007) [2023-12-27 16:47:28,735][20381] Updated weights for policy 1, policy_version 1607129 (0.0009) [2023-12-27 16:47:28,795][20381] Updated weights for policy 1, policy_version 1607139 (0.0008) [2023-12-27 16:47:28,847][20381] Updated weights for policy 1, policy_version 1607149 (0.0009) [2023-12-27 16:47:28,896][20381] Updated weights for policy 1, policy_version 1607159 (0.0008) [2023-12-27 16:47:29,128][20347] Updated weights for policy 0, policy_version 1608721 (0.0008) [2023-12-27 16:47:29,180][20347] Updated weights for policy 0, policy_version 1608731 (0.0008) [2023-12-27 16:47:29,248][20347] Updated weights for policy 0, policy_version 1608741 (0.0008) [2023-12-27 16:47:29,310][20347] Updated weights for policy 0, policy_version 1608751 (0.0008) [2023-12-27 16:47:29,622][20381] Updated weights for policy 1, policy_version 1607169 (0.0008) [2023-12-27 16:47:29,680][20381] Updated weights for policy 1, policy_version 1607179 (0.0008) [2023-12-27 16:47:29,739][20381] Updated weights for policy 1, policy_version 1607189 (0.0008) [2023-12-27 16:47:29,995][20347] Updated weights for policy 0, policy_version 1608761 (0.0007) [2023-12-27 16:47:30,057][20347] Updated weights for policy 0, policy_version 1608771 (0.0008) [2023-12-27 16:47:30,113][20347] Updated weights for policy 0, policy_version 1608781 (0.0008) [2023-12-27 16:47:30,466][20381] Updated weights for policy 1, policy_version 1607199 (0.0008) [2023-12-27 16:47:30,518][20381] Updated weights for policy 1, policy_version 1607209 (0.0007) [2023-12-27 16:47:30,579][20381] Updated weights for policy 1, policy_version 1607219 (0.0007) [2023-12-27 16:47:30,775][20347] Updated weights for policy 0, policy_version 1608791 (0.0006) [2023-12-27 16:47:30,818][20347] Updated weights for policy 0, policy_version 1608801 (0.0006) [2023-12-27 16:47:30,867][20347] Updated weights for policy 0, policy_version 1608811 (0.0007) [2023-12-27 16:47:31,323][20381] Updated weights for policy 1, policy_version 1607229 (0.0007) [2023-12-27 16:47:31,394][20381] Updated weights for policy 1, policy_version 1607239 (0.0008) [2023-12-27 16:47:31,446][20381] Updated weights for policy 1, policy_version 1607249 (0.0007) [2023-12-27 16:47:31,616][20347] Updated weights for policy 0, policy_version 1608821 (0.0008) [2023-12-27 16:47:31,683][20347] Updated weights for policy 0, policy_version 1608831 (0.0008) [2023-12-27 16:47:31,737][20347] Updated weights for policy 0, policy_version 1608841 (0.0006) [2023-12-27 16:47:32,190][20381] Updated weights for policy 1, policy_version 1607259 (0.0008) [2023-12-27 16:47:32,248][20381] Updated weights for policy 1, policy_version 1607269 (0.0008) [2023-12-27 16:47:32,304][20381] Updated weights for policy 1, policy_version 1607279 (0.0008) [2023-12-27 16:47:32,345][20347] Updated weights for policy 0, policy_version 1608851 (0.0007) [2023-12-27 16:47:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 823451648. Throughput: 0: 10099.4, 1: 10260.8. Samples: 823460264. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.720')] [2023-12-27 16:47:32,407][20347] Updated weights for policy 0, policy_version 1608861 (0.0008) [2023-12-27 16:47:32,456][20347] Updated weights for policy 0, policy_version 1608871 (0.0006) [2023-12-27 16:47:32,927][20381] Updated weights for policy 1, policy_version 1607289 (0.0007) [2023-12-27 16:47:32,979][20381] Updated weights for policy 1, policy_version 1607299 (0.0010) [2023-12-27 16:47:33,039][20381] Updated weights for policy 1, policy_version 1607309 (0.0007) [2023-12-27 16:47:33,099][20381] Updated weights for policy 1, policy_version 1607319 (0.0010) [2023-12-27 16:47:33,218][20347] Updated weights for policy 0, policy_version 1608881 (0.0010) [2023-12-27 16:47:33,274][20347] Updated weights for policy 0, policy_version 1608891 (0.0008) [2023-12-27 16:47:33,324][20347] Updated weights for policy 0, policy_version 1608901 (0.0008) [2023-12-27 16:47:33,373][20347] Updated weights for policy 0, policy_version 1608911 (0.0008) [2023-12-27 16:47:33,804][20381] Updated weights for policy 1, policy_version 1607329 (0.0006) [2023-12-27 16:47:33,867][20381] Updated weights for policy 1, policy_version 1607339 (0.0005) [2023-12-27 16:47:33,922][20381] Updated weights for policy 1, policy_version 1607349 (0.0005) [2023-12-27 16:47:33,996][20347] Updated weights for policy 0, policy_version 1608921 (0.0008) [2023-12-27 16:47:34,044][20347] Updated weights for policy 0, policy_version 1608931 (0.0007) [2023-12-27 16:47:34,092][20347] Updated weights for policy 0, policy_version 1608941 (0.0007) [2023-12-27 16:47:34,548][20381] Updated weights for policy 1, policy_version 1607359 (0.0008) [2023-12-27 16:47:34,608][20381] Updated weights for policy 1, policy_version 1607369 (0.0009) [2023-12-27 16:47:34,669][20381] Updated weights for policy 1, policy_version 1607379 (0.0008) [2023-12-27 16:47:34,770][20347] Updated weights for policy 0, policy_version 1608951 (0.0006) [2023-12-27 16:47:34,830][20347] Updated weights for policy 0, policy_version 1608961 (0.0010) [2023-12-27 16:47:34,889][20347] Updated weights for policy 0, policy_version 1608971 (0.0010) [2023-12-27 16:47:35,290][20381] Updated weights for policy 1, policy_version 1607389 (0.0007) [2023-12-27 16:47:35,342][20381] Updated weights for policy 1, policy_version 1607399 (0.0005) [2023-12-27 16:47:35,393][20381] Updated weights for policy 1, policy_version 1607409 (0.0005) [2023-12-27 16:47:35,474][20347] Updated weights for policy 0, policy_version 1608981 (0.0009) [2023-12-27 16:47:35,524][20347] Updated weights for policy 0, policy_version 1608991 (0.0006) [2023-12-27 16:47:35,582][20347] Updated weights for policy 0, policy_version 1609001 (0.0007) [2023-12-27 16:47:36,050][20381] Updated weights for policy 1, policy_version 1607419 (0.0006) [2023-12-27 16:47:36,106][20381] Updated weights for policy 1, policy_version 1607429 (0.0007) [2023-12-27 16:47:36,174][20381] Updated weights for policy 1, policy_version 1607439 (0.0009) [2023-12-27 16:47:36,274][20347] Updated weights for policy 0, policy_version 1609011 (0.0006) [2023-12-27 16:47:36,334][20347] Updated weights for policy 0, policy_version 1609021 (0.0006) [2023-12-27 16:47:36,394][20347] Updated weights for policy 0, policy_version 1609031 (0.0005) [2023-12-27 16:47:36,923][20381] Updated weights for policy 1, policy_version 1607449 (0.0007) [2023-12-27 16:47:36,928][20347] Updated weights for policy 0, policy_version 1609041 (0.0007) [2023-12-27 16:47:36,980][20381] Updated weights for policy 1, policy_version 1607459 (0.0007) [2023-12-27 16:47:36,985][20347] Updated weights for policy 0, policy_version 1609051 (0.0007) [2023-12-27 16:47:37,035][20347] Updated weights for policy 0, policy_version 1609061 (0.0005) [2023-12-27 16:47:37,041][20381] Updated weights for policy 1, policy_version 1607469 (0.0006) [2023-12-27 16:47:37,086][20347] Updated weights for policy 0, policy_version 1609071 (0.0006) [2023-12-27 16:47:37,100][20381] Updated weights for policy 1, policy_version 1607479 (0.0006) [2023-12-27 16:47:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 823558144. Throughput: 0: 10114.7, 1: 10332.8. Samples: 823523712. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.100')] [2023-12-27 16:47:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001607480_411557888.pth... [2023-12-27 16:47:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001609072_412000256.pth... [2023-12-27 16:47:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001607888_411697152.pth [2023-12-27 16:47:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001606296_411254784.pth [2023-12-27 16:47:37,649][20347] Updated weights for policy 0, policy_version 1609081 (0.0007) [2023-12-27 16:47:37,713][20347] Updated weights for policy 0, policy_version 1609091 (0.0006) [2023-12-27 16:47:37,745][20381] Updated weights for policy 1, policy_version 1607489 (0.0010) [2023-12-27 16:47:37,766][20347] Updated weights for policy 0, policy_version 1609101 (0.0005) [2023-12-27 16:47:37,790][20381] Updated weights for policy 1, policy_version 1607499 (0.0010) [2023-12-27 16:47:37,840][20381] Updated weights for policy 1, policy_version 1607509 (0.0006) [2023-12-27 16:47:38,371][20347] Updated weights for policy 0, policy_version 1609111 (0.0007) [2023-12-27 16:47:38,433][20347] Updated weights for policy 0, policy_version 1609121 (0.0007) [2023-12-27 16:47:38,477][20381] Updated weights for policy 1, policy_version 1607519 (0.0009) [2023-12-27 16:47:38,486][20347] Updated weights for policy 0, policy_version 1609131 (0.0007) [2023-12-27 16:47:38,536][20381] Updated weights for policy 1, policy_version 1607529 (0.0011) [2023-12-27 16:47:38,593][20381] Updated weights for policy 1, policy_version 1607539 (0.0010) [2023-12-27 16:47:39,207][20347] Updated weights for policy 0, policy_version 1609141 (0.0007) [2023-12-27 16:47:39,222][20381] Updated weights for policy 1, policy_version 1607549 (0.0006) [2023-12-27 16:47:39,273][20347] Updated weights for policy 0, policy_version 1609151 (0.0009) [2023-12-27 16:47:39,286][20381] Updated weights for policy 1, policy_version 1607559 (0.0010) [2023-12-27 16:47:39,339][20347] Updated weights for policy 0, policy_version 1609161 (0.0008) [2023-12-27 16:47:39,354][20381] Updated weights for policy 1, policy_version 1607569 (0.0008) [2023-12-27 16:47:40,088][20347] Updated weights for policy 0, policy_version 1609171 (0.0008) [2023-12-27 16:47:40,093][20381] Updated weights for policy 1, policy_version 1607579 (0.0009) [2023-12-27 16:47:40,146][20347] Updated weights for policy 0, policy_version 1609181 (0.0007) [2023-12-27 16:47:40,153][20381] Updated weights for policy 1, policy_version 1607589 (0.0008) [2023-12-27 16:47:40,209][20347] Updated weights for policy 0, policy_version 1609191 (0.0007) [2023-12-27 16:47:40,216][20381] Updated weights for policy 1, policy_version 1607599 (0.0009) [2023-12-27 16:47:40,882][20381] Updated weights for policy 1, policy_version 1607609 (0.0006) [2023-12-27 16:47:40,904][20347] Updated weights for policy 0, policy_version 1609201 (0.0008) [2023-12-27 16:47:40,934][20381] Updated weights for policy 1, policy_version 1607619 (0.0006) [2023-12-27 16:47:40,960][20347] Updated weights for policy 0, policy_version 1609211 (0.0007) [2023-12-27 16:47:40,992][20381] Updated weights for policy 1, policy_version 1607629 (0.0006) [2023-12-27 16:47:41,015][20347] Updated weights for policy 0, policy_version 1609221 (0.0007) [2023-12-27 16:47:41,054][20381] Updated weights for policy 1, policy_version 1607639 (0.0008) [2023-12-27 16:47:41,078][20347] Updated weights for policy 0, policy_version 1609231 (0.0006) [2023-12-27 16:47:41,797][20381] Updated weights for policy 1, policy_version 1607649 (0.0007) [2023-12-27 16:47:41,843][20347] Updated weights for policy 0, policy_version 1609241 (0.0009) [2023-12-27 16:47:41,849][20381] Updated weights for policy 1, policy_version 1607659 (0.0006) [2023-12-27 16:47:41,901][20347] Updated weights for policy 0, policy_version 1609251 (0.0007) [2023-12-27 16:47:41,902][20381] Updated weights for policy 1, policy_version 1607669 (0.0006) [2023-12-27 16:47:41,962][20347] Updated weights for policy 0, policy_version 1609261 (0.0007) [2023-12-27 16:47:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 823656448. Throughput: 0: 10167.5, 1: 10289.8. Samples: 823648320. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.450')] [2023-12-27 16:47:42,602][20381] Updated weights for policy 1, policy_version 1607679 (0.0008) [2023-12-27 16:47:42,657][20347] Updated weights for policy 0, policy_version 1609271 (0.0010) [2023-12-27 16:47:42,661][20381] Updated weights for policy 1, policy_version 1607689 (0.0007) [2023-12-27 16:47:42,710][20347] Updated weights for policy 0, policy_version 1609281 (0.0010) [2023-12-27 16:47:42,713][20381] Updated weights for policy 1, policy_version 1607699 (0.0007) [2023-12-27 16:47:42,762][20347] Updated weights for policy 0, policy_version 1609291 (0.0010) [2023-12-27 16:47:43,404][20381] Updated weights for policy 1, policy_version 1607709 (0.0007) [2023-12-27 16:47:43,455][20347] Updated weights for policy 0, policy_version 1609301 (0.0010) [2023-12-27 16:47:43,457][20381] Updated weights for policy 1, policy_version 1607719 (0.0006) [2023-12-27 16:47:43,507][20347] Updated weights for policy 0, policy_version 1609311 (0.0010) [2023-12-27 16:47:43,516][20381] Updated weights for policy 1, policy_version 1607729 (0.0005) [2023-12-27 16:47:43,562][20347] Updated weights for policy 0, policy_version 1609321 (0.0010) [2023-12-27 16:47:44,161][20347] Updated weights for policy 0, policy_version 1609331 (0.0006) [2023-12-27 16:47:44,187][20381] Updated weights for policy 1, policy_version 1607739 (0.0005) [2023-12-27 16:47:44,215][20347] Updated weights for policy 0, policy_version 1609341 (0.0007) [2023-12-27 16:47:44,244][20381] Updated weights for policy 1, policy_version 1607749 (0.0005) [2023-12-27 16:47:44,264][20347] Updated weights for policy 0, policy_version 1609351 (0.0006) [2023-12-27 16:47:44,298][20381] Updated weights for policy 1, policy_version 1607759 (0.0007) [2023-12-27 16:47:44,965][20347] Updated weights for policy 0, policy_version 1609361 (0.0008) [2023-12-27 16:47:44,993][20381] Updated weights for policy 1, policy_version 1607769 (0.0008) [2023-12-27 16:47:45,031][20347] Updated weights for policy 0, policy_version 1609371 (0.0011) [2023-12-27 16:47:45,044][20381] Updated weights for policy 1, policy_version 1607779 (0.0006) [2023-12-27 16:47:45,080][20347] Updated weights for policy 0, policy_version 1609381 (0.0011) [2023-12-27 16:47:45,100][20381] Updated weights for policy 1, policy_version 1607789 (0.0005) [2023-12-27 16:47:45,140][20347] Updated weights for policy 0, policy_version 1609391 (0.0011) [2023-12-27 16:47:45,150][20381] Updated weights for policy 1, policy_version 1607799 (0.0006) [2023-12-27 16:47:45,835][20347] Updated weights for policy 0, policy_version 1609401 (0.0010) [2023-12-27 16:47:45,868][20381] Updated weights for policy 1, policy_version 1607809 (0.0009) [2023-12-27 16:47:45,890][20347] Updated weights for policy 0, policy_version 1609411 (0.0010) [2023-12-27 16:47:45,922][20381] Updated weights for policy 1, policy_version 1607819 (0.0010) [2023-12-27 16:47:45,945][20347] Updated weights for policy 0, policy_version 1609421 (0.0010) [2023-12-27 16:47:45,977][20381] Updated weights for policy 1, policy_version 1607829 (0.0006) [2023-12-27 16:47:46,626][20347] Updated weights for policy 0, policy_version 1609431 (0.0011) [2023-12-27 16:47:46,673][20381] Updated weights for policy 1, policy_version 1607839 (0.0008) [2023-12-27 16:47:46,681][20347] Updated weights for policy 0, policy_version 1609441 (0.0010) [2023-12-27 16:47:46,722][20381] Updated weights for policy 1, policy_version 1607849 (0.0008) [2023-12-27 16:47:46,731][20347] Updated weights for policy 0, policy_version 1609451 (0.0010) [2023-12-27 16:47:46,774][20381] Updated weights for policy 1, policy_version 1607859 (0.0007) [2023-12-27 16:47:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 823754752. Throughput: 0: 10193.9, 1: 10239.8. Samples: 823768468. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.680')] [2023-12-27 16:47:47,423][20347] Updated weights for policy 0, policy_version 1609461 (0.0010) [2023-12-27 16:47:47,476][20347] Updated weights for policy 0, policy_version 1609471 (0.0011) [2023-12-27 16:47:47,485][20381] Updated weights for policy 1, policy_version 1607869 (0.0007) [2023-12-27 16:47:47,521][20347] Updated weights for policy 0, policy_version 1609481 (0.0010) [2023-12-27 16:47:47,538][20381] Updated weights for policy 1, policy_version 1607879 (0.0005) [2023-12-27 16:47:47,584][20381] Updated weights for policy 1, policy_version 1607889 (0.0006) [2023-12-27 16:47:48,232][20347] Updated weights for policy 0, policy_version 1609491 (0.0010) [2023-12-27 16:47:48,302][20347] Updated weights for policy 0, policy_version 1609501 (0.0008) [2023-12-27 16:47:48,308][20381] Updated weights for policy 1, policy_version 1607899 (0.0007) [2023-12-27 16:47:48,367][20347] Updated weights for policy 0, policy_version 1609511 (0.0010) [2023-12-27 16:47:48,368][20381] Updated weights for policy 1, policy_version 1607909 (0.0007) [2023-12-27 16:47:48,425][20381] Updated weights for policy 1, policy_version 1607919 (0.0006) [2023-12-27 16:47:49,050][20347] Updated weights for policy 0, policy_version 1609521 (0.0010) [2023-12-27 16:47:49,074][20381] Updated weights for policy 1, policy_version 1607929 (0.0006) [2023-12-27 16:47:49,108][20347] Updated weights for policy 0, policy_version 1609531 (0.0010) [2023-12-27 16:47:49,133][20381] Updated weights for policy 1, policy_version 1607939 (0.0011) [2023-12-27 16:47:49,158][20347] Updated weights for policy 0, policy_version 1609541 (0.0005) [2023-12-27 16:47:49,182][20381] Updated weights for policy 1, policy_version 1607949 (0.0010) [2023-12-27 16:47:49,208][20347] Updated weights for policy 0, policy_version 1609551 (0.0010) [2023-12-27 16:47:49,243][20381] Updated weights for policy 1, policy_version 1607959 (0.0009) [2023-12-27 16:47:49,890][20347] Updated weights for policy 0, policy_version 1609561 (0.0008) [2023-12-27 16:47:49,934][20381] Updated weights for policy 1, policy_version 1607969 (0.0010) [2023-12-27 16:47:49,948][20347] Updated weights for policy 0, policy_version 1609571 (0.0007) [2023-12-27 16:47:50,000][20381] Updated weights for policy 1, policy_version 1607979 (0.0011) [2023-12-27 16:47:50,010][20347] Updated weights for policy 0, policy_version 1609581 (0.0006) [2023-12-27 16:47:50,063][20381] Updated weights for policy 1, policy_version 1607989 (0.0011) [2023-12-27 16:47:50,698][20347] Updated weights for policy 0, policy_version 1609591 (0.0007) [2023-12-27 16:47:50,736][20381] Updated weights for policy 1, policy_version 1607999 (0.0008) [2023-12-27 16:47:50,756][20347] Updated weights for policy 0, policy_version 1609601 (0.0008) [2023-12-27 16:47:50,796][20381] Updated weights for policy 1, policy_version 1608009 (0.0006) [2023-12-27 16:47:50,812][20347] Updated weights for policy 0, policy_version 1609611 (0.0006) [2023-12-27 16:47:50,860][20381] Updated weights for policy 1, policy_version 1608019 (0.0009) [2023-12-27 16:47:51,512][20347] Updated weights for policy 0, policy_version 1609621 (0.0007) [2023-12-27 16:47:51,542][20381] Updated weights for policy 1, policy_version 1608029 (0.0007) [2023-12-27 16:47:51,566][20347] Updated weights for policy 0, policy_version 1609631 (0.0006) [2023-12-27 16:47:51,609][20381] Updated weights for policy 1, policy_version 1608039 (0.0007) [2023-12-27 16:47:51,623][20347] Updated weights for policy 0, policy_version 1609641 (0.0007) [2023-12-27 16:47:51,670][20381] Updated weights for policy 1, policy_version 1608049 (0.0007) [2023-12-27 16:47:52,225][20347] Updated weights for policy 0, policy_version 1609651 (0.0006) [2023-12-27 16:47:52,293][20347] Updated weights for policy 0, policy_version 1609661 (0.0008) [2023-12-27 16:47:52,349][20347] Updated weights for policy 0, policy_version 1609671 (0.0007) [2023-12-27 16:47:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 823853056. Throughput: 0: 10189.3, 1: 10186.3. Samples: 823828476. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '834.360')] [2023-12-27 16:47:52,400][20381] Updated weights for policy 1, policy_version 1608059 (0.0009) [2023-12-27 16:47:52,410][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001609680_412155904.pth... [2023-12-27 16:47:52,413][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001608464_411844608.pth [2023-12-27 16:47:52,452][20381] Updated weights for policy 1, policy_version 1608069 (0.0006) [2023-12-27 16:47:52,506][20381] Updated weights for policy 1, policy_version 1608079 (0.0008) [2023-12-27 16:47:52,550][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001608088_411713536.pth... [2023-12-27 16:47:52,554][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001606904_411410432.pth [2023-12-27 16:47:53,037][20347] Updated weights for policy 0, policy_version 1609681 (0.0008) [2023-12-27 16:47:53,094][20347] Updated weights for policy 0, policy_version 1609691 (0.0005) [2023-12-27 16:47:53,153][20347] Updated weights for policy 0, policy_version 1609701 (0.0005) [2023-12-27 16:47:53,211][20347] Updated weights for policy 0, policy_version 1609711 (0.0005) [2023-12-27 16:47:53,215][20381] Updated weights for policy 1, policy_version 1608089 (0.0008) [2023-12-27 16:47:53,269][20381] Updated weights for policy 1, policy_version 1608099 (0.0008) [2023-12-27 16:47:53,333][20381] Updated weights for policy 1, policy_version 1608109 (0.0006) [2023-12-27 16:47:53,398][20381] Updated weights for policy 1, policy_version 1608119 (0.0007) [2023-12-27 16:47:53,794][20347] Updated weights for policy 0, policy_version 1609721 (0.0005) [2023-12-27 16:47:53,847][20347] Updated weights for policy 0, policy_version 1609731 (0.0007) [2023-12-27 16:47:53,894][20347] Updated weights for policy 0, policy_version 1609741 (0.0005) [2023-12-27 16:47:54,095][20381] Updated weights for policy 1, policy_version 1608129 (0.0009) [2023-12-27 16:47:54,159][20381] Updated weights for policy 1, policy_version 1608139 (0.0008) [2023-12-27 16:47:54,227][20381] Updated weights for policy 1, policy_version 1608149 (0.0008) [2023-12-27 16:47:54,512][20347] Updated weights for policy 0, policy_version 1609751 (0.0008) [2023-12-27 16:47:54,574][20347] Updated weights for policy 0, policy_version 1609761 (0.0008) [2023-12-27 16:47:54,633][20347] Updated weights for policy 0, policy_version 1609771 (0.0007) [2023-12-27 16:47:54,838][20381] Updated weights for policy 1, policy_version 1608159 (0.0007) [2023-12-27 16:47:54,897][20381] Updated weights for policy 1, policy_version 1608169 (0.0005) [2023-12-27 16:47:54,962][20381] Updated weights for policy 1, policy_version 1608179 (0.0007) [2023-12-27 16:47:55,292][20347] Updated weights for policy 0, policy_version 1609781 (0.0008) [2023-12-27 16:47:55,343][20347] Updated weights for policy 0, policy_version 1609791 (0.0008) [2023-12-27 16:47:55,394][20347] Updated weights for policy 0, policy_version 1609801 (0.0007) [2023-12-27 16:47:55,601][20381] Updated weights for policy 1, policy_version 1608189 (0.0007) [2023-12-27 16:47:55,667][20381] Updated weights for policy 1, policy_version 1608199 (0.0005) [2023-12-27 16:47:55,732][20381] Updated weights for policy 1, policy_version 1608209 (0.0005) [2023-12-27 16:47:56,033][20347] Updated weights for policy 0, policy_version 1609811 (0.0010) [2023-12-27 16:47:56,085][20347] Updated weights for policy 0, policy_version 1609821 (0.0010) [2023-12-27 16:47:56,134][20347] Updated weights for policy 0, policy_version 1609831 (0.0010) [2023-12-27 16:47:56,294][20381] Updated weights for policy 1, policy_version 1608219 (0.0006) [2023-12-27 16:47:56,353][20381] Updated weights for policy 1, policy_version 1608229 (0.0009) [2023-12-27 16:47:56,413][20381] Updated weights for policy 1, policy_version 1608239 (0.0010) [2023-12-27 16:47:56,738][20347] Updated weights for policy 0, policy_version 1609841 (0.0010) [2023-12-27 16:47:56,793][20347] Updated weights for policy 0, policy_version 1609851 (0.0005) [2023-12-27 16:47:56,867][20347] Updated weights for policy 0, policy_version 1609861 (0.0005) [2023-12-27 16:47:56,910][20347] Updated weights for policy 0, policy_version 1609871 (0.0005) [2023-12-27 16:47:57,033][20381] Updated weights for policy 1, policy_version 1608249 (0.0010) [2023-12-27 16:47:57,093][20381] Updated weights for policy 1, policy_version 1608259 (0.0005) [2023-12-27 16:47:57,147][20381] Updated weights for policy 1, policy_version 1608269 (0.0005) [2023-12-27 16:47:57,201][20381] Updated weights for policy 1, policy_version 1608279 (0.0005) [2023-12-27 16:47:57,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 823967744. Throughput: 0: 10293.4, 1: 10139.1. Samples: 823954540. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:47:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '902.750')] [2023-12-27 16:47:57,501][20347] Updated weights for policy 0, policy_version 1609881 (0.0007) [2023-12-27 16:47:57,557][20347] Updated weights for policy 0, policy_version 1609891 (0.0008) [2023-12-27 16:47:57,613][20347] Updated weights for policy 0, policy_version 1609901 (0.0008) [2023-12-27 16:47:57,792][20381] Updated weights for policy 1, policy_version 1608289 (0.0007) [2023-12-27 16:47:57,836][20381] Updated weights for policy 1, policy_version 1608299 (0.0008) [2023-12-27 16:47:57,885][20381] Updated weights for policy 1, policy_version 1608309 (0.0007) [2023-12-27 16:47:58,307][20347] Updated weights for policy 0, policy_version 1609911 (0.0008) [2023-12-27 16:47:58,377][20347] Updated weights for policy 0, policy_version 1609921 (0.0008) [2023-12-27 16:47:58,435][20347] Updated weights for policy 0, policy_version 1609931 (0.0007) [2023-12-27 16:47:58,600][20381] Updated weights for policy 1, policy_version 1608319 (0.0007) [2023-12-27 16:47:58,662][20381] Updated weights for policy 1, policy_version 1608329 (0.0008) [2023-12-27 16:47:58,725][20381] Updated weights for policy 1, policy_version 1608339 (0.0011) [2023-12-27 16:47:59,212][20347] Updated weights for policy 0, policy_version 1609941 (0.0006) [2023-12-27 16:47:59,293][20347] Updated weights for policy 0, policy_version 1609951 (0.0007) [2023-12-27 16:47:59,363][20347] Updated weights for policy 0, policy_version 1609961 (0.0008) [2023-12-27 16:47:59,503][20381] Updated weights for policy 1, policy_version 1608349 (0.0008) [2023-12-27 16:47:59,570][20381] Updated weights for policy 1, policy_version 1608359 (0.0007) [2023-12-27 16:47:59,628][20381] Updated weights for policy 1, policy_version 1608369 (0.0009) [2023-12-27 16:48:00,028][20347] Updated weights for policy 0, policy_version 1609971 (0.0009) [2023-12-27 16:48:00,075][20347] Updated weights for policy 0, policy_version 1609981 (0.0008) [2023-12-27 16:48:00,129][20347] Updated weights for policy 0, policy_version 1609991 (0.0008) [2023-12-27 16:48:00,265][20381] Updated weights for policy 1, policy_version 1608379 (0.0008) [2023-12-27 16:48:00,318][20381] Updated weights for policy 1, policy_version 1608389 (0.0007) [2023-12-27 16:48:00,381][20381] Updated weights for policy 1, policy_version 1608399 (0.0007) [2023-12-27 16:48:00,829][20347] Updated weights for policy 0, policy_version 1610001 (0.0008) [2023-12-27 16:48:00,885][20347] Updated weights for policy 0, policy_version 1610011 (0.0007) [2023-12-27 16:48:00,935][20347] Updated weights for policy 0, policy_version 1610021 (0.0007) [2023-12-27 16:48:00,985][20347] Updated weights for policy 0, policy_version 1610031 (0.0007) [2023-12-27 16:48:01,018][20381] Updated weights for policy 1, policy_version 1608409 (0.0007) [2023-12-27 16:48:01,068][20381] Updated weights for policy 1, policy_version 1608419 (0.0007) [2023-12-27 16:48:01,123][20381] Updated weights for policy 1, policy_version 1608429 (0.0007) [2023-12-27 16:48:01,185][20381] Updated weights for policy 1, policy_version 1608439 (0.0008) [2023-12-27 16:48:01,669][20347] Updated weights for policy 0, policy_version 1610041 (0.0007) [2023-12-27 16:48:01,743][20347] Updated weights for policy 0, policy_version 1610051 (0.0007) [2023-12-27 16:48:01,806][20347] Updated weights for policy 0, policy_version 1610061 (0.0005) [2023-12-27 16:48:01,964][20381] Updated weights for policy 1, policy_version 1608449 (0.0009) [2023-12-27 16:48:02,018][20381] Updated weights for policy 1, policy_version 1608459 (0.0007) [2023-12-27 16:48:02,077][20381] Updated weights for policy 1, policy_version 1608469 (0.0008) [2023-12-27 16:48:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 824066048. Throughput: 0: 10346.3, 1: 10062.1. Samples: 824076892. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:48:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.150')] [2023-12-27 16:48:02,468][20347] Updated weights for policy 0, policy_version 1610071 (0.0006) [2023-12-27 16:48:02,532][20347] Updated weights for policy 0, policy_version 1610081 (0.0006) [2023-12-27 16:48:02,600][20347] Updated weights for policy 0, policy_version 1610091 (0.0007) [2023-12-27 16:48:02,784][20381] Updated weights for policy 1, policy_version 1608479 (0.0008) [2023-12-27 16:48:02,832][20381] Updated weights for policy 1, policy_version 1608489 (0.0007) [2023-12-27 16:48:02,887][20381] Updated weights for policy 1, policy_version 1608499 (0.0008) [2023-12-27 16:48:03,230][20347] Updated weights for policy 0, policy_version 1610101 (0.0007) [2023-12-27 16:48:03,274][20347] Updated weights for policy 0, policy_version 1610111 (0.0007) [2023-12-27 16:48:03,332][20347] Updated weights for policy 0, policy_version 1610121 (0.0007) [2023-12-27 16:48:03,580][20381] Updated weights for policy 1, policy_version 1608509 (0.0007) [2023-12-27 16:48:03,636][20381] Updated weights for policy 1, policy_version 1608519 (0.0011) [2023-12-27 16:48:03,696][20381] Updated weights for policy 1, policy_version 1608529 (0.0009) [2023-12-27 16:48:04,023][20347] Updated weights for policy 0, policy_version 1610131 (0.0007) [2023-12-27 16:48:04,083][20347] Updated weights for policy 0, policy_version 1610141 (0.0008) [2023-12-27 16:48:04,147][20347] Updated weights for policy 0, policy_version 1610151 (0.0008) [2023-12-27 16:48:04,402][20381] Updated weights for policy 1, policy_version 1608539 (0.0006) [2023-12-27 16:48:04,462][20381] Updated weights for policy 1, policy_version 1608549 (0.0009) [2023-12-27 16:48:04,508][20381] Updated weights for policy 1, policy_version 1608559 (0.0009) [2023-12-27 16:48:04,828][20347] Updated weights for policy 0, policy_version 1610161 (0.0010) [2023-12-27 16:48:04,890][20347] Updated weights for policy 0, policy_version 1610171 (0.0005) [2023-12-27 16:48:04,953][20347] Updated weights for policy 0, policy_version 1610181 (0.0005) [2023-12-27 16:48:05,014][20347] Updated weights for policy 0, policy_version 1610191 (0.0005) [2023-12-27 16:48:05,167][20381] Updated weights for policy 1, policy_version 1608569 (0.0009) [2023-12-27 16:48:05,213][20381] Updated weights for policy 1, policy_version 1608579 (0.0010) [2023-12-27 16:48:05,262][20381] Updated weights for policy 1, policy_version 1608589 (0.0008) [2023-12-27 16:48:05,310][20381] Updated weights for policy 1, policy_version 1608599 (0.0007) [2023-12-27 16:48:05,573][20347] Updated weights for policy 0, policy_version 1610201 (0.0005) [2023-12-27 16:48:05,628][20347] Updated weights for policy 0, policy_version 1610211 (0.0007) [2023-12-27 16:48:05,693][20347] Updated weights for policy 0, policy_version 1610221 (0.0008) [2023-12-27 16:48:05,959][20381] Updated weights for policy 1, policy_version 1608609 (0.0008) [2023-12-27 16:48:06,014][20381] Updated weights for policy 1, policy_version 1608619 (0.0008) [2023-12-27 16:48:06,065][20381] Updated weights for policy 1, policy_version 1608629 (0.0010) [2023-12-27 16:48:06,343][20347] Updated weights for policy 0, policy_version 1610231 (0.0009) [2023-12-27 16:48:06,405][20347] Updated weights for policy 0, policy_version 1610241 (0.0007) [2023-12-27 16:48:06,469][20347] Updated weights for policy 0, policy_version 1610251 (0.0009) [2023-12-27 16:48:06,709][20381] Updated weights for policy 1, policy_version 1608639 (0.0010) [2023-12-27 16:48:06,726][20292] KL-divergence is very high: 117.7616 [2023-12-27 16:48:06,768][20381] Updated weights for policy 1, policy_version 1608649 (0.0010) [2023-12-27 16:48:06,773][20292] KL-divergence is very high: 157.2537 [2023-12-27 16:48:06,826][20381] Updated weights for policy 1, policy_version 1608659 (0.0010) [2023-12-27 16:48:07,175][20347] Updated weights for policy 0, policy_version 1610261 (0.0009) [2023-12-27 16:48:07,228][20347] Updated weights for policy 0, policy_version 1610271 (0.0007) [2023-12-27 16:48:07,278][20347] Updated weights for policy 0, policy_version 1610281 (0.0007) [2023-12-27 16:48:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 824172544. Throughput: 0: 10363.8, 1: 10097.4. Samples: 824138008. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:48:07,363][19170] Avg episode reward: [(0, '990.270'), (1, '951.130')] [2023-12-27 16:48:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001608664_411860992.pth... [2023-12-27 16:48:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001610288_412311552.pth... [2023-12-27 16:48:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001609072_412000256.pth [2023-12-27 16:48:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001607480_411557888.pth [2023-12-27 16:48:07,529][20381] Updated weights for policy 1, policy_version 1608669 (0.0010) [2023-12-27 16:48:07,580][20381] Updated weights for policy 1, policy_version 1608679 (0.0010) [2023-12-27 16:48:07,635][20381] Updated weights for policy 1, policy_version 1608689 (0.0010) [2023-12-27 16:48:07,939][20347] Updated weights for policy 0, policy_version 1610291 (0.0007) [2023-12-27 16:48:07,997][20347] Updated weights for policy 0, policy_version 1610301 (0.0006) [2023-12-27 16:48:08,050][20347] Updated weights for policy 0, policy_version 1610311 (0.0010) [2023-12-27 16:48:08,298][20381] Updated weights for policy 1, policy_version 1608699 (0.0009) [2023-12-27 16:48:08,365][20381] Updated weights for policy 1, policy_version 1608709 (0.0007) [2023-12-27 16:48:08,421][20381] Updated weights for policy 1, policy_version 1608719 (0.0006) [2023-12-27 16:48:08,734][20347] Updated weights for policy 0, policy_version 1610321 (0.0010) [2023-12-27 16:48:08,790][20347] Updated weights for policy 0, policy_version 1610331 (0.0007) [2023-12-27 16:48:08,840][20347] Updated weights for policy 0, policy_version 1610341 (0.0008) [2023-12-27 16:48:08,890][20347] Updated weights for policy 0, policy_version 1610351 (0.0007) [2023-12-27 16:48:09,048][20381] Updated weights for policy 1, policy_version 1608729 (0.0006) [2023-12-27 16:48:09,111][20381] Updated weights for policy 1, policy_version 1608739 (0.0008) [2023-12-27 16:48:09,174][20381] Updated weights for policy 1, policy_version 1608749 (0.0007) [2023-12-27 16:48:09,238][20381] Updated weights for policy 1, policy_version 1608759 (0.0008) [2023-12-27 16:48:09,604][20347] Updated weights for policy 0, policy_version 1610361 (0.0010) [2023-12-27 16:48:09,661][20347] Updated weights for policy 0, policy_version 1610371 (0.0011) [2023-12-27 16:48:09,718][20347] Updated weights for policy 0, policy_version 1610381 (0.0010) [2023-12-27 16:48:09,995][20381] Updated weights for policy 1, policy_version 1608769 (0.0008) [2023-12-27 16:48:10,057][20381] Updated weights for policy 1, policy_version 1608779 (0.0008) [2023-12-27 16:48:10,117][20381] Updated weights for policy 1, policy_version 1608789 (0.0008) [2023-12-27 16:48:10,462][20347] Updated weights for policy 0, policy_version 1610391 (0.0007) [2023-12-27 16:48:10,526][20347] Updated weights for policy 0, policy_version 1610401 (0.0006) [2023-12-27 16:48:10,590][20347] Updated weights for policy 0, policy_version 1610411 (0.0006) [2023-12-27 16:48:10,763][20381] Updated weights for policy 1, policy_version 1608799 (0.0006) [2023-12-27 16:48:10,828][20381] Updated weights for policy 1, policy_version 1608809 (0.0005) [2023-12-27 16:48:10,885][20381] Updated weights for policy 1, policy_version 1608819 (0.0005) [2023-12-27 16:48:11,265][20347] Updated weights for policy 0, policy_version 1610421 (0.0007) [2023-12-27 16:48:11,327][20347] Updated weights for policy 0, policy_version 1610431 (0.0007) [2023-12-27 16:48:11,390][20347] Updated weights for policy 0, policy_version 1610441 (0.0007) [2023-12-27 16:48:11,539][20381] Updated weights for policy 1, policy_version 1608829 (0.0005) [2023-12-27 16:48:11,601][20381] Updated weights for policy 1, policy_version 1608839 (0.0006) [2023-12-27 16:48:11,666][20381] Updated weights for policy 1, policy_version 1608849 (0.0009) [2023-12-27 16:48:12,107][20347] Updated weights for policy 0, policy_version 1610451 (0.0009) [2023-12-27 16:48:12,164][20347] Updated weights for policy 0, policy_version 1610461 (0.0011) [2023-12-27 16:48:12,223][20347] Updated weights for policy 0, policy_version 1610471 (0.0011) [2023-12-27 16:48:12,344][20381] Updated weights for policy 1, policy_version 1608859 (0.0008) [2023-12-27 16:48:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 824270848. Throughput: 0: 10382.2, 1: 10183.0. Samples: 824263012. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:48:12,363][19170] Avg episode reward: [(0, '990.270'), (1, '931.600')] [2023-12-27 16:48:12,412][20381] Updated weights for policy 1, policy_version 1608869 (0.0011) [2023-12-27 16:48:12,464][20381] Updated weights for policy 1, policy_version 1608879 (0.0010) [2023-12-27 16:48:12,938][20347] Updated weights for policy 0, policy_version 1610481 (0.0010) [2023-12-27 16:48:12,989][20347] Updated weights for policy 0, policy_version 1610491 (0.0005) [2023-12-27 16:48:13,051][20347] Updated weights for policy 0, policy_version 1610501 (0.0005) [2023-12-27 16:48:13,110][20347] Updated weights for policy 0, policy_version 1610511 (0.0005) [2023-12-27 16:48:13,156][20381] Updated weights for policy 1, policy_version 1608889 (0.0010) [2023-12-27 16:48:13,217][20381] Updated weights for policy 1, policy_version 1608899 (0.0010) [2023-12-27 16:48:13,265][20381] Updated weights for policy 1, policy_version 1608909 (0.0010) [2023-12-27 16:48:13,313][20381] Updated weights for policy 1, policy_version 1608919 (0.0010) [2023-12-27 16:48:13,660][20347] Updated weights for policy 0, policy_version 1610521 (0.0007) [2023-12-27 16:48:13,717][20347] Updated weights for policy 0, policy_version 1610531 (0.0010) [2023-12-27 16:48:13,765][20347] Updated weights for policy 0, policy_version 1610541 (0.0010) [2023-12-27 16:48:13,964][20381] Updated weights for policy 1, policy_version 1608929 (0.0010) [2023-12-27 16:48:14,016][20381] Updated weights for policy 1, policy_version 1608939 (0.0007) [2023-12-27 16:48:14,065][20381] Updated weights for policy 1, policy_version 1608949 (0.0011) [2023-12-27 16:48:14,384][20347] Updated weights for policy 0, policy_version 1610551 (0.0007) [2023-12-27 16:48:14,438][20347] Updated weights for policy 0, policy_version 1610561 (0.0010) [2023-12-27 16:48:14,498][20347] Updated weights for policy 0, policy_version 1610571 (0.0010) [2023-12-27 16:48:14,752][20381] Updated weights for policy 1, policy_version 1608959 (0.0010) [2023-12-27 16:48:14,813][20381] Updated weights for policy 1, policy_version 1608969 (0.0008) [2023-12-27 16:48:14,876][20381] Updated weights for policy 1, policy_version 1608979 (0.0006) [2023-12-27 16:48:15,126][20347] Updated weights for policy 0, policy_version 1610581 (0.0010) [2023-12-27 16:48:15,185][20347] Updated weights for policy 0, policy_version 1610591 (0.0010) [2023-12-27 16:48:15,248][20347] Updated weights for policy 0, policy_version 1610601 (0.0009) [2023-12-27 16:48:15,499][20381] Updated weights for policy 1, policy_version 1608989 (0.0006) [2023-12-27 16:48:15,551][20381] Updated weights for policy 1, policy_version 1608999 (0.0005) [2023-12-27 16:48:15,608][20381] Updated weights for policy 1, policy_version 1609009 (0.0007) [2023-12-27 16:48:15,821][20347] Updated weights for policy 0, policy_version 1610611 (0.0007) [2023-12-27 16:48:15,888][20347] Updated weights for policy 0, policy_version 1610621 (0.0010) [2023-12-27 16:48:15,950][20347] Updated weights for policy 0, policy_version 1610631 (0.0008) [2023-12-27 16:48:16,204][20381] Updated weights for policy 1, policy_version 1609019 (0.0009) [2023-12-27 16:48:16,258][20381] Updated weights for policy 1, policy_version 1609029 (0.0007) [2023-12-27 16:48:16,306][20381] Updated weights for policy 1, policy_version 1609039 (0.0008) [2023-12-27 16:48:16,558][20347] Updated weights for policy 0, policy_version 1610641 (0.0006) [2023-12-27 16:48:16,611][20347] Updated weights for policy 0, policy_version 1610651 (0.0009) [2023-12-27 16:48:16,662][20347] Updated weights for policy 0, policy_version 1610661 (0.0010) [2023-12-27 16:48:16,710][20347] Updated weights for policy 0, policy_version 1610671 (0.0010) [2023-12-27 16:48:16,927][20381] Updated weights for policy 1, policy_version 1609049 (0.0007) [2023-12-27 16:48:16,983][20381] Updated weights for policy 1, policy_version 1609059 (0.0008) [2023-12-27 16:48:17,037][20381] Updated weights for policy 1, policy_version 1609069 (0.0007) [2023-12-27 16:48:17,089][20381] Updated weights for policy 1, policy_version 1609079 (0.0008) [2023-12-27 16:48:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 824377344. Throughput: 0: 10385.9, 1: 10262.7. Samples: 824389448. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:48:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.560')] [2023-12-27 16:48:17,406][20347] Updated weights for policy 0, policy_version 1610681 (0.0010) [2023-12-27 16:48:17,464][20347] Updated weights for policy 0, policy_version 1610691 (0.0010) [2023-12-27 16:48:17,522][20347] Updated weights for policy 0, policy_version 1610701 (0.0010) [2023-12-27 16:48:17,723][20381] Updated weights for policy 1, policy_version 1609089 (0.0010) [2023-12-27 16:48:17,787][20381] Updated weights for policy 1, policy_version 1609099 (0.0008) [2023-12-27 16:48:17,854][20381] Updated weights for policy 1, policy_version 1609109 (0.0007) [2023-12-27 16:48:18,172][20347] Updated weights for policy 0, policy_version 1610711 (0.0008) [2023-12-27 16:48:18,234][20347] Updated weights for policy 0, policy_version 1610721 (0.0008) [2023-12-27 16:48:18,291][20347] Updated weights for policy 0, policy_version 1610731 (0.0007) [2023-12-27 16:48:18,481][20381] Updated weights for policy 1, policy_version 1609119 (0.0010) [2023-12-27 16:48:18,544][20381] Updated weights for policy 1, policy_version 1609129 (0.0011) [2023-12-27 16:48:18,603][20381] Updated weights for policy 1, policy_version 1609139 (0.0011) [2023-12-27 16:48:19,028][20347] Updated weights for policy 0, policy_version 1610741 (0.0008) [2023-12-27 16:48:19,080][20347] Updated weights for policy 0, policy_version 1610751 (0.0008) [2023-12-27 16:48:19,135][20347] Updated weights for policy 0, policy_version 1610761 (0.0008) [2023-12-27 16:48:19,232][20381] Updated weights for policy 1, policy_version 1609149 (0.0009) [2023-12-27 16:48:19,292][20381] Updated weights for policy 1, policy_version 1609159 (0.0006) [2023-12-27 16:48:19,352][20381] Updated weights for policy 1, policy_version 1609169 (0.0007) [2023-12-27 16:48:19,823][20347] Updated weights for policy 0, policy_version 1610771 (0.0008) [2023-12-27 16:48:19,881][20347] Updated weights for policy 0, policy_version 1610781 (0.0007) [2023-12-27 16:48:19,944][20347] Updated weights for policy 0, policy_version 1610791 (0.0007) [2023-12-27 16:48:20,108][20381] Updated weights for policy 1, policy_version 1609179 (0.0007) [2023-12-27 16:48:20,172][20381] Updated weights for policy 1, policy_version 1609189 (0.0009) [2023-12-27 16:48:20,236][20381] Updated weights for policy 1, policy_version 1609199 (0.0008) [2023-12-27 16:48:20,638][20347] Updated weights for policy 0, policy_version 1610801 (0.0008) [2023-12-27 16:48:20,697][20347] Updated weights for policy 0, policy_version 1610811 (0.0008) [2023-12-27 16:48:20,760][20347] Updated weights for policy 0, policy_version 1610821 (0.0008) [2023-12-27 16:48:20,823][20347] Updated weights for policy 0, policy_version 1610831 (0.0008) [2023-12-27 16:48:20,875][20381] Updated weights for policy 1, policy_version 1609209 (0.0007) [2023-12-27 16:48:20,937][20381] Updated weights for policy 1, policy_version 1609219 (0.0005) [2023-12-27 16:48:20,997][20381] Updated weights for policy 1, policy_version 1609229 (0.0007) [2023-12-27 16:48:21,064][20381] Updated weights for policy 1, policy_version 1609239 (0.0008) [2023-12-27 16:48:21,657][20347] Updated weights for policy 0, policy_version 1610841 (0.0008) [2023-12-27 16:48:21,722][20347] Updated weights for policy 0, policy_version 1610851 (0.0008) [2023-12-27 16:48:21,791][20347] Updated weights for policy 0, policy_version 1610861 (0.0009) [2023-12-27 16:48:21,836][20381] Updated weights for policy 1, policy_version 1609249 (0.0007) [2023-12-27 16:48:21,900][20381] Updated weights for policy 1, policy_version 1609259 (0.0008) [2023-12-27 16:48:21,957][20381] Updated weights for policy 1, policy_version 1609269 (0.0008) [2023-12-27 16:48:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 824475648. Throughput: 0: 10353.0, 1: 10239.1. Samples: 824450356. Policy #0 lag: (min: 31.0, avg: 39.9, max: 63.0) [2023-12-27 16:48:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.580')] [2023-12-27 16:48:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001609272_412016640.pth... [2023-12-27 16:48:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001610864_412459008.pth... [2023-12-27 16:48:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001609680_412155904.pth [2023-12-27 16:48:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001608088_411713536.pth [2023-12-27 16:48:22,529][20347] Updated weights for policy 0, policy_version 1610871 (0.0008) [2023-12-27 16:48:22,594][20347] Updated weights for policy 0, policy_version 1610881 (0.0009) [2023-12-27 16:48:22,629][20381] Updated weights for policy 1, policy_version 1609279 (0.0007) [2023-12-27 16:48:22,654][20347] Updated weights for policy 0, policy_version 1610891 (0.0008) [2023-12-27 16:48:22,685][20381] Updated weights for policy 1, policy_version 1609289 (0.0006) [2023-12-27 16:48:22,746][20381] Updated weights for policy 1, policy_version 1609299 (0.0007) [2023-12-27 16:48:23,354][20347] Updated weights for policy 0, policy_version 1610901 (0.0007) [2023-12-27 16:48:23,412][20347] Updated weights for policy 0, policy_version 1610911 (0.0005) [2023-12-27 16:48:23,428][20381] Updated weights for policy 1, policy_version 1609309 (0.0009) [2023-12-27 16:48:23,477][20347] Updated weights for policy 0, policy_version 1610921 (0.0006) [2023-12-27 16:48:23,482][20381] Updated weights for policy 1, policy_version 1609319 (0.0008) [2023-12-27 16:48:23,543][20381] Updated weights for policy 1, policy_version 1609329 (0.0007) [2023-12-27 16:48:24,059][20347] Updated weights for policy 0, policy_version 1610931 (0.0007) [2023-12-27 16:48:24,125][20347] Updated weights for policy 0, policy_version 1610941 (0.0006) [2023-12-27 16:48:24,187][20347] Updated weights for policy 0, policy_version 1610951 (0.0006) [2023-12-27 16:48:24,191][20381] Updated weights for policy 1, policy_version 1609339 (0.0007) [2023-12-27 16:48:24,249][20381] Updated weights for policy 1, policy_version 1609349 (0.0007) [2023-12-27 16:48:24,297][20381] Updated weights for policy 1, policy_version 1609359 (0.0007) [2023-12-27 16:48:24,840][20347] Updated weights for policy 0, policy_version 1610961 (0.0006) [2023-12-27 16:48:24,887][20347] Updated weights for policy 0, policy_version 1610971 (0.0007) [2023-12-27 16:48:24,939][20347] Updated weights for policy 0, policy_version 1610981 (0.0007) [2023-12-27 16:48:24,978][20381] Updated weights for policy 1, policy_version 1609369 (0.0007) [2023-12-27 16:48:24,994][20347] Updated weights for policy 0, policy_version 1610991 (0.0008) [2023-12-27 16:48:25,025][20381] Updated weights for policy 1, policy_version 1609379 (0.0006) [2023-12-27 16:48:25,070][20381] Updated weights for policy 1, policy_version 1609389 (0.0010) [2023-12-27 16:48:25,122][20381] Updated weights for policy 1, policy_version 1609399 (0.0010) [2023-12-27 16:48:25,707][20347] Updated weights for policy 0, policy_version 1611001 (0.0007) [2023-12-27 16:48:25,762][20347] Updated weights for policy 0, policy_version 1611011 (0.0006) [2023-12-27 16:48:25,788][20291] KL-divergence is very high: 473.5778 [2023-12-27 16:48:25,801][20381] Updated weights for policy 1, policy_version 1609409 (0.0011) [2023-12-27 16:48:25,822][20347] Updated weights for policy 0, policy_version 1611021 (0.0005) [2023-12-27 16:48:25,832][20291] KL-divergence is very high: 408.3795 [2023-12-27 16:48:25,853][20381] Updated weights for policy 1, policy_version 1609419 (0.0010) [2023-12-27 16:48:25,912][20381] Updated weights for policy 1, policy_version 1609429 (0.0010) [2023-12-27 16:48:26,552][20381] Updated weights for policy 1, policy_version 1609439 (0.0007) [2023-12-27 16:48:26,555][20347] Updated weights for policy 0, policy_version 1611031 (0.0008) [2023-12-27 16:48:26,599][20381] Updated weights for policy 1, policy_version 1609449 (0.0005) [2023-12-27 16:48:26,602][20347] Updated weights for policy 0, policy_version 1611041 (0.0007) [2023-12-27 16:48:26,649][20381] Updated weights for policy 1, policy_version 1609459 (0.0007) [2023-12-27 16:48:26,651][20347] Updated weights for policy 0, policy_version 1611051 (0.0007) [2023-12-27 16:48:27,193][20381] Updated weights for policy 1, policy_version 1609469 (0.0010) [2023-12-27 16:48:27,238][20347] Updated weights for policy 0, policy_version 1611061 (0.0005) [2023-12-27 16:48:27,247][20381] Updated weights for policy 1, policy_version 1609479 (0.0010) [2023-12-27 16:48:27,293][20347] Updated weights for policy 0, policy_version 1611071 (0.0007) [2023-12-27 16:48:27,298][20381] Updated weights for policy 1, policy_version 1609489 (0.0010) [2023-12-27 16:48:27,345][20347] Updated weights for policy 0, policy_version 1611081 (0.0010) [2023-12-27 16:48:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 824582144. Throughput: 0: 10238.7, 1: 10232.9. Samples: 824569544. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:27,363][19170] Avg episode reward: [(0, '990.280'), (1, '892.570')] [2023-12-27 16:48:27,958][20381] Updated weights for policy 1, policy_version 1609499 (0.0010) [2023-12-27 16:48:27,959][20347] Updated weights for policy 0, policy_version 1611091 (0.0010) [2023-12-27 16:48:28,011][20347] Updated weights for policy 0, policy_version 1611101 (0.0010) [2023-12-27 16:48:28,012][20381] Updated weights for policy 1, policy_version 1609509 (0.0008) [2023-12-27 16:48:28,062][20347] Updated weights for policy 0, policy_version 1611111 (0.0010) [2023-12-27 16:48:28,067][20381] Updated weights for policy 1, policy_version 1609519 (0.0006) [2023-12-27 16:48:28,720][20347] Updated weights for policy 0, policy_version 1611121 (0.0009) [2023-12-27 16:48:28,778][20347] Updated weights for policy 0, policy_version 1611131 (0.0007) [2023-12-27 16:48:28,779][20381] Updated weights for policy 1, policy_version 1609529 (0.0008) [2023-12-27 16:48:28,829][20381] Updated weights for policy 1, policy_version 1609539 (0.0006) [2023-12-27 16:48:28,837][20347] Updated weights for policy 0, policy_version 1611141 (0.0007) [2023-12-27 16:48:28,886][20347] Updated weights for policy 0, policy_version 1611151 (0.0005) [2023-12-27 16:48:28,887][20381] Updated weights for policy 1, policy_version 1609549 (0.0006) [2023-12-27 16:48:28,942][20381] Updated weights for policy 1, policy_version 1609559 (0.0008) [2023-12-27 16:48:29,580][20347] Updated weights for policy 0, policy_version 1611161 (0.0005) [2023-12-27 16:48:29,625][20381] Updated weights for policy 1, policy_version 1609569 (0.0008) [2023-12-27 16:48:29,649][20347] Updated weights for policy 0, policy_version 1611171 (0.0005) [2023-12-27 16:48:29,676][20381] Updated weights for policy 1, policy_version 1609579 (0.0010) [2023-12-27 16:48:29,712][20347] Updated weights for policy 0, policy_version 1611181 (0.0007) [2023-12-27 16:48:29,729][20381] Updated weights for policy 1, policy_version 1609589 (0.0011) [2023-12-27 16:48:30,291][20347] Updated weights for policy 0, policy_version 1611191 (0.0009) [2023-12-27 16:48:30,348][20347] Updated weights for policy 0, policy_version 1611201 (0.0009) [2023-12-27 16:48:30,404][20347] Updated weights for policy 0, policy_version 1611211 (0.0008) [2023-12-27 16:48:30,461][20381] Updated weights for policy 1, policy_version 1609599 (0.0011) [2023-12-27 16:48:30,515][20381] Updated weights for policy 1, policy_version 1609609 (0.0010) [2023-12-27 16:48:30,569][20381] Updated weights for policy 1, policy_version 1609619 (0.0010) [2023-12-27 16:48:30,985][20347] Updated weights for policy 0, policy_version 1611221 (0.0008) [2023-12-27 16:48:31,050][20347] Updated weights for policy 0, policy_version 1611231 (0.0006) [2023-12-27 16:48:31,111][20347] Updated weights for policy 0, policy_version 1611241 (0.0007) [2023-12-27 16:48:31,346][20381] Updated weights for policy 1, policy_version 1609629 (0.0011) [2023-12-27 16:48:31,430][20381] Updated weights for policy 1, policy_version 1609639 (0.0009) [2023-12-27 16:48:31,491][20381] Updated weights for policy 1, policy_version 1609649 (0.0006) [2023-12-27 16:48:31,839][20347] Updated weights for policy 0, policy_version 1611251 (0.0009) [2023-12-27 16:48:31,906][20347] Updated weights for policy 0, policy_version 1611261 (0.0010) [2023-12-27 16:48:31,964][20347] Updated weights for policy 0, policy_version 1611271 (0.0010) [2023-12-27 16:48:32,163][20381] Updated weights for policy 1, policy_version 1609659 (0.0006) [2023-12-27 16:48:32,225][20381] Updated weights for policy 1, policy_version 1609669 (0.0005) [2023-12-27 16:48:32,289][20381] Updated weights for policy 1, policy_version 1609679 (0.0007) [2023-12-27 16:48:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 824688640. Throughput: 0: 10325.9, 1: 10287.9. Samples: 824696092. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:32,363][19170] Avg episode reward: [(0, '990.280'), (1, '873.430')] [2023-12-27 16:48:32,633][20347] Updated weights for policy 0, policy_version 1611281 (0.0010) [2023-12-27 16:48:32,689][20347] Updated weights for policy 0, policy_version 1611291 (0.0007) [2023-12-27 16:48:32,739][20347] Updated weights for policy 0, policy_version 1611301 (0.0010) [2023-12-27 16:48:32,794][20347] Updated weights for policy 0, policy_version 1611311 (0.0011) [2023-12-27 16:48:32,967][20381] Updated weights for policy 1, policy_version 1609689 (0.0008) [2023-12-27 16:48:33,023][20381] Updated weights for policy 1, policy_version 1609699 (0.0007) [2023-12-27 16:48:33,078][20381] Updated weights for policy 1, policy_version 1609709 (0.0007) [2023-12-27 16:48:33,135][20381] Updated weights for policy 1, policy_version 1609719 (0.0007) [2023-12-27 16:48:33,482][20347] Updated weights for policy 0, policy_version 1611321 (0.0007) [2023-12-27 16:48:33,532][20347] Updated weights for policy 0, policy_version 1611331 (0.0005) [2023-12-27 16:48:33,582][20347] Updated weights for policy 0, policy_version 1611341 (0.0005) [2023-12-27 16:48:33,850][20381] Updated weights for policy 1, policy_version 1609729 (0.0005) [2023-12-27 16:48:33,916][20381] Updated weights for policy 1, policy_version 1609739 (0.0005) [2023-12-27 16:48:33,981][20381] Updated weights for policy 1, policy_version 1609749 (0.0005) [2023-12-27 16:48:34,223][20347] Updated weights for policy 0, policy_version 1611351 (0.0005) [2023-12-27 16:48:34,277][20347] Updated weights for policy 0, policy_version 1611361 (0.0006) [2023-12-27 16:48:34,341][20347] Updated weights for policy 0, policy_version 1611371 (0.0009) [2023-12-27 16:48:34,588][20381] Updated weights for policy 1, policy_version 1609759 (0.0006) [2023-12-27 16:48:34,650][20381] Updated weights for policy 1, policy_version 1609769 (0.0007) [2023-12-27 16:48:34,704][20381] Updated weights for policy 1, policy_version 1609779 (0.0005) [2023-12-27 16:48:34,976][20347] Updated weights for policy 0, policy_version 1611381 (0.0009) [2023-12-27 16:48:35,037][20347] Updated weights for policy 0, policy_version 1611391 (0.0006) [2023-12-27 16:48:35,103][20347] Updated weights for policy 0, policy_version 1611401 (0.0005) [2023-12-27 16:48:35,330][20381] Updated weights for policy 1, policy_version 1609789 (0.0005) [2023-12-27 16:48:35,387][20381] Updated weights for policy 1, policy_version 1609799 (0.0007) [2023-12-27 16:48:35,432][20292] KL-divergence is very high: 115.2976 [2023-12-27 16:48:35,445][20381] Updated weights for policy 1, policy_version 1609809 (0.0007) [2023-12-27 16:48:35,476][20292] KL-divergence is very high: 171.4070 [2023-12-27 16:48:35,483][20292] KL-divergence is very high: 169.3323 [2023-12-27 16:48:35,719][20347] Updated weights for policy 0, policy_version 1611411 (0.0007) [2023-12-27 16:48:35,767][20347] Updated weights for policy 0, policy_version 1611421 (0.0010) [2023-12-27 16:48:35,822][20347] Updated weights for policy 0, policy_version 1611431 (0.0010) [2023-12-27 16:48:36,014][20381] Updated weights for policy 1, policy_version 1609819 (0.0007) [2023-12-27 16:48:36,065][20381] Updated weights for policy 1, policy_version 1609829 (0.0010) [2023-12-27 16:48:36,134][20381] Updated weights for policy 1, policy_version 1609839 (0.0010) [2023-12-27 16:48:36,541][20347] Updated weights for policy 0, policy_version 1611441 (0.0010) [2023-12-27 16:48:36,601][20347] Updated weights for policy 0, policy_version 1611451 (0.0011) [2023-12-27 16:48:36,667][20347] Updated weights for policy 0, policy_version 1611461 (0.0008) [2023-12-27 16:48:36,723][20347] Updated weights for policy 0, policy_version 1611471 (0.0007) [2023-12-27 16:48:36,834][20381] Updated weights for policy 1, policy_version 1609849 (0.0009) [2023-12-27 16:48:36,894][20381] Updated weights for policy 1, policy_version 1609859 (0.0008) [2023-12-27 16:48:36,954][20381] Updated weights for policy 1, policy_version 1609869 (0.0008) [2023-12-27 16:48:37,013][20381] Updated weights for policy 1, policy_version 1609879 (0.0008) [2023-12-27 16:48:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 824786944. Throughput: 0: 10361.7, 1: 10320.6. Samples: 824759180. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '815.040')] [2023-12-27 16:48:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001609880_412172288.pth... [2023-12-27 16:48:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001608664_411860992.pth [2023-12-27 16:48:37,418][20347] Updated weights for policy 0, policy_version 1611481 (0.0006) [2023-12-27 16:48:37,469][20347] Updated weights for policy 0, policy_version 1611491 (0.0005) [2023-12-27 16:48:37,523][20347] Updated weights for policy 0, policy_version 1611501 (0.0006) [2023-12-27 16:48:37,535][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001611504_412622848.pth... [2023-12-27 16:48:37,538][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001610288_412311552.pth [2023-12-27 16:48:37,594][20381] Updated weights for policy 1, policy_version 1609889 (0.0008) [2023-12-27 16:48:37,646][20381] Updated weights for policy 1, policy_version 1609899 (0.0007) [2023-12-27 16:48:37,693][20381] Updated weights for policy 1, policy_version 1609909 (0.0008) [2023-12-27 16:48:38,155][20347] Updated weights for policy 0, policy_version 1611511 (0.0007) [2023-12-27 16:48:38,217][20347] Updated weights for policy 0, policy_version 1611521 (0.0007) [2023-12-27 16:48:38,273][20347] Updated weights for policy 0, policy_version 1611531 (0.0005) [2023-12-27 16:48:38,423][20381] Updated weights for policy 1, policy_version 1609919 (0.0007) [2023-12-27 16:48:38,486][20381] Updated weights for policy 1, policy_version 1609929 (0.0009) [2023-12-27 16:48:38,545][20381] Updated weights for policy 1, policy_version 1609939 (0.0007) [2023-12-27 16:48:38,944][20347] Updated weights for policy 0, policy_version 1611541 (0.0007) [2023-12-27 16:48:39,000][20347] Updated weights for policy 0, policy_version 1611551 (0.0008) [2023-12-27 16:48:39,051][20347] Updated weights for policy 0, policy_version 1611561 (0.0006) [2023-12-27 16:48:39,254][20381] Updated weights for policy 1, policy_version 1609949 (0.0008) [2023-12-27 16:48:39,312][20381] Updated weights for policy 1, policy_version 1609959 (0.0008) [2023-12-27 16:48:39,378][20381] Updated weights for policy 1, policy_version 1609969 (0.0007) [2023-12-27 16:48:39,788][20347] Updated weights for policy 0, policy_version 1611571 (0.0006) [2023-12-27 16:48:39,859][20347] Updated weights for policy 0, policy_version 1611581 (0.0007) [2023-12-27 16:48:39,924][20347] Updated weights for policy 0, policy_version 1611591 (0.0006) [2023-12-27 16:48:39,974][20381] Updated weights for policy 1, policy_version 1609979 (0.0007) [2023-12-27 16:48:40,038][20381] Updated weights for policy 1, policy_version 1609989 (0.0008) [2023-12-27 16:48:40,101][20381] Updated weights for policy 1, policy_version 1609999 (0.0008) [2023-12-27 16:48:40,551][20347] Updated weights for policy 0, policy_version 1611601 (0.0007) [2023-12-27 16:48:40,605][20347] Updated weights for policy 0, policy_version 1611611 (0.0006) [2023-12-27 16:48:40,659][20347] Updated weights for policy 0, policy_version 1611621 (0.0005) [2023-12-27 16:48:40,710][20347] Updated weights for policy 0, policy_version 1611631 (0.0005) [2023-12-27 16:48:40,823][20381] Updated weights for policy 1, policy_version 1610009 (0.0008) [2023-12-27 16:48:40,879][20381] Updated weights for policy 1, policy_version 1610019 (0.0007) [2023-12-27 16:48:40,928][20381] Updated weights for policy 1, policy_version 1610029 (0.0009) [2023-12-27 16:48:40,974][20381] Updated weights for policy 1, policy_version 1610039 (0.0009) [2023-12-27 16:48:41,301][20347] Updated weights for policy 0, policy_version 1611641 (0.0006) [2023-12-27 16:48:41,367][20347] Updated weights for policy 0, policy_version 1611651 (0.0008) [2023-12-27 16:48:41,435][20347] Updated weights for policy 0, policy_version 1611661 (0.0006) [2023-12-27 16:48:41,859][20381] Updated weights for policy 1, policy_version 1610049 (0.0008) [2023-12-27 16:48:41,924][20381] Updated weights for policy 1, policy_version 1610059 (0.0007) [2023-12-27 16:48:41,985][20381] Updated weights for policy 1, policy_version 1610069 (0.0006) [2023-12-27 16:48:42,138][20347] Updated weights for policy 0, policy_version 1611671 (0.0006) [2023-12-27 16:48:42,187][20347] Updated weights for policy 0, policy_version 1611681 (0.0005) [2023-12-27 16:48:42,237][20347] Updated weights for policy 0, policy_version 1611691 (0.0009) [2023-12-27 16:48:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 824893440. Throughput: 0: 10328.1, 1: 10306.2. Samples: 824883080. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:42,363][19170] Avg episode reward: [(0, '990.250'), (1, '775.810')] [2023-12-27 16:48:42,640][20381] Updated weights for policy 1, policy_version 1610079 (0.0006) [2023-12-27 16:48:42,701][20381] Updated weights for policy 1, policy_version 1610089 (0.0006) [2023-12-27 16:48:42,761][20381] Updated weights for policy 1, policy_version 1610099 (0.0005) [2023-12-27 16:48:42,973][20347] Updated weights for policy 0, policy_version 1611701 (0.0007) [2023-12-27 16:48:43,019][20347] Updated weights for policy 0, policy_version 1611711 (0.0007) [2023-12-27 16:48:43,072][20347] Updated weights for policy 0, policy_version 1611721 (0.0008) [2023-12-27 16:48:43,371][20381] Updated weights for policy 1, policy_version 1610109 (0.0005) [2023-12-27 16:48:43,427][20381] Updated weights for policy 1, policy_version 1610119 (0.0005) [2023-12-27 16:48:43,488][20381] Updated weights for policy 1, policy_version 1610129 (0.0009) [2023-12-27 16:48:43,814][20347] Updated weights for policy 0, policy_version 1611731 (0.0007) [2023-12-27 16:48:43,865][20347] Updated weights for policy 0, policy_version 1611741 (0.0008) [2023-12-27 16:48:43,920][20347] Updated weights for policy 0, policy_version 1611751 (0.0008) [2023-12-27 16:48:44,027][20381] Updated weights for policy 1, policy_version 1610139 (0.0009) [2023-12-27 16:48:44,081][20381] Updated weights for policy 1, policy_version 1610149 (0.0005) [2023-12-27 16:48:44,136][20381] Updated weights for policy 1, policy_version 1610159 (0.0009) [2023-12-27 16:48:44,656][20347] Updated weights for policy 0, policy_version 1611761 (0.0008) [2023-12-27 16:48:44,705][20347] Updated weights for policy 0, policy_version 1611771 (0.0007) [2023-12-27 16:48:44,767][20347] Updated weights for policy 0, policy_version 1611781 (0.0007) [2023-12-27 16:48:44,772][20381] Updated weights for policy 1, policy_version 1610169 (0.0010) [2023-12-27 16:48:44,834][20347] Updated weights for policy 0, policy_version 1611791 (0.0008) [2023-12-27 16:48:44,834][20381] Updated weights for policy 1, policy_version 1610179 (0.0009) [2023-12-27 16:48:44,892][20381] Updated weights for policy 1, policy_version 1610189 (0.0006) [2023-12-27 16:48:44,951][20381] Updated weights for policy 1, policy_version 1610199 (0.0005) [2023-12-27 16:48:45,566][20381] Updated weights for policy 1, policy_version 1610209 (0.0006) [2023-12-27 16:48:45,580][20347] Updated weights for policy 0, policy_version 1611801 (0.0007) [2023-12-27 16:48:45,617][20381] Updated weights for policy 1, policy_version 1610219 (0.0006) [2023-12-27 16:48:45,633][20347] Updated weights for policy 0, policy_version 1611811 (0.0007) [2023-12-27 16:48:45,671][20381] Updated weights for policy 1, policy_version 1610229 (0.0008) [2023-12-27 16:48:45,688][20347] Updated weights for policy 0, policy_version 1611821 (0.0006) [2023-12-27 16:48:46,231][20381] Updated weights for policy 1, policy_version 1610239 (0.0007) [2023-12-27 16:48:46,289][20381] Updated weights for policy 1, policy_version 1610249 (0.0007) [2023-12-27 16:48:46,348][20381] Updated weights for policy 1, policy_version 1610259 (0.0006) [2023-12-27 16:48:46,431][20347] Updated weights for policy 0, policy_version 1611831 (0.0008) [2023-12-27 16:48:46,474][20291] KL-divergence is very high: 103.3126 [2023-12-27 16:48:46,486][20347] Updated weights for policy 0, policy_version 1611841 (0.0008) [2023-12-27 16:48:46,515][20291] KL-divergence is very high: 141.7177 [2023-12-27 16:48:46,535][20347] Updated weights for policy 0, policy_version 1611851 (0.0008) [2023-12-27 16:48:46,889][20381] Updated weights for policy 1, policy_version 1610269 (0.0005) [2023-12-27 16:48:46,944][20381] Updated weights for policy 1, policy_version 1610279 (0.0005) [2023-12-27 16:48:47,001][20381] Updated weights for policy 1, policy_version 1610289 (0.0005) [2023-12-27 16:48:47,324][20347] Updated weights for policy 0, policy_version 1611861 (0.0008) [2023-12-27 16:48:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 824991744. Throughput: 0: 10218.3, 1: 10435.2. Samples: 825006300. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:47,363][19170] Avg episode reward: [(0, '980.500'), (1, '814.290')] [2023-12-27 16:48:47,381][20347] Updated weights for policy 0, policy_version 1611871 (0.0007) [2023-12-27 16:48:47,435][20347] Updated weights for policy 0, policy_version 1611881 (0.0008) [2023-12-27 16:48:47,590][20381] Updated weights for policy 1, policy_version 1610299 (0.0006) [2023-12-27 16:48:47,645][20381] Updated weights for policy 1, policy_version 1610309 (0.0008) [2023-12-27 16:48:47,695][20381] Updated weights for policy 1, policy_version 1610319 (0.0008) [2023-12-27 16:48:48,129][20347] Updated weights for policy 0, policy_version 1611891 (0.0007) [2023-12-27 16:48:48,188][20347] Updated weights for policy 0, policy_version 1611901 (0.0007) [2023-12-27 16:48:48,243][20347] Updated weights for policy 0, policy_version 1611911 (0.0007) [2023-12-27 16:48:48,393][20381] Updated weights for policy 1, policy_version 1610329 (0.0008) [2023-12-27 16:48:48,452][20381] Updated weights for policy 1, policy_version 1610339 (0.0008) [2023-12-27 16:48:48,497][20381] Updated weights for policy 1, policy_version 1610349 (0.0008) [2023-12-27 16:48:48,549][20381] Updated weights for policy 1, policy_version 1610359 (0.0008) [2023-12-27 16:48:48,939][20347] Updated weights for policy 0, policy_version 1611921 (0.0008) [2023-12-27 16:48:48,993][20347] Updated weights for policy 0, policy_version 1611931 (0.0010) [2023-12-27 16:48:49,043][20347] Updated weights for policy 0, policy_version 1611941 (0.0009) [2023-12-27 16:48:49,091][20347] Updated weights for policy 0, policy_version 1611951 (0.0009) [2023-12-27 16:48:49,322][20381] Updated weights for policy 1, policy_version 1610369 (0.0007) [2023-12-27 16:48:49,384][20381] Updated weights for policy 1, policy_version 1610379 (0.0006) [2023-12-27 16:48:49,433][20381] Updated weights for policy 1, policy_version 1610389 (0.0007) [2023-12-27 16:48:49,839][20347] Updated weights for policy 0, policy_version 1611961 (0.0010) [2023-12-27 16:48:49,899][20347] Updated weights for policy 0, policy_version 1611971 (0.0008) [2023-12-27 16:48:49,963][20347] Updated weights for policy 0, policy_version 1611981 (0.0012) [2023-12-27 16:48:50,095][20381] Updated weights for policy 1, policy_version 1610399 (0.0006) [2023-12-27 16:48:50,157][20381] Updated weights for policy 1, policy_version 1610409 (0.0005) [2023-12-27 16:48:50,218][20381] Updated weights for policy 1, policy_version 1610419 (0.0006) [2023-12-27 16:48:50,629][20347] Updated weights for policy 0, policy_version 1611991 (0.0008) [2023-12-27 16:48:50,687][20347] Updated weights for policy 0, policy_version 1612001 (0.0007) [2023-12-27 16:48:50,752][20347] Updated weights for policy 0, policy_version 1612011 (0.0011) [2023-12-27 16:48:50,849][20381] Updated weights for policy 1, policy_version 1610429 (0.0006) [2023-12-27 16:48:50,910][20381] Updated weights for policy 1, policy_version 1610439 (0.0007) [2023-12-27 16:48:50,969][20381] Updated weights for policy 1, policy_version 1610449 (0.0008) [2023-12-27 16:48:51,473][20347] Updated weights for policy 0, policy_version 1612021 (0.0009) [2023-12-27 16:48:51,531][20347] Updated weights for policy 0, policy_version 1612031 (0.0008) [2023-12-27 16:48:51,580][20347] Updated weights for policy 0, policy_version 1612041 (0.0006) [2023-12-27 16:48:51,740][20381] Updated weights for policy 1, policy_version 1610459 (0.0007) [2023-12-27 16:48:51,800][20381] Updated weights for policy 1, policy_version 1610469 (0.0008) [2023-12-27 16:48:51,862][20381] Updated weights for policy 1, policy_version 1610479 (0.0008) [2023-12-27 16:48:52,277][20347] Updated weights for policy 0, policy_version 1612051 (0.0008) [2023-12-27 16:48:52,343][20347] Updated weights for policy 0, policy_version 1612061 (0.0007) [2023-12-27 16:48:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 825090048. Throughput: 0: 10180.7, 1: 10424.1. Samples: 825065224. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:52,363][19170] Avg episode reward: [(0, '990.250'), (1, '804.950')] [2023-12-27 16:48:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001610488_412327936.pth... [2023-12-27 16:48:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001609272_412016640.pth [2023-12-27 16:48:52,416][20347] Updated weights for policy 0, policy_version 1612071 (0.0008) [2023-12-27 16:48:52,464][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001612080_412770304.pth... [2023-12-27 16:48:52,467][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001610864_412459008.pth [2023-12-27 16:48:52,610][20381] Updated weights for policy 1, policy_version 1610489 (0.0008) [2023-12-27 16:48:52,670][20381] Updated weights for policy 1, policy_version 1610499 (0.0007) [2023-12-27 16:48:52,726][20381] Updated weights for policy 1, policy_version 1610509 (0.0006) [2023-12-27 16:48:52,781][20381] Updated weights for policy 1, policy_version 1610519 (0.0006) [2023-12-27 16:48:53,092][20347] Updated weights for policy 0, policy_version 1612081 (0.0007) [2023-12-27 16:48:53,143][20347] Updated weights for policy 0, policy_version 1612091 (0.0008) [2023-12-27 16:48:53,194][20347] Updated weights for policy 0, policy_version 1612101 (0.0007) [2023-12-27 16:48:53,247][20347] Updated weights for policy 0, policy_version 1612111 (0.0007) [2023-12-27 16:48:53,460][20381] Updated weights for policy 1, policy_version 1610529 (0.0008) [2023-12-27 16:48:53,512][20381] Updated weights for policy 1, policy_version 1610539 (0.0007) [2023-12-27 16:48:53,567][20381] Updated weights for policy 1, policy_version 1610549 (0.0005) [2023-12-27 16:48:53,906][20347] Updated weights for policy 0, policy_version 1612121 (0.0009) [2023-12-27 16:48:53,959][20347] Updated weights for policy 0, policy_version 1612131 (0.0007) [2023-12-27 16:48:54,021][20347] Updated weights for policy 0, policy_version 1612141 (0.0005) [2023-12-27 16:48:54,223][20381] Updated weights for policy 1, policy_version 1610559 (0.0009) [2023-12-27 16:48:54,272][20381] Updated weights for policy 1, policy_version 1610569 (0.0011) [2023-12-27 16:48:54,322][20381] Updated weights for policy 1, policy_version 1610579 (0.0011) [2023-12-27 16:48:54,683][20347] Updated weights for policy 0, policy_version 1612151 (0.0006) [2023-12-27 16:48:54,740][20347] Updated weights for policy 0, policy_version 1612161 (0.0010) [2023-12-27 16:48:54,797][20347] Updated weights for policy 0, policy_version 1612171 (0.0006) [2023-12-27 16:48:55,019][20381] Updated weights for policy 1, policy_version 1610589 (0.0011) [2023-12-27 16:48:55,077][20381] Updated weights for policy 1, policy_version 1610599 (0.0010) [2023-12-27 16:48:55,135][20381] Updated weights for policy 1, policy_version 1610609 (0.0011) [2023-12-27 16:48:55,390][20347] Updated weights for policy 0, policy_version 1612181 (0.0007) [2023-12-27 16:48:55,445][20347] Updated weights for policy 0, policy_version 1612191 (0.0006) [2023-12-27 16:48:55,490][20347] Updated weights for policy 0, policy_version 1612201 (0.0005) [2023-12-27 16:48:55,787][20381] Updated weights for policy 1, policy_version 1610619 (0.0009) [2023-12-27 16:48:55,841][20381] Updated weights for policy 1, policy_version 1610629 (0.0006) [2023-12-27 16:48:55,890][20381] Updated weights for policy 1, policy_version 1610639 (0.0007) [2023-12-27 16:48:56,123][20347] Updated weights for policy 0, policy_version 1612211 (0.0006) [2023-12-27 16:48:56,179][20347] Updated weights for policy 0, policy_version 1612221 (0.0008) [2023-12-27 16:48:56,239][20347] Updated weights for policy 0, policy_version 1612231 (0.0008) [2023-12-27 16:48:56,551][20381] Updated weights for policy 1, policy_version 1610649 (0.0005) [2023-12-27 16:48:56,599][20381] Updated weights for policy 1, policy_version 1610659 (0.0008) [2023-12-27 16:48:56,644][20381] Updated weights for policy 1, policy_version 1610669 (0.0007) [2023-12-27 16:48:56,695][20381] Updated weights for policy 1, policy_version 1610679 (0.0007) [2023-12-27 16:48:56,911][20347] Updated weights for policy 0, policy_version 1612241 (0.0007) [2023-12-27 16:48:56,964][20347] Updated weights for policy 0, policy_version 1612251 (0.0007) [2023-12-27 16:48:57,013][20347] Updated weights for policy 0, policy_version 1612261 (0.0008) [2023-12-27 16:48:57,064][20347] Updated weights for policy 0, policy_version 1612271 (0.0007) [2023-12-27 16:48:57,335][20381] Updated weights for policy 1, policy_version 1610689 (0.0007) [2023-12-27 16:48:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 825196544. Throughput: 0: 10192.8, 1: 10363.1. Samples: 825188028. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.060')] [2023-12-27 16:48:57,383][20381] Updated weights for policy 1, policy_version 1610699 (0.0007) [2023-12-27 16:48:57,433][20381] Updated weights for policy 1, policy_version 1610709 (0.0007) [2023-12-27 16:48:57,729][20347] Updated weights for policy 0, policy_version 1612281 (0.0010) [2023-12-27 16:48:57,786][20347] Updated weights for policy 0, policy_version 1612291 (0.0010) [2023-12-27 16:48:57,843][20347] Updated weights for policy 0, policy_version 1612301 (0.0007) [2023-12-27 16:48:58,145][20381] Updated weights for policy 1, policy_version 1610719 (0.0007) [2023-12-27 16:48:58,202][20381] Updated weights for policy 1, policy_version 1610729 (0.0007) [2023-12-27 16:48:58,263][20381] Updated weights for policy 1, policy_version 1610739 (0.0008) [2023-12-27 16:48:58,484][20347] Updated weights for policy 0, policy_version 1612311 (0.0008) [2023-12-27 16:48:58,544][20347] Updated weights for policy 0, policy_version 1612321 (0.0008) [2023-12-27 16:48:58,597][20347] Updated weights for policy 0, policy_version 1612331 (0.0009) [2023-12-27 16:48:58,983][20381] Updated weights for policy 1, policy_version 1610749 (0.0007) [2023-12-27 16:48:59,037][20381] Updated weights for policy 1, policy_version 1610759 (0.0006) [2023-12-27 16:48:59,096][20381] Updated weights for policy 1, policy_version 1610769 (0.0008) [2023-12-27 16:48:59,376][20347] Updated weights for policy 0, policy_version 1612341 (0.0008) [2023-12-27 16:48:59,436][20347] Updated weights for policy 0, policy_version 1612351 (0.0008) [2023-12-27 16:48:59,494][20347] Updated weights for policy 0, policy_version 1612361 (0.0007) [2023-12-27 16:48:59,811][20381] Updated weights for policy 1, policy_version 1610779 (0.0007) [2023-12-27 16:48:59,875][20381] Updated weights for policy 1, policy_version 1610789 (0.0008) [2023-12-27 16:48:59,938][20381] Updated weights for policy 1, policy_version 1610799 (0.0006) [2023-12-27 16:49:00,155][20347] Updated weights for policy 0, policy_version 1612371 (0.0007) [2023-12-27 16:49:00,211][20347] Updated weights for policy 0, policy_version 1612381 (0.0007) [2023-12-27 16:49:00,268][20347] Updated weights for policy 0, policy_version 1612391 (0.0007) [2023-12-27 16:49:00,530][20381] Updated weights for policy 1, policy_version 1610809 (0.0006) [2023-12-27 16:49:00,579][20381] Updated weights for policy 1, policy_version 1610819 (0.0005) [2023-12-27 16:49:00,638][20381] Updated weights for policy 1, policy_version 1610829 (0.0006) [2023-12-27 16:49:00,705][20381] Updated weights for policy 1, policy_version 1610839 (0.0006) [2023-12-27 16:49:00,903][20347] Updated weights for policy 0, policy_version 1612401 (0.0006) [2023-12-27 16:49:00,958][20347] Updated weights for policy 0, policy_version 1612411 (0.0008) [2023-12-27 16:49:01,009][20347] Updated weights for policy 0, policy_version 1612421 (0.0006) [2023-12-27 16:49:01,072][20347] Updated weights for policy 0, policy_version 1612431 (0.0008) [2023-12-27 16:49:01,266][20381] Updated weights for policy 1, policy_version 1610849 (0.0007) [2023-12-27 16:49:01,319][20381] Updated weights for policy 1, policy_version 1610859 (0.0009) [2023-12-27 16:49:01,381][20381] Updated weights for policy 1, policy_version 1610869 (0.0007) [2023-12-27 16:49:01,812][20347] Updated weights for policy 0, policy_version 1612441 (0.0007) [2023-12-27 16:49:01,868][20347] Updated weights for policy 0, policy_version 1612451 (0.0008) [2023-12-27 16:49:01,926][20347] Updated weights for policy 0, policy_version 1612461 (0.0008) [2023-12-27 16:49:02,076][20381] Updated weights for policy 1, policy_version 1610879 (0.0008) [2023-12-27 16:49:02,125][20381] Updated weights for policy 1, policy_version 1610889 (0.0008) [2023-12-27 16:49:02,174][20381] Updated weights for policy 1, policy_version 1610899 (0.0007) [2023-12-27 16:49:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 825303040. Throughput: 0: 10123.7, 1: 10341.7. Samples: 825310392. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '796.180')] [2023-12-27 16:49:02,658][20347] Updated weights for policy 0, policy_version 1612471 (0.0007) [2023-12-27 16:49:02,711][20347] Updated weights for policy 0, policy_version 1612481 (0.0008) [2023-12-27 16:49:02,766][20347] Updated weights for policy 0, policy_version 1612491 (0.0006) [2023-12-27 16:49:02,820][20381] Updated weights for policy 1, policy_version 1610909 (0.0006) [2023-12-27 16:49:02,876][20381] Updated weights for policy 1, policy_version 1610919 (0.0006) [2023-12-27 16:49:02,938][20381] Updated weights for policy 1, policy_version 1610929 (0.0006) [2023-12-27 16:49:03,509][20347] Updated weights for policy 0, policy_version 1612501 (0.0007) [2023-12-27 16:49:03,532][20381] Updated weights for policy 1, policy_version 1610939 (0.0006) [2023-12-27 16:49:03,558][20347] Updated weights for policy 0, policy_version 1612511 (0.0007) [2023-12-27 16:49:03,591][20381] Updated weights for policy 1, policy_version 1610949 (0.0006) [2023-12-27 16:49:03,608][20347] Updated weights for policy 0, policy_version 1612521 (0.0006) [2023-12-27 16:49:03,646][20381] Updated weights for policy 1, policy_version 1610959 (0.0006) [2023-12-27 16:49:04,323][20381] Updated weights for policy 1, policy_version 1610969 (0.0008) [2023-12-27 16:49:04,376][20381] Updated weights for policy 1, policy_version 1610979 (0.0007) [2023-12-27 16:49:04,388][20347] Updated weights for policy 0, policy_version 1612531 (0.0006) [2023-12-27 16:49:04,436][20381] Updated weights for policy 1, policy_version 1610989 (0.0006) [2023-12-27 16:49:04,449][20347] Updated weights for policy 0, policy_version 1612541 (0.0006) [2023-12-27 16:49:04,492][20381] Updated weights for policy 1, policy_version 1610999 (0.0006) [2023-12-27 16:49:04,509][20347] Updated weights for policy 0, policy_version 1612551 (0.0006) [2023-12-27 16:49:05,173][20381] Updated weights for policy 1, policy_version 1611009 (0.0007) [2023-12-27 16:49:05,194][20347] Updated weights for policy 0, policy_version 1612561 (0.0007) [2023-12-27 16:49:05,230][20381] Updated weights for policy 1, policy_version 1611019 (0.0006) [2023-12-27 16:49:05,255][20347] Updated weights for policy 0, policy_version 1612571 (0.0008) [2023-12-27 16:49:05,286][20381] Updated weights for policy 1, policy_version 1611029 (0.0007) [2023-12-27 16:49:05,318][20347] Updated weights for policy 0, policy_version 1612581 (0.0007) [2023-12-27 16:49:05,373][20347] Updated weights for policy 0, policy_version 1612591 (0.0007) [2023-12-27 16:49:05,979][20381] Updated weights for policy 1, policy_version 1611039 (0.0007) [2023-12-27 16:49:06,035][20381] Updated weights for policy 1, policy_version 1611049 (0.0006) [2023-12-27 16:49:06,052][20347] Updated weights for policy 0, policy_version 1612601 (0.0006) [2023-12-27 16:49:06,093][20381] Updated weights for policy 1, policy_version 1611059 (0.0008) [2023-12-27 16:49:06,106][20347] Updated weights for policy 0, policy_version 1612611 (0.0007) [2023-12-27 16:49:06,160][20347] Updated weights for policy 0, policy_version 1612621 (0.0008) [2023-12-27 16:49:06,784][20347] Updated weights for policy 0, policy_version 1612631 (0.0007) [2023-12-27 16:49:06,831][20381] Updated weights for policy 1, policy_version 1611069 (0.0008) [2023-12-27 16:49:06,843][20347] Updated weights for policy 0, policy_version 1612641 (0.0006) [2023-12-27 16:49:06,885][20381] Updated weights for policy 1, policy_version 1611079 (0.0007) [2023-12-27 16:49:06,902][20347] Updated weights for policy 0, policy_version 1612651 (0.0007) [2023-12-27 16:49:06,939][20381] Updated weights for policy 1, policy_version 1611089 (0.0006) [2023-12-27 16:49:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 825401344. Throughput: 0: 10111.2, 1: 10362.6. Samples: 825371676. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '825.540')] [2023-12-27 16:49:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001612656_412917760.pth... [2023-12-27 16:49:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001611096_412483584.pth... [2023-12-27 16:49:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001609880_412172288.pth [2023-12-27 16:49:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001611504_412622848.pth [2023-12-27 16:49:07,482][20347] Updated weights for policy 0, policy_version 1612661 (0.0007) [2023-12-27 16:49:07,537][20347] Updated weights for policy 0, policy_version 1612671 (0.0008) [2023-12-27 16:49:07,592][20347] Updated weights for policy 0, policy_version 1612681 (0.0007) [2023-12-27 16:49:07,666][20381] Updated weights for policy 1, policy_version 1611099 (0.0008) [2023-12-27 16:49:07,719][20381] Updated weights for policy 1, policy_version 1611109 (0.0009) [2023-12-27 16:49:07,772][20381] Updated weights for policy 1, policy_version 1611119 (0.0008) [2023-12-27 16:49:08,227][20347] Updated weights for policy 0, policy_version 1612691 (0.0007) [2023-12-27 16:49:08,288][20347] Updated weights for policy 0, policy_version 1612701 (0.0008) [2023-12-27 16:49:08,350][20347] Updated weights for policy 0, policy_version 1612711 (0.0007) [2023-12-27 16:49:08,472][20381] Updated weights for policy 1, policy_version 1611129 (0.0007) [2023-12-27 16:49:08,531][20381] Updated weights for policy 1, policy_version 1611139 (0.0008) [2023-12-27 16:49:08,592][20381] Updated weights for policy 1, policy_version 1611149 (0.0009) [2023-12-27 16:49:08,641][20381] Updated weights for policy 1, policy_version 1611159 (0.0007) [2023-12-27 16:49:08,982][20347] Updated weights for policy 0, policy_version 1612721 (0.0008) [2023-12-27 16:49:09,036][20347] Updated weights for policy 0, policy_version 1612731 (0.0008) [2023-12-27 16:49:09,091][20347] Updated weights for policy 0, policy_version 1612741 (0.0008) [2023-12-27 16:49:09,147][20347] Updated weights for policy 0, policy_version 1612751 (0.0008) [2023-12-27 16:49:09,356][20381] Updated weights for policy 1, policy_version 1611169 (0.0008) [2023-12-27 16:49:09,424][20381] Updated weights for policy 1, policy_version 1611179 (0.0008) [2023-12-27 16:49:09,484][20381] Updated weights for policy 1, policy_version 1611189 (0.0007) [2023-12-27 16:49:09,849][20347] Updated weights for policy 0, policy_version 1612761 (0.0008) [2023-12-27 16:49:09,915][20347] Updated weights for policy 0, policy_version 1612771 (0.0007) [2023-12-27 16:49:09,985][20347] Updated weights for policy 0, policy_version 1612781 (0.0008) [2023-12-27 16:49:10,227][20381] Updated weights for policy 1, policy_version 1611199 (0.0008) [2023-12-27 16:49:10,291][20381] Updated weights for policy 1, policy_version 1611209 (0.0008) [2023-12-27 16:49:10,351][20381] Updated weights for policy 1, policy_version 1611219 (0.0008) [2023-12-27 16:49:10,635][20347] Updated weights for policy 0, policy_version 1612791 (0.0008) [2023-12-27 16:49:10,687][20347] Updated weights for policy 0, policy_version 1612801 (0.0007) [2023-12-27 16:49:10,741][20347] Updated weights for policy 0, policy_version 1612811 (0.0005) [2023-12-27 16:49:11,034][20381] Updated weights for policy 1, policy_version 1611229 (0.0008) [2023-12-27 16:49:11,092][20381] Updated weights for policy 1, policy_version 1611239 (0.0009) [2023-12-27 16:49:11,159][20381] Updated weights for policy 1, policy_version 1611249 (0.0009) [2023-12-27 16:49:11,465][20347] Updated weights for policy 0, policy_version 1612821 (0.0007) [2023-12-27 16:49:11,527][20347] Updated weights for policy 0, policy_version 1612831 (0.0010) [2023-12-27 16:49:11,593][20347] Updated weights for policy 0, policy_version 1612841 (0.0011) [2023-12-27 16:49:11,923][20381] Updated weights for policy 1, policy_version 1611259 (0.0008) [2023-12-27 16:49:11,983][20381] Updated weights for policy 1, policy_version 1611269 (0.0009) [2023-12-27 16:49:12,042][20381] Updated weights for policy 1, policy_version 1611279 (0.0008) [2023-12-27 16:49:12,227][20347] Updated weights for policy 0, policy_version 1612851 (0.0009) [2023-12-27 16:49:12,280][20347] Updated weights for policy 0, policy_version 1612861 (0.0007) [2023-12-27 16:49:12,342][20347] Updated weights for policy 0, policy_version 1612871 (0.0007) [2023-12-27 16:49:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 825499648. Throughput: 0: 10227.2, 1: 10312.0. Samples: 825493808. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:12,363][19170] Avg episode reward: [(0, '990.240'), (1, '795.100')] [2023-12-27 16:49:12,779][20381] Updated weights for policy 1, policy_version 1611289 (0.0009) [2023-12-27 16:49:12,835][20381] Updated weights for policy 1, policy_version 1611299 (0.0008) [2023-12-27 16:49:12,900][20381] Updated weights for policy 1, policy_version 1611309 (0.0007) [2023-12-27 16:49:12,964][20347] Updated weights for policy 0, policy_version 1612881 (0.0007) [2023-12-27 16:49:12,968][20381] Updated weights for policy 1, policy_version 1611319 (0.0008) [2023-12-27 16:49:13,017][20347] Updated weights for policy 0, policy_version 1612891 (0.0005) [2023-12-27 16:49:13,064][20347] Updated weights for policy 0, policy_version 1612901 (0.0005) [2023-12-27 16:49:13,120][20347] Updated weights for policy 0, policy_version 1612911 (0.0005) [2023-12-27 16:49:13,635][20381] Updated weights for policy 1, policy_version 1611329 (0.0006) [2023-12-27 16:49:13,694][20381] Updated weights for policy 1, policy_version 1611339 (0.0006) [2023-12-27 16:49:13,746][20381] Updated weights for policy 1, policy_version 1611349 (0.0005) [2023-12-27 16:49:13,752][20347] Updated weights for policy 0, policy_version 1612921 (0.0007) [2023-12-27 16:49:13,806][20347] Updated weights for policy 0, policy_version 1612931 (0.0007) [2023-12-27 16:49:13,860][20347] Updated weights for policy 0, policy_version 1612941 (0.0007) [2023-12-27 16:49:14,397][20381] Updated weights for policy 1, policy_version 1611359 (0.0006) [2023-12-27 16:49:14,446][20381] Updated weights for policy 1, policy_version 1611369 (0.0006) [2023-12-27 16:49:14,500][20381] Updated weights for policy 1, policy_version 1611379 (0.0006) [2023-12-27 16:49:14,513][20347] Updated weights for policy 0, policy_version 1612951 (0.0009) [2023-12-27 16:49:14,564][20347] Updated weights for policy 0, policy_version 1612961 (0.0010) [2023-12-27 16:49:14,612][20347] Updated weights for policy 0, policy_version 1612971 (0.0010) [2023-12-27 16:49:15,187][20381] Updated weights for policy 1, policy_version 1611389 (0.0006) [2023-12-27 16:49:15,245][20381] Updated weights for policy 1, policy_version 1611399 (0.0007) [2023-12-27 16:49:15,308][20381] Updated weights for policy 1, policy_version 1611409 (0.0007) [2023-12-27 16:49:15,322][20347] Updated weights for policy 0, policy_version 1612981 (0.0010) [2023-12-27 16:49:15,371][20347] Updated weights for policy 0, policy_version 1612991 (0.0010) [2023-12-27 16:49:15,420][20347] Updated weights for policy 0, policy_version 1613001 (0.0010) [2023-12-27 16:49:16,011][20381] Updated weights for policy 1, policy_version 1611419 (0.0006) [2023-12-27 16:49:16,064][20381] Updated weights for policy 1, policy_version 1611429 (0.0008) [2023-12-27 16:49:16,066][20347] Updated weights for policy 0, policy_version 1613011 (0.0007) [2023-12-27 16:49:16,116][20381] Updated weights for policy 1, policy_version 1611439 (0.0005) [2023-12-27 16:49:16,122][20347] Updated weights for policy 0, policy_version 1613021 (0.0005) [2023-12-27 16:49:16,175][20347] Updated weights for policy 0, policy_version 1613031 (0.0006) [2023-12-27 16:49:16,815][20381] Updated weights for policy 1, policy_version 1611449 (0.0006) [2023-12-27 16:49:16,850][20347] Updated weights for policy 0, policy_version 1613041 (0.0008) [2023-12-27 16:49:16,864][20381] Updated weights for policy 1, policy_version 1611459 (0.0007) [2023-12-27 16:49:16,907][20347] Updated weights for policy 0, policy_version 1613051 (0.0010) [2023-12-27 16:49:16,919][20381] Updated weights for policy 1, policy_version 1611469 (0.0006) [2023-12-27 16:49:16,966][20381] Updated weights for policy 1, policy_version 1611479 (0.0007) [2023-12-27 16:49:16,967][20347] Updated weights for policy 0, policy_version 1613061 (0.0008) [2023-12-27 16:49:17,029][20347] Updated weights for policy 0, policy_version 1613071 (0.0007) [2023-12-27 16:49:17,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 825606144. Throughput: 0: 10170.1, 1: 10255.9. Samples: 825615264. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:17,363][19170] Avg episode reward: [(0, '990.240'), (1, '814.540')] [2023-12-27 16:49:17,643][20347] Updated weights for policy 0, policy_version 1613081 (0.0005) [2023-12-27 16:49:17,692][20347] Updated weights for policy 0, policy_version 1613091 (0.0005) [2023-12-27 16:49:17,694][20381] Updated weights for policy 1, policy_version 1611489 (0.0007) [2023-12-27 16:49:17,744][20347] Updated weights for policy 0, policy_version 1613101 (0.0007) [2023-12-27 16:49:17,750][20381] Updated weights for policy 1, policy_version 1611499 (0.0007) [2023-12-27 16:49:17,815][20381] Updated weights for policy 1, policy_version 1611509 (0.0007) [2023-12-27 16:49:18,274][20347] Updated weights for policy 0, policy_version 1613111 (0.0005) [2023-12-27 16:49:18,332][20347] Updated weights for policy 0, policy_version 1613121 (0.0006) [2023-12-27 16:49:18,397][20347] Updated weights for policy 0, policy_version 1613131 (0.0007) [2023-12-27 16:49:18,451][20381] Updated weights for policy 1, policy_version 1611519 (0.0008) [2023-12-27 16:49:18,505][20381] Updated weights for policy 1, policy_version 1611529 (0.0005) [2023-12-27 16:49:18,553][20381] Updated weights for policy 1, policy_version 1611539 (0.0006) [2023-12-27 16:49:19,038][20347] Updated weights for policy 0, policy_version 1613141 (0.0008) [2023-12-27 16:49:19,099][20347] Updated weights for policy 0, policy_version 1613151 (0.0008) [2023-12-27 16:49:19,161][20347] Updated weights for policy 0, policy_version 1613161 (0.0007) [2023-12-27 16:49:19,291][20381] Updated weights for policy 1, policy_version 1611549 (0.0006) [2023-12-27 16:49:19,321][20292] KL-divergence is very high: 106.5585 [2023-12-27 16:49:19,335][20292] KL-divergence is very high: 132.0315 [2023-12-27 16:49:19,347][20381] Updated weights for policy 1, policy_version 1611559 (0.0009) [2023-12-27 16:49:19,363][20292] KL-divergence is very high: 111.1858 [2023-12-27 16:49:19,379][20292] KL-divergence is very high: 106.0323 [2023-12-27 16:49:19,396][20292] KL-divergence is very high: 105.2672 [2023-12-27 16:49:19,421][20381] Updated weights for policy 1, policy_version 1611569 (0.0008) [2023-12-27 16:49:19,916][20347] Updated weights for policy 0, policy_version 1613171 (0.0006) [2023-12-27 16:49:19,982][20347] Updated weights for policy 0, policy_version 1613181 (0.0007) [2023-12-27 16:49:20,041][20347] Updated weights for policy 0, policy_version 1613191 (0.0008) [2023-12-27 16:49:20,180][20381] Updated weights for policy 1, policy_version 1611579 (0.0007) [2023-12-27 16:49:20,236][20381] Updated weights for policy 1, policy_version 1611589 (0.0008) [2023-12-27 16:49:20,286][20381] Updated weights for policy 1, policy_version 1611599 (0.0005) [2023-12-27 16:49:20,707][20347] Updated weights for policy 0, policy_version 1613201 (0.0008) [2023-12-27 16:49:20,768][20347] Updated weights for policy 0, policy_version 1613211 (0.0008) [2023-12-27 16:49:20,826][20347] Updated weights for policy 0, policy_version 1613221 (0.0007) [2023-12-27 16:49:20,886][20347] Updated weights for policy 0, policy_version 1613231 (0.0008) [2023-12-27 16:49:20,953][20381] Updated weights for policy 1, policy_version 1611609 (0.0006) [2023-12-27 16:49:21,019][20381] Updated weights for policy 1, policy_version 1611619 (0.0006) [2023-12-27 16:49:21,086][20381] Updated weights for policy 1, policy_version 1611629 (0.0007) [2023-12-27 16:49:21,148][20381] Updated weights for policy 1, policy_version 1611639 (0.0006) [2023-12-27 16:49:21,668][20347] Updated weights for policy 0, policy_version 1613241 (0.0009) [2023-12-27 16:49:21,735][20347] Updated weights for policy 0, policy_version 1613251 (0.0008) [2023-12-27 16:49:21,794][20347] Updated weights for policy 0, policy_version 1613261 (0.0008) [2023-12-27 16:49:21,829][20381] Updated weights for policy 1, policy_version 1611649 (0.0010) [2023-12-27 16:49:21,891][20381] Updated weights for policy 1, policy_version 1611659 (0.0008) [2023-12-27 16:49:21,956][20381] Updated weights for policy 1, policy_version 1611669 (0.0005) [2023-12-27 16:49:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 825704448. Throughput: 0: 10180.5, 1: 10211.1. Samples: 825676804. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:22,363][19170] Avg episode reward: [(0, '990.240'), (1, '775.730')] [2023-12-27 16:49:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001611672_412631040.pth... [2023-12-27 16:49:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001613264_413073408.pth... [2023-12-27 16:49:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001610488_412327936.pth [2023-12-27 16:49:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001612080_412770304.pth [2023-12-27 16:49:22,513][20347] Updated weights for policy 0, policy_version 1613271 (0.0007) [2023-12-27 16:49:22,569][20347] Updated weights for policy 0, policy_version 1613281 (0.0008) [2023-12-27 16:49:22,595][20381] Updated weights for policy 1, policy_version 1611679 (0.0008) [2023-12-27 16:49:22,623][20347] Updated weights for policy 0, policy_version 1613291 (0.0007) [2023-12-27 16:49:22,656][20381] Updated weights for policy 1, policy_version 1611689 (0.0008) [2023-12-27 16:49:22,717][20381] Updated weights for policy 1, policy_version 1611699 (0.0006) [2023-12-27 16:49:23,329][20347] Updated weights for policy 0, policy_version 1613301 (0.0006) [2023-12-27 16:49:23,369][20381] Updated weights for policy 1, policy_version 1611709 (0.0011) [2023-12-27 16:49:23,393][20347] Updated weights for policy 0, policy_version 1613311 (0.0006) [2023-12-27 16:49:23,424][20381] Updated weights for policy 1, policy_version 1611719 (0.0010) [2023-12-27 16:49:23,446][20347] Updated weights for policy 0, policy_version 1613321 (0.0009) [2023-12-27 16:49:23,479][20381] Updated weights for policy 1, policy_version 1611729 (0.0009) [2023-12-27 16:49:23,964][20347] Updated weights for policy 0, policy_version 1613331 (0.0006) [2023-12-27 16:49:24,021][20347] Updated weights for policy 0, policy_version 1613341 (0.0008) [2023-12-27 16:49:24,079][20347] Updated weights for policy 0, policy_version 1613351 (0.0005) [2023-12-27 16:49:24,143][20381] Updated weights for policy 1, policy_version 1611739 (0.0009) [2023-12-27 16:49:24,200][20381] Updated weights for policy 1, policy_version 1611749 (0.0011) [2023-12-27 16:49:24,255][20381] Updated weights for policy 1, policy_version 1611759 (0.0010) [2023-12-27 16:49:24,685][20347] Updated weights for policy 0, policy_version 1613361 (0.0008) [2023-12-27 16:49:24,738][20347] Updated weights for policy 0, policy_version 1613371 (0.0010) [2023-12-27 16:49:24,785][20347] Updated weights for policy 0, policy_version 1613381 (0.0010) [2023-12-27 16:49:24,833][20347] Updated weights for policy 0, policy_version 1613391 (0.0005) [2023-12-27 16:49:24,863][20381] Updated weights for policy 1, policy_version 1611769 (0.0010) [2023-12-27 16:49:24,912][20381] Updated weights for policy 1, policy_version 1611779 (0.0005) [2023-12-27 16:49:24,966][20381] Updated weights for policy 1, policy_version 1611789 (0.0005) [2023-12-27 16:49:25,027][20381] Updated weights for policy 1, policy_version 1611799 (0.0005) [2023-12-27 16:49:25,458][20347] Updated weights for policy 0, policy_version 1613401 (0.0010) [2023-12-27 16:49:25,511][20347] Updated weights for policy 0, policy_version 1613411 (0.0006) [2023-12-27 16:49:25,567][20347] Updated weights for policy 0, policy_version 1613421 (0.0005) [2023-12-27 16:49:25,590][20381] Updated weights for policy 1, policy_version 1611809 (0.0006) [2023-12-27 16:49:25,648][20381] Updated weights for policy 1, policy_version 1611819 (0.0010) [2023-12-27 16:49:25,697][20381] Updated weights for policy 1, policy_version 1611829 (0.0010) [2023-12-27 16:49:26,164][20347] Updated weights for policy 0, policy_version 1613431 (0.0007) [2023-12-27 16:49:26,230][20347] Updated weights for policy 0, policy_version 1613441 (0.0005) [2023-12-27 16:49:26,287][20347] Updated weights for policy 0, policy_version 1613451 (0.0008) [2023-12-27 16:49:26,359][20381] Updated weights for policy 1, policy_version 1611839 (0.0010) [2023-12-27 16:49:26,410][20381] Updated weights for policy 1, policy_version 1611849 (0.0010) [2023-12-27 16:49:26,459][20381] Updated weights for policy 1, policy_version 1611859 (0.0006) [2023-12-27 16:49:26,870][20347] Updated weights for policy 0, policy_version 1613461 (0.0008) [2023-12-27 16:49:26,931][20347] Updated weights for policy 0, policy_version 1613471 (0.0010) [2023-12-27 16:49:26,990][20347] Updated weights for policy 0, policy_version 1613481 (0.0010) [2023-12-27 16:49:27,052][20381] Updated weights for policy 1, policy_version 1611869 (0.0008) [2023-12-27 16:49:27,105][20381] Updated weights for policy 1, policy_version 1611879 (0.0006) [2023-12-27 16:49:27,151][20381] Updated weights for policy 1, policy_version 1611889 (0.0005) [2023-12-27 16:49:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.4, 300 sec: 20521.6). Total num frames: 825819136. Throughput: 0: 10204.4, 1: 10258.8. Samples: 825803928. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:27,364][19170] Avg episode reward: [(0, '980.580'), (1, '794.400')] [2023-12-27 16:49:27,574][20347] Updated weights for policy 0, policy_version 1613491 (0.0008) [2023-12-27 16:49:27,632][20347] Updated weights for policy 0, policy_version 1613501 (0.0005) [2023-12-27 16:49:27,687][20347] Updated weights for policy 0, policy_version 1613511 (0.0005) [2023-12-27 16:49:27,785][20381] Updated weights for policy 1, policy_version 1611899 (0.0007) [2023-12-27 16:49:27,832][20381] Updated weights for policy 1, policy_version 1611909 (0.0010) [2023-12-27 16:49:27,883][20381] Updated weights for policy 1, policy_version 1611919 (0.0010) [2023-12-27 16:49:28,219][20347] Updated weights for policy 0, policy_version 1613521 (0.0006) [2023-12-27 16:49:28,274][20347] Updated weights for policy 0, policy_version 1613531 (0.0010) [2023-12-27 16:49:28,341][20347] Updated weights for policy 0, policy_version 1613541 (0.0010) [2023-12-27 16:49:28,398][20347] Updated weights for policy 0, policy_version 1613551 (0.0007) [2023-12-27 16:49:28,608][20381] Updated weights for policy 1, policy_version 1611929 (0.0008) [2023-12-27 16:49:28,664][20381] Updated weights for policy 1, policy_version 1611939 (0.0009) [2023-12-27 16:49:28,719][20381] Updated weights for policy 1, policy_version 1611949 (0.0011) [2023-12-27 16:49:28,776][20381] Updated weights for policy 1, policy_version 1611959 (0.0011) [2023-12-27 16:49:29,035][20347] Updated weights for policy 0, policy_version 1613561 (0.0010) [2023-12-27 16:49:29,081][20347] Updated weights for policy 0, policy_version 1613571 (0.0010) [2023-12-27 16:49:29,126][20347] Updated weights for policy 0, policy_version 1613581 (0.0009) [2023-12-27 16:49:29,423][20381] Updated weights for policy 1, policy_version 1611969 (0.0011) [2023-12-27 16:49:29,488][20381] Updated weights for policy 1, policy_version 1611979 (0.0011) [2023-12-27 16:49:29,546][20381] Updated weights for policy 1, policy_version 1611989 (0.0010) [2023-12-27 16:49:29,861][20347] Updated weights for policy 0, policy_version 1613591 (0.0010) [2023-12-27 16:49:29,919][20347] Updated weights for policy 0, policy_version 1613601 (0.0010) [2023-12-27 16:49:29,978][20347] Updated weights for policy 0, policy_version 1613611 (0.0009) [2023-12-27 16:49:30,217][20381] Updated weights for policy 1, policy_version 1611999 (0.0010) [2023-12-27 16:49:30,275][20381] Updated weights for policy 1, policy_version 1612009 (0.0007) [2023-12-27 16:49:30,326][20381] Updated weights for policy 1, policy_version 1612019 (0.0007) [2023-12-27 16:49:30,628][20347] Updated weights for policy 0, policy_version 1613621 (0.0009) [2023-12-27 16:49:30,690][20347] Updated weights for policy 0, policy_version 1613631 (0.0005) [2023-12-27 16:49:30,753][20347] Updated weights for policy 0, policy_version 1613641 (0.0005) [2023-12-27 16:49:30,929][20381] Updated weights for policy 1, policy_version 1612029 (0.0006) [2023-12-27 16:49:30,972][20381] Updated weights for policy 1, policy_version 1612039 (0.0005) [2023-12-27 16:49:31,023][20381] Updated weights for policy 1, policy_version 1612049 (0.0006) [2023-12-27 16:49:31,313][20347] Updated weights for policy 0, policy_version 1613651 (0.0006) [2023-12-27 16:49:31,381][20347] Updated weights for policy 0, policy_version 1613661 (0.0007) [2023-12-27 16:49:31,447][20347] Updated weights for policy 0, policy_version 1613671 (0.0009) [2023-12-27 16:49:31,746][20381] Updated weights for policy 1, policy_version 1612059 (0.0008) [2023-12-27 16:49:31,810][20381] Updated weights for policy 1, policy_version 1612069 (0.0005) [2023-12-27 16:49:31,872][20381] Updated weights for policy 1, policy_version 1612079 (0.0005) [2023-12-27 16:49:32,222][20347] Updated weights for policy 0, policy_version 1613681 (0.0009) [2023-12-27 16:49:32,278][20347] Updated weights for policy 0, policy_version 1613691 (0.0008) [2023-12-27 16:49:32,340][20347] Updated weights for policy 0, policy_version 1613701 (0.0008) [2023-12-27 16:49:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 825917440. Throughput: 0: 10383.4, 1: 10204.8. Samples: 825932768. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:32,363][19170] Avg episode reward: [(0, '971.000'), (1, '814.570')] [2023-12-27 16:49:32,409][20347] Updated weights for policy 0, policy_version 1613711 (0.0008) [2023-12-27 16:49:32,463][20381] Updated weights for policy 1, policy_version 1612089 (0.0005) [2023-12-27 16:49:32,517][20381] Updated weights for policy 1, policy_version 1612099 (0.0005) [2023-12-27 16:49:32,572][20381] Updated weights for policy 1, policy_version 1612109 (0.0006) [2023-12-27 16:49:32,635][20381] Updated weights for policy 1, policy_version 1612119 (0.0006) [2023-12-27 16:49:33,088][20347] Updated weights for policy 0, policy_version 1613721 (0.0009) [2023-12-27 16:49:33,140][20347] Updated weights for policy 0, policy_version 1613731 (0.0010) [2023-12-27 16:49:33,194][20347] Updated weights for policy 0, policy_version 1613741 (0.0010) [2023-12-27 16:49:33,256][20381] Updated weights for policy 1, policy_version 1612129 (0.0008) [2023-12-27 16:49:33,308][20381] Updated weights for policy 1, policy_version 1612139 (0.0007) [2023-12-27 16:49:33,361][20381] Updated weights for policy 1, policy_version 1612149 (0.0006) [2023-12-27 16:49:33,870][20347] Updated weights for policy 0, policy_version 1613751 (0.0010) [2023-12-27 16:49:33,924][20347] Updated weights for policy 0, policy_version 1613761 (0.0010) [2023-12-27 16:49:33,978][20347] Updated weights for policy 0, policy_version 1613771 (0.0010) [2023-12-27 16:49:33,979][20381] Updated weights for policy 1, policy_version 1612159 (0.0005) [2023-12-27 16:49:34,032][20381] Updated weights for policy 1, policy_version 1612169 (0.0006) [2023-12-27 16:49:34,085][20381] Updated weights for policy 1, policy_version 1612179 (0.0007) [2023-12-27 16:49:34,742][20347] Updated weights for policy 0, policy_version 1613781 (0.0011) [2023-12-27 16:49:34,798][20347] Updated weights for policy 0, policy_version 1613791 (0.0010) [2023-12-27 16:49:34,837][20381] Updated weights for policy 1, policy_version 1612189 (0.0007) [2023-12-27 16:49:34,849][20347] Updated weights for policy 0, policy_version 1613801 (0.0010) [2023-12-27 16:49:34,900][20381] Updated weights for policy 1, policy_version 1612199 (0.0005) [2023-12-27 16:49:34,965][20381] Updated weights for policy 1, policy_version 1612209 (0.0005) [2023-12-27 16:49:35,473][20347] Updated weights for policy 0, policy_version 1613811 (0.0009) [2023-12-27 16:49:35,535][20347] Updated weights for policy 0, policy_version 1613821 (0.0008) [2023-12-27 16:49:35,591][20347] Updated weights for policy 0, policy_version 1613831 (0.0005) [2023-12-27 16:49:35,621][20381] Updated weights for policy 1, policy_version 1612219 (0.0005) [2023-12-27 16:49:35,668][20381] Updated weights for policy 1, policy_version 1612229 (0.0008) [2023-12-27 16:49:35,714][20381] Updated weights for policy 1, policy_version 1612239 (0.0010) [2023-12-27 16:49:36,262][20347] Updated weights for policy 0, policy_version 1613841 (0.0006) [2023-12-27 16:49:36,324][20347] Updated weights for policy 0, policy_version 1613851 (0.0008) [2023-12-27 16:49:36,355][20381] Updated weights for policy 1, policy_version 1612249 (0.0008) [2023-12-27 16:49:36,380][20347] Updated weights for policy 0, policy_version 1613861 (0.0006) [2023-12-27 16:49:36,410][20381] Updated weights for policy 1, policy_version 1612259 (0.0010) [2023-12-27 16:49:36,441][20347] Updated weights for policy 0, policy_version 1613871 (0.0007) [2023-12-27 16:49:36,463][20381] Updated weights for policy 1, policy_version 1612269 (0.0011) [2023-12-27 16:49:36,514][20381] Updated weights for policy 1, policy_version 1612279 (0.0011) [2023-12-27 16:49:37,111][20347] Updated weights for policy 0, policy_version 1613881 (0.0007) [2023-12-27 16:49:37,165][20347] Updated weights for policy 0, policy_version 1613891 (0.0008) [2023-12-27 16:49:37,205][20381] Updated weights for policy 1, policy_version 1612289 (0.0008) [2023-12-27 16:49:37,215][20347] Updated weights for policy 0, policy_version 1613901 (0.0008) [2023-12-27 16:49:37,254][20381] Updated weights for policy 1, policy_version 1612299 (0.0006) [2023-12-27 16:49:37,303][20381] Updated weights for policy 1, policy_version 1612309 (0.0010) [2023-12-27 16:49:37,363][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 826032128. Throughput: 0: 10394.4, 1: 10230.5. Samples: 825993352. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:37,364][19170] Avg episode reward: [(0, '980.660'), (1, '805.510')] [2023-12-27 16:49:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001613904_413237248.pth... [2023-12-27 16:49:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001612312_412794880.pth... [2023-12-27 16:49:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001612656_412917760.pth [2023-12-27 16:49:37,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001611096_412483584.pth [2023-12-27 16:49:37,933][20381] Updated weights for policy 1, policy_version 1612319 (0.0007) [2023-12-27 16:49:37,934][20347] Updated weights for policy 0, policy_version 1613911 (0.0006) [2023-12-27 16:49:37,985][20347] Updated weights for policy 0, policy_version 1613921 (0.0005) [2023-12-27 16:49:37,991][20381] Updated weights for policy 1, policy_version 1612329 (0.0005) [2023-12-27 16:49:38,043][20381] Updated weights for policy 1, policy_version 1612339 (0.0005) [2023-12-27 16:49:38,046][20347] Updated weights for policy 0, policy_version 1613931 (0.0006) [2023-12-27 16:49:38,612][20381] Updated weights for policy 1, policy_version 1612349 (0.0008) [2023-12-27 16:49:38,673][20381] Updated weights for policy 1, policy_version 1612359 (0.0010) [2023-12-27 16:49:38,683][20347] Updated weights for policy 0, policy_version 1613941 (0.0006) [2023-12-27 16:49:38,733][20381] Updated weights for policy 1, policy_version 1612369 (0.0011) [2023-12-27 16:49:38,745][20347] Updated weights for policy 0, policy_version 1613951 (0.0006) [2023-12-27 16:49:38,802][20347] Updated weights for policy 0, policy_version 1613961 (0.0007) [2023-12-27 16:49:39,420][20381] Updated weights for policy 1, policy_version 1612379 (0.0010) [2023-12-27 16:49:39,473][20381] Updated weights for policy 1, policy_version 1612389 (0.0006) [2023-12-27 16:49:39,529][20381] Updated weights for policy 1, policy_version 1612399 (0.0005) [2023-12-27 16:49:39,555][20347] Updated weights for policy 0, policy_version 1613971 (0.0008) [2023-12-27 16:49:39,614][20347] Updated weights for policy 0, policy_version 1613981 (0.0008) [2023-12-27 16:49:39,677][20347] Updated weights for policy 0, policy_version 1613991 (0.0008) [2023-12-27 16:49:40,270][20381] Updated weights for policy 1, policy_version 1612409 (0.0007) [2023-12-27 16:49:40,331][20381] Updated weights for policy 1, policy_version 1612419 (0.0006) [2023-12-27 16:49:40,361][20347] Updated weights for policy 0, policy_version 1614001 (0.0011) [2023-12-27 16:49:40,394][20381] Updated weights for policy 1, policy_version 1612429 (0.0006) [2023-12-27 16:49:40,421][20347] Updated weights for policy 0, policy_version 1614011 (0.0008) [2023-12-27 16:49:40,453][20381] Updated weights for policy 1, policy_version 1612439 (0.0008) [2023-12-27 16:49:40,482][20347] Updated weights for policy 0, policy_version 1614021 (0.0006) [2023-12-27 16:49:40,533][20347] Updated weights for policy 0, policy_version 1614031 (0.0009) [2023-12-27 16:49:41,118][20381] Updated weights for policy 1, policy_version 1612449 (0.0008) [2023-12-27 16:49:41,189][20381] Updated weights for policy 1, policy_version 1612459 (0.0008) [2023-12-27 16:49:41,253][20381] Updated weights for policy 1, policy_version 1612469 (0.0008) [2023-12-27 16:49:41,285][20347] Updated weights for policy 0, policy_version 1614041 (0.0010) [2023-12-27 16:49:41,354][20347] Updated weights for policy 0, policy_version 1614051 (0.0008) [2023-12-27 16:49:41,422][20347] Updated weights for policy 0, policy_version 1614061 (0.0008) [2023-12-27 16:49:41,994][20381] Updated weights for policy 1, policy_version 1612479 (0.0008) [2023-12-27 16:49:42,049][20381] Updated weights for policy 1, policy_version 1612489 (0.0008) [2023-12-27 16:49:42,107][20381] Updated weights for policy 1, policy_version 1612499 (0.0006) [2023-12-27 16:49:42,123][20347] Updated weights for policy 0, policy_version 1614071 (0.0010) [2023-12-27 16:49:42,185][20347] Updated weights for policy 0, policy_version 1614081 (0.0010) [2023-12-27 16:49:42,245][20347] Updated weights for policy 0, policy_version 1614091 (0.0010) [2023-12-27 16:49:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 826130432. Throughput: 0: 10362.3, 1: 10286.0. Samples: 826117200. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.480')] [2023-12-27 16:49:42,856][20381] Updated weights for policy 1, policy_version 1612509 (0.0008) [2023-12-27 16:49:42,864][20347] Updated weights for policy 0, policy_version 1614101 (0.0008) [2023-12-27 16:49:42,912][20381] Updated weights for policy 1, policy_version 1612519 (0.0007) [2023-12-27 16:49:42,918][20347] Updated weights for policy 0, policy_version 1614111 (0.0005) [2023-12-27 16:49:42,968][20347] Updated weights for policy 0, policy_version 1614121 (0.0005) [2023-12-27 16:49:42,972][20381] Updated weights for policy 1, policy_version 1612529 (0.0008) [2023-12-27 16:49:43,560][20381] Updated weights for policy 1, policy_version 1612539 (0.0006) [2023-12-27 16:49:43,614][20381] Updated weights for policy 1, policy_version 1612549 (0.0005) [2023-12-27 16:49:43,631][20347] Updated weights for policy 0, policy_version 1614131 (0.0007) [2023-12-27 16:49:43,664][20381] Updated weights for policy 1, policy_version 1612559 (0.0005) [2023-12-27 16:49:43,683][20347] Updated weights for policy 0, policy_version 1614141 (0.0010) [2023-12-27 16:49:43,748][20347] Updated weights for policy 0, policy_version 1614151 (0.0010) [2023-12-27 16:49:44,189][20381] Updated weights for policy 1, policy_version 1612569 (0.0006) [2023-12-27 16:49:44,240][20381] Updated weights for policy 1, policy_version 1612579 (0.0007) [2023-12-27 16:49:44,297][20381] Updated weights for policy 1, policy_version 1612589 (0.0008) [2023-12-27 16:49:44,345][20381] Updated weights for policy 1, policy_version 1612599 (0.0008) [2023-12-27 16:49:44,429][20347] Updated weights for policy 0, policy_version 1614161 (0.0010) [2023-12-27 16:49:44,471][20347] Updated weights for policy 0, policy_version 1614171 (0.0010) [2023-12-27 16:49:44,515][20347] Updated weights for policy 0, policy_version 1614181 (0.0010) [2023-12-27 16:49:44,570][20347] Updated weights for policy 0, policy_version 1614191 (0.0010) [2023-12-27 16:49:44,989][20381] Updated weights for policy 1, policy_version 1612609 (0.0005) [2023-12-27 16:49:45,050][20381] Updated weights for policy 1, policy_version 1612619 (0.0006) [2023-12-27 16:49:45,114][20381] Updated weights for policy 1, policy_version 1612629 (0.0006) [2023-12-27 16:49:45,296][20347] Updated weights for policy 0, policy_version 1614201 (0.0010) [2023-12-27 16:49:45,346][20347] Updated weights for policy 0, policy_version 1614211 (0.0010) [2023-12-27 16:49:45,392][20347] Updated weights for policy 0, policy_version 1614221 (0.0010) [2023-12-27 16:49:45,744][20381] Updated weights for policy 1, policy_version 1612639 (0.0009) [2023-12-27 16:49:45,789][20381] Updated weights for policy 1, policy_version 1612649 (0.0010) [2023-12-27 16:49:45,841][20381] Updated weights for policy 1, policy_version 1612659 (0.0010) [2023-12-27 16:49:46,095][20347] Updated weights for policy 0, policy_version 1614231 (0.0010) [2023-12-27 16:49:46,147][20347] Updated weights for policy 0, policy_version 1614241 (0.0010) [2023-12-27 16:49:46,194][20347] Updated weights for policy 0, policy_version 1614251 (0.0010) [2023-12-27 16:49:46,556][20381] Updated weights for policy 1, policy_version 1612669 (0.0010) [2023-12-27 16:49:46,614][20381] Updated weights for policy 1, policy_version 1612679 (0.0010) [2023-12-27 16:49:46,672][20381] Updated weights for policy 1, policy_version 1612689 (0.0010) [2023-12-27 16:49:46,865][20347] Updated weights for policy 0, policy_version 1614261 (0.0008) [2023-12-27 16:49:46,916][20347] Updated weights for policy 0, policy_version 1614271 (0.0005) [2023-12-27 16:49:46,965][20347] Updated weights for policy 0, policy_version 1614281 (0.0005) [2023-12-27 16:49:47,268][20381] Updated weights for policy 1, policy_version 1612699 (0.0008) [2023-12-27 16:49:47,324][20381] Updated weights for policy 1, policy_version 1612709 (0.0008) [2023-12-27 16:49:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 826228736. Throughput: 0: 10383.4, 1: 10336.3. Samples: 826242776. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '814.160')] [2023-12-27 16:49:47,388][20381] Updated weights for policy 1, policy_version 1612719 (0.0008) [2023-12-27 16:49:47,501][20347] Updated weights for policy 0, policy_version 1614291 (0.0005) [2023-12-27 16:49:47,557][20347] Updated weights for policy 0, policy_version 1614301 (0.0005) [2023-12-27 16:49:47,610][20347] Updated weights for policy 0, policy_version 1614311 (0.0007) [2023-12-27 16:49:48,120][20381] Updated weights for policy 1, policy_version 1612729 (0.0007) [2023-12-27 16:49:48,181][20381] Updated weights for policy 1, policy_version 1612739 (0.0007) [2023-12-27 16:49:48,237][20347] Updated weights for policy 0, policy_version 1614321 (0.0010) [2023-12-27 16:49:48,246][20381] Updated weights for policy 1, policy_version 1612749 (0.0007) [2023-12-27 16:49:48,298][20347] Updated weights for policy 0, policy_version 1614331 (0.0010) [2023-12-27 16:49:48,304][20381] Updated weights for policy 1, policy_version 1612759 (0.0008) [2023-12-27 16:49:48,362][20347] Updated weights for policy 0, policy_version 1614341 (0.0009) [2023-12-27 16:49:48,425][20347] Updated weights for policy 0, policy_version 1614351 (0.0007) [2023-12-27 16:49:48,972][20381] Updated weights for policy 1, policy_version 1612769 (0.0007) [2023-12-27 16:49:49,029][20381] Updated weights for policy 1, policy_version 1612779 (0.0008) [2023-12-27 16:49:49,084][20381] Updated weights for policy 1, policy_version 1612789 (0.0008) [2023-12-27 16:49:49,119][20347] Updated weights for policy 0, policy_version 1614361 (0.0006) [2023-12-27 16:49:49,174][20347] Updated weights for policy 0, policy_version 1614371 (0.0008) [2023-12-27 16:49:49,230][20347] Updated weights for policy 0, policy_version 1614381 (0.0009) [2023-12-27 16:49:49,835][20381] Updated weights for policy 1, policy_version 1612799 (0.0008) [2023-12-27 16:49:49,854][20347] Updated weights for policy 0, policy_version 1614391 (0.0010) [2023-12-27 16:49:49,897][20381] Updated weights for policy 1, policy_version 1612809 (0.0007) [2023-12-27 16:49:49,911][20347] Updated weights for policy 0, policy_version 1614401 (0.0007) [2023-12-27 16:49:49,960][20381] Updated weights for policy 1, policy_version 1612819 (0.0006) [2023-12-27 16:49:49,972][20347] Updated weights for policy 0, policy_version 1614411 (0.0008) [2023-12-27 16:49:50,622][20347] Updated weights for policy 0, policy_version 1614421 (0.0005) [2023-12-27 16:49:50,632][20381] Updated weights for policy 1, policy_version 1612829 (0.0008) [2023-12-27 16:49:50,674][20347] Updated weights for policy 0, policy_version 1614431 (0.0007) [2023-12-27 16:49:50,688][20381] Updated weights for policy 1, policy_version 1612839 (0.0006) [2023-12-27 16:49:50,722][20292] KL-divergence is very high: 134.2356 [2023-12-27 16:49:50,725][20347] Updated weights for policy 0, policy_version 1614441 (0.0006) [2023-12-27 16:49:50,743][20381] Updated weights for policy 1, policy_version 1612849 (0.0007) [2023-12-27 16:49:50,748][20292] KL-divergence is very high: 176.7039 [2023-12-27 16:49:50,763][20292] KL-divergence is very high: 150.1727 [2023-12-27 16:49:51,379][20347] Updated weights for policy 0, policy_version 1614451 (0.0006) [2023-12-27 16:49:51,442][20347] Updated weights for policy 0, policy_version 1614461 (0.0009) [2023-12-27 16:49:51,502][20347] Updated weights for policy 0, policy_version 1614471 (0.0008) [2023-12-27 16:49:51,510][20292] KL-divergence is very high: 113.3945 [2023-12-27 16:49:51,529][20381] Updated weights for policy 1, policy_version 1612859 (0.0008) [2023-12-27 16:49:51,584][20381] Updated weights for policy 1, policy_version 1612869 (0.0006) [2023-12-27 16:49:51,650][20381] Updated weights for policy 1, policy_version 1612879 (0.0008) [2023-12-27 16:49:52,212][20347] Updated weights for policy 0, policy_version 1614481 (0.0007) [2023-12-27 16:49:52,279][20347] Updated weights for policy 0, policy_version 1614491 (0.0007) [2023-12-27 16:49:52,343][20347] Updated weights for policy 0, policy_version 1614501 (0.0008) [2023-12-27 16:49:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 826327040. Throughput: 0: 10453.1, 1: 10260.9. Samples: 826303808. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '786.060')] [2023-12-27 16:49:52,375][20381] Updated weights for policy 1, policy_version 1612889 (0.0008) [2023-12-27 16:49:52,409][20347] Updated weights for policy 0, policy_version 1614511 (0.0009) [2023-12-27 16:49:52,414][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001614512_413392896.pth... [2023-12-27 16:49:52,417][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001613264_413073408.pth [2023-12-27 16:49:52,432][20381] Updated weights for policy 1, policy_version 1612899 (0.0006) [2023-12-27 16:49:52,494][20381] Updated weights for policy 1, policy_version 1612909 (0.0006) [2023-12-27 16:49:52,549][20381] Updated weights for policy 1, policy_version 1612919 (0.0009) [2023-12-27 16:49:52,554][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001612920_412950528.pth... [2023-12-27 16:49:52,557][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001611672_412631040.pth [2023-12-27 16:49:53,047][20347] Updated weights for policy 0, policy_version 1614521 (0.0005) [2023-12-27 16:49:53,096][20347] Updated weights for policy 0, policy_version 1614531 (0.0005) [2023-12-27 16:49:53,143][20347] Updated weights for policy 0, policy_version 1614541 (0.0007) [2023-12-27 16:49:53,251][20381] Updated weights for policy 1, policy_version 1612929 (0.0009) [2023-12-27 16:49:53,309][20381] Updated weights for policy 1, policy_version 1612939 (0.0010) [2023-12-27 16:49:53,364][20381] Updated weights for policy 1, policy_version 1612949 (0.0010) [2023-12-27 16:49:53,827][20347] Updated weights for policy 0, policy_version 1614551 (0.0007) [2023-12-27 16:49:53,880][20347] Updated weights for policy 0, policy_version 1614561 (0.0007) [2023-12-27 16:49:53,937][20347] Updated weights for policy 0, policy_version 1614571 (0.0007) [2023-12-27 16:49:54,056][20381] Updated weights for policy 1, policy_version 1612959 (0.0010) [2023-12-27 16:49:54,113][20381] Updated weights for policy 1, policy_version 1612969 (0.0010) [2023-12-27 16:49:54,171][20381] Updated weights for policy 1, policy_version 1612979 (0.0009) [2023-12-27 16:49:54,653][20347] Updated weights for policy 0, policy_version 1614581 (0.0007) [2023-12-27 16:49:54,701][20347] Updated weights for policy 0, policy_version 1614591 (0.0007) [2023-12-27 16:49:54,750][20347] Updated weights for policy 0, policy_version 1614601 (0.0007) [2023-12-27 16:49:54,772][20381] Updated weights for policy 1, policy_version 1612989 (0.0008) [2023-12-27 16:49:54,833][20381] Updated weights for policy 1, policy_version 1612999 (0.0011) [2023-12-27 16:49:54,884][20381] Updated weights for policy 1, policy_version 1613009 (0.0010) [2023-12-27 16:49:55,462][20381] Updated weights for policy 1, policy_version 1613019 (0.0009) [2023-12-27 16:49:55,484][20347] Updated weights for policy 0, policy_version 1614611 (0.0006) [2023-12-27 16:49:55,511][20381] Updated weights for policy 1, policy_version 1613029 (0.0007) [2023-12-27 16:49:55,541][20347] Updated weights for policy 0, policy_version 1614621 (0.0007) [2023-12-27 16:49:55,558][20381] Updated weights for policy 1, policy_version 1613039 (0.0007) [2023-12-27 16:49:55,598][20347] Updated weights for policy 0, policy_version 1614631 (0.0006) [2023-12-27 16:49:56,160][20381] Updated weights for policy 1, policy_version 1613049 (0.0005) [2023-12-27 16:49:56,223][20381] Updated weights for policy 1, policy_version 1613059 (0.0007) [2023-12-27 16:49:56,275][20381] Updated weights for policy 1, policy_version 1613069 (0.0007) [2023-12-27 16:49:56,316][20347] Updated weights for policy 0, policy_version 1614641 (0.0007) [2023-12-27 16:49:56,329][20381] Updated weights for policy 1, policy_version 1613079 (0.0007) [2023-12-27 16:49:56,371][20347] Updated weights for policy 0, policy_version 1614651 (0.0006) [2023-12-27 16:49:56,427][20347] Updated weights for policy 0, policy_version 1614661 (0.0008) [2023-12-27 16:49:56,490][20347] Updated weights for policy 0, policy_version 1614671 (0.0007) [2023-12-27 16:49:56,917][20381] Updated weights for policy 1, policy_version 1613089 (0.0005) [2023-12-27 16:49:56,971][20381] Updated weights for policy 1, policy_version 1613099 (0.0005) [2023-12-27 16:49:57,019][20381] Updated weights for policy 1, policy_version 1613109 (0.0005) [2023-12-27 16:49:57,221][20347] Updated weights for policy 0, policy_version 1614681 (0.0008) [2023-12-27 16:49:57,276][20347] Updated weights for policy 0, policy_version 1614691 (0.0007) [2023-12-27 16:49:57,338][20347] Updated weights for policy 0, policy_version 1614701 (0.0007) [2023-12-27 16:49:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 826441728. Throughput: 0: 10370.3, 1: 10350.9. Samples: 826426264. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:49:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.540')] [2023-12-27 16:49:57,590][20381] Updated weights for policy 1, policy_version 1613119 (0.0005) [2023-12-27 16:49:57,642][20381] Updated weights for policy 1, policy_version 1613129 (0.0006) [2023-12-27 16:49:57,699][20381] Updated weights for policy 1, policy_version 1613139 (0.0006) [2023-12-27 16:49:57,968][20347] Updated weights for policy 0, policy_version 1614711 (0.0009) [2023-12-27 16:49:58,016][20347] Updated weights for policy 0, policy_version 1614721 (0.0010) [2023-12-27 16:49:58,072][20347] Updated weights for policy 0, policy_version 1614731 (0.0010) [2023-12-27 16:49:58,313][20381] Updated weights for policy 1, policy_version 1613149 (0.0011) [2023-12-27 16:49:58,376][20381] Updated weights for policy 1, policy_version 1613159 (0.0009) [2023-12-27 16:49:58,440][20381] Updated weights for policy 1, policy_version 1613169 (0.0009) [2023-12-27 16:49:58,868][20347] Updated weights for policy 0, policy_version 1614741 (0.0010) [2023-12-27 16:49:58,931][20347] Updated weights for policy 0, policy_version 1614751 (0.0011) [2023-12-27 16:49:58,982][20347] Updated weights for policy 0, policy_version 1614761 (0.0010) [2023-12-27 16:49:59,233][20381] Updated weights for policy 1, policy_version 1613179 (0.0010) [2023-12-27 16:49:59,297][20381] Updated weights for policy 1, policy_version 1613189 (0.0007) [2023-12-27 16:49:59,365][20381] Updated weights for policy 1, policy_version 1613199 (0.0009) [2023-12-27 16:49:59,756][20347] Updated weights for policy 0, policy_version 1614771 (0.0009) [2023-12-27 16:49:59,814][20347] Updated weights for policy 0, policy_version 1614781 (0.0010) [2023-12-27 16:49:59,877][20347] Updated weights for policy 0, policy_version 1614791 (0.0009) [2023-12-27 16:50:00,047][20381] Updated weights for policy 1, policy_version 1613209 (0.0010) [2023-12-27 16:50:00,106][20381] Updated weights for policy 1, policy_version 1613219 (0.0008) [2023-12-27 16:50:00,167][20381] Updated weights for policy 1, policy_version 1613229 (0.0008) [2023-12-27 16:50:00,230][20381] Updated weights for policy 1, policy_version 1613239 (0.0008) [2023-12-27 16:50:00,546][20347] Updated weights for policy 0, policy_version 1614801 (0.0009) [2023-12-27 16:50:00,600][20347] Updated weights for policy 0, policy_version 1614811 (0.0010) [2023-12-27 16:50:00,653][20347] Updated weights for policy 0, policy_version 1614821 (0.0007) [2023-12-27 16:50:00,707][20347] Updated weights for policy 0, policy_version 1614831 (0.0005) [2023-12-27 16:50:00,891][20381] Updated weights for policy 1, policy_version 1613249 (0.0008) [2023-12-27 16:50:00,955][20381] Updated weights for policy 1, policy_version 1613259 (0.0008) [2023-12-27 16:50:01,018][20381] Updated weights for policy 1, policy_version 1613269 (0.0008) [2023-12-27 16:50:01,354][20347] Updated weights for policy 0, policy_version 1614841 (0.0009) [2023-12-27 16:50:01,421][20347] Updated weights for policy 0, policy_version 1614851 (0.0007) [2023-12-27 16:50:01,482][20347] Updated weights for policy 0, policy_version 1614861 (0.0010) [2023-12-27 16:50:01,697][20381] Updated weights for policy 1, policy_version 1613279 (0.0008) [2023-12-27 16:50:01,760][20381] Updated weights for policy 1, policy_version 1613289 (0.0008) [2023-12-27 16:50:01,819][20381] Updated weights for policy 1, policy_version 1613299 (0.0006) [2023-12-27 16:50:02,162][20347] Updated weights for policy 0, policy_version 1614871 (0.0005) [2023-12-27 16:50:02,227][20347] Updated weights for policy 0, policy_version 1614881 (0.0009) [2023-12-27 16:50:02,285][20347] Updated weights for policy 0, policy_version 1614891 (0.0010) [2023-12-27 16:50:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 826540032. Throughput: 0: 10322.2, 1: 10430.1. Samples: 826549120. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:50:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '814.980')] [2023-12-27 16:50:02,448][20381] Updated weights for policy 1, policy_version 1613309 (0.0006) [2023-12-27 16:50:02,507][20381] Updated weights for policy 1, policy_version 1613319 (0.0006) [2023-12-27 16:50:02,563][20381] Updated weights for policy 1, policy_version 1613329 (0.0006) [2023-12-27 16:50:02,918][20347] Updated weights for policy 0, policy_version 1614901 (0.0008) [2023-12-27 16:50:02,969][20347] Updated weights for policy 0, policy_version 1614911 (0.0005) [2023-12-27 16:50:03,016][20347] Updated weights for policy 0, policy_version 1614921 (0.0005) [2023-12-27 16:50:03,147][20381] Updated weights for policy 1, policy_version 1613339 (0.0006) [2023-12-27 16:50:03,217][20381] Updated weights for policy 1, policy_version 1613349 (0.0007) [2023-12-27 16:50:03,278][20381] Updated weights for policy 1, policy_version 1613359 (0.0006) [2023-12-27 16:50:03,608][20347] Updated weights for policy 0, policy_version 1614931 (0.0006) [2023-12-27 16:50:03,655][20347] Updated weights for policy 0, policy_version 1614941 (0.0005) [2023-12-27 16:50:03,703][20347] Updated weights for policy 0, policy_version 1614951 (0.0007) [2023-12-27 16:50:03,902][20381] Updated weights for policy 1, policy_version 1613369 (0.0007) [2023-12-27 16:50:03,961][20381] Updated weights for policy 1, policy_version 1613379 (0.0005) [2023-12-27 16:50:04,016][20381] Updated weights for policy 1, policy_version 1613389 (0.0005) [2023-12-27 16:50:04,069][20381] Updated weights for policy 1, policy_version 1613399 (0.0005) [2023-12-27 16:50:04,394][20347] Updated weights for policy 0, policy_version 1614961 (0.0008) [2023-12-27 16:50:04,461][20347] Updated weights for policy 0, policy_version 1614971 (0.0008) [2023-12-27 16:50:04,525][20347] Updated weights for policy 0, policy_version 1614981 (0.0008) [2023-12-27 16:50:04,578][20347] Updated weights for policy 0, policy_version 1614991 (0.0009) [2023-12-27 16:50:04,717][20381] Updated weights for policy 1, policy_version 1613409 (0.0010) [2023-12-27 16:50:04,773][20381] Updated weights for policy 1, policy_version 1613419 (0.0010) [2023-12-27 16:50:04,828][20381] Updated weights for policy 1, policy_version 1613429 (0.0010) [2023-12-27 16:50:05,316][20347] Updated weights for policy 0, policy_version 1615001 (0.0007) [2023-12-27 16:50:05,367][20347] Updated weights for policy 0, policy_version 1615011 (0.0005) [2023-12-27 16:50:05,418][20347] Updated weights for policy 0, policy_version 1615021 (0.0005) [2023-12-27 16:50:05,425][20381] Updated weights for policy 1, policy_version 1613439 (0.0007) [2023-12-27 16:50:05,477][20381] Updated weights for policy 1, policy_version 1613449 (0.0005) [2023-12-27 16:50:05,528][20381] Updated weights for policy 1, policy_version 1613459 (0.0005) [2023-12-27 16:50:06,060][20347] Updated weights for policy 0, policy_version 1615031 (0.0005) [2023-12-27 16:50:06,123][20381] Updated weights for policy 1, policy_version 1613469 (0.0007) [2023-12-27 16:50:06,128][20347] Updated weights for policy 0, policy_version 1615041 (0.0007) [2023-12-27 16:50:06,182][20381] Updated weights for policy 1, policy_version 1613479 (0.0006) [2023-12-27 16:50:06,191][20347] Updated weights for policy 0, policy_version 1615051 (0.0007) [2023-12-27 16:50:06,244][20381] Updated weights for policy 1, policy_version 1613489 (0.0009) [2023-12-27 16:50:06,888][20347] Updated weights for policy 0, policy_version 1615061 (0.0007) [2023-12-27 16:50:06,944][20347] Updated weights for policy 0, policy_version 1615071 (0.0007) [2023-12-27 16:50:06,954][20381] Updated weights for policy 1, policy_version 1613499 (0.0009) [2023-12-27 16:50:06,994][20347] Updated weights for policy 0, policy_version 1615081 (0.0006) [2023-12-27 16:50:07,009][20381] Updated weights for policy 1, policy_version 1613509 (0.0008) [2023-12-27 16:50:07,064][20381] Updated weights for policy 1, policy_version 1613519 (0.0008) [2023-12-27 16:50:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 826646528. Throughput: 0: 10307.1, 1: 10504.4. Samples: 826613320. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:50:07,363][19170] Avg episode reward: [(0, '980.450'), (1, '853.490')] [2023-12-27 16:50:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001613528_413106176.pth... [2023-12-27 16:50:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001615088_413540352.pth... [2023-12-27 16:50:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001613904_413237248.pth [2023-12-27 16:50:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001612312_412794880.pth [2023-12-27 16:50:07,683][20381] Updated weights for policy 1, policy_version 1613529 (0.0008) [2023-12-27 16:50:07,685][20347] Updated weights for policy 0, policy_version 1615091 (0.0007) [2023-12-27 16:50:07,733][20381] Updated weights for policy 1, policy_version 1613539 (0.0006) [2023-12-27 16:50:07,739][20347] Updated weights for policy 0, policy_version 1615101 (0.0006) [2023-12-27 16:50:07,783][20381] Updated weights for policy 1, policy_version 1613549 (0.0005) [2023-12-27 16:50:07,792][20347] Updated weights for policy 0, policy_version 1615111 (0.0005) [2023-12-27 16:50:07,833][20381] Updated weights for policy 1, policy_version 1613559 (0.0005) [2023-12-27 16:50:08,499][20347] Updated weights for policy 0, policy_version 1615121 (0.0007) [2023-12-27 16:50:08,527][20381] Updated weights for policy 1, policy_version 1613569 (0.0007) [2023-12-27 16:50:08,556][20347] Updated weights for policy 0, policy_version 1615131 (0.0006) [2023-12-27 16:50:08,588][20381] Updated weights for policy 1, policy_version 1613579 (0.0006) [2023-12-27 16:50:08,622][20347] Updated weights for policy 0, policy_version 1615141 (0.0006) [2023-12-27 16:50:08,644][20381] Updated weights for policy 1, policy_version 1613589 (0.0005) [2023-12-27 16:50:08,679][20347] Updated weights for policy 0, policy_version 1615151 (0.0007) [2023-12-27 16:50:09,256][20381] Updated weights for policy 1, policy_version 1613599 (0.0008) [2023-12-27 16:50:09,295][20292] KL-divergence is very high: 100.7811 [2023-12-27 16:50:09,321][20381] Updated weights for policy 1, policy_version 1613609 (0.0008) [2023-12-27 16:50:09,389][20381] Updated weights for policy 1, policy_version 1613619 (0.0008) [2023-12-27 16:50:09,431][20347] Updated weights for policy 0, policy_version 1615161 (0.0008) [2023-12-27 16:50:09,489][20347] Updated weights for policy 0, policy_version 1615171 (0.0008) [2023-12-27 16:50:09,552][20347] Updated weights for policy 0, policy_version 1615181 (0.0008) [2023-12-27 16:50:10,045][20381] Updated weights for policy 1, policy_version 1613629 (0.0009) [2023-12-27 16:50:10,102][20381] Updated weights for policy 1, policy_version 1613639 (0.0009) [2023-12-27 16:50:10,165][20381] Updated weights for policy 1, policy_version 1613649 (0.0009) [2023-12-27 16:50:10,276][20347] Updated weights for policy 0, policy_version 1615191 (0.0008) [2023-12-27 16:50:10,331][20347] Updated weights for policy 0, policy_version 1615201 (0.0007) [2023-12-27 16:50:10,385][20347] Updated weights for policy 0, policy_version 1615211 (0.0007) [2023-12-27 16:50:10,815][20381] Updated weights for policy 1, policy_version 1613659 (0.0007) [2023-12-27 16:50:10,875][20381] Updated weights for policy 1, policy_version 1613669 (0.0010) [2023-12-27 16:50:10,934][20381] Updated weights for policy 1, policy_version 1613679 (0.0011) [2023-12-27 16:50:11,078][20347] Updated weights for policy 0, policy_version 1615221 (0.0007) [2023-12-27 16:50:11,139][20347] Updated weights for policy 0, policy_version 1615231 (0.0007) [2023-12-27 16:50:11,198][20347] Updated weights for policy 0, policy_version 1615241 (0.0007) [2023-12-27 16:50:11,680][20381] Updated weights for policy 1, policy_version 1613689 (0.0008) [2023-12-27 16:50:11,750][20381] Updated weights for policy 1, policy_version 1613699 (0.0008) [2023-12-27 16:50:11,804][20381] Updated weights for policy 1, policy_version 1613709 (0.0008) [2023-12-27 16:50:11,850][20381] Updated weights for policy 1, policy_version 1613719 (0.0008) [2023-12-27 16:50:11,918][20347] Updated weights for policy 0, policy_version 1615251 (0.0007) [2023-12-27 16:50:11,982][20347] Updated weights for policy 0, policy_version 1615261 (0.0007) [2023-12-27 16:50:12,041][20347] Updated weights for policy 0, policy_version 1615271 (0.0007) [2023-12-27 16:50:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 826744832. Throughput: 0: 10214.2, 1: 10490.0. Samples: 826735612. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:50:12,363][19170] Avg episode reward: [(0, '980.430'), (1, '863.580')] [2023-12-27 16:50:12,600][20381] Updated weights for policy 1, policy_version 1613729 (0.0008) [2023-12-27 16:50:12,661][20381] Updated weights for policy 1, policy_version 1613739 (0.0008) [2023-12-27 16:50:12,711][20381] Updated weights for policy 1, policy_version 1613749 (0.0008) [2023-12-27 16:50:12,716][20347] Updated weights for policy 0, policy_version 1615281 (0.0006) [2023-12-27 16:50:12,779][20347] Updated weights for policy 0, policy_version 1615291 (0.0007) [2023-12-27 16:50:12,839][20347] Updated weights for policy 0, policy_version 1615301 (0.0008) [2023-12-27 16:50:12,893][20347] Updated weights for policy 0, policy_version 1615311 (0.0007) [2023-12-27 16:50:13,468][20381] Updated weights for policy 1, policy_version 1613759 (0.0007) [2023-12-27 16:50:13,485][20347] Updated weights for policy 0, policy_version 1615321 (0.0008) [2023-12-27 16:50:13,523][20381] Updated weights for policy 1, policy_version 1613769 (0.0008) [2023-12-27 16:50:13,531][20347] Updated weights for policy 0, policy_version 1615331 (0.0010) [2023-12-27 16:50:13,577][20381] Updated weights for policy 1, policy_version 1613779 (0.0006) [2023-12-27 16:50:13,583][20347] Updated weights for policy 0, policy_version 1615341 (0.0010) [2023-12-27 16:50:14,185][20381] Updated weights for policy 1, policy_version 1613789 (0.0006) [2023-12-27 16:50:14,239][20381] Updated weights for policy 1, policy_version 1613799 (0.0007) [2023-12-27 16:50:14,264][20347] Updated weights for policy 0, policy_version 1615351 (0.0007) [2023-12-27 16:50:14,297][20381] Updated weights for policy 1, policy_version 1613809 (0.0005) [2023-12-27 16:50:14,325][20347] Updated weights for policy 0, policy_version 1615361 (0.0007) [2023-12-27 16:50:14,374][20347] Updated weights for policy 0, policy_version 1615371 (0.0007) [2023-12-27 16:50:14,889][20381] Updated weights for policy 1, policy_version 1613819 (0.0006) [2023-12-27 16:50:14,947][20381] Updated weights for policy 1, policy_version 1613829 (0.0007) [2023-12-27 16:50:15,007][20381] Updated weights for policy 1, policy_version 1613839 (0.0009) [2023-12-27 16:50:15,041][20347] Updated weights for policy 0, policy_version 1615381 (0.0005) [2023-12-27 16:50:15,089][20347] Updated weights for policy 0, policy_version 1615391 (0.0005) [2023-12-27 16:50:15,142][20347] Updated weights for policy 0, policy_version 1615401 (0.0006) [2023-12-27 16:50:15,760][20381] Updated weights for policy 1, policy_version 1613849 (0.0009) [2023-12-27 16:50:15,762][20347] Updated weights for policy 0, policy_version 1615411 (0.0006) [2023-12-27 16:50:15,813][20381] Updated weights for policy 1, policy_version 1613859 (0.0006) [2023-12-27 16:50:15,815][20347] Updated weights for policy 0, policy_version 1615421 (0.0006) [2023-12-27 16:50:15,865][20381] Updated weights for policy 1, policy_version 1613869 (0.0006) [2023-12-27 16:50:15,867][20347] Updated weights for policy 0, policy_version 1615431 (0.0007) [2023-12-27 16:50:15,921][20381] Updated weights for policy 1, policy_version 1613879 (0.0005) [2023-12-27 16:50:16,524][20347] Updated weights for policy 0, policy_version 1615441 (0.0006) [2023-12-27 16:50:16,581][20347] Updated weights for policy 0, policy_version 1615451 (0.0006) [2023-12-27 16:50:16,635][20347] Updated weights for policy 0, policy_version 1615461 (0.0005) [2023-12-27 16:50:16,640][20381] Updated weights for policy 1, policy_version 1613889 (0.0007) [2023-12-27 16:50:16,685][20347] Updated weights for policy 0, policy_version 1615471 (0.0006) [2023-12-27 16:50:16,698][20381] Updated weights for policy 1, policy_version 1613899 (0.0006) [2023-12-27 16:50:16,755][20381] Updated weights for policy 1, policy_version 1613909 (0.0008) [2023-12-27 16:50:17,263][20347] Updated weights for policy 0, policy_version 1615481 (0.0005) [2023-12-27 16:50:17,317][20347] Updated weights for policy 0, policy_version 1615491 (0.0006) [2023-12-27 16:50:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 826843136. Throughput: 0: 10194.8, 1: 10369.4. Samples: 826858156. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:50:17,363][19170] Avg episode reward: [(0, '970.770'), (1, '843.600')] [2023-12-27 16:50:17,376][20347] Updated weights for policy 0, policy_version 1615501 (0.0008) [2023-12-27 16:50:17,500][20381] Updated weights for policy 1, policy_version 1613919 (0.0008) [2023-12-27 16:50:17,552][20381] Updated weights for policy 1, policy_version 1613929 (0.0008) [2023-12-27 16:50:17,610][20381] Updated weights for policy 1, policy_version 1613939 (0.0005) [2023-12-27 16:50:18,038][20347] Updated weights for policy 0, policy_version 1615511 (0.0007) [2023-12-27 16:50:18,091][20347] Updated weights for policy 0, policy_version 1615521 (0.0007) [2023-12-27 16:50:18,146][20347] Updated weights for policy 0, policy_version 1615531 (0.0007) [2023-12-27 16:50:18,202][20381] Updated weights for policy 1, policy_version 1613949 (0.0005) [2023-12-27 16:50:18,255][20381] Updated weights for policy 1, policy_version 1613959 (0.0007) [2023-12-27 16:50:18,308][20381] Updated weights for policy 1, policy_version 1613969 (0.0008) [2023-12-27 16:50:18,822][20347] Updated weights for policy 0, policy_version 1615541 (0.0007) [2023-12-27 16:50:18,891][20347] Updated weights for policy 0, policy_version 1615551 (0.0006) [2023-12-27 16:50:18,919][20381] Updated weights for policy 1, policy_version 1613979 (0.0007) [2023-12-27 16:50:18,958][20347] Updated weights for policy 0, policy_version 1615561 (0.0005) [2023-12-27 16:50:18,983][20381] Updated weights for policy 1, policy_version 1613989 (0.0006) [2023-12-27 16:50:19,041][20381] Updated weights for policy 1, policy_version 1613999 (0.0007) [2023-12-27 16:50:19,683][20347] Updated weights for policy 0, policy_version 1615571 (0.0006) [2023-12-27 16:50:19,734][20381] Updated weights for policy 1, policy_version 1614009 (0.0008) [2023-12-27 16:50:19,741][20347] Updated weights for policy 0, policy_version 1615581 (0.0007) [2023-12-27 16:50:19,791][20381] Updated weights for policy 1, policy_version 1614019 (0.0008) [2023-12-27 16:50:19,801][20347] Updated weights for policy 0, policy_version 1615591 (0.0007) [2023-12-27 16:50:19,853][20381] Updated weights for policy 1, policy_version 1614029 (0.0006) [2023-12-27 16:50:19,915][20381] Updated weights for policy 1, policy_version 1614039 (0.0007) [2023-12-27 16:50:20,530][20347] Updated weights for policy 0, policy_version 1615601 (0.0007) [2023-12-27 16:50:20,598][20347] Updated weights for policy 0, policy_version 1615611 (0.0008) [2023-12-27 16:50:20,631][20381] Updated weights for policy 1, policy_version 1614049 (0.0007) [2023-12-27 16:50:20,662][20347] Updated weights for policy 0, policy_version 1615621 (0.0008) [2023-12-27 16:50:20,688][20381] Updated weights for policy 1, policy_version 1614059 (0.0006) [2023-12-27 16:50:20,713][20347] Updated weights for policy 0, policy_version 1615631 (0.0006) [2023-12-27 16:50:20,746][20381] Updated weights for policy 1, policy_version 1614069 (0.0007) [2023-12-27 16:50:21,408][20381] Updated weights for policy 1, policy_version 1614079 (0.0008) [2023-12-27 16:50:21,457][20347] Updated weights for policy 0, policy_version 1615641 (0.0008) [2023-12-27 16:50:21,469][20381] Updated weights for policy 1, policy_version 1614089 (0.0008) [2023-12-27 16:50:21,507][20347] Updated weights for policy 0, policy_version 1615651 (0.0007) [2023-12-27 16:50:21,528][20381] Updated weights for policy 1, policy_version 1614099 (0.0008) [2023-12-27 16:50:21,563][20347] Updated weights for policy 0, policy_version 1615661 (0.0006) [2023-12-27 16:50:22,281][20381] Updated weights for policy 1, policy_version 1614109 (0.0008) [2023-12-27 16:50:22,345][20381] Updated weights for policy 1, policy_version 1614119 (0.0007) [2023-12-27 16:50:22,346][20347] Updated weights for policy 0, policy_version 1615671 (0.0007) [2023-12-27 16:50:22,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 826941440. Throughput: 0: 10217.8, 1: 10379.2. Samples: 826920216. Policy #0 lag: (min: 31.0, avg: 37.4, max: 63.0) [2023-12-27 16:50:22,363][19170] Avg episode reward: [(0, '980.560'), (1, '912.070')] [2023-12-27 16:50:22,409][20347] Updated weights for policy 0, policy_version 1615681 (0.0007) [2023-12-27 16:50:22,414][20381] Updated weights for policy 1, policy_version 1614129 (0.0007) [2023-12-27 16:50:22,454][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001614136_413261824.pth... [2023-12-27 16:50:22,459][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001612920_412950528.pth [2023-12-27 16:50:22,467][20347] Updated weights for policy 0, policy_version 1615691 (0.0007) [2023-12-27 16:50:22,493][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001615696_413696000.pth... [2023-12-27 16:50:22,496][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001614512_413392896.pth [2023-12-27 16:50:23,053][20381] Updated weights for policy 1, policy_version 1614139 (0.0007) [2023-12-27 16:50:23,112][20381] Updated weights for policy 1, policy_version 1614149 (0.0006) [2023-12-27 16:50:23,120][20347] Updated weights for policy 0, policy_version 1615701 (0.0006) [2023-12-27 16:50:23,160][20381] Updated weights for policy 1, policy_version 1614159 (0.0006) [2023-12-27 16:50:23,165][20347] Updated weights for policy 0, policy_version 1615711 (0.0005) [2023-12-27 16:50:23,215][20347] Updated weights for policy 0, policy_version 1615721 (0.0005) [2023-12-27 16:50:23,793][20381] Updated weights for policy 1, policy_version 1614169 (0.0006) [2023-12-27 16:50:23,857][20381] Updated weights for policy 1, policy_version 1614179 (0.0008) [2023-12-27 16:50:23,917][20381] Updated weights for policy 1, policy_version 1614189 (0.0008) [2023-12-27 16:50:23,950][20347] Updated weights for policy 0, policy_version 1615731 (0.0008) [2023-12-27 16:50:23,979][20381] Updated weights for policy 1, policy_version 1614199 (0.0005) [2023-12-27 16:50:24,010][20347] Updated weights for policy 0, policy_version 1615741 (0.0008) [2023-12-27 16:50:24,063][20347] Updated weights for policy 0, policy_version 1615751 (0.0007) [2023-12-27 16:50:24,622][20381] Updated weights for policy 1, policy_version 1614209 (0.0007) [2023-12-27 16:50:24,685][20381] Updated weights for policy 1, policy_version 1614219 (0.0008) [2023-12-27 16:50:24,732][20347] Updated weights for policy 0, policy_version 1615761 (0.0007) [2023-12-27 16:50:24,746][20381] Updated weights for policy 1, policy_version 1614229 (0.0008) [2023-12-27 16:50:24,789][20347] Updated weights for policy 0, policy_version 1615771 (0.0005) [2023-12-27 16:50:24,845][20347] Updated weights for policy 0, policy_version 1615781 (0.0005) [2023-12-27 16:50:24,900][20347] Updated weights for policy 0, policy_version 1615791 (0.0010) [2023-12-27 16:50:25,420][20381] Updated weights for policy 1, policy_version 1614239 (0.0009) [2023-12-27 16:50:25,477][20381] Updated weights for policy 1, policy_version 1614249 (0.0010) [2023-12-27 16:50:25,534][20347] Updated weights for policy 0, policy_version 1615801 (0.0010) [2023-12-27 16:50:25,534][20381] Updated weights for policy 1, policy_version 1614259 (0.0008) [2023-12-27 16:50:25,584][20347] Updated weights for policy 0, policy_version 1615811 (0.0010) [2023-12-27 16:50:25,654][20347] Updated weights for policy 0, policy_version 1615821 (0.0010) [2023-12-27 16:50:26,122][20381] Updated weights for policy 1, policy_version 1614269 (0.0008) [2023-12-27 16:50:26,184][20381] Updated weights for policy 1, policy_version 1614279 (0.0011) [2023-12-27 16:50:26,231][20381] Updated weights for policy 1, policy_version 1614289 (0.0009) [2023-12-27 16:50:26,285][20347] Updated weights for policy 0, policy_version 1615831 (0.0010) [2023-12-27 16:50:26,337][20347] Updated weights for policy 0, policy_version 1615841 (0.0009) [2023-12-27 16:50:26,383][20347] Updated weights for policy 0, policy_version 1615851 (0.0006) [2023-12-27 16:50:26,839][20381] Updated weights for policy 1, policy_version 1614299 (0.0009) [2023-12-27 16:50:26,887][20381] Updated weights for policy 1, policy_version 1614309 (0.0007) [2023-12-27 16:50:26,934][20381] Updated weights for policy 1, policy_version 1614319 (0.0008) [2023-12-27 16:50:27,009][20347] Updated weights for policy 0, policy_version 1615861 (0.0007) [2023-12-27 16:50:27,063][20347] Updated weights for policy 0, policy_version 1615871 (0.0005) [2023-12-27 16:50:27,122][20347] Updated weights for policy 0, policy_version 1615881 (0.0007) [2023-12-27 16:50:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 827056128. Throughput: 0: 10180.0, 1: 10355.5. Samples: 827041300. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:27,363][19170] Avg episode reward: [(0, '990.270'), (1, '941.500')] [2023-12-27 16:50:27,634][20347] Updated weights for policy 0, policy_version 1615891 (0.0008) [2023-12-27 16:50:27,682][20381] Updated weights for policy 1, policy_version 1614329 (0.0007) [2023-12-27 16:50:27,689][20347] Updated weights for policy 0, policy_version 1615901 (0.0006) [2023-12-27 16:50:27,734][20381] Updated weights for policy 1, policy_version 1614339 (0.0007) [2023-12-27 16:50:27,743][20347] Updated weights for policy 0, policy_version 1615911 (0.0007) [2023-12-27 16:50:27,782][20381] Updated weights for policy 1, policy_version 1614349 (0.0005) [2023-12-27 16:50:27,832][20381] Updated weights for policy 1, policy_version 1614359 (0.0007) [2023-12-27 16:50:28,299][20347] Updated weights for policy 0, policy_version 1615921 (0.0010) [2023-12-27 16:50:28,359][20347] Updated weights for policy 0, policy_version 1615931 (0.0010) [2023-12-27 16:50:28,418][20347] Updated weights for policy 0, policy_version 1615941 (0.0008) [2023-12-27 16:50:28,477][20347] Updated weights for policy 0, policy_version 1615951 (0.0007) [2023-12-27 16:50:28,511][20381] Updated weights for policy 1, policy_version 1614369 (0.0007) [2023-12-27 16:50:28,566][20381] Updated weights for policy 1, policy_version 1614379 (0.0006) [2023-12-27 16:50:28,615][20381] Updated weights for policy 1, policy_version 1614389 (0.0006) [2023-12-27 16:50:29,149][20347] Updated weights for policy 0, policy_version 1615961 (0.0005) [2023-12-27 16:50:29,205][20347] Updated weights for policy 0, policy_version 1615971 (0.0006) [2023-12-27 16:50:29,274][20347] Updated weights for policy 0, policy_version 1615981 (0.0007) [2023-12-27 16:50:29,328][20381] Updated weights for policy 1, policy_version 1614399 (0.0008) [2023-12-27 16:50:29,399][20381] Updated weights for policy 1, policy_version 1614409 (0.0008) [2023-12-27 16:50:29,455][20381] Updated weights for policy 1, policy_version 1614419 (0.0005) [2023-12-27 16:50:30,007][20347] Updated weights for policy 0, policy_version 1615991 (0.0006) [2023-12-27 16:50:30,068][20347] Updated weights for policy 0, policy_version 1616001 (0.0005) [2023-12-27 16:50:30,120][20381] Updated weights for policy 1, policy_version 1614429 (0.0005) [2023-12-27 16:50:30,126][20347] Updated weights for policy 0, policy_version 1616011 (0.0005) [2023-12-27 16:50:30,180][20381] Updated weights for policy 1, policy_version 1614439 (0.0005) [2023-12-27 16:50:30,243][20381] Updated weights for policy 1, policy_version 1614449 (0.0006) [2023-12-27 16:50:30,715][20347] Updated weights for policy 0, policy_version 1616021 (0.0006) [2023-12-27 16:50:30,762][20347] Updated weights for policy 0, policy_version 1616031 (0.0005) [2023-12-27 16:50:30,808][20347] Updated weights for policy 0, policy_version 1616041 (0.0005) [2023-12-27 16:50:30,929][20381] Updated weights for policy 1, policy_version 1614459 (0.0006) [2023-12-27 16:50:30,978][20381] Updated weights for policy 1, policy_version 1614469 (0.0005) [2023-12-27 16:50:31,028][20381] Updated weights for policy 1, policy_version 1614479 (0.0006) [2023-12-27 16:50:31,444][20347] Updated weights for policy 0, policy_version 1616051 (0.0006) [2023-12-27 16:50:31,500][20347] Updated weights for policy 0, policy_version 1616061 (0.0008) [2023-12-27 16:50:31,555][20347] Updated weights for policy 0, policy_version 1616071 (0.0007) [2023-12-27 16:50:31,703][20381] Updated weights for policy 1, policy_version 1614489 (0.0008) [2023-12-27 16:50:31,765][20381] Updated weights for policy 1, policy_version 1614499 (0.0008) [2023-12-27 16:50:31,821][20381] Updated weights for policy 1, policy_version 1614509 (0.0008) [2023-12-27 16:50:31,872][20381] Updated weights for policy 1, policy_version 1614519 (0.0008) [2023-12-27 16:50:32,291][20347] Updated weights for policy 0, policy_version 1616081 (0.0007) [2023-12-27 16:50:32,350][20347] Updated weights for policy 0, policy_version 1616091 (0.0008) [2023-12-27 16:50:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 827154432. Throughput: 0: 10276.9, 1: 10311.6. Samples: 827169264. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.480')] [2023-12-27 16:50:32,412][20347] Updated weights for policy 0, policy_version 1616101 (0.0007) [2023-12-27 16:50:32,461][20347] Updated weights for policy 0, policy_version 1616111 (0.0005) [2023-12-27 16:50:32,468][20381] Updated weights for policy 1, policy_version 1614529 (0.0006) [2023-12-27 16:50:32,526][20381] Updated weights for policy 1, policy_version 1614539 (0.0010) [2023-12-27 16:50:32,577][20381] Updated weights for policy 1, policy_version 1614549 (0.0009) [2023-12-27 16:50:33,082][20347] Updated weights for policy 0, policy_version 1616121 (0.0007) [2023-12-27 16:50:33,135][20347] Updated weights for policy 0, policy_version 1616131 (0.0007) [2023-12-27 16:50:33,186][20347] Updated weights for policy 0, policy_version 1616141 (0.0007) [2023-12-27 16:50:33,225][20381] Updated weights for policy 1, policy_version 1614559 (0.0008) [2023-12-27 16:50:33,281][20381] Updated weights for policy 1, policy_version 1614569 (0.0008) [2023-12-27 16:50:33,337][20381] Updated weights for policy 1, policy_version 1614579 (0.0008) [2023-12-27 16:50:33,823][20347] Updated weights for policy 0, policy_version 1616151 (0.0007) [2023-12-27 16:50:33,868][20347] Updated weights for policy 0, policy_version 1616161 (0.0005) [2023-12-27 16:50:33,916][20347] Updated weights for policy 0, policy_version 1616171 (0.0005) [2023-12-27 16:50:33,981][20381] Updated weights for policy 1, policy_version 1614589 (0.0007) [2023-12-27 16:50:34,035][20381] Updated weights for policy 1, policy_version 1614599 (0.0005) [2023-12-27 16:50:34,082][20381] Updated weights for policy 1, policy_version 1614609 (0.0005) [2023-12-27 16:50:34,533][20347] Updated weights for policy 0, policy_version 1616181 (0.0008) [2023-12-27 16:50:34,585][20347] Updated weights for policy 0, policy_version 1616191 (0.0010) [2023-12-27 16:50:34,644][20347] Updated weights for policy 0, policy_version 1616201 (0.0011) [2023-12-27 16:50:34,802][20381] Updated weights for policy 1, policy_version 1614619 (0.0007) [2023-12-27 16:50:34,857][20381] Updated weights for policy 1, policy_version 1614629 (0.0008) [2023-12-27 16:50:34,909][20381] Updated weights for policy 1, policy_version 1614639 (0.0007) [2023-12-27 16:50:35,344][20347] Updated weights for policy 0, policy_version 1616211 (0.0010) [2023-12-27 16:50:35,399][20347] Updated weights for policy 0, policy_version 1616221 (0.0010) [2023-12-27 16:50:35,456][20347] Updated weights for policy 0, policy_version 1616231 (0.0010) [2023-12-27 16:50:35,624][20381] Updated weights for policy 1, policy_version 1614649 (0.0007) [2023-12-27 16:50:35,679][20381] Updated weights for policy 1, policy_version 1614659 (0.0005) [2023-12-27 16:50:35,738][20381] Updated weights for policy 1, policy_version 1614669 (0.0005) [2023-12-27 16:50:35,791][20381] Updated weights for policy 1, policy_version 1614679 (0.0005) [2023-12-27 16:50:36,116][20347] Updated weights for policy 0, policy_version 1616241 (0.0010) [2023-12-27 16:50:36,174][20347] Updated weights for policy 0, policy_version 1616251 (0.0010) [2023-12-27 16:50:36,237][20347] Updated weights for policy 0, policy_version 1616261 (0.0010) [2023-12-27 16:50:36,299][20347] Updated weights for policy 0, policy_version 1616271 (0.0010) [2023-12-27 16:50:36,411][20381] Updated weights for policy 1, policy_version 1614689 (0.0007) [2023-12-27 16:50:36,471][20381] Updated weights for policy 1, policy_version 1614699 (0.0008) [2023-12-27 16:50:36,527][20381] Updated weights for policy 1, policy_version 1614709 (0.0008) [2023-12-27 16:50:37,019][20347] Updated weights for policy 0, policy_version 1616281 (0.0011) [2023-12-27 16:50:37,072][20291] KL-divergence is very high: 115.9018 [2023-12-27 16:50:37,072][20347] Updated weights for policy 0, policy_version 1616291 (0.0010) [2023-12-27 16:50:37,127][20347] Updated weights for policy 0, policy_version 1616301 (0.0010) [2023-12-27 16:50:37,244][20381] Updated weights for policy 1, policy_version 1614719 (0.0008) [2023-12-27 16:50:37,308][20381] Updated weights for policy 1, policy_version 1614729 (0.0007) [2023-12-27 16:50:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 827260928. Throughput: 0: 10268.5, 1: 10365.3. Samples: 827232324. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:37,363][19170] Avg episode reward: [(0, '990.220'), (1, '912.320')] [2023-12-27 16:50:37,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001616304_413851648.pth... [2023-12-27 16:50:37,376][20381] Updated weights for policy 1, policy_version 1614739 (0.0007) [2023-12-27 16:50:37,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001615088_413540352.pth [2023-12-27 16:50:37,407][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001614744_413417472.pth... [2023-12-27 16:50:37,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001613528_413106176.pth [2023-12-27 16:50:37,849][20347] Updated weights for policy 0, policy_version 1616311 (0.0007) [2023-12-27 16:50:37,915][20347] Updated weights for policy 0, policy_version 1616321 (0.0006) [2023-12-27 16:50:37,987][20347] Updated weights for policy 0, policy_version 1616331 (0.0006) [2023-12-27 16:50:37,994][20381] Updated weights for policy 1, policy_version 1614749 (0.0005) [2023-12-27 16:50:38,053][20381] Updated weights for policy 1, policy_version 1614759 (0.0005) [2023-12-27 16:50:38,115][20381] Updated weights for policy 1, policy_version 1614769 (0.0006) [2023-12-27 16:50:38,633][20347] Updated weights for policy 0, policy_version 1616341 (0.0007) [2023-12-27 16:50:38,649][20381] Updated weights for policy 1, policy_version 1614779 (0.0005) [2023-12-27 16:50:38,686][20347] Updated weights for policy 0, policy_version 1616351 (0.0008) [2023-12-27 16:50:38,702][20381] Updated weights for policy 1, policy_version 1614789 (0.0006) [2023-12-27 16:50:38,737][20347] Updated weights for policy 0, policy_version 1616361 (0.0009) [2023-12-27 16:50:38,761][20381] Updated weights for policy 1, policy_version 1614799 (0.0006) [2023-12-27 16:50:39,383][20381] Updated weights for policy 1, policy_version 1614809 (0.0007) [2023-12-27 16:50:39,431][20347] Updated weights for policy 0, policy_version 1616371 (0.0005) [2023-12-27 16:50:39,453][20381] Updated weights for policy 1, policy_version 1614819 (0.0008) [2023-12-27 16:50:39,490][20347] Updated weights for policy 0, policy_version 1616381 (0.0007) [2023-12-27 16:50:39,516][20381] Updated weights for policy 1, policy_version 1614829 (0.0007) [2023-12-27 16:50:39,548][20347] Updated weights for policy 0, policy_version 1616391 (0.0008) [2023-12-27 16:50:39,576][20381] Updated weights for policy 1, policy_version 1614839 (0.0006) [2023-12-27 16:50:40,263][20347] Updated weights for policy 0, policy_version 1616401 (0.0006) [2023-12-27 16:50:40,294][20381] Updated weights for policy 1, policy_version 1614849 (0.0007) [2023-12-27 16:50:40,322][20347] Updated weights for policy 0, policy_version 1616411 (0.0007) [2023-12-27 16:50:40,351][20381] Updated weights for policy 1, policy_version 1614859 (0.0006) [2023-12-27 16:50:40,383][20347] Updated weights for policy 0, policy_version 1616421 (0.0008) [2023-12-27 16:50:40,412][20381] Updated weights for policy 1, policy_version 1614869 (0.0006) [2023-12-27 16:50:40,443][20347] Updated weights for policy 0, policy_version 1616431 (0.0007) [2023-12-27 16:50:41,048][20381] Updated weights for policy 1, policy_version 1614879 (0.0007) [2023-12-27 16:50:41,114][20381] Updated weights for policy 1, policy_version 1614889 (0.0006) [2023-12-27 16:50:41,177][20347] Updated weights for policy 0, policy_version 1616441 (0.0008) [2023-12-27 16:50:41,182][20381] Updated weights for policy 1, policy_version 1614899 (0.0007) [2023-12-27 16:50:41,236][20347] Updated weights for policy 0, policy_version 1616451 (0.0006) [2023-12-27 16:50:41,297][20347] Updated weights for policy 0, policy_version 1616461 (0.0008) [2023-12-27 16:50:41,909][20381] Updated weights for policy 1, policy_version 1614909 (0.0007) [2023-12-27 16:50:41,974][20381] Updated weights for policy 1, policy_version 1614919 (0.0007) [2023-12-27 16:50:42,002][20347] Updated weights for policy 0, policy_version 1616471 (0.0007) [2023-12-27 16:50:42,033][20381] Updated weights for policy 1, policy_version 1614929 (0.0007) [2023-12-27 16:50:42,064][20347] Updated weights for policy 0, policy_version 1616481 (0.0006) [2023-12-27 16:50:42,121][20347] Updated weights for policy 0, policy_version 1616491 (0.0008) [2023-12-27 16:50:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 827367424. Throughput: 0: 10256.4, 1: 10393.2. Samples: 827355496. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:42,363][19170] Avg episode reward: [(0, '980.490'), (1, '892.570')] [2023-12-27 16:50:42,712][20381] Updated weights for policy 1, policy_version 1614939 (0.0006) [2023-12-27 16:50:42,764][20381] Updated weights for policy 1, policy_version 1614949 (0.0007) [2023-12-27 16:50:42,814][20347] Updated weights for policy 0, policy_version 1616501 (0.0007) [2023-12-27 16:50:42,824][20381] Updated weights for policy 1, policy_version 1614959 (0.0005) [2023-12-27 16:50:42,872][20347] Updated weights for policy 0, policy_version 1616511 (0.0007) [2023-12-27 16:50:42,928][20347] Updated weights for policy 0, policy_version 1616521 (0.0005) [2023-12-27 16:50:43,422][20381] Updated weights for policy 1, policy_version 1614969 (0.0006) [2023-12-27 16:50:43,473][20381] Updated weights for policy 1, policy_version 1614979 (0.0007) [2023-12-27 16:50:43,475][20347] Updated weights for policy 0, policy_version 1616531 (0.0005) [2023-12-27 16:50:43,526][20381] Updated weights for policy 1, policy_version 1614989 (0.0007) [2023-12-27 16:50:43,528][20347] Updated weights for policy 0, policy_version 1616541 (0.0005) [2023-12-27 16:50:43,574][20381] Updated weights for policy 1, policy_version 1614999 (0.0006) [2023-12-27 16:50:43,576][20347] Updated weights for policy 0, policy_version 1616551 (0.0005) [2023-12-27 16:50:44,152][20347] Updated weights for policy 0, policy_version 1616561 (0.0007) [2023-12-27 16:50:44,212][20347] Updated weights for policy 0, policy_version 1616571 (0.0005) [2023-12-27 16:50:44,270][20347] Updated weights for policy 0, policy_version 1616581 (0.0005) [2023-12-27 16:50:44,302][20381] Updated weights for policy 1, policy_version 1615009 (0.0008) [2023-12-27 16:50:44,328][20347] Updated weights for policy 0, policy_version 1616591 (0.0005) [2023-12-27 16:50:44,349][20381] Updated weights for policy 1, policy_version 1615019 (0.0008) [2023-12-27 16:50:44,393][20381] Updated weights for policy 1, policy_version 1615029 (0.0008) [2023-12-27 16:50:44,907][20347] Updated weights for policy 0, policy_version 1616601 (0.0007) [2023-12-27 16:50:44,962][20347] Updated weights for policy 0, policy_version 1616611 (0.0008) [2023-12-27 16:50:45,015][20347] Updated weights for policy 0, policy_version 1616621 (0.0007) [2023-12-27 16:50:45,148][20381] Updated weights for policy 1, policy_version 1615039 (0.0007) [2023-12-27 16:50:45,195][20381] Updated weights for policy 1, policy_version 1615049 (0.0008) [2023-12-27 16:50:45,243][20381] Updated weights for policy 1, policy_version 1615059 (0.0006) [2023-12-27 16:50:45,668][20347] Updated weights for policy 0, policy_version 1616631 (0.0006) [2023-12-27 16:50:45,723][20347] Updated weights for policy 0, policy_version 1616641 (0.0005) [2023-12-27 16:50:45,773][20347] Updated weights for policy 0, policy_version 1616651 (0.0005) [2023-12-27 16:50:45,983][20381] Updated weights for policy 1, policy_version 1615069 (0.0007) [2023-12-27 16:50:46,033][20381] Updated weights for policy 1, policy_version 1615079 (0.0008) [2023-12-27 16:50:46,082][20381] Updated weights for policy 1, policy_version 1615089 (0.0008) [2023-12-27 16:50:46,325][20347] Updated weights for policy 0, policy_version 1616661 (0.0006) [2023-12-27 16:50:46,376][20347] Updated weights for policy 0, policy_version 1616671 (0.0007) [2023-12-27 16:50:46,424][20347] Updated weights for policy 0, policy_version 1616681 (0.0007) [2023-12-27 16:50:46,794][20381] Updated weights for policy 1, policy_version 1615099 (0.0009) [2023-12-27 16:50:46,841][20381] Updated weights for policy 1, policy_version 1615109 (0.0008) [2023-12-27 16:50:46,906][20381] Updated weights for policy 1, policy_version 1615119 (0.0007) [2023-12-27 16:50:47,096][20347] Updated weights for policy 0, policy_version 1616691 (0.0007) [2023-12-27 16:50:47,157][20347] Updated weights for policy 0, policy_version 1616701 (0.0007) [2023-12-27 16:50:47,217][20347] Updated weights for policy 0, policy_version 1616711 (0.0007) [2023-12-27 16:50:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 827473920. Throughput: 0: 10385.0, 1: 10326.1. Samples: 827481120. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:47,364][19170] Avg episode reward: [(0, '980.480'), (1, '902.310')] [2023-12-27 16:50:47,629][20381] Updated weights for policy 1, policy_version 1615129 (0.0008) [2023-12-27 16:50:47,683][20381] Updated weights for policy 1, policy_version 1615139 (0.0007) [2023-12-27 16:50:47,747][20381] Updated weights for policy 1, policy_version 1615149 (0.0007) [2023-12-27 16:50:47,790][20347] Updated weights for policy 0, policy_version 1616721 (0.0005) [2023-12-27 16:50:47,792][20381] Updated weights for policy 1, policy_version 1615159 (0.0007) [2023-12-27 16:50:47,841][20347] Updated weights for policy 0, policy_version 1616731 (0.0007) [2023-12-27 16:50:47,887][20347] Updated weights for policy 0, policy_version 1616741 (0.0007) [2023-12-27 16:50:47,932][20347] Updated weights for policy 0, policy_version 1616751 (0.0007) [2023-12-27 16:50:48,541][20381] Updated weights for policy 1, policy_version 1615169 (0.0007) [2023-12-27 16:50:48,566][20347] Updated weights for policy 0, policy_version 1616761 (0.0009) [2023-12-27 16:50:48,596][20381] Updated weights for policy 1, policy_version 1615179 (0.0006) [2023-12-27 16:50:48,627][20347] Updated weights for policy 0, policy_version 1616771 (0.0009) [2023-12-27 16:50:48,649][20381] Updated weights for policy 1, policy_version 1615189 (0.0007) [2023-12-27 16:50:48,679][20347] Updated weights for policy 0, policy_version 1616781 (0.0007) [2023-12-27 16:50:49,275][20347] Updated weights for policy 0, policy_version 1616791 (0.0006) [2023-12-27 16:50:49,331][20347] Updated weights for policy 0, policy_version 1616801 (0.0007) [2023-12-27 16:50:49,394][20347] Updated weights for policy 0, policy_version 1616811 (0.0007) [2023-12-27 16:50:49,425][20381] Updated weights for policy 1, policy_version 1615199 (0.0006) [2023-12-27 16:50:49,486][20381] Updated weights for policy 1, policy_version 1615209 (0.0006) [2023-12-27 16:50:49,547][20381] Updated weights for policy 1, policy_version 1615219 (0.0005) [2023-12-27 16:50:50,074][20347] Updated weights for policy 0, policy_version 1616821 (0.0005) [2023-12-27 16:50:50,123][20347] Updated weights for policy 0, policy_version 1616831 (0.0007) [2023-12-27 16:50:50,163][20381] Updated weights for policy 1, policy_version 1615229 (0.0007) [2023-12-27 16:50:50,173][20347] Updated weights for policy 0, policy_version 1616841 (0.0006) [2023-12-27 16:50:50,211][20381] Updated weights for policy 1, policy_version 1615239 (0.0006) [2023-12-27 16:50:50,260][20381] Updated weights for policy 1, policy_version 1615249 (0.0008) [2023-12-27 16:50:50,840][20347] Updated weights for policy 0, policy_version 1616851 (0.0006) [2023-12-27 16:50:50,903][20347] Updated weights for policy 0, policy_version 1616861 (0.0006) [2023-12-27 16:50:50,967][20347] Updated weights for policy 0, policy_version 1616871 (0.0007) [2023-12-27 16:50:51,004][20381] Updated weights for policy 1, policy_version 1615259 (0.0007) [2023-12-27 16:50:51,064][20381] Updated weights for policy 1, policy_version 1615269 (0.0007) [2023-12-27 16:50:51,117][20381] Updated weights for policy 1, policy_version 1615279 (0.0006) [2023-12-27 16:50:51,722][20347] Updated weights for policy 0, policy_version 1616881 (0.0006) [2023-12-27 16:50:51,789][20347] Updated weights for policy 0, policy_version 1616891 (0.0008) [2023-12-27 16:50:51,839][20347] Updated weights for policy 0, policy_version 1616901 (0.0006) [2023-12-27 16:50:51,840][20381] Updated weights for policy 1, policy_version 1615289 (0.0008) [2023-12-27 16:50:51,892][20347] Updated weights for policy 0, policy_version 1616911 (0.0007) [2023-12-27 16:50:51,900][20381] Updated weights for policy 1, policy_version 1615299 (0.0008) [2023-12-27 16:50:51,957][20381] Updated weights for policy 1, policy_version 1615309 (0.0005) [2023-12-27 16:50:52,005][20381] Updated weights for policy 1, policy_version 1615319 (0.0009) [2023-12-27 16:50:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 827572224. Throughput: 0: 10418.1, 1: 10251.8. Samples: 827543468. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:52,363][19170] Avg episode reward: [(0, '970.720'), (1, '902.230')] [2023-12-27 16:50:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001615320_413564928.pth... [2023-12-27 16:50:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001616912_414007296.pth... [2023-12-27 16:50:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001614136_413261824.pth [2023-12-27 16:50:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001615696_413696000.pth [2023-12-27 16:50:52,572][20347] Updated weights for policy 0, policy_version 1616921 (0.0008) [2023-12-27 16:50:52,625][20347] Updated weights for policy 0, policy_version 1616931 (0.0007) [2023-12-27 16:50:52,654][20381] Updated weights for policy 1, policy_version 1615329 (0.0010) [2023-12-27 16:50:52,678][20347] Updated weights for policy 0, policy_version 1616941 (0.0006) [2023-12-27 16:50:52,710][20381] Updated weights for policy 1, policy_version 1615339 (0.0011) [2023-12-27 16:50:52,768][20381] Updated weights for policy 1, policy_version 1615349 (0.0010) [2023-12-27 16:50:53,385][20347] Updated weights for policy 0, policy_version 1616951 (0.0007) [2023-12-27 16:50:53,413][20381] Updated weights for policy 1, policy_version 1615359 (0.0009) [2023-12-27 16:50:53,428][20347] Updated weights for policy 0, policy_version 1616961 (0.0008) [2023-12-27 16:50:53,473][20381] Updated weights for policy 1, policy_version 1615369 (0.0011) [2023-12-27 16:50:53,493][20347] Updated weights for policy 0, policy_version 1616971 (0.0006) [2023-12-27 16:50:53,518][20381] Updated weights for policy 1, policy_version 1615379 (0.0010) [2023-12-27 16:50:54,198][20347] Updated weights for policy 0, policy_version 1616981 (0.0009) [2023-12-27 16:50:54,216][20381] Updated weights for policy 1, policy_version 1615389 (0.0008) [2023-12-27 16:50:54,257][20347] Updated weights for policy 0, policy_version 1616991 (0.0011) [2023-12-27 16:50:54,275][20381] Updated weights for policy 1, policy_version 1615399 (0.0006) [2023-12-27 16:50:54,311][20347] Updated weights for policy 0, policy_version 1617001 (0.0011) [2023-12-27 16:50:54,334][20381] Updated weights for policy 1, policy_version 1615409 (0.0006) [2023-12-27 16:50:55,009][20347] Updated weights for policy 0, policy_version 1617011 (0.0009) [2023-12-27 16:50:55,053][20381] Updated weights for policy 1, policy_version 1615419 (0.0008) [2023-12-27 16:50:55,075][20347] Updated weights for policy 0, policy_version 1617021 (0.0006) [2023-12-27 16:50:55,099][20381] Updated weights for policy 1, policy_version 1615429 (0.0008) [2023-12-27 16:50:55,135][20347] Updated weights for policy 0, policy_version 1617031 (0.0007) [2023-12-27 16:50:55,144][20381] Updated weights for policy 1, policy_version 1615439 (0.0008) [2023-12-27 16:50:55,736][20347] Updated weights for policy 0, policy_version 1617041 (0.0006) [2023-12-27 16:50:55,785][20347] Updated weights for policy 0, policy_version 1617051 (0.0008) [2023-12-27 16:50:55,826][20381] Updated weights for policy 1, policy_version 1615449 (0.0006) [2023-12-27 16:50:55,842][20347] Updated weights for policy 0, policy_version 1617061 (0.0007) [2023-12-27 16:50:55,883][20381] Updated weights for policy 1, policy_version 1615459 (0.0007) [2023-12-27 16:50:55,902][20347] Updated weights for policy 0, policy_version 1617071 (0.0007) [2023-12-27 16:50:55,935][20381] Updated weights for policy 1, policy_version 1615469 (0.0008) [2023-12-27 16:50:55,987][20381] Updated weights for policy 1, policy_version 1615479 (0.0008) [2023-12-27 16:50:56,600][20347] Updated weights for policy 0, policy_version 1617081 (0.0007) [2023-12-27 16:50:56,620][20381] Updated weights for policy 1, policy_version 1615489 (0.0006) [2023-12-27 16:50:56,650][20347] Updated weights for policy 0, policy_version 1617091 (0.0007) [2023-12-27 16:50:56,671][20381] Updated weights for policy 1, policy_version 1615499 (0.0005) [2023-12-27 16:50:56,702][20347] Updated weights for policy 0, policy_version 1617101 (0.0007) [2023-12-27 16:50:56,720][20381] Updated weights for policy 1, policy_version 1615509 (0.0005) [2023-12-27 16:50:57,238][20381] Updated weights for policy 1, policy_version 1615519 (0.0009) [2023-12-27 16:50:57,291][20381] Updated weights for policy 1, policy_version 1615529 (0.0010) [2023-12-27 16:50:57,350][20381] Updated weights for policy 1, policy_version 1615539 (0.0010) [2023-12-27 16:50:57,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 827670528. Throughput: 0: 10450.8, 1: 10185.2. Samples: 827664228. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:50:57,363][19170] Avg episode reward: [(0, '990.300'), (1, '863.160')] [2023-12-27 16:50:57,470][20347] Updated weights for policy 0, policy_version 1617112 (0.0008) [2023-12-27 16:50:57,520][20347] Updated weights for policy 0, policy_version 1617122 (0.0007) [2023-12-27 16:50:57,565][20347] Updated weights for policy 0, policy_version 1617132 (0.0007) [2023-12-27 16:50:58,024][20381] Updated weights for policy 1, policy_version 1615549 (0.0010) [2023-12-27 16:50:58,068][20381] Updated weights for policy 1, policy_version 1615559 (0.0010) [2023-12-27 16:50:58,119][20381] Updated weights for policy 1, policy_version 1615569 (0.0010) [2023-12-27 16:50:58,223][20347] Updated weights for policy 0, policy_version 1617142 (0.0008) [2023-12-27 16:50:58,278][20347] Updated weights for policy 0, policy_version 1617152 (0.0007) [2023-12-27 16:50:58,340][20347] Updated weights for policy 0, policy_version 1617162 (0.0007) [2023-12-27 16:50:58,979][20381] Updated weights for policy 1, policy_version 1615579 (0.0009) [2023-12-27 16:50:59,048][20381] Updated weights for policy 1, policy_version 1615589 (0.0010) [2023-12-27 16:50:59,117][20381] Updated weights for policy 1, policy_version 1615599 (0.0009) [2023-12-27 16:50:59,171][20347] Updated weights for policy 0, policy_version 1617172 (0.0008) [2023-12-27 16:50:59,239][20347] Updated weights for policy 0, policy_version 1617182 (0.0011) [2023-12-27 16:50:59,302][20347] Updated weights for policy 0, policy_version 1617192 (0.0007) [2023-12-27 16:50:59,841][20381] Updated weights for policy 1, policy_version 1615609 (0.0008) [2023-12-27 16:50:59,901][20381] Updated weights for policy 1, policy_version 1615619 (0.0008) [2023-12-27 16:50:59,962][20381] Updated weights for policy 1, policy_version 1615629 (0.0008) [2023-12-27 16:51:00,018][20381] Updated weights for policy 1, policy_version 1615639 (0.0007) [2023-12-27 16:51:00,035][20347] Updated weights for policy 0, policy_version 1617202 (0.0007) [2023-12-27 16:51:00,083][20347] Updated weights for policy 0, policy_version 1617212 (0.0009) [2023-12-27 16:51:00,138][20347] Updated weights for policy 0, policy_version 1617222 (0.0007) [2023-12-27 16:51:00,201][20347] Updated weights for policy 0, policy_version 1617232 (0.0005) [2023-12-27 16:51:00,684][20381] Updated weights for policy 1, policy_version 1615649 (0.0005) [2023-12-27 16:51:00,732][20381] Updated weights for policy 1, policy_version 1615659 (0.0007) [2023-12-27 16:51:00,783][20381] Updated weights for policy 1, policy_version 1615669 (0.0005) [2023-12-27 16:51:00,876][20347] Updated weights for policy 0, policy_version 1617242 (0.0007) [2023-12-27 16:51:00,924][20347] Updated weights for policy 0, policy_version 1617252 (0.0007) [2023-12-27 16:51:00,978][20347] Updated weights for policy 0, policy_version 1617262 (0.0007) [2023-12-27 16:51:01,435][20381] Updated weights for policy 1, policy_version 1615679 (0.0006) [2023-12-27 16:51:01,495][20381] Updated weights for policy 1, policy_version 1615689 (0.0006) [2023-12-27 16:51:01,553][20381] Updated weights for policy 1, policy_version 1615699 (0.0005) [2023-12-27 16:51:01,739][20347] Updated weights for policy 0, policy_version 1617272 (0.0009) [2023-12-27 16:51:01,806][20347] Updated weights for policy 0, policy_version 1617282 (0.0008) [2023-12-27 16:51:01,873][20347] Updated weights for policy 0, policy_version 1617292 (0.0008) [2023-12-27 16:51:02,224][20381] Updated weights for policy 1, policy_version 1615709 (0.0006) [2023-12-27 16:51:02,286][20381] Updated weights for policy 1, policy_version 1615719 (0.0009) [2023-12-27 16:51:02,345][20381] Updated weights for policy 1, policy_version 1615729 (0.0008) [2023-12-27 16:51:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 827768832. Throughput: 0: 10319.8, 1: 10250.3. Samples: 827783812. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:02,363][19170] Avg episode reward: [(0, '970.880'), (1, '873.050')] [2023-12-27 16:51:02,541][20347] Updated weights for policy 0, policy_version 1617302 (0.0006) [2023-12-27 16:51:02,595][20347] Updated weights for policy 0, policy_version 1617312 (0.0007) [2023-12-27 16:51:02,650][20291] KL-divergence is very high: 436.9013 [2023-12-27 16:51:02,655][20347] Updated weights for policy 0, policy_version 1617322 (0.0007) [2023-12-27 16:51:02,661][20291] KL-divergence is very high: 295.8435 [2023-12-27 16:51:03,069][20381] Updated weights for policy 1, policy_version 1615739 (0.0009) [2023-12-27 16:51:03,122][20381] Updated weights for policy 1, policy_version 1615749 (0.0008) [2023-12-27 16:51:03,177][20381] Updated weights for policy 1, policy_version 1615759 (0.0007) [2023-12-27 16:51:03,315][20347] Updated weights for policy 0, policy_version 1617332 (0.0007) [2023-12-27 16:51:03,372][20347] Updated weights for policy 0, policy_version 1617342 (0.0007) [2023-12-27 16:51:03,427][20347] Updated weights for policy 0, policy_version 1617352 (0.0007) [2023-12-27 16:51:03,828][20381] Updated weights for policy 1, policy_version 1615769 (0.0008) [2023-12-27 16:51:03,890][20381] Updated weights for policy 1, policy_version 1615779 (0.0008) [2023-12-27 16:51:03,946][20381] Updated weights for policy 1, policy_version 1615789 (0.0007) [2023-12-27 16:51:04,011][20381] Updated weights for policy 1, policy_version 1615799 (0.0007) [2023-12-27 16:51:04,143][20347] Updated weights for policy 0, policy_version 1617362 (0.0008) [2023-12-27 16:51:04,205][20347] Updated weights for policy 0, policy_version 1617372 (0.0010) [2023-12-27 16:51:04,270][20347] Updated weights for policy 0, policy_version 1617382 (0.0011) [2023-12-27 16:51:04,328][20347] Updated weights for policy 0, policy_version 1617392 (0.0011) [2023-12-27 16:51:04,693][20381] Updated weights for policy 1, policy_version 1615809 (0.0007) [2023-12-27 16:51:04,751][20381] Updated weights for policy 1, policy_version 1615819 (0.0008) [2023-12-27 16:51:04,806][20381] Updated weights for policy 1, policy_version 1615829 (0.0006) [2023-12-27 16:51:05,018][20347] Updated weights for policy 0, policy_version 1617402 (0.0008) [2023-12-27 16:51:05,076][20347] Updated weights for policy 0, policy_version 1617412 (0.0008) [2023-12-27 16:51:05,131][20347] Updated weights for policy 0, policy_version 1617422 (0.0008) [2023-12-27 16:51:05,573][20381] Updated weights for policy 1, policy_version 1615839 (0.0007) [2023-12-27 16:51:05,627][20381] Updated weights for policy 1, policy_version 1615849 (0.0008) [2023-12-27 16:51:05,683][20381] Updated weights for policy 1, policy_version 1615859 (0.0009) [2023-12-27 16:51:05,792][20347] Updated weights for policy 0, policy_version 1617432 (0.0007) [2023-12-27 16:51:05,851][20347] Updated weights for policy 0, policy_version 1617442 (0.0007) [2023-12-27 16:51:05,902][20347] Updated weights for policy 0, policy_version 1617452 (0.0007) [2023-12-27 16:51:06,396][20381] Updated weights for policy 1, policy_version 1615869 (0.0007) [2023-12-27 16:51:06,459][20381] Updated weights for policy 1, policy_version 1615879 (0.0005) [2023-12-27 16:51:06,520][20381] Updated weights for policy 1, policy_version 1615889 (0.0007) [2023-12-27 16:51:06,625][20347] Updated weights for policy 0, policy_version 1617462 (0.0008) [2023-12-27 16:51:06,688][20347] Updated weights for policy 0, policy_version 1617472 (0.0008) [2023-12-27 16:51:06,747][20347] Updated weights for policy 0, policy_version 1617482 (0.0008) [2023-12-27 16:51:07,128][20381] Updated weights for policy 1, policy_version 1615899 (0.0009) [2023-12-27 16:51:07,186][20381] Updated weights for policy 1, policy_version 1615909 (0.0010) [2023-12-27 16:51:07,245][20381] Updated weights for policy 1, policy_version 1615919 (0.0010) [2023-12-27 16:51:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 827875328. Throughput: 0: 10312.3, 1: 10208.3. Samples: 827843640. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:07,363][19170] Avg episode reward: [(0, '970.880'), (1, '873.080')] [2023-12-27 16:51:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001615928_413720576.pth... [2023-12-27 16:51:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001617488_414154752.pth... [2023-12-27 16:51:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001614744_413417472.pth [2023-12-27 16:51:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001616304_413851648.pth [2023-12-27 16:51:07,470][20347] Updated weights for policy 0, policy_version 1617492 (0.0007) [2023-12-27 16:51:07,523][20347] Updated weights for policy 0, policy_version 1617502 (0.0007) [2023-12-27 16:51:07,568][20347] Updated weights for policy 0, policy_version 1617512 (0.0007) [2023-12-27 16:51:07,861][20381] Updated weights for policy 1, policy_version 1615929 (0.0009) [2023-12-27 16:51:07,923][20381] Updated weights for policy 1, policy_version 1615939 (0.0006) [2023-12-27 16:51:07,983][20381] Updated weights for policy 1, policy_version 1615949 (0.0006) [2023-12-27 16:51:08,046][20381] Updated weights for policy 1, policy_version 1615959 (0.0006) [2023-12-27 16:51:08,176][20347] Updated weights for policy 0, policy_version 1617522 (0.0007) [2023-12-27 16:51:08,237][20347] Updated weights for policy 0, policy_version 1617532 (0.0006) [2023-12-27 16:51:08,293][20347] Updated weights for policy 0, policy_version 1617542 (0.0005) [2023-12-27 16:51:08,357][20347] Updated weights for policy 0, policy_version 1617552 (0.0007) [2023-12-27 16:51:08,647][20381] Updated weights for policy 1, policy_version 1615969 (0.0007) [2023-12-27 16:51:08,707][20381] Updated weights for policy 1, policy_version 1615979 (0.0011) [2023-12-27 16:51:08,771][20381] Updated weights for policy 1, policy_version 1615989 (0.0010) [2023-12-27 16:51:09,034][20347] Updated weights for policy 0, policy_version 1617562 (0.0007) [2023-12-27 16:51:09,088][20347] Updated weights for policy 0, policy_version 1617572 (0.0007) [2023-12-27 16:51:09,143][20347] Updated weights for policy 0, policy_version 1617582 (0.0008) [2023-12-27 16:51:09,468][20381] Updated weights for policy 1, policy_version 1615999 (0.0009) [2023-12-27 16:51:09,534][20381] Updated weights for policy 1, policy_version 1616009 (0.0008) [2023-12-27 16:51:09,592][20381] Updated weights for policy 1, policy_version 1616019 (0.0008) [2023-12-27 16:51:09,933][20347] Updated weights for policy 0, policy_version 1617592 (0.0008) [2023-12-27 16:51:09,993][20347] Updated weights for policy 0, policy_version 1617602 (0.0007) [2023-12-27 16:51:10,049][20347] Updated weights for policy 0, policy_version 1617612 (0.0007) [2023-12-27 16:51:10,288][20381] Updated weights for policy 1, policy_version 1616029 (0.0007) [2023-12-27 16:51:10,353][20381] Updated weights for policy 1, policy_version 1616039 (0.0006) [2023-12-27 16:51:10,412][20381] Updated weights for policy 1, policy_version 1616049 (0.0007) [2023-12-27 16:51:10,776][20347] Updated weights for policy 0, policy_version 1617622 (0.0007) [2023-12-27 16:51:10,839][20347] Updated weights for policy 0, policy_version 1617632 (0.0006) [2023-12-27 16:51:10,901][20347] Updated weights for policy 0, policy_version 1617642 (0.0007) [2023-12-27 16:51:11,051][20381] Updated weights for policy 1, policy_version 1616059 (0.0008) [2023-12-27 16:51:11,109][20381] Updated weights for policy 1, policy_version 1616069 (0.0008) [2023-12-27 16:51:11,177][20381] Updated weights for policy 1, policy_version 1616079 (0.0008) [2023-12-27 16:51:11,602][20347] Updated weights for policy 0, policy_version 1617652 (0.0008) [2023-12-27 16:51:11,666][20347] Updated weights for policy 0, policy_version 1617662 (0.0009) [2023-12-27 16:51:11,725][20347] Updated weights for policy 0, policy_version 1617672 (0.0007) [2023-12-27 16:51:11,989][20381] Updated weights for policy 1, policy_version 1616089 (0.0008) [2023-12-27 16:51:12,047][20381] Updated weights for policy 1, policy_version 1616099 (0.0007) [2023-12-27 16:51:12,106][20381] Updated weights for policy 1, policy_version 1616109 (0.0005) [2023-12-27 16:51:12,173][20381] Updated weights for policy 1, policy_version 1616119 (0.0006) [2023-12-27 16:51:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 827973632. Throughput: 0: 10327.3, 1: 10208.0. Samples: 827965388. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:12,363][19170] Avg episode reward: [(0, '990.250'), (1, '863.200')] [2023-12-27 16:51:12,424][20347] Updated weights for policy 0, policy_version 1617682 (0.0007) [2023-12-27 16:51:12,489][20347] Updated weights for policy 0, policy_version 1617692 (0.0008) [2023-12-27 16:51:12,539][20347] Updated weights for policy 0, policy_version 1617702 (0.0007) [2023-12-27 16:51:12,592][20347] Updated weights for policy 0, policy_version 1617712 (0.0007) [2023-12-27 16:51:12,787][20381] Updated weights for policy 1, policy_version 1616129 (0.0006) [2023-12-27 16:51:12,840][20381] Updated weights for policy 1, policy_version 1616139 (0.0007) [2023-12-27 16:51:12,896][20381] Updated weights for policy 1, policy_version 1616149 (0.0008) [2023-12-27 16:51:13,301][20347] Updated weights for policy 0, policy_version 1617722 (0.0007) [2023-12-27 16:51:13,356][20347] Updated weights for policy 0, policy_version 1617732 (0.0006) [2023-12-27 16:51:13,419][20347] Updated weights for policy 0, policy_version 1617742 (0.0006) [2023-12-27 16:51:13,520][20381] Updated weights for policy 1, policy_version 1616159 (0.0006) [2023-12-27 16:51:13,586][20381] Updated weights for policy 1, policy_version 1616169 (0.0007) [2023-12-27 16:51:13,647][20381] Updated weights for policy 1, policy_version 1616179 (0.0010) [2023-12-27 16:51:13,936][20347] Updated weights for policy 0, policy_version 1617752 (0.0005) [2023-12-27 16:51:13,986][20347] Updated weights for policy 0, policy_version 1617762 (0.0006) [2023-12-27 16:51:14,034][20347] Updated weights for policy 0, policy_version 1617772 (0.0006) [2023-12-27 16:51:14,293][20381] Updated weights for policy 1, policy_version 1616189 (0.0011) [2023-12-27 16:51:14,354][20381] Updated weights for policy 1, policy_version 1616199 (0.0011) [2023-12-27 16:51:14,415][20381] Updated weights for policy 1, policy_version 1616209 (0.0011) [2023-12-27 16:51:14,685][20347] Updated weights for policy 0, policy_version 1617782 (0.0008) [2023-12-27 16:51:14,751][20347] Updated weights for policy 0, policy_version 1617792 (0.0007) [2023-12-27 16:51:14,815][20347] Updated weights for policy 0, policy_version 1617802 (0.0007) [2023-12-27 16:51:15,080][20381] Updated weights for policy 1, policy_version 1616219 (0.0009) [2023-12-27 16:51:15,150][20381] Updated weights for policy 1, policy_version 1616229 (0.0007) [2023-12-27 16:51:15,215][20381] Updated weights for policy 1, policy_version 1616239 (0.0007) [2023-12-27 16:51:15,534][20347] Updated weights for policy 0, policy_version 1617812 (0.0007) [2023-12-27 16:51:15,586][20347] Updated weights for policy 0, policy_version 1617822 (0.0007) [2023-12-27 16:51:15,644][20347] Updated weights for policy 0, policy_version 1617832 (0.0007) [2023-12-27 16:51:15,846][20381] Updated weights for policy 1, policy_version 1616249 (0.0007) [2023-12-27 16:51:15,902][20381] Updated weights for policy 1, policy_version 1616259 (0.0008) [2023-12-27 16:51:15,954][20381] Updated weights for policy 1, policy_version 1616269 (0.0008) [2023-12-27 16:51:16,013][20381] Updated weights for policy 1, policy_version 1616279 (0.0008) [2023-12-27 16:51:16,332][20347] Updated weights for policy 0, policy_version 1617842 (0.0007) [2023-12-27 16:51:16,389][20347] Updated weights for policy 0, policy_version 1617852 (0.0008) [2023-12-27 16:51:16,444][20347] Updated weights for policy 0, policy_version 1617862 (0.0007) [2023-12-27 16:51:16,499][20347] Updated weights for policy 0, policy_version 1617872 (0.0008) [2023-12-27 16:51:16,698][20381] Updated weights for policy 1, policy_version 1616289 (0.0005) [2023-12-27 16:51:16,756][20381] Updated weights for policy 1, policy_version 1616299 (0.0010) [2023-12-27 16:51:16,815][20381] Updated weights for policy 1, policy_version 1616309 (0.0011) [2023-12-27 16:51:17,194][20347] Updated weights for policy 0, policy_version 1617882 (0.0008) [2023-12-27 16:51:17,243][20347] Updated weights for policy 0, policy_version 1617892 (0.0007) [2023-12-27 16:51:17,288][20347] Updated weights for policy 0, policy_version 1617902 (0.0007) [2023-12-27 16:51:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 828080128. Throughput: 0: 10229.6, 1: 10217.0. Samples: 828089360. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:17,363][19170] Avg episode reward: [(0, '990.200'), (1, '804.900')] [2023-12-27 16:51:17,463][20381] Updated weights for policy 1, policy_version 1616319 (0.0008) [2023-12-27 16:51:17,520][20381] Updated weights for policy 1, policy_version 1616329 (0.0008) [2023-12-27 16:51:17,578][20381] Updated weights for policy 1, policy_version 1616339 (0.0008) [2023-12-27 16:51:18,015][20347] Updated weights for policy 0, policy_version 1617912 (0.0008) [2023-12-27 16:51:18,073][20347] Updated weights for policy 0, policy_version 1617922 (0.0008) [2023-12-27 16:51:18,128][20347] Updated weights for policy 0, policy_version 1617932 (0.0008) [2023-12-27 16:51:18,250][20381] Updated weights for policy 1, policy_version 1616349 (0.0007) [2023-12-27 16:51:18,299][20381] Updated weights for policy 1, policy_version 1616359 (0.0008) [2023-12-27 16:51:18,353][20381] Updated weights for policy 1, policy_version 1616369 (0.0007) [2023-12-27 16:51:18,805][20347] Updated weights for policy 0, policy_version 1617942 (0.0007) [2023-12-27 16:51:18,859][20347] Updated weights for policy 0, policy_version 1617952 (0.0008) [2023-12-27 16:51:18,907][20347] Updated weights for policy 0, policy_version 1617962 (0.0008) [2023-12-27 16:51:19,028][20381] Updated weights for policy 1, policy_version 1616379 (0.0009) [2023-12-27 16:51:19,084][20381] Updated weights for policy 1, policy_version 1616389 (0.0007) [2023-12-27 16:51:19,149][20381] Updated weights for policy 1, policy_version 1616399 (0.0007) [2023-12-27 16:51:19,631][20347] Updated weights for policy 0, policy_version 1617972 (0.0008) [2023-12-27 16:51:19,688][20347] Updated weights for policy 0, policy_version 1617982 (0.0007) [2023-12-27 16:51:19,747][20347] Updated weights for policy 0, policy_version 1617992 (0.0007) [2023-12-27 16:51:19,876][20381] Updated weights for policy 1, policy_version 1616409 (0.0006) [2023-12-27 16:51:19,941][20381] Updated weights for policy 1, policy_version 1616419 (0.0011) [2023-12-27 16:51:20,001][20381] Updated weights for policy 1, policy_version 1616429 (0.0011) [2023-12-27 16:51:20,063][20381] Updated weights for policy 1, policy_version 1616439 (0.0011) [2023-12-27 16:51:20,491][20347] Updated weights for policy 0, policy_version 1618002 (0.0007) [2023-12-27 16:51:20,552][20347] Updated weights for policy 0, policy_version 1618012 (0.0008) [2023-12-27 16:51:20,614][20347] Updated weights for policy 0, policy_version 1618022 (0.0007) [2023-12-27 16:51:20,675][20347] Updated weights for policy 0, policy_version 1618032 (0.0007) [2023-12-27 16:51:20,723][20381] Updated weights for policy 1, policy_version 1616449 (0.0007) [2023-12-27 16:51:20,783][20381] Updated weights for policy 1, policy_version 1616459 (0.0008) [2023-12-27 16:51:20,843][20381] Updated weights for policy 1, policy_version 1616469 (0.0007) [2023-12-27 16:51:21,430][20347] Updated weights for policy 0, policy_version 1618042 (0.0009) [2023-12-27 16:51:21,486][20347] Updated weights for policy 0, policy_version 1618052 (0.0008) [2023-12-27 16:51:21,506][20381] Updated weights for policy 1, policy_version 1616479 (0.0006) [2023-12-27 16:51:21,551][20347] Updated weights for policy 0, policy_version 1618062 (0.0007) [2023-12-27 16:51:21,559][20381] Updated weights for policy 1, policy_version 1616489 (0.0007) [2023-12-27 16:51:21,625][20381] Updated weights for policy 1, policy_version 1616499 (0.0008) [2023-12-27 16:51:22,280][20347] Updated weights for policy 0, policy_version 1618072 (0.0010) [2023-12-27 16:51:22,339][20381] Updated weights for policy 1, policy_version 1616509 (0.0007) [2023-12-27 16:51:22,346][20347] Updated weights for policy 0, policy_version 1618082 (0.0007) [2023-12-27 16:51:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 828170240. Throughput: 0: 10167.5, 1: 10191.8. Samples: 828148492. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:22,363][19170] Avg episode reward: [(0, '990.200'), (1, '813.890')] [2023-12-27 16:51:22,401][20381] Updated weights for policy 1, policy_version 1616519 (0.0008) [2023-12-27 16:51:22,409][20347] Updated weights for policy 0, policy_version 1618092 (0.0008) [2023-12-27 16:51:22,428][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001618096_414310400.pth... [2023-12-27 16:51:22,431][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001616912_414007296.pth [2023-12-27 16:51:22,453][20381] Updated weights for policy 1, policy_version 1616529 (0.0008) [2023-12-27 16:51:22,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001616536_413876224.pth... [2023-12-27 16:51:22,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001615320_413564928.pth [2023-12-27 16:51:23,101][20347] Updated weights for policy 0, policy_version 1618102 (0.0007) [2023-12-27 16:51:23,154][20381] Updated weights for policy 1, policy_version 1616539 (0.0007) [2023-12-27 16:51:23,160][20347] Updated weights for policy 0, policy_version 1618112 (0.0007) [2023-12-27 16:51:23,206][20381] Updated weights for policy 1, policy_version 1616549 (0.0006) [2023-12-27 16:51:23,213][20347] Updated weights for policy 0, policy_version 1618122 (0.0005) [2023-12-27 16:51:23,262][20381] Updated weights for policy 1, policy_version 1616559 (0.0007) [2023-12-27 16:51:23,901][20347] Updated weights for policy 0, policy_version 1618132 (0.0006) [2023-12-27 16:51:23,935][20381] Updated weights for policy 1, policy_version 1616569 (0.0005) [2023-12-27 16:51:23,959][20347] Updated weights for policy 0, policy_version 1618142 (0.0008) [2023-12-27 16:51:23,991][20381] Updated weights for policy 1, policy_version 1616579 (0.0005) [2023-12-27 16:51:24,018][20347] Updated weights for policy 0, policy_version 1618152 (0.0007) [2023-12-27 16:51:24,050][20381] Updated weights for policy 1, policy_version 1616589 (0.0010) [2023-12-27 16:51:24,108][20381] Updated weights for policy 1, policy_version 1616599 (0.0010) [2023-12-27 16:51:24,605][20347] Updated weights for policy 0, policy_version 1618162 (0.0006) [2023-12-27 16:51:24,665][20347] Updated weights for policy 0, policy_version 1618172 (0.0006) [2023-12-27 16:51:24,716][20347] Updated weights for policy 0, policy_version 1618182 (0.0007) [2023-12-27 16:51:24,772][20347] Updated weights for policy 0, policy_version 1618192 (0.0006) [2023-12-27 16:51:24,783][20381] Updated weights for policy 1, policy_version 1616609 (0.0010) [2023-12-27 16:51:24,832][20381] Updated weights for policy 1, policy_version 1616619 (0.0010) [2023-12-27 16:51:24,887][20381] Updated weights for policy 1, policy_version 1616629 (0.0010) [2023-12-27 16:51:25,414][20347] Updated weights for policy 0, policy_version 1618202 (0.0007) [2023-12-27 16:51:25,465][20347] Updated weights for policy 0, policy_version 1618212 (0.0007) [2023-12-27 16:51:25,526][20347] Updated weights for policy 0, policy_version 1618222 (0.0006) [2023-12-27 16:51:25,593][20381] Updated weights for policy 1, policy_version 1616639 (0.0010) [2023-12-27 16:51:25,651][20381] Updated weights for policy 1, policy_version 1616649 (0.0007) [2023-12-27 16:51:25,705][20381] Updated weights for policy 1, policy_version 1616659 (0.0005) [2023-12-27 16:51:26,199][20347] Updated weights for policy 0, policy_version 1618232 (0.0007) [2023-12-27 16:51:26,253][20347] Updated weights for policy 0, policy_version 1618242 (0.0007) [2023-12-27 16:51:26,304][20347] Updated weights for policy 0, policy_version 1618252 (0.0007) [2023-12-27 16:51:26,364][20381] Updated weights for policy 1, policy_version 1616669 (0.0010) [2023-12-27 16:51:26,409][20381] Updated weights for policy 1, policy_version 1616679 (0.0010) [2023-12-27 16:51:26,457][20381] Updated weights for policy 1, policy_version 1616689 (0.0010) [2023-12-27 16:51:26,987][20347] Updated weights for policy 0, policy_version 1618262 (0.0007) [2023-12-27 16:51:27,036][20347] Updated weights for policy 0, policy_version 1618272 (0.0007) [2023-12-27 16:51:27,085][20347] Updated weights for policy 0, policy_version 1618282 (0.0007) [2023-12-27 16:51:27,162][20381] Updated weights for policy 1, policy_version 1616699 (0.0010) [2023-12-27 16:51:27,219][20381] Updated weights for policy 1, policy_version 1616709 (0.0009) [2023-12-27 16:51:27,266][20381] Updated weights for policy 1, policy_version 1616719 (0.0006) [2023-12-27 16:51:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 828284928. Throughput: 0: 10190.1, 1: 10133.0. Samples: 828270036. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '902.210')] [2023-12-27 16:51:27,791][20347] Updated weights for policy 0, policy_version 1618292 (0.0006) [2023-12-27 16:51:27,846][20347] Updated weights for policy 0, policy_version 1618302 (0.0007) [2023-12-27 16:51:27,860][20381] Updated weights for policy 1, policy_version 1616729 (0.0005) [2023-12-27 16:51:27,902][20347] Updated weights for policy 0, policy_version 1618312 (0.0007) [2023-12-27 16:51:27,913][20381] Updated weights for policy 1, policy_version 1616739 (0.0005) [2023-12-27 16:51:27,962][20381] Updated weights for policy 1, policy_version 1616749 (0.0006) [2023-12-27 16:51:28,010][20381] Updated weights for policy 1, policy_version 1616759 (0.0007) [2023-12-27 16:51:28,604][20347] Updated weights for policy 0, policy_version 1618322 (0.0007) [2023-12-27 16:51:28,665][20347] Updated weights for policy 0, policy_version 1618332 (0.0006) [2023-12-27 16:51:28,682][20381] Updated weights for policy 1, policy_version 1616769 (0.0011) [2023-12-27 16:51:28,721][20347] Updated weights for policy 0, policy_version 1618342 (0.0005) [2023-12-27 16:51:28,738][20381] Updated weights for policy 1, policy_version 1616779 (0.0011) [2023-12-27 16:51:28,777][20347] Updated weights for policy 0, policy_version 1618352 (0.0006) [2023-12-27 16:51:28,795][20381] Updated weights for policy 1, policy_version 1616789 (0.0009) [2023-12-27 16:51:29,405][20347] Updated weights for policy 0, policy_version 1618362 (0.0008) [2023-12-27 16:51:29,461][20347] Updated weights for policy 0, policy_version 1618372 (0.0008) [2023-12-27 16:51:29,474][20381] Updated weights for policy 1, policy_version 1616799 (0.0011) [2023-12-27 16:51:29,517][20347] Updated weights for policy 0, policy_version 1618382 (0.0008) [2023-12-27 16:51:29,527][20381] Updated weights for policy 1, policy_version 1616809 (0.0011) [2023-12-27 16:51:29,579][20381] Updated weights for policy 1, policy_version 1616819 (0.0010) [2023-12-27 16:51:30,267][20381] Updated weights for policy 1, policy_version 1616829 (0.0007) [2023-12-27 16:51:30,270][20347] Updated weights for policy 0, policy_version 1618392 (0.0006) [2023-12-27 16:51:30,323][20381] Updated weights for policy 1, policy_version 1616839 (0.0007) [2023-12-27 16:51:30,328][20347] Updated weights for policy 0, policy_version 1618402 (0.0006) [2023-12-27 16:51:30,383][20381] Updated weights for policy 1, policy_version 1616849 (0.0006) [2023-12-27 16:51:30,383][20347] Updated weights for policy 0, policy_version 1618412 (0.0007) [2023-12-27 16:51:30,973][20347] Updated weights for policy 0, policy_version 1618422 (0.0007) [2023-12-27 16:51:30,975][20381] Updated weights for policy 1, policy_version 1616859 (0.0008) [2023-12-27 16:51:31,025][20347] Updated weights for policy 0, policy_version 1618432 (0.0008) [2023-12-27 16:51:31,031][20381] Updated weights for policy 1, policy_version 1616869 (0.0009) [2023-12-27 16:51:31,088][20381] Updated weights for policy 1, policy_version 1616879 (0.0011) [2023-12-27 16:51:31,098][20347] Updated weights for policy 0, policy_version 1618443 (0.0007) [2023-12-27 16:51:31,763][20381] Updated weights for policy 1, policy_version 1616889 (0.0010) [2023-12-27 16:51:31,824][20347] Updated weights for policy 0, policy_version 1618453 (0.0007) [2023-12-27 16:51:31,824][20381] Updated weights for policy 1, policy_version 1616899 (0.0008) [2023-12-27 16:51:31,877][20347] Updated weights for policy 0, policy_version 1618463 (0.0006) [2023-12-27 16:51:31,879][20381] Updated weights for policy 1, policy_version 1616909 (0.0008) [2023-12-27 16:51:31,928][20347] Updated weights for policy 0, policy_version 1618473 (0.0005) [2023-12-27 16:51:31,933][20381] Updated weights for policy 1, policy_version 1616919 (0.0007) [2023-12-27 16:51:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 828383232. Throughput: 0: 10081.7, 1: 10209.5. Samples: 828394224. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:32,363][19170] Avg episode reward: [(0, '980.540'), (1, '796.170')] [2023-12-27 16:51:32,635][20347] Updated weights for policy 0, policy_version 1618483 (0.0007) [2023-12-27 16:51:32,640][20381] Updated weights for policy 1, policy_version 1616929 (0.0007) [2023-12-27 16:51:32,696][20347] Updated weights for policy 0, policy_version 1618493 (0.0005) [2023-12-27 16:51:32,702][20381] Updated weights for policy 1, policy_version 1616939 (0.0007) [2023-12-27 16:51:32,754][20347] Updated weights for policy 0, policy_version 1618503 (0.0006) [2023-12-27 16:51:32,763][20381] Updated weights for policy 1, policy_version 1616949 (0.0008) [2023-12-27 16:51:33,329][20347] Updated weights for policy 0, policy_version 1618513 (0.0005) [2023-12-27 16:51:33,374][20347] Updated weights for policy 0, policy_version 1618523 (0.0007) [2023-12-27 16:51:33,421][20347] Updated weights for policy 0, policy_version 1618533 (0.0007) [2023-12-27 16:51:33,431][20381] Updated weights for policy 1, policy_version 1616959 (0.0006) [2023-12-27 16:51:33,465][20347] Updated weights for policy 0, policy_version 1618543 (0.0007) [2023-12-27 16:51:33,482][20381] Updated weights for policy 1, policy_version 1616969 (0.0005) [2023-12-27 16:51:33,534][20381] Updated weights for policy 1, policy_version 1616979 (0.0010) [2023-12-27 16:51:34,065][20381] Updated weights for policy 1, policy_version 1616989 (0.0006) [2023-12-27 16:51:34,127][20381] Updated weights for policy 1, policy_version 1616999 (0.0006) [2023-12-27 16:51:34,190][20381] Updated weights for policy 1, policy_version 1617009 (0.0010) [2023-12-27 16:51:34,264][20347] Updated weights for policy 0, policy_version 1618553 (0.0008) [2023-12-27 16:51:34,325][20347] Updated weights for policy 0, policy_version 1618563 (0.0008) [2023-12-27 16:51:34,384][20347] Updated weights for policy 0, policy_version 1618573 (0.0007) [2023-12-27 16:51:34,799][20381] Updated weights for policy 1, policy_version 1617019 (0.0006) [2023-12-27 16:51:34,858][20381] Updated weights for policy 1, policy_version 1617029 (0.0005) [2023-12-27 16:51:34,916][20381] Updated weights for policy 1, policy_version 1617039 (0.0005) [2023-12-27 16:51:35,010][20347] Updated weights for policy 0, policy_version 1618583 (0.0008) [2023-12-27 16:51:35,065][20347] Updated weights for policy 0, policy_version 1618593 (0.0007) [2023-12-27 16:51:35,116][20347] Updated weights for policy 0, policy_version 1618603 (0.0008) [2023-12-27 16:51:35,528][20381] Updated weights for policy 1, policy_version 1617049 (0.0006) [2023-12-27 16:51:35,589][20381] Updated weights for policy 1, policy_version 1617059 (0.0008) [2023-12-27 16:51:35,640][20381] Updated weights for policy 1, policy_version 1617069 (0.0009) [2023-12-27 16:51:35,686][20381] Updated weights for policy 1, policy_version 1617079 (0.0010) [2023-12-27 16:51:35,851][20347] Updated weights for policy 0, policy_version 1618613 (0.0007) [2023-12-27 16:51:35,911][20347] Updated weights for policy 0, policy_version 1618623 (0.0008) [2023-12-27 16:51:35,970][20347] Updated weights for policy 0, policy_version 1618633 (0.0007) [2023-12-27 16:51:36,367][20381] Updated weights for policy 1, policy_version 1617089 (0.0010) [2023-12-27 16:51:36,434][20381] Updated weights for policy 1, policy_version 1617099 (0.0007) [2023-12-27 16:51:36,503][20381] Updated weights for policy 1, policy_version 1617109 (0.0007) [2023-12-27 16:51:36,654][20347] Updated weights for policy 0, policy_version 1618643 (0.0007) [2023-12-27 16:51:36,717][20347] Updated weights for policy 0, policy_version 1618653 (0.0006) [2023-12-27 16:51:36,770][20347] Updated weights for policy 0, policy_version 1618663 (0.0007) [2023-12-27 16:51:37,139][20381] Updated weights for policy 1, policy_version 1617119 (0.0010) [2023-12-27 16:51:37,198][20381] Updated weights for policy 1, policy_version 1617129 (0.0011) [2023-12-27 16:51:37,253][20381] Updated weights for policy 1, policy_version 1617139 (0.0011) [2023-12-27 16:51:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 828489728. Throughput: 0: 10039.4, 1: 10292.3. Samples: 828458392. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:37,363][19170] Avg episode reward: [(0, '960.950'), (1, '805.640')] [2023-12-27 16:51:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001617144_414031872.pth... [2023-12-27 16:51:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001618672_414457856.pth... [2023-12-27 16:51:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001615928_413720576.pth [2023-12-27 16:51:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001617488_414154752.pth [2023-12-27 16:51:37,404][20347] Updated weights for policy 0, policy_version 1618673 (0.0009) [2023-12-27 16:51:37,459][20347] Updated weights for policy 0, policy_version 1618683 (0.0007) [2023-12-27 16:51:37,507][20347] Updated weights for policy 0, policy_version 1618693 (0.0007) [2023-12-27 16:51:37,555][20347] Updated weights for policy 0, policy_version 1618703 (0.0007) [2023-12-27 16:51:37,941][20381] Updated weights for policy 1, policy_version 1617149 (0.0011) [2023-12-27 16:51:37,990][20381] Updated weights for policy 1, policy_version 1617159 (0.0011) [2023-12-27 16:51:38,039][20381] Updated weights for policy 1, policy_version 1617169 (0.0010) [2023-12-27 16:51:38,230][20347] Updated weights for policy 0, policy_version 1618713 (0.0009) [2023-12-27 16:51:38,284][20347] Updated weights for policy 0, policy_version 1618723 (0.0006) [2023-12-27 16:51:38,360][20347] Updated weights for policy 0, policy_version 1618735 (0.0009) [2023-12-27 16:51:38,734][20381] Updated weights for policy 1, policy_version 1617179 (0.0011) [2023-12-27 16:51:38,788][20381] Updated weights for policy 1, policy_version 1617189 (0.0007) [2023-12-27 16:51:38,836][20381] Updated weights for policy 1, policy_version 1617199 (0.0008) [2023-12-27 16:51:39,036][20347] Updated weights for policy 0, policy_version 1618745 (0.0008) [2023-12-27 16:51:39,087][20347] Updated weights for policy 0, policy_version 1618755 (0.0008) [2023-12-27 16:51:39,136][20347] Updated weights for policy 0, policy_version 1618765 (0.0009) [2023-12-27 16:51:39,552][20381] Updated weights for policy 1, policy_version 1617209 (0.0007) [2023-12-27 16:51:39,619][20381] Updated weights for policy 1, policy_version 1617219 (0.0008) [2023-12-27 16:51:39,682][20381] Updated weights for policy 1, policy_version 1617229 (0.0008) [2023-12-27 16:51:39,743][20381] Updated weights for policy 1, policy_version 1617239 (0.0008) [2023-12-27 16:51:39,909][20347] Updated weights for policy 0, policy_version 1618775 (0.0011) [2023-12-27 16:51:39,978][20347] Updated weights for policy 0, policy_version 1618785 (0.0010) [2023-12-27 16:51:40,039][20347] Updated weights for policy 0, policy_version 1618795 (0.0011) [2023-12-27 16:51:40,471][20381] Updated weights for policy 1, policy_version 1617249 (0.0008) [2023-12-27 16:51:40,534][20381] Updated weights for policy 1, policy_version 1617259 (0.0008) [2023-12-27 16:51:40,592][20381] Updated weights for policy 1, policy_version 1617269 (0.0007) [2023-12-27 16:51:40,665][20347] Updated weights for policy 0, policy_version 1618805 (0.0010) [2023-12-27 16:51:40,722][20347] Updated weights for policy 0, policy_version 1618815 (0.0010) [2023-12-27 16:51:40,777][20347] Updated weights for policy 0, policy_version 1618825 (0.0010) [2023-12-27 16:51:41,359][20381] Updated weights for policy 1, policy_version 1617279 (0.0008) [2023-12-27 16:51:41,425][20381] Updated weights for policy 1, policy_version 1617289 (0.0008) [2023-12-27 16:51:41,484][20381] Updated weights for policy 1, policy_version 1617299 (0.0008) [2023-12-27 16:51:41,520][20347] Updated weights for policy 0, policy_version 1618835 (0.0009) [2023-12-27 16:51:41,570][20347] Updated weights for policy 0, policy_version 1618845 (0.0007) [2023-12-27 16:51:41,643][20347] Updated weights for policy 0, policy_version 1618855 (0.0007) [2023-12-27 16:51:42,214][20381] Updated weights for policy 1, policy_version 1617309 (0.0007) [2023-12-27 16:51:42,274][20381] Updated weights for policy 1, policy_version 1617319 (0.0006) [2023-12-27 16:51:42,327][20347] Updated weights for policy 0, policy_version 1618865 (0.0009) [2023-12-27 16:51:42,340][20381] Updated weights for policy 1, policy_version 1617329 (0.0007) [2023-12-27 16:51:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 828579840. Throughput: 0: 10035.8, 1: 10285.0. Samples: 828578668. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:42,363][19170] Avg episode reward: [(0, '951.150'), (1, '834.210')] [2023-12-27 16:51:42,387][20347] Updated weights for policy 0, policy_version 1618875 (0.0007) [2023-12-27 16:51:42,449][20347] Updated weights for policy 0, policy_version 1618885 (0.0007) [2023-12-27 16:51:42,511][20347] Updated weights for policy 0, policy_version 1618895 (0.0006) [2023-12-27 16:51:42,986][20381] Updated weights for policy 1, policy_version 1617339 (0.0008) [2023-12-27 16:51:43,039][20381] Updated weights for policy 1, policy_version 1617349 (0.0009) [2023-12-27 16:51:43,091][20381] Updated weights for policy 1, policy_version 1617359 (0.0007) [2023-12-27 16:51:43,096][20347] Updated weights for policy 0, policy_version 1618905 (0.0006) [2023-12-27 16:51:43,149][20347] Updated weights for policy 0, policy_version 1618915 (0.0007) [2023-12-27 16:51:43,206][20347] Updated weights for policy 0, policy_version 1618925 (0.0008) [2023-12-27 16:51:43,760][20381] Updated weights for policy 1, policy_version 1617369 (0.0006) [2023-12-27 16:51:43,821][20381] Updated weights for policy 1, policy_version 1617379 (0.0007) [2023-12-27 16:51:43,867][20347] Updated weights for policy 0, policy_version 1618935 (0.0006) [2023-12-27 16:51:43,874][20381] Updated weights for policy 1, policy_version 1617389 (0.0010) [2023-12-27 16:51:43,918][20347] Updated weights for policy 0, policy_version 1618945 (0.0009) [2023-12-27 16:51:43,929][20381] Updated weights for policy 1, policy_version 1617399 (0.0011) [2023-12-27 16:51:43,975][20347] Updated weights for policy 0, policy_version 1618955 (0.0008) [2023-12-27 16:51:44,492][20381] Updated weights for policy 1, policy_version 1617409 (0.0006) [2023-12-27 16:51:44,546][20381] Updated weights for policy 1, policy_version 1617419 (0.0005) [2023-12-27 16:51:44,614][20381] Updated weights for policy 1, policy_version 1617429 (0.0005) [2023-12-27 16:51:44,691][20347] Updated weights for policy 0, policy_version 1618965 (0.0007) [2023-12-27 16:51:44,751][20347] Updated weights for policy 0, policy_version 1618975 (0.0005) [2023-12-27 16:51:44,809][20347] Updated weights for policy 0, policy_version 1618985 (0.0006) [2023-12-27 16:51:45,203][20381] Updated weights for policy 1, policy_version 1617439 (0.0007) [2023-12-27 16:51:45,270][20381] Updated weights for policy 1, policy_version 1617449 (0.0008) [2023-12-27 16:51:45,330][20381] Updated weights for policy 1, policy_version 1617459 (0.0008) [2023-12-27 16:51:45,486][20347] Updated weights for policy 0, policy_version 1618995 (0.0007) [2023-12-27 16:51:45,540][20347] Updated weights for policy 0, policy_version 1619005 (0.0008) [2023-12-27 16:51:45,595][20347] Updated weights for policy 0, policy_version 1619015 (0.0007) [2023-12-27 16:51:45,936][20381] Updated weights for policy 1, policy_version 1617469 (0.0007) [2023-12-27 16:51:45,995][20381] Updated weights for policy 1, policy_version 1617479 (0.0005) [2023-12-27 16:51:46,055][20381] Updated weights for policy 1, policy_version 1617489 (0.0006) [2023-12-27 16:51:46,216][20347] Updated weights for policy 0, policy_version 1619025 (0.0008) [2023-12-27 16:51:46,267][20347] Updated weights for policy 0, policy_version 1619035 (0.0005) [2023-12-27 16:51:46,315][20347] Updated weights for policy 0, policy_version 1619045 (0.0005) [2023-12-27 16:51:46,364][20347] Updated weights for policy 0, policy_version 1619055 (0.0005) [2023-12-27 16:51:46,676][20381] Updated weights for policy 1, policy_version 1617499 (0.0006) [2023-12-27 16:51:46,730][20381] Updated weights for policy 1, policy_version 1617509 (0.0008) [2023-12-27 16:51:46,786][20381] Updated weights for policy 1, policy_version 1617519 (0.0008) [2023-12-27 16:51:46,995][20347] Updated weights for policy 0, policy_version 1619065 (0.0009) [2023-12-27 16:51:47,057][20347] Updated weights for policy 0, policy_version 1619075 (0.0008) [2023-12-27 16:51:47,111][20347] Updated weights for policy 0, policy_version 1619085 (0.0007) [2023-12-27 16:51:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 828694528. Throughput: 0: 10130.0, 1: 10337.0. Samples: 828704828. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:47,363][19170] Avg episode reward: [(0, '931.580'), (1, '864.250')] [2023-12-27 16:51:47,445][20381] Updated weights for policy 1, policy_version 1617529 (0.0008) [2023-12-27 16:51:47,500][20381] Updated weights for policy 1, policy_version 1617539 (0.0011) [2023-12-27 16:51:47,546][20381] Updated weights for policy 1, policy_version 1617549 (0.0010) [2023-12-27 16:51:47,598][20381] Updated weights for policy 1, policy_version 1617559 (0.0010) [2023-12-27 16:51:47,775][20347] Updated weights for policy 0, policy_version 1619095 (0.0009) [2023-12-27 16:51:47,833][20347] Updated weights for policy 0, policy_version 1619105 (0.0007) [2023-12-27 16:51:47,885][20347] Updated weights for policy 0, policy_version 1619115 (0.0005) [2023-12-27 16:51:48,210][20381] Updated weights for policy 1, policy_version 1617569 (0.0005) [2023-12-27 16:51:48,273][20381] Updated weights for policy 1, policy_version 1617579 (0.0006) [2023-12-27 16:51:48,333][20381] Updated weights for policy 1, policy_version 1617589 (0.0007) [2023-12-27 16:51:48,518][20347] Updated weights for policy 0, policy_version 1619125 (0.0008) [2023-12-27 16:51:48,570][20347] Updated weights for policy 0, policy_version 1619135 (0.0010) [2023-12-27 16:51:48,626][20347] Updated weights for policy 0, policy_version 1619145 (0.0010) [2023-12-27 16:51:48,643][20291] KL-divergence is very high: 233.0630 [2023-12-27 16:51:48,871][20381] Updated weights for policy 1, policy_version 1617599 (0.0006) [2023-12-27 16:51:48,922][20381] Updated weights for policy 1, policy_version 1617609 (0.0005) [2023-12-27 16:51:48,976][20381] Updated weights for policy 1, policy_version 1617619 (0.0005) [2023-12-27 16:51:49,322][20347] Updated weights for policy 0, policy_version 1619155 (0.0010) [2023-12-27 16:51:49,340][20291] KL-divergence is very high: 126.8847 [2023-12-27 16:51:49,400][20347] Updated weights for policy 0, policy_version 1619165 (0.0010) [2023-12-27 16:51:49,405][20291] KL-divergence is very high: 120.4005 [2023-12-27 16:51:49,450][20291] KL-divergence is very high: 164.2544 [2023-12-27 16:51:49,456][20347] Updated weights for policy 0, policy_version 1619175 (0.0010) [2023-12-27 16:51:49,497][20291] KL-divergence is very high: 153.2014 [2023-12-27 16:51:49,571][20381] Updated weights for policy 1, policy_version 1617629 (0.0008) [2023-12-27 16:51:49,626][20381] Updated weights for policy 1, policy_version 1617639 (0.0008) [2023-12-27 16:51:49,683][20381] Updated weights for policy 1, policy_version 1617649 (0.0005) [2023-12-27 16:51:50,172][20347] Updated weights for policy 0, policy_version 1619185 (0.0010) [2023-12-27 16:51:50,230][20347] Updated weights for policy 0, policy_version 1619195 (0.0010) [2023-12-27 16:51:50,285][20347] Updated weights for policy 0, policy_version 1619205 (0.0010) [2023-12-27 16:51:50,309][20381] Updated weights for policy 1, policy_version 1617659 (0.0010) [2023-12-27 16:51:50,340][20347] Updated weights for policy 0, policy_version 1619215 (0.0011) [2023-12-27 16:51:50,366][20381] Updated weights for policy 1, policy_version 1617669 (0.0011) [2023-12-27 16:51:50,422][20381] Updated weights for policy 1, policy_version 1617679 (0.0006) [2023-12-27 16:51:50,996][20347] Updated weights for policy 0, policy_version 1619225 (0.0010) [2023-12-27 16:51:51,025][20381] Updated weights for policy 1, policy_version 1617689 (0.0006) [2023-12-27 16:51:51,061][20347] Updated weights for policy 0, policy_version 1619235 (0.0010) [2023-12-27 16:51:51,095][20381] Updated weights for policy 1, policy_version 1617699 (0.0007) [2023-12-27 16:51:51,121][20347] Updated weights for policy 0, policy_version 1619245 (0.0010) [2023-12-27 16:51:51,164][20381] Updated weights for policy 1, policy_version 1617709 (0.0007) [2023-12-27 16:51:51,231][20381] Updated weights for policy 1, policy_version 1617719 (0.0008) [2023-12-27 16:51:51,839][20347] Updated weights for policy 0, policy_version 1619255 (0.0009) [2023-12-27 16:51:51,881][20381] Updated weights for policy 1, policy_version 1617729 (0.0007) [2023-12-27 16:51:51,902][20347] Updated weights for policy 0, policy_version 1619265 (0.0006) [2023-12-27 16:51:51,940][20381] Updated weights for policy 1, policy_version 1617739 (0.0006) [2023-12-27 16:51:51,970][20347] Updated weights for policy 0, policy_version 1619275 (0.0006) [2023-12-27 16:51:52,004][20381] Updated weights for policy 1, policy_version 1617749 (0.0006) [2023-12-27 16:51:52,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 828801024. Throughput: 0: 10142.2, 1: 10433.9. Samples: 828769560. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:52,363][19170] Avg episode reward: [(0, '921.830'), (1, '843.330')] [2023-12-27 16:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001617752_414187520.pth... [2023-12-27 16:51:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001619280_414613504.pth... [2023-12-27 16:51:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001616536_413876224.pth [2023-12-27 16:51:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001618096_414310400.pth [2023-12-27 16:51:52,658][20381] Updated weights for policy 1, policy_version 1617759 (0.0006) [2023-12-27 16:51:52,675][20347] Updated weights for policy 0, policy_version 1619285 (0.0007) [2023-12-27 16:51:52,716][20381] Updated weights for policy 1, policy_version 1617769 (0.0006) [2023-12-27 16:51:52,733][20347] Updated weights for policy 0, policy_version 1619295 (0.0007) [2023-12-27 16:51:52,774][20381] Updated weights for policy 1, policy_version 1617779 (0.0006) [2023-12-27 16:51:52,787][20347] Updated weights for policy 0, policy_version 1619305 (0.0006) [2023-12-27 16:51:53,392][20347] Updated weights for policy 0, policy_version 1619315 (0.0006) [2023-12-27 16:51:53,448][20347] Updated weights for policy 0, policy_version 1619325 (0.0007) [2023-12-27 16:51:53,489][20381] Updated weights for policy 1, policy_version 1617789 (0.0006) [2023-12-27 16:51:53,499][20347] Updated weights for policy 0, policy_version 1619335 (0.0006) [2023-12-27 16:51:53,544][20381] Updated weights for policy 1, policy_version 1617799 (0.0007) [2023-12-27 16:51:53,593][20381] Updated weights for policy 1, policy_version 1617809 (0.0008) [2023-12-27 16:51:54,184][20347] Updated weights for policy 0, policy_version 1619345 (0.0006) [2023-12-27 16:51:54,236][20347] Updated weights for policy 0, policy_version 1619355 (0.0008) [2023-12-27 16:51:54,283][20381] Updated weights for policy 1, policy_version 1617819 (0.0008) [2023-12-27 16:51:54,293][20347] Updated weights for policy 0, policy_version 1619365 (0.0007) [2023-12-27 16:51:54,333][20381] Updated weights for policy 1, policy_version 1617829 (0.0005) [2023-12-27 16:51:54,352][20347] Updated weights for policy 0, policy_version 1619375 (0.0007) [2023-12-27 16:51:54,387][20381] Updated weights for policy 1, policy_version 1617839 (0.0005) [2023-12-27 16:51:55,002][20347] Updated weights for policy 0, policy_version 1619385 (0.0008) [2023-12-27 16:51:55,057][20347] Updated weights for policy 0, policy_version 1619395 (0.0008) [2023-12-27 16:51:55,087][20381] Updated weights for policy 1, policy_version 1617849 (0.0007) [2023-12-27 16:51:55,111][20347] Updated weights for policy 0, policy_version 1619405 (0.0006) [2023-12-27 16:51:55,155][20381] Updated weights for policy 1, policy_version 1617859 (0.0006) [2023-12-27 16:51:55,212][20381] Updated weights for policy 1, policy_version 1617869 (0.0008) [2023-12-27 16:51:55,272][20381] Updated weights for policy 1, policy_version 1617879 (0.0008) [2023-12-27 16:51:55,669][20347] Updated weights for policy 0, policy_version 1619415 (0.0006) [2023-12-27 16:51:55,723][20347] Updated weights for policy 0, policy_version 1619425 (0.0008) [2023-12-27 16:51:55,775][20347] Updated weights for policy 0, policy_version 1619435 (0.0008) [2023-12-27 16:51:55,993][20381] Updated weights for policy 1, policy_version 1617889 (0.0006) [2023-12-27 16:51:56,041][20381] Updated weights for policy 1, policy_version 1617899 (0.0005) [2023-12-27 16:51:56,091][20381] Updated weights for policy 1, policy_version 1617909 (0.0006) [2023-12-27 16:51:56,507][20347] Updated weights for policy 0, policy_version 1619445 (0.0008) [2023-12-27 16:51:56,554][20347] Updated weights for policy 0, policy_version 1619455 (0.0005) [2023-12-27 16:51:56,597][20347] Updated weights for policy 0, policy_version 1619465 (0.0005) [2023-12-27 16:51:56,660][20381] Updated weights for policy 1, policy_version 1617919 (0.0006) [2023-12-27 16:51:56,711][20381] Updated weights for policy 1, policy_version 1617929 (0.0006) [2023-12-27 16:51:56,755][20381] Updated weights for policy 1, policy_version 1617939 (0.0007) [2023-12-27 16:51:57,221][20347] Updated weights for policy 0, policy_version 1619475 (0.0006) [2023-12-27 16:51:57,278][20347] Updated weights for policy 0, policy_version 1619485 (0.0008) [2023-12-27 16:51:57,334][20347] Updated weights for policy 0, policy_version 1619495 (0.0007) [2023-12-27 16:51:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 828899328. Throughput: 0: 10194.3, 1: 10418.2. Samples: 828892952. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:51:57,363][19170] Avg episode reward: [(0, '960.940'), (1, '882.710')] [2023-12-27 16:51:57,425][20381] Updated weights for policy 1, policy_version 1617949 (0.0008) [2023-12-27 16:51:57,471][20381] Updated weights for policy 1, policy_version 1617959 (0.0007) [2023-12-27 16:51:57,516][20381] Updated weights for policy 1, policy_version 1617969 (0.0008) [2023-12-27 16:51:57,990][20347] Updated weights for policy 0, policy_version 1619505 (0.0007) [2023-12-27 16:51:58,041][20347] Updated weights for policy 0, policy_version 1619515 (0.0005) [2023-12-27 16:51:58,085][20347] Updated weights for policy 0, policy_version 1619525 (0.0005) [2023-12-27 16:51:58,131][20347] Updated weights for policy 0, policy_version 1619535 (0.0005) [2023-12-27 16:51:58,148][20381] Updated weights for policy 1, policy_version 1617979 (0.0007) [2023-12-27 16:51:58,213][20381] Updated weights for policy 1, policy_version 1617989 (0.0008) [2023-12-27 16:51:58,273][20381] Updated weights for policy 1, policy_version 1617999 (0.0008) [2023-12-27 16:51:58,844][20347] Updated weights for policy 0, policy_version 1619545 (0.0008) [2023-12-27 16:51:58,905][20347] Updated weights for policy 0, policy_version 1619555 (0.0006) [2023-12-27 16:51:58,977][20347] Updated weights for policy 0, policy_version 1619565 (0.0008) [2023-12-27 16:51:59,071][20381] Updated weights for policy 1, policy_version 1618009 (0.0008) [2023-12-27 16:51:59,130][20381] Updated weights for policy 1, policy_version 1618019 (0.0011) [2023-12-27 16:51:59,195][20381] Updated weights for policy 1, policy_version 1618029 (0.0010) [2023-12-27 16:51:59,263][20381] Updated weights for policy 1, policy_version 1618039 (0.0009) [2023-12-27 16:51:59,673][20347] Updated weights for policy 0, policy_version 1619575 (0.0009) [2023-12-27 16:51:59,729][20347] Updated weights for policy 0, policy_version 1619585 (0.0011) [2023-12-27 16:51:59,787][20347] Updated weights for policy 0, policy_version 1619595 (0.0008) [2023-12-27 16:52:00,009][20381] Updated weights for policy 1, policy_version 1618049 (0.0011) [2023-12-27 16:52:00,055][20381] Updated weights for policy 1, policy_version 1618059 (0.0006) [2023-12-27 16:52:00,114][20381] Updated weights for policy 1, policy_version 1618069 (0.0006) [2023-12-27 16:52:00,477][20347] Updated weights for policy 0, policy_version 1619605 (0.0008) [2023-12-27 16:52:00,537][20347] Updated weights for policy 0, policy_version 1619615 (0.0007) [2023-12-27 16:52:00,593][20347] Updated weights for policy 0, policy_version 1619625 (0.0007) [2023-12-27 16:52:00,787][20381] Updated weights for policy 1, policy_version 1618079 (0.0005) [2023-12-27 16:52:00,842][20381] Updated weights for policy 1, policy_version 1618089 (0.0008) [2023-12-27 16:52:00,899][20381] Updated weights for policy 1, policy_version 1618099 (0.0010) [2023-12-27 16:52:01,254][20347] Updated weights for policy 0, policy_version 1619635 (0.0007) [2023-12-27 16:52:01,320][20347] Updated weights for policy 0, policy_version 1619645 (0.0008) [2023-12-27 16:52:01,384][20347] Updated weights for policy 0, policy_version 1619655 (0.0008) [2023-12-27 16:52:01,564][20381] Updated weights for policy 1, policy_version 1618109 (0.0010) [2023-12-27 16:52:01,627][20381] Updated weights for policy 1, policy_version 1618119 (0.0010) [2023-12-27 16:52:01,689][20381] Updated weights for policy 1, policy_version 1618129 (0.0007) [2023-12-27 16:52:02,003][20347] Updated weights for policy 0, policy_version 1619665 (0.0005) [2023-12-27 16:52:02,055][20347] Updated weights for policy 0, policy_version 1619675 (0.0006) [2023-12-27 16:52:02,120][20347] Updated weights for policy 0, policy_version 1619685 (0.0005) [2023-12-27 16:52:02,185][20347] Updated weights for policy 0, policy_version 1619695 (0.0005) [2023-12-27 16:52:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 829005824. Throughput: 0: 10203.8, 1: 10381.6. Samples: 829015704. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:52:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '873.200')] [2023-12-27 16:52:02,406][20381] Updated weights for policy 1, policy_version 1618139 (0.0011) [2023-12-27 16:52:02,467][20381] Updated weights for policy 1, policy_version 1618149 (0.0011) [2023-12-27 16:52:02,523][20381] Updated weights for policy 1, policy_version 1618159 (0.0010) [2023-12-27 16:52:02,760][20347] Updated weights for policy 0, policy_version 1619705 (0.0007) [2023-12-27 16:52:02,816][20347] Updated weights for policy 0, policy_version 1619715 (0.0008) [2023-12-27 16:52:02,879][20347] Updated weights for policy 0, policy_version 1619725 (0.0007) [2023-12-27 16:52:03,230][20381] Updated weights for policy 1, policy_version 1618169 (0.0011) [2023-12-27 16:52:03,284][20381] Updated weights for policy 1, policy_version 1618179 (0.0007) [2023-12-27 16:52:03,331][20381] Updated weights for policy 1, policy_version 1618189 (0.0008) [2023-12-27 16:52:03,379][20381] Updated weights for policy 1, policy_version 1618199 (0.0007) [2023-12-27 16:52:03,446][20347] Updated weights for policy 0, policy_version 1619735 (0.0006) [2023-12-27 16:52:03,506][20347] Updated weights for policy 0, policy_version 1619745 (0.0006) [2023-12-27 16:52:03,555][20347] Updated weights for policy 0, policy_version 1619755 (0.0005) [2023-12-27 16:52:04,089][20381] Updated weights for policy 1, policy_version 1618209 (0.0011) [2023-12-27 16:52:04,155][20381] Updated weights for policy 1, policy_version 1618219 (0.0011) [2023-12-27 16:52:04,194][20347] Updated weights for policy 0, policy_version 1619765 (0.0008) [2023-12-27 16:52:04,221][20381] Updated weights for policy 1, policy_version 1618229 (0.0011) [2023-12-27 16:52:04,251][20347] Updated weights for policy 0, policy_version 1619775 (0.0010) [2023-12-27 16:52:04,317][20347] Updated weights for policy 0, policy_version 1619785 (0.0008) [2023-12-27 16:52:04,862][20381] Updated weights for policy 1, policy_version 1618239 (0.0007) [2023-12-27 16:52:04,920][20381] Updated weights for policy 1, policy_version 1618249 (0.0005) [2023-12-27 16:52:04,979][20381] Updated weights for policy 1, policy_version 1618259 (0.0006) [2023-12-27 16:52:04,979][20347] Updated weights for policy 0, policy_version 1619795 (0.0007) [2023-12-27 16:52:05,033][20347] Updated weights for policy 0, policy_version 1619805 (0.0008) [2023-12-27 16:52:05,078][20347] Updated weights for policy 0, policy_version 1619815 (0.0010) [2023-12-27 16:52:05,488][20381] Updated weights for policy 1, policy_version 1618269 (0.0007) [2023-12-27 16:52:05,547][20381] Updated weights for policy 1, policy_version 1618279 (0.0006) [2023-12-27 16:52:05,600][20381] Updated weights for policy 1, policy_version 1618289 (0.0006) [2023-12-27 16:52:05,858][20347] Updated weights for policy 0, policy_version 1619825 (0.0010) [2023-12-27 16:52:05,912][20347] Updated weights for policy 0, policy_version 1619835 (0.0006) [2023-12-27 16:52:05,965][20347] Updated weights for policy 0, policy_version 1619845 (0.0006) [2023-12-27 16:52:06,011][20347] Updated weights for policy 0, policy_version 1619855 (0.0010) [2023-12-27 16:52:06,221][20381] Updated weights for policy 1, policy_version 1618299 (0.0007) [2023-12-27 16:52:06,270][20381] Updated weights for policy 1, policy_version 1618309 (0.0010) [2023-12-27 16:52:06,326][20381] Updated weights for policy 1, policy_version 1618319 (0.0010) [2023-12-27 16:52:06,748][20347] Updated weights for policy 0, policy_version 1619865 (0.0010) [2023-12-27 16:52:06,810][20347] Updated weights for policy 0, policy_version 1619875 (0.0010) [2023-12-27 16:52:06,869][20347] Updated weights for policy 0, policy_version 1619885 (0.0010) [2023-12-27 16:52:07,044][20381] Updated weights for policy 1, policy_version 1618329 (0.0011) [2023-12-27 16:52:07,089][20381] Updated weights for policy 1, policy_version 1618339 (0.0010) [2023-12-27 16:52:07,137][20381] Updated weights for policy 1, policy_version 1618349 (0.0010) [2023-12-27 16:52:07,186][20381] Updated weights for policy 1, policy_version 1618359 (0.0010) [2023-12-27 16:52:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 829112320. Throughput: 0: 10281.6, 1: 10418.1. Samples: 829079980. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:52:07,364][19170] Avg episode reward: [(0, '980.510'), (1, '892.680')] [2023-12-27 16:52:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001618360_414343168.pth... [2023-12-27 16:52:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001619888_414769152.pth... [2023-12-27 16:52:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001617144_414031872.pth [2023-12-27 16:52:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001618672_414457856.pth [2023-12-27 16:52:07,530][20347] Updated weights for policy 0, policy_version 1619895 (0.0009) [2023-12-27 16:52:07,585][20347] Updated weights for policy 0, policy_version 1619905 (0.0010) [2023-12-27 16:52:07,630][20347] Updated weights for policy 0, policy_version 1619915 (0.0010) [2023-12-27 16:52:07,855][20381] Updated weights for policy 1, policy_version 1618369 (0.0011) [2023-12-27 16:52:07,913][20381] Updated weights for policy 1, policy_version 1618379 (0.0011) [2023-12-27 16:52:07,971][20381] Updated weights for policy 1, policy_version 1618389 (0.0010) [2023-12-27 16:52:08,229][20347] Updated weights for policy 0, policy_version 1619925 (0.0008) [2023-12-27 16:52:08,277][20347] Updated weights for policy 0, policy_version 1619935 (0.0005) [2023-12-27 16:52:08,335][20347] Updated weights for policy 0, policy_version 1619945 (0.0006) [2023-12-27 16:52:08,604][20381] Updated weights for policy 1, policy_version 1618399 (0.0011) [2023-12-27 16:52:08,649][20381] Updated weights for policy 1, policy_version 1618409 (0.0010) [2023-12-27 16:52:08,701][20381] Updated weights for policy 1, policy_version 1618419 (0.0011) [2023-12-27 16:52:09,039][20347] Updated weights for policy 0, policy_version 1619955 (0.0009) [2023-12-27 16:52:09,096][20347] Updated weights for policy 0, policy_version 1619965 (0.0010) [2023-12-27 16:52:09,153][20347] Updated weights for policy 0, policy_version 1619975 (0.0010) [2023-12-27 16:52:09,437][20381] Updated weights for policy 1, policy_version 1618429 (0.0010) [2023-12-27 16:52:09,497][20381] Updated weights for policy 1, policy_version 1618439 (0.0011) [2023-12-27 16:52:09,559][20381] Updated weights for policy 1, policy_version 1618449 (0.0011) [2023-12-27 16:52:09,875][20347] Updated weights for policy 0, policy_version 1619985 (0.0011) [2023-12-27 16:52:09,940][20347] Updated weights for policy 0, policy_version 1619995 (0.0009) [2023-12-27 16:52:10,007][20347] Updated weights for policy 0, policy_version 1620005 (0.0006) [2023-12-27 16:52:10,070][20347] Updated weights for policy 0, policy_version 1620015 (0.0006) [2023-12-27 16:52:10,271][20381] Updated weights for policy 1, policy_version 1618459 (0.0010) [2023-12-27 16:52:10,332][20381] Updated weights for policy 1, policy_version 1618469 (0.0007) [2023-12-27 16:52:10,395][20381] Updated weights for policy 1, policy_version 1618479 (0.0007) [2023-12-27 16:52:10,683][20347] Updated weights for policy 0, policy_version 1620025 (0.0010) [2023-12-27 16:52:10,748][20347] Updated weights for policy 0, policy_version 1620035 (0.0011) [2023-12-27 16:52:10,803][20347] Updated weights for policy 0, policy_version 1620045 (0.0011) [2023-12-27 16:52:11,032][20381] Updated weights for policy 1, policy_version 1618489 (0.0006) [2023-12-27 16:52:11,102][20381] Updated weights for policy 1, policy_version 1618499 (0.0007) [2023-12-27 16:52:11,174][20381] Updated weights for policy 1, policy_version 1618509 (0.0009) [2023-12-27 16:52:11,237][20381] Updated weights for policy 1, policy_version 1618519 (0.0008) [2023-12-27 16:52:11,557][20347] Updated weights for policy 0, policy_version 1620055 (0.0009) [2023-12-27 16:52:11,622][20347] Updated weights for policy 0, policy_version 1620065 (0.0008) [2023-12-27 16:52:11,686][20347] Updated weights for policy 0, policy_version 1620075 (0.0008) [2023-12-27 16:52:11,966][20381] Updated weights for policy 1, policy_version 1618529 (0.0009) [2023-12-27 16:52:12,023][20381] Updated weights for policy 1, policy_version 1618539 (0.0008) [2023-12-27 16:52:12,081][20381] Updated weights for policy 1, policy_version 1618549 (0.0008) [2023-12-27 16:52:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 829210624. Throughput: 0: 10292.5, 1: 10441.9. Samples: 829203084. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:52:12,363][19170] Avg episode reward: [(0, '980.540'), (1, '902.290')] [2023-12-27 16:52:12,375][20347] Updated weights for policy 0, policy_version 1620085 (0.0009) [2023-12-27 16:52:12,435][20347] Updated weights for policy 0, policy_version 1620095 (0.0009) [2023-12-27 16:52:12,488][20347] Updated weights for policy 0, policy_version 1620105 (0.0008) [2023-12-27 16:52:12,772][20381] Updated weights for policy 1, policy_version 1618559 (0.0009) [2023-12-27 16:52:12,832][20381] Updated weights for policy 1, policy_version 1618569 (0.0009) [2023-12-27 16:52:12,886][20381] Updated weights for policy 1, policy_version 1618579 (0.0009) [2023-12-27 16:52:13,148][20347] Updated weights for policy 0, policy_version 1620115 (0.0009) [2023-12-27 16:52:13,195][20347] Updated weights for policy 0, policy_version 1620125 (0.0007) [2023-12-27 16:52:13,255][20347] Updated weights for policy 0, policy_version 1620135 (0.0007) [2023-12-27 16:52:13,544][20381] Updated weights for policy 1, policy_version 1618589 (0.0008) [2023-12-27 16:52:13,599][20381] Updated weights for policy 1, policy_version 1618599 (0.0007) [2023-12-27 16:52:13,654][20381] Updated weights for policy 1, policy_version 1618609 (0.0005) [2023-12-27 16:52:13,878][20347] Updated weights for policy 0, policy_version 1620145 (0.0007) [2023-12-27 16:52:13,943][20347] Updated weights for policy 0, policy_version 1620155 (0.0006) [2023-12-27 16:52:14,006][20347] Updated weights for policy 0, policy_version 1620165 (0.0008) [2023-12-27 16:52:14,063][20347] Updated weights for policy 0, policy_version 1620175 (0.0007) [2023-12-27 16:52:14,317][20381] Updated weights for policy 1, policy_version 1618619 (0.0008) [2023-12-27 16:52:14,372][20381] Updated weights for policy 1, policy_version 1618629 (0.0008) [2023-12-27 16:52:14,426][20381] Updated weights for policy 1, policy_version 1618639 (0.0007) [2023-12-27 16:52:14,708][20347] Updated weights for policy 0, policy_version 1620185 (0.0005) [2023-12-27 16:52:14,761][20347] Updated weights for policy 0, policy_version 1620195 (0.0007) [2023-12-27 16:52:14,822][20347] Updated weights for policy 0, policy_version 1620205 (0.0006) [2023-12-27 16:52:15,213][20381] Updated weights for policy 1, policy_version 1618649 (0.0008) [2023-12-27 16:52:15,267][20381] Updated weights for policy 1, policy_version 1618659 (0.0009) [2023-12-27 16:52:15,325][20381] Updated weights for policy 1, policy_version 1618669 (0.0009) [2023-12-27 16:52:15,368][20347] Updated weights for policy 0, policy_version 1620215 (0.0005) [2023-12-27 16:52:15,379][20381] Updated weights for policy 1, policy_version 1618679 (0.0008) [2023-12-27 16:52:15,429][20347] Updated weights for policy 0, policy_version 1620225 (0.0005) [2023-12-27 16:52:15,488][20347] Updated weights for policy 0, policy_version 1620235 (0.0007) [2023-12-27 16:52:16,104][20381] Updated weights for policy 1, policy_version 1618689 (0.0008) [2023-12-27 16:52:16,133][20347] Updated weights for policy 0, policy_version 1620245 (0.0007) [2023-12-27 16:52:16,161][20381] Updated weights for policy 1, policy_version 1618699 (0.0005) [2023-12-27 16:52:16,194][20347] Updated weights for policy 0, policy_version 1620255 (0.0007) [2023-12-27 16:52:16,222][20381] Updated weights for policy 1, policy_version 1618709 (0.0007) [2023-12-27 16:52:16,250][20347] Updated weights for policy 0, policy_version 1620265 (0.0006) [2023-12-27 16:52:16,792][20381] Updated weights for policy 1, policy_version 1618719 (0.0005) [2023-12-27 16:52:16,845][20381] Updated weights for policy 1, policy_version 1618729 (0.0005) [2023-12-27 16:52:16,905][20381] Updated weights for policy 1, policy_version 1618739 (0.0010) [2023-12-27 16:52:16,981][20347] Updated weights for policy 0, policy_version 1620275 (0.0008) [2023-12-27 16:52:17,030][20347] Updated weights for policy 0, policy_version 1620285 (0.0008) [2023-12-27 16:52:17,077][20347] Updated weights for policy 0, policy_version 1620295 (0.0007) [2023-12-27 16:52:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 829317120. Throughput: 0: 10329.2, 1: 10387.5. Samples: 829326476. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:52:17,363][19170] Avg episode reward: [(0, '990.260'), (1, '855.500')] [2023-12-27 16:52:17,540][20381] Updated weights for policy 1, policy_version 1618749 (0.0009) [2023-12-27 16:52:17,595][20381] Updated weights for policy 1, policy_version 1618759 (0.0005) [2023-12-27 16:52:17,656][20381] Updated weights for policy 1, policy_version 1618769 (0.0006) [2023-12-27 16:52:17,807][20347] Updated weights for policy 0, policy_version 1620305 (0.0007) [2023-12-27 16:52:17,855][20347] Updated weights for policy 0, policy_version 1620315 (0.0008) [2023-12-27 16:52:17,909][20347] Updated weights for policy 0, policy_version 1620325 (0.0008) [2023-12-27 16:52:17,969][20347] Updated weights for policy 0, policy_version 1620335 (0.0007) [2023-12-27 16:52:18,244][20381] Updated weights for policy 1, policy_version 1618779 (0.0008) [2023-12-27 16:52:18,308][20381] Updated weights for policy 1, policy_version 1618789 (0.0008) [2023-12-27 16:52:18,375][20381] Updated weights for policy 1, policy_version 1618799 (0.0008) [2023-12-27 16:52:18,757][20347] Updated weights for policy 0, policy_version 1620345 (0.0007) [2023-12-27 16:52:18,815][20347] Updated weights for policy 0, policy_version 1620355 (0.0007) [2023-12-27 16:52:18,878][20347] Updated weights for policy 0, policy_version 1620365 (0.0007) [2023-12-27 16:52:19,034][20381] Updated weights for policy 1, policy_version 1618809 (0.0008) [2023-12-27 16:52:19,095][20381] Updated weights for policy 1, policy_version 1618819 (0.0010) [2023-12-27 16:52:19,150][20381] Updated weights for policy 1, policy_version 1618829 (0.0010) [2023-12-27 16:52:19,212][20381] Updated weights for policy 1, policy_version 1618839 (0.0010) [2023-12-27 16:52:19,581][20347] Updated weights for policy 0, policy_version 1620375 (0.0007) [2023-12-27 16:52:19,637][20347] Updated weights for policy 0, policy_version 1620385 (0.0007) [2023-12-27 16:52:19,696][20347] Updated weights for policy 0, policy_version 1620395 (0.0007) [2023-12-27 16:52:19,974][20381] Updated weights for policy 1, policy_version 1618849 (0.0006) [2023-12-27 16:52:20,040][20381] Updated weights for policy 1, policy_version 1618859 (0.0006) [2023-12-27 16:52:20,097][20381] Updated weights for policy 1, policy_version 1618869 (0.0009) [2023-12-27 16:52:20,432][20347] Updated weights for policy 0, policy_version 1620405 (0.0007) [2023-12-27 16:52:20,490][20347] Updated weights for policy 0, policy_version 1620415 (0.0005) [2023-12-27 16:52:20,550][20347] Updated weights for policy 0, policy_version 1620425 (0.0006) [2023-12-27 16:52:20,783][20381] Updated weights for policy 1, policy_version 1618879 (0.0009) [2023-12-27 16:52:20,837][20381] Updated weights for policy 1, policy_version 1618889 (0.0008) [2023-12-27 16:52:20,893][20381] Updated weights for policy 1, policy_version 1618899 (0.0007) [2023-12-27 16:52:21,226][20347] Updated weights for policy 0, policy_version 1620435 (0.0007) [2023-12-27 16:52:21,293][20347] Updated weights for policy 0, policy_version 1620445 (0.0008) [2023-12-27 16:52:21,364][20347] Updated weights for policy 0, policy_version 1620455 (0.0009) [2023-12-27 16:52:21,599][20381] Updated weights for policy 1, policy_version 1618909 (0.0008) [2023-12-27 16:52:21,666][20381] Updated weights for policy 1, policy_version 1618919 (0.0008) [2023-12-27 16:52:21,717][20381] Updated weights for policy 1, policy_version 1618929 (0.0007) [2023-12-27 16:52:22,080][20347] Updated weights for policy 0, policy_version 1620465 (0.0008) [2023-12-27 16:52:22,129][20347] Updated weights for policy 0, policy_version 1620475 (0.0008) [2023-12-27 16:52:22,178][20347] Updated weights for policy 0, policy_version 1620485 (0.0009) [2023-12-27 16:52:22,228][20347] Updated weights for policy 0, policy_version 1620495 (0.0008) [2023-12-27 16:52:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 829415424. Throughput: 0: 10276.2, 1: 10349.6. Samples: 829386552. Policy #0 lag: (min: 30.0, avg: 37.7, max: 62.0) [2023-12-27 16:52:22,363][19170] Avg episode reward: [(0, '990.250'), (1, '845.270')] [2023-12-27 16:52:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001618936_414490624.pth... [2023-12-27 16:52:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001620496_414924800.pth... [2023-12-27 16:52:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001617752_414187520.pth [2023-12-27 16:52:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001619280_414613504.pth [2023-12-27 16:52:22,408][20381] Updated weights for policy 1, policy_version 1618939 (0.0008) [2023-12-27 16:52:22,471][20381] Updated weights for policy 1, policy_version 1618949 (0.0008) [2023-12-27 16:52:22,530][20381] Updated weights for policy 1, policy_version 1618959 (0.0008) [2023-12-27 16:52:23,020][20347] Updated weights for policy 0, policy_version 1620505 (0.0008) [2023-12-27 16:52:23,070][20347] Updated weights for policy 0, policy_version 1620515 (0.0008) [2023-12-27 16:52:23,124][20347] Updated weights for policy 0, policy_version 1620525 (0.0007) [2023-12-27 16:52:23,175][20381] Updated weights for policy 1, policy_version 1618969 (0.0007) [2023-12-27 16:52:23,231][20381] Updated weights for policy 1, policy_version 1618979 (0.0007) [2023-12-27 16:52:23,284][20381] Updated weights for policy 1, policy_version 1618989 (0.0008) [2023-12-27 16:52:23,335][20381] Updated weights for policy 1, policy_version 1618999 (0.0008) [2023-12-27 16:52:23,800][20347] Updated weights for policy 0, policy_version 1620535 (0.0007) [2023-12-27 16:52:23,845][20347] Updated weights for policy 0, policy_version 1620545 (0.0008) [2023-12-27 16:52:23,900][20347] Updated weights for policy 0, policy_version 1620555 (0.0007) [2023-12-27 16:52:24,034][20381] Updated weights for policy 1, policy_version 1619009 (0.0006) [2023-12-27 16:52:24,086][20381] Updated weights for policy 1, policy_version 1619019 (0.0006) [2023-12-27 16:52:24,131][20381] Updated weights for policy 1, policy_version 1619029 (0.0005) [2023-12-27 16:52:24,574][20347] Updated weights for policy 0, policy_version 1620565 (0.0007) [2023-12-27 16:52:24,629][20347] Updated weights for policy 0, policy_version 1620575 (0.0008) [2023-12-27 16:52:24,686][20347] Updated weights for policy 0, policy_version 1620585 (0.0008) [2023-12-27 16:52:24,718][20381] Updated weights for policy 1, policy_version 1619039 (0.0006) [2023-12-27 16:52:24,767][20381] Updated weights for policy 1, policy_version 1619049 (0.0008) [2023-12-27 16:52:24,823][20381] Updated weights for policy 1, policy_version 1619059 (0.0008) [2023-12-27 16:52:25,394][20347] Updated weights for policy 0, policy_version 1620595 (0.0008) [2023-12-27 16:52:25,443][20381] Updated weights for policy 1, policy_version 1619069 (0.0006) [2023-12-27 16:52:25,445][20347] Updated weights for policy 0, policy_version 1620605 (0.0006) [2023-12-27 16:52:25,495][20381] Updated weights for policy 1, policy_version 1619079 (0.0005) [2023-12-27 16:52:25,500][20347] Updated weights for policy 0, policy_version 1620615 (0.0006) [2023-12-27 16:52:25,543][20381] Updated weights for policy 1, policy_version 1619089 (0.0005) [2023-12-27 16:52:26,173][20347] Updated weights for policy 0, policy_version 1620625 (0.0006) [2023-12-27 16:52:26,228][20347] Updated weights for policy 0, policy_version 1620635 (0.0007) [2023-12-27 16:52:26,239][20381] Updated weights for policy 1, policy_version 1619099 (0.0007) [2023-12-27 16:52:26,278][20347] Updated weights for policy 0, policy_version 1620645 (0.0006) [2023-12-27 16:52:26,292][20381] Updated weights for policy 1, policy_version 1619109 (0.0007) [2023-12-27 16:52:26,327][20347] Updated weights for policy 0, policy_version 1620655 (0.0006) [2023-12-27 16:52:26,344][20381] Updated weights for policy 1, policy_version 1619119 (0.0006) [2023-12-27 16:52:27,015][20347] Updated weights for policy 0, policy_version 1620665 (0.0007) [2023-12-27 16:52:27,042][20381] Updated weights for policy 1, policy_version 1619129 (0.0008) [2023-12-27 16:52:27,065][20347] Updated weights for policy 0, policy_version 1620675 (0.0006) [2023-12-27 16:52:27,093][20381] Updated weights for policy 1, policy_version 1619139 (0.0005) [2023-12-27 16:52:27,113][20347] Updated weights for policy 0, policy_version 1620685 (0.0006) [2023-12-27 16:52:27,144][20381] Updated weights for policy 1, policy_version 1619149 (0.0006) [2023-12-27 16:52:27,193][20381] Updated weights for policy 1, policy_version 1619159 (0.0008) [2023-12-27 16:52:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 829521920. Throughput: 0: 10260.7, 1: 10400.0. Samples: 829508400. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:27,363][19170] Avg episode reward: [(0, '990.250'), (1, '892.550')] [2023-12-27 16:52:27,780][20381] Updated weights for policy 1, policy_version 1619169 (0.0005) [2023-12-27 16:52:27,828][20347] Updated weights for policy 0, policy_version 1620695 (0.0005) [2023-12-27 16:52:27,830][20381] Updated weights for policy 1, policy_version 1619179 (0.0008) [2023-12-27 16:52:27,879][20381] Updated weights for policy 1, policy_version 1619189 (0.0007) [2023-12-27 16:52:27,885][20347] Updated weights for policy 0, policy_version 1620705 (0.0005) [2023-12-27 16:52:27,940][20347] Updated weights for policy 0, policy_version 1620715 (0.0005) [2023-12-27 16:52:28,556][20347] Updated weights for policy 0, policy_version 1620725 (0.0006) [2023-12-27 16:52:28,584][20381] Updated weights for policy 1, policy_version 1619199 (0.0007) [2023-12-27 16:52:28,605][20347] Updated weights for policy 0, policy_version 1620735 (0.0006) [2023-12-27 16:52:28,640][20381] Updated weights for policy 1, policy_version 1619209 (0.0006) [2023-12-27 16:52:28,650][20347] Updated weights for policy 0, policy_version 1620745 (0.0005) [2023-12-27 16:52:28,698][20381] Updated weights for policy 1, policy_version 1619219 (0.0006) [2023-12-27 16:52:29,347][20347] Updated weights for policy 0, policy_version 1620755 (0.0006) [2023-12-27 16:52:29,414][20347] Updated weights for policy 0, policy_version 1620765 (0.0007) [2023-12-27 16:52:29,423][20381] Updated weights for policy 1, policy_version 1619229 (0.0008) [2023-12-27 16:52:29,473][20347] Updated weights for policy 0, policy_version 1620775 (0.0007) [2023-12-27 16:52:29,479][20381] Updated weights for policy 1, policy_version 1619239 (0.0005) [2023-12-27 16:52:29,536][20381] Updated weights for policy 1, policy_version 1619249 (0.0007) [2023-12-27 16:52:30,132][20347] Updated weights for policy 0, policy_version 1620785 (0.0007) [2023-12-27 16:52:30,184][20347] Updated weights for policy 0, policy_version 1620795 (0.0005) [2023-12-27 16:52:30,238][20347] Updated weights for policy 0, policy_version 1620805 (0.0007) [2023-12-27 16:52:30,240][20381] Updated weights for policy 1, policy_version 1619259 (0.0008) [2023-12-27 16:52:30,292][20381] Updated weights for policy 1, policy_version 1619269 (0.0007) [2023-12-27 16:52:30,294][20347] Updated weights for policy 0, policy_version 1620815 (0.0007) [2023-12-27 16:52:30,344][20381] Updated weights for policy 1, policy_version 1619279 (0.0009) [2023-12-27 16:52:30,901][20347] Updated weights for policy 0, policy_version 1620825 (0.0005) [2023-12-27 16:52:30,947][20347] Updated weights for policy 0, policy_version 1620835 (0.0005) [2023-12-27 16:52:30,959][20381] Updated weights for policy 1, policy_version 1619289 (0.0007) [2023-12-27 16:52:30,995][20347] Updated weights for policy 0, policy_version 1620845 (0.0005) [2023-12-27 16:52:31,008][20381] Updated weights for policy 1, policy_version 1619299 (0.0005) [2023-12-27 16:52:31,093][20381] Updated weights for policy 1, policy_version 1619309 (0.0007) [2023-12-27 16:52:31,159][20381] Updated weights for policy 1, policy_version 1619319 (0.0008) [2023-12-27 16:52:31,814][20347] Updated weights for policy 0, policy_version 1620855 (0.0006) [2023-12-27 16:52:31,872][20381] Updated weights for policy 1, policy_version 1619329 (0.0005) [2023-12-27 16:52:31,877][20347] Updated weights for policy 0, policy_version 1620865 (0.0008) [2023-12-27 16:52:31,926][20381] Updated weights for policy 1, policy_version 1619339 (0.0005) [2023-12-27 16:52:31,939][20347] Updated weights for policy 0, policy_version 1620875 (0.0008) [2023-12-27 16:52:31,986][20381] Updated weights for policy 1, policy_version 1619349 (0.0006) [2023-12-27 16:52:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 829620224. Throughput: 0: 10238.4, 1: 10340.2. Samples: 829630864. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.480')] [2023-12-27 16:52:32,598][20347] Updated weights for policy 0, policy_version 1620885 (0.0008) [2023-12-27 16:52:32,633][20381] Updated weights for policy 1, policy_version 1619359 (0.0006) [2023-12-27 16:52:32,647][20347] Updated weights for policy 0, policy_version 1620895 (0.0008) [2023-12-27 16:52:32,698][20347] Updated weights for policy 0, policy_version 1620905 (0.0007) [2023-12-27 16:52:32,699][20381] Updated weights for policy 1, policy_version 1619369 (0.0006) [2023-12-27 16:52:32,740][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-27 16:52:33,325][20347] Updated weights for policy 0, policy_version 1620915 (0.0007) [2023-12-27 16:52:33,377][20347] Updated weights for policy 0, policy_version 1620925 (0.0006) [2023-12-27 16:52:33,423][20347] Updated weights for policy 0, policy_version 1620935 (0.0005) [2023-12-27 16:52:33,428][20381] Updated weights for policy 1, policy_version 1619379 (0.0007) [2023-12-27 16:52:33,477][20381] Updated weights for policy 1, policy_version 1619389 (0.0007) [2023-12-27 16:52:33,534][20381] Updated weights for policy 1, policy_version 1619399 (0.0007) [2023-12-27 16:52:34,021][20347] Updated weights for policy 0, policy_version 1620945 (0.0006) [2023-12-27 16:52:34,076][20347] Updated weights for policy 0, policy_version 1620955 (0.0007) [2023-12-27 16:52:34,153][20347] Updated weights for policy 0, policy_version 1620967 (0.0007) [2023-12-27 16:52:34,252][20381] Updated weights for policy 1, policy_version 1619409 (0.0008) [2023-12-27 16:52:34,307][20381] Updated weights for policy 1, policy_version 1619419 (0.0008) [2023-12-27 16:52:34,374][20381] Updated weights for policy 1, policy_version 1619429 (0.0008) [2023-12-27 16:52:34,426][20381] Updated weights for policy 1, policy_version 1619439 (0.0008) [2023-12-27 16:52:34,906][20347] Updated weights for policy 0, policy_version 1620977 (0.0008) [2023-12-27 16:52:34,954][20347] Updated weights for policy 0, policy_version 1620987 (0.0006) [2023-12-27 16:52:35,004][20347] Updated weights for policy 0, policy_version 1620997 (0.0007) [2023-12-27 16:52:35,054][20347] Updated weights for policy 0, policy_version 1621007 (0.0008) [2023-12-27 16:52:35,157][20381] Updated weights for policy 1, policy_version 1619449 (0.0007) [2023-12-27 16:52:35,217][20381] Updated weights for policy 1, policy_version 1619459 (0.0007) [2023-12-27 16:52:35,279][20381] Updated weights for policy 1, policy_version 1619469 (0.0007) [2023-12-27 16:52:35,692][20347] Updated weights for policy 0, policy_version 1621017 (0.0008) [2023-12-27 16:52:35,753][20347] Updated weights for policy 0, policy_version 1621027 (0.0009) [2023-12-27 16:52:35,814][20347] Updated weights for policy 0, policy_version 1621037 (0.0008) [2023-12-27 16:52:35,996][20381] Updated weights for policy 1, policy_version 1619479 (0.0008) [2023-12-27 16:52:36,049][20381] Updated weights for policy 1, policy_version 1619489 (0.0008) [2023-12-27 16:52:36,110][20381] Updated weights for policy 1, policy_version 1619499 (0.0008) [2023-12-27 16:52:36,519][20347] Updated weights for policy 0, policy_version 1621047 (0.0007) [2023-12-27 16:52:36,572][20347] Updated weights for policy 0, policy_version 1621057 (0.0005) [2023-12-27 16:52:36,635][20347] Updated weights for policy 0, policy_version 1621067 (0.0006) [2023-12-27 16:52:36,831][20381] Updated weights for policy 1, policy_version 1619509 (0.0008) [2023-12-27 16:52:36,881][20381] Updated weights for policy 1, policy_version 1619519 (0.0007) [2023-12-27 16:52:36,928][20381] Updated weights for policy 1, policy_version 1619529 (0.0008) [2023-12-27 16:52:37,225][20347] Updated weights for policy 0, policy_version 1621077 (0.0005) [2023-12-27 16:52:37,281][20347] Updated weights for policy 0, policy_version 1621087 (0.0005) [2023-12-27 16:52:37,334][20347] Updated weights for policy 0, policy_version 1621097 (0.0007) [2023-12-27 16:52:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 829718528. Throughput: 0: 10259.8, 1: 10244.3. Samples: 829692244. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.800')] [2023-12-27 16:52:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001619536_414646272.pth... [2023-12-27 16:52:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001618360_414343168.pth [2023-12-27 16:52:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001621104_415080448.pth... [2023-12-27 16:52:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001619888_414769152.pth [2023-12-27 16:52:37,648][20381] Updated weights for policy 1, policy_version 1619539 (0.0007) [2023-12-27 16:52:37,703][20381] Updated weights for policy 1, policy_version 1619549 (0.0008) [2023-12-27 16:52:37,759][20381] Updated weights for policy 1, policy_version 1619559 (0.0008) [2023-12-27 16:52:37,957][20347] Updated weights for policy 0, policy_version 1621107 (0.0008) [2023-12-27 16:52:38,011][20347] Updated weights for policy 0, policy_version 1621117 (0.0008) [2023-12-27 16:52:38,061][20347] Updated weights for policy 0, policy_version 1621127 (0.0008) [2023-12-27 16:52:38,470][20381] Updated weights for policy 1, policy_version 1619569 (0.0007) [2023-12-27 16:52:38,525][20381] Updated weights for policy 1, policy_version 1619579 (0.0005) [2023-12-27 16:52:38,583][20381] Updated weights for policy 1, policy_version 1619589 (0.0007) [2023-12-27 16:52:38,640][20381] Updated weights for policy 1, policy_version 1619599 (0.0008) [2023-12-27 16:52:38,730][20347] Updated weights for policy 0, policy_version 1621137 (0.0007) [2023-12-27 16:52:38,783][20347] Updated weights for policy 0, policy_version 1621147 (0.0008) [2023-12-27 16:52:38,843][20347] Updated weights for policy 0, policy_version 1621157 (0.0008) [2023-12-27 16:52:38,893][20347] Updated weights for policy 0, policy_version 1621167 (0.0008) [2023-12-27 16:52:39,347][20381] Updated weights for policy 1, policy_version 1619609 (0.0008) [2023-12-27 16:52:39,420][20381] Updated weights for policy 1, policy_version 1619619 (0.0009) [2023-12-27 16:52:39,480][20381] Updated weights for policy 1, policy_version 1619629 (0.0008) [2023-12-27 16:52:39,538][20347] Updated weights for policy 0, policy_version 1621177 (0.0006) [2023-12-27 16:52:39,592][20347] Updated weights for policy 0, policy_version 1621187 (0.0006) [2023-12-27 16:52:39,645][20347] Updated weights for policy 0, policy_version 1621197 (0.0007) [2023-12-27 16:52:40,193][20292] KL-divergence is very high: 106.7208 [2023-12-27 16:52:40,221][20381] Updated weights for policy 1, policy_version 1619639 (0.0008) [2023-12-27 16:52:40,278][20381] Updated weights for policy 1, policy_version 1619649 (0.0008) [2023-12-27 16:52:40,339][20381] Updated weights for policy 1, policy_version 1619659 (0.0008) [2023-12-27 16:52:40,366][20347] Updated weights for policy 0, policy_version 1621207 (0.0007) [2023-12-27 16:52:40,420][20347] Updated weights for policy 0, policy_version 1621217 (0.0009) [2023-12-27 16:52:40,477][20347] Updated weights for policy 0, policy_version 1621227 (0.0008) [2023-12-27 16:52:40,937][20381] Updated weights for policy 1, policy_version 1619669 (0.0006) [2023-12-27 16:52:40,990][20381] Updated weights for policy 1, policy_version 1619679 (0.0008) [2023-12-27 16:52:41,051][20381] Updated weights for policy 1, policy_version 1619689 (0.0008) [2023-12-27 16:52:41,177][20347] Updated weights for policy 0, policy_version 1621237 (0.0008) [2023-12-27 16:52:41,242][20347] Updated weights for policy 0, policy_version 1621247 (0.0007) [2023-12-27 16:52:41,307][20347] Updated weights for policy 0, policy_version 1621257 (0.0008) [2023-12-27 16:52:41,822][20381] Updated weights for policy 1, policy_version 1619699 (0.0009) [2023-12-27 16:52:41,885][20381] Updated weights for policy 1, policy_version 1619709 (0.0008) [2023-12-27 16:52:41,947][20381] Updated weights for policy 1, policy_version 1619719 (0.0008) [2023-12-27 16:52:42,041][20347] Updated weights for policy 0, policy_version 1621267 (0.0007) [2023-12-27 16:52:42,096][20347] Updated weights for policy 0, policy_version 1621277 (0.0007) [2023-12-27 16:52:42,155][20347] Updated weights for policy 0, policy_version 1621287 (0.0005) [2023-12-27 16:52:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 829825024. Throughput: 0: 10268.6, 1: 10192.1. Samples: 829813684. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.680')] [2023-12-27 16:52:42,711][20381] Updated weights for policy 1, policy_version 1619729 (0.0008) [2023-12-27 16:52:42,775][20381] Updated weights for policy 1, policy_version 1619739 (0.0008) [2023-12-27 16:52:42,787][20347] Updated weights for policy 0, policy_version 1621297 (0.0007) [2023-12-27 16:52:42,834][20381] Updated weights for policy 1, policy_version 1619749 (0.0008) [2023-12-27 16:52:42,844][20347] Updated weights for policy 0, policy_version 1621307 (0.0006) [2023-12-27 16:52:42,892][20381] Updated weights for policy 1, policy_version 1619759 (0.0006) [2023-12-27 16:52:42,901][20347] Updated weights for policy 0, policy_version 1621317 (0.0006) [2023-12-27 16:52:42,956][20347] Updated weights for policy 0, policy_version 1621327 (0.0008) [2023-12-27 16:52:43,583][20347] Updated weights for policy 0, policy_version 1621337 (0.0007) [2023-12-27 16:52:43,584][20381] Updated weights for policy 1, policy_version 1619769 (0.0005) [2023-12-27 16:52:43,630][20381] Updated weights for policy 1, policy_version 1619779 (0.0007) [2023-12-27 16:52:43,640][20347] Updated weights for policy 0, policy_version 1621347 (0.0006) [2023-12-27 16:52:43,672][20381] Updated weights for policy 1, policy_version 1619789 (0.0008) [2023-12-27 16:52:43,700][20347] Updated weights for policy 0, policy_version 1621357 (0.0005) [2023-12-27 16:52:44,284][20347] Updated weights for policy 0, policy_version 1621367 (0.0009) [2023-12-27 16:52:44,339][20347] Updated weights for policy 0, policy_version 1621377 (0.0010) [2023-12-27 16:52:44,394][20347] Updated weights for policy 0, policy_version 1621387 (0.0010) [2023-12-27 16:52:44,410][20381] Updated weights for policy 1, policy_version 1619799 (0.0006) [2023-12-27 16:52:44,467][20381] Updated weights for policy 1, policy_version 1619809 (0.0006) [2023-12-27 16:52:44,522][20381] Updated weights for policy 1, policy_version 1619819 (0.0006) [2023-12-27 16:52:45,093][20347] Updated weights for policy 0, policy_version 1621397 (0.0011) [2023-12-27 16:52:45,152][20347] Updated weights for policy 0, policy_version 1621407 (0.0010) [2023-12-27 16:52:45,192][20381] Updated weights for policy 1, policy_version 1619829 (0.0005) [2023-12-27 16:52:45,212][20347] Updated weights for policy 0, policy_version 1621417 (0.0010) [2023-12-27 16:52:45,247][20381] Updated weights for policy 1, policy_version 1619839 (0.0006) [2023-12-27 16:52:45,307][20381] Updated weights for policy 1, policy_version 1619849 (0.0007) [2023-12-27 16:52:45,886][20381] Updated weights for policy 1, policy_version 1619859 (0.0007) [2023-12-27 16:52:45,894][20347] Updated weights for policy 0, policy_version 1621427 (0.0010) [2023-12-27 16:52:45,937][20381] Updated weights for policy 1, policy_version 1619869 (0.0005) [2023-12-27 16:52:45,946][20347] Updated weights for policy 0, policy_version 1621437 (0.0010) [2023-12-27 16:52:45,989][20381] Updated weights for policy 1, policy_version 1619879 (0.0005) [2023-12-27 16:52:45,995][20347] Updated weights for policy 0, policy_version 1621447 (0.0010) [2023-12-27 16:52:46,539][20381] Updated weights for policy 1, policy_version 1619889 (0.0005) [2023-12-27 16:52:46,587][20381] Updated weights for policy 1, policy_version 1619899 (0.0007) [2023-12-27 16:52:46,646][20381] Updated weights for policy 1, policy_version 1619909 (0.0005) [2023-12-27 16:52:46,690][20347] Updated weights for policy 0, policy_version 1621457 (0.0010) [2023-12-27 16:52:46,697][20381] Updated weights for policy 1, policy_version 1619919 (0.0005) [2023-12-27 16:52:46,753][20347] Updated weights for policy 0, policy_version 1621467 (0.0010) [2023-12-27 16:52:46,810][20347] Updated weights for policy 0, policy_version 1621477 (0.0010) [2023-12-27 16:52:46,865][20347] Updated weights for policy 0, policy_version 1621487 (0.0010) [2023-12-27 16:52:47,341][20381] Updated weights for policy 1, policy_version 1619929 (0.0008) [2023-12-27 16:52:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 829923328. Throughput: 0: 10270.2, 1: 10220.8. Samples: 829937800. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.640')] [2023-12-27 16:52:47,409][20381] Updated weights for policy 1, policy_version 1619939 (0.0007) [2023-12-27 16:52:47,456][20347] Updated weights for policy 0, policy_version 1621497 (0.0006) [2023-12-27 16:52:47,465][20381] Updated weights for policy 1, policy_version 1619949 (0.0008) [2023-12-27 16:52:47,507][20347] Updated weights for policy 0, policy_version 1621507 (0.0005) [2023-12-27 16:52:47,553][20347] Updated weights for policy 0, policy_version 1621517 (0.0006) [2023-12-27 16:52:48,037][20381] Updated weights for policy 1, policy_version 1619959 (0.0008) [2023-12-27 16:52:48,084][20381] Updated weights for policy 1, policy_version 1619969 (0.0007) [2023-12-27 16:52:48,135][20381] Updated weights for policy 1, policy_version 1619979 (0.0007) [2023-12-27 16:52:48,175][20347] Updated weights for policy 0, policy_version 1621527 (0.0009) [2023-12-27 16:52:48,235][20347] Updated weights for policy 0, policy_version 1621537 (0.0010) [2023-12-27 16:52:48,296][20347] Updated weights for policy 0, policy_version 1621547 (0.0010) [2023-12-27 16:52:48,880][20381] Updated weights for policy 1, policy_version 1619989 (0.0007) [2023-12-27 16:52:48,936][20381] Updated weights for policy 1, policy_version 1619999 (0.0008) [2023-12-27 16:52:48,964][20347] Updated weights for policy 0, policy_version 1621557 (0.0008) [2023-12-27 16:52:48,992][20381] Updated weights for policy 1, policy_version 1620009 (0.0007) [2023-12-27 16:52:49,028][20347] Updated weights for policy 0, policy_version 1621567 (0.0006) [2023-12-27 16:52:49,085][20347] Updated weights for policy 0, policy_version 1621577 (0.0006) [2023-12-27 16:52:49,639][20381] Updated weights for policy 1, policy_version 1620019 (0.0006) [2023-12-27 16:52:49,703][20381] Updated weights for policy 1, policy_version 1620029 (0.0008) [2023-12-27 16:52:49,756][20347] Updated weights for policy 0, policy_version 1621587 (0.0007) [2023-12-27 16:52:49,757][20381] Updated weights for policy 1, policy_version 1620039 (0.0007) [2023-12-27 16:52:49,815][20347] Updated weights for policy 0, policy_version 1621597 (0.0006) [2023-12-27 16:52:49,880][20347] Updated weights for policy 0, policy_version 1621607 (0.0007) [2023-12-27 16:52:50,450][20381] Updated weights for policy 1, policy_version 1620049 (0.0008) [2023-12-27 16:52:50,503][20381] Updated weights for policy 1, policy_version 1620059 (0.0011) [2023-12-27 16:52:50,557][20381] Updated weights for policy 1, policy_version 1620069 (0.0010) [2023-12-27 16:52:50,594][20347] Updated weights for policy 0, policy_version 1621617 (0.0008) [2023-12-27 16:52:50,615][20381] Updated weights for policy 1, policy_version 1620079 (0.0007) [2023-12-27 16:52:50,646][20347] Updated weights for policy 0, policy_version 1621627 (0.0007) [2023-12-27 16:52:50,707][20347] Updated weights for policy 0, policy_version 1621637 (0.0007) [2023-12-27 16:52:50,765][20347] Updated weights for policy 0, policy_version 1621647 (0.0008) [2023-12-27 16:52:51,297][20381] Updated weights for policy 1, policy_version 1620089 (0.0008) [2023-12-27 16:52:51,362][20381] Updated weights for policy 1, policy_version 1620099 (0.0010) [2023-12-27 16:52:51,427][20381] Updated weights for policy 1, policy_version 1620109 (0.0008) [2023-12-27 16:52:51,537][20347] Updated weights for policy 0, policy_version 1621657 (0.0008) [2023-12-27 16:52:51,595][20347] Updated weights for policy 0, policy_version 1621667 (0.0008) [2023-12-27 16:52:51,665][20347] Updated weights for policy 0, policy_version 1621677 (0.0010) [2023-12-27 16:52:52,043][20381] Updated weights for policy 1, policy_version 1620119 (0.0007) [2023-12-27 16:52:52,095][20381] Updated weights for policy 1, policy_version 1620129 (0.0005) [2023-12-27 16:52:52,145][20381] Updated weights for policy 1, policy_version 1620139 (0.0005) [2023-12-27 16:52:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 830029824. Throughput: 0: 10248.3, 1: 10210.7. Samples: 830000632. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.660')] [2023-12-27 16:52:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001620144_414801920.pth... [2023-12-27 16:52:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001621680_415227904.pth... [2023-12-27 16:52:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001620496_414924800.pth [2023-12-27 16:52:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001618936_414490624.pth [2023-12-27 16:52:52,443][20347] Updated weights for policy 0, policy_version 1621687 (0.0008) [2023-12-27 16:52:52,507][20347] Updated weights for policy 0, policy_version 1621697 (0.0008) [2023-12-27 16:52:52,552][20347] Updated weights for policy 0, policy_version 1621707 (0.0008) [2023-12-27 16:52:52,742][20381] Updated weights for policy 1, policy_version 1620149 (0.0006) [2023-12-27 16:52:52,797][20381] Updated weights for policy 1, policy_version 1620159 (0.0005) [2023-12-27 16:52:52,852][20381] Updated weights for policy 1, policy_version 1620169 (0.0005) [2023-12-27 16:52:53,287][20347] Updated weights for policy 0, policy_version 1621717 (0.0008) [2023-12-27 16:52:53,343][20347] Updated weights for policy 0, policy_version 1621727 (0.0008) [2023-12-27 16:52:53,398][20347] Updated weights for policy 0, policy_version 1621737 (0.0008) [2023-12-27 16:52:53,486][20381] Updated weights for policy 1, policy_version 1620179 (0.0007) [2023-12-27 16:52:53,539][20381] Updated weights for policy 1, policy_version 1620189 (0.0008) [2023-12-27 16:52:53,585][20381] Updated weights for policy 1, policy_version 1620199 (0.0007) [2023-12-27 16:52:54,047][20347] Updated weights for policy 0, policy_version 1621747 (0.0007) [2023-12-27 16:52:54,095][20347] Updated weights for policy 0, policy_version 1621757 (0.0008) [2023-12-27 16:52:54,152][20347] Updated weights for policy 0, policy_version 1621767 (0.0008) [2023-12-27 16:52:54,216][20381] Updated weights for policy 1, policy_version 1620209 (0.0007) [2023-12-27 16:52:54,274][20381] Updated weights for policy 1, policy_version 1620219 (0.0009) [2023-12-27 16:52:54,331][20381] Updated weights for policy 1, policy_version 1620229 (0.0007) [2023-12-27 16:52:54,393][20381] Updated weights for policy 1, policy_version 1620239 (0.0007) [2023-12-27 16:52:54,847][20347] Updated weights for policy 0, policy_version 1621777 (0.0010) [2023-12-27 16:52:54,907][20347] Updated weights for policy 0, policy_version 1621787 (0.0009) [2023-12-27 16:52:54,966][20381] Updated weights for policy 1, policy_version 1620249 (0.0007) [2023-12-27 16:52:54,969][20347] Updated weights for policy 0, policy_version 1621797 (0.0006) [2023-12-27 16:52:55,024][20381] Updated weights for policy 1, policy_version 1620259 (0.0009) [2023-12-27 16:52:55,029][20347] Updated weights for policy 0, policy_version 1621807 (0.0010) [2023-12-27 16:52:55,073][20381] Updated weights for policy 1, policy_version 1620269 (0.0010) [2023-12-27 16:52:55,587][20347] Updated weights for policy 0, policy_version 1621817 (0.0010) [2023-12-27 16:52:55,644][20347] Updated weights for policy 0, policy_version 1621827 (0.0010) [2023-12-27 16:52:55,699][20347] Updated weights for policy 0, policy_version 1621837 (0.0010) [2023-12-27 16:52:55,764][20381] Updated weights for policy 1, policy_version 1620279 (0.0010) [2023-12-27 16:52:55,819][20381] Updated weights for policy 1, policy_version 1620289 (0.0010) [2023-12-27 16:52:55,877][20381] Updated weights for policy 1, policy_version 1620299 (0.0011) [2023-12-27 16:52:56,365][20347] Updated weights for policy 0, policy_version 1621847 (0.0010) [2023-12-27 16:52:56,409][20347] Updated weights for policy 0, policy_version 1621857 (0.0010) [2023-12-27 16:52:56,459][20347] Updated weights for policy 0, policy_version 1621867 (0.0010) [2023-12-27 16:52:56,475][20381] Updated weights for policy 1, policy_version 1620309 (0.0010) [2023-12-27 16:52:56,529][20381] Updated weights for policy 1, policy_version 1620319 (0.0010) [2023-12-27 16:52:56,588][20381] Updated weights for policy 1, policy_version 1620329 (0.0010) [2023-12-27 16:52:57,080][20347] Updated weights for policy 0, policy_version 1621877 (0.0010) [2023-12-27 16:52:57,132][20347] Updated weights for policy 0, policy_version 1621887 (0.0010) [2023-12-27 16:52:57,167][20381] Updated weights for policy 1, policy_version 1620339 (0.0009) [2023-12-27 16:52:57,190][20347] Updated weights for policy 0, policy_version 1621897 (0.0010) [2023-12-27 16:52:57,237][20381] Updated weights for policy 1, policy_version 1620349 (0.0005) [2023-12-27 16:52:57,304][20381] Updated weights for policy 1, policy_version 1620359 (0.0006) [2023-12-27 16:52:57,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 830144512. Throughput: 0: 10226.0, 1: 10270.8. Samples: 830125444. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:52:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '892.370')] [2023-12-27 16:52:57,757][20347] Updated weights for policy 0, policy_version 1621907 (0.0009) [2023-12-27 16:52:57,808][20347] Updated weights for policy 0, policy_version 1621917 (0.0005) [2023-12-27 16:52:57,851][20347] Updated weights for policy 0, policy_version 1621927 (0.0005) [2023-12-27 16:52:58,005][20381] Updated weights for policy 1, policy_version 1620369 (0.0007) [2023-12-27 16:52:58,072][20381] Updated weights for policy 1, policy_version 1620379 (0.0005) [2023-12-27 16:52:58,135][20381] Updated weights for policy 1, policy_version 1620389 (0.0007) [2023-12-27 16:52:58,181][20381] Updated weights for policy 1, policy_version 1620399 (0.0007) [2023-12-27 16:52:58,557][20347] Updated weights for policy 0, policy_version 1621937 (0.0009) [2023-12-27 16:52:58,629][20347] Updated weights for policy 0, policy_version 1621947 (0.0009) [2023-12-27 16:52:58,694][20347] Updated weights for policy 0, policy_version 1621957 (0.0008) [2023-12-27 16:52:58,762][20347] Updated weights for policy 0, policy_version 1621967 (0.0009) [2023-12-27 16:52:58,922][20381] Updated weights for policy 1, policy_version 1620409 (0.0009) [2023-12-27 16:52:58,978][20381] Updated weights for policy 1, policy_version 1620419 (0.0008) [2023-12-27 16:52:59,036][20381] Updated weights for policy 1, policy_version 1620429 (0.0008) [2023-12-27 16:52:59,527][20347] Updated weights for policy 0, policy_version 1621977 (0.0008) [2023-12-27 16:52:59,583][20347] Updated weights for policy 0, policy_version 1621987 (0.0008) [2023-12-27 16:52:59,635][20347] Updated weights for policy 0, policy_version 1621997 (0.0006) [2023-12-27 16:52:59,750][20381] Updated weights for policy 1, policy_version 1620439 (0.0006) [2023-12-27 16:52:59,811][20381] Updated weights for policy 1, policy_version 1620449 (0.0006) [2023-12-27 16:52:59,875][20381] Updated weights for policy 1, policy_version 1620459 (0.0008) [2023-12-27 16:53:00,303][20347] Updated weights for policy 0, policy_version 1622007 (0.0008) [2023-12-27 16:53:00,361][20347] Updated weights for policy 0, policy_version 1622017 (0.0007) [2023-12-27 16:53:00,416][20347] Updated weights for policy 0, policy_version 1622027 (0.0007) [2023-12-27 16:53:00,566][20381] Updated weights for policy 1, policy_version 1620469 (0.0007) [2023-12-27 16:53:00,625][20381] Updated weights for policy 1, policy_version 1620479 (0.0007) [2023-12-27 16:53:00,684][20381] Updated weights for policy 1, policy_version 1620489 (0.0008) [2023-12-27 16:53:01,070][20347] Updated weights for policy 0, policy_version 1622037 (0.0007) [2023-12-27 16:53:01,127][20347] Updated weights for policy 0, policy_version 1622047 (0.0008) [2023-12-27 16:53:01,197][20347] Updated weights for policy 0, policy_version 1622057 (0.0006) [2023-12-27 16:53:01,365][20381] Updated weights for policy 1, policy_version 1620499 (0.0008) [2023-12-27 16:53:01,436][20381] Updated weights for policy 1, policy_version 1620509 (0.0006) [2023-12-27 16:53:01,500][20381] Updated weights for policy 1, policy_version 1620519 (0.0007) [2023-12-27 16:53:01,930][20347] Updated weights for policy 0, policy_version 1622067 (0.0007) [2023-12-27 16:53:01,977][20347] Updated weights for policy 0, policy_version 1622077 (0.0007) [2023-12-27 16:53:02,035][20347] Updated weights for policy 0, policy_version 1622087 (0.0007) [2023-12-27 16:53:02,182][20381] Updated weights for policy 1, policy_version 1620529 (0.0008) [2023-12-27 16:53:02,232][20381] Updated weights for policy 1, policy_version 1620539 (0.0010) [2023-12-27 16:53:02,291][20381] Updated weights for policy 1, policy_version 1620549 (0.0008) [2023-12-27 16:53:02,352][20381] Updated weights for policy 1, policy_version 1620559 (0.0008) [2023-12-27 16:53:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 830242816. Throughput: 0: 10207.2, 1: 10269.2. Samples: 830247912. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.350')] [2023-12-27 16:53:02,771][20347] Updated weights for policy 0, policy_version 1622097 (0.0007) [2023-12-27 16:53:02,818][20347] Updated weights for policy 0, policy_version 1622107 (0.0005) [2023-12-27 16:53:02,869][20347] Updated weights for policy 0, policy_version 1622117 (0.0005) [2023-12-27 16:53:02,930][20347] Updated weights for policy 0, policy_version 1622127 (0.0007) [2023-12-27 16:53:03,015][20381] Updated weights for policy 1, policy_version 1620569 (0.0008) [2023-12-27 16:53:03,062][20381] Updated weights for policy 1, policy_version 1620579 (0.0008) [2023-12-27 16:53:03,116][20381] Updated weights for policy 1, policy_version 1620589 (0.0007) [2023-12-27 16:53:03,546][20347] Updated weights for policy 0, policy_version 1622137 (0.0007) [2023-12-27 16:53:03,588][20347] Updated weights for policy 0, policy_version 1622147 (0.0007) [2023-12-27 16:53:03,631][20347] Updated weights for policy 0, policy_version 1622157 (0.0005) [2023-12-27 16:53:03,826][20381] Updated weights for policy 1, policy_version 1620599 (0.0006) [2023-12-27 16:53:03,887][20381] Updated weights for policy 1, policy_version 1620609 (0.0008) [2023-12-27 16:53:03,951][20381] Updated weights for policy 1, policy_version 1620619 (0.0008) [2023-12-27 16:53:04,268][20347] Updated weights for policy 0, policy_version 1622167 (0.0007) [2023-12-27 16:53:04,330][20347] Updated weights for policy 0, policy_version 1622177 (0.0007) [2023-12-27 16:53:04,388][20347] Updated weights for policy 0, policy_version 1622187 (0.0006) [2023-12-27 16:53:04,662][20381] Updated weights for policy 1, policy_version 1620629 (0.0009) [2023-12-27 16:53:04,717][20381] Updated weights for policy 1, policy_version 1620639 (0.0006) [2023-12-27 16:53:04,769][20381] Updated weights for policy 1, policy_version 1620649 (0.0005) [2023-12-27 16:53:04,993][20347] Updated weights for policy 0, policy_version 1622197 (0.0005) [2023-12-27 16:53:05,045][20347] Updated weights for policy 0, policy_version 1622207 (0.0005) [2023-12-27 16:53:05,100][20347] Updated weights for policy 0, policy_version 1622217 (0.0006) [2023-12-27 16:53:05,348][20381] Updated weights for policy 1, policy_version 1620659 (0.0005) [2023-12-27 16:53:05,393][20381] Updated weights for policy 1, policy_version 1620669 (0.0005) [2023-12-27 16:53:05,443][20381] Updated weights for policy 1, policy_version 1620679 (0.0007) [2023-12-27 16:53:05,719][20347] Updated weights for policy 0, policy_version 1622227 (0.0010) [2023-12-27 16:53:05,785][20347] Updated weights for policy 0, policy_version 1622237 (0.0007) [2023-12-27 16:53:05,846][20347] Updated weights for policy 0, policy_version 1622247 (0.0006) [2023-12-27 16:53:06,097][20381] Updated weights for policy 1, policy_version 1620689 (0.0008) [2023-12-27 16:53:06,167][20381] Updated weights for policy 1, policy_version 1620699 (0.0007) [2023-12-27 16:53:06,223][20381] Updated weights for policy 1, policy_version 1620709 (0.0009) [2023-12-27 16:53:06,279][20381] Updated weights for policy 1, policy_version 1620719 (0.0009) [2023-12-27 16:53:06,534][20347] Updated weights for policy 0, policy_version 1622257 (0.0008) [2023-12-27 16:53:06,599][20347] Updated weights for policy 0, policy_version 1622267 (0.0007) [2023-12-27 16:53:06,665][20347] Updated weights for policy 0, policy_version 1622277 (0.0008) [2023-12-27 16:53:06,733][20347] Updated weights for policy 0, policy_version 1622287 (0.0007) [2023-12-27 16:53:07,000][20381] Updated weights for policy 1, policy_version 1620729 (0.0010) [2023-12-27 16:53:07,048][20381] Updated weights for policy 1, policy_version 1620739 (0.0010) [2023-12-27 16:53:07,101][20381] Updated weights for policy 1, policy_version 1620749 (0.0011) [2023-12-27 16:53:07,263][20347] Updated weights for policy 0, policy_version 1622297 (0.0006) [2023-12-27 16:53:07,323][20347] Updated weights for policy 0, policy_version 1622307 (0.0008) [2023-12-27 16:53:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 830341120. Throughput: 0: 10295.1, 1: 10256.1. Samples: 830311356. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:07,363][19170] Avg episode reward: [(0, '990.310'), (1, '844.500')] [2023-12-27 16:53:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001620752_414957568.pth... [2023-12-27 16:53:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001619536_414646272.pth [2023-12-27 16:53:07,379][20347] Updated weights for policy 0, policy_version 1622317 (0.0009) [2023-12-27 16:53:07,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001622320_415391744.pth... [2023-12-27 16:53:07,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001621104_415080448.pth [2023-12-27 16:53:07,704][20381] Updated weights for policy 1, policy_version 1620759 (0.0008) [2023-12-27 16:53:07,754][20381] Updated weights for policy 1, policy_version 1620769 (0.0007) [2023-12-27 16:53:07,804][20381] Updated weights for policy 1, policy_version 1620779 (0.0005) [2023-12-27 16:53:08,097][20347] Updated weights for policy 0, policy_version 1622327 (0.0010) [2023-12-27 16:53:08,140][20347] Updated weights for policy 0, policy_version 1622337 (0.0010) [2023-12-27 16:53:08,191][20347] Updated weights for policy 0, policy_version 1622347 (0.0010) [2023-12-27 16:53:08,449][20381] Updated weights for policy 1, policy_version 1620789 (0.0007) [2023-12-27 16:53:08,508][20381] Updated weights for policy 1, policy_version 1620799 (0.0008) [2023-12-27 16:53:08,564][20381] Updated weights for policy 1, policy_version 1620809 (0.0005) [2023-12-27 16:53:08,954][20347] Updated weights for policy 0, policy_version 1622357 (0.0010) [2023-12-27 16:53:09,006][20347] Updated weights for policy 0, policy_version 1622367 (0.0009) [2023-12-27 16:53:09,055][20347] Updated weights for policy 0, policy_version 1622377 (0.0008) [2023-12-27 16:53:09,089][20381] Updated weights for policy 1, policy_version 1620819 (0.0005) [2023-12-27 16:53:09,154][20381] Updated weights for policy 1, policy_version 1620829 (0.0005) [2023-12-27 16:53:09,213][20381] Updated weights for policy 1, policy_version 1620839 (0.0009) [2023-12-27 16:53:09,823][20347] Updated weights for policy 0, policy_version 1622387 (0.0007) [2023-12-27 16:53:09,888][20347] Updated weights for policy 0, policy_version 1622397 (0.0007) [2023-12-27 16:53:09,935][20381] Updated weights for policy 1, policy_version 1620849 (0.0009) [2023-12-27 16:53:09,957][20347] Updated weights for policy 0, policy_version 1622407 (0.0008) [2023-12-27 16:53:09,996][20381] Updated weights for policy 1, policy_version 1620859 (0.0008) [2023-12-27 16:53:10,062][20381] Updated weights for policy 1, policy_version 1620869 (0.0008) [2023-12-27 16:53:10,122][20381] Updated weights for policy 1, policy_version 1620879 (0.0011) [2023-12-27 16:53:10,563][20347] Updated weights for policy 0, policy_version 1622417 (0.0006) [2023-12-27 16:53:10,624][20347] Updated weights for policy 0, policy_version 1622427 (0.0007) [2023-12-27 16:53:10,685][20347] Updated weights for policy 0, policy_version 1622437 (0.0010) [2023-12-27 16:53:10,741][20347] Updated weights for policy 0, policy_version 1622447 (0.0011) [2023-12-27 16:53:10,841][20381] Updated weights for policy 1, policy_version 1620889 (0.0006) [2023-12-27 16:53:10,898][20381] Updated weights for policy 1, policy_version 1620899 (0.0007) [2023-12-27 16:53:10,961][20381] Updated weights for policy 1, policy_version 1620909 (0.0010) [2023-12-27 16:53:11,508][20347] Updated weights for policy 0, policy_version 1622457 (0.0008) [2023-12-27 16:53:11,571][20347] Updated weights for policy 0, policy_version 1622467 (0.0007) [2023-12-27 16:53:11,637][20347] Updated weights for policy 0, policy_version 1622477 (0.0009) [2023-12-27 16:53:11,671][20381] Updated weights for policy 1, policy_version 1620919 (0.0011) [2023-12-27 16:53:11,731][20381] Updated weights for policy 1, policy_version 1620929 (0.0010) [2023-12-27 16:53:11,793][20381] Updated weights for policy 1, policy_version 1620939 (0.0007) [2023-12-27 16:53:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 830439424. Throughput: 0: 10321.5, 1: 10281.8. Samples: 830435548. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:12,363][19170] Avg episode reward: [(0, '990.310'), (1, '922.570')] [2023-12-27 16:53:12,429][20347] Updated weights for policy 0, policy_version 1622487 (0.0007) [2023-12-27 16:53:12,470][20381] Updated weights for policy 1, policy_version 1620949 (0.0009) [2023-12-27 16:53:12,487][20347] Updated weights for policy 0, policy_version 1622497 (0.0008) [2023-12-27 16:53:12,526][20381] Updated weights for policy 1, policy_version 1620959 (0.0008) [2023-12-27 16:53:12,547][20347] Updated weights for policy 0, policy_version 1622507 (0.0006) [2023-12-27 16:53:12,588][20381] Updated weights for policy 1, policy_version 1620969 (0.0007) [2023-12-27 16:53:13,259][20347] Updated weights for policy 0, policy_version 1622517 (0.0008) [2023-12-27 16:53:13,271][20381] Updated weights for policy 1, policy_version 1620979 (0.0008) [2023-12-27 16:53:13,317][20347] Updated weights for policy 0, policy_version 1622527 (0.0006) [2023-12-27 16:53:13,320][20381] Updated weights for policy 1, policy_version 1620989 (0.0011) [2023-12-27 16:53:13,366][20381] Updated weights for policy 1, policy_version 1620999 (0.0010) [2023-12-27 16:53:13,368][20347] Updated weights for policy 0, policy_version 1622537 (0.0006) [2023-12-27 16:53:13,958][20347] Updated weights for policy 0, policy_version 1622547 (0.0006) [2023-12-27 16:53:14,003][20347] Updated weights for policy 0, policy_version 1622557 (0.0007) [2023-12-27 16:53:14,051][20347] Updated weights for policy 0, policy_version 1622567 (0.0007) [2023-12-27 16:53:14,069][20381] Updated weights for policy 1, policy_version 1621009 (0.0010) [2023-12-27 16:53:14,118][20381] Updated weights for policy 1, policy_version 1621019 (0.0009) [2023-12-27 16:53:14,174][20381] Updated weights for policy 1, policy_version 1621029 (0.0005) [2023-12-27 16:53:14,226][20381] Updated weights for policy 1, policy_version 1621039 (0.0006) [2023-12-27 16:53:14,813][20347] Updated weights for policy 0, policy_version 1622577 (0.0006) [2023-12-27 16:53:14,826][20381] Updated weights for policy 1, policy_version 1621049 (0.0009) [2023-12-27 16:53:14,867][20347] Updated weights for policy 0, policy_version 1622587 (0.0006) [2023-12-27 16:53:14,883][20381] Updated weights for policy 1, policy_version 1621059 (0.0010) [2023-12-27 16:53:14,934][20347] Updated weights for policy 0, policy_version 1622597 (0.0007) [2023-12-27 16:53:14,946][20381] Updated weights for policy 1, policy_version 1621069 (0.0008) [2023-12-27 16:53:14,995][20347] Updated weights for policy 0, policy_version 1622607 (0.0008) [2023-12-27 16:53:15,613][20347] Updated weights for policy 0, policy_version 1622617 (0.0006) [2023-12-27 16:53:15,646][20381] Updated weights for policy 1, policy_version 1621079 (0.0011) [2023-12-27 16:53:15,668][20347] Updated weights for policy 0, policy_version 1622627 (0.0010) [2023-12-27 16:53:15,701][20381] Updated weights for policy 1, policy_version 1621089 (0.0010) [2023-12-27 16:53:15,718][20347] Updated weights for policy 0, policy_version 1622637 (0.0010) [2023-12-27 16:53:15,756][20381] Updated weights for policy 1, policy_version 1621099 (0.0010) [2023-12-27 16:53:16,312][20381] Updated weights for policy 1, policy_version 1621109 (0.0008) [2023-12-27 16:53:16,376][20381] Updated weights for policy 1, policy_version 1621119 (0.0007) [2023-12-27 16:53:16,404][20347] Updated weights for policy 0, policy_version 1622647 (0.0008) [2023-12-27 16:53:16,433][20381] Updated weights for policy 1, policy_version 1621129 (0.0006) [2023-12-27 16:53:16,463][20347] Updated weights for policy 0, policy_version 1622657 (0.0007) [2023-12-27 16:53:16,528][20347] Updated weights for policy 0, policy_version 1622667 (0.0008) [2023-12-27 16:53:16,967][20381] Updated weights for policy 1, policy_version 1621139 (0.0005) [2023-12-27 16:53:17,022][20381] Updated weights for policy 1, policy_version 1621149 (0.0006) [2023-12-27 16:53:17,074][20381] Updated weights for policy 1, policy_version 1621159 (0.0008) [2023-12-27 16:53:17,113][20347] Updated weights for policy 0, policy_version 1622677 (0.0006) [2023-12-27 16:53:17,174][20347] Updated weights for policy 0, policy_version 1622687 (0.0007) [2023-12-27 16:53:17,228][20347] Updated weights for policy 0, policy_version 1622697 (0.0006) [2023-12-27 16:53:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 830554112. Throughput: 0: 10311.6, 1: 10345.2. Samples: 830560416. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:17,363][19170] Avg episode reward: [(0, '990.310'), (1, '941.310')] [2023-12-27 16:53:17,743][20381] Updated weights for policy 1, policy_version 1621169 (0.0008) [2023-12-27 16:53:17,801][20381] Updated weights for policy 1, policy_version 1621179 (0.0007) [2023-12-27 16:53:17,842][20347] Updated weights for policy 0, policy_version 1622707 (0.0006) [2023-12-27 16:53:17,859][20381] Updated weights for policy 1, policy_version 1621189 (0.0005) [2023-12-27 16:53:17,894][20347] Updated weights for policy 0, policy_version 1622717 (0.0008) [2023-12-27 16:53:17,917][20381] Updated weights for policy 1, policy_version 1621199 (0.0005) [2023-12-27 16:53:17,951][20347] Updated weights for policy 0, policy_version 1622727 (0.0008) [2023-12-27 16:53:18,466][20381] Updated weights for policy 1, policy_version 1621209 (0.0009) [2023-12-27 16:53:18,521][20381] Updated weights for policy 1, policy_version 1621219 (0.0008) [2023-12-27 16:53:18,584][20381] Updated weights for policy 1, policy_version 1621229 (0.0008) [2023-12-27 16:53:18,685][20347] Updated weights for policy 0, policy_version 1622737 (0.0007) [2023-12-27 16:53:18,740][20347] Updated weights for policy 0, policy_version 1622747 (0.0007) [2023-12-27 16:53:18,798][20347] Updated weights for policy 0, policy_version 1622757 (0.0007) [2023-12-27 16:53:18,854][20347] Updated weights for policy 0, policy_version 1622767 (0.0008) [2023-12-27 16:53:19,297][20381] Updated weights for policy 1, policy_version 1621239 (0.0008) [2023-12-27 16:53:19,361][20381] Updated weights for policy 1, policy_version 1621249 (0.0009) [2023-12-27 16:53:19,420][20381] Updated weights for policy 1, policy_version 1621259 (0.0006) [2023-12-27 16:53:19,600][20347] Updated weights for policy 0, policy_version 1622777 (0.0008) [2023-12-27 16:53:19,663][20347] Updated weights for policy 0, policy_version 1622787 (0.0009) [2023-12-27 16:53:19,730][20347] Updated weights for policy 0, policy_version 1622797 (0.0009) [2023-12-27 16:53:20,089][20381] Updated weights for policy 1, policy_version 1621269 (0.0007) [2023-12-27 16:53:20,144][20381] Updated weights for policy 1, policy_version 1621279 (0.0008) [2023-12-27 16:53:20,202][20381] Updated weights for policy 1, policy_version 1621289 (0.0008) [2023-12-27 16:53:20,438][20347] Updated weights for policy 0, policy_version 1622807 (0.0009) [2023-12-27 16:53:20,498][20347] Updated weights for policy 0, policy_version 1622817 (0.0008) [2023-12-27 16:53:20,557][20347] Updated weights for policy 0, policy_version 1622827 (0.0007) [2023-12-27 16:53:20,847][20381] Updated weights for policy 1, policy_version 1621299 (0.0008) [2023-12-27 16:53:20,900][20381] Updated weights for policy 1, policy_version 1621309 (0.0005) [2023-12-27 16:53:20,962][20381] Updated weights for policy 1, policy_version 1621319 (0.0007) [2023-12-27 16:53:21,334][20347] Updated weights for policy 0, policy_version 1622837 (0.0007) [2023-12-27 16:53:21,401][20347] Updated weights for policy 0, policy_version 1622847 (0.0007) [2023-12-27 16:53:21,469][20347] Updated weights for policy 0, policy_version 1622857 (0.0008) [2023-12-27 16:53:21,698][20381] Updated weights for policy 1, policy_version 1621329 (0.0007) [2023-12-27 16:53:21,771][20381] Updated weights for policy 1, policy_version 1621339 (0.0009) [2023-12-27 16:53:21,836][20381] Updated weights for policy 1, policy_version 1621349 (0.0008) [2023-12-27 16:53:21,902][20381] Updated weights for policy 1, policy_version 1621359 (0.0008) [2023-12-27 16:53:22,213][20347] Updated weights for policy 0, policy_version 1622867 (0.0007) [2023-12-27 16:53:22,280][20347] Updated weights for policy 0, policy_version 1622877 (0.0008) [2023-12-27 16:53:22,349][20347] Updated weights for policy 0, policy_version 1622887 (0.0007) [2023-12-27 16:53:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 830644224. Throughput: 0: 10265.3, 1: 10378.1. Samples: 830621200. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.220')] [2023-12-27 16:53:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001621360_415113216.pth... [2023-12-27 16:53:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001620144_414801920.pth [2023-12-27 16:53:22,409][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001622896_415539200.pth... [2023-12-27 16:53:22,413][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001621680_415227904.pth [2023-12-27 16:53:22,566][20381] Updated weights for policy 1, policy_version 1621369 (0.0009) [2023-12-27 16:53:22,623][20381] Updated weights for policy 1, policy_version 1621379 (0.0009) [2023-12-27 16:53:22,687][20381] Updated weights for policy 1, policy_version 1621389 (0.0006) [2023-12-27 16:53:23,044][20347] Updated weights for policy 0, policy_version 1622897 (0.0005) [2023-12-27 16:53:23,102][20347] Updated weights for policy 0, policy_version 1622907 (0.0006) [2023-12-27 16:53:23,158][20347] Updated weights for policy 0, policy_version 1622917 (0.0008) [2023-12-27 16:53:23,215][20347] Updated weights for policy 0, policy_version 1622927 (0.0008) [2023-12-27 16:53:23,316][20381] Updated weights for policy 1, policy_version 1621399 (0.0006) [2023-12-27 16:53:23,361][20381] Updated weights for policy 1, policy_version 1621409 (0.0009) [2023-12-27 16:53:23,406][20381] Updated weights for policy 1, policy_version 1621419 (0.0008) [2023-12-27 16:53:23,862][20347] Updated weights for policy 0, policy_version 1622937 (0.0008) [2023-12-27 16:53:23,920][20347] Updated weights for policy 0, policy_version 1622947 (0.0008) [2023-12-27 16:53:23,973][20347] Updated weights for policy 0, policy_version 1622957 (0.0008) [2023-12-27 16:53:24,097][20381] Updated weights for policy 1, policy_version 1621429 (0.0008) [2023-12-27 16:53:24,152][20381] Updated weights for policy 1, policy_version 1621439 (0.0007) [2023-12-27 16:53:24,205][20381] Updated weights for policy 1, policy_version 1621449 (0.0008) [2023-12-27 16:53:24,698][20347] Updated weights for policy 0, policy_version 1622967 (0.0007) [2023-12-27 16:53:24,753][20347] Updated weights for policy 0, policy_version 1622977 (0.0008) [2023-12-27 16:53:24,800][20347] Updated weights for policy 0, policy_version 1622987 (0.0007) [2023-12-27 16:53:24,828][20381] Updated weights for policy 1, policy_version 1621459 (0.0007) [2023-12-27 16:53:24,875][20381] Updated weights for policy 1, policy_version 1621469 (0.0007) [2023-12-27 16:53:24,923][20381] Updated weights for policy 1, policy_version 1621479 (0.0007) [2023-12-27 16:53:25,485][20347] Updated weights for policy 0, policy_version 1622997 (0.0008) [2023-12-27 16:53:25,542][20347] Updated weights for policy 0, policy_version 1623007 (0.0008) [2023-12-27 16:53:25,597][20347] Updated weights for policy 0, policy_version 1623017 (0.0008) [2023-12-27 16:53:25,622][20381] Updated weights for policy 1, policy_version 1621489 (0.0008) [2023-12-27 16:53:25,673][20381] Updated weights for policy 1, policy_version 1621499 (0.0010) [2023-12-27 16:53:25,719][20381] Updated weights for policy 1, policy_version 1621509 (0.0009) [2023-12-27 16:53:25,760][20381] Updated weights for policy 1, policy_version 1621519 (0.0007) [2023-12-27 16:53:26,192][20347] Updated weights for policy 0, policy_version 1623027 (0.0007) [2023-12-27 16:53:26,253][20347] Updated weights for policy 0, policy_version 1623037 (0.0005) [2023-12-27 16:53:26,307][20347] Updated weights for policy 0, policy_version 1623047 (0.0007) [2023-12-27 16:53:26,512][20381] Updated weights for policy 1, policy_version 1621529 (0.0010) [2023-12-27 16:53:26,567][20381] Updated weights for policy 1, policy_version 1621539 (0.0010) [2023-12-27 16:53:26,621][20381] Updated weights for policy 1, policy_version 1621549 (0.0010) [2023-12-27 16:53:26,902][20347] Updated weights for policy 0, policy_version 1623057 (0.0005) [2023-12-27 16:53:26,957][20347] Updated weights for policy 0, policy_version 1623067 (0.0005) [2023-12-27 16:53:27,010][20347] Updated weights for policy 0, policy_version 1623077 (0.0010) [2023-12-27 16:53:27,058][20347] Updated weights for policy 0, policy_version 1623087 (0.0010) [2023-12-27 16:53:27,240][20381] Updated weights for policy 1, policy_version 1621559 (0.0007) [2023-12-27 16:53:27,288][20381] Updated weights for policy 1, policy_version 1621569 (0.0005) [2023-12-27 16:53:27,341][20381] Updated weights for policy 1, policy_version 1621579 (0.0009) [2023-12-27 16:53:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 830750720. Throughput: 0: 10194.1, 1: 10438.9. Samples: 830742164. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.560')] [2023-12-27 16:53:27,667][20347] Updated weights for policy 0, policy_version 1623097 (0.0010) [2023-12-27 16:53:27,714][20347] Updated weights for policy 0, policy_version 1623107 (0.0010) [2023-12-27 16:53:27,767][20347] Updated weights for policy 0, policy_version 1623117 (0.0008) [2023-12-27 16:53:27,937][20381] Updated weights for policy 1, policy_version 1621589 (0.0005) [2023-12-27 16:53:27,992][20381] Updated weights for policy 1, policy_version 1621599 (0.0005) [2023-12-27 16:53:28,043][20381] Updated weights for policy 1, policy_version 1621609 (0.0006) [2023-12-27 16:53:28,435][20347] Updated weights for policy 0, policy_version 1623127 (0.0006) [2023-12-27 16:53:28,493][20347] Updated weights for policy 0, policy_version 1623137 (0.0010) [2023-12-27 16:53:28,551][20347] Updated weights for policy 0, policy_version 1623147 (0.0010) [2023-12-27 16:53:28,611][20381] Updated weights for policy 1, policy_version 1621619 (0.0005) [2023-12-27 16:53:28,671][20381] Updated weights for policy 1, policy_version 1621629 (0.0006) [2023-12-27 16:53:28,729][20381] Updated weights for policy 1, policy_version 1621639 (0.0007) [2023-12-27 16:53:29,177][20347] Updated weights for policy 0, policy_version 1623157 (0.0008) [2023-12-27 16:53:29,237][20347] Updated weights for policy 0, policy_version 1623167 (0.0007) [2023-12-27 16:53:29,300][20347] Updated weights for policy 0, policy_version 1623177 (0.0008) [2023-12-27 16:53:29,338][20381] Updated weights for policy 1, policy_version 1621649 (0.0011) [2023-12-27 16:53:29,402][20381] Updated weights for policy 1, policy_version 1621659 (0.0008) [2023-12-27 16:53:29,461][20381] Updated weights for policy 1, policy_version 1621669 (0.0005) [2023-12-27 16:53:29,512][20381] Updated weights for policy 1, policy_version 1621679 (0.0005) [2023-12-27 16:53:30,002][20347] Updated weights for policy 0, policy_version 1623187 (0.0007) [2023-12-27 16:53:30,057][20347] Updated weights for policy 0, policy_version 1623197 (0.0006) [2023-12-27 16:53:30,123][20347] Updated weights for policy 0, policy_version 1623207 (0.0007) [2023-12-27 16:53:30,151][20381] Updated weights for policy 1, policy_version 1621689 (0.0006) [2023-12-27 16:53:30,210][20381] Updated weights for policy 1, policy_version 1621699 (0.0008) [2023-12-27 16:53:30,267][20381] Updated weights for policy 1, policy_version 1621709 (0.0005) [2023-12-27 16:53:30,762][20347] Updated weights for policy 0, policy_version 1623217 (0.0007) [2023-12-27 16:53:30,814][20381] Updated weights for policy 1, policy_version 1621719 (0.0005) [2023-12-27 16:53:30,819][20347] Updated weights for policy 0, policy_version 1623227 (0.0010) [2023-12-27 16:53:30,861][20381] Updated weights for policy 1, policy_version 1621729 (0.0005) [2023-12-27 16:53:30,867][20347] Updated weights for policy 0, policy_version 1623237 (0.0010) [2023-12-27 16:53:30,911][20381] Updated weights for policy 1, policy_version 1621739 (0.0005) [2023-12-27 16:53:30,923][20347] Updated weights for policy 0, policy_version 1623247 (0.0010) [2023-12-27 16:53:31,614][20347] Updated weights for policy 0, policy_version 1623257 (0.0008) [2023-12-27 16:53:31,651][20381] Updated weights for policy 1, policy_version 1621749 (0.0008) [2023-12-27 16:53:31,682][20347] Updated weights for policy 0, policy_version 1623267 (0.0007) [2023-12-27 16:53:31,711][20381] Updated weights for policy 1, policy_version 1621759 (0.0008) [2023-12-27 16:53:31,751][20347] Updated weights for policy 0, policy_version 1623277 (0.0008) [2023-12-27 16:53:31,779][20381] Updated weights for policy 1, policy_version 1621769 (0.0008) [2023-12-27 16:53:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 830857216. Throughput: 0: 10220.6, 1: 10537.1. Samples: 830871896. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.060')] [2023-12-27 16:53:32,406][20381] Updated weights for policy 1, policy_version 1621779 (0.0008) [2023-12-27 16:53:32,466][20381] Updated weights for policy 1, policy_version 1621789 (0.0008) [2023-12-27 16:53:32,502][20292] KL-divergence is very high: 125.7521 [2023-12-27 16:53:32,520][20347] Updated weights for policy 0, policy_version 1623287 (0.0006) [2023-12-27 16:53:32,526][20381] Updated weights for policy 1, policy_version 1621799 (0.0007) [2023-12-27 16:53:32,529][20292] KL-divergence is very high: 113.3602 [2023-12-27 16:53:32,544][20292] KL-divergence is very high: 108.3228 [2023-12-27 16:53:32,577][20347] Updated weights for policy 0, policy_version 1623297 (0.0006) [2023-12-27 16:53:32,634][20347] Updated weights for policy 0, policy_version 1623308 (0.0008) [2023-12-27 16:53:33,202][20381] Updated weights for policy 1, policy_version 1621809 (0.0006) [2023-12-27 16:53:33,263][20381] Updated weights for policy 1, policy_version 1621819 (0.0008) [2023-12-27 16:53:33,318][20381] Updated weights for policy 1, policy_version 1621829 (0.0006) [2023-12-27 16:53:33,327][20347] Updated weights for policy 0, policy_version 1623318 (0.0007) [2023-12-27 16:53:33,364][20381] Updated weights for policy 1, policy_version 1621839 (0.0006) [2023-12-27 16:53:33,380][20347] Updated weights for policy 0, policy_version 1623328 (0.0006) [2023-12-27 16:53:33,431][20347] Updated weights for policy 0, policy_version 1623338 (0.0007) [2023-12-27 16:53:34,036][20381] Updated weights for policy 1, policy_version 1621849 (0.0008) [2023-12-27 16:53:34,073][20347] Updated weights for policy 0, policy_version 1623348 (0.0009) [2023-12-27 16:53:34,096][20381] Updated weights for policy 1, policy_version 1621859 (0.0006) [2023-12-27 16:53:34,131][20347] Updated weights for policy 0, policy_version 1623358 (0.0010) [2023-12-27 16:53:34,163][20381] Updated weights for policy 1, policy_version 1621869 (0.0007) [2023-12-27 16:53:34,191][20347] Updated weights for policy 0, policy_version 1623368 (0.0008) [2023-12-27 16:53:34,820][20381] Updated weights for policy 1, policy_version 1621879 (0.0011) [2023-12-27 16:53:34,871][20347] Updated weights for policy 0, policy_version 1623378 (0.0006) [2023-12-27 16:53:34,878][20381] Updated weights for policy 1, policy_version 1621889 (0.0011) [2023-12-27 16:53:34,929][20381] Updated weights for policy 1, policy_version 1621899 (0.0006) [2023-12-27 16:53:34,929][20347] Updated weights for policy 0, policy_version 1623388 (0.0006) [2023-12-27 16:53:34,983][20347] Updated weights for policy 0, policy_version 1623398 (0.0007) [2023-12-27 16:53:35,030][20347] Updated weights for policy 0, policy_version 1623408 (0.0008) [2023-12-27 16:53:35,501][20381] Updated weights for policy 1, policy_version 1621909 (0.0005) [2023-12-27 16:53:35,558][20381] Updated weights for policy 1, policy_version 1621919 (0.0007) [2023-12-27 16:53:35,586][20347] Updated weights for policy 0, policy_version 1623418 (0.0005) [2023-12-27 16:53:35,622][20381] Updated weights for policy 1, policy_version 1621929 (0.0010) [2023-12-27 16:53:35,637][20347] Updated weights for policy 0, policy_version 1623428 (0.0005) [2023-12-27 16:53:35,693][20347] Updated weights for policy 0, policy_version 1623438 (0.0008) [2023-12-27 16:53:36,324][20381] Updated weights for policy 1, policy_version 1621939 (0.0009) [2023-12-27 16:53:36,379][20381] Updated weights for policy 1, policy_version 1621949 (0.0007) [2023-12-27 16:53:36,388][20347] Updated weights for policy 0, policy_version 1623448 (0.0009) [2023-12-27 16:53:36,436][20381] Updated weights for policy 1, policy_version 1621959 (0.0006) [2023-12-27 16:53:36,440][20347] Updated weights for policy 0, policy_version 1623458 (0.0007) [2023-12-27 16:53:36,500][20347] Updated weights for policy 0, policy_version 1623468 (0.0008) [2023-12-27 16:53:37,143][20381] Updated weights for policy 1, policy_version 1621969 (0.0008) [2023-12-27 16:53:37,204][20381] Updated weights for policy 1, policy_version 1621979 (0.0007) [2023-12-27 16:53:37,208][20347] Updated weights for policy 0, policy_version 1623478 (0.0009) [2023-12-27 16:53:37,261][20381] Updated weights for policy 1, policy_version 1621989 (0.0006) [2023-12-27 16:53:37,263][20347] Updated weights for policy 0, policy_version 1623488 (0.0010) [2023-12-27 16:53:37,318][20381] Updated weights for policy 1, policy_version 1621999 (0.0005) [2023-12-27 16:53:37,319][20347] Updated weights for policy 0, policy_version 1623498 (0.0005) [2023-12-27 16:53:37,362][19170] Fps is (10 sec: 22118.3, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 830971904. Throughput: 0: 10197.4, 1: 10538.9. Samples: 830933768. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.180')] [2023-12-27 16:53:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001622000_415277056.pth... [2023-12-27 16:53:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001623504_415694848.pth... [2023-12-27 16:53:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001620752_414957568.pth [2023-12-27 16:53:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001622320_415391744.pth [2023-12-27 16:53:37,884][20381] Updated weights for policy 1, policy_version 1622009 (0.0007) [2023-12-27 16:53:37,940][20381] Updated weights for policy 1, policy_version 1622019 (0.0009) [2023-12-27 16:53:37,954][20347] Updated weights for policy 0, policy_version 1623508 (0.0007) [2023-12-27 16:53:37,996][20381] Updated weights for policy 1, policy_version 1622029 (0.0007) [2023-12-27 16:53:38,013][20347] Updated weights for policy 0, policy_version 1623518 (0.0008) [2023-12-27 16:53:38,059][20347] Updated weights for policy 0, policy_version 1623528 (0.0009) [2023-12-27 16:53:38,552][20381] Updated weights for policy 1, policy_version 1622039 (0.0009) [2023-12-27 16:53:38,601][20381] Updated weights for policy 1, policy_version 1622049 (0.0010) [2023-12-27 16:53:38,649][20381] Updated weights for policy 1, policy_version 1622059 (0.0010) [2023-12-27 16:53:38,735][20347] Updated weights for policy 0, policy_version 1623538 (0.0010) [2023-12-27 16:53:38,800][20347] Updated weights for policy 0, policy_version 1623548 (0.0008) [2023-12-27 16:53:38,868][20347] Updated weights for policy 0, policy_version 1623558 (0.0007) [2023-12-27 16:53:38,934][20347] Updated weights for policy 0, policy_version 1623568 (0.0008) [2023-12-27 16:53:39,295][20381] Updated weights for policy 1, policy_version 1622069 (0.0009) [2023-12-27 16:53:39,354][20381] Updated weights for policy 1, policy_version 1622079 (0.0008) [2023-12-27 16:53:39,438][20381] Updated weights for policy 1, policy_version 1622089 (0.0008) [2023-12-27 16:53:39,580][20347] Updated weights for policy 0, policy_version 1623578 (0.0007) [2023-12-27 16:53:39,642][20347] Updated weights for policy 0, policy_version 1623588 (0.0007) [2023-12-27 16:53:39,707][20347] Updated weights for policy 0, policy_version 1623598 (0.0007) [2023-12-27 16:53:40,166][20381] Updated weights for policy 1, policy_version 1622099 (0.0008) [2023-12-27 16:53:40,226][20381] Updated weights for policy 1, policy_version 1622109 (0.0008) [2023-12-27 16:53:40,279][20381] Updated weights for policy 1, policy_version 1622119 (0.0008) [2023-12-27 16:53:40,399][20347] Updated weights for policy 0, policy_version 1623608 (0.0007) [2023-12-27 16:53:40,460][20347] Updated weights for policy 0, policy_version 1623618 (0.0008) [2023-12-27 16:53:40,525][20347] Updated weights for policy 0, policy_version 1623628 (0.0007) [2023-12-27 16:53:40,948][20381] Updated weights for policy 1, policy_version 1622129 (0.0008) [2023-12-27 16:53:41,005][20381] Updated weights for policy 1, policy_version 1622139 (0.0005) [2023-12-27 16:53:41,073][20381] Updated weights for policy 1, policy_version 1622149 (0.0008) [2023-12-27 16:53:41,135][20381] Updated weights for policy 1, policy_version 1622159 (0.0007) [2023-12-27 16:53:41,218][20347] Updated weights for policy 0, policy_version 1623638 (0.0010) [2023-12-27 16:53:41,283][20347] Updated weights for policy 0, policy_version 1623648 (0.0008) [2023-12-27 16:53:41,347][20347] Updated weights for policy 0, policy_version 1623658 (0.0008) [2023-12-27 16:53:41,844][20381] Updated weights for policy 1, policy_version 1622169 (0.0006) [2023-12-27 16:53:41,907][20381] Updated weights for policy 1, policy_version 1622179 (0.0006) [2023-12-27 16:53:41,968][20381] Updated weights for policy 1, policy_version 1622189 (0.0006) [2023-12-27 16:53:42,107][20347] Updated weights for policy 0, policy_version 1623668 (0.0009) [2023-12-27 16:53:42,164][20347] Updated weights for policy 0, policy_version 1623678 (0.0007) [2023-12-27 16:53:42,224][20347] Updated weights for policy 0, policy_version 1623688 (0.0008) [2023-12-27 16:53:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 831070208. Throughput: 0: 10253.0, 1: 10503.7. Samples: 831059496. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.430')] [2023-12-27 16:53:42,630][20381] Updated weights for policy 1, policy_version 1622199 (0.0008) [2023-12-27 16:53:42,696][20381] Updated weights for policy 1, policy_version 1622209 (0.0008) [2023-12-27 16:53:42,759][20381] Updated weights for policy 1, policy_version 1622220 (0.0006) [2023-12-27 16:53:42,926][20347] Updated weights for policy 0, policy_version 1623698 (0.0008) [2023-12-27 16:53:42,973][20347] Updated weights for policy 0, policy_version 1623708 (0.0008) [2023-12-27 16:53:43,033][20347] Updated weights for policy 0, policy_version 1623718 (0.0008) [2023-12-27 16:53:43,090][20347] Updated weights for policy 0, policy_version 1623728 (0.0008) [2023-12-27 16:53:43,386][20381] Updated weights for policy 1, policy_version 1622230 (0.0006) [2023-12-27 16:53:43,451][20381] Updated weights for policy 1, policy_version 1622240 (0.0008) [2023-12-27 16:53:43,515][20381] Updated weights for policy 1, policy_version 1622250 (0.0008) [2023-12-27 16:53:43,747][20347] Updated weights for policy 0, policy_version 1623738 (0.0007) [2023-12-27 16:53:43,810][20347] Updated weights for policy 0, policy_version 1623748 (0.0007) [2023-12-27 16:53:43,865][20347] Updated weights for policy 0, policy_version 1623758 (0.0008) [2023-12-27 16:53:44,219][20381] Updated weights for policy 1, policy_version 1622260 (0.0008) [2023-12-27 16:53:44,271][20381] Updated weights for policy 1, policy_version 1622270 (0.0008) [2023-12-27 16:53:44,332][20381] Updated weights for policy 1, policy_version 1622280 (0.0008) [2023-12-27 16:53:44,540][20347] Updated weights for policy 0, policy_version 1623768 (0.0008) [2023-12-27 16:53:44,598][20347] Updated weights for policy 0, policy_version 1623778 (0.0007) [2023-12-27 16:53:44,652][20347] Updated weights for policy 0, policy_version 1623788 (0.0008) [2023-12-27 16:53:45,062][20381] Updated weights for policy 1, policy_version 1622290 (0.0007) [2023-12-27 16:53:45,122][20381] Updated weights for policy 1, policy_version 1622300 (0.0008) [2023-12-27 16:53:45,182][20381] Updated weights for policy 1, policy_version 1622310 (0.0008) [2023-12-27 16:53:45,242][20381] Updated weights for policy 1, policy_version 1622320 (0.0008) [2023-12-27 16:53:45,334][20347] Updated weights for policy 0, policy_version 1623798 (0.0008) [2023-12-27 16:53:45,389][20347] Updated weights for policy 0, policy_version 1623808 (0.0008) [2023-12-27 16:53:45,440][20347] Updated weights for policy 0, policy_version 1623818 (0.0009) [2023-12-27 16:53:45,880][20381] Updated weights for policy 1, policy_version 1622330 (0.0007) [2023-12-27 16:53:45,939][20381] Updated weights for policy 1, policy_version 1622340 (0.0008) [2023-12-27 16:53:45,989][20381] Updated weights for policy 1, policy_version 1622350 (0.0006) [2023-12-27 16:53:46,167][20347] Updated weights for policy 0, policy_version 1623828 (0.0008) [2023-12-27 16:53:46,216][20347] Updated weights for policy 0, policy_version 1623838 (0.0008) [2023-12-27 16:53:46,265][20347] Updated weights for policy 0, policy_version 1623848 (0.0007) [2023-12-27 16:53:46,619][20381] Updated weights for policy 1, policy_version 1622360 (0.0008) [2023-12-27 16:53:46,680][20381] Updated weights for policy 1, policy_version 1622370 (0.0008) [2023-12-27 16:53:46,743][20381] Updated weights for policy 1, policy_version 1622380 (0.0007) [2023-12-27 16:53:47,003][20347] Updated weights for policy 0, policy_version 1623858 (0.0008) [2023-12-27 16:53:47,063][20347] Updated weights for policy 0, policy_version 1623868 (0.0008) [2023-12-27 16:53:47,122][20347] Updated weights for policy 0, policy_version 1623878 (0.0007) [2023-12-27 16:53:47,176][20347] Updated weights for policy 0, policy_version 1623888 (0.0008) [2023-12-27 16:53:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 831168512. Throughput: 0: 10187.4, 1: 10519.8. Samples: 831179736. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.650')] [2023-12-27 16:53:47,390][20381] Updated weights for policy 1, policy_version 1622390 (0.0006) [2023-12-27 16:53:47,443][20381] Updated weights for policy 1, policy_version 1622400 (0.0005) [2023-12-27 16:53:47,490][20381] Updated weights for policy 1, policy_version 1622410 (0.0005) [2023-12-27 16:53:47,864][20347] Updated weights for policy 0, policy_version 1623898 (0.0011) [2023-12-27 16:53:47,918][20347] Updated weights for policy 0, policy_version 1623908 (0.0010) [2023-12-27 16:53:47,972][20347] Updated weights for policy 0, policy_version 1623918 (0.0010) [2023-12-27 16:53:48,031][20381] Updated weights for policy 1, policy_version 1622420 (0.0006) [2023-12-27 16:53:48,090][20381] Updated weights for policy 1, policy_version 1622430 (0.0007) [2023-12-27 16:53:48,149][20381] Updated weights for policy 1, policy_version 1622440 (0.0007) [2023-12-27 16:53:48,682][20347] Updated weights for policy 0, policy_version 1623928 (0.0008) [2023-12-27 16:53:48,726][20381] Updated weights for policy 1, policy_version 1622450 (0.0007) [2023-12-27 16:53:48,743][20347] Updated weights for policy 0, policy_version 1623938 (0.0007) [2023-12-27 16:53:48,784][20381] Updated weights for policy 1, policy_version 1622460 (0.0008) [2023-12-27 16:53:48,803][20347] Updated weights for policy 0, policy_version 1623948 (0.0009) [2023-12-27 16:53:48,834][20381] Updated weights for policy 1, policy_version 1622470 (0.0008) [2023-12-27 16:53:48,894][20381] Updated weights for policy 1, policy_version 1622480 (0.0008) [2023-12-27 16:53:49,463][20347] Updated weights for policy 0, policy_version 1623958 (0.0007) [2023-12-27 16:53:49,531][20347] Updated weights for policy 0, policy_version 1623968 (0.0007) [2023-12-27 16:53:49,584][20347] Updated weights for policy 0, policy_version 1623978 (0.0005) [2023-12-27 16:53:49,585][20381] Updated weights for policy 1, policy_version 1622490 (0.0007) [2023-12-27 16:53:49,647][20381] Updated weights for policy 1, policy_version 1622500 (0.0005) [2023-12-27 16:53:49,709][20381] Updated weights for policy 1, policy_version 1622510 (0.0005) [2023-12-27 16:53:50,272][20347] Updated weights for policy 0, policy_version 1623988 (0.0007) [2023-12-27 16:53:50,306][20381] Updated weights for policy 1, policy_version 1622520 (0.0006) [2023-12-27 16:53:50,333][20347] Updated weights for policy 0, policy_version 1623998 (0.0008) [2023-12-27 16:53:50,355][20381] Updated weights for policy 1, policy_version 1622530 (0.0006) [2023-12-27 16:53:50,391][20347] Updated weights for policy 0, policy_version 1624008 (0.0008) [2023-12-27 16:53:50,408][20381] Updated weights for policy 1, policy_version 1622540 (0.0007) [2023-12-27 16:53:51,087][20381] Updated weights for policy 1, policy_version 1622550 (0.0007) [2023-12-27 16:53:51,107][20347] Updated weights for policy 0, policy_version 1624018 (0.0007) [2023-12-27 16:53:51,148][20381] Updated weights for policy 1, policy_version 1622560 (0.0006) [2023-12-27 16:53:51,168][20347] Updated weights for policy 0, policy_version 1624028 (0.0008) [2023-12-27 16:53:51,201][20381] Updated weights for policy 1, policy_version 1622570 (0.0006) [2023-12-27 16:53:51,219][20347] Updated weights for policy 0, policy_version 1624038 (0.0005) [2023-12-27 16:53:51,282][20347] Updated weights for policy 0, policy_version 1624048 (0.0008) [2023-12-27 16:53:51,918][20381] Updated weights for policy 1, policy_version 1622580 (0.0008) [2023-12-27 16:53:51,975][20381] Updated weights for policy 1, policy_version 1622590 (0.0007) [2023-12-27 16:53:52,040][20381] Updated weights for policy 1, policy_version 1622600 (0.0007) [2023-12-27 16:53:52,082][20347] Updated weights for policy 0, policy_version 1624058 (0.0011) [2023-12-27 16:53:52,136][20347] Updated weights for policy 0, policy_version 1624068 (0.0010) [2023-12-27 16:53:52,188][20347] Updated weights for policy 0, policy_version 1624078 (0.0005) [2023-12-27 16:53:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 831275008. Throughput: 0: 10112.6, 1: 10584.4. Samples: 831242724. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.800')] [2023-12-27 16:53:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001624080_415842304.pth... [2023-12-27 16:53:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001622608_415432704.pth... [2023-12-27 16:53:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001621360_415113216.pth [2023-12-27 16:53:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001622896_415539200.pth [2023-12-27 16:53:52,759][20381] Updated weights for policy 1, policy_version 1622610 (0.0007) [2023-12-27 16:53:52,815][20381] Updated weights for policy 1, policy_version 1622620 (0.0006) [2023-12-27 16:53:52,829][20347] Updated weights for policy 0, policy_version 1624088 (0.0007) [2023-12-27 16:53:52,873][20381] Updated weights for policy 1, policy_version 1622630 (0.0006) [2023-12-27 16:53:52,891][20347] Updated weights for policy 0, policy_version 1624098 (0.0008) [2023-12-27 16:53:52,929][20381] Updated weights for policy 1, policy_version 1622640 (0.0007) [2023-12-27 16:53:52,958][20347] Updated weights for policy 0, policy_version 1624108 (0.0007) [2023-12-27 16:53:53,603][20347] Updated weights for policy 0, policy_version 1624118 (0.0006) [2023-12-27 16:53:53,635][20381] Updated weights for policy 1, policy_version 1622650 (0.0007) [2023-12-27 16:53:53,655][20347] Updated weights for policy 0, policy_version 1624128 (0.0006) [2023-12-27 16:53:53,683][20381] Updated weights for policy 1, policy_version 1622660 (0.0005) [2023-12-27 16:53:53,711][20347] Updated weights for policy 0, policy_version 1624138 (0.0006) [2023-12-27 16:53:53,728][20381] Updated weights for policy 1, policy_version 1622670 (0.0005) [2023-12-27 16:53:54,408][20347] Updated weights for policy 0, policy_version 1624148 (0.0006) [2023-12-27 16:53:54,425][20381] Updated weights for policy 1, policy_version 1622680 (0.0005) [2023-12-27 16:53:54,461][20347] Updated weights for policy 0, policy_version 1624158 (0.0006) [2023-12-27 16:53:54,484][20381] Updated weights for policy 1, policy_version 1622690 (0.0008) [2023-12-27 16:53:54,520][20347] Updated weights for policy 0, policy_version 1624168 (0.0005) [2023-12-27 16:53:54,533][20381] Updated weights for policy 1, policy_version 1622700 (0.0006) [2023-12-27 16:53:55,160][20347] Updated weights for policy 0, policy_version 1624178 (0.0007) [2023-12-27 16:53:55,190][20381] Updated weights for policy 1, policy_version 1622710 (0.0006) [2023-12-27 16:53:55,216][20347] Updated weights for policy 0, policy_version 1624188 (0.0007) [2023-12-27 16:53:55,241][20381] Updated weights for policy 1, policy_version 1622720 (0.0007) [2023-12-27 16:53:55,266][20347] Updated weights for policy 0, policy_version 1624198 (0.0006) [2023-12-27 16:53:55,293][20381] Updated weights for policy 1, policy_version 1622730 (0.0006) [2023-12-27 16:53:55,316][20347] Updated weights for policy 0, policy_version 1624208 (0.0006) [2023-12-27 16:53:55,971][20347] Updated weights for policy 0, policy_version 1624218 (0.0008) [2023-12-27 16:53:55,977][20381] Updated weights for policy 1, policy_version 1622740 (0.0006) [2023-12-27 16:53:56,028][20347] Updated weights for policy 0, policy_version 1624228 (0.0008) [2023-12-27 16:53:56,039][20381] Updated weights for policy 1, policy_version 1622750 (0.0005) [2023-12-27 16:53:56,081][20347] Updated weights for policy 0, policy_version 1624238 (0.0008) [2023-12-27 16:53:56,103][20381] Updated weights for policy 1, policy_version 1622760 (0.0005) [2023-12-27 16:53:56,639][20381] Updated weights for policy 1, policy_version 1622770 (0.0006) [2023-12-27 16:53:56,699][20381] Updated weights for policy 1, policy_version 1622780 (0.0009) [2023-12-27 16:53:56,750][20347] Updated weights for policy 0, policy_version 1624248 (0.0005) [2023-12-27 16:53:56,752][20381] Updated weights for policy 1, policy_version 1622790 (0.0006) [2023-12-27 16:53:56,800][20347] Updated weights for policy 0, policy_version 1624258 (0.0006) [2023-12-27 16:53:56,804][20381] Updated weights for policy 1, policy_version 1622800 (0.0006) [2023-12-27 16:53:56,854][20347] Updated weights for policy 0, policy_version 1624268 (0.0005) [2023-12-27 16:53:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 831373312. Throughput: 0: 10111.7, 1: 10534.9. Samples: 831364648. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:53:57,363][19170] Avg episode reward: [(0, '990.280'), (1, '853.770')] [2023-12-27 16:53:57,461][20347] Updated weights for policy 0, policy_version 1624278 (0.0006) [2023-12-27 16:53:57,509][20347] Updated weights for policy 0, policy_version 1624288 (0.0006) [2023-12-27 16:53:57,526][20381] Updated weights for policy 1, policy_version 1622810 (0.0005) [2023-12-27 16:53:57,564][20347] Updated weights for policy 0, policy_version 1624298 (0.0006) [2023-12-27 16:53:57,577][20381] Updated weights for policy 1, policy_version 1622820 (0.0005) [2023-12-27 16:53:57,625][20381] Updated weights for policy 1, policy_version 1622830 (0.0007) [2023-12-27 16:53:58,158][20347] Updated weights for policy 0, policy_version 1624308 (0.0008) [2023-12-27 16:53:58,214][20347] Updated weights for policy 0, policy_version 1624318 (0.0008) [2023-12-27 16:53:58,275][20347] Updated weights for policy 0, policy_version 1624328 (0.0010) [2023-12-27 16:53:58,293][20381] Updated weights for policy 1, policy_version 1622840 (0.0011) [2023-12-27 16:53:58,353][20381] Updated weights for policy 1, policy_version 1622850 (0.0009) [2023-12-27 16:53:58,418][20381] Updated weights for policy 1, policy_version 1622860 (0.0007) [2023-12-27 16:53:59,162][20347] Updated weights for policy 0, policy_version 1624338 (0.0009) [2023-12-27 16:53:59,263][20347] Updated weights for policy 0, policy_version 1624350 (0.0010) [2023-12-27 16:53:59,289][20381] Updated weights for policy 1, policy_version 1622870 (0.0009) [2023-12-27 16:53:59,326][20347] Updated weights for policy 0, policy_version 1624360 (0.0009) [2023-12-27 16:53:59,346][20381] Updated weights for policy 1, policy_version 1622880 (0.0008) [2023-12-27 16:53:59,410][20381] Updated weights for policy 1, policy_version 1622890 (0.0011) [2023-12-27 16:54:00,010][20347] Updated weights for policy 0, policy_version 1624370 (0.0008) [2023-12-27 16:54:00,071][20347] Updated weights for policy 0, policy_version 1624380 (0.0007) [2023-12-27 16:54:00,093][20381] Updated weights for policy 1, policy_version 1622900 (0.0008) [2023-12-27 16:54:00,124][20347] Updated weights for policy 0, policy_version 1624390 (0.0007) [2023-12-27 16:54:00,145][20381] Updated weights for policy 1, policy_version 1622910 (0.0005) [2023-12-27 16:54:00,176][20347] Updated weights for policy 0, policy_version 1624400 (0.0006) [2023-12-27 16:54:00,201][20381] Updated weights for policy 1, policy_version 1622920 (0.0006) [2023-12-27 16:54:00,799][20381] Updated weights for policy 1, policy_version 1622930 (0.0008) [2023-12-27 16:54:00,859][20381] Updated weights for policy 1, policy_version 1622940 (0.0010) [2023-12-27 16:54:00,861][20347] Updated weights for policy 0, policy_version 1624410 (0.0006) [2023-12-27 16:54:00,910][20347] Updated weights for policy 0, policy_version 1624420 (0.0009) [2023-12-27 16:54:00,911][20381] Updated weights for policy 1, policy_version 1622950 (0.0010) [2023-12-27 16:54:00,960][20381] Updated weights for policy 1, policy_version 1622960 (0.0010) [2023-12-27 16:54:00,961][20347] Updated weights for policy 0, policy_version 1624430 (0.0008) [2023-12-27 16:54:01,669][20381] Updated weights for policy 1, policy_version 1622970 (0.0009) [2023-12-27 16:54:01,705][20347] Updated weights for policy 0, policy_version 1624440 (0.0010) [2023-12-27 16:54:01,716][20381] Updated weights for policy 1, policy_version 1622980 (0.0007) [2023-12-27 16:54:01,770][20347] Updated weights for policy 0, policy_version 1624450 (0.0009) [2023-12-27 16:54:01,775][20381] Updated weights for policy 1, policy_version 1622990 (0.0012) [2023-12-27 16:54:01,822][20347] Updated weights for policy 0, policy_version 1624460 (0.0007) [2023-12-27 16:54:02,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 831471616. Throughput: 0: 10108.3, 1: 10426.5. Samples: 831484480. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:54:02,363][19170] Avg episode reward: [(0, '990.280'), (1, '834.290')] [2023-12-27 16:54:02,449][20347] Updated weights for policy 0, policy_version 1624470 (0.0008) [2023-12-27 16:54:02,508][20347] Updated weights for policy 0, policy_version 1624480 (0.0007) [2023-12-27 16:54:02,554][20381] Updated weights for policy 1, policy_version 1623000 (0.0007) [2023-12-27 16:54:02,571][20347] Updated weights for policy 0, policy_version 1624490 (0.0006) [2023-12-27 16:54:02,615][20381] Updated weights for policy 1, policy_version 1623010 (0.0006) [2023-12-27 16:54:02,668][20381] Updated weights for policy 1, policy_version 1623020 (0.0007) [2023-12-27 16:54:03,179][20347] Updated weights for policy 0, policy_version 1624500 (0.0007) [2023-12-27 16:54:03,238][20347] Updated weights for policy 0, policy_version 1624510 (0.0008) [2023-12-27 16:54:03,293][20347] Updated weights for policy 0, policy_version 1624520 (0.0007) [2023-12-27 16:54:03,383][20381] Updated weights for policy 1, policy_version 1623030 (0.0006) [2023-12-27 16:54:03,444][20381] Updated weights for policy 1, policy_version 1623040 (0.0005) [2023-12-27 16:54:03,502][20381] Updated weights for policy 1, policy_version 1623050 (0.0005) [2023-12-27 16:54:03,932][20347] Updated weights for policy 0, policy_version 1624530 (0.0007) [2023-12-27 16:54:03,989][20347] Updated weights for policy 0, policy_version 1624540 (0.0007) [2023-12-27 16:54:04,052][20347] Updated weights for policy 0, policy_version 1624550 (0.0008) [2023-12-27 16:54:04,120][20347] Updated weights for policy 0, policy_version 1624560 (0.0007) [2023-12-27 16:54:04,144][20381] Updated weights for policy 1, policy_version 1623060 (0.0006) [2023-12-27 16:54:04,201][20381] Updated weights for policy 1, policy_version 1623070 (0.0008) [2023-12-27 16:54:04,259][20381] Updated weights for policy 1, policy_version 1623080 (0.0007) [2023-12-27 16:54:04,874][20347] Updated weights for policy 0, policy_version 1624570 (0.0008) [2023-12-27 16:54:04,922][20347] Updated weights for policy 0, policy_version 1624580 (0.0008) [2023-12-27 16:54:04,961][20381] Updated weights for policy 1, policy_version 1623090 (0.0006) [2023-12-27 16:54:04,977][20347] Updated weights for policy 0, policy_version 1624590 (0.0009) [2023-12-27 16:54:05,007][20381] Updated weights for policy 1, policy_version 1623100 (0.0005) [2023-12-27 16:54:05,065][20381] Updated weights for policy 1, policy_version 1623110 (0.0005) [2023-12-27 16:54:05,124][20381] Updated weights for policy 1, policy_version 1623120 (0.0005) [2023-12-27 16:54:05,650][20347] Updated weights for policy 0, policy_version 1624600 (0.0008) [2023-12-27 16:54:05,713][20347] Updated weights for policy 0, policy_version 1624610 (0.0008) [2023-12-27 16:54:05,736][20381] Updated weights for policy 1, policy_version 1623130 (0.0009) [2023-12-27 16:54:05,761][20347] Updated weights for policy 0, policy_version 1624620 (0.0006) [2023-12-27 16:54:05,785][20381] Updated weights for policy 1, policy_version 1623140 (0.0005) [2023-12-27 16:54:05,832][20381] Updated weights for policy 1, policy_version 1623150 (0.0007) [2023-12-27 16:54:06,456][20347] Updated weights for policy 0, policy_version 1624630 (0.0008) [2023-12-27 16:54:06,499][20381] Updated weights for policy 1, policy_version 1623160 (0.0007) [2023-12-27 16:54:06,506][20292] KL-divergence is very high: 155.3582 [2023-12-27 16:54:06,517][20347] Updated weights for policy 0, policy_version 1624640 (0.0010) [2023-12-27 16:54:06,555][20292] KL-divergence is very high: 227.3018 [2023-12-27 16:54:06,562][20381] Updated weights for policy 1, policy_version 1623170 (0.0006) [2023-12-27 16:54:06,571][20347] Updated weights for policy 0, policy_version 1624650 (0.0008) [2023-12-27 16:54:06,606][20292] KL-divergence is very high: 171.8916 [2023-12-27 16:54:06,627][20381] Updated weights for policy 1, policy_version 1623180 (0.0007) [2023-12-27 16:54:07,246][20347] Updated weights for policy 0, policy_version 1624660 (0.0008) [2023-12-27 16:54:07,294][20347] Updated weights for policy 0, policy_version 1624670 (0.0010) [2023-12-27 16:54:07,327][20381] Updated weights for policy 1, policy_version 1623190 (0.0009) [2023-12-27 16:54:07,351][20347] Updated weights for policy 0, policy_version 1624680 (0.0010) [2023-12-27 16:54:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 831569920. Throughput: 0: 10139.1, 1: 10414.5. Samples: 831546108. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:54:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.470')] [2023-12-27 16:54:07,386][20381] Updated weights for policy 1, policy_version 1623200 (0.0011) [2023-12-27 16:54:07,394][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001624688_415997952.pth... [2023-12-27 16:54:07,398][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001623504_415694848.pth [2023-12-27 16:54:07,444][20381] Updated weights for policy 1, policy_version 1623210 (0.0010) [2023-12-27 16:54:07,478][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001623216_415588352.pth... [2023-12-27 16:54:07,482][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001622000_415277056.pth [2023-12-27 16:54:08,035][20347] Updated weights for policy 0, policy_version 1624690 (0.0010) [2023-12-27 16:54:08,090][20347] Updated weights for policy 0, policy_version 1624700 (0.0010) [2023-12-27 16:54:08,139][20381] Updated weights for policy 1, policy_version 1623220 (0.0011) [2023-12-27 16:54:08,142][20347] Updated weights for policy 0, policy_version 1624710 (0.0010) [2023-12-27 16:54:08,189][20381] Updated weights for policy 1, policy_version 1623230 (0.0010) [2023-12-27 16:54:08,193][20347] Updated weights for policy 0, policy_version 1624720 (0.0009) [2023-12-27 16:54:08,239][20381] Updated weights for policy 1, policy_version 1623241 (0.0009) [2023-12-27 16:54:08,938][20347] Updated weights for policy 0, policy_version 1624730 (0.0007) [2023-12-27 16:54:08,974][20381] Updated weights for policy 1, policy_version 1623251 (0.0009) [2023-12-27 16:54:08,991][20347] Updated weights for policy 0, policy_version 1624740 (0.0007) [2023-12-27 16:54:09,023][20381] Updated weights for policy 1, policy_version 1623261 (0.0010) [2023-12-27 16:54:09,050][20347] Updated weights for policy 0, policy_version 1624750 (0.0006) [2023-12-27 16:54:09,078][20381] Updated weights for policy 1, policy_version 1623271 (0.0010) [2023-12-27 16:54:09,784][20347] Updated weights for policy 0, policy_version 1624760 (0.0007) [2023-12-27 16:54:09,814][20381] Updated weights for policy 1, policy_version 1623281 (0.0010) [2023-12-27 16:54:09,846][20347] Updated weights for policy 0, policy_version 1624770 (0.0007) [2023-12-27 16:54:09,878][20381] Updated weights for policy 1, policy_version 1623291 (0.0008) [2023-12-27 16:54:09,907][20347] Updated weights for policy 0, policy_version 1624780 (0.0007) [2023-12-27 16:54:09,937][20381] Updated weights for policy 1, policy_version 1623301 (0.0006) [2023-12-27 16:54:09,994][20381] Updated weights for policy 1, policy_version 1623311 (0.0009) [2023-12-27 16:54:10,587][20347] Updated weights for policy 0, policy_version 1624790 (0.0007) [2023-12-27 16:54:10,639][20347] Updated weights for policy 0, policy_version 1624800 (0.0008) [2023-12-27 16:54:10,689][20347] Updated weights for policy 0, policy_version 1624810 (0.0007) [2023-12-27 16:54:10,715][20381] Updated weights for policy 1, policy_version 1623321 (0.0007) [2023-12-27 16:54:10,775][20381] Updated weights for policy 1, policy_version 1623331 (0.0008) [2023-12-27 16:54:10,832][20381] Updated weights for policy 1, policy_version 1623341 (0.0009) [2023-12-27 16:54:11,425][20347] Updated weights for policy 0, policy_version 1624820 (0.0007) [2023-12-27 16:54:11,489][20347] Updated weights for policy 0, policy_version 1624830 (0.0006) [2023-12-27 16:54:11,544][20381] Updated weights for policy 1, policy_version 1623351 (0.0008) [2023-12-27 16:54:11,551][20347] Updated weights for policy 0, policy_version 1624840 (0.0007) [2023-12-27 16:54:11,603][20381] Updated weights for policy 1, policy_version 1623361 (0.0006) [2023-12-27 16:54:11,669][20381] Updated weights for policy 1, policy_version 1623371 (0.0007) [2023-12-27 16:54:12,250][20347] Updated weights for policy 0, policy_version 1624850 (0.0008) [2023-12-27 16:54:12,314][20347] Updated weights for policy 0, policy_version 1624860 (0.0011) [2023-12-27 16:54:12,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 831668224. Throughput: 0: 10158.9, 1: 10369.5. Samples: 831665944. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:54:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.450')] [2023-12-27 16:54:12,378][20347] Updated weights for policy 0, policy_version 1624870 (0.0011) [2023-12-27 16:54:12,388][20381] Updated weights for policy 1, policy_version 1623381 (0.0007) [2023-12-27 16:54:12,432][20347] Updated weights for policy 0, policy_version 1624880 (0.0009) [2023-12-27 16:54:12,447][20381] Updated weights for policy 1, policy_version 1623391 (0.0007) [2023-12-27 16:54:12,505][20381] Updated weights for policy 1, policy_version 1623401 (0.0008) [2023-12-27 16:54:13,111][20347] Updated weights for policy 0, policy_version 1624890 (0.0010) [2023-12-27 16:54:13,159][20347] Updated weights for policy 0, policy_version 1624900 (0.0010) [2023-12-27 16:54:13,223][20347] Updated weights for policy 0, policy_version 1624910 (0.0006) [2023-12-27 16:54:13,238][20381] Updated weights for policy 1, policy_version 1623411 (0.0008) [2023-12-27 16:54:13,281][20292] KL-divergence is very high: 658.2127 [2023-12-27 16:54:13,289][20292] KL-divergence is very high: 438.8553 [2023-12-27 16:54:13,302][20381] Updated weights for policy 1, policy_version 1623421 (0.0008) [2023-12-27 16:54:13,328][20292] KL-divergence is very high: 786.5106 [2023-12-27 16:54:13,334][20292] KL-divergence is very high: 467.8422 [2023-12-27 16:54:13,356][20381] Updated weights for policy 1, policy_version 1623431 (0.0009) [2023-12-27 16:54:13,374][20292] KL-divergence is very high: 484.5235 [2023-12-27 16:54:13,380][20292] KL-divergence is very high: 218.1690 [2023-12-27 16:54:13,402][20292] KL-divergence is very high: 115.8516 [2023-12-27 16:54:13,768][20347] Updated weights for policy 0, policy_version 1624920 (0.0005) [2023-12-27 16:54:13,819][20347] Updated weights for policy 0, policy_version 1624930 (0.0009) [2023-12-27 16:54:13,866][20347] Updated weights for policy 0, policy_version 1624940 (0.0010) [2023-12-27 16:54:14,080][20381] Updated weights for policy 1, policy_version 1623441 (0.0008) [2023-12-27 16:54:14,135][20381] Updated weights for policy 1, policy_version 1623451 (0.0007) [2023-12-27 16:54:14,192][20381] Updated weights for policy 1, policy_version 1623461 (0.0007) [2023-12-27 16:54:14,254][20381] Updated weights for policy 1, policy_version 1623471 (0.0007) [2023-12-27 16:54:14,550][20347] Updated weights for policy 0, policy_version 1624950 (0.0007) [2023-12-27 16:54:14,609][20347] Updated weights for policy 0, policy_version 1624960 (0.0007) [2023-12-27 16:54:14,669][20347] Updated weights for policy 0, policy_version 1624970 (0.0006) [2023-12-27 16:54:14,992][20381] Updated weights for policy 1, policy_version 1623481 (0.0008) [2023-12-27 16:54:15,064][20381] Updated weights for policy 1, policy_version 1623491 (0.0008) [2023-12-27 16:54:15,117][20381] Updated weights for policy 1, policy_version 1623501 (0.0008) [2023-12-27 16:54:15,309][20347] Updated weights for policy 0, policy_version 1624980 (0.0005) [2023-12-27 16:54:15,368][20347] Updated weights for policy 0, policy_version 1624990 (0.0007) [2023-12-27 16:54:15,427][20347] Updated weights for policy 0, policy_version 1625000 (0.0007) [2023-12-27 16:54:15,809][20381] Updated weights for policy 1, policy_version 1623511 (0.0008) [2023-12-27 16:54:15,853][20381] Updated weights for policy 1, policy_version 1623521 (0.0007) [2023-12-27 16:54:15,902][20381] Updated weights for policy 1, policy_version 1623531 (0.0007) [2023-12-27 16:54:16,101][20347] Updated weights for policy 0, policy_version 1625010 (0.0008) [2023-12-27 16:54:16,158][20347] Updated weights for policy 0, policy_version 1625020 (0.0005) [2023-12-27 16:54:16,207][20347] Updated weights for policy 0, policy_version 1625030 (0.0005) [2023-12-27 16:54:16,258][20347] Updated weights for policy 0, policy_version 1625040 (0.0005) [2023-12-27 16:54:16,639][20381] Updated weights for policy 1, policy_version 1623541 (0.0009) [2023-12-27 16:54:16,710][20381] Updated weights for policy 1, policy_version 1623551 (0.0011) [2023-12-27 16:54:16,767][20381] Updated weights for policy 1, policy_version 1623561 (0.0011) [2023-12-27 16:54:16,778][20347] Updated weights for policy 0, policy_version 1625050 (0.0006) [2023-12-27 16:54:16,836][20347] Updated weights for policy 0, policy_version 1625060 (0.0007) [2023-12-27 16:54:16,898][20347] Updated weights for policy 0, policy_version 1625070 (0.0011) [2023-12-27 16:54:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 831774720. Throughput: 0: 10172.2, 1: 10181.0. Samples: 831787792. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:54:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.110')] [2023-12-27 16:54:17,448][20381] Updated weights for policy 1, policy_version 1623571 (0.0011) [2023-12-27 16:54:17,513][20381] Updated weights for policy 1, policy_version 1623581 (0.0011) [2023-12-27 16:54:17,523][20347] Updated weights for policy 0, policy_version 1625080 (0.0007) [2023-12-27 16:54:17,575][20381] Updated weights for policy 1, policy_version 1623591 (0.0011) [2023-12-27 16:54:17,581][20347] Updated weights for policy 0, policy_version 1625090 (0.0008) [2023-12-27 16:54:17,639][20347] Updated weights for policy 0, policy_version 1625100 (0.0005) [2023-12-27 16:54:18,231][20381] Updated weights for policy 1, policy_version 1623601 (0.0011) [2023-12-27 16:54:18,237][20347] Updated weights for policy 0, policy_version 1625110 (0.0009) [2023-12-27 16:54:18,283][20347] Updated weights for policy 0, policy_version 1625120 (0.0010) [2023-12-27 16:54:18,283][20381] Updated weights for policy 1, policy_version 1623611 (0.0010) [2023-12-27 16:54:18,333][20381] Updated weights for policy 1, policy_version 1623621 (0.0010) [2023-12-27 16:54:18,351][20347] Updated weights for policy 0, policy_version 1625130 (0.0006) [2023-12-27 16:54:18,398][20381] Updated weights for policy 1, policy_version 1623631 (0.0010) [2023-12-27 16:54:19,081][20347] Updated weights for policy 0, policy_version 1625140 (0.0007) [2023-12-27 16:54:19,086][20381] Updated weights for policy 1, policy_version 1623641 (0.0011) [2023-12-27 16:54:19,131][20381] Updated weights for policy 1, policy_version 1623651 (0.0010) [2023-12-27 16:54:19,132][20347] Updated weights for policy 0, policy_version 1625150 (0.0005) [2023-12-27 16:54:19,176][20381] Updated weights for policy 1, policy_version 1623661 (0.0010) [2023-12-27 16:54:19,189][20347] Updated weights for policy 0, policy_version 1625160 (0.0008) [2023-12-27 16:54:19,822][20347] Updated weights for policy 0, policy_version 1625170 (0.0008) [2023-12-27 16:54:19,890][20347] Updated weights for policy 0, policy_version 1625180 (0.0007) [2023-12-27 16:54:19,949][20381] Updated weights for policy 1, policy_version 1623671 (0.0010) [2023-12-27 16:54:19,963][20347] Updated weights for policy 0, policy_version 1625190 (0.0008) [2023-12-27 16:54:20,008][20381] Updated weights for policy 1, policy_version 1623681 (0.0010) [2023-12-27 16:54:20,024][20347] Updated weights for policy 0, policy_version 1625200 (0.0008) [2023-12-27 16:54:20,065][20381] Updated weights for policy 1, policy_version 1623691 (0.0011) [2023-12-27 16:54:20,751][20347] Updated weights for policy 0, policy_version 1625210 (0.0008) [2023-12-27 16:54:20,761][20381] Updated weights for policy 1, policy_version 1623701 (0.0009) [2023-12-27 16:54:20,806][20347] Updated weights for policy 0, policy_version 1625220 (0.0006) [2023-12-27 16:54:20,816][20381] Updated weights for policy 1, policy_version 1623711 (0.0010) [2023-12-27 16:54:20,860][20347] Updated weights for policy 0, policy_version 1625230 (0.0005) [2023-12-27 16:54:20,869][20381] Updated weights for policy 1, policy_version 1623721 (0.0011) [2023-12-27 16:54:21,598][20347] Updated weights for policy 0, policy_version 1625240 (0.0007) [2023-12-27 16:54:21,668][20347] Updated weights for policy 0, policy_version 1625250 (0.0007) [2023-12-27 16:54:21,681][20381] Updated weights for policy 1, policy_version 1623731 (0.0010) [2023-12-27 16:54:21,726][20347] Updated weights for policy 0, policy_version 1625260 (0.0007) [2023-12-27 16:54:21,752][20381] Updated weights for policy 1, policy_version 1623741 (0.0008) [2023-12-27 16:54:21,815][20381] Updated weights for policy 1, policy_version 1623751 (0.0008) [2023-12-27 16:54:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 831873024. Throughput: 0: 10183.6, 1: 10141.4. Samples: 831848392. Policy #0 lag: (min: 18.0, avg: 30.5, max: 50.0) [2023-12-27 16:54:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.230')] [2023-12-27 16:54:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001623760_415727616.pth... [2023-12-27 16:54:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001625264_416145408.pth... [2023-12-27 16:54:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001622608_415432704.pth [2023-12-27 16:54:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001624080_415842304.pth [2023-12-27 16:54:22,442][20347] Updated weights for policy 0, policy_version 1625270 (0.0008) [2023-12-27 16:54:22,507][20347] Updated weights for policy 0, policy_version 1625280 (0.0009) [2023-12-27 16:54:22,535][20381] Updated weights for policy 1, policy_version 1623761 (0.0008) [2023-12-27 16:54:22,564][20347] Updated weights for policy 0, policy_version 1625290 (0.0007) [2023-12-27 16:54:22,594][20381] Updated weights for policy 1, policy_version 1623771 (0.0006) [2023-12-27 16:54:22,647][20381] Updated weights for policy 1, policy_version 1623781 (0.0006) [2023-12-27 16:54:22,701][20381] Updated weights for policy 1, policy_version 1623791 (0.0008) [2023-12-27 16:54:23,222][20347] Updated weights for policy 0, policy_version 1625300 (0.0008) [2023-12-27 16:54:23,272][20347] Updated weights for policy 0, policy_version 1625310 (0.0010) [2023-12-27 16:54:23,318][20347] Updated weights for policy 0, policy_version 1625320 (0.0007) [2023-12-27 16:54:23,352][20381] Updated weights for policy 1, policy_version 1623801 (0.0007) [2023-12-27 16:54:23,412][20381] Updated weights for policy 1, policy_version 1623811 (0.0007) [2023-12-27 16:54:23,472][20381] Updated weights for policy 1, policy_version 1623821 (0.0005) [2023-12-27 16:54:23,967][20347] Updated weights for policy 0, policy_version 1625330 (0.0006) [2023-12-27 16:54:24,025][20347] Updated weights for policy 0, policy_version 1625340 (0.0005) [2023-12-27 16:54:24,075][20381] Updated weights for policy 1, policy_version 1623831 (0.0009) [2023-12-27 16:54:24,078][20347] Updated weights for policy 0, policy_version 1625350 (0.0005) [2023-12-27 16:54:24,130][20381] Updated weights for policy 1, policy_version 1623841 (0.0011) [2023-12-27 16:54:24,133][20347] Updated weights for policy 0, policy_version 1625360 (0.0005) [2023-12-27 16:54:24,185][20381] Updated weights for policy 1, policy_version 1623851 (0.0011) [2023-12-27 16:54:24,728][20347] Updated weights for policy 0, policy_version 1625370 (0.0006) [2023-12-27 16:54:24,784][20347] Updated weights for policy 0, policy_version 1625380 (0.0007) [2023-12-27 16:54:24,833][20347] Updated weights for policy 0, policy_version 1625390 (0.0005) [2023-12-27 16:54:24,883][20381] Updated weights for policy 1, policy_version 1623861 (0.0011) [2023-12-27 16:54:24,948][20381] Updated weights for policy 1, policy_version 1623871 (0.0011) [2023-12-27 16:54:25,008][20381] Updated weights for policy 1, policy_version 1623881 (0.0010) [2023-12-27 16:54:25,445][20347] Updated weights for policy 0, policy_version 1625400 (0.0007) [2023-12-27 16:54:25,501][20347] Updated weights for policy 0, policy_version 1625410 (0.0006) [2023-12-27 16:54:25,556][20347] Updated weights for policy 0, policy_version 1625420 (0.0005) [2023-12-27 16:54:25,664][20381] Updated weights for policy 1, policy_version 1623891 (0.0006) [2023-12-27 16:54:25,718][20381] Updated weights for policy 1, policy_version 1623901 (0.0008) [2023-12-27 16:54:25,770][20381] Updated weights for policy 1, policy_version 1623911 (0.0008) [2023-12-27 16:54:26,239][20347] Updated weights for policy 0, policy_version 1625430 (0.0008) [2023-12-27 16:54:26,300][20347] Updated weights for policy 0, policy_version 1625440 (0.0007) [2023-12-27 16:54:26,352][20347] Updated weights for policy 0, policy_version 1625450 (0.0007) [2023-12-27 16:54:26,366][20381] Updated weights for policy 1, policy_version 1623921 (0.0007) [2023-12-27 16:54:26,417][20381] Updated weights for policy 1, policy_version 1623931 (0.0007) [2023-12-27 16:54:26,466][20381] Updated weights for policy 1, policy_version 1623941 (0.0008) [2023-12-27 16:54:26,515][20381] Updated weights for policy 1, policy_version 1623951 (0.0007) [2023-12-27 16:54:26,972][20347] Updated weights for policy 0, policy_version 1625460 (0.0006) [2023-12-27 16:54:27,034][20347] Updated weights for policy 0, policy_version 1625470 (0.0005) [2023-12-27 16:54:27,084][20347] Updated weights for policy 0, policy_version 1625480 (0.0005) [2023-12-27 16:54:27,195][20381] Updated weights for policy 1, policy_version 1623961 (0.0007) [2023-12-27 16:54:27,243][20381] Updated weights for policy 1, policy_version 1623971 (0.0009) [2023-12-27 16:54:27,296][20381] Updated weights for policy 1, policy_version 1623981 (0.0007) [2023-12-27 16:54:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 831987712. Throughput: 0: 10175.2, 1: 10108.2. Samples: 831972248. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.370')] [2023-12-27 16:54:27,652][20347] Updated weights for policy 0, policy_version 1625490 (0.0005) [2023-12-27 16:54:27,702][20347] Updated weights for policy 0, policy_version 1625500 (0.0010) [2023-12-27 16:54:27,757][20347] Updated weights for policy 0, policy_version 1625510 (0.0010) [2023-12-27 16:54:27,802][20347] Updated weights for policy 0, policy_version 1625520 (0.0010) [2023-12-27 16:54:27,968][20381] Updated weights for policy 1, policy_version 1623991 (0.0007) [2023-12-27 16:54:28,014][20381] Updated weights for policy 1, policy_version 1624001 (0.0008) [2023-12-27 16:54:28,064][20381] Updated weights for policy 1, policy_version 1624011 (0.0007) [2023-12-27 16:54:28,427][20347] Updated weights for policy 0, policy_version 1625530 (0.0007) [2023-12-27 16:54:28,482][20347] Updated weights for policy 0, policy_version 1625540 (0.0008) [2023-12-27 16:54:28,530][20347] Updated weights for policy 0, policy_version 1625550 (0.0007) [2023-12-27 16:54:28,744][20381] Updated weights for policy 1, policy_version 1624021 (0.0008) [2023-12-27 16:54:28,808][20381] Updated weights for policy 1, policy_version 1624031 (0.0008) [2023-12-27 16:54:28,862][20381] Updated weights for policy 1, policy_version 1624041 (0.0008) [2023-12-27 16:54:29,177][20347] Updated weights for policy 0, policy_version 1625560 (0.0007) [2023-12-27 16:54:29,250][20347] Updated weights for policy 0, policy_version 1625570 (0.0009) [2023-12-27 16:54:29,312][20347] Updated weights for policy 0, policy_version 1625580 (0.0005) [2023-12-27 16:54:29,496][20381] Updated weights for policy 1, policy_version 1624051 (0.0006) [2023-12-27 16:54:29,558][20381] Updated weights for policy 1, policy_version 1624061 (0.0008) [2023-12-27 16:54:29,622][20381] Updated weights for policy 1, policy_version 1624071 (0.0008) [2023-12-27 16:54:30,004][20347] Updated weights for policy 0, policy_version 1625590 (0.0007) [2023-12-27 16:54:30,051][20347] Updated weights for policy 0, policy_version 1625600 (0.0008) [2023-12-27 16:54:30,106][20347] Updated weights for policy 0, policy_version 1625610 (0.0007) [2023-12-27 16:54:30,363][20381] Updated weights for policy 1, policy_version 1624081 (0.0007) [2023-12-27 16:54:30,419][20381] Updated weights for policy 1, policy_version 1624091 (0.0008) [2023-12-27 16:54:30,469][20381] Updated weights for policy 1, policy_version 1624101 (0.0008) [2023-12-27 16:54:30,523][20381] Updated weights for policy 1, policy_version 1624111 (0.0008) [2023-12-27 16:54:30,812][20347] Updated weights for policy 0, policy_version 1625620 (0.0007) [2023-12-27 16:54:30,860][20347] Updated weights for policy 0, policy_version 1625630 (0.0005) [2023-12-27 16:54:30,906][20347] Updated weights for policy 0, policy_version 1625640 (0.0005) [2023-12-27 16:54:31,170][20381] Updated weights for policy 1, policy_version 1624121 (0.0008) [2023-12-27 16:54:31,225][20381] Updated weights for policy 1, policy_version 1624131 (0.0007) [2023-12-27 16:54:31,286][20381] Updated weights for policy 1, policy_version 1624141 (0.0010) [2023-12-27 16:54:31,585][20347] Updated weights for policy 0, policy_version 1625650 (0.0008) [2023-12-27 16:54:31,655][20347] Updated weights for policy 0, policy_version 1625660 (0.0008) [2023-12-27 16:54:31,718][20347] Updated weights for policy 0, policy_version 1625670 (0.0008) [2023-12-27 16:54:31,782][20347] Updated weights for policy 0, policy_version 1625680 (0.0008) [2023-12-27 16:54:32,012][20381] Updated weights for policy 1, policy_version 1624151 (0.0009) [2023-12-27 16:54:32,063][20381] Updated weights for policy 1, policy_version 1624161 (0.0007) [2023-12-27 16:54:32,117][20381] Updated weights for policy 1, policy_version 1624171 (0.0007) [2023-12-27 16:54:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 832086016. Throughput: 0: 10285.1, 1: 10103.0. Samples: 832097200. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.340')] [2023-12-27 16:54:32,453][20347] Updated weights for policy 0, policy_version 1625690 (0.0007) [2023-12-27 16:54:32,508][20347] Updated weights for policy 0, policy_version 1625700 (0.0007) [2023-12-27 16:54:32,557][20347] Updated weights for policy 0, policy_version 1625710 (0.0007) [2023-12-27 16:54:32,781][20381] Updated weights for policy 1, policy_version 1624181 (0.0008) [2023-12-27 16:54:32,834][20381] Updated weights for policy 1, policy_version 1624191 (0.0009) [2023-12-27 16:54:32,890][20381] Updated weights for policy 1, policy_version 1624201 (0.0005) [2023-12-27 16:54:33,277][20347] Updated weights for policy 0, policy_version 1625720 (0.0007) [2023-12-27 16:54:33,330][20347] Updated weights for policy 0, policy_version 1625730 (0.0007) [2023-12-27 16:54:33,381][20347] Updated weights for policy 0, policy_version 1625740 (0.0008) [2023-12-27 16:54:33,413][20381] Updated weights for policy 1, policy_version 1624211 (0.0005) [2023-12-27 16:54:33,465][20381] Updated weights for policy 1, policy_version 1624221 (0.0006) [2023-12-27 16:54:33,516][20381] Updated weights for policy 1, policy_version 1624231 (0.0005) [2023-12-27 16:54:34,093][20381] Updated weights for policy 1, policy_version 1624241 (0.0008) [2023-12-27 16:54:34,124][20347] Updated weights for policy 0, policy_version 1625750 (0.0007) [2023-12-27 16:54:34,157][20381] Updated weights for policy 1, policy_version 1624251 (0.0010) [2023-12-27 16:54:34,186][20347] Updated weights for policy 0, policy_version 1625760 (0.0006) [2023-12-27 16:54:34,215][20381] Updated weights for policy 1, policy_version 1624261 (0.0010) [2023-12-27 16:54:34,246][20347] Updated weights for policy 0, policy_version 1625770 (0.0006) [2023-12-27 16:54:34,277][20381] Updated weights for policy 1, policy_version 1624271 (0.0009) [2023-12-27 16:54:34,845][20347] Updated weights for policy 0, policy_version 1625780 (0.0006) [2023-12-27 16:54:34,918][20347] Updated weights for policy 0, policy_version 1625790 (0.0006) [2023-12-27 16:54:34,981][20381] Updated weights for policy 1, policy_version 1624281 (0.0010) [2023-12-27 16:54:34,982][20347] Updated weights for policy 0, policy_version 1625800 (0.0005) [2023-12-27 16:54:35,034][20381] Updated weights for policy 1, policy_version 1624291 (0.0009) [2023-12-27 16:54:35,091][20381] Updated weights for policy 1, policy_version 1624301 (0.0007) [2023-12-27 16:54:35,525][20347] Updated weights for policy 0, policy_version 1625810 (0.0006) [2023-12-27 16:54:35,583][20347] Updated weights for policy 0, policy_version 1625820 (0.0007) [2023-12-27 16:54:35,637][20347] Updated weights for policy 0, policy_version 1625830 (0.0007) [2023-12-27 16:54:35,687][20347] Updated weights for policy 0, policy_version 1625840 (0.0005) [2023-12-27 16:54:35,747][20381] Updated weights for policy 1, policy_version 1624311 (0.0011) [2023-12-27 16:54:35,802][20381] Updated weights for policy 1, policy_version 1624321 (0.0010) [2023-12-27 16:54:35,864][20381] Updated weights for policy 1, policy_version 1624331 (0.0011) [2023-12-27 16:54:36,319][20347] Updated weights for policy 0, policy_version 1625850 (0.0011) [2023-12-27 16:54:36,382][20347] Updated weights for policy 0, policy_version 1625860 (0.0009) [2023-12-27 16:54:36,442][20347] Updated weights for policy 0, policy_version 1625870 (0.0006) [2023-12-27 16:54:36,593][20381] Updated weights for policy 1, policy_version 1624341 (0.0011) [2023-12-27 16:54:36,657][20381] Updated weights for policy 1, policy_version 1624351 (0.0011) [2023-12-27 16:54:36,726][20381] Updated weights for policy 1, policy_version 1624361 (0.0010) [2023-12-27 16:54:37,046][20347] Updated weights for policy 0, policy_version 1625881 (0.0007) [2023-12-27 16:54:37,104][20347] Updated weights for policy 0, policy_version 1625891 (0.0008) [2023-12-27 16:54:37,160][20347] Updated weights for policy 0, policy_version 1625901 (0.0008) [2023-12-27 16:54:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 832192512. Throughput: 0: 10309.0, 1: 10092.3. Samples: 832160784. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.370')] [2023-12-27 16:54:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001625904_416309248.pth... [2023-12-27 16:54:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001624688_415997952.pth [2023-12-27 16:54:37,382][20381] Updated weights for policy 1, policy_version 1624371 (0.0007) [2023-12-27 16:54:37,431][20381] Updated weights for policy 1, policy_version 1624381 (0.0005) [2023-12-27 16:54:37,493][20381] Updated weights for policy 1, policy_version 1624391 (0.0006) [2023-12-27 16:54:37,539][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001624400_415891456.pth... [2023-12-27 16:54:37,542][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001623216_415588352.pth [2023-12-27 16:54:37,836][20347] Updated weights for policy 0, policy_version 1625911 (0.0006) [2023-12-27 16:54:37,883][20347] Updated weights for policy 0, policy_version 1625921 (0.0010) [2023-12-27 16:54:37,937][20347] Updated weights for policy 0, policy_version 1625931 (0.0006) [2023-12-27 16:54:38,062][20381] Updated weights for policy 1, policy_version 1624401 (0.0005) [2023-12-27 16:54:38,124][20381] Updated weights for policy 1, policy_version 1624411 (0.0005) [2023-12-27 16:54:38,174][20381] Updated weights for policy 1, policy_version 1624421 (0.0007) [2023-12-27 16:54:38,226][20381] Updated weights for policy 1, policy_version 1624431 (0.0007) [2023-12-27 16:54:38,598][20347] Updated weights for policy 0, policy_version 1625941 (0.0007) [2023-12-27 16:54:38,655][20347] Updated weights for policy 0, policy_version 1625951 (0.0007) [2023-12-27 16:54:38,717][20347] Updated weights for policy 0, policy_version 1625961 (0.0007) [2023-12-27 16:54:38,911][20381] Updated weights for policy 1, policy_version 1624441 (0.0007) [2023-12-27 16:54:38,968][20381] Updated weights for policy 1, policy_version 1624451 (0.0011) [2023-12-27 16:54:39,026][20381] Updated weights for policy 1, policy_version 1624461 (0.0010) [2023-12-27 16:54:39,491][20347] Updated weights for policy 0, policy_version 1625971 (0.0007) [2023-12-27 16:54:39,554][20347] Updated weights for policy 0, policy_version 1625981 (0.0005) [2023-12-27 16:54:39,614][20347] Updated weights for policy 0, policy_version 1625991 (0.0006) [2023-12-27 16:54:39,802][20381] Updated weights for policy 1, policy_version 1624471 (0.0009) [2023-12-27 16:54:39,869][20381] Updated weights for policy 1, policy_version 1624481 (0.0008) [2023-12-27 16:54:39,936][20381] Updated weights for policy 1, policy_version 1624491 (0.0007) [2023-12-27 16:54:40,279][20347] Updated weights for policy 0, policy_version 1626001 (0.0006) [2023-12-27 16:54:40,345][20347] Updated weights for policy 0, policy_version 1626011 (0.0011) [2023-12-27 16:54:40,407][20347] Updated weights for policy 0, policy_version 1626021 (0.0010) [2023-12-27 16:54:40,470][20347] Updated weights for policy 0, policy_version 1626031 (0.0010) [2023-12-27 16:54:40,553][20381] Updated weights for policy 1, policy_version 1624501 (0.0007) [2023-12-27 16:54:40,611][20381] Updated weights for policy 1, policy_version 1624511 (0.0007) [2023-12-27 16:54:40,668][20381] Updated weights for policy 1, policy_version 1624521 (0.0005) [2023-12-27 16:54:41,191][20347] Updated weights for policy 0, policy_version 1626041 (0.0012) [2023-12-27 16:54:41,255][20347] Updated weights for policy 0, policy_version 1626051 (0.0008) [2023-12-27 16:54:41,301][20381] Updated weights for policy 1, policy_version 1624531 (0.0006) [2023-12-27 16:54:41,313][20347] Updated weights for policy 0, policy_version 1626061 (0.0009) [2023-12-27 16:54:41,372][20381] Updated weights for policy 1, policy_version 1624541 (0.0008) [2023-12-27 16:54:41,446][20381] Updated weights for policy 1, policy_version 1624551 (0.0008) [2023-12-27 16:54:42,067][20347] Updated weights for policy 0, policy_version 1626071 (0.0009) [2023-12-27 16:54:42,123][20347] Updated weights for policy 0, policy_version 1626081 (0.0010) [2023-12-27 16:54:42,150][20381] Updated weights for policy 1, policy_version 1624561 (0.0007) [2023-12-27 16:54:42,176][20347] Updated weights for policy 0, policy_version 1626091 (0.0010) [2023-12-27 16:54:42,206][20381] Updated weights for policy 1, policy_version 1624571 (0.0007) [2023-12-27 16:54:42,269][20381] Updated weights for policy 1, policy_version 1624581 (0.0008) [2023-12-27 16:54:42,330][20381] Updated weights for policy 1, policy_version 1624591 (0.0006) [2023-12-27 16:54:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 832299008. Throughput: 0: 10345.2, 1: 10122.0. Samples: 832285668. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.650')] [2023-12-27 16:54:42,884][20347] Updated weights for policy 0, policy_version 1626101 (0.0011) [2023-12-27 16:54:42,942][20347] Updated weights for policy 0, policy_version 1626111 (0.0010) [2023-12-27 16:54:42,964][20381] Updated weights for policy 1, policy_version 1624601 (0.0006) [2023-12-27 16:54:43,008][20347] Updated weights for policy 0, policy_version 1626121 (0.0010) [2023-12-27 16:54:43,016][20381] Updated weights for policy 1, policy_version 1624611 (0.0005) [2023-12-27 16:54:43,077][20381] Updated weights for policy 1, policy_version 1624621 (0.0006) [2023-12-27 16:54:43,619][20347] Updated weights for policy 0, policy_version 1626131 (0.0008) [2023-12-27 16:54:43,669][20347] Updated weights for policy 0, policy_version 1626141 (0.0008) [2023-12-27 16:54:43,719][20347] Updated weights for policy 0, policy_version 1626151 (0.0007) [2023-12-27 16:54:43,746][20381] Updated weights for policy 1, policy_version 1624631 (0.0009) [2023-12-27 16:54:43,809][20381] Updated weights for policy 1, policy_version 1624641 (0.0007) [2023-12-27 16:54:43,856][20381] Updated weights for policy 1, policy_version 1624651 (0.0007) [2023-12-27 16:54:44,419][20347] Updated weights for policy 0, policy_version 1626161 (0.0010) [2023-12-27 16:54:44,477][20347] Updated weights for policy 0, policy_version 1626171 (0.0010) [2023-12-27 16:54:44,533][20347] Updated weights for policy 0, policy_version 1626181 (0.0010) [2023-12-27 16:54:44,550][20381] Updated weights for policy 1, policy_version 1624661 (0.0006) [2023-12-27 16:54:44,582][20347] Updated weights for policy 0, policy_version 1626191 (0.0010) [2023-12-27 16:54:44,595][20381] Updated weights for policy 1, policy_version 1624671 (0.0005) [2023-12-27 16:54:44,649][20381] Updated weights for policy 1, policy_version 1624681 (0.0007) [2023-12-27 16:54:45,276][20347] Updated weights for policy 0, policy_version 1626201 (0.0010) [2023-12-27 16:54:45,323][20347] Updated weights for policy 0, policy_version 1626211 (0.0010) [2023-12-27 16:54:45,370][20347] Updated weights for policy 0, policy_version 1626221 (0.0010) [2023-12-27 16:54:45,371][20381] Updated weights for policy 1, policy_version 1624691 (0.0007) [2023-12-27 16:54:45,418][20381] Updated weights for policy 1, policy_version 1624701 (0.0008) [2023-12-27 16:54:45,469][20381] Updated weights for policy 1, policy_version 1624711 (0.0007) [2023-12-27 16:54:46,089][20347] Updated weights for policy 0, policy_version 1626231 (0.0010) [2023-12-27 16:54:46,137][20347] Updated weights for policy 0, policy_version 1626241 (0.0010) [2023-12-27 16:54:46,183][20381] Updated weights for policy 1, policy_version 1624721 (0.0007) [2023-12-27 16:54:46,185][20347] Updated weights for policy 0, policy_version 1626251 (0.0010) [2023-12-27 16:54:46,235][20381] Updated weights for policy 1, policy_version 1624731 (0.0006) [2023-12-27 16:54:46,282][20381] Updated weights for policy 1, policy_version 1624741 (0.0007) [2023-12-27 16:54:46,330][20381] Updated weights for policy 1, policy_version 1624751 (0.0007) [2023-12-27 16:54:46,873][20347] Updated weights for policy 0, policy_version 1626261 (0.0010) [2023-12-27 16:54:46,925][20347] Updated weights for policy 0, policy_version 1626271 (0.0010) [2023-12-27 16:54:46,985][20347] Updated weights for policy 0, policy_version 1626281 (0.0010) [2023-12-27 16:54:47,021][20381] Updated weights for policy 1, policy_version 1624761 (0.0010) [2023-12-27 16:54:47,074][20381] Updated weights for policy 1, policy_version 1624771 (0.0010) [2023-12-27 16:54:47,125][20381] Updated weights for policy 1, policy_version 1624781 (0.0010) [2023-12-27 16:54:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 832397312. Throughput: 0: 10330.6, 1: 10135.5. Samples: 832405456. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.470')] [2023-12-27 16:54:47,652][20347] Updated weights for policy 0, policy_version 1626291 (0.0010) [2023-12-27 16:54:47,697][20347] Updated weights for policy 0, policy_version 1626301 (0.0010) [2023-12-27 16:54:47,745][20347] Updated weights for policy 0, policy_version 1626311 (0.0010) [2023-12-27 16:54:47,810][20381] Updated weights for policy 1, policy_version 1624791 (0.0007) [2023-12-27 16:54:47,859][20381] Updated weights for policy 1, policy_version 1624801 (0.0007) [2023-12-27 16:54:47,907][20381] Updated weights for policy 1, policy_version 1624811 (0.0007) [2023-12-27 16:54:48,452][20347] Updated weights for policy 0, policy_version 1626321 (0.0010) [2023-12-27 16:54:48,517][20347] Updated weights for policy 0, policy_version 1626331 (0.0011) [2023-12-27 16:54:48,582][20347] Updated weights for policy 0, policy_version 1626341 (0.0011) [2023-12-27 16:54:48,644][20347] Updated weights for policy 0, policy_version 1626351 (0.0010) [2023-12-27 16:54:48,659][20381] Updated weights for policy 1, policy_version 1624821 (0.0007) [2023-12-27 16:54:48,719][20381] Updated weights for policy 1, policy_version 1624831 (0.0005) [2023-12-27 16:54:48,782][20381] Updated weights for policy 1, policy_version 1624841 (0.0006) [2023-12-27 16:54:49,311][20347] Updated weights for policy 0, policy_version 1626361 (0.0009) [2023-12-27 16:54:49,379][20347] Updated weights for policy 0, policy_version 1626371 (0.0010) [2023-12-27 16:54:49,440][20381] Updated weights for policy 1, policy_version 1624851 (0.0008) [2023-12-27 16:54:49,444][20347] Updated weights for policy 0, policy_version 1626381 (0.0010) [2023-12-27 16:54:49,502][20381] Updated weights for policy 1, policy_version 1624861 (0.0008) [2023-12-27 16:54:49,556][20381] Updated weights for policy 1, policy_version 1624871 (0.0008) [2023-12-27 16:54:50,140][20347] Updated weights for policy 0, policy_version 1626391 (0.0011) [2023-12-27 16:54:50,199][20347] Updated weights for policy 0, policy_version 1626401 (0.0010) [2023-12-27 16:54:50,257][20347] Updated weights for policy 0, policy_version 1626411 (0.0011) [2023-12-27 16:54:50,305][20381] Updated weights for policy 1, policy_version 1624881 (0.0008) [2023-12-27 16:54:50,365][20381] Updated weights for policy 1, policy_version 1624891 (0.0007) [2023-12-27 16:54:50,425][20381] Updated weights for policy 1, policy_version 1624901 (0.0008) [2023-12-27 16:54:50,480][20381] Updated weights for policy 1, policy_version 1624911 (0.0008) [2023-12-27 16:54:50,965][20347] Updated weights for policy 0, policy_version 1626421 (0.0010) [2023-12-27 16:54:51,027][20347] Updated weights for policy 0, policy_version 1626431 (0.0011) [2023-12-27 16:54:51,089][20347] Updated weights for policy 0, policy_version 1626441 (0.0009) [2023-12-27 16:54:51,212][20381] Updated weights for policy 1, policy_version 1624921 (0.0005) [2023-12-27 16:54:51,269][20381] Updated weights for policy 1, policy_version 1624931 (0.0007) [2023-12-27 16:54:51,330][20381] Updated weights for policy 1, policy_version 1624941 (0.0009) [2023-12-27 16:54:51,763][20347] Updated weights for policy 0, policy_version 1626451 (0.0009) [2023-12-27 16:54:51,839][20347] Updated weights for policy 0, policy_version 1626461 (0.0009) [2023-12-27 16:54:51,894][20347] Updated weights for policy 0, policy_version 1626471 (0.0007) [2023-12-27 16:54:52,058][20381] Updated weights for policy 1, policy_version 1624951 (0.0009) [2023-12-27 16:54:52,115][20381] Updated weights for policy 1, policy_version 1624961 (0.0009) [2023-12-27 16:54:52,177][20381] Updated weights for policy 1, policy_version 1624971 (0.0011) [2023-12-27 16:54:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 832495616. Throughput: 0: 10314.4, 1: 10119.8. Samples: 832465648. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.040')] [2023-12-27 16:54:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001624976_416038912.pth... [2023-12-27 16:54:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001626480_416456704.pth... [2023-12-27 16:54:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001623760_415727616.pth [2023-12-27 16:54:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001625264_416145408.pth [2023-12-27 16:54:52,577][20347] Updated weights for policy 0, policy_version 1626481 (0.0007) [2023-12-27 16:54:52,630][20347] Updated weights for policy 0, policy_version 1626491 (0.0009) [2023-12-27 16:54:52,685][20347] Updated weights for policy 0, policy_version 1626501 (0.0007) [2023-12-27 16:54:52,742][20347] Updated weights for policy 0, policy_version 1626511 (0.0008) [2023-12-27 16:54:52,895][20381] Updated weights for policy 1, policy_version 1624981 (0.0011) [2023-12-27 16:54:52,947][20381] Updated weights for policy 1, policy_version 1624991 (0.0011) [2023-12-27 16:54:53,013][20381] Updated weights for policy 1, policy_version 1625001 (0.0011) [2023-12-27 16:54:53,363][20347] Updated weights for policy 0, policy_version 1626521 (0.0007) [2023-12-27 16:54:53,422][20347] Updated weights for policy 0, policy_version 1626531 (0.0007) [2023-12-27 16:54:53,486][20347] Updated weights for policy 0, policy_version 1626541 (0.0007) [2023-12-27 16:54:53,708][20381] Updated weights for policy 1, policy_version 1625011 (0.0011) [2023-12-27 16:54:53,754][20381] Updated weights for policy 1, policy_version 1625021 (0.0010) [2023-12-27 16:54:53,813][20381] Updated weights for policy 1, policy_version 1625031 (0.0011) [2023-12-27 16:54:54,156][20347] Updated weights for policy 0, policy_version 1626551 (0.0008) [2023-12-27 16:54:54,209][20347] Updated weights for policy 0, policy_version 1626561 (0.0007) [2023-12-27 16:54:54,269][20347] Updated weights for policy 0, policy_version 1626571 (0.0007) [2023-12-27 16:54:54,513][20381] Updated weights for policy 1, policy_version 1625041 (0.0011) [2023-12-27 16:54:54,575][20381] Updated weights for policy 1, policy_version 1625051 (0.0010) [2023-12-27 16:54:54,624][20381] Updated weights for policy 1, policy_version 1625061 (0.0010) [2023-12-27 16:54:54,672][20381] Updated weights for policy 1, policy_version 1625071 (0.0010) [2023-12-27 16:54:54,932][20347] Updated weights for policy 0, policy_version 1626581 (0.0007) [2023-12-27 16:54:54,992][20347] Updated weights for policy 0, policy_version 1626591 (0.0007) [2023-12-27 16:54:55,059][20347] Updated weights for policy 0, policy_version 1626601 (0.0007) [2023-12-27 16:54:55,295][20381] Updated weights for policy 1, policy_version 1625081 (0.0010) [2023-12-27 16:54:55,343][20381] Updated weights for policy 1, policy_version 1625091 (0.0010) [2023-12-27 16:54:55,392][20381] Updated weights for policy 1, policy_version 1625101 (0.0010) [2023-12-27 16:54:55,729][20347] Updated weights for policy 0, policy_version 1626611 (0.0008) [2023-12-27 16:54:55,780][20347] Updated weights for policy 0, policy_version 1626621 (0.0007) [2023-12-27 16:54:55,841][20347] Updated weights for policy 0, policy_version 1626631 (0.0006) [2023-12-27 16:54:56,101][20381] Updated weights for policy 1, policy_version 1625111 (0.0011) [2023-12-27 16:54:56,147][20381] Updated weights for policy 1, policy_version 1625121 (0.0011) [2023-12-27 16:54:56,196][20381] Updated weights for policy 1, policy_version 1625131 (0.0011) [2023-12-27 16:54:56,477][20347] Updated weights for policy 0, policy_version 1626641 (0.0006) [2023-12-27 16:54:56,531][20347] Updated weights for policy 0, policy_version 1626651 (0.0010) [2023-12-27 16:54:56,582][20347] Updated weights for policy 0, policy_version 1626661 (0.0010) [2023-12-27 16:54:56,624][20347] Updated weights for policy 0, policy_version 1626671 (0.0007) [2023-12-27 16:54:56,874][20381] Updated weights for policy 1, policy_version 1625141 (0.0010) [2023-12-27 16:54:56,925][20381] Updated weights for policy 1, policy_version 1625151 (0.0010) [2023-12-27 16:54:56,979][20381] Updated weights for policy 1, policy_version 1625161 (0.0010) [2023-12-27 16:54:57,219][20347] Updated weights for policy 0, policy_version 1626681 (0.0007) [2023-12-27 16:54:57,267][20347] Updated weights for policy 0, policy_version 1626691 (0.0010) [2023-12-27 16:54:57,322][20347] Updated weights for policy 0, policy_version 1626701 (0.0010) [2023-12-27 16:54:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 832602112. Throughput: 0: 10335.3, 1: 10111.3. Samples: 832586040. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:54:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.990')] [2023-12-27 16:54:57,675][20381] Updated weights for policy 1, policy_version 1625171 (0.0010) [2023-12-27 16:54:57,740][20381] Updated weights for policy 1, policy_version 1625181 (0.0011) [2023-12-27 16:54:57,801][20381] Updated weights for policy 1, policy_version 1625191 (0.0010) [2023-12-27 16:54:57,998][20347] Updated weights for policy 0, policy_version 1626711 (0.0010) [2023-12-27 16:54:58,049][20347] Updated weights for policy 0, policy_version 1626721 (0.0010) [2023-12-27 16:54:58,103][20347] Updated weights for policy 0, policy_version 1626731 (0.0010) [2023-12-27 16:54:58,493][20381] Updated weights for policy 1, policy_version 1625201 (0.0011) [2023-12-27 16:54:58,556][20381] Updated weights for policy 1, policy_version 1625211 (0.0011) [2023-12-27 16:54:58,617][20381] Updated weights for policy 1, policy_version 1625221 (0.0011) [2023-12-27 16:54:58,677][20381] Updated weights for policy 1, policy_version 1625231 (0.0010) [2023-12-27 16:54:58,933][20347] Updated weights for policy 0, policy_version 1626741 (0.0009) [2023-12-27 16:54:58,996][20347] Updated weights for policy 0, policy_version 1626751 (0.0009) [2023-12-27 16:54:59,057][20347] Updated weights for policy 0, policy_version 1626761 (0.0010) [2023-12-27 16:54:59,411][20381] Updated weights for policy 1, policy_version 1625241 (0.0007) [2023-12-27 16:54:59,469][20381] Updated weights for policy 1, policy_version 1625251 (0.0005) [2023-12-27 16:54:59,527][20381] Updated weights for policy 1, policy_version 1625261 (0.0005) [2023-12-27 16:54:59,785][20347] Updated weights for policy 0, policy_version 1626771 (0.0009) [2023-12-27 16:54:59,860][20347] Updated weights for policy 0, policy_version 1626781 (0.0007) [2023-12-27 16:54:59,925][20347] Updated weights for policy 0, policy_version 1626791 (0.0008) [2023-12-27 16:55:00,185][20381] Updated weights for policy 1, policy_version 1625271 (0.0008) [2023-12-27 16:55:00,242][20381] Updated weights for policy 1, policy_version 1625281 (0.0009) [2023-12-27 16:55:00,297][20381] Updated weights for policy 1, policy_version 1625291 (0.0011) [2023-12-27 16:55:00,602][20347] Updated weights for policy 0, policy_version 1626801 (0.0008) [2023-12-27 16:55:00,660][20347] Updated weights for policy 0, policy_version 1626811 (0.0007) [2023-12-27 16:55:00,710][20347] Updated weights for policy 0, policy_version 1626821 (0.0006) [2023-12-27 16:55:00,757][20347] Updated weights for policy 0, policy_version 1626831 (0.0007) [2023-12-27 16:55:00,997][20381] Updated weights for policy 1, policy_version 1625301 (0.0011) [2023-12-27 16:55:01,054][20381] Updated weights for policy 1, policy_version 1625311 (0.0011) [2023-12-27 16:55:01,121][20381] Updated weights for policy 1, policy_version 1625321 (0.0011) [2023-12-27 16:55:01,505][20347] Updated weights for policy 0, policy_version 1626841 (0.0007) [2023-12-27 16:55:01,562][20347] Updated weights for policy 0, policy_version 1626851 (0.0009) [2023-12-27 16:55:01,618][20347] Updated weights for policy 0, policy_version 1626861 (0.0008) [2023-12-27 16:55:01,810][20381] Updated weights for policy 1, policy_version 1625331 (0.0010) [2023-12-27 16:55:01,856][20381] Updated weights for policy 1, policy_version 1625341 (0.0008) [2023-12-27 16:55:01,913][20381] Updated weights for policy 1, policy_version 1625351 (0.0008) [2023-12-27 16:55:02,339][20347] Updated weights for policy 0, policy_version 1626871 (0.0007) [2023-12-27 16:55:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 832692224. Throughput: 0: 10248.3, 1: 10150.4. Samples: 832705732. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.950')] [2023-12-27 16:55:02,398][20347] Updated weights for policy 0, policy_version 1626881 (0.0006) [2023-12-27 16:55:02,447][20347] Updated weights for policy 0, policy_version 1626891 (0.0007) [2023-12-27 16:55:02,614][20381] Updated weights for policy 1, policy_version 1625361 (0.0008) [2023-12-27 16:55:02,674][20381] Updated weights for policy 1, policy_version 1625371 (0.0007) [2023-12-27 16:55:02,732][20381] Updated weights for policy 1, policy_version 1625381 (0.0007) [2023-12-27 16:55:02,788][20381] Updated weights for policy 1, policy_version 1625391 (0.0008) [2023-12-27 16:55:03,150][20347] Updated weights for policy 0, policy_version 1626901 (0.0009) [2023-12-27 16:55:03,204][20347] Updated weights for policy 0, policy_version 1626911 (0.0008) [2023-12-27 16:55:03,261][20347] Updated weights for policy 0, policy_version 1626921 (0.0005) [2023-12-27 16:55:03,397][20381] Updated weights for policy 1, policy_version 1625401 (0.0005) [2023-12-27 16:55:03,443][20381] Updated weights for policy 1, policy_version 1625411 (0.0005) [2023-12-27 16:55:03,488][20381] Updated weights for policy 1, policy_version 1625421 (0.0005) [2023-12-27 16:55:03,848][20347] Updated weights for policy 0, policy_version 1626931 (0.0007) [2023-12-27 16:55:03,907][20347] Updated weights for policy 0, policy_version 1626941 (0.0009) [2023-12-27 16:55:03,969][20347] Updated weights for policy 0, policy_version 1626951 (0.0008) [2023-12-27 16:55:04,125][20381] Updated weights for policy 1, policy_version 1625431 (0.0007) [2023-12-27 16:55:04,187][20381] Updated weights for policy 1, policy_version 1625441 (0.0007) [2023-12-27 16:55:04,247][20381] Updated weights for policy 1, policy_version 1625451 (0.0008) [2023-12-27 16:55:04,671][20347] Updated weights for policy 0, policy_version 1626961 (0.0008) [2023-12-27 16:55:04,729][20347] Updated weights for policy 0, policy_version 1626971 (0.0011) [2023-12-27 16:55:04,784][20347] Updated weights for policy 0, policy_version 1626981 (0.0010) [2023-12-27 16:55:04,838][20347] Updated weights for policy 0, policy_version 1626991 (0.0010) [2023-12-27 16:55:04,889][20381] Updated weights for policy 1, policy_version 1625461 (0.0007) [2023-12-27 16:55:04,948][20381] Updated weights for policy 1, policy_version 1625471 (0.0005) [2023-12-27 16:55:05,015][20381] Updated weights for policy 1, policy_version 1625481 (0.0005) [2023-12-27 16:55:05,497][20347] Updated weights for policy 0, policy_version 1627001 (0.0006) [2023-12-27 16:55:05,550][20347] Updated weights for policy 0, policy_version 1627011 (0.0008) [2023-12-27 16:55:05,560][20381] Updated weights for policy 1, policy_version 1625491 (0.0009) [2023-12-27 16:55:05,604][20347] Updated weights for policy 0, policy_version 1627021 (0.0010) [2023-12-27 16:55:05,609][20381] Updated weights for policy 1, policy_version 1625501 (0.0011) [2023-12-27 16:55:05,658][20381] Updated weights for policy 1, policy_version 1625511 (0.0010) [2023-12-27 16:55:06,251][20381] Updated weights for policy 1, policy_version 1625521 (0.0007) [2023-12-27 16:55:06,292][20347] Updated weights for policy 0, policy_version 1627031 (0.0010) [2023-12-27 16:55:06,299][20381] Updated weights for policy 1, policy_version 1625531 (0.0008) [2023-12-27 16:55:06,337][20347] Updated weights for policy 0, policy_version 1627041 (0.0010) [2023-12-27 16:55:06,344][20381] Updated weights for policy 1, policy_version 1625541 (0.0008) [2023-12-27 16:55:06,383][20347] Updated weights for policy 0, policy_version 1627051 (0.0009) [2023-12-27 16:55:06,390][20381] Updated weights for policy 1, policy_version 1625551 (0.0007) [2023-12-27 16:55:07,111][20347] Updated weights for policy 0, policy_version 1627061 (0.0007) [2023-12-27 16:55:07,130][20381] Updated weights for policy 1, policy_version 1625561 (0.0007) [2023-12-27 16:55:07,178][20347] Updated weights for policy 0, policy_version 1627071 (0.0008) [2023-12-27 16:55:07,194][20381] Updated weights for policy 1, policy_version 1625571 (0.0008) [2023-12-27 16:55:07,196][20292] KL-divergence is very high: 114.2684 [2023-12-27 16:55:07,239][20347] Updated weights for policy 0, policy_version 1627081 (0.0006) [2023-12-27 16:55:07,250][20292] KL-divergence is very high: 190.2898 [2023-12-27 16:55:07,262][20381] Updated weights for policy 1, policy_version 1625581 (0.0006) [2023-12-27 16:55:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 832806912. Throughput: 0: 10235.7, 1: 10220.0. Samples: 832768900. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:07,363][19170] Avg episode reward: [(0, '990.260'), (1, '844.130')] [2023-12-27 16:55:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001625584_416194560.pth... [2023-12-27 16:55:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001627088_416612352.pth... [2023-12-27 16:55:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001624400_415891456.pth [2023-12-27 16:55:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001625904_416309248.pth [2023-12-27 16:55:07,824][20347] Updated weights for policy 0, policy_version 1627091 (0.0005) [2023-12-27 16:55:07,866][20381] Updated weights for policy 1, policy_version 1625591 (0.0007) [2023-12-27 16:55:07,875][20347] Updated weights for policy 0, policy_version 1627101 (0.0005) [2023-12-27 16:55:07,923][20347] Updated weights for policy 0, policy_version 1627111 (0.0006) [2023-12-27 16:55:07,923][20381] Updated weights for policy 1, policy_version 1625601 (0.0007) [2023-12-27 16:55:07,988][20381] Updated weights for policy 1, policy_version 1625611 (0.0006) [2023-12-27 16:55:08,580][20347] Updated weights for policy 0, policy_version 1627121 (0.0007) [2023-12-27 16:55:08,639][20347] Updated weights for policy 0, policy_version 1627131 (0.0008) [2023-12-27 16:55:08,675][20381] Updated weights for policy 1, policy_version 1625621 (0.0007) [2023-12-27 16:55:08,703][20347] Updated weights for policy 0, policy_version 1627141 (0.0007) [2023-12-27 16:55:08,727][20381] Updated weights for policy 1, policy_version 1625631 (0.0006) [2023-12-27 16:55:08,760][20347] Updated weights for policy 0, policy_version 1627151 (0.0007) [2023-12-27 16:55:08,783][20381] Updated weights for policy 1, policy_version 1625641 (0.0006) [2023-12-27 16:55:09,357][20347] Updated weights for policy 0, policy_version 1627161 (0.0008) [2023-12-27 16:55:09,426][20347] Updated weights for policy 0, policy_version 1627171 (0.0008) [2023-12-27 16:55:09,488][20347] Updated weights for policy 0, policy_version 1627181 (0.0008) [2023-12-27 16:55:09,584][20381] Updated weights for policy 1, policy_version 1625651 (0.0009) [2023-12-27 16:55:09,646][20381] Updated weights for policy 1, policy_version 1625661 (0.0008) [2023-12-27 16:55:09,702][20381] Updated weights for policy 1, policy_version 1625671 (0.0007) [2023-12-27 16:55:10,202][20347] Updated weights for policy 0, policy_version 1627191 (0.0008) [2023-12-27 16:55:10,261][20347] Updated weights for policy 0, policy_version 1627201 (0.0007) [2023-12-27 16:55:10,321][20347] Updated weights for policy 0, policy_version 1627211 (0.0008) [2023-12-27 16:55:10,422][20381] Updated weights for policy 1, policy_version 1625681 (0.0008) [2023-12-27 16:55:10,483][20381] Updated weights for policy 1, policy_version 1625691 (0.0008) [2023-12-27 16:55:10,543][20381] Updated weights for policy 1, policy_version 1625701 (0.0008) [2023-12-27 16:55:10,597][20381] Updated weights for policy 1, policy_version 1625711 (0.0007) [2023-12-27 16:55:10,996][20347] Updated weights for policy 0, policy_version 1627221 (0.0008) [2023-12-27 16:55:11,061][20347] Updated weights for policy 0, policy_version 1627231 (0.0007) [2023-12-27 16:55:11,118][20347] Updated weights for policy 0, policy_version 1627241 (0.0006) [2023-12-27 16:55:11,278][20381] Updated weights for policy 1, policy_version 1625721 (0.0006) [2023-12-27 16:55:11,336][20381] Updated weights for policy 1, policy_version 1625731 (0.0007) [2023-12-27 16:55:11,400][20381] Updated weights for policy 1, policy_version 1625741 (0.0008) [2023-12-27 16:55:11,873][20347] Updated weights for policy 0, policy_version 1627251 (0.0006) [2023-12-27 16:55:11,932][20347] Updated weights for policy 0, policy_version 1627261 (0.0006) [2023-12-27 16:55:11,999][20347] Updated weights for policy 0, policy_version 1627271 (0.0006) [2023-12-27 16:55:12,183][20381] Updated weights for policy 1, policy_version 1625751 (0.0007) [2023-12-27 16:55:12,236][20381] Updated weights for policy 1, policy_version 1625761 (0.0006) [2023-12-27 16:55:12,299][20381] Updated weights for policy 1, policy_version 1625771 (0.0007) [2023-12-27 16:55:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 832905216. Throughput: 0: 10245.1, 1: 10218.7. Samples: 832893116. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:12,363][19170] Avg episode reward: [(0, '990.260'), (1, '873.260')] [2023-12-27 16:55:12,631][20347] Updated weights for policy 0, policy_version 1627281 (0.0007) [2023-12-27 16:55:12,693][20347] Updated weights for policy 0, policy_version 1627291 (0.0007) [2023-12-27 16:55:12,751][20347] Updated weights for policy 0, policy_version 1627301 (0.0005) [2023-12-27 16:55:12,819][20347] Updated weights for policy 0, policy_version 1627311 (0.0006) [2023-12-27 16:55:12,899][20381] Updated weights for policy 1, policy_version 1625781 (0.0008) [2023-12-27 16:55:12,959][20381] Updated weights for policy 1, policy_version 1625791 (0.0008) [2023-12-27 16:55:13,009][20381] Updated weights for policy 1, policy_version 1625801 (0.0008) [2023-12-27 16:55:13,410][20347] Updated weights for policy 0, policy_version 1627321 (0.0005) [2023-12-27 16:55:13,462][20347] Updated weights for policy 0, policy_version 1627331 (0.0005) [2023-12-27 16:55:13,514][20347] Updated weights for policy 0, policy_version 1627341 (0.0005) [2023-12-27 16:55:13,670][20381] Updated weights for policy 1, policy_version 1625811 (0.0007) [2023-12-27 16:55:13,718][20381] Updated weights for policy 1, policy_version 1625821 (0.0005) [2023-12-27 16:55:13,776][20381] Updated weights for policy 1, policy_version 1625831 (0.0006) [2023-12-27 16:55:14,112][20347] Updated weights for policy 0, policy_version 1627351 (0.0005) [2023-12-27 16:55:14,179][20347] Updated weights for policy 0, policy_version 1627361 (0.0005) [2023-12-27 16:55:14,248][20347] Updated weights for policy 0, policy_version 1627371 (0.0005) [2023-12-27 16:55:14,387][20381] Updated weights for policy 1, policy_version 1625841 (0.0006) [2023-12-27 16:55:14,452][20381] Updated weights for policy 1, policy_version 1625851 (0.0011) [2023-12-27 16:55:14,507][20381] Updated weights for policy 1, policy_version 1625861 (0.0007) [2023-12-27 16:55:14,566][20381] Updated weights for policy 1, policy_version 1625871 (0.0005) [2023-12-27 16:55:14,764][20347] Updated weights for policy 0, policy_version 1627381 (0.0006) [2023-12-27 16:55:14,826][20347] Updated weights for policy 0, policy_version 1627391 (0.0006) [2023-12-27 16:55:14,878][20347] Updated weights for policy 0, policy_version 1627401 (0.0008) [2023-12-27 16:55:15,195][20381] Updated weights for policy 1, policy_version 1625881 (0.0010) [2023-12-27 16:55:15,246][20381] Updated weights for policy 1, policy_version 1625891 (0.0008) [2023-12-27 16:55:15,305][20381] Updated weights for policy 1, policy_version 1625901 (0.0005) [2023-12-27 16:55:15,613][20347] Updated weights for policy 0, policy_version 1627411 (0.0007) [2023-12-27 16:55:15,663][20347] Updated weights for policy 0, policy_version 1627421 (0.0007) [2023-12-27 16:55:15,709][20347] Updated weights for policy 0, policy_version 1627431 (0.0007) [2023-12-27 16:55:15,883][20381] Updated weights for policy 1, policy_version 1625911 (0.0005) [2023-12-27 16:55:15,945][20381] Updated weights for policy 1, policy_version 1625921 (0.0007) [2023-12-27 16:55:16,004][20381] Updated weights for policy 1, policy_version 1625931 (0.0007) [2023-12-27 16:55:16,324][20347] Updated weights for policy 0, policy_version 1627441 (0.0007) [2023-12-27 16:55:16,371][20347] Updated weights for policy 0, policy_version 1627451 (0.0007) [2023-12-27 16:55:16,429][20347] Updated weights for policy 0, policy_version 1627461 (0.0007) [2023-12-27 16:55:16,478][20347] Updated weights for policy 0, policy_version 1627471 (0.0008) [2023-12-27 16:55:16,589][20381] Updated weights for policy 1, policy_version 1625941 (0.0009) [2023-12-27 16:55:16,646][20381] Updated weights for policy 1, policy_version 1625951 (0.0005) [2023-12-27 16:55:16,704][20381] Updated weights for policy 1, policy_version 1625961 (0.0010) [2023-12-27 16:55:17,109][20347] Updated weights for policy 0, policy_version 1627481 (0.0008) [2023-12-27 16:55:17,153][20347] Updated weights for policy 0, policy_version 1627491 (0.0005) [2023-12-27 16:55:17,197][20347] Updated weights for policy 0, policy_version 1627501 (0.0005) [2023-12-27 16:55:17,359][20381] Updated weights for policy 1, policy_version 1625971 (0.0010) [2023-12-27 16:55:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 833011712. Throughput: 0: 10273.1, 1: 10281.8. Samples: 833022168. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.230')] [2023-12-27 16:55:17,410][20381] Updated weights for policy 1, policy_version 1625981 (0.0010) [2023-12-27 16:55:17,460][20381] Updated weights for policy 1, policy_version 1625991 (0.0008) [2023-12-27 16:55:17,851][20347] Updated weights for policy 0, policy_version 1627511 (0.0007) [2023-12-27 16:55:17,907][20347] Updated weights for policy 0, policy_version 1627521 (0.0007) [2023-12-27 16:55:17,968][20347] Updated weights for policy 0, policy_version 1627531 (0.0007) [2023-12-27 16:55:18,149][20381] Updated weights for policy 1, policy_version 1626001 (0.0008) [2023-12-27 16:55:18,199][20381] Updated weights for policy 1, policy_version 1626011 (0.0008) [2023-12-27 16:55:18,254][20381] Updated weights for policy 1, policy_version 1626021 (0.0008) [2023-12-27 16:55:18,300][20381] Updated weights for policy 1, policy_version 1626031 (0.0008) [2023-12-27 16:55:18,688][20347] Updated weights for policy 0, policy_version 1627541 (0.0008) [2023-12-27 16:55:18,744][20347] Updated weights for policy 0, policy_version 1627551 (0.0007) [2023-12-27 16:55:18,807][20347] Updated weights for policy 0, policy_version 1627561 (0.0005) [2023-12-27 16:55:18,988][20381] Updated weights for policy 1, policy_version 1626041 (0.0009) [2023-12-27 16:55:19,043][20381] Updated weights for policy 1, policy_version 1626051 (0.0009) [2023-12-27 16:55:19,091][20381] Updated weights for policy 1, policy_version 1626061 (0.0007) [2023-12-27 16:55:19,355][20347] Updated weights for policy 0, policy_version 1627571 (0.0007) [2023-12-27 16:55:19,419][20347] Updated weights for policy 0, policy_version 1627581 (0.0007) [2023-12-27 16:55:19,480][20347] Updated weights for policy 0, policy_version 1627591 (0.0007) [2023-12-27 16:55:19,864][20381] Updated weights for policy 1, policy_version 1626071 (0.0009) [2023-12-27 16:55:19,927][20381] Updated weights for policy 1, policy_version 1626081 (0.0009) [2023-12-27 16:55:19,987][20381] Updated weights for policy 1, policy_version 1626091 (0.0008) [2023-12-27 16:55:20,214][20347] Updated weights for policy 0, policy_version 1627601 (0.0007) [2023-12-27 16:55:20,276][20347] Updated weights for policy 0, policy_version 1627611 (0.0008) [2023-12-27 16:55:20,334][20347] Updated weights for policy 0, policy_version 1627621 (0.0007) [2023-12-27 16:55:20,393][20347] Updated weights for policy 0, policy_version 1627631 (0.0008) [2023-12-27 16:55:20,768][20381] Updated weights for policy 1, policy_version 1626101 (0.0008) [2023-12-27 16:55:20,828][20381] Updated weights for policy 1, policy_version 1626111 (0.0008) [2023-12-27 16:55:20,894][20381] Updated weights for policy 1, policy_version 1626121 (0.0008) [2023-12-27 16:55:21,052][20347] Updated weights for policy 0, policy_version 1627641 (0.0007) [2023-12-27 16:55:21,118][20347] Updated weights for policy 0, policy_version 1627651 (0.0008) [2023-12-27 16:55:21,177][20347] Updated weights for policy 0, policy_version 1627661 (0.0008) [2023-12-27 16:55:21,659][20381] Updated weights for policy 1, policy_version 1626131 (0.0008) [2023-12-27 16:55:21,717][20381] Updated weights for policy 1, policy_version 1626141 (0.0008) [2023-12-27 16:55:21,781][20381] Updated weights for policy 1, policy_version 1626151 (0.0008) [2023-12-27 16:55:21,834][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000004 [2023-12-27 16:55:21,886][20347] Updated weights for policy 0, policy_version 1627671 (0.0009) [2023-12-27 16:55:21,952][20347] Updated weights for policy 0, policy_version 1627681 (0.0009) [2023-12-27 16:55:22,011][20347] Updated weights for policy 0, policy_version 1627691 (0.0007) [2023-12-27 16:55:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 833110016. Throughput: 0: 10301.9, 1: 10196.5. Samples: 833083208. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.750')] [2023-12-27 16:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001626160_416342016.pth... [2023-12-27 16:55:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001627696_416768000.pth... [2023-12-27 16:55:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001624976_416038912.pth [2023-12-27 16:55:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001626480_416456704.pth [2023-12-27 16:55:22,554][20381] Updated weights for policy 1, policy_version 1626161 (0.0009) [2023-12-27 16:55:22,617][20381] Updated weights for policy 1, policy_version 1626171 (0.0006) [2023-12-27 16:55:22,682][20381] Updated weights for policy 1, policy_version 1626181 (0.0007) [2023-12-27 16:55:22,685][20347] Updated weights for policy 0, policy_version 1627701 (0.0007) [2023-12-27 16:55:22,744][20347] Updated weights for policy 0, policy_version 1627711 (0.0007) [2023-12-27 16:55:22,745][20381] Updated weights for policy 1, policy_version 1626191 (0.0009) [2023-12-27 16:55:22,801][20347] Updated weights for policy 0, policy_version 1627721 (0.0007) [2023-12-27 16:55:23,410][20381] Updated weights for policy 1, policy_version 1626201 (0.0011) [2023-12-27 16:55:23,466][20381] Updated weights for policy 1, policy_version 1626211 (0.0011) [2023-12-27 16:55:23,485][20347] Updated weights for policy 0, policy_version 1627731 (0.0009) [2023-12-27 16:55:23,522][20381] Updated weights for policy 1, policy_version 1626221 (0.0011) [2023-12-27 16:55:23,570][20347] Updated weights for policy 0, policy_version 1627741 (0.0006) [2023-12-27 16:55:23,622][20347] Updated weights for policy 0, policy_version 1627751 (0.0008) [2023-12-27 16:55:24,136][20381] Updated weights for policy 1, policy_version 1626231 (0.0007) [2023-12-27 16:55:24,178][20347] Updated weights for policy 0, policy_version 1627761 (0.0007) [2023-12-27 16:55:24,201][20381] Updated weights for policy 1, policy_version 1626241 (0.0006) [2023-12-27 16:55:24,235][20347] Updated weights for policy 0, policy_version 1627771 (0.0006) [2023-12-27 16:55:24,260][20381] Updated weights for policy 1, policy_version 1626251 (0.0008) [2023-12-27 16:55:24,293][20347] Updated weights for policy 0, policy_version 1627781 (0.0007) [2023-12-27 16:55:24,347][20347] Updated weights for policy 0, policy_version 1627791 (0.0008) [2023-12-27 16:55:24,849][20381] Updated weights for policy 1, policy_version 1626261 (0.0006) [2023-12-27 16:55:24,906][20381] Updated weights for policy 1, policy_version 1626271 (0.0009) [2023-12-27 16:55:24,964][20381] Updated weights for policy 1, policy_version 1626281 (0.0005) [2023-12-27 16:55:24,969][20347] Updated weights for policy 0, policy_version 1627801 (0.0006) [2023-12-27 16:55:25,028][20347] Updated weights for policy 0, policy_version 1627811 (0.0009) [2023-12-27 16:55:25,079][20347] Updated weights for policy 0, policy_version 1627821 (0.0010) [2023-12-27 16:55:25,547][20381] Updated weights for policy 1, policy_version 1626291 (0.0006) [2023-12-27 16:55:25,598][20381] Updated weights for policy 1, policy_version 1626301 (0.0008) [2023-12-27 16:55:25,643][20381] Updated weights for policy 1, policy_version 1626311 (0.0009) [2023-12-27 16:55:25,679][20347] Updated weights for policy 0, policy_version 1627831 (0.0009) [2023-12-27 16:55:25,724][20347] Updated weights for policy 0, policy_version 1627841 (0.0010) [2023-12-27 16:55:25,780][20347] Updated weights for policy 0, policy_version 1627851 (0.0006) [2023-12-27 16:55:26,336][20381] Updated weights for policy 1, policy_version 1626321 (0.0009) [2023-12-27 16:55:26,391][20347] Updated weights for policy 0, policy_version 1627861 (0.0006) [2023-12-27 16:55:26,413][20381] Updated weights for policy 1, policy_version 1626331 (0.0008) [2023-12-27 16:55:26,448][20347] Updated weights for policy 0, policy_version 1627871 (0.0010) [2023-12-27 16:55:26,472][20381] Updated weights for policy 1, policy_version 1626341 (0.0007) [2023-12-27 16:55:26,505][20347] Updated weights for policy 0, policy_version 1627881 (0.0009) [2023-12-27 16:55:26,526][20381] Updated weights for policy 1, policy_version 1626351 (0.0009) [2023-12-27 16:55:27,070][20347] Updated weights for policy 0, policy_version 1627891 (0.0007) [2023-12-27 16:55:27,124][20347] Updated weights for policy 0, policy_version 1627901 (0.0010) [2023-12-27 16:55:27,178][20381] Updated weights for policy 1, policy_version 1626361 (0.0010) [2023-12-27 16:55:27,181][20347] Updated weights for policy 0, policy_version 1627911 (0.0010) [2023-12-27 16:55:27,233][20381] Updated weights for policy 1, policy_version 1626371 (0.0010) [2023-12-27 16:55:27,287][20381] Updated weights for policy 1, policy_version 1626381 (0.0010) [2023-12-27 16:55:27,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 833224704. Throughput: 0: 10321.9, 1: 10175.1. Samples: 833208036. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '882.930')] [2023-12-27 16:55:27,807][20347] Updated weights for policy 0, policy_version 1627921 (0.0010) [2023-12-27 16:55:27,854][20347] Updated weights for policy 0, policy_version 1627931 (0.0005) [2023-12-27 16:55:27,877][20381] Updated weights for policy 1, policy_version 1626391 (0.0005) [2023-12-27 16:55:27,898][20347] Updated weights for policy 0, policy_version 1627941 (0.0006) [2023-12-27 16:55:27,923][20381] Updated weights for policy 1, policy_version 1626401 (0.0005) [2023-12-27 16:55:27,944][20347] Updated weights for policy 0, policy_version 1627951 (0.0007) [2023-12-27 16:55:27,968][20381] Updated weights for policy 1, policy_version 1626411 (0.0005) [2023-12-27 16:55:28,568][20347] Updated weights for policy 0, policy_version 1627961 (0.0005) [2023-12-27 16:55:28,628][20381] Updated weights for policy 1, policy_version 1626421 (0.0008) [2023-12-27 16:55:28,629][20347] Updated weights for policy 0, policy_version 1627971 (0.0007) [2023-12-27 16:55:28,690][20347] Updated weights for policy 0, policy_version 1627981 (0.0010) [2023-12-27 16:55:28,692][20381] Updated weights for policy 1, policy_version 1626431 (0.0009) [2023-12-27 16:55:28,747][20381] Updated weights for policy 1, policy_version 1626441 (0.0011) [2023-12-27 16:55:29,330][20347] Updated weights for policy 0, policy_version 1627991 (0.0008) [2023-12-27 16:55:29,400][20347] Updated weights for policy 0, policy_version 1628001 (0.0010) [2023-12-27 16:55:29,434][20381] Updated weights for policy 1, policy_version 1626451 (0.0009) [2023-12-27 16:55:29,451][20347] Updated weights for policy 0, policy_version 1628011 (0.0006) [2023-12-27 16:55:29,488][20381] Updated weights for policy 1, policy_version 1626461 (0.0006) [2023-12-27 16:55:29,540][20381] Updated weights for policy 1, policy_version 1626471 (0.0008) [2023-12-27 16:55:30,157][20347] Updated weights for policy 0, policy_version 1628021 (0.0008) [2023-12-27 16:55:30,203][20347] Updated weights for policy 0, policy_version 1628031 (0.0010) [2023-12-27 16:55:30,254][20347] Updated weights for policy 0, policy_version 1628041 (0.0010) [2023-12-27 16:55:30,258][20381] Updated weights for policy 1, policy_version 1626481 (0.0007) [2023-12-27 16:55:30,317][20381] Updated weights for policy 1, policy_version 1626491 (0.0005) [2023-12-27 16:55:30,379][20381] Updated weights for policy 1, policy_version 1626501 (0.0006) [2023-12-27 16:55:30,439][20381] Updated weights for policy 1, policy_version 1626511 (0.0008) [2023-12-27 16:55:30,852][20347] Updated weights for policy 0, policy_version 1628051 (0.0009) [2023-12-27 16:55:30,914][20347] Updated weights for policy 0, policy_version 1628061 (0.0006) [2023-12-27 16:55:30,974][20347] Updated weights for policy 0, policy_version 1628071 (0.0009) [2023-12-27 16:55:31,076][20381] Updated weights for policy 1, policy_version 1626521 (0.0007) [2023-12-27 16:55:31,125][20381] Updated weights for policy 1, policy_version 1626531 (0.0008) [2023-12-27 16:55:31,190][20381] Updated weights for policy 1, policy_version 1626541 (0.0007) [2023-12-27 16:55:31,665][20347] Updated weights for policy 0, policy_version 1628081 (0.0008) [2023-12-27 16:55:31,721][20347] Updated weights for policy 0, policy_version 1628091 (0.0008) [2023-12-27 16:55:31,782][20347] Updated weights for policy 0, policy_version 1628101 (0.0007) [2023-12-27 16:55:31,842][20347] Updated weights for policy 0, policy_version 1628111 (0.0008) [2023-12-27 16:55:31,901][20381] Updated weights for policy 1, policy_version 1626551 (0.0008) [2023-12-27 16:55:31,961][20381] Updated weights for policy 1, policy_version 1626561 (0.0008) [2023-12-27 16:55:32,020][20381] Updated weights for policy 1, policy_version 1626571 (0.0008) [2023-12-27 16:55:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 833323008. Throughput: 0: 10414.2, 1: 10227.5. Samples: 833334332. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.000')] [2023-12-27 16:55:32,582][20347] Updated weights for policy 0, policy_version 1628121 (0.0008) [2023-12-27 16:55:32,641][20347] Updated weights for policy 0, policy_version 1628131 (0.0008) [2023-12-27 16:55:32,693][20381] Updated weights for policy 1, policy_version 1626581 (0.0007) [2023-12-27 16:55:32,700][20347] Updated weights for policy 0, policy_version 1628141 (0.0006) [2023-12-27 16:55:32,749][20381] Updated weights for policy 1, policy_version 1626591 (0.0005) [2023-12-27 16:55:32,810][20381] Updated weights for policy 1, policy_version 1626601 (0.0005) [2023-12-27 16:55:33,311][20381] Updated weights for policy 1, policy_version 1626611 (0.0006) [2023-12-27 16:55:33,361][20381] Updated weights for policy 1, policy_version 1626621 (0.0005) [2023-12-27 16:55:33,374][20347] Updated weights for policy 0, policy_version 1628151 (0.0007) [2023-12-27 16:55:33,417][20381] Updated weights for policy 1, policy_version 1626631 (0.0008) [2023-12-27 16:55:33,426][20347] Updated weights for policy 0, policy_version 1628161 (0.0007) [2023-12-27 16:55:33,478][20347] Updated weights for policy 0, policy_version 1628171 (0.0006) [2023-12-27 16:55:34,035][20381] Updated weights for policy 1, policy_version 1626641 (0.0010) [2023-12-27 16:55:34,093][20381] Updated weights for policy 1, policy_version 1626651 (0.0007) [2023-12-27 16:55:34,148][20381] Updated weights for policy 1, policy_version 1626661 (0.0006) [2023-12-27 16:55:34,195][20347] Updated weights for policy 0, policy_version 1628181 (0.0007) [2023-12-27 16:55:34,210][20381] Updated weights for policy 1, policy_version 1626671 (0.0009) [2023-12-27 16:55:34,258][20347] Updated weights for policy 0, policy_version 1628191 (0.0006) [2023-12-27 16:55:34,317][20347] Updated weights for policy 0, policy_version 1628201 (0.0008) [2023-12-27 16:55:34,837][20381] Updated weights for policy 1, policy_version 1626681 (0.0007) [2023-12-27 16:55:34,887][20381] Updated weights for policy 1, policy_version 1626691 (0.0007) [2023-12-27 16:55:34,937][20381] Updated weights for policy 1, policy_version 1626701 (0.0005) [2023-12-27 16:55:35,038][20347] Updated weights for policy 0, policy_version 1628211 (0.0006) [2023-12-27 16:55:35,099][20347] Updated weights for policy 0, policy_version 1628221 (0.0006) [2023-12-27 16:55:35,156][20347] Updated weights for policy 0, policy_version 1628231 (0.0010) [2023-12-27 16:55:35,531][20381] Updated weights for policy 1, policy_version 1626711 (0.0005) [2023-12-27 16:55:35,586][20381] Updated weights for policy 1, policy_version 1626721 (0.0005) [2023-12-27 16:55:35,634][20381] Updated weights for policy 1, policy_version 1626731 (0.0005) [2023-12-27 16:55:35,735][20347] Updated weights for policy 0, policy_version 1628241 (0.0009) [2023-12-27 16:55:35,801][20347] Updated weights for policy 0, policy_version 1628251 (0.0006) [2023-12-27 16:55:35,856][20347] Updated weights for policy 0, policy_version 1628261 (0.0008) [2023-12-27 16:55:35,911][20347] Updated weights for policy 0, policy_version 1628271 (0.0008) [2023-12-27 16:55:36,189][20381] Updated weights for policy 1, policy_version 1626741 (0.0008) [2023-12-27 16:55:36,252][20381] Updated weights for policy 1, policy_version 1626751 (0.0009) [2023-12-27 16:55:36,312][20381] Updated weights for policy 1, policy_version 1626761 (0.0011) [2023-12-27 16:55:36,568][20347] Updated weights for policy 0, policy_version 1628281 (0.0007) [2023-12-27 16:55:36,623][20347] Updated weights for policy 0, policy_version 1628291 (0.0007) [2023-12-27 16:55:36,679][20347] Updated weights for policy 0, policy_version 1628301 (0.0007) [2023-12-27 16:55:36,973][20381] Updated weights for policy 1, policy_version 1626771 (0.0009) [2023-12-27 16:55:37,029][20381] Updated weights for policy 1, policy_version 1626781 (0.0008) [2023-12-27 16:55:37,080][20381] Updated weights for policy 1, policy_version 1626791 (0.0008) [2023-12-27 16:55:37,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 833429504. Throughput: 0: 10402.6, 1: 10321.3. Samples: 833398220. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.390')] [2023-12-27 16:55:37,368][20347] Updated weights for policy 0, policy_version 1628311 (0.0008) [2023-12-27 16:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001626800_416505856.pth... [2023-12-27 16:55:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001625584_416194560.pth [2023-12-27 16:55:37,431][20347] Updated weights for policy 0, policy_version 1628321 (0.0008) [2023-12-27 16:55:37,491][20347] Updated weights for policy 0, policy_version 1628331 (0.0008) [2023-12-27 16:55:37,518][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001628336_416931840.pth... [2023-12-27 16:55:37,523][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001627088_416612352.pth [2023-12-27 16:55:37,830][20381] Updated weights for policy 1, policy_version 1626801 (0.0009) [2023-12-27 16:55:37,888][20381] Updated weights for policy 1, policy_version 1626811 (0.0011) [2023-12-27 16:55:37,946][20381] Updated weights for policy 1, policy_version 1626821 (0.0011) [2023-12-27 16:55:38,001][20381] Updated weights for policy 1, policy_version 1626831 (0.0010) [2023-12-27 16:55:38,197][20347] Updated weights for policy 0, policy_version 1628341 (0.0008) [2023-12-27 16:55:38,249][20347] Updated weights for policy 0, policy_version 1628351 (0.0007) [2023-12-27 16:55:38,299][20347] Updated weights for policy 0, policy_version 1628361 (0.0007) [2023-12-27 16:55:38,700][20381] Updated weights for policy 1, policy_version 1626841 (0.0011) [2023-12-27 16:55:38,762][20381] Updated weights for policy 1, policy_version 1626851 (0.0011) [2023-12-27 16:55:38,821][20381] Updated weights for policy 1, policy_version 1626861 (0.0011) [2023-12-27 16:55:39,035][20347] Updated weights for policy 0, policy_version 1628371 (0.0008) [2023-12-27 16:55:39,089][20347] Updated weights for policy 0, policy_version 1628381 (0.0007) [2023-12-27 16:55:39,145][20347] Updated weights for policy 0, policy_version 1628391 (0.0007) [2023-12-27 16:55:39,562][20381] Updated weights for policy 1, policy_version 1626871 (0.0008) [2023-12-27 16:55:39,620][20381] Updated weights for policy 1, policy_version 1626881 (0.0009) [2023-12-27 16:55:39,675][20381] Updated weights for policy 1, policy_version 1626891 (0.0009) [2023-12-27 16:55:39,837][20347] Updated weights for policy 0, policy_version 1628401 (0.0007) [2023-12-27 16:55:39,899][20347] Updated weights for policy 0, policy_version 1628411 (0.0007) [2023-12-27 16:55:39,963][20347] Updated weights for policy 0, policy_version 1628421 (0.0008) [2023-12-27 16:55:40,032][20347] Updated weights for policy 0, policy_version 1628431 (0.0006) [2023-12-27 16:55:40,375][20381] Updated weights for policy 1, policy_version 1626901 (0.0009) [2023-12-27 16:55:40,439][20381] Updated weights for policy 1, policy_version 1626911 (0.0010) [2023-12-27 16:55:40,509][20381] Updated weights for policy 1, policy_version 1626921 (0.0006) [2023-12-27 16:55:40,737][20347] Updated weights for policy 0, policy_version 1628441 (0.0010) [2023-12-27 16:55:40,789][20347] Updated weights for policy 0, policy_version 1628451 (0.0010) [2023-12-27 16:55:40,845][20347] Updated weights for policy 0, policy_version 1628461 (0.0010) [2023-12-27 16:55:41,238][20381] Updated weights for policy 1, policy_version 1626931 (0.0007) [2023-12-27 16:55:41,304][20381] Updated weights for policy 1, policy_version 1626941 (0.0008) [2023-12-27 16:55:41,363][20381] Updated weights for policy 1, policy_version 1626951 (0.0007) [2023-12-27 16:55:41,545][20347] Updated weights for policy 0, policy_version 1628471 (0.0011) [2023-12-27 16:55:41,601][20347] Updated weights for policy 0, policy_version 1628481 (0.0011) [2023-12-27 16:55:41,668][20347] Updated weights for policy 0, policy_version 1628491 (0.0008) [2023-12-27 16:55:42,057][20381] Updated weights for policy 1, policy_version 1626961 (0.0008) [2023-12-27 16:55:42,109][20381] Updated weights for policy 1, policy_version 1626971 (0.0010) [2023-12-27 16:55:42,160][20381] Updated weights for policy 1, policy_version 1626981 (0.0010) [2023-12-27 16:55:42,212][20381] Updated weights for policy 1, policy_version 1626991 (0.0010) [2023-12-27 16:55:42,312][20347] Updated weights for policy 0, policy_version 1628501 (0.0007) [2023-12-27 16:55:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 833527808. Throughput: 0: 10387.4, 1: 10360.2. Samples: 833519680. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.410')] [2023-12-27 16:55:42,382][20347] Updated weights for policy 0, policy_version 1628511 (0.0009) [2023-12-27 16:55:42,441][20347] Updated weights for policy 0, policy_version 1628521 (0.0010) [2023-12-27 16:55:42,960][20381] Updated weights for policy 1, policy_version 1627001 (0.0007) [2023-12-27 16:55:43,022][20381] Updated weights for policy 1, policy_version 1627011 (0.0009) [2023-12-27 16:55:43,085][20381] Updated weights for policy 1, policy_version 1627021 (0.0007) [2023-12-27 16:55:43,126][20347] Updated weights for policy 0, policy_version 1628531 (0.0010) [2023-12-27 16:55:43,183][20347] Updated weights for policy 0, policy_version 1628541 (0.0007) [2023-12-27 16:55:43,242][20347] Updated weights for policy 0, policy_version 1628551 (0.0008) [2023-12-27 16:55:43,708][20381] Updated weights for policy 1, policy_version 1627031 (0.0007) [2023-12-27 16:55:43,753][20381] Updated weights for policy 1, policy_version 1627041 (0.0007) [2023-12-27 16:55:43,811][20381] Updated weights for policy 1, policy_version 1627051 (0.0008) [2023-12-27 16:55:43,893][20347] Updated weights for policy 0, policy_version 1628561 (0.0009) [2023-12-27 16:55:43,954][20347] Updated weights for policy 0, policy_version 1628571 (0.0010) [2023-12-27 16:55:44,007][20347] Updated weights for policy 0, policy_version 1628581 (0.0010) [2023-12-27 16:55:44,061][20347] Updated weights for policy 0, policy_version 1628591 (0.0010) [2023-12-27 16:55:44,501][20381] Updated weights for policy 1, policy_version 1627061 (0.0010) [2023-12-27 16:55:44,559][20381] Updated weights for policy 1, policy_version 1627071 (0.0011) [2023-12-27 16:55:44,617][20381] Updated weights for policy 1, policy_version 1627081 (0.0010) [2023-12-27 16:55:44,749][20347] Updated weights for policy 0, policy_version 1628601 (0.0010) [2023-12-27 16:55:44,814][20347] Updated weights for policy 0, policy_version 1628611 (0.0007) [2023-12-27 16:55:44,883][20347] Updated weights for policy 0, policy_version 1628621 (0.0007) [2023-12-27 16:55:45,315][20381] Updated weights for policy 1, policy_version 1627091 (0.0009) [2023-12-27 16:55:45,365][20381] Updated weights for policy 1, policy_version 1627101 (0.0007) [2023-12-27 16:55:45,433][20381] Updated weights for policy 1, policy_version 1627111 (0.0007) [2023-12-27 16:55:45,530][20347] Updated weights for policy 0, policy_version 1628631 (0.0008) [2023-12-27 16:55:45,588][20347] Updated weights for policy 0, policy_version 1628641 (0.0005) [2023-12-27 16:55:45,646][20347] Updated weights for policy 0, policy_version 1628651 (0.0005) [2023-12-27 16:55:46,134][20381] Updated weights for policy 1, policy_version 1627121 (0.0007) [2023-12-27 16:55:46,190][20347] Updated weights for policy 0, policy_version 1628661 (0.0007) [2023-12-27 16:55:46,200][20381] Updated weights for policy 1, policy_version 1627131 (0.0006) [2023-12-27 16:55:46,241][20347] Updated weights for policy 0, policy_version 1628671 (0.0007) [2023-12-27 16:55:46,258][20381] Updated weights for policy 1, policy_version 1627141 (0.0005) [2023-12-27 16:55:46,295][20347] Updated weights for policy 0, policy_version 1628681 (0.0008) [2023-12-27 16:55:46,310][20381] Updated weights for policy 1, policy_version 1627151 (0.0008) [2023-12-27 16:55:46,988][20347] Updated weights for policy 0, policy_version 1628691 (0.0007) [2023-12-27 16:55:47,000][20381] Updated weights for policy 1, policy_version 1627161 (0.0007) [2023-12-27 16:55:47,038][20347] Updated weights for policy 0, policy_version 1628701 (0.0008) [2023-12-27 16:55:47,058][20381] Updated weights for policy 1, policy_version 1627171 (0.0009) [2023-12-27 16:55:47,090][20347] Updated weights for policy 0, policy_version 1628711 (0.0005) [2023-12-27 16:55:47,114][20381] Updated weights for policy 1, policy_version 1627181 (0.0006) [2023-12-27 16:55:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 833634304. Throughput: 0: 10446.3, 1: 10359.1. Samples: 833641976. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.480')] [2023-12-27 16:55:47,681][20347] Updated weights for policy 0, policy_version 1628721 (0.0006) [2023-12-27 16:55:47,738][20347] Updated weights for policy 0, policy_version 1628731 (0.0007) [2023-12-27 16:55:47,789][20347] Updated weights for policy 0, policy_version 1628741 (0.0007) [2023-12-27 16:55:47,826][20381] Updated weights for policy 1, policy_version 1627191 (0.0007) [2023-12-27 16:55:47,845][20347] Updated weights for policy 0, policy_version 1628751 (0.0005) [2023-12-27 16:55:47,873][20381] Updated weights for policy 1, policy_version 1627201 (0.0008) [2023-12-27 16:55:47,926][20381] Updated weights for policy 1, policy_version 1627211 (0.0008) [2023-12-27 16:55:48,472][20347] Updated weights for policy 0, policy_version 1628761 (0.0005) [2023-12-27 16:55:48,529][20347] Updated weights for policy 0, policy_version 1628771 (0.0005) [2023-12-27 16:55:48,592][20347] Updated weights for policy 0, policy_version 1628781 (0.0007) [2023-12-27 16:55:48,688][20381] Updated weights for policy 1, policy_version 1627221 (0.0007) [2023-12-27 16:55:48,748][20381] Updated weights for policy 1, policy_version 1627231 (0.0009) [2023-12-27 16:55:48,804][20381] Updated weights for policy 1, policy_version 1627241 (0.0008) [2023-12-27 16:55:49,237][20347] Updated weights for policy 0, policy_version 1628791 (0.0008) [2023-12-27 16:55:49,288][20347] Updated weights for policy 0, policy_version 1628801 (0.0008) [2023-12-27 16:55:49,350][20347] Updated weights for policy 0, policy_version 1628811 (0.0007) [2023-12-27 16:55:49,533][20381] Updated weights for policy 1, policy_version 1627251 (0.0008) [2023-12-27 16:55:49,583][20381] Updated weights for policy 1, policy_version 1627261 (0.0008) [2023-12-27 16:55:49,643][20381] Updated weights for policy 1, policy_version 1627271 (0.0008) [2023-12-27 16:55:50,110][20347] Updated weights for policy 0, policy_version 1628821 (0.0008) [2023-12-27 16:55:50,155][20347] Updated weights for policy 0, policy_version 1628831 (0.0008) [2023-12-27 16:55:50,211][20347] Updated weights for policy 0, policy_version 1628841 (0.0008) [2023-12-27 16:55:50,344][20381] Updated weights for policy 1, policy_version 1627281 (0.0009) [2023-12-27 16:55:50,390][20381] Updated weights for policy 1, policy_version 1627291 (0.0008) [2023-12-27 16:55:50,433][20381] Updated weights for policy 1, policy_version 1627301 (0.0007) [2023-12-27 16:55:50,481][20381] Updated weights for policy 1, policy_version 1627311 (0.0005) [2023-12-27 16:55:50,986][20347] Updated weights for policy 0, policy_version 1628851 (0.0007) [2023-12-27 16:55:51,056][20347] Updated weights for policy 0, policy_version 1628861 (0.0011) [2023-12-27 16:55:51,116][20381] Updated weights for policy 1, policy_version 1627321 (0.0008) [2023-12-27 16:55:51,120][20347] Updated weights for policy 0, policy_version 1628871 (0.0007) [2023-12-27 16:55:51,173][20381] Updated weights for policy 1, policy_version 1627331 (0.0008) [2023-12-27 16:55:51,229][20381] Updated weights for policy 1, policy_version 1627341 (0.0008) [2023-12-27 16:55:51,839][20347] Updated weights for policy 0, policy_version 1628881 (0.0007) [2023-12-27 16:55:51,898][20347] Updated weights for policy 0, policy_version 1628891 (0.0007) [2023-12-27 16:55:51,960][20347] Updated weights for policy 0, policy_version 1628901 (0.0008) [2023-12-27 16:55:51,993][20381] Updated weights for policy 1, policy_version 1627351 (0.0006) [2023-12-27 16:55:52,018][20347] Updated weights for policy 0, policy_version 1628911 (0.0006) [2023-12-27 16:55:52,053][20381] Updated weights for policy 1, policy_version 1627361 (0.0007) [2023-12-27 16:55:52,109][20381] Updated weights for policy 1, policy_version 1627371 (0.0008) [2023-12-27 16:55:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 833732608. Throughput: 0: 10468.4, 1: 10287.5. Samples: 833702920. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.070')] [2023-12-27 16:55:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001627376_416653312.pth... [2023-12-27 16:55:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001628912_417079296.pth... [2023-12-27 16:55:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001627696_416768000.pth [2023-12-27 16:55:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001626160_416342016.pth [2023-12-27 16:55:52,716][20347] Updated weights for policy 0, policy_version 1628921 (0.0007) [2023-12-27 16:55:52,769][20347] Updated weights for policy 0, policy_version 1628931 (0.0008) [2023-12-27 16:55:52,824][20381] Updated weights for policy 1, policy_version 1627381 (0.0007) [2023-12-27 16:55:52,829][20347] Updated weights for policy 0, policy_version 1628941 (0.0006) [2023-12-27 16:55:52,881][20381] Updated weights for policy 1, policy_version 1627391 (0.0008) [2023-12-27 16:55:52,927][20381] Updated weights for policy 1, policy_version 1627401 (0.0009) [2023-12-27 16:55:53,436][20347] Updated weights for policy 0, policy_version 1628951 (0.0007) [2023-12-27 16:55:53,487][20347] Updated weights for policy 0, policy_version 1628961 (0.0007) [2023-12-27 16:55:53,538][20347] Updated weights for policy 0, policy_version 1628971 (0.0007) [2023-12-27 16:55:53,659][20381] Updated weights for policy 1, policy_version 1627411 (0.0010) [2023-12-27 16:55:53,716][20381] Updated weights for policy 1, policy_version 1627421 (0.0008) [2023-12-27 16:55:53,768][20381] Updated weights for policy 1, policy_version 1627431 (0.0007) [2023-12-27 16:55:54,212][20347] Updated weights for policy 0, policy_version 1628981 (0.0006) [2023-12-27 16:55:54,264][20347] Updated weights for policy 0, policy_version 1628991 (0.0005) [2023-12-27 16:55:54,319][20347] Updated weights for policy 0, policy_version 1629001 (0.0006) [2023-12-27 16:55:54,449][20381] Updated weights for policy 1, policy_version 1627441 (0.0008) [2023-12-27 16:55:54,498][20381] Updated weights for policy 1, policy_version 1627451 (0.0005) [2023-12-27 16:55:54,552][20381] Updated weights for policy 1, policy_version 1627461 (0.0005) [2023-12-27 16:55:54,594][20292] KL-divergence is very high: 114.4037 [2023-12-27 16:55:54,601][20381] Updated weights for policy 1, policy_version 1627471 (0.0009) [2023-12-27 16:55:54,930][20347] Updated weights for policy 0, policy_version 1629011 (0.0006) [2023-12-27 16:55:54,981][20347] Updated weights for policy 0, policy_version 1629021 (0.0007) [2023-12-27 16:55:55,045][20347] Updated weights for policy 0, policy_version 1629031 (0.0008) [2023-12-27 16:55:55,244][20381] Updated weights for policy 1, policy_version 1627481 (0.0007) [2023-12-27 16:55:55,292][20381] Updated weights for policy 1, policy_version 1627491 (0.0005) [2023-12-27 16:55:55,342][20381] Updated weights for policy 1, policy_version 1627501 (0.0005) [2023-12-27 16:55:55,767][20347] Updated weights for policy 0, policy_version 1629041 (0.0006) [2023-12-27 16:55:55,817][20347] Updated weights for policy 0, policy_version 1629051 (0.0007) [2023-12-27 16:55:55,866][20347] Updated weights for policy 0, policy_version 1629061 (0.0008) [2023-12-27 16:55:55,910][20381] Updated weights for policy 1, policy_version 1627511 (0.0009) [2023-12-27 16:55:55,915][20347] Updated weights for policy 0, policy_version 1629071 (0.0006) [2023-12-27 16:55:55,968][20381] Updated weights for policy 1, policy_version 1627521 (0.0010) [2023-12-27 16:55:56,021][20381] Updated weights for policy 1, policy_version 1627531 (0.0011) [2023-12-27 16:55:56,559][20347] Updated weights for policy 0, policy_version 1629081 (0.0007) [2023-12-27 16:55:56,608][20347] Updated weights for policy 0, policy_version 1629091 (0.0007) [2023-12-27 16:55:56,622][20381] Updated weights for policy 1, policy_version 1627541 (0.0008) [2023-12-27 16:55:56,665][20347] Updated weights for policy 0, policy_version 1629101 (0.0007) [2023-12-27 16:55:56,667][20381] Updated weights for policy 1, policy_version 1627551 (0.0010) [2023-12-27 16:55:56,715][20381] Updated weights for policy 1, policy_version 1627561 (0.0010) [2023-12-27 16:55:57,356][20347] Updated weights for policy 0, policy_version 1629111 (0.0007) [2023-12-27 16:55:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 833830912. Throughput: 0: 10418.5, 1: 10304.6. Samples: 833825656. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:55:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.790')] [2023-12-27 16:55:57,406][20347] Updated weights for policy 0, policy_version 1629121 (0.0007) [2023-12-27 16:55:57,417][20381] Updated weights for policy 1, policy_version 1627571 (0.0010) [2023-12-27 16:55:57,448][20347] Updated weights for policy 0, policy_version 1629131 (0.0005) [2023-12-27 16:55:57,462][20381] Updated weights for policy 1, policy_version 1627581 (0.0010) [2023-12-27 16:55:57,511][20381] Updated weights for policy 1, policy_version 1627591 (0.0008) [2023-12-27 16:55:58,112][20347] Updated weights for policy 0, policy_version 1629141 (0.0008) [2023-12-27 16:55:58,150][20381] Updated weights for policy 1, policy_version 1627601 (0.0006) [2023-12-27 16:55:58,168][20347] Updated weights for policy 0, policy_version 1629151 (0.0006) [2023-12-27 16:55:58,211][20381] Updated weights for policy 1, policy_version 1627611 (0.0009) [2023-12-27 16:55:58,233][20347] Updated weights for policy 0, policy_version 1629161 (0.0009) [2023-12-27 16:55:58,269][20381] Updated weights for policy 1, policy_version 1627621 (0.0009) [2023-12-27 16:55:58,330][20381] Updated weights for policy 1, policy_version 1627631 (0.0008) [2023-12-27 16:55:58,990][20347] Updated weights for policy 0, policy_version 1629171 (0.0009) [2023-12-27 16:55:59,050][20347] Updated weights for policy 0, policy_version 1629181 (0.0008) [2023-12-27 16:55:59,110][20347] Updated weights for policy 0, policy_version 1629191 (0.0007) [2023-12-27 16:55:59,134][20381] Updated weights for policy 1, policy_version 1627641 (0.0010) [2023-12-27 16:55:59,191][20381] Updated weights for policy 1, policy_version 1627651 (0.0010) [2023-12-27 16:55:59,249][20381] Updated weights for policy 1, policy_version 1627661 (0.0008) [2023-12-27 16:55:59,860][20347] Updated weights for policy 0, policy_version 1629201 (0.0005) [2023-12-27 16:55:59,893][20381] Updated weights for policy 1, policy_version 1627671 (0.0007) [2023-12-27 16:55:59,922][20347] Updated weights for policy 0, policy_version 1629211 (0.0007) [2023-12-27 16:55:59,954][20381] Updated weights for policy 1, policy_version 1627681 (0.0009) [2023-12-27 16:55:59,980][20347] Updated weights for policy 0, policy_version 1629221 (0.0007) [2023-12-27 16:56:00,013][20381] Updated weights for policy 1, policy_version 1627691 (0.0008) [2023-12-27 16:56:00,036][20347] Updated weights for policy 0, policy_version 1629231 (0.0007) [2023-12-27 16:56:00,715][20347] Updated weights for policy 0, policy_version 1629241 (0.0006) [2023-12-27 16:56:00,724][20381] Updated weights for policy 1, policy_version 1627701 (0.0007) [2023-12-27 16:56:00,766][20347] Updated weights for policy 0, policy_version 1629251 (0.0005) [2023-12-27 16:56:00,776][20381] Updated weights for policy 1, policy_version 1627711 (0.0006) [2023-12-27 16:56:00,831][20381] Updated weights for policy 1, policy_version 1627721 (0.0006) [2023-12-27 16:56:00,832][20347] Updated weights for policy 0, policy_version 1629261 (0.0008) [2023-12-27 16:56:01,542][20381] Updated weights for policy 1, policy_version 1627731 (0.0007) [2023-12-27 16:56:01,566][20347] Updated weights for policy 0, policy_version 1629271 (0.0007) [2023-12-27 16:56:01,586][20381] Updated weights for policy 1, policy_version 1627741 (0.0006) [2023-12-27 16:56:01,628][20347] Updated weights for policy 0, policy_version 1629281 (0.0008) [2023-12-27 16:56:01,646][20381] Updated weights for policy 1, policy_version 1627751 (0.0007) [2023-12-27 16:56:01,687][20347] Updated weights for policy 0, policy_version 1629291 (0.0008) [2023-12-27 16:56:02,315][20381] Updated weights for policy 1, policy_version 1627761 (0.0007) [2023-12-27 16:56:02,358][20347] Updated weights for policy 0, policy_version 1629301 (0.0008) [2023-12-27 16:56:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 833929216. Throughput: 0: 10319.5, 1: 10213.1. Samples: 833946140. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:56:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.550')] [2023-12-27 16:56:02,375][20381] Updated weights for policy 1, policy_version 1627771 (0.0009) [2023-12-27 16:56:02,411][20347] Updated weights for policy 0, policy_version 1629311 (0.0006) [2023-12-27 16:56:02,432][20381] Updated weights for policy 1, policy_version 1627781 (0.0008) [2023-12-27 16:56:02,471][20347] Updated weights for policy 0, policy_version 1629321 (0.0006) [2023-12-27 16:56:02,486][20381] Updated weights for policy 1, policy_version 1627791 (0.0006) [2023-12-27 16:56:03,128][20347] Updated weights for policy 0, policy_version 1629331 (0.0007) [2023-12-27 16:56:03,136][20381] Updated weights for policy 1, policy_version 1627801 (0.0006) [2023-12-27 16:56:03,178][20347] Updated weights for policy 0, policy_version 1629341 (0.0008) [2023-12-27 16:56:03,194][20381] Updated weights for policy 1, policy_version 1627811 (0.0006) [2023-12-27 16:56:03,229][20347] Updated weights for policy 0, policy_version 1629351 (0.0008) [2023-12-27 16:56:03,246][20381] Updated weights for policy 1, policy_version 1627821 (0.0008) [2023-12-27 16:56:03,884][20381] Updated weights for policy 1, policy_version 1627831 (0.0009) [2023-12-27 16:56:03,939][20381] Updated weights for policy 1, policy_version 1627841 (0.0008) [2023-12-27 16:56:03,973][20347] Updated weights for policy 0, policy_version 1629361 (0.0006) [2023-12-27 16:56:03,998][20381] Updated weights for policy 1, policy_version 1627851 (0.0008) [2023-12-27 16:56:04,034][20347] Updated weights for policy 0, policy_version 1629371 (0.0006) [2023-12-27 16:56:04,088][20347] Updated weights for policy 0, policy_version 1629381 (0.0007) [2023-12-27 16:56:04,148][20347] Updated weights for policy 0, policy_version 1629391 (0.0006) [2023-12-27 16:56:04,746][20381] Updated weights for policy 1, policy_version 1627861 (0.0009) [2023-12-27 16:56:04,799][20381] Updated weights for policy 1, policy_version 1627871 (0.0010) [2023-12-27 16:56:04,836][20347] Updated weights for policy 0, policy_version 1629401 (0.0007) [2023-12-27 16:56:04,853][20381] Updated weights for policy 1, policy_version 1627881 (0.0011) [2023-12-27 16:56:04,892][20347] Updated weights for policy 0, policy_version 1629411 (0.0006) [2023-12-27 16:56:04,943][20347] Updated weights for policy 0, policy_version 1629421 (0.0007) [2023-12-27 16:56:05,443][20381] Updated weights for policy 1, policy_version 1627891 (0.0010) [2023-12-27 16:56:05,507][20381] Updated weights for policy 1, policy_version 1627901 (0.0010) [2023-12-27 16:56:05,566][20381] Updated weights for policy 1, policy_version 1627911 (0.0011) [2023-12-27 16:56:05,655][20347] Updated weights for policy 0, policy_version 1629431 (0.0008) [2023-12-27 16:56:05,707][20347] Updated weights for policy 0, policy_version 1629441 (0.0007) [2023-12-27 16:56:05,760][20347] Updated weights for policy 0, policy_version 1629451 (0.0006) [2023-12-27 16:56:06,234][20381] Updated weights for policy 1, policy_version 1627921 (0.0010) [2023-12-27 16:56:06,297][20381] Updated weights for policy 1, policy_version 1627931 (0.0007) [2023-12-27 16:56:06,353][20381] Updated weights for policy 1, policy_version 1627941 (0.0007) [2023-12-27 16:56:06,414][20381] Updated weights for policy 1, policy_version 1627951 (0.0007) [2023-12-27 16:56:06,466][20347] Updated weights for policy 0, policy_version 1629461 (0.0007) [2023-12-27 16:56:06,527][20347] Updated weights for policy 0, policy_version 1629471 (0.0008) [2023-12-27 16:56:06,584][20347] Updated weights for policy 0, policy_version 1629481 (0.0008) [2023-12-27 16:56:07,048][20381] Updated weights for policy 1, policy_version 1627961 (0.0008) [2023-12-27 16:56:07,093][20381] Updated weights for policy 1, policy_version 1627971 (0.0007) [2023-12-27 16:56:07,147][20381] Updated weights for policy 1, policy_version 1627981 (0.0008) [2023-12-27 16:56:07,336][20347] Updated weights for policy 0, policy_version 1629491 (0.0007) [2023-12-27 16:56:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 834035712. Throughput: 0: 10277.2, 1: 10255.7. Samples: 834007192. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:56:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.800')] [2023-12-27 16:56:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001627984_416808960.pth... [2023-12-27 16:56:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001626800_416505856.pth [2023-12-27 16:56:07,395][20347] Updated weights for policy 0, policy_version 1629501 (0.0006) [2023-12-27 16:56:07,465][20347] Updated weights for policy 0, policy_version 1629511 (0.0007) [2023-12-27 16:56:07,518][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001629520_417234944.pth... [2023-12-27 16:56:07,522][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001628336_416931840.pth [2023-12-27 16:56:07,800][20381] Updated weights for policy 1, policy_version 1627991 (0.0007) [2023-12-27 16:56:07,863][20381] Updated weights for policy 1, policy_version 1628001 (0.0009) [2023-12-27 16:56:07,929][20381] Updated weights for policy 1, policy_version 1628011 (0.0008) [2023-12-27 16:56:08,199][20347] Updated weights for policy 0, policy_version 1629521 (0.0008) [2023-12-27 16:56:08,270][20347] Updated weights for policy 0, policy_version 1629531 (0.0006) [2023-12-27 16:56:08,331][20347] Updated weights for policy 0, policy_version 1629541 (0.0007) [2023-12-27 16:56:08,394][20347] Updated weights for policy 0, policy_version 1629551 (0.0006) [2023-12-27 16:56:08,611][20381] Updated weights for policy 1, policy_version 1628021 (0.0007) [2023-12-27 16:56:08,671][20381] Updated weights for policy 1, policy_version 1628031 (0.0008) [2023-12-27 16:56:08,730][20381] Updated weights for policy 1, policy_version 1628041 (0.0008) [2023-12-27 16:56:09,008][20347] Updated weights for policy 0, policy_version 1629561 (0.0007) [2023-12-27 16:56:09,058][20347] Updated weights for policy 0, policy_version 1629571 (0.0005) [2023-12-27 16:56:09,115][20347] Updated weights for policy 0, policy_version 1629581 (0.0007) [2023-12-27 16:56:09,505][20381] Updated weights for policy 1, policy_version 1628051 (0.0008) [2023-12-27 16:56:09,556][20381] Updated weights for policy 1, policy_version 1628061 (0.0008) [2023-12-27 16:56:09,617][20381] Updated weights for policy 1, policy_version 1628071 (0.0008) [2023-12-27 16:56:09,847][20347] Updated weights for policy 0, policy_version 1629591 (0.0009) [2023-12-27 16:56:09,905][20347] Updated weights for policy 0, policy_version 1629601 (0.0008) [2023-12-27 16:56:09,965][20347] Updated weights for policy 0, policy_version 1629611 (0.0008) [2023-12-27 16:56:10,327][20381] Updated weights for policy 1, policy_version 1628081 (0.0008) [2023-12-27 16:56:10,392][20381] Updated weights for policy 1, policy_version 1628091 (0.0007) [2023-12-27 16:56:10,448][20381] Updated weights for policy 1, policy_version 1628101 (0.0005) [2023-12-27 16:56:10,504][20381] Updated weights for policy 1, policy_version 1628111 (0.0007) [2023-12-27 16:56:10,677][20347] Updated weights for policy 0, policy_version 1629621 (0.0008) [2023-12-27 16:56:10,735][20347] Updated weights for policy 0, policy_version 1629631 (0.0006) [2023-12-27 16:56:10,787][20347] Updated weights for policy 0, policy_version 1629641 (0.0008) [2023-12-27 16:56:11,204][20381] Updated weights for policy 1, policy_version 1628121 (0.0008) [2023-12-27 16:56:11,268][20381] Updated weights for policy 1, policy_version 1628131 (0.0008) [2023-12-27 16:56:11,327][20381] Updated weights for policy 1, policy_version 1628141 (0.0009) [2023-12-27 16:56:11,523][20347] Updated weights for policy 0, policy_version 1629651 (0.0008) [2023-12-27 16:56:11,582][20347] Updated weights for policy 0, policy_version 1629661 (0.0010) [2023-12-27 16:56:11,640][20347] Updated weights for policy 0, policy_version 1629671 (0.0008) [2023-12-27 16:56:12,056][20381] Updated weights for policy 1, policy_version 1628151 (0.0007) [2023-12-27 16:56:12,120][20381] Updated weights for policy 1, policy_version 1628161 (0.0008) [2023-12-27 16:56:12,182][20381] Updated weights for policy 1, policy_version 1628171 (0.0011) [2023-12-27 16:56:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 834134016. Throughput: 0: 10177.7, 1: 10229.5. Samples: 834126352. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:56:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.330')] [2023-12-27 16:56:12,398][20347] Updated weights for policy 0, policy_version 1629681 (0.0008) [2023-12-27 16:56:12,462][20347] Updated weights for policy 0, policy_version 1629691 (0.0010) [2023-12-27 16:56:12,515][20347] Updated weights for policy 0, policy_version 1629701 (0.0010) [2023-12-27 16:56:12,571][20347] Updated weights for policy 0, policy_version 1629711 (0.0010) [2023-12-27 16:56:12,901][20381] Updated weights for policy 1, policy_version 1628181 (0.0009) [2023-12-27 16:56:12,956][20381] Updated weights for policy 1, policy_version 1628191 (0.0007) [2023-12-27 16:56:12,999][20381] Updated weights for policy 1, policy_version 1628201 (0.0007) [2023-12-27 16:56:13,254][20347] Updated weights for policy 0, policy_version 1629721 (0.0010) [2023-12-27 16:56:13,306][20347] Updated weights for policy 0, policy_version 1629731 (0.0010) [2023-12-27 16:56:13,361][20347] Updated weights for policy 0, policy_version 1629741 (0.0010) [2023-12-27 16:56:13,621][20381] Updated weights for policy 1, policy_version 1628211 (0.0007) [2023-12-27 16:56:13,669][20381] Updated weights for policy 1, policy_version 1628221 (0.0005) [2023-12-27 16:56:13,720][20381] Updated weights for policy 1, policy_version 1628231 (0.0005) [2023-12-27 16:56:13,953][20347] Updated weights for policy 0, policy_version 1629751 (0.0010) [2023-12-27 16:56:14,002][20347] Updated weights for policy 0, policy_version 1629761 (0.0010) [2023-12-27 16:56:14,055][20347] Updated weights for policy 0, policy_version 1629771 (0.0010) [2023-12-27 16:56:14,335][20381] Updated weights for policy 1, policy_version 1628241 (0.0006) [2023-12-27 16:56:14,393][20381] Updated weights for policy 1, policy_version 1628251 (0.0011) [2023-12-27 16:56:14,449][20381] Updated weights for policy 1, policy_version 1628261 (0.0010) [2023-12-27 16:56:14,502][20381] Updated weights for policy 1, policy_version 1628271 (0.0010) [2023-12-27 16:56:14,748][20347] Updated weights for policy 0, policy_version 1629781 (0.0008) [2023-12-27 16:56:14,817][20347] Updated weights for policy 0, policy_version 1629791 (0.0007) [2023-12-27 16:56:14,877][20347] Updated weights for policy 0, policy_version 1629801 (0.0006) [2023-12-27 16:56:15,257][20381] Updated weights for policy 1, policy_version 1628281 (0.0008) [2023-12-27 16:56:15,313][20381] Updated weights for policy 1, policy_version 1628291 (0.0008) [2023-12-27 16:56:15,372][20381] Updated weights for policy 1, policy_version 1628301 (0.0008) [2023-12-27 16:56:15,462][20347] Updated weights for policy 0, policy_version 1629811 (0.0005) [2023-12-27 16:56:15,517][20347] Updated weights for policy 0, policy_version 1629821 (0.0005) [2023-12-27 16:56:15,563][20347] Updated weights for policy 0, policy_version 1629831 (0.0005) [2023-12-27 16:56:16,126][20381] Updated weights for policy 1, policy_version 1628311 (0.0008) [2023-12-27 16:56:16,161][20347] Updated weights for policy 0, policy_version 1629841 (0.0005) [2023-12-27 16:56:16,185][20381] Updated weights for policy 1, policy_version 1628321 (0.0008) [2023-12-27 16:56:16,226][20347] Updated weights for policy 0, policy_version 1629851 (0.0005) [2023-12-27 16:56:16,245][20381] Updated weights for policy 1, policy_version 1628331 (0.0008) [2023-12-27 16:56:16,280][20347] Updated weights for policy 0, policy_version 1629861 (0.0005) [2023-12-27 16:56:16,338][20347] Updated weights for policy 0, policy_version 1629871 (0.0006) [2023-12-27 16:56:16,953][20381] Updated weights for policy 1, policy_version 1628341 (0.0007) [2023-12-27 16:56:16,970][20347] Updated weights for policy 0, policy_version 1629881 (0.0006) [2023-12-27 16:56:17,008][20381] Updated weights for policy 1, policy_version 1628351 (0.0008) [2023-12-27 16:56:17,030][20347] Updated weights for policy 0, policy_version 1629891 (0.0006) [2023-12-27 16:56:17,065][20381] Updated weights for policy 1, policy_version 1628361 (0.0006) [2023-12-27 16:56:17,088][20347] Updated weights for policy 0, policy_version 1629901 (0.0006) [2023-12-27 16:56:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 834240512. Throughput: 0: 10155.1, 1: 10172.0. Samples: 834249052. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:56:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.170')] [2023-12-27 16:56:17,768][20347] Updated weights for policy 0, policy_version 1629911 (0.0007) [2023-12-27 16:56:17,770][20381] Updated weights for policy 1, policy_version 1628371 (0.0006) [2023-12-27 16:56:17,820][20347] Updated weights for policy 0, policy_version 1629921 (0.0006) [2023-12-27 16:56:17,822][20381] Updated weights for policy 1, policy_version 1628381 (0.0005) [2023-12-27 16:56:17,876][20381] Updated weights for policy 1, policy_version 1628391 (0.0008) [2023-12-27 16:56:17,880][20347] Updated weights for policy 0, policy_version 1629931 (0.0006) [2023-12-27 16:56:18,542][20381] Updated weights for policy 1, policy_version 1628401 (0.0006) [2023-12-27 16:56:18,567][20347] Updated weights for policy 0, policy_version 1629941 (0.0006) [2023-12-27 16:56:18,602][20381] Updated weights for policy 1, policy_version 1628411 (0.0011) [2023-12-27 16:56:18,623][20347] Updated weights for policy 0, policy_version 1629951 (0.0005) [2023-12-27 16:56:18,656][20381] Updated weights for policy 1, policy_version 1628421 (0.0011) [2023-12-27 16:56:18,679][20347] Updated weights for policy 0, policy_version 1629961 (0.0006) [2023-12-27 16:56:18,713][20381] Updated weights for policy 1, policy_version 1628431 (0.0011) [2023-12-27 16:56:19,349][20381] Updated weights for policy 1, policy_version 1628441 (0.0009) [2023-12-27 16:56:19,381][20347] Updated weights for policy 0, policy_version 1629971 (0.0008) [2023-12-27 16:56:19,413][20381] Updated weights for policy 1, policy_version 1628451 (0.0010) [2023-12-27 16:56:19,436][20347] Updated weights for policy 0, policy_version 1629981 (0.0008) [2023-12-27 16:56:19,468][20381] Updated weights for policy 1, policy_version 1628461 (0.0010) [2023-12-27 16:56:19,496][20347] Updated weights for policy 0, policy_version 1629991 (0.0008) [2023-12-27 16:56:20,126][20381] Updated weights for policy 1, policy_version 1628471 (0.0008) [2023-12-27 16:56:20,186][20381] Updated weights for policy 1, policy_version 1628481 (0.0008) [2023-12-27 16:56:20,241][20381] Updated weights for policy 1, policy_version 1628491 (0.0006) [2023-12-27 16:56:20,305][20347] Updated weights for policy 0, policy_version 1630001 (0.0008) [2023-12-27 16:56:20,361][20347] Updated weights for policy 0, policy_version 1630011 (0.0007) [2023-12-27 16:56:20,419][20347] Updated weights for policy 0, policy_version 1630021 (0.0009) [2023-12-27 16:56:20,476][20347] Updated weights for policy 0, policy_version 1630031 (0.0008) [2023-12-27 16:56:20,872][20381] Updated weights for policy 1, policy_version 1628501 (0.0006) [2023-12-27 16:56:20,931][20381] Updated weights for policy 1, policy_version 1628511 (0.0008) [2023-12-27 16:56:20,988][20381] Updated weights for policy 1, policy_version 1628521 (0.0007) [2023-12-27 16:56:21,267][20347] Updated weights for policy 0, policy_version 1630041 (0.0009) [2023-12-27 16:56:21,331][20347] Updated weights for policy 0, policy_version 1630051 (0.0007) [2023-12-27 16:56:21,404][20347] Updated weights for policy 0, policy_version 1630061 (0.0009) [2023-12-27 16:56:21,671][20381] Updated weights for policy 1, policy_version 1628531 (0.0006) [2023-12-27 16:56:21,726][20381] Updated weights for policy 1, policy_version 1628541 (0.0008) [2023-12-27 16:56:21,791][20381] Updated weights for policy 1, policy_version 1628551 (0.0009) [2023-12-27 16:56:22,065][20347] Updated weights for policy 0, policy_version 1630071 (0.0007) [2023-12-27 16:56:22,128][20347] Updated weights for policy 0, policy_version 1630081 (0.0007) [2023-12-27 16:56:22,188][20347] Updated weights for policy 0, policy_version 1630091 (0.0008) [2023-12-27 16:56:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 834338816. Throughput: 0: 10143.6, 1: 10106.1. Samples: 834309460. Policy #0 lag: (min: 3.0, avg: 20.2, max: 35.0) [2023-12-27 16:56:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.890')] [2023-12-27 16:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001628560_416956416.pth... [2023-12-27 16:56:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001630096_417382400.pth... [2023-12-27 16:56:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001627376_416653312.pth [2023-12-27 16:56:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001628912_417079296.pth [2023-12-27 16:56:22,525][20381] Updated weights for policy 1, policy_version 1628561 (0.0009) [2023-12-27 16:56:22,591][20381] Updated weights for policy 1, policy_version 1628571 (0.0008) [2023-12-27 16:56:22,660][20381] Updated weights for policy 1, policy_version 1628581 (0.0009) [2023-12-27 16:56:22,729][20381] Updated weights for policy 1, policy_version 1628591 (0.0008) [2023-12-27 16:56:22,909][20347] Updated weights for policy 0, policy_version 1630101 (0.0008) [2023-12-27 16:56:22,967][20347] Updated weights for policy 0, policy_version 1630111 (0.0007) [2023-12-27 16:56:23,026][20347] Updated weights for policy 0, policy_version 1630121 (0.0008) [2023-12-27 16:56:23,435][20381] Updated weights for policy 1, policy_version 1628601 (0.0010) [2023-12-27 16:56:23,489][20381] Updated weights for policy 1, policy_version 1628611 (0.0010) [2023-12-27 16:56:23,541][20381] Updated weights for policy 1, policy_version 1628621 (0.0010) [2023-12-27 16:56:23,736][20347] Updated weights for policy 0, policy_version 1630131 (0.0008) [2023-12-27 16:56:23,787][20347] Updated weights for policy 0, policy_version 1630141 (0.0007) [2023-12-27 16:56:23,835][20347] Updated weights for policy 0, policy_version 1630151 (0.0007) [2023-12-27 16:56:24,217][20381] Updated weights for policy 1, policy_version 1628631 (0.0010) [2023-12-27 16:56:24,272][20381] Updated weights for policy 1, policy_version 1628641 (0.0010) [2023-12-27 16:56:24,320][20381] Updated weights for policy 1, policy_version 1628651 (0.0010) [2023-12-27 16:56:24,554][20347] Updated weights for policy 0, policy_version 1630161 (0.0007) [2023-12-27 16:56:24,610][20347] Updated weights for policy 0, policy_version 1630171 (0.0008) [2023-12-27 16:56:24,673][20347] Updated weights for policy 0, policy_version 1630181 (0.0008) [2023-12-27 16:56:24,735][20347] Updated weights for policy 0, policy_version 1630191 (0.0008) [2023-12-27 16:56:24,910][20381] Updated weights for policy 1, policy_version 1628661 (0.0010) [2023-12-27 16:56:24,968][20381] Updated weights for policy 1, policy_version 1628671 (0.0010) [2023-12-27 16:56:25,026][20381] Updated weights for policy 1, policy_version 1628681 (0.0011) [2023-12-27 16:56:25,443][20347] Updated weights for policy 0, policy_version 1630201 (0.0008) [2023-12-27 16:56:25,486][20347] Updated weights for policy 0, policy_version 1630211 (0.0007) [2023-12-27 16:56:25,544][20347] Updated weights for policy 0, policy_version 1630221 (0.0007) [2023-12-27 16:56:25,684][20381] Updated weights for policy 1, policy_version 1628691 (0.0009) [2023-12-27 16:56:25,741][20381] Updated weights for policy 1, policy_version 1628701 (0.0005) [2023-12-27 16:56:25,796][20381] Updated weights for policy 1, policy_version 1628711 (0.0005) [2023-12-27 16:56:26,272][20347] Updated weights for policy 0, policy_version 1630231 (0.0008) [2023-12-27 16:56:26,321][20347] Updated weights for policy 0, policy_version 1630241 (0.0007) [2023-12-27 16:56:26,372][20347] Updated weights for policy 0, policy_version 1630251 (0.0007) [2023-12-27 16:56:26,420][20381] Updated weights for policy 1, policy_version 1628721 (0.0007) [2023-12-27 16:56:26,479][20381] Updated weights for policy 1, policy_version 1628731 (0.0007) [2023-12-27 16:56:26,530][20381] Updated weights for policy 1, policy_version 1628741 (0.0007) [2023-12-27 16:56:26,575][20381] Updated weights for policy 1, policy_version 1628751 (0.0008) [2023-12-27 16:56:27,084][20347] Updated weights for policy 0, policy_version 1630261 (0.0008) [2023-12-27 16:56:27,140][20347] Updated weights for policy 0, policy_version 1630271 (0.0007) [2023-12-27 16:56:27,202][20347] Updated weights for policy 0, policy_version 1630281 (0.0008) [2023-12-27 16:56:27,261][20381] Updated weights for policy 1, policy_version 1628761 (0.0007) [2023-12-27 16:56:27,317][20381] Updated weights for policy 1, policy_version 1628771 (0.0007) [2023-12-27 16:56:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 834437120. Throughput: 0: 10081.8, 1: 10146.7. Samples: 834429960. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.740')] [2023-12-27 16:56:27,370][20381] Updated weights for policy 1, policy_version 1628781 (0.0007) [2023-12-27 16:56:27,902][20347] Updated weights for policy 0, policy_version 1630291 (0.0009) [2023-12-27 16:56:27,960][20347] Updated weights for policy 0, policy_version 1630301 (0.0007) [2023-12-27 16:56:28,002][20381] Updated weights for policy 1, policy_version 1628791 (0.0006) [2023-12-27 16:56:28,008][20347] Updated weights for policy 0, policy_version 1630311 (0.0007) [2023-12-27 16:56:28,059][20381] Updated weights for policy 1, policy_version 1628801 (0.0007) [2023-12-27 16:56:28,116][20381] Updated weights for policy 1, policy_version 1628811 (0.0008) [2023-12-27 16:56:28,739][20347] Updated weights for policy 0, policy_version 1630321 (0.0006) [2023-12-27 16:56:28,800][20347] Updated weights for policy 0, policy_version 1630331 (0.0007) [2023-12-27 16:56:28,801][20381] Updated weights for policy 1, policy_version 1628821 (0.0007) [2023-12-27 16:56:28,855][20347] Updated weights for policy 0, policy_version 1630341 (0.0007) [2023-12-27 16:56:28,857][20381] Updated weights for policy 1, policy_version 1628831 (0.0007) [2023-12-27 16:56:28,909][20381] Updated weights for policy 1, policy_version 1628841 (0.0007) [2023-12-27 16:56:28,914][20347] Updated weights for policy 0, policy_version 1630351 (0.0005) [2023-12-27 16:56:29,509][20381] Updated weights for policy 1, policy_version 1628851 (0.0006) [2023-12-27 16:56:29,563][20381] Updated weights for policy 1, policy_version 1628861 (0.0005) [2023-12-27 16:56:29,616][20347] Updated weights for policy 0, policy_version 1630361 (0.0010) [2023-12-27 16:56:29,617][20381] Updated weights for policy 1, policy_version 1628871 (0.0005) [2023-12-27 16:56:29,671][20347] Updated weights for policy 0, policy_version 1630371 (0.0010) [2023-12-27 16:56:29,730][20347] Updated weights for policy 0, policy_version 1630381 (0.0010) [2023-12-27 16:56:30,283][20381] Updated weights for policy 1, policy_version 1628881 (0.0006) [2023-12-27 16:56:30,342][20381] Updated weights for policy 1, policy_version 1628891 (0.0006) [2023-12-27 16:56:30,365][20347] Updated weights for policy 0, policy_version 1630391 (0.0010) [2023-12-27 16:56:30,398][20381] Updated weights for policy 1, policy_version 1628901 (0.0006) [2023-12-27 16:56:30,419][20347] Updated weights for policy 0, policy_version 1630401 (0.0006) [2023-12-27 16:56:30,453][20381] Updated weights for policy 1, policy_version 1628911 (0.0008) [2023-12-27 16:56:30,475][20347] Updated weights for policy 0, policy_version 1630411 (0.0005) [2023-12-27 16:56:31,000][20347] Updated weights for policy 0, policy_version 1630421 (0.0005) [2023-12-27 16:56:31,059][20347] Updated weights for policy 0, policy_version 1630431 (0.0010) [2023-12-27 16:56:31,118][20347] Updated weights for policy 0, policy_version 1630441 (0.0008) [2023-12-27 16:56:31,161][20381] Updated weights for policy 1, policy_version 1628921 (0.0011) [2023-12-27 16:56:31,217][20381] Updated weights for policy 1, policy_version 1628931 (0.0011) [2023-12-27 16:56:31,280][20381] Updated weights for policy 1, policy_version 1628941 (0.0010) [2023-12-27 16:56:31,870][20347] Updated weights for policy 0, policy_version 1630451 (0.0008) [2023-12-27 16:56:31,934][20347] Updated weights for policy 0, policy_version 1630461 (0.0008) [2023-12-27 16:56:31,997][20381] Updated weights for policy 1, policy_version 1628951 (0.0011) [2023-12-27 16:56:31,999][20347] Updated weights for policy 0, policy_version 1630471 (0.0008) [2023-12-27 16:56:32,055][20381] Updated weights for policy 1, policy_version 1628961 (0.0011) [2023-12-27 16:56:32,116][20381] Updated weights for policy 1, policy_version 1628971 (0.0011) [2023-12-27 16:56:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 834543616. Throughput: 0: 10052.6, 1: 10179.7. Samples: 834552424. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.510')] [2023-12-27 16:56:32,659][20347] Updated weights for policy 0, policy_version 1630481 (0.0008) [2023-12-27 16:56:32,723][20347] Updated weights for policy 0, policy_version 1630491 (0.0005) [2023-12-27 16:56:32,782][20347] Updated weights for policy 0, policy_version 1630501 (0.0005) [2023-12-27 16:56:32,798][20381] Updated weights for policy 1, policy_version 1628981 (0.0011) [2023-12-27 16:56:32,841][20347] Updated weights for policy 0, policy_version 1630511 (0.0006) [2023-12-27 16:56:32,853][20381] Updated weights for policy 1, policy_version 1628991 (0.0011) [2023-12-27 16:56:32,908][20381] Updated weights for policy 1, policy_version 1629001 (0.0007) [2023-12-27 16:56:33,452][20347] Updated weights for policy 0, policy_version 1630521 (0.0006) [2023-12-27 16:56:33,501][20347] Updated weights for policy 0, policy_version 1630531 (0.0007) [2023-12-27 16:56:33,552][20347] Updated weights for policy 0, policy_version 1630541 (0.0010) [2023-12-27 16:56:33,586][20381] Updated weights for policy 1, policy_version 1629011 (0.0011) [2023-12-27 16:56:33,634][20381] Updated weights for policy 1, policy_version 1629021 (0.0010) [2023-12-27 16:56:33,694][20381] Updated weights for policy 1, policy_version 1629031 (0.0010) [2023-12-27 16:56:34,199][20347] Updated weights for policy 0, policy_version 1630551 (0.0008) [2023-12-27 16:56:34,257][20347] Updated weights for policy 0, policy_version 1630561 (0.0007) [2023-12-27 16:56:34,319][20347] Updated weights for policy 0, policy_version 1630571 (0.0007) [2023-12-27 16:56:34,392][20381] Updated weights for policy 1, policy_version 1629041 (0.0010) [2023-12-27 16:56:34,447][20381] Updated weights for policy 1, policy_version 1629051 (0.0008) [2023-12-27 16:56:34,507][20381] Updated weights for policy 1, policy_version 1629061 (0.0008) [2023-12-27 16:56:34,564][20381] Updated weights for policy 1, policy_version 1629071 (0.0008) [2023-12-27 16:56:35,006][20347] Updated weights for policy 0, policy_version 1630581 (0.0008) [2023-12-27 16:56:35,057][20347] Updated weights for policy 0, policy_version 1630591 (0.0008) [2023-12-27 16:56:35,100][20347] Updated weights for policy 0, policy_version 1630601 (0.0007) [2023-12-27 16:56:35,225][20381] Updated weights for policy 1, policy_version 1629081 (0.0010) [2023-12-27 16:56:35,285][20381] Updated weights for policy 1, policy_version 1629091 (0.0011) [2023-12-27 16:56:35,348][20381] Updated weights for policy 1, policy_version 1629101 (0.0011) [2023-12-27 16:56:35,824][20347] Updated weights for policy 0, policy_version 1630611 (0.0007) [2023-12-27 16:56:35,872][20347] Updated weights for policy 0, policy_version 1630621 (0.0007) [2023-12-27 16:56:35,927][20347] Updated weights for policy 0, policy_version 1630631 (0.0007) [2023-12-27 16:56:36,033][20381] Updated weights for policy 1, policy_version 1629111 (0.0009) [2023-12-27 16:56:36,091][20381] Updated weights for policy 1, policy_version 1629121 (0.0007) [2023-12-27 16:56:36,156][20381] Updated weights for policy 1, policy_version 1629131 (0.0010) [2023-12-27 16:56:36,636][20347] Updated weights for policy 0, policy_version 1630641 (0.0008) [2023-12-27 16:56:36,693][20347] Updated weights for policy 0, policy_version 1630651 (0.0008) [2023-12-27 16:56:36,752][20347] Updated weights for policy 0, policy_version 1630661 (0.0005) [2023-12-27 16:56:36,808][20347] Updated weights for policy 0, policy_version 1630671 (0.0005) [2023-12-27 16:56:36,848][20381] Updated weights for policy 1, policy_version 1629141 (0.0009) [2023-12-27 16:56:36,906][20381] Updated weights for policy 1, policy_version 1629151 (0.0010) [2023-12-27 16:56:36,961][20381] Updated weights for policy 1, policy_version 1629161 (0.0010) [2023-12-27 16:56:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 834641920. Throughput: 0: 10057.2, 1: 10214.5. Samples: 834615144. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.610')] [2023-12-27 16:56:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001629168_417112064.pth... [2023-12-27 16:56:37,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001627984_416808960.pth [2023-12-27 16:56:37,406][20347] Updated weights for policy 0, policy_version 1630681 (0.0007) [2023-12-27 16:56:37,460][20347] Updated weights for policy 0, policy_version 1630691 (0.0007) [2023-12-27 16:56:37,510][20347] Updated weights for policy 0, policy_version 1630701 (0.0007) [2023-12-27 16:56:37,524][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001630704_417538048.pth... [2023-12-27 16:56:37,527][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001629520_417234944.pth [2023-12-27 16:56:37,579][20381] Updated weights for policy 1, policy_version 1629171 (0.0008) [2023-12-27 16:56:37,643][20381] Updated weights for policy 1, policy_version 1629181 (0.0009) [2023-12-27 16:56:37,708][20381] Updated weights for policy 1, policy_version 1629191 (0.0008) [2023-12-27 16:56:38,251][20347] Updated weights for policy 0, policy_version 1630711 (0.0008) [2023-12-27 16:56:38,315][20347] Updated weights for policy 0, policy_version 1630721 (0.0007) [2023-12-27 16:56:38,382][20347] Updated weights for policy 0, policy_version 1630731 (0.0008) [2023-12-27 16:56:38,406][20381] Updated weights for policy 1, policy_version 1629201 (0.0007) [2023-12-27 16:56:38,466][20381] Updated weights for policy 1, policy_version 1629211 (0.0011) [2023-12-27 16:56:38,526][20381] Updated weights for policy 1, policy_version 1629221 (0.0010) [2023-12-27 16:56:38,586][20381] Updated weights for policy 1, policy_version 1629231 (0.0011) [2023-12-27 16:56:39,028][20347] Updated weights for policy 0, policy_version 1630741 (0.0008) [2023-12-27 16:56:39,077][20347] Updated weights for policy 0, policy_version 1630751 (0.0009) [2023-12-27 16:56:39,127][20347] Updated weights for policy 0, policy_version 1630761 (0.0007) [2023-12-27 16:56:39,209][20381] Updated weights for policy 1, policy_version 1629241 (0.0006) [2023-12-27 16:56:39,269][20381] Updated weights for policy 1, policy_version 1629251 (0.0010) [2023-12-27 16:56:39,329][20381] Updated weights for policy 1, policy_version 1629261 (0.0011) [2023-12-27 16:56:39,791][20347] Updated weights for policy 0, policy_version 1630771 (0.0007) [2023-12-27 16:56:39,853][20347] Updated weights for policy 0, policy_version 1630781 (0.0007) [2023-12-27 16:56:39,905][20347] Updated weights for policy 0, policy_version 1630791 (0.0007) [2023-12-27 16:56:40,054][20381] Updated weights for policy 1, policy_version 1629271 (0.0008) [2023-12-27 16:56:40,116][20381] Updated weights for policy 1, policy_version 1629281 (0.0008) [2023-12-27 16:56:40,177][20381] Updated weights for policy 1, policy_version 1629291 (0.0008) [2023-12-27 16:56:40,600][20347] Updated weights for policy 0, policy_version 1630801 (0.0008) [2023-12-27 16:56:40,658][20347] Updated weights for policy 0, policy_version 1630811 (0.0007) [2023-12-27 16:56:40,708][20347] Updated weights for policy 0, policy_version 1630821 (0.0007) [2023-12-27 16:56:40,764][20347] Updated weights for policy 0, policy_version 1630831 (0.0007) [2023-12-27 16:56:40,901][20381] Updated weights for policy 1, policy_version 1629301 (0.0008) [2023-12-27 16:56:40,955][20381] Updated weights for policy 1, policy_version 1629311 (0.0007) [2023-12-27 16:56:41,015][20381] Updated weights for policy 1, policy_version 1629321 (0.0007) [2023-12-27 16:56:41,539][20347] Updated weights for policy 0, policy_version 1630841 (0.0007) [2023-12-27 16:56:41,600][20347] Updated weights for policy 0, policy_version 1630851 (0.0007) [2023-12-27 16:56:41,672][20347] Updated weights for policy 0, policy_version 1630861 (0.0006) [2023-12-27 16:56:41,797][20381] Updated weights for policy 1, policy_version 1629331 (0.0007) [2023-12-27 16:56:41,858][20381] Updated weights for policy 1, policy_version 1629341 (0.0008) [2023-12-27 16:56:41,917][20381] Updated weights for policy 1, policy_version 1629351 (0.0006) [2023-12-27 16:56:42,326][20347] Updated weights for policy 0, policy_version 1630871 (0.0007) [2023-12-27 16:56:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 834740224. Throughput: 0: 10053.2, 1: 10161.7. Samples: 834735324. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.440')] [2023-12-27 16:56:42,405][20347] Updated weights for policy 0, policy_version 1630881 (0.0009) [2023-12-27 16:56:42,470][20347] Updated weights for policy 0, policy_version 1630891 (0.0008) [2023-12-27 16:56:42,627][20381] Updated weights for policy 1, policy_version 1629361 (0.0009) [2023-12-27 16:56:42,683][20381] Updated weights for policy 1, policy_version 1629371 (0.0008) [2023-12-27 16:56:42,736][20381] Updated weights for policy 1, policy_version 1629381 (0.0007) [2023-12-27 16:56:42,791][20381] Updated weights for policy 1, policy_version 1629391 (0.0009) [2023-12-27 16:56:43,111][20347] Updated weights for policy 0, policy_version 1630901 (0.0008) [2023-12-27 16:56:43,156][20347] Updated weights for policy 0, policy_version 1630911 (0.0007) [2023-12-27 16:56:43,210][20347] Updated weights for policy 0, policy_version 1630921 (0.0007) [2023-12-27 16:56:43,495][20381] Updated weights for policy 1, policy_version 1629401 (0.0007) [2023-12-27 16:56:43,549][20381] Updated weights for policy 1, policy_version 1629411 (0.0007) [2023-12-27 16:56:43,595][20381] Updated weights for policy 1, policy_version 1629421 (0.0010) [2023-12-27 16:56:43,823][20347] Updated weights for policy 0, policy_version 1630931 (0.0007) [2023-12-27 16:56:43,879][20347] Updated weights for policy 0, policy_version 1630941 (0.0008) [2023-12-27 16:56:43,929][20347] Updated weights for policy 0, policy_version 1630951 (0.0007) [2023-12-27 16:56:44,280][20381] Updated weights for policy 1, policy_version 1629431 (0.0005) [2023-12-27 16:56:44,344][20381] Updated weights for policy 1, policy_version 1629441 (0.0006) [2023-12-27 16:56:44,409][20381] Updated weights for policy 1, policy_version 1629451 (0.0009) [2023-12-27 16:56:44,537][20347] Updated weights for policy 0, policy_version 1630961 (0.0005) [2023-12-27 16:56:44,589][20347] Updated weights for policy 0, policy_version 1630971 (0.0005) [2023-12-27 16:56:44,638][20347] Updated weights for policy 0, policy_version 1630981 (0.0007) [2023-12-27 16:56:44,688][20347] Updated weights for policy 0, policy_version 1630991 (0.0006) [2023-12-27 16:56:45,088][20381] Updated weights for policy 1, policy_version 1629461 (0.0011) [2023-12-27 16:56:45,149][20381] Updated weights for policy 1, policy_version 1629471 (0.0011) [2023-12-27 16:56:45,214][20381] Updated weights for policy 1, policy_version 1629481 (0.0010) [2023-12-27 16:56:45,300][20347] Updated weights for policy 0, policy_version 1631001 (0.0008) [2023-12-27 16:56:45,359][20347] Updated weights for policy 0, policy_version 1631011 (0.0011) [2023-12-27 16:56:45,415][20347] Updated weights for policy 0, policy_version 1631021 (0.0010) [2023-12-27 16:56:45,897][20381] Updated weights for policy 1, policy_version 1629491 (0.0011) [2023-12-27 16:56:45,955][20381] Updated weights for policy 1, policy_version 1629501 (0.0010) [2023-12-27 16:56:46,004][20347] Updated weights for policy 0, policy_version 1631031 (0.0009) [2023-12-27 16:56:46,013][20381] Updated weights for policy 1, policy_version 1629511 (0.0008) [2023-12-27 16:56:46,060][20347] Updated weights for policy 0, policy_version 1631041 (0.0007) [2023-12-27 16:56:46,110][20347] Updated weights for policy 0, policy_version 1631051 (0.0007) [2023-12-27 16:56:46,660][20381] Updated weights for policy 1, policy_version 1629521 (0.0011) [2023-12-27 16:56:46,663][20347] Updated weights for policy 0, policy_version 1631061 (0.0005) [2023-12-27 16:56:46,714][20381] Updated weights for policy 1, policy_version 1629531 (0.0009) [2023-12-27 16:56:46,717][20347] Updated weights for policy 0, policy_version 1631071 (0.0006) [2023-12-27 16:56:46,770][20381] Updated weights for policy 1, policy_version 1629541 (0.0010) [2023-12-27 16:56:46,775][20347] Updated weights for policy 0, policy_version 1631081 (0.0007) [2023-12-27 16:56:46,820][20381] Updated weights for policy 1, policy_version 1629551 (0.0011) [2023-12-27 16:56:47,354][20347] Updated weights for policy 0, policy_version 1631091 (0.0007) [2023-12-27 16:56:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 834846720. Throughput: 0: 10178.7, 1: 10168.2. Samples: 834861748. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.030')] [2023-12-27 16:56:47,410][20381] Updated weights for policy 1, policy_version 1629561 (0.0011) [2023-12-27 16:56:47,411][20347] Updated weights for policy 0, policy_version 1631101 (0.0007) [2023-12-27 16:56:47,463][20381] Updated weights for policy 1, policy_version 1629571 (0.0011) [2023-12-27 16:56:47,467][20347] Updated weights for policy 0, policy_version 1631111 (0.0008) [2023-12-27 16:56:47,520][20381] Updated weights for policy 1, policy_version 1629581 (0.0005) [2023-12-27 16:56:48,067][20381] Updated weights for policy 1, policy_version 1629591 (0.0008) [2023-12-27 16:56:48,123][20381] Updated weights for policy 1, policy_version 1629601 (0.0010) [2023-12-27 16:56:48,164][20347] Updated weights for policy 0, policy_version 1631121 (0.0008) [2023-12-27 16:56:48,181][20381] Updated weights for policy 1, policy_version 1629611 (0.0011) [2023-12-27 16:56:48,212][20347] Updated weights for policy 0, policy_version 1631131 (0.0006) [2023-12-27 16:56:48,267][20347] Updated weights for policy 0, policy_version 1631141 (0.0009) [2023-12-27 16:56:48,326][20347] Updated weights for policy 0, policy_version 1631151 (0.0007) [2023-12-27 16:56:48,873][20381] Updated weights for policy 1, policy_version 1629621 (0.0007) [2023-12-27 16:56:48,925][20381] Updated weights for policy 1, policy_version 1629631 (0.0005) [2023-12-27 16:56:48,974][20381] Updated weights for policy 1, policy_version 1629641 (0.0009) [2023-12-27 16:56:49,064][20347] Updated weights for policy 0, policy_version 1631161 (0.0010) [2023-12-27 16:56:49,123][20347] Updated weights for policy 0, policy_version 1631171 (0.0010) [2023-12-27 16:56:49,181][20347] Updated weights for policy 0, policy_version 1631181 (0.0010) [2023-12-27 16:56:49,594][20381] Updated weights for policy 1, policy_version 1629651 (0.0009) [2023-12-27 16:56:49,653][20381] Updated weights for policy 1, policy_version 1629661 (0.0006) [2023-12-27 16:56:49,710][20381] Updated weights for policy 1, policy_version 1629671 (0.0008) [2023-12-27 16:56:49,879][20347] Updated weights for policy 0, policy_version 1631191 (0.0008) [2023-12-27 16:56:49,948][20347] Updated weights for policy 0, policy_version 1631201 (0.0009) [2023-12-27 16:56:50,012][20347] Updated weights for policy 0, policy_version 1631211 (0.0010) [2023-12-27 16:56:50,420][20381] Updated weights for policy 1, policy_version 1629681 (0.0009) [2023-12-27 16:56:50,469][20381] Updated weights for policy 1, policy_version 1629691 (0.0011) [2023-12-27 16:56:50,521][20381] Updated weights for policy 1, policy_version 1629701 (0.0010) [2023-12-27 16:56:50,585][20381] Updated weights for policy 1, policy_version 1629711 (0.0011) [2023-12-27 16:56:50,670][20347] Updated weights for policy 0, policy_version 1631221 (0.0010) [2023-12-27 16:56:50,729][20347] Updated weights for policy 0, policy_version 1631231 (0.0010) [2023-12-27 16:56:50,784][20347] Updated weights for policy 0, policy_version 1631241 (0.0010) [2023-12-27 16:56:51,258][20381] Updated weights for policy 1, policy_version 1629721 (0.0011) [2023-12-27 16:56:51,319][20381] Updated weights for policy 1, policy_version 1629731 (0.0010) [2023-12-27 16:56:51,387][20381] Updated weights for policy 1, policy_version 1629741 (0.0011) [2023-12-27 16:56:51,446][20347] Updated weights for policy 0, policy_version 1631251 (0.0011) [2023-12-27 16:56:51,493][20347] Updated weights for policy 0, policy_version 1631261 (0.0008) [2023-12-27 16:56:51,544][20347] Updated weights for policy 0, policy_version 1631271 (0.0006) [2023-12-27 16:56:52,070][20381] Updated weights for policy 1, policy_version 1629751 (0.0010) [2023-12-27 16:56:52,126][20381] Updated weights for policy 1, policy_version 1629761 (0.0011) [2023-12-27 16:56:52,185][20381] Updated weights for policy 1, policy_version 1629771 (0.0011) [2023-12-27 16:56:52,211][20347] Updated weights for policy 0, policy_version 1631281 (0.0010) [2023-12-27 16:56:52,263][20347] Updated weights for policy 0, policy_version 1631291 (0.0010) [2023-12-27 16:56:52,320][20347] Updated weights for policy 0, policy_version 1631301 (0.0011) [2023-12-27 16:56:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 834953216. Throughput: 0: 10178.3, 1: 10201.0. Samples: 834924260. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.210')] [2023-12-27 16:56:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001629776_417267712.pth... [2023-12-27 16:56:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001628560_416956416.pth [2023-12-27 16:56:52,383][20347] Updated weights for policy 0, policy_version 1631311 (0.0010) [2023-12-27 16:56:52,386][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001631312_417693696.pth... [2023-12-27 16:56:52,389][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001630096_417382400.pth [2023-12-27 16:56:52,814][20381] Updated weights for policy 1, policy_version 1629781 (0.0011) [2023-12-27 16:56:52,872][20381] Updated weights for policy 1, policy_version 1629791 (0.0011) [2023-12-27 16:56:52,921][20381] Updated weights for policy 1, policy_version 1629801 (0.0010) [2023-12-27 16:56:53,075][20347] Updated weights for policy 0, policy_version 1631321 (0.0008) [2023-12-27 16:56:53,137][20347] Updated weights for policy 0, policy_version 1631331 (0.0007) [2023-12-27 16:56:53,198][20347] Updated weights for policy 0, policy_version 1631341 (0.0009) [2023-12-27 16:56:53,576][20381] Updated weights for policy 1, policy_version 1629811 (0.0008) [2023-12-27 16:56:53,625][20381] Updated weights for policy 1, policy_version 1629821 (0.0008) [2023-12-27 16:56:53,675][20381] Updated weights for policy 1, policy_version 1629831 (0.0007) [2023-12-27 16:56:53,909][20347] Updated weights for policy 0, policy_version 1631351 (0.0008) [2023-12-27 16:56:53,968][20347] Updated weights for policy 0, policy_version 1631361 (0.0008) [2023-12-27 16:56:54,016][20347] Updated weights for policy 0, policy_version 1631371 (0.0008) [2023-12-27 16:56:54,288][20381] Updated weights for policy 1, policy_version 1629841 (0.0008) [2023-12-27 16:56:54,342][20381] Updated weights for policy 1, policy_version 1629851 (0.0008) [2023-12-27 16:56:54,401][20381] Updated weights for policy 1, policy_version 1629861 (0.0007) [2023-12-27 16:56:54,462][20381] Updated weights for policy 1, policy_version 1629871 (0.0008) [2023-12-27 16:56:54,685][20347] Updated weights for policy 0, policy_version 1631381 (0.0009) [2023-12-27 16:56:54,741][20347] Updated weights for policy 0, policy_version 1631391 (0.0008) [2023-12-27 16:56:54,797][20347] Updated weights for policy 0, policy_version 1631401 (0.0008) [2023-12-27 16:56:55,184][20381] Updated weights for policy 1, policy_version 1629881 (0.0008) [2023-12-27 16:56:55,247][20381] Updated weights for policy 1, policy_version 1629891 (0.0007) [2023-12-27 16:56:55,307][20381] Updated weights for policy 1, policy_version 1629901 (0.0007) [2023-12-27 16:56:55,420][20347] Updated weights for policy 0, policy_version 1631411 (0.0007) [2023-12-27 16:56:55,463][20347] Updated weights for policy 0, policy_version 1631421 (0.0006) [2023-12-27 16:56:55,508][20347] Updated weights for policy 0, policy_version 1631431 (0.0006) [2023-12-27 16:56:55,953][20381] Updated weights for policy 1, policy_version 1629911 (0.0006) [2023-12-27 16:56:56,006][20381] Updated weights for policy 1, policy_version 1629921 (0.0005) [2023-12-27 16:56:56,060][20381] Updated weights for policy 1, policy_version 1629931 (0.0005) [2023-12-27 16:56:56,259][20347] Updated weights for policy 0, policy_version 1631441 (0.0007) [2023-12-27 16:56:56,304][20347] Updated weights for policy 0, policy_version 1631451 (0.0007) [2023-12-27 16:56:56,356][20347] Updated weights for policy 0, policy_version 1631461 (0.0007) [2023-12-27 16:56:56,409][20347] Updated weights for policy 0, policy_version 1631471 (0.0007) [2023-12-27 16:56:56,653][20381] Updated weights for policy 1, policy_version 1629941 (0.0008) [2023-12-27 16:56:56,698][20381] Updated weights for policy 1, policy_version 1629951 (0.0007) [2023-12-27 16:56:56,743][20381] Updated weights for policy 1, policy_version 1629961 (0.0007) [2023-12-27 16:56:57,085][20347] Updated weights for policy 0, policy_version 1631481 (0.0006) [2023-12-27 16:56:57,133][20347] Updated weights for policy 0, policy_version 1631491 (0.0005) [2023-12-27 16:56:57,182][20347] Updated weights for policy 0, policy_version 1631501 (0.0005) [2023-12-27 16:56:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 835059712. Throughput: 0: 10224.9, 1: 10246.7. Samples: 835047576. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:56:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.980')] [2023-12-27 16:56:57,468][20381] Updated weights for policy 1, policy_version 1629971 (0.0008) [2023-12-27 16:56:57,515][20381] Updated weights for policy 1, policy_version 1629981 (0.0007) [2023-12-27 16:56:57,556][20381] Updated weights for policy 1, policy_version 1629991 (0.0007) [2023-12-27 16:56:57,796][20347] Updated weights for policy 0, policy_version 1631511 (0.0008) [2023-12-27 16:56:57,842][20347] Updated weights for policy 0, policy_version 1631521 (0.0008) [2023-12-27 16:56:57,888][20347] Updated weights for policy 0, policy_version 1631531 (0.0006) [2023-12-27 16:56:58,300][20381] Updated weights for policy 1, policy_version 1630001 (0.0008) [2023-12-27 16:56:58,380][20381] Updated weights for policy 1, policy_version 1630011 (0.0009) [2023-12-27 16:56:58,443][20381] Updated weights for policy 1, policy_version 1630021 (0.0007) [2023-12-27 16:56:58,506][20381] Updated weights for policy 1, policy_version 1630031 (0.0008) [2023-12-27 16:56:58,674][20347] Updated weights for policy 0, policy_version 1631541 (0.0007) [2023-12-27 16:56:58,739][20347] Updated weights for policy 0, policy_version 1631551 (0.0008) [2023-12-27 16:56:58,803][20347] Updated weights for policy 0, policy_version 1631561 (0.0007) [2023-12-27 16:56:59,279][20381] Updated weights for policy 1, policy_version 1630041 (0.0008) [2023-12-27 16:56:59,334][20381] Updated weights for policy 1, policy_version 1630051 (0.0008) [2023-12-27 16:56:59,398][20381] Updated weights for policy 1, policy_version 1630061 (0.0008) [2023-12-27 16:56:59,450][20347] Updated weights for policy 0, policy_version 1631571 (0.0007) [2023-12-27 16:56:59,503][20347] Updated weights for policy 0, policy_version 1631581 (0.0008) [2023-12-27 16:56:59,555][20347] Updated weights for policy 0, policy_version 1631591 (0.0006) [2023-12-27 16:56:59,999][20381] Updated weights for policy 1, policy_version 1630071 (0.0008) [2023-12-27 16:57:00,054][20381] Updated weights for policy 1, policy_version 1630081 (0.0007) [2023-12-27 16:57:00,110][20381] Updated weights for policy 1, policy_version 1630091 (0.0006) [2023-12-27 16:57:00,323][20347] Updated weights for policy 0, policy_version 1631601 (0.0009) [2023-12-27 16:57:00,379][20347] Updated weights for policy 0, policy_version 1631611 (0.0007) [2023-12-27 16:57:00,428][20347] Updated weights for policy 0, policy_version 1631621 (0.0008) [2023-12-27 16:57:00,472][20347] Updated weights for policy 0, policy_version 1631631 (0.0009) [2023-12-27 16:57:00,680][20381] Updated weights for policy 1, policy_version 1630101 (0.0007) [2023-12-27 16:57:00,728][20381] Updated weights for policy 1, policy_version 1630111 (0.0008) [2023-12-27 16:57:00,776][20381] Updated weights for policy 1, policy_version 1630121 (0.0007) [2023-12-27 16:57:01,220][20347] Updated weights for policy 0, policy_version 1631641 (0.0008) [2023-12-27 16:57:01,276][20347] Updated weights for policy 0, policy_version 1631651 (0.0007) [2023-12-27 16:57:01,337][20347] Updated weights for policy 0, policy_version 1631661 (0.0007) [2023-12-27 16:57:01,436][20381] Updated weights for policy 1, policy_version 1630131 (0.0008) [2023-12-27 16:57:01,491][20381] Updated weights for policy 1, policy_version 1630141 (0.0007) [2023-12-27 16:57:01,545][20381] Updated weights for policy 1, policy_version 1630151 (0.0007) [2023-12-27 16:57:02,107][20347] Updated weights for policy 0, policy_version 1631671 (0.0008) [2023-12-27 16:57:02,163][20347] Updated weights for policy 0, policy_version 1631681 (0.0008) [2023-12-27 16:57:02,215][20347] Updated weights for policy 0, policy_version 1631691 (0.0007) [2023-12-27 16:57:02,237][20381] Updated weights for policy 1, policy_version 1630161 (0.0007) [2023-12-27 16:57:02,295][20381] Updated weights for policy 1, policy_version 1630171 (0.0007) [2023-12-27 16:57:02,357][20381] Updated weights for policy 1, policy_version 1630181 (0.0008) [2023-12-27 16:57:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 835158016. Throughput: 0: 10142.7, 1: 10301.4. Samples: 835169040. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.770')] [2023-12-27 16:57:02,413][20381] Updated weights for policy 1, policy_version 1630191 (0.0008) [2023-12-27 16:57:02,966][20347] Updated weights for policy 0, policy_version 1631701 (0.0006) [2023-12-27 16:57:03,024][20347] Updated weights for policy 0, policy_version 1631711 (0.0005) [2023-12-27 16:57:03,075][20347] Updated weights for policy 0, policy_version 1631721 (0.0005) [2023-12-27 16:57:03,135][20381] Updated weights for policy 1, policy_version 1630201 (0.0008) [2023-12-27 16:57:03,190][20381] Updated weights for policy 1, policy_version 1630211 (0.0009) [2023-12-27 16:57:03,252][20381] Updated weights for policy 1, policy_version 1630221 (0.0005) [2023-12-27 16:57:03,634][20347] Updated weights for policy 0, policy_version 1631731 (0.0006) [2023-12-27 16:57:03,689][20347] Updated weights for policy 0, policy_version 1631741 (0.0008) [2023-12-27 16:57:03,741][20347] Updated weights for policy 0, policy_version 1631751 (0.0008) [2023-12-27 16:57:03,797][20381] Updated weights for policy 1, policy_version 1630231 (0.0006) [2023-12-27 16:57:03,850][20381] Updated weights for policy 1, policy_version 1630241 (0.0008) [2023-12-27 16:57:03,915][20381] Updated weights for policy 1, policy_version 1630251 (0.0007) [2023-12-27 16:57:04,440][20347] Updated weights for policy 0, policy_version 1631761 (0.0006) [2023-12-27 16:57:04,499][20347] Updated weights for policy 0, policy_version 1631771 (0.0007) [2023-12-27 16:57:04,555][20347] Updated weights for policy 0, policy_version 1631781 (0.0007) [2023-12-27 16:57:04,613][20347] Updated weights for policy 0, policy_version 1631791 (0.0010) [2023-12-27 16:57:04,623][20381] Updated weights for policy 1, policy_version 1630261 (0.0007) [2023-12-27 16:57:04,681][20381] Updated weights for policy 1, policy_version 1630271 (0.0007) [2023-12-27 16:57:04,739][20381] Updated weights for policy 1, policy_version 1630281 (0.0008) [2023-12-27 16:57:05,229][20347] Updated weights for policy 0, policy_version 1631801 (0.0006) [2023-12-27 16:57:05,281][20347] Updated weights for policy 0, policy_version 1631811 (0.0005) [2023-12-27 16:57:05,332][20347] Updated weights for policy 0, policy_version 1631821 (0.0005) [2023-12-27 16:57:05,370][20381] Updated weights for policy 1, policy_version 1630291 (0.0008) [2023-12-27 16:57:05,416][20381] Updated weights for policy 1, policy_version 1630301 (0.0005) [2023-12-27 16:57:05,462][20381] Updated weights for policy 1, policy_version 1630311 (0.0005) [2023-12-27 16:57:05,966][20347] Updated weights for policy 0, policy_version 1631831 (0.0005) [2023-12-27 16:57:06,025][20347] Updated weights for policy 0, policy_version 1631841 (0.0005) [2023-12-27 16:57:06,074][20347] Updated weights for policy 0, policy_version 1631851 (0.0005) [2023-12-27 16:57:06,111][20381] Updated weights for policy 1, policy_version 1630321 (0.0006) [2023-12-27 16:57:06,174][20381] Updated weights for policy 1, policy_version 1630331 (0.0008) [2023-12-27 16:57:06,239][20381] Updated weights for policy 1, policy_version 1630341 (0.0007) [2023-12-27 16:57:06,303][20381] Updated weights for policy 1, policy_version 1630351 (0.0007) [2023-12-27 16:57:06,739][20347] Updated weights for policy 0, policy_version 1631861 (0.0008) [2023-12-27 16:57:06,798][20347] Updated weights for policy 0, policy_version 1631871 (0.0008) [2023-12-27 16:57:06,852][20347] Updated weights for policy 0, policy_version 1631881 (0.0007) [2023-12-27 16:57:06,991][20381] Updated weights for policy 1, policy_version 1630361 (0.0006) [2023-12-27 16:57:07,047][20381] Updated weights for policy 1, policy_version 1630371 (0.0007) [2023-12-27 16:57:07,094][20381] Updated weights for policy 1, policy_version 1630381 (0.0008) [2023-12-27 16:57:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 835264512. Throughput: 0: 10198.1, 1: 10295.9. Samples: 835231684. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.250')] [2023-12-27 16:57:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001631888_417841152.pth... [2023-12-27 16:57:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001630384_417423360.pth... [2023-12-27 16:57:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001629168_417112064.pth [2023-12-27 16:57:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001630704_417538048.pth [2023-12-27 16:57:07,498][20347] Updated weights for policy 0, policy_version 1631891 (0.0009) [2023-12-27 16:57:07,555][20347] Updated weights for policy 0, policy_version 1631901 (0.0010) [2023-12-27 16:57:07,616][20347] Updated weights for policy 0, policy_version 1631911 (0.0010) [2023-12-27 16:57:07,749][20381] Updated weights for policy 1, policy_version 1630391 (0.0008) [2023-12-27 16:57:07,799][20381] Updated weights for policy 1, policy_version 1630401 (0.0007) [2023-12-27 16:57:07,857][20381] Updated weights for policy 1, policy_version 1630411 (0.0006) [2023-12-27 16:57:08,277][20347] Updated weights for policy 0, policy_version 1631921 (0.0007) [2023-12-27 16:57:08,343][20347] Updated weights for policy 0, policy_version 1631931 (0.0008) [2023-12-27 16:57:08,402][20347] Updated weights for policy 0, policy_version 1631941 (0.0008) [2023-12-27 16:57:08,457][20347] Updated weights for policy 0, policy_version 1631951 (0.0009) [2023-12-27 16:57:08,542][20381] Updated weights for policy 1, policy_version 1630421 (0.0006) [2023-12-27 16:57:08,601][20381] Updated weights for policy 1, policy_version 1630431 (0.0007) [2023-12-27 16:57:08,652][20381] Updated weights for policy 1, policy_version 1630441 (0.0007) [2023-12-27 16:57:09,170][20347] Updated weights for policy 0, policy_version 1631961 (0.0008) [2023-12-27 16:57:09,236][20347] Updated weights for policy 0, policy_version 1631971 (0.0006) [2023-12-27 16:57:09,301][20347] Updated weights for policy 0, policy_version 1631981 (0.0006) [2023-12-27 16:57:09,407][20381] Updated weights for policy 1, policy_version 1630451 (0.0008) [2023-12-27 16:57:09,465][20381] Updated weights for policy 1, policy_version 1630461 (0.0008) [2023-12-27 16:57:09,523][20381] Updated weights for policy 1, policy_version 1630471 (0.0008) [2023-12-27 16:57:09,989][20347] Updated weights for policy 0, policy_version 1631991 (0.0009) [2023-12-27 16:57:10,042][20347] Updated weights for policy 0, policy_version 1632001 (0.0009) [2023-12-27 16:57:10,092][20347] Updated weights for policy 0, policy_version 1632011 (0.0009) [2023-12-27 16:57:10,311][20381] Updated weights for policy 1, policy_version 1630481 (0.0008) [2023-12-27 16:57:10,370][20381] Updated weights for policy 1, policy_version 1630491 (0.0006) [2023-12-27 16:57:10,431][20381] Updated weights for policy 1, policy_version 1630501 (0.0007) [2023-12-27 16:57:10,488][20381] Updated weights for policy 1, policy_version 1630511 (0.0008) [2023-12-27 16:57:10,775][20347] Updated weights for policy 0, policy_version 1632021 (0.0008) [2023-12-27 16:57:10,832][20347] Updated weights for policy 0, policy_version 1632031 (0.0006) [2023-12-27 16:57:10,889][20347] Updated weights for policy 0, policy_version 1632041 (0.0007) [2023-12-27 16:57:11,265][20381] Updated weights for policy 1, policy_version 1630521 (0.0008) [2023-12-27 16:57:11,335][20381] Updated weights for policy 1, policy_version 1630531 (0.0008) [2023-12-27 16:57:11,408][20381] Updated weights for policy 1, policy_version 1630541 (0.0008) [2023-12-27 16:57:11,539][20347] Updated weights for policy 0, policy_version 1632051 (0.0010) [2023-12-27 16:57:11,592][20347] Updated weights for policy 0, policy_version 1632061 (0.0010) [2023-12-27 16:57:11,657][20347] Updated weights for policy 0, policy_version 1632071 (0.0007) [2023-12-27 16:57:12,124][20381] Updated weights for policy 1, policy_version 1630551 (0.0007) [2023-12-27 16:57:12,184][20381] Updated weights for policy 1, policy_version 1630561 (0.0008) [2023-12-27 16:57:12,237][20381] Updated weights for policy 1, policy_version 1630571 (0.0009) [2023-12-27 16:57:12,330][20347] Updated weights for policy 0, policy_version 1632081 (0.0010) [2023-12-27 16:57:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 835362816. Throughput: 0: 10285.8, 1: 10221.5. Samples: 835352788. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.780')] [2023-12-27 16:57:12,397][20347] Updated weights for policy 0, policy_version 1632091 (0.0009) [2023-12-27 16:57:12,462][20347] Updated weights for policy 0, policy_version 1632101 (0.0008) [2023-12-27 16:57:12,523][20347] Updated weights for policy 0, policy_version 1632111 (0.0008) [2023-12-27 16:57:12,914][20381] Updated weights for policy 1, policy_version 1630581 (0.0008) [2023-12-27 16:57:12,963][20381] Updated weights for policy 1, policy_version 1630591 (0.0009) [2023-12-27 16:57:13,016][20381] Updated weights for policy 1, policy_version 1630601 (0.0009) [2023-12-27 16:57:13,175][20347] Updated weights for policy 0, policy_version 1632121 (0.0007) [2023-12-27 16:57:13,235][20347] Updated weights for policy 0, policy_version 1632131 (0.0009) [2023-12-27 16:57:13,286][20347] Updated weights for policy 0, policy_version 1632141 (0.0010) [2023-12-27 16:57:13,724][20381] Updated weights for policy 1, policy_version 1630611 (0.0008) [2023-12-27 16:57:13,790][20381] Updated weights for policy 1, policy_version 1630621 (0.0005) [2023-12-27 16:57:13,852][20381] Updated weights for policy 1, policy_version 1630631 (0.0008) [2023-12-27 16:57:13,886][20347] Updated weights for policy 0, policy_version 1632151 (0.0008) [2023-12-27 16:57:13,934][20347] Updated weights for policy 0, policy_version 1632161 (0.0005) [2023-12-27 16:57:13,980][20347] Updated weights for policy 0, policy_version 1632171 (0.0005) [2023-12-27 16:57:14,497][20381] Updated weights for policy 1, policy_version 1630641 (0.0007) [2023-12-27 16:57:14,561][20381] Updated weights for policy 1, policy_version 1630651 (0.0008) [2023-12-27 16:57:14,612][20381] Updated weights for policy 1, policy_version 1630661 (0.0005) [2023-12-27 16:57:14,614][20347] Updated weights for policy 0, policy_version 1632181 (0.0007) [2023-12-27 16:57:14,664][20381] Updated weights for policy 1, policy_version 1630671 (0.0006) [2023-12-27 16:57:14,673][20347] Updated weights for policy 0, policy_version 1632191 (0.0006) [2023-12-27 16:57:14,717][20347] Updated weights for policy 0, policy_version 1632201 (0.0005) [2023-12-27 16:57:15,325][20381] Updated weights for policy 1, policy_version 1630681 (0.0007) [2023-12-27 16:57:15,371][20347] Updated weights for policy 0, policy_version 1632211 (0.0007) [2023-12-27 16:57:15,381][20381] Updated weights for policy 1, policy_version 1630691 (0.0007) [2023-12-27 16:57:15,424][20347] Updated weights for policy 0, policy_version 1632221 (0.0008) [2023-12-27 16:57:15,432][20381] Updated weights for policy 1, policy_version 1630701 (0.0005) [2023-12-27 16:57:15,478][20347] Updated weights for policy 0, policy_version 1632231 (0.0008) [2023-12-27 16:57:16,080][20381] Updated weights for policy 1, policy_version 1630711 (0.0007) [2023-12-27 16:57:16,128][20381] Updated weights for policy 1, policy_version 1630721 (0.0007) [2023-12-27 16:57:16,164][20347] Updated weights for policy 0, policy_version 1632241 (0.0008) [2023-12-27 16:57:16,181][20381] Updated weights for policy 1, policy_version 1630731 (0.0007) [2023-12-27 16:57:16,193][20292] KL-divergence is very high: 108.3364 [2023-12-27 16:57:16,222][20347] Updated weights for policy 0, policy_version 1632251 (0.0007) [2023-12-27 16:57:16,280][20347] Updated weights for policy 0, policy_version 1632261 (0.0007) [2023-12-27 16:57:16,339][20347] Updated weights for policy 0, policy_version 1632271 (0.0007) [2023-12-27 16:57:16,815][20381] Updated weights for policy 1, policy_version 1630741 (0.0006) [2023-12-27 16:57:16,875][20381] Updated weights for policy 1, policy_version 1630751 (0.0008) [2023-12-27 16:57:16,937][20381] Updated weights for policy 1, policy_version 1630761 (0.0011) [2023-12-27 16:57:16,999][20347] Updated weights for policy 0, policy_version 1632281 (0.0005) [2023-12-27 16:57:17,043][20347] Updated weights for policy 0, policy_version 1632291 (0.0005) [2023-12-27 16:57:17,089][20347] Updated weights for policy 0, policy_version 1632301 (0.0005) [2023-12-27 16:57:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 835469312. Throughput: 0: 10338.4, 1: 10256.6. Samples: 835479196. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.320')] [2023-12-27 16:57:17,517][20381] Updated weights for policy 1, policy_version 1630771 (0.0010) [2023-12-27 16:57:17,568][20381] Updated weights for policy 1, policy_version 1630781 (0.0010) [2023-12-27 16:57:17,619][20381] Updated weights for policy 1, policy_version 1630791 (0.0010) [2023-12-27 16:57:17,726][20347] Updated weights for policy 0, policy_version 1632311 (0.0007) [2023-12-27 16:57:17,774][20347] Updated weights for policy 0, policy_version 1632321 (0.0007) [2023-12-27 16:57:17,827][20347] Updated weights for policy 0, policy_version 1632331 (0.0007) [2023-12-27 16:57:18,291][20381] Updated weights for policy 1, policy_version 1630801 (0.0010) [2023-12-27 16:57:18,356][20381] Updated weights for policy 1, policy_version 1630811 (0.0011) [2023-12-27 16:57:18,411][20381] Updated weights for policy 1, policy_version 1630821 (0.0010) [2023-12-27 16:57:18,465][20381] Updated weights for policy 1, policy_version 1630831 (0.0008) [2023-12-27 16:57:18,504][20347] Updated weights for policy 0, policy_version 1632341 (0.0008) [2023-12-27 16:57:18,556][20347] Updated weights for policy 0, policy_version 1632351 (0.0007) [2023-12-27 16:57:18,614][20347] Updated weights for policy 0, policy_version 1632361 (0.0007) [2023-12-27 16:57:19,112][20381] Updated weights for policy 1, policy_version 1630841 (0.0008) [2023-12-27 16:57:19,159][20381] Updated weights for policy 1, policy_version 1630851 (0.0007) [2023-12-27 16:57:19,201][20381] Updated weights for policy 1, policy_version 1630861 (0.0007) [2023-12-27 16:57:19,317][20347] Updated weights for policy 0, policy_version 1632371 (0.0006) [2023-12-27 16:57:19,381][20347] Updated weights for policy 0, policy_version 1632381 (0.0008) [2023-12-27 16:57:19,435][20347] Updated weights for policy 0, policy_version 1632391 (0.0009) [2023-12-27 16:57:20,038][20381] Updated weights for policy 1, policy_version 1630871 (0.0010) [2023-12-27 16:57:20,094][20381] Updated weights for policy 1, policy_version 1630881 (0.0011) [2023-12-27 16:57:20,158][20381] Updated weights for policy 1, policy_version 1630891 (0.0011) [2023-12-27 16:57:20,167][20347] Updated weights for policy 0, policy_version 1632401 (0.0009) [2023-12-27 16:57:20,222][20347] Updated weights for policy 0, policy_version 1632411 (0.0008) [2023-12-27 16:57:20,279][20347] Updated weights for policy 0, policy_version 1632421 (0.0008) [2023-12-27 16:57:20,339][20347] Updated weights for policy 0, policy_version 1632431 (0.0008) [2023-12-27 16:57:20,764][20381] Updated weights for policy 1, policy_version 1630901 (0.0008) [2023-12-27 16:57:20,826][20381] Updated weights for policy 1, policy_version 1630911 (0.0008) [2023-12-27 16:57:20,887][20381] Updated weights for policy 1, policy_version 1630921 (0.0007) [2023-12-27 16:57:21,085][20347] Updated weights for policy 0, policy_version 1632441 (0.0009) [2023-12-27 16:57:21,150][20347] Updated weights for policy 0, policy_version 1632451 (0.0008) [2023-12-27 16:57:21,203][20347] Updated weights for policy 0, policy_version 1632461 (0.0007) [2023-12-27 16:57:21,551][20381] Updated weights for policy 1, policy_version 1630931 (0.0006) [2023-12-27 16:57:21,611][20381] Updated weights for policy 1, policy_version 1630941 (0.0008) [2023-12-27 16:57:21,683][20381] Updated weights for policy 1, policy_version 1630951 (0.0008) [2023-12-27 16:57:21,948][20347] Updated weights for policy 0, policy_version 1632471 (0.0008) [2023-12-27 16:57:22,008][20347] Updated weights for policy 0, policy_version 1632481 (0.0008) [2023-12-27 16:57:22,061][20347] Updated weights for policy 0, policy_version 1632491 (0.0008) [2023-12-27 16:57:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 835567616. Throughput: 0: 10313.4, 1: 10240.2. Samples: 835540056. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.740')] [2023-12-27 16:57:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001632496_417996800.pth... [2023-12-27 16:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001630960_417570816.pth... [2023-12-27 16:57:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001629776_417267712.pth [2023-12-27 16:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001631312_417693696.pth [2023-12-27 16:57:22,391][20381] Updated weights for policy 1, policy_version 1630961 (0.0008) [2023-12-27 16:57:22,449][20381] Updated weights for policy 1, policy_version 1630971 (0.0006) [2023-12-27 16:57:22,503][20381] Updated weights for policy 1, policy_version 1630981 (0.0006) [2023-12-27 16:57:22,561][20381] Updated weights for policy 1, policy_version 1630991 (0.0007) [2023-12-27 16:57:22,834][20347] Updated weights for policy 0, policy_version 1632501 (0.0008) [2023-12-27 16:57:22,898][20347] Updated weights for policy 0, policy_version 1632511 (0.0008) [2023-12-27 16:57:22,965][20347] Updated weights for policy 0, policy_version 1632521 (0.0008) [2023-12-27 16:57:23,152][20381] Updated weights for policy 1, policy_version 1631001 (0.0007) [2023-12-27 16:57:23,217][20381] Updated weights for policy 1, policy_version 1631011 (0.0005) [2023-12-27 16:57:23,272][20381] Updated weights for policy 1, policy_version 1631021 (0.0005) [2023-12-27 16:57:23,599][20347] Updated weights for policy 0, policy_version 1632531 (0.0007) [2023-12-27 16:57:23,657][20347] Updated weights for policy 0, policy_version 1632541 (0.0006) [2023-12-27 16:57:23,707][20347] Updated weights for policy 0, policy_version 1632551 (0.0005) [2023-12-27 16:57:23,851][20381] Updated weights for policy 1, policy_version 1631031 (0.0007) [2023-12-27 16:57:23,897][20381] Updated weights for policy 1, policy_version 1631041 (0.0007) [2023-12-27 16:57:23,944][20381] Updated weights for policy 1, policy_version 1631051 (0.0008) [2023-12-27 16:57:24,282][20347] Updated weights for policy 0, policy_version 1632561 (0.0006) [2023-12-27 16:57:24,339][20347] Updated weights for policy 0, policy_version 1632571 (0.0005) [2023-12-27 16:57:24,395][20347] Updated weights for policy 0, policy_version 1632581 (0.0005) [2023-12-27 16:57:24,453][20347] Updated weights for policy 0, policy_version 1632591 (0.0007) [2023-12-27 16:57:24,667][20381] Updated weights for policy 1, policy_version 1631061 (0.0008) [2023-12-27 16:57:24,717][20381] Updated weights for policy 1, policy_version 1631071 (0.0008) [2023-12-27 16:57:24,766][20381] Updated weights for policy 1, policy_version 1631081 (0.0007) [2023-12-27 16:57:25,066][20347] Updated weights for policy 0, policy_version 1632601 (0.0006) [2023-12-27 16:57:25,110][20347] Updated weights for policy 0, policy_version 1632611 (0.0009) [2023-12-27 16:57:25,158][20347] Updated weights for policy 0, policy_version 1632621 (0.0007) [2023-12-27 16:57:25,391][20381] Updated weights for policy 1, policy_version 1631091 (0.0007) [2023-12-27 16:57:25,454][20381] Updated weights for policy 1, policy_version 1631101 (0.0007) [2023-12-27 16:57:25,512][20381] Updated weights for policy 1, policy_version 1631111 (0.0005) [2023-12-27 16:57:25,804][20347] Updated weights for policy 0, policy_version 1632631 (0.0006) [2023-12-27 16:57:25,854][20347] Updated weights for policy 0, policy_version 1632641 (0.0007) [2023-12-27 16:57:25,901][20347] Updated weights for policy 0, policy_version 1632651 (0.0005) [2023-12-27 16:57:26,070][20381] Updated weights for policy 1, policy_version 1631121 (0.0005) [2023-12-27 16:57:26,127][20381] Updated weights for policy 1, policy_version 1631131 (0.0007) [2023-12-27 16:57:26,184][20381] Updated weights for policy 1, policy_version 1631141 (0.0008) [2023-12-27 16:57:26,239][20381] Updated weights for policy 1, policy_version 1631151 (0.0006) [2023-12-27 16:57:26,503][20347] Updated weights for policy 0, policy_version 1632661 (0.0006) [2023-12-27 16:57:26,558][20347] Updated weights for policy 0, policy_version 1632671 (0.0005) [2023-12-27 16:57:26,627][20347] Updated weights for policy 0, policy_version 1632681 (0.0005) [2023-12-27 16:57:26,957][20381] Updated weights for policy 1, policy_version 1631161 (0.0008) [2023-12-27 16:57:27,021][20381] Updated weights for policy 1, policy_version 1631171 (0.0009) [2023-12-27 16:57:27,074][20381] Updated weights for policy 1, policy_version 1631181 (0.0007) [2023-12-27 16:57:27,144][20347] Updated weights for policy 0, policy_version 1632691 (0.0006) [2023-12-27 16:57:27,199][20347] Updated weights for policy 0, policy_version 1632701 (0.0007) [2023-12-27 16:57:27,254][20347] Updated weights for policy 0, policy_version 1632711 (0.0008) [2023-12-27 16:57:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 835682304. Throughput: 0: 10341.7, 1: 10338.9. Samples: 835665952. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.730')] [2023-12-27 16:57:27,735][20381] Updated weights for policy 1, policy_version 1631191 (0.0008) [2023-12-27 16:57:27,779][20381] Updated weights for policy 1, policy_version 1631201 (0.0009) [2023-12-27 16:57:27,823][20381] Updated weights for policy 1, policy_version 1631211 (0.0008) [2023-12-27 16:57:27,868][20347] Updated weights for policy 0, policy_version 1632721 (0.0007) [2023-12-27 16:57:27,926][20347] Updated weights for policy 0, policy_version 1632731 (0.0005) [2023-12-27 16:57:27,986][20347] Updated weights for policy 0, policy_version 1632741 (0.0005) [2023-12-27 16:57:28,045][20347] Updated weights for policy 0, policy_version 1632751 (0.0005) [2023-12-27 16:57:28,546][20381] Updated weights for policy 1, policy_version 1631221 (0.0008) [2023-12-27 16:57:28,581][20347] Updated weights for policy 0, policy_version 1632761 (0.0005) [2023-12-27 16:57:28,602][20381] Updated weights for policy 1, policy_version 1631231 (0.0007) [2023-12-27 16:57:28,635][20347] Updated weights for policy 0, policy_version 1632771 (0.0005) [2023-12-27 16:57:28,659][20381] Updated weights for policy 1, policy_version 1631241 (0.0008) [2023-12-27 16:57:28,691][20347] Updated weights for policy 0, policy_version 1632781 (0.0006) [2023-12-27 16:57:29,283][20347] Updated weights for policy 0, policy_version 1632791 (0.0008) [2023-12-27 16:57:29,342][20347] Updated weights for policy 0, policy_version 1632801 (0.0008) [2023-12-27 16:57:29,412][20347] Updated weights for policy 0, policy_version 1632811 (0.0009) [2023-12-27 16:57:29,431][20381] Updated weights for policy 1, policy_version 1631251 (0.0007) [2023-12-27 16:57:29,492][20381] Updated weights for policy 1, policy_version 1631261 (0.0007) [2023-12-27 16:57:29,560][20381] Updated weights for policy 1, policy_version 1631271 (0.0008) [2023-12-27 16:57:30,118][20347] Updated weights for policy 0, policy_version 1632821 (0.0006) [2023-12-27 16:57:30,147][20381] Updated weights for policy 1, policy_version 1631281 (0.0007) [2023-12-27 16:57:30,177][20347] Updated weights for policy 0, policy_version 1632831 (0.0007) [2023-12-27 16:57:30,211][20381] Updated weights for policy 1, policy_version 1631291 (0.0009) [2023-12-27 16:57:30,236][20347] Updated weights for policy 0, policy_version 1632841 (0.0007) [2023-12-27 16:57:30,271][20381] Updated weights for policy 1, policy_version 1631301 (0.0006) [2023-12-27 16:57:30,327][20381] Updated weights for policy 1, policy_version 1631311 (0.0008) [2023-12-27 16:57:30,895][20381] Updated weights for policy 1, policy_version 1631321 (0.0007) [2023-12-27 16:57:30,933][20347] Updated weights for policy 0, policy_version 1632851 (0.0006) [2023-12-27 16:57:30,942][20381] Updated weights for policy 1, policy_version 1631331 (0.0007) [2023-12-27 16:57:30,986][20347] Updated weights for policy 0, policy_version 1632861 (0.0007) [2023-12-27 16:57:30,988][20381] Updated weights for policy 1, policy_version 1631341 (0.0008) [2023-12-27 16:57:31,040][20347] Updated weights for policy 0, policy_version 1632871 (0.0006) [2023-12-27 16:57:31,748][20381] Updated weights for policy 1, policy_version 1631351 (0.0008) [2023-12-27 16:57:31,778][20347] Updated weights for policy 0, policy_version 1632881 (0.0010) [2023-12-27 16:57:31,811][20381] Updated weights for policy 1, policy_version 1631361 (0.0009) [2023-12-27 16:57:31,839][20347] Updated weights for policy 0, policy_version 1632891 (0.0010) [2023-12-27 16:57:31,867][20381] Updated weights for policy 1, policy_version 1631371 (0.0006) [2023-12-27 16:57:31,894][20347] Updated weights for policy 0, policy_version 1632901 (0.0010) [2023-12-27 16:57:31,946][20347] Updated weights for policy 0, policy_version 1632911 (0.0010) [2023-12-27 16:57:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 835780608. Throughput: 0: 10310.5, 1: 10351.9. Samples: 835791556. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.920')] [2023-12-27 16:57:32,557][20381] Updated weights for policy 1, policy_version 1631381 (0.0007) [2023-12-27 16:57:32,610][20381] Updated weights for policy 1, policy_version 1631391 (0.0008) [2023-12-27 16:57:32,656][20347] Updated weights for policy 0, policy_version 1632921 (0.0006) [2023-12-27 16:57:32,662][20381] Updated weights for policy 1, policy_version 1631401 (0.0008) [2023-12-27 16:57:32,713][20347] Updated weights for policy 0, policy_version 1632931 (0.0010) [2023-12-27 16:57:32,768][20347] Updated weights for policy 0, policy_version 1632941 (0.0010) [2023-12-27 16:57:33,381][20347] Updated weights for policy 0, policy_version 1632951 (0.0008) [2023-12-27 16:57:33,394][20381] Updated weights for policy 1, policy_version 1631411 (0.0006) [2023-12-27 16:57:33,444][20347] Updated weights for policy 0, policy_version 1632961 (0.0011) [2023-12-27 16:57:33,455][20381] Updated weights for policy 1, policy_version 1631421 (0.0006) [2023-12-27 16:57:33,499][20347] Updated weights for policy 0, policy_version 1632971 (0.0006) [2023-12-27 16:57:33,514][20381] Updated weights for policy 1, policy_version 1631431 (0.0007) [2023-12-27 16:57:34,041][20347] Updated weights for policy 0, policy_version 1632981 (0.0005) [2023-12-27 16:57:34,099][20347] Updated weights for policy 0, policy_version 1632991 (0.0006) [2023-12-27 16:57:34,150][20347] Updated weights for policy 0, policy_version 1633001 (0.0009) [2023-12-27 16:57:34,183][20381] Updated weights for policy 1, policy_version 1631441 (0.0009) [2023-12-27 16:57:34,235][20381] Updated weights for policy 1, policy_version 1631451 (0.0010) [2023-12-27 16:57:34,297][20381] Updated weights for policy 1, policy_version 1631461 (0.0010) [2023-12-27 16:57:34,355][20381] Updated weights for policy 1, policy_version 1631471 (0.0010) [2023-12-27 16:57:34,799][20347] Updated weights for policy 0, policy_version 1633011 (0.0010) [2023-12-27 16:57:34,844][20347] Updated weights for policy 0, policy_version 1633021 (0.0010) [2023-12-27 16:57:34,895][20347] Updated weights for policy 0, policy_version 1633031 (0.0010) [2023-12-27 16:57:35,052][20381] Updated weights for policy 1, policy_version 1631481 (0.0010) [2023-12-27 16:57:35,105][20381] Updated weights for policy 1, policy_version 1631491 (0.0007) [2023-12-27 16:57:35,163][20381] Updated weights for policy 1, policy_version 1631501 (0.0009) [2023-12-27 16:57:35,543][20347] Updated weights for policy 0, policy_version 1633041 (0.0010) [2023-12-27 16:57:35,604][20347] Updated weights for policy 0, policy_version 1633051 (0.0008) [2023-12-27 16:57:35,662][20347] Updated weights for policy 0, policy_version 1633061 (0.0005) [2023-12-27 16:57:35,713][20347] Updated weights for policy 0, policy_version 1633071 (0.0006) [2023-12-27 16:57:35,759][20381] Updated weights for policy 1, policy_version 1631511 (0.0005) [2023-12-27 16:57:35,807][20381] Updated weights for policy 1, policy_version 1631521 (0.0005) [2023-12-27 16:57:35,867][20381] Updated weights for policy 1, policy_version 1631531 (0.0008) [2023-12-27 16:57:36,399][20347] Updated weights for policy 0, policy_version 1633081 (0.0006) [2023-12-27 16:57:36,462][20347] Updated weights for policy 0, policy_version 1633091 (0.0008) [2023-12-27 16:57:36,500][20381] Updated weights for policy 1, policy_version 1631541 (0.0008) [2023-12-27 16:57:36,527][20347] Updated weights for policy 0, policy_version 1633101 (0.0007) [2023-12-27 16:57:36,562][20381] Updated weights for policy 1, policy_version 1631551 (0.0010) [2023-12-27 16:57:36,616][20381] Updated weights for policy 1, policy_version 1631561 (0.0011) [2023-12-27 16:57:37,147][20347] Updated weights for policy 0, policy_version 1633111 (0.0007) [2023-12-27 16:57:37,216][20347] Updated weights for policy 0, policy_version 1633121 (0.0008) [2023-12-27 16:57:37,269][20381] Updated weights for policy 1, policy_version 1631571 (0.0009) [2023-12-27 16:57:37,276][20347] Updated weights for policy 0, policy_version 1633131 (0.0009) [2023-12-27 16:57:37,332][20381] Updated weights for policy 1, policy_version 1631581 (0.0005) [2023-12-27 16:57:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 835887104. Throughput: 0: 10379.7, 1: 10285.0. Samples: 835854168. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.860')] [2023-12-27 16:57:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001633136_418160640.pth... [2023-12-27 16:57:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001631888_417841152.pth [2023-12-27 16:57:37,398][20381] Updated weights for policy 1, policy_version 1631591 (0.0006) [2023-12-27 16:57:37,456][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001631600_417734656.pth... [2023-12-27 16:57:37,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001630384_417423360.pth [2023-12-27 16:57:37,978][20381] Updated weights for policy 1, policy_version 1631601 (0.0006) [2023-12-27 16:57:37,992][20347] Updated weights for policy 0, policy_version 1633141 (0.0007) [2023-12-27 16:57:38,031][20381] Updated weights for policy 1, policy_version 1631611 (0.0007) [2023-12-27 16:57:38,045][20347] Updated weights for policy 0, policy_version 1633151 (0.0006) [2023-12-27 16:57:38,081][20381] Updated weights for policy 1, policy_version 1631621 (0.0006) [2023-12-27 16:57:38,102][20347] Updated weights for policy 0, policy_version 1633161 (0.0006) [2023-12-27 16:57:38,133][20381] Updated weights for policy 1, policy_version 1631631 (0.0006) [2023-12-27 16:57:38,787][20347] Updated weights for policy 0, policy_version 1633171 (0.0006) [2023-12-27 16:57:38,842][20347] Updated weights for policy 0, policy_version 1633181 (0.0007) [2023-12-27 16:57:38,848][20381] Updated weights for policy 1, policy_version 1631641 (0.0006) [2023-12-27 16:57:38,899][20347] Updated weights for policy 0, policy_version 1633191 (0.0006) [2023-12-27 16:57:38,902][20381] Updated weights for policy 1, policy_version 1631651 (0.0007) [2023-12-27 16:57:38,958][20381] Updated weights for policy 1, policy_version 1631661 (0.0006) [2023-12-27 16:57:39,622][20347] Updated weights for policy 0, policy_version 1633201 (0.0007) [2023-12-27 16:57:39,650][20381] Updated weights for policy 1, policy_version 1631671 (0.0007) [2023-12-27 16:57:39,676][20347] Updated weights for policy 0, policy_version 1633211 (0.0006) [2023-12-27 16:57:39,704][20381] Updated weights for policy 1, policy_version 1631681 (0.0006) [2023-12-27 16:57:39,726][20347] Updated weights for policy 0, policy_version 1633221 (0.0005) [2023-12-27 16:57:39,759][20381] Updated weights for policy 1, policy_version 1631691 (0.0007) [2023-12-27 16:57:39,790][20347] Updated weights for policy 0, policy_version 1633231 (0.0007) [2023-12-27 16:57:40,470][20381] Updated weights for policy 1, policy_version 1631701 (0.0007) [2023-12-27 16:57:40,512][20347] Updated weights for policy 0, policy_version 1633241 (0.0007) [2023-12-27 16:57:40,530][20381] Updated weights for policy 1, policy_version 1631711 (0.0007) [2023-12-27 16:57:40,569][20347] Updated weights for policy 0, policy_version 1633251 (0.0008) [2023-12-27 16:57:40,588][20381] Updated weights for policy 1, policy_version 1631721 (0.0006) [2023-12-27 16:57:40,631][20347] Updated weights for policy 0, policy_version 1633261 (0.0009) [2023-12-27 16:57:41,369][20381] Updated weights for policy 1, policy_version 1631731 (0.0009) [2023-12-27 16:57:41,432][20381] Updated weights for policy 1, policy_version 1631741 (0.0011) [2023-12-27 16:57:41,434][20347] Updated weights for policy 0, policy_version 1633271 (0.0009) [2023-12-27 16:57:41,490][20381] Updated weights for policy 1, policy_version 1631751 (0.0011) [2023-12-27 16:57:41,494][20347] Updated weights for policy 0, policy_version 1633281 (0.0006) [2023-12-27 16:57:41,558][20347] Updated weights for policy 0, policy_version 1633291 (0.0006) [2023-12-27 16:57:42,207][20347] Updated weights for policy 0, policy_version 1633301 (0.0008) [2023-12-27 16:57:42,213][20381] Updated weights for policy 1, policy_version 1631761 (0.0010) [2023-12-27 16:57:42,264][20381] Updated weights for policy 1, policy_version 1631771 (0.0006) [2023-12-27 16:57:42,272][20347] Updated weights for policy 0, policy_version 1633311 (0.0010) [2023-12-27 16:57:42,329][20381] Updated weights for policy 1, policy_version 1631781 (0.0008) [2023-12-27 16:57:42,339][20347] Updated weights for policy 0, policy_version 1633321 (0.0010) [2023-12-27 16:57:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 835977216. Throughput: 0: 10365.5, 1: 10289.3. Samples: 835977044. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.360')] [2023-12-27 16:57:42,397][20381] Updated weights for policy 1, policy_version 1631791 (0.0007) [2023-12-27 16:57:42,983][20347] Updated weights for policy 0, policy_version 1633331 (0.0008) [2023-12-27 16:57:43,047][20347] Updated weights for policy 0, policy_version 1633341 (0.0006) [2023-12-27 16:57:43,068][20381] Updated weights for policy 1, policy_version 1631801 (0.0010) [2023-12-27 16:57:43,105][20347] Updated weights for policy 0, policy_version 1633351 (0.0006) [2023-12-27 16:57:43,124][20381] Updated weights for policy 1, policy_version 1631811 (0.0010) [2023-12-27 16:57:43,176][20381] Updated weights for policy 1, policy_version 1631821 (0.0010) [2023-12-27 16:57:43,704][20347] Updated weights for policy 0, policy_version 1633361 (0.0006) [2023-12-27 16:57:43,765][20347] Updated weights for policy 0, policy_version 1633371 (0.0005) [2023-12-27 16:57:43,824][20347] Updated weights for policy 0, policy_version 1633381 (0.0005) [2023-12-27 16:57:43,868][20381] Updated weights for policy 1, policy_version 1631831 (0.0010) [2023-12-27 16:57:43,879][20347] Updated weights for policy 0, policy_version 1633391 (0.0005) [2023-12-27 16:57:43,914][20381] Updated weights for policy 1, policy_version 1631841 (0.0010) [2023-12-27 16:57:43,962][20381] Updated weights for policy 1, policy_version 1631851 (0.0010) [2023-12-27 16:57:44,397][20347] Updated weights for policy 0, policy_version 1633401 (0.0010) [2023-12-27 16:57:44,459][20347] Updated weights for policy 0, policy_version 1633411 (0.0010) [2023-12-27 16:57:44,513][20347] Updated weights for policy 0, policy_version 1633421 (0.0010) [2023-12-27 16:57:44,667][20381] Updated weights for policy 1, policy_version 1631861 (0.0010) [2023-12-27 16:57:44,723][20381] Updated weights for policy 1, policy_version 1631871 (0.0011) [2023-12-27 16:57:44,780][20381] Updated weights for policy 1, policy_version 1631881 (0.0009) [2023-12-27 16:57:45,247][20347] Updated weights for policy 0, policy_version 1633431 (0.0008) [2023-12-27 16:57:45,302][20347] Updated weights for policy 0, policy_version 1633441 (0.0007) [2023-12-27 16:57:45,355][20347] Updated weights for policy 0, policy_version 1633451 (0.0007) [2023-12-27 16:57:45,453][20381] Updated weights for policy 1, policy_version 1631891 (0.0010) [2023-12-27 16:57:45,504][20381] Updated weights for policy 1, policy_version 1631901 (0.0008) [2023-12-27 16:57:45,559][20381] Updated weights for policy 1, policy_version 1631911 (0.0006) [2023-12-27 16:57:45,946][20347] Updated weights for policy 0, policy_version 1633461 (0.0005) [2023-12-27 16:57:46,008][20347] Updated weights for policy 0, policy_version 1633471 (0.0005) [2023-12-27 16:57:46,075][20347] Updated weights for policy 0, policy_version 1633481 (0.0006) [2023-12-27 16:57:46,174][20381] Updated weights for policy 1, policy_version 1631921 (0.0010) [2023-12-27 16:57:46,233][20381] Updated weights for policy 1, policy_version 1631931 (0.0005) [2023-12-27 16:57:46,289][20381] Updated weights for policy 1, policy_version 1631941 (0.0005) [2023-12-27 16:57:46,341][20381] Updated weights for policy 1, policy_version 1631951 (0.0007) [2023-12-27 16:57:46,682][20347] Updated weights for policy 0, policy_version 1633491 (0.0007) [2023-12-27 16:57:46,730][20347] Updated weights for policy 0, policy_version 1633501 (0.0007) [2023-12-27 16:57:46,783][20347] Updated weights for policy 0, policy_version 1633511 (0.0007) [2023-12-27 16:57:46,949][20381] Updated weights for policy 1, policy_version 1631961 (0.0010) [2023-12-27 16:57:47,010][20381] Updated weights for policy 1, policy_version 1631971 (0.0010) [2023-12-27 16:57:47,061][20381] Updated weights for policy 1, policy_version 1631981 (0.0010) [2023-12-27 16:57:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 836091904. Throughput: 0: 10469.1, 1: 10280.7. Samples: 836102780. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:47,363][19170] Avg episode reward: [(0, '990.280'), (1, '931.950')] [2023-12-27 16:57:47,427][20347] Updated weights for policy 0, policy_version 1633521 (0.0007) [2023-12-27 16:57:47,480][20347] Updated weights for policy 0, policy_version 1633531 (0.0005) [2023-12-27 16:57:47,538][20347] Updated weights for policy 0, policy_version 1633541 (0.0008) [2023-12-27 16:57:47,594][20347] Updated weights for policy 0, policy_version 1633551 (0.0007) [2023-12-27 16:57:47,725][20381] Updated weights for policy 1, policy_version 1631991 (0.0010) [2023-12-27 16:57:47,776][20381] Updated weights for policy 1, policy_version 1632001 (0.0010) [2023-12-27 16:57:47,821][20381] Updated weights for policy 1, policy_version 1632011 (0.0010) [2023-12-27 16:57:48,215][20347] Updated weights for policy 0, policy_version 1633561 (0.0008) [2023-12-27 16:57:48,265][20347] Updated weights for policy 0, policy_version 1633571 (0.0008) [2023-12-27 16:57:48,315][20347] Updated weights for policy 0, policy_version 1633581 (0.0007) [2023-12-27 16:57:48,491][20381] Updated weights for policy 1, policy_version 1632021 (0.0010) [2023-12-27 16:57:48,553][20381] Updated weights for policy 1, policy_version 1632031 (0.0010) [2023-12-27 16:57:48,606][20381] Updated weights for policy 1, policy_version 1632041 (0.0010) [2023-12-27 16:57:48,979][20347] Updated weights for policy 0, policy_version 1633591 (0.0007) [2023-12-27 16:57:49,041][20347] Updated weights for policy 0, policy_version 1633601 (0.0007) [2023-12-27 16:57:49,102][20347] Updated weights for policy 0, policy_version 1633611 (0.0007) [2023-12-27 16:57:49,260][20381] Updated weights for policy 1, policy_version 1632051 (0.0010) [2023-12-27 16:57:49,323][20381] Updated weights for policy 1, policy_version 1632061 (0.0010) [2023-12-27 16:57:49,387][20381] Updated weights for policy 1, policy_version 1632071 (0.0011) [2023-12-27 16:57:49,803][20347] Updated weights for policy 0, policy_version 1633621 (0.0008) [2023-12-27 16:57:49,868][20347] Updated weights for policy 0, policy_version 1633631 (0.0007) [2023-12-27 16:57:49,930][20347] Updated weights for policy 0, policy_version 1633641 (0.0007) [2023-12-27 16:57:50,084][20381] Updated weights for policy 1, policy_version 1632081 (0.0008) [2023-12-27 16:57:50,144][20381] Updated weights for policy 1, policy_version 1632091 (0.0007) [2023-12-27 16:57:50,212][20381] Updated weights for policy 1, policy_version 1632101 (0.0007) [2023-12-27 16:57:50,262][20381] Updated weights for policy 1, policy_version 1632111 (0.0007) [2023-12-27 16:57:50,589][20347] Updated weights for policy 0, policy_version 1633651 (0.0008) [2023-12-27 16:57:50,648][20347] Updated weights for policy 0, policy_version 1633661 (0.0006) [2023-12-27 16:57:50,713][20347] Updated weights for policy 0, policy_version 1633671 (0.0007) [2023-12-27 16:57:50,998][20381] Updated weights for policy 1, policy_version 1632121 (0.0008) [2023-12-27 16:57:51,064][20381] Updated weights for policy 1, policy_version 1632131 (0.0008) [2023-12-27 16:57:51,125][20381] Updated weights for policy 1, policy_version 1632141 (0.0007) [2023-12-27 16:57:51,424][20347] Updated weights for policy 0, policy_version 1633681 (0.0008) [2023-12-27 16:57:51,479][20347] Updated weights for policy 0, policy_version 1633691 (0.0007) [2023-12-27 16:57:51,555][20347] Updated weights for policy 0, policy_version 1633701 (0.0009) [2023-12-27 16:57:51,611][20347] Updated weights for policy 0, policy_version 1633711 (0.0009) [2023-12-27 16:57:51,907][20381] Updated weights for policy 1, policy_version 1632151 (0.0008) [2023-12-27 16:57:51,959][20381] Updated weights for policy 1, policy_version 1632161 (0.0008) [2023-12-27 16:57:52,028][20381] Updated weights for policy 1, policy_version 1632171 (0.0008) [2023-12-27 16:57:52,317][20347] Updated weights for policy 0, policy_version 1633721 (0.0007) [2023-12-27 16:57:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 836190208. Throughput: 0: 10461.3, 1: 10272.9. Samples: 836164724. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:52,363][19170] Avg episode reward: [(0, '990.280'), (1, '912.460')] [2023-12-27 16:57:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001632176_417882112.pth... [2023-12-27 16:57:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001630960_417570816.pth [2023-12-27 16:57:52,379][20347] Updated weights for policy 0, policy_version 1633731 (0.0008) [2023-12-27 16:57:52,436][20347] Updated weights for policy 0, policy_version 1633741 (0.0007) [2023-12-27 16:57:52,454][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001633744_418316288.pth... [2023-12-27 16:57:52,458][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001632496_417996800.pth [2023-12-27 16:57:52,769][20381] Updated weights for policy 1, policy_version 1632181 (0.0008) [2023-12-27 16:57:52,829][20381] Updated weights for policy 1, policy_version 1632191 (0.0008) [2023-12-27 16:57:52,885][20381] Updated weights for policy 1, policy_version 1632201 (0.0008) [2023-12-27 16:57:53,118][20347] Updated weights for policy 0, policy_version 1633751 (0.0006) [2023-12-27 16:57:53,171][20347] Updated weights for policy 0, policy_version 1633761 (0.0006) [2023-12-27 16:57:53,220][20347] Updated weights for policy 0, policy_version 1633771 (0.0006) [2023-12-27 16:57:53,631][20381] Updated weights for policy 1, policy_version 1632211 (0.0007) [2023-12-27 16:57:53,692][20381] Updated weights for policy 1, policy_version 1632221 (0.0008) [2023-12-27 16:57:53,744][20381] Updated weights for policy 1, policy_version 1632231 (0.0010) [2023-12-27 16:57:53,856][20347] Updated weights for policy 0, policy_version 1633781 (0.0006) [2023-12-27 16:57:53,908][20347] Updated weights for policy 0, policy_version 1633791 (0.0007) [2023-12-27 16:57:53,962][20347] Updated weights for policy 0, policy_version 1633801 (0.0007) [2023-12-27 16:57:54,376][20381] Updated weights for policy 1, policy_version 1632241 (0.0010) [2023-12-27 16:57:54,435][20381] Updated weights for policy 1, policy_version 1632251 (0.0007) [2023-12-27 16:57:54,493][20381] Updated weights for policy 1, policy_version 1632261 (0.0011) [2023-12-27 16:57:54,552][20381] Updated weights for policy 1, policy_version 1632271 (0.0011) [2023-12-27 16:57:54,588][20347] Updated weights for policy 0, policy_version 1633811 (0.0007) [2023-12-27 16:57:54,644][20347] Updated weights for policy 0, policy_version 1633821 (0.0005) [2023-12-27 16:57:54,695][20347] Updated weights for policy 0, policy_version 1633831 (0.0005) [2023-12-27 16:57:55,198][20381] Updated weights for policy 1, policy_version 1632281 (0.0006) [2023-12-27 16:57:55,246][20381] Updated weights for policy 1, policy_version 1632291 (0.0010) [2023-12-27 16:57:55,293][20381] Updated weights for policy 1, policy_version 1632301 (0.0010) [2023-12-27 16:57:55,307][20347] Updated weights for policy 0, policy_version 1633841 (0.0006) [2023-12-27 16:57:55,357][20347] Updated weights for policy 0, policy_version 1633851 (0.0007) [2023-12-27 16:57:55,407][20347] Updated weights for policy 0, policy_version 1633861 (0.0007) [2023-12-27 16:57:55,457][20347] Updated weights for policy 0, policy_version 1633871 (0.0007) [2023-12-27 16:57:55,984][20381] Updated weights for policy 1, policy_version 1632311 (0.0008) [2023-12-27 16:57:56,029][20381] Updated weights for policy 1, policy_version 1632321 (0.0007) [2023-12-27 16:57:56,072][20381] Updated weights for policy 1, policy_version 1632331 (0.0007) [2023-12-27 16:57:56,101][20347] Updated weights for policy 0, policy_version 1633881 (0.0007) [2023-12-27 16:57:56,153][20347] Updated weights for policy 0, policy_version 1633891 (0.0010) [2023-12-27 16:57:56,198][20347] Updated weights for policy 0, policy_version 1633901 (0.0010) [2023-12-27 16:57:56,757][20381] Updated weights for policy 1, policy_version 1632341 (0.0009) [2023-12-27 16:57:56,799][20381] Updated weights for policy 1, policy_version 1632351 (0.0010) [2023-12-27 16:57:56,803][20347] Updated weights for policy 0, policy_version 1633911 (0.0007) [2023-12-27 16:57:56,843][20381] Updated weights for policy 1, policy_version 1632361 (0.0007) [2023-12-27 16:57:56,861][20347] Updated weights for policy 0, policy_version 1633921 (0.0005) [2023-12-27 16:57:56,922][20347] Updated weights for policy 0, policy_version 1633931 (0.0005) [2023-12-27 16:57:57,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 836296704. Throughput: 0: 10489.6, 1: 10254.9. Samples: 836286292. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:57:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.800')] [2023-12-27 16:57:57,462][20347] Updated weights for policy 0, policy_version 1633941 (0.0008) [2023-12-27 16:57:57,514][20347] Updated weights for policy 0, policy_version 1633951 (0.0007) [2023-12-27 16:57:57,521][20381] Updated weights for policy 1, policy_version 1632371 (0.0006) [2023-12-27 16:57:57,571][20381] Updated weights for policy 1, policy_version 1632381 (0.0005) [2023-12-27 16:57:57,571][20347] Updated weights for policy 0, policy_version 1633961 (0.0008) [2023-12-27 16:57:57,620][20381] Updated weights for policy 1, policy_version 1632391 (0.0006) [2023-12-27 16:57:58,185][20347] Updated weights for policy 0, policy_version 1633971 (0.0010) [2023-12-27 16:57:58,219][20381] Updated weights for policy 1, policy_version 1632401 (0.0010) [2023-12-27 16:57:58,246][20347] Updated weights for policy 0, policy_version 1633981 (0.0009) [2023-12-27 16:57:58,278][20381] Updated weights for policy 1, policy_version 1632411 (0.0009) [2023-12-27 16:57:58,303][20347] Updated weights for policy 0, policy_version 1633991 (0.0009) [2023-12-27 16:57:58,337][20381] Updated weights for policy 1, policy_version 1632421 (0.0008) [2023-12-27 16:57:58,396][20381] Updated weights for policy 1, policy_version 1632431 (0.0008) [2023-12-27 16:57:59,121][20347] Updated weights for policy 0, policy_version 1634001 (0.0008) [2023-12-27 16:57:59,154][20381] Updated weights for policy 1, policy_version 1632441 (0.0010) [2023-12-27 16:57:59,173][20347] Updated weights for policy 0, policy_version 1634011 (0.0008) [2023-12-27 16:57:59,211][20381] Updated weights for policy 1, policy_version 1632451 (0.0009) [2023-12-27 16:57:59,233][20347] Updated weights for policy 0, policy_version 1634021 (0.0008) [2023-12-27 16:57:59,282][20381] Updated weights for policy 1, policy_version 1632461 (0.0010) [2023-12-27 16:57:59,297][20347] Updated weights for policy 0, policy_version 1634031 (0.0007) [2023-12-27 16:57:59,930][20381] Updated weights for policy 1, policy_version 1632471 (0.0008) [2023-12-27 16:57:59,992][20381] Updated weights for policy 1, policy_version 1632481 (0.0008) [2023-12-27 16:58:00,025][20347] Updated weights for policy 0, policy_version 1634041 (0.0006) [2023-12-27 16:58:00,049][20381] Updated weights for policy 1, policy_version 1632491 (0.0009) [2023-12-27 16:58:00,079][20347] Updated weights for policy 0, policy_version 1634051 (0.0006) [2023-12-27 16:58:00,136][20347] Updated weights for policy 0, policy_version 1634061 (0.0006) [2023-12-27 16:58:00,673][20381] Updated weights for policy 1, policy_version 1632501 (0.0006) [2023-12-27 16:58:00,707][20347] Updated weights for policy 0, policy_version 1634071 (0.0008) [2023-12-27 16:58:00,720][20381] Updated weights for policy 1, policy_version 1632511 (0.0007) [2023-12-27 16:58:00,764][20347] Updated weights for policy 0, policy_version 1634081 (0.0007) [2023-12-27 16:58:00,770][20381] Updated weights for policy 1, policy_version 1632521 (0.0006) [2023-12-27 16:58:00,811][20347] Updated weights for policy 0, policy_version 1634091 (0.0007) [2023-12-27 16:58:01,417][20381] Updated weights for policy 1, policy_version 1632531 (0.0006) [2023-12-27 16:58:01,476][20381] Updated weights for policy 1, policy_version 1632541 (0.0005) [2023-12-27 16:58:01,499][20347] Updated weights for policy 0, policy_version 1634101 (0.0008) [2023-12-27 16:58:01,538][20381] Updated weights for policy 1, policy_version 1632551 (0.0006) [2023-12-27 16:58:01,554][20347] Updated weights for policy 0, policy_version 1634111 (0.0008) [2023-12-27 16:58:01,603][20347] Updated weights for policy 0, policy_version 1634121 (0.0008) [2023-12-27 16:58:02,226][20381] Updated weights for policy 1, policy_version 1632561 (0.0009) [2023-12-27 16:58:02,282][20381] Updated weights for policy 1, policy_version 1632571 (0.0008) [2023-12-27 16:58:02,341][20381] Updated weights for policy 1, policy_version 1632581 (0.0007) [2023-12-27 16:58:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 836395008. Throughput: 0: 10470.1, 1: 10252.6. Samples: 836411720. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:58:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.760')] [2023-12-27 16:58:02,367][20347] Updated weights for policy 0, policy_version 1634131 (0.0009) [2023-12-27 16:58:02,398][20381] Updated weights for policy 1, policy_version 1632591 (0.0006) [2023-12-27 16:58:02,421][20347] Updated weights for policy 0, policy_version 1634141 (0.0007) [2023-12-27 16:58:02,469][20347] Updated weights for policy 0, policy_version 1634151 (0.0007) [2023-12-27 16:58:03,061][20381] Updated weights for policy 1, policy_version 1632601 (0.0010) [2023-12-27 16:58:03,109][20381] Updated weights for policy 1, policy_version 1632611 (0.0010) [2023-12-27 16:58:03,142][20347] Updated weights for policy 0, policy_version 1634161 (0.0008) [2023-12-27 16:58:03,153][20381] Updated weights for policy 1, policy_version 1632621 (0.0010) [2023-12-27 16:58:03,194][20347] Updated weights for policy 0, policy_version 1634171 (0.0007) [2023-12-27 16:58:03,253][20347] Updated weights for policy 0, policy_version 1634181 (0.0005) [2023-12-27 16:58:03,319][20347] Updated weights for policy 0, policy_version 1634191 (0.0005) [2023-12-27 16:58:03,698][20381] Updated weights for policy 1, policy_version 1632631 (0.0006) [2023-12-27 16:58:03,755][20381] Updated weights for policy 1, policy_version 1632641 (0.0005) [2023-12-27 16:58:03,809][20381] Updated weights for policy 1, policy_version 1632651 (0.0005) [2023-12-27 16:58:03,942][20347] Updated weights for policy 0, policy_version 1634201 (0.0008) [2023-12-27 16:58:04,004][20347] Updated weights for policy 0, policy_version 1634211 (0.0012) [2023-12-27 16:58:04,058][20347] Updated weights for policy 0, policy_version 1634221 (0.0010) [2023-12-27 16:58:04,508][20381] Updated weights for policy 1, policy_version 1632661 (0.0008) [2023-12-27 16:58:04,564][20381] Updated weights for policy 1, policy_version 1632671 (0.0009) [2023-12-27 16:58:04,619][20381] Updated weights for policy 1, policy_version 1632681 (0.0007) [2023-12-27 16:58:04,741][20347] Updated weights for policy 0, policy_version 1634231 (0.0011) [2023-12-27 16:58:04,789][20347] Updated weights for policy 0, policy_version 1634241 (0.0010) [2023-12-27 16:58:04,843][20347] Updated weights for policy 0, policy_version 1634251 (0.0010) [2023-12-27 16:58:05,240][20381] Updated weights for policy 1, policy_version 1632691 (0.0009) [2023-12-27 16:58:05,291][20381] Updated weights for policy 1, policy_version 1632701 (0.0010) [2023-12-27 16:58:05,349][20381] Updated weights for policy 1, policy_version 1632711 (0.0010) [2023-12-27 16:58:05,498][20347] Updated weights for policy 0, policy_version 1634261 (0.0010) [2023-12-27 16:58:05,554][20347] Updated weights for policy 0, policy_version 1634271 (0.0007) [2023-12-27 16:58:05,615][20347] Updated weights for policy 0, policy_version 1634281 (0.0007) [2023-12-27 16:58:06,042][20381] Updated weights for policy 1, policy_version 1632721 (0.0010) [2023-12-27 16:58:06,092][20381] Updated weights for policy 1, policy_version 1632731 (0.0010) [2023-12-27 16:58:06,155][20381] Updated weights for policy 1, policy_version 1632741 (0.0009) [2023-12-27 16:58:06,197][20347] Updated weights for policy 0, policy_version 1634291 (0.0009) [2023-12-27 16:58:06,221][20381] Updated weights for policy 1, policy_version 1632751 (0.0010) [2023-12-27 16:58:06,258][20347] Updated weights for policy 0, policy_version 1634301 (0.0007) [2023-12-27 16:58:06,323][20347] Updated weights for policy 0, policy_version 1634311 (0.0007) [2023-12-27 16:58:06,913][20381] Updated weights for policy 1, policy_version 1632761 (0.0011) [2023-12-27 16:58:06,963][20381] Updated weights for policy 1, policy_version 1632771 (0.0011) [2023-12-27 16:58:07,020][20381] Updated weights for policy 1, policy_version 1632781 (0.0011) [2023-12-27 16:58:07,040][20347] Updated weights for policy 0, policy_version 1634321 (0.0007) [2023-12-27 16:58:07,094][20347] Updated weights for policy 0, policy_version 1634331 (0.0007) [2023-12-27 16:58:07,150][20347] Updated weights for policy 0, policy_version 1634341 (0.0007) [2023-12-27 16:58:07,213][20347] Updated weights for policy 0, policy_version 1634351 (0.0008) [2023-12-27 16:58:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 836509696. Throughput: 0: 10464.4, 1: 10308.1. Samples: 836474816. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:58:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.630')] [2023-12-27 16:58:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001632784_418037760.pth... [2023-12-27 16:58:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001634352_418471936.pth... [2023-12-27 16:58:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001631600_417734656.pth [2023-12-27 16:58:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001633136_418160640.pth [2023-12-27 16:58:07,700][20381] Updated weights for policy 1, policy_version 1632791 (0.0007) [2023-12-27 16:58:07,753][20381] Updated weights for policy 1, policy_version 1632801 (0.0005) [2023-12-27 16:58:07,800][20381] Updated weights for policy 1, policy_version 1632811 (0.0005) [2023-12-27 16:58:07,945][20347] Updated weights for policy 0, policy_version 1634361 (0.0009) [2023-12-27 16:58:08,004][20347] Updated weights for policy 0, policy_version 1634371 (0.0010) [2023-12-27 16:58:08,054][20347] Updated weights for policy 0, policy_version 1634381 (0.0010) [2023-12-27 16:58:08,446][20381] Updated weights for policy 1, policy_version 1632821 (0.0008) [2023-12-27 16:58:08,503][20381] Updated weights for policy 1, policy_version 1632831 (0.0005) [2023-12-27 16:58:08,568][20381] Updated weights for policy 1, policy_version 1632841 (0.0006) [2023-12-27 16:58:08,719][20347] Updated weights for policy 0, policy_version 1634391 (0.0010) [2023-12-27 16:58:08,774][20347] Updated weights for policy 0, policy_version 1634401 (0.0010) [2023-12-27 16:58:08,823][20347] Updated weights for policy 0, policy_version 1634411 (0.0010) [2023-12-27 16:58:09,166][20381] Updated weights for policy 1, policy_version 1632851 (0.0008) [2023-12-27 16:58:09,231][20381] Updated weights for policy 1, policy_version 1632861 (0.0007) [2023-12-27 16:58:09,291][20381] Updated weights for policy 1, policy_version 1632871 (0.0007) [2023-12-27 16:58:09,527][20347] Updated weights for policy 0, policy_version 1634421 (0.0009) [2023-12-27 16:58:09,582][20347] Updated weights for policy 0, policy_version 1634431 (0.0010) [2023-12-27 16:58:09,646][20347] Updated weights for policy 0, policy_version 1634441 (0.0010) [2023-12-27 16:58:09,991][20381] Updated weights for policy 1, policy_version 1632881 (0.0008) [2023-12-27 16:58:10,056][20381] Updated weights for policy 1, policy_version 1632891 (0.0006) [2023-12-27 16:58:10,107][20381] Updated weights for policy 1, policy_version 1632901 (0.0006) [2023-12-27 16:58:10,161][20381] Updated weights for policy 1, policy_version 1632911 (0.0006) [2023-12-27 16:58:10,361][20347] Updated weights for policy 0, policy_version 1634451 (0.0010) [2023-12-27 16:58:10,413][20347] Updated weights for policy 0, policy_version 1634461 (0.0008) [2023-12-27 16:58:10,474][20347] Updated weights for policy 0, policy_version 1634471 (0.0008) [2023-12-27 16:58:10,732][20381] Updated weights for policy 1, policy_version 1632921 (0.0007) [2023-12-27 16:58:10,783][20381] Updated weights for policy 1, policy_version 1632931 (0.0009) [2023-12-27 16:58:10,835][20381] Updated weights for policy 1, policy_version 1632941 (0.0006) [2023-12-27 16:58:11,207][20347] Updated weights for policy 0, policy_version 1634481 (0.0007) [2023-12-27 16:58:11,273][20347] Updated weights for policy 0, policy_version 1634491 (0.0008) [2023-12-27 16:58:11,342][20347] Updated weights for policy 0, policy_version 1634501 (0.0008) [2023-12-27 16:58:11,408][20347] Updated weights for policy 0, policy_version 1634511 (0.0007) [2023-12-27 16:58:11,517][20381] Updated weights for policy 1, policy_version 1632951 (0.0005) [2023-12-27 16:58:11,575][20381] Updated weights for policy 1, policy_version 1632961 (0.0008) [2023-12-27 16:58:11,639][20381] Updated weights for policy 1, policy_version 1632971 (0.0009) [2023-12-27 16:58:12,129][20347] Updated weights for policy 0, policy_version 1634521 (0.0008) [2023-12-27 16:58:12,191][20347] Updated weights for policy 0, policy_version 1634531 (0.0005) [2023-12-27 16:58:12,257][20347] Updated weights for policy 0, policy_version 1634541 (0.0008) [2023-12-27 16:58:12,350][20381] Updated weights for policy 1, policy_version 1632981 (0.0009) [2023-12-27 16:58:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 836608000. Throughput: 0: 10446.1, 1: 10268.1. Samples: 836598088. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:58:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.410')] [2023-12-27 16:58:12,425][20381] Updated weights for policy 1, policy_version 1632991 (0.0008) [2023-12-27 16:58:12,485][20381] Updated weights for policy 1, policy_version 1633001 (0.0008) [2023-12-27 16:58:12,947][20347] Updated weights for policy 0, policy_version 1634551 (0.0008) [2023-12-27 16:58:13,004][20347] Updated weights for policy 0, policy_version 1634561 (0.0008) [2023-12-27 16:58:13,059][20347] Updated weights for policy 0, policy_version 1634571 (0.0007) [2023-12-27 16:58:13,149][20381] Updated weights for policy 1, policy_version 1633011 (0.0006) [2023-12-27 16:58:13,198][20381] Updated weights for policy 1, policy_version 1633021 (0.0008) [2023-12-27 16:58:13,250][20381] Updated weights for policy 1, policy_version 1633031 (0.0008) [2023-12-27 16:58:13,807][20347] Updated weights for policy 0, policy_version 1634581 (0.0008) [2023-12-27 16:58:13,831][20381] Updated weights for policy 1, policy_version 1633041 (0.0007) [2023-12-27 16:58:13,864][20347] Updated weights for policy 0, policy_version 1634591 (0.0008) [2023-12-27 16:58:13,882][20381] Updated weights for policy 1, policy_version 1633051 (0.0005) [2023-12-27 16:58:13,917][20347] Updated weights for policy 0, policy_version 1634601 (0.0007) [2023-12-27 16:58:13,932][20381] Updated weights for policy 1, policy_version 1633061 (0.0005) [2023-12-27 16:58:13,977][20381] Updated weights for policy 1, policy_version 1633071 (0.0008) [2023-12-27 16:58:14,604][20347] Updated weights for policy 0, policy_version 1634611 (0.0008) [2023-12-27 16:58:14,661][20381] Updated weights for policy 1, policy_version 1633081 (0.0006) [2023-12-27 16:58:14,670][20347] Updated weights for policy 0, policy_version 1634621 (0.0006) [2023-12-27 16:58:14,713][20381] Updated weights for policy 1, policy_version 1633091 (0.0006) [2023-12-27 16:58:14,733][20347] Updated weights for policy 0, policy_version 1634631 (0.0007) [2023-12-27 16:58:14,799][20381] Updated weights for policy 1, policy_version 1633101 (0.0008) [2023-12-27 16:58:15,439][20381] Updated weights for policy 1, policy_version 1633111 (0.0006) [2023-12-27 16:58:15,466][20347] Updated weights for policy 0, policy_version 1634641 (0.0007) [2023-12-27 16:58:15,490][20381] Updated weights for policy 1, policy_version 1633121 (0.0006) [2023-12-27 16:58:15,528][20347] Updated weights for policy 0, policy_version 1634651 (0.0006) [2023-12-27 16:58:15,545][20381] Updated weights for policy 1, policy_version 1633131 (0.0005) [2023-12-27 16:58:15,588][20347] Updated weights for policy 0, policy_version 1634661 (0.0006) [2023-12-27 16:58:15,643][20347] Updated weights for policy 0, policy_version 1634671 (0.0008) [2023-12-27 16:58:16,161][20381] Updated weights for policy 1, policy_version 1633141 (0.0007) [2023-12-27 16:58:16,225][20381] Updated weights for policy 1, policy_version 1633151 (0.0008) [2023-12-27 16:58:16,288][20381] Updated weights for policy 1, policy_version 1633161 (0.0008) [2023-12-27 16:58:16,348][20347] Updated weights for policy 0, policy_version 1634681 (0.0006) [2023-12-27 16:58:16,398][20347] Updated weights for policy 0, policy_version 1634691 (0.0008) [2023-12-27 16:58:16,448][20347] Updated weights for policy 0, policy_version 1634701 (0.0009) [2023-12-27 16:58:16,888][20381] Updated weights for policy 1, policy_version 1633171 (0.0007) [2023-12-27 16:58:16,949][20381] Updated weights for policy 1, policy_version 1633181 (0.0006) [2023-12-27 16:58:16,999][20381] Updated weights for policy 1, policy_version 1633191 (0.0007) [2023-12-27 16:58:17,137][20347] Updated weights for policy 0, policy_version 1634711 (0.0009) [2023-12-27 16:58:17,185][20347] Updated weights for policy 0, policy_version 1634721 (0.0009) [2023-12-27 16:58:17,241][20347] Updated weights for policy 0, policy_version 1634731 (0.0009) [2023-12-27 16:58:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 836714496. Throughput: 0: 10302.0, 1: 10329.7. Samples: 836719980. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:58:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.650')] [2023-12-27 16:58:17,673][20381] Updated weights for policy 1, policy_version 1633201 (0.0007) [2023-12-27 16:58:17,728][20381] Updated weights for policy 1, policy_version 1633211 (0.0007) [2023-12-27 16:58:17,784][20381] Updated weights for policy 1, policy_version 1633221 (0.0007) [2023-12-27 16:58:17,840][20381] Updated weights for policy 1, policy_version 1633231 (0.0008) [2023-12-27 16:58:17,898][20347] Updated weights for policy 0, policy_version 1634741 (0.0007) [2023-12-27 16:58:17,945][20347] Updated weights for policy 0, policy_version 1634751 (0.0007) [2023-12-27 16:58:17,991][20347] Updated weights for policy 0, policy_version 1634761 (0.0010) [2023-12-27 16:58:18,533][20381] Updated weights for policy 1, policy_version 1633241 (0.0007) [2023-12-27 16:58:18,602][20381] Updated weights for policy 1, policy_version 1633251 (0.0008) [2023-12-27 16:58:18,663][20381] Updated weights for policy 1, policy_version 1633261 (0.0007) [2023-12-27 16:58:18,698][20347] Updated weights for policy 0, policy_version 1634771 (0.0008) [2023-12-27 16:58:18,759][20347] Updated weights for policy 0, policy_version 1634781 (0.0008) [2023-12-27 16:58:18,822][20347] Updated weights for policy 0, policy_version 1634791 (0.0009) [2023-12-27 16:58:19,356][20381] Updated weights for policy 1, policy_version 1633271 (0.0008) [2023-12-27 16:58:19,420][20381] Updated weights for policy 1, policy_version 1633281 (0.0008) [2023-12-27 16:58:19,478][20381] Updated weights for policy 1, policy_version 1633291 (0.0008) [2023-12-27 16:58:19,548][20347] Updated weights for policy 0, policy_version 1634801 (0.0008) [2023-12-27 16:58:19,609][20347] Updated weights for policy 0, policy_version 1634811 (0.0007) [2023-12-27 16:58:19,662][20347] Updated weights for policy 0, policy_version 1634821 (0.0008) [2023-12-27 16:58:19,722][20347] Updated weights for policy 0, policy_version 1634831 (0.0007) [2023-12-27 16:58:20,194][20381] Updated weights for policy 1, policy_version 1633301 (0.0007) [2023-12-27 16:58:20,249][20381] Updated weights for policy 1, policy_version 1633311 (0.0006) [2023-12-27 16:58:20,307][20381] Updated weights for policy 1, policy_version 1633321 (0.0006) [2023-12-27 16:58:20,472][20347] Updated weights for policy 0, policy_version 1634841 (0.0008) [2023-12-27 16:58:20,526][20347] Updated weights for policy 0, policy_version 1634851 (0.0008) [2023-12-27 16:58:20,593][20347] Updated weights for policy 0, policy_version 1634861 (0.0008) [2023-12-27 16:58:20,894][20381] Updated weights for policy 1, policy_version 1633331 (0.0007) [2023-12-27 16:58:20,947][20381] Updated weights for policy 1, policy_version 1633341 (0.0008) [2023-12-27 16:58:21,001][20381] Updated weights for policy 1, policy_version 1633351 (0.0007) [2023-12-27 16:58:21,337][20347] Updated weights for policy 0, policy_version 1634871 (0.0007) [2023-12-27 16:58:21,413][20347] Updated weights for policy 0, policy_version 1634881 (0.0008) [2023-12-27 16:58:21,474][20347] Updated weights for policy 0, policy_version 1634891 (0.0010) [2023-12-27 16:58:21,781][20381] Updated weights for policy 1, policy_version 1633361 (0.0009) [2023-12-27 16:58:21,842][20381] Updated weights for policy 1, policy_version 1633371 (0.0006) [2023-12-27 16:58:21,904][20381] Updated weights for policy 1, policy_version 1633381 (0.0009) [2023-12-27 16:58:21,965][20381] Updated weights for policy 1, policy_version 1633391 (0.0007) [2023-12-27 16:58:22,189][20347] Updated weights for policy 0, policy_version 1634901 (0.0010) [2023-12-27 16:58:22,247][20347] Updated weights for policy 0, policy_version 1634911 (0.0010) [2023-12-27 16:58:22,316][20347] Updated weights for policy 0, policy_version 1634921 (0.0007) [2023-12-27 16:58:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 836812800. Throughput: 0: 10239.5, 1: 10333.6. Samples: 836779960. Policy #0 lag: (min: 31.0, avg: 32.8, max: 63.0) [2023-12-27 16:58:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.530')] [2023-12-27 16:58:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001633392_418193408.pth... [2023-12-27 16:58:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001634928_418619392.pth... [2023-12-27 16:58:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001633744_418316288.pth [2023-12-27 16:58:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001632176_417882112.pth [2023-12-27 16:58:22,679][20381] Updated weights for policy 1, policy_version 1633401 (0.0011) [2023-12-27 16:58:22,742][20381] Updated weights for policy 1, policy_version 1633411 (0.0011) [2023-12-27 16:58:22,802][20381] Updated weights for policy 1, policy_version 1633421 (0.0011) [2023-12-27 16:58:23,053][20347] Updated weights for policy 0, policy_version 1634931 (0.0007) [2023-12-27 16:58:23,105][20347] Updated weights for policy 0, policy_version 1634941 (0.0006) [2023-12-27 16:58:23,159][20347] Updated weights for policy 0, policy_version 1634951 (0.0005) [2023-12-27 16:58:23,418][20381] Updated weights for policy 1, policy_version 1633431 (0.0007) [2023-12-27 16:58:23,479][20381] Updated weights for policy 1, policy_version 1633441 (0.0009) [2023-12-27 16:58:23,535][20381] Updated weights for policy 1, policy_version 1633451 (0.0008) [2023-12-27 16:58:23,729][20347] Updated weights for policy 0, policy_version 1634961 (0.0005) [2023-12-27 16:58:23,787][20347] Updated weights for policy 0, policy_version 1634971 (0.0007) [2023-12-27 16:58:23,839][20347] Updated weights for policy 0, policy_version 1634981 (0.0009) [2023-12-27 16:58:23,891][20347] Updated weights for policy 0, policy_version 1634991 (0.0005) [2023-12-27 16:58:24,263][20381] Updated weights for policy 1, policy_version 1633461 (0.0008) [2023-12-27 16:58:24,322][20381] Updated weights for policy 1, policy_version 1633471 (0.0006) [2023-12-27 16:58:24,378][20381] Updated weights for policy 1, policy_version 1633481 (0.0006) [2023-12-27 16:58:24,544][20347] Updated weights for policy 0, policy_version 1635001 (0.0006) [2023-12-27 16:58:24,613][20347] Updated weights for policy 0, policy_version 1635011 (0.0008) [2023-12-27 16:58:24,682][20347] Updated weights for policy 0, policy_version 1635021 (0.0005) [2023-12-27 16:58:24,992][20381] Updated weights for policy 1, policy_version 1633491 (0.0007) [2023-12-27 16:58:25,047][20381] Updated weights for policy 1, policy_version 1633501 (0.0006) [2023-12-27 16:58:25,098][20381] Updated weights for policy 1, policy_version 1633511 (0.0007) [2023-12-27 16:58:25,269][20347] Updated weights for policy 0, policy_version 1635031 (0.0005) [2023-12-27 16:58:25,316][20347] Updated weights for policy 0, policy_version 1635041 (0.0005) [2023-12-27 16:58:25,362][20347] Updated weights for policy 0, policy_version 1635051 (0.0005) [2023-12-27 16:58:25,751][20381] Updated weights for policy 1, policy_version 1633521 (0.0008) [2023-12-27 16:58:25,802][20381] Updated weights for policy 1, policy_version 1633531 (0.0007) [2023-12-27 16:58:25,855][20381] Updated weights for policy 1, policy_version 1633541 (0.0007) [2023-12-27 16:58:25,909][20381] Updated weights for policy 1, policy_version 1633551 (0.0010) [2023-12-27 16:58:25,986][20347] Updated weights for policy 0, policy_version 1635061 (0.0007) [2023-12-27 16:58:26,032][20347] Updated weights for policy 0, policy_version 1635071 (0.0008) [2023-12-27 16:58:26,079][20347] Updated weights for policy 0, policy_version 1635081 (0.0008) [2023-12-27 16:58:26,620][20381] Updated weights for policy 1, policy_version 1633561 (0.0008) [2023-12-27 16:58:26,677][20381] Updated weights for policy 1, policy_version 1633571 (0.0008) [2023-12-27 16:58:26,727][20381] Updated weights for policy 1, policy_version 1633581 (0.0007) [2023-12-27 16:58:26,732][20347] Updated weights for policy 0, policy_version 1635091 (0.0008) [2023-12-27 16:58:26,779][20347] Updated weights for policy 0, policy_version 1635101 (0.0006) [2023-12-27 16:58:26,822][20347] Updated weights for policy 0, policy_version 1635111 (0.0005) [2023-12-27 16:58:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 836911104. Throughput: 0: 10275.8, 1: 10310.2. Samples: 836903416. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.660')] [2023-12-27 16:58:27,425][20347] Updated weights for policy 0, policy_version 1635121 (0.0005) [2023-12-27 16:58:27,456][20381] Updated weights for policy 1, policy_version 1633591 (0.0006) [2023-12-27 16:58:27,482][20347] Updated weights for policy 0, policy_version 1635131 (0.0007) [2023-12-27 16:58:27,515][20381] Updated weights for policy 1, policy_version 1633601 (0.0007) [2023-12-27 16:58:27,535][20347] Updated weights for policy 0, policy_version 1635141 (0.0006) [2023-12-27 16:58:27,560][20381] Updated weights for policy 1, policy_version 1633611 (0.0005) [2023-12-27 16:58:27,584][20347] Updated weights for policy 0, policy_version 1635151 (0.0005) [2023-12-27 16:58:28,155][20381] Updated weights for policy 1, policy_version 1633621 (0.0005) [2023-12-27 16:58:28,206][20381] Updated weights for policy 1, policy_version 1633631 (0.0007) [2023-12-27 16:58:28,234][20347] Updated weights for policy 0, policy_version 1635161 (0.0005) [2023-12-27 16:58:28,262][20381] Updated weights for policy 1, policy_version 1633641 (0.0008) [2023-12-27 16:58:28,291][20347] Updated weights for policy 0, policy_version 1635171 (0.0006) [2023-12-27 16:58:28,354][20347] Updated weights for policy 0, policy_version 1635181 (0.0008) [2023-12-27 16:58:28,939][20347] Updated weights for policy 0, policy_version 1635191 (0.0010) [2023-12-27 16:58:28,988][20381] Updated weights for policy 1, policy_version 1633651 (0.0007) [2023-12-27 16:58:28,997][20347] Updated weights for policy 0, policy_version 1635201 (0.0010) [2023-12-27 16:58:29,042][20381] Updated weights for policy 1, policy_version 1633661 (0.0006) [2023-12-27 16:58:29,044][20347] Updated weights for policy 0, policy_version 1635211 (0.0010) [2023-12-27 16:58:29,101][20381] Updated weights for policy 1, policy_version 1633671 (0.0006) [2023-12-27 16:58:29,770][20381] Updated weights for policy 1, policy_version 1633681 (0.0007) [2023-12-27 16:58:29,805][20347] Updated weights for policy 0, policy_version 1635221 (0.0009) [2023-12-27 16:58:29,829][20381] Updated weights for policy 1, policy_version 1633691 (0.0006) [2023-12-27 16:58:29,872][20347] Updated weights for policy 0, policy_version 1635231 (0.0007) [2023-12-27 16:58:29,895][20381] Updated weights for policy 1, policy_version 1633701 (0.0007) [2023-12-27 16:58:29,925][20347] Updated weights for policy 0, policy_version 1635241 (0.0007) [2023-12-27 16:58:29,956][20381] Updated weights for policy 1, policy_version 1633711 (0.0006) [2023-12-27 16:58:30,554][20347] Updated weights for policy 0, policy_version 1635251 (0.0006) [2023-12-27 16:58:30,605][20347] Updated weights for policy 0, policy_version 1635261 (0.0005) [2023-12-27 16:58:30,615][20381] Updated weights for policy 1, policy_version 1633721 (0.0008) [2023-12-27 16:58:30,659][20347] Updated weights for policy 0, policy_version 1635271 (0.0008) [2023-12-27 16:58:30,671][20381] Updated weights for policy 1, policy_version 1633731 (0.0007) [2023-12-27 16:58:30,726][20381] Updated weights for policy 1, policy_version 1633741 (0.0006) [2023-12-27 16:58:31,257][20347] Updated weights for policy 0, policy_version 1635281 (0.0010) [2023-12-27 16:58:31,327][20347] Updated weights for policy 0, policy_version 1635291 (0.0006) [2023-12-27 16:58:31,391][20347] Updated weights for policy 0, policy_version 1635301 (0.0008) [2023-12-27 16:58:31,439][20381] Updated weights for policy 1, policy_version 1633751 (0.0008) [2023-12-27 16:58:31,452][20347] Updated weights for policy 0, policy_version 1635311 (0.0008) [2023-12-27 16:58:31,497][20381] Updated weights for policy 1, policy_version 1633761 (0.0008) [2023-12-27 16:58:31,561][20381] Updated weights for policy 1, policy_version 1633771 (0.0008) [2023-12-27 16:58:32,100][20347] Updated weights for policy 0, policy_version 1635321 (0.0005) [2023-12-27 16:58:32,153][20347] Updated weights for policy 0, policy_version 1635331 (0.0005) [2023-12-27 16:58:32,209][20347] Updated weights for policy 0, policy_version 1635341 (0.0005) [2023-12-27 16:58:32,273][20381] Updated weights for policy 1, policy_version 1633781 (0.0008) [2023-12-27 16:58:32,327][20381] Updated weights for policy 1, policy_version 1633791 (0.0007) [2023-12-27 16:58:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 837017600. Throughput: 0: 10296.2, 1: 10281.1. Samples: 837028760. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.870')] [2023-12-27 16:58:32,383][20381] Updated weights for policy 1, policy_version 1633801 (0.0007) [2023-12-27 16:58:32,780][20347] Updated weights for policy 0, policy_version 1635351 (0.0008) [2023-12-27 16:58:32,825][20347] Updated weights for policy 0, policy_version 1635361 (0.0010) [2023-12-27 16:58:32,878][20347] Updated weights for policy 0, policy_version 1635371 (0.0010) [2023-12-27 16:58:33,120][20381] Updated weights for policy 1, policy_version 1633811 (0.0008) [2023-12-27 16:58:33,168][20381] Updated weights for policy 1, policy_version 1633821 (0.0008) [2023-12-27 16:58:33,225][20381] Updated weights for policy 1, policy_version 1633831 (0.0009) [2023-12-27 16:58:33,490][20347] Updated weights for policy 0, policy_version 1635381 (0.0009) [2023-12-27 16:58:33,545][20347] Updated weights for policy 0, policy_version 1635391 (0.0007) [2023-12-27 16:58:33,602][20347] Updated weights for policy 0, policy_version 1635401 (0.0007) [2023-12-27 16:58:33,945][20381] Updated weights for policy 1, policy_version 1633841 (0.0008) [2023-12-27 16:58:33,999][20381] Updated weights for policy 1, policy_version 1633851 (0.0007) [2023-12-27 16:58:34,049][20381] Updated weights for policy 1, policy_version 1633861 (0.0009) [2023-12-27 16:58:34,108][20381] Updated weights for policy 1, policy_version 1633871 (0.0009) [2023-12-27 16:58:34,259][20347] Updated weights for policy 0, policy_version 1635411 (0.0007) [2023-12-27 16:58:34,318][20347] Updated weights for policy 0, policy_version 1635421 (0.0008) [2023-12-27 16:58:34,381][20347] Updated weights for policy 0, policy_version 1635431 (0.0008) [2023-12-27 16:58:34,858][20381] Updated weights for policy 1, policy_version 1633881 (0.0008) [2023-12-27 16:58:34,903][20381] Updated weights for policy 1, policy_version 1633891 (0.0008) [2023-12-27 16:58:34,963][20381] Updated weights for policy 1, policy_version 1633901 (0.0006) [2023-12-27 16:58:35,024][20347] Updated weights for policy 0, policy_version 1635441 (0.0008) [2023-12-27 16:58:35,073][20347] Updated weights for policy 0, policy_version 1635451 (0.0005) [2023-12-27 16:58:35,124][20347] Updated weights for policy 0, policy_version 1635461 (0.0006) [2023-12-27 16:58:35,181][20347] Updated weights for policy 0, policy_version 1635471 (0.0007) [2023-12-27 16:58:35,615][20381] Updated weights for policy 1, policy_version 1633911 (0.0007) [2023-12-27 16:58:35,670][20381] Updated weights for policy 1, policy_version 1633921 (0.0008) [2023-12-27 16:58:35,725][20381] Updated weights for policy 1, policy_version 1633931 (0.0007) [2023-12-27 16:58:35,860][20347] Updated weights for policy 0, policy_version 1635481 (0.0005) [2023-12-27 16:58:35,910][20347] Updated weights for policy 0, policy_version 1635491 (0.0005) [2023-12-27 16:58:35,960][20347] Updated weights for policy 0, policy_version 1635501 (0.0006) [2023-12-27 16:58:36,502][20381] Updated weights for policy 1, policy_version 1633941 (0.0007) [2023-12-27 16:58:36,564][20347] Updated weights for policy 0, policy_version 1635511 (0.0007) [2023-12-27 16:58:36,572][20381] Updated weights for policy 1, policy_version 1633951 (0.0006) [2023-12-27 16:58:36,624][20347] Updated weights for policy 0, policy_version 1635521 (0.0007) [2023-12-27 16:58:36,639][20381] Updated weights for policy 1, policy_version 1633961 (0.0007) [2023-12-27 16:58:36,683][20347] Updated weights for policy 0, policy_version 1635531 (0.0007) [2023-12-27 16:58:37,282][20381] Updated weights for policy 1, policy_version 1633971 (0.0007) [2023-12-27 16:58:37,336][20347] Updated weights for policy 0, policy_version 1635541 (0.0006) [2023-12-27 16:58:37,345][20381] Updated weights for policy 1, policy_version 1633981 (0.0007) [2023-12-27 16:58:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 837115904. Throughput: 0: 10333.0, 1: 10242.7. Samples: 837090628. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.650')] [2023-12-27 16:58:37,393][20347] Updated weights for policy 0, policy_version 1635551 (0.0008) [2023-12-27 16:58:37,407][20381] Updated weights for policy 1, policy_version 1633991 (0.0007) [2023-12-27 16:58:37,451][20347] Updated weights for policy 0, policy_version 1635561 (0.0010) [2023-12-27 16:58:37,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001634000_418349056.pth... [2023-12-27 16:58:37,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001632784_418037760.pth [2023-12-27 16:58:37,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001635568_418783232.pth... [2023-12-27 16:58:37,495][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001634352_418471936.pth [2023-12-27 16:58:38,056][20347] Updated weights for policy 0, policy_version 1635571 (0.0005) [2023-12-27 16:58:38,123][20347] Updated weights for policy 0, policy_version 1635581 (0.0010) [2023-12-27 16:58:38,142][20381] Updated weights for policy 1, policy_version 1634001 (0.0006) [2023-12-27 16:58:38,183][20347] Updated weights for policy 0, policy_version 1635591 (0.0010) [2023-12-27 16:58:38,205][20381] Updated weights for policy 1, policy_version 1634011 (0.0008) [2023-12-27 16:58:38,270][20381] Updated weights for policy 1, policy_version 1634021 (0.0007) [2023-12-27 16:58:38,333][20381] Updated weights for policy 1, policy_version 1634031 (0.0008) [2023-12-27 16:58:38,879][20347] Updated weights for policy 0, policy_version 1635601 (0.0008) [2023-12-27 16:58:38,940][20347] Updated weights for policy 0, policy_version 1635611 (0.0008) [2023-12-27 16:58:38,999][20347] Updated weights for policy 0, policy_version 1635621 (0.0007) [2023-12-27 16:58:39,050][20381] Updated weights for policy 1, policy_version 1634041 (0.0008) [2023-12-27 16:58:39,060][20347] Updated weights for policy 0, policy_version 1635631 (0.0006) [2023-12-27 16:58:39,110][20381] Updated weights for policy 1, policy_version 1634051 (0.0006) [2023-12-27 16:58:39,167][20381] Updated weights for policy 1, policy_version 1634061 (0.0008) [2023-12-27 16:58:39,744][20347] Updated weights for policy 0, policy_version 1635641 (0.0010) [2023-12-27 16:58:39,802][20347] Updated weights for policy 0, policy_version 1635651 (0.0010) [2023-12-27 16:58:39,863][20347] Updated weights for policy 0, policy_version 1635661 (0.0009) [2023-12-27 16:58:39,916][20381] Updated weights for policy 1, policy_version 1634071 (0.0008) [2023-12-27 16:58:39,991][20381] Updated weights for policy 1, policy_version 1634081 (0.0008) [2023-12-27 16:58:40,050][20381] Updated weights for policy 1, policy_version 1634091 (0.0006) [2023-12-27 16:58:40,613][20347] Updated weights for policy 0, policy_version 1635671 (0.0008) [2023-12-27 16:58:40,634][20381] Updated weights for policy 1, policy_version 1634101 (0.0007) [2023-12-27 16:58:40,659][20347] Updated weights for policy 0, policy_version 1635681 (0.0006) [2023-12-27 16:58:40,690][20381] Updated weights for policy 1, policy_version 1634111 (0.0007) [2023-12-27 16:58:40,716][20347] Updated weights for policy 0, policy_version 1635691 (0.0006) [2023-12-27 16:58:40,745][20381] Updated weights for policy 1, policy_version 1634121 (0.0006) [2023-12-27 16:58:41,454][20347] Updated weights for policy 0, policy_version 1635701 (0.0007) [2023-12-27 16:58:41,488][20381] Updated weights for policy 1, policy_version 1634131 (0.0008) [2023-12-27 16:58:41,514][20347] Updated weights for policy 0, policy_version 1635711 (0.0008) [2023-12-27 16:58:41,544][20381] Updated weights for policy 1, policy_version 1634141 (0.0007) [2023-12-27 16:58:41,570][20347] Updated weights for policy 0, policy_version 1635721 (0.0008) [2023-12-27 16:58:41,602][20381] Updated weights for policy 1, policy_version 1634151 (0.0008) [2023-12-27 16:58:42,287][20347] Updated weights for policy 0, policy_version 1635731 (0.0008) [2023-12-27 16:58:42,347][20347] Updated weights for policy 0, policy_version 1635741 (0.0007) [2023-12-27 16:58:42,363][20381] Updated weights for policy 1, policy_version 1634161 (0.0008) [2023-12-27 16:58:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 837214208. Throughput: 0: 10292.9, 1: 10260.6. Samples: 837211200. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.130')] [2023-12-27 16:58:42,412][20347] Updated weights for policy 0, policy_version 1635751 (0.0007) [2023-12-27 16:58:42,427][20381] Updated weights for policy 1, policy_version 1634171 (0.0007) [2023-12-27 16:58:42,487][20381] Updated weights for policy 1, policy_version 1634181 (0.0006) [2023-12-27 16:58:42,546][20381] Updated weights for policy 1, policy_version 1634191 (0.0007) [2023-12-27 16:58:43,082][20347] Updated weights for policy 0, policy_version 1635761 (0.0008) [2023-12-27 16:58:43,133][20347] Updated weights for policy 0, policy_version 1635771 (0.0010) [2023-12-27 16:58:43,143][20381] Updated weights for policy 1, policy_version 1634201 (0.0006) [2023-12-27 16:58:43,181][20347] Updated weights for policy 0, policy_version 1635781 (0.0010) [2023-12-27 16:58:43,194][20381] Updated weights for policy 1, policy_version 1634211 (0.0005) [2023-12-27 16:58:43,233][20347] Updated weights for policy 0, policy_version 1635791 (0.0010) [2023-12-27 16:58:43,254][20381] Updated weights for policy 1, policy_version 1634221 (0.0006) [2023-12-27 16:58:43,859][20347] Updated weights for policy 0, policy_version 1635801 (0.0006) [2023-12-27 16:58:43,923][20347] Updated weights for policy 0, policy_version 1635811 (0.0006) [2023-12-27 16:58:43,932][20381] Updated weights for policy 1, policy_version 1634231 (0.0006) [2023-12-27 16:58:43,972][20347] Updated weights for policy 0, policy_version 1635821 (0.0005) [2023-12-27 16:58:43,986][20381] Updated weights for policy 1, policy_version 1634241 (0.0006) [2023-12-27 16:58:44,048][20381] Updated weights for policy 1, policy_version 1634251 (0.0007) [2023-12-27 16:58:44,603][20347] Updated weights for policy 0, policy_version 1635831 (0.0007) [2023-12-27 16:58:44,651][20347] Updated weights for policy 0, policy_version 1635841 (0.0008) [2023-12-27 16:58:44,710][20347] Updated weights for policy 0, policy_version 1635851 (0.0008) [2023-12-27 16:58:44,751][20381] Updated weights for policy 1, policy_version 1634261 (0.0007) [2023-12-27 16:58:44,815][20381] Updated weights for policy 1, policy_version 1634271 (0.0007) [2023-12-27 16:58:44,875][20381] Updated weights for policy 1, policy_version 1634281 (0.0008) [2023-12-27 16:58:45,338][20347] Updated weights for policy 0, policy_version 1635861 (0.0005) [2023-12-27 16:58:45,389][20347] Updated weights for policy 0, policy_version 1635871 (0.0007) [2023-12-27 16:58:45,441][20347] Updated weights for policy 0, policy_version 1635881 (0.0007) [2023-12-27 16:58:45,588][20381] Updated weights for policy 1, policy_version 1634291 (0.0007) [2023-12-27 16:58:45,639][20381] Updated weights for policy 1, policy_version 1634301 (0.0008) [2023-12-27 16:58:45,689][20381] Updated weights for policy 1, policy_version 1634311 (0.0007) [2023-12-27 16:58:46,129][20347] Updated weights for policy 0, policy_version 1635891 (0.0008) [2023-12-27 16:58:46,176][20347] Updated weights for policy 0, policy_version 1635901 (0.0007) [2023-12-27 16:58:46,224][20347] Updated weights for policy 0, policy_version 1635911 (0.0009) [2023-12-27 16:58:46,352][20381] Updated weights for policy 1, policy_version 1634321 (0.0008) [2023-12-27 16:58:46,403][20381] Updated weights for policy 1, policy_version 1634331 (0.0009) [2023-12-27 16:58:46,455][20381] Updated weights for policy 1, policy_version 1634341 (0.0007) [2023-12-27 16:58:46,500][20381] Updated weights for policy 1, policy_version 1634351 (0.0008) [2023-12-27 16:58:46,797][20347] Updated weights for policy 0, policy_version 1635921 (0.0007) [2023-12-27 16:58:46,842][20347] Updated weights for policy 0, policy_version 1635931 (0.0008) [2023-12-27 16:58:46,899][20347] Updated weights for policy 0, policy_version 1635941 (0.0006) [2023-12-27 16:58:46,967][20347] Updated weights for policy 0, policy_version 1635951 (0.0006) [2023-12-27 16:58:47,153][20381] Updated weights for policy 1, policy_version 1634361 (0.0006) [2023-12-27 16:58:47,207][20381] Updated weights for policy 1, policy_version 1634371 (0.0007) [2023-12-27 16:58:47,267][20381] Updated weights for policy 1, policy_version 1634381 (0.0008) [2023-12-27 16:58:47,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 837328896. Throughput: 0: 10313.5, 1: 10216.1. Samples: 837335552. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.960')] [2023-12-27 16:58:47,685][20347] Updated weights for policy 0, policy_version 1635961 (0.0006) [2023-12-27 16:58:47,734][20347] Updated weights for policy 0, policy_version 1635971 (0.0005) [2023-12-27 16:58:47,790][20347] Updated weights for policy 0, policy_version 1635981 (0.0006) [2023-12-27 16:58:47,822][20381] Updated weights for policy 1, policy_version 1634391 (0.0006) [2023-12-27 16:58:47,877][20381] Updated weights for policy 1, policy_version 1634401 (0.0009) [2023-12-27 16:58:47,936][20381] Updated weights for policy 1, policy_version 1634411 (0.0011) [2023-12-27 16:58:48,406][20347] Updated weights for policy 0, policy_version 1635991 (0.0007) [2023-12-27 16:58:48,459][20347] Updated weights for policy 0, policy_version 1636001 (0.0007) [2023-12-27 16:58:48,515][20347] Updated weights for policy 0, policy_version 1636011 (0.0008) [2023-12-27 16:58:48,657][20381] Updated weights for policy 1, policy_version 1634421 (0.0010) [2023-12-27 16:58:48,718][20381] Updated weights for policy 1, policy_version 1634431 (0.0008) [2023-12-27 16:58:48,775][20381] Updated weights for policy 1, policy_version 1634441 (0.0008) [2023-12-27 16:58:49,155][20347] Updated weights for policy 0, policy_version 1636021 (0.0006) [2023-12-27 16:58:49,218][20347] Updated weights for policy 0, policy_version 1636031 (0.0007) [2023-12-27 16:58:49,280][20347] Updated weights for policy 0, policy_version 1636041 (0.0008) [2023-12-27 16:58:49,492][20381] Updated weights for policy 1, policy_version 1634451 (0.0007) [2023-12-27 16:58:49,559][20381] Updated weights for policy 1, policy_version 1634461 (0.0006) [2023-12-27 16:58:49,621][20381] Updated weights for policy 1, policy_version 1634471 (0.0006) [2023-12-27 16:58:49,994][20347] Updated weights for policy 0, policy_version 1636051 (0.0008) [2023-12-27 16:58:50,053][20347] Updated weights for policy 0, policy_version 1636061 (0.0006) [2023-12-27 16:58:50,108][20347] Updated weights for policy 0, policy_version 1636071 (0.0007) [2023-12-27 16:58:50,204][20381] Updated weights for policy 1, policy_version 1634481 (0.0006) [2023-12-27 16:58:50,266][20381] Updated weights for policy 1, policy_version 1634491 (0.0008) [2023-12-27 16:58:50,323][20381] Updated weights for policy 1, policy_version 1634501 (0.0008) [2023-12-27 16:58:50,376][20381] Updated weights for policy 1, policy_version 1634511 (0.0008) [2023-12-27 16:58:50,803][20347] Updated weights for policy 0, policy_version 1636081 (0.0009) [2023-12-27 16:58:50,857][20347] Updated weights for policy 0, policy_version 1636091 (0.0008) [2023-12-27 16:58:50,916][20347] Updated weights for policy 0, policy_version 1636101 (0.0008) [2023-12-27 16:58:50,973][20347] Updated weights for policy 0, policy_version 1636111 (0.0008) [2023-12-27 16:58:51,073][20381] Updated weights for policy 1, policy_version 1634521 (0.0009) [2023-12-27 16:58:51,130][20381] Updated weights for policy 1, policy_version 1634531 (0.0011) [2023-12-27 16:58:51,195][20381] Updated weights for policy 1, policy_version 1634541 (0.0008) [2023-12-27 16:58:51,672][20347] Updated weights for policy 0, policy_version 1636121 (0.0009) [2023-12-27 16:58:51,733][20347] Updated weights for policy 0, policy_version 1636131 (0.0007) [2023-12-27 16:58:51,793][20347] Updated weights for policy 0, policy_version 1636141 (0.0006) [2023-12-27 16:58:51,948][20381] Updated weights for policy 1, policy_version 1634551 (0.0011) [2023-12-27 16:58:52,004][20381] Updated weights for policy 1, policy_version 1634561 (0.0010) [2023-12-27 16:58:52,065][20381] Updated weights for policy 1, policy_version 1634571 (0.0007) [2023-12-27 16:58:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 837427200. Throughput: 0: 10323.3, 1: 10208.2. Samples: 837398732. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.210')] [2023-12-27 16:58:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001636144_418930688.pth... [2023-12-27 16:58:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001634576_418496512.pth... [2023-12-27 16:58:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001634928_418619392.pth [2023-12-27 16:58:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001633392_418193408.pth [2023-12-27 16:58:52,443][20347] Updated weights for policy 0, policy_version 1636151 (0.0006) [2023-12-27 16:58:52,499][20347] Updated weights for policy 0, policy_version 1636161 (0.0006) [2023-12-27 16:58:52,552][20347] Updated weights for policy 0, policy_version 1636171 (0.0008) [2023-12-27 16:58:52,765][20381] Updated weights for policy 1, policy_version 1634581 (0.0009) [2023-12-27 16:58:52,819][20381] Updated weights for policy 1, policy_version 1634591 (0.0008) [2023-12-27 16:58:52,875][20381] Updated weights for policy 1, policy_version 1634601 (0.0007) [2023-12-27 16:58:53,232][20347] Updated weights for policy 0, policy_version 1636181 (0.0008) [2023-12-27 16:58:53,287][20347] Updated weights for policy 0, policy_version 1636191 (0.0006) [2023-12-27 16:58:53,336][20347] Updated weights for policy 0, policy_version 1636201 (0.0010) [2023-12-27 16:58:53,565][20381] Updated weights for policy 1, policy_version 1634611 (0.0008) [2023-12-27 16:58:53,623][20381] Updated weights for policy 1, policy_version 1634621 (0.0006) [2023-12-27 16:58:53,674][20381] Updated weights for policy 1, policy_version 1634631 (0.0005) [2023-12-27 16:58:53,887][20347] Updated weights for policy 0, policy_version 1636211 (0.0010) [2023-12-27 16:58:53,936][20347] Updated weights for policy 0, policy_version 1636221 (0.0010) [2023-12-27 16:58:53,995][20347] Updated weights for policy 0, policy_version 1636231 (0.0010) [2023-12-27 16:58:54,296][20381] Updated weights for policy 1, policy_version 1634641 (0.0006) [2023-12-27 16:58:54,358][20381] Updated weights for policy 1, policy_version 1634651 (0.0009) [2023-12-27 16:58:54,410][20381] Updated weights for policy 1, policy_version 1634661 (0.0009) [2023-12-27 16:58:54,472][20381] Updated weights for policy 1, policy_version 1634671 (0.0011) [2023-12-27 16:58:54,636][20347] Updated weights for policy 0, policy_version 1636241 (0.0008) [2023-12-27 16:58:54,695][20347] Updated weights for policy 0, policy_version 1636251 (0.0010) [2023-12-27 16:58:54,754][20347] Updated weights for policy 0, policy_version 1636261 (0.0010) [2023-12-27 16:58:54,810][20347] Updated weights for policy 0, policy_version 1636271 (0.0010) [2023-12-27 16:58:55,172][20381] Updated weights for policy 1, policy_version 1634681 (0.0011) [2023-12-27 16:58:55,234][20381] Updated weights for policy 1, policy_version 1634691 (0.0011) [2023-12-27 16:58:55,291][20381] Updated weights for policy 1, policy_version 1634701 (0.0007) [2023-12-27 16:58:55,443][20347] Updated weights for policy 0, policy_version 1636281 (0.0010) [2023-12-27 16:58:55,495][20347] Updated weights for policy 0, policy_version 1636291 (0.0010) [2023-12-27 16:58:55,547][20347] Updated weights for policy 0, policy_version 1636301 (0.0010) [2023-12-27 16:58:55,877][20381] Updated weights for policy 1, policy_version 1634711 (0.0006) [2023-12-27 16:58:55,931][20381] Updated weights for policy 1, policy_version 1634721 (0.0005) [2023-12-27 16:58:55,983][20381] Updated weights for policy 1, policy_version 1634731 (0.0007) [2023-12-27 16:58:56,226][20347] Updated weights for policy 0, policy_version 1636311 (0.0010) [2023-12-27 16:58:56,280][20347] Updated weights for policy 0, policy_version 1636321 (0.0010) [2023-12-27 16:58:56,335][20347] Updated weights for policy 0, policy_version 1636331 (0.0010) [2023-12-27 16:58:56,607][20381] Updated weights for policy 1, policy_version 1634741 (0.0008) [2023-12-27 16:58:56,659][20381] Updated weights for policy 1, policy_version 1634751 (0.0008) [2023-12-27 16:58:56,709][20381] Updated weights for policy 1, policy_version 1634761 (0.0007) [2023-12-27 16:58:57,008][20347] Updated weights for policy 0, policy_version 1636341 (0.0010) [2023-12-27 16:58:57,056][20347] Updated weights for policy 0, policy_version 1636351 (0.0010) [2023-12-27 16:58:57,103][20347] Updated weights for policy 0, policy_version 1636361 (0.0010) [2023-12-27 16:58:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 837533696. Throughput: 0: 10385.7, 1: 10177.6. Samples: 837523436. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:58:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.270')] [2023-12-27 16:58:57,374][20381] Updated weights for policy 1, policy_version 1634771 (0.0008) [2023-12-27 16:58:57,427][20381] Updated weights for policy 1, policy_version 1634781 (0.0010) [2023-12-27 16:58:57,481][20381] Updated weights for policy 1, policy_version 1634791 (0.0010) [2023-12-27 16:58:57,787][20347] Updated weights for policy 0, policy_version 1636371 (0.0008) [2023-12-27 16:58:57,844][20347] Updated weights for policy 0, policy_version 1636381 (0.0006) [2023-12-27 16:58:57,896][20347] Updated weights for policy 0, policy_version 1636391 (0.0007) [2023-12-27 16:58:58,154][20381] Updated weights for policy 1, policy_version 1634801 (0.0010) [2023-12-27 16:58:58,213][20381] Updated weights for policy 1, policy_version 1634811 (0.0009) [2023-12-27 16:58:58,273][20381] Updated weights for policy 1, policy_version 1634821 (0.0010) [2023-12-27 16:58:58,330][20381] Updated weights for policy 1, policy_version 1634831 (0.0010) [2023-12-27 16:58:58,578][20347] Updated weights for policy 0, policy_version 1636401 (0.0007) [2023-12-27 16:58:58,642][20347] Updated weights for policy 0, policy_version 1636411 (0.0007) [2023-12-27 16:58:58,704][20347] Updated weights for policy 0, policy_version 1636421 (0.0008) [2023-12-27 16:58:58,772][20347] Updated weights for policy 0, policy_version 1636431 (0.0009) [2023-12-27 16:58:59,162][20381] Updated weights for policy 1, policy_version 1634841 (0.0008) [2023-12-27 16:58:59,220][20381] Updated weights for policy 1, policy_version 1634851 (0.0008) [2023-12-27 16:58:59,289][20381] Updated weights for policy 1, policy_version 1634861 (0.0007) [2023-12-27 16:58:59,510][20347] Updated weights for policy 0, policy_version 1636441 (0.0009) [2023-12-27 16:58:59,577][20347] Updated weights for policy 0, policy_version 1636451 (0.0008) [2023-12-27 16:58:59,636][20347] Updated weights for policy 0, policy_version 1636461 (0.0007) [2023-12-27 16:59:00,016][20381] Updated weights for policy 1, policy_version 1634871 (0.0008) [2023-12-27 16:59:00,078][20381] Updated weights for policy 1, policy_version 1634881 (0.0008) [2023-12-27 16:59:00,134][20381] Updated weights for policy 1, policy_version 1634891 (0.0008) [2023-12-27 16:59:00,241][20347] Updated weights for policy 0, policy_version 1636471 (0.0007) [2023-12-27 16:59:00,300][20347] Updated weights for policy 0, policy_version 1636481 (0.0008) [2023-12-27 16:59:00,356][20347] Updated weights for policy 0, policy_version 1636491 (0.0008) [2023-12-27 16:59:00,807][20381] Updated weights for policy 1, policy_version 1634901 (0.0005) [2023-12-27 16:59:00,868][20381] Updated weights for policy 1, policy_version 1634911 (0.0005) [2023-12-27 16:59:00,924][20381] Updated weights for policy 1, policy_version 1634921 (0.0005) [2023-12-27 16:59:00,952][20347] Updated weights for policy 0, policy_version 1636501 (0.0007) [2023-12-27 16:59:01,016][20347] Updated weights for policy 0, policy_version 1636511 (0.0007) [2023-12-27 16:59:01,078][20347] Updated weights for policy 0, policy_version 1636521 (0.0008) [2023-12-27 16:59:01,569][20381] Updated weights for policy 1, policy_version 1634931 (0.0006) [2023-12-27 16:59:01,626][20381] Updated weights for policy 1, policy_version 1634941 (0.0009) [2023-12-27 16:59:01,680][20381] Updated weights for policy 1, policy_version 1634951 (0.0007) [2023-12-27 16:59:01,798][20347] Updated weights for policy 0, policy_version 1636531 (0.0008) [2023-12-27 16:59:01,853][20347] Updated weights for policy 0, policy_version 1636541 (0.0008) [2023-12-27 16:59:01,909][20347] Updated weights for policy 0, policy_version 1636551 (0.0011) [2023-12-27 16:59:02,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 837632000. Throughput: 0: 10448.7, 1: 10104.8. Samples: 837644888. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.400')] [2023-12-27 16:59:02,393][20381] Updated weights for policy 1, policy_version 1634961 (0.0008) [2023-12-27 16:59:02,452][20381] Updated weights for policy 1, policy_version 1634971 (0.0007) [2023-12-27 16:59:02,504][20381] Updated weights for policy 1, policy_version 1634981 (0.0006) [2023-12-27 16:59:02,557][20381] Updated weights for policy 1, policy_version 1634991 (0.0011) [2023-12-27 16:59:02,595][20347] Updated weights for policy 0, policy_version 1636561 (0.0010) [2023-12-27 16:59:02,649][20347] Updated weights for policy 0, policy_version 1636571 (0.0005) [2023-12-27 16:59:02,697][20347] Updated weights for policy 0, policy_version 1636581 (0.0006) [2023-12-27 16:59:02,747][20347] Updated weights for policy 0, policy_version 1636591 (0.0007) [2023-12-27 16:59:03,187][20381] Updated weights for policy 1, policy_version 1635001 (0.0008) [2023-12-27 16:59:03,246][20381] Updated weights for policy 1, policy_version 1635011 (0.0010) [2023-12-27 16:59:03,294][20347] Updated weights for policy 0, policy_version 1636601 (0.0007) [2023-12-27 16:59:03,300][20381] Updated weights for policy 1, policy_version 1635021 (0.0010) [2023-12-27 16:59:03,352][20347] Updated weights for policy 0, policy_version 1636611 (0.0005) [2023-12-27 16:59:03,401][20347] Updated weights for policy 0, policy_version 1636621 (0.0005) [2023-12-27 16:59:03,935][20347] Updated weights for policy 0, policy_version 1636631 (0.0007) [2023-12-27 16:59:03,994][20381] Updated weights for policy 1, policy_version 1635031 (0.0008) [2023-12-27 16:59:03,996][20347] Updated weights for policy 0, policy_version 1636641 (0.0008) [2023-12-27 16:59:04,042][20381] Updated weights for policy 1, policy_version 1635041 (0.0007) [2023-12-27 16:59:04,058][20347] Updated weights for policy 0, policy_version 1636651 (0.0008) [2023-12-27 16:59:04,101][20381] Updated weights for policy 1, policy_version 1635051 (0.0006) [2023-12-27 16:59:04,701][20347] Updated weights for policy 0, policy_version 1636661 (0.0006) [2023-12-27 16:59:04,747][20381] Updated weights for policy 1, policy_version 1635061 (0.0007) [2023-12-27 16:59:04,751][20347] Updated weights for policy 0, policy_version 1636671 (0.0007) [2023-12-27 16:59:04,804][20347] Updated weights for policy 0, policy_version 1636681 (0.0006) [2023-12-27 16:59:04,810][20381] Updated weights for policy 1, policy_version 1635071 (0.0005) [2023-12-27 16:59:04,876][20381] Updated weights for policy 1, policy_version 1635081 (0.0005) [2023-12-27 16:59:05,408][20381] Updated weights for policy 1, policy_version 1635091 (0.0007) [2023-12-27 16:59:05,454][20347] Updated weights for policy 0, policy_version 1636691 (0.0006) [2023-12-27 16:59:05,459][20381] Updated weights for policy 1, policy_version 1635101 (0.0006) [2023-12-27 16:59:05,501][20347] Updated weights for policy 0, policy_version 1636701 (0.0008) [2023-12-27 16:59:05,511][20381] Updated weights for policy 1, policy_version 1635111 (0.0005) [2023-12-27 16:59:05,555][20347] Updated weights for policy 0, policy_version 1636711 (0.0008) [2023-12-27 16:59:06,066][20381] Updated weights for policy 1, policy_version 1635121 (0.0005) [2023-12-27 16:59:06,131][20292] KL-divergence is very high: 144.9008 [2023-12-27 16:59:06,131][20381] Updated weights for policy 1, policy_version 1635131 (0.0007) [2023-12-27 16:59:06,203][20347] Updated weights for policy 0, policy_version 1636721 (0.0006) [2023-12-27 16:59:06,212][20292] KL-divergence is very high: 187.0010 [2023-12-27 16:59:06,227][20381] Updated weights for policy 1, policy_version 1635141 (0.0008) [2023-12-27 16:59:06,261][20347] Updated weights for policy 0, policy_version 1636731 (0.0006) [2023-12-27 16:59:06,265][20292] KL-divergence is very high: 115.9000 [2023-12-27 16:59:06,291][20381] Updated weights for policy 1, policy_version 1635151 (0.0007) [2023-12-27 16:59:06,313][20347] Updated weights for policy 0, policy_version 1636741 (0.0007) [2023-12-27 16:59:06,377][20347] Updated weights for policy 0, policy_version 1636751 (0.0008) [2023-12-27 16:59:06,917][20381] Updated weights for policy 1, policy_version 1635161 (0.0005) [2023-12-27 16:59:06,969][20381] Updated weights for policy 1, policy_version 1635171 (0.0005) [2023-12-27 16:59:07,025][20381] Updated weights for policy 1, policy_version 1635181 (0.0005) [2023-12-27 16:59:07,072][20347] Updated weights for policy 0, policy_version 1636761 (0.0005) [2023-12-27 16:59:07,135][20347] Updated weights for policy 0, policy_version 1636771 (0.0006) [2023-12-27 16:59:07,196][20347] Updated weights for policy 0, policy_version 1636781 (0.0007) [2023-12-27 16:59:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 837746688. Throughput: 0: 10548.9, 1: 10160.0. Samples: 837711860. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.610')] [2023-12-27 16:59:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001635184_418652160.pth... [2023-12-27 16:59:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001636784_419094528.pth... [2023-12-27 16:59:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001635568_418783232.pth [2023-12-27 16:59:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001634000_418349056.pth [2023-12-27 16:59:07,647][20381] Updated weights for policy 1, policy_version 1635191 (0.0007) [2023-12-27 16:59:07,697][20381] Updated weights for policy 1, policy_version 1635201 (0.0006) [2023-12-27 16:59:07,753][20381] Updated weights for policy 1, policy_version 1635211 (0.0006) [2023-12-27 16:59:07,824][20347] Updated weights for policy 0, policy_version 1636791 (0.0010) [2023-12-27 16:59:07,888][20347] Updated weights for policy 0, policy_version 1636801 (0.0010) [2023-12-27 16:59:07,936][20347] Updated weights for policy 0, policy_version 1636811 (0.0010) [2023-12-27 16:59:08,426][20381] Updated weights for policy 1, policy_version 1635221 (0.0008) [2023-12-27 16:59:08,478][20381] Updated weights for policy 1, policy_version 1635231 (0.0011) [2023-12-27 16:59:08,531][20381] Updated weights for policy 1, policy_version 1635241 (0.0011) [2023-12-27 16:59:08,609][20347] Updated weights for policy 0, policy_version 1636821 (0.0010) [2023-12-27 16:59:08,659][20347] Updated weights for policy 0, policy_version 1636831 (0.0010) [2023-12-27 16:59:08,716][20347] Updated weights for policy 0, policy_version 1636841 (0.0010) [2023-12-27 16:59:09,190][20381] Updated weights for policy 1, policy_version 1635251 (0.0009) [2023-12-27 16:59:09,261][20381] Updated weights for policy 1, policy_version 1635261 (0.0009) [2023-12-27 16:59:09,328][20381] Updated weights for policy 1, policy_version 1635271 (0.0009) [2023-12-27 16:59:09,443][20347] Updated weights for policy 0, policy_version 1636851 (0.0010) [2023-12-27 16:59:09,497][20347] Updated weights for policy 0, policy_version 1636861 (0.0008) [2023-12-27 16:59:09,551][20347] Updated weights for policy 0, policy_version 1636871 (0.0008) [2023-12-27 16:59:10,059][20381] Updated weights for policy 1, policy_version 1635281 (0.0009) [2023-12-27 16:59:10,114][20381] Updated weights for policy 1, policy_version 1635291 (0.0008) [2023-12-27 16:59:10,171][20381] Updated weights for policy 1, policy_version 1635301 (0.0007) [2023-12-27 16:59:10,232][20381] Updated weights for policy 1, policy_version 1635311 (0.0007) [2023-12-27 16:59:10,274][20347] Updated weights for policy 0, policy_version 1636881 (0.0007) [2023-12-27 16:59:10,326][20347] Updated weights for policy 0, policy_version 1636891 (0.0008) [2023-12-27 16:59:10,381][20347] Updated weights for policy 0, policy_version 1636901 (0.0008) [2023-12-27 16:59:10,436][20347] Updated weights for policy 0, policy_version 1636911 (0.0008) [2023-12-27 16:59:10,988][20381] Updated weights for policy 1, policy_version 1635321 (0.0007) [2023-12-27 16:59:11,002][20347] Updated weights for policy 0, policy_version 1636921 (0.0006) [2023-12-27 16:59:11,043][20381] Updated weights for policy 1, policy_version 1635331 (0.0006) [2023-12-27 16:59:11,062][20347] Updated weights for policy 0, policy_version 1636931 (0.0007) [2023-12-27 16:59:11,102][20381] Updated weights for policy 1, policy_version 1635341 (0.0006) [2023-12-27 16:59:11,123][20347] Updated weights for policy 0, policy_version 1636941 (0.0006) [2023-12-27 16:59:11,897][20381] Updated weights for policy 1, policy_version 1635351 (0.0007) [2023-12-27 16:59:11,950][20347] Updated weights for policy 0, policy_version 1636951 (0.0008) [2023-12-27 16:59:11,957][20381] Updated weights for policy 1, policy_version 1635361 (0.0007) [2023-12-27 16:59:12,009][20347] Updated weights for policy 0, policy_version 1636961 (0.0008) [2023-12-27 16:59:12,016][20381] Updated weights for policy 1, policy_version 1635371 (0.0007) [2023-12-27 16:59:12,068][20347] Updated weights for policy 0, policy_version 1636971 (0.0008) [2023-12-27 16:59:12,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 837844992. Throughput: 0: 10544.1, 1: 10165.3. Samples: 837835352. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:12,364][19170] Avg episode reward: [(0, '1000.000'), (1, '902.510')] [2023-12-27 16:59:12,655][20381] Updated weights for policy 1, policy_version 1635381 (0.0006) [2023-12-27 16:59:12,718][20381] Updated weights for policy 1, policy_version 1635391 (0.0005) [2023-12-27 16:59:12,781][20381] Updated weights for policy 1, policy_version 1635401 (0.0008) [2023-12-27 16:59:12,814][20347] Updated weights for policy 0, policy_version 1636981 (0.0007) [2023-12-27 16:59:12,872][20347] Updated weights for policy 0, policy_version 1636991 (0.0007) [2023-12-27 16:59:12,927][20347] Updated weights for policy 0, policy_version 1637001 (0.0007) [2023-12-27 16:59:13,428][20381] Updated weights for policy 1, policy_version 1635411 (0.0007) [2023-12-27 16:59:13,478][20381] Updated weights for policy 1, policy_version 1635421 (0.0008) [2023-12-27 16:59:13,510][20347] Updated weights for policy 0, policy_version 1637011 (0.0007) [2023-12-27 16:59:13,529][20381] Updated weights for policy 1, policy_version 1635431 (0.0006) [2023-12-27 16:59:13,563][20347] Updated weights for policy 0, policy_version 1637021 (0.0009) [2023-12-27 16:59:13,616][20347] Updated weights for policy 0, policy_version 1637031 (0.0007) [2023-12-27 16:59:14,224][20381] Updated weights for policy 1, policy_version 1635441 (0.0006) [2023-12-27 16:59:14,279][20381] Updated weights for policy 1, policy_version 1635451 (0.0007) [2023-12-27 16:59:14,281][20347] Updated weights for policy 0, policy_version 1637041 (0.0008) [2023-12-27 16:59:14,336][20347] Updated weights for policy 0, policy_version 1637051 (0.0005) [2023-12-27 16:59:14,338][20381] Updated weights for policy 1, policy_version 1635461 (0.0006) [2023-12-27 16:59:14,390][20347] Updated weights for policy 0, policy_version 1637061 (0.0006) [2023-12-27 16:59:14,396][20381] Updated weights for policy 1, policy_version 1635471 (0.0006) [2023-12-27 16:59:14,448][20347] Updated weights for policy 0, policy_version 1637071 (0.0007) [2023-12-27 16:59:15,088][20347] Updated weights for policy 0, policy_version 1637081 (0.0008) [2023-12-27 16:59:15,115][20381] Updated weights for policy 1, policy_version 1635481 (0.0008) [2023-12-27 16:59:15,146][20347] Updated weights for policy 0, policy_version 1637091 (0.0008) [2023-12-27 16:59:15,174][20381] Updated weights for policy 1, policy_version 1635491 (0.0008) [2023-12-27 16:59:15,200][20347] Updated weights for policy 0, policy_version 1637101 (0.0007) [2023-12-27 16:59:15,229][20381] Updated weights for policy 1, policy_version 1635501 (0.0008) [2023-12-27 16:59:15,784][20347] Updated weights for policy 0, policy_version 1637111 (0.0006) [2023-12-27 16:59:15,842][20347] Updated weights for policy 0, policy_version 1637121 (0.0007) [2023-12-27 16:59:15,887][20347] Updated weights for policy 0, policy_version 1637131 (0.0007) [2023-12-27 16:59:16,003][20381] Updated weights for policy 1, policy_version 1635511 (0.0010) [2023-12-27 16:59:16,051][20381] Updated weights for policy 1, policy_version 1635521 (0.0010) [2023-12-27 16:59:16,111][20381] Updated weights for policy 1, policy_version 1635531 (0.0010) [2023-12-27 16:59:16,545][20347] Updated weights for policy 0, policy_version 1637141 (0.0008) [2023-12-27 16:59:16,588][20347] Updated weights for policy 0, policy_version 1637151 (0.0007) [2023-12-27 16:59:16,634][20347] Updated weights for policy 0, policy_version 1637161 (0.0007) [2023-12-27 16:59:16,797][20381] Updated weights for policy 1, policy_version 1635541 (0.0011) [2023-12-27 16:59:16,812][20292] KL-divergence is very high: 171.7653 [2023-12-27 16:59:16,822][20292] KL-divergence is very high: 220.5190 [2023-12-27 16:59:16,828][20292] KL-divergence is very high: 123.3340 [2023-12-27 16:59:16,846][20381] Updated weights for policy 1, policy_version 1635551 (0.0009) [2023-12-27 16:59:16,852][20292] KL-divergence is very high: 278.7335 [2023-12-27 16:59:16,863][20292] KL-divergence is very high: 230.5798 [2023-12-27 16:59:16,895][20292] KL-divergence is very high: 140.7714 [2023-12-27 16:59:16,901][20381] Updated weights for policy 1, policy_version 1635561 (0.0010) [2023-12-27 16:59:16,904][20292] KL-divergence is very high: 101.6107 [2023-12-27 16:59:17,313][20347] Updated weights for policy 0, policy_version 1637171 (0.0007) [2023-12-27 16:59:17,360][20347] Updated weights for policy 0, policy_version 1637181 (0.0006) [2023-12-27 16:59:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 837943296. Throughput: 0: 10495.8, 1: 10144.8. Samples: 837957588. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:17,363][19170] Avg episode reward: [(0, '990.240'), (1, '863.500')] [2023-12-27 16:59:17,408][20347] Updated weights for policy 0, policy_version 1637191 (0.0009) [2023-12-27 16:59:17,596][20381] Updated weights for policy 1, policy_version 1635571 (0.0011) [2023-12-27 16:59:17,651][20381] Updated weights for policy 1, policy_version 1635581 (0.0011) [2023-12-27 16:59:17,710][20381] Updated weights for policy 1, policy_version 1635591 (0.0011) [2023-12-27 16:59:18,080][20347] Updated weights for policy 0, policy_version 1637201 (0.0010) [2023-12-27 16:59:18,137][20347] Updated weights for policy 0, policy_version 1637211 (0.0010) [2023-12-27 16:59:18,202][20347] Updated weights for policy 0, policy_version 1637221 (0.0010) [2023-12-27 16:59:18,257][20347] Updated weights for policy 0, policy_version 1637231 (0.0010) [2023-12-27 16:59:18,372][20381] Updated weights for policy 1, policy_version 1635601 (0.0010) [2023-12-27 16:59:18,436][20381] Updated weights for policy 1, policy_version 1635611 (0.0008) [2023-12-27 16:59:18,495][20381] Updated weights for policy 1, policy_version 1635621 (0.0008) [2023-12-27 16:59:18,553][20381] Updated weights for policy 1, policy_version 1635631 (0.0008) [2023-12-27 16:59:18,946][20347] Updated weights for policy 0, policy_version 1637241 (0.0007) [2023-12-27 16:59:19,007][20347] Updated weights for policy 0, policy_version 1637251 (0.0007) [2023-12-27 16:59:19,064][20347] Updated weights for policy 0, policy_version 1637261 (0.0008) [2023-12-27 16:59:19,232][20381] Updated weights for policy 1, policy_version 1635641 (0.0011) [2023-12-27 16:59:19,288][20381] Updated weights for policy 1, policy_version 1635651 (0.0009) [2023-12-27 16:59:19,354][20381] Updated weights for policy 1, policy_version 1635661 (0.0011) [2023-12-27 16:59:19,737][20347] Updated weights for policy 0, policy_version 1637271 (0.0007) [2023-12-27 16:59:19,799][20347] Updated weights for policy 0, policy_version 1637281 (0.0008) [2023-12-27 16:59:19,862][20347] Updated weights for policy 0, policy_version 1637291 (0.0008) [2023-12-27 16:59:20,054][20381] Updated weights for policy 1, policy_version 1635671 (0.0009) [2023-12-27 16:59:20,108][20381] Updated weights for policy 1, policy_version 1635681 (0.0008) [2023-12-27 16:59:20,166][20381] Updated weights for policy 1, policy_version 1635691 (0.0008) [2023-12-27 16:59:20,552][20347] Updated weights for policy 0, policy_version 1637301 (0.0008) [2023-12-27 16:59:20,610][20347] Updated weights for policy 0, policy_version 1637311 (0.0008) [2023-12-27 16:59:20,655][20347] Updated weights for policy 0, policy_version 1637321 (0.0007) [2023-12-27 16:59:20,912][20381] Updated weights for policy 1, policy_version 1635701 (0.0009) [2023-12-27 16:59:20,972][20381] Updated weights for policy 1, policy_version 1635711 (0.0011) [2023-12-27 16:59:21,035][20381] Updated weights for policy 1, policy_version 1635721 (0.0010) [2023-12-27 16:59:21,425][20347] Updated weights for policy 0, policy_version 1637331 (0.0007) [2023-12-27 16:59:21,486][20347] Updated weights for policy 0, policy_version 1637341 (0.0007) [2023-12-27 16:59:21,544][20347] Updated weights for policy 0, policy_version 1637351 (0.0006) [2023-12-27 16:59:21,812][20381] Updated weights for policy 1, policy_version 1635731 (0.0009) [2023-12-27 16:59:21,873][20381] Updated weights for policy 1, policy_version 1635741 (0.0008) [2023-12-27 16:59:21,936][20381] Updated weights for policy 1, policy_version 1635751 (0.0008) [2023-12-27 16:59:22,257][20347] Updated weights for policy 0, policy_version 1637361 (0.0007) [2023-12-27 16:59:22,314][20347] Updated weights for policy 0, policy_version 1637371 (0.0008) [2023-12-27 16:59:22,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 838041600. Throughput: 0: 10428.1, 1: 10158.6. Samples: 838017028. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:22,363][19170] Avg episode reward: [(0, '990.240'), (1, '853.940')] [2023-12-27 16:59:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001635760_418799616.pth... [2023-12-27 16:59:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001634576_418496512.pth [2023-12-27 16:59:22,386][20347] Updated weights for policy 0, policy_version 1637381 (0.0007) [2023-12-27 16:59:22,446][20347] Updated weights for policy 0, policy_version 1637391 (0.0008) [2023-12-27 16:59:22,453][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001637392_419250176.pth... [2023-12-27 16:59:22,458][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001636144_418930688.pth [2023-12-27 16:59:22,633][20381] Updated weights for policy 1, policy_version 1635761 (0.0006) [2023-12-27 16:59:22,692][20381] Updated weights for policy 1, policy_version 1635771 (0.0007) [2023-12-27 16:59:22,749][20381] Updated weights for policy 1, policy_version 1635781 (0.0008) [2023-12-27 16:59:22,809][20381] Updated weights for policy 1, policy_version 1635791 (0.0008) [2023-12-27 16:59:23,123][20347] Updated weights for policy 0, policy_version 1637401 (0.0009) [2023-12-27 16:59:23,177][20347] Updated weights for policy 0, policy_version 1637411 (0.0008) [2023-12-27 16:59:23,231][20347] Updated weights for policy 0, policy_version 1637421 (0.0008) [2023-12-27 16:59:23,420][20381] Updated weights for policy 1, policy_version 1635801 (0.0007) [2023-12-27 16:59:23,480][20381] Updated weights for policy 1, policy_version 1635811 (0.0008) [2023-12-27 16:59:23,534][20381] Updated weights for policy 1, policy_version 1635821 (0.0008) [2023-12-27 16:59:23,881][20347] Updated weights for policy 0, policy_version 1637431 (0.0007) [2023-12-27 16:59:23,940][20347] Updated weights for policy 0, policy_version 1637441 (0.0008) [2023-12-27 16:59:23,992][20347] Updated weights for policy 0, policy_version 1637451 (0.0007) [2023-12-27 16:59:24,242][20381] Updated weights for policy 1, policy_version 1635831 (0.0007) [2023-12-27 16:59:24,306][20381] Updated weights for policy 1, policy_version 1635841 (0.0007) [2023-12-27 16:59:24,366][20381] Updated weights for policy 1, policy_version 1635851 (0.0008) [2023-12-27 16:59:24,748][20347] Updated weights for policy 0, policy_version 1637461 (0.0007) [2023-12-27 16:59:24,797][20347] Updated weights for policy 0, policy_version 1637471 (0.0007) [2023-12-27 16:59:24,858][20347] Updated weights for policy 0, policy_version 1637481 (0.0007) [2023-12-27 16:59:24,986][20381] Updated weights for policy 1, policy_version 1635861 (0.0007) [2023-12-27 16:59:25,046][20381] Updated weights for policy 1, policy_version 1635871 (0.0006) [2023-12-27 16:59:25,103][20381] Updated weights for policy 1, policy_version 1635881 (0.0007) [2023-12-27 16:59:25,449][20347] Updated weights for policy 0, policy_version 1637491 (0.0005) [2023-12-27 16:59:25,502][20347] Updated weights for policy 0, policy_version 1637501 (0.0005) [2023-12-27 16:59:25,546][20347] Updated weights for policy 0, policy_version 1637511 (0.0005) [2023-12-27 16:59:25,642][20381] Updated weights for policy 1, policy_version 1635891 (0.0007) [2023-12-27 16:59:25,690][20381] Updated weights for policy 1, policy_version 1635901 (0.0007) [2023-12-27 16:59:25,748][20381] Updated weights for policy 1, policy_version 1635911 (0.0008) [2023-12-27 16:59:26,123][20347] Updated weights for policy 0, policy_version 1637521 (0.0006) [2023-12-27 16:59:26,165][20347] Updated weights for policy 0, policy_version 1637531 (0.0007) [2023-12-27 16:59:26,220][20347] Updated weights for policy 0, policy_version 1637541 (0.0007) [2023-12-27 16:59:26,268][20347] Updated weights for policy 0, policy_version 1637551 (0.0009) [2023-12-27 16:59:26,351][20381] Updated weights for policy 1, policy_version 1635921 (0.0007) [2023-12-27 16:59:26,407][20381] Updated weights for policy 1, policy_version 1635931 (0.0005) [2023-12-27 16:59:26,456][20381] Updated weights for policy 1, policy_version 1635941 (0.0007) [2023-12-27 16:59:26,501][20381] Updated weights for policy 1, policy_version 1635951 (0.0008) [2023-12-27 16:59:26,951][20347] Updated weights for policy 0, policy_version 1637561 (0.0008) [2023-12-27 16:59:26,996][20347] Updated weights for policy 0, policy_version 1637571 (0.0007) [2023-12-27 16:59:27,043][20347] Updated weights for policy 0, policy_version 1637581 (0.0007) [2023-12-27 16:59:27,148][20381] Updated weights for policy 1, policy_version 1635961 (0.0005) [2023-12-27 16:59:27,205][20381] Updated weights for policy 1, policy_version 1635971 (0.0005) [2023-12-27 16:59:27,261][20381] Updated weights for policy 1, policy_version 1635981 (0.0006) [2023-12-27 16:59:27,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 838156288. Throughput: 0: 10439.5, 1: 10225.1. Samples: 838141104. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.150')] [2023-12-27 16:59:27,756][20347] Updated weights for policy 0, policy_version 1637591 (0.0009) [2023-12-27 16:59:27,775][20381] Updated weights for policy 1, policy_version 1635991 (0.0007) [2023-12-27 16:59:27,811][20347] Updated weights for policy 0, policy_version 1637601 (0.0005) [2023-12-27 16:59:27,828][20381] Updated weights for policy 1, policy_version 1636001 (0.0007) [2023-12-27 16:59:27,862][20347] Updated weights for policy 0, policy_version 1637611 (0.0005) [2023-12-27 16:59:27,884][20381] Updated weights for policy 1, policy_version 1636011 (0.0006) [2023-12-27 16:59:28,465][20381] Updated weights for policy 1, policy_version 1636021 (0.0007) [2023-12-27 16:59:28,472][20347] Updated weights for policy 0, policy_version 1637621 (0.0007) [2023-12-27 16:59:28,528][20381] Updated weights for policy 1, policy_version 1636031 (0.0006) [2023-12-27 16:59:28,529][20347] Updated weights for policy 0, policy_version 1637631 (0.0010) [2023-12-27 16:59:28,579][20381] Updated weights for policy 1, policy_version 1636041 (0.0005) [2023-12-27 16:59:28,589][20347] Updated weights for policy 0, policy_version 1637641 (0.0007) [2023-12-27 16:59:29,138][20347] Updated weights for policy 0, policy_version 1637651 (0.0005) [2023-12-27 16:59:29,165][20381] Updated weights for policy 1, policy_version 1636051 (0.0005) [2023-12-27 16:59:29,192][20347] Updated weights for policy 0, policy_version 1637661 (0.0006) [2023-12-27 16:59:29,219][20381] Updated weights for policy 1, policy_version 1636061 (0.0006) [2023-12-27 16:59:29,255][20347] Updated weights for policy 0, policy_version 1637671 (0.0009) [2023-12-27 16:59:29,284][20381] Updated weights for policy 1, policy_version 1636071 (0.0007) [2023-12-27 16:59:29,953][20381] Updated weights for policy 1, policy_version 1636081 (0.0007) [2023-12-27 16:59:29,955][20347] Updated weights for policy 0, policy_version 1637681 (0.0010) [2023-12-27 16:59:30,004][20347] Updated weights for policy 0, policy_version 1637691 (0.0010) [2023-12-27 16:59:30,011][20381] Updated weights for policy 1, policy_version 1636091 (0.0008) [2023-12-27 16:59:30,053][20347] Updated weights for policy 0, policy_version 1637701 (0.0010) [2023-12-27 16:59:30,069][20381] Updated weights for policy 1, policy_version 1636101 (0.0008) [2023-12-27 16:59:30,108][20347] Updated weights for policy 0, policy_version 1637711 (0.0010) [2023-12-27 16:59:30,120][20381] Updated weights for policy 1, policy_version 1636111 (0.0008) [2023-12-27 16:59:30,730][20381] Updated weights for policy 1, policy_version 1636121 (0.0006) [2023-12-27 16:59:30,779][20381] Updated weights for policy 1, policy_version 1636131 (0.0006) [2023-12-27 16:59:30,808][20347] Updated weights for policy 0, policy_version 1637721 (0.0009) [2023-12-27 16:59:30,832][20381] Updated weights for policy 1, policy_version 1636141 (0.0007) [2023-12-27 16:59:30,866][20347] Updated weights for policy 0, policy_version 1637731 (0.0010) [2023-12-27 16:59:30,916][20347] Updated weights for policy 0, policy_version 1637741 (0.0010) [2023-12-27 16:59:31,465][20381] Updated weights for policy 1, policy_version 1636151 (0.0007) [2023-12-27 16:59:31,528][20381] Updated weights for policy 1, policy_version 1636161 (0.0006) [2023-12-27 16:59:31,588][20381] Updated weights for policy 1, policy_version 1636171 (0.0007) [2023-12-27 16:59:31,672][20347] Updated weights for policy 0, policy_version 1637751 (0.0007) [2023-12-27 16:59:31,741][20347] Updated weights for policy 0, policy_version 1637761 (0.0009) [2023-12-27 16:59:31,800][20347] Updated weights for policy 0, policy_version 1637771 (0.0011) [2023-12-27 16:59:32,249][20381] Updated weights for policy 1, policy_version 1636181 (0.0009) [2023-12-27 16:59:32,312][20381] Updated weights for policy 1, policy_version 1636191 (0.0010) [2023-12-27 16:59:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 838254592. Throughput: 0: 10405.8, 1: 10365.3. Samples: 838270248. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.870')] [2023-12-27 16:59:32,376][20381] Updated weights for policy 1, policy_version 1636201 (0.0011) [2023-12-27 16:59:32,442][20347] Updated weights for policy 0, policy_version 1637781 (0.0009) [2023-12-27 16:59:32,494][20347] Updated weights for policy 0, policy_version 1637791 (0.0007) [2023-12-27 16:59:32,553][20347] Updated weights for policy 0, policy_version 1637801 (0.0008) [2023-12-27 16:59:33,056][20381] Updated weights for policy 1, policy_version 1636211 (0.0011) [2023-12-27 16:59:33,113][20381] Updated weights for policy 1, policy_version 1636221 (0.0010) [2023-12-27 16:59:33,149][20347] Updated weights for policy 0, policy_version 1637811 (0.0007) [2023-12-27 16:59:33,158][20381] Updated weights for policy 1, policy_version 1636231 (0.0010) [2023-12-27 16:59:33,197][20347] Updated weights for policy 0, policy_version 1637821 (0.0010) [2023-12-27 16:59:33,253][20347] Updated weights for policy 0, policy_version 1637831 (0.0010) [2023-12-27 16:59:33,809][20381] Updated weights for policy 1, policy_version 1636241 (0.0010) [2023-12-27 16:59:33,857][20381] Updated weights for policy 1, policy_version 1636251 (0.0009) [2023-12-27 16:59:33,900][20347] Updated weights for policy 0, policy_version 1637841 (0.0010) [2023-12-27 16:59:33,912][20381] Updated weights for policy 1, policy_version 1636261 (0.0006) [2023-12-27 16:59:33,945][20347] Updated weights for policy 0, policy_version 1637851 (0.0007) [2023-12-27 16:59:33,967][20381] Updated weights for policy 1, policy_version 1636271 (0.0006) [2023-12-27 16:59:34,003][20347] Updated weights for policy 0, policy_version 1637861 (0.0006) [2023-12-27 16:59:34,060][20347] Updated weights for policy 0, policy_version 1637871 (0.0005) [2023-12-27 16:59:34,603][20381] Updated weights for policy 1, policy_version 1636281 (0.0010) [2023-12-27 16:59:34,662][20381] Updated weights for policy 1, policy_version 1636291 (0.0010) [2023-12-27 16:59:34,715][20381] Updated weights for policy 1, policy_version 1636301 (0.0009) [2023-12-27 16:59:34,738][20347] Updated weights for policy 0, policy_version 1637881 (0.0006) [2023-12-27 16:59:34,793][20347] Updated weights for policy 0, policy_version 1637891 (0.0007) [2023-12-27 16:59:34,859][20347] Updated weights for policy 0, policy_version 1637901 (0.0008) [2023-12-27 16:59:35,347][20381] Updated weights for policy 1, policy_version 1636311 (0.0007) [2023-12-27 16:59:35,400][20381] Updated weights for policy 1, policy_version 1636321 (0.0005) [2023-12-27 16:59:35,427][20347] Updated weights for policy 0, policy_version 1637911 (0.0006) [2023-12-27 16:59:35,457][20381] Updated weights for policy 1, policy_version 1636331 (0.0005) [2023-12-27 16:59:35,483][20347] Updated weights for policy 0, policy_version 1637921 (0.0005) [2023-12-27 16:59:35,529][20347] Updated weights for policy 0, policy_version 1637931 (0.0005) [2023-12-27 16:59:36,003][20381] Updated weights for policy 1, policy_version 1636341 (0.0008) [2023-12-27 16:59:36,062][20381] Updated weights for policy 1, policy_version 1636351 (0.0006) [2023-12-27 16:59:36,128][20381] Updated weights for policy 1, policy_version 1636361 (0.0007) [2023-12-27 16:59:36,214][20347] Updated weights for policy 0, policy_version 1637941 (0.0007) [2023-12-27 16:59:36,281][20347] Updated weights for policy 0, policy_version 1637951 (0.0009) [2023-12-27 16:59:36,346][20347] Updated weights for policy 0, policy_version 1637961 (0.0010) [2023-12-27 16:59:36,776][20381] Updated weights for policy 1, policy_version 1636371 (0.0011) [2023-12-27 16:59:36,837][20381] Updated weights for policy 1, policy_version 1636381 (0.0011) [2023-12-27 16:59:36,896][20381] Updated weights for policy 1, policy_version 1636391 (0.0011) [2023-12-27 16:59:37,023][20347] Updated weights for policy 0, policy_version 1637971 (0.0010) [2023-12-27 16:59:37,074][20347] Updated weights for policy 0, policy_version 1637981 (0.0010) [2023-12-27 16:59:37,129][20347] Updated weights for policy 0, policy_version 1637991 (0.0010) [2023-12-27 16:59:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 838369280. Throughput: 0: 10440.9, 1: 10355.8. Samples: 838334584. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.060')] [2023-12-27 16:59:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001636400_418963456.pth... [2023-12-27 16:59:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001638000_419405824.pth... [2023-12-27 16:59:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001636784_419094528.pth [2023-12-27 16:59:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001635184_418652160.pth [2023-12-27 16:59:37,585][20381] Updated weights for policy 1, policy_version 1636401 (0.0011) [2023-12-27 16:59:37,637][20381] Updated weights for policy 1, policy_version 1636411 (0.0007) [2023-12-27 16:59:37,696][20381] Updated weights for policy 1, policy_version 1636421 (0.0007) [2023-12-27 16:59:37,751][20381] Updated weights for policy 1, policy_version 1636431 (0.0007) [2023-12-27 16:59:37,772][20347] Updated weights for policy 0, policy_version 1638001 (0.0010) [2023-12-27 16:59:37,818][20347] Updated weights for policy 0, policy_version 1638011 (0.0008) [2023-12-27 16:59:37,863][20347] Updated weights for policy 0, policy_version 1638021 (0.0010) [2023-12-27 16:59:37,921][20347] Updated weights for policy 0, policy_version 1638031 (0.0010) [2023-12-27 16:59:38,444][20381] Updated weights for policy 1, policy_version 1636441 (0.0008) [2023-12-27 16:59:38,490][20381] Updated weights for policy 1, policy_version 1636451 (0.0008) [2023-12-27 16:59:38,549][20381] Updated weights for policy 1, policy_version 1636461 (0.0008) [2023-12-27 16:59:38,607][20347] Updated weights for policy 0, policy_version 1638041 (0.0008) [2023-12-27 16:59:38,652][20347] Updated weights for policy 0, policy_version 1638051 (0.0008) [2023-12-27 16:59:38,695][20347] Updated weights for policy 0, policy_version 1638061 (0.0008) [2023-12-27 16:59:39,265][20381] Updated weights for policy 1, policy_version 1636471 (0.0007) [2023-12-27 16:59:39,325][20381] Updated weights for policy 1, policy_version 1636481 (0.0008) [2023-12-27 16:59:39,395][20381] Updated weights for policy 1, policy_version 1636491 (0.0008) [2023-12-27 16:59:39,445][20347] Updated weights for policy 0, policy_version 1638071 (0.0008) [2023-12-27 16:59:39,494][20347] Updated weights for policy 0, policy_version 1638081 (0.0007) [2023-12-27 16:59:39,546][20347] Updated weights for policy 0, policy_version 1638091 (0.0008) [2023-12-27 16:59:40,182][20381] Updated weights for policy 1, policy_version 1636501 (0.0008) [2023-12-27 16:59:40,246][20381] Updated weights for policy 1, policy_version 1636511 (0.0008) [2023-12-27 16:59:40,313][20381] Updated weights for policy 1, policy_version 1636521 (0.0007) [2023-12-27 16:59:40,335][20347] Updated weights for policy 0, policy_version 1638101 (0.0008) [2023-12-27 16:59:40,397][20347] Updated weights for policy 0, policy_version 1638111 (0.0008) [2023-12-27 16:59:40,458][20347] Updated weights for policy 0, policy_version 1638121 (0.0008) [2023-12-27 16:59:41,003][20381] Updated weights for policy 1, policy_version 1636531 (0.0008) [2023-12-27 16:59:41,070][20381] Updated weights for policy 1, policy_version 1636541 (0.0007) [2023-12-27 16:59:41,110][20347] Updated weights for policy 0, policy_version 1638131 (0.0009) [2023-12-27 16:59:41,136][20381] Updated weights for policy 1, policy_version 1636551 (0.0006) [2023-12-27 16:59:41,173][20347] Updated weights for policy 0, policy_version 1638141 (0.0008) [2023-12-27 16:59:41,228][20347] Updated weights for policy 0, policy_version 1638151 (0.0005) [2023-12-27 16:59:41,857][20381] Updated weights for policy 1, policy_version 1636561 (0.0008) [2023-12-27 16:59:41,918][20381] Updated weights for policy 1, policy_version 1636571 (0.0008) [2023-12-27 16:59:41,933][20347] Updated weights for policy 0, policy_version 1638161 (0.0007) [2023-12-27 16:59:41,978][20381] Updated weights for policy 1, policy_version 1636581 (0.0008) [2023-12-27 16:59:41,987][20347] Updated weights for policy 0, policy_version 1638171 (0.0007) [2023-12-27 16:59:42,038][20381] Updated weights for policy 1, policy_version 1636591 (0.0006) [2023-12-27 16:59:42,044][20347] Updated weights for policy 0, policy_version 1638181 (0.0007) [2023-12-27 16:59:42,105][20347] Updated weights for policy 0, policy_version 1638191 (0.0008) [2023-12-27 16:59:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 838467584. Throughput: 0: 10399.3, 1: 10327.6. Samples: 838456144. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:42,363][19170] Avg episode reward: [(0, '990.320'), (1, '883.070')] [2023-12-27 16:59:42,726][20381] Updated weights for policy 1, policy_version 1636601 (0.0007) [2023-12-27 16:59:42,734][20347] Updated weights for policy 0, policy_version 1638201 (0.0006) [2023-12-27 16:59:42,787][20381] Updated weights for policy 1, policy_version 1636611 (0.0008) [2023-12-27 16:59:42,792][20347] Updated weights for policy 0, policy_version 1638211 (0.0005) [2023-12-27 16:59:42,843][20381] Updated weights for policy 1, policy_version 1636621 (0.0007) [2023-12-27 16:59:42,850][20347] Updated weights for policy 0, policy_version 1638221 (0.0006) [2023-12-27 16:59:43,387][20347] Updated weights for policy 0, policy_version 1638231 (0.0007) [2023-12-27 16:59:43,449][20347] Updated weights for policy 0, policy_version 1638241 (0.0005) [2023-12-27 16:59:43,503][20347] Updated weights for policy 0, policy_version 1638251 (0.0007) [2023-12-27 16:59:43,540][20381] Updated weights for policy 1, policy_version 1636631 (0.0008) [2023-12-27 16:59:43,586][20381] Updated weights for policy 1, policy_version 1636641 (0.0007) [2023-12-27 16:59:43,630][20381] Updated weights for policy 1, policy_version 1636651 (0.0008) [2023-12-27 16:59:44,031][20347] Updated weights for policy 0, policy_version 1638261 (0.0005) [2023-12-27 16:59:44,087][20347] Updated weights for policy 0, policy_version 1638271 (0.0006) [2023-12-27 16:59:44,146][20347] Updated weights for policy 0, policy_version 1638281 (0.0007) [2023-12-27 16:59:44,331][20381] Updated weights for policy 1, policy_version 1636661 (0.0007) [2023-12-27 16:59:44,387][20381] Updated weights for policy 1, policy_version 1636671 (0.0006) [2023-12-27 16:59:44,458][20381] Updated weights for policy 1, policy_version 1636681 (0.0006) [2023-12-27 16:59:44,777][20347] Updated weights for policy 0, policy_version 1638291 (0.0010) [2023-12-27 16:59:44,839][20347] Updated weights for policy 0, policy_version 1638301 (0.0009) [2023-12-27 16:59:44,898][20347] Updated weights for policy 0, policy_version 1638311 (0.0008) [2023-12-27 16:59:45,126][20381] Updated weights for policy 1, policy_version 1636691 (0.0011) [2023-12-27 16:59:45,182][20381] Updated weights for policy 1, policy_version 1636701 (0.0011) [2023-12-27 16:59:45,232][20381] Updated weights for policy 1, policy_version 1636711 (0.0011) [2023-12-27 16:59:45,620][20347] Updated weights for policy 0, policy_version 1638321 (0.0008) [2023-12-27 16:59:45,679][20347] Updated weights for policy 0, policy_version 1638331 (0.0005) [2023-12-27 16:59:45,741][20347] Updated weights for policy 0, policy_version 1638341 (0.0006) [2023-12-27 16:59:45,794][20347] Updated weights for policy 0, policy_version 1638351 (0.0007) [2023-12-27 16:59:45,889][20381] Updated weights for policy 1, policy_version 1636721 (0.0010) [2023-12-27 16:59:45,947][20381] Updated weights for policy 1, policy_version 1636731 (0.0009) [2023-12-27 16:59:46,002][20381] Updated weights for policy 1, policy_version 1636741 (0.0009) [2023-12-27 16:59:46,055][20381] Updated weights for policy 1, policy_version 1636751 (0.0005) [2023-12-27 16:59:46,397][20347] Updated weights for policy 0, policy_version 1638361 (0.0008) [2023-12-27 16:59:46,444][20347] Updated weights for policy 0, policy_version 1638371 (0.0007) [2023-12-27 16:59:46,500][20347] Updated weights for policy 0, policy_version 1638381 (0.0007) [2023-12-27 16:59:46,683][20381] Updated weights for policy 1, policy_version 1636761 (0.0010) [2023-12-27 16:59:46,731][20381] Updated weights for policy 1, policy_version 1636771 (0.0010) [2023-12-27 16:59:46,792][20381] Updated weights for policy 1, policy_version 1636781 (0.0010) [2023-12-27 16:59:47,179][20347] Updated weights for policy 0, policy_version 1638391 (0.0008) [2023-12-27 16:59:47,226][20347] Updated weights for policy 0, policy_version 1638401 (0.0008) [2023-12-27 16:59:47,274][20347] Updated weights for policy 0, policy_version 1638411 (0.0008) [2023-12-27 16:59:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 838574080. Throughput: 0: 10484.3, 1: 10360.9. Samples: 838582920. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:47,363][19170] Avg episode reward: [(0, '971.050'), (1, '873.500')] [2023-12-27 16:59:47,417][20381] Updated weights for policy 1, policy_version 1636791 (0.0009) [2023-12-27 16:59:47,469][20381] Updated weights for policy 1, policy_version 1636801 (0.0007) [2023-12-27 16:59:47,528][20381] Updated weights for policy 1, policy_version 1636811 (0.0008) [2023-12-27 16:59:47,974][20347] Updated weights for policy 0, policy_version 1638421 (0.0006) [2023-12-27 16:59:48,025][20347] Updated weights for policy 0, policy_version 1638431 (0.0007) [2023-12-27 16:59:48,078][20347] Updated weights for policy 0, policy_version 1638441 (0.0007) [2023-12-27 16:59:48,141][20381] Updated weights for policy 1, policy_version 1636821 (0.0006) [2023-12-27 16:59:48,196][20381] Updated weights for policy 1, policy_version 1636831 (0.0006) [2023-12-27 16:59:48,252][20381] Updated weights for policy 1, policy_version 1636841 (0.0005) [2023-12-27 16:59:48,740][20347] Updated weights for policy 0, policy_version 1638451 (0.0006) [2023-12-27 16:59:48,805][20347] Updated weights for policy 0, policy_version 1638461 (0.0008) [2023-12-27 16:59:48,854][20381] Updated weights for policy 1, policy_version 1636851 (0.0006) [2023-12-27 16:59:48,870][20347] Updated weights for policy 0, policy_version 1638471 (0.0008) [2023-12-27 16:59:48,910][20381] Updated weights for policy 1, policy_version 1636861 (0.0005) [2023-12-27 16:59:48,969][20381] Updated weights for policy 1, policy_version 1636871 (0.0009) [2023-12-27 16:59:49,566][20347] Updated weights for policy 0, policy_version 1638481 (0.0007) [2023-12-27 16:59:49,621][20347] Updated weights for policy 0, policy_version 1638491 (0.0008) [2023-12-27 16:59:49,636][20381] Updated weights for policy 1, policy_version 1636881 (0.0008) [2023-12-27 16:59:49,683][20347] Updated weights for policy 0, policy_version 1638501 (0.0007) [2023-12-27 16:59:49,695][20381] Updated weights for policy 1, policy_version 1636891 (0.0007) [2023-12-27 16:59:49,737][20347] Updated weights for policy 0, policy_version 1638511 (0.0007) [2023-12-27 16:59:49,750][20381] Updated weights for policy 1, policy_version 1636901 (0.0006) [2023-12-27 16:59:49,807][20381] Updated weights for policy 1, policy_version 1636911 (0.0005) [2023-12-27 16:59:50,430][20381] Updated weights for policy 1, policy_version 1636921 (0.0007) [2023-12-27 16:59:50,486][20381] Updated weights for policy 1, policy_version 1636931 (0.0006) [2023-12-27 16:59:50,519][20347] Updated weights for policy 0, policy_version 1638521 (0.0007) [2023-12-27 16:59:50,543][20381] Updated weights for policy 1, policy_version 1636941 (0.0006) [2023-12-27 16:59:50,580][20347] Updated weights for policy 0, policy_version 1638531 (0.0007) [2023-12-27 16:59:50,638][20347] Updated weights for policy 0, policy_version 1638541 (0.0007) [2023-12-27 16:59:51,201][20381] Updated weights for policy 1, policy_version 1636951 (0.0007) [2023-12-27 16:59:51,260][20381] Updated weights for policy 1, policy_version 1636961 (0.0007) [2023-12-27 16:59:51,316][20381] Updated weights for policy 1, policy_version 1636971 (0.0007) [2023-12-27 16:59:51,344][20347] Updated weights for policy 0, policy_version 1638551 (0.0008) [2023-12-27 16:59:51,409][20347] Updated weights for policy 0, policy_version 1638561 (0.0008) [2023-12-27 16:59:51,468][20347] Updated weights for policy 0, policy_version 1638571 (0.0007) [2023-12-27 16:59:52,002][20381] Updated weights for policy 1, policy_version 1636981 (0.0007) [2023-12-27 16:59:52,063][20381] Updated weights for policy 1, policy_version 1636991 (0.0006) [2023-12-27 16:59:52,123][20381] Updated weights for policy 1, policy_version 1637001 (0.0008) [2023-12-27 16:59:52,235][20347] Updated weights for policy 0, policy_version 1638581 (0.0008) [2023-12-27 16:59:52,297][20347] Updated weights for policy 0, policy_version 1638591 (0.0009) [2023-12-27 16:59:52,357][20347] Updated weights for policy 0, policy_version 1638601 (0.0008) [2023-12-27 16:59:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 838672384. Throughput: 0: 10372.6, 1: 10359.0. Samples: 838644784. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:52,363][19170] Avg episode reward: [(0, '980.730'), (1, '834.590')] [2023-12-27 16:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001637008_419119104.pth... [2023-12-27 16:59:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001635760_418799616.pth [2023-12-27 16:59:52,375][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001637008_419119104.pth [2023-12-27 16:59:52,402][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001638608_419561472.pth... [2023-12-27 16:59:52,407][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001637392_419250176.pth [2023-12-27 16:59:52,407][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001638608_419561472.pth [2023-12-27 16:59:52,710][20381] Updated weights for policy 1, policy_version 1637011 (0.0008) [2023-12-27 16:59:52,766][20381] Updated weights for policy 1, policy_version 1637021 (0.0008) [2023-12-27 16:59:52,825][20381] Updated weights for policy 1, policy_version 1637031 (0.0007) [2023-12-27 16:59:53,061][20347] Updated weights for policy 0, policy_version 1638611 (0.0007) [2023-12-27 16:59:53,117][20347] Updated weights for policy 0, policy_version 1638621 (0.0005) [2023-12-27 16:59:53,171][20347] Updated weights for policy 0, policy_version 1638631 (0.0006) [2023-12-27 16:59:53,473][20381] Updated weights for policy 1, policy_version 1637041 (0.0008) [2023-12-27 16:59:53,526][20381] Updated weights for policy 1, policy_version 1637051 (0.0005) [2023-12-27 16:59:53,569][20381] Updated weights for policy 1, policy_version 1637061 (0.0005) [2023-12-27 16:59:53,624][20381] Updated weights for policy 1, policy_version 1637071 (0.0006) [2023-12-27 16:59:53,759][20347] Updated weights for policy 0, policy_version 1638641 (0.0005) [2023-12-27 16:59:53,817][20347] Updated weights for policy 0, policy_version 1638651 (0.0005) [2023-12-27 16:59:53,870][20347] Updated weights for policy 0, policy_version 1638661 (0.0005) [2023-12-27 16:59:53,920][20347] Updated weights for policy 0, policy_version 1638671 (0.0006) [2023-12-27 16:59:54,252][20381] Updated weights for policy 1, policy_version 1637081 (0.0008) [2023-12-27 16:59:54,304][20381] Updated weights for policy 1, policy_version 1637091 (0.0008) [2023-12-27 16:59:54,362][20381] Updated weights for policy 1, policy_version 1637101 (0.0007) [2023-12-27 16:59:54,558][20347] Updated weights for policy 0, policy_version 1638681 (0.0007) [2023-12-27 16:59:54,622][20347] Updated weights for policy 0, policy_version 1638691 (0.0008) [2023-12-27 16:59:54,687][20347] Updated weights for policy 0, policy_version 1638701 (0.0008) [2023-12-27 16:59:55,066][20381] Updated weights for policy 1, policy_version 1637111 (0.0008) [2023-12-27 16:59:55,123][20381] Updated weights for policy 1, policy_version 1637121 (0.0008) [2023-12-27 16:59:55,180][20381] Updated weights for policy 1, policy_version 1637131 (0.0008) [2023-12-27 16:59:55,341][20347] Updated weights for policy 0, policy_version 1638711 (0.0007) [2023-12-27 16:59:55,397][20347] Updated weights for policy 0, policy_version 1638721 (0.0006) [2023-12-27 16:59:55,466][20347] Updated weights for policy 0, policy_version 1638731 (0.0007) [2023-12-27 16:59:55,895][20381] Updated weights for policy 1, policy_version 1637141 (0.0009) [2023-12-27 16:59:55,958][20381] Updated weights for policy 1, policy_version 1637151 (0.0010) [2023-12-27 16:59:56,005][20381] Updated weights for policy 1, policy_version 1637161 (0.0010) [2023-12-27 16:59:56,118][20347] Updated weights for policy 0, policy_version 1638741 (0.0008) [2023-12-27 16:59:56,173][20347] Updated weights for policy 0, policy_version 1638751 (0.0008) [2023-12-27 16:59:56,222][20347] Updated weights for policy 0, policy_version 1638761 (0.0010) [2023-12-27 16:59:56,683][20381] Updated weights for policy 1, policy_version 1637171 (0.0010) [2023-12-27 16:59:56,731][20381] Updated weights for policy 1, policy_version 1637181 (0.0010) [2023-12-27 16:59:56,778][20381] Updated weights for policy 1, policy_version 1637191 (0.0010) [2023-12-27 16:59:56,877][20347] Updated weights for policy 0, policy_version 1638771 (0.0010) [2023-12-27 16:59:56,931][20347] Updated weights for policy 0, policy_version 1638781 (0.0010) [2023-12-27 16:59:56,975][20347] Updated weights for policy 0, policy_version 1638791 (0.0010) [2023-12-27 16:59:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 838778880. Throughput: 0: 10343.8, 1: 10388.3. Samples: 838768292. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 16:59:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '893.010')] [2023-12-27 16:59:57,404][20381] Updated weights for policy 1, policy_version 1637201 (0.0009) [2023-12-27 16:59:57,449][20381] Updated weights for policy 1, policy_version 1637211 (0.0010) [2023-12-27 16:59:57,497][20381] Updated weights for policy 1, policy_version 1637221 (0.0009) [2023-12-27 16:59:57,551][20381] Updated weights for policy 1, policy_version 1637231 (0.0009) [2023-12-27 16:59:57,665][20347] Updated weights for policy 0, policy_version 1638801 (0.0010) [2023-12-27 16:59:57,722][20347] Updated weights for policy 0, policy_version 1638811 (0.0010) [2023-12-27 16:59:57,780][20347] Updated weights for policy 0, policy_version 1638821 (0.0010) [2023-12-27 16:59:57,837][20347] Updated weights for policy 0, policy_version 1638831 (0.0009) [2023-12-27 16:59:58,135][20381] Updated weights for policy 1, policy_version 1637241 (0.0008) [2023-12-27 16:59:58,190][20381] Updated weights for policy 1, policy_version 1637251 (0.0007) [2023-12-27 16:59:58,255][20381] Updated weights for policy 1, policy_version 1637261 (0.0008) [2023-12-27 16:59:58,592][20347] Updated weights for policy 0, policy_version 1638841 (0.0008) [2023-12-27 16:59:58,651][20347] Updated weights for policy 0, policy_version 1638851 (0.0008) [2023-12-27 16:59:58,710][20347] Updated weights for policy 0, policy_version 1638861 (0.0008) [2023-12-27 16:59:59,050][20381] Updated weights for policy 1, policy_version 1637271 (0.0007) [2023-12-27 16:59:59,109][20381] Updated weights for policy 1, policy_version 1637281 (0.0007) [2023-12-27 16:59:59,167][20381] Updated weights for policy 1, policy_version 1637291 (0.0007) [2023-12-27 16:59:59,494][20347] Updated weights for policy 0, policy_version 1638871 (0.0006) [2023-12-27 16:59:59,551][20347] Updated weights for policy 0, policy_version 1638881 (0.0007) [2023-12-27 16:59:59,613][20347] Updated weights for policy 0, policy_version 1638891 (0.0010) [2023-12-27 16:59:59,895][20381] Updated weights for policy 1, policy_version 1637301 (0.0006) [2023-12-27 16:59:59,954][20381] Updated weights for policy 1, policy_version 1637311 (0.0007) [2023-12-27 17:00:00,005][20381] Updated weights for policy 1, policy_version 1637321 (0.0007) [2023-12-27 17:00:00,245][20347] Updated weights for policy 0, policy_version 1638901 (0.0008) [2023-12-27 17:00:00,299][20347] Updated weights for policy 0, policy_version 1638911 (0.0005) [2023-12-27 17:00:00,357][20347] Updated weights for policy 0, policy_version 1638921 (0.0005) [2023-12-27 17:00:00,768][20381] Updated weights for policy 1, policy_version 1637331 (0.0008) [2023-12-27 17:00:00,825][20381] Updated weights for policy 1, policy_version 1637341 (0.0007) [2023-12-27 17:00:00,882][20381] Updated weights for policy 1, policy_version 1637351 (0.0007) [2023-12-27 17:00:00,959][20347] Updated weights for policy 0, policy_version 1638931 (0.0008) [2023-12-27 17:00:01,009][20347] Updated weights for policy 0, policy_version 1638941 (0.0005) [2023-12-27 17:00:01,069][20347] Updated weights for policy 0, policy_version 1638951 (0.0007) [2023-12-27 17:00:01,611][20381] Updated weights for policy 1, policy_version 1637361 (0.0007) [2023-12-27 17:00:01,677][20381] Updated weights for policy 1, policy_version 1637371 (0.0008) [2023-12-27 17:00:01,696][20347] Updated weights for policy 0, policy_version 1638961 (0.0006) [2023-12-27 17:00:01,741][20381] Updated weights for policy 1, policy_version 1637381 (0.0006) [2023-12-27 17:00:01,762][20347] Updated weights for policy 0, policy_version 1638971 (0.0008) [2023-12-27 17:00:01,795][20381] Updated weights for policy 1, policy_version 1637391 (0.0006) [2023-12-27 17:00:01,814][20347] Updated weights for policy 0, policy_version 1638981 (0.0006) [2023-12-27 17:00:01,875][20347] Updated weights for policy 0, policy_version 1638991 (0.0007) [2023-12-27 17:00:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 838877184. Throughput: 0: 10326.4, 1: 10403.6. Samples: 838890440. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 17:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.070')] [2023-12-27 17:00:02,441][20381] Updated weights for policy 1, policy_version 1637401 (0.0007) [2023-12-27 17:00:02,492][20381] Updated weights for policy 1, policy_version 1637411 (0.0007) [2023-12-27 17:00:02,547][20347] Updated weights for policy 0, policy_version 1639001 (0.0006) [2023-12-27 17:00:02,548][20381] Updated weights for policy 1, policy_version 1637421 (0.0006) [2023-12-27 17:00:02,601][20347] Updated weights for policy 0, policy_version 1639011 (0.0005) [2023-12-27 17:00:02,659][20347] Updated weights for policy 0, policy_version 1639021 (0.0006) [2023-12-27 17:00:03,226][20381] Updated weights for policy 1, policy_version 1637431 (0.0008) [2023-12-27 17:00:03,278][20381] Updated weights for policy 1, policy_version 1637441 (0.0010) [2023-12-27 17:00:03,295][20347] Updated weights for policy 0, policy_version 1639031 (0.0008) [2023-12-27 17:00:03,327][20381] Updated weights for policy 1, policy_version 1637451 (0.0010) [2023-12-27 17:00:03,351][20347] Updated weights for policy 0, policy_version 1639041 (0.0010) [2023-12-27 17:00:03,406][20347] Updated weights for policy 0, policy_version 1639051 (0.0010) [2023-12-27 17:00:04,010][20381] Updated weights for policy 1, policy_version 1637461 (0.0008) [2023-12-27 17:00:04,046][20347] Updated weights for policy 0, policy_version 1639061 (0.0007) [2023-12-27 17:00:04,062][20381] Updated weights for policy 1, policy_version 1637471 (0.0008) [2023-12-27 17:00:04,111][20347] Updated weights for policy 0, policy_version 1639071 (0.0007) [2023-12-27 17:00:04,126][20381] Updated weights for policy 1, policy_version 1637481 (0.0008) [2023-12-27 17:00:04,171][20347] Updated weights for policy 0, policy_version 1639081 (0.0007) [2023-12-27 17:00:04,761][20381] Updated weights for policy 1, policy_version 1637491 (0.0007) [2023-12-27 17:00:04,822][20381] Updated weights for policy 1, policy_version 1637501 (0.0005) [2023-12-27 17:00:04,876][20347] Updated weights for policy 0, policy_version 1639091 (0.0007) [2023-12-27 17:00:04,880][20381] Updated weights for policy 1, policy_version 1637511 (0.0010) [2023-12-27 17:00:04,929][20347] Updated weights for policy 0, policy_version 1639101 (0.0005) [2023-12-27 17:00:04,980][20347] Updated weights for policy 0, policy_version 1639111 (0.0006) [2023-12-27 17:00:05,500][20381] Updated weights for policy 1, policy_version 1637521 (0.0010) [2023-12-27 17:00:05,553][20381] Updated weights for policy 1, policy_version 1637531 (0.0008) [2023-12-27 17:00:05,599][20381] Updated weights for policy 1, policy_version 1637541 (0.0007) [2023-12-27 17:00:05,648][20381] Updated weights for policy 1, policy_version 1637551 (0.0006) [2023-12-27 17:00:05,669][20347] Updated weights for policy 0, policy_version 1639121 (0.0007) [2023-12-27 17:00:05,724][20347] Updated weights for policy 0, policy_version 1639131 (0.0006) [2023-12-27 17:00:05,785][20347] Updated weights for policy 0, policy_version 1639141 (0.0006) [2023-12-27 17:00:05,845][20347] Updated weights for policy 0, policy_version 1639151 (0.0007) [2023-12-27 17:00:06,333][20381] Updated weights for policy 1, policy_version 1637561 (0.0010) [2023-12-27 17:00:06,383][20381] Updated weights for policy 1, policy_version 1637571 (0.0011) [2023-12-27 17:00:06,440][20381] Updated weights for policy 1, policy_version 1637581 (0.0011) [2023-12-27 17:00:06,537][20347] Updated weights for policy 0, policy_version 1639161 (0.0007) [2023-12-27 17:00:06,600][20347] Updated weights for policy 0, policy_version 1639171 (0.0007) [2023-12-27 17:00:06,659][20347] Updated weights for policy 0, policy_version 1639181 (0.0007) [2023-12-27 17:00:07,128][20381] Updated weights for policy 1, policy_version 1637591 (0.0008) [2023-12-27 17:00:07,187][20381] Updated weights for policy 1, policy_version 1637601 (0.0007) [2023-12-27 17:00:07,244][20381] Updated weights for policy 1, policy_version 1637611 (0.0007) [2023-12-27 17:00:07,359][20347] Updated weights for policy 0, policy_version 1639191 (0.0009) [2023-12-27 17:00:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 838983680. Throughput: 0: 10352.2, 1: 10441.8. Samples: 838952756. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 17:00:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.070')] [2023-12-27 17:00:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001637616_419274752.pth... [2023-12-27 17:00:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001636400_418963456.pth [2023-12-27 17:00:07,410][20347] Updated weights for policy 0, policy_version 1639201 (0.0010) [2023-12-27 17:00:07,459][20347] Updated weights for policy 0, policy_version 1639211 (0.0010) [2023-12-27 17:00:07,480][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001639216_419717120.pth... [2023-12-27 17:00:07,483][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001638000_419405824.pth [2023-12-27 17:00:07,785][20381] Updated weights for policy 1, policy_version 1637621 (0.0005) [2023-12-27 17:00:07,839][20381] Updated weights for policy 1, policy_version 1637631 (0.0007) [2023-12-27 17:00:07,894][20381] Updated weights for policy 1, policy_version 1637641 (0.0007) [2023-12-27 17:00:08,115][20347] Updated weights for policy 0, policy_version 1639221 (0.0008) [2023-12-27 17:00:08,170][20347] Updated weights for policy 0, policy_version 1639231 (0.0005) [2023-12-27 17:00:08,229][20347] Updated weights for policy 0, policy_version 1639241 (0.0005) [2023-12-27 17:00:08,566][20381] Updated weights for policy 1, policy_version 1637651 (0.0006) [2023-12-27 17:00:08,630][20381] Updated weights for policy 1, policy_version 1637661 (0.0008) [2023-12-27 17:00:08,691][20381] Updated weights for policy 1, policy_version 1637671 (0.0008) [2023-12-27 17:00:08,872][20347] Updated weights for policy 0, policy_version 1639251 (0.0006) [2023-12-27 17:00:08,925][20347] Updated weights for policy 0, policy_version 1639261 (0.0008) [2023-12-27 17:00:08,971][20347] Updated weights for policy 0, policy_version 1639271 (0.0008) [2023-12-27 17:00:09,396][20381] Updated weights for policy 1, policy_version 1637681 (0.0008) [2023-12-27 17:00:09,461][20381] Updated weights for policy 1, policy_version 1637691 (0.0007) [2023-12-27 17:00:09,520][20381] Updated weights for policy 1, policy_version 1637701 (0.0008) [2023-12-27 17:00:09,577][20381] Updated weights for policy 1, policy_version 1637711 (0.0008) [2023-12-27 17:00:09,761][20347] Updated weights for policy 0, policy_version 1639281 (0.0007) [2023-12-27 17:00:09,816][20347] Updated weights for policy 0, policy_version 1639291 (0.0009) [2023-12-27 17:00:09,886][20347] Updated weights for policy 0, policy_version 1639301 (0.0007) [2023-12-27 17:00:09,950][20347] Updated weights for policy 0, policy_version 1639311 (0.0008) [2023-12-27 17:00:10,322][20381] Updated weights for policy 1, policy_version 1637721 (0.0007) [2023-12-27 17:00:10,382][20381] Updated weights for policy 1, policy_version 1637731 (0.0008) [2023-12-27 17:00:10,440][20381] Updated weights for policy 1, policy_version 1637741 (0.0007) [2023-12-27 17:00:10,611][20347] Updated weights for policy 0, policy_version 1639321 (0.0008) [2023-12-27 17:00:10,669][20347] Updated weights for policy 0, policy_version 1639331 (0.0008) [2023-12-27 17:00:10,728][20347] Updated weights for policy 0, policy_version 1639341 (0.0008) [2023-12-27 17:00:11,239][20381] Updated weights for policy 1, policy_version 1637751 (0.0008) [2023-12-27 17:00:11,295][20381] Updated weights for policy 1, policy_version 1637761 (0.0008) [2023-12-27 17:00:11,351][20381] Updated weights for policy 1, policy_version 1637771 (0.0008) [2023-12-27 17:00:11,440][20347] Updated weights for policy 0, policy_version 1639351 (0.0007) [2023-12-27 17:00:11,502][20347] Updated weights for policy 0, policy_version 1639361 (0.0006) [2023-12-27 17:00:11,560][20347] Updated weights for policy 0, policy_version 1639371 (0.0007) [2023-12-27 17:00:12,134][20381] Updated weights for policy 1, policy_version 1637781 (0.0008) [2023-12-27 17:00:12,191][20381] Updated weights for policy 1, policy_version 1637791 (0.0007) [2023-12-27 17:00:12,193][20347] Updated weights for policy 0, policy_version 1639381 (0.0005) [2023-12-27 17:00:12,244][20381] Updated weights for policy 1, policy_version 1637801 (0.0008) [2023-12-27 17:00:12,249][20347] Updated weights for policy 0, policy_version 1639391 (0.0006) [2023-12-27 17:00:12,304][20347] Updated weights for policy 0, policy_version 1639401 (0.0007) [2023-12-27 17:00:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.2, 300 sec: 20605.0). Total num frames: 839090176. Throughput: 0: 10328.8, 1: 10412.1. Samples: 839074444. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 17:00:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.750')] [2023-12-27 17:00:12,921][20381] Updated weights for policy 1, policy_version 1637811 (0.0007) [2023-12-27 17:00:12,978][20381] Updated weights for policy 1, policy_version 1637821 (0.0008) [2023-12-27 17:00:13,002][20347] Updated weights for policy 0, policy_version 1639411 (0.0008) [2023-12-27 17:00:13,033][20381] Updated weights for policy 1, policy_version 1637831 (0.0006) [2023-12-27 17:00:13,058][20347] Updated weights for policy 0, policy_version 1639421 (0.0005) [2023-12-27 17:00:13,112][20347] Updated weights for policy 0, policy_version 1639431 (0.0007) [2023-12-27 17:00:13,753][20347] Updated weights for policy 0, policy_version 1639441 (0.0006) [2023-12-27 17:00:13,755][20381] Updated weights for policy 1, policy_version 1637841 (0.0006) [2023-12-27 17:00:13,810][20381] Updated weights for policy 1, policy_version 1637851 (0.0007) [2023-12-27 17:00:13,822][20347] Updated weights for policy 0, policy_version 1639451 (0.0009) [2023-12-27 17:00:13,862][20381] Updated weights for policy 1, policy_version 1637861 (0.0007) [2023-12-27 17:00:13,868][20347] Updated weights for policy 0, policy_version 1639461 (0.0008) [2023-12-27 17:00:13,919][20381] Updated weights for policy 1, policy_version 1637871 (0.0005) [2023-12-27 17:00:13,927][20347] Updated weights for policy 0, policy_version 1639471 (0.0007) [2023-12-27 17:00:14,564][20381] Updated weights for policy 1, policy_version 1637881 (0.0007) [2023-12-27 17:00:14,614][20347] Updated weights for policy 0, policy_version 1639481 (0.0006) [2023-12-27 17:00:14,617][20381] Updated weights for policy 1, policy_version 1637891 (0.0006) [2023-12-27 17:00:14,662][20347] Updated weights for policy 0, policy_version 1639491 (0.0005) [2023-12-27 17:00:14,672][20381] Updated weights for policy 1, policy_version 1637901 (0.0007) [2023-12-27 17:00:14,716][20347] Updated weights for policy 0, policy_version 1639501 (0.0007) [2023-12-27 17:00:15,392][20381] Updated weights for policy 1, policy_version 1637911 (0.0006) [2023-12-27 17:00:15,430][20347] Updated weights for policy 0, policy_version 1639511 (0.0007) [2023-12-27 17:00:15,453][20381] Updated weights for policy 1, policy_version 1637921 (0.0009) [2023-12-27 17:00:15,485][20347] Updated weights for policy 0, policy_version 1639521 (0.0006) [2023-12-27 17:00:15,515][20381] Updated weights for policy 1, policy_version 1637931 (0.0006) [2023-12-27 17:00:15,542][20347] Updated weights for policy 0, policy_version 1639531 (0.0006) [2023-12-27 17:00:16,192][20347] Updated weights for policy 0, policy_version 1639541 (0.0006) [2023-12-27 17:00:16,200][20381] Updated weights for policy 1, policy_version 1637941 (0.0006) [2023-12-27 17:00:16,246][20347] Updated weights for policy 0, policy_version 1639551 (0.0006) [2023-12-27 17:00:16,256][20381] Updated weights for policy 1, policy_version 1637951 (0.0008) [2023-12-27 17:00:16,303][20347] Updated weights for policy 0, policy_version 1639561 (0.0005) [2023-12-27 17:00:16,311][20381] Updated weights for policy 1, policy_version 1637961 (0.0011) [2023-12-27 17:00:16,877][20347] Updated weights for policy 0, policy_version 1639571 (0.0006) [2023-12-27 17:00:16,928][20347] Updated weights for policy 0, policy_version 1639581 (0.0008) [2023-12-27 17:00:16,958][20381] Updated weights for policy 1, policy_version 1637971 (0.0009) [2023-12-27 17:00:16,980][20347] Updated weights for policy 0, policy_version 1639591 (0.0008) [2023-12-27 17:00:17,005][20381] Updated weights for policy 1, policy_version 1637981 (0.0006) [2023-12-27 17:00:17,055][20381] Updated weights for policy 1, policy_version 1637991 (0.0007) [2023-12-27 17:00:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 839188480. Throughput: 0: 10331.6, 1: 10248.5. Samples: 839196348. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 17:00:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.400')] [2023-12-27 17:00:17,624][20381] Updated weights for policy 1, policy_version 1638001 (0.0007) [2023-12-27 17:00:17,678][20381] Updated weights for policy 1, policy_version 1638011 (0.0005) [2023-12-27 17:00:17,724][20381] Updated weights for policy 1, policy_version 1638021 (0.0005) [2023-12-27 17:00:17,724][20347] Updated weights for policy 0, policy_version 1639601 (0.0006) [2023-12-27 17:00:17,774][20381] Updated weights for policy 1, policy_version 1638031 (0.0007) [2023-12-27 17:00:17,780][20347] Updated weights for policy 0, policy_version 1639611 (0.0006) [2023-12-27 17:00:17,841][20347] Updated weights for policy 0, policy_version 1639621 (0.0007) [2023-12-27 17:00:17,894][20347] Updated weights for policy 0, policy_version 1639631 (0.0008) [2023-12-27 17:00:18,373][20381] Updated weights for policy 1, policy_version 1638041 (0.0008) [2023-12-27 17:00:18,429][20381] Updated weights for policy 1, policy_version 1638051 (0.0008) [2023-12-27 17:00:18,489][20381] Updated weights for policy 1, policy_version 1638061 (0.0005) [2023-12-27 17:00:18,628][20347] Updated weights for policy 0, policy_version 1639641 (0.0008) [2023-12-27 17:00:18,696][20347] Updated weights for policy 0, policy_version 1639651 (0.0008) [2023-12-27 17:00:18,781][20347] Updated weights for policy 0, policy_version 1639661 (0.0008) [2023-12-27 17:00:19,103][20381] Updated weights for policy 1, policy_version 1638071 (0.0006) [2023-12-27 17:00:19,155][20381] Updated weights for policy 1, policy_version 1638081 (0.0008) [2023-12-27 17:00:19,204][20381] Updated weights for policy 1, policy_version 1638091 (0.0008) [2023-12-27 17:00:19,456][20347] Updated weights for policy 0, policy_version 1639671 (0.0007) [2023-12-27 17:00:19,521][20347] Updated weights for policy 0, policy_version 1639681 (0.0006) [2023-12-27 17:00:19,580][20347] Updated weights for policy 0, policy_version 1639691 (0.0008) [2023-12-27 17:00:19,998][20381] Updated weights for policy 1, policy_version 1638101 (0.0007) [2023-12-27 17:00:20,061][20381] Updated weights for policy 1, policy_version 1638111 (0.0006) [2023-12-27 17:00:20,124][20381] Updated weights for policy 1, policy_version 1638121 (0.0006) [2023-12-27 17:00:20,349][20347] Updated weights for policy 0, policy_version 1639701 (0.0008) [2023-12-27 17:00:20,401][20347] Updated weights for policy 0, policy_version 1639711 (0.0008) [2023-12-27 17:00:20,454][20347] Updated weights for policy 0, policy_version 1639721 (0.0008) [2023-12-27 17:00:20,733][20381] Updated weights for policy 1, policy_version 1638131 (0.0007) [2023-12-27 17:00:20,798][20381] Updated weights for policy 1, policy_version 1638141 (0.0008) [2023-12-27 17:00:20,856][20381] Updated weights for policy 1, policy_version 1638151 (0.0008) [2023-12-27 17:00:21,206][20347] Updated weights for policy 0, policy_version 1639731 (0.0009) [2023-12-27 17:00:21,270][20347] Updated weights for policy 0, policy_version 1639741 (0.0007) [2023-12-27 17:00:21,326][20347] Updated weights for policy 0, policy_version 1639751 (0.0007) [2023-12-27 17:00:21,554][20381] Updated weights for policy 1, policy_version 1638161 (0.0008) [2023-12-27 17:00:21,620][20381] Updated weights for policy 1, policy_version 1638171 (0.0007) [2023-12-27 17:00:21,689][20381] Updated weights for policy 1, policy_version 1638181 (0.0008) [2023-12-27 17:00:21,758][20381] Updated weights for policy 1, policy_version 1638191 (0.0009) [2023-12-27 17:00:22,100][20347] Updated weights for policy 0, policy_version 1639761 (0.0008) [2023-12-27 17:00:22,158][20347] Updated weights for policy 0, policy_version 1639771 (0.0008) [2023-12-27 17:00:22,218][20347] Updated weights for policy 0, policy_version 1639781 (0.0008) [2023-12-27 17:00:22,278][20347] Updated weights for policy 0, policy_version 1639791 (0.0006) [2023-12-27 17:00:22,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 839286784. Throughput: 0: 10238.6, 1: 10261.7. Samples: 839257100. Policy #0 lag: (min: 18.0, avg: 26.9, max: 50.0) [2023-12-27 17:00:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.820')] [2023-12-27 17:00:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001638192_419422208.pth... [2023-12-27 17:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001639792_419864576.pth... [2023-12-27 17:00:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001638608_419561472.pth [2023-12-27 17:00:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001637008_419119104.pth [2023-12-27 17:00:22,475][20381] Updated weights for policy 1, policy_version 1638201 (0.0007) [2023-12-27 17:00:22,529][20381] Updated weights for policy 1, policy_version 1638211 (0.0008) [2023-12-27 17:00:22,579][20381] Updated weights for policy 1, policy_version 1638221 (0.0009) [2023-12-27 17:00:22,950][20347] Updated weights for policy 0, policy_version 1639801 (0.0007) [2023-12-27 17:00:23,010][20347] Updated weights for policy 0, policy_version 1639811 (0.0008) [2023-12-27 17:00:23,070][20347] Updated weights for policy 0, policy_version 1639821 (0.0007) [2023-12-27 17:00:23,318][20381] Updated weights for policy 1, policy_version 1638231 (0.0006) [2023-12-27 17:00:23,363][20381] Updated weights for policy 1, policy_version 1638241 (0.0005) [2023-12-27 17:00:23,416][20381] Updated weights for policy 1, policy_version 1638251 (0.0005) [2023-12-27 17:00:23,784][20347] Updated weights for policy 0, policy_version 1639831 (0.0007) [2023-12-27 17:00:23,836][20347] Updated weights for policy 0, policy_version 1639841 (0.0007) [2023-12-27 17:00:23,887][20347] Updated weights for policy 0, policy_version 1639851 (0.0007) [2023-12-27 17:00:24,042][20381] Updated weights for policy 1, policy_version 1638261 (0.0008) [2023-12-27 17:00:24,102][20381] Updated weights for policy 1, policy_version 1638271 (0.0010) [2023-12-27 17:00:24,150][20381] Updated weights for policy 1, policy_version 1638281 (0.0010) [2023-12-27 17:00:24,578][20347] Updated weights for policy 0, policy_version 1639861 (0.0005) [2023-12-27 17:00:24,626][20347] Updated weights for policy 0, policy_version 1639871 (0.0005) [2023-12-27 17:00:24,679][20347] Updated weights for policy 0, policy_version 1639881 (0.0005) [2023-12-27 17:00:24,829][20381] Updated weights for policy 1, policy_version 1638291 (0.0010) [2023-12-27 17:00:24,891][20381] Updated weights for policy 1, policy_version 1638301 (0.0011) [2023-12-27 17:00:24,958][20381] Updated weights for policy 1, policy_version 1638311 (0.0010) [2023-12-27 17:00:25,312][20347] Updated weights for policy 0, policy_version 1639891 (0.0006) [2023-12-27 17:00:25,377][20347] Updated weights for policy 0, policy_version 1639901 (0.0007) [2023-12-27 17:00:25,432][20347] Updated weights for policy 0, policy_version 1639911 (0.0007) [2023-12-27 17:00:25,645][20381] Updated weights for policy 1, policy_version 1638321 (0.0009) [2023-12-27 17:00:25,698][20381] Updated weights for policy 1, policy_version 1638331 (0.0008) [2023-12-27 17:00:25,756][20381] Updated weights for policy 1, policy_version 1638341 (0.0008) [2023-12-27 17:00:25,810][20381] Updated weights for policy 1, policy_version 1638351 (0.0006) [2023-12-27 17:00:26,038][20347] Updated weights for policy 0, policy_version 1639921 (0.0008) [2023-12-27 17:00:26,089][20347] Updated weights for policy 0, policy_version 1639931 (0.0006) [2023-12-27 17:00:26,139][20347] Updated weights for policy 0, policy_version 1639941 (0.0007) [2023-12-27 17:00:26,196][20347] Updated weights for policy 0, policy_version 1639951 (0.0009) [2023-12-27 17:00:26,500][20381] Updated weights for policy 1, policy_version 1638361 (0.0007) [2023-12-27 17:00:26,554][20381] Updated weights for policy 1, policy_version 1638371 (0.0007) [2023-12-27 17:00:26,611][20381] Updated weights for policy 1, policy_version 1638381 (0.0007) [2023-12-27 17:00:26,834][20347] Updated weights for policy 0, policy_version 1639961 (0.0006) [2023-12-27 17:00:26,884][20347] Updated weights for policy 0, policy_version 1639971 (0.0005) [2023-12-27 17:00:26,941][20347] Updated weights for policy 0, policy_version 1639981 (0.0005) [2023-12-27 17:00:27,296][20381] Updated weights for policy 1, policy_version 1638391 (0.0007) [2023-12-27 17:00:27,352][20381] Updated weights for policy 1, policy_version 1638401 (0.0005) [2023-12-27 17:00:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 839385088. Throughput: 0: 10198.6, 1: 10279.4. Samples: 839377652. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.550')] [2023-12-27 17:00:27,414][20381] Updated weights for policy 1, policy_version 1638411 (0.0005) [2023-12-27 17:00:27,511][20347] Updated weights for policy 0, policy_version 1639991 (0.0005) [2023-12-27 17:00:27,563][20347] Updated weights for policy 0, policy_version 1640001 (0.0007) [2023-12-27 17:00:27,608][20347] Updated weights for policy 0, policy_version 1640011 (0.0009) [2023-12-27 17:00:28,063][20381] Updated weights for policy 1, policy_version 1638421 (0.0006) [2023-12-27 17:00:28,113][20381] Updated weights for policy 1, policy_version 1638431 (0.0007) [2023-12-27 17:00:28,168][20381] Updated weights for policy 1, policy_version 1638441 (0.0007) [2023-12-27 17:00:28,198][20347] Updated weights for policy 0, policy_version 1640021 (0.0008) [2023-12-27 17:00:28,249][20347] Updated weights for policy 0, policy_version 1640031 (0.0005) [2023-12-27 17:00:28,306][20347] Updated weights for policy 0, policy_version 1640041 (0.0005) [2023-12-27 17:00:28,852][20381] Updated weights for policy 1, policy_version 1638451 (0.0007) [2023-12-27 17:00:28,902][20381] Updated weights for policy 1, policy_version 1638461 (0.0007) [2023-12-27 17:00:28,953][20381] Updated weights for policy 1, policy_version 1638471 (0.0006) [2023-12-27 17:00:28,966][20347] Updated weights for policy 0, policy_version 1640051 (0.0009) [2023-12-27 17:00:29,016][20347] Updated weights for policy 0, policy_version 1640061 (0.0010) [2023-12-27 17:00:29,074][20347] Updated weights for policy 0, policy_version 1640071 (0.0010) [2023-12-27 17:00:29,588][20381] Updated weights for policy 1, policy_version 1638481 (0.0005) [2023-12-27 17:00:29,656][20381] Updated weights for policy 1, policy_version 1638491 (0.0007) [2023-12-27 17:00:29,717][20381] Updated weights for policy 1, policy_version 1638501 (0.0008) [2023-12-27 17:00:29,756][20347] Updated weights for policy 0, policy_version 1640081 (0.0006) [2023-12-27 17:00:29,778][20381] Updated weights for policy 1, policy_version 1638511 (0.0007) [2023-12-27 17:00:29,814][20347] Updated weights for policy 0, policy_version 1640091 (0.0006) [2023-12-27 17:00:29,876][20347] Updated weights for policy 0, policy_version 1640101 (0.0008) [2023-12-27 17:00:29,940][20347] Updated weights for policy 0, policy_version 1640111 (0.0006) [2023-12-27 17:00:30,498][20381] Updated weights for policy 1, policy_version 1638521 (0.0008) [2023-12-27 17:00:30,551][20381] Updated weights for policy 1, policy_version 1638531 (0.0008) [2023-12-27 17:00:30,589][20347] Updated weights for policy 0, policy_version 1640121 (0.0006) [2023-12-27 17:00:30,603][20381] Updated weights for policy 1, policy_version 1638541 (0.0006) [2023-12-27 17:00:30,640][20347] Updated weights for policy 0, policy_version 1640131 (0.0007) [2023-12-27 17:00:30,697][20347] Updated weights for policy 0, policy_version 1640141 (0.0007) [2023-12-27 17:00:31,271][20381] Updated weights for policy 1, policy_version 1638551 (0.0008) [2023-12-27 17:00:31,334][20381] Updated weights for policy 1, policy_version 1638561 (0.0009) [2023-12-27 17:00:31,388][20347] Updated weights for policy 0, policy_version 1640151 (0.0009) [2023-12-27 17:00:31,391][20381] Updated weights for policy 1, policy_version 1638571 (0.0006) [2023-12-27 17:00:31,440][20347] Updated weights for policy 0, policy_version 1640161 (0.0010) [2023-12-27 17:00:31,493][20347] Updated weights for policy 0, policy_version 1640171 (0.0008) [2023-12-27 17:00:32,117][20381] Updated weights for policy 1, policy_version 1638581 (0.0007) [2023-12-27 17:00:32,176][20381] Updated weights for policy 1, policy_version 1638591 (0.0007) [2023-12-27 17:00:32,188][20347] Updated weights for policy 0, policy_version 1640181 (0.0010) [2023-12-27 17:00:32,233][20381] Updated weights for policy 1, policy_version 1638601 (0.0007) [2023-12-27 17:00:32,243][20347] Updated weights for policy 0, policy_version 1640191 (0.0010) [2023-12-27 17:00:32,300][20347] Updated weights for policy 0, policy_version 1640201 (0.0007) [2023-12-27 17:00:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 839499776. Throughput: 0: 10184.5, 1: 10248.8. Samples: 839502420. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.790')] [2023-12-27 17:00:32,879][20381] Updated weights for policy 1, policy_version 1638611 (0.0008) [2023-12-27 17:00:32,936][20381] Updated weights for policy 1, policy_version 1638621 (0.0008) [2023-12-27 17:00:33,001][20381] Updated weights for policy 1, policy_version 1638631 (0.0010) [2023-12-27 17:00:33,006][20347] Updated weights for policy 0, policy_version 1640211 (0.0006) [2023-12-27 17:00:33,057][20347] Updated weights for policy 0, policy_version 1640221 (0.0005) [2023-12-27 17:00:33,113][20347] Updated weights for policy 0, policy_version 1640231 (0.0007) [2023-12-27 17:00:33,655][20381] Updated weights for policy 1, policy_version 1638641 (0.0010) [2023-12-27 17:00:33,709][20381] Updated weights for policy 1, policy_version 1638651 (0.0006) [2023-12-27 17:00:33,766][20381] Updated weights for policy 1, policy_version 1638661 (0.0007) [2023-12-27 17:00:33,802][20347] Updated weights for policy 0, policy_version 1640241 (0.0008) [2023-12-27 17:00:33,815][20381] Updated weights for policy 1, policy_version 1638671 (0.0006) [2023-12-27 17:00:33,852][20347] Updated weights for policy 0, policy_version 1640251 (0.0009) [2023-12-27 17:00:33,904][20347] Updated weights for policy 0, policy_version 1640261 (0.0005) [2023-12-27 17:00:33,959][20347] Updated weights for policy 0, policy_version 1640271 (0.0009) [2023-12-27 17:00:34,436][20381] Updated weights for policy 1, policy_version 1638681 (0.0008) [2023-12-27 17:00:34,497][20381] Updated weights for policy 1, policy_version 1638691 (0.0008) [2023-12-27 17:00:34,565][20381] Updated weights for policy 1, policy_version 1638701 (0.0008) [2023-12-27 17:00:34,647][20347] Updated weights for policy 0, policy_version 1640281 (0.0010) [2023-12-27 17:00:34,706][20347] Updated weights for policy 0, policy_version 1640291 (0.0008) [2023-12-27 17:00:34,767][20347] Updated weights for policy 0, policy_version 1640301 (0.0006) [2023-12-27 17:00:35,225][20381] Updated weights for policy 1, policy_version 1638711 (0.0007) [2023-12-27 17:00:35,287][20381] Updated weights for policy 1, policy_version 1638721 (0.0006) [2023-12-27 17:00:35,343][20381] Updated weights for policy 1, policy_version 1638731 (0.0005) [2023-12-27 17:00:35,422][20347] Updated weights for policy 0, policy_version 1640311 (0.0009) [2023-12-27 17:00:35,478][20347] Updated weights for policy 0, policy_version 1640321 (0.0007) [2023-12-27 17:00:35,523][20347] Updated weights for policy 0, policy_version 1640331 (0.0008) [2023-12-27 17:00:35,922][20381] Updated weights for policy 1, policy_version 1638741 (0.0005) [2023-12-27 17:00:35,980][20381] Updated weights for policy 1, policy_version 1638751 (0.0006) [2023-12-27 17:00:36,032][20381] Updated weights for policy 1, policy_version 1638761 (0.0008) [2023-12-27 17:00:36,170][20347] Updated weights for policy 0, policy_version 1640341 (0.0008) [2023-12-27 17:00:36,232][20347] Updated weights for policy 0, policy_version 1640351 (0.0008) [2023-12-27 17:00:36,291][20347] Updated weights for policy 0, policy_version 1640361 (0.0007) [2023-12-27 17:00:36,737][20381] Updated weights for policy 1, policy_version 1638771 (0.0008) [2023-12-27 17:00:36,796][20381] Updated weights for policy 1, policy_version 1638781 (0.0008) [2023-12-27 17:00:36,848][20381] Updated weights for policy 1, policy_version 1638791 (0.0007) [2023-12-27 17:00:37,022][20347] Updated weights for policy 0, policy_version 1640371 (0.0008) [2023-12-27 17:00:37,083][20347] Updated weights for policy 0, policy_version 1640381 (0.0008) [2023-12-27 17:00:37,138][20347] Updated weights for policy 0, policy_version 1640391 (0.0008) [2023-12-27 17:00:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 839598080. Throughput: 0: 10213.3, 1: 10238.4. Samples: 839565112. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '824.660')] [2023-12-27 17:00:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001638800_419577856.pth... [2023-12-27 17:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001640400_420020224.pth... [2023-12-27 17:00:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001637616_419274752.pth [2023-12-27 17:00:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001639216_419717120.pth [2023-12-27 17:00:37,488][20381] Updated weights for policy 1, policy_version 1638801 (0.0007) [2023-12-27 17:00:37,547][20381] Updated weights for policy 1, policy_version 1638811 (0.0007) [2023-12-27 17:00:37,597][20381] Updated weights for policy 1, policy_version 1638821 (0.0008) [2023-12-27 17:00:37,657][20381] Updated weights for policy 1, policy_version 1638831 (0.0006) [2023-12-27 17:00:37,874][20347] Updated weights for policy 0, policy_version 1640401 (0.0005) [2023-12-27 17:00:37,927][20347] Updated weights for policy 0, policy_version 1640411 (0.0008) [2023-12-27 17:00:37,979][20347] Updated weights for policy 0, policy_version 1640421 (0.0008) [2023-12-27 17:00:38,035][20347] Updated weights for policy 0, policy_version 1640431 (0.0008) [2023-12-27 17:00:38,282][20381] Updated weights for policy 1, policy_version 1638841 (0.0008) [2023-12-27 17:00:38,347][20381] Updated weights for policy 1, policy_version 1638851 (0.0008) [2023-12-27 17:00:38,408][20381] Updated weights for policy 1, policy_version 1638861 (0.0009) [2023-12-27 17:00:38,768][20347] Updated weights for policy 0, policy_version 1640441 (0.0008) [2023-12-27 17:00:38,826][20347] Updated weights for policy 0, policy_version 1640451 (0.0007) [2023-12-27 17:00:38,888][20347] Updated weights for policy 0, policy_version 1640461 (0.0008) [2023-12-27 17:00:39,115][20381] Updated weights for policy 1, policy_version 1638871 (0.0009) [2023-12-27 17:00:39,168][20381] Updated weights for policy 1, policy_version 1638881 (0.0008) [2023-12-27 17:00:39,233][20381] Updated weights for policy 1, policy_version 1638891 (0.0008) [2023-12-27 17:00:39,618][20347] Updated weights for policy 0, policy_version 1640471 (0.0008) [2023-12-27 17:00:39,670][20347] Updated weights for policy 0, policy_version 1640481 (0.0007) [2023-12-27 17:00:39,729][20347] Updated weights for policy 0, policy_version 1640491 (0.0007) [2023-12-27 17:00:40,043][20381] Updated weights for policy 1, policy_version 1638901 (0.0008) [2023-12-27 17:00:40,106][20381] Updated weights for policy 1, policy_version 1638911 (0.0007) [2023-12-27 17:00:40,173][20381] Updated weights for policy 1, policy_version 1638921 (0.0008) [2023-12-27 17:00:40,442][20347] Updated weights for policy 0, policy_version 1640501 (0.0007) [2023-12-27 17:00:40,502][20347] Updated weights for policy 0, policy_version 1640511 (0.0008) [2023-12-27 17:00:40,556][20347] Updated weights for policy 0, policy_version 1640521 (0.0008) [2023-12-27 17:00:40,876][20381] Updated weights for policy 1, policy_version 1638931 (0.0007) [2023-12-27 17:00:40,936][20381] Updated weights for policy 1, policy_version 1638941 (0.0007) [2023-12-27 17:00:40,988][20381] Updated weights for policy 1, policy_version 1638951 (0.0008) [2023-12-27 17:00:41,331][20347] Updated weights for policy 0, policy_version 1640531 (0.0007) [2023-12-27 17:00:41,397][20347] Updated weights for policy 0, policy_version 1640541 (0.0008) [2023-12-27 17:00:41,459][20347] Updated weights for policy 0, policy_version 1640551 (0.0008) [2023-12-27 17:00:41,827][20381] Updated weights for policy 1, policy_version 1638961 (0.0007) [2023-12-27 17:00:41,886][20381] Updated weights for policy 1, policy_version 1638971 (0.0009) [2023-12-27 17:00:41,949][20381] Updated weights for policy 1, policy_version 1638981 (0.0009) [2023-12-27 17:00:42,004][20381] Updated weights for policy 1, policy_version 1638991 (0.0009) [2023-12-27 17:00:42,362][19170] Fps is (10 sec: 18841.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 839688192. Throughput: 0: 10163.1, 1: 10176.5. Samples: 839683568. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '815.190')] [2023-12-27 17:00:42,393][20347] Updated weights for policy 0, policy_version 1640561 (0.0008) [2023-12-27 17:00:42,446][20347] Updated weights for policy 0, policy_version 1640571 (0.0009) [2023-12-27 17:00:42,496][20347] Updated weights for policy 0, policy_version 1640581 (0.0009) [2023-12-27 17:00:42,556][20347] Updated weights for policy 0, policy_version 1640591 (0.0010) [2023-12-27 17:00:42,898][20381] Updated weights for policy 1, policy_version 1639001 (0.0009) [2023-12-27 17:00:42,956][20381] Updated weights for policy 1, policy_version 1639011 (0.0009) [2023-12-27 17:00:43,009][20381] Updated weights for policy 1, policy_version 1639021 (0.0009) [2023-12-27 17:00:43,432][20347] Updated weights for policy 0, policy_version 1640601 (0.0010) [2023-12-27 17:00:43,484][20347] Updated weights for policy 0, policy_version 1640611 (0.0010) [2023-12-27 17:00:43,539][20347] Updated weights for policy 0, policy_version 1640621 (0.0009) [2023-12-27 17:00:43,697][20381] Updated weights for policy 1, policy_version 1639031 (0.0010) [2023-12-27 17:00:43,758][20381] Updated weights for policy 1, policy_version 1639041 (0.0011) [2023-12-27 17:00:43,814][20381] Updated weights for policy 1, policy_version 1639051 (0.0011) [2023-12-27 17:00:44,242][20347] Updated weights for policy 0, policy_version 1640631 (0.0008) [2023-12-27 17:00:44,304][20347] Updated weights for policy 0, policy_version 1640641 (0.0006) [2023-12-27 17:00:44,370][20347] Updated weights for policy 0, policy_version 1640651 (0.0006) [2023-12-27 17:00:44,465][20381] Updated weights for policy 1, policy_version 1639061 (0.0009) [2023-12-27 17:00:44,524][20381] Updated weights for policy 1, policy_version 1639071 (0.0008) [2023-12-27 17:00:44,579][20381] Updated weights for policy 1, policy_version 1639081 (0.0007) [2023-12-27 17:00:45,062][20347] Updated weights for policy 0, policy_version 1640661 (0.0008) [2023-12-27 17:00:45,123][20347] Updated weights for policy 0, policy_version 1640671 (0.0009) [2023-12-27 17:00:45,182][20347] Updated weights for policy 0, policy_version 1640681 (0.0009) [2023-12-27 17:00:45,323][20381] Updated weights for policy 1, policy_version 1639091 (0.0010) [2023-12-27 17:00:45,383][20381] Updated weights for policy 1, policy_version 1639101 (0.0011) [2023-12-27 17:00:45,433][20381] Updated weights for policy 1, policy_version 1639111 (0.0011) [2023-12-27 17:00:45,894][20347] Updated weights for policy 0, policy_version 1640691 (0.0008) [2023-12-27 17:00:45,955][20347] Updated weights for policy 0, policy_version 1640701 (0.0008) [2023-12-27 17:00:46,010][20347] Updated weights for policy 0, policy_version 1640711 (0.0007) [2023-12-27 17:00:46,126][20381] Updated weights for policy 1, policy_version 1639121 (0.0011) [2023-12-27 17:00:46,181][20381] Updated weights for policy 1, policy_version 1639131 (0.0011) [2023-12-27 17:00:46,233][20381] Updated weights for policy 1, policy_version 1639141 (0.0011) [2023-12-27 17:00:46,286][20381] Updated weights for policy 1, policy_version 1639151 (0.0011) [2023-12-27 17:00:46,650][20347] Updated weights for policy 0, policy_version 1640721 (0.0007) [2023-12-27 17:00:46,704][20347] Updated weights for policy 0, policy_version 1640731 (0.0007) [2023-12-27 17:00:46,763][20347] Updated weights for policy 0, policy_version 1640741 (0.0006) [2023-12-27 17:00:46,815][20347] Updated weights for policy 0, policy_version 1640751 (0.0008) [2023-12-27 17:00:46,979][20381] Updated weights for policy 1, policy_version 1639161 (0.0008) [2023-12-27 17:00:47,036][20381] Updated weights for policy 1, policy_version 1639171 (0.0008) [2023-12-27 17:00:47,095][20381] Updated weights for policy 1, policy_version 1639181 (0.0007) [2023-12-27 17:00:47,362][19170] Fps is (10 sec: 18842.1, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 839786496. Throughput: 0: 10022.9, 1: 10098.4. Samples: 839795896. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.610')] [2023-12-27 17:00:47,461][20347] Updated weights for policy 0, policy_version 1640761 (0.0005) [2023-12-27 17:00:47,513][20347] Updated weights for policy 0, policy_version 1640771 (0.0008) [2023-12-27 17:00:47,573][20347] Updated weights for policy 0, policy_version 1640781 (0.0009) [2023-12-27 17:00:47,820][20381] Updated weights for policy 1, policy_version 1639191 (0.0007) [2023-12-27 17:00:47,874][20381] Updated weights for policy 1, policy_version 1639201 (0.0005) [2023-12-27 17:00:47,942][20381] Updated weights for policy 1, policy_version 1639211 (0.0007) [2023-12-27 17:00:48,201][20347] Updated weights for policy 0, policy_version 1640791 (0.0008) [2023-12-27 17:00:48,256][20347] Updated weights for policy 0, policy_version 1640801 (0.0007) [2023-12-27 17:00:48,314][20347] Updated weights for policy 0, policy_version 1640811 (0.0007) [2023-12-27 17:00:48,618][20381] Updated weights for policy 1, policy_version 1639221 (0.0009) [2023-12-27 17:00:48,680][20381] Updated weights for policy 1, policy_version 1639231 (0.0008) [2023-12-27 17:00:48,744][20381] Updated weights for policy 1, policy_version 1639241 (0.0007) [2023-12-27 17:00:48,954][20347] Updated weights for policy 0, policy_version 1640821 (0.0009) [2023-12-27 17:00:49,009][20347] Updated weights for policy 0, policy_version 1640831 (0.0007) [2023-12-27 17:00:49,067][20347] Updated weights for policy 0, policy_version 1640841 (0.0006) [2023-12-27 17:00:49,464][20381] Updated weights for policy 1, policy_version 1639251 (0.0006) [2023-12-27 17:00:49,533][20381] Updated weights for policy 1, policy_version 1639261 (0.0009) [2023-12-27 17:00:49,594][20381] Updated weights for policy 1, policy_version 1639271 (0.0007) [2023-12-27 17:00:49,745][20347] Updated weights for policy 0, policy_version 1640851 (0.0008) [2023-12-27 17:00:49,810][20347] Updated weights for policy 0, policy_version 1640861 (0.0008) [2023-12-27 17:00:49,876][20347] Updated weights for policy 0, policy_version 1640871 (0.0007) [2023-12-27 17:00:50,349][20381] Updated weights for policy 1, policy_version 1639281 (0.0007) [2023-12-27 17:00:50,405][20381] Updated weights for policy 1, policy_version 1639291 (0.0008) [2023-12-27 17:00:50,459][20381] Updated weights for policy 1, policy_version 1639301 (0.0008) [2023-12-27 17:00:50,517][20381] Updated weights for policy 1, policy_version 1639311 (0.0007) [2023-12-27 17:00:50,575][20347] Updated weights for policy 0, policy_version 1640881 (0.0006) [2023-12-27 17:00:50,632][20347] Updated weights for policy 0, policy_version 1640891 (0.0007) [2023-12-27 17:00:50,696][20347] Updated weights for policy 0, policy_version 1640901 (0.0009) [2023-12-27 17:00:50,758][20347] Updated weights for policy 0, policy_version 1640911 (0.0011) [2023-12-27 17:00:51,328][20381] Updated weights for policy 1, policy_version 1639321 (0.0008) [2023-12-27 17:00:51,392][20381] Updated weights for policy 1, policy_version 1639331 (0.0008) [2023-12-27 17:00:51,445][20381] Updated weights for policy 1, policy_version 1639341 (0.0008) [2023-12-27 17:00:51,514][20347] Updated weights for policy 0, policy_version 1640921 (0.0008) [2023-12-27 17:00:51,571][20347] Updated weights for policy 0, policy_version 1640931 (0.0007) [2023-12-27 17:00:51,632][20347] Updated weights for policy 0, policy_version 1640941 (0.0008) [2023-12-27 17:00:52,182][20381] Updated weights for policy 1, policy_version 1639351 (0.0007) [2023-12-27 17:00:52,243][20381] Updated weights for policy 1, policy_version 1639361 (0.0007) [2023-12-27 17:00:52,307][20381] Updated weights for policy 1, policy_version 1639371 (0.0008) [2023-12-27 17:00:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 839884800. Throughput: 0: 10030.5, 1: 10063.1. Samples: 839856968. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.150')] [2023-12-27 17:00:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001639376_419725312.pth... [2023-12-27 17:00:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001638192_419422208.pth [2023-12-27 17:00:52,381][20347] Updated weights for policy 0, policy_version 1640951 (0.0008) [2023-12-27 17:00:52,440][20347] Updated weights for policy 0, policy_version 1640961 (0.0008) [2023-12-27 17:00:52,499][20347] Updated weights for policy 0, policy_version 1640971 (0.0008) [2023-12-27 17:00:52,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001640976_420167680.pth... [2023-12-27 17:00:52,530][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001639792_419864576.pth [2023-12-27 17:00:52,893][20381] Updated weights for policy 1, policy_version 1639381 (0.0007) [2023-12-27 17:00:52,950][20381] Updated weights for policy 1, policy_version 1639391 (0.0005) [2023-12-27 17:00:53,016][20381] Updated weights for policy 1, policy_version 1639401 (0.0005) [2023-12-27 17:00:53,175][20347] Updated weights for policy 0, policy_version 1640981 (0.0006) [2023-12-27 17:00:53,222][20347] Updated weights for policy 0, policy_version 1640991 (0.0005) [2023-12-27 17:00:53,272][20347] Updated weights for policy 0, policy_version 1641001 (0.0005) [2023-12-27 17:00:53,572][20381] Updated weights for policy 1, policy_version 1639411 (0.0006) [2023-12-27 17:00:53,617][20381] Updated weights for policy 1, policy_version 1639421 (0.0007) [2023-12-27 17:00:53,671][20381] Updated weights for policy 1, policy_version 1639431 (0.0008) [2023-12-27 17:00:53,910][20347] Updated weights for policy 0, policy_version 1641011 (0.0009) [2023-12-27 17:00:53,963][20347] Updated weights for policy 0, policy_version 1641021 (0.0007) [2023-12-27 17:00:54,023][20347] Updated weights for policy 0, policy_version 1641031 (0.0007) [2023-12-27 17:00:54,316][20381] Updated weights for policy 1, policy_version 1639441 (0.0007) [2023-12-27 17:00:54,370][20381] Updated weights for policy 1, policy_version 1639451 (0.0009) [2023-12-27 17:00:54,418][20381] Updated weights for policy 1, policy_version 1639461 (0.0007) [2023-12-27 17:00:54,469][20381] Updated weights for policy 1, policy_version 1639471 (0.0007) [2023-12-27 17:00:54,744][20347] Updated weights for policy 0, policy_version 1641041 (0.0008) [2023-12-27 17:00:54,793][20347] Updated weights for policy 0, policy_version 1641051 (0.0008) [2023-12-27 17:00:54,835][20347] Updated weights for policy 0, policy_version 1641061 (0.0007) [2023-12-27 17:00:54,882][20347] Updated weights for policy 0, policy_version 1641071 (0.0008) [2023-12-27 17:00:55,195][20381] Updated weights for policy 1, policy_version 1639481 (0.0008) [2023-12-27 17:00:55,253][20381] Updated weights for policy 1, policy_version 1639491 (0.0008) [2023-12-27 17:00:55,312][20381] Updated weights for policy 1, policy_version 1639501 (0.0007) [2023-12-27 17:00:55,612][20347] Updated weights for policy 0, policy_version 1641081 (0.0007) [2023-12-27 17:00:55,660][20347] Updated weights for policy 0, policy_version 1641091 (0.0007) [2023-12-27 17:00:55,710][20347] Updated weights for policy 0, policy_version 1641101 (0.0006) [2023-12-27 17:00:56,012][20381] Updated weights for policy 1, policy_version 1639511 (0.0008) [2023-12-27 17:00:56,080][20381] Updated weights for policy 1, policy_version 1639521 (0.0008) [2023-12-27 17:00:56,146][20381] Updated weights for policy 1, policy_version 1639531 (0.0007) [2023-12-27 17:00:56,362][20347] Updated weights for policy 0, policy_version 1641111 (0.0007) [2023-12-27 17:00:56,409][20347] Updated weights for policy 0, policy_version 1641121 (0.0007) [2023-12-27 17:00:56,458][20347] Updated weights for policy 0, policy_version 1641131 (0.0008) [2023-12-27 17:00:56,787][20381] Updated weights for policy 1, policy_version 1639541 (0.0008) [2023-12-27 17:00:56,835][20381] Updated weights for policy 1, policy_version 1639551 (0.0008) [2023-12-27 17:00:56,881][20381] Updated weights for policy 1, policy_version 1639561 (0.0007) [2023-12-27 17:00:57,166][20347] Updated weights for policy 0, policy_version 1641141 (0.0007) [2023-12-27 17:00:57,222][20347] Updated weights for policy 0, policy_version 1641151 (0.0005) [2023-12-27 17:00:57,273][20347] Updated weights for policy 0, policy_version 1641161 (0.0005) [2023-12-27 17:00:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 839991296. Throughput: 0: 10010.2, 1: 10057.9. Samples: 839977512. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:00:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.910')] [2023-12-27 17:00:57,505][20381] Updated weights for policy 1, policy_version 1639571 (0.0007) [2023-12-27 17:00:57,558][20381] Updated weights for policy 1, policy_version 1639581 (0.0006) [2023-12-27 17:00:57,609][20381] Updated weights for policy 1, policy_version 1639591 (0.0007) [2023-12-27 17:00:57,853][20347] Updated weights for policy 0, policy_version 1641171 (0.0005) [2023-12-27 17:00:57,913][20347] Updated weights for policy 0, policy_version 1641181 (0.0007) [2023-12-27 17:00:57,969][20347] Updated weights for policy 0, policy_version 1641191 (0.0008) [2023-12-27 17:00:58,204][20381] Updated weights for policy 1, policy_version 1639601 (0.0008) [2023-12-27 17:00:58,264][20381] Updated weights for policy 1, policy_version 1639611 (0.0006) [2023-12-27 17:00:58,322][20381] Updated weights for policy 1, policy_version 1639621 (0.0007) [2023-12-27 17:00:58,387][20381] Updated weights for policy 1, policy_version 1639631 (0.0008) [2023-12-27 17:00:58,737][20347] Updated weights for policy 0, policy_version 1641201 (0.0007) [2023-12-27 17:00:58,795][20347] Updated weights for policy 0, policy_version 1641211 (0.0008) [2023-12-27 17:00:58,868][20347] Updated weights for policy 0, policy_version 1641221 (0.0008) [2023-12-27 17:00:58,933][20347] Updated weights for policy 0, policy_version 1641231 (0.0008) [2023-12-27 17:00:59,066][20381] Updated weights for policy 1, policy_version 1639641 (0.0008) [2023-12-27 17:00:59,124][20381] Updated weights for policy 1, policy_version 1639651 (0.0008) [2023-12-27 17:00:59,181][20381] Updated weights for policy 1, policy_version 1639661 (0.0007) [2023-12-27 17:00:59,570][20347] Updated weights for policy 0, policy_version 1641241 (0.0006) [2023-12-27 17:00:59,631][20347] Updated weights for policy 0, policy_version 1641251 (0.0008) [2023-12-27 17:00:59,697][20347] Updated weights for policy 0, policy_version 1641261 (0.0007) [2023-12-27 17:00:59,882][20381] Updated weights for policy 1, policy_version 1639671 (0.0008) [2023-12-27 17:00:59,944][20381] Updated weights for policy 1, policy_version 1639681 (0.0007) [2023-12-27 17:01:00,010][20381] Updated weights for policy 1, policy_version 1639691 (0.0007) [2023-12-27 17:01:00,427][20347] Updated weights for policy 0, policy_version 1641271 (0.0008) [2023-12-27 17:01:00,481][20347] Updated weights for policy 0, policy_version 1641281 (0.0007) [2023-12-27 17:01:00,535][20347] Updated weights for policy 0, policy_version 1641291 (0.0006) [2023-12-27 17:01:00,660][20381] Updated weights for policy 1, policy_version 1639701 (0.0007) [2023-12-27 17:01:00,709][20381] Updated weights for policy 1, policy_version 1639711 (0.0008) [2023-12-27 17:01:00,760][20381] Updated weights for policy 1, policy_version 1639721 (0.0008) [2023-12-27 17:01:01,090][20347] Updated weights for policy 0, policy_version 1641301 (0.0007) [2023-12-27 17:01:01,150][20347] Updated weights for policy 0, policy_version 1641311 (0.0007) [2023-12-27 17:01:01,216][20347] Updated weights for policy 0, policy_version 1641321 (0.0008) [2023-12-27 17:01:01,461][20381] Updated weights for policy 1, policy_version 1639731 (0.0008) [2023-12-27 17:01:01,522][20381] Updated weights for policy 1, policy_version 1639741 (0.0007) [2023-12-27 17:01:01,575][20381] Updated weights for policy 1, policy_version 1639751 (0.0008) [2023-12-27 17:01:01,979][20347] Updated weights for policy 0, policy_version 1641331 (0.0008) [2023-12-27 17:01:02,028][20347] Updated weights for policy 0, policy_version 1641341 (0.0008) [2023-12-27 17:01:02,083][20347] Updated weights for policy 0, policy_version 1641351 (0.0007) [2023-12-27 17:01:02,354][20381] Updated weights for policy 1, policy_version 1639761 (0.0008) [2023-12-27 17:01:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 840089600. Throughput: 0: 9989.8, 1: 10109.7. Samples: 840100824. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:02,365][19170] Avg episode reward: [(0, '1000.000'), (1, '902.450')] [2023-12-27 17:01:02,420][20381] Updated weights for policy 1, policy_version 1639771 (0.0007) [2023-12-27 17:01:02,464][20381] Updated weights for policy 1, policy_version 1639781 (0.0005) [2023-12-27 17:01:02,513][20381] Updated weights for policy 1, policy_version 1639791 (0.0005) [2023-12-27 17:01:02,716][20347] Updated weights for policy 0, policy_version 1641361 (0.0005) [2023-12-27 17:01:02,777][20347] Updated weights for policy 0, policy_version 1641371 (0.0006) [2023-12-27 17:01:02,838][20347] Updated weights for policy 0, policy_version 1641381 (0.0007) [2023-12-27 17:01:02,894][20347] Updated weights for policy 0, policy_version 1641391 (0.0007) [2023-12-27 17:01:03,159][20381] Updated weights for policy 1, policy_version 1639801 (0.0005) [2023-12-27 17:01:03,210][20381] Updated weights for policy 1, policy_version 1639811 (0.0005) [2023-12-27 17:01:03,265][20381] Updated weights for policy 1, policy_version 1639821 (0.0005) [2023-12-27 17:01:03,484][20347] Updated weights for policy 0, policy_version 1641401 (0.0010) [2023-12-27 17:01:03,540][20347] Updated weights for policy 0, policy_version 1641411 (0.0010) [2023-12-27 17:01:03,598][20347] Updated weights for policy 0, policy_version 1641421 (0.0010) [2023-12-27 17:01:03,955][20381] Updated weights for policy 1, policy_version 1639831 (0.0007) [2023-12-27 17:01:04,010][20381] Updated weights for policy 1, policy_version 1639841 (0.0007) [2023-12-27 17:01:04,075][20381] Updated weights for policy 1, policy_version 1639851 (0.0007) [2023-12-27 17:01:04,300][20347] Updated weights for policy 0, policy_version 1641431 (0.0009) [2023-12-27 17:01:04,360][20347] Updated weights for policy 0, policy_version 1641441 (0.0009) [2023-12-27 17:01:04,419][20347] Updated weights for policy 0, policy_version 1641451 (0.0010) [2023-12-27 17:01:04,809][20381] Updated weights for policy 1, policy_version 1639861 (0.0008) [2023-12-27 17:01:04,865][20381] Updated weights for policy 1, policy_version 1639871 (0.0008) [2023-12-27 17:01:04,912][20381] Updated weights for policy 1, policy_version 1639881 (0.0007) [2023-12-27 17:01:05,039][20347] Updated weights for policy 0, policy_version 1641461 (0.0009) [2023-12-27 17:01:05,100][20347] Updated weights for policy 0, policy_version 1641471 (0.0005) [2023-12-27 17:01:05,158][20347] Updated weights for policy 0, policy_version 1641481 (0.0005) [2023-12-27 17:01:05,676][20381] Updated weights for policy 1, policy_version 1639891 (0.0008) [2023-12-27 17:01:05,706][20347] Updated weights for policy 0, policy_version 1641491 (0.0005) [2023-12-27 17:01:05,734][20381] Updated weights for policy 1, policy_version 1639901 (0.0008) [2023-12-27 17:01:05,764][20347] Updated weights for policy 0, policy_version 1641501 (0.0009) [2023-12-27 17:01:05,785][20381] Updated weights for policy 1, policy_version 1639911 (0.0006) [2023-12-27 17:01:05,819][20347] Updated weights for policy 0, policy_version 1641511 (0.0010) [2023-12-27 17:01:06,466][20381] Updated weights for policy 1, policy_version 1639921 (0.0008) [2023-12-27 17:01:06,473][20347] Updated weights for policy 0, policy_version 1641521 (0.0010) [2023-12-27 17:01:06,524][20381] Updated weights for policy 1, policy_version 1639931 (0.0007) [2023-12-27 17:01:06,537][20347] Updated weights for policy 0, policy_version 1641531 (0.0007) [2023-12-27 17:01:06,589][20381] Updated weights for policy 1, policy_version 1639941 (0.0006) [2023-12-27 17:01:06,595][20347] Updated weights for policy 0, policy_version 1641541 (0.0006) [2023-12-27 17:01:06,651][20347] Updated weights for policy 0, policy_version 1641551 (0.0007) [2023-12-27 17:01:06,660][20381] Updated weights for policy 1, policy_version 1639951 (0.0007) [2023-12-27 17:01:07,289][20381] Updated weights for policy 1, policy_version 1639961 (0.0007) [2023-12-27 17:01:07,347][20381] Updated weights for policy 1, policy_version 1639971 (0.0006) [2023-12-27 17:01:07,361][20347] Updated weights for policy 0, policy_version 1641561 (0.0007) [2023-12-27 17:01:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20070.4, 300 sec: 20521.7). Total num frames: 840187904. Throughput: 0: 10079.5, 1: 10053.8. Samples: 840163096. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '775.500')] [2023-12-27 17:01:07,398][20381] Updated weights for policy 1, policy_version 1639981 (0.0006) [2023-12-27 17:01:07,414][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001639984_419880960.pth... [2023-12-27 17:01:07,418][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001638800_419577856.pth [2023-12-27 17:01:07,419][20347] Updated weights for policy 0, policy_version 1641571 (0.0006) [2023-12-27 17:01:07,477][20347] Updated weights for policy 0, policy_version 1641581 (0.0008) [2023-12-27 17:01:07,491][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001641584_420323328.pth... [2023-12-27 17:01:07,495][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001640400_420020224.pth [2023-12-27 17:01:08,060][20347] Updated weights for policy 0, policy_version 1641591 (0.0006) [2023-12-27 17:01:08,064][20381] Updated weights for policy 1, policy_version 1639991 (0.0006) [2023-12-27 17:01:08,115][20347] Updated weights for policy 0, policy_version 1641601 (0.0005) [2023-12-27 17:01:08,123][20381] Updated weights for policy 1, policy_version 1640001 (0.0006) [2023-12-27 17:01:08,163][20347] Updated weights for policy 0, policy_version 1641611 (0.0010) [2023-12-27 17:01:08,173][20381] Updated weights for policy 1, policy_version 1640011 (0.0006) [2023-12-27 17:01:08,829][20381] Updated weights for policy 1, policy_version 1640021 (0.0008) [2023-12-27 17:01:08,879][20347] Updated weights for policy 0, policy_version 1641621 (0.0007) [2023-12-27 17:01:08,892][20381] Updated weights for policy 1, policy_version 1640031 (0.0011) [2023-12-27 17:01:08,942][20347] Updated weights for policy 0, policy_version 1641631 (0.0008) [2023-12-27 17:01:08,954][20381] Updated weights for policy 1, policy_version 1640041 (0.0011) [2023-12-27 17:01:08,993][20347] Updated weights for policy 0, policy_version 1641641 (0.0009) [2023-12-27 17:01:09,664][20381] Updated weights for policy 1, policy_version 1640051 (0.0011) [2023-12-27 17:01:09,682][20347] Updated weights for policy 0, policy_version 1641651 (0.0010) [2023-12-27 17:01:09,723][20381] Updated weights for policy 1, policy_version 1640061 (0.0011) [2023-12-27 17:01:09,744][20347] Updated weights for policy 0, policy_version 1641661 (0.0010) [2023-12-27 17:01:09,784][20381] Updated weights for policy 1, policy_version 1640071 (0.0011) [2023-12-27 17:01:09,808][20347] Updated weights for policy 0, policy_version 1641671 (0.0010) [2023-12-27 17:01:10,500][20347] Updated weights for policy 0, policy_version 1641681 (0.0011) [2023-12-27 17:01:10,512][20381] Updated weights for policy 1, policy_version 1640081 (0.0010) [2023-12-27 17:01:10,563][20347] Updated weights for policy 0, policy_version 1641691 (0.0011) [2023-12-27 17:01:10,578][20381] Updated weights for policy 1, policy_version 1640091 (0.0008) [2023-12-27 17:01:10,618][20347] Updated weights for policy 0, policy_version 1641701 (0.0010) [2023-12-27 17:01:10,643][20381] Updated weights for policy 1, policy_version 1640101 (0.0008) [2023-12-27 17:01:10,673][20347] Updated weights for policy 0, policy_version 1641711 (0.0010) [2023-12-27 17:01:10,708][20381] Updated weights for policy 1, policy_version 1640111 (0.0010) [2023-12-27 17:01:11,312][20347] Updated weights for policy 0, policy_version 1641721 (0.0008) [2023-12-27 17:01:11,383][20347] Updated weights for policy 0, policy_version 1641731 (0.0008) [2023-12-27 17:01:11,403][20381] Updated weights for policy 1, policy_version 1640121 (0.0008) [2023-12-27 17:01:11,449][20347] Updated weights for policy 0, policy_version 1641741 (0.0008) [2023-12-27 17:01:11,456][20381] Updated weights for policy 1, policy_version 1640131 (0.0006) [2023-12-27 17:01:11,513][20381] Updated weights for policy 1, policy_version 1640141 (0.0008) [2023-12-27 17:01:12,216][20347] Updated weights for policy 0, policy_version 1641751 (0.0006) [2023-12-27 17:01:12,260][20381] Updated weights for policy 1, policy_version 1640151 (0.0007) [2023-12-27 17:01:12,280][20347] Updated weights for policy 0, policy_version 1641761 (0.0008) [2023-12-27 17:01:12,324][20381] Updated weights for policy 1, policy_version 1640161 (0.0006) [2023-12-27 17:01:12,344][20347] Updated weights for policy 0, policy_version 1641771 (0.0007) [2023-12-27 17:01:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 19933.8, 300 sec: 20493.9). Total num frames: 840286208. Throughput: 0: 10128.6, 1: 10039.7. Samples: 840285228. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '785.360')] [2023-12-27 17:01:12,392][20381] Updated weights for policy 1, policy_version 1640171 (0.0008) [2023-12-27 17:01:13,075][20381] Updated weights for policy 1, policy_version 1640181 (0.0008) [2023-12-27 17:01:13,077][20347] Updated weights for policy 0, policy_version 1641781 (0.0008) [2023-12-27 17:01:13,123][20347] Updated weights for policy 0, policy_version 1641791 (0.0007) [2023-12-27 17:01:13,135][20381] Updated weights for policy 1, policy_version 1640191 (0.0008) [2023-12-27 17:01:13,168][20347] Updated weights for policy 0, policy_version 1641801 (0.0007) [2023-12-27 17:01:13,196][20381] Updated weights for policy 1, policy_version 1640201 (0.0007) [2023-12-27 17:01:13,882][20381] Updated weights for policy 1, policy_version 1640211 (0.0008) [2023-12-27 17:01:13,888][20347] Updated weights for policy 0, policy_version 1641811 (0.0008) [2023-12-27 17:01:13,936][20381] Updated weights for policy 1, policy_version 1640221 (0.0010) [2023-12-27 17:01:13,942][20347] Updated weights for policy 0, policy_version 1641821 (0.0010) [2023-12-27 17:01:13,982][20381] Updated weights for policy 1, policy_version 1640231 (0.0010) [2023-12-27 17:01:13,996][20347] Updated weights for policy 0, policy_version 1641831 (0.0010) [2023-12-27 17:01:14,593][20347] Updated weights for policy 0, policy_version 1641841 (0.0010) [2023-12-27 17:01:14,650][20347] Updated weights for policy 0, policy_version 1641851 (0.0010) [2023-12-27 17:01:14,671][20381] Updated weights for policy 1, policy_version 1640241 (0.0006) [2023-12-27 17:01:14,695][20347] Updated weights for policy 0, policy_version 1641861 (0.0010) [2023-12-27 17:01:14,719][20381] Updated weights for policy 1, policy_version 1640251 (0.0008) [2023-12-27 17:01:14,753][20347] Updated weights for policy 0, policy_version 1641871 (0.0010) [2023-12-27 17:01:14,780][20381] Updated weights for policy 1, policy_version 1640261 (0.0010) [2023-12-27 17:01:14,838][20381] Updated weights for policy 1, policy_version 1640271 (0.0009) [2023-12-27 17:01:15,437][20347] Updated weights for policy 0, policy_version 1641881 (0.0010) [2023-12-27 17:01:15,493][20347] Updated weights for policy 0, policy_version 1641891 (0.0010) [2023-12-27 17:01:15,542][20381] Updated weights for policy 1, policy_version 1640281 (0.0011) [2023-12-27 17:01:15,545][20347] Updated weights for policy 0, policy_version 1641901 (0.0010) [2023-12-27 17:01:15,597][20381] Updated weights for policy 1, policy_version 1640291 (0.0011) [2023-12-27 17:01:15,652][20381] Updated weights for policy 1, policy_version 1640301 (0.0009) [2023-12-27 17:01:16,220][20347] Updated weights for policy 0, policy_version 1641911 (0.0010) [2023-12-27 17:01:16,283][20347] Updated weights for policy 0, policy_version 1641921 (0.0010) [2023-12-27 17:01:16,336][20381] Updated weights for policy 1, policy_version 1640311 (0.0007) [2023-12-27 17:01:16,337][20347] Updated weights for policy 0, policy_version 1641931 (0.0010) [2023-12-27 17:01:16,387][20381] Updated weights for policy 1, policy_version 1640321 (0.0006) [2023-12-27 17:01:16,439][20381] Updated weights for policy 1, policy_version 1640331 (0.0007) [2023-12-27 17:01:16,944][20347] Updated weights for policy 0, policy_version 1641941 (0.0010) [2023-12-27 17:01:16,994][20347] Updated weights for policy 0, policy_version 1641951 (0.0010) [2023-12-27 17:01:17,034][20381] Updated weights for policy 1, policy_version 1640341 (0.0006) [2023-12-27 17:01:17,043][20347] Updated weights for policy 0, policy_version 1641961 (0.0010) [2023-12-27 17:01:17,083][20381] Updated weights for policy 1, policy_version 1640351 (0.0005) [2023-12-27 17:01:17,131][20381] Updated weights for policy 1, policy_version 1640361 (0.0007) [2023-12-27 17:01:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 840400896. Throughput: 0: 10048.4, 1: 10038.7. Samples: 840406336. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '755.900')] [2023-12-27 17:01:17,712][20347] Updated weights for policy 0, policy_version 1641971 (0.0010) [2023-12-27 17:01:17,757][20381] Updated weights for policy 1, policy_version 1640371 (0.0009) [2023-12-27 17:01:17,770][20347] Updated weights for policy 0, policy_version 1641981 (0.0010) [2023-12-27 17:01:17,807][20381] Updated weights for policy 1, policy_version 1640381 (0.0009) [2023-12-27 17:01:17,825][20347] Updated weights for policy 0, policy_version 1641991 (0.0010) [2023-12-27 17:01:17,856][20381] Updated weights for policy 1, policy_version 1640391 (0.0005) [2023-12-27 17:01:18,480][20347] Updated weights for policy 0, policy_version 1642001 (0.0010) [2023-12-27 17:01:18,499][20381] Updated weights for policy 1, policy_version 1640401 (0.0006) [2023-12-27 17:01:18,545][20347] Updated weights for policy 0, policy_version 1642011 (0.0007) [2023-12-27 17:01:18,561][20381] Updated weights for policy 1, policy_version 1640411 (0.0010) [2023-12-27 17:01:18,605][20347] Updated weights for policy 0, policy_version 1642021 (0.0006) [2023-12-27 17:01:18,619][20381] Updated weights for policy 1, policy_version 1640421 (0.0008) [2023-12-27 17:01:18,659][20347] Updated weights for policy 0, policy_version 1642031 (0.0006) [2023-12-27 17:01:18,679][20381] Updated weights for policy 1, policy_version 1640431 (0.0006) [2023-12-27 17:01:19,312][20347] Updated weights for policy 0, policy_version 1642041 (0.0008) [2023-12-27 17:01:19,378][20381] Updated weights for policy 1, policy_version 1640441 (0.0006) [2023-12-27 17:01:19,380][20347] Updated weights for policy 0, policy_version 1642051 (0.0008) [2023-12-27 17:01:19,432][20347] Updated weights for policy 0, policy_version 1642061 (0.0008) [2023-12-27 17:01:19,434][20381] Updated weights for policy 1, policy_version 1640451 (0.0006) [2023-12-27 17:01:19,504][20381] Updated weights for policy 1, policy_version 1640461 (0.0007) [2023-12-27 17:01:20,152][20381] Updated weights for policy 1, policy_version 1640471 (0.0008) [2023-12-27 17:01:20,207][20381] Updated weights for policy 1, policy_version 1640481 (0.0007) [2023-12-27 17:01:20,219][20347] Updated weights for policy 0, policy_version 1642071 (0.0007) [2023-12-27 17:01:20,259][20381] Updated weights for policy 1, policy_version 1640491 (0.0007) [2023-12-27 17:01:20,276][20347] Updated weights for policy 0, policy_version 1642081 (0.0006) [2023-12-27 17:01:20,337][20347] Updated weights for policy 0, policy_version 1642091 (0.0007) [2023-12-27 17:01:20,946][20381] Updated weights for policy 1, policy_version 1640501 (0.0006) [2023-12-27 17:01:21,008][20381] Updated weights for policy 1, policy_version 1640511 (0.0007) [2023-12-27 17:01:21,071][20381] Updated weights for policy 1, policy_version 1640521 (0.0008) [2023-12-27 17:01:21,112][20347] Updated weights for policy 0, policy_version 1642101 (0.0008) [2023-12-27 17:01:21,171][20347] Updated weights for policy 0, policy_version 1642111 (0.0008) [2023-12-27 17:01:21,226][20347] Updated weights for policy 0, policy_version 1642121 (0.0008) [2023-12-27 17:01:21,793][20381] Updated weights for policy 1, policy_version 1640531 (0.0006) [2023-12-27 17:01:21,842][20381] Updated weights for policy 1, policy_version 1640541 (0.0007) [2023-12-27 17:01:21,892][20381] Updated weights for policy 1, policy_version 1640551 (0.0008) [2023-12-27 17:01:22,001][20347] Updated weights for policy 0, policy_version 1642131 (0.0008) [2023-12-27 17:01:22,061][20347] Updated weights for policy 0, policy_version 1642141 (0.0008) [2023-12-27 17:01:22,124][20347] Updated weights for policy 0, policy_version 1642151 (0.0006) [2023-12-27 17:01:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20207.0, 300 sec: 20549.4). Total num frames: 840499200. Throughput: 0: 10047.9, 1: 10030.8. Samples: 840468648. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '765.620')] [2023-12-27 17:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001640560_420028416.pth... [2023-12-27 17:01:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001642160_420470784.pth... [2023-12-27 17:01:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001639376_419725312.pth [2023-12-27 17:01:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001640976_420167680.pth [2023-12-27 17:01:22,635][20381] Updated weights for policy 1, policy_version 1640561 (0.0008) [2023-12-27 17:01:22,691][20381] Updated weights for policy 1, policy_version 1640571 (0.0008) [2023-12-27 17:01:22,752][20381] Updated weights for policy 1, policy_version 1640581 (0.0008) [2023-12-27 17:01:22,814][20381] Updated weights for policy 1, policy_version 1640591 (0.0009) [2023-12-27 17:01:22,844][20347] Updated weights for policy 0, policy_version 1642161 (0.0008) [2023-12-27 17:01:22,900][20347] Updated weights for policy 0, policy_version 1642171 (0.0005) [2023-12-27 17:01:22,957][20347] Updated weights for policy 0, policy_version 1642181 (0.0005) [2023-12-27 17:01:23,013][20347] Updated weights for policy 0, policy_version 1642191 (0.0006) [2023-12-27 17:01:23,562][20381] Updated weights for policy 1, policy_version 1640601 (0.0007) [2023-12-27 17:01:23,607][20347] Updated weights for policy 0, policy_version 1642201 (0.0010) [2023-12-27 17:01:23,620][20381] Updated weights for policy 1, policy_version 1640611 (0.0007) [2023-12-27 17:01:23,658][20347] Updated weights for policy 0, policy_version 1642211 (0.0010) [2023-12-27 17:01:23,669][20381] Updated weights for policy 1, policy_version 1640621 (0.0006) [2023-12-27 17:01:23,706][20347] Updated weights for policy 0, policy_version 1642221 (0.0010) [2023-12-27 17:01:24,312][20381] Updated weights for policy 1, policy_version 1640631 (0.0007) [2023-12-27 17:01:24,359][20347] Updated weights for policy 0, policy_version 1642231 (0.0010) [2023-12-27 17:01:24,365][20381] Updated weights for policy 1, policy_version 1640641 (0.0006) [2023-12-27 17:01:24,417][20381] Updated weights for policy 1, policy_version 1640651 (0.0005) [2023-12-27 17:01:24,419][20347] Updated weights for policy 0, policy_version 1642241 (0.0011) [2023-12-27 17:01:24,468][20347] Updated weights for policy 0, policy_version 1642251 (0.0011) [2023-12-27 17:01:25,083][20381] Updated weights for policy 1, policy_version 1640661 (0.0007) [2023-12-27 17:01:25,136][20381] Updated weights for policy 1, policy_version 1640671 (0.0007) [2023-12-27 17:01:25,148][20347] Updated weights for policy 0, policy_version 1642261 (0.0010) [2023-12-27 17:01:25,190][20381] Updated weights for policy 1, policy_version 1640681 (0.0008) [2023-12-27 17:01:25,203][20347] Updated weights for policy 0, policy_version 1642271 (0.0010) [2023-12-27 17:01:25,260][20347] Updated weights for policy 0, policy_version 1642281 (0.0010) [2023-12-27 17:01:25,918][20381] Updated weights for policy 1, policy_version 1640691 (0.0007) [2023-12-27 17:01:25,947][20347] Updated weights for policy 0, policy_version 1642291 (0.0011) [2023-12-27 17:01:25,972][20381] Updated weights for policy 1, policy_version 1640701 (0.0010) [2023-12-27 17:01:26,001][20347] Updated weights for policy 0, policy_version 1642301 (0.0010) [2023-12-27 17:01:26,031][20381] Updated weights for policy 1, policy_version 1640711 (0.0010) [2023-12-27 17:01:26,051][20347] Updated weights for policy 0, policy_version 1642311 (0.0010) [2023-12-27 17:01:26,664][20381] Updated weights for policy 1, policy_version 1640721 (0.0010) [2023-12-27 17:01:26,717][20381] Updated weights for policy 1, policy_version 1640731 (0.0008) [2023-12-27 17:01:26,729][20347] Updated weights for policy 0, policy_version 1642321 (0.0010) [2023-12-27 17:01:26,777][20381] Updated weights for policy 1, policy_version 1640741 (0.0007) [2023-12-27 17:01:26,786][20347] Updated weights for policy 0, policy_version 1642331 (0.0008) [2023-12-27 17:01:26,828][20381] Updated weights for policy 1, policy_version 1640751 (0.0005) [2023-12-27 17:01:26,836][20347] Updated weights for policy 0, policy_version 1642341 (0.0008) [2023-12-27 17:01:26,883][20347] Updated weights for policy 0, policy_version 1642351 (0.0006) [2023-12-27 17:01:27,344][20381] Updated weights for policy 1, policy_version 1640761 (0.0008) [2023-12-27 17:01:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 840597504. Throughput: 0: 10087.7, 1: 10010.9. Samples: 840588008. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.670')] [2023-12-27 17:01:27,395][20381] Updated weights for policy 1, policy_version 1640771 (0.0009) [2023-12-27 17:01:27,456][20381] Updated weights for policy 1, policy_version 1640781 (0.0005) [2023-12-27 17:01:27,549][20347] Updated weights for policy 0, policy_version 1642361 (0.0006) [2023-12-27 17:01:27,601][20347] Updated weights for policy 0, policy_version 1642371 (0.0010) [2023-12-27 17:01:27,665][20347] Updated weights for policy 0, policy_version 1642381 (0.0010) [2023-12-27 17:01:28,049][20381] Updated weights for policy 1, policy_version 1640791 (0.0005) [2023-12-27 17:01:28,098][20381] Updated weights for policy 1, policy_version 1640801 (0.0006) [2023-12-27 17:01:28,147][20381] Updated weights for policy 1, policy_version 1640811 (0.0007) [2023-12-27 17:01:28,306][20347] Updated weights for policy 0, policy_version 1642391 (0.0010) [2023-12-27 17:01:28,368][20347] Updated weights for policy 0, policy_version 1642401 (0.0011) [2023-12-27 17:01:28,424][20347] Updated weights for policy 0, policy_version 1642411 (0.0010) [2023-12-27 17:01:28,792][20381] Updated weights for policy 1, policy_version 1640821 (0.0006) [2023-12-27 17:01:28,844][20381] Updated weights for policy 1, policy_version 1640831 (0.0008) [2023-12-27 17:01:28,899][20381] Updated weights for policy 1, policy_version 1640841 (0.0007) [2023-12-27 17:01:29,079][20347] Updated weights for policy 0, policy_version 1642421 (0.0010) [2023-12-27 17:01:29,139][20347] Updated weights for policy 0, policy_version 1642431 (0.0010) [2023-12-27 17:01:29,198][20347] Updated weights for policy 0, policy_version 1642441 (0.0009) [2023-12-27 17:01:29,629][20381] Updated weights for policy 1, policy_version 1640851 (0.0007) [2023-12-27 17:01:29,677][20381] Updated weights for policy 1, policy_version 1640861 (0.0007) [2023-12-27 17:01:29,729][20381] Updated weights for policy 1, policy_version 1640871 (0.0007) [2023-12-27 17:01:29,869][20347] Updated weights for policy 0, policy_version 1642451 (0.0008) [2023-12-27 17:01:29,930][20347] Updated weights for policy 0, policy_version 1642461 (0.0010) [2023-12-27 17:01:29,994][20347] Updated weights for policy 0, policy_version 1642471 (0.0008) [2023-12-27 17:01:30,505][20381] Updated weights for policy 1, policy_version 1640881 (0.0007) [2023-12-27 17:01:30,566][20381] Updated weights for policy 1, policy_version 1640891 (0.0008) [2023-12-27 17:01:30,624][20381] Updated weights for policy 1, policy_version 1640901 (0.0007) [2023-12-27 17:01:30,645][20347] Updated weights for policy 0, policy_version 1642481 (0.0006) [2023-12-27 17:01:30,680][20381] Updated weights for policy 1, policy_version 1640911 (0.0006) [2023-12-27 17:01:30,696][20347] Updated weights for policy 0, policy_version 1642491 (0.0010) [2023-12-27 17:01:30,740][20347] Updated weights for policy 0, policy_version 1642501 (0.0010) [2023-12-27 17:01:30,784][20347] Updated weights for policy 0, policy_version 1642511 (0.0010) [2023-12-27 17:01:31,285][20381] Updated weights for policy 1, policy_version 1640921 (0.0008) [2023-12-27 17:01:31,345][20381] Updated weights for policy 1, policy_version 1640931 (0.0008) [2023-12-27 17:01:31,417][20381] Updated weights for policy 1, policy_version 1640941 (0.0008) [2023-12-27 17:01:31,484][20347] Updated weights for policy 0, policy_version 1642521 (0.0007) [2023-12-27 17:01:31,536][20347] Updated weights for policy 0, policy_version 1642531 (0.0008) [2023-12-27 17:01:31,595][20347] Updated weights for policy 0, policy_version 1642541 (0.0007) [2023-12-27 17:01:32,109][20381] Updated weights for policy 1, policy_version 1640951 (0.0008) [2023-12-27 17:01:32,159][20381] Updated weights for policy 1, policy_version 1640961 (0.0007) [2023-12-27 17:01:32,213][20381] Updated weights for policy 1, policy_version 1640971 (0.0008) [2023-12-27 17:01:32,310][20347] Updated weights for policy 0, policy_version 1642551 (0.0007) [2023-12-27 17:01:32,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20070.3, 300 sec: 20549.4). Total num frames: 840704000. Throughput: 0: 10229.0, 1: 10172.3. Samples: 840713960. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:32,363][19170] Avg episode reward: [(0, '990.220'), (1, '804.630')] [2023-12-27 17:01:32,375][20347] Updated weights for policy 0, policy_version 1642561 (0.0007) [2023-12-27 17:01:32,432][20347] Updated weights for policy 0, policy_version 1642571 (0.0007) [2023-12-27 17:01:32,950][20381] Updated weights for policy 1, policy_version 1640981 (0.0008) [2023-12-27 17:01:33,006][20381] Updated weights for policy 1, policy_version 1640991 (0.0010) [2023-12-27 17:01:33,068][20381] Updated weights for policy 1, policy_version 1641001 (0.0010) [2023-12-27 17:01:33,097][20347] Updated weights for policy 0, policy_version 1642581 (0.0006) [2023-12-27 17:01:33,151][20347] Updated weights for policy 0, policy_version 1642591 (0.0007) [2023-12-27 17:01:33,200][20347] Updated weights for policy 0, policy_version 1642601 (0.0006) [2023-12-27 17:01:33,633][20381] Updated weights for policy 1, policy_version 1641011 (0.0009) [2023-12-27 17:01:33,691][20381] Updated weights for policy 1, policy_version 1641021 (0.0005) [2023-12-27 17:01:33,740][20381] Updated weights for policy 1, policy_version 1641031 (0.0005) [2023-12-27 17:01:33,766][20347] Updated weights for policy 0, policy_version 1642611 (0.0006) [2023-12-27 17:01:33,812][20347] Updated weights for policy 0, policy_version 1642621 (0.0008) [2023-12-27 17:01:33,860][20347] Updated weights for policy 0, policy_version 1642631 (0.0009) [2023-12-27 17:01:34,259][20381] Updated weights for policy 1, policy_version 1641041 (0.0006) [2023-12-27 17:01:34,325][20381] Updated weights for policy 1, policy_version 1641051 (0.0009) [2023-12-27 17:01:34,390][20381] Updated weights for policy 1, policy_version 1641061 (0.0008) [2023-12-27 17:01:34,458][20381] Updated weights for policy 1, policy_version 1641071 (0.0009) [2023-12-27 17:01:34,498][20347] Updated weights for policy 0, policy_version 1642641 (0.0008) [2023-12-27 17:01:34,552][20347] Updated weights for policy 0, policy_version 1642651 (0.0008) [2023-12-27 17:01:34,601][20347] Updated weights for policy 0, policy_version 1642661 (0.0008) [2023-12-27 17:01:34,656][20347] Updated weights for policy 0, policy_version 1642671 (0.0008) [2023-12-27 17:01:35,156][20381] Updated weights for policy 1, policy_version 1641081 (0.0008) [2023-12-27 17:01:35,208][20381] Updated weights for policy 1, policy_version 1641091 (0.0008) [2023-12-27 17:01:35,290][20381] Updated weights for policy 1, policy_version 1641101 (0.0009) [2023-12-27 17:01:35,358][20347] Updated weights for policy 0, policy_version 1642681 (0.0005) [2023-12-27 17:01:35,413][20347] Updated weights for policy 0, policy_version 1642691 (0.0005) [2023-12-27 17:01:35,466][20347] Updated weights for policy 0, policy_version 1642701 (0.0005) [2023-12-27 17:01:36,005][20347] Updated weights for policy 0, policy_version 1642711 (0.0005) [2023-12-27 17:01:36,026][20381] Updated weights for policy 1, policy_version 1641111 (0.0006) [2023-12-27 17:01:36,057][20347] Updated weights for policy 0, policy_version 1642721 (0.0007) [2023-12-27 17:01:36,068][20381] Updated weights for policy 1, policy_version 1641121 (0.0007) [2023-12-27 17:01:36,115][20347] Updated weights for policy 0, policy_version 1642731 (0.0007) [2023-12-27 17:01:36,127][20381] Updated weights for policy 1, policy_version 1641131 (0.0006) [2023-12-27 17:01:36,770][20381] Updated weights for policy 1, policy_version 1641141 (0.0008) [2023-12-27 17:01:36,827][20381] Updated weights for policy 1, policy_version 1641151 (0.0008) [2023-12-27 17:01:36,854][20347] Updated weights for policy 0, policy_version 1642741 (0.0006) [2023-12-27 17:01:36,883][20381] Updated weights for policy 1, policy_version 1641161 (0.0006) [2023-12-27 17:01:36,910][20347] Updated weights for policy 0, policy_version 1642751 (0.0005) [2023-12-27 17:01:36,969][20347] Updated weights for policy 0, policy_version 1642761 (0.0006) [2023-12-27 17:01:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20207.0, 300 sec: 20577.2). Total num frames: 840810496. Throughput: 0: 10247.4, 1: 10233.5. Samples: 840778608. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:37,363][19170] Avg episode reward: [(0, '970.800'), (1, '785.210')] [2023-12-27 17:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001641168_420184064.pth... [2023-12-27 17:01:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001642768_420626432.pth... [2023-12-27 17:01:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001639984_419880960.pth [2023-12-27 17:01:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001641584_420323328.pth [2023-12-27 17:01:37,573][20381] Updated weights for policy 1, policy_version 1641171 (0.0009) [2023-12-27 17:01:37,635][20381] Updated weights for policy 1, policy_version 1641181 (0.0008) [2023-12-27 17:01:37,647][20347] Updated weights for policy 0, policy_version 1642771 (0.0009) [2023-12-27 17:01:37,685][20381] Updated weights for policy 1, policy_version 1641191 (0.0008) [2023-12-27 17:01:37,702][20292] KL-divergence is very high: 139.4902 [2023-12-27 17:01:37,708][20292] KL-divergence is very high: 115.0584 [2023-12-27 17:01:37,713][20347] Updated weights for policy 0, policy_version 1642781 (0.0011) [2023-12-27 17:01:37,726][20292] KL-divergence is very high: 111.1683 [2023-12-27 17:01:37,770][20347] Updated weights for policy 0, policy_version 1642791 (0.0007) [2023-12-27 17:01:38,382][20381] Updated weights for policy 1, policy_version 1641201 (0.0008) [2023-12-27 17:01:38,435][20347] Updated weights for policy 0, policy_version 1642801 (0.0008) [2023-12-27 17:01:38,446][20381] Updated weights for policy 1, policy_version 1641211 (0.0008) [2023-12-27 17:01:38,491][20347] Updated weights for policy 0, policy_version 1642811 (0.0007) [2023-12-27 17:01:38,506][20381] Updated weights for policy 1, policy_version 1641221 (0.0007) [2023-12-27 17:01:38,552][20347] Updated weights for policy 0, policy_version 1642821 (0.0009) [2023-12-27 17:01:38,569][20381] Updated weights for policy 1, policy_version 1641231 (0.0006) [2023-12-27 17:01:38,611][20347] Updated weights for policy 0, policy_version 1642831 (0.0007) [2023-12-27 17:01:39,218][20347] Updated weights for policy 0, policy_version 1642841 (0.0008) [2023-12-27 17:01:39,267][20381] Updated weights for policy 1, policy_version 1641241 (0.0008) [2023-12-27 17:01:39,288][20347] Updated weights for policy 0, policy_version 1642851 (0.0008) [2023-12-27 17:01:39,334][20381] Updated weights for policy 1, policy_version 1641251 (0.0010) [2023-12-27 17:01:39,353][20347] Updated weights for policy 0, policy_version 1642861 (0.0007) [2023-12-27 17:01:39,401][20381] Updated weights for policy 1, policy_version 1641261 (0.0011) [2023-12-27 17:01:40,146][20347] Updated weights for policy 0, policy_version 1642871 (0.0008) [2023-12-27 17:01:40,152][20381] Updated weights for policy 1, policy_version 1641271 (0.0009) [2023-12-27 17:01:40,205][20347] Updated weights for policy 0, policy_version 1642881 (0.0007) [2023-12-27 17:01:40,206][20381] Updated weights for policy 1, policy_version 1641281 (0.0007) [2023-12-27 17:01:40,268][20381] Updated weights for policy 1, policy_version 1641291 (0.0010) [2023-12-27 17:01:40,271][20347] Updated weights for policy 0, policy_version 1642891 (0.0007) [2023-12-27 17:01:40,979][20381] Updated weights for policy 1, policy_version 1641301 (0.0010) [2023-12-27 17:01:40,979][20347] Updated weights for policy 0, policy_version 1642901 (0.0008) [2023-12-27 17:01:41,031][20347] Updated weights for policy 0, policy_version 1642911 (0.0007) [2023-12-27 17:01:41,033][20381] Updated weights for policy 1, policy_version 1641311 (0.0010) [2023-12-27 17:01:41,088][20347] Updated weights for policy 0, policy_version 1642921 (0.0008) [2023-12-27 17:01:41,094][20381] Updated weights for policy 1, policy_version 1641321 (0.0007) [2023-12-27 17:01:41,838][20347] Updated weights for policy 0, policy_version 1642931 (0.0008) [2023-12-27 17:01:41,873][20381] Updated weights for policy 1, policy_version 1641331 (0.0008) [2023-12-27 17:01:41,901][20347] Updated weights for policy 0, policy_version 1642941 (0.0007) [2023-12-27 17:01:41,932][20381] Updated weights for policy 1, policy_version 1641341 (0.0008) [2023-12-27 17:01:41,956][20347] Updated weights for policy 0, policy_version 1642951 (0.0007) [2023-12-27 17:01:41,991][20381] Updated weights for policy 1, policy_version 1641351 (0.0007) [2023-12-27 17:01:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 840908800. Throughput: 0: 10282.7, 1: 10193.8. Samples: 840898952. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:42,363][19170] Avg episode reward: [(0, '970.850'), (1, '765.510')] [2023-12-27 17:01:42,616][20381] Updated weights for policy 1, policy_version 1641361 (0.0006) [2023-12-27 17:01:42,649][20347] Updated weights for policy 0, policy_version 1642961 (0.0008) [2023-12-27 17:01:42,677][20381] Updated weights for policy 1, policy_version 1641371 (0.0006) [2023-12-27 17:01:42,714][20347] Updated weights for policy 0, policy_version 1642971 (0.0008) [2023-12-27 17:01:42,738][20381] Updated weights for policy 1, policy_version 1641381 (0.0006) [2023-12-27 17:01:42,767][20347] Updated weights for policy 0, policy_version 1642981 (0.0006) [2023-12-27 17:01:42,798][20381] Updated weights for policy 1, policy_version 1641391 (0.0008) [2023-12-27 17:01:42,828][20347] Updated weights for policy 0, policy_version 1642991 (0.0007) [2023-12-27 17:01:43,396][20381] Updated weights for policy 1, policy_version 1641401 (0.0005) [2023-12-27 17:01:43,452][20381] Updated weights for policy 1, policy_version 1641411 (0.0008) [2023-12-27 17:01:43,488][20347] Updated weights for policy 0, policy_version 1643001 (0.0005) [2023-12-27 17:01:43,513][20381] Updated weights for policy 1, policy_version 1641421 (0.0006) [2023-12-27 17:01:43,550][20347] Updated weights for policy 0, policy_version 1643011 (0.0006) [2023-12-27 17:01:43,600][20347] Updated weights for policy 0, policy_version 1643021 (0.0007) [2023-12-27 17:01:44,160][20381] Updated weights for policy 1, policy_version 1641431 (0.0009) [2023-12-27 17:01:44,216][20381] Updated weights for policy 1, policy_version 1641441 (0.0010) [2023-12-27 17:01:44,248][20347] Updated weights for policy 0, policy_version 1643031 (0.0005) [2023-12-27 17:01:44,268][20381] Updated weights for policy 1, policy_version 1641451 (0.0010) [2023-12-27 17:01:44,299][20347] Updated weights for policy 0, policy_version 1643041 (0.0005) [2023-12-27 17:01:44,346][20347] Updated weights for policy 0, policy_version 1643051 (0.0007) [2023-12-27 17:01:44,847][20381] Updated weights for policy 1, policy_version 1641461 (0.0010) [2023-12-27 17:01:44,900][20381] Updated weights for policy 1, policy_version 1641471 (0.0011) [2023-12-27 17:01:44,953][20347] Updated weights for policy 0, policy_version 1643061 (0.0008) [2023-12-27 17:01:44,956][20381] Updated weights for policy 1, policy_version 1641481 (0.0010) [2023-12-27 17:01:45,015][20347] Updated weights for policy 0, policy_version 1643071 (0.0006) [2023-12-27 17:01:45,072][20347] Updated weights for policy 0, policy_version 1643081 (0.0008) [2023-12-27 17:01:45,655][20381] Updated weights for policy 1, policy_version 1641491 (0.0011) [2023-12-27 17:01:45,703][20381] Updated weights for policy 1, policy_version 1641501 (0.0010) [2023-12-27 17:01:45,749][20381] Updated weights for policy 1, policy_version 1641511 (0.0009) [2023-12-27 17:01:45,785][20347] Updated weights for policy 0, policy_version 1643091 (0.0007) [2023-12-27 17:01:45,842][20347] Updated weights for policy 0, policy_version 1643101 (0.0006) [2023-12-27 17:01:45,904][20347] Updated weights for policy 0, policy_version 1643111 (0.0006) [2023-12-27 17:01:46,415][20381] Updated weights for policy 1, policy_version 1641521 (0.0007) [2023-12-27 17:01:46,473][20381] Updated weights for policy 1, policy_version 1641531 (0.0005) [2023-12-27 17:01:46,477][20347] Updated weights for policy 0, policy_version 1643121 (0.0006) [2023-12-27 17:01:46,526][20347] Updated weights for policy 0, policy_version 1643131 (0.0010) [2023-12-27 17:01:46,528][20381] Updated weights for policy 1, policy_version 1641541 (0.0006) [2023-12-27 17:01:46,581][20381] Updated weights for policy 1, policy_version 1641551 (0.0005) [2023-12-27 17:01:46,581][20347] Updated weights for policy 0, policy_version 1643141 (0.0007) [2023-12-27 17:01:46,638][20347] Updated weights for policy 0, policy_version 1643151 (0.0007) [2023-12-27 17:01:47,184][20381] Updated weights for policy 1, policy_version 1641561 (0.0007) [2023-12-27 17:01:47,244][20381] Updated weights for policy 1, policy_version 1641571 (0.0006) [2023-12-27 17:01:47,258][20347] Updated weights for policy 0, policy_version 1643161 (0.0008) [2023-12-27 17:01:47,301][20381] Updated weights for policy 1, policy_version 1641581 (0.0007) [2023-12-27 17:01:47,311][20347] Updated weights for policy 0, policy_version 1643171 (0.0005) [2023-12-27 17:01:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 841015296. Throughput: 0: 10316.3, 1: 10230.8. Samples: 841025440. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:47,363][19170] Avg episode reward: [(0, '980.510'), (1, '794.940')] [2023-12-27 17:01:47,363][20347] Updated weights for policy 0, policy_version 1643181 (0.0005) [2023-12-27 17:01:47,978][20381] Updated weights for policy 1, policy_version 1641591 (0.0006) [2023-12-27 17:01:47,985][20347] Updated weights for policy 0, policy_version 1643191 (0.0005) [2023-12-27 17:01:48,027][20381] Updated weights for policy 1, policy_version 1641601 (0.0007) [2023-12-27 17:01:48,043][20347] Updated weights for policy 0, policy_version 1643201 (0.0005) [2023-12-27 17:01:48,080][20381] Updated weights for policy 1, policy_version 1641611 (0.0010) [2023-12-27 17:01:48,095][20347] Updated weights for policy 0, policy_version 1643211 (0.0007) [2023-12-27 17:01:48,671][20381] Updated weights for policy 1, policy_version 1641621 (0.0009) [2023-12-27 17:01:48,716][20347] Updated weights for policy 0, policy_version 1643221 (0.0008) [2023-12-27 17:01:48,737][20381] Updated weights for policy 1, policy_version 1641631 (0.0011) [2023-12-27 17:01:48,773][20347] Updated weights for policy 0, policy_version 1643231 (0.0011) [2023-12-27 17:01:48,797][20381] Updated weights for policy 1, policy_version 1641641 (0.0008) [2023-12-27 17:01:48,834][20347] Updated weights for policy 0, policy_version 1643241 (0.0010) [2023-12-27 17:01:49,495][20381] Updated weights for policy 1, policy_version 1641651 (0.0009) [2023-12-27 17:01:49,532][20347] Updated weights for policy 0, policy_version 1643251 (0.0009) [2023-12-27 17:01:49,557][20381] Updated weights for policy 1, policy_version 1641661 (0.0011) [2023-12-27 17:01:49,592][20347] Updated weights for policy 0, policy_version 1643261 (0.0005) [2023-12-27 17:01:49,613][20381] Updated weights for policy 1, policy_version 1641671 (0.0011) [2023-12-27 17:01:49,654][20347] Updated weights for policy 0, policy_version 1643271 (0.0007) [2023-12-27 17:01:50,274][20347] Updated weights for policy 0, policy_version 1643281 (0.0007) [2023-12-27 17:01:50,325][20381] Updated weights for policy 1, policy_version 1641681 (0.0010) [2023-12-27 17:01:50,329][20347] Updated weights for policy 0, policy_version 1643291 (0.0007) [2023-12-27 17:01:50,383][20381] Updated weights for policy 1, policy_version 1641691 (0.0007) [2023-12-27 17:01:50,388][20347] Updated weights for policy 0, policy_version 1643301 (0.0008) [2023-12-27 17:01:50,445][20381] Updated weights for policy 1, policy_version 1641701 (0.0005) [2023-12-27 17:01:50,447][20347] Updated weights for policy 0, policy_version 1643311 (0.0008) [2023-12-27 17:01:50,496][20381] Updated weights for policy 1, policy_version 1641711 (0.0007) [2023-12-27 17:01:51,095][20381] Updated weights for policy 1, policy_version 1641721 (0.0007) [2023-12-27 17:01:51,160][20381] Updated weights for policy 1, policy_version 1641731 (0.0006) [2023-12-27 17:01:51,168][20347] Updated weights for policy 0, policy_version 1643321 (0.0007) [2023-12-27 17:01:51,219][20381] Updated weights for policy 1, policy_version 1641741 (0.0006) [2023-12-27 17:01:51,225][20347] Updated weights for policy 0, policy_version 1643331 (0.0007) [2023-12-27 17:01:51,287][20347] Updated weights for policy 0, policy_version 1643341 (0.0007) [2023-12-27 17:01:51,960][20381] Updated weights for policy 1, policy_version 1641751 (0.0006) [2023-12-27 17:01:52,021][20381] Updated weights for policy 1, policy_version 1641761 (0.0006) [2023-12-27 17:01:52,027][20347] Updated weights for policy 0, policy_version 1643351 (0.0007) [2023-12-27 17:01:52,081][20381] Updated weights for policy 1, policy_version 1641771 (0.0007) [2023-12-27 17:01:52,083][20347] Updated weights for policy 0, policy_version 1643361 (0.0006) [2023-12-27 17:01:52,144][20347] Updated weights for policy 0, policy_version 1643371 (0.0007) [2023-12-27 17:01:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 841121792. Throughput: 0: 10303.3, 1: 10247.6. Samples: 841087888. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:52,363][19170] Avg episode reward: [(0, '990.240'), (1, '853.630')] [2023-12-27 17:01:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001643376_420782080.pth... [2023-12-27 17:01:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001641776_420339712.pth... [2023-12-27 17:01:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001640560_420028416.pth [2023-12-27 17:01:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001642160_420470784.pth [2023-12-27 17:01:52,819][20381] Updated weights for policy 1, policy_version 1641781 (0.0008) [2023-12-27 17:01:52,865][20347] Updated weights for policy 0, policy_version 1643381 (0.0007) [2023-12-27 17:01:52,882][20381] Updated weights for policy 1, policy_version 1641791 (0.0008) [2023-12-27 17:01:52,924][20347] Updated weights for policy 0, policy_version 1643391 (0.0006) [2023-12-27 17:01:52,934][20381] Updated weights for policy 1, policy_version 1641801 (0.0006) [2023-12-27 17:01:52,989][20347] Updated weights for policy 0, policy_version 1643401 (0.0005) [2023-12-27 17:01:53,573][20381] Updated weights for policy 1, policy_version 1641811 (0.0007) [2023-12-27 17:01:53,579][20347] Updated weights for policy 0, policy_version 1643411 (0.0006) [2023-12-27 17:01:53,618][20381] Updated weights for policy 1, policy_version 1641821 (0.0010) [2023-12-27 17:01:53,640][20347] Updated weights for policy 0, policy_version 1643421 (0.0006) [2023-12-27 17:01:53,663][20381] Updated weights for policy 1, policy_version 1641831 (0.0008) [2023-12-27 17:01:53,697][20347] Updated weights for policy 0, policy_version 1643431 (0.0007) [2023-12-27 17:01:54,302][20347] Updated weights for policy 0, policy_version 1643441 (0.0007) [2023-12-27 17:01:54,357][20347] Updated weights for policy 0, policy_version 1643451 (0.0006) [2023-12-27 17:01:54,360][20381] Updated weights for policy 1, policy_version 1641841 (0.0008) [2023-12-27 17:01:54,416][20347] Updated weights for policy 0, policy_version 1643461 (0.0006) [2023-12-27 17:01:54,423][20381] Updated weights for policy 1, policy_version 1641851 (0.0007) [2023-12-27 17:01:54,469][20347] Updated weights for policy 0, policy_version 1643471 (0.0006) [2023-12-27 17:01:54,475][20381] Updated weights for policy 1, policy_version 1641861 (0.0005) [2023-12-27 17:01:54,527][20381] Updated weights for policy 1, policy_version 1641871 (0.0007) [2023-12-27 17:01:55,089][20347] Updated weights for policy 0, policy_version 1643481 (0.0008) [2023-12-27 17:01:55,149][20347] Updated weights for policy 0, policy_version 1643491 (0.0008) [2023-12-27 17:01:55,198][20347] Updated weights for policy 0, policy_version 1643501 (0.0007) [2023-12-27 17:01:55,210][20381] Updated weights for policy 1, policy_version 1641881 (0.0006) [2023-12-27 17:01:55,268][20381] Updated weights for policy 1, policy_version 1641891 (0.0008) [2023-12-27 17:01:55,331][20381] Updated weights for policy 1, policy_version 1641901 (0.0007) [2023-12-27 17:01:55,953][20347] Updated weights for policy 0, policy_version 1643511 (0.0007) [2023-12-27 17:01:55,961][20381] Updated weights for policy 1, policy_version 1641911 (0.0006) [2023-12-27 17:01:56,007][20347] Updated weights for policy 0, policy_version 1643521 (0.0007) [2023-12-27 17:01:56,019][20381] Updated weights for policy 1, policy_version 1641921 (0.0007) [2023-12-27 17:01:56,072][20347] Updated weights for policy 0, policy_version 1643531 (0.0008) [2023-12-27 17:01:56,078][20381] Updated weights for policy 1, policy_version 1641931 (0.0006) [2023-12-27 17:01:56,622][20347] Updated weights for policy 0, policy_version 1643541 (0.0006) [2023-12-27 17:01:56,668][20347] Updated weights for policy 0, policy_version 1643551 (0.0005) [2023-12-27 17:01:56,711][20347] Updated weights for policy 0, policy_version 1643561 (0.0005) [2023-12-27 17:01:56,768][20381] Updated weights for policy 1, policy_version 1641941 (0.0007) [2023-12-27 17:01:56,813][20381] Updated weights for policy 1, policy_version 1641951 (0.0008) [2023-12-27 17:01:56,866][20381] Updated weights for policy 1, policy_version 1641961 (0.0009) [2023-12-27 17:01:57,251][20347] Updated weights for policy 0, policy_version 1643571 (0.0005) [2023-12-27 17:01:57,305][20347] Updated weights for policy 0, policy_version 1643581 (0.0006) [2023-12-27 17:01:57,308][20291] KL-divergence is very high: 597.1934 [2023-12-27 17:01:57,346][20291] KL-divergence is very high: 633.5059 [2023-12-27 17:01:57,357][20347] Updated weights for policy 0, policy_version 1643591 (0.0008) [2023-12-27 17:01:57,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 841220096. Throughput: 0: 10280.2, 1: 10279.4. Samples: 841210408. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:01:57,363][19170] Avg episode reward: [(0, '990.220'), (1, '765.980')] [2023-12-27 17:01:57,387][20291] KL-divergence is very high: 294.7719 [2023-12-27 17:01:57,503][20381] Updated weights for policy 1, policy_version 1641971 (0.0008) [2023-12-27 17:01:57,556][20381] Updated weights for policy 1, policy_version 1641981 (0.0005) [2023-12-27 17:01:57,598][20381] Updated weights for policy 1, policy_version 1641991 (0.0005) [2023-12-27 17:01:57,966][20347] Updated weights for policy 0, policy_version 1643601 (0.0008) [2023-12-27 17:01:58,023][20347] Updated weights for policy 0, policy_version 1643611 (0.0010) [2023-12-27 17:01:58,071][20347] Updated weights for policy 0, policy_version 1643621 (0.0010) [2023-12-27 17:01:58,124][20347] Updated weights for policy 0, policy_version 1643631 (0.0010) [2023-12-27 17:01:58,245][20381] Updated weights for policy 1, policy_version 1642001 (0.0007) [2023-12-27 17:01:58,307][20381] Updated weights for policy 1, policy_version 1642011 (0.0010) [2023-12-27 17:01:58,373][20381] Updated weights for policy 1, policy_version 1642021 (0.0008) [2023-12-27 17:01:58,438][20381] Updated weights for policy 1, policy_version 1642031 (0.0007) [2023-12-27 17:01:58,956][20347] Updated weights for policy 0, policy_version 1643641 (0.0008) [2023-12-27 17:01:59,016][20347] Updated weights for policy 0, policy_version 1643651 (0.0008) [2023-12-27 17:01:59,074][20347] Updated weights for policy 0, policy_version 1643661 (0.0008) [2023-12-27 17:01:59,194][20381] Updated weights for policy 1, policy_version 1642041 (0.0010) [2023-12-27 17:01:59,262][20381] Updated weights for policy 1, policy_version 1642051 (0.0010) [2023-12-27 17:01:59,332][20381] Updated weights for policy 1, policy_version 1642061 (0.0008) [2023-12-27 17:01:59,871][20347] Updated weights for policy 0, policy_version 1643671 (0.0008) [2023-12-27 17:01:59,926][20347] Updated weights for policy 0, policy_version 1643681 (0.0007) [2023-12-27 17:01:59,994][20347] Updated weights for policy 0, policy_version 1643691 (0.0008) [2023-12-27 17:02:00,028][20381] Updated weights for policy 1, policy_version 1642071 (0.0009) [2023-12-27 17:02:00,088][20381] Updated weights for policy 1, policy_version 1642081 (0.0010) [2023-12-27 17:02:00,147][20381] Updated weights for policy 1, policy_version 1642091 (0.0008) [2023-12-27 17:02:00,664][20347] Updated weights for policy 0, policy_version 1643701 (0.0009) [2023-12-27 17:02:00,722][20347] Updated weights for policy 0, policy_version 1643711 (0.0010) [2023-12-27 17:02:00,729][20381] Updated weights for policy 1, policy_version 1642101 (0.0005) [2023-12-27 17:02:00,786][20347] Updated weights for policy 0, policy_version 1643721 (0.0010) [2023-12-27 17:02:00,793][20381] Updated weights for policy 1, policy_version 1642111 (0.0005) [2023-12-27 17:02:00,849][20381] Updated weights for policy 1, policy_version 1642121 (0.0007) [2023-12-27 17:02:01,438][20381] Updated weights for policy 1, policy_version 1642131 (0.0006) [2023-12-27 17:02:01,443][20347] Updated weights for policy 0, policy_version 1643731 (0.0009) [2023-12-27 17:02:01,502][20381] Updated weights for policy 1, policy_version 1642141 (0.0009) [2023-12-27 17:02:01,502][20347] Updated weights for policy 0, policy_version 1643741 (0.0006) [2023-12-27 17:02:01,562][20347] Updated weights for policy 0, policy_version 1643751 (0.0006) [2023-12-27 17:02:01,572][20381] Updated weights for policy 1, policy_version 1642151 (0.0010) [2023-12-27 17:02:02,235][20347] Updated weights for policy 0, policy_version 1643761 (0.0007) [2023-12-27 17:02:02,242][20381] Updated weights for policy 1, policy_version 1642161 (0.0011) [2023-12-27 17:02:02,297][20347] Updated weights for policy 0, policy_version 1643771 (0.0008) [2023-12-27 17:02:02,301][20381] Updated weights for policy 1, policy_version 1642171 (0.0011) [2023-12-27 17:02:02,351][20347] Updated weights for policy 0, policy_version 1643781 (0.0006) [2023-12-27 17:02:02,360][20381] Updated weights for policy 1, policy_version 1642181 (0.0011) [2023-12-27 17:02:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 841318400. Throughput: 0: 10323.6, 1: 10325.7. Samples: 841335552. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:02:02,363][19170] Avg episode reward: [(0, '970.740'), (1, '795.440')] [2023-12-27 17:02:02,415][20347] Updated weights for policy 0, policy_version 1643791 (0.0007) [2023-12-27 17:02:02,425][20381] Updated weights for policy 1, policy_version 1642191 (0.0011) [2023-12-27 17:02:03,016][20381] Updated weights for policy 1, policy_version 1642201 (0.0008) [2023-12-27 17:02:03,029][20347] Updated weights for policy 0, policy_version 1643801 (0.0007) [2023-12-27 17:02:03,067][20381] Updated weights for policy 1, policy_version 1642211 (0.0010) [2023-12-27 17:02:03,081][20347] Updated weights for policy 0, policy_version 1643811 (0.0005) [2023-12-27 17:02:03,114][20381] Updated weights for policy 1, policy_version 1642221 (0.0010) [2023-12-27 17:02:03,135][20347] Updated weights for policy 0, policy_version 1643821 (0.0005) [2023-12-27 17:02:03,749][20381] Updated weights for policy 1, policy_version 1642231 (0.0008) [2023-12-27 17:02:03,799][20381] Updated weights for policy 1, policy_version 1642241 (0.0006) [2023-12-27 17:02:03,812][20347] Updated weights for policy 0, policy_version 1643831 (0.0009) [2023-12-27 17:02:03,860][20381] Updated weights for policy 1, policy_version 1642251 (0.0007) [2023-12-27 17:02:03,869][20347] Updated weights for policy 0, policy_version 1643841 (0.0010) [2023-12-27 17:02:03,921][20347] Updated weights for policy 0, policy_version 1643851 (0.0010) [2023-12-27 17:02:04,541][20381] Updated weights for policy 1, policy_version 1642261 (0.0007) [2023-12-27 17:02:04,599][20381] Updated weights for policy 1, policy_version 1642271 (0.0007) [2023-12-27 17:02:04,611][20347] Updated weights for policy 0, policy_version 1643861 (0.0011) [2023-12-27 17:02:04,661][20381] Updated weights for policy 1, policy_version 1642281 (0.0008) [2023-12-27 17:02:04,668][20347] Updated weights for policy 0, policy_version 1643871 (0.0011) [2023-12-27 17:02:04,726][20347] Updated weights for policy 0, policy_version 1643881 (0.0010) [2023-12-27 17:02:05,284][20381] Updated weights for policy 1, policy_version 1642291 (0.0007) [2023-12-27 17:02:05,338][20381] Updated weights for policy 1, policy_version 1642301 (0.0010) [2023-12-27 17:02:05,398][20381] Updated weights for policy 1, policy_version 1642311 (0.0010) [2023-12-27 17:02:05,402][20347] Updated weights for policy 0, policy_version 1643891 (0.0010) [2023-12-27 17:02:05,452][20347] Updated weights for policy 0, policy_version 1643901 (0.0010) [2023-12-27 17:02:05,509][20347] Updated weights for policy 0, policy_version 1643911 (0.0010) [2023-12-27 17:02:06,032][20381] Updated weights for policy 1, policy_version 1642321 (0.0009) [2023-12-27 17:02:06,076][20347] Updated weights for policy 0, policy_version 1643921 (0.0006) [2023-12-27 17:02:06,091][20381] Updated weights for policy 1, policy_version 1642331 (0.0005) [2023-12-27 17:02:06,142][20347] Updated weights for policy 0, policy_version 1643931 (0.0007) [2023-12-27 17:02:06,154][20381] Updated weights for policy 1, policy_version 1642341 (0.0008) [2023-12-27 17:02:06,201][20347] Updated weights for policy 0, policy_version 1643941 (0.0007) [2023-12-27 17:02:06,215][20381] Updated weights for policy 1, policy_version 1642351 (0.0007) [2023-12-27 17:02:06,260][20347] Updated weights for policy 0, policy_version 1643951 (0.0008) [2023-12-27 17:02:06,866][20347] Updated weights for policy 0, policy_version 1643961 (0.0008) [2023-12-27 17:02:06,927][20347] Updated weights for policy 0, policy_version 1643971 (0.0006) [2023-12-27 17:02:06,934][20381] Updated weights for policy 1, policy_version 1642361 (0.0006) [2023-12-27 17:02:06,978][20347] Updated weights for policy 0, policy_version 1643981 (0.0005) [2023-12-27 17:02:06,987][20381] Updated weights for policy 1, policy_version 1642371 (0.0008) [2023-12-27 17:02:07,044][20381] Updated weights for policy 1, policy_version 1642381 (0.0008) [2023-12-27 17:02:07,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 841433088. Throughput: 0: 10325.7, 1: 10349.3. Samples: 841399024. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:02:07,363][19170] Avg episode reward: [(0, '961.000'), (1, '795.290')] [2023-12-27 17:02:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001643984_420937728.pth... [2023-12-27 17:02:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001642384_420495360.pth... [2023-12-27 17:02:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001641168_420184064.pth [2023-12-27 17:02:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001642768_420626432.pth [2023-12-27 17:02:07,599][20347] Updated weights for policy 0, policy_version 1643991 (0.0007) [2023-12-27 17:02:07,655][20347] Updated weights for policy 0, policy_version 1644001 (0.0008) [2023-12-27 17:02:07,715][20347] Updated weights for policy 0, policy_version 1644011 (0.0006) [2023-12-27 17:02:07,749][20381] Updated weights for policy 1, policy_version 1642391 (0.0007) [2023-12-27 17:02:07,796][20381] Updated weights for policy 1, policy_version 1642401 (0.0007) [2023-12-27 17:02:07,851][20381] Updated weights for policy 1, policy_version 1642411 (0.0007) [2023-12-27 17:02:08,321][20347] Updated weights for policy 0, policy_version 1644021 (0.0008) [2023-12-27 17:02:08,388][20347] Updated weights for policy 0, policy_version 1644031 (0.0008) [2023-12-27 17:02:08,450][20347] Updated weights for policy 0, policy_version 1644041 (0.0009) [2023-12-27 17:02:08,496][20381] Updated weights for policy 1, policy_version 1642421 (0.0007) [2023-12-27 17:02:08,558][20381] Updated weights for policy 1, policy_version 1642431 (0.0007) [2023-12-27 17:02:08,618][20381] Updated weights for policy 1, policy_version 1642441 (0.0009) [2023-12-27 17:02:09,079][20347] Updated weights for policy 0, policy_version 1644051 (0.0007) [2023-12-27 17:02:09,137][20347] Updated weights for policy 0, policy_version 1644061 (0.0009) [2023-12-27 17:02:09,195][20347] Updated weights for policy 0, policy_version 1644071 (0.0010) [2023-12-27 17:02:09,312][20381] Updated weights for policy 1, policy_version 1642451 (0.0010) [2023-12-27 17:02:09,380][20381] Updated weights for policy 1, policy_version 1642461 (0.0007) [2023-12-27 17:02:09,449][20381] Updated weights for policy 1, policy_version 1642471 (0.0008) [2023-12-27 17:02:09,957][20347] Updated weights for policy 0, policy_version 1644081 (0.0008) [2023-12-27 17:02:10,029][20347] Updated weights for policy 0, policy_version 1644091 (0.0008) [2023-12-27 17:02:10,097][20347] Updated weights for policy 0, policy_version 1644101 (0.0007) [2023-12-27 17:02:10,168][20347] Updated weights for policy 0, policy_version 1644111 (0.0007) [2023-12-27 17:02:10,194][20381] Updated weights for policy 1, policy_version 1642481 (0.0006) [2023-12-27 17:02:10,251][20381] Updated weights for policy 1, policy_version 1642491 (0.0008) [2023-12-27 17:02:10,309][20381] Updated weights for policy 1, policy_version 1642501 (0.0007) [2023-12-27 17:02:10,369][20381] Updated weights for policy 1, policy_version 1642511 (0.0007) [2023-12-27 17:02:10,849][20347] Updated weights for policy 0, policy_version 1644121 (0.0008) [2023-12-27 17:02:10,908][20347] Updated weights for policy 0, policy_version 1644131 (0.0008) [2023-12-27 17:02:10,966][20347] Updated weights for policy 0, policy_version 1644141 (0.0008) [2023-12-27 17:02:11,066][20381] Updated weights for policy 1, policy_version 1642521 (0.0008) [2023-12-27 17:02:11,132][20381] Updated weights for policy 1, policy_version 1642531 (0.0007) [2023-12-27 17:02:11,193][20381] Updated weights for policy 1, policy_version 1642541 (0.0007) [2023-12-27 17:02:11,691][20347] Updated weights for policy 0, policy_version 1644151 (0.0008) [2023-12-27 17:02:11,751][20347] Updated weights for policy 0, policy_version 1644161 (0.0009) [2023-12-27 17:02:11,810][20347] Updated weights for policy 0, policy_version 1644171 (0.0009) [2023-12-27 17:02:11,929][20381] Updated weights for policy 1, policy_version 1642551 (0.0007) [2023-12-27 17:02:11,985][20381] Updated weights for policy 1, policy_version 1642561 (0.0007) [2023-12-27 17:02:12,048][20381] Updated weights for policy 1, policy_version 1642571 (0.0007) [2023-12-27 17:02:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 841531392. Throughput: 0: 10392.4, 1: 10361.4. Samples: 841521928. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:02:12,363][19170] Avg episode reward: [(0, '970.730'), (1, '824.250')] [2023-12-27 17:02:12,505][20347] Updated weights for policy 0, policy_version 1644181 (0.0008) [2023-12-27 17:02:12,523][20291] KL-divergence is very high: 180.4710 [2023-12-27 17:02:12,562][20347] Updated weights for policy 0, policy_version 1644191 (0.0008) [2023-12-27 17:02:12,569][20291] KL-divergence is very high: 221.3899 [2023-12-27 17:02:12,617][20291] KL-divergence is very high: 121.7572 [2023-12-27 17:02:12,623][20347] Updated weights for policy 0, policy_version 1644201 (0.0008) [2023-12-27 17:02:12,706][20381] Updated weights for policy 1, policy_version 1642581 (0.0006) [2023-12-27 17:02:12,757][20381] Updated weights for policy 1, policy_version 1642591 (0.0009) [2023-12-27 17:02:12,811][20381] Updated weights for policy 1, policy_version 1642601 (0.0009) [2023-12-27 17:02:13,232][20347] Updated weights for policy 0, policy_version 1644211 (0.0007) [2023-12-27 17:02:13,286][20347] Updated weights for policy 0, policy_version 1644221 (0.0007) [2023-12-27 17:02:13,335][20347] Updated weights for policy 0, policy_version 1644231 (0.0007) [2023-12-27 17:02:13,524][20381] Updated weights for policy 1, policy_version 1642611 (0.0008) [2023-12-27 17:02:13,577][20381] Updated weights for policy 1, policy_version 1642621 (0.0007) [2023-12-27 17:02:13,628][20381] Updated weights for policy 1, policy_version 1642631 (0.0008) [2023-12-27 17:02:13,969][20347] Updated weights for policy 0, policy_version 1644241 (0.0007) [2023-12-27 17:02:14,024][20347] Updated weights for policy 0, policy_version 1644251 (0.0007) [2023-12-27 17:02:14,081][20347] Updated weights for policy 0, policy_version 1644261 (0.0008) [2023-12-27 17:02:14,137][20347] Updated weights for policy 0, policy_version 1644271 (0.0007) [2023-12-27 17:02:14,346][20381] Updated weights for policy 1, policy_version 1642641 (0.0008) [2023-12-27 17:02:14,404][20381] Updated weights for policy 1, policy_version 1642651 (0.0007) [2023-12-27 17:02:14,462][20381] Updated weights for policy 1, policy_version 1642661 (0.0007) [2023-12-27 17:02:14,516][20381] Updated weights for policy 1, policy_version 1642671 (0.0007) [2023-12-27 17:02:14,828][20347] Updated weights for policy 0, policy_version 1644281 (0.0007) [2023-12-27 17:02:14,885][20347] Updated weights for policy 0, policy_version 1644291 (0.0006) [2023-12-27 17:02:14,943][20347] Updated weights for policy 0, policy_version 1644301 (0.0007) [2023-12-27 17:02:15,231][20381] Updated weights for policy 1, policy_version 1642681 (0.0008) [2023-12-27 17:02:15,295][20381] Updated weights for policy 1, policy_version 1642691 (0.0007) [2023-12-27 17:02:15,355][20381] Updated weights for policy 1, policy_version 1642701 (0.0008) [2023-12-27 17:02:15,604][20347] Updated weights for policy 0, policy_version 1644311 (0.0006) [2023-12-27 17:02:15,673][20347] Updated weights for policy 0, policy_version 1644321 (0.0006) [2023-12-27 17:02:15,727][20347] Updated weights for policy 0, policy_version 1644331 (0.0005) [2023-12-27 17:02:16,070][20381] Updated weights for policy 1, policy_version 1642711 (0.0007) [2023-12-27 17:02:16,122][20381] Updated weights for policy 1, policy_version 1642721 (0.0009) [2023-12-27 17:02:16,178][20381] Updated weights for policy 1, policy_version 1642731 (0.0009) [2023-12-27 17:02:16,280][20347] Updated weights for policy 0, policy_version 1644341 (0.0006) [2023-12-27 17:02:16,343][20347] Updated weights for policy 0, policy_version 1644351 (0.0007) [2023-12-27 17:02:16,395][20347] Updated weights for policy 0, policy_version 1644361 (0.0008) [2023-12-27 17:02:16,896][20381] Updated weights for policy 1, policy_version 1642741 (0.0007) [2023-12-27 17:02:16,951][20381] Updated weights for policy 1, policy_version 1642751 (0.0007) [2023-12-27 17:02:17,003][20381] Updated weights for policy 1, policy_version 1642761 (0.0007) [2023-12-27 17:02:17,077][20347] Updated weights for policy 0, policy_version 1644371 (0.0008) [2023-12-27 17:02:17,130][20347] Updated weights for policy 0, policy_version 1644381 (0.0006) [2023-12-27 17:02:17,176][20347] Updated weights for policy 0, policy_version 1644391 (0.0005) [2023-12-27 17:02:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 841637888. Throughput: 0: 10424.3, 1: 10249.9. Samples: 841644296. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:02:17,363][19170] Avg episode reward: [(0, '980.500'), (1, '833.980')] [2023-12-27 17:02:17,662][20381] Updated weights for policy 1, policy_version 1642771 (0.0007) [2023-12-27 17:02:17,707][20381] Updated weights for policy 1, policy_version 1642781 (0.0007) [2023-12-27 17:02:17,756][20381] Updated weights for policy 1, policy_version 1642791 (0.0005) [2023-12-27 17:02:17,872][20347] Updated weights for policy 0, policy_version 1644401 (0.0006) [2023-12-27 17:02:17,919][20347] Updated weights for policy 0, policy_version 1644411 (0.0010) [2023-12-27 17:02:17,976][20347] Updated weights for policy 0, policy_version 1644421 (0.0009) [2023-12-27 17:02:18,034][20347] Updated weights for policy 0, policy_version 1644431 (0.0008) [2023-12-27 17:02:18,322][20381] Updated weights for policy 1, policy_version 1642801 (0.0006) [2023-12-27 17:02:18,386][20381] Updated weights for policy 1, policy_version 1642811 (0.0006) [2023-12-27 17:02:18,446][20381] Updated weights for policy 1, policy_version 1642821 (0.0009) [2023-12-27 17:02:18,504][20381] Updated weights for policy 1, policy_version 1642831 (0.0007) [2023-12-27 17:02:18,758][20347] Updated weights for policy 0, policy_version 1644441 (0.0008) [2023-12-27 17:02:18,810][20347] Updated weights for policy 0, policy_version 1644451 (0.0008) [2023-12-27 17:02:18,869][20347] Updated weights for policy 0, policy_version 1644461 (0.0008) [2023-12-27 17:02:19,188][20381] Updated weights for policy 1, policy_version 1642841 (0.0008) [2023-12-27 17:02:19,253][20381] Updated weights for policy 1, policy_version 1642851 (0.0008) [2023-12-27 17:02:19,316][20381] Updated weights for policy 1, policy_version 1642861 (0.0008) [2023-12-27 17:02:19,564][20347] Updated weights for policy 0, policy_version 1644471 (0.0007) [2023-12-27 17:02:19,616][20347] Updated weights for policy 0, policy_version 1644481 (0.0008) [2023-12-27 17:02:19,679][20347] Updated weights for policy 0, policy_version 1644491 (0.0008) [2023-12-27 17:02:19,994][20381] Updated weights for policy 1, policy_version 1642871 (0.0006) [2023-12-27 17:02:20,047][20381] Updated weights for policy 1, policy_version 1642881 (0.0006) [2023-12-27 17:02:20,098][20381] Updated weights for policy 1, policy_version 1642891 (0.0005) [2023-12-27 17:02:20,421][20347] Updated weights for policy 0, policy_version 1644501 (0.0009) [2023-12-27 17:02:20,482][20347] Updated weights for policy 0, policy_version 1644511 (0.0007) [2023-12-27 17:02:20,543][20347] Updated weights for policy 0, policy_version 1644521 (0.0008) [2023-12-27 17:02:20,792][20381] Updated weights for policy 1, policy_version 1642901 (0.0008) [2023-12-27 17:02:20,854][20381] Updated weights for policy 1, policy_version 1642911 (0.0008) [2023-12-27 17:02:20,917][20381] Updated weights for policy 1, policy_version 1642921 (0.0008) [2023-12-27 17:02:21,265][20347] Updated weights for policy 0, policy_version 1644531 (0.0009) [2023-12-27 17:02:21,329][20347] Updated weights for policy 0, policy_version 1644541 (0.0008) [2023-12-27 17:02:21,401][20347] Updated weights for policy 0, policy_version 1644551 (0.0008) [2023-12-27 17:02:21,634][20381] Updated weights for policy 1, policy_version 1642931 (0.0009) [2023-12-27 17:02:21,698][20381] Updated weights for policy 1, policy_version 1642941 (0.0007) [2023-12-27 17:02:21,768][20381] Updated weights for policy 1, policy_version 1642951 (0.0008) [2023-12-27 17:02:22,162][20347] Updated weights for policy 0, policy_version 1644561 (0.0007) [2023-12-27 17:02:22,220][20347] Updated weights for policy 0, policy_version 1644571 (0.0005) [2023-12-27 17:02:22,281][20347] Updated weights for policy 0, policy_version 1644581 (0.0007) [2023-12-27 17:02:22,331][20347] Updated weights for policy 0, policy_version 1644591 (0.0008) [2023-12-27 17:02:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 841736192. Throughput: 0: 10364.5, 1: 10241.4. Samples: 841705876. Policy #0 lag: (min: 2.0, avg: 14.0, max: 34.0) [2023-12-27 17:02:22,363][19170] Avg episode reward: [(0, '990.250'), (1, '716.920')] [2023-12-27 17:02:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001642960_420642816.pth... [2023-12-27 17:02:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001644592_421093376.pth... [2023-12-27 17:02:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001641776_420339712.pth [2023-12-27 17:02:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001643376_420782080.pth [2023-12-27 17:02:22,456][20381] Updated weights for policy 1, policy_version 1642961 (0.0008) [2023-12-27 17:02:22,517][20381] Updated weights for policy 1, policy_version 1642971 (0.0008) [2023-12-27 17:02:22,582][20381] Updated weights for policy 1, policy_version 1642981 (0.0006) [2023-12-27 17:02:22,647][20381] Updated weights for policy 1, policy_version 1642991 (0.0006) [2023-12-27 17:02:23,053][20347] Updated weights for policy 0, policy_version 1644601 (0.0007) [2023-12-27 17:02:23,100][20347] Updated weights for policy 0, policy_version 1644611 (0.0008) [2023-12-27 17:02:23,147][20347] Updated weights for policy 0, policy_version 1644621 (0.0007) [2023-12-27 17:02:23,288][20381] Updated weights for policy 1, policy_version 1643001 (0.0007) [2023-12-27 17:02:23,340][20381] Updated weights for policy 1, policy_version 1643011 (0.0007) [2023-12-27 17:02:23,394][20381] Updated weights for policy 1, policy_version 1643021 (0.0010) [2023-12-27 17:02:23,803][20347] Updated weights for policy 0, policy_version 1644631 (0.0008) [2023-12-27 17:02:23,855][20347] Updated weights for policy 0, policy_version 1644641 (0.0007) [2023-12-27 17:02:23,905][20347] Updated weights for policy 0, policy_version 1644651 (0.0005) [2023-12-27 17:02:24,081][20381] Updated weights for policy 1, policy_version 1643031 (0.0009) [2023-12-27 17:02:24,143][20381] Updated weights for policy 1, policy_version 1643041 (0.0007) [2023-12-27 17:02:24,204][20381] Updated weights for policy 1, policy_version 1643051 (0.0008) [2023-12-27 17:02:24,535][20347] Updated weights for policy 0, policy_version 1644661 (0.0007) [2023-12-27 17:02:24,579][20347] Updated weights for policy 0, policy_version 1644671 (0.0006) [2023-12-27 17:02:24,627][20347] Updated weights for policy 0, policy_version 1644681 (0.0007) [2023-12-27 17:02:24,921][20381] Updated weights for policy 1, policy_version 1643061 (0.0007) [2023-12-27 17:02:24,984][20381] Updated weights for policy 1, policy_version 1643071 (0.0007) [2023-12-27 17:02:25,047][20381] Updated weights for policy 1, policy_version 1643081 (0.0007) [2023-12-27 17:02:25,259][20347] Updated weights for policy 0, policy_version 1644691 (0.0007) [2023-12-27 17:02:25,305][20347] Updated weights for policy 0, policy_version 1644701 (0.0005) [2023-12-27 17:02:25,351][20347] Updated weights for policy 0, policy_version 1644711 (0.0005) [2023-12-27 17:02:25,766][20381] Updated weights for policy 1, policy_version 1643091 (0.0008) [2023-12-27 17:02:25,834][20381] Updated weights for policy 1, policy_version 1643101 (0.0007) [2023-12-27 17:02:25,898][20381] Updated weights for policy 1, policy_version 1643111 (0.0008) [2023-12-27 17:02:25,957][20347] Updated weights for policy 0, policy_version 1644721 (0.0008) [2023-12-27 17:02:26,018][20347] Updated weights for policy 0, policy_version 1644731 (0.0006) [2023-12-27 17:02:26,071][20347] Updated weights for policy 0, policy_version 1644741 (0.0005) [2023-12-27 17:02:26,128][20347] Updated weights for policy 0, policy_version 1644751 (0.0006) [2023-12-27 17:02:26,657][20347] Updated weights for policy 0, policy_version 1644761 (0.0005) [2023-12-27 17:02:26,663][20381] Updated weights for policy 1, policy_version 1643121 (0.0008) [2023-12-27 17:02:26,710][20347] Updated weights for policy 0, policy_version 1644771 (0.0005) [2023-12-27 17:02:26,715][20381] Updated weights for policy 1, policy_version 1643131 (0.0007) [2023-12-27 17:02:26,735][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000006 [2023-12-27 17:02:26,772][20381] Updated weights for policy 1, policy_version 1643141 (0.0008) [2023-12-27 17:02:26,816][20381] Updated weights for policy 1, policy_version 1643151 (0.0007) [2023-12-27 17:02:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 841834496. Throughput: 0: 10363.5, 1: 10236.6. Samples: 841825956. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '766.020')] [2023-12-27 17:02:27,389][20347] Updated weights for policy 0, policy_version 1644781 (0.0006) [2023-12-27 17:02:27,442][20347] Updated weights for policy 0, policy_version 1644791 (0.0007) [2023-12-27 17:02:27,489][20347] Updated weights for policy 0, policy_version 1644801 (0.0007) [2023-12-27 17:02:27,527][20381] Updated weights for policy 1, policy_version 1643161 (0.0007) [2023-12-27 17:02:27,583][20381] Updated weights for policy 1, policy_version 1643171 (0.0008) [2023-12-27 17:02:27,630][20381] Updated weights for policy 1, policy_version 1643181 (0.0007) [2023-12-27 17:02:28,190][20347] Updated weights for policy 0, policy_version 1644811 (0.0006) [2023-12-27 17:02:28,245][20347] Updated weights for policy 0, policy_version 1644821 (0.0008) [2023-12-27 17:02:28,293][20381] Updated weights for policy 1, policy_version 1643191 (0.0008) [2023-12-27 17:02:28,302][20347] Updated weights for policy 0, policy_version 1644831 (0.0005) [2023-12-27 17:02:28,345][20381] Updated weights for policy 1, policy_version 1643201 (0.0008) [2023-12-27 17:02:28,396][20381] Updated weights for policy 1, policy_version 1643211 (0.0006) [2023-12-27 17:02:28,905][20347] Updated weights for policy 0, policy_version 1644841 (0.0006) [2023-12-27 17:02:28,965][20347] Updated weights for policy 0, policy_version 1644851 (0.0007) [2023-12-27 17:02:29,023][20347] Updated weights for policy 0, policy_version 1644861 (0.0008) [2023-12-27 17:02:29,082][20347] Updated weights for policy 0, policy_version 1644871 (0.0007) [2023-12-27 17:02:29,129][20381] Updated weights for policy 1, policy_version 1643221 (0.0008) [2023-12-27 17:02:29,189][20381] Updated weights for policy 1, policy_version 1643231 (0.0007) [2023-12-27 17:02:29,254][20381] Updated weights for policy 1, policy_version 1643241 (0.0007) [2023-12-27 17:02:29,770][20347] Updated weights for policy 0, policy_version 1644881 (0.0010) [2023-12-27 17:02:29,835][20347] Updated weights for policy 0, policy_version 1644891 (0.0009) [2023-12-27 17:02:29,899][20347] Updated weights for policy 0, policy_version 1644901 (0.0008) [2023-12-27 17:02:29,939][20381] Updated weights for policy 1, policy_version 1643251 (0.0008) [2023-12-27 17:02:30,003][20381] Updated weights for policy 1, policy_version 1643261 (0.0008) [2023-12-27 17:02:30,054][20381] Updated weights for policy 1, policy_version 1643271 (0.0007) [2023-12-27 17:02:30,563][20347] Updated weights for policy 0, policy_version 1644911 (0.0007) [2023-12-27 17:02:30,612][20347] Updated weights for policy 0, policy_version 1644921 (0.0008) [2023-12-27 17:02:30,657][20347] Updated weights for policy 0, policy_version 1644931 (0.0008) [2023-12-27 17:02:30,747][20381] Updated weights for policy 1, policy_version 1643281 (0.0007) [2023-12-27 17:02:30,804][20381] Updated weights for policy 1, policy_version 1643291 (0.0006) [2023-12-27 17:02:30,857][20381] Updated weights for policy 1, policy_version 1643301 (0.0005) [2023-12-27 17:02:30,914][20381] Updated weights for policy 1, policy_version 1643311 (0.0005) [2023-12-27 17:02:31,344][20347] Updated weights for policy 0, policy_version 1644941 (0.0007) [2023-12-27 17:02:31,406][20347] Updated weights for policy 0, policy_version 1644951 (0.0007) [2023-12-27 17:02:31,463][20347] Updated weights for policy 0, policy_version 1644961 (0.0006) [2023-12-27 17:02:31,563][20381] Updated weights for policy 1, policy_version 1643321 (0.0009) [2023-12-27 17:02:31,617][20381] Updated weights for policy 1, policy_version 1643331 (0.0008) [2023-12-27 17:02:31,676][20381] Updated weights for policy 1, policy_version 1643341 (0.0009) [2023-12-27 17:02:32,052][20347] Updated weights for policy 0, policy_version 1644971 (0.0008) [2023-12-27 17:02:32,107][20347] Updated weights for policy 0, policy_version 1644981 (0.0006) [2023-12-27 17:02:32,168][20347] Updated weights for policy 0, policy_version 1644991 (0.0005) [2023-12-27 17:02:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 841940992. Throughput: 0: 10414.3, 1: 10149.7. Samples: 841950820. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.890')] [2023-12-27 17:02:32,400][20381] Updated weights for policy 1, policy_version 1643351 (0.0009) [2023-12-27 17:02:32,453][20381] Updated weights for policy 1, policy_version 1643361 (0.0008) [2023-12-27 17:02:32,454][20292] KL-divergence is very high: 124.1578 [2023-12-27 17:02:32,501][20292] KL-divergence is very high: 209.4591 [2023-12-27 17:02:32,512][20381] Updated weights for policy 1, policy_version 1643371 (0.0008) [2023-12-27 17:02:32,837][20347] Updated weights for policy 0, policy_version 1645001 (0.0006) [2023-12-27 17:02:32,888][20347] Updated weights for policy 0, policy_version 1645011 (0.0007) [2023-12-27 17:02:32,936][20347] Updated weights for policy 0, policy_version 1645021 (0.0007) [2023-12-27 17:02:32,983][20347] Updated weights for policy 0, policy_version 1645031 (0.0007) [2023-12-27 17:02:33,146][20292] KL-divergence is very high: 185.2745 [2023-12-27 17:02:33,177][20381] Updated weights for policy 1, policy_version 1643381 (0.0006) [2023-12-27 17:02:33,194][20292] KL-divergence is very high: 128.5857 [2023-12-27 17:02:33,231][20381] Updated weights for policy 1, policy_version 1643391 (0.0007) [2023-12-27 17:02:33,285][20381] Updated weights for policy 1, policy_version 1643401 (0.0010) [2023-12-27 17:02:33,697][20347] Updated weights for policy 0, policy_version 1645041 (0.0008) [2023-12-27 17:02:33,765][20347] Updated weights for policy 0, policy_version 1645051 (0.0007) [2023-12-27 17:02:33,822][20347] Updated weights for policy 0, policy_version 1645061 (0.0008) [2023-12-27 17:02:33,864][20381] Updated weights for policy 1, policy_version 1643411 (0.0006) [2023-12-27 17:02:33,912][20381] Updated weights for policy 1, policy_version 1643421 (0.0005) [2023-12-27 17:02:33,967][20381] Updated weights for policy 1, policy_version 1643431 (0.0005) [2023-12-27 17:02:34,431][20347] Updated weights for policy 0, policy_version 1645071 (0.0006) [2023-12-27 17:02:34,491][20347] Updated weights for policy 0, policy_version 1645081 (0.0007) [2023-12-27 17:02:34,558][20347] Updated weights for policy 0, policy_version 1645091 (0.0008) [2023-12-27 17:02:34,578][20381] Updated weights for policy 1, policy_version 1643441 (0.0005) [2023-12-27 17:02:34,628][20381] Updated weights for policy 1, policy_version 1643451 (0.0007) [2023-12-27 17:02:34,689][20381] Updated weights for policy 1, policy_version 1643461 (0.0008) [2023-12-27 17:02:34,749][20381] Updated weights for policy 1, policy_version 1643471 (0.0008) [2023-12-27 17:02:35,259][20347] Updated weights for policy 0, policy_version 1645101 (0.0008) [2023-12-27 17:02:35,307][20347] Updated weights for policy 0, policy_version 1645111 (0.0007) [2023-12-27 17:02:35,350][20347] Updated weights for policy 0, policy_version 1645121 (0.0007) [2023-12-27 17:02:35,438][20381] Updated weights for policy 1, policy_version 1643481 (0.0007) [2023-12-27 17:02:35,494][20381] Updated weights for policy 1, policy_version 1643491 (0.0008) [2023-12-27 17:02:35,551][20381] Updated weights for policy 1, policy_version 1643501 (0.0007) [2023-12-27 17:02:36,077][20347] Updated weights for policy 0, policy_version 1645131 (0.0007) [2023-12-27 17:02:36,134][20347] Updated weights for policy 0, policy_version 1645141 (0.0008) [2023-12-27 17:02:36,192][20347] Updated weights for policy 0, policy_version 1645151 (0.0007) [2023-12-27 17:02:36,231][20381] Updated weights for policy 1, policy_version 1643511 (0.0008) [2023-12-27 17:02:36,289][20381] Updated weights for policy 1, policy_version 1643521 (0.0007) [2023-12-27 17:02:36,351][20381] Updated weights for policy 1, policy_version 1643531 (0.0007) [2023-12-27 17:02:36,859][20347] Updated weights for policy 0, policy_version 1645161 (0.0007) [2023-12-27 17:02:36,923][20347] Updated weights for policy 0, policy_version 1645171 (0.0005) [2023-12-27 17:02:36,981][20347] Updated weights for policy 0, policy_version 1645181 (0.0005) [2023-12-27 17:02:37,016][20381] Updated weights for policy 1, policy_version 1643541 (0.0006) [2023-12-27 17:02:37,040][20347] Updated weights for policy 0, policy_version 1645191 (0.0006) [2023-12-27 17:02:37,070][20381] Updated weights for policy 1, policy_version 1643551 (0.0005) [2023-12-27 17:02:37,102][20292] KL-divergence is very high: 106.0636 [2023-12-27 17:02:37,121][20381] Updated weights for policy 1, policy_version 1643561 (0.0005) [2023-12-27 17:02:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 842047488. Throughput: 0: 10388.8, 1: 10191.0. Samples: 842013980. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '794.810')] [2023-12-27 17:02:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001645192_421249024.pth... [2023-12-27 17:02:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001643568_420798464.pth... [2023-12-27 17:02:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001642384_420495360.pth [2023-12-27 17:02:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001643984_420937728.pth [2023-12-27 17:02:37,569][20347] Updated weights for policy 0, policy_version 1645201 (0.0007) [2023-12-27 17:02:37,628][20347] Updated weights for policy 0, policy_version 1645211 (0.0008) [2023-12-27 17:02:37,688][20347] Updated weights for policy 0, policy_version 1645221 (0.0008) [2023-12-27 17:02:37,751][20381] Updated weights for policy 1, policy_version 1643571 (0.0008) [2023-12-27 17:02:37,807][20381] Updated weights for policy 1, policy_version 1643581 (0.0008) [2023-12-27 17:02:37,871][20381] Updated weights for policy 1, policy_version 1643591 (0.0008) [2023-12-27 17:02:38,304][20347] Updated weights for policy 0, policy_version 1645231 (0.0007) [2023-12-27 17:02:38,365][20347] Updated weights for policy 0, policy_version 1645241 (0.0008) [2023-12-27 17:02:38,419][20347] Updated weights for policy 0, policy_version 1645251 (0.0005) [2023-12-27 17:02:38,557][20381] Updated weights for policy 1, policy_version 1643601 (0.0008) [2023-12-27 17:02:38,617][20381] Updated weights for policy 1, policy_version 1643611 (0.0008) [2023-12-27 17:02:38,676][20381] Updated weights for policy 1, policy_version 1643621 (0.0008) [2023-12-27 17:02:38,737][20381] Updated weights for policy 1, policy_version 1643631 (0.0007) [2023-12-27 17:02:39,152][20347] Updated weights for policy 0, policy_version 1645261 (0.0006) [2023-12-27 17:02:39,201][20347] Updated weights for policy 0, policy_version 1645271 (0.0007) [2023-12-27 17:02:39,269][20347] Updated weights for policy 0, policy_version 1645281 (0.0008) [2023-12-27 17:02:39,302][20381] Updated weights for policy 1, policy_version 1643641 (0.0008) [2023-12-27 17:02:39,370][20381] Updated weights for policy 1, policy_version 1643651 (0.0008) [2023-12-27 17:02:39,441][20381] Updated weights for policy 1, policy_version 1643661 (0.0008) [2023-12-27 17:02:40,038][20347] Updated weights for policy 0, policy_version 1645291 (0.0006) [2023-12-27 17:02:40,090][20347] Updated weights for policy 0, policy_version 1645301 (0.0007) [2023-12-27 17:02:40,149][20347] Updated weights for policy 0, policy_version 1645311 (0.0006) [2023-12-27 17:02:40,165][20381] Updated weights for policy 1, policy_version 1643671 (0.0008) [2023-12-27 17:02:40,227][20381] Updated weights for policy 1, policy_version 1643681 (0.0011) [2023-12-27 17:02:40,289][20381] Updated weights for policy 1, policy_version 1643691 (0.0011) [2023-12-27 17:02:40,862][20347] Updated weights for policy 0, policy_version 1645321 (0.0008) [2023-12-27 17:02:40,922][20347] Updated weights for policy 0, policy_version 1645331 (0.0006) [2023-12-27 17:02:40,928][20381] Updated weights for policy 1, policy_version 1643701 (0.0011) [2023-12-27 17:02:40,984][20347] Updated weights for policy 0, policy_version 1645341 (0.0007) [2023-12-27 17:02:40,994][20381] Updated weights for policy 1, policy_version 1643711 (0.0009) [2023-12-27 17:02:41,047][20347] Updated weights for policy 0, policy_version 1645351 (0.0007) [2023-12-27 17:02:41,060][20381] Updated weights for policy 1, policy_version 1643721 (0.0008) [2023-12-27 17:02:41,779][20381] Updated weights for policy 1, policy_version 1643731 (0.0007) [2023-12-27 17:02:41,835][20347] Updated weights for policy 0, policy_version 1645361 (0.0008) [2023-12-27 17:02:41,841][20381] Updated weights for policy 1, policy_version 1643741 (0.0006) [2023-12-27 17:02:41,902][20381] Updated weights for policy 1, policy_version 1643751 (0.0008) [2023-12-27 17:02:41,903][20347] Updated weights for policy 0, policy_version 1645371 (0.0007) [2023-12-27 17:02:41,970][20347] Updated weights for policy 0, policy_version 1645381 (0.0007) [2023-12-27 17:02:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 842145792. Throughput: 0: 10390.6, 1: 10188.8. Samples: 842136480. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.530')] [2023-12-27 17:02:42,608][20347] Updated weights for policy 0, policy_version 1645391 (0.0007) [2023-12-27 17:02:42,642][20381] Updated weights for policy 1, policy_version 1643761 (0.0008) [2023-12-27 17:02:42,666][20347] Updated weights for policy 0, policy_version 1645401 (0.0011) [2023-12-27 17:02:42,701][20381] Updated weights for policy 1, policy_version 1643771 (0.0010) [2023-12-27 17:02:42,724][20347] Updated weights for policy 0, policy_version 1645411 (0.0008) [2023-12-27 17:02:42,751][20381] Updated weights for policy 1, policy_version 1643781 (0.0009) [2023-12-27 17:02:42,801][20381] Updated weights for policy 1, policy_version 1643791 (0.0006) [2023-12-27 17:02:43,296][20347] Updated weights for policy 0, policy_version 1645421 (0.0005) [2023-12-27 17:02:43,354][20347] Updated weights for policy 0, policy_version 1645431 (0.0005) [2023-12-27 17:02:43,411][20347] Updated weights for policy 0, policy_version 1645441 (0.0006) [2023-12-27 17:02:43,455][20381] Updated weights for policy 1, policy_version 1643801 (0.0010) [2023-12-27 17:02:43,506][20381] Updated weights for policy 1, policy_version 1643811 (0.0010) [2023-12-27 17:02:43,554][20381] Updated weights for policy 1, policy_version 1643821 (0.0010) [2023-12-27 17:02:44,015][20347] Updated weights for policy 0, policy_version 1645451 (0.0006) [2023-12-27 17:02:44,073][20347] Updated weights for policy 0, policy_version 1645461 (0.0007) [2023-12-27 17:02:44,126][20347] Updated weights for policy 0, policy_version 1645471 (0.0008) [2023-12-27 17:02:44,236][20381] Updated weights for policy 1, policy_version 1643831 (0.0010) [2023-12-27 17:02:44,297][20381] Updated weights for policy 1, policy_version 1643841 (0.0010) [2023-12-27 17:02:44,349][20381] Updated weights for policy 1, policy_version 1643851 (0.0010) [2023-12-27 17:02:44,845][20347] Updated weights for policy 0, policy_version 1645481 (0.0008) [2023-12-27 17:02:44,907][20347] Updated weights for policy 0, policy_version 1645491 (0.0007) [2023-12-27 17:02:44,974][20347] Updated weights for policy 0, policy_version 1645501 (0.0005) [2023-12-27 17:02:45,038][20347] Updated weights for policy 0, policy_version 1645511 (0.0007) [2023-12-27 17:02:45,055][20381] Updated weights for policy 1, policy_version 1643861 (0.0010) [2023-12-27 17:02:45,112][20381] Updated weights for policy 1, policy_version 1643871 (0.0011) [2023-12-27 17:02:45,171][20381] Updated weights for policy 1, policy_version 1643881 (0.0011) [2023-12-27 17:02:45,695][20347] Updated weights for policy 0, policy_version 1645521 (0.0010) [2023-12-27 17:02:45,755][20347] Updated weights for policy 0, policy_version 1645531 (0.0010) [2023-12-27 17:02:45,814][20347] Updated weights for policy 0, policy_version 1645541 (0.0009) [2023-12-27 17:02:45,833][20381] Updated weights for policy 1, policy_version 1643891 (0.0009) [2023-12-27 17:02:45,883][20381] Updated weights for policy 1, policy_version 1643901 (0.0007) [2023-12-27 17:02:45,939][20381] Updated weights for policy 1, policy_version 1643911 (0.0005) [2023-12-27 17:02:46,477][20347] Updated weights for policy 0, policy_version 1645551 (0.0007) [2023-12-27 17:02:46,501][20381] Updated weights for policy 1, policy_version 1643921 (0.0006) [2023-12-27 17:02:46,531][20347] Updated weights for policy 0, policy_version 1645561 (0.0010) [2023-12-27 17:02:46,559][20381] Updated weights for policy 1, policy_version 1643931 (0.0008) [2023-12-27 17:02:46,574][20347] Updated weights for policy 0, policy_version 1645571 (0.0010) [2023-12-27 17:02:46,607][20381] Updated weights for policy 1, policy_version 1643941 (0.0010) [2023-12-27 17:02:46,656][20381] Updated weights for policy 1, policy_version 1643951 (0.0010) [2023-12-27 17:02:47,160][20347] Updated weights for policy 0, policy_version 1645581 (0.0010) [2023-12-27 17:02:47,220][20347] Updated weights for policy 0, policy_version 1645591 (0.0010) [2023-12-27 17:02:47,280][20347] Updated weights for policy 0, policy_version 1645601 (0.0010) [2023-12-27 17:02:47,334][20381] Updated weights for policy 1, policy_version 1643961 (0.0010) [2023-12-27 17:02:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 842252288. Throughput: 0: 10417.7, 1: 10180.5. Samples: 842262476. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:47,363][19170] Avg episode reward: [(0, '980.500'), (1, '931.810')] [2023-12-27 17:02:47,383][20381] Updated weights for policy 1, policy_version 1643971 (0.0010) [2023-12-27 17:02:47,428][20381] Updated weights for policy 1, policy_version 1643981 (0.0010) [2023-12-27 17:02:47,940][20347] Updated weights for policy 0, policy_version 1645611 (0.0010) [2023-12-27 17:02:48,014][20347] Updated weights for policy 0, policy_version 1645621 (0.0010) [2023-12-27 17:02:48,040][20381] Updated weights for policy 1, policy_version 1643991 (0.0010) [2023-12-27 17:02:48,065][20347] Updated weights for policy 0, policy_version 1645631 (0.0010) [2023-12-27 17:02:48,089][20381] Updated weights for policy 1, policy_version 1644001 (0.0010) [2023-12-27 17:02:48,134][20381] Updated weights for policy 1, policy_version 1644011 (0.0010) [2023-12-27 17:02:48,723][20347] Updated weights for policy 0, policy_version 1645641 (0.0010) [2023-12-27 17:02:48,779][20347] Updated weights for policy 0, policy_version 1645651 (0.0008) [2023-12-27 17:02:48,834][20347] Updated weights for policy 0, policy_version 1645661 (0.0010) [2023-12-27 17:02:48,864][20381] Updated weights for policy 1, policy_version 1644021 (0.0010) [2023-12-27 17:02:48,894][20347] Updated weights for policy 0, policy_version 1645671 (0.0007) [2023-12-27 17:02:48,916][20381] Updated weights for policy 1, policy_version 1644031 (0.0007) [2023-12-27 17:02:48,969][20381] Updated weights for policy 1, policy_version 1644041 (0.0010) [2023-12-27 17:02:49,534][20347] Updated weights for policy 0, policy_version 1645681 (0.0008) [2023-12-27 17:02:49,592][20347] Updated weights for policy 0, policy_version 1645691 (0.0008) [2023-12-27 17:02:49,647][20347] Updated weights for policy 0, policy_version 1645701 (0.0007) [2023-12-27 17:02:49,679][20381] Updated weights for policy 1, policy_version 1644051 (0.0010) [2023-12-27 17:02:49,741][20381] Updated weights for policy 1, policy_version 1644061 (0.0009) [2023-12-27 17:02:49,806][20381] Updated weights for policy 1, policy_version 1644071 (0.0008) [2023-12-27 17:02:50,360][20347] Updated weights for policy 0, policy_version 1645711 (0.0006) [2023-12-27 17:02:50,419][20347] Updated weights for policy 0, policy_version 1645721 (0.0005) [2023-12-27 17:02:50,477][20347] Updated weights for policy 0, policy_version 1645731 (0.0006) [2023-12-27 17:02:50,490][20381] Updated weights for policy 1, policy_version 1644081 (0.0008) [2023-12-27 17:02:50,538][20381] Updated weights for policy 1, policy_version 1644091 (0.0005) [2023-12-27 17:02:50,595][20381] Updated weights for policy 1, policy_version 1644101 (0.0009) [2023-12-27 17:02:50,656][20381] Updated weights for policy 1, policy_version 1644111 (0.0011) [2023-12-27 17:02:51,057][20347] Updated weights for policy 0, policy_version 1645741 (0.0007) [2023-12-27 17:02:51,117][20347] Updated weights for policy 0, policy_version 1645751 (0.0007) [2023-12-27 17:02:51,177][20347] Updated weights for policy 0, policy_version 1645761 (0.0007) [2023-12-27 17:02:51,357][20381] Updated weights for policy 1, policy_version 1644121 (0.0008) [2023-12-27 17:02:51,420][20381] Updated weights for policy 1, policy_version 1644131 (0.0008) [2023-12-27 17:02:51,480][20381] Updated weights for policy 1, policy_version 1644141 (0.0006) [2023-12-27 17:02:51,961][20347] Updated weights for policy 0, policy_version 1645771 (0.0007) [2023-12-27 17:02:52,023][20347] Updated weights for policy 0, policy_version 1645781 (0.0008) [2023-12-27 17:02:52,091][20347] Updated weights for policy 0, policy_version 1645791 (0.0007) [2023-12-27 17:02:52,161][20381] Updated weights for policy 1, policy_version 1644151 (0.0006) [2023-12-27 17:02:52,223][20381] Updated weights for policy 1, policy_version 1644161 (0.0010) [2023-12-27 17:02:52,286][20381] Updated weights for policy 1, policy_version 1644171 (0.0011) [2023-12-27 17:02:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 842358784. Throughput: 0: 10406.8, 1: 10139.3. Samples: 842323596. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:52,363][19170] Avg episode reward: [(0, '980.500'), (1, '824.520')] [2023-12-27 17:02:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001644176_420954112.pth... [2023-12-27 17:02:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001645800_421404672.pth... [2023-12-27 17:02:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001642960_420642816.pth [2023-12-27 17:02:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001644592_421093376.pth [2023-12-27 17:02:52,738][20347] Updated weights for policy 0, policy_version 1645801 (0.0008) [2023-12-27 17:02:52,798][20347] Updated weights for policy 0, policy_version 1645811 (0.0006) [2023-12-27 17:02:52,854][20347] Updated weights for policy 0, policy_version 1645821 (0.0005) [2023-12-27 17:02:52,905][20347] Updated weights for policy 0, policy_version 1645831 (0.0005) [2023-12-27 17:02:53,004][20381] Updated weights for policy 1, policy_version 1644181 (0.0009) [2023-12-27 17:02:53,056][20381] Updated weights for policy 1, policy_version 1644191 (0.0007) [2023-12-27 17:02:53,108][20381] Updated weights for policy 1, policy_version 1644201 (0.0006) [2023-12-27 17:02:53,541][20347] Updated weights for policy 0, policy_version 1645841 (0.0005) [2023-12-27 17:02:53,601][20347] Updated weights for policy 0, policy_version 1645851 (0.0006) [2023-12-27 17:02:53,653][20347] Updated weights for policy 0, policy_version 1645861 (0.0005) [2023-12-27 17:02:53,712][20381] Updated weights for policy 1, policy_version 1644211 (0.0007) [2023-12-27 17:02:53,767][20381] Updated weights for policy 1, policy_version 1644221 (0.0010) [2023-12-27 17:02:53,828][20381] Updated weights for policy 1, policy_version 1644231 (0.0011) [2023-12-27 17:02:54,254][20347] Updated weights for policy 0, policy_version 1645871 (0.0007) [2023-12-27 17:02:54,309][20347] Updated weights for policy 0, policy_version 1645881 (0.0007) [2023-12-27 17:02:54,364][20347] Updated weights for policy 0, policy_version 1645891 (0.0008) [2023-12-27 17:02:54,519][20381] Updated weights for policy 1, policy_version 1644241 (0.0010) [2023-12-27 17:02:54,578][20381] Updated weights for policy 1, policy_version 1644251 (0.0009) [2023-12-27 17:02:54,634][20381] Updated weights for policy 1, policy_version 1644261 (0.0008) [2023-12-27 17:02:54,680][20381] Updated weights for policy 1, policy_version 1644271 (0.0008) [2023-12-27 17:02:55,062][20347] Updated weights for policy 0, policy_version 1645901 (0.0006) [2023-12-27 17:02:55,127][20347] Updated weights for policy 0, policy_version 1645911 (0.0005) [2023-12-27 17:02:55,187][20347] Updated weights for policy 0, policy_version 1645921 (0.0005) [2023-12-27 17:02:55,355][20381] Updated weights for policy 1, policy_version 1644281 (0.0006) [2023-12-27 17:02:55,408][20381] Updated weights for policy 1, policy_version 1644291 (0.0005) [2023-12-27 17:02:55,469][20381] Updated weights for policy 1, policy_version 1644301 (0.0005) [2023-12-27 17:02:55,714][20347] Updated weights for policy 0, policy_version 1645931 (0.0005) [2023-12-27 17:02:55,772][20347] Updated weights for policy 0, policy_version 1645941 (0.0007) [2023-12-27 17:02:55,828][20347] Updated weights for policy 0, policy_version 1645951 (0.0008) [2023-12-27 17:02:56,030][20381] Updated weights for policy 1, policy_version 1644311 (0.0006) [2023-12-27 17:02:56,078][20381] Updated weights for policy 1, policy_version 1644321 (0.0007) [2023-12-27 17:02:56,133][20381] Updated weights for policy 1, policy_version 1644331 (0.0008) [2023-12-27 17:02:56,559][20347] Updated weights for policy 0, policy_version 1645961 (0.0008) [2023-12-27 17:02:56,619][20347] Updated weights for policy 0, policy_version 1645971 (0.0008) [2023-12-27 17:02:56,685][20347] Updated weights for policy 0, policy_version 1645981 (0.0007) [2023-12-27 17:02:56,750][20347] Updated weights for policy 0, policy_version 1645991 (0.0005) [2023-12-27 17:02:56,776][20381] Updated weights for policy 1, policy_version 1644341 (0.0008) [2023-12-27 17:02:56,824][20381] Updated weights for policy 1, policy_version 1644351 (0.0008) [2023-12-27 17:02:56,868][20381] Updated weights for policy 1, policy_version 1644361 (0.0009) [2023-12-27 17:02:57,315][20347] Updated weights for policy 0, policy_version 1646001 (0.0007) [2023-12-27 17:02:57,363][20347] Updated weights for policy 0, policy_version 1646011 (0.0008) [2023-12-27 17:02:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 842457088. Throughput: 0: 10431.5, 1: 10187.6. Samples: 842449792. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:02:57,363][19170] Avg episode reward: [(0, '980.540'), (1, '814.820')] [2023-12-27 17:02:57,409][20347] Updated weights for policy 0, policy_version 1646021 (0.0007) [2023-12-27 17:02:57,445][20381] Updated weights for policy 1, policy_version 1644371 (0.0006) [2023-12-27 17:02:57,496][20381] Updated weights for policy 1, policy_version 1644381 (0.0007) [2023-12-27 17:02:57,539][20381] Updated weights for policy 1, policy_version 1644391 (0.0007) [2023-12-27 17:02:58,057][20347] Updated weights for policy 0, policy_version 1646031 (0.0006) [2023-12-27 17:02:58,104][20347] Updated weights for policy 0, policy_version 1646041 (0.0007) [2023-12-27 17:02:58,147][20347] Updated weights for policy 0, policy_version 1646051 (0.0007) [2023-12-27 17:02:58,167][20381] Updated weights for policy 1, policy_version 1644401 (0.0008) [2023-12-27 17:02:58,229][20381] Updated weights for policy 1, policy_version 1644411 (0.0010) [2023-12-27 17:02:58,283][20381] Updated weights for policy 1, policy_version 1644421 (0.0010) [2023-12-27 17:02:58,344][20381] Updated weights for policy 1, policy_version 1644431 (0.0008) [2023-12-27 17:02:58,932][20347] Updated weights for policy 0, policy_version 1646061 (0.0006) [2023-12-27 17:02:58,993][20347] Updated weights for policy 0, policy_version 1646071 (0.0006) [2023-12-27 17:02:59,048][20381] Updated weights for policy 1, policy_version 1644441 (0.0010) [2023-12-27 17:02:59,048][20347] Updated weights for policy 0, policy_version 1646081 (0.0009) [2023-12-27 17:02:59,099][20381] Updated weights for policy 1, policy_version 1644451 (0.0010) [2023-12-27 17:02:59,162][20381] Updated weights for policy 1, policy_version 1644461 (0.0010) [2023-12-27 17:02:59,699][20347] Updated weights for policy 0, policy_version 1646091 (0.0008) [2023-12-27 17:02:59,757][20347] Updated weights for policy 0, policy_version 1646101 (0.0010) [2023-12-27 17:02:59,807][20347] Updated weights for policy 0, policy_version 1646111 (0.0010) [2023-12-27 17:02:59,882][20381] Updated weights for policy 1, policy_version 1644471 (0.0008) [2023-12-27 17:02:59,945][20381] Updated weights for policy 1, policy_version 1644481 (0.0008) [2023-12-27 17:03:00,005][20381] Updated weights for policy 1, policy_version 1644491 (0.0008) [2023-12-27 17:03:00,489][20347] Updated weights for policy 0, policy_version 1646121 (0.0009) [2023-12-27 17:03:00,539][20347] Updated weights for policy 0, policy_version 1646131 (0.0006) [2023-12-27 17:03:00,582][20381] Updated weights for policy 1, policy_version 1644501 (0.0008) [2023-12-27 17:03:00,592][20347] Updated weights for policy 0, policy_version 1646141 (0.0005) [2023-12-27 17:03:00,637][20381] Updated weights for policy 1, policy_version 1644511 (0.0010) [2023-12-27 17:03:00,647][20347] Updated weights for policy 0, policy_version 1646151 (0.0005) [2023-12-27 17:03:00,690][20381] Updated weights for policy 1, policy_version 1644521 (0.0010) [2023-12-27 17:03:01,279][20347] Updated weights for policy 0, policy_version 1646161 (0.0008) [2023-12-27 17:03:01,323][20381] Updated weights for policy 1, policy_version 1644531 (0.0009) [2023-12-27 17:03:01,333][20347] Updated weights for policy 0, policy_version 1646171 (0.0007) [2023-12-27 17:03:01,380][20381] Updated weights for policy 1, policy_version 1644541 (0.0007) [2023-12-27 17:03:01,394][20347] Updated weights for policy 0, policy_version 1646181 (0.0006) [2023-12-27 17:03:01,445][20381] Updated weights for policy 1, policy_version 1644551 (0.0007) [2023-12-27 17:03:02,103][20381] Updated weights for policy 1, policy_version 1644561 (0.0008) [2023-12-27 17:03:02,129][20347] Updated weights for policy 0, policy_version 1646191 (0.0007) [2023-12-27 17:03:02,159][20381] Updated weights for policy 1, policy_version 1644571 (0.0005) [2023-12-27 17:03:02,182][20347] Updated weights for policy 0, policy_version 1646201 (0.0007) [2023-12-27 17:03:02,214][20381] Updated weights for policy 1, policy_version 1644581 (0.0006) [2023-12-27 17:03:02,242][20347] Updated weights for policy 0, policy_version 1646211 (0.0005) [2023-12-27 17:03:02,260][20381] Updated weights for policy 1, policy_version 1644591 (0.0007) [2023-12-27 17:03:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 842571776. Throughput: 0: 10381.2, 1: 10312.0. Samples: 842575488. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:02,363][19170] Avg episode reward: [(0, '960.990'), (1, '824.550')] [2023-12-27 17:03:02,856][20347] Updated weights for policy 0, policy_version 1646221 (0.0008) [2023-12-27 17:03:02,915][20347] Updated weights for policy 0, policy_version 1646231 (0.0007) [2023-12-27 17:03:02,965][20381] Updated weights for policy 1, policy_version 1644601 (0.0005) [2023-12-27 17:03:02,974][20347] Updated weights for policy 0, policy_version 1646241 (0.0007) [2023-12-27 17:03:03,014][20381] Updated weights for policy 1, policy_version 1644611 (0.0006) [2023-12-27 17:03:03,069][20381] Updated weights for policy 1, policy_version 1644621 (0.0007) [2023-12-27 17:03:03,602][20347] Updated weights for policy 0, policy_version 1646251 (0.0006) [2023-12-27 17:03:03,646][20347] Updated weights for policy 0, policy_version 1646261 (0.0007) [2023-12-27 17:03:03,692][20347] Updated weights for policy 0, policy_version 1646271 (0.0005) [2023-12-27 17:03:03,794][20381] Updated weights for policy 1, policy_version 1644631 (0.0008) [2023-12-27 17:03:03,841][20381] Updated weights for policy 1, policy_version 1644641 (0.0009) [2023-12-27 17:03:03,907][20381] Updated weights for policy 1, policy_version 1644651 (0.0008) [2023-12-27 17:03:04,322][20347] Updated weights for policy 0, policy_version 1646281 (0.0005) [2023-12-27 17:03:04,384][20347] Updated weights for policy 0, policy_version 1646291 (0.0008) [2023-12-27 17:03:04,441][20347] Updated weights for policy 0, policy_version 1646301 (0.0006) [2023-12-27 17:03:04,496][20347] Updated weights for policy 0, policy_version 1646311 (0.0007) [2023-12-27 17:03:04,675][20381] Updated weights for policy 1, policy_version 1644661 (0.0008) [2023-12-27 17:03:04,731][20381] Updated weights for policy 1, policy_version 1644671 (0.0008) [2023-12-27 17:03:04,790][20381] Updated weights for policy 1, policy_version 1644681 (0.0007) [2023-12-27 17:03:05,206][20347] Updated weights for policy 0, policy_version 1646321 (0.0009) [2023-12-27 17:03:05,264][20347] Updated weights for policy 0, policy_version 1646331 (0.0010) [2023-12-27 17:03:05,327][20347] Updated weights for policy 0, policy_version 1646341 (0.0009) [2023-12-27 17:03:05,411][20381] Updated weights for policy 1, policy_version 1644691 (0.0007) [2023-12-27 17:03:05,469][20381] Updated weights for policy 1, policy_version 1644701 (0.0005) [2023-12-27 17:03:05,521][20381] Updated weights for policy 1, policy_version 1644711 (0.0005) [2023-12-27 17:03:06,016][20347] Updated weights for policy 0, policy_version 1646351 (0.0009) [2023-12-27 17:03:06,061][20347] Updated weights for policy 0, policy_version 1646361 (0.0010) [2023-12-27 17:03:06,116][20347] Updated weights for policy 0, policy_version 1646371 (0.0009) [2023-12-27 17:03:06,181][20381] Updated weights for policy 1, policy_version 1644721 (0.0007) [2023-12-27 17:03:06,254][20381] Updated weights for policy 1, policy_version 1644731 (0.0007) [2023-12-27 17:03:06,323][20381] Updated weights for policy 1, policy_version 1644741 (0.0008) [2023-12-27 17:03:06,383][20381] Updated weights for policy 1, policy_version 1644751 (0.0009) [2023-12-27 17:03:06,808][20347] Updated weights for policy 0, policy_version 1646381 (0.0008) [2023-12-27 17:03:06,860][20347] Updated weights for policy 0, policy_version 1646391 (0.0008) [2023-12-27 17:03:06,919][20347] Updated weights for policy 0, policy_version 1646401 (0.0008) [2023-12-27 17:03:07,056][20381] Updated weights for policy 1, policy_version 1644761 (0.0006) [2023-12-27 17:03:07,103][20381] Updated weights for policy 1, policy_version 1644771 (0.0005) [2023-12-27 17:03:07,166][20381] Updated weights for policy 1, policy_version 1644781 (0.0005) [2023-12-27 17:03:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 842670080. Throughput: 0: 10439.0, 1: 10243.1. Samples: 842636572. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:07,363][19170] Avg episode reward: [(0, '960.950'), (1, '843.830')] [2023-12-27 17:03:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001644784_421109760.pth... [2023-12-27 17:03:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001646408_421560320.pth... [2023-12-27 17:03:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001645192_421249024.pth [2023-12-27 17:03:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001643568_420798464.pth [2023-12-27 17:03:07,641][20347] Updated weights for policy 0, policy_version 1646411 (0.0008) [2023-12-27 17:03:07,696][20347] Updated weights for policy 0, policy_version 1646421 (0.0007) [2023-12-27 17:03:07,750][20347] Updated weights for policy 0, policy_version 1646431 (0.0007) [2023-12-27 17:03:07,780][20381] Updated weights for policy 1, policy_version 1644791 (0.0008) [2023-12-27 17:03:07,793][20291] KL-divergence is very high: 142.1639 [2023-12-27 17:03:07,829][20381] Updated weights for policy 1, policy_version 1644801 (0.0011) [2023-12-27 17:03:07,890][20381] Updated weights for policy 1, policy_version 1644811 (0.0011) [2023-12-27 17:03:08,323][20347] Updated weights for policy 0, policy_version 1646441 (0.0006) [2023-12-27 17:03:08,385][20347] Updated weights for policy 0, policy_version 1646451 (0.0007) [2023-12-27 17:03:08,444][20347] Updated weights for policy 0, policy_version 1646461 (0.0008) [2023-12-27 17:03:08,502][20347] Updated weights for policy 0, policy_version 1646471 (0.0007) [2023-12-27 17:03:08,553][20381] Updated weights for policy 1, policy_version 1644821 (0.0007) [2023-12-27 17:03:08,600][20381] Updated weights for policy 1, policy_version 1644831 (0.0005) [2023-12-27 17:03:08,655][20381] Updated weights for policy 1, policy_version 1644841 (0.0005) [2023-12-27 17:03:09,201][20381] Updated weights for policy 1, policy_version 1644851 (0.0005) [2023-12-27 17:03:09,238][20347] Updated weights for policy 0, policy_version 1646481 (0.0008) [2023-12-27 17:03:09,263][20381] Updated weights for policy 1, policy_version 1644861 (0.0008) [2023-12-27 17:03:09,289][20347] Updated weights for policy 0, policy_version 1646491 (0.0007) [2023-12-27 17:03:09,311][20381] Updated weights for policy 1, policy_version 1644871 (0.0008) [2023-12-27 17:03:09,340][20347] Updated weights for policy 0, policy_version 1646501 (0.0007) [2023-12-27 17:03:10,073][20347] Updated weights for policy 0, policy_version 1646511 (0.0007) [2023-12-27 17:03:10,084][20381] Updated weights for policy 1, policy_version 1644881 (0.0009) [2023-12-27 17:03:10,135][20347] Updated weights for policy 0, policy_version 1646521 (0.0007) [2023-12-27 17:03:10,138][20381] Updated weights for policy 1, policy_version 1644891 (0.0008) [2023-12-27 17:03:10,190][20381] Updated weights for policy 1, policy_version 1644901 (0.0008) [2023-12-27 17:03:10,191][20347] Updated weights for policy 0, policy_version 1646531 (0.0008) [2023-12-27 17:03:10,248][20381] Updated weights for policy 1, policy_version 1644911 (0.0006) [2023-12-27 17:03:10,861][20347] Updated weights for policy 0, policy_version 1646541 (0.0008) [2023-12-27 17:03:10,906][20347] Updated weights for policy 0, policy_version 1646551 (0.0010) [2023-12-27 17:03:10,957][20347] Updated weights for policy 0, policy_version 1646561 (0.0011) [2023-12-27 17:03:10,981][20381] Updated weights for policy 1, policy_version 1644921 (0.0006) [2023-12-27 17:03:11,049][20381] Updated weights for policy 1, policy_version 1644931 (0.0007) [2023-12-27 17:03:11,097][20381] Updated weights for policy 1, policy_version 1644941 (0.0008) [2023-12-27 17:03:11,810][20347] Updated weights for policy 0, policy_version 1646571 (0.0009) [2023-12-27 17:03:11,871][20347] Updated weights for policy 0, policy_version 1646581 (0.0007) [2023-12-27 17:03:11,908][20381] Updated weights for policy 1, policy_version 1644951 (0.0008) [2023-12-27 17:03:11,929][20347] Updated weights for policy 0, policy_version 1646591 (0.0009) [2023-12-27 17:03:11,962][20381] Updated weights for policy 1, policy_version 1644961 (0.0007) [2023-12-27 17:03:12,021][20381] Updated weights for policy 1, policy_version 1644971 (0.0008) [2023-12-27 17:03:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 842768384. Throughput: 0: 10409.1, 1: 10330.5. Samples: 842759236. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:12,363][19170] Avg episode reward: [(0, '970.760'), (1, '863.370')] [2023-12-27 17:03:12,654][20347] Updated weights for policy 0, policy_version 1646601 (0.0007) [2023-12-27 17:03:12,716][20347] Updated weights for policy 0, policy_version 1646611 (0.0010) [2023-12-27 17:03:12,756][20381] Updated weights for policy 1, policy_version 1644981 (0.0007) [2023-12-27 17:03:12,779][20347] Updated weights for policy 0, policy_version 1646621 (0.0011) [2023-12-27 17:03:12,811][20381] Updated weights for policy 1, policy_version 1644991 (0.0005) [2023-12-27 17:03:12,831][20347] Updated weights for policy 0, policy_version 1646631 (0.0010) [2023-12-27 17:03:12,872][20381] Updated weights for policy 1, policy_version 1645001 (0.0006) [2023-12-27 17:03:13,397][20347] Updated weights for policy 0, policy_version 1646641 (0.0006) [2023-12-27 17:03:13,456][20347] Updated weights for policy 0, policy_version 1646651 (0.0005) [2023-12-27 17:03:13,515][20347] Updated weights for policy 0, policy_version 1646661 (0.0008) [2023-12-27 17:03:13,617][20381] Updated weights for policy 1, policy_version 1645011 (0.0008) [2023-12-27 17:03:13,677][20381] Updated weights for policy 1, policy_version 1645021 (0.0008) [2023-12-27 17:03:13,737][20381] Updated weights for policy 1, policy_version 1645031 (0.0007) [2023-12-27 17:03:14,116][20347] Updated weights for policy 0, policy_version 1646671 (0.0010) [2023-12-27 17:03:14,170][20347] Updated weights for policy 0, policy_version 1646681 (0.0010) [2023-12-27 17:03:14,221][20347] Updated weights for policy 0, policy_version 1646691 (0.0010) [2023-12-27 17:03:14,393][20381] Updated weights for policy 1, policy_version 1645041 (0.0007) [2023-12-27 17:03:14,450][20381] Updated weights for policy 1, policy_version 1645051 (0.0007) [2023-12-27 17:03:14,501][20381] Updated weights for policy 1, policy_version 1645061 (0.0007) [2023-12-27 17:03:14,559][20381] Updated weights for policy 1, policy_version 1645071 (0.0007) [2023-12-27 17:03:14,918][20347] Updated weights for policy 0, policy_version 1646701 (0.0010) [2023-12-27 17:03:14,978][20347] Updated weights for policy 0, policy_version 1646711 (0.0011) [2023-12-27 17:03:15,040][20347] Updated weights for policy 0, policy_version 1646721 (0.0011) [2023-12-27 17:03:15,307][20381] Updated weights for policy 1, policy_version 1645081 (0.0008) [2023-12-27 17:03:15,368][20381] Updated weights for policy 1, policy_version 1645091 (0.0007) [2023-12-27 17:03:15,427][20381] Updated weights for policy 1, policy_version 1645101 (0.0008) [2023-12-27 17:03:15,730][20347] Updated weights for policy 0, policy_version 1646731 (0.0010) [2023-12-27 17:03:15,787][20347] Updated weights for policy 0, policy_version 1646741 (0.0010) [2023-12-27 17:03:15,835][20347] Updated weights for policy 0, policy_version 1646751 (0.0010) [2023-12-27 17:03:16,094][20381] Updated weights for policy 1, policy_version 1645111 (0.0006) [2023-12-27 17:03:16,156][20381] Updated weights for policy 1, policy_version 1645121 (0.0007) [2023-12-27 17:03:16,223][20381] Updated weights for policy 1, policy_version 1645131 (0.0007) [2023-12-27 17:03:16,416][20347] Updated weights for policy 0, policy_version 1646761 (0.0009) [2023-12-27 17:03:16,469][20347] Updated weights for policy 0, policy_version 1646771 (0.0009) [2023-12-27 17:03:16,520][20347] Updated weights for policy 0, policy_version 1646781 (0.0010) [2023-12-27 17:03:16,571][20347] Updated weights for policy 0, policy_version 1646791 (0.0010) [2023-12-27 17:03:16,870][20381] Updated weights for policy 1, policy_version 1645141 (0.0006) [2023-12-27 17:03:16,927][20381] Updated weights for policy 1, policy_version 1645151 (0.0005) [2023-12-27 17:03:16,984][20381] Updated weights for policy 1, policy_version 1645161 (0.0008) [2023-12-27 17:03:17,226][20347] Updated weights for policy 0, policy_version 1646801 (0.0008) [2023-12-27 17:03:17,292][20347] Updated weights for policy 0, policy_version 1646811 (0.0007) [2023-12-27 17:03:17,342][20347] Updated weights for policy 0, policy_version 1646821 (0.0007) [2023-12-27 17:03:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 842874880. Throughput: 0: 10355.5, 1: 10319.1. Samples: 842881176. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:17,363][19170] Avg episode reward: [(0, '970.690'), (1, '824.480')] [2023-12-27 17:03:17,585][20381] Updated weights for policy 1, policy_version 1645171 (0.0007) [2023-12-27 17:03:17,649][20381] Updated weights for policy 1, policy_version 1645181 (0.0005) [2023-12-27 17:03:17,702][20381] Updated weights for policy 1, policy_version 1645191 (0.0006) [2023-12-27 17:03:17,949][20347] Updated weights for policy 0, policy_version 1646831 (0.0006) [2023-12-27 17:03:17,998][20347] Updated weights for policy 0, policy_version 1646841 (0.0006) [2023-12-27 17:03:18,057][20347] Updated weights for policy 0, policy_version 1646851 (0.0008) [2023-12-27 17:03:18,288][20381] Updated weights for policy 1, policy_version 1645201 (0.0010) [2023-12-27 17:03:18,345][20381] Updated weights for policy 1, policy_version 1645211 (0.0008) [2023-12-27 17:03:18,405][20381] Updated weights for policy 1, policy_version 1645221 (0.0008) [2023-12-27 17:03:18,421][20292] KL-divergence is very high: 188.1746 [2023-12-27 17:03:18,461][20381] Updated weights for policy 1, policy_version 1645231 (0.0007) [2023-12-27 17:03:18,690][20347] Updated weights for policy 0, policy_version 1646861 (0.0007) [2023-12-27 17:03:18,747][20347] Updated weights for policy 0, policy_version 1646871 (0.0008) [2023-12-27 17:03:18,810][20347] Updated weights for policy 0, policy_version 1646881 (0.0008) [2023-12-27 17:03:19,176][20381] Updated weights for policy 1, policy_version 1645241 (0.0008) [2023-12-27 17:03:19,242][20381] Updated weights for policy 1, policy_version 1645251 (0.0008) [2023-12-27 17:03:19,303][20381] Updated weights for policy 1, policy_version 1645261 (0.0008) [2023-12-27 17:03:19,476][20347] Updated weights for policy 0, policy_version 1646891 (0.0008) [2023-12-27 17:03:19,541][20347] Updated weights for policy 0, policy_version 1646901 (0.0009) [2023-12-27 17:03:19,601][20347] Updated weights for policy 0, policy_version 1646911 (0.0007) [2023-12-27 17:03:20,014][20381] Updated weights for policy 1, policy_version 1645271 (0.0008) [2023-12-27 17:03:20,077][20381] Updated weights for policy 1, policy_version 1645281 (0.0008) [2023-12-27 17:03:20,137][20381] Updated weights for policy 1, policy_version 1645291 (0.0008) [2023-12-27 17:03:20,316][20347] Updated weights for policy 0, policy_version 1646921 (0.0008) [2023-12-27 17:03:20,373][20347] Updated weights for policy 0, policy_version 1646931 (0.0008) [2023-12-27 17:03:20,428][20347] Updated weights for policy 0, policy_version 1646941 (0.0008) [2023-12-27 17:03:20,491][20347] Updated weights for policy 0, policy_version 1646951 (0.0008) [2023-12-27 17:03:20,840][20381] Updated weights for policy 1, policy_version 1645301 (0.0008) [2023-12-27 17:03:20,899][20381] Updated weights for policy 1, policy_version 1645311 (0.0008) [2023-12-27 17:03:20,960][20381] Updated weights for policy 1, policy_version 1645321 (0.0008) [2023-12-27 17:03:21,241][20347] Updated weights for policy 0, policy_version 1646961 (0.0008) [2023-12-27 17:03:21,304][20347] Updated weights for policy 0, policy_version 1646971 (0.0007) [2023-12-27 17:03:21,378][20347] Updated weights for policy 0, policy_version 1646981 (0.0009) [2023-12-27 17:03:21,713][20381] Updated weights for policy 1, policy_version 1645331 (0.0007) [2023-12-27 17:03:21,783][20381] Updated weights for policy 1, policy_version 1645341 (0.0008) [2023-12-27 17:03:21,837][20381] Updated weights for policy 1, policy_version 1645351 (0.0007) [2023-12-27 17:03:22,087][20347] Updated weights for policy 0, policy_version 1646991 (0.0007) [2023-12-27 17:03:22,140][20347] Updated weights for policy 0, policy_version 1647001 (0.0008) [2023-12-27 17:03:22,191][20347] Updated weights for policy 0, policy_version 1647011 (0.0008) [2023-12-27 17:03:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 842973184. Throughput: 0: 10364.6, 1: 10276.9. Samples: 842942844. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:22,363][19170] Avg episode reward: [(0, '970.670'), (1, '814.750')] [2023-12-27 17:03:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001645360_421257216.pth... [2023-12-27 17:03:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001647016_421715968.pth... [2023-12-27 17:03:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001644176_420954112.pth [2023-12-27 17:03:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001645800_421404672.pth [2023-12-27 17:03:22,556][20381] Updated weights for policy 1, policy_version 1645361 (0.0008) [2023-12-27 17:03:22,610][20381] Updated weights for policy 1, policy_version 1645371 (0.0007) [2023-12-27 17:03:22,668][20381] Updated weights for policy 1, policy_version 1645381 (0.0006) [2023-12-27 17:03:22,728][20381] Updated weights for policy 1, policy_version 1645391 (0.0007) [2023-12-27 17:03:22,979][20347] Updated weights for policy 0, policy_version 1647021 (0.0007) [2023-12-27 17:03:23,024][20347] Updated weights for policy 0, policy_version 1647031 (0.0006) [2023-12-27 17:03:23,070][20347] Updated weights for policy 0, policy_version 1647041 (0.0007) [2023-12-27 17:03:23,401][20381] Updated weights for policy 1, policy_version 1645401 (0.0007) [2023-12-27 17:03:23,462][20381] Updated weights for policy 1, policy_version 1645411 (0.0005) [2023-12-27 17:03:23,518][20381] Updated weights for policy 1, policy_version 1645421 (0.0006) [2023-12-27 17:03:23,730][20347] Updated weights for policy 0, policy_version 1647051 (0.0007) [2023-12-27 17:03:23,785][20347] Updated weights for policy 0, policy_version 1647061 (0.0005) [2023-12-27 17:03:23,841][20347] Updated weights for policy 0, policy_version 1647071 (0.0007) [2023-12-27 17:03:24,054][20381] Updated weights for policy 1, policy_version 1645431 (0.0005) [2023-12-27 17:03:24,105][20381] Updated weights for policy 1, policy_version 1645441 (0.0005) [2023-12-27 17:03:24,154][20381] Updated weights for policy 1, policy_version 1645451 (0.0005) [2023-12-27 17:03:24,565][20347] Updated weights for policy 0, policy_version 1647081 (0.0008) [2023-12-27 17:03:24,616][20347] Updated weights for policy 0, policy_version 1647091 (0.0008) [2023-12-27 17:03:24,668][20347] Updated weights for policy 0, policy_version 1647101 (0.0009) [2023-12-27 17:03:24,714][20347] Updated weights for policy 0, policy_version 1647111 (0.0008) [2023-12-27 17:03:24,725][20381] Updated weights for policy 1, policy_version 1645461 (0.0005) [2023-12-27 17:03:24,781][20381] Updated weights for policy 1, policy_version 1645471 (0.0005) [2023-12-27 17:03:24,833][20381] Updated weights for policy 1, policy_version 1645481 (0.0005) [2023-12-27 17:03:25,341][20347] Updated weights for policy 0, policy_version 1647121 (0.0008) [2023-12-27 17:03:25,363][20381] Updated weights for policy 1, policy_version 1645491 (0.0005) [2023-12-27 17:03:25,396][20347] Updated weights for policy 0, policy_version 1647131 (0.0007) [2023-12-27 17:03:25,416][20381] Updated weights for policy 1, policy_version 1645501 (0.0005) [2023-12-27 17:03:25,445][20347] Updated weights for policy 0, policy_version 1647141 (0.0007) [2023-12-27 17:03:25,472][20381] Updated weights for policy 1, policy_version 1645511 (0.0005) [2023-12-27 17:03:26,010][20381] Updated weights for policy 1, policy_version 1645521 (0.0005) [2023-12-27 17:03:26,066][20381] Updated weights for policy 1, policy_version 1645531 (0.0005) [2023-12-27 17:03:26,118][20381] Updated weights for policy 1, policy_version 1645541 (0.0005) [2023-12-27 17:03:26,169][20381] Updated weights for policy 1, policy_version 1645551 (0.0005) [2023-12-27 17:03:26,220][20347] Updated weights for policy 0, policy_version 1647151 (0.0007) [2023-12-27 17:03:26,269][20347] Updated weights for policy 0, policy_version 1647161 (0.0008) [2023-12-27 17:03:26,323][20347] Updated weights for policy 0, policy_version 1647171 (0.0007) [2023-12-27 17:03:26,750][20381] Updated weights for policy 1, policy_version 1645561 (0.0005) [2023-12-27 17:03:26,806][20381] Updated weights for policy 1, policy_version 1645571 (0.0005) [2023-12-27 17:03:26,864][20381] Updated weights for policy 1, policy_version 1645581 (0.0007) [2023-12-27 17:03:27,078][20347] Updated weights for policy 0, policy_version 1647181 (0.0007) [2023-12-27 17:03:27,125][20347] Updated weights for policy 0, policy_version 1647191 (0.0007) [2023-12-27 17:03:27,172][20347] Updated weights for policy 0, policy_version 1647201 (0.0007) [2023-12-27 17:03:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 843079680. Throughput: 0: 10311.6, 1: 10382.2. Samples: 843067696. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:27,363][19170] Avg episode reward: [(0, '970.800'), (1, '844.070')] [2023-12-27 17:03:27,493][20381] Updated weights for policy 1, policy_version 1645591 (0.0007) [2023-12-27 17:03:27,536][20381] Updated weights for policy 1, policy_version 1645601 (0.0005) [2023-12-27 17:03:27,579][20381] Updated weights for policy 1, policy_version 1645611 (0.0005) [2023-12-27 17:03:27,799][20347] Updated weights for policy 0, policy_version 1647211 (0.0007) [2023-12-27 17:03:27,852][20347] Updated weights for policy 0, policy_version 1647221 (0.0006) [2023-12-27 17:03:27,896][20347] Updated weights for policy 0, policy_version 1647231 (0.0008) [2023-12-27 17:03:28,137][20381] Updated weights for policy 1, policy_version 1645621 (0.0008) [2023-12-27 17:03:28,188][20381] Updated weights for policy 1, policy_version 1645631 (0.0010) [2023-12-27 17:03:28,240][20381] Updated weights for policy 1, policy_version 1645641 (0.0009) [2023-12-27 17:03:28,517][20347] Updated weights for policy 0, policy_version 1647241 (0.0008) [2023-12-27 17:03:28,574][20347] Updated weights for policy 0, policy_version 1647251 (0.0008) [2023-12-27 17:03:28,629][20347] Updated weights for policy 0, policy_version 1647261 (0.0007) [2023-12-27 17:03:28,686][20347] Updated weights for policy 0, policy_version 1647271 (0.0007) [2023-12-27 17:03:28,911][20381] Updated weights for policy 1, policy_version 1645651 (0.0007) [2023-12-27 17:03:28,956][20381] Updated weights for policy 1, policy_version 1645661 (0.0005) [2023-12-27 17:03:29,014][20381] Updated weights for policy 1, policy_version 1645671 (0.0007) [2023-12-27 17:03:29,367][20347] Updated weights for policy 0, policy_version 1647281 (0.0008) [2023-12-27 17:03:29,437][20347] Updated weights for policy 0, policy_version 1647291 (0.0009) [2023-12-27 17:03:29,497][20347] Updated weights for policy 0, policy_version 1647301 (0.0008) [2023-12-27 17:03:29,644][20381] Updated weights for policy 1, policy_version 1645681 (0.0010) [2023-12-27 17:03:29,692][20381] Updated weights for policy 1, policy_version 1645691 (0.0007) [2023-12-27 17:03:29,742][20292] KL-divergence is very high: 121.2059 [2023-12-27 17:03:29,742][20381] Updated weights for policy 1, policy_version 1645701 (0.0007) [2023-12-27 17:03:29,793][20381] Updated weights for policy 1, policy_version 1645711 (0.0005) [2023-12-27 17:03:30,197][20347] Updated weights for policy 0, policy_version 1647311 (0.0006) [2023-12-27 17:03:30,254][20347] Updated weights for policy 0, policy_version 1647321 (0.0005) [2023-12-27 17:03:30,313][20347] Updated weights for policy 0, policy_version 1647331 (0.0006) [2023-12-27 17:03:30,510][20381] Updated weights for policy 1, policy_version 1645721 (0.0005) [2023-12-27 17:03:30,558][20381] Updated weights for policy 1, policy_version 1645731 (0.0006) [2023-12-27 17:03:30,606][20381] Updated weights for policy 1, policy_version 1645741 (0.0005) [2023-12-27 17:03:30,916][20347] Updated weights for policy 0, policy_version 1647341 (0.0005) [2023-12-27 17:03:30,960][20347] Updated weights for policy 0, policy_version 1647351 (0.0005) [2023-12-27 17:03:31,007][20347] Updated weights for policy 0, policy_version 1647361 (0.0005) [2023-12-27 17:03:31,222][20381] Updated weights for policy 1, policy_version 1645751 (0.0007) [2023-12-27 17:03:31,289][20381] Updated weights for policy 1, policy_version 1645761 (0.0008) [2023-12-27 17:03:31,353][20381] Updated weights for policy 1, policy_version 1645771 (0.0007) [2023-12-27 17:03:31,678][20347] Updated weights for policy 0, policy_version 1647371 (0.0006) [2023-12-27 17:03:31,739][20347] Updated weights for policy 0, policy_version 1647381 (0.0008) [2023-12-27 17:03:31,799][20347] Updated weights for policy 0, policy_version 1647391 (0.0008) [2023-12-27 17:03:32,001][20381] Updated weights for policy 1, policy_version 1645781 (0.0007) [2023-12-27 17:03:32,064][20381] Updated weights for policy 1, policy_version 1645791 (0.0005) [2023-12-27 17:03:32,122][20381] Updated weights for policy 1, policy_version 1645801 (0.0008) [2023-12-27 17:03:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 843186176. Throughput: 0: 10283.0, 1: 10450.6. Samples: 843195488. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:32,363][19170] Avg episode reward: [(0, '970.700'), (1, '892.560')] [2023-12-27 17:03:32,453][20347] Updated weights for policy 0, policy_version 1647401 (0.0008) [2023-12-27 17:03:32,516][20347] Updated weights for policy 0, policy_version 1647411 (0.0008) [2023-12-27 17:03:32,578][20347] Updated weights for policy 0, policy_version 1647421 (0.0007) [2023-12-27 17:03:32,633][20347] Updated weights for policy 0, policy_version 1647431 (0.0008) [2023-12-27 17:03:32,764][20381] Updated weights for policy 1, policy_version 1645811 (0.0007) [2023-12-27 17:03:32,824][20381] Updated weights for policy 1, policy_version 1645821 (0.0009) [2023-12-27 17:03:32,881][20381] Updated weights for policy 1, policy_version 1645831 (0.0008) [2023-12-27 17:03:33,281][20347] Updated weights for policy 0, policy_version 1647441 (0.0007) [2023-12-27 17:03:33,342][20347] Updated weights for policy 0, policy_version 1647451 (0.0009) [2023-12-27 17:03:33,395][20347] Updated weights for policy 0, policy_version 1647461 (0.0007) [2023-12-27 17:03:33,414][20381] Updated weights for policy 1, policy_version 1645841 (0.0007) [2023-12-27 17:03:33,476][20381] Updated weights for policy 1, policy_version 1645851 (0.0008) [2023-12-27 17:03:33,535][20381] Updated weights for policy 1, policy_version 1645861 (0.0008) [2023-12-27 17:03:33,595][20381] Updated weights for policy 1, policy_version 1645871 (0.0005) [2023-12-27 17:03:34,016][20347] Updated weights for policy 0, policy_version 1647471 (0.0007) [2023-12-27 17:03:34,076][20347] Updated weights for policy 0, policy_version 1647481 (0.0006) [2023-12-27 17:03:34,129][20347] Updated weights for policy 0, policy_version 1647491 (0.0006) [2023-12-27 17:03:34,263][20381] Updated weights for policy 1, policy_version 1645881 (0.0008) [2023-12-27 17:03:34,326][20381] Updated weights for policy 1, policy_version 1645891 (0.0008) [2023-12-27 17:03:34,397][20381] Updated weights for policy 1, policy_version 1645901 (0.0008) [2023-12-27 17:03:34,828][20347] Updated weights for policy 0, policy_version 1647501 (0.0006) [2023-12-27 17:03:34,876][20347] Updated weights for policy 0, policy_version 1647511 (0.0005) [2023-12-27 17:03:34,935][20347] Updated weights for policy 0, policy_version 1647521 (0.0006) [2023-12-27 17:03:35,080][20381] Updated weights for policy 1, policy_version 1645911 (0.0006) [2023-12-27 17:03:35,128][20381] Updated weights for policy 1, policy_version 1645921 (0.0007) [2023-12-27 17:03:35,175][20381] Updated weights for policy 1, policy_version 1645931 (0.0008) [2023-12-27 17:03:35,570][20347] Updated weights for policy 0, policy_version 1647531 (0.0007) [2023-12-27 17:03:35,634][20347] Updated weights for policy 0, policy_version 1647541 (0.0006) [2023-12-27 17:03:35,692][20347] Updated weights for policy 0, policy_version 1647551 (0.0006) [2023-12-27 17:03:35,885][20381] Updated weights for policy 1, policy_version 1645941 (0.0007) [2023-12-27 17:03:35,944][20381] Updated weights for policy 1, policy_version 1645951 (0.0008) [2023-12-27 17:03:36,001][20381] Updated weights for policy 1, policy_version 1645961 (0.0008) [2023-12-27 17:03:36,282][20347] Updated weights for policy 0, policy_version 1647561 (0.0007) [2023-12-27 17:03:36,341][20347] Updated weights for policy 0, policy_version 1647571 (0.0008) [2023-12-27 17:03:36,401][20347] Updated weights for policy 0, policy_version 1647581 (0.0007) [2023-12-27 17:03:36,458][20347] Updated weights for policy 0, policy_version 1647591 (0.0008) [2023-12-27 17:03:36,695][20381] Updated weights for policy 1, policy_version 1645971 (0.0007) [2023-12-27 17:03:36,753][20381] Updated weights for policy 1, policy_version 1645981 (0.0009) [2023-12-27 17:03:36,815][20381] Updated weights for policy 1, policy_version 1645991 (0.0011) [2023-12-27 17:03:37,122][20347] Updated weights for policy 0, policy_version 1647601 (0.0008) [2023-12-27 17:03:37,183][20347] Updated weights for policy 0, policy_version 1647611 (0.0007) [2023-12-27 17:03:37,236][20347] Updated weights for policy 0, policy_version 1647621 (0.0008) [2023-12-27 17:03:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 843292672. Throughput: 0: 10323.7, 1: 10480.5. Samples: 843259788. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:37,363][19170] Avg episode reward: [(0, '980.490'), (1, '883.050')] [2023-12-27 17:03:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001646000_421421056.pth... [2023-12-27 17:03:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001647624_421871616.pth... [2023-12-27 17:03:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001644784_421109760.pth [2023-12-27 17:03:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001646408_421560320.pth [2023-12-27 17:03:37,516][20381] Updated weights for policy 1, policy_version 1646001 (0.0009) [2023-12-27 17:03:37,569][20381] Updated weights for policy 1, policy_version 1646011 (0.0008) [2023-12-27 17:03:37,622][20381] Updated weights for policy 1, policy_version 1646021 (0.0008) [2023-12-27 17:03:37,677][20381] Updated weights for policy 1, policy_version 1646031 (0.0007) [2023-12-27 17:03:37,919][20347] Updated weights for policy 0, policy_version 1647631 (0.0006) [2023-12-27 17:03:37,970][20347] Updated weights for policy 0, policy_version 1647641 (0.0005) [2023-12-27 17:03:38,031][20347] Updated weights for policy 0, policy_version 1647651 (0.0005) [2023-12-27 17:03:38,420][20381] Updated weights for policy 1, policy_version 1646041 (0.0008) [2023-12-27 17:03:38,472][20381] Updated weights for policy 1, policy_version 1646051 (0.0008) [2023-12-27 17:03:38,526][20381] Updated weights for policy 1, policy_version 1646061 (0.0008) [2023-12-27 17:03:38,641][20347] Updated weights for policy 0, policy_version 1647661 (0.0008) [2023-12-27 17:03:38,701][20347] Updated weights for policy 0, policy_version 1647671 (0.0011) [2023-12-27 17:03:38,759][20347] Updated weights for policy 0, policy_version 1647681 (0.0010) [2023-12-27 17:03:39,213][20381] Updated weights for policy 1, policy_version 1646071 (0.0008) [2023-12-27 17:03:39,272][20381] Updated weights for policy 1, policy_version 1646081 (0.0008) [2023-12-27 17:03:39,325][20381] Updated weights for policy 1, policy_version 1646091 (0.0008) [2023-12-27 17:03:39,484][20347] Updated weights for policy 0, policy_version 1647691 (0.0009) [2023-12-27 17:03:39,552][20347] Updated weights for policy 0, policy_version 1647701 (0.0006) [2023-12-27 17:03:39,618][20347] Updated weights for policy 0, policy_version 1647711 (0.0006) [2023-12-27 17:03:40,059][20381] Updated weights for policy 1, policy_version 1646101 (0.0008) [2023-12-27 17:03:40,124][20381] Updated weights for policy 1, policy_version 1646111 (0.0007) [2023-12-27 17:03:40,185][20381] Updated weights for policy 1, policy_version 1646121 (0.0008) [2023-12-27 17:03:40,371][20347] Updated weights for policy 0, policy_version 1647721 (0.0007) [2023-12-27 17:03:40,421][20347] Updated weights for policy 0, policy_version 1647731 (0.0007) [2023-12-27 17:03:40,474][20347] Updated weights for policy 0, policy_version 1647741 (0.0007) [2023-12-27 17:03:40,533][20347] Updated weights for policy 0, policy_version 1647751 (0.0007) [2023-12-27 17:03:40,847][20381] Updated weights for policy 1, policy_version 1646131 (0.0008) [2023-12-27 17:03:40,908][20381] Updated weights for policy 1, policy_version 1646141 (0.0010) [2023-12-27 17:03:40,965][20381] Updated weights for policy 1, policy_version 1646151 (0.0010) [2023-12-27 17:03:41,311][20347] Updated weights for policy 0, policy_version 1647761 (0.0007) [2023-12-27 17:03:41,377][20347] Updated weights for policy 0, policy_version 1647771 (0.0008) [2023-12-27 17:03:41,442][20347] Updated weights for policy 0, policy_version 1647781 (0.0008) [2023-12-27 17:03:41,703][20381] Updated weights for policy 1, policy_version 1646161 (0.0010) [2023-12-27 17:03:41,766][20381] Updated weights for policy 1, policy_version 1646171 (0.0011) [2023-12-27 17:03:41,829][20381] Updated weights for policy 1, policy_version 1646181 (0.0011) [2023-12-27 17:03:41,890][20381] Updated weights for policy 1, policy_version 1646191 (0.0010) [2023-12-27 17:03:42,258][20347] Updated weights for policy 0, policy_version 1647791 (0.0008) [2023-12-27 17:03:42,320][20347] Updated weights for policy 0, policy_version 1647801 (0.0007) [2023-12-27 17:03:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 843382784. Throughput: 0: 10258.3, 1: 10417.3. Samples: 843380196. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:42,363][19170] Avg episode reward: [(0, '970.780'), (1, '853.870')] [2023-12-27 17:03:42,388][20347] Updated weights for policy 0, policy_version 1647811 (0.0007) [2023-12-27 17:03:42,559][20381] Updated weights for policy 1, policy_version 1646201 (0.0006) [2023-12-27 17:03:42,610][20381] Updated weights for policy 1, policy_version 1646211 (0.0005) [2023-12-27 17:03:42,663][20381] Updated weights for policy 1, policy_version 1646221 (0.0008) [2023-12-27 17:03:43,107][20347] Updated weights for policy 0, policy_version 1647821 (0.0007) [2023-12-27 17:03:43,168][20347] Updated weights for policy 0, policy_version 1647831 (0.0007) [2023-12-27 17:03:43,227][20347] Updated weights for policy 0, policy_version 1647841 (0.0008) [2023-12-27 17:03:43,337][20381] Updated weights for policy 1, policy_version 1646231 (0.0010) [2023-12-27 17:03:43,390][20381] Updated weights for policy 1, policy_version 1646241 (0.0010) [2023-12-27 17:03:43,445][20381] Updated weights for policy 1, policy_version 1646251 (0.0010) [2023-12-27 17:03:43,910][20347] Updated weights for policy 0, policy_version 1647851 (0.0007) [2023-12-27 17:03:43,969][20347] Updated weights for policy 0, policy_version 1647861 (0.0008) [2023-12-27 17:03:44,022][20347] Updated weights for policy 0, policy_version 1647871 (0.0008) [2023-12-27 17:03:44,156][20381] Updated weights for policy 1, policy_version 1646261 (0.0009) [2023-12-27 17:03:44,213][20381] Updated weights for policy 1, policy_version 1646271 (0.0008) [2023-12-27 17:03:44,265][20381] Updated weights for policy 1, policy_version 1646281 (0.0007) [2023-12-27 17:03:44,714][20347] Updated weights for policy 0, policy_version 1647881 (0.0008) [2023-12-27 17:03:44,768][20347] Updated weights for policy 0, policy_version 1647891 (0.0008) [2023-12-27 17:03:44,830][20347] Updated weights for policy 0, policy_version 1647901 (0.0009) [2023-12-27 17:03:44,889][20347] Updated weights for policy 0, policy_version 1647911 (0.0009) [2023-12-27 17:03:44,951][20381] Updated weights for policy 1, policy_version 1646291 (0.0008) [2023-12-27 17:03:45,012][20381] Updated weights for policy 1, policy_version 1646301 (0.0007) [2023-12-27 17:03:45,073][20381] Updated weights for policy 1, policy_version 1646311 (0.0008) [2023-12-27 17:03:45,572][20347] Updated weights for policy 0, policy_version 1647921 (0.0005) [2023-12-27 17:03:45,618][20347] Updated weights for policy 0, policy_version 1647931 (0.0005) [2023-12-27 17:03:45,673][20347] Updated weights for policy 0, policy_version 1647941 (0.0005) [2023-12-27 17:03:45,774][20381] Updated weights for policy 1, policy_version 1646321 (0.0008) [2023-12-27 17:03:45,830][20381] Updated weights for policy 1, policy_version 1646331 (0.0009) [2023-12-27 17:03:45,884][20381] Updated weights for policy 1, policy_version 1646341 (0.0007) [2023-12-27 17:03:45,937][20381] Updated weights for policy 1, policy_version 1646351 (0.0007) [2023-12-27 17:03:46,229][20347] Updated weights for policy 0, policy_version 1647951 (0.0007) [2023-12-27 17:03:46,285][20347] Updated weights for policy 0, policy_version 1647961 (0.0008) [2023-12-27 17:03:46,343][20347] Updated weights for policy 0, policy_version 1647971 (0.0005) [2023-12-27 17:03:46,611][20381] Updated weights for policy 1, policy_version 1646361 (0.0007) [2023-12-27 17:03:46,660][20381] Updated weights for policy 1, policy_version 1646371 (0.0005) [2023-12-27 17:03:46,722][20381] Updated weights for policy 1, policy_version 1646381 (0.0005) [2023-12-27 17:03:46,974][20347] Updated weights for policy 0, policy_version 1647981 (0.0008) [2023-12-27 17:03:47,036][20347] Updated weights for policy 0, policy_version 1647991 (0.0010) [2023-12-27 17:03:47,091][20347] Updated weights for policy 0, policy_version 1648001 (0.0010) [2023-12-27 17:03:47,315][20381] Updated weights for policy 1, policy_version 1646391 (0.0007) [2023-12-27 17:03:47,358][20381] Updated weights for policy 1, policy_version 1646401 (0.0006) [2023-12-27 17:03:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 843489280. Throughput: 0: 10245.3, 1: 10360.0. Samples: 843502728. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:47,363][19170] Avg episode reward: [(0, '960.970'), (1, '805.310')] [2023-12-27 17:03:47,403][20381] Updated weights for policy 1, policy_version 1646411 (0.0005) [2023-12-27 17:03:47,746][20347] Updated weights for policy 0, policy_version 1648011 (0.0007) [2023-12-27 17:03:47,802][20347] Updated weights for policy 0, policy_version 1648021 (0.0008) [2023-12-27 17:03:47,864][20347] Updated weights for policy 0, policy_version 1648031 (0.0010) [2023-12-27 17:03:47,992][20381] Updated weights for policy 1, policy_version 1646421 (0.0005) [2023-12-27 17:03:48,051][20381] Updated weights for policy 1, policy_version 1646431 (0.0006) [2023-12-27 17:03:48,103][20381] Updated weights for policy 1, policy_version 1646441 (0.0005) [2023-12-27 17:03:48,544][20347] Updated weights for policy 0, policy_version 1648041 (0.0010) [2023-12-27 17:03:48,599][20347] Updated weights for policy 0, policy_version 1648051 (0.0008) [2023-12-27 17:03:48,647][20347] Updated weights for policy 0, policy_version 1648061 (0.0008) [2023-12-27 17:03:48,692][20381] Updated weights for policy 1, policy_version 1646451 (0.0005) [2023-12-27 17:03:48,702][20347] Updated weights for policy 0, policy_version 1648071 (0.0008) [2023-12-27 17:03:48,750][20381] Updated weights for policy 1, policy_version 1646461 (0.0007) [2023-12-27 17:03:48,812][20381] Updated weights for policy 1, policy_version 1646471 (0.0007) [2023-12-27 17:03:49,419][20347] Updated weights for policy 0, policy_version 1648081 (0.0010) [2023-12-27 17:03:49,477][20347] Updated weights for policy 0, policy_version 1648091 (0.0010) [2023-12-27 17:03:49,536][20347] Updated weights for policy 0, policy_version 1648101 (0.0010) [2023-12-27 17:03:49,553][20381] Updated weights for policy 1, policy_version 1646481 (0.0007) [2023-12-27 17:03:49,602][20381] Updated weights for policy 1, policy_version 1646491 (0.0007) [2023-12-27 17:03:49,664][20381] Updated weights for policy 1, policy_version 1646501 (0.0007) [2023-12-27 17:03:49,726][20381] Updated weights for policy 1, policy_version 1646511 (0.0009) [2023-12-27 17:03:50,223][20347] Updated weights for policy 0, policy_version 1648111 (0.0011) [2023-12-27 17:03:50,287][20347] Updated weights for policy 0, policy_version 1648121 (0.0011) [2023-12-27 17:03:50,340][20347] Updated weights for policy 0, policy_version 1648131 (0.0010) [2023-12-27 17:03:50,447][20381] Updated weights for policy 1, policy_version 1646521 (0.0011) [2023-12-27 17:03:50,508][20381] Updated weights for policy 1, policy_version 1646531 (0.0010) [2023-12-27 17:03:50,567][20381] Updated weights for policy 1, policy_version 1646541 (0.0010) [2023-12-27 17:03:51,029][20347] Updated weights for policy 0, policy_version 1648141 (0.0011) [2023-12-27 17:03:51,083][20347] Updated weights for policy 0, policy_version 1648151 (0.0008) [2023-12-27 17:03:51,142][20347] Updated weights for policy 0, policy_version 1648161 (0.0008) [2023-12-27 17:03:51,239][20381] Updated weights for policy 1, policy_version 1646551 (0.0007) [2023-12-27 17:03:51,296][20381] Updated weights for policy 1, policy_version 1646561 (0.0009) [2023-12-27 17:03:51,357][20381] Updated weights for policy 1, policy_version 1646571 (0.0010) [2023-12-27 17:03:51,880][20347] Updated weights for policy 0, policy_version 1648171 (0.0009) [2023-12-27 17:03:51,946][20347] Updated weights for policy 0, policy_version 1648181 (0.0008) [2023-12-27 17:03:51,993][20347] Updated weights for policy 0, policy_version 1648191 (0.0008) [2023-12-27 17:03:52,063][20381] Updated weights for policy 1, policy_version 1646581 (0.0010) [2023-12-27 17:03:52,124][20381] Updated weights for policy 1, policy_version 1646591 (0.0010) [2023-12-27 17:03:52,185][20381] Updated weights for policy 1, policy_version 1646601 (0.0010) [2023-12-27 17:03:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 843595776. Throughput: 0: 10200.2, 1: 10419.5. Samples: 843564456. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:52,363][19170] Avg episode reward: [(0, '980.380'), (1, '804.980')] [2023-12-27 17:03:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001646608_421576704.pth... [2023-12-27 17:03:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001648200_422019072.pth... [2023-12-27 17:03:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001645360_421257216.pth [2023-12-27 17:03:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001647016_421715968.pth [2023-12-27 17:03:52,722][20347] Updated weights for policy 0, policy_version 1648201 (0.0008) [2023-12-27 17:03:52,779][20347] Updated weights for policy 0, policy_version 1648211 (0.0007) [2023-12-27 17:03:52,834][20347] Updated weights for policy 0, policy_version 1648221 (0.0006) [2023-12-27 17:03:52,885][20347] Updated weights for policy 0, policy_version 1648231 (0.0006) [2023-12-27 17:03:52,890][20381] Updated weights for policy 1, policy_version 1646611 (0.0010) [2023-12-27 17:03:52,958][20381] Updated weights for policy 1, policy_version 1646621 (0.0007) [2023-12-27 17:03:53,020][20381] Updated weights for policy 1, policy_version 1646631 (0.0008) [2023-12-27 17:03:53,454][20347] Updated weights for policy 0, policy_version 1648241 (0.0007) [2023-12-27 17:03:53,507][20347] Updated weights for policy 0, policy_version 1648251 (0.0007) [2023-12-27 17:03:53,563][20347] Updated weights for policy 0, policy_version 1648261 (0.0008) [2023-12-27 17:03:53,698][20381] Updated weights for policy 1, policy_version 1646641 (0.0007) [2023-12-27 17:03:53,759][20381] Updated weights for policy 1, policy_version 1646651 (0.0007) [2023-12-27 17:03:53,821][20381] Updated weights for policy 1, policy_version 1646661 (0.0011) [2023-12-27 17:03:53,880][20381] Updated weights for policy 1, policy_version 1646671 (0.0011) [2023-12-27 17:03:54,225][20347] Updated weights for policy 0, policy_version 1648271 (0.0008) [2023-12-27 17:03:54,287][20347] Updated weights for policy 0, policy_version 1648281 (0.0006) [2023-12-27 17:03:54,345][20347] Updated weights for policy 0, policy_version 1648291 (0.0010) [2023-12-27 17:03:54,540][20381] Updated weights for policy 1, policy_version 1646681 (0.0010) [2023-12-27 17:03:54,596][20381] Updated weights for policy 1, policy_version 1646691 (0.0010) [2023-12-27 17:03:54,649][20381] Updated weights for policy 1, policy_version 1646701 (0.0010) [2023-12-27 17:03:54,962][20347] Updated weights for policy 0, policy_version 1648301 (0.0007) [2023-12-27 17:03:55,027][20347] Updated weights for policy 0, policy_version 1648311 (0.0006) [2023-12-27 17:03:55,089][20347] Updated weights for policy 0, policy_version 1648321 (0.0007) [2023-12-27 17:03:55,365][20381] Updated weights for policy 1, policy_version 1646711 (0.0010) [2023-12-27 17:03:55,419][20381] Updated weights for policy 1, policy_version 1646721 (0.0010) [2023-12-27 17:03:55,474][20381] Updated weights for policy 1, policy_version 1646731 (0.0010) [2023-12-27 17:03:55,718][20347] Updated weights for policy 0, policy_version 1648331 (0.0007) [2023-12-27 17:03:55,766][20347] Updated weights for policy 0, policy_version 1648341 (0.0008) [2023-12-27 17:03:55,813][20347] Updated weights for policy 0, policy_version 1648351 (0.0007) [2023-12-27 17:03:56,140][20381] Updated weights for policy 1, policy_version 1646741 (0.0008) [2023-12-27 17:03:56,194][20381] Updated weights for policy 1, policy_version 1646751 (0.0005) [2023-12-27 17:03:56,250][20381] Updated weights for policy 1, policy_version 1646761 (0.0007) [2023-12-27 17:03:56,487][20347] Updated weights for policy 0, policy_version 1648361 (0.0007) [2023-12-27 17:03:56,488][20291] KL-divergence is very high: 172.2224 [2023-12-27 17:03:56,540][20291] KL-divergence is very high: 438.5125 [2023-12-27 17:03:56,548][20347] Updated weights for policy 0, policy_version 1648371 (0.0005) [2023-12-27 17:03:56,582][20291] KL-divergence is very high: 473.5182 [2023-12-27 17:03:56,605][20347] Updated weights for policy 0, policy_version 1648381 (0.0006) [2023-12-27 17:03:56,628][20291] KL-divergence is very high: 375.4352 [2023-12-27 17:03:56,665][20347] Updated weights for policy 0, policy_version 1648391 (0.0005) [2023-12-27 17:03:56,900][20381] Updated weights for policy 1, policy_version 1646771 (0.0008) [2023-12-27 17:03:56,961][20381] Updated weights for policy 1, policy_version 1646781 (0.0008) [2023-12-27 17:03:57,026][20381] Updated weights for policy 1, policy_version 1646791 (0.0008) [2023-12-27 17:03:57,242][20347] Updated weights for policy 0, policy_version 1648401 (0.0006) [2023-12-27 17:03:57,295][20347] Updated weights for policy 0, policy_version 1648411 (0.0009) [2023-12-27 17:03:57,348][20347] Updated weights for policy 0, policy_version 1648421 (0.0009) [2023-12-27 17:03:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 843702272. Throughput: 0: 10243.2, 1: 10364.2. Samples: 843686568. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:03:57,363][19170] Avg episode reward: [(0, '970.600'), (1, '853.490')] [2023-12-27 17:03:57,754][20381] Updated weights for policy 1, policy_version 1646801 (0.0008) [2023-12-27 17:03:57,811][20381] Updated weights for policy 1, policy_version 1646811 (0.0006) [2023-12-27 17:03:57,859][20381] Updated weights for policy 1, policy_version 1646821 (0.0010) [2023-12-27 17:03:57,908][20381] Updated weights for policy 1, policy_version 1646831 (0.0011) [2023-12-27 17:03:58,039][20347] Updated weights for policy 0, policy_version 1648431 (0.0007) [2023-12-27 17:03:58,097][20347] Updated weights for policy 0, policy_version 1648441 (0.0007) [2023-12-27 17:03:58,147][20347] Updated weights for policy 0, policy_version 1648451 (0.0007) [2023-12-27 17:03:58,602][20381] Updated weights for policy 1, policy_version 1646841 (0.0008) [2023-12-27 17:03:58,667][20381] Updated weights for policy 1, policy_version 1646851 (0.0007) [2023-12-27 17:03:58,731][20381] Updated weights for policy 1, policy_version 1646861 (0.0008) [2023-12-27 17:03:58,957][20347] Updated weights for policy 0, policy_version 1648461 (0.0007) [2023-12-27 17:03:59,017][20347] Updated weights for policy 0, policy_version 1648471 (0.0007) [2023-12-27 17:03:59,075][20347] Updated weights for policy 0, policy_version 1648481 (0.0006) [2023-12-27 17:03:59,463][20381] Updated weights for policy 1, policy_version 1646871 (0.0008) [2023-12-27 17:03:59,521][20381] Updated weights for policy 1, policy_version 1646881 (0.0008) [2023-12-27 17:03:59,576][20381] Updated weights for policy 1, policy_version 1646891 (0.0007) [2023-12-27 17:03:59,714][20347] Updated weights for policy 0, policy_version 1648491 (0.0006) [2023-12-27 17:03:59,775][20347] Updated weights for policy 0, policy_version 1648501 (0.0007) [2023-12-27 17:03:59,837][20347] Updated weights for policy 0, policy_version 1648511 (0.0008) [2023-12-27 17:04:00,220][20381] Updated weights for policy 1, policy_version 1646901 (0.0007) [2023-12-27 17:04:00,281][20381] Updated weights for policy 1, policy_version 1646911 (0.0010) [2023-12-27 17:04:00,340][20381] Updated weights for policy 1, policy_version 1646921 (0.0010) [2023-12-27 17:04:00,565][20347] Updated weights for policy 0, policy_version 1648521 (0.0007) [2023-12-27 17:04:00,624][20347] Updated weights for policy 0, policy_version 1648531 (0.0007) [2023-12-27 17:04:00,681][20347] Updated weights for policy 0, policy_version 1648541 (0.0008) [2023-12-27 17:04:00,743][20347] Updated weights for policy 0, policy_version 1648551 (0.0008) [2023-12-27 17:04:00,982][20381] Updated weights for policy 1, policy_version 1646931 (0.0010) [2023-12-27 17:04:01,038][20381] Updated weights for policy 1, policy_version 1646941 (0.0010) [2023-12-27 17:04:01,100][20381] Updated weights for policy 1, policy_version 1646951 (0.0010) [2023-12-27 17:04:01,418][20291] KL-divergence is very high: 351.1118 [2023-12-27 17:04:01,425][20347] Updated weights for policy 0, policy_version 1648561 (0.0008) [2023-12-27 17:04:01,470][20291] KL-divergence is very high: 394.6333 [2023-12-27 17:04:01,491][20347] Updated weights for policy 0, policy_version 1648571 (0.0006) [2023-12-27 17:04:01,524][20291] KL-divergence is very high: 200.2590 [2023-12-27 17:04:01,558][20347] Updated weights for policy 0, policy_version 1648581 (0.0008) [2023-12-27 17:04:01,804][20381] Updated weights for policy 1, policy_version 1646961 (0.0009) [2023-12-27 17:04:01,868][20381] Updated weights for policy 1, policy_version 1646971 (0.0007) [2023-12-27 17:04:01,923][20381] Updated weights for policy 1, policy_version 1646981 (0.0010) [2023-12-27 17:04:01,983][20381] Updated weights for policy 1, policy_version 1646991 (0.0011) [2023-12-27 17:04:02,229][20347] Updated weights for policy 0, policy_version 1648591 (0.0007) [2023-12-27 17:04:02,288][20347] Updated weights for policy 0, policy_version 1648601 (0.0007) [2023-12-27 17:04:02,344][20347] Updated weights for policy 0, policy_version 1648611 (0.0008) [2023-12-27 17:04:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 843792384. Throughput: 0: 10196.0, 1: 10378.1. Samples: 843807012. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:02,363][19170] Avg episode reward: [(0, '960.860'), (1, '872.960')] [2023-12-27 17:04:02,652][20381] Updated weights for policy 1, policy_version 1647001 (0.0008) [2023-12-27 17:04:02,709][20381] Updated weights for policy 1, policy_version 1647011 (0.0008) [2023-12-27 17:04:02,766][20381] Updated weights for policy 1, policy_version 1647021 (0.0008) [2023-12-27 17:04:03,038][20347] Updated weights for policy 0, policy_version 1648621 (0.0008) [2023-12-27 17:04:03,082][20347] Updated weights for policy 0, policy_version 1648631 (0.0007) [2023-12-27 17:04:03,129][20347] Updated weights for policy 0, policy_version 1648641 (0.0007) [2023-12-27 17:04:03,402][20381] Updated weights for policy 1, policy_version 1647031 (0.0006) [2023-12-27 17:04:03,454][20381] Updated weights for policy 1, policy_version 1647041 (0.0006) [2023-12-27 17:04:03,506][20381] Updated weights for policy 1, policy_version 1647051 (0.0006) [2023-12-27 17:04:03,895][20347] Updated weights for policy 0, policy_version 1648651 (0.0008) [2023-12-27 17:04:03,961][20347] Updated weights for policy 0, policy_version 1648661 (0.0007) [2023-12-27 17:04:04,023][20347] Updated weights for policy 0, policy_version 1648671 (0.0007) [2023-12-27 17:04:04,092][20381] Updated weights for policy 1, policy_version 1647061 (0.0005) [2023-12-27 17:04:04,152][20381] Updated weights for policy 1, policy_version 1647071 (0.0005) [2023-12-27 17:04:04,201][20381] Updated weights for policy 1, policy_version 1647081 (0.0006) [2023-12-27 17:04:04,689][20347] Updated weights for policy 0, policy_version 1648681 (0.0008) [2023-12-27 17:04:04,739][20347] Updated weights for policy 0, policy_version 1648691 (0.0005) [2023-12-27 17:04:04,779][20381] Updated weights for policy 1, policy_version 1647091 (0.0007) [2023-12-27 17:04:04,797][20347] Updated weights for policy 0, policy_version 1648701 (0.0005) [2023-12-27 17:04:04,836][20381] Updated weights for policy 1, policy_version 1647101 (0.0006) [2023-12-27 17:04:04,846][20347] Updated weights for policy 0, policy_version 1648711 (0.0005) [2023-12-27 17:04:04,895][20381] Updated weights for policy 1, policy_version 1647111 (0.0011) [2023-12-27 17:04:05,515][20347] Updated weights for policy 0, policy_version 1648721 (0.0010) [2023-12-27 17:04:05,536][20381] Updated weights for policy 1, policy_version 1647121 (0.0010) [2023-12-27 17:04:05,569][20347] Updated weights for policy 0, policy_version 1648731 (0.0010) [2023-12-27 17:04:05,584][20381] Updated weights for policy 1, policy_version 1647131 (0.0010) [2023-12-27 17:04:05,619][20347] Updated weights for policy 0, policy_version 1648741 (0.0010) [2023-12-27 17:04:05,632][20381] Updated weights for policy 1, policy_version 1647141 (0.0007) [2023-12-27 17:04:05,684][20381] Updated weights for policy 1, policy_version 1647151 (0.0005) [2023-12-27 17:04:06,275][20381] Updated weights for policy 1, policy_version 1647161 (0.0009) [2023-12-27 17:04:06,315][20347] Updated weights for policy 0, policy_version 1648751 (0.0010) [2023-12-27 17:04:06,333][20381] Updated weights for policy 1, policy_version 1647171 (0.0011) [2023-12-27 17:04:06,375][20347] Updated weights for policy 0, policy_version 1648761 (0.0010) [2023-12-27 17:04:06,389][20381] Updated weights for policy 1, policy_version 1647181 (0.0011) [2023-12-27 17:04:06,431][20347] Updated weights for policy 0, policy_version 1648771 (0.0010) [2023-12-27 17:04:07,066][20347] Updated weights for policy 0, policy_version 1648781 (0.0009) [2023-12-27 17:04:07,081][20381] Updated weights for policy 1, policy_version 1647191 (0.0011) [2023-12-27 17:04:07,122][20347] Updated weights for policy 0, policy_version 1648791 (0.0008) [2023-12-27 17:04:07,131][20381] Updated weights for policy 1, policy_version 1647201 (0.0011) [2023-12-27 17:04:07,181][20381] Updated weights for policy 1, policy_version 1647211 (0.0011) [2023-12-27 17:04:07,181][20347] Updated weights for policy 0, policy_version 1648801 (0.0009) [2023-12-27 17:04:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 843907072. Throughput: 0: 10171.5, 1: 10440.9. Samples: 843870404. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:07,363][19170] Avg episode reward: [(0, '980.450'), (1, '872.960')] [2023-12-27 17:04:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001647216_421732352.pth... [2023-12-27 17:04:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001648808_422174720.pth... [2023-12-27 17:04:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001646000_421421056.pth [2023-12-27 17:04:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001647624_421871616.pth [2023-12-27 17:04:07,843][20381] Updated weights for policy 1, policy_version 1647221 (0.0011) [2023-12-27 17:04:07,850][20347] Updated weights for policy 0, policy_version 1648811 (0.0009) [2023-12-27 17:04:07,892][20381] Updated weights for policy 1, policy_version 1647231 (0.0010) [2023-12-27 17:04:07,910][20347] Updated weights for policy 0, policy_version 1648821 (0.0005) [2023-12-27 17:04:07,946][20381] Updated weights for policy 1, policy_version 1647241 (0.0010) [2023-12-27 17:04:07,977][20347] Updated weights for policy 0, policy_version 1648831 (0.0005) [2023-12-27 17:04:08,524][20347] Updated weights for policy 0, policy_version 1648841 (0.0007) [2023-12-27 17:04:08,585][20347] Updated weights for policy 0, policy_version 1648851 (0.0010) [2023-12-27 17:04:08,640][20347] Updated weights for policy 0, policy_version 1648861 (0.0010) [2023-12-27 17:04:08,673][20381] Updated weights for policy 1, policy_version 1647251 (0.0010) [2023-12-27 17:04:08,698][20347] Updated weights for policy 0, policy_version 1648871 (0.0010) [2023-12-27 17:04:08,732][20381] Updated weights for policy 1, policy_version 1647261 (0.0010) [2023-12-27 17:04:08,787][20381] Updated weights for policy 1, policy_version 1647271 (0.0010) [2023-12-27 17:04:09,395][20347] Updated weights for policy 0, policy_version 1648881 (0.0009) [2023-12-27 17:04:09,465][20347] Updated weights for policy 0, policy_version 1648891 (0.0009) [2023-12-27 17:04:09,520][20381] Updated weights for policy 1, policy_version 1647281 (0.0010) [2023-12-27 17:04:09,525][20347] Updated weights for policy 0, policy_version 1648901 (0.0011) [2023-12-27 17:04:09,578][20381] Updated weights for policy 1, policy_version 1647291 (0.0009) [2023-12-27 17:04:09,628][20381] Updated weights for policy 1, policy_version 1647301 (0.0010) [2023-12-27 17:04:09,683][20381] Updated weights for policy 1, policy_version 1647311 (0.0009) [2023-12-27 17:04:10,288][20347] Updated weights for policy 0, policy_version 1648911 (0.0009) [2023-12-27 17:04:10,340][20381] Updated weights for policy 1, policy_version 1647321 (0.0008) [2023-12-27 17:04:10,348][20347] Updated weights for policy 0, policy_version 1648921 (0.0007) [2023-12-27 17:04:10,401][20381] Updated weights for policy 1, policy_version 1647331 (0.0011) [2023-12-27 17:04:10,408][20347] Updated weights for policy 0, policy_version 1648931 (0.0009) [2023-12-27 17:04:10,458][20381] Updated weights for policy 1, policy_version 1647341 (0.0011) [2023-12-27 17:04:11,039][20347] Updated weights for policy 0, policy_version 1648941 (0.0010) [2023-12-27 17:04:11,100][20347] Updated weights for policy 0, policy_version 1648951 (0.0012) [2023-12-27 17:04:11,166][20347] Updated weights for policy 0, policy_version 1648961 (0.0008) [2023-12-27 17:04:11,176][20381] Updated weights for policy 1, policy_version 1647351 (0.0011) [2023-12-27 17:04:11,237][20381] Updated weights for policy 1, policy_version 1647361 (0.0010) [2023-12-27 17:04:11,301][20381] Updated weights for policy 1, policy_version 1647371 (0.0010) [2023-12-27 17:04:11,900][20347] Updated weights for policy 0, policy_version 1648971 (0.0007) [2023-12-27 17:04:11,957][20347] Updated weights for policy 0, policy_version 1648981 (0.0007) [2023-12-27 17:04:12,014][20347] Updated weights for policy 0, policy_version 1648991 (0.0007) [2023-12-27 17:04:12,091][20381] Updated weights for policy 1, policy_version 1647381 (0.0010) [2023-12-27 17:04:12,149][20381] Updated weights for policy 1, policy_version 1647391 (0.0011) [2023-12-27 17:04:12,199][20381] Updated weights for policy 1, policy_version 1647401 (0.0011) [2023-12-27 17:04:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 844005376. Throughput: 0: 10250.3, 1: 10337.9. Samples: 843994164. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:12,363][19170] Avg episode reward: [(0, '980.460'), (1, '873.770')] [2023-12-27 17:04:12,721][20347] Updated weights for policy 0, policy_version 1649001 (0.0008) [2023-12-27 17:04:12,781][20347] Updated weights for policy 0, policy_version 1649011 (0.0009) [2023-12-27 17:04:12,845][20347] Updated weights for policy 0, policy_version 1649021 (0.0006) [2023-12-27 17:04:12,888][20381] Updated weights for policy 1, policy_version 1647411 (0.0009) [2023-12-27 17:04:12,897][20347] Updated weights for policy 0, policy_version 1649031 (0.0007) [2023-12-27 17:04:12,949][20381] Updated weights for policy 1, policy_version 1647421 (0.0007) [2023-12-27 17:04:13,004][20381] Updated weights for policy 1, policy_version 1647431 (0.0005) [2023-12-27 17:04:13,465][20347] Updated weights for policy 0, policy_version 1649041 (0.0006) [2023-12-27 17:04:13,514][20347] Updated weights for policy 0, policy_version 1649051 (0.0005) [2023-12-27 17:04:13,565][20347] Updated weights for policy 0, policy_version 1649061 (0.0005) [2023-12-27 17:04:13,642][20381] Updated weights for policy 1, policy_version 1647441 (0.0008) [2023-12-27 17:04:13,694][20381] Updated weights for policy 1, policy_version 1647451 (0.0006) [2023-12-27 17:04:13,751][20381] Updated weights for policy 1, policy_version 1647461 (0.0006) [2023-12-27 17:04:13,796][20381] Updated weights for policy 1, policy_version 1647471 (0.0009) [2023-12-27 17:04:14,133][20347] Updated weights for policy 0, policy_version 1649071 (0.0006) [2023-12-27 17:04:14,195][20347] Updated weights for policy 0, policy_version 1649081 (0.0005) [2023-12-27 17:04:14,257][20347] Updated weights for policy 0, policy_version 1649091 (0.0006) [2023-12-27 17:04:14,468][20381] Updated weights for policy 1, policy_version 1647481 (0.0007) [2023-12-27 17:04:14,516][20381] Updated weights for policy 1, policy_version 1647491 (0.0007) [2023-12-27 17:04:14,568][20381] Updated weights for policy 1, policy_version 1647501 (0.0007) [2023-12-27 17:04:14,825][20347] Updated weights for policy 0, policy_version 1649101 (0.0006) [2023-12-27 17:04:14,896][20347] Updated weights for policy 0, policy_version 1649111 (0.0008) [2023-12-27 17:04:14,957][20347] Updated weights for policy 0, policy_version 1649121 (0.0008) [2023-12-27 17:04:15,285][20381] Updated weights for policy 1, policy_version 1647511 (0.0008) [2023-12-27 17:04:15,346][20381] Updated weights for policy 1, policy_version 1647521 (0.0008) [2023-12-27 17:04:15,408][20381] Updated weights for policy 1, policy_version 1647531 (0.0006) [2023-12-27 17:04:15,672][20347] Updated weights for policy 0, policy_version 1649131 (0.0008) [2023-12-27 17:04:15,726][20347] Updated weights for policy 0, policy_version 1649141 (0.0009) [2023-12-27 17:04:15,778][20347] Updated weights for policy 0, policy_version 1649151 (0.0008) [2023-12-27 17:04:15,982][20381] Updated weights for policy 1, policy_version 1647541 (0.0006) [2023-12-27 17:04:16,044][20381] Updated weights for policy 1, policy_version 1647551 (0.0006) [2023-12-27 17:04:16,104][20381] Updated weights for policy 1, policy_version 1647561 (0.0006) [2023-12-27 17:04:16,510][20347] Updated weights for policy 0, policy_version 1649161 (0.0007) [2023-12-27 17:04:16,574][20347] Updated weights for policy 0, policy_version 1649171 (0.0008) [2023-12-27 17:04:16,622][20347] Updated weights for policy 0, policy_version 1649181 (0.0010) [2023-12-27 17:04:16,675][20347] Updated weights for policy 0, policy_version 1649191 (0.0010) [2023-12-27 17:04:16,699][20381] Updated weights for policy 1, policy_version 1647571 (0.0006) [2023-12-27 17:04:16,757][20381] Updated weights for policy 1, policy_version 1647581 (0.0007) [2023-12-27 17:04:16,813][20381] Updated weights for policy 1, policy_version 1647591 (0.0008) [2023-12-27 17:04:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 844103680. Throughput: 0: 10247.1, 1: 10259.1. Samples: 844118268. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:17,363][19170] Avg episode reward: [(0, '970.740'), (1, '863.990')] [2023-12-27 17:04:17,391][20347] Updated weights for policy 0, policy_version 1649201 (0.0010) [2023-12-27 17:04:17,452][20347] Updated weights for policy 0, policy_version 1649211 (0.0010) [2023-12-27 17:04:17,501][20381] Updated weights for policy 1, policy_version 1647601 (0.0007) [2023-12-27 17:04:17,505][20347] Updated weights for policy 0, policy_version 1649221 (0.0008) [2023-12-27 17:04:17,559][20381] Updated weights for policy 1, policy_version 1647611 (0.0007) [2023-12-27 17:04:17,615][20381] Updated weights for policy 1, policy_version 1647621 (0.0007) [2023-12-27 17:04:17,675][20381] Updated weights for policy 1, policy_version 1647631 (0.0007) [2023-12-27 17:04:18,107][20347] Updated weights for policy 0, policy_version 1649231 (0.0005) [2023-12-27 17:04:18,169][20347] Updated weights for policy 0, policy_version 1649241 (0.0006) [2023-12-27 17:04:18,230][20347] Updated weights for policy 0, policy_version 1649251 (0.0010) [2023-12-27 17:04:18,361][20381] Updated weights for policy 1, policy_version 1647641 (0.0007) [2023-12-27 17:04:18,428][20381] Updated weights for policy 1, policy_version 1647651 (0.0006) [2023-12-27 17:04:18,493][20381] Updated weights for policy 1, policy_version 1647661 (0.0008) [2023-12-27 17:04:18,834][20347] Updated weights for policy 0, policy_version 1649261 (0.0010) [2023-12-27 17:04:18,896][20347] Updated weights for policy 0, policy_version 1649271 (0.0011) [2023-12-27 17:04:18,953][20347] Updated weights for policy 0, policy_version 1649281 (0.0009) [2023-12-27 17:04:19,120][20381] Updated weights for policy 1, policy_version 1647671 (0.0007) [2023-12-27 17:04:19,175][20381] Updated weights for policy 1, policy_version 1647681 (0.0008) [2023-12-27 17:04:19,232][20381] Updated weights for policy 1, policy_version 1647691 (0.0008) [2023-12-27 17:04:19,639][20347] Updated weights for policy 0, policy_version 1649291 (0.0008) [2023-12-27 17:04:19,697][20347] Updated weights for policy 0, policy_version 1649301 (0.0008) [2023-12-27 17:04:19,748][20347] Updated weights for policy 0, policy_version 1649311 (0.0008) [2023-12-27 17:04:19,969][20381] Updated weights for policy 1, policy_version 1647701 (0.0008) [2023-12-27 17:04:20,026][20381] Updated weights for policy 1, policy_version 1647711 (0.0008) [2023-12-27 17:04:20,090][20381] Updated weights for policy 1, policy_version 1647721 (0.0008) [2023-12-27 17:04:20,517][20347] Updated weights for policy 0, policy_version 1649321 (0.0008) [2023-12-27 17:04:20,582][20347] Updated weights for policy 0, policy_version 1649331 (0.0008) [2023-12-27 17:04:20,652][20347] Updated weights for policy 0, policy_version 1649341 (0.0008) [2023-12-27 17:04:20,709][20347] Updated weights for policy 0, policy_version 1649351 (0.0007) [2023-12-27 17:04:20,749][20381] Updated weights for policy 1, policy_version 1647731 (0.0008) [2023-12-27 17:04:20,806][20381] Updated weights for policy 1, policy_version 1647741 (0.0008) [2023-12-27 17:04:20,859][20381] Updated weights for policy 1, policy_version 1647751 (0.0007) [2023-12-27 17:04:21,435][20347] Updated weights for policy 0, policy_version 1649361 (0.0008) [2023-12-27 17:04:21,497][20347] Updated weights for policy 0, policy_version 1649371 (0.0008) [2023-12-27 17:04:21,555][20347] Updated weights for policy 0, policy_version 1649381 (0.0006) [2023-12-27 17:04:21,605][20381] Updated weights for policy 1, policy_version 1647761 (0.0006) [2023-12-27 17:04:21,669][20381] Updated weights for policy 1, policy_version 1647771 (0.0009) [2023-12-27 17:04:21,740][20381] Updated weights for policy 1, policy_version 1647781 (0.0008) [2023-12-27 17:04:21,808][20381] Updated weights for policy 1, policy_version 1647791 (0.0007) [2023-12-27 17:04:22,187][20347] Updated weights for policy 0, policy_version 1649391 (0.0007) [2023-12-27 17:04:22,243][20347] Updated weights for policy 0, policy_version 1649401 (0.0007) [2023-12-27 17:04:22,305][20347] Updated weights for policy 0, policy_version 1649411 (0.0008) [2023-12-27 17:04:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 844210176. Throughput: 0: 10218.5, 1: 10216.4. Samples: 844179360. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:22,363][19170] Avg episode reward: [(0, '980.550'), (1, '863.310')] [2023-12-27 17:04:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001647792_421879808.pth... [2023-12-27 17:04:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001649416_422330368.pth... [2023-12-27 17:04:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001646608_421576704.pth [2023-12-27 17:04:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001648200_422019072.pth [2023-12-27 17:04:22,543][20381] Updated weights for policy 1, policy_version 1647801 (0.0008) [2023-12-27 17:04:22,599][20381] Updated weights for policy 1, policy_version 1647811 (0.0006) [2023-12-27 17:04:22,647][20381] Updated weights for policy 1, policy_version 1647821 (0.0005) [2023-12-27 17:04:23,006][20347] Updated weights for policy 0, policy_version 1649421 (0.0006) [2023-12-27 17:04:23,059][20347] Updated weights for policy 0, policy_version 1649431 (0.0006) [2023-12-27 17:04:23,108][20347] Updated weights for policy 0, policy_version 1649441 (0.0005) [2023-12-27 17:04:23,322][20381] Updated weights for policy 1, policy_version 1647831 (0.0007) [2023-12-27 17:04:23,366][20381] Updated weights for policy 1, policy_version 1647841 (0.0006) [2023-12-27 17:04:23,413][20381] Updated weights for policy 1, policy_version 1647851 (0.0007) [2023-12-27 17:04:23,698][20347] Updated weights for policy 0, policy_version 1649451 (0.0006) [2023-12-27 17:04:23,756][20347] Updated weights for policy 0, policy_version 1649461 (0.0008) [2023-12-27 17:04:23,801][20347] Updated weights for policy 0, policy_version 1649471 (0.0005) [2023-12-27 17:04:24,098][20381] Updated weights for policy 1, policy_version 1647861 (0.0007) [2023-12-27 17:04:24,147][20381] Updated weights for policy 1, policy_version 1647871 (0.0007) [2023-12-27 17:04:24,206][20381] Updated weights for policy 1, policy_version 1647881 (0.0008) [2023-12-27 17:04:24,377][20347] Updated weights for policy 0, policy_version 1649481 (0.0005) [2023-12-27 17:04:24,426][20347] Updated weights for policy 0, policy_version 1649491 (0.0006) [2023-12-27 17:04:24,475][20347] Updated weights for policy 0, policy_version 1649501 (0.0007) [2023-12-27 17:04:24,520][20347] Updated weights for policy 0, policy_version 1649511 (0.0008) [2023-12-27 17:04:24,842][20381] Updated weights for policy 1, policy_version 1647891 (0.0010) [2023-12-27 17:04:24,897][20381] Updated weights for policy 1, policy_version 1647901 (0.0010) [2023-12-27 17:04:24,954][20381] Updated weights for policy 1, policy_version 1647911 (0.0010) [2023-12-27 17:04:25,102][20347] Updated weights for policy 0, policy_version 1649521 (0.0007) [2023-12-27 17:04:25,162][20347] Updated weights for policy 0, policy_version 1649531 (0.0007) [2023-12-27 17:04:25,226][20347] Updated weights for policy 0, policy_version 1649541 (0.0007) [2023-12-27 17:04:25,644][20381] Updated weights for policy 1, policy_version 1647921 (0.0010) [2023-12-27 17:04:25,703][20381] Updated weights for policy 1, policy_version 1647931 (0.0008) [2023-12-27 17:04:25,770][20381] Updated weights for policy 1, policy_version 1647941 (0.0007) [2023-12-27 17:04:25,824][20381] Updated weights for policy 1, policy_version 1647951 (0.0007) [2023-12-27 17:04:25,861][20347] Updated weights for policy 0, policy_version 1649551 (0.0006) [2023-12-27 17:04:25,919][20347] Updated weights for policy 0, policy_version 1649561 (0.0006) [2023-12-27 17:04:25,975][20347] Updated weights for policy 0, policy_version 1649571 (0.0007) [2023-12-27 17:04:26,472][20381] Updated weights for policy 1, policy_version 1647961 (0.0008) [2023-12-27 17:04:26,527][20381] Updated weights for policy 1, policy_version 1647971 (0.0008) [2023-12-27 17:04:26,576][20381] Updated weights for policy 1, policy_version 1647981 (0.0007) [2023-12-27 17:04:26,620][20347] Updated weights for policy 0, policy_version 1649581 (0.0008) [2023-12-27 17:04:26,678][20347] Updated weights for policy 0, policy_version 1649591 (0.0005) [2023-12-27 17:04:26,729][20347] Updated weights for policy 0, policy_version 1649601 (0.0005) [2023-12-27 17:04:27,250][20381] Updated weights for policy 1, policy_version 1647991 (0.0007) [2023-12-27 17:04:27,314][20381] Updated weights for policy 1, policy_version 1648001 (0.0008) [2023-12-27 17:04:27,356][20347] Updated weights for policy 0, policy_version 1649611 (0.0005) [2023-12-27 17:04:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 844308480. Throughput: 0: 10280.1, 1: 10250.5. Samples: 844304072. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-12-27 17:04:27,363][19170] Avg episode reward: [(0, '990.270'), (1, '873.130')] [2023-12-27 17:04:27,372][20381] Updated weights for policy 1, policy_version 1648011 (0.0007) [2023-12-27 17:04:27,418][20347] Updated weights for policy 0, policy_version 1649621 (0.0006) [2023-12-27 17:04:27,485][20347] Updated weights for policy 0, policy_version 1649631 (0.0008) [2023-12-27 17:04:27,920][20381] Updated weights for policy 1, policy_version 1648021 (0.0006) [2023-12-27 17:04:27,979][20381] Updated weights for policy 1, policy_version 1648031 (0.0007) [2023-12-27 17:04:28,037][20381] Updated weights for policy 1, policy_version 1648041 (0.0005) [2023-12-27 17:04:28,072][20347] Updated weights for policy 0, policy_version 1649641 (0.0005) [2023-12-27 17:04:28,122][20347] Updated weights for policy 0, policy_version 1649651 (0.0005) [2023-12-27 17:04:28,170][20347] Updated weights for policy 0, policy_version 1649661 (0.0006) [2023-12-27 17:04:28,219][20347] Updated weights for policy 0, policy_version 1649671 (0.0007) [2023-12-27 17:04:28,719][20381] Updated weights for policy 1, policy_version 1648051 (0.0006) [2023-12-27 17:04:28,775][20381] Updated weights for policy 1, policy_version 1648061 (0.0007) [2023-12-27 17:04:28,835][20381] Updated weights for policy 1, policy_version 1648071 (0.0006) [2023-12-27 17:04:28,852][20347] Updated weights for policy 0, policy_version 1649681 (0.0007) [2023-12-27 17:04:28,909][20347] Updated weights for policy 0, policy_version 1649691 (0.0007) [2023-12-27 17:04:28,958][20347] Updated weights for policy 0, policy_version 1649701 (0.0008) [2023-12-27 17:04:29,539][20381] Updated weights for policy 1, policy_version 1648081 (0.0005) [2023-12-27 17:04:29,603][20381] Updated weights for policy 1, policy_version 1648091 (0.0007) [2023-12-27 17:04:29,632][20347] Updated weights for policy 0, policy_version 1649711 (0.0007) [2023-12-27 17:04:29,649][20381] Updated weights for policy 1, policy_version 1648101 (0.0007) [2023-12-27 17:04:29,681][20347] Updated weights for policy 0, policy_version 1649721 (0.0006) [2023-12-27 17:04:29,695][20381] Updated weights for policy 1, policy_version 1648111 (0.0005) [2023-12-27 17:04:29,741][20347] Updated weights for policy 0, policy_version 1649731 (0.0007) [2023-12-27 17:04:30,393][20381] Updated weights for policy 1, policy_version 1648121 (0.0007) [2023-12-27 17:04:30,421][20347] Updated weights for policy 0, policy_version 1649741 (0.0007) [2023-12-27 17:04:30,452][20381] Updated weights for policy 1, policy_version 1648131 (0.0006) [2023-12-27 17:04:30,473][20347] Updated weights for policy 0, policy_version 1649751 (0.0005) [2023-12-27 17:04:30,508][20381] Updated weights for policy 1, policy_version 1648141 (0.0006) [2023-12-27 17:04:30,522][20347] Updated weights for policy 0, policy_version 1649761 (0.0005) [2023-12-27 17:04:31,115][20347] Updated weights for policy 0, policy_version 1649771 (0.0006) [2023-12-27 17:04:31,126][20381] Updated weights for policy 1, policy_version 1648151 (0.0007) [2023-12-27 17:04:31,179][20347] Updated weights for policy 0, policy_version 1649781 (0.0007) [2023-12-27 17:04:31,191][20381] Updated weights for policy 1, policy_version 1648161 (0.0008) [2023-12-27 17:04:31,242][20347] Updated weights for policy 0, policy_version 1649791 (0.0007) [2023-12-27 17:04:31,252][20381] Updated weights for policy 1, policy_version 1648171 (0.0006) [2023-12-27 17:04:31,922][20381] Updated weights for policy 1, policy_version 1648181 (0.0007) [2023-12-27 17:04:31,980][20381] Updated weights for policy 1, policy_version 1648191 (0.0007) [2023-12-27 17:04:31,997][20347] Updated weights for policy 0, policy_version 1649801 (0.0008) [2023-12-27 17:04:32,034][20381] Updated weights for policy 1, policy_version 1648201 (0.0006) [2023-12-27 17:04:32,050][20347] Updated weights for policy 0, policy_version 1649811 (0.0006) [2023-12-27 17:04:32,100][20347] Updated weights for policy 0, policy_version 1649821 (0.0007) [2023-12-27 17:04:32,150][20347] Updated weights for policy 0, policy_version 1649831 (0.0006) [2023-12-27 17:04:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 844423168. Throughput: 0: 10352.8, 1: 10266.3. Samples: 844430588. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:32,363][19170] Avg episode reward: [(0, '980.470'), (1, '785.490')] [2023-12-27 17:04:32,744][20381] Updated weights for policy 1, policy_version 1648211 (0.0009) [2023-12-27 17:04:32,796][20381] Updated weights for policy 1, policy_version 1648221 (0.0010) [2023-12-27 17:04:32,851][20347] Updated weights for policy 0, policy_version 1649841 (0.0007) [2023-12-27 17:04:32,852][20381] Updated weights for policy 1, policy_version 1648231 (0.0009) [2023-12-27 17:04:32,908][20347] Updated weights for policy 0, policy_version 1649851 (0.0007) [2023-12-27 17:04:32,961][20347] Updated weights for policy 0, policy_version 1649861 (0.0007) [2023-12-27 17:04:33,416][20381] Updated weights for policy 1, policy_version 1648241 (0.0007) [2023-12-27 17:04:33,477][20381] Updated weights for policy 1, policy_version 1648251 (0.0007) [2023-12-27 17:04:33,528][20381] Updated weights for policy 1, policy_version 1648261 (0.0010) [2023-12-27 17:04:33,582][20381] Updated weights for policy 1, policy_version 1648271 (0.0010) [2023-12-27 17:04:33,642][20347] Updated weights for policy 0, policy_version 1649871 (0.0008) [2023-12-27 17:04:33,688][20347] Updated weights for policy 0, policy_version 1649881 (0.0007) [2023-12-27 17:04:33,751][20347] Updated weights for policy 0, policy_version 1649891 (0.0005) [2023-12-27 17:04:34,202][20381] Updated weights for policy 1, policy_version 1648281 (0.0008) [2023-12-27 17:04:34,262][20381] Updated weights for policy 1, policy_version 1648291 (0.0009) [2023-12-27 17:04:34,322][20381] Updated weights for policy 1, policy_version 1648301 (0.0011) [2023-12-27 17:04:34,438][20347] Updated weights for policy 0, policy_version 1649901 (0.0006) [2023-12-27 17:04:34,501][20347] Updated weights for policy 0, policy_version 1649911 (0.0007) [2023-12-27 17:04:34,569][20347] Updated weights for policy 0, policy_version 1649921 (0.0005) [2023-12-27 17:04:35,002][20381] Updated weights for policy 1, policy_version 1648311 (0.0007) [2023-12-27 17:04:35,061][20381] Updated weights for policy 1, policy_version 1648321 (0.0005) [2023-12-27 17:04:35,112][20381] Updated weights for policy 1, policy_version 1648331 (0.0005) [2023-12-27 17:04:35,222][20347] Updated weights for policy 0, policy_version 1649931 (0.0006) [2023-12-27 17:04:35,270][20347] Updated weights for policy 0, policy_version 1649941 (0.0006) [2023-12-27 17:04:35,318][20347] Updated weights for policy 0, policy_version 1649951 (0.0008) [2023-12-27 17:04:35,681][20381] Updated weights for policy 1, policy_version 1648341 (0.0007) [2023-12-27 17:04:35,735][20381] Updated weights for policy 1, policy_version 1648351 (0.0007) [2023-12-27 17:04:35,785][20381] Updated weights for policy 1, policy_version 1648361 (0.0010) [2023-12-27 17:04:35,918][20347] Updated weights for policy 0, policy_version 1649961 (0.0008) [2023-12-27 17:04:35,977][20347] Updated weights for policy 0, policy_version 1649971 (0.0007) [2023-12-27 17:04:36,041][20347] Updated weights for policy 0, policy_version 1649981 (0.0007) [2023-12-27 17:04:36,097][20347] Updated weights for policy 0, policy_version 1649991 (0.0007) [2023-12-27 17:04:36,510][20381] Updated weights for policy 1, policy_version 1648371 (0.0011) [2023-12-27 17:04:36,571][20381] Updated weights for policy 1, policy_version 1648381 (0.0011) [2023-12-27 17:04:36,634][20381] Updated weights for policy 1, policy_version 1648391 (0.0011) [2023-12-27 17:04:36,797][20347] Updated weights for policy 0, policy_version 1650001 (0.0007) [2023-12-27 17:04:36,853][20347] Updated weights for policy 0, policy_version 1650011 (0.0007) [2023-12-27 17:04:36,916][20347] Updated weights for policy 0, policy_version 1650021 (0.0008) [2023-12-27 17:04:37,177][20381] Updated weights for policy 1, policy_version 1648401 (0.0006) [2023-12-27 17:04:37,237][20381] Updated weights for policy 1, policy_version 1648411 (0.0005) [2023-12-27 17:04:37,294][20381] Updated weights for policy 1, policy_version 1648421 (0.0005) [2023-12-27 17:04:37,342][20381] Updated weights for policy 1, policy_version 1648431 (0.0006) [2023-12-27 17:04:37,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 844529664. Throughput: 0: 10367.1, 1: 10289.9. Samples: 844494016. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:37,363][19170] Avg episode reward: [(0, '980.470'), (1, '746.210')] [2023-12-27 17:04:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001648432_422043648.pth... [2023-12-27 17:04:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001650024_422486016.pth... [2023-12-27 17:04:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001647216_421732352.pth [2023-12-27 17:04:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001648808_422174720.pth [2023-12-27 17:04:37,552][20347] Updated weights for policy 0, policy_version 1650031 (0.0008) [2023-12-27 17:04:37,613][20347] Updated weights for policy 0, policy_version 1650041 (0.0006) [2023-12-27 17:04:37,671][20347] Updated weights for policy 0, policy_version 1650051 (0.0005) [2023-12-27 17:04:38,000][20381] Updated weights for policy 1, policy_version 1648441 (0.0006) [2023-12-27 17:04:38,054][20381] Updated weights for policy 1, policy_version 1648451 (0.0005) [2023-12-27 17:04:38,107][20381] Updated weights for policy 1, policy_version 1648461 (0.0005) [2023-12-27 17:04:38,333][20347] Updated weights for policy 0, policy_version 1650061 (0.0007) [2023-12-27 17:04:38,397][20347] Updated weights for policy 0, policy_version 1650071 (0.0008) [2023-12-27 17:04:38,445][20347] Updated weights for policy 0, policy_version 1650081 (0.0008) [2023-12-27 17:04:38,743][20381] Updated weights for policy 1, policy_version 1648471 (0.0006) [2023-12-27 17:04:38,796][20381] Updated weights for policy 1, policy_version 1648481 (0.0008) [2023-12-27 17:04:38,852][20381] Updated weights for policy 1, policy_version 1648491 (0.0007) [2023-12-27 17:04:39,195][20347] Updated weights for policy 0, policy_version 1650091 (0.0008) [2023-12-27 17:04:39,261][20347] Updated weights for policy 0, policy_version 1650101 (0.0008) [2023-12-27 17:04:39,317][20347] Updated weights for policy 0, policy_version 1650111 (0.0007) [2023-12-27 17:04:39,505][20381] Updated weights for policy 1, policy_version 1648501 (0.0007) [2023-12-27 17:04:39,567][20381] Updated weights for policy 1, policy_version 1648511 (0.0008) [2023-12-27 17:04:39,624][20381] Updated weights for policy 1, policy_version 1648521 (0.0008) [2023-12-27 17:04:40,025][20347] Updated weights for policy 0, policy_version 1650121 (0.0008) [2023-12-27 17:04:40,086][20347] Updated weights for policy 0, policy_version 1650131 (0.0007) [2023-12-27 17:04:40,147][20347] Updated weights for policy 0, policy_version 1650141 (0.0008) [2023-12-27 17:04:40,215][20347] Updated weights for policy 0, policy_version 1650151 (0.0008) [2023-12-27 17:04:40,315][20381] Updated weights for policy 1, policy_version 1648531 (0.0007) [2023-12-27 17:04:40,377][20381] Updated weights for policy 1, policy_version 1648541 (0.0008) [2023-12-27 17:04:40,437][20381] Updated weights for policy 1, policy_version 1648551 (0.0009) [2023-12-27 17:04:40,867][20347] Updated weights for policy 0, policy_version 1650161 (0.0006) [2023-12-27 17:04:40,928][20347] Updated weights for policy 0, policy_version 1650171 (0.0005) [2023-12-27 17:04:40,981][20347] Updated weights for policy 0, policy_version 1650181 (0.0006) [2023-12-27 17:04:41,120][20381] Updated weights for policy 1, policy_version 1648561 (0.0007) [2023-12-27 17:04:41,186][20381] Updated weights for policy 1, policy_version 1648571 (0.0007) [2023-12-27 17:04:41,249][20381] Updated weights for policy 1, policy_version 1648581 (0.0009) [2023-12-27 17:04:41,314][20381] Updated weights for policy 1, policy_version 1648591 (0.0009) [2023-12-27 17:04:41,644][20347] Updated weights for policy 0, policy_version 1650191 (0.0008) [2023-12-27 17:04:41,706][20347] Updated weights for policy 0, policy_version 1650201 (0.0008) [2023-12-27 17:04:41,770][20347] Updated weights for policy 0, policy_version 1650211 (0.0008) [2023-12-27 17:04:42,060][20381] Updated weights for policy 1, policy_version 1648601 (0.0006) [2023-12-27 17:04:42,119][20381] Updated weights for policy 1, policy_version 1648611 (0.0006) [2023-12-27 17:04:42,176][20381] Updated weights for policy 1, policy_version 1648621 (0.0007) [2023-12-27 17:04:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 844627968. Throughput: 0: 10347.5, 1: 10359.0. Samples: 844618360. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:42,363][19170] Avg episode reward: [(0, '990.240'), (1, '824.130')] [2023-12-27 17:04:42,526][20347] Updated weights for policy 0, policy_version 1650221 (0.0008) [2023-12-27 17:04:42,589][20347] Updated weights for policy 0, policy_version 1650231 (0.0008) [2023-12-27 17:04:42,643][20347] Updated weights for policy 0, policy_version 1650241 (0.0008) [2023-12-27 17:04:42,828][20381] Updated weights for policy 1, policy_version 1648631 (0.0006) [2023-12-27 17:04:42,891][20381] Updated weights for policy 1, policy_version 1648641 (0.0007) [2023-12-27 17:04:42,952][20381] Updated weights for policy 1, policy_version 1648651 (0.0007) [2023-12-27 17:04:43,357][20347] Updated weights for policy 0, policy_version 1650251 (0.0007) [2023-12-27 17:04:43,409][20347] Updated weights for policy 0, policy_version 1650261 (0.0007) [2023-12-27 17:04:43,466][20347] Updated weights for policy 0, policy_version 1650271 (0.0007) [2023-12-27 17:04:43,649][20381] Updated weights for policy 1, policy_version 1648661 (0.0008) [2023-12-27 17:04:43,705][20381] Updated weights for policy 1, policy_version 1648671 (0.0007) [2023-12-27 17:04:43,763][20381] Updated weights for policy 1, policy_version 1648681 (0.0007) [2023-12-27 17:04:44,142][20347] Updated weights for policy 0, policy_version 1650281 (0.0008) [2023-12-27 17:04:44,195][20347] Updated weights for policy 0, policy_version 1650291 (0.0009) [2023-12-27 17:04:44,252][20347] Updated weights for policy 0, policy_version 1650301 (0.0008) [2023-12-27 17:04:44,312][20347] Updated weights for policy 0, policy_version 1650311 (0.0009) [2023-12-27 17:04:44,470][20381] Updated weights for policy 1, policy_version 1648691 (0.0007) [2023-12-27 17:04:44,528][20381] Updated weights for policy 1, policy_version 1648701 (0.0005) [2023-12-27 17:04:44,573][20381] Updated weights for policy 1, policy_version 1648711 (0.0005) [2023-12-27 17:04:44,973][20347] Updated weights for policy 0, policy_version 1650321 (0.0010) [2023-12-27 17:04:45,033][20347] Updated weights for policy 0, policy_version 1650331 (0.0010) [2023-12-27 17:04:45,092][20347] Updated weights for policy 0, policy_version 1650341 (0.0010) [2023-12-27 17:04:45,180][20381] Updated weights for policy 1, policy_version 1648721 (0.0006) [2023-12-27 17:04:45,241][20381] Updated weights for policy 1, policy_version 1648731 (0.0007) [2023-12-27 17:04:45,296][20381] Updated weights for policy 1, policy_version 1648741 (0.0008) [2023-12-27 17:04:45,354][20381] Updated weights for policy 1, policy_version 1648751 (0.0007) [2023-12-27 17:04:45,787][20347] Updated weights for policy 0, policy_version 1650351 (0.0008) [2023-12-27 17:04:45,846][20347] Updated weights for policy 0, policy_version 1650361 (0.0006) [2023-12-27 17:04:45,903][20347] Updated weights for policy 0, policy_version 1650371 (0.0005) [2023-12-27 17:04:46,073][20381] Updated weights for policy 1, policy_version 1648761 (0.0008) [2023-12-27 17:04:46,129][20381] Updated weights for policy 1, policy_version 1648771 (0.0008) [2023-12-27 17:04:46,175][20381] Updated weights for policy 1, policy_version 1648781 (0.0008) [2023-12-27 17:04:46,519][20347] Updated weights for policy 0, policy_version 1650381 (0.0005) [2023-12-27 17:04:46,575][20347] Updated weights for policy 0, policy_version 1650391 (0.0005) [2023-12-27 17:04:46,626][20347] Updated weights for policy 0, policy_version 1650401 (0.0007) [2023-12-27 17:04:46,888][20381] Updated weights for policy 1, policy_version 1648791 (0.0007) [2023-12-27 17:04:46,937][20381] Updated weights for policy 1, policy_version 1648801 (0.0008) [2023-12-27 17:04:46,989][20381] Updated weights for policy 1, policy_version 1648811 (0.0008) [2023-12-27 17:04:47,229][20347] Updated weights for policy 0, policy_version 1650411 (0.0007) [2023-12-27 17:04:47,290][20347] Updated weights for policy 0, policy_version 1650421 (0.0007) [2023-12-27 17:04:47,345][20347] Updated weights for policy 0, policy_version 1650431 (0.0008) [2023-12-27 17:04:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 844726272. Throughput: 0: 10363.6, 1: 10351.1. Samples: 844739172. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:47,363][19170] Avg episode reward: [(0, '980.490'), (1, '824.210')] [2023-12-27 17:04:47,704][20381] Updated weights for policy 1, policy_version 1648821 (0.0007) [2023-12-27 17:04:47,763][20381] Updated weights for policy 1, policy_version 1648831 (0.0005) [2023-12-27 17:04:47,825][20381] Updated weights for policy 1, policy_version 1648841 (0.0005) [2023-12-27 17:04:48,059][20347] Updated weights for policy 0, policy_version 1650441 (0.0008) [2023-12-27 17:04:48,115][20347] Updated weights for policy 0, policy_version 1650451 (0.0009) [2023-12-27 17:04:48,170][20347] Updated weights for policy 0, policy_version 1650461 (0.0007) [2023-12-27 17:04:48,227][20347] Updated weights for policy 0, policy_version 1650471 (0.0007) [2023-12-27 17:04:48,384][20381] Updated weights for policy 1, policy_version 1648851 (0.0007) [2023-12-27 17:04:48,439][20381] Updated weights for policy 1, policy_version 1648861 (0.0007) [2023-12-27 17:04:48,503][20381] Updated weights for policy 1, policy_version 1648871 (0.0005) [2023-12-27 17:04:49,019][20347] Updated weights for policy 0, policy_version 1650481 (0.0008) [2023-12-27 17:04:49,075][20347] Updated weights for policy 0, policy_version 1650491 (0.0008) [2023-12-27 17:04:49,077][20381] Updated weights for policy 1, policy_version 1648881 (0.0005) [2023-12-27 17:04:49,128][20347] Updated weights for policy 0, policy_version 1650501 (0.0006) [2023-12-27 17:04:49,130][20381] Updated weights for policy 1, policy_version 1648891 (0.0005) [2023-12-27 17:04:49,190][20381] Updated weights for policy 1, policy_version 1648901 (0.0008) [2023-12-27 17:04:49,258][20381] Updated weights for policy 1, policy_version 1648911 (0.0007) [2023-12-27 17:04:49,861][20347] Updated weights for policy 0, policy_version 1650511 (0.0010) [2023-12-27 17:04:49,923][20347] Updated weights for policy 0, policy_version 1650521 (0.0007) [2023-12-27 17:04:49,961][20381] Updated weights for policy 1, policy_version 1648921 (0.0007) [2023-12-27 17:04:49,981][20347] Updated weights for policy 0, policy_version 1650531 (0.0006) [2023-12-27 17:04:50,030][20381] Updated weights for policy 1, policy_version 1648931 (0.0008) [2023-12-27 17:04:50,095][20381] Updated weights for policy 1, policy_version 1648941 (0.0008) [2023-12-27 17:04:50,738][20347] Updated weights for policy 0, policy_version 1650541 (0.0007) [2023-12-27 17:04:50,761][20381] Updated weights for policy 1, policy_version 1648951 (0.0009) [2023-12-27 17:04:50,796][20347] Updated weights for policy 0, policy_version 1650551 (0.0007) [2023-12-27 17:04:50,817][20381] Updated weights for policy 1, policy_version 1648961 (0.0007) [2023-12-27 17:04:50,853][20347] Updated weights for policy 0, policy_version 1650561 (0.0008) [2023-12-27 17:04:50,871][20381] Updated weights for policy 1, policy_version 1648971 (0.0007) [2023-12-27 17:04:51,591][20347] Updated weights for policy 0, policy_version 1650571 (0.0008) [2023-12-27 17:04:51,653][20381] Updated weights for policy 1, policy_version 1648981 (0.0009) [2023-12-27 17:04:51,657][20347] Updated weights for policy 0, policy_version 1650581 (0.0008) [2023-12-27 17:04:51,715][20381] Updated weights for policy 1, policy_version 1648991 (0.0008) [2023-12-27 17:04:51,719][20347] Updated weights for policy 0, policy_version 1650591 (0.0007) [2023-12-27 17:04:51,781][20381] Updated weights for policy 1, policy_version 1649001 (0.0007) [2023-12-27 17:04:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 844824576. Throughput: 0: 10328.5, 1: 10327.8. Samples: 844799940. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:52,363][20347] Updated weights for policy 0, policy_version 1650601 (0.0006) [2023-12-27 17:04:52,363][19170] Avg episode reward: [(0, '980.490'), (1, '795.160')] [2023-12-27 17:04:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001649008_422191104.pth... [2023-12-27 17:04:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001647792_421879808.pth [2023-12-27 17:04:52,431][20347] Updated weights for policy 0, policy_version 1650611 (0.0008) [2023-12-27 17:04:52,483][20347] Updated weights for policy 0, policy_version 1650621 (0.0007) [2023-12-27 17:04:52,490][20381] Updated weights for policy 1, policy_version 1649011 (0.0008) [2023-12-27 17:04:52,541][20347] Updated weights for policy 0, policy_version 1650631 (0.0008) [2023-12-27 17:04:52,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001650632_422641664.pth... [2023-12-27 17:04:52,546][20381] Updated weights for policy 1, policy_version 1649021 (0.0006) [2023-12-27 17:04:52,548][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001649416_422330368.pth [2023-12-27 17:04:52,603][20381] Updated weights for policy 1, policy_version 1649031 (0.0007) [2023-12-27 17:04:53,233][20381] Updated weights for policy 1, policy_version 1649041 (0.0007) [2023-12-27 17:04:53,271][20347] Updated weights for policy 0, policy_version 1650641 (0.0007) [2023-12-27 17:04:53,286][20381] Updated weights for policy 1, policy_version 1649051 (0.0005) [2023-12-27 17:04:53,322][20292] KL-divergence is very high: 130.0693 [2023-12-27 17:04:53,322][20347] Updated weights for policy 0, policy_version 1650651 (0.0008) [2023-12-27 17:04:53,345][20381] Updated weights for policy 1, policy_version 1649061 (0.0007) [2023-12-27 17:04:53,366][20292] KL-divergence is very high: 105.3148 [2023-12-27 17:04:53,368][20347] Updated weights for policy 0, policy_version 1650661 (0.0007) [2023-12-27 17:04:53,399][20381] Updated weights for policy 1, policy_version 1649071 (0.0010) [2023-12-27 17:04:53,958][20381] Updated weights for policy 1, policy_version 1649081 (0.0006) [2023-12-27 17:04:54,015][20381] Updated weights for policy 1, policy_version 1649091 (0.0008) [2023-12-27 17:04:54,024][20347] Updated weights for policy 0, policy_version 1650671 (0.0007) [2023-12-27 17:04:54,065][20381] Updated weights for policy 1, policy_version 1649101 (0.0006) [2023-12-27 17:04:54,078][20347] Updated weights for policy 0, policy_version 1650681 (0.0007) [2023-12-27 17:04:54,140][20347] Updated weights for policy 0, policy_version 1650691 (0.0008) [2023-12-27 17:04:54,741][20381] Updated weights for policy 1, policy_version 1649111 (0.0007) [2023-12-27 17:04:54,744][20347] Updated weights for policy 0, policy_version 1650701 (0.0006) [2023-12-27 17:04:54,800][20381] Updated weights for policy 1, policy_version 1649121 (0.0007) [2023-12-27 17:04:54,800][20347] Updated weights for policy 0, policy_version 1650711 (0.0005) [2023-12-27 17:04:54,848][20381] Updated weights for policy 1, policy_version 1649131 (0.0007) [2023-12-27 17:04:54,850][20347] Updated weights for policy 0, policy_version 1650721 (0.0005) [2023-12-27 17:04:55,417][20347] Updated weights for policy 0, policy_version 1650731 (0.0005) [2023-12-27 17:04:55,462][20381] Updated weights for policy 1, policy_version 1649141 (0.0008) [2023-12-27 17:04:55,464][20347] Updated weights for policy 0, policy_version 1650741 (0.0005) [2023-12-27 17:04:55,513][20347] Updated weights for policy 0, policy_version 1650751 (0.0005) [2023-12-27 17:04:55,517][20381] Updated weights for policy 1, policy_version 1649151 (0.0011) [2023-12-27 17:04:55,574][20381] Updated weights for policy 1, policy_version 1649161 (0.0011) [2023-12-27 17:04:56,106][20347] Updated weights for policy 0, policy_version 1650761 (0.0006) [2023-12-27 17:04:56,153][20347] Updated weights for policy 0, policy_version 1650771 (0.0007) [2023-12-27 17:04:56,196][20347] Updated weights for policy 0, policy_version 1650781 (0.0007) [2023-12-27 17:04:56,246][20381] Updated weights for policy 1, policy_version 1649171 (0.0011) [2023-12-27 17:04:56,249][20347] Updated weights for policy 0, policy_version 1650791 (0.0007) [2023-12-27 17:04:56,306][20381] Updated weights for policy 1, policy_version 1649181 (0.0011) [2023-12-27 17:04:56,367][20381] Updated weights for policy 1, policy_version 1649191 (0.0007) [2023-12-27 17:04:56,939][20347] Updated weights for policy 0, policy_version 1650801 (0.0007) [2023-12-27 17:04:56,994][20347] Updated weights for policy 0, policy_version 1650811 (0.0007) [2023-12-27 17:04:56,999][20381] Updated weights for policy 1, policy_version 1649201 (0.0009) [2023-12-27 17:04:57,048][20347] Updated weights for policy 0, policy_version 1650821 (0.0007) [2023-12-27 17:04:57,053][20381] Updated weights for policy 1, policy_version 1649211 (0.0010) [2023-12-27 17:04:57,108][20381] Updated weights for policy 1, policy_version 1649221 (0.0010) [2023-12-27 17:04:57,155][20381] Updated weights for policy 1, policy_version 1649231 (0.0010) [2023-12-27 17:04:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 844939264. Throughput: 0: 10346.6, 1: 10351.2. Samples: 844925564. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:04:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '795.130')] [2023-12-27 17:04:57,724][20347] Updated weights for policy 0, policy_version 1650831 (0.0005) [2023-12-27 17:04:57,786][20347] Updated weights for policy 0, policy_version 1650841 (0.0006) [2023-12-27 17:04:57,842][20347] Updated weights for policy 0, policy_version 1650851 (0.0006) [2023-12-27 17:04:57,859][20381] Updated weights for policy 1, policy_version 1649241 (0.0011) [2023-12-27 17:04:57,914][20381] Updated weights for policy 1, policy_version 1649251 (0.0010) [2023-12-27 17:04:57,977][20381] Updated weights for policy 1, policy_version 1649262 (0.0008) [2023-12-27 17:04:58,518][20347] Updated weights for policy 0, policy_version 1650861 (0.0006) [2023-12-27 17:04:58,587][20347] Updated weights for policy 0, policy_version 1650871 (0.0009) [2023-12-27 17:04:58,655][20347] Updated weights for policy 0, policy_version 1650881 (0.0007) [2023-12-27 17:04:58,731][20381] Updated weights for policy 1, policy_version 1649272 (0.0008) [2023-12-27 17:04:58,799][20381] Updated weights for policy 1, policy_version 1649282 (0.0009) [2023-12-27 17:04:58,859][20381] Updated weights for policy 1, policy_version 1649292 (0.0009) [2023-12-27 17:04:59,443][20347] Updated weights for policy 0, policy_version 1650891 (0.0008) [2023-12-27 17:04:59,499][20347] Updated weights for policy 0, policy_version 1650901 (0.0008) [2023-12-27 17:04:59,558][20347] Updated weights for policy 0, policy_version 1650911 (0.0006) [2023-12-27 17:04:59,634][20381] Updated weights for policy 1, policy_version 1649302 (0.0008) [2023-12-27 17:04:59,696][20381] Updated weights for policy 1, policy_version 1649312 (0.0008) [2023-12-27 17:04:59,755][20381] Updated weights for policy 1, policy_version 1649322 (0.0006) [2023-12-27 17:05:00,248][20347] Updated weights for policy 0, policy_version 1650921 (0.0007) [2023-12-27 17:05:00,306][20347] Updated weights for policy 0, policy_version 1650931 (0.0008) [2023-12-27 17:05:00,364][20347] Updated weights for policy 0, policy_version 1650941 (0.0010) [2023-12-27 17:05:00,409][20381] Updated weights for policy 1, policy_version 1649332 (0.0007) [2023-12-27 17:05:00,420][20347] Updated weights for policy 0, policy_version 1650951 (0.0010) [2023-12-27 17:05:00,467][20381] Updated weights for policy 1, policy_version 1649342 (0.0006) [2023-12-27 17:05:00,515][20381] Updated weights for policy 1, policy_version 1649352 (0.0005) [2023-12-27 17:05:01,008][20347] Updated weights for policy 0, policy_version 1650961 (0.0005) [2023-12-27 17:05:01,072][20347] Updated weights for policy 0, policy_version 1650971 (0.0009) [2023-12-27 17:05:01,127][20347] Updated weights for policy 0, policy_version 1650981 (0.0007) [2023-12-27 17:05:01,179][20381] Updated weights for policy 1, policy_version 1649362 (0.0007) [2023-12-27 17:05:01,233][20381] Updated weights for policy 1, policy_version 1649372 (0.0008) [2023-12-27 17:05:01,284][20381] Updated weights for policy 1, policy_version 1649382 (0.0009) [2023-12-27 17:05:01,343][20381] Updated weights for policy 1, policy_version 1649392 (0.0008) [2023-12-27 17:05:01,800][20347] Updated weights for policy 0, policy_version 1650991 (0.0010) [2023-12-27 17:05:01,855][20347] Updated weights for policy 0, policy_version 1651001 (0.0010) [2023-12-27 17:05:01,914][20347] Updated weights for policy 0, policy_version 1651011 (0.0010) [2023-12-27 17:05:02,101][20381] Updated weights for policy 1, policy_version 1649402 (0.0008) [2023-12-27 17:05:02,153][20381] Updated weights for policy 1, policy_version 1649412 (0.0008) [2023-12-27 17:05:02,217][20381] Updated weights for policy 1, policy_version 1649422 (0.0007) [2023-12-27 17:05:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 845037568. Throughput: 0: 10300.1, 1: 10290.3. Samples: 845044836. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '736.480')] [2023-12-27 17:05:02,543][20347] Updated weights for policy 0, policy_version 1651021 (0.0010) [2023-12-27 17:05:02,605][20347] Updated weights for policy 0, policy_version 1651031 (0.0007) [2023-12-27 17:05:02,666][20347] Updated weights for policy 0, policy_version 1651041 (0.0008) [2023-12-27 17:05:02,907][20381] Updated weights for policy 1, policy_version 1649432 (0.0009) [2023-12-27 17:05:02,970][20381] Updated weights for policy 1, policy_version 1649442 (0.0008) [2023-12-27 17:05:03,031][20381] Updated weights for policy 1, policy_version 1649452 (0.0007) [2023-12-27 17:05:03,318][20347] Updated weights for policy 0, policy_version 1651051 (0.0009) [2023-12-27 17:05:03,381][20347] Updated weights for policy 0, policy_version 1651061 (0.0010) [2023-12-27 17:05:03,431][20347] Updated weights for policy 0, policy_version 1651071 (0.0010) [2023-12-27 17:05:03,726][20381] Updated weights for policy 1, policy_version 1649462 (0.0007) [2023-12-27 17:05:03,779][20381] Updated weights for policy 1, policy_version 1649472 (0.0007) [2023-12-27 17:05:03,829][20381] Updated weights for policy 1, policy_version 1649482 (0.0007) [2023-12-27 17:05:04,111][20347] Updated weights for policy 0, policy_version 1651081 (0.0010) [2023-12-27 17:05:04,168][20347] Updated weights for policy 0, policy_version 1651091 (0.0008) [2023-12-27 17:05:04,230][20347] Updated weights for policy 0, policy_version 1651101 (0.0007) [2023-12-27 17:05:04,294][20347] Updated weights for policy 0, policy_version 1651111 (0.0008) [2023-12-27 17:05:04,542][20381] Updated weights for policy 1, policy_version 1649492 (0.0009) [2023-12-27 17:05:04,603][20381] Updated weights for policy 1, policy_version 1649502 (0.0009) [2023-12-27 17:05:04,644][20381] Updated weights for policy 1, policy_version 1649512 (0.0010) [2023-12-27 17:05:04,969][20347] Updated weights for policy 0, policy_version 1651121 (0.0008) [2023-12-27 17:05:05,028][20347] Updated weights for policy 0, policy_version 1651131 (0.0007) [2023-12-27 17:05:05,087][20347] Updated weights for policy 0, policy_version 1651141 (0.0008) [2023-12-27 17:05:05,242][20381] Updated weights for policy 1, policy_version 1649522 (0.0009) [2023-12-27 17:05:05,297][20381] Updated weights for policy 1, policy_version 1649532 (0.0006) [2023-12-27 17:05:05,341][20381] Updated weights for policy 1, policy_version 1649542 (0.0010) [2023-12-27 17:05:05,392][20381] Updated weights for policy 1, policy_version 1649552 (0.0010) [2023-12-27 17:05:05,714][20347] Updated weights for policy 0, policy_version 1651151 (0.0007) [2023-12-27 17:05:05,780][20347] Updated weights for policy 0, policy_version 1651161 (0.0008) [2023-12-27 17:05:05,844][20347] Updated weights for policy 0, policy_version 1651171 (0.0008) [2023-12-27 17:05:06,088][20381] Updated weights for policy 1, policy_version 1649562 (0.0006) [2023-12-27 17:05:06,149][20381] Updated weights for policy 1, policy_version 1649572 (0.0008) [2023-12-27 17:05:06,216][20381] Updated weights for policy 1, policy_version 1649582 (0.0006) [2023-12-27 17:05:06,467][20347] Updated weights for policy 0, policy_version 1651181 (0.0008) [2023-12-27 17:05:06,529][20347] Updated weights for policy 0, policy_version 1651191 (0.0009) [2023-12-27 17:05:06,591][20347] Updated weights for policy 0, policy_version 1651201 (0.0007) [2023-12-27 17:05:06,798][20381] Updated weights for policy 1, policy_version 1649592 (0.0006) [2023-12-27 17:05:06,859][20381] Updated weights for policy 1, policy_version 1649602 (0.0007) [2023-12-27 17:05:06,909][20381] Updated weights for policy 1, policy_version 1649612 (0.0009) [2023-12-27 17:05:07,276][20347] Updated weights for policy 0, policy_version 1651211 (0.0008) [2023-12-27 17:05:07,323][20347] Updated weights for policy 0, policy_version 1651221 (0.0008) [2023-12-27 17:05:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 845135872. Throughput: 0: 10301.1, 1: 10307.8. Samples: 845106760. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:07,363][19170] Avg episode reward: [(0, '990.250'), (1, '726.780')] [2023-12-27 17:05:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001649616_422346752.pth... [2023-12-27 17:05:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001648432_422043648.pth [2023-12-27 17:05:07,372][20347] Updated weights for policy 0, policy_version 1651231 (0.0007) [2023-12-27 17:05:07,412][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001651240_422797312.pth... [2023-12-27 17:05:07,417][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001650024_422486016.pth [2023-12-27 17:05:07,609][20381] Updated weights for policy 1, policy_version 1649622 (0.0008) [2023-12-27 17:05:07,658][20381] Updated weights for policy 1, policy_version 1649632 (0.0009) [2023-12-27 17:05:07,707][20381] Updated weights for policy 1, policy_version 1649642 (0.0009) [2023-12-27 17:05:07,998][20347] Updated weights for policy 0, policy_version 1651241 (0.0007) [2023-12-27 17:05:08,057][20347] Updated weights for policy 0, policy_version 1651251 (0.0010) [2023-12-27 17:05:08,118][20347] Updated weights for policy 0, policy_version 1651261 (0.0010) [2023-12-27 17:05:08,179][20347] Updated weights for policy 0, policy_version 1651271 (0.0010) [2023-12-27 17:05:08,426][20381] Updated weights for policy 1, policy_version 1649652 (0.0007) [2023-12-27 17:05:08,483][20381] Updated weights for policy 1, policy_version 1649662 (0.0007) [2023-12-27 17:05:08,536][20381] Updated weights for policy 1, policy_version 1649672 (0.0007) [2023-12-27 17:05:08,915][20347] Updated weights for policy 0, policy_version 1651281 (0.0008) [2023-12-27 17:05:08,975][20347] Updated weights for policy 0, policy_version 1651291 (0.0009) [2023-12-27 17:05:09,033][20347] Updated weights for policy 0, policy_version 1651301 (0.0007) [2023-12-27 17:05:09,190][20381] Updated weights for policy 1, policy_version 1649682 (0.0007) [2023-12-27 17:05:09,250][20381] Updated weights for policy 1, policy_version 1649692 (0.0008) [2023-12-27 17:05:09,307][20381] Updated weights for policy 1, policy_version 1649702 (0.0007) [2023-12-27 17:05:09,370][20381] Updated weights for policy 1, policy_version 1649712 (0.0008) [2023-12-27 17:05:09,817][20347] Updated weights for policy 0, policy_version 1651311 (0.0009) [2023-12-27 17:05:09,878][20347] Updated weights for policy 0, policy_version 1651321 (0.0009) [2023-12-27 17:05:09,942][20347] Updated weights for policy 0, policy_version 1651331 (0.0008) [2023-12-27 17:05:10,140][20381] Updated weights for policy 1, policy_version 1649722 (0.0008) [2023-12-27 17:05:10,193][20381] Updated weights for policy 1, policy_version 1649732 (0.0007) [2023-12-27 17:05:10,244][20381] Updated weights for policy 1, policy_version 1649742 (0.0006) [2023-12-27 17:05:10,712][20347] Updated weights for policy 0, policy_version 1651341 (0.0007) [2023-12-27 17:05:10,760][20347] Updated weights for policy 0, policy_version 1651351 (0.0007) [2023-12-27 17:05:10,814][20347] Updated weights for policy 0, policy_version 1651361 (0.0007) [2023-12-27 17:05:10,869][20381] Updated weights for policy 1, policy_version 1649752 (0.0010) [2023-12-27 17:05:10,922][20381] Updated weights for policy 1, policy_version 1649762 (0.0010) [2023-12-27 17:05:10,973][20381] Updated weights for policy 1, policy_version 1649772 (0.0010) [2023-12-27 17:05:11,548][20347] Updated weights for policy 0, policy_version 1651371 (0.0006) [2023-12-27 17:05:11,613][20347] Updated weights for policy 0, policy_version 1651381 (0.0008) [2023-12-27 17:05:11,680][20347] Updated weights for policy 0, policy_version 1651391 (0.0007) [2023-12-27 17:05:11,787][20381] Updated weights for policy 1, policy_version 1649782 (0.0008) [2023-12-27 17:05:11,849][20292] KL-divergence is very high: 121.6288 [2023-12-27 17:05:11,854][20381] Updated weights for policy 1, policy_version 1649792 (0.0006) [2023-12-27 17:05:11,902][20292] KL-divergence is very high: 145.4551 [2023-12-27 17:05:11,923][20381] Updated weights for policy 1, policy_version 1649802 (0.0006) [2023-12-27 17:05:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 845234176. Throughput: 0: 10217.3, 1: 10312.9. Samples: 845227932. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:12,363][19170] Avg episode reward: [(0, '980.470'), (1, '727.420')] [2023-12-27 17:05:12,389][20347] Updated weights for policy 0, policy_version 1651401 (0.0008) [2023-12-27 17:05:12,455][20347] Updated weights for policy 0, policy_version 1651411 (0.0007) [2023-12-27 17:05:12,512][20347] Updated weights for policy 0, policy_version 1651421 (0.0008) [2023-12-27 17:05:12,573][20347] Updated weights for policy 0, policy_version 1651431 (0.0007) [2023-12-27 17:05:12,600][20381] Updated weights for policy 1, policy_version 1649812 (0.0009) [2023-12-27 17:05:12,663][20381] Updated weights for policy 1, policy_version 1649822 (0.0008) [2023-12-27 17:05:12,728][20381] Updated weights for policy 1, policy_version 1649832 (0.0008) [2023-12-27 17:05:13,201][20347] Updated weights for policy 0, policy_version 1651441 (0.0010) [2023-12-27 17:05:13,246][20347] Updated weights for policy 0, policy_version 1651451 (0.0010) [2023-12-27 17:05:13,296][20347] Updated weights for policy 0, policy_version 1651461 (0.0010) [2023-12-27 17:05:13,300][20381] Updated weights for policy 1, policy_version 1649842 (0.0008) [2023-12-27 17:05:13,355][20381] Updated weights for policy 1, policy_version 1649852 (0.0007) [2023-12-27 17:05:13,404][20381] Updated weights for policy 1, policy_version 1649862 (0.0008) [2023-12-27 17:05:13,456][20381] Updated weights for policy 1, policy_version 1649872 (0.0008) [2023-12-27 17:05:14,001][20347] Updated weights for policy 0, policy_version 1651471 (0.0010) [2023-12-27 17:05:14,056][20347] Updated weights for policy 0, policy_version 1651481 (0.0010) [2023-12-27 17:05:14,065][20381] Updated weights for policy 1, policy_version 1649882 (0.0005) [2023-12-27 17:05:14,111][20347] Updated weights for policy 0, policy_version 1651491 (0.0010) [2023-12-27 17:05:14,122][20381] Updated weights for policy 1, policy_version 1649892 (0.0008) [2023-12-27 17:05:14,177][20381] Updated weights for policy 1, policy_version 1649902 (0.0007) [2023-12-27 17:05:14,720][20347] Updated weights for policy 0, policy_version 1651501 (0.0008) [2023-12-27 17:05:14,785][20347] Updated weights for policy 0, policy_version 1651511 (0.0010) [2023-12-27 17:05:14,845][20347] Updated weights for policy 0, policy_version 1651521 (0.0011) [2023-12-27 17:05:14,854][20381] Updated weights for policy 1, policy_version 1649912 (0.0006) [2023-12-27 17:05:14,912][20381] Updated weights for policy 1, policy_version 1649922 (0.0008) [2023-12-27 17:05:14,972][20381] Updated weights for policy 1, policy_version 1649932 (0.0008) [2023-12-27 17:05:15,483][20347] Updated weights for policy 0, policy_version 1651531 (0.0009) [2023-12-27 17:05:15,529][20347] Updated weights for policy 0, policy_version 1651541 (0.0005) [2023-12-27 17:05:15,592][20347] Updated weights for policy 0, policy_version 1651551 (0.0010) [2023-12-27 17:05:15,704][20381] Updated weights for policy 1, policy_version 1649942 (0.0007) [2023-12-27 17:05:15,764][20381] Updated weights for policy 1, policy_version 1649952 (0.0007) [2023-12-27 17:05:15,815][20381] Updated weights for policy 1, policy_version 1649962 (0.0007) [2023-12-27 17:05:16,201][20347] Updated weights for policy 0, policy_version 1651561 (0.0010) [2023-12-27 17:05:16,254][20347] Updated weights for policy 0, policy_version 1651571 (0.0008) [2023-12-27 17:05:16,335][20347] Updated weights for policy 0, policy_version 1651581 (0.0009) [2023-12-27 17:05:16,392][20347] Updated weights for policy 0, policy_version 1651591 (0.0010) [2023-12-27 17:05:16,530][20381] Updated weights for policy 1, policy_version 1649972 (0.0008) [2023-12-27 17:05:16,579][20381] Updated weights for policy 1, policy_version 1649982 (0.0008) [2023-12-27 17:05:16,621][20381] Updated weights for policy 1, policy_version 1649992 (0.0008) [2023-12-27 17:05:16,944][20347] Updated weights for policy 0, policy_version 1651601 (0.0010) [2023-12-27 17:05:16,995][20347] Updated weights for policy 0, policy_version 1651611 (0.0005) [2023-12-27 17:05:17,056][20347] Updated weights for policy 0, policy_version 1651621 (0.0009) [2023-12-27 17:05:17,323][20381] Updated weights for policy 1, policy_version 1650002 (0.0006) [2023-12-27 17:05:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 845340672. Throughput: 0: 10217.8, 1: 10286.4. Samples: 845353276. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:17,363][19170] Avg episode reward: [(0, '970.730'), (1, '717.230')] [2023-12-27 17:05:17,383][20381] Updated weights for policy 1, policy_version 1650012 (0.0008) [2023-12-27 17:05:17,449][20381] Updated weights for policy 1, policy_version 1650022 (0.0008) [2023-12-27 17:05:17,517][20381] Updated weights for policy 1, policy_version 1650032 (0.0008) [2023-12-27 17:05:17,722][20347] Updated weights for policy 0, policy_version 1651631 (0.0008) [2023-12-27 17:05:17,773][20347] Updated weights for policy 0, policy_version 1651641 (0.0008) [2023-12-27 17:05:17,821][20347] Updated weights for policy 0, policy_version 1651651 (0.0006) [2023-12-27 17:05:18,221][20381] Updated weights for policy 1, policy_version 1650042 (0.0008) [2023-12-27 17:05:18,282][20381] Updated weights for policy 1, policy_version 1650052 (0.0008) [2023-12-27 17:05:18,345][20381] Updated weights for policy 1, policy_version 1650062 (0.0008) [2023-12-27 17:05:18,447][20347] Updated weights for policy 0, policy_version 1651661 (0.0008) [2023-12-27 17:05:18,503][20347] Updated weights for policy 0, policy_version 1651671 (0.0008) [2023-12-27 17:05:18,559][20347] Updated weights for policy 0, policy_version 1651681 (0.0008) [2023-12-27 17:05:19,082][20381] Updated weights for policy 1, policy_version 1650072 (0.0007) [2023-12-27 17:05:19,128][20381] Updated weights for policy 1, policy_version 1650082 (0.0008) [2023-12-27 17:05:19,172][20347] Updated weights for policy 0, policy_version 1651691 (0.0007) [2023-12-27 17:05:19,182][20381] Updated weights for policy 1, policy_version 1650092 (0.0009) [2023-12-27 17:05:19,239][20347] Updated weights for policy 0, policy_version 1651701 (0.0007) [2023-12-27 17:05:19,299][20347] Updated weights for policy 0, policy_version 1651711 (0.0007) [2023-12-27 17:05:19,876][20381] Updated weights for policy 1, policy_version 1650102 (0.0008) [2023-12-27 17:05:19,947][20381] Updated weights for policy 1, policy_version 1650112 (0.0008) [2023-12-27 17:05:20,010][20381] Updated weights for policy 1, policy_version 1650122 (0.0008) [2023-12-27 17:05:20,027][20347] Updated weights for policy 0, policy_version 1651721 (0.0007) [2023-12-27 17:05:20,086][20347] Updated weights for policy 0, policy_version 1651731 (0.0008) [2023-12-27 17:05:20,148][20347] Updated weights for policy 0, policy_version 1651741 (0.0007) [2023-12-27 17:05:20,212][20347] Updated weights for policy 0, policy_version 1651751 (0.0008) [2023-12-27 17:05:20,668][20381] Updated weights for policy 1, policy_version 1650132 (0.0007) [2023-12-27 17:05:20,724][20381] Updated weights for policy 1, policy_version 1650142 (0.0008) [2023-12-27 17:05:20,781][20381] Updated weights for policy 1, policy_version 1650152 (0.0008) [2023-12-27 17:05:20,910][20347] Updated weights for policy 0, policy_version 1651761 (0.0007) [2023-12-27 17:05:20,970][20347] Updated weights for policy 0, policy_version 1651771 (0.0006) [2023-12-27 17:05:21,033][20347] Updated weights for policy 0, policy_version 1651781 (0.0008) [2023-12-27 17:05:21,556][20381] Updated weights for policy 1, policy_version 1650162 (0.0008) [2023-12-27 17:05:21,619][20381] Updated weights for policy 1, policy_version 1650172 (0.0006) [2023-12-27 17:05:21,679][20381] Updated weights for policy 1, policy_version 1650182 (0.0008) [2023-12-27 17:05:21,753][20381] Updated weights for policy 1, policy_version 1650192 (0.0009) [2023-12-27 17:05:21,784][20347] Updated weights for policy 0, policy_version 1651791 (0.0008) [2023-12-27 17:05:21,838][20347] Updated weights for policy 0, policy_version 1651801 (0.0006) [2023-12-27 17:05:21,884][20347] Updated weights for policy 0, policy_version 1651811 (0.0007) [2023-12-27 17:05:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 845438976. Throughput: 0: 10227.7, 1: 10199.1. Samples: 845413224. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '726.780')] [2023-12-27 17:05:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001650192_422494208.pth... [2023-12-27 17:05:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001651816_422944768.pth... [2023-12-27 17:05:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001649008_422191104.pth [2023-12-27 17:05:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001650632_422641664.pth [2023-12-27 17:05:22,470][20381] Updated weights for policy 1, policy_version 1650202 (0.0007) [2023-12-27 17:05:22,529][20381] Updated weights for policy 1, policy_version 1650212 (0.0007) [2023-12-27 17:05:22,591][20347] Updated weights for policy 0, policy_version 1651821 (0.0009) [2023-12-27 17:05:22,593][20381] Updated weights for policy 1, policy_version 1650222 (0.0007) [2023-12-27 17:05:22,659][20347] Updated weights for policy 0, policy_version 1651831 (0.0010) [2023-12-27 17:05:22,722][20347] Updated weights for policy 0, policy_version 1651841 (0.0008) [2023-12-27 17:05:23,244][20381] Updated weights for policy 1, policy_version 1650232 (0.0007) [2023-12-27 17:05:23,294][20381] Updated weights for policy 1, policy_version 1650242 (0.0007) [2023-12-27 17:05:23,342][20381] Updated weights for policy 1, policy_version 1650252 (0.0007) [2023-12-27 17:05:23,386][20347] Updated weights for policy 0, policy_version 1651851 (0.0011) [2023-12-27 17:05:23,431][20347] Updated weights for policy 0, policy_version 1651861 (0.0010) [2023-12-27 17:05:23,482][20347] Updated weights for policy 0, policy_version 1651871 (0.0010) [2023-12-27 17:05:24,051][20381] Updated weights for policy 1, policy_version 1650262 (0.0007) [2023-12-27 17:05:24,099][20381] Updated weights for policy 1, policy_version 1650272 (0.0008) [2023-12-27 17:05:24,147][20381] Updated weights for policy 1, policy_version 1650282 (0.0007) [2023-12-27 17:05:24,189][20347] Updated weights for policy 0, policy_version 1651881 (0.0010) [2023-12-27 17:05:24,245][20347] Updated weights for policy 0, policy_version 1651891 (0.0009) [2023-12-27 17:05:24,303][20347] Updated weights for policy 0, policy_version 1651901 (0.0010) [2023-12-27 17:05:24,362][20347] Updated weights for policy 0, policy_version 1651911 (0.0010) [2023-12-27 17:05:24,871][20381] Updated weights for policy 1, policy_version 1650292 (0.0007) [2023-12-27 17:05:24,921][20381] Updated weights for policy 1, policy_version 1650302 (0.0007) [2023-12-27 17:05:24,976][20381] Updated weights for policy 1, policy_version 1650312 (0.0006) [2023-12-27 17:05:25,019][20347] Updated weights for policy 0, policy_version 1651921 (0.0008) [2023-12-27 17:05:25,064][20347] Updated weights for policy 0, policy_version 1651931 (0.0005) [2023-12-27 17:05:25,110][20347] Updated weights for policy 0, policy_version 1651941 (0.0005) [2023-12-27 17:05:25,712][20347] Updated weights for policy 0, policy_version 1651951 (0.0005) [2023-12-27 17:05:25,712][20381] Updated weights for policy 1, policy_version 1650322 (0.0007) [2023-12-27 17:05:25,766][20381] Updated weights for policy 1, policy_version 1650332 (0.0007) [2023-12-27 17:05:25,769][20347] Updated weights for policy 0, policy_version 1651961 (0.0006) [2023-12-27 17:05:25,822][20381] Updated weights for policy 1, policy_version 1650342 (0.0010) [2023-12-27 17:05:25,824][20347] Updated weights for policy 0, policy_version 1651971 (0.0006) [2023-12-27 17:05:25,880][20381] Updated weights for policy 1, policy_version 1650352 (0.0009) [2023-12-27 17:05:26,489][20347] Updated weights for policy 0, policy_version 1651981 (0.0005) [2023-12-27 17:05:26,510][20381] Updated weights for policy 1, policy_version 1650362 (0.0007) [2023-12-27 17:05:26,536][20347] Updated weights for policy 0, policy_version 1651991 (0.0007) [2023-12-27 17:05:26,563][20381] Updated weights for policy 1, policy_version 1650372 (0.0007) [2023-12-27 17:05:26,584][20347] Updated weights for policy 0, policy_version 1652001 (0.0005) [2023-12-27 17:05:26,615][20381] Updated weights for policy 1, policy_version 1650382 (0.0007) [2023-12-27 17:05:27,234][20347] Updated weights for policy 0, policy_version 1652011 (0.0006) [2023-12-27 17:05:27,290][20347] Updated weights for policy 0, policy_version 1652021 (0.0005) [2023-12-27 17:05:27,299][20381] Updated weights for policy 1, policy_version 1650392 (0.0008) [2023-12-27 17:05:27,323][20291] KL-divergence is very high: 280.6129 [2023-12-27 17:05:27,346][20347] Updated weights for policy 0, policy_version 1652031 (0.0007) [2023-12-27 17:05:27,352][20381] Updated weights for policy 1, policy_version 1650402 (0.0005) [2023-12-27 17:05:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 845537280. Throughput: 0: 10221.6, 1: 10120.6. Samples: 845533760. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:27,363][19170] Avg episode reward: [(0, '980.450'), (1, '739.020')] [2023-12-27 17:05:27,363][20291] KL-divergence is very high: 309.4579 [2023-12-27 17:05:27,406][20381] Updated weights for policy 1, policy_version 1650412 (0.0005) [2023-12-27 17:05:28,002][20347] Updated weights for policy 0, policy_version 1652041 (0.0007) [2023-12-27 17:05:28,030][20381] Updated weights for policy 1, policy_version 1650422 (0.0006) [2023-12-27 17:05:28,054][20347] Updated weights for policy 0, policy_version 1652051 (0.0008) [2023-12-27 17:05:28,085][20381] Updated weights for policy 1, policy_version 1650432 (0.0005) [2023-12-27 17:05:28,112][20347] Updated weights for policy 0, policy_version 1652061 (0.0008) [2023-12-27 17:05:28,135][20381] Updated weights for policy 1, policy_version 1650442 (0.0008) [2023-12-27 17:05:28,166][20347] Updated weights for policy 0, policy_version 1652071 (0.0007) [2023-12-27 17:05:28,853][20381] Updated weights for policy 1, policy_version 1650452 (0.0006) [2023-12-27 17:05:28,873][20347] Updated weights for policy 0, policy_version 1652081 (0.0010) [2023-12-27 17:05:28,905][20381] Updated weights for policy 1, policy_version 1650462 (0.0005) [2023-12-27 17:05:28,928][20347] Updated weights for policy 0, policy_version 1652091 (0.0010) [2023-12-27 17:05:28,957][20381] Updated weights for policy 1, policy_version 1650472 (0.0006) [2023-12-27 17:05:28,983][20347] Updated weights for policy 0, policy_version 1652101 (0.0008) [2023-12-27 17:05:29,665][20347] Updated weights for policy 0, policy_version 1652111 (0.0007) [2023-12-27 17:05:29,674][20381] Updated weights for policy 1, policy_version 1650482 (0.0008) [2023-12-27 17:05:29,714][20347] Updated weights for policy 0, policy_version 1652121 (0.0005) [2023-12-27 17:05:29,723][20381] Updated weights for policy 1, policy_version 1650492 (0.0008) [2023-12-27 17:05:29,764][20347] Updated weights for policy 0, policy_version 1652131 (0.0005) [2023-12-27 17:05:29,770][20381] Updated weights for policy 1, policy_version 1650502 (0.0007) [2023-12-27 17:05:29,826][20381] Updated weights for policy 1, policy_version 1650512 (0.0006) [2023-12-27 17:05:30,399][20347] Updated weights for policy 0, policy_version 1652141 (0.0007) [2023-12-27 17:05:30,459][20347] Updated weights for policy 0, policy_version 1652151 (0.0007) [2023-12-27 17:05:30,522][20347] Updated weights for policy 0, policy_version 1652161 (0.0007) [2023-12-27 17:05:30,557][20381] Updated weights for policy 1, policy_version 1650522 (0.0006) [2023-12-27 17:05:30,613][20381] Updated weights for policy 1, policy_version 1650532 (0.0005) [2023-12-27 17:05:30,672][20381] Updated weights for policy 1, policy_version 1650542 (0.0006) [2023-12-27 17:05:31,176][20347] Updated weights for policy 0, policy_version 1652171 (0.0008) [2023-12-27 17:05:31,235][20347] Updated weights for policy 0, policy_version 1652181 (0.0006) [2023-12-27 17:05:31,292][20381] Updated weights for policy 1, policy_version 1650552 (0.0006) [2023-12-27 17:05:31,301][20347] Updated weights for policy 0, policy_version 1652191 (0.0008) [2023-12-27 17:05:31,349][20381] Updated weights for policy 1, policy_version 1650562 (0.0006) [2023-12-27 17:05:31,412][20381] Updated weights for policy 1, policy_version 1650572 (0.0007) [2023-12-27 17:05:32,017][20347] Updated weights for policy 0, policy_version 1652201 (0.0007) [2023-12-27 17:05:32,024][20381] Updated weights for policy 1, policy_version 1650582 (0.0007) [2023-12-27 17:05:32,074][20347] Updated weights for policy 0, policy_version 1652211 (0.0005) [2023-12-27 17:05:32,084][20381] Updated weights for policy 1, policy_version 1650592 (0.0008) [2023-12-27 17:05:32,126][20347] Updated weights for policy 0, policy_version 1652221 (0.0006) [2023-12-27 17:05:32,140][20381] Updated weights for policy 1, policy_version 1650602 (0.0008) [2023-12-27 17:05:32,177][20347] Updated weights for policy 0, policy_version 1652231 (0.0007) [2023-12-27 17:05:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 845651968. Throughput: 0: 10237.5, 1: 10175.0. Samples: 845657736. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:32,363][19170] Avg episode reward: [(0, '961.030'), (1, '756.040')] [2023-12-27 17:05:32,837][20347] Updated weights for policy 0, policy_version 1652241 (0.0007) [2023-12-27 17:05:32,864][20381] Updated weights for policy 1, policy_version 1650612 (0.0007) [2023-12-27 17:05:32,889][20347] Updated weights for policy 0, policy_version 1652251 (0.0005) [2023-12-27 17:05:32,937][20347] Updated weights for policy 0, policy_version 1652261 (0.0005) [2023-12-27 17:05:32,940][20381] Updated weights for policy 1, policy_version 1650622 (0.0007) [2023-12-27 17:05:32,986][20381] Updated weights for policy 1, policy_version 1650632 (0.0008) [2023-12-27 17:05:33,479][20347] Updated weights for policy 0, policy_version 1652271 (0.0005) [2023-12-27 17:05:33,536][20347] Updated weights for policy 0, policy_version 1652281 (0.0005) [2023-12-27 17:05:33,586][20347] Updated weights for policy 0, policy_version 1652291 (0.0005) [2023-12-27 17:05:33,695][20381] Updated weights for policy 1, policy_version 1650642 (0.0008) [2023-12-27 17:05:33,746][20381] Updated weights for policy 1, policy_version 1650652 (0.0008) [2023-12-27 17:05:33,801][20381] Updated weights for policy 1, policy_version 1650662 (0.0008) [2023-12-27 17:05:33,855][20381] Updated weights for policy 1, policy_version 1650672 (0.0008) [2023-12-27 17:05:34,252][20347] Updated weights for policy 0, policy_version 1652301 (0.0006) [2023-12-27 17:05:34,309][20347] Updated weights for policy 0, policy_version 1652311 (0.0008) [2023-12-27 17:05:34,370][20347] Updated weights for policy 0, policy_version 1652321 (0.0008) [2023-12-27 17:05:34,518][20381] Updated weights for policy 1, policy_version 1650682 (0.0008) [2023-12-27 17:05:34,571][20381] Updated weights for policy 1, policy_version 1650692 (0.0008) [2023-12-27 17:05:34,621][20381] Updated weights for policy 1, policy_version 1650702 (0.0008) [2023-12-27 17:05:35,076][20347] Updated weights for policy 0, policy_version 1652331 (0.0008) [2023-12-27 17:05:35,128][20347] Updated weights for policy 0, policy_version 1652341 (0.0007) [2023-12-27 17:05:35,171][20347] Updated weights for policy 0, policy_version 1652351 (0.0007) [2023-12-27 17:05:35,319][20381] Updated weights for policy 1, policy_version 1650712 (0.0010) [2023-12-27 17:05:35,379][20381] Updated weights for policy 1, policy_version 1650722 (0.0006) [2023-12-27 17:05:35,440][20381] Updated weights for policy 1, policy_version 1650732 (0.0009) [2023-12-27 17:05:35,773][20347] Updated weights for policy 0, policy_version 1652361 (0.0007) [2023-12-27 17:05:35,833][20347] Updated weights for policy 0, policy_version 1652371 (0.0005) [2023-12-27 17:05:35,890][20347] Updated weights for policy 0, policy_version 1652381 (0.0008) [2023-12-27 17:05:35,941][20347] Updated weights for policy 0, policy_version 1652391 (0.0007) [2023-12-27 17:05:36,080][20381] Updated weights for policy 1, policy_version 1650742 (0.0009) [2023-12-27 17:05:36,140][20381] Updated weights for policy 1, policy_version 1650752 (0.0009) [2023-12-27 17:05:36,195][20381] Updated weights for policy 1, policy_version 1650762 (0.0009) [2023-12-27 17:05:36,626][20347] Updated weights for policy 0, policy_version 1652401 (0.0006) [2023-12-27 17:05:36,686][20347] Updated weights for policy 0, policy_version 1652411 (0.0007) [2023-12-27 17:05:36,746][20347] Updated weights for policy 0, policy_version 1652421 (0.0008) [2023-12-27 17:05:36,841][20381] Updated weights for policy 1, policy_version 1650772 (0.0009) [2023-12-27 17:05:36,900][20381] Updated weights for policy 1, policy_version 1650782 (0.0011) [2023-12-27 17:05:36,959][20381] Updated weights for policy 1, policy_version 1650792 (0.0011) [2023-12-27 17:05:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 845750272. Throughput: 0: 10327.0, 1: 10130.9. Samples: 845720540. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:37,363][19170] Avg episode reward: [(0, '970.830'), (1, '775.530')] [2023-12-27 17:05:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001650800_422649856.pth... [2023-12-27 17:05:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001649616_422346752.pth [2023-12-27 17:05:37,377][20347] Updated weights for policy 0, policy_version 1652431 (0.0007) [2023-12-27 17:05:37,426][20347] Updated weights for policy 0, policy_version 1652441 (0.0008) [2023-12-27 17:05:37,478][20347] Updated weights for policy 0, policy_version 1652451 (0.0006) [2023-12-27 17:05:37,502][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001652456_423108608.pth... [2023-12-27 17:05:37,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001651240_422797312.pth [2023-12-27 17:05:37,650][20381] Updated weights for policy 1, policy_version 1650802 (0.0010) [2023-12-27 17:05:37,699][20381] Updated weights for policy 1, policy_version 1650812 (0.0010) [2023-12-27 17:05:37,747][20381] Updated weights for policy 1, policy_version 1650822 (0.0009) [2023-12-27 17:05:37,800][20381] Updated weights for policy 1, policy_version 1650832 (0.0008) [2023-12-27 17:05:38,077][20347] Updated weights for policy 0, policy_version 1652461 (0.0007) [2023-12-27 17:05:38,125][20347] Updated weights for policy 0, policy_version 1652471 (0.0008) [2023-12-27 17:05:38,175][20347] Updated weights for policy 0, policy_version 1652481 (0.0008) [2023-12-27 17:05:38,518][20381] Updated weights for policy 1, policy_version 1650842 (0.0007) [2023-12-27 17:05:38,578][20381] Updated weights for policy 1, policy_version 1650852 (0.0008) [2023-12-27 17:05:38,640][20381] Updated weights for policy 1, policy_version 1650862 (0.0008) [2023-12-27 17:05:38,815][20347] Updated weights for policy 0, policy_version 1652491 (0.0008) [2023-12-27 17:05:38,866][20347] Updated weights for policy 0, policy_version 1652501 (0.0007) [2023-12-27 17:05:38,924][20347] Updated weights for policy 0, policy_version 1652511 (0.0008) [2023-12-27 17:05:39,371][20381] Updated weights for policy 1, policy_version 1650872 (0.0008) [2023-12-27 17:05:39,440][20381] Updated weights for policy 1, policy_version 1650882 (0.0009) [2023-12-27 17:05:39,506][20381] Updated weights for policy 1, policy_version 1650892 (0.0008) [2023-12-27 17:05:39,657][20347] Updated weights for policy 0, policy_version 1652521 (0.0007) [2023-12-27 17:05:39,726][20347] Updated weights for policy 0, policy_version 1652531 (0.0007) [2023-12-27 17:05:39,791][20347] Updated weights for policy 0, policy_version 1652541 (0.0007) [2023-12-27 17:05:39,860][20347] Updated weights for policy 0, policy_version 1652551 (0.0008) [2023-12-27 17:05:40,182][20381] Updated weights for policy 1, policy_version 1650902 (0.0009) [2023-12-27 17:05:40,244][20381] Updated weights for policy 1, policy_version 1650912 (0.0006) [2023-12-27 17:05:40,305][20381] Updated weights for policy 1, policy_version 1650922 (0.0006) [2023-12-27 17:05:40,464][20347] Updated weights for policy 0, policy_version 1652561 (0.0007) [2023-12-27 17:05:40,520][20347] Updated weights for policy 0, policy_version 1652571 (0.0008) [2023-12-27 17:05:40,570][20347] Updated weights for policy 0, policy_version 1652581 (0.0007) [2023-12-27 17:05:40,953][20381] Updated weights for policy 1, policy_version 1650932 (0.0007) [2023-12-27 17:05:41,011][20381] Updated weights for policy 1, policy_version 1650942 (0.0009) [2023-12-27 17:05:41,074][20381] Updated weights for policy 1, policy_version 1650952 (0.0008) [2023-12-27 17:05:41,264][20347] Updated weights for policy 0, policy_version 1652591 (0.0009) [2023-12-27 17:05:41,324][20347] Updated weights for policy 0, policy_version 1652601 (0.0006) [2023-12-27 17:05:41,385][20347] Updated weights for policy 0, policy_version 1652611 (0.0007) [2023-12-27 17:05:41,892][20381] Updated weights for policy 1, policy_version 1650962 (0.0006) [2023-12-27 17:05:41,949][20381] Updated weights for policy 1, policy_version 1650972 (0.0008) [2023-12-27 17:05:42,009][20381] Updated weights for policy 1, policy_version 1650982 (0.0008) [2023-12-27 17:05:42,061][20381] Updated weights for policy 1, policy_version 1650992 (0.0005) [2023-12-27 17:05:42,096][20347] Updated weights for policy 0, policy_version 1652621 (0.0008) [2023-12-27 17:05:42,153][20347] Updated weights for policy 0, policy_version 1652631 (0.0008) [2023-12-27 17:05:42,202][20347] Updated weights for policy 0, policy_version 1652641 (0.0009) [2023-12-27 17:05:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 845856768. Throughput: 0: 10340.2, 1: 10074.1. Samples: 845844208. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:42,363][19170] Avg episode reward: [(0, '990.250'), (1, '804.720')] [2023-12-27 17:05:42,722][20381] Updated weights for policy 1, policy_version 1651002 (0.0008) [2023-12-27 17:05:42,780][20381] Updated weights for policy 1, policy_version 1651012 (0.0008) [2023-12-27 17:05:42,836][20381] Updated weights for policy 1, policy_version 1651022 (0.0007) [2023-12-27 17:05:42,935][20347] Updated weights for policy 0, policy_version 1652651 (0.0009) [2023-12-27 17:05:42,981][20347] Updated weights for policy 0, policy_version 1652661 (0.0008) [2023-12-27 17:05:43,035][20347] Updated weights for policy 0, policy_version 1652671 (0.0007) [2023-12-27 17:05:43,560][20381] Updated weights for policy 1, policy_version 1651032 (0.0007) [2023-12-27 17:05:43,623][20381] Updated weights for policy 1, policy_version 1651042 (0.0008) [2023-12-27 17:05:43,652][20347] Updated weights for policy 0, policy_version 1652681 (0.0005) [2023-12-27 17:05:43,680][20381] Updated weights for policy 1, policy_version 1651052 (0.0008) [2023-12-27 17:05:43,700][20347] Updated weights for policy 0, policy_version 1652691 (0.0005) [2023-12-27 17:05:43,752][20347] Updated weights for policy 0, policy_version 1652701 (0.0008) [2023-12-27 17:05:43,805][20347] Updated weights for policy 0, policy_version 1652711 (0.0007) [2023-12-27 17:05:44,359][20381] Updated weights for policy 1, policy_version 1651062 (0.0007) [2023-12-27 17:05:44,412][20381] Updated weights for policy 1, policy_version 1651072 (0.0007) [2023-12-27 17:05:44,461][20347] Updated weights for policy 0, policy_version 1652721 (0.0006) [2023-12-27 17:05:44,463][20381] Updated weights for policy 1, policy_version 1651082 (0.0006) [2023-12-27 17:05:44,517][20347] Updated weights for policy 0, policy_version 1652731 (0.0006) [2023-12-27 17:05:44,573][20347] Updated weights for policy 0, policy_version 1652741 (0.0007) [2023-12-27 17:05:45,163][20381] Updated weights for policy 1, policy_version 1651092 (0.0009) [2023-12-27 17:05:45,219][20381] Updated weights for policy 1, policy_version 1651102 (0.0011) [2023-12-27 17:05:45,231][20347] Updated weights for policy 0, policy_version 1652751 (0.0010) [2023-12-27 17:05:45,271][20381] Updated weights for policy 1, policy_version 1651112 (0.0010) [2023-12-27 17:05:45,286][20347] Updated weights for policy 0, policy_version 1652761 (0.0010) [2023-12-27 17:05:45,338][20347] Updated weights for policy 0, policy_version 1652771 (0.0010) [2023-12-27 17:05:45,949][20347] Updated weights for policy 0, policy_version 1652781 (0.0008) [2023-12-27 17:05:45,961][20381] Updated weights for policy 1, policy_version 1651122 (0.0007) [2023-12-27 17:05:45,967][20291] KL-divergence is very high: 250.3732 [2023-12-27 17:05:46,004][20347] Updated weights for policy 0, policy_version 1652791 (0.0007) [2023-12-27 17:05:46,010][20381] Updated weights for policy 1, policy_version 1651132 (0.0011) [2023-12-27 17:05:46,012][20291] KL-divergence is very high: 387.6400 [2023-12-27 17:05:46,060][20291] KL-divergence is very high: 336.3841 [2023-12-27 17:05:46,066][20381] Updated weights for policy 1, policy_version 1651142 (0.0011) [2023-12-27 17:05:46,067][20347] Updated weights for policy 0, policy_version 1652801 (0.0005) [2023-12-27 17:05:46,125][20381] Updated weights for policy 1, policy_version 1651152 (0.0011) [2023-12-27 17:05:46,627][20347] Updated weights for policy 0, policy_version 1652811 (0.0005) [2023-12-27 17:05:46,682][20347] Updated weights for policy 0, policy_version 1652821 (0.0005) [2023-12-27 17:05:46,733][20347] Updated weights for policy 0, policy_version 1652831 (0.0005) [2023-12-27 17:05:46,832][20381] Updated weights for policy 1, policy_version 1651162 (0.0009) [2023-12-27 17:05:46,886][20381] Updated weights for policy 1, policy_version 1651172 (0.0005) [2023-12-27 17:05:46,952][20381] Updated weights for policy 1, policy_version 1651182 (0.0005) [2023-12-27 17:05:47,254][20347] Updated weights for policy 0, policy_version 1652841 (0.0006) [2023-12-27 17:05:47,304][20347] Updated weights for policy 0, policy_version 1652851 (0.0006) [2023-12-27 17:05:47,355][20347] Updated weights for policy 0, policy_version 1652861 (0.0010) [2023-12-27 17:05:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 845955072. Throughput: 0: 10434.0, 1: 10102.0. Samples: 845968960. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:47,363][19170] Avg episode reward: [(0, '990.250'), (1, '814.590')] [2023-12-27 17:05:47,403][20347] Updated weights for policy 0, policy_version 1652871 (0.0010) [2023-12-27 17:05:47,494][20381] Updated weights for policy 1, policy_version 1651192 (0.0009) [2023-12-27 17:05:47,538][20381] Updated weights for policy 1, policy_version 1651202 (0.0010) [2023-12-27 17:05:47,593][20381] Updated weights for policy 1, policy_version 1651212 (0.0010) [2023-12-27 17:05:48,047][20347] Updated weights for policy 0, policy_version 1652881 (0.0010) [2023-12-27 17:05:48,110][20347] Updated weights for policy 0, policy_version 1652891 (0.0011) [2023-12-27 17:05:48,170][20347] Updated weights for policy 0, policy_version 1652901 (0.0011) [2023-12-27 17:05:48,302][20381] Updated weights for policy 1, policy_version 1651222 (0.0011) [2023-12-27 17:05:48,372][20381] Updated weights for policy 1, policy_version 1651232 (0.0009) [2023-12-27 17:05:48,431][20381] Updated weights for policy 1, policy_version 1651242 (0.0008) [2023-12-27 17:05:48,854][20347] Updated weights for policy 0, policy_version 1652911 (0.0008) [2023-12-27 17:05:48,917][20347] Updated weights for policy 0, policy_version 1652921 (0.0007) [2023-12-27 17:05:48,966][20347] Updated weights for policy 0, policy_version 1652931 (0.0007) [2023-12-27 17:05:49,118][20381] Updated weights for policy 1, policy_version 1651252 (0.0008) [2023-12-27 17:05:49,170][20381] Updated weights for policy 1, policy_version 1651262 (0.0006) [2023-12-27 17:05:49,236][20381] Updated weights for policy 1, policy_version 1651272 (0.0007) [2023-12-27 17:05:49,731][20347] Updated weights for policy 0, policy_version 1652941 (0.0009) [2023-12-27 17:05:49,793][20347] Updated weights for policy 0, policy_version 1652951 (0.0011) [2023-12-27 17:05:49,859][20347] Updated weights for policy 0, policy_version 1652961 (0.0013) [2023-12-27 17:05:49,930][20381] Updated weights for policy 1, policy_version 1651282 (0.0008) [2023-12-27 17:05:49,987][20381] Updated weights for policy 1, policy_version 1651292 (0.0007) [2023-12-27 17:05:50,046][20381] Updated weights for policy 1, policy_version 1651302 (0.0008) [2023-12-27 17:05:50,095][20381] Updated weights for policy 1, policy_version 1651312 (0.0007) [2023-12-27 17:05:50,637][20347] Updated weights for policy 0, policy_version 1652971 (0.0008) [2023-12-27 17:05:50,689][20291] KL-divergence is very high: 276.7860 [2023-12-27 17:05:50,701][20347] Updated weights for policy 0, policy_version 1652981 (0.0007) [2023-12-27 17:05:50,722][20381] Updated weights for policy 1, policy_version 1651322 (0.0007) [2023-12-27 17:05:50,738][20291] KL-divergence is very high: 366.8559 [2023-12-27 17:05:50,761][20347] Updated weights for policy 0, policy_version 1652991 (0.0005) [2023-12-27 17:05:50,785][20381] Updated weights for policy 1, policy_version 1651332 (0.0005) [2023-12-27 17:05:50,785][20291] KL-divergence is very high: 212.0145 [2023-12-27 17:05:50,845][20381] Updated weights for policy 1, policy_version 1651342 (0.0009) [2023-12-27 17:05:51,385][20347] Updated weights for policy 0, policy_version 1653001 (0.0007) [2023-12-27 17:05:51,451][20347] Updated weights for policy 0, policy_version 1653011 (0.0007) [2023-12-27 17:05:51,501][20347] Updated weights for policy 0, policy_version 1653021 (0.0007) [2023-12-27 17:05:51,520][20381] Updated weights for policy 1, policy_version 1651352 (0.0011) [2023-12-27 17:05:51,549][20347] Updated weights for policy 0, policy_version 1653031 (0.0007) [2023-12-27 17:05:51,578][20381] Updated weights for policy 1, policy_version 1651362 (0.0010) [2023-12-27 17:05:51,642][20381] Updated weights for policy 1, policy_version 1651372 (0.0011) [2023-12-27 17:05:52,298][20381] Updated weights for policy 1, policy_version 1651382 (0.0011) [2023-12-27 17:05:52,354][20381] Updated weights for policy 1, policy_version 1651392 (0.0010) [2023-12-27 17:05:52,357][20347] Updated weights for policy 0, policy_version 1653041 (0.0006) [2023-12-27 17:05:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 846053376. Throughput: 0: 10410.7, 1: 10099.6. Samples: 846029720. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:52,363][19170] Avg episode reward: [(0, '980.450'), (1, '785.210')] [2023-12-27 17:05:52,421][20347] Updated weights for policy 0, policy_version 1653051 (0.0007) [2023-12-27 17:05:52,424][20381] Updated weights for policy 1, policy_version 1651402 (0.0009) [2023-12-27 17:05:52,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001651408_422805504.pth... [2023-12-27 17:05:52,461][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001650192_422494208.pth [2023-12-27 17:05:52,481][20347] Updated weights for policy 0, policy_version 1653061 (0.0008) [2023-12-27 17:05:52,495][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001653064_423264256.pth... [2023-12-27 17:05:52,499][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001651816_422944768.pth [2023-12-27 17:05:53,091][20381] Updated weights for policy 1, policy_version 1651412 (0.0011) [2023-12-27 17:05:53,137][20381] Updated weights for policy 1, policy_version 1651422 (0.0010) [2023-12-27 17:05:53,180][20381] Updated weights for policy 1, policy_version 1651432 (0.0010) [2023-12-27 17:05:53,191][20347] Updated weights for policy 0, policy_version 1653071 (0.0008) [2023-12-27 17:05:53,246][20347] Updated weights for policy 0, policy_version 1653081 (0.0007) [2023-12-27 17:05:53,289][20347] Updated weights for policy 0, policy_version 1653091 (0.0007) [2023-12-27 17:05:53,787][20381] Updated weights for policy 1, policy_version 1651442 (0.0010) [2023-12-27 17:05:53,842][20381] Updated weights for policy 1, policy_version 1651452 (0.0008) [2023-12-27 17:05:53,896][20381] Updated weights for policy 1, policy_version 1651462 (0.0006) [2023-12-27 17:05:53,943][20381] Updated weights for policy 1, policy_version 1651472 (0.0007) [2023-12-27 17:05:53,977][20347] Updated weights for policy 0, policy_version 1653101 (0.0007) [2023-12-27 17:05:54,034][20347] Updated weights for policy 0, policy_version 1653111 (0.0007) [2023-12-27 17:05:54,077][20347] Updated weights for policy 0, policy_version 1653121 (0.0007) [2023-12-27 17:05:54,598][20381] Updated weights for policy 1, policy_version 1651482 (0.0010) [2023-12-27 17:05:54,651][20381] Updated weights for policy 1, policy_version 1651492 (0.0007) [2023-12-27 17:05:54,704][20381] Updated weights for policy 1, policy_version 1651502 (0.0005) [2023-12-27 17:05:54,818][20347] Updated weights for policy 0, policy_version 1653131 (0.0008) [2023-12-27 17:05:54,862][20347] Updated weights for policy 0, policy_version 1653141 (0.0008) [2023-12-27 17:05:54,908][20347] Updated weights for policy 0, policy_version 1653151 (0.0007) [2023-12-27 17:05:55,309][20381] Updated weights for policy 1, policy_version 1651512 (0.0009) [2023-12-27 17:05:55,357][20381] Updated weights for policy 1, policy_version 1651522 (0.0010) [2023-12-27 17:05:55,417][20381] Updated weights for policy 1, policy_version 1651532 (0.0010) [2023-12-27 17:05:55,642][20347] Updated weights for policy 0, policy_version 1653161 (0.0007) [2023-12-27 17:05:55,700][20347] Updated weights for policy 0, policy_version 1653171 (0.0007) [2023-12-27 17:05:55,764][20347] Updated weights for policy 0, policy_version 1653181 (0.0007) [2023-12-27 17:05:55,821][20347] Updated weights for policy 0, policy_version 1653191 (0.0007) [2023-12-27 17:05:56,077][20381] Updated weights for policy 1, policy_version 1651542 (0.0011) [2023-12-27 17:05:56,138][20381] Updated weights for policy 1, policy_version 1651552 (0.0011) [2023-12-27 17:05:56,196][20381] Updated weights for policy 1, policy_version 1651562 (0.0006) [2023-12-27 17:05:56,521][20347] Updated weights for policy 0, policy_version 1653201 (0.0006) [2023-12-27 17:05:56,581][20347] Updated weights for policy 0, policy_version 1653211 (0.0006) [2023-12-27 17:05:56,625][20347] Updated weights for policy 0, policy_version 1653221 (0.0008) [2023-12-27 17:05:56,756][20381] Updated weights for policy 1, policy_version 1651572 (0.0006) [2023-12-27 17:05:56,808][20381] Updated weights for policy 1, policy_version 1651582 (0.0007) [2023-12-27 17:05:56,865][20381] Updated weights for policy 1, policy_version 1651592 (0.0005) [2023-12-27 17:05:57,237][20347] Updated weights for policy 0, policy_version 1653231 (0.0007) [2023-12-27 17:05:57,288][20347] Updated weights for policy 0, policy_version 1653241 (0.0006) [2023-12-27 17:05:57,340][20347] Updated weights for policy 0, policy_version 1653251 (0.0007) [2023-12-27 17:05:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 846159872. Throughput: 0: 10380.0, 1: 10169.3. Samples: 846152648. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:05:57,363][19170] Avg episode reward: [(0, '980.400'), (1, '765.580')] [2023-12-27 17:05:57,444][20381] Updated weights for policy 1, policy_version 1651602 (0.0005) [2023-12-27 17:05:57,493][20381] Updated weights for policy 1, policy_version 1651612 (0.0005) [2023-12-27 17:05:57,543][20381] Updated weights for policy 1, policy_version 1651622 (0.0005) [2023-12-27 17:05:57,588][20381] Updated weights for policy 1, policy_version 1651632 (0.0005) [2023-12-27 17:05:58,096][20347] Updated weights for policy 0, policy_version 1653261 (0.0007) [2023-12-27 17:05:58,157][20347] Updated weights for policy 0, policy_version 1653271 (0.0007) [2023-12-27 17:05:58,180][20381] Updated weights for policy 1, policy_version 1651642 (0.0007) [2023-12-27 17:05:58,217][20347] Updated weights for policy 0, policy_version 1653281 (0.0007) [2023-12-27 17:05:58,243][20381] Updated weights for policy 1, policy_version 1651652 (0.0008) [2023-12-27 17:05:58,303][20381] Updated weights for policy 1, policy_version 1651662 (0.0007) [2023-12-27 17:05:59,005][20347] Updated weights for policy 0, policy_version 1653291 (0.0008) [2023-12-27 17:05:59,041][20381] Updated weights for policy 1, policy_version 1651672 (0.0007) [2023-12-27 17:05:59,063][20347] Updated weights for policy 0, policy_version 1653301 (0.0006) [2023-12-27 17:05:59,098][20381] Updated weights for policy 1, policy_version 1651682 (0.0007) [2023-12-27 17:05:59,124][20347] Updated weights for policy 0, policy_version 1653311 (0.0007) [2023-12-27 17:05:59,159][20381] Updated weights for policy 1, policy_version 1651692 (0.0009) [2023-12-27 17:05:59,761][20347] Updated weights for policy 0, policy_version 1653321 (0.0007) [2023-12-27 17:05:59,829][20347] Updated weights for policy 0, policy_version 1653331 (0.0006) [2023-12-27 17:05:59,858][20381] Updated weights for policy 1, policy_version 1651702 (0.0007) [2023-12-27 17:05:59,879][20347] Updated weights for policy 0, policy_version 1653341 (0.0006) [2023-12-27 17:05:59,915][20381] Updated weights for policy 1, policy_version 1651712 (0.0007) [2023-12-27 17:05:59,943][20347] Updated weights for policy 0, policy_version 1653351 (0.0007) [2023-12-27 17:05:59,973][20381] Updated weights for policy 1, policy_version 1651722 (0.0006) [2023-12-27 17:06:00,603][20347] Updated weights for policy 0, policy_version 1653361 (0.0006) [2023-12-27 17:06:00,628][20381] Updated weights for policy 1, policy_version 1651732 (0.0006) [2023-12-27 17:06:00,652][20347] Updated weights for policy 0, policy_version 1653371 (0.0006) [2023-12-27 17:06:00,677][20381] Updated weights for policy 1, policy_version 1651742 (0.0006) [2023-12-27 17:06:00,708][20347] Updated weights for policy 0, policy_version 1653381 (0.0007) [2023-12-27 17:06:00,732][20381] Updated weights for policy 1, policy_version 1651752 (0.0005) [2023-12-27 17:06:01,344][20347] Updated weights for policy 0, policy_version 1653391 (0.0007) [2023-12-27 17:06:01,407][20347] Updated weights for policy 0, policy_version 1653401 (0.0008) [2023-12-27 17:06:01,467][20347] Updated weights for policy 0, policy_version 1653411 (0.0008) [2023-12-27 17:06:01,472][20381] Updated weights for policy 1, policy_version 1651762 (0.0006) [2023-12-27 17:06:01,528][20381] Updated weights for policy 1, policy_version 1651772 (0.0008) [2023-12-27 17:06:01,580][20381] Updated weights for policy 1, policy_version 1651782 (0.0008) [2023-12-27 17:06:01,643][20381] Updated weights for policy 1, policy_version 1651792 (0.0008) [2023-12-27 17:06:02,116][20347] Updated weights for policy 0, policy_version 1653421 (0.0006) [2023-12-27 17:06:02,178][20347] Updated weights for policy 0, policy_version 1653431 (0.0007) [2023-12-27 17:06:02,236][20347] Updated weights for policy 0, policy_version 1653441 (0.0006) [2023-12-27 17:06:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 846266368. Throughput: 0: 10311.7, 1: 10197.9. Samples: 846276208. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:02,363][19170] Avg episode reward: [(0, '970.670'), (1, '756.090')] [2023-12-27 17:06:02,380][20381] Updated weights for policy 1, policy_version 1651802 (0.0007) [2023-12-27 17:06:02,439][20381] Updated weights for policy 1, policy_version 1651812 (0.0006) [2023-12-27 17:06:02,500][20381] Updated weights for policy 1, policy_version 1651822 (0.0007) [2023-12-27 17:06:02,928][20347] Updated weights for policy 0, policy_version 1653451 (0.0007) [2023-12-27 17:06:02,973][20347] Updated weights for policy 0, policy_version 1653461 (0.0006) [2023-12-27 17:06:03,026][20347] Updated weights for policy 0, policy_version 1653471 (0.0005) [2023-12-27 17:06:03,121][20381] Updated weights for policy 1, policy_version 1651832 (0.0007) [2023-12-27 17:06:03,167][20381] Updated weights for policy 1, policy_version 1651842 (0.0007) [2023-12-27 17:06:03,216][20381] Updated weights for policy 1, policy_version 1651852 (0.0005) [2023-12-27 17:06:03,665][20347] Updated weights for policy 0, policy_version 1653481 (0.0007) [2023-12-27 17:06:03,717][20347] Updated weights for policy 0, policy_version 1653491 (0.0005) [2023-12-27 17:06:03,767][20347] Updated weights for policy 0, policy_version 1653501 (0.0006) [2023-12-27 17:06:03,778][20381] Updated weights for policy 1, policy_version 1651862 (0.0005) [2023-12-27 17:06:03,813][20347] Updated weights for policy 0, policy_version 1653511 (0.0005) [2023-12-27 17:06:03,839][20381] Updated weights for policy 1, policy_version 1651872 (0.0007) [2023-12-27 17:06:03,898][20381] Updated weights for policy 1, policy_version 1651882 (0.0007) [2023-12-27 17:06:04,490][20381] Updated weights for policy 1, policy_version 1651892 (0.0006) [2023-12-27 17:06:04,505][20347] Updated weights for policy 0, policy_version 1653521 (0.0006) [2023-12-27 17:06:04,548][20381] Updated weights for policy 1, policy_version 1651902 (0.0006) [2023-12-27 17:06:04,565][20347] Updated weights for policy 0, policy_version 1653531 (0.0006) [2023-12-27 17:06:04,606][20381] Updated weights for policy 1, policy_version 1651912 (0.0005) [2023-12-27 17:06:04,620][20347] Updated weights for policy 0, policy_version 1653541 (0.0007) [2023-12-27 17:06:05,221][20381] Updated weights for policy 1, policy_version 1651922 (0.0006) [2023-12-27 17:06:05,270][20381] Updated weights for policy 1, policy_version 1651932 (0.0008) [2023-12-27 17:06:05,291][20347] Updated weights for policy 0, policy_version 1653551 (0.0009) [2023-12-27 17:06:05,322][20381] Updated weights for policy 1, policy_version 1651942 (0.0005) [2023-12-27 17:06:05,346][20347] Updated weights for policy 0, policy_version 1653561 (0.0010) [2023-12-27 17:06:05,377][20381] Updated weights for policy 1, policy_version 1651952 (0.0006) [2023-12-27 17:06:05,400][20347] Updated weights for policy 0, policy_version 1653571 (0.0010) [2023-12-27 17:06:06,040][20381] Updated weights for policy 1, policy_version 1651962 (0.0006) [2023-12-27 17:06:06,042][20347] Updated weights for policy 0, policy_version 1653581 (0.0009) [2023-12-27 17:06:06,086][20381] Updated weights for policy 1, policy_version 1651972 (0.0008) [2023-12-27 17:06:06,093][20347] Updated weights for policy 0, policy_version 1653591 (0.0008) [2023-12-27 17:06:06,144][20381] Updated weights for policy 1, policy_version 1651982 (0.0007) [2023-12-27 17:06:06,149][20347] Updated weights for policy 0, policy_version 1653601 (0.0008) [2023-12-27 17:06:06,824][20381] Updated weights for policy 1, policy_version 1651992 (0.0006) [2023-12-27 17:06:06,850][20347] Updated weights for policy 0, policy_version 1653611 (0.0008) [2023-12-27 17:06:06,873][20381] Updated weights for policy 1, policy_version 1652002 (0.0005) [2023-12-27 17:06:06,908][20347] Updated weights for policy 0, policy_version 1653621 (0.0006) [2023-12-27 17:06:06,922][20381] Updated weights for policy 1, policy_version 1652012 (0.0006) [2023-12-27 17:06:06,964][20347] Updated weights for policy 0, policy_version 1653631 (0.0010) [2023-12-27 17:06:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 846372864. Throughput: 0: 10314.4, 1: 10306.4. Samples: 846341160. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:07,364][19170] Avg episode reward: [(0, '951.110'), (1, '794.960')] [2023-12-27 17:06:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001652016_422961152.pth... [2023-12-27 17:06:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001653640_423411712.pth... [2023-12-27 17:06:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001652456_423108608.pth [2023-12-27 17:06:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001650800_422649856.pth [2023-12-27 17:06:07,490][20381] Updated weights for policy 1, policy_version 1652022 (0.0007) [2023-12-27 17:06:07,547][20381] Updated weights for policy 1, policy_version 1652032 (0.0007) [2023-12-27 17:06:07,602][20381] Updated weights for policy 1, policy_version 1652042 (0.0007) [2023-12-27 17:06:07,607][20347] Updated weights for policy 0, policy_version 1653641 (0.0010) [2023-12-27 17:06:07,667][20347] Updated weights for policy 0, policy_version 1653651 (0.0007) [2023-12-27 17:06:07,732][20347] Updated weights for policy 0, policy_version 1653661 (0.0006) [2023-12-27 17:06:07,793][20347] Updated weights for policy 0, policy_version 1653671 (0.0009) [2023-12-27 17:06:08,229][20381] Updated weights for policy 1, policy_version 1652052 (0.0007) [2023-12-27 17:06:08,274][20381] Updated weights for policy 1, policy_version 1652062 (0.0006) [2023-12-27 17:06:08,326][20381] Updated weights for policy 1, policy_version 1652072 (0.0008) [2023-12-27 17:06:08,406][20347] Updated weights for policy 0, policy_version 1653681 (0.0010) [2023-12-27 17:06:08,465][20347] Updated weights for policy 0, policy_version 1653691 (0.0010) [2023-12-27 17:06:08,523][20347] Updated weights for policy 0, policy_version 1653701 (0.0010) [2023-12-27 17:06:08,969][20381] Updated weights for policy 1, policy_version 1652082 (0.0008) [2023-12-27 17:06:09,013][20381] Updated weights for policy 1, policy_version 1652092 (0.0008) [2023-12-27 17:06:09,057][20381] Updated weights for policy 1, policy_version 1652102 (0.0007) [2023-12-27 17:06:09,103][20381] Updated weights for policy 1, policy_version 1652112 (0.0007) [2023-12-27 17:06:09,204][20347] Updated weights for policy 0, policy_version 1653711 (0.0007) [2023-12-27 17:06:09,270][20347] Updated weights for policy 0, policy_version 1653721 (0.0008) [2023-12-27 17:06:09,321][20347] Updated weights for policy 0, policy_version 1653731 (0.0008) [2023-12-27 17:06:09,907][20381] Updated weights for policy 1, policy_version 1652122 (0.0008) [2023-12-27 17:06:09,974][20381] Updated weights for policy 1, policy_version 1652132 (0.0008) [2023-12-27 17:06:10,042][20381] Updated weights for policy 1, policy_version 1652142 (0.0007) [2023-12-27 17:06:10,071][20347] Updated weights for policy 0, policy_version 1653741 (0.0009) [2023-12-27 17:06:10,127][20347] Updated weights for policy 0, policy_version 1653751 (0.0010) [2023-12-27 17:06:10,183][20347] Updated weights for policy 0, policy_version 1653761 (0.0010) [2023-12-27 17:06:10,630][20381] Updated weights for policy 1, policy_version 1652152 (0.0010) [2023-12-27 17:06:10,685][20381] Updated weights for policy 1, policy_version 1652162 (0.0010) [2023-12-27 17:06:10,733][20381] Updated weights for policy 1, policy_version 1652172 (0.0006) [2023-12-27 17:06:10,868][20347] Updated weights for policy 0, policy_version 1653771 (0.0010) [2023-12-27 17:06:10,917][20347] Updated weights for policy 0, policy_version 1653781 (0.0010) [2023-12-27 17:06:10,966][20347] Updated weights for policy 0, policy_version 1653791 (0.0010) [2023-12-27 17:06:11,402][20381] Updated weights for policy 1, policy_version 1652182 (0.0007) [2023-12-27 17:06:11,462][20381] Updated weights for policy 1, policy_version 1652192 (0.0008) [2023-12-27 17:06:11,524][20381] Updated weights for policy 1, policy_version 1652202 (0.0007) [2023-12-27 17:06:11,727][20347] Updated weights for policy 0, policy_version 1653801 (0.0010) [2023-12-27 17:06:11,791][20347] Updated weights for policy 0, policy_version 1653811 (0.0007) [2023-12-27 17:06:11,845][20347] Updated weights for policy 0, policy_version 1653821 (0.0008) [2023-12-27 17:06:11,902][20347] Updated weights for policy 0, policy_version 1653831 (0.0008) [2023-12-27 17:06:12,228][20381] Updated weights for policy 1, policy_version 1652212 (0.0008) [2023-12-27 17:06:12,289][20381] Updated weights for policy 1, policy_version 1652222 (0.0008) [2023-12-27 17:06:12,352][20381] Updated weights for policy 1, policy_version 1652232 (0.0008) [2023-12-27 17:06:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 846471168. Throughput: 0: 10326.6, 1: 10401.1. Samples: 846466512. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:12,363][19170] Avg episode reward: [(0, '941.280'), (1, '824.110')] [2023-12-27 17:06:12,654][20347] Updated weights for policy 0, policy_version 1653841 (0.0008) [2023-12-27 17:06:12,706][20347] Updated weights for policy 0, policy_version 1653851 (0.0008) [2023-12-27 17:06:12,759][20347] Updated weights for policy 0, policy_version 1653861 (0.0008) [2023-12-27 17:06:13,055][20381] Updated weights for policy 1, policy_version 1652242 (0.0008) [2023-12-27 17:06:13,118][20381] Updated weights for policy 1, policy_version 1652252 (0.0007) [2023-12-27 17:06:13,168][20381] Updated weights for policy 1, policy_version 1652262 (0.0005) [2023-12-27 17:06:13,223][20381] Updated weights for policy 1, policy_version 1652272 (0.0005) [2023-12-27 17:06:13,508][20347] Updated weights for policy 0, policy_version 1653871 (0.0008) [2023-12-27 17:06:13,566][20347] Updated weights for policy 0, policy_version 1653881 (0.0007) [2023-12-27 17:06:13,620][20347] Updated weights for policy 0, policy_version 1653891 (0.0007) [2023-12-27 17:06:13,818][20381] Updated weights for policy 1, policy_version 1652282 (0.0008) [2023-12-27 17:06:13,871][20381] Updated weights for policy 1, policy_version 1652292 (0.0007) [2023-12-27 17:06:13,926][20381] Updated weights for policy 1, policy_version 1652302 (0.0008) [2023-12-27 17:06:14,308][20347] Updated weights for policy 0, policy_version 1653901 (0.0007) [2023-12-27 17:06:14,366][20347] Updated weights for policy 0, policy_version 1653911 (0.0007) [2023-12-27 17:06:14,419][20347] Updated weights for policy 0, policy_version 1653921 (0.0008) [2023-12-27 17:06:14,633][20381] Updated weights for policy 1, policy_version 1652312 (0.0006) [2023-12-27 17:06:14,680][20381] Updated weights for policy 1, policy_version 1652322 (0.0008) [2023-12-27 17:06:14,725][20381] Updated weights for policy 1, policy_version 1652332 (0.0010) [2023-12-27 17:06:15,091][20347] Updated weights for policy 0, policy_version 1653931 (0.0008) [2023-12-27 17:06:15,143][20347] Updated weights for policy 0, policy_version 1653941 (0.0007) [2023-12-27 17:06:15,202][20347] Updated weights for policy 0, policy_version 1653951 (0.0007) [2023-12-27 17:06:15,467][20381] Updated weights for policy 1, policy_version 1652342 (0.0010) [2023-12-27 17:06:15,522][20381] Updated weights for policy 1, policy_version 1652352 (0.0011) [2023-12-27 17:06:15,574][20381] Updated weights for policy 1, policy_version 1652362 (0.0007) [2023-12-27 17:06:15,895][20347] Updated weights for policy 0, policy_version 1653961 (0.0007) [2023-12-27 17:06:15,948][20347] Updated weights for policy 0, policy_version 1653971 (0.0005) [2023-12-27 17:06:16,001][20347] Updated weights for policy 0, policy_version 1653981 (0.0007) [2023-12-27 17:06:16,054][20347] Updated weights for policy 0, policy_version 1653991 (0.0007) [2023-12-27 17:06:16,215][20381] Updated weights for policy 1, policy_version 1652372 (0.0007) [2023-12-27 17:06:16,267][20381] Updated weights for policy 1, policy_version 1652382 (0.0009) [2023-12-27 17:06:16,328][20381] Updated weights for policy 1, policy_version 1652392 (0.0006) [2023-12-27 17:06:16,704][20347] Updated weights for policy 0, policy_version 1654001 (0.0010) [2023-12-27 17:06:16,759][20347] Updated weights for policy 0, policy_version 1654011 (0.0010) [2023-12-27 17:06:16,818][20347] Updated weights for policy 0, policy_version 1654021 (0.0010) [2023-12-27 17:06:16,928][20381] Updated weights for policy 1, policy_version 1652402 (0.0005) [2023-12-27 17:06:16,985][20381] Updated weights for policy 1, policy_version 1652412 (0.0008) [2023-12-27 17:06:17,050][20381] Updated weights for policy 1, policy_version 1652422 (0.0007) [2023-12-27 17:06:17,098][20381] Updated weights for policy 1, policy_version 1652432 (0.0008) [2023-12-27 17:06:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 846577664. Throughput: 0: 10265.9, 1: 10395.2. Samples: 846587484. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:17,363][19170] Avg episode reward: [(0, '931.550'), (1, '746.310')] [2023-12-27 17:06:17,466][20347] Updated weights for policy 0, policy_version 1654031 (0.0007) [2023-12-27 17:06:17,514][20347] Updated weights for policy 0, policy_version 1654041 (0.0005) [2023-12-27 17:06:17,560][20347] Updated weights for policy 0, policy_version 1654051 (0.0005) [2023-12-27 17:06:17,831][20381] Updated weights for policy 1, policy_version 1652442 (0.0005) [2023-12-27 17:06:17,882][20381] Updated weights for policy 1, policy_version 1652452 (0.0005) [2023-12-27 17:06:17,932][20381] Updated weights for policy 1, policy_version 1652462 (0.0006) [2023-12-27 17:06:18,148][20347] Updated weights for policy 0, policy_version 1654061 (0.0008) [2023-12-27 17:06:18,211][20347] Updated weights for policy 0, policy_version 1654071 (0.0011) [2023-12-27 17:06:18,269][20347] Updated weights for policy 0, policy_version 1654081 (0.0010) [2023-12-27 17:06:18,557][20381] Updated weights for policy 1, policy_version 1652472 (0.0007) [2023-12-27 17:06:18,623][20381] Updated weights for policy 1, policy_version 1652482 (0.0008) [2023-12-27 17:06:18,689][20381] Updated weights for policy 1, policy_version 1652492 (0.0008) [2023-12-27 17:06:19,006][20347] Updated weights for policy 0, policy_version 1654091 (0.0010) [2023-12-27 17:06:19,054][20347] Updated weights for policy 0, policy_version 1654101 (0.0008) [2023-12-27 17:06:19,115][20347] Updated weights for policy 0, policy_version 1654111 (0.0008) [2023-12-27 17:06:19,381][20381] Updated weights for policy 1, policy_version 1652502 (0.0008) [2023-12-27 17:06:19,438][20381] Updated weights for policy 1, policy_version 1652512 (0.0008) [2023-12-27 17:06:19,499][20381] Updated weights for policy 1, policy_version 1652522 (0.0007) [2023-12-27 17:06:19,871][20347] Updated weights for policy 0, policy_version 1654121 (0.0009) [2023-12-27 17:06:19,933][20347] Updated weights for policy 0, policy_version 1654131 (0.0007) [2023-12-27 17:06:19,994][20347] Updated weights for policy 0, policy_version 1654141 (0.0008) [2023-12-27 17:06:20,059][20347] Updated weights for policy 0, policy_version 1654151 (0.0007) [2023-12-27 17:06:20,245][20381] Updated weights for policy 1, policy_version 1652532 (0.0007) [2023-12-27 17:06:20,305][20381] Updated weights for policy 1, policy_version 1652542 (0.0007) [2023-12-27 17:06:20,366][20381] Updated weights for policy 1, policy_version 1652552 (0.0008) [2023-12-27 17:06:20,775][20347] Updated weights for policy 0, policy_version 1654161 (0.0010) [2023-12-27 17:06:20,835][20347] Updated weights for policy 0, policy_version 1654171 (0.0011) [2023-12-27 17:06:20,899][20347] Updated weights for policy 0, policy_version 1654181 (0.0011) [2023-12-27 17:06:21,113][20381] Updated weights for policy 1, policy_version 1652562 (0.0008) [2023-12-27 17:06:21,180][20381] Updated weights for policy 1, policy_version 1652572 (0.0008) [2023-12-27 17:06:21,251][20381] Updated weights for policy 1, policy_version 1652582 (0.0006) [2023-12-27 17:06:21,322][20381] Updated weights for policy 1, policy_version 1652592 (0.0008) [2023-12-27 17:06:21,640][20347] Updated weights for policy 0, policy_version 1654191 (0.0010) [2023-12-27 17:06:21,703][20347] Updated weights for policy 0, policy_version 1654201 (0.0008) [2023-12-27 17:06:21,770][20347] Updated weights for policy 0, policy_version 1654211 (0.0007) [2023-12-27 17:06:22,024][20381] Updated weights for policy 1, policy_version 1652602 (0.0010) [2023-12-27 17:06:22,092][20381] Updated weights for policy 1, policy_version 1652612 (0.0011) [2023-12-27 17:06:22,153][20381] Updated weights for policy 1, policy_version 1652622 (0.0011) [2023-12-27 17:06:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 846675968. Throughput: 0: 10230.1, 1: 10387.4. Samples: 846648328. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:22,363][19170] Avg episode reward: [(0, '960.920'), (1, '697.020')] [2023-12-27 17:06:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001652624_423116800.pth... [2023-12-27 17:06:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001654216_423559168.pth... [2023-12-27 17:06:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001651408_422805504.pth [2023-12-27 17:06:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001653064_423264256.pth [2023-12-27 17:06:22,474][20347] Updated weights for policy 0, policy_version 1654221 (0.0008) [2023-12-27 17:06:22,528][20347] Updated weights for policy 0, policy_version 1654231 (0.0008) [2023-12-27 17:06:22,588][20347] Updated weights for policy 0, policy_version 1654241 (0.0008) [2023-12-27 17:06:22,854][20381] Updated weights for policy 1, policy_version 1652632 (0.0009) [2023-12-27 17:06:22,915][20381] Updated weights for policy 1, policy_version 1652642 (0.0008) [2023-12-27 17:06:22,978][20381] Updated weights for policy 1, policy_version 1652652 (0.0008) [2023-12-27 17:06:23,282][20347] Updated weights for policy 0, policy_version 1654251 (0.0008) [2023-12-27 17:06:23,335][20347] Updated weights for policy 0, policy_version 1654261 (0.0009) [2023-12-27 17:06:23,395][20347] Updated weights for policy 0, policy_version 1654271 (0.0009) [2023-12-27 17:06:23,622][20381] Updated weights for policy 1, policy_version 1652662 (0.0006) [2023-12-27 17:06:23,667][20381] Updated weights for policy 1, policy_version 1652672 (0.0005) [2023-12-27 17:06:23,716][20381] Updated weights for policy 1, policy_version 1652682 (0.0005) [2023-12-27 17:06:24,148][20347] Updated weights for policy 0, policy_version 1654281 (0.0007) [2023-12-27 17:06:24,209][20347] Updated weights for policy 0, policy_version 1654291 (0.0006) [2023-12-27 17:06:24,265][20347] Updated weights for policy 0, policy_version 1654301 (0.0007) [2023-12-27 17:06:24,315][20347] Updated weights for policy 0, policy_version 1654311 (0.0007) [2023-12-27 17:06:24,317][20381] Updated weights for policy 1, policy_version 1652692 (0.0006) [2023-12-27 17:06:24,380][20381] Updated weights for policy 1, policy_version 1652702 (0.0006) [2023-12-27 17:06:24,442][20381] Updated weights for policy 1, policy_version 1652712 (0.0005) [2023-12-27 17:06:25,001][20347] Updated weights for policy 0, policy_version 1654321 (0.0008) [2023-12-27 17:06:25,034][20381] Updated weights for policy 1, policy_version 1652722 (0.0006) [2023-12-27 17:06:25,058][20347] Updated weights for policy 0, policy_version 1654331 (0.0006) [2023-12-27 17:06:25,086][20381] Updated weights for policy 1, policy_version 1652732 (0.0006) [2023-12-27 17:06:25,114][20347] Updated weights for policy 0, policy_version 1654341 (0.0008) [2023-12-27 17:06:25,139][20381] Updated weights for policy 1, policy_version 1652742 (0.0006) [2023-12-27 17:06:25,196][20381] Updated weights for policy 1, policy_version 1652752 (0.0005) [2023-12-27 17:06:25,741][20381] Updated weights for policy 1, policy_version 1652762 (0.0008) [2023-12-27 17:06:25,799][20381] Updated weights for policy 1, policy_version 1652772 (0.0007) [2023-12-27 17:06:25,857][20381] Updated weights for policy 1, policy_version 1652782 (0.0007) [2023-12-27 17:06:25,874][20347] Updated weights for policy 0, policy_version 1654351 (0.0006) [2023-12-27 17:06:25,929][20347] Updated weights for policy 0, policy_version 1654361 (0.0008) [2023-12-27 17:06:25,982][20347] Updated weights for policy 0, policy_version 1654371 (0.0008) [2023-12-27 17:06:26,526][20381] Updated weights for policy 1, policy_version 1652792 (0.0009) [2023-12-27 17:06:26,583][20381] Updated weights for policy 1, policy_version 1652802 (0.0010) [2023-12-27 17:06:26,632][20381] Updated weights for policy 1, policy_version 1652812 (0.0010) [2023-12-27 17:06:26,648][20347] Updated weights for policy 0, policy_version 1654381 (0.0008) [2023-12-27 17:06:26,701][20347] Updated weights for policy 0, policy_version 1654391 (0.0005) [2023-12-27 17:06:26,753][20347] Updated weights for policy 0, policy_version 1654401 (0.0005) [2023-12-27 17:06:27,327][20381] Updated weights for policy 1, policy_version 1652822 (0.0008) [2023-12-27 17:06:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 846774272. Throughput: 0: 10101.7, 1: 10451.0. Samples: 846769076. Policy #0 lag: (min: 28.0, avg: 32.7, max: 55.0) [2023-12-27 17:06:27,363][19170] Avg episode reward: [(0, '990.200'), (1, '697.200')] [2023-12-27 17:06:27,383][20381] Updated weights for policy 1, policy_version 1652832 (0.0008) [2023-12-27 17:06:27,404][20347] Updated weights for policy 0, policy_version 1654411 (0.0008) [2023-12-27 17:06:27,441][20381] Updated weights for policy 1, policy_version 1652842 (0.0010) [2023-12-27 17:06:27,462][20347] Updated weights for policy 0, policy_version 1654421 (0.0010) [2023-12-27 17:06:27,520][20347] Updated weights for policy 0, policy_version 1654431 (0.0010) [2023-12-27 17:06:28,041][20381] Updated weights for policy 1, policy_version 1652852 (0.0008) [2023-12-27 17:06:28,086][20347] Updated weights for policy 0, policy_version 1654441 (0.0007) [2023-12-27 17:06:28,088][20381] Updated weights for policy 1, policy_version 1652862 (0.0008) [2023-12-27 17:06:28,150][20381] Updated weights for policy 1, policy_version 1652872 (0.0011) [2023-12-27 17:06:28,151][20347] Updated weights for policy 0, policy_version 1654451 (0.0008) [2023-12-27 17:06:28,209][20347] Updated weights for policy 0, policy_version 1654461 (0.0009) [2023-12-27 17:06:28,258][20347] Updated weights for policy 0, policy_version 1654471 (0.0010) [2023-12-27 17:06:28,854][20381] Updated weights for policy 1, policy_version 1652882 (0.0011) [2023-12-27 17:06:28,909][20381] Updated weights for policy 1, policy_version 1652892 (0.0008) [2023-12-27 17:06:28,914][20347] Updated weights for policy 0, policy_version 1654481 (0.0010) [2023-12-27 17:06:28,961][20381] Updated weights for policy 1, policy_version 1652902 (0.0011) [2023-12-27 17:06:28,969][20347] Updated weights for policy 0, policy_version 1654491 (0.0010) [2023-12-27 17:06:29,013][20381] Updated weights for policy 1, policy_version 1652912 (0.0009) [2023-12-27 17:06:29,023][20347] Updated weights for policy 0, policy_version 1654501 (0.0009) [2023-12-27 17:06:29,617][20381] Updated weights for policy 1, policy_version 1652922 (0.0010) [2023-12-27 17:06:29,675][20381] Updated weights for policy 1, policy_version 1652932 (0.0010) [2023-12-27 17:06:29,714][20347] Updated weights for policy 0, policy_version 1654511 (0.0009) [2023-12-27 17:06:29,734][20381] Updated weights for policy 1, policy_version 1652942 (0.0010) [2023-12-27 17:06:29,777][20347] Updated weights for policy 0, policy_version 1654521 (0.0011) [2023-12-27 17:06:29,834][20347] Updated weights for policy 0, policy_version 1654531 (0.0010) [2023-12-27 17:06:30,399][20381] Updated weights for policy 1, policy_version 1652952 (0.0008) [2023-12-27 17:06:30,449][20381] Updated weights for policy 1, policy_version 1652962 (0.0005) [2023-12-27 17:06:30,503][20381] Updated weights for policy 1, policy_version 1652972 (0.0005) [2023-12-27 17:06:30,508][20347] Updated weights for policy 0, policy_version 1654541 (0.0009) [2023-12-27 17:06:30,562][20347] Updated weights for policy 0, policy_version 1654551 (0.0010) [2023-12-27 17:06:30,611][20347] Updated weights for policy 0, policy_version 1654561 (0.0010) [2023-12-27 17:06:31,165][20381] Updated weights for policy 1, policy_version 1652982 (0.0006) [2023-12-27 17:06:31,218][20381] Updated weights for policy 1, policy_version 1652992 (0.0005) [2023-12-27 17:06:31,278][20381] Updated weights for policy 1, policy_version 1653002 (0.0009) [2023-12-27 17:06:31,317][20347] Updated weights for policy 0, policy_version 1654571 (0.0010) [2023-12-27 17:06:31,361][20291] KL-divergence is very high: 254.3762 [2023-12-27 17:06:31,375][20347] Updated weights for policy 0, policy_version 1654581 (0.0008) [2023-12-27 17:06:31,415][20291] KL-divergence is very high: 196.5712 [2023-12-27 17:06:31,439][20347] Updated weights for policy 0, policy_version 1654591 (0.0010) [2023-12-27 17:06:31,964][20381] Updated weights for policy 1, policy_version 1653012 (0.0007) [2023-12-27 17:06:32,018][20381] Updated weights for policy 1, policy_version 1653022 (0.0008) [2023-12-27 17:06:32,082][20381] Updated weights for policy 1, policy_version 1653032 (0.0008) [2023-12-27 17:06:32,125][20347] Updated weights for policy 0, policy_version 1654601 (0.0010) [2023-12-27 17:06:32,184][20347] Updated weights for policy 0, policy_version 1654611 (0.0010) [2023-12-27 17:06:32,242][20347] Updated weights for policy 0, policy_version 1654621 (0.0008) [2023-12-27 17:06:32,311][20347] Updated weights for policy 0, policy_version 1654631 (0.0008) [2023-12-27 17:06:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 846888960. Throughput: 0: 10062.6, 1: 10493.6. Samples: 846893988. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:32,363][19170] Avg episode reward: [(0, '970.670'), (1, '736.760')] [2023-12-27 17:06:32,787][20381] Updated weights for policy 1, policy_version 1653042 (0.0007) [2023-12-27 17:06:32,843][20381] Updated weights for policy 1, policy_version 1653052 (0.0005) [2023-12-27 17:06:32,899][20381] Updated weights for policy 1, policy_version 1653062 (0.0005) [2023-12-27 17:06:32,961][20381] Updated weights for policy 1, policy_version 1653072 (0.0005) [2023-12-27 17:06:32,970][20347] Updated weights for policy 0, policy_version 1654641 (0.0006) [2023-12-27 17:06:33,021][20347] Updated weights for policy 0, policy_version 1654651 (0.0010) [2023-12-27 17:06:33,066][20347] Updated weights for policy 0, policy_version 1654661 (0.0010) [2023-12-27 17:06:33,467][20381] Updated weights for policy 1, policy_version 1653082 (0.0005) [2023-12-27 17:06:33,514][20381] Updated weights for policy 1, policy_version 1653092 (0.0009) [2023-12-27 17:06:33,560][20381] Updated weights for policy 1, policy_version 1653102 (0.0010) [2023-12-27 17:06:33,708][20347] Updated weights for policy 0, policy_version 1654671 (0.0007) [2023-12-27 17:06:33,770][20347] Updated weights for policy 0, policy_version 1654681 (0.0005) [2023-12-27 17:06:33,828][20347] Updated weights for policy 0, policy_version 1654691 (0.0008) [2023-12-27 17:06:34,246][20381] Updated weights for policy 1, policy_version 1653112 (0.0008) [2023-12-27 17:06:34,309][20381] Updated weights for policy 1, policy_version 1653122 (0.0008) [2023-12-27 17:06:34,362][20381] Updated weights for policy 1, policy_version 1653132 (0.0007) [2023-12-27 17:06:34,507][20347] Updated weights for policy 0, policy_version 1654701 (0.0007) [2023-12-27 17:06:34,561][20347] Updated weights for policy 0, policy_version 1654711 (0.0006) [2023-12-27 17:06:34,627][20347] Updated weights for policy 0, policy_version 1654721 (0.0007) [2023-12-27 17:06:35,016][20381] Updated weights for policy 1, policy_version 1653142 (0.0007) [2023-12-27 17:06:35,065][20381] Updated weights for policy 1, policy_version 1653153 (0.0008) [2023-12-27 17:06:35,118][20381] Updated weights for policy 1, policy_version 1653163 (0.0007) [2023-12-27 17:06:35,316][20347] Updated weights for policy 0, policy_version 1654731 (0.0007) [2023-12-27 17:06:35,366][20347] Updated weights for policy 0, policy_version 1654741 (0.0006) [2023-12-27 17:06:35,410][20347] Updated weights for policy 0, policy_version 1654751 (0.0006) [2023-12-27 17:06:35,815][20381] Updated weights for policy 1, policy_version 1653173 (0.0009) [2023-12-27 17:06:35,873][20381] Updated weights for policy 1, policy_version 1653183 (0.0009) [2023-12-27 17:06:35,931][20381] Updated weights for policy 1, policy_version 1653193 (0.0007) [2023-12-27 17:06:36,037][20347] Updated weights for policy 0, policy_version 1654761 (0.0006) [2023-12-27 17:06:36,088][20347] Updated weights for policy 0, policy_version 1654771 (0.0007) [2023-12-27 17:06:36,147][20347] Updated weights for policy 0, policy_version 1654781 (0.0008) [2023-12-27 17:06:36,197][20347] Updated weights for policy 0, policy_version 1654791 (0.0006) [2023-12-27 17:06:36,667][20381] Updated weights for policy 1, policy_version 1653203 (0.0007) [2023-12-27 17:06:36,719][20381] Updated weights for policy 1, policy_version 1653213 (0.0007) [2023-12-27 17:06:36,780][20381] Updated weights for policy 1, policy_version 1653223 (0.0010) [2023-12-27 17:06:36,919][20347] Updated weights for policy 0, policy_version 1654801 (0.0008) [2023-12-27 17:06:36,970][20347] Updated weights for policy 0, policy_version 1654811 (0.0008) [2023-12-27 17:06:37,018][20347] Updated weights for policy 0, policy_version 1654821 (0.0007) [2023-12-27 17:06:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 846987264. Throughput: 0: 10085.7, 1: 10526.8. Samples: 846957284. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:37,363][19170] Avg episode reward: [(0, '951.130'), (1, '677.960')] [2023-12-27 17:06:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001654824_423714816.pth... [2023-12-27 17:06:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001653232_423272448.pth... [2023-12-27 17:06:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001653640_423411712.pth [2023-12-27 17:06:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001652016_422961152.pth [2023-12-27 17:06:37,440][20381] Updated weights for policy 1, policy_version 1653233 (0.0010) [2023-12-27 17:06:37,490][20381] Updated weights for policy 1, policy_version 1653243 (0.0005) [2023-12-27 17:06:37,536][20381] Updated weights for policy 1, policy_version 1653253 (0.0005) [2023-12-27 17:06:37,583][20381] Updated weights for policy 1, policy_version 1653263 (0.0005) [2023-12-27 17:06:37,700][20347] Updated weights for policy 0, policy_version 1654831 (0.0009) [2023-12-27 17:06:37,756][20347] Updated weights for policy 0, policy_version 1654841 (0.0011) [2023-12-27 17:06:37,818][20347] Updated weights for policy 0, policy_version 1654851 (0.0009) [2023-12-27 17:06:38,227][20381] Updated weights for policy 1, policy_version 1653273 (0.0010) [2023-12-27 17:06:38,278][20381] Updated weights for policy 1, policy_version 1653283 (0.0007) [2023-12-27 17:06:38,326][20381] Updated weights for policy 1, policy_version 1653293 (0.0007) [2023-12-27 17:06:38,497][20347] Updated weights for policy 0, policy_version 1654861 (0.0007) [2023-12-27 17:06:38,543][20291] KL-divergence is very high: 122.4109 [2023-12-27 17:06:38,554][20347] Updated weights for policy 0, policy_version 1654871 (0.0008) [2023-12-27 17:06:38,590][20291] KL-divergence is very high: 115.4875 [2023-12-27 17:06:38,610][20347] Updated weights for policy 0, policy_version 1654881 (0.0008) [2023-12-27 17:06:39,062][20381] Updated weights for policy 1, policy_version 1653303 (0.0006) [2023-12-27 17:06:39,119][20381] Updated weights for policy 1, policy_version 1653313 (0.0006) [2023-12-27 17:06:39,182][20381] Updated weights for policy 1, policy_version 1653323 (0.0006) [2023-12-27 17:06:39,261][20347] Updated weights for policy 0, policy_version 1654891 (0.0008) [2023-12-27 17:06:39,323][20347] Updated weights for policy 0, policy_version 1654901 (0.0008) [2023-12-27 17:06:39,395][20347] Updated weights for policy 0, policy_version 1654911 (0.0008) [2023-12-27 17:06:39,904][20381] Updated weights for policy 1, policy_version 1653333 (0.0008) [2023-12-27 17:06:39,964][20381] Updated weights for policy 1, policy_version 1653343 (0.0008) [2023-12-27 17:06:40,024][20381] Updated weights for policy 1, policy_version 1653353 (0.0007) [2023-12-27 17:06:40,091][20347] Updated weights for policy 0, policy_version 1654921 (0.0007) [2023-12-27 17:06:40,152][20347] Updated weights for policy 0, policy_version 1654931 (0.0006) [2023-12-27 17:06:40,213][20347] Updated weights for policy 0, policy_version 1654941 (0.0005) [2023-12-27 17:06:40,276][20347] Updated weights for policy 0, policy_version 1654951 (0.0007) [2023-12-27 17:06:40,736][20381] Updated weights for policy 1, policy_version 1653363 (0.0007) [2023-12-27 17:06:40,787][20381] Updated weights for policy 1, policy_version 1653373 (0.0007) [2023-12-27 17:06:40,836][20381] Updated weights for policy 1, policy_version 1653383 (0.0007) [2023-12-27 17:06:40,925][20347] Updated weights for policy 0, policy_version 1654961 (0.0010) [2023-12-27 17:06:40,983][20347] Updated weights for policy 0, policy_version 1654971 (0.0010) [2023-12-27 17:06:41,047][20347] Updated weights for policy 0, policy_version 1654981 (0.0010) [2023-12-27 17:06:41,546][20381] Updated weights for policy 1, policy_version 1653393 (0.0007) [2023-12-27 17:06:41,605][20381] Updated weights for policy 1, policy_version 1653403 (0.0006) [2023-12-27 17:06:41,666][20381] Updated weights for policy 1, policy_version 1653413 (0.0008) [2023-12-27 17:06:41,728][20381] Updated weights for policy 1, policy_version 1653423 (0.0008) [2023-12-27 17:06:41,787][20347] Updated weights for policy 0, policy_version 1654991 (0.0008) [2023-12-27 17:06:41,852][20347] Updated weights for policy 0, policy_version 1655001 (0.0008) [2023-12-27 17:06:41,910][20347] Updated weights for policy 0, policy_version 1655011 (0.0008) [2023-12-27 17:06:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 847085568. Throughput: 0: 10136.2, 1: 10443.1. Samples: 847078716. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:42,363][19170] Avg episode reward: [(0, '941.370'), (1, '648.550')] [2023-12-27 17:06:42,395][20381] Updated weights for policy 1, policy_version 1653433 (0.0008) [2023-12-27 17:06:42,451][20381] Updated weights for policy 1, policy_version 1653443 (0.0007) [2023-12-27 17:06:42,504][20381] Updated weights for policy 1, policy_version 1653453 (0.0007) [2023-12-27 17:06:42,655][20347] Updated weights for policy 0, policy_version 1655021 (0.0007) [2023-12-27 17:06:42,703][20347] Updated weights for policy 0, policy_version 1655031 (0.0007) [2023-12-27 17:06:42,754][20347] Updated weights for policy 0, policy_version 1655041 (0.0008) [2023-12-27 17:06:43,213][20381] Updated weights for policy 1, policy_version 1653463 (0.0007) [2023-12-27 17:06:43,269][20381] Updated weights for policy 1, policy_version 1653473 (0.0009) [2023-12-27 17:06:43,327][20381] Updated weights for policy 1, policy_version 1653483 (0.0006) [2023-12-27 17:06:43,492][20347] Updated weights for policy 0, policy_version 1655051 (0.0007) [2023-12-27 17:06:43,548][20347] Updated weights for policy 0, policy_version 1655061 (0.0009) [2023-12-27 17:06:43,604][20347] Updated weights for policy 0, policy_version 1655071 (0.0008) [2023-12-27 17:06:43,925][20381] Updated weights for policy 1, policy_version 1653493 (0.0005) [2023-12-27 17:06:43,987][20381] Updated weights for policy 1, policy_version 1653503 (0.0005) [2023-12-27 17:06:44,042][20381] Updated weights for policy 1, policy_version 1653513 (0.0007) [2023-12-27 17:06:44,272][20347] Updated weights for policy 0, policy_version 1655081 (0.0007) [2023-12-27 17:06:44,319][20347] Updated weights for policy 0, policy_version 1655091 (0.0007) [2023-12-27 17:06:44,367][20347] Updated weights for policy 0, policy_version 1655101 (0.0007) [2023-12-27 17:06:44,416][20347] Updated weights for policy 0, policy_version 1655111 (0.0008) [2023-12-27 17:06:44,681][20381] Updated weights for policy 1, policy_version 1653523 (0.0007) [2023-12-27 17:06:44,732][20381] Updated weights for policy 1, policy_version 1653533 (0.0008) [2023-12-27 17:06:44,786][20381] Updated weights for policy 1, policy_version 1653543 (0.0007) [2023-12-27 17:06:45,159][20347] Updated weights for policy 0, policy_version 1655121 (0.0009) [2023-12-27 17:06:45,212][20347] Updated weights for policy 0, policy_version 1655131 (0.0009) [2023-12-27 17:06:45,261][20347] Updated weights for policy 0, policy_version 1655141 (0.0009) [2023-12-27 17:06:45,588][20381] Updated weights for policy 1, policy_version 1653553 (0.0008) [2023-12-27 17:06:45,642][20381] Updated weights for policy 1, policy_version 1653563 (0.0008) [2023-12-27 17:06:45,691][20381] Updated weights for policy 1, policy_version 1653573 (0.0007) [2023-12-27 17:06:45,739][20381] Updated weights for policy 1, policy_version 1653583 (0.0009) [2023-12-27 17:06:45,984][20347] Updated weights for policy 0, policy_version 1655151 (0.0005) [2023-12-27 17:06:46,047][20347] Updated weights for policy 0, policy_version 1655161 (0.0006) [2023-12-27 17:06:46,096][20347] Updated weights for policy 0, policy_version 1655171 (0.0010) [2023-12-27 17:06:46,405][20381] Updated weights for policy 1, policy_version 1653593 (0.0005) [2023-12-27 17:06:46,461][20381] Updated weights for policy 1, policy_version 1653603 (0.0005) [2023-12-27 17:06:46,514][20381] Updated weights for policy 1, policy_version 1653613 (0.0005) [2023-12-27 17:06:46,742][20347] Updated weights for policy 0, policy_version 1655181 (0.0008) [2023-12-27 17:06:46,797][20347] Updated weights for policy 0, policy_version 1655191 (0.0005) [2023-12-27 17:06:46,848][20347] Updated weights for policy 0, policy_version 1655201 (0.0010) [2023-12-27 17:06:47,012][20381] Updated weights for policy 1, policy_version 1653623 (0.0005) [2023-12-27 17:06:47,068][20381] Updated weights for policy 1, policy_version 1653633 (0.0005) [2023-12-27 17:06:47,124][20381] Updated weights for policy 1, policy_version 1653643 (0.0005) [2023-12-27 17:06:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 847192064. Throughput: 0: 10102.3, 1: 10467.1. Samples: 847201832. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:47,363][19170] Avg episode reward: [(0, '941.340'), (1, '707.170')] [2023-12-27 17:06:47,505][20347] Updated weights for policy 0, policy_version 1655211 (0.0009) [2023-12-27 17:06:47,560][20347] Updated weights for policy 0, policy_version 1655221 (0.0006) [2023-12-27 17:06:47,608][20347] Updated weights for policy 0, policy_version 1655231 (0.0008) [2023-12-27 17:06:47,692][20381] Updated weights for policy 1, policy_version 1653653 (0.0007) [2023-12-27 17:06:47,740][20381] Updated weights for policy 1, policy_version 1653663 (0.0007) [2023-12-27 17:06:47,803][20381] Updated weights for policy 1, policy_version 1653673 (0.0006) [2023-12-27 17:06:48,307][20347] Updated weights for policy 0, policy_version 1655241 (0.0008) [2023-12-27 17:06:48,376][20347] Updated weights for policy 0, policy_version 1655252 (0.0010) [2023-12-27 17:06:48,431][20381] Updated weights for policy 1, policy_version 1653683 (0.0007) [2023-12-27 17:06:48,436][20347] Updated weights for policy 0, policy_version 1655262 (0.0006) [2023-12-27 17:06:48,490][20381] Updated weights for policy 1, policy_version 1653693 (0.0008) [2023-12-27 17:06:48,497][20347] Updated weights for policy 0, policy_version 1655272 (0.0006) [2023-12-27 17:06:48,541][20381] Updated weights for policy 1, policy_version 1653703 (0.0008) [2023-12-27 17:06:49,114][20347] Updated weights for policy 0, policy_version 1655282 (0.0007) [2023-12-27 17:06:49,168][20347] Updated weights for policy 0, policy_version 1655292 (0.0007) [2023-12-27 17:06:49,239][20347] Updated weights for policy 0, policy_version 1655302 (0.0008) [2023-12-27 17:06:49,244][20381] Updated weights for policy 1, policy_version 1653713 (0.0008) [2023-12-27 17:06:49,305][20381] Updated weights for policy 1, policy_version 1653723 (0.0009) [2023-12-27 17:06:49,368][20381] Updated weights for policy 1, policy_version 1653733 (0.0010) [2023-12-27 17:06:49,428][20381] Updated weights for policy 1, policy_version 1653743 (0.0010) [2023-12-27 17:06:50,006][20347] Updated weights for policy 0, policy_version 1655312 (0.0008) [2023-12-27 17:06:50,063][20347] Updated weights for policy 0, policy_version 1655322 (0.0007) [2023-12-27 17:06:50,117][20347] Updated weights for policy 0, policy_version 1655332 (0.0007) [2023-12-27 17:06:50,214][20381] Updated weights for policy 1, policy_version 1653753 (0.0011) [2023-12-27 17:06:50,268][20381] Updated weights for policy 1, policy_version 1653763 (0.0011) [2023-12-27 17:06:50,327][20381] Updated weights for policy 1, policy_version 1653773 (0.0011) [2023-12-27 17:06:50,803][20347] Updated weights for policy 0, policy_version 1655342 (0.0006) [2023-12-27 17:06:50,855][20347] Updated weights for policy 0, policy_version 1655352 (0.0005) [2023-12-27 17:06:50,907][20347] Updated weights for policy 0, policy_version 1655362 (0.0005) [2023-12-27 17:06:51,061][20381] Updated weights for policy 1, policy_version 1653783 (0.0011) [2023-12-27 17:06:51,117][20381] Updated weights for policy 1, policy_version 1653793 (0.0011) [2023-12-27 17:06:51,170][20381] Updated weights for policy 1, policy_version 1653803 (0.0010) [2023-12-27 17:06:51,578][20347] Updated weights for policy 0, policy_version 1655372 (0.0007) [2023-12-27 17:06:51,642][20347] Updated weights for policy 0, policy_version 1655382 (0.0008) [2023-12-27 17:06:51,710][20347] Updated weights for policy 0, policy_version 1655392 (0.0007) [2023-12-27 17:06:51,895][20381] Updated weights for policy 1, policy_version 1653813 (0.0008) [2023-12-27 17:06:51,944][20381] Updated weights for policy 1, policy_version 1653823 (0.0009) [2023-12-27 17:06:52,002][20381] Updated weights for policy 1, policy_version 1653833 (0.0009) [2023-12-27 17:06:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 847290368. Throughput: 0: 10081.5, 1: 10395.5. Samples: 847262624. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:52,363][19170] Avg episode reward: [(0, '970.600'), (1, '756.110')] [2023-12-27 17:06:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001653840_423428096.pth... [2023-12-27 17:06:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001655400_423862272.pth... [2023-12-27 17:06:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001652624_423116800.pth [2023-12-27 17:06:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001654216_423559168.pth [2023-12-27 17:06:52,426][20347] Updated weights for policy 0, policy_version 1655402 (0.0006) [2023-12-27 17:06:52,479][20347] Updated weights for policy 0, policy_version 1655412 (0.0007) [2023-12-27 17:06:52,541][20347] Updated weights for policy 0, policy_version 1655422 (0.0007) [2023-12-27 17:06:52,600][20347] Updated weights for policy 0, policy_version 1655432 (0.0007) [2023-12-27 17:06:52,732][20381] Updated weights for policy 1, policy_version 1653843 (0.0008) [2023-12-27 17:06:52,791][20381] Updated weights for policy 1, policy_version 1653853 (0.0007) [2023-12-27 17:06:52,846][20381] Updated weights for policy 1, policy_version 1653863 (0.0007) [2023-12-27 17:06:53,263][20347] Updated weights for policy 0, policy_version 1655442 (0.0008) [2023-12-27 17:06:53,324][20347] Updated weights for policy 0, policy_version 1655452 (0.0008) [2023-12-27 17:06:53,390][20347] Updated weights for policy 0, policy_version 1655462 (0.0008) [2023-12-27 17:06:53,515][20381] Updated weights for policy 1, policy_version 1653873 (0.0007) [2023-12-27 17:06:53,561][20381] Updated weights for policy 1, policy_version 1653883 (0.0008) [2023-12-27 17:06:53,616][20381] Updated weights for policy 1, policy_version 1653893 (0.0007) [2023-12-27 17:06:53,668][20381] Updated weights for policy 1, policy_version 1653903 (0.0008) [2023-12-27 17:06:54,090][20347] Updated weights for policy 0, policy_version 1655472 (0.0009) [2023-12-27 17:06:54,154][20347] Updated weights for policy 0, policy_version 1655482 (0.0008) [2023-12-27 17:06:54,214][20347] Updated weights for policy 0, policy_version 1655492 (0.0007) [2023-12-27 17:06:54,321][20381] Updated weights for policy 1, policy_version 1653913 (0.0008) [2023-12-27 17:06:54,374][20381] Updated weights for policy 1, policy_version 1653923 (0.0008) [2023-12-27 17:06:54,432][20381] Updated weights for policy 1, policy_version 1653933 (0.0007) [2023-12-27 17:06:54,867][20347] Updated weights for policy 0, policy_version 1655502 (0.0008) [2023-12-27 17:06:54,918][20347] Updated weights for policy 0, policy_version 1655512 (0.0007) [2023-12-27 17:06:54,962][20347] Updated weights for policy 0, policy_version 1655522 (0.0007) [2023-12-27 17:06:55,083][20381] Updated weights for policy 1, policy_version 1653943 (0.0009) [2023-12-27 17:06:55,141][20381] Updated weights for policy 1, policy_version 1653953 (0.0008) [2023-12-27 17:06:55,204][20381] Updated weights for policy 1, policy_version 1653963 (0.0005) [2023-12-27 17:06:55,726][20347] Updated weights for policy 0, policy_version 1655532 (0.0008) [2023-12-27 17:06:55,785][20381] Updated weights for policy 1, policy_version 1653973 (0.0008) [2023-12-27 17:06:55,791][20347] Updated weights for policy 0, policy_version 1655542 (0.0007) [2023-12-27 17:06:55,836][20381] Updated weights for policy 1, policy_version 1653983 (0.0010) [2023-12-27 17:06:55,839][20347] Updated weights for policy 0, policy_version 1655552 (0.0005) [2023-12-27 17:06:55,892][20381] Updated weights for policy 1, policy_version 1653993 (0.0010) [2023-12-27 17:06:56,471][20347] Updated weights for policy 0, policy_version 1655562 (0.0007) [2023-12-27 17:06:56,520][20347] Updated weights for policy 0, policy_version 1655572 (0.0008) [2023-12-27 17:06:56,567][20347] Updated weights for policy 0, policy_version 1655582 (0.0007) [2023-12-27 17:06:56,585][20381] Updated weights for policy 1, policy_version 1654003 (0.0011) [2023-12-27 17:06:56,619][20347] Updated weights for policy 0, policy_version 1655592 (0.0007) [2023-12-27 17:06:56,638][20381] Updated weights for policy 1, policy_version 1654013 (0.0008) [2023-12-27 17:06:56,695][20381] Updated weights for policy 1, policy_version 1654023 (0.0007) [2023-12-27 17:06:57,195][20347] Updated weights for policy 0, policy_version 1655602 (0.0007) [2023-12-27 17:06:57,243][20347] Updated weights for policy 0, policy_version 1655612 (0.0008) [2023-12-27 17:06:57,266][20381] Updated weights for policy 1, policy_version 1654033 (0.0005) [2023-12-27 17:06:57,291][20347] Updated weights for policy 0, policy_version 1655622 (0.0007) [2023-12-27 17:06:57,316][20381] Updated weights for policy 1, policy_version 1654043 (0.0006) [2023-12-27 17:06:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 847396864. Throughput: 0: 10045.6, 1: 10334.2. Samples: 847383600. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:06:57,363][19170] Avg episode reward: [(0, '970.610'), (1, '687.610')] [2023-12-27 17:06:57,367][20381] Updated weights for policy 1, policy_version 1654053 (0.0007) [2023-12-27 17:06:57,423][20381] Updated weights for policy 1, policy_version 1654063 (0.0006) [2023-12-27 17:06:57,959][20347] Updated weights for policy 0, policy_version 1655632 (0.0007) [2023-12-27 17:06:58,001][20381] Updated weights for policy 1, policy_version 1654073 (0.0006) [2023-12-27 17:06:58,019][20347] Updated weights for policy 0, policy_version 1655642 (0.0007) [2023-12-27 17:06:58,057][20381] Updated weights for policy 1, policy_version 1654083 (0.0005) [2023-12-27 17:06:58,074][20347] Updated weights for policy 0, policy_version 1655652 (0.0007) [2023-12-27 17:06:58,106][20381] Updated weights for policy 1, policy_version 1654093 (0.0005) [2023-12-27 17:06:58,804][20381] Updated weights for policy 1, policy_version 1654103 (0.0008) [2023-12-27 17:06:58,870][20347] Updated weights for policy 0, policy_version 1655662 (0.0008) [2023-12-27 17:06:58,874][20381] Updated weights for policy 1, policy_version 1654113 (0.0008) [2023-12-27 17:06:58,930][20347] Updated weights for policy 0, policy_version 1655672 (0.0006) [2023-12-27 17:06:58,935][20381] Updated weights for policy 1, policy_version 1654123 (0.0008) [2023-12-27 17:06:58,987][20347] Updated weights for policy 0, policy_version 1655682 (0.0007) [2023-12-27 17:06:59,647][20381] Updated weights for policy 1, policy_version 1654133 (0.0008) [2023-12-27 17:06:59,706][20381] Updated weights for policy 1, policy_version 1654143 (0.0008) [2023-12-27 17:06:59,717][20347] Updated weights for policy 0, policy_version 1655692 (0.0008) [2023-12-27 17:06:59,764][20381] Updated weights for policy 1, policy_version 1654153 (0.0009) [2023-12-27 17:06:59,769][20347] Updated weights for policy 0, policy_version 1655702 (0.0009) [2023-12-27 17:06:59,824][20347] Updated weights for policy 0, policy_version 1655712 (0.0007) [2023-12-27 17:07:00,456][20381] Updated weights for policy 1, policy_version 1654163 (0.0006) [2023-12-27 17:07:00,509][20381] Updated weights for policy 1, policy_version 1654173 (0.0008) [2023-12-27 17:07:00,552][20347] Updated weights for policy 0, policy_version 1655722 (0.0008) [2023-12-27 17:07:00,557][20381] Updated weights for policy 1, policy_version 1654183 (0.0008) [2023-12-27 17:07:00,598][20347] Updated weights for policy 0, policy_version 1655732 (0.0005) [2023-12-27 17:07:00,646][20347] Updated weights for policy 0, policy_version 1655742 (0.0005) [2023-12-27 17:07:00,694][20347] Updated weights for policy 0, policy_version 1655752 (0.0005) [2023-12-27 17:07:01,336][20347] Updated weights for policy 0, policy_version 1655762 (0.0008) [2023-12-27 17:07:01,350][20381] Updated weights for policy 1, policy_version 1654193 (0.0007) [2023-12-27 17:07:01,400][20347] Updated weights for policy 0, policy_version 1655772 (0.0009) [2023-12-27 17:07:01,418][20381] Updated weights for policy 1, policy_version 1654203 (0.0007) [2023-12-27 17:07:01,453][20347] Updated weights for policy 0, policy_version 1655782 (0.0006) [2023-12-27 17:07:01,479][20381] Updated weights for policy 1, policy_version 1654213 (0.0007) [2023-12-27 17:07:01,537][20381] Updated weights for policy 1, policy_version 1654223 (0.0007) [2023-12-27 17:07:02,142][20347] Updated weights for policy 0, policy_version 1655792 (0.0009) [2023-12-27 17:07:02,200][20347] Updated weights for policy 0, policy_version 1655802 (0.0010) [2023-12-27 17:07:02,234][20381] Updated weights for policy 1, policy_version 1654233 (0.0008) [2023-12-27 17:07:02,259][20347] Updated weights for policy 0, policy_version 1655812 (0.0011) [2023-12-27 17:07:02,297][20381] Updated weights for policy 1, policy_version 1654243 (0.0006) [2023-12-27 17:07:02,355][20381] Updated weights for policy 1, policy_version 1654253 (0.0007) [2023-12-27 17:07:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 847495168. Throughput: 0: 10102.7, 1: 10325.9. Samples: 847506772. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:02,363][19170] Avg episode reward: [(0, '980.420'), (1, '629.840')] [2023-12-27 17:07:02,904][20347] Updated weights for policy 0, policy_version 1655822 (0.0007) [2023-12-27 17:07:02,963][20347] Updated weights for policy 0, policy_version 1655832 (0.0006) [2023-12-27 17:07:03,014][20347] Updated weights for policy 0, policy_version 1655842 (0.0005) [2023-12-27 17:07:03,061][20381] Updated weights for policy 1, policy_version 1654263 (0.0007) [2023-12-27 17:07:03,120][20381] Updated weights for policy 1, policy_version 1654273 (0.0007) [2023-12-27 17:07:03,178][20381] Updated weights for policy 1, policy_version 1654283 (0.0008) [2023-12-27 17:07:03,632][20347] Updated weights for policy 0, policy_version 1655852 (0.0006) [2023-12-27 17:07:03,688][20347] Updated weights for policy 0, policy_version 1655862 (0.0007) [2023-12-27 17:07:03,742][20347] Updated weights for policy 0, policy_version 1655872 (0.0008) [2023-12-27 17:07:03,871][20381] Updated weights for policy 1, policy_version 1654293 (0.0007) [2023-12-27 17:07:03,931][20381] Updated weights for policy 1, policy_version 1654303 (0.0007) [2023-12-27 17:07:03,990][20381] Updated weights for policy 1, policy_version 1654313 (0.0005) [2023-12-27 17:07:04,451][20347] Updated weights for policy 0, policy_version 1655882 (0.0007) [2023-12-27 17:07:04,499][20347] Updated weights for policy 0, policy_version 1655892 (0.0007) [2023-12-27 17:07:04,551][20347] Updated weights for policy 0, policy_version 1655902 (0.0007) [2023-12-27 17:07:04,603][20347] Updated weights for policy 0, policy_version 1655912 (0.0007) [2023-12-27 17:07:04,652][20381] Updated weights for policy 1, policy_version 1654323 (0.0006) [2023-12-27 17:07:04,708][20381] Updated weights for policy 1, policy_version 1654333 (0.0008) [2023-12-27 17:07:04,755][20381] Updated weights for policy 1, policy_version 1654343 (0.0007) [2023-12-27 17:07:05,295][20347] Updated weights for policy 0, policy_version 1655922 (0.0005) [2023-12-27 17:07:05,349][20347] Updated weights for policy 0, policy_version 1655932 (0.0005) [2023-12-27 17:07:05,400][20347] Updated weights for policy 0, policy_version 1655942 (0.0005) [2023-12-27 17:07:05,492][20381] Updated weights for policy 1, policy_version 1654353 (0.0008) [2023-12-27 17:07:05,548][20381] Updated weights for policy 1, policy_version 1654363 (0.0008) [2023-12-27 17:07:05,601][20381] Updated weights for policy 1, policy_version 1654373 (0.0007) [2023-12-27 17:07:05,655][20381] Updated weights for policy 1, policy_version 1654383 (0.0008) [2023-12-27 17:07:06,037][20347] Updated weights for policy 0, policy_version 1655952 (0.0009) [2023-12-27 17:07:06,086][20347] Updated weights for policy 0, policy_version 1655962 (0.0010) [2023-12-27 17:07:06,150][20347] Updated weights for policy 0, policy_version 1655972 (0.0010) [2023-12-27 17:07:06,366][20381] Updated weights for policy 1, policy_version 1654393 (0.0009) [2023-12-27 17:07:06,419][20381] Updated weights for policy 1, policy_version 1654403 (0.0011) [2023-12-27 17:07:06,477][20381] Updated weights for policy 1, policy_version 1654413 (0.0011) [2023-12-27 17:07:06,767][20347] Updated weights for policy 0, policy_version 1655982 (0.0008) [2023-12-27 17:07:06,823][20347] Updated weights for policy 0, policy_version 1655992 (0.0010) [2023-12-27 17:07:06,881][20347] Updated weights for policy 0, policy_version 1656002 (0.0010) [2023-12-27 17:07:07,183][20381] Updated weights for policy 1, policy_version 1654423 (0.0010) [2023-12-27 17:07:07,248][20381] Updated weights for policy 1, policy_version 1654433 (0.0005) [2023-12-27 17:07:07,309][20381] Updated weights for policy 1, policy_version 1654443 (0.0010) [2023-12-27 17:07:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 847601664. Throughput: 0: 10107.2, 1: 10320.8. Samples: 847567588. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:07,363][19170] Avg episode reward: [(0, '970.630'), (1, '599.720')] [2023-12-27 17:07:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001656008_424017920.pth... [2023-12-27 17:07:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001654448_423583744.pth... [2023-12-27 17:07:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001654824_423714816.pth [2023-12-27 17:07:07,383][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001653232_423272448.pth [2023-12-27 17:07:07,563][20347] Updated weights for policy 0, policy_version 1656012 (0.0010) [2023-12-27 17:07:07,614][20347] Updated weights for policy 0, policy_version 1656022 (0.0010) [2023-12-27 17:07:07,665][20347] Updated weights for policy 0, policy_version 1656032 (0.0007) [2023-12-27 17:07:07,966][20381] Updated weights for policy 1, policy_version 1654453 (0.0010) [2023-12-27 17:07:08,015][20381] Updated weights for policy 1, policy_version 1654463 (0.0010) [2023-12-27 17:07:08,064][20381] Updated weights for policy 1, policy_version 1654473 (0.0010) [2023-12-27 17:07:08,254][20347] Updated weights for policy 0, policy_version 1656042 (0.0006) [2023-12-27 17:07:08,303][20347] Updated weights for policy 0, policy_version 1656052 (0.0006) [2023-12-27 17:07:08,362][20347] Updated weights for policy 0, policy_version 1656062 (0.0009) [2023-12-27 17:07:08,412][20347] Updated weights for policy 0, policy_version 1656072 (0.0010) [2023-12-27 17:07:08,743][20381] Updated weights for policy 1, policy_version 1654483 (0.0010) [2023-12-27 17:07:08,802][20381] Updated weights for policy 1, policy_version 1654493 (0.0011) [2023-12-27 17:07:08,859][20381] Updated weights for policy 1, policy_version 1654503 (0.0011) [2023-12-27 17:07:09,096][20347] Updated weights for policy 0, policy_version 1656082 (0.0009) [2023-12-27 17:07:09,152][20347] Updated weights for policy 0, policy_version 1656092 (0.0010) [2023-12-27 17:07:09,207][20347] Updated weights for policy 0, policy_version 1656102 (0.0010) [2023-12-27 17:07:09,525][20381] Updated weights for policy 1, policy_version 1654513 (0.0010) [2023-12-27 17:07:09,576][20381] Updated weights for policy 1, policy_version 1654523 (0.0010) [2023-12-27 17:07:09,635][20381] Updated weights for policy 1, policy_version 1654533 (0.0010) [2023-12-27 17:07:09,700][20381] Updated weights for policy 1, policy_version 1654543 (0.0010) [2023-12-27 17:07:09,969][20347] Updated weights for policy 0, policy_version 1656112 (0.0007) [2023-12-27 17:07:10,035][20347] Updated weights for policy 0, policy_version 1656122 (0.0006) [2023-12-27 17:07:10,085][20347] Updated weights for policy 0, policy_version 1656132 (0.0006) [2023-12-27 17:07:10,458][20381] Updated weights for policy 1, policy_version 1654553 (0.0009) [2023-12-27 17:07:10,523][20381] Updated weights for policy 1, policy_version 1654563 (0.0008) [2023-12-27 17:07:10,577][20381] Updated weights for policy 1, policy_version 1654573 (0.0008) [2023-12-27 17:07:10,673][20347] Updated weights for policy 0, policy_version 1656142 (0.0007) [2023-12-27 17:07:10,736][20347] Updated weights for policy 0, policy_version 1656152 (0.0007) [2023-12-27 17:07:10,796][20347] Updated weights for policy 0, policy_version 1656162 (0.0009) [2023-12-27 17:07:11,320][20381] Updated weights for policy 1, policy_version 1654583 (0.0008) [2023-12-27 17:07:11,393][20381] Updated weights for policy 1, policy_version 1654593 (0.0008) [2023-12-27 17:07:11,451][20381] Updated weights for policy 1, policy_version 1654603 (0.0010) [2023-12-27 17:07:11,506][20347] Updated weights for policy 0, policy_version 1656172 (0.0006) [2023-12-27 17:07:11,562][20347] Updated weights for policy 0, policy_version 1656182 (0.0007) [2023-12-27 17:07:11,633][20347] Updated weights for policy 0, policy_version 1656192 (0.0008) [2023-12-27 17:07:12,210][20381] Updated weights for policy 1, policy_version 1654613 (0.0008) [2023-12-27 17:07:12,269][20381] Updated weights for policy 1, policy_version 1654623 (0.0008) [2023-12-27 17:07:12,299][20347] Updated weights for policy 0, policy_version 1656202 (0.0006) [2023-12-27 17:07:12,333][20381] Updated weights for policy 1, policy_version 1654633 (0.0007) [2023-12-27 17:07:12,362][20347] Updated weights for policy 0, policy_version 1656212 (0.0007) [2023-12-27 17:07:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 847691776. Throughput: 0: 10225.1, 1: 10241.0. Samples: 847690052. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:12,363][19170] Avg episode reward: [(0, '970.630'), (1, '658.250')] [2023-12-27 17:07:12,422][20347] Updated weights for policy 0, policy_version 1656222 (0.0008) [2023-12-27 17:07:13,056][20381] Updated weights for policy 1, policy_version 1654643 (0.0008) [2023-12-27 17:07:13,112][20381] Updated weights for policy 1, policy_version 1654653 (0.0008) [2023-12-27 17:07:13,124][20347] Updated weights for policy 0, policy_version 1656233 (0.0008) [2023-12-27 17:07:13,173][20381] Updated weights for policy 1, policy_version 1654663 (0.0008) [2023-12-27 17:07:13,178][20347] Updated weights for policy 0, policy_version 1656243 (0.0005) [2023-12-27 17:07:13,224][20347] Updated weights for policy 0, policy_version 1656253 (0.0005) [2023-12-27 17:07:13,272][20347] Updated weights for policy 0, policy_version 1656263 (0.0007) [2023-12-27 17:07:13,811][20381] Updated weights for policy 1, policy_version 1654673 (0.0006) [2023-12-27 17:07:13,864][20292] KL-divergence is very high: 198.8551 [2023-12-27 17:07:13,871][20381] Updated weights for policy 1, policy_version 1654683 (0.0005) [2023-12-27 17:07:13,913][20292] KL-divergence is very high: 226.5990 [2023-12-27 17:07:13,930][20381] Updated weights for policy 1, policy_version 1654693 (0.0007) [2023-12-27 17:07:13,936][20347] Updated weights for policy 0, policy_version 1656273 (0.0005) [2023-12-27 17:07:13,956][20292] KL-divergence is very high: 181.7489 [2023-12-27 17:07:13,980][20381] Updated weights for policy 1, policy_version 1654703 (0.0006) [2023-12-27 17:07:13,993][20347] Updated weights for policy 0, policy_version 1656283 (0.0006) [2023-12-27 17:07:14,051][20347] Updated weights for policy 0, policy_version 1656293 (0.0008) [2023-12-27 17:07:14,646][20381] Updated weights for policy 1, policy_version 1654713 (0.0006) [2023-12-27 17:07:14,693][20347] Updated weights for policy 0, policy_version 1656303 (0.0007) [2023-12-27 17:07:14,703][20381] Updated weights for policy 1, policy_version 1654723 (0.0006) [2023-12-27 17:07:14,747][20347] Updated weights for policy 0, policy_version 1656313 (0.0005) [2023-12-27 17:07:14,763][20381] Updated weights for policy 1, policy_version 1654733 (0.0006) [2023-12-27 17:07:14,813][20347] Updated weights for policy 0, policy_version 1656323 (0.0007) [2023-12-27 17:07:15,430][20381] Updated weights for policy 1, policy_version 1654743 (0.0008) [2023-12-27 17:07:15,482][20381] Updated weights for policy 1, policy_version 1654753 (0.0007) [2023-12-27 17:07:15,491][20347] Updated weights for policy 0, policy_version 1656333 (0.0007) [2023-12-27 17:07:15,531][20381] Updated weights for policy 1, policy_version 1654763 (0.0006) [2023-12-27 17:07:15,536][20347] Updated weights for policy 0, policy_version 1656343 (0.0006) [2023-12-27 17:07:15,580][20347] Updated weights for policy 0, policy_version 1656353 (0.0006) [2023-12-27 17:07:16,168][20381] Updated weights for policy 1, policy_version 1654773 (0.0006) [2023-12-27 17:07:16,227][20381] Updated weights for policy 1, policy_version 1654783 (0.0008) [2023-12-27 17:07:16,260][20347] Updated weights for policy 0, policy_version 1656363 (0.0010) [2023-12-27 17:07:16,285][20381] Updated weights for policy 1, policy_version 1654793 (0.0005) [2023-12-27 17:07:16,311][20291] KL-divergence is very high: 203.0738 [2023-12-27 17:07:16,317][20347] Updated weights for policy 0, policy_version 1656373 (0.0010) [2023-12-27 17:07:16,334][20291] KL-divergence is very high: 233.5724 [2023-12-27 17:07:16,347][20291] KL-divergence is very high: 204.7273 [2023-12-27 17:07:16,360][20291] KL-divergence is very high: 270.1475 [2023-12-27 17:07:16,379][20347] Updated weights for policy 0, policy_version 1656383 (0.0010) [2023-12-27 17:07:16,388][20291] KL-divergence is very high: 131.0208 [2023-12-27 17:07:16,858][20381] Updated weights for policy 1, policy_version 1654803 (0.0005) [2023-12-27 17:07:16,916][20381] Updated weights for policy 1, policy_version 1654813 (0.0007) [2023-12-27 17:07:16,968][20381] Updated weights for policy 1, policy_version 1654823 (0.0006) [2023-12-27 17:07:17,010][20347] Updated weights for policy 0, policy_version 1656393 (0.0006) [2023-12-27 17:07:17,056][20347] Updated weights for policy 0, policy_version 1656403 (0.0005) [2023-12-27 17:07:17,107][20347] Updated weights for policy 0, policy_version 1656413 (0.0005) [2023-12-27 17:07:17,165][20347] Updated weights for policy 0, policy_version 1656423 (0.0007) [2023-12-27 17:07:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 847806464. Throughput: 0: 10216.2, 1: 10231.4. Samples: 847814128. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:17,363][19170] Avg episode reward: [(0, '980.530'), (1, '716.810')] [2023-12-27 17:07:17,550][20381] Updated weights for policy 1, policy_version 1654833 (0.0007) [2023-12-27 17:07:17,613][20381] Updated weights for policy 1, policy_version 1654843 (0.0008) [2023-12-27 17:07:17,682][20381] Updated weights for policy 1, policy_version 1654853 (0.0008) [2023-12-27 17:07:17,751][20381] Updated weights for policy 1, policy_version 1654863 (0.0006) [2023-12-27 17:07:17,833][20347] Updated weights for policy 0, policy_version 1656433 (0.0010) [2023-12-27 17:07:17,881][20347] Updated weights for policy 0, policy_version 1656443 (0.0010) [2023-12-27 17:07:17,926][20347] Updated weights for policy 0, policy_version 1656453 (0.0010) [2023-12-27 17:07:18,383][20381] Updated weights for policy 1, policy_version 1654873 (0.0009) [2023-12-27 17:07:18,442][20381] Updated weights for policy 1, policy_version 1654883 (0.0010) [2023-12-27 17:07:18,507][20381] Updated weights for policy 1, policy_version 1654893 (0.0011) [2023-12-27 17:07:18,645][20347] Updated weights for policy 0, policy_version 1656463 (0.0010) [2023-12-27 17:07:18,702][20347] Updated weights for policy 0, policy_version 1656473 (0.0011) [2023-12-27 17:07:18,757][20347] Updated weights for policy 0, policy_version 1656483 (0.0010) [2023-12-27 17:07:19,183][20381] Updated weights for policy 1, policy_version 1654903 (0.0007) [2023-12-27 17:07:19,247][20381] Updated weights for policy 1, policy_version 1654913 (0.0008) [2023-12-27 17:07:19,307][20381] Updated weights for policy 1, policy_version 1654923 (0.0008) [2023-12-27 17:07:19,423][20347] Updated weights for policy 0, policy_version 1656493 (0.0010) [2023-12-27 17:07:19,484][20347] Updated weights for policy 0, policy_version 1656503 (0.0007) [2023-12-27 17:07:19,543][20347] Updated weights for policy 0, policy_version 1656513 (0.0007) [2023-12-27 17:07:20,008][20381] Updated weights for policy 1, policy_version 1654933 (0.0007) [2023-12-27 17:07:20,067][20381] Updated weights for policy 1, policy_version 1654943 (0.0008) [2023-12-27 17:07:20,128][20381] Updated weights for policy 1, policy_version 1654953 (0.0008) [2023-12-27 17:07:20,235][20347] Updated weights for policy 0, policy_version 1656523 (0.0007) [2023-12-27 17:07:20,297][20347] Updated weights for policy 0, policy_version 1656533 (0.0007) [2023-12-27 17:07:20,355][20347] Updated weights for policy 0, policy_version 1656543 (0.0006) [2023-12-27 17:07:20,892][20381] Updated weights for policy 1, policy_version 1654963 (0.0007) [2023-12-27 17:07:20,955][20381] Updated weights for policy 1, policy_version 1654973 (0.0007) [2023-12-27 17:07:21,012][20381] Updated weights for policy 1, policy_version 1654983 (0.0008) [2023-12-27 17:07:21,075][20347] Updated weights for policy 0, policy_version 1656553 (0.0007) [2023-12-27 17:07:21,135][20347] Updated weights for policy 0, policy_version 1656563 (0.0008) [2023-12-27 17:07:21,193][20347] Updated weights for policy 0, policy_version 1656573 (0.0008) [2023-12-27 17:07:21,264][20347] Updated weights for policy 0, policy_version 1656583 (0.0008) [2023-12-27 17:07:21,722][20381] Updated weights for policy 1, policy_version 1654993 (0.0008) [2023-12-27 17:07:21,783][20381] Updated weights for policy 1, policy_version 1655003 (0.0009) [2023-12-27 17:07:21,832][20381] Updated weights for policy 1, policy_version 1655013 (0.0007) [2023-12-27 17:07:21,891][20381] Updated weights for policy 1, policy_version 1655023 (0.0008) [2023-12-27 17:07:22,023][20347] Updated weights for policy 0, policy_version 1656593 (0.0007) [2023-12-27 17:07:22,085][20347] Updated weights for policy 0, policy_version 1656603 (0.0007) [2023-12-27 17:07:22,151][20347] Updated weights for policy 0, policy_version 1656613 (0.0009) [2023-12-27 17:07:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 847904768. Throughput: 0: 10203.6, 1: 10193.4. Samples: 847875148. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:22,363][19170] Avg episode reward: [(0, '951.110'), (1, '717.160')] [2023-12-27 17:07:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001655024_423731200.pth... [2023-12-27 17:07:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001656616_424173568.pth... [2023-12-27 17:07:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001653840_423428096.pth [2023-12-27 17:07:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001655400_423862272.pth [2023-12-27 17:07:22,603][20381] Updated weights for policy 1, policy_version 1655033 (0.0007) [2023-12-27 17:07:22,666][20381] Updated weights for policy 1, policy_version 1655043 (0.0007) [2023-12-27 17:07:22,727][20381] Updated weights for policy 1, policy_version 1655053 (0.0008) [2023-12-27 17:07:22,865][20347] Updated weights for policy 0, policy_version 1656623 (0.0008) [2023-12-27 17:07:22,925][20347] Updated weights for policy 0, policy_version 1656633 (0.0008) [2023-12-27 17:07:22,982][20347] Updated weights for policy 0, policy_version 1656643 (0.0008) [2023-12-27 17:07:23,398][20381] Updated weights for policy 1, policy_version 1655063 (0.0006) [2023-12-27 17:07:23,453][20381] Updated weights for policy 1, policy_version 1655073 (0.0006) [2023-12-27 17:07:23,508][20381] Updated weights for policy 1, policy_version 1655083 (0.0007) [2023-12-27 17:07:23,700][20347] Updated weights for policy 0, policy_version 1656653 (0.0007) [2023-12-27 17:07:23,756][20347] Updated weights for policy 0, policy_version 1656663 (0.0007) [2023-12-27 17:07:23,815][20347] Updated weights for policy 0, policy_version 1656673 (0.0008) [2023-12-27 17:07:24,093][20381] Updated weights for policy 1, policy_version 1655093 (0.0006) [2023-12-27 17:07:24,147][20381] Updated weights for policy 1, policy_version 1655103 (0.0008) [2023-12-27 17:07:24,210][20381] Updated weights for policy 1, policy_version 1655113 (0.0010) [2023-12-27 17:07:24,528][20347] Updated weights for policy 0, policy_version 1656683 (0.0008) [2023-12-27 17:07:24,586][20347] Updated weights for policy 0, policy_version 1656693 (0.0010) [2023-12-27 17:07:24,635][20347] Updated weights for policy 0, policy_version 1656703 (0.0007) [2023-12-27 17:07:24,838][20381] Updated weights for policy 1, policy_version 1655123 (0.0010) [2023-12-27 17:07:24,888][20381] Updated weights for policy 1, policy_version 1655133 (0.0010) [2023-12-27 17:07:24,938][20381] Updated weights for policy 1, policy_version 1655143 (0.0010) [2023-12-27 17:07:25,270][20347] Updated weights for policy 0, policy_version 1656713 (0.0007) [2023-12-27 17:07:25,325][20347] Updated weights for policy 0, policy_version 1656723 (0.0009) [2023-12-27 17:07:25,379][20347] Updated weights for policy 0, policy_version 1656733 (0.0007) [2023-12-27 17:07:25,430][20347] Updated weights for policy 0, policy_version 1656743 (0.0007) [2023-12-27 17:07:25,604][20381] Updated weights for policy 1, policy_version 1655153 (0.0011) [2023-12-27 17:07:25,655][20381] Updated weights for policy 1, policy_version 1655163 (0.0010) [2023-12-27 17:07:25,707][20381] Updated weights for policy 1, policy_version 1655173 (0.0010) [2023-12-27 17:07:25,765][20381] Updated weights for policy 1, policy_version 1655183 (0.0011) [2023-12-27 17:07:26,078][20347] Updated weights for policy 0, policy_version 1656753 (0.0005) [2023-12-27 17:07:26,137][20347] Updated weights for policy 0, policy_version 1656763 (0.0010) [2023-12-27 17:07:26,198][20347] Updated weights for policy 0, policy_version 1656773 (0.0010) [2023-12-27 17:07:26,377][20381] Updated weights for policy 1, policy_version 1655193 (0.0007) [2023-12-27 17:07:26,424][20381] Updated weights for policy 1, policy_version 1655203 (0.0010) [2023-12-27 17:07:26,472][20381] Updated weights for policy 1, policy_version 1655213 (0.0010) [2023-12-27 17:07:26,844][20347] Updated weights for policy 0, policy_version 1656783 (0.0010) [2023-12-27 17:07:26,907][20347] Updated weights for policy 0, policy_version 1656793 (0.0010) [2023-12-27 17:07:26,964][20347] Updated weights for policy 0, policy_version 1656803 (0.0010) [2023-12-27 17:07:27,018][20381] Updated weights for policy 1, policy_version 1655223 (0.0007) [2023-12-27 17:07:27,068][20381] Updated weights for policy 1, policy_version 1655233 (0.0005) [2023-12-27 17:07:27,118][20381] Updated weights for policy 1, policy_version 1655243 (0.0005) [2023-12-27 17:07:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 848011264. Throughput: 0: 10164.0, 1: 10232.7. Samples: 847996568. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:27,363][19170] Avg episode reward: [(0, '960.810'), (1, '707.420')] [2023-12-27 17:07:27,616][20347] Updated weights for policy 0, policy_version 1656813 (0.0010) [2023-12-27 17:07:27,670][20347] Updated weights for policy 0, policy_version 1656823 (0.0010) [2023-12-27 17:07:27,691][20381] Updated weights for policy 1, policy_version 1655253 (0.0008) [2023-12-27 17:07:27,733][20347] Updated weights for policy 0, policy_version 1656833 (0.0010) [2023-12-27 17:07:27,744][20381] Updated weights for policy 1, policy_version 1655263 (0.0008) [2023-12-27 17:07:27,796][20381] Updated weights for policy 1, policy_version 1655273 (0.0006) [2023-12-27 17:07:28,341][20381] Updated weights for policy 1, policy_version 1655283 (0.0007) [2023-12-27 17:07:28,397][20381] Updated weights for policy 1, policy_version 1655293 (0.0011) [2023-12-27 17:07:28,427][20347] Updated weights for policy 0, policy_version 1656843 (0.0010) [2023-12-27 17:07:28,455][20381] Updated weights for policy 1, policy_version 1655303 (0.0007) [2023-12-27 17:07:28,480][20347] Updated weights for policy 0, policy_version 1656853 (0.0010) [2023-12-27 17:07:28,534][20347] Updated weights for policy 0, policy_version 1656863 (0.0010) [2023-12-27 17:07:29,104][20381] Updated weights for policy 1, policy_version 1655313 (0.0006) [2023-12-27 17:07:29,161][20381] Updated weights for policy 1, policy_version 1655323 (0.0008) [2023-12-27 17:07:29,216][20381] Updated weights for policy 1, policy_version 1655333 (0.0006) [2023-12-27 17:07:29,258][20347] Updated weights for policy 0, policy_version 1656873 (0.0010) [2023-12-27 17:07:29,280][20381] Updated weights for policy 1, policy_version 1655343 (0.0008) [2023-12-27 17:07:29,314][20347] Updated weights for policy 0, policy_version 1656883 (0.0006) [2023-12-27 17:07:29,373][20347] Updated weights for policy 0, policy_version 1656893 (0.0007) [2023-12-27 17:07:29,429][20347] Updated weights for policy 0, policy_version 1656903 (0.0005) [2023-12-27 17:07:30,014][20381] Updated weights for policy 1, policy_version 1655353 (0.0008) [2023-12-27 17:07:30,068][20381] Updated weights for policy 1, policy_version 1655363 (0.0008) [2023-12-27 17:07:30,093][20347] Updated weights for policy 0, policy_version 1656913 (0.0006) [2023-12-27 17:07:30,120][20381] Updated weights for policy 1, policy_version 1655373 (0.0010) [2023-12-27 17:07:30,149][20347] Updated weights for policy 0, policy_version 1656923 (0.0006) [2023-12-27 17:07:30,198][20347] Updated weights for policy 0, policy_version 1656933 (0.0007) [2023-12-27 17:07:30,798][20381] Updated weights for policy 1, policy_version 1655383 (0.0010) [2023-12-27 17:07:30,852][20381] Updated weights for policy 1, policy_version 1655393 (0.0010) [2023-12-27 17:07:30,864][20347] Updated weights for policy 0, policy_version 1656943 (0.0007) [2023-12-27 17:07:30,908][20381] Updated weights for policy 1, policy_version 1655403 (0.0009) [2023-12-27 17:07:30,916][20347] Updated weights for policy 0, policy_version 1656953 (0.0008) [2023-12-27 17:07:30,969][20347] Updated weights for policy 0, policy_version 1656963 (0.0006) [2023-12-27 17:07:31,584][20381] Updated weights for policy 1, policy_version 1655413 (0.0009) [2023-12-27 17:07:31,644][20381] Updated weights for policy 1, policy_version 1655423 (0.0007) [2023-12-27 17:07:31,681][20347] Updated weights for policy 0, policy_version 1656973 (0.0008) [2023-12-27 17:07:31,699][20381] Updated weights for policy 1, policy_version 1655433 (0.0006) [2023-12-27 17:07:31,740][20347] Updated weights for policy 0, policy_version 1656983 (0.0007) [2023-12-27 17:07:31,795][20347] Updated weights for policy 0, policy_version 1656993 (0.0008) [2023-12-27 17:07:32,308][20381] Updated weights for policy 1, policy_version 1655443 (0.0008) [2023-12-27 17:07:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 848109568. Throughput: 0: 10188.5, 1: 10284.7. Samples: 848123128. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:32,363][19170] Avg episode reward: [(0, '970.650'), (1, '765.860')] [2023-12-27 17:07:32,373][20381] Updated weights for policy 1, policy_version 1655453 (0.0010) [2023-12-27 17:07:32,436][20381] Updated weights for policy 1, policy_version 1655463 (0.0010) [2023-12-27 17:07:32,565][20347] Updated weights for policy 0, policy_version 1657003 (0.0007) [2023-12-27 17:07:32,626][20347] Updated weights for policy 0, policy_version 1657013 (0.0007) [2023-12-27 17:07:32,689][20347] Updated weights for policy 0, policy_version 1657023 (0.0006) [2023-12-27 17:07:33,126][20381] Updated weights for policy 1, policy_version 1655473 (0.0009) [2023-12-27 17:07:33,176][20381] Updated weights for policy 1, policy_version 1655483 (0.0007) [2023-12-27 17:07:33,243][20381] Updated weights for policy 1, policy_version 1655493 (0.0007) [2023-12-27 17:07:33,250][20347] Updated weights for policy 0, policy_version 1657033 (0.0006) [2023-12-27 17:07:33,294][20381] Updated weights for policy 1, policy_version 1655503 (0.0007) [2023-12-27 17:07:33,307][20347] Updated weights for policy 0, policy_version 1657043 (0.0005) [2023-12-27 17:07:33,354][20347] Updated weights for policy 0, policy_version 1657053 (0.0006) [2023-12-27 17:07:33,406][20347] Updated weights for policy 0, policy_version 1657063 (0.0006) [2023-12-27 17:07:33,870][20381] Updated weights for policy 1, policy_version 1655513 (0.0007) [2023-12-27 17:07:33,923][20381] Updated weights for policy 1, policy_version 1655523 (0.0006) [2023-12-27 17:07:33,973][20381] Updated weights for policy 1, policy_version 1655533 (0.0005) [2023-12-27 17:07:34,110][20347] Updated weights for policy 0, policy_version 1657073 (0.0008) [2023-12-27 17:07:34,172][20347] Updated weights for policy 0, policy_version 1657083 (0.0010) [2023-12-27 17:07:34,229][20347] Updated weights for policy 0, policy_version 1657093 (0.0008) [2023-12-27 17:07:34,656][20381] Updated weights for policy 1, policy_version 1655543 (0.0007) [2023-12-27 17:07:34,716][20381] Updated weights for policy 1, policy_version 1655553 (0.0008) [2023-12-27 17:07:34,777][20381] Updated weights for policy 1, policy_version 1655563 (0.0008) [2023-12-27 17:07:34,909][20347] Updated weights for policy 0, policy_version 1657103 (0.0007) [2023-12-27 17:07:34,967][20347] Updated weights for policy 0, policy_version 1657113 (0.0007) [2023-12-27 17:07:35,024][20347] Updated weights for policy 0, policy_version 1657123 (0.0008) [2023-12-27 17:07:35,413][20381] Updated weights for policy 1, policy_version 1655573 (0.0006) [2023-12-27 17:07:35,467][20381] Updated weights for policy 1, policy_version 1655583 (0.0007) [2023-12-27 17:07:35,512][20381] Updated weights for policy 1, policy_version 1655593 (0.0008) [2023-12-27 17:07:35,712][20347] Updated weights for policy 0, policy_version 1657133 (0.0008) [2023-12-27 17:07:35,769][20347] Updated weights for policy 0, policy_version 1657143 (0.0008) [2023-12-27 17:07:35,828][20347] Updated weights for policy 0, policy_version 1657153 (0.0008) [2023-12-27 17:07:36,153][20381] Updated weights for policy 1, policy_version 1655603 (0.0007) [2023-12-27 17:07:36,212][20381] Updated weights for policy 1, policy_version 1655613 (0.0006) [2023-12-27 17:07:36,270][20381] Updated weights for policy 1, policy_version 1655623 (0.0007) [2023-12-27 17:07:36,560][20347] Updated weights for policy 0, policy_version 1657163 (0.0008) [2023-12-27 17:07:36,622][20347] Updated weights for policy 0, policy_version 1657173 (0.0011) [2023-12-27 17:07:36,685][20347] Updated weights for policy 0, policy_version 1657183 (0.0011) [2023-12-27 17:07:36,934][20381] Updated weights for policy 1, policy_version 1655633 (0.0006) [2023-12-27 17:07:36,986][20381] Updated weights for policy 1, policy_version 1655643 (0.0010) [2023-12-27 17:07:37,047][20381] Updated weights for policy 1, policy_version 1655653 (0.0010) [2023-12-27 17:07:37,105][20381] Updated weights for policy 1, policy_version 1655663 (0.0010) [2023-12-27 17:07:37,309][20347] Updated weights for policy 0, policy_version 1657193 (0.0010) [2023-12-27 17:07:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 848216064. Throughput: 0: 10202.1, 1: 10305.7. Samples: 848185472. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:37,363][19170] Avg episode reward: [(0, '931.610'), (1, '726.630')] [2023-12-27 17:07:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001655664_423895040.pth... [2023-12-27 17:07:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001654448_423583744.pth [2023-12-27 17:07:37,384][20347] Updated weights for policy 0, policy_version 1657203 (0.0006) [2023-12-27 17:07:37,449][20347] Updated weights for policy 0, policy_version 1657213 (0.0008) [2023-12-27 17:07:37,516][20347] Updated weights for policy 0, policy_version 1657223 (0.0007) [2023-12-27 17:07:37,518][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001657224_424329216.pth... [2023-12-27 17:07:37,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001656008_424017920.pth [2023-12-27 17:07:37,809][20381] Updated weights for policy 1, policy_version 1655673 (0.0011) [2023-12-27 17:07:37,874][20381] Updated weights for policy 1, policy_version 1655683 (0.0011) [2023-12-27 17:07:37,933][20381] Updated weights for policy 1, policy_version 1655693 (0.0011) [2023-12-27 17:07:38,078][20347] Updated weights for policy 0, policy_version 1657233 (0.0006) [2023-12-27 17:07:38,132][20347] Updated weights for policy 0, policy_version 1657243 (0.0007) [2023-12-27 17:07:38,188][20347] Updated weights for policy 0, policy_version 1657253 (0.0005) [2023-12-27 17:07:38,613][20381] Updated weights for policy 1, policy_version 1655703 (0.0011) [2023-12-27 17:07:38,665][20381] Updated weights for policy 1, policy_version 1655713 (0.0011) [2023-12-27 17:07:38,728][20381] Updated weights for policy 1, policy_version 1655723 (0.0011) [2023-12-27 17:07:38,861][20347] Updated weights for policy 0, policy_version 1657263 (0.0006) [2023-12-27 17:07:38,924][20347] Updated weights for policy 0, policy_version 1657273 (0.0006) [2023-12-27 17:07:38,977][20347] Updated weights for policy 0, policy_version 1657283 (0.0011) [2023-12-27 17:07:39,427][20381] Updated weights for policy 1, policy_version 1655733 (0.0011) [2023-12-27 17:07:39,490][20381] Updated weights for policy 1, policy_version 1655743 (0.0011) [2023-12-27 17:07:39,555][20381] Updated weights for policy 1, policy_version 1655753 (0.0011) [2023-12-27 17:07:39,677][20347] Updated weights for policy 0, policy_version 1657293 (0.0009) [2023-12-27 17:07:39,738][20347] Updated weights for policy 0, policy_version 1657303 (0.0007) [2023-12-27 17:07:39,798][20347] Updated weights for policy 0, policy_version 1657313 (0.0008) [2023-12-27 17:07:40,289][20381] Updated weights for policy 1, policy_version 1655763 (0.0011) [2023-12-27 17:07:40,348][20381] Updated weights for policy 1, policy_version 1655773 (0.0009) [2023-12-27 17:07:40,416][20381] Updated weights for policy 1, policy_version 1655783 (0.0008) [2023-12-27 17:07:40,503][20347] Updated weights for policy 0, policy_version 1657323 (0.0007) [2023-12-27 17:07:40,556][20347] Updated weights for policy 0, policy_version 1657333 (0.0005) [2023-12-27 17:07:40,620][20347] Updated weights for policy 0, policy_version 1657343 (0.0005) [2023-12-27 17:07:40,989][20381] Updated weights for policy 1, policy_version 1655793 (0.0011) [2023-12-27 17:07:41,044][20381] Updated weights for policy 1, policy_version 1655803 (0.0010) [2023-12-27 17:07:41,099][20381] Updated weights for policy 1, policy_version 1655813 (0.0008) [2023-12-27 17:07:41,163][20381] Updated weights for policy 1, policy_version 1655823 (0.0009) [2023-12-27 17:07:41,238][20347] Updated weights for policy 0, policy_version 1657353 (0.0005) [2023-12-27 17:07:41,302][20347] Updated weights for policy 0, policy_version 1657363 (0.0008) [2023-12-27 17:07:41,371][20347] Updated weights for policy 0, policy_version 1657373 (0.0007) [2023-12-27 17:07:41,433][20347] Updated weights for policy 0, policy_version 1657383 (0.0007) [2023-12-27 17:07:41,940][20381] Updated weights for policy 1, policy_version 1655833 (0.0006) [2023-12-27 17:07:41,996][20381] Updated weights for policy 1, policy_version 1655843 (0.0006) [2023-12-27 17:07:42,050][20381] Updated weights for policy 1, policy_version 1655853 (0.0006) [2023-12-27 17:07:42,169][20347] Updated weights for policy 0, policy_version 1657393 (0.0008) [2023-12-27 17:07:42,227][20347] Updated weights for policy 0, policy_version 1657403 (0.0007) [2023-12-27 17:07:42,289][20347] Updated weights for policy 0, policy_version 1657413 (0.0008) [2023-12-27 17:07:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 848322560. Throughput: 0: 10255.5, 1: 10315.9. Samples: 848309312. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:42,363][19170] Avg episode reward: [(0, '912.090'), (1, '746.400')] [2023-12-27 17:07:42,737][20381] Updated weights for policy 1, policy_version 1655863 (0.0010) [2023-12-27 17:07:42,794][20381] Updated weights for policy 1, policy_version 1655873 (0.0011) [2023-12-27 17:07:42,841][20381] Updated weights for policy 1, policy_version 1655883 (0.0011) [2023-12-27 17:07:42,989][20347] Updated weights for policy 0, policy_version 1657423 (0.0007) [2023-12-27 17:07:43,048][20347] Updated weights for policy 0, policy_version 1657433 (0.0007) [2023-12-27 17:07:43,097][20347] Updated weights for policy 0, policy_version 1657443 (0.0009) [2023-12-27 17:07:43,495][20381] Updated weights for policy 1, policy_version 1655893 (0.0010) [2023-12-27 17:07:43,546][20381] Updated weights for policy 1, policy_version 1655903 (0.0010) [2023-12-27 17:07:43,597][20381] Updated weights for policy 1, policy_version 1655913 (0.0010) [2023-12-27 17:07:43,811][20347] Updated weights for policy 0, policy_version 1657453 (0.0008) [2023-12-27 17:07:43,873][20347] Updated weights for policy 0, policy_version 1657463 (0.0007) [2023-12-27 17:07:43,927][20347] Updated weights for policy 0, policy_version 1657473 (0.0008) [2023-12-27 17:07:44,269][20381] Updated weights for policy 1, policy_version 1655923 (0.0010) [2023-12-27 17:07:44,331][20381] Updated weights for policy 1, policy_version 1655933 (0.0011) [2023-12-27 17:07:44,383][20381] Updated weights for policy 1, policy_version 1655943 (0.0010) [2023-12-27 17:07:44,592][20347] Updated weights for policy 0, policy_version 1657483 (0.0007) [2023-12-27 17:07:44,640][20347] Updated weights for policy 0, policy_version 1657493 (0.0010) [2023-12-27 17:07:44,698][20347] Updated weights for policy 0, policy_version 1657503 (0.0010) [2023-12-27 17:07:45,084][20381] Updated weights for policy 1, policy_version 1655953 (0.0010) [2023-12-27 17:07:45,144][20381] Updated weights for policy 1, policy_version 1655963 (0.0011) [2023-12-27 17:07:45,210][20381] Updated weights for policy 1, policy_version 1655973 (0.0011) [2023-12-27 17:07:45,266][20381] Updated weights for policy 1, policy_version 1655983 (0.0008) [2023-12-27 17:07:45,327][20347] Updated weights for policy 0, policy_version 1657513 (0.0010) [2023-12-27 17:07:45,384][20347] Updated weights for policy 0, policy_version 1657523 (0.0006) [2023-12-27 17:07:45,439][20347] Updated weights for policy 0, policy_version 1657533 (0.0008) [2023-12-27 17:07:45,527][20347] Updated weights for policy 0, policy_version 1657543 (0.0009) [2023-12-27 17:07:45,947][20381] Updated weights for policy 1, policy_version 1655993 (0.0006) [2023-12-27 17:07:46,005][20381] Updated weights for policy 1, policy_version 1656003 (0.0009) [2023-12-27 17:07:46,057][20381] Updated weights for policy 1, policy_version 1656013 (0.0011) [2023-12-27 17:07:46,094][20347] Updated weights for policy 0, policy_version 1657553 (0.0006) [2023-12-27 17:07:46,145][20347] Updated weights for policy 0, policy_version 1657563 (0.0005) [2023-12-27 17:07:46,205][20347] Updated weights for policy 0, policy_version 1657573 (0.0005) [2023-12-27 17:07:46,707][20381] Updated weights for policy 1, policy_version 1656023 (0.0011) [2023-12-27 17:07:46,755][20381] Updated weights for policy 1, policy_version 1656033 (0.0010) [2023-12-27 17:07:46,771][20347] Updated weights for policy 0, policy_version 1657583 (0.0006) [2023-12-27 17:07:46,804][20381] Updated weights for policy 1, policy_version 1656043 (0.0010) [2023-12-27 17:07:46,829][20347] Updated weights for policy 0, policy_version 1657593 (0.0006) [2023-12-27 17:07:46,882][20347] Updated weights for policy 0, policy_version 1657603 (0.0007) [2023-12-27 17:07:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 848420864. Throughput: 0: 10274.9, 1: 10310.6. Samples: 848433120. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:47,363][19170] Avg episode reward: [(0, '960.900'), (1, '843.890')] [2023-12-27 17:07:47,487][20381] Updated weights for policy 1, policy_version 1656053 (0.0011) [2023-12-27 17:07:47,515][20347] Updated weights for policy 0, policy_version 1657613 (0.0008) [2023-12-27 17:07:47,541][20381] Updated weights for policy 1, policy_version 1656063 (0.0011) [2023-12-27 17:07:47,572][20347] Updated weights for policy 0, policy_version 1657623 (0.0007) [2023-12-27 17:07:47,598][20381] Updated weights for policy 1, policy_version 1656073 (0.0011) [2023-12-27 17:07:47,623][20347] Updated weights for policy 0, policy_version 1657633 (0.0007) [2023-12-27 17:07:48,257][20381] Updated weights for policy 1, policy_version 1656083 (0.0011) [2023-12-27 17:07:48,306][20381] Updated weights for policy 1, policy_version 1656093 (0.0010) [2023-12-27 17:07:48,330][20347] Updated weights for policy 0, policy_version 1657643 (0.0007) [2023-12-27 17:07:48,368][20381] Updated weights for policy 1, policy_version 1656103 (0.0008) [2023-12-27 17:07:48,391][20347] Updated weights for policy 0, policy_version 1657653 (0.0006) [2023-12-27 17:07:48,448][20347] Updated weights for policy 0, policy_version 1657663 (0.0005) [2023-12-27 17:07:49,054][20381] Updated weights for policy 1, policy_version 1656113 (0.0009) [2023-12-27 17:07:49,095][20347] Updated weights for policy 0, policy_version 1657673 (0.0005) [2023-12-27 17:07:49,106][20381] Updated weights for policy 1, policy_version 1656123 (0.0010) [2023-12-27 17:07:49,149][20347] Updated weights for policy 0, policy_version 1657683 (0.0005) [2023-12-27 17:07:49,151][20381] Updated weights for policy 1, policy_version 1656133 (0.0010) [2023-12-27 17:07:49,200][20347] Updated weights for policy 0, policy_version 1657693 (0.0006) [2023-12-27 17:07:49,206][20381] Updated weights for policy 1, policy_version 1656143 (0.0010) [2023-12-27 17:07:49,261][20347] Updated weights for policy 0, policy_version 1657703 (0.0009) [2023-12-27 17:07:49,946][20381] Updated weights for policy 1, policy_version 1656153 (0.0008) [2023-12-27 17:07:49,957][20347] Updated weights for policy 0, policy_version 1657713 (0.0008) [2023-12-27 17:07:50,008][20381] Updated weights for policy 1, policy_version 1656163 (0.0007) [2023-12-27 17:07:50,022][20347] Updated weights for policy 0, policy_version 1657723 (0.0008) [2023-12-27 17:07:50,068][20381] Updated weights for policy 1, policy_version 1656173 (0.0006) [2023-12-27 17:07:50,077][20347] Updated weights for policy 0, policy_version 1657733 (0.0008) [2023-12-27 17:07:50,725][20347] Updated weights for policy 0, policy_version 1657743 (0.0010) [2023-12-27 17:07:50,778][20381] Updated weights for policy 1, policy_version 1656183 (0.0009) [2023-12-27 17:07:50,788][20347] Updated weights for policy 0, policy_version 1657753 (0.0011) [2023-12-27 17:07:50,836][20381] Updated weights for policy 1, policy_version 1656193 (0.0011) [2023-12-27 17:07:50,854][20347] Updated weights for policy 0, policy_version 1657763 (0.0010) [2023-12-27 17:07:50,899][20381] Updated weights for policy 1, policy_version 1656203 (0.0011) [2023-12-27 17:07:51,612][20347] Updated weights for policy 0, policy_version 1657773 (0.0009) [2023-12-27 17:07:51,637][20381] Updated weights for policy 1, policy_version 1656213 (0.0011) [2023-12-27 17:07:51,671][20347] Updated weights for policy 0, policy_version 1657783 (0.0009) [2023-12-27 17:07:51,697][20381] Updated weights for policy 1, policy_version 1656223 (0.0010) [2023-12-27 17:07:51,730][20347] Updated weights for policy 0, policy_version 1657793 (0.0007) [2023-12-27 17:07:51,760][20381] Updated weights for policy 1, policy_version 1656233 (0.0009) [2023-12-27 17:07:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 848519168. Throughput: 0: 10270.2, 1: 10312.6. Samples: 848493812. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:52,363][19170] Avg episode reward: [(0, '951.160'), (1, '726.610')] [2023-12-27 17:07:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001656240_424042496.pth... [2023-12-27 17:07:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001657800_424476672.pth... [2023-12-27 17:07:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001656616_424173568.pth [2023-12-27 17:07:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001655024_423731200.pth [2023-12-27 17:07:52,405][20381] Updated weights for policy 1, policy_version 1656243 (0.0006) [2023-12-27 17:07:52,411][20347] Updated weights for policy 0, policy_version 1657803 (0.0008) [2023-12-27 17:07:52,455][20381] Updated weights for policy 1, policy_version 1656253 (0.0005) [2023-12-27 17:07:52,466][20347] Updated weights for policy 0, policy_version 1657813 (0.0006) [2023-12-27 17:07:52,507][20381] Updated weights for policy 1, policy_version 1656263 (0.0008) [2023-12-27 17:07:52,514][20347] Updated weights for policy 0, policy_version 1657823 (0.0006) [2023-12-27 17:07:53,160][20381] Updated weights for policy 1, policy_version 1656273 (0.0007) [2023-12-27 17:07:53,207][20381] Updated weights for policy 1, policy_version 1656283 (0.0006) [2023-12-27 17:07:53,216][20347] Updated weights for policy 0, policy_version 1657833 (0.0009) [2023-12-27 17:07:53,259][20381] Updated weights for policy 1, policy_version 1656293 (0.0005) [2023-12-27 17:07:53,274][20347] Updated weights for policy 0, policy_version 1657843 (0.0010) [2023-12-27 17:07:53,309][20381] Updated weights for policy 1, policy_version 1656303 (0.0006) [2023-12-27 17:07:53,332][20347] Updated weights for policy 0, policy_version 1657853 (0.0010) [2023-12-27 17:07:53,386][20347] Updated weights for policy 0, policy_version 1657863 (0.0010) [2023-12-27 17:07:53,986][20347] Updated weights for policy 0, policy_version 1657873 (0.0008) [2023-12-27 17:07:54,007][20381] Updated weights for policy 1, policy_version 1656313 (0.0009) [2023-12-27 17:07:54,039][20347] Updated weights for policy 0, policy_version 1657883 (0.0010) [2023-12-27 17:07:54,062][20381] Updated weights for policy 1, policy_version 1656323 (0.0010) [2023-12-27 17:07:54,095][20347] Updated weights for policy 0, policy_version 1657893 (0.0010) [2023-12-27 17:07:54,112][20381] Updated weights for policy 1, policy_version 1656333 (0.0006) [2023-12-27 17:07:54,691][20381] Updated weights for policy 1, policy_version 1656343 (0.0010) [2023-12-27 17:07:54,749][20381] Updated weights for policy 1, policy_version 1656353 (0.0011) [2023-12-27 17:07:54,765][20347] Updated weights for policy 0, policy_version 1657903 (0.0010) [2023-12-27 17:07:54,805][20381] Updated weights for policy 1, policy_version 1656363 (0.0011) [2023-12-27 17:07:54,821][20347] Updated weights for policy 0, policy_version 1657913 (0.0010) [2023-12-27 17:07:54,877][20347] Updated weights for policy 0, policy_version 1657923 (0.0010) [2023-12-27 17:07:55,477][20381] Updated weights for policy 1, policy_version 1656373 (0.0011) [2023-12-27 17:07:55,505][20347] Updated weights for policy 0, policy_version 1657933 (0.0008) [2023-12-27 17:07:55,526][20381] Updated weights for policy 1, policy_version 1656383 (0.0011) [2023-12-27 17:07:55,556][20347] Updated weights for policy 0, policy_version 1657943 (0.0006) [2023-12-27 17:07:55,576][20381] Updated weights for policy 1, policy_version 1656393 (0.0011) [2023-12-27 17:07:55,614][20347] Updated weights for policy 0, policy_version 1657953 (0.0006) [2023-12-27 17:07:56,190][20381] Updated weights for policy 1, policy_version 1656403 (0.0010) [2023-12-27 17:07:56,228][20347] Updated weights for policy 0, policy_version 1657963 (0.0005) [2023-12-27 17:07:56,241][20381] Updated weights for policy 1, policy_version 1656413 (0.0008) [2023-12-27 17:07:56,285][20347] Updated weights for policy 0, policy_version 1657973 (0.0005) [2023-12-27 17:07:56,289][20381] Updated weights for policy 1, policy_version 1656423 (0.0008) [2023-12-27 17:07:56,341][20347] Updated weights for policy 0, policy_version 1657983 (0.0006) [2023-12-27 17:07:56,926][20381] Updated weights for policy 1, policy_version 1656433 (0.0007) [2023-12-27 17:07:56,983][20381] Updated weights for policy 1, policy_version 1656443 (0.0006) [2023-12-27 17:07:57,000][20347] Updated weights for policy 0, policy_version 1657993 (0.0009) [2023-12-27 17:07:57,036][20381] Updated weights for policy 1, policy_version 1656453 (0.0006) [2023-12-27 17:07:57,052][20347] Updated weights for policy 0, policy_version 1658003 (0.0010) [2023-12-27 17:07:57,080][20381] Updated weights for policy 1, policy_version 1656463 (0.0005) [2023-12-27 17:07:57,102][20347] Updated weights for policy 0, policy_version 1658013 (0.0010) [2023-12-27 17:07:57,157][20347] Updated weights for policy 0, policy_version 1658023 (0.0009) [2023-12-27 17:07:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 848633856. Throughput: 0: 10266.1, 1: 10395.1. Samples: 848619804. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:07:57,363][19170] Avg episode reward: [(0, '960.930'), (1, '726.780')] [2023-12-27 17:07:57,719][20381] Updated weights for policy 1, policy_version 1656473 (0.0010) [2023-12-27 17:07:57,746][20347] Updated weights for policy 0, policy_version 1658033 (0.0005) [2023-12-27 17:07:57,768][20381] Updated weights for policy 1, policy_version 1656483 (0.0008) [2023-12-27 17:07:57,802][20347] Updated weights for policy 0, policy_version 1658043 (0.0005) [2023-12-27 17:07:57,820][20381] Updated weights for policy 1, policy_version 1656493 (0.0008) [2023-12-27 17:07:57,857][20347] Updated weights for policy 0, policy_version 1658053 (0.0005) [2023-12-27 17:07:58,439][20347] Updated weights for policy 0, policy_version 1658063 (0.0007) [2023-12-27 17:07:58,506][20347] Updated weights for policy 0, policy_version 1658073 (0.0008) [2023-12-27 17:07:58,560][20381] Updated weights for policy 1, policy_version 1656503 (0.0008) [2023-12-27 17:07:58,574][20347] Updated weights for policy 0, policy_version 1658083 (0.0008) [2023-12-27 17:07:58,623][20381] Updated weights for policy 1, policy_version 1656513 (0.0008) [2023-12-27 17:07:58,687][20381] Updated weights for policy 1, policy_version 1656523 (0.0007) [2023-12-27 17:07:59,309][20347] Updated weights for policy 0, policy_version 1658093 (0.0006) [2023-12-27 17:07:59,379][20347] Updated weights for policy 0, policy_version 1658103 (0.0009) [2023-12-27 17:07:59,434][20347] Updated weights for policy 0, policy_version 1658113 (0.0010) [2023-12-27 17:07:59,461][20381] Updated weights for policy 1, policy_version 1656533 (0.0007) [2023-12-27 17:07:59,519][20381] Updated weights for policy 1, policy_version 1656543 (0.0007) [2023-12-27 17:07:59,568][20381] Updated weights for policy 1, policy_version 1656553 (0.0007) [2023-12-27 17:08:00,109][20347] Updated weights for policy 0, policy_version 1658123 (0.0010) [2023-12-27 17:08:00,167][20347] Updated weights for policy 0, policy_version 1658133 (0.0010) [2023-12-27 17:08:00,217][20347] Updated weights for policy 0, policy_version 1658143 (0.0010) [2023-12-27 17:08:00,281][20381] Updated weights for policy 1, policy_version 1656563 (0.0008) [2023-12-27 17:08:00,333][20381] Updated weights for policy 1, policy_version 1656573 (0.0007) [2023-12-27 17:08:00,410][20381] Updated weights for policy 1, policy_version 1656583 (0.0008) [2023-12-27 17:08:00,836][20347] Updated weights for policy 0, policy_version 1658153 (0.0010) [2023-12-27 17:08:00,887][20347] Updated weights for policy 0, policy_version 1658163 (0.0007) [2023-12-27 17:08:00,936][20347] Updated weights for policy 0, policy_version 1658173 (0.0005) [2023-12-27 17:08:00,986][20347] Updated weights for policy 0, policy_version 1658183 (0.0008) [2023-12-27 17:08:01,020][20381] Updated weights for policy 1, policy_version 1656593 (0.0007) [2023-12-27 17:08:01,070][20381] Updated weights for policy 1, policy_version 1656603 (0.0008) [2023-12-27 17:08:01,119][20381] Updated weights for policy 1, policy_version 1656613 (0.0008) [2023-12-27 17:08:01,180][20381] Updated weights for policy 1, policy_version 1656623 (0.0008) [2023-12-27 17:08:01,710][20347] Updated weights for policy 0, policy_version 1658193 (0.0007) [2023-12-27 17:08:01,767][20347] Updated weights for policy 0, policy_version 1658203 (0.0008) [2023-12-27 17:08:01,829][20347] Updated weights for policy 0, policy_version 1658213 (0.0005) [2023-12-27 17:08:01,975][20381] Updated weights for policy 1, policy_version 1656633 (0.0007) [2023-12-27 17:08:02,033][20381] Updated weights for policy 1, policy_version 1656643 (0.0009) [2023-12-27 17:08:02,104][20381] Updated weights for policy 1, policy_version 1656653 (0.0008) [2023-12-27 17:08:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 848732160. Throughput: 0: 10287.5, 1: 10340.5. Samples: 848742388. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:02,363][19170] Avg episode reward: [(0, '980.410'), (1, '736.910')] [2023-12-27 17:08:02,431][20347] Updated weights for policy 0, policy_version 1658223 (0.0009) [2023-12-27 17:08:02,494][20347] Updated weights for policy 0, policy_version 1658233 (0.0008) [2023-12-27 17:08:02,553][20347] Updated weights for policy 0, policy_version 1658243 (0.0009) [2023-12-27 17:08:02,768][20381] Updated weights for policy 1, policy_version 1656663 (0.0010) [2023-12-27 17:08:02,817][20381] Updated weights for policy 1, policy_version 1656673 (0.0010) [2023-12-27 17:08:02,864][20381] Updated weights for policy 1, policy_version 1656683 (0.0010) [2023-12-27 17:08:02,875][20292] KL-divergence is very high: 119.3640 [2023-12-27 17:08:03,168][20347] Updated weights for policy 0, policy_version 1658253 (0.0007) [2023-12-27 17:08:03,227][20347] Updated weights for policy 0, policy_version 1658263 (0.0005) [2023-12-27 17:08:03,273][20347] Updated weights for policy 0, policy_version 1658273 (0.0006) [2023-12-27 17:08:03,556][20381] Updated weights for policy 1, policy_version 1656693 (0.0010) [2023-12-27 17:08:03,612][20381] Updated weights for policy 1, policy_version 1656703 (0.0005) [2023-12-27 17:08:03,668][20381] Updated weights for policy 1, policy_version 1656713 (0.0005) [2023-12-27 17:08:03,851][20347] Updated weights for policy 0, policy_version 1658283 (0.0007) [2023-12-27 17:08:03,904][20347] Updated weights for policy 0, policy_version 1658293 (0.0007) [2023-12-27 17:08:03,967][20347] Updated weights for policy 0, policy_version 1658303 (0.0009) [2023-12-27 17:08:04,297][20381] Updated weights for policy 1, policy_version 1656723 (0.0011) [2023-12-27 17:08:04,353][20381] Updated weights for policy 1, policy_version 1656733 (0.0009) [2023-12-27 17:08:04,416][20381] Updated weights for policy 1, policy_version 1656743 (0.0009) [2023-12-27 17:08:04,576][20347] Updated weights for policy 0, policy_version 1658313 (0.0010) [2023-12-27 17:08:04,623][20347] Updated weights for policy 0, policy_version 1658323 (0.0007) [2023-12-27 17:08:04,675][20347] Updated weights for policy 0, policy_version 1658333 (0.0005) [2023-12-27 17:08:04,722][20347] Updated weights for policy 0, policy_version 1658343 (0.0007) [2023-12-27 17:08:05,026][20381] Updated weights for policy 1, policy_version 1656753 (0.0010) [2023-12-27 17:08:05,084][20381] Updated weights for policy 1, policy_version 1656763 (0.0011) [2023-12-27 17:08:05,142][20381] Updated weights for policy 1, policy_version 1656773 (0.0010) [2023-12-27 17:08:05,197][20381] Updated weights for policy 1, policy_version 1656783 (0.0011) [2023-12-27 17:08:05,307][20347] Updated weights for policy 0, policy_version 1658353 (0.0006) [2023-12-27 17:08:05,350][20347] Updated weights for policy 0, policy_version 1658363 (0.0006) [2023-12-27 17:08:05,397][20347] Updated weights for policy 0, policy_version 1658373 (0.0005) [2023-12-27 17:08:05,843][20381] Updated weights for policy 1, policy_version 1656793 (0.0010) [2023-12-27 17:08:05,888][20381] Updated weights for policy 1, policy_version 1656803 (0.0010) [2023-12-27 17:08:05,937][20381] Updated weights for policy 1, policy_version 1656813 (0.0010) [2023-12-27 17:08:06,003][20347] Updated weights for policy 0, policy_version 1658383 (0.0007) [2023-12-27 17:08:06,051][20347] Updated weights for policy 0, policy_version 1658393 (0.0006) [2023-12-27 17:08:06,094][20347] Updated weights for policy 0, policy_version 1658403 (0.0007) [2023-12-27 17:08:06,619][20381] Updated weights for policy 1, policy_version 1656823 (0.0009) [2023-12-27 17:08:06,675][20381] Updated weights for policy 1, policy_version 1656833 (0.0007) [2023-12-27 17:08:06,720][20381] Updated weights for policy 1, policy_version 1656843 (0.0007) [2023-12-27 17:08:06,876][20347] Updated weights for policy 0, policy_version 1658413 (0.0007) [2023-12-27 17:08:06,927][20347] Updated weights for policy 0, policy_version 1658423 (0.0009) [2023-12-27 17:08:06,982][20347] Updated weights for policy 0, policy_version 1658433 (0.0007) [2023-12-27 17:08:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 848838656. Throughput: 0: 10370.0, 1: 10369.0. Samples: 848808404. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:07,363][19170] Avg episode reward: [(0, '980.410'), (1, '736.640')] [2023-12-27 17:08:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001658440_424640512.pth... [2023-12-27 17:08:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001657224_424329216.pth [2023-12-27 17:08:07,386][20381] Updated weights for policy 1, policy_version 1656853 (0.0008) [2023-12-27 17:08:07,442][20381] Updated weights for policy 1, policy_version 1656863 (0.0007) [2023-12-27 17:08:07,500][20381] Updated weights for policy 1, policy_version 1656873 (0.0008) [2023-12-27 17:08:07,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001656880_424206336.pth... [2023-12-27 17:08:07,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001655664_423895040.pth [2023-12-27 17:08:07,645][20347] Updated weights for policy 0, policy_version 1658443 (0.0008) [2023-12-27 17:08:07,698][20347] Updated weights for policy 0, policy_version 1658453 (0.0008) [2023-12-27 17:08:07,747][20347] Updated weights for policy 0, policy_version 1658463 (0.0005) [2023-12-27 17:08:08,121][20381] Updated weights for policy 1, policy_version 1656883 (0.0008) [2023-12-27 17:08:08,180][20381] Updated weights for policy 1, policy_version 1656893 (0.0007) [2023-12-27 17:08:08,237][20381] Updated weights for policy 1, policy_version 1656903 (0.0007) [2023-12-27 17:08:08,404][20347] Updated weights for policy 0, policy_version 1658473 (0.0006) [2023-12-27 17:08:08,456][20347] Updated weights for policy 0, policy_version 1658483 (0.0007) [2023-12-27 17:08:08,512][20347] Updated weights for policy 0, policy_version 1658493 (0.0007) [2023-12-27 17:08:08,570][20347] Updated weights for policy 0, policy_version 1658503 (0.0005) [2023-12-27 17:08:08,924][20381] Updated weights for policy 1, policy_version 1656913 (0.0008) [2023-12-27 17:08:08,984][20381] Updated weights for policy 1, policy_version 1656923 (0.0010) [2023-12-27 17:08:09,041][20381] Updated weights for policy 1, policy_version 1656933 (0.0010) [2023-12-27 17:08:09,086][20381] Updated weights for policy 1, policy_version 1656943 (0.0010) [2023-12-27 17:08:09,277][20347] Updated weights for policy 0, policy_version 1658513 (0.0008) [2023-12-27 17:08:09,340][20347] Updated weights for policy 0, policy_version 1658523 (0.0007) [2023-12-27 17:08:09,421][20347] Updated weights for policy 0, policy_version 1658533 (0.0007) [2023-12-27 17:08:09,783][20381] Updated weights for policy 1, policy_version 1656953 (0.0007) [2023-12-27 17:08:09,849][20381] Updated weights for policy 1, policy_version 1656963 (0.0009) [2023-12-27 17:08:09,911][20381] Updated weights for policy 1, policy_version 1656973 (0.0008) [2023-12-27 17:08:10,072][20347] Updated weights for policy 0, policy_version 1658543 (0.0006) [2023-12-27 17:08:10,134][20347] Updated weights for policy 0, policy_version 1658553 (0.0006) [2023-12-27 17:08:10,187][20347] Updated weights for policy 0, policy_version 1658563 (0.0006) [2023-12-27 17:08:10,602][20381] Updated weights for policy 1, policy_version 1656983 (0.0006) [2023-12-27 17:08:10,666][20381] Updated weights for policy 1, policy_version 1656993 (0.0008) [2023-12-27 17:08:10,726][20381] Updated weights for policy 1, policy_version 1657003 (0.0005) [2023-12-27 17:08:10,828][20347] Updated weights for policy 0, policy_version 1658573 (0.0007) [2023-12-27 17:08:10,891][20347] Updated weights for policy 0, policy_version 1658583 (0.0008) [2023-12-27 17:08:10,959][20347] Updated weights for policy 0, policy_version 1658593 (0.0008) [2023-12-27 17:08:11,355][20381] Updated weights for policy 1, policy_version 1657013 (0.0008) [2023-12-27 17:08:11,412][20381] Updated weights for policy 1, policy_version 1657023 (0.0011) [2023-12-27 17:08:11,472][20381] Updated weights for policy 1, policy_version 1657033 (0.0011) [2023-12-27 17:08:11,786][20347] Updated weights for policy 0, policy_version 1658603 (0.0007) [2023-12-27 17:08:11,850][20347] Updated weights for policy 0, policy_version 1658613 (0.0006) [2023-12-27 17:08:11,919][20347] Updated weights for policy 0, policy_version 1658623 (0.0010) [2023-12-27 17:08:12,198][20381] Updated weights for policy 1, policy_version 1657043 (0.0010) [2023-12-27 17:08:12,255][20381] Updated weights for policy 1, policy_version 1657053 (0.0010) [2023-12-27 17:08:12,315][20381] Updated weights for policy 1, policy_version 1657063 (0.0009) [2023-12-27 17:08:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 848936960. Throughput: 0: 10424.3, 1: 10391.9. Samples: 848933296. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:12,363][19170] Avg episode reward: [(0, '990.210'), (1, '765.980')] [2023-12-27 17:08:12,591][20347] Updated weights for policy 0, policy_version 1658633 (0.0010) [2023-12-27 17:08:12,649][20347] Updated weights for policy 0, policy_version 1658643 (0.0010) [2023-12-27 17:08:12,711][20347] Updated weights for policy 0, policy_version 1658653 (0.0011) [2023-12-27 17:08:12,774][20347] Updated weights for policy 0, policy_version 1658663 (0.0011) [2023-12-27 17:08:12,977][20381] Updated weights for policy 1, policy_version 1657073 (0.0008) [2023-12-27 17:08:13,040][20381] Updated weights for policy 1, policy_version 1657083 (0.0005) [2023-12-27 17:08:13,102][20381] Updated weights for policy 1, policy_version 1657093 (0.0005) [2023-12-27 17:08:13,167][20381] Updated weights for policy 1, policy_version 1657103 (0.0006) [2023-12-27 17:08:13,409][20347] Updated weights for policy 0, policy_version 1658673 (0.0008) [2023-12-27 17:08:13,463][20347] Updated weights for policy 0, policy_version 1658683 (0.0007) [2023-12-27 17:08:13,525][20347] Updated weights for policy 0, policy_version 1658693 (0.0008) [2023-12-27 17:08:13,742][20381] Updated weights for policy 1, policy_version 1657113 (0.0007) [2023-12-27 17:08:13,797][20381] Updated weights for policy 1, policy_version 1657123 (0.0008) [2023-12-27 17:08:13,859][20381] Updated weights for policy 1, policy_version 1657133 (0.0008) [2023-12-27 17:08:14,156][20347] Updated weights for policy 0, policy_version 1658703 (0.0005) [2023-12-27 17:08:14,218][20347] Updated weights for policy 0, policy_version 1658713 (0.0010) [2023-12-27 17:08:14,281][20347] Updated weights for policy 0, policy_version 1658723 (0.0010) [2023-12-27 17:08:14,489][20381] Updated weights for policy 1, policy_version 1657143 (0.0006) [2023-12-27 17:08:14,539][20381] Updated weights for policy 1, policy_version 1657153 (0.0008) [2023-12-27 17:08:14,589][20381] Updated weights for policy 1, policy_version 1657163 (0.0008) [2023-12-27 17:08:14,917][20347] Updated weights for policy 0, policy_version 1658733 (0.0010) [2023-12-27 17:08:14,980][20347] Updated weights for policy 0, policy_version 1658743 (0.0010) [2023-12-27 17:08:15,034][20347] Updated weights for policy 0, policy_version 1658753 (0.0011) [2023-12-27 17:08:15,284][20381] Updated weights for policy 1, policy_version 1657173 (0.0008) [2023-12-27 17:08:15,336][20381] Updated weights for policy 1, policy_version 1657183 (0.0008) [2023-12-27 17:08:15,381][20381] Updated weights for policy 1, policy_version 1657193 (0.0008) [2023-12-27 17:08:15,688][20347] Updated weights for policy 0, policy_version 1658763 (0.0009) [2023-12-27 17:08:15,736][20347] Updated weights for policy 0, policy_version 1658773 (0.0005) [2023-12-27 17:08:15,786][20347] Updated weights for policy 0, policy_version 1658783 (0.0007) [2023-12-27 17:08:16,065][20381] Updated weights for policy 1, policy_version 1657203 (0.0008) [2023-12-27 17:08:16,116][20381] Updated weights for policy 1, policy_version 1657213 (0.0009) [2023-12-27 17:08:16,166][20381] Updated weights for policy 1, policy_version 1657223 (0.0009) [2023-12-27 17:08:16,368][20347] Updated weights for policy 0, policy_version 1658793 (0.0007) [2023-12-27 17:08:16,423][20347] Updated weights for policy 0, policy_version 1658803 (0.0005) [2023-12-27 17:08:16,478][20347] Updated weights for policy 0, policy_version 1658813 (0.0006) [2023-12-27 17:08:16,531][20347] Updated weights for policy 0, policy_version 1658823 (0.0008) [2023-12-27 17:08:16,863][20381] Updated weights for policy 1, policy_version 1657233 (0.0006) [2023-12-27 17:08:16,925][20381] Updated weights for policy 1, policy_version 1657243 (0.0005) [2023-12-27 17:08:16,987][20381] Updated weights for policy 1, policy_version 1657253 (0.0011) [2023-12-27 17:08:17,048][20381] Updated weights for policy 1, policy_version 1657263 (0.0010) [2023-12-27 17:08:17,113][20347] Updated weights for policy 0, policy_version 1658833 (0.0007) [2023-12-27 17:08:17,173][20347] Updated weights for policy 0, policy_version 1658843 (0.0008) [2023-12-27 17:08:17,230][20347] Updated weights for policy 0, policy_version 1658853 (0.0008) [2023-12-27 17:08:17,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 849051648. Throughput: 0: 10490.2, 1: 10301.4. Samples: 849058744. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:17,363][19170] Avg episode reward: [(0, '980.450'), (1, '795.230')] [2023-12-27 17:08:17,671][20381] Updated weights for policy 1, policy_version 1657273 (0.0008) [2023-12-27 17:08:17,716][20381] Updated weights for policy 1, policy_version 1657283 (0.0007) [2023-12-27 17:08:17,766][20381] Updated weights for policy 1, policy_version 1657293 (0.0011) [2023-12-27 17:08:17,869][20347] Updated weights for policy 0, policy_version 1658863 (0.0007) [2023-12-27 17:08:17,922][20347] Updated weights for policy 0, policy_version 1658873 (0.0007) [2023-12-27 17:08:17,978][20347] Updated weights for policy 0, policy_version 1658883 (0.0008) [2023-12-27 17:08:18,424][20381] Updated weights for policy 1, policy_version 1657303 (0.0009) [2023-12-27 17:08:18,492][20381] Updated weights for policy 1, policy_version 1657313 (0.0008) [2023-12-27 17:08:18,557][20381] Updated weights for policy 1, policy_version 1657323 (0.0008) [2023-12-27 17:08:18,598][20347] Updated weights for policy 0, policy_version 1658893 (0.0007) [2023-12-27 17:08:18,657][20347] Updated weights for policy 0, policy_version 1658903 (0.0007) [2023-12-27 17:08:18,707][20347] Updated weights for policy 0, policy_version 1658913 (0.0008) [2023-12-27 17:08:19,100][20381] Updated weights for policy 1, policy_version 1657333 (0.0008) [2023-12-27 17:08:19,167][20381] Updated weights for policy 1, policy_version 1657343 (0.0009) [2023-12-27 17:08:19,219][20381] Updated weights for policy 1, policy_version 1657353 (0.0008) [2023-12-27 17:08:19,463][20347] Updated weights for policy 0, policy_version 1658923 (0.0008) [2023-12-27 17:08:19,534][20347] Updated weights for policy 0, policy_version 1658933 (0.0009) [2023-12-27 17:08:19,598][20347] Updated weights for policy 0, policy_version 1658943 (0.0007) [2023-12-27 17:08:19,923][20381] Updated weights for policy 1, policy_version 1657363 (0.0009) [2023-12-27 17:08:19,988][20381] Updated weights for policy 1, policy_version 1657373 (0.0011) [2023-12-27 17:08:20,049][20381] Updated weights for policy 1, policy_version 1657383 (0.0007) [2023-12-27 17:08:20,314][20347] Updated weights for policy 0, policy_version 1658953 (0.0008) [2023-12-27 17:08:20,378][20347] Updated weights for policy 0, policy_version 1658963 (0.0008) [2023-12-27 17:08:20,430][20347] Updated weights for policy 0, policy_version 1658973 (0.0007) [2023-12-27 17:08:20,473][20347] Updated weights for policy 0, policy_version 1658983 (0.0007) [2023-12-27 17:08:20,670][20381] Updated weights for policy 1, policy_version 1657393 (0.0006) [2023-12-27 17:08:20,728][20381] Updated weights for policy 1, policy_version 1657403 (0.0011) [2023-12-27 17:08:20,781][20381] Updated weights for policy 1, policy_version 1657413 (0.0008) [2023-12-27 17:08:20,841][20381] Updated weights for policy 1, policy_version 1657423 (0.0007) [2023-12-27 17:08:21,175][20347] Updated weights for policy 0, policy_version 1658993 (0.0008) [2023-12-27 17:08:21,235][20347] Updated weights for policy 0, policy_version 1659003 (0.0006) [2023-12-27 17:08:21,301][20347] Updated weights for policy 0, policy_version 1659013 (0.0008) [2023-12-27 17:08:21,583][20381] Updated weights for policy 1, policy_version 1657433 (0.0007) [2023-12-27 17:08:21,655][20381] Updated weights for policy 1, policy_version 1657443 (0.0011) [2023-12-27 17:08:21,715][20381] Updated weights for policy 1, policy_version 1657453 (0.0008) [2023-12-27 17:08:21,984][20347] Updated weights for policy 0, policy_version 1659023 (0.0007) [2023-12-27 17:08:22,043][20347] Updated weights for policy 0, policy_version 1659033 (0.0008) [2023-12-27 17:08:22,103][20347] Updated weights for policy 0, policy_version 1659043 (0.0007) [2023-12-27 17:08:22,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 849149952. Throughput: 0: 10477.1, 1: 10321.3. Samples: 849121396. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:22,363][19170] Avg episode reward: [(0, '990.240'), (1, '804.640')] [2023-12-27 17:08:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001659048_424796160.pth... [2023-12-27 17:08:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001657456_424353792.pth... [2023-12-27 17:08:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001656240_424042496.pth [2023-12-27 17:08:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001657800_424476672.pth [2023-12-27 17:08:22,453][20381] Updated weights for policy 1, policy_version 1657463 (0.0009) [2023-12-27 17:08:22,510][20381] Updated weights for policy 1, policy_version 1657473 (0.0007) [2023-12-27 17:08:22,571][20381] Updated weights for policy 1, policy_version 1657483 (0.0007) [2023-12-27 17:08:22,801][20347] Updated weights for policy 0, policy_version 1659053 (0.0007) [2023-12-27 17:08:22,857][20347] Updated weights for policy 0, policy_version 1659063 (0.0007) [2023-12-27 17:08:22,913][20347] Updated weights for policy 0, policy_version 1659073 (0.0007) [2023-12-27 17:08:23,229][20381] Updated weights for policy 1, policy_version 1657493 (0.0007) [2023-12-27 17:08:23,281][20381] Updated weights for policy 1, policy_version 1657503 (0.0008) [2023-12-27 17:08:23,332][20381] Updated weights for policy 1, policy_version 1657513 (0.0009) [2023-12-27 17:08:23,560][20347] Updated weights for policy 0, policy_version 1659083 (0.0007) [2023-12-27 17:08:23,613][20347] Updated weights for policy 0, policy_version 1659093 (0.0006) [2023-12-27 17:08:23,666][20347] Updated weights for policy 0, policy_version 1659103 (0.0005) [2023-12-27 17:08:23,947][20381] Updated weights for policy 1, policy_version 1657523 (0.0008) [2023-12-27 17:08:23,999][20381] Updated weights for policy 1, policy_version 1657533 (0.0007) [2023-12-27 17:08:24,055][20381] Updated weights for policy 1, policy_version 1657543 (0.0007) [2023-12-27 17:08:24,213][20347] Updated weights for policy 0, policy_version 1659113 (0.0008) [2023-12-27 17:08:24,272][20347] Updated weights for policy 0, policy_version 1659123 (0.0007) [2023-12-27 17:08:24,326][20347] Updated weights for policy 0, policy_version 1659133 (0.0005) [2023-12-27 17:08:24,375][20347] Updated weights for policy 0, policy_version 1659143 (0.0005) [2023-12-27 17:08:24,777][20381] Updated weights for policy 1, policy_version 1657553 (0.0008) [2023-12-27 17:08:24,830][20381] Updated weights for policy 1, policy_version 1657563 (0.0005) [2023-12-27 17:08:24,885][20381] Updated weights for policy 1, policy_version 1657573 (0.0005) [2023-12-27 17:08:24,943][20381] Updated weights for policy 1, policy_version 1657583 (0.0007) [2023-12-27 17:08:25,030][20347] Updated weights for policy 0, policy_version 1659153 (0.0005) [2023-12-27 17:08:25,087][20347] Updated weights for policy 0, policy_version 1659163 (0.0006) [2023-12-27 17:08:25,144][20347] Updated weights for policy 0, policy_version 1659173 (0.0008) [2023-12-27 17:08:25,526][20381] Updated weights for policy 1, policy_version 1657593 (0.0007) [2023-12-27 17:08:25,592][20381] Updated weights for policy 1, policy_version 1657603 (0.0010) [2023-12-27 17:08:25,639][20381] Updated weights for policy 1, policy_version 1657613 (0.0010) [2023-12-27 17:08:25,813][20347] Updated weights for policy 0, policy_version 1659183 (0.0009) [2023-12-27 17:08:25,862][20347] Updated weights for policy 0, policy_version 1659193 (0.0010) [2023-12-27 17:08:25,911][20347] Updated weights for policy 0, policy_version 1659203 (0.0008) [2023-12-27 17:08:26,302][20381] Updated weights for policy 1, policy_version 1657623 (0.0010) [2023-12-27 17:08:26,368][20381] Updated weights for policy 1, policy_version 1657633 (0.0010) [2023-12-27 17:08:26,419][20381] Updated weights for policy 1, policy_version 1657643 (0.0007) [2023-12-27 17:08:26,608][20347] Updated weights for policy 0, policy_version 1659213 (0.0008) [2023-12-27 17:08:26,663][20347] Updated weights for policy 0, policy_version 1659223 (0.0010) [2023-12-27 17:08:26,719][20347] Updated weights for policy 0, policy_version 1659233 (0.0010) [2023-12-27 17:08:26,948][20381] Updated weights for policy 1, policy_version 1657653 (0.0009) [2023-12-27 17:08:27,004][20381] Updated weights for policy 1, policy_version 1657663 (0.0008) [2023-12-27 17:08:27,061][20381] Updated weights for policy 1, policy_version 1657673 (0.0006) [2023-12-27 17:08:27,341][20347] Updated weights for policy 0, policy_version 1659243 (0.0009) [2023-12-27 17:08:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 849256448. Throughput: 0: 10484.3, 1: 10319.2. Samples: 849245468. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-12-27 17:08:27,363][19170] Avg episode reward: [(0, '980.400'), (1, '795.100')] [2023-12-27 17:08:27,396][20347] Updated weights for policy 0, policy_version 1659253 (0.0005) [2023-12-27 17:08:27,454][20347] Updated weights for policy 0, policy_version 1659263 (0.0005) [2023-12-27 17:08:27,672][20381] Updated weights for policy 1, policy_version 1657683 (0.0007) [2023-12-27 17:08:27,719][20381] Updated weights for policy 1, policy_version 1657693 (0.0010) [2023-12-27 17:08:27,776][20381] Updated weights for policy 1, policy_version 1657703 (0.0010) [2023-12-27 17:08:27,997][20347] Updated weights for policy 0, policy_version 1659273 (0.0005) [2023-12-27 17:08:28,045][20347] Updated weights for policy 0, policy_version 1659283 (0.0007) [2023-12-27 17:08:28,096][20347] Updated weights for policy 0, policy_version 1659293 (0.0010) [2023-12-27 17:08:28,147][20347] Updated weights for policy 0, policy_version 1659303 (0.0010) [2023-12-27 17:08:28,488][20381] Updated weights for policy 1, policy_version 1657713 (0.0010) [2023-12-27 17:08:28,546][20381] Updated weights for policy 1, policy_version 1657723 (0.0011) [2023-12-27 17:08:28,610][20381] Updated weights for policy 1, policy_version 1657733 (0.0008) [2023-12-27 17:08:28,668][20381] Updated weights for policy 1, policy_version 1657743 (0.0007) [2023-12-27 17:08:28,830][20347] Updated weights for policy 0, policy_version 1659313 (0.0010) [2023-12-27 17:08:28,888][20347] Updated weights for policy 0, policy_version 1659323 (0.0010) [2023-12-27 17:08:28,942][20347] Updated weights for policy 0, policy_version 1659333 (0.0010) [2023-12-27 17:08:29,224][20381] Updated weights for policy 1, policy_version 1657753 (0.0006) [2023-12-27 17:08:29,287][20381] Updated weights for policy 1, policy_version 1657763 (0.0009) [2023-12-27 17:08:29,353][20381] Updated weights for policy 1, policy_version 1657773 (0.0007) [2023-12-27 17:08:29,577][20347] Updated weights for policy 0, policy_version 1659343 (0.0010) [2023-12-27 17:08:29,634][20347] Updated weights for policy 0, policy_version 1659353 (0.0010) [2023-12-27 17:08:29,682][20347] Updated weights for policy 0, policy_version 1659363 (0.0010) [2023-12-27 17:08:30,093][20381] Updated weights for policy 1, policy_version 1657783 (0.0007) [2023-12-27 17:08:30,154][20381] Updated weights for policy 1, policy_version 1657793 (0.0007) [2023-12-27 17:08:30,216][20381] Updated weights for policy 1, policy_version 1657803 (0.0007) [2023-12-27 17:08:30,382][20347] Updated weights for policy 0, policy_version 1659373 (0.0009) [2023-12-27 17:08:30,446][20347] Updated weights for policy 0, policy_version 1659383 (0.0008) [2023-12-27 17:08:30,499][20347] Updated weights for policy 0, policy_version 1659393 (0.0006) [2023-12-27 17:08:30,884][20381] Updated weights for policy 1, policy_version 1657813 (0.0007) [2023-12-27 17:08:30,943][20381] Updated weights for policy 1, policy_version 1657823 (0.0006) [2023-12-27 17:08:31,002][20381] Updated weights for policy 1, policy_version 1657833 (0.0008) [2023-12-27 17:08:31,122][20347] Updated weights for policy 0, policy_version 1659403 (0.0006) [2023-12-27 17:08:31,187][20347] Updated weights for policy 0, policy_version 1659413 (0.0007) [2023-12-27 17:08:31,246][20347] Updated weights for policy 0, policy_version 1659423 (0.0008) [2023-12-27 17:08:31,702][20381] Updated weights for policy 1, policy_version 1657843 (0.0007) [2023-12-27 17:08:31,770][20381] Updated weights for policy 1, policy_version 1657853 (0.0007) [2023-12-27 17:08:31,822][20381] Updated weights for policy 1, policy_version 1657863 (0.0008) [2023-12-27 17:08:31,951][20347] Updated weights for policy 0, policy_version 1659433 (0.0007) [2023-12-27 17:08:32,007][20347] Updated weights for policy 0, policy_version 1659443 (0.0007) [2023-12-27 17:08:32,061][20347] Updated weights for policy 0, policy_version 1659453 (0.0008) [2023-12-27 17:08:32,121][20347] Updated weights for policy 0, policy_version 1659463 (0.0007) [2023-12-27 17:08:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 849362944. Throughput: 0: 10517.7, 1: 10381.4. Samples: 849373580. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:32,363][19170] Avg episode reward: [(0, '970.650'), (1, '814.560')] [2023-12-27 17:08:32,459][20381] Updated weights for policy 1, policy_version 1657873 (0.0006) [2023-12-27 17:08:32,510][20381] Updated weights for policy 1, policy_version 1657883 (0.0005) [2023-12-27 17:08:32,567][20381] Updated weights for policy 1, policy_version 1657893 (0.0006) [2023-12-27 17:08:32,621][20381] Updated weights for policy 1, policy_version 1657903 (0.0007) [2023-12-27 17:08:32,861][20347] Updated weights for policy 0, policy_version 1659473 (0.0008) [2023-12-27 17:08:32,914][20347] Updated weights for policy 0, policy_version 1659483 (0.0007) [2023-12-27 17:08:32,972][20347] Updated weights for policy 0, policy_version 1659493 (0.0007) [2023-12-27 17:08:33,232][20381] Updated weights for policy 1, policy_version 1657913 (0.0010) [2023-12-27 17:08:33,297][20381] Updated weights for policy 1, policy_version 1657923 (0.0010) [2023-12-27 17:08:33,349][20381] Updated weights for policy 1, policy_version 1657933 (0.0010) [2023-12-27 17:08:33,679][20347] Updated weights for policy 0, policy_version 1659503 (0.0007) [2023-12-27 17:08:33,736][20347] Updated weights for policy 0, policy_version 1659513 (0.0007) [2023-12-27 17:08:33,789][20347] Updated weights for policy 0, policy_version 1659523 (0.0007) [2023-12-27 17:08:34,024][20381] Updated weights for policy 1, policy_version 1657943 (0.0007) [2023-12-27 17:08:34,079][20381] Updated weights for policy 1, policy_version 1657953 (0.0010) [2023-12-27 17:08:34,137][20381] Updated weights for policy 1, policy_version 1657963 (0.0011) [2023-12-27 17:08:34,495][20347] Updated weights for policy 0, policy_version 1659533 (0.0007) [2023-12-27 17:08:34,547][20347] Updated weights for policy 0, policy_version 1659543 (0.0007) [2023-12-27 17:08:34,603][20347] Updated weights for policy 0, policy_version 1659553 (0.0007) [2023-12-27 17:08:34,845][20381] Updated weights for policy 1, policy_version 1657973 (0.0009) [2023-12-27 17:08:34,900][20381] Updated weights for policy 1, policy_version 1657983 (0.0007) [2023-12-27 17:08:34,958][20381] Updated weights for policy 1, policy_version 1657993 (0.0006) [2023-12-27 17:08:35,299][20347] Updated weights for policy 0, policy_version 1659563 (0.0007) [2023-12-27 17:08:35,350][20347] Updated weights for policy 0, policy_version 1659573 (0.0010) [2023-12-27 17:08:35,399][20347] Updated weights for policy 0, policy_version 1659583 (0.0010) [2023-12-27 17:08:35,593][20381] Updated weights for policy 1, policy_version 1658003 (0.0007) [2023-12-27 17:08:35,651][20381] Updated weights for policy 1, policy_version 1658013 (0.0005) [2023-12-27 17:08:35,714][20381] Updated weights for policy 1, policy_version 1658023 (0.0008) [2023-12-27 17:08:36,063][20347] Updated weights for policy 0, policy_version 1659593 (0.0010) [2023-12-27 17:08:36,125][20347] Updated weights for policy 0, policy_version 1659603 (0.0011) [2023-12-27 17:08:36,191][20347] Updated weights for policy 0, policy_version 1659613 (0.0011) [2023-12-27 17:08:36,254][20347] Updated weights for policy 0, policy_version 1659623 (0.0010) [2023-12-27 17:08:36,281][20381] Updated weights for policy 1, policy_version 1658033 (0.0010) [2023-12-27 17:08:36,342][20381] Updated weights for policy 1, policy_version 1658043 (0.0007) [2023-12-27 17:08:36,395][20381] Updated weights for policy 1, policy_version 1658053 (0.0007) [2023-12-27 17:08:36,441][20381] Updated weights for policy 1, policy_version 1658063 (0.0007) [2023-12-27 17:08:36,976][20347] Updated weights for policy 0, policy_version 1659633 (0.0008) [2023-12-27 17:08:37,027][20347] Updated weights for policy 0, policy_version 1659643 (0.0007) [2023-12-27 17:08:37,086][20347] Updated weights for policy 0, policy_version 1659653 (0.0006) [2023-12-27 17:08:37,088][20381] Updated weights for policy 1, policy_version 1658073 (0.0010) [2023-12-27 17:08:37,144][20381] Updated weights for policy 1, policy_version 1658083 (0.0011) [2023-12-27 17:08:37,203][20381] Updated weights for policy 1, policy_version 1658093 (0.0011) [2023-12-27 17:08:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 849469440. Throughput: 0: 10488.4, 1: 10409.4. Samples: 849434216. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:37,363][19170] Avg episode reward: [(0, '990.250'), (1, '814.620')] [2023-12-27 17:08:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001658096_424517632.pth... [2023-12-27 17:08:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001659656_424951808.pth... [2023-12-27 17:08:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001658440_424640512.pth [2023-12-27 17:08:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001656880_424206336.pth [2023-12-27 17:08:37,787][20347] Updated weights for policy 0, policy_version 1659663 (0.0005) [2023-12-27 17:08:37,839][20347] Updated weights for policy 0, policy_version 1659673 (0.0005) [2023-12-27 17:08:37,872][20381] Updated weights for policy 1, policy_version 1658103 (0.0008) [2023-12-27 17:08:37,893][20347] Updated weights for policy 0, policy_version 1659683 (0.0006) [2023-12-27 17:08:37,926][20381] Updated weights for policy 1, policy_version 1658113 (0.0008) [2023-12-27 17:08:37,982][20381] Updated weights for policy 1, policy_version 1658123 (0.0005) [2023-12-27 17:08:38,493][20347] Updated weights for policy 0, policy_version 1659693 (0.0007) [2023-12-27 17:08:38,548][20347] Updated weights for policy 0, policy_version 1659703 (0.0008) [2023-12-27 17:08:38,611][20347] Updated weights for policy 0, policy_version 1659713 (0.0007) [2023-12-27 17:08:38,619][20381] Updated weights for policy 1, policy_version 1658133 (0.0006) [2023-12-27 17:08:38,681][20381] Updated weights for policy 1, policy_version 1658143 (0.0007) [2023-12-27 17:08:38,739][20381] Updated weights for policy 1, policy_version 1658153 (0.0008) [2023-12-27 17:08:39,224][20347] Updated weights for policy 0, policy_version 1659723 (0.0009) [2023-12-27 17:08:39,278][20347] Updated weights for policy 0, policy_version 1659733 (0.0008) [2023-12-27 17:08:39,328][20347] Updated weights for policy 0, policy_version 1659743 (0.0007) [2023-12-27 17:08:39,438][20381] Updated weights for policy 1, policy_version 1658163 (0.0008) [2023-12-27 17:08:39,497][20381] Updated weights for policy 1, policy_version 1658173 (0.0007) [2023-12-27 17:08:39,554][20381] Updated weights for policy 1, policy_version 1658183 (0.0007) [2023-12-27 17:08:40,059][20347] Updated weights for policy 0, policy_version 1659753 (0.0009) [2023-12-27 17:08:40,111][20347] Updated weights for policy 0, policy_version 1659763 (0.0007) [2023-12-27 17:08:40,170][20347] Updated weights for policy 0, policy_version 1659773 (0.0010) [2023-12-27 17:08:40,234][20347] Updated weights for policy 0, policy_version 1659783 (0.0011) [2023-12-27 17:08:40,324][20381] Updated weights for policy 1, policy_version 1658193 (0.0007) [2023-12-27 17:08:40,383][20381] Updated weights for policy 1, policy_version 1658203 (0.0007) [2023-12-27 17:08:40,439][20381] Updated weights for policy 1, policy_version 1658213 (0.0008) [2023-12-27 17:08:40,496][20381] Updated weights for policy 1, policy_version 1658223 (0.0007) [2023-12-27 17:08:40,903][20291] KL-divergence is very high: 102.9873 [2023-12-27 17:08:40,907][20347] Updated weights for policy 0, policy_version 1659793 (0.0007) [2023-12-27 17:08:40,917][20291] KL-divergence is very high: 182.1269 [2023-12-27 17:08:40,926][20291] KL-divergence is very high: 220.0943 [2023-12-27 17:08:40,940][20291] KL-divergence is very high: 185.0903 [2023-12-27 17:08:40,954][20291] KL-divergence is very high: 129.5240 [2023-12-27 17:08:40,955][20347] Updated weights for policy 0, policy_version 1659803 (0.0008) [2023-12-27 17:08:41,009][20347] Updated weights for policy 0, policy_version 1659813 (0.0007) [2023-12-27 17:08:41,224][20381] Updated weights for policy 1, policy_version 1658233 (0.0010) [2023-12-27 17:08:41,285][20381] Updated weights for policy 1, policy_version 1658243 (0.0011) [2023-12-27 17:08:41,342][20381] Updated weights for policy 1, policy_version 1658253 (0.0011) [2023-12-27 17:08:41,808][20347] Updated weights for policy 0, policy_version 1659823 (0.0006) [2023-12-27 17:08:41,868][20347] Updated weights for policy 0, policy_version 1659833 (0.0006) [2023-12-27 17:08:41,933][20347] Updated weights for policy 0, policy_version 1659843 (0.0005) [2023-12-27 17:08:42,052][20381] Updated weights for policy 1, policy_version 1658263 (0.0011) [2023-12-27 17:08:42,098][20381] Updated weights for policy 1, policy_version 1658273 (0.0005) [2023-12-27 17:08:42,149][20381] Updated weights for policy 1, policy_version 1658283 (0.0006) [2023-12-27 17:08:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 849567744. Throughput: 0: 10451.1, 1: 10379.8. Samples: 849557196. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:42,363][19170] Avg episode reward: [(0, '970.680'), (1, '863.520')] [2023-12-27 17:08:42,580][20347] Updated weights for policy 0, policy_version 1659853 (0.0006) [2023-12-27 17:08:42,632][20347] Updated weights for policy 0, policy_version 1659863 (0.0007) [2023-12-27 17:08:42,684][20347] Updated weights for policy 0, policy_version 1659873 (0.0007) [2023-12-27 17:08:42,826][20381] Updated weights for policy 1, policy_version 1658293 (0.0007) [2023-12-27 17:08:42,891][20381] Updated weights for policy 1, policy_version 1658303 (0.0008) [2023-12-27 17:08:42,953][20381] Updated weights for policy 1, policy_version 1658313 (0.0008) [2023-12-27 17:08:43,330][20347] Updated weights for policy 0, policy_version 1659883 (0.0007) [2023-12-27 17:08:43,382][20347] Updated weights for policy 0, policy_version 1659893 (0.0007) [2023-12-27 17:08:43,438][20347] Updated weights for policy 0, policy_version 1659903 (0.0008) [2023-12-27 17:08:43,593][20381] Updated weights for policy 1, policy_version 1658323 (0.0008) [2023-12-27 17:08:43,639][20381] Updated weights for policy 1, policy_version 1658333 (0.0008) [2023-12-27 17:08:43,701][20381] Updated weights for policy 1, policy_version 1658343 (0.0008) [2023-12-27 17:08:44,038][20347] Updated weights for policy 0, policy_version 1659913 (0.0008) [2023-12-27 17:08:44,095][20347] Updated weights for policy 0, policy_version 1659923 (0.0008) [2023-12-27 17:08:44,150][20347] Updated weights for policy 0, policy_version 1659933 (0.0008) [2023-12-27 17:08:44,212][20347] Updated weights for policy 0, policy_version 1659944 (0.0006) [2023-12-27 17:08:44,374][20381] Updated weights for policy 1, policy_version 1658353 (0.0008) [2023-12-27 17:08:44,433][20381] Updated weights for policy 1, policy_version 1658363 (0.0007) [2023-12-27 17:08:44,498][20381] Updated weights for policy 1, policy_version 1658373 (0.0006) [2023-12-27 17:08:44,555][20381] Updated weights for policy 1, policy_version 1658383 (0.0005) [2023-12-27 17:08:44,891][20347] Updated weights for policy 0, policy_version 1659954 (0.0006) [2023-12-27 17:08:44,950][20347] Updated weights for policy 0, policy_version 1659964 (0.0006) [2023-12-27 17:08:45,010][20347] Updated weights for policy 0, policy_version 1659974 (0.0006) [2023-12-27 17:08:45,231][20381] Updated weights for policy 1, policy_version 1658393 (0.0007) [2023-12-27 17:08:45,287][20381] Updated weights for policy 1, policy_version 1658403 (0.0007) [2023-12-27 17:08:45,335][20381] Updated weights for policy 1, policy_version 1658413 (0.0008) [2023-12-27 17:08:45,636][20347] Updated weights for policy 0, policy_version 1659984 (0.0010) [2023-12-27 17:08:45,684][20347] Updated weights for policy 0, policy_version 1659994 (0.0010) [2023-12-27 17:08:45,730][20347] Updated weights for policy 0, policy_version 1660004 (0.0010) [2023-12-27 17:08:45,985][20381] Updated weights for policy 1, policy_version 1658423 (0.0008) [2023-12-27 17:08:46,043][20381] Updated weights for policy 1, policy_version 1658433 (0.0007) [2023-12-27 17:08:46,100][20381] Updated weights for policy 1, policy_version 1658443 (0.0008) [2023-12-27 17:08:46,372][20347] Updated weights for policy 0, policy_version 1660014 (0.0007) [2023-12-27 17:08:46,424][20347] Updated weights for policy 0, policy_version 1660024 (0.0005) [2023-12-27 17:08:46,474][20347] Updated weights for policy 0, policy_version 1660034 (0.0005) [2023-12-27 17:08:46,841][20381] Updated weights for policy 1, policy_version 1658453 (0.0007) [2023-12-27 17:08:46,892][20381] Updated weights for policy 1, policy_version 1658463 (0.0008) [2023-12-27 17:08:46,941][20381] Updated weights for policy 1, policy_version 1658473 (0.0007) [2023-12-27 17:08:47,030][20347] Updated weights for policy 0, policy_version 1660044 (0.0005) [2023-12-27 17:08:47,089][20347] Updated weights for policy 0, policy_version 1660054 (0.0005) [2023-12-27 17:08:47,147][20347] Updated weights for policy 0, policy_version 1660064 (0.0008) [2023-12-27 17:08:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20889.5, 300 sec: 20605.0). Total num frames: 849674240. Throughput: 0: 10480.7, 1: 10414.1. Samples: 849682656. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:47,363][19170] Avg episode reward: [(0, '941.420'), (1, '882.910')] [2023-12-27 17:08:47,674][20381] Updated weights for policy 1, policy_version 1658483 (0.0008) [2023-12-27 17:08:47,723][20347] Updated weights for policy 0, policy_version 1660074 (0.0010) [2023-12-27 17:08:47,724][20381] Updated weights for policy 1, policy_version 1658493 (0.0008) [2023-12-27 17:08:47,777][20381] Updated weights for policy 1, policy_version 1658503 (0.0008) [2023-12-27 17:08:47,778][20347] Updated weights for policy 0, policy_version 1660084 (0.0005) [2023-12-27 17:08:47,838][20347] Updated weights for policy 0, policy_version 1660094 (0.0006) [2023-12-27 17:08:47,899][20347] Updated weights for policy 0, policy_version 1660104 (0.0005) [2023-12-27 17:08:48,460][20347] Updated weights for policy 0, policy_version 1660114 (0.0007) [2023-12-27 17:08:48,522][20347] Updated weights for policy 0, policy_version 1660124 (0.0010) [2023-12-27 17:08:48,539][20381] Updated weights for policy 1, policy_version 1658513 (0.0008) [2023-12-27 17:08:48,579][20347] Updated weights for policy 0, policy_version 1660134 (0.0009) [2023-12-27 17:08:48,595][20381] Updated weights for policy 1, policy_version 1658523 (0.0005) [2023-12-27 17:08:48,655][20381] Updated weights for policy 1, policy_version 1658533 (0.0007) [2023-12-27 17:08:48,716][20381] Updated weights for policy 1, policy_version 1658543 (0.0007) [2023-12-27 17:08:49,246][20347] Updated weights for policy 0, policy_version 1660144 (0.0008) [2023-12-27 17:08:49,304][20347] Updated weights for policy 0, policy_version 1660154 (0.0010) [2023-12-27 17:08:49,370][20347] Updated weights for policy 0, policy_version 1660164 (0.0010) [2023-12-27 17:08:49,416][20381] Updated weights for policy 1, policy_version 1658553 (0.0008) [2023-12-27 17:08:49,468][20381] Updated weights for policy 1, policy_version 1658563 (0.0008) [2023-12-27 17:08:49,520][20381] Updated weights for policy 1, policy_version 1658573 (0.0008) [2023-12-27 17:08:50,100][20347] Updated weights for policy 0, policy_version 1660174 (0.0007) [2023-12-27 17:08:50,162][20347] Updated weights for policy 0, policy_version 1660184 (0.0006) [2023-12-27 17:08:50,225][20347] Updated weights for policy 0, policy_version 1660194 (0.0006) [2023-12-27 17:08:50,276][20381] Updated weights for policy 1, policy_version 1658583 (0.0007) [2023-12-27 17:08:50,335][20381] Updated weights for policy 1, policy_version 1658593 (0.0008) [2023-12-27 17:08:50,395][20381] Updated weights for policy 1, policy_version 1658603 (0.0008) [2023-12-27 17:08:50,856][20347] Updated weights for policy 0, policy_version 1660204 (0.0006) [2023-12-27 17:08:50,917][20347] Updated weights for policy 0, policy_version 1660214 (0.0008) [2023-12-27 17:08:50,976][20347] Updated weights for policy 0, policy_version 1660224 (0.0007) [2023-12-27 17:08:51,142][20381] Updated weights for policy 1, policy_version 1658613 (0.0008) [2023-12-27 17:08:51,206][20381] Updated weights for policy 1, policy_version 1658623 (0.0008) [2023-12-27 17:08:51,262][20381] Updated weights for policy 1, policy_version 1658633 (0.0008) [2023-12-27 17:08:51,687][20347] Updated weights for policy 0, policy_version 1660234 (0.0008) [2023-12-27 17:08:51,751][20347] Updated weights for policy 0, policy_version 1660244 (0.0007) [2023-12-27 17:08:51,818][20347] Updated weights for policy 0, policy_version 1660254 (0.0006) [2023-12-27 17:08:51,881][20347] Updated weights for policy 0, policy_version 1660264 (0.0008) [2023-12-27 17:08:51,974][20381] Updated weights for policy 1, policy_version 1658643 (0.0008) [2023-12-27 17:08:52,037][20381] Updated weights for policy 1, policy_version 1658653 (0.0008) [2023-12-27 17:08:52,085][20381] Updated weights for policy 1, policy_version 1658663 (0.0008) [2023-12-27 17:08:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 849772544. Throughput: 0: 10448.0, 1: 10351.6. Samples: 849744388. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:52,363][19170] Avg episode reward: [(0, '960.960'), (1, '863.430')] [2023-12-27 17:08:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001658672_424665088.pth... [2023-12-27 17:08:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001657456_424353792.pth [2023-12-27 17:08:52,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001660264_425107456.pth... [2023-12-27 17:08:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001659048_424796160.pth [2023-12-27 17:08:52,566][20347] Updated weights for policy 0, policy_version 1660274 (0.0008) [2023-12-27 17:08:52,624][20347] Updated weights for policy 0, policy_version 1660284 (0.0007) [2023-12-27 17:08:52,688][20347] Updated weights for policy 0, policy_version 1660294 (0.0009) [2023-12-27 17:08:52,824][20381] Updated weights for policy 1, policy_version 1658673 (0.0008) [2023-12-27 17:08:52,882][20381] Updated weights for policy 1, policy_version 1658683 (0.0008) [2023-12-27 17:08:52,938][20381] Updated weights for policy 1, policy_version 1658693 (0.0008) [2023-12-27 17:08:52,990][20381] Updated weights for policy 1, policy_version 1658703 (0.0008) [2023-12-27 17:08:53,384][20347] Updated weights for policy 0, policy_version 1660304 (0.0008) [2023-12-27 17:08:53,440][20347] Updated weights for policy 0, policy_version 1660314 (0.0007) [2023-12-27 17:08:53,499][20347] Updated weights for policy 0, policy_version 1660324 (0.0008) [2023-12-27 17:08:53,690][20381] Updated weights for policy 1, policy_version 1658713 (0.0007) [2023-12-27 17:08:53,754][20381] Updated weights for policy 1, policy_version 1658723 (0.0008) [2023-12-27 17:08:53,786][20292] KL-divergence is very high: 120.8704 [2023-12-27 17:08:53,804][20292] KL-divergence is very high: 101.3285 [2023-12-27 17:08:53,806][20381] Updated weights for policy 1, policy_version 1658733 (0.0009) [2023-12-27 17:08:54,077][20347] Updated weights for policy 0, policy_version 1660334 (0.0009) [2023-12-27 17:08:54,136][20347] Updated weights for policy 0, policy_version 1660344 (0.0007) [2023-12-27 17:08:54,189][20347] Updated weights for policy 0, policy_version 1660354 (0.0007) [2023-12-27 17:08:54,487][20381] Updated weights for policy 1, policy_version 1658743 (0.0006) [2023-12-27 17:08:54,536][20381] Updated weights for policy 1, policy_version 1658753 (0.0008) [2023-12-27 17:08:54,581][20381] Updated weights for policy 1, policy_version 1658763 (0.0008) [2023-12-27 17:08:54,878][20347] Updated weights for policy 0, policy_version 1660364 (0.0009) [2023-12-27 17:08:54,925][20347] Updated weights for policy 0, policy_version 1660374 (0.0008) [2023-12-27 17:08:54,973][20347] Updated weights for policy 0, policy_version 1660384 (0.0007) [2023-12-27 17:08:55,260][20381] Updated weights for policy 1, policy_version 1658773 (0.0008) [2023-12-27 17:08:55,313][20381] Updated weights for policy 1, policy_version 1658783 (0.0008) [2023-12-27 17:08:55,361][20381] Updated weights for policy 1, policy_version 1658793 (0.0006) [2023-12-27 17:08:55,671][20347] Updated weights for policy 0, policy_version 1660394 (0.0006) [2023-12-27 17:08:55,727][20347] Updated weights for policy 0, policy_version 1660404 (0.0007) [2023-12-27 17:08:55,782][20347] Updated weights for policy 0, policy_version 1660414 (0.0008) [2023-12-27 17:08:55,839][20347] Updated weights for policy 0, policy_version 1660424 (0.0007) [2023-12-27 17:08:55,911][20381] Updated weights for policy 1, policy_version 1658803 (0.0005) [2023-12-27 17:08:55,963][20381] Updated weights for policy 1, policy_version 1658813 (0.0008) [2023-12-27 17:08:56,010][20381] Updated weights for policy 1, policy_version 1658823 (0.0010) [2023-12-27 17:08:56,451][20347] Updated weights for policy 0, policy_version 1660434 (0.0007) [2023-12-27 17:08:56,510][20347] Updated weights for policy 0, policy_version 1660444 (0.0007) [2023-12-27 17:08:56,564][20347] Updated weights for policy 0, policy_version 1660454 (0.0005) [2023-12-27 17:08:56,752][20381] Updated weights for policy 1, policy_version 1658833 (0.0009) [2023-12-27 17:08:56,804][20381] Updated weights for policy 1, policy_version 1658843 (0.0008) [2023-12-27 17:08:56,852][20381] Updated weights for policy 1, policy_version 1658853 (0.0008) [2023-12-27 17:08:56,908][20381] Updated weights for policy 1, policy_version 1658863 (0.0009) [2023-12-27 17:08:57,189][20347] Updated weights for policy 0, policy_version 1660464 (0.0007) [2023-12-27 17:08:57,244][20347] Updated weights for policy 0, policy_version 1660474 (0.0005) [2023-12-27 17:08:57,294][20291] KL-divergence is very high: 130.4361 [2023-12-27 17:08:57,305][20347] Updated weights for policy 0, policy_version 1660484 (0.0006) [2023-12-27 17:08:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 849879040. Throughput: 0: 10438.5, 1: 10287.0. Samples: 849865944. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:08:57,363][19170] Avg episode reward: [(0, '970.650'), (1, '824.250')] [2023-12-27 17:08:57,659][20381] Updated weights for policy 1, policy_version 1658873 (0.0008) [2023-12-27 17:08:57,703][20381] Updated weights for policy 1, policy_version 1658883 (0.0008) [2023-12-27 17:08:57,758][20381] Updated weights for policy 1, policy_version 1658893 (0.0006) [2023-12-27 17:08:57,879][20347] Updated weights for policy 0, policy_version 1660494 (0.0008) [2023-12-27 17:08:57,939][20347] Updated weights for policy 0, policy_version 1660504 (0.0010) [2023-12-27 17:08:57,991][20347] Updated weights for policy 0, policy_version 1660514 (0.0005) [2023-12-27 17:08:58,468][20381] Updated weights for policy 1, policy_version 1658903 (0.0008) [2023-12-27 17:08:58,534][20381] Updated weights for policy 1, policy_version 1658913 (0.0011) [2023-12-27 17:08:58,597][20381] Updated weights for policy 1, policy_version 1658923 (0.0010) [2023-12-27 17:08:58,735][20347] Updated weights for policy 0, policy_version 1660524 (0.0006) [2023-12-27 17:08:58,794][20347] Updated weights for policy 0, policy_version 1660534 (0.0007) [2023-12-27 17:08:58,866][20347] Updated weights for policy 0, policy_version 1660544 (0.0011) [2023-12-27 17:08:59,273][20381] Updated weights for policy 1, policy_version 1658933 (0.0009) [2023-12-27 17:08:59,339][20381] Updated weights for policy 1, policy_version 1658943 (0.0006) [2023-12-27 17:08:59,405][20381] Updated weights for policy 1, policy_version 1658953 (0.0008) [2023-12-27 17:08:59,656][20347] Updated weights for policy 0, policy_version 1660554 (0.0008) [2023-12-27 17:08:59,714][20347] Updated weights for policy 0, policy_version 1660564 (0.0008) [2023-12-27 17:08:59,772][20347] Updated weights for policy 0, policy_version 1660574 (0.0007) [2023-12-27 17:08:59,830][20347] Updated weights for policy 0, policy_version 1660584 (0.0008) [2023-12-27 17:09:00,101][20381] Updated weights for policy 1, policy_version 1658963 (0.0009) [2023-12-27 17:09:00,151][20381] Updated weights for policy 1, policy_version 1658973 (0.0006) [2023-12-27 17:09:00,197][20381] Updated weights for policy 1, policy_version 1658983 (0.0007) [2023-12-27 17:09:00,463][20347] Updated weights for policy 0, policy_version 1660594 (0.0006) [2023-12-27 17:09:00,526][20347] Updated weights for policy 0, policy_version 1660604 (0.0005) [2023-12-27 17:09:00,587][20347] Updated weights for policy 0, policy_version 1660614 (0.0005) [2023-12-27 17:09:00,932][20381] Updated weights for policy 1, policy_version 1658993 (0.0008) [2023-12-27 17:09:00,984][20381] Updated weights for policy 1, policy_version 1659003 (0.0008) [2023-12-27 17:09:01,043][20381] Updated weights for policy 1, policy_version 1659013 (0.0009) [2023-12-27 17:09:01,117][20381] Updated weights for policy 1, policy_version 1659023 (0.0009) [2023-12-27 17:09:01,141][20347] Updated weights for policy 0, policy_version 1660624 (0.0007) [2023-12-27 17:09:01,151][20291] KL-divergence is very high: 123.5883 [2023-12-27 17:09:01,197][20291] KL-divergence is very high: 234.7216 [2023-12-27 17:09:01,204][20347] Updated weights for policy 0, policy_version 1660634 (0.0007) [2023-12-27 17:09:01,243][20291] KL-divergence is very high: 135.9028 [2023-12-27 17:09:01,266][20347] Updated weights for policy 0, policy_version 1660644 (0.0008) [2023-12-27 17:09:01,828][20381] Updated weights for policy 1, policy_version 1659033 (0.0006) [2023-12-27 17:09:01,879][20381] Updated weights for policy 1, policy_version 1659043 (0.0006) [2023-12-27 17:09:01,938][20381] Updated weights for policy 1, policy_version 1659053 (0.0005) [2023-12-27 17:09:02,031][20347] Updated weights for policy 0, policy_version 1660654 (0.0008) [2023-12-27 17:09:02,093][20347] Updated weights for policy 0, policy_version 1660664 (0.0008) [2023-12-27 17:09:02,156][20347] Updated weights for policy 0, policy_version 1660674 (0.0008) [2023-12-27 17:09:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 849977344. Throughput: 0: 10420.8, 1: 10222.1. Samples: 849987676. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:02,363][19170] Avg episode reward: [(0, '960.900'), (1, '853.650')] [2023-12-27 17:09:02,572][20381] Updated weights for policy 1, policy_version 1659063 (0.0007) [2023-12-27 17:09:02,621][20381] Updated weights for policy 1, policy_version 1659073 (0.0008) [2023-12-27 17:09:02,672][20381] Updated weights for policy 1, policy_version 1659083 (0.0007) [2023-12-27 17:09:02,777][20347] Updated weights for policy 0, policy_version 1660684 (0.0008) [2023-12-27 17:09:02,836][20347] Updated weights for policy 0, policy_version 1660694 (0.0007) [2023-12-27 17:09:02,895][20347] Updated weights for policy 0, policy_version 1660704 (0.0005) [2023-12-27 17:09:03,430][20347] Updated weights for policy 0, policy_version 1660714 (0.0005) [2023-12-27 17:09:03,433][20381] Updated weights for policy 1, policy_version 1659093 (0.0007) [2023-12-27 17:09:03,482][20347] Updated weights for policy 0, policy_version 1660724 (0.0007) [2023-12-27 17:09:03,494][20381] Updated weights for policy 1, policy_version 1659103 (0.0006) [2023-12-27 17:09:03,537][20347] Updated weights for policy 0, policy_version 1660734 (0.0006) [2023-12-27 17:09:03,539][20381] Updated weights for policy 1, policy_version 1659113 (0.0007) [2023-12-27 17:09:03,595][20347] Updated weights for policy 0, policy_version 1660744 (0.0007) [2023-12-27 17:09:04,225][20347] Updated weights for policy 0, policy_version 1660754 (0.0007) [2023-12-27 17:09:04,265][20381] Updated weights for policy 1, policy_version 1659123 (0.0007) [2023-12-27 17:09:04,286][20347] Updated weights for policy 0, policy_version 1660764 (0.0006) [2023-12-27 17:09:04,315][20381] Updated weights for policy 1, policy_version 1659133 (0.0006) [2023-12-27 17:09:04,346][20347] Updated weights for policy 0, policy_version 1660774 (0.0007) [2023-12-27 17:09:04,365][20381] Updated weights for policy 1, policy_version 1659143 (0.0007) [2023-12-27 17:09:05,000][20381] Updated weights for policy 1, policy_version 1659153 (0.0006) [2023-12-27 17:09:05,015][20347] Updated weights for policy 0, policy_version 1660784 (0.0008) [2023-12-27 17:09:05,052][20381] Updated weights for policy 1, policy_version 1659163 (0.0005) [2023-12-27 17:09:05,072][20347] Updated weights for policy 0, policy_version 1660794 (0.0010) [2023-12-27 17:09:05,112][20381] Updated weights for policy 1, policy_version 1659173 (0.0006) [2023-12-27 17:09:05,128][20347] Updated weights for policy 0, policy_version 1660804 (0.0009) [2023-12-27 17:09:05,175][20381] Updated weights for policy 1, policy_version 1659183 (0.0007) [2023-12-27 17:09:05,739][20347] Updated weights for policy 0, policy_version 1660814 (0.0007) [2023-12-27 17:09:05,790][20347] Updated weights for policy 0, policy_version 1660824 (0.0005) [2023-12-27 17:09:05,822][20381] Updated weights for policy 1, policy_version 1659193 (0.0007) [2023-12-27 17:09:05,853][20347] Updated weights for policy 0, policy_version 1660834 (0.0005) [2023-12-27 17:09:05,874][20381] Updated weights for policy 1, policy_version 1659203 (0.0008) [2023-12-27 17:09:05,928][20381] Updated weights for policy 1, policy_version 1659213 (0.0005) [2023-12-27 17:09:06,445][20347] Updated weights for policy 0, policy_version 1660844 (0.0007) [2023-12-27 17:09:06,495][20347] Updated weights for policy 0, policy_version 1660854 (0.0008) [2023-12-27 17:09:06,558][20347] Updated weights for policy 0, policy_version 1660864 (0.0008) [2023-12-27 17:09:06,567][20381] Updated weights for policy 1, policy_version 1659223 (0.0008) [2023-12-27 17:09:06,626][20381] Updated weights for policy 1, policy_version 1659233 (0.0006) [2023-12-27 17:09:06,689][20381] Updated weights for policy 1, policy_version 1659243 (0.0006) [2023-12-27 17:09:07,258][20347] Updated weights for policy 0, policy_version 1660874 (0.0007) [2023-12-27 17:09:07,319][20347] Updated weights for policy 0, policy_version 1660884 (0.0007) [2023-12-27 17:09:07,325][20381] Updated weights for policy 1, policy_version 1659253 (0.0008) [2023-12-27 17:09:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 850075648. Throughput: 0: 10476.4, 1: 10168.3. Samples: 850050404. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:07,363][19170] Avg episode reward: [(0, '970.680'), (1, '892.720')] [2023-12-27 17:09:07,369][20347] Updated weights for policy 0, policy_version 1660894 (0.0006) [2023-12-27 17:09:07,377][20381] Updated weights for policy 1, policy_version 1659263 (0.0008) [2023-12-27 17:09:07,416][20292] KL-divergence is very high: 159.9991 [2023-12-27 17:09:07,420][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001660904_425271296.pth... [2023-12-27 17:09:07,422][20347] Updated weights for policy 0, policy_version 1660904 (0.0005) [2023-12-27 17:09:07,424][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001659656_424951808.pth [2023-12-27 17:09:07,431][20381] Updated weights for policy 1, policy_version 1659273 (0.0008) [2023-12-27 17:09:07,463][20292] KL-divergence is very high: 131.7454 [2023-12-27 17:09:07,474][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001659280_424820736.pth... [2023-12-27 17:09:07,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001658096_424517632.pth [2023-12-27 17:09:07,997][20347] Updated weights for policy 0, policy_version 1660914 (0.0007) [2023-12-27 17:09:08,043][20347] Updated weights for policy 0, policy_version 1660924 (0.0009) [2023-12-27 17:09:08,091][20381] Updated weights for policy 1, policy_version 1659283 (0.0008) [2023-12-27 17:09:08,103][20347] Updated weights for policy 0, policy_version 1660934 (0.0007) [2023-12-27 17:09:08,139][20381] Updated weights for policy 1, policy_version 1659293 (0.0008) [2023-12-27 17:09:08,186][20381] Updated weights for policy 1, policy_version 1659303 (0.0007) [2023-12-27 17:09:08,767][20347] Updated weights for policy 0, policy_version 1660944 (0.0010) [2023-12-27 17:09:08,819][20347] Updated weights for policy 0, policy_version 1660954 (0.0010) [2023-12-27 17:09:08,865][20347] Updated weights for policy 0, policy_version 1660964 (0.0010) [2023-12-27 17:09:08,915][20381] Updated weights for policy 1, policy_version 1659313 (0.0008) [2023-12-27 17:09:08,967][20381] Updated weights for policy 1, policy_version 1659323 (0.0007) [2023-12-27 17:09:09,017][20381] Updated weights for policy 1, policy_version 1659333 (0.0009) [2023-12-27 17:09:09,062][20381] Updated weights for policy 1, policy_version 1659343 (0.0008) [2023-12-27 17:09:09,549][20347] Updated weights for policy 0, policy_version 1660974 (0.0010) [2023-12-27 17:09:09,609][20347] Updated weights for policy 0, policy_version 1660984 (0.0011) [2023-12-27 17:09:09,668][20347] Updated weights for policy 0, policy_version 1660994 (0.0010) [2023-12-27 17:09:09,795][20381] Updated weights for policy 1, policy_version 1659353 (0.0007) [2023-12-27 17:09:09,856][20381] Updated weights for policy 1, policy_version 1659363 (0.0008) [2023-12-27 17:09:09,917][20381] Updated weights for policy 1, policy_version 1659373 (0.0007) [2023-12-27 17:09:10,387][20347] Updated weights for policy 0, policy_version 1661004 (0.0010) [2023-12-27 17:09:10,457][20347] Updated weights for policy 0, policy_version 1661014 (0.0009) [2023-12-27 17:09:10,518][20347] Updated weights for policy 0, policy_version 1661024 (0.0010) [2023-12-27 17:09:10,630][20381] Updated weights for policy 1, policy_version 1659383 (0.0008) [2023-12-27 17:09:10,686][20381] Updated weights for policy 1, policy_version 1659393 (0.0008) [2023-12-27 17:09:10,745][20381] Updated weights for policy 1, policy_version 1659403 (0.0007) [2023-12-27 17:09:11,226][20347] Updated weights for policy 0, policy_version 1661034 (0.0009) [2023-12-27 17:09:11,286][20347] Updated weights for policy 0, policy_version 1661044 (0.0007) [2023-12-27 17:09:11,347][20347] Updated weights for policy 0, policy_version 1661054 (0.0008) [2023-12-27 17:09:11,418][20347] Updated weights for policy 0, policy_version 1661064 (0.0008) [2023-12-27 17:09:11,502][20381] Updated weights for policy 1, policy_version 1659413 (0.0008) [2023-12-27 17:09:11,561][20381] Updated weights for policy 1, policy_version 1659423 (0.0007) [2023-12-27 17:09:11,616][20381] Updated weights for policy 1, policy_version 1659433 (0.0008) [2023-12-27 17:09:12,151][20347] Updated weights for policy 0, policy_version 1661074 (0.0008) [2023-12-27 17:09:12,207][20347] Updated weights for policy 0, policy_version 1661084 (0.0007) [2023-12-27 17:09:12,264][20347] Updated weights for policy 0, policy_version 1661094 (0.0008) [2023-12-27 17:09:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 850182144. Throughput: 0: 10492.4, 1: 10155.6. Samples: 850174632. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:12,363][19170] Avg episode reward: [(0, '960.960'), (1, '921.880')] [2023-12-27 17:09:12,368][20381] Updated weights for policy 1, policy_version 1659443 (0.0009) [2023-12-27 17:09:12,434][20381] Updated weights for policy 1, policy_version 1659453 (0.0011) [2023-12-27 17:09:12,490][20381] Updated weights for policy 1, policy_version 1659463 (0.0011) [2023-12-27 17:09:12,977][20347] Updated weights for policy 0, policy_version 1661104 (0.0006) [2023-12-27 17:09:13,032][20347] Updated weights for policy 0, policy_version 1661114 (0.0010) [2023-12-27 17:09:13,083][20347] Updated weights for policy 0, policy_version 1661124 (0.0010) [2023-12-27 17:09:13,198][20381] Updated weights for policy 1, policy_version 1659473 (0.0010) [2023-12-27 17:09:13,246][20381] Updated weights for policy 1, policy_version 1659483 (0.0007) [2023-12-27 17:09:13,297][20381] Updated weights for policy 1, policy_version 1659493 (0.0010) [2023-12-27 17:09:13,355][20381] Updated weights for policy 1, policy_version 1659503 (0.0011) [2023-12-27 17:09:13,725][20347] Updated weights for policy 0, policy_version 1661134 (0.0010) [2023-12-27 17:09:13,774][20347] Updated weights for policy 0, policy_version 1661144 (0.0010) [2023-12-27 17:09:13,825][20347] Updated weights for policy 0, policy_version 1661154 (0.0010) [2023-12-27 17:09:14,045][20381] Updated weights for policy 1, policy_version 1659513 (0.0011) [2023-12-27 17:09:14,106][20381] Updated weights for policy 1, policy_version 1659523 (0.0011) [2023-12-27 17:09:14,157][20381] Updated weights for policy 1, policy_version 1659533 (0.0010) [2023-12-27 17:09:14,476][20347] Updated weights for policy 0, policy_version 1661164 (0.0010) [2023-12-27 17:09:14,524][20347] Updated weights for policy 0, policy_version 1661174 (0.0010) [2023-12-27 17:09:14,569][20347] Updated weights for policy 0, policy_version 1661184 (0.0010) [2023-12-27 17:09:14,799][20381] Updated weights for policy 1, policy_version 1659543 (0.0010) [2023-12-27 17:09:14,861][20381] Updated weights for policy 1, policy_version 1659553 (0.0008) [2023-12-27 17:09:14,928][20381] Updated weights for policy 1, policy_version 1659563 (0.0006) [2023-12-27 17:09:15,346][20347] Updated weights for policy 0, policy_version 1661194 (0.0010) [2023-12-27 17:09:15,396][20347] Updated weights for policy 0, policy_version 1661204 (0.0007) [2023-12-27 17:09:15,443][20347] Updated weights for policy 0, policy_version 1661214 (0.0007) [2023-12-27 17:09:15,491][20347] Updated weights for policy 0, policy_version 1661224 (0.0007) [2023-12-27 17:09:15,569][20381] Updated weights for policy 1, policy_version 1659573 (0.0005) [2023-12-27 17:09:15,625][20381] Updated weights for policy 1, policy_version 1659583 (0.0005) [2023-12-27 17:09:15,688][20381] Updated weights for policy 1, policy_version 1659593 (0.0005) [2023-12-27 17:09:16,097][20347] Updated weights for policy 0, policy_version 1661234 (0.0005) [2023-12-27 17:09:16,156][20347] Updated weights for policy 0, policy_version 1661244 (0.0005) [2023-12-27 17:09:16,212][20347] Updated weights for policy 0, policy_version 1661254 (0.0005) [2023-12-27 17:09:16,225][20381] Updated weights for policy 1, policy_version 1659603 (0.0005) [2023-12-27 17:09:16,271][20381] Updated weights for policy 1, policy_version 1659613 (0.0005) [2023-12-27 17:09:16,332][20381] Updated weights for policy 1, policy_version 1659623 (0.0010) [2023-12-27 17:09:16,747][20347] Updated weights for policy 0, policy_version 1661264 (0.0006) [2023-12-27 17:09:16,799][20347] Updated weights for policy 0, policy_version 1661274 (0.0007) [2023-12-27 17:09:16,852][20347] Updated weights for policy 0, policy_version 1661284 (0.0010) [2023-12-27 17:09:16,868][20381] Updated weights for policy 1, policy_version 1659633 (0.0010) [2023-12-27 17:09:16,929][20381] Updated weights for policy 1, policy_version 1659643 (0.0007) [2023-12-27 17:09:16,984][20381] Updated weights for policy 1, policy_version 1659653 (0.0010) [2023-12-27 17:09:17,032][20381] Updated weights for policy 1, policy_version 1659664 (0.0010) [2023-12-27 17:09:17,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.4, 300 sec: 20605.0). Total num frames: 850288640. Throughput: 0: 10450.3, 1: 10145.9. Samples: 850300412. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:17,364][19170] Avg episode reward: [(0, '960.940'), (1, '853.640')] [2023-12-27 17:09:17,433][20347] Updated weights for policy 0, policy_version 1661294 (0.0007) [2023-12-27 17:09:17,492][20347] Updated weights for policy 0, policy_version 1661304 (0.0007) [2023-12-27 17:09:17,548][20347] Updated weights for policy 0, policy_version 1661314 (0.0008) [2023-12-27 17:09:17,665][20381] Updated weights for policy 1, policy_version 1659674 (0.0005) [2023-12-27 17:09:17,726][20381] Updated weights for policy 1, policy_version 1659684 (0.0005) [2023-12-27 17:09:17,783][20381] Updated weights for policy 1, policy_version 1659694 (0.0005) [2023-12-27 17:09:18,202][20347] Updated weights for policy 0, policy_version 1661324 (0.0007) [2023-12-27 17:09:18,258][20347] Updated weights for policy 0, policy_version 1661334 (0.0010) [2023-12-27 17:09:18,313][20347] Updated weights for policy 0, policy_version 1661344 (0.0010) [2023-12-27 17:09:18,389][20381] Updated weights for policy 1, policy_version 1659704 (0.0006) [2023-12-27 17:09:18,449][20381] Updated weights for policy 1, policy_version 1659714 (0.0007) [2023-12-27 17:09:18,514][20381] Updated weights for policy 1, policy_version 1659724 (0.0008) [2023-12-27 17:09:19,012][20347] Updated weights for policy 0, policy_version 1661354 (0.0009) [2023-12-27 17:09:19,077][20347] Updated weights for policy 0, policy_version 1661364 (0.0007) [2023-12-27 17:09:19,120][20381] Updated weights for policy 1, policy_version 1659734 (0.0008) [2023-12-27 17:09:19,133][20347] Updated weights for policy 0, policy_version 1661374 (0.0008) [2023-12-27 17:09:19,172][20381] Updated weights for policy 1, policy_version 1659744 (0.0005) [2023-12-27 17:09:19,186][20347] Updated weights for policy 0, policy_version 1661384 (0.0006) [2023-12-27 17:09:19,240][20381] Updated weights for policy 1, policy_version 1659754 (0.0007) [2023-12-27 17:09:19,944][20347] Updated weights for policy 0, policy_version 1661394 (0.0008) [2023-12-27 17:09:19,991][20381] Updated weights for policy 1, policy_version 1659764 (0.0006) [2023-12-27 17:09:20,000][20347] Updated weights for policy 0, policy_version 1661404 (0.0009) [2023-12-27 17:09:20,050][20381] Updated weights for policy 1, policy_version 1659774 (0.0007) [2023-12-27 17:09:20,061][20347] Updated weights for policy 0, policy_version 1661414 (0.0009) [2023-12-27 17:09:20,108][20381] Updated weights for policy 1, policy_version 1659784 (0.0006) [2023-12-27 17:09:20,780][20347] Updated weights for policy 0, policy_version 1661424 (0.0008) [2023-12-27 17:09:20,822][20381] Updated weights for policy 1, policy_version 1659794 (0.0007) [2023-12-27 17:09:20,840][20347] Updated weights for policy 0, policy_version 1661434 (0.0006) [2023-12-27 17:09:20,881][20381] Updated weights for policy 1, policy_version 1659804 (0.0008) [2023-12-27 17:09:20,901][20347] Updated weights for policy 0, policy_version 1661444 (0.0006) [2023-12-27 17:09:20,940][20381] Updated weights for policy 1, policy_version 1659814 (0.0007) [2023-12-27 17:09:20,996][20381] Updated weights for policy 1, policy_version 1659824 (0.0008) [2023-12-27 17:09:21,574][20347] Updated weights for policy 0, policy_version 1661454 (0.0007) [2023-12-27 17:09:21,638][20347] Updated weights for policy 0, policy_version 1661464 (0.0008) [2023-12-27 17:09:21,710][20347] Updated weights for policy 0, policy_version 1661474 (0.0009) [2023-12-27 17:09:21,804][20381] Updated weights for policy 1, policy_version 1659834 (0.0008) [2023-12-27 17:09:21,859][20381] Updated weights for policy 1, policy_version 1659844 (0.0007) [2023-12-27 17:09:21,918][20381] Updated weights for policy 1, policy_version 1659854 (0.0008) [2023-12-27 17:09:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 850386944. Throughput: 0: 10463.1, 1: 10142.0. Samples: 850361444. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:22,363][19170] Avg episode reward: [(0, '980.390'), (1, '863.530')] [2023-12-27 17:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001659856_424968192.pth... [2023-12-27 17:09:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001661480_425418752.pth... [2023-12-27 17:09:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001658672_424665088.pth [2023-12-27 17:09:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001660264_425107456.pth [2023-12-27 17:09:22,423][20347] Updated weights for policy 0, policy_version 1661484 (0.0009) [2023-12-27 17:09:22,483][20347] Updated weights for policy 0, policy_version 1661494 (0.0007) [2023-12-27 17:09:22,544][20347] Updated weights for policy 0, policy_version 1661504 (0.0007) [2023-12-27 17:09:22,614][20381] Updated weights for policy 1, policy_version 1659864 (0.0007) [2023-12-27 17:09:22,679][20381] Updated weights for policy 1, policy_version 1659874 (0.0009) [2023-12-27 17:09:22,749][20381] Updated weights for policy 1, policy_version 1659884 (0.0008) [2023-12-27 17:09:23,270][20347] Updated weights for policy 0, policy_version 1661514 (0.0007) [2023-12-27 17:09:23,330][20347] Updated weights for policy 0, policy_version 1661524 (0.0006) [2023-12-27 17:09:23,340][20381] Updated weights for policy 1, policy_version 1659894 (0.0008) [2023-12-27 17:09:23,385][20347] Updated weights for policy 0, policy_version 1661534 (0.0006) [2023-12-27 17:09:23,396][20381] Updated weights for policy 1, policy_version 1659904 (0.0008) [2023-12-27 17:09:23,436][20347] Updated weights for policy 0, policy_version 1661544 (0.0006) [2023-12-27 17:09:23,442][20381] Updated weights for policy 1, policy_version 1659914 (0.0008) [2023-12-27 17:09:24,011][20347] Updated weights for policy 0, policy_version 1661554 (0.0008) [2023-12-27 17:09:24,058][20347] Updated weights for policy 0, policy_version 1661564 (0.0009) [2023-12-27 17:09:24,113][20347] Updated weights for policy 0, policy_version 1661574 (0.0010) [2023-12-27 17:09:24,131][20381] Updated weights for policy 1, policy_version 1659924 (0.0008) [2023-12-27 17:09:24,202][20381] Updated weights for policy 1, policy_version 1659934 (0.0008) [2023-12-27 17:09:24,264][20381] Updated weights for policy 1, policy_version 1659944 (0.0008) [2023-12-27 17:09:24,825][20347] Updated weights for policy 0, policy_version 1661584 (0.0010) [2023-12-27 17:09:24,872][20381] Updated weights for policy 1, policy_version 1659954 (0.0007) [2023-12-27 17:09:24,886][20347] Updated weights for policy 0, policy_version 1661594 (0.0010) [2023-12-27 17:09:24,923][20381] Updated weights for policy 1, policy_version 1659964 (0.0006) [2023-12-27 17:09:24,946][20347] Updated weights for policy 0, policy_version 1661604 (0.0010) [2023-12-27 17:09:24,981][20381] Updated weights for policy 1, policy_version 1659974 (0.0009) [2023-12-27 17:09:25,033][20381] Updated weights for policy 1, policy_version 1659984 (0.0011) [2023-12-27 17:09:25,514][20347] Updated weights for policy 0, policy_version 1661614 (0.0009) [2023-12-27 17:09:25,565][20347] Updated weights for policy 0, policy_version 1661624 (0.0008) [2023-12-27 17:09:25,619][20347] Updated weights for policy 0, policy_version 1661634 (0.0007) [2023-12-27 17:09:25,636][20381] Updated weights for policy 1, policy_version 1659994 (0.0011) [2023-12-27 17:09:25,687][20381] Updated weights for policy 1, policy_version 1660004 (0.0010) [2023-12-27 17:09:25,753][20381] Updated weights for policy 1, policy_version 1660014 (0.0011) [2023-12-27 17:09:26,232][20347] Updated weights for policy 0, policy_version 1661644 (0.0005) [2023-12-27 17:09:26,287][20347] Updated weights for policy 0, policy_version 1661654 (0.0006) [2023-12-27 17:09:26,342][20381] Updated weights for policy 1, policy_version 1660024 (0.0010) [2023-12-27 17:09:26,348][20347] Updated weights for policy 0, policy_version 1661664 (0.0006) [2023-12-27 17:09:26,390][20381] Updated weights for policy 1, policy_version 1660034 (0.0010) [2023-12-27 17:09:26,440][20381] Updated weights for policy 1, policy_version 1660044 (0.0009) [2023-12-27 17:09:27,000][20347] Updated weights for policy 0, policy_version 1661674 (0.0005) [2023-12-27 17:09:27,053][20347] Updated weights for policy 0, policy_version 1661684 (0.0005) [2023-12-27 17:09:27,109][20347] Updated weights for policy 0, policy_version 1661694 (0.0005) [2023-12-27 17:09:27,111][20381] Updated weights for policy 1, policy_version 1660054 (0.0008) [2023-12-27 17:09:27,157][20347] Updated weights for policy 0, policy_version 1661704 (0.0005) [2023-12-27 17:09:27,165][20381] Updated weights for policy 1, policy_version 1660064 (0.0008) [2023-12-27 17:09:27,211][20381] Updated weights for policy 1, policy_version 1660074 (0.0005) [2023-12-27 17:09:27,363][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 850501632. Throughput: 0: 10483.1, 1: 10162.8. Samples: 850486264. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:27,363][19170] Avg episode reward: [(0, '980.490'), (1, '882.820')] [2023-12-27 17:09:27,726][20347] Updated weights for policy 0, policy_version 1661714 (0.0008) [2023-12-27 17:09:27,780][20381] Updated weights for policy 1, policy_version 1660084 (0.0006) [2023-12-27 17:09:27,784][20347] Updated weights for policy 0, policy_version 1661724 (0.0009) [2023-12-27 17:09:27,833][20381] Updated weights for policy 1, policy_version 1660094 (0.0008) [2023-12-27 17:09:27,837][20347] Updated weights for policy 0, policy_version 1661734 (0.0008) [2023-12-27 17:09:27,886][20381] Updated weights for policy 1, policy_version 1660104 (0.0008) [2023-12-27 17:09:28,482][20347] Updated weights for policy 0, policy_version 1661744 (0.0010) [2023-12-27 17:09:28,539][20347] Updated weights for policy 0, policy_version 1661754 (0.0009) [2023-12-27 17:09:28,579][20381] Updated weights for policy 1, policy_version 1660114 (0.0008) [2023-12-27 17:09:28,600][20347] Updated weights for policy 0, policy_version 1661764 (0.0005) [2023-12-27 17:09:28,629][20381] Updated weights for policy 1, policy_version 1660124 (0.0011) [2023-12-27 17:09:28,678][20381] Updated weights for policy 1, policy_version 1660134 (0.0011) [2023-12-27 17:09:28,724][20381] Updated weights for policy 1, policy_version 1660144 (0.0010) [2023-12-27 17:09:29,331][20347] Updated weights for policy 0, policy_version 1661774 (0.0007) [2023-12-27 17:09:29,402][20347] Updated weights for policy 0, policy_version 1661784 (0.0008) [2023-12-27 17:09:29,441][20381] Updated weights for policy 1, policy_version 1660154 (0.0010) [2023-12-27 17:09:29,472][20347] Updated weights for policy 0, policy_version 1661794 (0.0008) [2023-12-27 17:09:29,501][20381] Updated weights for policy 1, policy_version 1660164 (0.0011) [2023-12-27 17:09:29,561][20381] Updated weights for policy 1, policy_version 1660174 (0.0011) [2023-12-27 17:09:30,103][20347] Updated weights for policy 0, policy_version 1661804 (0.0008) [2023-12-27 17:09:30,170][20347] Updated weights for policy 0, policy_version 1661814 (0.0008) [2023-12-27 17:09:30,230][20347] Updated weights for policy 0, policy_version 1661824 (0.0008) [2023-12-27 17:09:30,260][20381] Updated weights for policy 1, policy_version 1660184 (0.0010) [2023-12-27 17:09:30,315][20381] Updated weights for policy 1, policy_version 1660194 (0.0010) [2023-12-27 17:09:30,369][20381] Updated weights for policy 1, policy_version 1660204 (0.0010) [2023-12-27 17:09:30,828][20347] Updated weights for policy 0, policy_version 1661834 (0.0008) [2023-12-27 17:09:30,879][20347] Updated weights for policy 0, policy_version 1661844 (0.0005) [2023-12-27 17:09:30,927][20347] Updated weights for policy 0, policy_version 1661854 (0.0005) [2023-12-27 17:09:30,976][20347] Updated weights for policy 0, policy_version 1661864 (0.0005) [2023-12-27 17:09:31,006][20381] Updated weights for policy 1, policy_version 1660214 (0.0009) [2023-12-27 17:09:31,066][20381] Updated weights for policy 1, policy_version 1660224 (0.0007) [2023-12-27 17:09:31,130][20381] Updated weights for policy 1, policy_version 1660234 (0.0008) [2023-12-27 17:09:31,612][20347] Updated weights for policy 0, policy_version 1661874 (0.0007) [2023-12-27 17:09:31,678][20347] Updated weights for policy 0, policy_version 1661884 (0.0007) [2023-12-27 17:09:31,743][20347] Updated weights for policy 0, policy_version 1661894 (0.0007) [2023-12-27 17:09:31,845][20381] Updated weights for policy 1, policy_version 1660244 (0.0008) [2023-12-27 17:09:31,902][20381] Updated weights for policy 1, policy_version 1660254 (0.0008) [2023-12-27 17:09:31,952][20381] Updated weights for policy 1, policy_version 1660264 (0.0007) [2023-12-27 17:09:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 850599936. Throughput: 0: 10482.2, 1: 10179.8. Samples: 850612444. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:32,363][19170] Avg episode reward: [(0, '980.600'), (1, '824.120')] [2023-12-27 17:09:32,420][20347] Updated weights for policy 0, policy_version 1661904 (0.0007) [2023-12-27 17:09:32,472][20347] Updated weights for policy 0, policy_version 1661914 (0.0006) [2023-12-27 17:09:32,530][20347] Updated weights for policy 0, policy_version 1661924 (0.0006) [2023-12-27 17:09:32,692][20381] Updated weights for policy 1, policy_version 1660274 (0.0008) [2023-12-27 17:09:32,758][20381] Updated weights for policy 1, policy_version 1660284 (0.0011) [2023-12-27 17:09:32,824][20381] Updated weights for policy 1, policy_version 1660294 (0.0011) [2023-12-27 17:09:32,886][20381] Updated weights for policy 1, policy_version 1660304 (0.0011) [2023-12-27 17:09:33,083][20347] Updated weights for policy 0, policy_version 1661934 (0.0007) [2023-12-27 17:09:33,134][20347] Updated weights for policy 0, policy_version 1661944 (0.0006) [2023-12-27 17:09:33,189][20347] Updated weights for policy 0, policy_version 1661954 (0.0007) [2023-12-27 17:09:33,515][20381] Updated weights for policy 1, policy_version 1660314 (0.0005) [2023-12-27 17:09:33,578][20381] Updated weights for policy 1, policy_version 1660324 (0.0005) [2023-12-27 17:09:33,637][20381] Updated weights for policy 1, policy_version 1660334 (0.0010) [2023-12-27 17:09:33,835][20347] Updated weights for policy 0, policy_version 1661964 (0.0007) [2023-12-27 17:09:33,891][20347] Updated weights for policy 0, policy_version 1661974 (0.0007) [2023-12-27 17:09:33,944][20347] Updated weights for policy 0, policy_version 1661984 (0.0007) [2023-12-27 17:09:34,270][20381] Updated weights for policy 1, policy_version 1660344 (0.0009) [2023-12-27 17:09:34,326][20381] Updated weights for policy 1, policy_version 1660354 (0.0007) [2023-12-27 17:09:34,385][20381] Updated weights for policy 1, policy_version 1660364 (0.0007) [2023-12-27 17:09:34,650][20347] Updated weights for policy 0, policy_version 1661994 (0.0007) [2023-12-27 17:09:34,708][20347] Updated weights for policy 0, policy_version 1662004 (0.0007) [2023-12-27 17:09:34,767][20347] Updated weights for policy 0, policy_version 1662014 (0.0007) [2023-12-27 17:09:34,822][20347] Updated weights for policy 0, policy_version 1662024 (0.0007) [2023-12-27 17:09:35,041][20381] Updated weights for policy 1, policy_version 1660374 (0.0006) [2023-12-27 17:09:35,096][20381] Updated weights for policy 1, policy_version 1660384 (0.0005) [2023-12-27 17:09:35,148][20381] Updated weights for policy 1, policy_version 1660394 (0.0006) [2023-12-27 17:09:35,552][20347] Updated weights for policy 0, policy_version 1662034 (0.0007) [2023-12-27 17:09:35,610][20347] Updated weights for policy 0, policy_version 1662044 (0.0007) [2023-12-27 17:09:35,668][20347] Updated weights for policy 0, policy_version 1662054 (0.0007) [2023-12-27 17:09:35,753][20381] Updated weights for policy 1, policy_version 1660404 (0.0009) [2023-12-27 17:09:35,805][20381] Updated weights for policy 1, policy_version 1660414 (0.0011) [2023-12-27 17:09:35,858][20381] Updated weights for policy 1, policy_version 1660424 (0.0007) [2023-12-27 17:09:36,316][20347] Updated weights for policy 0, policy_version 1662064 (0.0006) [2023-12-27 17:09:36,380][20347] Updated weights for policy 0, policy_version 1662074 (0.0006) [2023-12-27 17:09:36,444][20347] Updated weights for policy 0, policy_version 1662084 (0.0007) [2023-12-27 17:09:36,620][20381] Updated weights for policy 1, policy_version 1660434 (0.0010) [2023-12-27 17:09:36,685][20381] Updated weights for policy 1, policy_version 1660444 (0.0008) [2023-12-27 17:09:36,744][20381] Updated weights for policy 1, policy_version 1660454 (0.0011) [2023-12-27 17:09:36,802][20381] Updated weights for policy 1, policy_version 1660464 (0.0011) [2023-12-27 17:09:37,028][20347] Updated weights for policy 0, policy_version 1662094 (0.0006) [2023-12-27 17:09:37,089][20347] Updated weights for policy 0, policy_version 1662104 (0.0005) [2023-12-27 17:09:37,152][20347] Updated weights for policy 0, policy_version 1662114 (0.0008) [2023-12-27 17:09:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20604.9). Total num frames: 850706432. Throughput: 0: 10459.6, 1: 10228.8. Samples: 850675372. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:37,364][19170] Avg episode reward: [(0, '980.510'), (1, '892.660')] [2023-12-27 17:09:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001662120_425582592.pth... [2023-12-27 17:09:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001660904_425271296.pth [2023-12-27 17:09:37,390][20381] Updated weights for policy 1, policy_version 1660474 (0.0011) [2023-12-27 17:09:37,439][20381] Updated weights for policy 1, policy_version 1660484 (0.0011) [2023-12-27 17:09:37,494][20381] Updated weights for policy 1, policy_version 1660494 (0.0011) [2023-12-27 17:09:37,503][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001660496_425132032.pth... [2023-12-27 17:09:37,507][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001659280_424820736.pth [2023-12-27 17:09:37,702][20347] Updated weights for policy 0, policy_version 1662124 (0.0006) [2023-12-27 17:09:37,763][20347] Updated weights for policy 0, policy_version 1662134 (0.0006) [2023-12-27 17:09:37,818][20347] Updated weights for policy 0, policy_version 1662144 (0.0007) [2023-12-27 17:09:38,199][20381] Updated weights for policy 1, policy_version 1660504 (0.0011) [2023-12-27 17:09:38,247][20381] Updated weights for policy 1, policy_version 1660514 (0.0010) [2023-12-27 17:09:38,302][20381] Updated weights for policy 1, policy_version 1660524 (0.0011) [2023-12-27 17:09:38,369][20347] Updated weights for policy 0, policy_version 1662154 (0.0007) [2023-12-27 17:09:38,426][20347] Updated weights for policy 0, policy_version 1662164 (0.0008) [2023-12-27 17:09:38,487][20347] Updated weights for policy 0, policy_version 1662174 (0.0008) [2023-12-27 17:09:38,543][20347] Updated weights for policy 0, policy_version 1662184 (0.0008) [2023-12-27 17:09:38,969][20381] Updated weights for policy 1, policy_version 1660534 (0.0011) [2023-12-27 17:09:39,022][20381] Updated weights for policy 1, policy_version 1660544 (0.0010) [2023-12-27 17:09:39,080][20381] Updated weights for policy 1, policy_version 1660554 (0.0010) [2023-12-27 17:09:39,280][20347] Updated weights for policy 0, policy_version 1662194 (0.0007) [2023-12-27 17:09:39,342][20347] Updated weights for policy 0, policy_version 1662204 (0.0006) [2023-12-27 17:09:39,413][20347] Updated weights for policy 0, policy_version 1662214 (0.0008) [2023-12-27 17:09:39,761][20381] Updated weights for policy 1, policy_version 1660564 (0.0009) [2023-12-27 17:09:39,815][20381] Updated weights for policy 1, policy_version 1660574 (0.0008) [2023-12-27 17:09:39,887][20381] Updated weights for policy 1, policy_version 1660584 (0.0007) [2023-12-27 17:09:40,111][20347] Updated weights for policy 0, policy_version 1662224 (0.0006) [2023-12-27 17:09:40,173][20347] Updated weights for policy 0, policy_version 1662234 (0.0006) [2023-12-27 17:09:40,239][20347] Updated weights for policy 0, policy_version 1662244 (0.0006) [2023-12-27 17:09:40,611][20381] Updated weights for policy 1, policy_version 1660594 (0.0007) [2023-12-27 17:09:40,670][20381] Updated weights for policy 1, policy_version 1660604 (0.0007) [2023-12-27 17:09:40,725][20381] Updated weights for policy 1, policy_version 1660614 (0.0006) [2023-12-27 17:09:40,794][20381] Updated weights for policy 1, policy_version 1660624 (0.0006) [2023-12-27 17:09:40,837][20347] Updated weights for policy 0, policy_version 1662254 (0.0007) [2023-12-27 17:09:40,894][20347] Updated weights for policy 0, policy_version 1662264 (0.0008) [2023-12-27 17:09:40,940][20347] Updated weights for policy 0, policy_version 1662274 (0.0006) [2023-12-27 17:09:41,422][20381] Updated weights for policy 1, policy_version 1660634 (0.0008) [2023-12-27 17:09:41,482][20381] Updated weights for policy 1, policy_version 1660644 (0.0005) [2023-12-27 17:09:41,541][20381] Updated weights for policy 1, policy_version 1660654 (0.0008) [2023-12-27 17:09:41,660][20347] Updated weights for policy 0, policy_version 1662284 (0.0006) [2023-12-27 17:09:41,721][20347] Updated weights for policy 0, policy_version 1662294 (0.0008) [2023-12-27 17:09:41,788][20347] Updated weights for policy 0, policy_version 1662304 (0.0008) [2023-12-27 17:09:42,254][20381] Updated weights for policy 1, policy_version 1660664 (0.0008) [2023-12-27 17:09:42,315][20381] Updated weights for policy 1, policy_version 1660674 (0.0008) [2023-12-27 17:09:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 850804736. Throughput: 0: 10516.4, 1: 10269.1. Samples: 850801296. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:42,363][19170] Avg episode reward: [(0, '980.410'), (1, '941.570')] [2023-12-27 17:09:42,377][20381] Updated weights for policy 1, policy_version 1660684 (0.0008) [2023-12-27 17:09:42,416][20347] Updated weights for policy 0, policy_version 1662314 (0.0007) [2023-12-27 17:09:42,475][20347] Updated weights for policy 0, policy_version 1662324 (0.0008) [2023-12-27 17:09:42,537][20347] Updated weights for policy 0, policy_version 1662334 (0.0008) [2023-12-27 17:09:42,594][20347] Updated weights for policy 0, policy_version 1662344 (0.0008) [2023-12-27 17:09:43,019][20381] Updated weights for policy 1, policy_version 1660694 (0.0008) [2023-12-27 17:09:43,062][20381] Updated weights for policy 1, policy_version 1660704 (0.0008) [2023-12-27 17:09:43,108][20381] Updated weights for policy 1, policy_version 1660714 (0.0008) [2023-12-27 17:09:43,324][20347] Updated weights for policy 0, policy_version 1662354 (0.0008) [2023-12-27 17:09:43,375][20347] Updated weights for policy 0, policy_version 1662364 (0.0007) [2023-12-27 17:09:43,422][20347] Updated weights for policy 0, policy_version 1662374 (0.0007) [2023-12-27 17:09:43,799][20381] Updated weights for policy 1, policy_version 1660724 (0.0008) [2023-12-27 17:09:43,852][20381] Updated weights for policy 1, policy_version 1660734 (0.0007) [2023-12-27 17:09:43,901][20381] Updated weights for policy 1, policy_version 1660744 (0.0007) [2023-12-27 17:09:44,123][20347] Updated weights for policy 0, policy_version 1662384 (0.0008) [2023-12-27 17:09:44,182][20347] Updated weights for policy 0, policy_version 1662394 (0.0008) [2023-12-27 17:09:44,233][20347] Updated weights for policy 0, policy_version 1662404 (0.0007) [2023-12-27 17:09:44,569][20381] Updated weights for policy 1, policy_version 1660754 (0.0007) [2023-12-27 17:09:44,632][20381] Updated weights for policy 1, policy_version 1660764 (0.0011) [2023-12-27 17:09:44,688][20381] Updated weights for policy 1, policy_version 1660774 (0.0008) [2023-12-27 17:09:44,754][20381] Updated weights for policy 1, policy_version 1660784 (0.0008) [2023-12-27 17:09:44,912][20347] Updated weights for policy 0, policy_version 1662414 (0.0008) [2023-12-27 17:09:44,978][20347] Updated weights for policy 0, policy_version 1662424 (0.0006) [2023-12-27 17:09:45,044][20347] Updated weights for policy 0, policy_version 1662434 (0.0006) [2023-12-27 17:09:45,436][20381] Updated weights for policy 1, policy_version 1660794 (0.0008) [2023-12-27 17:09:45,484][20381] Updated weights for policy 1, policy_version 1660804 (0.0007) [2023-12-27 17:09:45,539][20381] Updated weights for policy 1, policy_version 1660814 (0.0007) [2023-12-27 17:09:45,766][20347] Updated weights for policy 0, policy_version 1662444 (0.0008) [2023-12-27 17:09:45,815][20347] Updated weights for policy 0, policy_version 1662454 (0.0008) [2023-12-27 17:09:45,871][20347] Updated weights for policy 0, policy_version 1662464 (0.0007) [2023-12-27 17:09:46,238][20381] Updated weights for policy 1, policy_version 1660824 (0.0008) [2023-12-27 17:09:46,297][20381] Updated weights for policy 1, policy_version 1660834 (0.0008) [2023-12-27 17:09:46,352][20381] Updated weights for policy 1, policy_version 1660844 (0.0008) [2023-12-27 17:09:46,556][20347] Updated weights for policy 0, policy_version 1662474 (0.0007) [2023-12-27 17:09:46,621][20347] Updated weights for policy 0, policy_version 1662484 (0.0008) [2023-12-27 17:09:46,680][20347] Updated weights for policy 0, policy_version 1662494 (0.0006) [2023-12-27 17:09:46,729][20347] Updated weights for policy 0, policy_version 1662504 (0.0005) [2023-12-27 17:09:47,069][20381] Updated weights for policy 1, policy_version 1660854 (0.0008) [2023-12-27 17:09:47,125][20381] Updated weights for policy 1, policy_version 1660864 (0.0007) [2023-12-27 17:09:47,174][20381] Updated weights for policy 1, policy_version 1660874 (0.0010) [2023-12-27 17:09:47,264][20347] Updated weights for policy 0, policy_version 1662514 (0.0005) [2023-12-27 17:09:47,320][20347] Updated weights for policy 0, policy_version 1662524 (0.0005) [2023-12-27 17:09:47,362][19170] Fps is (10 sec: 20480.9, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 850911232. Throughput: 0: 10481.6, 1: 10283.0. Samples: 850922084. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:47,363][19170] Avg episode reward: [(0, '990.200'), (1, '853.690')] [2023-12-27 17:09:47,371][20347] Updated weights for policy 0, policy_version 1662534 (0.0005) [2023-12-27 17:09:47,881][20381] Updated weights for policy 1, policy_version 1660884 (0.0007) [2023-12-27 17:09:47,935][20381] Updated weights for policy 1, policy_version 1660894 (0.0005) [2023-12-27 17:09:47,952][20347] Updated weights for policy 0, policy_version 1662544 (0.0005) [2023-12-27 17:09:47,992][20381] Updated weights for policy 1, policy_version 1660904 (0.0007) [2023-12-27 17:09:48,005][20347] Updated weights for policy 0, policy_version 1662554 (0.0006) [2023-12-27 17:09:48,054][20347] Updated weights for policy 0, policy_version 1662564 (0.0008) [2023-12-27 17:09:48,652][20347] Updated weights for policy 0, policy_version 1662574 (0.0006) [2023-12-27 17:09:48,705][20347] Updated weights for policy 0, policy_version 1662584 (0.0005) [2023-12-27 17:09:48,719][20381] Updated weights for policy 1, policy_version 1660914 (0.0006) [2023-12-27 17:09:48,763][20347] Updated weights for policy 0, policy_version 1662594 (0.0005) [2023-12-27 17:09:48,776][20381] Updated weights for policy 1, policy_version 1660924 (0.0007) [2023-12-27 17:09:48,839][20381] Updated weights for policy 1, policy_version 1660934 (0.0008) [2023-12-27 17:09:48,892][20381] Updated weights for policy 1, policy_version 1660944 (0.0009) [2023-12-27 17:09:49,411][20347] Updated weights for policy 0, policy_version 1662604 (0.0006) [2023-12-27 17:09:49,462][20347] Updated weights for policy 0, policy_version 1662614 (0.0005) [2023-12-27 17:09:49,516][20347] Updated weights for policy 0, policy_version 1662624 (0.0007) [2023-12-27 17:09:49,593][20381] Updated weights for policy 1, policy_version 1660954 (0.0007) [2023-12-27 17:09:49,649][20381] Updated weights for policy 1, policy_version 1660964 (0.0008) [2023-12-27 17:09:49,711][20381] Updated weights for policy 1, policy_version 1660974 (0.0006) [2023-12-27 17:09:50,186][20347] Updated weights for policy 0, policy_version 1662634 (0.0006) [2023-12-27 17:09:50,244][20347] Updated weights for policy 0, policy_version 1662644 (0.0008) [2023-12-27 17:09:50,308][20347] Updated weights for policy 0, policy_version 1662654 (0.0006) [2023-12-27 17:09:50,370][20347] Updated weights for policy 0, policy_version 1662664 (0.0008) [2023-12-27 17:09:50,381][20381] Updated weights for policy 1, policy_version 1660984 (0.0006) [2023-12-27 17:09:50,446][20381] Updated weights for policy 1, policy_version 1660994 (0.0008) [2023-12-27 17:09:50,499][20381] Updated weights for policy 1, policy_version 1661004 (0.0008) [2023-12-27 17:09:51,037][20347] Updated weights for policy 0, policy_version 1662674 (0.0007) [2023-12-27 17:09:51,099][20347] Updated weights for policy 0, policy_version 1662684 (0.0008) [2023-12-27 17:09:51,164][20347] Updated weights for policy 0, policy_version 1662694 (0.0008) [2023-12-27 17:09:51,233][20381] Updated weights for policy 1, policy_version 1661014 (0.0008) [2023-12-27 17:09:51,298][20381] Updated weights for policy 1, policy_version 1661024 (0.0008) [2023-12-27 17:09:51,358][20381] Updated weights for policy 1, policy_version 1661034 (0.0008) [2023-12-27 17:09:51,870][20347] Updated weights for policy 0, policy_version 1662704 (0.0008) [2023-12-27 17:09:51,926][20347] Updated weights for policy 0, policy_version 1662714 (0.0007) [2023-12-27 17:09:51,980][20347] Updated weights for policy 0, policy_version 1662724 (0.0006) [2023-12-27 17:09:52,090][20381] Updated weights for policy 1, policy_version 1661044 (0.0007) [2023-12-27 17:09:52,138][20381] Updated weights for policy 1, policy_version 1661054 (0.0007) [2023-12-27 17:09:52,191][20381] Updated weights for policy 1, policy_version 1661064 (0.0008) [2023-12-27 17:09:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 851017728. Throughput: 0: 10487.5, 1: 10294.8. Samples: 850985612. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:52,363][19170] Avg episode reward: [(0, '990.230'), (1, '795.350')] [2023-12-27 17:09:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001661072_425279488.pth... [2023-12-27 17:09:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001662728_425738240.pth... [2023-12-27 17:09:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001659856_424968192.pth [2023-12-27 17:09:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001661480_425418752.pth [2023-12-27 17:09:52,701][20347] Updated weights for policy 0, policy_version 1662734 (0.0007) [2023-12-27 17:09:52,750][20347] Updated weights for policy 0, policy_version 1662744 (0.0006) [2023-12-27 17:09:52,791][20381] Updated weights for policy 1, policy_version 1661074 (0.0008) [2023-12-27 17:09:52,808][20347] Updated weights for policy 0, policy_version 1662754 (0.0007) [2023-12-27 17:09:52,850][20381] Updated weights for policy 1, policy_version 1661084 (0.0006) [2023-12-27 17:09:52,900][20381] Updated weights for policy 1, policy_version 1661094 (0.0007) [2023-12-27 17:09:52,956][20381] Updated weights for policy 1, policy_version 1661104 (0.0008) [2023-12-27 17:09:53,488][20347] Updated weights for policy 0, policy_version 1662764 (0.0007) [2023-12-27 17:09:53,540][20347] Updated weights for policy 0, policy_version 1662774 (0.0007) [2023-12-27 17:09:53,591][20347] Updated weights for policy 0, policy_version 1662784 (0.0008) [2023-12-27 17:09:53,665][20381] Updated weights for policy 1, policy_version 1661114 (0.0009) [2023-12-27 17:09:53,720][20381] Updated weights for policy 1, policy_version 1661124 (0.0008) [2023-12-27 17:09:53,774][20381] Updated weights for policy 1, policy_version 1661134 (0.0008) [2023-12-27 17:09:54,258][20347] Updated weights for policy 0, policy_version 1662794 (0.0007) [2023-12-27 17:09:54,314][20347] Updated weights for policy 0, policy_version 1662804 (0.0008) [2023-12-27 17:09:54,377][20347] Updated weights for policy 0, policy_version 1662814 (0.0007) [2023-12-27 17:09:54,434][20347] Updated weights for policy 0, policy_version 1662824 (0.0007) [2023-12-27 17:09:54,508][20381] Updated weights for policy 1, policy_version 1661144 (0.0008) [2023-12-27 17:09:54,568][20381] Updated weights for policy 1, policy_version 1661154 (0.0007) [2023-12-27 17:09:54,623][20381] Updated weights for policy 1, policy_version 1661164 (0.0008) [2023-12-27 17:09:55,097][20347] Updated weights for policy 0, policy_version 1662834 (0.0006) [2023-12-27 17:09:55,153][20347] Updated weights for policy 0, policy_version 1662844 (0.0008) [2023-12-27 17:09:55,213][20347] Updated weights for policy 0, policy_version 1662854 (0.0007) [2023-12-27 17:09:55,333][20381] Updated weights for policy 1, policy_version 1661174 (0.0008) [2023-12-27 17:09:55,385][20381] Updated weights for policy 1, policy_version 1661184 (0.0007) [2023-12-27 17:09:55,441][20381] Updated weights for policy 1, policy_version 1661194 (0.0008) [2023-12-27 17:09:55,897][20347] Updated weights for policy 0, policy_version 1662864 (0.0008) [2023-12-27 17:09:55,951][20347] Updated weights for policy 0, policy_version 1662874 (0.0008) [2023-12-27 17:09:55,997][20347] Updated weights for policy 0, policy_version 1662884 (0.0008) [2023-12-27 17:09:56,115][20381] Updated weights for policy 1, policy_version 1661204 (0.0008) [2023-12-27 17:09:56,169][20381] Updated weights for policy 1, policy_version 1661214 (0.0008) [2023-12-27 17:09:56,217][20381] Updated weights for policy 1, policy_version 1661224 (0.0010) [2023-12-27 17:09:56,634][20347] Updated weights for policy 0, policy_version 1662894 (0.0006) [2023-12-27 17:09:56,689][20347] Updated weights for policy 0, policy_version 1662904 (0.0005) [2023-12-27 17:09:56,740][20347] Updated weights for policy 0, policy_version 1662914 (0.0007) [2023-12-27 17:09:56,908][20381] Updated weights for policy 1, policy_version 1661234 (0.0009) [2023-12-27 17:09:56,965][20381] Updated weights for policy 1, policy_version 1661244 (0.0006) [2023-12-27 17:09:57,013][20381] Updated weights for policy 1, policy_version 1661254 (0.0010) [2023-12-27 17:09:57,071][20381] Updated weights for policy 1, policy_version 1661264 (0.0010) [2023-12-27 17:09:57,312][20347] Updated weights for policy 0, policy_version 1662924 (0.0008) [2023-12-27 17:09:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 851116032. Throughput: 0: 10434.0, 1: 10260.4. Samples: 851105876. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:09:57,363][19170] Avg episode reward: [(0, '980.430'), (1, '824.360')] [2023-12-27 17:09:57,365][20347] Updated weights for policy 0, policy_version 1662934 (0.0009) [2023-12-27 17:09:57,409][20347] Updated weights for policy 0, policy_version 1662944 (0.0009) [2023-12-27 17:09:57,703][20381] Updated weights for policy 1, policy_version 1661274 (0.0010) [2023-12-27 17:09:57,754][20381] Updated weights for policy 1, policy_version 1661284 (0.0010) [2023-12-27 17:09:57,795][20381] Updated weights for policy 1, policy_version 1661294 (0.0010) [2023-12-27 17:09:58,113][20347] Updated weights for policy 0, policy_version 1662954 (0.0007) [2023-12-27 17:09:58,160][20347] Updated weights for policy 0, policy_version 1662964 (0.0007) [2023-12-27 17:09:58,219][20347] Updated weights for policy 0, policy_version 1662974 (0.0008) [2023-12-27 17:09:58,281][20347] Updated weights for policy 0, policy_version 1662984 (0.0008) [2023-12-27 17:09:58,494][20381] Updated weights for policy 1, policy_version 1661304 (0.0011) [2023-12-27 17:09:58,557][20381] Updated weights for policy 1, policy_version 1661314 (0.0010) [2023-12-27 17:09:58,610][20381] Updated weights for policy 1, policy_version 1661324 (0.0010) [2023-12-27 17:09:59,075][20347] Updated weights for policy 0, policy_version 1662994 (0.0008) [2023-12-27 17:09:59,127][20347] Updated weights for policy 0, policy_version 1663004 (0.0008) [2023-12-27 17:09:59,179][20347] Updated weights for policy 0, policy_version 1663014 (0.0008) [2023-12-27 17:09:59,381][20381] Updated weights for policy 1, policy_version 1661334 (0.0009) [2023-12-27 17:09:59,438][20381] Updated weights for policy 1, policy_version 1661344 (0.0008) [2023-12-27 17:09:59,499][20381] Updated weights for policy 1, policy_version 1661354 (0.0005) [2023-12-27 17:09:59,834][20347] Updated weights for policy 0, policy_version 1663024 (0.0007) [2023-12-27 17:09:59,894][20347] Updated weights for policy 0, policy_version 1663034 (0.0008) [2023-12-27 17:09:59,956][20347] Updated weights for policy 0, policy_version 1663044 (0.0008) [2023-12-27 17:10:00,174][20381] Updated weights for policy 1, policy_version 1661364 (0.0006) [2023-12-27 17:10:00,233][20381] Updated weights for policy 1, policy_version 1661374 (0.0008) [2023-12-27 17:10:00,282][20381] Updated weights for policy 1, policy_version 1661384 (0.0008) [2023-12-27 17:10:00,724][20347] Updated weights for policy 0, policy_version 1663054 (0.0009) [2023-12-27 17:10:00,784][20347] Updated weights for policy 0, policy_version 1663064 (0.0007) [2023-12-27 17:10:00,840][20347] Updated weights for policy 0, policy_version 1663074 (0.0007) [2023-12-27 17:10:00,911][20381] Updated weights for policy 1, policy_version 1661394 (0.0009) [2023-12-27 17:10:00,963][20381] Updated weights for policy 1, policy_version 1661404 (0.0010) [2023-12-27 17:10:01,012][20381] Updated weights for policy 1, policy_version 1661414 (0.0010) [2023-12-27 17:10:01,073][20381] Updated weights for policy 1, policy_version 1661424 (0.0011) [2023-12-27 17:10:01,543][20347] Updated weights for policy 0, policy_version 1663084 (0.0006) [2023-12-27 17:10:01,592][20347] Updated weights for policy 0, policy_version 1663094 (0.0006) [2023-12-27 17:10:01,651][20347] Updated weights for policy 0, policy_version 1663104 (0.0009) [2023-12-27 17:10:01,864][20381] Updated weights for policy 1, policy_version 1661434 (0.0008) [2023-12-27 17:10:01,922][20381] Updated weights for policy 1, policy_version 1661444 (0.0008) [2023-12-27 17:10:01,981][20381] Updated weights for policy 1, policy_version 1661454 (0.0009) [2023-12-27 17:10:02,339][20347] Updated weights for policy 0, policy_version 1663114 (0.0009) [2023-12-27 17:10:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 851214336. Throughput: 0: 10401.0, 1: 10192.3. Samples: 851227108. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:02,363][19170] Avg episode reward: [(0, '960.860'), (1, '814.650')] [2023-12-27 17:10:02,409][20347] Updated weights for policy 0, policy_version 1663124 (0.0010) [2023-12-27 17:10:02,468][20347] Updated weights for policy 0, policy_version 1663134 (0.0010) [2023-12-27 17:10:02,525][20347] Updated weights for policy 0, policy_version 1663144 (0.0010) [2023-12-27 17:10:02,695][20381] Updated weights for policy 1, policy_version 1661464 (0.0010) [2023-12-27 17:10:02,749][20381] Updated weights for policy 1, policy_version 1661474 (0.0010) [2023-12-27 17:10:02,820][20381] Updated weights for policy 1, policy_version 1661484 (0.0010) [2023-12-27 17:10:03,154][20347] Updated weights for policy 0, policy_version 1663154 (0.0010) [2023-12-27 17:10:03,218][20347] Updated weights for policy 0, policy_version 1663164 (0.0010) [2023-12-27 17:10:03,269][20347] Updated weights for policy 0, policy_version 1663174 (0.0010) [2023-12-27 17:10:03,402][20381] Updated weights for policy 1, policy_version 1661494 (0.0009) [2023-12-27 17:10:03,453][20381] Updated weights for policy 1, policy_version 1661504 (0.0007) [2023-12-27 17:10:03,503][20381] Updated weights for policy 1, policy_version 1661514 (0.0007) [2023-12-27 17:10:03,893][20347] Updated weights for policy 0, policy_version 1663184 (0.0010) [2023-12-27 17:10:03,955][20347] Updated weights for policy 0, policy_version 1663194 (0.0010) [2023-12-27 17:10:04,014][20347] Updated weights for policy 0, policy_version 1663204 (0.0010) [2023-12-27 17:10:04,218][20381] Updated weights for policy 1, policy_version 1661524 (0.0008) [2023-12-27 17:10:04,273][20381] Updated weights for policy 1, policy_version 1661534 (0.0007) [2023-12-27 17:10:04,328][20381] Updated weights for policy 1, policy_version 1661544 (0.0008) [2023-12-27 17:10:04,669][20347] Updated weights for policy 0, policy_version 1663214 (0.0008) [2023-12-27 17:10:04,712][20347] Updated weights for policy 0, policy_version 1663224 (0.0007) [2023-12-27 17:10:04,760][20347] Updated weights for policy 0, policy_version 1663234 (0.0008) [2023-12-27 17:10:05,027][20381] Updated weights for policy 1, policy_version 1661554 (0.0008) [2023-12-27 17:10:05,077][20381] Updated weights for policy 1, policy_version 1661564 (0.0008) [2023-12-27 17:10:05,126][20381] Updated weights for policy 1, policy_version 1661574 (0.0008) [2023-12-27 17:10:05,171][20381] Updated weights for policy 1, policy_version 1661584 (0.0007) [2023-12-27 17:10:05,464][20347] Updated weights for policy 0, policy_version 1663244 (0.0008) [2023-12-27 17:10:05,515][20347] Updated weights for policy 0, policy_version 1663254 (0.0007) [2023-12-27 17:10:05,560][20347] Updated weights for policy 0, policy_version 1663264 (0.0007) [2023-12-27 17:10:05,836][20381] Updated weights for policy 1, policy_version 1661594 (0.0005) [2023-12-27 17:10:05,887][20381] Updated weights for policy 1, policy_version 1661604 (0.0005) [2023-12-27 17:10:05,935][20381] Updated weights for policy 1, policy_version 1661614 (0.0005) [2023-12-27 17:10:06,307][20347] Updated weights for policy 0, policy_version 1663274 (0.0007) [2023-12-27 17:10:06,363][20347] Updated weights for policy 0, policy_version 1663284 (0.0007) [2023-12-27 17:10:06,423][20347] Updated weights for policy 0, policy_version 1663294 (0.0007) [2023-12-27 17:10:06,479][20347] Updated weights for policy 0, policy_version 1663304 (0.0008) [2023-12-27 17:10:06,565][20381] Updated weights for policy 1, policy_version 1661624 (0.0007) [2023-12-27 17:10:06,612][20381] Updated weights for policy 1, policy_version 1661634 (0.0008) [2023-12-27 17:10:06,659][20381] Updated weights for policy 1, policy_version 1661644 (0.0010) [2023-12-27 17:10:07,121][20347] Updated weights for policy 0, policy_version 1663314 (0.0007) [2023-12-27 17:10:07,170][20347] Updated weights for policy 0, policy_version 1663324 (0.0008) [2023-12-27 17:10:07,225][20347] Updated weights for policy 0, policy_version 1663334 (0.0007) [2023-12-27 17:10:07,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 851320832. Throughput: 0: 10431.5, 1: 10183.5. Samples: 851289124. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:07,364][19170] Avg episode reward: [(0, '941.300'), (1, '863.360')] [2023-12-27 17:10:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001663336_425893888.pth... [2023-12-27 17:10:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001662120_425582592.pth [2023-12-27 17:10:07,409][20381] Updated weights for policy 1, policy_version 1661654 (0.0009) [2023-12-27 17:10:07,463][20381] Updated weights for policy 1, policy_version 1661664 (0.0007) [2023-12-27 17:10:07,523][20381] Updated weights for policy 1, policy_version 1661674 (0.0008) [2023-12-27 17:10:07,553][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001661680_425435136.pth... [2023-12-27 17:10:07,557][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001660496_425132032.pth [2023-12-27 17:10:07,920][20347] Updated weights for policy 0, policy_version 1663344 (0.0007) [2023-12-27 17:10:07,969][20347] Updated weights for policy 0, policy_version 1663354 (0.0007) [2023-12-27 17:10:08,015][20347] Updated weights for policy 0, policy_version 1663364 (0.0008) [2023-12-27 17:10:08,151][20381] Updated weights for policy 1, policy_version 1661684 (0.0007) [2023-12-27 17:10:08,199][20381] Updated weights for policy 1, policy_version 1661694 (0.0005) [2023-12-27 17:10:08,251][20381] Updated weights for policy 1, policy_version 1661704 (0.0006) [2023-12-27 17:10:08,752][20347] Updated weights for policy 0, policy_version 1663374 (0.0009) [2023-12-27 17:10:08,812][20347] Updated weights for policy 0, policy_version 1663384 (0.0008) [2023-12-27 17:10:08,846][20381] Updated weights for policy 1, policy_version 1661714 (0.0006) [2023-12-27 17:10:08,873][20347] Updated weights for policy 0, policy_version 1663394 (0.0008) [2023-12-27 17:10:08,902][20381] Updated weights for policy 1, policy_version 1661724 (0.0006) [2023-12-27 17:10:08,958][20381] Updated weights for policy 1, policy_version 1661734 (0.0008) [2023-12-27 17:10:09,010][20381] Updated weights for policy 1, policy_version 1661744 (0.0007) [2023-12-27 17:10:09,562][20347] Updated weights for policy 0, policy_version 1663404 (0.0007) [2023-12-27 17:10:09,608][20347] Updated weights for policy 0, policy_version 1663414 (0.0007) [2023-12-27 17:10:09,658][20347] Updated weights for policy 0, policy_version 1663424 (0.0008) [2023-12-27 17:10:09,798][20381] Updated weights for policy 1, policy_version 1661754 (0.0008) [2023-12-27 17:10:09,860][20381] Updated weights for policy 1, policy_version 1661764 (0.0009) [2023-12-27 17:10:09,918][20381] Updated weights for policy 1, policy_version 1661774 (0.0008) [2023-12-27 17:10:10,401][20347] Updated weights for policy 0, policy_version 1663434 (0.0007) [2023-12-27 17:10:10,462][20347] Updated weights for policy 0, policy_version 1663444 (0.0008) [2023-12-27 17:10:10,529][20347] Updated weights for policy 0, policy_version 1663454 (0.0008) [2023-12-27 17:10:10,591][20347] Updated weights for policy 0, policy_version 1663464 (0.0008) [2023-12-27 17:10:10,657][20381] Updated weights for policy 1, policy_version 1661784 (0.0007) [2023-12-27 17:10:10,717][20381] Updated weights for policy 1, policy_version 1661794 (0.0008) [2023-12-27 17:10:10,772][20381] Updated weights for policy 1, policy_version 1661804 (0.0007) [2023-12-27 17:10:11,297][20347] Updated weights for policy 0, policy_version 1663474 (0.0007) [2023-12-27 17:10:11,354][20347] Updated weights for policy 0, policy_version 1663484 (0.0008) [2023-12-27 17:10:11,422][20347] Updated weights for policy 0, policy_version 1663494 (0.0008) [2023-12-27 17:10:11,570][20381] Updated weights for policy 1, policy_version 1661814 (0.0008) [2023-12-27 17:10:11,637][20381] Updated weights for policy 1, policy_version 1661824 (0.0008) [2023-12-27 17:10:11,644][20292] KL-divergence is very high: 102.2484 [2023-12-27 17:10:11,697][20381] Updated weights for policy 1, policy_version 1661834 (0.0007) [2023-12-27 17:10:12,171][20347] Updated weights for policy 0, policy_version 1663504 (0.0007) [2023-12-27 17:10:12,217][20347] Updated weights for policy 0, policy_version 1663514 (0.0007) [2023-12-27 17:10:12,237][20291] KL-divergence is very high: 215.5225 [2023-12-27 17:10:12,255][20291] KL-divergence is very high: 272.9849 [2023-12-27 17:10:12,271][20347] Updated weights for policy 0, policy_version 1663524 (0.0007) [2023-12-27 17:10:12,285][20291] KL-divergence is very high: 246.5522 [2023-12-27 17:10:12,354][20381] Updated weights for policy 1, policy_version 1661844 (0.0007) [2023-12-27 17:10:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 851419136. Throughput: 0: 10368.1, 1: 10158.5. Samples: 851409960. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:12,363][19170] Avg episode reward: [(0, '951.110'), (1, '863.430')] [2023-12-27 17:10:12,422][20381] Updated weights for policy 1, policy_version 1661854 (0.0007) [2023-12-27 17:10:12,471][20381] Updated weights for policy 1, policy_version 1661864 (0.0007) [2023-12-27 17:10:12,952][20347] Updated weights for policy 0, policy_version 1663534 (0.0007) [2023-12-27 17:10:13,006][20347] Updated weights for policy 0, policy_version 1663544 (0.0006) [2023-12-27 17:10:13,063][20347] Updated weights for policy 0, policy_version 1663554 (0.0006) [2023-12-27 17:10:13,191][20381] Updated weights for policy 1, policy_version 1661874 (0.0006) [2023-12-27 17:10:13,247][20381] Updated weights for policy 1, policy_version 1661884 (0.0007) [2023-12-27 17:10:13,302][20381] Updated weights for policy 1, policy_version 1661894 (0.0005) [2023-12-27 17:10:13,346][20381] Updated weights for policy 1, policy_version 1661904 (0.0005) [2023-12-27 17:10:13,712][20347] Updated weights for policy 0, policy_version 1663564 (0.0010) [2023-12-27 17:10:13,765][20347] Updated weights for policy 0, policy_version 1663574 (0.0010) [2023-12-27 17:10:13,819][20347] Updated weights for policy 0, policy_version 1663584 (0.0010) [2023-12-27 17:10:14,017][20381] Updated weights for policy 1, policy_version 1661914 (0.0005) [2023-12-27 17:10:14,074][20381] Updated weights for policy 1, policy_version 1661924 (0.0009) [2023-12-27 17:10:14,129][20381] Updated weights for policy 1, policy_version 1661934 (0.0010) [2023-12-27 17:10:14,500][20347] Updated weights for policy 0, policy_version 1663594 (0.0010) [2023-12-27 17:10:14,559][20347] Updated weights for policy 0, policy_version 1663604 (0.0010) [2023-12-27 17:10:14,623][20347] Updated weights for policy 0, policy_version 1663614 (0.0010) [2023-12-27 17:10:14,683][20347] Updated weights for policy 0, policy_version 1663624 (0.0010) [2023-12-27 17:10:14,791][20381] Updated weights for policy 1, policy_version 1661944 (0.0010) [2023-12-27 17:10:14,855][20381] Updated weights for policy 1, policy_version 1661954 (0.0008) [2023-12-27 17:10:14,915][20381] Updated weights for policy 1, policy_version 1661964 (0.0008) [2023-12-27 17:10:15,423][20347] Updated weights for policy 0, policy_version 1663634 (0.0007) [2023-12-27 17:10:15,485][20347] Updated weights for policy 0, policy_version 1663644 (0.0007) [2023-12-27 17:10:15,546][20347] Updated weights for policy 0, policy_version 1663654 (0.0007) [2023-12-27 17:10:15,550][20381] Updated weights for policy 1, policy_version 1661974 (0.0011) [2023-12-27 17:10:15,600][20381] Updated weights for policy 1, policy_version 1661984 (0.0010) [2023-12-27 17:10:15,649][20381] Updated weights for policy 1, policy_version 1661994 (0.0010) [2023-12-27 17:10:16,207][20381] Updated weights for policy 1, policy_version 1662004 (0.0008) [2023-12-27 17:10:16,266][20381] Updated weights for policy 1, policy_version 1662014 (0.0011) [2023-12-27 17:10:16,276][20347] Updated weights for policy 0, policy_version 1663664 (0.0008) [2023-12-27 17:10:16,322][20381] Updated weights for policy 1, policy_version 1662024 (0.0011) [2023-12-27 17:10:16,328][20347] Updated weights for policy 0, policy_version 1663674 (0.0005) [2023-12-27 17:10:16,376][20347] Updated weights for policy 0, policy_version 1663684 (0.0006) [2023-12-27 17:10:16,979][20347] Updated weights for policy 0, policy_version 1663694 (0.0007) [2023-12-27 17:10:16,997][20381] Updated weights for policy 1, policy_version 1662034 (0.0010) [2023-12-27 17:10:17,038][20347] Updated weights for policy 0, policy_version 1663704 (0.0009) [2023-12-27 17:10:17,042][20292] KL-divergence is very high: 100.4398 [2023-12-27 17:10:17,048][20292] KL-divergence is very high: 126.9535 [2023-12-27 17:10:17,055][20381] Updated weights for policy 1, policy_version 1662044 (0.0011) [2023-12-27 17:10:17,089][20292] KL-divergence is very high: 102.0007 [2023-12-27 17:10:17,093][20347] Updated weights for policy 0, policy_version 1663714 (0.0006) [2023-12-27 17:10:17,104][20381] Updated weights for policy 1, policy_version 1662054 (0.0010) [2023-12-27 17:10:17,159][20381] Updated weights for policy 1, policy_version 1662064 (0.0011) [2023-12-27 17:10:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 851525632. Throughput: 0: 10266.0, 1: 10170.0. Samples: 851532060. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:17,363][19170] Avg episode reward: [(0, '960.870'), (1, '795.020')] [2023-12-27 17:10:17,792][20347] Updated weights for policy 0, policy_version 1663724 (0.0008) [2023-12-27 17:10:17,847][20347] Updated weights for policy 0, policy_version 1663734 (0.0006) [2023-12-27 17:10:17,859][20381] Updated weights for policy 1, policy_version 1662074 (0.0008) [2023-12-27 17:10:17,902][20347] Updated weights for policy 0, policy_version 1663744 (0.0005) [2023-12-27 17:10:17,910][20381] Updated weights for policy 1, policy_version 1662084 (0.0007) [2023-12-27 17:10:17,965][20381] Updated weights for policy 1, policy_version 1662094 (0.0006) [2023-12-27 17:10:18,559][20347] Updated weights for policy 0, policy_version 1663754 (0.0005) [2023-12-27 17:10:18,603][20381] Updated weights for policy 1, policy_version 1662104 (0.0007) [2023-12-27 17:10:18,610][20347] Updated weights for policy 0, policy_version 1663764 (0.0005) [2023-12-27 17:10:18,662][20381] Updated weights for policy 1, policy_version 1662114 (0.0009) [2023-12-27 17:10:18,664][20347] Updated weights for policy 0, policy_version 1663774 (0.0005) [2023-12-27 17:10:18,716][20381] Updated weights for policy 1, policy_version 1662124 (0.0011) [2023-12-27 17:10:18,719][20347] Updated weights for policy 0, policy_version 1663784 (0.0008) [2023-12-27 17:10:19,411][20347] Updated weights for policy 0, policy_version 1663794 (0.0007) [2023-12-27 17:10:19,433][20381] Updated weights for policy 1, policy_version 1662134 (0.0009) [2023-12-27 17:10:19,461][20347] Updated weights for policy 0, policy_version 1663804 (0.0008) [2023-12-27 17:10:19,495][20381] Updated weights for policy 1, policy_version 1662144 (0.0010) [2023-12-27 17:10:19,525][20347] Updated weights for policy 0, policy_version 1663814 (0.0007) [2023-12-27 17:10:19,553][20381] Updated weights for policy 1, policy_version 1662154 (0.0007) [2023-12-27 17:10:20,213][20347] Updated weights for policy 0, policy_version 1663824 (0.0008) [2023-12-27 17:10:20,263][20347] Updated weights for policy 0, policy_version 1663834 (0.0008) [2023-12-27 17:10:20,301][20381] Updated weights for policy 1, policy_version 1662164 (0.0009) [2023-12-27 17:10:20,322][20347] Updated weights for policy 0, policy_version 1663844 (0.0009) [2023-12-27 17:10:20,368][20381] Updated weights for policy 1, policy_version 1662174 (0.0006) [2023-12-27 17:10:20,433][20381] Updated weights for policy 1, policy_version 1662184 (0.0006) [2023-12-27 17:10:21,039][20347] Updated weights for policy 0, policy_version 1663854 (0.0007) [2023-12-27 17:10:21,104][20347] Updated weights for policy 0, policy_version 1663864 (0.0007) [2023-12-27 17:10:21,131][20381] Updated weights for policy 1, policy_version 1662194 (0.0006) [2023-12-27 17:10:21,167][20347] Updated weights for policy 0, policy_version 1663874 (0.0008) [2023-12-27 17:10:21,199][20381] Updated weights for policy 1, policy_version 1662204 (0.0008) [2023-12-27 17:10:21,261][20381] Updated weights for policy 1, policy_version 1662214 (0.0007) [2023-12-27 17:10:21,331][20381] Updated weights for policy 1, policy_version 1662224 (0.0007) [2023-12-27 17:10:21,888][20347] Updated weights for policy 0, policy_version 1663884 (0.0007) [2023-12-27 17:10:21,941][20347] Updated weights for policy 0, policy_version 1663894 (0.0007) [2023-12-27 17:10:21,999][20347] Updated weights for policy 0, policy_version 1663904 (0.0007) [2023-12-27 17:10:22,097][20381] Updated weights for policy 1, policy_version 1662234 (0.0008) [2023-12-27 17:10:22,160][20381] Updated weights for policy 1, policy_version 1662244 (0.0008) [2023-12-27 17:10:22,217][20381] Updated weights for policy 1, policy_version 1662254 (0.0008) [2023-12-27 17:10:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 851623936. Throughput: 0: 10235.4, 1: 10136.0. Samples: 851592076. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:22,363][19170] Avg episode reward: [(0, '970.600'), (1, '833.830')] [2023-12-27 17:10:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001662256_425582592.pth... [2023-12-27 17:10:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001663912_426041344.pth... [2023-12-27 17:10:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001661072_425279488.pth [2023-12-27 17:10:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001662728_425738240.pth [2023-12-27 17:10:22,714][20347] Updated weights for policy 0, policy_version 1663914 (0.0007) [2023-12-27 17:10:22,773][20347] Updated weights for policy 0, policy_version 1663924 (0.0007) [2023-12-27 17:10:22,830][20347] Updated weights for policy 0, policy_version 1663934 (0.0007) [2023-12-27 17:10:22,888][20347] Updated weights for policy 0, policy_version 1663944 (0.0007) [2023-12-27 17:10:22,902][20381] Updated weights for policy 1, policy_version 1662264 (0.0008) [2023-12-27 17:10:22,954][20381] Updated weights for policy 1, policy_version 1662274 (0.0007) [2023-12-27 17:10:23,000][20381] Updated weights for policy 1, policy_version 1662284 (0.0008) [2023-12-27 17:10:23,508][20347] Updated weights for policy 0, policy_version 1663954 (0.0005) [2023-12-27 17:10:23,562][20347] Updated weights for policy 0, policy_version 1663964 (0.0010) [2023-12-27 17:10:23,621][20347] Updated weights for policy 0, policy_version 1663974 (0.0008) [2023-12-27 17:10:23,701][20381] Updated weights for policy 1, policy_version 1662294 (0.0010) [2023-12-27 17:10:23,753][20381] Updated weights for policy 1, policy_version 1662304 (0.0010) [2023-12-27 17:10:23,801][20381] Updated weights for policy 1, policy_version 1662314 (0.0010) [2023-12-27 17:10:24,259][20347] Updated weights for policy 0, policy_version 1663984 (0.0010) [2023-12-27 17:10:24,309][20347] Updated weights for policy 0, policy_version 1663994 (0.0010) [2023-12-27 17:10:24,361][20347] Updated weights for policy 0, policy_version 1664004 (0.0010) [2023-12-27 17:10:24,423][20381] Updated weights for policy 1, policy_version 1662324 (0.0010) [2023-12-27 17:10:24,490][20381] Updated weights for policy 1, policy_version 1662334 (0.0010) [2023-12-27 17:10:24,551][20381] Updated weights for policy 1, policy_version 1662344 (0.0011) [2023-12-27 17:10:25,029][20347] Updated weights for policy 0, policy_version 1664014 (0.0008) [2023-12-27 17:10:25,085][20347] Updated weights for policy 0, policy_version 1664024 (0.0007) [2023-12-27 17:10:25,135][20347] Updated weights for policy 0, policy_version 1664034 (0.0005) [2023-12-27 17:10:25,144][20381] Updated weights for policy 1, policy_version 1662354 (0.0009) [2023-12-27 17:10:25,194][20381] Updated weights for policy 1, policy_version 1662364 (0.0008) [2023-12-27 17:10:25,244][20381] Updated weights for policy 1, policy_version 1662374 (0.0011) [2023-12-27 17:10:25,308][20381] Updated weights for policy 1, policy_version 1662384 (0.0010) [2023-12-27 17:10:25,764][20347] Updated weights for policy 0, policy_version 1664044 (0.0006) [2023-12-27 17:10:25,815][20347] Updated weights for policy 0, policy_version 1664054 (0.0007) [2023-12-27 17:10:25,847][20381] Updated weights for policy 1, policy_version 1662394 (0.0008) [2023-12-27 17:10:25,871][20347] Updated weights for policy 0, policy_version 1664064 (0.0005) [2023-12-27 17:10:25,905][20381] Updated weights for policy 1, policy_version 1662404 (0.0011) [2023-12-27 17:10:25,961][20381] Updated weights for policy 1, policy_version 1662414 (0.0011) [2023-12-27 17:10:26,564][20347] Updated weights for policy 0, policy_version 1664074 (0.0006) [2023-12-27 17:10:26,625][20347] Updated weights for policy 0, policy_version 1664084 (0.0005) [2023-12-27 17:10:26,630][20381] Updated weights for policy 1, policy_version 1662424 (0.0011) [2023-12-27 17:10:26,678][20347] Updated weights for policy 0, policy_version 1664094 (0.0005) [2023-12-27 17:10:26,684][20381] Updated weights for policy 1, policy_version 1662434 (0.0010) [2023-12-27 17:10:26,725][20347] Updated weights for policy 0, policy_version 1664104 (0.0005) [2023-12-27 17:10:26,738][20381] Updated weights for policy 1, policy_version 1662444 (0.0010) [2023-12-27 17:10:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 851722240. Throughput: 0: 10181.5, 1: 10159.7. Samples: 851716648. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) [2023-12-27 17:10:27,363][19170] Avg episode reward: [(0, '951.050'), (1, '853.300')] [2023-12-27 17:10:27,383][20347] Updated weights for policy 0, policy_version 1664114 (0.0008) [2023-12-27 17:10:27,386][20381] Updated weights for policy 1, policy_version 1662454 (0.0010) [2023-12-27 17:10:27,434][20347] Updated weights for policy 0, policy_version 1664124 (0.0007) [2023-12-27 17:10:27,440][20381] Updated weights for policy 1, policy_version 1662464 (0.0010) [2023-12-27 17:10:27,485][20381] Updated weights for policy 1, policy_version 1662474 (0.0010) [2023-12-27 17:10:27,486][20347] Updated weights for policy 0, policy_version 1664134 (0.0007) [2023-12-27 17:10:28,151][20347] Updated weights for policy 0, policy_version 1664144 (0.0010) [2023-12-27 17:10:28,183][20381] Updated weights for policy 1, policy_version 1662484 (0.0010) [2023-12-27 17:10:28,194][20347] Updated weights for policy 0, policy_version 1664154 (0.0005) [2023-12-27 17:10:28,231][20381] Updated weights for policy 1, policy_version 1662494 (0.0010) [2023-12-27 17:10:28,245][20347] Updated weights for policy 0, policy_version 1664164 (0.0005) [2023-12-27 17:10:28,279][20381] Updated weights for policy 1, policy_version 1662504 (0.0010) [2023-12-27 17:10:28,916][20347] Updated weights for policy 0, policy_version 1664174 (0.0007) [2023-12-27 17:10:28,972][20347] Updated weights for policy 0, policy_version 1664184 (0.0007) [2023-12-27 17:10:28,980][20381] Updated weights for policy 1, policy_version 1662514 (0.0010) [2023-12-27 17:10:29,026][20347] Updated weights for policy 0, policy_version 1664194 (0.0005) [2023-12-27 17:10:29,038][20381] Updated weights for policy 1, policy_version 1662524 (0.0010) [2023-12-27 17:10:29,089][20381] Updated weights for policy 1, policy_version 1662534 (0.0010) [2023-12-27 17:10:29,161][20381] Updated weights for policy 1, policy_version 1662544 (0.0010) [2023-12-27 17:10:29,703][20347] Updated weights for policy 0, policy_version 1664204 (0.0006) [2023-12-27 17:10:29,768][20347] Updated weights for policy 0, policy_version 1664214 (0.0007) [2023-12-27 17:10:29,820][20347] Updated weights for policy 0, policy_version 1664224 (0.0006) [2023-12-27 17:10:29,843][20381] Updated weights for policy 1, policy_version 1662554 (0.0009) [2023-12-27 17:10:29,899][20381] Updated weights for policy 1, policy_version 1662564 (0.0007) [2023-12-27 17:10:29,963][20381] Updated weights for policy 1, policy_version 1662574 (0.0009) [2023-12-27 17:10:30,490][20347] Updated weights for policy 0, policy_version 1664234 (0.0008) [2023-12-27 17:10:30,549][20347] Updated weights for policy 0, policy_version 1664244 (0.0007) [2023-12-27 17:10:30,603][20347] Updated weights for policy 0, policy_version 1664254 (0.0008) [2023-12-27 17:10:30,658][20381] Updated weights for policy 1, policy_version 1662584 (0.0005) [2023-12-27 17:10:30,665][20347] Updated weights for policy 0, policy_version 1664264 (0.0007) [2023-12-27 17:10:30,711][20381] Updated weights for policy 1, policy_version 1662594 (0.0005) [2023-12-27 17:10:30,765][20381] Updated weights for policy 1, policy_version 1662604 (0.0007) [2023-12-27 17:10:31,403][20347] Updated weights for policy 0, policy_version 1664274 (0.0008) [2023-12-27 17:10:31,435][20381] Updated weights for policy 1, policy_version 1662614 (0.0006) [2023-12-27 17:10:31,459][20347] Updated weights for policy 0, policy_version 1664284 (0.0007) [2023-12-27 17:10:31,494][20381] Updated weights for policy 1, policy_version 1662624 (0.0008) [2023-12-27 17:10:31,514][20347] Updated weights for policy 0, policy_version 1664294 (0.0006) [2023-12-27 17:10:31,553][20381] Updated weights for policy 1, policy_version 1662634 (0.0006) [2023-12-27 17:10:32,194][20347] Updated weights for policy 0, policy_version 1664304 (0.0006) [2023-12-27 17:10:32,257][20347] Updated weights for policy 0, policy_version 1664314 (0.0008) [2023-12-27 17:10:32,281][20381] Updated weights for policy 1, policy_version 1662644 (0.0006) [2023-12-27 17:10:32,317][20347] Updated weights for policy 0, policy_version 1664324 (0.0006) [2023-12-27 17:10:32,335][20381] Updated weights for policy 1, policy_version 1662654 (0.0006) [2023-12-27 17:10:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 851828736. Throughput: 0: 10199.2, 1: 10173.3. Samples: 851838848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:32,363][19170] Avg episode reward: [(0, '941.270'), (1, '863.110')] [2023-12-27 17:10:32,399][20381] Updated weights for policy 1, policy_version 1662664 (0.0009) [2023-12-27 17:10:32,953][20347] Updated weights for policy 0, policy_version 1664334 (0.0009) [2023-12-27 17:10:33,002][20347] Updated weights for policy 0, policy_version 1664344 (0.0010) [2023-12-27 17:10:33,042][20381] Updated weights for policy 1, policy_version 1662674 (0.0009) [2023-12-27 17:10:33,054][20347] Updated weights for policy 0, policy_version 1664354 (0.0007) [2023-12-27 17:10:33,102][20381] Updated weights for policy 1, policy_version 1662684 (0.0010) [2023-12-27 17:10:33,156][20381] Updated weights for policy 1, policy_version 1662694 (0.0010) [2023-12-27 17:10:33,211][20381] Updated weights for policy 1, policy_version 1662704 (0.0010) [2023-12-27 17:10:33,607][20347] Updated weights for policy 0, policy_version 1664364 (0.0005) [2023-12-27 17:10:33,661][20347] Updated weights for policy 0, policy_version 1664374 (0.0005) [2023-12-27 17:10:33,715][20347] Updated weights for policy 0, policy_version 1664384 (0.0007) [2023-12-27 17:10:33,894][20381] Updated weights for policy 1, policy_version 1662714 (0.0006) [2023-12-27 17:10:33,945][20381] Updated weights for policy 1, policy_version 1662724 (0.0005) [2023-12-27 17:10:33,990][20381] Updated weights for policy 1, policy_version 1662734 (0.0005) [2023-12-27 17:10:34,310][20347] Updated weights for policy 0, policy_version 1664394 (0.0008) [2023-12-27 17:10:34,369][20347] Updated weights for policy 0, policy_version 1664404 (0.0007) [2023-12-27 17:10:34,426][20347] Updated weights for policy 0, policy_version 1664414 (0.0008) [2023-12-27 17:10:34,488][20347] Updated weights for policy 0, policy_version 1664424 (0.0007) [2023-12-27 17:10:34,663][20381] Updated weights for policy 1, policy_version 1662744 (0.0010) [2023-12-27 17:10:34,719][20381] Updated weights for policy 1, policy_version 1662754 (0.0010) [2023-12-27 17:10:34,774][20381] Updated weights for policy 1, policy_version 1662764 (0.0010) [2023-12-27 17:10:35,123][20347] Updated weights for policy 0, policy_version 1664434 (0.0005) [2023-12-27 17:10:35,173][20347] Updated weights for policy 0, policy_version 1664444 (0.0005) [2023-12-27 17:10:35,224][20347] Updated weights for policy 0, policy_version 1664454 (0.0005) [2023-12-27 17:10:35,429][20381] Updated weights for policy 1, policy_version 1662774 (0.0011) [2023-12-27 17:10:35,488][20381] Updated weights for policy 1, policy_version 1662784 (0.0009) [2023-12-27 17:10:35,540][20381] Updated weights for policy 1, policy_version 1662794 (0.0010) [2023-12-27 17:10:35,831][20347] Updated weights for policy 0, policy_version 1664464 (0.0007) [2023-12-27 17:10:35,901][20347] Updated weights for policy 0, policy_version 1664474 (0.0008) [2023-12-27 17:10:35,960][20347] Updated weights for policy 0, policy_version 1664484 (0.0008) [2023-12-27 17:10:36,122][20381] Updated weights for policy 1, policy_version 1662804 (0.0009) [2023-12-27 17:10:36,178][20381] Updated weights for policy 1, policy_version 1662814 (0.0008) [2023-12-27 17:10:36,232][20381] Updated weights for policy 1, policy_version 1662824 (0.0008) [2023-12-27 17:10:36,654][20347] Updated weights for policy 0, policy_version 1664494 (0.0007) [2023-12-27 17:10:36,718][20347] Updated weights for policy 0, policy_version 1664504 (0.0008) [2023-12-27 17:10:36,780][20347] Updated weights for policy 0, policy_version 1664514 (0.0007) [2023-12-27 17:10:36,894][20381] Updated weights for policy 1, policy_version 1662834 (0.0007) [2023-12-27 17:10:36,951][20381] Updated weights for policy 1, policy_version 1662844 (0.0007) [2023-12-27 17:10:37,008][20381] Updated weights for policy 1, policy_version 1662854 (0.0008) [2023-12-27 17:10:37,068][20381] Updated weights for policy 1, policy_version 1662864 (0.0008) [2023-12-27 17:10:37,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20480.0, 300 sec: 20604.9). Total num frames: 851935232. Throughput: 0: 10203.5, 1: 10184.1. Samples: 851903060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:37,364][19170] Avg episode reward: [(0, '980.420'), (1, '853.540')] [2023-12-27 17:10:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001662864_425738240.pth... [2023-12-27 17:10:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001664520_426196992.pth... [2023-12-27 17:10:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001661680_425435136.pth [2023-12-27 17:10:37,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001663336_425893888.pth [2023-12-27 17:10:37,481][20347] Updated weights for policy 0, policy_version 1664524 (0.0007) [2023-12-27 17:10:37,543][20347] Updated weights for policy 0, policy_version 1664534 (0.0008) [2023-12-27 17:10:37,603][20347] Updated weights for policy 0, policy_version 1664544 (0.0006) [2023-12-27 17:10:37,744][20381] Updated weights for policy 1, policy_version 1662874 (0.0006) [2023-12-27 17:10:37,790][20381] Updated weights for policy 1, policy_version 1662884 (0.0005) [2023-12-27 17:10:37,826][20292] KL-divergence is very high: 175.2792 [2023-12-27 17:10:37,835][20381] Updated weights for policy 1, policy_version 1662894 (0.0005) [2023-12-27 17:10:38,225][20347] Updated weights for policy 0, policy_version 1664554 (0.0006) [2023-12-27 17:10:38,280][20347] Updated weights for policy 0, policy_version 1664564 (0.0010) [2023-12-27 17:10:38,341][20292] KL-divergence is very high: 152.2162 [2023-12-27 17:10:38,342][20347] Updated weights for policy 0, policy_version 1664574 (0.0009) [2023-12-27 17:10:38,385][20381] Updated weights for policy 1, policy_version 1662904 (0.0007) [2023-12-27 17:10:38,408][20347] Updated weights for policy 0, policy_version 1664584 (0.0007) [2023-12-27 17:10:38,441][20381] Updated weights for policy 1, policy_version 1662914 (0.0006) [2023-12-27 17:10:38,484][20292] KL-divergence is very high: 111.1939 [2023-12-27 17:10:38,497][20381] Updated weights for policy 1, policy_version 1662924 (0.0008) [2023-12-27 17:10:39,046][20347] Updated weights for policy 0, policy_version 1664594 (0.0005) [2023-12-27 17:10:39,098][20347] Updated weights for policy 0, policy_version 1664604 (0.0005) [2023-12-27 17:10:39,157][20347] Updated weights for policy 0, policy_version 1664614 (0.0005) [2023-12-27 17:10:39,195][20381] Updated weights for policy 1, policy_version 1662934 (0.0007) [2023-12-27 17:10:39,258][20381] Updated weights for policy 1, policy_version 1662944 (0.0008) [2023-12-27 17:10:39,314][20381] Updated weights for policy 1, policy_version 1662954 (0.0008) [2023-12-27 17:10:39,923][20347] Updated weights for policy 0, policy_version 1664624 (0.0009) [2023-12-27 17:10:39,982][20347] Updated weights for policy 0, policy_version 1664634 (0.0009) [2023-12-27 17:10:40,039][20347] Updated weights for policy 0, policy_version 1664644 (0.0011) [2023-12-27 17:10:40,052][20381] Updated weights for policy 1, policy_version 1662964 (0.0008) [2023-12-27 17:10:40,104][20381] Updated weights for policy 1, policy_version 1662974 (0.0007) [2023-12-27 17:10:40,148][20381] Updated weights for policy 1, policy_version 1662984 (0.0007) [2023-12-27 17:10:40,732][20347] Updated weights for policy 0, policy_version 1664654 (0.0010) [2023-12-27 17:10:40,784][20347] Updated weights for policy 0, policy_version 1664664 (0.0010) [2023-12-27 17:10:40,831][20347] Updated weights for policy 0, policy_version 1664674 (0.0010) [2023-12-27 17:10:40,880][20381] Updated weights for policy 1, policy_version 1662994 (0.0007) [2023-12-27 17:10:40,934][20381] Updated weights for policy 1, policy_version 1663004 (0.0006) [2023-12-27 17:10:40,998][20381] Updated weights for policy 1, policy_version 1663014 (0.0006) [2023-12-27 17:10:41,066][20381] Updated weights for policy 1, policy_version 1663024 (0.0008) [2023-12-27 17:10:41,611][20347] Updated weights for policy 0, policy_version 1664684 (0.0010) [2023-12-27 17:10:41,678][20347] Updated weights for policy 0, policy_version 1664694 (0.0011) [2023-12-27 17:10:41,753][20347] Updated weights for policy 0, policy_version 1664704 (0.0010) [2023-12-27 17:10:41,800][20381] Updated weights for policy 1, policy_version 1663034 (0.0008) [2023-12-27 17:10:41,856][20381] Updated weights for policy 1, policy_version 1663044 (0.0007) [2023-12-27 17:10:41,913][20381] Updated weights for policy 1, policy_version 1663054 (0.0007) [2023-12-27 17:10:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 852033536. Throughput: 0: 10222.6, 1: 10257.0. Samples: 852027456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:42,363][19170] Avg episode reward: [(0, '980.470'), (1, '843.760')] [2023-12-27 17:10:42,414][20347] Updated weights for policy 0, policy_version 1664714 (0.0009) [2023-12-27 17:10:42,477][20347] Updated weights for policy 0, policy_version 1664724 (0.0008) [2023-12-27 17:10:42,540][20347] Updated weights for policy 0, policy_version 1664734 (0.0007) [2023-12-27 17:10:42,600][20347] Updated weights for policy 0, policy_version 1664744 (0.0007) [2023-12-27 17:10:42,702][20381] Updated weights for policy 1, policy_version 1663064 (0.0008) [2023-12-27 17:10:42,757][20381] Updated weights for policy 1, policy_version 1663074 (0.0008) [2023-12-27 17:10:42,817][20381] Updated weights for policy 1, policy_version 1663084 (0.0008) [2023-12-27 17:10:43,270][20347] Updated weights for policy 0, policy_version 1664754 (0.0007) [2023-12-27 17:10:43,313][20347] Updated weights for policy 0, policy_version 1664764 (0.0008) [2023-12-27 17:10:43,364][20347] Updated weights for policy 0, policy_version 1664774 (0.0007) [2023-12-27 17:10:43,520][20381] Updated weights for policy 1, policy_version 1663094 (0.0007) [2023-12-27 17:10:43,573][20381] Updated weights for policy 1, policy_version 1663104 (0.0007) [2023-12-27 17:10:43,631][20381] Updated weights for policy 1, policy_version 1663114 (0.0008) [2023-12-27 17:10:44,009][20347] Updated weights for policy 0, policy_version 1664784 (0.0006) [2023-12-27 17:10:44,068][20347] Updated weights for policy 0, policy_version 1664794 (0.0005) [2023-12-27 17:10:44,121][20347] Updated weights for policy 0, policy_version 1664804 (0.0007) [2023-12-27 17:10:44,246][20381] Updated weights for policy 1, policy_version 1663124 (0.0006) [2023-12-27 17:10:44,296][20381] Updated weights for policy 1, policy_version 1663134 (0.0009) [2023-12-27 17:10:44,342][20381] Updated weights for policy 1, policy_version 1663144 (0.0008) [2023-12-27 17:10:44,661][20347] Updated weights for policy 0, policy_version 1664814 (0.0006) [2023-12-27 17:10:44,712][20347] Updated weights for policy 0, policy_version 1664824 (0.0006) [2023-12-27 17:10:44,756][20347] Updated weights for policy 0, policy_version 1664834 (0.0007) [2023-12-27 17:10:45,036][20381] Updated weights for policy 1, policy_version 1663154 (0.0007) [2023-12-27 17:10:45,094][20381] Updated weights for policy 1, policy_version 1663164 (0.0007) [2023-12-27 17:10:45,152][20381] Updated weights for policy 1, policy_version 1663174 (0.0007) [2023-12-27 17:10:45,207][20381] Updated weights for policy 1, policy_version 1663184 (0.0008) [2023-12-27 17:10:45,529][20347] Updated weights for policy 0, policy_version 1664844 (0.0008) [2023-12-27 17:10:45,589][20347] Updated weights for policy 0, policy_version 1664854 (0.0010) [2023-12-27 17:10:45,645][20347] Updated weights for policy 0, policy_version 1664864 (0.0010) [2023-12-27 17:10:45,920][20381] Updated weights for policy 1, policy_version 1663194 (0.0007) [2023-12-27 17:10:45,965][20381] Updated weights for policy 1, policy_version 1663204 (0.0007) [2023-12-27 17:10:46,010][20381] Updated weights for policy 1, policy_version 1663214 (0.0007) [2023-12-27 17:10:46,327][20347] Updated weights for policy 0, policy_version 1664874 (0.0010) [2023-12-27 17:10:46,390][20347] Updated weights for policy 0, policy_version 1664884 (0.0010) [2023-12-27 17:10:46,444][20347] Updated weights for policy 0, policy_version 1664894 (0.0010) [2023-12-27 17:10:46,495][20347] Updated weights for policy 0, policy_version 1664904 (0.0010) [2023-12-27 17:10:46,687][20381] Updated weights for policy 1, policy_version 1663224 (0.0009) [2023-12-27 17:10:46,740][20381] Updated weights for policy 1, policy_version 1663234 (0.0007) [2023-12-27 17:10:46,786][20381] Updated weights for policy 1, policy_version 1663244 (0.0010) [2023-12-27 17:10:47,088][20347] Updated weights for policy 0, policy_version 1664914 (0.0005) [2023-12-27 17:10:47,138][20347] Updated weights for policy 0, policy_version 1664924 (0.0006) [2023-12-27 17:10:47,196][20347] Updated weights for policy 0, policy_version 1664934 (0.0008) [2023-12-27 17:10:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 852140032. Throughput: 0: 10269.4, 1: 10236.3. Samples: 852149860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:47,363][19170] Avg episode reward: [(0, '970.660'), (1, '941.370')] [2023-12-27 17:10:47,526][20381] Updated weights for policy 1, policy_version 1663254 (0.0010) [2023-12-27 17:10:47,577][20381] Updated weights for policy 1, policy_version 1663264 (0.0010) [2023-12-27 17:10:47,625][20381] Updated weights for policy 1, policy_version 1663274 (0.0010) [2023-12-27 17:10:47,774][20347] Updated weights for policy 0, policy_version 1664944 (0.0007) [2023-12-27 17:10:47,819][20347] Updated weights for policy 0, policy_version 1664954 (0.0006) [2023-12-27 17:10:47,865][20347] Updated weights for policy 0, policy_version 1664964 (0.0007) [2023-12-27 17:10:48,227][20381] Updated weights for policy 1, policy_version 1663284 (0.0010) [2023-12-27 17:10:48,287][20381] Updated weights for policy 1, policy_version 1663294 (0.0010) [2023-12-27 17:10:48,347][20381] Updated weights for policy 1, policy_version 1663304 (0.0010) [2023-12-27 17:10:48,617][20347] Updated weights for policy 0, policy_version 1664974 (0.0006) [2023-12-27 17:10:48,674][20347] Updated weights for policy 0, policy_version 1664984 (0.0007) [2023-12-27 17:10:48,727][20347] Updated weights for policy 0, policy_version 1664994 (0.0009) [2023-12-27 17:10:48,958][20381] Updated weights for policy 1, policy_version 1663314 (0.0007) [2023-12-27 17:10:49,015][20381] Updated weights for policy 1, policy_version 1663324 (0.0005) [2023-12-27 17:10:49,074][20381] Updated weights for policy 1, policy_version 1663334 (0.0005) [2023-12-27 17:10:49,126][20381] Updated weights for policy 1, policy_version 1663344 (0.0005) [2023-12-27 17:10:49,499][20347] Updated weights for policy 0, policy_version 1665004 (0.0008) [2023-12-27 17:10:49,561][20347] Updated weights for policy 0, policy_version 1665014 (0.0010) [2023-12-27 17:10:49,616][20347] Updated weights for policy 0, policy_version 1665024 (0.0010) [2023-12-27 17:10:49,768][20381] Updated weights for policy 1, policy_version 1663354 (0.0007) [2023-12-27 17:10:49,827][20381] Updated weights for policy 1, policy_version 1663364 (0.0008) [2023-12-27 17:10:49,889][20381] Updated weights for policy 1, policy_version 1663374 (0.0007) [2023-12-27 17:10:50,342][20347] Updated weights for policy 0, policy_version 1665034 (0.0010) [2023-12-27 17:10:50,404][20347] Updated weights for policy 0, policy_version 1665044 (0.0006) [2023-12-27 17:10:50,462][20347] Updated weights for policy 0, policy_version 1665054 (0.0007) [2023-12-27 17:10:50,515][20381] Updated weights for policy 1, policy_version 1663384 (0.0006) [2023-12-27 17:10:50,516][20347] Updated weights for policy 0, policy_version 1665064 (0.0007) [2023-12-27 17:10:50,573][20381] Updated weights for policy 1, policy_version 1663394 (0.0006) [2023-12-27 17:10:50,632][20381] Updated weights for policy 1, policy_version 1663404 (0.0007) [2023-12-27 17:10:51,191][20347] Updated weights for policy 0, policy_version 1665074 (0.0008) [2023-12-27 17:10:51,240][20347] Updated weights for policy 0, policy_version 1665084 (0.0007) [2023-12-27 17:10:51,271][20381] Updated weights for policy 1, policy_version 1663414 (0.0008) [2023-12-27 17:10:51,298][20347] Updated weights for policy 0, policy_version 1665094 (0.0008) [2023-12-27 17:10:51,335][20381] Updated weights for policy 1, policy_version 1663424 (0.0007) [2023-12-27 17:10:51,405][20381] Updated weights for policy 1, policy_version 1663434 (0.0008) [2023-12-27 17:10:52,072][20381] Updated weights for policy 1, policy_version 1663444 (0.0006) [2023-12-27 17:10:52,104][20347] Updated weights for policy 0, policy_version 1665104 (0.0006) [2023-12-27 17:10:52,133][20381] Updated weights for policy 1, policy_version 1663454 (0.0007) [2023-12-27 17:10:52,160][20347] Updated weights for policy 0, policy_version 1665114 (0.0007) [2023-12-27 17:10:52,195][20381] Updated weights for policy 1, policy_version 1663464 (0.0008) [2023-12-27 17:10:52,219][20347] Updated weights for policy 0, policy_version 1665124 (0.0007) [2023-12-27 17:10:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 852246528. Throughput: 0: 10239.5, 1: 10278.0. Samples: 852212408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:52,363][19170] Avg episode reward: [(0, '951.000'), (1, '892.900')] [2023-12-27 17:10:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001663472_425893888.pth... [2023-12-27 17:10:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001665128_426352640.pth... [2023-12-27 17:10:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001662256_425582592.pth [2023-12-27 17:10:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001663912_426041344.pth [2023-12-27 17:10:52,884][20381] Updated weights for policy 1, policy_version 1663474 (0.0009) [2023-12-27 17:10:52,940][20347] Updated weights for policy 0, policy_version 1665134 (0.0006) [2023-12-27 17:10:52,945][20381] Updated weights for policy 1, policy_version 1663484 (0.0011) [2023-12-27 17:10:52,995][20347] Updated weights for policy 0, policy_version 1665144 (0.0006) [2023-12-27 17:10:53,000][20381] Updated weights for policy 1, policy_version 1663494 (0.0010) [2023-12-27 17:10:53,054][20347] Updated weights for policy 0, policy_version 1665154 (0.0006) [2023-12-27 17:10:53,059][20381] Updated weights for policy 1, policy_version 1663504 (0.0010) [2023-12-27 17:10:53,712][20347] Updated weights for policy 0, policy_version 1665164 (0.0007) [2023-12-27 17:10:53,717][20381] Updated weights for policy 1, policy_version 1663514 (0.0006) [2023-12-27 17:10:53,767][20347] Updated weights for policy 0, policy_version 1665174 (0.0007) [2023-12-27 17:10:53,773][20381] Updated weights for policy 1, policy_version 1663524 (0.0006) [2023-12-27 17:10:53,825][20381] Updated weights for policy 1, policy_version 1663534 (0.0006) [2023-12-27 17:10:53,827][20347] Updated weights for policy 0, policy_version 1665184 (0.0006) [2023-12-27 17:10:54,458][20381] Updated weights for policy 1, policy_version 1663544 (0.0005) [2023-12-27 17:10:54,514][20381] Updated weights for policy 1, policy_version 1663554 (0.0006) [2023-12-27 17:10:54,521][20347] Updated weights for policy 0, policy_version 1665194 (0.0006) [2023-12-27 17:10:54,561][20381] Updated weights for policy 1, policy_version 1663564 (0.0005) [2023-12-27 17:10:54,570][20347] Updated weights for policy 0, policy_version 1665204 (0.0007) [2023-12-27 17:10:54,621][20347] Updated weights for policy 0, policy_version 1665214 (0.0007) [2023-12-27 17:10:54,680][20347] Updated weights for policy 0, policy_version 1665224 (0.0008) [2023-12-27 17:10:55,134][20381] Updated weights for policy 1, policy_version 1663574 (0.0007) [2023-12-27 17:10:55,192][20381] Updated weights for policy 1, policy_version 1663584 (0.0008) [2023-12-27 17:10:55,240][20381] Updated weights for policy 1, policy_version 1663594 (0.0008) [2023-12-27 17:10:55,374][20347] Updated weights for policy 0, policy_version 1665234 (0.0008) [2023-12-27 17:10:55,420][20347] Updated weights for policy 0, policy_version 1665244 (0.0008) [2023-12-27 17:10:55,474][20347] Updated weights for policy 0, policy_version 1665254 (0.0009) [2023-12-27 17:10:55,789][20381] Updated weights for policy 1, policy_version 1663604 (0.0006) [2023-12-27 17:10:55,847][20381] Updated weights for policy 1, policy_version 1663614 (0.0007) [2023-12-27 17:10:55,902][20381] Updated weights for policy 1, policy_version 1663624 (0.0007) [2023-12-27 17:10:56,200][20347] Updated weights for policy 0, policy_version 1665264 (0.0010) [2023-12-27 17:10:56,249][20347] Updated weights for policy 0, policy_version 1665274 (0.0010) [2023-12-27 17:10:56,299][20347] Updated weights for policy 0, policy_version 1665284 (0.0008) [2023-12-27 17:10:56,561][20381] Updated weights for policy 1, policy_version 1663634 (0.0008) [2023-12-27 17:10:56,619][20381] Updated weights for policy 1, policy_version 1663644 (0.0009) [2023-12-27 17:10:56,677][20381] Updated weights for policy 1, policy_version 1663654 (0.0007) [2023-12-27 17:10:56,735][20381] Updated weights for policy 1, policy_version 1663664 (0.0008) [2023-12-27 17:10:56,893][20347] Updated weights for policy 0, policy_version 1665294 (0.0007) [2023-12-27 17:10:56,944][20347] Updated weights for policy 0, policy_version 1665304 (0.0006) [2023-12-27 17:10:56,998][20347] Updated weights for policy 0, policy_version 1665314 (0.0005) [2023-12-27 17:10:57,295][20381] Updated weights for policy 1, policy_version 1663674 (0.0006) [2023-12-27 17:10:57,350][20381] Updated weights for policy 1, policy_version 1663684 (0.0007) [2023-12-27 17:10:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 852344832. Throughput: 0: 10213.7, 1: 10345.0. Samples: 852335104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:10:57,363][19170] Avg episode reward: [(0, '950.990'), (1, '853.480')] [2023-12-27 17:10:57,407][20381] Updated weights for policy 1, policy_version 1663694 (0.0006) [2023-12-27 17:10:57,551][20347] Updated weights for policy 0, policy_version 1665324 (0.0006) [2023-12-27 17:10:57,616][20347] Updated weights for policy 0, policy_version 1665334 (0.0006) [2023-12-27 17:10:57,678][20347] Updated weights for policy 0, policy_version 1665344 (0.0010) [2023-12-27 17:10:57,935][20381] Updated weights for policy 1, policy_version 1663704 (0.0008) [2023-12-27 17:10:57,982][20381] Updated weights for policy 1, policy_version 1663714 (0.0010) [2023-12-27 17:10:58,036][20381] Updated weights for policy 1, policy_version 1663724 (0.0010) [2023-12-27 17:10:58,277][20347] Updated weights for policy 0, policy_version 1665354 (0.0008) [2023-12-27 17:10:58,336][20347] Updated weights for policy 0, policy_version 1665364 (0.0007) [2023-12-27 17:10:58,401][20347] Updated weights for policy 0, policy_version 1665374 (0.0007) [2023-12-27 17:10:58,468][20347] Updated weights for policy 0, policy_version 1665384 (0.0007) [2023-12-27 17:10:58,776][20381] Updated weights for policy 1, policy_version 1663734 (0.0009) [2023-12-27 17:10:58,840][20381] Updated weights for policy 1, policy_version 1663744 (0.0008) [2023-12-27 17:10:58,913][20381] Updated weights for policy 1, policy_version 1663754 (0.0008) [2023-12-27 17:10:59,280][20347] Updated weights for policy 0, policy_version 1665394 (0.0009) [2023-12-27 17:10:59,343][20347] Updated weights for policy 0, policy_version 1665404 (0.0007) [2023-12-27 17:10:59,406][20347] Updated weights for policy 0, policy_version 1665414 (0.0007) [2023-12-27 17:10:59,684][20381] Updated weights for policy 1, policy_version 1663764 (0.0008) [2023-12-27 17:10:59,742][20381] Updated weights for policy 1, policy_version 1663774 (0.0007) [2023-12-27 17:10:59,799][20381] Updated weights for policy 1, policy_version 1663784 (0.0010) [2023-12-27 17:11:00,152][20347] Updated weights for policy 0, policy_version 1665424 (0.0007) [2023-12-27 17:11:00,198][20347] Updated weights for policy 0, policy_version 1665434 (0.0007) [2023-12-27 17:11:00,248][20347] Updated weights for policy 0, policy_version 1665444 (0.0007) [2023-12-27 17:11:00,504][20381] Updated weights for policy 1, policy_version 1663794 (0.0010) [2023-12-27 17:11:00,557][20381] Updated weights for policy 1, policy_version 1663804 (0.0011) [2023-12-27 17:11:00,606][20381] Updated weights for policy 1, policy_version 1663814 (0.0009) [2023-12-27 17:11:00,662][20381] Updated weights for policy 1, policy_version 1663824 (0.0005) [2023-12-27 17:11:00,919][20347] Updated weights for policy 0, policy_version 1665454 (0.0007) [2023-12-27 17:11:00,966][20347] Updated weights for policy 0, policy_version 1665464 (0.0007) [2023-12-27 17:11:01,018][20347] Updated weights for policy 0, policy_version 1665474 (0.0006) [2023-12-27 17:11:01,257][20381] Updated weights for policy 1, policy_version 1663834 (0.0008) [2023-12-27 17:11:01,318][20381] Updated weights for policy 1, policy_version 1663844 (0.0008) [2023-12-27 17:11:01,372][20381] Updated weights for policy 1, policy_version 1663854 (0.0008) [2023-12-27 17:11:01,687][20347] Updated weights for policy 0, policy_version 1665484 (0.0008) [2023-12-27 17:11:01,736][20347] Updated weights for policy 0, policy_version 1665494 (0.0007) [2023-12-27 17:11:01,804][20347] Updated weights for policy 0, policy_version 1665504 (0.0008) [2023-12-27 17:11:02,080][20381] Updated weights for policy 1, policy_version 1663864 (0.0008) [2023-12-27 17:11:02,136][20381] Updated weights for policy 1, policy_version 1663874 (0.0007) [2023-12-27 17:11:02,192][20381] Updated weights for policy 1, policy_version 1663884 (0.0008) [2023-12-27 17:11:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 852451328. Throughput: 0: 10273.0, 1: 10359.5. Samples: 852460520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:02,363][19170] Avg episode reward: [(0, '941.130'), (1, '824.300')] [2023-12-27 17:11:02,528][20347] Updated weights for policy 0, policy_version 1665514 (0.0008) [2023-12-27 17:11:02,586][20347] Updated weights for policy 0, policy_version 1665524 (0.0007) [2023-12-27 17:11:02,593][20291] KL-divergence is very high: 115.0053 [2023-12-27 17:11:02,637][20291] KL-divergence is very high: 118.2115 [2023-12-27 17:11:02,643][20347] Updated weights for policy 0, policy_version 1665534 (0.0008) [2023-12-27 17:11:02,694][20347] Updated weights for policy 0, policy_version 1665544 (0.0007) [2023-12-27 17:11:02,910][20381] Updated weights for policy 1, policy_version 1663894 (0.0009) [2023-12-27 17:11:02,966][20381] Updated weights for policy 1, policy_version 1663904 (0.0011) [2023-12-27 17:11:03,025][20381] Updated weights for policy 1, policy_version 1663914 (0.0011) [2023-12-27 17:11:03,426][20347] Updated weights for policy 0, policy_version 1665554 (0.0008) [2023-12-27 17:11:03,480][20347] Updated weights for policy 0, policy_version 1665564 (0.0008) [2023-12-27 17:11:03,532][20347] Updated weights for policy 0, policy_version 1665574 (0.0009) [2023-12-27 17:11:03,604][20381] Updated weights for policy 1, policy_version 1663924 (0.0008) [2023-12-27 17:11:03,661][20381] Updated weights for policy 1, policy_version 1663934 (0.0009) [2023-12-27 17:11:03,712][20381] Updated weights for policy 1, policy_version 1663944 (0.0005) [2023-12-27 17:11:04,278][20347] Updated weights for policy 0, policy_version 1665584 (0.0007) [2023-12-27 17:11:04,330][20381] Updated weights for policy 1, policy_version 1663954 (0.0006) [2023-12-27 17:11:04,338][20347] Updated weights for policy 0, policy_version 1665594 (0.0007) [2023-12-27 17:11:04,391][20381] Updated weights for policy 1, policy_version 1663964 (0.0007) [2023-12-27 17:11:04,399][20347] Updated weights for policy 0, policy_version 1665604 (0.0009) [2023-12-27 17:11:04,456][20381] Updated weights for policy 1, policy_version 1663974 (0.0007) [2023-12-27 17:11:04,515][20381] Updated weights for policy 1, policy_version 1663984 (0.0007) [2023-12-27 17:11:05,095][20381] Updated weights for policy 1, policy_version 1663994 (0.0005) [2023-12-27 17:11:05,137][20347] Updated weights for policy 0, policy_version 1665614 (0.0007) [2023-12-27 17:11:05,149][20381] Updated weights for policy 1, policy_version 1664004 (0.0005) [2023-12-27 17:11:05,190][20347] Updated weights for policy 0, policy_version 1665624 (0.0008) [2023-12-27 17:11:05,203][20381] Updated weights for policy 1, policy_version 1664014 (0.0005) [2023-12-27 17:11:05,237][20347] Updated weights for policy 0, policy_version 1665634 (0.0007) [2023-12-27 17:11:05,855][20381] Updated weights for policy 1, policy_version 1664024 (0.0010) [2023-12-27 17:11:05,861][20292] KL-divergence is very high: 177.0416 [2023-12-27 17:11:05,913][20292] KL-divergence is very high: 213.5986 [2023-12-27 17:11:05,918][20381] Updated weights for policy 1, policy_version 1664034 (0.0010) [2023-12-27 17:11:05,940][20347] Updated weights for policy 0, policy_version 1665644 (0.0008) [2023-12-27 17:11:05,961][20292] KL-divergence is very high: 165.1696 [2023-12-27 17:11:05,974][20381] Updated weights for policy 1, policy_version 1664044 (0.0008) [2023-12-27 17:11:05,997][20347] Updated weights for policy 0, policy_version 1665654 (0.0008) [2023-12-27 17:11:06,049][20347] Updated weights for policy 0, policy_version 1665664 (0.0009) [2023-12-27 17:11:06,645][20381] Updated weights for policy 1, policy_version 1664054 (0.0009) [2023-12-27 17:11:06,717][20381] Updated weights for policy 1, policy_version 1664064 (0.0011) [2023-12-27 17:11:06,723][20347] Updated weights for policy 0, policy_version 1665674 (0.0007) [2023-12-27 17:11:06,776][20347] Updated weights for policy 0, policy_version 1665684 (0.0010) [2023-12-27 17:11:06,779][20381] Updated weights for policy 1, policy_version 1664074 (0.0011) [2023-12-27 17:11:06,838][20347] Updated weights for policy 0, policy_version 1665694 (0.0009) [2023-12-27 17:11:06,892][20347] Updated weights for policy 0, policy_version 1665704 (0.0007) [2023-12-27 17:11:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 852549632. Throughput: 0: 10241.4, 1: 10443.7. Samples: 852522908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:07,363][19170] Avg episode reward: [(0, '921.690'), (1, '892.820')] [2023-12-27 17:11:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001665704_426500096.pth... [2023-12-27 17:11:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001664080_426049536.pth... [2023-12-27 17:11:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001664520_426196992.pth [2023-12-27 17:11:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001662864_425738240.pth [2023-12-27 17:11:07,460][20381] Updated weights for policy 1, policy_version 1664084 (0.0011) [2023-12-27 17:11:07,522][20381] Updated weights for policy 1, policy_version 1664094 (0.0011) [2023-12-27 17:11:07,578][20347] Updated weights for policy 0, policy_version 1665714 (0.0005) [2023-12-27 17:11:07,583][20381] Updated weights for policy 1, policy_version 1664104 (0.0011) [2023-12-27 17:11:07,636][20347] Updated weights for policy 0, policy_version 1665724 (0.0007) [2023-12-27 17:11:07,690][20347] Updated weights for policy 0, policy_version 1665734 (0.0007) [2023-12-27 17:11:08,186][20381] Updated weights for policy 1, policy_version 1664114 (0.0011) [2023-12-27 17:11:08,233][20381] Updated weights for policy 1, policy_version 1664124 (0.0010) [2023-12-27 17:11:08,281][20381] Updated weights for policy 1, policy_version 1664134 (0.0010) [2023-12-27 17:11:08,332][20381] Updated weights for policy 1, policy_version 1664144 (0.0010) [2023-12-27 17:11:08,378][20347] Updated weights for policy 0, policy_version 1665744 (0.0007) [2023-12-27 17:11:08,438][20347] Updated weights for policy 0, policy_version 1665754 (0.0008) [2023-12-27 17:11:08,497][20347] Updated weights for policy 0, policy_version 1665764 (0.0008) [2023-12-27 17:11:09,097][20381] Updated weights for policy 1, policy_version 1664154 (0.0006) [2023-12-27 17:11:09,148][20347] Updated weights for policy 0, policy_version 1665774 (0.0008) [2023-12-27 17:11:09,163][20381] Updated weights for policy 1, policy_version 1664164 (0.0007) [2023-12-27 17:11:09,198][20347] Updated weights for policy 0, policy_version 1665784 (0.0006) [2023-12-27 17:11:09,226][20381] Updated weights for policy 1, policy_version 1664174 (0.0008) [2023-12-27 17:11:09,260][20347] Updated weights for policy 0, policy_version 1665794 (0.0011) [2023-12-27 17:11:09,958][20347] Updated weights for policy 0, policy_version 1665804 (0.0009) [2023-12-27 17:11:09,961][20381] Updated weights for policy 1, policy_version 1664184 (0.0007) [2023-12-27 17:11:10,020][20347] Updated weights for policy 0, policy_version 1665814 (0.0009) [2023-12-27 17:11:10,021][20381] Updated weights for policy 1, policy_version 1664194 (0.0006) [2023-12-27 17:11:10,077][20347] Updated weights for policy 0, policy_version 1665824 (0.0009) [2023-12-27 17:11:10,081][20381] Updated weights for policy 1, policy_version 1664204 (0.0006) [2023-12-27 17:11:10,691][20381] Updated weights for policy 1, policy_version 1664214 (0.0010) [2023-12-27 17:11:10,740][20381] Updated weights for policy 1, policy_version 1664224 (0.0010) [2023-12-27 17:11:10,783][20347] Updated weights for policy 0, policy_version 1665834 (0.0006) [2023-12-27 17:11:10,792][20381] Updated weights for policy 1, policy_version 1664234 (0.0011) [2023-12-27 17:11:10,837][20347] Updated weights for policy 0, policy_version 1665844 (0.0005) [2023-12-27 17:11:10,890][20347] Updated weights for policy 0, policy_version 1665854 (0.0006) [2023-12-27 17:11:10,947][20347] Updated weights for policy 0, policy_version 1665864 (0.0005) [2023-12-27 17:11:11,518][20381] Updated weights for policy 1, policy_version 1664244 (0.0011) [2023-12-27 17:11:11,571][20381] Updated weights for policy 1, policy_version 1664254 (0.0007) [2023-12-27 17:11:11,641][20381] Updated weights for policy 1, policy_version 1664264 (0.0011) [2023-12-27 17:11:11,700][20347] Updated weights for policy 0, policy_version 1665874 (0.0007) [2023-12-27 17:11:11,769][20347] Updated weights for policy 0, policy_version 1665884 (0.0007) [2023-12-27 17:11:11,833][20347] Updated weights for policy 0, policy_version 1665894 (0.0007) [2023-12-27 17:11:12,314][20381] Updated weights for policy 1, policy_version 1664274 (0.0008) [2023-12-27 17:11:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 852647936. Throughput: 0: 10223.8, 1: 10394.2. Samples: 852644460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:12,363][19170] Avg episode reward: [(0, '931.530'), (1, '902.530')] [2023-12-27 17:11:12,381][20381] Updated weights for policy 1, policy_version 1664284 (0.0007) [2023-12-27 17:11:12,437][20381] Updated weights for policy 1, policy_version 1664294 (0.0008) [2023-12-27 17:11:12,499][20381] Updated weights for policy 1, policy_version 1664304 (0.0007) [2023-12-27 17:11:12,524][20347] Updated weights for policy 0, policy_version 1665904 (0.0007) [2023-12-27 17:11:12,585][20347] Updated weights for policy 0, policy_version 1665914 (0.0009) [2023-12-27 17:11:12,646][20347] Updated weights for policy 0, policy_version 1665924 (0.0006) [2023-12-27 17:11:13,184][20381] Updated weights for policy 1, policy_version 1664314 (0.0008) [2023-12-27 17:11:13,243][20381] Updated weights for policy 1, policy_version 1664324 (0.0008) [2023-12-27 17:11:13,305][20381] Updated weights for policy 1, policy_version 1664334 (0.0008) [2023-12-27 17:11:13,339][20347] Updated weights for policy 0, policy_version 1665934 (0.0007) [2023-12-27 17:11:13,395][20347] Updated weights for policy 0, policy_version 1665944 (0.0008) [2023-12-27 17:11:13,451][20347] Updated weights for policy 0, policy_version 1665954 (0.0007) [2023-12-27 17:11:13,973][20381] Updated weights for policy 1, policy_version 1664344 (0.0008) [2023-12-27 17:11:14,019][20381] Updated weights for policy 1, policy_version 1664354 (0.0009) [2023-12-27 17:11:14,066][20381] Updated weights for policy 1, policy_version 1664364 (0.0007) [2023-12-27 17:11:14,073][20347] Updated weights for policy 0, policy_version 1665964 (0.0007) [2023-12-27 17:11:14,124][20347] Updated weights for policy 0, policy_version 1665974 (0.0005) [2023-12-27 17:11:14,181][20347] Updated weights for policy 0, policy_version 1665984 (0.0005) [2023-12-27 17:11:14,712][20347] Updated weights for policy 0, policy_version 1665994 (0.0006) [2023-12-27 17:11:14,723][20381] Updated weights for policy 1, policy_version 1664374 (0.0008) [2023-12-27 17:11:14,773][20347] Updated weights for policy 0, policy_version 1666004 (0.0006) [2023-12-27 17:11:14,788][20381] Updated weights for policy 1, policy_version 1664384 (0.0008) [2023-12-27 17:11:14,840][20347] Updated weights for policy 0, policy_version 1666014 (0.0008) [2023-12-27 17:11:14,846][20381] Updated weights for policy 1, policy_version 1664394 (0.0007) [2023-12-27 17:11:14,903][20347] Updated weights for policy 0, policy_version 1666024 (0.0007) [2023-12-27 17:11:15,547][20347] Updated weights for policy 0, policy_version 1666034 (0.0006) [2023-12-27 17:11:15,594][20381] Updated weights for policy 1, policy_version 1664404 (0.0007) [2023-12-27 17:11:15,610][20347] Updated weights for policy 0, policy_version 1666044 (0.0007) [2023-12-27 17:11:15,648][20381] Updated weights for policy 1, policy_version 1664414 (0.0007) [2023-12-27 17:11:15,670][20347] Updated weights for policy 0, policy_version 1666054 (0.0008) [2023-12-27 17:11:15,700][20381] Updated weights for policy 1, policy_version 1664424 (0.0007) [2023-12-27 17:11:16,255][20347] Updated weights for policy 0, policy_version 1666064 (0.0007) [2023-12-27 17:11:16,313][20347] Updated weights for policy 0, policy_version 1666074 (0.0008) [2023-12-27 17:11:16,375][20347] Updated weights for policy 0, policy_version 1666084 (0.0007) [2023-12-27 17:11:16,436][20381] Updated weights for policy 1, policy_version 1664434 (0.0007) [2023-12-27 17:11:16,485][20381] Updated weights for policy 1, policy_version 1664444 (0.0008) [2023-12-27 17:11:16,540][20381] Updated weights for policy 1, policy_version 1664454 (0.0008) [2023-12-27 17:11:16,589][20381] Updated weights for policy 1, policy_version 1664464 (0.0007) [2023-12-27 17:11:17,039][20347] Updated weights for policy 0, policy_version 1666094 (0.0009) [2023-12-27 17:11:17,099][20347] Updated weights for policy 0, policy_version 1666104 (0.0010) [2023-12-27 17:11:17,157][20347] Updated weights for policy 0, policy_version 1666114 (0.0010) [2023-12-27 17:11:17,298][20381] Updated weights for policy 1, policy_version 1664474 (0.0007) [2023-12-27 17:11:17,354][20381] Updated weights for policy 1, policy_version 1664484 (0.0008) [2023-12-27 17:11:17,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 852754432. Throughput: 0: 10249.9, 1: 10368.5. Samples: 852766680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:17,364][19170] Avg episode reward: [(0, '941.250'), (1, '882.770')] [2023-12-27 17:11:17,397][20381] Updated weights for policy 1, policy_version 1664494 (0.0007) [2023-12-27 17:11:17,816][20347] Updated weights for policy 0, policy_version 1666124 (0.0010) [2023-12-27 17:11:17,873][20347] Updated weights for policy 0, policy_version 1666134 (0.0010) [2023-12-27 17:11:17,931][20347] Updated weights for policy 0, policy_version 1666144 (0.0010) [2023-12-27 17:11:18,133][20381] Updated weights for policy 1, policy_version 1664504 (0.0008) [2023-12-27 17:11:18,189][20381] Updated weights for policy 1, policy_version 1664514 (0.0008) [2023-12-27 17:11:18,241][20381] Updated weights for policy 1, policy_version 1664524 (0.0008) [2023-12-27 17:11:18,614][20347] Updated weights for policy 0, policy_version 1666154 (0.0010) [2023-12-27 17:11:18,670][20347] Updated weights for policy 0, policy_version 1666164 (0.0007) [2023-12-27 17:11:18,726][20347] Updated weights for policy 0, policy_version 1666174 (0.0008) [2023-12-27 17:11:18,783][20347] Updated weights for policy 0, policy_version 1666184 (0.0007) [2023-12-27 17:11:18,988][20381] Updated weights for policy 1, policy_version 1664534 (0.0007) [2023-12-27 17:11:19,040][20381] Updated weights for policy 1, policy_version 1664544 (0.0009) [2023-12-27 17:11:19,091][20381] Updated weights for policy 1, policy_version 1664554 (0.0008) [2023-12-27 17:11:19,386][20347] Updated weights for policy 0, policy_version 1666194 (0.0010) [2023-12-27 17:11:19,451][20347] Updated weights for policy 0, policy_version 1666204 (0.0007) [2023-12-27 17:11:19,511][20347] Updated weights for policy 0, policy_version 1666214 (0.0010) [2023-12-27 17:11:19,851][20381] Updated weights for policy 1, policy_version 1664564 (0.0010) [2023-12-27 17:11:19,916][20381] Updated weights for policy 1, policy_version 1664574 (0.0007) [2023-12-27 17:11:19,973][20381] Updated weights for policy 1, policy_version 1664584 (0.0007) [2023-12-27 17:11:20,164][20347] Updated weights for policy 0, policy_version 1666224 (0.0010) [2023-12-27 17:11:20,225][20347] Updated weights for policy 0, policy_version 1666234 (0.0010) [2023-12-27 17:11:20,274][20347] Updated weights for policy 0, policy_version 1666244 (0.0010) [2023-12-27 17:11:20,649][20381] Updated weights for policy 1, policy_version 1664594 (0.0008) [2023-12-27 17:11:20,707][20381] Updated weights for policy 1, policy_version 1664604 (0.0007) [2023-12-27 17:11:20,765][20381] Updated weights for policy 1, policy_version 1664614 (0.0008) [2023-12-27 17:11:20,831][20381] Updated weights for policy 1, policy_version 1664624 (0.0008) [2023-12-27 17:11:20,952][20347] Updated weights for policy 0, policy_version 1666254 (0.0008) [2023-12-27 17:11:21,016][20347] Updated weights for policy 0, policy_version 1666264 (0.0009) [2023-12-27 17:11:21,080][20347] Updated weights for policy 0, policy_version 1666274 (0.0008) [2023-12-27 17:11:21,497][20381] Updated weights for policy 1, policy_version 1664634 (0.0008) [2023-12-27 17:11:21,556][20381] Updated weights for policy 1, policy_version 1664644 (0.0008) [2023-12-27 17:11:21,609][20381] Updated weights for policy 1, policy_version 1664654 (0.0008) [2023-12-27 17:11:21,804][20347] Updated weights for policy 0, policy_version 1666284 (0.0007) [2023-12-27 17:11:21,869][20347] Updated weights for policy 0, policy_version 1666294 (0.0006) [2023-12-27 17:11:21,935][20347] Updated weights for policy 0, policy_version 1666304 (0.0006) [2023-12-27 17:11:22,342][20381] Updated weights for policy 1, policy_version 1664664 (0.0008) [2023-12-27 17:11:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 852852736. Throughput: 0: 10209.0, 1: 10322.6. Samples: 852826972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:22,363][19170] Avg episode reward: [(0, '951.040'), (1, '931.730')] [2023-12-27 17:11:22,380][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001666312_426655744.pth... [2023-12-27 17:11:22,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001665128_426352640.pth [2023-12-27 17:11:22,406][20381] Updated weights for policy 1, policy_version 1664674 (0.0008) [2023-12-27 17:11:22,469][20381] Updated weights for policy 1, policy_version 1664684 (0.0008) [2023-12-27 17:11:22,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001664688_426205184.pth... [2023-12-27 17:11:22,500][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001663472_425893888.pth [2023-12-27 17:11:22,619][20347] Updated weights for policy 0, policy_version 1666314 (0.0008) [2023-12-27 17:11:22,681][20347] Updated weights for policy 0, policy_version 1666324 (0.0008) [2023-12-27 17:11:22,742][20347] Updated weights for policy 0, policy_version 1666334 (0.0007) [2023-12-27 17:11:22,809][20347] Updated weights for policy 0, policy_version 1666344 (0.0008) [2023-12-27 17:11:23,175][20381] Updated weights for policy 1, policy_version 1664694 (0.0008) [2023-12-27 17:11:23,228][20381] Updated weights for policy 1, policy_version 1664704 (0.0007) [2023-12-27 17:11:23,281][20381] Updated weights for policy 1, policy_version 1664714 (0.0008) [2023-12-27 17:11:23,469][20347] Updated weights for policy 0, policy_version 1666354 (0.0006) [2023-12-27 17:11:23,519][20347] Updated weights for policy 0, policy_version 1666364 (0.0005) [2023-12-27 17:11:23,571][20347] Updated weights for policy 0, policy_version 1666374 (0.0006) [2023-12-27 17:11:23,972][20381] Updated weights for policy 1, policy_version 1664724 (0.0007) [2023-12-27 17:11:24,028][20381] Updated weights for policy 1, policy_version 1664734 (0.0008) [2023-12-27 17:11:24,084][20381] Updated weights for policy 1, policy_version 1664744 (0.0008) [2023-12-27 17:11:24,219][20347] Updated weights for policy 0, policy_version 1666384 (0.0006) [2023-12-27 17:11:24,279][20347] Updated weights for policy 0, policy_version 1666394 (0.0008) [2023-12-27 17:11:24,338][20347] Updated weights for policy 0, policy_version 1666404 (0.0008) [2023-12-27 17:11:24,742][20381] Updated weights for policy 1, policy_version 1664754 (0.0007) [2023-12-27 17:11:24,794][20381] Updated weights for policy 1, policy_version 1664764 (0.0008) [2023-12-27 17:11:24,845][20381] Updated weights for policy 1, policy_version 1664774 (0.0008) [2023-12-27 17:11:24,847][20292] KL-divergence is very high: 157.5391 [2023-12-27 17:11:24,892][20292] KL-divergence is very high: 207.1692 [2023-12-27 17:11:24,903][20381] Updated weights for policy 1, policy_version 1664784 (0.0007) [2023-12-27 17:11:25,028][20347] Updated weights for policy 0, policy_version 1666414 (0.0008) [2023-12-27 17:11:25,077][20347] Updated weights for policy 0, policy_version 1666424 (0.0007) [2023-12-27 17:11:25,132][20347] Updated weights for policy 0, policy_version 1666434 (0.0008) [2023-12-27 17:11:25,547][20292] KL-divergence is very high: 203.3181 [2023-12-27 17:11:25,590][20292] KL-divergence is very high: 130.3426 [2023-12-27 17:11:25,595][20381] Updated weights for policy 1, policy_version 1664794 (0.0008) [2023-12-27 17:11:25,647][20381] Updated weights for policy 1, policy_version 1664804 (0.0007) [2023-12-27 17:11:25,700][20381] Updated weights for policy 1, policy_version 1664814 (0.0008) [2023-12-27 17:11:25,841][20347] Updated weights for policy 0, policy_version 1666444 (0.0008) [2023-12-27 17:11:25,886][20347] Updated weights for policy 0, policy_version 1666454 (0.0007) [2023-12-27 17:11:25,931][20347] Updated weights for policy 0, policy_version 1666464 (0.0008) [2023-12-27 17:11:26,351][20381] Updated weights for policy 1, policy_version 1664824 (0.0008) [2023-12-27 17:11:26,407][20381] Updated weights for policy 1, policy_version 1664834 (0.0007) [2023-12-27 17:11:26,454][20381] Updated weights for policy 1, policy_version 1664844 (0.0005) [2023-12-27 17:11:26,595][20347] Updated weights for policy 0, policy_version 1666474 (0.0008) [2023-12-27 17:11:26,649][20347] Updated weights for policy 0, policy_version 1666484 (0.0008) [2023-12-27 17:11:26,704][20347] Updated weights for policy 0, policy_version 1666494 (0.0008) [2023-12-27 17:11:26,757][20347] Updated weights for policy 0, policy_version 1666504 (0.0009) [2023-12-27 17:11:27,002][20381] Updated weights for policy 1, policy_version 1664854 (0.0006) [2023-12-27 17:11:27,057][20381] Updated weights for policy 1, policy_version 1664864 (0.0005) [2023-12-27 17:11:27,118][20381] Updated weights for policy 1, policy_version 1664874 (0.0005) [2023-12-27 17:11:27,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 852959232. Throughput: 0: 10175.7, 1: 10272.6. Samples: 852947628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:27,363][19170] Avg episode reward: [(0, '931.640'), (1, '902.700')] [2023-12-27 17:11:27,511][20347] Updated weights for policy 0, policy_version 1666514 (0.0008) [2023-12-27 17:11:27,556][20347] Updated weights for policy 0, policy_version 1666524 (0.0007) [2023-12-27 17:11:27,609][20347] Updated weights for policy 0, policy_version 1666534 (0.0007) [2023-12-27 17:11:27,687][20381] Updated weights for policy 1, policy_version 1664884 (0.0006) [2023-12-27 17:11:27,739][20381] Updated weights for policy 1, policy_version 1664894 (0.0005) [2023-12-27 17:11:27,784][20381] Updated weights for policy 1, policy_version 1664904 (0.0005) [2023-12-27 17:11:28,312][20381] Updated weights for policy 1, policy_version 1664914 (0.0006) [2023-12-27 17:11:28,376][20381] Updated weights for policy 1, policy_version 1664924 (0.0007) [2023-12-27 17:11:28,392][20347] Updated weights for policy 0, policy_version 1666544 (0.0007) [2023-12-27 17:11:28,433][20381] Updated weights for policy 1, policy_version 1664934 (0.0006) [2023-12-27 17:11:28,453][20347] Updated weights for policy 0, policy_version 1666554 (0.0006) [2023-12-27 17:11:28,491][20381] Updated weights for policy 1, policy_version 1664944 (0.0005) [2023-12-27 17:11:28,511][20347] Updated weights for policy 0, policy_version 1666564 (0.0007) [2023-12-27 17:11:29,125][20381] Updated weights for policy 1, policy_version 1664954 (0.0008) [2023-12-27 17:11:29,166][20347] Updated weights for policy 0, policy_version 1666574 (0.0006) [2023-12-27 17:11:29,175][20381] Updated weights for policy 1, policy_version 1664964 (0.0007) [2023-12-27 17:11:29,223][20347] Updated weights for policy 0, policy_version 1666584 (0.0007) [2023-12-27 17:11:29,233][20381] Updated weights for policy 1, policy_version 1664974 (0.0007) [2023-12-27 17:11:29,287][20347] Updated weights for policy 0, policy_version 1666594 (0.0008) [2023-12-27 17:11:29,959][20381] Updated weights for policy 1, policy_version 1664984 (0.0008) [2023-12-27 17:11:30,005][20381] Updated weights for policy 1, policy_version 1664994 (0.0007) [2023-12-27 17:11:30,024][20347] Updated weights for policy 0, policy_version 1666604 (0.0009) [2023-12-27 17:11:30,054][20381] Updated weights for policy 1, policy_version 1665004 (0.0007) [2023-12-27 17:11:30,085][20347] Updated weights for policy 0, policy_version 1666614 (0.0005) [2023-12-27 17:11:30,142][20347] Updated weights for policy 0, policy_version 1666624 (0.0007) [2023-12-27 17:11:30,714][20347] Updated weights for policy 0, policy_version 1666634 (0.0008) [2023-12-27 17:11:30,772][20347] Updated weights for policy 0, policy_version 1666644 (0.0008) [2023-12-27 17:11:30,794][20381] Updated weights for policy 1, policy_version 1665014 (0.0007) [2023-12-27 17:11:30,819][20347] Updated weights for policy 0, policy_version 1666654 (0.0008) [2023-12-27 17:11:30,850][20381] Updated weights for policy 1, policy_version 1665024 (0.0008) [2023-12-27 17:11:30,872][20347] Updated weights for policy 0, policy_version 1666664 (0.0010) [2023-12-27 17:11:30,903][20381] Updated weights for policy 1, policy_version 1665034 (0.0008) [2023-12-27 17:11:31,565][20381] Updated weights for policy 1, policy_version 1665044 (0.0007) [2023-12-27 17:11:31,571][20347] Updated weights for policy 0, policy_version 1666674 (0.0007) [2023-12-27 17:11:31,623][20381] Updated weights for policy 1, policy_version 1665054 (0.0007) [2023-12-27 17:11:31,635][20347] Updated weights for policy 0, policy_version 1666684 (0.0007) [2023-12-27 17:11:31,687][20381] Updated weights for policy 1, policy_version 1665064 (0.0006) [2023-12-27 17:11:31,689][20347] Updated weights for policy 0, policy_version 1666694 (0.0009) [2023-12-27 17:11:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 853057536. Throughput: 0: 10116.9, 1: 10412.2. Samples: 853073668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:32,363][19170] Avg episode reward: [(0, '882.580'), (1, '892.830')] [2023-12-27 17:11:32,365][20381] Updated weights for policy 1, policy_version 1665074 (0.0008) [2023-12-27 17:11:32,418][20347] Updated weights for policy 0, policy_version 1666704 (0.0011) [2023-12-27 17:11:32,420][20381] Updated weights for policy 1, policy_version 1665084 (0.0005) [2023-12-27 17:11:32,471][20381] Updated weights for policy 1, policy_version 1665094 (0.0005) [2023-12-27 17:11:32,481][20347] Updated weights for policy 0, policy_version 1666714 (0.0011) [2023-12-27 17:11:32,524][20381] Updated weights for policy 1, policy_version 1665104 (0.0006) [2023-12-27 17:11:32,537][20347] Updated weights for policy 0, policy_version 1666724 (0.0010) [2023-12-27 17:11:33,173][20347] Updated weights for policy 0, policy_version 1666734 (0.0010) [2023-12-27 17:11:33,231][20347] Updated weights for policy 0, policy_version 1666744 (0.0010) [2023-12-27 17:11:33,236][20381] Updated weights for policy 1, policy_version 1665114 (0.0005) [2023-12-27 17:11:33,284][20347] Updated weights for policy 0, policy_version 1666754 (0.0008) [2023-12-27 17:11:33,293][20381] Updated weights for policy 1, policy_version 1665124 (0.0006) [2023-12-27 17:11:33,345][20381] Updated weights for policy 1, policy_version 1665134 (0.0006) [2023-12-27 17:11:33,866][20347] Updated weights for policy 0, policy_version 1666764 (0.0009) [2023-12-27 17:11:33,919][20347] Updated weights for policy 0, policy_version 1666774 (0.0006) [2023-12-27 17:11:33,970][20347] Updated weights for policy 0, policy_version 1666784 (0.0009) [2023-12-27 17:11:34,069][20381] Updated weights for policy 1, policy_version 1665144 (0.0007) [2023-12-27 17:11:34,125][20381] Updated weights for policy 1, policy_version 1665154 (0.0008) [2023-12-27 17:11:34,181][20381] Updated weights for policy 1, policy_version 1665164 (0.0006) [2023-12-27 17:11:34,620][20347] Updated weights for policy 0, policy_version 1666794 (0.0006) [2023-12-27 17:11:34,687][20347] Updated weights for policy 0, policy_version 1666804 (0.0010) [2023-12-27 17:11:34,747][20347] Updated weights for policy 0, policy_version 1666814 (0.0009) [2023-12-27 17:11:34,795][20347] Updated weights for policy 0, policy_version 1666824 (0.0007) [2023-12-27 17:11:34,933][20381] Updated weights for policy 1, policy_version 1665174 (0.0007) [2023-12-27 17:11:34,988][20381] Updated weights for policy 1, policy_version 1665184 (0.0008) [2023-12-27 17:11:35,043][20381] Updated weights for policy 1, policy_version 1665194 (0.0010) [2023-12-27 17:11:35,471][20347] Updated weights for policy 0, policy_version 1666834 (0.0007) [2023-12-27 17:11:35,526][20347] Updated weights for policy 0, policy_version 1666844 (0.0008) [2023-12-27 17:11:35,587][20347] Updated weights for policy 0, policy_version 1666854 (0.0008) [2023-12-27 17:11:35,710][20381] Updated weights for policy 1, policy_version 1665204 (0.0010) [2023-12-27 17:11:35,769][20381] Updated weights for policy 1, policy_version 1665214 (0.0010) [2023-12-27 17:11:35,826][20381] Updated weights for policy 1, policy_version 1665224 (0.0010) [2023-12-27 17:11:36,308][20347] Updated weights for policy 0, policy_version 1666864 (0.0008) [2023-12-27 17:11:36,378][20347] Updated weights for policy 0, policy_version 1666874 (0.0007) [2023-12-27 17:11:36,449][20347] Updated weights for policy 0, policy_version 1666884 (0.0008) [2023-12-27 17:11:36,454][20381] Updated weights for policy 1, policy_version 1665234 (0.0010) [2023-12-27 17:11:36,516][20381] Updated weights for policy 1, policy_version 1665244 (0.0006) [2023-12-27 17:11:36,571][20381] Updated weights for policy 1, policy_version 1665254 (0.0007) [2023-12-27 17:11:36,618][20381] Updated weights for policy 1, policy_version 1665264 (0.0010) [2023-12-27 17:11:37,051][20347] Updated weights for policy 0, policy_version 1666894 (0.0006) [2023-12-27 17:11:37,104][20347] Updated weights for policy 0, policy_version 1666904 (0.0006) [2023-12-27 17:11:37,164][20347] Updated weights for policy 0, policy_version 1666914 (0.0007) [2023-12-27 17:11:37,219][20381] Updated weights for policy 1, policy_version 1665274 (0.0005) [2023-12-27 17:11:37,269][20381] Updated weights for policy 1, policy_version 1665284 (0.0005) [2023-12-27 17:11:37,321][20381] Updated weights for policy 1, policy_version 1665294 (0.0005) [2023-12-27 17:11:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.7, 300 sec: 20632.7). Total num frames: 853172224. Throughput: 0: 10156.4, 1: 10339.4. Samples: 853134716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:37,363][19170] Avg episode reward: [(0, '931.440'), (1, '892.580')] [2023-12-27 17:11:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001666920_426811392.pth... [2023-12-27 17:11:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001665296_426360832.pth... [2023-12-27 17:11:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001665704_426500096.pth [2023-12-27 17:11:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001664080_426049536.pth [2023-12-27 17:11:37,860][20347] Updated weights for policy 0, policy_version 1666924 (0.0007) [2023-12-27 17:11:37,912][20347] Updated weights for policy 0, policy_version 1666934 (0.0007) [2023-12-27 17:11:37,933][20381] Updated weights for policy 1, policy_version 1665304 (0.0006) [2023-12-27 17:11:37,962][20347] Updated weights for policy 0, policy_version 1666944 (0.0005) [2023-12-27 17:11:37,983][20381] Updated weights for policy 1, policy_version 1665314 (0.0006) [2023-12-27 17:11:38,030][20381] Updated weights for policy 1, policy_version 1665324 (0.0008) [2023-12-27 17:11:38,588][20347] Updated weights for policy 0, policy_version 1666954 (0.0006) [2023-12-27 17:11:38,642][20347] Updated weights for policy 0, policy_version 1666964 (0.0008) [2023-12-27 17:11:38,699][20347] Updated weights for policy 0, policy_version 1666974 (0.0007) [2023-12-27 17:11:38,759][20347] Updated weights for policy 0, policy_version 1666984 (0.0007) [2023-12-27 17:11:38,761][20381] Updated weights for policy 1, policy_version 1665334 (0.0007) [2023-12-27 17:11:38,818][20381] Updated weights for policy 1, policy_version 1665344 (0.0007) [2023-12-27 17:11:38,878][20381] Updated weights for policy 1, policy_version 1665354 (0.0008) [2023-12-27 17:11:39,440][20347] Updated weights for policy 0, policy_version 1666994 (0.0008) [2023-12-27 17:11:39,505][20347] Updated weights for policy 0, policy_version 1667004 (0.0008) [2023-12-27 17:11:39,564][20347] Updated weights for policy 0, policy_version 1667014 (0.0008) [2023-12-27 17:11:39,655][20381] Updated weights for policy 1, policy_version 1665364 (0.0008) [2023-12-27 17:11:39,713][20381] Updated weights for policy 1, policy_version 1665374 (0.0006) [2023-12-27 17:11:39,776][20381] Updated weights for policy 1, policy_version 1665384 (0.0006) [2023-12-27 17:11:40,302][20347] Updated weights for policy 0, policy_version 1667024 (0.0007) [2023-12-27 17:11:40,358][20347] Updated weights for policy 0, policy_version 1667034 (0.0008) [2023-12-27 17:11:40,425][20347] Updated weights for policy 0, policy_version 1667044 (0.0007) [2023-12-27 17:11:40,434][20381] Updated weights for policy 1, policy_version 1665394 (0.0007) [2023-12-27 17:11:40,487][20381] Updated weights for policy 1, policy_version 1665404 (0.0009) [2023-12-27 17:11:40,545][20381] Updated weights for policy 1, policy_version 1665414 (0.0008) [2023-12-27 17:11:40,594][20381] Updated weights for policy 1, policy_version 1665424 (0.0008) [2023-12-27 17:11:41,046][20347] Updated weights for policy 0, policy_version 1667054 (0.0006) [2023-12-27 17:11:41,114][20347] Updated weights for policy 0, policy_version 1667064 (0.0007) [2023-12-27 17:11:41,179][20347] Updated weights for policy 0, policy_version 1667074 (0.0008) [2023-12-27 17:11:41,201][20291] KL-divergence is very high: 166.9456 [2023-12-27 17:11:41,253][20381] Updated weights for policy 1, policy_version 1665434 (0.0007) [2023-12-27 17:11:41,315][20381] Updated weights for policy 1, policy_version 1665444 (0.0008) [2023-12-27 17:11:41,376][20381] Updated weights for policy 1, policy_version 1665454 (0.0007) [2023-12-27 17:11:41,918][20347] Updated weights for policy 0, policy_version 1667084 (0.0008) [2023-12-27 17:11:41,977][20347] Updated weights for policy 0, policy_version 1667094 (0.0010) [2023-12-27 17:11:42,036][20347] Updated weights for policy 0, policy_version 1667104 (0.0010) [2023-12-27 17:11:42,077][20381] Updated weights for policy 1, policy_version 1665464 (0.0009) [2023-12-27 17:11:42,141][20381] Updated weights for policy 1, policy_version 1665474 (0.0006) [2023-12-27 17:11:42,190][20381] Updated weights for policy 1, policy_version 1665484 (0.0008) [2023-12-27 17:11:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 853270528. Throughput: 0: 10232.0, 1: 10288.9. Samples: 853258544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:42,363][19170] Avg episode reward: [(0, '931.460'), (1, '912.150')] [2023-12-27 17:11:42,721][20347] Updated weights for policy 0, policy_version 1667114 (0.0009) [2023-12-27 17:11:42,784][20347] Updated weights for policy 0, policy_version 1667124 (0.0006) [2023-12-27 17:11:42,845][20347] Updated weights for policy 0, policy_version 1667134 (0.0009) [2023-12-27 17:11:42,903][20347] Updated weights for policy 0, policy_version 1667144 (0.0010) [2023-12-27 17:11:42,938][20381] Updated weights for policy 1, policy_version 1665494 (0.0008) [2023-12-27 17:11:42,995][20381] Updated weights for policy 1, policy_version 1665504 (0.0008) [2023-12-27 17:11:43,045][20381] Updated weights for policy 1, policy_version 1665514 (0.0006) [2023-12-27 17:11:43,470][20347] Updated weights for policy 0, policy_version 1667154 (0.0005) [2023-12-27 17:11:43,524][20347] Updated weights for policy 0, policy_version 1667164 (0.0005) [2023-12-27 17:11:43,580][20347] Updated weights for policy 0, policy_version 1667174 (0.0008) [2023-12-27 17:11:43,725][20381] Updated weights for policy 1, policy_version 1665524 (0.0006) [2023-12-27 17:11:43,784][20381] Updated weights for policy 1, policy_version 1665534 (0.0005) [2023-12-27 17:11:43,839][20381] Updated weights for policy 1, policy_version 1665544 (0.0005) [2023-12-27 17:11:44,170][20347] Updated weights for policy 0, policy_version 1667184 (0.0007) [2023-12-27 17:11:44,227][20347] Updated weights for policy 0, policy_version 1667194 (0.0007) [2023-12-27 17:11:44,279][20347] Updated weights for policy 0, policy_version 1667204 (0.0007) [2023-12-27 17:11:44,429][20381] Updated weights for policy 1, policy_version 1665554 (0.0006) [2023-12-27 17:11:44,485][20381] Updated weights for policy 1, policy_version 1665564 (0.0009) [2023-12-27 17:11:44,527][20381] Updated weights for policy 1, policy_version 1665574 (0.0006) [2023-12-27 17:11:44,588][20381] Updated weights for policy 1, policy_version 1665584 (0.0007) [2023-12-27 17:11:45,014][20347] Updated weights for policy 0, policy_version 1667214 (0.0007) [2023-12-27 17:11:45,078][20347] Updated weights for policy 0, policy_version 1667224 (0.0008) [2023-12-27 17:11:45,139][20347] Updated weights for policy 0, policy_version 1667234 (0.0008) [2023-12-27 17:11:45,271][20381] Updated weights for policy 1, policy_version 1665594 (0.0007) [2023-12-27 17:11:45,327][20381] Updated weights for policy 1, policy_version 1665604 (0.0007) [2023-12-27 17:11:45,380][20381] Updated weights for policy 1, policy_version 1665614 (0.0008) [2023-12-27 17:11:45,867][20347] Updated weights for policy 0, policy_version 1667244 (0.0008) [2023-12-27 17:11:45,931][20347] Updated weights for policy 0, policy_version 1667254 (0.0008) [2023-12-27 17:11:45,989][20347] Updated weights for policy 0, policy_version 1667264 (0.0008) [2023-12-27 17:11:46,036][20381] Updated weights for policy 1, policy_version 1665624 (0.0007) [2023-12-27 17:11:46,094][20381] Updated weights for policy 1, policy_version 1665634 (0.0006) [2023-12-27 17:11:46,158][20381] Updated weights for policy 1, policy_version 1665644 (0.0006) [2023-12-27 17:11:46,634][20347] Updated weights for policy 0, policy_version 1667274 (0.0005) [2023-12-27 17:11:46,688][20347] Updated weights for policy 0, policy_version 1667284 (0.0006) [2023-12-27 17:11:46,733][20347] Updated weights for policy 0, policy_version 1667294 (0.0007) [2023-12-27 17:11:46,783][20347] Updated weights for policy 0, policy_version 1667304 (0.0007) [2023-12-27 17:11:46,785][20381] Updated weights for policy 1, policy_version 1665654 (0.0007) [2023-12-27 17:11:46,845][20381] Updated weights for policy 1, policy_version 1665664 (0.0007) [2023-12-27 17:11:46,893][20381] Updated weights for policy 1, policy_version 1665674 (0.0008) [2023-12-27 17:11:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 853368832. Throughput: 0: 10205.8, 1: 10264.4. Samples: 853381684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:47,364][19170] Avg episode reward: [(0, '941.240'), (1, '951.220')] [2023-12-27 17:11:47,438][20347] Updated weights for policy 0, policy_version 1667314 (0.0006) [2023-12-27 17:11:47,486][20347] Updated weights for policy 0, policy_version 1667324 (0.0005) [2023-12-27 17:11:47,539][20347] Updated weights for policy 0, policy_version 1667334 (0.0005) [2023-12-27 17:11:47,613][20381] Updated weights for policy 1, policy_version 1665684 (0.0009) [2023-12-27 17:11:47,658][20381] Updated weights for policy 1, policy_version 1665694 (0.0010) [2023-12-27 17:11:47,703][20381] Updated weights for policy 1, policy_version 1665704 (0.0010) [2023-12-27 17:11:48,164][20347] Updated weights for policy 0, policy_version 1667344 (0.0005) [2023-12-27 17:11:48,222][20347] Updated weights for policy 0, policy_version 1667354 (0.0005) [2023-12-27 17:11:48,285][20347] Updated weights for policy 0, policy_version 1667364 (0.0006) [2023-12-27 17:11:48,420][20381] Updated weights for policy 1, policy_version 1665714 (0.0010) [2023-12-27 17:11:48,479][20381] Updated weights for policy 1, policy_version 1665724 (0.0011) [2023-12-27 17:11:48,528][20381] Updated weights for policy 1, policy_version 1665734 (0.0010) [2023-12-27 17:11:48,577][20381] Updated weights for policy 1, policy_version 1665744 (0.0009) [2023-12-27 17:11:48,857][20347] Updated weights for policy 0, policy_version 1667374 (0.0005) [2023-12-27 17:11:48,916][20347] Updated weights for policy 0, policy_version 1667384 (0.0009) [2023-12-27 17:11:48,974][20347] Updated weights for policy 0, policy_version 1667394 (0.0010) [2023-12-27 17:11:49,253][20381] Updated weights for policy 1, policy_version 1665754 (0.0008) [2023-12-27 17:11:49,301][20381] Updated weights for policy 1, policy_version 1665764 (0.0008) [2023-12-27 17:11:49,357][20381] Updated weights for policy 1, policy_version 1665774 (0.0008) [2023-12-27 17:11:49,629][20347] Updated weights for policy 0, policy_version 1667404 (0.0009) [2023-12-27 17:11:49,689][20347] Updated weights for policy 0, policy_version 1667414 (0.0007) [2023-12-27 17:11:49,752][20347] Updated weights for policy 0, policy_version 1667424 (0.0006) [2023-12-27 17:11:50,135][20381] Updated weights for policy 1, policy_version 1665784 (0.0008) [2023-12-27 17:11:50,196][20381] Updated weights for policy 1, policy_version 1665794 (0.0008) [2023-12-27 17:11:50,252][20381] Updated weights for policy 1, policy_version 1665804 (0.0008) [2023-12-27 17:11:50,419][20347] Updated weights for policy 0, policy_version 1667434 (0.0006) [2023-12-27 17:11:50,481][20347] Updated weights for policy 0, policy_version 1667444 (0.0006) [2023-12-27 17:11:50,545][20347] Updated weights for policy 0, policy_version 1667454 (0.0006) [2023-12-27 17:11:50,609][20347] Updated weights for policy 0, policy_version 1667464 (0.0008) [2023-12-27 17:11:50,933][20381] Updated weights for policy 1, policy_version 1665814 (0.0006) [2023-12-27 17:11:50,990][20381] Updated weights for policy 1, policy_version 1665824 (0.0005) [2023-12-27 17:11:51,054][20381] Updated weights for policy 1, policy_version 1665834 (0.0007) [2023-12-27 17:11:51,313][20347] Updated weights for policy 0, policy_version 1667474 (0.0011) [2023-12-27 17:11:51,379][20347] Updated weights for policy 0, policy_version 1667484 (0.0010) [2023-12-27 17:11:51,436][20347] Updated weights for policy 0, policy_version 1667494 (0.0008) [2023-12-27 17:11:51,680][20381] Updated weights for policy 1, policy_version 1665844 (0.0006) [2023-12-27 17:11:51,748][20381] Updated weights for policy 1, policy_version 1665854 (0.0007) [2023-12-27 17:11:51,801][20381] Updated weights for policy 1, policy_version 1665864 (0.0010) [2023-12-27 17:11:52,157][20347] Updated weights for policy 0, policy_version 1667504 (0.0006) [2023-12-27 17:11:52,202][20347] Updated weights for policy 0, policy_version 1667514 (0.0005) [2023-12-27 17:11:52,252][20347] Updated weights for policy 0, policy_version 1667524 (0.0006) [2023-12-27 17:11:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 853475328. Throughput: 0: 10291.1, 1: 10177.7. Samples: 853444004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:52,363][19170] Avg episode reward: [(0, '951.000'), (1, '951.270')] [2023-12-27 17:11:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001665872_426508288.pth... [2023-12-27 17:11:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001667528_426967040.pth... [2023-12-27 17:11:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001664688_426205184.pth [2023-12-27 17:11:52,391][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001666312_426655744.pth [2023-12-27 17:11:52,514][20381] Updated weights for policy 1, policy_version 1665874 (0.0009) [2023-12-27 17:11:52,572][20381] Updated weights for policy 1, policy_version 1665884 (0.0007) [2023-12-27 17:11:52,631][20381] Updated weights for policy 1, policy_version 1665894 (0.0008) [2023-12-27 17:11:52,690][20381] Updated weights for policy 1, policy_version 1665904 (0.0007) [2023-12-27 17:11:52,958][20347] Updated weights for policy 0, policy_version 1667534 (0.0008) [2023-12-27 17:11:53,006][20347] Updated weights for policy 0, policy_version 1667544 (0.0008) [2023-12-27 17:11:53,060][20347] Updated weights for policy 0, policy_version 1667554 (0.0007) [2023-12-27 17:11:53,354][20381] Updated weights for policy 1, policy_version 1665914 (0.0007) [2023-12-27 17:11:53,409][20381] Updated weights for policy 1, policy_version 1665924 (0.0009) [2023-12-27 17:11:53,458][20381] Updated weights for policy 1, policy_version 1665934 (0.0007) [2023-12-27 17:11:53,642][20347] Updated weights for policy 0, policy_version 1667564 (0.0007) [2023-12-27 17:11:53,703][20347] Updated weights for policy 0, policy_version 1667574 (0.0010) [2023-12-27 17:11:53,756][20347] Updated weights for policy 0, policy_version 1667584 (0.0008) [2023-12-27 17:11:54,141][20381] Updated weights for policy 1, policy_version 1665944 (0.0007) [2023-12-27 17:11:54,202][20381] Updated weights for policy 1, policy_version 1665954 (0.0008) [2023-12-27 17:11:54,265][20381] Updated weights for policy 1, policy_version 1665964 (0.0008) [2023-12-27 17:11:54,331][20347] Updated weights for policy 0, policy_version 1667594 (0.0008) [2023-12-27 17:11:54,390][20347] Updated weights for policy 0, policy_version 1667604 (0.0005) [2023-12-27 17:11:54,452][20347] Updated weights for policy 0, policy_version 1667614 (0.0007) [2023-12-27 17:11:54,506][20347] Updated weights for policy 0, policy_version 1667624 (0.0007) [2023-12-27 17:11:54,900][20381] Updated weights for policy 1, policy_version 1665974 (0.0008) [2023-12-27 17:11:54,961][20381] Updated weights for policy 1, policy_version 1665984 (0.0010) [2023-12-27 17:11:55,028][20381] Updated weights for policy 1, policy_version 1665994 (0.0011) [2023-12-27 17:11:55,154][20347] Updated weights for policy 0, policy_version 1667634 (0.0005) [2023-12-27 17:11:55,214][20347] Updated weights for policy 0, policy_version 1667644 (0.0005) [2023-12-27 17:11:55,266][20347] Updated weights for policy 0, policy_version 1667654 (0.0005) [2023-12-27 17:11:55,637][20381] Updated weights for policy 1, policy_version 1666004 (0.0008) [2023-12-27 17:11:55,698][20381] Updated weights for policy 1, policy_version 1666014 (0.0006) [2023-12-27 17:11:55,749][20381] Updated weights for policy 1, policy_version 1666024 (0.0008) [2023-12-27 17:11:55,784][20347] Updated weights for policy 0, policy_version 1667664 (0.0005) [2023-12-27 17:11:55,837][20347] Updated weights for policy 0, policy_version 1667674 (0.0005) [2023-12-27 17:11:55,895][20347] Updated weights for policy 0, policy_version 1667684 (0.0005) [2023-12-27 17:11:56,372][20381] Updated weights for policy 1, policy_version 1666034 (0.0007) [2023-12-27 17:11:56,431][20381] Updated weights for policy 1, policy_version 1666044 (0.0005) [2023-12-27 17:11:56,485][20381] Updated weights for policy 1, policy_version 1666054 (0.0007) [2023-12-27 17:11:56,536][20347] Updated weights for policy 0, policy_version 1667694 (0.0008) [2023-12-27 17:11:56,548][20381] Updated weights for policy 1, policy_version 1666064 (0.0009) [2023-12-27 17:11:56,594][20347] Updated weights for policy 0, policy_version 1667704 (0.0010) [2023-12-27 17:11:56,652][20347] Updated weights for policy 0, policy_version 1667714 (0.0010) [2023-12-27 17:11:57,157][20381] Updated weights for policy 1, policy_version 1666074 (0.0008) [2023-12-27 17:11:57,207][20381] Updated weights for policy 1, policy_version 1666084 (0.0007) [2023-12-27 17:11:57,239][20347] Updated weights for policy 0, policy_version 1667724 (0.0010) [2023-12-27 17:11:57,255][20381] Updated weights for policy 1, policy_version 1666094 (0.0006) [2023-12-27 17:11:57,289][20347] Updated weights for policy 0, policy_version 1667734 (0.0010) [2023-12-27 17:11:57,350][20347] Updated weights for policy 0, policy_version 1667744 (0.0010) [2023-12-27 17:11:57,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.4, 300 sec: 20632.7). Total num frames: 853581824. Throughput: 0: 10382.2, 1: 10224.5. Samples: 853571772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:11:57,364][19170] Avg episode reward: [(0, '931.430'), (1, '883.050')] [2023-12-27 17:11:57,848][20381] Updated weights for policy 1, policy_version 1666104 (0.0008) [2023-12-27 17:11:57,907][20381] Updated weights for policy 1, policy_version 1666114 (0.0007) [2023-12-27 17:11:57,963][20381] Updated weights for policy 1, policy_version 1666124 (0.0008) [2023-12-27 17:11:57,991][20347] Updated weights for policy 0, policy_version 1667754 (0.0009) [2023-12-27 17:11:58,056][20347] Updated weights for policy 0, policy_version 1667764 (0.0005) [2023-12-27 17:11:58,108][20347] Updated weights for policy 0, policy_version 1667774 (0.0005) [2023-12-27 17:11:58,157][20347] Updated weights for policy 0, policy_version 1667784 (0.0006) [2023-12-27 17:11:58,729][20381] Updated weights for policy 1, policy_version 1666134 (0.0008) [2023-12-27 17:11:58,798][20381] Updated weights for policy 1, policy_version 1666144 (0.0013) [2023-12-27 17:11:58,869][20381] Updated weights for policy 1, policy_version 1666154 (0.0008) [2023-12-27 17:11:58,884][20347] Updated weights for policy 0, policy_version 1667794 (0.0010) [2023-12-27 17:11:58,945][20347] Updated weights for policy 0, policy_version 1667804 (0.0007) [2023-12-27 17:11:59,007][20347] Updated weights for policy 0, policy_version 1667814 (0.0011) [2023-12-27 17:11:59,657][20381] Updated weights for policy 1, policy_version 1666164 (0.0007) [2023-12-27 17:11:59,716][20381] Updated weights for policy 1, policy_version 1666174 (0.0007) [2023-12-27 17:11:59,737][20347] Updated weights for policy 0, policy_version 1667824 (0.0011) [2023-12-27 17:11:59,777][20381] Updated weights for policy 1, policy_version 1666184 (0.0005) [2023-12-27 17:11:59,794][20347] Updated weights for policy 0, policy_version 1667834 (0.0009) [2023-12-27 17:11:59,855][20347] Updated weights for policy 0, policy_version 1667844 (0.0010) [2023-12-27 17:12:00,441][20381] Updated weights for policy 1, policy_version 1666194 (0.0007) [2023-12-27 17:12:00,479][20347] Updated weights for policy 0, policy_version 1667854 (0.0006) [2023-12-27 17:12:00,494][20381] Updated weights for policy 1, policy_version 1666204 (0.0005) [2023-12-27 17:12:00,537][20347] Updated weights for policy 0, policy_version 1667864 (0.0005) [2023-12-27 17:12:00,546][20381] Updated weights for policy 1, policy_version 1666214 (0.0007) [2023-12-27 17:12:00,588][20347] Updated weights for policy 0, policy_version 1667874 (0.0005) [2023-12-27 17:12:00,591][20381] Updated weights for policy 1, policy_version 1666224 (0.0007) [2023-12-27 17:12:01,151][20347] Updated weights for policy 0, policy_version 1667884 (0.0006) [2023-12-27 17:12:01,214][20347] Updated weights for policy 0, policy_version 1667894 (0.0007) [2023-12-27 17:12:01,270][20347] Updated weights for policy 0, policy_version 1667904 (0.0007) [2023-12-27 17:12:01,335][20381] Updated weights for policy 1, policy_version 1666234 (0.0010) [2023-12-27 17:12:01,397][20381] Updated weights for policy 1, policy_version 1666244 (0.0008) [2023-12-27 17:12:01,452][20381] Updated weights for policy 1, policy_version 1666254 (0.0005) [2023-12-27 17:12:01,977][20347] Updated weights for policy 0, policy_version 1667914 (0.0007) [2023-12-27 17:12:02,031][20347] Updated weights for policy 0, policy_version 1667924 (0.0005) [2023-12-27 17:12:02,085][20347] Updated weights for policy 0, policy_version 1667934 (0.0006) [2023-12-27 17:12:02,111][20381] Updated weights for policy 1, policy_version 1666264 (0.0010) [2023-12-27 17:12:02,141][20347] Updated weights for policy 0, policy_version 1667944 (0.0006) [2023-12-27 17:12:02,160][20381] Updated weights for policy 1, policy_version 1666274 (0.0011) [2023-12-27 17:12:02,212][20381] Updated weights for policy 1, policy_version 1666284 (0.0010) [2023-12-27 17:12:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 853688320. Throughput: 0: 10385.9, 1: 10250.5. Samples: 853695316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:02,363][19170] Avg episode reward: [(0, '960.810'), (1, '873.140')] [2023-12-27 17:12:02,793][20347] Updated weights for policy 0, policy_version 1667954 (0.0008) [2023-12-27 17:12:02,847][20347] Updated weights for policy 0, policy_version 1667964 (0.0005) [2023-12-27 17:12:02,902][20347] Updated weights for policy 0, policy_version 1667974 (0.0005) [2023-12-27 17:12:02,914][20381] Updated weights for policy 1, policy_version 1666294 (0.0009) [2023-12-27 17:12:02,959][20381] Updated weights for policy 1, policy_version 1666304 (0.0008) [2023-12-27 17:12:03,014][20381] Updated weights for policy 1, policy_version 1666314 (0.0008) [2023-12-27 17:12:03,415][20347] Updated weights for policy 0, policy_version 1667984 (0.0008) [2023-12-27 17:12:03,462][20347] Updated weights for policy 0, policy_version 1667994 (0.0010) [2023-12-27 17:12:03,510][20347] Updated weights for policy 0, policy_version 1668004 (0.0007) [2023-12-27 17:12:03,642][20381] Updated weights for policy 1, policy_version 1666324 (0.0009) [2023-12-27 17:12:03,697][20381] Updated weights for policy 1, policy_version 1666334 (0.0007) [2023-12-27 17:12:03,742][20381] Updated weights for policy 1, policy_version 1666344 (0.0005) [2023-12-27 17:12:04,081][20347] Updated weights for policy 0, policy_version 1668014 (0.0005) [2023-12-27 17:12:04,137][20347] Updated weights for policy 0, policy_version 1668024 (0.0008) [2023-12-27 17:12:04,198][20347] Updated weights for policy 0, policy_version 1668034 (0.0008) [2023-12-27 17:12:04,408][20381] Updated weights for policy 1, policy_version 1666354 (0.0006) [2023-12-27 17:12:04,463][20381] Updated weights for policy 1, policy_version 1666364 (0.0007) [2023-12-27 17:12:04,516][20381] Updated weights for policy 1, policy_version 1666374 (0.0006) [2023-12-27 17:12:04,583][20381] Updated weights for policy 1, policy_version 1666384 (0.0006) [2023-12-27 17:12:04,903][20347] Updated weights for policy 0, policy_version 1668044 (0.0007) [2023-12-27 17:12:04,960][20347] Updated weights for policy 0, policy_version 1668054 (0.0007) [2023-12-27 17:12:05,016][20347] Updated weights for policy 0, policy_version 1668064 (0.0007) [2023-12-27 17:12:05,254][20381] Updated weights for policy 1, policy_version 1666394 (0.0008) [2023-12-27 17:12:05,307][20381] Updated weights for policy 1, policy_version 1666404 (0.0009) [2023-12-27 17:12:05,355][20381] Updated weights for policy 1, policy_version 1666414 (0.0009) [2023-12-27 17:12:05,635][20347] Updated weights for policy 0, policy_version 1668074 (0.0006) [2023-12-27 17:12:05,702][20347] Updated weights for policy 0, policy_version 1668084 (0.0007) [2023-12-27 17:12:05,750][20347] Updated weights for policy 0, policy_version 1668094 (0.0006) [2023-12-27 17:12:05,802][20347] Updated weights for policy 0, policy_version 1668104 (0.0007) [2023-12-27 17:12:06,062][20381] Updated weights for policy 1, policy_version 1666424 (0.0008) [2023-12-27 17:12:06,128][20381] Updated weights for policy 1, policy_version 1666434 (0.0008) [2023-12-27 17:12:06,191][20381] Updated weights for policy 1, policy_version 1666444 (0.0008) [2023-12-27 17:12:06,503][20347] Updated weights for policy 0, policy_version 1668114 (0.0008) [2023-12-27 17:12:06,563][20347] Updated weights for policy 0, policy_version 1668124 (0.0007) [2023-12-27 17:12:06,623][20347] Updated weights for policy 0, policy_version 1668134 (0.0008) [2023-12-27 17:12:06,902][20381] Updated weights for policy 1, policy_version 1666454 (0.0006) [2023-12-27 17:12:06,950][20381] Updated weights for policy 1, policy_version 1666464 (0.0005) [2023-12-27 17:12:07,007][20381] Updated weights for policy 1, policy_version 1666474 (0.0005) [2023-12-27 17:12:07,329][20347] Updated weights for policy 0, policy_version 1668144 (0.0007) [2023-12-27 17:12:07,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 853786624. Throughput: 0: 10427.2, 1: 10310.7. Samples: 853760176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:07,363][19170] Avg episode reward: [(0, '960.810'), (1, '941.370')] [2023-12-27 17:12:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001666480_426663936.pth... [2023-12-27 17:12:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001665296_426360832.pth [2023-12-27 17:12:07,383][20347] Updated weights for policy 0, policy_version 1668154 (0.0008) [2023-12-27 17:12:07,436][20347] Updated weights for policy 0, policy_version 1668164 (0.0008) [2023-12-27 17:12:07,452][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001668168_427130880.pth... [2023-12-27 17:12:07,456][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001666920_426811392.pth [2023-12-27 17:12:07,622][20381] Updated weights for policy 1, policy_version 1666484 (0.0007) [2023-12-27 17:12:07,674][20381] Updated weights for policy 1, policy_version 1666494 (0.0007) [2023-12-27 17:12:07,724][20381] Updated weights for policy 1, policy_version 1666504 (0.0007) [2023-12-27 17:12:08,126][20347] Updated weights for policy 0, policy_version 1668174 (0.0008) [2023-12-27 17:12:08,187][20347] Updated weights for policy 0, policy_version 1668184 (0.0007) [2023-12-27 17:12:08,258][20347] Updated weights for policy 0, policy_version 1668194 (0.0007) [2023-12-27 17:12:08,402][20381] Updated weights for policy 1, policy_version 1666514 (0.0007) [2023-12-27 17:12:08,459][20381] Updated weights for policy 1, policy_version 1666524 (0.0008) [2023-12-27 17:12:08,514][20381] Updated weights for policy 1, policy_version 1666534 (0.0008) [2023-12-27 17:12:08,571][20381] Updated weights for policy 1, policy_version 1666544 (0.0008) [2023-12-27 17:12:08,964][20347] Updated weights for policy 0, policy_version 1668204 (0.0007) [2023-12-27 17:12:09,019][20347] Updated weights for policy 0, policy_version 1668214 (0.0005) [2023-12-27 17:12:09,075][20347] Updated weights for policy 0, policy_version 1668224 (0.0005) [2023-12-27 17:12:09,228][20381] Updated weights for policy 1, policy_version 1666554 (0.0008) [2023-12-27 17:12:09,282][20381] Updated weights for policy 1, policy_version 1666564 (0.0007) [2023-12-27 17:12:09,338][20381] Updated weights for policy 1, policy_version 1666574 (0.0007) [2023-12-27 17:12:09,717][20347] Updated weights for policy 0, policy_version 1668234 (0.0005) [2023-12-27 17:12:09,770][20347] Updated weights for policy 0, policy_version 1668244 (0.0006) [2023-12-27 17:12:09,830][20347] Updated weights for policy 0, policy_version 1668254 (0.0007) [2023-12-27 17:12:09,895][20347] Updated weights for policy 0, policy_version 1668264 (0.0007) [2023-12-27 17:12:10,117][20381] Updated weights for policy 1, policy_version 1666584 (0.0008) [2023-12-27 17:12:10,178][20381] Updated weights for policy 1, policy_version 1666594 (0.0009) [2023-12-27 17:12:10,232][20381] Updated weights for policy 1, policy_version 1666604 (0.0008) [2023-12-27 17:12:10,513][20347] Updated weights for policy 0, policy_version 1668274 (0.0007) [2023-12-27 17:12:10,562][20347] Updated weights for policy 0, policy_version 1668284 (0.0008) [2023-12-27 17:12:10,612][20347] Updated weights for policy 0, policy_version 1668294 (0.0007) [2023-12-27 17:12:10,969][20381] Updated weights for policy 1, policy_version 1666614 (0.0008) [2023-12-27 17:12:11,039][20381] Updated weights for policy 1, policy_version 1666624 (0.0008) [2023-12-27 17:12:11,102][20381] Updated weights for policy 1, policy_version 1666634 (0.0007) [2023-12-27 17:12:11,350][20347] Updated weights for policy 0, policy_version 1668304 (0.0007) [2023-12-27 17:12:11,425][20347] Updated weights for policy 0, policy_version 1668314 (0.0008) [2023-12-27 17:12:11,476][20347] Updated weights for policy 0, policy_version 1668324 (0.0007) [2023-12-27 17:12:11,919][20381] Updated weights for policy 1, policy_version 1666644 (0.0009) [2023-12-27 17:12:11,974][20381] Updated weights for policy 1, policy_version 1666654 (0.0008) [2023-12-27 17:12:12,028][20381] Updated weights for policy 1, policy_version 1666664 (0.0008) [2023-12-27 17:12:12,209][20347] Updated weights for policy 0, policy_version 1668334 (0.0008) [2023-12-27 17:12:12,269][20347] Updated weights for policy 0, policy_version 1668344 (0.0008) [2023-12-27 17:12:12,331][20347] Updated weights for policy 0, policy_version 1668354 (0.0007) [2023-12-27 17:12:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 853884928. Throughput: 0: 10463.2, 1: 10279.4. Samples: 853881044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:12,363][19170] Avg episode reward: [(0, '951.020'), (1, '902.350')] [2023-12-27 17:12:12,763][20381] Updated weights for policy 1, policy_version 1666674 (0.0007) [2023-12-27 17:12:12,821][20381] Updated weights for policy 1, policy_version 1666684 (0.0008) [2023-12-27 17:12:12,880][20381] Updated weights for policy 1, policy_version 1666694 (0.0007) [2023-12-27 17:12:12,927][20381] Updated weights for policy 1, policy_version 1666704 (0.0008) [2023-12-27 17:12:13,008][20347] Updated weights for policy 0, policy_version 1668364 (0.0008) [2023-12-27 17:12:13,067][20347] Updated weights for policy 0, policy_version 1668374 (0.0007) [2023-12-27 17:12:13,122][20347] Updated weights for policy 0, policy_version 1668384 (0.0007) [2023-12-27 17:12:13,590][20381] Updated weights for policy 1, policy_version 1666714 (0.0011) [2023-12-27 17:12:13,639][20381] Updated weights for policy 1, policy_version 1666724 (0.0010) [2023-12-27 17:12:13,695][20381] Updated weights for policy 1, policy_version 1666734 (0.0010) [2023-12-27 17:12:13,829][20347] Updated weights for policy 0, policy_version 1668394 (0.0007) [2023-12-27 17:12:13,892][20347] Updated weights for policy 0, policy_version 1668404 (0.0008) [2023-12-27 17:12:13,957][20347] Updated weights for policy 0, policy_version 1668414 (0.0007) [2023-12-27 17:12:14,019][20347] Updated weights for policy 0, policy_version 1668424 (0.0007) [2023-12-27 17:12:14,329][20381] Updated weights for policy 1, policy_version 1666744 (0.0006) [2023-12-27 17:12:14,374][20381] Updated weights for policy 1, policy_version 1666754 (0.0005) [2023-12-27 17:12:14,422][20381] Updated weights for policy 1, policy_version 1666764 (0.0005) [2023-12-27 17:12:14,687][20347] Updated weights for policy 0, policy_version 1668434 (0.0007) [2023-12-27 17:12:14,744][20347] Updated weights for policy 0, policy_version 1668444 (0.0007) [2023-12-27 17:12:14,801][20347] Updated weights for policy 0, policy_version 1668454 (0.0008) [2023-12-27 17:12:15,091][20381] Updated weights for policy 1, policy_version 1666774 (0.0009) [2023-12-27 17:12:15,151][20381] Updated weights for policy 1, policy_version 1666784 (0.0011) [2023-12-27 17:12:15,211][20381] Updated weights for policy 1, policy_version 1666794 (0.0011) [2023-12-27 17:12:15,541][20347] Updated weights for policy 0, policy_version 1668464 (0.0007) [2023-12-27 17:12:15,589][20347] Updated weights for policy 0, policy_version 1668474 (0.0007) [2023-12-27 17:12:15,643][20347] Updated weights for policy 0, policy_version 1668484 (0.0007) [2023-12-27 17:12:15,888][20381] Updated weights for policy 1, policy_version 1666804 (0.0011) [2023-12-27 17:12:15,949][20381] Updated weights for policy 1, policy_version 1666814 (0.0010) [2023-12-27 17:12:16,003][20381] Updated weights for policy 1, policy_version 1666824 (0.0010) [2023-12-27 17:12:16,356][20347] Updated weights for policy 0, policy_version 1668494 (0.0008) [2023-12-27 17:12:16,410][20347] Updated weights for policy 0, policy_version 1668504 (0.0007) [2023-12-27 17:12:16,474][20347] Updated weights for policy 0, policy_version 1668514 (0.0007) [2023-12-27 17:12:16,668][20381] Updated weights for policy 1, policy_version 1666834 (0.0010) [2023-12-27 17:12:16,726][20381] Updated weights for policy 1, policy_version 1666844 (0.0008) [2023-12-27 17:12:16,781][20381] Updated weights for policy 1, policy_version 1666854 (0.0008) [2023-12-27 17:12:16,837][20381] Updated weights for policy 1, policy_version 1666864 (0.0007) [2023-12-27 17:12:17,185][20347] Updated weights for policy 0, policy_version 1668524 (0.0008) [2023-12-27 17:12:17,230][20347] Updated weights for policy 0, policy_version 1668534 (0.0008) [2023-12-27 17:12:17,285][20347] Updated weights for policy 0, policy_version 1668544 (0.0008) [2023-12-27 17:12:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 853991424. Throughput: 0: 10425.3, 1: 10171.3. Samples: 854000516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:17,363][19170] Avg episode reward: [(0, '960.820'), (1, '853.780')] [2023-12-27 17:12:17,500][20381] Updated weights for policy 1, policy_version 1666874 (0.0008) [2023-12-27 17:12:17,550][20381] Updated weights for policy 1, policy_version 1666884 (0.0008) [2023-12-27 17:12:17,602][20381] Updated weights for policy 1, policy_version 1666894 (0.0007) [2023-12-27 17:12:17,957][20347] Updated weights for policy 0, policy_version 1668554 (0.0007) [2023-12-27 17:12:18,012][20347] Updated weights for policy 0, policy_version 1668564 (0.0008) [2023-12-27 17:12:18,066][20347] Updated weights for policy 0, policy_version 1668574 (0.0008) [2023-12-27 17:12:18,119][20347] Updated weights for policy 0, policy_version 1668584 (0.0008) [2023-12-27 17:12:18,309][20381] Updated weights for policy 1, policy_version 1666904 (0.0008) [2023-12-27 17:12:18,366][20381] Updated weights for policy 1, policy_version 1666914 (0.0007) [2023-12-27 17:12:18,412][20381] Updated weights for policy 1, policy_version 1666924 (0.0007) [2023-12-27 17:12:18,816][20347] Updated weights for policy 0, policy_version 1668594 (0.0008) [2023-12-27 17:12:18,869][20347] Updated weights for policy 0, policy_version 1668604 (0.0010) [2023-12-27 17:12:18,919][20347] Updated weights for policy 0, policy_version 1668614 (0.0009) [2023-12-27 17:12:19,176][20381] Updated weights for policy 1, policy_version 1666934 (0.0009) [2023-12-27 17:12:19,260][20381] Updated weights for policy 1, policy_version 1666944 (0.0007) [2023-12-27 17:12:19,320][20381] Updated weights for policy 1, policy_version 1666954 (0.0007) [2023-12-27 17:12:19,581][20347] Updated weights for policy 0, policy_version 1668624 (0.0009) [2023-12-27 17:12:19,649][20347] Updated weights for policy 0, policy_version 1668634 (0.0007) [2023-12-27 17:12:19,714][20347] Updated weights for policy 0, policy_version 1668644 (0.0008) [2023-12-27 17:12:20,050][20381] Updated weights for policy 1, policy_version 1666964 (0.0008) [2023-12-27 17:12:20,108][20381] Updated weights for policy 1, policy_version 1666974 (0.0009) [2023-12-27 17:12:20,157][20381] Updated weights for policy 1, policy_version 1666984 (0.0007) [2023-12-27 17:12:20,353][20347] Updated weights for policy 0, policy_version 1668654 (0.0008) [2023-12-27 17:12:20,364][20291] KL-divergence is very high: 136.1086 [2023-12-27 17:12:20,377][20291] KL-divergence is very high: 121.9203 [2023-12-27 17:12:20,392][20291] KL-divergence is very high: 155.8640 [2023-12-27 17:12:20,414][20347] Updated weights for policy 0, policy_version 1668664 (0.0006) [2023-12-27 17:12:20,417][20291] KL-divergence is very high: 159.1131 [2023-12-27 17:12:20,429][20291] KL-divergence is very high: 121.1375 [2023-12-27 17:12:20,482][20347] Updated weights for policy 0, policy_version 1668674 (0.0005) [2023-12-27 17:12:20,775][20381] Updated weights for policy 1, policy_version 1666994 (0.0006) [2023-12-27 17:12:20,835][20381] Updated weights for policy 1, policy_version 1667004 (0.0008) [2023-12-27 17:12:20,896][20381] Updated weights for policy 1, policy_version 1667014 (0.0008) [2023-12-27 17:12:20,960][20381] Updated weights for policy 1, policy_version 1667024 (0.0007) [2023-12-27 17:12:21,134][20347] Updated weights for policy 0, policy_version 1668684 (0.0008) [2023-12-27 17:12:21,197][20347] Updated weights for policy 0, policy_version 1668694 (0.0007) [2023-12-27 17:12:21,265][20347] Updated weights for policy 0, policy_version 1668704 (0.0006) [2023-12-27 17:12:21,711][20381] Updated weights for policy 1, policy_version 1667034 (0.0008) [2023-12-27 17:12:21,776][20381] Updated weights for policy 1, policy_version 1667044 (0.0008) [2023-12-27 17:12:21,827][20381] Updated weights for policy 1, policy_version 1667054 (0.0009) [2023-12-27 17:12:21,954][20347] Updated weights for policy 0, policy_version 1668714 (0.0007) [2023-12-27 17:12:22,008][20347] Updated weights for policy 0, policy_version 1668724 (0.0010) [2023-12-27 17:12:22,066][20347] Updated weights for policy 0, policy_version 1668734 (0.0010) [2023-12-27 17:12:22,120][20347] Updated weights for policy 0, policy_version 1668744 (0.0011) [2023-12-27 17:12:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 854089728. Throughput: 0: 10403.8, 1: 10160.1. Samples: 854060092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:22,363][19170] Avg episode reward: [(0, '941.280'), (1, '892.880')] [2023-12-27 17:12:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001667056_426811392.pth... [2023-12-27 17:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001668744_427278336.pth... [2023-12-27 17:12:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001665872_426508288.pth [2023-12-27 17:12:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001667528_426967040.pth [2023-12-27 17:12:22,531][20381] Updated weights for policy 1, policy_version 1667064 (0.0008) [2023-12-27 17:12:22,586][20381] Updated weights for policy 1, policy_version 1667074 (0.0008) [2023-12-27 17:12:22,645][20381] Updated weights for policy 1, policy_version 1667084 (0.0008) [2023-12-27 17:12:22,841][20347] Updated weights for policy 0, policy_version 1668754 (0.0007) [2023-12-27 17:12:22,912][20347] Updated weights for policy 0, policy_version 1668764 (0.0008) [2023-12-27 17:12:22,964][20347] Updated weights for policy 0, policy_version 1668774 (0.0008) [2023-12-27 17:12:23,335][20381] Updated weights for policy 1, policy_version 1667094 (0.0008) [2023-12-27 17:12:23,397][20381] Updated weights for policy 1, policy_version 1667104 (0.0008) [2023-12-27 17:12:23,448][20381] Updated weights for policy 1, policy_version 1667114 (0.0007) [2023-12-27 17:12:23,640][20347] Updated weights for policy 0, policy_version 1668784 (0.0008) [2023-12-27 17:12:23,700][20347] Updated weights for policy 0, policy_version 1668794 (0.0007) [2023-12-27 17:12:23,763][20347] Updated weights for policy 0, policy_version 1668804 (0.0008) [2023-12-27 17:12:24,082][20381] Updated weights for policy 1, policy_version 1667124 (0.0007) [2023-12-27 17:12:24,138][20381] Updated weights for policy 1, policy_version 1667134 (0.0006) [2023-12-27 17:12:24,189][20381] Updated weights for policy 1, policy_version 1667144 (0.0008) [2023-12-27 17:12:24,426][20347] Updated weights for policy 0, policy_version 1668814 (0.0008) [2023-12-27 17:12:24,487][20347] Updated weights for policy 0, policy_version 1668824 (0.0006) [2023-12-27 17:12:24,554][20347] Updated weights for policy 0, policy_version 1668834 (0.0005) [2023-12-27 17:12:24,873][20381] Updated weights for policy 1, policy_version 1667154 (0.0009) [2023-12-27 17:12:24,927][20381] Updated weights for policy 1, policy_version 1667164 (0.0010) [2023-12-27 17:12:24,987][20381] Updated weights for policy 1, policy_version 1667174 (0.0010) [2023-12-27 17:12:25,039][20381] Updated weights for policy 1, policy_version 1667184 (0.0010) [2023-12-27 17:12:25,195][20347] Updated weights for policy 0, policy_version 1668844 (0.0006) [2023-12-27 17:12:25,261][20347] Updated weights for policy 0, policy_version 1668854 (0.0005) [2023-12-27 17:12:25,314][20347] Updated weights for policy 0, policy_version 1668864 (0.0006) [2023-12-27 17:12:25,679][20381] Updated weights for policy 1, policy_version 1667194 (0.0010) [2023-12-27 17:12:25,724][20381] Updated weights for policy 1, policy_version 1667204 (0.0010) [2023-12-27 17:12:25,772][20381] Updated weights for policy 1, policy_version 1667214 (0.0010) [2023-12-27 17:12:25,944][20347] Updated weights for policy 0, policy_version 1668874 (0.0009) [2023-12-27 17:12:26,005][20347] Updated weights for policy 0, policy_version 1668884 (0.0005) [2023-12-27 17:12:26,061][20347] Updated weights for policy 0, policy_version 1668894 (0.0006) [2023-12-27 17:12:26,116][20347] Updated weights for policy 0, policy_version 1668904 (0.0007) [2023-12-27 17:12:26,492][20381] Updated weights for policy 1, policy_version 1667224 (0.0006) [2023-12-27 17:12:26,548][20381] Updated weights for policy 1, policy_version 1667234 (0.0006) [2023-12-27 17:12:26,601][20381] Updated weights for policy 1, policy_version 1667244 (0.0010) [2023-12-27 17:12:26,697][20347] Updated weights for policy 0, policy_version 1668914 (0.0005) [2023-12-27 17:12:26,740][20347] Updated weights for policy 0, policy_version 1668924 (0.0005) [2023-12-27 17:12:26,790][20347] Updated weights for policy 0, policy_version 1668934 (0.0005) [2023-12-27 17:12:27,220][20381] Updated weights for policy 1, policy_version 1667254 (0.0010) [2023-12-27 17:12:27,264][20381] Updated weights for policy 1, policy_version 1667264 (0.0010) [2023-12-27 17:12:27,313][20347] Updated weights for policy 0, policy_version 1668944 (0.0005) [2023-12-27 17:12:27,317][20381] Updated weights for policy 1, policy_version 1667274 (0.0008) [2023-12-27 17:12:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 854196224. Throughput: 0: 10415.9, 1: 10147.3. Samples: 854183884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:12:27,363][19170] Avg episode reward: [(0, '960.850'), (1, '912.280')] [2023-12-27 17:12:27,365][20347] Updated weights for policy 0, policy_version 1668954 (0.0007) [2023-12-27 17:12:27,420][20347] Updated weights for policy 0, policy_version 1668964 (0.0006) [2023-12-27 17:12:27,949][20381] Updated weights for policy 1, policy_version 1667284 (0.0007) [2023-12-27 17:12:28,000][20381] Updated weights for policy 1, policy_version 1667294 (0.0007) [2023-12-27 17:12:28,052][20381] Updated weights for policy 1, policy_version 1667304 (0.0010) [2023-12-27 17:12:28,068][20347] Updated weights for policy 0, policy_version 1668974 (0.0010) [2023-12-27 17:12:28,124][20347] Updated weights for policy 0, policy_version 1668984 (0.0010) [2023-12-27 17:12:28,176][20347] Updated weights for policy 0, policy_version 1668994 (0.0010) [2023-12-27 17:12:28,743][20381] Updated weights for policy 1, policy_version 1667314 (0.0011) [2023-12-27 17:12:28,794][20381] Updated weights for policy 1, policy_version 1667324 (0.0010) [2023-12-27 17:12:28,853][20381] Updated weights for policy 1, policy_version 1667334 (0.0008) [2023-12-27 17:12:28,893][20347] Updated weights for policy 0, policy_version 1669004 (0.0009) [2023-12-27 17:12:28,909][20381] Updated weights for policy 1, policy_version 1667344 (0.0006) [2023-12-27 17:12:28,943][20347] Updated weights for policy 0, policy_version 1669014 (0.0008) [2023-12-27 17:12:29,001][20347] Updated weights for policy 0, policy_version 1669024 (0.0007) [2023-12-27 17:12:29,554][20381] Updated weights for policy 1, policy_version 1667354 (0.0006) [2023-12-27 17:12:29,604][20381] Updated weights for policy 1, policy_version 1667364 (0.0005) [2023-12-27 17:12:29,654][20381] Updated weights for policy 1, policy_version 1667374 (0.0005) [2023-12-27 17:12:29,768][20347] Updated weights for policy 0, policy_version 1669034 (0.0007) [2023-12-27 17:12:29,826][20347] Updated weights for policy 0, policy_version 1669044 (0.0007) [2023-12-27 17:12:29,884][20347] Updated weights for policy 0, policy_version 1669054 (0.0008) [2023-12-27 17:12:29,932][20347] Updated weights for policy 0, policy_version 1669064 (0.0008) [2023-12-27 17:12:30,284][20381] Updated weights for policy 1, policy_version 1667384 (0.0007) [2023-12-27 17:12:30,338][20381] Updated weights for policy 1, policy_version 1667394 (0.0008) [2023-12-27 17:12:30,401][20381] Updated weights for policy 1, policy_version 1667404 (0.0008) [2023-12-27 17:12:30,669][20347] Updated weights for policy 0, policy_version 1669074 (0.0007) [2023-12-27 17:12:30,717][20347] Updated weights for policy 0, policy_version 1669084 (0.0008) [2023-12-27 17:12:30,763][20347] Updated weights for policy 0, policy_version 1669094 (0.0007) [2023-12-27 17:12:31,062][20381] Updated weights for policy 1, policy_version 1667414 (0.0008) [2023-12-27 17:12:31,114][20381] Updated weights for policy 1, policy_version 1667424 (0.0006) [2023-12-27 17:12:31,177][20381] Updated weights for policy 1, policy_version 1667434 (0.0009) [2023-12-27 17:12:31,436][20347] Updated weights for policy 0, policy_version 1669104 (0.0008) [2023-12-27 17:12:31,485][20347] Updated weights for policy 0, policy_version 1669114 (0.0007) [2023-12-27 17:12:31,530][20347] Updated weights for policy 0, policy_version 1669124 (0.0007) [2023-12-27 17:12:31,863][20381] Updated weights for policy 1, policy_version 1667444 (0.0007) [2023-12-27 17:12:31,915][20381] Updated weights for policy 1, policy_version 1667454 (0.0007) [2023-12-27 17:12:31,974][20381] Updated weights for policy 1, policy_version 1667464 (0.0007) [2023-12-27 17:12:32,256][20347] Updated weights for policy 0, policy_version 1669134 (0.0007) [2023-12-27 17:12:32,310][20347] Updated weights for policy 0, policy_version 1669144 (0.0009) [2023-12-27 17:12:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 854294528. Throughput: 0: 10439.4, 1: 10181.8. Samples: 854309636. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:32,363][19170] Avg episode reward: [(0, '990.200'), (1, '844.010')] [2023-12-27 17:12:32,367][20347] Updated weights for policy 0, policy_version 1669154 (0.0008) [2023-12-27 17:12:32,682][20381] Updated weights for policy 1, policy_version 1667474 (0.0008) [2023-12-27 17:12:32,736][20381] Updated weights for policy 1, policy_version 1667484 (0.0007) [2023-12-27 17:12:32,790][20381] Updated weights for policy 1, policy_version 1667494 (0.0010) [2023-12-27 17:12:32,847][20381] Updated weights for policy 1, policy_version 1667504 (0.0009) [2023-12-27 17:12:32,997][20347] Updated weights for policy 0, policy_version 1669164 (0.0008) [2023-12-27 17:12:33,063][20347] Updated weights for policy 0, policy_version 1669174 (0.0005) [2023-12-27 17:12:33,125][20347] Updated weights for policy 0, policy_version 1669184 (0.0006) [2023-12-27 17:12:33,545][20381] Updated weights for policy 1, policy_version 1667514 (0.0008) [2023-12-27 17:12:33,601][20381] Updated weights for policy 1, policy_version 1667524 (0.0006) [2023-12-27 17:12:33,646][20381] Updated weights for policy 1, policy_version 1667534 (0.0006) [2023-12-27 17:12:33,671][20347] Updated weights for policy 0, policy_version 1669194 (0.0006) [2023-12-27 17:12:33,724][20347] Updated weights for policy 0, policy_version 1669204 (0.0010) [2023-12-27 17:12:33,774][20347] Updated weights for policy 0, policy_version 1669214 (0.0010) [2023-12-27 17:12:33,822][20347] Updated weights for policy 0, policy_version 1669224 (0.0010) [2023-12-27 17:12:34,277][20381] Updated weights for policy 1, policy_version 1667544 (0.0008) [2023-12-27 17:12:34,336][20381] Updated weights for policy 1, policy_version 1667554 (0.0007) [2023-12-27 17:12:34,392][20381] Updated weights for policy 1, policy_version 1667564 (0.0008) [2023-12-27 17:12:34,512][20347] Updated weights for policy 0, policy_version 1669234 (0.0008) [2023-12-27 17:12:34,571][20347] Updated weights for policy 0, policy_version 1669244 (0.0008) [2023-12-27 17:12:34,634][20347] Updated weights for policy 0, policy_version 1669254 (0.0008) [2023-12-27 17:12:35,082][20381] Updated weights for policy 1, policy_version 1667574 (0.0006) [2023-12-27 17:12:35,142][20381] Updated weights for policy 1, policy_version 1667584 (0.0005) [2023-12-27 17:12:35,196][20381] Updated weights for policy 1, policy_version 1667594 (0.0005) [2023-12-27 17:12:35,348][20347] Updated weights for policy 0, policy_version 1669264 (0.0009) [2023-12-27 17:12:35,397][20347] Updated weights for policy 0, policy_version 1669274 (0.0011) [2023-12-27 17:12:35,452][20347] Updated weights for policy 0, policy_version 1669284 (0.0010) [2023-12-27 17:12:35,730][20381] Updated weights for policy 1, policy_version 1667604 (0.0005) [2023-12-27 17:12:35,782][20381] Updated weights for policy 1, policy_version 1667614 (0.0006) [2023-12-27 17:12:35,833][20381] Updated weights for policy 1, policy_version 1667624 (0.0008) [2023-12-27 17:12:36,083][20347] Updated weights for policy 0, policy_version 1669294 (0.0008) [2023-12-27 17:12:36,148][20347] Updated weights for policy 0, policy_version 1669304 (0.0007) [2023-12-27 17:12:36,205][20347] Updated weights for policy 0, policy_version 1669314 (0.0008) [2023-12-27 17:12:36,501][20381] Updated weights for policy 1, policy_version 1667634 (0.0008) [2023-12-27 17:12:36,565][20381] Updated weights for policy 1, policy_version 1667644 (0.0009) [2023-12-27 17:12:36,633][20381] Updated weights for policy 1, policy_version 1667654 (0.0008) [2023-12-27 17:12:36,701][20381] Updated weights for policy 1, policy_version 1667664 (0.0008) [2023-12-27 17:12:36,920][20347] Updated weights for policy 0, policy_version 1669324 (0.0009) [2023-12-27 17:12:36,975][20347] Updated weights for policy 0, policy_version 1669334 (0.0010) [2023-12-27 17:12:37,032][20347] Updated weights for policy 0, policy_version 1669344 (0.0010) [2023-12-27 17:12:37,352][20381] Updated weights for policy 1, policy_version 1667674 (0.0006) [2023-12-27 17:12:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 854401024. Throughput: 0: 10416.0, 1: 10212.3. Samples: 854372276. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:37,363][19170] Avg episode reward: [(0, '951.060'), (1, '883.170')] [2023-12-27 17:12:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001669352_427433984.pth... [2023-12-27 17:12:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001668168_427130880.pth [2023-12-27 17:12:37,414][20381] Updated weights for policy 1, policy_version 1667684 (0.0008) [2023-12-27 17:12:37,469][20381] Updated weights for policy 1, policy_version 1667694 (0.0005) [2023-12-27 17:12:37,481][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001667696_426975232.pth... [2023-12-27 17:12:37,486][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001666480_426663936.pth [2023-12-27 17:12:37,679][20347] Updated weights for policy 0, policy_version 1669354 (0.0009) [2023-12-27 17:12:37,737][20347] Updated weights for policy 0, policy_version 1669364 (0.0005) [2023-12-27 17:12:37,798][20347] Updated weights for policy 0, policy_version 1669374 (0.0006) [2023-12-27 17:12:37,857][20347] Updated weights for policy 0, policy_version 1669384 (0.0006) [2023-12-27 17:12:38,066][20381] Updated weights for policy 1, policy_version 1667704 (0.0008) [2023-12-27 17:12:38,130][20381] Updated weights for policy 1, policy_version 1667714 (0.0009) [2023-12-27 17:12:38,189][20381] Updated weights for policy 1, policy_version 1667724 (0.0011) [2023-12-27 17:12:38,462][20347] Updated weights for policy 0, policy_version 1669394 (0.0006) [2023-12-27 17:12:38,516][20347] Updated weights for policy 0, policy_version 1669404 (0.0006) [2023-12-27 17:12:38,570][20347] Updated weights for policy 0, policy_version 1669414 (0.0006) [2023-12-27 17:12:38,878][20381] Updated weights for policy 1, policy_version 1667734 (0.0007) [2023-12-27 17:12:38,942][20381] Updated weights for policy 1, policy_version 1667744 (0.0006) [2023-12-27 17:12:38,992][20381] Updated weights for policy 1, policy_version 1667754 (0.0007) [2023-12-27 17:12:39,196][20347] Updated weights for policy 0, policy_version 1669424 (0.0005) [2023-12-27 17:12:39,261][20347] Updated weights for policy 0, policy_version 1669434 (0.0007) [2023-12-27 17:12:39,308][20347] Updated weights for policy 0, policy_version 1669444 (0.0008) [2023-12-27 17:12:39,638][20381] Updated weights for policy 1, policy_version 1667764 (0.0007) [2023-12-27 17:12:39,695][20381] Updated weights for policy 1, policy_version 1667774 (0.0005) [2023-12-27 17:12:39,745][20381] Updated weights for policy 1, policy_version 1667784 (0.0005) [2023-12-27 17:12:40,043][20347] Updated weights for policy 0, policy_version 1669454 (0.0008) [2023-12-27 17:12:40,099][20347] Updated weights for policy 0, policy_version 1669464 (0.0008) [2023-12-27 17:12:40,160][20347] Updated weights for policy 0, policy_version 1669474 (0.0007) [2023-12-27 17:12:40,441][20381] Updated weights for policy 1, policy_version 1667794 (0.0007) [2023-12-27 17:12:40,506][20381] Updated weights for policy 1, policy_version 1667804 (0.0008) [2023-12-27 17:12:40,563][20381] Updated weights for policy 1, policy_version 1667814 (0.0008) [2023-12-27 17:12:40,618][20381] Updated weights for policy 1, policy_version 1667824 (0.0008) [2023-12-27 17:12:40,851][20347] Updated weights for policy 0, policy_version 1669484 (0.0008) [2023-12-27 17:12:40,904][20347] Updated weights for policy 0, policy_version 1669494 (0.0006) [2023-12-27 17:12:40,957][20347] Updated weights for policy 0, policy_version 1669504 (0.0005) [2023-12-27 17:12:41,343][20381] Updated weights for policy 1, policy_version 1667834 (0.0008) [2023-12-27 17:12:41,431][20381] Updated weights for policy 1, policy_version 1667844 (0.0009) [2023-12-27 17:12:41,498][20381] Updated weights for policy 1, policy_version 1667854 (0.0007) [2023-12-27 17:12:41,691][20347] Updated weights for policy 0, policy_version 1669514 (0.0006) [2023-12-27 17:12:41,760][20347] Updated weights for policy 0, policy_version 1669524 (0.0008) [2023-12-27 17:12:41,814][20347] Updated weights for policy 0, policy_version 1669534 (0.0007) [2023-12-27 17:12:41,871][20347] Updated weights for policy 0, policy_version 1669544 (0.0007) [2023-12-27 17:12:42,276][20381] Updated weights for policy 1, policy_version 1667864 (0.0008) [2023-12-27 17:12:42,334][20381] Updated weights for policy 1, policy_version 1667874 (0.0007) [2023-12-27 17:12:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 854499328. Throughput: 0: 10350.4, 1: 10205.1. Samples: 854496760. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:42,363][19170] Avg episode reward: [(0, '941.250'), (1, '873.560')] [2023-12-27 17:12:42,398][20381] Updated weights for policy 1, policy_version 1667884 (0.0008) [2023-12-27 17:12:42,509][20291] KL-divergence is very high: 118.7672 [2023-12-27 17:12:42,519][20347] Updated weights for policy 0, policy_version 1669554 (0.0009) [2023-12-27 17:12:42,569][20347] Updated weights for policy 0, policy_version 1669564 (0.0008) [2023-12-27 17:12:42,621][20347] Updated weights for policy 0, policy_version 1669574 (0.0005) [2023-12-27 17:12:43,065][20381] Updated weights for policy 1, policy_version 1667894 (0.0008) [2023-12-27 17:12:43,108][20381] Updated weights for policy 1, policy_version 1667904 (0.0007) [2023-12-27 17:12:43,164][20381] Updated weights for policy 1, policy_version 1667914 (0.0007) [2023-12-27 17:12:43,231][20347] Updated weights for policy 0, policy_version 1669584 (0.0008) [2023-12-27 17:12:43,287][20347] Updated weights for policy 0, policy_version 1669594 (0.0007) [2023-12-27 17:12:43,313][20291] KL-divergence is very high: 126.8174 [2023-12-27 17:12:43,341][20347] Updated weights for policy 0, policy_version 1669604 (0.0008) [2023-12-27 17:12:43,359][20291] KL-divergence is very high: 160.9987 [2023-12-27 17:12:43,922][20381] Updated weights for policy 1, policy_version 1667924 (0.0008) [2023-12-27 17:12:43,971][20381] Updated weights for policy 1, policy_version 1667934 (0.0007) [2023-12-27 17:12:44,010][20347] Updated weights for policy 0, policy_version 1669614 (0.0006) [2023-12-27 17:12:44,027][20381] Updated weights for policy 1, policy_version 1667944 (0.0007) [2023-12-27 17:12:44,056][20347] Updated weights for policy 0, policy_version 1669624 (0.0007) [2023-12-27 17:12:44,116][20347] Updated weights for policy 0, policy_version 1669634 (0.0007) [2023-12-27 17:12:44,723][20381] Updated weights for policy 1, policy_version 1667954 (0.0007) [2023-12-27 17:12:44,785][20381] Updated weights for policy 1, policy_version 1667964 (0.0008) [2023-12-27 17:12:44,805][20347] Updated weights for policy 0, policy_version 1669644 (0.0008) [2023-12-27 17:12:44,842][20381] Updated weights for policy 1, policy_version 1667974 (0.0007) [2023-12-27 17:12:44,871][20347] Updated weights for policy 0, policy_version 1669654 (0.0006) [2023-12-27 17:12:44,896][20381] Updated weights for policy 1, policy_version 1667984 (0.0006) [2023-12-27 17:12:44,932][20347] Updated weights for policy 0, policy_version 1669664 (0.0008) [2023-12-27 17:12:45,597][20381] Updated weights for policy 1, policy_version 1667994 (0.0008) [2023-12-27 17:12:45,623][20347] Updated weights for policy 0, policy_version 1669674 (0.0008) [2023-12-27 17:12:45,643][20381] Updated weights for policy 1, policy_version 1668004 (0.0007) [2023-12-27 17:12:45,682][20347] Updated weights for policy 0, policy_version 1669684 (0.0007) [2023-12-27 17:12:45,695][20381] Updated weights for policy 1, policy_version 1668014 (0.0005) [2023-12-27 17:12:45,734][20347] Updated weights for policy 0, policy_version 1669694 (0.0006) [2023-12-27 17:12:45,794][20347] Updated weights for policy 0, policy_version 1669704 (0.0007) [2023-12-27 17:12:46,384][20381] Updated weights for policy 1, policy_version 1668024 (0.0008) [2023-12-27 17:12:46,451][20381] Updated weights for policy 1, policy_version 1668034 (0.0006) [2023-12-27 17:12:46,502][20347] Updated weights for policy 0, policy_version 1669714 (0.0006) [2023-12-27 17:12:46,515][20381] Updated weights for policy 1, policy_version 1668044 (0.0007) [2023-12-27 17:12:46,565][20347] Updated weights for policy 0, policy_version 1669724 (0.0006) [2023-12-27 17:12:46,622][20347] Updated weights for policy 0, policy_version 1669734 (0.0008) [2023-12-27 17:12:47,196][20381] Updated weights for policy 1, policy_version 1668054 (0.0006) [2023-12-27 17:12:47,254][20381] Updated weights for policy 1, policy_version 1668064 (0.0008) [2023-12-27 17:12:47,266][20347] Updated weights for policy 0, policy_version 1669744 (0.0006) [2023-12-27 17:12:47,297][20381] Updated weights for policy 1, policy_version 1668074 (0.0007) [2023-12-27 17:12:47,316][20347] Updated weights for policy 0, policy_version 1669754 (0.0005) [2023-12-27 17:12:47,362][20347] Updated weights for policy 0, policy_version 1669764 (0.0005) [2023-12-27 17:12:47,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 854605824. Throughput: 0: 10309.5, 1: 10172.8. Samples: 854617028. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:47,364][19170] Avg episode reward: [(0, '921.690'), (1, '912.400')] [2023-12-27 17:12:47,928][20347] Updated weights for policy 0, policy_version 1669774 (0.0005) [2023-12-27 17:12:47,981][20347] Updated weights for policy 0, policy_version 1669784 (0.0005) [2023-12-27 17:12:48,036][20381] Updated weights for policy 1, policy_version 1668084 (0.0007) [2023-12-27 17:12:48,038][20347] Updated weights for policy 0, policy_version 1669794 (0.0007) [2023-12-27 17:12:48,091][20381] Updated weights for policy 1, policy_version 1668094 (0.0007) [2023-12-27 17:12:48,136][20381] Updated weights for policy 1, policy_version 1668104 (0.0008) [2023-12-27 17:12:48,679][20347] Updated weights for policy 0, policy_version 1669804 (0.0006) [2023-12-27 17:12:48,734][20347] Updated weights for policy 0, policy_version 1669814 (0.0007) [2023-12-27 17:12:48,791][20347] Updated weights for policy 0, policy_version 1669824 (0.0008) [2023-12-27 17:12:48,855][20381] Updated weights for policy 1, policy_version 1668114 (0.0007) [2023-12-27 17:12:48,915][20381] Updated weights for policy 1, policy_version 1668124 (0.0007) [2023-12-27 17:12:48,972][20381] Updated weights for policy 1, policy_version 1668134 (0.0007) [2023-12-27 17:12:49,028][20381] Updated weights for policy 1, policy_version 1668144 (0.0008) [2023-12-27 17:12:49,421][20347] Updated weights for policy 0, policy_version 1669834 (0.0006) [2023-12-27 17:12:49,477][20347] Updated weights for policy 0, policy_version 1669844 (0.0007) [2023-12-27 17:12:49,534][20347] Updated weights for policy 0, policy_version 1669854 (0.0006) [2023-12-27 17:12:49,592][20347] Updated weights for policy 0, policy_version 1669864 (0.0006) [2023-12-27 17:12:49,766][20381] Updated weights for policy 1, policy_version 1668154 (0.0005) [2023-12-27 17:12:49,832][20381] Updated weights for policy 1, policy_version 1668164 (0.0006) [2023-12-27 17:12:49,899][20381] Updated weights for policy 1, policy_version 1668174 (0.0007) [2023-12-27 17:12:50,244][20347] Updated weights for policy 0, policy_version 1669874 (0.0008) [2023-12-27 17:12:50,306][20347] Updated weights for policy 0, policy_version 1669884 (0.0008) [2023-12-27 17:12:50,362][20347] Updated weights for policy 0, policy_version 1669894 (0.0007) [2023-12-27 17:12:50,502][20381] Updated weights for policy 1, policy_version 1668184 (0.0008) [2023-12-27 17:12:50,561][20381] Updated weights for policy 1, policy_version 1668194 (0.0007) [2023-12-27 17:12:50,624][20381] Updated weights for policy 1, policy_version 1668204 (0.0007) [2023-12-27 17:12:50,979][20347] Updated weights for policy 0, policy_version 1669904 (0.0008) [2023-12-27 17:12:51,046][20347] Updated weights for policy 0, policy_version 1669914 (0.0008) [2023-12-27 17:12:51,103][20347] Updated weights for policy 0, policy_version 1669924 (0.0008) [2023-12-27 17:12:51,313][20381] Updated weights for policy 1, policy_version 1668214 (0.0008) [2023-12-27 17:12:51,378][20381] Updated weights for policy 1, policy_version 1668224 (0.0009) [2023-12-27 17:12:51,434][20381] Updated weights for policy 1, policy_version 1668234 (0.0011) [2023-12-27 17:12:51,860][20347] Updated weights for policy 0, policy_version 1669934 (0.0008) [2023-12-27 17:12:51,919][20347] Updated weights for policy 0, policy_version 1669944 (0.0008) [2023-12-27 17:12:51,987][20347] Updated weights for policy 0, policy_version 1669954 (0.0007) [2023-12-27 17:12:52,136][20381] Updated weights for policy 1, policy_version 1668244 (0.0011) [2023-12-27 17:12:52,187][20381] Updated weights for policy 1, policy_version 1668254 (0.0010) [2023-12-27 17:12:52,240][20381] Updated weights for policy 1, policy_version 1668264 (0.0010) [2023-12-27 17:12:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 854712320. Throughput: 0: 10301.0, 1: 10128.6. Samples: 854679512. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:52,363][19170] Avg episode reward: [(0, '941.250'), (1, '951.380')] [2023-12-27 17:12:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001668272_427122688.pth... [2023-12-27 17:12:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001669960_427589632.pth... [2023-12-27 17:12:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001668744_427278336.pth [2023-12-27 17:12:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001667056_426811392.pth [2023-12-27 17:12:52,664][20347] Updated weights for policy 0, policy_version 1669964 (0.0008) [2023-12-27 17:12:52,730][20347] Updated weights for policy 0, policy_version 1669974 (0.0008) [2023-12-27 17:12:52,789][20347] Updated weights for policy 0, policy_version 1669984 (0.0006) [2023-12-27 17:12:52,872][20381] Updated weights for policy 1, policy_version 1668274 (0.0010) [2023-12-27 17:12:52,937][20381] Updated weights for policy 1, policy_version 1668284 (0.0006) [2023-12-27 17:12:52,994][20381] Updated weights for policy 1, policy_version 1668294 (0.0007) [2023-12-27 17:12:53,039][20381] Updated weights for policy 1, policy_version 1668304 (0.0007) [2023-12-27 17:12:53,443][20347] Updated weights for policy 0, policy_version 1669994 (0.0006) [2023-12-27 17:12:53,494][20347] Updated weights for policy 0, policy_version 1670004 (0.0010) [2023-12-27 17:12:53,539][20347] Updated weights for policy 0, policy_version 1670014 (0.0010) [2023-12-27 17:12:53,597][20347] Updated weights for policy 0, policy_version 1670024 (0.0010) [2023-12-27 17:12:53,707][20381] Updated weights for policy 1, policy_version 1668314 (0.0008) [2023-12-27 17:12:53,756][20381] Updated weights for policy 1, policy_version 1668324 (0.0007) [2023-12-27 17:12:53,812][20381] Updated weights for policy 1, policy_version 1668334 (0.0008) [2023-12-27 17:12:54,274][20347] Updated weights for policy 0, policy_version 1670034 (0.0008) [2023-12-27 17:12:54,330][20347] Updated weights for policy 0, policy_version 1670044 (0.0006) [2023-12-27 17:12:54,387][20347] Updated weights for policy 0, policy_version 1670054 (0.0007) [2023-12-27 17:12:54,452][20381] Updated weights for policy 1, policy_version 1668344 (0.0008) [2023-12-27 17:12:54,501][20381] Updated weights for policy 1, policy_version 1668354 (0.0007) [2023-12-27 17:12:54,554][20381] Updated weights for policy 1, policy_version 1668364 (0.0008) [2023-12-27 17:12:55,043][20347] Updated weights for policy 0, policy_version 1670064 (0.0006) [2023-12-27 17:12:55,107][20347] Updated weights for policy 0, policy_version 1670074 (0.0006) [2023-12-27 17:12:55,156][20347] Updated weights for policy 0, policy_version 1670084 (0.0007) [2023-12-27 17:12:55,232][20381] Updated weights for policy 1, policy_version 1668374 (0.0008) [2023-12-27 17:12:55,288][20381] Updated weights for policy 1, policy_version 1668384 (0.0008) [2023-12-27 17:12:55,342][20381] Updated weights for policy 1, policy_version 1668394 (0.0007) [2023-12-27 17:12:55,837][20347] Updated weights for policy 0, policy_version 1670094 (0.0006) [2023-12-27 17:12:55,888][20347] Updated weights for policy 0, policy_version 1670104 (0.0005) [2023-12-27 17:12:55,937][20347] Updated weights for policy 0, policy_version 1670114 (0.0007) [2023-12-27 17:12:56,024][20381] Updated weights for policy 1, policy_version 1668404 (0.0010) [2023-12-27 17:12:56,090][20381] Updated weights for policy 1, policy_version 1668414 (0.0011) [2023-12-27 17:12:56,149][20381] Updated weights for policy 1, policy_version 1668424 (0.0011) [2023-12-27 17:12:56,496][20347] Updated weights for policy 0, policy_version 1670124 (0.0007) [2023-12-27 17:12:56,539][20347] Updated weights for policy 0, policy_version 1670134 (0.0005) [2023-12-27 17:12:56,587][20347] Updated weights for policy 0, policy_version 1670144 (0.0005) [2023-12-27 17:12:56,826][20381] Updated weights for policy 1, policy_version 1668434 (0.0011) [2023-12-27 17:12:56,870][20381] Updated weights for policy 1, policy_version 1668444 (0.0010) [2023-12-27 17:12:56,921][20381] Updated weights for policy 1, policy_version 1668454 (0.0010) [2023-12-27 17:12:56,978][20381] Updated weights for policy 1, policy_version 1668464 (0.0010) [2023-12-27 17:12:57,164][20347] Updated weights for policy 0, policy_version 1670154 (0.0005) [2023-12-27 17:12:57,217][20347] Updated weights for policy 0, policy_version 1670164 (0.0006) [2023-12-27 17:12:57,269][20347] Updated weights for policy 0, policy_version 1670174 (0.0007) [2023-12-27 17:12:57,326][20347] Updated weights for policy 0, policy_version 1670184 (0.0007) [2023-12-27 17:12:57,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20616.7, 300 sec: 20632.7). Total num frames: 854818816. Throughput: 0: 10291.4, 1: 10182.0. Samples: 854802344. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:12:57,363][19170] Avg episode reward: [(0, '960.880'), (1, '931.770')] [2023-12-27 17:12:57,681][20381] Updated weights for policy 1, policy_version 1668474 (0.0010) [2023-12-27 17:12:57,729][20381] Updated weights for policy 1, policy_version 1668484 (0.0010) [2023-12-27 17:12:57,777][20381] Updated weights for policy 1, policy_version 1668494 (0.0010) [2023-12-27 17:12:57,955][20347] Updated weights for policy 0, policy_version 1670194 (0.0005) [2023-12-27 17:12:58,016][20347] Updated weights for policy 0, policy_version 1670204 (0.0006) [2023-12-27 17:12:58,062][20347] Updated weights for policy 0, policy_version 1670214 (0.0007) [2023-12-27 17:12:58,494][20381] Updated weights for policy 1, policy_version 1668504 (0.0010) [2023-12-27 17:12:58,555][20381] Updated weights for policy 1, policy_version 1668514 (0.0011) [2023-12-27 17:12:58,618][20381] Updated weights for policy 1, policy_version 1668524 (0.0011) [2023-12-27 17:12:58,792][20347] Updated weights for policy 0, policy_version 1670224 (0.0008) [2023-12-27 17:12:58,857][20347] Updated weights for policy 0, policy_version 1670234 (0.0008) [2023-12-27 17:12:58,931][20347] Updated weights for policy 0, policy_version 1670244 (0.0008) [2023-12-27 17:12:59,424][20381] Updated weights for policy 1, policy_version 1668534 (0.0008) [2023-12-27 17:12:59,478][20381] Updated weights for policy 1, policy_version 1668544 (0.0007) [2023-12-27 17:12:59,537][20381] Updated weights for policy 1, policy_version 1668554 (0.0008) [2023-12-27 17:12:59,715][20347] Updated weights for policy 0, policy_version 1670254 (0.0007) [2023-12-27 17:12:59,767][20347] Updated weights for policy 0, policy_version 1670264 (0.0008) [2023-12-27 17:12:59,820][20347] Updated weights for policy 0, policy_version 1670274 (0.0008) [2023-12-27 17:13:00,258][20381] Updated weights for policy 1, policy_version 1668564 (0.0006) [2023-12-27 17:13:00,319][20381] Updated weights for policy 1, policy_version 1668574 (0.0005) [2023-12-27 17:13:00,383][20381] Updated weights for policy 1, policy_version 1668584 (0.0005) [2023-12-27 17:13:00,588][20347] Updated weights for policy 0, policy_version 1670284 (0.0008) [2023-12-27 17:13:00,638][20347] Updated weights for policy 0, policy_version 1670294 (0.0010) [2023-12-27 17:13:00,690][20347] Updated weights for policy 0, policy_version 1670304 (0.0010) [2023-12-27 17:13:01,004][20381] Updated weights for policy 1, policy_version 1668594 (0.0008) [2023-12-27 17:13:01,062][20381] Updated weights for policy 1, policy_version 1668604 (0.0009) [2023-12-27 17:13:01,121][20381] Updated weights for policy 1, policy_version 1668614 (0.0010) [2023-12-27 17:13:01,186][20381] Updated weights for policy 1, policy_version 1668624 (0.0008) [2023-12-27 17:13:01,375][20347] Updated weights for policy 0, policy_version 1670314 (0.0007) [2023-12-27 17:13:01,435][20347] Updated weights for policy 0, policy_version 1670324 (0.0008) [2023-12-27 17:13:01,486][20347] Updated weights for policy 0, policy_version 1670334 (0.0008) [2023-12-27 17:13:01,536][20347] Updated weights for policy 0, policy_version 1670344 (0.0008) [2023-12-27 17:13:01,836][20381] Updated weights for policy 1, policy_version 1668634 (0.0006) [2023-12-27 17:13:01,887][20381] Updated weights for policy 1, policy_version 1668644 (0.0006) [2023-12-27 17:13:01,942][20381] Updated weights for policy 1, policy_version 1668654 (0.0006) [2023-12-27 17:13:02,235][20347] Updated weights for policy 0, policy_version 1670354 (0.0010) [2023-12-27 17:13:02,291][20347] Updated weights for policy 0, policy_version 1670364 (0.0007) [2023-12-27 17:13:02,346][20347] Updated weights for policy 0, policy_version 1670374 (0.0009) [2023-12-27 17:13:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 854917120. Throughput: 0: 10356.3, 1: 10177.3. Samples: 854924524. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:02,363][19170] Avg episode reward: [(0, '951.160'), (1, '931.680')] [2023-12-27 17:13:02,568][20381] Updated weights for policy 1, policy_version 1668664 (0.0007) [2023-12-27 17:13:02,632][20381] Updated weights for policy 1, policy_version 1668674 (0.0007) [2023-12-27 17:13:02,691][20381] Updated weights for policy 1, policy_version 1668684 (0.0007) [2023-12-27 17:13:02,996][20347] Updated weights for policy 0, policy_version 1670384 (0.0008) [2023-12-27 17:13:03,050][20347] Updated weights for policy 0, policy_version 1670394 (0.0010) [2023-12-27 17:13:03,098][20347] Updated weights for policy 0, policy_version 1670404 (0.0010) [2023-12-27 17:13:03,313][20381] Updated weights for policy 1, policy_version 1668694 (0.0007) [2023-12-27 17:13:03,355][20381] Updated weights for policy 1, policy_version 1668704 (0.0005) [2023-12-27 17:13:03,400][20381] Updated weights for policy 1, policy_version 1668714 (0.0005) [2023-12-27 17:13:03,749][20347] Updated weights for policy 0, policy_version 1670414 (0.0007) [2023-12-27 17:13:03,809][20347] Updated weights for policy 0, policy_version 1670424 (0.0005) [2023-12-27 17:13:03,872][20347] Updated weights for policy 0, policy_version 1670434 (0.0007) [2023-12-27 17:13:03,992][20381] Updated weights for policy 1, policy_version 1668724 (0.0007) [2023-12-27 17:13:04,045][20381] Updated weights for policy 1, policy_version 1668734 (0.0010) [2023-12-27 17:13:04,099][20381] Updated weights for policy 1, policy_version 1668744 (0.0010) [2023-12-27 17:13:04,431][20347] Updated weights for policy 0, policy_version 1670444 (0.0006) [2023-12-27 17:13:04,492][20347] Updated weights for policy 0, policy_version 1670454 (0.0008) [2023-12-27 17:13:04,555][20347] Updated weights for policy 0, policy_version 1670464 (0.0007) [2023-12-27 17:13:04,811][20381] Updated weights for policy 1, policy_version 1668754 (0.0011) [2023-12-27 17:13:04,869][20381] Updated weights for policy 1, policy_version 1668764 (0.0010) [2023-12-27 17:13:04,917][20381] Updated weights for policy 1, policy_version 1668774 (0.0010) [2023-12-27 17:13:04,968][20381] Updated weights for policy 1, policy_version 1668784 (0.0010) [2023-12-27 17:13:05,191][20347] Updated weights for policy 0, policy_version 1670474 (0.0008) [2023-12-27 17:13:05,240][20347] Updated weights for policy 0, policy_version 1670484 (0.0005) [2023-12-27 17:13:05,291][20347] Updated weights for policy 0, policy_version 1670494 (0.0005) [2023-12-27 17:13:05,358][20347] Updated weights for policy 0, policy_version 1670504 (0.0006) [2023-12-27 17:13:05,584][20381] Updated weights for policy 1, policy_version 1668794 (0.0007) [2023-12-27 17:13:05,641][20381] Updated weights for policy 1, policy_version 1668804 (0.0010) [2023-12-27 17:13:05,688][20381] Updated weights for policy 1, policy_version 1668814 (0.0010) [2023-12-27 17:13:05,896][20347] Updated weights for policy 0, policy_version 1670514 (0.0005) [2023-12-27 17:13:05,952][20347] Updated weights for policy 0, policy_version 1670524 (0.0006) [2023-12-27 17:13:06,004][20347] Updated weights for policy 0, policy_version 1670534 (0.0010) [2023-12-27 17:13:06,267][20381] Updated weights for policy 1, policy_version 1668824 (0.0011) [2023-12-27 17:13:06,330][20381] Updated weights for policy 1, policy_version 1668834 (0.0011) [2023-12-27 17:13:06,392][20381] Updated weights for policy 1, policy_version 1668844 (0.0010) [2023-12-27 17:13:06,650][20347] Updated weights for policy 0, policy_version 1670544 (0.0011) [2023-12-27 17:13:06,700][20347] Updated weights for policy 0, policy_version 1670554 (0.0010) [2023-12-27 17:13:06,752][20347] Updated weights for policy 0, policy_version 1670564 (0.0010) [2023-12-27 17:13:07,074][20381] Updated weights for policy 1, policy_version 1668854 (0.0008) [2023-12-27 17:13:07,130][20381] Updated weights for policy 1, policy_version 1668864 (0.0007) [2023-12-27 17:13:07,185][20381] Updated weights for policy 1, policy_version 1668874 (0.0007) [2023-12-27 17:13:07,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 855023616. Throughput: 0: 10401.8, 1: 10254.3. Samples: 854989616. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:07,363][19170] Avg episode reward: [(0, '951.040'), (1, '892.800')] [2023-12-27 17:13:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001668880_427278336.pth... [2023-12-27 17:13:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001670568_427745280.pth... [2023-12-27 17:13:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001669352_427433984.pth [2023-12-27 17:13:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001667696_426975232.pth [2023-12-27 17:13:07,437][20347] Updated weights for policy 0, policy_version 1670574 (0.0009) [2023-12-27 17:13:07,489][20347] Updated weights for policy 0, policy_version 1670584 (0.0007) [2023-12-27 17:13:07,549][20347] Updated weights for policy 0, policy_version 1670594 (0.0005) [2023-12-27 17:13:07,830][20381] Updated weights for policy 1, policy_version 1668884 (0.0007) [2023-12-27 17:13:07,887][20381] Updated weights for policy 1, policy_version 1668894 (0.0009) [2023-12-27 17:13:07,950][20381] Updated weights for policy 1, policy_version 1668904 (0.0007) [2023-12-27 17:13:08,108][20347] Updated weights for policy 0, policy_version 1670604 (0.0005) [2023-12-27 17:13:08,168][20347] Updated weights for policy 0, policy_version 1670614 (0.0006) [2023-12-27 17:13:08,216][20347] Updated weights for policy 0, policy_version 1670624 (0.0005) [2023-12-27 17:13:08,674][20381] Updated weights for policy 1, policy_version 1668914 (0.0007) [2023-12-27 17:13:08,722][20381] Updated weights for policy 1, policy_version 1668924 (0.0008) [2023-12-27 17:13:08,774][20381] Updated weights for policy 1, policy_version 1668934 (0.0006) [2023-12-27 17:13:08,805][20347] Updated weights for policy 0, policy_version 1670634 (0.0005) [2023-12-27 17:13:08,827][20381] Updated weights for policy 1, policy_version 1668944 (0.0007) [2023-12-27 17:13:08,863][20347] Updated weights for policy 0, policy_version 1670644 (0.0006) [2023-12-27 17:13:08,913][20347] Updated weights for policy 0, policy_version 1670654 (0.0007) [2023-12-27 17:13:08,965][20347] Updated weights for policy 0, policy_version 1670664 (0.0008) [2023-12-27 17:13:09,554][20381] Updated weights for policy 1, policy_version 1668954 (0.0007) [2023-12-27 17:13:09,614][20381] Updated weights for policy 1, policy_version 1668964 (0.0007) [2023-12-27 17:13:09,675][20381] Updated weights for policy 1, policy_version 1668974 (0.0008) [2023-12-27 17:13:09,744][20347] Updated weights for policy 0, policy_version 1670674 (0.0006) [2023-12-27 17:13:09,803][20347] Updated weights for policy 0, policy_version 1670684 (0.0007) [2023-12-27 17:13:09,868][20347] Updated weights for policy 0, policy_version 1670694 (0.0008) [2023-12-27 17:13:10,441][20381] Updated weights for policy 1, policy_version 1668984 (0.0008) [2023-12-27 17:13:10,502][20381] Updated weights for policy 1, policy_version 1668994 (0.0008) [2023-12-27 17:13:10,525][20347] Updated weights for policy 0, policy_version 1670704 (0.0007) [2023-12-27 17:13:10,557][20381] Updated weights for policy 1, policy_version 1669004 (0.0007) [2023-12-27 17:13:10,571][20347] Updated weights for policy 0, policy_version 1670714 (0.0006) [2023-12-27 17:13:10,616][20347] Updated weights for policy 0, policy_version 1670724 (0.0007) [2023-12-27 17:13:11,289][20381] Updated weights for policy 1, policy_version 1669014 (0.0007) [2023-12-27 17:13:11,332][20347] Updated weights for policy 0, policy_version 1670734 (0.0007) [2023-12-27 17:13:11,352][20381] Updated weights for policy 1, policy_version 1669024 (0.0007) [2023-12-27 17:13:11,403][20347] Updated weights for policy 0, policy_version 1670744 (0.0007) [2023-12-27 17:13:11,426][20381] Updated weights for policy 1, policy_version 1669034 (0.0009) [2023-12-27 17:13:11,466][20347] Updated weights for policy 0, policy_version 1670754 (0.0009) [2023-12-27 17:13:12,156][20381] Updated weights for policy 1, policy_version 1669044 (0.0009) [2023-12-27 17:13:12,167][20347] Updated weights for policy 0, policy_version 1670764 (0.0009) [2023-12-27 17:13:12,211][20381] Updated weights for policy 1, policy_version 1669054 (0.0011) [2023-12-27 17:13:12,228][20347] Updated weights for policy 0, policy_version 1670774 (0.0011) [2023-12-27 17:13:12,271][20381] Updated weights for policy 1, policy_version 1669064 (0.0011) [2023-12-27 17:13:12,289][20347] Updated weights for policy 0, policy_version 1670784 (0.0011) [2023-12-27 17:13:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 855130112. Throughput: 0: 10456.3, 1: 10238.9. Samples: 855115172. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:12,363][19170] Avg episode reward: [(0, '951.010'), (1, '863.630')] [2023-12-27 17:13:12,959][20381] Updated weights for policy 1, policy_version 1669074 (0.0010) [2023-12-27 17:13:12,997][20347] Updated weights for policy 0, policy_version 1670794 (0.0010) [2023-12-27 17:13:13,010][20381] Updated weights for policy 1, policy_version 1669084 (0.0006) [2023-12-27 17:13:13,050][20347] Updated weights for policy 0, policy_version 1670804 (0.0007) [2023-12-27 17:13:13,056][20381] Updated weights for policy 1, policy_version 1669094 (0.0005) [2023-12-27 17:13:13,105][20347] Updated weights for policy 0, policy_version 1670814 (0.0006) [2023-12-27 17:13:13,110][20381] Updated weights for policy 1, policy_version 1669104 (0.0007) [2023-12-27 17:13:13,159][20347] Updated weights for policy 0, policy_version 1670824 (0.0006) [2023-12-27 17:13:13,726][20347] Updated weights for policy 0, policy_version 1670834 (0.0005) [2023-12-27 17:13:13,787][20347] Updated weights for policy 0, policy_version 1670844 (0.0005) [2023-12-27 17:13:13,799][20381] Updated weights for policy 1, policy_version 1669114 (0.0008) [2023-12-27 17:13:13,848][20347] Updated weights for policy 0, policy_version 1670854 (0.0005) [2023-12-27 17:13:13,853][20381] Updated weights for policy 1, policy_version 1669124 (0.0007) [2023-12-27 17:13:13,909][20381] Updated weights for policy 1, policy_version 1669134 (0.0008) [2023-12-27 17:13:14,382][20347] Updated weights for policy 0, policy_version 1670864 (0.0005) [2023-12-27 17:13:14,441][20347] Updated weights for policy 0, policy_version 1670874 (0.0005) [2023-12-27 17:13:14,493][20347] Updated weights for policy 0, policy_version 1670884 (0.0005) [2023-12-27 17:13:14,640][20381] Updated weights for policy 1, policy_version 1669144 (0.0010) [2023-12-27 17:13:14,686][20381] Updated weights for policy 1, policy_version 1669154 (0.0010) [2023-12-27 17:13:14,743][20381] Updated weights for policy 1, policy_version 1669165 (0.0009) [2023-12-27 17:13:15,047][20347] Updated weights for policy 0, policy_version 1670894 (0.0008) [2023-12-27 17:13:15,116][20347] Updated weights for policy 0, policy_version 1670904 (0.0011) [2023-12-27 17:13:15,182][20347] Updated weights for policy 0, policy_version 1670914 (0.0011) [2023-12-27 17:13:15,416][20381] Updated weights for policy 1, policy_version 1669175 (0.0006) [2023-12-27 17:13:15,471][20381] Updated weights for policy 1, policy_version 1669185 (0.0006) [2023-12-27 17:13:15,531][20381] Updated weights for policy 1, policy_version 1669195 (0.0008) [2023-12-27 17:13:15,802][20347] Updated weights for policy 0, policy_version 1670924 (0.0011) [2023-12-27 17:13:15,864][20347] Updated weights for policy 0, policy_version 1670934 (0.0009) [2023-12-27 17:13:15,916][20347] Updated weights for policy 0, policy_version 1670944 (0.0005) [2023-12-27 17:13:16,145][20381] Updated weights for policy 1, policy_version 1669205 (0.0007) [2023-12-27 17:13:16,197][20381] Updated weights for policy 1, policy_version 1669215 (0.0006) [2023-12-27 17:13:16,246][20381] Updated weights for policy 1, policy_version 1669225 (0.0005) [2023-12-27 17:13:16,555][20347] Updated weights for policy 0, policy_version 1670954 (0.0008) [2023-12-27 17:13:16,616][20347] Updated weights for policy 0, policy_version 1670964 (0.0010) [2023-12-27 17:13:16,670][20347] Updated weights for policy 0, policy_version 1670974 (0.0010) [2023-12-27 17:13:16,738][20347] Updated weights for policy 0, policy_version 1670984 (0.0011) [2023-12-27 17:13:16,833][20381] Updated weights for policy 1, policy_version 1669235 (0.0005) [2023-12-27 17:13:16,899][20381] Updated weights for policy 1, policy_version 1669245 (0.0006) [2023-12-27 17:13:16,963][20381] Updated weights for policy 1, policy_version 1669255 (0.0011) [2023-12-27 17:13:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 855228416. Throughput: 0: 10499.3, 1: 10206.5. Samples: 855241396. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:17,363][19170] Avg episode reward: [(0, '960.840'), (1, '873.290')] [2023-12-27 17:13:17,421][20347] Updated weights for policy 0, policy_version 1670994 (0.0010) [2023-12-27 17:13:17,478][20347] Updated weights for policy 0, policy_version 1671004 (0.0010) [2023-12-27 17:13:17,532][20347] Updated weights for policy 0, policy_version 1671014 (0.0010) [2023-12-27 17:13:17,623][20381] Updated weights for policy 1, policy_version 1669265 (0.0011) [2023-12-27 17:13:17,671][20381] Updated weights for policy 1, policy_version 1669275 (0.0010) [2023-12-27 17:13:17,725][20381] Updated weights for policy 1, policy_version 1669285 (0.0010) [2023-12-27 17:13:17,780][20381] Updated weights for policy 1, policy_version 1669295 (0.0010) [2023-12-27 17:13:18,192][20347] Updated weights for policy 0, policy_version 1671024 (0.0008) [2023-12-27 17:13:18,246][20347] Updated weights for policy 0, policy_version 1671034 (0.0010) [2023-12-27 17:13:18,304][20347] Updated weights for policy 0, policy_version 1671044 (0.0010) [2023-12-27 17:13:18,484][20381] Updated weights for policy 1, policy_version 1669305 (0.0011) [2023-12-27 17:13:18,544][20381] Updated weights for policy 1, policy_version 1669315 (0.0011) [2023-12-27 17:13:18,606][20381] Updated weights for policy 1, policy_version 1669325 (0.0011) [2023-12-27 17:13:18,931][20347] Updated weights for policy 0, policy_version 1671054 (0.0010) [2023-12-27 17:13:18,981][20347] Updated weights for policy 0, policy_version 1671064 (0.0010) [2023-12-27 17:13:19,028][20347] Updated weights for policy 0, policy_version 1671074 (0.0010) [2023-12-27 17:13:19,258][20381] Updated weights for policy 1, policy_version 1669335 (0.0010) [2023-12-27 17:13:19,321][20381] Updated weights for policy 1, policy_version 1669345 (0.0007) [2023-12-27 17:13:19,383][20381] Updated weights for policy 1, policy_version 1669355 (0.0007) [2023-12-27 17:13:19,766][20347] Updated weights for policy 0, policy_version 1671084 (0.0008) [2023-12-27 17:13:19,829][20347] Updated weights for policy 0, policy_version 1671094 (0.0006) [2023-12-27 17:13:19,893][20347] Updated weights for policy 0, policy_version 1671104 (0.0009) [2023-12-27 17:13:20,090][20381] Updated weights for policy 1, policy_version 1669365 (0.0009) [2023-12-27 17:13:20,156][20381] Updated weights for policy 1, policy_version 1669375 (0.0010) [2023-12-27 17:13:20,204][20292] KL-divergence is very high: 160.0194 [2023-12-27 17:13:20,217][20381] Updated weights for policy 1, policy_version 1669385 (0.0007) [2023-12-27 17:13:20,250][20292] KL-divergence is very high: 197.9319 [2023-12-27 17:13:20,549][20347] Updated weights for policy 0, policy_version 1671114 (0.0010) [2023-12-27 17:13:20,613][20347] Updated weights for policy 0, policy_version 1671124 (0.0008) [2023-12-27 17:13:20,681][20347] Updated weights for policy 0, policy_version 1671134 (0.0010) [2023-12-27 17:13:20,743][20347] Updated weights for policy 0, policy_version 1671144 (0.0007) [2023-12-27 17:13:20,919][20381] Updated weights for policy 1, policy_version 1669395 (0.0008) [2023-12-27 17:13:20,979][20381] Updated weights for policy 1, policy_version 1669405 (0.0007) [2023-12-27 17:13:21,040][20381] Updated weights for policy 1, policy_version 1669415 (0.0007) [2023-12-27 17:13:21,464][20347] Updated weights for policy 0, policy_version 1671154 (0.0008) [2023-12-27 17:13:21,536][20347] Updated weights for policy 0, policy_version 1671164 (0.0008) [2023-12-27 17:13:21,579][20291] KL-divergence is very high: 126.8742 [2023-12-27 17:13:21,594][20347] Updated weights for policy 0, policy_version 1671174 (0.0007) [2023-12-27 17:13:21,783][20381] Updated weights for policy 1, policy_version 1669425 (0.0007) [2023-12-27 17:13:21,845][20381] Updated weights for policy 1, policy_version 1669435 (0.0007) [2023-12-27 17:13:21,908][20381] Updated weights for policy 1, policy_version 1669445 (0.0008) [2023-12-27 17:13:21,969][20381] Updated weights for policy 1, policy_version 1669455 (0.0007) [2023-12-27 17:13:22,242][20347] Updated weights for policy 0, policy_version 1671184 (0.0008) [2023-12-27 17:13:22,302][20347] Updated weights for policy 0, policy_version 1671194 (0.0007) [2023-12-27 17:13:22,362][20347] Updated weights for policy 0, policy_version 1671204 (0.0008) [2023-12-27 17:13:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 855326720. Throughput: 0: 10483.8, 1: 10187.0. Samples: 855302464. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:22,364][19170] Avg episode reward: [(0, '970.590'), (1, '892.600')] [2023-12-27 17:13:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001669456_427425792.pth... [2023-12-27 17:13:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001668272_427122688.pth [2023-12-27 17:13:22,391][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001671208_427909120.pth... [2023-12-27 17:13:22,395][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001669960_427589632.pth [2023-12-27 17:13:22,701][20381] Updated weights for policy 1, policy_version 1669465 (0.0008) [2023-12-27 17:13:22,763][20381] Updated weights for policy 1, policy_version 1669475 (0.0008) [2023-12-27 17:13:22,813][20381] Updated weights for policy 1, policy_version 1669485 (0.0007) [2023-12-27 17:13:23,041][20347] Updated weights for policy 0, policy_version 1671214 (0.0008) [2023-12-27 17:13:23,102][20347] Updated weights for policy 0, policy_version 1671224 (0.0009) [2023-12-27 17:13:23,167][20347] Updated weights for policy 0, policy_version 1671234 (0.0007) [2023-12-27 17:13:23,410][20381] Updated weights for policy 1, policy_version 1669495 (0.0009) [2023-12-27 17:13:23,458][20381] Updated weights for policy 1, policy_version 1669505 (0.0008) [2023-12-27 17:13:23,506][20381] Updated weights for policy 1, policy_version 1669515 (0.0008) [2023-12-27 17:13:23,875][20347] Updated weights for policy 0, policy_version 1671244 (0.0008) [2023-12-27 17:13:23,922][20347] Updated weights for policy 0, policy_version 1671254 (0.0007) [2023-12-27 17:13:23,972][20347] Updated weights for policy 0, policy_version 1671264 (0.0007) [2023-12-27 17:13:24,139][20381] Updated weights for policy 1, policy_version 1669525 (0.0007) [2023-12-27 17:13:24,203][20381] Updated weights for policy 1, policy_version 1669535 (0.0008) [2023-12-27 17:13:24,261][20381] Updated weights for policy 1, policy_version 1669545 (0.0007) [2023-12-27 17:13:24,646][20347] Updated weights for policy 0, policy_version 1671274 (0.0007) [2023-12-27 17:13:24,695][20347] Updated weights for policy 0, policy_version 1671284 (0.0005) [2023-12-27 17:13:24,740][20347] Updated weights for policy 0, policy_version 1671294 (0.0005) [2023-12-27 17:13:24,800][20347] Updated weights for policy 0, policy_version 1671304 (0.0007) [2023-12-27 17:13:24,906][20381] Updated weights for policy 1, policy_version 1669555 (0.0007) [2023-12-27 17:13:24,957][20381] Updated weights for policy 1, policy_version 1669565 (0.0005) [2023-12-27 17:13:25,011][20381] Updated weights for policy 1, policy_version 1669575 (0.0005) [2023-12-27 17:13:25,421][20347] Updated weights for policy 0, policy_version 1671314 (0.0008) [2023-12-27 17:13:25,475][20347] Updated weights for policy 0, policy_version 1671324 (0.0006) [2023-12-27 17:13:25,487][20291] KL-divergence is very high: 112.4391 [2023-12-27 17:13:25,533][20347] Updated weights for policy 0, policy_version 1671334 (0.0005) [2023-12-27 17:13:25,671][20381] Updated weights for policy 1, policy_version 1669585 (0.0007) [2023-12-27 17:13:25,731][20381] Updated weights for policy 1, policy_version 1669595 (0.0008) [2023-12-27 17:13:25,790][20381] Updated weights for policy 1, policy_version 1669605 (0.0008) [2023-12-27 17:13:25,845][20381] Updated weights for policy 1, policy_version 1669615 (0.0006) [2023-12-27 17:13:26,139][20347] Updated weights for policy 0, policy_version 1671344 (0.0007) [2023-12-27 17:13:26,190][20347] Updated weights for policy 0, policy_version 1671354 (0.0008) [2023-12-27 17:13:26,239][20347] Updated weights for policy 0, policy_version 1671364 (0.0007) [2023-12-27 17:13:26,480][20381] Updated weights for policy 1, policy_version 1669625 (0.0006) [2023-12-27 17:13:26,537][20381] Updated weights for policy 1, policy_version 1669635 (0.0010) [2023-12-27 17:13:26,585][20381] Updated weights for policy 1, policy_version 1669645 (0.0010) [2023-12-27 17:13:26,891][20347] Updated weights for policy 0, policy_version 1671374 (0.0007) [2023-12-27 17:13:26,947][20347] Updated weights for policy 0, policy_version 1671384 (0.0007) [2023-12-27 17:13:27,001][20347] Updated weights for policy 0, policy_version 1671394 (0.0006) [2023-12-27 17:13:27,258][20381] Updated weights for policy 1, policy_version 1669655 (0.0010) [2023-12-27 17:13:27,314][20381] Updated weights for policy 1, policy_version 1669665 (0.0010) [2023-12-27 17:13:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 855433216. Throughput: 0: 10481.5, 1: 10157.5. Samples: 855425516. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:27,363][19170] Avg episode reward: [(0, '960.800'), (1, '931.610')] [2023-12-27 17:13:27,370][20381] Updated weights for policy 1, policy_version 1669675 (0.0008) [2023-12-27 17:13:27,606][20347] Updated weights for policy 0, policy_version 1671404 (0.0006) [2023-12-27 17:13:27,651][20347] Updated weights for policy 0, policy_version 1671414 (0.0008) [2023-12-27 17:13:27,694][20347] Updated weights for policy 0, policy_version 1671424 (0.0008) [2023-12-27 17:13:27,945][20381] Updated weights for policy 1, policy_version 1669685 (0.0010) [2023-12-27 17:13:27,993][20381] Updated weights for policy 1, policy_version 1669695 (0.0005) [2023-12-27 17:13:28,045][20381] Updated weights for policy 1, policy_version 1669705 (0.0005) [2023-12-27 17:13:28,384][20347] Updated weights for policy 0, policy_version 1671434 (0.0007) [2023-12-27 17:13:28,439][20347] Updated weights for policy 0, policy_version 1671444 (0.0008) [2023-12-27 17:13:28,491][20347] Updated weights for policy 0, policy_version 1671454 (0.0007) [2023-12-27 17:13:28,545][20347] Updated weights for policy 0, policy_version 1671464 (0.0008) [2023-12-27 17:13:28,657][20381] Updated weights for policy 1, policy_version 1669715 (0.0005) [2023-12-27 17:13:28,719][20381] Updated weights for policy 1, policy_version 1669725 (0.0006) [2023-12-27 17:13:28,773][20381] Updated weights for policy 1, policy_version 1669735 (0.0007) [2023-12-27 17:13:29,268][20347] Updated weights for policy 0, policy_version 1671474 (0.0008) [2023-12-27 17:13:29,333][20347] Updated weights for policy 0, policy_version 1671484 (0.0007) [2023-12-27 17:13:29,382][20381] Updated weights for policy 1, policy_version 1669745 (0.0005) [2023-12-27 17:13:29,393][20347] Updated weights for policy 0, policy_version 1671494 (0.0007) [2023-12-27 17:13:29,441][20381] Updated weights for policy 1, policy_version 1669755 (0.0006) [2023-12-27 17:13:29,504][20381] Updated weights for policy 1, policy_version 1669765 (0.0005) [2023-12-27 17:13:29,565][20381] Updated weights for policy 1, policy_version 1669775 (0.0005) [2023-12-27 17:13:30,101][20347] Updated weights for policy 0, policy_version 1671504 (0.0006) [2023-12-27 17:13:30,144][20381] Updated weights for policy 1, policy_version 1669785 (0.0007) [2023-12-27 17:13:30,152][20347] Updated weights for policy 0, policy_version 1671514 (0.0006) [2023-12-27 17:13:30,203][20381] Updated weights for policy 1, policy_version 1669795 (0.0006) [2023-12-27 17:13:30,208][20347] Updated weights for policy 0, policy_version 1671524 (0.0006) [2023-12-27 17:13:30,266][20381] Updated weights for policy 1, policy_version 1669805 (0.0006) [2023-12-27 17:13:30,815][20347] Updated weights for policy 0, policy_version 1671534 (0.0008) [2023-12-27 17:13:30,877][20347] Updated weights for policy 0, policy_version 1671544 (0.0010) [2023-12-27 17:13:30,883][20381] Updated weights for policy 1, policy_version 1669815 (0.0005) [2023-12-27 17:13:30,933][20381] Updated weights for policy 1, policy_version 1669825 (0.0005) [2023-12-27 17:13:30,936][20347] Updated weights for policy 0, policy_version 1671554 (0.0010) [2023-12-27 17:13:30,980][20381] Updated weights for policy 1, policy_version 1669835 (0.0010) [2023-12-27 17:13:30,980][20292] KL-divergence is very high: 141.4975 [2023-12-27 17:13:31,663][20347] Updated weights for policy 0, policy_version 1671564 (0.0010) [2023-12-27 17:13:31,680][20381] Updated weights for policy 1, policy_version 1669845 (0.0008) [2023-12-27 17:13:31,712][20347] Updated weights for policy 0, policy_version 1671574 (0.0010) [2023-12-27 17:13:31,740][20381] Updated weights for policy 1, policy_version 1669855 (0.0006) [2023-12-27 17:13:31,773][20347] Updated weights for policy 0, policy_version 1671584 (0.0008) [2023-12-27 17:13:31,797][20381] Updated weights for policy 1, policy_version 1669865 (0.0006) [2023-12-27 17:13:32,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 855539712. Throughput: 0: 10493.2, 1: 10311.9. Samples: 855553260. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:32,363][19170] Avg episode reward: [(0, '980.430'), (1, '921.860')] [2023-12-27 17:13:32,388][20347] Updated weights for policy 0, policy_version 1671594 (0.0007) [2023-12-27 17:13:32,449][20347] Updated weights for policy 0, policy_version 1671604 (0.0006) [2023-12-27 17:13:32,463][20381] Updated weights for policy 1, policy_version 1669875 (0.0008) [2023-12-27 17:13:32,509][20347] Updated weights for policy 0, policy_version 1671614 (0.0010) [2023-12-27 17:13:32,518][20381] Updated weights for policy 1, policy_version 1669885 (0.0005) [2023-12-27 17:13:32,567][20347] Updated weights for policy 0, policy_version 1671624 (0.0010) [2023-12-27 17:13:32,574][20381] Updated weights for policy 1, policy_version 1669895 (0.0005) [2023-12-27 17:13:33,170][20381] Updated weights for policy 1, policy_version 1669905 (0.0005) [2023-12-27 17:13:33,221][20381] Updated weights for policy 1, policy_version 1669915 (0.0005) [2023-12-27 17:13:33,262][20347] Updated weights for policy 0, policy_version 1671634 (0.0010) [2023-12-27 17:13:33,275][20381] Updated weights for policy 1, policy_version 1669925 (0.0006) [2023-12-27 17:13:33,317][20347] Updated weights for policy 0, policy_version 1671644 (0.0005) [2023-12-27 17:13:33,326][20381] Updated weights for policy 1, policy_version 1669935 (0.0008) [2023-12-27 17:13:33,367][20347] Updated weights for policy 0, policy_version 1671654 (0.0007) [2023-12-27 17:13:33,907][20347] Updated weights for policy 0, policy_version 1671664 (0.0006) [2023-12-27 17:13:33,965][20347] Updated weights for policy 0, policy_version 1671674 (0.0005) [2023-12-27 17:13:34,016][20347] Updated weights for policy 0, policy_version 1671684 (0.0005) [2023-12-27 17:13:34,051][20381] Updated weights for policy 1, policy_version 1669945 (0.0008) [2023-12-27 17:13:34,110][20381] Updated weights for policy 1, policy_version 1669955 (0.0008) [2023-12-27 17:13:34,175][20381] Updated weights for policy 1, policy_version 1669965 (0.0008) [2023-12-27 17:13:34,579][20347] Updated weights for policy 0, policy_version 1671694 (0.0008) [2023-12-27 17:13:34,628][20347] Updated weights for policy 0, policy_version 1671704 (0.0006) [2023-12-27 17:13:34,689][20347] Updated weights for policy 0, policy_version 1671714 (0.0008) [2023-12-27 17:13:34,886][20381] Updated weights for policy 1, policy_version 1669975 (0.0008) [2023-12-27 17:13:34,931][20381] Updated weights for policy 1, policy_version 1669985 (0.0008) [2023-12-27 17:13:34,984][20381] Updated weights for policy 1, policy_version 1669995 (0.0006) [2023-12-27 17:13:35,340][20347] Updated weights for policy 0, policy_version 1671724 (0.0011) [2023-12-27 17:13:35,391][20347] Updated weights for policy 0, policy_version 1671734 (0.0010) [2023-12-27 17:13:35,452][20347] Updated weights for policy 0, policy_version 1671744 (0.0010) [2023-12-27 17:13:35,662][20381] Updated weights for policy 1, policy_version 1670005 (0.0007) [2023-12-27 17:13:35,716][20381] Updated weights for policy 1, policy_version 1670015 (0.0008) [2023-12-27 17:13:35,769][20381] Updated weights for policy 1, policy_version 1670025 (0.0007) [2023-12-27 17:13:36,122][20347] Updated weights for policy 0, policy_version 1671754 (0.0010) [2023-12-27 17:13:36,191][20347] Updated weights for policy 0, policy_version 1671764 (0.0008) [2023-12-27 17:13:36,255][20347] Updated weights for policy 0, policy_version 1671774 (0.0010) [2023-12-27 17:13:36,324][20347] Updated weights for policy 0, policy_version 1671784 (0.0010) [2023-12-27 17:13:36,390][20381] Updated weights for policy 1, policy_version 1670035 (0.0006) [2023-12-27 17:13:36,445][20381] Updated weights for policy 1, policy_version 1670045 (0.0008) [2023-12-27 17:13:36,499][20381] Updated weights for policy 1, policy_version 1670055 (0.0007) [2023-12-27 17:13:37,044][20347] Updated weights for policy 0, policy_version 1671794 (0.0005) [2023-12-27 17:13:37,093][20381] Updated weights for policy 1, policy_version 1670065 (0.0006) [2023-12-27 17:13:37,110][20347] Updated weights for policy 0, policy_version 1671804 (0.0007) [2023-12-27 17:13:37,148][20381] Updated weights for policy 1, policy_version 1670075 (0.0006) [2023-12-27 17:13:37,165][20347] Updated weights for policy 0, policy_version 1671814 (0.0006) [2023-12-27 17:13:37,213][20381] Updated weights for policy 1, policy_version 1670085 (0.0008) [2023-12-27 17:13:37,278][20381] Updated weights for policy 1, policy_version 1670095 (0.0008) [2023-12-27 17:13:37,363][19170] Fps is (10 sec: 22118.0, 60 sec: 20889.5, 300 sec: 20632.7). Total num frames: 855654400. Throughput: 0: 10515.2, 1: 10338.1. Samples: 855617912. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.950')] [2023-12-27 17:13:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001670096_427589632.pth... [2023-12-27 17:13:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001671816_428064768.pth... [2023-12-27 17:13:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001670568_427745280.pth [2023-12-27 17:13:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001668880_427278336.pth [2023-12-27 17:13:37,773][20347] Updated weights for policy 0, policy_version 1671824 (0.0005) [2023-12-27 17:13:37,830][20347] Updated weights for policy 0, policy_version 1671834 (0.0007) [2023-12-27 17:13:37,885][20347] Updated weights for policy 0, policy_version 1671844 (0.0007) [2023-12-27 17:13:37,909][20381] Updated weights for policy 1, policy_version 1670105 (0.0007) [2023-12-27 17:13:37,967][20381] Updated weights for policy 1, policy_version 1670115 (0.0010) [2023-12-27 17:13:38,021][20381] Updated weights for policy 1, policy_version 1670125 (0.0010) [2023-12-27 17:13:38,563][20347] Updated weights for policy 0, policy_version 1671854 (0.0008) [2023-12-27 17:13:38,618][20347] Updated weights for policy 0, policy_version 1671864 (0.0010) [2023-12-27 17:13:38,671][20347] Updated weights for policy 0, policy_version 1671874 (0.0010) [2023-12-27 17:13:38,683][20381] Updated weights for policy 1, policy_version 1670135 (0.0010) [2023-12-27 17:13:38,740][20381] Updated weights for policy 1, policy_version 1670145 (0.0010) [2023-12-27 17:13:38,800][20381] Updated weights for policy 1, policy_version 1670155 (0.0011) [2023-12-27 17:13:39,441][20347] Updated weights for policy 0, policy_version 1671884 (0.0011) [2023-12-27 17:13:39,469][20381] Updated weights for policy 1, policy_version 1670165 (0.0009) [2023-12-27 17:13:39,496][20347] Updated weights for policy 0, policy_version 1671894 (0.0011) [2023-12-27 17:13:39,522][20381] Updated weights for policy 1, policy_version 1670175 (0.0011) [2023-12-27 17:13:39,548][20347] Updated weights for policy 0, policy_version 1671904 (0.0010) [2023-12-27 17:13:39,578][20381] Updated weights for policy 1, policy_version 1670185 (0.0010) [2023-12-27 17:13:40,280][20347] Updated weights for policy 0, policy_version 1671914 (0.0010) [2023-12-27 17:13:40,315][20381] Updated weights for policy 1, policy_version 1670195 (0.0011) [2023-12-27 17:13:40,336][20347] Updated weights for policy 0, policy_version 1671924 (0.0010) [2023-12-27 17:13:40,367][20381] Updated weights for policy 1, policy_version 1670205 (0.0009) [2023-12-27 17:13:40,399][20347] Updated weights for policy 0, policy_version 1671934 (0.0010) [2023-12-27 17:13:40,425][20381] Updated weights for policy 1, policy_version 1670215 (0.0010) [2023-12-27 17:13:40,458][20347] Updated weights for policy 0, policy_version 1671944 (0.0008) [2023-12-27 17:13:41,155][20381] Updated weights for policy 1, policy_version 1670225 (0.0011) [2023-12-27 17:13:41,212][20381] Updated weights for policy 1, policy_version 1670235 (0.0008) [2023-12-27 17:13:41,223][20347] Updated weights for policy 0, policy_version 1671954 (0.0007) [2023-12-27 17:13:41,274][20381] Updated weights for policy 1, policy_version 1670245 (0.0010) [2023-12-27 17:13:41,289][20347] Updated weights for policy 0, policy_version 1671964 (0.0007) [2023-12-27 17:13:41,335][20381] Updated weights for policy 1, policy_version 1670255 (0.0009) [2023-12-27 17:13:41,352][20347] Updated weights for policy 0, policy_version 1671974 (0.0006) [2023-12-27 17:13:42,091][20347] Updated weights for policy 0, policy_version 1671984 (0.0011) [2023-12-27 17:13:42,094][20381] Updated weights for policy 1, policy_version 1670265 (0.0006) [2023-12-27 17:13:42,153][20381] Updated weights for policy 1, policy_version 1670275 (0.0007) [2023-12-27 17:13:42,161][20347] Updated weights for policy 0, policy_version 1671994 (0.0010) [2023-12-27 17:13:42,214][20381] Updated weights for policy 1, policy_version 1670285 (0.0006) [2023-12-27 17:13:42,216][20347] Updated weights for policy 0, policy_version 1672004 (0.0010) [2023-12-27 17:13:42,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 855752704. Throughput: 0: 10488.1, 1: 10354.8. Samples: 855740276. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:42,363][19170] Avg episode reward: [(0, '980.430'), (1, '931.750')] [2023-12-27 17:13:42,903][20381] Updated weights for policy 1, policy_version 1670295 (0.0007) [2023-12-27 17:13:42,930][20347] Updated weights for policy 0, policy_version 1672014 (0.0007) [2023-12-27 17:13:42,951][20381] Updated weights for policy 1, policy_version 1670305 (0.0005) [2023-12-27 17:13:42,982][20347] Updated weights for policy 0, policy_version 1672024 (0.0007) [2023-12-27 17:13:43,003][20381] Updated weights for policy 1, policy_version 1670315 (0.0005) [2023-12-27 17:13:43,036][20347] Updated weights for policy 0, policy_version 1672034 (0.0008) [2023-12-27 17:13:43,637][20347] Updated weights for policy 0, policy_version 1672044 (0.0007) [2023-12-27 17:13:43,645][20381] Updated weights for policy 1, policy_version 1670325 (0.0007) [2023-12-27 17:13:43,691][20347] Updated weights for policy 0, policy_version 1672054 (0.0005) [2023-12-27 17:13:43,704][20381] Updated weights for policy 1, policy_version 1670335 (0.0007) [2023-12-27 17:13:43,743][20347] Updated weights for policy 0, policy_version 1672064 (0.0005) [2023-12-27 17:13:43,758][20381] Updated weights for policy 1, policy_version 1670345 (0.0009) [2023-12-27 17:13:44,355][20381] Updated weights for policy 1, policy_version 1670355 (0.0007) [2023-12-27 17:13:44,370][20347] Updated weights for policy 0, policy_version 1672074 (0.0005) [2023-12-27 17:13:44,417][20381] Updated weights for policy 1, policy_version 1670365 (0.0008) [2023-12-27 17:13:44,427][20347] Updated weights for policy 0, policy_version 1672084 (0.0005) [2023-12-27 17:13:44,473][20381] Updated weights for policy 1, policy_version 1670375 (0.0006) [2023-12-27 17:13:44,485][20347] Updated weights for policy 0, policy_version 1672094 (0.0006) [2023-12-27 17:13:44,531][20347] Updated weights for policy 0, policy_version 1672104 (0.0006) [2023-12-27 17:13:45,111][20381] Updated weights for policy 1, policy_version 1670385 (0.0005) [2023-12-27 17:13:45,174][20381] Updated weights for policy 1, policy_version 1670395 (0.0006) [2023-12-27 17:13:45,238][20381] Updated weights for policy 1, policy_version 1670405 (0.0007) [2023-12-27 17:13:45,254][20347] Updated weights for policy 0, policy_version 1672114 (0.0006) [2023-12-27 17:13:45,295][20381] Updated weights for policy 1, policy_version 1670415 (0.0006) [2023-12-27 17:13:45,308][20347] Updated weights for policy 0, policy_version 1672124 (0.0008) [2023-12-27 17:13:45,366][20347] Updated weights for policy 0, policy_version 1672134 (0.0008) [2023-12-27 17:13:45,847][20381] Updated weights for policy 1, policy_version 1670425 (0.0005) [2023-12-27 17:13:45,899][20381] Updated weights for policy 1, policy_version 1670435 (0.0005) [2023-12-27 17:13:45,961][20381] Updated weights for policy 1, policy_version 1670445 (0.0005) [2023-12-27 17:13:46,019][20347] Updated weights for policy 0, policy_version 1672144 (0.0007) [2023-12-27 17:13:46,063][20347] Updated weights for policy 0, policy_version 1672154 (0.0005) [2023-12-27 17:13:46,114][20347] Updated weights for policy 0, policy_version 1672164 (0.0005) [2023-12-27 17:13:46,578][20381] Updated weights for policy 1, policy_version 1670455 (0.0007) [2023-12-27 17:13:46,629][20381] Updated weights for policy 1, policy_version 1670465 (0.0007) [2023-12-27 17:13:46,685][20381] Updated weights for policy 1, policy_version 1670475 (0.0007) [2023-12-27 17:13:46,716][20347] Updated weights for policy 0, policy_version 1672174 (0.0006) [2023-12-27 17:13:46,776][20347] Updated weights for policy 0, policy_version 1672184 (0.0006) [2023-12-27 17:13:46,837][20347] Updated weights for policy 0, policy_version 1672194 (0.0006) [2023-12-27 17:13:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.2, 300 sec: 20605.0). Total num frames: 855851008. Throughput: 0: 10504.4, 1: 10413.0. Samples: 855865804. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:47,363][19170] Avg episode reward: [(0, '980.430'), (1, '882.930')] [2023-12-27 17:13:47,389][20381] Updated weights for policy 1, policy_version 1670485 (0.0008) [2023-12-27 17:13:47,447][20381] Updated weights for policy 1, policy_version 1670495 (0.0006) [2023-12-27 17:13:47,511][20381] Updated weights for policy 1, policy_version 1670505 (0.0006) [2023-12-27 17:13:47,526][20347] Updated weights for policy 0, policy_version 1672204 (0.0008) [2023-12-27 17:13:47,577][20347] Updated weights for policy 0, policy_version 1672214 (0.0007) [2023-12-27 17:13:47,625][20347] Updated weights for policy 0, policy_version 1672224 (0.0007) [2023-12-27 17:13:48,089][20381] Updated weights for policy 1, policy_version 1670515 (0.0007) [2023-12-27 17:13:48,135][20381] Updated weights for policy 1, policy_version 1670525 (0.0007) [2023-12-27 17:13:48,189][20381] Updated weights for policy 1, policy_version 1670535 (0.0007) [2023-12-27 17:13:48,335][20347] Updated weights for policy 0, policy_version 1672234 (0.0008) [2023-12-27 17:13:48,395][20347] Updated weights for policy 0, policy_version 1672244 (0.0010) [2023-12-27 17:13:48,456][20347] Updated weights for policy 0, policy_version 1672254 (0.0010) [2023-12-27 17:13:48,506][20347] Updated weights for policy 0, policy_version 1672264 (0.0010) [2023-12-27 17:13:48,801][20381] Updated weights for policy 1, policy_version 1670545 (0.0009) [2023-12-27 17:13:48,856][20381] Updated weights for policy 1, policy_version 1670555 (0.0007) [2023-12-27 17:13:48,910][20381] Updated weights for policy 1, policy_version 1670565 (0.0007) [2023-12-27 17:13:48,967][20381] Updated weights for policy 1, policy_version 1670575 (0.0006) [2023-12-27 17:13:49,211][20347] Updated weights for policy 0, policy_version 1672274 (0.0010) [2023-12-27 17:13:49,280][20347] Updated weights for policy 0, policy_version 1672284 (0.0010) [2023-12-27 17:13:49,337][20347] Updated weights for policy 0, policy_version 1672294 (0.0008) [2023-12-27 17:13:49,587][20381] Updated weights for policy 1, policy_version 1670585 (0.0005) [2023-12-27 17:13:49,651][20381] Updated weights for policy 1, policy_version 1670595 (0.0010) [2023-12-27 17:13:49,707][20381] Updated weights for policy 1, policy_version 1670605 (0.0010) [2023-12-27 17:13:50,017][20347] Updated weights for policy 0, policy_version 1672304 (0.0007) [2023-12-27 17:13:50,077][20347] Updated weights for policy 0, policy_version 1672314 (0.0008) [2023-12-27 17:13:50,136][20347] Updated weights for policy 0, policy_version 1672324 (0.0008) [2023-12-27 17:13:50,381][20381] Updated weights for policy 1, policy_version 1670615 (0.0007) [2023-12-27 17:13:50,438][20381] Updated weights for policy 1, policy_version 1670625 (0.0006) [2023-12-27 17:13:50,492][20381] Updated weights for policy 1, policy_version 1670635 (0.0010) [2023-12-27 17:13:50,856][20347] Updated weights for policy 0, policy_version 1672334 (0.0007) [2023-12-27 17:13:50,904][20347] Updated weights for policy 0, policy_version 1672344 (0.0008) [2023-12-27 17:13:50,955][20347] Updated weights for policy 0, policy_version 1672354 (0.0008) [2023-12-27 17:13:51,145][20381] Updated weights for policy 1, policy_version 1670645 (0.0009) [2023-12-27 17:13:51,214][20381] Updated weights for policy 1, policy_version 1670655 (0.0006) [2023-12-27 17:13:51,282][20381] Updated weights for policy 1, policy_version 1670665 (0.0007) [2023-12-27 17:13:51,673][20347] Updated weights for policy 0, policy_version 1672364 (0.0007) [2023-12-27 17:13:51,740][20347] Updated weights for policy 0, policy_version 1672374 (0.0007) [2023-12-27 17:13:51,797][20347] Updated weights for policy 0, policy_version 1672384 (0.0007) [2023-12-27 17:13:51,983][20381] Updated weights for policy 1, policy_version 1670675 (0.0008) [2023-12-27 17:13:52,030][20381] Updated weights for policy 1, policy_version 1670685 (0.0008) [2023-12-27 17:13:52,084][20381] Updated weights for policy 1, policy_version 1670695 (0.0008) [2023-12-27 17:13:52,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20753.1, 300 sec: 20604.9). Total num frames: 855957504. Throughput: 0: 10431.2, 1: 10428.8. Samples: 855928316. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:52,363][19170] Avg episode reward: [(0, '990.210'), (1, '921.860')] [2023-12-27 17:13:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001670704_427745280.pth... [2023-12-27 17:13:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001672392_428212224.pth... [2023-12-27 17:13:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001669456_427425792.pth [2023-12-27 17:13:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001671208_427909120.pth [2023-12-27 17:13:52,486][20347] Updated weights for policy 0, policy_version 1672394 (0.0007) [2023-12-27 17:13:52,549][20347] Updated weights for policy 0, policy_version 1672404 (0.0008) [2023-12-27 17:13:52,606][20347] Updated weights for policy 0, policy_version 1672414 (0.0008) [2023-12-27 17:13:52,659][20347] Updated weights for policy 0, policy_version 1672424 (0.0008) [2023-12-27 17:13:52,791][20381] Updated weights for policy 1, policy_version 1670705 (0.0008) [2023-12-27 17:13:52,837][20381] Updated weights for policy 1, policy_version 1670715 (0.0008) [2023-12-27 17:13:52,890][20381] Updated weights for policy 1, policy_version 1670725 (0.0008) [2023-12-27 17:13:52,943][20381] Updated weights for policy 1, policy_version 1670735 (0.0008) [2023-12-27 17:13:53,355][20347] Updated weights for policy 0, policy_version 1672434 (0.0008) [2023-12-27 17:13:53,409][20347] Updated weights for policy 0, policy_version 1672444 (0.0007) [2023-12-27 17:13:53,464][20347] Updated weights for policy 0, policy_version 1672454 (0.0008) [2023-12-27 17:13:53,577][20381] Updated weights for policy 1, policy_version 1670745 (0.0008) [2023-12-27 17:13:53,623][20381] Updated weights for policy 1, policy_version 1670755 (0.0007) [2023-12-27 17:13:53,678][20381] Updated weights for policy 1, policy_version 1670765 (0.0008) [2023-12-27 17:13:54,099][20347] Updated weights for policy 0, policy_version 1672464 (0.0007) [2023-12-27 17:13:54,162][20347] Updated weights for policy 0, policy_version 1672474 (0.0007) [2023-12-27 17:13:54,214][20347] Updated weights for policy 0, policy_version 1672484 (0.0007) [2023-12-27 17:13:54,338][20381] Updated weights for policy 1, policy_version 1670775 (0.0006) [2023-12-27 17:13:54,394][20381] Updated weights for policy 1, policy_version 1670785 (0.0007) [2023-12-27 17:13:54,450][20381] Updated weights for policy 1, policy_version 1670795 (0.0008) [2023-12-27 17:13:54,903][20347] Updated weights for policy 0, policy_version 1672494 (0.0006) [2023-12-27 17:13:54,954][20347] Updated weights for policy 0, policy_version 1672504 (0.0006) [2023-12-27 17:13:55,015][20347] Updated weights for policy 0, policy_version 1672514 (0.0008) [2023-12-27 17:13:55,110][20292] KL-divergence is very high: 132.2117 [2023-12-27 17:13:55,139][20381] Updated weights for policy 1, policy_version 1670805 (0.0007) [2023-12-27 17:13:55,155][20292] KL-divergence is very high: 144.8968 [2023-12-27 17:13:55,192][20381] Updated weights for policy 1, policy_version 1670815 (0.0008) [2023-12-27 17:13:55,199][20292] KL-divergence is very high: 127.2720 [2023-12-27 17:13:55,242][20292] KL-divergence is very high: 121.0547 [2023-12-27 17:13:55,248][20381] Updated weights for policy 1, policy_version 1670825 (0.0008) [2023-12-27 17:13:55,678][20347] Updated weights for policy 0, policy_version 1672524 (0.0007) [2023-12-27 17:13:55,730][20347] Updated weights for policy 0, policy_version 1672534 (0.0006) [2023-12-27 17:13:55,788][20347] Updated weights for policy 0, policy_version 1672544 (0.0005) [2023-12-27 17:13:55,892][20381] Updated weights for policy 1, policy_version 1670835 (0.0007) [2023-12-27 17:13:55,952][20381] Updated weights for policy 1, policy_version 1670845 (0.0006) [2023-12-27 17:13:56,012][20381] Updated weights for policy 1, policy_version 1670855 (0.0008) [2023-12-27 17:13:56,432][20347] Updated weights for policy 0, policy_version 1672554 (0.0007) [2023-12-27 17:13:56,479][20347] Updated weights for policy 0, policy_version 1672564 (0.0007) [2023-12-27 17:13:56,523][20347] Updated weights for policy 0, policy_version 1672574 (0.0008) [2023-12-27 17:13:56,568][20347] Updated weights for policy 0, policy_version 1672584 (0.0008) [2023-12-27 17:13:56,658][20381] Updated weights for policy 1, policy_version 1670865 (0.0009) [2023-12-27 17:13:56,719][20381] Updated weights for policy 1, policy_version 1670875 (0.0005) [2023-12-27 17:13:56,770][20381] Updated weights for policy 1, policy_version 1670885 (0.0005) [2023-12-27 17:13:56,826][20381] Updated weights for policy 1, policy_version 1670895 (0.0006) [2023-12-27 17:13:57,336][20347] Updated weights for policy 0, policy_version 1672594 (0.0007) [2023-12-27 17:13:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 856055808. Throughput: 0: 10330.0, 1: 10444.1. Samples: 856050008. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:13:57,363][19170] Avg episode reward: [(0, '990.210'), (1, '863.620')] [2023-12-27 17:13:57,382][20381] Updated weights for policy 1, policy_version 1670905 (0.0007) [2023-12-27 17:13:57,388][20347] Updated weights for policy 0, policy_version 1672604 (0.0007) [2023-12-27 17:13:57,442][20381] Updated weights for policy 1, policy_version 1670915 (0.0005) [2023-12-27 17:13:57,448][20347] Updated weights for policy 0, policy_version 1672614 (0.0008) [2023-12-27 17:13:57,502][20381] Updated weights for policy 1, policy_version 1670925 (0.0005) [2023-12-27 17:13:58,042][20381] Updated weights for policy 1, policy_version 1670935 (0.0005) [2023-12-27 17:13:58,095][20347] Updated weights for policy 0, policy_version 1672624 (0.0006) [2023-12-27 17:13:58,098][20381] Updated weights for policy 1, policy_version 1670945 (0.0005) [2023-12-27 17:13:58,149][20381] Updated weights for policy 1, policy_version 1670955 (0.0005) [2023-12-27 17:13:58,152][20347] Updated weights for policy 0, policy_version 1672634 (0.0005) [2023-12-27 17:13:58,217][20347] Updated weights for policy 0, policy_version 1672644 (0.0007) [2023-12-27 17:13:58,897][20381] Updated weights for policy 1, policy_version 1670965 (0.0006) [2023-12-27 17:13:58,959][20381] Updated weights for policy 1, policy_version 1670975 (0.0008) [2023-12-27 17:13:58,965][20347] Updated weights for policy 0, policy_version 1672654 (0.0008) [2023-12-27 17:13:59,015][20381] Updated weights for policy 1, policy_version 1670985 (0.0008) [2023-12-27 17:13:59,021][20347] Updated weights for policy 0, policy_version 1672664 (0.0007) [2023-12-27 17:13:59,076][20347] Updated weights for policy 0, policy_version 1672674 (0.0006) [2023-12-27 17:13:59,760][20381] Updated weights for policy 1, policy_version 1670995 (0.0008) [2023-12-27 17:13:59,812][20381] Updated weights for policy 1, policy_version 1671005 (0.0008) [2023-12-27 17:13:59,832][20347] Updated weights for policy 0, policy_version 1672684 (0.0007) [2023-12-27 17:13:59,881][20381] Updated weights for policy 1, policy_version 1671015 (0.0007) [2023-12-27 17:13:59,893][20347] Updated weights for policy 0, policy_version 1672694 (0.0008) [2023-12-27 17:13:59,953][20347] Updated weights for policy 0, policy_version 1672704 (0.0007) [2023-12-27 17:14:00,550][20347] Updated weights for policy 0, policy_version 1672714 (0.0007) [2023-12-27 17:14:00,567][20381] Updated weights for policy 1, policy_version 1671025 (0.0006) [2023-12-27 17:14:00,608][20347] Updated weights for policy 0, policy_version 1672724 (0.0006) [2023-12-27 17:14:00,620][20381] Updated weights for policy 1, policy_version 1671035 (0.0007) [2023-12-27 17:14:00,661][20347] Updated weights for policy 0, policy_version 1672734 (0.0006) [2023-12-27 17:14:00,672][20381] Updated weights for policy 1, policy_version 1671045 (0.0007) [2023-12-27 17:14:00,711][20347] Updated weights for policy 0, policy_version 1672744 (0.0006) [2023-12-27 17:14:00,722][20381] Updated weights for policy 1, policy_version 1671055 (0.0007) [2023-12-27 17:14:01,422][20347] Updated weights for policy 0, policy_version 1672754 (0.0008) [2023-12-27 17:14:01,473][20347] Updated weights for policy 0, policy_version 1672764 (0.0006) [2023-12-27 17:14:01,487][20381] Updated weights for policy 1, policy_version 1671065 (0.0008) [2023-12-27 17:14:01,529][20347] Updated weights for policy 0, policy_version 1672774 (0.0006) [2023-12-27 17:14:01,545][20381] Updated weights for policy 1, policy_version 1671075 (0.0008) [2023-12-27 17:14:01,597][20381] Updated weights for policy 1, policy_version 1671085 (0.0009) [2023-12-27 17:14:02,185][20347] Updated weights for policy 0, policy_version 1672784 (0.0007) [2023-12-27 17:14:02,243][20347] Updated weights for policy 0, policy_version 1672794 (0.0007) [2023-12-27 17:14:02,308][20347] Updated weights for policy 0, policy_version 1672804 (0.0008) [2023-12-27 17:14:02,346][20381] Updated weights for policy 1, policy_version 1671095 (0.0006) [2023-12-27 17:14:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 856162304. Throughput: 0: 10228.5, 1: 10443.3. Samples: 856171628. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.410')] [2023-12-27 17:14:02,406][20381] Updated weights for policy 1, policy_version 1671105 (0.0007) [2023-12-27 17:14:02,464][20381] Updated weights for policy 1, policy_version 1671115 (0.0007) [2023-12-27 17:14:03,025][20347] Updated weights for policy 0, policy_version 1672814 (0.0007) [2023-12-27 17:14:03,072][20381] Updated weights for policy 1, policy_version 1671125 (0.0009) [2023-12-27 17:14:03,085][20347] Updated weights for policy 0, policy_version 1672824 (0.0005) [2023-12-27 17:14:03,130][20381] Updated weights for policy 1, policy_version 1671135 (0.0010) [2023-12-27 17:14:03,139][20347] Updated weights for policy 0, policy_version 1672834 (0.0006) [2023-12-27 17:14:03,178][20381] Updated weights for policy 1, policy_version 1671145 (0.0010) [2023-12-27 17:14:03,760][20347] Updated weights for policy 0, policy_version 1672844 (0.0005) [2023-12-27 17:14:03,794][20381] Updated weights for policy 1, policy_version 1671155 (0.0009) [2023-12-27 17:14:03,811][20347] Updated weights for policy 0, policy_version 1672854 (0.0006) [2023-12-27 17:14:03,859][20381] Updated weights for policy 1, policy_version 1671165 (0.0007) [2023-12-27 17:14:03,877][20347] Updated weights for policy 0, policy_version 1672864 (0.0009) [2023-12-27 17:14:03,915][20381] Updated weights for policy 1, policy_version 1671175 (0.0006) [2023-12-27 17:14:04,547][20347] Updated weights for policy 0, policy_version 1672874 (0.0010) [2023-12-27 17:14:04,601][20381] Updated weights for policy 1, policy_version 1671185 (0.0007) [2023-12-27 17:14:04,605][20347] Updated weights for policy 0, policy_version 1672884 (0.0010) [2023-12-27 17:14:04,651][20381] Updated weights for policy 1, policy_version 1671195 (0.0005) [2023-12-27 17:14:04,653][20347] Updated weights for policy 0, policy_version 1672894 (0.0010) [2023-12-27 17:14:04,695][20381] Updated weights for policy 1, policy_version 1671205 (0.0005) [2023-12-27 17:14:04,701][20347] Updated weights for policy 0, policy_version 1672904 (0.0010) [2023-12-27 17:14:04,745][20381] Updated weights for policy 1, policy_version 1671215 (0.0007) [2023-12-27 17:14:05,362][20347] Updated weights for policy 0, policy_version 1672914 (0.0008) [2023-12-27 17:14:05,416][20347] Updated weights for policy 0, policy_version 1672924 (0.0008) [2023-12-27 17:14:05,453][20381] Updated weights for policy 1, policy_version 1671225 (0.0007) [2023-12-27 17:14:05,462][20347] Updated weights for policy 0, policy_version 1672934 (0.0007) [2023-12-27 17:14:05,512][20381] Updated weights for policy 1, policy_version 1671235 (0.0006) [2023-12-27 17:14:05,570][20381] Updated weights for policy 1, policy_version 1671245 (0.0007) [2023-12-27 17:14:06,148][20347] Updated weights for policy 0, policy_version 1672944 (0.0008) [2023-12-27 17:14:06,211][20347] Updated weights for policy 0, policy_version 1672954 (0.0008) [2023-12-27 17:14:06,242][20381] Updated weights for policy 1, policy_version 1671255 (0.0007) [2023-12-27 17:14:06,267][20347] Updated weights for policy 0, policy_version 1672964 (0.0008) [2023-12-27 17:14:06,306][20381] Updated weights for policy 1, policy_version 1671265 (0.0007) [2023-12-27 17:14:06,371][20381] Updated weights for policy 1, policy_version 1671275 (0.0008) [2023-12-27 17:14:06,943][20347] Updated weights for policy 0, policy_version 1672974 (0.0006) [2023-12-27 17:14:06,997][20347] Updated weights for policy 0, policy_version 1672984 (0.0006) [2023-12-27 17:14:07,046][20347] Updated weights for policy 0, policy_version 1672994 (0.0008) [2023-12-27 17:14:07,059][20381] Updated weights for policy 1, policy_version 1671285 (0.0006) [2023-12-27 17:14:07,107][20381] Updated weights for policy 1, policy_version 1671295 (0.0007) [2023-12-27 17:14:07,155][20381] Updated weights for policy 1, policy_version 1671305 (0.0006) [2023-12-27 17:14:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 856268800. Throughput: 0: 10233.2, 1: 10481.1. Samples: 856234604. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:07,363][19170] Avg episode reward: [(0, '990.200'), (1, '970.760')] [2023-12-27 17:14:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001673000_428367872.pth... [2023-12-27 17:14:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001671312_427900928.pth... [2023-12-27 17:14:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001671816_428064768.pth [2023-12-27 17:14:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001670096_427589632.pth [2023-12-27 17:14:07,686][20347] Updated weights for policy 0, policy_version 1673004 (0.0007) [2023-12-27 17:14:07,752][20347] Updated weights for policy 0, policy_version 1673014 (0.0008) [2023-12-27 17:14:07,778][20381] Updated weights for policy 1, policy_version 1671315 (0.0009) [2023-12-27 17:14:07,818][20347] Updated weights for policy 0, policy_version 1673024 (0.0009) [2023-12-27 17:14:07,836][20381] Updated weights for policy 1, policy_version 1671325 (0.0005) [2023-12-27 17:14:07,891][20381] Updated weights for policy 1, policy_version 1671335 (0.0010) [2023-12-27 17:14:08,478][20347] Updated weights for policy 0, policy_version 1673034 (0.0007) [2023-12-27 17:14:08,514][20381] Updated weights for policy 1, policy_version 1671345 (0.0006) [2023-12-27 17:14:08,535][20347] Updated weights for policy 0, policy_version 1673044 (0.0007) [2023-12-27 17:14:08,571][20381] Updated weights for policy 1, policy_version 1671355 (0.0006) [2023-12-27 17:14:08,595][20347] Updated weights for policy 0, policy_version 1673054 (0.0007) [2023-12-27 17:14:08,623][20381] Updated weights for policy 1, policy_version 1671365 (0.0006) [2023-12-27 17:14:08,653][20347] Updated weights for policy 0, policy_version 1673064 (0.0008) [2023-12-27 17:14:08,679][20381] Updated weights for policy 1, policy_version 1671375 (0.0008) [2023-12-27 17:14:09,326][20347] Updated weights for policy 0, policy_version 1673074 (0.0008) [2023-12-27 17:14:09,390][20347] Updated weights for policy 0, policy_version 1673084 (0.0008) [2023-12-27 17:14:09,427][20381] Updated weights for policy 1, policy_version 1671385 (0.0007) [2023-12-27 17:14:09,456][20347] Updated weights for policy 0, policy_version 1673094 (0.0012) [2023-12-27 17:14:09,485][20381] Updated weights for policy 1, policy_version 1671395 (0.0009) [2023-12-27 17:14:09,548][20381] Updated weights for policy 1, policy_version 1671405 (0.0008) [2023-12-27 17:14:10,220][20347] Updated weights for policy 0, policy_version 1673104 (0.0006) [2023-12-27 17:14:10,283][20347] Updated weights for policy 0, policy_version 1673114 (0.0005) [2023-12-27 17:14:10,336][20347] Updated weights for policy 0, policy_version 1673124 (0.0006) [2023-12-27 17:14:10,358][20381] Updated weights for policy 1, policy_version 1671415 (0.0008) [2023-12-27 17:14:10,416][20381] Updated weights for policy 1, policy_version 1671425 (0.0009) [2023-12-27 17:14:10,478][20381] Updated weights for policy 1, policy_version 1671435 (0.0008) [2023-12-27 17:14:10,877][20347] Updated weights for policy 0, policy_version 1673134 (0.0007) [2023-12-27 17:14:10,928][20347] Updated weights for policy 0, policy_version 1673144 (0.0005) [2023-12-27 17:14:10,982][20347] Updated weights for policy 0, policy_version 1673154 (0.0005) [2023-12-27 17:14:11,259][20381] Updated weights for policy 1, policy_version 1671445 (0.0006) [2023-12-27 17:14:11,316][20381] Updated weights for policy 1, policy_version 1671455 (0.0007) [2023-12-27 17:14:11,387][20381] Updated weights for policy 1, policy_version 1671465 (0.0007) [2023-12-27 17:14:11,669][20347] Updated weights for policy 0, policy_version 1673164 (0.0008) [2023-12-27 17:14:11,739][20347] Updated weights for policy 0, policy_version 1673174 (0.0008) [2023-12-27 17:14:11,801][20347] Updated weights for policy 0, policy_version 1673184 (0.0008) [2023-12-27 17:14:12,056][20381] Updated weights for policy 1, policy_version 1671475 (0.0007) [2023-12-27 17:14:12,109][20381] Updated weights for policy 1, policy_version 1671485 (0.0008) [2023-12-27 17:14:12,170][20381] Updated weights for policy 1, policy_version 1671495 (0.0008) [2023-12-27 17:14:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 856367104. Throughput: 0: 10255.3, 1: 10436.4. Samples: 856356644. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:12,363][19170] Avg episode reward: [(0, '990.200'), (1, '922.160')] [2023-12-27 17:14:12,405][20347] Updated weights for policy 0, policy_version 1673194 (0.0008) [2023-12-27 17:14:12,453][20347] Updated weights for policy 0, policy_version 1673204 (0.0007) [2023-12-27 17:14:12,510][20347] Updated weights for policy 0, policy_version 1673214 (0.0007) [2023-12-27 17:14:12,567][20347] Updated weights for policy 0, policy_version 1673224 (0.0007) [2023-12-27 17:14:12,950][20381] Updated weights for policy 1, policy_version 1671505 (0.0009) [2023-12-27 17:14:13,009][20381] Updated weights for policy 1, policy_version 1671515 (0.0008) [2023-12-27 17:14:13,059][20381] Updated weights for policy 1, policy_version 1671525 (0.0009) [2023-12-27 17:14:13,112][20381] Updated weights for policy 1, policy_version 1671535 (0.0009) [2023-12-27 17:14:13,162][20347] Updated weights for policy 0, policy_version 1673234 (0.0007) [2023-12-27 17:14:13,222][20347] Updated weights for policy 0, policy_version 1673244 (0.0008) [2023-12-27 17:14:13,287][20347] Updated weights for policy 0, policy_version 1673254 (0.0007) [2023-12-27 17:14:13,809][20381] Updated weights for policy 1, policy_version 1671545 (0.0007) [2023-12-27 17:14:13,867][20381] Updated weights for policy 1, policy_version 1671555 (0.0007) [2023-12-27 17:14:13,924][20347] Updated weights for policy 0, policy_version 1673264 (0.0007) [2023-12-27 17:14:13,926][20381] Updated weights for policy 1, policy_version 1671565 (0.0007) [2023-12-27 17:14:13,974][20347] Updated weights for policy 0, policy_version 1673274 (0.0007) [2023-12-27 17:14:14,030][20347] Updated weights for policy 0, policy_version 1673284 (0.0007) [2023-12-27 17:14:14,611][20381] Updated weights for policy 1, policy_version 1671575 (0.0007) [2023-12-27 17:14:14,656][20347] Updated weights for policy 0, policy_version 1673294 (0.0006) [2023-12-27 17:14:14,661][20381] Updated weights for policy 1, policy_version 1671585 (0.0007) [2023-12-27 17:14:14,710][20347] Updated weights for policy 0, policy_version 1673304 (0.0005) [2023-12-27 17:14:14,713][20381] Updated weights for policy 1, policy_version 1671595 (0.0007) [2023-12-27 17:14:14,762][20347] Updated weights for policy 0, policy_version 1673314 (0.0006) [2023-12-27 17:14:15,417][20347] Updated weights for policy 0, policy_version 1673324 (0.0012) [2023-12-27 17:14:15,434][20381] Updated weights for policy 1, policy_version 1671605 (0.0008) [2023-12-27 17:14:15,472][20347] Updated weights for policy 0, policy_version 1673334 (0.0010) [2023-12-27 17:14:15,480][20381] Updated weights for policy 1, policy_version 1671615 (0.0008) [2023-12-27 17:14:15,520][20347] Updated weights for policy 0, policy_version 1673344 (0.0007) [2023-12-27 17:14:15,532][20381] Updated weights for policy 1, policy_version 1671625 (0.0007) [2023-12-27 17:14:16,149][20347] Updated weights for policy 0, policy_version 1673354 (0.0006) [2023-12-27 17:14:16,202][20381] Updated weights for policy 1, policy_version 1671635 (0.0009) [2023-12-27 17:14:16,209][20347] Updated weights for policy 0, policy_version 1673364 (0.0010) [2023-12-27 17:14:16,255][20381] Updated weights for policy 1, policy_version 1671645 (0.0009) [2023-12-27 17:14:16,259][20347] Updated weights for policy 0, policy_version 1673374 (0.0006) [2023-12-27 17:14:16,307][20381] Updated weights for policy 1, policy_version 1671655 (0.0008) [2023-12-27 17:14:16,312][20347] Updated weights for policy 0, policy_version 1673384 (0.0005) [2023-12-27 17:14:16,834][20347] Updated weights for policy 0, policy_version 1673394 (0.0010) [2023-12-27 17:14:16,894][20347] Updated weights for policy 0, policy_version 1673404 (0.0010) [2023-12-27 17:14:16,949][20347] Updated weights for policy 0, policy_version 1673414 (0.0010) [2023-12-27 17:14:16,989][20381] Updated weights for policy 1, policy_version 1671665 (0.0007) [2023-12-27 17:14:17,039][20381] Updated weights for policy 1, policy_version 1671675 (0.0008) [2023-12-27 17:14:17,088][20381] Updated weights for policy 1, policy_version 1671685 (0.0008) [2023-12-27 17:14:17,135][20381] Updated weights for policy 1, policy_version 1671695 (0.0005) [2023-12-27 17:14:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 856473600. Throughput: 0: 10336.4, 1: 10284.1. Samples: 856481176. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:17,363][19170] Avg episode reward: [(0, '990.210'), (1, '941.750')] [2023-12-27 17:14:17,586][20347] Updated weights for policy 0, policy_version 1673424 (0.0006) [2023-12-27 17:14:17,649][20347] Updated weights for policy 0, policy_version 1673434 (0.0007) [2023-12-27 17:14:17,711][20347] Updated weights for policy 0, policy_version 1673444 (0.0008) [2023-12-27 17:14:17,854][20381] Updated weights for policy 1, policy_version 1671705 (0.0007) [2023-12-27 17:14:17,910][20381] Updated weights for policy 1, policy_version 1671715 (0.0006) [2023-12-27 17:14:17,956][20381] Updated weights for policy 1, policy_version 1671725 (0.0008) [2023-12-27 17:14:18,365][20347] Updated weights for policy 0, policy_version 1673454 (0.0008) [2023-12-27 17:14:18,426][20347] Updated weights for policy 0, policy_version 1673464 (0.0008) [2023-12-27 17:14:18,479][20347] Updated weights for policy 0, policy_version 1673474 (0.0008) [2023-12-27 17:14:18,679][20381] Updated weights for policy 1, policy_version 1671735 (0.0010) [2023-12-27 17:14:18,732][20381] Updated weights for policy 1, policy_version 1671745 (0.0010) [2023-12-27 17:14:18,782][20381] Updated weights for policy 1, policy_version 1671755 (0.0010) [2023-12-27 17:14:19,111][20347] Updated weights for policy 0, policy_version 1673484 (0.0007) [2023-12-27 17:14:19,184][20347] Updated weights for policy 0, policy_version 1673494 (0.0008) [2023-12-27 17:14:19,251][20347] Updated weights for policy 0, policy_version 1673506 (0.0010) [2023-12-27 17:14:19,468][20381] Updated weights for policy 1, policy_version 1671765 (0.0008) [2023-12-27 17:14:19,538][20381] Updated weights for policy 1, policy_version 1671775 (0.0008) [2023-12-27 17:14:19,602][20381] Updated weights for policy 1, policy_version 1671785 (0.0009) [2023-12-27 17:14:19,879][20347] Updated weights for policy 0, policy_version 1673516 (0.0008) [2023-12-27 17:14:19,942][20347] Updated weights for policy 0, policy_version 1673526 (0.0008) [2023-12-27 17:14:20,013][20347] Updated weights for policy 0, policy_version 1673536 (0.0008) [2023-12-27 17:14:20,336][20381] Updated weights for policy 1, policy_version 1671795 (0.0008) [2023-12-27 17:14:20,404][20381] Updated weights for policy 1, policy_version 1671805 (0.0009) [2023-12-27 17:14:20,464][20381] Updated weights for policy 1, policy_version 1671815 (0.0007) [2023-12-27 17:14:20,669][20347] Updated weights for policy 0, policy_version 1673546 (0.0008) [2023-12-27 17:14:20,733][20347] Updated weights for policy 0, policy_version 1673556 (0.0011) [2023-12-27 17:14:20,795][20347] Updated weights for policy 0, policy_version 1673566 (0.0009) [2023-12-27 17:14:20,855][20347] Updated weights for policy 0, policy_version 1673576 (0.0010) [2023-12-27 17:14:21,178][20381] Updated weights for policy 1, policy_version 1671825 (0.0008) [2023-12-27 17:14:21,234][20381] Updated weights for policy 1, policy_version 1671835 (0.0007) [2023-12-27 17:14:21,294][20381] Updated weights for policy 1, policy_version 1671845 (0.0007) [2023-12-27 17:14:21,348][20381] Updated weights for policy 1, policy_version 1671855 (0.0008) [2023-12-27 17:14:21,556][20347] Updated weights for policy 0, policy_version 1673586 (0.0010) [2023-12-27 17:14:21,623][20347] Updated weights for policy 0, policy_version 1673596 (0.0010) [2023-12-27 17:14:21,684][20347] Updated weights for policy 0, policy_version 1673606 (0.0011) [2023-12-27 17:14:22,108][20381] Updated weights for policy 1, policy_version 1671865 (0.0008) [2023-12-27 17:14:22,155][20381] Updated weights for policy 1, policy_version 1671875 (0.0009) [2023-12-27 17:14:22,200][20381] Updated weights for policy 1, policy_version 1671885 (0.0008) [2023-12-27 17:14:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 856571904. Throughput: 0: 10283.2, 1: 10259.6. Samples: 856542340. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:22,363][19170] Avg episode reward: [(0, '990.210'), (1, '951.360')] [2023-12-27 17:14:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001671888_428048384.pth... [2023-12-27 17:14:22,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001670704_427745280.pth [2023-12-27 17:14:22,409][20347] Updated weights for policy 0, policy_version 1673616 (0.0011) [2023-12-27 17:14:22,472][20347] Updated weights for policy 0, policy_version 1673626 (0.0008) [2023-12-27 17:14:22,533][20347] Updated weights for policy 0, policy_version 1673636 (0.0008) [2023-12-27 17:14:22,555][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001673640_428531712.pth... [2023-12-27 17:14:22,559][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001672392_428212224.pth [2023-12-27 17:14:22,903][20381] Updated weights for policy 1, policy_version 1671895 (0.0007) [2023-12-27 17:14:22,962][20381] Updated weights for policy 1, policy_version 1671905 (0.0008) [2023-12-27 17:14:23,022][20381] Updated weights for policy 1, policy_version 1671915 (0.0006) [2023-12-27 17:14:23,289][20347] Updated weights for policy 0, policy_version 1673646 (0.0008) [2023-12-27 17:14:23,346][20347] Updated weights for policy 0, policy_version 1673656 (0.0006) [2023-12-27 17:14:23,402][20347] Updated weights for policy 0, policy_version 1673666 (0.0008) [2023-12-27 17:14:23,597][20381] Updated weights for policy 1, policy_version 1671925 (0.0005) [2023-12-27 17:14:23,659][20381] Updated weights for policy 1, policy_version 1671935 (0.0006) [2023-12-27 17:14:23,713][20381] Updated weights for policy 1, policy_version 1671945 (0.0007) [2023-12-27 17:14:24,103][20347] Updated weights for policy 0, policy_version 1673676 (0.0007) [2023-12-27 17:14:24,151][20347] Updated weights for policy 0, policy_version 1673686 (0.0007) [2023-12-27 17:14:24,211][20347] Updated weights for policy 0, policy_version 1673696 (0.0007) [2023-12-27 17:14:24,332][20381] Updated weights for policy 1, policy_version 1671955 (0.0008) [2023-12-27 17:14:24,385][20381] Updated weights for policy 1, policy_version 1671965 (0.0010) [2023-12-27 17:14:24,440][20381] Updated weights for policy 1, policy_version 1671975 (0.0011) [2023-12-27 17:14:24,916][20347] Updated weights for policy 0, policy_version 1673706 (0.0008) [2023-12-27 17:14:24,969][20347] Updated weights for policy 0, policy_version 1673716 (0.0007) [2023-12-27 17:14:25,028][20347] Updated weights for policy 0, policy_version 1673726 (0.0008) [2023-12-27 17:14:25,090][20347] Updated weights for policy 0, policy_version 1673736 (0.0007) [2023-12-27 17:14:25,123][20381] Updated weights for policy 1, policy_version 1671985 (0.0010) [2023-12-27 17:14:25,178][20381] Updated weights for policy 1, policy_version 1671995 (0.0010) [2023-12-27 17:14:25,233][20381] Updated weights for policy 1, policy_version 1672005 (0.0011) [2023-12-27 17:14:25,288][20381] Updated weights for policy 1, policy_version 1672015 (0.0011) [2023-12-27 17:14:25,790][20347] Updated weights for policy 0, policy_version 1673746 (0.0010) [2023-12-27 17:14:25,839][20347] Updated weights for policy 0, policy_version 1673756 (0.0010) [2023-12-27 17:14:25,885][20347] Updated weights for policy 0, policy_version 1673766 (0.0010) [2023-12-27 17:14:25,985][20381] Updated weights for policy 1, policy_version 1672025 (0.0006) [2023-12-27 17:14:26,042][20381] Updated weights for policy 1, policy_version 1672035 (0.0009) [2023-12-27 17:14:26,100][20381] Updated weights for policy 1, policy_version 1672045 (0.0005) [2023-12-27 17:14:26,491][20347] Updated weights for policy 0, policy_version 1673776 (0.0010) [2023-12-27 17:14:26,552][20347] Updated weights for policy 0, policy_version 1673786 (0.0010) [2023-12-27 17:14:26,607][20347] Updated weights for policy 0, policy_version 1673796 (0.0010) [2023-12-27 17:14:26,673][20381] Updated weights for policy 1, policy_version 1672055 (0.0009) [2023-12-27 17:14:26,722][20381] Updated weights for policy 1, policy_version 1672065 (0.0010) [2023-12-27 17:14:26,768][20381] Updated weights for policy 1, policy_version 1672075 (0.0010) [2023-12-27 17:14:27,287][20347] Updated weights for policy 0, policy_version 1673806 (0.0010) [2023-12-27 17:14:27,337][20347] Updated weights for policy 0, policy_version 1673816 (0.0008) [2023-12-27 17:14:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 856670208. Throughput: 0: 10265.6, 1: 10249.4. Samples: 856663448. Policy #0 lag: (min: 27.0, avg: 36.9, max: 59.0) [2023-12-27 17:14:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '960.970')] [2023-12-27 17:14:27,398][20347] Updated weights for policy 0, policy_version 1673826 (0.0007) [2023-12-27 17:14:27,442][20381] Updated weights for policy 1, policy_version 1672085 (0.0008) [2023-12-27 17:14:27,492][20381] Updated weights for policy 1, policy_version 1672095 (0.0006) [2023-12-27 17:14:27,546][20381] Updated weights for policy 1, policy_version 1672105 (0.0007) [2023-12-27 17:14:27,988][20347] Updated weights for policy 0, policy_version 1673836 (0.0007) [2023-12-27 17:14:28,053][20347] Updated weights for policy 0, policy_version 1673846 (0.0010) [2023-12-27 17:14:28,107][20347] Updated weights for policy 0, policy_version 1673856 (0.0010) [2023-12-27 17:14:28,188][20381] Updated weights for policy 1, policy_version 1672115 (0.0008) [2023-12-27 17:14:28,236][20381] Updated weights for policy 1, policy_version 1672125 (0.0008) [2023-12-27 17:14:28,288][20381] Updated weights for policy 1, policy_version 1672135 (0.0008) [2023-12-27 17:14:28,751][20347] Updated weights for policy 0, policy_version 1673866 (0.0010) [2023-12-27 17:14:28,810][20347] Updated weights for policy 0, policy_version 1673876 (0.0010) [2023-12-27 17:14:28,866][20347] Updated weights for policy 0, policy_version 1673886 (0.0010) [2023-12-27 17:14:28,919][20347] Updated weights for policy 0, policy_version 1673896 (0.0010) [2023-12-27 17:14:28,987][20381] Updated weights for policy 1, policy_version 1672145 (0.0007) [2023-12-27 17:14:29,043][20381] Updated weights for policy 1, policy_version 1672155 (0.0006) [2023-12-27 17:14:29,102][20381] Updated weights for policy 1, policy_version 1672165 (0.0009) [2023-12-27 17:14:29,162][20381] Updated weights for policy 1, policy_version 1672175 (0.0009) [2023-12-27 17:14:29,624][20347] Updated weights for policy 0, policy_version 1673906 (0.0010) [2023-12-27 17:14:29,673][20347] Updated weights for policy 0, policy_version 1673916 (0.0010) [2023-12-27 17:14:29,718][20347] Updated weights for policy 0, policy_version 1673926 (0.0010) [2023-12-27 17:14:29,908][20381] Updated weights for policy 1, policy_version 1672185 (0.0009) [2023-12-27 17:14:29,970][20381] Updated weights for policy 1, policy_version 1672195 (0.0009) [2023-12-27 17:14:30,026][20381] Updated weights for policy 1, policy_version 1672205 (0.0008) [2023-12-27 17:14:30,464][20347] Updated weights for policy 0, policy_version 1673936 (0.0010) [2023-12-27 17:14:30,516][20347] Updated weights for policy 0, policy_version 1673946 (0.0010) [2023-12-27 17:14:30,575][20347] Updated weights for policy 0, policy_version 1673956 (0.0009) [2023-12-27 17:14:30,688][20381] Updated weights for policy 1, policy_version 1672215 (0.0009) [2023-12-27 17:14:30,739][20381] Updated weights for policy 1, policy_version 1672225 (0.0008) [2023-12-27 17:14:30,786][20381] Updated weights for policy 1, policy_version 1672235 (0.0007) [2023-12-27 17:14:31,236][20347] Updated weights for policy 0, policy_version 1673966 (0.0010) [2023-12-27 17:14:31,301][20347] Updated weights for policy 0, policy_version 1673976 (0.0007) [2023-12-27 17:14:31,368][20347] Updated weights for policy 0, policy_version 1673986 (0.0008) [2023-12-27 17:14:31,501][20381] Updated weights for policy 1, policy_version 1672245 (0.0008) [2023-12-27 17:14:31,557][20381] Updated weights for policy 1, policy_version 1672255 (0.0007) [2023-12-27 17:14:31,618][20381] Updated weights for policy 1, policy_version 1672265 (0.0007) [2023-12-27 17:14:32,100][20347] Updated weights for policy 0, policy_version 1673996 (0.0007) [2023-12-27 17:14:32,159][20347] Updated weights for policy 0, policy_version 1674006 (0.0007) [2023-12-27 17:14:32,218][20347] Updated weights for policy 0, policy_version 1674016 (0.0007) [2023-12-27 17:14:32,230][20291] KL-divergence is very high: 129.7543 [2023-12-27 17:14:32,250][20381] Updated weights for policy 1, policy_version 1672275 (0.0007) [2023-12-27 17:14:32,313][20381] Updated weights for policy 1, policy_version 1672285 (0.0007) [2023-12-27 17:14:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 856776704. Throughput: 0: 10262.2, 1: 10213.5. Samples: 856787212. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:32,363][19170] Avg episode reward: [(0, '980.450'), (1, '922.040')] [2023-12-27 17:14:32,375][20381] Updated weights for policy 1, policy_version 1672295 (0.0008) [2023-12-27 17:14:32,930][20347] Updated weights for policy 0, policy_version 1674026 (0.0008) [2023-12-27 17:14:32,984][20381] Updated weights for policy 1, policy_version 1672305 (0.0008) [2023-12-27 17:14:32,993][20347] Updated weights for policy 0, policy_version 1674036 (0.0006) [2023-12-27 17:14:33,038][20381] Updated weights for policy 1, policy_version 1672315 (0.0007) [2023-12-27 17:14:33,053][20347] Updated weights for policy 0, policy_version 1674046 (0.0006) [2023-12-27 17:14:33,085][20381] Updated weights for policy 1, policy_version 1672325 (0.0006) [2023-12-27 17:14:33,113][20347] Updated weights for policy 0, policy_version 1674056 (0.0005) [2023-12-27 17:14:33,147][20381] Updated weights for policy 1, policy_version 1672335 (0.0008) [2023-12-27 17:14:33,625][20347] Updated weights for policy 0, policy_version 1674066 (0.0005) [2023-12-27 17:14:33,645][20291] KL-divergence is very high: 138.0695 [2023-12-27 17:14:33,675][20347] Updated weights for policy 0, policy_version 1674076 (0.0008) [2023-12-27 17:14:33,683][20291] KL-divergence is very high: 135.8477 [2023-12-27 17:14:33,721][20347] Updated weights for policy 0, policy_version 1674086 (0.0010) [2023-12-27 17:14:33,858][20381] Updated weights for policy 1, policy_version 1672345 (0.0007) [2023-12-27 17:14:33,907][20381] Updated weights for policy 1, policy_version 1672355 (0.0008) [2023-12-27 17:14:33,954][20381] Updated weights for policy 1, policy_version 1672365 (0.0009) [2023-12-27 17:14:34,405][20347] Updated weights for policy 0, policy_version 1674096 (0.0009) [2023-12-27 17:14:34,462][20347] Updated weights for policy 0, policy_version 1674106 (0.0008) [2023-12-27 17:14:34,515][20347] Updated weights for policy 0, policy_version 1674116 (0.0008) [2023-12-27 17:14:34,702][20381] Updated weights for policy 1, policy_version 1672375 (0.0007) [2023-12-27 17:14:34,762][20381] Updated weights for policy 1, policy_version 1672385 (0.0007) [2023-12-27 17:14:34,821][20381] Updated weights for policy 1, policy_version 1672395 (0.0005) [2023-12-27 17:14:35,172][20347] Updated weights for policy 0, policy_version 1674126 (0.0009) [2023-12-27 17:14:35,227][20347] Updated weights for policy 0, policy_version 1674136 (0.0008) [2023-12-27 17:14:35,283][20347] Updated weights for policy 0, policy_version 1674146 (0.0006) [2023-12-27 17:14:35,484][20381] Updated weights for policy 1, policy_version 1672405 (0.0009) [2023-12-27 17:14:35,536][20381] Updated weights for policy 1, policy_version 1672415 (0.0010) [2023-12-27 17:14:35,584][20381] Updated weights for policy 1, policy_version 1672425 (0.0009) [2023-12-27 17:14:35,926][20347] Updated weights for policy 0, policy_version 1674156 (0.0010) [2023-12-27 17:14:35,974][20347] Updated weights for policy 0, policy_version 1674166 (0.0010) [2023-12-27 17:14:36,027][20347] Updated weights for policy 0, policy_version 1674176 (0.0010) [2023-12-27 17:14:36,215][20381] Updated weights for policy 1, policy_version 1672435 (0.0008) [2023-12-27 17:14:36,273][20381] Updated weights for policy 1, policy_version 1672445 (0.0007) [2023-12-27 17:14:36,331][20381] Updated weights for policy 1, policy_version 1672455 (0.0007) [2023-12-27 17:14:36,759][20347] Updated weights for policy 0, policy_version 1674186 (0.0010) [2023-12-27 17:14:36,817][20347] Updated weights for policy 0, policy_version 1674196 (0.0010) [2023-12-27 17:14:36,875][20347] Updated weights for policy 0, policy_version 1674206 (0.0010) [2023-12-27 17:14:36,929][20347] Updated weights for policy 0, policy_version 1674216 (0.0010) [2023-12-27 17:14:37,003][20381] Updated weights for policy 1, policy_version 1672465 (0.0006) [2023-12-27 17:14:37,052][20381] Updated weights for policy 1, policy_version 1672475 (0.0005) [2023-12-27 17:14:37,101][20381] Updated weights for policy 1, policy_version 1672485 (0.0005) [2023-12-27 17:14:37,146][20381] Updated weights for policy 1, policy_version 1672495 (0.0005) [2023-12-27 17:14:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 856883200. Throughput: 0: 10319.9, 1: 10166.2. Samples: 856850184. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:37,363][19170] Avg episode reward: [(0, '970.720'), (1, '931.740')] [2023-12-27 17:14:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001672496_428204032.pth... [2023-12-27 17:14:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001674216_428679168.pth... [2023-12-27 17:14:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001671312_427900928.pth [2023-12-27 17:14:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001673000_428367872.pth [2023-12-27 17:14:37,643][20347] Updated weights for policy 0, policy_version 1674226 (0.0007) [2023-12-27 17:14:37,705][20347] Updated weights for policy 0, policy_version 1674236 (0.0008) [2023-12-27 17:14:37,766][20347] Updated weights for policy 0, policy_version 1674246 (0.0006) [2023-12-27 17:14:37,773][20381] Updated weights for policy 1, policy_version 1672505 (0.0010) [2023-12-27 17:14:37,840][20381] Updated weights for policy 1, policy_version 1672515 (0.0005) [2023-12-27 17:14:37,897][20381] Updated weights for policy 1, policy_version 1672525 (0.0007) [2023-12-27 17:14:38,439][20347] Updated weights for policy 0, policy_version 1674256 (0.0007) [2023-12-27 17:14:38,502][20347] Updated weights for policy 0, policy_version 1674266 (0.0007) [2023-12-27 17:14:38,549][20381] Updated weights for policy 1, policy_version 1672535 (0.0010) [2023-12-27 17:14:38,563][20347] Updated weights for policy 0, policy_version 1674276 (0.0006) [2023-12-27 17:14:38,606][20381] Updated weights for policy 1, policy_version 1672545 (0.0009) [2023-12-27 17:14:38,660][20381] Updated weights for policy 1, policy_version 1672555 (0.0011) [2023-12-27 17:14:39,252][20347] Updated weights for policy 0, policy_version 1674286 (0.0008) [2023-12-27 17:14:39,314][20347] Updated weights for policy 0, policy_version 1674296 (0.0007) [2023-12-27 17:14:39,342][20381] Updated weights for policy 1, policy_version 1672565 (0.0010) [2023-12-27 17:14:39,381][20347] Updated weights for policy 0, policy_version 1674306 (0.0008) [2023-12-27 17:14:39,418][20381] Updated weights for policy 1, policy_version 1672575 (0.0008) [2023-12-27 17:14:39,483][20381] Updated weights for policy 1, policy_version 1672585 (0.0007) [2023-12-27 17:14:40,172][20347] Updated weights for policy 0, policy_version 1674316 (0.0008) [2023-12-27 17:14:40,178][20381] Updated weights for policy 1, policy_version 1672595 (0.0008) [2023-12-27 17:14:40,234][20347] Updated weights for policy 0, policy_version 1674326 (0.0008) [2023-12-27 17:14:40,237][20381] Updated weights for policy 1, policy_version 1672605 (0.0007) [2023-12-27 17:14:40,293][20381] Updated weights for policy 1, policy_version 1672615 (0.0006) [2023-12-27 17:14:40,299][20347] Updated weights for policy 0, policy_version 1674336 (0.0007) [2023-12-27 17:14:40,921][20381] Updated weights for policy 1, policy_version 1672625 (0.0007) [2023-12-27 17:14:40,951][20347] Updated weights for policy 0, policy_version 1674346 (0.0005) [2023-12-27 17:14:40,980][20381] Updated weights for policy 1, policy_version 1672635 (0.0005) [2023-12-27 17:14:41,005][20347] Updated weights for policy 0, policy_version 1674356 (0.0005) [2023-12-27 17:14:41,046][20381] Updated weights for policy 1, policy_version 1672645 (0.0007) [2023-12-27 17:14:41,071][20347] Updated weights for policy 0, policy_version 1674366 (0.0008) [2023-12-27 17:14:41,105][20381] Updated weights for policy 1, policy_version 1672655 (0.0007) [2023-12-27 17:14:41,133][20347] Updated weights for policy 0, policy_version 1674376 (0.0007) [2023-12-27 17:14:41,790][20381] Updated weights for policy 1, policy_version 1672665 (0.0007) [2023-12-27 17:14:41,851][20381] Updated weights for policy 1, policy_version 1672675 (0.0006) [2023-12-27 17:14:41,880][20347] Updated weights for policy 0, policy_version 1674386 (0.0007) [2023-12-27 17:14:41,912][20381] Updated weights for policy 1, policy_version 1672685 (0.0006) [2023-12-27 17:14:41,940][20347] Updated weights for policy 0, policy_version 1674396 (0.0008) [2023-12-27 17:14:41,999][20347] Updated weights for policy 0, policy_version 1674406 (0.0007) [2023-12-27 17:14:42,363][19170] Fps is (10 sec: 20478.7, 60 sec: 20479.8, 300 sec: 20577.1). Total num frames: 856981504. Throughput: 0: 10310.0, 1: 10189.6. Samples: 856972504. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:42,364][19170] Avg episode reward: [(0, '980.510'), (1, '970.770')] [2023-12-27 17:14:42,563][20381] Updated weights for policy 1, policy_version 1672695 (0.0007) [2023-12-27 17:14:42,626][20381] Updated weights for policy 1, policy_version 1672705 (0.0007) [2023-12-27 17:14:42,668][20347] Updated weights for policy 0, policy_version 1674416 (0.0010) [2023-12-27 17:14:42,687][20381] Updated weights for policy 1, policy_version 1672715 (0.0009) [2023-12-27 17:14:42,722][20347] Updated weights for policy 0, policy_version 1674426 (0.0005) [2023-12-27 17:14:42,781][20347] Updated weights for policy 0, policy_version 1674436 (0.0011) [2023-12-27 17:14:43,371][20347] Updated weights for policy 0, policy_version 1674446 (0.0007) [2023-12-27 17:14:43,412][20381] Updated weights for policy 1, policy_version 1672725 (0.0009) [2023-12-27 17:14:43,429][20347] Updated weights for policy 0, policy_version 1674456 (0.0005) [2023-12-27 17:14:43,470][20381] Updated weights for policy 1, policy_version 1672735 (0.0010) [2023-12-27 17:14:43,481][20347] Updated weights for policy 0, policy_version 1674466 (0.0005) [2023-12-27 17:14:43,525][20381] Updated weights for policy 1, policy_version 1672745 (0.0010) [2023-12-27 17:14:44,040][20347] Updated weights for policy 0, policy_version 1674476 (0.0005) [2023-12-27 17:14:44,095][20347] Updated weights for policy 0, policy_version 1674486 (0.0005) [2023-12-27 17:14:44,154][20347] Updated weights for policy 0, policy_version 1674496 (0.0007) [2023-12-27 17:14:44,158][20381] Updated weights for policy 1, policy_version 1672755 (0.0009) [2023-12-27 17:14:44,224][20381] Updated weights for policy 1, policy_version 1672765 (0.0008) [2023-12-27 17:14:44,288][20381] Updated weights for policy 1, policy_version 1672775 (0.0008) [2023-12-27 17:14:44,769][20347] Updated weights for policy 0, policy_version 1674506 (0.0007) [2023-12-27 17:14:44,827][20347] Updated weights for policy 0, policy_version 1674516 (0.0008) [2023-12-27 17:14:44,883][20347] Updated weights for policy 0, policy_version 1674526 (0.0008) [2023-12-27 17:14:44,944][20347] Updated weights for policy 0, policy_version 1674536 (0.0007) [2023-12-27 17:14:44,987][20381] Updated weights for policy 1, policy_version 1672785 (0.0007) [2023-12-27 17:14:45,045][20381] Updated weights for policy 1, policy_version 1672795 (0.0011) [2023-12-27 17:14:45,102][20381] Updated weights for policy 1, policy_version 1672805 (0.0011) [2023-12-27 17:14:45,156][20381] Updated weights for policy 1, policy_version 1672815 (0.0011) [2023-12-27 17:14:45,626][20347] Updated weights for policy 0, policy_version 1674546 (0.0005) [2023-12-27 17:14:45,679][20347] Updated weights for policy 0, policy_version 1674556 (0.0006) [2023-12-27 17:14:45,733][20347] Updated weights for policy 0, policy_version 1674566 (0.0008) [2023-12-27 17:14:45,852][20381] Updated weights for policy 1, policy_version 1672825 (0.0011) [2023-12-27 17:14:45,910][20381] Updated weights for policy 1, policy_version 1672835 (0.0011) [2023-12-27 17:14:45,971][20381] Updated weights for policy 1, policy_version 1672845 (0.0011) [2023-12-27 17:14:46,353][20347] Updated weights for policy 0, policy_version 1674576 (0.0005) [2023-12-27 17:14:46,399][20347] Updated weights for policy 0, policy_version 1674586 (0.0005) [2023-12-27 17:14:46,453][20347] Updated weights for policy 0, policy_version 1674596 (0.0005) [2023-12-27 17:14:46,681][20381] Updated weights for policy 1, policy_version 1672855 (0.0009) [2023-12-27 17:14:46,740][20381] Updated weights for policy 1, policy_version 1672865 (0.0009) [2023-12-27 17:14:46,805][20381] Updated weights for policy 1, policy_version 1672875 (0.0010) [2023-12-27 17:14:47,018][20347] Updated weights for policy 0, policy_version 1674606 (0.0006) [2023-12-27 17:14:47,067][20347] Updated weights for policy 0, policy_version 1674616 (0.0005) [2023-12-27 17:14:47,117][20347] Updated weights for policy 0, policy_version 1674626 (0.0005) [2023-12-27 17:14:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 857088000. Throughput: 0: 10431.9, 1: 10144.2. Samples: 857097552. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.510')] [2023-12-27 17:14:47,449][20381] Updated weights for policy 1, policy_version 1672885 (0.0008) [2023-12-27 17:14:47,511][20381] Updated weights for policy 1, policy_version 1672895 (0.0010) [2023-12-27 17:14:47,567][20381] Updated weights for policy 1, policy_version 1672905 (0.0010) [2023-12-27 17:14:47,735][20347] Updated weights for policy 0, policy_version 1674636 (0.0005) [2023-12-27 17:14:47,792][20347] Updated weights for policy 0, policy_version 1674646 (0.0009) [2023-12-27 17:14:47,845][20347] Updated weights for policy 0, policy_version 1674656 (0.0009) [2023-12-27 17:14:48,162][20381] Updated weights for policy 1, policy_version 1672915 (0.0010) [2023-12-27 17:14:48,214][20381] Updated weights for policy 1, policy_version 1672925 (0.0010) [2023-12-27 17:14:48,266][20381] Updated weights for policy 1, policy_version 1672935 (0.0010) [2023-12-27 17:14:48,549][20347] Updated weights for policy 0, policy_version 1674666 (0.0007) [2023-12-27 17:14:48,611][20347] Updated weights for policy 0, policy_version 1674676 (0.0008) [2023-12-27 17:14:48,660][20347] Updated weights for policy 0, policy_version 1674686 (0.0008) [2023-12-27 17:14:48,722][20347] Updated weights for policy 0, policy_version 1674696 (0.0007) [2023-12-27 17:14:49,019][20381] Updated weights for policy 1, policy_version 1672945 (0.0010) [2023-12-27 17:14:49,068][20381] Updated weights for policy 1, policy_version 1672955 (0.0010) [2023-12-27 17:14:49,128][20381] Updated weights for policy 1, policy_version 1672965 (0.0010) [2023-12-27 17:14:49,192][20381] Updated weights for policy 1, policy_version 1672975 (0.0010) [2023-12-27 17:14:49,444][20347] Updated weights for policy 0, policy_version 1674706 (0.0008) [2023-12-27 17:14:49,502][20347] Updated weights for policy 0, policy_version 1674716 (0.0008) [2023-12-27 17:14:49,561][20347] Updated weights for policy 0, policy_version 1674726 (0.0008) [2023-12-27 17:14:49,882][20381] Updated weights for policy 1, policy_version 1672985 (0.0007) [2023-12-27 17:14:49,949][20381] Updated weights for policy 1, policy_version 1672995 (0.0008) [2023-12-27 17:14:50,010][20381] Updated weights for policy 1, policy_version 1673005 (0.0008) [2023-12-27 17:14:50,274][20347] Updated weights for policy 0, policy_version 1674736 (0.0008) [2023-12-27 17:14:50,336][20347] Updated weights for policy 0, policy_version 1674746 (0.0007) [2023-12-27 17:14:50,401][20347] Updated weights for policy 0, policy_version 1674756 (0.0008) [2023-12-27 17:14:50,723][20381] Updated weights for policy 1, policy_version 1673015 (0.0006) [2023-12-27 17:14:50,784][20381] Updated weights for policy 1, policy_version 1673025 (0.0005) [2023-12-27 17:14:50,843][20381] Updated weights for policy 1, policy_version 1673035 (0.0007) [2023-12-27 17:14:51,115][20347] Updated weights for policy 0, policy_version 1674766 (0.0007) [2023-12-27 17:14:51,177][20347] Updated weights for policy 0, policy_version 1674776 (0.0007) [2023-12-27 17:14:51,230][20347] Updated weights for policy 0, policy_version 1674786 (0.0006) [2023-12-27 17:14:51,494][20381] Updated weights for policy 1, policy_version 1673045 (0.0007) [2023-12-27 17:14:51,556][20381] Updated weights for policy 1, policy_version 1673055 (0.0007) [2023-12-27 17:14:51,618][20381] Updated weights for policy 1, policy_version 1673065 (0.0008) [2023-12-27 17:14:51,906][20347] Updated weights for policy 0, policy_version 1674796 (0.0007) [2023-12-27 17:14:51,967][20347] Updated weights for policy 0, policy_version 1674806 (0.0005) [2023-12-27 17:14:52,023][20347] Updated weights for policy 0, policy_version 1674816 (0.0006) [2023-12-27 17:14:52,352][20381] Updated weights for policy 1, policy_version 1673075 (0.0009) [2023-12-27 17:14:52,362][19170] Fps is (10 sec: 20481.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 857186304. Throughput: 0: 10406.7, 1: 10121.7. Samples: 857158384. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.380')] [2023-12-27 17:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001674824_428834816.pth... [2023-12-27 17:14:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001673640_428531712.pth [2023-12-27 17:14:52,418][20381] Updated weights for policy 1, policy_version 1673085 (0.0006) [2023-12-27 17:14:52,473][20381] Updated weights for policy 1, policy_version 1673095 (0.0006) [2023-12-27 17:14:52,527][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001673104_428359680.pth... [2023-12-27 17:14:52,530][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001671888_428048384.pth [2023-12-27 17:14:52,644][20347] Updated weights for policy 0, policy_version 1674826 (0.0008) [2023-12-27 17:14:52,709][20347] Updated weights for policy 0, policy_version 1674836 (0.0008) [2023-12-27 17:14:52,772][20347] Updated weights for policy 0, policy_version 1674846 (0.0008) [2023-12-27 17:14:52,824][20347] Updated weights for policy 0, policy_version 1674856 (0.0007) [2023-12-27 17:14:53,149][20381] Updated weights for policy 1, policy_version 1673105 (0.0006) [2023-12-27 17:14:53,201][20381] Updated weights for policy 1, policy_version 1673115 (0.0006) [2023-12-27 17:14:53,257][20381] Updated weights for policy 1, policy_version 1673125 (0.0007) [2023-12-27 17:14:53,317][20381] Updated weights for policy 1, policy_version 1673135 (0.0007) [2023-12-27 17:14:53,511][20347] Updated weights for policy 0, policy_version 1674866 (0.0008) [2023-12-27 17:14:53,567][20347] Updated weights for policy 0, policy_version 1674876 (0.0010) [2023-12-27 17:14:53,619][20347] Updated weights for policy 0, policy_version 1674886 (0.0010) [2023-12-27 17:14:53,914][20381] Updated weights for policy 1, policy_version 1673145 (0.0008) [2023-12-27 17:14:53,965][20381] Updated weights for policy 1, policy_version 1673155 (0.0005) [2023-12-27 17:14:54,028][20381] Updated weights for policy 1, policy_version 1673165 (0.0005) [2023-12-27 17:14:54,212][20347] Updated weights for policy 0, policy_version 1674896 (0.0006) [2023-12-27 17:14:54,279][20347] Updated weights for policy 0, policy_version 1674906 (0.0005) [2023-12-27 17:14:54,336][20347] Updated weights for policy 0, policy_version 1674916 (0.0005) [2023-12-27 17:14:54,634][20381] Updated weights for policy 1, policy_version 1673175 (0.0007) [2023-12-27 17:14:54,687][20381] Updated weights for policy 1, policy_version 1673185 (0.0008) [2023-12-27 17:14:54,740][20381] Updated weights for policy 1, policy_version 1673195 (0.0009) [2023-12-27 17:14:54,834][20347] Updated weights for policy 0, policy_version 1674926 (0.0005) [2023-12-27 17:14:54,890][20347] Updated weights for policy 0, policy_version 1674936 (0.0005) [2023-12-27 17:14:54,944][20347] Updated weights for policy 0, policy_version 1674946 (0.0007) [2023-12-27 17:14:55,461][20381] Updated weights for policy 1, policy_version 1673205 (0.0006) [2023-12-27 17:14:55,523][20381] Updated weights for policy 1, policy_version 1673215 (0.0009) [2023-12-27 17:14:55,549][20347] Updated weights for policy 0, policy_version 1674956 (0.0008) [2023-12-27 17:14:55,577][20381] Updated weights for policy 1, policy_version 1673225 (0.0007) [2023-12-27 17:14:55,601][20347] Updated weights for policy 0, policy_version 1674966 (0.0010) [2023-12-27 17:14:55,648][20347] Updated weights for policy 0, policy_version 1674976 (0.0010) [2023-12-27 17:14:56,131][20381] Updated weights for policy 1, policy_version 1673235 (0.0006) [2023-12-27 17:14:56,191][20381] Updated weights for policy 1, policy_version 1673245 (0.0008) [2023-12-27 17:14:56,252][20381] Updated weights for policy 1, policy_version 1673255 (0.0008) [2023-12-27 17:14:56,340][20347] Updated weights for policy 0, policy_version 1674986 (0.0010) [2023-12-27 17:14:56,397][20347] Updated weights for policy 0, policy_version 1674996 (0.0010) [2023-12-27 17:14:56,451][20347] Updated weights for policy 0, policy_version 1675006 (0.0010) [2023-12-27 17:14:56,494][20347] Updated weights for policy 0, policy_version 1675016 (0.0009) [2023-12-27 17:14:56,809][20381] Updated weights for policy 1, policy_version 1673265 (0.0007) [2023-12-27 17:14:56,855][20381] Updated weights for policy 1, policy_version 1673275 (0.0007) [2023-12-27 17:14:56,906][20381] Updated weights for policy 1, policy_version 1673285 (0.0005) [2023-12-27 17:14:56,962][20381] Updated weights for policy 1, policy_version 1673295 (0.0006) [2023-12-27 17:14:57,114][20347] Updated weights for policy 0, policy_version 1675026 (0.0010) [2023-12-27 17:14:57,171][20347] Updated weights for policy 0, policy_version 1675036 (0.0010) [2023-12-27 17:14:57,229][20347] Updated weights for policy 0, policy_version 1675046 (0.0010) [2023-12-27 17:14:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 857300992. Throughput: 0: 10441.7, 1: 10200.6. Samples: 857285544. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:14:57,363][19170] Avg episode reward: [(0, '990.230'), (1, '912.270')] [2023-12-27 17:14:57,603][20381] Updated weights for policy 1, policy_version 1673305 (0.0005) [2023-12-27 17:14:57,649][20381] Updated weights for policy 1, policy_version 1673315 (0.0007) [2023-12-27 17:14:57,699][20381] Updated weights for policy 1, policy_version 1673325 (0.0005) [2023-12-27 17:14:57,885][20347] Updated weights for policy 0, policy_version 1675056 (0.0010) [2023-12-27 17:14:57,935][20347] Updated weights for policy 0, policy_version 1675066 (0.0010) [2023-12-27 17:14:57,983][20347] Updated weights for policy 0, policy_version 1675076 (0.0010) [2023-12-27 17:14:58,305][20381] Updated weights for policy 1, policy_version 1673335 (0.0007) [2023-12-27 17:14:58,369][20381] Updated weights for policy 1, policy_version 1673345 (0.0008) [2023-12-27 17:14:58,434][20381] Updated weights for policy 1, policy_version 1673355 (0.0008) [2023-12-27 17:14:58,709][20347] Updated weights for policy 0, policy_version 1675086 (0.0009) [2023-12-27 17:14:58,772][20347] Updated weights for policy 0, policy_version 1675096 (0.0007) [2023-12-27 17:14:58,834][20347] Updated weights for policy 0, policy_version 1675106 (0.0008) [2023-12-27 17:14:59,242][20381] Updated weights for policy 1, policy_version 1673365 (0.0008) [2023-12-27 17:14:59,306][20381] Updated weights for policy 1, policy_version 1673375 (0.0007) [2023-12-27 17:14:59,386][20381] Updated weights for policy 1, policy_version 1673385 (0.0007) [2023-12-27 17:14:59,585][20347] Updated weights for policy 0, policy_version 1675116 (0.0007) [2023-12-27 17:14:59,640][20347] Updated weights for policy 0, policy_version 1675126 (0.0007) [2023-12-27 17:14:59,690][20347] Updated weights for policy 0, policy_version 1675136 (0.0007) [2023-12-27 17:15:00,033][20381] Updated weights for policy 1, policy_version 1673395 (0.0008) [2023-12-27 17:15:00,093][20381] Updated weights for policy 1, policy_version 1673405 (0.0006) [2023-12-27 17:15:00,152][20381] Updated weights for policy 1, policy_version 1673415 (0.0006) [2023-12-27 17:15:00,331][20347] Updated weights for policy 0, policy_version 1675146 (0.0007) [2023-12-27 17:15:00,378][20347] Updated weights for policy 0, policy_version 1675156 (0.0008) [2023-12-27 17:15:00,435][20347] Updated weights for policy 0, policy_version 1675166 (0.0008) [2023-12-27 17:15:00,495][20347] Updated weights for policy 0, policy_version 1675176 (0.0007) [2023-12-27 17:15:00,796][20381] Updated weights for policy 1, policy_version 1673425 (0.0006) [2023-12-27 17:15:00,855][20381] Updated weights for policy 1, policy_version 1673435 (0.0008) [2023-12-27 17:15:00,912][20381] Updated weights for policy 1, policy_version 1673445 (0.0007) [2023-12-27 17:15:00,965][20381] Updated weights for policy 1, policy_version 1673455 (0.0009) [2023-12-27 17:15:01,075][20347] Updated weights for policy 0, policy_version 1675186 (0.0008) [2023-12-27 17:15:01,126][20347] Updated weights for policy 0, policy_version 1675196 (0.0008) [2023-12-27 17:15:01,191][20347] Updated weights for policy 0, policy_version 1675206 (0.0008) [2023-12-27 17:15:01,688][20381] Updated weights for policy 1, policy_version 1673465 (0.0008) [2023-12-27 17:15:01,744][20381] Updated weights for policy 1, policy_version 1673475 (0.0010) [2023-12-27 17:15:01,790][20381] Updated weights for policy 1, policy_version 1673485 (0.0010) [2023-12-27 17:15:01,980][20347] Updated weights for policy 0, policy_version 1675216 (0.0010) [2023-12-27 17:15:02,040][20347] Updated weights for policy 0, policy_version 1675226 (0.0011) [2023-12-27 17:15:02,099][20347] Updated weights for policy 0, policy_version 1675236 (0.0011) [2023-12-27 17:15:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 857399296. Throughput: 0: 10355.7, 1: 10258.5. Samples: 857408816. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:02,364][19170] Avg episode reward: [(0, '990.230'), (1, '922.020')] [2023-12-27 17:15:02,495][20381] Updated weights for policy 1, policy_version 1673495 (0.0008) [2023-12-27 17:15:02,538][20381] Updated weights for policy 1, policy_version 1673505 (0.0005) [2023-12-27 17:15:02,591][20381] Updated weights for policy 1, policy_version 1673515 (0.0006) [2023-12-27 17:15:02,737][20347] Updated weights for policy 0, policy_version 1675246 (0.0009) [2023-12-27 17:15:02,795][20347] Updated weights for policy 0, policy_version 1675256 (0.0008) [2023-12-27 17:15:02,863][20347] Updated weights for policy 0, policy_version 1675266 (0.0009) [2023-12-27 17:15:03,208][20381] Updated weights for policy 1, policy_version 1673525 (0.0009) [2023-12-27 17:15:03,255][20381] Updated weights for policy 1, policy_version 1673535 (0.0010) [2023-12-27 17:15:03,303][20381] Updated weights for policy 1, policy_version 1673545 (0.0010) [2023-12-27 17:15:03,468][20347] Updated weights for policy 0, policy_version 1675276 (0.0008) [2023-12-27 17:15:03,519][20347] Updated weights for policy 0, policy_version 1675286 (0.0005) [2023-12-27 17:15:03,567][20347] Updated weights for policy 0, policy_version 1675296 (0.0005) [2023-12-27 17:15:03,955][20381] Updated weights for policy 1, policy_version 1673555 (0.0009) [2023-12-27 17:15:04,012][20381] Updated weights for policy 1, policy_version 1673565 (0.0007) [2023-12-27 17:15:04,066][20381] Updated weights for policy 1, policy_version 1673575 (0.0009) [2023-12-27 17:15:04,101][20292] KL-divergence is very high: 134.5994 [2023-12-27 17:15:04,170][20347] Updated weights for policy 0, policy_version 1675306 (0.0005) [2023-12-27 17:15:04,230][20347] Updated weights for policy 0, policy_version 1675316 (0.0008) [2023-12-27 17:15:04,294][20347] Updated weights for policy 0, policy_version 1675326 (0.0007) [2023-12-27 17:15:04,355][20347] Updated weights for policy 0, policy_version 1675336 (0.0008) [2023-12-27 17:15:04,804][20381] Updated weights for policy 1, policy_version 1673585 (0.0008) [2023-12-27 17:15:04,856][20381] Updated weights for policy 1, policy_version 1673595 (0.0010) [2023-12-27 17:15:04,909][20381] Updated weights for policy 1, policy_version 1673605 (0.0010) [2023-12-27 17:15:04,961][20381] Updated weights for policy 1, policy_version 1673615 (0.0007) [2023-12-27 17:15:05,046][20347] Updated weights for policy 0, policy_version 1675346 (0.0007) [2023-12-27 17:15:05,105][20347] Updated weights for policy 0, policy_version 1675356 (0.0008) [2023-12-27 17:15:05,156][20347] Updated weights for policy 0, policy_version 1675366 (0.0007) [2023-12-27 17:15:05,646][20381] Updated weights for policy 1, policy_version 1673625 (0.0009) [2023-12-27 17:15:05,689][20381] Updated weights for policy 1, policy_version 1673635 (0.0007) [2023-12-27 17:15:05,738][20381] Updated weights for policy 1, policy_version 1673645 (0.0009) [2023-12-27 17:15:05,788][20347] Updated weights for policy 0, policy_version 1675376 (0.0008) [2023-12-27 17:15:05,845][20347] Updated weights for policy 0, policy_version 1675386 (0.0007) [2023-12-27 17:15:05,895][20347] Updated weights for policy 0, policy_version 1675396 (0.0007) [2023-12-27 17:15:06,521][20381] Updated weights for policy 1, policy_version 1673655 (0.0008) [2023-12-27 17:15:06,588][20381] Updated weights for policy 1, policy_version 1673665 (0.0008) [2023-12-27 17:15:06,590][20347] Updated weights for policy 0, policy_version 1675406 (0.0005) [2023-12-27 17:15:06,650][20381] Updated weights for policy 1, policy_version 1673675 (0.0006) [2023-12-27 17:15:06,652][20347] Updated weights for policy 0, policy_version 1675416 (0.0006) [2023-12-27 17:15:06,717][20347] Updated weights for policy 0, policy_version 1675426 (0.0006) [2023-12-27 17:15:07,336][20381] Updated weights for policy 1, policy_version 1673685 (0.0007) [2023-12-27 17:15:07,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20479.9, 300 sec: 20604.9). Total num frames: 857497600. Throughput: 0: 10371.3, 1: 10292.5. Samples: 857472212. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '941.520')] [2023-12-27 17:15:07,381][20347] Updated weights for policy 0, policy_version 1675436 (0.0008) [2023-12-27 17:15:07,387][20381] Updated weights for policy 1, policy_version 1673695 (0.0006) [2023-12-27 17:15:07,430][20347] Updated weights for policy 0, policy_version 1675446 (0.0010) [2023-12-27 17:15:07,442][20381] Updated weights for policy 1, policy_version 1673705 (0.0005) [2023-12-27 17:15:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001673712_428515328.pth... [2023-12-27 17:15:07,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001672496_428204032.pth [2023-12-27 17:15:07,487][20347] Updated weights for policy 0, policy_version 1675456 (0.0010) [2023-12-27 17:15:07,532][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001675464_428998656.pth... [2023-12-27 17:15:07,536][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001674216_428679168.pth [2023-12-27 17:15:08,129][20347] Updated weights for policy 0, policy_version 1675466 (0.0009) [2023-12-27 17:15:08,156][20381] Updated weights for policy 1, policy_version 1673715 (0.0006) [2023-12-27 17:15:08,182][20347] Updated weights for policy 0, policy_version 1675476 (0.0005) [2023-12-27 17:15:08,212][20381] Updated weights for policy 1, policy_version 1673725 (0.0008) [2023-12-27 17:15:08,234][20347] Updated weights for policy 0, policy_version 1675486 (0.0005) [2023-12-27 17:15:08,263][20381] Updated weights for policy 1, policy_version 1673735 (0.0007) [2023-12-27 17:15:08,287][20347] Updated weights for policy 0, policy_version 1675496 (0.0008) [2023-12-27 17:15:08,935][20381] Updated weights for policy 1, policy_version 1673745 (0.0005) [2023-12-27 17:15:09,000][20381] Updated weights for policy 1, policy_version 1673755 (0.0008) [2023-12-27 17:15:09,002][20347] Updated weights for policy 0, policy_version 1675506 (0.0008) [2023-12-27 17:15:09,048][20347] Updated weights for policy 0, policy_version 1675516 (0.0007) [2023-12-27 17:15:09,061][20381] Updated weights for policy 1, policy_version 1673765 (0.0007) [2023-12-27 17:15:09,097][20347] Updated weights for policy 0, policy_version 1675526 (0.0007) [2023-12-27 17:15:09,121][20381] Updated weights for policy 1, policy_version 1673775 (0.0009) [2023-12-27 17:15:09,789][20381] Updated weights for policy 1, policy_version 1673785 (0.0010) [2023-12-27 17:15:09,805][20347] Updated weights for policy 0, policy_version 1675536 (0.0008) [2023-12-27 17:15:09,856][20381] Updated weights for policy 1, policy_version 1673795 (0.0010) [2023-12-27 17:15:09,869][20347] Updated weights for policy 0, policy_version 1675546 (0.0009) [2023-12-27 17:15:09,906][20381] Updated weights for policy 1, policy_version 1673805 (0.0007) [2023-12-27 17:15:09,927][20347] Updated weights for policy 0, policy_version 1675556 (0.0007) [2023-12-27 17:15:10,591][20347] Updated weights for policy 0, policy_version 1675566 (0.0011) [2023-12-27 17:15:10,647][20347] Updated weights for policy 0, policy_version 1675576 (0.0010) [2023-12-27 17:15:10,661][20381] Updated weights for policy 1, policy_version 1673815 (0.0010) [2023-12-27 17:15:10,704][20347] Updated weights for policy 0, policy_version 1675586 (0.0009) [2023-12-27 17:15:10,721][20381] Updated weights for policy 1, policy_version 1673825 (0.0011) [2023-12-27 17:15:10,777][20381] Updated weights for policy 1, policy_version 1673835 (0.0011) [2023-12-27 17:15:11,368][20347] Updated weights for policy 0, policy_version 1675596 (0.0009) [2023-12-27 17:15:11,434][20347] Updated weights for policy 0, policy_version 1675606 (0.0008) [2023-12-27 17:15:11,491][20347] Updated weights for policy 0, policy_version 1675616 (0.0007) [2023-12-27 17:15:11,561][20381] Updated weights for policy 1, policy_version 1673845 (0.0009) [2023-12-27 17:15:11,611][20381] Updated weights for policy 1, policy_version 1673855 (0.0008) [2023-12-27 17:15:11,677][20381] Updated weights for policy 1, policy_version 1673865 (0.0008) [2023-12-27 17:15:12,221][20347] Updated weights for policy 0, policy_version 1675626 (0.0007) [2023-12-27 17:15:12,281][20347] Updated weights for policy 0, policy_version 1675636 (0.0007) [2023-12-27 17:15:12,317][20381] Updated weights for policy 1, policy_version 1673875 (0.0009) [2023-12-27 17:15:12,339][20347] Updated weights for policy 0, policy_version 1675646 (0.0007) [2023-12-27 17:15:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 857595904. Throughput: 0: 10426.5, 1: 10224.2. Samples: 857592728. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:12,363][19170] Avg episode reward: [(0, '990.210'), (1, '921.980')] [2023-12-27 17:15:12,383][20381] Updated weights for policy 1, policy_version 1673885 (0.0009) [2023-12-27 17:15:12,395][20347] Updated weights for policy 0, policy_version 1675656 (0.0007) [2023-12-27 17:15:12,439][20381] Updated weights for policy 1, policy_version 1673895 (0.0010) [2023-12-27 17:15:13,079][20381] Updated weights for policy 1, policy_version 1673905 (0.0010) [2023-12-27 17:15:13,115][20347] Updated weights for policy 0, policy_version 1675666 (0.0007) [2023-12-27 17:15:13,127][20381] Updated weights for policy 1, policy_version 1673915 (0.0010) [2023-12-27 17:15:13,169][20347] Updated weights for policy 0, policy_version 1675676 (0.0006) [2023-12-27 17:15:13,182][20381] Updated weights for policy 1, policy_version 1673925 (0.0010) [2023-12-27 17:15:13,220][20347] Updated weights for policy 0, policy_version 1675686 (0.0005) [2023-12-27 17:15:13,236][20381] Updated weights for policy 1, policy_version 1673935 (0.0009) [2023-12-27 17:15:13,846][20347] Updated weights for policy 0, policy_version 1675696 (0.0006) [2023-12-27 17:15:13,880][20381] Updated weights for policy 1, policy_version 1673945 (0.0008) [2023-12-27 17:15:13,897][20347] Updated weights for policy 0, policy_version 1675706 (0.0005) [2023-12-27 17:15:13,933][20381] Updated weights for policy 1, policy_version 1673955 (0.0010) [2023-12-27 17:15:13,954][20347] Updated weights for policy 0, policy_version 1675716 (0.0006) [2023-12-27 17:15:13,984][20381] Updated weights for policy 1, policy_version 1673965 (0.0010) [2023-12-27 17:15:14,551][20347] Updated weights for policy 0, policy_version 1675726 (0.0010) [2023-12-27 17:15:14,611][20347] Updated weights for policy 0, policy_version 1675736 (0.0009) [2023-12-27 17:15:14,660][20381] Updated weights for policy 1, policy_version 1673975 (0.0009) [2023-12-27 17:15:14,671][20347] Updated weights for policy 0, policy_version 1675746 (0.0008) [2023-12-27 17:15:14,721][20381] Updated weights for policy 1, policy_version 1673985 (0.0010) [2023-12-27 17:15:14,790][20381] Updated weights for policy 1, policy_version 1673995 (0.0011) [2023-12-27 17:15:15,384][20347] Updated weights for policy 0, policy_version 1675756 (0.0009) [2023-12-27 17:15:15,433][20347] Updated weights for policy 0, policy_version 1675766 (0.0007) [2023-12-27 17:15:15,447][20381] Updated weights for policy 1, policy_version 1674005 (0.0011) [2023-12-27 17:15:15,486][20347] Updated weights for policy 0, policy_version 1675776 (0.0006) [2023-12-27 17:15:15,500][20381] Updated weights for policy 1, policy_version 1674015 (0.0008) [2023-12-27 17:15:15,553][20381] Updated weights for policy 1, policy_version 1674025 (0.0007) [2023-12-27 17:15:16,179][20381] Updated weights for policy 1, policy_version 1674035 (0.0007) [2023-12-27 17:15:16,193][20347] Updated weights for policy 0, policy_version 1675786 (0.0005) [2023-12-27 17:15:16,228][20381] Updated weights for policy 1, policy_version 1674045 (0.0005) [2023-12-27 17:15:16,246][20347] Updated weights for policy 0, policy_version 1675796 (0.0007) [2023-12-27 17:15:16,286][20381] Updated weights for policy 1, policy_version 1674055 (0.0005) [2023-12-27 17:15:16,296][20347] Updated weights for policy 0, policy_version 1675806 (0.0008) [2023-12-27 17:15:16,344][20347] Updated weights for policy 0, policy_version 1675816 (0.0007) [2023-12-27 17:15:16,941][20347] Updated weights for policy 0, policy_version 1675826 (0.0008) [2023-12-27 17:15:16,965][20381] Updated weights for policy 1, policy_version 1674065 (0.0006) [2023-12-27 17:15:16,987][20347] Updated weights for policy 0, policy_version 1675836 (0.0006) [2023-12-27 17:15:17,015][20381] Updated weights for policy 1, policy_version 1674075 (0.0007) [2023-12-27 17:15:17,037][20347] Updated weights for policy 0, policy_version 1675846 (0.0006) [2023-12-27 17:15:17,058][20381] Updated weights for policy 1, policy_version 1674085 (0.0007) [2023-12-27 17:15:17,103][20381] Updated weights for policy 1, policy_version 1674095 (0.0008) [2023-12-27 17:15:17,363][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 857710592. Throughput: 0: 10435.4, 1: 10240.9. Samples: 857717648. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '902.440')] [2023-12-27 17:15:17,636][20347] Updated weights for policy 0, policy_version 1675856 (0.0007) [2023-12-27 17:15:17,693][20347] Updated weights for policy 0, policy_version 1675866 (0.0005) [2023-12-27 17:15:17,753][20347] Updated weights for policy 0, policy_version 1675876 (0.0007) [2023-12-27 17:15:17,853][20381] Updated weights for policy 1, policy_version 1674105 (0.0007) [2023-12-27 17:15:17,909][20381] Updated weights for policy 1, policy_version 1674115 (0.0007) [2023-12-27 17:15:17,968][20381] Updated weights for policy 1, policy_version 1674125 (0.0008) [2023-12-27 17:15:18,506][20347] Updated weights for policy 0, policy_version 1675886 (0.0008) [2023-12-27 17:15:18,569][20347] Updated weights for policy 0, policy_version 1675896 (0.0008) [2023-12-27 17:15:18,621][20381] Updated weights for policy 1, policy_version 1674135 (0.0006) [2023-12-27 17:15:18,630][20347] Updated weights for policy 0, policy_version 1675906 (0.0008) [2023-12-27 17:15:18,671][20381] Updated weights for policy 1, policy_version 1674145 (0.0007) [2023-12-27 17:15:18,729][20381] Updated weights for policy 1, policy_version 1674155 (0.0007) [2023-12-27 17:15:19,217][20347] Updated weights for policy 0, policy_version 1675916 (0.0007) [2023-12-27 17:15:19,278][20347] Updated weights for policy 0, policy_version 1675926 (0.0008) [2023-12-27 17:15:19,341][20347] Updated weights for policy 0, policy_version 1675936 (0.0008) [2023-12-27 17:15:19,462][20381] Updated weights for policy 1, policy_version 1674165 (0.0008) [2023-12-27 17:15:19,522][20381] Updated weights for policy 1, policy_version 1674175 (0.0008) [2023-12-27 17:15:19,571][20381] Updated weights for policy 1, policy_version 1674185 (0.0008) [2023-12-27 17:15:20,016][20347] Updated weights for policy 0, policy_version 1675946 (0.0008) [2023-12-27 17:15:20,081][20347] Updated weights for policy 0, policy_version 1675956 (0.0009) [2023-12-27 17:15:20,141][20347] Updated weights for policy 0, policy_version 1675966 (0.0007) [2023-12-27 17:15:20,204][20347] Updated weights for policy 0, policy_version 1675976 (0.0008) [2023-12-27 17:15:20,352][20381] Updated weights for policy 1, policy_version 1674195 (0.0009) [2023-12-27 17:15:20,411][20381] Updated weights for policy 1, policy_version 1674205 (0.0011) [2023-12-27 17:15:20,470][20381] Updated weights for policy 1, policy_version 1674215 (0.0011) [2023-12-27 17:15:20,902][20347] Updated weights for policy 0, policy_version 1675986 (0.0007) [2023-12-27 17:15:20,959][20347] Updated weights for policy 0, policy_version 1675996 (0.0008) [2023-12-27 17:15:21,024][20347] Updated weights for policy 0, policy_version 1676006 (0.0007) [2023-12-27 17:15:21,182][20381] Updated weights for policy 1, policy_version 1674225 (0.0010) [2023-12-27 17:15:21,238][20381] Updated weights for policy 1, policy_version 1674235 (0.0011) [2023-12-27 17:15:21,311][20381] Updated weights for policy 1, policy_version 1674245 (0.0011) [2023-12-27 17:15:21,367][20381] Updated weights for policy 1, policy_version 1674255 (0.0007) [2023-12-27 17:15:21,743][20347] Updated weights for policy 0, policy_version 1676016 (0.0008) [2023-12-27 17:15:21,803][20347] Updated weights for policy 0, policy_version 1676026 (0.0007) [2023-12-27 17:15:21,863][20347] Updated weights for policy 0, policy_version 1676036 (0.0007) [2023-12-27 17:15:22,139][20381] Updated weights for policy 1, policy_version 1674265 (0.0008) [2023-12-27 17:15:22,193][20381] Updated weights for policy 1, policy_version 1674275 (0.0005) [2023-12-27 17:15:22,258][20381] Updated weights for policy 1, policy_version 1674285 (0.0007) [2023-12-27 17:15:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 857808896. Throughput: 0: 10413.4, 1: 10206.0. Samples: 857778060. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:22,363][19170] Avg episode reward: [(0, '990.250'), (1, '921.960')] [2023-12-27 17:15:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001674288_428662784.pth... [2023-12-27 17:15:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001676040_429146112.pth... [2023-12-27 17:15:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001673104_428359680.pth [2023-12-27 17:15:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001674824_428834816.pth [2023-12-27 17:15:22,519][20347] Updated weights for policy 0, policy_version 1676046 (0.0008) [2023-12-27 17:15:22,582][20347] Updated weights for policy 0, policy_version 1676056 (0.0006) [2023-12-27 17:15:22,645][20347] Updated weights for policy 0, policy_version 1676066 (0.0009) [2023-12-27 17:15:22,959][20381] Updated weights for policy 1, policy_version 1674295 (0.0007) [2023-12-27 17:15:23,010][20381] Updated weights for policy 1, policy_version 1674305 (0.0006) [2023-12-27 17:15:23,055][20381] Updated weights for policy 1, policy_version 1674315 (0.0005) [2023-12-27 17:15:23,215][20347] Updated weights for policy 0, policy_version 1676076 (0.0007) [2023-12-27 17:15:23,275][20347] Updated weights for policy 0, policy_version 1676086 (0.0006) [2023-12-27 17:15:23,329][20347] Updated weights for policy 0, policy_version 1676096 (0.0007) [2023-12-27 17:15:23,641][20381] Updated weights for policy 1, policy_version 1674325 (0.0007) [2023-12-27 17:15:23,688][20381] Updated weights for policy 1, policy_version 1674335 (0.0005) [2023-12-27 17:15:23,734][20381] Updated weights for policy 1, policy_version 1674345 (0.0005) [2023-12-27 17:15:23,887][20347] Updated weights for policy 0, policy_version 1676106 (0.0008) [2023-12-27 17:15:23,943][20347] Updated weights for policy 0, policy_version 1676116 (0.0005) [2023-12-27 17:15:23,994][20347] Updated weights for policy 0, policy_version 1676126 (0.0005) [2023-12-27 17:15:24,041][20347] Updated weights for policy 0, policy_version 1676136 (0.0007) [2023-12-27 17:15:24,382][20381] Updated weights for policy 1, policy_version 1674355 (0.0008) [2023-12-27 17:15:24,434][20381] Updated weights for policy 1, policy_version 1674365 (0.0010) [2023-12-27 17:15:24,491][20381] Updated weights for policy 1, policy_version 1674375 (0.0010) [2023-12-27 17:15:24,673][20347] Updated weights for policy 0, policy_version 1676146 (0.0008) [2023-12-27 17:15:24,730][20347] Updated weights for policy 0, policy_version 1676156 (0.0007) [2023-12-27 17:15:24,792][20347] Updated weights for policy 0, policy_version 1676166 (0.0007) [2023-12-27 17:15:25,160][20381] Updated weights for policy 1, policy_version 1674385 (0.0009) [2023-12-27 17:15:25,221][20381] Updated weights for policy 1, policy_version 1674395 (0.0005) [2023-12-27 17:15:25,279][20381] Updated weights for policy 1, policy_version 1674405 (0.0005) [2023-12-27 17:15:25,336][20381] Updated weights for policy 1, policy_version 1674415 (0.0005) [2023-12-27 17:15:25,458][20347] Updated weights for policy 0, policy_version 1676176 (0.0008) [2023-12-27 17:15:25,512][20347] Updated weights for policy 0, policy_version 1676186 (0.0008) [2023-12-27 17:15:25,570][20347] Updated weights for policy 0, policy_version 1676196 (0.0007) [2023-12-27 17:15:25,856][20381] Updated weights for policy 1, policy_version 1674425 (0.0005) [2023-12-27 17:15:25,912][20381] Updated weights for policy 1, policy_version 1674435 (0.0006) [2023-12-27 17:15:25,964][20381] Updated weights for policy 1, policy_version 1674445 (0.0009) [2023-12-27 17:15:26,239][20347] Updated weights for policy 0, policy_version 1676206 (0.0008) [2023-12-27 17:15:26,290][20347] Updated weights for policy 0, policy_version 1676216 (0.0007) [2023-12-27 17:15:26,335][20347] Updated weights for policy 0, policy_version 1676226 (0.0007) [2023-12-27 17:15:26,625][20381] Updated weights for policy 1, policy_version 1674455 (0.0007) [2023-12-27 17:15:26,709][20381] Updated weights for policy 1, policy_version 1674465 (0.0007) [2023-12-27 17:15:26,755][20381] Updated weights for policy 1, policy_version 1674475 (0.0007) [2023-12-27 17:15:27,049][20347] Updated weights for policy 0, policy_version 1676236 (0.0008) [2023-12-27 17:15:27,107][20347] Updated weights for policy 0, policy_version 1676246 (0.0007) [2023-12-27 17:15:27,166][20347] Updated weights for policy 0, policy_version 1676256 (0.0006) [2023-12-27 17:15:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20752.9, 300 sec: 20632.7). Total num frames: 857915392. Throughput: 0: 10504.7, 1: 10209.4. Samples: 857904632. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:27,364][19170] Avg episode reward: [(0, '990.250'), (1, '951.160')] [2023-12-27 17:15:27,401][20381] Updated weights for policy 1, policy_version 1674485 (0.0007) [2023-12-27 17:15:27,466][20381] Updated weights for policy 1, policy_version 1674495 (0.0008) [2023-12-27 17:15:27,526][20381] Updated weights for policy 1, policy_version 1674505 (0.0008) [2023-12-27 17:15:27,764][20347] Updated weights for policy 0, policy_version 1676266 (0.0006) [2023-12-27 17:15:27,831][20347] Updated weights for policy 0, policy_version 1676276 (0.0007) [2023-12-27 17:15:27,879][20347] Updated weights for policy 0, policy_version 1676286 (0.0005) [2023-12-27 17:15:27,925][20347] Updated weights for policy 0, policy_version 1676296 (0.0008) [2023-12-27 17:15:28,245][20381] Updated weights for policy 1, policy_version 1674515 (0.0007) [2023-12-27 17:15:28,297][20381] Updated weights for policy 1, policy_version 1674525 (0.0007) [2023-12-27 17:15:28,353][20381] Updated weights for policy 1, policy_version 1674535 (0.0007) [2023-12-27 17:15:28,601][20347] Updated weights for policy 0, policy_version 1676306 (0.0008) [2023-12-27 17:15:28,662][20347] Updated weights for policy 0, policy_version 1676316 (0.0008) [2023-12-27 17:15:28,723][20347] Updated weights for policy 0, policy_version 1676326 (0.0007) [2023-12-27 17:15:29,021][20381] Updated weights for policy 1, policy_version 1674545 (0.0008) [2023-12-27 17:15:29,076][20381] Updated weights for policy 1, policy_version 1674555 (0.0008) [2023-12-27 17:15:29,127][20381] Updated weights for policy 1, policy_version 1674565 (0.0007) [2023-12-27 17:15:29,182][20381] Updated weights for policy 1, policy_version 1674575 (0.0005) [2023-12-27 17:15:29,431][20347] Updated weights for policy 0, policy_version 1676336 (0.0009) [2023-12-27 17:15:29,483][20347] Updated weights for policy 0, policy_version 1676346 (0.0007) [2023-12-27 17:15:29,532][20347] Updated weights for policy 0, policy_version 1676356 (0.0007) [2023-12-27 17:15:29,858][20381] Updated weights for policy 1, policy_version 1674585 (0.0009) [2023-12-27 17:15:29,924][20381] Updated weights for policy 1, policy_version 1674595 (0.0007) [2023-12-27 17:15:29,986][20381] Updated weights for policy 1, policy_version 1674605 (0.0007) [2023-12-27 17:15:30,279][20347] Updated weights for policy 0, policy_version 1676366 (0.0007) [2023-12-27 17:15:30,338][20347] Updated weights for policy 0, policy_version 1676376 (0.0008) [2023-12-27 17:15:30,392][20347] Updated weights for policy 0, policy_version 1676386 (0.0007) [2023-12-27 17:15:30,628][20381] Updated weights for policy 1, policy_version 1674615 (0.0006) [2023-12-27 17:15:30,688][20381] Updated weights for policy 1, policy_version 1674625 (0.0006) [2023-12-27 17:15:30,750][20381] Updated weights for policy 1, policy_version 1674635 (0.0006) [2023-12-27 17:15:30,995][20347] Updated weights for policy 0, policy_version 1676396 (0.0006) [2023-12-27 17:15:31,056][20347] Updated weights for policy 0, policy_version 1676406 (0.0009) [2023-12-27 17:15:31,125][20347] Updated weights for policy 0, policy_version 1676416 (0.0008) [2023-12-27 17:15:31,333][20381] Updated weights for policy 1, policy_version 1674645 (0.0007) [2023-12-27 17:15:31,405][20381] Updated weights for policy 1, policy_version 1674655 (0.0008) [2023-12-27 17:15:31,469][20381] Updated weights for policy 1, policy_version 1674665 (0.0006) [2023-12-27 17:15:31,891][20347] Updated weights for policy 0, policy_version 1676426 (0.0007) [2023-12-27 17:15:31,952][20347] Updated weights for policy 0, policy_version 1676436 (0.0006) [2023-12-27 17:15:32,003][20347] Updated weights for policy 0, policy_version 1676446 (0.0007) [2023-12-27 17:15:32,056][20347] Updated weights for policy 0, policy_version 1676456 (0.0007) [2023-12-27 17:15:32,122][20381] Updated weights for policy 1, policy_version 1674675 (0.0006) [2023-12-27 17:15:32,174][20381] Updated weights for policy 1, policy_version 1674685 (0.0007) [2023-12-27 17:15:32,224][20381] Updated weights for policy 1, policy_version 1674695 (0.0007) [2023-12-27 17:15:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 858021888. Throughput: 0: 10406.5, 1: 10271.8. Samples: 858028076. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.000')] [2023-12-27 17:15:32,770][20347] Updated weights for policy 0, policy_version 1676466 (0.0007) [2023-12-27 17:15:32,817][20347] Updated weights for policy 0, policy_version 1676476 (0.0008) [2023-12-27 17:15:32,878][20347] Updated weights for policy 0, policy_version 1676486 (0.0008) [2023-12-27 17:15:32,918][20381] Updated weights for policy 1, policy_version 1674705 (0.0008) [2023-12-27 17:15:32,976][20381] Updated weights for policy 1, policy_version 1674715 (0.0010) [2023-12-27 17:15:33,031][20381] Updated weights for policy 1, policy_version 1674725 (0.0007) [2023-12-27 17:15:33,085][20381] Updated weights for policy 1, policy_version 1674735 (0.0005) [2023-12-27 17:15:33,545][20347] Updated weights for policy 0, policy_version 1676496 (0.0010) [2023-12-27 17:15:33,596][20347] Updated weights for policy 0, policy_version 1676506 (0.0010) [2023-12-27 17:15:33,647][20347] Updated weights for policy 0, policy_version 1676516 (0.0010) [2023-12-27 17:15:33,684][20381] Updated weights for policy 1, policy_version 1674745 (0.0005) [2023-12-27 17:15:33,726][20381] Updated weights for policy 1, policy_version 1674755 (0.0005) [2023-12-27 17:15:33,771][20381] Updated weights for policy 1, policy_version 1674765 (0.0005) [2023-12-27 17:15:34,335][20347] Updated weights for policy 0, policy_version 1676526 (0.0010) [2023-12-27 17:15:34,393][20347] Updated weights for policy 0, policy_version 1676536 (0.0010) [2023-12-27 17:15:34,414][20381] Updated weights for policy 1, policy_version 1674775 (0.0005) [2023-12-27 17:15:34,450][20347] Updated weights for policy 0, policy_version 1676546 (0.0010) [2023-12-27 17:15:34,467][20381] Updated weights for policy 1, policy_version 1674785 (0.0006) [2023-12-27 17:15:34,526][20381] Updated weights for policy 1, policy_version 1674795 (0.0008) [2023-12-27 17:15:35,089][20347] Updated weights for policy 0, policy_version 1676556 (0.0009) [2023-12-27 17:15:35,140][20347] Updated weights for policy 0, policy_version 1676566 (0.0007) [2023-12-27 17:15:35,193][20347] Updated weights for policy 0, policy_version 1676576 (0.0008) [2023-12-27 17:15:35,229][20381] Updated weights for policy 1, policy_version 1674805 (0.0008) [2023-12-27 17:15:35,281][20381] Updated weights for policy 1, policy_version 1674815 (0.0008) [2023-12-27 17:15:35,330][20381] Updated weights for policy 1, policy_version 1674825 (0.0007) [2023-12-27 17:15:35,788][20347] Updated weights for policy 0, policy_version 1676586 (0.0006) [2023-12-27 17:15:35,841][20347] Updated weights for policy 0, policy_version 1676596 (0.0005) [2023-12-27 17:15:35,886][20347] Updated weights for policy 0, policy_version 1676606 (0.0005) [2023-12-27 17:15:35,942][20347] Updated weights for policy 0, policy_version 1676616 (0.0007) [2023-12-27 17:15:36,066][20381] Updated weights for policy 1, policy_version 1674835 (0.0008) [2023-12-27 17:15:36,133][20381] Updated weights for policy 1, policy_version 1674845 (0.0007) [2023-12-27 17:15:36,193][20381] Updated weights for policy 1, policy_version 1674855 (0.0007) [2023-12-27 17:15:36,580][20347] Updated weights for policy 0, policy_version 1676626 (0.0005) [2023-12-27 17:15:36,646][20347] Updated weights for policy 0, policy_version 1676636 (0.0006) [2023-12-27 17:15:36,705][20347] Updated weights for policy 0, policy_version 1676646 (0.0007) [2023-12-27 17:15:36,771][20381] Updated weights for policy 1, policy_version 1674865 (0.0005) [2023-12-27 17:15:36,831][20381] Updated weights for policy 1, policy_version 1674875 (0.0005) [2023-12-27 17:15:36,882][20381] Updated weights for policy 1, policy_version 1674885 (0.0008) [2023-12-27 17:15:36,934][20381] Updated weights for policy 1, policy_version 1674895 (0.0008) [2023-12-27 17:15:37,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 858120192. Throughput: 0: 10425.6, 1: 10289.7. Samples: 858090572. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:37,363][19170] Avg episode reward: [(0, '990.230'), (1, '824.370')] [2023-12-27 17:15:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001674896_428818432.pth... [2023-12-27 17:15:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001673712_428515328.pth [2023-12-27 17:15:37,381][20347] Updated weights for policy 0, policy_version 1676656 (0.0008) [2023-12-27 17:15:37,438][20347] Updated weights for policy 0, policy_version 1676666 (0.0008) [2023-12-27 17:15:37,489][20347] Updated weights for policy 0, policy_version 1676676 (0.0007) [2023-12-27 17:15:37,509][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001676680_429309952.pth... [2023-12-27 17:15:37,513][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001675464_428998656.pth [2023-12-27 17:15:37,523][20381] Updated weights for policy 1, policy_version 1674905 (0.0008) [2023-12-27 17:15:37,582][20381] Updated weights for policy 1, policy_version 1674915 (0.0007) [2023-12-27 17:15:37,639][20381] Updated weights for policy 1, policy_version 1674925 (0.0008) [2023-12-27 17:15:38,214][20347] Updated weights for policy 0, policy_version 1676686 (0.0008) [2023-12-27 17:15:38,272][20347] Updated weights for policy 0, policy_version 1676696 (0.0007) [2023-12-27 17:15:38,305][20381] Updated weights for policy 1, policy_version 1674935 (0.0008) [2023-12-27 17:15:38,330][20347] Updated weights for policy 0, policy_version 1676706 (0.0008) [2023-12-27 17:15:38,360][20381] Updated weights for policy 1, policy_version 1674945 (0.0007) [2023-12-27 17:15:38,410][20381] Updated weights for policy 1, policy_version 1674955 (0.0007) [2023-12-27 17:15:39,016][20347] Updated weights for policy 0, policy_version 1676716 (0.0008) [2023-12-27 17:15:39,071][20347] Updated weights for policy 0, policy_version 1676726 (0.0005) [2023-12-27 17:15:39,124][20347] Updated weights for policy 0, policy_version 1676736 (0.0006) [2023-12-27 17:15:39,124][20381] Updated weights for policy 1, policy_version 1674965 (0.0008) [2023-12-27 17:15:39,181][20381] Updated weights for policy 1, policy_version 1674975 (0.0009) [2023-12-27 17:15:39,229][20381] Updated weights for policy 1, policy_version 1674985 (0.0008) [2023-12-27 17:15:39,772][20347] Updated weights for policy 0, policy_version 1676746 (0.0005) [2023-12-27 17:15:39,835][20347] Updated weights for policy 0, policy_version 1676756 (0.0008) [2023-12-27 17:15:39,905][20347] Updated weights for policy 0, policy_version 1676766 (0.0008) [2023-12-27 17:15:39,968][20347] Updated weights for policy 0, policy_version 1676776 (0.0007) [2023-12-27 17:15:40,030][20381] Updated weights for policy 1, policy_version 1674995 (0.0007) [2023-12-27 17:15:40,093][20381] Updated weights for policy 1, policy_version 1675005 (0.0007) [2023-12-27 17:15:40,159][20381] Updated weights for policy 1, policy_version 1675015 (0.0008) [2023-12-27 17:15:40,672][20347] Updated weights for policy 0, policy_version 1676786 (0.0008) [2023-12-27 17:15:40,730][20347] Updated weights for policy 0, policy_version 1676796 (0.0008) [2023-12-27 17:15:40,784][20347] Updated weights for policy 0, policy_version 1676806 (0.0007) [2023-12-27 17:15:40,813][20381] Updated weights for policy 1, policy_version 1675025 (0.0007) [2023-12-27 17:15:40,871][20381] Updated weights for policy 1, policy_version 1675035 (0.0008) [2023-12-27 17:15:40,937][20381] Updated weights for policy 1, policy_version 1675045 (0.0008) [2023-12-27 17:15:40,995][20381] Updated weights for policy 1, policy_version 1675055 (0.0008) [2023-12-27 17:15:41,545][20347] Updated weights for policy 0, policy_version 1676816 (0.0008) [2023-12-27 17:15:41,607][20347] Updated weights for policy 0, policy_version 1676826 (0.0007) [2023-12-27 17:15:41,688][20347] Updated weights for policy 0, policy_version 1676836 (0.0008) [2023-12-27 17:15:41,800][20381] Updated weights for policy 1, policy_version 1675065 (0.0007) [2023-12-27 17:15:41,863][20381] Updated weights for policy 1, policy_version 1675075 (0.0008) [2023-12-27 17:15:41,923][20381] Updated weights for policy 1, policy_version 1675085 (0.0008) [2023-12-27 17:15:42,363][19170] Fps is (10 sec: 19660.0, 60 sec: 20616.6, 300 sec: 20604.9). Total num frames: 858218496. Throughput: 0: 10355.0, 1: 10251.6. Samples: 858212852. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:42,364][19170] Avg episode reward: [(0, '990.230'), (1, '873.220')] [2023-12-27 17:15:42,420][20347] Updated weights for policy 0, policy_version 1676846 (0.0008) [2023-12-27 17:15:42,476][20347] Updated weights for policy 0, policy_version 1676856 (0.0007) [2023-12-27 17:15:42,543][20347] Updated weights for policy 0, policy_version 1676866 (0.0006) [2023-12-27 17:15:42,661][20381] Updated weights for policy 1, policy_version 1675095 (0.0008) [2023-12-27 17:15:42,728][20381] Updated weights for policy 1, policy_version 1675105 (0.0008) [2023-12-27 17:15:42,778][20381] Updated weights for policy 1, policy_version 1675115 (0.0008) [2023-12-27 17:15:43,183][20347] Updated weights for policy 0, policy_version 1676876 (0.0006) [2023-12-27 17:15:43,236][20347] Updated weights for policy 0, policy_version 1676886 (0.0010) [2023-12-27 17:15:43,299][20347] Updated weights for policy 0, policy_version 1676896 (0.0005) [2023-12-27 17:15:43,470][20381] Updated weights for policy 1, policy_version 1675125 (0.0008) [2023-12-27 17:15:43,521][20381] Updated weights for policy 1, policy_version 1675135 (0.0009) [2023-12-27 17:15:43,534][20292] KL-divergence is very high: 118.6359 [2023-12-27 17:15:43,579][20381] Updated weights for policy 1, policy_version 1675145 (0.0008) [2023-12-27 17:15:43,881][20347] Updated weights for policy 0, policy_version 1676906 (0.0007) [2023-12-27 17:15:43,939][20347] Updated weights for policy 0, policy_version 1676916 (0.0010) [2023-12-27 17:15:43,998][20347] Updated weights for policy 0, policy_version 1676926 (0.0010) [2023-12-27 17:15:44,055][20347] Updated weights for policy 0, policy_version 1676936 (0.0010) [2023-12-27 17:15:44,259][20381] Updated weights for policy 1, policy_version 1675155 (0.0009) [2023-12-27 17:15:44,312][20381] Updated weights for policy 1, policy_version 1675165 (0.0010) [2023-12-27 17:15:44,364][20381] Updated weights for policy 1, policy_version 1675175 (0.0010) [2023-12-27 17:15:44,749][20347] Updated weights for policy 0, policy_version 1676946 (0.0008) [2023-12-27 17:15:44,808][20347] Updated weights for policy 0, policy_version 1676956 (0.0010) [2023-12-27 17:15:44,867][20347] Updated weights for policy 0, policy_version 1676966 (0.0010) [2023-12-27 17:15:45,054][20381] Updated weights for policy 1, policy_version 1675185 (0.0010) [2023-12-27 17:15:45,114][20381] Updated weights for policy 1, policy_version 1675195 (0.0008) [2023-12-27 17:15:45,171][20381] Updated weights for policy 1, policy_version 1675205 (0.0008) [2023-12-27 17:15:45,240][20381] Updated weights for policy 1, policy_version 1675215 (0.0008) [2023-12-27 17:15:45,608][20347] Updated weights for policy 0, policy_version 1676976 (0.0008) [2023-12-27 17:15:45,664][20347] Updated weights for policy 0, policy_version 1676986 (0.0007) [2023-12-27 17:15:45,723][20347] Updated weights for policy 0, policy_version 1676996 (0.0008) [2023-12-27 17:15:45,936][20381] Updated weights for policy 1, policy_version 1675225 (0.0010) [2023-12-27 17:15:45,992][20381] Updated weights for policy 1, policy_version 1675235 (0.0011) [2023-12-27 17:15:46,050][20381] Updated weights for policy 1, policy_version 1675245 (0.0011) [2023-12-27 17:15:46,378][20347] Updated weights for policy 0, policy_version 1677006 (0.0008) [2023-12-27 17:15:46,431][20347] Updated weights for policy 0, policy_version 1677016 (0.0010) [2023-12-27 17:15:46,488][20347] Updated weights for policy 0, policy_version 1677026 (0.0009) [2023-12-27 17:15:46,746][20381] Updated weights for policy 1, policy_version 1675255 (0.0011) [2023-12-27 17:15:46,798][20381] Updated weights for policy 1, policy_version 1675265 (0.0011) [2023-12-27 17:15:46,851][20381] Updated weights for policy 1, policy_version 1675275 (0.0009) [2023-12-27 17:15:47,182][20347] Updated weights for policy 0, policy_version 1677036 (0.0008) [2023-12-27 17:15:47,244][20347] Updated weights for policy 0, policy_version 1677046 (0.0010) [2023-12-27 17:15:47,305][20347] Updated weights for policy 0, policy_version 1677056 (0.0010) [2023-12-27 17:15:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 858324992. Throughput: 0: 10327.7, 1: 10208.5. Samples: 858332940. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:47,363][19170] Avg episode reward: [(0, '990.200'), (1, '912.310')] [2023-12-27 17:15:47,479][20381] Updated weights for policy 1, policy_version 1675285 (0.0010) [2023-12-27 17:15:47,538][20381] Updated weights for policy 1, policy_version 1675295 (0.0011) [2023-12-27 17:15:47,600][20381] Updated weights for policy 1, policy_version 1675305 (0.0011) [2023-12-27 17:15:47,993][20347] Updated weights for policy 0, policy_version 1677066 (0.0010) [2023-12-27 17:15:48,063][20347] Updated weights for policy 0, policy_version 1677076 (0.0008) [2023-12-27 17:15:48,124][20347] Updated weights for policy 0, policy_version 1677086 (0.0009) [2023-12-27 17:15:48,192][20347] Updated weights for policy 0, policy_version 1677096 (0.0008) [2023-12-27 17:15:48,270][20381] Updated weights for policy 1, policy_version 1675315 (0.0009) [2023-12-27 17:15:48,332][20381] Updated weights for policy 1, policy_version 1675325 (0.0008) [2023-12-27 17:15:48,400][20381] Updated weights for policy 1, policy_version 1675335 (0.0008) [2023-12-27 17:15:48,915][20347] Updated weights for policy 0, policy_version 1677106 (0.0005) [2023-12-27 17:15:48,975][20347] Updated weights for policy 0, policy_version 1677116 (0.0006) [2023-12-27 17:15:49,038][20347] Updated weights for policy 0, policy_version 1677126 (0.0005) [2023-12-27 17:15:49,121][20381] Updated weights for policy 1, policy_version 1675345 (0.0008) [2023-12-27 17:15:49,184][20381] Updated weights for policy 1, policy_version 1675355 (0.0008) [2023-12-27 17:15:49,249][20381] Updated weights for policy 1, policy_version 1675365 (0.0008) [2023-12-27 17:15:49,312][20381] Updated weights for policy 1, policy_version 1675375 (0.0008) [2023-12-27 17:15:49,673][20347] Updated weights for policy 0, policy_version 1677136 (0.0008) [2023-12-27 17:15:49,742][20347] Updated weights for policy 0, policy_version 1677147 (0.0007) [2023-12-27 17:15:49,798][20347] Updated weights for policy 0, policy_version 1677157 (0.0006) [2023-12-27 17:15:49,986][20381] Updated weights for policy 1, policy_version 1675385 (0.0008) [2023-12-27 17:15:50,035][20381] Updated weights for policy 1, policy_version 1675395 (0.0008) [2023-12-27 17:15:50,091][20381] Updated weights for policy 1, policy_version 1675405 (0.0008) [2023-12-27 17:15:50,513][20347] Updated weights for policy 0, policy_version 1677167 (0.0010) [2023-12-27 17:15:50,570][20347] Updated weights for policy 0, policy_version 1677177 (0.0011) [2023-12-27 17:15:50,622][20347] Updated weights for policy 0, policy_version 1677187 (0.0009) [2023-12-27 17:15:50,896][20381] Updated weights for policy 1, policy_version 1675415 (0.0008) [2023-12-27 17:15:50,947][20381] Updated weights for policy 1, policy_version 1675425 (0.0008) [2023-12-27 17:15:51,001][20381] Updated weights for policy 1, policy_version 1675435 (0.0005) [2023-12-27 17:15:51,289][20347] Updated weights for policy 0, policy_version 1677197 (0.0007) [2023-12-27 17:15:51,354][20347] Updated weights for policy 0, policy_version 1677207 (0.0008) [2023-12-27 17:15:51,417][20347] Updated weights for policy 0, policy_version 1677217 (0.0008) [2023-12-27 17:15:51,742][20381] Updated weights for policy 1, policy_version 1675445 (0.0007) [2023-12-27 17:15:51,792][20381] Updated weights for policy 1, policy_version 1675455 (0.0007) [2023-12-27 17:15:51,850][20381] Updated weights for policy 1, policy_version 1675465 (0.0005) [2023-12-27 17:15:52,065][20347] Updated weights for policy 0, policy_version 1677227 (0.0007) [2023-12-27 17:15:52,118][20347] Updated weights for policy 0, policy_version 1677237 (0.0008) [2023-12-27 17:15:52,175][20347] Updated weights for policy 0, policy_version 1677247 (0.0008) [2023-12-27 17:15:52,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 858423296. Throughput: 0: 10260.9, 1: 10171.9. Samples: 858391684. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:52,363][19170] Avg episode reward: [(0, '980.420'), (1, '931.730')] [2023-12-27 17:15:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001675472_428965888.pth... [2023-12-27 17:15:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001677256_429457408.pth... [2023-12-27 17:15:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001674288_428662784.pth [2023-12-27 17:15:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001676040_429146112.pth [2023-12-27 17:15:52,500][20381] Updated weights for policy 1, policy_version 1675475 (0.0005) [2023-12-27 17:15:52,551][20381] Updated weights for policy 1, policy_version 1675485 (0.0006) [2023-12-27 17:15:52,600][20381] Updated weights for policy 1, policy_version 1675495 (0.0005) [2023-12-27 17:15:52,955][20347] Updated weights for policy 0, policy_version 1677257 (0.0007) [2023-12-27 17:15:53,003][20347] Updated weights for policy 0, policy_version 1677267 (0.0008) [2023-12-27 17:15:53,069][20347] Updated weights for policy 0, policy_version 1677277 (0.0008) [2023-12-27 17:15:53,120][20347] Updated weights for policy 0, policy_version 1677287 (0.0008) [2023-12-27 17:15:53,147][20381] Updated weights for policy 1, policy_version 1675505 (0.0007) [2023-12-27 17:15:53,209][20381] Updated weights for policy 1, policy_version 1675515 (0.0007) [2023-12-27 17:15:53,261][20381] Updated weights for policy 1, policy_version 1675525 (0.0005) [2023-12-27 17:15:53,317][20381] Updated weights for policy 1, policy_version 1675535 (0.0005) [2023-12-27 17:15:53,854][20381] Updated weights for policy 1, policy_version 1675545 (0.0010) [2023-12-27 17:15:53,897][20347] Updated weights for policy 0, policy_version 1677297 (0.0006) [2023-12-27 17:15:53,909][20381] Updated weights for policy 1, policy_version 1675555 (0.0010) [2023-12-27 17:15:53,945][20347] Updated weights for policy 0, policy_version 1677307 (0.0006) [2023-12-27 17:15:53,964][20381] Updated weights for policy 1, policy_version 1675565 (0.0007) [2023-12-27 17:15:54,002][20347] Updated weights for policy 0, policy_version 1677317 (0.0008) [2023-12-27 17:15:54,499][20381] Updated weights for policy 1, policy_version 1675575 (0.0005) [2023-12-27 17:15:54,553][20381] Updated weights for policy 1, policy_version 1675585 (0.0008) [2023-12-27 17:15:54,611][20381] Updated weights for policy 1, policy_version 1675595 (0.0010) [2023-12-27 17:15:54,745][20347] Updated weights for policy 0, policy_version 1677327 (0.0008) [2023-12-27 17:15:54,802][20347] Updated weights for policy 0, policy_version 1677337 (0.0008) [2023-12-27 17:15:54,849][20347] Updated weights for policy 0, policy_version 1677347 (0.0008) [2023-12-27 17:15:55,221][20381] Updated weights for policy 1, policy_version 1675605 (0.0011) [2023-12-27 17:15:55,282][20381] Updated weights for policy 1, policy_version 1675615 (0.0010) [2023-12-27 17:15:55,340][20381] Updated weights for policy 1, policy_version 1675625 (0.0010) [2023-12-27 17:15:55,506][20347] Updated weights for policy 0, policy_version 1677357 (0.0006) [2023-12-27 17:15:55,558][20347] Updated weights for policy 0, policy_version 1677367 (0.0005) [2023-12-27 17:15:55,614][20347] Updated weights for policy 0, policy_version 1677377 (0.0005) [2023-12-27 17:15:55,962][20381] Updated weights for policy 1, policy_version 1675635 (0.0009) [2023-12-27 17:15:56,017][20381] Updated weights for policy 1, policy_version 1675645 (0.0006) [2023-12-27 17:15:56,074][20381] Updated weights for policy 1, policy_version 1675655 (0.0006) [2023-12-27 17:15:56,278][20347] Updated weights for policy 0, policy_version 1677387 (0.0006) [2023-12-27 17:15:56,330][20347] Updated weights for policy 0, policy_version 1677397 (0.0008) [2023-12-27 17:15:56,399][20347] Updated weights for policy 0, policy_version 1677407 (0.0009) [2023-12-27 17:15:56,723][20381] Updated weights for policy 1, policy_version 1675665 (0.0008) [2023-12-27 17:15:56,786][20381] Updated weights for policy 1, policy_version 1675675 (0.0008) [2023-12-27 17:15:56,838][20381] Updated weights for policy 1, policy_version 1675685 (0.0009) [2023-12-27 17:15:56,889][20381] Updated weights for policy 1, policy_version 1675695 (0.0005) [2023-12-27 17:15:56,998][20347] Updated weights for policy 0, policy_version 1677417 (0.0008) [2023-12-27 17:15:57,056][20347] Updated weights for policy 0, policy_version 1677427 (0.0010) [2023-12-27 17:15:57,107][20347] Updated weights for policy 0, policy_version 1677437 (0.0010) [2023-12-27 17:15:57,155][20347] Updated weights for policy 0, policy_version 1677447 (0.0010) [2023-12-27 17:15:57,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 858529792. Throughput: 0: 10217.0, 1: 10325.1. Samples: 858517128. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:15:57,363][19170] Avg episode reward: [(0, '990.220'), (1, '912.200')] [2023-12-27 17:15:57,461][20381] Updated weights for policy 1, policy_version 1675705 (0.0010) [2023-12-27 17:15:57,513][20381] Updated weights for policy 1, policy_version 1675715 (0.0007) [2023-12-27 17:15:57,563][20381] Updated weights for policy 1, policy_version 1675725 (0.0005) [2023-12-27 17:15:57,836][20347] Updated weights for policy 0, policy_version 1677457 (0.0006) [2023-12-27 17:15:57,890][20347] Updated weights for policy 0, policy_version 1677467 (0.0006) [2023-12-27 17:15:57,948][20347] Updated weights for policy 0, policy_version 1677477 (0.0010) [2023-12-27 17:15:58,220][20381] Updated weights for policy 1, policy_version 1675735 (0.0007) [2023-12-27 17:15:58,284][20381] Updated weights for policy 1, policy_version 1675745 (0.0007) [2023-12-27 17:15:58,357][20381] Updated weights for policy 1, policy_version 1675755 (0.0009) [2023-12-27 17:15:58,617][20347] Updated weights for policy 0, policy_version 1677487 (0.0008) [2023-12-27 17:15:58,676][20347] Updated weights for policy 0, policy_version 1677497 (0.0008) [2023-12-27 17:15:58,746][20347] Updated weights for policy 0, policy_version 1677507 (0.0008) [2023-12-27 17:15:59,125][20381] Updated weights for policy 1, policy_version 1675765 (0.0007) [2023-12-27 17:15:59,184][20381] Updated weights for policy 1, policy_version 1675775 (0.0011) [2023-12-27 17:15:59,246][20381] Updated weights for policy 1, policy_version 1675785 (0.0009) [2023-12-27 17:15:59,527][20347] Updated weights for policy 0, policy_version 1677517 (0.0008) [2023-12-27 17:15:59,589][20347] Updated weights for policy 0, policy_version 1677527 (0.0010) [2023-12-27 17:15:59,651][20347] Updated weights for policy 0, policy_version 1677537 (0.0010) [2023-12-27 17:15:59,964][20381] Updated weights for policy 1, policy_version 1675795 (0.0007) [2023-12-27 17:16:00,029][20381] Updated weights for policy 1, policy_version 1675805 (0.0010) [2023-12-27 17:16:00,088][20381] Updated weights for policy 1, policy_version 1675815 (0.0011) [2023-12-27 17:16:00,332][20347] Updated weights for policy 0, policy_version 1677547 (0.0009) [2023-12-27 17:16:00,391][20347] Updated weights for policy 0, policy_version 1677557 (0.0008) [2023-12-27 17:16:00,451][20347] Updated weights for policy 0, policy_version 1677567 (0.0005) [2023-12-27 17:16:00,766][20381] Updated weights for policy 1, policy_version 1675825 (0.0011) [2023-12-27 17:16:00,820][20381] Updated weights for policy 1, policy_version 1675835 (0.0010) [2023-12-27 17:16:00,872][20381] Updated weights for policy 1, policy_version 1675845 (0.0011) [2023-12-27 17:16:00,932][20381] Updated weights for policy 1, policy_version 1675855 (0.0011) [2023-12-27 17:16:01,066][20347] Updated weights for policy 0, policy_version 1677577 (0.0006) [2023-12-27 17:16:01,118][20347] Updated weights for policy 0, policy_version 1677587 (0.0010) [2023-12-27 17:16:01,178][20347] Updated weights for policy 0, policy_version 1677597 (0.0010) [2023-12-27 17:16:01,231][20347] Updated weights for policy 0, policy_version 1677607 (0.0009) [2023-12-27 17:16:01,611][20381] Updated weights for policy 1, policy_version 1675865 (0.0009) [2023-12-27 17:16:01,674][20381] Updated weights for policy 1, policy_version 1675875 (0.0009) [2023-12-27 17:16:01,733][20381] Updated weights for policy 1, policy_version 1675885 (0.0008) [2023-12-27 17:16:01,989][20347] Updated weights for policy 0, policy_version 1677617 (0.0007) [2023-12-27 17:16:02,046][20347] Updated weights for policy 0, policy_version 1677627 (0.0006) [2023-12-27 17:16:02,104][20347] Updated weights for policy 0, policy_version 1677637 (0.0008) [2023-12-27 17:16:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 858628096. Throughput: 0: 10183.6, 1: 10275.2. Samples: 858638292. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.220')] [2023-12-27 17:16:02,504][20381] Updated weights for policy 1, policy_version 1675895 (0.0008) [2023-12-27 17:16:02,552][20381] Updated weights for policy 1, policy_version 1675905 (0.0008) [2023-12-27 17:16:02,597][20381] Updated weights for policy 1, policy_version 1675915 (0.0007) [2023-12-27 17:16:02,761][20347] Updated weights for policy 0, policy_version 1677647 (0.0006) [2023-12-27 17:16:02,821][20347] Updated weights for policy 0, policy_version 1677657 (0.0007) [2023-12-27 17:16:02,881][20347] Updated weights for policy 0, policy_version 1677667 (0.0007) [2023-12-27 17:16:03,324][20381] Updated weights for policy 1, policy_version 1675925 (0.0008) [2023-12-27 17:16:03,385][20381] Updated weights for policy 1, policy_version 1675935 (0.0009) [2023-12-27 17:16:03,434][20381] Updated weights for policy 1, policy_version 1675945 (0.0005) [2023-12-27 17:16:03,504][20347] Updated weights for policy 0, policy_version 1677677 (0.0006) [2023-12-27 17:16:03,555][20347] Updated weights for policy 0, policy_version 1677687 (0.0006) [2023-12-27 17:16:03,608][20347] Updated weights for policy 0, policy_version 1677697 (0.0006) [2023-12-27 17:16:04,195][20381] Updated weights for policy 1, policy_version 1675955 (0.0007) [2023-12-27 17:16:04,254][20381] Updated weights for policy 1, policy_version 1675965 (0.0008) [2023-12-27 17:16:04,265][20347] Updated weights for policy 0, policy_version 1677707 (0.0006) [2023-12-27 17:16:04,318][20381] Updated weights for policy 1, policy_version 1675975 (0.0006) [2023-12-27 17:16:04,323][20347] Updated weights for policy 0, policy_version 1677717 (0.0007) [2023-12-27 17:16:04,388][20347] Updated weights for policy 0, policy_version 1677727 (0.0006) [2023-12-27 17:16:04,900][20381] Updated weights for policy 1, policy_version 1675985 (0.0007) [2023-12-27 17:16:04,963][20381] Updated weights for policy 1, policy_version 1675995 (0.0007) [2023-12-27 17:16:05,021][20381] Updated weights for policy 1, policy_version 1676005 (0.0010) [2023-12-27 17:16:05,057][20347] Updated weights for policy 0, policy_version 1677737 (0.0007) [2023-12-27 17:16:05,073][20381] Updated weights for policy 1, policy_version 1676015 (0.0010) [2023-12-27 17:16:05,116][20347] Updated weights for policy 0, policy_version 1677747 (0.0006) [2023-12-27 17:16:05,176][20347] Updated weights for policy 0, policy_version 1677757 (0.0007) [2023-12-27 17:16:05,237][20347] Updated weights for policy 0, policy_version 1677767 (0.0007) [2023-12-27 17:16:05,699][20381] Updated weights for policy 1, policy_version 1676025 (0.0008) [2023-12-27 17:16:05,752][20381] Updated weights for policy 1, policy_version 1676035 (0.0007) [2023-12-27 17:16:05,804][20381] Updated weights for policy 1, policy_version 1676045 (0.0006) [2023-12-27 17:16:05,854][20347] Updated weights for policy 0, policy_version 1677777 (0.0009) [2023-12-27 17:16:05,900][20347] Updated weights for policy 0, policy_version 1677787 (0.0010) [2023-12-27 17:16:05,951][20347] Updated weights for policy 0, policy_version 1677797 (0.0010) [2023-12-27 17:16:06,500][20381] Updated weights for policy 1, policy_version 1676055 (0.0007) [2023-12-27 17:16:06,560][20381] Updated weights for policy 1, policy_version 1676065 (0.0007) [2023-12-27 17:16:06,619][20381] Updated weights for policy 1, policy_version 1676075 (0.0008) [2023-12-27 17:16:06,676][20347] Updated weights for policy 0, policy_version 1677807 (0.0007) [2023-12-27 17:16:06,733][20347] Updated weights for policy 0, policy_version 1677817 (0.0008) [2023-12-27 17:16:06,790][20347] Updated weights for policy 0, policy_version 1677827 (0.0008) [2023-12-27 17:16:07,264][20381] Updated weights for policy 1, policy_version 1676085 (0.0007) [2023-12-27 17:16:07,321][20381] Updated weights for policy 1, policy_version 1676095 (0.0007) [2023-12-27 17:16:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.1, 300 sec: 20605.0). Total num frames: 858726400. Throughput: 0: 10191.6, 1: 10305.0. Samples: 858700404. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:07,363][19170] Avg episode reward: [(0, '990.270'), (1, '931.740')] [2023-12-27 17:16:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001677832_429604864.pth... [2023-12-27 17:16:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001676680_429309952.pth [2023-12-27 17:16:07,380][20381] Updated weights for policy 1, policy_version 1676105 (0.0008) [2023-12-27 17:16:07,420][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001676112_429129728.pth... [2023-12-27 17:16:07,421][20347] Updated weights for policy 0, policy_version 1677837 (0.0008) [2023-12-27 17:16:07,424][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001674896_428818432.pth [2023-12-27 17:16:07,479][20347] Updated weights for policy 0, policy_version 1677847 (0.0008) [2023-12-27 17:16:07,543][20347] Updated weights for policy 0, policy_version 1677857 (0.0008) [2023-12-27 17:16:07,958][20381] Updated weights for policy 1, policy_version 1676115 (0.0008) [2023-12-27 17:16:08,012][20381] Updated weights for policy 1, policy_version 1676125 (0.0007) [2023-12-27 17:16:08,067][20381] Updated weights for policy 1, policy_version 1676135 (0.0010) [2023-12-27 17:16:08,248][20347] Updated weights for policy 0, policy_version 1677867 (0.0007) [2023-12-27 17:16:08,301][20347] Updated weights for policy 0, policy_version 1677877 (0.0005) [2023-12-27 17:16:08,360][20347] Updated weights for policy 0, policy_version 1677887 (0.0007) [2023-12-27 17:16:08,788][20381] Updated weights for policy 1, policy_version 1676145 (0.0010) [2023-12-27 17:16:08,837][20381] Updated weights for policy 1, policy_version 1676155 (0.0008) [2023-12-27 17:16:08,886][20381] Updated weights for policy 1, policy_version 1676165 (0.0008) [2023-12-27 17:16:08,944][20381] Updated weights for policy 1, policy_version 1676175 (0.0008) [2023-12-27 17:16:09,049][20347] Updated weights for policy 0, policy_version 1677897 (0.0007) [2023-12-27 17:16:09,103][20347] Updated weights for policy 0, policy_version 1677907 (0.0008) [2023-12-27 17:16:09,160][20347] Updated weights for policy 0, policy_version 1677917 (0.0007) [2023-12-27 17:16:09,213][20347] Updated weights for policy 0, policy_version 1677927 (0.0007) [2023-12-27 17:16:09,636][20381] Updated weights for policy 1, policy_version 1676185 (0.0007) [2023-12-27 17:16:09,696][20381] Updated weights for policy 1, policy_version 1676195 (0.0008) [2023-12-27 17:16:09,753][20381] Updated weights for policy 1, policy_version 1676205 (0.0009) [2023-12-27 17:16:09,937][20347] Updated weights for policy 0, policy_version 1677937 (0.0007) [2023-12-27 17:16:09,999][20347] Updated weights for policy 0, policy_version 1677947 (0.0008) [2023-12-27 17:16:10,065][20347] Updated weights for policy 0, policy_version 1677957 (0.0008) [2023-12-27 17:16:10,470][20381] Updated weights for policy 1, policy_version 1676215 (0.0008) [2023-12-27 17:16:10,482][20292] KL-divergence is very high: 133.8817 [2023-12-27 17:16:10,531][20381] Updated weights for policy 1, policy_version 1676225 (0.0005) [2023-12-27 17:16:10,594][20381] Updated weights for policy 1, policy_version 1676235 (0.0006) [2023-12-27 17:16:10,719][20347] Updated weights for policy 0, policy_version 1677967 (0.0007) [2023-12-27 17:16:10,764][20347] Updated weights for policy 0, policy_version 1677977 (0.0006) [2023-12-27 17:16:10,819][20347] Updated weights for policy 0, policy_version 1677987 (0.0008) [2023-12-27 17:16:11,264][20292] KL-divergence is very high: 151.2046 [2023-12-27 17:16:11,283][20381] Updated weights for policy 1, policy_version 1676245 (0.0008) [2023-12-27 17:16:11,335][20381] Updated weights for policy 1, policy_version 1676255 (0.0007) [2023-12-27 17:16:11,405][20381] Updated weights for policy 1, policy_version 1676265 (0.0008) [2023-12-27 17:16:11,535][20347] Updated weights for policy 0, policy_version 1677997 (0.0008) [2023-12-27 17:16:11,593][20347] Updated weights for policy 0, policy_version 1678007 (0.0008) [2023-12-27 17:16:11,659][20347] Updated weights for policy 0, policy_version 1678017 (0.0008) [2023-12-27 17:16:12,096][20381] Updated weights for policy 1, policy_version 1676275 (0.0007) [2023-12-27 17:16:12,145][20381] Updated weights for policy 1, policy_version 1676285 (0.0007) [2023-12-27 17:16:12,201][20381] Updated weights for policy 1, policy_version 1676295 (0.0007) [2023-12-27 17:16:12,278][20347] Updated weights for policy 0, policy_version 1678027 (0.0006) [2023-12-27 17:16:12,341][20347] Updated weights for policy 0, policy_version 1678037 (0.0006) [2023-12-27 17:16:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 858832896. Throughput: 0: 10138.4, 1: 10282.3. Samples: 858823556. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:12,363][19170] Avg episode reward: [(0, '990.270'), (1, '824.380')] [2023-12-27 17:16:12,405][20347] Updated weights for policy 0, policy_version 1678047 (0.0009) [2023-12-27 17:16:12,960][20381] Updated weights for policy 1, policy_version 1676305 (0.0008) [2023-12-27 17:16:13,017][20381] Updated weights for policy 1, policy_version 1676315 (0.0005) [2023-12-27 17:16:13,075][20381] Updated weights for policy 1, policy_version 1676325 (0.0007) [2023-12-27 17:16:13,109][20347] Updated weights for policy 0, policy_version 1678057 (0.0008) [2023-12-27 17:16:13,124][20381] Updated weights for policy 1, policy_version 1676335 (0.0006) [2023-12-27 17:16:13,166][20347] Updated weights for policy 0, policy_version 1678067 (0.0008) [2023-12-27 17:16:13,231][20347] Updated weights for policy 0, policy_version 1678077 (0.0008) [2023-12-27 17:16:13,282][20347] Updated weights for policy 0, policy_version 1678087 (0.0007) [2023-12-27 17:16:13,817][20381] Updated weights for policy 1, policy_version 1676345 (0.0008) [2023-12-27 17:16:13,869][20381] Updated weights for policy 1, policy_version 1676355 (0.0009) [2023-12-27 17:16:13,900][20347] Updated weights for policy 0, policy_version 1678097 (0.0006) [2023-12-27 17:16:13,932][20381] Updated weights for policy 1, policy_version 1676365 (0.0007) [2023-12-27 17:16:13,951][20347] Updated weights for policy 0, policy_version 1678107 (0.0005) [2023-12-27 17:16:13,996][20347] Updated weights for policy 0, policy_version 1678117 (0.0005) [2023-12-27 17:16:14,602][20347] Updated weights for policy 0, policy_version 1678127 (0.0009) [2023-12-27 17:16:14,656][20347] Updated weights for policy 0, policy_version 1678137 (0.0010) [2023-12-27 17:16:14,666][20381] Updated weights for policy 1, policy_version 1676375 (0.0006) [2023-12-27 17:16:14,708][20347] Updated weights for policy 0, policy_version 1678147 (0.0010) [2023-12-27 17:16:14,728][20381] Updated weights for policy 1, policy_version 1676385 (0.0005) [2023-12-27 17:16:14,791][20381] Updated weights for policy 1, policy_version 1676395 (0.0008) [2023-12-27 17:16:15,432][20347] Updated weights for policy 0, policy_version 1678157 (0.0009) [2023-12-27 17:16:15,445][20381] Updated weights for policy 1, policy_version 1676405 (0.0006) [2023-12-27 17:16:15,480][20347] Updated weights for policy 0, policy_version 1678167 (0.0007) [2023-12-27 17:16:15,493][20381] Updated weights for policy 1, policy_version 1676415 (0.0007) [2023-12-27 17:16:15,530][20347] Updated weights for policy 0, policy_version 1678177 (0.0008) [2023-12-27 17:16:15,550][20381] Updated weights for policy 1, policy_version 1676425 (0.0007) [2023-12-27 17:16:16,211][20347] Updated weights for policy 0, policy_version 1678187 (0.0006) [2023-12-27 17:16:16,219][20381] Updated weights for policy 1, policy_version 1676435 (0.0007) [2023-12-27 17:16:16,266][20347] Updated weights for policy 0, policy_version 1678197 (0.0005) [2023-12-27 17:16:16,267][20381] Updated weights for policy 1, policy_version 1676445 (0.0007) [2023-12-27 17:16:16,312][20381] Updated weights for policy 1, policy_version 1676455 (0.0006) [2023-12-27 17:16:16,324][20347] Updated weights for policy 0, policy_version 1678207 (0.0007) [2023-12-27 17:16:16,967][20347] Updated weights for policy 0, policy_version 1678217 (0.0007) [2023-12-27 17:16:17,011][20381] Updated weights for policy 1, policy_version 1676465 (0.0007) [2023-12-27 17:16:17,031][20347] Updated weights for policy 0, policy_version 1678227 (0.0006) [2023-12-27 17:16:17,063][20381] Updated weights for policy 1, policy_version 1676475 (0.0005) [2023-12-27 17:16:17,083][20347] Updated weights for policy 0, policy_version 1678237 (0.0006) [2023-12-27 17:16:17,119][20381] Updated weights for policy 1, policy_version 1676485 (0.0006) [2023-12-27 17:16:17,136][20347] Updated weights for policy 0, policy_version 1678247 (0.0006) [2023-12-27 17:16:17,169][20381] Updated weights for policy 1, policy_version 1676495 (0.0007) [2023-12-27 17:16:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.1, 300 sec: 20632.7). Total num frames: 858939392. Throughput: 0: 10161.4, 1: 10214.3. Samples: 858944980. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.140')] [2023-12-27 17:16:17,785][20347] Updated weights for policy 0, policy_version 1678257 (0.0008) [2023-12-27 17:16:17,840][20347] Updated weights for policy 0, policy_version 1678267 (0.0007) [2023-12-27 17:16:17,874][20381] Updated weights for policy 1, policy_version 1676505 (0.0008) [2023-12-27 17:16:17,897][20347] Updated weights for policy 0, policy_version 1678277 (0.0006) [2023-12-27 17:16:17,925][20381] Updated weights for policy 1, policy_version 1676515 (0.0008) [2023-12-27 17:16:17,977][20381] Updated weights for policy 1, policy_version 1676525 (0.0008) [2023-12-27 17:16:18,667][20347] Updated weights for policy 0, policy_version 1678287 (0.0007) [2023-12-27 17:16:18,700][20381] Updated weights for policy 1, policy_version 1676535 (0.0007) [2023-12-27 17:16:18,724][20347] Updated weights for policy 0, policy_version 1678297 (0.0007) [2023-12-27 17:16:18,756][20381] Updated weights for policy 1, policy_version 1676545 (0.0006) [2023-12-27 17:16:18,778][20347] Updated weights for policy 0, policy_version 1678307 (0.0005) [2023-12-27 17:16:18,811][20381] Updated weights for policy 1, policy_version 1676555 (0.0006) [2023-12-27 17:16:19,470][20381] Updated weights for policy 1, policy_version 1676565 (0.0008) [2023-12-27 17:16:19,511][20347] Updated weights for policy 0, policy_version 1678317 (0.0008) [2023-12-27 17:16:19,535][20381] Updated weights for policy 1, policy_version 1676575 (0.0006) [2023-12-27 17:16:19,568][20347] Updated weights for policy 0, policy_version 1678327 (0.0011) [2023-12-27 17:16:19,601][20381] Updated weights for policy 1, policy_version 1676585 (0.0006) [2023-12-27 17:16:19,628][20347] Updated weights for policy 0, policy_version 1678337 (0.0010) [2023-12-27 17:16:20,331][20381] Updated weights for policy 1, policy_version 1676595 (0.0008) [2023-12-27 17:16:20,352][20347] Updated weights for policy 0, policy_version 1678347 (0.0009) [2023-12-27 17:16:20,400][20381] Updated weights for policy 1, policy_version 1676605 (0.0009) [2023-12-27 17:16:20,410][20347] Updated weights for policy 0, policy_version 1678357 (0.0007) [2023-12-27 17:16:20,465][20381] Updated weights for policy 1, policy_version 1676615 (0.0008) [2023-12-27 17:16:20,468][20347] Updated weights for policy 0, policy_version 1678367 (0.0006) [2023-12-27 17:16:21,162][20381] Updated weights for policy 1, policy_version 1676625 (0.0008) [2023-12-27 17:16:21,204][20347] Updated weights for policy 0, policy_version 1678377 (0.0005) [2023-12-27 17:16:21,226][20381] Updated weights for policy 1, policy_version 1676635 (0.0008) [2023-12-27 17:16:21,272][20347] Updated weights for policy 0, policy_version 1678387 (0.0007) [2023-12-27 17:16:21,284][20381] Updated weights for policy 1, policy_version 1676645 (0.0008) [2023-12-27 17:16:21,325][20347] Updated weights for policy 0, policy_version 1678397 (0.0011) [2023-12-27 17:16:21,347][20381] Updated weights for policy 1, policy_version 1676655 (0.0008) [2023-12-27 17:16:21,399][20347] Updated weights for policy 0, policy_version 1678407 (0.0008) [2023-12-27 17:16:22,089][20381] Updated weights for policy 1, policy_version 1676665 (0.0011) [2023-12-27 17:16:22,119][20347] Updated weights for policy 0, policy_version 1678417 (0.0010) [2023-12-27 17:16:22,148][20381] Updated weights for policy 1, policy_version 1676675 (0.0010) [2023-12-27 17:16:22,175][20347] Updated weights for policy 0, policy_version 1678427 (0.0010) [2023-12-27 17:16:22,205][20381] Updated weights for policy 1, policy_version 1676685 (0.0011) [2023-12-27 17:16:22,235][20347] Updated weights for policy 0, policy_version 1678437 (0.0007) [2023-12-27 17:16:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 859037696. Throughput: 0: 10132.5, 1: 10173.9. Samples: 859004364. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.420')] [2023-12-27 17:16:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001678440_429760512.pth... [2023-12-27 17:16:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001676688_429277184.pth... [2023-12-27 17:16:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001675472_428965888.pth [2023-12-27 17:16:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001677256_429457408.pth [2023-12-27 17:16:22,933][20347] Updated weights for policy 0, policy_version 1678447 (0.0006) [2023-12-27 17:16:22,954][20381] Updated weights for policy 1, policy_version 1676695 (0.0011) [2023-12-27 17:16:22,984][20347] Updated weights for policy 0, policy_version 1678457 (0.0006) [2023-12-27 17:16:23,004][20381] Updated weights for policy 1, policy_version 1676705 (0.0010) [2023-12-27 17:16:23,037][20347] Updated weights for policy 0, policy_version 1678467 (0.0006) [2023-12-27 17:16:23,057][20381] Updated weights for policy 1, policy_version 1676715 (0.0011) [2023-12-27 17:16:23,689][20347] Updated weights for policy 0, policy_version 1678477 (0.0006) [2023-12-27 17:16:23,748][20347] Updated weights for policy 0, policy_version 1678487 (0.0006) [2023-12-27 17:16:23,751][20381] Updated weights for policy 1, policy_version 1676725 (0.0011) [2023-12-27 17:16:23,802][20347] Updated weights for policy 0, policy_version 1678497 (0.0007) [2023-12-27 17:16:23,805][20381] Updated weights for policy 1, policy_version 1676735 (0.0010) [2023-12-27 17:16:23,862][20381] Updated weights for policy 1, policy_version 1676745 (0.0010) [2023-12-27 17:16:24,461][20347] Updated weights for policy 0, policy_version 1678507 (0.0007) [2023-12-27 17:16:24,491][20381] Updated weights for policy 1, policy_version 1676755 (0.0009) [2023-12-27 17:16:24,516][20347] Updated weights for policy 0, policy_version 1678517 (0.0010) [2023-12-27 17:16:24,547][20381] Updated weights for policy 1, policy_version 1676765 (0.0007) [2023-12-27 17:16:24,565][20347] Updated weights for policy 0, policy_version 1678527 (0.0010) [2023-12-27 17:16:24,591][20381] Updated weights for policy 1, policy_version 1676775 (0.0008) [2023-12-27 17:16:25,177][20381] Updated weights for policy 1, policy_version 1676785 (0.0005) [2023-12-27 17:16:25,220][20347] Updated weights for policy 0, policy_version 1678537 (0.0010) [2023-12-27 17:16:25,240][20381] Updated weights for policy 1, policy_version 1676795 (0.0009) [2023-12-27 17:16:25,279][20347] Updated weights for policy 0, policy_version 1678547 (0.0005) [2023-12-27 17:16:25,296][20381] Updated weights for policy 1, policy_version 1676805 (0.0010) [2023-12-27 17:16:25,341][20347] Updated weights for policy 0, policy_version 1678557 (0.0005) [2023-12-27 17:16:25,350][20381] Updated weights for policy 1, policy_version 1676815 (0.0010) [2023-12-27 17:16:25,403][20347] Updated weights for policy 0, policy_version 1678567 (0.0007) [2023-12-27 17:16:25,955][20347] Updated weights for policy 0, policy_version 1678577 (0.0008) [2023-12-27 17:16:25,987][20381] Updated weights for policy 1, policy_version 1676825 (0.0006) [2023-12-27 17:16:26,016][20347] Updated weights for policy 0, policy_version 1678587 (0.0006) [2023-12-27 17:16:26,043][20381] Updated weights for policy 1, policy_version 1676835 (0.0008) [2023-12-27 17:16:26,074][20347] Updated weights for policy 0, policy_version 1678597 (0.0006) [2023-12-27 17:16:26,098][20381] Updated weights for policy 1, policy_version 1676845 (0.0007) [2023-12-27 17:16:26,694][20347] Updated weights for policy 0, policy_version 1678607 (0.0006) [2023-12-27 17:16:26,731][20381] Updated weights for policy 1, policy_version 1676855 (0.0008) [2023-12-27 17:16:26,749][20347] Updated weights for policy 0, policy_version 1678617 (0.0007) [2023-12-27 17:16:26,787][20381] Updated weights for policy 1, policy_version 1676865 (0.0006) [2023-12-27 17:16:26,804][20347] Updated weights for policy 0, policy_version 1678627 (0.0006) [2023-12-27 17:16:26,833][20381] Updated weights for policy 1, policy_version 1676875 (0.0005) [2023-12-27 17:16:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.6, 300 sec: 20605.0). Total num frames: 859136000. Throughput: 0: 10138.0, 1: 10183.2. Samples: 859127296. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) [2023-12-27 17:16:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.940')] [2023-12-27 17:16:27,431][20381] Updated weights for policy 1, policy_version 1676885 (0.0008) [2023-12-27 17:16:27,431][20347] Updated weights for policy 0, policy_version 1678637 (0.0006) [2023-12-27 17:16:27,480][20347] Updated weights for policy 0, policy_version 1678647 (0.0005) [2023-12-27 17:16:27,485][20381] Updated weights for policy 1, policy_version 1676895 (0.0010) [2023-12-27 17:16:27,528][20347] Updated weights for policy 0, policy_version 1678657 (0.0010) [2023-12-27 17:16:27,530][20381] Updated weights for policy 1, policy_version 1676905 (0.0008) [2023-12-27 17:16:28,120][20347] Updated weights for policy 0, policy_version 1678667 (0.0010) [2023-12-27 17:16:28,177][20347] Updated weights for policy 0, policy_version 1678677 (0.0010) [2023-12-27 17:16:28,207][20381] Updated weights for policy 1, policy_version 1676915 (0.0007) [2023-12-27 17:16:28,232][20347] Updated weights for policy 0, policy_version 1678687 (0.0008) [2023-12-27 17:16:28,273][20381] Updated weights for policy 1, policy_version 1676925 (0.0005) [2023-12-27 17:16:28,336][20381] Updated weights for policy 1, policy_version 1676935 (0.0006) [2023-12-27 17:16:28,876][20347] Updated weights for policy 0, policy_version 1678697 (0.0006) [2023-12-27 17:16:28,924][20381] Updated weights for policy 1, policy_version 1676945 (0.0009) [2023-12-27 17:16:28,931][20347] Updated weights for policy 0, policy_version 1678707 (0.0007) [2023-12-27 17:16:28,984][20381] Updated weights for policy 1, policy_version 1676955 (0.0005) [2023-12-27 17:16:28,988][20347] Updated weights for policy 0, policy_version 1678717 (0.0005) [2023-12-27 17:16:29,040][20381] Updated weights for policy 1, policy_version 1676965 (0.0005) [2023-12-27 17:16:29,045][20347] Updated weights for policy 0, policy_version 1678727 (0.0005) [2023-12-27 17:16:29,100][20381] Updated weights for policy 1, policy_version 1676975 (0.0007) [2023-12-27 17:16:29,654][20347] Updated weights for policy 0, policy_version 1678737 (0.0005) [2023-12-27 17:16:29,708][20381] Updated weights for policy 1, policy_version 1676985 (0.0006) [2023-12-27 17:16:29,716][20347] Updated weights for policy 0, policy_version 1678747 (0.0005) [2023-12-27 17:16:29,747][20292] KL-divergence is very high: 124.3927 [2023-12-27 17:16:29,762][20381] Updated weights for policy 1, policy_version 1676995 (0.0006) [2023-12-27 17:16:29,776][20347] Updated weights for policy 0, policy_version 1678757 (0.0005) [2023-12-27 17:16:29,829][20381] Updated weights for policy 1, policy_version 1677005 (0.0007) [2023-12-27 17:16:30,421][20347] Updated weights for policy 0, policy_version 1678767 (0.0007) [2023-12-27 17:16:30,476][20347] Updated weights for policy 0, policy_version 1678777 (0.0007) [2023-12-27 17:16:30,519][20381] Updated weights for policy 1, policy_version 1677015 (0.0007) [2023-12-27 17:16:30,535][20347] Updated weights for policy 0, policy_version 1678787 (0.0006) [2023-12-27 17:16:30,575][20381] Updated weights for policy 1, policy_version 1677025 (0.0008) [2023-12-27 17:16:30,632][20381] Updated weights for policy 1, policy_version 1677035 (0.0008) [2023-12-27 17:16:31,158][20347] Updated weights for policy 0, policy_version 1678797 (0.0007) [2023-12-27 17:16:31,219][20347] Updated weights for policy 0, policy_version 1678807 (0.0006) [2023-12-27 17:16:31,281][20347] Updated weights for policy 0, policy_version 1678817 (0.0007) [2023-12-27 17:16:31,373][20381] Updated weights for policy 1, policy_version 1677045 (0.0007) [2023-12-27 17:16:31,438][20381] Updated weights for policy 1, policy_version 1677055 (0.0009) [2023-12-27 17:16:31,495][20381] Updated weights for policy 1, policy_version 1677065 (0.0008) [2023-12-27 17:16:31,982][20347] Updated weights for policy 0, policy_version 1678827 (0.0007) [2023-12-27 17:16:32,032][20347] Updated weights for policy 0, policy_version 1678837 (0.0008) [2023-12-27 17:16:32,078][20347] Updated weights for policy 0, policy_version 1678847 (0.0007) [2023-12-27 17:16:32,099][20381] Updated weights for policy 1, policy_version 1677075 (0.0008) [2023-12-27 17:16:32,160][20381] Updated weights for policy 1, policy_version 1677085 (0.0006) [2023-12-27 17:16:32,226][20381] Updated weights for policy 1, policy_version 1677095 (0.0006) [2023-12-27 17:16:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 859250688. Throughput: 0: 10242.3, 1: 10281.5. Samples: 859256512. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.270')] [2023-12-27 17:16:32,782][20347] Updated weights for policy 0, policy_version 1678857 (0.0007) [2023-12-27 17:16:32,826][20381] Updated weights for policy 1, policy_version 1677105 (0.0007) [2023-12-27 17:16:32,836][20347] Updated weights for policy 0, policy_version 1678867 (0.0007) [2023-12-27 17:16:32,885][20381] Updated weights for policy 1, policy_version 1677115 (0.0005) [2023-12-27 17:16:32,890][20347] Updated weights for policy 0, policy_version 1678877 (0.0007) [2023-12-27 17:16:32,935][20347] Updated weights for policy 0, policy_version 1678887 (0.0006) [2023-12-27 17:16:32,940][20381] Updated weights for policy 1, policy_version 1677125 (0.0007) [2023-12-27 17:16:32,994][20381] Updated weights for policy 1, policy_version 1677135 (0.0007) [2023-12-27 17:16:33,616][20347] Updated weights for policy 0, policy_version 1678897 (0.0007) [2023-12-27 17:16:33,678][20347] Updated weights for policy 0, policy_version 1678907 (0.0007) [2023-12-27 17:16:33,687][20381] Updated weights for policy 1, policy_version 1677145 (0.0006) [2023-12-27 17:16:33,734][20347] Updated weights for policy 0, policy_version 1678917 (0.0008) [2023-12-27 17:16:33,747][20381] Updated weights for policy 1, policy_version 1677155 (0.0005) [2023-12-27 17:16:33,805][20381] Updated weights for policy 1, policy_version 1677165 (0.0006) [2023-12-27 17:16:34,400][20347] Updated weights for policy 0, policy_version 1678927 (0.0008) [2023-12-27 17:16:34,451][20347] Updated weights for policy 0, policy_version 1678937 (0.0007) [2023-12-27 17:16:34,471][20381] Updated weights for policy 1, policy_version 1677175 (0.0007) [2023-12-27 17:16:34,507][20347] Updated weights for policy 0, policy_version 1678947 (0.0006) [2023-12-27 17:16:34,533][20381] Updated weights for policy 1, policy_version 1677185 (0.0007) [2023-12-27 17:16:34,601][20381] Updated weights for policy 1, policy_version 1677195 (0.0008) [2023-12-27 17:16:35,215][20347] Updated weights for policy 0, policy_version 1678957 (0.0007) [2023-12-27 17:16:35,271][20381] Updated weights for policy 1, policy_version 1677205 (0.0006) [2023-12-27 17:16:35,275][20347] Updated weights for policy 0, policy_version 1678967 (0.0008) [2023-12-27 17:16:35,326][20381] Updated weights for policy 1, policy_version 1677215 (0.0005) [2023-12-27 17:16:35,333][20347] Updated weights for policy 0, policy_version 1678977 (0.0008) [2023-12-27 17:16:35,382][20381] Updated weights for policy 1, policy_version 1677225 (0.0007) [2023-12-27 17:16:35,899][20347] Updated weights for policy 0, policy_version 1678987 (0.0009) [2023-12-27 17:16:35,952][20347] Updated weights for policy 0, policy_version 1678997 (0.0009) [2023-12-27 17:16:36,005][20381] Updated weights for policy 1, policy_version 1677235 (0.0006) [2023-12-27 17:16:36,007][20347] Updated weights for policy 0, policy_version 1679007 (0.0010) [2023-12-27 17:16:36,065][20381] Updated weights for policy 1, policy_version 1677245 (0.0005) [2023-12-27 17:16:36,125][20381] Updated weights for policy 1, policy_version 1677255 (0.0007) [2023-12-27 17:16:36,697][20347] Updated weights for policy 0, policy_version 1679017 (0.0010) [2023-12-27 17:16:36,749][20347] Updated weights for policy 0, policy_version 1679027 (0.0010) [2023-12-27 17:16:36,770][20381] Updated weights for policy 1, policy_version 1677265 (0.0007) [2023-12-27 17:16:36,799][20347] Updated weights for policy 0, policy_version 1679037 (0.0010) [2023-12-27 17:16:36,817][20381] Updated weights for policy 1, policy_version 1677275 (0.0005) [2023-12-27 17:16:36,852][20347] Updated weights for policy 0, policy_version 1679047 (0.0011) [2023-12-27 17:16:36,863][20381] Updated weights for policy 1, policy_version 1677285 (0.0006) [2023-12-27 17:16:36,911][20381] Updated weights for policy 1, policy_version 1677295 (0.0006) [2023-12-27 17:16:37,363][19170] Fps is (10 sec: 21298.2, 60 sec: 20479.9, 300 sec: 20604.9). Total num frames: 859348992. Throughput: 0: 10265.7, 1: 10320.5. Samples: 859318072. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:37,364][19170] Avg episode reward: [(0, '990.400'), (1, '883.170')] [2023-12-27 17:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001679048_429916160.pth... [2023-12-27 17:16:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001677296_429432832.pth... [2023-12-27 17:16:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001677832_429604864.pth [2023-12-27 17:16:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001676112_429129728.pth [2023-12-27 17:16:37,546][20347] Updated weights for policy 0, policy_version 1679057 (0.0010) [2023-12-27 17:16:37,600][20347] Updated weights for policy 0, policy_version 1679067 (0.0010) [2023-12-27 17:16:37,613][20381] Updated weights for policy 1, policy_version 1677305 (0.0006) [2023-12-27 17:16:37,654][20347] Updated weights for policy 0, policy_version 1679077 (0.0007) [2023-12-27 17:16:37,669][20381] Updated weights for policy 1, policy_version 1677315 (0.0009) [2023-12-27 17:16:37,722][20381] Updated weights for policy 1, policy_version 1677325 (0.0007) [2023-12-27 17:16:38,310][20381] Updated weights for policy 1, policy_version 1677335 (0.0007) [2023-12-27 17:16:38,350][20347] Updated weights for policy 0, policy_version 1679087 (0.0009) [2023-12-27 17:16:38,370][20381] Updated weights for policy 1, policy_version 1677345 (0.0009) [2023-12-27 17:16:38,412][20347] Updated weights for policy 0, policy_version 1679097 (0.0008) [2023-12-27 17:16:38,428][20381] Updated weights for policy 1, policy_version 1677355 (0.0006) [2023-12-27 17:16:38,468][20347] Updated weights for policy 0, policy_version 1679107 (0.0008) [2023-12-27 17:16:39,102][20381] Updated weights for policy 1, policy_version 1677365 (0.0008) [2023-12-27 17:16:39,155][20381] Updated weights for policy 1, policy_version 1677375 (0.0008) [2023-12-27 17:16:39,199][20381] Updated weights for policy 1, policy_version 1677385 (0.0007) [2023-12-27 17:16:39,208][20347] Updated weights for policy 0, policy_version 1679117 (0.0007) [2023-12-27 17:16:39,273][20347] Updated weights for policy 0, policy_version 1679127 (0.0007) [2023-12-27 17:16:39,327][20347] Updated weights for policy 0, policy_version 1679137 (0.0008) [2023-12-27 17:16:39,984][20381] Updated weights for policy 1, policy_version 1677395 (0.0008) [2023-12-27 17:16:39,995][20347] Updated weights for policy 0, policy_version 1679147 (0.0006) [2023-12-27 17:16:40,046][20381] Updated weights for policy 1, policy_version 1677405 (0.0006) [2023-12-27 17:16:40,052][20347] Updated weights for policy 0, policy_version 1679157 (0.0009) [2023-12-27 17:16:40,103][20381] Updated weights for policy 1, policy_version 1677415 (0.0006) [2023-12-27 17:16:40,115][20347] Updated weights for policy 0, policy_version 1679167 (0.0010) [2023-12-27 17:16:40,762][20381] Updated weights for policy 1, policy_version 1677425 (0.0006) [2023-12-27 17:16:40,796][20347] Updated weights for policy 0, policy_version 1679177 (0.0010) [2023-12-27 17:16:40,820][20381] Updated weights for policy 1, policy_version 1677435 (0.0007) [2023-12-27 17:16:40,855][20347] Updated weights for policy 0, policy_version 1679187 (0.0011) [2023-12-27 17:16:40,876][20381] Updated weights for policy 1, policy_version 1677445 (0.0005) [2023-12-27 17:16:40,904][20347] Updated weights for policy 0, policy_version 1679197 (0.0010) [2023-12-27 17:16:40,934][20381] Updated weights for policy 1, policy_version 1677455 (0.0007) [2023-12-27 17:16:40,953][20347] Updated weights for policy 0, policy_version 1679207 (0.0010) [2023-12-27 17:16:41,727][20381] Updated weights for policy 1, policy_version 1677465 (0.0008) [2023-12-27 17:16:41,742][20347] Updated weights for policy 0, policy_version 1679217 (0.0008) [2023-12-27 17:16:41,796][20381] Updated weights for policy 1, policy_version 1677475 (0.0007) [2023-12-27 17:16:41,808][20347] Updated weights for policy 0, policy_version 1679227 (0.0008) [2023-12-27 17:16:41,855][20381] Updated weights for policy 1, policy_version 1677485 (0.0007) [2023-12-27 17:16:41,861][20347] Updated weights for policy 0, policy_version 1679237 (0.0008) [2023-12-27 17:16:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.2, 300 sec: 20605.0). Total num frames: 859447296. Throughput: 0: 10289.4, 1: 10222.1. Samples: 859440144. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:42,363][19170] Avg episode reward: [(0, '990.400'), (1, '902.650')] [2023-12-27 17:16:42,552][20381] Updated weights for policy 1, policy_version 1677495 (0.0007) [2023-12-27 17:16:42,593][20347] Updated weights for policy 0, policy_version 1679247 (0.0006) [2023-12-27 17:16:42,612][20381] Updated weights for policy 1, policy_version 1677505 (0.0007) [2023-12-27 17:16:42,654][20347] Updated weights for policy 0, policy_version 1679257 (0.0006) [2023-12-27 17:16:42,675][20381] Updated weights for policy 1, policy_version 1677515 (0.0008) [2023-12-27 17:16:42,720][20347] Updated weights for policy 0, policy_version 1679267 (0.0006) [2023-12-27 17:16:43,232][20347] Updated weights for policy 0, policy_version 1679277 (0.0006) [2023-12-27 17:16:43,283][20347] Updated weights for policy 0, policy_version 1679287 (0.0005) [2023-12-27 17:16:43,331][20347] Updated weights for policy 0, policy_version 1679297 (0.0005) [2023-12-27 17:16:43,394][20381] Updated weights for policy 1, policy_version 1677525 (0.0007) [2023-12-27 17:16:43,441][20381] Updated weights for policy 1, policy_version 1677535 (0.0008) [2023-12-27 17:16:43,490][20381] Updated weights for policy 1, policy_version 1677545 (0.0008) [2023-12-27 17:16:43,881][20347] Updated weights for policy 0, policy_version 1679307 (0.0006) [2023-12-27 17:16:43,941][20347] Updated weights for policy 0, policy_version 1679317 (0.0007) [2023-12-27 17:16:43,997][20347] Updated weights for policy 0, policy_version 1679327 (0.0007) [2023-12-27 17:16:44,287][20381] Updated weights for policy 1, policy_version 1677555 (0.0009) [2023-12-27 17:16:44,333][20381] Updated weights for policy 1, policy_version 1677565 (0.0007) [2023-12-27 17:16:44,379][20381] Updated weights for policy 1, policy_version 1677575 (0.0006) [2023-12-27 17:16:44,628][20347] Updated weights for policy 0, policy_version 1679337 (0.0008) [2023-12-27 17:16:44,686][20347] Updated weights for policy 0, policy_version 1679347 (0.0008) [2023-12-27 17:16:44,735][20347] Updated weights for policy 0, policy_version 1679357 (0.0007) [2023-12-27 17:16:44,794][20347] Updated weights for policy 0, policy_version 1679367 (0.0009) [2023-12-27 17:16:45,118][20381] Updated weights for policy 1, policy_version 1677585 (0.0007) [2023-12-27 17:16:45,177][20381] Updated weights for policy 1, policy_version 1677595 (0.0010) [2023-12-27 17:16:45,240][20381] Updated weights for policy 1, policy_version 1677605 (0.0011) [2023-12-27 17:16:45,295][20381] Updated weights for policy 1, policy_version 1677615 (0.0010) [2023-12-27 17:16:45,497][20347] Updated weights for policy 0, policy_version 1679377 (0.0008) [2023-12-27 17:16:45,556][20347] Updated weights for policy 0, policy_version 1679387 (0.0008) [2023-12-27 17:16:45,613][20347] Updated weights for policy 0, policy_version 1679397 (0.0008) [2023-12-27 17:16:45,871][20381] Updated weights for policy 1, policy_version 1677625 (0.0010) [2023-12-27 17:16:45,927][20381] Updated weights for policy 1, policy_version 1677635 (0.0011) [2023-12-27 17:16:45,982][20381] Updated weights for policy 1, policy_version 1677645 (0.0011) [2023-12-27 17:16:46,266][20347] Updated weights for policy 0, policy_version 1679407 (0.0007) [2023-12-27 17:16:46,313][20347] Updated weights for policy 0, policy_version 1679417 (0.0008) [2023-12-27 17:16:46,367][20347] Updated weights for policy 0, policy_version 1679427 (0.0008) [2023-12-27 17:16:46,596][20381] Updated weights for policy 1, policy_version 1677655 (0.0011) [2023-12-27 17:16:46,652][20381] Updated weights for policy 1, policy_version 1677665 (0.0011) [2023-12-27 17:16:46,709][20381] Updated weights for policy 1, policy_version 1677675 (0.0010) [2023-12-27 17:16:47,001][20347] Updated weights for policy 0, policy_version 1679437 (0.0006) [2023-12-27 17:16:47,056][20347] Updated weights for policy 0, policy_version 1679447 (0.0005) [2023-12-27 17:16:47,140][20347] Updated weights for policy 0, policy_version 1679457 (0.0005) [2023-12-27 17:16:47,344][20381] Updated weights for policy 1, policy_version 1677685 (0.0008) [2023-12-27 17:16:47,363][19170] Fps is (10 sec: 20480.5, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 859553792. Throughput: 0: 10366.5, 1: 10233.1. Samples: 859565276. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:47,364][19170] Avg episode reward: [(0, '990.250'), (1, '941.580')] [2023-12-27 17:16:47,409][20381] Updated weights for policy 1, policy_version 1677695 (0.0005) [2023-12-27 17:16:47,464][20381] Updated weights for policy 1, policy_version 1677705 (0.0006) [2023-12-27 17:16:47,667][20347] Updated weights for policy 0, policy_version 1679467 (0.0005) [2023-12-27 17:16:47,716][20347] Updated weights for policy 0, policy_version 1679477 (0.0005) [2023-12-27 17:16:47,769][20347] Updated weights for policy 0, policy_version 1679487 (0.0006) [2023-12-27 17:16:47,971][20381] Updated weights for policy 1, policy_version 1677715 (0.0005) [2023-12-27 17:16:48,023][20381] Updated weights for policy 1, policy_version 1677725 (0.0005) [2023-12-27 17:16:48,081][20381] Updated weights for policy 1, policy_version 1677735 (0.0005) [2023-12-27 17:16:48,347][20347] Updated weights for policy 0, policy_version 1679497 (0.0006) [2023-12-27 17:16:48,411][20347] Updated weights for policy 0, policy_version 1679507 (0.0007) [2023-12-27 17:16:48,480][20347] Updated weights for policy 0, policy_version 1679517 (0.0006) [2023-12-27 17:16:48,542][20347] Updated weights for policy 0, policy_version 1679527 (0.0007) [2023-12-27 17:16:48,770][20381] Updated weights for policy 1, policy_version 1677745 (0.0010) [2023-12-27 17:16:48,839][20381] Updated weights for policy 1, policy_version 1677755 (0.0011) [2023-12-27 17:16:48,897][20381] Updated weights for policy 1, policy_version 1677765 (0.0010) [2023-12-27 17:16:48,959][20381] Updated weights for policy 1, policy_version 1677775 (0.0011) [2023-12-27 17:16:49,202][20347] Updated weights for policy 0, policy_version 1679537 (0.0008) [2023-12-27 17:16:49,281][20347] Updated weights for policy 0, policy_version 1679547 (0.0007) [2023-12-27 17:16:49,344][20347] Updated weights for policy 0, policy_version 1679557 (0.0008) [2023-12-27 17:16:49,646][20381] Updated weights for policy 1, policy_version 1677785 (0.0011) [2023-12-27 17:16:49,705][20381] Updated weights for policy 1, policy_version 1677795 (0.0011) [2023-12-27 17:16:49,761][20381] Updated weights for policy 1, policy_version 1677805 (0.0011) [2023-12-27 17:16:50,060][20347] Updated weights for policy 0, policy_version 1679567 (0.0008) [2023-12-27 17:16:50,128][20347] Updated weights for policy 0, policy_version 1679577 (0.0008) [2023-12-27 17:16:50,189][20347] Updated weights for policy 0, policy_version 1679587 (0.0007) [2023-12-27 17:16:50,493][20381] Updated weights for policy 1, policy_version 1677815 (0.0007) [2023-12-27 17:16:50,539][20381] Updated weights for policy 1, policy_version 1677825 (0.0010) [2023-12-27 17:16:50,601][20381] Updated weights for policy 1, policy_version 1677835 (0.0010) [2023-12-27 17:16:50,906][20347] Updated weights for policy 0, policy_version 1679597 (0.0009) [2023-12-27 17:16:50,969][20347] Updated weights for policy 0, policy_version 1679607 (0.0010) [2023-12-27 17:16:51,034][20347] Updated weights for policy 0, policy_version 1679617 (0.0009) [2023-12-27 17:16:51,349][20381] Updated weights for policy 1, policy_version 1677845 (0.0009) [2023-12-27 17:16:51,409][20381] Updated weights for policy 1, policy_version 1677855 (0.0011) [2023-12-27 17:16:51,462][20381] Updated weights for policy 1, policy_version 1677865 (0.0011) [2023-12-27 17:16:51,787][20347] Updated weights for policy 0, policy_version 1679627 (0.0008) [2023-12-27 17:16:51,843][20347] Updated weights for policy 0, policy_version 1679637 (0.0008) [2023-12-27 17:16:51,896][20347] Updated weights for policy 0, policy_version 1679647 (0.0008) [2023-12-27 17:16:52,136][20381] Updated weights for policy 1, policy_version 1677875 (0.0009) [2023-12-27 17:16:52,193][20381] Updated weights for policy 1, policy_version 1677885 (0.0005) [2023-12-27 17:16:52,254][20381] Updated weights for policy 1, policy_version 1677895 (0.0006) [2023-12-27 17:16:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 859660288. Throughput: 0: 10380.4, 1: 10261.5. Samples: 859629292. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:52,363][19170] Avg episode reward: [(0, '980.460'), (1, '931.790')] [2023-12-27 17:16:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001677904_429588480.pth... [2023-12-27 17:16:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001679656_430071808.pth... [2023-12-27 17:16:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001676688_429277184.pth [2023-12-27 17:16:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001678440_429760512.pth [2023-12-27 17:16:52,602][20347] Updated weights for policy 0, policy_version 1679657 (0.0007) [2023-12-27 17:16:52,661][20347] Updated weights for policy 0, policy_version 1679667 (0.0006) [2023-12-27 17:16:52,723][20347] Updated weights for policy 0, policy_version 1679677 (0.0006) [2023-12-27 17:16:52,784][20347] Updated weights for policy 0, policy_version 1679687 (0.0007) [2023-12-27 17:16:52,845][20381] Updated weights for policy 1, policy_version 1677905 (0.0008) [2023-12-27 17:16:52,912][20381] Updated weights for policy 1, policy_version 1677915 (0.0008) [2023-12-27 17:16:52,976][20381] Updated weights for policy 1, policy_version 1677925 (0.0005) [2023-12-27 17:16:53,047][20381] Updated weights for policy 1, policy_version 1677935 (0.0006) [2023-12-27 17:16:53,447][20347] Updated weights for policy 0, policy_version 1679697 (0.0007) [2023-12-27 17:16:53,502][20347] Updated weights for policy 0, policy_version 1679707 (0.0009) [2023-12-27 17:16:53,549][20347] Updated weights for policy 0, policy_version 1679717 (0.0008) [2023-12-27 17:16:53,580][20381] Updated weights for policy 1, policy_version 1677945 (0.0006) [2023-12-27 17:16:53,648][20381] Updated weights for policy 1, policy_version 1677955 (0.0005) [2023-12-27 17:16:53,713][20381] Updated weights for policy 1, policy_version 1677965 (0.0005) [2023-12-27 17:16:54,176][20347] Updated weights for policy 0, policy_version 1679727 (0.0006) [2023-12-27 17:16:54,230][20347] Updated weights for policy 0, policy_version 1679737 (0.0005) [2023-12-27 17:16:54,286][20347] Updated weights for policy 0, policy_version 1679747 (0.0006) [2023-12-27 17:16:54,329][20381] Updated weights for policy 1, policy_version 1677975 (0.0010) [2023-12-27 17:16:54,384][20381] Updated weights for policy 1, policy_version 1677985 (0.0010) [2023-12-27 17:16:54,439][20381] Updated weights for policy 1, policy_version 1677995 (0.0010) [2023-12-27 17:16:54,979][20347] Updated weights for policy 0, policy_version 1679757 (0.0007) [2023-12-27 17:16:55,042][20347] Updated weights for policy 0, policy_version 1679767 (0.0007) [2023-12-27 17:16:55,085][20381] Updated weights for policy 1, policy_version 1678005 (0.0010) [2023-12-27 17:16:55,090][20347] Updated weights for policy 0, policy_version 1679777 (0.0005) [2023-12-27 17:16:55,144][20381] Updated weights for policy 1, policy_version 1678015 (0.0010) [2023-12-27 17:16:55,203][20381] Updated weights for policy 1, policy_version 1678025 (0.0011) [2023-12-27 17:16:55,756][20347] Updated weights for policy 0, policy_version 1679787 (0.0005) [2023-12-27 17:16:55,825][20347] Updated weights for policy 0, policy_version 1679797 (0.0006) [2023-12-27 17:16:55,874][20381] Updated weights for policy 1, policy_version 1678035 (0.0010) [2023-12-27 17:16:55,888][20347] Updated weights for policy 0, policy_version 1679807 (0.0005) [2023-12-27 17:16:55,929][20381] Updated weights for policy 1, policy_version 1678045 (0.0010) [2023-12-27 17:16:55,988][20381] Updated weights for policy 1, policy_version 1678055 (0.0006) [2023-12-27 17:16:56,448][20347] Updated weights for policy 0, policy_version 1679817 (0.0006) [2023-12-27 17:16:56,505][20347] Updated weights for policy 0, policy_version 1679827 (0.0005) [2023-12-27 17:16:56,555][20347] Updated weights for policy 0, policy_version 1679837 (0.0005) [2023-12-27 17:16:56,607][20347] Updated weights for policy 0, policy_version 1679847 (0.0007) [2023-12-27 17:16:56,622][20381] Updated weights for policy 1, policy_version 1678065 (0.0006) [2023-12-27 17:16:56,676][20381] Updated weights for policy 1, policy_version 1678075 (0.0010) [2023-12-27 17:16:56,730][20381] Updated weights for policy 1, policy_version 1678085 (0.0010) [2023-12-27 17:16:56,789][20381] Updated weights for policy 1, policy_version 1678095 (0.0008) [2023-12-27 17:16:57,222][20347] Updated weights for policy 0, policy_version 1679857 (0.0005) [2023-12-27 17:16:57,266][20347] Updated weights for policy 0, policy_version 1679867 (0.0005) [2023-12-27 17:16:57,317][20347] Updated weights for policy 0, policy_version 1679877 (0.0006) [2023-12-27 17:16:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 859766784. Throughput: 0: 10363.4, 1: 10306.0. Samples: 859753680. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:16:57,363][19170] Avg episode reward: [(0, '990.210'), (1, '931.690')] [2023-12-27 17:16:57,412][20381] Updated weights for policy 1, policy_version 1678105 (0.0006) [2023-12-27 17:16:57,464][20381] Updated weights for policy 1, policy_version 1678115 (0.0007) [2023-12-27 17:16:57,513][20381] Updated weights for policy 1, policy_version 1678125 (0.0010) [2023-12-27 17:16:58,041][20347] Updated weights for policy 0, policy_version 1679887 (0.0008) [2023-12-27 17:16:58,086][20381] Updated weights for policy 1, policy_version 1678135 (0.0008) [2023-12-27 17:16:58,096][20347] Updated weights for policy 0, policy_version 1679897 (0.0010) [2023-12-27 17:16:58,137][20381] Updated weights for policy 1, policy_version 1678145 (0.0005) [2023-12-27 17:16:58,149][20347] Updated weights for policy 0, policy_version 1679907 (0.0008) [2023-12-27 17:16:58,202][20381] Updated weights for policy 1, policy_version 1678155 (0.0007) [2023-12-27 17:16:58,931][20347] Updated weights for policy 0, policy_version 1679917 (0.0009) [2023-12-27 17:16:58,984][20381] Updated weights for policy 1, policy_version 1678165 (0.0008) [2023-12-27 17:16:58,992][20347] Updated weights for policy 0, policy_version 1679927 (0.0008) [2023-12-27 17:16:59,044][20381] Updated weights for policy 1, policy_version 1678175 (0.0007) [2023-12-27 17:16:59,046][20347] Updated weights for policy 0, policy_version 1679937 (0.0007) [2023-12-27 17:16:59,105][20381] Updated weights for policy 1, policy_version 1678185 (0.0006) [2023-12-27 17:16:59,832][20347] Updated weights for policy 0, policy_version 1679947 (0.0006) [2023-12-27 17:16:59,869][20381] Updated weights for policy 1, policy_version 1678195 (0.0007) [2023-12-27 17:16:59,891][20347] Updated weights for policy 0, policy_version 1679957 (0.0008) [2023-12-27 17:16:59,924][20381] Updated weights for policy 1, policy_version 1678205 (0.0008) [2023-12-27 17:16:59,951][20347] Updated weights for policy 0, policy_version 1679967 (0.0008) [2023-12-27 17:16:59,978][20381] Updated weights for policy 1, policy_version 1678215 (0.0008) [2023-12-27 17:17:00,629][20347] Updated weights for policy 0, policy_version 1679977 (0.0006) [2023-12-27 17:17:00,647][20381] Updated weights for policy 1, policy_version 1678225 (0.0008) [2023-12-27 17:17:00,676][20347] Updated weights for policy 0, policy_version 1679987 (0.0010) [2023-12-27 17:17:00,701][20381] Updated weights for policy 1, policy_version 1678235 (0.0010) [2023-12-27 17:17:00,730][20347] Updated weights for policy 0, policy_version 1679997 (0.0010) [2023-12-27 17:17:00,760][20381] Updated weights for policy 1, policy_version 1678245 (0.0009) [2023-12-27 17:17:00,779][20347] Updated weights for policy 0, policy_version 1680007 (0.0010) [2023-12-27 17:17:00,818][20381] Updated weights for policy 1, policy_version 1678255 (0.0010) [2023-12-27 17:17:01,454][20347] Updated weights for policy 0, policy_version 1680017 (0.0010) [2023-12-27 17:17:01,506][20381] Updated weights for policy 1, policy_version 1678265 (0.0006) [2023-12-27 17:17:01,509][20347] Updated weights for policy 0, policy_version 1680027 (0.0010) [2023-12-27 17:17:01,557][20381] Updated weights for policy 1, policy_version 1678275 (0.0005) [2023-12-27 17:17:01,564][20347] Updated weights for policy 0, policy_version 1680037 (0.0010) [2023-12-27 17:17:01,613][20381] Updated weights for policy 1, policy_version 1678285 (0.0007) [2023-12-27 17:17:02,218][20347] Updated weights for policy 0, policy_version 1680047 (0.0008) [2023-12-27 17:17:02,278][20347] Updated weights for policy 0, policy_version 1680057 (0.0010) [2023-12-27 17:17:02,313][20381] Updated weights for policy 1, policy_version 1678295 (0.0007) [2023-12-27 17:17:02,342][20347] Updated weights for policy 0, policy_version 1680067 (0.0007) [2023-12-27 17:17:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 859856896. Throughput: 0: 10341.1, 1: 10319.6. Samples: 859874712. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.400')] [2023-12-27 17:17:02,373][20381] Updated weights for policy 1, policy_version 1678305 (0.0009) [2023-12-27 17:17:02,444][20381] Updated weights for policy 1, policy_version 1678315 (0.0008) [2023-12-27 17:17:03,063][20347] Updated weights for policy 0, policy_version 1680077 (0.0008) [2023-12-27 17:17:03,091][20381] Updated weights for policy 1, policy_version 1678325 (0.0007) [2023-12-27 17:17:03,120][20347] Updated weights for policy 0, policy_version 1680087 (0.0006) [2023-12-27 17:17:03,135][20381] Updated weights for policy 1, policy_version 1678335 (0.0006) [2023-12-27 17:17:03,177][20347] Updated weights for policy 0, policy_version 1680097 (0.0007) [2023-12-27 17:17:03,186][20381] Updated weights for policy 1, policy_version 1678345 (0.0006) [2023-12-27 17:17:03,797][20381] Updated weights for policy 1, policy_version 1678355 (0.0006) [2023-12-27 17:17:03,802][20347] Updated weights for policy 0, policy_version 1680107 (0.0007) [2023-12-27 17:17:03,855][20381] Updated weights for policy 1, policy_version 1678365 (0.0007) [2023-12-27 17:17:03,861][20347] Updated weights for policy 0, policy_version 1680117 (0.0007) [2023-12-27 17:17:03,913][20381] Updated weights for policy 1, policy_version 1678375 (0.0006) [2023-12-27 17:17:03,919][20347] Updated weights for policy 0, policy_version 1680127 (0.0006) [2023-12-27 17:17:04,613][20347] Updated weights for policy 0, policy_version 1680137 (0.0007) [2023-12-27 17:17:04,614][20381] Updated weights for policy 1, policy_version 1678385 (0.0006) [2023-12-27 17:17:04,661][20347] Updated weights for policy 0, policy_version 1680147 (0.0005) [2023-12-27 17:17:04,666][20381] Updated weights for policy 1, policy_version 1678395 (0.0006) [2023-12-27 17:17:04,710][20347] Updated weights for policy 0, policy_version 1680157 (0.0006) [2023-12-27 17:17:04,716][20381] Updated weights for policy 1, policy_version 1678405 (0.0005) [2023-12-27 17:17:04,759][20347] Updated weights for policy 0, policy_version 1680167 (0.0006) [2023-12-27 17:17:04,765][20381] Updated weights for policy 1, policy_version 1678415 (0.0005) [2023-12-27 17:17:05,382][20347] Updated weights for policy 0, policy_version 1680177 (0.0006) [2023-12-27 17:17:05,388][20291] KL-divergence is very high: 164.5433 [2023-12-27 17:17:05,427][20291] KL-divergence is very high: 190.0888 [2023-12-27 17:17:05,433][20347] Updated weights for policy 0, policy_version 1680187 (0.0007) [2023-12-27 17:17:05,445][20381] Updated weights for policy 1, policy_version 1678425 (0.0007) [2023-12-27 17:17:05,484][20347] Updated weights for policy 0, policy_version 1680197 (0.0006) [2023-12-27 17:17:05,493][20381] Updated weights for policy 1, policy_version 1678435 (0.0005) [2023-12-27 17:17:05,541][20381] Updated weights for policy 1, policy_version 1678445 (0.0007) [2023-12-27 17:17:06,129][20347] Updated weights for policy 0, policy_version 1680207 (0.0007) [2023-12-27 17:17:06,188][20347] Updated weights for policy 0, policy_version 1680217 (0.0010) [2023-12-27 17:17:06,248][20347] Updated weights for policy 0, policy_version 1680227 (0.0008) [2023-12-27 17:17:06,303][20381] Updated weights for policy 1, policy_version 1678455 (0.0006) [2023-12-27 17:17:06,360][20381] Updated weights for policy 1, policy_version 1678465 (0.0007) [2023-12-27 17:17:06,419][20381] Updated weights for policy 1, policy_version 1678475 (0.0007) [2023-12-27 17:17:06,957][20347] Updated weights for policy 0, policy_version 1680237 (0.0007) [2023-12-27 17:17:07,005][20347] Updated weights for policy 0, policy_version 1680247 (0.0007) [2023-12-27 17:17:07,028][20381] Updated weights for policy 1, policy_version 1678485 (0.0007) [2023-12-27 17:17:07,049][20347] Updated weights for policy 0, policy_version 1680257 (0.0005) [2023-12-27 17:17:07,087][20381] Updated weights for policy 1, policy_version 1678495 (0.0006) [2023-12-27 17:17:07,147][20381] Updated weights for policy 1, policy_version 1678505 (0.0007) [2023-12-27 17:17:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 859971584. Throughput: 0: 10378.1, 1: 10349.4. Samples: 859937100. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:07,363][19170] Avg episode reward: [(0, '990.210'), (1, '892.740')] [2023-12-27 17:17:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001678512_429744128.pth... [2023-12-27 17:17:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001680264_430227456.pth... [2023-12-27 17:17:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001679048_429916160.pth [2023-12-27 17:17:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001677296_429432832.pth [2023-12-27 17:17:07,694][20381] Updated weights for policy 1, policy_version 1678515 (0.0007) [2023-12-27 17:17:07,696][20347] Updated weights for policy 0, policy_version 1680267 (0.0007) [2023-12-27 17:17:07,752][20381] Updated weights for policy 1, policy_version 1678525 (0.0009) [2023-12-27 17:17:07,754][20347] Updated weights for policy 0, policy_version 1680277 (0.0010) [2023-12-27 17:17:07,808][20381] Updated weights for policy 1, policy_version 1678535 (0.0010) [2023-12-27 17:17:07,813][20347] Updated weights for policy 0, policy_version 1680287 (0.0010) [2023-12-27 17:17:08,484][20381] Updated weights for policy 1, policy_version 1678545 (0.0010) [2023-12-27 17:17:08,492][20347] Updated weights for policy 0, policy_version 1680297 (0.0009) [2023-12-27 17:17:08,544][20381] Updated weights for policy 1, policy_version 1678555 (0.0011) [2023-12-27 17:17:08,546][20347] Updated weights for policy 0, policy_version 1680307 (0.0009) [2023-12-27 17:17:08,611][20347] Updated weights for policy 0, policy_version 1680317 (0.0008) [2023-12-27 17:17:08,613][20381] Updated weights for policy 1, policy_version 1678565 (0.0006) [2023-12-27 17:17:08,675][20347] Updated weights for policy 0, policy_version 1680327 (0.0011) [2023-12-27 17:17:08,678][20381] Updated weights for policy 1, policy_version 1678575 (0.0009) [2023-12-27 17:17:09,333][20381] Updated weights for policy 1, policy_version 1678585 (0.0010) [2023-12-27 17:17:09,339][20347] Updated weights for policy 0, policy_version 1680337 (0.0010) [2023-12-27 17:17:09,398][20381] Updated weights for policy 1, policy_version 1678595 (0.0009) [2023-12-27 17:17:09,403][20347] Updated weights for policy 0, policy_version 1680347 (0.0008) [2023-12-27 17:17:09,463][20381] Updated weights for policy 1, policy_version 1678605 (0.0010) [2023-12-27 17:17:09,465][20347] Updated weights for policy 0, policy_version 1680357 (0.0008) [2023-12-27 17:17:10,176][20381] Updated weights for policy 1, policy_version 1678615 (0.0008) [2023-12-27 17:17:10,234][20381] Updated weights for policy 1, policy_version 1678625 (0.0008) [2023-12-27 17:17:10,237][20347] Updated weights for policy 0, policy_version 1680367 (0.0008) [2023-12-27 17:17:10,291][20381] Updated weights for policy 1, policy_version 1678635 (0.0007) [2023-12-27 17:17:10,299][20347] Updated weights for policy 0, policy_version 1680377 (0.0006) [2023-12-27 17:17:10,358][20347] Updated weights for policy 0, policy_version 1680387 (0.0008) [2023-12-27 17:17:10,886][20381] Updated weights for policy 1, policy_version 1678645 (0.0007) [2023-12-27 17:17:10,944][20381] Updated weights for policy 1, policy_version 1678655 (0.0008) [2023-12-27 17:17:10,996][20381] Updated weights for policy 1, policy_version 1678665 (0.0008) [2023-12-27 17:17:11,074][20347] Updated weights for policy 0, policy_version 1680397 (0.0008) [2023-12-27 17:17:11,138][20347] Updated weights for policy 0, policy_version 1680407 (0.0008) [2023-12-27 17:17:11,204][20347] Updated weights for policy 0, policy_version 1680417 (0.0010) [2023-12-27 17:17:11,777][20381] Updated weights for policy 1, policy_version 1678675 (0.0008) [2023-12-27 17:17:11,834][20381] Updated weights for policy 1, policy_version 1678685 (0.0007) [2023-12-27 17:17:11,892][20381] Updated weights for policy 1, policy_version 1678695 (0.0006) [2023-12-27 17:17:11,914][20347] Updated weights for policy 0, policy_version 1680427 (0.0009) [2023-12-27 17:17:11,973][20347] Updated weights for policy 0, policy_version 1680437 (0.0008) [2023-12-27 17:17:12,029][20347] Updated weights for policy 0, policy_version 1680447 (0.0007) [2023-12-27 17:17:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 860069888. Throughput: 0: 10364.2, 1: 10387.6. Samples: 860061124. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:12,363][19170] Avg episode reward: [(0, '970.610'), (1, '931.850')] [2023-12-27 17:17:12,505][20381] Updated weights for policy 1, policy_version 1678705 (0.0005) [2023-12-27 17:17:12,559][20381] Updated weights for policy 1, policy_version 1678715 (0.0005) [2023-12-27 17:17:12,624][20381] Updated weights for policy 1, policy_version 1678725 (0.0005) [2023-12-27 17:17:12,683][20381] Updated weights for policy 1, policy_version 1678735 (0.0007) [2023-12-27 17:17:12,795][20347] Updated weights for policy 0, policy_version 1680457 (0.0007) [2023-12-27 17:17:12,849][20347] Updated weights for policy 0, policy_version 1680467 (0.0008) [2023-12-27 17:17:12,909][20347] Updated weights for policy 0, policy_version 1680477 (0.0008) [2023-12-27 17:17:12,971][20347] Updated weights for policy 0, policy_version 1680487 (0.0008) [2023-12-27 17:17:13,274][20381] Updated weights for policy 1, policy_version 1678745 (0.0007) [2023-12-27 17:17:13,319][20381] Updated weights for policy 1, policy_version 1678755 (0.0007) [2023-12-27 17:17:13,368][20381] Updated weights for policy 1, policy_version 1678765 (0.0008) [2023-12-27 17:17:13,663][20347] Updated weights for policy 0, policy_version 1680497 (0.0006) [2023-12-27 17:17:13,723][20347] Updated weights for policy 0, policy_version 1680507 (0.0006) [2023-12-27 17:17:13,774][20347] Updated weights for policy 0, policy_version 1680517 (0.0007) [2023-12-27 17:17:14,041][20381] Updated weights for policy 1, policy_version 1678775 (0.0005) [2023-12-27 17:17:14,097][20381] Updated weights for policy 1, policy_version 1678785 (0.0007) [2023-12-27 17:17:14,152][20381] Updated weights for policy 1, policy_version 1678795 (0.0006) [2023-12-27 17:17:14,401][20347] Updated weights for policy 0, policy_version 1680527 (0.0007) [2023-12-27 17:17:14,448][20347] Updated weights for policy 0, policy_version 1680537 (0.0007) [2023-12-27 17:17:14,504][20347] Updated weights for policy 0, policy_version 1680547 (0.0008) [2023-12-27 17:17:14,738][20381] Updated weights for policy 1, policy_version 1678805 (0.0006) [2023-12-27 17:17:14,794][20381] Updated weights for policy 1, policy_version 1678815 (0.0007) [2023-12-27 17:17:14,855][20381] Updated weights for policy 1, policy_version 1678825 (0.0008) [2023-12-27 17:17:15,199][20347] Updated weights for policy 0, policy_version 1680557 (0.0007) [2023-12-27 17:17:15,262][20347] Updated weights for policy 0, policy_version 1680567 (0.0008) [2023-12-27 17:17:15,321][20347] Updated weights for policy 0, policy_version 1680577 (0.0008) [2023-12-27 17:17:15,544][20381] Updated weights for policy 1, policy_version 1678835 (0.0008) [2023-12-27 17:17:15,589][20381] Updated weights for policy 1, policy_version 1678845 (0.0007) [2023-12-27 17:17:15,633][20381] Updated weights for policy 1, policy_version 1678855 (0.0008) [2023-12-27 17:17:16,009][20347] Updated weights for policy 0, policy_version 1680587 (0.0008) [2023-12-27 17:17:16,065][20347] Updated weights for policy 0, policy_version 1680597 (0.0009) [2023-12-27 17:17:16,120][20347] Updated weights for policy 0, policy_version 1680607 (0.0010) [2023-12-27 17:17:16,362][20381] Updated weights for policy 1, policy_version 1678865 (0.0007) [2023-12-27 17:17:16,413][20381] Updated weights for policy 1, policy_version 1678875 (0.0007) [2023-12-27 17:17:16,466][20381] Updated weights for policy 1, policy_version 1678885 (0.0008) [2023-12-27 17:17:16,512][20381] Updated weights for policy 1, policy_version 1678895 (0.0007) [2023-12-27 17:17:16,753][20347] Updated weights for policy 0, policy_version 1680617 (0.0010) [2023-12-27 17:17:16,801][20347] Updated weights for policy 0, policy_version 1680627 (0.0010) [2023-12-27 17:17:16,849][20347] Updated weights for policy 0, policy_version 1680637 (0.0010) [2023-12-27 17:17:16,897][20347] Updated weights for policy 0, policy_version 1680647 (0.0010) [2023-12-27 17:17:17,230][20381] Updated weights for policy 1, policy_version 1678905 (0.0008) [2023-12-27 17:17:17,288][20381] Updated weights for policy 1, policy_version 1678915 (0.0007) [2023-12-27 17:17:17,352][20381] Updated weights for policy 1, policy_version 1678925 (0.0007) [2023-12-27 17:17:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 860168192. Throughput: 0: 10245.3, 1: 10356.1. Samples: 860183576. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:17,363][19170] Avg episode reward: [(0, '980.400'), (1, '961.100')] [2023-12-27 17:17:17,570][20347] Updated weights for policy 0, policy_version 1680657 (0.0008) [2023-12-27 17:17:17,629][20347] Updated weights for policy 0, policy_version 1680667 (0.0008) [2023-12-27 17:17:17,681][20347] Updated weights for policy 0, policy_version 1680677 (0.0008) [2023-12-27 17:17:17,971][20381] Updated weights for policy 1, policy_version 1678935 (0.0007) [2023-12-27 17:17:18,032][20381] Updated weights for policy 1, policy_version 1678945 (0.0007) [2023-12-27 17:17:18,089][20381] Updated weights for policy 1, policy_version 1678955 (0.0007) [2023-12-27 17:17:18,411][20347] Updated weights for policy 0, policy_version 1680687 (0.0010) [2023-12-27 17:17:18,482][20347] Updated weights for policy 0, policy_version 1680697 (0.0010) [2023-12-27 17:17:18,536][20347] Updated weights for policy 0, policy_version 1680707 (0.0010) [2023-12-27 17:17:18,786][20381] Updated weights for policy 1, policy_version 1678965 (0.0007) [2023-12-27 17:17:18,838][20381] Updated weights for policy 1, policy_version 1678975 (0.0008) [2023-12-27 17:17:18,900][20381] Updated weights for policy 1, policy_version 1678985 (0.0007) [2023-12-27 17:17:19,213][20347] Updated weights for policy 0, policy_version 1680717 (0.0008) [2023-12-27 17:17:19,275][20347] Updated weights for policy 0, policy_version 1680727 (0.0009) [2023-12-27 17:17:19,334][20347] Updated weights for policy 0, policy_version 1680737 (0.0007) [2023-12-27 17:17:19,589][20381] Updated weights for policy 1, policy_version 1678995 (0.0007) [2023-12-27 17:17:19,638][20381] Updated weights for policy 1, policy_version 1679005 (0.0007) [2023-12-27 17:17:19,682][20381] Updated weights for policy 1, policy_version 1679015 (0.0007) [2023-12-27 17:17:20,061][20347] Updated weights for policy 0, policy_version 1680747 (0.0008) [2023-12-27 17:17:20,120][20347] Updated weights for policy 0, policy_version 1680757 (0.0006) [2023-12-27 17:17:20,173][20347] Updated weights for policy 0, policy_version 1680767 (0.0007) [2023-12-27 17:17:20,444][20381] Updated weights for policy 1, policy_version 1679025 (0.0007) [2023-12-27 17:17:20,494][20381] Updated weights for policy 1, policy_version 1679035 (0.0010) [2023-12-27 17:17:20,544][20381] Updated weights for policy 1, policy_version 1679045 (0.0009) [2023-12-27 17:17:20,604][20381] Updated weights for policy 1, policy_version 1679055 (0.0010) [2023-12-27 17:17:20,854][20347] Updated weights for policy 0, policy_version 1680777 (0.0008) [2023-12-27 17:17:20,921][20347] Updated weights for policy 0, policy_version 1680787 (0.0011) [2023-12-27 17:17:20,989][20347] Updated weights for policy 0, policy_version 1680797 (0.0011) [2023-12-27 17:17:21,055][20347] Updated weights for policy 0, policy_version 1680807 (0.0009) [2023-12-27 17:17:21,375][20381] Updated weights for policy 1, policy_version 1679065 (0.0011) [2023-12-27 17:17:21,441][20381] Updated weights for policy 1, policy_version 1679075 (0.0011) [2023-12-27 17:17:21,494][20381] Updated weights for policy 1, policy_version 1679085 (0.0010) [2023-12-27 17:17:21,792][20347] Updated weights for policy 0, policy_version 1680817 (0.0007) [2023-12-27 17:17:21,861][20347] Updated weights for policy 0, policy_version 1680827 (0.0006) [2023-12-27 17:17:21,928][20347] Updated weights for policy 0, policy_version 1680837 (0.0006) [2023-12-27 17:17:22,213][20381] Updated weights for policy 1, policy_version 1679095 (0.0011) [2023-12-27 17:17:22,267][20381] Updated weights for policy 1, policy_version 1679105 (0.0010) [2023-12-27 17:17:22,330][20381] Updated weights for policy 1, policy_version 1679115 (0.0008) [2023-12-27 17:17:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 860274688. Throughput: 0: 10230.0, 1: 10330.8. Samples: 860243296. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:22,363][19170] Avg episode reward: [(0, '970.620'), (1, '990.240')] [2023-12-27 17:17:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001679120_429899776.pth... [2023-12-27 17:17:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001680840_430374912.pth... [2023-12-27 17:17:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001677904_429588480.pth [2023-12-27 17:17:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001679656_430071808.pth [2023-12-27 17:17:22,668][20347] Updated weights for policy 0, policy_version 1680847 (0.0007) [2023-12-27 17:17:22,730][20347] Updated weights for policy 0, policy_version 1680857 (0.0009) [2023-12-27 17:17:22,794][20347] Updated weights for policy 0, policy_version 1680867 (0.0008) [2023-12-27 17:17:23,023][20381] Updated weights for policy 1, policy_version 1679125 (0.0009) [2023-12-27 17:17:23,083][20381] Updated weights for policy 1, policy_version 1679135 (0.0008) [2023-12-27 17:17:23,146][20381] Updated weights for policy 1, policy_version 1679145 (0.0007) [2023-12-27 17:17:23,464][20347] Updated weights for policy 0, policy_version 1680877 (0.0008) [2023-12-27 17:17:23,524][20347] Updated weights for policy 0, policy_version 1680887 (0.0011) [2023-12-27 17:17:23,579][20347] Updated weights for policy 0, policy_version 1680897 (0.0010) [2023-12-27 17:17:23,768][20381] Updated weights for policy 1, policy_version 1679155 (0.0007) [2023-12-27 17:17:23,818][20381] Updated weights for policy 1, policy_version 1679165 (0.0007) [2023-12-27 17:17:23,864][20381] Updated weights for policy 1, policy_version 1679175 (0.0005) [2023-12-27 17:17:24,250][20347] Updated weights for policy 0, policy_version 1680907 (0.0011) [2023-12-27 17:17:24,305][20347] Updated weights for policy 0, policy_version 1680917 (0.0010) [2023-12-27 17:17:24,362][20347] Updated weights for policy 0, policy_version 1680927 (0.0010) [2023-12-27 17:17:24,547][20381] Updated weights for policy 1, policy_version 1679185 (0.0008) [2023-12-27 17:17:24,605][20381] Updated weights for policy 1, policy_version 1679195 (0.0008) [2023-12-27 17:17:24,660][20381] Updated weights for policy 1, policy_version 1679205 (0.0007) [2023-12-27 17:17:24,710][20381] Updated weights for policy 1, policy_version 1679215 (0.0007) [2023-12-27 17:17:24,926][20347] Updated weights for policy 0, policy_version 1680937 (0.0008) [2023-12-27 17:17:24,987][20347] Updated weights for policy 0, policy_version 1680947 (0.0005) [2023-12-27 17:17:25,034][20347] Updated weights for policy 0, policy_version 1680957 (0.0005) [2023-12-27 17:17:25,085][20347] Updated weights for policy 0, policy_version 1680967 (0.0005) [2023-12-27 17:17:25,410][20381] Updated weights for policy 1, policy_version 1679225 (0.0010) [2023-12-27 17:17:25,461][20381] Updated weights for policy 1, policy_version 1679235 (0.0011) [2023-12-27 17:17:25,525][20381] Updated weights for policy 1, policy_version 1679245 (0.0010) [2023-12-27 17:17:25,710][20347] Updated weights for policy 0, policy_version 1680977 (0.0010) [2023-12-27 17:17:25,762][20347] Updated weights for policy 0, policy_version 1680987 (0.0009) [2023-12-27 17:17:25,810][20347] Updated weights for policy 0, policy_version 1680997 (0.0005) [2023-12-27 17:17:26,149][20381] Updated weights for policy 1, policy_version 1679255 (0.0011) [2023-12-27 17:17:26,210][20381] Updated weights for policy 1, policy_version 1679265 (0.0008) [2023-12-27 17:17:26,264][20381] Updated weights for policy 1, policy_version 1679275 (0.0008) [2023-12-27 17:17:26,449][20347] Updated weights for policy 0, policy_version 1681007 (0.0009) [2023-12-27 17:17:26,504][20347] Updated weights for policy 0, policy_version 1681017 (0.0008) [2023-12-27 17:17:26,556][20347] Updated weights for policy 0, policy_version 1681027 (0.0006) [2023-12-27 17:17:26,904][20381] Updated weights for policy 1, policy_version 1679285 (0.0008) [2023-12-27 17:17:26,948][20381] Updated weights for policy 1, policy_version 1679295 (0.0007) [2023-12-27 17:17:27,003][20381] Updated weights for policy 1, policy_version 1679305 (0.0010) [2023-12-27 17:17:27,143][20347] Updated weights for policy 0, policy_version 1681037 (0.0007) [2023-12-27 17:17:27,200][20347] Updated weights for policy 0, policy_version 1681047 (0.0007) [2023-12-27 17:17:27,246][20347] Updated weights for policy 0, policy_version 1681057 (0.0007) [2023-12-27 17:17:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 860381184. Throughput: 0: 10247.6, 1: 10321.1. Samples: 860365732. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:27,363][19170] Avg episode reward: [(0, '960.840'), (1, '990.220')] [2023-12-27 17:17:27,698][20381] Updated weights for policy 1, policy_version 1679315 (0.0010) [2023-12-27 17:17:27,743][20381] Updated weights for policy 1, policy_version 1679325 (0.0010) [2023-12-27 17:17:27,794][20381] Updated weights for policy 1, policy_version 1679335 (0.0010) [2023-12-27 17:17:27,832][20347] Updated weights for policy 0, policy_version 1681067 (0.0007) [2023-12-27 17:17:27,884][20347] Updated weights for policy 0, policy_version 1681077 (0.0007) [2023-12-27 17:17:27,937][20347] Updated weights for policy 0, policy_version 1681087 (0.0007) [2023-12-27 17:17:28,422][20381] Updated weights for policy 1, policy_version 1679345 (0.0011) [2023-12-27 17:17:28,478][20381] Updated weights for policy 1, policy_version 1679355 (0.0006) [2023-12-27 17:17:28,537][20381] Updated weights for policy 1, policy_version 1679365 (0.0005) [2023-12-27 17:17:28,546][20347] Updated weights for policy 0, policy_version 1681097 (0.0005) [2023-12-27 17:17:28,597][20381] Updated weights for policy 1, policy_version 1679375 (0.0010) [2023-12-27 17:17:28,603][20347] Updated weights for policy 0, policy_version 1681107 (0.0006) [2023-12-27 17:17:28,662][20347] Updated weights for policy 0, policy_version 1681117 (0.0008) [2023-12-27 17:17:28,725][20347] Updated weights for policy 0, policy_version 1681127 (0.0009) [2023-12-27 17:17:29,238][20381] Updated weights for policy 1, policy_version 1679385 (0.0008) [2023-12-27 17:17:29,282][20347] Updated weights for policy 0, policy_version 1681137 (0.0008) [2023-12-27 17:17:29,300][20381] Updated weights for policy 1, policy_version 1679395 (0.0008) [2023-12-27 17:17:29,339][20347] Updated weights for policy 0, policy_version 1681147 (0.0007) [2023-12-27 17:17:29,361][20381] Updated weights for policy 1, policy_version 1679405 (0.0008) [2023-12-27 17:17:29,406][20347] Updated weights for policy 0, policy_version 1681157 (0.0010) [2023-12-27 17:17:30,090][20381] Updated weights for policy 1, policy_version 1679415 (0.0008) [2023-12-27 17:17:30,099][20347] Updated weights for policy 0, policy_version 1681167 (0.0010) [2023-12-27 17:17:30,157][20381] Updated weights for policy 1, policy_version 1679425 (0.0006) [2023-12-27 17:17:30,159][20347] Updated weights for policy 0, policy_version 1681177 (0.0010) [2023-12-27 17:17:30,219][20347] Updated weights for policy 0, policy_version 1681187 (0.0011) [2023-12-27 17:17:30,223][20381] Updated weights for policy 1, policy_version 1679435 (0.0007) [2023-12-27 17:17:30,836][20347] Updated weights for policy 0, policy_version 1681197 (0.0008) [2023-12-27 17:17:30,840][20381] Updated weights for policy 1, policy_version 1679445 (0.0008) [2023-12-27 17:17:30,884][20347] Updated weights for policy 0, policy_version 1681207 (0.0005) [2023-12-27 17:17:30,890][20381] Updated weights for policy 1, policy_version 1679455 (0.0005) [2023-12-27 17:17:30,931][20347] Updated weights for policy 0, policy_version 1681217 (0.0006) [2023-12-27 17:17:30,941][20381] Updated weights for policy 1, policy_version 1679465 (0.0005) [2023-12-27 17:17:31,582][20381] Updated weights for policy 1, policy_version 1679475 (0.0006) [2023-12-27 17:17:31,646][20381] Updated weights for policy 1, policy_version 1679485 (0.0008) [2023-12-27 17:17:31,683][20347] Updated weights for policy 0, policy_version 1681227 (0.0007) [2023-12-27 17:17:31,700][20381] Updated weights for policy 1, policy_version 1679495 (0.0006) [2023-12-27 17:17:31,753][20347] Updated weights for policy 0, policy_version 1681237 (0.0008) [2023-12-27 17:17:31,802][20347] Updated weights for policy 0, policy_version 1681247 (0.0010) [2023-12-27 17:17:32,304][20381] Updated weights for policy 1, policy_version 1679505 (0.0008) [2023-12-27 17:17:32,354][20381] Updated weights for policy 1, policy_version 1679515 (0.0006) [2023-12-27 17:17:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 860479488. Throughput: 0: 10250.4, 1: 10373.2. Samples: 860493336. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:32,363][19170] Avg episode reward: [(0, '990.220'), (1, '951.160')] [2023-12-27 17:17:32,410][20381] Updated weights for policy 1, policy_version 1679525 (0.0008) [2023-12-27 17:17:32,462][20381] Updated weights for policy 1, policy_version 1679535 (0.0007) [2023-12-27 17:17:32,553][20347] Updated weights for policy 0, policy_version 1681257 (0.0007) [2023-12-27 17:17:32,612][20347] Updated weights for policy 0, policy_version 1681267 (0.0007) [2023-12-27 17:17:32,670][20347] Updated weights for policy 0, policy_version 1681277 (0.0009) [2023-12-27 17:17:32,719][20347] Updated weights for policy 0, policy_version 1681287 (0.0009) [2023-12-27 17:17:33,123][20381] Updated weights for policy 1, policy_version 1679545 (0.0008) [2023-12-27 17:17:33,187][20381] Updated weights for policy 1, policy_version 1679555 (0.0008) [2023-12-27 17:17:33,243][20381] Updated weights for policy 1, policy_version 1679565 (0.0008) [2023-12-27 17:17:33,362][20347] Updated weights for policy 0, policy_version 1681297 (0.0005) [2023-12-27 17:17:33,423][20347] Updated weights for policy 0, policy_version 1681307 (0.0005) [2023-12-27 17:17:33,477][20347] Updated weights for policy 0, policy_version 1681317 (0.0005) [2023-12-27 17:17:33,889][20381] Updated weights for policy 1, policy_version 1679575 (0.0009) [2023-12-27 17:17:33,936][20381] Updated weights for policy 1, policy_version 1679585 (0.0006) [2023-12-27 17:17:33,998][20381] Updated weights for policy 1, policy_version 1679595 (0.0010) [2023-12-27 17:17:34,146][20347] Updated weights for policy 0, policy_version 1681327 (0.0007) [2023-12-27 17:17:34,208][20347] Updated weights for policy 0, policy_version 1681337 (0.0007) [2023-12-27 17:17:34,267][20347] Updated weights for policy 0, policy_version 1681347 (0.0007) [2023-12-27 17:17:34,659][20381] Updated weights for policy 1, policy_version 1679605 (0.0006) [2023-12-27 17:17:34,710][20381] Updated weights for policy 1, policy_version 1679615 (0.0006) [2023-12-27 17:17:34,757][20381] Updated weights for policy 1, policy_version 1679625 (0.0008) [2023-12-27 17:17:34,919][20347] Updated weights for policy 0, policy_version 1681357 (0.0007) [2023-12-27 17:17:34,967][20347] Updated weights for policy 0, policy_version 1681367 (0.0008) [2023-12-27 17:17:35,016][20347] Updated weights for policy 0, policy_version 1681377 (0.0007) [2023-12-27 17:17:35,453][20381] Updated weights for policy 1, policy_version 1679635 (0.0008) [2023-12-27 17:17:35,508][20381] Updated weights for policy 1, policy_version 1679645 (0.0007) [2023-12-27 17:17:35,560][20381] Updated weights for policy 1, policy_version 1679655 (0.0008) [2023-12-27 17:17:35,740][20347] Updated weights for policy 0, policy_version 1681387 (0.0007) [2023-12-27 17:17:35,792][20347] Updated weights for policy 0, policy_version 1681397 (0.0010) [2023-12-27 17:17:35,838][20347] Updated weights for policy 0, policy_version 1681407 (0.0010) [2023-12-27 17:17:36,291][20381] Updated weights for policy 1, policy_version 1679665 (0.0008) [2023-12-27 17:17:36,339][20381] Updated weights for policy 1, policy_version 1679675 (0.0008) [2023-12-27 17:17:36,391][20381] Updated weights for policy 1, policy_version 1679685 (0.0008) [2023-12-27 17:17:36,442][20381] Updated weights for policy 1, policy_version 1679695 (0.0007) [2023-12-27 17:17:36,544][20347] Updated weights for policy 0, policy_version 1681417 (0.0010) [2023-12-27 17:17:36,608][20347] Updated weights for policy 0, policy_version 1681427 (0.0011) [2023-12-27 17:17:36,668][20347] Updated weights for policy 0, policy_version 1681437 (0.0011) [2023-12-27 17:17:36,721][20347] Updated weights for policy 0, policy_version 1681447 (0.0008) [2023-12-27 17:17:37,143][20381] Updated weights for policy 1, policy_version 1679705 (0.0006) [2023-12-27 17:17:37,206][20381] Updated weights for policy 1, policy_version 1679715 (0.0006) [2023-12-27 17:17:37,273][20381] Updated weights for policy 1, policy_version 1679725 (0.0005) [2023-12-27 17:17:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 860585984. Throughput: 0: 10219.2, 1: 10364.3. Samples: 860555548. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.700')] [2023-12-27 17:17:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001679728_430055424.pth... [2023-12-27 17:17:37,374][20347] Updated weights for policy 0, policy_version 1681457 (0.0010) [2023-12-27 17:17:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001678512_429744128.pth [2023-12-27 17:17:37,430][20347] Updated weights for policy 0, policy_version 1681467 (0.0007) [2023-12-27 17:17:37,487][20347] Updated weights for policy 0, policy_version 1681477 (0.0005) [2023-12-27 17:17:37,506][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001681480_430538752.pth... [2023-12-27 17:17:37,510][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001680264_430227456.pth [2023-12-27 17:17:37,885][20381] Updated weights for policy 1, policy_version 1679735 (0.0005) [2023-12-27 17:17:37,950][20381] Updated weights for policy 1, policy_version 1679745 (0.0005) [2023-12-27 17:17:38,014][20381] Updated weights for policy 1, policy_version 1679755 (0.0005) [2023-12-27 17:17:38,097][20347] Updated weights for policy 0, policy_version 1681487 (0.0009) [2023-12-27 17:17:38,149][20347] Updated weights for policy 0, policy_version 1681497 (0.0010) [2023-12-27 17:17:38,201][20347] Updated weights for policy 0, policy_version 1681507 (0.0010) [2023-12-27 17:17:38,585][20381] Updated weights for policy 1, policy_version 1679765 (0.0006) [2023-12-27 17:17:38,641][20381] Updated weights for policy 1, policy_version 1679775 (0.0006) [2023-12-27 17:17:38,700][20381] Updated weights for policy 1, policy_version 1679785 (0.0007) [2023-12-27 17:17:38,929][20347] Updated weights for policy 0, policy_version 1681517 (0.0010) [2023-12-27 17:17:38,983][20347] Updated weights for policy 0, policy_version 1681527 (0.0010) [2023-12-27 17:17:39,038][20347] Updated weights for policy 0, policy_version 1681537 (0.0010) [2023-12-27 17:17:39,400][20381] Updated weights for policy 1, policy_version 1679795 (0.0008) [2023-12-27 17:17:39,472][20381] Updated weights for policy 1, policy_version 1679805 (0.0009) [2023-12-27 17:17:39,530][20381] Updated weights for policy 1, policy_version 1679815 (0.0009) [2023-12-27 17:17:39,789][20347] Updated weights for policy 0, policy_version 1681547 (0.0010) [2023-12-27 17:17:39,851][20347] Updated weights for policy 0, policy_version 1681557 (0.0009) [2023-12-27 17:17:39,907][20347] Updated weights for policy 0, policy_version 1681567 (0.0007) [2023-12-27 17:17:40,289][20381] Updated weights for policy 1, policy_version 1679825 (0.0008) [2023-12-27 17:17:40,335][20381] Updated weights for policy 1, policy_version 1679835 (0.0010) [2023-12-27 17:17:40,395][20381] Updated weights for policy 1, policy_version 1679845 (0.0007) [2023-12-27 17:17:40,448][20381] Updated weights for policy 1, policy_version 1679855 (0.0010) [2023-12-27 17:17:40,601][20347] Updated weights for policy 0, policy_version 1681577 (0.0008) [2023-12-27 17:17:40,659][20347] Updated weights for policy 0, policy_version 1681587 (0.0006) [2023-12-27 17:17:40,717][20347] Updated weights for policy 0, policy_version 1681597 (0.0008) [2023-12-27 17:17:40,770][20347] Updated weights for policy 0, policy_version 1681607 (0.0008) [2023-12-27 17:17:41,120][20381] Updated weights for policy 1, policy_version 1679865 (0.0010) [2023-12-27 17:17:41,189][20381] Updated weights for policy 1, policy_version 1679875 (0.0011) [2023-12-27 17:17:41,249][20381] Updated weights for policy 1, policy_version 1679885 (0.0010) [2023-12-27 17:17:41,499][20347] Updated weights for policy 0, policy_version 1681617 (0.0008) [2023-12-27 17:17:41,561][20347] Updated weights for policy 0, policy_version 1681627 (0.0007) [2023-12-27 17:17:41,626][20347] Updated weights for policy 0, policy_version 1681637 (0.0008) [2023-12-27 17:17:41,940][20381] Updated weights for policy 1, policy_version 1679895 (0.0007) [2023-12-27 17:17:41,998][20381] Updated weights for policy 1, policy_version 1679905 (0.0007) [2023-12-27 17:17:42,060][20381] Updated weights for policy 1, policy_version 1679915 (0.0007) [2023-12-27 17:17:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 860684288. Throughput: 0: 10221.5, 1: 10306.1. Samples: 860677424. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:42,364][19170] Avg episode reward: [(0, '990.220'), (1, '961.070')] [2023-12-27 17:17:42,380][20347] Updated weights for policy 0, policy_version 1681647 (0.0008) [2023-12-27 17:17:42,437][20347] Updated weights for policy 0, policy_version 1681657 (0.0007) [2023-12-27 17:17:42,490][20347] Updated weights for policy 0, policy_version 1681667 (0.0006) [2023-12-27 17:17:42,725][20381] Updated weights for policy 1, policy_version 1679925 (0.0007) [2023-12-27 17:17:42,785][20381] Updated weights for policy 1, policy_version 1679935 (0.0006) [2023-12-27 17:17:42,837][20381] Updated weights for policy 1, policy_version 1679945 (0.0005) [2023-12-27 17:17:43,092][20347] Updated weights for policy 0, policy_version 1681677 (0.0008) [2023-12-27 17:17:43,139][20347] Updated weights for policy 0, policy_version 1681687 (0.0008) [2023-12-27 17:17:43,187][20347] Updated weights for policy 0, policy_version 1681697 (0.0008) [2023-12-27 17:17:43,491][20381] Updated weights for policy 1, policy_version 1679955 (0.0007) [2023-12-27 17:17:43,539][20381] Updated weights for policy 1, policy_version 1679965 (0.0007) [2023-12-27 17:17:43,596][20381] Updated weights for policy 1, policy_version 1679975 (0.0007) [2023-12-27 17:17:43,861][20347] Updated weights for policy 0, policy_version 1681707 (0.0007) [2023-12-27 17:17:43,917][20347] Updated weights for policy 0, policy_version 1681717 (0.0006) [2023-12-27 17:17:43,969][20347] Updated weights for policy 0, policy_version 1681727 (0.0008) [2023-12-27 17:17:44,303][20381] Updated weights for policy 1, policy_version 1679985 (0.0007) [2023-12-27 17:17:44,362][20381] Updated weights for policy 1, policy_version 1679995 (0.0008) [2023-12-27 17:17:44,410][20381] Updated weights for policy 1, policy_version 1680005 (0.0008) [2023-12-27 17:17:44,458][20381] Updated weights for policy 1, policy_version 1680015 (0.0008) [2023-12-27 17:17:44,613][20347] Updated weights for policy 0, policy_version 1681737 (0.0006) [2023-12-27 17:17:44,669][20347] Updated weights for policy 0, policy_version 1681747 (0.0008) [2023-12-27 17:17:44,720][20347] Updated weights for policy 0, policy_version 1681757 (0.0007) [2023-12-27 17:17:44,779][20347] Updated weights for policy 0, policy_version 1681767 (0.0007) [2023-12-27 17:17:45,179][20381] Updated weights for policy 1, policy_version 1680025 (0.0007) [2023-12-27 17:17:45,240][20381] Updated weights for policy 1, policy_version 1680035 (0.0006) [2023-12-27 17:17:45,298][20381] Updated weights for policy 1, policy_version 1680045 (0.0006) [2023-12-27 17:17:45,490][20347] Updated weights for policy 0, policy_version 1681777 (0.0008) [2023-12-27 17:17:45,548][20347] Updated weights for policy 0, policy_version 1681787 (0.0008) [2023-12-27 17:17:45,606][20347] Updated weights for policy 0, policy_version 1681797 (0.0008) [2023-12-27 17:17:45,868][20381] Updated weights for policy 1, policy_version 1680055 (0.0005) [2023-12-27 17:17:45,923][20381] Updated weights for policy 1, policy_version 1680065 (0.0005) [2023-12-27 17:17:45,975][20381] Updated weights for policy 1, policy_version 1680075 (0.0008) [2023-12-27 17:17:46,340][20347] Updated weights for policy 0, policy_version 1681807 (0.0009) [2023-12-27 17:17:46,394][20347] Updated weights for policy 0, policy_version 1681817 (0.0010) [2023-12-27 17:17:46,443][20347] Updated weights for policy 0, policy_version 1681827 (0.0007) [2023-12-27 17:17:46,587][20381] Updated weights for policy 1, policy_version 1680085 (0.0006) [2023-12-27 17:17:46,638][20381] Updated weights for policy 1, policy_version 1680095 (0.0005) [2023-12-27 17:17:46,691][20381] Updated weights for policy 1, policy_version 1680105 (0.0005) [2023-12-27 17:17:47,113][20347] Updated weights for policy 0, policy_version 1681837 (0.0008) [2023-12-27 17:17:47,173][20347] Updated weights for policy 0, policy_version 1681847 (0.0007) [2023-12-27 17:17:47,230][20347] Updated weights for policy 0, policy_version 1681857 (0.0007) [2023-12-27 17:17:47,284][20381] Updated weights for policy 1, policy_version 1680115 (0.0006) [2023-12-27 17:17:47,348][20381] Updated weights for policy 1, policy_version 1680125 (0.0007) [2023-12-27 17:17:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 860790784. Throughput: 0: 10215.9, 1: 10377.9. Samples: 860801432. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:47,363][19170] Avg episode reward: [(0, '980.460'), (1, '912.250')] [2023-12-27 17:17:47,400][20381] Updated weights for policy 1, policy_version 1680135 (0.0008) [2023-12-27 17:17:47,930][20347] Updated weights for policy 0, policy_version 1681867 (0.0008) [2023-12-27 17:17:47,975][20347] Updated weights for policy 0, policy_version 1681877 (0.0008) [2023-12-27 17:17:48,016][20381] Updated weights for policy 1, policy_version 1680145 (0.0007) [2023-12-27 17:17:48,028][20347] Updated weights for policy 0, policy_version 1681887 (0.0008) [2023-12-27 17:17:48,069][20381] Updated weights for policy 1, policy_version 1680155 (0.0006) [2023-12-27 17:17:48,114][20381] Updated weights for policy 1, policy_version 1680165 (0.0006) [2023-12-27 17:17:48,161][20381] Updated weights for policy 1, policy_version 1680175 (0.0007) [2023-12-27 17:17:48,759][20347] Updated weights for policy 0, policy_version 1681897 (0.0006) [2023-12-27 17:17:48,814][20347] Updated weights for policy 0, policy_version 1681907 (0.0008) [2023-12-27 17:17:48,853][20381] Updated weights for policy 1, policy_version 1680185 (0.0006) [2023-12-27 17:17:48,870][20347] Updated weights for policy 0, policy_version 1681917 (0.0006) [2023-12-27 17:17:48,913][20381] Updated weights for policy 1, policy_version 1680195 (0.0005) [2023-12-27 17:17:48,929][20347] Updated weights for policy 0, policy_version 1681927 (0.0008) [2023-12-27 17:17:48,968][20381] Updated weights for policy 1, policy_version 1680205 (0.0007) [2023-12-27 17:17:49,607][20381] Updated weights for policy 1, policy_version 1680215 (0.0008) [2023-12-27 17:17:49,677][20381] Updated weights for policy 1, policy_version 1680225 (0.0008) [2023-12-27 17:17:49,698][20347] Updated weights for policy 0, policy_version 1681937 (0.0010) [2023-12-27 17:17:49,744][20381] Updated weights for policy 1, policy_version 1680235 (0.0008) [2023-12-27 17:17:49,754][20347] Updated weights for policy 0, policy_version 1681947 (0.0011) [2023-12-27 17:17:49,807][20347] Updated weights for policy 0, policy_version 1681957 (0.0009) [2023-12-27 17:17:50,425][20381] Updated weights for policy 1, policy_version 1680245 (0.0008) [2023-12-27 17:17:50,480][20381] Updated weights for policy 1, policy_version 1680255 (0.0008) [2023-12-27 17:17:50,510][20347] Updated weights for policy 0, policy_version 1681967 (0.0010) [2023-12-27 17:17:50,535][20381] Updated weights for policy 1, policy_version 1680265 (0.0006) [2023-12-27 17:17:50,573][20347] Updated weights for policy 0, policy_version 1681977 (0.0008) [2023-12-27 17:17:50,636][20347] Updated weights for policy 0, policy_version 1681987 (0.0008) [2023-12-27 17:17:51,271][20347] Updated weights for policy 0, policy_version 1681997 (0.0007) [2023-12-27 17:17:51,314][20381] Updated weights for policy 1, policy_version 1680275 (0.0007) [2023-12-27 17:17:51,324][20347] Updated weights for policy 0, policy_version 1682007 (0.0007) [2023-12-27 17:17:51,373][20381] Updated weights for policy 1, policy_version 1680285 (0.0008) [2023-12-27 17:17:51,390][20347] Updated weights for policy 0, policy_version 1682017 (0.0007) [2023-12-27 17:17:51,438][20381] Updated weights for policy 1, policy_version 1680295 (0.0007) [2023-12-27 17:17:52,087][20381] Updated weights for policy 1, policy_version 1680305 (0.0007) [2023-12-27 17:17:52,129][20347] Updated weights for policy 0, policy_version 1682027 (0.0006) [2023-12-27 17:17:52,139][20381] Updated weights for policy 1, policy_version 1680315 (0.0005) [2023-12-27 17:17:52,184][20347] Updated weights for policy 0, policy_version 1682037 (0.0008) [2023-12-27 17:17:52,188][20381] Updated weights for policy 1, policy_version 1680325 (0.0005) [2023-12-27 17:17:52,238][20381] Updated weights for policy 1, policy_version 1680335 (0.0006) [2023-12-27 17:17:52,241][20347] Updated weights for policy 0, policy_version 1682047 (0.0008) [2023-12-27 17:17:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 860897280. Throughput: 0: 10159.2, 1: 10375.0. Samples: 860861140. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:52,363][19170] Avg episode reward: [(0, '990.240'), (1, '931.720')] [2023-12-27 17:17:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001680336_430211072.pth... [2023-12-27 17:17:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001682056_430686208.pth... [2023-12-27 17:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001680840_430374912.pth [2023-12-27 17:17:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001679120_429899776.pth [2023-12-27 17:17:52,919][20347] Updated weights for policy 0, policy_version 1682057 (0.0007) [2023-12-27 17:17:52,939][20381] Updated weights for policy 1, policy_version 1680345 (0.0006) [2023-12-27 17:17:52,977][20347] Updated weights for policy 0, policy_version 1682067 (0.0006) [2023-12-27 17:17:52,991][20381] Updated weights for policy 1, policy_version 1680355 (0.0006) [2023-12-27 17:17:53,033][20347] Updated weights for policy 0, policy_version 1682077 (0.0006) [2023-12-27 17:17:53,038][20381] Updated weights for policy 1, policy_version 1680365 (0.0007) [2023-12-27 17:17:53,084][20347] Updated weights for policy 0, policy_version 1682087 (0.0007) [2023-12-27 17:17:53,672][20381] Updated weights for policy 1, policy_version 1680375 (0.0007) [2023-12-27 17:17:53,717][20381] Updated weights for policy 1, policy_version 1680385 (0.0007) [2023-12-27 17:17:53,760][20347] Updated weights for policy 0, policy_version 1682097 (0.0006) [2023-12-27 17:17:53,765][20381] Updated weights for policy 1, policy_version 1680395 (0.0005) [2023-12-27 17:17:53,813][20347] Updated weights for policy 0, policy_version 1682107 (0.0006) [2023-12-27 17:17:53,865][20347] Updated weights for policy 0, policy_version 1682117 (0.0008) [2023-12-27 17:17:54,417][20381] Updated weights for policy 1, policy_version 1680405 (0.0005) [2023-12-27 17:17:54,481][20381] Updated weights for policy 1, policy_version 1680415 (0.0005) [2023-12-27 17:17:54,565][20381] Updated weights for policy 1, policy_version 1680425 (0.0005) [2023-12-27 17:17:54,566][20347] Updated weights for policy 0, policy_version 1682127 (0.0008) [2023-12-27 17:17:54,619][20347] Updated weights for policy 0, policy_version 1682137 (0.0008) [2023-12-27 17:17:54,676][20347] Updated weights for policy 0, policy_version 1682147 (0.0007) [2023-12-27 17:17:55,143][20381] Updated weights for policy 1, policy_version 1680435 (0.0006) [2023-12-27 17:17:55,197][20381] Updated weights for policy 1, policy_version 1680445 (0.0007) [2023-12-27 17:17:55,251][20381] Updated weights for policy 1, policy_version 1680455 (0.0008) [2023-12-27 17:17:55,365][20347] Updated weights for policy 0, policy_version 1682157 (0.0007) [2023-12-27 17:17:55,411][20347] Updated weights for policy 0, policy_version 1682167 (0.0008) [2023-12-27 17:17:55,462][20347] Updated weights for policy 0, policy_version 1682177 (0.0008) [2023-12-27 17:17:55,916][20381] Updated weights for policy 1, policy_version 1680465 (0.0007) [2023-12-27 17:17:55,970][20381] Updated weights for policy 1, policy_version 1680475 (0.0006) [2023-12-27 17:17:56,023][20381] Updated weights for policy 1, policy_version 1680485 (0.0007) [2023-12-27 17:17:56,074][20381] Updated weights for policy 1, policy_version 1680495 (0.0006) [2023-12-27 17:17:56,207][20347] Updated weights for policy 0, policy_version 1682187 (0.0007) [2023-12-27 17:17:56,259][20347] Updated weights for policy 0, policy_version 1682197 (0.0008) [2023-12-27 17:17:56,307][20347] Updated weights for policy 0, policy_version 1682207 (0.0008) [2023-12-27 17:17:56,639][20381] Updated weights for policy 1, policy_version 1680505 (0.0006) [2023-12-27 17:17:56,697][20381] Updated weights for policy 1, policy_version 1680515 (0.0005) [2023-12-27 17:17:56,744][20381] Updated weights for policy 1, policy_version 1680525 (0.0005) [2023-12-27 17:17:57,013][20347] Updated weights for policy 0, policy_version 1682217 (0.0008) [2023-12-27 17:17:57,064][20347] Updated weights for policy 0, policy_version 1682227 (0.0007) [2023-12-27 17:17:57,108][20347] Updated weights for policy 0, policy_version 1682237 (0.0007) [2023-12-27 17:17:57,160][20347] Updated weights for policy 0, policy_version 1682247 (0.0008) [2023-12-27 17:17:57,248][20381] Updated weights for policy 1, policy_version 1680535 (0.0005) [2023-12-27 17:17:57,302][20381] Updated weights for policy 1, policy_version 1680545 (0.0005) [2023-12-27 17:17:57,362][20381] Updated weights for policy 1, policy_version 1680555 (0.0007) [2023-12-27 17:17:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 860995584. Throughput: 0: 10152.2, 1: 10363.3. Samples: 860984320. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:17:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.330')] [2023-12-27 17:17:57,909][20347] Updated weights for policy 0, policy_version 1682257 (0.0008) [2023-12-27 17:17:57,962][20347] Updated weights for policy 0, policy_version 1682267 (0.0008) [2023-12-27 17:17:57,972][20381] Updated weights for policy 1, policy_version 1680565 (0.0007) [2023-12-27 17:17:58,011][20347] Updated weights for policy 0, policy_version 1682277 (0.0006) [2023-12-27 17:17:58,025][20381] Updated weights for policy 1, policy_version 1680575 (0.0011) [2023-12-27 17:17:58,081][20381] Updated weights for policy 1, policy_version 1680585 (0.0011) [2023-12-27 17:17:58,723][20347] Updated weights for policy 0, policy_version 1682287 (0.0007) [2023-12-27 17:17:58,791][20347] Updated weights for policy 0, policy_version 1682297 (0.0012) [2023-12-27 17:17:58,858][20347] Updated weights for policy 0, policy_version 1682307 (0.0011) [2023-12-27 17:17:58,898][20381] Updated weights for policy 1, policy_version 1680595 (0.0010) [2023-12-27 17:17:58,959][20381] Updated weights for policy 1, policy_version 1680605 (0.0008) [2023-12-27 17:17:59,017][20381] Updated weights for policy 1, policy_version 1680615 (0.0008) [2023-12-27 17:17:59,638][20347] Updated weights for policy 0, policy_version 1682317 (0.0006) [2023-12-27 17:17:59,685][20347] Updated weights for policy 0, policy_version 1682327 (0.0008) [2023-12-27 17:17:59,704][20381] Updated weights for policy 1, policy_version 1680625 (0.0008) [2023-12-27 17:17:59,729][20347] Updated weights for policy 0, policy_version 1682337 (0.0006) [2023-12-27 17:17:59,756][20381] Updated weights for policy 1, policy_version 1680635 (0.0005) [2023-12-27 17:17:59,812][20381] Updated weights for policy 1, policy_version 1680645 (0.0007) [2023-12-27 17:17:59,873][20381] Updated weights for policy 1, policy_version 1680655 (0.0009) [2023-12-27 17:18:00,452][20347] Updated weights for policy 0, policy_version 1682347 (0.0005) [2023-12-27 17:18:00,510][20347] Updated weights for policy 0, policy_version 1682357 (0.0007) [2023-12-27 17:18:00,569][20347] Updated weights for policy 0, policy_version 1682367 (0.0009) [2023-12-27 17:18:00,623][20381] Updated weights for policy 1, policy_version 1680665 (0.0006) [2023-12-27 17:18:00,680][20381] Updated weights for policy 1, policy_version 1680675 (0.0005) [2023-12-27 17:18:00,733][20381] Updated weights for policy 1, policy_version 1680685 (0.0005) [2023-12-27 17:18:01,286][20347] Updated weights for policy 0, policy_version 1682377 (0.0008) [2023-12-27 17:18:01,341][20347] Updated weights for policy 0, policy_version 1682387 (0.0008) [2023-12-27 17:18:01,380][20381] Updated weights for policy 1, policy_version 1680695 (0.0007) [2023-12-27 17:18:01,407][20347] Updated weights for policy 0, policy_version 1682397 (0.0008) [2023-12-27 17:18:01,443][20381] Updated weights for policy 1, policy_version 1680705 (0.0007) [2023-12-27 17:18:01,464][20347] Updated weights for policy 0, policy_version 1682407 (0.0006) [2023-12-27 17:18:01,496][20381] Updated weights for policy 1, policy_version 1680715 (0.0006) [2023-12-27 17:18:02,113][20347] Updated weights for policy 0, policy_version 1682417 (0.0007) [2023-12-27 17:18:02,159][20381] Updated weights for policy 1, policy_version 1680725 (0.0009) [2023-12-27 17:18:02,159][20347] Updated weights for policy 0, policy_version 1682427 (0.0006) [2023-12-27 17:18:02,209][20347] Updated weights for policy 0, policy_version 1682437 (0.0005) [2023-12-27 17:18:02,212][20381] Updated weights for policy 1, policy_version 1680735 (0.0010) [2023-12-27 17:18:02,265][20381] Updated weights for policy 1, policy_version 1680745 (0.0009) [2023-12-27 17:18:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 861102080. Throughput: 0: 10124.1, 1: 10371.6. Samples: 861105880. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.010')] [2023-12-27 17:18:02,868][20347] Updated weights for policy 0, policy_version 1682447 (0.0008) [2023-12-27 17:18:02,924][20347] Updated weights for policy 0, policy_version 1682457 (0.0006) [2023-12-27 17:18:02,950][20381] Updated weights for policy 1, policy_version 1680755 (0.0007) [2023-12-27 17:18:02,979][20347] Updated weights for policy 0, policy_version 1682467 (0.0005) [2023-12-27 17:18:02,998][20381] Updated weights for policy 1, policy_version 1680765 (0.0005) [2023-12-27 17:18:03,047][20381] Updated weights for policy 1, policy_version 1680775 (0.0005) [2023-12-27 17:18:03,631][20347] Updated weights for policy 0, policy_version 1682477 (0.0005) [2023-12-27 17:18:03,683][20347] Updated weights for policy 0, policy_version 1682487 (0.0005) [2023-12-27 17:18:03,696][20381] Updated weights for policy 1, policy_version 1680785 (0.0006) [2023-12-27 17:18:03,742][20347] Updated weights for policy 0, policy_version 1682497 (0.0005) [2023-12-27 17:18:03,755][20381] Updated weights for policy 1, policy_version 1680795 (0.0010) [2023-12-27 17:18:03,807][20381] Updated weights for policy 1, policy_version 1680805 (0.0008) [2023-12-27 17:18:03,866][20381] Updated weights for policy 1, policy_version 1680815 (0.0007) [2023-12-27 17:18:04,439][20347] Updated weights for policy 0, policy_version 1682507 (0.0006) [2023-12-27 17:18:04,499][20347] Updated weights for policy 0, policy_version 1682517 (0.0008) [2023-12-27 17:18:04,557][20347] Updated weights for policy 0, policy_version 1682527 (0.0006) [2023-12-27 17:18:04,579][20381] Updated weights for policy 1, policy_version 1680825 (0.0007) [2023-12-27 17:18:04,630][20381] Updated weights for policy 1, policy_version 1680835 (0.0007) [2023-12-27 17:18:04,676][20381] Updated weights for policy 1, policy_version 1680845 (0.0007) [2023-12-27 17:18:05,263][20347] Updated weights for policy 0, policy_version 1682537 (0.0008) [2023-12-27 17:18:05,323][20347] Updated weights for policy 0, policy_version 1682547 (0.0007) [2023-12-27 17:18:05,353][20381] Updated weights for policy 1, policy_version 1680855 (0.0010) [2023-12-27 17:18:05,380][20347] Updated weights for policy 0, policy_version 1682557 (0.0005) [2023-12-27 17:18:05,411][20381] Updated weights for policy 1, policy_version 1680865 (0.0010) [2023-12-27 17:18:05,437][20347] Updated weights for policy 0, policy_version 1682567 (0.0007) [2023-12-27 17:18:05,461][20381] Updated weights for policy 1, policy_version 1680875 (0.0007) [2023-12-27 17:18:06,085][20347] Updated weights for policy 0, policy_version 1682577 (0.0007) [2023-12-27 17:18:06,104][20381] Updated weights for policy 1, policy_version 1680885 (0.0006) [2023-12-27 17:18:06,148][20347] Updated weights for policy 0, policy_version 1682587 (0.0008) [2023-12-27 17:18:06,168][20381] Updated weights for policy 1, policy_version 1680895 (0.0007) [2023-12-27 17:18:06,203][20347] Updated weights for policy 0, policy_version 1682597 (0.0005) [2023-12-27 17:18:06,234][20381] Updated weights for policy 1, policy_version 1680905 (0.0008) [2023-12-27 17:18:06,837][20347] Updated weights for policy 0, policy_version 1682607 (0.0006) [2023-12-27 17:18:06,891][20347] Updated weights for policy 0, policy_version 1682617 (0.0005) [2023-12-27 17:18:06,934][20381] Updated weights for policy 1, policy_version 1680915 (0.0007) [2023-12-27 17:18:06,951][20347] Updated weights for policy 0, policy_version 1682627 (0.0006) [2023-12-27 17:18:06,987][20381] Updated weights for policy 1, policy_version 1680925 (0.0006) [2023-12-27 17:18:07,043][20381] Updated weights for policy 1, policy_version 1680935 (0.0008) [2023-12-27 17:18:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 861200384. Throughput: 0: 10157.2, 1: 10398.6. Samples: 861168312. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.730')] [2023-12-27 17:18:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001680944_430366720.pth... [2023-12-27 17:18:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001682632_430833664.pth... [2023-12-27 17:18:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001681480_430538752.pth [2023-12-27 17:18:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001679728_430055424.pth [2023-12-27 17:18:07,609][20347] Updated weights for policy 0, policy_version 1682637 (0.0005) [2023-12-27 17:18:07,640][20381] Updated weights for policy 1, policy_version 1680945 (0.0008) [2023-12-27 17:18:07,665][20347] Updated weights for policy 0, policy_version 1682647 (0.0005) [2023-12-27 17:18:07,697][20381] Updated weights for policy 1, policy_version 1680955 (0.0006) [2023-12-27 17:18:07,716][20347] Updated weights for policy 0, policy_version 1682657 (0.0005) [2023-12-27 17:18:07,754][20381] Updated weights for policy 1, policy_version 1680965 (0.0006) [2023-12-27 17:18:07,811][20381] Updated weights for policy 1, policy_version 1680975 (0.0005) [2023-12-27 17:18:08,298][20347] Updated weights for policy 0, policy_version 1682667 (0.0007) [2023-12-27 17:18:08,359][20347] Updated weights for policy 0, policy_version 1682677 (0.0010) [2023-12-27 17:18:08,392][20381] Updated weights for policy 1, policy_version 1680985 (0.0007) [2023-12-27 17:18:08,417][20347] Updated weights for policy 0, policy_version 1682687 (0.0006) [2023-12-27 17:18:08,449][20381] Updated weights for policy 1, policy_version 1680995 (0.0007) [2023-12-27 17:18:08,507][20381] Updated weights for policy 1, policy_version 1681005 (0.0007) [2023-12-27 17:18:09,083][20347] Updated weights for policy 0, policy_version 1682697 (0.0007) [2023-12-27 17:18:09,135][20347] Updated weights for policy 0, policy_version 1682707 (0.0007) [2023-12-27 17:18:09,190][20347] Updated weights for policy 0, policy_version 1682717 (0.0007) [2023-12-27 17:18:09,197][20381] Updated weights for policy 1, policy_version 1681015 (0.0006) [2023-12-27 17:18:09,257][20347] Updated weights for policy 0, policy_version 1682727 (0.0008) [2023-12-27 17:18:09,262][20381] Updated weights for policy 1, policy_version 1681025 (0.0008) [2023-12-27 17:18:09,320][20381] Updated weights for policy 1, policy_version 1681035 (0.0006) [2023-12-27 17:18:10,009][20381] Updated weights for policy 1, policy_version 1681045 (0.0007) [2023-12-27 17:18:10,038][20347] Updated weights for policy 0, policy_version 1682737 (0.0008) [2023-12-27 17:18:10,066][20381] Updated weights for policy 1, policy_version 1681055 (0.0006) [2023-12-27 17:18:10,095][20347] Updated weights for policy 0, policy_version 1682747 (0.0006) [2023-12-27 17:18:10,123][20381] Updated weights for policy 1, policy_version 1681065 (0.0006) [2023-12-27 17:18:10,145][20347] Updated weights for policy 0, policy_version 1682757 (0.0006) [2023-12-27 17:18:10,826][20347] Updated weights for policy 0, policy_version 1682767 (0.0008) [2023-12-27 17:18:10,842][20381] Updated weights for policy 1, policy_version 1681075 (0.0008) [2023-12-27 17:18:10,886][20347] Updated weights for policy 0, policy_version 1682777 (0.0007) [2023-12-27 17:18:10,906][20381] Updated weights for policy 1, policy_version 1681085 (0.0005) [2023-12-27 17:18:10,938][20347] Updated weights for policy 0, policy_version 1682787 (0.0008) [2023-12-27 17:18:10,960][20381] Updated weights for policy 1, policy_version 1681095 (0.0005) [2023-12-27 17:18:11,755][20347] Updated weights for policy 0, policy_version 1682797 (0.0006) [2023-12-27 17:18:11,773][20381] Updated weights for policy 1, policy_version 1681105 (0.0007) [2023-12-27 17:18:11,812][20347] Updated weights for policy 0, policy_version 1682807 (0.0007) [2023-12-27 17:18:11,832][20381] Updated weights for policy 1, policy_version 1681115 (0.0008) [2023-12-27 17:18:11,872][20347] Updated weights for policy 0, policy_version 1682817 (0.0007) [2023-12-27 17:18:11,891][20381] Updated weights for policy 1, policy_version 1681125 (0.0008) [2023-12-27 17:18:11,943][20381] Updated weights for policy 1, policy_version 1681135 (0.0008) [2023-12-27 17:18:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 861298688. Throughput: 0: 10158.8, 1: 10414.5. Samples: 861291532. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.790')] [2023-12-27 17:18:12,528][20347] Updated weights for policy 0, policy_version 1682827 (0.0007) [2023-12-27 17:18:12,593][20347] Updated weights for policy 0, policy_version 1682837 (0.0009) [2023-12-27 17:18:12,659][20347] Updated weights for policy 0, policy_version 1682847 (0.0008) [2023-12-27 17:18:12,713][20381] Updated weights for policy 1, policy_version 1681145 (0.0007) [2023-12-27 17:18:12,776][20381] Updated weights for policy 1, policy_version 1681155 (0.0005) [2023-12-27 17:18:12,835][20381] Updated weights for policy 1, policy_version 1681165 (0.0006) [2023-12-27 17:18:13,331][20347] Updated weights for policy 0, policy_version 1682857 (0.0005) [2023-12-27 17:18:13,389][20347] Updated weights for policy 0, policy_version 1682867 (0.0007) [2023-12-27 17:18:13,444][20347] Updated weights for policy 0, policy_version 1682877 (0.0005) [2023-12-27 17:18:13,485][20381] Updated weights for policy 1, policy_version 1681175 (0.0008) [2023-12-27 17:18:13,496][20347] Updated weights for policy 0, policy_version 1682887 (0.0006) [2023-12-27 17:18:13,542][20381] Updated weights for policy 1, policy_version 1681185 (0.0008) [2023-12-27 17:18:13,600][20381] Updated weights for policy 1, policy_version 1681195 (0.0007) [2023-12-27 17:18:14,116][20347] Updated weights for policy 0, policy_version 1682897 (0.0006) [2023-12-27 17:18:14,167][20347] Updated weights for policy 0, policy_version 1682907 (0.0005) [2023-12-27 17:18:14,222][20347] Updated weights for policy 0, policy_version 1682917 (0.0005) [2023-12-27 17:18:14,269][20381] Updated weights for policy 1, policy_version 1681205 (0.0008) [2023-12-27 17:18:14,330][20381] Updated weights for policy 1, policy_version 1681215 (0.0008) [2023-12-27 17:18:14,382][20381] Updated weights for policy 1, policy_version 1681225 (0.0008) [2023-12-27 17:18:14,816][20347] Updated weights for policy 0, policy_version 1682927 (0.0009) [2023-12-27 17:18:14,882][20347] Updated weights for policy 0, policy_version 1682937 (0.0010) [2023-12-27 17:18:14,950][20347] Updated weights for policy 0, policy_version 1682947 (0.0006) [2023-12-27 17:18:15,025][20381] Updated weights for policy 1, policy_version 1681235 (0.0007) [2023-12-27 17:18:15,082][20381] Updated weights for policy 1, policy_version 1681245 (0.0007) [2023-12-27 17:18:15,141][20381] Updated weights for policy 1, policy_version 1681255 (0.0008) [2023-12-27 17:18:15,627][20347] Updated weights for policy 0, policy_version 1682957 (0.0009) [2023-12-27 17:18:15,681][20347] Updated weights for policy 0, policy_version 1682967 (0.0010) [2023-12-27 17:18:15,735][20347] Updated weights for policy 0, policy_version 1682977 (0.0010) [2023-12-27 17:18:15,762][20381] Updated weights for policy 1, policy_version 1681265 (0.0008) [2023-12-27 17:18:15,824][20381] Updated weights for policy 1, policy_version 1681275 (0.0005) [2023-12-27 17:18:15,878][20381] Updated weights for policy 1, policy_version 1681285 (0.0007) [2023-12-27 17:18:15,919][20292] KL-divergence is very high: 177.4530 [2023-12-27 17:18:15,929][20381] Updated weights for policy 1, policy_version 1681295 (0.0007) [2023-12-27 17:18:16,400][20347] Updated weights for policy 0, policy_version 1682987 (0.0009) [2023-12-27 17:18:16,459][20347] Updated weights for policy 0, policy_version 1682997 (0.0005) [2023-12-27 17:18:16,508][20347] Updated weights for policy 0, policy_version 1683007 (0.0007) [2023-12-27 17:18:16,588][20381] Updated weights for policy 1, policy_version 1681305 (0.0007) [2023-12-27 17:18:16,633][20381] Updated weights for policy 1, policy_version 1681315 (0.0008) [2023-12-27 17:18:16,679][20381] Updated weights for policy 1, policy_version 1681325 (0.0007) [2023-12-27 17:18:17,171][20347] Updated weights for policy 0, policy_version 1683017 (0.0007) [2023-12-27 17:18:17,224][20347] Updated weights for policy 0, policy_version 1683027 (0.0008) [2023-12-27 17:18:17,283][20347] Updated weights for policy 0, policy_version 1683037 (0.0007) [2023-12-27 17:18:17,343][20347] Updated weights for policy 0, policy_version 1683047 (0.0008) [2023-12-27 17:18:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 861405184. Throughput: 0: 10108.4, 1: 10365.0. Samples: 861414644. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '932.050')] [2023-12-27 17:18:17,386][20381] Updated weights for policy 1, policy_version 1681335 (0.0007) [2023-12-27 17:18:17,441][20381] Updated weights for policy 1, policy_version 1681345 (0.0007) [2023-12-27 17:18:17,487][20381] Updated weights for policy 1, policy_version 1681355 (0.0007) [2023-12-27 17:18:18,036][20347] Updated weights for policy 0, policy_version 1683057 (0.0008) [2023-12-27 17:18:18,090][20347] Updated weights for policy 0, policy_version 1683067 (0.0007) [2023-12-27 17:18:18,148][20347] Updated weights for policy 0, policy_version 1683077 (0.0006) [2023-12-27 17:18:18,167][20381] Updated weights for policy 1, policy_version 1681365 (0.0008) [2023-12-27 17:18:18,232][20381] Updated weights for policy 1, policy_version 1681375 (0.0007) [2023-12-27 17:18:18,284][20381] Updated weights for policy 1, policy_version 1681385 (0.0008) [2023-12-27 17:18:18,851][20347] Updated weights for policy 0, policy_version 1683087 (0.0006) [2023-12-27 17:18:18,908][20347] Updated weights for policy 0, policy_version 1683097 (0.0009) [2023-12-27 17:18:18,931][20381] Updated weights for policy 1, policy_version 1681395 (0.0008) [2023-12-27 17:18:18,965][20347] Updated weights for policy 0, policy_version 1683107 (0.0007) [2023-12-27 17:18:18,989][20381] Updated weights for policy 1, policy_version 1681405 (0.0005) [2023-12-27 17:18:19,043][20381] Updated weights for policy 1, policy_version 1681415 (0.0005) [2023-12-27 17:18:19,645][20347] Updated weights for policy 0, policy_version 1683117 (0.0008) [2023-12-27 17:18:19,697][20347] Updated weights for policy 0, policy_version 1683127 (0.0008) [2023-12-27 17:18:19,744][20381] Updated weights for policy 1, policy_version 1681425 (0.0006) [2023-12-27 17:18:19,749][20347] Updated weights for policy 0, policy_version 1683137 (0.0007) [2023-12-27 17:18:19,801][20381] Updated weights for policy 1, policy_version 1681435 (0.0011) [2023-12-27 17:18:19,870][20381] Updated weights for policy 1, policy_version 1681445 (0.0010) [2023-12-27 17:18:19,939][20381] Updated weights for policy 1, policy_version 1681455 (0.0009) [2023-12-27 17:18:20,498][20347] Updated weights for policy 0, policy_version 1683147 (0.0007) [2023-12-27 17:18:20,553][20347] Updated weights for policy 0, policy_version 1683157 (0.0007) [2023-12-27 17:18:20,581][20381] Updated weights for policy 1, policy_version 1681465 (0.0008) [2023-12-27 17:18:20,620][20347] Updated weights for policy 0, policy_version 1683167 (0.0006) [2023-12-27 17:18:20,634][20381] Updated weights for policy 1, policy_version 1681475 (0.0007) [2023-12-27 17:18:20,690][20381] Updated weights for policy 1, policy_version 1681485 (0.0009) [2023-12-27 17:18:21,402][20347] Updated weights for policy 0, policy_version 1683177 (0.0007) [2023-12-27 17:18:21,413][20381] Updated weights for policy 1, policy_version 1681495 (0.0006) [2023-12-27 17:18:21,461][20347] Updated weights for policy 0, policy_version 1683187 (0.0008) [2023-12-27 17:18:21,466][20381] Updated weights for policy 1, policy_version 1681505 (0.0008) [2023-12-27 17:18:21,515][20381] Updated weights for policy 1, policy_version 1681515 (0.0007) [2023-12-27 17:18:21,520][20347] Updated weights for policy 0, policy_version 1683197 (0.0008) [2023-12-27 17:18:21,586][20347] Updated weights for policy 0, policy_version 1683207 (0.0007) [2023-12-27 17:18:22,293][20381] Updated weights for policy 1, policy_version 1681525 (0.0006) [2023-12-27 17:18:22,303][20347] Updated weights for policy 0, policy_version 1683217 (0.0008) [2023-12-27 17:18:22,354][20381] Updated weights for policy 1, policy_version 1681535 (0.0007) [2023-12-27 17:18:22,360][20347] Updated weights for policy 0, policy_version 1683227 (0.0008) [2023-12-27 17:18:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 861495296. Throughput: 0: 10076.3, 1: 10354.1. Samples: 861474912. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:22,363][19170] Avg episode reward: [(0, '980.500'), (1, '922.190')] [2023-12-27 17:18:22,416][20381] Updated weights for policy 1, policy_version 1681545 (0.0006) [2023-12-27 17:18:22,419][20347] Updated weights for policy 0, policy_version 1683237 (0.0007) [2023-12-27 17:18:22,437][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001683240_430989312.pth... [2023-12-27 17:18:22,440][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001682056_430686208.pth [2023-12-27 17:18:22,451][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001681552_430522368.pth... [2023-12-27 17:18:22,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001680336_430211072.pth [2023-12-27 17:18:23,021][20381] Updated weights for policy 1, policy_version 1681555 (0.0007) [2023-12-27 17:18:23,076][20381] Updated weights for policy 1, policy_version 1681565 (0.0007) [2023-12-27 17:18:23,139][20381] Updated weights for policy 1, policy_version 1681575 (0.0007) [2023-12-27 17:18:23,152][20347] Updated weights for policy 0, policy_version 1683247 (0.0005) [2023-12-27 17:18:23,207][20347] Updated weights for policy 0, policy_version 1683257 (0.0006) [2023-12-27 17:18:23,266][20347] Updated weights for policy 0, policy_version 1683267 (0.0008) [2023-12-27 17:18:23,718][20381] Updated weights for policy 1, policy_version 1681585 (0.0006) [2023-12-27 17:18:23,778][20381] Updated weights for policy 1, policy_version 1681595 (0.0005) [2023-12-27 17:18:23,831][20381] Updated weights for policy 1, policy_version 1681605 (0.0005) [2023-12-27 17:18:23,891][20381] Updated weights for policy 1, policy_version 1681615 (0.0009) [2023-12-27 17:18:24,001][20347] Updated weights for policy 0, policy_version 1683277 (0.0009) [2023-12-27 17:18:24,057][20347] Updated weights for policy 0, policy_version 1683287 (0.0008) [2023-12-27 17:18:24,114][20347] Updated weights for policy 0, policy_version 1683297 (0.0008) [2023-12-27 17:18:24,454][20381] Updated weights for policy 1, policy_version 1681625 (0.0011) [2023-12-27 17:18:24,503][20381] Updated weights for policy 1, policy_version 1681635 (0.0010) [2023-12-27 17:18:24,552][20381] Updated weights for policy 1, policy_version 1681645 (0.0011) [2023-12-27 17:18:24,866][20347] Updated weights for policy 0, policy_version 1683307 (0.0007) [2023-12-27 17:18:24,929][20347] Updated weights for policy 0, policy_version 1683317 (0.0006) [2023-12-27 17:18:24,988][20347] Updated weights for policy 0, policy_version 1683327 (0.0005) [2023-12-27 17:18:25,171][20381] Updated weights for policy 1, policy_version 1681655 (0.0008) [2023-12-27 17:18:25,225][20381] Updated weights for policy 1, policy_version 1681665 (0.0010) [2023-12-27 17:18:25,284][20381] Updated weights for policy 1, policy_version 1681675 (0.0011) [2023-12-27 17:18:25,602][20347] Updated weights for policy 0, policy_version 1683337 (0.0010) [2023-12-27 17:18:25,656][20347] Updated weights for policy 0, policy_version 1683347 (0.0007) [2023-12-27 17:18:25,713][20347] Updated weights for policy 0, policy_version 1683357 (0.0007) [2023-12-27 17:18:25,763][20347] Updated weights for policy 0, policy_version 1683367 (0.0007) [2023-12-27 17:18:25,963][20381] Updated weights for policy 1, policy_version 1681685 (0.0010) [2023-12-27 17:18:26,021][20381] Updated weights for policy 1, policy_version 1681695 (0.0011) [2023-12-27 17:18:26,071][20381] Updated weights for policy 1, policy_version 1681705 (0.0010) [2023-12-27 17:18:26,433][20347] Updated weights for policy 0, policy_version 1683377 (0.0007) [2023-12-27 17:18:26,477][20347] Updated weights for policy 0, policy_version 1683387 (0.0006) [2023-12-27 17:18:26,533][20347] Updated weights for policy 0, policy_version 1683397 (0.0005) [2023-12-27 17:18:26,739][20381] Updated weights for policy 1, policy_version 1681715 (0.0006) [2023-12-27 17:18:26,789][20381] Updated weights for policy 1, policy_version 1681725 (0.0005) [2023-12-27 17:18:26,847][20381] Updated weights for policy 1, policy_version 1681735 (0.0005) [2023-12-27 17:18:27,133][20347] Updated weights for policy 0, policy_version 1683407 (0.0006) [2023-12-27 17:18:27,188][20347] Updated weights for policy 0, policy_version 1683417 (0.0007) [2023-12-27 17:18:27,237][20347] Updated weights for policy 0, policy_version 1683427 (0.0005) [2023-12-27 17:18:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 861609984. Throughput: 0: 10011.6, 1: 10401.9. Samples: 861596028. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:27,363][19170] Avg episode reward: [(0, '980.500'), (1, '960.930')] [2023-12-27 17:18:27,433][20381] Updated weights for policy 1, policy_version 1681745 (0.0005) [2023-12-27 17:18:27,487][20381] Updated weights for policy 1, policy_version 1681755 (0.0006) [2023-12-27 17:18:27,532][20381] Updated weights for policy 1, policy_version 1681765 (0.0008) [2023-12-27 17:18:27,576][20381] Updated weights for policy 1, policy_version 1681775 (0.0010) [2023-12-27 17:18:27,872][20347] Updated weights for policy 0, policy_version 1683437 (0.0006) [2023-12-27 17:18:27,922][20347] Updated weights for policy 0, policy_version 1683447 (0.0005) [2023-12-27 17:18:27,966][20347] Updated weights for policy 0, policy_version 1683457 (0.0005) [2023-12-27 17:18:28,255][20381] Updated weights for policy 1, policy_version 1681785 (0.0010) [2023-12-27 17:18:28,306][20381] Updated weights for policy 1, policy_version 1681795 (0.0010) [2023-12-27 17:18:28,361][20381] Updated weights for policy 1, policy_version 1681805 (0.0010) [2023-12-27 17:18:28,571][20347] Updated weights for policy 0, policy_version 1683467 (0.0007) [2023-12-27 17:18:28,632][20347] Updated weights for policy 0, policy_version 1683477 (0.0008) [2023-12-27 17:18:28,690][20347] Updated weights for policy 0, policy_version 1683487 (0.0008) [2023-12-27 17:18:29,021][20381] Updated weights for policy 1, policy_version 1681815 (0.0006) [2023-12-27 17:18:29,084][20381] Updated weights for policy 1, policy_version 1681825 (0.0009) [2023-12-27 17:18:29,129][20381] Updated weights for policy 1, policy_version 1681835 (0.0008) [2023-12-27 17:18:29,337][20347] Updated weights for policy 0, policy_version 1683497 (0.0007) [2023-12-27 17:18:29,401][20347] Updated weights for policy 0, policy_version 1683507 (0.0010) [2023-12-27 17:18:29,459][20347] Updated weights for policy 0, policy_version 1683517 (0.0007) [2023-12-27 17:18:29,519][20347] Updated weights for policy 0, policy_version 1683527 (0.0009) [2023-12-27 17:18:29,810][20381] Updated weights for policy 1, policy_version 1681845 (0.0006) [2023-12-27 17:18:29,868][20381] Updated weights for policy 1, policy_version 1681855 (0.0010) [2023-12-27 17:18:29,934][20381] Updated weights for policy 1, policy_version 1681865 (0.0010) [2023-12-27 17:18:30,209][20347] Updated weights for policy 0, policy_version 1683537 (0.0007) [2023-12-27 17:18:30,271][20347] Updated weights for policy 0, policy_version 1683547 (0.0005) [2023-12-27 17:18:30,336][20347] Updated weights for policy 0, policy_version 1683557 (0.0007) [2023-12-27 17:18:30,621][20381] Updated weights for policy 1, policy_version 1681875 (0.0007) [2023-12-27 17:18:30,678][20381] Updated weights for policy 1, policy_version 1681885 (0.0007) [2023-12-27 17:18:30,722][20381] Updated weights for policy 1, policy_version 1681895 (0.0007) [2023-12-27 17:18:30,947][20347] Updated weights for policy 0, policy_version 1683567 (0.0007) [2023-12-27 17:18:30,997][20347] Updated weights for policy 0, policy_version 1683577 (0.0010) [2023-12-27 17:18:31,058][20347] Updated weights for policy 0, policy_version 1683587 (0.0010) [2023-12-27 17:18:31,483][20381] Updated weights for policy 1, policy_version 1681905 (0.0008) [2023-12-27 17:18:31,539][20381] Updated weights for policy 1, policy_version 1681915 (0.0007) [2023-12-27 17:18:31,596][20381] Updated weights for policy 1, policy_version 1681925 (0.0008) [2023-12-27 17:18:31,654][20381] Updated weights for policy 1, policy_version 1681935 (0.0009) [2023-12-27 17:18:31,764][20347] Updated weights for policy 0, policy_version 1683597 (0.0009) [2023-12-27 17:18:31,821][20347] Updated weights for policy 0, policy_version 1683607 (0.0008) [2023-12-27 17:18:31,871][20347] Updated weights for policy 0, policy_version 1683617 (0.0008) [2023-12-27 17:18:32,305][20381] Updated weights for policy 1, policy_version 1681945 (0.0010) [2023-12-27 17:18:32,362][20381] Updated weights for policy 1, policy_version 1681955 (0.0010) [2023-12-27 17:18:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 861708288. Throughput: 0: 10077.1, 1: 10380.4. Samples: 861722020. Policy #0 lag: (min: 31.0, avg: 33.3, max: 63.0) [2023-12-27 17:18:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '980.500')] [2023-12-27 17:18:32,433][20381] Updated weights for policy 1, policy_version 1681965 (0.0009) [2023-12-27 17:18:32,663][20347] Updated weights for policy 0, policy_version 1683627 (0.0008) [2023-12-27 17:18:32,706][20347] Updated weights for policy 0, policy_version 1683637 (0.0008) [2023-12-27 17:18:32,750][20347] Updated weights for policy 0, policy_version 1683647 (0.0007) [2023-12-27 17:18:33,072][20381] Updated weights for policy 1, policy_version 1681975 (0.0010) [2023-12-27 17:18:33,119][20381] Updated weights for policy 1, policy_version 1681985 (0.0010) [2023-12-27 17:18:33,161][20381] Updated weights for policy 1, policy_version 1681995 (0.0010) [2023-12-27 17:18:33,335][20347] Updated weights for policy 0, policy_version 1683657 (0.0007) [2023-12-27 17:18:33,394][20347] Updated weights for policy 0, policy_version 1683667 (0.0008) [2023-12-27 17:18:33,458][20347] Updated weights for policy 0, policy_version 1683677 (0.0008) [2023-12-27 17:18:33,522][20347] Updated weights for policy 0, policy_version 1683687 (0.0008) [2023-12-27 17:18:33,844][20381] Updated weights for policy 1, policy_version 1682005 (0.0008) [2023-12-27 17:18:33,899][20381] Updated weights for policy 1, policy_version 1682015 (0.0005) [2023-12-27 17:18:33,957][20381] Updated weights for policy 1, policy_version 1682025 (0.0005) [2023-12-27 17:18:34,151][20347] Updated weights for policy 0, policy_version 1683697 (0.0008) [2023-12-27 17:18:34,217][20347] Updated weights for policy 0, policy_version 1683707 (0.0010) [2023-12-27 17:18:34,276][20347] Updated weights for policy 0, policy_version 1683717 (0.0010) [2023-12-27 17:18:34,586][20381] Updated weights for policy 1, policy_version 1682035 (0.0005) [2023-12-27 17:18:34,648][20381] Updated weights for policy 1, policy_version 1682045 (0.0009) [2023-12-27 17:18:34,709][20381] Updated weights for policy 1, policy_version 1682055 (0.0010) [2023-12-27 17:18:34,968][20347] Updated weights for policy 0, policy_version 1683727 (0.0011) [2023-12-27 17:18:35,022][20347] Updated weights for policy 0, policy_version 1683737 (0.0009) [2023-12-27 17:18:35,068][20347] Updated weights for policy 0, policy_version 1683747 (0.0005) [2023-12-27 17:18:35,371][20381] Updated weights for policy 1, policy_version 1682065 (0.0010) [2023-12-27 17:18:35,426][20381] Updated weights for policy 1, policy_version 1682075 (0.0010) [2023-12-27 17:18:35,475][20381] Updated weights for policy 1, policy_version 1682085 (0.0010) [2023-12-27 17:18:35,524][20381] Updated weights for policy 1, policy_version 1682095 (0.0010) [2023-12-27 17:18:35,676][20347] Updated weights for policy 0, policy_version 1683757 (0.0006) [2023-12-27 17:18:35,742][20347] Updated weights for policy 0, policy_version 1683767 (0.0005) [2023-12-27 17:18:35,799][20347] Updated weights for policy 0, policy_version 1683777 (0.0007) [2023-12-27 17:18:36,176][20381] Updated weights for policy 1, policy_version 1682105 (0.0008) [2023-12-27 17:18:36,235][20381] Updated weights for policy 1, policy_version 1682115 (0.0011) [2023-12-27 17:18:36,291][20381] Updated weights for policy 1, policy_version 1682125 (0.0011) [2023-12-27 17:18:36,421][20347] Updated weights for policy 0, policy_version 1683787 (0.0009) [2023-12-27 17:18:36,484][20347] Updated weights for policy 0, policy_version 1683797 (0.0006) [2023-12-27 17:18:36,546][20347] Updated weights for policy 0, policy_version 1683807 (0.0007) [2023-12-27 17:18:36,988][20381] Updated weights for policy 1, policy_version 1682135 (0.0010) [2023-12-27 17:18:37,048][20381] Updated weights for policy 1, policy_version 1682145 (0.0010) [2023-12-27 17:18:37,107][20381] Updated weights for policy 1, policy_version 1682155 (0.0010) [2023-12-27 17:18:37,140][20347] Updated weights for policy 0, policy_version 1683817 (0.0006) [2023-12-27 17:18:37,201][20347] Updated weights for policy 0, policy_version 1683827 (0.0008) [2023-12-27 17:18:37,263][20347] Updated weights for policy 0, policy_version 1683837 (0.0005) [2023-12-27 17:18:37,324][20347] Updated weights for policy 0, policy_version 1683847 (0.0005) [2023-12-27 17:18:37,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 861822976. Throughput: 0: 10138.4, 1: 10390.5. Samples: 861784940. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:18:37,364][19170] Avg episode reward: [(0, '990.250'), (1, '990.260')] [2023-12-27 17:18:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001682160_430678016.pth... [2023-12-27 17:18:37,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001683848_431144960.pth... [2023-12-27 17:18:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001680944_430366720.pth [2023-12-27 17:18:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001682632_430833664.pth [2023-12-27 17:18:37,791][20381] Updated weights for policy 1, policy_version 1682165 (0.0011) [2023-12-27 17:18:37,844][20381] Updated weights for policy 1, policy_version 1682175 (0.0011) [2023-12-27 17:18:37,892][20347] Updated weights for policy 0, policy_version 1683857 (0.0005) [2023-12-27 17:18:37,900][20381] Updated weights for policy 1, policy_version 1682185 (0.0010) [2023-12-27 17:18:37,944][20347] Updated weights for policy 0, policy_version 1683867 (0.0005) [2023-12-27 17:18:37,991][20347] Updated weights for policy 0, policy_version 1683877 (0.0005) [2023-12-27 17:18:38,584][20347] Updated weights for policy 0, policy_version 1683887 (0.0007) [2023-12-27 17:18:38,627][20381] Updated weights for policy 1, policy_version 1682195 (0.0009) [2023-12-27 17:18:38,636][20347] Updated weights for policy 0, policy_version 1683897 (0.0008) [2023-12-27 17:18:38,679][20381] Updated weights for policy 1, policy_version 1682205 (0.0007) [2023-12-27 17:18:38,685][20347] Updated weights for policy 0, policy_version 1683907 (0.0008) [2023-12-27 17:18:38,728][20381] Updated weights for policy 1, policy_version 1682215 (0.0007) [2023-12-27 17:18:39,374][20347] Updated weights for policy 0, policy_version 1683917 (0.0008) [2023-12-27 17:18:39,438][20347] Updated weights for policy 0, policy_version 1683927 (0.0008) [2023-12-27 17:18:39,488][20381] Updated weights for policy 1, policy_version 1682225 (0.0007) [2023-12-27 17:18:39,497][20347] Updated weights for policy 0, policy_version 1683937 (0.0009) [2023-12-27 17:18:39,551][20381] Updated weights for policy 1, policy_version 1682235 (0.0008) [2023-12-27 17:18:39,618][20381] Updated weights for policy 1, policy_version 1682245 (0.0007) [2023-12-27 17:18:39,682][20381] Updated weights for policy 1, policy_version 1682255 (0.0005) [2023-12-27 17:18:40,256][20347] Updated weights for policy 0, policy_version 1683947 (0.0008) [2023-12-27 17:18:40,313][20347] Updated weights for policy 0, policy_version 1683957 (0.0008) [2023-12-27 17:18:40,376][20347] Updated weights for policy 0, policy_version 1683967 (0.0007) [2023-12-27 17:18:40,390][20381] Updated weights for policy 1, policy_version 1682265 (0.0006) [2023-12-27 17:18:40,448][20381] Updated weights for policy 1, policy_version 1682275 (0.0007) [2023-12-27 17:18:40,511][20381] Updated weights for policy 1, policy_version 1682285 (0.0007) [2023-12-27 17:18:41,045][20347] Updated weights for policy 0, policy_version 1683977 (0.0007) [2023-12-27 17:18:41,103][20347] Updated weights for policy 0, policy_version 1683987 (0.0008) [2023-12-27 17:18:41,173][20347] Updated weights for policy 0, policy_version 1683997 (0.0008) [2023-12-27 17:18:41,229][20347] Updated weights for policy 0, policy_version 1684007 (0.0009) [2023-12-27 17:18:41,241][20381] Updated weights for policy 1, policy_version 1682295 (0.0009) [2023-12-27 17:18:41,302][20381] Updated weights for policy 1, policy_version 1682305 (0.0008) [2023-12-27 17:18:41,373][20381] Updated weights for policy 1, policy_version 1682315 (0.0008) [2023-12-27 17:18:42,012][20347] Updated weights for policy 0, policy_version 1684017 (0.0008) [2023-12-27 17:18:42,072][20347] Updated weights for policy 0, policy_version 1684027 (0.0008) [2023-12-27 17:18:42,100][20381] Updated weights for policy 1, policy_version 1682325 (0.0008) [2023-12-27 17:18:42,129][20347] Updated weights for policy 0, policy_version 1684037 (0.0006) [2023-12-27 17:18:42,154][20381] Updated weights for policy 1, policy_version 1682335 (0.0006) [2023-12-27 17:18:42,212][20381] Updated weights for policy 1, policy_version 1682345 (0.0008) [2023-12-27 17:18:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 861921280. Throughput: 0: 10226.1, 1: 10324.7. Samples: 861909108. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:18:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '980.520')] [2023-12-27 17:18:42,853][20347] Updated weights for policy 0, policy_version 1684047 (0.0007) [2023-12-27 17:18:42,904][20381] Updated weights for policy 1, policy_version 1682355 (0.0008) [2023-12-27 17:18:42,906][20347] Updated weights for policy 0, policy_version 1684057 (0.0007) [2023-12-27 17:18:42,956][20347] Updated weights for policy 0, policy_version 1684067 (0.0005) [2023-12-27 17:18:42,961][20381] Updated weights for policy 1, policy_version 1682365 (0.0007) [2023-12-27 17:18:43,016][20381] Updated weights for policy 1, policy_version 1682375 (0.0008) [2023-12-27 17:18:43,673][20347] Updated weights for policy 0, policy_version 1684077 (0.0008) [2023-12-27 17:18:43,703][20381] Updated weights for policy 1, policy_version 1682385 (0.0007) [2023-12-27 17:18:43,719][20347] Updated weights for policy 0, policy_version 1684087 (0.0007) [2023-12-27 17:18:43,757][20381] Updated weights for policy 1, policy_version 1682395 (0.0006) [2023-12-27 17:18:43,768][20347] Updated weights for policy 0, policy_version 1684097 (0.0006) [2023-12-27 17:18:43,812][20381] Updated weights for policy 1, policy_version 1682405 (0.0005) [2023-12-27 17:18:43,869][20381] Updated weights for policy 1, policy_version 1682415 (0.0008) [2023-12-27 17:18:44,428][20347] Updated weights for policy 0, policy_version 1684107 (0.0006) [2023-12-27 17:18:44,493][20347] Updated weights for policy 0, policy_version 1684117 (0.0006) [2023-12-27 17:18:44,555][20347] Updated weights for policy 0, policy_version 1684127 (0.0007) [2023-12-27 17:18:44,568][20381] Updated weights for policy 1, policy_version 1682425 (0.0006) [2023-12-27 17:18:44,632][20381] Updated weights for policy 1, policy_version 1682435 (0.0006) [2023-12-27 17:18:44,697][20381] Updated weights for policy 1, policy_version 1682445 (0.0008) [2023-12-27 17:18:45,220][20347] Updated weights for policy 0, policy_version 1684137 (0.0007) [2023-12-27 17:18:45,276][20347] Updated weights for policy 0, policy_version 1684147 (0.0008) [2023-12-27 17:18:45,336][20347] Updated weights for policy 0, policy_version 1684157 (0.0007) [2023-12-27 17:18:45,385][20347] Updated weights for policy 0, policy_version 1684167 (0.0006) [2023-12-27 17:18:45,430][20381] Updated weights for policy 1, policy_version 1682455 (0.0008) [2023-12-27 17:18:45,481][20381] Updated weights for policy 1, policy_version 1682465 (0.0009) [2023-12-27 17:18:45,527][20381] Updated weights for policy 1, policy_version 1682475 (0.0009) [2023-12-27 17:18:45,976][20347] Updated weights for policy 0, policy_version 1684177 (0.0008) [2023-12-27 17:18:46,035][20347] Updated weights for policy 0, policy_version 1684187 (0.0008) [2023-12-27 17:18:46,092][20347] Updated weights for policy 0, policy_version 1684197 (0.0007) [2023-12-27 17:18:46,246][20381] Updated weights for policy 1, policy_version 1682485 (0.0008) [2023-12-27 17:18:46,302][20381] Updated weights for policy 1, policy_version 1682495 (0.0008) [2023-12-27 17:18:46,362][20381] Updated weights for policy 1, policy_version 1682505 (0.0008) [2023-12-27 17:18:46,640][20347] Updated weights for policy 0, policy_version 1684207 (0.0008) [2023-12-27 17:18:46,694][20347] Updated weights for policy 0, policy_version 1684217 (0.0008) [2023-12-27 17:18:46,752][20347] Updated weights for policy 0, policy_version 1684227 (0.0008) [2023-12-27 17:18:47,100][20381] Updated weights for policy 1, policy_version 1682515 (0.0009) [2023-12-27 17:18:47,165][20381] Updated weights for policy 1, policy_version 1682525 (0.0008) [2023-12-27 17:18:47,223][20381] Updated weights for policy 1, policy_version 1682535 (0.0008) [2023-12-27 17:18:47,358][20347] Updated weights for policy 0, policy_version 1684237 (0.0006) [2023-12-27 17:18:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 862019584. Throughput: 0: 10291.6, 1: 10228.9. Samples: 862029304. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:18:47,363][19170] Avg episode reward: [(0, '990.220'), (1, '961.130')] [2023-12-27 17:18:47,416][20347] Updated weights for policy 0, policy_version 1684247 (0.0006) [2023-12-27 17:18:47,472][20347] Updated weights for policy 0, policy_version 1684257 (0.0005) [2023-12-27 17:18:47,901][20381] Updated weights for policy 1, policy_version 1682545 (0.0008) [2023-12-27 17:18:47,958][20381] Updated weights for policy 1, policy_version 1682555 (0.0005) [2023-12-27 17:18:48,008][20381] Updated weights for policy 1, policy_version 1682565 (0.0005) [2023-12-27 17:18:48,070][20381] Updated weights for policy 1, policy_version 1682575 (0.0005) [2023-12-27 17:18:48,091][20347] Updated weights for policy 0, policy_version 1684267 (0.0005) [2023-12-27 17:18:48,146][20347] Updated weights for policy 0, policy_version 1684277 (0.0007) [2023-12-27 17:18:48,198][20347] Updated weights for policy 0, policy_version 1684287 (0.0007) [2023-12-27 17:18:48,699][20381] Updated weights for policy 1, policy_version 1682585 (0.0010) [2023-12-27 17:18:48,758][20381] Updated weights for policy 1, policy_version 1682595 (0.0010) [2023-12-27 17:18:48,814][20381] Updated weights for policy 1, policy_version 1682605 (0.0010) [2023-12-27 17:18:48,920][20347] Updated weights for policy 0, policy_version 1684297 (0.0008) [2023-12-27 17:18:48,970][20347] Updated weights for policy 0, policy_version 1684307 (0.0007) [2023-12-27 17:18:49,024][20347] Updated weights for policy 0, policy_version 1684317 (0.0007) [2023-12-27 17:18:49,080][20347] Updated weights for policy 0, policy_version 1684327 (0.0007) [2023-12-27 17:18:49,463][20381] Updated weights for policy 1, policy_version 1682615 (0.0008) [2023-12-27 17:18:49,516][20381] Updated weights for policy 1, policy_version 1682625 (0.0008) [2023-12-27 17:18:49,573][20381] Updated weights for policy 1, policy_version 1682635 (0.0008) [2023-12-27 17:18:49,783][20347] Updated weights for policy 0, policy_version 1684337 (0.0007) [2023-12-27 17:18:49,848][20347] Updated weights for policy 0, policy_version 1684347 (0.0008) [2023-12-27 17:18:49,910][20347] Updated weights for policy 0, policy_version 1684357 (0.0007) [2023-12-27 17:18:50,269][20381] Updated weights for policy 1, policy_version 1682645 (0.0008) [2023-12-27 17:18:50,332][20381] Updated weights for policy 1, policy_version 1682655 (0.0005) [2023-12-27 17:18:50,389][20381] Updated weights for policy 1, policy_version 1682665 (0.0006) [2023-12-27 17:18:50,615][20347] Updated weights for policy 0, policy_version 1684367 (0.0010) [2023-12-27 17:18:50,668][20347] Updated weights for policy 0, policy_version 1684377 (0.0010) [2023-12-27 17:18:50,721][20347] Updated weights for policy 0, policy_version 1684387 (0.0007) [2023-12-27 17:18:51,033][20381] Updated weights for policy 1, policy_version 1682675 (0.0006) [2023-12-27 17:18:51,096][20381] Updated weights for policy 1, policy_version 1682685 (0.0009) [2023-12-27 17:18:51,151][20381] Updated weights for policy 1, policy_version 1682695 (0.0008) [2023-12-27 17:18:51,425][20347] Updated weights for policy 0, policy_version 1684397 (0.0007) [2023-12-27 17:18:51,476][20347] Updated weights for policy 0, policy_version 1684407 (0.0010) [2023-12-27 17:18:51,524][20347] Updated weights for policy 0, policy_version 1684417 (0.0010) [2023-12-27 17:18:51,952][20381] Updated weights for policy 1, policy_version 1682705 (0.0007) [2023-12-27 17:18:52,005][20381] Updated weights for policy 1, policy_version 1682715 (0.0008) [2023-12-27 17:18:52,064][20381] Updated weights for policy 1, policy_version 1682725 (0.0006) [2023-12-27 17:18:52,123][20381] Updated weights for policy 1, policy_version 1682735 (0.0005) [2023-12-27 17:18:52,237][20347] Updated weights for policy 0, policy_version 1684427 (0.0009) [2023-12-27 17:18:52,292][20347] Updated weights for policy 0, policy_version 1684437 (0.0008) [2023-12-27 17:18:52,350][20347] Updated weights for policy 0, policy_version 1684447 (0.0007) [2023-12-27 17:18:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 862117888. Throughput: 0: 10280.9, 1: 10234.0. Samples: 862091484. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:18:52,363][19170] Avg episode reward: [(0, '990.220'), (1, '970.860')] [2023-12-27 17:18:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001682736_430825472.pth... [2023-12-27 17:18:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001681552_430522368.pth [2023-12-27 17:18:52,403][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001684456_431300608.pth... [2023-12-27 17:18:52,406][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001683240_430989312.pth [2023-12-27 17:18:52,789][20381] Updated weights for policy 1, policy_version 1682745 (0.0007) [2023-12-27 17:18:52,846][20381] Updated weights for policy 1, policy_version 1682755 (0.0007) [2023-12-27 17:18:52,892][20381] Updated weights for policy 1, policy_version 1682765 (0.0007) [2023-12-27 17:18:53,068][20347] Updated weights for policy 0, policy_version 1684457 (0.0008) [2023-12-27 17:18:53,127][20347] Updated weights for policy 0, policy_version 1684467 (0.0007) [2023-12-27 17:18:53,178][20347] Updated weights for policy 0, policy_version 1684477 (0.0007) [2023-12-27 17:18:53,231][20347] Updated weights for policy 0, policy_version 1684487 (0.0008) [2023-12-27 17:18:53,499][20381] Updated weights for policy 1, policy_version 1682775 (0.0010) [2023-12-27 17:18:53,552][20381] Updated weights for policy 1, policy_version 1682785 (0.0011) [2023-12-27 17:18:53,601][20381] Updated weights for policy 1, policy_version 1682795 (0.0011) [2023-12-27 17:18:53,907][20347] Updated weights for policy 0, policy_version 1684497 (0.0010) [2023-12-27 17:18:53,956][20347] Updated weights for policy 0, policy_version 1684507 (0.0010) [2023-12-27 17:18:54,011][20347] Updated weights for policy 0, policy_version 1684517 (0.0010) [2023-12-27 17:18:54,307][20381] Updated weights for policy 1, policy_version 1682805 (0.0010) [2023-12-27 17:18:54,364][20381] Updated weights for policy 1, policy_version 1682815 (0.0007) [2023-12-27 17:18:54,423][20381] Updated weights for policy 1, policy_version 1682825 (0.0008) [2023-12-27 17:18:54,675][20347] Updated weights for policy 0, policy_version 1684527 (0.0008) [2023-12-27 17:18:54,724][20347] Updated weights for policy 0, policy_version 1684537 (0.0010) [2023-12-27 17:18:54,782][20347] Updated weights for policy 0, policy_version 1684547 (0.0010) [2023-12-27 17:18:55,022][20381] Updated weights for policy 1, policy_version 1682835 (0.0007) [2023-12-27 17:18:55,099][20381] Updated weights for policy 1, policy_version 1682845 (0.0006) [2023-12-27 17:18:55,157][20381] Updated weights for policy 1, policy_version 1682855 (0.0007) [2023-12-27 17:18:55,439][20347] Updated weights for policy 0, policy_version 1684557 (0.0010) [2023-12-27 17:18:55,490][20347] Updated weights for policy 0, policy_version 1684567 (0.0005) [2023-12-27 17:18:55,518][20291] KL-divergence is very high: 111.8639 [2023-12-27 17:18:55,537][20347] Updated weights for policy 0, policy_version 1684577 (0.0005) [2023-12-27 17:18:55,791][20381] Updated weights for policy 1, policy_version 1682865 (0.0007) [2023-12-27 17:18:55,852][20381] Updated weights for policy 1, policy_version 1682875 (0.0006) [2023-12-27 17:18:55,915][20381] Updated weights for policy 1, policy_version 1682885 (0.0007) [2023-12-27 17:18:55,975][20381] Updated weights for policy 1, policy_version 1682895 (0.0008) [2023-12-27 17:18:56,154][20347] Updated weights for policy 0, policy_version 1684587 (0.0005) [2023-12-27 17:18:56,215][20347] Updated weights for policy 0, policy_version 1684597 (0.0006) [2023-12-27 17:18:56,278][20347] Updated weights for policy 0, policy_version 1684607 (0.0010) [2023-12-27 17:18:56,658][20381] Updated weights for policy 1, policy_version 1682905 (0.0008) [2023-12-27 17:18:56,701][20381] Updated weights for policy 1, policy_version 1682915 (0.0007) [2023-12-27 17:18:56,757][20381] Updated weights for policy 1, policy_version 1682925 (0.0008) [2023-12-27 17:18:56,886][20347] Updated weights for policy 0, policy_version 1684617 (0.0010) [2023-12-27 17:18:56,942][20347] Updated weights for policy 0, policy_version 1684627 (0.0005) [2023-12-27 17:18:57,000][20347] Updated weights for policy 0, policy_version 1684637 (0.0005) [2023-12-27 17:18:57,066][20347] Updated weights for policy 0, policy_version 1684647 (0.0005) [2023-12-27 17:18:57,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 862224384. Throughput: 0: 10285.3, 1: 10244.0. Samples: 862215352. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:18:57,364][19170] Avg episode reward: [(0, '980.500'), (1, '941.480')] [2023-12-27 17:18:57,485][20381] Updated weights for policy 1, policy_version 1682935 (0.0007) [2023-12-27 17:18:57,541][20381] Updated weights for policy 1, policy_version 1682945 (0.0007) [2023-12-27 17:18:57,596][20381] Updated weights for policy 1, policy_version 1682955 (0.0007) [2023-12-27 17:18:57,669][20347] Updated weights for policy 0, policy_version 1684657 (0.0009) [2023-12-27 17:18:57,717][20347] Updated weights for policy 0, policy_version 1684667 (0.0010) [2023-12-27 17:18:57,768][20347] Updated weights for policy 0, policy_version 1684677 (0.0010) [2023-12-27 17:18:58,274][20381] Updated weights for policy 1, policy_version 1682965 (0.0008) [2023-12-27 17:18:58,323][20381] Updated weights for policy 1, policy_version 1682975 (0.0010) [2023-12-27 17:18:58,389][20381] Updated weights for policy 1, policy_version 1682985 (0.0009) [2023-12-27 17:18:58,450][20347] Updated weights for policy 0, policy_version 1684687 (0.0008) [2023-12-27 17:18:58,509][20347] Updated weights for policy 0, policy_version 1684697 (0.0010) [2023-12-27 17:18:58,572][20347] Updated weights for policy 0, policy_version 1684707 (0.0008) [2023-12-27 17:18:59,110][20381] Updated weights for policy 1, policy_version 1682995 (0.0008) [2023-12-27 17:18:59,166][20381] Updated weights for policy 1, policy_version 1683005 (0.0011) [2023-12-27 17:18:59,231][20381] Updated weights for policy 1, policy_version 1683015 (0.0010) [2023-12-27 17:18:59,413][20347] Updated weights for policy 0, policy_version 1684717 (0.0008) [2023-12-27 17:18:59,472][20347] Updated weights for policy 0, policy_version 1684727 (0.0007) [2023-12-27 17:18:59,534][20347] Updated weights for policy 0, policy_version 1684737 (0.0007) [2023-12-27 17:18:59,980][20381] Updated weights for policy 1, policy_version 1683025 (0.0008) [2023-12-27 17:19:00,030][20381] Updated weights for policy 1, policy_version 1683035 (0.0010) [2023-12-27 17:19:00,089][20381] Updated weights for policy 1, policy_version 1683045 (0.0010) [2023-12-27 17:19:00,137][20381] Updated weights for policy 1, policy_version 1683055 (0.0008) [2023-12-27 17:19:00,235][20347] Updated weights for policy 0, policy_version 1684747 (0.0007) [2023-12-27 17:19:00,288][20347] Updated weights for policy 0, policy_version 1684757 (0.0005) [2023-12-27 17:19:00,343][20347] Updated weights for policy 0, policy_version 1684767 (0.0007) [2023-12-27 17:19:00,801][20381] Updated weights for policy 1, policy_version 1683065 (0.0010) [2023-12-27 17:19:00,859][20381] Updated weights for policy 1, policy_version 1683075 (0.0007) [2023-12-27 17:19:00,917][20381] Updated weights for policy 1, policy_version 1683085 (0.0005) [2023-12-27 17:19:01,025][20347] Updated weights for policy 0, policy_version 1684777 (0.0008) [2023-12-27 17:19:01,092][20347] Updated weights for policy 0, policy_version 1684787 (0.0007) [2023-12-27 17:19:01,153][20347] Updated weights for policy 0, policy_version 1684797 (0.0008) [2023-12-27 17:19:01,210][20347] Updated weights for policy 0, policy_version 1684807 (0.0005) [2023-12-27 17:19:01,536][20381] Updated weights for policy 1, policy_version 1683095 (0.0009) [2023-12-27 17:19:01,595][20381] Updated weights for policy 1, policy_version 1683105 (0.0010) [2023-12-27 17:19:01,655][20381] Updated weights for policy 1, policy_version 1683115 (0.0010) [2023-12-27 17:19:01,915][20347] Updated weights for policy 0, policy_version 1684817 (0.0007) [2023-12-27 17:19:01,971][20347] Updated weights for policy 0, policy_version 1684827 (0.0007) [2023-12-27 17:19:02,030][20347] Updated weights for policy 0, policy_version 1684837 (0.0007) [2023-12-27 17:19:02,349][20381] Updated weights for policy 1, policy_version 1683125 (0.0011) [2023-12-27 17:19:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 862322688. Throughput: 0: 10250.3, 1: 10219.6. Samples: 862335792. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:02,363][19170] Avg episode reward: [(0, '980.500'), (1, '873.320')] [2023-12-27 17:19:02,377][20292] KL-divergence is very high: 168.3021 [2023-12-27 17:19:02,390][20292] KL-divergence is very high: 197.5696 [2023-12-27 17:19:02,402][20292] KL-divergence is very high: 226.2287 [2023-12-27 17:19:02,413][20381] Updated weights for policy 1, policy_version 1683135 (0.0011) [2023-12-27 17:19:02,422][20292] KL-divergence is very high: 274.3849 [2023-12-27 17:19:02,432][20292] KL-divergence is very high: 213.1744 [2023-12-27 17:19:02,442][20292] KL-divergence is very high: 170.3950 [2023-12-27 17:19:02,461][20292] KL-divergence is very high: 123.4937 [2023-12-27 17:19:02,462][20381] Updated weights for policy 1, policy_version 1683145 (0.0008) [2023-12-27 17:19:02,623][20347] Updated weights for policy 0, policy_version 1684847 (0.0006) [2023-12-27 17:19:02,691][20347] Updated weights for policy 0, policy_version 1684857 (0.0005) [2023-12-27 17:19:02,761][20347] Updated weights for policy 0, policy_version 1684867 (0.0006) [2023-12-27 17:19:03,146][20381] Updated weights for policy 1, policy_version 1683155 (0.0007) [2023-12-27 17:19:03,201][20381] Updated weights for policy 1, policy_version 1683165 (0.0005) [2023-12-27 17:19:03,265][20381] Updated weights for policy 1, policy_version 1683175 (0.0005) [2023-12-27 17:19:03,432][20347] Updated weights for policy 0, policy_version 1684877 (0.0008) [2023-12-27 17:19:03,485][20347] Updated weights for policy 0, policy_version 1684887 (0.0008) [2023-12-27 17:19:03,535][20347] Updated weights for policy 0, policy_version 1684897 (0.0008) [2023-12-27 17:19:03,824][20381] Updated weights for policy 1, policy_version 1683185 (0.0008) [2023-12-27 17:19:03,885][20381] Updated weights for policy 1, policy_version 1683195 (0.0008) [2023-12-27 17:19:03,940][20381] Updated weights for policy 1, policy_version 1683205 (0.0008) [2023-12-27 17:19:03,996][20381] Updated weights for policy 1, policy_version 1683215 (0.0008) [2023-12-27 17:19:04,257][20347] Updated weights for policy 0, policy_version 1684907 (0.0007) [2023-12-27 17:19:04,316][20347] Updated weights for policy 0, policy_version 1684917 (0.0007) [2023-12-27 17:19:04,369][20347] Updated weights for policy 0, policy_version 1684927 (0.0005) [2023-12-27 17:19:04,743][20381] Updated weights for policy 1, policy_version 1683225 (0.0008) [2023-12-27 17:19:04,792][20381] Updated weights for policy 1, policy_version 1683235 (0.0008) [2023-12-27 17:19:04,845][20381] Updated weights for policy 1, policy_version 1683245 (0.0008) [2023-12-27 17:19:05,013][20347] Updated weights for policy 0, policy_version 1684937 (0.0005) [2023-12-27 17:19:05,076][20347] Updated weights for policy 0, policy_version 1684947 (0.0008) [2023-12-27 17:19:05,133][20347] Updated weights for policy 0, policy_version 1684957 (0.0008) [2023-12-27 17:19:05,192][20347] Updated weights for policy 0, policy_version 1684967 (0.0008) [2023-12-27 17:19:05,569][20381] Updated weights for policy 1, policy_version 1683255 (0.0006) [2023-12-27 17:19:05,614][20381] Updated weights for policy 1, policy_version 1683265 (0.0008) [2023-12-27 17:19:05,660][20381] Updated weights for policy 1, policy_version 1683275 (0.0006) [2023-12-27 17:19:05,923][20347] Updated weights for policy 0, policy_version 1684977 (0.0008) [2023-12-27 17:19:05,968][20347] Updated weights for policy 0, policy_version 1684987 (0.0007) [2023-12-27 17:19:06,014][20347] Updated weights for policy 0, policy_version 1684997 (0.0008) [2023-12-27 17:19:06,362][20381] Updated weights for policy 1, policy_version 1683285 (0.0007) [2023-12-27 17:19:06,416][20381] Updated weights for policy 1, policy_version 1683295 (0.0008) [2023-12-27 17:19:06,466][20381] Updated weights for policy 1, policy_version 1683305 (0.0007) [2023-12-27 17:19:06,731][20347] Updated weights for policy 0, policy_version 1685007 (0.0007) [2023-12-27 17:19:06,793][20347] Updated weights for policy 0, policy_version 1685017 (0.0005) [2023-12-27 17:19:06,859][20347] Updated weights for policy 0, policy_version 1685027 (0.0008) [2023-12-27 17:19:07,234][20381] Updated weights for policy 1, policy_version 1683315 (0.0008) [2023-12-27 17:19:07,292][20381] Updated weights for policy 1, policy_version 1683325 (0.0008) [2023-12-27 17:19:07,348][20381] Updated weights for policy 1, policy_version 1683335 (0.0009) [2023-12-27 17:19:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 862420992. Throughput: 0: 10293.5, 1: 10210.3. Samples: 862397584. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:07,363][19170] Avg episode reward: [(0, '990.290'), (1, '902.600')] [2023-12-27 17:19:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001685032_431448064.pth... [2023-12-27 17:19:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001683848_431144960.pth [2023-12-27 17:19:07,404][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001683344_430981120.pth... [2023-12-27 17:19:07,408][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001682160_430678016.pth [2023-12-27 17:19:07,414][20347] Updated weights for policy 0, policy_version 1685037 (0.0008) [2023-12-27 17:19:07,462][20347] Updated weights for policy 0, policy_version 1685047 (0.0007) [2023-12-27 17:19:07,517][20347] Updated weights for policy 0, policy_version 1685057 (0.0007) [2023-12-27 17:19:07,926][20381] Updated weights for policy 1, policy_version 1683345 (0.0008) [2023-12-27 17:19:07,973][20381] Updated weights for policy 1, policy_version 1683355 (0.0008) [2023-12-27 17:19:08,025][20381] Updated weights for policy 1, policy_version 1683365 (0.0007) [2023-12-27 17:19:08,080][20381] Updated weights for policy 1, policy_version 1683375 (0.0007) [2023-12-27 17:19:08,266][20347] Updated weights for policy 0, policy_version 1685067 (0.0007) [2023-12-27 17:19:08,323][20347] Updated weights for policy 0, policy_version 1685077 (0.0007) [2023-12-27 17:19:08,390][20347] Updated weights for policy 0, policy_version 1685087 (0.0007) [2023-12-27 17:19:08,755][20381] Updated weights for policy 1, policy_version 1683385 (0.0008) [2023-12-27 17:19:08,807][20381] Updated weights for policy 1, policy_version 1683395 (0.0005) [2023-12-27 17:19:08,866][20381] Updated weights for policy 1, policy_version 1683405 (0.0005) [2023-12-27 17:19:09,016][20347] Updated weights for policy 0, policy_version 1685097 (0.0008) [2023-12-27 17:19:09,072][20347] Updated weights for policy 0, policy_version 1685107 (0.0008) [2023-12-27 17:19:09,135][20347] Updated weights for policy 0, policy_version 1685117 (0.0008) [2023-12-27 17:19:09,195][20347] Updated weights for policy 0, policy_version 1685127 (0.0005) [2023-12-27 17:19:09,522][20381] Updated weights for policy 1, policy_version 1683415 (0.0006) [2023-12-27 17:19:09,581][20381] Updated weights for policy 1, policy_version 1683425 (0.0006) [2023-12-27 17:19:09,635][20381] Updated weights for policy 1, policy_version 1683435 (0.0006) [2023-12-27 17:19:09,934][20347] Updated weights for policy 0, policy_version 1685137 (0.0010) [2023-12-27 17:19:09,988][20347] Updated weights for policy 0, policy_version 1685147 (0.0011) [2023-12-27 17:19:10,042][20347] Updated weights for policy 0, policy_version 1685157 (0.0011) [2023-12-27 17:19:10,323][20381] Updated weights for policy 1, policy_version 1683445 (0.0006) [2023-12-27 17:19:10,383][20381] Updated weights for policy 1, policy_version 1683455 (0.0006) [2023-12-27 17:19:10,442][20381] Updated weights for policy 1, policy_version 1683465 (0.0006) [2023-12-27 17:19:10,694][20347] Updated weights for policy 0, policy_version 1685167 (0.0006) [2023-12-27 17:19:10,756][20347] Updated weights for policy 0, policy_version 1685177 (0.0009) [2023-12-27 17:19:10,810][20347] Updated weights for policy 0, policy_version 1685187 (0.0010) [2023-12-27 17:19:11,068][20381] Updated weights for policy 1, policy_version 1683475 (0.0008) [2023-12-27 17:19:11,127][20381] Updated weights for policy 1, policy_version 1683485 (0.0008) [2023-12-27 17:19:11,189][20381] Updated weights for policy 1, policy_version 1683495 (0.0008) [2023-12-27 17:19:11,513][20347] Updated weights for policy 0, policy_version 1685197 (0.0008) [2023-12-27 17:19:11,566][20347] Updated weights for policy 0, policy_version 1685207 (0.0007) [2023-12-27 17:19:11,625][20347] Updated weights for policy 0, policy_version 1685217 (0.0008) [2023-12-27 17:19:11,931][20381] Updated weights for policy 1, policy_version 1683505 (0.0009) [2023-12-27 17:19:11,986][20381] Updated weights for policy 1, policy_version 1683515 (0.0006) [2023-12-27 17:19:12,038][20381] Updated weights for policy 1, policy_version 1683525 (0.0006) [2023-12-27 17:19:12,102][20381] Updated weights for policy 1, policy_version 1683535 (0.0006) [2023-12-27 17:19:12,327][20347] Updated weights for policy 0, policy_version 1685227 (0.0010) [2023-12-27 17:19:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 862527488. Throughput: 0: 10365.4, 1: 10170.4. Samples: 862520140. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:12,363][19170] Avg episode reward: [(0, '990.290'), (1, '951.340')] [2023-12-27 17:19:12,392][20347] Updated weights for policy 0, policy_version 1685237 (0.0007) [2023-12-27 17:19:12,442][20347] Updated weights for policy 0, policy_version 1685247 (0.0005) [2023-12-27 17:19:12,761][20381] Updated weights for policy 1, policy_version 1683545 (0.0006) [2023-12-27 17:19:12,821][20381] Updated weights for policy 1, policy_version 1683555 (0.0006) [2023-12-27 17:19:12,887][20381] Updated weights for policy 1, policy_version 1683565 (0.0009) [2023-12-27 17:19:13,116][20347] Updated weights for policy 0, policy_version 1685257 (0.0006) [2023-12-27 17:19:13,166][20347] Updated weights for policy 0, policy_version 1685267 (0.0005) [2023-12-27 17:19:13,218][20347] Updated weights for policy 0, policy_version 1685277 (0.0005) [2023-12-27 17:19:13,276][20347] Updated weights for policy 0, policy_version 1685287 (0.0005) [2023-12-27 17:19:13,504][20381] Updated weights for policy 1, policy_version 1683575 (0.0006) [2023-12-27 17:19:13,555][20381] Updated weights for policy 1, policy_version 1683585 (0.0005) [2023-12-27 17:19:13,610][20381] Updated weights for policy 1, policy_version 1683595 (0.0007) [2023-12-27 17:19:13,860][20347] Updated weights for policy 0, policy_version 1685297 (0.0010) [2023-12-27 17:19:13,901][20347] Updated weights for policy 0, policy_version 1685307 (0.0010) [2023-12-27 17:19:13,948][20347] Updated weights for policy 0, policy_version 1685317 (0.0010) [2023-12-27 17:19:14,202][20381] Updated weights for policy 1, policy_version 1683605 (0.0006) [2023-12-27 17:19:14,251][20381] Updated weights for policy 1, policy_version 1683615 (0.0007) [2023-12-27 17:19:14,300][20381] Updated weights for policy 1, policy_version 1683625 (0.0008) [2023-12-27 17:19:14,614][20347] Updated weights for policy 0, policy_version 1685327 (0.0011) [2023-12-27 17:19:14,673][20347] Updated weights for policy 0, policy_version 1685337 (0.0010) [2023-12-27 17:19:14,735][20347] Updated weights for policy 0, policy_version 1685347 (0.0011) [2023-12-27 17:19:15,032][20381] Updated weights for policy 1, policy_version 1683635 (0.0008) [2023-12-27 17:19:15,098][20381] Updated weights for policy 1, policy_version 1683645 (0.0008) [2023-12-27 17:19:15,153][20381] Updated weights for policy 1, policy_version 1683655 (0.0008) [2023-12-27 17:19:15,446][20347] Updated weights for policy 0, policy_version 1685357 (0.0011) [2023-12-27 17:19:15,503][20347] Updated weights for policy 0, policy_version 1685367 (0.0010) [2023-12-27 17:19:15,561][20347] Updated weights for policy 0, policy_version 1685377 (0.0010) [2023-12-27 17:19:15,873][20381] Updated weights for policy 1, policy_version 1683665 (0.0007) [2023-12-27 17:19:15,928][20381] Updated weights for policy 1, policy_version 1683675 (0.0005) [2023-12-27 17:19:15,980][20381] Updated weights for policy 1, policy_version 1683685 (0.0008) [2023-12-27 17:19:16,039][20381] Updated weights for policy 1, policy_version 1683695 (0.0010) [2023-12-27 17:19:16,161][20347] Updated weights for policy 0, policy_version 1685387 (0.0009) [2023-12-27 17:19:16,213][20347] Updated weights for policy 0, policy_version 1685397 (0.0008) [2023-12-27 17:19:16,273][20347] Updated weights for policy 0, policy_version 1685407 (0.0008) [2023-12-27 17:19:16,632][20381] Updated weights for policy 1, policy_version 1683705 (0.0010) [2023-12-27 17:19:16,687][20381] Updated weights for policy 1, policy_version 1683715 (0.0011) [2023-12-27 17:19:16,742][20381] Updated weights for policy 1, policy_version 1683725 (0.0010) [2023-12-27 17:19:16,975][20347] Updated weights for policy 0, policy_version 1685417 (0.0007) [2023-12-27 17:19:17,029][20347] Updated weights for policy 0, policy_version 1685427 (0.0007) [2023-12-27 17:19:17,073][20347] Updated weights for policy 0, policy_version 1685437 (0.0007) [2023-12-27 17:19:17,116][20347] Updated weights for policy 0, policy_version 1685447 (0.0007) [2023-12-27 17:19:17,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 862633984. Throughput: 0: 10335.1, 1: 10165.5. Samples: 862644548. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.260')] [2023-12-27 17:19:17,449][20381] Updated weights for policy 1, policy_version 1683735 (0.0011) [2023-12-27 17:19:17,507][20381] Updated weights for policy 1, policy_version 1683745 (0.0010) [2023-12-27 17:19:17,561][20381] Updated weights for policy 1, policy_version 1683755 (0.0010) [2023-12-27 17:19:17,800][20347] Updated weights for policy 0, policy_version 1685457 (0.0006) [2023-12-27 17:19:17,862][20347] Updated weights for policy 0, policy_version 1685467 (0.0009) [2023-12-27 17:19:17,921][20347] Updated weights for policy 0, policy_version 1685477 (0.0008) [2023-12-27 17:19:18,225][20381] Updated weights for policy 1, policy_version 1683765 (0.0010) [2023-12-27 17:19:18,277][20381] Updated weights for policy 1, policy_version 1683775 (0.0010) [2023-12-27 17:19:18,335][20381] Updated weights for policy 1, policy_version 1683785 (0.0010) [2023-12-27 17:19:18,497][20347] Updated weights for policy 0, policy_version 1685487 (0.0006) [2023-12-27 17:19:18,550][20347] Updated weights for policy 0, policy_version 1685497 (0.0006) [2023-12-27 17:19:18,602][20347] Updated weights for policy 0, policy_version 1685507 (0.0006) [2023-12-27 17:19:18,945][20381] Updated weights for policy 1, policy_version 1683795 (0.0007) [2023-12-27 17:19:19,008][20381] Updated weights for policy 1, policy_version 1683805 (0.0006) [2023-12-27 17:19:19,067][20381] Updated weights for policy 1, policy_version 1683815 (0.0008) [2023-12-27 17:19:19,318][20347] Updated weights for policy 0, policy_version 1685517 (0.0009) [2023-12-27 17:19:19,382][20347] Updated weights for policy 0, policy_version 1685527 (0.0008) [2023-12-27 17:19:19,444][20347] Updated weights for policy 0, policy_version 1685537 (0.0008) [2023-12-27 17:19:19,747][20381] Updated weights for policy 1, policy_version 1683825 (0.0010) [2023-12-27 17:19:19,803][20381] Updated weights for policy 1, policy_version 1683835 (0.0007) [2023-12-27 17:19:19,868][20381] Updated weights for policy 1, policy_version 1683845 (0.0007) [2023-12-27 17:19:19,935][20381] Updated weights for policy 1, policy_version 1683855 (0.0007) [2023-12-27 17:19:20,222][20347] Updated weights for policy 0, policy_version 1685547 (0.0008) [2023-12-27 17:19:20,278][20347] Updated weights for policy 0, policy_version 1685557 (0.0008) [2023-12-27 17:19:20,336][20347] Updated weights for policy 0, policy_version 1685567 (0.0008) [2023-12-27 17:19:20,602][20381] Updated weights for policy 1, policy_version 1683865 (0.0008) [2023-12-27 17:19:20,658][20381] Updated weights for policy 1, policy_version 1683875 (0.0008) [2023-12-27 17:19:20,715][20381] Updated weights for policy 1, policy_version 1683885 (0.0008) [2023-12-27 17:19:21,162][20347] Updated weights for policy 0, policy_version 1685577 (0.0008) [2023-12-27 17:19:21,225][20347] Updated weights for policy 0, policy_version 1685587 (0.0008) [2023-12-27 17:19:21,287][20347] Updated weights for policy 0, policy_version 1685597 (0.0007) [2023-12-27 17:19:21,351][20347] Updated weights for policy 0, policy_version 1685607 (0.0006) [2023-12-27 17:19:21,444][20381] Updated weights for policy 1, policy_version 1683895 (0.0009) [2023-12-27 17:19:21,502][20381] Updated weights for policy 1, policy_version 1683905 (0.0008) [2023-12-27 17:19:21,559][20381] Updated weights for policy 1, policy_version 1683915 (0.0007) [2023-12-27 17:19:22,042][20347] Updated weights for policy 0, policy_version 1685617 (0.0008) [2023-12-27 17:19:22,103][20347] Updated weights for policy 0, policy_version 1685627 (0.0007) [2023-12-27 17:19:22,158][20347] Updated weights for policy 0, policy_version 1685637 (0.0007) [2023-12-27 17:19:22,285][20381] Updated weights for policy 1, policy_version 1683925 (0.0008) [2023-12-27 17:19:22,350][20381] Updated weights for policy 1, policy_version 1683935 (0.0010) [2023-12-27 17:19:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 862732288. Throughput: 0: 10319.0, 1: 10155.1. Samples: 862706272. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:22,363][19170] Avg episode reward: [(0, '990.240'), (1, '970.780')] [2023-12-27 17:19:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001685640_431603712.pth... [2023-12-27 17:19:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001684456_431300608.pth [2023-12-27 17:19:22,415][20381] Updated weights for policy 1, policy_version 1683945 (0.0010) [2023-12-27 17:19:22,451][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001683952_431136768.pth... [2023-12-27 17:19:22,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001682736_430825472.pth [2023-12-27 17:19:22,881][20347] Updated weights for policy 0, policy_version 1685647 (0.0008) [2023-12-27 17:19:22,933][20347] Updated weights for policy 0, policy_version 1685657 (0.0008) [2023-12-27 17:19:22,985][20347] Updated weights for policy 0, policy_version 1685667 (0.0007) [2023-12-27 17:19:23,111][20381] Updated weights for policy 1, policy_version 1683955 (0.0009) [2023-12-27 17:19:23,170][20381] Updated weights for policy 1, policy_version 1683965 (0.0005) [2023-12-27 17:19:23,222][20381] Updated weights for policy 1, policy_version 1683975 (0.0005) [2023-12-27 17:19:23,645][20347] Updated weights for policy 0, policy_version 1685677 (0.0009) [2023-12-27 17:19:23,689][20347] Updated weights for policy 0, policy_version 1685687 (0.0010) [2023-12-27 17:19:23,738][20347] Updated weights for policy 0, policy_version 1685697 (0.0010) [2023-12-27 17:19:23,865][20381] Updated weights for policy 1, policy_version 1683985 (0.0009) [2023-12-27 17:19:23,929][20381] Updated weights for policy 1, policy_version 1683995 (0.0011) [2023-12-27 17:19:23,978][20381] Updated weights for policy 1, policy_version 1684005 (0.0009) [2023-12-27 17:19:24,029][20381] Updated weights for policy 1, policy_version 1684015 (0.0009) [2023-12-27 17:19:24,397][20347] Updated weights for policy 0, policy_version 1685707 (0.0010) [2023-12-27 17:19:24,457][20347] Updated weights for policy 0, policy_version 1685717 (0.0010) [2023-12-27 17:19:24,509][20347] Updated weights for policy 0, policy_version 1685727 (0.0010) [2023-12-27 17:19:24,678][20381] Updated weights for policy 1, policy_version 1684025 (0.0007) [2023-12-27 17:19:24,736][20381] Updated weights for policy 1, policy_version 1684035 (0.0008) [2023-12-27 17:19:24,798][20381] Updated weights for policy 1, policy_version 1684045 (0.0008) [2023-12-27 17:19:25,120][20347] Updated weights for policy 0, policy_version 1685737 (0.0010) [2023-12-27 17:19:25,188][20347] Updated weights for policy 0, policy_version 1685747 (0.0005) [2023-12-27 17:19:25,249][20347] Updated weights for policy 0, policy_version 1685757 (0.0008) [2023-12-27 17:19:25,310][20347] Updated weights for policy 0, policy_version 1685767 (0.0011) [2023-12-27 17:19:25,413][20381] Updated weights for policy 1, policy_version 1684055 (0.0007) [2023-12-27 17:19:25,471][20381] Updated weights for policy 1, policy_version 1684065 (0.0007) [2023-12-27 17:19:25,526][20381] Updated weights for policy 1, policy_version 1684075 (0.0007) [2023-12-27 17:19:25,945][20347] Updated weights for policy 0, policy_version 1685777 (0.0010) [2023-12-27 17:19:25,996][20347] Updated weights for policy 0, policy_version 1685787 (0.0010) [2023-12-27 17:19:26,048][20347] Updated weights for policy 0, policy_version 1685797 (0.0010) [2023-12-27 17:19:26,091][20381] Updated weights for policy 1, policy_version 1684085 (0.0009) [2023-12-27 17:19:26,150][20381] Updated weights for policy 1, policy_version 1684095 (0.0011) [2023-12-27 17:19:26,210][20381] Updated weights for policy 1, policy_version 1684105 (0.0010) [2023-12-27 17:19:26,716][20347] Updated weights for policy 0, policy_version 1685807 (0.0010) [2023-12-27 17:19:26,764][20347] Updated weights for policy 0, policy_version 1685817 (0.0010) [2023-12-27 17:19:26,813][20347] Updated weights for policy 0, policy_version 1685827 (0.0010) [2023-12-27 17:19:26,894][20381] Updated weights for policy 1, policy_version 1684115 (0.0009) [2023-12-27 17:19:26,944][20381] Updated weights for policy 1, policy_version 1684125 (0.0007) [2023-12-27 17:19:27,000][20381] Updated weights for policy 1, policy_version 1684135 (0.0008) [2023-12-27 17:19:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 862838784. Throughput: 0: 10227.4, 1: 10214.2. Samples: 862828980. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '980.530')] [2023-12-27 17:19:27,425][20347] Updated weights for policy 0, policy_version 1685837 (0.0008) [2023-12-27 17:19:27,484][20347] Updated weights for policy 0, policy_version 1685847 (0.0005) [2023-12-27 17:19:27,538][20347] Updated weights for policy 0, policy_version 1685857 (0.0010) [2023-12-27 17:19:27,737][20381] Updated weights for policy 1, policy_version 1684145 (0.0007) [2023-12-27 17:19:27,788][20381] Updated weights for policy 1, policy_version 1684155 (0.0009) [2023-12-27 17:19:27,841][20381] Updated weights for policy 1, policy_version 1684165 (0.0011) [2023-12-27 17:19:27,896][20381] Updated weights for policy 1, policy_version 1684175 (0.0006) [2023-12-27 17:19:28,187][20347] Updated weights for policy 0, policy_version 1685867 (0.0010) [2023-12-27 17:19:28,235][20347] Updated weights for policy 0, policy_version 1685877 (0.0010) [2023-12-27 17:19:28,286][20347] Updated weights for policy 0, policy_version 1685887 (0.0007) [2023-12-27 17:19:28,572][20381] Updated weights for policy 1, policy_version 1684185 (0.0010) [2023-12-27 17:19:28,628][20381] Updated weights for policy 1, policy_version 1684195 (0.0010) [2023-12-27 17:19:28,677][20381] Updated weights for policy 1, policy_version 1684205 (0.0011) [2023-12-27 17:19:28,952][20347] Updated weights for policy 0, policy_version 1685897 (0.0008) [2023-12-27 17:19:29,016][20347] Updated weights for policy 0, policy_version 1685907 (0.0007) [2023-12-27 17:19:29,067][20347] Updated weights for policy 0, policy_version 1685917 (0.0007) [2023-12-27 17:19:29,123][20347] Updated weights for policy 0, policy_version 1685927 (0.0007) [2023-12-27 17:19:29,389][20381] Updated weights for policy 1, policy_version 1684215 (0.0009) [2023-12-27 17:19:29,444][20381] Updated weights for policy 1, policy_version 1684225 (0.0010) [2023-12-27 17:19:29,497][20381] Updated weights for policy 1, policy_version 1684235 (0.0009) [2023-12-27 17:19:29,840][20347] Updated weights for policy 0, policy_version 1685937 (0.0007) [2023-12-27 17:19:29,908][20347] Updated weights for policy 0, policy_version 1685947 (0.0007) [2023-12-27 17:19:29,969][20347] Updated weights for policy 0, policy_version 1685957 (0.0009) [2023-12-27 17:19:30,128][20381] Updated weights for policy 1, policy_version 1684245 (0.0008) [2023-12-27 17:19:30,183][20381] Updated weights for policy 1, policy_version 1684255 (0.0010) [2023-12-27 17:19:30,242][20381] Updated weights for policy 1, policy_version 1684265 (0.0010) [2023-12-27 17:19:30,581][20347] Updated weights for policy 0, policy_version 1685967 (0.0010) [2023-12-27 17:19:30,630][20347] Updated weights for policy 0, policy_version 1685977 (0.0010) [2023-12-27 17:19:30,690][20347] Updated weights for policy 0, policy_version 1685987 (0.0010) [2023-12-27 17:19:30,940][20381] Updated weights for policy 1, policy_version 1684275 (0.0010) [2023-12-27 17:19:30,990][20381] Updated weights for policy 1, policy_version 1684285 (0.0007) [2023-12-27 17:19:31,050][20381] Updated weights for policy 1, policy_version 1684295 (0.0008) [2023-12-27 17:19:31,380][20347] Updated weights for policy 0, policy_version 1685997 (0.0009) [2023-12-27 17:19:31,437][20347] Updated weights for policy 0, policy_version 1686007 (0.0007) [2023-12-27 17:19:31,491][20347] Updated weights for policy 0, policy_version 1686017 (0.0008) [2023-12-27 17:19:31,802][20381] Updated weights for policy 1, policy_version 1684305 (0.0008) [2023-12-27 17:19:31,865][20381] Updated weights for policy 1, policy_version 1684315 (0.0007) [2023-12-27 17:19:31,919][20381] Updated weights for policy 1, policy_version 1684325 (0.0007) [2023-12-27 17:19:31,972][20381] Updated weights for policy 1, policy_version 1684335 (0.0006) [2023-12-27 17:19:32,217][20347] Updated weights for policy 0, policy_version 1686027 (0.0007) [2023-12-27 17:19:32,280][20347] Updated weights for policy 0, policy_version 1686037 (0.0008) [2023-12-27 17:19:32,339][20347] Updated weights for policy 0, policy_version 1686047 (0.0007) [2023-12-27 17:19:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 862937088. Throughput: 0: 10238.9, 1: 10237.7. Samples: 862950748. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '970.760')] [2023-12-27 17:19:32,715][20381] Updated weights for policy 1, policy_version 1684345 (0.0007) [2023-12-27 17:19:32,775][20381] Updated weights for policy 1, policy_version 1684355 (0.0008) [2023-12-27 17:19:32,831][20381] Updated weights for policy 1, policy_version 1684365 (0.0009) [2023-12-27 17:19:32,943][20347] Updated weights for policy 0, policy_version 1686058 (0.0008) [2023-12-27 17:19:33,000][20347] Updated weights for policy 0, policy_version 1686068 (0.0010) [2023-12-27 17:19:33,061][20347] Updated weights for policy 0, policy_version 1686078 (0.0005) [2023-12-27 17:19:33,114][20347] Updated weights for policy 0, policy_version 1686088 (0.0007) [2023-12-27 17:19:33,505][20381] Updated weights for policy 1, policy_version 1684375 (0.0006) [2023-12-27 17:19:33,561][20381] Updated weights for policy 1, policy_version 1684385 (0.0005) [2023-12-27 17:19:33,615][20381] Updated weights for policy 1, policy_version 1684395 (0.0007) [2023-12-27 17:19:33,759][20347] Updated weights for policy 0, policy_version 1686098 (0.0008) [2023-12-27 17:19:33,819][20347] Updated weights for policy 0, policy_version 1686108 (0.0005) [2023-12-27 17:19:33,885][20347] Updated weights for policy 0, policy_version 1686118 (0.0005) [2023-12-27 17:19:34,209][20381] Updated weights for policy 1, policy_version 1684405 (0.0011) [2023-12-27 17:19:34,266][20381] Updated weights for policy 1, policy_version 1684415 (0.0011) [2023-12-27 17:19:34,330][20381] Updated weights for policy 1, policy_version 1684425 (0.0010) [2023-12-27 17:19:34,483][20347] Updated weights for policy 0, policy_version 1686128 (0.0009) [2023-12-27 17:19:34,548][20347] Updated weights for policy 0, policy_version 1686138 (0.0011) [2023-12-27 17:19:34,611][20347] Updated weights for policy 0, policy_version 1686148 (0.0010) [2023-12-27 17:19:34,999][20381] Updated weights for policy 1, policy_version 1684435 (0.0007) [2023-12-27 17:19:35,055][20381] Updated weights for policy 1, policy_version 1684445 (0.0007) [2023-12-27 17:19:35,110][20381] Updated weights for policy 1, policy_version 1684455 (0.0007) [2023-12-27 17:19:35,266][20347] Updated weights for policy 0, policy_version 1686158 (0.0010) [2023-12-27 17:19:35,326][20347] Updated weights for policy 0, policy_version 1686168 (0.0011) [2023-12-27 17:19:35,392][20347] Updated weights for policy 0, policy_version 1686178 (0.0009) [2023-12-27 17:19:35,787][20381] Updated weights for policy 1, policy_version 1684465 (0.0006) [2023-12-27 17:19:35,845][20381] Updated weights for policy 1, policy_version 1684475 (0.0010) [2023-12-27 17:19:35,898][20381] Updated weights for policy 1, policy_version 1684485 (0.0011) [2023-12-27 17:19:35,955][20381] Updated weights for policy 1, policy_version 1684495 (0.0011) [2023-12-27 17:19:36,037][20347] Updated weights for policy 0, policy_version 1686188 (0.0007) [2023-12-27 17:19:36,071][20291] KL-divergence is very high: 113.3750 [2023-12-27 17:19:36,087][20347] Updated weights for policy 0, policy_version 1686198 (0.0007) [2023-12-27 17:19:36,153][20347] Updated weights for policy 0, policy_version 1686208 (0.0007) [2023-12-27 17:19:36,598][20381] Updated weights for policy 1, policy_version 1684505 (0.0008) [2023-12-27 17:19:36,658][20381] Updated weights for policy 1, policy_version 1684515 (0.0006) [2023-12-27 17:19:36,722][20381] Updated weights for policy 1, policy_version 1684525 (0.0007) [2023-12-27 17:19:36,893][20347] Updated weights for policy 0, policy_version 1686218 (0.0008) [2023-12-27 17:19:36,943][20347] Updated weights for policy 0, policy_version 1686228 (0.0008) [2023-12-27 17:19:36,990][20347] Updated weights for policy 0, policy_version 1686238 (0.0007) [2023-12-27 17:19:37,039][20347] Updated weights for policy 0, policy_version 1686248 (0.0007) [2023-12-27 17:19:37,354][20381] Updated weights for policy 1, policy_version 1684535 (0.0007) [2023-12-27 17:19:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20549.5). Total num frames: 863043584. Throughput: 0: 10289.7, 1: 10225.1. Samples: 863014648. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:37,363][19170] Avg episode reward: [(0, '970.730'), (1, '970.850')] [2023-12-27 17:19:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001686248_431759360.pth... [2023-12-27 17:19:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001685032_431448064.pth [2023-12-27 17:19:37,410][20381] Updated weights for policy 1, policy_version 1684545 (0.0005) [2023-12-27 17:19:37,470][20381] Updated weights for policy 1, policy_version 1684555 (0.0006) [2023-12-27 17:19:37,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001684560_431292416.pth... [2023-12-27 17:19:37,502][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001683344_430981120.pth [2023-12-27 17:19:37,815][20347] Updated weights for policy 0, policy_version 1686258 (0.0007) [2023-12-27 17:19:37,870][20347] Updated weights for policy 0, policy_version 1686268 (0.0008) [2023-12-27 17:19:37,918][20347] Updated weights for policy 0, policy_version 1686278 (0.0007) [2023-12-27 17:19:38,064][20381] Updated weights for policy 1, policy_version 1684565 (0.0008) [2023-12-27 17:19:38,118][20381] Updated weights for policy 1, policy_version 1684575 (0.0010) [2023-12-27 17:19:38,172][20381] Updated weights for policy 1, policy_version 1684585 (0.0010) [2023-12-27 17:19:38,625][20347] Updated weights for policy 0, policy_version 1686288 (0.0008) [2023-12-27 17:19:38,687][20347] Updated weights for policy 0, policy_version 1686298 (0.0008) [2023-12-27 17:19:38,746][20347] Updated weights for policy 0, policy_version 1686308 (0.0009) [2023-12-27 17:19:38,855][20381] Updated weights for policy 1, policy_version 1684595 (0.0009) [2023-12-27 17:19:38,917][20381] Updated weights for policy 1, policy_version 1684605 (0.0005) [2023-12-27 17:19:38,975][20381] Updated weights for policy 1, policy_version 1684615 (0.0005) [2023-12-27 17:19:39,503][20347] Updated weights for policy 0, policy_version 1686318 (0.0007) [2023-12-27 17:19:39,559][20347] Updated weights for policy 0, policy_version 1686328 (0.0008) [2023-12-27 17:19:39,597][20381] Updated weights for policy 1, policy_version 1684625 (0.0006) [2023-12-27 17:19:39,607][20347] Updated weights for policy 0, policy_version 1686338 (0.0009) [2023-12-27 17:19:39,654][20381] Updated weights for policy 1, policy_version 1684635 (0.0011) [2023-12-27 17:19:39,718][20381] Updated weights for policy 1, policy_version 1684645 (0.0009) [2023-12-27 17:19:39,779][20381] Updated weights for policy 1, policy_version 1684655 (0.0006) [2023-12-27 17:19:40,323][20347] Updated weights for policy 0, policy_version 1686348 (0.0010) [2023-12-27 17:19:40,383][20347] Updated weights for policy 0, policy_version 1686358 (0.0010) [2023-12-27 17:19:40,445][20347] Updated weights for policy 0, policy_version 1686368 (0.0007) [2023-12-27 17:19:40,445][20381] Updated weights for policy 1, policy_version 1684665 (0.0010) [2023-12-27 17:19:40,507][20381] Updated weights for policy 1, policy_version 1684675 (0.0011) [2023-12-27 17:19:40,566][20381] Updated weights for policy 1, policy_version 1684685 (0.0011) [2023-12-27 17:19:40,996][20347] Updated weights for policy 0, policy_version 1686378 (0.0005) [2023-12-27 17:19:41,060][20347] Updated weights for policy 0, policy_version 1686388 (0.0009) [2023-12-27 17:19:41,118][20347] Updated weights for policy 0, policy_version 1686398 (0.0008) [2023-12-27 17:19:41,180][20347] Updated weights for policy 0, policy_version 1686408 (0.0007) [2023-12-27 17:19:41,245][20381] Updated weights for policy 1, policy_version 1684695 (0.0009) [2023-12-27 17:19:41,310][20381] Updated weights for policy 1, policy_version 1684705 (0.0009) [2023-12-27 17:19:41,374][20381] Updated weights for policy 1, policy_version 1684715 (0.0011) [2023-12-27 17:19:41,902][20347] Updated weights for policy 0, policy_version 1686418 (0.0008) [2023-12-27 17:19:41,954][20347] Updated weights for policy 0, policy_version 1686428 (0.0008) [2023-12-27 17:19:42,006][20347] Updated weights for policy 0, policy_version 1686438 (0.0008) [2023-12-27 17:19:42,149][20381] Updated weights for policy 1, policy_version 1684725 (0.0008) [2023-12-27 17:19:42,199][20381] Updated weights for policy 1, policy_version 1684735 (0.0007) [2023-12-27 17:19:42,255][20381] Updated weights for policy 1, policy_version 1684745 (0.0007) [2023-12-27 17:19:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 863150080. Throughput: 0: 10243.2, 1: 10262.4. Samples: 863138104. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:42,363][19170] Avg episode reward: [(0, '960.940'), (1, '970.800')] [2023-12-27 17:19:42,807][20347] Updated weights for policy 0, policy_version 1686448 (0.0008) [2023-12-27 17:19:42,862][20347] Updated weights for policy 0, policy_version 1686458 (0.0005) [2023-12-27 17:19:42,915][20347] Updated weights for policy 0, policy_version 1686468 (0.0005) [2023-12-27 17:19:42,999][20381] Updated weights for policy 1, policy_version 1684755 (0.0008) [2023-12-27 17:19:43,047][20381] Updated weights for policy 1, policy_version 1684765 (0.0009) [2023-12-27 17:19:43,095][20381] Updated weights for policy 1, policy_version 1684775 (0.0008) [2023-12-27 17:19:43,516][20347] Updated weights for policy 0, policy_version 1686478 (0.0006) [2023-12-27 17:19:43,564][20347] Updated weights for policy 0, policy_version 1686488 (0.0007) [2023-12-27 17:19:43,617][20347] Updated weights for policy 0, policy_version 1686498 (0.0007) [2023-12-27 17:19:43,792][20381] Updated weights for policy 1, policy_version 1684785 (0.0008) [2023-12-27 17:19:43,840][20381] Updated weights for policy 1, policy_version 1684795 (0.0005) [2023-12-27 17:19:43,894][20381] Updated weights for policy 1, policy_version 1684805 (0.0005) [2023-12-27 17:19:43,941][20381] Updated weights for policy 1, policy_version 1684815 (0.0006) [2023-12-27 17:19:44,337][20347] Updated weights for policy 0, policy_version 1686508 (0.0007) [2023-12-27 17:19:44,393][20347] Updated weights for policy 0, policy_version 1686518 (0.0008) [2023-12-27 17:19:44,442][20347] Updated weights for policy 0, policy_version 1686528 (0.0008) [2023-12-27 17:19:44,544][20381] Updated weights for policy 1, policy_version 1684825 (0.0007) [2023-12-27 17:19:44,593][20381] Updated weights for policy 1, policy_version 1684835 (0.0006) [2023-12-27 17:19:44,639][20381] Updated weights for policy 1, policy_version 1684845 (0.0005) [2023-12-27 17:19:45,169][20347] Updated weights for policy 0, policy_version 1686538 (0.0008) [2023-12-27 17:19:45,239][20347] Updated weights for policy 0, policy_version 1686548 (0.0007) [2023-12-27 17:19:45,298][20381] Updated weights for policy 1, policy_version 1684855 (0.0007) [2023-12-27 17:19:45,300][20347] Updated weights for policy 0, policy_version 1686558 (0.0008) [2023-12-27 17:19:45,353][20381] Updated weights for policy 1, policy_version 1684865 (0.0007) [2023-12-27 17:19:45,355][20347] Updated weights for policy 0, policy_version 1686568 (0.0005) [2023-12-27 17:19:45,407][20381] Updated weights for policy 1, policy_version 1684875 (0.0007) [2023-12-27 17:19:46,040][20347] Updated weights for policy 0, policy_version 1686578 (0.0009) [2023-12-27 17:19:46,068][20381] Updated weights for policy 1, policy_version 1684885 (0.0007) [2023-12-27 17:19:46,097][20347] Updated weights for policy 0, policy_version 1686588 (0.0007) [2023-12-27 17:19:46,126][20381] Updated weights for policy 1, policy_version 1684895 (0.0005) [2023-12-27 17:19:46,149][20347] Updated weights for policy 0, policy_version 1686598 (0.0008) [2023-12-27 17:19:46,184][20381] Updated weights for policy 1, policy_version 1684905 (0.0005) [2023-12-27 17:19:46,723][20381] Updated weights for policy 1, policy_version 1684915 (0.0009) [2023-12-27 17:19:46,776][20381] Updated weights for policy 1, policy_version 1684925 (0.0008) [2023-12-27 17:19:46,828][20381] Updated weights for policy 1, policy_version 1684935 (0.0008) [2023-12-27 17:19:46,891][20347] Updated weights for policy 0, policy_version 1686608 (0.0007) [2023-12-27 17:19:46,937][20347] Updated weights for policy 0, policy_version 1686618 (0.0008) [2023-12-27 17:19:46,995][20347] Updated weights for policy 0, policy_version 1686628 (0.0008) [2023-12-27 17:19:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 863248384. Throughput: 0: 10184.8, 1: 10329.0. Samples: 863258912. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:47,363][19170] Avg episode reward: [(0, '970.680'), (1, '980.510')] [2023-12-27 17:19:47,475][20381] Updated weights for policy 1, policy_version 1684945 (0.0007) [2023-12-27 17:19:47,531][20381] Updated weights for policy 1, policy_version 1684955 (0.0007) [2023-12-27 17:19:47,586][20381] Updated weights for policy 1, policy_version 1684965 (0.0008) [2023-12-27 17:19:47,644][20381] Updated weights for policy 1, policy_version 1684975 (0.0007) [2023-12-27 17:19:47,722][20347] Updated weights for policy 0, policy_version 1686638 (0.0009) [2023-12-27 17:19:47,774][20347] Updated weights for policy 0, policy_version 1686648 (0.0010) [2023-12-27 17:19:47,826][20347] Updated weights for policy 0, policy_version 1686658 (0.0010) [2023-12-27 17:19:48,335][20381] Updated weights for policy 1, policy_version 1684985 (0.0008) [2023-12-27 17:19:48,394][20381] Updated weights for policy 1, policy_version 1684995 (0.0007) [2023-12-27 17:19:48,457][20381] Updated weights for policy 1, policy_version 1685005 (0.0005) [2023-12-27 17:19:48,505][20347] Updated weights for policy 0, policy_version 1686668 (0.0009) [2023-12-27 17:19:48,556][20347] Updated weights for policy 0, policy_version 1686678 (0.0008) [2023-12-27 17:19:48,616][20347] Updated weights for policy 0, policy_version 1686688 (0.0009) [2023-12-27 17:19:49,054][20381] Updated weights for policy 1, policy_version 1685015 (0.0006) [2023-12-27 17:19:49,111][20381] Updated weights for policy 1, policy_version 1685025 (0.0008) [2023-12-27 17:19:49,161][20381] Updated weights for policy 1, policy_version 1685035 (0.0010) [2023-12-27 17:19:49,350][20347] Updated weights for policy 0, policy_version 1686698 (0.0011) [2023-12-27 17:19:49,410][20347] Updated weights for policy 0, policy_version 1686708 (0.0008) [2023-12-27 17:19:49,464][20347] Updated weights for policy 0, policy_version 1686718 (0.0006) [2023-12-27 17:19:49,524][20347] Updated weights for policy 0, policy_version 1686728 (0.0007) [2023-12-27 17:19:49,860][20381] Updated weights for policy 1, policy_version 1685045 (0.0010) [2023-12-27 17:19:49,923][20381] Updated weights for policy 1, policy_version 1685055 (0.0011) [2023-12-27 17:19:49,987][20381] Updated weights for policy 1, policy_version 1685065 (0.0010) [2023-12-27 17:19:50,244][20347] Updated weights for policy 0, policy_version 1686738 (0.0008) [2023-12-27 17:19:50,301][20347] Updated weights for policy 0, policy_version 1686748 (0.0008) [2023-12-27 17:19:50,361][20347] Updated weights for policy 0, policy_version 1686758 (0.0009) [2023-12-27 17:19:50,588][20381] Updated weights for policy 1, policy_version 1685075 (0.0009) [2023-12-27 17:19:50,643][20381] Updated weights for policy 1, policy_version 1685085 (0.0006) [2023-12-27 17:19:50,700][20381] Updated weights for policy 1, policy_version 1685095 (0.0005) [2023-12-27 17:19:51,095][20347] Updated weights for policy 0, policy_version 1686768 (0.0008) [2023-12-27 17:19:51,164][20347] Updated weights for policy 0, policy_version 1686778 (0.0009) [2023-12-27 17:19:51,227][20347] Updated weights for policy 0, policy_version 1686788 (0.0010) [2023-12-27 17:19:51,342][20381] Updated weights for policy 1, policy_version 1685105 (0.0006) [2023-12-27 17:19:51,412][20381] Updated weights for policy 1, policy_version 1685115 (0.0008) [2023-12-27 17:19:51,474][20381] Updated weights for policy 1, policy_version 1685125 (0.0008) [2023-12-27 17:19:51,528][20381] Updated weights for policy 1, policy_version 1685135 (0.0008) [2023-12-27 17:19:51,937][20347] Updated weights for policy 0, policy_version 1686798 (0.0007) [2023-12-27 17:19:51,997][20347] Updated weights for policy 0, policy_version 1686808 (0.0005) [2023-12-27 17:19:52,062][20347] Updated weights for policy 0, policy_version 1686818 (0.0005) [2023-12-27 17:19:52,231][20381] Updated weights for policy 1, policy_version 1685145 (0.0008) [2023-12-27 17:19:52,289][20381] Updated weights for policy 1, policy_version 1685155 (0.0009) [2023-12-27 17:19:52,335][20381] Updated weights for policy 1, policy_version 1685165 (0.0008) [2023-12-27 17:19:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 863354880. Throughput: 0: 10154.2, 1: 10346.0. Samples: 863320096. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:52,363][19170] Avg episode reward: [(0, '960.930'), (1, '980.590')] [2023-12-27 17:19:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001685168_431448064.pth... [2023-12-27 17:19:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001686824_431906816.pth... [2023-12-27 17:19:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001683952_431136768.pth [2023-12-27 17:19:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001685640_431603712.pth [2023-12-27 17:19:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001685168_431448064.pth [2023-12-27 17:19:52,373][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001686824_431906816.pth [2023-12-27 17:19:52,612][20347] Updated weights for policy 0, policy_version 1686828 (0.0008) [2023-12-27 17:19:52,673][20347] Updated weights for policy 0, policy_version 1686838 (0.0007) [2023-12-27 17:19:52,735][20347] Updated weights for policy 0, policy_version 1686848 (0.0005) [2023-12-27 17:19:53,110][20381] Updated weights for policy 1, policy_version 1685175 (0.0008) [2023-12-27 17:19:53,152][20381] Updated weights for policy 1, policy_version 1685185 (0.0007) [2023-12-27 17:19:53,207][20381] Updated weights for policy 1, policy_version 1685195 (0.0007) [2023-12-27 17:19:53,282][20347] Updated weights for policy 0, policy_version 1686858 (0.0006) [2023-12-27 17:19:53,344][20347] Updated weights for policy 0, policy_version 1686868 (0.0007) [2023-12-27 17:19:53,394][20347] Updated weights for policy 0, policy_version 1686878 (0.0008) [2023-12-27 17:19:53,446][20347] Updated weights for policy 0, policy_version 1686888 (0.0008) [2023-12-27 17:19:53,862][20381] Updated weights for policy 1, policy_version 1685205 (0.0007) [2023-12-27 17:19:53,920][20381] Updated weights for policy 1, policy_version 1685215 (0.0007) [2023-12-27 17:19:53,979][20381] Updated weights for policy 1, policy_version 1685225 (0.0005) [2023-12-27 17:19:54,165][20347] Updated weights for policy 0, policy_version 1686898 (0.0008) [2023-12-27 17:19:54,222][20347] Updated weights for policy 0, policy_version 1686908 (0.0008) [2023-12-27 17:19:54,278][20347] Updated weights for policy 0, policy_version 1686918 (0.0008) [2023-12-27 17:19:54,562][20381] Updated weights for policy 1, policy_version 1685235 (0.0006) [2023-12-27 17:19:54,611][20381] Updated weights for policy 1, policy_version 1685245 (0.0007) [2023-12-27 17:19:54,655][20381] Updated weights for policy 1, policy_version 1685255 (0.0007) [2023-12-27 17:19:54,971][20347] Updated weights for policy 0, policy_version 1686928 (0.0007) [2023-12-27 17:19:55,034][20347] Updated weights for policy 0, policy_version 1686938 (0.0008) [2023-12-27 17:19:55,091][20347] Updated weights for policy 0, policy_version 1686948 (0.0007) [2023-12-27 17:19:55,356][20381] Updated weights for policy 1, policy_version 1685265 (0.0007) [2023-12-27 17:19:55,417][20381] Updated weights for policy 1, policy_version 1685275 (0.0007) [2023-12-27 17:19:55,474][20381] Updated weights for policy 1, policy_version 1685285 (0.0008) [2023-12-27 17:19:55,533][20381] Updated weights for policy 1, policy_version 1685295 (0.0007) [2023-12-27 17:19:55,746][20347] Updated weights for policy 0, policy_version 1686958 (0.0006) [2023-12-27 17:19:55,806][20347] Updated weights for policy 0, policy_version 1686968 (0.0009) [2023-12-27 17:19:55,858][20347] Updated weights for policy 0, policy_version 1686978 (0.0010) [2023-12-27 17:19:56,242][20381] Updated weights for policy 1, policy_version 1685305 (0.0006) [2023-12-27 17:19:56,292][20381] Updated weights for policy 1, policy_version 1685315 (0.0005) [2023-12-27 17:19:56,337][20381] Updated weights for policy 1, policy_version 1685325 (0.0005) [2023-12-27 17:19:56,495][20347] Updated weights for policy 0, policy_version 1686988 (0.0010) [2023-12-27 17:19:56,545][20347] Updated weights for policy 0, policy_version 1686998 (0.0009) [2023-12-27 17:19:56,592][20347] Updated weights for policy 0, policy_version 1687008 (0.0005) [2023-12-27 17:19:56,941][20381] Updated weights for policy 1, policy_version 1685335 (0.0005) [2023-12-27 17:19:57,005][20381] Updated weights for policy 1, policy_version 1685345 (0.0009) [2023-12-27 17:19:57,063][20381] Updated weights for policy 1, policy_version 1685355 (0.0010) [2023-12-27 17:19:57,258][20347] Updated weights for policy 0, policy_version 1687018 (0.0007) [2023-12-27 17:19:57,315][20347] Updated weights for policy 0, policy_version 1687028 (0.0008) [2023-12-27 17:19:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20521.7). Total num frames: 863453184. Throughput: 0: 10176.2, 1: 10346.8. Samples: 863443672. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:19:57,363][19170] Avg episode reward: [(0, '951.190'), (1, '990.340')] [2023-12-27 17:19:57,373][20347] Updated weights for policy 0, policy_version 1687038 (0.0009) [2023-12-27 17:19:57,417][20347] Updated weights for policy 0, policy_version 1687048 (0.0008) [2023-12-27 17:19:57,679][20381] Updated weights for policy 1, policy_version 1685365 (0.0008) [2023-12-27 17:19:57,733][20381] Updated weights for policy 1, policy_version 1685375 (0.0006) [2023-12-27 17:19:57,789][20381] Updated weights for policy 1, policy_version 1685385 (0.0007) [2023-12-27 17:19:58,095][20347] Updated weights for policy 0, policy_version 1687058 (0.0010) [2023-12-27 17:19:58,146][20347] Updated weights for policy 0, policy_version 1687068 (0.0010) [2023-12-27 17:19:58,203][20347] Updated weights for policy 0, policy_version 1687078 (0.0009) [2023-12-27 17:19:58,437][20381] Updated weights for policy 1, policy_version 1685395 (0.0008) [2023-12-27 17:19:58,503][20381] Updated weights for policy 1, policy_version 1685405 (0.0007) [2023-12-27 17:19:58,572][20381] Updated weights for policy 1, policy_version 1685415 (0.0008) [2023-12-27 17:19:58,966][20347] Updated weights for policy 0, policy_version 1687088 (0.0009) [2023-12-27 17:19:59,020][20347] Updated weights for policy 0, policy_version 1687098 (0.0009) [2023-12-27 17:19:59,078][20347] Updated weights for policy 0, policy_version 1687108 (0.0009) [2023-12-27 17:19:59,484][20381] Updated weights for policy 1, policy_version 1685425 (0.0010) [2023-12-27 17:19:59,550][20381] Updated weights for policy 1, policy_version 1685435 (0.0007) [2023-12-27 17:19:59,606][20381] Updated weights for policy 1, policy_version 1685445 (0.0007) [2023-12-27 17:19:59,664][20381] Updated weights for policy 1, policy_version 1685455 (0.0007) [2023-12-27 17:19:59,905][20347] Updated weights for policy 0, policy_version 1687118 (0.0008) [2023-12-27 17:19:59,966][20347] Updated weights for policy 0, policy_version 1687128 (0.0009) [2023-12-27 17:20:00,027][20347] Updated weights for policy 0, policy_version 1687138 (0.0008) [2023-12-27 17:20:00,341][20381] Updated weights for policy 1, policy_version 1685465 (0.0010) [2023-12-27 17:20:00,402][20381] Updated weights for policy 1, policy_version 1685475 (0.0010) [2023-12-27 17:20:00,460][20381] Updated weights for policy 1, policy_version 1685485 (0.0008) [2023-12-27 17:20:00,685][20347] Updated weights for policy 0, policy_version 1687148 (0.0006) [2023-12-27 17:20:00,734][20347] Updated weights for policy 0, policy_version 1687158 (0.0007) [2023-12-27 17:20:00,780][20347] Updated weights for policy 0, policy_version 1687168 (0.0007) [2023-12-27 17:20:01,087][20381] Updated weights for policy 1, policy_version 1685495 (0.0007) [2023-12-27 17:20:01,151][20381] Updated weights for policy 1, policy_version 1685505 (0.0008) [2023-12-27 17:20:01,213][20381] Updated weights for policy 1, policy_version 1685515 (0.0007) [2023-12-27 17:20:01,504][20347] Updated weights for policy 0, policy_version 1687178 (0.0007) [2023-12-27 17:20:01,563][20347] Updated weights for policy 0, policy_version 1687188 (0.0007) [2023-12-27 17:20:01,647][20347] Updated weights for policy 0, policy_version 1687198 (0.0007) [2023-12-27 17:20:01,705][20347] Updated weights for policy 0, policy_version 1687208 (0.0005) [2023-12-27 17:20:01,938][20381] Updated weights for policy 1, policy_version 1685525 (0.0010) [2023-12-27 17:20:01,997][20381] Updated weights for policy 1, policy_version 1685535 (0.0011) [2023-12-27 17:20:02,057][20381] Updated weights for policy 1, policy_version 1685545 (0.0010) [2023-12-27 17:20:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 863551488. Throughput: 0: 10105.8, 1: 10295.8. Samples: 863562620. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:02,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:20:02,405][20347] Updated weights for policy 0, policy_version 1687218 (0.0007) [2023-12-27 17:20:02,463][20347] Updated weights for policy 0, policy_version 1687228 (0.0008) [2023-12-27 17:20:02,522][20347] Updated weights for policy 0, policy_version 1687238 (0.0008) [2023-12-27 17:20:02,765][20381] Updated weights for policy 1, policy_version 1685555 (0.0008) [2023-12-27 17:20:02,822][20381] Updated weights for policy 1, policy_version 1685565 (0.0006) [2023-12-27 17:20:02,879][20381] Updated weights for policy 1, policy_version 1685575 (0.0009) [2023-12-27 17:20:03,223][20347] Updated weights for policy 0, policy_version 1687248 (0.0008) [2023-12-27 17:20:03,276][20347] Updated weights for policy 0, policy_version 1687258 (0.0007) [2023-12-27 17:20:03,327][20347] Updated weights for policy 0, policy_version 1687268 (0.0007) [2023-12-27 17:20:03,535][20381] Updated weights for policy 1, policy_version 1685585 (0.0007) [2023-12-27 17:20:03,589][20381] Updated weights for policy 1, policy_version 1685595 (0.0007) [2023-12-27 17:20:03,647][20381] Updated weights for policy 1, policy_version 1685605 (0.0007) [2023-12-27 17:20:03,701][20381] Updated weights for policy 1, policy_version 1685615 (0.0008) [2023-12-27 17:20:04,023][20347] Updated weights for policy 0, policy_version 1687278 (0.0006) [2023-12-27 17:20:04,070][20347] Updated weights for policy 0, policy_version 1687288 (0.0005) [2023-12-27 17:20:04,123][20347] Updated weights for policy 0, policy_version 1687298 (0.0006) [2023-12-27 17:20:04,371][20381] Updated weights for policy 1, policy_version 1685625 (0.0007) [2023-12-27 17:20:04,420][20381] Updated weights for policy 1, policy_version 1685635 (0.0007) [2023-12-27 17:20:04,469][20381] Updated weights for policy 1, policy_version 1685645 (0.0007) [2023-12-27 17:20:04,848][20347] Updated weights for policy 0, policy_version 1687308 (0.0008) [2023-12-27 17:20:04,895][20347] Updated weights for policy 0, policy_version 1687318 (0.0010) [2023-12-27 17:20:04,947][20347] Updated weights for policy 0, policy_version 1687328 (0.0010) [2023-12-27 17:20:05,014][20381] Updated weights for policy 1, policy_version 1685655 (0.0005) [2023-12-27 17:20:05,067][20381] Updated weights for policy 1, policy_version 1685665 (0.0006) [2023-12-27 17:20:05,123][20381] Updated weights for policy 1, policy_version 1685675 (0.0005) [2023-12-27 17:20:05,617][20347] Updated weights for policy 0, policy_version 1687338 (0.0010) [2023-12-27 17:20:05,673][20347] Updated weights for policy 0, policy_version 1687348 (0.0010) [2023-12-27 17:20:05,719][20347] Updated weights for policy 0, policy_version 1687358 (0.0010) [2023-12-27 17:20:05,775][20347] Updated weights for policy 0, policy_version 1687368 (0.0011) [2023-12-27 17:20:05,812][20381] Updated weights for policy 1, policy_version 1685685 (0.0008) [2023-12-27 17:20:05,869][20381] Updated weights for policy 1, policy_version 1685695 (0.0007) [2023-12-27 17:20:05,916][20381] Updated weights for policy 1, policy_version 1685705 (0.0007) [2023-12-27 17:20:06,518][20347] Updated weights for policy 0, policy_version 1687378 (0.0010) [2023-12-27 17:20:06,578][20347] Updated weights for policy 0, policy_version 1687388 (0.0011) [2023-12-27 17:20:06,638][20381] Updated weights for policy 1, policy_version 1685715 (0.0007) [2023-12-27 17:20:06,641][20347] Updated weights for policy 0, policy_version 1687398 (0.0011) [2023-12-27 17:20:06,700][20381] Updated weights for policy 1, policy_version 1685725 (0.0007) [2023-12-27 17:20:06,754][20381] Updated weights for policy 1, policy_version 1685735 (0.0009) [2023-12-27 17:20:07,319][20347] Updated weights for policy 0, policy_version 1687408 (0.0010) [2023-12-27 17:20:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 863649792. Throughput: 0: 10080.3, 1: 10325.1. Samples: 863624512. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:07,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:20:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001685744_431595520.pth... [2023-12-27 17:20:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001684560_431292416.pth [2023-12-27 17:20:07,375][20347] Updated weights for policy 0, policy_version 1687418 (0.0010) [2023-12-27 17:20:07,391][20381] Updated weights for policy 1, policy_version 1685745 (0.0010) [2023-12-27 17:20:07,432][20347] Updated weights for policy 0, policy_version 1687428 (0.0011) [2023-12-27 17:20:07,446][20381] Updated weights for policy 1, policy_version 1685755 (0.0007) [2023-12-27 17:20:07,450][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001687432_432062464.pth... [2023-12-27 17:20:07,452][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001686248_431759360.pth [2023-12-27 17:20:07,496][20381] Updated weights for policy 1, policy_version 1685765 (0.0005) [2023-12-27 17:20:07,553][20381] Updated weights for policy 1, policy_version 1685775 (0.0006) [2023-12-27 17:20:08,109][20381] Updated weights for policy 1, policy_version 1685785 (0.0005) [2023-12-27 17:20:08,114][20347] Updated weights for policy 0, policy_version 1687438 (0.0010) [2023-12-27 17:20:08,165][20381] Updated weights for policy 1, policy_version 1685795 (0.0005) [2023-12-27 17:20:08,169][20347] Updated weights for policy 0, policy_version 1687448 (0.0010) [2023-12-27 17:20:08,224][20381] Updated weights for policy 1, policy_version 1685805 (0.0005) [2023-12-27 17:20:08,227][20347] Updated weights for policy 0, policy_version 1687458 (0.0010) [2023-12-27 17:20:08,845][20381] Updated weights for policy 1, policy_version 1685815 (0.0009) [2023-12-27 17:20:08,904][20381] Updated weights for policy 1, policy_version 1685825 (0.0011) [2023-12-27 17:20:08,913][20347] Updated weights for policy 0, policy_version 1687468 (0.0008) [2023-12-27 17:20:08,956][20381] Updated weights for policy 1, policy_version 1685835 (0.0011) [2023-12-27 17:20:08,964][20347] Updated weights for policy 0, policy_version 1687478 (0.0005) [2023-12-27 17:20:09,015][20347] Updated weights for policy 0, policy_version 1687488 (0.0005) [2023-12-27 17:20:09,666][20381] Updated weights for policy 1, policy_version 1685845 (0.0009) [2023-12-27 17:20:09,727][20381] Updated weights for policy 1, policy_version 1685855 (0.0008) [2023-12-27 17:20:09,744][20347] Updated weights for policy 0, policy_version 1687498 (0.0007) [2023-12-27 17:20:09,784][20381] Updated weights for policy 1, policy_version 1685865 (0.0005) [2023-12-27 17:20:09,801][20347] Updated weights for policy 0, policy_version 1687508 (0.0006) [2023-12-27 17:20:09,860][20347] Updated weights for policy 0, policy_version 1687518 (0.0008) [2023-12-27 17:20:09,909][20347] Updated weights for policy 0, policy_version 1687528 (0.0008) [2023-12-27 17:20:10,504][20381] Updated weights for policy 1, policy_version 1685875 (0.0010) [2023-12-27 17:20:10,555][20381] Updated weights for policy 1, policy_version 1685885 (0.0008) [2023-12-27 17:20:10,600][20381] Updated weights for policy 1, policy_version 1685895 (0.0007) [2023-12-27 17:20:10,603][20347] Updated weights for policy 0, policy_version 1687538 (0.0007) [2023-12-27 17:20:10,660][20347] Updated weights for policy 0, policy_version 1687548 (0.0007) [2023-12-27 17:20:10,721][20347] Updated weights for policy 0, policy_version 1687558 (0.0008) [2023-12-27 17:20:11,310][20381] Updated weights for policy 1, policy_version 1685905 (0.0007) [2023-12-27 17:20:11,376][20381] Updated weights for policy 1, policy_version 1685915 (0.0007) [2023-12-27 17:20:11,431][20381] Updated weights for policy 1, policy_version 1685925 (0.0006) [2023-12-27 17:20:11,477][20381] Updated weights for policy 1, policy_version 1685935 (0.0007) [2023-12-27 17:20:11,523][20347] Updated weights for policy 0, policy_version 1687568 (0.0008) [2023-12-27 17:20:11,577][20347] Updated weights for policy 0, policy_version 1687578 (0.0008) [2023-12-27 17:20:11,645][20347] Updated weights for policy 0, policy_version 1687588 (0.0007) [2023-12-27 17:20:12,221][20381] Updated weights for policy 1, policy_version 1685945 (0.0008) [2023-12-27 17:20:12,279][20381] Updated weights for policy 1, policy_version 1685955 (0.0008) [2023-12-27 17:20:12,300][20347] Updated weights for policy 0, policy_version 1687598 (0.0007) [2023-12-27 17:20:12,342][20381] Updated weights for policy 1, policy_version 1685965 (0.0008) [2023-12-27 17:20:12,358][20347] Updated weights for policy 0, policy_version 1687608 (0.0009) [2023-12-27 17:20:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 863756288. Throughput: 0: 10078.5, 1: 10330.3. Samples: 863747376. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:12,363][19170] Avg episode reward: [(0, '980.640'), (1, '1000.000')] [2023-12-27 17:20:12,425][20347] Updated weights for policy 0, policy_version 1687618 (0.0010) [2023-12-27 17:20:13,038][20347] Updated weights for policy 0, policy_version 1687628 (0.0010) [2023-12-27 17:20:13,074][20381] Updated weights for policy 1, policy_version 1685975 (0.0006) [2023-12-27 17:20:13,084][20347] Updated weights for policy 0, policy_version 1687638 (0.0010) [2023-12-27 17:20:13,127][20381] Updated weights for policy 1, policy_version 1685985 (0.0005) [2023-12-27 17:20:13,133][20347] Updated weights for policy 0, policy_version 1687648 (0.0010) [2023-12-27 17:20:13,176][20381] Updated weights for policy 1, policy_version 1685995 (0.0006) [2023-12-27 17:20:13,776][20381] Updated weights for policy 1, policy_version 1686005 (0.0009) [2023-12-27 17:20:13,799][20347] Updated weights for policy 0, policy_version 1687658 (0.0009) [2023-12-27 17:20:13,828][20381] Updated weights for policy 1, policy_version 1686015 (0.0011) [2023-12-27 17:20:13,850][20347] Updated weights for policy 0, policy_version 1687668 (0.0008) [2023-12-27 17:20:13,883][20381] Updated weights for policy 1, policy_version 1686025 (0.0011) [2023-12-27 17:20:13,911][20347] Updated weights for policy 0, policy_version 1687678 (0.0010) [2023-12-27 17:20:13,975][20347] Updated weights for policy 0, policy_version 1687688 (0.0009) [2023-12-27 17:20:14,575][20381] Updated weights for policy 1, policy_version 1686035 (0.0011) [2023-12-27 17:20:14,587][20347] Updated weights for policy 0, policy_version 1687698 (0.0008) [2023-12-27 17:20:14,630][20347] Updated weights for policy 0, policy_version 1687708 (0.0008) [2023-12-27 17:20:14,631][20381] Updated weights for policy 1, policy_version 1686045 (0.0006) [2023-12-27 17:20:14,679][20381] Updated weights for policy 1, policy_version 1686055 (0.0006) [2023-12-27 17:20:14,687][20347] Updated weights for policy 0, policy_version 1687718 (0.0008) [2023-12-27 17:20:15,305][20381] Updated weights for policy 1, policy_version 1686065 (0.0005) [2023-12-27 17:20:15,366][20381] Updated weights for policy 1, policy_version 1686075 (0.0009) [2023-12-27 17:20:15,424][20381] Updated weights for policy 1, policy_version 1686085 (0.0010) [2023-12-27 17:20:15,476][20347] Updated weights for policy 0, policy_version 1687728 (0.0009) [2023-12-27 17:20:15,478][20381] Updated weights for policy 1, policy_version 1686095 (0.0011) [2023-12-27 17:20:15,531][20347] Updated weights for policy 0, policy_version 1687738 (0.0010) [2023-12-27 17:20:15,588][20347] Updated weights for policy 0, policy_version 1687748 (0.0010) [2023-12-27 17:20:16,117][20381] Updated weights for policy 1, policy_version 1686105 (0.0010) [2023-12-27 17:20:16,176][20347] Updated weights for policy 0, policy_version 1687758 (0.0010) [2023-12-27 17:20:16,177][20381] Updated weights for policy 1, policy_version 1686115 (0.0007) [2023-12-27 17:20:16,231][20347] Updated weights for policy 0, policy_version 1687768 (0.0010) [2023-12-27 17:20:16,233][20381] Updated weights for policy 1, policy_version 1686125 (0.0005) [2023-12-27 17:20:16,282][20347] Updated weights for policy 0, policy_version 1687778 (0.0010) [2023-12-27 17:20:16,830][20381] Updated weights for policy 1, policy_version 1686135 (0.0006) [2023-12-27 17:20:16,869][20347] Updated weights for policy 0, policy_version 1687788 (0.0010) [2023-12-27 17:20:16,878][20381] Updated weights for policy 1, policy_version 1686145 (0.0005) [2023-12-27 17:20:16,920][20347] Updated weights for policy 0, policy_version 1687798 (0.0010) [2023-12-27 17:20:16,927][20381] Updated weights for policy 1, policy_version 1686155 (0.0005) [2023-12-27 17:20:16,971][20347] Updated weights for policy 0, policy_version 1687808 (0.0010) [2023-12-27 17:20:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 863862784. Throughput: 0: 10102.8, 1: 10392.5. Samples: 863873040. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:17,363][19170] Avg episode reward: [(0, '980.640'), (1, '990.450')] [2023-12-27 17:20:17,574][20381] Updated weights for policy 1, policy_version 1686165 (0.0007) [2023-12-27 17:20:17,621][20381] Updated weights for policy 1, policy_version 1686175 (0.0007) [2023-12-27 17:20:17,623][20347] Updated weights for policy 0, policy_version 1687818 (0.0009) [2023-12-27 17:20:17,666][20381] Updated weights for policy 1, policy_version 1686185 (0.0005) [2023-12-27 17:20:17,675][20347] Updated weights for policy 0, policy_version 1687828 (0.0007) [2023-12-27 17:20:17,738][20347] Updated weights for policy 0, policy_version 1687838 (0.0005) [2023-12-27 17:20:17,787][20347] Updated weights for policy 0, policy_version 1687848 (0.0005) [2023-12-27 17:20:18,408][20381] Updated weights for policy 1, policy_version 1686195 (0.0006) [2023-12-27 17:20:18,440][20347] Updated weights for policy 0, policy_version 1687858 (0.0007) [2023-12-27 17:20:18,458][20381] Updated weights for policy 1, policy_version 1686205 (0.0006) [2023-12-27 17:20:18,497][20347] Updated weights for policy 0, policy_version 1687868 (0.0007) [2023-12-27 17:20:18,514][20381] Updated weights for policy 1, policy_version 1686215 (0.0006) [2023-12-27 17:20:18,550][20347] Updated weights for policy 0, policy_version 1687878 (0.0005) [2023-12-27 17:20:19,229][20347] Updated weights for policy 0, policy_version 1687888 (0.0007) [2023-12-27 17:20:19,235][20381] Updated weights for policy 1, policy_version 1686225 (0.0008) [2023-12-27 17:20:19,289][20347] Updated weights for policy 0, policy_version 1687898 (0.0008) [2023-12-27 17:20:19,291][20381] Updated weights for policy 1, policy_version 1686235 (0.0007) [2023-12-27 17:20:19,348][20347] Updated weights for policy 0, policy_version 1687908 (0.0008) [2023-12-27 17:20:19,350][20381] Updated weights for policy 1, policy_version 1686245 (0.0007) [2023-12-27 17:20:19,420][20381] Updated weights for policy 1, policy_version 1686255 (0.0008) [2023-12-27 17:20:20,107][20347] Updated weights for policy 0, policy_version 1687918 (0.0007) [2023-12-27 17:20:20,173][20347] Updated weights for policy 0, policy_version 1687928 (0.0006) [2023-12-27 17:20:20,175][20381] Updated weights for policy 1, policy_version 1686265 (0.0008) [2023-12-27 17:20:20,235][20381] Updated weights for policy 1, policy_version 1686275 (0.0007) [2023-12-27 17:20:20,241][20347] Updated weights for policy 0, policy_version 1687938 (0.0006) [2023-12-27 17:20:20,293][20381] Updated weights for policy 1, policy_version 1686285 (0.0007) [2023-12-27 17:20:20,862][20347] Updated weights for policy 0, policy_version 1687948 (0.0005) [2023-12-27 17:20:20,923][20347] Updated weights for policy 0, policy_version 1687958 (0.0006) [2023-12-27 17:20:20,986][20347] Updated weights for policy 0, policy_version 1687968 (0.0006) [2023-12-27 17:20:21,045][20381] Updated weights for policy 1, policy_version 1686295 (0.0008) [2023-12-27 17:20:21,110][20381] Updated weights for policy 1, policy_version 1686305 (0.0009) [2023-12-27 17:20:21,180][20381] Updated weights for policy 1, policy_version 1686315 (0.0007) [2023-12-27 17:20:21,633][20347] Updated weights for policy 0, policy_version 1687978 (0.0007) [2023-12-27 17:20:21,685][20347] Updated weights for policy 0, policy_version 1687988 (0.0008) [2023-12-27 17:20:21,749][20347] Updated weights for policy 0, policy_version 1687998 (0.0008) [2023-12-27 17:20:21,801][20347] Updated weights for policy 0, policy_version 1688008 (0.0007) [2023-12-27 17:20:21,872][20381] Updated weights for policy 1, policy_version 1686325 (0.0007) [2023-12-27 17:20:21,931][20381] Updated weights for policy 1, policy_version 1686335 (0.0008) [2023-12-27 17:20:21,983][20381] Updated weights for policy 1, policy_version 1686345 (0.0008) [2023-12-27 17:20:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 863961088. Throughput: 0: 10064.2, 1: 10348.5. Samples: 863933220. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.450')] [2023-12-27 17:20:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001686352_431751168.pth... [2023-12-27 17:20:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001688008_432209920.pth... [2023-12-27 17:20:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001685168_431448064.pth [2023-12-27 17:20:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001686824_431906816.pth [2023-12-27 17:20:22,493][20347] Updated weights for policy 0, policy_version 1688018 (0.0007) [2023-12-27 17:20:22,564][20347] Updated weights for policy 0, policy_version 1688028 (0.0007) [2023-12-27 17:20:22,626][20347] Updated weights for policy 0, policy_version 1688038 (0.0005) [2023-12-27 17:20:22,736][20381] Updated weights for policy 1, policy_version 1686355 (0.0008) [2023-12-27 17:20:22,786][20381] Updated weights for policy 1, policy_version 1686365 (0.0008) [2023-12-27 17:20:22,838][20381] Updated weights for policy 1, policy_version 1686375 (0.0008) [2023-12-27 17:20:23,274][20347] Updated weights for policy 0, policy_version 1688048 (0.0008) [2023-12-27 17:20:23,337][20347] Updated weights for policy 0, policy_version 1688058 (0.0006) [2023-12-27 17:20:23,387][20347] Updated weights for policy 0, policy_version 1688068 (0.0006) [2023-12-27 17:20:23,494][20381] Updated weights for policy 1, policy_version 1686385 (0.0007) [2023-12-27 17:20:23,550][20381] Updated weights for policy 1, policy_version 1686395 (0.0006) [2023-12-27 17:20:23,606][20381] Updated weights for policy 1, policy_version 1686405 (0.0007) [2023-12-27 17:20:23,665][20381] Updated weights for policy 1, policy_version 1686415 (0.0008) [2023-12-27 17:20:24,019][20347] Updated weights for policy 0, policy_version 1688078 (0.0005) [2023-12-27 17:20:24,075][20347] Updated weights for policy 0, policy_version 1688088 (0.0005) [2023-12-27 17:20:24,141][20347] Updated weights for policy 0, policy_version 1688098 (0.0005) [2023-12-27 17:20:24,252][20381] Updated weights for policy 1, policy_version 1686425 (0.0008) [2023-12-27 17:20:24,307][20381] Updated weights for policy 1, policy_version 1686435 (0.0008) [2023-12-27 17:20:24,366][20381] Updated weights for policy 1, policy_version 1686445 (0.0007) [2023-12-27 17:20:24,703][20347] Updated weights for policy 0, policy_version 1688108 (0.0007) [2023-12-27 17:20:24,753][20347] Updated weights for policy 0, policy_version 1688118 (0.0008) [2023-12-27 17:20:24,802][20347] Updated weights for policy 0, policy_version 1688128 (0.0007) [2023-12-27 17:20:25,019][20381] Updated weights for policy 1, policy_version 1686455 (0.0006) [2023-12-27 17:20:25,074][20381] Updated weights for policy 1, policy_version 1686465 (0.0005) [2023-12-27 17:20:25,137][20381] Updated weights for policy 1, policy_version 1686475 (0.0005) [2023-12-27 17:20:25,545][20347] Updated weights for policy 0, policy_version 1688138 (0.0008) [2023-12-27 17:20:25,608][20347] Updated weights for policy 0, policy_version 1688148 (0.0007) [2023-12-27 17:20:25,660][20347] Updated weights for policy 0, policy_version 1688158 (0.0007) [2023-12-27 17:20:25,713][20347] Updated weights for policy 0, policy_version 1688168 (0.0007) [2023-12-27 17:20:25,723][20381] Updated weights for policy 1, policy_version 1686485 (0.0006) [2023-12-27 17:20:25,775][20381] Updated weights for policy 1, policy_version 1686495 (0.0008) [2023-12-27 17:20:25,832][20381] Updated weights for policy 1, policy_version 1686505 (0.0007) [2023-12-27 17:20:26,287][20347] Updated weights for policy 0, policy_version 1688178 (0.0006) [2023-12-27 17:20:26,333][20347] Updated weights for policy 0, policy_version 1688188 (0.0005) [2023-12-27 17:20:26,386][20347] Updated weights for policy 0, policy_version 1688198 (0.0005) [2023-12-27 17:20:26,589][20381] Updated weights for policy 1, policy_version 1686515 (0.0007) [2023-12-27 17:20:26,634][20381] Updated weights for policy 1, policy_version 1686525 (0.0007) [2023-12-27 17:20:26,679][20381] Updated weights for policy 1, policy_version 1686535 (0.0008) [2023-12-27 17:20:27,045][20347] Updated weights for policy 0, policy_version 1688208 (0.0007) [2023-12-27 17:20:27,090][20347] Updated weights for policy 0, policy_version 1688218 (0.0007) [2023-12-27 17:20:27,140][20347] Updated weights for policy 0, policy_version 1688228 (0.0007) [2023-12-27 17:20:27,349][20381] Updated weights for policy 1, policy_version 1686545 (0.0007) [2023-12-27 17:20:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 864067584. Throughput: 0: 10129.1, 1: 10286.2. Samples: 864056792. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:20:27,409][20381] Updated weights for policy 1, policy_version 1686555 (0.0005) [2023-12-27 17:20:27,467][20381] Updated weights for policy 1, policy_version 1686565 (0.0005) [2023-12-27 17:20:27,522][20381] Updated weights for policy 1, policy_version 1686575 (0.0005) [2023-12-27 17:20:27,833][20347] Updated weights for policy 0, policy_version 1688238 (0.0008) [2023-12-27 17:20:27,888][20347] Updated weights for policy 0, policy_version 1688248 (0.0008) [2023-12-27 17:20:27,946][20347] Updated weights for policy 0, policy_version 1688258 (0.0007) [2023-12-27 17:20:28,099][20381] Updated weights for policy 1, policy_version 1686585 (0.0007) [2023-12-27 17:20:28,153][20381] Updated weights for policy 1, policy_version 1686595 (0.0006) [2023-12-27 17:20:28,214][20381] Updated weights for policy 1, policy_version 1686605 (0.0005) [2023-12-27 17:20:28,612][20347] Updated weights for policy 0, policy_version 1688268 (0.0007) [2023-12-27 17:20:28,669][20347] Updated weights for policy 0, policy_version 1688278 (0.0007) [2023-12-27 17:20:28,720][20347] Updated weights for policy 0, policy_version 1688288 (0.0007) [2023-12-27 17:20:28,861][20381] Updated weights for policy 1, policy_version 1686615 (0.0008) [2023-12-27 17:20:28,925][20381] Updated weights for policy 1, policy_version 1686625 (0.0007) [2023-12-27 17:20:28,978][20381] Updated weights for policy 1, policy_version 1686635 (0.0005) [2023-12-27 17:20:29,393][20347] Updated weights for policy 0, policy_version 1688298 (0.0005) [2023-12-27 17:20:29,443][20347] Updated weights for policy 0, policy_version 1688308 (0.0006) [2023-12-27 17:20:29,505][20347] Updated weights for policy 0, policy_version 1688318 (0.0005) [2023-12-27 17:20:29,556][20347] Updated weights for policy 0, policy_version 1688328 (0.0007) [2023-12-27 17:20:29,639][20381] Updated weights for policy 1, policy_version 1686645 (0.0009) [2023-12-27 17:20:29,699][20381] Updated weights for policy 1, policy_version 1686655 (0.0011) [2023-12-27 17:20:29,759][20381] Updated weights for policy 1, policy_version 1686665 (0.0011) [2023-12-27 17:20:30,243][20347] Updated weights for policy 0, policy_version 1688338 (0.0007) [2023-12-27 17:20:30,298][20347] Updated weights for policy 0, policy_version 1688348 (0.0007) [2023-12-27 17:20:30,356][20347] Updated weights for policy 0, policy_version 1688358 (0.0008) [2023-12-27 17:20:30,430][20381] Updated weights for policy 1, policy_version 1686675 (0.0011) [2023-12-27 17:20:30,490][20381] Updated weights for policy 1, policy_version 1686685 (0.0011) [2023-12-27 17:20:30,546][20381] Updated weights for policy 1, policy_version 1686695 (0.0006) [2023-12-27 17:20:31,093][20347] Updated weights for policy 0, policy_version 1688368 (0.0008) [2023-12-27 17:20:31,140][20381] Updated weights for policy 1, policy_version 1686705 (0.0006) [2023-12-27 17:20:31,153][20347] Updated weights for policy 0, policy_version 1688378 (0.0006) [2023-12-27 17:20:31,199][20381] Updated weights for policy 1, policy_version 1686715 (0.0007) [2023-12-27 17:20:31,213][20347] Updated weights for policy 0, policy_version 1688388 (0.0006) [2023-12-27 17:20:31,258][20381] Updated weights for policy 1, policy_version 1686725 (0.0009) [2023-12-27 17:20:31,316][20381] Updated weights for policy 1, policy_version 1686735 (0.0007) [2023-12-27 17:20:31,890][20347] Updated weights for policy 0, policy_version 1688398 (0.0007) [2023-12-27 17:20:31,945][20347] Updated weights for policy 0, policy_version 1688408 (0.0007) [2023-12-27 17:20:31,999][20347] Updated weights for policy 0, policy_version 1688418 (0.0007) [2023-12-27 17:20:32,045][20381] Updated weights for policy 1, policy_version 1686745 (0.0008) [2023-12-27 17:20:32,091][20381] Updated weights for policy 1, policy_version 1686755 (0.0008) [2023-12-27 17:20:32,141][20381] Updated weights for policy 1, policy_version 1686765 (0.0008) [2023-12-27 17:20:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 864174080. Throughput: 0: 10199.7, 1: 10286.7. Samples: 864180800. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-12-27 17:20:32,363][19170] Avg episode reward: [(0, '970.730'), (1, '1000.000')] [2023-12-27 17:20:32,730][20347] Updated weights for policy 0, policy_version 1688428 (0.0007) [2023-12-27 17:20:32,785][20347] Updated weights for policy 0, policy_version 1688438 (0.0008) [2023-12-27 17:20:32,812][20381] Updated weights for policy 1, policy_version 1686775 (0.0008) [2023-12-27 17:20:32,833][20347] Updated weights for policy 0, policy_version 1688448 (0.0007) [2023-12-27 17:20:32,874][20381] Updated weights for policy 1, policy_version 1686785 (0.0007) [2023-12-27 17:20:32,937][20381] Updated weights for policy 1, policy_version 1686795 (0.0008) [2023-12-27 17:20:33,512][20347] Updated weights for policy 0, policy_version 1688458 (0.0005) [2023-12-27 17:20:33,546][20381] Updated weights for policy 1, policy_version 1686805 (0.0007) [2023-12-27 17:20:33,559][20347] Updated weights for policy 0, policy_version 1688468 (0.0006) [2023-12-27 17:20:33,603][20381] Updated weights for policy 1, policy_version 1686815 (0.0007) [2023-12-27 17:20:33,613][20347] Updated weights for policy 0, policy_version 1688478 (0.0006) [2023-12-27 17:20:33,657][20381] Updated weights for policy 1, policy_version 1686825 (0.0007) [2023-12-27 17:20:33,674][20347] Updated weights for policy 0, policy_version 1688488 (0.0006) [2023-12-27 17:20:34,284][20347] Updated weights for policy 0, policy_version 1688498 (0.0007) [2023-12-27 17:20:34,337][20347] Updated weights for policy 0, policy_version 1688508 (0.0008) [2023-12-27 17:20:34,372][20381] Updated weights for policy 1, policy_version 1686835 (0.0008) [2023-12-27 17:20:34,392][20347] Updated weights for policy 0, policy_version 1688518 (0.0009) [2023-12-27 17:20:34,430][20381] Updated weights for policy 1, policy_version 1686845 (0.0008) [2023-12-27 17:20:34,496][20381] Updated weights for policy 1, policy_version 1686855 (0.0008) [2023-12-27 17:20:35,072][20347] Updated weights for policy 0, policy_version 1688528 (0.0007) [2023-12-27 17:20:35,137][20347] Updated weights for policy 0, policy_version 1688538 (0.0007) [2023-12-27 17:20:35,165][20381] Updated weights for policy 1, policy_version 1686865 (0.0008) [2023-12-27 17:20:35,199][20347] Updated weights for policy 0, policy_version 1688548 (0.0006) [2023-12-27 17:20:35,217][20381] Updated weights for policy 1, policy_version 1686875 (0.0010) [2023-12-27 17:20:35,268][20381] Updated weights for policy 1, policy_version 1686885 (0.0010) [2023-12-27 17:20:35,318][20381] Updated weights for policy 1, policy_version 1686895 (0.0010) [2023-12-27 17:20:35,872][20347] Updated weights for policy 0, policy_version 1688558 (0.0006) [2023-12-27 17:20:35,931][20347] Updated weights for policy 0, policy_version 1688568 (0.0007) [2023-12-27 17:20:35,982][20347] Updated weights for policy 0, policy_version 1688578 (0.0007) [2023-12-27 17:20:36,035][20381] Updated weights for policy 1, policy_version 1686905 (0.0011) [2023-12-27 17:20:36,088][20381] Updated weights for policy 1, policy_version 1686915 (0.0010) [2023-12-27 17:20:36,152][20381] Updated weights for policy 1, policy_version 1686925 (0.0006) [2023-12-27 17:20:36,634][20347] Updated weights for policy 0, policy_version 1688588 (0.0008) [2023-12-27 17:20:36,687][20347] Updated weights for policy 0, policy_version 1688598 (0.0006) [2023-12-27 17:20:36,740][20347] Updated weights for policy 0, policy_version 1688608 (0.0005) [2023-12-27 17:20:36,808][20381] Updated weights for policy 1, policy_version 1686935 (0.0006) [2023-12-27 17:20:36,871][20381] Updated weights for policy 1, policy_version 1686945 (0.0005) [2023-12-27 17:20:36,934][20381] Updated weights for policy 1, policy_version 1686955 (0.0009) [2023-12-27 17:20:37,357][20347] Updated weights for policy 0, policy_version 1688618 (0.0006) [2023-12-27 17:20:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 864272384. Throughput: 0: 10233.1, 1: 10287.5. Samples: 864243520. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:20:37,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001686960_431906816.pth... [2023-12-27 17:20:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001685744_431595520.pth [2023-12-27 17:20:37,412][20347] Updated weights for policy 0, policy_version 1688628 (0.0010) [2023-12-27 17:20:37,464][20347] Updated weights for policy 0, policy_version 1688638 (0.0010) [2023-12-27 17:20:37,518][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001688648_432373760.pth... [2023-12-27 17:20:37,520][20347] Updated weights for policy 0, policy_version 1688648 (0.0010) [2023-12-27 17:20:37,521][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001687432_432062464.pth [2023-12-27 17:20:37,545][20381] Updated weights for policy 1, policy_version 1686965 (0.0011) [2023-12-27 17:20:37,597][20381] Updated weights for policy 1, policy_version 1686975 (0.0008) [2023-12-27 17:20:37,657][20381] Updated weights for policy 1, policy_version 1686985 (0.0010) [2023-12-27 17:20:38,196][20347] Updated weights for policy 0, policy_version 1688658 (0.0007) [2023-12-27 17:20:38,252][20347] Updated weights for policy 0, policy_version 1688668 (0.0007) [2023-12-27 17:20:38,299][20347] Updated weights for policy 0, policy_version 1688678 (0.0008) [2023-12-27 17:20:38,350][20381] Updated weights for policy 1, policy_version 1686995 (0.0010) [2023-12-27 17:20:38,411][20381] Updated weights for policy 1, policy_version 1687005 (0.0008) [2023-12-27 17:20:38,471][20381] Updated weights for policy 1, policy_version 1687015 (0.0006) [2023-12-27 17:20:39,053][20347] Updated weights for policy 0, policy_version 1688688 (0.0008) [2023-12-27 17:20:39,114][20347] Updated weights for policy 0, policy_version 1688698 (0.0007) [2023-12-27 17:20:39,153][20381] Updated weights for policy 1, policy_version 1687025 (0.0006) [2023-12-27 17:20:39,166][20347] Updated weights for policy 0, policy_version 1688708 (0.0009) [2023-12-27 17:20:39,212][20381] Updated weights for policy 1, policy_version 1687035 (0.0006) [2023-12-27 17:20:39,275][20381] Updated weights for policy 1, policy_version 1687045 (0.0007) [2023-12-27 17:20:39,333][20381] Updated weights for policy 1, policy_version 1687055 (0.0008) [2023-12-27 17:20:39,866][20347] Updated weights for policy 0, policy_version 1688718 (0.0008) [2023-12-27 17:20:39,929][20347] Updated weights for policy 0, policy_version 1688728 (0.0008) [2023-12-27 17:20:39,991][20347] Updated weights for policy 0, policy_version 1688738 (0.0008) [2023-12-27 17:20:40,085][20381] Updated weights for policy 1, policy_version 1687065 (0.0008) [2023-12-27 17:20:40,145][20381] Updated weights for policy 1, policy_version 1687075 (0.0007) [2023-12-27 17:20:40,205][20381] Updated weights for policy 1, policy_version 1687085 (0.0007) [2023-12-27 17:20:40,736][20347] Updated weights for policy 0, policy_version 1688748 (0.0008) [2023-12-27 17:20:40,800][20347] Updated weights for policy 0, policy_version 1688758 (0.0009) [2023-12-27 17:20:40,857][20347] Updated weights for policy 0, policy_version 1688768 (0.0007) [2023-12-27 17:20:40,869][20381] Updated weights for policy 1, policy_version 1687095 (0.0006) [2023-12-27 17:20:40,928][20381] Updated weights for policy 1, policy_version 1687105 (0.0008) [2023-12-27 17:20:40,986][20381] Updated weights for policy 1, policy_version 1687115 (0.0006) [2023-12-27 17:20:41,600][20347] Updated weights for policy 0, policy_version 1688778 (0.0008) [2023-12-27 17:20:41,669][20347] Updated weights for policy 0, policy_version 1688788 (0.0008) [2023-12-27 17:20:41,720][20381] Updated weights for policy 1, policy_version 1687125 (0.0007) [2023-12-27 17:20:41,734][20347] Updated weights for policy 0, policy_version 1688798 (0.0008) [2023-12-27 17:20:41,783][20381] Updated weights for policy 1, policy_version 1687135 (0.0008) [2023-12-27 17:20:41,792][20347] Updated weights for policy 0, policy_version 1688808 (0.0008) [2023-12-27 17:20:41,837][20381] Updated weights for policy 1, policy_version 1687145 (0.0006) [2023-12-27 17:20:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 864370688. Throughput: 0: 10186.2, 1: 10270.2. Samples: 864364212. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:20:42,363][19170] Avg episode reward: [(0, '960.870'), (1, '990.260')] [2023-12-27 17:20:42,457][20347] Updated weights for policy 0, policy_version 1688818 (0.0007) [2023-12-27 17:20:42,518][20347] Updated weights for policy 0, policy_version 1688828 (0.0008) [2023-12-27 17:20:42,578][20347] Updated weights for policy 0, policy_version 1688838 (0.0008) [2023-12-27 17:20:42,612][20381] Updated weights for policy 1, policy_version 1687155 (0.0007) [2023-12-27 17:20:42,669][20381] Updated weights for policy 1, policy_version 1687165 (0.0008) [2023-12-27 17:20:42,726][20381] Updated weights for policy 1, policy_version 1687175 (0.0007) [2023-12-27 17:20:43,274][20347] Updated weights for policy 0, policy_version 1688848 (0.0006) [2023-12-27 17:20:43,326][20347] Updated weights for policy 0, policy_version 1688858 (0.0008) [2023-12-27 17:20:43,345][20381] Updated weights for policy 1, policy_version 1687185 (0.0008) [2023-12-27 17:20:43,383][20347] Updated weights for policy 0, policy_version 1688868 (0.0007) [2023-12-27 17:20:43,401][20381] Updated weights for policy 1, policy_version 1687195 (0.0005) [2023-12-27 17:20:43,452][20381] Updated weights for policy 1, policy_version 1687205 (0.0005) [2023-12-27 17:20:43,504][20381] Updated weights for policy 1, policy_version 1687215 (0.0008) [2023-12-27 17:20:44,030][20347] Updated weights for policy 0, policy_version 1688878 (0.0006) [2023-12-27 17:20:44,097][20347] Updated weights for policy 0, policy_version 1688888 (0.0006) [2023-12-27 17:20:44,160][20347] Updated weights for policy 0, policy_version 1688898 (0.0009) [2023-12-27 17:20:44,198][20381] Updated weights for policy 1, policy_version 1687225 (0.0006) [2023-12-27 17:20:44,251][20381] Updated weights for policy 1, policy_version 1687235 (0.0009) [2023-12-27 17:20:44,308][20381] Updated weights for policy 1, policy_version 1687245 (0.0008) [2023-12-27 17:20:44,824][20347] Updated weights for policy 0, policy_version 1688908 (0.0009) [2023-12-27 17:20:44,887][20347] Updated weights for policy 0, policy_version 1688918 (0.0008) [2023-12-27 17:20:44,943][20347] Updated weights for policy 0, policy_version 1688928 (0.0008) [2023-12-27 17:20:44,991][20381] Updated weights for policy 1, policy_version 1687255 (0.0008) [2023-12-27 17:20:45,047][20381] Updated weights for policy 1, policy_version 1687265 (0.0006) [2023-12-27 17:20:45,105][20381] Updated weights for policy 1, policy_version 1687275 (0.0005) [2023-12-27 17:20:45,673][20347] Updated weights for policy 0, policy_version 1688938 (0.0007) [2023-12-27 17:20:45,690][20381] Updated weights for policy 1, policy_version 1687285 (0.0006) [2023-12-27 17:20:45,727][20347] Updated weights for policy 0, policy_version 1688948 (0.0007) [2023-12-27 17:20:45,759][20381] Updated weights for policy 1, policy_version 1687295 (0.0008) [2023-12-27 17:20:45,774][20347] Updated weights for policy 0, policy_version 1688958 (0.0005) [2023-12-27 17:20:45,820][20381] Updated weights for policy 1, policy_version 1687305 (0.0006) [2023-12-27 17:20:45,825][20347] Updated weights for policy 0, policy_version 1688968 (0.0006) [2023-12-27 17:20:46,505][20347] Updated weights for policy 0, policy_version 1688978 (0.0007) [2023-12-27 17:20:46,511][20381] Updated weights for policy 1, policy_version 1687315 (0.0007) [2023-12-27 17:20:46,561][20347] Updated weights for policy 0, policy_version 1688988 (0.0006) [2023-12-27 17:20:46,566][20381] Updated weights for policy 1, policy_version 1687325 (0.0005) [2023-12-27 17:20:46,615][20347] Updated weights for policy 0, policy_version 1688998 (0.0006) [2023-12-27 17:20:46,617][20381] Updated weights for policy 1, policy_version 1687335 (0.0005) [2023-12-27 17:20:47,304][20347] Updated weights for policy 0, policy_version 1689008 (0.0008) [2023-12-27 17:20:47,308][20381] Updated weights for policy 1, policy_version 1687345 (0.0007) [2023-12-27 17:20:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 864468992. Throughput: 0: 10213.3, 1: 10294.4. Samples: 864485468. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:20:47,363][20381] Updated weights for policy 1, policy_version 1687355 (0.0007) [2023-12-27 17:20:47,363][19170] Avg episode reward: [(0, '960.900'), (1, '990.260')] [2023-12-27 17:20:47,364][20347] Updated weights for policy 0, policy_version 1689018 (0.0007) [2023-12-27 17:20:47,417][20347] Updated weights for policy 0, policy_version 1689028 (0.0006) [2023-12-27 17:20:47,420][20381] Updated weights for policy 1, policy_version 1687365 (0.0006) [2023-12-27 17:20:47,480][20381] Updated weights for policy 1, policy_version 1687375 (0.0006) [2023-12-27 17:20:48,033][20381] Updated weights for policy 1, policy_version 1687385 (0.0007) [2023-12-27 17:20:48,091][20381] Updated weights for policy 1, policy_version 1687395 (0.0008) [2023-12-27 17:20:48,139][20381] Updated weights for policy 1, policy_version 1687405 (0.0007) [2023-12-27 17:20:48,155][20347] Updated weights for policy 0, policy_version 1689038 (0.0007) [2023-12-27 17:20:48,209][20347] Updated weights for policy 0, policy_version 1689048 (0.0007) [2023-12-27 17:20:48,259][20347] Updated weights for policy 0, policy_version 1689058 (0.0007) [2023-12-27 17:20:48,823][20381] Updated weights for policy 1, policy_version 1687415 (0.0007) [2023-12-27 17:20:48,878][20381] Updated weights for policy 1, policy_version 1687425 (0.0007) [2023-12-27 17:20:48,930][20381] Updated weights for policy 1, policy_version 1687435 (0.0006) [2023-12-27 17:20:48,955][20347] Updated weights for policy 0, policy_version 1689068 (0.0008) [2023-12-27 17:20:49,014][20347] Updated weights for policy 0, policy_version 1689078 (0.0010) [2023-12-27 17:20:49,075][20347] Updated weights for policy 0, policy_version 1689088 (0.0010) [2023-12-27 17:20:49,669][20381] Updated weights for policy 1, policy_version 1687445 (0.0008) [2023-12-27 17:20:49,721][20347] Updated weights for policy 0, policy_version 1689098 (0.0010) [2023-12-27 17:20:49,726][20381] Updated weights for policy 1, policy_version 1687455 (0.0011) [2023-12-27 17:20:49,774][20347] Updated weights for policy 0, policy_version 1689108 (0.0010) [2023-12-27 17:20:49,782][20381] Updated weights for policy 1, policy_version 1687465 (0.0006) [2023-12-27 17:20:49,830][20347] Updated weights for policy 0, policy_version 1689118 (0.0009) [2023-12-27 17:20:49,888][20347] Updated weights for policy 0, policy_version 1689128 (0.0006) [2023-12-27 17:20:50,420][20381] Updated weights for policy 1, policy_version 1687475 (0.0006) [2023-12-27 17:20:50,469][20381] Updated weights for policy 1, policy_version 1687485 (0.0007) [2023-12-27 17:20:50,521][20381] Updated weights for policy 1, policy_version 1687495 (0.0006) [2023-12-27 17:20:50,611][20347] Updated weights for policy 0, policy_version 1689138 (0.0008) [2023-12-27 17:20:50,667][20347] Updated weights for policy 0, policy_version 1689148 (0.0006) [2023-12-27 17:20:50,727][20347] Updated weights for policy 0, policy_version 1689158 (0.0007) [2023-12-27 17:20:51,151][20381] Updated weights for policy 1, policy_version 1687505 (0.0007) [2023-12-27 17:20:51,216][20381] Updated weights for policy 1, policy_version 1687515 (0.0006) [2023-12-27 17:20:51,278][20381] Updated weights for policy 1, policy_version 1687525 (0.0006) [2023-12-27 17:20:51,338][20381] Updated weights for policy 1, policy_version 1687535 (0.0007) [2023-12-27 17:20:51,348][20347] Updated weights for policy 0, policy_version 1689168 (0.0007) [2023-12-27 17:20:51,413][20347] Updated weights for policy 0, policy_version 1689178 (0.0006) [2023-12-27 17:20:51,467][20347] Updated weights for policy 0, policy_version 1689188 (0.0008) [2023-12-27 17:20:52,064][20381] Updated weights for policy 1, policy_version 1687545 (0.0006) [2023-12-27 17:20:52,123][20381] Updated weights for policy 1, policy_version 1687555 (0.0006) [2023-12-27 17:20:52,178][20381] Updated weights for policy 1, policy_version 1687565 (0.0008) [2023-12-27 17:20:52,230][20347] Updated weights for policy 0, policy_version 1689198 (0.0006) [2023-12-27 17:20:52,292][20347] Updated weights for policy 0, policy_version 1689208 (0.0007) [2023-12-27 17:20:52,354][20347] Updated weights for policy 0, policy_version 1689218 (0.0007) [2023-12-27 17:20:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 864575488. Throughput: 0: 10217.9, 1: 10279.6. Samples: 864546900. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:20:52,363][19170] Avg episode reward: [(0, '970.690'), (1, '1000.000')] [2023-12-27 17:20:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001687568_432062464.pth... [2023-12-27 17:20:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001686352_431751168.pth [2023-12-27 17:20:52,388][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001689224_432521216.pth... [2023-12-27 17:20:52,393][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001688008_432209920.pth [2023-12-27 17:20:52,864][20381] Updated weights for policy 1, policy_version 1687575 (0.0005) [2023-12-27 17:20:52,920][20381] Updated weights for policy 1, policy_version 1687585 (0.0005) [2023-12-27 17:20:52,985][20381] Updated weights for policy 1, policy_version 1687595 (0.0005) [2023-12-27 17:20:53,042][20347] Updated weights for policy 0, policy_version 1689228 (0.0006) [2023-12-27 17:20:53,100][20347] Updated weights for policy 0, policy_version 1689238 (0.0005) [2023-12-27 17:20:53,162][20347] Updated weights for policy 0, policy_version 1689248 (0.0006) [2023-12-27 17:20:53,488][20381] Updated weights for policy 1, policy_version 1687605 (0.0008) [2023-12-27 17:20:53,540][20381] Updated weights for policy 1, policy_version 1687615 (0.0010) [2023-12-27 17:20:53,595][20381] Updated weights for policy 1, policy_version 1687625 (0.0011) [2023-12-27 17:20:53,759][20347] Updated weights for policy 0, policy_version 1689258 (0.0006) [2023-12-27 17:20:53,812][20347] Updated weights for policy 0, policy_version 1689268 (0.0007) [2023-12-27 17:20:53,868][20347] Updated weights for policy 0, policy_version 1689278 (0.0007) [2023-12-27 17:20:53,925][20347] Updated weights for policy 0, policy_version 1689288 (0.0007) [2023-12-27 17:20:54,297][20381] Updated weights for policy 1, policy_version 1687635 (0.0011) [2023-12-27 17:20:54,353][20381] Updated weights for policy 1, policy_version 1687645 (0.0011) [2023-12-27 17:20:54,409][20381] Updated weights for policy 1, policy_version 1687655 (0.0008) [2023-12-27 17:20:54,619][20347] Updated weights for policy 0, policy_version 1689298 (0.0007) [2023-12-27 17:20:54,674][20347] Updated weights for policy 0, policy_version 1689308 (0.0007) [2023-12-27 17:20:54,727][20347] Updated weights for policy 0, policy_version 1689318 (0.0007) [2023-12-27 17:20:55,079][20381] Updated weights for policy 1, policy_version 1687665 (0.0005) [2023-12-27 17:20:55,132][20381] Updated weights for policy 1, policy_version 1687675 (0.0007) [2023-12-27 17:20:55,181][20381] Updated weights for policy 1, policy_version 1687685 (0.0007) [2023-12-27 17:20:55,229][20381] Updated weights for policy 1, policy_version 1687695 (0.0008) [2023-12-27 17:20:55,419][20347] Updated weights for policy 0, policy_version 1689328 (0.0005) [2023-12-27 17:20:55,473][20347] Updated weights for policy 0, policy_version 1689338 (0.0007) [2023-12-27 17:20:55,518][20347] Updated weights for policy 0, policy_version 1689348 (0.0008) [2023-12-27 17:20:55,925][20381] Updated weights for policy 1, policy_version 1687705 (0.0006) [2023-12-27 17:20:55,975][20381] Updated weights for policy 1, policy_version 1687715 (0.0005) [2023-12-27 17:20:56,029][20381] Updated weights for policy 1, policy_version 1687725 (0.0007) [2023-12-27 17:20:56,209][20347] Updated weights for policy 0, policy_version 1689358 (0.0006) [2023-12-27 17:20:56,259][20347] Updated weights for policy 0, policy_version 1689368 (0.0006) [2023-12-27 17:20:56,322][20347] Updated weights for policy 0, policy_version 1689378 (0.0008) [2023-12-27 17:20:56,678][20381] Updated weights for policy 1, policy_version 1687735 (0.0007) [2023-12-27 17:20:56,740][20381] Updated weights for policy 1, policy_version 1687745 (0.0008) [2023-12-27 17:20:56,803][20381] Updated weights for policy 1, policy_version 1687756 (0.0009) [2023-12-27 17:20:56,987][20347] Updated weights for policy 0, policy_version 1689388 (0.0007) [2023-12-27 17:20:57,041][20347] Updated weights for policy 0, policy_version 1689398 (0.0009) [2023-12-27 17:20:57,093][20347] Updated weights for policy 0, policy_version 1689408 (0.0007) [2023-12-27 17:20:57,350][20381] Updated weights for policy 1, policy_version 1687766 (0.0008) [2023-12-27 17:20:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 864681984. Throughput: 0: 10234.6, 1: 10297.8. Samples: 864671332. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:20:57,363][19170] Avg episode reward: [(0, '961.050'), (1, '1000.000')] [2023-12-27 17:20:57,408][20381] Updated weights for policy 1, policy_version 1687776 (0.0011) [2023-12-27 17:20:57,461][20381] Updated weights for policy 1, policy_version 1687786 (0.0008) [2023-12-27 17:20:57,724][20347] Updated weights for policy 0, policy_version 1689418 (0.0007) [2023-12-27 17:20:57,777][20347] Updated weights for policy 0, policy_version 1689428 (0.0007) [2023-12-27 17:20:57,830][20347] Updated weights for policy 0, policy_version 1689438 (0.0007) [2023-12-27 17:20:57,877][20347] Updated weights for policy 0, policy_version 1689448 (0.0008) [2023-12-27 17:20:58,094][20381] Updated weights for policy 1, policy_version 1687796 (0.0005) [2023-12-27 17:20:58,143][20381] Updated weights for policy 1, policy_version 1687806 (0.0006) [2023-12-27 17:20:58,205][20381] Updated weights for policy 1, policy_version 1687816 (0.0007) [2023-12-27 17:20:58,558][20347] Updated weights for policy 0, policy_version 1689458 (0.0008) [2023-12-27 17:20:58,614][20347] Updated weights for policy 0, policy_version 1689468 (0.0008) [2023-12-27 17:20:58,674][20347] Updated weights for policy 0, policy_version 1689478 (0.0007) [2023-12-27 17:20:59,001][20381] Updated weights for policy 1, policy_version 1687826 (0.0008) [2023-12-27 17:20:59,055][20381] Updated weights for policy 1, policy_version 1687836 (0.0007) [2023-12-27 17:20:59,117][20381] Updated weights for policy 1, policy_version 1687846 (0.0008) [2023-12-27 17:20:59,180][20381] Updated weights for policy 1, policy_version 1687856 (0.0008) [2023-12-27 17:20:59,451][20347] Updated weights for policy 0, policy_version 1689488 (0.0007) [2023-12-27 17:20:59,512][20347] Updated weights for policy 0, policy_version 1689498 (0.0008) [2023-12-27 17:20:59,563][20347] Updated weights for policy 0, policy_version 1689508 (0.0007) [2023-12-27 17:20:59,925][20381] Updated weights for policy 1, policy_version 1687866 (0.0008) [2023-12-27 17:20:59,989][20381] Updated weights for policy 1, policy_version 1687876 (0.0008) [2023-12-27 17:21:00,048][20381] Updated weights for policy 1, policy_version 1687886 (0.0011) [2023-12-27 17:21:00,322][20347] Updated weights for policy 0, policy_version 1689518 (0.0008) [2023-12-27 17:21:00,372][20347] Updated weights for policy 0, policy_version 1689528 (0.0007) [2023-12-27 17:21:00,422][20347] Updated weights for policy 0, policy_version 1689538 (0.0006) [2023-12-27 17:21:00,708][20381] Updated weights for policy 1, policy_version 1687896 (0.0006) [2023-12-27 17:21:00,769][20381] Updated weights for policy 1, policy_version 1687906 (0.0006) [2023-12-27 17:21:00,817][20381] Updated weights for policy 1, policy_version 1687916 (0.0005) [2023-12-27 17:21:01,123][20347] Updated weights for policy 0, policy_version 1689548 (0.0006) [2023-12-27 17:21:01,185][20347] Updated weights for policy 0, policy_version 1689558 (0.0008) [2023-12-27 17:21:01,243][20347] Updated weights for policy 0, policy_version 1689568 (0.0008) [2023-12-27 17:21:01,439][20381] Updated weights for policy 1, policy_version 1687926 (0.0008) [2023-12-27 17:21:01,494][20381] Updated weights for policy 1, policy_version 1687936 (0.0010) [2023-12-27 17:21:01,544][20381] Updated weights for policy 1, policy_version 1687946 (0.0008) [2023-12-27 17:21:01,929][20347] Updated weights for policy 0, policy_version 1689578 (0.0007) [2023-12-27 17:21:01,978][20347] Updated weights for policy 0, policy_version 1689588 (0.0007) [2023-12-27 17:21:02,028][20347] Updated weights for policy 0, policy_version 1689598 (0.0007) [2023-12-27 17:21:02,085][20347] Updated weights for policy 0, policy_version 1689608 (0.0007) [2023-12-27 17:21:02,301][20381] Updated weights for policy 1, policy_version 1687956 (0.0008) [2023-12-27 17:21:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 864780288. Throughput: 0: 10167.7, 1: 10262.2. Samples: 864792384. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:02,363][19170] Avg episode reward: [(0, '970.810'), (1, '1000.000')] [2023-12-27 17:21:02,369][20381] Updated weights for policy 1, policy_version 1687966 (0.0007) [2023-12-27 17:21:02,427][20381] Updated weights for policy 1, policy_version 1687976 (0.0006) [2023-12-27 17:21:02,825][20347] Updated weights for policy 0, policy_version 1689618 (0.0007) [2023-12-27 17:21:02,884][20347] Updated weights for policy 0, policy_version 1689628 (0.0008) [2023-12-27 17:21:02,940][20347] Updated weights for policy 0, policy_version 1689638 (0.0005) [2023-12-27 17:21:03,066][20381] Updated weights for policy 1, policy_version 1687986 (0.0006) [2023-12-27 17:21:03,111][20381] Updated weights for policy 1, policy_version 1687996 (0.0007) [2023-12-27 17:21:03,156][20381] Updated weights for policy 1, policy_version 1688006 (0.0009) [2023-12-27 17:21:03,202][20381] Updated weights for policy 1, policy_version 1688016 (0.0008) [2023-12-27 17:21:03,553][20347] Updated weights for policy 0, policy_version 1689648 (0.0008) [2023-12-27 17:21:03,606][20347] Updated weights for policy 0, policy_version 1689658 (0.0008) [2023-12-27 17:21:03,661][20347] Updated weights for policy 0, policy_version 1689668 (0.0007) [2023-12-27 17:21:03,808][20381] Updated weights for policy 1, policy_version 1688026 (0.0011) [2023-12-27 17:21:03,869][20381] Updated weights for policy 1, policy_version 1688036 (0.0009) [2023-12-27 17:21:03,926][20381] Updated weights for policy 1, policy_version 1688046 (0.0008) [2023-12-27 17:21:04,289][20347] Updated weights for policy 0, policy_version 1689678 (0.0006) [2023-12-27 17:21:04,347][20347] Updated weights for policy 0, policy_version 1689688 (0.0007) [2023-12-27 17:21:04,403][20347] Updated weights for policy 0, policy_version 1689698 (0.0010) [2023-12-27 17:21:04,695][20381] Updated weights for policy 1, policy_version 1688056 (0.0008) [2023-12-27 17:21:04,743][20381] Updated weights for policy 1, policy_version 1688066 (0.0007) [2023-12-27 17:21:04,795][20381] Updated weights for policy 1, policy_version 1688076 (0.0006) [2023-12-27 17:21:05,036][20347] Updated weights for policy 0, policy_version 1689708 (0.0008) [2023-12-27 17:21:05,083][20347] Updated weights for policy 0, policy_version 1689718 (0.0007) [2023-12-27 17:21:05,145][20347] Updated weights for policy 0, policy_version 1689728 (0.0006) [2023-12-27 17:21:05,547][20381] Updated weights for policy 1, policy_version 1688086 (0.0009) [2023-12-27 17:21:05,596][20381] Updated weights for policy 1, policy_version 1688096 (0.0010) [2023-12-27 17:21:05,650][20381] Updated weights for policy 1, policy_version 1688106 (0.0010) [2023-12-27 17:21:05,773][20347] Updated weights for policy 0, policy_version 1689738 (0.0007) [2023-12-27 17:21:05,832][20347] Updated weights for policy 0, policy_version 1689748 (0.0007) [2023-12-27 17:21:05,893][20347] Updated weights for policy 0, policy_version 1689758 (0.0008) [2023-12-27 17:21:05,957][20347] Updated weights for policy 0, policy_version 1689768 (0.0008) [2023-12-27 17:21:06,331][20381] Updated weights for policy 1, policy_version 1688116 (0.0009) [2023-12-27 17:21:06,396][20381] Updated weights for policy 1, policy_version 1688126 (0.0006) [2023-12-27 17:21:06,458][20381] Updated weights for policy 1, policy_version 1688136 (0.0007) [2023-12-27 17:21:06,732][20347] Updated weights for policy 0, policy_version 1689778 (0.0005) [2023-12-27 17:21:06,796][20347] Updated weights for policy 0, policy_version 1689788 (0.0005) [2023-12-27 17:21:06,854][20347] Updated weights for policy 0, policy_version 1689798 (0.0007) [2023-12-27 17:21:06,967][20381] Updated weights for policy 1, policy_version 1688146 (0.0006) [2023-12-27 17:21:07,010][20381] Updated weights for policy 1, policy_version 1688156 (0.0007) [2023-12-27 17:21:07,057][20381] Updated weights for policy 1, policy_version 1688166 (0.0007) [2023-12-27 17:21:07,103][20381] Updated weights for policy 1, policy_version 1688176 (0.0008) [2023-12-27 17:21:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 864886784. Throughput: 0: 10184.1, 1: 10299.7. Samples: 864854996. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:07,364][19170] Avg episode reward: [(0, '990.380'), (1, '1000.000')] [2023-12-27 17:21:07,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001688176_432218112.pth... [2023-12-27 17:21:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001689800_432668672.pth... [2023-12-27 17:21:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001688648_432373760.pth [2023-12-27 17:21:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001686960_431906816.pth [2023-12-27 17:21:07,503][20347] Updated weights for policy 0, policy_version 1689808 (0.0009) [2023-12-27 17:21:07,549][20347] Updated weights for policy 0, policy_version 1689818 (0.0009) [2023-12-27 17:21:07,606][20347] Updated weights for policy 0, policy_version 1689828 (0.0008) [2023-12-27 17:21:07,814][20381] Updated weights for policy 1, policy_version 1688186 (0.0005) [2023-12-27 17:21:07,873][20381] Updated weights for policy 1, policy_version 1688196 (0.0006) [2023-12-27 17:21:07,934][20381] Updated weights for policy 1, policy_version 1688206 (0.0008) [2023-12-27 17:21:08,329][20347] Updated weights for policy 0, policy_version 1689838 (0.0007) [2023-12-27 17:21:08,397][20347] Updated weights for policy 0, policy_version 1689848 (0.0008) [2023-12-27 17:21:08,455][20347] Updated weights for policy 0, policy_version 1689858 (0.0009) [2023-12-27 17:21:08,534][20381] Updated weights for policy 1, policy_version 1688216 (0.0010) [2023-12-27 17:21:08,589][20381] Updated weights for policy 1, policy_version 1688226 (0.0010) [2023-12-27 17:21:08,644][20381] Updated weights for policy 1, policy_version 1688236 (0.0010) [2023-12-27 17:21:09,090][20347] Updated weights for policy 0, policy_version 1689868 (0.0007) [2023-12-27 17:21:09,151][20347] Updated weights for policy 0, policy_version 1689878 (0.0008) [2023-12-27 17:21:09,217][20347] Updated weights for policy 0, policy_version 1689888 (0.0008) [2023-12-27 17:21:09,293][20381] Updated weights for policy 1, policy_version 1688246 (0.0009) [2023-12-27 17:21:09,352][20381] Updated weights for policy 1, policy_version 1688256 (0.0007) [2023-12-27 17:21:09,424][20381] Updated weights for policy 1, policy_version 1688266 (0.0008) [2023-12-27 17:21:09,829][20347] Updated weights for policy 0, policy_version 1689898 (0.0007) [2023-12-27 17:21:09,892][20347] Updated weights for policy 0, policy_version 1689908 (0.0007) [2023-12-27 17:21:09,954][20347] Updated weights for policy 0, policy_version 1689918 (0.0008) [2023-12-27 17:21:10,007][20347] Updated weights for policy 0, policy_version 1689928 (0.0007) [2023-12-27 17:21:10,229][20381] Updated weights for policy 1, policy_version 1688276 (0.0007) [2023-12-27 17:21:10,276][20381] Updated weights for policy 1, policy_version 1688286 (0.0008) [2023-12-27 17:21:10,335][20381] Updated weights for policy 1, policy_version 1688296 (0.0008) [2023-12-27 17:21:10,711][20347] Updated weights for policy 0, policy_version 1689938 (0.0008) [2023-12-27 17:21:10,769][20347] Updated weights for policy 0, policy_version 1689948 (0.0007) [2023-12-27 17:21:10,825][20347] Updated weights for policy 0, policy_version 1689958 (0.0007) [2023-12-27 17:21:11,028][20381] Updated weights for policy 1, policy_version 1688306 (0.0007) [2023-12-27 17:21:11,088][20381] Updated weights for policy 1, policy_version 1688316 (0.0007) [2023-12-27 17:21:11,153][20381] Updated weights for policy 1, policy_version 1688326 (0.0006) [2023-12-27 17:21:11,218][20381] Updated weights for policy 1, policy_version 1688336 (0.0007) [2023-12-27 17:21:11,626][20347] Updated weights for policy 0, policy_version 1689968 (0.0007) [2023-12-27 17:21:11,687][20347] Updated weights for policy 0, policy_version 1689978 (0.0008) [2023-12-27 17:21:11,756][20347] Updated weights for policy 0, policy_version 1689988 (0.0009) [2023-12-27 17:21:11,961][20381] Updated weights for policy 1, policy_version 1688346 (0.0008) [2023-12-27 17:21:12,024][20381] Updated weights for policy 1, policy_version 1688356 (0.0008) [2023-12-27 17:21:12,076][20381] Updated weights for policy 1, policy_version 1688366 (0.0008) [2023-12-27 17:21:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 864985088. Throughput: 0: 10129.5, 1: 10325.5. Samples: 864977268. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:12,363][19170] Avg episode reward: [(0, '951.340'), (1, '1000.000')] [2023-12-27 17:21:12,502][20347] Updated weights for policy 0, policy_version 1689998 (0.0007) [2023-12-27 17:21:12,564][20347] Updated weights for policy 0, policy_version 1690008 (0.0006) [2023-12-27 17:21:12,622][20347] Updated weights for policy 0, policy_version 1690018 (0.0007) [2023-12-27 17:21:12,748][20381] Updated weights for policy 1, policy_version 1688376 (0.0006) [2023-12-27 17:21:12,802][20381] Updated weights for policy 1, policy_version 1688386 (0.0010) [2023-12-27 17:21:12,826][20292] KL-divergence is very high: 117.4061 [2023-12-27 17:21:12,865][20381] Updated weights for policy 1, policy_version 1688396 (0.0011) [2023-12-27 17:21:13,313][20347] Updated weights for policy 0, policy_version 1690028 (0.0006) [2023-12-27 17:21:13,360][20347] Updated weights for policy 0, policy_version 1690038 (0.0007) [2023-12-27 17:21:13,407][20347] Updated weights for policy 0, policy_version 1690048 (0.0007) [2023-12-27 17:21:13,512][20381] Updated weights for policy 1, policy_version 1688406 (0.0008) [2023-12-27 17:21:13,562][20381] Updated weights for policy 1, policy_version 1688416 (0.0006) [2023-12-27 17:21:13,570][20292] KL-divergence is very high: 113.0226 [2023-12-27 17:21:13,617][20381] Updated weights for policy 1, policy_version 1688426 (0.0006) [2023-12-27 17:21:14,069][20347] Updated weights for policy 0, policy_version 1690058 (0.0007) [2023-12-27 17:21:14,125][20347] Updated weights for policy 0, policy_version 1690068 (0.0007) [2023-12-27 17:21:14,181][20347] Updated weights for policy 0, policy_version 1690078 (0.0006) [2023-12-27 17:21:14,241][20381] Updated weights for policy 1, policy_version 1688436 (0.0007) [2023-12-27 17:21:14,243][20347] Updated weights for policy 0, policy_version 1690088 (0.0007) [2023-12-27 17:21:14,289][20381] Updated weights for policy 1, policy_version 1688446 (0.0007) [2023-12-27 17:21:14,341][20381] Updated weights for policy 1, policy_version 1688456 (0.0007) [2023-12-27 17:21:14,821][20347] Updated weights for policy 0, policy_version 1690098 (0.0008) [2023-12-27 17:21:14,878][20347] Updated weights for policy 0, policy_version 1690108 (0.0008) [2023-12-27 17:21:14,937][20347] Updated weights for policy 0, policy_version 1690118 (0.0008) [2023-12-27 17:21:15,068][20381] Updated weights for policy 1, policy_version 1688466 (0.0007) [2023-12-27 17:21:15,136][20381] Updated weights for policy 1, policy_version 1688476 (0.0009) [2023-12-27 17:21:15,198][20381] Updated weights for policy 1, policy_version 1688486 (0.0006) [2023-12-27 17:21:15,251][20381] Updated weights for policy 1, policy_version 1688496 (0.0006) [2023-12-27 17:21:15,592][20347] Updated weights for policy 0, policy_version 1690128 (0.0006) [2023-12-27 17:21:15,638][20347] Updated weights for policy 0, policy_version 1690138 (0.0008) [2023-12-27 17:21:15,693][20347] Updated weights for policy 0, policy_version 1690148 (0.0007) [2023-12-27 17:21:15,928][20381] Updated weights for policy 1, policy_version 1688506 (0.0008) [2023-12-27 17:21:15,985][20381] Updated weights for policy 1, policy_version 1688516 (0.0007) [2023-12-27 17:21:16,037][20381] Updated weights for policy 1, policy_version 1688526 (0.0007) [2023-12-27 17:21:16,308][20347] Updated weights for policy 0, policy_version 1690158 (0.0008) [2023-12-27 17:21:16,366][20347] Updated weights for policy 0, policy_version 1690168 (0.0007) [2023-12-27 17:21:16,421][20347] Updated weights for policy 0, policy_version 1690178 (0.0008) [2023-12-27 17:21:16,775][20381] Updated weights for policy 1, policy_version 1688536 (0.0010) [2023-12-27 17:21:16,837][20381] Updated weights for policy 1, policy_version 1688546 (0.0011) [2023-12-27 17:21:16,894][20381] Updated weights for policy 1, policy_version 1688556 (0.0011) [2023-12-27 17:21:17,072][20347] Updated weights for policy 0, policy_version 1690188 (0.0007) [2023-12-27 17:21:17,126][20347] Updated weights for policy 0, policy_version 1690198 (0.0005) [2023-12-27 17:21:17,182][20347] Updated weights for policy 0, policy_version 1690208 (0.0005) [2023-12-27 17:21:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 865091584. Throughput: 0: 10156.9, 1: 10272.4. Samples: 865100120. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:17,363][19170] Avg episode reward: [(0, '960.900'), (1, '980.590')] [2023-12-27 17:21:17,537][20381] Updated weights for policy 1, policy_version 1688566 (0.0011) [2023-12-27 17:21:17,588][20381] Updated weights for policy 1, policy_version 1688576 (0.0010) [2023-12-27 17:21:17,646][20381] Updated weights for policy 1, policy_version 1688586 (0.0010) [2023-12-27 17:21:17,781][20347] Updated weights for policy 0, policy_version 1690218 (0.0006) [2023-12-27 17:21:17,832][20347] Updated weights for policy 0, policy_version 1690228 (0.0007) [2023-12-27 17:21:17,882][20347] Updated weights for policy 0, policy_version 1690238 (0.0007) [2023-12-27 17:21:17,939][20347] Updated weights for policy 0, policy_version 1690248 (0.0007) [2023-12-27 17:21:18,345][20381] Updated weights for policy 1, policy_version 1688596 (0.0010) [2023-12-27 17:21:18,401][20381] Updated weights for policy 1, policy_version 1688606 (0.0011) [2023-12-27 17:21:18,464][20381] Updated weights for policy 1, policy_version 1688616 (0.0011) [2023-12-27 17:21:18,641][20347] Updated weights for policy 0, policy_version 1690258 (0.0006) [2023-12-27 17:21:18,696][20347] Updated weights for policy 0, policy_version 1690268 (0.0005) [2023-12-27 17:21:18,746][20347] Updated weights for policy 0, policy_version 1690278 (0.0006) [2023-12-27 17:21:19,113][20381] Updated weights for policy 1, policy_version 1688626 (0.0010) [2023-12-27 17:21:19,169][20381] Updated weights for policy 1, policy_version 1688636 (0.0010) [2023-12-27 17:21:19,230][20381] Updated weights for policy 1, policy_version 1688646 (0.0011) [2023-12-27 17:21:19,288][20381] Updated weights for policy 1, policy_version 1688656 (0.0011) [2023-12-27 17:21:19,426][20347] Updated weights for policy 0, policy_version 1690288 (0.0007) [2023-12-27 17:21:19,491][20347] Updated weights for policy 0, policy_version 1690298 (0.0007) [2023-12-27 17:21:19,546][20347] Updated weights for policy 0, policy_version 1690308 (0.0007) [2023-12-27 17:21:20,006][20381] Updated weights for policy 1, policy_version 1688666 (0.0011) [2023-12-27 17:21:20,054][20381] Updated weights for policy 1, policy_version 1688676 (0.0011) [2023-12-27 17:21:20,101][20381] Updated weights for policy 1, policy_version 1688686 (0.0011) [2023-12-27 17:21:20,272][20347] Updated weights for policy 0, policy_version 1690318 (0.0007) [2023-12-27 17:21:20,335][20347] Updated weights for policy 0, policy_version 1690328 (0.0008) [2023-12-27 17:21:20,394][20347] Updated weights for policy 0, policy_version 1690338 (0.0007) [2023-12-27 17:21:20,844][20381] Updated weights for policy 1, policy_version 1688696 (0.0010) [2023-12-27 17:21:20,905][20381] Updated weights for policy 1, policy_version 1688706 (0.0008) [2023-12-27 17:21:20,967][20381] Updated weights for policy 1, policy_version 1688716 (0.0008) [2023-12-27 17:21:21,094][20347] Updated weights for policy 0, policy_version 1690348 (0.0007) [2023-12-27 17:21:21,158][20347] Updated weights for policy 0, policy_version 1690358 (0.0007) [2023-12-27 17:21:21,217][20347] Updated weights for policy 0, policy_version 1690368 (0.0008) [2023-12-27 17:21:21,767][20381] Updated weights for policy 1, policy_version 1688726 (0.0008) [2023-12-27 17:21:21,823][20381] Updated weights for policy 1, policy_version 1688736 (0.0006) [2023-12-27 17:21:21,889][20381] Updated weights for policy 1, policy_version 1688746 (0.0007) [2023-12-27 17:21:21,891][20347] Updated weights for policy 0, policy_version 1690378 (0.0007) [2023-12-27 17:21:21,949][20347] Updated weights for policy 0, policy_version 1690388 (0.0006) [2023-12-27 17:21:22,006][20347] Updated weights for policy 0, policy_version 1690398 (0.0008) [2023-12-27 17:21:22,065][20347] Updated weights for policy 0, policy_version 1690408 (0.0007) [2023-12-27 17:21:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 865189888. Throughput: 0: 10139.0, 1: 10238.7. Samples: 865160512. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:22,363][19170] Avg episode reward: [(0, '980.500'), (1, '980.590')] [2023-12-27 17:21:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001688752_432365568.pth... [2023-12-27 17:21:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001690408_432824320.pth... [2023-12-27 17:21:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001687568_432062464.pth [2023-12-27 17:21:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001689224_432521216.pth [2023-12-27 17:21:22,631][20381] Updated weights for policy 1, policy_version 1688756 (0.0008) [2023-12-27 17:21:22,692][20381] Updated weights for policy 1, policy_version 1688766 (0.0011) [2023-12-27 17:21:22,748][20381] Updated weights for policy 1, policy_version 1688776 (0.0010) [2023-12-27 17:21:22,754][20347] Updated weights for policy 0, policy_version 1690418 (0.0006) [2023-12-27 17:21:22,814][20347] Updated weights for policy 0, policy_version 1690428 (0.0006) [2023-12-27 17:21:22,877][20347] Updated weights for policy 0, policy_version 1690438 (0.0008) [2023-12-27 17:21:23,419][20381] Updated weights for policy 1, policy_version 1688786 (0.0010) [2023-12-27 17:21:23,477][20381] Updated weights for policy 1, policy_version 1688796 (0.0010) [2023-12-27 17:21:23,511][20347] Updated weights for policy 0, policy_version 1690448 (0.0006) [2023-12-27 17:21:23,528][20381] Updated weights for policy 1, policy_version 1688806 (0.0010) [2023-12-27 17:21:23,572][20347] Updated weights for policy 0, policy_version 1690458 (0.0005) [2023-12-27 17:21:23,582][20381] Updated weights for policy 1, policy_version 1688816 (0.0010) [2023-12-27 17:21:23,624][20347] Updated weights for policy 0, policy_version 1690468 (0.0005) [2023-12-27 17:21:24,158][20347] Updated weights for policy 0, policy_version 1690478 (0.0005) [2023-12-27 17:21:24,209][20347] Updated weights for policy 0, policy_version 1690488 (0.0007) [2023-12-27 17:21:24,223][20381] Updated weights for policy 1, policy_version 1688826 (0.0006) [2023-12-27 17:21:24,262][20347] Updated weights for policy 0, policy_version 1690498 (0.0007) [2023-12-27 17:21:24,284][20381] Updated weights for policy 1, policy_version 1688836 (0.0006) [2023-12-27 17:21:24,334][20381] Updated weights for policy 1, policy_version 1688846 (0.0006) [2023-12-27 17:21:24,935][20381] Updated weights for policy 1, policy_version 1688856 (0.0006) [2023-12-27 17:21:24,941][20347] Updated weights for policy 0, policy_version 1690508 (0.0005) [2023-12-27 17:21:24,991][20381] Updated weights for policy 1, policy_version 1688866 (0.0006) [2023-12-27 17:21:24,999][20347] Updated weights for policy 0, policy_version 1690518 (0.0009) [2023-12-27 17:21:25,048][20381] Updated weights for policy 1, policy_version 1688876 (0.0007) [2023-12-27 17:21:25,058][20347] Updated weights for policy 0, policy_version 1690528 (0.0011) [2023-12-27 17:21:25,654][20381] Updated weights for policy 1, policy_version 1688886 (0.0006) [2023-12-27 17:21:25,710][20381] Updated weights for policy 1, policy_version 1688896 (0.0007) [2023-12-27 17:21:25,726][20347] Updated weights for policy 0, policy_version 1690538 (0.0009) [2023-12-27 17:21:25,766][20381] Updated weights for policy 1, policy_version 1688906 (0.0006) [2023-12-27 17:21:25,782][20347] Updated weights for policy 0, policy_version 1690548 (0.0007) [2023-12-27 17:21:25,828][20347] Updated weights for policy 0, policy_version 1690558 (0.0010) [2023-12-27 17:21:25,876][20347] Updated weights for policy 0, policy_version 1690568 (0.0010) [2023-12-27 17:21:26,326][20381] Updated weights for policy 1, policy_version 1688916 (0.0005) [2023-12-27 17:21:26,377][20381] Updated weights for policy 1, policy_version 1688926 (0.0008) [2023-12-27 17:21:26,429][20381] Updated weights for policy 1, policy_version 1688936 (0.0006) [2023-12-27 17:21:26,518][20347] Updated weights for policy 0, policy_version 1690578 (0.0009) [2023-12-27 17:21:26,574][20347] Updated weights for policy 0, policy_version 1690588 (0.0005) [2023-12-27 17:21:26,634][20347] Updated weights for policy 0, policy_version 1690598 (0.0008) [2023-12-27 17:21:27,060][20381] Updated weights for policy 1, policy_version 1688946 (0.0007) [2023-12-27 17:21:27,108][20381] Updated weights for policy 1, policy_version 1688956 (0.0008) [2023-12-27 17:21:27,160][20381] Updated weights for policy 1, policy_version 1688966 (0.0005) [2023-12-27 17:21:27,192][20347] Updated weights for policy 0, policy_version 1690608 (0.0006) [2023-12-27 17:21:27,215][20381] Updated weights for policy 1, policy_version 1688976 (0.0006) [2023-12-27 17:21:27,240][20347] Updated weights for policy 0, policy_version 1690618 (0.0010) [2023-12-27 17:21:27,297][20347] Updated weights for policy 0, policy_version 1690628 (0.0009) [2023-12-27 17:21:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 865304576. Throughput: 0: 10205.1, 1: 10283.7. Samples: 865286212. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:27,363][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 17:21:27,790][20381] Updated weights for policy 1, policy_version 1688986 (0.0005) [2023-12-27 17:21:27,853][20381] Updated weights for policy 1, policy_version 1688996 (0.0005) [2023-12-27 17:21:27,890][20347] Updated weights for policy 0, policy_version 1690638 (0.0008) [2023-12-27 17:21:27,905][20381] Updated weights for policy 1, policy_version 1689006 (0.0005) [2023-12-27 17:21:27,934][20347] Updated weights for policy 0, policy_version 1690648 (0.0010) [2023-12-27 17:21:27,995][20347] Updated weights for policy 0, policy_version 1690658 (0.0010) [2023-12-27 17:21:28,459][20381] Updated weights for policy 1, policy_version 1689016 (0.0006) [2023-12-27 17:21:28,513][20381] Updated weights for policy 1, policy_version 1689026 (0.0007) [2023-12-27 17:21:28,569][20381] Updated weights for policy 1, policy_version 1689036 (0.0007) [2023-12-27 17:21:28,603][20347] Updated weights for policy 0, policy_version 1690668 (0.0009) [2023-12-27 17:21:28,670][20347] Updated weights for policy 0, policy_version 1690678 (0.0006) [2023-12-27 17:21:28,721][20347] Updated weights for policy 0, policy_version 1690688 (0.0005) [2023-12-27 17:21:29,165][20381] Updated weights for policy 1, policy_version 1689046 (0.0006) [2023-12-27 17:21:29,221][20381] Updated weights for policy 1, policy_version 1689056 (0.0006) [2023-12-27 17:21:29,280][20381] Updated weights for policy 1, policy_version 1689066 (0.0007) [2023-12-27 17:21:29,292][20347] Updated weights for policy 0, policy_version 1690698 (0.0006) [2023-12-27 17:21:29,356][20347] Updated weights for policy 0, policy_version 1690708 (0.0007) [2023-12-27 17:21:29,417][20347] Updated weights for policy 0, policy_version 1690718 (0.0006) [2023-12-27 17:21:29,483][20347] Updated weights for policy 0, policy_version 1690728 (0.0006) [2023-12-27 17:21:29,923][20381] Updated weights for policy 1, policy_version 1689076 (0.0006) [2023-12-27 17:21:29,980][20381] Updated weights for policy 1, policy_version 1689086 (0.0006) [2023-12-27 17:21:30,039][20381] Updated weights for policy 1, policy_version 1689096 (0.0007) [2023-12-27 17:21:30,086][20347] Updated weights for policy 0, policy_version 1690738 (0.0007) [2023-12-27 17:21:30,135][20347] Updated weights for policy 0, policy_version 1690748 (0.0008) [2023-12-27 17:21:30,184][20347] Updated weights for policy 0, policy_version 1690758 (0.0007) [2023-12-27 17:21:30,653][20381] Updated weights for policy 1, policy_version 1689106 (0.0007) [2023-12-27 17:21:30,711][20381] Updated weights for policy 1, policy_version 1689116 (0.0010) [2023-12-27 17:21:30,762][20381] Updated weights for policy 1, policy_version 1689126 (0.0010) [2023-12-27 17:21:30,809][20381] Updated weights for policy 1, policy_version 1689136 (0.0010) [2023-12-27 17:21:30,828][20347] Updated weights for policy 0, policy_version 1690768 (0.0006) [2023-12-27 17:21:30,889][20347] Updated weights for policy 0, policy_version 1690778 (0.0005) [2023-12-27 17:21:30,935][20347] Updated weights for policy 0, policy_version 1690788 (0.0005) [2023-12-27 17:21:31,504][20381] Updated weights for policy 1, policy_version 1689146 (0.0008) [2023-12-27 17:21:31,564][20381] Updated weights for policy 1, policy_version 1689156 (0.0007) [2023-12-27 17:21:31,604][20347] Updated weights for policy 0, policy_version 1690798 (0.0006) [2023-12-27 17:21:31,626][20381] Updated weights for policy 1, policy_version 1689166 (0.0007) [2023-12-27 17:21:31,667][20347] Updated weights for policy 0, policy_version 1690808 (0.0008) [2023-12-27 17:21:31,711][20347] Updated weights for policy 0, policy_version 1690818 (0.0008) [2023-12-27 17:21:32,330][20381] Updated weights for policy 1, policy_version 1689176 (0.0007) [2023-12-27 17:21:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 865402880. Throughput: 0: 10345.2, 1: 10384.6. Samples: 865418308. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:32,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:21:32,397][20381] Updated weights for policy 1, policy_version 1689186 (0.0007) [2023-12-27 17:21:32,448][20347] Updated weights for policy 0, policy_version 1690828 (0.0007) [2023-12-27 17:21:32,460][20381] Updated weights for policy 1, policy_version 1689196 (0.0008) [2023-12-27 17:21:32,501][20347] Updated weights for policy 0, policy_version 1690838 (0.0007) [2023-12-27 17:21:32,556][20347] Updated weights for policy 0, policy_version 1690848 (0.0008) [2023-12-27 17:21:33,138][20381] Updated weights for policy 1, policy_version 1689206 (0.0009) [2023-12-27 17:21:33,199][20347] Updated weights for policy 0, policy_version 1690858 (0.0006) [2023-12-27 17:21:33,203][20381] Updated weights for policy 1, policy_version 1689216 (0.0011) [2023-12-27 17:21:33,250][20347] Updated weights for policy 0, policy_version 1690868 (0.0007) [2023-12-27 17:21:33,267][20381] Updated weights for policy 1, policy_version 1689226 (0.0011) [2023-12-27 17:21:33,308][20347] Updated weights for policy 0, policy_version 1690878 (0.0006) [2023-12-27 17:21:33,365][20347] Updated weights for policy 0, policy_version 1690888 (0.0007) [2023-12-27 17:21:33,925][20381] Updated weights for policy 1, policy_version 1689236 (0.0010) [2023-12-27 17:21:33,964][20347] Updated weights for policy 0, policy_version 1690898 (0.0005) [2023-12-27 17:21:33,977][20381] Updated weights for policy 1, policy_version 1689246 (0.0010) [2023-12-27 17:21:34,020][20347] Updated weights for policy 0, policy_version 1690908 (0.0006) [2023-12-27 17:21:34,034][20381] Updated weights for policy 1, policy_version 1689256 (0.0010) [2023-12-27 17:21:34,072][20347] Updated weights for policy 0, policy_version 1690918 (0.0006) [2023-12-27 17:21:34,720][20381] Updated weights for policy 1, policy_version 1689266 (0.0010) [2023-12-27 17:21:34,777][20381] Updated weights for policy 1, policy_version 1689276 (0.0011) [2023-12-27 17:21:34,787][20347] Updated weights for policy 0, policy_version 1690928 (0.0008) [2023-12-27 17:21:34,837][20381] Updated weights for policy 1, policy_version 1689286 (0.0009) [2023-12-27 17:21:34,849][20347] Updated weights for policy 0, policy_version 1690938 (0.0006) [2023-12-27 17:21:34,895][20381] Updated weights for policy 1, policy_version 1689296 (0.0010) [2023-12-27 17:21:34,912][20347] Updated weights for policy 0, policy_version 1690948 (0.0007) [2023-12-27 17:21:35,520][20347] Updated weights for policy 0, policy_version 1690958 (0.0007) [2023-12-27 17:21:35,568][20347] Updated weights for policy 0, policy_version 1690968 (0.0006) [2023-12-27 17:21:35,569][20381] Updated weights for policy 1, policy_version 1689306 (0.0011) [2023-12-27 17:21:35,626][20347] Updated weights for policy 0, policy_version 1690978 (0.0005) [2023-12-27 17:21:35,627][20381] Updated weights for policy 1, policy_version 1689316 (0.0010) [2023-12-27 17:21:35,685][20381] Updated weights for policy 1, policy_version 1689326 (0.0009) [2023-12-27 17:21:36,225][20347] Updated weights for policy 0, policy_version 1690988 (0.0006) [2023-12-27 17:21:36,273][20347] Updated weights for policy 0, policy_version 1690998 (0.0006) [2023-12-27 17:21:36,300][20381] Updated weights for policy 1, policy_version 1689336 (0.0008) [2023-12-27 17:21:36,329][20347] Updated weights for policy 0, policy_version 1691008 (0.0005) [2023-12-27 17:21:36,360][20381] Updated weights for policy 1, policy_version 1689346 (0.0008) [2023-12-27 17:21:36,413][20381] Updated weights for policy 1, policy_version 1689356 (0.0007) [2023-12-27 17:21:36,911][20347] Updated weights for policy 0, policy_version 1691018 (0.0006) [2023-12-27 17:21:36,964][20347] Updated weights for policy 0, policy_version 1691028 (0.0005) [2023-12-27 17:21:37,026][20347] Updated weights for policy 0, policy_version 1691038 (0.0006) [2023-12-27 17:21:37,048][20381] Updated weights for policy 1, policy_version 1689366 (0.0007) [2023-12-27 17:21:37,085][20347] Updated weights for policy 0, policy_version 1691048 (0.0005) [2023-12-27 17:21:37,104][20381] Updated weights for policy 1, policy_version 1689376 (0.0007) [2023-12-27 17:21:37,154][20381] Updated weights for policy 1, policy_version 1689386 (0.0010) [2023-12-27 17:21:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 865517568. Throughput: 0: 10382.1, 1: 10345.2. Samples: 865479632. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:21:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001689392_432529408.pth... [2023-12-27 17:21:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001691048_432988160.pth... [2023-12-27 17:21:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001688176_432218112.pth [2023-12-27 17:21:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001689800_432668672.pth [2023-12-27 17:21:37,675][20347] Updated weights for policy 0, policy_version 1691058 (0.0007) [2023-12-27 17:21:37,732][20347] Updated weights for policy 0, policy_version 1691068 (0.0008) [2023-12-27 17:21:37,780][20347] Updated weights for policy 0, policy_version 1691078 (0.0008) [2023-12-27 17:21:37,792][20381] Updated weights for policy 1, policy_version 1689396 (0.0008) [2023-12-27 17:21:37,844][20381] Updated weights for policy 1, policy_version 1689406 (0.0009) [2023-12-27 17:21:37,897][20381] Updated weights for policy 1, policy_version 1689416 (0.0007) [2023-12-27 17:21:38,484][20347] Updated weights for policy 0, policy_version 1691088 (0.0007) [2023-12-27 17:21:38,548][20347] Updated weights for policy 0, policy_version 1691098 (0.0007) [2023-12-27 17:21:38,550][20381] Updated weights for policy 1, policy_version 1689426 (0.0006) [2023-12-27 17:21:38,604][20347] Updated weights for policy 0, policy_version 1691108 (0.0007) [2023-12-27 17:21:38,606][20381] Updated weights for policy 1, policy_version 1689436 (0.0005) [2023-12-27 17:21:38,664][20381] Updated weights for policy 1, policy_version 1689446 (0.0008) [2023-12-27 17:21:38,724][20381] Updated weights for policy 1, policy_version 1689456 (0.0008) [2023-12-27 17:21:39,348][20381] Updated weights for policy 1, policy_version 1689466 (0.0007) [2023-12-27 17:21:39,378][20347] Updated weights for policy 0, policy_version 1691118 (0.0008) [2023-12-27 17:21:39,422][20381] Updated weights for policy 1, policy_version 1689476 (0.0008) [2023-12-27 17:21:39,447][20347] Updated weights for policy 0, policy_version 1691128 (0.0008) [2023-12-27 17:21:39,487][20381] Updated weights for policy 1, policy_version 1689486 (0.0006) [2023-12-27 17:21:39,507][20347] Updated weights for policy 0, policy_version 1691138 (0.0008) [2023-12-27 17:21:40,154][20381] Updated weights for policy 1, policy_version 1689496 (0.0008) [2023-12-27 17:21:40,180][20347] Updated weights for policy 0, policy_version 1691148 (0.0006) [2023-12-27 17:21:40,212][20381] Updated weights for policy 1, policy_version 1689506 (0.0007) [2023-12-27 17:21:40,237][20347] Updated weights for policy 0, policy_version 1691158 (0.0006) [2023-12-27 17:21:40,281][20381] Updated weights for policy 1, policy_version 1689516 (0.0006) [2023-12-27 17:21:40,298][20347] Updated weights for policy 0, policy_version 1691168 (0.0006) [2023-12-27 17:21:40,932][20347] Updated weights for policy 0, policy_version 1691178 (0.0008) [2023-12-27 17:21:40,981][20381] Updated weights for policy 1, policy_version 1689526 (0.0006) [2023-12-27 17:21:40,990][20347] Updated weights for policy 0, policy_version 1691188 (0.0011) [2023-12-27 17:21:41,042][20381] Updated weights for policy 1, policy_version 1689536 (0.0007) [2023-12-27 17:21:41,056][20347] Updated weights for policy 0, policy_version 1691198 (0.0010) [2023-12-27 17:21:41,105][20381] Updated weights for policy 1, policy_version 1689546 (0.0006) [2023-12-27 17:21:41,121][20347] Updated weights for policy 0, policy_version 1691208 (0.0011) [2023-12-27 17:21:41,836][20347] Updated weights for policy 0, policy_version 1691218 (0.0010) [2023-12-27 17:21:41,837][20381] Updated weights for policy 1, policy_version 1689556 (0.0007) [2023-12-27 17:21:41,888][20381] Updated weights for policy 1, policy_version 1689566 (0.0006) [2023-12-27 17:21:41,894][20347] Updated weights for policy 0, policy_version 1691228 (0.0011) [2023-12-27 17:21:41,946][20347] Updated weights for policy 0, policy_version 1691238 (0.0010) [2023-12-27 17:21:41,948][20381] Updated weights for policy 1, policy_version 1689576 (0.0005) [2023-12-27 17:21:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 865615872. Throughput: 0: 10438.2, 1: 10326.4. Samples: 865605740. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:42,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:21:42,625][20381] Updated weights for policy 1, policy_version 1689586 (0.0008) [2023-12-27 17:21:42,672][20347] Updated weights for policy 0, policy_version 1691248 (0.0010) [2023-12-27 17:21:42,685][20381] Updated weights for policy 1, policy_version 1689596 (0.0005) [2023-12-27 17:21:42,732][20347] Updated weights for policy 0, policy_version 1691258 (0.0010) [2023-12-27 17:21:42,745][20381] Updated weights for policy 1, policy_version 1689606 (0.0005) [2023-12-27 17:21:42,791][20347] Updated weights for policy 0, policy_version 1691268 (0.0010) [2023-12-27 17:21:42,808][20381] Updated weights for policy 1, policy_version 1689616 (0.0006) [2023-12-27 17:21:43,386][20381] Updated weights for policy 1, policy_version 1689626 (0.0007) [2023-12-27 17:21:43,442][20381] Updated weights for policy 1, policy_version 1689636 (0.0007) [2023-12-27 17:21:43,448][20347] Updated weights for policy 0, policy_version 1691278 (0.0008) [2023-12-27 17:21:43,496][20381] Updated weights for policy 1, policy_version 1689646 (0.0006) [2023-12-27 17:21:43,505][20347] Updated weights for policy 0, policy_version 1691288 (0.0010) [2023-12-27 17:21:43,553][20347] Updated weights for policy 0, policy_version 1691298 (0.0010) [2023-12-27 17:21:44,184][20347] Updated weights for policy 0, policy_version 1691308 (0.0009) [2023-12-27 17:21:44,194][20381] Updated weights for policy 1, policy_version 1689656 (0.0007) [2023-12-27 17:21:44,241][20381] Updated weights for policy 1, policy_version 1689666 (0.0007) [2023-12-27 17:21:44,242][20347] Updated weights for policy 0, policy_version 1691318 (0.0006) [2023-12-27 17:21:44,294][20381] Updated weights for policy 1, policy_version 1689676 (0.0007) [2023-12-27 17:21:44,296][20347] Updated weights for policy 0, policy_version 1691328 (0.0008) [2023-12-27 17:21:44,948][20347] Updated weights for policy 0, policy_version 1691338 (0.0006) [2023-12-27 17:21:44,986][20381] Updated weights for policy 1, policy_version 1689686 (0.0008) [2023-12-27 17:21:45,004][20347] Updated weights for policy 0, policy_version 1691348 (0.0008) [2023-12-27 17:21:45,043][20381] Updated weights for policy 1, policy_version 1689696 (0.0006) [2023-12-27 17:21:45,061][20347] Updated weights for policy 0, policy_version 1691358 (0.0011) [2023-12-27 17:21:45,092][20381] Updated weights for policy 1, policy_version 1689706 (0.0005) [2023-12-27 17:21:45,116][20347] Updated weights for policy 0, policy_version 1691368 (0.0010) [2023-12-27 17:21:45,774][20347] Updated weights for policy 0, policy_version 1691378 (0.0007) [2023-12-27 17:21:45,823][20381] Updated weights for policy 1, policy_version 1689716 (0.0008) [2023-12-27 17:21:45,827][20347] Updated weights for policy 0, policy_version 1691388 (0.0008) [2023-12-27 17:21:45,878][20347] Updated weights for policy 0, policy_version 1691398 (0.0005) [2023-12-27 17:21:45,881][20381] Updated weights for policy 1, policy_version 1689726 (0.0007) [2023-12-27 17:21:45,946][20381] Updated weights for policy 1, policy_version 1689736 (0.0008) [2023-12-27 17:21:46,496][20347] Updated weights for policy 0, policy_version 1691408 (0.0005) [2023-12-27 17:21:46,549][20347] Updated weights for policy 0, policy_version 1691418 (0.0007) [2023-12-27 17:21:46,601][20347] Updated weights for policy 0, policy_version 1691428 (0.0005) [2023-12-27 17:21:46,607][20381] Updated weights for policy 1, policy_version 1689746 (0.0007) [2023-12-27 17:21:46,667][20381] Updated weights for policy 1, policy_version 1689756 (0.0005) [2023-12-27 17:21:46,722][20381] Updated weights for policy 1, policy_version 1689766 (0.0005) [2023-12-27 17:21:46,772][20381] Updated weights for policy 1, policy_version 1689776 (0.0005) [2023-12-27 17:21:47,271][20347] Updated weights for policy 0, policy_version 1691438 (0.0006) [2023-12-27 17:21:47,331][20347] Updated weights for policy 0, policy_version 1691448 (0.0007) [2023-12-27 17:21:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 865714176. Throughput: 0: 10485.4, 1: 10345.2. Samples: 865729764. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:47,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:21:47,383][20347] Updated weights for policy 0, policy_version 1691458 (0.0006) [2023-12-27 17:21:47,400][20381] Updated weights for policy 1, policy_version 1689786 (0.0011) [2023-12-27 17:21:47,461][20381] Updated weights for policy 1, policy_version 1689796 (0.0006) [2023-12-27 17:21:47,528][20381] Updated weights for policy 1, policy_version 1689806 (0.0006) [2023-12-27 17:21:48,023][20347] Updated weights for policy 0, policy_version 1691468 (0.0005) [2023-12-27 17:21:48,071][20347] Updated weights for policy 0, policy_version 1691478 (0.0006) [2023-12-27 17:21:48,085][20381] Updated weights for policy 1, policy_version 1689816 (0.0006) [2023-12-27 17:21:48,117][20347] Updated weights for policy 0, policy_version 1691488 (0.0006) [2023-12-27 17:21:48,149][20381] Updated weights for policy 1, policy_version 1689826 (0.0005) [2023-12-27 17:21:48,210][20381] Updated weights for policy 1, policy_version 1689836 (0.0006) [2023-12-27 17:21:48,783][20347] Updated weights for policy 0, policy_version 1691498 (0.0007) [2023-12-27 17:21:48,841][20347] Updated weights for policy 0, policy_version 1691508 (0.0008) [2023-12-27 17:21:48,896][20381] Updated weights for policy 1, policy_version 1689846 (0.0008) [2023-12-27 17:21:48,901][20347] Updated weights for policy 0, policy_version 1691518 (0.0007) [2023-12-27 17:21:48,954][20347] Updated weights for policy 0, policy_version 1691528 (0.0006) [2023-12-27 17:21:48,956][20381] Updated weights for policy 1, policy_version 1689856 (0.0010) [2023-12-27 17:21:49,016][20381] Updated weights for policy 1, policy_version 1689866 (0.0010) [2023-12-27 17:21:49,602][20347] Updated weights for policy 0, policy_version 1691538 (0.0007) [2023-12-27 17:21:49,663][20347] Updated weights for policy 0, policy_version 1691548 (0.0008) [2023-12-27 17:21:49,724][20347] Updated weights for policy 0, policy_version 1691558 (0.0007) [2023-12-27 17:21:49,726][20381] Updated weights for policy 1, policy_version 1689876 (0.0011) [2023-12-27 17:21:49,784][20381] Updated weights for policy 1, policy_version 1689886 (0.0008) [2023-12-27 17:21:49,847][20381] Updated weights for policy 1, policy_version 1689896 (0.0008) [2023-12-27 17:21:50,440][20347] Updated weights for policy 0, policy_version 1691568 (0.0007) [2023-12-27 17:21:50,501][20347] Updated weights for policy 0, policy_version 1691578 (0.0008) [2023-12-27 17:21:50,517][20381] Updated weights for policy 1, policy_version 1689906 (0.0011) [2023-12-27 17:21:50,552][20347] Updated weights for policy 0, policy_version 1691588 (0.0005) [2023-12-27 17:21:50,591][20381] Updated weights for policy 1, policy_version 1689916 (0.0009) [2023-12-27 17:21:50,647][20381] Updated weights for policy 1, policy_version 1689926 (0.0008) [2023-12-27 17:21:50,701][20381] Updated weights for policy 1, policy_version 1689936 (0.0010) [2023-12-27 17:21:51,282][20347] Updated weights for policy 0, policy_version 1691598 (0.0007) [2023-12-27 17:21:51,340][20347] Updated weights for policy 0, policy_version 1691608 (0.0008) [2023-12-27 17:21:51,409][20347] Updated weights for policy 0, policy_version 1691618 (0.0007) [2023-12-27 17:21:51,437][20381] Updated weights for policy 1, policy_version 1689946 (0.0008) [2023-12-27 17:21:51,499][20381] Updated weights for policy 1, policy_version 1689956 (0.0007) [2023-12-27 17:21:51,558][20381] Updated weights for policy 1, policy_version 1689966 (0.0007) [2023-12-27 17:21:52,126][20347] Updated weights for policy 0, policy_version 1691628 (0.0006) [2023-12-27 17:21:52,173][20347] Updated weights for policy 0, policy_version 1691638 (0.0005) [2023-12-27 17:21:52,225][20347] Updated weights for policy 0, policy_version 1691648 (0.0005) [2023-12-27 17:21:52,275][20381] Updated weights for policy 1, policy_version 1689976 (0.0007) [2023-12-27 17:21:52,336][20381] Updated weights for policy 1, policy_version 1689986 (0.0008) [2023-12-27 17:21:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 865820672. Throughput: 0: 10480.7, 1: 10352.8. Samples: 865792496. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:52,363][19170] Avg episode reward: [(0, '970.710'), (1, '1000.000')] [2023-12-27 17:21:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001691656_433143808.pth... [2023-12-27 17:21:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001690408_432824320.pth [2023-12-27 17:21:52,389][20381] Updated weights for policy 1, policy_version 1689996 (0.0006) [2023-12-27 17:21:52,408][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001690000_432685056.pth... [2023-12-27 17:21:52,411][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001688752_432365568.pth [2023-12-27 17:21:52,916][20347] Updated weights for policy 0, policy_version 1691658 (0.0007) [2023-12-27 17:21:52,968][20347] Updated weights for policy 0, policy_version 1691668 (0.0007) [2023-12-27 17:21:53,023][20347] Updated weights for policy 0, policy_version 1691678 (0.0007) [2023-12-27 17:21:53,036][20381] Updated weights for policy 1, policy_version 1690006 (0.0006) [2023-12-27 17:21:53,073][20347] Updated weights for policy 0, policy_version 1691688 (0.0007) [2023-12-27 17:21:53,088][20381] Updated weights for policy 1, policy_version 1690016 (0.0008) [2023-12-27 17:21:53,140][20381] Updated weights for policy 1, policy_version 1690026 (0.0008) [2023-12-27 17:21:53,786][20347] Updated weights for policy 0, policy_version 1691698 (0.0007) [2023-12-27 17:21:53,788][20381] Updated weights for policy 1, policy_version 1690036 (0.0008) [2023-12-27 17:21:53,832][20381] Updated weights for policy 1, policy_version 1690046 (0.0007) [2023-12-27 17:21:53,834][20347] Updated weights for policy 0, policy_version 1691708 (0.0006) [2023-12-27 17:21:53,889][20381] Updated weights for policy 1, policy_version 1690056 (0.0007) [2023-12-27 17:21:53,897][20347] Updated weights for policy 0, policy_version 1691718 (0.0005) [2023-12-27 17:21:54,551][20381] Updated weights for policy 1, policy_version 1690066 (0.0008) [2023-12-27 17:21:54,579][20347] Updated weights for policy 0, policy_version 1691728 (0.0005) [2023-12-27 17:21:54,600][20381] Updated weights for policy 1, policy_version 1690076 (0.0010) [2023-12-27 17:21:54,632][20347] Updated weights for policy 0, policy_version 1691738 (0.0006) [2023-12-27 17:21:54,655][20381] Updated weights for policy 1, policy_version 1690086 (0.0006) [2023-12-27 17:21:54,683][20347] Updated weights for policy 0, policy_version 1691748 (0.0008) [2023-12-27 17:21:54,711][20381] Updated weights for policy 1, policy_version 1690096 (0.0005) [2023-12-27 17:21:55,331][20381] Updated weights for policy 1, policy_version 1690106 (0.0011) [2023-12-27 17:21:55,377][20381] Updated weights for policy 1, policy_version 1690116 (0.0006) [2023-12-27 17:21:55,410][20347] Updated weights for policy 0, policy_version 1691758 (0.0007) [2023-12-27 17:21:55,431][20381] Updated weights for policy 1, policy_version 1690126 (0.0005) [2023-12-27 17:21:55,464][20347] Updated weights for policy 0, policy_version 1691768 (0.0007) [2023-12-27 17:21:55,516][20347] Updated weights for policy 0, policy_version 1691778 (0.0008) [2023-12-27 17:21:56,012][20381] Updated weights for policy 1, policy_version 1690136 (0.0006) [2023-12-27 17:21:56,067][20381] Updated weights for policy 1, policy_version 1690146 (0.0005) [2023-12-27 17:21:56,123][20381] Updated weights for policy 1, policy_version 1690156 (0.0005) [2023-12-27 17:21:56,266][20347] Updated weights for policy 0, policy_version 1691788 (0.0009) [2023-12-27 17:21:56,324][20347] Updated weights for policy 0, policy_version 1691798 (0.0007) [2023-12-27 17:21:56,376][20347] Updated weights for policy 0, policy_version 1691808 (0.0007) [2023-12-27 17:21:56,695][20381] Updated weights for policy 1, policy_version 1690166 (0.0005) [2023-12-27 17:21:56,756][20381] Updated weights for policy 1, policy_version 1690176 (0.0006) [2023-12-27 17:21:56,813][20381] Updated weights for policy 1, policy_version 1690186 (0.0010) [2023-12-27 17:21:57,086][20347] Updated weights for policy 0, policy_version 1691818 (0.0007) [2023-12-27 17:21:57,146][20347] Updated weights for policy 0, policy_version 1691828 (0.0007) [2023-12-27 17:21:57,209][20347] Updated weights for policy 0, policy_version 1691838 (0.0007) [2023-12-27 17:21:57,269][20347] Updated weights for policy 0, policy_version 1691848 (0.0007) [2023-12-27 17:21:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 865927168. Throughput: 0: 10447.0, 1: 10388.0. Samples: 865914844. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:21:57,363][19170] Avg episode reward: [(0, '970.720'), (1, '1000.000')] [2023-12-27 17:21:57,444][20381] Updated weights for policy 1, policy_version 1690196 (0.0009) [2023-12-27 17:21:57,489][20381] Updated weights for policy 1, policy_version 1690206 (0.0010) [2023-12-27 17:21:57,538][20381] Updated weights for policy 1, policy_version 1690216 (0.0010) [2023-12-27 17:21:57,935][20347] Updated weights for policy 0, policy_version 1691858 (0.0007) [2023-12-27 17:21:57,996][20347] Updated weights for policy 0, policy_version 1691868 (0.0006) [2023-12-27 17:21:58,047][20347] Updated weights for policy 0, policy_version 1691878 (0.0007) [2023-12-27 17:21:58,158][20381] Updated weights for policy 1, policy_version 1690226 (0.0010) [2023-12-27 17:21:58,215][20381] Updated weights for policy 1, policy_version 1690236 (0.0011) [2023-12-27 17:21:58,276][20381] Updated weights for policy 1, policy_version 1690246 (0.0011) [2023-12-27 17:21:58,340][20381] Updated weights for policy 1, policy_version 1690256 (0.0010) [2023-12-27 17:21:58,884][20347] Updated weights for policy 0, policy_version 1691888 (0.0006) [2023-12-27 17:21:58,947][20347] Updated weights for policy 0, policy_version 1691898 (0.0007) [2023-12-27 17:21:59,005][20347] Updated weights for policy 0, policy_version 1691908 (0.0007) [2023-12-27 17:21:59,090][20381] Updated weights for policy 1, policy_version 1690266 (0.0008) [2023-12-27 17:21:59,141][20381] Updated weights for policy 1, policy_version 1690276 (0.0008) [2023-12-27 17:21:59,200][20381] Updated weights for policy 1, policy_version 1690286 (0.0007) [2023-12-27 17:21:59,703][20347] Updated weights for policy 0, policy_version 1691918 (0.0007) [2023-12-27 17:21:59,769][20347] Updated weights for policy 0, policy_version 1691928 (0.0008) [2023-12-27 17:21:59,831][20347] Updated weights for policy 0, policy_version 1691938 (0.0007) [2023-12-27 17:21:59,839][20381] Updated weights for policy 1, policy_version 1690296 (0.0007) [2023-12-27 17:21:59,898][20381] Updated weights for policy 1, policy_version 1690306 (0.0006) [2023-12-27 17:21:59,961][20381] Updated weights for policy 1, policy_version 1690316 (0.0007) [2023-12-27 17:22:00,520][20347] Updated weights for policy 0, policy_version 1691948 (0.0008) [2023-12-27 17:22:00,567][20347] Updated weights for policy 0, policy_version 1691958 (0.0006) [2023-12-27 17:22:00,610][20381] Updated weights for policy 1, policy_version 1690326 (0.0008) [2023-12-27 17:22:00,616][20347] Updated weights for policy 0, policy_version 1691968 (0.0006) [2023-12-27 17:22:00,668][20381] Updated weights for policy 1, policy_version 1690336 (0.0010) [2023-12-27 17:22:00,722][20381] Updated weights for policy 1, policy_version 1690346 (0.0010) [2023-12-27 17:22:01,274][20347] Updated weights for policy 0, policy_version 1691978 (0.0009) [2023-12-27 17:22:01,330][20347] Updated weights for policy 0, policy_version 1691988 (0.0007) [2023-12-27 17:22:01,396][20347] Updated weights for policy 0, policy_version 1691998 (0.0009) [2023-12-27 17:22:01,445][20381] Updated weights for policy 1, policy_version 1690356 (0.0008) [2023-12-27 17:22:01,450][20347] Updated weights for policy 0, policy_version 1692008 (0.0008) [2023-12-27 17:22:01,491][20381] Updated weights for policy 1, policy_version 1690366 (0.0005) [2023-12-27 17:22:01,548][20381] Updated weights for policy 1, policy_version 1690376 (0.0006) [2023-12-27 17:22:02,171][20347] Updated weights for policy 0, policy_version 1692018 (0.0010) [2023-12-27 17:22:02,198][20381] Updated weights for policy 1, policy_version 1690386 (0.0006) [2023-12-27 17:22:02,230][20347] Updated weights for policy 0, policy_version 1692028 (0.0010) [2023-12-27 17:22:02,249][20381] Updated weights for policy 1, policy_version 1690396 (0.0007) [2023-12-27 17:22:02,287][20347] Updated weights for policy 0, policy_version 1692038 (0.0010) [2023-12-27 17:22:02,307][20381] Updated weights for policy 1, policy_version 1690406 (0.0007) [2023-12-27 17:22:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 866025472. Throughput: 0: 10367.8, 1: 10439.0. Samples: 866036420. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:02,363][19170] Avg episode reward: [(0, '970.720'), (1, '1000.000')] [2023-12-27 17:22:02,367][20381] Updated weights for policy 1, policy_version 1690416 (0.0008) [2023-12-27 17:22:02,944][20347] Updated weights for policy 0, policy_version 1692048 (0.0006) [2023-12-27 17:22:02,990][20347] Updated weights for policy 0, policy_version 1692058 (0.0007) [2023-12-27 17:22:03,020][20381] Updated weights for policy 1, policy_version 1690426 (0.0006) [2023-12-27 17:22:03,039][20347] Updated weights for policy 0, policy_version 1692068 (0.0005) [2023-12-27 17:22:03,080][20381] Updated weights for policy 1, policy_version 1690436 (0.0005) [2023-12-27 17:22:03,130][20381] Updated weights for policy 1, policy_version 1690446 (0.0008) [2023-12-27 17:22:03,617][20347] Updated weights for policy 0, policy_version 1692078 (0.0005) [2023-12-27 17:22:03,675][20347] Updated weights for policy 0, policy_version 1692088 (0.0008) [2023-12-27 17:22:03,729][20347] Updated weights for policy 0, policy_version 1692098 (0.0010) [2023-12-27 17:22:03,809][20381] Updated weights for policy 1, policy_version 1690456 (0.0011) [2023-12-27 17:22:03,860][20381] Updated weights for policy 1, policy_version 1690466 (0.0009) [2023-12-27 17:22:03,925][20381] Updated weights for policy 1, policy_version 1690476 (0.0007) [2023-12-27 17:22:04,379][20347] Updated weights for policy 0, policy_version 1692108 (0.0010) [2023-12-27 17:22:04,432][20347] Updated weights for policy 0, policy_version 1692118 (0.0010) [2023-12-27 17:22:04,488][20347] Updated weights for policy 0, policy_version 1692128 (0.0010) [2023-12-27 17:22:04,588][20381] Updated weights for policy 1, policy_version 1690486 (0.0007) [2023-12-27 17:22:04,643][20381] Updated weights for policy 1, policy_version 1690496 (0.0010) [2023-12-27 17:22:04,692][20381] Updated weights for policy 1, policy_version 1690506 (0.0011) [2023-12-27 17:22:05,122][20347] Updated weights for policy 0, policy_version 1692138 (0.0005) [2023-12-27 17:22:05,178][20347] Updated weights for policy 0, policy_version 1692148 (0.0006) [2023-12-27 17:22:05,235][20347] Updated weights for policy 0, policy_version 1692158 (0.0007) [2023-12-27 17:22:05,292][20347] Updated weights for policy 0, policy_version 1692168 (0.0007) [2023-12-27 17:22:05,404][20381] Updated weights for policy 1, policy_version 1690516 (0.0009) [2023-12-27 17:22:05,455][20381] Updated weights for policy 1, policy_version 1690526 (0.0005) [2023-12-27 17:22:05,503][20381] Updated weights for policy 1, policy_version 1690536 (0.0005) [2023-12-27 17:22:05,900][20347] Updated weights for policy 0, policy_version 1692178 (0.0007) [2023-12-27 17:22:05,954][20347] Updated weights for policy 0, policy_version 1692188 (0.0007) [2023-12-27 17:22:06,008][20347] Updated weights for policy 0, policy_version 1692198 (0.0007) [2023-12-27 17:22:06,110][20381] Updated weights for policy 1, policy_version 1690546 (0.0010) [2023-12-27 17:22:06,172][20381] Updated weights for policy 1, policy_version 1690556 (0.0010) [2023-12-27 17:22:06,232][20381] Updated weights for policy 1, policy_version 1690566 (0.0007) [2023-12-27 17:22:06,293][20381] Updated weights for policy 1, policy_version 1690576 (0.0006) [2023-12-27 17:22:06,712][20347] Updated weights for policy 0, policy_version 1692208 (0.0007) [2023-12-27 17:22:06,772][20347] Updated weights for policy 0, policy_version 1692218 (0.0008) [2023-12-27 17:22:06,827][20347] Updated weights for policy 0, policy_version 1692228 (0.0006) [2023-12-27 17:22:06,910][20381] Updated weights for policy 1, policy_version 1690586 (0.0005) [2023-12-27 17:22:06,967][20381] Updated weights for policy 1, policy_version 1690596 (0.0008) [2023-12-27 17:22:07,012][20381] Updated weights for policy 1, policy_version 1690606 (0.0007) [2023-12-27 17:22:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.2, 300 sec: 20549.4). Total num frames: 866131968. Throughput: 0: 10408.5, 1: 10462.5. Samples: 866099708. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:07,363][19170] Avg episode reward: [(0, '980.510'), (1, '1000.000')] [2023-12-27 17:22:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001692232_433291264.pth... [2023-12-27 17:22:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001690608_432840704.pth... [2023-12-27 17:22:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001689392_432529408.pth [2023-12-27 17:22:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001691048_432988160.pth [2023-12-27 17:22:07,451][20347] Updated weights for policy 0, policy_version 1692238 (0.0008) [2023-12-27 17:22:07,507][20347] Updated weights for policy 0, policy_version 1692248 (0.0010) [2023-12-27 17:22:07,558][20347] Updated weights for policy 0, policy_version 1692258 (0.0010) [2023-12-27 17:22:07,589][20381] Updated weights for policy 1, policy_version 1690616 (0.0005) [2023-12-27 17:22:07,640][20381] Updated weights for policy 1, policy_version 1690626 (0.0007) [2023-12-27 17:22:07,693][20381] Updated weights for policy 1, policy_version 1690636 (0.0007) [2023-12-27 17:22:08,183][20347] Updated weights for policy 0, policy_version 1692268 (0.0008) [2023-12-27 17:22:08,245][20347] Updated weights for policy 0, policy_version 1692278 (0.0005) [2023-12-27 17:22:08,309][20347] Updated weights for policy 0, policy_version 1692288 (0.0006) [2023-12-27 17:22:08,436][20381] Updated weights for policy 1, policy_version 1690646 (0.0007) [2023-12-27 17:22:08,453][20292] KL-divergence is very high: 124.8927 [2023-12-27 17:22:08,491][20381] Updated weights for policy 1, policy_version 1690656 (0.0007) [2023-12-27 17:22:08,500][20292] KL-divergence is very high: 108.4291 [2023-12-27 17:22:08,557][20381] Updated weights for policy 1, policy_version 1690666 (0.0007) [2023-12-27 17:22:09,003][20347] Updated weights for policy 0, policy_version 1692298 (0.0010) [2023-12-27 17:22:09,060][20347] Updated weights for policy 0, policy_version 1692308 (0.0006) [2023-12-27 17:22:09,126][20347] Updated weights for policy 0, policy_version 1692318 (0.0005) [2023-12-27 17:22:09,154][20381] Updated weights for policy 1, policy_version 1690676 (0.0007) [2023-12-27 17:22:09,181][20347] Updated weights for policy 0, policy_version 1692328 (0.0005) [2023-12-27 17:22:09,215][20381] Updated weights for policy 1, policy_version 1690686 (0.0006) [2023-12-27 17:22:09,279][20381] Updated weights for policy 1, policy_version 1690696 (0.0008) [2023-12-27 17:22:09,902][20347] Updated weights for policy 0, policy_version 1692338 (0.0008) [2023-12-27 17:22:09,944][20381] Updated weights for policy 1, policy_version 1690706 (0.0008) [2023-12-27 17:22:09,968][20347] Updated weights for policy 0, policy_version 1692348 (0.0007) [2023-12-27 17:22:10,010][20381] Updated weights for policy 1, policy_version 1690716 (0.0007) [2023-12-27 17:22:10,038][20347] Updated weights for policy 0, policy_version 1692358 (0.0007) [2023-12-27 17:22:10,067][20381] Updated weights for policy 1, policy_version 1690726 (0.0006) [2023-12-27 17:22:10,132][20381] Updated weights for policy 1, policy_version 1690736 (0.0006) [2023-12-27 17:22:10,739][20381] Updated weights for policy 1, policy_version 1690746 (0.0007) [2023-12-27 17:22:10,740][20347] Updated weights for policy 0, policy_version 1692368 (0.0006) [2023-12-27 17:22:10,790][20381] Updated weights for policy 1, policy_version 1690756 (0.0006) [2023-12-27 17:22:10,796][20347] Updated weights for policy 0, policy_version 1692378 (0.0006) [2023-12-27 17:22:10,843][20381] Updated weights for policy 1, policy_version 1690766 (0.0006) [2023-12-27 17:22:10,849][20347] Updated weights for policy 0, policy_version 1692388 (0.0005) [2023-12-27 17:22:11,583][20347] Updated weights for policy 0, policy_version 1692398 (0.0007) [2023-12-27 17:22:11,601][20381] Updated weights for policy 1, policy_version 1690776 (0.0008) [2023-12-27 17:22:11,655][20347] Updated weights for policy 0, policy_version 1692408 (0.0008) [2023-12-27 17:22:11,667][20381] Updated weights for policy 1, policy_version 1690786 (0.0007) [2023-12-27 17:22:11,717][20347] Updated weights for policy 0, policy_version 1692418 (0.0008) [2023-12-27 17:22:11,733][20381] Updated weights for policy 1, policy_version 1690796 (0.0007) [2023-12-27 17:22:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 866230272. Throughput: 0: 10378.5, 1: 10491.6. Samples: 866225364. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.250')] [2023-12-27 17:22:12,372][20347] Updated weights for policy 0, policy_version 1692428 (0.0008) [2023-12-27 17:22:12,436][20347] Updated weights for policy 0, policy_version 1692438 (0.0007) [2023-12-27 17:22:12,460][20381] Updated weights for policy 1, policy_version 1690806 (0.0008) [2023-12-27 17:22:12,491][20347] Updated weights for policy 0, policy_version 1692448 (0.0006) [2023-12-27 17:22:12,515][20381] Updated weights for policy 1, policy_version 1690816 (0.0007) [2023-12-27 17:22:12,568][20381] Updated weights for policy 1, policy_version 1690826 (0.0007) [2023-12-27 17:22:13,184][20347] Updated weights for policy 0, policy_version 1692458 (0.0007) [2023-12-27 17:22:13,239][20347] Updated weights for policy 0, policy_version 1692468 (0.0007) [2023-12-27 17:22:13,240][20381] Updated weights for policy 1, policy_version 1690836 (0.0007) [2023-12-27 17:22:13,287][20347] Updated weights for policy 0, policy_version 1692478 (0.0005) [2023-12-27 17:22:13,292][20381] Updated weights for policy 1, policy_version 1690846 (0.0008) [2023-12-27 17:22:13,343][20347] Updated weights for policy 0, policy_version 1692488 (0.0005) [2023-12-27 17:22:13,345][20381] Updated weights for policy 1, policy_version 1690856 (0.0008) [2023-12-27 17:22:13,977][20347] Updated weights for policy 0, policy_version 1692498 (0.0006) [2023-12-27 17:22:14,032][20347] Updated weights for policy 0, policy_version 1692508 (0.0007) [2023-12-27 17:22:14,060][20381] Updated weights for policy 1, policy_version 1690866 (0.0008) [2023-12-27 17:22:14,091][20347] Updated weights for policy 0, policy_version 1692518 (0.0008) [2023-12-27 17:22:14,117][20381] Updated weights for policy 1, policy_version 1690876 (0.0007) [2023-12-27 17:22:14,173][20381] Updated weights for policy 1, policy_version 1690886 (0.0008) [2023-12-27 17:22:14,223][20381] Updated weights for policy 1, policy_version 1690896 (0.0008) [2023-12-27 17:22:14,719][20347] Updated weights for policy 0, policy_version 1692528 (0.0007) [2023-12-27 17:22:14,780][20347] Updated weights for policy 0, policy_version 1692538 (0.0009) [2023-12-27 17:22:14,842][20347] Updated weights for policy 0, policy_version 1692548 (0.0008) [2023-12-27 17:22:14,943][20381] Updated weights for policy 1, policy_version 1690906 (0.0008) [2023-12-27 17:22:14,999][20381] Updated weights for policy 1, policy_version 1690916 (0.0007) [2023-12-27 17:22:15,055][20381] Updated weights for policy 1, policy_version 1690926 (0.0007) [2023-12-27 17:22:15,515][20347] Updated weights for policy 0, policy_version 1692558 (0.0006) [2023-12-27 17:22:15,579][20347] Updated weights for policy 0, policy_version 1692568 (0.0005) [2023-12-27 17:22:15,636][20347] Updated weights for policy 0, policy_version 1692578 (0.0007) [2023-12-27 17:22:15,803][20381] Updated weights for policy 1, policy_version 1690936 (0.0006) [2023-12-27 17:22:15,863][20381] Updated weights for policy 1, policy_version 1690946 (0.0008) [2023-12-27 17:22:15,925][20381] Updated weights for policy 1, policy_version 1690956 (0.0008) [2023-12-27 17:22:16,196][20347] Updated weights for policy 0, policy_version 1692588 (0.0009) [2023-12-27 17:22:16,245][20347] Updated weights for policy 0, policy_version 1692598 (0.0010) [2023-12-27 17:22:16,303][20347] Updated weights for policy 0, policy_version 1692608 (0.0010) [2023-12-27 17:22:16,625][20381] Updated weights for policy 1, policy_version 1690966 (0.0008) [2023-12-27 17:22:16,686][20381] Updated weights for policy 1, policy_version 1690976 (0.0007) [2023-12-27 17:22:16,744][20381] Updated weights for policy 1, policy_version 1690986 (0.0008) [2023-12-27 17:22:17,002][20347] Updated weights for policy 0, policy_version 1692618 (0.0011) [2023-12-27 17:22:17,058][20347] Updated weights for policy 0, policy_version 1692628 (0.0010) [2023-12-27 17:22:17,111][20347] Updated weights for policy 0, policy_version 1692638 (0.0010) [2023-12-27 17:22:17,166][20347] Updated weights for policy 0, policy_version 1692648 (0.0011) [2023-12-27 17:22:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 866336768. Throughput: 0: 10283.9, 1: 10348.6. Samples: 866346772. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:17,363][19170] Avg episode reward: [(0, '990.260'), (1, '980.510')] [2023-12-27 17:22:17,417][20381] Updated weights for policy 1, policy_version 1690996 (0.0007) [2023-12-27 17:22:17,470][20381] Updated weights for policy 1, policy_version 1691006 (0.0008) [2023-12-27 17:22:17,528][20381] Updated weights for policy 1, policy_version 1691016 (0.0008) [2023-12-27 17:22:17,834][20347] Updated weights for policy 0, policy_version 1692658 (0.0008) [2023-12-27 17:22:17,890][20347] Updated weights for policy 0, policy_version 1692668 (0.0007) [2023-12-27 17:22:17,942][20347] Updated weights for policy 0, policy_version 1692678 (0.0006) [2023-12-27 17:22:18,093][20381] Updated weights for policy 1, policy_version 1691026 (0.0007) [2023-12-27 17:22:18,149][20381] Updated weights for policy 1, policy_version 1691036 (0.0007) [2023-12-27 17:22:18,209][20381] Updated weights for policy 1, policy_version 1691046 (0.0005) [2023-12-27 17:22:18,270][20381] Updated weights for policy 1, policy_version 1691056 (0.0005) [2023-12-27 17:22:18,640][20347] Updated weights for policy 0, policy_version 1692688 (0.0006) [2023-12-27 17:22:18,705][20347] Updated weights for policy 0, policy_version 1692698 (0.0006) [2023-12-27 17:22:18,764][20347] Updated weights for policy 0, policy_version 1692708 (0.0005) [2023-12-27 17:22:18,977][20381] Updated weights for policy 1, policy_version 1691066 (0.0007) [2023-12-27 17:22:19,023][20381] Updated weights for policy 1, policy_version 1691076 (0.0008) [2023-12-27 17:22:19,076][20381] Updated weights for policy 1, policy_version 1691086 (0.0009) [2023-12-27 17:22:19,405][20347] Updated weights for policy 0, policy_version 1692718 (0.0008) [2023-12-27 17:22:19,466][20347] Updated weights for policy 0, policy_version 1692728 (0.0007) [2023-12-27 17:22:19,529][20347] Updated weights for policy 0, policy_version 1692738 (0.0008) [2023-12-27 17:22:19,806][20381] Updated weights for policy 1, policy_version 1691096 (0.0008) [2023-12-27 17:22:19,864][20381] Updated weights for policy 1, policy_version 1691106 (0.0008) [2023-12-27 17:22:19,918][20381] Updated weights for policy 1, policy_version 1691116 (0.0008) [2023-12-27 17:22:20,261][20347] Updated weights for policy 0, policy_version 1692748 (0.0008) [2023-12-27 17:22:20,317][20347] Updated weights for policy 0, policy_version 1692758 (0.0008) [2023-12-27 17:22:20,382][20347] Updated weights for policy 0, policy_version 1692768 (0.0007) [2023-12-27 17:22:20,649][20381] Updated weights for policy 1, policy_version 1691126 (0.0008) [2023-12-27 17:22:20,708][20381] Updated weights for policy 1, policy_version 1691136 (0.0008) [2023-12-27 17:22:20,763][20381] Updated weights for policy 1, policy_version 1691146 (0.0008) [2023-12-27 17:22:21,124][20347] Updated weights for policy 0, policy_version 1692778 (0.0009) [2023-12-27 17:22:21,182][20347] Updated weights for policy 0, policy_version 1692788 (0.0008) [2023-12-27 17:22:21,235][20347] Updated weights for policy 0, policy_version 1692798 (0.0007) [2023-12-27 17:22:21,296][20347] Updated weights for policy 0, policy_version 1692808 (0.0008) [2023-12-27 17:22:21,521][20381] Updated weights for policy 1, policy_version 1691156 (0.0007) [2023-12-27 17:22:21,584][20381] Updated weights for policy 1, policy_version 1691166 (0.0008) [2023-12-27 17:22:21,649][20381] Updated weights for policy 1, policy_version 1691176 (0.0009) [2023-12-27 17:22:22,048][20347] Updated weights for policy 0, policy_version 1692818 (0.0006) [2023-12-27 17:22:22,108][20347] Updated weights for policy 0, policy_version 1692828 (0.0007) [2023-12-27 17:22:22,161][20347] Updated weights for policy 0, policy_version 1692838 (0.0008) [2023-12-27 17:22:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 866435072. Throughput: 0: 10263.9, 1: 10356.5. Samples: 866407548. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:22,363][19170] Avg episode reward: [(0, '990.260'), (1, '1000.000')] [2023-12-27 17:22:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001692840_433446912.pth... [2023-12-27 17:22:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001691184_432988160.pth... [2023-12-27 17:22:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001691656_433143808.pth [2023-12-27 17:22:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001690000_432685056.pth [2023-12-27 17:22:22,402][20381] Updated weights for policy 1, policy_version 1691186 (0.0009) [2023-12-27 17:22:22,455][20381] Updated weights for policy 1, policy_version 1691196 (0.0008) [2023-12-27 17:22:22,512][20381] Updated weights for policy 1, policy_version 1691206 (0.0007) [2023-12-27 17:22:22,566][20381] Updated weights for policy 1, policy_version 1691216 (0.0007) [2023-12-27 17:22:22,857][20347] Updated weights for policy 0, policy_version 1692848 (0.0008) [2023-12-27 17:22:22,917][20347] Updated weights for policy 0, policy_version 1692858 (0.0008) [2023-12-27 17:22:22,984][20347] Updated weights for policy 0, policy_version 1692868 (0.0006) [2023-12-27 17:22:23,280][20381] Updated weights for policy 1, policy_version 1691226 (0.0006) [2023-12-27 17:22:23,336][20381] Updated weights for policy 1, policy_version 1691236 (0.0005) [2023-12-27 17:22:23,399][20381] Updated weights for policy 1, policy_version 1691246 (0.0005) [2023-12-27 17:22:23,625][20347] Updated weights for policy 0, policy_version 1692878 (0.0005) [2023-12-27 17:22:23,687][20347] Updated weights for policy 0, policy_version 1692888 (0.0005) [2023-12-27 17:22:23,750][20347] Updated weights for policy 0, policy_version 1692898 (0.0005) [2023-12-27 17:22:23,982][20381] Updated weights for policy 1, policy_version 1691256 (0.0006) [2023-12-27 17:22:24,043][20381] Updated weights for policy 1, policy_version 1691266 (0.0005) [2023-12-27 17:22:24,095][20381] Updated weights for policy 1, policy_version 1691276 (0.0005) [2023-12-27 17:22:24,337][20347] Updated weights for policy 0, policy_version 1692908 (0.0007) [2023-12-27 17:22:24,399][20347] Updated weights for policy 0, policy_version 1692918 (0.0008) [2023-12-27 17:22:24,456][20347] Updated weights for policy 0, policy_version 1692928 (0.0009) [2023-12-27 17:22:24,617][20381] Updated weights for policy 1, policy_version 1691286 (0.0005) [2023-12-27 17:22:24,684][20381] Updated weights for policy 1, policy_version 1691296 (0.0006) [2023-12-27 17:22:24,743][20381] Updated weights for policy 1, policy_version 1691306 (0.0008) [2023-12-27 17:22:25,141][20347] Updated weights for policy 0, policy_version 1692938 (0.0008) [2023-12-27 17:22:25,189][20347] Updated weights for policy 0, policy_version 1692948 (0.0007) [2023-12-27 17:22:25,241][20347] Updated weights for policy 0, policy_version 1692958 (0.0006) [2023-12-27 17:22:25,291][20347] Updated weights for policy 0, policy_version 1692968 (0.0005) [2023-12-27 17:22:25,364][20381] Updated weights for policy 1, policy_version 1691316 (0.0009) [2023-12-27 17:22:25,409][20381] Updated weights for policy 1, policy_version 1691326 (0.0007) [2023-12-27 17:22:25,466][20381] Updated weights for policy 1, policy_version 1691336 (0.0006) [2023-12-27 17:22:25,969][20347] Updated weights for policy 0, policy_version 1692978 (0.0007) [2023-12-27 17:22:26,023][20347] Updated weights for policy 0, policy_version 1692988 (0.0008) [2023-12-27 17:22:26,078][20347] Updated weights for policy 0, policy_version 1692998 (0.0008) [2023-12-27 17:22:26,110][20381] Updated weights for policy 1, policy_version 1691346 (0.0006) [2023-12-27 17:22:26,157][20381] Updated weights for policy 1, policy_version 1691356 (0.0008) [2023-12-27 17:22:26,203][20381] Updated weights for policy 1, policy_version 1691366 (0.0007) [2023-12-27 17:22:26,250][20381] Updated weights for policy 1, policy_version 1691376 (0.0008) [2023-12-27 17:22:26,711][20347] Updated weights for policy 0, policy_version 1693008 (0.0007) [2023-12-27 17:22:26,754][20347] Updated weights for policy 0, policy_version 1693018 (0.0005) [2023-12-27 17:22:26,807][20347] Updated weights for policy 0, policy_version 1693028 (0.0005) [2023-12-27 17:22:27,003][20381] Updated weights for policy 1, policy_version 1691386 (0.0008) [2023-12-27 17:22:27,064][20381] Updated weights for policy 1, policy_version 1691396 (0.0007) [2023-12-27 17:22:27,119][20381] Updated weights for policy 1, policy_version 1691406 (0.0007) [2023-12-27 17:22:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 866541568. Throughput: 0: 10198.0, 1: 10362.0. Samples: 866530940. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:22:27,423][20347] Updated weights for policy 0, policy_version 1693038 (0.0006) [2023-12-27 17:22:27,471][20347] Updated weights for policy 0, policy_version 1693048 (0.0007) [2023-12-27 17:22:27,520][20347] Updated weights for policy 0, policy_version 1693058 (0.0008) [2023-12-27 17:22:27,691][20381] Updated weights for policy 1, policy_version 1691416 (0.0008) [2023-12-27 17:22:27,736][20381] Updated weights for policy 1, policy_version 1691426 (0.0008) [2023-12-27 17:22:27,781][20381] Updated weights for policy 1, policy_version 1691436 (0.0007) [2023-12-27 17:22:28,176][20347] Updated weights for policy 0, policy_version 1693068 (0.0007) [2023-12-27 17:22:28,231][20347] Updated weights for policy 0, policy_version 1693078 (0.0006) [2023-12-27 17:22:28,288][20347] Updated weights for policy 0, policy_version 1693088 (0.0005) [2023-12-27 17:22:28,491][20381] Updated weights for policy 1, policy_version 1691446 (0.0007) [2023-12-27 17:22:28,556][20381] Updated weights for policy 1, policy_version 1691456 (0.0008) [2023-12-27 17:22:28,623][20381] Updated weights for policy 1, policy_version 1691466 (0.0007) [2023-12-27 17:22:28,958][20347] Updated weights for policy 0, policy_version 1693098 (0.0007) [2023-12-27 17:22:29,006][20347] Updated weights for policy 0, policy_version 1693108 (0.0008) [2023-12-27 17:22:29,060][20347] Updated weights for policy 0, policy_version 1693118 (0.0007) [2023-12-27 17:22:29,107][20347] Updated weights for policy 0, policy_version 1693128 (0.0008) [2023-12-27 17:22:29,317][20381] Updated weights for policy 1, policy_version 1691476 (0.0009) [2023-12-27 17:22:29,396][20381] Updated weights for policy 1, policy_version 1691486 (0.0009) [2023-12-27 17:22:29,455][20381] Updated weights for policy 1, policy_version 1691496 (0.0008) [2023-12-27 17:22:29,827][20347] Updated weights for policy 0, policy_version 1693138 (0.0007) [2023-12-27 17:22:29,884][20347] Updated weights for policy 0, policy_version 1693148 (0.0008) [2023-12-27 17:22:29,946][20347] Updated weights for policy 0, policy_version 1693158 (0.0007) [2023-12-27 17:22:30,089][20381] Updated weights for policy 1, policy_version 1691506 (0.0008) [2023-12-27 17:22:30,153][20381] Updated weights for policy 1, policy_version 1691516 (0.0008) [2023-12-27 17:22:30,213][20381] Updated weights for policy 1, policy_version 1691526 (0.0007) [2023-12-27 17:22:30,273][20381] Updated weights for policy 1, policy_version 1691536 (0.0008) [2023-12-27 17:22:30,543][20347] Updated weights for policy 0, policy_version 1693168 (0.0007) [2023-12-27 17:22:30,589][20347] Updated weights for policy 0, policy_version 1693178 (0.0005) [2023-12-27 17:22:30,643][20347] Updated weights for policy 0, policy_version 1693188 (0.0006) [2023-12-27 17:22:30,928][20381] Updated weights for policy 1, policy_version 1691546 (0.0007) [2023-12-27 17:22:30,988][20381] Updated weights for policy 1, policy_version 1691556 (0.0008) [2023-12-27 17:22:31,043][20381] Updated weights for policy 1, policy_version 1691566 (0.0006) [2023-12-27 17:22:31,265][20347] Updated weights for policy 0, policy_version 1693198 (0.0008) [2023-12-27 17:22:31,314][20347] Updated weights for policy 0, policy_version 1693208 (0.0007) [2023-12-27 17:22:31,382][20347] Updated weights for policy 0, policy_version 1693218 (0.0008) [2023-12-27 17:22:31,808][20381] Updated weights for policy 1, policy_version 1691576 (0.0008) [2023-12-27 17:22:31,867][20381] Updated weights for policy 1, policy_version 1691586 (0.0007) [2023-12-27 17:22:31,933][20381] Updated weights for policy 1, policy_version 1691596 (0.0008) [2023-12-27 17:22:32,028][20347] Updated weights for policy 0, policy_version 1693228 (0.0007) [2023-12-27 17:22:32,081][20347] Updated weights for policy 0, policy_version 1693238 (0.0007) [2023-12-27 17:22:32,131][20347] Updated weights for policy 0, policy_version 1693248 (0.0007) [2023-12-27 17:22:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 866648064. Throughput: 0: 10261.2, 1: 10331.3. Samples: 866656428. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 17:22:32,363][19170] Avg episode reward: [(0, '980.530'), (1, '1000.000')] [2023-12-27 17:22:32,573][20381] Updated weights for policy 1, policy_version 1691606 (0.0007) [2023-12-27 17:22:32,638][20381] Updated weights for policy 1, policy_version 1691616 (0.0006) [2023-12-27 17:22:32,702][20381] Updated weights for policy 1, policy_version 1691626 (0.0007) [2023-12-27 17:22:32,828][20347] Updated weights for policy 0, policy_version 1693258 (0.0006) [2023-12-27 17:22:32,878][20347] Updated weights for policy 0, policy_version 1693268 (0.0007) [2023-12-27 17:22:32,937][20347] Updated weights for policy 0, policy_version 1693278 (0.0008) [2023-12-27 17:22:32,991][20347] Updated weights for policy 0, policy_version 1693288 (0.0007) [2023-12-27 17:22:33,343][20381] Updated weights for policy 1, policy_version 1691636 (0.0007) [2023-12-27 17:22:33,398][20381] Updated weights for policy 1, policy_version 1691646 (0.0007) [2023-12-27 17:22:33,449][20381] Updated weights for policy 1, policy_version 1691656 (0.0007) [2023-12-27 17:22:33,630][20347] Updated weights for policy 0, policy_version 1693298 (0.0008) [2023-12-27 17:22:33,685][20347] Updated weights for policy 0, policy_version 1693308 (0.0008) [2023-12-27 17:22:33,740][20347] Updated weights for policy 0, policy_version 1693318 (0.0007) [2023-12-27 17:22:34,132][20381] Updated weights for policy 1, policy_version 1691666 (0.0008) [2023-12-27 17:22:34,199][20381] Updated weights for policy 1, policy_version 1691676 (0.0009) [2023-12-27 17:22:34,268][20381] Updated weights for policy 1, policy_version 1691686 (0.0008) [2023-12-27 17:22:34,335][20381] Updated weights for policy 1, policy_version 1691696 (0.0008) [2023-12-27 17:22:34,404][20347] Updated weights for policy 0, policy_version 1693328 (0.0007) [2023-12-27 17:22:34,465][20347] Updated weights for policy 0, policy_version 1693338 (0.0007) [2023-12-27 17:22:34,524][20347] Updated weights for policy 0, policy_version 1693348 (0.0007) [2023-12-27 17:22:35,021][20381] Updated weights for policy 1, policy_version 1691706 (0.0010) [2023-12-27 17:22:35,077][20381] Updated weights for policy 1, policy_version 1691716 (0.0011) [2023-12-27 17:22:35,129][20381] Updated weights for policy 1, policy_version 1691726 (0.0010) [2023-12-27 17:22:35,190][20347] Updated weights for policy 0, policy_version 1693358 (0.0008) [2023-12-27 17:22:35,254][20347] Updated weights for policy 0, policy_version 1693368 (0.0009) [2023-12-27 17:22:35,309][20347] Updated weights for policy 0, policy_version 1693378 (0.0008) [2023-12-27 17:22:35,827][20381] Updated weights for policy 1, policy_version 1691736 (0.0011) [2023-12-27 17:22:35,877][20381] Updated weights for policy 1, policy_version 1691746 (0.0011) [2023-12-27 17:22:35,926][20381] Updated weights for policy 1, policy_version 1691756 (0.0009) [2023-12-27 17:22:36,020][20347] Updated weights for policy 0, policy_version 1693388 (0.0007) [2023-12-27 17:22:36,076][20347] Updated weights for policy 0, policy_version 1693398 (0.0009) [2023-12-27 17:22:36,144][20347] Updated weights for policy 0, policy_version 1693408 (0.0008) [2023-12-27 17:22:36,615][20381] Updated weights for policy 1, policy_version 1691766 (0.0007) [2023-12-27 17:22:36,669][20381] Updated weights for policy 1, policy_version 1691776 (0.0007) [2023-12-27 17:22:36,731][20381] Updated weights for policy 1, policy_version 1691786 (0.0008) [2023-12-27 17:22:36,904][20347] Updated weights for policy 0, policy_version 1693418 (0.0008) [2023-12-27 17:22:36,963][20347] Updated weights for policy 0, policy_version 1693428 (0.0008) [2023-12-27 17:22:37,013][20347] Updated weights for policy 0, policy_version 1693438 (0.0008) [2023-12-27 17:22:37,068][20347] Updated weights for policy 0, policy_version 1693448 (0.0008) [2023-12-27 17:22:37,338][20381] Updated weights for policy 1, policy_version 1691796 (0.0007) [2023-12-27 17:22:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 866746368. Throughput: 0: 10241.7, 1: 10333.1. Samples: 866718364. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:22:37,363][19170] Avg episode reward: [(0, '960.940'), (1, '1000.000')] [2023-12-27 17:22:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001693448_433602560.pth... [2023-12-27 17:22:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001692232_433291264.pth [2023-12-27 17:22:37,390][20381] Updated weights for policy 1, policy_version 1691806 (0.0005) [2023-12-27 17:22:37,447][20381] Updated weights for policy 1, policy_version 1691816 (0.0007) [2023-12-27 17:22:37,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001691824_433152000.pth... [2023-12-27 17:22:37,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001690608_432840704.pth [2023-12-27 17:22:37,820][20347] Updated weights for policy 0, policy_version 1693458 (0.0007) [2023-12-27 17:22:37,892][20347] Updated weights for policy 0, policy_version 1693468 (0.0007) [2023-12-27 17:22:37,950][20347] Updated weights for policy 0, policy_version 1693478 (0.0008) [2023-12-27 17:22:38,105][20381] Updated weights for policy 1, policy_version 1691826 (0.0007) [2023-12-27 17:22:38,152][20381] Updated weights for policy 1, policy_version 1691836 (0.0005) [2023-12-27 17:22:38,206][20381] Updated weights for policy 1, policy_version 1691846 (0.0007) [2023-12-27 17:22:38,260][20381] Updated weights for policy 1, policy_version 1691856 (0.0007) [2023-12-27 17:22:38,636][20347] Updated weights for policy 0, policy_version 1693488 (0.0006) [2023-12-27 17:22:38,698][20347] Updated weights for policy 0, policy_version 1693498 (0.0007) [2023-12-27 17:22:38,756][20347] Updated weights for policy 0, policy_version 1693508 (0.0006) [2023-12-27 17:22:38,886][20381] Updated weights for policy 1, policy_version 1691866 (0.0010) [2023-12-27 17:22:38,943][20381] Updated weights for policy 1, policy_version 1691876 (0.0010) [2023-12-27 17:22:39,001][20381] Updated weights for policy 1, policy_version 1691886 (0.0010) [2023-12-27 17:22:39,449][20347] Updated weights for policy 0, policy_version 1693518 (0.0007) [2023-12-27 17:22:39,496][20347] Updated weights for policy 0, policy_version 1693528 (0.0008) [2023-12-27 17:22:39,556][20347] Updated weights for policy 0, policy_version 1693538 (0.0008) [2023-12-27 17:22:39,738][20381] Updated weights for policy 1, policy_version 1691896 (0.0011) [2023-12-27 17:22:39,797][20381] Updated weights for policy 1, policy_version 1691906 (0.0011) [2023-12-27 17:22:39,861][20381] Updated weights for policy 1, policy_version 1691916 (0.0008) [2023-12-27 17:22:40,368][20347] Updated weights for policy 0, policy_version 1693548 (0.0008) [2023-12-27 17:22:40,427][20347] Updated weights for policy 0, policy_version 1693558 (0.0008) [2023-12-27 17:22:40,491][20347] Updated weights for policy 0, policy_version 1693568 (0.0008) [2023-12-27 17:22:40,547][20381] Updated weights for policy 1, policy_version 1691926 (0.0009) [2023-12-27 17:22:40,606][20381] Updated weights for policy 1, policy_version 1691936 (0.0011) [2023-12-27 17:22:40,666][20381] Updated weights for policy 1, policy_version 1691946 (0.0009) [2023-12-27 17:22:41,230][20347] Updated weights for policy 0, policy_version 1693578 (0.0006) [2023-12-27 17:22:41,292][20347] Updated weights for policy 0, policy_version 1693588 (0.0007) [2023-12-27 17:22:41,309][20381] Updated weights for policy 1, policy_version 1691956 (0.0009) [2023-12-27 17:22:41,351][20347] Updated weights for policy 0, policy_version 1693598 (0.0007) [2023-12-27 17:22:41,382][20381] Updated weights for policy 1, policy_version 1691966 (0.0010) [2023-12-27 17:22:41,419][20347] Updated weights for policy 0, policy_version 1693608 (0.0007) [2023-12-27 17:22:41,447][20381] Updated weights for policy 1, policy_version 1691976 (0.0008) [2023-12-27 17:22:42,129][20347] Updated weights for policy 0, policy_version 1693618 (0.0006) [2023-12-27 17:22:42,165][20381] Updated weights for policy 1, policy_version 1691986 (0.0009) [2023-12-27 17:22:42,191][20347] Updated weights for policy 0, policy_version 1693628 (0.0006) [2023-12-27 17:22:42,237][20381] Updated weights for policy 1, policy_version 1691996 (0.0008) [2023-12-27 17:22:42,249][20347] Updated weights for policy 0, policy_version 1693638 (0.0007) [2023-12-27 17:22:42,306][20381] Updated weights for policy 1, policy_version 1692006 (0.0008) [2023-12-27 17:22:42,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 866844672. Throughput: 0: 10199.9, 1: 10311.0. Samples: 866837836. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:22:42,363][19170] Avg episode reward: [(0, '980.410'), (1, '1000.000')] [2023-12-27 17:22:42,374][20381] Updated weights for policy 1, policy_version 1692016 (0.0008) [2023-12-27 17:22:42,950][20347] Updated weights for policy 0, policy_version 1693648 (0.0006) [2023-12-27 17:22:43,001][20347] Updated weights for policy 0, policy_version 1693658 (0.0008) [2023-12-27 17:22:43,014][20381] Updated weights for policy 1, policy_version 1692026 (0.0006) [2023-12-27 17:22:43,057][20347] Updated weights for policy 0, policy_version 1693668 (0.0007) [2023-12-27 17:22:43,070][20381] Updated weights for policy 1, policy_version 1692036 (0.0005) [2023-12-27 17:22:43,129][20381] Updated weights for policy 1, policy_version 1692046 (0.0007) [2023-12-27 17:22:43,688][20381] Updated weights for policy 1, policy_version 1692056 (0.0007) [2023-12-27 17:22:43,742][20381] Updated weights for policy 1, policy_version 1692066 (0.0007) [2023-12-27 17:22:43,769][20347] Updated weights for policy 0, policy_version 1693678 (0.0007) [2023-12-27 17:22:43,795][20381] Updated weights for policy 1, policy_version 1692076 (0.0006) [2023-12-27 17:22:43,813][20347] Updated weights for policy 0, policy_version 1693688 (0.0006) [2023-12-27 17:22:43,861][20347] Updated weights for policy 0, policy_version 1693698 (0.0007) [2023-12-27 17:22:44,379][20381] Updated weights for policy 1, policy_version 1692086 (0.0007) [2023-12-27 17:22:44,425][20381] Updated weights for policy 1, policy_version 1692096 (0.0010) [2023-12-27 17:22:44,473][20347] Updated weights for policy 0, policy_version 1693708 (0.0007) [2023-12-27 17:22:44,474][20381] Updated weights for policy 1, policy_version 1692106 (0.0010) [2023-12-27 17:22:44,530][20347] Updated weights for policy 0, policy_version 1693718 (0.0007) [2023-12-27 17:22:44,584][20347] Updated weights for policy 0, policy_version 1693728 (0.0008) [2023-12-27 17:22:45,090][20381] Updated weights for policy 1, policy_version 1692116 (0.0009) [2023-12-27 17:22:45,144][20381] Updated weights for policy 1, policy_version 1692126 (0.0008) [2023-12-27 17:22:45,198][20381] Updated weights for policy 1, policy_version 1692136 (0.0011) [2023-12-27 17:22:45,288][20347] Updated weights for policy 0, policy_version 1693738 (0.0009) [2023-12-27 17:22:45,347][20347] Updated weights for policy 0, policy_version 1693748 (0.0005) [2023-12-27 17:22:45,400][20347] Updated weights for policy 0, policy_version 1693758 (0.0007) [2023-12-27 17:22:45,452][20347] Updated weights for policy 0, policy_version 1693768 (0.0007) [2023-12-27 17:22:45,788][20381] Updated weights for policy 1, policy_version 1692146 (0.0007) [2023-12-27 17:22:45,852][20381] Updated weights for policy 1, policy_version 1692156 (0.0010) [2023-12-27 17:22:45,919][20381] Updated weights for policy 1, policy_version 1692166 (0.0009) [2023-12-27 17:22:45,982][20381] Updated weights for policy 1, policy_version 1692176 (0.0007) [2023-12-27 17:22:46,006][20347] Updated weights for policy 0, policy_version 1693778 (0.0009) [2023-12-27 17:22:46,057][20347] Updated weights for policy 0, policy_version 1693788 (0.0009) [2023-12-27 17:22:46,105][20347] Updated weights for policy 0, policy_version 1693798 (0.0009) [2023-12-27 17:22:46,531][20381] Updated weights for policy 1, policy_version 1692186 (0.0007) [2023-12-27 17:22:46,579][20381] Updated weights for policy 1, policy_version 1692196 (0.0008) [2023-12-27 17:22:46,625][20381] Updated weights for policy 1, policy_version 1692206 (0.0008) [2023-12-27 17:22:46,864][20347] Updated weights for policy 0, policy_version 1693808 (0.0010) [2023-12-27 17:22:46,910][20347] Updated weights for policy 0, policy_version 1693818 (0.0010) [2023-12-27 17:22:46,972][20347] Updated weights for policy 0, policy_version 1693828 (0.0010) [2023-12-27 17:22:47,312][20381] Updated weights for policy 1, policy_version 1692216 (0.0007) [2023-12-27 17:22:47,358][20381] Updated weights for policy 1, policy_version 1692226 (0.0007) [2023-12-27 17:22:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 866951168. Throughput: 0: 10252.0, 1: 10374.4. Samples: 866964608. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:22:47,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:22:47,409][20381] Updated weights for policy 1, policy_version 1692236 (0.0007) [2023-12-27 17:22:47,602][20347] Updated weights for policy 0, policy_version 1693838 (0.0010) [2023-12-27 17:22:47,668][20347] Updated weights for policy 0, policy_version 1693848 (0.0008) [2023-12-27 17:22:47,723][20347] Updated weights for policy 0, policy_version 1693858 (0.0006) [2023-12-27 17:22:48,031][20381] Updated weights for policy 1, policy_version 1692246 (0.0009) [2023-12-27 17:22:48,084][20381] Updated weights for policy 1, policy_version 1692256 (0.0010) [2023-12-27 17:22:48,128][20381] Updated weights for policy 1, policy_version 1692266 (0.0009) [2023-12-27 17:22:48,364][20347] Updated weights for policy 0, policy_version 1693868 (0.0007) [2023-12-27 17:22:48,427][20347] Updated weights for policy 0, policy_version 1693878 (0.0007) [2023-12-27 17:22:48,481][20347] Updated weights for policy 0, policy_version 1693888 (0.0007) [2023-12-27 17:22:48,820][20381] Updated weights for policy 1, policy_version 1692276 (0.0007) [2023-12-27 17:22:48,881][20381] Updated weights for policy 1, policy_version 1692286 (0.0011) [2023-12-27 17:22:48,934][20381] Updated weights for policy 1, policy_version 1692296 (0.0011) [2023-12-27 17:22:49,178][20347] Updated weights for policy 0, policy_version 1693898 (0.0007) [2023-12-27 17:22:49,250][20347] Updated weights for policy 0, policy_version 1693908 (0.0008) [2023-12-27 17:22:49,304][20347] Updated weights for policy 0, policy_version 1693918 (0.0008) [2023-12-27 17:22:49,371][20347] Updated weights for policy 0, policy_version 1693928 (0.0007) [2023-12-27 17:22:49,658][20381] Updated weights for policy 1, policy_version 1692306 (0.0010) [2023-12-27 17:22:49,714][20381] Updated weights for policy 1, policy_version 1692316 (0.0008) [2023-12-27 17:22:49,767][20381] Updated weights for policy 1, policy_version 1692326 (0.0008) [2023-12-27 17:22:49,828][20381] Updated weights for policy 1, policy_version 1692336 (0.0009) [2023-12-27 17:22:50,132][20347] Updated weights for policy 0, policy_version 1693938 (0.0007) [2023-12-27 17:22:50,186][20347] Updated weights for policy 0, policy_version 1693948 (0.0007) [2023-12-27 17:22:50,245][20347] Updated weights for policy 0, policy_version 1693958 (0.0007) [2023-12-27 17:22:50,631][20381] Updated weights for policy 1, policy_version 1692346 (0.0007) [2023-12-27 17:22:50,708][20381] Updated weights for policy 1, policy_version 1692356 (0.0007) [2023-12-27 17:22:50,760][20381] Updated weights for policy 1, policy_version 1692366 (0.0008) [2023-12-27 17:22:50,887][20347] Updated weights for policy 0, policy_version 1693968 (0.0007) [2023-12-27 17:22:50,951][20347] Updated weights for policy 0, policy_version 1693978 (0.0008) [2023-12-27 17:22:51,009][20347] Updated weights for policy 0, policy_version 1693988 (0.0008) [2023-12-27 17:22:51,476][20381] Updated weights for policy 1, policy_version 1692376 (0.0007) [2023-12-27 17:22:51,535][20381] Updated weights for policy 1, policy_version 1692386 (0.0008) [2023-12-27 17:22:51,595][20381] Updated weights for policy 1, policy_version 1692396 (0.0008) [2023-12-27 17:22:51,697][20347] Updated weights for policy 0, policy_version 1693998 (0.0008) [2023-12-27 17:22:51,767][20347] Updated weights for policy 0, policy_version 1694008 (0.0008) [2023-12-27 17:22:51,822][20347] Updated weights for policy 0, policy_version 1694018 (0.0008) [2023-12-27 17:22:52,298][20381] Updated weights for policy 1, policy_version 1692406 (0.0008) [2023-12-27 17:22:52,359][20381] Updated weights for policy 1, policy_version 1692416 (0.0008) [2023-12-27 17:22:52,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 867049472. Throughput: 0: 10205.6, 1: 10356.5. Samples: 867025004. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:22:52,363][19170] Avg episode reward: [(0, '980.490'), (1, '1000.000')] [2023-12-27 17:22:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001694024_433750016.pth... [2023-12-27 17:22:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001692840_433446912.pth [2023-12-27 17:22:52,426][20381] Updated weights for policy 1, policy_version 1692426 (0.0006) [2023-12-27 17:22:52,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001692432_433307648.pth... [2023-12-27 17:22:52,473][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001691184_432988160.pth [2023-12-27 17:22:52,524][20347] Updated weights for policy 0, policy_version 1694028 (0.0007) [2023-12-27 17:22:52,586][20347] Updated weights for policy 0, policy_version 1694038 (0.0007) [2023-12-27 17:22:52,653][20347] Updated weights for policy 0, policy_version 1694048 (0.0009) [2023-12-27 17:22:53,031][20381] Updated weights for policy 1, policy_version 1692436 (0.0007) [2023-12-27 17:22:53,086][20381] Updated weights for policy 1, policy_version 1692446 (0.0007) [2023-12-27 17:22:53,132][20381] Updated weights for policy 1, policy_version 1692456 (0.0008) [2023-12-27 17:22:53,296][20347] Updated weights for policy 0, policy_version 1694058 (0.0008) [2023-12-27 17:22:53,354][20347] Updated weights for policy 0, policy_version 1694068 (0.0007) [2023-12-27 17:22:53,402][20347] Updated weights for policy 0, policy_version 1694078 (0.0007) [2023-12-27 17:22:53,446][20347] Updated weights for policy 0, policy_version 1694088 (0.0007) [2023-12-27 17:22:53,879][20381] Updated weights for policy 1, policy_version 1692466 (0.0008) [2023-12-27 17:22:53,925][20381] Updated weights for policy 1, policy_version 1692476 (0.0009) [2023-12-27 17:22:53,973][20381] Updated weights for policy 1, policy_version 1692486 (0.0007) [2023-12-27 17:22:54,029][20381] Updated weights for policy 1, policy_version 1692496 (0.0008) [2023-12-27 17:22:54,067][20347] Updated weights for policy 0, policy_version 1694098 (0.0010) [2023-12-27 17:22:54,115][20347] Updated weights for policy 0, policy_version 1694108 (0.0010) [2023-12-27 17:22:54,177][20347] Updated weights for policy 0, policy_version 1694118 (0.0007) [2023-12-27 17:22:54,761][20381] Updated weights for policy 1, policy_version 1692506 (0.0008) [2023-12-27 17:22:54,779][20347] Updated weights for policy 0, policy_version 1694128 (0.0007) [2023-12-27 17:22:54,818][20381] Updated weights for policy 1, policy_version 1692516 (0.0007) [2023-12-27 17:22:54,828][20347] Updated weights for policy 0, policy_version 1694138 (0.0006) [2023-12-27 17:22:54,876][20347] Updated weights for policy 0, policy_version 1694148 (0.0007) [2023-12-27 17:22:54,879][20381] Updated weights for policy 1, policy_version 1692526 (0.0005) [2023-12-27 17:22:55,473][20381] Updated weights for policy 1, policy_version 1692536 (0.0005) [2023-12-27 17:22:55,538][20381] Updated weights for policy 1, policy_version 1692546 (0.0005) [2023-12-27 17:22:55,544][20347] Updated weights for policy 0, policy_version 1694158 (0.0008) [2023-12-27 17:22:55,591][20381] Updated weights for policy 1, policy_version 1692556 (0.0006) [2023-12-27 17:22:55,596][20347] Updated weights for policy 0, policy_version 1694168 (0.0006) [2023-12-27 17:22:55,647][20347] Updated weights for policy 0, policy_version 1694178 (0.0010) [2023-12-27 17:22:56,250][20381] Updated weights for policy 1, policy_version 1692566 (0.0008) [2023-12-27 17:22:56,283][20347] Updated weights for policy 0, policy_version 1694188 (0.0010) [2023-12-27 17:22:56,300][20381] Updated weights for policy 1, policy_version 1692576 (0.0006) [2023-12-27 17:22:56,335][20347] Updated weights for policy 0, policy_version 1694198 (0.0010) [2023-12-27 17:22:56,348][20381] Updated weights for policy 1, policy_version 1692586 (0.0005) [2023-12-27 17:22:56,386][20347] Updated weights for policy 0, policy_version 1694208 (0.0010) [2023-12-27 17:22:57,028][20381] Updated weights for policy 1, policy_version 1692596 (0.0006) [2023-12-27 17:22:57,072][20347] Updated weights for policy 0, policy_version 1694218 (0.0010) [2023-12-27 17:22:57,078][20381] Updated weights for policy 1, policy_version 1692606 (0.0005) [2023-12-27 17:22:57,127][20347] Updated weights for policy 0, policy_version 1694228 (0.0007) [2023-12-27 17:22:57,134][20381] Updated weights for policy 1, policy_version 1692616 (0.0007) [2023-12-27 17:22:57,181][20347] Updated weights for policy 0, policy_version 1694238 (0.0007) [2023-12-27 17:22:57,246][20347] Updated weights for policy 0, policy_version 1694248 (0.0010) [2023-12-27 17:22:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 867164160. Throughput: 0: 10244.4, 1: 10277.0. Samples: 867148824. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:22:57,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:22:57,719][20381] Updated weights for policy 1, policy_version 1692626 (0.0006) [2023-12-27 17:22:57,767][20381] Updated weights for policy 1, policy_version 1692636 (0.0008) [2023-12-27 17:22:57,814][20381] Updated weights for policy 1, policy_version 1692646 (0.0007) [2023-12-27 17:22:57,864][20381] Updated weights for policy 1, policy_version 1692656 (0.0008) [2023-12-27 17:22:57,866][20347] Updated weights for policy 0, policy_version 1694258 (0.0008) [2023-12-27 17:22:57,918][20347] Updated weights for policy 0, policy_version 1694268 (0.0006) [2023-12-27 17:22:57,971][20347] Updated weights for policy 0, policy_version 1694278 (0.0005) [2023-12-27 17:22:58,632][20347] Updated weights for policy 0, policy_version 1694288 (0.0006) [2023-12-27 17:22:58,661][20381] Updated weights for policy 1, policy_version 1692666 (0.0007) [2023-12-27 17:22:58,691][20347] Updated weights for policy 0, policy_version 1694298 (0.0007) [2023-12-27 17:22:58,723][20381] Updated weights for policy 1, policy_version 1692676 (0.0010) [2023-12-27 17:22:58,756][20347] Updated weights for policy 0, policy_version 1694308 (0.0007) [2023-12-27 17:22:58,792][20381] Updated weights for policy 1, policy_version 1692686 (0.0008) [2023-12-27 17:22:59,479][20347] Updated weights for policy 0, policy_version 1694318 (0.0007) [2023-12-27 17:22:59,520][20381] Updated weights for policy 1, policy_version 1692696 (0.0008) [2023-12-27 17:22:59,529][20347] Updated weights for policy 0, policy_version 1694328 (0.0007) [2023-12-27 17:22:59,580][20381] Updated weights for policy 1, policy_version 1692706 (0.0006) [2023-12-27 17:22:59,581][20347] Updated weights for policy 0, policy_version 1694338 (0.0005) [2023-12-27 17:22:59,636][20381] Updated weights for policy 1, policy_version 1692716 (0.0007) [2023-12-27 17:23:00,269][20347] Updated weights for policy 0, policy_version 1694348 (0.0006) [2023-12-27 17:23:00,278][20381] Updated weights for policy 1, policy_version 1692726 (0.0007) [2023-12-27 17:23:00,318][20347] Updated weights for policy 0, policy_version 1694358 (0.0006) [2023-12-27 17:23:00,331][20381] Updated weights for policy 1, policy_version 1692736 (0.0006) [2023-12-27 17:23:00,373][20347] Updated weights for policy 0, policy_version 1694368 (0.0006) [2023-12-27 17:23:00,391][20381] Updated weights for policy 1, policy_version 1692746 (0.0006) [2023-12-27 17:23:01,004][20381] Updated weights for policy 1, policy_version 1692756 (0.0005) [2023-12-27 17:23:01,057][20381] Updated weights for policy 1, policy_version 1692766 (0.0007) [2023-12-27 17:23:01,075][20347] Updated weights for policy 0, policy_version 1694378 (0.0006) [2023-12-27 17:23:01,119][20381] Updated weights for policy 1, policy_version 1692776 (0.0006) [2023-12-27 17:23:01,143][20347] Updated weights for policy 0, policy_version 1694388 (0.0006) [2023-12-27 17:23:01,218][20347] Updated weights for policy 0, policy_version 1694398 (0.0009) [2023-12-27 17:23:01,273][20347] Updated weights for policy 0, policy_version 1694408 (0.0011) [2023-12-27 17:23:01,933][20381] Updated weights for policy 1, policy_version 1692786 (0.0008) [2023-12-27 17:23:01,991][20381] Updated weights for policy 1, policy_version 1692796 (0.0008) [2023-12-27 17:23:02,046][20381] Updated weights for policy 1, policy_version 1692806 (0.0007) [2023-12-27 17:23:02,079][20347] Updated weights for policy 0, policy_version 1694418 (0.0007) [2023-12-27 17:23:02,104][20381] Updated weights for policy 1, policy_version 1692816 (0.0007) [2023-12-27 17:23:02,135][20347] Updated weights for policy 0, policy_version 1694428 (0.0010) [2023-12-27 17:23:02,184][20347] Updated weights for policy 0, policy_version 1694438 (0.0010) [2023-12-27 17:23:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 867262464. Throughput: 0: 10193.5, 1: 10282.9. Samples: 867268212. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:02,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 17:23:02,743][20381] Updated weights for policy 1, policy_version 1692826 (0.0005) [2023-12-27 17:23:02,799][20381] Updated weights for policy 1, policy_version 1692836 (0.0006) [2023-12-27 17:23:02,842][20347] Updated weights for policy 0, policy_version 1694448 (0.0006) [2023-12-27 17:23:02,849][20381] Updated weights for policy 1, policy_version 1692846 (0.0007) [2023-12-27 17:23:02,896][20347] Updated weights for policy 0, policy_version 1694458 (0.0005) [2023-12-27 17:23:02,959][20347] Updated weights for policy 0, policy_version 1694468 (0.0006) [2023-12-27 17:23:03,507][20381] Updated weights for policy 1, policy_version 1692856 (0.0009) [2023-12-27 17:23:03,565][20381] Updated weights for policy 1, policy_version 1692866 (0.0009) [2023-12-27 17:23:03,575][20347] Updated weights for policy 0, policy_version 1694478 (0.0006) [2023-12-27 17:23:03,613][20381] Updated weights for policy 1, policy_version 1692876 (0.0006) [2023-12-27 17:23:03,625][20347] Updated weights for policy 0, policy_version 1694488 (0.0007) [2023-12-27 17:23:03,671][20347] Updated weights for policy 0, policy_version 1694498 (0.0005) [2023-12-27 17:23:04,271][20381] Updated weights for policy 1, policy_version 1692886 (0.0007) [2023-12-27 17:23:04,329][20381] Updated weights for policy 1, policy_version 1692896 (0.0007) [2023-12-27 17:23:04,362][20347] Updated weights for policy 0, policy_version 1694508 (0.0008) [2023-12-27 17:23:04,383][20381] Updated weights for policy 1, policy_version 1692906 (0.0006) [2023-12-27 17:23:04,422][20347] Updated weights for policy 0, policy_version 1694518 (0.0011) [2023-12-27 17:23:04,479][20347] Updated weights for policy 0, policy_version 1694528 (0.0011) [2023-12-27 17:23:05,047][20381] Updated weights for policy 1, policy_version 1692916 (0.0005) [2023-12-27 17:23:05,102][20381] Updated weights for policy 1, policy_version 1692926 (0.0005) [2023-12-27 17:23:05,156][20381] Updated weights for policy 1, policy_version 1692936 (0.0005) [2023-12-27 17:23:05,160][20347] Updated weights for policy 0, policy_version 1694538 (0.0010) [2023-12-27 17:23:05,214][20347] Updated weights for policy 0, policy_version 1694548 (0.0008) [2023-12-27 17:23:05,263][20347] Updated weights for policy 0, policy_version 1694558 (0.0009) [2023-12-27 17:23:05,317][20347] Updated weights for policy 0, policy_version 1694568 (0.0009) [2023-12-27 17:23:05,816][20381] Updated weights for policy 1, policy_version 1692946 (0.0006) [2023-12-27 17:23:05,865][20381] Updated weights for policy 1, policy_version 1692956 (0.0008) [2023-12-27 17:23:05,909][20381] Updated weights for policy 1, policy_version 1692966 (0.0008) [2023-12-27 17:23:05,949][20347] Updated weights for policy 0, policy_version 1694578 (0.0006) [2023-12-27 17:23:05,966][20381] Updated weights for policy 1, policy_version 1692976 (0.0008) [2023-12-27 17:23:06,015][20347] Updated weights for policy 0, policy_version 1694588 (0.0007) [2023-12-27 17:23:06,079][20347] Updated weights for policy 0, policy_version 1694598 (0.0008) [2023-12-27 17:23:06,744][20381] Updated weights for policy 1, policy_version 1692986 (0.0006) [2023-12-27 17:23:06,746][20347] Updated weights for policy 0, policy_version 1694608 (0.0010) [2023-12-27 17:23:06,797][20381] Updated weights for policy 1, policy_version 1692996 (0.0006) [2023-12-27 17:23:06,798][20347] Updated weights for policy 0, policy_version 1694618 (0.0011) [2023-12-27 17:23:06,843][20381] Updated weights for policy 1, policy_version 1693006 (0.0007) [2023-12-27 17:23:06,851][20347] Updated weights for policy 0, policy_version 1694628 (0.0010) [2023-12-27 17:23:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 867360768. Throughput: 0: 10212.8, 1: 10330.4. Samples: 867331992. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:07,363][19170] Avg episode reward: [(0, '960.860'), (1, '1000.000')] [2023-12-27 17:23:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001693008_433455104.pth... [2023-12-27 17:23:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001694632_433905664.pth... [2023-12-27 17:23:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001693448_433602560.pth [2023-12-27 17:23:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001691824_433152000.pth [2023-12-27 17:23:07,550][20347] Updated weights for policy 0, policy_version 1694638 (0.0010) [2023-12-27 17:23:07,551][20381] Updated weights for policy 1, policy_version 1693016 (0.0006) [2023-12-27 17:23:07,600][20381] Updated weights for policy 1, policy_version 1693026 (0.0007) [2023-12-27 17:23:07,608][20347] Updated weights for policy 0, policy_version 1694648 (0.0008) [2023-12-27 17:23:07,659][20381] Updated weights for policy 1, policy_version 1693036 (0.0007) [2023-12-27 17:23:07,670][20347] Updated weights for policy 0, policy_version 1694658 (0.0010) [2023-12-27 17:23:08,333][20347] Updated weights for policy 0, policy_version 1694668 (0.0011) [2023-12-27 17:23:08,339][20381] Updated weights for policy 1, policy_version 1693046 (0.0008) [2023-12-27 17:23:08,397][20381] Updated weights for policy 1, policy_version 1693056 (0.0006) [2023-12-27 17:23:08,402][20347] Updated weights for policy 0, policy_version 1694678 (0.0010) [2023-12-27 17:23:08,451][20381] Updated weights for policy 1, policy_version 1693066 (0.0006) [2023-12-27 17:23:08,463][20347] Updated weights for policy 0, policy_version 1694688 (0.0010) [2023-12-27 17:23:09,065][20347] Updated weights for policy 0, policy_version 1694698 (0.0009) [2023-12-27 17:23:09,124][20347] Updated weights for policy 0, policy_version 1694708 (0.0006) [2023-12-27 17:23:09,137][20381] Updated weights for policy 1, policy_version 1693076 (0.0006) [2023-12-27 17:23:09,173][20347] Updated weights for policy 0, policy_version 1694718 (0.0006) [2023-12-27 17:23:09,186][20381] Updated weights for policy 1, policy_version 1693086 (0.0005) [2023-12-27 17:23:09,229][20347] Updated weights for policy 0, policy_version 1694728 (0.0006) [2023-12-27 17:23:09,266][20381] Updated weights for policy 1, policy_version 1693096 (0.0006) [2023-12-27 17:23:09,918][20381] Updated weights for policy 1, policy_version 1693106 (0.0008) [2023-12-27 17:23:09,981][20347] Updated weights for policy 0, policy_version 1694738 (0.0008) [2023-12-27 17:23:09,987][20381] Updated weights for policy 1, policy_version 1693116 (0.0008) [2023-12-27 17:23:10,042][20347] Updated weights for policy 0, policy_version 1694748 (0.0006) [2023-12-27 17:23:10,044][20381] Updated weights for policy 1, policy_version 1693126 (0.0008) [2023-12-27 17:23:10,104][20381] Updated weights for policy 1, policy_version 1693136 (0.0007) [2023-12-27 17:23:10,104][20347] Updated weights for policy 0, policy_version 1694758 (0.0006) [2023-12-27 17:23:10,756][20381] Updated weights for policy 1, policy_version 1693146 (0.0007) [2023-12-27 17:23:10,767][20347] Updated weights for policy 0, policy_version 1694768 (0.0007) [2023-12-27 17:23:10,810][20381] Updated weights for policy 1, policy_version 1693156 (0.0006) [2023-12-27 17:23:10,826][20347] Updated weights for policy 0, policy_version 1694778 (0.0008) [2023-12-27 17:23:10,871][20381] Updated weights for policy 1, policy_version 1693166 (0.0006) [2023-12-27 17:23:10,884][20347] Updated weights for policy 0, policy_version 1694788 (0.0007) [2023-12-27 17:23:11,575][20381] Updated weights for policy 1, policy_version 1693176 (0.0008) [2023-12-27 17:23:11,639][20381] Updated weights for policy 1, policy_version 1693186 (0.0008) [2023-12-27 17:23:11,653][20347] Updated weights for policy 0, policy_version 1694798 (0.0007) [2023-12-27 17:23:11,697][20381] Updated weights for policy 1, policy_version 1693196 (0.0007) [2023-12-27 17:23:11,710][20347] Updated weights for policy 0, policy_version 1694808 (0.0007) [2023-12-27 17:23:11,779][20347] Updated weights for policy 0, policy_version 1694818 (0.0008) [2023-12-27 17:23:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 867459072. Throughput: 0: 10223.2, 1: 10276.2. Samples: 867453412. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:12,363][19170] Avg episode reward: [(0, '951.070'), (1, '1000.000')] [2023-12-27 17:23:12,437][20381] Updated weights for policy 1, policy_version 1693206 (0.0008) [2023-12-27 17:23:12,497][20381] Updated weights for policy 1, policy_version 1693216 (0.0008) [2023-12-27 17:23:12,532][20347] Updated weights for policy 0, policy_version 1694828 (0.0008) [2023-12-27 17:23:12,553][20381] Updated weights for policy 1, policy_version 1693226 (0.0005) [2023-12-27 17:23:12,588][20347] Updated weights for policy 0, policy_version 1694838 (0.0007) [2023-12-27 17:23:12,646][20347] Updated weights for policy 0, policy_version 1694848 (0.0007) [2023-12-27 17:23:13,206][20381] Updated weights for policy 1, policy_version 1693236 (0.0006) [2023-12-27 17:23:13,260][20381] Updated weights for policy 1, policy_version 1693246 (0.0005) [2023-12-27 17:23:13,315][20381] Updated weights for policy 1, policy_version 1693256 (0.0005) [2023-12-27 17:23:13,356][20347] Updated weights for policy 0, policy_version 1694858 (0.0008) [2023-12-27 17:23:13,423][20347] Updated weights for policy 0, policy_version 1694868 (0.0010) [2023-12-27 17:23:13,489][20347] Updated weights for policy 0, policy_version 1694878 (0.0010) [2023-12-27 17:23:13,549][20347] Updated weights for policy 0, policy_version 1694888 (0.0010) [2023-12-27 17:23:13,917][20381] Updated weights for policy 1, policy_version 1693266 (0.0006) [2023-12-27 17:23:13,972][20381] Updated weights for policy 1, policy_version 1693276 (0.0007) [2023-12-27 17:23:14,021][20381] Updated weights for policy 1, policy_version 1693286 (0.0008) [2023-12-27 17:23:14,065][20381] Updated weights for policy 1, policy_version 1693296 (0.0008) [2023-12-27 17:23:14,168][20347] Updated weights for policy 0, policy_version 1694898 (0.0006) [2023-12-27 17:23:14,222][20347] Updated weights for policy 0, policy_version 1694908 (0.0007) [2023-12-27 17:23:14,279][20347] Updated weights for policy 0, policy_version 1694918 (0.0005) [2023-12-27 17:23:14,794][20381] Updated weights for policy 1, policy_version 1693306 (0.0007) [2023-12-27 17:23:14,852][20381] Updated weights for policy 1, policy_version 1693316 (0.0007) [2023-12-27 17:23:14,901][20347] Updated weights for policy 0, policy_version 1694928 (0.0006) [2023-12-27 17:23:14,912][20381] Updated weights for policy 1, policy_version 1693326 (0.0007) [2023-12-27 17:23:14,964][20347] Updated weights for policy 0, policy_version 1694938 (0.0006) [2023-12-27 17:23:15,020][20347] Updated weights for policy 0, policy_version 1694948 (0.0008) [2023-12-27 17:23:15,613][20381] Updated weights for policy 1, policy_version 1693336 (0.0007) [2023-12-27 17:23:15,663][20381] Updated weights for policy 1, policy_version 1693346 (0.0007) [2023-12-27 17:23:15,666][20347] Updated weights for policy 0, policy_version 1694958 (0.0006) [2023-12-27 17:23:15,713][20347] Updated weights for policy 0, policy_version 1694968 (0.0005) [2023-12-27 17:23:15,713][20381] Updated weights for policy 1, policy_version 1693356 (0.0007) [2023-12-27 17:23:15,769][20347] Updated weights for policy 0, policy_version 1694978 (0.0006) [2023-12-27 17:23:16,293][20381] Updated weights for policy 1, policy_version 1693366 (0.0006) [2023-12-27 17:23:16,348][20381] Updated weights for policy 1, policy_version 1693376 (0.0006) [2023-12-27 17:23:16,397][20381] Updated weights for policy 1, policy_version 1693386 (0.0006) [2023-12-27 17:23:16,473][20347] Updated weights for policy 0, policy_version 1694988 (0.0009) [2023-12-27 17:23:16,521][20347] Updated weights for policy 0, policy_version 1694998 (0.0007) [2023-12-27 17:23:16,582][20347] Updated weights for policy 0, policy_version 1695008 (0.0008) [2023-12-27 17:23:17,040][20381] Updated weights for policy 1, policy_version 1693396 (0.0006) [2023-12-27 17:23:17,090][20381] Updated weights for policy 1, policy_version 1693406 (0.0007) [2023-12-27 17:23:17,146][20381] Updated weights for policy 1, policy_version 1693416 (0.0005) [2023-12-27 17:23:17,259][20347] Updated weights for policy 0, policy_version 1695018 (0.0006) [2023-12-27 17:23:17,318][20347] Updated weights for policy 0, policy_version 1695028 (0.0007) [2023-12-27 17:23:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 867565568. Throughput: 0: 10127.1, 1: 10321.6. Samples: 867576620. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:17,363][19170] Avg episode reward: [(0, '951.200'), (1, '1000.000')] [2023-12-27 17:23:17,373][20347] Updated weights for policy 0, policy_version 1695038 (0.0008) [2023-12-27 17:23:17,429][20347] Updated weights for policy 0, policy_version 1695048 (0.0007) [2023-12-27 17:23:17,762][20381] Updated weights for policy 1, policy_version 1693426 (0.0006) [2023-12-27 17:23:17,814][20381] Updated weights for policy 1, policy_version 1693436 (0.0008) [2023-12-27 17:23:17,864][20381] Updated weights for policy 1, policy_version 1693446 (0.0007) [2023-12-27 17:23:17,921][20381] Updated weights for policy 1, policy_version 1693456 (0.0008) [2023-12-27 17:23:18,142][20347] Updated weights for policy 0, policy_version 1695058 (0.0008) [2023-12-27 17:23:18,187][20347] Updated weights for policy 0, policy_version 1695068 (0.0007) [2023-12-27 17:23:18,244][20347] Updated weights for policy 0, policy_version 1695078 (0.0006) [2023-12-27 17:23:18,570][20381] Updated weights for policy 1, policy_version 1693466 (0.0007) [2023-12-27 17:23:18,633][20381] Updated weights for policy 1, policy_version 1693476 (0.0007) [2023-12-27 17:23:18,693][20381] Updated weights for policy 1, policy_version 1693486 (0.0008) [2023-12-27 17:23:18,978][20347] Updated weights for policy 0, policy_version 1695088 (0.0006) [2023-12-27 17:23:19,029][20347] Updated weights for policy 0, policy_version 1695098 (0.0006) [2023-12-27 17:23:19,077][20347] Updated weights for policy 0, policy_version 1695108 (0.0010) [2023-12-27 17:23:19,391][20381] Updated weights for policy 1, policy_version 1693496 (0.0007) [2023-12-27 17:23:19,443][20381] Updated weights for policy 1, policy_version 1693506 (0.0008) [2023-12-27 17:23:19,502][20381] Updated weights for policy 1, policy_version 1693516 (0.0008) [2023-12-27 17:23:19,770][20347] Updated weights for policy 0, policy_version 1695118 (0.0007) [2023-12-27 17:23:19,831][20347] Updated weights for policy 0, policy_version 1695128 (0.0006) [2023-12-27 17:23:19,893][20347] Updated weights for policy 0, policy_version 1695138 (0.0007) [2023-12-27 17:23:20,250][20381] Updated weights for policy 1, policy_version 1693526 (0.0007) [2023-12-27 17:23:20,300][20381] Updated weights for policy 1, policy_version 1693536 (0.0008) [2023-12-27 17:23:20,351][20381] Updated weights for policy 1, policy_version 1693546 (0.0008) [2023-12-27 17:23:20,598][20347] Updated weights for policy 0, policy_version 1695148 (0.0007) [2023-12-27 17:23:20,655][20347] Updated weights for policy 0, policy_version 1695158 (0.0008) [2023-12-27 17:23:20,709][20347] Updated weights for policy 0, policy_version 1695168 (0.0007) [2023-12-27 17:23:21,167][20381] Updated weights for policy 1, policy_version 1693556 (0.0006) [2023-12-27 17:23:21,227][20381] Updated weights for policy 1, policy_version 1693566 (0.0008) [2023-12-27 17:23:21,287][20381] Updated weights for policy 1, policy_version 1693576 (0.0008) [2023-12-27 17:23:21,472][20347] Updated weights for policy 0, policy_version 1695178 (0.0008) [2023-12-27 17:23:21,525][20347] Updated weights for policy 0, policy_version 1695188 (0.0008) [2023-12-27 17:23:21,581][20347] Updated weights for policy 0, policy_version 1695198 (0.0007) [2023-12-27 17:23:21,643][20347] Updated weights for policy 0, policy_version 1695208 (0.0008) [2023-12-27 17:23:22,006][20381] Updated weights for policy 1, policy_version 1693586 (0.0008) [2023-12-27 17:23:22,057][20381] Updated weights for policy 1, policy_version 1693596 (0.0008) [2023-12-27 17:23:22,111][20381] Updated weights for policy 1, policy_version 1693606 (0.0007) [2023-12-27 17:23:22,170][20381] Updated weights for policy 1, policy_version 1693616 (0.0007) [2023-12-27 17:23:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 867663872. Throughput: 0: 10095.5, 1: 10318.5. Samples: 867636992. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:22,363][19170] Avg episode reward: [(0, '980.510'), (1, '1000.000')] [2023-12-27 17:23:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001693616_433610752.pth... [2023-12-27 17:23:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001692432_433307648.pth [2023-12-27 17:23:22,411][20347] Updated weights for policy 0, policy_version 1695218 (0.0008) [2023-12-27 17:23:22,468][20347] Updated weights for policy 0, policy_version 1695228 (0.0008) [2023-12-27 17:23:22,519][20347] Updated weights for policy 0, policy_version 1695238 (0.0007) [2023-12-27 17:23:22,529][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001695240_434061312.pth... [2023-12-27 17:23:22,533][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001694024_433750016.pth [2023-12-27 17:23:22,896][20381] Updated weights for policy 1, policy_version 1693626 (0.0005) [2023-12-27 17:23:22,951][20381] Updated weights for policy 1, policy_version 1693636 (0.0008) [2023-12-27 17:23:23,001][20381] Updated weights for policy 1, policy_version 1693646 (0.0010) [2023-12-27 17:23:23,163][20347] Updated weights for policy 0, policy_version 1695248 (0.0008) [2023-12-27 17:23:23,217][20347] Updated weights for policy 0, policy_version 1695258 (0.0007) [2023-12-27 17:23:23,263][20347] Updated weights for policy 0, policy_version 1695268 (0.0005) [2023-12-27 17:23:23,650][20381] Updated weights for policy 1, policy_version 1693656 (0.0008) [2023-12-27 17:23:23,695][20381] Updated weights for policy 1, policy_version 1693666 (0.0005) [2023-12-27 17:23:23,743][20381] Updated weights for policy 1, policy_version 1693676 (0.0005) [2023-12-27 17:23:23,879][20347] Updated weights for policy 0, policy_version 1695278 (0.0007) [2023-12-27 17:23:23,933][20347] Updated weights for policy 0, policy_version 1695288 (0.0005) [2023-12-27 17:23:23,984][20347] Updated weights for policy 0, policy_version 1695298 (0.0009) [2023-12-27 17:23:24,331][20381] Updated weights for policy 1, policy_version 1693686 (0.0006) [2023-12-27 17:23:24,392][20381] Updated weights for policy 1, policy_version 1693696 (0.0008) [2023-12-27 17:23:24,450][20381] Updated weights for policy 1, policy_version 1693706 (0.0010) [2023-12-27 17:23:24,681][20347] Updated weights for policy 0, policy_version 1695308 (0.0009) [2023-12-27 17:23:24,741][20347] Updated weights for policy 0, policy_version 1695318 (0.0006) [2023-12-27 17:23:24,792][20347] Updated weights for policy 0, policy_version 1695328 (0.0005) [2023-12-27 17:23:25,066][20381] Updated weights for policy 1, policy_version 1693716 (0.0008) [2023-12-27 17:23:25,124][20381] Updated weights for policy 1, policy_version 1693726 (0.0007) [2023-12-27 17:23:25,183][20381] Updated weights for policy 1, policy_version 1693736 (0.0010) [2023-12-27 17:23:25,467][20347] Updated weights for policy 0, policy_version 1695338 (0.0006) [2023-12-27 17:23:25,513][20347] Updated weights for policy 0, policy_version 1695348 (0.0007) [2023-12-27 17:23:25,564][20347] Updated weights for policy 0, policy_version 1695358 (0.0008) [2023-12-27 17:23:25,623][20347] Updated weights for policy 0, policy_version 1695368 (0.0008) [2023-12-27 17:23:25,765][20381] Updated weights for policy 1, policy_version 1693746 (0.0009) [2023-12-27 17:23:25,828][20381] Updated weights for policy 1, policy_version 1693756 (0.0008) [2023-12-27 17:23:25,883][20381] Updated weights for policy 1, policy_version 1693766 (0.0009) [2023-12-27 17:23:25,931][20381] Updated weights for policy 1, policy_version 1693776 (0.0010) [2023-12-27 17:23:26,351][20347] Updated weights for policy 0, policy_version 1695378 (0.0008) [2023-12-27 17:23:26,410][20347] Updated weights for policy 0, policy_version 1695388 (0.0009) [2023-12-27 17:23:26,463][20347] Updated weights for policy 0, policy_version 1695398 (0.0007) [2023-12-27 17:23:26,526][20381] Updated weights for policy 1, policy_version 1693786 (0.0010) [2023-12-27 17:23:26,583][20381] Updated weights for policy 1, policy_version 1693796 (0.0006) [2023-12-27 17:23:26,631][20381] Updated weights for policy 1, policy_version 1693806 (0.0008) [2023-12-27 17:23:27,172][20347] Updated weights for policy 0, policy_version 1695408 (0.0007) [2023-12-27 17:23:27,221][20347] Updated weights for policy 0, policy_version 1695418 (0.0007) [2023-12-27 17:23:27,267][20347] Updated weights for policy 0, policy_version 1695428 (0.0007) [2023-12-27 17:23:27,310][20381] Updated weights for policy 1, policy_version 1693816 (0.0009) [2023-12-27 17:23:27,359][20381] Updated weights for policy 1, policy_version 1693826 (0.0010) [2023-12-27 17:23:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 867770368. Throughput: 0: 10179.6, 1: 10330.5. Samples: 867760788. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:23:27,410][20381] Updated weights for policy 1, policy_version 1693836 (0.0010) [2023-12-27 17:23:27,967][20347] Updated weights for policy 0, policy_version 1695438 (0.0007) [2023-12-27 17:23:28,026][20347] Updated weights for policy 0, policy_version 1695448 (0.0007) [2023-12-27 17:23:28,072][20347] Updated weights for policy 0, policy_version 1695458 (0.0007) [2023-12-27 17:23:28,113][20381] Updated weights for policy 1, policy_version 1693846 (0.0010) [2023-12-27 17:23:28,158][20381] Updated weights for policy 1, policy_version 1693856 (0.0010) [2023-12-27 17:23:28,212][20381] Updated weights for policy 1, policy_version 1693866 (0.0010) [2023-12-27 17:23:28,746][20347] Updated weights for policy 0, policy_version 1695468 (0.0007) [2023-12-27 17:23:28,802][20347] Updated weights for policy 0, policy_version 1695478 (0.0005) [2023-12-27 17:23:28,806][20381] Updated weights for policy 1, policy_version 1693876 (0.0007) [2023-12-27 17:23:28,862][20381] Updated weights for policy 1, policy_version 1693886 (0.0008) [2023-12-27 17:23:28,865][20347] Updated weights for policy 0, policy_version 1695488 (0.0006) [2023-12-27 17:23:28,923][20381] Updated weights for policy 1, policy_version 1693896 (0.0006) [2023-12-27 17:23:29,444][20347] Updated weights for policy 0, policy_version 1695498 (0.0006) [2023-12-27 17:23:29,491][20347] Updated weights for policy 0, policy_version 1695508 (0.0008) [2023-12-27 17:23:29,548][20347] Updated weights for policy 0, policy_version 1695518 (0.0008) [2023-12-27 17:23:29,610][20347] Updated weights for policy 0, policy_version 1695528 (0.0008) [2023-12-27 17:23:29,617][20381] Updated weights for policy 1, policy_version 1693906 (0.0007) [2023-12-27 17:23:29,674][20381] Updated weights for policy 1, policy_version 1693916 (0.0006) [2023-12-27 17:23:29,728][20381] Updated weights for policy 1, policy_version 1693926 (0.0007) [2023-12-27 17:23:29,778][20381] Updated weights for policy 1, policy_version 1693936 (0.0007) [2023-12-27 17:23:30,304][20347] Updated weights for policy 0, policy_version 1695538 (0.0005) [2023-12-27 17:23:30,361][20347] Updated weights for policy 0, policy_version 1695548 (0.0007) [2023-12-27 17:23:30,392][20381] Updated weights for policy 1, policy_version 1693946 (0.0006) [2023-12-27 17:23:30,421][20347] Updated weights for policy 0, policy_version 1695558 (0.0007) [2023-12-27 17:23:30,455][20381] Updated weights for policy 1, policy_version 1693956 (0.0005) [2023-12-27 17:23:30,516][20381] Updated weights for policy 1, policy_version 1693966 (0.0006) [2023-12-27 17:23:31,065][20347] Updated weights for policy 0, policy_version 1695568 (0.0008) [2023-12-27 17:23:31,123][20347] Updated weights for policy 0, policy_version 1695578 (0.0006) [2023-12-27 17:23:31,167][20381] Updated weights for policy 1, policy_version 1693976 (0.0008) [2023-12-27 17:23:31,186][20347] Updated weights for policy 0, policy_version 1695588 (0.0009) [2023-12-27 17:23:31,220][20381] Updated weights for policy 1, policy_version 1693986 (0.0006) [2023-12-27 17:23:31,278][20381] Updated weights for policy 1, policy_version 1693996 (0.0008) [2023-12-27 17:23:31,898][20347] Updated weights for policy 0, policy_version 1695598 (0.0007) [2023-12-27 17:23:31,962][20347] Updated weights for policy 0, policy_version 1695608 (0.0007) [2023-12-27 17:23:31,978][20381] Updated weights for policy 1, policy_version 1694006 (0.0007) [2023-12-27 17:23:32,018][20347] Updated weights for policy 0, policy_version 1695618 (0.0005) [2023-12-27 17:23:32,031][20381] Updated weights for policy 1, policy_version 1694016 (0.0006) [2023-12-27 17:23:32,085][20381] Updated weights for policy 1, policy_version 1694026 (0.0008) [2023-12-27 17:23:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 867876864. Throughput: 0: 10187.5, 1: 10275.7. Samples: 867885452. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:32,363][19170] Avg episode reward: [(0, '990.320'), (1, '1000.000')] [2023-12-27 17:23:32,665][20347] Updated weights for policy 0, policy_version 1695628 (0.0006) [2023-12-27 17:23:32,726][20347] Updated weights for policy 0, policy_version 1695638 (0.0007) [2023-12-27 17:23:32,777][20381] Updated weights for policy 1, policy_version 1694036 (0.0009) [2023-12-27 17:23:32,783][20347] Updated weights for policy 0, policy_version 1695648 (0.0007) [2023-12-27 17:23:32,830][20381] Updated weights for policy 1, policy_version 1694046 (0.0010) [2023-12-27 17:23:32,881][20381] Updated weights for policy 1, policy_version 1694056 (0.0010) [2023-12-27 17:23:33,403][20347] Updated weights for policy 0, policy_version 1695658 (0.0005) [2023-12-27 17:23:33,447][20347] Updated weights for policy 0, policy_version 1695668 (0.0007) [2023-12-27 17:23:33,494][20347] Updated weights for policy 0, policy_version 1695678 (0.0007) [2023-12-27 17:23:33,534][20381] Updated weights for policy 1, policy_version 1694066 (0.0010) [2023-12-27 17:23:33,549][20347] Updated weights for policy 0, policy_version 1695688 (0.0008) [2023-12-27 17:23:33,578][20381] Updated weights for policy 1, policy_version 1694076 (0.0010) [2023-12-27 17:23:33,638][20381] Updated weights for policy 1, policy_version 1694086 (0.0010) [2023-12-27 17:23:33,693][20381] Updated weights for policy 1, policy_version 1694096 (0.0005) [2023-12-27 17:23:34,178][20347] Updated weights for policy 0, policy_version 1695698 (0.0008) [2023-12-27 17:23:34,241][20347] Updated weights for policy 0, policy_version 1695708 (0.0007) [2023-12-27 17:23:34,302][20381] Updated weights for policy 1, policy_version 1694106 (0.0011) [2023-12-27 17:23:34,303][20347] Updated weights for policy 0, policy_version 1695718 (0.0006) [2023-12-27 17:23:34,371][20381] Updated weights for policy 1, policy_version 1694116 (0.0011) [2023-12-27 17:23:34,439][20381] Updated weights for policy 1, policy_version 1694126 (0.0010) [2023-12-27 17:23:35,010][20347] Updated weights for policy 0, policy_version 1695728 (0.0008) [2023-12-27 17:23:35,063][20347] Updated weights for policy 0, policy_version 1695738 (0.0008) [2023-12-27 17:23:35,092][20381] Updated weights for policy 1, policy_version 1694136 (0.0005) [2023-12-27 17:23:35,114][20347] Updated weights for policy 0, policy_version 1695748 (0.0007) [2023-12-27 17:23:35,146][20381] Updated weights for policy 1, policy_version 1694146 (0.0007) [2023-12-27 17:23:35,211][20381] Updated weights for policy 1, policy_version 1694156 (0.0010) [2023-12-27 17:23:35,743][20347] Updated weights for policy 0, policy_version 1695758 (0.0007) [2023-12-27 17:23:35,791][20347] Updated weights for policy 0, policy_version 1695768 (0.0005) [2023-12-27 17:23:35,839][20347] Updated weights for policy 0, policy_version 1695778 (0.0005) [2023-12-27 17:23:35,866][20381] Updated weights for policy 1, policy_version 1694166 (0.0008) [2023-12-27 17:23:35,915][20381] Updated weights for policy 1, policy_version 1694176 (0.0010) [2023-12-27 17:23:35,974][20381] Updated weights for policy 1, policy_version 1694186 (0.0010) [2023-12-27 17:23:36,517][20347] Updated weights for policy 0, policy_version 1695788 (0.0007) [2023-12-27 17:23:36,570][20347] Updated weights for policy 0, policy_version 1695798 (0.0008) [2023-12-27 17:23:36,626][20347] Updated weights for policy 0, policy_version 1695808 (0.0008) [2023-12-27 17:23:36,672][20381] Updated weights for policy 1, policy_version 1694196 (0.0011) [2023-12-27 17:23:36,730][20381] Updated weights for policy 1, policy_version 1694206 (0.0011) [2023-12-27 17:23:36,789][20381] Updated weights for policy 1, policy_version 1694216 (0.0011) [2023-12-27 17:23:37,301][20347] Updated weights for policy 0, policy_version 1695818 (0.0007) [2023-12-27 17:23:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 867975168. Throughput: 0: 10213.4, 1: 10321.1. Samples: 867949056. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:37,363][19170] Avg episode reward: [(0, '990.320'), (1, '1000.000')] [2023-12-27 17:23:37,363][20347] Updated weights for policy 0, policy_version 1695828 (0.0007) [2023-12-27 17:23:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001694224_433766400.pth... [2023-12-27 17:23:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001693008_433455104.pth [2023-12-27 17:23:37,425][20347] Updated weights for policy 0, policy_version 1695838 (0.0007) [2023-12-27 17:23:37,481][20381] Updated weights for policy 1, policy_version 1694226 (0.0010) [2023-12-27 17:23:37,496][20347] Updated weights for policy 0, policy_version 1695848 (0.0006) [2023-12-27 17:23:37,496][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001695848_434216960.pth... [2023-12-27 17:23:37,501][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001694632_433905664.pth [2023-12-27 17:23:37,545][20381] Updated weights for policy 1, policy_version 1694236 (0.0009) [2023-12-27 17:23:37,604][20381] Updated weights for policy 1, policy_version 1694246 (0.0006) [2023-12-27 17:23:37,666][20381] Updated weights for policy 1, policy_version 1694256 (0.0008) [2023-12-27 17:23:38,211][20347] Updated weights for policy 0, policy_version 1695858 (0.0006) [2023-12-27 17:23:38,266][20381] Updated weights for policy 1, policy_version 1694266 (0.0007) [2023-12-27 17:23:38,267][20347] Updated weights for policy 0, policy_version 1695868 (0.0008) [2023-12-27 17:23:38,327][20381] Updated weights for policy 1, policy_version 1694276 (0.0007) [2023-12-27 17:23:38,341][20347] Updated weights for policy 0, policy_version 1695878 (0.0007) [2023-12-27 17:23:38,388][20381] Updated weights for policy 1, policy_version 1694286 (0.0008) [2023-12-27 17:23:39,028][20347] Updated weights for policy 0, policy_version 1695888 (0.0007) [2023-12-27 17:23:39,058][20381] Updated weights for policy 1, policy_version 1694296 (0.0006) [2023-12-27 17:23:39,074][20347] Updated weights for policy 0, policy_version 1695898 (0.0006) [2023-12-27 17:23:39,120][20381] Updated weights for policy 1, policy_version 1694306 (0.0006) [2023-12-27 17:23:39,120][20347] Updated weights for policy 0, policy_version 1695908 (0.0007) [2023-12-27 17:23:39,183][20381] Updated weights for policy 1, policy_version 1694316 (0.0006) [2023-12-27 17:23:39,890][20381] Updated weights for policy 1, policy_version 1694326 (0.0007) [2023-12-27 17:23:39,925][20347] Updated weights for policy 0, policy_version 1695918 (0.0006) [2023-12-27 17:23:39,959][20381] Updated weights for policy 1, policy_version 1694336 (0.0010) [2023-12-27 17:23:39,991][20347] Updated weights for policy 0, policy_version 1695928 (0.0008) [2023-12-27 17:23:40,025][20381] Updated weights for policy 1, policy_version 1694346 (0.0007) [2023-12-27 17:23:40,056][20347] Updated weights for policy 0, policy_version 1695938 (0.0007) [2023-12-27 17:23:40,673][20381] Updated weights for policy 1, policy_version 1694356 (0.0008) [2023-12-27 17:23:40,707][20347] Updated weights for policy 0, policy_version 1695948 (0.0007) [2023-12-27 17:23:40,740][20381] Updated weights for policy 1, policy_version 1694366 (0.0007) [2023-12-27 17:23:40,762][20347] Updated weights for policy 0, policy_version 1695958 (0.0006) [2023-12-27 17:23:40,802][20381] Updated weights for policy 1, policy_version 1694376 (0.0006) [2023-12-27 17:23:40,823][20347] Updated weights for policy 0, policy_version 1695968 (0.0006) [2023-12-27 17:23:41,501][20381] Updated weights for policy 1, policy_version 1694386 (0.0007) [2023-12-27 17:23:41,556][20381] Updated weights for policy 1, policy_version 1694396 (0.0008) [2023-12-27 17:23:41,610][20347] Updated weights for policy 0, policy_version 1695978 (0.0007) [2023-12-27 17:23:41,616][20381] Updated weights for policy 1, policy_version 1694406 (0.0007) [2023-12-27 17:23:41,686][20381] Updated weights for policy 1, policy_version 1694416 (0.0008) [2023-12-27 17:23:41,686][20347] Updated weights for policy 0, policy_version 1695988 (0.0007) [2023-12-27 17:23:41,754][20347] Updated weights for policy 0, policy_version 1695998 (0.0007) [2023-12-27 17:23:41,822][20347] Updated weights for policy 0, policy_version 1696008 (0.0008) [2023-12-27 17:23:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 868073472. Throughput: 0: 10136.9, 1: 10329.5. Samples: 868069812. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:42,363][19170] Avg episode reward: [(0, '990.200'), (1, '1000.000')] [2023-12-27 17:23:42,429][20381] Updated weights for policy 1, policy_version 1694426 (0.0008) [2023-12-27 17:23:42,485][20381] Updated weights for policy 1, policy_version 1694436 (0.0006) [2023-12-27 17:23:42,511][20347] Updated weights for policy 0, policy_version 1696018 (0.0008) [2023-12-27 17:23:42,550][20381] Updated weights for policy 1, policy_version 1694446 (0.0006) [2023-12-27 17:23:42,563][20347] Updated weights for policy 0, policy_version 1696028 (0.0009) [2023-12-27 17:23:42,615][20347] Updated weights for policy 0, policy_version 1696038 (0.0010) [2023-12-27 17:23:43,240][20347] Updated weights for policy 0, policy_version 1696048 (0.0007) [2023-12-27 17:23:43,265][20381] Updated weights for policy 1, policy_version 1694456 (0.0007) [2023-12-27 17:23:43,293][20347] Updated weights for policy 0, policy_version 1696058 (0.0010) [2023-12-27 17:23:43,310][20381] Updated weights for policy 1, policy_version 1694466 (0.0006) [2023-12-27 17:23:43,351][20347] Updated weights for policy 0, policy_version 1696068 (0.0010) [2023-12-27 17:23:43,360][20381] Updated weights for policy 1, policy_version 1694476 (0.0005) [2023-12-27 17:23:44,027][20381] Updated weights for policy 1, policy_version 1694486 (0.0009) [2023-12-27 17:23:44,034][20347] Updated weights for policy 0, policy_version 1696078 (0.0008) [2023-12-27 17:23:44,080][20381] Updated weights for policy 1, policy_version 1694496 (0.0010) [2023-12-27 17:23:44,082][20347] Updated weights for policy 0, policy_version 1696088 (0.0005) [2023-12-27 17:23:44,135][20381] Updated weights for policy 1, policy_version 1694506 (0.0008) [2023-12-27 17:23:44,136][20347] Updated weights for policy 0, policy_version 1696098 (0.0009) [2023-12-27 17:23:44,680][20381] Updated weights for policy 1, policy_version 1694516 (0.0008) [2023-12-27 17:23:44,713][20347] Updated weights for policy 0, policy_version 1696108 (0.0010) [2023-12-27 17:23:44,732][20381] Updated weights for policy 1, policy_version 1694526 (0.0010) [2023-12-27 17:23:44,765][20347] Updated weights for policy 0, policy_version 1696118 (0.0010) [2023-12-27 17:23:44,783][20381] Updated weights for policy 1, policy_version 1694536 (0.0011) [2023-12-27 17:23:44,828][20347] Updated weights for policy 0, policy_version 1696128 (0.0010) [2023-12-27 17:23:45,484][20381] Updated weights for policy 1, policy_version 1694546 (0.0010) [2023-12-27 17:23:45,505][20347] Updated weights for policy 0, policy_version 1696138 (0.0006) [2023-12-27 17:23:45,539][20381] Updated weights for policy 1, policy_version 1694556 (0.0005) [2023-12-27 17:23:45,565][20347] Updated weights for policy 0, policy_version 1696148 (0.0010) [2023-12-27 17:23:45,594][20381] Updated weights for policy 1, policy_version 1694566 (0.0005) [2023-12-27 17:23:45,627][20347] Updated weights for policy 0, policy_version 1696158 (0.0010) [2023-12-27 17:23:45,654][20381] Updated weights for policy 1, policy_version 1694576 (0.0005) [2023-12-27 17:23:45,682][20347] Updated weights for policy 0, policy_version 1696168 (0.0010) [2023-12-27 17:23:46,163][20381] Updated weights for policy 1, policy_version 1694586 (0.0009) [2023-12-27 17:23:46,207][20381] Updated weights for policy 1, policy_version 1694596 (0.0005) [2023-12-27 17:23:46,268][20381] Updated weights for policy 1, policy_version 1694606 (0.0010) [2023-12-27 17:23:46,361][20347] Updated weights for policy 0, policy_version 1696178 (0.0005) [2023-12-27 17:23:46,423][20347] Updated weights for policy 0, policy_version 1696188 (0.0006) [2023-12-27 17:23:46,478][20347] Updated weights for policy 0, policy_version 1696198 (0.0005) [2023-12-27 17:23:46,946][20381] Updated weights for policy 1, policy_version 1694616 (0.0011) [2023-12-27 17:23:47,005][20381] Updated weights for policy 1, policy_version 1694626 (0.0011) [2023-12-27 17:23:47,032][20347] Updated weights for policy 0, policy_version 1696208 (0.0007) [2023-12-27 17:23:47,054][20381] Updated weights for policy 1, policy_version 1694636 (0.0011) [2023-12-27 17:23:47,083][20347] Updated weights for policy 0, policy_version 1696218 (0.0008) [2023-12-27 17:23:47,140][20347] Updated weights for policy 0, policy_version 1696228 (0.0007) [2023-12-27 17:23:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 868188160. Throughput: 0: 10203.3, 1: 10404.4. Samples: 868195556. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:47,363][19170] Avg episode reward: [(0, '951.150'), (1, '1000.000')] [2023-12-27 17:23:47,757][20381] Updated weights for policy 1, policy_version 1694646 (0.0011) [2023-12-27 17:23:47,794][20347] Updated weights for policy 0, policy_version 1696238 (0.0007) [2023-12-27 17:23:47,807][20381] Updated weights for policy 1, policy_version 1694656 (0.0010) [2023-12-27 17:23:47,852][20381] Updated weights for policy 1, policy_version 1694666 (0.0011) [2023-12-27 17:23:47,853][20347] Updated weights for policy 0, policy_version 1696248 (0.0006) [2023-12-27 17:23:47,910][20347] Updated weights for policy 0, policy_version 1696258 (0.0006) [2023-12-27 17:23:48,501][20381] Updated weights for policy 1, policy_version 1694676 (0.0010) [2023-12-27 17:23:48,560][20381] Updated weights for policy 1, policy_version 1694686 (0.0011) [2023-12-27 17:23:48,582][20347] Updated weights for policy 0, policy_version 1696268 (0.0007) [2023-12-27 17:23:48,623][20381] Updated weights for policy 1, policy_version 1694696 (0.0011) [2023-12-27 17:23:48,637][20347] Updated weights for policy 0, policy_version 1696278 (0.0008) [2023-12-27 17:23:48,693][20347] Updated weights for policy 0, policy_version 1696288 (0.0007) [2023-12-27 17:23:49,305][20381] Updated weights for policy 1, policy_version 1694706 (0.0010) [2023-12-27 17:23:49,333][20347] Updated weights for policy 0, policy_version 1696298 (0.0009) [2023-12-27 17:23:49,362][20381] Updated weights for policy 1, policy_version 1694716 (0.0010) [2023-12-27 17:23:49,390][20347] Updated weights for policy 0, policy_version 1696308 (0.0011) [2023-12-27 17:23:49,421][20381] Updated weights for policy 1, policy_version 1694727 (0.0008) [2023-12-27 17:23:49,451][20347] Updated weights for policy 0, policy_version 1696318 (0.0008) [2023-12-27 17:23:49,510][20347] Updated weights for policy 0, policy_version 1696328 (0.0010) [2023-12-27 17:23:50,114][20381] Updated weights for policy 1, policy_version 1694737 (0.0009) [2023-12-27 17:23:50,170][20381] Updated weights for policy 1, policy_version 1694747 (0.0011) [2023-12-27 17:23:50,170][20347] Updated weights for policy 0, policy_version 1696338 (0.0011) [2023-12-27 17:23:50,223][20347] Updated weights for policy 0, policy_version 1696348 (0.0010) [2023-12-27 17:23:50,230][20381] Updated weights for policy 1, policy_version 1694757 (0.0010) [2023-12-27 17:23:50,275][20347] Updated weights for policy 0, policy_version 1696358 (0.0010) [2023-12-27 17:23:50,292][20381] Updated weights for policy 1, policy_version 1694767 (0.0010) [2023-12-27 17:23:50,951][20381] Updated weights for policy 1, policy_version 1694777 (0.0010) [2023-12-27 17:23:50,960][20347] Updated weights for policy 0, policy_version 1696368 (0.0006) [2023-12-27 17:23:51,006][20381] Updated weights for policy 1, policy_version 1694787 (0.0011) [2023-12-27 17:23:51,024][20347] Updated weights for policy 0, policy_version 1696378 (0.0006) [2023-12-27 17:23:51,068][20381] Updated weights for policy 1, policy_version 1694797 (0.0009) [2023-12-27 17:23:51,084][20347] Updated weights for policy 0, policy_version 1696388 (0.0007) [2023-12-27 17:23:51,793][20381] Updated weights for policy 1, policy_version 1694807 (0.0007) [2023-12-27 17:23:51,799][20347] Updated weights for policy 0, policy_version 1696398 (0.0007) [2023-12-27 17:23:51,854][20347] Updated weights for policy 0, policy_version 1696408 (0.0007) [2023-12-27 17:23:51,857][20381] Updated weights for policy 1, policy_version 1694817 (0.0008) [2023-12-27 17:23:51,907][20347] Updated weights for policy 0, policy_version 1696418 (0.0007) [2023-12-27 17:23:51,912][20381] Updated weights for policy 1, policy_version 1694827 (0.0008) [2023-12-27 17:23:52,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 868286464. Throughput: 0: 10204.4, 1: 10365.8. Samples: 868257656. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:52,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 17:23:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001694832_433922048.pth... [2023-12-27 17:23:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001696424_434364416.pth... [2023-12-27 17:23:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001693616_433610752.pth [2023-12-27 17:23:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001695240_434061312.pth [2023-12-27 17:23:52,619][20381] Updated weights for policy 1, policy_version 1694837 (0.0009) [2023-12-27 17:23:52,649][20347] Updated weights for policy 0, policy_version 1696428 (0.0008) [2023-12-27 17:23:52,672][20381] Updated weights for policy 1, policy_version 1694847 (0.0011) [2023-12-27 17:23:52,712][20347] Updated weights for policy 0, policy_version 1696438 (0.0009) [2023-12-27 17:23:52,734][20381] Updated weights for policy 1, policy_version 1694857 (0.0011) [2023-12-27 17:23:52,774][20347] Updated weights for policy 0, policy_version 1696448 (0.0010) [2023-12-27 17:23:53,342][20347] Updated weights for policy 0, policy_version 1696458 (0.0007) [2023-12-27 17:23:53,355][20381] Updated weights for policy 1, policy_version 1694867 (0.0009) [2023-12-27 17:23:53,403][20347] Updated weights for policy 0, policy_version 1696468 (0.0005) [2023-12-27 17:23:53,407][20381] Updated weights for policy 1, policy_version 1694877 (0.0009) [2023-12-27 17:23:53,461][20347] Updated weights for policy 0, policy_version 1696478 (0.0005) [2023-12-27 17:23:53,466][20381] Updated weights for policy 1, policy_version 1694887 (0.0011) [2023-12-27 17:23:53,516][20347] Updated weights for policy 0, policy_version 1696488 (0.0005) [2023-12-27 17:23:54,017][20381] Updated weights for policy 1, policy_version 1694897 (0.0010) [2023-12-27 17:23:54,031][20347] Updated weights for policy 0, policy_version 1696498 (0.0007) [2023-12-27 17:23:54,079][20381] Updated weights for policy 1, policy_version 1694907 (0.0009) [2023-12-27 17:23:54,084][20347] Updated weights for policy 0, policy_version 1696508 (0.0007) [2023-12-27 17:23:54,140][20347] Updated weights for policy 0, policy_version 1696518 (0.0007) [2023-12-27 17:23:54,141][20381] Updated weights for policy 1, policy_version 1694917 (0.0010) [2023-12-27 17:23:54,207][20381] Updated weights for policy 1, policy_version 1694927 (0.0008) [2023-12-27 17:23:54,796][20347] Updated weights for policy 0, policy_version 1696528 (0.0006) [2023-12-27 17:23:54,798][20381] Updated weights for policy 1, policy_version 1694937 (0.0008) [2023-12-27 17:23:54,849][20381] Updated weights for policy 1, policy_version 1694947 (0.0007) [2023-12-27 17:23:54,852][20347] Updated weights for policy 0, policy_version 1696538 (0.0005) [2023-12-27 17:23:54,894][20381] Updated weights for policy 1, policy_version 1694957 (0.0005) [2023-12-27 17:23:54,906][20347] Updated weights for policy 0, policy_version 1696548 (0.0006) [2023-12-27 17:23:55,591][20381] Updated weights for policy 1, policy_version 1694967 (0.0007) [2023-12-27 17:23:55,600][20347] Updated weights for policy 0, policy_version 1696558 (0.0007) [2023-12-27 17:23:55,636][20381] Updated weights for policy 1, policy_version 1694977 (0.0005) [2023-12-27 17:23:55,645][20347] Updated weights for policy 0, policy_version 1696568 (0.0005) [2023-12-27 17:23:55,681][20381] Updated weights for policy 1, policy_version 1694987 (0.0005) [2023-12-27 17:23:55,694][20347] Updated weights for policy 0, policy_version 1696578 (0.0005) [2023-12-27 17:23:56,269][20381] Updated weights for policy 1, policy_version 1694997 (0.0005) [2023-12-27 17:23:56,314][20347] Updated weights for policy 0, policy_version 1696588 (0.0007) [2023-12-27 17:23:56,327][20381] Updated weights for policy 1, policy_version 1695007 (0.0005) [2023-12-27 17:23:56,355][20347] Updated weights for policy 0, policy_version 1696598 (0.0007) [2023-12-27 17:23:56,381][20381] Updated weights for policy 1, policy_version 1695017 (0.0005) [2023-12-27 17:23:56,402][20347] Updated weights for policy 0, policy_version 1696608 (0.0006) [2023-12-27 17:23:56,955][20381] Updated weights for policy 1, policy_version 1695027 (0.0006) [2023-12-27 17:23:57,003][20381] Updated weights for policy 1, policy_version 1695037 (0.0008) [2023-12-27 17:23:57,066][20381] Updated weights for policy 1, policy_version 1695047 (0.0008) [2023-12-27 17:23:57,079][20347] Updated weights for policy 0, policy_version 1696618 (0.0006) [2023-12-27 17:23:57,133][20347] Updated weights for policy 0, policy_version 1696628 (0.0006) [2023-12-27 17:23:57,185][20347] Updated weights for policy 0, policy_version 1696638 (0.0006) [2023-12-27 17:23:57,236][20347] Updated weights for policy 0, policy_version 1696648 (0.0008) [2023-12-27 17:23:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 868401152. Throughput: 0: 10243.0, 1: 10442.0. Samples: 868384232. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:23:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:23:57,720][20381] Updated weights for policy 1, policy_version 1695057 (0.0007) [2023-12-27 17:23:57,768][20381] Updated weights for policy 1, policy_version 1695067 (0.0010) [2023-12-27 17:23:57,823][20381] Updated weights for policy 1, policy_version 1695077 (0.0010) [2023-12-27 17:23:57,825][20347] Updated weights for policy 0, policy_version 1696658 (0.0010) [2023-12-27 17:23:57,877][20347] Updated weights for policy 0, policy_version 1696668 (0.0010) [2023-12-27 17:23:57,879][20381] Updated weights for policy 1, policy_version 1695087 (0.0010) [2023-12-27 17:23:57,925][20347] Updated weights for policy 0, policy_version 1696678 (0.0010) [2023-12-27 17:23:58,597][20381] Updated weights for policy 1, policy_version 1695097 (0.0008) [2023-12-27 17:23:58,627][20347] Updated weights for policy 0, policy_version 1696688 (0.0008) [2023-12-27 17:23:58,661][20381] Updated weights for policy 1, policy_version 1695107 (0.0011) [2023-12-27 17:23:58,688][20347] Updated weights for policy 0, policy_version 1696698 (0.0007) [2023-12-27 17:23:58,727][20381] Updated weights for policy 1, policy_version 1695117 (0.0012) [2023-12-27 17:23:58,756][20347] Updated weights for policy 0, policy_version 1696708 (0.0008) [2023-12-27 17:23:59,516][20381] Updated weights for policy 1, policy_version 1695127 (0.0007) [2023-12-27 17:23:59,536][20347] Updated weights for policy 0, policy_version 1696718 (0.0007) [2023-12-27 17:23:59,571][20381] Updated weights for policy 1, policy_version 1695137 (0.0008) [2023-12-27 17:23:59,591][20347] Updated weights for policy 0, policy_version 1696728 (0.0007) [2023-12-27 17:23:59,625][20381] Updated weights for policy 1, policy_version 1695147 (0.0006) [2023-12-27 17:23:59,652][20347] Updated weights for policy 0, policy_version 1696738 (0.0007) [2023-12-27 17:24:00,341][20381] Updated weights for policy 1, policy_version 1695157 (0.0007) [2023-12-27 17:24:00,343][20347] Updated weights for policy 0, policy_version 1696748 (0.0006) [2023-12-27 17:24:00,395][20347] Updated weights for policy 0, policy_version 1696758 (0.0006) [2023-12-27 17:24:00,397][20381] Updated weights for policy 1, policy_version 1695167 (0.0008) [2023-12-27 17:24:00,446][20347] Updated weights for policy 0, policy_version 1696768 (0.0006) [2023-12-27 17:24:00,454][20381] Updated weights for policy 1, policy_version 1695177 (0.0007) [2023-12-27 17:24:01,023][20381] Updated weights for policy 1, policy_version 1695187 (0.0007) [2023-12-27 17:24:01,093][20381] Updated weights for policy 1, policy_version 1695197 (0.0009) [2023-12-27 17:24:01,156][20381] Updated weights for policy 1, policy_version 1695207 (0.0008) [2023-12-27 17:24:01,198][20347] Updated weights for policy 0, policy_version 1696778 (0.0006) [2023-12-27 17:24:01,261][20347] Updated weights for policy 0, policy_version 1696788 (0.0008) [2023-12-27 17:24:01,316][20347] Updated weights for policy 0, policy_version 1696798 (0.0006) [2023-12-27 17:24:01,379][20347] Updated weights for policy 0, policy_version 1696808 (0.0007) [2023-12-27 17:24:01,783][20381] Updated weights for policy 1, policy_version 1695217 (0.0008) [2023-12-27 17:24:01,838][20381] Updated weights for policy 1, policy_version 1695227 (0.0007) [2023-12-27 17:24:01,893][20381] Updated weights for policy 1, policy_version 1695237 (0.0007) [2023-12-27 17:24:01,948][20381] Updated weights for policy 1, policy_version 1695247 (0.0007) [2023-12-27 17:24:02,107][20347] Updated weights for policy 0, policy_version 1696818 (0.0007) [2023-12-27 17:24:02,170][20347] Updated weights for policy 0, policy_version 1696828 (0.0006) [2023-12-27 17:24:02,228][20347] Updated weights for policy 0, policy_version 1696838 (0.0005) [2023-12-27 17:24:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 868499456. Throughput: 0: 10242.0, 1: 10429.8. Samples: 868506852. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:24:02,649][20381] Updated weights for policy 1, policy_version 1695257 (0.0008) [2023-12-27 17:24:02,714][20381] Updated weights for policy 1, policy_version 1695267 (0.0007) [2023-12-27 17:24:02,778][20381] Updated weights for policy 1, policy_version 1695277 (0.0007) [2023-12-27 17:24:02,864][20347] Updated weights for policy 0, policy_version 1696848 (0.0009) [2023-12-27 17:24:02,916][20347] Updated weights for policy 0, policy_version 1696858 (0.0010) [2023-12-27 17:24:02,967][20347] Updated weights for policy 0, policy_version 1696868 (0.0006) [2023-12-27 17:24:03,419][20381] Updated weights for policy 1, policy_version 1695287 (0.0009) [2023-12-27 17:24:03,468][20381] Updated weights for policy 1, policy_version 1695297 (0.0010) [2023-12-27 17:24:03,526][20381] Updated weights for policy 1, policy_version 1695307 (0.0005) [2023-12-27 17:24:03,578][20347] Updated weights for policy 0, policy_version 1696878 (0.0007) [2023-12-27 17:24:03,629][20347] Updated weights for policy 0, policy_version 1696888 (0.0006) [2023-12-27 17:24:03,679][20347] Updated weights for policy 0, policy_version 1696898 (0.0006) [2023-12-27 17:24:04,103][20381] Updated weights for policy 1, policy_version 1695317 (0.0008) [2023-12-27 17:24:04,162][20381] Updated weights for policy 1, policy_version 1695327 (0.0010) [2023-12-27 17:24:04,220][20381] Updated weights for policy 1, policy_version 1695337 (0.0011) [2023-12-27 17:24:04,320][20347] Updated weights for policy 0, policy_version 1696908 (0.0006) [2023-12-27 17:24:04,373][20347] Updated weights for policy 0, policy_version 1696918 (0.0008) [2023-12-27 17:24:04,428][20347] Updated weights for policy 0, policy_version 1696928 (0.0007) [2023-12-27 17:24:04,929][20381] Updated weights for policy 1, policy_version 1695347 (0.0011) [2023-12-27 17:24:04,977][20381] Updated weights for policy 1, policy_version 1695357 (0.0010) [2023-12-27 17:24:05,026][20381] Updated weights for policy 1, policy_version 1695367 (0.0010) [2023-12-27 17:24:05,140][20347] Updated weights for policy 0, policy_version 1696938 (0.0007) [2023-12-27 17:24:05,183][20347] Updated weights for policy 0, policy_version 1696948 (0.0007) [2023-12-27 17:24:05,237][20347] Updated weights for policy 0, policy_version 1696958 (0.0007) [2023-12-27 17:24:05,297][20347] Updated weights for policy 0, policy_version 1696968 (0.0008) [2023-12-27 17:24:05,733][20381] Updated weights for policy 1, policy_version 1695377 (0.0011) [2023-12-27 17:24:05,794][20381] Updated weights for policy 1, policy_version 1695387 (0.0011) [2023-12-27 17:24:05,843][20381] Updated weights for policy 1, policy_version 1695397 (0.0010) [2023-12-27 17:24:05,900][20381] Updated weights for policy 1, policy_version 1695407 (0.0011) [2023-12-27 17:24:06,010][20347] Updated weights for policy 0, policy_version 1696978 (0.0008) [2023-12-27 17:24:06,059][20347] Updated weights for policy 0, policy_version 1696988 (0.0009) [2023-12-27 17:24:06,115][20347] Updated weights for policy 0, policy_version 1696998 (0.0008) [2023-12-27 17:24:06,525][20381] Updated weights for policy 1, policy_version 1695417 (0.0011) [2023-12-27 17:24:06,581][20381] Updated weights for policy 1, policy_version 1695427 (0.0011) [2023-12-27 17:24:06,630][20381] Updated weights for policy 1, policy_version 1695437 (0.0007) [2023-12-27 17:24:06,753][20347] Updated weights for policy 0, policy_version 1697008 (0.0006) [2023-12-27 17:24:06,808][20347] Updated weights for policy 0, policy_version 1697018 (0.0008) [2023-12-27 17:24:06,857][20347] Updated weights for policy 0, policy_version 1697028 (0.0006) [2023-12-27 17:24:07,308][20381] Updated weights for policy 1, policy_version 1695447 (0.0009) [2023-12-27 17:24:07,354][20381] Updated weights for policy 1, policy_version 1695457 (0.0011) [2023-12-27 17:24:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 868597760. Throughput: 0: 10296.0, 1: 10445.4. Samples: 868570356. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:07,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:24:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001697032_434520064.pth... [2023-12-27 17:24:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001695848_434216960.pth [2023-12-27 17:24:07,399][20381] Updated weights for policy 1, policy_version 1695467 (0.0006) [2023-12-27 17:24:07,422][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001695472_434085888.pth... [2023-12-27 17:24:07,425][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001694224_433766400.pth [2023-12-27 17:24:07,504][20347] Updated weights for policy 0, policy_version 1697038 (0.0008) [2023-12-27 17:24:07,563][20347] Updated weights for policy 0, policy_version 1697048 (0.0011) [2023-12-27 17:24:07,633][20347] Updated weights for policy 0, policy_version 1697058 (0.0011) [2023-12-27 17:24:08,125][20381] Updated weights for policy 1, policy_version 1695477 (0.0007) [2023-12-27 17:24:08,182][20381] Updated weights for policy 1, policy_version 1695487 (0.0009) [2023-12-27 17:24:08,233][20381] Updated weights for policy 1, policy_version 1695497 (0.0008) [2023-12-27 17:24:08,258][20347] Updated weights for policy 0, policy_version 1697068 (0.0010) [2023-12-27 17:24:08,314][20347] Updated weights for policy 0, policy_version 1697078 (0.0008) [2023-12-27 17:24:08,371][20347] Updated weights for policy 0, policy_version 1697088 (0.0010) [2023-12-27 17:24:08,845][20381] Updated weights for policy 1, policy_version 1695507 (0.0007) [2023-12-27 17:24:08,902][20381] Updated weights for policy 1, policy_version 1695517 (0.0008) [2023-12-27 17:24:08,954][20381] Updated weights for policy 1, policy_version 1695527 (0.0008) [2023-12-27 17:24:09,051][20347] Updated weights for policy 0, policy_version 1697098 (0.0011) [2023-12-27 17:24:09,099][20347] Updated weights for policy 0, policy_version 1697108 (0.0010) [2023-12-27 17:24:09,154][20347] Updated weights for policy 0, policy_version 1697118 (0.0010) [2023-12-27 17:24:09,206][20347] Updated weights for policy 0, policy_version 1697128 (0.0010) [2023-12-27 17:24:09,629][20381] Updated weights for policy 1, policy_version 1695537 (0.0008) [2023-12-27 17:24:09,682][20381] Updated weights for policy 1, policy_version 1695547 (0.0006) [2023-12-27 17:24:09,734][20381] Updated weights for policy 1, policy_version 1695557 (0.0007) [2023-12-27 17:24:09,795][20381] Updated weights for policy 1, policy_version 1695567 (0.0008) [2023-12-27 17:24:10,085][20347] Updated weights for policy 0, policy_version 1697138 (0.0008) [2023-12-27 17:24:10,152][20347] Updated weights for policy 0, policy_version 1697148 (0.0007) [2023-12-27 17:24:10,220][20347] Updated weights for policy 0, policy_version 1697158 (0.0007) [2023-12-27 17:24:10,473][20381] Updated weights for policy 1, policy_version 1695577 (0.0008) [2023-12-27 17:24:10,530][20381] Updated weights for policy 1, policy_version 1695587 (0.0008) [2023-12-27 17:24:10,589][20381] Updated weights for policy 1, policy_version 1695597 (0.0007) [2023-12-27 17:24:10,936][20347] Updated weights for policy 0, policy_version 1697168 (0.0008) [2023-12-27 17:24:10,998][20347] Updated weights for policy 0, policy_version 1697178 (0.0007) [2023-12-27 17:24:11,069][20347] Updated weights for policy 0, policy_version 1697188 (0.0008) [2023-12-27 17:24:11,282][20381] Updated weights for policy 1, policy_version 1695607 (0.0008) [2023-12-27 17:24:11,344][20381] Updated weights for policy 1, policy_version 1695617 (0.0008) [2023-12-27 17:24:11,414][20381] Updated weights for policy 1, policy_version 1695627 (0.0009) [2023-12-27 17:24:11,844][20347] Updated weights for policy 0, policy_version 1697198 (0.0007) [2023-12-27 17:24:11,905][20347] Updated weights for policy 0, policy_version 1697208 (0.0007) [2023-12-27 17:24:11,966][20347] Updated weights for policy 0, policy_version 1697218 (0.0008) [2023-12-27 17:24:12,156][20381] Updated weights for policy 1, policy_version 1695637 (0.0008) [2023-12-27 17:24:12,209][20381] Updated weights for policy 1, policy_version 1695647 (0.0005) [2023-12-27 17:24:12,271][20381] Updated weights for policy 1, policy_version 1695657 (0.0007) [2023-12-27 17:24:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 868704256. Throughput: 0: 10277.4, 1: 10422.1. Samples: 868692268. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:12,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 17:24:12,686][20347] Updated weights for policy 0, policy_version 1697228 (0.0007) [2023-12-27 17:24:12,746][20347] Updated weights for policy 0, policy_version 1697238 (0.0007) [2023-12-27 17:24:12,809][20347] Updated weights for policy 0, policy_version 1697248 (0.0007) [2023-12-27 17:24:12,978][20381] Updated weights for policy 1, policy_version 1695667 (0.0008) [2023-12-27 17:24:13,033][20381] Updated weights for policy 1, policy_version 1695677 (0.0006) [2023-12-27 17:24:13,093][20381] Updated weights for policy 1, policy_version 1695687 (0.0006) [2023-12-27 17:24:13,534][20347] Updated weights for policy 0, policy_version 1697258 (0.0007) [2023-12-27 17:24:13,587][20347] Updated weights for policy 0, policy_version 1697268 (0.0008) [2023-12-27 17:24:13,632][20381] Updated weights for policy 1, policy_version 1695697 (0.0005) [2023-12-27 17:24:13,645][20347] Updated weights for policy 0, policy_version 1697278 (0.0007) [2023-12-27 17:24:13,687][20381] Updated weights for policy 1, policy_version 1695707 (0.0007) [2023-12-27 17:24:13,697][20347] Updated weights for policy 0, policy_version 1697288 (0.0005) [2023-12-27 17:24:13,741][20381] Updated weights for policy 1, policy_version 1695717 (0.0008) [2023-12-27 17:24:13,797][20381] Updated weights for policy 1, policy_version 1695727 (0.0008) [2023-12-27 17:24:14,380][20347] Updated weights for policy 0, policy_version 1697298 (0.0007) [2023-12-27 17:24:14,383][20381] Updated weights for policy 1, policy_version 1695737 (0.0006) [2023-12-27 17:24:14,429][20381] Updated weights for policy 1, policy_version 1695747 (0.0005) [2023-12-27 17:24:14,436][20347] Updated weights for policy 0, policy_version 1697308 (0.0008) [2023-12-27 17:24:14,476][20381] Updated weights for policy 1, policy_version 1695757 (0.0005) [2023-12-27 17:24:14,495][20347] Updated weights for policy 0, policy_version 1697318 (0.0008) [2023-12-27 17:24:15,062][20381] Updated weights for policy 1, policy_version 1695767 (0.0006) [2023-12-27 17:24:15,124][20381] Updated weights for policy 1, policy_version 1695777 (0.0006) [2023-12-27 17:24:15,183][20381] Updated weights for policy 1, policy_version 1695787 (0.0006) [2023-12-27 17:24:15,226][20347] Updated weights for policy 0, policy_version 1697328 (0.0008) [2023-12-27 17:24:15,287][20347] Updated weights for policy 0, policy_version 1697338 (0.0007) [2023-12-27 17:24:15,370][20347] Updated weights for policy 0, policy_version 1697348 (0.0008) [2023-12-27 17:24:15,806][20381] Updated weights for policy 1, policy_version 1695797 (0.0009) [2023-12-27 17:24:15,861][20381] Updated weights for policy 1, policy_version 1695807 (0.0010) [2023-12-27 17:24:15,909][20381] Updated weights for policy 1, policy_version 1695817 (0.0010) [2023-12-27 17:24:16,036][20347] Updated weights for policy 0, policy_version 1697358 (0.0008) [2023-12-27 17:24:16,089][20347] Updated weights for policy 0, policy_version 1697368 (0.0007) [2023-12-27 17:24:16,143][20347] Updated weights for policy 0, policy_version 1697378 (0.0007) [2023-12-27 17:24:16,581][20381] Updated weights for policy 1, policy_version 1695827 (0.0010) [2023-12-27 17:24:16,639][20381] Updated weights for policy 1, policy_version 1695837 (0.0006) [2023-12-27 17:24:16,694][20381] Updated weights for policy 1, policy_version 1695847 (0.0005) [2023-12-27 17:24:16,865][20347] Updated weights for policy 0, policy_version 1697388 (0.0007) [2023-12-27 17:24:16,924][20347] Updated weights for policy 0, policy_version 1697398 (0.0008) [2023-12-27 17:24:16,976][20347] Updated weights for policy 0, policy_version 1697408 (0.0008) [2023-12-27 17:24:17,316][20381] Updated weights for policy 1, policy_version 1695857 (0.0006) [2023-12-27 17:24:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 868802560. Throughput: 0: 10189.0, 1: 10458.9. Samples: 868814608. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:24:17,364][20381] Updated weights for policy 1, policy_version 1695867 (0.0010) [2023-12-27 17:24:17,415][20381] Updated weights for policy 1, policy_version 1695877 (0.0010) [2023-12-27 17:24:17,471][20381] Updated weights for policy 1, policy_version 1695887 (0.0008) [2023-12-27 17:24:17,644][20347] Updated weights for policy 0, policy_version 1697418 (0.0006) [2023-12-27 17:24:17,696][20347] Updated weights for policy 0, policy_version 1697428 (0.0005) [2023-12-27 17:24:17,748][20347] Updated weights for policy 0, policy_version 1697438 (0.0005) [2023-12-27 17:24:17,799][20347] Updated weights for policy 0, policy_version 1697448 (0.0005) [2023-12-27 17:24:18,096][20381] Updated weights for policy 1, policy_version 1695897 (0.0008) [2023-12-27 17:24:18,154][20381] Updated weights for policy 1, policy_version 1695907 (0.0007) [2023-12-27 17:24:18,215][20381] Updated weights for policy 1, policy_version 1695917 (0.0008) [2023-12-27 17:24:18,452][20347] Updated weights for policy 0, policy_version 1697458 (0.0011) [2023-12-27 17:24:18,500][20347] Updated weights for policy 0, policy_version 1697468 (0.0010) [2023-12-27 17:24:18,555][20347] Updated weights for policy 0, policy_version 1697478 (0.0010) [2023-12-27 17:24:18,877][20381] Updated weights for policy 1, policy_version 1695927 (0.0006) [2023-12-27 17:24:18,937][20381] Updated weights for policy 1, policy_version 1695937 (0.0007) [2023-12-27 17:24:18,989][20381] Updated weights for policy 1, policy_version 1695947 (0.0008) [2023-12-27 17:24:19,280][20347] Updated weights for policy 0, policy_version 1697488 (0.0009) [2023-12-27 17:24:19,340][20347] Updated weights for policy 0, policy_version 1697498 (0.0008) [2023-12-27 17:24:19,412][20347] Updated weights for policy 0, policy_version 1697508 (0.0008) [2023-12-27 17:24:19,641][20381] Updated weights for policy 1, policy_version 1695957 (0.0007) [2023-12-27 17:24:19,698][20381] Updated weights for policy 1, policy_version 1695967 (0.0007) [2023-12-27 17:24:19,763][20381] Updated weights for policy 1, policy_version 1695977 (0.0008) [2023-12-27 17:24:20,173][20347] Updated weights for policy 0, policy_version 1697518 (0.0008) [2023-12-27 17:24:20,232][20347] Updated weights for policy 0, policy_version 1697528 (0.0008) [2023-12-27 17:24:20,295][20347] Updated weights for policy 0, policy_version 1697538 (0.0008) [2023-12-27 17:24:20,425][20381] Updated weights for policy 1, policy_version 1695987 (0.0008) [2023-12-27 17:24:20,490][20381] Updated weights for policy 1, policy_version 1695997 (0.0007) [2023-12-27 17:24:20,553][20381] Updated weights for policy 1, policy_version 1696007 (0.0008) [2023-12-27 17:24:21,053][20347] Updated weights for policy 0, policy_version 1697548 (0.0009) [2023-12-27 17:24:21,105][20347] Updated weights for policy 0, policy_version 1697558 (0.0008) [2023-12-27 17:24:21,166][20347] Updated weights for policy 0, policy_version 1697568 (0.0008) [2023-12-27 17:24:21,212][20381] Updated weights for policy 1, policy_version 1696017 (0.0006) [2023-12-27 17:24:21,277][20381] Updated weights for policy 1, policy_version 1696027 (0.0008) [2023-12-27 17:24:21,343][20381] Updated weights for policy 1, policy_version 1696037 (0.0006) [2023-12-27 17:24:21,412][20381] Updated weights for policy 1, policy_version 1696047 (0.0008) [2023-12-27 17:24:21,947][20347] Updated weights for policy 0, policy_version 1697578 (0.0007) [2023-12-27 17:24:22,001][20347] Updated weights for policy 0, policy_version 1697588 (0.0006) [2023-12-27 17:24:22,056][20347] Updated weights for policy 0, policy_version 1697598 (0.0007) [2023-12-27 17:24:22,112][20381] Updated weights for policy 1, policy_version 1696057 (0.0006) [2023-12-27 17:24:22,114][20347] Updated weights for policy 0, policy_version 1697608 (0.0006) [2023-12-27 17:24:22,169][20381] Updated weights for policy 1, policy_version 1696067 (0.0008) [2023-12-27 17:24:22,222][20381] Updated weights for policy 1, policy_version 1696077 (0.0008) [2023-12-27 17:24:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 868909056. Throughput: 0: 10160.9, 1: 10444.4. Samples: 868876300. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:22,363][19170] Avg episode reward: [(0, '990.230'), (1, '1000.000')] [2023-12-27 17:24:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001696080_434241536.pth... [2023-12-27 17:24:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001697608_434667520.pth... [2023-12-27 17:24:22,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001694832_433922048.pth [2023-12-27 17:24:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001696424_434364416.pth [2023-12-27 17:24:22,809][20347] Updated weights for policy 0, policy_version 1697618 (0.0008) [2023-12-27 17:24:22,870][20347] Updated weights for policy 0, policy_version 1697628 (0.0008) [2023-12-27 17:24:22,936][20347] Updated weights for policy 0, policy_version 1697638 (0.0008) [2023-12-27 17:24:22,953][20381] Updated weights for policy 1, policy_version 1696087 (0.0006) [2023-12-27 17:24:23,018][20381] Updated weights for policy 1, policy_version 1696097 (0.0007) [2023-12-27 17:24:23,078][20381] Updated weights for policy 1, policy_version 1696107 (0.0008) [2023-12-27 17:24:23,649][20347] Updated weights for policy 0, policy_version 1697648 (0.0008) [2023-12-27 17:24:23,707][20347] Updated weights for policy 0, policy_version 1697658 (0.0008) [2023-12-27 17:24:23,724][20381] Updated weights for policy 1, policy_version 1696117 (0.0006) [2023-12-27 17:24:23,759][20347] Updated weights for policy 0, policy_version 1697668 (0.0006) [2023-12-27 17:24:23,772][20381] Updated weights for policy 1, policy_version 1696127 (0.0005) [2023-12-27 17:24:23,829][20381] Updated weights for policy 1, policy_version 1696137 (0.0005) [2023-12-27 17:24:24,389][20381] Updated weights for policy 1, policy_version 1696147 (0.0006) [2023-12-27 17:24:24,442][20381] Updated weights for policy 1, policy_version 1696157 (0.0007) [2023-12-27 17:24:24,496][20381] Updated weights for policy 1, policy_version 1696167 (0.0006) [2023-12-27 17:24:24,509][20347] Updated weights for policy 0, policy_version 1697678 (0.0006) [2023-12-27 17:24:24,564][20347] Updated weights for policy 0, policy_version 1697688 (0.0007) [2023-12-27 17:24:24,622][20347] Updated weights for policy 0, policy_version 1697698 (0.0006) [2023-12-27 17:24:25,145][20381] Updated weights for policy 1, policy_version 1696177 (0.0007) [2023-12-27 17:24:25,194][20381] Updated weights for policy 1, policy_version 1696187 (0.0006) [2023-12-27 17:24:25,208][20347] Updated weights for policy 0, policy_version 1697708 (0.0005) [2023-12-27 17:24:25,239][20381] Updated weights for policy 1, policy_version 1696197 (0.0006) [2023-12-27 17:24:25,272][20347] Updated weights for policy 0, policy_version 1697718 (0.0005) [2023-12-27 17:24:25,294][20381] Updated weights for policy 1, policy_version 1696207 (0.0006) [2023-12-27 17:24:25,331][20347] Updated weights for policy 0, policy_version 1697728 (0.0007) [2023-12-27 17:24:25,878][20381] Updated weights for policy 1, policy_version 1696217 (0.0008) [2023-12-27 17:24:25,937][20381] Updated weights for policy 1, policy_version 1696227 (0.0007) [2023-12-27 17:24:25,998][20381] Updated weights for policy 1, policy_version 1696237 (0.0009) [2023-12-27 17:24:26,019][20347] Updated weights for policy 0, policy_version 1697738 (0.0007) [2023-12-27 17:24:26,074][20347] Updated weights for policy 0, policy_version 1697748 (0.0007) [2023-12-27 17:24:26,130][20347] Updated weights for policy 0, policy_version 1697758 (0.0007) [2023-12-27 17:24:26,191][20347] Updated weights for policy 0, policy_version 1697768 (0.0007) [2023-12-27 17:24:26,664][20381] Updated weights for policy 1, policy_version 1696247 (0.0010) [2023-12-27 17:24:26,712][20381] Updated weights for policy 1, policy_version 1696257 (0.0008) [2023-12-27 17:24:26,755][20381] Updated weights for policy 1, policy_version 1696267 (0.0007) [2023-12-27 17:24:26,777][20347] Updated weights for policy 0, policy_version 1697778 (0.0005) [2023-12-27 17:24:26,821][20347] Updated weights for policy 0, policy_version 1697788 (0.0005) [2023-12-27 17:24:26,873][20347] Updated weights for policy 0, policy_version 1697798 (0.0005) [2023-12-27 17:24:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 869007360. Throughput: 0: 10130.0, 1: 10497.8. Samples: 868998064. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:27,363][19170] Avg episode reward: [(0, '990.230'), (1, '1000.000')] [2023-12-27 17:24:27,427][20347] Updated weights for policy 0, policy_version 1697808 (0.0005) [2023-12-27 17:24:27,483][20347] Updated weights for policy 0, policy_version 1697818 (0.0007) [2023-12-27 17:24:27,526][20381] Updated weights for policy 1, policy_version 1696277 (0.0007) [2023-12-27 17:24:27,537][20347] Updated weights for policy 0, policy_version 1697828 (0.0010) [2023-12-27 17:24:27,580][20381] Updated weights for policy 1, policy_version 1696287 (0.0007) [2023-12-27 17:24:27,642][20381] Updated weights for policy 1, policy_version 1696297 (0.0006) [2023-12-27 17:24:28,161][20347] Updated weights for policy 0, policy_version 1697838 (0.0009) [2023-12-27 17:24:28,218][20347] Updated weights for policy 0, policy_version 1697848 (0.0008) [2023-12-27 17:24:28,263][20381] Updated weights for policy 1, policy_version 1696307 (0.0007) [2023-12-27 17:24:28,279][20347] Updated weights for policy 0, policy_version 1697858 (0.0006) [2023-12-27 17:24:28,310][20381] Updated weights for policy 1, policy_version 1696317 (0.0007) [2023-12-27 17:24:28,370][20381] Updated weights for policy 1, policy_version 1696327 (0.0008) [2023-12-27 17:24:28,970][20347] Updated weights for policy 0, policy_version 1697868 (0.0010) [2023-12-27 17:24:29,022][20347] Updated weights for policy 0, policy_version 1697878 (0.0010) [2023-12-27 17:24:29,032][20291] KL-divergence is very high: 113.8712 [2023-12-27 17:24:29,041][20381] Updated weights for policy 1, policy_version 1696337 (0.0008) [2023-12-27 17:24:29,076][20347] Updated weights for policy 0, policy_version 1697888 (0.0010) [2023-12-27 17:24:29,096][20381] Updated weights for policy 1, policy_version 1696347 (0.0007) [2023-12-27 17:24:29,154][20381] Updated weights for policy 1, policy_version 1696357 (0.0006) [2023-12-27 17:24:29,215][20381] Updated weights for policy 1, policy_version 1696367 (0.0007) [2023-12-27 17:24:29,768][20347] Updated weights for policy 0, policy_version 1697898 (0.0010) [2023-12-27 17:24:29,831][20347] Updated weights for policy 0, policy_version 1697908 (0.0010) [2023-12-27 17:24:29,891][20381] Updated weights for policy 1, policy_version 1696377 (0.0007) [2023-12-27 17:24:29,891][20347] Updated weights for policy 0, policy_version 1697918 (0.0008) [2023-12-27 17:24:29,949][20347] Updated weights for policy 0, policy_version 1697928 (0.0008) [2023-12-27 17:24:29,951][20381] Updated weights for policy 1, policy_version 1696387 (0.0007) [2023-12-27 17:24:30,006][20381] Updated weights for policy 1, policy_version 1696397 (0.0008) [2023-12-27 17:24:30,627][20347] Updated weights for policy 0, policy_version 1697938 (0.0011) [2023-12-27 17:24:30,662][20381] Updated weights for policy 1, policy_version 1696407 (0.0006) [2023-12-27 17:24:30,685][20347] Updated weights for policy 0, policy_version 1697948 (0.0010) [2023-12-27 17:24:30,724][20381] Updated weights for policy 1, policy_version 1696417 (0.0005) [2023-12-27 17:24:30,740][20347] Updated weights for policy 0, policy_version 1697958 (0.0010) [2023-12-27 17:24:30,779][20381] Updated weights for policy 1, policy_version 1696427 (0.0006) [2023-12-27 17:24:31,440][20381] Updated weights for policy 1, policy_version 1696437 (0.0006) [2023-12-27 17:24:31,483][20347] Updated weights for policy 0, policy_version 1697968 (0.0008) [2023-12-27 17:24:31,501][20381] Updated weights for policy 1, policy_version 1696447 (0.0006) [2023-12-27 17:24:31,537][20347] Updated weights for policy 0, policy_version 1697978 (0.0008) [2023-12-27 17:24:31,555][20381] Updated weights for policy 1, policy_version 1696457 (0.0007) [2023-12-27 17:24:31,591][20347] Updated weights for policy 0, policy_version 1697988 (0.0006) [2023-12-27 17:24:32,248][20347] Updated weights for policy 0, policy_version 1697998 (0.0007) [2023-12-27 17:24:32,289][20381] Updated weights for policy 1, policy_version 1696467 (0.0007) [2023-12-27 17:24:32,308][20347] Updated weights for policy 0, policy_version 1698008 (0.0006) [2023-12-27 17:24:32,354][20381] Updated weights for policy 1, policy_version 1696477 (0.0007) [2023-12-27 17:24:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 869105664. Throughput: 0: 10133.0, 1: 10454.8. Samples: 869122004. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-12-27 17:24:32,363][19170] Avg episode reward: [(0, '990.200'), (1, '1000.000')] [2023-12-27 17:24:32,365][20347] Updated weights for policy 0, policy_version 1698018 (0.0008) [2023-12-27 17:24:32,412][20381] Updated weights for policy 1, policy_version 1696487 (0.0007) [2023-12-27 17:24:33,030][20347] Updated weights for policy 0, policy_version 1698028 (0.0007) [2023-12-27 17:24:33,044][20381] Updated weights for policy 1, policy_version 1696497 (0.0005) [2023-12-27 17:24:33,083][20347] Updated weights for policy 0, policy_version 1698038 (0.0005) [2023-12-27 17:24:33,095][20381] Updated weights for policy 1, policy_version 1696507 (0.0007) [2023-12-27 17:24:33,134][20347] Updated weights for policy 0, policy_version 1698048 (0.0005) [2023-12-27 17:24:33,146][20381] Updated weights for policy 1, policy_version 1696517 (0.0009) [2023-12-27 17:24:33,203][20381] Updated weights for policy 1, policy_version 1696527 (0.0010) [2023-12-27 17:24:33,655][20347] Updated weights for policy 0, policy_version 1698058 (0.0006) [2023-12-27 17:24:33,702][20347] Updated weights for policy 0, policy_version 1698068 (0.0007) [2023-12-27 17:24:33,752][20347] Updated weights for policy 0, policy_version 1698078 (0.0007) [2023-12-27 17:24:33,791][20381] Updated weights for policy 1, policy_version 1696537 (0.0009) [2023-12-27 17:24:33,798][20347] Updated weights for policy 0, policy_version 1698088 (0.0008) [2023-12-27 17:24:33,845][20381] Updated weights for policy 1, policy_version 1696547 (0.0010) [2023-12-27 17:24:33,902][20381] Updated weights for policy 1, policy_version 1696557 (0.0010) [2023-12-27 17:24:34,466][20347] Updated weights for policy 0, policy_version 1698098 (0.0008) [2023-12-27 17:24:34,524][20347] Updated weights for policy 0, policy_version 1698108 (0.0007) [2023-12-27 17:24:34,580][20347] Updated weights for policy 0, policy_version 1698118 (0.0007) [2023-12-27 17:24:34,581][20381] Updated weights for policy 1, policy_version 1696567 (0.0011) [2023-12-27 17:24:34,635][20381] Updated weights for policy 1, policy_version 1696577 (0.0010) [2023-12-27 17:24:34,689][20381] Updated weights for policy 1, policy_version 1696587 (0.0011) [2023-12-27 17:24:35,246][20347] Updated weights for policy 0, policy_version 1698128 (0.0006) [2023-12-27 17:24:35,295][20347] Updated weights for policy 0, policy_version 1698138 (0.0005) [2023-12-27 17:24:35,340][20381] Updated weights for policy 1, policy_version 1696597 (0.0011) [2023-12-27 17:24:35,350][20347] Updated weights for policy 0, policy_version 1698148 (0.0007) [2023-12-27 17:24:35,392][20381] Updated weights for policy 1, policy_version 1696607 (0.0010) [2023-12-27 17:24:35,446][20381] Updated weights for policy 1, policy_version 1696617 (0.0010) [2023-12-27 17:24:35,912][20347] Updated weights for policy 0, policy_version 1698158 (0.0007) [2023-12-27 17:24:35,968][20347] Updated weights for policy 0, policy_version 1698168 (0.0007) [2023-12-27 17:24:36,021][20347] Updated weights for policy 0, policy_version 1698178 (0.0007) [2023-12-27 17:24:36,145][20381] Updated weights for policy 1, policy_version 1696627 (0.0010) [2023-12-27 17:24:36,201][20381] Updated weights for policy 1, policy_version 1696637 (0.0010) [2023-12-27 17:24:36,258][20381] Updated weights for policy 1, policy_version 1696647 (0.0011) [2023-12-27 17:24:36,746][20347] Updated weights for policy 0, policy_version 1698188 (0.0006) [2023-12-27 17:24:36,801][20347] Updated weights for policy 0, policy_version 1698198 (0.0005) [2023-12-27 17:24:36,860][20347] Updated weights for policy 0, policy_version 1698208 (0.0005) [2023-12-27 17:24:36,892][20381] Updated weights for policy 1, policy_version 1696657 (0.0011) [2023-12-27 17:24:36,949][20381] Updated weights for policy 1, policy_version 1696667 (0.0011) [2023-12-27 17:24:37,002][20381] Updated weights for policy 1, policy_version 1696677 (0.0011) [2023-12-27 17:24:37,058][20381] Updated weights for policy 1, policy_version 1696687 (0.0011) [2023-12-27 17:24:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 869220352. Throughput: 0: 10171.2, 1: 10492.4. Samples: 869187520. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:24:37,364][19170] Avg episode reward: [(0, '980.570'), (1, '1000.000')] [2023-12-27 17:24:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001696688_434397184.pth... [2023-12-27 17:24:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001698216_434823168.pth... [2023-12-27 17:24:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001695472_434085888.pth [2023-12-27 17:24:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001697032_434520064.pth [2023-12-27 17:24:37,440][20347] Updated weights for policy 0, policy_version 1698218 (0.0006) [2023-12-27 17:24:37,497][20347] Updated weights for policy 0, policy_version 1698228 (0.0007) [2023-12-27 17:24:37,557][20347] Updated weights for policy 0, policy_version 1698238 (0.0007) [2023-12-27 17:24:37,620][20347] Updated weights for policy 0, policy_version 1698248 (0.0007) [2023-12-27 17:24:37,769][20381] Updated weights for policy 1, policy_version 1696697 (0.0011) [2023-12-27 17:24:37,831][20381] Updated weights for policy 1, policy_version 1696707 (0.0011) [2023-12-27 17:24:37,892][20381] Updated weights for policy 1, policy_version 1696717 (0.0007) [2023-12-27 17:24:38,368][20347] Updated weights for policy 0, policy_version 1698258 (0.0008) [2023-12-27 17:24:38,428][20347] Updated weights for policy 0, policy_version 1698268 (0.0008) [2023-12-27 17:24:38,491][20347] Updated weights for policy 0, policy_version 1698278 (0.0008) [2023-12-27 17:24:38,626][20381] Updated weights for policy 1, policy_version 1696727 (0.0008) [2023-12-27 17:24:38,679][20381] Updated weights for policy 1, policy_version 1696737 (0.0009) [2023-12-27 17:24:38,739][20381] Updated weights for policy 1, policy_version 1696747 (0.0009) [2023-12-27 17:24:39,209][20347] Updated weights for policy 0, policy_version 1698288 (0.0007) [2023-12-27 17:24:39,272][20347] Updated weights for policy 0, policy_version 1698298 (0.0007) [2023-12-27 17:24:39,332][20347] Updated weights for policy 0, policy_version 1698308 (0.0006) [2023-12-27 17:24:39,533][20381] Updated weights for policy 1, policy_version 1696757 (0.0007) [2023-12-27 17:24:39,588][20381] Updated weights for policy 1, policy_version 1696767 (0.0009) [2023-12-27 17:24:39,652][20381] Updated weights for policy 1, policy_version 1696777 (0.0007) [2023-12-27 17:24:40,050][20347] Updated weights for policy 0, policy_version 1698318 (0.0008) [2023-12-27 17:24:40,114][20347] Updated weights for policy 0, policy_version 1698328 (0.0007) [2023-12-27 17:24:40,172][20347] Updated weights for policy 0, policy_version 1698338 (0.0008) [2023-12-27 17:24:40,311][20381] Updated weights for policy 1, policy_version 1696787 (0.0009) [2023-12-27 17:24:40,365][20381] Updated weights for policy 1, policy_version 1696797 (0.0006) [2023-12-27 17:24:40,416][20381] Updated weights for policy 1, policy_version 1696807 (0.0008) [2023-12-27 17:24:40,787][20347] Updated weights for policy 0, policy_version 1698348 (0.0009) [2023-12-27 17:24:40,849][20347] Updated weights for policy 0, policy_version 1698358 (0.0009) [2023-12-27 17:24:40,904][20347] Updated weights for policy 0, policy_version 1698368 (0.0008) [2023-12-27 17:24:41,006][20381] Updated weights for policy 1, policy_version 1696817 (0.0006) [2023-12-27 17:24:41,073][20381] Updated weights for policy 1, policy_version 1696827 (0.0007) [2023-12-27 17:24:41,134][20381] Updated weights for policy 1, policy_version 1696837 (0.0008) [2023-12-27 17:24:41,198][20381] Updated weights for policy 1, policy_version 1696847 (0.0007) [2023-12-27 17:24:41,668][20347] Updated weights for policy 0, policy_version 1698378 (0.0006) [2023-12-27 17:24:41,721][20347] Updated weights for policy 0, policy_version 1698388 (0.0008) [2023-12-27 17:24:41,791][20347] Updated weights for policy 0, policy_version 1698398 (0.0007) [2023-12-27 17:24:41,847][20347] Updated weights for policy 0, policy_version 1698408 (0.0008) [2023-12-27 17:24:41,928][20381] Updated weights for policy 1, policy_version 1696857 (0.0008) [2023-12-27 17:24:41,992][20381] Updated weights for policy 1, policy_version 1696867 (0.0008) [2023-12-27 17:24:42,059][20381] Updated weights for policy 1, policy_version 1696877 (0.0008) [2023-12-27 17:24:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 869318656. Throughput: 0: 10149.1, 1: 10426.5. Samples: 869310140. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:24:42,363][19170] Avg episode reward: [(0, '990.290'), (1, '1000.000')] [2023-12-27 17:24:42,632][20347] Updated weights for policy 0, policy_version 1698418 (0.0008) [2023-12-27 17:24:42,690][20347] Updated weights for policy 0, policy_version 1698428 (0.0008) [2023-12-27 17:24:42,743][20347] Updated weights for policy 0, policy_version 1698438 (0.0008) [2023-12-27 17:24:42,849][20381] Updated weights for policy 1, policy_version 1696887 (0.0010) [2023-12-27 17:24:42,907][20381] Updated weights for policy 1, policy_version 1696897 (0.0011) [2023-12-27 17:24:42,961][20381] Updated weights for policy 1, policy_version 1696907 (0.0009) [2023-12-27 17:24:43,467][20347] Updated weights for policy 0, policy_version 1698448 (0.0007) [2023-12-27 17:24:43,517][20347] Updated weights for policy 0, policy_version 1698458 (0.0007) [2023-12-27 17:24:43,570][20347] Updated weights for policy 0, policy_version 1698468 (0.0007) [2023-12-27 17:24:43,598][20381] Updated weights for policy 1, policy_version 1696917 (0.0008) [2023-12-27 17:24:43,652][20381] Updated weights for policy 1, policy_version 1696927 (0.0010) [2023-12-27 17:24:43,710][20381] Updated weights for policy 1, policy_version 1696937 (0.0010) [2023-12-27 17:24:44,276][20347] Updated weights for policy 0, policy_version 1698478 (0.0006) [2023-12-27 17:24:44,327][20347] Updated weights for policy 0, policy_version 1698488 (0.0006) [2023-12-27 17:24:44,366][20381] Updated weights for policy 1, policy_version 1696947 (0.0008) [2023-12-27 17:24:44,374][20347] Updated weights for policy 0, policy_version 1698498 (0.0005) [2023-12-27 17:24:44,427][20381] Updated weights for policy 1, policy_version 1696957 (0.0006) [2023-12-27 17:24:44,486][20381] Updated weights for policy 1, policy_version 1696967 (0.0005) [2023-12-27 17:24:44,982][20347] Updated weights for policy 0, policy_version 1698508 (0.0008) [2023-12-27 17:24:45,039][20347] Updated weights for policy 0, policy_version 1698518 (0.0007) [2023-12-27 17:24:45,066][20381] Updated weights for policy 1, policy_version 1696977 (0.0005) [2023-12-27 17:24:45,094][20347] Updated weights for policy 0, policy_version 1698528 (0.0008) [2023-12-27 17:24:45,123][20381] Updated weights for policy 1, policy_version 1696987 (0.0006) [2023-12-27 17:24:45,182][20381] Updated weights for policy 1, policy_version 1696997 (0.0007) [2023-12-27 17:24:45,242][20381] Updated weights for policy 1, policy_version 1697007 (0.0008) [2023-12-27 17:24:45,737][20347] Updated weights for policy 0, policy_version 1698538 (0.0006) [2023-12-27 17:24:45,794][20347] Updated weights for policy 0, policy_version 1698548 (0.0005) [2023-12-27 17:24:45,851][20347] Updated weights for policy 0, policy_version 1698558 (0.0006) [2023-12-27 17:24:45,907][20347] Updated weights for policy 0, policy_version 1698568 (0.0008) [2023-12-27 17:24:45,972][20381] Updated weights for policy 1, policy_version 1697017 (0.0007) [2023-12-27 17:24:46,028][20381] Updated weights for policy 1, policy_version 1697027 (0.0006) [2023-12-27 17:24:46,094][20381] Updated weights for policy 1, policy_version 1697037 (0.0005) [2023-12-27 17:24:46,482][20347] Updated weights for policy 0, policy_version 1698578 (0.0005) [2023-12-27 17:24:46,529][20347] Updated weights for policy 0, policy_version 1698588 (0.0010) [2023-12-27 17:24:46,576][20347] Updated weights for policy 0, policy_version 1698598 (0.0010) [2023-12-27 17:24:46,703][20381] Updated weights for policy 1, policy_version 1697047 (0.0007) [2023-12-27 17:24:46,761][20381] Updated weights for policy 1, policy_version 1697057 (0.0008) [2023-12-27 17:24:46,820][20381] Updated weights for policy 1, policy_version 1697067 (0.0008) [2023-12-27 17:24:47,212][20347] Updated weights for policy 0, policy_version 1698608 (0.0010) [2023-12-27 17:24:47,267][20347] Updated weights for policy 0, policy_version 1698618 (0.0010) [2023-12-27 17:24:47,327][20347] Updated weights for policy 0, policy_version 1698628 (0.0010) [2023-12-27 17:24:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 869425152. Throughput: 0: 10170.2, 1: 10407.0. Samples: 869432820. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:24:47,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:24:47,465][20381] Updated weights for policy 1, policy_version 1697077 (0.0007) [2023-12-27 17:24:47,515][20381] Updated weights for policy 1, policy_version 1697087 (0.0005) [2023-12-27 17:24:47,571][20381] Updated weights for policy 1, policy_version 1697097 (0.0005) [2023-12-27 17:24:47,990][20347] Updated weights for policy 0, policy_version 1698638 (0.0010) [2023-12-27 17:24:48,038][20347] Updated weights for policy 0, policy_version 1698648 (0.0010) [2023-12-27 17:24:48,083][20347] Updated weights for policy 0, policy_version 1698658 (0.0009) [2023-12-27 17:24:48,121][20381] Updated weights for policy 1, policy_version 1697107 (0.0005) [2023-12-27 17:24:48,166][20381] Updated weights for policy 1, policy_version 1697117 (0.0008) [2023-12-27 17:24:48,214][20381] Updated weights for policy 1, policy_version 1697127 (0.0007) [2023-12-27 17:24:48,802][20347] Updated weights for policy 0, policy_version 1698668 (0.0009) [2023-12-27 17:24:48,860][20347] Updated weights for policy 0, policy_version 1698678 (0.0007) [2023-12-27 17:24:48,915][20347] Updated weights for policy 0, policy_version 1698688 (0.0007) [2023-12-27 17:24:48,940][20381] Updated weights for policy 1, policy_version 1697137 (0.0007) [2023-12-27 17:24:48,993][20381] Updated weights for policy 1, policy_version 1697147 (0.0011) [2023-12-27 17:24:49,052][20381] Updated weights for policy 1, policy_version 1697157 (0.0011) [2023-12-27 17:24:49,098][20381] Updated weights for policy 1, policy_version 1697167 (0.0011) [2023-12-27 17:24:49,597][20347] Updated weights for policy 0, policy_version 1698698 (0.0005) [2023-12-27 17:24:49,654][20347] Updated weights for policy 0, policy_version 1698708 (0.0005) [2023-12-27 17:24:49,722][20347] Updated weights for policy 0, policy_version 1698718 (0.0005) [2023-12-27 17:24:49,755][20381] Updated weights for policy 1, policy_version 1697177 (0.0011) [2023-12-27 17:24:49,780][20347] Updated weights for policy 0, policy_version 1698728 (0.0005) [2023-12-27 17:24:49,809][20381] Updated weights for policy 1, policy_version 1697187 (0.0011) [2023-12-27 17:24:49,870][20381] Updated weights for policy 1, policy_version 1697197 (0.0011) [2023-12-27 17:24:50,423][20347] Updated weights for policy 0, policy_version 1698738 (0.0007) [2023-12-27 17:24:50,478][20347] Updated weights for policy 0, policy_version 1698748 (0.0005) [2023-12-27 17:24:50,523][20347] Updated weights for policy 0, policy_version 1698758 (0.0006) [2023-12-27 17:24:50,578][20381] Updated weights for policy 1, policy_version 1697207 (0.0010) [2023-12-27 17:24:50,636][20381] Updated weights for policy 1, policy_version 1697217 (0.0010) [2023-12-27 17:24:50,682][20381] Updated weights for policy 1, policy_version 1697227 (0.0011) [2023-12-27 17:24:51,226][20347] Updated weights for policy 0, policy_version 1698768 (0.0008) [2023-12-27 17:24:51,291][20347] Updated weights for policy 0, policy_version 1698778 (0.0008) [2023-12-27 17:24:51,356][20347] Updated weights for policy 0, policy_version 1698788 (0.0008) [2023-12-27 17:24:51,446][20381] Updated weights for policy 1, policy_version 1697237 (0.0010) [2023-12-27 17:24:51,507][20381] Updated weights for policy 1, policy_version 1697247 (0.0008) [2023-12-27 17:24:51,568][20381] Updated weights for policy 1, policy_version 1697257 (0.0007) [2023-12-27 17:24:52,045][20347] Updated weights for policy 0, policy_version 1698798 (0.0007) [2023-12-27 17:24:52,106][20347] Updated weights for policy 0, policy_version 1698808 (0.0006) [2023-12-27 17:24:52,161][20347] Updated weights for policy 0, policy_version 1698818 (0.0007) [2023-12-27 17:24:52,264][20381] Updated weights for policy 1, policy_version 1697267 (0.0008) [2023-12-27 17:24:52,316][20381] Updated weights for policy 1, policy_version 1697277 (0.0011) [2023-12-27 17:24:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 869523456. Throughput: 0: 10146.3, 1: 10423.6. Samples: 869496004. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:24:52,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:24:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001698824_434978816.pth... [2023-12-27 17:24:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001697608_434667520.pth [2023-12-27 17:24:52,380][20381] Updated weights for policy 1, policy_version 1697287 (0.0010) [2023-12-27 17:24:52,437][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001697296_434552832.pth... [2023-12-27 17:24:52,442][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001696080_434241536.pth [2023-12-27 17:24:52,848][20347] Updated weights for policy 0, policy_version 1698828 (0.0007) [2023-12-27 17:24:52,912][20347] Updated weights for policy 0, policy_version 1698838 (0.0007) [2023-12-27 17:24:52,974][20347] Updated weights for policy 0, policy_version 1698848 (0.0006) [2023-12-27 17:24:53,110][20381] Updated weights for policy 1, policy_version 1697297 (0.0009) [2023-12-27 17:24:53,169][20381] Updated weights for policy 1, policy_version 1697307 (0.0008) [2023-12-27 17:24:53,223][20381] Updated weights for policy 1, policy_version 1697317 (0.0008) [2023-12-27 17:24:53,279][20381] Updated weights for policy 1, policy_version 1697327 (0.0008) [2023-12-27 17:24:53,540][20347] Updated weights for policy 0, policy_version 1698858 (0.0007) [2023-12-27 17:24:53,599][20347] Updated weights for policy 0, policy_version 1698868 (0.0008) [2023-12-27 17:24:53,647][20347] Updated weights for policy 0, policy_version 1698878 (0.0007) [2023-12-27 17:24:53,698][20347] Updated weights for policy 0, policy_version 1698888 (0.0005) [2023-12-27 17:24:54,025][20381] Updated weights for policy 1, policy_version 1697337 (0.0006) [2023-12-27 17:24:54,088][20381] Updated weights for policy 1, policy_version 1697347 (0.0005) [2023-12-27 17:24:54,148][20381] Updated weights for policy 1, policy_version 1697357 (0.0005) [2023-12-27 17:24:54,330][20347] Updated weights for policy 0, policy_version 1698898 (0.0008) [2023-12-27 17:24:54,389][20347] Updated weights for policy 0, policy_version 1698908 (0.0008) [2023-12-27 17:24:54,441][20347] Updated weights for policy 0, policy_version 1698918 (0.0006) [2023-12-27 17:24:54,720][20381] Updated weights for policy 1, policy_version 1697367 (0.0009) [2023-12-27 17:24:54,782][20381] Updated weights for policy 1, policy_version 1697377 (0.0006) [2023-12-27 17:24:54,835][20381] Updated weights for policy 1, policy_version 1697387 (0.0005) [2023-12-27 17:24:55,024][20347] Updated weights for policy 0, policy_version 1698928 (0.0006) [2023-12-27 17:24:55,076][20347] Updated weights for policy 0, policy_version 1698938 (0.0007) [2023-12-27 17:24:55,129][20347] Updated weights for policy 0, policy_version 1698948 (0.0009) [2023-12-27 17:24:55,477][20381] Updated weights for policy 1, policy_version 1697397 (0.0006) [2023-12-27 17:24:55,535][20381] Updated weights for policy 1, policy_version 1697407 (0.0008) [2023-12-27 17:24:55,590][20381] Updated weights for policy 1, policy_version 1697417 (0.0007) [2023-12-27 17:24:55,736][20347] Updated weights for policy 0, policy_version 1698958 (0.0006) [2023-12-27 17:24:55,779][20347] Updated weights for policy 0, policy_version 1698968 (0.0005) [2023-12-27 17:24:55,831][20347] Updated weights for policy 0, policy_version 1698978 (0.0005) [2023-12-27 17:24:56,285][20381] Updated weights for policy 1, policy_version 1697427 (0.0008) [2023-12-27 17:24:56,333][20381] Updated weights for policy 1, policy_version 1697437 (0.0009) [2023-12-27 17:24:56,379][20381] Updated weights for policy 1, policy_version 1697447 (0.0009) [2023-12-27 17:24:56,434][20347] Updated weights for policy 0, policy_version 1698988 (0.0006) [2023-12-27 17:24:56,487][20347] Updated weights for policy 0, policy_version 1698998 (0.0005) [2023-12-27 17:24:56,533][20347] Updated weights for policy 0, policy_version 1699008 (0.0005) [2023-12-27 17:24:56,997][20381] Updated weights for policy 1, policy_version 1697457 (0.0007) [2023-12-27 17:24:57,053][20381] Updated weights for policy 1, policy_version 1697467 (0.0006) [2023-12-27 17:24:57,104][20381] Updated weights for policy 1, policy_version 1697477 (0.0007) [2023-12-27 17:24:57,162][20381] Updated weights for policy 1, policy_version 1697487 (0.0005) [2023-12-27 17:24:57,185][20347] Updated weights for policy 0, policy_version 1699018 (0.0006) [2023-12-27 17:24:57,227][20347] Updated weights for policy 0, policy_version 1699028 (0.0008) [2023-12-27 17:24:57,272][20347] Updated weights for policy 0, policy_version 1699038 (0.0008) [2023-12-27 17:24:57,322][20347] Updated weights for policy 0, policy_version 1699048 (0.0007) [2023-12-27 17:24:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 869638144. Throughput: 0: 10248.5, 1: 10376.8. Samples: 869620404. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:24:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:24:57,746][20381] Updated weights for policy 1, policy_version 1697497 (0.0006) [2023-12-27 17:24:57,804][20381] Updated weights for policy 1, policy_version 1697507 (0.0005) [2023-12-27 17:24:57,866][20381] Updated weights for policy 1, policy_version 1697517 (0.0005) [2023-12-27 17:24:58,071][20347] Updated weights for policy 0, policy_version 1699058 (0.0005) [2023-12-27 17:24:58,119][20347] Updated weights for policy 0, policy_version 1699068 (0.0008) [2023-12-27 17:24:58,165][20347] Updated weights for policy 0, policy_version 1699078 (0.0008) [2023-12-27 17:24:58,454][20381] Updated weights for policy 1, policy_version 1697527 (0.0007) [2023-12-27 17:24:58,506][20381] Updated weights for policy 1, policy_version 1697537 (0.0008) [2023-12-27 17:24:58,555][20381] Updated weights for policy 1, policy_version 1697547 (0.0008) [2023-12-27 17:24:58,986][20347] Updated weights for policy 0, policy_version 1699088 (0.0006) [2023-12-27 17:24:59,040][20347] Updated weights for policy 0, policy_version 1699098 (0.0005) [2023-12-27 17:24:59,093][20347] Updated weights for policy 0, policy_version 1699108 (0.0006) [2023-12-27 17:24:59,345][20381] Updated weights for policy 1, policy_version 1697557 (0.0007) [2023-12-27 17:24:59,404][20381] Updated weights for policy 1, policy_version 1697567 (0.0007) [2023-12-27 17:24:59,456][20381] Updated weights for policy 1, policy_version 1697577 (0.0007) [2023-12-27 17:24:59,814][20347] Updated weights for policy 0, policy_version 1699118 (0.0008) [2023-12-27 17:24:59,871][20347] Updated weights for policy 0, policy_version 1699128 (0.0008) [2023-12-27 17:24:59,933][20347] Updated weights for policy 0, policy_version 1699138 (0.0008) [2023-12-27 17:25:00,169][20381] Updated weights for policy 1, policy_version 1697587 (0.0007) [2023-12-27 17:25:00,232][20381] Updated weights for policy 1, policy_version 1697597 (0.0007) [2023-12-27 17:25:00,292][20381] Updated weights for policy 1, policy_version 1697607 (0.0008) [2023-12-27 17:25:00,643][20347] Updated weights for policy 0, policy_version 1699148 (0.0009) [2023-12-27 17:25:00,704][20347] Updated weights for policy 0, policy_version 1699158 (0.0008) [2023-12-27 17:25:00,755][20347] Updated weights for policy 0, policy_version 1699168 (0.0009) [2023-12-27 17:25:00,901][20381] Updated weights for policy 1, policy_version 1697617 (0.0007) [2023-12-27 17:25:00,951][20381] Updated weights for policy 1, policy_version 1697627 (0.0010) [2023-12-27 17:25:00,995][20381] Updated weights for policy 1, policy_version 1697637 (0.0007) [2023-12-27 17:25:01,059][20381] Updated weights for policy 1, policy_version 1697647 (0.0007) [2023-12-27 17:25:01,519][20347] Updated weights for policy 0, policy_version 1699178 (0.0008) [2023-12-27 17:25:01,562][20291] KL-divergence is very high: 152.5118 [2023-12-27 17:25:01,581][20347] Updated weights for policy 0, policy_version 1699188 (0.0005) [2023-12-27 17:25:01,613][20291] KL-divergence is very high: 211.3740 [2023-12-27 17:25:01,647][20347] Updated weights for policy 0, policy_version 1699198 (0.0006) [2023-12-27 17:25:01,663][20291] KL-divergence is very high: 136.1026 [2023-12-27 17:25:01,702][20347] Updated weights for policy 0, policy_version 1699208 (0.0007) [2023-12-27 17:25:01,785][20381] Updated weights for policy 1, policy_version 1697657 (0.0007) [2023-12-27 17:25:01,844][20381] Updated weights for policy 1, policy_version 1697667 (0.0009) [2023-12-27 17:25:01,900][20381] Updated weights for policy 1, policy_version 1697677 (0.0008) [2023-12-27 17:25:02,357][20347] Updated weights for policy 0, policy_version 1699218 (0.0007) [2023-12-27 17:25:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 869728256. Throughput: 0: 10286.7, 1: 10322.5. Samples: 869742020. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:25:02,421][20347] Updated weights for policy 0, policy_version 1699228 (0.0007) [2023-12-27 17:25:02,490][20347] Updated weights for policy 0, policy_version 1699238 (0.0007) [2023-12-27 17:25:02,589][20381] Updated weights for policy 1, policy_version 1697687 (0.0007) [2023-12-27 17:25:02,642][20381] Updated weights for policy 1, policy_version 1697697 (0.0007) [2023-12-27 17:25:02,701][20381] Updated weights for policy 1, policy_version 1697707 (0.0008) [2023-12-27 17:25:03,188][20347] Updated weights for policy 0, policy_version 1699248 (0.0007) [2023-12-27 17:25:03,245][20347] Updated weights for policy 0, policy_version 1699258 (0.0008) [2023-12-27 17:25:03,298][20347] Updated weights for policy 0, policy_version 1699268 (0.0007) [2023-12-27 17:25:03,332][20381] Updated weights for policy 1, policy_version 1697717 (0.0007) [2023-12-27 17:25:03,392][20381] Updated weights for policy 1, policy_version 1697727 (0.0008) [2023-12-27 17:25:03,440][20381] Updated weights for policy 1, policy_version 1697737 (0.0007) [2023-12-27 17:25:03,954][20347] Updated weights for policy 0, policy_version 1699278 (0.0007) [2023-12-27 17:25:04,010][20347] Updated weights for policy 0, policy_version 1699288 (0.0008) [2023-12-27 17:25:04,068][20347] Updated weights for policy 0, policy_version 1699298 (0.0008) [2023-12-27 17:25:04,080][20381] Updated weights for policy 1, policy_version 1697747 (0.0008) [2023-12-27 17:25:04,136][20381] Updated weights for policy 1, policy_version 1697757 (0.0008) [2023-12-27 17:25:04,195][20381] Updated weights for policy 1, policy_version 1697767 (0.0008) [2023-12-27 17:25:04,766][20347] Updated weights for policy 0, policy_version 1699308 (0.0008) [2023-12-27 17:25:04,823][20347] Updated weights for policy 0, policy_version 1699318 (0.0007) [2023-12-27 17:25:04,885][20381] Updated weights for policy 1, policy_version 1697777 (0.0008) [2023-12-27 17:25:04,885][20347] Updated weights for policy 0, policy_version 1699328 (0.0008) [2023-12-27 17:25:04,939][20381] Updated weights for policy 1, policy_version 1697787 (0.0005) [2023-12-27 17:25:04,987][20381] Updated weights for policy 1, policy_version 1697797 (0.0005) [2023-12-27 17:25:05,040][20381] Updated weights for policy 1, policy_version 1697807 (0.0005) [2023-12-27 17:25:05,566][20381] Updated weights for policy 1, policy_version 1697817 (0.0005) [2023-12-27 17:25:05,611][20347] Updated weights for policy 0, policy_version 1699338 (0.0008) [2023-12-27 17:25:05,617][20381] Updated weights for policy 1, policy_version 1697827 (0.0007) [2023-12-27 17:25:05,656][20347] Updated weights for policy 0, policy_version 1699348 (0.0005) [2023-12-27 17:25:05,672][20381] Updated weights for policy 1, policy_version 1697837 (0.0005) [2023-12-27 17:25:05,702][20347] Updated weights for policy 0, policy_version 1699358 (0.0006) [2023-12-27 17:25:05,746][20347] Updated weights for policy 0, policy_version 1699368 (0.0008) [2023-12-27 17:25:06,265][20381] Updated weights for policy 1, policy_version 1697847 (0.0007) [2023-12-27 17:25:06,328][20381] Updated weights for policy 1, policy_version 1697857 (0.0008) [2023-12-27 17:25:06,397][20381] Updated weights for policy 1, policy_version 1697867 (0.0008) [2023-12-27 17:25:06,499][20347] Updated weights for policy 0, policy_version 1699378 (0.0009) [2023-12-27 17:25:06,563][20347] Updated weights for policy 0, policy_version 1699388 (0.0008) [2023-12-27 17:25:06,625][20347] Updated weights for policy 0, policy_version 1699398 (0.0010) [2023-12-27 17:25:07,004][20381] Updated weights for policy 1, policy_version 1697877 (0.0009) [2023-12-27 17:25:07,066][20381] Updated weights for policy 1, policy_version 1697887 (0.0011) [2023-12-27 17:25:07,125][20381] Updated weights for policy 1, policy_version 1697897 (0.0008) [2023-12-27 17:25:07,200][20347] Updated weights for policy 0, policy_version 1699408 (0.0008) [2023-12-27 17:25:07,259][20347] Updated weights for policy 0, policy_version 1699418 (0.0010) [2023-12-27 17:25:07,313][20347] Updated weights for policy 0, policy_version 1699428 (0.0008) [2023-12-27 17:25:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 869842944. Throughput: 0: 10285.0, 1: 10342.3. Samples: 869804524. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:07,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:25:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001697904_434708480.pth... [2023-12-27 17:25:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001699432_435134464.pth... [2023-12-27 17:25:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001696688_434397184.pth [2023-12-27 17:25:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001698216_434823168.pth [2023-12-27 17:25:07,767][20381] Updated weights for policy 1, policy_version 1697907 (0.0007) [2023-12-27 17:25:07,832][20381] Updated weights for policy 1, policy_version 1697917 (0.0007) [2023-12-27 17:25:07,882][20381] Updated weights for policy 1, policy_version 1697927 (0.0007) [2023-12-27 17:25:07,982][20347] Updated weights for policy 0, policy_version 1699438 (0.0007) [2023-12-27 17:25:08,042][20347] Updated weights for policy 0, policy_version 1699448 (0.0006) [2023-12-27 17:25:08,099][20347] Updated weights for policy 0, policy_version 1699458 (0.0008) [2023-12-27 17:25:08,529][20381] Updated weights for policy 1, policy_version 1697937 (0.0005) [2023-12-27 17:25:08,588][20381] Updated weights for policy 1, policy_version 1697947 (0.0005) [2023-12-27 17:25:08,638][20381] Updated weights for policy 1, policy_version 1697957 (0.0005) [2023-12-27 17:25:08,692][20381] Updated weights for policy 1, policy_version 1697967 (0.0006) [2023-12-27 17:25:08,819][20347] Updated weights for policy 0, policy_version 1699468 (0.0006) [2023-12-27 17:25:08,868][20347] Updated weights for policy 0, policy_version 1699478 (0.0007) [2023-12-27 17:25:08,931][20347] Updated weights for policy 0, policy_version 1699488 (0.0008) [2023-12-27 17:25:09,390][20381] Updated weights for policy 1, policy_version 1697977 (0.0009) [2023-12-27 17:25:09,462][20381] Updated weights for policy 1, policy_version 1697987 (0.0009) [2023-12-27 17:25:09,523][20381] Updated weights for policy 1, policy_version 1697997 (0.0010) [2023-12-27 17:25:09,656][20347] Updated weights for policy 0, policy_version 1699498 (0.0009) [2023-12-27 17:25:09,717][20347] Updated weights for policy 0, policy_version 1699508 (0.0007) [2023-12-27 17:25:09,771][20347] Updated weights for policy 0, policy_version 1699518 (0.0006) [2023-12-27 17:25:09,830][20347] Updated weights for policy 0, policy_version 1699528 (0.0007) [2023-12-27 17:25:10,260][20381] Updated weights for policy 1, policy_version 1698007 (0.0010) [2023-12-27 17:25:10,324][20381] Updated weights for policy 1, policy_version 1698017 (0.0011) [2023-12-27 17:25:10,386][20381] Updated weights for policy 1, policy_version 1698027 (0.0011) [2023-12-27 17:25:10,484][20347] Updated weights for policy 0, policy_version 1699538 (0.0007) [2023-12-27 17:25:10,545][20347] Updated weights for policy 0, policy_version 1699548 (0.0008) [2023-12-27 17:25:10,604][20347] Updated weights for policy 0, policy_version 1699558 (0.0008) [2023-12-27 17:25:11,029][20381] Updated weights for policy 1, policy_version 1698037 (0.0009) [2023-12-27 17:25:11,089][20381] Updated weights for policy 1, policy_version 1698047 (0.0007) [2023-12-27 17:25:11,166][20381] Updated weights for policy 1, policy_version 1698057 (0.0009) [2023-12-27 17:25:11,348][20347] Updated weights for policy 0, policy_version 1699568 (0.0006) [2023-12-27 17:25:11,413][20347] Updated weights for policy 0, policy_version 1699578 (0.0008) [2023-12-27 17:25:11,471][20347] Updated weights for policy 0, policy_version 1699588 (0.0007) [2023-12-27 17:25:11,918][20381] Updated weights for policy 1, policy_version 1698067 (0.0008) [2023-12-27 17:25:11,980][20381] Updated weights for policy 1, policy_version 1698077 (0.0007) [2023-12-27 17:25:12,037][20381] Updated weights for policy 1, policy_version 1698087 (0.0008) [2023-12-27 17:25:12,143][20347] Updated weights for policy 0, policy_version 1699598 (0.0007) [2023-12-27 17:25:12,210][20347] Updated weights for policy 0, policy_version 1699608 (0.0008) [2023-12-27 17:25:12,269][20347] Updated weights for policy 0, policy_version 1699618 (0.0008) [2023-12-27 17:25:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 869941248. Throughput: 0: 10322.5, 1: 10349.3. Samples: 869928292. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:25:12,697][20381] Updated weights for policy 1, policy_version 1698097 (0.0007) [2023-12-27 17:25:12,755][20381] Updated weights for policy 1, policy_version 1698107 (0.0009) [2023-12-27 17:25:12,809][20381] Updated weights for policy 1, policy_version 1698117 (0.0008) [2023-12-27 17:25:12,862][20381] Updated weights for policy 1, policy_version 1698127 (0.0008) [2023-12-27 17:25:12,924][20347] Updated weights for policy 0, policy_version 1699628 (0.0007) [2023-12-27 17:25:12,982][20347] Updated weights for policy 0, policy_version 1699638 (0.0008) [2023-12-27 17:25:13,042][20347] Updated weights for policy 0, policy_version 1699648 (0.0008) [2023-12-27 17:25:13,551][20381] Updated weights for policy 1, policy_version 1698137 (0.0008) [2023-12-27 17:25:13,619][20381] Updated weights for policy 1, policy_version 1698147 (0.0008) [2023-12-27 17:25:13,683][20381] Updated weights for policy 1, policy_version 1698157 (0.0008) [2023-12-27 17:25:13,771][20347] Updated weights for policy 0, policy_version 1699658 (0.0007) [2023-12-27 17:25:13,825][20347] Updated weights for policy 0, policy_version 1699668 (0.0005) [2023-12-27 17:25:13,894][20347] Updated weights for policy 0, policy_version 1699678 (0.0006) [2023-12-27 17:25:13,952][20347] Updated weights for policy 0, policy_version 1699688 (0.0005) [2023-12-27 17:25:14,294][20381] Updated weights for policy 1, policy_version 1698167 (0.0007) [2023-12-27 17:25:14,355][20381] Updated weights for policy 1, policy_version 1698177 (0.0008) [2023-12-27 17:25:14,409][20381] Updated weights for policy 1, policy_version 1698187 (0.0007) [2023-12-27 17:25:14,537][20347] Updated weights for policy 0, policy_version 1699698 (0.0010) [2023-12-27 17:25:14,589][20347] Updated weights for policy 0, policy_version 1699708 (0.0010) [2023-12-27 17:25:14,641][20347] Updated weights for policy 0, policy_version 1699718 (0.0010) [2023-12-27 17:25:15,065][20381] Updated weights for policy 1, policy_version 1698197 (0.0007) [2023-12-27 17:25:15,127][20381] Updated weights for policy 1, policy_version 1698207 (0.0005) [2023-12-27 17:25:15,190][20381] Updated weights for policy 1, policy_version 1698217 (0.0007) [2023-12-27 17:25:15,366][20347] Updated weights for policy 0, policy_version 1699728 (0.0010) [2023-12-27 17:25:15,426][20347] Updated weights for policy 0, policy_version 1699738 (0.0010) [2023-12-27 17:25:15,475][20347] Updated weights for policy 0, policy_version 1699748 (0.0010) [2023-12-27 17:25:15,819][20381] Updated weights for policy 1, policy_version 1698227 (0.0008) [2023-12-27 17:25:15,872][20381] Updated weights for policy 1, policy_version 1698237 (0.0006) [2023-12-27 17:25:15,927][20381] Updated weights for policy 1, policy_version 1698247 (0.0006) [2023-12-27 17:25:16,163][20347] Updated weights for policy 0, policy_version 1699758 (0.0010) [2023-12-27 17:25:16,221][20347] Updated weights for policy 0, policy_version 1699768 (0.0010) [2023-12-27 17:25:16,267][20347] Updated weights for policy 0, policy_version 1699778 (0.0010) [2023-12-27 17:25:16,577][20381] Updated weights for policy 1, policy_version 1698257 (0.0008) [2023-12-27 17:25:16,621][20381] Updated weights for policy 1, policy_version 1698267 (0.0008) [2023-12-27 17:25:16,666][20381] Updated weights for policy 1, policy_version 1698277 (0.0008) [2023-12-27 17:25:16,718][20381] Updated weights for policy 1, policy_version 1698287 (0.0008) [2023-12-27 17:25:16,865][20347] Updated weights for policy 0, policy_version 1699788 (0.0008) [2023-12-27 17:25:16,923][20347] Updated weights for policy 0, policy_version 1699798 (0.0009) [2023-12-27 17:25:16,976][20347] Updated weights for policy 0, policy_version 1699808 (0.0010) [2023-12-27 17:25:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 870039552. Throughput: 0: 10287.2, 1: 10389.0. Samples: 870052432. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:17,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:25:17,388][20381] Updated weights for policy 1, policy_version 1698297 (0.0007) [2023-12-27 17:25:17,450][20381] Updated weights for policy 1, policy_version 1698307 (0.0005) [2023-12-27 17:25:17,512][20381] Updated weights for policy 1, policy_version 1698317 (0.0005) [2023-12-27 17:25:17,627][20347] Updated weights for policy 0, policy_version 1699818 (0.0010) [2023-12-27 17:25:17,689][20347] Updated weights for policy 0, policy_version 1699828 (0.0010) [2023-12-27 17:25:17,752][20347] Updated weights for policy 0, policy_version 1699838 (0.0010) [2023-12-27 17:25:17,810][20347] Updated weights for policy 0, policy_version 1699848 (0.0010) [2023-12-27 17:25:18,085][20381] Updated weights for policy 1, policy_version 1698327 (0.0007) [2023-12-27 17:25:18,145][20381] Updated weights for policy 1, policy_version 1698337 (0.0007) [2023-12-27 17:25:18,203][20381] Updated weights for policy 1, policy_version 1698347 (0.0007) [2023-12-27 17:25:18,482][20347] Updated weights for policy 0, policy_version 1699858 (0.0010) [2023-12-27 17:25:18,543][20347] Updated weights for policy 0, policy_version 1699868 (0.0010) [2023-12-27 17:25:18,606][20347] Updated weights for policy 0, policy_version 1699878 (0.0010) [2023-12-27 17:25:18,917][20381] Updated weights for policy 1, policy_version 1698357 (0.0008) [2023-12-27 17:25:18,970][20381] Updated weights for policy 1, policy_version 1698367 (0.0008) [2023-12-27 17:25:19,021][20381] Updated weights for policy 1, policy_version 1698377 (0.0007) [2023-12-27 17:25:19,269][20347] Updated weights for policy 0, policy_version 1699888 (0.0008) [2023-12-27 17:25:19,320][20347] Updated weights for policy 0, policy_version 1699898 (0.0007) [2023-12-27 17:25:19,398][20347] Updated weights for policy 0, policy_version 1699908 (0.0008) [2023-12-27 17:25:19,779][20381] Updated weights for policy 1, policy_version 1698387 (0.0008) [2023-12-27 17:25:19,845][20381] Updated weights for policy 1, policy_version 1698397 (0.0008) [2023-12-27 17:25:19,908][20381] Updated weights for policy 1, policy_version 1698407 (0.0007) [2023-12-27 17:25:20,097][20347] Updated weights for policy 0, policy_version 1699918 (0.0007) [2023-12-27 17:25:20,160][20347] Updated weights for policy 0, policy_version 1699928 (0.0007) [2023-12-27 17:25:20,209][20347] Updated weights for policy 0, policy_version 1699938 (0.0006) [2023-12-27 17:25:20,595][20381] Updated weights for policy 1, policy_version 1698417 (0.0006) [2023-12-27 17:25:20,657][20381] Updated weights for policy 1, policy_version 1698427 (0.0007) [2023-12-27 17:25:20,718][20381] Updated weights for policy 1, policy_version 1698437 (0.0008) [2023-12-27 17:25:20,777][20381] Updated weights for policy 1, policy_version 1698447 (0.0008) [2023-12-27 17:25:20,893][20347] Updated weights for policy 0, policy_version 1699948 (0.0007) [2023-12-27 17:25:20,953][20347] Updated weights for policy 0, policy_version 1699958 (0.0007) [2023-12-27 17:25:21,012][20347] Updated weights for policy 0, policy_version 1699968 (0.0008) [2023-12-27 17:25:21,481][20381] Updated weights for policy 1, policy_version 1698457 (0.0007) [2023-12-27 17:25:21,537][20381] Updated weights for policy 1, policy_version 1698467 (0.0006) [2023-12-27 17:25:21,590][20381] Updated weights for policy 1, policy_version 1698477 (0.0006) [2023-12-27 17:25:21,763][20347] Updated weights for policy 0, policy_version 1699978 (0.0009) [2023-12-27 17:25:21,817][20347] Updated weights for policy 0, policy_version 1699988 (0.0008) [2023-12-27 17:25:21,869][20347] Updated weights for policy 0, policy_version 1699998 (0.0007) [2023-12-27 17:25:21,923][20347] Updated weights for policy 0, policy_version 1700008 (0.0008) [2023-12-27 17:25:22,281][20381] Updated weights for policy 1, policy_version 1698487 (0.0008) [2023-12-27 17:25:22,335][20381] Updated weights for policy 1, policy_version 1698497 (0.0010) [2023-12-27 17:25:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 870137856. Throughput: 0: 10238.0, 1: 10337.2. Samples: 870113400. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:22,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:25:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001700008_435281920.pth... [2023-12-27 17:25:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001698824_434978816.pth [2023-12-27 17:25:22,401][20381] Updated weights for policy 1, policy_version 1698507 (0.0008) [2023-12-27 17:25:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001698512_434864128.pth... [2023-12-27 17:25:22,434][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001697296_434552832.pth [2023-12-27 17:25:22,654][20347] Updated weights for policy 0, policy_version 1700018 (0.0005) [2023-12-27 17:25:22,719][20347] Updated weights for policy 0, policy_version 1700028 (0.0009) [2023-12-27 17:25:22,777][20347] Updated weights for policy 0, policy_version 1700038 (0.0010) [2023-12-27 17:25:23,075][20381] Updated weights for policy 1, policy_version 1698517 (0.0006) [2023-12-27 17:25:23,126][20381] Updated weights for policy 1, policy_version 1698527 (0.0005) [2023-12-27 17:25:23,172][20381] Updated weights for policy 1, policy_version 1698537 (0.0005) [2023-12-27 17:25:23,451][20347] Updated weights for policy 0, policy_version 1700048 (0.0010) [2023-12-27 17:25:23,510][20347] Updated weights for policy 0, policy_version 1700058 (0.0010) [2023-12-27 17:25:23,565][20347] Updated weights for policy 0, policy_version 1700068 (0.0010) [2023-12-27 17:25:23,784][20381] Updated weights for policy 1, policy_version 1698547 (0.0007) [2023-12-27 17:25:23,840][20381] Updated weights for policy 1, policy_version 1698557 (0.0005) [2023-12-27 17:25:23,893][20381] Updated weights for policy 1, policy_version 1698567 (0.0005) [2023-12-27 17:25:24,207][20347] Updated weights for policy 0, policy_version 1700078 (0.0008) [2023-12-27 17:25:24,267][20347] Updated weights for policy 0, policy_version 1700088 (0.0007) [2023-12-27 17:25:24,318][20347] Updated weights for policy 0, policy_version 1700098 (0.0007) [2023-12-27 17:25:24,585][20381] Updated weights for policy 1, policy_version 1698577 (0.0008) [2023-12-27 17:25:24,631][20381] Updated weights for policy 1, policy_version 1698587 (0.0008) [2023-12-27 17:25:24,683][20381] Updated weights for policy 1, policy_version 1698597 (0.0007) [2023-12-27 17:25:24,730][20381] Updated weights for policy 1, policy_version 1698607 (0.0007) [2023-12-27 17:25:24,959][20347] Updated weights for policy 0, policy_version 1700108 (0.0006) [2023-12-27 17:25:25,008][20347] Updated weights for policy 0, policy_version 1700118 (0.0005) [2023-12-27 17:25:25,057][20347] Updated weights for policy 0, policy_version 1700128 (0.0005) [2023-12-27 17:25:25,380][20381] Updated weights for policy 1, policy_version 1698617 (0.0005) [2023-12-27 17:25:25,443][20381] Updated weights for policy 1, policy_version 1698627 (0.0005) [2023-12-27 17:25:25,498][20381] Updated weights for policy 1, policy_version 1698637 (0.0006) [2023-12-27 17:25:25,718][20347] Updated weights for policy 0, policy_version 1700138 (0.0007) [2023-12-27 17:25:25,776][20347] Updated weights for policy 0, policy_version 1700148 (0.0008) [2023-12-27 17:25:25,830][20347] Updated weights for policy 0, policy_version 1700158 (0.0006) [2023-12-27 17:25:25,884][20347] Updated weights for policy 0, policy_version 1700168 (0.0007) [2023-12-27 17:25:26,131][20381] Updated weights for policy 1, policy_version 1698647 (0.0007) [2023-12-27 17:25:26,184][20381] Updated weights for policy 1, policy_version 1698657 (0.0008) [2023-12-27 17:25:26,238][20381] Updated weights for policy 1, policy_version 1698667 (0.0008) [2023-12-27 17:25:26,534][20347] Updated weights for policy 0, policy_version 1700178 (0.0009) [2023-12-27 17:25:26,583][20347] Updated weights for policy 0, policy_version 1700188 (0.0009) [2023-12-27 17:25:26,627][20347] Updated weights for policy 0, policy_version 1700198 (0.0009) [2023-12-27 17:25:26,852][20381] Updated weights for policy 1, policy_version 1698677 (0.0008) [2023-12-27 17:25:26,903][20381] Updated weights for policy 1, policy_version 1698687 (0.0008) [2023-12-27 17:25:26,951][20381] Updated weights for policy 1, policy_version 1698697 (0.0008) [2023-12-27 17:25:27,340][20347] Updated weights for policy 0, policy_version 1700208 (0.0007) [2023-12-27 17:25:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 870244352. Throughput: 0: 10211.7, 1: 10355.5. Samples: 870235660. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:27,363][19170] Avg episode reward: [(0, '960.850'), (1, '1000.000')] [2023-12-27 17:25:27,390][20347] Updated weights for policy 0, policy_version 1700218 (0.0005) [2023-12-27 17:25:27,439][20347] Updated weights for policy 0, policy_version 1700228 (0.0005) [2023-12-27 17:25:27,546][20381] Updated weights for policy 1, policy_version 1698707 (0.0006) [2023-12-27 17:25:27,596][20381] Updated weights for policy 1, policy_version 1698717 (0.0007) [2023-12-27 17:25:27,649][20381] Updated weights for policy 1, policy_version 1698727 (0.0007) [2023-12-27 17:25:27,983][20347] Updated weights for policy 0, policy_version 1700238 (0.0007) [2023-12-27 17:25:28,036][20347] Updated weights for policy 0, policy_version 1700248 (0.0007) [2023-12-27 17:25:28,093][20347] Updated weights for policy 0, policy_version 1700258 (0.0006) [2023-12-27 17:25:28,369][20381] Updated weights for policy 1, policy_version 1698737 (0.0008) [2023-12-27 17:25:28,425][20381] Updated weights for policy 1, policy_version 1698747 (0.0006) [2023-12-27 17:25:28,480][20381] Updated weights for policy 1, policy_version 1698757 (0.0007) [2023-12-27 17:25:28,543][20381] Updated weights for policy 1, policy_version 1698767 (0.0005) [2023-12-27 17:25:28,762][20347] Updated weights for policy 0, policy_version 1700268 (0.0007) [2023-12-27 17:25:28,807][20347] Updated weights for policy 0, policy_version 1700278 (0.0007) [2023-12-27 17:25:28,859][20347] Updated weights for policy 0, policy_version 1700288 (0.0008) [2023-12-27 17:25:29,191][20381] Updated weights for policy 1, policy_version 1698777 (0.0007) [2023-12-27 17:25:29,256][20381] Updated weights for policy 1, policy_version 1698787 (0.0006) [2023-12-27 17:25:29,307][20381] Updated weights for policy 1, policy_version 1698797 (0.0008) [2023-12-27 17:25:29,546][20347] Updated weights for policy 0, policy_version 1700298 (0.0007) [2023-12-27 17:25:29,601][20347] Updated weights for policy 0, policy_version 1700308 (0.0007) [2023-12-27 17:25:29,656][20347] Updated weights for policy 0, policy_version 1700318 (0.0007) [2023-12-27 17:25:29,711][20347] Updated weights for policy 0, policy_version 1700328 (0.0006) [2023-12-27 17:25:29,992][20381] Updated weights for policy 1, policy_version 1698807 (0.0008) [2023-12-27 17:25:30,055][20381] Updated weights for policy 1, policy_version 1698817 (0.0008) [2023-12-27 17:25:30,110][20381] Updated weights for policy 1, policy_version 1698827 (0.0008) [2023-12-27 17:25:30,399][20347] Updated weights for policy 0, policy_version 1700338 (0.0007) [2023-12-27 17:25:30,457][20347] Updated weights for policy 0, policy_version 1700348 (0.0007) [2023-12-27 17:25:30,509][20347] Updated weights for policy 0, policy_version 1700358 (0.0007) [2023-12-27 17:25:30,846][20381] Updated weights for policy 1, policy_version 1698837 (0.0007) [2023-12-27 17:25:30,892][20381] Updated weights for policy 1, policy_version 1698847 (0.0008) [2023-12-27 17:25:30,934][20381] Updated weights for policy 1, policy_version 1698857 (0.0008) [2023-12-27 17:25:31,174][20347] Updated weights for policy 0, policy_version 1700368 (0.0008) [2023-12-27 17:25:31,230][20347] Updated weights for policy 0, policy_version 1700378 (0.0006) [2023-12-27 17:25:31,289][20347] Updated weights for policy 0, policy_version 1700388 (0.0008) [2023-12-27 17:25:31,710][20381] Updated weights for policy 1, policy_version 1698867 (0.0007) [2023-12-27 17:25:31,770][20381] Updated weights for policy 1, policy_version 1698877 (0.0008) [2023-12-27 17:25:31,824][20381] Updated weights for policy 1, policy_version 1698887 (0.0007) [2023-12-27 17:25:31,996][20347] Updated weights for policy 0, policy_version 1700398 (0.0009) [2023-12-27 17:25:32,051][20347] Updated weights for policy 0, policy_version 1700408 (0.0008) [2023-12-27 17:25:32,113][20347] Updated weights for policy 0, policy_version 1700418 (0.0008) [2023-12-27 17:25:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 870350848. Throughput: 0: 10236.8, 1: 10361.7. Samples: 870359756. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:32,363][19170] Avg episode reward: [(0, '980.430'), (1, '1000.000')] [2023-12-27 17:25:32,542][20381] Updated weights for policy 1, policy_version 1698897 (0.0008) [2023-12-27 17:25:32,597][20381] Updated weights for policy 1, policy_version 1698907 (0.0008) [2023-12-27 17:25:32,650][20381] Updated weights for policy 1, policy_version 1698917 (0.0008) [2023-12-27 17:25:32,709][20381] Updated weights for policy 1, policy_version 1698927 (0.0008) [2023-12-27 17:25:32,757][20347] Updated weights for policy 0, policy_version 1700428 (0.0007) [2023-12-27 17:25:32,822][20347] Updated weights for policy 0, policy_version 1700438 (0.0005) [2023-12-27 17:25:32,879][20347] Updated weights for policy 0, policy_version 1700448 (0.0006) [2023-12-27 17:25:33,429][20347] Updated weights for policy 0, policy_version 1700458 (0.0007) [2023-12-27 17:25:33,449][20381] Updated weights for policy 1, policy_version 1698937 (0.0007) [2023-12-27 17:25:33,485][20347] Updated weights for policy 0, policy_version 1700468 (0.0006) [2023-12-27 17:25:33,502][20381] Updated weights for policy 1, policy_version 1698947 (0.0007) [2023-12-27 17:25:33,549][20347] Updated weights for policy 0, policy_version 1700478 (0.0006) [2023-12-27 17:25:33,551][20381] Updated weights for policy 1, policy_version 1698957 (0.0007) [2023-12-27 17:25:33,602][20347] Updated weights for policy 0, policy_version 1700488 (0.0007) [2023-12-27 17:25:34,221][20347] Updated weights for policy 0, policy_version 1700498 (0.0006) [2023-12-27 17:25:34,278][20347] Updated weights for policy 0, policy_version 1700508 (0.0007) [2023-12-27 17:25:34,291][20381] Updated weights for policy 1, policy_version 1698967 (0.0007) [2023-12-27 17:25:34,333][20347] Updated weights for policy 0, policy_version 1700518 (0.0007) [2023-12-27 17:25:34,344][20381] Updated weights for policy 1, policy_version 1698977 (0.0008) [2023-12-27 17:25:34,399][20381] Updated weights for policy 1, policy_version 1698987 (0.0008) [2023-12-27 17:25:34,952][20347] Updated weights for policy 0, policy_version 1700528 (0.0006) [2023-12-27 17:25:35,001][20347] Updated weights for policy 0, policy_version 1700538 (0.0007) [2023-12-27 17:25:35,055][20347] Updated weights for policy 0, policy_version 1700548 (0.0008) [2023-12-27 17:25:35,141][20381] Updated weights for policy 1, policy_version 1698997 (0.0008) [2023-12-27 17:25:35,202][20381] Updated weights for policy 1, policy_version 1699007 (0.0008) [2023-12-27 17:25:35,264][20381] Updated weights for policy 1, policy_version 1699017 (0.0008) [2023-12-27 17:25:35,630][20347] Updated weights for policy 0, policy_version 1700558 (0.0006) [2023-12-27 17:25:35,693][20347] Updated weights for policy 0, policy_version 1700568 (0.0009) [2023-12-27 17:25:35,748][20347] Updated weights for policy 0, policy_version 1700578 (0.0007) [2023-12-27 17:25:35,808][20381] Updated weights for policy 1, policy_version 1699027 (0.0008) [2023-12-27 17:25:35,861][20381] Updated weights for policy 1, policy_version 1699037 (0.0009) [2023-12-27 17:25:35,922][20381] Updated weights for policy 1, policy_version 1699047 (0.0007) [2023-12-27 17:25:36,338][20347] Updated weights for policy 0, policy_version 1700588 (0.0007) [2023-12-27 17:25:36,400][20347] Updated weights for policy 0, policy_version 1700598 (0.0008) [2023-12-27 17:25:36,462][20347] Updated weights for policy 0, policy_version 1700608 (0.0011) [2023-12-27 17:25:36,694][20381] Updated weights for policy 1, policy_version 1699057 (0.0008) [2023-12-27 17:25:36,743][20381] Updated weights for policy 1, policy_version 1699067 (0.0007) [2023-12-27 17:25:36,796][20381] Updated weights for policy 1, policy_version 1699077 (0.0007) [2023-12-27 17:25:36,852][20381] Updated weights for policy 1, policy_version 1699087 (0.0008) [2023-12-27 17:25:37,106][20347] Updated weights for policy 0, policy_version 1700618 (0.0008) [2023-12-27 17:25:37,161][20347] Updated weights for policy 0, policy_version 1700628 (0.0007) [2023-12-27 17:25:37,215][20347] Updated weights for policy 0, policy_version 1700638 (0.0008) [2023-12-27 17:25:37,266][20347] Updated weights for policy 0, policy_version 1700648 (0.0007) [2023-12-27 17:25:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 870457344. Throughput: 0: 10291.7, 1: 10279.9. Samples: 870421724. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:37,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:25:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001699088_435011584.pth... [2023-12-27 17:25:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001700648_435445760.pth... [2023-12-27 17:25:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001699432_435134464.pth [2023-12-27 17:25:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001697904_434708480.pth [2023-12-27 17:25:37,573][20381] Updated weights for policy 1, policy_version 1699097 (0.0008) [2023-12-27 17:25:37,626][20381] Updated weights for policy 1, policy_version 1699107 (0.0008) [2023-12-27 17:25:37,691][20381] Updated weights for policy 1, policy_version 1699117 (0.0008) [2023-12-27 17:25:37,944][20347] Updated weights for policy 0, policy_version 1700658 (0.0006) [2023-12-27 17:25:38,004][20347] Updated weights for policy 0, policy_version 1700668 (0.0008) [2023-12-27 17:25:38,052][20347] Updated weights for policy 0, policy_version 1700678 (0.0008) [2023-12-27 17:25:38,367][20381] Updated weights for policy 1, policy_version 1699127 (0.0007) [2023-12-27 17:25:38,425][20381] Updated weights for policy 1, policy_version 1699137 (0.0006) [2023-12-27 17:25:38,486][20381] Updated weights for policy 1, policy_version 1699147 (0.0006) [2023-12-27 17:25:38,753][20347] Updated weights for policy 0, policy_version 1700688 (0.0008) [2023-12-27 17:25:38,824][20347] Updated weights for policy 0, policy_version 1700698 (0.0010) [2023-12-27 17:25:38,881][20347] Updated weights for policy 0, policy_version 1700708 (0.0007) [2023-12-27 17:25:39,085][20381] Updated weights for policy 1, policy_version 1699157 (0.0008) [2023-12-27 17:25:39,140][20381] Updated weights for policy 1, policy_version 1699167 (0.0011) [2023-12-27 17:25:39,188][20381] Updated weights for policy 1, policy_version 1699177 (0.0010) [2023-12-27 17:25:39,628][20347] Updated weights for policy 0, policy_version 1700718 (0.0007) [2023-12-27 17:25:39,685][20347] Updated weights for policy 0, policy_version 1700728 (0.0008) [2023-12-27 17:25:39,750][20347] Updated weights for policy 0, policy_version 1700738 (0.0008) [2023-12-27 17:25:39,907][20381] Updated weights for policy 1, policy_version 1699187 (0.0012) [2023-12-27 17:25:39,968][20381] Updated weights for policy 1, policy_version 1699197 (0.0008) [2023-12-27 17:25:40,035][20381] Updated weights for policy 1, policy_version 1699207 (0.0007) [2023-12-27 17:25:40,488][20347] Updated weights for policy 0, policy_version 1700748 (0.0008) [2023-12-27 17:25:40,548][20347] Updated weights for policy 0, policy_version 1700758 (0.0006) [2023-12-27 17:25:40,604][20347] Updated weights for policy 0, policy_version 1700768 (0.0006) [2023-12-27 17:25:40,747][20381] Updated weights for policy 1, policy_version 1699217 (0.0010) [2023-12-27 17:25:40,804][20381] Updated weights for policy 1, policy_version 1699227 (0.0006) [2023-12-27 17:25:40,867][20381] Updated weights for policy 1, policy_version 1699237 (0.0005) [2023-12-27 17:25:40,934][20381] Updated weights for policy 1, policy_version 1699247 (0.0005) [2023-12-27 17:25:41,284][20347] Updated weights for policy 0, policy_version 1700778 (0.0006) [2023-12-27 17:25:41,349][20347] Updated weights for policy 0, policy_version 1700788 (0.0010) [2023-12-27 17:25:41,413][20347] Updated weights for policy 0, policy_version 1700798 (0.0009) [2023-12-27 17:25:41,467][20347] Updated weights for policy 0, policy_version 1700808 (0.0011) [2023-12-27 17:25:41,660][20381] Updated weights for policy 1, policy_version 1699257 (0.0007) [2023-12-27 17:25:41,719][20381] Updated weights for policy 1, policy_version 1699267 (0.0008) [2023-12-27 17:25:41,780][20381] Updated weights for policy 1, policy_version 1699277 (0.0009) [2023-12-27 17:25:42,130][20347] Updated weights for policy 0, policy_version 1700818 (0.0008) [2023-12-27 17:25:42,180][20347] Updated weights for policy 0, policy_version 1700828 (0.0007) [2023-12-27 17:25:42,233][20347] Updated weights for policy 0, policy_version 1700838 (0.0008) [2023-12-27 17:25:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 870555648. Throughput: 0: 10251.7, 1: 10300.7. Samples: 870545264. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:42,363][19170] Avg episode reward: [(0, '970.770'), (1, '1000.000')] [2023-12-27 17:25:42,542][20381] Updated weights for policy 1, policy_version 1699287 (0.0008) [2023-12-27 17:25:42,594][20381] Updated weights for policy 1, policy_version 1699297 (0.0006) [2023-12-27 17:25:42,644][20381] Updated weights for policy 1, policy_version 1699307 (0.0005) [2023-12-27 17:25:42,928][20347] Updated weights for policy 0, policy_version 1700848 (0.0006) [2023-12-27 17:25:42,978][20347] Updated weights for policy 0, policy_version 1700858 (0.0005) [2023-12-27 17:25:43,027][20347] Updated weights for policy 0, policy_version 1700868 (0.0005) [2023-12-27 17:25:43,279][20381] Updated weights for policy 1, policy_version 1699317 (0.0005) [2023-12-27 17:25:43,336][20381] Updated weights for policy 1, policy_version 1699327 (0.0005) [2023-12-27 17:25:43,387][20381] Updated weights for policy 1, policy_version 1699337 (0.0005) [2023-12-27 17:25:43,600][20347] Updated weights for policy 0, policy_version 1700878 (0.0005) [2023-12-27 17:25:43,665][20347] Updated weights for policy 0, policy_version 1700888 (0.0008) [2023-12-27 17:25:43,729][20347] Updated weights for policy 0, policy_version 1700898 (0.0006) [2023-12-27 17:25:43,958][20381] Updated weights for policy 1, policy_version 1699347 (0.0006) [2023-12-27 17:25:44,011][20381] Updated weights for policy 1, policy_version 1699357 (0.0011) [2023-12-27 17:25:44,064][20381] Updated weights for policy 1, policy_version 1699367 (0.0011) [2023-12-27 17:25:44,365][20347] Updated weights for policy 0, policy_version 1700908 (0.0007) [2023-12-27 17:25:44,423][20347] Updated weights for policy 0, policy_version 1700918 (0.0008) [2023-12-27 17:25:44,472][20347] Updated weights for policy 0, policy_version 1700928 (0.0007) [2023-12-27 17:25:44,739][20381] Updated weights for policy 1, policy_version 1699377 (0.0010) [2023-12-27 17:25:44,803][20381] Updated weights for policy 1, policy_version 1699387 (0.0007) [2023-12-27 17:25:44,863][20381] Updated weights for policy 1, policy_version 1699397 (0.0008) [2023-12-27 17:25:44,926][20381] Updated weights for policy 1, policy_version 1699407 (0.0008) [2023-12-27 17:25:45,202][20347] Updated weights for policy 0, policy_version 1700938 (0.0007) [2023-12-27 17:25:45,265][20347] Updated weights for policy 0, policy_version 1700948 (0.0007) [2023-12-27 17:25:45,333][20347] Updated weights for policy 0, policy_version 1700958 (0.0008) [2023-12-27 17:25:45,394][20347] Updated weights for policy 0, policy_version 1700968 (0.0008) [2023-12-27 17:25:45,568][20381] Updated weights for policy 1, policy_version 1699417 (0.0008) [2023-12-27 17:25:45,614][20381] Updated weights for policy 1, policy_version 1699427 (0.0008) [2023-12-27 17:25:45,666][20381] Updated weights for policy 1, policy_version 1699437 (0.0007) [2023-12-27 17:25:46,114][20347] Updated weights for policy 0, policy_version 1700978 (0.0007) [2023-12-27 17:25:46,173][20347] Updated weights for policy 0, policy_version 1700988 (0.0007) [2023-12-27 17:25:46,232][20347] Updated weights for policy 0, policy_version 1700998 (0.0008) [2023-12-27 17:25:46,344][20381] Updated weights for policy 1, policy_version 1699447 (0.0007) [2023-12-27 17:25:46,398][20381] Updated weights for policy 1, policy_version 1699457 (0.0007) [2023-12-27 17:25:46,444][20381] Updated weights for policy 1, policy_version 1699467 (0.0007) [2023-12-27 17:25:46,959][20347] Updated weights for policy 0, policy_version 1701008 (0.0006) [2023-12-27 17:25:47,013][20347] Updated weights for policy 0, policy_version 1701018 (0.0007) [2023-12-27 17:25:47,034][20381] Updated weights for policy 1, policy_version 1699477 (0.0007) [2023-12-27 17:25:47,066][20347] Updated weights for policy 0, policy_version 1701028 (0.0007) [2023-12-27 17:25:47,087][20381] Updated weights for policy 1, policy_version 1699487 (0.0006) [2023-12-27 17:25:47,139][20381] Updated weights for policy 1, policy_version 1699497 (0.0008) [2023-12-27 17:25:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.4, 300 sec: 20632.7). Total num frames: 870662144. Throughput: 0: 10272.7, 1: 10317.1. Samples: 870668564. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:47,364][19170] Avg episode reward: [(0, '980.550'), (1, '1000.000')] [2023-12-27 17:25:47,760][20347] Updated weights for policy 0, policy_version 1701038 (0.0007) [2023-12-27 17:25:47,814][20347] Updated weights for policy 0, policy_version 1701048 (0.0008) [2023-12-27 17:25:47,820][20381] Updated weights for policy 1, policy_version 1699507 (0.0007) [2023-12-27 17:25:47,872][20347] Updated weights for policy 0, policy_version 1701058 (0.0006) [2023-12-27 17:25:47,877][20381] Updated weights for policy 1, policy_version 1699517 (0.0006) [2023-12-27 17:25:47,928][20381] Updated weights for policy 1, policy_version 1699527 (0.0005) [2023-12-27 17:25:48,573][20381] Updated weights for policy 1, policy_version 1699537 (0.0005) [2023-12-27 17:25:48,577][20347] Updated weights for policy 0, policy_version 1701068 (0.0007) [2023-12-27 17:25:48,635][20347] Updated weights for policy 0, policy_version 1701078 (0.0005) [2023-12-27 17:25:48,636][20381] Updated weights for policy 1, policy_version 1699547 (0.0007) [2023-12-27 17:25:48,692][20347] Updated weights for policy 0, policy_version 1701088 (0.0007) [2023-12-27 17:25:48,694][20381] Updated weights for policy 1, policy_version 1699557 (0.0006) [2023-12-27 17:25:48,747][20381] Updated weights for policy 1, policy_version 1699567 (0.0005) [2023-12-27 17:25:49,391][20381] Updated weights for policy 1, policy_version 1699577 (0.0008) [2023-12-27 17:25:49,411][20347] Updated weights for policy 0, policy_version 1701098 (0.0007) [2023-12-27 17:25:49,454][20381] Updated weights for policy 1, policy_version 1699587 (0.0007) [2023-12-27 17:25:49,470][20347] Updated weights for policy 0, policy_version 1701108 (0.0006) [2023-12-27 17:25:49,513][20381] Updated weights for policy 1, policy_version 1699597 (0.0006) [2023-12-27 17:25:49,526][20347] Updated weights for policy 0, policy_version 1701118 (0.0006) [2023-12-27 17:25:49,585][20347] Updated weights for policy 0, policy_version 1701128 (0.0008) [2023-12-27 17:25:50,231][20381] Updated weights for policy 1, policy_version 1699607 (0.0007) [2023-12-27 17:25:50,297][20381] Updated weights for policy 1, policy_version 1699617 (0.0007) [2023-12-27 17:25:50,314][20347] Updated weights for policy 0, policy_version 1701138 (0.0006) [2023-12-27 17:25:50,357][20381] Updated weights for policy 1, policy_version 1699627 (0.0007) [2023-12-27 17:25:50,377][20347] Updated weights for policy 0, policy_version 1701148 (0.0007) [2023-12-27 17:25:50,447][20347] Updated weights for policy 0, policy_version 1701158 (0.0007) [2023-12-27 17:25:51,004][20381] Updated weights for policy 1, policy_version 1699637 (0.0006) [2023-12-27 17:25:51,072][20381] Updated weights for policy 1, policy_version 1699647 (0.0007) [2023-12-27 17:25:51,138][20381] Updated weights for policy 1, policy_version 1699657 (0.0007) [2023-12-27 17:25:51,179][20347] Updated weights for policy 0, policy_version 1701168 (0.0008) [2023-12-27 17:25:51,247][20347] Updated weights for policy 0, policy_version 1701178 (0.0008) [2023-12-27 17:25:51,313][20347] Updated weights for policy 0, policy_version 1701188 (0.0008) [2023-12-27 17:25:51,829][20381] Updated weights for policy 1, policy_version 1699667 (0.0007) [2023-12-27 17:25:51,894][20381] Updated weights for policy 1, policy_version 1699677 (0.0008) [2023-12-27 17:25:51,951][20347] Updated weights for policy 0, policy_version 1701198 (0.0008) [2023-12-27 17:25:51,954][20381] Updated weights for policy 1, policy_version 1699687 (0.0007) [2023-12-27 17:25:52,003][20347] Updated weights for policy 0, policy_version 1701208 (0.0008) [2023-12-27 17:25:52,062][20347] Updated weights for policy 0, policy_version 1701218 (0.0008) [2023-12-27 17:25:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 870760448. Throughput: 0: 10253.0, 1: 10298.6. Samples: 870729348. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:52,363][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 17:25:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001699696_435167232.pth... [2023-12-27 17:25:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001701224_435593216.pth... [2023-12-27 17:25:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001698512_434864128.pth [2023-12-27 17:25:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001700008_435281920.pth [2023-12-27 17:25:52,652][20381] Updated weights for policy 1, policy_version 1699697 (0.0006) [2023-12-27 17:25:52,718][20381] Updated weights for policy 1, policy_version 1699707 (0.0008) [2023-12-27 17:25:52,729][20347] Updated weights for policy 0, policy_version 1701228 (0.0009) [2023-12-27 17:25:52,769][20381] Updated weights for policy 1, policy_version 1699717 (0.0007) [2023-12-27 17:25:52,789][20347] Updated weights for policy 0, policy_version 1701238 (0.0011) [2023-12-27 17:25:52,828][20381] Updated weights for policy 1, policy_version 1699727 (0.0008) [2023-12-27 17:25:52,850][20347] Updated weights for policy 0, policy_version 1701248 (0.0006) [2023-12-27 17:25:53,438][20381] Updated weights for policy 1, policy_version 1699737 (0.0007) [2023-12-27 17:25:53,472][20347] Updated weights for policy 0, policy_version 1701258 (0.0009) [2023-12-27 17:25:53,499][20381] Updated weights for policy 1, policy_version 1699747 (0.0008) [2023-12-27 17:25:53,527][20347] Updated weights for policy 0, policy_version 1701268 (0.0008) [2023-12-27 17:25:53,546][20381] Updated weights for policy 1, policy_version 1699757 (0.0008) [2023-12-27 17:25:53,582][20347] Updated weights for policy 0, policy_version 1701278 (0.0007) [2023-12-27 17:25:53,636][20347] Updated weights for policy 0, policy_version 1701288 (0.0007) [2023-12-27 17:25:54,220][20381] Updated weights for policy 1, policy_version 1699767 (0.0006) [2023-12-27 17:25:54,227][20347] Updated weights for policy 0, policy_version 1701298 (0.0008) [2023-12-27 17:25:54,271][20381] Updated weights for policy 1, policy_version 1699777 (0.0005) [2023-12-27 17:25:54,277][20347] Updated weights for policy 0, policy_version 1701308 (0.0008) [2023-12-27 17:25:54,327][20381] Updated weights for policy 1, policy_version 1699787 (0.0006) [2023-12-27 17:25:54,335][20347] Updated weights for policy 0, policy_version 1701318 (0.0006) [2023-12-27 17:25:54,909][20381] Updated weights for policy 1, policy_version 1699797 (0.0006) [2023-12-27 17:25:54,960][20381] Updated weights for policy 1, policy_version 1699807 (0.0007) [2023-12-27 17:25:55,021][20347] Updated weights for policy 0, policy_version 1701328 (0.0007) [2023-12-27 17:25:55,022][20381] Updated weights for policy 1, policy_version 1699817 (0.0008) [2023-12-27 17:25:55,071][20347] Updated weights for policy 0, policy_version 1701338 (0.0006) [2023-12-27 17:25:55,118][20347] Updated weights for policy 0, policy_version 1701348 (0.0008) [2023-12-27 17:25:55,658][20381] Updated weights for policy 1, policy_version 1699827 (0.0006) [2023-12-27 17:25:55,714][20381] Updated weights for policy 1, policy_version 1699837 (0.0008) [2023-12-27 17:25:55,770][20381] Updated weights for policy 1, policy_version 1699847 (0.0008) [2023-12-27 17:25:55,848][20347] Updated weights for policy 0, policy_version 1701358 (0.0008) [2023-12-27 17:25:55,896][20347] Updated weights for policy 0, policy_version 1701368 (0.0007) [2023-12-27 17:25:55,953][20347] Updated weights for policy 0, policy_version 1701378 (0.0007) [2023-12-27 17:25:56,381][20381] Updated weights for policy 1, policy_version 1699857 (0.0008) [2023-12-27 17:25:56,435][20381] Updated weights for policy 1, policy_version 1699867 (0.0010) [2023-12-27 17:25:56,488][20381] Updated weights for policy 1, policy_version 1699877 (0.0007) [2023-12-27 17:25:56,544][20381] Updated weights for policy 1, policy_version 1699887 (0.0005) [2023-12-27 17:25:56,575][20347] Updated weights for policy 0, policy_version 1701388 (0.0008) [2023-12-27 17:25:56,630][20347] Updated weights for policy 0, policy_version 1701398 (0.0006) [2023-12-27 17:25:56,688][20347] Updated weights for policy 0, policy_version 1701408 (0.0007) [2023-12-27 17:25:57,113][20381] Updated weights for policy 1, policy_version 1699897 (0.0005) [2023-12-27 17:25:57,170][20381] Updated weights for policy 1, policy_version 1699907 (0.0005) [2023-12-27 17:25:57,225][20381] Updated weights for policy 1, policy_version 1699917 (0.0005) [2023-12-27 17:25:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 870866944. Throughput: 0: 10294.3, 1: 10285.1. Samples: 870854364. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:25:57,363][19170] Avg episode reward: [(0, '970.670'), (1, '1000.000')] [2023-12-27 17:25:57,382][20347] Updated weights for policy 0, policy_version 1701418 (0.0008) [2023-12-27 17:25:57,428][20347] Updated weights for policy 0, policy_version 1701428 (0.0005) [2023-12-27 17:25:57,476][20347] Updated weights for policy 0, policy_version 1701438 (0.0005) [2023-12-27 17:25:57,519][20347] Updated weights for policy 0, policy_version 1701448 (0.0005) [2023-12-27 17:25:57,782][20381] Updated weights for policy 1, policy_version 1699927 (0.0005) [2023-12-27 17:25:57,833][20381] Updated weights for policy 1, policy_version 1699937 (0.0006) [2023-12-27 17:25:57,884][20381] Updated weights for policy 1, policy_version 1699947 (0.0007) [2023-12-27 17:25:58,106][20347] Updated weights for policy 0, policy_version 1701458 (0.0008) [2023-12-27 17:25:58,151][20347] Updated weights for policy 0, policy_version 1701468 (0.0008) [2023-12-27 17:25:58,201][20347] Updated weights for policy 0, policy_version 1701478 (0.0007) [2023-12-27 17:25:58,528][20381] Updated weights for policy 1, policy_version 1699957 (0.0007) [2023-12-27 17:25:58,590][20381] Updated weights for policy 1, policy_version 1699967 (0.0010) [2023-12-27 17:25:58,653][20381] Updated weights for policy 1, policy_version 1699977 (0.0008) [2023-12-27 17:25:59,043][20347] Updated weights for policy 0, policy_version 1701488 (0.0010) [2023-12-27 17:25:59,104][20347] Updated weights for policy 0, policy_version 1701498 (0.0010) [2023-12-27 17:25:59,154][20347] Updated weights for policy 0, policy_version 1701508 (0.0010) [2023-12-27 17:25:59,427][20381] Updated weights for policy 1, policy_version 1699987 (0.0009) [2023-12-27 17:25:59,490][20381] Updated weights for policy 1, policy_version 1699997 (0.0007) [2023-12-27 17:25:59,551][20381] Updated weights for policy 1, policy_version 1700007 (0.0005) [2023-12-27 17:25:59,828][20347] Updated weights for policy 0, policy_version 1701518 (0.0009) [2023-12-27 17:25:59,891][20347] Updated weights for policy 0, policy_version 1701528 (0.0008) [2023-12-27 17:25:59,956][20347] Updated weights for policy 0, policy_version 1701538 (0.0008) [2023-12-27 17:26:00,181][20381] Updated weights for policy 1, policy_version 1700017 (0.0005) [2023-12-27 17:26:00,232][20381] Updated weights for policy 1, policy_version 1700027 (0.0008) [2023-12-27 17:26:00,277][20381] Updated weights for policy 1, policy_version 1700037 (0.0008) [2023-12-27 17:26:00,326][20381] Updated weights for policy 1, policy_version 1700047 (0.0009) [2023-12-27 17:26:00,584][20347] Updated weights for policy 0, policy_version 1701548 (0.0007) [2023-12-27 17:26:00,627][20347] Updated weights for policy 0, policy_version 1701558 (0.0005) [2023-12-27 17:26:00,675][20347] Updated weights for policy 0, policy_version 1701568 (0.0005) [2023-12-27 17:26:01,067][20381] Updated weights for policy 1, policy_version 1700057 (0.0008) [2023-12-27 17:26:01,136][20381] Updated weights for policy 1, policy_version 1700067 (0.0008) [2023-12-27 17:26:01,200][20381] Updated weights for policy 1, policy_version 1700077 (0.0007) [2023-12-27 17:26:01,363][20347] Updated weights for policy 0, policy_version 1701578 (0.0006) [2023-12-27 17:26:01,425][20347] Updated weights for policy 0, policy_version 1701588 (0.0007) [2023-12-27 17:26:01,479][20347] Updated weights for policy 0, policy_version 1701598 (0.0009) [2023-12-27 17:26:01,534][20347] Updated weights for policy 0, policy_version 1701608 (0.0007) [2023-12-27 17:26:01,925][20381] Updated weights for policy 1, policy_version 1700087 (0.0007) [2023-12-27 17:26:01,987][20381] Updated weights for policy 1, policy_version 1700097 (0.0008) [2023-12-27 17:26:02,050][20381] Updated weights for policy 1, policy_version 1700107 (0.0007) [2023-12-27 17:26:02,292][20347] Updated weights for policy 0, policy_version 1701618 (0.0008) [2023-12-27 17:26:02,353][20347] Updated weights for policy 0, policy_version 1701628 (0.0007) [2023-12-27 17:26:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 870965248. Throughput: 0: 10311.3, 1: 10283.4. Samples: 870979192. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:02,363][19170] Avg episode reward: [(0, '970.710'), (1, '1000.000')] [2023-12-27 17:26:02,415][20347] Updated weights for policy 0, policy_version 1701638 (0.0007) [2023-12-27 17:26:02,695][20381] Updated weights for policy 1, policy_version 1700117 (0.0007) [2023-12-27 17:26:02,751][20381] Updated weights for policy 1, policy_version 1700127 (0.0007) [2023-12-27 17:26:02,807][20381] Updated weights for policy 1, policy_version 1700137 (0.0008) [2023-12-27 17:26:03,089][20347] Updated weights for policy 0, policy_version 1701648 (0.0008) [2023-12-27 17:26:03,145][20347] Updated weights for policy 0, policy_version 1701658 (0.0008) [2023-12-27 17:26:03,193][20347] Updated weights for policy 0, policy_version 1701668 (0.0008) [2023-12-27 17:26:03,452][20381] Updated weights for policy 1, policy_version 1700147 (0.0007) [2023-12-27 17:26:03,497][20381] Updated weights for policy 1, policy_version 1700157 (0.0008) [2023-12-27 17:26:03,547][20381] Updated weights for policy 1, policy_version 1700167 (0.0008) [2023-12-27 17:26:03,925][20347] Updated weights for policy 0, policy_version 1701678 (0.0008) [2023-12-27 17:26:03,979][20347] Updated weights for policy 0, policy_version 1701688 (0.0006) [2023-12-27 17:26:04,035][20347] Updated weights for policy 0, policy_version 1701698 (0.0007) [2023-12-27 17:26:04,277][20381] Updated weights for policy 1, policy_version 1700177 (0.0008) [2023-12-27 17:26:04,351][20381] Updated weights for policy 1, policy_version 1700187 (0.0008) [2023-12-27 17:26:04,402][20381] Updated weights for policy 1, policy_version 1700197 (0.0008) [2023-12-27 17:26:04,457][20381] Updated weights for policy 1, policy_version 1700207 (0.0008) [2023-12-27 17:26:04,697][20347] Updated weights for policy 0, policy_version 1701708 (0.0007) [2023-12-27 17:26:04,755][20347] Updated weights for policy 0, policy_version 1701718 (0.0007) [2023-12-27 17:26:04,803][20347] Updated weights for policy 0, policy_version 1701728 (0.0007) [2023-12-27 17:26:05,123][20381] Updated weights for policy 1, policy_version 1700217 (0.0010) [2023-12-27 17:26:05,170][20381] Updated weights for policy 1, policy_version 1700227 (0.0010) [2023-12-27 17:26:05,220][20381] Updated weights for policy 1, policy_version 1700237 (0.0009) [2023-12-27 17:26:05,443][20347] Updated weights for policy 0, policy_version 1701738 (0.0007) [2023-12-27 17:26:05,497][20347] Updated weights for policy 0, policy_version 1701748 (0.0007) [2023-12-27 17:26:05,555][20347] Updated weights for policy 0, policy_version 1701758 (0.0007) [2023-12-27 17:26:05,611][20347] Updated weights for policy 0, policy_version 1701768 (0.0007) [2023-12-27 17:26:05,887][20381] Updated weights for policy 1, policy_version 1700247 (0.0007) [2023-12-27 17:26:05,946][20381] Updated weights for policy 1, policy_version 1700257 (0.0007) [2023-12-27 17:26:05,996][20381] Updated weights for policy 1, policy_version 1700267 (0.0009) [2023-12-27 17:26:06,235][20347] Updated weights for policy 0, policy_version 1701778 (0.0008) [2023-12-27 17:26:06,287][20347] Updated weights for policy 0, policy_version 1701788 (0.0008) [2023-12-27 17:26:06,347][20347] Updated weights for policy 0, policy_version 1701798 (0.0008) [2023-12-27 17:26:06,710][20381] Updated weights for policy 1, policy_version 1700277 (0.0009) [2023-12-27 17:26:06,762][20381] Updated weights for policy 1, policy_version 1700287 (0.0008) [2023-12-27 17:26:06,811][20381] Updated weights for policy 1, policy_version 1700297 (0.0007) [2023-12-27 17:26:07,036][20347] Updated weights for policy 0, policy_version 1701808 (0.0010) [2023-12-27 17:26:07,095][20347] Updated weights for policy 0, policy_version 1701818 (0.0010) [2023-12-27 17:26:07,150][20347] Updated weights for policy 0, policy_version 1701828 (0.0010) [2023-12-27 17:26:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20479.9, 300 sec: 20632.7). Total num frames: 871071744. Throughput: 0: 10293.2, 1: 10300.6. Samples: 871040124. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:07,364][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 17:26:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001700304_435322880.pth... [2023-12-27 17:26:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001701832_435748864.pth... [2023-12-27 17:26:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001700648_435445760.pth [2023-12-27 17:26:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001699088_435011584.pth [2023-12-27 17:26:07,504][20381] Updated weights for policy 1, policy_version 1700307 (0.0007) [2023-12-27 17:26:07,556][20381] Updated weights for policy 1, policy_version 1700317 (0.0005) [2023-12-27 17:26:07,613][20381] Updated weights for policy 1, policy_version 1700327 (0.0008) [2023-12-27 17:26:07,828][20347] Updated weights for policy 0, policy_version 1701838 (0.0010) [2023-12-27 17:26:07,876][20347] Updated weights for policy 0, policy_version 1701848 (0.0010) [2023-12-27 17:26:07,927][20347] Updated weights for policy 0, policy_version 1701858 (0.0010) [2023-12-27 17:26:08,264][20381] Updated weights for policy 1, policy_version 1700337 (0.0007) [2023-12-27 17:26:08,327][20381] Updated weights for policy 1, policy_version 1700347 (0.0006) [2023-12-27 17:26:08,389][20381] Updated weights for policy 1, policy_version 1700357 (0.0009) [2023-12-27 17:26:08,443][20381] Updated weights for policy 1, policy_version 1700367 (0.0008) [2023-12-27 17:26:08,621][20347] Updated weights for policy 0, policy_version 1701868 (0.0010) [2023-12-27 17:26:08,685][20347] Updated weights for policy 0, policy_version 1701878 (0.0010) [2023-12-27 17:26:08,738][20347] Updated weights for policy 0, policy_version 1701888 (0.0010) [2023-12-27 17:26:09,077][20381] Updated weights for policy 1, policy_version 1700377 (0.0011) [2023-12-27 17:26:09,133][20381] Updated weights for policy 1, policy_version 1700387 (0.0009) [2023-12-27 17:26:09,190][20381] Updated weights for policy 1, policy_version 1700397 (0.0011) [2023-12-27 17:26:09,503][20347] Updated weights for policy 0, policy_version 1701898 (0.0010) [2023-12-27 17:26:09,553][20347] Updated weights for policy 0, policy_version 1701908 (0.0009) [2023-12-27 17:26:09,610][20347] Updated weights for policy 0, policy_version 1701918 (0.0009) [2023-12-27 17:26:09,667][20347] Updated weights for policy 0, policy_version 1701928 (0.0009) [2023-12-27 17:26:09,982][20381] Updated weights for policy 1, policy_version 1700407 (0.0011) [2023-12-27 17:26:10,043][20381] Updated weights for policy 1, policy_version 1700417 (0.0011) [2023-12-27 17:26:10,105][20381] Updated weights for policy 1, policy_version 1700427 (0.0011) [2023-12-27 17:26:10,365][20347] Updated weights for policy 0, policy_version 1701938 (0.0006) [2023-12-27 17:26:10,426][20347] Updated weights for policy 0, policy_version 1701948 (0.0007) [2023-12-27 17:26:10,486][20347] Updated weights for policy 0, policy_version 1701958 (0.0007) [2023-12-27 17:26:10,792][20381] Updated weights for policy 1, policy_version 1700437 (0.0011) [2023-12-27 17:26:10,848][20381] Updated weights for policy 1, policy_version 1700447 (0.0010) [2023-12-27 17:26:10,908][20381] Updated weights for policy 1, policy_version 1700457 (0.0011) [2023-12-27 17:26:11,179][20347] Updated weights for policy 0, policy_version 1701968 (0.0009) [2023-12-27 17:26:11,256][20347] Updated weights for policy 0, policy_version 1701979 (0.0007) [2023-12-27 17:26:11,317][20347] Updated weights for policy 0, policy_version 1701989 (0.0008) [2023-12-27 17:26:11,673][20381] Updated weights for policy 1, policy_version 1700467 (0.0011) [2023-12-27 17:26:11,745][20381] Updated weights for policy 1, policy_version 1700477 (0.0010) [2023-12-27 17:26:11,804][20381] Updated weights for policy 1, policy_version 1700487 (0.0008) [2023-12-27 17:26:12,042][20347] Updated weights for policy 0, policy_version 1701999 (0.0007) [2023-12-27 17:26:12,109][20347] Updated weights for policy 0, policy_version 1702009 (0.0007) [2023-12-27 17:26:12,178][20347] Updated weights for policy 0, policy_version 1702019 (0.0008) [2023-12-27 17:26:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 871170048. Throughput: 0: 10314.9, 1: 10264.1. Samples: 871161716. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:12,363][19170] Avg episode reward: [(0, '990.260'), (1, '1000.000')] [2023-12-27 17:26:12,537][20381] Updated weights for policy 1, policy_version 1700497 (0.0008) [2023-12-27 17:26:12,594][20381] Updated weights for policy 1, policy_version 1700507 (0.0008) [2023-12-27 17:26:12,656][20381] Updated weights for policy 1, policy_version 1700517 (0.0007) [2023-12-27 17:26:12,721][20381] Updated weights for policy 1, policy_version 1700527 (0.0005) [2023-12-27 17:26:12,806][20347] Updated weights for policy 0, policy_version 1702029 (0.0007) [2023-12-27 17:26:12,865][20347] Updated weights for policy 0, policy_version 1702039 (0.0006) [2023-12-27 17:26:12,924][20347] Updated weights for policy 0, policy_version 1702049 (0.0008) [2023-12-27 17:26:13,335][20381] Updated weights for policy 1, policy_version 1700537 (0.0010) [2023-12-27 17:26:13,387][20381] Updated weights for policy 1, policy_version 1700547 (0.0010) [2023-12-27 17:26:13,445][20381] Updated weights for policy 1, policy_version 1700557 (0.0011) [2023-12-27 17:26:13,591][20347] Updated weights for policy 0, policy_version 1702059 (0.0008) [2023-12-27 17:26:13,638][20347] Updated weights for policy 0, policy_version 1702069 (0.0007) [2023-12-27 17:26:13,692][20347] Updated weights for policy 0, policy_version 1702079 (0.0007) [2023-12-27 17:26:14,138][20381] Updated weights for policy 1, policy_version 1700567 (0.0011) [2023-12-27 17:26:14,184][20381] Updated weights for policy 1, policy_version 1700577 (0.0010) [2023-12-27 17:26:14,250][20381] Updated weights for policy 1, policy_version 1700587 (0.0011) [2023-12-27 17:26:14,393][20347] Updated weights for policy 0, policy_version 1702089 (0.0007) [2023-12-27 17:26:14,443][20347] Updated weights for policy 0, policy_version 1702099 (0.0008) [2023-12-27 17:26:14,504][20347] Updated weights for policy 0, policy_version 1702109 (0.0008) [2023-12-27 17:26:14,561][20347] Updated weights for policy 0, policy_version 1702119 (0.0008) [2023-12-27 17:26:14,877][20381] Updated weights for policy 1, policy_version 1700597 (0.0009) [2023-12-27 17:26:14,942][20381] Updated weights for policy 1, policy_version 1700607 (0.0007) [2023-12-27 17:26:15,010][20381] Updated weights for policy 1, policy_version 1700617 (0.0006) [2023-12-27 17:26:15,376][20347] Updated weights for policy 0, policy_version 1702129 (0.0007) [2023-12-27 17:26:15,427][20347] Updated weights for policy 0, policy_version 1702139 (0.0007) [2023-12-27 17:26:15,485][20347] Updated weights for policy 0, policy_version 1702149 (0.0007) [2023-12-27 17:26:15,618][20381] Updated weights for policy 1, policy_version 1700627 (0.0007) [2023-12-27 17:26:15,672][20381] Updated weights for policy 1, policy_version 1700637 (0.0008) [2023-12-27 17:26:15,722][20381] Updated weights for policy 1, policy_version 1700647 (0.0007) [2023-12-27 17:26:16,209][20347] Updated weights for policy 0, policy_version 1702159 (0.0007) [2023-12-27 17:26:16,274][20347] Updated weights for policy 0, policy_version 1702169 (0.0006) [2023-12-27 17:26:16,337][20347] Updated weights for policy 0, policy_version 1702179 (0.0005) [2023-12-27 17:26:16,348][20381] Updated weights for policy 1, policy_version 1700657 (0.0007) [2023-12-27 17:26:16,409][20381] Updated weights for policy 1, policy_version 1700667 (0.0010) [2023-12-27 17:26:16,470][20381] Updated weights for policy 1, policy_version 1700677 (0.0010) [2023-12-27 17:26:16,526][20381] Updated weights for policy 1, policy_version 1700687 (0.0009) [2023-12-27 17:26:16,878][20347] Updated weights for policy 0, policy_version 1702189 (0.0007) [2023-12-27 17:26:16,930][20347] Updated weights for policy 0, policy_version 1702199 (0.0007) [2023-12-27 17:26:16,982][20347] Updated weights for policy 0, policy_version 1702209 (0.0006) [2023-12-27 17:26:17,134][20381] Updated weights for policy 1, policy_version 1700697 (0.0006) [2023-12-27 17:26:17,192][20381] Updated weights for policy 1, policy_version 1700707 (0.0007) [2023-12-27 17:26:17,250][20381] Updated weights for policy 1, policy_version 1700717 (0.0007) [2023-12-27 17:26:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 871276544. Throughput: 0: 10241.4, 1: 10292.8. Samples: 871283792. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:17,363][19170] Avg episode reward: [(0, '970.750'), (1, '1000.000')] [2023-12-27 17:26:17,708][20347] Updated weights for policy 0, policy_version 1702219 (0.0007) [2023-12-27 17:26:17,763][20347] Updated weights for policy 0, policy_version 1702229 (0.0009) [2023-12-27 17:26:17,812][20347] Updated weights for policy 0, policy_version 1702239 (0.0007) [2023-12-27 17:26:17,848][20381] Updated weights for policy 1, policy_version 1700727 (0.0007) [2023-12-27 17:26:17,903][20381] Updated weights for policy 1, policy_version 1700737 (0.0007) [2023-12-27 17:26:17,947][20381] Updated weights for policy 1, policy_version 1700747 (0.0008) [2023-12-27 17:26:18,425][20347] Updated weights for policy 0, policy_version 1702249 (0.0005) [2023-12-27 17:26:18,488][20347] Updated weights for policy 0, policy_version 1702259 (0.0006) [2023-12-27 17:26:18,553][20347] Updated weights for policy 0, policy_version 1702269 (0.0008) [2023-12-27 17:26:18,612][20347] Updated weights for policy 0, policy_version 1702279 (0.0005) [2023-12-27 17:26:18,636][20381] Updated weights for policy 1, policy_version 1700757 (0.0008) [2023-12-27 17:26:18,694][20381] Updated weights for policy 1, policy_version 1700767 (0.0007) [2023-12-27 17:26:18,753][20381] Updated weights for policy 1, policy_version 1700777 (0.0005) [2023-12-27 17:26:19,251][20347] Updated weights for policy 0, policy_version 1702289 (0.0008) [2023-12-27 17:26:19,310][20347] Updated weights for policy 0, policy_version 1702299 (0.0007) [2023-12-27 17:26:19,362][20291] KL-divergence is very high: 165.6732 [2023-12-27 17:26:19,385][20347] Updated weights for policy 0, policy_version 1702309 (0.0008) [2023-12-27 17:26:19,402][20381] Updated weights for policy 1, policy_version 1700787 (0.0006) [2023-12-27 17:26:19,456][20381] Updated weights for policy 1, policy_version 1700797 (0.0008) [2023-12-27 17:26:19,517][20381] Updated weights for policy 1, policy_version 1700807 (0.0008) [2023-12-27 17:26:20,075][20347] Updated weights for policy 0, policy_version 1702319 (0.0007) [2023-12-27 17:26:20,142][20347] Updated weights for policy 0, policy_version 1702329 (0.0007) [2023-12-27 17:26:20,209][20347] Updated weights for policy 0, policy_version 1702339 (0.0008) [2023-12-27 17:26:20,286][20381] Updated weights for policy 1, policy_version 1700817 (0.0008) [2023-12-27 17:26:20,354][20381] Updated weights for policy 1, policy_version 1700827 (0.0010) [2023-12-27 17:26:20,404][20381] Updated weights for policy 1, policy_version 1700837 (0.0008) [2023-12-27 17:26:20,457][20381] Updated weights for policy 1, policy_version 1700847 (0.0007) [2023-12-27 17:26:20,882][20347] Updated weights for policy 0, policy_version 1702349 (0.0007) [2023-12-27 17:26:20,944][20347] Updated weights for policy 0, policy_version 1702359 (0.0008) [2023-12-27 17:26:21,004][20347] Updated weights for policy 0, policy_version 1702369 (0.0008) [2023-12-27 17:26:21,210][20381] Updated weights for policy 1, policy_version 1700857 (0.0010) [2023-12-27 17:26:21,279][20381] Updated weights for policy 1, policy_version 1700867 (0.0008) [2023-12-27 17:26:21,340][20381] Updated weights for policy 1, policy_version 1700877 (0.0010) [2023-12-27 17:26:21,818][20347] Updated weights for policy 0, policy_version 1702379 (0.0008) [2023-12-27 17:26:21,878][20347] Updated weights for policy 0, policy_version 1702389 (0.0006) [2023-12-27 17:26:21,938][20347] Updated weights for policy 0, policy_version 1702399 (0.0006) [2023-12-27 17:26:22,005][20381] Updated weights for policy 1, policy_version 1700887 (0.0009) [2023-12-27 17:26:22,065][20381] Updated weights for policy 1, policy_version 1700897 (0.0007) [2023-12-27 17:26:22,122][20381] Updated weights for policy 1, policy_version 1700907 (0.0008) [2023-12-27 17:26:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 871374848. Throughput: 0: 10202.9, 1: 10338.7. Samples: 871346092. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:22,363][19170] Avg episode reward: [(0, '970.690'), (1, '1000.000')] [2023-12-27 17:26:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001700912_435478528.pth... [2023-12-27 17:26:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001702408_435896320.pth... [2023-12-27 17:26:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001699696_435167232.pth [2023-12-27 17:26:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001701224_435593216.pth [2023-12-27 17:26:22,560][20347] Updated weights for policy 0, policy_version 1702409 (0.0006) [2023-12-27 17:26:22,617][20347] Updated weights for policy 0, policy_version 1702419 (0.0009) [2023-12-27 17:26:22,678][20347] Updated weights for policy 0, policy_version 1702429 (0.0008) [2023-12-27 17:26:22,738][20347] Updated weights for policy 0, policy_version 1702439 (0.0008) [2023-12-27 17:26:22,889][20381] Updated weights for policy 1, policy_version 1700917 (0.0008) [2023-12-27 17:26:22,945][20381] Updated weights for policy 1, policy_version 1700927 (0.0009) [2023-12-27 17:26:23,003][20381] Updated weights for policy 1, policy_version 1700937 (0.0008) [2023-12-27 17:26:23,460][20347] Updated weights for policy 0, policy_version 1702449 (0.0006) [2023-12-27 17:26:23,517][20347] Updated weights for policy 0, policy_version 1702459 (0.0005) [2023-12-27 17:26:23,575][20347] Updated weights for policy 0, policy_version 1702469 (0.0005) [2023-12-27 17:26:23,635][20381] Updated weights for policy 1, policy_version 1700947 (0.0007) [2023-12-27 17:26:23,686][20381] Updated weights for policy 1, policy_version 1700957 (0.0005) [2023-12-27 17:26:23,747][20381] Updated weights for policy 1, policy_version 1700967 (0.0007) [2023-12-27 17:26:24,212][20347] Updated weights for policy 0, policy_version 1702479 (0.0007) [2023-12-27 17:26:24,263][20347] Updated weights for policy 0, policy_version 1702489 (0.0007) [2023-12-27 17:26:24,324][20347] Updated weights for policy 0, policy_version 1702499 (0.0006) [2023-12-27 17:26:24,358][20381] Updated weights for policy 1, policy_version 1700977 (0.0010) [2023-12-27 17:26:24,417][20381] Updated weights for policy 1, policy_version 1700987 (0.0010) [2023-12-27 17:26:24,468][20381] Updated weights for policy 1, policy_version 1700997 (0.0009) [2023-12-27 17:26:24,520][20381] Updated weights for policy 1, policy_version 1701007 (0.0010) [2023-12-27 17:26:24,988][20347] Updated weights for policy 0, policy_version 1702509 (0.0009) [2023-12-27 17:26:25,041][20347] Updated weights for policy 0, policy_version 1702519 (0.0010) [2023-12-27 17:26:25,095][20347] Updated weights for policy 0, policy_version 1702529 (0.0010) [2023-12-27 17:26:25,177][20381] Updated weights for policy 1, policy_version 1701017 (0.0008) [2023-12-27 17:26:25,231][20381] Updated weights for policy 1, policy_version 1701027 (0.0006) [2023-12-27 17:26:25,283][20381] Updated weights for policy 1, policy_version 1701037 (0.0007) [2023-12-27 17:26:25,774][20347] Updated weights for policy 0, policy_version 1702539 (0.0010) [2023-12-27 17:26:25,828][20347] Updated weights for policy 0, policy_version 1702549 (0.0010) [2023-12-27 17:26:25,886][20347] Updated weights for policy 0, policy_version 1702559 (0.0010) [2023-12-27 17:26:25,904][20381] Updated weights for policy 1, policy_version 1701047 (0.0008) [2023-12-27 17:26:25,952][20381] Updated weights for policy 1, policy_version 1701057 (0.0008) [2023-12-27 17:26:25,995][20381] Updated weights for policy 1, policy_version 1701067 (0.0009) [2023-12-27 17:26:26,561][20347] Updated weights for policy 0, policy_version 1702569 (0.0010) [2023-12-27 17:26:26,618][20347] Updated weights for policy 0, policy_version 1702579 (0.0010) [2023-12-27 17:26:26,626][20381] Updated weights for policy 1, policy_version 1701077 (0.0005) [2023-12-27 17:26:26,668][20347] Updated weights for policy 0, policy_version 1702589 (0.0007) [2023-12-27 17:26:26,681][20381] Updated weights for policy 1, policy_version 1701087 (0.0006) [2023-12-27 17:26:26,718][20347] Updated weights for policy 0, policy_version 1702599 (0.0005) [2023-12-27 17:26:26,733][20381] Updated weights for policy 1, policy_version 1701097 (0.0008) [2023-12-27 17:26:27,312][20347] Updated weights for policy 0, policy_version 1702609 (0.0010) [2023-12-27 17:26:27,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 871473152. Throughput: 0: 10142.3, 1: 10364.0. Samples: 871468048. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:27,363][19170] Avg episode reward: [(0, '990.200'), (1, '1000.000')] [2023-12-27 17:26:27,371][20347] Updated weights for policy 0, policy_version 1702619 (0.0010) [2023-12-27 17:26:27,425][20381] Updated weights for policy 1, policy_version 1701107 (0.0007) [2023-12-27 17:26:27,426][20347] Updated weights for policy 0, policy_version 1702629 (0.0010) [2023-12-27 17:26:27,480][20381] Updated weights for policy 1, policy_version 1701117 (0.0005) [2023-12-27 17:26:27,537][20381] Updated weights for policy 1, policy_version 1701127 (0.0005) [2023-12-27 17:26:27,995][20347] Updated weights for policy 0, policy_version 1702639 (0.0007) [2023-12-27 17:26:28,049][20347] Updated weights for policy 0, policy_version 1702649 (0.0005) [2023-12-27 17:26:28,085][20381] Updated weights for policy 1, policy_version 1701137 (0.0008) [2023-12-27 17:26:28,102][20347] Updated weights for policy 0, policy_version 1702659 (0.0005) [2023-12-27 17:26:28,146][20381] Updated weights for policy 1, policy_version 1701147 (0.0010) [2023-12-27 17:26:28,200][20381] Updated weights for policy 1, policy_version 1701157 (0.0010) [2023-12-27 17:26:28,251][20381] Updated weights for policy 1, policy_version 1701167 (0.0010) [2023-12-27 17:26:28,725][20347] Updated weights for policy 0, policy_version 1702669 (0.0005) [2023-12-27 17:26:28,783][20347] Updated weights for policy 0, policy_version 1702679 (0.0006) [2023-12-27 17:26:28,841][20347] Updated weights for policy 0, policy_version 1702689 (0.0007) [2023-12-27 17:26:28,968][20381] Updated weights for policy 1, policy_version 1701177 (0.0011) [2023-12-27 17:26:29,020][20381] Updated weights for policy 1, policy_version 1701187 (0.0010) [2023-12-27 17:26:29,071][20381] Updated weights for policy 1, policy_version 1701197 (0.0010) [2023-12-27 17:26:29,479][20347] Updated weights for policy 0, policy_version 1702699 (0.0007) [2023-12-27 17:26:29,535][20347] Updated weights for policy 0, policy_version 1702709 (0.0005) [2023-12-27 17:26:29,595][20347] Updated weights for policy 0, policy_version 1702719 (0.0006) [2023-12-27 17:26:29,778][20381] Updated weights for policy 1, policy_version 1701207 (0.0008) [2023-12-27 17:26:29,843][20381] Updated weights for policy 1, policy_version 1701217 (0.0010) [2023-12-27 17:26:29,899][20381] Updated weights for policy 1, policy_version 1701227 (0.0007) [2023-12-27 17:26:30,234][20347] Updated weights for policy 0, policy_version 1702729 (0.0006) [2023-12-27 17:26:30,290][20347] Updated weights for policy 0, policy_version 1702739 (0.0005) [2023-12-27 17:26:30,352][20347] Updated weights for policy 0, policy_version 1702749 (0.0006) [2023-12-27 17:26:30,405][20347] Updated weights for policy 0, policy_version 1702759 (0.0007) [2023-12-27 17:26:30,587][20381] Updated weights for policy 1, policy_version 1701237 (0.0009) [2023-12-27 17:26:30,642][20381] Updated weights for policy 1, policy_version 1701247 (0.0010) [2023-12-27 17:26:30,694][20381] Updated weights for policy 1, policy_version 1701257 (0.0010) [2023-12-27 17:26:31,065][20347] Updated weights for policy 0, policy_version 1702769 (0.0007) [2023-12-27 17:26:31,124][20347] Updated weights for policy 0, policy_version 1702779 (0.0007) [2023-12-27 17:26:31,188][20347] Updated weights for policy 0, policy_version 1702789 (0.0011) [2023-12-27 17:26:31,423][20381] Updated weights for policy 1, policy_version 1701267 (0.0008) [2023-12-27 17:26:31,484][20381] Updated weights for policy 1, policy_version 1701277 (0.0008) [2023-12-27 17:26:31,545][20381] Updated weights for policy 1, policy_version 1701287 (0.0008) [2023-12-27 17:26:31,944][20347] Updated weights for policy 0, policy_version 1702799 (0.0006) [2023-12-27 17:26:32,003][20347] Updated weights for policy 0, policy_version 1702809 (0.0007) [2023-12-27 17:26:32,063][20347] Updated weights for policy 0, policy_version 1702819 (0.0007) [2023-12-27 17:26:32,290][20381] Updated weights for policy 1, policy_version 1701297 (0.0008) [2023-12-27 17:26:32,353][20381] Updated weights for policy 1, policy_version 1701307 (0.0008) [2023-12-27 17:26:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 871579648. Throughput: 0: 10230.2, 1: 10314.9. Samples: 871593088. Policy #0 lag: (min: 10.0, avg: 17.0, max: 42.0) [2023-12-27 17:26:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:26:32,412][20381] Updated weights for policy 1, policy_version 1701317 (0.0008) [2023-12-27 17:26:32,464][20381] Updated weights for policy 1, policy_version 1701327 (0.0008) [2023-12-27 17:26:32,744][20347] Updated weights for policy 0, policy_version 1702829 (0.0007) [2023-12-27 17:26:32,805][20347] Updated weights for policy 0, policy_version 1702839 (0.0008) [2023-12-27 17:26:32,857][20347] Updated weights for policy 0, policy_version 1702849 (0.0008) [2023-12-27 17:26:33,186][20381] Updated weights for policy 1, policy_version 1701337 (0.0009) [2023-12-27 17:26:33,231][20381] Updated weights for policy 1, policy_version 1701347 (0.0008) [2023-12-27 17:26:33,282][20381] Updated weights for policy 1, policy_version 1701357 (0.0008) [2023-12-27 17:26:33,417][20347] Updated weights for policy 0, policy_version 1702859 (0.0007) [2023-12-27 17:26:33,461][20347] Updated weights for policy 0, policy_version 1702869 (0.0007) [2023-12-27 17:26:33,507][20347] Updated weights for policy 0, policy_version 1702879 (0.0008) [2023-12-27 17:26:34,032][20381] Updated weights for policy 1, policy_version 1701367 (0.0007) [2023-12-27 17:26:34,083][20381] Updated weights for policy 1, policy_version 1701377 (0.0008) [2023-12-27 17:26:34,131][20381] Updated weights for policy 1, policy_version 1701387 (0.0007) [2023-12-27 17:26:34,154][20347] Updated weights for policy 0, policy_version 1702889 (0.0006) [2023-12-27 17:26:34,210][20347] Updated weights for policy 0, policy_version 1702899 (0.0008) [2023-12-27 17:26:34,266][20347] Updated weights for policy 0, policy_version 1702909 (0.0008) [2023-12-27 17:26:34,328][20347] Updated weights for policy 0, policy_version 1702919 (0.0008) [2023-12-27 17:26:34,857][20381] Updated weights for policy 1, policy_version 1701397 (0.0007) [2023-12-27 17:26:34,918][20381] Updated weights for policy 1, policy_version 1701407 (0.0005) [2023-12-27 17:26:34,981][20381] Updated weights for policy 1, policy_version 1701417 (0.0006) [2023-12-27 17:26:34,990][20347] Updated weights for policy 0, policy_version 1702929 (0.0007) [2023-12-27 17:26:35,049][20347] Updated weights for policy 0, policy_version 1702939 (0.0008) [2023-12-27 17:26:35,110][20347] Updated weights for policy 0, policy_version 1702949 (0.0008) [2023-12-27 17:26:35,602][20381] Updated weights for policy 1, policy_version 1701427 (0.0006) [2023-12-27 17:26:35,650][20381] Updated weights for policy 1, policy_version 1701437 (0.0005) [2023-12-27 17:26:35,694][20381] Updated weights for policy 1, policy_version 1701447 (0.0005) [2023-12-27 17:26:35,741][20347] Updated weights for policy 0, policy_version 1702959 (0.0006) [2023-12-27 17:26:35,808][20347] Updated weights for policy 0, policy_version 1702969 (0.0006) [2023-12-27 17:26:35,867][20347] Updated weights for policy 0, policy_version 1702979 (0.0007) [2023-12-27 17:26:36,256][20381] Updated weights for policy 1, policy_version 1701457 (0.0005) [2023-12-27 17:26:36,313][20381] Updated weights for policy 1, policy_version 1701467 (0.0006) [2023-12-27 17:26:36,379][20381] Updated weights for policy 1, policy_version 1701477 (0.0006) [2023-12-27 17:26:36,443][20381] Updated weights for policy 1, policy_version 1701487 (0.0006) [2023-12-27 17:26:36,571][20347] Updated weights for policy 0, policy_version 1702989 (0.0008) [2023-12-27 17:26:36,628][20347] Updated weights for policy 0, policy_version 1702999 (0.0010) [2023-12-27 17:26:36,687][20347] Updated weights for policy 0, policy_version 1703009 (0.0010) [2023-12-27 17:26:37,031][20381] Updated weights for policy 1, policy_version 1701497 (0.0010) [2023-12-27 17:26:37,085][20381] Updated weights for policy 1, policy_version 1701507 (0.0010) [2023-12-27 17:26:37,140][20381] Updated weights for policy 1, policy_version 1701517 (0.0009) [2023-12-27 17:26:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 871686144. Throughput: 0: 10293.9, 1: 10267.7. Samples: 871654616. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:26:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:26:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001701520_435634176.pth... [2023-12-27 17:26:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001700304_435322880.pth [2023-12-27 17:26:37,383][20347] Updated weights for policy 0, policy_version 1703019 (0.0010) [2023-12-27 17:26:37,434][20347] Updated weights for policy 0, policy_version 1703029 (0.0007) [2023-12-27 17:26:37,489][20347] Updated weights for policy 0, policy_version 1703039 (0.0010) [2023-12-27 17:26:37,531][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001703048_436060160.pth... [2023-12-27 17:26:37,534][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001701832_435748864.pth [2023-12-27 17:26:37,775][20381] Updated weights for policy 1, policy_version 1701527 (0.0006) [2023-12-27 17:26:37,838][20381] Updated weights for policy 1, policy_version 1701537 (0.0008) [2023-12-27 17:26:37,901][20381] Updated weights for policy 1, policy_version 1701547 (0.0007) [2023-12-27 17:26:38,095][20347] Updated weights for policy 0, policy_version 1703049 (0.0010) [2023-12-27 17:26:38,147][20347] Updated weights for policy 0, policy_version 1703059 (0.0008) [2023-12-27 17:26:38,199][20347] Updated weights for policy 0, policy_version 1703069 (0.0009) [2023-12-27 17:26:38,257][20347] Updated weights for policy 0, policy_version 1703079 (0.0008) [2023-12-27 17:26:38,554][20381] Updated weights for policy 1, policy_version 1701557 (0.0008) [2023-12-27 17:26:38,615][20381] Updated weights for policy 1, policy_version 1701567 (0.0008) [2023-12-27 17:26:38,672][20381] Updated weights for policy 1, policy_version 1701577 (0.0008) [2023-12-27 17:26:38,926][20347] Updated weights for policy 0, policy_version 1703089 (0.0007) [2023-12-27 17:26:38,984][20347] Updated weights for policy 0, policy_version 1703099 (0.0008) [2023-12-27 17:26:39,051][20347] Updated weights for policy 0, policy_version 1703109 (0.0008) [2023-12-27 17:26:39,339][20381] Updated weights for policy 1, policy_version 1701587 (0.0008) [2023-12-27 17:26:39,410][20381] Updated weights for policy 1, policy_version 1701597 (0.0008) [2023-12-27 17:26:39,471][20381] Updated weights for policy 1, policy_version 1701607 (0.0008) [2023-12-27 17:26:39,864][20347] Updated weights for policy 0, policy_version 1703119 (0.0008) [2023-12-27 17:26:39,921][20347] Updated weights for policy 0, policy_version 1703129 (0.0008) [2023-12-27 17:26:39,984][20347] Updated weights for policy 0, policy_version 1703139 (0.0008) [2023-12-27 17:26:40,221][20381] Updated weights for policy 1, policy_version 1701617 (0.0008) [2023-12-27 17:26:40,278][20381] Updated weights for policy 1, policy_version 1701627 (0.0007) [2023-12-27 17:26:40,336][20381] Updated weights for policy 1, policy_version 1701637 (0.0008) [2023-12-27 17:26:40,390][20381] Updated weights for policy 1, policy_version 1701647 (0.0008) [2023-12-27 17:26:40,645][20347] Updated weights for policy 0, policy_version 1703149 (0.0007) [2023-12-27 17:26:40,704][20347] Updated weights for policy 0, policy_version 1703159 (0.0008) [2023-12-27 17:26:40,764][20347] Updated weights for policy 0, policy_version 1703169 (0.0006) [2023-12-27 17:26:41,065][20381] Updated weights for policy 1, policy_version 1701657 (0.0008) [2023-12-27 17:26:41,117][20381] Updated weights for policy 1, policy_version 1701667 (0.0008) [2023-12-27 17:26:41,181][20381] Updated weights for policy 1, policy_version 1701677 (0.0008) [2023-12-27 17:26:41,522][20347] Updated weights for policy 0, policy_version 1703179 (0.0009) [2023-12-27 17:26:41,575][20347] Updated weights for policy 0, policy_version 1703189 (0.0011) [2023-12-27 17:26:41,650][20347] Updated weights for policy 0, policy_version 1703199 (0.0011) [2023-12-27 17:26:41,960][20381] Updated weights for policy 1, policy_version 1701687 (0.0009) [2023-12-27 17:26:42,017][20381] Updated weights for policy 1, policy_version 1701697 (0.0009) [2023-12-27 17:26:42,082][20381] Updated weights for policy 1, policy_version 1701707 (0.0010) [2023-12-27 17:26:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 871784448. Throughput: 0: 10253.8, 1: 10270.0. Samples: 871777936. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:26:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:26:42,425][20347] Updated weights for policy 0, policy_version 1703209 (0.0009) [2023-12-27 17:26:42,489][20347] Updated weights for policy 0, policy_version 1703219 (0.0007) [2023-12-27 17:26:42,549][20347] Updated weights for policy 0, policy_version 1703229 (0.0008) [2023-12-27 17:26:42,609][20347] Updated weights for policy 0, policy_version 1703239 (0.0008) [2023-12-27 17:26:42,807][20381] Updated weights for policy 1, policy_version 1701717 (0.0008) [2023-12-27 17:26:42,863][20381] Updated weights for policy 1, policy_version 1701727 (0.0007) [2023-12-27 17:26:42,918][20381] Updated weights for policy 1, policy_version 1701737 (0.0008) [2023-12-27 17:26:43,289][20347] Updated weights for policy 0, policy_version 1703249 (0.0007) [2023-12-27 17:26:43,341][20347] Updated weights for policy 0, policy_version 1703259 (0.0007) [2023-12-27 17:26:43,387][20347] Updated weights for policy 0, policy_version 1703269 (0.0007) [2023-12-27 17:26:43,551][20381] Updated weights for policy 1, policy_version 1701747 (0.0008) [2023-12-27 17:26:43,613][20381] Updated weights for policy 1, policy_version 1701757 (0.0006) [2023-12-27 17:26:43,664][20381] Updated weights for policy 1, policy_version 1701767 (0.0008) [2023-12-27 17:26:44,064][20347] Updated weights for policy 0, policy_version 1703279 (0.0008) [2023-12-27 17:26:44,128][20347] Updated weights for policy 0, policy_version 1703289 (0.0009) [2023-12-27 17:26:44,177][20347] Updated weights for policy 0, policy_version 1703299 (0.0009) [2023-12-27 17:26:44,251][20381] Updated weights for policy 1, policy_version 1701777 (0.0007) [2023-12-27 17:26:44,312][20381] Updated weights for policy 1, policy_version 1701787 (0.0005) [2023-12-27 17:26:44,358][20381] Updated weights for policy 1, policy_version 1701797 (0.0005) [2023-12-27 17:26:44,405][20381] Updated weights for policy 1, policy_version 1701807 (0.0008) [2023-12-27 17:26:44,797][20347] Updated weights for policy 0, policy_version 1703309 (0.0008) [2023-12-27 17:26:44,859][20347] Updated weights for policy 0, policy_version 1703319 (0.0006) [2023-12-27 17:26:44,919][20347] Updated weights for policy 0, policy_version 1703329 (0.0006) [2023-12-27 17:26:45,162][20381] Updated weights for policy 1, policy_version 1701817 (0.0009) [2023-12-27 17:26:45,223][20381] Updated weights for policy 1, policy_version 1701827 (0.0008) [2023-12-27 17:26:45,288][20381] Updated weights for policy 1, policy_version 1701837 (0.0008) [2023-12-27 17:26:45,604][20347] Updated weights for policy 0, policy_version 1703339 (0.0006) [2023-12-27 17:26:45,662][20347] Updated weights for policy 0, policy_version 1703349 (0.0006) [2023-12-27 17:26:45,725][20347] Updated weights for policy 0, policy_version 1703359 (0.0005) [2023-12-27 17:26:45,999][20381] Updated weights for policy 1, policy_version 1701847 (0.0007) [2023-12-27 17:26:46,054][20381] Updated weights for policy 1, policy_version 1701857 (0.0007) [2023-12-27 17:26:46,105][20381] Updated weights for policy 1, policy_version 1701867 (0.0007) [2023-12-27 17:26:46,405][20347] Updated weights for policy 0, policy_version 1703369 (0.0007) [2023-12-27 17:26:46,453][20347] Updated weights for policy 0, policy_version 1703379 (0.0008) [2023-12-27 17:26:46,516][20347] Updated weights for policy 0, policy_version 1703389 (0.0007) [2023-12-27 17:26:46,573][20347] Updated weights for policy 0, policy_version 1703399 (0.0008) [2023-12-27 17:26:46,715][20381] Updated weights for policy 1, policy_version 1701877 (0.0009) [2023-12-27 17:26:46,765][20381] Updated weights for policy 1, policy_version 1701887 (0.0011) [2023-12-27 17:26:46,813][20381] Updated weights for policy 1, policy_version 1701897 (0.0010) [2023-12-27 17:26:47,140][20347] Updated weights for policy 0, policy_version 1703409 (0.0006) [2023-12-27 17:26:47,190][20347] Updated weights for policy 0, policy_version 1703419 (0.0006) [2023-12-27 17:26:47,242][20347] Updated weights for policy 0, policy_version 1703429 (0.0006) [2023-12-27 17:26:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 871890944. Throughput: 0: 10226.4, 1: 10240.4. Samples: 871900200. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:26:47,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 17:26:47,514][20381] Updated weights for policy 1, policy_version 1701907 (0.0011) [2023-12-27 17:26:47,576][20381] Updated weights for policy 1, policy_version 1701917 (0.0011) [2023-12-27 17:26:47,642][20381] Updated weights for policy 1, policy_version 1701927 (0.0010) [2023-12-27 17:26:47,889][20347] Updated weights for policy 0, policy_version 1703439 (0.0006) [2023-12-27 17:26:47,939][20347] Updated weights for policy 0, policy_version 1703449 (0.0009) [2023-12-27 17:26:47,989][20347] Updated weights for policy 0, policy_version 1703459 (0.0010) [2023-12-27 17:26:48,297][20381] Updated weights for policy 1, policy_version 1701937 (0.0010) [2023-12-27 17:26:48,365][20381] Updated weights for policy 1, policy_version 1701947 (0.0009) [2023-12-27 17:26:48,429][20381] Updated weights for policy 1, policy_version 1701957 (0.0011) [2023-12-27 17:26:48,497][20381] Updated weights for policy 1, policy_version 1701967 (0.0011) [2023-12-27 17:26:48,713][20347] Updated weights for policy 0, policy_version 1703469 (0.0007) [2023-12-27 17:26:48,771][20347] Updated weights for policy 0, policy_version 1703479 (0.0007) [2023-12-27 17:26:48,829][20347] Updated weights for policy 0, policy_version 1703489 (0.0007) [2023-12-27 17:26:49,156][20381] Updated weights for policy 1, policy_version 1701977 (0.0010) [2023-12-27 17:26:49,206][20381] Updated weights for policy 1, policy_version 1701987 (0.0010) [2023-12-27 17:26:49,264][20381] Updated weights for policy 1, policy_version 1701997 (0.0008) [2023-12-27 17:26:49,515][20347] Updated weights for policy 0, policy_version 1703499 (0.0007) [2023-12-27 17:26:49,575][20347] Updated weights for policy 0, policy_version 1703509 (0.0008) [2023-12-27 17:26:49,635][20347] Updated weights for policy 0, policy_version 1703519 (0.0007) [2023-12-27 17:26:49,985][20381] Updated weights for policy 1, policy_version 1702007 (0.0007) [2023-12-27 17:26:50,045][20381] Updated weights for policy 1, policy_version 1702017 (0.0008) [2023-12-27 17:26:50,091][20381] Updated weights for policy 1, policy_version 1702027 (0.0008) [2023-12-27 17:26:50,240][20347] Updated weights for policy 0, policy_version 1703529 (0.0005) [2023-12-27 17:26:50,298][20347] Updated weights for policy 0, policy_version 1703539 (0.0005) [2023-12-27 17:26:50,358][20347] Updated weights for policy 0, policy_version 1703549 (0.0008) [2023-12-27 17:26:50,420][20347] Updated weights for policy 0, policy_version 1703559 (0.0005) [2023-12-27 17:26:50,842][20381] Updated weights for policy 1, policy_version 1702037 (0.0007) [2023-12-27 17:26:50,905][20381] Updated weights for policy 1, policy_version 1702047 (0.0008) [2023-12-27 17:26:50,958][20381] Updated weights for policy 1, policy_version 1702057 (0.0008) [2023-12-27 17:26:51,066][20347] Updated weights for policy 0, policy_version 1703569 (0.0007) [2023-12-27 17:26:51,129][20347] Updated weights for policy 0, policy_version 1703579 (0.0008) [2023-12-27 17:26:51,191][20347] Updated weights for policy 0, policy_version 1703589 (0.0006) [2023-12-27 17:26:51,696][20381] Updated weights for policy 1, policy_version 1702067 (0.0007) [2023-12-27 17:26:51,763][20381] Updated weights for policy 1, policy_version 1702077 (0.0009) [2023-12-27 17:26:51,817][20381] Updated weights for policy 1, policy_version 1702087 (0.0008) [2023-12-27 17:26:51,879][20347] Updated weights for policy 0, policy_version 1703599 (0.0006) [2023-12-27 17:26:51,935][20347] Updated weights for policy 0, policy_version 1703609 (0.0008) [2023-12-27 17:26:51,991][20347] Updated weights for policy 0, policy_version 1703619 (0.0007) [2023-12-27 17:26:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 871989248. Throughput: 0: 10237.5, 1: 10209.4. Samples: 871960236. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:26:52,363][19170] Avg episode reward: [(0, '990.350'), (1, '1000.000')] [2023-12-27 17:26:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001702096_435781632.pth... [2023-12-27 17:26:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001703624_436207616.pth... [2023-12-27 17:26:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001700912_435478528.pth [2023-12-27 17:26:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001702408_435896320.pth [2023-12-27 17:26:52,480][20381] Updated weights for policy 1, policy_version 1702097 (0.0007) [2023-12-27 17:26:52,543][20381] Updated weights for policy 1, policy_version 1702107 (0.0007) [2023-12-27 17:26:52,599][20381] Updated weights for policy 1, policy_version 1702117 (0.0008) [2023-12-27 17:26:52,652][20381] Updated weights for policy 1, policy_version 1702127 (0.0008) [2023-12-27 17:26:52,698][20347] Updated weights for policy 0, policy_version 1703629 (0.0008) [2023-12-27 17:26:52,750][20347] Updated weights for policy 0, policy_version 1703639 (0.0007) [2023-12-27 17:26:52,811][20347] Updated weights for policy 0, policy_version 1703649 (0.0006) [2023-12-27 17:26:53,390][20381] Updated weights for policy 1, policy_version 1702137 (0.0007) [2023-12-27 17:26:53,445][20381] Updated weights for policy 1, policy_version 1702147 (0.0009) [2023-12-27 17:26:53,497][20347] Updated weights for policy 0, policy_version 1703659 (0.0007) [2023-12-27 17:26:53,499][20381] Updated weights for policy 1, policy_version 1702157 (0.0010) [2023-12-27 17:26:53,547][20347] Updated weights for policy 0, policy_version 1703669 (0.0006) [2023-12-27 17:26:53,594][20347] Updated weights for policy 0, policy_version 1703679 (0.0007) [2023-12-27 17:26:54,132][20381] Updated weights for policy 1, policy_version 1702167 (0.0009) [2023-12-27 17:26:54,199][20381] Updated weights for policy 1, policy_version 1702177 (0.0008) [2023-12-27 17:26:54,257][20381] Updated weights for policy 1, policy_version 1702187 (0.0008) [2023-12-27 17:26:54,304][20347] Updated weights for policy 0, policy_version 1703689 (0.0008) [2023-12-27 17:26:54,368][20347] Updated weights for policy 0, policy_version 1703699 (0.0008) [2023-12-27 17:26:54,427][20347] Updated weights for policy 0, policy_version 1703709 (0.0007) [2023-12-27 17:26:54,489][20347] Updated weights for policy 0, policy_version 1703719 (0.0007) [2023-12-27 17:26:54,936][20381] Updated weights for policy 1, policy_version 1702197 (0.0008) [2023-12-27 17:26:54,990][20381] Updated weights for policy 1, policy_version 1702207 (0.0008) [2023-12-27 17:26:55,047][20381] Updated weights for policy 1, policy_version 1702217 (0.0008) [2023-12-27 17:26:55,186][20347] Updated weights for policy 0, policy_version 1703729 (0.0007) [2023-12-27 17:26:55,241][20347] Updated weights for policy 0, policy_version 1703739 (0.0007) [2023-12-27 17:26:55,295][20347] Updated weights for policy 0, policy_version 1703749 (0.0008) [2023-12-27 17:26:55,755][20381] Updated weights for policy 1, policy_version 1702227 (0.0008) [2023-12-27 17:26:55,812][20381] Updated weights for policy 1, policy_version 1702237 (0.0007) [2023-12-27 17:26:55,867][20381] Updated weights for policy 1, policy_version 1702247 (0.0007) [2023-12-27 17:26:55,929][20347] Updated weights for policy 0, policy_version 1703759 (0.0006) [2023-12-27 17:26:55,984][20347] Updated weights for policy 0, policy_version 1703769 (0.0008) [2023-12-27 17:26:56,030][20347] Updated weights for policy 0, policy_version 1703779 (0.0007) [2023-12-27 17:26:56,604][20381] Updated weights for policy 1, policy_version 1702257 (0.0008) [2023-12-27 17:26:56,627][20347] Updated weights for policy 0, policy_version 1703789 (0.0006) [2023-12-27 17:26:56,659][20381] Updated weights for policy 1, policy_version 1702267 (0.0007) [2023-12-27 17:26:56,681][20347] Updated weights for policy 0, policy_version 1703799 (0.0005) [2023-12-27 17:26:56,708][20381] Updated weights for policy 1, policy_version 1702277 (0.0008) [2023-12-27 17:26:56,737][20347] Updated weights for policy 0, policy_version 1703809 (0.0005) [2023-12-27 17:26:56,752][20381] Updated weights for policy 1, policy_version 1702287 (0.0007) [2023-12-27 17:26:57,327][20347] Updated weights for policy 0, policy_version 1703819 (0.0007) [2023-12-27 17:26:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 872087552. Throughput: 0: 10235.7, 1: 10206.5. Samples: 872081616. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:26:57,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:26:57,381][20347] Updated weights for policy 0, policy_version 1703829 (0.0010) [2023-12-27 17:26:57,438][20347] Updated weights for policy 0, policy_version 1703839 (0.0010) [2023-12-27 17:26:57,476][20381] Updated weights for policy 1, policy_version 1702297 (0.0006) [2023-12-27 17:26:57,530][20381] Updated weights for policy 1, policy_version 1702307 (0.0006) [2023-12-27 17:26:57,585][20381] Updated weights for policy 1, policy_version 1702317 (0.0005) [2023-12-27 17:26:58,109][20347] Updated weights for policy 0, policy_version 1703849 (0.0010) [2023-12-27 17:26:58,160][20347] Updated weights for policy 0, policy_version 1703859 (0.0010) [2023-12-27 17:26:58,226][20347] Updated weights for policy 0, policy_version 1703869 (0.0010) [2023-12-27 17:26:58,228][20381] Updated weights for policy 1, policy_version 1702327 (0.0007) [2023-12-27 17:26:58,286][20347] Updated weights for policy 0, policy_version 1703879 (0.0011) [2023-12-27 17:26:58,289][20381] Updated weights for policy 1, policy_version 1702337 (0.0008) [2023-12-27 17:26:58,378][20381] Updated weights for policy 1, policy_version 1702347 (0.0012) [2023-12-27 17:26:59,067][20347] Updated weights for policy 0, policy_version 1703889 (0.0008) [2023-12-27 17:26:59,130][20347] Updated weights for policy 0, policy_version 1703899 (0.0007) [2023-12-27 17:26:59,191][20347] Updated weights for policy 0, policy_version 1703909 (0.0007) [2023-12-27 17:26:59,234][20381] Updated weights for policy 1, policy_version 1702357 (0.0008) [2023-12-27 17:26:59,292][20381] Updated weights for policy 1, policy_version 1702367 (0.0008) [2023-12-27 17:26:59,339][20381] Updated weights for policy 1, policy_version 1702377 (0.0008) [2023-12-27 17:26:59,850][20347] Updated weights for policy 0, policy_version 1703919 (0.0009) [2023-12-27 17:26:59,909][20347] Updated weights for policy 0, policy_version 1703929 (0.0008) [2023-12-27 17:26:59,968][20347] Updated weights for policy 0, policy_version 1703939 (0.0008) [2023-12-27 17:27:00,066][20381] Updated weights for policy 1, policy_version 1702387 (0.0009) [2023-12-27 17:27:00,122][20381] Updated weights for policy 1, policy_version 1702397 (0.0011) [2023-12-27 17:27:00,177][20381] Updated weights for policy 1, policy_version 1702407 (0.0010) [2023-12-27 17:27:00,614][20347] Updated weights for policy 0, policy_version 1703949 (0.0007) [2023-12-27 17:27:00,661][20347] Updated weights for policy 0, policy_version 1703959 (0.0005) [2023-12-27 17:27:00,705][20347] Updated weights for policy 0, policy_version 1703969 (0.0005) [2023-12-27 17:27:00,867][20381] Updated weights for policy 1, policy_version 1702417 (0.0010) [2023-12-27 17:27:00,916][20381] Updated weights for policy 1, policy_version 1702427 (0.0007) [2023-12-27 17:27:00,971][20381] Updated weights for policy 1, policy_version 1702437 (0.0010) [2023-12-27 17:27:01,026][20381] Updated weights for policy 1, policy_version 1702447 (0.0008) [2023-12-27 17:27:01,416][20347] Updated weights for policy 0, policy_version 1703979 (0.0008) [2023-12-27 17:27:01,477][20347] Updated weights for policy 0, policy_version 1703989 (0.0008) [2023-12-27 17:27:01,535][20347] Updated weights for policy 0, policy_version 1703999 (0.0008) [2023-12-27 17:27:01,668][20381] Updated weights for policy 1, policy_version 1702457 (0.0007) [2023-12-27 17:27:01,735][20381] Updated weights for policy 1, policy_version 1702467 (0.0008) [2023-12-27 17:27:01,792][20381] Updated weights for policy 1, policy_version 1702477 (0.0009) [2023-12-27 17:27:02,218][20347] Updated weights for policy 0, policy_version 1704009 (0.0008) [2023-12-27 17:27:02,285][20347] Updated weights for policy 0, policy_version 1704019 (0.0007) [2023-12-27 17:27:02,348][20347] Updated weights for policy 0, policy_version 1704029 (0.0009) [2023-12-27 17:27:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 872185856. Throughput: 0: 10302.8, 1: 10149.5. Samples: 872204148. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:02,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 17:27:02,409][20381] Updated weights for policy 1, policy_version 1702487 (0.0007) [2023-12-27 17:27:02,416][20347] Updated weights for policy 0, policy_version 1704039 (0.0007) [2023-12-27 17:27:02,475][20381] Updated weights for policy 1, policy_version 1702497 (0.0005) [2023-12-27 17:27:02,539][20381] Updated weights for policy 1, policy_version 1702507 (0.0005) [2023-12-27 17:27:03,057][20347] Updated weights for policy 0, policy_version 1704049 (0.0010) [2023-12-27 17:27:03,120][20347] Updated weights for policy 0, policy_version 1704059 (0.0010) [2023-12-27 17:27:03,169][20381] Updated weights for policy 1, policy_version 1702517 (0.0007) [2023-12-27 17:27:03,202][20347] Updated weights for policy 0, policy_version 1704069 (0.0010) [2023-12-27 17:27:03,223][20381] Updated weights for policy 1, policy_version 1702527 (0.0005) [2023-12-27 17:27:03,283][20381] Updated weights for policy 1, policy_version 1702537 (0.0005) [2023-12-27 17:27:03,768][20347] Updated weights for policy 0, policy_version 1704079 (0.0007) [2023-12-27 17:27:03,816][20347] Updated weights for policy 0, policy_version 1704089 (0.0009) [2023-12-27 17:27:03,877][20347] Updated weights for policy 0, policy_version 1704099 (0.0011) [2023-12-27 17:27:03,880][20381] Updated weights for policy 1, policy_version 1702547 (0.0005) [2023-12-27 17:27:03,939][20381] Updated weights for policy 1, policy_version 1702557 (0.0006) [2023-12-27 17:27:04,001][20381] Updated weights for policy 1, policy_version 1702567 (0.0006) [2023-12-27 17:27:04,591][20347] Updated weights for policy 0, policy_version 1704109 (0.0010) [2023-12-27 17:27:04,640][20347] Updated weights for policy 0, policy_version 1704119 (0.0010) [2023-12-27 17:27:04,678][20381] Updated weights for policy 1, policy_version 1702577 (0.0006) [2023-12-27 17:27:04,685][20347] Updated weights for policy 0, policy_version 1704129 (0.0010) [2023-12-27 17:27:04,731][20381] Updated weights for policy 1, policy_version 1702587 (0.0006) [2023-12-27 17:27:04,788][20381] Updated weights for policy 1, policy_version 1702597 (0.0007) [2023-12-27 17:27:04,839][20381] Updated weights for policy 1, policy_version 1702607 (0.0007) [2023-12-27 17:27:05,392][20347] Updated weights for policy 0, policy_version 1704139 (0.0010) [2023-12-27 17:27:05,440][20347] Updated weights for policy 0, policy_version 1704149 (0.0010) [2023-12-27 17:27:05,488][20347] Updated weights for policy 0, policy_version 1704159 (0.0010) [2023-12-27 17:27:05,545][20381] Updated weights for policy 1, policy_version 1702617 (0.0007) [2023-12-27 17:27:05,595][20381] Updated weights for policy 1, policy_version 1702627 (0.0007) [2023-12-27 17:27:05,636][20381] Updated weights for policy 1, policy_version 1702637 (0.0007) [2023-12-27 17:27:06,115][20347] Updated weights for policy 0, policy_version 1704169 (0.0010) [2023-12-27 17:27:06,184][20347] Updated weights for policy 0, policy_version 1704179 (0.0011) [2023-12-27 17:27:06,244][20347] Updated weights for policy 0, policy_version 1704189 (0.0011) [2023-12-27 17:27:06,282][20381] Updated weights for policy 1, policy_version 1702647 (0.0009) [2023-12-27 17:27:06,306][20347] Updated weights for policy 0, policy_version 1704199 (0.0010) [2023-12-27 17:27:06,339][20381] Updated weights for policy 1, policy_version 1702657 (0.0010) [2023-12-27 17:27:06,402][20381] Updated weights for policy 1, policy_version 1702667 (0.0006) [2023-12-27 17:27:06,931][20347] Updated weights for policy 0, policy_version 1704209 (0.0010) [2023-12-27 17:27:06,988][20347] Updated weights for policy 0, policy_version 1704219 (0.0010) [2023-12-27 17:27:07,042][20381] Updated weights for policy 1, policy_version 1702677 (0.0008) [2023-12-27 17:27:07,045][20347] Updated weights for policy 0, policy_version 1704229 (0.0008) [2023-12-27 17:27:07,101][20381] Updated weights for policy 1, policy_version 1702687 (0.0006) [2023-12-27 17:27:07,149][20381] Updated weights for policy 1, policy_version 1702697 (0.0008) [2023-12-27 17:27:07,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 872300544. Throughput: 0: 10298.0, 1: 10168.6. Samples: 872267092. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:07,364][19170] Avg episode reward: [(0, '960.940'), (1, '1000.000')] [2023-12-27 17:27:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001702704_435937280.pth... [2023-12-27 17:27:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001704232_436363264.pth... [2023-12-27 17:27:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001701520_435634176.pth [2023-12-27 17:27:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001703048_436060160.pth [2023-12-27 17:27:07,635][20347] Updated weights for policy 0, policy_version 1704239 (0.0005) [2023-12-27 17:27:07,698][20347] Updated weights for policy 0, policy_version 1704249 (0.0005) [2023-12-27 17:27:07,729][20381] Updated weights for policy 1, policy_version 1702707 (0.0006) [2023-12-27 17:27:07,756][20347] Updated weights for policy 0, policy_version 1704259 (0.0008) [2023-12-27 17:27:07,792][20381] Updated weights for policy 1, policy_version 1702717 (0.0006) [2023-12-27 17:27:07,852][20381] Updated weights for policy 1, policy_version 1702727 (0.0006) [2023-12-27 17:27:08,346][20347] Updated weights for policy 0, policy_version 1704269 (0.0007) [2023-12-27 17:27:08,384][20291] KL-divergence is very high: 196.4955 [2023-12-27 17:27:08,408][20347] Updated weights for policy 0, policy_version 1704279 (0.0008) [2023-12-27 17:27:08,434][20291] KL-divergence is very high: 225.6175 [2023-12-27 17:27:08,470][20347] Updated weights for policy 0, policy_version 1704289 (0.0008) [2023-12-27 17:27:08,481][20291] KL-divergence is very high: 139.0794 [2023-12-27 17:27:08,484][20381] Updated weights for policy 1, policy_version 1702737 (0.0006) [2023-12-27 17:27:08,539][20381] Updated weights for policy 1, policy_version 1702747 (0.0010) [2023-12-27 17:27:08,600][20381] Updated weights for policy 1, policy_version 1702757 (0.0010) [2023-12-27 17:27:08,651][20381] Updated weights for policy 1, policy_version 1702767 (0.0010) [2023-12-27 17:27:09,100][20347] Updated weights for policy 0, policy_version 1704299 (0.0006) [2023-12-27 17:27:09,152][20347] Updated weights for policy 0, policy_version 1704309 (0.0005) [2023-12-27 17:27:09,209][20347] Updated weights for policy 0, policy_version 1704319 (0.0007) [2023-12-27 17:27:09,383][20381] Updated weights for policy 1, policy_version 1702777 (0.0009) [2023-12-27 17:27:09,447][20381] Updated weights for policy 1, policy_version 1702787 (0.0008) [2023-12-27 17:27:09,501][20381] Updated weights for policy 1, policy_version 1702797 (0.0008) [2023-12-27 17:27:09,932][20347] Updated weights for policy 0, policy_version 1704329 (0.0008) [2023-12-27 17:27:09,990][20347] Updated weights for policy 0, policy_version 1704339 (0.0008) [2023-12-27 17:27:10,046][20347] Updated weights for policy 0, policy_version 1704349 (0.0008) [2023-12-27 17:27:10,097][20347] Updated weights for policy 0, policy_version 1704359 (0.0007) [2023-12-27 17:27:10,229][20381] Updated weights for policy 1, policy_version 1702807 (0.0007) [2023-12-27 17:27:10,286][20381] Updated weights for policy 1, policy_version 1702817 (0.0008) [2023-12-27 17:27:10,341][20381] Updated weights for policy 1, policy_version 1702827 (0.0007) [2023-12-27 17:27:10,824][20347] Updated weights for policy 0, policy_version 1704369 (0.0007) [2023-12-27 17:27:10,879][20347] Updated weights for policy 0, policy_version 1704379 (0.0008) [2023-12-27 17:27:10,936][20347] Updated weights for policy 0, policy_version 1704389 (0.0007) [2023-12-27 17:27:11,013][20381] Updated weights for policy 1, policy_version 1702837 (0.0007) [2023-12-27 17:27:11,091][20381] Updated weights for policy 1, policy_version 1702847 (0.0008) [2023-12-27 17:27:11,152][20381] Updated weights for policy 1, policy_version 1702857 (0.0008) [2023-12-27 17:27:11,695][20347] Updated weights for policy 0, policy_version 1704399 (0.0008) [2023-12-27 17:27:11,753][20347] Updated weights for policy 0, policy_version 1704409 (0.0008) [2023-12-27 17:27:11,814][20347] Updated weights for policy 0, policy_version 1704419 (0.0007) [2023-12-27 17:27:11,821][20381] Updated weights for policy 1, policy_version 1702867 (0.0007) [2023-12-27 17:27:11,884][20381] Updated weights for policy 1, policy_version 1702877 (0.0006) [2023-12-27 17:27:11,937][20381] Updated weights for policy 1, policy_version 1702887 (0.0008) [2023-12-27 17:27:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 872398848. Throughput: 0: 10368.4, 1: 10175.1. Samples: 872392504. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:12,363][19170] Avg episode reward: [(0, '921.840'), (1, '1000.000')] [2023-12-27 17:27:12,534][20347] Updated weights for policy 0, policy_version 1704429 (0.0007) [2023-12-27 17:27:12,586][20347] Updated weights for policy 0, policy_version 1704439 (0.0006) [2023-12-27 17:27:12,588][20381] Updated weights for policy 1, policy_version 1702897 (0.0007) [2023-12-27 17:27:12,644][20347] Updated weights for policy 0, policy_version 1704449 (0.0007) [2023-12-27 17:27:12,651][20381] Updated weights for policy 1, policy_version 1702907 (0.0007) [2023-12-27 17:27:12,708][20381] Updated weights for policy 1, policy_version 1702917 (0.0007) [2023-12-27 17:27:12,766][20381] Updated weights for policy 1, policy_version 1702927 (0.0008) [2023-12-27 17:27:13,296][20347] Updated weights for policy 0, policy_version 1704459 (0.0008) [2023-12-27 17:27:13,350][20347] Updated weights for policy 0, policy_version 1704469 (0.0007) [2023-12-27 17:27:13,373][20291] KL-divergence is very high: 105.1853 [2023-12-27 17:27:13,409][20347] Updated weights for policy 0, policy_version 1704479 (0.0008) [2023-12-27 17:27:13,481][20381] Updated weights for policy 1, policy_version 1702937 (0.0008) [2023-12-27 17:27:13,542][20381] Updated weights for policy 1, policy_version 1702947 (0.0008) [2023-12-27 17:27:13,598][20381] Updated weights for policy 1, policy_version 1702957 (0.0008) [2023-12-27 17:27:14,056][20347] Updated weights for policy 0, policy_version 1704489 (0.0007) [2023-12-27 17:27:14,107][20347] Updated weights for policy 0, policy_version 1704499 (0.0007) [2023-12-27 17:27:14,161][20347] Updated weights for policy 0, policy_version 1704509 (0.0007) [2023-12-27 17:27:14,213][20347] Updated weights for policy 0, policy_version 1704519 (0.0008) [2023-12-27 17:27:14,291][20381] Updated weights for policy 1, policy_version 1702967 (0.0009) [2023-12-27 17:27:14,348][20381] Updated weights for policy 1, policy_version 1702977 (0.0007) [2023-12-27 17:27:14,415][20381] Updated weights for policy 1, policy_version 1702987 (0.0008) [2023-12-27 17:27:14,878][20347] Updated weights for policy 0, policy_version 1704529 (0.0008) [2023-12-27 17:27:14,941][20347] Updated weights for policy 0, policy_version 1704539 (0.0007) [2023-12-27 17:27:14,998][20347] Updated weights for policy 0, policy_version 1704549 (0.0008) [2023-12-27 17:27:15,134][20381] Updated weights for policy 1, policy_version 1702997 (0.0009) [2023-12-27 17:27:15,193][20381] Updated weights for policy 1, policy_version 1703007 (0.0008) [2023-12-27 17:27:15,257][20381] Updated weights for policy 1, policy_version 1703017 (0.0008) [2023-12-27 17:27:15,653][20347] Updated weights for policy 0, policy_version 1704559 (0.0008) [2023-12-27 17:27:15,706][20347] Updated weights for policy 0, policy_version 1704569 (0.0008) [2023-12-27 17:27:15,758][20347] Updated weights for policy 0, policy_version 1704579 (0.0006) [2023-12-27 17:27:15,967][20381] Updated weights for policy 1, policy_version 1703027 (0.0008) [2023-12-27 17:27:16,028][20381] Updated weights for policy 1, policy_version 1703037 (0.0008) [2023-12-27 17:27:16,079][20381] Updated weights for policy 1, policy_version 1703047 (0.0008) [2023-12-27 17:27:16,434][20347] Updated weights for policy 0, policy_version 1704589 (0.0006) [2023-12-27 17:27:16,493][20347] Updated weights for policy 0, policy_version 1704599 (0.0007) [2023-12-27 17:27:16,553][20347] Updated weights for policy 0, policy_version 1704609 (0.0007) [2023-12-27 17:27:16,758][20381] Updated weights for policy 1, policy_version 1703057 (0.0008) [2023-12-27 17:27:16,807][20381] Updated weights for policy 1, policy_version 1703067 (0.0007) [2023-12-27 17:27:16,852][20381] Updated weights for policy 1, policy_version 1703077 (0.0007) [2023-12-27 17:27:16,895][20381] Updated weights for policy 1, policy_version 1703087 (0.0008) [2023-12-27 17:27:17,225][20347] Updated weights for policy 0, policy_version 1704619 (0.0007) [2023-12-27 17:27:17,260][20291] KL-divergence is very high: 288.2452 [2023-12-27 17:27:17,278][20291] KL-divergence is very high: 257.8569 [2023-12-27 17:27:17,284][20347] Updated weights for policy 0, policy_version 1704629 (0.0008) [2023-12-27 17:27:17,305][20291] KL-divergence is very high: 422.5565 [2023-12-27 17:27:17,325][20291] KL-divergence is very high: 213.8284 [2023-12-27 17:27:17,347][20347] Updated weights for policy 0, policy_version 1704639 (0.0008) [2023-12-27 17:27:17,357][20291] KL-divergence is very high: 197.4871 [2023-12-27 17:27:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 872497152. Throughput: 0: 10303.4, 1: 10133.2. Samples: 872512736. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:17,363][19170] Avg episode reward: [(0, '931.570'), (1, '990.430')] [2023-12-27 17:27:17,631][20381] Updated weights for policy 1, policy_version 1703097 (0.0008) [2023-12-27 17:27:17,687][20381] Updated weights for policy 1, policy_version 1703107 (0.0007) [2023-12-27 17:27:17,741][20381] Updated weights for policy 1, policy_version 1703117 (0.0007) [2023-12-27 17:27:18,033][20347] Updated weights for policy 0, policy_version 1704649 (0.0007) [2023-12-27 17:27:18,091][20347] Updated weights for policy 0, policy_version 1704659 (0.0008) [2023-12-27 17:27:18,146][20347] Updated weights for policy 0, policy_version 1704669 (0.0008) [2023-12-27 17:27:18,204][20347] Updated weights for policy 0, policy_version 1704679 (0.0008) [2023-12-27 17:27:18,386][20381] Updated weights for policy 1, policy_version 1703127 (0.0007) [2023-12-27 17:27:18,453][20381] Updated weights for policy 1, policy_version 1703137 (0.0008) [2023-12-27 17:27:18,512][20381] Updated weights for policy 1, policy_version 1703147 (0.0008) [2023-12-27 17:27:18,886][20347] Updated weights for policy 0, policy_version 1704689 (0.0010) [2023-12-27 17:27:18,946][20347] Updated weights for policy 0, policy_version 1704699 (0.0011) [2023-12-27 17:27:19,005][20347] Updated weights for policy 0, policy_version 1704709 (0.0010) [2023-12-27 17:27:19,223][20381] Updated weights for policy 1, policy_version 1703157 (0.0008) [2023-12-27 17:27:19,286][20381] Updated weights for policy 1, policy_version 1703167 (0.0008) [2023-12-27 17:27:19,361][20381] Updated weights for policy 1, policy_version 1703177 (0.0008) [2023-12-27 17:27:19,718][20347] Updated weights for policy 0, policy_version 1704719 (0.0011) [2023-12-27 17:27:19,780][20347] Updated weights for policy 0, policy_version 1704729 (0.0011) [2023-12-27 17:27:19,853][20347] Updated weights for policy 0, policy_version 1704739 (0.0009) [2023-12-27 17:27:20,083][20381] Updated weights for policy 1, policy_version 1703187 (0.0008) [2023-12-27 17:27:20,146][20381] Updated weights for policy 1, policy_version 1703197 (0.0007) [2023-12-27 17:27:20,208][20381] Updated weights for policy 1, policy_version 1703207 (0.0006) [2023-12-27 17:27:20,569][20347] Updated weights for policy 0, policy_version 1704749 (0.0009) [2023-12-27 17:27:20,631][20347] Updated weights for policy 0, policy_version 1704759 (0.0010) [2023-12-27 17:27:20,695][20347] Updated weights for policy 0, policy_version 1704769 (0.0011) [2023-12-27 17:27:20,808][20381] Updated weights for policy 1, policy_version 1703217 (0.0007) [2023-12-27 17:27:20,872][20381] Updated weights for policy 1, policy_version 1703227 (0.0008) [2023-12-27 17:27:20,931][20381] Updated weights for policy 1, policy_version 1703237 (0.0008) [2023-12-27 17:27:20,993][20381] Updated weights for policy 1, policy_version 1703247 (0.0006) [2023-12-27 17:27:21,434][20347] Updated weights for policy 0, policy_version 1704779 (0.0010) [2023-12-27 17:27:21,493][20347] Updated weights for policy 0, policy_version 1704789 (0.0007) [2023-12-27 17:27:21,550][20347] Updated weights for policy 0, policy_version 1704799 (0.0007) [2023-12-27 17:27:21,711][20381] Updated weights for policy 1, policy_version 1703257 (0.0008) [2023-12-27 17:27:21,779][20381] Updated weights for policy 1, policy_version 1703267 (0.0007) [2023-12-27 17:27:21,838][20381] Updated weights for policy 1, policy_version 1703277 (0.0008) [2023-12-27 17:27:22,318][20347] Updated weights for policy 0, policy_version 1704809 (0.0008) [2023-12-27 17:27:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 872595456. Throughput: 0: 10240.6, 1: 10136.1. Samples: 872571568. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:22,363][19170] Avg episode reward: [(0, '970.630'), (1, '990.430')] [2023-12-27 17:27:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001703280_436084736.pth... [2023-12-27 17:27:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001702096_435781632.pth [2023-12-27 17:27:22,389][20347] Updated weights for policy 0, policy_version 1704819 (0.0007) [2023-12-27 17:27:22,443][20347] Updated weights for policy 0, policy_version 1704829 (0.0006) [2023-12-27 17:27:22,465][20381] Updated weights for policy 1, policy_version 1703287 (0.0006) [2023-12-27 17:27:22,497][20347] Updated weights for policy 0, policy_version 1704839 (0.0006) [2023-12-27 17:27:22,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001704840_436518912.pth... [2023-12-27 17:27:22,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001703624_436207616.pth [2023-12-27 17:27:22,526][20381] Updated weights for policy 1, policy_version 1703297 (0.0006) [2023-12-27 17:27:22,590][20381] Updated weights for policy 1, policy_version 1703307 (0.0008) [2023-12-27 17:27:23,208][20381] Updated weights for policy 1, policy_version 1703317 (0.0009) [2023-12-27 17:27:23,234][20347] Updated weights for policy 0, policy_version 1704849 (0.0010) [2023-12-27 17:27:23,264][20381] Updated weights for policy 1, policy_version 1703327 (0.0010) [2023-12-27 17:27:23,282][20347] Updated weights for policy 0, policy_version 1704859 (0.0010) [2023-12-27 17:27:23,312][20381] Updated weights for policy 1, policy_version 1703337 (0.0010) [2023-12-27 17:27:23,336][20347] Updated weights for policy 0, policy_version 1704869 (0.0010) [2023-12-27 17:27:23,966][20381] Updated weights for policy 1, policy_version 1703347 (0.0009) [2023-12-27 17:27:24,018][20347] Updated weights for policy 0, policy_version 1704879 (0.0008) [2023-12-27 17:27:24,019][20381] Updated weights for policy 1, policy_version 1703357 (0.0006) [2023-12-27 17:27:24,065][20347] Updated weights for policy 0, policy_version 1704889 (0.0008) [2023-12-27 17:27:24,066][20381] Updated weights for policy 1, policy_version 1703367 (0.0006) [2023-12-27 17:27:24,113][20347] Updated weights for policy 0, policy_version 1704899 (0.0008) [2023-12-27 17:27:24,783][20381] Updated weights for policy 1, policy_version 1703377 (0.0005) [2023-12-27 17:27:24,803][20347] Updated weights for policy 0, policy_version 1704909 (0.0009) [2023-12-27 17:27:24,835][20381] Updated weights for policy 1, policy_version 1703387 (0.0006) [2023-12-27 17:27:24,855][20347] Updated weights for policy 0, policy_version 1704919 (0.0010) [2023-12-27 17:27:24,885][20381] Updated weights for policy 1, policy_version 1703397 (0.0008) [2023-12-27 17:27:24,910][20347] Updated weights for policy 0, policy_version 1704929 (0.0006) [2023-12-27 17:27:24,940][20381] Updated weights for policy 1, policy_version 1703407 (0.0006) [2023-12-27 17:27:25,525][20381] Updated weights for policy 1, policy_version 1703417 (0.0005) [2023-12-27 17:27:25,575][20381] Updated weights for policy 1, policy_version 1703427 (0.0005) [2023-12-27 17:27:25,580][20347] Updated weights for policy 0, policy_version 1704939 (0.0006) [2023-12-27 17:27:25,628][20381] Updated weights for policy 1, policy_version 1703437 (0.0005) [2023-12-27 17:27:25,642][20347] Updated weights for policy 0, policy_version 1704949 (0.0007) [2023-12-27 17:27:25,701][20347] Updated weights for policy 0, policy_version 1704959 (0.0008) [2023-12-27 17:27:26,147][20381] Updated weights for policy 1, policy_version 1703447 (0.0005) [2023-12-27 17:27:26,198][20381] Updated weights for policy 1, policy_version 1703457 (0.0006) [2023-12-27 17:27:26,252][20381] Updated weights for policy 1, policy_version 1703467 (0.0009) [2023-12-27 17:27:26,313][20347] Updated weights for policy 0, policy_version 1704969 (0.0007) [2023-12-27 17:27:26,361][20347] Updated weights for policy 0, policy_version 1704979 (0.0005) [2023-12-27 17:27:26,421][20347] Updated weights for policy 0, policy_version 1704989 (0.0005) [2023-12-27 17:27:26,476][20347] Updated weights for policy 0, policy_version 1704999 (0.0005) [2023-12-27 17:27:26,918][20381] Updated weights for policy 1, policy_version 1703477 (0.0009) [2023-12-27 17:27:26,967][20381] Updated weights for policy 1, policy_version 1703487 (0.0008) [2023-12-27 17:27:27,025][20381] Updated weights for policy 1, policy_version 1703497 (0.0008) [2023-12-27 17:27:27,122][20347] Updated weights for policy 0, policy_version 1705009 (0.0007) [2023-12-27 17:27:27,171][20347] Updated weights for policy 0, policy_version 1705019 (0.0006) [2023-12-27 17:27:27,220][20347] Updated weights for policy 0, policy_version 1705029 (0.0007) [2023-12-27 17:27:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 872710144. Throughput: 0: 10219.7, 1: 10175.4. Samples: 872695716. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:27:27,608][20381] Updated weights for policy 1, policy_version 1703507 (0.0007) [2023-12-27 17:27:27,664][20381] Updated weights for policy 1, policy_version 1703517 (0.0005) [2023-12-27 17:27:27,709][20381] Updated weights for policy 1, policy_version 1703527 (0.0008) [2023-12-27 17:27:27,897][20347] Updated weights for policy 0, policy_version 1705039 (0.0008) [2023-12-27 17:27:27,949][20347] Updated weights for policy 0, policy_version 1705049 (0.0007) [2023-12-27 17:27:28,000][20347] Updated weights for policy 0, policy_version 1705059 (0.0007) [2023-12-27 17:27:28,295][20381] Updated weights for policy 1, policy_version 1703537 (0.0007) [2023-12-27 17:27:28,359][20381] Updated weights for policy 1, policy_version 1703547 (0.0007) [2023-12-27 17:27:28,418][20381] Updated weights for policy 1, policy_version 1703557 (0.0006) [2023-12-27 17:27:28,478][20381] Updated weights for policy 1, policy_version 1703567 (0.0006) [2023-12-27 17:27:28,685][20347] Updated weights for policy 0, policy_version 1705069 (0.0008) [2023-12-27 17:27:28,735][20347] Updated weights for policy 0, policy_version 1705079 (0.0008) [2023-12-27 17:27:28,791][20347] Updated weights for policy 0, policy_version 1705089 (0.0009) [2023-12-27 17:27:29,059][20381] Updated weights for policy 1, policy_version 1703577 (0.0008) [2023-12-27 17:27:29,112][20381] Updated weights for policy 1, policy_version 1703587 (0.0008) [2023-12-27 17:27:29,167][20381] Updated weights for policy 1, policy_version 1703597 (0.0008) [2023-12-27 17:27:29,550][20347] Updated weights for policy 0, policy_version 1705099 (0.0009) [2023-12-27 17:27:29,612][20347] Updated weights for policy 0, policy_version 1705109 (0.0010) [2023-12-27 17:27:29,670][20347] Updated weights for policy 0, policy_version 1705119 (0.0010) [2023-12-27 17:27:29,982][20381] Updated weights for policy 1, policy_version 1703607 (0.0007) [2023-12-27 17:27:30,048][20381] Updated weights for policy 1, policy_version 1703617 (0.0005) [2023-12-27 17:27:30,110][20381] Updated weights for policy 1, policy_version 1703627 (0.0006) [2023-12-27 17:27:30,290][20347] Updated weights for policy 0, policy_version 1705129 (0.0011) [2023-12-27 17:27:30,355][20347] Updated weights for policy 0, policy_version 1705139 (0.0009) [2023-12-27 17:27:30,418][20347] Updated weights for policy 0, policy_version 1705149 (0.0009) [2023-12-27 17:27:30,475][20347] Updated weights for policy 0, policy_version 1705159 (0.0010) [2023-12-27 17:27:30,738][20381] Updated weights for policy 1, policy_version 1703637 (0.0007) [2023-12-27 17:27:30,795][20381] Updated weights for policy 1, policy_version 1703647 (0.0008) [2023-12-27 17:27:30,852][20381] Updated weights for policy 1, policy_version 1703657 (0.0008) [2023-12-27 17:27:31,089][20347] Updated weights for policy 0, policy_version 1705169 (0.0008) [2023-12-27 17:27:31,155][20347] Updated weights for policy 0, policy_version 1705179 (0.0008) [2023-12-27 17:27:31,218][20347] Updated weights for policy 0, policy_version 1705189 (0.0008) [2023-12-27 17:27:31,615][20381] Updated weights for policy 1, policy_version 1703667 (0.0008) [2023-12-27 17:27:31,676][20381] Updated weights for policy 1, policy_version 1703677 (0.0008) [2023-12-27 17:27:31,735][20381] Updated weights for policy 1, policy_version 1703687 (0.0008) [2023-12-27 17:27:31,889][20347] Updated weights for policy 0, policy_version 1705199 (0.0008) [2023-12-27 17:27:31,944][20347] Updated weights for policy 0, policy_version 1705209 (0.0007) [2023-12-27 17:27:31,997][20347] Updated weights for policy 0, policy_version 1705219 (0.0008) [2023-12-27 17:27:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 872808448. Throughput: 0: 10243.6, 1: 10211.6. Samples: 872820684. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 17:27:32,436][20381] Updated weights for policy 1, policy_version 1703697 (0.0006) [2023-12-27 17:27:32,494][20381] Updated weights for policy 1, policy_version 1703707 (0.0008) [2023-12-27 17:27:32,542][20381] Updated weights for policy 1, policy_version 1703717 (0.0007) [2023-12-27 17:27:32,598][20381] Updated weights for policy 1, policy_version 1703727 (0.0007) [2023-12-27 17:27:32,744][20347] Updated weights for policy 0, policy_version 1705229 (0.0008) [2023-12-27 17:27:32,804][20347] Updated weights for policy 0, policy_version 1705239 (0.0008) [2023-12-27 17:27:32,859][20347] Updated weights for policy 0, policy_version 1705249 (0.0007) [2023-12-27 17:27:33,228][20381] Updated weights for policy 1, policy_version 1703737 (0.0009) [2023-12-27 17:27:33,283][20381] Updated weights for policy 1, policy_version 1703747 (0.0008) [2023-12-27 17:27:33,345][20381] Updated weights for policy 1, policy_version 1703757 (0.0008) [2023-12-27 17:27:33,485][20347] Updated weights for policy 0, policy_version 1705259 (0.0008) [2023-12-27 17:27:33,540][20347] Updated weights for policy 0, policy_version 1705269 (0.0005) [2023-12-27 17:27:33,596][20347] Updated weights for policy 0, policy_version 1705279 (0.0005) [2023-12-27 17:27:34,039][20381] Updated weights for policy 1, policy_version 1703767 (0.0006) [2023-12-27 17:27:34,087][20381] Updated weights for policy 1, policy_version 1703777 (0.0005) [2023-12-27 17:27:34,140][20381] Updated weights for policy 1, policy_version 1703787 (0.0005) [2023-12-27 17:27:34,152][20347] Updated weights for policy 0, policy_version 1705289 (0.0006) [2023-12-27 17:27:34,213][20347] Updated weights for policy 0, policy_version 1705299 (0.0007) [2023-12-27 17:27:34,274][20347] Updated weights for policy 0, policy_version 1705309 (0.0010) [2023-12-27 17:27:34,343][20347] Updated weights for policy 0, policy_version 1705319 (0.0010) [2023-12-27 17:27:34,776][20381] Updated weights for policy 1, policy_version 1703797 (0.0008) [2023-12-27 17:27:34,827][20381] Updated weights for policy 1, policy_version 1703807 (0.0007) [2023-12-27 17:27:34,891][20381] Updated weights for policy 1, policy_version 1703817 (0.0008) [2023-12-27 17:27:34,986][20347] Updated weights for policy 0, policy_version 1705329 (0.0006) [2023-12-27 17:27:35,037][20347] Updated weights for policy 0, policy_version 1705339 (0.0007) [2023-12-27 17:27:35,095][20347] Updated weights for policy 0, policy_version 1705349 (0.0007) [2023-12-27 17:27:35,620][20381] Updated weights for policy 1, policy_version 1703827 (0.0007) [2023-12-27 17:27:35,680][20381] Updated weights for policy 1, policy_version 1703837 (0.0007) [2023-12-27 17:27:35,731][20347] Updated weights for policy 0, policy_version 1705359 (0.0006) [2023-12-27 17:27:35,741][20381] Updated weights for policy 1, policy_version 1703847 (0.0007) [2023-12-27 17:27:35,784][20347] Updated weights for policy 0, policy_version 1705369 (0.0007) [2023-12-27 17:27:35,842][20347] Updated weights for policy 0, policy_version 1705379 (0.0008) [2023-12-27 17:27:36,318][20381] Updated weights for policy 1, policy_version 1703857 (0.0006) [2023-12-27 17:27:36,378][20381] Updated weights for policy 1, policy_version 1703867 (0.0009) [2023-12-27 17:27:36,438][20381] Updated weights for policy 1, policy_version 1703877 (0.0008) [2023-12-27 17:27:36,498][20381] Updated weights for policy 1, policy_version 1703887 (0.0008) [2023-12-27 17:27:36,544][20347] Updated weights for policy 0, policy_version 1705389 (0.0008) [2023-12-27 17:27:36,611][20347] Updated weights for policy 0, policy_version 1705399 (0.0008) [2023-12-27 17:27:36,682][20347] Updated weights for policy 0, policy_version 1705409 (0.0007) [2023-12-27 17:27:37,090][20381] Updated weights for policy 1, policy_version 1703897 (0.0009) [2023-12-27 17:27:37,152][20381] Updated weights for policy 1, policy_version 1703907 (0.0008) [2023-12-27 17:27:37,205][20381] Updated weights for policy 1, policy_version 1703917 (0.0008) [2023-12-27 17:27:37,301][20347] Updated weights for policy 0, policy_version 1705419 (0.0009) [2023-12-27 17:27:37,359][20347] Updated weights for policy 0, policy_version 1705429 (0.0010) [2023-12-27 17:27:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 872914944. Throughput: 0: 10282.2, 1: 10248.8. Samples: 872884128. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:37,363][19170] Avg episode reward: [(0, '990.200'), (1, '1000.000')] [2023-12-27 17:27:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001703920_436248576.pth... [2023-12-27 17:27:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001702704_435937280.pth [2023-12-27 17:27:37,409][20347] Updated weights for policy 0, policy_version 1705439 (0.0010) [2023-12-27 17:27:37,455][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001705448_436674560.pth... [2023-12-27 17:27:37,459][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001704232_436363264.pth [2023-12-27 17:27:37,937][20381] Updated weights for policy 1, policy_version 1703927 (0.0007) [2023-12-27 17:27:37,997][20381] Updated weights for policy 1, policy_version 1703937 (0.0008) [2023-12-27 17:27:38,039][20347] Updated weights for policy 0, policy_version 1705449 (0.0010) [2023-12-27 17:27:38,053][20381] Updated weights for policy 1, policy_version 1703947 (0.0007) [2023-12-27 17:27:38,091][20347] Updated weights for policy 0, policy_version 1705459 (0.0005) [2023-12-27 17:27:38,142][20347] Updated weights for policy 0, policy_version 1705469 (0.0005) [2023-12-27 17:27:38,197][20347] Updated weights for policy 0, policy_version 1705479 (0.0005) [2023-12-27 17:27:38,791][20381] Updated weights for policy 1, policy_version 1703957 (0.0008) [2023-12-27 17:27:38,848][20381] Updated weights for policy 1, policy_version 1703967 (0.0007) [2023-12-27 17:27:38,854][20347] Updated weights for policy 0, policy_version 1705489 (0.0005) [2023-12-27 17:27:38,907][20381] Updated weights for policy 1, policy_version 1703977 (0.0007) [2023-12-27 17:27:38,912][20347] Updated weights for policy 0, policy_version 1705499 (0.0006) [2023-12-27 17:27:38,966][20347] Updated weights for policy 0, policy_version 1705509 (0.0005) [2023-12-27 17:27:39,621][20381] Updated weights for policy 1, policy_version 1703987 (0.0008) [2023-12-27 17:27:39,650][20347] Updated weights for policy 0, policy_version 1705519 (0.0008) [2023-12-27 17:27:39,683][20381] Updated weights for policy 1, policy_version 1703997 (0.0008) [2023-12-27 17:27:39,708][20347] Updated weights for policy 0, policy_version 1705529 (0.0007) [2023-12-27 17:27:39,745][20381] Updated weights for policy 1, policy_version 1704007 (0.0007) [2023-12-27 17:27:39,762][20347] Updated weights for policy 0, policy_version 1705539 (0.0006) [2023-12-27 17:27:40,466][20347] Updated weights for policy 0, policy_version 1705549 (0.0007) [2023-12-27 17:27:40,483][20381] Updated weights for policy 1, policy_version 1704017 (0.0006) [2023-12-27 17:27:40,528][20347] Updated weights for policy 0, policy_version 1705559 (0.0007) [2023-12-27 17:27:40,543][20381] Updated weights for policy 1, policy_version 1704027 (0.0005) [2023-12-27 17:27:40,585][20347] Updated weights for policy 0, policy_version 1705569 (0.0008) [2023-12-27 17:27:40,602][20381] Updated weights for policy 1, policy_version 1704037 (0.0005) [2023-12-27 17:27:40,661][20381] Updated weights for policy 1, policy_version 1704047 (0.0005) [2023-12-27 17:27:41,266][20381] Updated weights for policy 1, policy_version 1704057 (0.0007) [2023-12-27 17:27:41,321][20347] Updated weights for policy 0, policy_version 1705579 (0.0007) [2023-12-27 17:27:41,327][20381] Updated weights for policy 1, policy_version 1704067 (0.0008) [2023-12-27 17:27:41,397][20381] Updated weights for policy 1, policy_version 1704077 (0.0008) [2023-12-27 17:27:41,398][20347] Updated weights for policy 0, policy_version 1705589 (0.0008) [2023-12-27 17:27:41,459][20347] Updated weights for policy 0, policy_version 1705599 (0.0008) [2023-12-27 17:27:42,090][20381] Updated weights for policy 1, policy_version 1704087 (0.0007) [2023-12-27 17:27:42,143][20381] Updated weights for policy 1, policy_version 1704097 (0.0008) [2023-12-27 17:27:42,196][20381] Updated weights for policy 1, policy_version 1704107 (0.0008) [2023-12-27 17:27:42,208][20347] Updated weights for policy 0, policy_version 1705609 (0.0009) [2023-12-27 17:27:42,275][20347] Updated weights for policy 0, policy_version 1705619 (0.0006) [2023-12-27 17:27:42,335][20347] Updated weights for policy 0, policy_version 1705629 (0.0007) [2023-12-27 17:27:42,363][19170] Fps is (10 sec: 20478.4, 60 sec: 20479.7, 300 sec: 20549.4). Total num frames: 873013248. Throughput: 0: 10287.2, 1: 10271.4. Samples: 873006768. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:42,364][19170] Avg episode reward: [(0, '980.400'), (1, '1000.000')] [2023-12-27 17:27:42,401][20347] Updated weights for policy 0, policy_version 1705639 (0.0009) [2023-12-27 17:27:42,865][20381] Updated weights for policy 1, policy_version 1704117 (0.0008) [2023-12-27 17:27:42,921][20381] Updated weights for policy 1, policy_version 1704127 (0.0008) [2023-12-27 17:27:42,975][20381] Updated weights for policy 1, policy_version 1704137 (0.0008) [2023-12-27 17:27:43,103][20347] Updated weights for policy 0, policy_version 1705649 (0.0008) [2023-12-27 17:27:43,153][20347] Updated weights for policy 0, policy_version 1705659 (0.0008) [2023-12-27 17:27:43,204][20347] Updated weights for policy 0, policy_version 1705669 (0.0008) [2023-12-27 17:27:43,682][20381] Updated weights for policy 1, policy_version 1704147 (0.0007) [2023-12-27 17:27:43,738][20381] Updated weights for policy 1, policy_version 1704157 (0.0008) [2023-12-27 17:27:43,793][20381] Updated weights for policy 1, policy_version 1704167 (0.0008) [2023-12-27 17:27:43,877][20347] Updated weights for policy 0, policy_version 1705679 (0.0008) [2023-12-27 17:27:43,939][20347] Updated weights for policy 0, policy_version 1705689 (0.0006) [2023-12-27 17:27:43,992][20347] Updated weights for policy 0, policy_version 1705699 (0.0005) [2023-12-27 17:27:44,517][20381] Updated weights for policy 1, policy_version 1704177 (0.0008) [2023-12-27 17:27:44,577][20381] Updated weights for policy 1, policy_version 1704187 (0.0008) [2023-12-27 17:27:44,630][20381] Updated weights for policy 1, policy_version 1704197 (0.0008) [2023-12-27 17:27:44,659][20347] Updated weights for policy 0, policy_version 1705709 (0.0007) [2023-12-27 17:27:44,687][20381] Updated weights for policy 1, policy_version 1704207 (0.0008) [2023-12-27 17:27:44,715][20347] Updated weights for policy 0, policy_version 1705719 (0.0007) [2023-12-27 17:27:44,767][20347] Updated weights for policy 0, policy_version 1705729 (0.0008) [2023-12-27 17:27:45,416][20381] Updated weights for policy 1, policy_version 1704217 (0.0007) [2023-12-27 17:27:45,435][20347] Updated weights for policy 0, policy_version 1705739 (0.0008) [2023-12-27 17:27:45,459][20381] Updated weights for policy 1, policy_version 1704227 (0.0007) [2023-12-27 17:27:45,487][20347] Updated weights for policy 0, policy_version 1705749 (0.0006) [2023-12-27 17:27:45,504][20381] Updated weights for policy 1, policy_version 1704237 (0.0005) [2023-12-27 17:27:45,538][20347] Updated weights for policy 0, policy_version 1705759 (0.0006) [2023-12-27 17:27:46,226][20381] Updated weights for policy 1, policy_version 1704247 (0.0007) [2023-12-27 17:27:46,236][20347] Updated weights for policy 0, policy_version 1705769 (0.0007) [2023-12-27 17:27:46,277][20381] Updated weights for policy 1, policy_version 1704257 (0.0006) [2023-12-27 17:27:46,299][20347] Updated weights for policy 0, policy_version 1705779 (0.0008) [2023-12-27 17:27:46,328][20381] Updated weights for policy 1, policy_version 1704267 (0.0008) [2023-12-27 17:27:46,357][20347] Updated weights for policy 0, policy_version 1705789 (0.0006) [2023-12-27 17:27:46,421][20347] Updated weights for policy 0, policy_version 1705799 (0.0008) [2023-12-27 17:27:47,065][20381] Updated weights for policy 1, policy_version 1704277 (0.0006) [2023-12-27 17:27:47,088][20347] Updated weights for policy 0, policy_version 1705809 (0.0010) [2023-12-27 17:27:47,115][20381] Updated weights for policy 1, policy_version 1704287 (0.0007) [2023-12-27 17:27:47,142][20347] Updated weights for policy 0, policy_version 1705819 (0.0010) [2023-12-27 17:27:47,167][20381] Updated weights for policy 1, policy_version 1704297 (0.0009) [2023-12-27 17:27:47,200][20347] Updated weights for policy 0, policy_version 1705829 (0.0007) [2023-12-27 17:27:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 873119744. Throughput: 0: 10235.4, 1: 10237.3. Samples: 873125424. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:47,363][19170] Avg episode reward: [(0, '961.000'), (1, '1000.000')] [2023-12-27 17:27:47,862][20347] Updated weights for policy 0, policy_version 1705839 (0.0006) [2023-12-27 17:27:47,884][20381] Updated weights for policy 1, policy_version 1704307 (0.0007) [2023-12-27 17:27:47,925][20347] Updated weights for policy 0, policy_version 1705849 (0.0005) [2023-12-27 17:27:47,940][20381] Updated weights for policy 1, policy_version 1704317 (0.0008) [2023-12-27 17:27:47,973][20347] Updated weights for policy 0, policy_version 1705859 (0.0005) [2023-12-27 17:27:47,998][20381] Updated weights for policy 1, policy_version 1704327 (0.0007) [2023-12-27 17:27:48,520][20347] Updated weights for policy 0, policy_version 1705869 (0.0006) [2023-12-27 17:27:48,578][20347] Updated weights for policy 0, policy_version 1705879 (0.0007) [2023-12-27 17:27:48,630][20291] KL-divergence is very high: 118.7737 [2023-12-27 17:27:48,642][20347] Updated weights for policy 0, policy_version 1705889 (0.0006) [2023-12-27 17:27:48,699][20381] Updated weights for policy 1, policy_version 1704337 (0.0007) [2023-12-27 17:27:48,766][20381] Updated weights for policy 1, policy_version 1704347 (0.0007) [2023-12-27 17:27:48,826][20381] Updated weights for policy 1, policy_version 1704357 (0.0008) [2023-12-27 17:27:48,887][20381] Updated weights for policy 1, policy_version 1704367 (0.0007) [2023-12-27 17:27:49,314][20347] Updated weights for policy 0, policy_version 1705899 (0.0007) [2023-12-27 17:27:49,378][20347] Updated weights for policy 0, policy_version 1705909 (0.0008) [2023-12-27 17:27:49,440][20347] Updated weights for policy 0, policy_version 1705919 (0.0008) [2023-12-27 17:27:49,555][20381] Updated weights for policy 1, policy_version 1704377 (0.0008) [2023-12-27 17:27:49,609][20381] Updated weights for policy 1, policy_version 1704387 (0.0008) [2023-12-27 17:27:49,665][20381] Updated weights for policy 1, policy_version 1704397 (0.0007) [2023-12-27 17:27:50,162][20347] Updated weights for policy 0, policy_version 1705929 (0.0008) [2023-12-27 17:27:50,214][20347] Updated weights for policy 0, policy_version 1705939 (0.0007) [2023-12-27 17:27:50,271][20347] Updated weights for policy 0, policy_version 1705949 (0.0007) [2023-12-27 17:27:50,331][20347] Updated weights for policy 0, policy_version 1705959 (0.0007) [2023-12-27 17:27:50,465][20381] Updated weights for policy 1, policy_version 1704407 (0.0008) [2023-12-27 17:27:50,519][20381] Updated weights for policy 1, policy_version 1704417 (0.0008) [2023-12-27 17:27:50,576][20381] Updated weights for policy 1, policy_version 1704427 (0.0008) [2023-12-27 17:27:51,063][20347] Updated weights for policy 0, policy_version 1705969 (0.0008) [2023-12-27 17:27:51,131][20347] Updated weights for policy 0, policy_version 1705979 (0.0008) [2023-12-27 17:27:51,190][20347] Updated weights for policy 0, policy_version 1705989 (0.0007) [2023-12-27 17:27:51,388][20381] Updated weights for policy 1, policy_version 1704437 (0.0008) [2023-12-27 17:27:51,446][20381] Updated weights for policy 1, policy_version 1704447 (0.0008) [2023-12-27 17:27:51,507][20381] Updated weights for policy 1, policy_version 1704457 (0.0009) [2023-12-27 17:27:51,905][20347] Updated weights for policy 0, policy_version 1705999 (0.0010) [2023-12-27 17:27:51,965][20347] Updated weights for policy 0, policy_version 1706009 (0.0010) [2023-12-27 17:27:52,021][20347] Updated weights for policy 0, policy_version 1706019 (0.0007) [2023-12-27 17:27:52,231][20381] Updated weights for policy 1, policy_version 1704467 (0.0008) [2023-12-27 17:27:52,292][20381] Updated weights for policy 1, policy_version 1704477 (0.0008) [2023-12-27 17:27:52,344][20381] Updated weights for policy 1, policy_version 1704487 (0.0007) [2023-12-27 17:27:52,362][19170] Fps is (10 sec: 19662.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 873209856. Throughput: 0: 10240.7, 1: 10200.0. Samples: 873186924. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:52,363][19170] Avg episode reward: [(0, '961.040'), (1, '1000.000')] [2023-12-27 17:27:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001706024_436822016.pth... [2023-12-27 17:27:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001704840_436518912.pth [2023-12-27 17:27:52,398][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001704496_436396032.pth... [2023-12-27 17:27:52,402][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001703280_436084736.pth [2023-12-27 17:27:52,735][20347] Updated weights for policy 0, policy_version 1706029 (0.0008) [2023-12-27 17:27:52,796][20347] Updated weights for policy 0, policy_version 1706039 (0.0008) [2023-12-27 17:27:52,849][20347] Updated weights for policy 0, policy_version 1706049 (0.0008) [2023-12-27 17:27:53,023][20381] Updated weights for policy 1, policy_version 1704497 (0.0008) [2023-12-27 17:27:53,086][20381] Updated weights for policy 1, policy_version 1704507 (0.0007) [2023-12-27 17:27:53,146][20381] Updated weights for policy 1, policy_version 1704517 (0.0009) [2023-12-27 17:27:53,205][20381] Updated weights for policy 1, policy_version 1704527 (0.0008) [2023-12-27 17:27:53,543][20347] Updated weights for policy 0, policy_version 1706059 (0.0007) [2023-12-27 17:27:53,604][20347] Updated weights for policy 0, policy_version 1706069 (0.0007) [2023-12-27 17:27:53,656][20347] Updated weights for policy 0, policy_version 1706079 (0.0007) [2023-12-27 17:27:53,871][20381] Updated weights for policy 1, policy_version 1704537 (0.0007) [2023-12-27 17:27:53,928][20381] Updated weights for policy 1, policy_version 1704547 (0.0008) [2023-12-27 17:27:53,983][20381] Updated weights for policy 1, policy_version 1704557 (0.0008) [2023-12-27 17:27:54,342][20347] Updated weights for policy 0, policy_version 1706089 (0.0007) [2023-12-27 17:27:54,394][20347] Updated weights for policy 0, policy_version 1706099 (0.0008) [2023-12-27 17:27:54,438][20347] Updated weights for policy 0, policy_version 1706109 (0.0007) [2023-12-27 17:27:54,486][20347] Updated weights for policy 0, policy_version 1706119 (0.0009) [2023-12-27 17:27:54,640][20381] Updated weights for policy 1, policy_version 1704567 (0.0009) [2023-12-27 17:27:54,705][20381] Updated weights for policy 1, policy_version 1704577 (0.0010) [2023-12-27 17:27:54,761][20381] Updated weights for policy 1, policy_version 1704587 (0.0011) [2023-12-27 17:27:55,239][20347] Updated weights for policy 0, policy_version 1706129 (0.0007) [2023-12-27 17:27:55,259][20291] KL-divergence is very high: 110.8200 [2023-12-27 17:27:55,293][20347] Updated weights for policy 0, policy_version 1706139 (0.0007) [2023-12-27 17:27:55,304][20291] KL-divergence is very high: 148.1579 [2023-12-27 17:27:55,345][20291] KL-divergence is very high: 113.3859 [2023-12-27 17:27:55,346][20347] Updated weights for policy 0, policy_version 1706149 (0.0007) [2023-12-27 17:27:55,471][20381] Updated weights for policy 1, policy_version 1704597 (0.0011) [2023-12-27 17:27:55,519][20381] Updated weights for policy 1, policy_version 1704607 (0.0010) [2023-12-27 17:27:55,576][20381] Updated weights for policy 1, policy_version 1704617 (0.0010) [2023-12-27 17:27:56,034][20347] Updated weights for policy 0, policy_version 1706159 (0.0007) [2023-12-27 17:27:56,084][20347] Updated weights for policy 0, policy_version 1706169 (0.0007) [2023-12-27 17:27:56,138][20347] Updated weights for policy 0, policy_version 1706179 (0.0007) [2023-12-27 17:27:56,256][20381] Updated weights for policy 1, policy_version 1704627 (0.0010) [2023-12-27 17:27:56,317][20381] Updated weights for policy 1, policy_version 1704637 (0.0010) [2023-12-27 17:27:56,375][20381] Updated weights for policy 1, policy_version 1704647 (0.0010) [2023-12-27 17:27:56,809][20347] Updated weights for policy 0, policy_version 1706189 (0.0009) [2023-12-27 17:27:56,859][20347] Updated weights for policy 0, policy_version 1706199 (0.0010) [2023-12-27 17:27:56,903][20347] Updated weights for policy 0, policy_version 1706209 (0.0010) [2023-12-27 17:27:57,013][20381] Updated weights for policy 1, policy_version 1704657 (0.0010) [2023-12-27 17:27:57,067][20381] Updated weights for policy 1, policy_version 1704667 (0.0005) [2023-12-27 17:27:57,126][20381] Updated weights for policy 1, policy_version 1704677 (0.0005) [2023-12-27 17:27:57,186][20381] Updated weights for policy 1, policy_version 1704687 (0.0006) [2023-12-27 17:27:57,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 873316352. Throughput: 0: 10144.3, 1: 10144.2. Samples: 873305484. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:27:57,363][19170] Avg episode reward: [(0, '960.890'), (1, '1000.000')] [2023-12-27 17:27:57,621][20347] Updated weights for policy 0, policy_version 1706219 (0.0009) [2023-12-27 17:27:57,669][20347] Updated weights for policy 0, policy_version 1706229 (0.0007) [2023-12-27 17:27:57,700][20381] Updated weights for policy 1, policy_version 1704697 (0.0010) [2023-12-27 17:27:57,725][20347] Updated weights for policy 0, policy_version 1706239 (0.0006) [2023-12-27 17:27:57,752][20381] Updated weights for policy 1, policy_version 1704707 (0.0011) [2023-12-27 17:27:57,800][20381] Updated weights for policy 1, policy_version 1704717 (0.0010) [2023-12-27 17:27:58,429][20347] Updated weights for policy 0, policy_version 1706249 (0.0006) [2023-12-27 17:27:58,485][20381] Updated weights for policy 1, policy_version 1704727 (0.0009) [2023-12-27 17:27:58,489][20347] Updated weights for policy 0, policy_version 1706259 (0.0008) [2023-12-27 17:27:58,546][20381] Updated weights for policy 1, policy_version 1704737 (0.0008) [2023-12-27 17:27:58,548][20347] Updated weights for policy 0, policy_version 1706269 (0.0008) [2023-12-27 17:27:58,603][20347] Updated weights for policy 0, policy_version 1706279 (0.0008) [2023-12-27 17:27:58,611][20381] Updated weights for policy 1, policy_version 1704747 (0.0007) [2023-12-27 17:27:59,311][20381] Updated weights for policy 1, policy_version 1704757 (0.0008) [2023-12-27 17:27:59,368][20381] Updated weights for policy 1, policy_version 1704767 (0.0009) [2023-12-27 17:27:59,429][20381] Updated weights for policy 1, policy_version 1704777 (0.0008) [2023-12-27 17:27:59,447][20347] Updated weights for policy 0, policy_version 1706289 (0.0006) [2023-12-27 17:27:59,506][20347] Updated weights for policy 0, policy_version 1706299 (0.0007) [2023-12-27 17:27:59,568][20347] Updated weights for policy 0, policy_version 1706309 (0.0008) [2023-12-27 17:28:00,116][20381] Updated weights for policy 1, policy_version 1704787 (0.0008) [2023-12-27 17:28:00,178][20381] Updated weights for policy 1, policy_version 1704797 (0.0007) [2023-12-27 17:28:00,209][20347] Updated weights for policy 0, policy_version 1706319 (0.0008) [2023-12-27 17:28:00,242][20381] Updated weights for policy 1, policy_version 1704807 (0.0008) [2023-12-27 17:28:00,265][20347] Updated weights for policy 0, policy_version 1706329 (0.0006) [2023-12-27 17:28:00,328][20347] Updated weights for policy 0, policy_version 1706339 (0.0006) [2023-12-27 17:28:00,891][20347] Updated weights for policy 0, policy_version 1706349 (0.0007) [2023-12-27 17:28:00,940][20381] Updated weights for policy 1, policy_version 1704817 (0.0008) [2023-12-27 17:28:00,955][20347] Updated weights for policy 0, policy_version 1706359 (0.0008) [2023-12-27 17:28:00,982][20381] Updated weights for policy 1, policy_version 1704827 (0.0009) [2023-12-27 17:28:01,018][20347] Updated weights for policy 0, policy_version 1706369 (0.0007) [2023-12-27 17:28:01,045][20381] Updated weights for policy 1, policy_version 1704837 (0.0006) [2023-12-27 17:28:01,107][20381] Updated weights for policy 1, policy_version 1704847 (0.0008) [2023-12-27 17:28:01,697][20347] Updated weights for policy 0, policy_version 1706379 (0.0008) [2023-12-27 17:28:01,763][20347] Updated weights for policy 0, policy_version 1706389 (0.0007) [2023-12-27 17:28:01,820][20347] Updated weights for policy 0, policy_version 1706399 (0.0006) [2023-12-27 17:28:01,823][20381] Updated weights for policy 1, policy_version 1704857 (0.0009) [2023-12-27 17:28:01,876][20381] Updated weights for policy 1, policy_version 1704867 (0.0010) [2023-12-27 17:28:01,935][20381] Updated weights for policy 1, policy_version 1704877 (0.0010) [2023-12-27 17:28:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 873414656. Throughput: 0: 10118.0, 1: 10201.3. Samples: 873427100. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:02,363][19170] Avg episode reward: [(0, '941.310'), (1, '1000.000')] [2023-12-27 17:28:02,471][20347] Updated weights for policy 0, policy_version 1706409 (0.0006) [2023-12-27 17:28:02,527][20347] Updated weights for policy 0, policy_version 1706419 (0.0007) [2023-12-27 17:28:02,574][20347] Updated weights for policy 0, policy_version 1706429 (0.0005) [2023-12-27 17:28:02,575][20381] Updated weights for policy 1, policy_version 1704887 (0.0009) [2023-12-27 17:28:02,619][20347] Updated weights for policy 0, policy_version 1706439 (0.0005) [2023-12-27 17:28:02,634][20381] Updated weights for policy 1, policy_version 1704897 (0.0007) [2023-12-27 17:28:02,695][20381] Updated weights for policy 1, policy_version 1704907 (0.0008) [2023-12-27 17:28:03,218][20347] Updated weights for policy 0, policy_version 1706449 (0.0006) [2023-12-27 17:28:03,273][20347] Updated weights for policy 0, policy_version 1706459 (0.0009) [2023-12-27 17:28:03,326][20347] Updated weights for policy 0, policy_version 1706469 (0.0008) [2023-12-27 17:28:03,351][20381] Updated weights for policy 1, policy_version 1704917 (0.0007) [2023-12-27 17:28:03,408][20381] Updated weights for policy 1, policy_version 1704927 (0.0008) [2023-12-27 17:28:03,454][20381] Updated weights for policy 1, policy_version 1704937 (0.0005) [2023-12-27 17:28:04,015][20381] Updated weights for policy 1, policy_version 1704947 (0.0006) [2023-12-27 17:28:04,034][20347] Updated weights for policy 0, policy_version 1706479 (0.0009) [2023-12-27 17:28:04,077][20381] Updated weights for policy 1, policy_version 1704957 (0.0010) [2023-12-27 17:28:04,091][20347] Updated weights for policy 0, policy_version 1706489 (0.0008) [2023-12-27 17:28:04,139][20381] Updated weights for policy 1, policy_version 1704967 (0.0006) [2023-12-27 17:28:04,145][20347] Updated weights for policy 0, policy_version 1706499 (0.0007) [2023-12-27 17:28:04,705][20381] Updated weights for policy 1, policy_version 1704977 (0.0006) [2023-12-27 17:28:04,767][20381] Updated weights for policy 1, policy_version 1704987 (0.0008) [2023-12-27 17:28:04,803][20347] Updated weights for policy 0, policy_version 1706509 (0.0006) [2023-12-27 17:28:04,825][20381] Updated weights for policy 1, policy_version 1704997 (0.0006) [2023-12-27 17:28:04,856][20347] Updated weights for policy 0, policy_version 1706519 (0.0005) [2023-12-27 17:28:04,879][20381] Updated weights for policy 1, policy_version 1705007 (0.0005) [2023-12-27 17:28:04,907][20347] Updated weights for policy 0, policy_version 1706529 (0.0006) [2023-12-27 17:28:05,506][20347] Updated weights for policy 0, policy_version 1706539 (0.0009) [2023-12-27 17:28:05,524][20381] Updated weights for policy 1, policy_version 1705017 (0.0009) [2023-12-27 17:28:05,555][20347] Updated weights for policy 0, policy_version 1706549 (0.0010) [2023-12-27 17:28:05,576][20381] Updated weights for policy 1, policy_version 1705027 (0.0005) [2023-12-27 17:28:05,604][20347] Updated weights for policy 0, policy_version 1706559 (0.0010) [2023-12-27 17:28:05,625][20381] Updated weights for policy 1, policy_version 1705037 (0.0007) [2023-12-27 17:28:06,296][20381] Updated weights for policy 1, policy_version 1705047 (0.0008) [2023-12-27 17:28:06,344][20347] Updated weights for policy 0, policy_version 1706569 (0.0010) [2023-12-27 17:28:06,361][20381] Updated weights for policy 1, policy_version 1705057 (0.0006) [2023-12-27 17:28:06,400][20347] Updated weights for policy 0, policy_version 1706579 (0.0006) [2023-12-27 17:28:06,427][20381] Updated weights for policy 1, policy_version 1705067 (0.0006) [2023-12-27 17:28:06,463][20347] Updated weights for policy 0, policy_version 1706589 (0.0008) [2023-12-27 17:28:06,520][20347] Updated weights for policy 0, policy_version 1706599 (0.0007) [2023-12-27 17:28:07,079][20381] Updated weights for policy 1, policy_version 1705077 (0.0008) [2023-12-27 17:28:07,137][20381] Updated weights for policy 1, policy_version 1705087 (0.0006) [2023-12-27 17:28:07,178][20347] Updated weights for policy 0, policy_version 1706609 (0.0007) [2023-12-27 17:28:07,192][20381] Updated weights for policy 1, policy_version 1705097 (0.0005) [2023-12-27 17:28:07,237][20347] Updated weights for policy 0, policy_version 1706619 (0.0007) [2023-12-27 17:28:07,293][20347] Updated weights for policy 0, policy_version 1706629 (0.0007) [2023-12-27 17:28:07,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 873529344. Throughput: 0: 10171.5, 1: 10297.4. Samples: 873492668. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:07,363][19170] Avg episode reward: [(0, '960.890'), (1, '1000.000')] [2023-12-27 17:28:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001705104_436551680.pth... [2023-12-27 17:28:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001706632_436977664.pth... [2023-12-27 17:28:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001703920_436248576.pth [2023-12-27 17:28:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001705448_436674560.pth [2023-12-27 17:28:07,860][20381] Updated weights for policy 1, policy_version 1705107 (0.0006) [2023-12-27 17:28:07,913][20347] Updated weights for policy 0, policy_version 1706639 (0.0006) [2023-12-27 17:28:07,915][20381] Updated weights for policy 1, policy_version 1705117 (0.0007) [2023-12-27 17:28:07,968][20347] Updated weights for policy 0, policy_version 1706649 (0.0005) [2023-12-27 17:28:07,971][20381] Updated weights for policy 1, policy_version 1705127 (0.0006) [2023-12-27 17:28:08,026][20347] Updated weights for policy 0, policy_version 1706659 (0.0007) [2023-12-27 17:28:08,628][20381] Updated weights for policy 1, policy_version 1705137 (0.0007) [2023-12-27 17:28:08,638][20347] Updated weights for policy 0, policy_version 1706669 (0.0006) [2023-12-27 17:28:08,684][20381] Updated weights for policy 1, policy_version 1705147 (0.0006) [2023-12-27 17:28:08,699][20347] Updated weights for policy 0, policy_version 1706679 (0.0007) [2023-12-27 17:28:08,741][20381] Updated weights for policy 1, policy_version 1705157 (0.0006) [2023-12-27 17:28:08,764][20347] Updated weights for policy 0, policy_version 1706689 (0.0008) [2023-12-27 17:28:08,797][20381] Updated weights for policy 1, policy_version 1705167 (0.0007) [2023-12-27 17:28:09,460][20347] Updated weights for policy 0, policy_version 1706699 (0.0007) [2023-12-27 17:28:09,504][20381] Updated weights for policy 1, policy_version 1705177 (0.0006) [2023-12-27 17:28:09,514][20347] Updated weights for policy 0, policy_version 1706709 (0.0007) [2023-12-27 17:28:09,563][20381] Updated weights for policy 1, policy_version 1705187 (0.0007) [2023-12-27 17:28:09,568][20347] Updated weights for policy 0, policy_version 1706719 (0.0006) [2023-12-27 17:28:09,625][20381] Updated weights for policy 1, policy_version 1705197 (0.0006) [2023-12-27 17:28:10,256][20347] Updated weights for policy 0, policy_version 1706729 (0.0009) [2023-12-27 17:28:10,315][20347] Updated weights for policy 0, policy_version 1706739 (0.0007) [2023-12-27 17:28:10,358][20381] Updated weights for policy 1, policy_version 1705207 (0.0007) [2023-12-27 17:28:10,374][20347] Updated weights for policy 0, policy_version 1706749 (0.0008) [2023-12-27 17:28:10,415][20381] Updated weights for policy 1, policy_version 1705217 (0.0006) [2023-12-27 17:28:10,432][20347] Updated weights for policy 0, policy_version 1706759 (0.0006) [2023-12-27 17:28:10,474][20381] Updated weights for policy 1, policy_version 1705227 (0.0007) [2023-12-27 17:28:11,125][20347] Updated weights for policy 0, policy_version 1706769 (0.0010) [2023-12-27 17:28:11,195][20347] Updated weights for policy 0, policy_version 1706779 (0.0009) [2023-12-27 17:28:11,207][20381] Updated weights for policy 1, policy_version 1705237 (0.0008) [2023-12-27 17:28:11,255][20347] Updated weights for policy 0, policy_version 1706789 (0.0008) [2023-12-27 17:28:11,267][20381] Updated weights for policy 1, policy_version 1705247 (0.0007) [2023-12-27 17:28:11,332][20381] Updated weights for policy 1, policy_version 1705257 (0.0009) [2023-12-27 17:28:11,968][20381] Updated weights for policy 1, policy_version 1705267 (0.0008) [2023-12-27 17:28:12,021][20347] Updated weights for policy 0, policy_version 1706799 (0.0005) [2023-12-27 17:28:12,023][20381] Updated weights for policy 1, policy_version 1705277 (0.0007) [2023-12-27 17:28:12,077][20347] Updated weights for policy 0, policy_version 1706809 (0.0006) [2023-12-27 17:28:12,078][20381] Updated weights for policy 1, policy_version 1705287 (0.0007) [2023-12-27 17:28:12,134][20347] Updated weights for policy 0, policy_version 1706819 (0.0006) [2023-12-27 17:28:12,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 873627648. Throughput: 0: 10248.9, 1: 10201.0. Samples: 873615960. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:12,363][19170] Avg episode reward: [(0, '970.660'), (1, '1000.000')] [2023-12-27 17:28:12,765][20381] Updated weights for policy 1, policy_version 1705297 (0.0007) [2023-12-27 17:28:12,831][20381] Updated weights for policy 1, policy_version 1705307 (0.0006) [2023-12-27 17:28:12,850][20347] Updated weights for policy 0, policy_version 1706829 (0.0008) [2023-12-27 17:28:12,887][20381] Updated weights for policy 1, policy_version 1705317 (0.0008) [2023-12-27 17:28:12,900][20347] Updated weights for policy 0, policy_version 1706839 (0.0006) [2023-12-27 17:28:12,940][20381] Updated weights for policy 1, policy_version 1705327 (0.0006) [2023-12-27 17:28:12,953][20347] Updated weights for policy 0, policy_version 1706849 (0.0006) [2023-12-27 17:28:13,575][20347] Updated weights for policy 0, policy_version 1706859 (0.0008) [2023-12-27 17:28:13,623][20381] Updated weights for policy 1, policy_version 1705337 (0.0007) [2023-12-27 17:28:13,625][20347] Updated weights for policy 0, policy_version 1706869 (0.0006) [2023-12-27 17:28:13,677][20347] Updated weights for policy 0, policy_version 1706879 (0.0005) [2023-12-27 17:28:13,683][20381] Updated weights for policy 1, policy_version 1705347 (0.0008) [2023-12-27 17:28:13,729][20381] Updated weights for policy 1, policy_version 1705357 (0.0007) [2023-12-27 17:28:14,342][20347] Updated weights for policy 0, policy_version 1706889 (0.0005) [2023-12-27 17:28:14,394][20347] Updated weights for policy 0, policy_version 1706899 (0.0007) [2023-12-27 17:28:14,416][20381] Updated weights for policy 1, policy_version 1705367 (0.0006) [2023-12-27 17:28:14,446][20347] Updated weights for policy 0, policy_version 1706909 (0.0007) [2023-12-27 17:28:14,465][20381] Updated weights for policy 1, policy_version 1705377 (0.0008) [2023-12-27 17:28:14,500][20347] Updated weights for policy 0, policy_version 1706919 (0.0007) [2023-12-27 17:28:14,514][20381] Updated weights for policy 1, policy_version 1705387 (0.0006) [2023-12-27 17:28:15,110][20347] Updated weights for policy 0, policy_version 1706929 (0.0006) [2023-12-27 17:28:15,168][20347] Updated weights for policy 0, policy_version 1706939 (0.0006) [2023-12-27 17:28:15,220][20347] Updated weights for policy 0, policy_version 1706949 (0.0006) [2023-12-27 17:28:15,225][20381] Updated weights for policy 1, policy_version 1705397 (0.0007) [2023-12-27 17:28:15,280][20381] Updated weights for policy 1, policy_version 1705407 (0.0007) [2023-12-27 17:28:15,334][20381] Updated weights for policy 1, policy_version 1705417 (0.0008) [2023-12-27 17:28:15,885][20347] Updated weights for policy 0, policy_version 1706959 (0.0007) [2023-12-27 17:28:15,941][20347] Updated weights for policy 0, policy_version 1706969 (0.0008) [2023-12-27 17:28:15,984][20381] Updated weights for policy 1, policy_version 1705427 (0.0007) [2023-12-27 17:28:15,999][20347] Updated weights for policy 0, policy_version 1706979 (0.0007) [2023-12-27 17:28:16,046][20381] Updated weights for policy 1, policy_version 1705437 (0.0006) [2023-12-27 17:28:16,108][20381] Updated weights for policy 1, policy_version 1705447 (0.0006) [2023-12-27 17:28:16,655][20347] Updated weights for policy 0, policy_version 1706989 (0.0008) [2023-12-27 17:28:16,667][20381] Updated weights for policy 1, policy_version 1705457 (0.0005) [2023-12-27 17:28:16,703][20347] Updated weights for policy 0, policy_version 1706999 (0.0006) [2023-12-27 17:28:16,725][20381] Updated weights for policy 1, policy_version 1705467 (0.0006) [2023-12-27 17:28:16,753][20347] Updated weights for policy 0, policy_version 1707009 (0.0008) [2023-12-27 17:28:16,785][20381] Updated weights for policy 1, policy_version 1705477 (0.0005) [2023-12-27 17:28:16,844][20381] Updated weights for policy 1, policy_version 1705487 (0.0006) [2023-12-27 17:28:17,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 873725952. Throughput: 0: 10231.6, 1: 10196.8. Samples: 873739964. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:17,364][19170] Avg episode reward: [(0, '980.430'), (1, '1000.000')] [2023-12-27 17:28:17,427][20347] Updated weights for policy 0, policy_version 1707019 (0.0008) [2023-12-27 17:28:17,481][20381] Updated weights for policy 1, policy_version 1705497 (0.0005) [2023-12-27 17:28:17,483][20347] Updated weights for policy 0, policy_version 1707029 (0.0010) [2023-12-27 17:28:17,539][20347] Updated weights for policy 0, policy_version 1707039 (0.0010) [2023-12-27 17:28:17,544][20381] Updated weights for policy 1, policy_version 1705507 (0.0005) [2023-12-27 17:28:17,601][20381] Updated weights for policy 1, policy_version 1705517 (0.0006) [2023-12-27 17:28:18,163][20347] Updated weights for policy 0, policy_version 1707049 (0.0010) [2023-12-27 17:28:18,222][20347] Updated weights for policy 0, policy_version 1707059 (0.0006) [2023-12-27 17:28:18,265][20381] Updated weights for policy 1, policy_version 1705527 (0.0007) [2023-12-27 17:28:18,275][20347] Updated weights for policy 0, policy_version 1707069 (0.0005) [2023-12-27 17:28:18,317][20381] Updated weights for policy 1, policy_version 1705537 (0.0008) [2023-12-27 17:28:18,335][20347] Updated weights for policy 0, policy_version 1707079 (0.0007) [2023-12-27 17:28:18,384][20381] Updated weights for policy 1, policy_version 1705547 (0.0008) [2023-12-27 17:28:18,960][20347] Updated weights for policy 0, policy_version 1707089 (0.0005) [2023-12-27 17:28:19,019][20347] Updated weights for policy 0, policy_version 1707099 (0.0005) [2023-12-27 17:28:19,076][20347] Updated weights for policy 0, policy_version 1707109 (0.0006) [2023-12-27 17:28:19,127][20381] Updated weights for policy 1, policy_version 1705557 (0.0009) [2023-12-27 17:28:19,174][20381] Updated weights for policy 1, policy_version 1705567 (0.0008) [2023-12-27 17:28:19,228][20381] Updated weights for policy 1, policy_version 1705577 (0.0006) [2023-12-27 17:28:19,752][20347] Updated weights for policy 0, policy_version 1707119 (0.0009) [2023-12-27 17:28:19,812][20347] Updated weights for policy 0, policy_version 1707129 (0.0011) [2023-12-27 17:28:19,880][20347] Updated weights for policy 0, policy_version 1707139 (0.0011) [2023-12-27 17:28:19,954][20381] Updated weights for policy 1, policy_version 1705587 (0.0007) [2023-12-27 17:28:20,013][20381] Updated weights for policy 1, policy_version 1705597 (0.0009) [2023-12-27 17:28:20,074][20381] Updated weights for policy 1, policy_version 1705607 (0.0007) [2023-12-27 17:28:20,574][20347] Updated weights for policy 0, policy_version 1707149 (0.0011) [2023-12-27 17:28:20,631][20347] Updated weights for policy 0, policy_version 1707159 (0.0011) [2023-12-27 17:28:20,686][20347] Updated weights for policy 0, policy_version 1707169 (0.0010) [2023-12-27 17:28:20,787][20381] Updated weights for policy 1, policy_version 1705617 (0.0006) [2023-12-27 17:28:20,852][20381] Updated weights for policy 1, policy_version 1705627 (0.0009) [2023-12-27 17:28:20,909][20381] Updated weights for policy 1, policy_version 1705637 (0.0006) [2023-12-27 17:28:20,975][20381] Updated weights for policy 1, policy_version 1705647 (0.0008) [2023-12-27 17:28:21,404][20347] Updated weights for policy 0, policy_version 1707179 (0.0010) [2023-12-27 17:28:21,461][20347] Updated weights for policy 0, policy_version 1707189 (0.0011) [2023-12-27 17:28:21,524][20347] Updated weights for policy 0, policy_version 1707199 (0.0011) [2023-12-27 17:28:21,660][20381] Updated weights for policy 1, policy_version 1705657 (0.0008) [2023-12-27 17:28:21,722][20381] Updated weights for policy 1, policy_version 1705667 (0.0008) [2023-12-27 17:28:21,787][20381] Updated weights for policy 1, policy_version 1705677 (0.0008) [2023-12-27 17:28:22,246][20347] Updated weights for policy 0, policy_version 1707209 (0.0010) [2023-12-27 17:28:22,313][20347] Updated weights for policy 0, policy_version 1707219 (0.0007) [2023-12-27 17:28:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 873824256. Throughput: 0: 10217.5, 1: 10162.6. Samples: 873801236. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:22,363][19170] Avg episode reward: [(0, '980.420'), (1, '1000.000')] [2023-12-27 17:28:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001705680_436699136.pth... [2023-12-27 17:28:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001704496_436396032.pth [2023-12-27 17:28:22,383][20347] Updated weights for policy 0, policy_version 1707229 (0.0007) [2023-12-27 17:28:22,445][20347] Updated weights for policy 0, policy_version 1707239 (0.0006) [2023-12-27 17:28:22,450][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001707240_437133312.pth... [2023-12-27 17:28:22,453][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001706024_436822016.pth [2023-12-27 17:28:22,503][20381] Updated weights for policy 1, policy_version 1705687 (0.0008) [2023-12-27 17:28:22,566][20381] Updated weights for policy 1, policy_version 1705697 (0.0010) [2023-12-27 17:28:22,630][20381] Updated weights for policy 1, policy_version 1705707 (0.0011) [2023-12-27 17:28:23,114][20347] Updated weights for policy 0, policy_version 1707249 (0.0005) [2023-12-27 17:28:23,178][20347] Updated weights for policy 0, policy_version 1707259 (0.0007) [2023-12-27 17:28:23,229][20347] Updated weights for policy 0, policy_version 1707269 (0.0007) [2023-12-27 17:28:23,267][20381] Updated weights for policy 1, policy_version 1705717 (0.0010) [2023-12-27 17:28:23,326][20381] Updated weights for policy 1, policy_version 1705727 (0.0010) [2023-12-27 17:28:23,379][20381] Updated weights for policy 1, policy_version 1705737 (0.0010) [2023-12-27 17:28:23,903][20347] Updated weights for policy 0, policy_version 1707279 (0.0008) [2023-12-27 17:28:23,962][20347] Updated weights for policy 0, policy_version 1707289 (0.0008) [2023-12-27 17:28:24,007][20381] Updated weights for policy 1, policy_version 1705747 (0.0009) [2023-12-27 17:28:24,020][20347] Updated weights for policy 0, policy_version 1707299 (0.0007) [2023-12-27 17:28:24,061][20381] Updated weights for policy 1, policy_version 1705757 (0.0006) [2023-12-27 17:28:24,118][20381] Updated weights for policy 1, policy_version 1705767 (0.0007) [2023-12-27 17:28:24,707][20347] Updated weights for policy 0, policy_version 1707309 (0.0008) [2023-12-27 17:28:24,764][20347] Updated weights for policy 0, policy_version 1707319 (0.0010) [2023-12-27 17:28:24,775][20381] Updated weights for policy 1, policy_version 1705777 (0.0008) [2023-12-27 17:28:24,824][20347] Updated weights for policy 0, policy_version 1707329 (0.0008) [2023-12-27 17:28:24,826][20381] Updated weights for policy 1, policy_version 1705787 (0.0005) [2023-12-27 17:28:24,881][20381] Updated weights for policy 1, policy_version 1705797 (0.0006) [2023-12-27 17:28:24,939][20381] Updated weights for policy 1, policy_version 1705807 (0.0008) [2023-12-27 17:28:25,449][20347] Updated weights for policy 0, policy_version 1707339 (0.0009) [2023-12-27 17:28:25,509][20347] Updated weights for policy 0, policy_version 1707349 (0.0005) [2023-12-27 17:28:25,561][20347] Updated weights for policy 0, policy_version 1707359 (0.0005) [2023-12-27 17:28:25,593][20381] Updated weights for policy 1, policy_version 1705817 (0.0008) [2023-12-27 17:28:25,648][20381] Updated weights for policy 1, policy_version 1705827 (0.0008) [2023-12-27 17:28:25,708][20381] Updated weights for policy 1, policy_version 1705837 (0.0007) [2023-12-27 17:28:26,142][20347] Updated weights for policy 0, policy_version 1707369 (0.0005) [2023-12-27 17:28:26,205][20347] Updated weights for policy 0, policy_version 1707379 (0.0005) [2023-12-27 17:28:26,267][20347] Updated weights for policy 0, policy_version 1707389 (0.0009) [2023-12-27 17:28:26,315][20347] Updated weights for policy 0, policy_version 1707399 (0.0009) [2023-12-27 17:28:26,342][20381] Updated weights for policy 1, policy_version 1705847 (0.0009) [2023-12-27 17:28:26,395][20381] Updated weights for policy 1, policy_version 1705857 (0.0008) [2023-12-27 17:28:26,457][20381] Updated weights for policy 1, policy_version 1705867 (0.0008) [2023-12-27 17:28:26,899][20347] Updated weights for policy 0, policy_version 1707409 (0.0010) [2023-12-27 17:28:26,959][20347] Updated weights for policy 0, policy_version 1707419 (0.0010) [2023-12-27 17:28:27,013][20347] Updated weights for policy 0, policy_version 1707429 (0.0010) [2023-12-27 17:28:27,139][20381] Updated weights for policy 1, policy_version 1705877 (0.0007) [2023-12-27 17:28:27,190][20381] Updated weights for policy 1, policy_version 1705887 (0.0005) [2023-12-27 17:28:27,240][20381] Updated weights for policy 1, policy_version 1705897 (0.0007) [2023-12-27 17:28:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 873938944. Throughput: 0: 10207.3, 1: 10188.0. Samples: 873924540. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:27,363][19170] Avg episode reward: [(0, '960.880'), (1, '1000.000')] [2023-12-27 17:28:27,653][20347] Updated weights for policy 0, policy_version 1707439 (0.0009) [2023-12-27 17:28:27,700][20347] Updated weights for policy 0, policy_version 1707449 (0.0005) [2023-12-27 17:28:27,754][20347] Updated weights for policy 0, policy_version 1707459 (0.0005) [2023-12-27 17:28:27,923][20381] Updated weights for policy 1, policy_version 1705907 (0.0007) [2023-12-27 17:28:27,966][20381] Updated weights for policy 1, policy_version 1705917 (0.0005) [2023-12-27 17:28:28,020][20381] Updated weights for policy 1, policy_version 1705927 (0.0007) [2023-12-27 17:28:28,401][20347] Updated weights for policy 0, policy_version 1707469 (0.0008) [2023-12-27 17:28:28,453][20347] Updated weights for policy 0, policy_version 1707479 (0.0010) [2023-12-27 17:28:28,505][20347] Updated weights for policy 0, policy_version 1707489 (0.0010) [2023-12-27 17:28:28,667][20381] Updated weights for policy 1, policy_version 1705937 (0.0007) [2023-12-27 17:28:28,716][20381] Updated weights for policy 1, policy_version 1705947 (0.0007) [2023-12-27 17:28:28,771][20381] Updated weights for policy 1, policy_version 1705957 (0.0010) [2023-12-27 17:28:28,819][20381] Updated weights for policy 1, policy_version 1705967 (0.0010) [2023-12-27 17:28:29,169][20347] Updated weights for policy 0, policy_version 1707499 (0.0010) [2023-12-27 17:28:29,221][20347] Updated weights for policy 0, policy_version 1707509 (0.0010) [2023-12-27 17:28:29,273][20347] Updated weights for policy 0, policy_version 1707519 (0.0009) [2023-12-27 17:28:29,523][20381] Updated weights for policy 1, policy_version 1705977 (0.0009) [2023-12-27 17:28:29,579][20381] Updated weights for policy 1, policy_version 1705987 (0.0009) [2023-12-27 17:28:29,636][20381] Updated weights for policy 1, policy_version 1705997 (0.0010) [2023-12-27 17:28:29,985][20347] Updated weights for policy 0, policy_version 1707529 (0.0010) [2023-12-27 17:28:30,047][20347] Updated weights for policy 0, policy_version 1707539 (0.0008) [2023-12-27 17:28:30,102][20347] Updated weights for policy 0, policy_version 1707549 (0.0008) [2023-12-27 17:28:30,157][20347] Updated weights for policy 0, policy_version 1707559 (0.0007) [2023-12-27 17:28:30,336][20381] Updated weights for policy 1, policy_version 1706007 (0.0010) [2023-12-27 17:28:30,391][20381] Updated weights for policy 1, policy_version 1706017 (0.0010) [2023-12-27 17:28:30,449][20381] Updated weights for policy 1, policy_version 1706027 (0.0010) [2023-12-27 17:28:30,785][20347] Updated weights for policy 0, policy_version 1707569 (0.0006) [2023-12-27 17:28:30,830][20347] Updated weights for policy 0, policy_version 1707579 (0.0006) [2023-12-27 17:28:30,887][20347] Updated weights for policy 0, policy_version 1707589 (0.0007) [2023-12-27 17:28:31,107][20381] Updated weights for policy 1, policy_version 1706037 (0.0010) [2023-12-27 17:28:31,171][20381] Updated weights for policy 1, policy_version 1706047 (0.0010) [2023-12-27 17:28:31,225][20381] Updated weights for policy 1, policy_version 1706057 (0.0011) [2023-12-27 17:28:31,615][20347] Updated weights for policy 0, policy_version 1707599 (0.0007) [2023-12-27 17:28:31,682][20347] Updated weights for policy 0, policy_version 1707609 (0.0007) [2023-12-27 17:28:31,752][20347] Updated weights for policy 0, policy_version 1707619 (0.0008) [2023-12-27 17:28:31,984][20381] Updated weights for policy 1, policy_version 1706067 (0.0010) [2023-12-27 17:28:32,031][20381] Updated weights for policy 1, policy_version 1706077 (0.0007) [2023-12-27 17:28:32,078][20381] Updated weights for policy 1, policy_version 1706087 (0.0007) [2023-12-27 17:28:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 874037248. Throughput: 0: 10281.1, 1: 10242.0. Samples: 874048960. Policy #0 lag: (min: 31.0, avg: 40.5, max: 63.0) [2023-12-27 17:28:32,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:28:32,386][20347] Updated weights for policy 0, policy_version 1707629 (0.0007) [2023-12-27 17:28:32,440][20347] Updated weights for policy 0, policy_version 1707639 (0.0008) [2023-12-27 17:28:32,490][20347] Updated weights for policy 0, policy_version 1707649 (0.0006) [2023-12-27 17:28:32,767][20381] Updated weights for policy 1, policy_version 1706097 (0.0007) [2023-12-27 17:28:32,817][20381] Updated weights for policy 1, policy_version 1706107 (0.0007) [2023-12-27 17:28:32,863][20381] Updated weights for policy 1, policy_version 1706117 (0.0007) [2023-12-27 17:28:32,915][20381] Updated weights for policy 1, policy_version 1706127 (0.0007) [2023-12-27 17:28:33,111][20347] Updated weights for policy 0, policy_version 1707659 (0.0006) [2023-12-27 17:28:33,154][20347] Updated weights for policy 0, policy_version 1707669 (0.0006) [2023-12-27 17:28:33,199][20347] Updated weights for policy 0, policy_version 1707679 (0.0005) [2023-12-27 17:28:33,543][20381] Updated weights for policy 1, policy_version 1706137 (0.0010) [2023-12-27 17:28:33,620][20381] Updated weights for policy 1, policy_version 1706147 (0.0008) [2023-12-27 17:28:33,689][20381] Updated weights for policy 1, policy_version 1706157 (0.0008) [2023-12-27 17:28:33,855][20347] Updated weights for policy 0, policy_version 1707689 (0.0008) [2023-12-27 17:28:33,907][20347] Updated weights for policy 0, policy_version 1707699 (0.0007) [2023-12-27 17:28:33,970][20347] Updated weights for policy 0, policy_version 1707709 (0.0008) [2023-12-27 17:28:34,018][20347] Updated weights for policy 0, policy_version 1707719 (0.0008) [2023-12-27 17:28:34,370][20381] Updated weights for policy 1, policy_version 1706167 (0.0008) [2023-12-27 17:28:34,437][20381] Updated weights for policy 1, policy_version 1706177 (0.0008) [2023-12-27 17:28:34,501][20381] Updated weights for policy 1, policy_version 1706187 (0.0008) [2023-12-27 17:28:34,676][20347] Updated weights for policy 0, policy_version 1707729 (0.0006) [2023-12-27 17:28:34,732][20347] Updated weights for policy 0, policy_version 1707739 (0.0008) [2023-12-27 17:28:34,783][20347] Updated weights for policy 0, policy_version 1707749 (0.0008) [2023-12-27 17:28:35,151][20381] Updated weights for policy 1, policy_version 1706197 (0.0008) [2023-12-27 17:28:35,207][20381] Updated weights for policy 1, policy_version 1706207 (0.0008) [2023-12-27 17:28:35,256][20381] Updated weights for policy 1, policy_version 1706217 (0.0007) [2023-12-27 17:28:35,524][20347] Updated weights for policy 0, policy_version 1707759 (0.0008) [2023-12-27 17:28:35,577][20347] Updated weights for policy 0, policy_version 1707769 (0.0007) [2023-12-27 17:28:35,625][20347] Updated weights for policy 0, policy_version 1707779 (0.0006) [2023-12-27 17:28:35,812][20381] Updated weights for policy 1, policy_version 1706227 (0.0007) [2023-12-27 17:28:35,876][20381] Updated weights for policy 1, policy_version 1706237 (0.0008) [2023-12-27 17:28:35,928][20381] Updated weights for policy 1, policy_version 1706247 (0.0008) [2023-12-27 17:28:36,343][20347] Updated weights for policy 0, policy_version 1707789 (0.0007) [2023-12-27 17:28:36,396][20347] Updated weights for policy 0, policy_version 1707799 (0.0008) [2023-12-27 17:28:36,456][20347] Updated weights for policy 0, policy_version 1707809 (0.0008) [2023-12-27 17:28:36,596][20381] Updated weights for policy 1, policy_version 1706257 (0.0007) [2023-12-27 17:28:36,656][20381] Updated weights for policy 1, policy_version 1706267 (0.0008) [2023-12-27 17:28:36,713][20381] Updated weights for policy 1, policy_version 1706277 (0.0008) [2023-12-27 17:28:36,777][20381] Updated weights for policy 1, policy_version 1706287 (0.0008) [2023-12-27 17:28:37,188][20347] Updated weights for policy 0, policy_version 1707819 (0.0008) [2023-12-27 17:28:37,238][20347] Updated weights for policy 0, policy_version 1707829 (0.0007) [2023-12-27 17:28:37,292][20347] Updated weights for policy 0, policy_version 1707839 (0.0008) [2023-12-27 17:28:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 874143744. Throughput: 0: 10283.6, 1: 10266.2. Samples: 874111664. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:28:37,363][19170] Avg episode reward: [(0, '950.770'), (1, '1000.000')] [2023-12-27 17:28:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001707848_437288960.pth... [2023-12-27 17:28:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001706632_436977664.pth [2023-12-27 17:28:37,404][20381] Updated weights for policy 1, policy_version 1706297 (0.0008) [2023-12-27 17:28:37,468][20381] Updated weights for policy 1, policy_version 1706307 (0.0007) [2023-12-27 17:28:37,515][20381] Updated weights for policy 1, policy_version 1706317 (0.0008) [2023-12-27 17:28:37,528][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001706320_436862976.pth... [2023-12-27 17:28:37,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001705104_436551680.pth [2023-12-27 17:28:38,010][20347] Updated weights for policy 0, policy_version 1707849 (0.0009) [2023-12-27 17:28:38,070][20347] Updated weights for policy 0, policy_version 1707859 (0.0006) [2023-12-27 17:28:38,123][20347] Updated weights for policy 0, policy_version 1707869 (0.0008) [2023-12-27 17:28:38,173][20347] Updated weights for policy 0, policy_version 1707879 (0.0009) [2023-12-27 17:28:38,219][20381] Updated weights for policy 1, policy_version 1706327 (0.0007) [2023-12-27 17:28:38,274][20381] Updated weights for policy 1, policy_version 1706337 (0.0008) [2023-12-27 17:28:38,346][20381] Updated weights for policy 1, policy_version 1706347 (0.0008) [2023-12-27 17:28:38,920][20347] Updated weights for policy 0, policy_version 1707889 (0.0008) [2023-12-27 17:28:38,975][20347] Updated weights for policy 0, policy_version 1707899 (0.0005) [2023-12-27 17:28:38,977][20381] Updated weights for policy 1, policy_version 1706357 (0.0007) [2023-12-27 17:28:39,024][20347] Updated weights for policy 0, policy_version 1707909 (0.0007) [2023-12-27 17:28:39,027][20381] Updated weights for policy 1, policy_version 1706367 (0.0008) [2023-12-27 17:28:39,071][20381] Updated weights for policy 1, policy_version 1706377 (0.0006) [2023-12-27 17:28:39,789][20347] Updated weights for policy 0, policy_version 1707919 (0.0007) [2023-12-27 17:28:39,795][20381] Updated weights for policy 1, policy_version 1706387 (0.0005) [2023-12-27 17:28:39,851][20347] Updated weights for policy 0, policy_version 1707929 (0.0007) [2023-12-27 17:28:39,864][20381] Updated weights for policy 1, policy_version 1706397 (0.0007) [2023-12-27 17:28:39,914][20347] Updated weights for policy 0, policy_version 1707939 (0.0006) [2023-12-27 17:28:39,924][20381] Updated weights for policy 1, policy_version 1706407 (0.0007) [2023-12-27 17:28:40,574][20347] Updated weights for policy 0, policy_version 1707949 (0.0006) [2023-12-27 17:28:40,611][20381] Updated weights for policy 1, policy_version 1706417 (0.0007) [2023-12-27 17:28:40,628][20347] Updated weights for policy 0, policy_version 1707959 (0.0007) [2023-12-27 17:28:40,664][20381] Updated weights for policy 1, policy_version 1706427 (0.0007) [2023-12-27 17:28:40,680][20347] Updated weights for policy 0, policy_version 1707969 (0.0006) [2023-12-27 17:28:40,716][20381] Updated weights for policy 1, policy_version 1706437 (0.0006) [2023-12-27 17:28:40,762][20381] Updated weights for policy 1, policy_version 1706447 (0.0009) [2023-12-27 17:28:41,363][20347] Updated weights for policy 0, policy_version 1707979 (0.0007) [2023-12-27 17:28:41,430][20347] Updated weights for policy 0, policy_version 1707989 (0.0007) [2023-12-27 17:28:41,490][20347] Updated weights for policy 0, policy_version 1707999 (0.0007) [2023-12-27 17:28:41,541][20381] Updated weights for policy 1, policy_version 1706457 (0.0008) [2023-12-27 17:28:41,603][20381] Updated weights for policy 1, policy_version 1706467 (0.0008) [2023-12-27 17:28:41,667][20381] Updated weights for policy 1, policy_version 1706477 (0.0008) [2023-12-27 17:28:42,225][20347] Updated weights for policy 0, policy_version 1708009 (0.0007) [2023-12-27 17:28:42,296][20347] Updated weights for policy 0, policy_version 1708019 (0.0008) [2023-12-27 17:28:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.7, 300 sec: 20493.9). Total num frames: 874233856. Throughput: 0: 10289.1, 1: 10312.7. Samples: 874232564. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:28:42,363][19170] Avg episode reward: [(0, '940.500'), (1, '1000.000')] [2023-12-27 17:28:42,369][20347] Updated weights for policy 0, policy_version 1708029 (0.0008) [2023-12-27 17:28:42,381][20381] Updated weights for policy 1, policy_version 1706487 (0.0007) [2023-12-27 17:28:42,431][20347] Updated weights for policy 0, policy_version 1708039 (0.0007) [2023-12-27 17:28:42,439][20381] Updated weights for policy 1, policy_version 1706497 (0.0006) [2023-12-27 17:28:42,504][20381] Updated weights for policy 1, policy_version 1706507 (0.0006) [2023-12-27 17:28:43,062][20347] Updated weights for policy 0, policy_version 1708049 (0.0008) [2023-12-27 17:28:43,118][20347] Updated weights for policy 0, policy_version 1708059 (0.0007) [2023-12-27 17:28:43,138][20381] Updated weights for policy 1, policy_version 1706517 (0.0007) [2023-12-27 17:28:43,173][20347] Updated weights for policy 0, policy_version 1708069 (0.0007) [2023-12-27 17:28:43,194][20381] Updated weights for policy 1, policy_version 1706527 (0.0006) [2023-12-27 17:28:43,246][20381] Updated weights for policy 1, policy_version 1706537 (0.0008) [2023-12-27 17:28:43,865][20347] Updated weights for policy 0, policy_version 1708079 (0.0006) [2023-12-27 17:28:43,867][20381] Updated weights for policy 1, policy_version 1706547 (0.0007) [2023-12-27 17:28:43,910][20347] Updated weights for policy 0, policy_version 1708089 (0.0007) [2023-12-27 17:28:43,913][20381] Updated weights for policy 1, policy_version 1706557 (0.0005) [2023-12-27 17:28:43,963][20347] Updated weights for policy 0, policy_version 1708099 (0.0006) [2023-12-27 17:28:43,965][20381] Updated weights for policy 1, policy_version 1706567 (0.0006) [2023-12-27 17:28:44,598][20347] Updated weights for policy 0, policy_version 1708109 (0.0007) [2023-12-27 17:28:44,642][20347] Updated weights for policy 0, policy_version 1708119 (0.0008) [2023-12-27 17:28:44,643][20381] Updated weights for policy 1, policy_version 1706577 (0.0006) [2023-12-27 17:28:44,691][20347] Updated weights for policy 0, policy_version 1708129 (0.0008) [2023-12-27 17:28:44,704][20381] Updated weights for policy 1, policy_version 1706587 (0.0007) [2023-12-27 17:28:44,755][20381] Updated weights for policy 1, policy_version 1706597 (0.0007) [2023-12-27 17:28:44,819][20381] Updated weights for policy 1, policy_version 1706607 (0.0007) [2023-12-27 17:28:45,407][20347] Updated weights for policy 0, policy_version 1708139 (0.0006) [2023-12-27 17:28:45,466][20347] Updated weights for policy 0, policy_version 1708149 (0.0008) [2023-12-27 17:28:45,498][20381] Updated weights for policy 1, policy_version 1706617 (0.0006) [2023-12-27 17:28:45,523][20347] Updated weights for policy 0, policy_version 1708159 (0.0007) [2023-12-27 17:28:45,551][20381] Updated weights for policy 1, policy_version 1706627 (0.0005) [2023-12-27 17:28:45,602][20381] Updated weights for policy 1, policy_version 1706637 (0.0007) [2023-12-27 17:28:46,227][20381] Updated weights for policy 1, policy_version 1706647 (0.0007) [2023-12-27 17:28:46,241][20347] Updated weights for policy 0, policy_version 1708169 (0.0007) [2023-12-27 17:28:46,273][20381] Updated weights for policy 1, policy_version 1706657 (0.0006) [2023-12-27 17:28:46,288][20347] Updated weights for policy 0, policy_version 1708179 (0.0005) [2023-12-27 17:28:46,326][20381] Updated weights for policy 1, policy_version 1706667 (0.0006) [2023-12-27 17:28:46,343][20347] Updated weights for policy 0, policy_version 1708189 (0.0006) [2023-12-27 17:28:46,399][20347] Updated weights for policy 0, policy_version 1708199 (0.0007) [2023-12-27 17:28:46,980][20381] Updated weights for policy 1, policy_version 1706677 (0.0007) [2023-12-27 17:28:47,029][20381] Updated weights for policy 1, policy_version 1706687 (0.0007) [2023-12-27 17:28:47,073][20381] Updated weights for policy 1, policy_version 1706697 (0.0007) [2023-12-27 17:28:47,108][20347] Updated weights for policy 0, policy_version 1708209 (0.0009) [2023-12-27 17:28:47,163][20347] Updated weights for policy 0, policy_version 1708219 (0.0008) [2023-12-27 17:28:47,215][20347] Updated weights for policy 0, policy_version 1708229 (0.0007) [2023-12-27 17:28:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 874348544. Throughput: 0: 10289.1, 1: 10327.6. Samples: 874354852. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:28:47,363][19170] Avg episode reward: [(0, '960.980'), (1, '1000.000')] [2023-12-27 17:28:47,788][20381] Updated weights for policy 1, policy_version 1706707 (0.0007) [2023-12-27 17:28:47,843][20381] Updated weights for policy 1, policy_version 1706717 (0.0007) [2023-12-27 17:28:47,900][20381] Updated weights for policy 1, policy_version 1706727 (0.0008) [2023-12-27 17:28:47,911][20347] Updated weights for policy 0, policy_version 1708239 (0.0006) [2023-12-27 17:28:47,966][20347] Updated weights for policy 0, policy_version 1708249 (0.0006) [2023-12-27 17:28:48,023][20347] Updated weights for policy 0, policy_version 1708259 (0.0008) [2023-12-27 17:28:48,640][20381] Updated weights for policy 1, policy_version 1706737 (0.0008) [2023-12-27 17:28:48,699][20381] Updated weights for policy 1, policy_version 1706747 (0.0008) [2023-12-27 17:28:48,726][20347] Updated weights for policy 0, policy_version 1708269 (0.0009) [2023-12-27 17:28:48,755][20381] Updated weights for policy 1, policy_version 1706757 (0.0007) [2023-12-27 17:28:48,783][20347] Updated weights for policy 0, policy_version 1708279 (0.0007) [2023-12-27 17:28:48,805][20381] Updated weights for policy 1, policy_version 1706767 (0.0006) [2023-12-27 17:28:48,835][20347] Updated weights for policy 0, policy_version 1708289 (0.0006) [2023-12-27 17:28:49,543][20347] Updated weights for policy 0, policy_version 1708299 (0.0007) [2023-12-27 17:28:49,552][20381] Updated weights for policy 1, policy_version 1706777 (0.0007) [2023-12-27 17:28:49,597][20347] Updated weights for policy 0, policy_version 1708309 (0.0006) [2023-12-27 17:28:49,603][20381] Updated weights for policy 1, policy_version 1706787 (0.0005) [2023-12-27 17:28:49,650][20347] Updated weights for policy 0, policy_version 1708319 (0.0006) [2023-12-27 17:28:49,661][20381] Updated weights for policy 1, policy_version 1706797 (0.0006) [2023-12-27 17:28:50,280][20347] Updated weights for policy 0, policy_version 1708329 (0.0007) [2023-12-27 17:28:50,335][20347] Updated weights for policy 0, policy_version 1708339 (0.0006) [2023-12-27 17:28:50,397][20347] Updated weights for policy 0, policy_version 1708349 (0.0007) [2023-12-27 17:28:50,428][20381] Updated weights for policy 1, policy_version 1706807 (0.0006) [2023-12-27 17:28:50,453][20347] Updated weights for policy 0, policy_version 1708359 (0.0007) [2023-12-27 17:28:50,484][20381] Updated weights for policy 1, policy_version 1706817 (0.0006) [2023-12-27 17:28:50,542][20381] Updated weights for policy 1, policy_version 1706827 (0.0008) [2023-12-27 17:28:51,137][20347] Updated weights for policy 0, policy_version 1708369 (0.0007) [2023-12-27 17:28:51,205][20347] Updated weights for policy 0, policy_version 1708379 (0.0006) [2023-12-27 17:28:51,212][20381] Updated weights for policy 1, policy_version 1706837 (0.0007) [2023-12-27 17:28:51,270][20381] Updated weights for policy 1, policy_version 1706847 (0.0007) [2023-12-27 17:28:51,271][20347] Updated weights for policy 0, policy_version 1708389 (0.0006) [2023-12-27 17:28:51,326][20381] Updated weights for policy 1, policy_version 1706857 (0.0010) [2023-12-27 17:28:52,034][20347] Updated weights for policy 0, policy_version 1708399 (0.0007) [2023-12-27 17:28:52,040][20381] Updated weights for policy 1, policy_version 1706867 (0.0009) [2023-12-27 17:28:52,088][20347] Updated weights for policy 0, policy_version 1708409 (0.0007) [2023-12-27 17:28:52,093][20381] Updated weights for policy 1, policy_version 1706877 (0.0006) [2023-12-27 17:28:52,149][20347] Updated weights for policy 0, policy_version 1708419 (0.0005) [2023-12-27 17:28:52,151][20381] Updated weights for policy 1, policy_version 1706887 (0.0008) [2023-12-27 17:28:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 874446848. Throughput: 0: 10241.6, 1: 10214.6. Samples: 874413200. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:28:52,363][19170] Avg episode reward: [(0, '970.770'), (1, '1000.000')] [2023-12-27 17:28:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001706896_437010432.pth... [2023-12-27 17:28:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001708424_437436416.pth... [2023-12-27 17:28:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001707240_437133312.pth [2023-12-27 17:28:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001705680_436699136.pth [2023-12-27 17:28:52,817][20347] Updated weights for policy 0, policy_version 1708429 (0.0006) [2023-12-27 17:28:52,862][20347] Updated weights for policy 0, policy_version 1708439 (0.0008) [2023-12-27 17:28:52,885][20381] Updated weights for policy 1, policy_version 1706897 (0.0007) [2023-12-27 17:28:52,914][20347] Updated weights for policy 0, policy_version 1708449 (0.0007) [2023-12-27 17:28:52,944][20381] Updated weights for policy 1, policy_version 1706907 (0.0007) [2023-12-27 17:28:53,005][20381] Updated weights for policy 1, policy_version 1706917 (0.0006) [2023-12-27 17:28:53,063][20381] Updated weights for policy 1, policy_version 1706927 (0.0006) [2023-12-27 17:28:53,570][20347] Updated weights for policy 0, policy_version 1708459 (0.0007) [2023-12-27 17:28:53,626][20347] Updated weights for policy 0, policy_version 1708469 (0.0011) [2023-12-27 17:28:53,631][20381] Updated weights for policy 1, policy_version 1706937 (0.0006) [2023-12-27 17:28:53,675][20347] Updated weights for policy 0, policy_version 1708479 (0.0010) [2023-12-27 17:28:53,690][20381] Updated weights for policy 1, policy_version 1706947 (0.0005) [2023-12-27 17:28:53,751][20381] Updated weights for policy 1, policy_version 1706957 (0.0005) [2023-12-27 17:28:54,338][20381] Updated weights for policy 1, policy_version 1706967 (0.0009) [2023-12-27 17:28:54,355][20347] Updated weights for policy 0, policy_version 1708489 (0.0010) [2023-12-27 17:28:54,398][20381] Updated weights for policy 1, policy_version 1706977 (0.0010) [2023-12-27 17:28:54,406][20347] Updated weights for policy 0, policy_version 1708499 (0.0010) [2023-12-27 17:28:54,453][20381] Updated weights for policy 1, policy_version 1706987 (0.0010) [2023-12-27 17:28:54,463][20347] Updated weights for policy 0, policy_version 1708509 (0.0010) [2023-12-27 17:28:54,526][20347] Updated weights for policy 0, policy_version 1708519 (0.0011) [2023-12-27 17:28:55,112][20381] Updated weights for policy 1, policy_version 1706997 (0.0011) [2023-12-27 17:28:55,147][20347] Updated weights for policy 0, policy_version 1708529 (0.0006) [2023-12-27 17:28:55,168][20381] Updated weights for policy 1, policy_version 1707007 (0.0011) [2023-12-27 17:28:55,217][20347] Updated weights for policy 0, policy_version 1708539 (0.0010) [2023-12-27 17:28:55,237][20381] Updated weights for policy 1, policy_version 1707017 (0.0007) [2023-12-27 17:28:55,277][20347] Updated weights for policy 0, policy_version 1708549 (0.0008) [2023-12-27 17:28:55,913][20381] Updated weights for policy 1, policy_version 1707027 (0.0006) [2023-12-27 17:28:55,934][20347] Updated weights for policy 0, policy_version 1708559 (0.0010) [2023-12-27 17:28:55,968][20381] Updated weights for policy 1, policy_version 1707037 (0.0005) [2023-12-27 17:28:55,991][20347] Updated weights for policy 0, policy_version 1708569 (0.0008) [2023-12-27 17:28:56,025][20381] Updated weights for policy 1, policy_version 1707047 (0.0005) [2023-12-27 17:28:56,048][20347] Updated weights for policy 0, policy_version 1708579 (0.0009) [2023-12-27 17:28:56,683][20381] Updated weights for policy 1, policy_version 1707057 (0.0006) [2023-12-27 17:28:56,710][20347] Updated weights for policy 0, policy_version 1708589 (0.0010) [2023-12-27 17:28:56,737][20381] Updated weights for policy 1, policy_version 1707067 (0.0005) [2023-12-27 17:28:56,764][20347] Updated weights for policy 0, policy_version 1708599 (0.0010) [2023-12-27 17:28:56,788][20381] Updated weights for policy 1, policy_version 1707077 (0.0005) [2023-12-27 17:28:56,818][20347] Updated weights for policy 0, policy_version 1708609 (0.0010) [2023-12-27 17:28:56,842][20381] Updated weights for policy 1, policy_version 1707087 (0.0005) [2023-12-27 17:28:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 874545152. Throughput: 0: 10231.7, 1: 10259.9. Samples: 874538076. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:28:57,363][19170] Avg episode reward: [(0, '980.400'), (1, '1000.000')] [2023-12-27 17:28:57,480][20347] Updated weights for policy 0, policy_version 1708619 (0.0009) [2023-12-27 17:28:57,534][20381] Updated weights for policy 1, policy_version 1707097 (0.0005) [2023-12-27 17:28:57,537][20347] Updated weights for policy 0, policy_version 1708629 (0.0006) [2023-12-27 17:28:57,588][20347] Updated weights for policy 0, policy_version 1708639 (0.0007) [2023-12-27 17:28:57,591][20381] Updated weights for policy 1, policy_version 1707107 (0.0005) [2023-12-27 17:28:57,655][20381] Updated weights for policy 1, policy_version 1707117 (0.0005) [2023-12-27 17:28:58,247][20381] Updated weights for policy 1, policy_version 1707127 (0.0008) [2023-12-27 17:28:58,268][20347] Updated weights for policy 0, policy_version 1708649 (0.0008) [2023-12-27 17:28:58,306][20381] Updated weights for policy 1, policy_version 1707137 (0.0008) [2023-12-27 17:28:58,325][20347] Updated weights for policy 0, policy_version 1708659 (0.0006) [2023-12-27 17:28:58,374][20381] Updated weights for policy 1, policy_version 1707147 (0.0008) [2023-12-27 17:28:58,393][20347] Updated weights for policy 0, policy_version 1708669 (0.0008) [2023-12-27 17:28:58,460][20347] Updated weights for policy 0, policy_version 1708679 (0.0007) [2023-12-27 17:28:59,144][20381] Updated weights for policy 1, policy_version 1707157 (0.0007) [2023-12-27 17:28:59,201][20381] Updated weights for policy 1, policy_version 1707167 (0.0007) [2023-12-27 17:28:59,201][20347] Updated weights for policy 0, policy_version 1708689 (0.0006) [2023-12-27 17:28:59,267][20381] Updated weights for policy 1, policy_version 1707177 (0.0009) [2023-12-27 17:28:59,274][20347] Updated weights for policy 0, policy_version 1708699 (0.0008) [2023-12-27 17:28:59,336][20347] Updated weights for policy 0, policy_version 1708709 (0.0007) [2023-12-27 17:28:59,947][20347] Updated weights for policy 0, policy_version 1708719 (0.0007) [2023-12-27 17:28:59,996][20381] Updated weights for policy 1, policy_version 1707187 (0.0006) [2023-12-27 17:29:00,010][20347] Updated weights for policy 0, policy_version 1708729 (0.0007) [2023-12-27 17:29:00,033][20291] KL-divergence is very high: 143.0426 [2023-12-27 17:29:00,050][20291] KL-divergence is very high: 164.7472 [2023-12-27 17:29:00,054][20381] Updated weights for policy 1, policy_version 1707197 (0.0006) [2023-12-27 17:29:00,056][20291] KL-divergence is very high: 154.3788 [2023-12-27 17:29:00,066][20347] Updated weights for policy 0, policy_version 1708739 (0.0007) [2023-12-27 17:29:00,111][20381] Updated weights for policy 1, policy_version 1707207 (0.0008) [2023-12-27 17:29:00,728][20347] Updated weights for policy 0, policy_version 1708749 (0.0009) [2023-12-27 17:29:00,784][20347] Updated weights for policy 0, policy_version 1708759 (0.0006) [2023-12-27 17:29:00,825][20381] Updated weights for policy 1, policy_version 1707217 (0.0008) [2023-12-27 17:29:00,838][20347] Updated weights for policy 0, policy_version 1708769 (0.0005) [2023-12-27 17:29:00,883][20381] Updated weights for policy 1, policy_version 1707227 (0.0005) [2023-12-27 17:29:00,937][20381] Updated weights for policy 1, policy_version 1707237 (0.0008) [2023-12-27 17:29:00,989][20381] Updated weights for policy 1, policy_version 1707247 (0.0006) [2023-12-27 17:29:01,487][20347] Updated weights for policy 0, policy_version 1708779 (0.0006) [2023-12-27 17:29:01,537][20347] Updated weights for policy 0, policy_version 1708789 (0.0006) [2023-12-27 17:29:01,595][20347] Updated weights for policy 0, policy_version 1708799 (0.0006) [2023-12-27 17:29:01,726][20381] Updated weights for policy 1, policy_version 1707257 (0.0008) [2023-12-27 17:29:01,778][20381] Updated weights for policy 1, policy_version 1707267 (0.0007) [2023-12-27 17:29:01,825][20381] Updated weights for policy 1, policy_version 1707277 (0.0007) [2023-12-27 17:29:02,319][20347] Updated weights for policy 0, policy_version 1708809 (0.0007) [2023-12-27 17:29:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 874643456. Throughput: 0: 10235.0, 1: 10183.9. Samples: 874658808. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:02,363][19170] Avg episode reward: [(0, '941.370'), (1, '990.270')] [2023-12-27 17:29:02,389][20347] Updated weights for policy 0, policy_version 1708819 (0.0007) [2023-12-27 17:29:02,450][20347] Updated weights for policy 0, policy_version 1708829 (0.0005) [2023-12-27 17:29:02,513][20347] Updated weights for policy 0, policy_version 1708839 (0.0006) [2023-12-27 17:29:02,518][20381] Updated weights for policy 1, policy_version 1707287 (0.0007) [2023-12-27 17:29:02,573][20381] Updated weights for policy 1, policy_version 1707297 (0.0005) [2023-12-27 17:29:02,629][20381] Updated weights for policy 1, policy_version 1707307 (0.0005) [2023-12-27 17:29:03,084][20347] Updated weights for policy 0, policy_version 1708849 (0.0007) [2023-12-27 17:29:03,137][20347] Updated weights for policy 0, policy_version 1708859 (0.0007) [2023-12-27 17:29:03,195][20347] Updated weights for policy 0, policy_version 1708869 (0.0005) [2023-12-27 17:29:03,253][20381] Updated weights for policy 1, policy_version 1707317 (0.0007) [2023-12-27 17:29:03,310][20381] Updated weights for policy 1, policy_version 1707327 (0.0006) [2023-12-27 17:29:03,370][20381] Updated weights for policy 1, policy_version 1707337 (0.0005) [2023-12-27 17:29:03,744][20347] Updated weights for policy 0, policy_version 1708879 (0.0007) [2023-12-27 17:29:03,802][20347] Updated weights for policy 0, policy_version 1708889 (0.0005) [2023-12-27 17:29:03,865][20347] Updated weights for policy 0, policy_version 1708899 (0.0007) [2023-12-27 17:29:03,895][20381] Updated weights for policy 1, policy_version 1707347 (0.0005) [2023-12-27 17:29:03,956][20381] Updated weights for policy 1, policy_version 1707357 (0.0006) [2023-12-27 17:29:04,021][20381] Updated weights for policy 1, policy_version 1707367 (0.0007) [2023-12-27 17:29:04,485][20347] Updated weights for policy 0, policy_version 1708909 (0.0008) [2023-12-27 17:29:04,541][20347] Updated weights for policy 0, policy_version 1708919 (0.0007) [2023-12-27 17:29:04,600][20347] Updated weights for policy 0, policy_version 1708929 (0.0005) [2023-12-27 17:29:04,716][20381] Updated weights for policy 1, policy_version 1707377 (0.0007) [2023-12-27 17:29:04,775][20381] Updated weights for policy 1, policy_version 1707387 (0.0008) [2023-12-27 17:29:04,837][20381] Updated weights for policy 1, policy_version 1707397 (0.0008) [2023-12-27 17:29:04,895][20381] Updated weights for policy 1, policy_version 1707407 (0.0008) [2023-12-27 17:29:05,204][20347] Updated weights for policy 0, policy_version 1708939 (0.0007) [2023-12-27 17:29:05,261][20347] Updated weights for policy 0, policy_version 1708949 (0.0008) [2023-12-27 17:29:05,318][20347] Updated weights for policy 0, policy_version 1708959 (0.0007) [2023-12-27 17:29:05,583][20381] Updated weights for policy 1, policy_version 1707417 (0.0005) [2023-12-27 17:29:05,635][20381] Updated weights for policy 1, policy_version 1707427 (0.0006) [2023-12-27 17:29:05,685][20381] Updated weights for policy 1, policy_version 1707437 (0.0006) [2023-12-27 17:29:05,958][20347] Updated weights for policy 0, policy_version 1708969 (0.0006) [2023-12-27 17:29:06,021][20347] Updated weights for policy 0, policy_version 1708979 (0.0006) [2023-12-27 17:29:06,079][20347] Updated weights for policy 0, policy_version 1708989 (0.0006) [2023-12-27 17:29:06,142][20347] Updated weights for policy 0, policy_version 1708999 (0.0007) [2023-12-27 17:29:06,340][20381] Updated weights for policy 1, policy_version 1707447 (0.0008) [2023-12-27 17:29:06,390][20381] Updated weights for policy 1, policy_version 1707457 (0.0008) [2023-12-27 17:29:06,444][20381] Updated weights for policy 1, policy_version 1707467 (0.0008) [2023-12-27 17:29:06,827][20347] Updated weights for policy 0, policy_version 1709009 (0.0006) [2023-12-27 17:29:06,891][20347] Updated weights for policy 0, policy_version 1709019 (0.0006) [2023-12-27 17:29:06,948][20347] Updated weights for policy 0, policy_version 1709029 (0.0006) [2023-12-27 17:29:07,177][20381] Updated weights for policy 1, policy_version 1707477 (0.0008) [2023-12-27 17:29:07,234][20381] Updated weights for policy 1, policy_version 1707487 (0.0006) [2023-12-27 17:29:07,289][20381] Updated weights for policy 1, policy_version 1707497 (0.0008) [2023-12-27 17:29:07,363][19170] Fps is (10 sec: 21298.1, 60 sec: 20479.8, 300 sec: 20521.6). Total num frames: 874758144. Throughput: 0: 10273.9, 1: 10233.9. Samples: 874724092. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:07,364][19170] Avg episode reward: [(0, '921.850'), (1, '990.270')] [2023-12-27 17:29:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001709032_437592064.pth... [2023-12-27 17:29:07,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001707504_437166080.pth... [2023-12-27 17:29:07,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001707848_437288960.pth [2023-12-27 17:29:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001706320_436862976.pth [2023-12-27 17:29:07,579][20347] Updated weights for policy 0, policy_version 1709039 (0.0007) [2023-12-27 17:29:07,634][20347] Updated weights for policy 0, policy_version 1709049 (0.0006) [2023-12-27 17:29:07,689][20347] Updated weights for policy 0, policy_version 1709059 (0.0005) [2023-12-27 17:29:07,902][20381] Updated weights for policy 1, policy_version 1707507 (0.0007) [2023-12-27 17:29:07,960][20381] Updated weights for policy 1, policy_version 1707517 (0.0006) [2023-12-27 17:29:08,017][20381] Updated weights for policy 1, policy_version 1707527 (0.0007) [2023-12-27 17:29:08,364][20347] Updated weights for policy 0, policy_version 1709069 (0.0006) [2023-12-27 17:29:08,424][20347] Updated weights for policy 0, policy_version 1709079 (0.0007) [2023-12-27 17:29:08,482][20347] Updated weights for policy 0, policy_version 1709089 (0.0008) [2023-12-27 17:29:08,719][20381] Updated weights for policy 1, policy_version 1707537 (0.0008) [2023-12-27 17:29:08,785][20381] Updated weights for policy 1, policy_version 1707547 (0.0010) [2023-12-27 17:29:08,840][20381] Updated weights for policy 1, policy_version 1707557 (0.0008) [2023-12-27 17:29:08,899][20381] Updated weights for policy 1, policy_version 1707567 (0.0008) [2023-12-27 17:29:09,094][20347] Updated weights for policy 0, policy_version 1709099 (0.0007) [2023-12-27 17:29:09,144][20347] Updated weights for policy 0, policy_version 1709109 (0.0005) [2023-12-27 17:29:09,202][20347] Updated weights for policy 0, policy_version 1709119 (0.0006) [2023-12-27 17:29:09,678][20381] Updated weights for policy 1, policy_version 1707577 (0.0007) [2023-12-27 17:29:09,738][20381] Updated weights for policy 1, policy_version 1707587 (0.0009) [2023-12-27 17:29:09,788][20381] Updated weights for policy 1, policy_version 1707597 (0.0010) [2023-12-27 17:29:09,826][20347] Updated weights for policy 0, policy_version 1709129 (0.0007) [2023-12-27 17:29:09,895][20347] Updated weights for policy 0, policy_version 1709139 (0.0007) [2023-12-27 17:29:09,963][20347] Updated weights for policy 0, policy_version 1709149 (0.0007) [2023-12-27 17:29:10,027][20347] Updated weights for policy 0, policy_version 1709159 (0.0006) [2023-12-27 17:29:10,576][20381] Updated weights for policy 1, policy_version 1707607 (0.0008) [2023-12-27 17:29:10,636][20381] Updated weights for policy 1, policy_version 1707617 (0.0007) [2023-12-27 17:29:10,690][20381] Updated weights for policy 1, policy_version 1707627 (0.0006) [2023-12-27 17:29:10,691][20347] Updated weights for policy 0, policy_version 1709169 (0.0010) [2023-12-27 17:29:10,744][20347] Updated weights for policy 0, policy_version 1709179 (0.0010) [2023-12-27 17:29:10,794][20347] Updated weights for policy 0, policy_version 1709189 (0.0010) [2023-12-27 17:29:11,374][20381] Updated weights for policy 1, policy_version 1707637 (0.0007) [2023-12-27 17:29:11,437][20381] Updated weights for policy 1, policy_version 1707647 (0.0007) [2023-12-27 17:29:11,500][20381] Updated weights for policy 1, policy_version 1707657 (0.0008) [2023-12-27 17:29:11,565][20347] Updated weights for policy 0, policy_version 1709199 (0.0009) [2023-12-27 17:29:11,626][20347] Updated weights for policy 0, policy_version 1709209 (0.0008) [2023-12-27 17:29:11,693][20347] Updated weights for policy 0, policy_version 1709219 (0.0008) [2023-12-27 17:29:12,181][20381] Updated weights for policy 1, policy_version 1707667 (0.0008) [2023-12-27 17:29:12,233][20381] Updated weights for policy 1, policy_version 1707677 (0.0007) [2023-12-27 17:29:12,296][20381] Updated weights for policy 1, policy_version 1707687 (0.0006) [2023-12-27 17:29:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 874856448. Throughput: 0: 10305.1, 1: 10181.9. Samples: 874846452. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:12,363][19170] Avg episode reward: [(0, '951.230'), (1, '1000.000')] [2023-12-27 17:29:12,378][20347] Updated weights for policy 0, policy_version 1709229 (0.0010) [2023-12-27 17:29:12,441][20347] Updated weights for policy 0, policy_version 1709240 (0.0008) [2023-12-27 17:29:12,500][20347] Updated weights for policy 0, policy_version 1709250 (0.0008) [2023-12-27 17:29:13,027][20381] Updated weights for policy 1, policy_version 1707697 (0.0006) [2023-12-27 17:29:13,076][20381] Updated weights for policy 1, policy_version 1707707 (0.0005) [2023-12-27 17:29:13,114][20347] Updated weights for policy 0, policy_version 1709260 (0.0008) [2023-12-27 17:29:13,127][20381] Updated weights for policy 1, policy_version 1707717 (0.0005) [2023-12-27 17:29:13,171][20347] Updated weights for policy 0, policy_version 1709270 (0.0006) [2023-12-27 17:29:13,180][20381] Updated weights for policy 1, policy_version 1707727 (0.0008) [2023-12-27 17:29:13,224][20347] Updated weights for policy 0, policy_version 1709280 (0.0007) [2023-12-27 17:29:13,792][20381] Updated weights for policy 1, policy_version 1707737 (0.0007) [2023-12-27 17:29:13,839][20381] Updated weights for policy 1, policy_version 1707747 (0.0006) [2023-12-27 17:29:13,863][20347] Updated weights for policy 0, policy_version 1709290 (0.0007) [2023-12-27 17:29:13,895][20381] Updated weights for policy 1, policy_version 1707757 (0.0005) [2023-12-27 17:29:13,913][20347] Updated weights for policy 0, policy_version 1709300 (0.0006) [2023-12-27 17:29:13,962][20347] Updated weights for policy 0, policy_version 1709310 (0.0005) [2023-12-27 17:29:14,007][20347] Updated weights for policy 0, policy_version 1709320 (0.0006) [2023-12-27 17:29:14,551][20381] Updated weights for policy 1, policy_version 1707767 (0.0008) [2023-12-27 17:29:14,608][20381] Updated weights for policy 1, policy_version 1707777 (0.0011) [2023-12-27 17:29:14,660][20381] Updated weights for policy 1, policy_version 1707787 (0.0011) [2023-12-27 17:29:14,687][20347] Updated weights for policy 0, policy_version 1709330 (0.0005) [2023-12-27 17:29:14,739][20347] Updated weights for policy 0, policy_version 1709340 (0.0006) [2023-12-27 17:29:14,806][20347] Updated weights for policy 0, policy_version 1709350 (0.0008) [2023-12-27 17:29:15,340][20381] Updated weights for policy 1, policy_version 1707797 (0.0011) [2023-12-27 17:29:15,397][20381] Updated weights for policy 1, policy_version 1707807 (0.0011) [2023-12-27 17:29:15,456][20381] Updated weights for policy 1, policy_version 1707817 (0.0011) [2023-12-27 17:29:15,466][20347] Updated weights for policy 0, policy_version 1709360 (0.0006) [2023-12-27 17:29:15,517][20347] Updated weights for policy 0, policy_version 1709370 (0.0005) [2023-12-27 17:29:15,569][20347] Updated weights for policy 0, policy_version 1709380 (0.0005) [2023-12-27 17:29:16,039][20381] Updated weights for policy 1, policy_version 1707827 (0.0009) [2023-12-27 17:29:16,086][20381] Updated weights for policy 1, policy_version 1707837 (0.0005) [2023-12-27 17:29:16,143][20381] Updated weights for policy 1, policy_version 1707847 (0.0009) [2023-12-27 17:29:16,164][20347] Updated weights for policy 0, policy_version 1709390 (0.0005) [2023-12-27 17:29:16,223][20347] Updated weights for policy 0, policy_version 1709400 (0.0007) [2023-12-27 17:29:16,278][20347] Updated weights for policy 0, policy_version 1709410 (0.0007) [2023-12-27 17:29:16,738][20381] Updated weights for policy 1, policy_version 1707857 (0.0010) [2023-12-27 17:29:16,789][20381] Updated weights for policy 1, policy_version 1707867 (0.0006) [2023-12-27 17:29:16,838][20381] Updated weights for policy 1, policy_version 1707877 (0.0010) [2023-12-27 17:29:16,889][20381] Updated weights for policy 1, policy_version 1707887 (0.0007) [2023-12-27 17:29:17,027][20347] Updated weights for policy 0, policy_version 1709420 (0.0007) [2023-12-27 17:29:17,080][20347] Updated weights for policy 0, policy_version 1709430 (0.0005) [2023-12-27 17:29:17,129][20347] Updated weights for policy 0, policy_version 1709440 (0.0007) [2023-12-27 17:29:17,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 874962944. Throughput: 0: 10279.5, 1: 10247.6. Samples: 874972684. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:17,363][19170] Avg episode reward: [(0, '980.540'), (1, '1000.000')] [2023-12-27 17:29:17,493][20381] Updated weights for policy 1, policy_version 1707897 (0.0005) [2023-12-27 17:29:17,551][20381] Updated weights for policy 1, policy_version 1707907 (0.0006) [2023-12-27 17:29:17,602][20381] Updated weights for policy 1, policy_version 1707917 (0.0007) [2023-12-27 17:29:17,769][20347] Updated weights for policy 0, policy_version 1709450 (0.0007) [2023-12-27 17:29:17,823][20347] Updated weights for policy 0, policy_version 1709460 (0.0005) [2023-12-27 17:29:17,878][20347] Updated weights for policy 0, policy_version 1709470 (0.0005) [2023-12-27 17:29:17,935][20347] Updated weights for policy 0, policy_version 1709480 (0.0005) [2023-12-27 17:29:18,203][20381] Updated weights for policy 1, policy_version 1707927 (0.0008) [2023-12-27 17:29:18,251][20381] Updated weights for policy 1, policy_version 1707937 (0.0008) [2023-12-27 17:29:18,299][20381] Updated weights for policy 1, policy_version 1707947 (0.0006) [2023-12-27 17:29:18,531][20347] Updated weights for policy 0, policy_version 1709490 (0.0008) [2023-12-27 17:29:18,593][20347] Updated weights for policy 0, policy_version 1709500 (0.0006) [2023-12-27 17:29:18,651][20347] Updated weights for policy 0, policy_version 1709510 (0.0008) [2023-12-27 17:29:18,956][20381] Updated weights for policy 1, policy_version 1707957 (0.0007) [2023-12-27 17:29:19,015][20381] Updated weights for policy 1, policy_version 1707967 (0.0007) [2023-12-27 17:29:19,072][20381] Updated weights for policy 1, policy_version 1707977 (0.0007) [2023-12-27 17:29:19,359][20347] Updated weights for policy 0, policy_version 1709520 (0.0008) [2023-12-27 17:29:19,418][20347] Updated weights for policy 0, policy_version 1709530 (0.0007) [2023-12-27 17:29:19,482][20347] Updated weights for policy 0, policy_version 1709540 (0.0006) [2023-12-27 17:29:19,789][20381] Updated weights for policy 1, policy_version 1707987 (0.0007) [2023-12-27 17:29:19,852][20381] Updated weights for policy 1, policy_version 1707997 (0.0008) [2023-12-27 17:29:19,912][20381] Updated weights for policy 1, policy_version 1708007 (0.0008) [2023-12-27 17:29:20,185][20347] Updated weights for policy 0, policy_version 1709550 (0.0008) [2023-12-27 17:29:20,243][20347] Updated weights for policy 0, policy_version 1709560 (0.0008) [2023-12-27 17:29:20,301][20347] Updated weights for policy 0, policy_version 1709570 (0.0009) [2023-12-27 17:29:20,607][20381] Updated weights for policy 1, policy_version 1708017 (0.0009) [2023-12-27 17:29:20,665][20381] Updated weights for policy 1, policy_version 1708027 (0.0008) [2023-12-27 17:29:20,720][20381] Updated weights for policy 1, policy_version 1708037 (0.0008) [2023-12-27 17:29:20,779][20381] Updated weights for policy 1, policy_version 1708047 (0.0008) [2023-12-27 17:29:21,001][20347] Updated weights for policy 0, policy_version 1709580 (0.0008) [2023-12-27 17:29:21,063][20347] Updated weights for policy 0, policy_version 1709590 (0.0007) [2023-12-27 17:29:21,127][20347] Updated weights for policy 0, policy_version 1709600 (0.0008) [2023-12-27 17:29:21,482][20381] Updated weights for policy 1, policy_version 1708057 (0.0008) [2023-12-27 17:29:21,542][20381] Updated weights for policy 1, policy_version 1708067 (0.0008) [2023-12-27 17:29:21,599][20381] Updated weights for policy 1, policy_version 1708077 (0.0008) [2023-12-27 17:29:21,833][20347] Updated weights for policy 0, policy_version 1709610 (0.0006) [2023-12-27 17:29:21,895][20347] Updated weights for policy 0, policy_version 1709620 (0.0007) [2023-12-27 17:29:21,958][20347] Updated weights for policy 0, policy_version 1709630 (0.0008) [2023-12-27 17:29:22,026][20347] Updated weights for policy 0, policy_version 1709640 (0.0008) [2023-12-27 17:29:22,349][20381] Updated weights for policy 1, policy_version 1708087 (0.0007) [2023-12-27 17:29:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 875061248. Throughput: 0: 10267.6, 1: 10254.5. Samples: 875035156. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:22,363][19170] Avg episode reward: [(0, '980.460'), (1, '1000.000')] [2023-12-27 17:29:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001709640_437747712.pth... [2023-12-27 17:29:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001708424_437436416.pth [2023-12-27 17:29:22,411][20381] Updated weights for policy 1, policy_version 1708097 (0.0009) [2023-12-27 17:29:22,464][20381] Updated weights for policy 1, policy_version 1708107 (0.0006) [2023-12-27 17:29:22,491][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001708112_437321728.pth... [2023-12-27 17:29:22,494][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001706896_437010432.pth [2023-12-27 17:29:22,720][20347] Updated weights for policy 0, policy_version 1709650 (0.0007) [2023-12-27 17:29:22,779][20347] Updated weights for policy 0, policy_version 1709660 (0.0008) [2023-12-27 17:29:22,834][20347] Updated weights for policy 0, policy_version 1709670 (0.0008) [2023-12-27 17:29:23,182][20381] Updated weights for policy 1, policy_version 1708117 (0.0008) [2023-12-27 17:29:23,231][20381] Updated weights for policy 1, policy_version 1708127 (0.0010) [2023-12-27 17:29:23,276][20381] Updated weights for policy 1, policy_version 1708137 (0.0010) [2023-12-27 17:29:23,556][20347] Updated weights for policy 0, policy_version 1709680 (0.0007) [2023-12-27 17:29:23,613][20347] Updated weights for policy 0, policy_version 1709690 (0.0007) [2023-12-27 17:29:23,670][20347] Updated weights for policy 0, policy_version 1709700 (0.0007) [2023-12-27 17:29:23,942][20381] Updated weights for policy 1, policy_version 1708147 (0.0007) [2023-12-27 17:29:24,000][20381] Updated weights for policy 1, policy_version 1708157 (0.0011) [2023-12-27 17:29:24,058][20381] Updated weights for policy 1, policy_version 1708167 (0.0009) [2023-12-27 17:29:24,357][20347] Updated weights for policy 0, policy_version 1709710 (0.0007) [2023-12-27 17:29:24,417][20347] Updated weights for policy 0, policy_version 1709720 (0.0007) [2023-12-27 17:29:24,472][20347] Updated weights for policy 0, policy_version 1709730 (0.0007) [2023-12-27 17:29:24,703][20381] Updated weights for policy 1, policy_version 1708177 (0.0006) [2023-12-27 17:29:24,755][20381] Updated weights for policy 1, policy_version 1708187 (0.0008) [2023-12-27 17:29:24,810][20381] Updated weights for policy 1, policy_version 1708197 (0.0008) [2023-12-27 17:29:24,870][20381] Updated weights for policy 1, policy_version 1708207 (0.0008) [2023-12-27 17:29:25,179][20347] Updated weights for policy 0, policy_version 1709740 (0.0008) [2023-12-27 17:29:25,241][20347] Updated weights for policy 0, policy_version 1709750 (0.0009) [2023-12-27 17:29:25,295][20347] Updated weights for policy 0, policy_version 1709760 (0.0009) [2023-12-27 17:29:25,499][20381] Updated weights for policy 1, policy_version 1708217 (0.0006) [2023-12-27 17:29:25,550][20381] Updated weights for policy 1, policy_version 1708227 (0.0006) [2023-12-27 17:29:25,609][20381] Updated weights for policy 1, policy_version 1708237 (0.0008) [2023-12-27 17:29:26,027][20347] Updated weights for policy 0, policy_version 1709770 (0.0008) [2023-12-27 17:29:26,083][20347] Updated weights for policy 0, policy_version 1709780 (0.0007) [2023-12-27 17:29:26,133][20347] Updated weights for policy 0, policy_version 1709790 (0.0005) [2023-12-27 17:29:26,190][20347] Updated weights for policy 0, policy_version 1709800 (0.0005) [2023-12-27 17:29:26,235][20381] Updated weights for policy 1, policy_version 1708247 (0.0008) [2023-12-27 17:29:26,293][20381] Updated weights for policy 1, policy_version 1708257 (0.0008) [2023-12-27 17:29:26,349][20381] Updated weights for policy 1, policy_version 1708267 (0.0008) [2023-12-27 17:29:26,825][20347] Updated weights for policy 0, policy_version 1709810 (0.0008) [2023-12-27 17:29:26,881][20347] Updated weights for policy 0, policy_version 1709820 (0.0008) [2023-12-27 17:29:26,938][20347] Updated weights for policy 0, policy_version 1709830 (0.0007) [2023-12-27 17:29:27,006][20381] Updated weights for policy 1, policy_version 1708277 (0.0007) [2023-12-27 17:29:27,057][20381] Updated weights for policy 1, policy_version 1708287 (0.0006) [2023-12-27 17:29:27,102][20381] Updated weights for policy 1, policy_version 1708297 (0.0005) [2023-12-27 17:29:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 875167744. Throughput: 0: 10264.7, 1: 10245.4. Samples: 875155520. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:27,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 17:29:27,606][20347] Updated weights for policy 0, policy_version 1709840 (0.0006) [2023-12-27 17:29:27,661][20347] Updated weights for policy 0, policy_version 1709850 (0.0005) [2023-12-27 17:29:27,710][20347] Updated weights for policy 0, policy_version 1709860 (0.0005) [2023-12-27 17:29:27,767][20381] Updated weights for policy 1, policy_version 1708307 (0.0006) [2023-12-27 17:29:27,820][20381] Updated weights for policy 1, policy_version 1708317 (0.0009) [2023-12-27 17:29:27,872][20381] Updated weights for policy 1, policy_version 1708327 (0.0007) [2023-12-27 17:29:28,335][20347] Updated weights for policy 0, policy_version 1709870 (0.0007) [2023-12-27 17:29:28,398][20347] Updated weights for policy 0, policy_version 1709880 (0.0008) [2023-12-27 17:29:28,454][20347] Updated weights for policy 0, policy_version 1709890 (0.0007) [2023-12-27 17:29:28,506][20381] Updated weights for policy 1, policy_version 1708337 (0.0006) [2023-12-27 17:29:28,557][20381] Updated weights for policy 1, policy_version 1708347 (0.0007) [2023-12-27 17:29:28,607][20381] Updated weights for policy 1, policy_version 1708357 (0.0005) [2023-12-27 17:29:28,657][20381] Updated weights for policy 1, policy_version 1708367 (0.0005) [2023-12-27 17:29:29,059][20347] Updated weights for policy 0, policy_version 1709900 (0.0006) [2023-12-27 17:29:29,113][20347] Updated weights for policy 0, policy_version 1709910 (0.0006) [2023-12-27 17:29:29,160][20347] Updated weights for policy 0, policy_version 1709920 (0.0007) [2023-12-27 17:29:29,289][20381] Updated weights for policy 1, policy_version 1708377 (0.0008) [2023-12-27 17:29:29,355][20381] Updated weights for policy 1, policy_version 1708387 (0.0009) [2023-12-27 17:29:29,414][20381] Updated weights for policy 1, policy_version 1708397 (0.0010) [2023-12-27 17:29:29,841][20347] Updated weights for policy 0, policy_version 1709930 (0.0007) [2023-12-27 17:29:29,907][20347] Updated weights for policy 0, policy_version 1709940 (0.0008) [2023-12-27 17:29:29,965][20347] Updated weights for policy 0, policy_version 1709950 (0.0008) [2023-12-27 17:29:30,017][20347] Updated weights for policy 0, policy_version 1709960 (0.0007) [2023-12-27 17:29:30,038][20381] Updated weights for policy 1, policy_version 1708407 (0.0011) [2023-12-27 17:29:30,086][20381] Updated weights for policy 1, policy_version 1708417 (0.0010) [2023-12-27 17:29:30,139][20381] Updated weights for policy 1, policy_version 1708427 (0.0010) [2023-12-27 17:29:30,700][20347] Updated weights for policy 0, policy_version 1709970 (0.0005) [2023-12-27 17:29:30,750][20347] Updated weights for policy 0, policy_version 1709980 (0.0005) [2023-12-27 17:29:30,801][20347] Updated weights for policy 0, policy_version 1709990 (0.0005) [2023-12-27 17:29:30,845][20381] Updated weights for policy 1, policy_version 1708437 (0.0009) [2023-12-27 17:29:30,896][20381] Updated weights for policy 1, policy_version 1708447 (0.0010) [2023-12-27 17:29:30,941][20381] Updated weights for policy 1, policy_version 1708457 (0.0010) [2023-12-27 17:29:31,466][20347] Updated weights for policy 0, policy_version 1710000 (0.0007) [2023-12-27 17:29:31,527][20347] Updated weights for policy 0, policy_version 1710010 (0.0008) [2023-12-27 17:29:31,591][20347] Updated weights for policy 0, policy_version 1710020 (0.0007) [2023-12-27 17:29:31,658][20381] Updated weights for policy 1, policy_version 1708467 (0.0009) [2023-12-27 17:29:31,718][20381] Updated weights for policy 1, policy_version 1708477 (0.0008) [2023-12-27 17:29:31,777][20381] Updated weights for policy 1, policy_version 1708487 (0.0008) [2023-12-27 17:29:32,304][20347] Updated weights for policy 0, policy_version 1710030 (0.0009) [2023-12-27 17:29:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 875266048. Throughput: 0: 10331.7, 1: 10266.8. Samples: 875281788. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:32,365][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 17:29:32,366][20347] Updated weights for policy 0, policy_version 1710040 (0.0010) [2023-12-27 17:29:32,428][20347] Updated weights for policy 0, policy_version 1710050 (0.0009) [2023-12-27 17:29:32,475][20381] Updated weights for policy 1, policy_version 1708497 (0.0008) [2023-12-27 17:29:32,525][20381] Updated weights for policy 1, policy_version 1708507 (0.0007) [2023-12-27 17:29:32,574][20381] Updated weights for policy 1, policy_version 1708517 (0.0007) [2023-12-27 17:29:32,625][20381] Updated weights for policy 1, policy_version 1708527 (0.0007) [2023-12-27 17:29:33,042][20347] Updated weights for policy 0, policy_version 1710060 (0.0010) [2023-12-27 17:29:33,091][20347] Updated weights for policy 0, policy_version 1710070 (0.0010) [2023-12-27 17:29:33,147][20347] Updated weights for policy 0, policy_version 1710080 (0.0010) [2023-12-27 17:29:33,244][20381] Updated weights for policy 1, policy_version 1708537 (0.0008) [2023-12-27 17:29:33,297][20381] Updated weights for policy 1, policy_version 1708547 (0.0006) [2023-12-27 17:29:33,352][20381] Updated weights for policy 1, policy_version 1708557 (0.0005) [2023-12-27 17:29:33,831][20347] Updated weights for policy 0, policy_version 1710090 (0.0009) [2023-12-27 17:29:33,883][20347] Updated weights for policy 0, policy_version 1710100 (0.0007) [2023-12-27 17:29:33,939][20347] Updated weights for policy 0, policy_version 1710110 (0.0005) [2023-12-27 17:29:33,994][20347] Updated weights for policy 0, policy_version 1710120 (0.0005) [2023-12-27 17:29:34,017][20381] Updated weights for policy 1, policy_version 1708567 (0.0006) [2023-12-27 17:29:34,067][20381] Updated weights for policy 1, policy_version 1708577 (0.0006) [2023-12-27 17:29:34,118][20381] Updated weights for policy 1, policy_version 1708587 (0.0007) [2023-12-27 17:29:34,673][20347] Updated weights for policy 0, policy_version 1710130 (0.0007) [2023-12-27 17:29:34,727][20347] Updated weights for policy 0, policy_version 1710140 (0.0005) [2023-12-27 17:29:34,781][20347] Updated weights for policy 0, policy_version 1710150 (0.0008) [2023-12-27 17:29:34,870][20381] Updated weights for policy 1, policy_version 1708597 (0.0009) [2023-12-27 17:29:34,921][20381] Updated weights for policy 1, policy_version 1708607 (0.0007) [2023-12-27 17:29:34,976][20381] Updated weights for policy 1, policy_version 1708617 (0.0005) [2023-12-27 17:29:35,388][20347] Updated weights for policy 0, policy_version 1710160 (0.0006) [2023-12-27 17:29:35,431][20347] Updated weights for policy 0, policy_version 1710170 (0.0005) [2023-12-27 17:29:35,486][20347] Updated weights for policy 0, policy_version 1710180 (0.0008) [2023-12-27 17:29:35,662][20381] Updated weights for policy 1, policy_version 1708627 (0.0006) [2023-12-27 17:29:35,716][20381] Updated weights for policy 1, policy_version 1708637 (0.0007) [2023-12-27 17:29:35,766][20381] Updated weights for policy 1, policy_version 1708647 (0.0007) [2023-12-27 17:29:36,187][20347] Updated weights for policy 0, policy_version 1710190 (0.0008) [2023-12-27 17:29:36,254][20347] Updated weights for policy 0, policy_version 1710200 (0.0008) [2023-12-27 17:29:36,324][20347] Updated weights for policy 0, policy_version 1710210 (0.0008) [2023-12-27 17:29:36,392][20381] Updated weights for policy 1, policy_version 1708657 (0.0006) [2023-12-27 17:29:36,455][20381] Updated weights for policy 1, policy_version 1708667 (0.0008) [2023-12-27 17:29:36,517][20381] Updated weights for policy 1, policy_version 1708677 (0.0007) [2023-12-27 17:29:36,583][20381] Updated weights for policy 1, policy_version 1708687 (0.0008) [2023-12-27 17:29:37,017][20347] Updated weights for policy 0, policy_version 1710220 (0.0007) [2023-12-27 17:29:37,073][20347] Updated weights for policy 0, policy_version 1710230 (0.0007) [2023-12-27 17:29:37,129][20347] Updated weights for policy 0, policy_version 1710240 (0.0007) [2023-12-27 17:29:37,215][20381] Updated weights for policy 1, policy_version 1708697 (0.0010) [2023-12-27 17:29:37,266][20381] Updated weights for policy 1, policy_version 1708707 (0.0009) [2023-12-27 17:29:37,319][20381] Updated weights for policy 1, policy_version 1708717 (0.0011) [2023-12-27 17:29:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 875380736. Throughput: 0: 10361.3, 1: 10318.5. Samples: 875343788. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:37,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:29:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001708720_437477376.pth... [2023-12-27 17:29:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001710248_437903360.pth... [2023-12-27 17:29:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001709032_437592064.pth [2023-12-27 17:29:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001707504_437166080.pth [2023-12-27 17:29:37,816][20347] Updated weights for policy 0, policy_version 1710250 (0.0007) [2023-12-27 17:29:37,868][20347] Updated weights for policy 0, policy_version 1710260 (0.0006) [2023-12-27 17:29:37,916][20347] Updated weights for policy 0, policy_version 1710270 (0.0006) [2023-12-27 17:29:37,962][20347] Updated weights for policy 0, policy_version 1710280 (0.0006) [2023-12-27 17:29:37,964][20381] Updated weights for policy 1, policy_version 1708727 (0.0007) [2023-12-27 17:29:38,025][20381] Updated weights for policy 1, policy_version 1708737 (0.0005) [2023-12-27 17:29:38,086][20381] Updated weights for policy 1, policy_version 1708747 (0.0005) [2023-12-27 17:29:38,673][20347] Updated weights for policy 0, policy_version 1710290 (0.0009) [2023-12-27 17:29:38,693][20381] Updated weights for policy 1, policy_version 1708757 (0.0008) [2023-12-27 17:29:38,733][20347] Updated weights for policy 0, policy_version 1710300 (0.0007) [2023-12-27 17:29:38,752][20381] Updated weights for policy 1, policy_version 1708767 (0.0006) [2023-12-27 17:29:38,790][20347] Updated weights for policy 0, policy_version 1710310 (0.0010) [2023-12-27 17:29:38,805][20381] Updated weights for policy 1, policy_version 1708777 (0.0005) [2023-12-27 17:29:39,442][20347] Updated weights for policy 0, policy_version 1710320 (0.0008) [2023-12-27 17:29:39,494][20381] Updated weights for policy 1, policy_version 1708787 (0.0007) [2023-12-27 17:29:39,501][20347] Updated weights for policy 0, policy_version 1710330 (0.0009) [2023-12-27 17:29:39,554][20381] Updated weights for policy 1, policy_version 1708797 (0.0007) [2023-12-27 17:29:39,563][20347] Updated weights for policy 0, policy_version 1710340 (0.0007) [2023-12-27 17:29:39,609][20381] Updated weights for policy 1, policy_version 1708807 (0.0006) [2023-12-27 17:29:40,223][20347] Updated weights for policy 0, policy_version 1710350 (0.0009) [2023-12-27 17:29:40,283][20347] Updated weights for policy 0, policy_version 1710360 (0.0010) [2023-12-27 17:29:40,308][20381] Updated weights for policy 1, policy_version 1708817 (0.0007) [2023-12-27 17:29:40,340][20347] Updated weights for policy 0, policy_version 1710370 (0.0010) [2023-12-27 17:29:40,370][20381] Updated weights for policy 1, policy_version 1708827 (0.0007) [2023-12-27 17:29:40,428][20381] Updated weights for policy 1, policy_version 1708837 (0.0007) [2023-12-27 17:29:40,476][20381] Updated weights for policy 1, policy_version 1708847 (0.0007) [2023-12-27 17:29:40,977][20347] Updated weights for policy 0, policy_version 1710380 (0.0007) [2023-12-27 17:29:41,038][20347] Updated weights for policy 0, policy_version 1710390 (0.0007) [2023-12-27 17:29:41,102][20347] Updated weights for policy 0, policy_version 1710400 (0.0010) [2023-12-27 17:29:41,244][20381] Updated weights for policy 1, policy_version 1708857 (0.0008) [2023-12-27 17:29:41,297][20381] Updated weights for policy 1, policy_version 1708867 (0.0006) [2023-12-27 17:29:41,348][20381] Updated weights for policy 1, policy_version 1708877 (0.0010) [2023-12-27 17:29:41,927][20347] Updated weights for policy 0, policy_version 1710410 (0.0011) [2023-12-27 17:29:41,988][20347] Updated weights for policy 0, policy_version 1710420 (0.0010) [2023-12-27 17:29:42,050][20347] Updated weights for policy 0, policy_version 1710430 (0.0010) [2023-12-27 17:29:42,080][20381] Updated weights for policy 1, policy_version 1708887 (0.0010) [2023-12-27 17:29:42,101][20347] Updated weights for policy 0, policy_version 1710440 (0.0010) [2023-12-27 17:29:42,138][20381] Updated weights for policy 1, policy_version 1708897 (0.0011) [2023-12-27 17:29:42,198][20381] Updated weights for policy 1, policy_version 1708907 (0.0009) [2023-12-27 17:29:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 875479040. Throughput: 0: 10356.0, 1: 10305.2. Samples: 875467832. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:42,363][19170] Avg episode reward: [(0, '970.750'), (1, '1000.000')] [2023-12-27 17:29:42,760][20347] Updated weights for policy 0, policy_version 1710450 (0.0007) [2023-12-27 17:29:42,816][20347] Updated weights for policy 0, policy_version 1710460 (0.0007) [2023-12-27 17:29:42,877][20347] Updated weights for policy 0, policy_version 1710470 (0.0008) [2023-12-27 17:29:42,880][20381] Updated weights for policy 1, policy_version 1708917 (0.0006) [2023-12-27 17:29:42,933][20381] Updated weights for policy 1, policy_version 1708927 (0.0005) [2023-12-27 17:29:42,983][20381] Updated weights for policy 1, policy_version 1708937 (0.0005) [2023-12-27 17:29:43,500][20347] Updated weights for policy 0, policy_version 1710480 (0.0006) [2023-12-27 17:29:43,505][20381] Updated weights for policy 1, policy_version 1708947 (0.0005) [2023-12-27 17:29:43,554][20347] Updated weights for policy 0, policy_version 1710490 (0.0005) [2023-12-27 17:29:43,558][20381] Updated weights for policy 1, policy_version 1708957 (0.0008) [2023-12-27 17:29:43,607][20381] Updated weights for policy 1, policy_version 1708967 (0.0008) [2023-12-27 17:29:43,610][20347] Updated weights for policy 0, policy_version 1710500 (0.0005) [2023-12-27 17:29:44,119][20347] Updated weights for policy 0, policy_version 1710510 (0.0006) [2023-12-27 17:29:44,169][20347] Updated weights for policy 0, policy_version 1710520 (0.0005) [2023-12-27 17:29:44,193][20381] Updated weights for policy 1, policy_version 1708977 (0.0008) [2023-12-27 17:29:44,227][20347] Updated weights for policy 0, policy_version 1710530 (0.0009) [2023-12-27 17:29:44,256][20381] Updated weights for policy 1, policy_version 1708987 (0.0008) [2023-12-27 17:29:44,317][20381] Updated weights for policy 1, policy_version 1708997 (0.0008) [2023-12-27 17:29:44,375][20381] Updated weights for policy 1, policy_version 1709007 (0.0007) [2023-12-27 17:29:44,856][20347] Updated weights for policy 0, policy_version 1710540 (0.0009) [2023-12-27 17:29:44,922][20347] Updated weights for policy 0, policy_version 1710550 (0.0009) [2023-12-27 17:29:44,985][20347] Updated weights for policy 0, policy_version 1710560 (0.0008) [2023-12-27 17:29:45,023][20381] Updated weights for policy 1, policy_version 1709017 (0.0006) [2023-12-27 17:29:45,080][20381] Updated weights for policy 1, policy_version 1709027 (0.0006) [2023-12-27 17:29:45,134][20381] Updated weights for policy 1, policy_version 1709037 (0.0008) [2023-12-27 17:29:45,634][20347] Updated weights for policy 0, policy_version 1710570 (0.0007) [2023-12-27 17:29:45,694][20347] Updated weights for policy 0, policy_version 1710580 (0.0005) [2023-12-27 17:29:45,758][20347] Updated weights for policy 0, policy_version 1710590 (0.0005) [2023-12-27 17:29:45,822][20347] Updated weights for policy 0, policy_version 1710600 (0.0005) [2023-12-27 17:29:45,868][20381] Updated weights for policy 1, policy_version 1709047 (0.0006) [2023-12-27 17:29:45,923][20381] Updated weights for policy 1, policy_version 1709057 (0.0005) [2023-12-27 17:29:45,979][20381] Updated weights for policy 1, policy_version 1709067 (0.0005) [2023-12-27 17:29:46,465][20347] Updated weights for policy 0, policy_version 1710610 (0.0008) [2023-12-27 17:29:46,530][20347] Updated weights for policy 0, policy_version 1710620 (0.0007) [2023-12-27 17:29:46,586][20347] Updated weights for policy 0, policy_version 1710630 (0.0010) [2023-12-27 17:29:46,596][20381] Updated weights for policy 1, policy_version 1709077 (0.0006) [2023-12-27 17:29:46,654][20381] Updated weights for policy 1, policy_version 1709087 (0.0008) [2023-12-27 17:29:46,706][20381] Updated weights for policy 1, policy_version 1709097 (0.0008) [2023-12-27 17:29:47,232][20347] Updated weights for policy 0, policy_version 1710640 (0.0007) [2023-12-27 17:29:47,277][20347] Updated weights for policy 0, policy_version 1710650 (0.0005) [2023-12-27 17:29:47,327][20347] Updated weights for policy 0, policy_version 1710660 (0.0005) [2023-12-27 17:29:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 875585536. Throughput: 0: 10390.4, 1: 10402.3. Samples: 875594480. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:47,363][19170] Avg episode reward: [(0, '931.860'), (1, '991.270')] [2023-12-27 17:29:47,409][20381] Updated weights for policy 1, policy_version 1709107 (0.0007) [2023-12-27 17:29:47,467][20381] Updated weights for policy 1, policy_version 1709117 (0.0007) [2023-12-27 17:29:47,512][20381] Updated weights for policy 1, policy_version 1709127 (0.0008) [2023-12-27 17:29:48,001][20347] Updated weights for policy 0, policy_version 1710670 (0.0007) [2023-12-27 17:29:48,055][20347] Updated weights for policy 0, policy_version 1710680 (0.0009) [2023-12-27 17:29:48,108][20347] Updated weights for policy 0, policy_version 1710690 (0.0008) [2023-12-27 17:29:48,169][20381] Updated weights for policy 1, policy_version 1709137 (0.0008) [2023-12-27 17:29:48,227][20381] Updated weights for policy 1, policy_version 1709147 (0.0007) [2023-12-27 17:29:48,283][20381] Updated weights for policy 1, policy_version 1709157 (0.0005) [2023-12-27 17:29:48,359][20381] Updated weights for policy 1, policy_version 1709167 (0.0007) [2023-12-27 17:29:48,833][20347] Updated weights for policy 0, policy_version 1710700 (0.0008) [2023-12-27 17:29:48,901][20347] Updated weights for policy 0, policy_version 1710710 (0.0008) [2023-12-27 17:29:48,937][20381] Updated weights for policy 1, policy_version 1709177 (0.0007) [2023-12-27 17:29:48,955][20347] Updated weights for policy 0, policy_version 1710720 (0.0006) [2023-12-27 17:29:48,996][20381] Updated weights for policy 1, policy_version 1709187 (0.0007) [2023-12-27 17:29:49,044][20381] Updated weights for policy 1, policy_version 1709197 (0.0007) [2023-12-27 17:29:49,710][20347] Updated weights for policy 0, policy_version 1710730 (0.0006) [2023-12-27 17:29:49,774][20347] Updated weights for policy 0, policy_version 1710740 (0.0006) [2023-12-27 17:29:49,784][20381] Updated weights for policy 1, policy_version 1709207 (0.0007) [2023-12-27 17:29:49,832][20347] Updated weights for policy 0, policy_version 1710750 (0.0007) [2023-12-27 17:29:49,844][20381] Updated weights for policy 1, policy_version 1709217 (0.0006) [2023-12-27 17:29:49,893][20347] Updated weights for policy 0, policy_version 1710760 (0.0007) [2023-12-27 17:29:49,896][20381] Updated weights for policy 1, policy_version 1709227 (0.0008) [2023-12-27 17:29:50,593][20347] Updated weights for policy 0, policy_version 1710770 (0.0008) [2023-12-27 17:29:50,649][20347] Updated weights for policy 0, policy_version 1710780 (0.0007) [2023-12-27 17:29:50,663][20381] Updated weights for policy 1, policy_version 1709237 (0.0008) [2023-12-27 17:29:50,706][20347] Updated weights for policy 0, policy_version 1710790 (0.0006) [2023-12-27 17:29:50,720][20381] Updated weights for policy 1, policy_version 1709247 (0.0008) [2023-12-27 17:29:50,768][20381] Updated weights for policy 1, policy_version 1709257 (0.0008) [2023-12-27 17:29:51,401][20347] Updated weights for policy 0, policy_version 1710800 (0.0007) [2023-12-27 17:29:51,459][20347] Updated weights for policy 0, policy_version 1710810 (0.0006) [2023-12-27 17:29:51,508][20347] Updated weights for policy 0, policy_version 1710820 (0.0007) [2023-12-27 17:29:51,534][20381] Updated weights for policy 1, policy_version 1709267 (0.0007) [2023-12-27 17:29:51,587][20381] Updated weights for policy 1, policy_version 1709277 (0.0007) [2023-12-27 17:29:51,654][20381] Updated weights for policy 1, policy_version 1709287 (0.0008) [2023-12-27 17:29:52,234][20347] Updated weights for policy 0, policy_version 1710830 (0.0008) [2023-12-27 17:29:52,298][20347] Updated weights for policy 0, policy_version 1710840 (0.0007) [2023-12-27 17:29:52,303][20381] Updated weights for policy 1, policy_version 1709297 (0.0007) [2023-12-27 17:29:52,356][20347] Updated weights for policy 0, policy_version 1710850 (0.0006) [2023-12-27 17:29:52,362][20381] Updated weights for policy 1, policy_version 1709307 (0.0007) [2023-12-27 17:29:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 875675648. Throughput: 0: 10303.8, 1: 10374.9. Samples: 875654628. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:52,363][19170] Avg episode reward: [(0, '922.080'), (1, '991.270')] [2023-12-27 17:29:52,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001710856_438059008.pth... [2023-12-27 17:29:52,397][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001709640_437747712.pth [2023-12-27 17:29:52,426][20381] Updated weights for policy 1, policy_version 1709317 (0.0007) [2023-12-27 17:29:52,490][20381] Updated weights for policy 1, policy_version 1709327 (0.0008) [2023-12-27 17:29:52,493][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001709328_437633024.pth... [2023-12-27 17:29:52,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001708112_437321728.pth [2023-12-27 17:29:53,102][20347] Updated weights for policy 0, policy_version 1710860 (0.0007) [2023-12-27 17:29:53,158][20381] Updated weights for policy 1, policy_version 1709337 (0.0007) [2023-12-27 17:29:53,161][20347] Updated weights for policy 0, policy_version 1710870 (0.0007) [2023-12-27 17:29:53,213][20381] Updated weights for policy 1, policy_version 1709347 (0.0006) [2023-12-27 17:29:53,226][20347] Updated weights for policy 0, policy_version 1710880 (0.0005) [2023-12-27 17:29:53,270][20381] Updated weights for policy 1, policy_version 1709357 (0.0007) [2023-12-27 17:29:53,874][20381] Updated weights for policy 1, policy_version 1709367 (0.0005) [2023-12-27 17:29:53,930][20381] Updated weights for policy 1, policy_version 1709377 (0.0007) [2023-12-27 17:29:53,934][20347] Updated weights for policy 0, policy_version 1710890 (0.0007) [2023-12-27 17:29:53,984][20381] Updated weights for policy 1, policy_version 1709387 (0.0005) [2023-12-27 17:29:53,989][20347] Updated weights for policy 0, policy_version 1710900 (0.0007) [2023-12-27 17:29:54,051][20347] Updated weights for policy 0, policy_version 1710910 (0.0007) [2023-12-27 17:29:54,110][20347] Updated weights for policy 0, policy_version 1710920 (0.0008) [2023-12-27 17:29:54,570][20381] Updated weights for policy 1, policy_version 1709397 (0.0007) [2023-12-27 17:29:54,633][20381] Updated weights for policy 1, policy_version 1709407 (0.0008) [2023-12-27 17:29:54,692][20381] Updated weights for policy 1, policy_version 1709417 (0.0008) [2023-12-27 17:29:54,773][20347] Updated weights for policy 0, policy_version 1710930 (0.0008) [2023-12-27 17:29:54,825][20347] Updated weights for policy 0, policy_version 1710940 (0.0007) [2023-12-27 17:29:54,852][20291] KL-divergence is very high: 102.9441 [2023-12-27 17:29:54,880][20347] Updated weights for policy 0, policy_version 1710950 (0.0007) [2023-12-27 17:29:55,381][20381] Updated weights for policy 1, policy_version 1709427 (0.0007) [2023-12-27 17:29:55,443][20381] Updated weights for policy 1, policy_version 1709437 (0.0008) [2023-12-27 17:29:55,505][20381] Updated weights for policy 1, policy_version 1709447 (0.0008) [2023-12-27 17:29:55,584][20347] Updated weights for policy 0, policy_version 1710960 (0.0008) [2023-12-27 17:29:55,629][20347] Updated weights for policy 0, policy_version 1710970 (0.0006) [2023-12-27 17:29:55,674][20347] Updated weights for policy 0, policy_version 1710980 (0.0007) [2023-12-27 17:29:56,215][20381] Updated weights for policy 1, policy_version 1709457 (0.0007) [2023-12-27 17:29:56,273][20381] Updated weights for policy 1, policy_version 1709467 (0.0009) [2023-12-27 17:29:56,326][20381] Updated weights for policy 1, policy_version 1709477 (0.0008) [2023-12-27 17:29:56,347][20347] Updated weights for policy 0, policy_version 1710990 (0.0007) [2023-12-27 17:29:56,379][20381] Updated weights for policy 1, policy_version 1709487 (0.0006) [2023-12-27 17:29:56,403][20347] Updated weights for policy 0, policy_version 1711000 (0.0006) [2023-12-27 17:29:56,453][20347] Updated weights for policy 0, policy_version 1711010 (0.0007) [2023-12-27 17:29:56,981][20381] Updated weights for policy 1, policy_version 1709497 (0.0007) [2023-12-27 17:29:57,036][20381] Updated weights for policy 1, policy_version 1709507 (0.0007) [2023-12-27 17:29:57,090][20381] Updated weights for policy 1, policy_version 1709517 (0.0007) [2023-12-27 17:29:57,125][20347] Updated weights for policy 0, policy_version 1711020 (0.0008) [2023-12-27 17:29:57,182][20347] Updated weights for policy 0, policy_version 1711030 (0.0010) [2023-12-27 17:29:57,231][20347] Updated weights for policy 0, policy_version 1711040 (0.0010) [2023-12-27 17:29:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 875790336. Throughput: 0: 10222.6, 1: 10422.3. Samples: 875775472. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:29:57,363][19170] Avg episode reward: [(0, '941.380'), (1, '1000.000')] [2023-12-27 17:29:57,757][20381] Updated weights for policy 1, policy_version 1709527 (0.0007) [2023-12-27 17:29:57,817][20381] Updated weights for policy 1, policy_version 1709537 (0.0008) [2023-12-27 17:29:57,834][20347] Updated weights for policy 0, policy_version 1711050 (0.0010) [2023-12-27 17:29:57,870][20381] Updated weights for policy 1, policy_version 1709547 (0.0006) [2023-12-27 17:29:57,882][20347] Updated weights for policy 0, policy_version 1711060 (0.0007) [2023-12-27 17:29:57,929][20347] Updated weights for policy 0, policy_version 1711070 (0.0007) [2023-12-27 17:29:57,972][20347] Updated weights for policy 0, policy_version 1711080 (0.0007) [2023-12-27 17:29:58,605][20381] Updated weights for policy 1, policy_version 1709557 (0.0008) [2023-12-27 17:29:58,670][20381] Updated weights for policy 1, policy_version 1709567 (0.0006) [2023-12-27 17:29:58,724][20347] Updated weights for policy 0, policy_version 1711090 (0.0007) [2023-12-27 17:29:58,729][20381] Updated weights for policy 1, policy_version 1709577 (0.0008) [2023-12-27 17:29:58,791][20347] Updated weights for policy 0, policy_version 1711100 (0.0008) [2023-12-27 17:29:58,855][20347] Updated weights for policy 0, policy_version 1711110 (0.0008) [2023-12-27 17:29:59,412][20381] Updated weights for policy 1, policy_version 1709587 (0.0008) [2023-12-27 17:29:59,456][20381] Updated weights for policy 1, policy_version 1709597 (0.0010) [2023-12-27 17:29:59,502][20381] Updated weights for policy 1, policy_version 1709607 (0.0010) [2023-12-27 17:29:59,663][20347] Updated weights for policy 0, policy_version 1711120 (0.0007) [2023-12-27 17:29:59,720][20347] Updated weights for policy 0, policy_version 1711130 (0.0007) [2023-12-27 17:29:59,770][20347] Updated weights for policy 0, policy_version 1711140 (0.0007) [2023-12-27 17:30:00,213][20381] Updated weights for policy 1, policy_version 1709617 (0.0010) [2023-12-27 17:30:00,270][20381] Updated weights for policy 1, policy_version 1709627 (0.0007) [2023-12-27 17:30:00,329][20381] Updated weights for policy 1, policy_version 1709637 (0.0006) [2023-12-27 17:30:00,385][20381] Updated weights for policy 1, policy_version 1709647 (0.0006) [2023-12-27 17:30:00,530][20347] Updated weights for policy 0, policy_version 1711150 (0.0008) [2023-12-27 17:30:00,577][20347] Updated weights for policy 0, policy_version 1711160 (0.0008) [2023-12-27 17:30:00,627][20347] Updated weights for policy 0, policy_version 1711170 (0.0007) [2023-12-27 17:30:00,947][20381] Updated weights for policy 1, policy_version 1709657 (0.0007) [2023-12-27 17:30:01,001][20381] Updated weights for policy 1, policy_version 1709667 (0.0006) [2023-12-27 17:30:01,075][20381] Updated weights for policy 1, policy_version 1709677 (0.0006) [2023-12-27 17:30:01,368][20347] Updated weights for policy 0, policy_version 1711180 (0.0009) [2023-12-27 17:30:01,433][20347] Updated weights for policy 0, policy_version 1711190 (0.0011) [2023-12-27 17:30:01,494][20347] Updated weights for policy 0, policy_version 1711200 (0.0009) [2023-12-27 17:30:01,685][20381] Updated weights for policy 1, policy_version 1709687 (0.0006) [2023-12-27 17:30:01,747][20381] Updated weights for policy 1, policy_version 1709697 (0.0008) [2023-12-27 17:30:01,806][20381] Updated weights for policy 1, policy_version 1709707 (0.0007) [2023-12-27 17:30:02,182][20347] Updated weights for policy 0, policy_version 1711210 (0.0009) [2023-12-27 17:30:02,239][20347] Updated weights for policy 0, policy_version 1711220 (0.0010) [2023-12-27 17:30:02,305][20347] Updated weights for policy 0, policy_version 1711230 (0.0010) [2023-12-27 17:30:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 875880448. Throughput: 0: 10158.4, 1: 10384.9. Samples: 875897132. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:02,363][19170] Avg episode reward: [(0, '970.750'), (1, '1000.000')] [2023-12-27 17:30:02,366][20347] Updated weights for policy 0, policy_version 1711240 (0.0010) [2023-12-27 17:30:02,460][20381] Updated weights for policy 1, policy_version 1709717 (0.0006) [2023-12-27 17:30:02,519][20381] Updated weights for policy 1, policy_version 1709727 (0.0007) [2023-12-27 17:30:02,565][20381] Updated weights for policy 1, policy_version 1709737 (0.0007) [2023-12-27 17:30:03,016][20347] Updated weights for policy 0, policy_version 1711250 (0.0010) [2023-12-27 17:30:03,060][20347] Updated weights for policy 0, policy_version 1711260 (0.0010) [2023-12-27 17:30:03,114][20347] Updated weights for policy 0, policy_version 1711270 (0.0010) [2023-12-27 17:30:03,209][20381] Updated weights for policy 1, policy_version 1709747 (0.0007) [2023-12-27 17:30:03,268][20381] Updated weights for policy 1, policy_version 1709757 (0.0007) [2023-12-27 17:30:03,322][20381] Updated weights for policy 1, policy_version 1709767 (0.0007) [2023-12-27 17:30:03,744][20347] Updated weights for policy 0, policy_version 1711280 (0.0007) [2023-12-27 17:30:03,795][20347] Updated weights for policy 0, policy_version 1711290 (0.0005) [2023-12-27 17:30:03,855][20347] Updated weights for policy 0, policy_version 1711300 (0.0006) [2023-12-27 17:30:04,010][20381] Updated weights for policy 1, policy_version 1709777 (0.0007) [2023-12-27 17:30:04,076][20381] Updated weights for policy 1, policy_version 1709787 (0.0006) [2023-12-27 17:30:04,146][20381] Updated weights for policy 1, policy_version 1709797 (0.0006) [2023-12-27 17:30:04,210][20381] Updated weights for policy 1, policy_version 1709807 (0.0006) [2023-12-27 17:30:04,549][20347] Updated weights for policy 0, policy_version 1711310 (0.0008) [2023-12-27 17:30:04,617][20347] Updated weights for policy 0, policy_version 1711320 (0.0005) [2023-12-27 17:30:04,683][20347] Updated weights for policy 0, policy_version 1711330 (0.0006) [2023-12-27 17:30:04,727][20381] Updated weights for policy 1, policy_version 1709817 (0.0008) [2023-12-27 17:30:04,781][20381] Updated weights for policy 1, policy_version 1709827 (0.0008) [2023-12-27 17:30:04,834][20381] Updated weights for policy 1, policy_version 1709837 (0.0009) [2023-12-27 17:30:05,211][20347] Updated weights for policy 0, policy_version 1711340 (0.0006) [2023-12-27 17:30:05,273][20347] Updated weights for policy 0, policy_version 1711350 (0.0006) [2023-12-27 17:30:05,332][20347] Updated weights for policy 0, policy_version 1711360 (0.0007) [2023-12-27 17:30:05,553][20381] Updated weights for policy 1, policy_version 1709847 (0.0005) [2023-12-27 17:30:05,603][20381] Updated weights for policy 1, policy_version 1709857 (0.0005) [2023-12-27 17:30:05,648][20381] Updated weights for policy 1, policy_version 1709867 (0.0005) [2023-12-27 17:30:06,019][20347] Updated weights for policy 0, policy_version 1711370 (0.0008) [2023-12-27 17:30:06,079][20347] Updated weights for policy 0, policy_version 1711380 (0.0008) [2023-12-27 17:30:06,139][20347] Updated weights for policy 0, policy_version 1711390 (0.0008) [2023-12-27 17:30:06,198][20347] Updated weights for policy 0, policy_version 1711400 (0.0007) [2023-12-27 17:30:06,252][20381] Updated weights for policy 1, policy_version 1709877 (0.0006) [2023-12-27 17:30:06,313][20381] Updated weights for policy 1, policy_version 1709887 (0.0005) [2023-12-27 17:30:06,377][20381] Updated weights for policy 1, policy_version 1709897 (0.0006) [2023-12-27 17:30:06,922][20347] Updated weights for policy 0, policy_version 1711410 (0.0008) [2023-12-27 17:30:06,978][20347] Updated weights for policy 0, policy_version 1711420 (0.0007) [2023-12-27 17:30:07,028][20347] Updated weights for policy 0, policy_version 1711430 (0.0007) [2023-12-27 17:30:07,029][20381] Updated weights for policy 1, policy_version 1709907 (0.0005) [2023-12-27 17:30:07,087][20381] Updated weights for policy 1, policy_version 1709917 (0.0007) [2023-12-27 17:30:07,147][20381] Updated weights for policy 1, policy_version 1709927 (0.0008) [2023-12-27 17:30:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.7, 300 sec: 20521.6). Total num frames: 875995136. Throughput: 0: 10179.9, 1: 10389.1. Samples: 875960760. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:07,363][19170] Avg episode reward: [(0, '980.540'), (1, '1000.000')] [2023-12-27 17:30:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001709936_437788672.pth... [2023-12-27 17:30:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001711432_438206464.pth... [2023-12-27 17:30:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001710248_437903360.pth [2023-12-27 17:30:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001708720_437477376.pth [2023-12-27 17:30:07,739][20347] Updated weights for policy 0, policy_version 1711440 (0.0007) [2023-12-27 17:30:07,765][20381] Updated weights for policy 1, policy_version 1709937 (0.0007) [2023-12-27 17:30:07,785][20347] Updated weights for policy 0, policy_version 1711450 (0.0007) [2023-12-27 17:30:07,820][20381] Updated weights for policy 1, policy_version 1709947 (0.0005) [2023-12-27 17:30:07,840][20347] Updated weights for policy 0, policy_version 1711460 (0.0007) [2023-12-27 17:30:07,875][20381] Updated weights for policy 1, policy_version 1709957 (0.0007) [2023-12-27 17:30:07,934][20381] Updated weights for policy 1, policy_version 1709967 (0.0008) [2023-12-27 17:30:08,538][20347] Updated weights for policy 0, policy_version 1711470 (0.0007) [2023-12-27 17:30:08,591][20347] Updated weights for policy 0, policy_version 1711480 (0.0007) [2023-12-27 17:30:08,593][20381] Updated weights for policy 1, policy_version 1709977 (0.0008) [2023-12-27 17:30:08,645][20381] Updated weights for policy 1, policy_version 1709987 (0.0006) [2023-12-27 17:30:08,647][20347] Updated weights for policy 0, policy_version 1711490 (0.0007) [2023-12-27 17:30:08,703][20381] Updated weights for policy 1, policy_version 1709997 (0.0006) [2023-12-27 17:30:09,389][20347] Updated weights for policy 0, policy_version 1711500 (0.0006) [2023-12-27 17:30:09,441][20381] Updated weights for policy 1, policy_version 1710007 (0.0008) [2023-12-27 17:30:09,454][20347] Updated weights for policy 0, policy_version 1711510 (0.0006) [2023-12-27 17:30:09,500][20381] Updated weights for policy 1, policy_version 1710017 (0.0008) [2023-12-27 17:30:09,509][20347] Updated weights for policy 0, policy_version 1711520 (0.0007) [2023-12-27 17:30:09,556][20381] Updated weights for policy 1, policy_version 1710027 (0.0008) [2023-12-27 17:30:10,269][20347] Updated weights for policy 0, policy_version 1711530 (0.0006) [2023-12-27 17:30:10,282][20381] Updated weights for policy 1, policy_version 1710037 (0.0007) [2023-12-27 17:30:10,333][20347] Updated weights for policy 0, policy_version 1711540 (0.0008) [2023-12-27 17:30:10,347][20381] Updated weights for policy 1, policy_version 1710047 (0.0006) [2023-12-27 17:30:10,398][20347] Updated weights for policy 0, policy_version 1711550 (0.0007) [2023-12-27 17:30:10,411][20381] Updated weights for policy 1, policy_version 1710057 (0.0008) [2023-12-27 17:30:10,455][20347] Updated weights for policy 0, policy_version 1711560 (0.0007) [2023-12-27 17:30:11,096][20381] Updated weights for policy 1, policy_version 1710067 (0.0007) [2023-12-27 17:30:11,146][20381] Updated weights for policy 1, policy_version 1710077 (0.0007) [2023-12-27 17:30:11,160][20347] Updated weights for policy 0, policy_version 1711570 (0.0006) [2023-12-27 17:30:11,210][20381] Updated weights for policy 1, policy_version 1710087 (0.0007) [2023-12-27 17:30:11,221][20347] Updated weights for policy 0, policy_version 1711580 (0.0007) [2023-12-27 17:30:11,279][20347] Updated weights for policy 0, policy_version 1711590 (0.0007) [2023-12-27 17:30:11,968][20381] Updated weights for policy 1, policy_version 1710097 (0.0007) [2023-12-27 17:30:12,005][20347] Updated weights for policy 0, policy_version 1711600 (0.0007) [2023-12-27 17:30:12,030][20381] Updated weights for policy 1, policy_version 1710107 (0.0006) [2023-12-27 17:30:12,063][20347] Updated weights for policy 0, policy_version 1711610 (0.0005) [2023-12-27 17:30:12,094][20381] Updated weights for policy 1, policy_version 1710117 (0.0008) [2023-12-27 17:30:12,121][20347] Updated weights for policy 0, policy_version 1711620 (0.0006) [2023-12-27 17:30:12,154][20381] Updated weights for policy 1, policy_version 1710127 (0.0008) [2023-12-27 17:30:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 876093440. Throughput: 0: 10181.5, 1: 10394.4. Samples: 876081436. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:12,363][19170] Avg episode reward: [(0, '970.730'), (1, '1000.000')] [2023-12-27 17:30:12,806][20347] Updated weights for policy 0, policy_version 1711630 (0.0005) [2023-12-27 17:30:12,807][20381] Updated weights for policy 1, policy_version 1710137 (0.0006) [2023-12-27 17:30:12,864][20381] Updated weights for policy 1, policy_version 1710147 (0.0008) [2023-12-27 17:30:12,865][20347] Updated weights for policy 0, policy_version 1711640 (0.0007) [2023-12-27 17:30:12,913][20347] Updated weights for policy 0, policy_version 1711650 (0.0010) [2023-12-27 17:30:12,918][20381] Updated weights for policy 1, policy_version 1710157 (0.0009) [2023-12-27 17:30:13,527][20381] Updated weights for policy 1, policy_version 1710167 (0.0007) [2023-12-27 17:30:13,571][20381] Updated weights for policy 1, policy_version 1710177 (0.0008) [2023-12-27 17:30:13,584][20347] Updated weights for policy 0, policy_version 1711660 (0.0010) [2023-12-27 17:30:13,617][20381] Updated weights for policy 1, policy_version 1710187 (0.0005) [2023-12-27 17:30:13,632][20347] Updated weights for policy 0, policy_version 1711670 (0.0010) [2023-12-27 17:30:13,683][20347] Updated weights for policy 0, policy_version 1711680 (0.0010) [2023-12-27 17:30:14,149][20381] Updated weights for policy 1, policy_version 1710197 (0.0005) [2023-12-27 17:30:14,203][20381] Updated weights for policy 1, policy_version 1710207 (0.0005) [2023-12-27 17:30:14,267][20381] Updated weights for policy 1, policy_version 1710217 (0.0010) [2023-12-27 17:30:14,374][20347] Updated weights for policy 0, policy_version 1711690 (0.0009) [2023-12-27 17:30:14,436][20347] Updated weights for policy 0, policy_version 1711700 (0.0009) [2023-12-27 17:30:14,488][20347] Updated weights for policy 0, policy_version 1711710 (0.0010) [2023-12-27 17:30:14,539][20347] Updated weights for policy 0, policy_version 1711720 (0.0010) [2023-12-27 17:30:14,919][20381] Updated weights for policy 1, policy_version 1710227 (0.0010) [2023-12-27 17:30:14,982][20381] Updated weights for policy 1, policy_version 1710237 (0.0009) [2023-12-27 17:30:15,041][20381] Updated weights for policy 1, policy_version 1710247 (0.0007) [2023-12-27 17:30:15,183][20347] Updated weights for policy 0, policy_version 1711730 (0.0010) [2023-12-27 17:30:15,242][20347] Updated weights for policy 0, policy_version 1711740 (0.0011) [2023-12-27 17:30:15,304][20347] Updated weights for policy 0, policy_version 1711750 (0.0011) [2023-12-27 17:30:15,696][20381] Updated weights for policy 1, policy_version 1710257 (0.0010) [2023-12-27 17:30:15,752][20381] Updated weights for policy 1, policy_version 1710267 (0.0011) [2023-12-27 17:30:15,808][20381] Updated weights for policy 1, policy_version 1710277 (0.0011) [2023-12-27 17:30:15,865][20381] Updated weights for policy 1, policy_version 1710287 (0.0011) [2023-12-27 17:30:15,925][20347] Updated weights for policy 0, policy_version 1711760 (0.0006) [2023-12-27 17:30:15,974][20347] Updated weights for policy 0, policy_version 1711770 (0.0009) [2023-12-27 17:30:16,019][20347] Updated weights for policy 0, policy_version 1711780 (0.0010) [2023-12-27 17:30:16,545][20381] Updated weights for policy 1, policy_version 1710297 (0.0010) [2023-12-27 17:30:16,593][20381] Updated weights for policy 1, policy_version 1710307 (0.0010) [2023-12-27 17:30:16,641][20381] Updated weights for policy 1, policy_version 1710317 (0.0010) [2023-12-27 17:30:16,697][20347] Updated weights for policy 0, policy_version 1711790 (0.0010) [2023-12-27 17:30:16,748][20347] Updated weights for policy 0, policy_version 1711800 (0.0010) [2023-12-27 17:30:16,796][20347] Updated weights for policy 0, policy_version 1711810 (0.0010) [2023-12-27 17:30:17,280][20381] Updated weights for policy 1, policy_version 1710327 (0.0007) [2023-12-27 17:30:17,342][20381] Updated weights for policy 1, policy_version 1710337 (0.0005) [2023-12-27 17:30:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 876191744. Throughput: 0: 10147.2, 1: 10427.0. Samples: 876207628. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:17,363][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 17:30:17,401][20381] Updated weights for policy 1, policy_version 1710347 (0.0005) [2023-12-27 17:30:17,481][20347] Updated weights for policy 0, policy_version 1711820 (0.0010) [2023-12-27 17:30:17,532][20347] Updated weights for policy 0, policy_version 1711830 (0.0009) [2023-12-27 17:30:17,596][20347] Updated weights for policy 0, policy_version 1711840 (0.0005) [2023-12-27 17:30:17,973][20381] Updated weights for policy 1, policy_version 1710357 (0.0008) [2023-12-27 17:30:18,030][20381] Updated weights for policy 1, policy_version 1710367 (0.0008) [2023-12-27 17:30:18,077][20381] Updated weights for policy 1, policy_version 1710377 (0.0007) [2023-12-27 17:30:18,212][20347] Updated weights for policy 0, policy_version 1711850 (0.0005) [2023-12-27 17:30:18,273][20347] Updated weights for policy 0, policy_version 1711860 (0.0005) [2023-12-27 17:30:18,339][20347] Updated weights for policy 0, policy_version 1711870 (0.0006) [2023-12-27 17:30:18,402][20347] Updated weights for policy 0, policy_version 1711880 (0.0008) [2023-12-27 17:30:18,754][20381] Updated weights for policy 1, policy_version 1710387 (0.0007) [2023-12-27 17:30:18,816][20381] Updated weights for policy 1, policy_version 1710397 (0.0008) [2023-12-27 17:30:18,882][20381] Updated weights for policy 1, policy_version 1710407 (0.0009) [2023-12-27 17:30:19,032][20347] Updated weights for policy 0, policy_version 1711890 (0.0007) [2023-12-27 17:30:19,098][20347] Updated weights for policy 0, policy_version 1711900 (0.0007) [2023-12-27 17:30:19,154][20347] Updated weights for policy 0, policy_version 1711910 (0.0008) [2023-12-27 17:30:19,593][20381] Updated weights for policy 1, policy_version 1710417 (0.0008) [2023-12-27 17:30:19,657][20381] Updated weights for policy 1, policy_version 1710427 (0.0008) [2023-12-27 17:30:19,717][20381] Updated weights for policy 1, policy_version 1710437 (0.0008) [2023-12-27 17:30:19,777][20381] Updated weights for policy 1, policy_version 1710447 (0.0008) [2023-12-27 17:30:19,894][20347] Updated weights for policy 0, policy_version 1711920 (0.0008) [2023-12-27 17:30:19,962][20347] Updated weights for policy 0, policy_version 1711930 (0.0007) [2023-12-27 17:30:20,025][20347] Updated weights for policy 0, policy_version 1711940 (0.0007) [2023-12-27 17:30:20,525][20381] Updated weights for policy 1, policy_version 1710457 (0.0006) [2023-12-27 17:30:20,586][20381] Updated weights for policy 1, policy_version 1710467 (0.0007) [2023-12-27 17:30:20,644][20381] Updated weights for policy 1, policy_version 1710477 (0.0006) [2023-12-27 17:30:20,782][20347] Updated weights for policy 0, policy_version 1711950 (0.0007) [2023-12-27 17:30:20,841][20347] Updated weights for policy 0, policy_version 1711960 (0.0007) [2023-12-27 17:30:20,902][20347] Updated weights for policy 0, policy_version 1711970 (0.0008) [2023-12-27 17:30:21,292][20381] Updated weights for policy 1, policy_version 1710487 (0.0007) [2023-12-27 17:30:21,354][20381] Updated weights for policy 1, policy_version 1710497 (0.0008) [2023-12-27 17:30:21,425][20381] Updated weights for policy 1, policy_version 1710507 (0.0009) [2023-12-27 17:30:21,615][20347] Updated weights for policy 0, policy_version 1711980 (0.0007) [2023-12-27 17:30:21,677][20347] Updated weights for policy 0, policy_version 1711990 (0.0008) [2023-12-27 17:30:21,753][20347] Updated weights for policy 0, policy_version 1712000 (0.0009) [2023-12-27 17:30:22,136][20381] Updated weights for policy 1, policy_version 1710517 (0.0008) [2023-12-27 17:30:22,187][20381] Updated weights for policy 1, policy_version 1710527 (0.0007) [2023-12-27 17:30:22,240][20381] Updated weights for policy 1, policy_version 1710537 (0.0007) [2023-12-27 17:30:22,356][20347] Updated weights for policy 0, policy_version 1712010 (0.0006) [2023-12-27 17:30:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 876298240. Throughput: 0: 10126.9, 1: 10412.0. Samples: 876268040. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:30:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001710544_437944320.pth... [2023-12-27 17:30:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001709328_437633024.pth [2023-12-27 17:30:22,432][20347] Updated weights for policy 0, policy_version 1712020 (0.0008) [2023-12-27 17:30:22,490][20347] Updated weights for policy 0, policy_version 1712030 (0.0007) [2023-12-27 17:30:22,550][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001712040_438362112.pth... [2023-12-27 17:30:22,551][20347] Updated weights for policy 0, policy_version 1712040 (0.0007) [2023-12-27 17:30:22,554][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001710856_438059008.pth [2023-12-27 17:30:22,971][20381] Updated weights for policy 1, policy_version 1710547 (0.0008) [2023-12-27 17:30:23,019][20381] Updated weights for policy 1, policy_version 1710557 (0.0007) [2023-12-27 17:30:23,068][20381] Updated weights for policy 1, policy_version 1710567 (0.0008) [2023-12-27 17:30:23,217][20347] Updated weights for policy 0, policy_version 1712050 (0.0007) [2023-12-27 17:30:23,261][20347] Updated weights for policy 0, policy_version 1712060 (0.0005) [2023-12-27 17:30:23,306][20347] Updated weights for policy 0, policy_version 1712070 (0.0005) [2023-12-27 17:30:23,796][20381] Updated weights for policy 1, policy_version 1710577 (0.0007) [2023-12-27 17:30:23,854][20381] Updated weights for policy 1, policy_version 1710587 (0.0007) [2023-12-27 17:30:23,904][20381] Updated weights for policy 1, policy_version 1710597 (0.0007) [2023-12-27 17:30:23,923][20347] Updated weights for policy 0, policy_version 1712080 (0.0008) [2023-12-27 17:30:23,933][20291] KL-divergence is very high: 615.3383 [2023-12-27 17:30:23,961][20381] Updated weights for policy 1, policy_version 1710607 (0.0008) [2023-12-27 17:30:23,978][20291] KL-divergence is very high: 1144.8473 [2023-12-27 17:30:23,978][20347] Updated weights for policy 0, policy_version 1712090 (0.0006) [2023-12-27 17:30:24,024][20291] KL-divergence is very high: 1250.6559 [2023-12-27 17:30:24,038][20347] Updated weights for policy 0, policy_version 1712100 (0.0007) [2023-12-27 17:30:24,634][20381] Updated weights for policy 1, policy_version 1710617 (0.0011) [2023-12-27 17:30:24,689][20381] Updated weights for policy 1, policy_version 1710627 (0.0010) [2023-12-27 17:30:24,734][20347] Updated weights for policy 0, policy_version 1712110 (0.0006) [2023-12-27 17:30:24,737][20381] Updated weights for policy 1, policy_version 1710637 (0.0010) [2023-12-27 17:30:24,790][20347] Updated weights for policy 0, policy_version 1712120 (0.0007) [2023-12-27 17:30:24,847][20347] Updated weights for policy 0, policy_version 1712130 (0.0007) [2023-12-27 17:30:25,436][20381] Updated weights for policy 1, policy_version 1710647 (0.0011) [2023-12-27 17:30:25,488][20381] Updated weights for policy 1, policy_version 1710657 (0.0010) [2023-12-27 17:30:25,527][20347] Updated weights for policy 0, policy_version 1712140 (0.0008) [2023-12-27 17:30:25,545][20381] Updated weights for policy 1, policy_version 1710667 (0.0011) [2023-12-27 17:30:25,590][20347] Updated weights for policy 0, policy_version 1712150 (0.0007) [2023-12-27 17:30:25,653][20347] Updated weights for policy 0, policy_version 1712160 (0.0007) [2023-12-27 17:30:26,219][20381] Updated weights for policy 1, policy_version 1710677 (0.0009) [2023-12-27 17:30:26,282][20381] Updated weights for policy 1, policy_version 1710687 (0.0008) [2023-12-27 17:30:26,315][20347] Updated weights for policy 0, policy_version 1712170 (0.0007) [2023-12-27 17:30:26,339][20381] Updated weights for policy 1, policy_version 1710697 (0.0008) [2023-12-27 17:30:26,380][20347] Updated weights for policy 0, policy_version 1712180 (0.0007) [2023-12-27 17:30:26,437][20347] Updated weights for policy 0, policy_version 1712190 (0.0007) [2023-12-27 17:30:26,487][20347] Updated weights for policy 0, policy_version 1712200 (0.0006) [2023-12-27 17:30:26,919][20381] Updated weights for policy 1, policy_version 1710707 (0.0007) [2023-12-27 17:30:26,970][20381] Updated weights for policy 1, policy_version 1710717 (0.0005) [2023-12-27 17:30:27,029][20381] Updated weights for policy 1, policy_version 1710727 (0.0005) [2023-12-27 17:30:27,225][20347] Updated weights for policy 0, policy_version 1712210 (0.0008) [2023-12-27 17:30:27,285][20347] Updated weights for policy 0, policy_version 1712220 (0.0008) [2023-12-27 17:30:27,337][20347] Updated weights for policy 0, policy_version 1712230 (0.0010) [2023-12-27 17:30:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 876404736. Throughput: 0: 10106.0, 1: 10373.4. Samples: 876389404. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:27,363][19170] Avg episode reward: [(0, '951.180'), (1, '1000.000')] [2023-12-27 17:30:27,579][20381] Updated weights for policy 1, policy_version 1710737 (0.0006) [2023-12-27 17:30:27,625][20381] Updated weights for policy 1, policy_version 1710747 (0.0007) [2023-12-27 17:30:27,674][20381] Updated weights for policy 1, policy_version 1710757 (0.0005) [2023-12-27 17:30:27,728][20381] Updated weights for policy 1, policy_version 1710767 (0.0008) [2023-12-27 17:30:27,972][20347] Updated weights for policy 0, policy_version 1712240 (0.0008) [2023-12-27 17:30:28,029][20347] Updated weights for policy 0, policy_version 1712250 (0.0006) [2023-12-27 17:30:28,088][20347] Updated weights for policy 0, policy_version 1712260 (0.0005) [2023-12-27 17:30:28,366][20381] Updated weights for policy 1, policy_version 1710777 (0.0008) [2023-12-27 17:30:28,438][20381] Updated weights for policy 1, policy_version 1710787 (0.0008) [2023-12-27 17:30:28,500][20381] Updated weights for policy 1, policy_version 1710797 (0.0008) [2023-12-27 17:30:28,778][20347] Updated weights for policy 0, policy_version 1712270 (0.0007) [2023-12-27 17:30:28,837][20347] Updated weights for policy 0, policy_version 1712280 (0.0008) [2023-12-27 17:30:28,893][20347] Updated weights for policy 0, policy_version 1712290 (0.0008) [2023-12-27 17:30:29,103][20381] Updated weights for policy 1, policy_version 1710807 (0.0006) [2023-12-27 17:30:29,156][20381] Updated weights for policy 1, policy_version 1710817 (0.0005) [2023-12-27 17:30:29,210][20381] Updated weights for policy 1, policy_version 1710827 (0.0006) [2023-12-27 17:30:29,655][20347] Updated weights for policy 0, policy_version 1712300 (0.0008) [2023-12-27 17:30:29,706][20347] Updated weights for policy 0, policy_version 1712310 (0.0007) [2023-12-27 17:30:29,764][20347] Updated weights for policy 0, policy_version 1712320 (0.0007) [2023-12-27 17:30:29,911][20381] Updated weights for policy 1, policy_version 1710837 (0.0008) [2023-12-27 17:30:29,978][20381] Updated weights for policy 1, policy_version 1710847 (0.0008) [2023-12-27 17:30:30,031][20381] Updated weights for policy 1, policy_version 1710857 (0.0008) [2023-12-27 17:30:30,465][20347] Updated weights for policy 0, policy_version 1712330 (0.0008) [2023-12-27 17:30:30,522][20347] Updated weights for policy 0, policy_version 1712340 (0.0007) [2023-12-27 17:30:30,572][20347] Updated weights for policy 0, policy_version 1712350 (0.0008) [2023-12-27 17:30:30,621][20347] Updated weights for policy 0, policy_version 1712360 (0.0008) [2023-12-27 17:30:30,697][20381] Updated weights for policy 1, policy_version 1710867 (0.0007) [2023-12-27 17:30:30,752][20381] Updated weights for policy 1, policy_version 1710877 (0.0007) [2023-12-27 17:30:30,802][20381] Updated weights for policy 1, policy_version 1710887 (0.0008) [2023-12-27 17:30:31,378][20347] Updated weights for policy 0, policy_version 1712370 (0.0008) [2023-12-27 17:30:31,445][20347] Updated weights for policy 0, policy_version 1712380 (0.0009) [2023-12-27 17:30:31,505][20381] Updated weights for policy 1, policy_version 1710897 (0.0007) [2023-12-27 17:30:31,506][20347] Updated weights for policy 0, policy_version 1712390 (0.0007) [2023-12-27 17:30:31,556][20381] Updated weights for policy 1, policy_version 1710907 (0.0008) [2023-12-27 17:30:31,606][20381] Updated weights for policy 1, policy_version 1710917 (0.0009) [2023-12-27 17:30:31,674][20381] Updated weights for policy 1, policy_version 1710927 (0.0008) [2023-12-27 17:30:32,225][20347] Updated weights for policy 0, policy_version 1712400 (0.0007) [2023-12-27 17:30:32,287][20347] Updated weights for policy 0, policy_version 1712410 (0.0008) [2023-12-27 17:30:32,339][20347] Updated weights for policy 0, policy_version 1712420 (0.0007) [2023-12-27 17:30:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 876494848. Throughput: 0: 10014.5, 1: 10395.7. Samples: 876512940. Policy #0 lag: (min: 8.0, avg: 34.8, max: 40.0) [2023-12-27 17:30:32,363][19170] Avg episode reward: [(0, '941.410'), (1, '990.300')] [2023-12-27 17:30:32,389][20381] Updated weights for policy 1, policy_version 1710937 (0.0008) [2023-12-27 17:30:32,448][20381] Updated weights for policy 1, policy_version 1710947 (0.0006) [2023-12-27 17:30:32,503][20381] Updated weights for policy 1, policy_version 1710957 (0.0008) [2023-12-27 17:30:32,980][20347] Updated weights for policy 0, policy_version 1712430 (0.0009) [2023-12-27 17:30:33,028][20347] Updated weights for policy 0, policy_version 1712440 (0.0009) [2023-12-27 17:30:33,074][20347] Updated weights for policy 0, policy_version 1712450 (0.0005) [2023-12-27 17:30:33,156][20381] Updated weights for policy 1, policy_version 1710967 (0.0007) [2023-12-27 17:30:33,208][20381] Updated weights for policy 1, policy_version 1710977 (0.0008) [2023-12-27 17:30:33,274][20381] Updated weights for policy 1, policy_version 1710987 (0.0008) [2023-12-27 17:30:33,691][20347] Updated weights for policy 0, policy_version 1712460 (0.0007) [2023-12-27 17:30:33,746][20347] Updated weights for policy 0, policy_version 1712470 (0.0005) [2023-12-27 17:30:33,801][20347] Updated weights for policy 0, policy_version 1712480 (0.0005) [2023-12-27 17:30:33,818][20381] Updated weights for policy 1, policy_version 1710997 (0.0007) [2023-12-27 17:30:33,865][20381] Updated weights for policy 1, policy_version 1711007 (0.0008) [2023-12-27 17:30:33,921][20381] Updated weights for policy 1, policy_version 1711017 (0.0009) [2023-12-27 17:30:34,406][20347] Updated weights for policy 0, policy_version 1712490 (0.0006) [2023-12-27 17:30:34,466][20347] Updated weights for policy 0, policy_version 1712500 (0.0008) [2023-12-27 17:30:34,518][20347] Updated weights for policy 0, policy_version 1712510 (0.0008) [2023-12-27 17:30:34,564][20347] Updated weights for policy 0, policy_version 1712520 (0.0007) [2023-12-27 17:30:34,673][20381] Updated weights for policy 1, policy_version 1711027 (0.0007) [2023-12-27 17:30:34,722][20381] Updated weights for policy 1, policy_version 1711037 (0.0007) [2023-12-27 17:30:34,776][20381] Updated weights for policy 1, policy_version 1711047 (0.0008) [2023-12-27 17:30:35,289][20347] Updated weights for policy 0, policy_version 1712530 (0.0010) [2023-12-27 17:30:35,341][20347] Updated weights for policy 0, policy_version 1712540 (0.0010) [2023-12-27 17:30:35,398][20347] Updated weights for policy 0, policy_version 1712550 (0.0011) [2023-12-27 17:30:35,492][20381] Updated weights for policy 1, policy_version 1711057 (0.0007) [2023-12-27 17:30:35,547][20381] Updated weights for policy 1, policy_version 1711067 (0.0008) [2023-12-27 17:30:35,599][20381] Updated weights for policy 1, policy_version 1711077 (0.0008) [2023-12-27 17:30:35,643][20381] Updated weights for policy 1, policy_version 1711087 (0.0007) [2023-12-27 17:30:36,086][20347] Updated weights for policy 0, policy_version 1712560 (0.0010) [2023-12-27 17:30:36,148][20347] Updated weights for policy 0, policy_version 1712571 (0.0008) [2023-12-27 17:30:36,203][20347] Updated weights for policy 0, policy_version 1712581 (0.0009) [2023-12-27 17:30:36,371][20381] Updated weights for policy 1, policy_version 1711097 (0.0008) [2023-12-27 17:30:36,424][20381] Updated weights for policy 1, policy_version 1711107 (0.0008) [2023-12-27 17:30:36,478][20381] Updated weights for policy 1, policy_version 1711117 (0.0008) [2023-12-27 17:30:36,892][20347] Updated weights for policy 0, policy_version 1712591 (0.0006) [2023-12-27 17:30:36,953][20347] Updated weights for policy 0, policy_version 1712601 (0.0006) [2023-12-27 17:30:37,009][20347] Updated weights for policy 0, policy_version 1712611 (0.0007) [2023-12-27 17:30:37,170][20381] Updated weights for policy 1, policy_version 1711127 (0.0006) [2023-12-27 17:30:37,229][20381] Updated weights for policy 1, policy_version 1711137 (0.0007) [2023-12-27 17:30:37,283][20381] Updated weights for policy 1, policy_version 1711147 (0.0010) [2023-12-27 17:30:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 876609536. Throughput: 0: 10066.5, 1: 10406.5. Samples: 876575912. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:30:37,363][19170] Avg episode reward: [(0, '970.730'), (1, '990.300')] [2023-12-27 17:30:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001712616_438509568.pth... [2023-12-27 17:30:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001711152_438099968.pth... [2023-12-27 17:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001709936_437788672.pth [2023-12-27 17:30:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001711432_438206464.pth [2023-12-27 17:30:37,714][20347] Updated weights for policy 0, policy_version 1712621 (0.0008) [2023-12-27 17:30:37,771][20347] Updated weights for policy 0, policy_version 1712631 (0.0010) [2023-12-27 17:30:37,825][20347] Updated weights for policy 0, policy_version 1712641 (0.0009) [2023-12-27 17:30:37,950][20381] Updated weights for policy 1, policy_version 1711157 (0.0009) [2023-12-27 17:30:38,002][20381] Updated weights for policy 1, policy_version 1711167 (0.0007) [2023-12-27 17:30:38,048][20381] Updated weights for policy 1, policy_version 1711177 (0.0006) [2023-12-27 17:30:38,449][20347] Updated weights for policy 0, policy_version 1712651 (0.0010) [2023-12-27 17:30:38,509][20347] Updated weights for policy 0, policy_version 1712661 (0.0010) [2023-12-27 17:30:38,575][20347] Updated weights for policy 0, policy_version 1712671 (0.0011) [2023-12-27 17:30:38,852][20381] Updated weights for policy 1, policy_version 1711187 (0.0008) [2023-12-27 17:30:38,905][20381] Updated weights for policy 1, policy_version 1711197 (0.0008) [2023-12-27 17:30:38,958][20381] Updated weights for policy 1, policy_version 1711207 (0.0008) [2023-12-27 17:30:39,290][20347] Updated weights for policy 0, policy_version 1712681 (0.0010) [2023-12-27 17:30:39,339][20347] Updated weights for policy 0, policy_version 1712691 (0.0008) [2023-12-27 17:30:39,410][20347] Updated weights for policy 0, policy_version 1712701 (0.0008) [2023-12-27 17:30:39,471][20347] Updated weights for policy 0, policy_version 1712711 (0.0008) [2023-12-27 17:30:39,718][20381] Updated weights for policy 1, policy_version 1711217 (0.0008) [2023-12-27 17:30:39,776][20381] Updated weights for policy 1, policy_version 1711227 (0.0009) [2023-12-27 17:30:39,839][20381] Updated weights for policy 1, policy_version 1711237 (0.0007) [2023-12-27 17:30:39,900][20381] Updated weights for policy 1, policy_version 1711247 (0.0009) [2023-12-27 17:30:40,266][20347] Updated weights for policy 0, policy_version 1712721 (0.0008) [2023-12-27 17:30:40,330][20347] Updated weights for policy 0, policy_version 1712731 (0.0007) [2023-12-27 17:30:40,393][20347] Updated weights for policy 0, policy_version 1712741 (0.0007) [2023-12-27 17:30:40,580][20381] Updated weights for policy 1, policy_version 1711257 (0.0009) [2023-12-27 17:30:40,640][20381] Updated weights for policy 1, policy_version 1711267 (0.0008) [2023-12-27 17:30:40,696][20381] Updated weights for policy 1, policy_version 1711277 (0.0008) [2023-12-27 17:30:41,055][20347] Updated weights for policy 0, policy_version 1712751 (0.0007) [2023-12-27 17:30:41,125][20347] Updated weights for policy 0, policy_version 1712761 (0.0009) [2023-12-27 17:30:41,199][20347] Updated weights for policy 0, policy_version 1712771 (0.0008) [2023-12-27 17:30:41,542][20381] Updated weights for policy 1, policy_version 1711287 (0.0010) [2023-12-27 17:30:41,600][20381] Updated weights for policy 1, policy_version 1711297 (0.0011) [2023-12-27 17:30:41,676][20381] Updated weights for policy 1, policy_version 1711307 (0.0009) [2023-12-27 17:30:41,979][20347] Updated weights for policy 0, policy_version 1712781 (0.0007) [2023-12-27 17:30:42,040][20347] Updated weights for policy 0, policy_version 1712791 (0.0010) [2023-12-27 17:30:42,103][20347] Updated weights for policy 0, policy_version 1712801 (0.0008) [2023-12-27 17:30:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 876699648. Throughput: 0: 10083.1, 1: 10312.2. Samples: 876693260. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:30:42,363][19170] Avg episode reward: [(0, '970.910'), (1, '1000.000')] [2023-12-27 17:30:42,440][20381] Updated weights for policy 1, policy_version 1711317 (0.0009) [2023-12-27 17:30:42,500][20381] Updated weights for policy 1, policy_version 1711327 (0.0007) [2023-12-27 17:30:42,556][20381] Updated weights for policy 1, policy_version 1711337 (0.0008) [2023-12-27 17:30:42,803][20347] Updated weights for policy 0, policy_version 1712811 (0.0008) [2023-12-27 17:30:42,858][20347] Updated weights for policy 0, policy_version 1712821 (0.0008) [2023-12-27 17:30:42,914][20347] Updated weights for policy 0, policy_version 1712831 (0.0007) [2023-12-27 17:30:43,256][20381] Updated weights for policy 1, policy_version 1711347 (0.0009) [2023-12-27 17:30:43,305][20381] Updated weights for policy 1, policy_version 1711357 (0.0010) [2023-12-27 17:30:43,364][20381] Updated weights for policy 1, policy_version 1711367 (0.0010) [2023-12-27 17:30:43,571][20347] Updated weights for policy 0, policy_version 1712841 (0.0007) [2023-12-27 17:30:43,625][20347] Updated weights for policy 0, policy_version 1712851 (0.0006) [2023-12-27 17:30:43,678][20347] Updated weights for policy 0, policy_version 1712861 (0.0006) [2023-12-27 17:30:43,725][20347] Updated weights for policy 0, policy_version 1712871 (0.0006) [2023-12-27 17:30:44,044][20381] Updated weights for policy 1, policy_version 1711377 (0.0010) [2023-12-27 17:30:44,106][20381] Updated weights for policy 1, policy_version 1711387 (0.0009) [2023-12-27 17:30:44,166][20381] Updated weights for policy 1, policy_version 1711397 (0.0011) [2023-12-27 17:30:44,226][20381] Updated weights for policy 1, policy_version 1711407 (0.0010) [2023-12-27 17:30:44,416][20347] Updated weights for policy 0, policy_version 1712881 (0.0005) [2023-12-27 17:30:44,464][20347] Updated weights for policy 0, policy_version 1712891 (0.0006) [2023-12-27 17:30:44,524][20347] Updated weights for policy 0, policy_version 1712901 (0.0007) [2023-12-27 17:30:44,902][20381] Updated weights for policy 1, policy_version 1711417 (0.0008) [2023-12-27 17:30:44,966][20381] Updated weights for policy 1, policy_version 1711427 (0.0008) [2023-12-27 17:30:45,031][20381] Updated weights for policy 1, policy_version 1711437 (0.0011) [2023-12-27 17:30:45,216][20347] Updated weights for policy 0, policy_version 1712911 (0.0008) [2023-12-27 17:30:45,277][20347] Updated weights for policy 0, policy_version 1712921 (0.0007) [2023-12-27 17:30:45,335][20347] Updated weights for policy 0, policy_version 1712931 (0.0008) [2023-12-27 17:30:45,747][20381] Updated weights for policy 1, policy_version 1711447 (0.0007) [2023-12-27 17:30:45,813][20381] Updated weights for policy 1, policy_version 1711457 (0.0005) [2023-12-27 17:30:45,876][20381] Updated weights for policy 1, policy_version 1711467 (0.0007) [2023-12-27 17:30:46,001][20347] Updated weights for policy 0, policy_version 1712941 (0.0008) [2023-12-27 17:30:46,060][20347] Updated weights for policy 0, policy_version 1712951 (0.0006) [2023-12-27 17:30:46,119][20347] Updated weights for policy 0, policy_version 1712961 (0.0006) [2023-12-27 17:30:46,508][20381] Updated weights for policy 1, policy_version 1711477 (0.0007) [2023-12-27 17:30:46,553][20381] Updated weights for policy 1, policy_version 1711487 (0.0007) [2023-12-27 17:30:46,599][20381] Updated weights for policy 1, policy_version 1711497 (0.0007) [2023-12-27 17:30:46,812][20347] Updated weights for policy 0, policy_version 1712971 (0.0007) [2023-12-27 17:30:46,861][20347] Updated weights for policy 0, policy_version 1712981 (0.0008) [2023-12-27 17:30:46,915][20347] Updated weights for policy 0, policy_version 1712991 (0.0008) [2023-12-27 17:30:47,247][20381] Updated weights for policy 1, policy_version 1711507 (0.0008) [2023-12-27 17:30:47,301][20381] Updated weights for policy 1, policy_version 1711517 (0.0008) [2023-12-27 17:30:47,362][20381] Updated weights for policy 1, policy_version 1711527 (0.0007) [2023-12-27 17:30:47,362][19170] Fps is (10 sec: 18841.6, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 876797952. Throughput: 0: 10075.0, 1: 10256.4. Samples: 876812044. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:30:47,363][19170] Avg episode reward: [(0, '980.670'), (1, '1000.000')] [2023-12-27 17:30:47,678][20347] Updated weights for policy 0, policy_version 1713001 (0.0008) [2023-12-27 17:30:47,742][20347] Updated weights for policy 0, policy_version 1713011 (0.0009) [2023-12-27 17:30:47,797][20347] Updated weights for policy 0, policy_version 1713021 (0.0007) [2023-12-27 17:30:47,861][20347] Updated weights for policy 0, policy_version 1713031 (0.0008) [2023-12-27 17:30:48,048][20381] Updated weights for policy 1, policy_version 1711537 (0.0007) [2023-12-27 17:30:48,111][20381] Updated weights for policy 1, policy_version 1711547 (0.0007) [2023-12-27 17:30:48,176][20381] Updated weights for policy 1, policy_version 1711557 (0.0007) [2023-12-27 17:30:48,239][20381] Updated weights for policy 1, policy_version 1711567 (0.0007) [2023-12-27 17:30:48,594][20347] Updated weights for policy 0, policy_version 1713041 (0.0007) [2023-12-27 17:30:48,658][20347] Updated weights for policy 0, policy_version 1713051 (0.0008) [2023-12-27 17:30:48,715][20347] Updated weights for policy 0, policy_version 1713061 (0.0006) [2023-12-27 17:30:48,947][20381] Updated weights for policy 1, policy_version 1711577 (0.0008) [2023-12-27 17:30:49,002][20381] Updated weights for policy 1, policy_version 1711587 (0.0006) [2023-12-27 17:30:49,054][20381] Updated weights for policy 1, policy_version 1711597 (0.0007) [2023-12-27 17:30:49,374][20347] Updated weights for policy 0, policy_version 1713071 (0.0008) [2023-12-27 17:30:49,437][20347] Updated weights for policy 0, policy_version 1713081 (0.0007) [2023-12-27 17:30:49,494][20347] Updated weights for policy 0, policy_version 1713091 (0.0008) [2023-12-27 17:30:49,769][20381] Updated weights for policy 1, policy_version 1711607 (0.0008) [2023-12-27 17:30:49,832][20381] Updated weights for policy 1, policy_version 1711617 (0.0008) [2023-12-27 17:30:49,890][20381] Updated weights for policy 1, policy_version 1711627 (0.0008) [2023-12-27 17:30:50,239][20347] Updated weights for policy 0, policy_version 1713101 (0.0007) [2023-12-27 17:30:50,302][20347] Updated weights for policy 0, policy_version 1713111 (0.0007) [2023-12-27 17:30:50,366][20347] Updated weights for policy 0, policy_version 1713121 (0.0007) [2023-12-27 17:30:50,677][20381] Updated weights for policy 1, policy_version 1711638 (0.0008) [2023-12-27 17:30:50,745][20381] Updated weights for policy 1, policy_version 1711648 (0.0009) [2023-12-27 17:30:50,810][20381] Updated weights for policy 1, policy_version 1711658 (0.0009) [2023-12-27 17:30:51,098][20347] Updated weights for policy 0, policy_version 1713131 (0.0008) [2023-12-27 17:30:51,166][20347] Updated weights for policy 0, policy_version 1713141 (0.0009) [2023-12-27 17:30:51,225][20347] Updated weights for policy 0, policy_version 1713151 (0.0008) [2023-12-27 17:30:51,503][20381] Updated weights for policy 1, policy_version 1711668 (0.0009) [2023-12-27 17:30:51,555][20381] Updated weights for policy 1, policy_version 1711678 (0.0009) [2023-12-27 17:30:51,605][20381] Updated weights for policy 1, policy_version 1711688 (0.0008) [2023-12-27 17:30:51,916][20347] Updated weights for policy 0, policy_version 1713161 (0.0010) [2023-12-27 17:30:51,976][20347] Updated weights for policy 0, policy_version 1713171 (0.0007) [2023-12-27 17:30:52,033][20347] Updated weights for policy 0, policy_version 1713181 (0.0006) [2023-12-27 17:30:52,097][20347] Updated weights for policy 0, policy_version 1713191 (0.0007) [2023-12-27 17:30:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 876896256. Throughput: 0: 10016.2, 1: 10198.9. Samples: 876870440. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:30:52,363][19170] Avg episode reward: [(0, '980.470'), (1, '990.300')] [2023-12-27 17:30:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001711696_438239232.pth... [2023-12-27 17:30:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001713192_438657024.pth... [2023-12-27 17:30:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001710544_437944320.pth [2023-12-27 17:30:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001712040_438362112.pth [2023-12-27 17:30:52,409][20381] Updated weights for policy 1, policy_version 1711698 (0.0008) [2023-12-27 17:30:52,459][20381] Updated weights for policy 1, policy_version 1711708 (0.0010) [2023-12-27 17:30:52,511][20381] Updated weights for policy 1, policy_version 1711718 (0.0010) [2023-12-27 17:30:52,573][20381] Updated weights for policy 1, policy_version 1711728 (0.0011) [2023-12-27 17:30:52,805][20347] Updated weights for policy 0, policy_version 1713201 (0.0007) [2023-12-27 17:30:52,867][20347] Updated weights for policy 0, policy_version 1713211 (0.0008) [2023-12-27 17:30:52,920][20347] Updated weights for policy 0, policy_version 1713221 (0.0007) [2023-12-27 17:30:53,274][20381] Updated weights for policy 1, policy_version 1711738 (0.0010) [2023-12-27 17:30:53,331][20381] Updated weights for policy 1, policy_version 1711748 (0.0010) [2023-12-27 17:30:53,389][20381] Updated weights for policy 1, policy_version 1711758 (0.0010) [2023-12-27 17:30:53,622][20347] Updated weights for policy 0, policy_version 1713231 (0.0008) [2023-12-27 17:30:53,669][20347] Updated weights for policy 0, policy_version 1713241 (0.0007) [2023-12-27 17:30:53,718][20347] Updated weights for policy 0, policy_version 1713251 (0.0007) [2023-12-27 17:30:54,054][20381] Updated weights for policy 1, policy_version 1711768 (0.0010) [2023-12-27 17:30:54,116][20381] Updated weights for policy 1, policy_version 1711778 (0.0010) [2023-12-27 17:30:54,176][20381] Updated weights for policy 1, policy_version 1711788 (0.0009) [2023-12-27 17:30:54,402][20347] Updated weights for policy 0, policy_version 1713261 (0.0009) [2023-12-27 17:30:54,459][20347] Updated weights for policy 0, policy_version 1713271 (0.0010) [2023-12-27 17:30:54,517][20347] Updated weights for policy 0, policy_version 1713281 (0.0010) [2023-12-27 17:30:54,807][20381] Updated weights for policy 1, policy_version 1711798 (0.0006) [2023-12-27 17:30:54,859][20381] Updated weights for policy 1, policy_version 1711808 (0.0008) [2023-12-27 17:30:54,915][20381] Updated weights for policy 1, policy_version 1711818 (0.0009) [2023-12-27 17:30:55,132][20347] Updated weights for policy 0, policy_version 1713291 (0.0008) [2023-12-27 17:30:55,183][20347] Updated weights for policy 0, policy_version 1713301 (0.0010) [2023-12-27 17:30:55,240][20347] Updated weights for policy 0, policy_version 1713311 (0.0010) [2023-12-27 17:30:55,483][20381] Updated weights for policy 1, policy_version 1711828 (0.0006) [2023-12-27 17:30:55,533][20381] Updated weights for policy 1, policy_version 1711838 (0.0005) [2023-12-27 17:30:55,588][20381] Updated weights for policy 1, policy_version 1711848 (0.0005) [2023-12-27 17:30:55,919][20347] Updated weights for policy 0, policy_version 1713321 (0.0010) [2023-12-27 17:30:55,968][20347] Updated weights for policy 0, policy_version 1713331 (0.0010) [2023-12-27 17:30:56,019][20347] Updated weights for policy 0, policy_version 1713341 (0.0010) [2023-12-27 17:30:56,066][20347] Updated weights for policy 0, policy_version 1713351 (0.0010) [2023-12-27 17:30:56,119][20381] Updated weights for policy 1, policy_version 1711858 (0.0006) [2023-12-27 17:30:56,174][20381] Updated weights for policy 1, policy_version 1711868 (0.0010) [2023-12-27 17:30:56,231][20381] Updated weights for policy 1, policy_version 1711878 (0.0008) [2023-12-27 17:30:56,287][20381] Updated weights for policy 1, policy_version 1711888 (0.0008) [2023-12-27 17:30:56,776][20347] Updated weights for policy 0, policy_version 1713361 (0.0007) [2023-12-27 17:30:56,833][20347] Updated weights for policy 0, policy_version 1713371 (0.0007) [2023-12-27 17:30:56,893][20347] Updated weights for policy 0, policy_version 1713381 (0.0007) [2023-12-27 17:30:56,913][20381] Updated weights for policy 1, policy_version 1711898 (0.0008) [2023-12-27 17:30:56,969][20381] Updated weights for policy 1, policy_version 1711908 (0.0011) [2023-12-27 17:30:57,025][20381] Updated weights for policy 1, policy_version 1711918 (0.0005) [2023-12-27 17:30:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 877002752. Throughput: 0: 10047.7, 1: 10216.0. Samples: 876993304. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:30:57,363][19170] Avg episode reward: [(0, '980.430'), (1, '990.300')] [2023-12-27 17:30:57,614][20347] Updated weights for policy 0, policy_version 1713391 (0.0006) [2023-12-27 17:30:57,630][20381] Updated weights for policy 1, policy_version 1711928 (0.0005) [2023-12-27 17:30:57,661][20347] Updated weights for policy 0, policy_version 1713401 (0.0007) [2023-12-27 17:30:57,680][20381] Updated weights for policy 1, policy_version 1711938 (0.0005) [2023-12-27 17:30:57,715][20347] Updated weights for policy 0, policy_version 1713411 (0.0006) [2023-12-27 17:30:57,735][20381] Updated weights for policy 1, policy_version 1711948 (0.0005) [2023-12-27 17:30:58,364][20381] Updated weights for policy 1, policy_version 1711958 (0.0006) [2023-12-27 17:30:58,437][20381] Updated weights for policy 1, policy_version 1711968 (0.0007) [2023-12-27 17:30:58,450][20347] Updated weights for policy 0, policy_version 1713421 (0.0007) [2023-12-27 17:30:58,500][20381] Updated weights for policy 1, policy_version 1711978 (0.0008) [2023-12-27 17:30:58,512][20347] Updated weights for policy 0, policy_version 1713431 (0.0008) [2023-12-27 17:30:58,576][20347] Updated weights for policy 0, policy_version 1713441 (0.0008) [2023-12-27 17:30:59,211][20381] Updated weights for policy 1, policy_version 1711988 (0.0008) [2023-12-27 17:30:59,284][20381] Updated weights for policy 1, policy_version 1711999 (0.0008) [2023-12-27 17:30:59,331][20347] Updated weights for policy 0, policy_version 1713451 (0.0007) [2023-12-27 17:30:59,350][20381] Updated weights for policy 1, policy_version 1712009 (0.0008) [2023-12-27 17:30:59,402][20347] Updated weights for policy 0, policy_version 1713461 (0.0008) [2023-12-27 17:30:59,465][20347] Updated weights for policy 0, policy_version 1713471 (0.0009) [2023-12-27 17:31:00,025][20381] Updated weights for policy 1, policy_version 1712019 (0.0008) [2023-12-27 17:31:00,078][20381] Updated weights for policy 1, policy_version 1712029 (0.0007) [2023-12-27 17:31:00,134][20381] Updated weights for policy 1, policy_version 1712039 (0.0008) [2023-12-27 17:31:00,174][20347] Updated weights for policy 0, policy_version 1713481 (0.0007) [2023-12-27 17:31:00,237][20347] Updated weights for policy 0, policy_version 1713491 (0.0008) [2023-12-27 17:31:00,296][20347] Updated weights for policy 0, policy_version 1713501 (0.0008) [2023-12-27 17:31:00,346][20347] Updated weights for policy 0, policy_version 1713511 (0.0008) [2023-12-27 17:31:00,811][20381] Updated weights for policy 1, policy_version 1712049 (0.0007) [2023-12-27 17:31:00,866][20381] Updated weights for policy 1, policy_version 1712059 (0.0007) [2023-12-27 17:31:00,917][20381] Updated weights for policy 1, policy_version 1712069 (0.0007) [2023-12-27 17:31:00,965][20381] Updated weights for policy 1, policy_version 1712079 (0.0006) [2023-12-27 17:31:01,065][20347] Updated weights for policy 0, policy_version 1713521 (0.0008) [2023-12-27 17:31:01,127][20347] Updated weights for policy 0, policy_version 1713531 (0.0007) [2023-12-27 17:31:01,190][20347] Updated weights for policy 0, policy_version 1713541 (0.0007) [2023-12-27 17:31:01,650][20381] Updated weights for policy 1, policy_version 1712089 (0.0006) [2023-12-27 17:31:01,706][20381] Updated weights for policy 1, policy_version 1712099 (0.0006) [2023-12-27 17:31:01,800][20381] Updated weights for policy 1, policy_version 1712109 (0.0008) [2023-12-27 17:31:01,884][20347] Updated weights for policy 0, policy_version 1713551 (0.0009) [2023-12-27 17:31:01,937][20347] Updated weights for policy 0, policy_version 1713561 (0.0010) [2023-12-27 17:31:02,001][20347] Updated weights for policy 0, policy_version 1713571 (0.0011) [2023-12-27 17:31:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 877101056. Throughput: 0: 9954.3, 1: 10160.4. Samples: 877112792. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:02,363][19170] Avg episode reward: [(0, '990.220'), (1, '990.270')] [2023-12-27 17:31:02,483][20381] Updated weights for policy 1, policy_version 1712119 (0.0007) [2023-12-27 17:31:02,547][20381] Updated weights for policy 1, policy_version 1712129 (0.0005) [2023-12-27 17:31:02,611][20381] Updated weights for policy 1, policy_version 1712139 (0.0006) [2023-12-27 17:31:02,662][20347] Updated weights for policy 0, policy_version 1713581 (0.0008) [2023-12-27 17:31:02,725][20347] Updated weights for policy 0, policy_version 1713591 (0.0007) [2023-12-27 17:31:02,784][20347] Updated weights for policy 0, policy_version 1713601 (0.0008) [2023-12-27 17:31:03,208][20381] Updated weights for policy 1, policy_version 1712149 (0.0007) [2023-12-27 17:31:03,263][20381] Updated weights for policy 1, policy_version 1712159 (0.0007) [2023-12-27 17:31:03,321][20347] Updated weights for policy 0, policy_version 1713611 (0.0005) [2023-12-27 17:31:03,323][20381] Updated weights for policy 1, policy_version 1712169 (0.0007) [2023-12-27 17:31:03,378][20347] Updated weights for policy 0, policy_version 1713621 (0.0005) [2023-12-27 17:31:03,437][20347] Updated weights for policy 0, policy_version 1713631 (0.0005) [2023-12-27 17:31:03,973][20381] Updated weights for policy 1, policy_version 1712179 (0.0009) [2023-12-27 17:31:04,031][20347] Updated weights for policy 0, policy_version 1713641 (0.0006) [2023-12-27 17:31:04,038][20381] Updated weights for policy 1, policy_version 1712189 (0.0006) [2023-12-27 17:31:04,094][20347] Updated weights for policy 0, policy_version 1713651 (0.0008) [2023-12-27 17:31:04,100][20381] Updated weights for policy 1, policy_version 1712199 (0.0006) [2023-12-27 17:31:04,156][20347] Updated weights for policy 0, policy_version 1713661 (0.0009) [2023-12-27 17:31:04,216][20347] Updated weights for policy 0, policy_version 1713671 (0.0008) [2023-12-27 17:31:04,690][20381] Updated weights for policy 1, policy_version 1712209 (0.0006) [2023-12-27 17:31:04,744][20381] Updated weights for policy 1, policy_version 1712219 (0.0007) [2023-12-27 17:31:04,806][20381] Updated weights for policy 1, policy_version 1712229 (0.0007) [2023-12-27 17:31:04,867][20381] Updated weights for policy 1, policy_version 1712239 (0.0008) [2023-12-27 17:31:04,942][20347] Updated weights for policy 0, policy_version 1713681 (0.0010) [2023-12-27 17:31:05,000][20347] Updated weights for policy 0, policy_version 1713691 (0.0010) [2023-12-27 17:31:05,062][20347] Updated weights for policy 0, policy_version 1713701 (0.0010) [2023-12-27 17:31:05,467][20381] Updated weights for policy 1, policy_version 1712249 (0.0008) [2023-12-27 17:31:05,521][20381] Updated weights for policy 1, policy_version 1712259 (0.0007) [2023-12-27 17:31:05,575][20381] Updated weights for policy 1, policy_version 1712269 (0.0008) [2023-12-27 17:31:05,728][20347] Updated weights for policy 0, policy_version 1713711 (0.0010) [2023-12-27 17:31:05,786][20347] Updated weights for policy 0, policy_version 1713721 (0.0010) [2023-12-27 17:31:05,843][20347] Updated weights for policy 0, policy_version 1713731 (0.0010) [2023-12-27 17:31:06,182][20381] Updated weights for policy 1, policy_version 1712279 (0.0007) [2023-12-27 17:31:06,238][20381] Updated weights for policy 1, policy_version 1712289 (0.0009) [2023-12-27 17:31:06,298][20381] Updated weights for policy 1, policy_version 1712299 (0.0008) [2023-12-27 17:31:06,530][20347] Updated weights for policy 0, policy_version 1713741 (0.0011) [2023-12-27 17:31:06,593][20347] Updated weights for policy 0, policy_version 1713751 (0.0011) [2023-12-27 17:31:06,662][20347] Updated weights for policy 0, policy_version 1713761 (0.0011) [2023-12-27 17:31:07,017][20381] Updated weights for policy 1, policy_version 1712309 (0.0009) [2023-12-27 17:31:07,074][20381] Updated weights for policy 1, policy_version 1712319 (0.0011) [2023-12-27 17:31:07,131][20381] Updated weights for policy 1, policy_version 1712329 (0.0011) [2023-12-27 17:31:07,247][20347] Updated weights for policy 0, policy_version 1713771 (0.0011) [2023-12-27 17:31:07,296][20347] Updated weights for policy 0, policy_version 1713781 (0.0010) [2023-12-27 17:31:07,341][20347] Updated weights for policy 0, policy_version 1713791 (0.0010) [2023-12-27 17:31:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 877207552. Throughput: 0: 10002.3, 1: 10206.2. Samples: 877177424. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:31:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001712336_438403072.pth... [2023-12-27 17:31:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001711152_438099968.pth [2023-12-27 17:31:07,386][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001713800_438812672.pth... [2023-12-27 17:31:07,391][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001712616_438509568.pth [2023-12-27 17:31:07,773][20381] Updated weights for policy 1, policy_version 1712339 (0.0009) [2023-12-27 17:31:07,829][20381] Updated weights for policy 1, policy_version 1712349 (0.0010) [2023-12-27 17:31:07,884][20381] Updated weights for policy 1, policy_version 1712359 (0.0010) [2023-12-27 17:31:08,032][20347] Updated weights for policy 0, policy_version 1713801 (0.0009) [2023-12-27 17:31:08,085][20347] Updated weights for policy 0, policy_version 1713811 (0.0007) [2023-12-27 17:31:08,143][20347] Updated weights for policy 0, policy_version 1713821 (0.0008) [2023-12-27 17:31:08,197][20347] Updated weights for policy 0, policy_version 1713831 (0.0007) [2023-12-27 17:31:08,568][20381] Updated weights for policy 1, policy_version 1712369 (0.0011) [2023-12-27 17:31:08,624][20381] Updated weights for policy 1, policy_version 1712379 (0.0011) [2023-12-27 17:31:08,676][20381] Updated weights for policy 1, policy_version 1712389 (0.0010) [2023-12-27 17:31:08,730][20381] Updated weights for policy 1, policy_version 1712399 (0.0011) [2023-12-27 17:31:08,795][20347] Updated weights for policy 0, policy_version 1713841 (0.0006) [2023-12-27 17:31:08,851][20347] Updated weights for policy 0, policy_version 1713851 (0.0007) [2023-12-27 17:31:08,908][20347] Updated weights for policy 0, policy_version 1713861 (0.0007) [2023-12-27 17:31:09,447][20381] Updated weights for policy 1, policy_version 1712409 (0.0009) [2023-12-27 17:31:09,506][20381] Updated weights for policy 1, policy_version 1712419 (0.0009) [2023-12-27 17:31:09,571][20381] Updated weights for policy 1, policy_version 1712429 (0.0006) [2023-12-27 17:31:09,669][20347] Updated weights for policy 0, policy_version 1713871 (0.0008) [2023-12-27 17:31:09,732][20347] Updated weights for policy 0, policy_version 1713881 (0.0008) [2023-12-27 17:31:09,795][20347] Updated weights for policy 0, policy_version 1713891 (0.0008) [2023-12-27 17:31:10,246][20381] Updated weights for policy 1, policy_version 1712439 (0.0008) [2023-12-27 17:31:10,301][20381] Updated weights for policy 1, policy_version 1712449 (0.0010) [2023-12-27 17:31:10,358][20381] Updated weights for policy 1, policy_version 1712459 (0.0011) [2023-12-27 17:31:10,525][20347] Updated weights for policy 0, policy_version 1713901 (0.0010) [2023-12-27 17:31:10,574][20347] Updated weights for policy 0, policy_version 1713911 (0.0009) [2023-12-27 17:31:10,629][20347] Updated weights for policy 0, policy_version 1713921 (0.0010) [2023-12-27 17:31:11,043][20381] Updated weights for policy 1, policy_version 1712469 (0.0009) [2023-12-27 17:31:11,100][20381] Updated weights for policy 1, policy_version 1712479 (0.0006) [2023-12-27 17:31:11,163][20381] Updated weights for policy 1, policy_version 1712489 (0.0008) [2023-12-27 17:31:11,285][20347] Updated weights for policy 0, policy_version 1713931 (0.0009) [2023-12-27 17:31:11,359][20347] Updated weights for policy 0, policy_version 1713941 (0.0008) [2023-12-27 17:31:11,418][20347] Updated weights for policy 0, policy_version 1713951 (0.0008) [2023-12-27 17:31:11,849][20381] Updated weights for policy 1, policy_version 1712499 (0.0008) [2023-12-27 17:31:11,910][20381] Updated weights for policy 1, policy_version 1712509 (0.0009) [2023-12-27 17:31:11,967][20381] Updated weights for policy 1, policy_version 1712519 (0.0007) [2023-12-27 17:31:12,175][20347] Updated weights for policy 0, policy_version 1713961 (0.0008) [2023-12-27 17:31:12,237][20347] Updated weights for policy 0, policy_version 1713971 (0.0008) [2023-12-27 17:31:12,296][20347] Updated weights for policy 0, policy_version 1713981 (0.0008) [2023-12-27 17:31:12,361][20347] Updated weights for policy 0, policy_version 1713991 (0.0008) [2023-12-27 17:31:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 877305856. Throughput: 0: 10014.4, 1: 10241.9. Samples: 877300936. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:12,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:31:12,664][20381] Updated weights for policy 1, policy_version 1712529 (0.0006) [2023-12-27 17:31:12,722][20381] Updated weights for policy 1, policy_version 1712539 (0.0008) [2023-12-27 17:31:12,781][20381] Updated weights for policy 1, policy_version 1712549 (0.0008) [2023-12-27 17:31:12,840][20381] Updated weights for policy 1, policy_version 1712559 (0.0007) [2023-12-27 17:31:13,063][20347] Updated weights for policy 0, policy_version 1714001 (0.0006) [2023-12-27 17:31:13,114][20347] Updated weights for policy 0, policy_version 1714011 (0.0007) [2023-12-27 17:31:13,161][20347] Updated weights for policy 0, policy_version 1714021 (0.0007) [2023-12-27 17:31:13,477][20381] Updated weights for policy 1, policy_version 1712569 (0.0011) [2023-12-27 17:31:13,519][20381] Updated weights for policy 1, policy_version 1712579 (0.0008) [2023-12-27 17:31:13,566][20381] Updated weights for policy 1, policy_version 1712589 (0.0005) [2023-12-27 17:31:13,909][20347] Updated weights for policy 0, policy_version 1714031 (0.0008) [2023-12-27 17:31:13,963][20347] Updated weights for policy 0, policy_version 1714041 (0.0006) [2023-12-27 17:31:14,013][20347] Updated weights for policy 0, policy_version 1714051 (0.0005) [2023-12-27 17:31:14,228][20381] Updated weights for policy 1, policy_version 1712599 (0.0005) [2023-12-27 17:31:14,285][20381] Updated weights for policy 1, policy_version 1712609 (0.0006) [2023-12-27 17:31:14,329][20381] Updated weights for policy 1, policy_version 1712619 (0.0007) [2023-12-27 17:31:14,694][20347] Updated weights for policy 0, policy_version 1714061 (0.0006) [2023-12-27 17:31:14,750][20347] Updated weights for policy 0, policy_version 1714071 (0.0007) [2023-12-27 17:31:14,819][20347] Updated weights for policy 0, policy_version 1714081 (0.0007) [2023-12-27 17:31:14,918][20381] Updated weights for policy 1, policy_version 1712629 (0.0007) [2023-12-27 17:31:14,973][20381] Updated weights for policy 1, policy_version 1712639 (0.0008) [2023-12-27 17:31:15,033][20381] Updated weights for policy 1, policy_version 1712649 (0.0008) [2023-12-27 17:31:15,442][20347] Updated weights for policy 0, policy_version 1714091 (0.0007) [2023-12-27 17:31:15,496][20347] Updated weights for policy 0, policy_version 1714101 (0.0007) [2023-12-27 17:31:15,548][20347] Updated weights for policy 0, policy_version 1714111 (0.0005) [2023-12-27 17:31:15,775][20381] Updated weights for policy 1, policy_version 1712659 (0.0008) [2023-12-27 17:31:15,830][20381] Updated weights for policy 1, policy_version 1712669 (0.0007) [2023-12-27 17:31:15,886][20381] Updated weights for policy 1, policy_version 1712679 (0.0007) [2023-12-27 17:31:16,193][20347] Updated weights for policy 0, policy_version 1714121 (0.0005) [2023-12-27 17:31:16,245][20347] Updated weights for policy 0, policy_version 1714131 (0.0005) [2023-12-27 17:31:16,305][20347] Updated weights for policy 0, policy_version 1714141 (0.0010) [2023-12-27 17:31:16,363][20347] Updated weights for policy 0, policy_version 1714151 (0.0010) [2023-12-27 17:31:16,535][20381] Updated weights for policy 1, policy_version 1712689 (0.0008) [2023-12-27 17:31:16,592][20381] Updated weights for policy 1, policy_version 1712699 (0.0007) [2023-12-27 17:31:16,647][20381] Updated weights for policy 1, policy_version 1712709 (0.0008) [2023-12-27 17:31:16,701][20381] Updated weights for policy 1, policy_version 1712719 (0.0007) [2023-12-27 17:31:17,008][20347] Updated weights for policy 0, policy_version 1714161 (0.0011) [2023-12-27 17:31:17,067][20347] Updated weights for policy 0, policy_version 1714171 (0.0010) [2023-12-27 17:31:17,128][20347] Updated weights for policy 0, policy_version 1714181 (0.0010) [2023-12-27 17:31:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 877412352. Throughput: 0: 10049.8, 1: 10189.3. Samples: 877423696. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:17,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:31:17,378][20381] Updated weights for policy 1, policy_version 1712729 (0.0008) [2023-12-27 17:31:17,436][20381] Updated weights for policy 1, policy_version 1712739 (0.0006) [2023-12-27 17:31:17,488][20381] Updated weights for policy 1, policy_version 1712749 (0.0008) [2023-12-27 17:31:17,788][20347] Updated weights for policy 0, policy_version 1714191 (0.0008) [2023-12-27 17:31:17,843][20347] Updated weights for policy 0, policy_version 1714201 (0.0010) [2023-12-27 17:31:17,892][20347] Updated weights for policy 0, policy_version 1714211 (0.0011) [2023-12-27 17:31:18,194][20381] Updated weights for policy 1, policy_version 1712759 (0.0008) [2023-12-27 17:31:18,244][20381] Updated weights for policy 1, policy_version 1712769 (0.0008) [2023-12-27 17:31:18,297][20381] Updated weights for policy 1, policy_version 1712779 (0.0009) [2023-12-27 17:31:18,498][20347] Updated weights for policy 0, policy_version 1714221 (0.0009) [2023-12-27 17:31:18,557][20347] Updated weights for policy 0, policy_version 1714231 (0.0007) [2023-12-27 17:31:18,614][20347] Updated weights for policy 0, policy_version 1714241 (0.0007) [2023-12-27 17:31:19,038][20381] Updated weights for policy 1, policy_version 1712789 (0.0008) [2023-12-27 17:31:19,089][20381] Updated weights for policy 1, policy_version 1712799 (0.0007) [2023-12-27 17:31:19,143][20381] Updated weights for policy 1, policy_version 1712809 (0.0008) [2023-12-27 17:31:19,296][20347] Updated weights for policy 0, policy_version 1714251 (0.0008) [2023-12-27 17:31:19,357][20347] Updated weights for policy 0, policy_version 1714261 (0.0009) [2023-12-27 17:31:19,428][20347] Updated weights for policy 0, policy_version 1714271 (0.0008) [2023-12-27 17:31:19,882][20381] Updated weights for policy 1, policy_version 1712819 (0.0008) [2023-12-27 17:31:19,946][20381] Updated weights for policy 1, policy_version 1712829 (0.0007) [2023-12-27 17:31:20,010][20381] Updated weights for policy 1, policy_version 1712839 (0.0006) [2023-12-27 17:31:20,100][20347] Updated weights for policy 0, policy_version 1714281 (0.0010) [2023-12-27 17:31:20,162][20347] Updated weights for policy 0, policy_version 1714291 (0.0011) [2023-12-27 17:31:20,222][20347] Updated weights for policy 0, policy_version 1714301 (0.0010) [2023-12-27 17:31:20,282][20347] Updated weights for policy 0, policy_version 1714311 (0.0009) [2023-12-27 17:31:20,606][20381] Updated weights for policy 1, policy_version 1712849 (0.0006) [2023-12-27 17:31:20,665][20381] Updated weights for policy 1, policy_version 1712859 (0.0008) [2023-12-27 17:31:20,723][20381] Updated weights for policy 1, policy_version 1712869 (0.0008) [2023-12-27 17:31:20,784][20381] Updated weights for policy 1, policy_version 1712879 (0.0007) [2023-12-27 17:31:21,048][20347] Updated weights for policy 0, policy_version 1714321 (0.0008) [2023-12-27 17:31:21,115][20347] Updated weights for policy 0, policy_version 1714331 (0.0009) [2023-12-27 17:31:21,181][20347] Updated weights for policy 0, policy_version 1714341 (0.0009) [2023-12-27 17:31:21,505][20381] Updated weights for policy 1, policy_version 1712889 (0.0006) [2023-12-27 17:31:21,554][20381] Updated weights for policy 1, policy_version 1712899 (0.0006) [2023-12-27 17:31:21,619][20381] Updated weights for policy 1, policy_version 1712909 (0.0007) [2023-12-27 17:31:22,003][20347] Updated weights for policy 0, policy_version 1714351 (0.0008) [2023-12-27 17:31:22,069][20347] Updated weights for policy 0, policy_version 1714361 (0.0008) [2023-12-27 17:31:22,131][20347] Updated weights for policy 0, policy_version 1714371 (0.0008) [2023-12-27 17:31:22,254][20381] Updated weights for policy 1, policy_version 1712919 (0.0008) [2023-12-27 17:31:22,323][20381] Updated weights for policy 1, policy_version 1712929 (0.0006) [2023-12-27 17:31:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 877510656. Throughput: 0: 10036.8, 1: 10153.9. Samples: 877484496. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:31:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001714376_438960128.pth... [2023-12-27 17:31:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001713192_438657024.pth [2023-12-27 17:31:22,384][20381] Updated weights for policy 1, policy_version 1712939 (0.0007) [2023-12-27 17:31:22,411][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001712944_438558720.pth... [2023-12-27 17:31:22,414][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001711696_438239232.pth [2023-12-27 17:31:22,878][20347] Updated weights for policy 0, policy_version 1714381 (0.0008) [2023-12-27 17:31:22,937][20347] Updated weights for policy 0, policy_version 1714391 (0.0007) [2023-12-27 17:31:22,992][20347] Updated weights for policy 0, policy_version 1714401 (0.0010) [2023-12-27 17:31:23,061][20381] Updated weights for policy 1, policy_version 1712949 (0.0007) [2023-12-27 17:31:23,117][20381] Updated weights for policy 1, policy_version 1712959 (0.0008) [2023-12-27 17:31:23,168][20381] Updated weights for policy 1, policy_version 1712969 (0.0008) [2023-12-27 17:31:23,594][20347] Updated weights for policy 0, policy_version 1714411 (0.0008) [2023-12-27 17:31:23,642][20347] Updated weights for policy 0, policy_version 1714421 (0.0005) [2023-12-27 17:31:23,700][20347] Updated weights for policy 0, policy_version 1714431 (0.0005) [2023-12-27 17:31:23,897][20381] Updated weights for policy 1, policy_version 1712979 (0.0007) [2023-12-27 17:31:23,951][20381] Updated weights for policy 1, policy_version 1712989 (0.0006) [2023-12-27 17:31:24,005][20381] Updated weights for policy 1, policy_version 1712999 (0.0005) [2023-12-27 17:31:24,323][20347] Updated weights for policy 0, policy_version 1714441 (0.0008) [2023-12-27 17:31:24,378][20347] Updated weights for policy 0, policy_version 1714451 (0.0010) [2023-12-27 17:31:24,427][20347] Updated weights for policy 0, policy_version 1714461 (0.0010) [2023-12-27 17:31:24,428][20291] KL-divergence is very high: 146.1683 [2023-12-27 17:31:24,464][20291] KL-divergence is very high: 141.5005 [2023-12-27 17:31:24,475][20347] Updated weights for policy 0, policy_version 1714471 (0.0007) [2023-12-27 17:31:24,687][20381] Updated weights for policy 1, policy_version 1713009 (0.0008) [2023-12-27 17:31:24,742][20381] Updated weights for policy 1, policy_version 1713019 (0.0007) [2023-12-27 17:31:24,800][20381] Updated weights for policy 1, policy_version 1713029 (0.0006) [2023-12-27 17:31:24,853][20381] Updated weights for policy 1, policy_version 1713039 (0.0007) [2023-12-27 17:31:25,142][20347] Updated weights for policy 0, policy_version 1714481 (0.0010) [2023-12-27 17:31:25,202][20347] Updated weights for policy 0, policy_version 1714491 (0.0010) [2023-12-27 17:31:25,265][20347] Updated weights for policy 0, policy_version 1714501 (0.0010) [2023-12-27 17:31:25,510][20381] Updated weights for policy 1, policy_version 1713049 (0.0007) [2023-12-27 17:31:25,557][20381] Updated weights for policy 1, policy_version 1713059 (0.0008) [2023-12-27 17:31:25,611][20381] Updated weights for policy 1, policy_version 1713069 (0.0006) [2023-12-27 17:31:25,917][20347] Updated weights for policy 0, policy_version 1714511 (0.0007) [2023-12-27 17:31:25,963][20347] Updated weights for policy 0, policy_version 1714521 (0.0005) [2023-12-27 17:31:26,010][20347] Updated weights for policy 0, policy_version 1714531 (0.0005) [2023-12-27 17:31:26,320][20381] Updated weights for policy 1, policy_version 1713079 (0.0007) [2023-12-27 17:31:26,369][20381] Updated weights for policy 1, policy_version 1713089 (0.0007) [2023-12-27 17:31:26,421][20381] Updated weights for policy 1, policy_version 1713099 (0.0007) [2023-12-27 17:31:26,594][20347] Updated weights for policy 0, policy_version 1714541 (0.0006) [2023-12-27 17:31:26,654][20347] Updated weights for policy 0, policy_version 1714551 (0.0008) [2023-12-27 17:31:26,707][20347] Updated weights for policy 0, policy_version 1714561 (0.0005) [2023-12-27 17:31:27,114][20381] Updated weights for policy 1, policy_version 1713109 (0.0008) [2023-12-27 17:31:27,173][20381] Updated weights for policy 1, policy_version 1713119 (0.0008) [2023-12-27 17:31:27,229][20381] Updated weights for policy 1, policy_version 1713129 (0.0008) [2023-12-27 17:31:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 877617152. Throughput: 0: 10053.4, 1: 10221.1. Samples: 877605612. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:31:27,371][20347] Updated weights for policy 0, policy_version 1714571 (0.0007) [2023-12-27 17:31:27,425][20347] Updated weights for policy 0, policy_version 1714581 (0.0008) [2023-12-27 17:31:27,479][20347] Updated weights for policy 0, policy_version 1714591 (0.0007) [2023-12-27 17:31:27,933][20381] Updated weights for policy 1, policy_version 1713139 (0.0008) [2023-12-27 17:31:27,989][20381] Updated weights for policy 1, policy_version 1713149 (0.0007) [2023-12-27 17:31:28,042][20381] Updated weights for policy 1, policy_version 1713159 (0.0007) [2023-12-27 17:31:28,179][20347] Updated weights for policy 0, policy_version 1714601 (0.0007) [2023-12-27 17:31:28,233][20347] Updated weights for policy 0, policy_version 1714611 (0.0006) [2023-12-27 17:31:28,281][20347] Updated weights for policy 0, policy_version 1714621 (0.0007) [2023-12-27 17:31:28,341][20347] Updated weights for policy 0, policy_version 1714631 (0.0007) [2023-12-27 17:31:28,644][20381] Updated weights for policy 1, policy_version 1713169 (0.0006) [2023-12-27 17:31:28,706][20381] Updated weights for policy 1, policy_version 1713179 (0.0006) [2023-12-27 17:31:28,761][20381] Updated weights for policy 1, policy_version 1713189 (0.0011) [2023-12-27 17:31:28,819][20381] Updated weights for policy 1, policy_version 1713199 (0.0011) [2023-12-27 17:31:29,011][20347] Updated weights for policy 0, policy_version 1714641 (0.0010) [2023-12-27 17:31:29,067][20347] Updated weights for policy 0, policy_version 1714651 (0.0006) [2023-12-27 17:31:29,120][20347] Updated weights for policy 0, policy_version 1714661 (0.0005) [2023-12-27 17:31:29,487][20381] Updated weights for policy 1, policy_version 1713209 (0.0007) [2023-12-27 17:31:29,538][20381] Updated weights for policy 1, policy_version 1713219 (0.0005) [2023-12-27 17:31:29,594][20381] Updated weights for policy 1, policy_version 1713229 (0.0007) [2023-12-27 17:31:29,792][20347] Updated weights for policy 0, policy_version 1714671 (0.0009) [2023-12-27 17:31:29,858][20347] Updated weights for policy 0, policy_version 1714681 (0.0010) [2023-12-27 17:31:29,922][20347] Updated weights for policy 0, policy_version 1714691 (0.0010) [2023-12-27 17:31:30,191][20381] Updated weights for policy 1, policy_version 1713239 (0.0007) [2023-12-27 17:31:30,249][20381] Updated weights for policy 1, policy_version 1713249 (0.0010) [2023-12-27 17:31:30,303][20381] Updated weights for policy 1, policy_version 1713259 (0.0010) [2023-12-27 17:31:30,610][20347] Updated weights for policy 0, policy_version 1714701 (0.0007) [2023-12-27 17:31:30,676][20347] Updated weights for policy 0, policy_version 1714711 (0.0005) [2023-12-27 17:31:30,732][20347] Updated weights for policy 0, policy_version 1714721 (0.0009) [2023-12-27 17:31:30,899][20381] Updated weights for policy 1, policy_version 1713269 (0.0010) [2023-12-27 17:31:30,948][20381] Updated weights for policy 1, policy_version 1713279 (0.0010) [2023-12-27 17:31:31,001][20381] Updated weights for policy 1, policy_version 1713289 (0.0010) [2023-12-27 17:31:31,299][20347] Updated weights for policy 0, policy_version 1714731 (0.0007) [2023-12-27 17:31:31,359][20347] Updated weights for policy 0, policy_version 1714741 (0.0007) [2023-12-27 17:31:31,426][20347] Updated weights for policy 0, policy_version 1714751 (0.0007) [2023-12-27 17:31:31,819][20381] Updated weights for policy 1, policy_version 1713299 (0.0007) [2023-12-27 17:31:31,881][20381] Updated weights for policy 1, policy_version 1713309 (0.0006) [2023-12-27 17:31:31,932][20381] Updated weights for policy 1, policy_version 1713319 (0.0007) [2023-12-27 17:31:32,163][20347] Updated weights for policy 0, policy_version 1714761 (0.0008) [2023-12-27 17:31:32,221][20347] Updated weights for policy 0, policy_version 1714771 (0.0008) [2023-12-27 17:31:32,293][20347] Updated weights for policy 0, policy_version 1714781 (0.0007) [2023-12-27 17:31:32,360][20347] Updated weights for policy 0, policy_version 1714791 (0.0007) [2023-12-27 17:31:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 877715456. Throughput: 0: 10127.8, 1: 10281.8. Samples: 877730476. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:32,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 17:31:32,525][20381] Updated weights for policy 1, policy_version 1713329 (0.0007) [2023-12-27 17:31:32,578][20381] Updated weights for policy 1, policy_version 1713339 (0.0007) [2023-12-27 17:31:32,628][20381] Updated weights for policy 1, policy_version 1713349 (0.0008) [2023-12-27 17:31:32,673][20381] Updated weights for policy 1, policy_version 1713359 (0.0007) [2023-12-27 17:31:33,024][20347] Updated weights for policy 0, policy_version 1714801 (0.0007) [2023-12-27 17:31:33,069][20347] Updated weights for policy 0, policy_version 1714811 (0.0008) [2023-12-27 17:31:33,114][20347] Updated weights for policy 0, policy_version 1714821 (0.0007) [2023-12-27 17:31:33,363][20381] Updated weights for policy 1, policy_version 1713369 (0.0008) [2023-12-27 17:31:33,412][20381] Updated weights for policy 1, policy_version 1713379 (0.0008) [2023-12-27 17:31:33,462][20381] Updated weights for policy 1, policy_version 1713389 (0.0007) [2023-12-27 17:31:33,822][20347] Updated weights for policy 0, policy_version 1714831 (0.0007) [2023-12-27 17:31:33,866][20347] Updated weights for policy 0, policy_version 1714841 (0.0007) [2023-12-27 17:31:33,923][20347] Updated weights for policy 0, policy_version 1714851 (0.0007) [2023-12-27 17:31:34,201][20381] Updated weights for policy 1, policy_version 1713399 (0.0006) [2023-12-27 17:31:34,268][20381] Updated weights for policy 1, policy_version 1713409 (0.0005) [2023-12-27 17:31:34,334][20381] Updated weights for policy 1, policy_version 1713419 (0.0005) [2023-12-27 17:31:34,659][20347] Updated weights for policy 0, policy_version 1714861 (0.0006) [2023-12-27 17:31:34,726][20347] Updated weights for policy 0, policy_version 1714871 (0.0008) [2023-12-27 17:31:34,788][20347] Updated weights for policy 0, policy_version 1714881 (0.0008) [2023-12-27 17:31:34,897][20381] Updated weights for policy 1, policy_version 1713429 (0.0006) [2023-12-27 17:31:34,960][20381] Updated weights for policy 1, policy_version 1713439 (0.0007) [2023-12-27 17:31:35,010][20381] Updated weights for policy 1, policy_version 1713449 (0.0008) [2023-12-27 17:31:35,436][20347] Updated weights for policy 0, policy_version 1714891 (0.0008) [2023-12-27 17:31:35,498][20347] Updated weights for policy 0, policy_version 1714901 (0.0006) [2023-12-27 17:31:35,556][20347] Updated weights for policy 0, policy_version 1714911 (0.0006) [2023-12-27 17:31:35,687][20381] Updated weights for policy 1, policy_version 1713459 (0.0007) [2023-12-27 17:31:35,749][20381] Updated weights for policy 1, policy_version 1713469 (0.0008) [2023-12-27 17:31:35,798][20381] Updated weights for policy 1, policy_version 1713479 (0.0007) [2023-12-27 17:31:36,170][20347] Updated weights for policy 0, policy_version 1714921 (0.0008) [2023-12-27 17:31:36,232][20347] Updated weights for policy 0, policy_version 1714931 (0.0007) [2023-12-27 17:31:36,296][20347] Updated weights for policy 0, policy_version 1714941 (0.0007) [2023-12-27 17:31:36,362][20347] Updated weights for policy 0, policy_version 1714951 (0.0006) [2023-12-27 17:31:36,492][20381] Updated weights for policy 1, policy_version 1713489 (0.0006) [2023-12-27 17:31:36,550][20381] Updated weights for policy 1, policy_version 1713499 (0.0008) [2023-12-27 17:31:36,607][20381] Updated weights for policy 1, policy_version 1713509 (0.0008) [2023-12-27 17:31:36,664][20381] Updated weights for policy 1, policy_version 1713519 (0.0008) [2023-12-27 17:31:37,076][20347] Updated weights for policy 0, policy_version 1714961 (0.0008) [2023-12-27 17:31:37,132][20347] Updated weights for policy 0, policy_version 1714971 (0.0007) [2023-12-27 17:31:37,180][20347] Updated weights for policy 0, policy_version 1714981 (0.0007) [2023-12-27 17:31:37,231][20381] Updated weights for policy 1, policy_version 1713529 (0.0007) [2023-12-27 17:31:37,289][20381] Updated weights for policy 1, policy_version 1713539 (0.0008) [2023-12-27 17:31:37,349][20381] Updated weights for policy 1, policy_version 1713549 (0.0008) [2023-12-27 17:31:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 877821952. Throughput: 0: 10133.4, 1: 10327.3. Samples: 877791172. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:37,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:31:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001713552_438714368.pth... [2023-12-27 17:31:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001714984_439115776.pth... [2023-12-27 17:31:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001712336_438403072.pth [2023-12-27 17:31:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001713800_438812672.pth [2023-12-27 17:31:37,895][20347] Updated weights for policy 0, policy_version 1714991 (0.0007) [2023-12-27 17:31:37,959][20347] Updated weights for policy 0, policy_version 1715001 (0.0008) [2023-12-27 17:31:38,000][20381] Updated weights for policy 1, policy_version 1713559 (0.0006) [2023-12-27 17:31:38,020][20347] Updated weights for policy 0, policy_version 1715011 (0.0006) [2023-12-27 17:31:38,056][20381] Updated weights for policy 1, policy_version 1713569 (0.0006) [2023-12-27 17:31:38,110][20381] Updated weights for policy 1, policy_version 1713579 (0.0008) [2023-12-27 17:31:38,701][20347] Updated weights for policy 0, policy_version 1715021 (0.0007) [2023-12-27 17:31:38,752][20347] Updated weights for policy 0, policy_version 1715031 (0.0008) [2023-12-27 17:31:38,809][20347] Updated weights for policy 0, policy_version 1715041 (0.0008) [2023-12-27 17:31:38,811][20381] Updated weights for policy 1, policy_version 1713589 (0.0007) [2023-12-27 17:31:38,866][20381] Updated weights for policy 1, policy_version 1713599 (0.0006) [2023-12-27 17:31:38,917][20381] Updated weights for policy 1, policy_version 1713609 (0.0008) [2023-12-27 17:31:39,516][20347] Updated weights for policy 0, policy_version 1715051 (0.0007) [2023-12-27 17:31:39,569][20347] Updated weights for policy 0, policy_version 1715061 (0.0007) [2023-12-27 17:31:39,626][20347] Updated weights for policy 0, policy_version 1715071 (0.0008) [2023-12-27 17:31:39,712][20381] Updated weights for policy 1, policy_version 1713619 (0.0008) [2023-12-27 17:31:39,774][20381] Updated weights for policy 1, policy_version 1713629 (0.0008) [2023-12-27 17:31:39,842][20381] Updated weights for policy 1, policy_version 1713639 (0.0009) [2023-12-27 17:31:40,303][20347] Updated weights for policy 0, policy_version 1715081 (0.0008) [2023-12-27 17:31:40,364][20347] Updated weights for policy 0, policy_version 1715091 (0.0008) [2023-12-27 17:31:40,424][20347] Updated weights for policy 0, policy_version 1715101 (0.0008) [2023-12-27 17:31:40,483][20347] Updated weights for policy 0, policy_version 1715111 (0.0007) [2023-12-27 17:31:40,581][20381] Updated weights for policy 1, policy_version 1713649 (0.0008) [2023-12-27 17:31:40,639][20381] Updated weights for policy 1, policy_version 1713659 (0.0005) [2023-12-27 17:31:40,687][20381] Updated weights for policy 1, policy_version 1713669 (0.0006) [2023-12-27 17:31:40,739][20381] Updated weights for policy 1, policy_version 1713679 (0.0006) [2023-12-27 17:31:41,197][20347] Updated weights for policy 0, policy_version 1715121 (0.0008) [2023-12-27 17:31:41,259][20347] Updated weights for policy 0, policy_version 1715131 (0.0008) [2023-12-27 17:31:41,318][20347] Updated weights for policy 0, policy_version 1715141 (0.0008) [2023-12-27 17:31:41,403][20381] Updated weights for policy 1, policy_version 1713689 (0.0008) [2023-12-27 17:31:41,462][20381] Updated weights for policy 1, policy_version 1713699 (0.0007) [2023-12-27 17:31:41,519][20381] Updated weights for policy 1, policy_version 1713709 (0.0008) [2023-12-27 17:31:42,090][20347] Updated weights for policy 0, policy_version 1715151 (0.0008) [2023-12-27 17:31:42,150][20347] Updated weights for policy 0, policy_version 1715161 (0.0008) [2023-12-27 17:31:42,207][20347] Updated weights for policy 0, policy_version 1715171 (0.0008) [2023-12-27 17:31:42,216][20381] Updated weights for policy 1, policy_version 1713719 (0.0007) [2023-12-27 17:31:42,276][20381] Updated weights for policy 1, policy_version 1713729 (0.0009) [2023-12-27 17:31:42,342][20381] Updated weights for policy 1, policy_version 1713739 (0.0008) [2023-12-27 17:31:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 877920256. Throughput: 0: 10138.3, 1: 10299.9. Samples: 877913024. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:31:42,944][20347] Updated weights for policy 0, policy_version 1715181 (0.0007) [2023-12-27 17:31:42,980][20381] Updated weights for policy 1, policy_version 1713749 (0.0006) [2023-12-27 17:31:43,005][20347] Updated weights for policy 0, policy_version 1715191 (0.0006) [2023-12-27 17:31:43,030][20381] Updated weights for policy 1, policy_version 1713759 (0.0006) [2023-12-27 17:31:43,062][20347] Updated weights for policy 0, policy_version 1715201 (0.0007) [2023-12-27 17:31:43,085][20381] Updated weights for policy 1, policy_version 1713769 (0.0006) [2023-12-27 17:31:43,745][20381] Updated weights for policy 1, policy_version 1713779 (0.0006) [2023-12-27 17:31:43,776][20347] Updated weights for policy 0, policy_version 1715211 (0.0007) [2023-12-27 17:31:43,794][20381] Updated weights for policy 1, policy_version 1713789 (0.0008) [2023-12-27 17:31:43,835][20347] Updated weights for policy 0, policy_version 1715221 (0.0007) [2023-12-27 17:31:43,850][20381] Updated weights for policy 1, policy_version 1713799 (0.0005) [2023-12-27 17:31:43,882][20347] Updated weights for policy 0, policy_version 1715231 (0.0006) [2023-12-27 17:31:44,534][20347] Updated weights for policy 0, policy_version 1715241 (0.0009) [2023-12-27 17:31:44,558][20381] Updated weights for policy 1, policy_version 1713809 (0.0008) [2023-12-27 17:31:44,587][20347] Updated weights for policy 0, policy_version 1715251 (0.0006) [2023-12-27 17:31:44,613][20381] Updated weights for policy 1, policy_version 1713819 (0.0008) [2023-12-27 17:31:44,634][20347] Updated weights for policy 0, policy_version 1715261 (0.0008) [2023-12-27 17:31:44,672][20381] Updated weights for policy 1, policy_version 1713829 (0.0009) [2023-12-27 17:31:44,682][20347] Updated weights for policy 0, policy_version 1715271 (0.0006) [2023-12-27 17:31:44,732][20381] Updated weights for policy 1, policy_version 1713839 (0.0007) [2023-12-27 17:31:45,375][20381] Updated weights for policy 1, policy_version 1713849 (0.0006) [2023-12-27 17:31:45,426][20381] Updated weights for policy 1, policy_version 1713859 (0.0005) [2023-12-27 17:31:45,431][20347] Updated weights for policy 0, policy_version 1715281 (0.0007) [2023-12-27 17:31:45,476][20381] Updated weights for policy 1, policy_version 1713869 (0.0006) [2023-12-27 17:31:45,492][20347] Updated weights for policy 0, policy_version 1715291 (0.0006) [2023-12-27 17:31:45,544][20347] Updated weights for policy 0, policy_version 1715301 (0.0005) [2023-12-27 17:31:46,105][20381] Updated weights for policy 1, policy_version 1713879 (0.0008) [2023-12-27 17:31:46,158][20381] Updated weights for policy 1, policy_version 1713889 (0.0006) [2023-12-27 17:31:46,165][20347] Updated weights for policy 0, policy_version 1715311 (0.0007) [2023-12-27 17:31:46,215][20381] Updated weights for policy 1, policy_version 1713899 (0.0006) [2023-12-27 17:31:46,230][20347] Updated weights for policy 0, policy_version 1715321 (0.0008) [2023-12-27 17:31:46,291][20347] Updated weights for policy 0, policy_version 1715331 (0.0008) [2023-12-27 17:31:46,890][20381] Updated weights for policy 1, policy_version 1713909 (0.0006) [2023-12-27 17:31:46,938][20381] Updated weights for policy 1, policy_version 1713919 (0.0009) [2023-12-27 17:31:46,977][20347] Updated weights for policy 0, policy_version 1715341 (0.0008) [2023-12-27 17:31:46,992][20381] Updated weights for policy 1, policy_version 1713929 (0.0006) [2023-12-27 17:31:47,037][20347] Updated weights for policy 0, policy_version 1715351 (0.0008) [2023-12-27 17:31:47,094][20347] Updated weights for policy 0, policy_version 1715361 (0.0008) [2023-12-27 17:31:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 878026752. Throughput: 0: 10180.8, 1: 10310.5. Samples: 878034900. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:47,363][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 17:31:47,628][20381] Updated weights for policy 1, policy_version 1713939 (0.0006) [2023-12-27 17:31:47,677][20381] Updated weights for policy 1, policy_version 1713949 (0.0006) [2023-12-27 17:31:47,733][20381] Updated weights for policy 1, policy_version 1713959 (0.0008) [2023-12-27 17:31:47,802][20347] Updated weights for policy 0, policy_version 1715371 (0.0008) [2023-12-27 17:31:47,852][20347] Updated weights for policy 0, policy_version 1715381 (0.0009) [2023-12-27 17:31:47,910][20347] Updated weights for policy 0, policy_version 1715391 (0.0006) [2023-12-27 17:31:48,330][20381] Updated weights for policy 1, policy_version 1713969 (0.0006) [2023-12-27 17:31:48,391][20381] Updated weights for policy 1, policy_version 1713979 (0.0007) [2023-12-27 17:31:48,451][20381] Updated weights for policy 1, policy_version 1713989 (0.0005) [2023-12-27 17:31:48,512][20381] Updated weights for policy 1, policy_version 1713999 (0.0005) [2023-12-27 17:31:48,665][20347] Updated weights for policy 0, policy_version 1715401 (0.0009) [2023-12-27 17:31:48,723][20347] Updated weights for policy 0, policy_version 1715411 (0.0008) [2023-12-27 17:31:48,786][20347] Updated weights for policy 0, policy_version 1715421 (0.0007) [2023-12-27 17:31:48,842][20347] Updated weights for policy 0, policy_version 1715431 (0.0008) [2023-12-27 17:31:49,059][20381] Updated weights for policy 1, policy_version 1714009 (0.0005) [2023-12-27 17:31:49,114][20381] Updated weights for policy 1, policy_version 1714019 (0.0006) [2023-12-27 17:31:49,165][20381] Updated weights for policy 1, policy_version 1714029 (0.0007) [2023-12-27 17:31:49,644][20347] Updated weights for policy 0, policy_version 1715441 (0.0008) [2023-12-27 17:31:49,715][20347] Updated weights for policy 0, policy_version 1715451 (0.0009) [2023-12-27 17:31:49,786][20347] Updated weights for policy 0, policy_version 1715461 (0.0008) [2023-12-27 17:31:49,803][20381] Updated weights for policy 1, policy_version 1714039 (0.0007) [2023-12-27 17:31:49,868][20381] Updated weights for policy 1, policy_version 1714049 (0.0009) [2023-12-27 17:31:49,928][20381] Updated weights for policy 1, policy_version 1714059 (0.0007) [2023-12-27 17:31:50,476][20347] Updated weights for policy 0, policy_version 1715471 (0.0007) [2023-12-27 17:31:50,531][20347] Updated weights for policy 0, policy_version 1715481 (0.0007) [2023-12-27 17:31:50,580][20381] Updated weights for policy 1, policy_version 1714069 (0.0007) [2023-12-27 17:31:50,596][20347] Updated weights for policy 0, policy_version 1715491 (0.0007) [2023-12-27 17:31:50,636][20381] Updated weights for policy 1, policy_version 1714079 (0.0006) [2023-12-27 17:31:50,689][20381] Updated weights for policy 1, policy_version 1714089 (0.0006) [2023-12-27 17:31:51,247][20347] Updated weights for policy 0, policy_version 1715501 (0.0008) [2023-12-27 17:31:51,308][20347] Updated weights for policy 0, policy_version 1715511 (0.0008) [2023-12-27 17:31:51,373][20347] Updated weights for policy 0, policy_version 1715521 (0.0008) [2023-12-27 17:31:51,444][20381] Updated weights for policy 1, policy_version 1714099 (0.0007) [2023-12-27 17:31:51,504][20381] Updated weights for policy 1, policy_version 1714109 (0.0006) [2023-12-27 17:31:51,567][20381] Updated weights for policy 1, policy_version 1714119 (0.0007) [2023-12-27 17:31:52,125][20347] Updated weights for policy 0, policy_version 1715531 (0.0008) [2023-12-27 17:31:52,186][20347] Updated weights for policy 0, policy_version 1715541 (0.0008) [2023-12-27 17:31:52,245][20347] Updated weights for policy 0, policy_version 1715551 (0.0008) [2023-12-27 17:31:52,296][20381] Updated weights for policy 1, policy_version 1714129 (0.0007) [2023-12-27 17:31:52,361][20381] Updated weights for policy 1, policy_version 1714139 (0.0008) [2023-12-27 17:31:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 878125056. Throughput: 0: 10086.9, 1: 10325.5. Samples: 878095984. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:52,363][19170] Avg episode reward: [(0, '980.520'), (1, '1000.000')] [2023-12-27 17:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001715560_439263232.pth... [2023-12-27 17:31:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001714376_438960128.pth [2023-12-27 17:31:52,420][20381] Updated weights for policy 1, policy_version 1714149 (0.0008) [2023-12-27 17:31:52,475][20381] Updated weights for policy 1, policy_version 1714159 (0.0007) [2023-12-27 17:31:52,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001714160_438870016.pth... [2023-12-27 17:31:52,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001712944_438558720.pth [2023-12-27 17:31:52,879][20347] Updated weights for policy 0, policy_version 1715561 (0.0007) [2023-12-27 17:31:52,924][20347] Updated weights for policy 0, policy_version 1715571 (0.0005) [2023-12-27 17:31:52,973][20347] Updated weights for policy 0, policy_version 1715581 (0.0005) [2023-12-27 17:31:53,022][20347] Updated weights for policy 0, policy_version 1715591 (0.0006) [2023-12-27 17:31:53,105][20381] Updated weights for policy 1, policy_version 1714169 (0.0008) [2023-12-27 17:31:53,162][20381] Updated weights for policy 1, policy_version 1714179 (0.0008) [2023-12-27 17:31:53,216][20381] Updated weights for policy 1, policy_version 1714189 (0.0007) [2023-12-27 17:31:53,708][20347] Updated weights for policy 0, policy_version 1715601 (0.0008) [2023-12-27 17:31:53,765][20347] Updated weights for policy 0, policy_version 1715611 (0.0007) [2023-12-27 17:31:53,814][20347] Updated weights for policy 0, policy_version 1715621 (0.0008) [2023-12-27 17:31:53,882][20381] Updated weights for policy 1, policy_version 1714199 (0.0008) [2023-12-27 17:31:53,944][20381] Updated weights for policy 1, policy_version 1714209 (0.0006) [2023-12-27 17:31:54,005][20381] Updated weights for policy 1, policy_version 1714219 (0.0006) [2023-12-27 17:31:54,532][20347] Updated weights for policy 0, policy_version 1715631 (0.0008) [2023-12-27 17:31:54,589][20381] Updated weights for policy 1, policy_version 1714229 (0.0008) [2023-12-27 17:31:54,591][20347] Updated weights for policy 0, policy_version 1715641 (0.0006) [2023-12-27 17:31:54,644][20381] Updated weights for policy 1, policy_version 1714239 (0.0009) [2023-12-27 17:31:54,646][20347] Updated weights for policy 0, policy_version 1715651 (0.0006) [2023-12-27 17:31:54,696][20381] Updated weights for policy 1, policy_version 1714249 (0.0008) [2023-12-27 17:31:55,347][20347] Updated weights for policy 0, policy_version 1715661 (0.0007) [2023-12-27 17:31:55,349][20381] Updated weights for policy 1, policy_version 1714259 (0.0007) [2023-12-27 17:31:55,394][20347] Updated weights for policy 0, policy_version 1715671 (0.0005) [2023-12-27 17:31:55,406][20381] Updated weights for policy 1, policy_version 1714269 (0.0010) [2023-12-27 17:31:55,441][20347] Updated weights for policy 0, policy_version 1715681 (0.0006) [2023-12-27 17:31:55,464][20381] Updated weights for policy 1, policy_version 1714279 (0.0010) [2023-12-27 17:31:56,147][20347] Updated weights for policy 0, policy_version 1715691 (0.0006) [2023-12-27 17:31:56,149][20381] Updated weights for policy 1, policy_version 1714289 (0.0011) [2023-12-27 17:31:56,199][20347] Updated weights for policy 0, policy_version 1715701 (0.0005) [2023-12-27 17:31:56,205][20381] Updated weights for policy 1, policy_version 1714299 (0.0011) [2023-12-27 17:31:56,256][20347] Updated weights for policy 0, policy_version 1715711 (0.0006) [2023-12-27 17:31:56,261][20381] Updated weights for policy 1, policy_version 1714309 (0.0010) [2023-12-27 17:31:56,313][20381] Updated weights for policy 1, policy_version 1714319 (0.0010) [2023-12-27 17:31:56,953][20347] Updated weights for policy 0, policy_version 1715721 (0.0005) [2023-12-27 17:31:56,996][20381] Updated weights for policy 1, policy_version 1714329 (0.0011) [2023-12-27 17:31:57,009][20347] Updated weights for policy 0, policy_version 1715731 (0.0005) [2023-12-27 17:31:57,055][20381] Updated weights for policy 1, policy_version 1714339 (0.0011) [2023-12-27 17:31:57,058][20347] Updated weights for policy 0, policy_version 1715741 (0.0005) [2023-12-27 17:31:57,103][20381] Updated weights for policy 1, policy_version 1714349 (0.0010) [2023-12-27 17:31:57,116][20347] Updated weights for policy 0, policy_version 1715751 (0.0007) [2023-12-27 17:31:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 878231552. Throughput: 0: 10058.0, 1: 10330.7. Samples: 878218424. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:31:57,363][19170] Avg episode reward: [(0, '990.310'), (1, '1000.000')] [2023-12-27 17:31:57,775][20347] Updated weights for policy 0, policy_version 1715761 (0.0005) [2023-12-27 17:31:57,796][20381] Updated weights for policy 1, policy_version 1714359 (0.0010) [2023-12-27 17:31:57,828][20347] Updated weights for policy 0, policy_version 1715771 (0.0005) [2023-12-27 17:31:57,840][20381] Updated weights for policy 1, policy_version 1714369 (0.0010) [2023-12-27 17:31:57,886][20347] Updated weights for policy 0, policy_version 1715781 (0.0005) [2023-12-27 17:31:57,889][20381] Updated weights for policy 1, policy_version 1714379 (0.0010) [2023-12-27 17:31:58,622][20347] Updated weights for policy 0, policy_version 1715791 (0.0007) [2023-12-27 17:31:58,626][20381] Updated weights for policy 1, policy_version 1714389 (0.0009) [2023-12-27 17:31:58,687][20381] Updated weights for policy 1, policy_version 1714399 (0.0008) [2023-12-27 17:31:58,689][20347] Updated weights for policy 0, policy_version 1715801 (0.0007) [2023-12-27 17:31:58,756][20381] Updated weights for policy 1, policy_version 1714409 (0.0008) [2023-12-27 17:31:58,762][20347] Updated weights for policy 0, policy_version 1715811 (0.0008) [2023-12-27 17:31:59,534][20381] Updated weights for policy 1, policy_version 1714419 (0.0008) [2023-12-27 17:31:59,566][20347] Updated weights for policy 0, policy_version 1715821 (0.0007) [2023-12-27 17:31:59,593][20381] Updated weights for policy 1, policy_version 1714429 (0.0006) [2023-12-27 17:31:59,625][20347] Updated weights for policy 0, policy_version 1715831 (0.0006) [2023-12-27 17:31:59,653][20381] Updated weights for policy 1, policy_version 1714439 (0.0009) [2023-12-27 17:31:59,680][20347] Updated weights for policy 0, policy_version 1715841 (0.0006) [2023-12-27 17:32:00,347][20347] Updated weights for policy 0, policy_version 1715851 (0.0006) [2023-12-27 17:32:00,404][20381] Updated weights for policy 1, policy_version 1714449 (0.0007) [2023-12-27 17:32:00,409][20347] Updated weights for policy 0, policy_version 1715861 (0.0005) [2023-12-27 17:32:00,464][20347] Updated weights for policy 0, policy_version 1715871 (0.0006) [2023-12-27 17:32:00,471][20381] Updated weights for policy 1, policy_version 1714459 (0.0007) [2023-12-27 17:32:00,523][20381] Updated weights for policy 1, policy_version 1714469 (0.0008) [2023-12-27 17:32:00,566][20381] Updated weights for policy 1, policy_version 1714479 (0.0007) [2023-12-27 17:32:01,081][20347] Updated weights for policy 0, policy_version 1715881 (0.0005) [2023-12-27 17:32:01,136][20347] Updated weights for policy 0, policy_version 1715891 (0.0008) [2023-12-27 17:32:01,197][20347] Updated weights for policy 0, policy_version 1715901 (0.0008) [2023-12-27 17:32:01,226][20381] Updated weights for policy 1, policy_version 1714489 (0.0007) [2023-12-27 17:32:01,255][20347] Updated weights for policy 0, policy_version 1715911 (0.0007) [2023-12-27 17:32:01,289][20381] Updated weights for policy 1, policy_version 1714499 (0.0007) [2023-12-27 17:32:01,350][20381] Updated weights for policy 1, policy_version 1714509 (0.0007) [2023-12-27 17:32:01,961][20347] Updated weights for policy 0, policy_version 1715921 (0.0008) [2023-12-27 17:32:02,009][20381] Updated weights for policy 1, policy_version 1714519 (0.0009) [2023-12-27 17:32:02,025][20347] Updated weights for policy 0, policy_version 1715931 (0.0006) [2023-12-27 17:32:02,065][20381] Updated weights for policy 1, policy_version 1714529 (0.0011) [2023-12-27 17:32:02,078][20347] Updated weights for policy 0, policy_version 1715941 (0.0007) [2023-12-27 17:32:02,120][20381] Updated weights for policy 1, policy_version 1714539 (0.0011) [2023-12-27 17:32:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 878329856. Throughput: 0: 10039.6, 1: 10270.1. Samples: 878337636. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:32:02,709][20347] Updated weights for policy 0, policy_version 1715951 (0.0008) [2023-12-27 17:32:02,763][20347] Updated weights for policy 0, policy_version 1715961 (0.0007) [2023-12-27 17:32:02,772][20381] Updated weights for policy 1, policy_version 1714549 (0.0008) [2023-12-27 17:32:02,817][20347] Updated weights for policy 0, policy_version 1715971 (0.0007) [2023-12-27 17:32:02,832][20381] Updated weights for policy 1, policy_version 1714559 (0.0007) [2023-12-27 17:32:02,889][20381] Updated weights for policy 1, policy_version 1714569 (0.0009) [2023-12-27 17:32:03,476][20381] Updated weights for policy 1, policy_version 1714579 (0.0009) [2023-12-27 17:32:03,539][20381] Updated weights for policy 1, policy_version 1714589 (0.0006) [2023-12-27 17:32:03,549][20347] Updated weights for policy 0, policy_version 1715981 (0.0007) [2023-12-27 17:32:03,591][20381] Updated weights for policy 1, policy_version 1714599 (0.0006) [2023-12-27 17:32:03,596][20347] Updated weights for policy 0, policy_version 1715991 (0.0006) [2023-12-27 17:32:03,646][20347] Updated weights for policy 0, policy_version 1716001 (0.0006) [2023-12-27 17:32:04,280][20381] Updated weights for policy 1, policy_version 1714609 (0.0005) [2023-12-27 17:32:04,286][20347] Updated weights for policy 0, policy_version 1716011 (0.0007) [2023-12-27 17:32:04,337][20381] Updated weights for policy 1, policy_version 1714619 (0.0008) [2023-12-27 17:32:04,350][20347] Updated weights for policy 0, policy_version 1716021 (0.0008) [2023-12-27 17:32:04,395][20381] Updated weights for policy 1, policy_version 1714629 (0.0008) [2023-12-27 17:32:04,416][20347] Updated weights for policy 0, policy_version 1716031 (0.0008) [2023-12-27 17:32:04,445][20381] Updated weights for policy 1, policy_version 1714639 (0.0008) [2023-12-27 17:32:05,079][20347] Updated weights for policy 0, policy_version 1716041 (0.0008) [2023-12-27 17:32:05,124][20381] Updated weights for policy 1, policy_version 1714649 (0.0010) [2023-12-27 17:32:05,137][20347] Updated weights for policy 0, policy_version 1716051 (0.0007) [2023-12-27 17:32:05,179][20381] Updated weights for policy 1, policy_version 1714659 (0.0010) [2023-12-27 17:32:05,192][20347] Updated weights for policy 0, policy_version 1716061 (0.0005) [2023-12-27 17:32:05,233][20381] Updated weights for policy 1, policy_version 1714669 (0.0010) [2023-12-27 17:32:05,249][20347] Updated weights for policy 0, policy_version 1716071 (0.0006) [2023-12-27 17:32:05,808][20381] Updated weights for policy 1, policy_version 1714679 (0.0006) [2023-12-27 17:32:05,854][20381] Updated weights for policy 1, policy_version 1714689 (0.0005) [2023-12-27 17:32:05,900][20381] Updated weights for policy 1, policy_version 1714699 (0.0005) [2023-12-27 17:32:05,983][20347] Updated weights for policy 0, policy_version 1716081 (0.0007) [2023-12-27 17:32:06,032][20347] Updated weights for policy 0, policy_version 1716091 (0.0005) [2023-12-27 17:32:06,082][20347] Updated weights for policy 0, policy_version 1716101 (0.0005) [2023-12-27 17:32:06,549][20381] Updated weights for policy 1, policy_version 1714709 (0.0006) [2023-12-27 17:32:06,600][20381] Updated weights for policy 1, policy_version 1714719 (0.0010) [2023-12-27 17:32:06,655][20381] Updated weights for policy 1, policy_version 1714729 (0.0009) [2023-12-27 17:32:06,712][20347] Updated weights for policy 0, policy_version 1716111 (0.0006) [2023-12-27 17:32:06,770][20347] Updated weights for policy 0, policy_version 1716121 (0.0008) [2023-12-27 17:32:06,826][20347] Updated weights for policy 0, policy_version 1716131 (0.0008) [2023-12-27 17:32:07,325][20381] Updated weights for policy 1, policy_version 1714739 (0.0007) [2023-12-27 17:32:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 878428160. Throughput: 0: 10031.0, 1: 10319.4. Samples: 878400260. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:32:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001716136_439410688.pth... [2023-12-27 17:32:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001714984_439115776.pth [2023-12-27 17:32:07,379][20381] Updated weights for policy 1, policy_version 1714749 (0.0006) [2023-12-27 17:32:07,435][20381] Updated weights for policy 1, policy_version 1714759 (0.0007) [2023-12-27 17:32:07,481][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001714768_439025664.pth... [2023-12-27 17:32:07,485][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001713552_438714368.pth [2023-12-27 17:32:07,522][20347] Updated weights for policy 0, policy_version 1716141 (0.0008) [2023-12-27 17:32:07,567][20347] Updated weights for policy 0, policy_version 1716151 (0.0007) [2023-12-27 17:32:07,612][20347] Updated weights for policy 0, policy_version 1716161 (0.0008) [2023-12-27 17:32:08,050][20381] Updated weights for policy 1, policy_version 1714769 (0.0007) [2023-12-27 17:32:08,108][20381] Updated weights for policy 1, policy_version 1714779 (0.0005) [2023-12-27 17:32:08,161][20381] Updated weights for policy 1, policy_version 1714789 (0.0005) [2023-12-27 17:32:08,217][20381] Updated weights for policy 1, policy_version 1714799 (0.0005) [2023-12-27 17:32:08,372][20347] Updated weights for policy 0, policy_version 1716171 (0.0007) [2023-12-27 17:32:08,437][20347] Updated weights for policy 0, policy_version 1716181 (0.0007) [2023-12-27 17:32:08,501][20347] Updated weights for policy 0, policy_version 1716191 (0.0008) [2023-12-27 17:32:08,874][20381] Updated weights for policy 1, policy_version 1714809 (0.0008) [2023-12-27 17:32:08,934][20381] Updated weights for policy 1, policy_version 1714819 (0.0008) [2023-12-27 17:32:08,987][20381] Updated weights for policy 1, policy_version 1714829 (0.0007) [2023-12-27 17:32:09,201][20347] Updated weights for policy 0, policy_version 1716201 (0.0007) [2023-12-27 17:32:09,263][20347] Updated weights for policy 0, policy_version 1716211 (0.0008) [2023-12-27 17:32:09,322][20347] Updated weights for policy 0, policy_version 1716221 (0.0008) [2023-12-27 17:32:09,383][20347] Updated weights for policy 0, policy_version 1716231 (0.0007) [2023-12-27 17:32:09,649][20381] Updated weights for policy 1, policy_version 1714839 (0.0008) [2023-12-27 17:32:09,714][20381] Updated weights for policy 1, policy_version 1714849 (0.0008) [2023-12-27 17:32:09,775][20381] Updated weights for policy 1, policy_version 1714859 (0.0008) [2023-12-27 17:32:10,165][20347] Updated weights for policy 0, policy_version 1716241 (0.0008) [2023-12-27 17:32:10,215][20347] Updated weights for policy 0, policy_version 1716251 (0.0009) [2023-12-27 17:32:10,271][20347] Updated weights for policy 0, policy_version 1716261 (0.0008) [2023-12-27 17:32:10,448][20381] Updated weights for policy 1, policy_version 1714869 (0.0007) [2023-12-27 17:32:10,507][20381] Updated weights for policy 1, policy_version 1714879 (0.0008) [2023-12-27 17:32:10,557][20381] Updated weights for policy 1, policy_version 1714889 (0.0006) [2023-12-27 17:32:10,941][20347] Updated weights for policy 0, policy_version 1716271 (0.0008) [2023-12-27 17:32:10,992][20347] Updated weights for policy 0, policy_version 1716281 (0.0007) [2023-12-27 17:32:11,057][20347] Updated weights for policy 0, policy_version 1716291 (0.0008) [2023-12-27 17:32:11,226][20381] Updated weights for policy 1, policy_version 1714899 (0.0005) [2023-12-27 17:32:11,294][20381] Updated weights for policy 1, policy_version 1714909 (0.0009) [2023-12-27 17:32:11,358][20381] Updated weights for policy 1, policy_version 1714919 (0.0008) [2023-12-27 17:32:11,916][20347] Updated weights for policy 0, policy_version 1716301 (0.0007) [2023-12-27 17:32:11,982][20347] Updated weights for policy 0, policy_version 1716311 (0.0008) [2023-12-27 17:32:12,044][20347] Updated weights for policy 0, policy_version 1716321 (0.0008) [2023-12-27 17:32:12,085][20381] Updated weights for policy 1, policy_version 1714929 (0.0008) [2023-12-27 17:32:12,146][20381] Updated weights for policy 1, policy_version 1714939 (0.0008) [2023-12-27 17:32:12,212][20381] Updated weights for policy 1, policy_version 1714949 (0.0007) [2023-12-27 17:32:12,274][20381] Updated weights for policy 1, policy_version 1714959 (0.0007) [2023-12-27 17:32:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 878534656. Throughput: 0: 9997.0, 1: 10400.4. Samples: 878523496. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:12,363][19170] Avg episode reward: [(0, '980.580'), (1, '1000.000')] [2023-12-27 17:32:12,752][20347] Updated weights for policy 0, policy_version 1716331 (0.0007) [2023-12-27 17:32:12,815][20347] Updated weights for policy 0, policy_version 1716341 (0.0009) [2023-12-27 17:32:12,868][20347] Updated weights for policy 0, policy_version 1716351 (0.0008) [2023-12-27 17:32:12,871][20381] Updated weights for policy 1, policy_version 1714969 (0.0006) [2023-12-27 17:32:12,929][20381] Updated weights for policy 1, policy_version 1714979 (0.0006) [2023-12-27 17:32:12,985][20381] Updated weights for policy 1, policy_version 1714989 (0.0007) [2023-12-27 17:32:13,557][20381] Updated weights for policy 1, policy_version 1714999 (0.0007) [2023-12-27 17:32:13,585][20347] Updated weights for policy 0, policy_version 1716361 (0.0008) [2023-12-27 17:32:13,613][20381] Updated weights for policy 1, policy_version 1715009 (0.0007) [2023-12-27 17:32:13,633][20347] Updated weights for policy 0, policy_version 1716371 (0.0010) [2023-12-27 17:32:13,671][20381] Updated weights for policy 1, policy_version 1715019 (0.0006) [2023-12-27 17:32:13,688][20347] Updated weights for policy 0, policy_version 1716381 (0.0010) [2023-12-27 17:32:13,744][20347] Updated weights for policy 0, policy_version 1716391 (0.0010) [2023-12-27 17:32:14,271][20381] Updated weights for policy 1, policy_version 1715029 (0.0005) [2023-12-27 17:32:14,324][20381] Updated weights for policy 1, policy_version 1715039 (0.0005) [2023-12-27 17:32:14,342][20347] Updated weights for policy 0, policy_version 1716401 (0.0009) [2023-12-27 17:32:14,381][20381] Updated weights for policy 1, policy_version 1715049 (0.0005) [2023-12-27 17:32:14,398][20347] Updated weights for policy 0, policy_version 1716411 (0.0008) [2023-12-27 17:32:14,453][20347] Updated weights for policy 0, policy_version 1716421 (0.0007) [2023-12-27 17:32:15,048][20381] Updated weights for policy 1, policy_version 1715059 (0.0006) [2023-12-27 17:32:15,103][20381] Updated weights for policy 1, policy_version 1715069 (0.0008) [2023-12-27 17:32:15,163][20381] Updated weights for policy 1, policy_version 1715079 (0.0008) [2023-12-27 17:32:15,173][20347] Updated weights for policy 0, policy_version 1716431 (0.0006) [2023-12-27 17:32:15,231][20347] Updated weights for policy 0, policy_version 1716441 (0.0007) [2023-12-27 17:32:15,289][20347] Updated weights for policy 0, policy_version 1716451 (0.0008) [2023-12-27 17:32:15,747][20381] Updated weights for policy 1, policy_version 1715089 (0.0008) [2023-12-27 17:32:15,802][20381] Updated weights for policy 1, policy_version 1715099 (0.0007) [2023-12-27 17:32:15,851][20381] Updated weights for policy 1, policy_version 1715109 (0.0007) [2023-12-27 17:32:15,898][20381] Updated weights for policy 1, policy_version 1715119 (0.0005) [2023-12-27 17:32:15,970][20347] Updated weights for policy 0, policy_version 1716461 (0.0008) [2023-12-27 17:32:16,031][20347] Updated weights for policy 0, policy_version 1716471 (0.0005) [2023-12-27 17:32:16,087][20347] Updated weights for policy 0, policy_version 1716481 (0.0005) [2023-12-27 17:32:16,574][20381] Updated weights for policy 1, policy_version 1715129 (0.0007) [2023-12-27 17:32:16,632][20381] Updated weights for policy 1, policy_version 1715139 (0.0007) [2023-12-27 17:32:16,637][20347] Updated weights for policy 0, policy_version 1716491 (0.0005) [2023-12-27 17:32:16,683][20347] Updated weights for policy 0, policy_version 1716501 (0.0005) [2023-12-27 17:32:16,684][20381] Updated weights for policy 1, policy_version 1715149 (0.0007) [2023-12-27 17:32:16,738][20347] Updated weights for policy 0, policy_version 1716511 (0.0006) [2023-12-27 17:32:17,315][20347] Updated weights for policy 0, policy_version 1716521 (0.0005) [2023-12-27 17:32:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 878632960. Throughput: 0: 9988.0, 1: 10427.9. Samples: 878649192. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:17,363][19170] Avg episode reward: [(0, '980.580'), (1, '990.270')] [2023-12-27 17:32:17,375][20347] Updated weights for policy 0, policy_version 1716531 (0.0006) [2023-12-27 17:32:17,432][20347] Updated weights for policy 0, policy_version 1716541 (0.0005) [2023-12-27 17:32:17,432][20381] Updated weights for policy 1, policy_version 1715159 (0.0008) [2023-12-27 17:32:17,484][20381] Updated weights for policy 1, policy_version 1715169 (0.0007) [2023-12-27 17:32:17,492][20347] Updated weights for policy 0, policy_version 1716551 (0.0005) [2023-12-27 17:32:17,537][20381] Updated weights for policy 1, policy_version 1715179 (0.0007) [2023-12-27 17:32:18,103][20347] Updated weights for policy 0, policy_version 1716561 (0.0007) [2023-12-27 17:32:18,161][20347] Updated weights for policy 0, policy_version 1716571 (0.0007) [2023-12-27 17:32:18,211][20347] Updated weights for policy 0, policy_version 1716581 (0.0006) [2023-12-27 17:32:18,228][20381] Updated weights for policy 1, policy_version 1715190 (0.0008) [2023-12-27 17:32:18,279][20381] Updated weights for policy 1, policy_version 1715200 (0.0008) [2023-12-27 17:32:18,334][20381] Updated weights for policy 1, policy_version 1715210 (0.0009) [2023-12-27 17:32:18,845][20347] Updated weights for policy 0, policy_version 1716591 (0.0007) [2023-12-27 17:32:18,894][20347] Updated weights for policy 0, policy_version 1716601 (0.0005) [2023-12-27 17:32:18,952][20347] Updated weights for policy 0, policy_version 1716611 (0.0005) [2023-12-27 17:32:19,074][20381] Updated weights for policy 1, policy_version 1715220 (0.0008) [2023-12-27 17:32:19,138][20381] Updated weights for policy 1, policy_version 1715230 (0.0007) [2023-12-27 17:32:19,199][20381] Updated weights for policy 1, policy_version 1715240 (0.0007) [2023-12-27 17:32:19,622][20347] Updated weights for policy 0, policy_version 1716621 (0.0006) [2023-12-27 17:32:19,688][20347] Updated weights for policy 0, policy_version 1716631 (0.0007) [2023-12-27 17:32:19,754][20347] Updated weights for policy 0, policy_version 1716641 (0.0006) [2023-12-27 17:32:19,862][20381] Updated weights for policy 1, policy_version 1715250 (0.0008) [2023-12-27 17:32:19,924][20381] Updated weights for policy 1, policy_version 1715260 (0.0008) [2023-12-27 17:32:19,988][20381] Updated weights for policy 1, policy_version 1715270 (0.0007) [2023-12-27 17:32:20,043][20381] Updated weights for policy 1, policy_version 1715280 (0.0006) [2023-12-27 17:32:20,429][20347] Updated weights for policy 0, policy_version 1716651 (0.0007) [2023-12-27 17:32:20,484][20347] Updated weights for policy 0, policy_version 1716661 (0.0007) [2023-12-27 17:32:20,544][20347] Updated weights for policy 0, policy_version 1716671 (0.0008) [2023-12-27 17:32:20,774][20381] Updated weights for policy 1, policy_version 1715290 (0.0008) [2023-12-27 17:32:20,829][20381] Updated weights for policy 1, policy_version 1715300 (0.0006) [2023-12-27 17:32:20,889][20381] Updated weights for policy 1, policy_version 1715310 (0.0011) [2023-12-27 17:32:21,275][20347] Updated weights for policy 0, policy_version 1716681 (0.0006) [2023-12-27 17:32:21,335][20347] Updated weights for policy 0, policy_version 1716691 (0.0009) [2023-12-27 17:32:21,403][20347] Updated weights for policy 0, policy_version 1716701 (0.0010) [2023-12-27 17:32:21,459][20347] Updated weights for policy 0, policy_version 1716711 (0.0007) [2023-12-27 17:32:21,658][20381] Updated weights for policy 1, policy_version 1715320 (0.0009) [2023-12-27 17:32:21,716][20381] Updated weights for policy 1, policy_version 1715330 (0.0008) [2023-12-27 17:32:21,794][20381] Updated weights for policy 1, policy_version 1715340 (0.0011) [2023-12-27 17:32:22,202][20347] Updated weights for policy 0, policy_version 1716721 (0.0007) [2023-12-27 17:32:22,251][20347] Updated weights for policy 0, policy_version 1716731 (0.0007) [2023-12-27 17:32:22,314][20347] Updated weights for policy 0, policy_version 1716741 (0.0007) [2023-12-27 17:32:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 878739456. Throughput: 0: 10037.0, 1: 10387.1. Samples: 878710256. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.270')] [2023-12-27 17:32:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001715344_439173120.pth... [2023-12-27 17:32:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001716744_439566336.pth... [2023-12-27 17:32:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001715560_439263232.pth [2023-12-27 17:32:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001714160_438870016.pth [2023-12-27 17:32:22,503][20381] Updated weights for policy 1, policy_version 1715350 (0.0011) [2023-12-27 17:32:22,563][20381] Updated weights for policy 1, policy_version 1715360 (0.0011) [2023-12-27 17:32:22,622][20381] Updated weights for policy 1, policy_version 1715370 (0.0011) [2023-12-27 17:32:23,120][20347] Updated weights for policy 0, policy_version 1716751 (0.0007) [2023-12-27 17:32:23,183][20347] Updated weights for policy 0, policy_version 1716761 (0.0008) [2023-12-27 17:32:23,241][20347] Updated weights for policy 0, policy_version 1716771 (0.0007) [2023-12-27 17:32:23,285][20381] Updated weights for policy 1, policy_version 1715380 (0.0009) [2023-12-27 17:32:23,343][20381] Updated weights for policy 1, policy_version 1715390 (0.0007) [2023-12-27 17:32:23,394][20381] Updated weights for policy 1, policy_version 1715400 (0.0007) [2023-12-27 17:32:23,935][20347] Updated weights for policy 0, policy_version 1716781 (0.0008) [2023-12-27 17:32:23,991][20347] Updated weights for policy 0, policy_version 1716791 (0.0007) [2023-12-27 17:32:24,049][20347] Updated weights for policy 0, policy_version 1716801 (0.0007) [2023-12-27 17:32:24,063][20381] Updated weights for policy 1, policy_version 1715410 (0.0008) [2023-12-27 17:32:24,124][20381] Updated weights for policy 1, policy_version 1715420 (0.0006) [2023-12-27 17:32:24,186][20381] Updated weights for policy 1, policy_version 1715430 (0.0007) [2023-12-27 17:32:24,248][20381] Updated weights for policy 1, policy_version 1715440 (0.0008) [2023-12-27 17:32:24,677][20347] Updated weights for policy 0, policy_version 1716811 (0.0006) [2023-12-27 17:32:24,722][20347] Updated weights for policy 0, policy_version 1716821 (0.0008) [2023-12-27 17:32:24,774][20347] Updated weights for policy 0, policy_version 1716831 (0.0008) [2023-12-27 17:32:24,941][20381] Updated weights for policy 1, policy_version 1715450 (0.0008) [2023-12-27 17:32:24,990][20381] Updated weights for policy 1, policy_version 1715460 (0.0007) [2023-12-27 17:32:25,042][20381] Updated weights for policy 1, policy_version 1715470 (0.0007) [2023-12-27 17:32:25,457][20347] Updated weights for policy 0, policy_version 1716841 (0.0007) [2023-12-27 17:32:25,501][20347] Updated weights for policy 0, policy_version 1716851 (0.0007) [2023-12-27 17:32:25,547][20347] Updated weights for policy 0, policy_version 1716861 (0.0006) [2023-12-27 17:32:25,591][20347] Updated weights for policy 0, policy_version 1716871 (0.0007) [2023-12-27 17:32:25,758][20381] Updated weights for policy 1, policy_version 1715480 (0.0008) [2023-12-27 17:32:25,814][20381] Updated weights for policy 1, policy_version 1715490 (0.0008) [2023-12-27 17:32:25,871][20381] Updated weights for policy 1, policy_version 1715500 (0.0008) [2023-12-27 17:32:26,299][20347] Updated weights for policy 0, policy_version 1716881 (0.0007) [2023-12-27 17:32:26,357][20347] Updated weights for policy 0, policy_version 1716891 (0.0005) [2023-12-27 17:32:26,408][20347] Updated weights for policy 0, policy_version 1716901 (0.0005) [2023-12-27 17:32:26,620][20381] Updated weights for policy 1, policy_version 1715510 (0.0008) [2023-12-27 17:32:26,666][20381] Updated weights for policy 1, policy_version 1715520 (0.0009) [2023-12-27 17:32:26,719][20381] Updated weights for policy 1, policy_version 1715530 (0.0009) [2023-12-27 17:32:26,939][20347] Updated weights for policy 0, policy_version 1716911 (0.0005) [2023-12-27 17:32:26,996][20347] Updated weights for policy 0, policy_version 1716921 (0.0005) [2023-12-27 17:32:27,054][20347] Updated weights for policy 0, policy_version 1716931 (0.0005) [2023-12-27 17:32:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 878837760. Throughput: 0: 9997.7, 1: 10332.3. Samples: 878827872. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.270')] [2023-12-27 17:32:27,452][20381] Updated weights for policy 1, policy_version 1715540 (0.0009) [2023-12-27 17:32:27,508][20381] Updated weights for policy 1, policy_version 1715550 (0.0007) [2023-12-27 17:32:27,563][20381] Updated weights for policy 1, policy_version 1715560 (0.0007) [2023-12-27 17:32:27,682][20347] Updated weights for policy 0, policy_version 1716941 (0.0006) [2023-12-27 17:32:27,738][20347] Updated weights for policy 0, policy_version 1716951 (0.0007) [2023-12-27 17:32:27,792][20347] Updated weights for policy 0, policy_version 1716961 (0.0007) [2023-12-27 17:32:28,258][20381] Updated weights for policy 1, policy_version 1715570 (0.0008) [2023-12-27 17:32:28,313][20381] Updated weights for policy 1, policy_version 1715580 (0.0010) [2023-12-27 17:32:28,376][20381] Updated weights for policy 1, policy_version 1715590 (0.0011) [2023-12-27 17:32:28,429][20381] Updated weights for policy 1, policy_version 1715600 (0.0010) [2023-12-27 17:32:28,472][20347] Updated weights for policy 0, policy_version 1716971 (0.0007) [2023-12-27 17:32:28,523][20347] Updated weights for policy 0, policy_version 1716981 (0.0005) [2023-12-27 17:32:28,578][20347] Updated weights for policy 0, policy_version 1716991 (0.0007) [2023-12-27 17:32:29,077][20381] Updated weights for policy 1, policy_version 1715610 (0.0007) [2023-12-27 17:32:29,126][20381] Updated weights for policy 1, policy_version 1715620 (0.0005) [2023-12-27 17:32:29,170][20381] Updated weights for policy 1, policy_version 1715630 (0.0005) [2023-12-27 17:32:29,240][20347] Updated weights for policy 0, policy_version 1717001 (0.0007) [2023-12-27 17:32:29,302][20347] Updated weights for policy 0, policy_version 1717011 (0.0007) [2023-12-27 17:32:29,365][20347] Updated weights for policy 0, policy_version 1717021 (0.0007) [2023-12-27 17:32:29,419][20347] Updated weights for policy 0, policy_version 1717031 (0.0005) [2023-12-27 17:32:29,957][20381] Updated weights for policy 1, policy_version 1715640 (0.0008) [2023-12-27 17:32:30,009][20381] Updated weights for policy 1, policy_version 1715650 (0.0009) [2023-12-27 17:32:30,063][20347] Updated weights for policy 0, policy_version 1717041 (0.0007) [2023-12-27 17:32:30,063][20381] Updated weights for policy 1, policy_version 1715660 (0.0006) [2023-12-27 17:32:30,116][20347] Updated weights for policy 0, policy_version 1717051 (0.0007) [2023-12-27 17:32:30,174][20347] Updated weights for policy 0, policy_version 1717061 (0.0007) [2023-12-27 17:32:30,719][20381] Updated weights for policy 1, policy_version 1715670 (0.0006) [2023-12-27 17:32:30,770][20381] Updated weights for policy 1, policy_version 1715680 (0.0007) [2023-12-27 17:32:30,818][20381] Updated weights for policy 1, policy_version 1715690 (0.0007) [2023-12-27 17:32:30,871][20347] Updated weights for policy 0, policy_version 1717071 (0.0009) [2023-12-27 17:32:30,916][20347] Updated weights for policy 0, policy_version 1717081 (0.0010) [2023-12-27 17:32:30,963][20347] Updated weights for policy 0, policy_version 1717091 (0.0010) [2023-12-27 17:32:31,544][20381] Updated weights for policy 1, policy_version 1715700 (0.0006) [2023-12-27 17:32:31,597][20381] Updated weights for policy 1, policy_version 1715710 (0.0007) [2023-12-27 17:32:31,665][20381] Updated weights for policy 1, policy_version 1715720 (0.0008) [2023-12-27 17:32:31,720][20347] Updated weights for policy 0, policy_version 1717101 (0.0009) [2023-12-27 17:32:31,790][20347] Updated weights for policy 0, policy_version 1717111 (0.0008) [2023-12-27 17:32:31,846][20347] Updated weights for policy 0, policy_version 1717121 (0.0008) [2023-12-27 17:32:32,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 878936064. Throughput: 0: 10083.7, 1: 10249.4. Samples: 878949892. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 17:32:32,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:32:32,406][20381] Updated weights for policy 1, policy_version 1715730 (0.0008) [2023-12-27 17:32:32,457][20381] Updated weights for policy 1, policy_version 1715740 (0.0008) [2023-12-27 17:32:32,507][20381] Updated weights for policy 1, policy_version 1715750 (0.0008) [2023-12-27 17:32:32,513][20347] Updated weights for policy 0, policy_version 1717131 (0.0007) [2023-12-27 17:32:32,563][20381] Updated weights for policy 1, policy_version 1715760 (0.0010) [2023-12-27 17:32:32,572][20347] Updated weights for policy 0, policy_version 1717141 (0.0006) [2023-12-27 17:32:32,626][20347] Updated weights for policy 0, policy_version 1717151 (0.0007) [2023-12-27 17:32:33,231][20381] Updated weights for policy 1, policy_version 1715770 (0.0008) [2023-12-27 17:32:33,280][20381] Updated weights for policy 1, policy_version 1715780 (0.0008) [2023-12-27 17:32:33,323][20347] Updated weights for policy 0, policy_version 1717161 (0.0008) [2023-12-27 17:32:33,332][20381] Updated weights for policy 1, policy_version 1715790 (0.0007) [2023-12-27 17:32:33,387][20347] Updated weights for policy 0, policy_version 1717171 (0.0008) [2023-12-27 17:32:33,445][20347] Updated weights for policy 0, policy_version 1717181 (0.0008) [2023-12-27 17:32:33,502][20347] Updated weights for policy 0, policy_version 1717191 (0.0010) [2023-12-27 17:32:33,974][20381] Updated weights for policy 1, policy_version 1715800 (0.0005) [2023-12-27 17:32:34,024][20381] Updated weights for policy 1, policy_version 1715810 (0.0006) [2023-12-27 17:32:34,069][20381] Updated weights for policy 1, policy_version 1715820 (0.0010) [2023-12-27 17:32:34,172][20347] Updated weights for policy 0, policy_version 1717201 (0.0010) [2023-12-27 17:32:34,232][20347] Updated weights for policy 0, policy_version 1717211 (0.0008) [2023-12-27 17:32:34,289][20347] Updated weights for policy 0, policy_version 1717221 (0.0007) [2023-12-27 17:32:34,778][20381] Updated weights for policy 1, policy_version 1715830 (0.0010) [2023-12-27 17:32:34,827][20381] Updated weights for policy 1, policy_version 1715840 (0.0010) [2023-12-27 17:32:34,880][20381] Updated weights for policy 1, policy_version 1715850 (0.0005) [2023-12-27 17:32:34,969][20347] Updated weights for policy 0, policy_version 1717231 (0.0009) [2023-12-27 17:32:35,024][20347] Updated weights for policy 0, policy_version 1717241 (0.0008) [2023-12-27 17:32:35,083][20347] Updated weights for policy 0, policy_version 1717251 (0.0009) [2023-12-27 17:32:35,481][20381] Updated weights for policy 1, policy_version 1715860 (0.0009) [2023-12-27 17:32:35,532][20381] Updated weights for policy 1, policy_version 1715870 (0.0010) [2023-12-27 17:32:35,592][20381] Updated weights for policy 1, policy_version 1715880 (0.0006) [2023-12-27 17:32:35,721][20347] Updated weights for policy 0, policy_version 1717261 (0.0008) [2023-12-27 17:32:35,784][20347] Updated weights for policy 0, policy_version 1717271 (0.0006) [2023-12-27 17:32:35,847][20347] Updated weights for policy 0, policy_version 1717281 (0.0008) [2023-12-27 17:32:36,187][20381] Updated weights for policy 1, policy_version 1715890 (0.0006) [2023-12-27 17:32:36,237][20381] Updated weights for policy 1, policy_version 1715900 (0.0011) [2023-12-27 17:32:36,290][20381] Updated weights for policy 1, policy_version 1715910 (0.0011) [2023-12-27 17:32:36,341][20381] Updated weights for policy 1, policy_version 1715920 (0.0011) [2023-12-27 17:32:36,472][20347] Updated weights for policy 0, policy_version 1717291 (0.0009) [2023-12-27 17:32:36,533][20347] Updated weights for policy 0, policy_version 1717301 (0.0007) [2023-12-27 17:32:36,590][20347] Updated weights for policy 0, policy_version 1717311 (0.0008) [2023-12-27 17:32:37,029][20381] Updated weights for policy 1, policy_version 1715930 (0.0008) [2023-12-27 17:32:37,079][20381] Updated weights for policy 1, policy_version 1715940 (0.0007) [2023-12-27 17:32:37,131][20381] Updated weights for policy 1, policy_version 1715950 (0.0008) [2023-12-27 17:32:37,250][20347] Updated weights for policy 0, policy_version 1717321 (0.0007) [2023-12-27 17:32:37,309][20347] Updated weights for policy 0, policy_version 1717331 (0.0005) [2023-12-27 17:32:37,354][20347] Updated weights for policy 0, policy_version 1717341 (0.0005) [2023-12-27 17:32:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 879042560. Throughput: 0: 10135.3, 1: 10221.4. Samples: 879012032. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:32:37,363][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:32:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001715952_439328768.pth... [2023-12-27 17:32:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001714768_439025664.pth [2023-12-27 17:32:37,408][20347] Updated weights for policy 0, policy_version 1717351 (0.0008) [2023-12-27 17:32:37,414][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001717352_439721984.pth... [2023-12-27 17:32:37,418][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001716136_439410688.pth [2023-12-27 17:32:37,856][20381] Updated weights for policy 1, policy_version 1715960 (0.0007) [2023-12-27 17:32:37,911][20381] Updated weights for policy 1, policy_version 1715970 (0.0010) [2023-12-27 17:32:37,968][20381] Updated weights for policy 1, policy_version 1715980 (0.0010) [2023-12-27 17:32:38,080][20347] Updated weights for policy 0, policy_version 1717361 (0.0008) [2023-12-27 17:32:38,157][20347] Updated weights for policy 0, policy_version 1717371 (0.0007) [2023-12-27 17:32:38,212][20347] Updated weights for policy 0, policy_version 1717381 (0.0007) [2023-12-27 17:32:38,654][20381] Updated weights for policy 1, policy_version 1715990 (0.0009) [2023-12-27 17:32:38,714][20381] Updated weights for policy 1, policy_version 1716000 (0.0008) [2023-12-27 17:32:38,772][20381] Updated weights for policy 1, policy_version 1716010 (0.0007) [2023-12-27 17:32:38,821][20347] Updated weights for policy 0, policy_version 1717391 (0.0007) [2023-12-27 17:32:38,883][20347] Updated weights for policy 0, policy_version 1717401 (0.0006) [2023-12-27 17:32:38,943][20347] Updated weights for policy 0, policy_version 1717411 (0.0007) [2023-12-27 17:32:39,481][20381] Updated weights for policy 1, policy_version 1716020 (0.0008) [2023-12-27 17:32:39,546][20347] Updated weights for policy 0, policy_version 1717421 (0.0006) [2023-12-27 17:32:39,553][20381] Updated weights for policy 1, policy_version 1716030 (0.0009) [2023-12-27 17:32:39,606][20347] Updated weights for policy 0, policy_version 1717431 (0.0006) [2023-12-27 17:32:39,615][20381] Updated weights for policy 1, policy_version 1716040 (0.0009) [2023-12-27 17:32:39,664][20347] Updated weights for policy 0, policy_version 1717441 (0.0006) [2023-12-27 17:32:40,408][20347] Updated weights for policy 0, policy_version 1717451 (0.0008) [2023-12-27 17:32:40,444][20381] Updated weights for policy 1, policy_version 1716050 (0.0009) [2023-12-27 17:32:40,463][20347] Updated weights for policy 0, policy_version 1717461 (0.0007) [2023-12-27 17:32:40,505][20381] Updated weights for policy 1, policy_version 1716060 (0.0006) [2023-12-27 17:32:40,520][20347] Updated weights for policy 0, policy_version 1717471 (0.0007) [2023-12-27 17:32:40,562][20381] Updated weights for policy 1, policy_version 1716070 (0.0006) [2023-12-27 17:32:40,616][20381] Updated weights for policy 1, policy_version 1716080 (0.0008) [2023-12-27 17:32:41,166][20347] Updated weights for policy 0, policy_version 1717481 (0.0006) [2023-12-27 17:32:41,228][20347] Updated weights for policy 0, policy_version 1717491 (0.0006) [2023-12-27 17:32:41,290][20347] Updated weights for policy 0, policy_version 1717501 (0.0007) [2023-12-27 17:32:41,339][20381] Updated weights for policy 1, policy_version 1716090 (0.0007) [2023-12-27 17:32:41,345][20347] Updated weights for policy 0, policy_version 1717511 (0.0007) [2023-12-27 17:32:41,405][20381] Updated weights for policy 1, policy_version 1716100 (0.0008) [2023-12-27 17:32:41,462][20381] Updated weights for policy 1, policy_version 1716110 (0.0008) [2023-12-27 17:32:42,026][20347] Updated weights for policy 0, policy_version 1717521 (0.0008) [2023-12-27 17:32:42,080][20347] Updated weights for policy 0, policy_version 1717531 (0.0008) [2023-12-27 17:32:42,133][20347] Updated weights for policy 0, policy_version 1717541 (0.0007) [2023-12-27 17:32:42,187][20381] Updated weights for policy 1, policy_version 1716120 (0.0007) [2023-12-27 17:32:42,244][20381] Updated weights for policy 1, policy_version 1716130 (0.0008) [2023-12-27 17:32:42,311][20381] Updated weights for policy 1, policy_version 1716140 (0.0008) [2023-12-27 17:32:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 879149056. Throughput: 0: 10198.2, 1: 10156.6. Samples: 879134392. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:32:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:32:42,887][20347] Updated weights for policy 0, policy_version 1717551 (0.0007) [2023-12-27 17:32:42,949][20347] Updated weights for policy 0, policy_version 1717561 (0.0007) [2023-12-27 17:32:43,003][20347] Updated weights for policy 0, policy_version 1717571 (0.0008) [2023-12-27 17:32:43,026][20381] Updated weights for policy 1, policy_version 1716150 (0.0007) [2023-12-27 17:32:43,086][20381] Updated weights for policy 1, policy_version 1716160 (0.0008) [2023-12-27 17:32:43,145][20381] Updated weights for policy 1, policy_version 1716170 (0.0007) [2023-12-27 17:32:43,584][20347] Updated weights for policy 0, policy_version 1717581 (0.0006) [2023-12-27 17:32:43,644][20347] Updated weights for policy 0, policy_version 1717591 (0.0005) [2023-12-27 17:32:43,701][20347] Updated weights for policy 0, policy_version 1717601 (0.0008) [2023-12-27 17:32:43,875][20381] Updated weights for policy 1, policy_version 1716180 (0.0007) [2023-12-27 17:32:43,927][20381] Updated weights for policy 1, policy_version 1716190 (0.0007) [2023-12-27 17:32:43,975][20381] Updated weights for policy 1, policy_version 1716200 (0.0007) [2023-12-27 17:32:44,327][20347] Updated weights for policy 0, policy_version 1717611 (0.0009) [2023-12-27 17:32:44,379][20347] Updated weights for policy 0, policy_version 1717621 (0.0010) [2023-12-27 17:32:44,424][20347] Updated weights for policy 0, policy_version 1717631 (0.0006) [2023-12-27 17:32:44,648][20381] Updated weights for policy 1, policy_version 1716210 (0.0007) [2023-12-27 17:32:44,703][20381] Updated weights for policy 1, policy_version 1716220 (0.0009) [2023-12-27 17:32:44,755][20381] Updated weights for policy 1, policy_version 1716230 (0.0007) [2023-12-27 17:32:44,822][20381] Updated weights for policy 1, policy_version 1716240 (0.0009) [2023-12-27 17:32:45,118][20347] Updated weights for policy 0, policy_version 1717641 (0.0009) [2023-12-27 17:32:45,176][20347] Updated weights for policy 0, policy_version 1717651 (0.0008) [2023-12-27 17:32:45,222][20347] Updated weights for policy 0, policy_version 1717661 (0.0007) [2023-12-27 17:32:45,273][20347] Updated weights for policy 0, policy_version 1717671 (0.0007) [2023-12-27 17:32:45,501][20381] Updated weights for policy 1, policy_version 1716250 (0.0008) [2023-12-27 17:32:45,547][20381] Updated weights for policy 1, policy_version 1716260 (0.0006) [2023-12-27 17:32:45,599][20381] Updated weights for policy 1, policy_version 1716270 (0.0006) [2023-12-27 17:32:46,001][20347] Updated weights for policy 0, policy_version 1717681 (0.0007) [2023-12-27 17:32:46,057][20347] Updated weights for policy 0, policy_version 1717691 (0.0008) [2023-12-27 17:32:46,101][20347] Updated weights for policy 0, policy_version 1717701 (0.0007) [2023-12-27 17:32:46,257][20381] Updated weights for policy 1, policy_version 1716280 (0.0008) [2023-12-27 17:32:46,312][20381] Updated weights for policy 1, policy_version 1716290 (0.0008) [2023-12-27 17:32:46,364][20381] Updated weights for policy 1, policy_version 1716300 (0.0008) [2023-12-27 17:32:46,742][20347] Updated weights for policy 0, policy_version 1717711 (0.0006) [2023-12-27 17:32:46,788][20347] Updated weights for policy 0, policy_version 1717721 (0.0005) [2023-12-27 17:32:46,844][20347] Updated weights for policy 0, policy_version 1717731 (0.0005) [2023-12-27 17:32:47,103][20381] Updated weights for policy 1, policy_version 1716310 (0.0007) [2023-12-27 17:32:47,154][20381] Updated weights for policy 1, policy_version 1716320 (0.0007) [2023-12-27 17:32:47,204][20381] Updated weights for policy 1, policy_version 1716330 (0.0007) [2023-12-27 17:32:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 879247360. Throughput: 0: 10249.9, 1: 10158.7. Samples: 879256028. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:32:47,363][19170] Avg episode reward: [(0, '990.380'), (1, '990.280')] [2023-12-27 17:32:47,491][20347] Updated weights for policy 0, policy_version 1717741 (0.0007) [2023-12-27 17:32:47,542][20347] Updated weights for policy 0, policy_version 1717751 (0.0007) [2023-12-27 17:32:47,595][20347] Updated weights for policy 0, policy_version 1717761 (0.0007) [2023-12-27 17:32:47,908][20381] Updated weights for policy 1, policy_version 1716340 (0.0007) [2023-12-27 17:32:47,966][20381] Updated weights for policy 1, policy_version 1716350 (0.0007) [2023-12-27 17:32:48,027][20381] Updated weights for policy 1, policy_version 1716360 (0.0008) [2023-12-27 17:32:48,290][20347] Updated weights for policy 0, policy_version 1717771 (0.0008) [2023-12-27 17:32:48,348][20347] Updated weights for policy 0, policy_version 1717781 (0.0007) [2023-12-27 17:32:48,401][20347] Updated weights for policy 0, policy_version 1717791 (0.0008) [2023-12-27 17:32:48,680][20381] Updated weights for policy 1, policy_version 1716370 (0.0007) [2023-12-27 17:32:48,742][20381] Updated weights for policy 1, policy_version 1716380 (0.0011) [2023-12-27 17:32:48,802][20381] Updated weights for policy 1, policy_version 1716390 (0.0011) [2023-12-27 17:32:48,857][20381] Updated weights for policy 1, policy_version 1716400 (0.0009) [2023-12-27 17:32:49,041][20347] Updated weights for policy 0, policy_version 1717801 (0.0007) [2023-12-27 17:32:49,099][20347] Updated weights for policy 0, policy_version 1717811 (0.0005) [2023-12-27 17:32:49,158][20347] Updated weights for policy 0, policy_version 1717821 (0.0007) [2023-12-27 17:32:49,216][20347] Updated weights for policy 0, policy_version 1717831 (0.0007) [2023-12-27 17:32:49,536][20381] Updated weights for policy 1, policy_version 1716410 (0.0008) [2023-12-27 17:32:49,597][20381] Updated weights for policy 1, policy_version 1716420 (0.0005) [2023-12-27 17:32:49,664][20381] Updated weights for policy 1, policy_version 1716430 (0.0006) [2023-12-27 17:32:49,892][20347] Updated weights for policy 0, policy_version 1717841 (0.0008) [2023-12-27 17:32:49,958][20347] Updated weights for policy 0, policy_version 1717851 (0.0007) [2023-12-27 17:32:50,014][20347] Updated weights for policy 0, policy_version 1717861 (0.0006) [2023-12-27 17:32:50,294][20381] Updated weights for policy 1, policy_version 1716440 (0.0006) [2023-12-27 17:32:50,350][20381] Updated weights for policy 1, policy_version 1716450 (0.0006) [2023-12-27 17:32:50,410][20381] Updated weights for policy 1, policy_version 1716460 (0.0011) [2023-12-27 17:32:50,716][20347] Updated weights for policy 0, policy_version 1717871 (0.0006) [2023-12-27 17:32:50,778][20347] Updated weights for policy 0, policy_version 1717881 (0.0007) [2023-12-27 17:32:50,842][20347] Updated weights for policy 0, policy_version 1717891 (0.0008) [2023-12-27 17:32:51,156][20381] Updated weights for policy 1, policy_version 1716470 (0.0009) [2023-12-27 17:32:51,218][20381] Updated weights for policy 1, policy_version 1716480 (0.0007) [2023-12-27 17:32:51,279][20381] Updated weights for policy 1, policy_version 1716490 (0.0008) [2023-12-27 17:32:51,525][20347] Updated weights for policy 0, policy_version 1717901 (0.0009) [2023-12-27 17:32:51,587][20347] Updated weights for policy 0, policy_version 1717912 (0.0009) [2023-12-27 17:32:51,651][20347] Updated weights for policy 0, policy_version 1717922 (0.0009) [2023-12-27 17:32:51,974][20381] Updated weights for policy 1, policy_version 1716500 (0.0007) [2023-12-27 17:32:52,038][20381] Updated weights for policy 1, policy_version 1716510 (0.0006) [2023-12-27 17:32:52,092][20381] Updated weights for policy 1, policy_version 1716520 (0.0006) [2023-12-27 17:32:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 879345664. Throughput: 0: 10247.7, 1: 10144.9. Samples: 879317928. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:32:52,363][19170] Avg episode reward: [(0, '990.380'), (1, '990.280')] [2023-12-27 17:32:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001716528_439476224.pth... [2023-12-27 17:32:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001717928_439869440.pth... [2023-12-27 17:32:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001715344_439173120.pth [2023-12-27 17:32:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001716744_439566336.pth [2023-12-27 17:32:52,407][20347] Updated weights for policy 0, policy_version 1717932 (0.0005) [2023-12-27 17:32:52,459][20347] Updated weights for policy 0, policy_version 1717942 (0.0005) [2023-12-27 17:32:52,509][20347] Updated weights for policy 0, policy_version 1717952 (0.0005) [2023-12-27 17:32:52,760][20381] Updated weights for policy 1, policy_version 1716530 (0.0008) [2023-12-27 17:32:52,818][20381] Updated weights for policy 1, policy_version 1716540 (0.0009) [2023-12-27 17:32:52,874][20381] Updated weights for policy 1, policy_version 1716550 (0.0009) [2023-12-27 17:32:52,934][20381] Updated weights for policy 1, policy_version 1716560 (0.0007) [2023-12-27 17:32:53,159][20347] Updated weights for policy 0, policy_version 1717962 (0.0006) [2023-12-27 17:32:53,216][20347] Updated weights for policy 0, policy_version 1717972 (0.0008) [2023-12-27 17:32:53,286][20347] Updated weights for policy 0, policy_version 1717982 (0.0007) [2023-12-27 17:32:53,354][20347] Updated weights for policy 0, policy_version 1717992 (0.0008) [2023-12-27 17:32:53,581][20381] Updated weights for policy 1, policy_version 1716570 (0.0006) [2023-12-27 17:32:53,634][20381] Updated weights for policy 1, policy_version 1716580 (0.0007) [2023-12-27 17:32:53,683][20381] Updated weights for policy 1, policy_version 1716590 (0.0008) [2023-12-27 17:32:54,040][20347] Updated weights for policy 0, policy_version 1718002 (0.0006) [2023-12-27 17:32:54,095][20347] Updated weights for policy 0, policy_version 1718012 (0.0008) [2023-12-27 17:32:54,159][20347] Updated weights for policy 0, policy_version 1718022 (0.0007) [2023-12-27 17:32:54,342][20381] Updated weights for policy 1, policy_version 1716600 (0.0010) [2023-12-27 17:32:54,390][20381] Updated weights for policy 1, policy_version 1716610 (0.0010) [2023-12-27 17:32:54,436][20381] Updated weights for policy 1, policy_version 1716620 (0.0011) [2023-12-27 17:32:54,762][20347] Updated weights for policy 0, policy_version 1718032 (0.0005) [2023-12-27 17:32:54,812][20347] Updated weights for policy 0, policy_version 1718042 (0.0006) [2023-12-27 17:32:54,864][20347] Updated weights for policy 0, policy_version 1718052 (0.0009) [2023-12-27 17:32:55,144][20381] Updated weights for policy 1, policy_version 1716630 (0.0011) [2023-12-27 17:32:55,200][20381] Updated weights for policy 1, policy_version 1716640 (0.0011) [2023-12-27 17:32:55,246][20381] Updated weights for policy 1, policy_version 1716650 (0.0010) [2023-12-27 17:32:55,456][20347] Updated weights for policy 0, policy_version 1718062 (0.0005) [2023-12-27 17:32:55,506][20347] Updated weights for policy 0, policy_version 1718072 (0.0005) [2023-12-27 17:32:55,557][20347] Updated weights for policy 0, policy_version 1718082 (0.0008) [2023-12-27 17:32:55,939][20381] Updated weights for policy 1, policy_version 1716660 (0.0010) [2023-12-27 17:32:55,987][20381] Updated weights for policy 1, policy_version 1716670 (0.0008) [2023-12-27 17:32:56,030][20381] Updated weights for policy 1, policy_version 1716680 (0.0007) [2023-12-27 17:32:56,105][20347] Updated weights for policy 0, policy_version 1718092 (0.0010) [2023-12-27 17:32:56,160][20347] Updated weights for policy 0, policy_version 1718102 (0.0007) [2023-12-27 17:32:56,217][20347] Updated weights for policy 0, policy_version 1718112 (0.0006) [2023-12-27 17:32:56,717][20381] Updated weights for policy 1, policy_version 1716690 (0.0006) [2023-12-27 17:32:56,768][20381] Updated weights for policy 1, policy_version 1716700 (0.0007) [2023-12-27 17:32:56,812][20381] Updated weights for policy 1, policy_version 1716710 (0.0008) [2023-12-27 17:32:56,855][20347] Updated weights for policy 0, policy_version 1718122 (0.0006) [2023-12-27 17:32:56,859][20381] Updated weights for policy 1, policy_version 1716720 (0.0008) [2023-12-27 17:32:56,908][20347] Updated weights for policy 0, policy_version 1718132 (0.0005) [2023-12-27 17:32:56,954][20347] Updated weights for policy 0, policy_version 1718142 (0.0006) [2023-12-27 17:32:56,999][20347] Updated weights for policy 0, policy_version 1718152 (0.0007) [2023-12-27 17:32:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 879452160. Throughput: 0: 10346.9, 1: 10069.5. Samples: 879442236. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:32:57,363][19170] Avg episode reward: [(0, '990.210'), (1, '990.310')] [2023-12-27 17:32:57,601][20381] Updated weights for policy 1, policy_version 1716730 (0.0011) [2023-12-27 17:32:57,645][20347] Updated weights for policy 0, policy_version 1718162 (0.0008) [2023-12-27 17:32:57,661][20381] Updated weights for policy 1, policy_version 1716740 (0.0011) [2023-12-27 17:32:57,694][20347] Updated weights for policy 0, policy_version 1718172 (0.0010) [2023-12-27 17:32:57,724][20381] Updated weights for policy 1, policy_version 1716750 (0.0011) [2023-12-27 17:32:57,746][20347] Updated weights for policy 0, policy_version 1718182 (0.0010) [2023-12-27 17:32:58,452][20381] Updated weights for policy 1, policy_version 1716760 (0.0007) [2023-12-27 17:32:58,464][20347] Updated weights for policy 0, policy_version 1718192 (0.0008) [2023-12-27 17:32:58,518][20381] Updated weights for policy 1, policy_version 1716770 (0.0007) [2023-12-27 17:32:58,525][20347] Updated weights for policy 0, policy_version 1718202 (0.0008) [2023-12-27 17:32:58,578][20381] Updated weights for policy 1, policy_version 1716780 (0.0008) [2023-12-27 17:32:58,585][20347] Updated weights for policy 0, policy_version 1718212 (0.0008) [2023-12-27 17:32:59,311][20347] Updated weights for policy 0, policy_version 1718222 (0.0007) [2023-12-27 17:32:59,350][20381] Updated weights for policy 1, policy_version 1716790 (0.0008) [2023-12-27 17:32:59,375][20347] Updated weights for policy 0, policy_version 1718232 (0.0009) [2023-12-27 17:32:59,411][20381] Updated weights for policy 1, policy_version 1716800 (0.0007) [2023-12-27 17:32:59,429][20347] Updated weights for policy 0, policy_version 1718242 (0.0007) [2023-12-27 17:32:59,469][20381] Updated weights for policy 1, policy_version 1716810 (0.0006) [2023-12-27 17:33:00,068][20381] Updated weights for policy 1, policy_version 1716820 (0.0006) [2023-12-27 17:33:00,122][20381] Updated weights for policy 1, policy_version 1716830 (0.0008) [2023-12-27 17:33:00,140][20347] Updated weights for policy 0, policy_version 1718252 (0.0007) [2023-12-27 17:33:00,176][20381] Updated weights for policy 1, policy_version 1716840 (0.0006) [2023-12-27 17:33:00,197][20347] Updated weights for policy 0, policy_version 1718262 (0.0007) [2023-12-27 17:33:00,252][20347] Updated weights for policy 0, policy_version 1718272 (0.0007) [2023-12-27 17:33:00,869][20381] Updated weights for policy 1, policy_version 1716850 (0.0007) [2023-12-27 17:33:00,874][20347] Updated weights for policy 0, policy_version 1718282 (0.0007) [2023-12-27 17:33:00,925][20347] Updated weights for policy 0, policy_version 1718292 (0.0005) [2023-12-27 17:33:00,928][20381] Updated weights for policy 1, policy_version 1716860 (0.0007) [2023-12-27 17:33:00,979][20347] Updated weights for policy 0, policy_version 1718302 (0.0005) [2023-12-27 17:33:00,984][20381] Updated weights for policy 1, policy_version 1716870 (0.0007) [2023-12-27 17:33:01,032][20347] Updated weights for policy 0, policy_version 1718312 (0.0006) [2023-12-27 17:33:01,043][20381] Updated weights for policy 1, policy_version 1716880 (0.0008) [2023-12-27 17:33:01,718][20347] Updated weights for policy 0, policy_version 1718322 (0.0008) [2023-12-27 17:33:01,764][20381] Updated weights for policy 1, policy_version 1716890 (0.0006) [2023-12-27 17:33:01,778][20347] Updated weights for policy 0, policy_version 1718332 (0.0006) [2023-12-27 17:33:01,818][20381] Updated weights for policy 1, policy_version 1716900 (0.0007) [2023-12-27 17:33:01,843][20347] Updated weights for policy 0, policy_version 1718342 (0.0005) [2023-12-27 17:33:01,877][20381] Updated weights for policy 1, policy_version 1716910 (0.0008) [2023-12-27 17:33:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 879550464. Throughput: 0: 10343.2, 1: 9973.1. Samples: 879563428. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:02,363][19170] Avg episode reward: [(0, '970.700'), (1, '990.310')] [2023-12-27 17:33:02,450][20347] Updated weights for policy 0, policy_version 1718352 (0.0006) [2023-12-27 17:33:02,520][20347] Updated weights for policy 0, policy_version 1718362 (0.0009) [2023-12-27 17:33:02,548][20381] Updated weights for policy 1, policy_version 1716920 (0.0007) [2023-12-27 17:33:02,585][20347] Updated weights for policy 0, policy_version 1718372 (0.0009) [2023-12-27 17:33:02,599][20381] Updated weights for policy 1, policy_version 1716930 (0.0007) [2023-12-27 17:33:02,647][20381] Updated weights for policy 1, policy_version 1716940 (0.0006) [2023-12-27 17:33:03,185][20347] Updated weights for policy 0, policy_version 1718382 (0.0007) [2023-12-27 17:33:03,245][20347] Updated weights for policy 0, policy_version 1718392 (0.0007) [2023-12-27 17:33:03,300][20347] Updated weights for policy 0, policy_version 1718402 (0.0008) [2023-12-27 17:33:03,334][20381] Updated weights for policy 1, policy_version 1716950 (0.0008) [2023-12-27 17:33:03,385][20381] Updated weights for policy 1, policy_version 1716960 (0.0007) [2023-12-27 17:33:03,435][20381] Updated weights for policy 1, policy_version 1716970 (0.0007) [2023-12-27 17:33:03,870][20347] Updated weights for policy 0, policy_version 1718412 (0.0009) [2023-12-27 17:33:03,924][20347] Updated weights for policy 0, policy_version 1718422 (0.0007) [2023-12-27 17:33:03,979][20347] Updated weights for policy 0, policy_version 1718432 (0.0008) [2023-12-27 17:33:04,144][20381] Updated weights for policy 1, policy_version 1716980 (0.0008) [2023-12-27 17:33:04,205][20381] Updated weights for policy 1, policy_version 1716990 (0.0006) [2023-12-27 17:33:04,268][20381] Updated weights for policy 1, policy_version 1717000 (0.0008) [2023-12-27 17:33:04,724][20347] Updated weights for policy 0, policy_version 1718442 (0.0007) [2023-12-27 17:33:04,783][20347] Updated weights for policy 0, policy_version 1718452 (0.0005) [2023-12-27 17:33:04,826][20381] Updated weights for policy 1, policy_version 1717010 (0.0005) [2023-12-27 17:33:04,834][20347] Updated weights for policy 0, policy_version 1718462 (0.0008) [2023-12-27 17:33:04,881][20381] Updated weights for policy 1, policy_version 1717020 (0.0005) [2023-12-27 17:33:04,883][20347] Updated weights for policy 0, policy_version 1718472 (0.0007) [2023-12-27 17:33:04,942][20381] Updated weights for policy 1, policy_version 1717030 (0.0005) [2023-12-27 17:33:04,999][20381] Updated weights for policy 1, policy_version 1717040 (0.0006) [2023-12-27 17:33:05,527][20347] Updated weights for policy 0, policy_version 1718482 (0.0006) [2023-12-27 17:33:05,581][20381] Updated weights for policy 1, policy_version 1717050 (0.0008) [2023-12-27 17:33:05,586][20347] Updated weights for policy 0, policy_version 1718492 (0.0005) [2023-12-27 17:33:05,638][20381] Updated weights for policy 1, policy_version 1717060 (0.0007) [2023-12-27 17:33:05,649][20347] Updated weights for policy 0, policy_version 1718502 (0.0005) [2023-12-27 17:33:05,690][20381] Updated weights for policy 1, policy_version 1717070 (0.0008) [2023-12-27 17:33:06,251][20347] Updated weights for policy 0, policy_version 1718512 (0.0008) [2023-12-27 17:33:06,297][20381] Updated weights for policy 1, policy_version 1717080 (0.0011) [2023-12-27 17:33:06,309][20347] Updated weights for policy 0, policy_version 1718522 (0.0007) [2023-12-27 17:33:06,358][20381] Updated weights for policy 1, policy_version 1717090 (0.0011) [2023-12-27 17:33:06,372][20347] Updated weights for policy 0, policy_version 1718532 (0.0009) [2023-12-27 17:33:06,416][20381] Updated weights for policy 1, policy_version 1717100 (0.0011) [2023-12-27 17:33:07,030][20347] Updated weights for policy 0, policy_version 1718542 (0.0005) [2023-12-27 17:33:07,081][20381] Updated weights for policy 1, policy_version 1717110 (0.0011) [2023-12-27 17:33:07,083][20347] Updated weights for policy 0, policy_version 1718552 (0.0007) [2023-12-27 17:33:07,128][20347] Updated weights for policy 0, policy_version 1718562 (0.0005) [2023-12-27 17:33:07,137][20381] Updated weights for policy 1, policy_version 1717120 (0.0011) [2023-12-27 17:33:07,197][20381] Updated weights for policy 1, policy_version 1717130 (0.0011) [2023-12-27 17:33:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 879665152. Throughput: 0: 10345.0, 1: 10033.2. Samples: 879627272. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:07,363][19170] Avg episode reward: [(0, '980.490'), (1, '1000.000')] [2023-12-27 17:33:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001717136_439631872.pth... [2023-12-27 17:33:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001718568_440033280.pth... [2023-12-27 17:33:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001715952_439328768.pth [2023-12-27 17:33:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001717352_439721984.pth [2023-12-27 17:33:07,798][20381] Updated weights for policy 1, policy_version 1717140 (0.0009) [2023-12-27 17:33:07,854][20347] Updated weights for policy 0, policy_version 1718572 (0.0006) [2023-12-27 17:33:07,857][20381] Updated weights for policy 1, policy_version 1717150 (0.0006) [2023-12-27 17:33:07,901][20347] Updated weights for policy 0, policy_version 1718582 (0.0007) [2023-12-27 17:33:07,903][20381] Updated weights for policy 1, policy_version 1717160 (0.0005) [2023-12-27 17:33:07,943][20347] Updated weights for policy 0, policy_version 1718592 (0.0009) [2023-12-27 17:33:08,608][20381] Updated weights for policy 1, policy_version 1717171 (0.0008) [2023-12-27 17:33:08,621][20347] Updated weights for policy 0, policy_version 1718602 (0.0009) [2023-12-27 17:33:08,669][20381] Updated weights for policy 1, policy_version 1717181 (0.0007) [2023-12-27 17:33:08,682][20347] Updated weights for policy 0, policy_version 1718612 (0.0008) [2023-12-27 17:33:08,728][20381] Updated weights for policy 1, policy_version 1717191 (0.0005) [2023-12-27 17:33:08,737][20347] Updated weights for policy 0, policy_version 1718622 (0.0007) [2023-12-27 17:33:08,788][20347] Updated weights for policy 0, policy_version 1718632 (0.0009) [2023-12-27 17:33:09,476][20381] Updated weights for policy 1, policy_version 1717201 (0.0006) [2023-12-27 17:33:09,530][20347] Updated weights for policy 0, policy_version 1718642 (0.0008) [2023-12-27 17:33:09,544][20381] Updated weights for policy 1, policy_version 1717211 (0.0007) [2023-12-27 17:33:09,597][20347] Updated weights for policy 0, policy_version 1718652 (0.0008) [2023-12-27 17:33:09,611][20381] Updated weights for policy 1, policy_version 1717221 (0.0007) [2023-12-27 17:33:09,661][20347] Updated weights for policy 0, policy_version 1718662 (0.0008) [2023-12-27 17:33:09,670][20381] Updated weights for policy 1, policy_version 1717231 (0.0007) [2023-12-27 17:33:10,311][20381] Updated weights for policy 1, policy_version 1717241 (0.0010) [2023-12-27 17:33:10,344][20347] Updated weights for policy 0, policy_version 1718672 (0.0006) [2023-12-27 17:33:10,379][20381] Updated weights for policy 1, policy_version 1717251 (0.0011) [2023-12-27 17:33:10,399][20347] Updated weights for policy 0, policy_version 1718682 (0.0007) [2023-12-27 17:33:10,445][20381] Updated weights for policy 1, policy_version 1717261 (0.0011) [2023-12-27 17:33:10,459][20347] Updated weights for policy 0, policy_version 1718692 (0.0007) [2023-12-27 17:33:11,030][20381] Updated weights for policy 1, policy_version 1717271 (0.0008) [2023-12-27 17:33:11,094][20381] Updated weights for policy 1, policy_version 1717281 (0.0010) [2023-12-27 17:33:11,158][20381] Updated weights for policy 1, policy_version 1717291 (0.0009) [2023-12-27 17:33:11,165][20347] Updated weights for policy 0, policy_version 1718702 (0.0008) [2023-12-27 17:33:11,232][20347] Updated weights for policy 0, policy_version 1718712 (0.0008) [2023-12-27 17:33:11,298][20347] Updated weights for policy 0, policy_version 1718722 (0.0007) [2023-12-27 17:33:11,912][20381] Updated weights for policy 1, policy_version 1717301 (0.0007) [2023-12-27 17:33:11,972][20381] Updated weights for policy 1, policy_version 1717311 (0.0007) [2023-12-27 17:33:12,023][20381] Updated weights for policy 1, policy_version 1717321 (0.0005) [2023-12-27 17:33:12,037][20347] Updated weights for policy 0, policy_version 1718732 (0.0007) [2023-12-27 17:33:12,102][20347] Updated weights for policy 0, policy_version 1718742 (0.0007) [2023-12-27 17:33:12,166][20347] Updated weights for policy 0, policy_version 1718752 (0.0006) [2023-12-27 17:33:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 879763456. Throughput: 0: 10405.4, 1: 10122.2. Samples: 879751616. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:12,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:33:12,693][20381] Updated weights for policy 1, policy_version 1717331 (0.0008) [2023-12-27 17:33:12,746][20381] Updated weights for policy 1, policy_version 1717341 (0.0011) [2023-12-27 17:33:12,802][20381] Updated weights for policy 1, policy_version 1717351 (0.0011) [2023-12-27 17:33:12,833][20347] Updated weights for policy 0, policy_version 1718762 (0.0008) [2023-12-27 17:33:12,887][20347] Updated weights for policy 0, policy_version 1718772 (0.0008) [2023-12-27 17:33:12,937][20347] Updated weights for policy 0, policy_version 1718782 (0.0007) [2023-12-27 17:33:12,986][20347] Updated weights for policy 0, policy_version 1718792 (0.0008) [2023-12-27 17:33:13,465][20381] Updated weights for policy 1, policy_version 1717361 (0.0011) [2023-12-27 17:33:13,520][20381] Updated weights for policy 1, policy_version 1717371 (0.0011) [2023-12-27 17:33:13,575][20381] Updated weights for policy 1, policy_version 1717381 (0.0010) [2023-12-27 17:33:13,620][20381] Updated weights for policy 1, policy_version 1717391 (0.0010) [2023-12-27 17:33:13,696][20347] Updated weights for policy 0, policy_version 1718802 (0.0008) [2023-12-27 17:33:13,746][20347] Updated weights for policy 0, policy_version 1718812 (0.0009) [2023-12-27 17:33:13,797][20347] Updated weights for policy 0, policy_version 1718822 (0.0009) [2023-12-27 17:33:14,254][20381] Updated weights for policy 1, policy_version 1717401 (0.0006) [2023-12-27 17:33:14,310][20381] Updated weights for policy 1, policy_version 1717411 (0.0006) [2023-12-27 17:33:14,370][20381] Updated weights for policy 1, policy_version 1717421 (0.0005) [2023-12-27 17:33:14,518][20347] Updated weights for policy 0, policy_version 1718832 (0.0008) [2023-12-27 17:33:14,573][20347] Updated weights for policy 0, policy_version 1718842 (0.0007) [2023-12-27 17:33:14,616][20347] Updated weights for policy 0, policy_version 1718852 (0.0007) [2023-12-27 17:33:14,978][20381] Updated weights for policy 1, policy_version 1717431 (0.0006) [2023-12-27 17:33:15,042][20381] Updated weights for policy 1, policy_version 1717441 (0.0006) [2023-12-27 17:33:15,107][20381] Updated weights for policy 1, policy_version 1717451 (0.0006) [2023-12-27 17:33:15,289][20347] Updated weights for policy 0, policy_version 1718862 (0.0007) [2023-12-27 17:33:15,352][20347] Updated weights for policy 0, policy_version 1718872 (0.0007) [2023-12-27 17:33:15,407][20347] Updated weights for policy 0, policy_version 1718882 (0.0008) [2023-12-27 17:33:15,737][20381] Updated weights for policy 1, policy_version 1717461 (0.0006) [2023-12-27 17:33:15,787][20381] Updated weights for policy 1, policy_version 1717471 (0.0005) [2023-12-27 17:33:15,841][20381] Updated weights for policy 1, policy_version 1717481 (0.0005) [2023-12-27 17:33:16,089][20347] Updated weights for policy 0, policy_version 1718892 (0.0007) [2023-12-27 17:33:16,134][20347] Updated weights for policy 0, policy_version 1718902 (0.0007) [2023-12-27 17:33:16,183][20347] Updated weights for policy 0, policy_version 1718912 (0.0008) [2023-12-27 17:33:16,472][20381] Updated weights for policy 1, policy_version 1717491 (0.0005) [2023-12-27 17:33:16,528][20381] Updated weights for policy 1, policy_version 1717501 (0.0005) [2023-12-27 17:33:16,574][20381] Updated weights for policy 1, policy_version 1717511 (0.0005) [2023-12-27 17:33:16,923][20347] Updated weights for policy 0, policy_version 1718922 (0.0007) [2023-12-27 17:33:16,973][20347] Updated weights for policy 0, policy_version 1718932 (0.0005) [2023-12-27 17:33:17,030][20347] Updated weights for policy 0, policy_version 1718942 (0.0005) [2023-12-27 17:33:17,075][20347] Updated weights for policy 0, policy_version 1718952 (0.0005) [2023-12-27 17:33:17,199][20381] Updated weights for policy 1, policy_version 1717521 (0.0006) [2023-12-27 17:33:17,258][20381] Updated weights for policy 1, policy_version 1717531 (0.0008) [2023-12-27 17:33:17,315][20381] Updated weights for policy 1, policy_version 1717541 (0.0008) [2023-12-27 17:33:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 879861760. Throughput: 0: 10316.3, 1: 10238.4. Samples: 879874852. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:17,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:33:17,364][20381] Updated weights for policy 1, policy_version 1717551 (0.0008) [2023-12-27 17:33:17,635][20347] Updated weights for policy 0, policy_version 1718962 (0.0009) [2023-12-27 17:33:17,680][20347] Updated weights for policy 0, policy_version 1718972 (0.0010) [2023-12-27 17:33:17,731][20347] Updated weights for policy 0, policy_version 1718982 (0.0010) [2023-12-27 17:33:18,014][20381] Updated weights for policy 1, policy_version 1717561 (0.0006) [2023-12-27 17:33:18,072][20381] Updated weights for policy 1, policy_version 1717571 (0.0005) [2023-12-27 17:33:18,127][20381] Updated weights for policy 1, policy_version 1717581 (0.0005) [2023-12-27 17:33:18,387][20347] Updated weights for policy 0, policy_version 1718992 (0.0007) [2023-12-27 17:33:18,446][20347] Updated weights for policy 0, policy_version 1719002 (0.0006) [2023-12-27 17:33:18,508][20347] Updated weights for policy 0, policy_version 1719012 (0.0006) [2023-12-27 17:33:18,753][20381] Updated weights for policy 1, policy_version 1717591 (0.0005) [2023-12-27 17:33:18,807][20381] Updated weights for policy 1, policy_version 1717601 (0.0007) [2023-12-27 17:33:18,853][20381] Updated weights for policy 1, policy_version 1717611 (0.0008) [2023-12-27 17:33:19,110][20347] Updated weights for policy 0, policy_version 1719022 (0.0005) [2023-12-27 17:33:19,161][20347] Updated weights for policy 0, policy_version 1719032 (0.0005) [2023-12-27 17:33:19,210][20347] Updated weights for policy 0, policy_version 1719042 (0.0005) [2023-12-27 17:33:19,661][20381] Updated weights for policy 1, policy_version 1717621 (0.0007) [2023-12-27 17:33:19,719][20381] Updated weights for policy 1, policy_version 1717631 (0.0006) [2023-12-27 17:33:19,778][20381] Updated weights for policy 1, policy_version 1717641 (0.0006) [2023-12-27 17:33:19,887][20347] Updated weights for policy 0, policy_version 1719052 (0.0008) [2023-12-27 17:33:19,951][20347] Updated weights for policy 0, policy_version 1719062 (0.0008) [2023-12-27 17:33:20,007][20347] Updated weights for policy 0, policy_version 1719072 (0.0008) [2023-12-27 17:33:20,461][20381] Updated weights for policy 1, policy_version 1717651 (0.0007) [2023-12-27 17:33:20,532][20381] Updated weights for policy 1, policy_version 1717661 (0.0009) [2023-12-27 17:33:20,596][20381] Updated weights for policy 1, policy_version 1717671 (0.0009) [2023-12-27 17:33:20,748][20347] Updated weights for policy 0, policy_version 1719082 (0.0008) [2023-12-27 17:33:20,813][20347] Updated weights for policy 0, policy_version 1719092 (0.0010) [2023-12-27 17:33:20,878][20347] Updated weights for policy 0, policy_version 1719102 (0.0011) [2023-12-27 17:33:20,884][20291] KL-divergence is very high: 113.0264 [2023-12-27 17:33:20,933][20291] KL-divergence is very high: 123.3100 [2023-12-27 17:33:20,940][20347] Updated weights for policy 0, policy_version 1719112 (0.0010) [2023-12-27 17:33:21,286][20381] Updated weights for policy 1, policy_version 1717681 (0.0008) [2023-12-27 17:33:21,347][20381] Updated weights for policy 1, policy_version 1717691 (0.0011) [2023-12-27 17:33:21,421][20381] Updated weights for policy 1, policy_version 1717701 (0.0012) [2023-12-27 17:33:21,479][20381] Updated weights for policy 1, policy_version 1717711 (0.0007) [2023-12-27 17:33:21,672][20347] Updated weights for policy 0, policy_version 1719122 (0.0008) [2023-12-27 17:33:21,743][20347] Updated weights for policy 0, policy_version 1719132 (0.0007) [2023-12-27 17:33:21,807][20347] Updated weights for policy 0, policy_version 1719142 (0.0007) [2023-12-27 17:33:22,151][20381] Updated weights for policy 1, policy_version 1717721 (0.0005) [2023-12-27 17:33:22,208][20381] Updated weights for policy 1, policy_version 1717731 (0.0005) [2023-12-27 17:33:22,268][20381] Updated weights for policy 1, policy_version 1717741 (0.0009) [2023-12-27 17:33:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 879968256. Throughput: 0: 10366.4, 1: 10211.6. Samples: 879938044. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:22,363][19170] Avg episode reward: [(0, '960.850'), (1, '990.270')] [2023-12-27 17:33:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001719144_440180736.pth... [2023-12-27 17:33:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001717744_439787520.pth... [2023-12-27 17:33:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001717928_439869440.pth [2023-12-27 17:33:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001716528_439476224.pth [2023-12-27 17:33:22,549][20347] Updated weights for policy 0, policy_version 1719152 (0.0007) [2023-12-27 17:33:22,607][20347] Updated weights for policy 0, policy_version 1719162 (0.0006) [2023-12-27 17:33:22,667][20347] Updated weights for policy 0, policy_version 1719172 (0.0008) [2023-12-27 17:33:23,014][20381] Updated weights for policy 1, policy_version 1717751 (0.0007) [2023-12-27 17:33:23,066][20381] Updated weights for policy 1, policy_version 1717761 (0.0008) [2023-12-27 17:33:23,129][20381] Updated weights for policy 1, policy_version 1717771 (0.0007) [2023-12-27 17:33:23,355][20347] Updated weights for policy 0, policy_version 1719182 (0.0007) [2023-12-27 17:33:23,413][20347] Updated weights for policy 0, policy_version 1719192 (0.0008) [2023-12-27 17:33:23,470][20347] Updated weights for policy 0, policy_version 1719202 (0.0008) [2023-12-27 17:33:23,778][20381] Updated weights for policy 1, policy_version 1717781 (0.0008) [2023-12-27 17:33:23,834][20381] Updated weights for policy 1, policy_version 1717791 (0.0008) [2023-12-27 17:33:23,887][20381] Updated weights for policy 1, policy_version 1717801 (0.0005) [2023-12-27 17:33:24,131][20347] Updated weights for policy 0, policy_version 1719212 (0.0008) [2023-12-27 17:33:24,201][20347] Updated weights for policy 0, policy_version 1719222 (0.0008) [2023-12-27 17:33:24,258][20347] Updated weights for policy 0, policy_version 1719232 (0.0008) [2023-12-27 17:33:24,509][20381] Updated weights for policy 1, policy_version 1717811 (0.0007) [2023-12-27 17:33:24,561][20381] Updated weights for policy 1, policy_version 1717821 (0.0010) [2023-12-27 17:33:24,615][20381] Updated weights for policy 1, policy_version 1717831 (0.0010) [2023-12-27 17:33:24,875][20347] Updated weights for policy 0, policy_version 1719242 (0.0007) [2023-12-27 17:33:24,928][20347] Updated weights for policy 0, policy_version 1719252 (0.0005) [2023-12-27 17:33:24,973][20347] Updated weights for policy 0, policy_version 1719262 (0.0006) [2023-12-27 17:33:25,023][20347] Updated weights for policy 0, policy_version 1719272 (0.0007) [2023-12-27 17:33:25,251][20381] Updated weights for policy 1, policy_version 1717841 (0.0010) [2023-12-27 17:33:25,309][20381] Updated weights for policy 1, policy_version 1717851 (0.0011) [2023-12-27 17:33:25,360][20381] Updated weights for policy 1, policy_version 1717861 (0.0010) [2023-12-27 17:33:25,408][20381] Updated weights for policy 1, policy_version 1717871 (0.0010) [2023-12-27 17:33:25,699][20347] Updated weights for policy 0, policy_version 1719282 (0.0007) [2023-12-27 17:33:25,744][20347] Updated weights for policy 0, policy_version 1719292 (0.0007) [2023-12-27 17:33:25,791][20347] Updated weights for policy 0, policy_version 1719302 (0.0007) [2023-12-27 17:33:26,092][20381] Updated weights for policy 1, policy_version 1717881 (0.0011) [2023-12-27 17:33:26,147][20381] Updated weights for policy 1, policy_version 1717891 (0.0011) [2023-12-27 17:33:26,206][20381] Updated weights for policy 1, policy_version 1717901 (0.0011) [2023-12-27 17:33:26,486][20347] Updated weights for policy 0, policy_version 1719312 (0.0008) [2023-12-27 17:33:26,539][20347] Updated weights for policy 0, policy_version 1719322 (0.0008) [2023-12-27 17:33:26,599][20347] Updated weights for policy 0, policy_version 1719332 (0.0007) [2023-12-27 17:33:26,898][20381] Updated weights for policy 1, policy_version 1717911 (0.0010) [2023-12-27 17:33:26,945][20381] Updated weights for policy 1, policy_version 1717921 (0.0010) [2023-12-27 17:33:27,001][20381] Updated weights for policy 1, policy_version 1717931 (0.0007) [2023-12-27 17:33:27,180][20347] Updated weights for policy 0, policy_version 1719342 (0.0007) [2023-12-27 17:33:27,229][20347] Updated weights for policy 0, policy_version 1719352 (0.0006) [2023-12-27 17:33:27,280][20347] Updated weights for policy 0, policy_version 1719362 (0.0005) [2023-12-27 17:33:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 880074752. Throughput: 0: 10293.6, 1: 10256.3. Samples: 880059136. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:27,363][19170] Avg episode reward: [(0, '970.640'), (1, '990.270')] [2023-12-27 17:33:27,617][20381] Updated weights for policy 1, policy_version 1717941 (0.0006) [2023-12-27 17:33:27,665][20381] Updated weights for policy 1, policy_version 1717951 (0.0010) [2023-12-27 17:33:27,727][20381] Updated weights for policy 1, policy_version 1717961 (0.0009) [2023-12-27 17:33:27,859][20347] Updated weights for policy 0, policy_version 1719372 (0.0006) [2023-12-27 17:33:27,910][20347] Updated weights for policy 0, policy_version 1719382 (0.0008) [2023-12-27 17:33:27,969][20347] Updated weights for policy 0, policy_version 1719392 (0.0007) [2023-12-27 17:33:28,387][20381] Updated weights for policy 1, policy_version 1717971 (0.0006) [2023-12-27 17:33:28,450][20381] Updated weights for policy 1, policy_version 1717981 (0.0008) [2023-12-27 17:33:28,511][20381] Updated weights for policy 1, policy_version 1717991 (0.0007) [2023-12-27 17:33:28,624][20347] Updated weights for policy 0, policy_version 1719402 (0.0007) [2023-12-27 17:33:28,685][20347] Updated weights for policy 0, policy_version 1719412 (0.0010) [2023-12-27 17:33:28,743][20347] Updated weights for policy 0, policy_version 1719422 (0.0010) [2023-12-27 17:33:28,803][20347] Updated weights for policy 0, policy_version 1719432 (0.0010) [2023-12-27 17:33:29,218][20381] Updated weights for policy 1, policy_version 1718001 (0.0007) [2023-12-27 17:33:29,282][20381] Updated weights for policy 1, policy_version 1718011 (0.0008) [2023-12-27 17:33:29,346][20381] Updated weights for policy 1, policy_version 1718021 (0.0007) [2023-12-27 17:33:29,411][20381] Updated weights for policy 1, policy_version 1718031 (0.0007) [2023-12-27 17:33:29,430][20347] Updated weights for policy 0, policy_version 1719442 (0.0006) [2023-12-27 17:33:29,488][20347] Updated weights for policy 0, policy_version 1719452 (0.0008) [2023-12-27 17:33:29,545][20347] Updated weights for policy 0, policy_version 1719462 (0.0007) [2023-12-27 17:33:29,984][20381] Updated weights for policy 1, policy_version 1718041 (0.0009) [2023-12-27 17:33:30,038][20381] Updated weights for policy 1, policy_version 1718051 (0.0010) [2023-12-27 17:33:30,091][20381] Updated weights for policy 1, policy_version 1718061 (0.0005) [2023-12-27 17:33:30,190][20347] Updated weights for policy 0, policy_version 1719472 (0.0006) [2023-12-27 17:33:30,251][20347] Updated weights for policy 0, policy_version 1719482 (0.0008) [2023-12-27 17:33:30,309][20347] Updated weights for policy 0, policy_version 1719492 (0.0007) [2023-12-27 17:33:30,679][20381] Updated weights for policy 1, policy_version 1718071 (0.0007) [2023-12-27 17:33:30,729][20381] Updated weights for policy 1, policy_version 1718081 (0.0008) [2023-12-27 17:33:30,773][20381] Updated weights for policy 1, policy_version 1718091 (0.0007) [2023-12-27 17:33:30,891][20347] Updated weights for policy 0, policy_version 1719502 (0.0006) [2023-12-27 17:33:30,940][20347] Updated weights for policy 0, policy_version 1719512 (0.0008) [2023-12-27 17:33:30,995][20347] Updated weights for policy 0, policy_version 1719522 (0.0008) [2023-12-27 17:33:31,511][20381] Updated weights for policy 1, policy_version 1718101 (0.0006) [2023-12-27 17:33:31,572][20381] Updated weights for policy 1, policy_version 1718111 (0.0007) [2023-12-27 17:33:31,636][20381] Updated weights for policy 1, policy_version 1718121 (0.0007) [2023-12-27 17:33:31,761][20347] Updated weights for policy 0, policy_version 1719532 (0.0007) [2023-12-27 17:33:31,818][20347] Updated weights for policy 0, policy_version 1719542 (0.0008) [2023-12-27 17:33:31,865][20347] Updated weights for policy 0, policy_version 1719552 (0.0008) [2023-12-27 17:33:32,237][20381] Updated weights for policy 1, policy_version 1718131 (0.0008) [2023-12-27 17:33:32,295][20381] Updated weights for policy 1, policy_version 1718141 (0.0008) [2023-12-27 17:33:32,357][20381] Updated weights for policy 1, policy_version 1718151 (0.0007) [2023-12-27 17:33:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 880173056. Throughput: 0: 10350.1, 1: 10347.1. Samples: 880187404. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:32,363][19170] Avg episode reward: [(0, '980.530'), (1, '1000.000')] [2023-12-27 17:33:32,600][20347] Updated weights for policy 0, policy_version 1719562 (0.0008) [2023-12-27 17:33:32,662][20347] Updated weights for policy 0, policy_version 1719572 (0.0007) [2023-12-27 17:33:32,727][20347] Updated weights for policy 0, policy_version 1719582 (0.0007) [2023-12-27 17:33:32,787][20347] Updated weights for policy 0, policy_version 1719592 (0.0008) [2023-12-27 17:33:33,026][20381] Updated weights for policy 1, policy_version 1718161 (0.0009) [2023-12-27 17:33:33,077][20381] Updated weights for policy 1, policy_version 1718171 (0.0010) [2023-12-27 17:33:33,131][20381] Updated weights for policy 1, policy_version 1718181 (0.0010) [2023-12-27 17:33:33,186][20381] Updated weights for policy 1, policy_version 1718191 (0.0010) [2023-12-27 17:33:33,473][20347] Updated weights for policy 0, policy_version 1719602 (0.0005) [2023-12-27 17:33:33,527][20347] Updated weights for policy 0, policy_version 1719612 (0.0005) [2023-12-27 17:33:33,580][20347] Updated weights for policy 0, policy_version 1719622 (0.0005) [2023-12-27 17:33:33,889][20381] Updated weights for policy 1, policy_version 1718201 (0.0007) [2023-12-27 17:33:33,941][20381] Updated weights for policy 1, policy_version 1718211 (0.0007) [2023-12-27 17:33:33,999][20381] Updated weights for policy 1, policy_version 1718221 (0.0007) [2023-12-27 17:33:34,123][20347] Updated weights for policy 0, policy_version 1719632 (0.0005) [2023-12-27 17:33:34,195][20347] Updated weights for policy 0, policy_version 1719642 (0.0007) [2023-12-27 17:33:34,252][20347] Updated weights for policy 0, policy_version 1719652 (0.0007) [2023-12-27 17:33:34,709][20381] Updated weights for policy 1, policy_version 1718231 (0.0008) [2023-12-27 17:33:34,778][20381] Updated weights for policy 1, policy_version 1718241 (0.0008) [2023-12-27 17:33:34,838][20381] Updated weights for policy 1, policy_version 1718251 (0.0008) [2023-12-27 17:33:34,913][20347] Updated weights for policy 0, policy_version 1719662 (0.0007) [2023-12-27 17:33:34,962][20347] Updated weights for policy 0, policy_version 1719672 (0.0005) [2023-12-27 17:33:35,014][20347] Updated weights for policy 0, policy_version 1719682 (0.0006) [2023-12-27 17:33:35,559][20381] Updated weights for policy 1, policy_version 1718261 (0.0007) [2023-12-27 17:33:35,579][20347] Updated weights for policy 0, policy_version 1719692 (0.0006) [2023-12-27 17:33:35,609][20381] Updated weights for policy 1, policy_version 1718271 (0.0006) [2023-12-27 17:33:35,628][20347] Updated weights for policy 0, policy_version 1719702 (0.0006) [2023-12-27 17:33:35,661][20381] Updated weights for policy 1, policy_version 1718281 (0.0008) [2023-12-27 17:33:35,681][20347] Updated weights for policy 0, policy_version 1719712 (0.0005) [2023-12-27 17:33:36,271][20347] Updated weights for policy 0, policy_version 1719722 (0.0006) [2023-12-27 17:33:36,329][20347] Updated weights for policy 0, policy_version 1719732 (0.0005) [2023-12-27 17:33:36,389][20347] Updated weights for policy 0, policy_version 1719742 (0.0008) [2023-12-27 17:33:36,431][20381] Updated weights for policy 1, policy_version 1718291 (0.0007) [2023-12-27 17:33:36,448][20347] Updated weights for policy 0, policy_version 1719752 (0.0009) [2023-12-27 17:33:36,495][20381] Updated weights for policy 1, policy_version 1718301 (0.0008) [2023-12-27 17:33:36,556][20381] Updated weights for policy 1, policy_version 1718311 (0.0008) [2023-12-27 17:33:37,107][20347] Updated weights for policy 0, policy_version 1719762 (0.0007) [2023-12-27 17:33:37,170][20347] Updated weights for policy 0, policy_version 1719772 (0.0011) [2023-12-27 17:33:37,217][20347] Updated weights for policy 0, policy_version 1719782 (0.0011) [2023-12-27 17:33:37,276][20381] Updated weights for policy 1, policy_version 1718321 (0.0007) [2023-12-27 17:33:37,335][20381] Updated weights for policy 1, policy_version 1718331 (0.0007) [2023-12-27 17:33:37,363][19170] Fps is (10 sec: 20479.1, 60 sec: 20616.4, 300 sec: 20493.9). Total num frames: 880279552. Throughput: 0: 10383.0, 1: 10318.5. Samples: 880249504. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:37,364][19170] Avg episode reward: [(0, '941.440'), (1, '1000.000')] [2023-12-27 17:33:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001719784_440344576.pth... [2023-12-27 17:33:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001718568_440033280.pth [2023-12-27 17:33:37,386][20381] Updated weights for policy 1, policy_version 1718341 (0.0005) [2023-12-27 17:33:37,445][20381] Updated weights for policy 1, policy_version 1718351 (0.0006) [2023-12-27 17:33:37,450][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001718352_439943168.pth... [2023-12-27 17:33:37,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001717136_439631872.pth [2023-12-27 17:33:37,884][20347] Updated weights for policy 0, policy_version 1719792 (0.0010) [2023-12-27 17:33:37,936][20347] Updated weights for policy 0, policy_version 1719802 (0.0010) [2023-12-27 17:33:37,986][20347] Updated weights for policy 0, policy_version 1719812 (0.0008) [2023-12-27 17:33:38,116][20381] Updated weights for policy 1, policy_version 1718361 (0.0006) [2023-12-27 17:33:38,163][20381] Updated weights for policy 1, policy_version 1718371 (0.0006) [2023-12-27 17:33:38,222][20381] Updated weights for policy 1, policy_version 1718381 (0.0005) [2023-12-27 17:33:38,659][20347] Updated weights for policy 0, policy_version 1719822 (0.0007) [2023-12-27 17:33:38,721][20347] Updated weights for policy 0, policy_version 1719832 (0.0009) [2023-12-27 17:33:38,778][20347] Updated weights for policy 0, policy_version 1719842 (0.0007) [2023-12-27 17:33:38,875][20381] Updated weights for policy 1, policy_version 1718391 (0.0008) [2023-12-27 17:33:38,938][20381] Updated weights for policy 1, policy_version 1718401 (0.0008) [2023-12-27 17:33:38,996][20381] Updated weights for policy 1, policy_version 1718411 (0.0007) [2023-12-27 17:33:39,422][20347] Updated weights for policy 0, policy_version 1719852 (0.0006) [2023-12-27 17:33:39,486][20347] Updated weights for policy 0, policy_version 1719862 (0.0006) [2023-12-27 17:33:39,553][20347] Updated weights for policy 0, policy_version 1719872 (0.0007) [2023-12-27 17:33:39,728][20381] Updated weights for policy 1, policy_version 1718421 (0.0007) [2023-12-27 17:33:39,784][20381] Updated weights for policy 1, policy_version 1718431 (0.0007) [2023-12-27 17:33:39,846][20381] Updated weights for policy 1, policy_version 1718441 (0.0008) [2023-12-27 17:33:40,234][20347] Updated weights for policy 0, policy_version 1719882 (0.0006) [2023-12-27 17:33:40,293][20347] Updated weights for policy 0, policy_version 1719892 (0.0008) [2023-12-27 17:33:40,352][20347] Updated weights for policy 0, policy_version 1719902 (0.0008) [2023-12-27 17:33:40,412][20347] Updated weights for policy 0, policy_version 1719912 (0.0008) [2023-12-27 17:33:40,591][20381] Updated weights for policy 1, policy_version 1718451 (0.0008) [2023-12-27 17:33:40,644][20381] Updated weights for policy 1, policy_version 1718461 (0.0008) [2023-12-27 17:33:40,696][20381] Updated weights for policy 1, policy_version 1718471 (0.0007) [2023-12-27 17:33:41,048][20347] Updated weights for policy 0, policy_version 1719922 (0.0007) [2023-12-27 17:33:41,116][20347] Updated weights for policy 0, policy_version 1719932 (0.0009) [2023-12-27 17:33:41,185][20347] Updated weights for policy 0, policy_version 1719942 (0.0008) [2023-12-27 17:33:41,443][20381] Updated weights for policy 1, policy_version 1718481 (0.0008) [2023-12-27 17:33:41,509][20381] Updated weights for policy 1, policy_version 1718491 (0.0010) [2023-12-27 17:33:41,575][20381] Updated weights for policy 1, policy_version 1718501 (0.0011) [2023-12-27 17:33:41,640][20381] Updated weights for policy 1, policy_version 1718511 (0.0011) [2023-12-27 17:33:41,958][20347] Updated weights for policy 0, policy_version 1719952 (0.0007) [2023-12-27 17:33:42,016][20347] Updated weights for policy 0, policy_version 1719962 (0.0007) [2023-12-27 17:33:42,073][20347] Updated weights for policy 0, policy_version 1719972 (0.0005) [2023-12-27 17:33:42,327][20381] Updated weights for policy 1, policy_version 1718521 (0.0010) [2023-12-27 17:33:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 880377856. Throughput: 0: 10395.6, 1: 10271.5. Samples: 880372256. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:42,363][19170] Avg episode reward: [(0, '912.070'), (1, '1000.000')] [2023-12-27 17:33:42,394][20381] Updated weights for policy 1, policy_version 1718531 (0.0007) [2023-12-27 17:33:42,449][20381] Updated weights for policy 1, policy_version 1718541 (0.0006) [2023-12-27 17:33:42,747][20347] Updated weights for policy 0, policy_version 1719982 (0.0007) [2023-12-27 17:33:42,807][20347] Updated weights for policy 0, policy_version 1719992 (0.0008) [2023-12-27 17:33:42,862][20347] Updated weights for policy 0, policy_version 1720002 (0.0007) [2023-12-27 17:33:43,098][20381] Updated weights for policy 1, policy_version 1718551 (0.0006) [2023-12-27 17:33:43,155][20381] Updated weights for policy 1, policy_version 1718561 (0.0008) [2023-12-27 17:33:43,212][20381] Updated weights for policy 1, policy_version 1718571 (0.0008) [2023-12-27 17:33:43,521][20347] Updated weights for policy 0, policy_version 1720012 (0.0007) [2023-12-27 17:33:43,576][20347] Updated weights for policy 0, policy_version 1720022 (0.0008) [2023-12-27 17:33:43,621][20347] Updated weights for policy 0, policy_version 1720032 (0.0008) [2023-12-27 17:33:43,901][20381] Updated weights for policy 1, policy_version 1718581 (0.0007) [2023-12-27 17:33:43,955][20381] Updated weights for policy 1, policy_version 1718591 (0.0008) [2023-12-27 17:33:44,004][20381] Updated weights for policy 1, policy_version 1718601 (0.0008) [2023-12-27 17:33:44,324][20347] Updated weights for policy 0, policy_version 1720042 (0.0007) [2023-12-27 17:33:44,376][20347] Updated weights for policy 0, policy_version 1720052 (0.0007) [2023-12-27 17:33:44,424][20347] Updated weights for policy 0, policy_version 1720062 (0.0008) [2023-12-27 17:33:44,472][20347] Updated weights for policy 0, policy_version 1720072 (0.0007) [2023-12-27 17:33:44,697][20381] Updated weights for policy 1, policy_version 1718611 (0.0008) [2023-12-27 17:33:44,758][20381] Updated weights for policy 1, policy_version 1718621 (0.0010) [2023-12-27 17:33:44,824][20381] Updated weights for policy 1, policy_version 1718631 (0.0009) [2023-12-27 17:33:45,212][20347] Updated weights for policy 0, policy_version 1720082 (0.0007) [2023-12-27 17:33:45,265][20347] Updated weights for policy 0, policy_version 1720092 (0.0007) [2023-12-27 17:33:45,327][20347] Updated weights for policy 0, policy_version 1720102 (0.0008) [2023-12-27 17:33:45,550][20381] Updated weights for policy 1, policy_version 1718641 (0.0010) [2023-12-27 17:33:45,599][20381] Updated weights for policy 1, policy_version 1718651 (0.0011) [2023-12-27 17:33:45,651][20381] Updated weights for policy 1, policy_version 1718661 (0.0010) [2023-12-27 17:33:45,710][20381] Updated weights for policy 1, policy_version 1718671 (0.0005) [2023-12-27 17:33:45,984][20347] Updated weights for policy 0, policy_version 1720112 (0.0007) [2023-12-27 17:33:46,031][20347] Updated weights for policy 0, policy_version 1720122 (0.0007) [2023-12-27 17:33:46,085][20347] Updated weights for policy 0, policy_version 1720132 (0.0007) [2023-12-27 17:33:46,315][20381] Updated weights for policy 1, policy_version 1718681 (0.0010) [2023-12-27 17:33:46,368][20381] Updated weights for policy 1, policy_version 1718691 (0.0007) [2023-12-27 17:33:46,434][20381] Updated weights for policy 1, policy_version 1718701 (0.0005) [2023-12-27 17:33:46,835][20347] Updated weights for policy 0, policy_version 1720142 (0.0008) [2023-12-27 17:33:46,898][20347] Updated weights for policy 0, policy_version 1720152 (0.0008) [2023-12-27 17:33:46,946][20347] Updated weights for policy 0, policy_version 1720162 (0.0007) [2023-12-27 17:33:47,008][20381] Updated weights for policy 1, policy_version 1718711 (0.0005) [2023-12-27 17:33:47,061][20381] Updated weights for policy 1, policy_version 1718721 (0.0005) [2023-12-27 17:33:47,118][20381] Updated weights for policy 1, policy_version 1718731 (0.0006) [2023-12-27 17:33:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 880484352. Throughput: 0: 10340.9, 1: 10332.5. Samples: 880493732. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:47,363][19170] Avg episode reward: [(0, '941.390'), (1, '1000.000')] [2023-12-27 17:33:47,668][20347] Updated weights for policy 0, policy_version 1720172 (0.0008) [2023-12-27 17:33:47,675][20381] Updated weights for policy 1, policy_version 1718741 (0.0008) [2023-12-27 17:33:47,730][20381] Updated weights for policy 1, policy_version 1718751 (0.0010) [2023-12-27 17:33:47,733][20347] Updated weights for policy 0, policy_version 1720182 (0.0007) [2023-12-27 17:33:47,778][20381] Updated weights for policy 1, policy_version 1718761 (0.0005) [2023-12-27 17:33:47,796][20347] Updated weights for policy 0, policy_version 1720192 (0.0007) [2023-12-27 17:33:48,443][20381] Updated weights for policy 1, policy_version 1718771 (0.0007) [2023-12-27 17:33:48,492][20347] Updated weights for policy 0, policy_version 1720202 (0.0009) [2023-12-27 17:33:48,498][20381] Updated weights for policy 1, policy_version 1718781 (0.0011) [2023-12-27 17:33:48,548][20347] Updated weights for policy 0, policy_version 1720212 (0.0010) [2023-12-27 17:33:48,554][20381] Updated weights for policy 1, policy_version 1718791 (0.0011) [2023-12-27 17:33:48,603][20347] Updated weights for policy 0, policy_version 1720222 (0.0011) [2023-12-27 17:33:48,655][20347] Updated weights for policy 0, policy_version 1720232 (0.0010) [2023-12-27 17:33:49,245][20381] Updated weights for policy 1, policy_version 1718801 (0.0010) [2023-12-27 17:33:49,281][20347] Updated weights for policy 0, policy_version 1720242 (0.0009) [2023-12-27 17:33:49,297][20381] Updated weights for policy 1, policy_version 1718811 (0.0008) [2023-12-27 17:33:49,335][20347] Updated weights for policy 0, policy_version 1720252 (0.0010) [2023-12-27 17:33:49,353][20381] Updated weights for policy 1, policy_version 1718821 (0.0008) [2023-12-27 17:33:49,394][20347] Updated weights for policy 0, policy_version 1720262 (0.0006) [2023-12-27 17:33:49,419][20381] Updated weights for policy 1, policy_version 1718831 (0.0009) [2023-12-27 17:33:50,078][20347] Updated weights for policy 0, policy_version 1720272 (0.0006) [2023-12-27 17:33:50,094][20381] Updated weights for policy 1, policy_version 1718841 (0.0007) [2023-12-27 17:33:50,123][20347] Updated weights for policy 0, policy_version 1720282 (0.0006) [2023-12-27 17:33:50,152][20381] Updated weights for policy 1, policy_version 1718851 (0.0007) [2023-12-27 17:33:50,166][20347] Updated weights for policy 0, policy_version 1720292 (0.0006) [2023-12-27 17:33:50,213][20381] Updated weights for policy 1, policy_version 1718861 (0.0007) [2023-12-27 17:33:50,909][20347] Updated weights for policy 0, policy_version 1720302 (0.0008) [2023-12-27 17:33:50,958][20381] Updated weights for policy 1, policy_version 1718871 (0.0006) [2023-12-27 17:33:50,960][20347] Updated weights for policy 0, policy_version 1720312 (0.0006) [2023-12-27 17:33:51,011][20347] Updated weights for policy 0, policy_version 1720322 (0.0006) [2023-12-27 17:33:51,016][20381] Updated weights for policy 1, policy_version 1718881 (0.0007) [2023-12-27 17:33:51,080][20381] Updated weights for policy 1, policy_version 1718891 (0.0007) [2023-12-27 17:33:51,775][20347] Updated weights for policy 0, policy_version 1720332 (0.0008) [2023-12-27 17:33:51,827][20381] Updated weights for policy 1, policy_version 1718901 (0.0007) [2023-12-27 17:33:51,829][20347] Updated weights for policy 0, policy_version 1720342 (0.0007) [2023-12-27 17:33:51,881][20381] Updated weights for policy 1, policy_version 1718911 (0.0006) [2023-12-27 17:33:51,883][20347] Updated weights for policy 0, policy_version 1720352 (0.0006) [2023-12-27 17:33:51,939][20381] Updated weights for policy 1, policy_version 1718921 (0.0006) [2023-12-27 17:33:52,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 880582656. Throughput: 0: 10308.2, 1: 10326.5. Samples: 880555840. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:52,363][19170] Avg episode reward: [(0, '970.680'), (1, '1000.000')] [2023-12-27 17:33:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001720360_440492032.pth... [2023-12-27 17:33:52,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001718928_440090624.pth... [2023-12-27 17:33:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001717744_439787520.pth [2023-12-27 17:33:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001719144_440180736.pth [2023-12-27 17:33:52,575][20347] Updated weights for policy 0, policy_version 1720362 (0.0006) [2023-12-27 17:33:52,640][20347] Updated weights for policy 0, policy_version 1720372 (0.0005) [2023-12-27 17:33:52,689][20381] Updated weights for policy 1, policy_version 1718931 (0.0007) [2023-12-27 17:33:52,699][20347] Updated weights for policy 0, policy_version 1720382 (0.0007) [2023-12-27 17:33:52,745][20381] Updated weights for policy 1, policy_version 1718941 (0.0006) [2023-12-27 17:33:52,750][20347] Updated weights for policy 0, policy_version 1720392 (0.0006) [2023-12-27 17:33:52,795][20381] Updated weights for policy 1, policy_version 1718951 (0.0008) [2023-12-27 17:33:53,328][20347] Updated weights for policy 0, policy_version 1720402 (0.0008) [2023-12-27 17:33:53,375][20347] Updated weights for policy 0, policy_version 1720412 (0.0007) [2023-12-27 17:33:53,439][20347] Updated weights for policy 0, policy_version 1720422 (0.0008) [2023-12-27 17:33:53,505][20381] Updated weights for policy 1, policy_version 1718961 (0.0009) [2023-12-27 17:33:53,557][20381] Updated weights for policy 1, policy_version 1718971 (0.0008) [2023-12-27 17:33:53,608][20381] Updated weights for policy 1, policy_version 1718981 (0.0007) [2023-12-27 17:33:53,668][20381] Updated weights for policy 1, policy_version 1718991 (0.0008) [2023-12-27 17:33:54,133][20347] Updated weights for policy 0, policy_version 1720432 (0.0010) [2023-12-27 17:33:54,179][20347] Updated weights for policy 0, policy_version 1720442 (0.0010) [2023-12-27 17:33:54,236][20347] Updated weights for policy 0, policy_version 1720452 (0.0010) [2023-12-27 17:33:54,355][20381] Updated weights for policy 1, policy_version 1719001 (0.0011) [2023-12-27 17:33:54,408][20381] Updated weights for policy 1, policy_version 1719011 (0.0010) [2023-12-27 17:33:54,469][20381] Updated weights for policy 1, policy_version 1719021 (0.0010) [2023-12-27 17:33:54,887][20347] Updated weights for policy 0, policy_version 1720462 (0.0007) [2023-12-27 17:33:54,948][20347] Updated weights for policy 0, policy_version 1720472 (0.0008) [2023-12-27 17:33:55,008][20347] Updated weights for policy 0, policy_version 1720482 (0.0008) [2023-12-27 17:33:55,170][20381] Updated weights for policy 1, policy_version 1719031 (0.0008) [2023-12-27 17:33:55,230][20381] Updated weights for policy 1, policy_version 1719041 (0.0008) [2023-12-27 17:33:55,291][20381] Updated weights for policy 1, policy_version 1719051 (0.0008) [2023-12-27 17:33:55,626][20347] Updated weights for policy 0, policy_version 1720492 (0.0006) [2023-12-27 17:33:55,681][20347] Updated weights for policy 0, policy_version 1720502 (0.0005) [2023-12-27 17:33:55,741][20347] Updated weights for policy 0, policy_version 1720512 (0.0006) [2023-12-27 17:33:55,951][20381] Updated weights for policy 1, policy_version 1719061 (0.0007) [2023-12-27 17:33:56,012][20381] Updated weights for policy 1, policy_version 1719071 (0.0006) [2023-12-27 17:33:56,073][20381] Updated weights for policy 1, policy_version 1719081 (0.0005) [2023-12-27 17:33:56,410][20347] Updated weights for policy 0, policy_version 1720522 (0.0007) [2023-12-27 17:33:56,460][20347] Updated weights for policy 0, policy_version 1720532 (0.0005) [2023-12-27 17:33:56,513][20347] Updated weights for policy 0, policy_version 1720542 (0.0005) [2023-12-27 17:33:56,570][20347] Updated weights for policy 0, policy_version 1720552 (0.0006) [2023-12-27 17:33:56,623][20381] Updated weights for policy 1, policy_version 1719091 (0.0006) [2023-12-27 17:33:56,689][20381] Updated weights for policy 1, policy_version 1719101 (0.0008) [2023-12-27 17:33:56,750][20381] Updated weights for policy 1, policy_version 1719111 (0.0008) [2023-12-27 17:33:57,118][20347] Updated weights for policy 0, policy_version 1720562 (0.0005) [2023-12-27 17:33:57,173][20347] Updated weights for policy 0, policy_version 1720572 (0.0005) [2023-12-27 17:33:57,236][20347] Updated weights for policy 0, policy_version 1720582 (0.0005) [2023-12-27 17:33:57,312][20381] Updated weights for policy 1, policy_version 1719121 (0.0005) [2023-12-27 17:33:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 880689152. Throughput: 0: 10312.1, 1: 10243.9. Samples: 880676640. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:33:57,363][19170] Avg episode reward: [(0, '980.460'), (1, '1000.000')] [2023-12-27 17:33:57,370][20381] Updated weights for policy 1, policy_version 1719131 (0.0009) [2023-12-27 17:33:57,429][20381] Updated weights for policy 1, policy_version 1719141 (0.0008) [2023-12-27 17:33:57,490][20381] Updated weights for policy 1, policy_version 1719151 (0.0010) [2023-12-27 17:33:57,746][20347] Updated weights for policy 0, policy_version 1720592 (0.0009) [2023-12-27 17:33:57,802][20347] Updated weights for policy 0, policy_version 1720602 (0.0010) [2023-12-27 17:33:57,856][20347] Updated weights for policy 0, policy_version 1720612 (0.0010) [2023-12-27 17:33:58,076][20381] Updated weights for policy 1, policy_version 1719161 (0.0010) [2023-12-27 17:33:58,128][20381] Updated weights for policy 1, policy_version 1719171 (0.0010) [2023-12-27 17:33:58,179][20381] Updated weights for policy 1, policy_version 1719181 (0.0010) [2023-12-27 17:33:58,620][20347] Updated weights for policy 0, policy_version 1720622 (0.0009) [2023-12-27 17:33:58,675][20347] Updated weights for policy 0, policy_version 1720632 (0.0008) [2023-12-27 17:33:58,743][20347] Updated weights for policy 0, policy_version 1720642 (0.0008) [2023-12-27 17:33:58,996][20381] Updated weights for policy 1, policy_version 1719191 (0.0008) [2023-12-27 17:33:59,055][20381] Updated weights for policy 1, policy_version 1719201 (0.0007) [2023-12-27 17:33:59,112][20381] Updated weights for policy 1, policy_version 1719211 (0.0007) [2023-12-27 17:33:59,513][20347] Updated weights for policy 0, policy_version 1720652 (0.0007) [2023-12-27 17:33:59,564][20347] Updated weights for policy 0, policy_version 1720662 (0.0005) [2023-12-27 17:33:59,615][20347] Updated weights for policy 0, policy_version 1720672 (0.0006) [2023-12-27 17:33:59,890][20381] Updated weights for policy 1, policy_version 1719221 (0.0007) [2023-12-27 17:33:59,955][20381] Updated weights for policy 1, policy_version 1719231 (0.0007) [2023-12-27 17:34:00,010][20381] Updated weights for policy 1, policy_version 1719241 (0.0007) [2023-12-27 17:34:00,277][20347] Updated weights for policy 0, policy_version 1720682 (0.0007) [2023-12-27 17:34:00,329][20347] Updated weights for policy 0, policy_version 1720692 (0.0008) [2023-12-27 17:34:00,379][20347] Updated weights for policy 0, policy_version 1720702 (0.0008) [2023-12-27 17:34:00,442][20347] Updated weights for policy 0, policy_version 1720712 (0.0007) [2023-12-27 17:34:00,680][20381] Updated weights for policy 1, policy_version 1719251 (0.0007) [2023-12-27 17:34:00,729][20381] Updated weights for policy 1, policy_version 1719261 (0.0007) [2023-12-27 17:34:00,783][20381] Updated weights for policy 1, policy_version 1719271 (0.0005) [2023-12-27 17:34:01,185][20347] Updated weights for policy 0, policy_version 1720722 (0.0008) [2023-12-27 17:34:01,234][20347] Updated weights for policy 0, policy_version 1720732 (0.0007) [2023-12-27 17:34:01,296][20347] Updated weights for policy 0, policy_version 1720742 (0.0008) [2023-12-27 17:34:01,453][20381] Updated weights for policy 1, policy_version 1719281 (0.0006) [2023-12-27 17:34:01,512][20381] Updated weights for policy 1, policy_version 1719291 (0.0010) [2023-12-27 17:34:01,565][20381] Updated weights for policy 1, policy_version 1719301 (0.0005) [2023-12-27 17:34:01,625][20381] Updated weights for policy 1, policy_version 1719311 (0.0007) [2023-12-27 17:34:02,064][20347] Updated weights for policy 0, policy_version 1720752 (0.0009) [2023-12-27 17:34:02,123][20347] Updated weights for policy 0, policy_version 1720762 (0.0010) [2023-12-27 17:34:02,182][20347] Updated weights for policy 0, policy_version 1720772 (0.0010) [2023-12-27 17:34:02,317][20381] Updated weights for policy 1, policy_version 1719321 (0.0008) [2023-12-27 17:34:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 880787456. Throughput: 0: 10354.4, 1: 10190.4. Samples: 880799368. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 17:34:02,380][20381] Updated weights for policy 1, policy_version 1719331 (0.0007) [2023-12-27 17:34:02,441][20381] Updated weights for policy 1, policy_version 1719341 (0.0007) [2023-12-27 17:34:02,848][20347] Updated weights for policy 0, policy_version 1720782 (0.0010) [2023-12-27 17:34:02,916][20347] Updated weights for policy 0, policy_version 1720792 (0.0011) [2023-12-27 17:34:02,987][20347] Updated weights for policy 0, policy_version 1720802 (0.0011) [2023-12-27 17:34:03,141][20381] Updated weights for policy 1, policy_version 1719351 (0.0008) [2023-12-27 17:34:03,195][20381] Updated weights for policy 1, policy_version 1719361 (0.0007) [2023-12-27 17:34:03,254][20381] Updated weights for policy 1, policy_version 1719371 (0.0008) [2023-12-27 17:34:03,625][20347] Updated weights for policy 0, policy_version 1720812 (0.0009) [2023-12-27 17:34:03,669][20347] Updated weights for policy 0, policy_version 1720822 (0.0010) [2023-12-27 17:34:03,714][20347] Updated weights for policy 0, policy_version 1720832 (0.0010) [2023-12-27 17:34:03,960][20381] Updated weights for policy 1, policy_version 1719381 (0.0007) [2023-12-27 17:34:04,019][20381] Updated weights for policy 1, policy_version 1719391 (0.0007) [2023-12-27 17:34:04,078][20381] Updated weights for policy 1, policy_version 1719401 (0.0007) [2023-12-27 17:34:04,405][20347] Updated weights for policy 0, policy_version 1720842 (0.0010) [2023-12-27 17:34:04,462][20347] Updated weights for policy 0, policy_version 1720852 (0.0008) [2023-12-27 17:34:04,519][20347] Updated weights for policy 0, policy_version 1720862 (0.0008) [2023-12-27 17:34:04,576][20347] Updated weights for policy 0, policy_version 1720872 (0.0008) [2023-12-27 17:34:04,769][20381] Updated weights for policy 1, policy_version 1719411 (0.0008) [2023-12-27 17:34:04,818][20381] Updated weights for policy 1, policy_version 1719421 (0.0008) [2023-12-27 17:34:04,867][20381] Updated weights for policy 1, policy_version 1719431 (0.0008) [2023-12-27 17:34:05,218][20347] Updated weights for policy 0, policy_version 1720882 (0.0005) [2023-12-27 17:34:05,269][20347] Updated weights for policy 0, policy_version 1720892 (0.0005) [2023-12-27 17:34:05,320][20347] Updated weights for policy 0, policy_version 1720902 (0.0005) [2023-12-27 17:34:05,608][20381] Updated weights for policy 1, policy_version 1719441 (0.0007) [2023-12-27 17:34:05,663][20381] Updated weights for policy 1, policy_version 1719451 (0.0008) [2023-12-27 17:34:05,725][20381] Updated weights for policy 1, policy_version 1719461 (0.0006) [2023-12-27 17:34:05,786][20381] Updated weights for policy 1, policy_version 1719471 (0.0009) [2023-12-27 17:34:05,894][20347] Updated weights for policy 0, policy_version 1720912 (0.0007) [2023-12-27 17:34:05,949][20347] Updated weights for policy 0, policy_version 1720922 (0.0008) [2023-12-27 17:34:06,013][20347] Updated weights for policy 0, policy_version 1720932 (0.0008) [2023-12-27 17:34:06,425][20381] Updated weights for policy 1, policy_version 1719481 (0.0007) [2023-12-27 17:34:06,495][20381] Updated weights for policy 1, policy_version 1719491 (0.0007) [2023-12-27 17:34:06,554][20381] Updated weights for policy 1, policy_version 1719501 (0.0009) [2023-12-27 17:34:06,733][20347] Updated weights for policy 0, policy_version 1720942 (0.0009) [2023-12-27 17:34:06,794][20347] Updated weights for policy 0, policy_version 1720952 (0.0011) [2023-12-27 17:34:06,859][20347] Updated weights for policy 0, policy_version 1720962 (0.0011) [2023-12-27 17:34:07,182][20381] Updated weights for policy 1, policy_version 1719511 (0.0008) [2023-12-27 17:34:07,240][20381] Updated weights for policy 1, policy_version 1719521 (0.0008) [2023-12-27 17:34:07,291][20381] Updated weights for policy 1, policy_version 1719531 (0.0007) [2023-12-27 17:34:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 880893952. Throughput: 0: 10315.8, 1: 10168.9. Samples: 880859864. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:07,364][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 17:34:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001719536_440246272.pth... [2023-12-27 17:34:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001720968_440647680.pth... [2023-12-27 17:34:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001718352_439943168.pth [2023-12-27 17:34:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001719784_440344576.pth [2023-12-27 17:34:07,517][20347] Updated weights for policy 0, policy_version 1720972 (0.0009) [2023-12-27 17:34:07,573][20347] Updated weights for policy 0, policy_version 1720982 (0.0008) [2023-12-27 17:34:07,620][20347] Updated weights for policy 0, policy_version 1720992 (0.0010) [2023-12-27 17:34:07,999][20381] Updated weights for policy 1, policy_version 1719541 (0.0008) [2023-12-27 17:34:08,049][20381] Updated weights for policy 1, policy_version 1719551 (0.0006) [2023-12-27 17:34:08,096][20381] Updated weights for policy 1, policy_version 1719561 (0.0007) [2023-12-27 17:34:08,237][20347] Updated weights for policy 0, policy_version 1721002 (0.0010) [2023-12-27 17:34:08,295][20347] Updated weights for policy 0, policy_version 1721012 (0.0010) [2023-12-27 17:34:08,361][20347] Updated weights for policy 0, policy_version 1721022 (0.0010) [2023-12-27 17:34:08,413][20347] Updated weights for policy 0, policy_version 1721032 (0.0010) [2023-12-27 17:34:08,762][20381] Updated weights for policy 1, policy_version 1719571 (0.0006) [2023-12-27 17:34:08,818][20381] Updated weights for policy 1, policy_version 1719581 (0.0008) [2023-12-27 17:34:08,874][20381] Updated weights for policy 1, policy_version 1719591 (0.0008) [2023-12-27 17:34:09,059][20347] Updated weights for policy 0, policy_version 1721042 (0.0005) [2023-12-27 17:34:09,116][20347] Updated weights for policy 0, policy_version 1721052 (0.0010) [2023-12-27 17:34:09,168][20347] Updated weights for policy 0, policy_version 1721062 (0.0010) [2023-12-27 17:34:09,599][20381] Updated weights for policy 1, policy_version 1719601 (0.0010) [2023-12-27 17:34:09,662][20381] Updated weights for policy 1, policy_version 1719611 (0.0009) [2023-12-27 17:34:09,720][20381] Updated weights for policy 1, policy_version 1719621 (0.0011) [2023-12-27 17:34:09,783][20381] Updated weights for policy 1, policy_version 1719631 (0.0011) [2023-12-27 17:34:09,890][20347] Updated weights for policy 0, policy_version 1721072 (0.0008) [2023-12-27 17:34:09,955][20347] Updated weights for policy 0, policy_version 1721082 (0.0008) [2023-12-27 17:34:10,017][20347] Updated weights for policy 0, policy_version 1721092 (0.0007) [2023-12-27 17:34:10,489][20381] Updated weights for policy 1, policy_version 1719641 (0.0007) [2023-12-27 17:34:10,552][20381] Updated weights for policy 1, policy_version 1719651 (0.0011) [2023-12-27 17:34:10,614][20381] Updated weights for policy 1, policy_version 1719661 (0.0010) [2023-12-27 17:34:10,641][20347] Updated weights for policy 0, policy_version 1721102 (0.0007) [2023-12-27 17:34:10,691][20347] Updated weights for policy 0, policy_version 1721112 (0.0008) [2023-12-27 17:34:10,744][20347] Updated weights for policy 0, policy_version 1721122 (0.0007) [2023-12-27 17:34:11,274][20381] Updated weights for policy 1, policy_version 1719671 (0.0007) [2023-12-27 17:34:11,334][20381] Updated weights for policy 1, policy_version 1719681 (0.0008) [2023-12-27 17:34:11,400][20381] Updated weights for policy 1, policy_version 1719691 (0.0008) [2023-12-27 17:34:11,505][20347] Updated weights for policy 0, policy_version 1721132 (0.0007) [2023-12-27 17:34:11,576][20347] Updated weights for policy 0, policy_version 1721142 (0.0006) [2023-12-27 17:34:11,645][20347] Updated weights for policy 0, policy_version 1721152 (0.0007) [2023-12-27 17:34:12,207][20381] Updated weights for policy 1, policy_version 1719701 (0.0007) [2023-12-27 17:34:12,267][20381] Updated weights for policy 1, policy_version 1719711 (0.0008) [2023-12-27 17:34:12,293][20347] Updated weights for policy 0, policy_version 1721162 (0.0008) [2023-12-27 17:34:12,325][20381] Updated weights for policy 1, policy_version 1719721 (0.0007) [2023-12-27 17:34:12,358][20347] Updated weights for policy 0, policy_version 1721172 (0.0008) [2023-12-27 17:34:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 880984064. Throughput: 0: 10374.7, 1: 10154.3. Samples: 880982944. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:12,363][19170] Avg episode reward: [(0, '941.360'), (1, '1000.000')] [2023-12-27 17:34:12,421][20347] Updated weights for policy 0, policy_version 1721182 (0.0008) [2023-12-27 17:34:12,472][20347] Updated weights for policy 0, policy_version 1721192 (0.0008) [2023-12-27 17:34:12,908][20381] Updated weights for policy 1, policy_version 1719731 (0.0007) [2023-12-27 17:34:12,969][20381] Updated weights for policy 1, policy_version 1719741 (0.0005) [2023-12-27 17:34:13,035][20381] Updated weights for policy 1, policy_version 1719751 (0.0006) [2023-12-27 17:34:13,234][20347] Updated weights for policy 0, policy_version 1721202 (0.0011) [2023-12-27 17:34:13,285][20347] Updated weights for policy 0, policy_version 1721212 (0.0010) [2023-12-27 17:34:13,342][20347] Updated weights for policy 0, policy_version 1721222 (0.0010) [2023-12-27 17:34:13,614][20381] Updated weights for policy 1, policy_version 1719761 (0.0008) [2023-12-27 17:34:13,667][20381] Updated weights for policy 1, policy_version 1719771 (0.0005) [2023-12-27 17:34:13,713][20381] Updated weights for policy 1, policy_version 1719781 (0.0005) [2023-12-27 17:34:13,764][20381] Updated weights for policy 1, policy_version 1719791 (0.0007) [2023-12-27 17:34:14,023][20347] Updated weights for policy 0, policy_version 1721232 (0.0007) [2023-12-27 17:34:14,080][20347] Updated weights for policy 0, policy_version 1721242 (0.0005) [2023-12-27 17:34:14,132][20347] Updated weights for policy 0, policy_version 1721252 (0.0005) [2023-12-27 17:34:14,339][20381] Updated weights for policy 1, policy_version 1719801 (0.0005) [2023-12-27 17:34:14,390][20381] Updated weights for policy 1, policy_version 1719811 (0.0005) [2023-12-27 17:34:14,446][20381] Updated weights for policy 1, policy_version 1719821 (0.0009) [2023-12-27 17:34:14,726][20347] Updated weights for policy 0, policy_version 1721262 (0.0005) [2023-12-27 17:34:14,803][20347] Updated weights for policy 0, policy_version 1721272 (0.0007) [2023-12-27 17:34:14,856][20347] Updated weights for policy 0, policy_version 1721282 (0.0011) [2023-12-27 17:34:15,055][20381] Updated weights for policy 1, policy_version 1719831 (0.0009) [2023-12-27 17:34:15,116][20381] Updated weights for policy 1, policy_version 1719841 (0.0011) [2023-12-27 17:34:15,175][20381] Updated weights for policy 1, policy_version 1719851 (0.0010) [2023-12-27 17:34:15,497][20347] Updated weights for policy 0, policy_version 1721292 (0.0009) [2023-12-27 17:34:15,542][20347] Updated weights for policy 0, policy_version 1721302 (0.0007) [2023-12-27 17:34:15,593][20347] Updated weights for policy 0, policy_version 1721312 (0.0007) [2023-12-27 17:34:15,850][20381] Updated weights for policy 1, policy_version 1719861 (0.0010) [2023-12-27 17:34:15,902][20381] Updated weights for policy 1, policy_version 1719871 (0.0007) [2023-12-27 17:34:15,956][20381] Updated weights for policy 1, policy_version 1719881 (0.0007) [2023-12-27 17:34:16,299][20347] Updated weights for policy 0, policy_version 1721322 (0.0007) [2023-12-27 17:34:16,356][20347] Updated weights for policy 0, policy_version 1721332 (0.0009) [2023-12-27 17:34:16,401][20347] Updated weights for policy 0, policy_version 1721342 (0.0010) [2023-12-27 17:34:16,456][20347] Updated weights for policy 0, policy_version 1721352 (0.0010) [2023-12-27 17:34:16,614][20381] Updated weights for policy 1, policy_version 1719891 (0.0008) [2023-12-27 17:34:16,659][20381] Updated weights for policy 1, policy_version 1719901 (0.0007) [2023-12-27 17:34:16,710][20381] Updated weights for policy 1, policy_version 1719911 (0.0007) [2023-12-27 17:34:17,133][20347] Updated weights for policy 0, policy_version 1721362 (0.0011) [2023-12-27 17:34:17,179][20347] Updated weights for policy 0, policy_version 1721372 (0.0010) [2023-12-27 17:34:17,224][20347] Updated weights for policy 0, policy_version 1721382 (0.0010) [2023-12-27 17:34:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 881098752. Throughput: 0: 10304.1, 1: 10162.1. Samples: 881108380. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:17,363][19170] Avg episode reward: [(0, '931.540'), (1, '990.290')] [2023-12-27 17:34:17,402][20381] Updated weights for policy 1, policy_version 1719921 (0.0007) [2023-12-27 17:34:17,456][20381] Updated weights for policy 1, policy_version 1719931 (0.0007) [2023-12-27 17:34:17,509][20381] Updated weights for policy 1, policy_version 1719941 (0.0007) [2023-12-27 17:34:17,559][20381] Updated weights for policy 1, policy_version 1719951 (0.0007) [2023-12-27 17:34:17,913][20347] Updated weights for policy 0, policy_version 1721392 (0.0006) [2023-12-27 17:34:17,961][20347] Updated weights for policy 0, policy_version 1721402 (0.0006) [2023-12-27 17:34:18,010][20347] Updated weights for policy 0, policy_version 1721412 (0.0008) [2023-12-27 17:34:18,274][20381] Updated weights for policy 1, policy_version 1719961 (0.0007) [2023-12-27 17:34:18,329][20381] Updated weights for policy 1, policy_version 1719971 (0.0008) [2023-12-27 17:34:18,386][20381] Updated weights for policy 1, policy_version 1719981 (0.0009) [2023-12-27 17:34:18,659][20347] Updated weights for policy 0, policy_version 1721422 (0.0007) [2023-12-27 17:34:18,711][20347] Updated weights for policy 0, policy_version 1721432 (0.0007) [2023-12-27 17:34:18,767][20347] Updated weights for policy 0, policy_version 1721442 (0.0008) [2023-12-27 17:34:19,034][20381] Updated weights for policy 1, policy_version 1719991 (0.0007) [2023-12-27 17:34:19,080][20381] Updated weights for policy 1, policy_version 1720001 (0.0008) [2023-12-27 17:34:19,137][20381] Updated weights for policy 1, policy_version 1720011 (0.0008) [2023-12-27 17:34:19,457][20347] Updated weights for policy 0, policy_version 1721452 (0.0007) [2023-12-27 17:34:19,525][20347] Updated weights for policy 0, policy_version 1721462 (0.0007) [2023-12-27 17:34:19,592][20347] Updated weights for policy 0, policy_version 1721472 (0.0005) [2023-12-27 17:34:19,896][20381] Updated weights for policy 1, policy_version 1720021 (0.0007) [2023-12-27 17:34:19,960][20381] Updated weights for policy 1, policy_version 1720031 (0.0007) [2023-12-27 17:34:20,023][20381] Updated weights for policy 1, policy_version 1720041 (0.0007) [2023-12-27 17:34:20,156][20347] Updated weights for policy 0, policy_version 1721482 (0.0008) [2023-12-27 17:34:20,214][20347] Updated weights for policy 0, policy_version 1721492 (0.0007) [2023-12-27 17:34:20,282][20347] Updated weights for policy 0, policy_version 1721502 (0.0008) [2023-12-27 17:34:20,341][20347] Updated weights for policy 0, policy_version 1721512 (0.0007) [2023-12-27 17:34:20,731][20381] Updated weights for policy 1, policy_version 1720051 (0.0009) [2023-12-27 17:34:20,794][20381] Updated weights for policy 1, policy_version 1720061 (0.0008) [2023-12-27 17:34:20,856][20381] Updated weights for policy 1, policy_version 1720071 (0.0008) [2023-12-27 17:34:21,042][20347] Updated weights for policy 0, policy_version 1721522 (0.0008) [2023-12-27 17:34:21,107][20347] Updated weights for policy 0, policy_version 1721532 (0.0009) [2023-12-27 17:34:21,180][20347] Updated weights for policy 0, policy_version 1721542 (0.0007) [2023-12-27 17:34:21,593][20381] Updated weights for policy 1, policy_version 1720081 (0.0007) [2023-12-27 17:34:21,668][20381] Updated weights for policy 1, policy_version 1720091 (0.0009) [2023-12-27 17:34:21,733][20381] Updated weights for policy 1, policy_version 1720101 (0.0008) [2023-12-27 17:34:21,790][20381] Updated weights for policy 1, policy_version 1720111 (0.0008) [2023-12-27 17:34:21,886][20347] Updated weights for policy 0, policy_version 1721552 (0.0006) [2023-12-27 17:34:21,938][20347] Updated weights for policy 0, policy_version 1721562 (0.0006) [2023-12-27 17:34:21,989][20347] Updated weights for policy 0, policy_version 1721572 (0.0005) [2023-12-27 17:34:22,009][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 17:34:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 881197056. Throughput: 0: 10291.9, 1: 10152.9. Samples: 881169512. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:22,363][19170] Avg episode reward: [(0, '970.650'), (1, '990.290')] [2023-12-27 17:34:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001720112_440393728.pth... [2023-12-27 17:34:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001721576_440803328.pth... [2023-12-27 17:34:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001718928_440090624.pth [2023-12-27 17:34:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001720360_440492032.pth [2023-12-27 17:34:22,484][20381] Updated weights for policy 1, policy_version 1720121 (0.0008) [2023-12-27 17:34:22,549][20381] Updated weights for policy 1, policy_version 1720131 (0.0008) [2023-12-27 17:34:22,609][20381] Updated weights for policy 1, policy_version 1720141 (0.0008) [2023-12-27 17:34:22,669][20347] Updated weights for policy 0, policy_version 1721582 (0.0007) [2023-12-27 17:34:22,736][20347] Updated weights for policy 0, policy_version 1721592 (0.0007) [2023-12-27 17:34:22,794][20347] Updated weights for policy 0, policy_version 1721602 (0.0007) [2023-12-27 17:34:23,313][20381] Updated weights for policy 1, policy_version 1720151 (0.0009) [2023-12-27 17:34:23,369][20381] Updated weights for policy 1, policy_version 1720161 (0.0007) [2023-12-27 17:34:23,425][20381] Updated weights for policy 1, policy_version 1720171 (0.0007) [2023-12-27 17:34:23,486][20347] Updated weights for policy 0, policy_version 1721612 (0.0007) [2023-12-27 17:34:23,546][20347] Updated weights for policy 0, policy_version 1721622 (0.0007) [2023-12-27 17:34:23,600][20347] Updated weights for policy 0, policy_version 1721632 (0.0007) [2023-12-27 17:34:24,029][20381] Updated weights for policy 1, policy_version 1720181 (0.0008) [2023-12-27 17:34:24,082][20381] Updated weights for policy 1, policy_version 1720191 (0.0007) [2023-12-27 17:34:24,140][20381] Updated weights for policy 1, policy_version 1720201 (0.0010) [2023-12-27 17:34:24,174][20347] Updated weights for policy 0, policy_version 1721642 (0.0007) [2023-12-27 17:34:24,227][20347] Updated weights for policy 0, policy_version 1721652 (0.0006) [2023-12-27 17:34:24,274][20347] Updated weights for policy 0, policy_version 1721662 (0.0005) [2023-12-27 17:34:24,322][20347] Updated weights for policy 0, policy_version 1721672 (0.0005) [2023-12-27 17:34:24,783][20381] Updated weights for policy 1, policy_version 1720211 (0.0010) [2023-12-27 17:34:24,834][20381] Updated weights for policy 1, policy_version 1720221 (0.0008) [2023-12-27 17:34:24,884][20381] Updated weights for policy 1, policy_version 1720231 (0.0008) [2023-12-27 17:34:24,986][20347] Updated weights for policy 0, policy_version 1721682 (0.0005) [2023-12-27 17:34:25,040][20347] Updated weights for policy 0, policy_version 1721692 (0.0007) [2023-12-27 17:34:25,098][20347] Updated weights for policy 0, policy_version 1721702 (0.0007) [2023-12-27 17:34:25,529][20381] Updated weights for policy 1, policy_version 1720241 (0.0007) [2023-12-27 17:34:25,587][20381] Updated weights for policy 1, policy_version 1720251 (0.0005) [2023-12-27 17:34:25,649][20381] Updated weights for policy 1, policy_version 1720261 (0.0006) [2023-12-27 17:34:25,702][20381] Updated weights for policy 1, policy_version 1720271 (0.0008) [2023-12-27 17:34:25,764][20347] Updated weights for policy 0, policy_version 1721712 (0.0006) [2023-12-27 17:34:25,824][20347] Updated weights for policy 0, policy_version 1721722 (0.0005) [2023-12-27 17:34:25,882][20347] Updated weights for policy 0, policy_version 1721732 (0.0005) [2023-12-27 17:34:26,289][20381] Updated weights for policy 1, policy_version 1720281 (0.0005) [2023-12-27 17:34:26,342][20381] Updated weights for policy 1, policy_version 1720291 (0.0010) [2023-12-27 17:34:26,393][20381] Updated weights for policy 1, policy_version 1720301 (0.0007) [2023-12-27 17:34:26,478][20347] Updated weights for policy 0, policy_version 1721742 (0.0007) [2023-12-27 17:34:26,528][20347] Updated weights for policy 0, policy_version 1721752 (0.0007) [2023-12-27 17:34:26,573][20347] Updated weights for policy 0, policy_version 1721762 (0.0005) [2023-12-27 17:34:27,026][20381] Updated weights for policy 1, policy_version 1720311 (0.0007) [2023-12-27 17:34:27,084][20381] Updated weights for policy 1, policy_version 1720321 (0.0005) [2023-12-27 17:34:27,127][20347] Updated weights for policy 0, policy_version 1721772 (0.0005) [2023-12-27 17:34:27,135][20381] Updated weights for policy 1, policy_version 1720331 (0.0005) [2023-12-27 17:34:27,180][20347] Updated weights for policy 0, policy_version 1721782 (0.0005) [2023-12-27 17:34:27,240][20347] Updated weights for policy 0, policy_version 1721792 (0.0005) [2023-12-27 17:34:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 881311744. Throughput: 0: 10249.1, 1: 10230.9. Samples: 881293860. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:27,363][19170] Avg episode reward: [(0, '970.660'), (1, '1000.000')] [2023-12-27 17:34:27,789][20381] Updated weights for policy 1, policy_version 1720341 (0.0007) [2023-12-27 17:34:27,807][20347] Updated weights for policy 0, policy_version 1721802 (0.0005) [2023-12-27 17:34:27,838][20381] Updated weights for policy 1, policy_version 1720351 (0.0007) [2023-12-27 17:34:27,851][20347] Updated weights for policy 0, policy_version 1721812 (0.0005) [2023-12-27 17:34:27,882][20381] Updated weights for policy 1, policy_version 1720361 (0.0007) [2023-12-27 17:34:27,897][20347] Updated weights for policy 0, policy_version 1721822 (0.0005) [2023-12-27 17:34:27,945][20347] Updated weights for policy 0, policy_version 1721832 (0.0007) [2023-12-27 17:34:28,623][20381] Updated weights for policy 1, policy_version 1720371 (0.0007) [2023-12-27 17:34:28,667][20347] Updated weights for policy 0, policy_version 1721842 (0.0005) [2023-12-27 17:34:28,680][20381] Updated weights for policy 1, policy_version 1720381 (0.0007) [2023-12-27 17:34:28,727][20347] Updated weights for policy 0, policy_version 1721852 (0.0006) [2023-12-27 17:34:28,732][20381] Updated weights for policy 1, policy_version 1720391 (0.0008) [2023-12-27 17:34:28,794][20347] Updated weights for policy 0, policy_version 1721862 (0.0008) [2023-12-27 17:34:29,371][20347] Updated weights for policy 0, policy_version 1721872 (0.0007) [2023-12-27 17:34:29,425][20347] Updated weights for policy 0, policy_version 1721882 (0.0008) [2023-12-27 17:34:29,484][20347] Updated weights for policy 0, policy_version 1721892 (0.0007) [2023-12-27 17:34:29,486][20381] Updated weights for policy 1, policy_version 1720401 (0.0007) [2023-12-27 17:34:29,549][20381] Updated weights for policy 1, policy_version 1720411 (0.0007) [2023-12-27 17:34:29,609][20381] Updated weights for policy 1, policy_version 1720421 (0.0007) [2023-12-27 17:34:29,666][20381] Updated weights for policy 1, policy_version 1720431 (0.0006) [2023-12-27 17:34:30,156][20347] Updated weights for policy 0, policy_version 1721902 (0.0007) [2023-12-27 17:34:30,205][20347] Updated weights for policy 0, policy_version 1721912 (0.0008) [2023-12-27 17:34:30,262][20347] Updated weights for policy 0, policy_version 1721922 (0.0007) [2023-12-27 17:34:30,348][20381] Updated weights for policy 1, policy_version 1720441 (0.0008) [2023-12-27 17:34:30,406][20381] Updated weights for policy 1, policy_version 1720451 (0.0008) [2023-12-27 17:34:30,458][20381] Updated weights for policy 1, policy_version 1720461 (0.0008) [2023-12-27 17:34:30,992][20347] Updated weights for policy 0, policy_version 1721932 (0.0008) [2023-12-27 17:34:31,054][20347] Updated weights for policy 0, policy_version 1721942 (0.0008) [2023-12-27 17:34:31,105][20381] Updated weights for policy 1, policy_version 1720471 (0.0010) [2023-12-27 17:34:31,110][20347] Updated weights for policy 0, policy_version 1721952 (0.0006) [2023-12-27 17:34:31,167][20381] Updated weights for policy 1, policy_version 1720481 (0.0011) [2023-12-27 17:34:31,232][20381] Updated weights for policy 1, policy_version 1720491 (0.0011) [2023-12-27 17:34:31,829][20347] Updated weights for policy 0, policy_version 1721962 (0.0007) [2023-12-27 17:34:31,883][20347] Updated weights for policy 0, policy_version 1721972 (0.0008) [2023-12-27 17:34:31,940][20381] Updated weights for policy 1, policy_version 1720501 (0.0009) [2023-12-27 17:34:31,941][20347] Updated weights for policy 0, policy_version 1721982 (0.0008) [2023-12-27 17:34:31,985][20381] Updated weights for policy 1, policy_version 1720511 (0.0006) [2023-12-27 17:34:31,995][20347] Updated weights for policy 0, policy_version 1721992 (0.0008) [2023-12-27 17:34:32,033][20381] Updated weights for policy 1, policy_version 1720521 (0.0007) [2023-12-27 17:34:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 881410048. Throughput: 0: 10338.3, 1: 10199.6. Samples: 881417932. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-12-27 17:34:32,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 17:34:32,692][20347] Updated weights for policy 0, policy_version 1722002 (0.0005) [2023-12-27 17:34:32,757][20347] Updated weights for policy 0, policy_version 1722012 (0.0005) [2023-12-27 17:34:32,776][20381] Updated weights for policy 1, policy_version 1720531 (0.0007) [2023-12-27 17:34:32,811][20347] Updated weights for policy 0, policy_version 1722022 (0.0008) [2023-12-27 17:34:32,823][20381] Updated weights for policy 1, policy_version 1720541 (0.0007) [2023-12-27 17:34:32,875][20381] Updated weights for policy 1, policy_version 1720551 (0.0007) [2023-12-27 17:34:33,399][20347] Updated weights for policy 0, policy_version 1722032 (0.0007) [2023-12-27 17:34:33,447][20347] Updated weights for policy 0, policy_version 1722042 (0.0007) [2023-12-27 17:34:33,504][20347] Updated weights for policy 0, policy_version 1722052 (0.0008) [2023-12-27 17:34:33,607][20381] Updated weights for policy 1, policy_version 1720561 (0.0007) [2023-12-27 17:34:33,659][20381] Updated weights for policy 1, policy_version 1720571 (0.0009) [2023-12-27 17:34:33,709][20381] Updated weights for policy 1, policy_version 1720581 (0.0010) [2023-12-27 17:34:33,760][20381] Updated weights for policy 1, policy_version 1720591 (0.0007) [2023-12-27 17:34:34,060][20347] Updated weights for policy 0, policy_version 1722062 (0.0006) [2023-12-27 17:34:34,117][20347] Updated weights for policy 0, policy_version 1722072 (0.0005) [2023-12-27 17:34:34,176][20347] Updated weights for policy 0, policy_version 1722082 (0.0007) [2023-12-27 17:34:34,481][20381] Updated weights for policy 1, policy_version 1720601 (0.0006) [2023-12-27 17:34:34,539][20381] Updated weights for policy 1, policy_version 1720611 (0.0006) [2023-12-27 17:34:34,602][20381] Updated weights for policy 1, policy_version 1720621 (0.0007) [2023-12-27 17:34:34,828][20347] Updated weights for policy 0, policy_version 1722092 (0.0007) [2023-12-27 17:34:34,875][20347] Updated weights for policy 0, policy_version 1722102 (0.0008) [2023-12-27 17:34:34,922][20347] Updated weights for policy 0, policy_version 1722112 (0.0007) [2023-12-27 17:34:35,243][20381] Updated weights for policy 1, policy_version 1720631 (0.0009) [2023-12-27 17:34:35,300][20381] Updated weights for policy 1, policy_version 1720641 (0.0010) [2023-12-27 17:34:35,358][20381] Updated weights for policy 1, policy_version 1720651 (0.0010) [2023-12-27 17:34:35,615][20347] Updated weights for policy 0, policy_version 1722122 (0.0007) [2023-12-27 17:34:35,668][20347] Updated weights for policy 0, policy_version 1722132 (0.0005) [2023-12-27 17:34:35,726][20347] Updated weights for policy 0, policy_version 1722142 (0.0005) [2023-12-27 17:34:35,779][20347] Updated weights for policy 0, policy_version 1722152 (0.0005) [2023-12-27 17:34:36,016][20381] Updated weights for policy 1, policy_version 1720661 (0.0009) [2023-12-27 17:34:36,070][20381] Updated weights for policy 1, policy_version 1720671 (0.0006) [2023-12-27 17:34:36,130][20381] Updated weights for policy 1, policy_version 1720681 (0.0007) [2023-12-27 17:34:36,378][20347] Updated weights for policy 0, policy_version 1722162 (0.0006) [2023-12-27 17:34:36,435][20347] Updated weights for policy 0, policy_version 1722172 (0.0005) [2023-12-27 17:34:36,495][20347] Updated weights for policy 0, policy_version 1722182 (0.0006) [2023-12-27 17:34:36,797][20381] Updated weights for policy 1, policy_version 1720691 (0.0007) [2023-12-27 17:34:36,859][20381] Updated weights for policy 1, policy_version 1720701 (0.0005) [2023-12-27 17:34:36,919][20381] Updated weights for policy 1, policy_version 1720711 (0.0005) [2023-12-27 17:34:37,187][20347] Updated weights for policy 0, policy_version 1722192 (0.0008) [2023-12-27 17:34:37,236][20347] Updated weights for policy 0, policy_version 1722202 (0.0008) [2023-12-27 17:34:37,281][20347] Updated weights for policy 0, policy_version 1722212 (0.0007) [2023-12-27 17:34:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.7, 300 sec: 20466.1). Total num frames: 881516544. Throughput: 0: 10389.4, 1: 10162.9. Samples: 881480688. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:34:37,363][19170] Avg episode reward: [(0, '970.670'), (1, '1000.000')] [2023-12-27 17:34:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001720720_440549376.pth... [2023-12-27 17:34:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001722216_440967168.pth... [2023-12-27 17:34:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001719536_440246272.pth [2023-12-27 17:34:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001720968_440647680.pth [2023-12-27 17:34:37,478][20381] Updated weights for policy 1, policy_version 1720721 (0.0006) [2023-12-27 17:34:37,529][20381] Updated weights for policy 1, policy_version 1720731 (0.0010) [2023-12-27 17:34:37,586][20381] Updated weights for policy 1, policy_version 1720741 (0.0009) [2023-12-27 17:34:37,642][20381] Updated weights for policy 1, policy_version 1720751 (0.0010) [2023-12-27 17:34:37,974][20347] Updated weights for policy 0, policy_version 1722222 (0.0008) [2023-12-27 17:34:38,033][20347] Updated weights for policy 0, policy_version 1722232 (0.0008) [2023-12-27 17:34:38,080][20347] Updated weights for policy 0, policy_version 1722242 (0.0007) [2023-12-27 17:34:38,308][20381] Updated weights for policy 1, policy_version 1720761 (0.0010) [2023-12-27 17:34:38,373][20381] Updated weights for policy 1, policy_version 1720771 (0.0011) [2023-12-27 17:34:38,424][20381] Updated weights for policy 1, policy_version 1720781 (0.0011) [2023-12-27 17:34:38,772][20347] Updated weights for policy 0, policy_version 1722252 (0.0008) [2023-12-27 17:34:38,833][20347] Updated weights for policy 0, policy_version 1722262 (0.0008) [2023-12-27 17:34:38,890][20347] Updated weights for policy 0, policy_version 1722272 (0.0006) [2023-12-27 17:34:39,137][20381] Updated weights for policy 1, policy_version 1720791 (0.0007) [2023-12-27 17:34:39,194][20381] Updated weights for policy 1, policy_version 1720801 (0.0005) [2023-12-27 17:34:39,258][20381] Updated weights for policy 1, policy_version 1720811 (0.0011) [2023-12-27 17:34:39,610][20347] Updated weights for policy 0, policy_version 1722282 (0.0008) [2023-12-27 17:34:39,671][20347] Updated weights for policy 0, policy_version 1722292 (0.0008) [2023-12-27 17:34:39,729][20347] Updated weights for policy 0, policy_version 1722302 (0.0010) [2023-12-27 17:34:39,787][20347] Updated weights for policy 0, policy_version 1722312 (0.0010) [2023-12-27 17:34:39,976][20381] Updated weights for policy 1, policy_version 1720821 (0.0009) [2023-12-27 17:34:40,040][20381] Updated weights for policy 1, policy_version 1720831 (0.0011) [2023-12-27 17:34:40,104][20381] Updated weights for policy 1, policy_version 1720841 (0.0010) [2023-12-27 17:34:40,470][20347] Updated weights for policy 0, policy_version 1722322 (0.0008) [2023-12-27 17:34:40,519][20347] Updated weights for policy 0, policy_version 1722332 (0.0007) [2023-12-27 17:34:40,572][20347] Updated weights for policy 0, policy_version 1722342 (0.0008) [2023-12-27 17:34:40,811][20381] Updated weights for policy 1, policy_version 1720851 (0.0010) [2023-12-27 17:34:40,866][20381] Updated weights for policy 1, policy_version 1720861 (0.0007) [2023-12-27 17:34:40,915][20381] Updated weights for policy 1, policy_version 1720871 (0.0009) [2023-12-27 17:34:41,288][20347] Updated weights for policy 0, policy_version 1722352 (0.0008) [2023-12-27 17:34:41,349][20347] Updated weights for policy 0, policy_version 1722362 (0.0008) [2023-12-27 17:34:41,420][20347] Updated weights for policy 0, policy_version 1722372 (0.0008) [2023-12-27 17:34:41,716][20381] Updated weights for policy 1, policy_version 1720881 (0.0008) [2023-12-27 17:34:41,791][20381] Updated weights for policy 1, policy_version 1720891 (0.0008) [2023-12-27 17:34:41,839][20381] Updated weights for policy 1, policy_version 1720901 (0.0008) [2023-12-27 17:34:41,898][20381] Updated weights for policy 1, policy_version 1720911 (0.0008) [2023-12-27 17:34:42,029][20347] Updated weights for policy 0, policy_version 1722382 (0.0006) [2023-12-27 17:34:42,086][20347] Updated weights for policy 0, policy_version 1722392 (0.0007) [2023-12-27 17:34:42,143][20347] Updated weights for policy 0, policy_version 1722402 (0.0008) [2023-12-27 17:34:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 881614848. Throughput: 0: 10395.4, 1: 10201.3. Samples: 881603488. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:34:42,363][19170] Avg episode reward: [(0, '960.850'), (1, '1000.000')] [2023-12-27 17:34:42,634][20381] Updated weights for policy 1, policy_version 1720921 (0.0008) [2023-12-27 17:34:42,691][20381] Updated weights for policy 1, policy_version 1720931 (0.0008) [2023-12-27 17:34:42,744][20381] Updated weights for policy 1, policy_version 1720941 (0.0008) [2023-12-27 17:34:42,855][20347] Updated weights for policy 0, policy_version 1722412 (0.0007) [2023-12-27 17:34:42,904][20347] Updated weights for policy 0, policy_version 1722422 (0.0005) [2023-12-27 17:34:42,958][20347] Updated weights for policy 0, policy_version 1722432 (0.0005) [2023-12-27 17:34:43,503][20381] Updated weights for policy 1, policy_version 1720951 (0.0007) [2023-12-27 17:34:43,533][20347] Updated weights for policy 0, policy_version 1722442 (0.0006) [2023-12-27 17:34:43,567][20381] Updated weights for policy 1, policy_version 1720961 (0.0007) [2023-12-27 17:34:43,592][20347] Updated weights for policy 0, policy_version 1722452 (0.0006) [2023-12-27 17:34:43,632][20381] Updated weights for policy 1, policy_version 1720971 (0.0007) [2023-12-27 17:34:43,652][20347] Updated weights for policy 0, policy_version 1722462 (0.0010) [2023-12-27 17:34:43,709][20347] Updated weights for policy 0, policy_version 1722472 (0.0008) [2023-12-27 17:34:44,294][20381] Updated weights for policy 1, policy_version 1720981 (0.0011) [2023-12-27 17:34:44,345][20381] Updated weights for policy 1, policy_version 1720991 (0.0010) [2023-12-27 17:34:44,382][20347] Updated weights for policy 0, policy_version 1722482 (0.0007) [2023-12-27 17:34:44,397][20381] Updated weights for policy 1, policy_version 1721001 (0.0010) [2023-12-27 17:34:44,439][20347] Updated weights for policy 0, policy_version 1722492 (0.0005) [2023-12-27 17:34:44,497][20347] Updated weights for policy 0, policy_version 1722502 (0.0007) [2023-12-27 17:34:45,069][20381] Updated weights for policy 1, policy_version 1721011 (0.0011) [2023-12-27 17:34:45,123][20381] Updated weights for policy 1, policy_version 1721021 (0.0011) [2023-12-27 17:34:45,184][20381] Updated weights for policy 1, policy_version 1721031 (0.0011) [2023-12-27 17:34:45,201][20347] Updated weights for policy 0, policy_version 1722512 (0.0007) [2023-12-27 17:34:45,259][20347] Updated weights for policy 0, policy_version 1722522 (0.0006) [2023-12-27 17:34:45,321][20347] Updated weights for policy 0, policy_version 1722532 (0.0006) [2023-12-27 17:34:45,870][20381] Updated weights for policy 1, policy_version 1721041 (0.0011) [2023-12-27 17:34:45,892][20347] Updated weights for policy 0, policy_version 1722542 (0.0008) [2023-12-27 17:34:45,923][20381] Updated weights for policy 1, policy_version 1721051 (0.0008) [2023-12-27 17:34:45,939][20347] Updated weights for policy 0, policy_version 1722552 (0.0007) [2023-12-27 17:34:45,968][20381] Updated weights for policy 1, policy_version 1721061 (0.0010) [2023-12-27 17:34:45,985][20347] Updated weights for policy 0, policy_version 1722562 (0.0005) [2023-12-27 17:34:46,017][20381] Updated weights for policy 1, policy_version 1721071 (0.0010) [2023-12-27 17:34:46,623][20347] Updated weights for policy 0, policy_version 1722572 (0.0007) [2023-12-27 17:34:46,658][20381] Updated weights for policy 1, policy_version 1721081 (0.0010) [2023-12-27 17:34:46,675][20347] Updated weights for policy 0, policy_version 1722582 (0.0010) [2023-12-27 17:34:46,713][20381] Updated weights for policy 1, policy_version 1721091 (0.0010) [2023-12-27 17:34:46,724][20347] Updated weights for policy 0, policy_version 1722592 (0.0009) [2023-12-27 17:34:46,769][20381] Updated weights for policy 1, policy_version 1721101 (0.0011) [2023-12-27 17:34:47,340][20347] Updated weights for policy 0, policy_version 1722602 (0.0005) [2023-12-27 17:34:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 881713152. Throughput: 0: 10456.5, 1: 10158.1. Samples: 881727028. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:34:47,363][19170] Avg episode reward: [(0, '980.430'), (1, '1000.000')] [2023-12-27 17:34:47,400][20347] Updated weights for policy 0, policy_version 1722612 (0.0005) [2023-12-27 17:34:47,451][20347] Updated weights for policy 0, policy_version 1722622 (0.0005) [2023-12-27 17:34:47,473][20381] Updated weights for policy 1, policy_version 1721111 (0.0008) [2023-12-27 17:34:47,505][20347] Updated weights for policy 0, policy_version 1722632 (0.0010) [2023-12-27 17:34:47,526][20381] Updated weights for policy 1, policy_version 1721121 (0.0006) [2023-12-27 17:34:47,586][20381] Updated weights for policy 1, policy_version 1721131 (0.0009) [2023-12-27 17:34:48,044][20347] Updated weights for policy 0, policy_version 1722642 (0.0010) [2023-12-27 17:34:48,105][20347] Updated weights for policy 0, policy_version 1722652 (0.0010) [2023-12-27 17:34:48,162][20347] Updated weights for policy 0, policy_version 1722662 (0.0010) [2023-12-27 17:34:48,227][20381] Updated weights for policy 1, policy_version 1721141 (0.0011) [2023-12-27 17:34:48,282][20381] Updated weights for policy 1, policy_version 1721151 (0.0010) [2023-12-27 17:34:48,348][20381] Updated weights for policy 1, policy_version 1721161 (0.0010) [2023-12-27 17:34:48,833][20347] Updated weights for policy 0, policy_version 1722672 (0.0007) [2023-12-27 17:34:48,901][20347] Updated weights for policy 0, policy_version 1722682 (0.0007) [2023-12-27 17:34:48,971][20381] Updated weights for policy 1, policy_version 1721171 (0.0008) [2023-12-27 17:34:48,972][20347] Updated weights for policy 0, policy_version 1722692 (0.0009) [2023-12-27 17:34:49,028][20381] Updated weights for policy 1, policy_version 1721181 (0.0005) [2023-12-27 17:34:49,083][20381] Updated weights for policy 1, policy_version 1721191 (0.0006) [2023-12-27 17:34:49,668][20347] Updated weights for policy 0, policy_version 1722702 (0.0009) [2023-12-27 17:34:49,716][20347] Updated weights for policy 0, policy_version 1722712 (0.0008) [2023-12-27 17:34:49,770][20347] Updated weights for policy 0, policy_version 1722722 (0.0008) [2023-12-27 17:34:49,823][20381] Updated weights for policy 1, policy_version 1721201 (0.0010) [2023-12-27 17:34:49,884][20381] Updated weights for policy 1, policy_version 1721211 (0.0009) [2023-12-27 17:34:49,952][20381] Updated weights for policy 1, policy_version 1721221 (0.0008) [2023-12-27 17:34:50,014][20381] Updated weights for policy 1, policy_version 1721231 (0.0008) [2023-12-27 17:34:50,463][20347] Updated weights for policy 0, policy_version 1722732 (0.0005) [2023-12-27 17:34:50,514][20347] Updated weights for policy 0, policy_version 1722742 (0.0006) [2023-12-27 17:34:50,570][20347] Updated weights for policy 0, policy_version 1722752 (0.0008) [2023-12-27 17:34:50,734][20381] Updated weights for policy 1, policy_version 1721241 (0.0007) [2023-12-27 17:34:50,789][20381] Updated weights for policy 1, policy_version 1721251 (0.0007) [2023-12-27 17:34:50,838][20381] Updated weights for policy 1, policy_version 1721261 (0.0005) [2023-12-27 17:34:51,308][20347] Updated weights for policy 0, policy_version 1722762 (0.0008) [2023-12-27 17:34:51,376][20347] Updated weights for policy 0, policy_version 1722772 (0.0008) [2023-12-27 17:34:51,429][20347] Updated weights for policy 0, policy_version 1722782 (0.0008) [2023-12-27 17:34:51,489][20347] Updated weights for policy 0, policy_version 1722792 (0.0008) [2023-12-27 17:34:51,506][20381] Updated weights for policy 1, policy_version 1721271 (0.0007) [2023-12-27 17:34:51,555][20381] Updated weights for policy 1, policy_version 1721281 (0.0007) [2023-12-27 17:34:51,615][20381] Updated weights for policy 1, policy_version 1721291 (0.0007) [2023-12-27 17:34:52,246][20381] Updated weights for policy 1, policy_version 1721301 (0.0007) [2023-12-27 17:34:52,284][20347] Updated weights for policy 0, policy_version 1722802 (0.0011) [2023-12-27 17:34:52,304][20381] Updated weights for policy 1, policy_version 1721311 (0.0007) [2023-12-27 17:34:52,341][20347] Updated weights for policy 0, policy_version 1722812 (0.0009) [2023-12-27 17:34:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 881811456. Throughput: 0: 10479.8, 1: 10178.5. Samples: 881789480. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:34:52,363][19170] Avg episode reward: [(0, '970.670'), (1, '990.270')] [2023-12-27 17:34:52,366][20381] Updated weights for policy 1, policy_version 1721321 (0.0007) [2023-12-27 17:34:52,408][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001721328_440705024.pth... [2023-12-27 17:34:52,411][20347] Updated weights for policy 0, policy_version 1722822 (0.0008) [2023-12-27 17:34:52,412][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001720112_440393728.pth [2023-12-27 17:34:52,419][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001722824_441122816.pth... [2023-12-27 17:34:52,423][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001721576_440803328.pth [2023-12-27 17:34:52,961][20381] Updated weights for policy 1, policy_version 1721331 (0.0006) [2023-12-27 17:34:52,997][20347] Updated weights for policy 0, policy_version 1722832 (0.0011) [2023-12-27 17:34:53,027][20381] Updated weights for policy 1, policy_version 1721341 (0.0005) [2023-12-27 17:34:53,050][20347] Updated weights for policy 0, policy_version 1722842 (0.0010) [2023-12-27 17:34:53,092][20381] Updated weights for policy 1, policy_version 1721351 (0.0006) [2023-12-27 17:34:53,105][20347] Updated weights for policy 0, policy_version 1722852 (0.0010) [2023-12-27 17:34:53,738][20381] Updated weights for policy 1, policy_version 1721361 (0.0006) [2023-12-27 17:34:53,793][20381] Updated weights for policy 1, policy_version 1721371 (0.0006) [2023-12-27 17:34:53,795][20347] Updated weights for policy 0, policy_version 1722862 (0.0010) [2023-12-27 17:34:53,850][20347] Updated weights for policy 0, policy_version 1722872 (0.0010) [2023-12-27 17:34:53,852][20381] Updated weights for policy 1, policy_version 1721381 (0.0005) [2023-12-27 17:34:53,898][20381] Updated weights for policy 1, policy_version 1721391 (0.0008) [2023-12-27 17:34:53,912][20347] Updated weights for policy 0, policy_version 1722882 (0.0010) [2023-12-27 17:34:54,574][20381] Updated weights for policy 1, policy_version 1721401 (0.0007) [2023-12-27 17:34:54,591][20347] Updated weights for policy 0, policy_version 1722892 (0.0010) [2023-12-27 17:34:54,629][20381] Updated weights for policy 1, policy_version 1721411 (0.0005) [2023-12-27 17:34:54,639][20347] Updated weights for policy 0, policy_version 1722902 (0.0010) [2023-12-27 17:34:54,681][20381] Updated weights for policy 1, policy_version 1721421 (0.0005) [2023-12-27 17:34:54,693][20347] Updated weights for policy 0, policy_version 1722912 (0.0010) [2023-12-27 17:34:55,374][20381] Updated weights for policy 1, policy_version 1721431 (0.0007) [2023-12-27 17:34:55,385][20347] Updated weights for policy 0, policy_version 1722922 (0.0008) [2023-12-27 17:34:55,434][20381] Updated weights for policy 1, policy_version 1721441 (0.0008) [2023-12-27 17:34:55,444][20347] Updated weights for policy 0, policy_version 1722932 (0.0010) [2023-12-27 17:34:55,486][20381] Updated weights for policy 1, policy_version 1721451 (0.0005) [2023-12-27 17:34:55,495][20347] Updated weights for policy 0, policy_version 1722942 (0.0010) [2023-12-27 17:34:55,551][20347] Updated weights for policy 0, policy_version 1722952 (0.0010) [2023-12-27 17:34:56,130][20347] Updated weights for policy 0, policy_version 1722962 (0.0007) [2023-12-27 17:34:56,185][20347] Updated weights for policy 0, policy_version 1722972 (0.0005) [2023-12-27 17:34:56,204][20381] Updated weights for policy 1, policy_version 1721461 (0.0006) [2023-12-27 17:34:56,237][20347] Updated weights for policy 0, policy_version 1722982 (0.0005) [2023-12-27 17:34:56,261][20381] Updated weights for policy 1, policy_version 1721471 (0.0008) [2023-12-27 17:34:56,316][20381] Updated weights for policy 1, policy_version 1721481 (0.0010) [2023-12-27 17:34:56,831][20347] Updated weights for policy 0, policy_version 1722992 (0.0007) [2023-12-27 17:34:56,888][20347] Updated weights for policy 0, policy_version 1723002 (0.0005) [2023-12-27 17:34:56,940][20347] Updated weights for policy 0, policy_version 1723012 (0.0005) [2023-12-27 17:34:56,974][20381] Updated weights for policy 1, policy_version 1721491 (0.0009) [2023-12-27 17:34:57,018][20381] Updated weights for policy 1, policy_version 1721501 (0.0008) [2023-12-27 17:34:57,065][20381] Updated weights for policy 1, policy_version 1721511 (0.0007) [2023-12-27 17:34:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 881926144. Throughput: 0: 10454.7, 1: 10209.9. Samples: 881912848. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:34:57,363][19170] Avg episode reward: [(0, '941.420'), (1, '990.270')] [2023-12-27 17:34:57,555][20347] Updated weights for policy 0, policy_version 1723022 (0.0008) [2023-12-27 17:34:57,600][20347] Updated weights for policy 0, policy_version 1723032 (0.0010) [2023-12-27 17:34:57,651][20347] Updated weights for policy 0, policy_version 1723042 (0.0010) [2023-12-27 17:34:57,717][20381] Updated weights for policy 1, policy_version 1721521 (0.0007) [2023-12-27 17:34:57,776][20381] Updated weights for policy 1, policy_version 1721531 (0.0006) [2023-12-27 17:34:57,827][20381] Updated weights for policy 1, policy_version 1721541 (0.0005) [2023-12-27 17:34:57,875][20381] Updated weights for policy 1, policy_version 1721551 (0.0005) [2023-12-27 17:34:58,339][20347] Updated weights for policy 0, policy_version 1723052 (0.0009) [2023-12-27 17:34:58,403][20347] Updated weights for policy 0, policy_version 1723062 (0.0008) [2023-12-27 17:34:58,464][20347] Updated weights for policy 0, policy_version 1723072 (0.0008) [2023-12-27 17:34:58,563][20381] Updated weights for policy 1, policy_version 1721561 (0.0008) [2023-12-27 17:34:58,621][20381] Updated weights for policy 1, policy_version 1721571 (0.0007) [2023-12-27 17:34:58,686][20381] Updated weights for policy 1, policy_version 1721581 (0.0007) [2023-12-27 17:34:59,397][20347] Updated weights for policy 0, policy_version 1723082 (0.0008) [2023-12-27 17:34:59,458][20347] Updated weights for policy 0, policy_version 1723092 (0.0006) [2023-12-27 17:34:59,514][20347] Updated weights for policy 0, policy_version 1723102 (0.0007) [2023-12-27 17:34:59,552][20381] Updated weights for policy 1, policy_version 1721591 (0.0008) [2023-12-27 17:34:59,572][20347] Updated weights for policy 0, policy_version 1723112 (0.0009) [2023-12-27 17:34:59,616][20381] Updated weights for policy 1, policy_version 1721601 (0.0008) [2023-12-27 17:34:59,667][20381] Updated weights for policy 1, policy_version 1721611 (0.0008) [2023-12-27 17:35:00,312][20381] Updated weights for policy 1, policy_version 1721621 (0.0007) [2023-12-27 17:35:00,322][20347] Updated weights for policy 0, policy_version 1723122 (0.0007) [2023-12-27 17:35:00,364][20381] Updated weights for policy 1, policy_version 1721631 (0.0005) [2023-12-27 17:35:00,382][20347] Updated weights for policy 0, policy_version 1723132 (0.0007) [2023-12-27 17:35:00,421][20381] Updated weights for policy 1, policy_version 1721641 (0.0006) [2023-12-27 17:35:00,447][20347] Updated weights for policy 0, policy_version 1723142 (0.0007) [2023-12-27 17:35:01,034][20381] Updated weights for policy 1, policy_version 1721651 (0.0007) [2023-12-27 17:35:01,059][20347] Updated weights for policy 0, policy_version 1723152 (0.0010) [2023-12-27 17:35:01,093][20381] Updated weights for policy 1, policy_version 1721661 (0.0006) [2023-12-27 17:35:01,112][20347] Updated weights for policy 0, policy_version 1723162 (0.0011) [2023-12-27 17:35:01,149][20381] Updated weights for policy 1, policy_version 1721671 (0.0007) [2023-12-27 17:35:01,175][20347] Updated weights for policy 0, policy_version 1723172 (0.0007) [2023-12-27 17:35:01,876][20347] Updated weights for policy 0, policy_version 1723182 (0.0007) [2023-12-27 17:35:01,901][20381] Updated weights for policy 1, policy_version 1721681 (0.0008) [2023-12-27 17:35:01,933][20347] Updated weights for policy 0, policy_version 1723192 (0.0006) [2023-12-27 17:35:01,962][20381] Updated weights for policy 1, policy_version 1721691 (0.0007) [2023-12-27 17:35:01,991][20347] Updated weights for policy 0, policy_version 1723202 (0.0006) [2023-12-27 17:35:02,016][20381] Updated weights for policy 1, policy_version 1721701 (0.0006) [2023-12-27 17:35:02,076][20381] Updated weights for policy 1, policy_version 1721711 (0.0008) [2023-12-27 17:35:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 882024448. Throughput: 0: 10435.4, 1: 10116.9. Samples: 882033232. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:02,363][19170] Avg episode reward: [(0, '931.620'), (1, '1000.000')] [2023-12-27 17:35:02,596][20347] Updated weights for policy 0, policy_version 1723212 (0.0005) [2023-12-27 17:35:02,658][20347] Updated weights for policy 0, policy_version 1723222 (0.0006) [2023-12-27 17:35:02,716][20347] Updated weights for policy 0, policy_version 1723232 (0.0007) [2023-12-27 17:35:02,847][20381] Updated weights for policy 1, policy_version 1721721 (0.0008) [2023-12-27 17:35:02,902][20381] Updated weights for policy 1, policy_version 1721731 (0.0008) [2023-12-27 17:35:02,953][20381] Updated weights for policy 1, policy_version 1721741 (0.0007) [2023-12-27 17:35:03,377][20347] Updated weights for policy 0, policy_version 1723242 (0.0007) [2023-12-27 17:35:03,434][20347] Updated weights for policy 0, policy_version 1723252 (0.0008) [2023-12-27 17:35:03,486][20347] Updated weights for policy 0, policy_version 1723262 (0.0008) [2023-12-27 17:35:03,543][20347] Updated weights for policy 0, policy_version 1723272 (0.0006) [2023-12-27 17:35:03,568][20381] Updated weights for policy 1, policy_version 1721751 (0.0007) [2023-12-27 17:35:03,620][20381] Updated weights for policy 1, policy_version 1721761 (0.0007) [2023-12-27 17:35:03,668][20381] Updated weights for policy 1, policy_version 1721771 (0.0008) [2023-12-27 17:35:04,248][20347] Updated weights for policy 0, policy_version 1723282 (0.0008) [2023-12-27 17:35:04,303][20347] Updated weights for policy 0, policy_version 1723292 (0.0008) [2023-12-27 17:35:04,362][20347] Updated weights for policy 0, policy_version 1723302 (0.0007) [2023-12-27 17:35:04,379][20381] Updated weights for policy 1, policy_version 1721781 (0.0007) [2023-12-27 17:35:04,429][20381] Updated weights for policy 1, policy_version 1721791 (0.0008) [2023-12-27 17:35:04,480][20381] Updated weights for policy 1, policy_version 1721801 (0.0007) [2023-12-27 17:35:05,084][20381] Updated weights for policy 1, policy_version 1721811 (0.0007) [2023-12-27 17:35:05,086][20347] Updated weights for policy 0, policy_version 1723312 (0.0006) [2023-12-27 17:35:05,140][20381] Updated weights for policy 1, policy_version 1721821 (0.0005) [2023-12-27 17:35:05,142][20347] Updated weights for policy 0, policy_version 1723322 (0.0005) [2023-12-27 17:35:05,185][20381] Updated weights for policy 1, policy_version 1721831 (0.0006) [2023-12-27 17:35:05,226][20347] Updated weights for policy 0, policy_version 1723332 (0.0007) [2023-12-27 17:35:05,885][20381] Updated weights for policy 1, policy_version 1721841 (0.0007) [2023-12-27 17:35:05,890][20347] Updated weights for policy 0, policy_version 1723342 (0.0008) [2023-12-27 17:35:05,940][20381] Updated weights for policy 1, policy_version 1721851 (0.0006) [2023-12-27 17:35:05,943][20347] Updated weights for policy 0, policy_version 1723352 (0.0006) [2023-12-27 17:35:05,986][20381] Updated weights for policy 1, policy_version 1721861 (0.0006) [2023-12-27 17:35:05,992][20347] Updated weights for policy 0, policy_version 1723362 (0.0006) [2023-12-27 17:35:06,042][20381] Updated weights for policy 1, policy_version 1721871 (0.0007) [2023-12-27 17:35:06,649][20347] Updated weights for policy 0, policy_version 1723372 (0.0006) [2023-12-27 17:35:06,715][20347] Updated weights for policy 0, policy_version 1723382 (0.0007) [2023-12-27 17:35:06,765][20381] Updated weights for policy 1, policy_version 1721881 (0.0007) [2023-12-27 17:35:06,767][20347] Updated weights for policy 0, policy_version 1723392 (0.0006) [2023-12-27 17:35:06,823][20381] Updated weights for policy 1, policy_version 1721891 (0.0010) [2023-12-27 17:35:06,883][20381] Updated weights for policy 1, policy_version 1721901 (0.0008) [2023-12-27 17:35:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.1, 300 sec: 20438.3). Total num frames: 882122752. Throughput: 0: 10416.6, 1: 10169.6. Samples: 882095892. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:07,363][19170] Avg episode reward: [(0, '951.120'), (1, '1000.000')] [2023-12-27 17:35:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001721904_440852480.pth... [2023-12-27 17:35:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001723400_441270272.pth... [2023-12-27 17:35:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001722216_440967168.pth [2023-12-27 17:35:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001720720_440549376.pth [2023-12-27 17:35:07,425][20347] Updated weights for policy 0, policy_version 1723402 (0.0009) [2023-12-27 17:35:07,477][20347] Updated weights for policy 0, policy_version 1723412 (0.0007) [2023-12-27 17:35:07,532][20347] Updated weights for policy 0, policy_version 1723422 (0.0008) [2023-12-27 17:35:07,589][20381] Updated weights for policy 1, policy_version 1721911 (0.0008) [2023-12-27 17:35:07,593][20347] Updated weights for policy 0, policy_version 1723432 (0.0007) [2023-12-27 17:35:07,644][20381] Updated weights for policy 1, policy_version 1721921 (0.0008) [2023-12-27 17:35:07,699][20381] Updated weights for policy 1, policy_version 1721931 (0.0008) [2023-12-27 17:35:08,295][20347] Updated weights for policy 0, policy_version 1723442 (0.0007) [2023-12-27 17:35:08,360][20347] Updated weights for policy 0, policy_version 1723452 (0.0007) [2023-12-27 17:35:08,378][20291] KL-divergence is very high: 140.3302 [2023-12-27 17:35:08,402][20381] Updated weights for policy 1, policy_version 1721941 (0.0008) [2023-12-27 17:35:08,418][20347] Updated weights for policy 0, policy_version 1723462 (0.0006) [2023-12-27 17:35:08,426][20291] KL-divergence is very high: 101.6043 [2023-12-27 17:35:08,465][20381] Updated weights for policy 1, policy_version 1721951 (0.0008) [2023-12-27 17:35:08,517][20381] Updated weights for policy 1, policy_version 1721961 (0.0008) [2023-12-27 17:35:09,068][20347] Updated weights for policy 0, policy_version 1723472 (0.0007) [2023-12-27 17:35:09,130][20347] Updated weights for policy 0, policy_version 1723482 (0.0008) [2023-12-27 17:35:09,174][20381] Updated weights for policy 1, policy_version 1721971 (0.0009) [2023-12-27 17:35:09,190][20347] Updated weights for policy 0, policy_version 1723492 (0.0008) [2023-12-27 17:35:09,236][20381] Updated weights for policy 1, policy_version 1721981 (0.0007) [2023-12-27 17:35:09,291][20381] Updated weights for policy 1, policy_version 1721991 (0.0008) [2023-12-27 17:35:09,896][20347] Updated weights for policy 0, policy_version 1723502 (0.0007) [2023-12-27 17:35:09,960][20347] Updated weights for policy 0, policy_version 1723512 (0.0007) [2023-12-27 17:35:09,974][20381] Updated weights for policy 1, policy_version 1722001 (0.0008) [2023-12-27 17:35:10,025][20347] Updated weights for policy 0, policy_version 1723522 (0.0007) [2023-12-27 17:35:10,034][20381] Updated weights for policy 1, policy_version 1722011 (0.0008) [2023-12-27 17:35:10,098][20381] Updated weights for policy 1, policy_version 1722021 (0.0008) [2023-12-27 17:35:10,160][20381] Updated weights for policy 1, policy_version 1722031 (0.0008) [2023-12-27 17:35:10,641][20347] Updated weights for policy 0, policy_version 1723532 (0.0006) [2023-12-27 17:35:10,704][20347] Updated weights for policy 0, policy_version 1723542 (0.0005) [2023-12-27 17:35:10,768][20347] Updated weights for policy 0, policy_version 1723552 (0.0005) [2023-12-27 17:35:10,901][20381] Updated weights for policy 1, policy_version 1722041 (0.0008) [2023-12-27 17:35:10,957][20381] Updated weights for policy 1, policy_version 1722051 (0.0007) [2023-12-27 17:35:11,011][20381] Updated weights for policy 1, policy_version 1722061 (0.0008) [2023-12-27 17:35:11,450][20347] Updated weights for policy 0, policy_version 1723562 (0.0008) [2023-12-27 17:35:11,513][20347] Updated weights for policy 0, policy_version 1723572 (0.0007) [2023-12-27 17:35:11,576][20347] Updated weights for policy 0, policy_version 1723582 (0.0008) [2023-12-27 17:35:11,640][20347] Updated weights for policy 0, policy_version 1723592 (0.0009) [2023-12-27 17:35:11,759][20381] Updated weights for policy 1, policy_version 1722071 (0.0009) [2023-12-27 17:35:11,811][20381] Updated weights for policy 1, policy_version 1722081 (0.0007) [2023-12-27 17:35:11,866][20381] Updated weights for policy 1, policy_version 1722091 (0.0007) [2023-12-27 17:35:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 882221056. Throughput: 0: 10410.2, 1: 10091.1. Samples: 882216420. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:12,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 17:35:12,418][20347] Updated weights for policy 0, policy_version 1723602 (0.0007) [2023-12-27 17:35:12,477][20347] Updated weights for policy 0, policy_version 1723612 (0.0006) [2023-12-27 17:35:12,533][20347] Updated weights for policy 0, policy_version 1723622 (0.0007) [2023-12-27 17:35:12,569][20381] Updated weights for policy 1, policy_version 1722101 (0.0008) [2023-12-27 17:35:12,628][20381] Updated weights for policy 1, policy_version 1722111 (0.0008) [2023-12-27 17:35:12,685][20381] Updated weights for policy 1, policy_version 1722121 (0.0008) [2023-12-27 17:35:13,130][20347] Updated weights for policy 0, policy_version 1723632 (0.0009) [2023-12-27 17:35:13,184][20347] Updated weights for policy 0, policy_version 1723642 (0.0010) [2023-12-27 17:35:13,239][20347] Updated weights for policy 0, policy_version 1723652 (0.0010) [2023-12-27 17:35:13,422][20381] Updated weights for policy 1, policy_version 1722131 (0.0007) [2023-12-27 17:35:13,479][20381] Updated weights for policy 1, policy_version 1722141 (0.0007) [2023-12-27 17:35:13,535][20381] Updated weights for policy 1, policy_version 1722151 (0.0007) [2023-12-27 17:35:13,897][20347] Updated weights for policy 0, policy_version 1723662 (0.0007) [2023-12-27 17:35:13,958][20347] Updated weights for policy 0, policy_version 1723672 (0.0006) [2023-12-27 17:35:14,017][20347] Updated weights for policy 0, policy_version 1723682 (0.0010) [2023-12-27 17:35:14,208][20381] Updated weights for policy 1, policy_version 1722161 (0.0008) [2023-12-27 17:35:14,273][20381] Updated weights for policy 1, policy_version 1722171 (0.0007) [2023-12-27 17:35:14,331][20381] Updated weights for policy 1, policy_version 1722181 (0.0005) [2023-12-27 17:35:14,395][20381] Updated weights for policy 1, policy_version 1722191 (0.0006) [2023-12-27 17:35:14,623][20347] Updated weights for policy 0, policy_version 1723692 (0.0009) [2023-12-27 17:35:14,678][20347] Updated weights for policy 0, policy_version 1723702 (0.0008) [2023-12-27 17:35:14,733][20347] Updated weights for policy 0, policy_version 1723712 (0.0007) [2023-12-27 17:35:15,141][20381] Updated weights for policy 1, policy_version 1722201 (0.0007) [2023-12-27 17:35:15,205][20381] Updated weights for policy 1, policy_version 1722211 (0.0008) [2023-12-27 17:35:15,260][20381] Updated weights for policy 1, policy_version 1722221 (0.0008) [2023-12-27 17:35:15,427][20347] Updated weights for policy 0, policy_version 1723722 (0.0007) [2023-12-27 17:35:15,479][20347] Updated weights for policy 0, policy_version 1723732 (0.0005) [2023-12-27 17:35:15,529][20347] Updated weights for policy 0, policy_version 1723742 (0.0005) [2023-12-27 17:35:15,574][20347] Updated weights for policy 0, policy_version 1723752 (0.0005) [2023-12-27 17:35:15,941][20381] Updated weights for policy 1, policy_version 1722231 (0.0009) [2023-12-27 17:35:15,988][20381] Updated weights for policy 1, policy_version 1722241 (0.0006) [2023-12-27 17:35:16,034][20381] Updated weights for policy 1, policy_version 1722251 (0.0005) [2023-12-27 17:35:16,143][20347] Updated weights for policy 0, policy_version 1723762 (0.0008) [2023-12-27 17:35:16,192][20347] Updated weights for policy 0, policy_version 1723772 (0.0010) [2023-12-27 17:35:16,244][20347] Updated weights for policy 0, policy_version 1723782 (0.0010) [2023-12-27 17:35:16,646][20381] Updated weights for policy 1, policy_version 1722261 (0.0008) [2023-12-27 17:35:16,690][20381] Updated weights for policy 1, policy_version 1722271 (0.0010) [2023-12-27 17:35:16,741][20381] Updated weights for policy 1, policy_version 1722281 (0.0008) [2023-12-27 17:35:16,840][20347] Updated weights for policy 0, policy_version 1723792 (0.0006) [2023-12-27 17:35:16,889][20347] Updated weights for policy 0, policy_version 1723802 (0.0005) [2023-12-27 17:35:16,948][20347] Updated weights for policy 0, policy_version 1723812 (0.0006) [2023-12-27 17:35:17,332][20381] Updated weights for policy 1, policy_version 1722291 (0.0005) [2023-12-27 17:35:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 882327552. Throughput: 0: 10413.5, 1: 10105.0. Samples: 882341264. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:17,363][19170] Avg episode reward: [(0, '941.340'), (1, '1000.000')] [2023-12-27 17:35:17,382][20381] Updated weights for policy 1, policy_version 1722301 (0.0008) [2023-12-27 17:35:17,434][20381] Updated weights for policy 1, policy_version 1722311 (0.0008) [2023-12-27 17:35:17,585][20347] Updated weights for policy 0, policy_version 1723822 (0.0010) [2023-12-27 17:35:17,642][20347] Updated weights for policy 0, policy_version 1723832 (0.0010) [2023-12-27 17:35:17,696][20347] Updated weights for policy 0, policy_version 1723842 (0.0010) [2023-12-27 17:35:18,036][20381] Updated weights for policy 1, policy_version 1722321 (0.0008) [2023-12-27 17:35:18,091][20381] Updated weights for policy 1, policy_version 1722331 (0.0005) [2023-12-27 17:35:18,139][20381] Updated weights for policy 1, policy_version 1722341 (0.0005) [2023-12-27 17:35:18,188][20381] Updated weights for policy 1, policy_version 1722351 (0.0005) [2023-12-27 17:35:18,356][20347] Updated weights for policy 0, policy_version 1723852 (0.0010) [2023-12-27 17:35:18,421][20347] Updated weights for policy 0, policy_version 1723862 (0.0008) [2023-12-27 17:35:18,477][20347] Updated weights for policy 0, policy_version 1723872 (0.0008) [2023-12-27 17:35:18,835][20381] Updated weights for policy 1, policy_version 1722361 (0.0010) [2023-12-27 17:35:18,893][20381] Updated weights for policy 1, policy_version 1722371 (0.0009) [2023-12-27 17:35:18,951][20381] Updated weights for policy 1, policy_version 1722381 (0.0006) [2023-12-27 17:35:19,191][20347] Updated weights for policy 0, policy_version 1723882 (0.0007) [2023-12-27 17:35:19,257][20347] Updated weights for policy 0, policy_version 1723892 (0.0009) [2023-12-27 17:35:19,310][20347] Updated weights for policy 0, policy_version 1723902 (0.0007) [2023-12-27 17:35:19,375][20347] Updated weights for policy 0, policy_version 1723912 (0.0008) [2023-12-27 17:35:19,679][20381] Updated weights for policy 1, policy_version 1722391 (0.0007) [2023-12-27 17:35:19,733][20381] Updated weights for policy 1, policy_version 1722401 (0.0008) [2023-12-27 17:35:19,782][20381] Updated weights for policy 1, policy_version 1722411 (0.0007) [2023-12-27 17:35:20,107][20347] Updated weights for policy 0, policy_version 1723922 (0.0011) [2023-12-27 17:35:20,161][20347] Updated weights for policy 0, policy_version 1723932 (0.0011) [2023-12-27 17:35:20,219][20347] Updated weights for policy 0, policy_version 1723942 (0.0011) [2023-12-27 17:35:20,531][20381] Updated weights for policy 1, policy_version 1722421 (0.0008) [2023-12-27 17:35:20,589][20381] Updated weights for policy 1, policy_version 1722431 (0.0008) [2023-12-27 17:35:20,647][20381] Updated weights for policy 1, policy_version 1722441 (0.0008) [2023-12-27 17:35:20,943][20347] Updated weights for policy 0, policy_version 1723952 (0.0011) [2023-12-27 17:35:21,002][20347] Updated weights for policy 0, policy_version 1723962 (0.0010) [2023-12-27 17:35:21,060][20347] Updated weights for policy 0, policy_version 1723972 (0.0010) [2023-12-27 17:35:21,391][20381] Updated weights for policy 1, policy_version 1722451 (0.0008) [2023-12-27 17:35:21,450][20381] Updated weights for policy 1, policy_version 1722461 (0.0008) [2023-12-27 17:35:21,499][20381] Updated weights for policy 1, policy_version 1722471 (0.0007) [2023-12-27 17:35:21,848][20347] Updated weights for policy 0, policy_version 1723982 (0.0010) [2023-12-27 17:35:21,903][20347] Updated weights for policy 0, policy_version 1723992 (0.0010) [2023-12-27 17:35:21,958][20347] Updated weights for policy 0, policy_version 1724002 (0.0010) [2023-12-27 17:35:22,231][20381] Updated weights for policy 1, policy_version 1722481 (0.0008) [2023-12-27 17:35:22,298][20381] Updated weights for policy 1, policy_version 1722491 (0.0008) [2023-12-27 17:35:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 882425856. Throughput: 0: 10352.3, 1: 10137.8. Samples: 882402744. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:22,363][19170] Avg episode reward: [(0, '921.720'), (1, '1000.000')] [2023-12-27 17:35:22,365][20381] Updated weights for policy 1, policy_version 1722501 (0.0008) [2023-12-27 17:35:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001724008_441425920.pth... [2023-12-27 17:35:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001722824_441122816.pth [2023-12-27 17:35:22,428][20381] Updated weights for policy 1, policy_version 1722511 (0.0008) [2023-12-27 17:35:22,433][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001722512_441008128.pth... [2023-12-27 17:35:22,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001721328_440705024.pth [2023-12-27 17:35:22,657][20347] Updated weights for policy 0, policy_version 1724012 (0.0010) [2023-12-27 17:35:22,709][20347] Updated weights for policy 0, policy_version 1724022 (0.0010) [2023-12-27 17:35:22,766][20347] Updated weights for policy 0, policy_version 1724032 (0.0010) [2023-12-27 17:35:23,172][20381] Updated weights for policy 1, policy_version 1722521 (0.0007) [2023-12-27 17:35:23,233][20381] Updated weights for policy 1, policy_version 1722531 (0.0008) [2023-12-27 17:35:23,295][20381] Updated weights for policy 1, policy_version 1722541 (0.0007) [2023-12-27 17:35:23,464][20347] Updated weights for policy 0, policy_version 1724042 (0.0009) [2023-12-27 17:35:23,520][20347] Updated weights for policy 0, policy_version 1724052 (0.0005) [2023-12-27 17:35:23,580][20347] Updated weights for policy 0, policy_version 1724062 (0.0006) [2023-12-27 17:35:23,646][20347] Updated weights for policy 0, policy_version 1724072 (0.0006) [2023-12-27 17:35:23,922][20381] Updated weights for policy 1, policy_version 1722551 (0.0006) [2023-12-27 17:35:23,985][20381] Updated weights for policy 1, policy_version 1722561 (0.0007) [2023-12-27 17:35:24,039][20381] Updated weights for policy 1, policy_version 1722571 (0.0005) [2023-12-27 17:35:24,257][20347] Updated weights for policy 0, policy_version 1724082 (0.0007) [2023-12-27 17:35:24,316][20347] Updated weights for policy 0, policy_version 1724092 (0.0008) [2023-12-27 17:35:24,374][20347] Updated weights for policy 0, policy_version 1724102 (0.0007) [2023-12-27 17:35:24,629][20381] Updated weights for policy 1, policy_version 1722581 (0.0008) [2023-12-27 17:35:24,688][20381] Updated weights for policy 1, policy_version 1722591 (0.0011) [2023-12-27 17:35:24,748][20381] Updated weights for policy 1, policy_version 1722601 (0.0009) [2023-12-27 17:35:25,022][20347] Updated weights for policy 0, policy_version 1724112 (0.0008) [2023-12-27 17:35:25,074][20347] Updated weights for policy 0, policy_version 1724122 (0.0006) [2023-12-27 17:35:25,130][20347] Updated weights for policy 0, policy_version 1724132 (0.0005) [2023-12-27 17:35:25,363][20381] Updated weights for policy 1, policy_version 1722611 (0.0007) [2023-12-27 17:35:25,418][20381] Updated weights for policy 1, policy_version 1722621 (0.0010) [2023-12-27 17:35:25,468][20381] Updated weights for policy 1, policy_version 1722631 (0.0011) [2023-12-27 17:35:25,760][20347] Updated weights for policy 0, policy_version 1724142 (0.0006) [2023-12-27 17:35:25,815][20347] Updated weights for policy 0, policy_version 1724152 (0.0007) [2023-12-27 17:35:25,868][20347] Updated weights for policy 0, policy_version 1724162 (0.0006) [2023-12-27 17:35:26,165][20381] Updated weights for policy 1, policy_version 1722641 (0.0011) [2023-12-27 17:35:26,221][20381] Updated weights for policy 1, policy_version 1722651 (0.0010) [2023-12-27 17:35:26,279][20381] Updated weights for policy 1, policy_version 1722661 (0.0010) [2023-12-27 17:35:26,344][20381] Updated weights for policy 1, policy_version 1722671 (0.0011) [2023-12-27 17:35:26,518][20347] Updated weights for policy 0, policy_version 1724172 (0.0006) [2023-12-27 17:35:26,569][20347] Updated weights for policy 0, policy_version 1724182 (0.0005) [2023-12-27 17:35:26,626][20347] Updated weights for policy 0, policy_version 1724192 (0.0005) [2023-12-27 17:35:27,007][20381] Updated weights for policy 1, policy_version 1722681 (0.0010) [2023-12-27 17:35:27,061][20381] Updated weights for policy 1, policy_version 1722691 (0.0010) [2023-12-27 17:35:27,105][20381] Updated weights for policy 1, policy_version 1722701 (0.0010) [2023-12-27 17:35:27,164][20347] Updated weights for policy 0, policy_version 1724202 (0.0006) [2023-12-27 17:35:27,222][20347] Updated weights for policy 0, policy_version 1724212 (0.0010) [2023-12-27 17:35:27,269][20347] Updated weights for policy 0, policy_version 1724222 (0.0008) [2023-12-27 17:35:27,317][20347] Updated weights for policy 0, policy_version 1724232 (0.0006) [2023-12-27 17:35:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 882540544. Throughput: 0: 10322.5, 1: 10134.0. Samples: 882524032. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:27,363][19170] Avg episode reward: [(0, '960.850'), (1, '1000.000')] [2023-12-27 17:35:27,798][20381] Updated weights for policy 1, policy_version 1722711 (0.0010) [2023-12-27 17:35:27,843][20381] Updated weights for policy 1, policy_version 1722721 (0.0010) [2023-12-27 17:35:27,893][20381] Updated weights for policy 1, policy_version 1722731 (0.0007) [2023-12-27 17:35:27,956][20347] Updated weights for policy 0, policy_version 1724242 (0.0005) [2023-12-27 17:35:28,006][20347] Updated weights for policy 0, policy_version 1724252 (0.0006) [2023-12-27 17:35:28,059][20347] Updated weights for policy 0, policy_version 1724262 (0.0005) [2023-12-27 17:35:28,566][20381] Updated weights for policy 1, policy_version 1722741 (0.0011) [2023-12-27 17:35:28,612][20381] Updated weights for policy 1, policy_version 1722751 (0.0011) [2023-12-27 17:35:28,658][20381] Updated weights for policy 1, policy_version 1722761 (0.0010) [2023-12-27 17:35:28,703][20347] Updated weights for policy 0, policy_version 1724272 (0.0007) [2023-12-27 17:35:28,704][20291] KL-divergence is very high: 100.4892 [2023-12-27 17:35:28,720][20291] KL-divergence is very high: 124.5339 [2023-12-27 17:35:28,746][20291] KL-divergence is very high: 120.2371 [2023-12-27 17:35:28,756][20347] Updated weights for policy 0, policy_version 1724282 (0.0008) [2023-12-27 17:35:28,810][20347] Updated weights for policy 0, policy_version 1724292 (0.0007) [2023-12-27 17:35:29,380][20381] Updated weights for policy 1, policy_version 1722771 (0.0009) [2023-12-27 17:35:29,440][20381] Updated weights for policy 1, policy_version 1722781 (0.0006) [2023-12-27 17:35:29,484][20347] Updated weights for policy 0, policy_version 1724302 (0.0010) [2023-12-27 17:35:29,500][20381] Updated weights for policy 1, policy_version 1722791 (0.0008) [2023-12-27 17:35:29,546][20347] Updated weights for policy 0, policy_version 1724312 (0.0005) [2023-12-27 17:35:29,601][20347] Updated weights for policy 0, policy_version 1724322 (0.0007) [2023-12-27 17:35:30,173][20381] Updated weights for policy 1, policy_version 1722801 (0.0007) [2023-12-27 17:35:30,229][20381] Updated weights for policy 1, policy_version 1722811 (0.0011) [2023-12-27 17:35:30,257][20347] Updated weights for policy 0, policy_version 1724332 (0.0007) [2023-12-27 17:35:30,289][20381] Updated weights for policy 1, policy_version 1722821 (0.0011) [2023-12-27 17:35:30,313][20347] Updated weights for policy 0, policy_version 1724342 (0.0005) [2023-12-27 17:35:30,342][20381] Updated weights for policy 1, policy_version 1722831 (0.0010) [2023-12-27 17:35:30,363][20347] Updated weights for policy 0, policy_version 1724352 (0.0005) [2023-12-27 17:35:31,002][20347] Updated weights for policy 0, policy_version 1724362 (0.0007) [2023-12-27 17:35:31,050][20381] Updated weights for policy 1, policy_version 1722841 (0.0013) [2023-12-27 17:35:31,061][20347] Updated weights for policy 0, policy_version 1724372 (0.0006) [2023-12-27 17:35:31,104][20381] Updated weights for policy 1, policy_version 1722851 (0.0008) [2023-12-27 17:35:31,124][20347] Updated weights for policy 0, policy_version 1724382 (0.0007) [2023-12-27 17:35:31,164][20381] Updated weights for policy 1, policy_version 1722861 (0.0009) [2023-12-27 17:35:31,184][20347] Updated weights for policy 0, policy_version 1724392 (0.0009) [2023-12-27 17:35:31,874][20347] Updated weights for policy 0, policy_version 1724402 (0.0008) [2023-12-27 17:35:31,883][20381] Updated weights for policy 1, policy_version 1722871 (0.0007) [2023-12-27 17:35:31,927][20347] Updated weights for policy 0, policy_version 1724412 (0.0007) [2023-12-27 17:35:31,940][20381] Updated weights for policy 1, policy_version 1722881 (0.0007) [2023-12-27 17:35:31,975][20347] Updated weights for policy 0, policy_version 1724422 (0.0005) [2023-12-27 17:35:31,996][20381] Updated weights for policy 1, policy_version 1722891 (0.0008) [2023-12-27 17:35:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 882638848. Throughput: 0: 10334.0, 1: 10151.5. Samples: 882648880. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:32,363][19170] Avg episode reward: [(0, '921.820'), (1, '1000.000')] [2023-12-27 17:35:32,650][20381] Updated weights for policy 1, policy_version 1722901 (0.0009) [2023-12-27 17:35:32,689][20347] Updated weights for policy 0, policy_version 1724432 (0.0006) [2023-12-27 17:35:32,708][20381] Updated weights for policy 1, policy_version 1722911 (0.0008) [2023-12-27 17:35:32,743][20347] Updated weights for policy 0, policy_version 1724442 (0.0006) [2023-12-27 17:35:32,767][20381] Updated weights for policy 1, policy_version 1722921 (0.0008) [2023-12-27 17:35:32,792][20347] Updated weights for policy 0, policy_version 1724452 (0.0005) [2023-12-27 17:35:33,358][20381] Updated weights for policy 1, policy_version 1722931 (0.0008) [2023-12-27 17:35:33,413][20381] Updated weights for policy 1, policy_version 1722941 (0.0007) [2023-12-27 17:35:33,442][20347] Updated weights for policy 0, policy_version 1724462 (0.0007) [2023-12-27 17:35:33,462][20381] Updated weights for policy 1, policy_version 1722951 (0.0007) [2023-12-27 17:35:33,494][20347] Updated weights for policy 0, policy_version 1724472 (0.0006) [2023-12-27 17:35:33,546][20347] Updated weights for policy 0, policy_version 1724482 (0.0007) [2023-12-27 17:35:34,133][20381] Updated weights for policy 1, policy_version 1722961 (0.0006) [2023-12-27 17:35:34,160][20347] Updated weights for policy 0, policy_version 1724492 (0.0007) [2023-12-27 17:35:34,194][20381] Updated weights for policy 1, policy_version 1722971 (0.0007) [2023-12-27 17:35:34,219][20347] Updated weights for policy 0, policy_version 1724502 (0.0006) [2023-12-27 17:35:34,255][20381] Updated weights for policy 1, policy_version 1722981 (0.0005) [2023-12-27 17:35:34,280][20347] Updated weights for policy 0, policy_version 1724512 (0.0006) [2023-12-27 17:35:34,319][20381] Updated weights for policy 1, policy_version 1722991 (0.0006) [2023-12-27 17:35:34,942][20381] Updated weights for policy 1, policy_version 1723001 (0.0009) [2023-12-27 17:35:34,947][20347] Updated weights for policy 0, policy_version 1724522 (0.0006) [2023-12-27 17:35:34,996][20347] Updated weights for policy 0, policy_version 1724532 (0.0005) [2023-12-27 17:35:35,006][20381] Updated weights for policy 1, policy_version 1723011 (0.0008) [2023-12-27 17:35:35,048][20347] Updated weights for policy 0, policy_version 1724542 (0.0006) [2023-12-27 17:35:35,065][20381] Updated weights for policy 1, policy_version 1723021 (0.0006) [2023-12-27 17:35:35,106][20347] Updated weights for policy 0, policy_version 1724552 (0.0007) [2023-12-27 17:35:35,604][20381] Updated weights for policy 1, policy_version 1723031 (0.0005) [2023-12-27 17:35:35,662][20381] Updated weights for policy 1, policy_version 1723041 (0.0010) [2023-12-27 17:35:35,719][20381] Updated weights for policy 1, policy_version 1723051 (0.0010) [2023-12-27 17:35:35,794][20347] Updated weights for policy 0, policy_version 1724562 (0.0009) [2023-12-27 17:35:35,848][20347] Updated weights for policy 0, policy_version 1724572 (0.0009) [2023-12-27 17:35:35,899][20347] Updated weights for policy 0, policy_version 1724582 (0.0007) [2023-12-27 17:35:36,281][20381] Updated weights for policy 1, policy_version 1723061 (0.0008) [2023-12-27 17:35:36,333][20381] Updated weights for policy 1, policy_version 1723071 (0.0011) [2023-12-27 17:35:36,386][20381] Updated weights for policy 1, policy_version 1723081 (0.0011) [2023-12-27 17:35:36,625][20347] Updated weights for policy 0, policy_version 1724592 (0.0008) [2023-12-27 17:35:36,681][20347] Updated weights for policy 0, policy_version 1724602 (0.0007) [2023-12-27 17:35:36,743][20347] Updated weights for policy 0, policy_version 1724612 (0.0006) [2023-12-27 17:35:37,065][20381] Updated weights for policy 1, policy_version 1723091 (0.0009) [2023-12-27 17:35:37,122][20381] Updated weights for policy 1, policy_version 1723101 (0.0010) [2023-12-27 17:35:37,174][20381] Updated weights for policy 1, policy_version 1723111 (0.0007) [2023-12-27 17:35:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 882745344. Throughput: 0: 10339.8, 1: 10194.9. Samples: 882713544. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:37,364][19170] Avg episode reward: [(0, '941.400'), (1, '1000.000')] [2023-12-27 17:35:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001724616_441581568.pth... [2023-12-27 17:35:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001723120_441163776.pth... [2023-12-27 17:35:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001721904_440852480.pth [2023-12-27 17:35:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001723400_441270272.pth [2023-12-27 17:35:37,433][20347] Updated weights for policy 0, policy_version 1724622 (0.0009) [2023-12-27 17:35:37,490][20347] Updated weights for policy 0, policy_version 1724632 (0.0010) [2023-12-27 17:35:37,549][20347] Updated weights for policy 0, policy_version 1724642 (0.0010) [2023-12-27 17:35:37,884][20381] Updated weights for policy 1, policy_version 1723121 (0.0008) [2023-12-27 17:35:37,943][20381] Updated weights for policy 1, policy_version 1723131 (0.0007) [2023-12-27 17:35:37,997][20381] Updated weights for policy 1, policy_version 1723141 (0.0009) [2023-12-27 17:35:38,049][20381] Updated weights for policy 1, policy_version 1723151 (0.0008) [2023-12-27 17:35:38,156][20347] Updated weights for policy 0, policy_version 1724652 (0.0009) [2023-12-27 17:35:38,208][20347] Updated weights for policy 0, policy_version 1724662 (0.0007) [2023-12-27 17:35:38,257][20347] Updated weights for policy 0, policy_version 1724672 (0.0007) [2023-12-27 17:35:38,775][20381] Updated weights for policy 1, policy_version 1723161 (0.0009) [2023-12-27 17:35:38,847][20381] Updated weights for policy 1, policy_version 1723171 (0.0008) [2023-12-27 17:35:38,903][20381] Updated weights for policy 1, policy_version 1723181 (0.0007) [2023-12-27 17:35:38,933][20347] Updated weights for policy 0, policy_version 1724682 (0.0006) [2023-12-27 17:35:38,986][20347] Updated weights for policy 0, policy_version 1724692 (0.0007) [2023-12-27 17:35:39,044][20347] Updated weights for policy 0, policy_version 1724702 (0.0006) [2023-12-27 17:35:39,101][20347] Updated weights for policy 0, policy_version 1724712 (0.0009) [2023-12-27 17:35:39,591][20381] Updated weights for policy 1, policy_version 1723191 (0.0009) [2023-12-27 17:35:39,650][20381] Updated weights for policy 1, policy_version 1723201 (0.0006) [2023-12-27 17:35:39,719][20381] Updated weights for policy 1, policy_version 1723211 (0.0008) [2023-12-27 17:35:39,798][20347] Updated weights for policy 0, policy_version 1724722 (0.0007) [2023-12-27 17:35:39,862][20347] Updated weights for policy 0, policy_version 1724732 (0.0009) [2023-12-27 17:35:39,931][20347] Updated weights for policy 0, policy_version 1724742 (0.0007) [2023-12-27 17:35:40,354][20381] Updated weights for policy 1, policy_version 1723221 (0.0010) [2023-12-27 17:35:40,422][20381] Updated weights for policy 1, policy_version 1723231 (0.0009) [2023-12-27 17:35:40,472][20381] Updated weights for policy 1, policy_version 1723241 (0.0010) [2023-12-27 17:35:40,629][20347] Updated weights for policy 0, policy_version 1724752 (0.0007) [2023-12-27 17:35:40,681][20347] Updated weights for policy 0, policy_version 1724762 (0.0005) [2023-12-27 17:35:40,734][20347] Updated weights for policy 0, policy_version 1724772 (0.0006) [2023-12-27 17:35:41,163][20381] Updated weights for policy 1, policy_version 1723251 (0.0011) [2023-12-27 17:35:41,224][20381] Updated weights for policy 1, policy_version 1723261 (0.0011) [2023-12-27 17:35:41,286][20381] Updated weights for policy 1, policy_version 1723271 (0.0008) [2023-12-27 17:35:41,514][20347] Updated weights for policy 0, policy_version 1724782 (0.0007) [2023-12-27 17:35:41,581][20347] Updated weights for policy 0, policy_version 1724792 (0.0007) [2023-12-27 17:35:41,645][20347] Updated weights for policy 0, policy_version 1724802 (0.0010) [2023-12-27 17:35:42,040][20381] Updated weights for policy 1, policy_version 1723281 (0.0010) [2023-12-27 17:35:42,102][20381] Updated weights for policy 1, policy_version 1723291 (0.0011) [2023-12-27 17:35:42,160][20381] Updated weights for policy 1, policy_version 1723301 (0.0010) [2023-12-27 17:35:42,210][20381] Updated weights for policy 1, policy_version 1723311 (0.0008) [2023-12-27 17:35:42,293][20347] Updated weights for policy 0, policy_version 1724812 (0.0007) [2023-12-27 17:35:42,353][20347] Updated weights for policy 0, policy_version 1724822 (0.0008) [2023-12-27 17:35:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 882843648. Throughput: 0: 10315.7, 1: 10194.9. Samples: 882835828. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:42,363][19170] Avg episode reward: [(0, '970.710'), (1, '1000.000')] [2023-12-27 17:35:42,420][20347] Updated weights for policy 0, policy_version 1724832 (0.0010) [2023-12-27 17:35:42,882][20381] Updated weights for policy 1, policy_version 1723321 (0.0008) [2023-12-27 17:35:42,937][20381] Updated weights for policy 1, policy_version 1723331 (0.0010) [2023-12-27 17:35:42,987][20381] Updated weights for policy 1, policy_version 1723341 (0.0010) [2023-12-27 17:35:43,131][20347] Updated weights for policy 0, policy_version 1724842 (0.0011) [2023-12-27 17:35:43,187][20347] Updated weights for policy 0, policy_version 1724852 (0.0010) [2023-12-27 17:35:43,243][20347] Updated weights for policy 0, policy_version 1724862 (0.0010) [2023-12-27 17:35:43,291][20347] Updated weights for policy 0, policy_version 1724872 (0.0010) [2023-12-27 17:35:43,664][20381] Updated weights for policy 1, policy_version 1723351 (0.0010) [2023-12-27 17:35:43,715][20381] Updated weights for policy 1, policy_version 1723361 (0.0010) [2023-12-27 17:35:43,766][20381] Updated weights for policy 1, policy_version 1723371 (0.0010) [2023-12-27 17:35:43,988][20347] Updated weights for policy 0, policy_version 1724882 (0.0010) [2023-12-27 17:35:44,051][20347] Updated weights for policy 0, policy_version 1724892 (0.0010) [2023-12-27 17:35:44,110][20347] Updated weights for policy 0, policy_version 1724902 (0.0010) [2023-12-27 17:35:44,451][20381] Updated weights for policy 1, policy_version 1723381 (0.0011) [2023-12-27 17:35:44,503][20381] Updated weights for policy 1, policy_version 1723391 (0.0011) [2023-12-27 17:35:44,566][20381] Updated weights for policy 1, policy_version 1723401 (0.0011) [2023-12-27 17:35:44,795][20347] Updated weights for policy 0, policy_version 1724912 (0.0009) [2023-12-27 17:35:44,848][20347] Updated weights for policy 0, policy_version 1724922 (0.0011) [2023-12-27 17:35:44,909][20347] Updated weights for policy 0, policy_version 1724932 (0.0009) [2023-12-27 17:35:45,234][20381] Updated weights for policy 1, policy_version 1723411 (0.0010) [2023-12-27 17:35:45,296][20381] Updated weights for policy 1, policy_version 1723421 (0.0011) [2023-12-27 17:35:45,355][20381] Updated weights for policy 1, policy_version 1723431 (0.0011) [2023-12-27 17:35:45,510][20347] Updated weights for policy 0, policy_version 1724942 (0.0007) [2023-12-27 17:35:45,567][20347] Updated weights for policy 0, policy_version 1724952 (0.0005) [2023-12-27 17:35:45,627][20347] Updated weights for policy 0, policy_version 1724962 (0.0008) [2023-12-27 17:35:45,999][20381] Updated weights for policy 1, policy_version 1723441 (0.0010) [2023-12-27 17:35:46,058][20381] Updated weights for policy 1, policy_version 1723451 (0.0006) [2023-12-27 17:35:46,114][20381] Updated weights for policy 1, policy_version 1723461 (0.0006) [2023-12-27 17:35:46,172][20381] Updated weights for policy 1, policy_version 1723471 (0.0006) [2023-12-27 17:35:46,255][20347] Updated weights for policy 0, policy_version 1724972 (0.0008) [2023-12-27 17:35:46,306][20347] Updated weights for policy 0, policy_version 1724982 (0.0005) [2023-12-27 17:35:46,319][20291] KL-divergence is very high: 201.6255 [2023-12-27 17:35:46,367][20347] Updated weights for policy 0, policy_version 1724992 (0.0005) [2023-12-27 17:35:46,369][20291] KL-divergence is very high: 297.9226 [2023-12-27 17:35:46,797][20381] Updated weights for policy 1, policy_version 1723481 (0.0006) [2023-12-27 17:35:46,851][20381] Updated weights for policy 1, policy_version 1723491 (0.0007) [2023-12-27 17:35:46,910][20381] Updated weights for policy 1, policy_version 1723501 (0.0006) [2023-12-27 17:35:46,941][20347] Updated weights for policy 0, policy_version 1725002 (0.0006) [2023-12-27 17:35:47,001][20347] Updated weights for policy 0, policy_version 1725012 (0.0008) [2023-12-27 17:35:47,052][20347] Updated weights for policy 0, policy_version 1725022 (0.0007) [2023-12-27 17:35:47,106][20347] Updated weights for policy 0, policy_version 1725032 (0.0005) [2023-12-27 17:35:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 882950144. Throughput: 0: 10359.4, 1: 10238.3. Samples: 882960128. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:47,363][19170] Avg episode reward: [(0, '960.920'), (1, '1000.000')] [2023-12-27 17:35:47,530][20381] Updated weights for policy 1, policy_version 1723511 (0.0010) [2023-12-27 17:35:47,575][20381] Updated weights for policy 1, policy_version 1723521 (0.0010) [2023-12-27 17:35:47,620][20381] Updated weights for policy 1, policy_version 1723531 (0.0010) [2023-12-27 17:35:47,803][20347] Updated weights for policy 0, policy_version 1725042 (0.0009) [2023-12-27 17:35:47,856][20347] Updated weights for policy 0, policy_version 1725052 (0.0008) [2023-12-27 17:35:47,916][20347] Updated weights for policy 0, policy_version 1725062 (0.0008) [2023-12-27 17:35:48,303][20381] Updated weights for policy 1, policy_version 1723541 (0.0009) [2023-12-27 17:35:48,365][20381] Updated weights for policy 1, policy_version 1723551 (0.0008) [2023-12-27 17:35:48,428][20381] Updated weights for policy 1, policy_version 1723561 (0.0007) [2023-12-27 17:35:48,609][20347] Updated weights for policy 0, policy_version 1725072 (0.0008) [2023-12-27 17:35:48,671][20347] Updated weights for policy 0, policy_version 1725082 (0.0007) [2023-12-27 17:35:48,729][20347] Updated weights for policy 0, policy_version 1725092 (0.0008) [2023-12-27 17:35:49,107][20381] Updated weights for policy 1, policy_version 1723571 (0.0007) [2023-12-27 17:35:49,160][20381] Updated weights for policy 1, policy_version 1723581 (0.0005) [2023-12-27 17:35:49,211][20381] Updated weights for policy 1, policy_version 1723591 (0.0005) [2023-12-27 17:35:49,423][20347] Updated weights for policy 0, policy_version 1725102 (0.0007) [2023-12-27 17:35:49,480][20347] Updated weights for policy 0, policy_version 1725112 (0.0007) [2023-12-27 17:35:49,539][20347] Updated weights for policy 0, policy_version 1725122 (0.0006) [2023-12-27 17:35:49,870][20381] Updated weights for policy 1, policy_version 1723601 (0.0007) [2023-12-27 17:35:49,927][20381] Updated weights for policy 1, policy_version 1723611 (0.0008) [2023-12-27 17:35:49,991][20381] Updated weights for policy 1, policy_version 1723621 (0.0006) [2023-12-27 17:35:50,057][20381] Updated weights for policy 1, policy_version 1723631 (0.0007) [2023-12-27 17:35:50,218][20347] Updated weights for policy 0, policy_version 1725132 (0.0007) [2023-12-27 17:35:50,281][20347] Updated weights for policy 0, policy_version 1725142 (0.0006) [2023-12-27 17:35:50,342][20347] Updated weights for policy 0, policy_version 1725152 (0.0008) [2023-12-27 17:35:50,673][20381] Updated weights for policy 1, policy_version 1723641 (0.0008) [2023-12-27 17:35:50,726][20381] Updated weights for policy 1, policy_version 1723651 (0.0006) [2023-12-27 17:35:50,784][20381] Updated weights for policy 1, policy_version 1723661 (0.0005) [2023-12-27 17:35:51,019][20347] Updated weights for policy 0, policy_version 1725162 (0.0008) [2023-12-27 17:35:51,079][20347] Updated weights for policy 0, policy_version 1725172 (0.0008) [2023-12-27 17:35:51,141][20347] Updated weights for policy 0, policy_version 1725182 (0.0009) [2023-12-27 17:35:51,204][20347] Updated weights for policy 0, policy_version 1725192 (0.0008) [2023-12-27 17:35:51,511][20381] Updated weights for policy 1, policy_version 1723671 (0.0008) [2023-12-27 17:35:51,578][20381] Updated weights for policy 1, policy_version 1723681 (0.0008) [2023-12-27 17:35:51,647][20381] Updated weights for policy 1, policy_version 1723691 (0.0009) [2023-12-27 17:35:51,878][20347] Updated weights for policy 0, policy_version 1725202 (0.0007) [2023-12-27 17:35:51,941][20347] Updated weights for policy 0, policy_version 1725212 (0.0007) [2023-12-27 17:35:52,002][20347] Updated weights for policy 0, policy_version 1725222 (0.0008) [2023-12-27 17:35:52,353][20381] Updated weights for policy 1, policy_version 1723701 (0.0007) [2023-12-27 17:35:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 883048448. Throughput: 0: 10328.0, 1: 10235.2. Samples: 883021244. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:52,363][19170] Avg episode reward: [(0, '912.000'), (1, '1000.000')] [2023-12-27 17:35:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001725224_441737216.pth... [2023-12-27 17:35:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001724008_441425920.pth [2023-12-27 17:35:52,418][20381] Updated weights for policy 1, policy_version 1723711 (0.0008) [2023-12-27 17:35:52,481][20381] Updated weights for policy 1, policy_version 1723721 (0.0007) [2023-12-27 17:35:52,523][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001723728_441319424.pth... [2023-12-27 17:35:52,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001722512_441008128.pth [2023-12-27 17:35:52,739][20347] Updated weights for policy 0, policy_version 1725232 (0.0009) [2023-12-27 17:35:52,791][20347] Updated weights for policy 0, policy_version 1725242 (0.0008) [2023-12-27 17:35:52,849][20347] Updated weights for policy 0, policy_version 1725252 (0.0006) [2023-12-27 17:35:53,152][20381] Updated weights for policy 1, policy_version 1723731 (0.0007) [2023-12-27 17:35:53,201][20381] Updated weights for policy 1, policy_version 1723741 (0.0007) [2023-12-27 17:35:53,252][20381] Updated weights for policy 1, policy_version 1723751 (0.0008) [2023-12-27 17:35:53,560][20347] Updated weights for policy 0, policy_version 1725262 (0.0007) [2023-12-27 17:35:53,626][20347] Updated weights for policy 0, policy_version 1725272 (0.0007) [2023-12-27 17:35:53,680][20347] Updated weights for policy 0, policy_version 1725282 (0.0007) [2023-12-27 17:35:53,910][20381] Updated weights for policy 1, policy_version 1723761 (0.0009) [2023-12-27 17:35:53,965][20381] Updated weights for policy 1, policy_version 1723771 (0.0005) [2023-12-27 17:35:54,015][20381] Updated weights for policy 1, policy_version 1723781 (0.0008) [2023-12-27 17:35:54,068][20381] Updated weights for policy 1, policy_version 1723791 (0.0007) [2023-12-27 17:35:54,298][20347] Updated weights for policy 0, policy_version 1725292 (0.0009) [2023-12-27 17:35:54,347][20347] Updated weights for policy 0, policy_version 1725302 (0.0009) [2023-12-27 17:35:54,400][20347] Updated weights for policy 0, policy_version 1725313 (0.0008) [2023-12-27 17:35:54,616][20381] Updated weights for policy 1, policy_version 1723801 (0.0010) [2023-12-27 17:35:54,674][20381] Updated weights for policy 1, policy_version 1723811 (0.0011) [2023-12-27 17:35:54,725][20381] Updated weights for policy 1, policy_version 1723821 (0.0010) [2023-12-27 17:35:55,140][20347] Updated weights for policy 0, policy_version 1725323 (0.0008) [2023-12-27 17:35:55,199][20347] Updated weights for policy 0, policy_version 1725333 (0.0008) [2023-12-27 17:35:55,254][20347] Updated weights for policy 0, policy_version 1725343 (0.0007) [2023-12-27 17:35:55,420][20381] Updated weights for policy 1, policy_version 1723831 (0.0010) [2023-12-27 17:35:55,481][20381] Updated weights for policy 1, policy_version 1723841 (0.0010) [2023-12-27 17:35:55,541][20381] Updated weights for policy 1, policy_version 1723851 (0.0010) [2023-12-27 17:35:55,947][20347] Updated weights for policy 0, policy_version 1725353 (0.0007) [2023-12-27 17:35:55,994][20347] Updated weights for policy 0, policy_version 1725363 (0.0007) [2023-12-27 17:35:56,053][20347] Updated weights for policy 0, policy_version 1725373 (0.0008) [2023-12-27 17:35:56,105][20347] Updated weights for policy 0, policy_version 1725383 (0.0007) [2023-12-27 17:35:56,216][20381] Updated weights for policy 1, policy_version 1723861 (0.0010) [2023-12-27 17:35:56,274][20381] Updated weights for policy 1, policy_version 1723871 (0.0010) [2023-12-27 17:35:56,325][20381] Updated weights for policy 1, policy_version 1723881 (0.0010) [2023-12-27 17:35:56,820][20347] Updated weights for policy 0, policy_version 1725393 (0.0008) [2023-12-27 17:35:56,876][20347] Updated weights for policy 0, policy_version 1725403 (0.0007) [2023-12-27 17:35:56,926][20347] Updated weights for policy 0, policy_version 1725413 (0.0008) [2023-12-27 17:35:56,983][20381] Updated weights for policy 1, policy_version 1723891 (0.0010) [2023-12-27 17:35:57,038][20381] Updated weights for policy 1, policy_version 1723901 (0.0011) [2023-12-27 17:35:57,092][20381] Updated weights for policy 1, policy_version 1723911 (0.0010) [2023-12-27 17:35:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 883154944. Throughput: 0: 10280.7, 1: 10331.1. Samples: 883143952. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:35:57,363][19170] Avg episode reward: [(0, '912.020'), (1, '1000.000')] [2023-12-27 17:35:57,532][20347] Updated weights for policy 0, policy_version 1725423 (0.0006) [2023-12-27 17:35:57,590][20347] Updated weights for policy 0, policy_version 1725433 (0.0008) [2023-12-27 17:35:57,641][20347] Updated weights for policy 0, policy_version 1725443 (0.0007) [2023-12-27 17:35:57,737][20381] Updated weights for policy 1, policy_version 1723921 (0.0010) [2023-12-27 17:35:57,793][20381] Updated weights for policy 1, policy_version 1723931 (0.0011) [2023-12-27 17:35:57,841][20381] Updated weights for policy 1, policy_version 1723941 (0.0010) [2023-12-27 17:35:57,902][20381] Updated weights for policy 1, policy_version 1723951 (0.0010) [2023-12-27 17:35:58,352][20347] Updated weights for policy 0, policy_version 1725453 (0.0007) [2023-12-27 17:35:58,414][20347] Updated weights for policy 0, policy_version 1725463 (0.0007) [2023-12-27 17:35:58,472][20347] Updated weights for policy 0, policy_version 1725473 (0.0008) [2023-12-27 17:35:58,579][20381] Updated weights for policy 1, policy_version 1723961 (0.0008) [2023-12-27 17:35:58,639][20381] Updated weights for policy 1, policy_version 1723971 (0.0007) [2023-12-27 17:35:58,695][20381] Updated weights for policy 1, policy_version 1723981 (0.0007) [2023-12-27 17:35:59,248][20347] Updated weights for policy 0, policy_version 1725483 (0.0008) [2023-12-27 17:35:59,323][20347] Updated weights for policy 0, policy_version 1725493 (0.0008) [2023-12-27 17:35:59,383][20347] Updated weights for policy 0, policy_version 1725503 (0.0007) [2023-12-27 17:35:59,485][20381] Updated weights for policy 1, policy_version 1723991 (0.0008) [2023-12-27 17:35:59,543][20381] Updated weights for policy 1, policy_version 1724001 (0.0010) [2023-12-27 17:35:59,599][20381] Updated weights for policy 1, policy_version 1724011 (0.0010) [2023-12-27 17:35:59,981][20347] Updated weights for policy 0, policy_version 1725513 (0.0006) [2023-12-27 17:36:00,042][20347] Updated weights for policy 0, policy_version 1725523 (0.0008) [2023-12-27 17:36:00,096][20347] Updated weights for policy 0, policy_version 1725533 (0.0010) [2023-12-27 17:36:00,153][20347] Updated weights for policy 0, policy_version 1725543 (0.0010) [2023-12-27 17:36:00,254][20381] Updated weights for policy 1, policy_version 1724021 (0.0010) [2023-12-27 17:36:00,339][20381] Updated weights for policy 1, policy_version 1724031 (0.0010) [2023-12-27 17:36:00,389][20381] Updated weights for policy 1, policy_version 1724041 (0.0010) [2023-12-27 17:36:00,731][20347] Updated weights for policy 0, policy_version 1725553 (0.0006) [2023-12-27 17:36:00,786][20347] Updated weights for policy 0, policy_version 1725563 (0.0005) [2023-12-27 17:36:00,843][20347] Updated weights for policy 0, policy_version 1725573 (0.0010) [2023-12-27 17:36:00,994][20381] Updated weights for policy 1, policy_version 1724051 (0.0009) [2023-12-27 17:36:01,052][20381] Updated weights for policy 1, policy_version 1724061 (0.0007) [2023-12-27 17:36:01,103][20381] Updated weights for policy 1, policy_version 1724071 (0.0008) [2023-12-27 17:36:01,451][20347] Updated weights for policy 0, policy_version 1725583 (0.0007) [2023-12-27 17:36:01,513][20347] Updated weights for policy 0, policy_version 1725593 (0.0009) [2023-12-27 17:36:01,574][20347] Updated weights for policy 0, policy_version 1725603 (0.0007) [2023-12-27 17:36:01,783][20381] Updated weights for policy 1, policy_version 1724081 (0.0008) [2023-12-27 17:36:01,842][20381] Updated weights for policy 1, policy_version 1724091 (0.0008) [2023-12-27 17:36:01,897][20381] Updated weights for policy 1, policy_version 1724101 (0.0007) [2023-12-27 17:36:01,951][20381] Updated weights for policy 1, policy_version 1724111 (0.0005) [2023-12-27 17:36:02,314][20347] Updated weights for policy 0, policy_version 1725613 (0.0008) [2023-12-27 17:36:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 883253248. Throughput: 0: 10247.7, 1: 10333.6. Samples: 883267424. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:02,363][19170] Avg episode reward: [(0, '931.580'), (1, '1000.000')] [2023-12-27 17:36:02,371][20347] Updated weights for policy 0, policy_version 1725623 (0.0008) [2023-12-27 17:36:02,430][20347] Updated weights for policy 0, policy_version 1725633 (0.0008) [2023-12-27 17:36:02,555][20381] Updated weights for policy 1, policy_version 1724121 (0.0007) [2023-12-27 17:36:02,612][20381] Updated weights for policy 1, policy_version 1724131 (0.0007) [2023-12-27 17:36:02,664][20381] Updated weights for policy 1, policy_version 1724141 (0.0005) [2023-12-27 17:36:03,143][20347] Updated weights for policy 0, policy_version 1725643 (0.0007) [2023-12-27 17:36:03,199][20347] Updated weights for policy 0, policy_version 1725653 (0.0007) [2023-12-27 17:36:03,253][20381] Updated weights for policy 1, policy_version 1724151 (0.0005) [2023-12-27 17:36:03,254][20347] Updated weights for policy 0, policy_version 1725663 (0.0006) [2023-12-27 17:36:03,308][20381] Updated weights for policy 1, policy_version 1724161 (0.0005) [2023-12-27 17:36:03,357][20381] Updated weights for policy 1, policy_version 1724171 (0.0006) [2023-12-27 17:36:03,834][20347] Updated weights for policy 0, policy_version 1725673 (0.0007) [2023-12-27 17:36:03,896][20347] Updated weights for policy 0, policy_version 1725683 (0.0008) [2023-12-27 17:36:03,897][20381] Updated weights for policy 1, policy_version 1724181 (0.0006) [2023-12-27 17:36:03,946][20347] Updated weights for policy 0, policy_version 1725693 (0.0008) [2023-12-27 17:36:03,953][20381] Updated weights for policy 1, policy_version 1724191 (0.0006) [2023-12-27 17:36:03,997][20347] Updated weights for policy 0, policy_version 1725703 (0.0007) [2023-12-27 17:36:04,018][20381] Updated weights for policy 1, policy_version 1724201 (0.0007) [2023-12-27 17:36:04,644][20347] Updated weights for policy 0, policy_version 1725713 (0.0006) [2023-12-27 17:36:04,671][20381] Updated weights for policy 1, policy_version 1724211 (0.0009) [2023-12-27 17:36:04,698][20347] Updated weights for policy 0, policy_version 1725723 (0.0006) [2023-12-27 17:36:04,729][20381] Updated weights for policy 1, policy_version 1724221 (0.0010) [2023-12-27 17:36:04,749][20347] Updated weights for policy 0, policy_version 1725733 (0.0005) [2023-12-27 17:36:04,788][20381] Updated weights for policy 1, policy_version 1724231 (0.0011) [2023-12-27 17:36:05,339][20347] Updated weights for policy 0, policy_version 1725743 (0.0009) [2023-12-27 17:36:05,396][20347] Updated weights for policy 0, policy_version 1725753 (0.0010) [2023-12-27 17:36:05,450][20347] Updated weights for policy 0, policy_version 1725763 (0.0010) [2023-12-27 17:36:05,451][20381] Updated weights for policy 1, policy_version 1724241 (0.0010) [2023-12-27 17:36:05,508][20381] Updated weights for policy 1, policy_version 1724251 (0.0010) [2023-12-27 17:36:05,555][20381] Updated weights for policy 1, policy_version 1724261 (0.0010) [2023-12-27 17:36:05,603][20381] Updated weights for policy 1, policy_version 1724271 (0.0010) [2023-12-27 17:36:06,093][20347] Updated weights for policy 0, policy_version 1725773 (0.0010) [2023-12-27 17:36:06,157][20347] Updated weights for policy 0, policy_version 1725783 (0.0007) [2023-12-27 17:36:06,213][20347] Updated weights for policy 0, policy_version 1725793 (0.0007) [2023-12-27 17:36:06,307][20381] Updated weights for policy 1, policy_version 1724281 (0.0008) [2023-12-27 17:36:06,365][20381] Updated weights for policy 1, policy_version 1724291 (0.0008) [2023-12-27 17:36:06,416][20381] Updated weights for policy 1, policy_version 1724301 (0.0007) [2023-12-27 17:36:06,874][20347] Updated weights for policy 0, policy_version 1725803 (0.0008) [2023-12-27 17:36:06,935][20347] Updated weights for policy 0, policy_version 1725813 (0.0011) [2023-12-27 17:36:06,994][20347] Updated weights for policy 0, policy_version 1725823 (0.0010) [2023-12-27 17:36:07,076][20381] Updated weights for policy 1, policy_version 1724311 (0.0007) [2023-12-27 17:36:07,133][20381] Updated weights for policy 1, policy_version 1724321 (0.0005) [2023-12-27 17:36:07,195][20381] Updated weights for policy 1, policy_version 1724331 (0.0007) [2023-12-27 17:36:07,363][19170] Fps is (10 sec: 21298.5, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 883367936. Throughput: 0: 10300.7, 1: 10376.8. Samples: 883333240. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:07,364][19170] Avg episode reward: [(0, '931.550'), (1, '1000.000')] [2023-12-27 17:36:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001725832_441892864.pth... [2023-12-27 17:36:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001724336_441475072.pth... [2023-12-27 17:36:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001724616_441581568.pth [2023-12-27 17:36:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001723120_441163776.pth [2023-12-27 17:36:07,603][20347] Updated weights for policy 0, policy_version 1725833 (0.0010) [2023-12-27 17:36:07,653][20347] Updated weights for policy 0, policy_version 1725843 (0.0005) [2023-12-27 17:36:07,715][20347] Updated weights for policy 0, policy_version 1725853 (0.0007) [2023-12-27 17:36:07,777][20347] Updated weights for policy 0, policy_version 1725863 (0.0009) [2023-12-27 17:36:07,808][20381] Updated weights for policy 1, policy_version 1724341 (0.0010) [2023-12-27 17:36:07,860][20381] Updated weights for policy 1, policy_version 1724351 (0.0008) [2023-12-27 17:36:07,922][20381] Updated weights for policy 1, policy_version 1724361 (0.0006) [2023-12-27 17:36:08,396][20347] Updated weights for policy 0, policy_version 1725873 (0.0008) [2023-12-27 17:36:08,451][20347] Updated weights for policy 0, policy_version 1725883 (0.0007) [2023-12-27 17:36:08,509][20347] Updated weights for policy 0, policy_version 1725893 (0.0009) [2023-12-27 17:36:08,648][20381] Updated weights for policy 1, policy_version 1724371 (0.0007) [2023-12-27 17:36:08,696][20381] Updated weights for policy 1, policy_version 1724381 (0.0008) [2023-12-27 17:36:08,743][20381] Updated weights for policy 1, policy_version 1724391 (0.0007) [2023-12-27 17:36:09,192][20347] Updated weights for policy 0, policy_version 1725903 (0.0009) [2023-12-27 17:36:09,256][20347] Updated weights for policy 0, policy_version 1725913 (0.0008) [2023-12-27 17:36:09,314][20347] Updated weights for policy 0, policy_version 1725923 (0.0008) [2023-12-27 17:36:09,527][20381] Updated weights for policy 1, policy_version 1724401 (0.0008) [2023-12-27 17:36:09,593][20381] Updated weights for policy 1, policy_version 1724411 (0.0009) [2023-12-27 17:36:09,650][20381] Updated weights for policy 1, policy_version 1724421 (0.0009) [2023-12-27 17:36:09,702][20381] Updated weights for policy 1, policy_version 1724431 (0.0008) [2023-12-27 17:36:10,044][20347] Updated weights for policy 0, policy_version 1725933 (0.0007) [2023-12-27 17:36:10,104][20347] Updated weights for policy 0, policy_version 1725943 (0.0008) [2023-12-27 17:36:10,167][20347] Updated weights for policy 0, policy_version 1725953 (0.0007) [2023-12-27 17:36:10,442][20381] Updated weights for policy 1, policy_version 1724441 (0.0008) [2023-12-27 17:36:10,499][20381] Updated weights for policy 1, policy_version 1724451 (0.0008) [2023-12-27 17:36:10,552][20381] Updated weights for policy 1, policy_version 1724461 (0.0008) [2023-12-27 17:36:10,738][20347] Updated weights for policy 0, policy_version 1725963 (0.0006) [2023-12-27 17:36:10,789][20347] Updated weights for policy 0, policy_version 1725973 (0.0009) [2023-12-27 17:36:10,841][20347] Updated weights for policy 0, policy_version 1725983 (0.0008) [2023-12-27 17:36:11,298][20381] Updated weights for policy 1, policy_version 1724471 (0.0008) [2023-12-27 17:36:11,363][20381] Updated weights for policy 1, policy_version 1724481 (0.0009) [2023-12-27 17:36:11,431][20381] Updated weights for policy 1, policy_version 1724491 (0.0008) [2023-12-27 17:36:11,547][20347] Updated weights for policy 0, policy_version 1725993 (0.0009) [2023-12-27 17:36:11,600][20347] Updated weights for policy 0, policy_version 1726003 (0.0007) [2023-12-27 17:36:11,663][20347] Updated weights for policy 0, policy_version 1726013 (0.0006) [2023-12-27 17:36:11,717][20347] Updated weights for policy 0, policy_version 1726023 (0.0007) [2023-12-27 17:36:12,103][20381] Updated weights for policy 1, policy_version 1724501 (0.0008) [2023-12-27 17:36:12,150][20381] Updated weights for policy 1, policy_version 1724511 (0.0008) [2023-12-27 17:36:12,200][20381] Updated weights for policy 1, policy_version 1724521 (0.0008) [2023-12-27 17:36:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 883466240. Throughput: 0: 10360.6, 1: 10360.1. Samples: 883456460. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:12,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:36:12,485][20347] Updated weights for policy 0, policy_version 1726033 (0.0008) [2023-12-27 17:36:12,544][20347] Updated weights for policy 0, policy_version 1726043 (0.0007) [2023-12-27 17:36:12,599][20347] Updated weights for policy 0, policy_version 1726053 (0.0007) [2023-12-27 17:36:12,965][20381] Updated weights for policy 1, policy_version 1724531 (0.0007) [2023-12-27 17:36:13,023][20381] Updated weights for policy 1, policy_version 1724541 (0.0008) [2023-12-27 17:36:13,076][20381] Updated weights for policy 1, policy_version 1724551 (0.0008) [2023-12-27 17:36:13,209][20347] Updated weights for policy 0, policy_version 1726063 (0.0009) [2023-12-27 17:36:13,258][20347] Updated weights for policy 0, policy_version 1726073 (0.0008) [2023-12-27 17:36:13,306][20347] Updated weights for policy 0, policy_version 1726083 (0.0007) [2023-12-27 17:36:13,727][20381] Updated weights for policy 1, policy_version 1724561 (0.0007) [2023-12-27 17:36:13,786][20381] Updated weights for policy 1, policy_version 1724571 (0.0007) [2023-12-27 17:36:13,841][20381] Updated weights for policy 1, policy_version 1724581 (0.0010) [2023-12-27 17:36:13,902][20381] Updated weights for policy 1, policy_version 1724591 (0.0010) [2023-12-27 17:36:14,041][20347] Updated weights for policy 0, policy_version 1726093 (0.0008) [2023-12-27 17:36:14,091][20347] Updated weights for policy 0, policy_version 1726103 (0.0007) [2023-12-27 17:36:14,144][20347] Updated weights for policy 0, policy_version 1726113 (0.0008) [2023-12-27 17:36:14,522][20381] Updated weights for policy 1, policy_version 1724601 (0.0011) [2023-12-27 17:36:14,588][20381] Updated weights for policy 1, policy_version 1724611 (0.0011) [2023-12-27 17:36:14,648][20381] Updated weights for policy 1, policy_version 1724621 (0.0011) [2023-12-27 17:36:14,851][20347] Updated weights for policy 0, policy_version 1726123 (0.0006) [2023-12-27 17:36:14,918][20347] Updated weights for policy 0, policy_version 1726133 (0.0006) [2023-12-27 17:36:14,982][20347] Updated weights for policy 0, policy_version 1726143 (0.0007) [2023-12-27 17:36:15,338][20381] Updated weights for policy 1, policy_version 1724631 (0.0009) [2023-12-27 17:36:15,391][20381] Updated weights for policy 1, policy_version 1724641 (0.0008) [2023-12-27 17:36:15,443][20381] Updated weights for policy 1, policy_version 1724651 (0.0009) [2023-12-27 17:36:15,610][20347] Updated weights for policy 0, policy_version 1726153 (0.0008) [2023-12-27 17:36:15,664][20347] Updated weights for policy 0, policy_version 1726163 (0.0007) [2023-12-27 17:36:15,720][20347] Updated weights for policy 0, policy_version 1726173 (0.0009) [2023-12-27 17:36:15,786][20347] Updated weights for policy 0, policy_version 1726183 (0.0008) [2023-12-27 17:36:16,128][20381] Updated weights for policy 1, policy_version 1724661 (0.0007) [2023-12-27 17:36:16,179][20381] Updated weights for policy 1, policy_version 1724671 (0.0006) [2023-12-27 17:36:16,232][20381] Updated weights for policy 1, policy_version 1724681 (0.0007) [2023-12-27 17:36:16,438][20347] Updated weights for policy 0, policy_version 1726193 (0.0005) [2023-12-27 17:36:16,498][20347] Updated weights for policy 0, policy_version 1726203 (0.0006) [2023-12-27 17:36:16,555][20347] Updated weights for policy 0, policy_version 1726213 (0.0008) [2023-12-27 17:36:16,835][20381] Updated weights for policy 1, policy_version 1724691 (0.0006) [2023-12-27 17:36:16,897][20381] Updated weights for policy 1, policy_version 1724701 (0.0007) [2023-12-27 17:36:16,959][20381] Updated weights for policy 1, policy_version 1724711 (0.0007) [2023-12-27 17:36:17,203][20347] Updated weights for policy 0, policy_version 1726223 (0.0006) [2023-12-27 17:36:17,259][20347] Updated weights for policy 0, policy_version 1726233 (0.0005) [2023-12-27 17:36:17,305][20347] Updated weights for policy 0, policy_version 1726243 (0.0006) [2023-12-27 17:36:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 883572736. Throughput: 0: 10290.2, 1: 10374.5. Samples: 883578796. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:17,364][19170] Avg episode reward: [(0, '941.470'), (1, '1000.000')] [2023-12-27 17:36:17,632][20381] Updated weights for policy 1, policy_version 1724721 (0.0008) [2023-12-27 17:36:17,676][20381] Updated weights for policy 1, policy_version 1724731 (0.0008) [2023-12-27 17:36:17,722][20381] Updated weights for policy 1, policy_version 1724741 (0.0008) [2023-12-27 17:36:17,774][20381] Updated weights for policy 1, policy_version 1724751 (0.0009) [2023-12-27 17:36:17,855][20347] Updated weights for policy 0, policy_version 1726253 (0.0005) [2023-12-27 17:36:17,906][20347] Updated weights for policy 0, policy_version 1726263 (0.0006) [2023-12-27 17:36:17,948][20347] Updated weights for policy 0, policy_version 1726273 (0.0007) [2023-12-27 17:36:18,445][20381] Updated weights for policy 1, policy_version 1724761 (0.0006) [2023-12-27 17:36:18,505][20381] Updated weights for policy 1, policy_version 1724771 (0.0007) [2023-12-27 17:36:18,565][20381] Updated weights for policy 1, policy_version 1724781 (0.0008) [2023-12-27 17:36:18,620][20347] Updated weights for policy 0, policy_version 1726283 (0.0008) [2023-12-27 17:36:18,669][20347] Updated weights for policy 0, policy_version 1726293 (0.0007) [2023-12-27 17:36:18,717][20347] Updated weights for policy 0, policy_version 1726303 (0.0007) [2023-12-27 17:36:19,180][20381] Updated weights for policy 1, policy_version 1724791 (0.0007) [2023-12-27 17:36:19,250][20381] Updated weights for policy 1, policy_version 1724801 (0.0008) [2023-12-27 17:36:19,311][20381] Updated weights for policy 1, policy_version 1724811 (0.0008) [2023-12-27 17:36:19,498][20347] Updated weights for policy 0, policy_version 1726313 (0.0008) [2023-12-27 17:36:19,561][20347] Updated weights for policy 0, policy_version 1726323 (0.0007) [2023-12-27 17:36:19,622][20347] Updated weights for policy 0, policy_version 1726333 (0.0009) [2023-12-27 17:36:19,679][20347] Updated weights for policy 0, policy_version 1726343 (0.0008) [2023-12-27 17:36:19,975][20381] Updated weights for policy 1, policy_version 1724821 (0.0007) [2023-12-27 17:36:20,041][20381] Updated weights for policy 1, policy_version 1724831 (0.0008) [2023-12-27 17:36:20,101][20381] Updated weights for policy 1, policy_version 1724841 (0.0008) [2023-12-27 17:36:20,425][20347] Updated weights for policy 0, policy_version 1726353 (0.0010) [2023-12-27 17:36:20,486][20347] Updated weights for policy 0, policy_version 1726363 (0.0008) [2023-12-27 17:36:20,551][20347] Updated weights for policy 0, policy_version 1726373 (0.0011) [2023-12-27 17:36:20,821][20381] Updated weights for policy 1, policy_version 1724851 (0.0007) [2023-12-27 17:36:20,883][20381] Updated weights for policy 1, policy_version 1724861 (0.0007) [2023-12-27 17:36:20,943][20381] Updated weights for policy 1, policy_version 1724871 (0.0008) [2023-12-27 17:36:21,265][20347] Updated weights for policy 0, policy_version 1726383 (0.0010) [2023-12-27 17:36:21,329][20347] Updated weights for policy 0, policy_version 1726393 (0.0010) [2023-12-27 17:36:21,408][20347] Updated weights for policy 0, policy_version 1726403 (0.0010) [2023-12-27 17:36:21,645][20381] Updated weights for policy 1, policy_version 1724881 (0.0008) [2023-12-27 17:36:21,700][20381] Updated weights for policy 1, policy_version 1724891 (0.0008) [2023-12-27 17:36:21,780][20381] Updated weights for policy 1, policy_version 1724901 (0.0008) [2023-12-27 17:36:21,836][20381] Updated weights for policy 1, policy_version 1724911 (0.0007) [2023-12-27 17:36:22,152][20347] Updated weights for policy 0, policy_version 1726413 (0.0009) [2023-12-27 17:36:22,217][20347] Updated weights for policy 0, policy_version 1726423 (0.0007) [2023-12-27 17:36:22,274][20347] Updated weights for policy 0, policy_version 1726433 (0.0009) [2023-12-27 17:36:22,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 883671040. Throughput: 0: 10261.6, 1: 10349.4. Samples: 883641036. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:22,363][19170] Avg episode reward: [(0, '941.470'), (1, '1000.000')] [2023-12-27 17:36:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001726440_442048512.pth... [2023-12-27 17:36:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001724912_441622528.pth... [2023-12-27 17:36:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001723728_441319424.pth [2023-12-27 17:36:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001725224_441737216.pth [2023-12-27 17:36:22,550][20381] Updated weights for policy 1, policy_version 1724921 (0.0008) [2023-12-27 17:36:22,610][20381] Updated weights for policy 1, policy_version 1724931 (0.0006) [2023-12-27 17:36:22,664][20381] Updated weights for policy 1, policy_version 1724941 (0.0008) [2023-12-27 17:36:22,982][20347] Updated weights for policy 0, policy_version 1726443 (0.0009) [2023-12-27 17:36:23,038][20347] Updated weights for policy 0, policy_version 1726453 (0.0005) [2023-12-27 17:36:23,094][20347] Updated weights for policy 0, policy_version 1726463 (0.0005) [2023-12-27 17:36:23,375][20381] Updated weights for policy 1, policy_version 1724951 (0.0008) [2023-12-27 17:36:23,421][20381] Updated weights for policy 1, policy_version 1724961 (0.0008) [2023-12-27 17:36:23,466][20381] Updated weights for policy 1, policy_version 1724971 (0.0008) [2023-12-27 17:36:23,649][20347] Updated weights for policy 0, policy_version 1726473 (0.0007) [2023-12-27 17:36:23,704][20347] Updated weights for policy 0, policy_version 1726483 (0.0007) [2023-12-27 17:36:23,756][20347] Updated weights for policy 0, policy_version 1726493 (0.0007) [2023-12-27 17:36:23,810][20347] Updated weights for policy 0, policy_version 1726503 (0.0007) [2023-12-27 17:36:24,148][20381] Updated weights for policy 1, policy_version 1724981 (0.0009) [2023-12-27 17:36:24,201][20381] Updated weights for policy 1, policy_version 1724991 (0.0010) [2023-12-27 17:36:24,254][20381] Updated weights for policy 1, policy_version 1725001 (0.0008) [2023-12-27 17:36:24,468][20347] Updated weights for policy 0, policy_version 1726513 (0.0010) [2023-12-27 17:36:24,513][20347] Updated weights for policy 0, policy_version 1726523 (0.0007) [2023-12-27 17:36:24,565][20347] Updated weights for policy 0, policy_version 1726533 (0.0006) [2023-12-27 17:36:24,882][20381] Updated weights for policy 1, policy_version 1725011 (0.0008) [2023-12-27 17:36:24,935][20381] Updated weights for policy 1, policy_version 1725021 (0.0005) [2023-12-27 17:36:24,994][20381] Updated weights for policy 1, policy_version 1725031 (0.0007) [2023-12-27 17:36:25,192][20347] Updated weights for policy 0, policy_version 1726543 (0.0009) [2023-12-27 17:36:25,242][20347] Updated weights for policy 0, policy_version 1726553 (0.0010) [2023-12-27 17:36:25,290][20347] Updated weights for policy 0, policy_version 1726563 (0.0010) [2023-12-27 17:36:25,674][20381] Updated weights for policy 1, policy_version 1725041 (0.0007) [2023-12-27 17:36:25,735][20381] Updated weights for policy 1, policy_version 1725051 (0.0008) [2023-12-27 17:36:25,793][20381] Updated weights for policy 1, policy_version 1725061 (0.0007) [2023-12-27 17:36:25,845][20381] Updated weights for policy 1, policy_version 1725071 (0.0007) [2023-12-27 17:36:25,916][20347] Updated weights for policy 0, policy_version 1726573 (0.0010) [2023-12-27 17:36:25,971][20347] Updated weights for policy 0, policy_version 1726583 (0.0011) [2023-12-27 17:36:26,023][20347] Updated weights for policy 0, policy_version 1726593 (0.0010) [2023-12-27 17:36:26,513][20381] Updated weights for policy 1, policy_version 1725081 (0.0005) [2023-12-27 17:36:26,561][20381] Updated weights for policy 1, policy_version 1725091 (0.0005) [2023-12-27 17:36:26,613][20381] Updated weights for policy 1, policy_version 1725101 (0.0005) [2023-12-27 17:36:26,657][20347] Updated weights for policy 0, policy_version 1726603 (0.0008) [2023-12-27 17:36:26,704][20347] Updated weights for policy 0, policy_version 1726613 (0.0005) [2023-12-27 17:36:26,752][20347] Updated weights for policy 0, policy_version 1726623 (0.0007) [2023-12-27 17:36:27,181][20381] Updated weights for policy 1, policy_version 1725111 (0.0007) [2023-12-27 17:36:27,231][20381] Updated weights for policy 1, policy_version 1725121 (0.0008) [2023-12-27 17:36:27,295][20381] Updated weights for policy 1, policy_version 1725132 (0.0008) [2023-12-27 17:36:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 883777536. Throughput: 0: 10311.7, 1: 10310.5. Samples: 883763824. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:27,363][19170] Avg episode reward: [(0, '970.650'), (1, '1000.000')] [2023-12-27 17:36:27,418][20347] Updated weights for policy 0, policy_version 1726633 (0.0010) [2023-12-27 17:36:27,478][20347] Updated weights for policy 0, policy_version 1726643 (0.0010) [2023-12-27 17:36:27,545][20347] Updated weights for policy 0, policy_version 1726653 (0.0010) [2023-12-27 17:36:27,599][20347] Updated weights for policy 0, policy_version 1726663 (0.0010) [2023-12-27 17:36:27,902][20381] Updated weights for policy 1, policy_version 1725142 (0.0007) [2023-12-27 17:36:27,950][20381] Updated weights for policy 1, policy_version 1725152 (0.0007) [2023-12-27 17:36:28,003][20381] Updated weights for policy 1, policy_version 1725162 (0.0008) [2023-12-27 17:36:28,264][20347] Updated weights for policy 0, policy_version 1726673 (0.0010) [2023-12-27 17:36:28,309][20347] Updated weights for policy 0, policy_version 1726683 (0.0010) [2023-12-27 17:36:28,365][20347] Updated weights for policy 0, policy_version 1726693 (0.0008) [2023-12-27 17:36:28,653][20381] Updated weights for policy 1, policy_version 1725172 (0.0006) [2023-12-27 17:36:28,712][20381] Updated weights for policy 1, policy_version 1725182 (0.0008) [2023-12-27 17:36:28,768][20381] Updated weights for policy 1, policy_version 1725192 (0.0008) [2023-12-27 17:36:28,970][20347] Updated weights for policy 0, policy_version 1726703 (0.0009) [2023-12-27 17:36:29,029][20347] Updated weights for policy 0, policy_version 1726713 (0.0008) [2023-12-27 17:36:29,079][20347] Updated weights for policy 0, policy_version 1726723 (0.0008) [2023-12-27 17:36:29,528][20381] Updated weights for policy 1, policy_version 1725202 (0.0007) [2023-12-27 17:36:29,590][20381] Updated weights for policy 1, policy_version 1725212 (0.0007) [2023-12-27 17:36:29,641][20381] Updated weights for policy 1, policy_version 1725222 (0.0008) [2023-12-27 17:36:29,694][20381] Updated weights for policy 1, policy_version 1725232 (0.0007) [2023-12-27 17:36:29,713][20347] Updated weights for policy 0, policy_version 1726733 (0.0010) [2023-12-27 17:36:29,771][20347] Updated weights for policy 0, policy_version 1726743 (0.0008) [2023-12-27 17:36:29,834][20347] Updated weights for policy 0, policy_version 1726753 (0.0006) [2023-12-27 17:36:30,404][20347] Updated weights for policy 0, policy_version 1726763 (0.0006) [2023-12-27 17:36:30,437][20381] Updated weights for policy 1, policy_version 1725242 (0.0007) [2023-12-27 17:36:30,458][20347] Updated weights for policy 0, policy_version 1726773 (0.0007) [2023-12-27 17:36:30,500][20381] Updated weights for policy 1, policy_version 1725252 (0.0006) [2023-12-27 17:36:30,502][20347] Updated weights for policy 0, policy_version 1726783 (0.0007) [2023-12-27 17:36:30,551][20381] Updated weights for policy 1, policy_version 1725262 (0.0006) [2023-12-27 17:36:31,224][20347] Updated weights for policy 0, policy_version 1726793 (0.0006) [2023-12-27 17:36:31,250][20381] Updated weights for policy 1, policy_version 1725272 (0.0007) [2023-12-27 17:36:31,285][20347] Updated weights for policy 0, policy_version 1726803 (0.0008) [2023-12-27 17:36:31,307][20381] Updated weights for policy 1, policy_version 1725282 (0.0007) [2023-12-27 17:36:31,340][20347] Updated weights for policy 0, policy_version 1726813 (0.0008) [2023-12-27 17:36:31,372][20381] Updated weights for policy 1, policy_version 1725292 (0.0007) [2023-12-27 17:36:31,408][20347] Updated weights for policy 0, policy_version 1726823 (0.0008) [2023-12-27 17:36:32,043][20381] Updated weights for policy 1, policy_version 1725302 (0.0007) [2023-12-27 17:36:32,102][20381] Updated weights for policy 1, policy_version 1725312 (0.0006) [2023-12-27 17:36:32,108][20347] Updated weights for policy 0, policy_version 1726833 (0.0007) [2023-12-27 17:36:32,155][20381] Updated weights for policy 1, policy_version 1725322 (0.0006) [2023-12-27 17:36:32,167][20347] Updated weights for policy 0, policy_version 1726843 (0.0006) [2023-12-27 17:36:32,225][20347] Updated weights for policy 0, policy_version 1726853 (0.0008) [2023-12-27 17:36:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 883884032. Throughput: 0: 10328.0, 1: 10319.9. Samples: 883889284. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:32,363][19170] Avg episode reward: [(0, '960.780'), (1, '1000.000')] [2023-12-27 17:36:32,832][20381] Updated weights for policy 1, policy_version 1725332 (0.0007) [2023-12-27 17:36:32,849][20347] Updated weights for policy 0, policy_version 1726863 (0.0008) [2023-12-27 17:36:32,896][20381] Updated weights for policy 1, policy_version 1725342 (0.0006) [2023-12-27 17:36:32,901][20347] Updated weights for policy 0, policy_version 1726873 (0.0007) [2023-12-27 17:36:32,955][20347] Updated weights for policy 0, policy_version 1726883 (0.0007) [2023-12-27 17:36:32,957][20381] Updated weights for policy 1, policy_version 1725352 (0.0007) [2023-12-27 17:36:33,586][20347] Updated weights for policy 0, policy_version 1726893 (0.0007) [2023-12-27 17:36:33,639][20347] Updated weights for policy 0, policy_version 1726903 (0.0007) [2023-12-27 17:36:33,645][20381] Updated weights for policy 1, policy_version 1725362 (0.0007) [2023-12-27 17:36:33,694][20347] Updated weights for policy 0, policy_version 1726913 (0.0006) [2023-12-27 17:36:33,699][20381] Updated weights for policy 1, policy_version 1725372 (0.0006) [2023-12-27 17:36:33,764][20381] Updated weights for policy 1, policy_version 1725382 (0.0007) [2023-12-27 17:36:33,824][20381] Updated weights for policy 1, policy_version 1725392 (0.0007) [2023-12-27 17:36:34,351][20347] Updated weights for policy 0, policy_version 1726923 (0.0006) [2023-12-27 17:36:34,406][20347] Updated weights for policy 0, policy_version 1726933 (0.0008) [2023-12-27 17:36:34,467][20347] Updated weights for policy 0, policy_version 1726943 (0.0008) [2023-12-27 17:36:34,540][20381] Updated weights for policy 1, policy_version 1725402 (0.0006) [2023-12-27 17:36:34,602][20381] Updated weights for policy 1, policy_version 1725412 (0.0006) [2023-12-27 17:36:34,662][20381] Updated weights for policy 1, policy_version 1725422 (0.0007) [2023-12-27 17:36:35,187][20347] Updated weights for policy 0, policy_version 1726953 (0.0006) [2023-12-27 17:36:35,247][20347] Updated weights for policy 0, policy_version 1726963 (0.0005) [2023-12-27 17:36:35,303][20347] Updated weights for policy 0, policy_version 1726973 (0.0006) [2023-12-27 17:36:35,313][20381] Updated weights for policy 1, policy_version 1725432 (0.0007) [2023-12-27 17:36:35,355][20347] Updated weights for policy 0, policy_version 1726983 (0.0005) [2023-12-27 17:36:35,373][20381] Updated weights for policy 1, policy_version 1725442 (0.0006) [2023-12-27 17:36:35,430][20381] Updated weights for policy 1, policy_version 1725452 (0.0007) [2023-12-27 17:36:36,010][20347] Updated weights for policy 0, policy_version 1726993 (0.0010) [2023-12-27 17:36:36,061][20347] Updated weights for policy 0, policy_version 1727003 (0.0010) [2023-12-27 17:36:36,117][20381] Updated weights for policy 1, policy_version 1725462 (0.0007) [2023-12-27 17:36:36,125][20347] Updated weights for policy 0, policy_version 1727013 (0.0010) [2023-12-27 17:36:36,180][20381] Updated weights for policy 1, policy_version 1725472 (0.0007) [2023-12-27 17:36:36,243][20381] Updated weights for policy 1, policy_version 1725482 (0.0008) [2023-12-27 17:36:36,825][20347] Updated weights for policy 0, policy_version 1727023 (0.0010) [2023-12-27 17:36:36,886][20347] Updated weights for policy 0, policy_version 1727033 (0.0010) [2023-12-27 17:36:36,931][20381] Updated weights for policy 1, policy_version 1725492 (0.0008) [2023-12-27 17:36:36,951][20347] Updated weights for policy 0, policy_version 1727043 (0.0010) [2023-12-27 17:36:36,991][20381] Updated weights for policy 1, policy_version 1725502 (0.0006) [2023-12-27 17:36:37,045][20381] Updated weights for policy 1, policy_version 1725512 (0.0007) [2023-12-27 17:36:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 883982336. Throughput: 0: 10377.8, 1: 10289.9. Samples: 883951288. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) [2023-12-27 17:36:37,363][19170] Avg episode reward: [(0, '960.780'), (1, '1000.000')] [2023-12-27 17:36:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001725520_441778176.pth... [2023-12-27 17:36:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001727048_442204160.pth... [2023-12-27 17:36:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001724336_441475072.pth [2023-12-27 17:36:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001725832_441892864.pth [2023-12-27 17:36:37,630][20347] Updated weights for policy 0, policy_version 1727053 (0.0011) [2023-12-27 17:36:37,682][20347] Updated weights for policy 0, policy_version 1727063 (0.0010) [2023-12-27 17:36:37,733][20347] Updated weights for policy 0, policy_version 1727073 (0.0010) [2023-12-27 17:36:37,739][20381] Updated weights for policy 1, policy_version 1725522 (0.0007) [2023-12-27 17:36:37,800][20381] Updated weights for policy 1, policy_version 1725532 (0.0011) [2023-12-27 17:36:37,859][20381] Updated weights for policy 1, policy_version 1725542 (0.0005) [2023-12-27 17:36:37,915][20381] Updated weights for policy 1, policy_version 1725552 (0.0009) [2023-12-27 17:36:38,437][20347] Updated weights for policy 0, policy_version 1727083 (0.0009) [2023-12-27 17:36:38,497][20347] Updated weights for policy 0, policy_version 1727093 (0.0008) [2023-12-27 17:36:38,559][20347] Updated weights for policy 0, policy_version 1727103 (0.0007) [2023-12-27 17:36:38,560][20381] Updated weights for policy 1, policy_version 1725562 (0.0010) [2023-12-27 17:36:38,613][20381] Updated weights for policy 1, policy_version 1725572 (0.0010) [2023-12-27 17:36:38,662][20381] Updated weights for policy 1, policy_version 1725582 (0.0010) [2023-12-27 17:36:39,267][20347] Updated weights for policy 0, policy_version 1727113 (0.0008) [2023-12-27 17:36:39,327][20347] Updated weights for policy 0, policy_version 1727123 (0.0011) [2023-12-27 17:36:39,378][20381] Updated weights for policy 1, policy_version 1725592 (0.0008) [2023-12-27 17:36:39,392][20347] Updated weights for policy 0, policy_version 1727133 (0.0009) [2023-12-27 17:36:39,442][20381] Updated weights for policy 1, policy_version 1725602 (0.0008) [2023-12-27 17:36:39,452][20347] Updated weights for policy 0, policy_version 1727143 (0.0008) [2023-12-27 17:36:39,501][20381] Updated weights for policy 1, policy_version 1725612 (0.0007) [2023-12-27 17:36:40,200][20347] Updated weights for policy 0, policy_version 1727153 (0.0008) [2023-12-27 17:36:40,259][20347] Updated weights for policy 0, policy_version 1727163 (0.0007) [2023-12-27 17:36:40,269][20381] Updated weights for policy 1, policy_version 1725622 (0.0008) [2023-12-27 17:36:40,313][20347] Updated weights for policy 0, policy_version 1727173 (0.0006) [2023-12-27 17:36:40,329][20381] Updated weights for policy 1, policy_version 1725632 (0.0007) [2023-12-27 17:36:40,387][20381] Updated weights for policy 1, policy_version 1725642 (0.0007) [2023-12-27 17:36:41,023][20381] Updated weights for policy 1, policy_version 1725652 (0.0007) [2023-12-27 17:36:41,060][20347] Updated weights for policy 0, policy_version 1727183 (0.0008) [2023-12-27 17:36:41,088][20381] Updated weights for policy 1, policy_version 1725662 (0.0007) [2023-12-27 17:36:41,120][20347] Updated weights for policy 0, policy_version 1727193 (0.0007) [2023-12-27 17:36:41,163][20381] Updated weights for policy 1, policy_version 1725672 (0.0007) [2023-12-27 17:36:41,184][20347] Updated weights for policy 0, policy_version 1727203 (0.0007) [2023-12-27 17:36:41,896][20347] Updated weights for policy 0, policy_version 1727213 (0.0007) [2023-12-27 17:36:41,899][20381] Updated weights for policy 1, policy_version 1725682 (0.0007) [2023-12-27 17:36:41,949][20347] Updated weights for policy 0, policy_version 1727223 (0.0008) [2023-12-27 17:36:41,962][20381] Updated weights for policy 1, policy_version 1725692 (0.0005) [2023-12-27 17:36:41,985][20291] KL-divergence is very high: 109.7627 [2023-12-27 17:36:42,009][20347] Updated weights for policy 0, policy_version 1727233 (0.0009) [2023-12-27 17:36:42,017][20381] Updated weights for policy 1, policy_version 1725702 (0.0006) [2023-12-27 17:36:42,075][20381] Updated weights for policy 1, policy_version 1725712 (0.0005) [2023-12-27 17:36:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 884080640. Throughput: 0: 10358.2, 1: 10207.9. Samples: 884069428. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:36:42,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:36:42,688][20381] Updated weights for policy 1, policy_version 1725722 (0.0011) [2023-12-27 17:36:42,742][20347] Updated weights for policy 0, policy_version 1727243 (0.0008) [2023-12-27 17:36:42,742][20381] Updated weights for policy 1, policy_version 1725732 (0.0010) [2023-12-27 17:36:42,792][20347] Updated weights for policy 0, policy_version 1727253 (0.0006) [2023-12-27 17:36:42,792][20381] Updated weights for policy 1, policy_version 1725742 (0.0011) [2023-12-27 17:36:42,842][20347] Updated weights for policy 0, policy_version 1727263 (0.0006) [2023-12-27 17:36:43,405][20347] Updated weights for policy 0, policy_version 1727273 (0.0005) [2023-12-27 17:36:43,461][20347] Updated weights for policy 0, policy_version 1727283 (0.0006) [2023-12-27 17:36:43,488][20381] Updated weights for policy 1, policy_version 1725752 (0.0008) [2023-12-27 17:36:43,522][20347] Updated weights for policy 0, policy_version 1727293 (0.0005) [2023-12-27 17:36:43,543][20381] Updated weights for policy 1, policy_version 1725762 (0.0007) [2023-12-27 17:36:43,580][20347] Updated weights for policy 0, policy_version 1727303 (0.0005) [2023-12-27 17:36:43,599][20381] Updated weights for policy 1, policy_version 1725772 (0.0007) [2023-12-27 17:36:44,078][20347] Updated weights for policy 0, policy_version 1727313 (0.0007) [2023-12-27 17:36:44,129][20347] Updated weights for policy 0, policy_version 1727323 (0.0007) [2023-12-27 17:36:44,180][20347] Updated weights for policy 0, policy_version 1727333 (0.0007) [2023-12-27 17:36:44,323][20381] Updated weights for policy 1, policy_version 1725782 (0.0007) [2023-12-27 17:36:44,377][20381] Updated weights for policy 1, policy_version 1725792 (0.0006) [2023-12-27 17:36:44,427][20381] Updated weights for policy 1, policy_version 1725802 (0.0007) [2023-12-27 17:36:44,929][20347] Updated weights for policy 0, policy_version 1727343 (0.0008) [2023-12-27 17:36:44,988][20291] KL-divergence is very high: 102.3913 [2023-12-27 17:36:44,994][20347] Updated weights for policy 0, policy_version 1727353 (0.0008) [2023-12-27 17:36:45,041][20291] KL-divergence is very high: 136.2764 [2023-12-27 17:36:45,062][20347] Updated weights for policy 0, policy_version 1727363 (0.0009) [2023-12-27 17:36:45,123][20381] Updated weights for policy 1, policy_version 1725812 (0.0006) [2023-12-27 17:36:45,183][20381] Updated weights for policy 1, policy_version 1725822 (0.0007) [2023-12-27 17:36:45,239][20381] Updated weights for policy 1, policy_version 1725832 (0.0005) [2023-12-27 17:36:45,776][20347] Updated weights for policy 0, policy_version 1727373 (0.0008) [2023-12-27 17:36:45,822][20347] Updated weights for policy 0, policy_version 1727383 (0.0007) [2023-12-27 17:36:45,848][20381] Updated weights for policy 1, policy_version 1725842 (0.0005) [2023-12-27 17:36:45,859][20291] KL-divergence is very high: 136.6114 [2023-12-27 17:36:45,877][20347] Updated weights for policy 0, policy_version 1727393 (0.0006) [2023-12-27 17:36:45,902][20381] Updated weights for policy 1, policy_version 1725852 (0.0006) [2023-12-27 17:36:45,905][20291] KL-divergence is very high: 103.2422 [2023-12-27 17:36:45,952][20381] Updated weights for policy 1, policy_version 1725862 (0.0007) [2023-12-27 17:36:46,017][20381] Updated weights for policy 1, policy_version 1725872 (0.0006) [2023-12-27 17:36:46,519][20347] Updated weights for policy 0, policy_version 1727403 (0.0006) [2023-12-27 17:36:46,580][20347] Updated weights for policy 0, policy_version 1727413 (0.0007) [2023-12-27 17:36:46,584][20381] Updated weights for policy 1, policy_version 1725882 (0.0008) [2023-12-27 17:36:46,635][20347] Updated weights for policy 0, policy_version 1727423 (0.0006) [2023-12-27 17:36:46,641][20381] Updated weights for policy 1, policy_version 1725892 (0.0007) [2023-12-27 17:36:46,690][20381] Updated weights for policy 1, policy_version 1725902 (0.0008) [2023-12-27 17:36:47,185][20347] Updated weights for policy 0, policy_version 1727433 (0.0006) [2023-12-27 17:36:47,236][20347] Updated weights for policy 0, policy_version 1727443 (0.0007) [2023-12-27 17:36:47,284][20347] Updated weights for policy 0, policy_version 1727453 (0.0006) [2023-12-27 17:36:47,332][20347] Updated weights for policy 0, policy_version 1727463 (0.0007) [2023-12-27 17:36:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 884187136. Throughput: 0: 10392.6, 1: 10221.2. Samples: 884195044. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:36:47,363][19170] Avg episode reward: [(0, '921.770'), (1, '1000.000')] [2023-12-27 17:36:47,372][20381] Updated weights for policy 1, policy_version 1725912 (0.0006) [2023-12-27 17:36:47,430][20381] Updated weights for policy 1, policy_version 1725922 (0.0006) [2023-12-27 17:36:47,483][20381] Updated weights for policy 1, policy_version 1725932 (0.0007) [2023-12-27 17:36:47,948][20347] Updated weights for policy 0, policy_version 1727473 (0.0008) [2023-12-27 17:36:48,003][20347] Updated weights for policy 0, policy_version 1727483 (0.0008) [2023-12-27 17:36:48,059][20347] Updated weights for policy 0, policy_version 1727493 (0.0008) [2023-12-27 17:36:48,164][20381] Updated weights for policy 1, policy_version 1725942 (0.0006) [2023-12-27 17:36:48,217][20381] Updated weights for policy 1, policy_version 1725952 (0.0005) [2023-12-27 17:36:48,275][20381] Updated weights for policy 1, policy_version 1725962 (0.0007) [2023-12-27 17:36:48,776][20347] Updated weights for policy 0, policy_version 1727503 (0.0008) [2023-12-27 17:36:48,837][20347] Updated weights for policy 0, policy_version 1727513 (0.0007) [2023-12-27 17:36:48,850][20381] Updated weights for policy 1, policy_version 1725972 (0.0008) [2023-12-27 17:36:48,890][20347] Updated weights for policy 0, policy_version 1727523 (0.0005) [2023-12-27 17:36:48,904][20381] Updated weights for policy 1, policy_version 1725982 (0.0006) [2023-12-27 17:36:48,962][20381] Updated weights for policy 1, policy_version 1725992 (0.0007) [2023-12-27 17:36:49,602][20347] Updated weights for policy 0, policy_version 1727533 (0.0007) [2023-12-27 17:36:49,658][20347] Updated weights for policy 0, policy_version 1727543 (0.0007) [2023-12-27 17:36:49,688][20381] Updated weights for policy 1, policy_version 1726002 (0.0007) [2023-12-27 17:36:49,713][20347] Updated weights for policy 0, policy_version 1727553 (0.0007) [2023-12-27 17:36:49,742][20381] Updated weights for policy 1, policy_version 1726012 (0.0006) [2023-12-27 17:36:49,800][20381] Updated weights for policy 1, policy_version 1726022 (0.0007) [2023-12-27 17:36:49,866][20381] Updated weights for policy 1, policy_version 1726032 (0.0008) [2023-12-27 17:36:50,434][20347] Updated weights for policy 0, policy_version 1727563 (0.0005) [2023-12-27 17:36:50,485][20347] Updated weights for policy 0, policy_version 1727573 (0.0005) [2023-12-27 17:36:50,540][20347] Updated weights for policy 0, policy_version 1727583 (0.0005) [2023-12-27 17:36:50,639][20381] Updated weights for policy 1, policy_version 1726042 (0.0007) [2023-12-27 17:36:50,691][20381] Updated weights for policy 1, policy_version 1726052 (0.0007) [2023-12-27 17:36:50,747][20381] Updated weights for policy 1, policy_version 1726062 (0.0006) [2023-12-27 17:36:51,263][20347] Updated weights for policy 0, policy_version 1727593 (0.0008) [2023-12-27 17:36:51,320][20347] Updated weights for policy 0, policy_version 1727603 (0.0006) [2023-12-27 17:36:51,383][20347] Updated weights for policy 0, policy_version 1727613 (0.0007) [2023-12-27 17:36:51,450][20347] Updated weights for policy 0, policy_version 1727623 (0.0008) [2023-12-27 17:36:51,450][20381] Updated weights for policy 1, policy_version 1726072 (0.0008) [2023-12-27 17:36:51,516][20381] Updated weights for policy 1, policy_version 1726082 (0.0007) [2023-12-27 17:36:51,582][20381] Updated weights for policy 1, policy_version 1726092 (0.0006) [2023-12-27 17:36:52,191][20347] Updated weights for policy 0, policy_version 1727633 (0.0007) [2023-12-27 17:36:52,215][20381] Updated weights for policy 1, policy_version 1726102 (0.0007) [2023-12-27 17:36:52,251][20347] Updated weights for policy 0, policy_version 1727643 (0.0007) [2023-12-27 17:36:52,268][20381] Updated weights for policy 1, policy_version 1726112 (0.0006) [2023-12-27 17:36:52,312][20347] Updated weights for policy 0, policy_version 1727653 (0.0006) [2023-12-27 17:36:52,328][20381] Updated weights for policy 1, policy_version 1726122 (0.0007) [2023-12-27 17:36:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 884285440. Throughput: 0: 10347.9, 1: 10169.4. Samples: 884256512. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:36:52,363][19170] Avg episode reward: [(0, '931.560'), (1, '1000.000')] [2023-12-27 17:36:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001726128_441933824.pth... [2023-12-27 17:36:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001727656_442359808.pth... [2023-12-27 17:36:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001724912_441622528.pth [2023-12-27 17:36:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001726440_442048512.pth [2023-12-27 17:36:52,941][20381] Updated weights for policy 1, policy_version 1726132 (0.0007) [2023-12-27 17:36:52,998][20381] Updated weights for policy 1, policy_version 1726142 (0.0006) [2023-12-27 17:36:53,058][20381] Updated weights for policy 1, policy_version 1726152 (0.0006) [2023-12-27 17:36:53,064][20347] Updated weights for policy 0, policy_version 1727663 (0.0007) [2023-12-27 17:36:53,123][20347] Updated weights for policy 0, policy_version 1727673 (0.0006) [2023-12-27 17:36:53,191][20347] Updated weights for policy 0, policy_version 1727683 (0.0007) [2023-12-27 17:36:53,624][20381] Updated weights for policy 1, policy_version 1726162 (0.0007) [2023-12-27 17:36:53,674][20381] Updated weights for policy 1, policy_version 1726172 (0.0008) [2023-12-27 17:36:53,719][20381] Updated weights for policy 1, policy_version 1726182 (0.0007) [2023-12-27 17:36:53,762][20381] Updated weights for policy 1, policy_version 1726192 (0.0008) [2023-12-27 17:36:53,814][20347] Updated weights for policy 0, policy_version 1727693 (0.0007) [2023-12-27 17:36:53,858][20347] Updated weights for policy 0, policy_version 1727703 (0.0007) [2023-12-27 17:36:53,911][20347] Updated weights for policy 0, policy_version 1727713 (0.0006) [2023-12-27 17:36:54,516][20381] Updated weights for policy 1, policy_version 1726202 (0.0008) [2023-12-27 17:36:54,526][20347] Updated weights for policy 0, policy_version 1727723 (0.0007) [2023-12-27 17:36:54,568][20381] Updated weights for policy 1, policy_version 1726212 (0.0006) [2023-12-27 17:36:54,583][20347] Updated weights for policy 0, policy_version 1727733 (0.0007) [2023-12-27 17:36:54,627][20381] Updated weights for policy 1, policy_version 1726222 (0.0006) [2023-12-27 17:36:54,629][20347] Updated weights for policy 0, policy_version 1727743 (0.0006) [2023-12-27 17:36:55,266][20347] Updated weights for policy 0, policy_version 1727753 (0.0008) [2023-12-27 17:36:55,299][20381] Updated weights for policy 1, policy_version 1726232 (0.0007) [2023-12-27 17:36:55,313][20347] Updated weights for policy 0, policy_version 1727763 (0.0007) [2023-12-27 17:36:55,348][20381] Updated weights for policy 1, policy_version 1726242 (0.0006) [2023-12-27 17:36:55,365][20347] Updated weights for policy 0, policy_version 1727773 (0.0005) [2023-12-27 17:36:55,399][20381] Updated weights for policy 1, policy_version 1726252 (0.0006) [2023-12-27 17:36:55,416][20347] Updated weights for policy 0, policy_version 1727783 (0.0005) [2023-12-27 17:36:56,064][20347] Updated weights for policy 0, policy_version 1727793 (0.0006) [2023-12-27 17:36:56,097][20381] Updated weights for policy 1, policy_version 1726262 (0.0006) [2023-12-27 17:36:56,121][20347] Updated weights for policy 0, policy_version 1727803 (0.0006) [2023-12-27 17:36:56,145][20381] Updated weights for policy 1, policy_version 1726272 (0.0010) [2023-12-27 17:36:56,171][20347] Updated weights for policy 0, policy_version 1727813 (0.0005) [2023-12-27 17:36:56,205][20381] Updated weights for policy 1, policy_version 1726282 (0.0010) [2023-12-27 17:36:56,813][20381] Updated weights for policy 1, policy_version 1726292 (0.0008) [2023-12-27 17:36:56,837][20347] Updated weights for policy 0, policy_version 1727823 (0.0007) [2023-12-27 17:36:56,861][20381] Updated weights for policy 1, policy_version 1726302 (0.0007) [2023-12-27 17:36:56,889][20347] Updated weights for policy 0, policy_version 1727833 (0.0006) [2023-12-27 17:36:56,914][20381] Updated weights for policy 1, policy_version 1726312 (0.0007) [2023-12-27 17:36:56,943][20347] Updated weights for policy 0, policy_version 1727843 (0.0007) [2023-12-27 17:36:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 884391936. Throughput: 0: 10323.1, 1: 10219.7. Samples: 884380888. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:36:57,363][19170] Avg episode reward: [(0, '951.080'), (1, '1000.000')] [2023-12-27 17:36:57,573][20347] Updated weights for policy 0, policy_version 1727853 (0.0008) [2023-12-27 17:36:57,575][20381] Updated weights for policy 1, policy_version 1726322 (0.0008) [2023-12-27 17:36:57,626][20347] Updated weights for policy 0, policy_version 1727863 (0.0008) [2023-12-27 17:36:57,633][20381] Updated weights for policy 1, policy_version 1726332 (0.0007) [2023-12-27 17:36:57,684][20347] Updated weights for policy 0, policy_version 1727873 (0.0010) [2023-12-27 17:36:57,693][20381] Updated weights for policy 1, policy_version 1726342 (0.0006) [2023-12-27 17:36:57,741][20381] Updated weights for policy 1, policy_version 1726352 (0.0006) [2023-12-27 17:36:58,328][20347] Updated weights for policy 0, policy_version 1727883 (0.0010) [2023-12-27 17:36:58,400][20347] Updated weights for policy 0, policy_version 1727893 (0.0009) [2023-12-27 17:36:58,459][20347] Updated weights for policy 0, policy_version 1727903 (0.0008) [2023-12-27 17:36:58,516][20381] Updated weights for policy 1, policy_version 1726362 (0.0007) [2023-12-27 17:36:58,576][20381] Updated weights for policy 1, policy_version 1726372 (0.0008) [2023-12-27 17:36:58,641][20381] Updated weights for policy 1, policy_version 1726382 (0.0007) [2023-12-27 17:36:59,283][20347] Updated weights for policy 0, policy_version 1727913 (0.0007) [2023-12-27 17:36:59,346][20347] Updated weights for policy 0, policy_version 1727923 (0.0010) [2023-12-27 17:36:59,409][20347] Updated weights for policy 0, policy_version 1727933 (0.0009) [2023-12-27 17:36:59,414][20381] Updated weights for policy 1, policy_version 1726392 (0.0009) [2023-12-27 17:36:59,466][20347] Updated weights for policy 0, policy_version 1727943 (0.0009) [2023-12-27 17:36:59,466][20381] Updated weights for policy 1, policy_version 1726402 (0.0008) [2023-12-27 17:36:59,521][20381] Updated weights for policy 1, policy_version 1726412 (0.0009) [2023-12-27 17:37:00,213][20347] Updated weights for policy 0, policy_version 1727953 (0.0008) [2023-12-27 17:37:00,257][20381] Updated weights for policy 1, policy_version 1726422 (0.0010) [2023-12-27 17:37:00,263][20347] Updated weights for policy 0, policy_version 1727963 (0.0006) [2023-12-27 17:37:00,314][20347] Updated weights for policy 0, policy_version 1727973 (0.0007) [2023-12-27 17:37:00,316][20381] Updated weights for policy 1, policy_version 1726432 (0.0010) [2023-12-27 17:37:00,375][20381] Updated weights for policy 1, policy_version 1726442 (0.0008) [2023-12-27 17:37:01,014][20381] Updated weights for policy 1, policy_version 1726452 (0.0007) [2023-12-27 17:37:01,056][20347] Updated weights for policy 0, policy_version 1727983 (0.0007) [2023-12-27 17:37:01,084][20381] Updated weights for policy 1, policy_version 1726462 (0.0008) [2023-12-27 17:37:01,108][20347] Updated weights for policy 0, policy_version 1727993 (0.0007) [2023-12-27 17:37:01,139][20381] Updated weights for policy 1, policy_version 1726472 (0.0008) [2023-12-27 17:37:01,171][20347] Updated weights for policy 0, policy_version 1728003 (0.0007) [2023-12-27 17:37:01,831][20381] Updated weights for policy 1, policy_version 1726482 (0.0008) [2023-12-27 17:37:01,890][20381] Updated weights for policy 1, policy_version 1726492 (0.0007) [2023-12-27 17:37:01,897][20347] Updated weights for policy 0, policy_version 1728013 (0.0008) [2023-12-27 17:37:01,948][20347] Updated weights for policy 0, policy_version 1728023 (0.0008) [2023-12-27 17:37:01,959][20381] Updated weights for policy 1, policy_version 1726502 (0.0008) [2023-12-27 17:37:02,004][20347] Updated weights for policy 0, policy_version 1728033 (0.0006) [2023-12-27 17:37:02,018][20381] Updated weights for policy 1, policy_version 1726512 (0.0007) [2023-12-27 17:37:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 884490240. Throughput: 0: 10271.1, 1: 10182.3. Samples: 884499192. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:02,363][19170] Avg episode reward: [(0, '951.120'), (1, '1000.000')] [2023-12-27 17:37:02,640][20347] Updated weights for policy 0, policy_version 1728043 (0.0006) [2023-12-27 17:37:02,687][20347] Updated weights for policy 0, policy_version 1728053 (0.0006) [2023-12-27 17:37:02,700][20381] Updated weights for policy 1, policy_version 1726522 (0.0006) [2023-12-27 17:37:02,732][20347] Updated weights for policy 0, policy_version 1728063 (0.0006) [2023-12-27 17:37:02,749][20381] Updated weights for policy 1, policy_version 1726532 (0.0005) [2023-12-27 17:37:02,815][20381] Updated weights for policy 1, policy_version 1726542 (0.0007) [2023-12-27 17:37:03,373][20381] Updated weights for policy 1, policy_version 1726552 (0.0006) [2023-12-27 17:37:03,431][20381] Updated weights for policy 1, policy_version 1726562 (0.0005) [2023-12-27 17:37:03,443][20347] Updated weights for policy 0, policy_version 1728073 (0.0007) [2023-12-27 17:37:03,490][20381] Updated weights for policy 1, policy_version 1726572 (0.0006) [2023-12-27 17:37:03,493][20347] Updated weights for policy 0, policy_version 1728083 (0.0005) [2023-12-27 17:37:03,540][20347] Updated weights for policy 0, policy_version 1728093 (0.0005) [2023-12-27 17:37:03,590][20347] Updated weights for policy 0, policy_version 1728103 (0.0007) [2023-12-27 17:37:04,089][20381] Updated weights for policy 1, policy_version 1726582 (0.0006) [2023-12-27 17:37:04,141][20381] Updated weights for policy 1, policy_version 1726592 (0.0006) [2023-12-27 17:37:04,196][20381] Updated weights for policy 1, policy_version 1726602 (0.0007) [2023-12-27 17:37:04,238][20347] Updated weights for policy 0, policy_version 1728113 (0.0010) [2023-12-27 17:37:04,309][20347] Updated weights for policy 0, policy_version 1728123 (0.0009) [2023-12-27 17:37:04,376][20347] Updated weights for policy 0, policy_version 1728133 (0.0005) [2023-12-27 17:37:04,904][20381] Updated weights for policy 1, policy_version 1726612 (0.0009) [2023-12-27 17:37:04,961][20381] Updated weights for policy 1, policy_version 1726622 (0.0011) [2023-12-27 17:37:04,986][20347] Updated weights for policy 0, policy_version 1728143 (0.0009) [2023-12-27 17:37:05,015][20381] Updated weights for policy 1, policy_version 1726632 (0.0009) [2023-12-27 17:37:05,047][20347] Updated weights for policy 0, policy_version 1728153 (0.0010) [2023-12-27 17:37:05,094][20347] Updated weights for policy 0, policy_version 1728163 (0.0007) [2023-12-27 17:37:05,697][20381] Updated weights for policy 1, policy_version 1726642 (0.0006) [2023-12-27 17:37:05,698][20347] Updated weights for policy 0, policy_version 1728173 (0.0006) [2023-12-27 17:37:05,743][20381] Updated weights for policy 1, policy_version 1726652 (0.0010) [2023-12-27 17:37:05,750][20347] Updated weights for policy 0, policy_version 1728183 (0.0008) [2023-12-27 17:37:05,792][20381] Updated weights for policy 1, policy_version 1726662 (0.0010) [2023-12-27 17:37:05,808][20347] Updated weights for policy 0, policy_version 1728193 (0.0008) [2023-12-27 17:37:05,848][20381] Updated weights for policy 1, policy_version 1726672 (0.0011) [2023-12-27 17:37:06,494][20347] Updated weights for policy 0, policy_version 1728203 (0.0006) [2023-12-27 17:37:06,547][20347] Updated weights for policy 0, policy_version 1728213 (0.0007) [2023-12-27 17:37:06,596][20347] Updated weights for policy 0, policy_version 1728223 (0.0006) [2023-12-27 17:37:06,607][20381] Updated weights for policy 1, policy_version 1726682 (0.0008) [2023-12-27 17:37:06,662][20381] Updated weights for policy 1, policy_version 1726692 (0.0008) [2023-12-27 17:37:06,724][20381] Updated weights for policy 1, policy_version 1726702 (0.0008) [2023-12-27 17:37:07,221][20347] Updated weights for policy 0, policy_version 1728233 (0.0006) [2023-12-27 17:37:07,276][20347] Updated weights for policy 0, policy_version 1728243 (0.0008) [2023-12-27 17:37:07,330][20347] Updated weights for policy 0, policy_version 1728253 (0.0007) [2023-12-27 17:37:07,357][20381] Updated weights for policy 1, policy_version 1726712 (0.0009) [2023-12-27 17:37:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 884588544. Throughput: 0: 10291.6, 1: 10206.8. Samples: 884563460. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:07,363][19170] Avg episode reward: [(0, '941.360'), (1, '1000.000')] [2023-12-27 17:37:07,388][20347] Updated weights for policy 0, policy_version 1728263 (0.0008) [2023-12-27 17:37:07,390][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001728264_442515456.pth... [2023-12-27 17:37:07,393][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001727048_442204160.pth [2023-12-27 17:37:07,412][20381] Updated weights for policy 1, policy_version 1726722 (0.0008) [2023-12-27 17:37:07,471][20381] Updated weights for policy 1, policy_version 1726732 (0.0008) [2023-12-27 17:37:07,492][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001726736_442089472.pth... [2023-12-27 17:37:07,496][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001725520_441778176.pth [2023-12-27 17:37:08,067][20381] Updated weights for policy 1, policy_version 1726742 (0.0007) [2023-12-27 17:37:08,083][20347] Updated weights for policy 0, policy_version 1728273 (0.0007) [2023-12-27 17:37:08,131][20381] Updated weights for policy 1, policy_version 1726752 (0.0007) [2023-12-27 17:37:08,137][20347] Updated weights for policy 0, policy_version 1728283 (0.0008) [2023-12-27 17:37:08,190][20381] Updated weights for policy 1, policy_version 1726762 (0.0006) [2023-12-27 17:37:08,196][20347] Updated weights for policy 0, policy_version 1728293 (0.0006) [2023-12-27 17:37:08,811][20381] Updated weights for policy 1, policy_version 1726772 (0.0006) [2023-12-27 17:37:08,872][20381] Updated weights for policy 1, policy_version 1726782 (0.0006) [2023-12-27 17:37:08,905][20347] Updated weights for policy 0, policy_version 1728303 (0.0007) [2023-12-27 17:37:08,939][20381] Updated weights for policy 1, policy_version 1726792 (0.0006) [2023-12-27 17:37:08,972][20347] Updated weights for policy 0, policy_version 1728313 (0.0006) [2023-12-27 17:37:09,027][20347] Updated weights for policy 0, policy_version 1728323 (0.0008) [2023-12-27 17:37:09,604][20381] Updated weights for policy 1, policy_version 1726802 (0.0007) [2023-12-27 17:37:09,656][20381] Updated weights for policy 1, policy_version 1726812 (0.0007) [2023-12-27 17:37:09,701][20381] Updated weights for policy 1, policy_version 1726822 (0.0007) [2023-12-27 17:37:09,733][20347] Updated weights for policy 0, policy_version 1728333 (0.0008) [2023-12-27 17:37:09,758][20381] Updated weights for policy 1, policy_version 1726832 (0.0007) [2023-12-27 17:37:09,790][20347] Updated weights for policy 0, policy_version 1728343 (0.0007) [2023-12-27 17:37:09,858][20347] Updated weights for policy 0, policy_version 1728353 (0.0007) [2023-12-27 17:37:10,457][20381] Updated weights for policy 1, policy_version 1726842 (0.0009) [2023-12-27 17:37:10,518][20381] Updated weights for policy 1, policy_version 1726852 (0.0011) [2023-12-27 17:37:10,569][20381] Updated weights for policy 1, policy_version 1726862 (0.0009) [2023-12-27 17:37:10,572][20347] Updated weights for policy 0, policy_version 1728363 (0.0008) [2023-12-27 17:37:10,631][20347] Updated weights for policy 0, policy_version 1728373 (0.0005) [2023-12-27 17:37:10,681][20347] Updated weights for policy 0, policy_version 1728383 (0.0005) [2023-12-27 17:37:11,265][20381] Updated weights for policy 1, policy_version 1726872 (0.0010) [2023-12-27 17:37:11,310][20347] Updated weights for policy 0, policy_version 1728393 (0.0005) [2023-12-27 17:37:11,335][20381] Updated weights for policy 1, policy_version 1726882 (0.0008) [2023-12-27 17:37:11,387][20347] Updated weights for policy 0, policy_version 1728403 (0.0008) [2023-12-27 17:37:11,409][20381] Updated weights for policy 1, policy_version 1726892 (0.0009) [2023-12-27 17:37:11,448][20347] Updated weights for policy 0, policy_version 1728413 (0.0007) [2023-12-27 17:37:11,508][20347] Updated weights for policy 0, policy_version 1728423 (0.0007) [2023-12-27 17:37:12,139][20381] Updated weights for policy 1, policy_version 1726902 (0.0007) [2023-12-27 17:37:12,200][20381] Updated weights for policy 1, policy_version 1726912 (0.0010) [2023-12-27 17:37:12,220][20347] Updated weights for policy 0, policy_version 1728433 (0.0006) [2023-12-27 17:37:12,262][20381] Updated weights for policy 1, policy_version 1726922 (0.0010) [2023-12-27 17:37:12,287][20347] Updated weights for policy 0, policy_version 1728443 (0.0007) [2023-12-27 17:37:12,350][20347] Updated weights for policy 0, policy_version 1728453 (0.0006) [2023-12-27 17:37:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 884695040. Throughput: 0: 10279.5, 1: 10245.1. Samples: 884687428. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:12,363][19170] Avg episode reward: [(0, '970.680'), (1, '1000.000')] [2023-12-27 17:37:12,914][20381] Updated weights for policy 1, policy_version 1726932 (0.0011) [2023-12-27 17:37:12,969][20381] Updated weights for policy 1, policy_version 1726942 (0.0010) [2023-12-27 17:37:13,018][20381] Updated weights for policy 1, policy_version 1726952 (0.0010) [2023-12-27 17:37:13,030][20347] Updated weights for policy 0, policy_version 1728463 (0.0007) [2023-12-27 17:37:13,089][20347] Updated weights for policy 0, policy_version 1728473 (0.0007) [2023-12-27 17:37:13,139][20347] Updated weights for policy 0, policy_version 1728483 (0.0007) [2023-12-27 17:37:13,696][20381] Updated weights for policy 1, policy_version 1726962 (0.0009) [2023-12-27 17:37:13,721][20347] Updated weights for policy 0, policy_version 1728493 (0.0006) [2023-12-27 17:37:13,747][20381] Updated weights for policy 1, policy_version 1726972 (0.0010) [2023-12-27 17:37:13,776][20347] Updated weights for policy 0, policy_version 1728503 (0.0010) [2023-12-27 17:37:13,802][20381] Updated weights for policy 1, policy_version 1726982 (0.0010) [2023-12-27 17:37:13,831][20347] Updated weights for policy 0, policy_version 1728513 (0.0010) [2023-12-27 17:37:13,853][20381] Updated weights for policy 1, policy_version 1726992 (0.0010) [2023-12-27 17:37:14,449][20347] Updated weights for policy 0, policy_version 1728523 (0.0009) [2023-12-27 17:37:14,504][20347] Updated weights for policy 0, policy_version 1728533 (0.0006) [2023-12-27 17:37:14,557][20381] Updated weights for policy 1, policy_version 1727002 (0.0007) [2023-12-27 17:37:14,566][20347] Updated weights for policy 0, policy_version 1728543 (0.0008) [2023-12-27 17:37:14,612][20381] Updated weights for policy 1, policy_version 1727012 (0.0007) [2023-12-27 17:37:14,665][20381] Updated weights for policy 1, policy_version 1727022 (0.0008) [2023-12-27 17:37:15,228][20347] Updated weights for policy 0, policy_version 1728553 (0.0007) [2023-12-27 17:37:15,286][20347] Updated weights for policy 0, policy_version 1728563 (0.0006) [2023-12-27 17:37:15,339][20381] Updated weights for policy 1, policy_version 1727032 (0.0008) [2023-12-27 17:37:15,342][20347] Updated weights for policy 0, policy_version 1728573 (0.0005) [2023-12-27 17:37:15,390][20381] Updated weights for policy 1, policy_version 1727042 (0.0006) [2023-12-27 17:37:15,392][20347] Updated weights for policy 0, policy_version 1728583 (0.0005) [2023-12-27 17:37:15,451][20381] Updated weights for policy 1, policy_version 1727052 (0.0007) [2023-12-27 17:37:16,072][20381] Updated weights for policy 1, policy_version 1727062 (0.0008) [2023-12-27 17:37:16,077][20347] Updated weights for policy 0, policy_version 1728593 (0.0006) [2023-12-27 17:37:16,127][20381] Updated weights for policy 1, policy_version 1727072 (0.0006) [2023-12-27 17:37:16,133][20347] Updated weights for policy 0, policy_version 1728603 (0.0005) [2023-12-27 17:37:16,184][20347] Updated weights for policy 0, policy_version 1728613 (0.0005) [2023-12-27 17:37:16,185][20381] Updated weights for policy 1, policy_version 1727082 (0.0006) [2023-12-27 17:37:16,751][20381] Updated weights for policy 1, policy_version 1727092 (0.0006) [2023-12-27 17:37:16,809][20381] Updated weights for policy 1, policy_version 1727102 (0.0010) [2023-12-27 17:37:16,866][20381] Updated weights for policy 1, policy_version 1727112 (0.0011) [2023-12-27 17:37:16,906][20347] Updated weights for policy 0, policy_version 1728623 (0.0006) [2023-12-27 17:37:16,963][20347] Updated weights for policy 0, policy_version 1728633 (0.0008) [2023-12-27 17:37:17,017][20347] Updated weights for policy 0, policy_version 1728643 (0.0008) [2023-12-27 17:37:17,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 884801536. Throughput: 0: 10244.2, 1: 10250.3. Samples: 884811544. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:17,364][19170] Avg episode reward: [(0, '951.020'), (1, '1000.000')] [2023-12-27 17:37:17,477][20381] Updated weights for policy 1, policy_version 1727122 (0.0011) [2023-12-27 17:37:17,527][20381] Updated weights for policy 1, policy_version 1727132 (0.0010) [2023-12-27 17:37:17,572][20381] Updated weights for policy 1, policy_version 1727142 (0.0010) [2023-12-27 17:37:17,624][20381] Updated weights for policy 1, policy_version 1727152 (0.0009) [2023-12-27 17:37:17,678][20347] Updated weights for policy 0, policy_version 1728653 (0.0006) [2023-12-27 17:37:17,736][20347] Updated weights for policy 0, policy_version 1728663 (0.0007) [2023-12-27 17:37:17,788][20347] Updated weights for policy 0, policy_version 1728673 (0.0009) [2023-12-27 17:37:18,236][20381] Updated weights for policy 1, policy_version 1727162 (0.0005) [2023-12-27 17:37:18,286][20381] Updated weights for policy 1, policy_version 1727172 (0.0005) [2023-12-27 17:37:18,344][20381] Updated weights for policy 1, policy_version 1727182 (0.0005) [2023-12-27 17:37:18,486][20347] Updated weights for policy 0, policy_version 1728683 (0.0009) [2023-12-27 17:37:18,550][20347] Updated weights for policy 0, policy_version 1728693 (0.0005) [2023-12-27 17:37:18,609][20347] Updated weights for policy 0, policy_version 1728703 (0.0005) [2023-12-27 17:37:19,010][20381] Updated weights for policy 1, policy_version 1727192 (0.0010) [2023-12-27 17:37:19,058][20381] Updated weights for policy 1, policy_version 1727202 (0.0010) [2023-12-27 17:37:19,104][20381] Updated weights for policy 1, policy_version 1727212 (0.0010) [2023-12-27 17:37:19,193][20347] Updated weights for policy 0, policy_version 1728713 (0.0007) [2023-12-27 17:37:19,261][20347] Updated weights for policy 0, policy_version 1728723 (0.0007) [2023-12-27 17:37:19,327][20347] Updated weights for policy 0, policy_version 1728733 (0.0006) [2023-12-27 17:37:19,388][20347] Updated weights for policy 0, policy_version 1728743 (0.0008) [2023-12-27 17:37:19,865][20381] Updated weights for policy 1, policy_version 1727222 (0.0009) [2023-12-27 17:37:19,926][20381] Updated weights for policy 1, policy_version 1727232 (0.0008) [2023-12-27 17:37:19,993][20381] Updated weights for policy 1, policy_version 1727242 (0.0011) [2023-12-27 17:37:20,095][20347] Updated weights for policy 0, policy_version 1728753 (0.0007) [2023-12-27 17:37:20,157][20347] Updated weights for policy 0, policy_version 1728763 (0.0005) [2023-12-27 17:37:20,224][20347] Updated weights for policy 0, policy_version 1728773 (0.0008) [2023-12-27 17:37:20,695][20381] Updated weights for policy 1, policy_version 1727252 (0.0010) [2023-12-27 17:37:20,759][20381] Updated weights for policy 1, policy_version 1727262 (0.0011) [2023-12-27 17:37:20,819][20381] Updated weights for policy 1, policy_version 1727272 (0.0011) [2023-12-27 17:37:20,869][20347] Updated weights for policy 0, policy_version 1728783 (0.0008) [2023-12-27 17:37:20,931][20347] Updated weights for policy 0, policy_version 1728793 (0.0007) [2023-12-27 17:37:20,985][20347] Updated weights for policy 0, policy_version 1728803 (0.0006) [2023-12-27 17:37:21,574][20381] Updated weights for policy 1, policy_version 1727282 (0.0010) [2023-12-27 17:37:21,644][20381] Updated weights for policy 1, policy_version 1727292 (0.0009) [2023-12-27 17:37:21,676][20347] Updated weights for policy 0, policy_version 1728813 (0.0008) [2023-12-27 17:37:21,705][20381] Updated weights for policy 1, policy_version 1727302 (0.0007) [2023-12-27 17:37:21,730][20347] Updated weights for policy 0, policy_version 1728823 (0.0007) [2023-12-27 17:37:21,769][20381] Updated weights for policy 1, policy_version 1727312 (0.0008) [2023-12-27 17:37:21,796][20347] Updated weights for policy 0, policy_version 1728833 (0.0007) [2023-12-27 17:37:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 884899840. Throughput: 0: 10225.4, 1: 10267.3. Samples: 884873456. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:22,363][19170] Avg episode reward: [(0, '911.900'), (1, '1000.000')] [2023-12-27 17:37:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001728840_442662912.pth... [2023-12-27 17:37:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001727312_442236928.pth... [2023-12-27 17:37:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001726128_441933824.pth [2023-12-27 17:37:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001727656_442359808.pth [2023-12-27 17:37:22,518][20347] Updated weights for policy 0, policy_version 1728843 (0.0006) [2023-12-27 17:37:22,581][20347] Updated weights for policy 0, policy_version 1728853 (0.0006) [2023-12-27 17:37:22,590][20381] Updated weights for policy 1, policy_version 1727322 (0.0008) [2023-12-27 17:37:22,644][20347] Updated weights for policy 0, policy_version 1728863 (0.0006) [2023-12-27 17:37:22,652][20381] Updated weights for policy 1, policy_version 1727332 (0.0008) [2023-12-27 17:37:22,710][20381] Updated weights for policy 1, policy_version 1727342 (0.0007) [2023-12-27 17:37:23,309][20347] Updated weights for policy 0, policy_version 1728873 (0.0005) [2023-12-27 17:37:23,349][20381] Updated weights for policy 1, policy_version 1727352 (0.0007) [2023-12-27 17:37:23,362][20347] Updated weights for policy 0, policy_version 1728883 (0.0008) [2023-12-27 17:37:23,401][20381] Updated weights for policy 1, policy_version 1727362 (0.0005) [2023-12-27 17:37:23,424][20347] Updated weights for policy 0, policy_version 1728893 (0.0006) [2023-12-27 17:37:23,452][20381] Updated weights for policy 1, policy_version 1727372 (0.0006) [2023-12-27 17:37:23,483][20347] Updated weights for policy 0, policy_version 1728903 (0.0007) [2023-12-27 17:37:24,090][20381] Updated weights for policy 1, policy_version 1727382 (0.0007) [2023-12-27 17:37:24,137][20381] Updated weights for policy 1, policy_version 1727392 (0.0007) [2023-12-27 17:37:24,174][20347] Updated weights for policy 0, policy_version 1728913 (0.0010) [2023-12-27 17:37:24,187][20381] Updated weights for policy 1, policy_version 1727402 (0.0007) [2023-12-27 17:37:24,230][20347] Updated weights for policy 0, policy_version 1728923 (0.0010) [2023-12-27 17:37:24,246][20291] KL-divergence is very high: 221.9888 [2023-12-27 17:37:24,285][20347] Updated weights for policy 0, policy_version 1728933 (0.0010) [2023-12-27 17:37:24,290][20291] KL-divergence is very high: 277.8777 [2023-12-27 17:37:24,822][20381] Updated weights for policy 1, policy_version 1727412 (0.0005) [2023-12-27 17:37:24,876][20347] Updated weights for policy 0, policy_version 1728943 (0.0010) [2023-12-27 17:37:24,884][20381] Updated weights for policy 1, policy_version 1727422 (0.0009) [2023-12-27 17:37:24,928][20347] Updated weights for policy 0, policy_version 1728953 (0.0010) [2023-12-27 17:37:24,943][20381] Updated weights for policy 1, policy_version 1727432 (0.0006) [2023-12-27 17:37:24,983][20347] Updated weights for policy 0, policy_version 1728963 (0.0006) [2023-12-27 17:37:25,567][20381] Updated weights for policy 1, policy_version 1727442 (0.0008) [2023-12-27 17:37:25,574][20347] Updated weights for policy 0, policy_version 1728973 (0.0008) [2023-12-27 17:37:25,618][20381] Updated weights for policy 1, policy_version 1727452 (0.0010) [2023-12-27 17:37:25,630][20347] Updated weights for policy 0, policy_version 1728983 (0.0010) [2023-12-27 17:37:25,663][20381] Updated weights for policy 1, policy_version 1727462 (0.0010) [2023-12-27 17:37:25,685][20347] Updated weights for policy 0, policy_version 1728993 (0.0010) [2023-12-27 17:37:25,711][20381] Updated weights for policy 1, policy_version 1727472 (0.0010) [2023-12-27 17:37:26,344][20381] Updated weights for policy 1, policy_version 1727482 (0.0006) [2023-12-27 17:37:26,371][20347] Updated weights for policy 0, policy_version 1729003 (0.0009) [2023-12-27 17:37:26,399][20381] Updated weights for policy 1, policy_version 1727492 (0.0006) [2023-12-27 17:37:26,427][20347] Updated weights for policy 0, policy_version 1729013 (0.0007) [2023-12-27 17:37:26,453][20381] Updated weights for policy 1, policy_version 1727502 (0.0006) [2023-12-27 17:37:26,489][20347] Updated weights for policy 0, policy_version 1729023 (0.0007) [2023-12-27 17:37:27,013][20381] Updated weights for policy 1, policy_version 1727512 (0.0007) [2023-12-27 17:37:27,067][20381] Updated weights for policy 1, policy_version 1727522 (0.0007) [2023-12-27 17:37:27,118][20381] Updated weights for policy 1, policy_version 1727532 (0.0007) [2023-12-27 17:37:27,158][20347] Updated weights for policy 0, policy_version 1729033 (0.0008) [2023-12-27 17:37:27,223][20347] Updated weights for policy 0, policy_version 1729043 (0.0008) [2023-12-27 17:37:27,280][20347] Updated weights for policy 0, policy_version 1729053 (0.0008) [2023-12-27 17:37:27,337][20347] Updated weights for policy 0, policy_version 1729063 (0.0007) [2023-12-27 17:37:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 885014528. Throughput: 0: 10304.7, 1: 10320.4. Samples: 884997560. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:27,363][19170] Avg episode reward: [(0, '892.410'), (1, '1000.000')] [2023-12-27 17:37:27,803][20381] Updated weights for policy 1, policy_version 1727542 (0.0007) [2023-12-27 17:37:27,854][20381] Updated weights for policy 1, policy_version 1727552 (0.0008) [2023-12-27 17:37:27,907][20381] Updated weights for policy 1, policy_version 1727562 (0.0008) [2023-12-27 17:37:28,015][20347] Updated weights for policy 0, policy_version 1729073 (0.0007) [2023-12-27 17:37:28,066][20347] Updated weights for policy 0, policy_version 1729083 (0.0006) [2023-12-27 17:37:28,120][20347] Updated weights for policy 0, policy_version 1729093 (0.0007) [2023-12-27 17:37:28,542][20381] Updated weights for policy 1, policy_version 1727572 (0.0005) [2023-12-27 17:37:28,607][20381] Updated weights for policy 1, policy_version 1727582 (0.0005) [2023-12-27 17:37:28,671][20381] Updated weights for policy 1, policy_version 1727592 (0.0008) [2023-12-27 17:37:28,840][20347] Updated weights for policy 0, policy_version 1729103 (0.0006) [2023-12-27 17:37:28,896][20347] Updated weights for policy 0, policy_version 1729113 (0.0007) [2023-12-27 17:37:28,952][20347] Updated weights for policy 0, policy_version 1729123 (0.0007) [2023-12-27 17:37:29,436][20381] Updated weights for policy 1, policy_version 1727602 (0.0008) [2023-12-27 17:37:29,482][20381] Updated weights for policy 1, policy_version 1727612 (0.0005) [2023-12-27 17:37:29,529][20381] Updated weights for policy 1, policy_version 1727622 (0.0005) [2023-12-27 17:37:29,575][20381] Updated weights for policy 1, policy_version 1727632 (0.0005) [2023-12-27 17:37:29,649][20347] Updated weights for policy 0, policy_version 1729133 (0.0007) [2023-12-27 17:37:29,708][20347] Updated weights for policy 0, policy_version 1729143 (0.0005) [2023-12-27 17:37:29,768][20347] Updated weights for policy 0, policy_version 1729153 (0.0007) [2023-12-27 17:37:30,257][20381] Updated weights for policy 1, policy_version 1727642 (0.0007) [2023-12-27 17:37:30,316][20381] Updated weights for policy 1, policy_version 1727652 (0.0008) [2023-12-27 17:37:30,378][20381] Updated weights for policy 1, policy_version 1727662 (0.0006) [2023-12-27 17:37:30,399][20347] Updated weights for policy 0, policy_version 1729163 (0.0006) [2023-12-27 17:37:30,448][20347] Updated weights for policy 0, policy_version 1729173 (0.0005) [2023-12-27 17:37:30,491][20347] Updated weights for policy 0, policy_version 1729183 (0.0007) [2023-12-27 17:37:31,056][20381] Updated weights for policy 1, policy_version 1727672 (0.0007) [2023-12-27 17:37:31,111][20381] Updated weights for policy 1, policy_version 1727682 (0.0008) [2023-12-27 17:37:31,178][20381] Updated weights for policy 1, policy_version 1727692 (0.0008) [2023-12-27 17:37:31,187][20347] Updated weights for policy 0, policy_version 1729193 (0.0008) [2023-12-27 17:37:31,254][20347] Updated weights for policy 0, policy_version 1729203 (0.0008) [2023-12-27 17:37:31,316][20347] Updated weights for policy 0, policy_version 1729213 (0.0008) [2023-12-27 17:37:31,384][20347] Updated weights for policy 0, policy_version 1729223 (0.0007) [2023-12-27 17:37:31,938][20381] Updated weights for policy 1, policy_version 1727702 (0.0006) [2023-12-27 17:37:31,990][20381] Updated weights for policy 1, policy_version 1727712 (0.0005) [2023-12-27 17:37:32,041][20381] Updated weights for policy 1, policy_version 1727722 (0.0008) [2023-12-27 17:37:32,067][20347] Updated weights for policy 0, policy_version 1729233 (0.0008) [2023-12-27 17:37:32,123][20347] Updated weights for policy 0, policy_version 1729243 (0.0005) [2023-12-27 17:37:32,183][20347] Updated weights for policy 0, policy_version 1729253 (0.0006) [2023-12-27 17:37:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 885112832. Throughput: 0: 10262.0, 1: 10314.8. Samples: 885121000. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:32,363][19170] Avg episode reward: [(0, '941.370'), (1, '1000.000')] [2023-12-27 17:37:32,708][20381] Updated weights for policy 1, policy_version 1727732 (0.0010) [2023-12-27 17:37:32,763][20381] Updated weights for policy 1, policy_version 1727742 (0.0010) [2023-12-27 17:37:32,811][20347] Updated weights for policy 0, policy_version 1729263 (0.0006) [2023-12-27 17:37:32,818][20381] Updated weights for policy 1, policy_version 1727752 (0.0010) [2023-12-27 17:37:32,870][20347] Updated weights for policy 0, policy_version 1729273 (0.0006) [2023-12-27 17:37:32,918][20347] Updated weights for policy 0, policy_version 1729283 (0.0005) [2023-12-27 17:37:33,478][20381] Updated weights for policy 1, policy_version 1727762 (0.0009) [2023-12-27 17:37:33,485][20347] Updated weights for policy 0, policy_version 1729293 (0.0005) [2023-12-27 17:37:33,535][20381] Updated weights for policy 1, policy_version 1727772 (0.0006) [2023-12-27 17:37:33,544][20347] Updated weights for policy 0, policy_version 1729303 (0.0005) [2023-12-27 17:37:33,588][20347] Updated weights for policy 0, policy_version 1729313 (0.0005) [2023-12-27 17:37:33,589][20381] Updated weights for policy 1, policy_version 1727782 (0.0008) [2023-12-27 17:37:33,642][20381] Updated weights for policy 1, policy_version 1727792 (0.0007) [2023-12-27 17:37:34,190][20347] Updated weights for policy 0, policy_version 1729323 (0.0006) [2023-12-27 17:37:34,249][20347] Updated weights for policy 0, policy_version 1729333 (0.0009) [2023-12-27 17:37:34,287][20381] Updated weights for policy 1, policy_version 1727802 (0.0010) [2023-12-27 17:37:34,311][20347] Updated weights for policy 0, policy_version 1729343 (0.0007) [2023-12-27 17:37:34,349][20381] Updated weights for policy 1, policy_version 1727812 (0.0008) [2023-12-27 17:37:34,415][20381] Updated weights for policy 1, policy_version 1727822 (0.0007) [2023-12-27 17:37:35,046][20347] Updated weights for policy 0, policy_version 1729353 (0.0007) [2023-12-27 17:37:35,100][20347] Updated weights for policy 0, policy_version 1729363 (0.0007) [2023-12-27 17:37:35,116][20381] Updated weights for policy 1, policy_version 1727832 (0.0010) [2023-12-27 17:37:35,157][20347] Updated weights for policy 0, policy_version 1729373 (0.0006) [2023-12-27 17:37:35,173][20381] Updated weights for policy 1, policy_version 1727842 (0.0010) [2023-12-27 17:37:35,209][20347] Updated weights for policy 0, policy_version 1729383 (0.0006) [2023-12-27 17:37:35,225][20381] Updated weights for policy 1, policy_version 1727852 (0.0010) [2023-12-27 17:37:35,845][20381] Updated weights for policy 1, policy_version 1727862 (0.0011) [2023-12-27 17:37:35,904][20381] Updated weights for policy 1, policy_version 1727872 (0.0005) [2023-12-27 17:37:35,919][20347] Updated weights for policy 0, policy_version 1729393 (0.0007) [2023-12-27 17:37:35,961][20381] Updated weights for policy 1, policy_version 1727882 (0.0005) [2023-12-27 17:37:35,970][20347] Updated weights for policy 0, policy_version 1729403 (0.0007) [2023-12-27 17:37:36,029][20347] Updated weights for policy 0, policy_version 1729413 (0.0006) [2023-12-27 17:37:36,543][20381] Updated weights for policy 1, policy_version 1727892 (0.0007) [2023-12-27 17:37:36,600][20381] Updated weights for policy 1, policy_version 1727902 (0.0008) [2023-12-27 17:37:36,660][20381] Updated weights for policy 1, policy_version 1727912 (0.0008) [2023-12-27 17:37:36,743][20347] Updated weights for policy 0, policy_version 1729423 (0.0008) [2023-12-27 17:37:36,796][20347] Updated weights for policy 0, policy_version 1729433 (0.0006) [2023-12-27 17:37:36,857][20347] Updated weights for policy 0, policy_version 1729443 (0.0008) [2023-12-27 17:37:37,261][20381] Updated weights for policy 1, policy_version 1727922 (0.0006) [2023-12-27 17:37:37,320][20381] Updated weights for policy 1, policy_version 1727932 (0.0008) [2023-12-27 17:37:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 885211136. Throughput: 0: 10306.0, 1: 10317.5. Samples: 885184564. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:37,363][19170] Avg episode reward: [(0, '931.530'), (1, '1000.000')] [2023-12-27 17:37:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001729448_442818560.pth... [2023-12-27 17:37:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001728264_442515456.pth [2023-12-27 17:37:37,381][20381] Updated weights for policy 1, policy_version 1727942 (0.0007) [2023-12-27 17:37:37,439][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001727952_442400768.pth... [2023-12-27 17:37:37,442][20381] Updated weights for policy 1, policy_version 1727952 (0.0008) [2023-12-27 17:37:37,444][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001726736_442089472.pth [2023-12-27 17:37:37,570][20347] Updated weights for policy 0, policy_version 1729453 (0.0007) [2023-12-27 17:37:37,623][20347] Updated weights for policy 0, policy_version 1729463 (0.0005) [2023-12-27 17:37:37,676][20347] Updated weights for policy 0, policy_version 1729473 (0.0006) [2023-12-27 17:37:38,062][20381] Updated weights for policy 1, policy_version 1727962 (0.0006) [2023-12-27 17:37:38,108][20381] Updated weights for policy 1, policy_version 1727972 (0.0006) [2023-12-27 17:37:38,159][20381] Updated weights for policy 1, policy_version 1727982 (0.0008) [2023-12-27 17:37:38,310][20347] Updated weights for policy 0, policy_version 1729483 (0.0008) [2023-12-27 17:37:38,379][20347] Updated weights for policy 0, policy_version 1729493 (0.0007) [2023-12-27 17:37:38,434][20347] Updated weights for policy 0, policy_version 1729503 (0.0008) [2023-12-27 17:37:38,842][20381] Updated weights for policy 1, policy_version 1727992 (0.0008) [2023-12-27 17:37:38,902][20381] Updated weights for policy 1, policy_version 1728002 (0.0008) [2023-12-27 17:37:38,949][20381] Updated weights for policy 1, policy_version 1728012 (0.0008) [2023-12-27 17:37:39,098][20347] Updated weights for policy 0, policy_version 1729513 (0.0008) [2023-12-27 17:37:39,162][20347] Updated weights for policy 0, policy_version 1729523 (0.0008) [2023-12-27 17:37:39,222][20347] Updated weights for policy 0, policy_version 1729533 (0.0007) [2023-12-27 17:37:39,286][20347] Updated weights for policy 0, policy_version 1729543 (0.0008) [2023-12-27 17:37:39,616][20381] Updated weights for policy 1, policy_version 1728022 (0.0007) [2023-12-27 17:37:39,677][20381] Updated weights for policy 1, policy_version 1728032 (0.0008) [2023-12-27 17:37:39,734][20381] Updated weights for policy 1, policy_version 1728042 (0.0008) [2023-12-27 17:37:40,020][20347] Updated weights for policy 0, policy_version 1729553 (0.0007) [2023-12-27 17:37:40,072][20347] Updated weights for policy 0, policy_version 1729563 (0.0007) [2023-12-27 17:37:40,130][20347] Updated weights for policy 0, policy_version 1729573 (0.0008) [2023-12-27 17:37:40,425][20381] Updated weights for policy 1, policy_version 1728052 (0.0007) [2023-12-27 17:37:40,485][20381] Updated weights for policy 1, policy_version 1728062 (0.0005) [2023-12-27 17:37:40,549][20381] Updated weights for policy 1, policy_version 1728072 (0.0006) [2023-12-27 17:37:40,866][20347] Updated weights for policy 0, policy_version 1729583 (0.0007) [2023-12-27 17:37:40,925][20347] Updated weights for policy 0, policy_version 1729593 (0.0006) [2023-12-27 17:37:40,986][20347] Updated weights for policy 0, policy_version 1729603 (0.0006) [2023-12-27 17:37:41,193][20381] Updated weights for policy 1, policy_version 1728082 (0.0008) [2023-12-27 17:37:41,259][20381] Updated weights for policy 1, policy_version 1728092 (0.0007) [2023-12-27 17:37:41,321][20381] Updated weights for policy 1, policy_version 1728102 (0.0008) [2023-12-27 17:37:41,389][20381] Updated weights for policy 1, policy_version 1728112 (0.0008) [2023-12-27 17:37:41,743][20347] Updated weights for policy 0, policy_version 1729613 (0.0008) [2023-12-27 17:37:41,806][20347] Updated weights for policy 0, policy_version 1729623 (0.0008) [2023-12-27 17:37:41,856][20347] Updated weights for policy 0, policy_version 1729633 (0.0007) [2023-12-27 17:37:42,157][20381] Updated weights for policy 1, policy_version 1728122 (0.0007) [2023-12-27 17:37:42,208][20381] Updated weights for policy 1, policy_version 1728132 (0.0008) [2023-12-27 17:37:42,263][20381] Updated weights for policy 1, policy_version 1728142 (0.0008) [2023-12-27 17:37:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 885317632. Throughput: 0: 10235.5, 1: 10356.4. Samples: 885307520. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:42,363][19170] Avg episode reward: [(0, '931.470'), (1, '1000.000')] [2023-12-27 17:37:42,545][20347] Updated weights for policy 0, policy_version 1729643 (0.0007) [2023-12-27 17:37:42,603][20347] Updated weights for policy 0, policy_version 1729653 (0.0008) [2023-12-27 17:37:42,657][20347] Updated weights for policy 0, policy_version 1729663 (0.0006) [2023-12-27 17:37:42,915][20381] Updated weights for policy 1, policy_version 1728152 (0.0007) [2023-12-27 17:37:42,967][20381] Updated weights for policy 1, policy_version 1728162 (0.0006) [2023-12-27 17:37:43,015][20381] Updated weights for policy 1, policy_version 1728172 (0.0006) [2023-12-27 17:37:43,270][20347] Updated weights for policy 0, policy_version 1729673 (0.0006) [2023-12-27 17:37:43,328][20347] Updated weights for policy 0, policy_version 1729683 (0.0010) [2023-12-27 17:37:43,383][20347] Updated weights for policy 0, policy_version 1729693 (0.0010) [2023-12-27 17:37:43,441][20347] Updated weights for policy 0, policy_version 1729703 (0.0010) [2023-12-27 17:37:43,583][20381] Updated weights for policy 1, policy_version 1728182 (0.0007) [2023-12-27 17:37:43,642][20381] Updated weights for policy 1, policy_version 1728192 (0.0006) [2023-12-27 17:37:43,696][20381] Updated weights for policy 1, policy_version 1728202 (0.0006) [2023-12-27 17:37:44,103][20347] Updated weights for policy 0, policy_version 1729713 (0.0008) [2023-12-27 17:37:44,153][20347] Updated weights for policy 0, policy_version 1729723 (0.0008) [2023-12-27 17:37:44,209][20347] Updated weights for policy 0, policy_version 1729733 (0.0008) [2023-12-27 17:37:44,278][20381] Updated weights for policy 1, policy_version 1728212 (0.0007) [2023-12-27 17:37:44,334][20381] Updated weights for policy 1, policy_version 1728222 (0.0006) [2023-12-27 17:37:44,393][20381] Updated weights for policy 1, policy_version 1728232 (0.0008) [2023-12-27 17:37:44,930][20347] Updated weights for policy 0, policy_version 1729743 (0.0008) [2023-12-27 17:37:44,992][20347] Updated weights for policy 0, policy_version 1729753 (0.0010) [2023-12-27 17:37:45,036][20381] Updated weights for policy 1, policy_version 1728242 (0.0007) [2023-12-27 17:37:45,054][20347] Updated weights for policy 0, policy_version 1729763 (0.0008) [2023-12-27 17:37:45,108][20381] Updated weights for policy 1, policy_version 1728252 (0.0006) [2023-12-27 17:37:45,162][20381] Updated weights for policy 1, policy_version 1728262 (0.0008) [2023-12-27 17:37:45,215][20381] Updated weights for policy 1, policy_version 1728272 (0.0008) [2023-12-27 17:37:45,662][20347] Updated weights for policy 0, policy_version 1729773 (0.0007) [2023-12-27 17:37:45,717][20347] Updated weights for policy 0, policy_version 1729783 (0.0006) [2023-12-27 17:37:45,772][20347] Updated weights for policy 0, policy_version 1729793 (0.0007) [2023-12-27 17:37:45,957][20381] Updated weights for policy 1, policy_version 1728282 (0.0010) [2023-12-27 17:37:46,005][20381] Updated weights for policy 1, policy_version 1728292 (0.0009) [2023-12-27 17:37:46,055][20381] Updated weights for policy 1, policy_version 1728302 (0.0009) [2023-12-27 17:37:46,302][20347] Updated weights for policy 0, policy_version 1729803 (0.0005) [2023-12-27 17:37:46,356][20347] Updated weights for policy 0, policy_version 1729813 (0.0005) [2023-12-27 17:37:46,407][20347] Updated weights for policy 0, policy_version 1729823 (0.0006) [2023-12-27 17:37:46,757][20381] Updated weights for policy 1, policy_version 1728312 (0.0008) [2023-12-27 17:37:46,813][20381] Updated weights for policy 1, policy_version 1728322 (0.0008) [2023-12-27 17:37:46,879][20381] Updated weights for policy 1, policy_version 1728332 (0.0006) [2023-12-27 17:37:47,013][20347] Updated weights for policy 0, policy_version 1729833 (0.0010) [2023-12-27 17:37:47,068][20347] Updated weights for policy 0, policy_version 1729843 (0.0010) [2023-12-27 17:37:47,122][20347] Updated weights for policy 0, policy_version 1729853 (0.0010) [2023-12-27 17:37:47,176][20347] Updated weights for policy 0, policy_version 1729863 (0.0010) [2023-12-27 17:37:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 885424128. Throughput: 0: 10343.2, 1: 10426.8. Samples: 885433840. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:47,363][19170] Avg episode reward: [(0, '941.360'), (1, '1000.000')] [2023-12-27 17:37:47,513][20381] Updated weights for policy 1, policy_version 1728342 (0.0007) [2023-12-27 17:37:47,570][20381] Updated weights for policy 1, policy_version 1728352 (0.0006) [2023-12-27 17:37:47,631][20381] Updated weights for policy 1, policy_version 1728362 (0.0006) [2023-12-27 17:37:47,867][20347] Updated weights for policy 0, policy_version 1729873 (0.0010) [2023-12-27 17:37:47,916][20347] Updated weights for policy 0, policy_version 1729883 (0.0010) [2023-12-27 17:37:47,972][20347] Updated weights for policy 0, policy_version 1729893 (0.0008) [2023-12-27 17:37:48,258][20381] Updated weights for policy 1, policy_version 1728372 (0.0008) [2023-12-27 17:37:48,321][20381] Updated weights for policy 1, policy_version 1728382 (0.0007) [2023-12-27 17:37:48,386][20381] Updated weights for policy 1, policy_version 1728392 (0.0008) [2023-12-27 17:37:48,616][20347] Updated weights for policy 0, policy_version 1729903 (0.0009) [2023-12-27 17:37:48,674][20347] Updated weights for policy 0, policy_version 1729913 (0.0010) [2023-12-27 17:37:48,737][20347] Updated weights for policy 0, policy_version 1729923 (0.0011) [2023-12-27 17:37:49,075][20381] Updated weights for policy 1, policy_version 1728402 (0.0009) [2023-12-27 17:37:49,129][20381] Updated weights for policy 1, policy_version 1728412 (0.0011) [2023-12-27 17:37:49,191][20381] Updated weights for policy 1, policy_version 1728422 (0.0011) [2023-12-27 17:37:49,273][20381] Updated weights for policy 1, policy_version 1728432 (0.0011) [2023-12-27 17:37:49,457][20347] Updated weights for policy 0, policy_version 1729933 (0.0010) [2023-12-27 17:37:49,509][20347] Updated weights for policy 0, policy_version 1729943 (0.0010) [2023-12-27 17:37:49,558][20347] Updated weights for policy 0, policy_version 1729953 (0.0010) [2023-12-27 17:37:49,974][20381] Updated weights for policy 1, policy_version 1728442 (0.0008) [2023-12-27 17:37:50,045][20381] Updated weights for policy 1, policy_version 1728452 (0.0007) [2023-12-27 17:37:50,114][20381] Updated weights for policy 1, policy_version 1728462 (0.0008) [2023-12-27 17:37:50,216][20347] Updated weights for policy 0, policy_version 1729963 (0.0010) [2023-12-27 17:37:50,267][20347] Updated weights for policy 0, policy_version 1729973 (0.0010) [2023-12-27 17:37:50,317][20347] Updated weights for policy 0, policy_version 1729983 (0.0011) [2023-12-27 17:37:50,804][20381] Updated weights for policy 1, policy_version 1728472 (0.0007) [2023-12-27 17:37:50,865][20381] Updated weights for policy 1, policy_version 1728482 (0.0008) [2023-12-27 17:37:50,925][20381] Updated weights for policy 1, policy_version 1728492 (0.0008) [2023-12-27 17:37:50,952][20347] Updated weights for policy 0, policy_version 1729993 (0.0008) [2023-12-27 17:37:51,010][20347] Updated weights for policy 0, policy_version 1730003 (0.0006) [2023-12-27 17:37:51,069][20347] Updated weights for policy 0, policy_version 1730013 (0.0011) [2023-12-27 17:37:51,124][20347] Updated weights for policy 0, policy_version 1730023 (0.0009) [2023-12-27 17:37:51,606][20381] Updated weights for policy 1, policy_version 1728502 (0.0007) [2023-12-27 17:37:51,669][20381] Updated weights for policy 1, policy_version 1728512 (0.0008) [2023-12-27 17:37:51,729][20381] Updated weights for policy 1, policy_version 1728522 (0.0008) [2023-12-27 17:37:51,901][20347] Updated weights for policy 0, policy_version 1730033 (0.0008) [2023-12-27 17:37:51,948][20347] Updated weights for policy 0, policy_version 1730043 (0.0008) [2023-12-27 17:37:51,996][20347] Updated weights for policy 0, policy_version 1730053 (0.0007) [2023-12-27 17:37:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 885522432. Throughput: 0: 10309.4, 1: 10365.6. Samples: 885493836. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:52,363][19170] Avg episode reward: [(0, '941.390'), (1, '1000.000')] [2023-12-27 17:37:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001730056_442974208.pth... [2023-12-27 17:37:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001728840_442662912.pth [2023-12-27 17:37:52,399][20381] Updated weights for policy 1, policy_version 1728532 (0.0009) [2023-12-27 17:37:52,464][20381] Updated weights for policy 1, policy_version 1728542 (0.0007) [2023-12-27 17:37:52,521][20381] Updated weights for policy 1, policy_version 1728552 (0.0008) [2023-12-27 17:37:52,564][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001728560_442556416.pth... [2023-12-27 17:37:52,568][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001727312_442236928.pth [2023-12-27 17:37:52,709][20347] Updated weights for policy 0, policy_version 1730064 (0.0008) [2023-12-27 17:37:52,758][20347] Updated weights for policy 0, policy_version 1730074 (0.0008) [2023-12-27 17:37:52,805][20347] Updated weights for policy 0, policy_version 1730084 (0.0007) [2023-12-27 17:37:53,137][20381] Updated weights for policy 1, policy_version 1728562 (0.0007) [2023-12-27 17:37:53,193][20381] Updated weights for policy 1, policy_version 1728572 (0.0006) [2023-12-27 17:37:53,243][20381] Updated weights for policy 1, policy_version 1728582 (0.0007) [2023-12-27 17:37:53,297][20381] Updated weights for policy 1, policy_version 1728592 (0.0005) [2023-12-27 17:37:53,491][20347] Updated weights for policy 0, policy_version 1730094 (0.0009) [2023-12-27 17:37:53,550][20347] Updated weights for policy 0, policy_version 1730104 (0.0010) [2023-12-27 17:37:53,608][20347] Updated weights for policy 0, policy_version 1730114 (0.0010) [2023-12-27 17:37:53,894][20381] Updated weights for policy 1, policy_version 1728602 (0.0008) [2023-12-27 17:37:53,943][20381] Updated weights for policy 1, policy_version 1728612 (0.0008) [2023-12-27 17:37:53,985][20381] Updated weights for policy 1, policy_version 1728622 (0.0008) [2023-12-27 17:37:54,239][20347] Updated weights for policy 0, policy_version 1730124 (0.0010) [2023-12-27 17:37:54,306][20347] Updated weights for policy 0, policy_version 1730134 (0.0010) [2023-12-27 17:37:54,366][20347] Updated weights for policy 0, policy_version 1730144 (0.0006) [2023-12-27 17:37:54,635][20381] Updated weights for policy 1, policy_version 1728632 (0.0006) [2023-12-27 17:37:54,691][20381] Updated weights for policy 1, policy_version 1728642 (0.0006) [2023-12-27 17:37:54,749][20381] Updated weights for policy 1, policy_version 1728652 (0.0006) [2023-12-27 17:37:55,006][20347] Updated weights for policy 0, policy_version 1730154 (0.0010) [2023-12-27 17:37:55,064][20347] Updated weights for policy 0, policy_version 1730164 (0.0010) [2023-12-27 17:37:55,125][20347] Updated weights for policy 0, policy_version 1730174 (0.0010) [2023-12-27 17:37:55,183][20347] Updated weights for policy 0, policy_version 1730184 (0.0008) [2023-12-27 17:37:55,299][20381] Updated weights for policy 1, policy_version 1728662 (0.0005) [2023-12-27 17:37:55,348][20381] Updated weights for policy 1, policy_version 1728672 (0.0005) [2023-12-27 17:37:55,408][20381] Updated weights for policy 1, policy_version 1728682 (0.0005) [2023-12-27 17:37:55,788][20347] Updated weights for policy 0, policy_version 1730194 (0.0005) [2023-12-27 17:37:55,836][20347] Updated weights for policy 0, policy_version 1730204 (0.0009) [2023-12-27 17:37:55,880][20347] Updated weights for policy 0, policy_version 1730214 (0.0007) [2023-12-27 17:37:56,074][20381] Updated weights for policy 1, policy_version 1728692 (0.0007) [2023-12-27 17:37:56,127][20381] Updated weights for policy 1, policy_version 1728702 (0.0005) [2023-12-27 17:37:56,188][20381] Updated weights for policy 1, policy_version 1728712 (0.0008) [2023-12-27 17:37:56,455][20347] Updated weights for policy 0, policy_version 1730224 (0.0006) [2023-12-27 17:37:56,508][20347] Updated weights for policy 0, policy_version 1730234 (0.0005) [2023-12-27 17:37:56,558][20347] Updated weights for policy 0, policy_version 1730244 (0.0005) [2023-12-27 17:37:56,897][20381] Updated weights for policy 1, policy_version 1728722 (0.0009) [2023-12-27 17:37:56,960][20381] Updated weights for policy 1, policy_version 1728732 (0.0007) [2023-12-27 17:37:57,007][20381] Updated weights for policy 1, policy_version 1728742 (0.0007) [2023-12-27 17:37:57,051][20381] Updated weights for policy 1, policy_version 1728752 (0.0007) [2023-12-27 17:37:57,169][20347] Updated weights for policy 0, policy_version 1730254 (0.0005) [2023-12-27 17:37:57,214][20347] Updated weights for policy 0, policy_version 1730264 (0.0005) [2023-12-27 17:37:57,262][20347] Updated weights for policy 0, policy_version 1730274 (0.0005) [2023-12-27 17:37:57,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 885637120. Throughput: 0: 10318.9, 1: 10427.4. Samples: 885621016. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:37:57,363][19170] Avg episode reward: [(0, '960.840'), (1, '1000.000')] [2023-12-27 17:37:57,697][20381] Updated weights for policy 1, policy_version 1728762 (0.0006) [2023-12-27 17:37:57,748][20381] Updated weights for policy 1, policy_version 1728772 (0.0005) [2023-12-27 17:37:57,804][20381] Updated weights for policy 1, policy_version 1728782 (0.0006) [2023-12-27 17:37:57,894][20347] Updated weights for policy 0, policy_version 1730284 (0.0007) [2023-12-27 17:37:57,949][20347] Updated weights for policy 0, policy_version 1730294 (0.0010) [2023-12-27 17:37:57,997][20347] Updated weights for policy 0, policy_version 1730304 (0.0010) [2023-12-27 17:37:58,478][20381] Updated weights for policy 1, policy_version 1728792 (0.0008) [2023-12-27 17:37:58,533][20381] Updated weights for policy 1, policy_version 1728802 (0.0007) [2023-12-27 17:37:58,604][20381] Updated weights for policy 1, policy_version 1728812 (0.0008) [2023-12-27 17:37:58,690][20347] Updated weights for policy 0, policy_version 1730314 (0.0010) [2023-12-27 17:37:58,758][20347] Updated weights for policy 0, policy_version 1730324 (0.0008) [2023-12-27 17:37:58,821][20347] Updated weights for policy 0, policy_version 1730334 (0.0008) [2023-12-27 17:37:58,888][20347] Updated weights for policy 0, policy_version 1730344 (0.0011) [2023-12-27 17:37:59,443][20381] Updated weights for policy 1, policy_version 1728822 (0.0006) [2023-12-27 17:37:59,511][20381] Updated weights for policy 1, policy_version 1728832 (0.0006) [2023-12-27 17:37:59,572][20381] Updated weights for policy 1, policy_version 1728842 (0.0008) [2023-12-27 17:37:59,726][20347] Updated weights for policy 0, policy_version 1730354 (0.0008) [2023-12-27 17:37:59,786][20347] Updated weights for policy 0, policy_version 1730364 (0.0008) [2023-12-27 17:37:59,846][20347] Updated weights for policy 0, policy_version 1730374 (0.0007) [2023-12-27 17:38:00,229][20381] Updated weights for policy 1, policy_version 1728852 (0.0007) [2023-12-27 17:38:00,287][20381] Updated weights for policy 1, policy_version 1728862 (0.0007) [2023-12-27 17:38:00,346][20381] Updated weights for policy 1, policy_version 1728872 (0.0007) [2023-12-27 17:38:00,542][20347] Updated weights for policy 0, policy_version 1730384 (0.0007) [2023-12-27 17:38:00,600][20347] Updated weights for policy 0, policy_version 1730394 (0.0007) [2023-12-27 17:38:00,658][20347] Updated weights for policy 0, policy_version 1730404 (0.0007) [2023-12-27 17:38:01,018][20381] Updated weights for policy 1, policy_version 1728882 (0.0006) [2023-12-27 17:38:01,082][20381] Updated weights for policy 1, policy_version 1728892 (0.0008) [2023-12-27 17:38:01,147][20381] Updated weights for policy 1, policy_version 1728902 (0.0007) [2023-12-27 17:38:01,200][20381] Updated weights for policy 1, policy_version 1728912 (0.0006) [2023-12-27 17:38:01,367][20347] Updated weights for policy 0, policy_version 1730414 (0.0007) [2023-12-27 17:38:01,424][20347] Updated weights for policy 0, policy_version 1730424 (0.0007) [2023-12-27 17:38:01,471][20347] Updated weights for policy 0, policy_version 1730434 (0.0008) [2023-12-27 17:38:01,778][20381] Updated weights for policy 1, policy_version 1728922 (0.0008) [2023-12-27 17:38:01,835][20381] Updated weights for policy 1, policy_version 1728932 (0.0008) [2023-12-27 17:38:01,894][20381] Updated weights for policy 1, policy_version 1728942 (0.0007) [2023-12-27 17:38:02,231][20347] Updated weights for policy 0, policy_version 1730444 (0.0008) [2023-12-27 17:38:02,289][20347] Updated weights for policy 0, policy_version 1730454 (0.0008) [2023-12-27 17:38:02,354][20347] Updated weights for policy 0, policy_version 1730464 (0.0008) [2023-12-27 17:38:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 885727232. Throughput: 0: 10287.0, 1: 10400.1. Samples: 885742456. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:02,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:38:02,462][20381] Updated weights for policy 1, policy_version 1728952 (0.0007) [2023-12-27 17:38:02,521][20381] Updated weights for policy 1, policy_version 1728962 (0.0008) [2023-12-27 17:38:02,573][20381] Updated weights for policy 1, policy_version 1728972 (0.0007) [2023-12-27 17:38:03,057][20347] Updated weights for policy 0, policy_version 1730474 (0.0007) [2023-12-27 17:38:03,103][20347] Updated weights for policy 0, policy_version 1730484 (0.0007) [2023-12-27 17:38:03,151][20347] Updated weights for policy 0, policy_version 1730494 (0.0007) [2023-12-27 17:38:03,209][20347] Updated weights for policy 0, policy_version 1730504 (0.0007) [2023-12-27 17:38:03,251][20381] Updated weights for policy 1, policy_version 1728982 (0.0008) [2023-12-27 17:38:03,293][20381] Updated weights for policy 1, policy_version 1728992 (0.0006) [2023-12-27 17:38:03,352][20381] Updated weights for policy 1, policy_version 1729002 (0.0005) [2023-12-27 17:38:03,938][20347] Updated weights for policy 0, policy_version 1730514 (0.0007) [2023-12-27 17:38:03,975][20381] Updated weights for policy 1, policy_version 1729012 (0.0005) [2023-12-27 17:38:03,995][20347] Updated weights for policy 0, policy_version 1730524 (0.0007) [2023-12-27 17:38:04,027][20381] Updated weights for policy 1, policy_version 1729022 (0.0006) [2023-12-27 17:38:04,048][20347] Updated weights for policy 0, policy_version 1730534 (0.0006) [2023-12-27 17:38:04,080][20381] Updated weights for policy 1, policy_version 1729032 (0.0007) [2023-12-27 17:38:04,709][20347] Updated weights for policy 0, policy_version 1730544 (0.0007) [2023-12-27 17:38:04,774][20347] Updated weights for policy 0, policy_version 1730554 (0.0007) [2023-12-27 17:38:04,828][20381] Updated weights for policy 1, policy_version 1729042 (0.0007) [2023-12-27 17:38:04,836][20347] Updated weights for policy 0, policy_version 1730564 (0.0007) [2023-12-27 17:38:04,887][20381] Updated weights for policy 1, policy_version 1729052 (0.0008) [2023-12-27 17:38:04,934][20381] Updated weights for policy 1, policy_version 1729062 (0.0009) [2023-12-27 17:38:04,993][20381] Updated weights for policy 1, policy_version 1729072 (0.0007) [2023-12-27 17:38:05,452][20347] Updated weights for policy 0, policy_version 1730574 (0.0005) [2023-12-27 17:38:05,513][20347] Updated weights for policy 0, policy_version 1730584 (0.0005) [2023-12-27 17:38:05,573][20347] Updated weights for policy 0, policy_version 1730594 (0.0007) [2023-12-27 17:38:05,712][20381] Updated weights for policy 1, policy_version 1729082 (0.0008) [2023-12-27 17:38:05,767][20381] Updated weights for policy 1, policy_version 1729092 (0.0007) [2023-12-27 17:38:05,826][20381] Updated weights for policy 1, policy_version 1729102 (0.0008) [2023-12-27 17:38:06,152][20347] Updated weights for policy 0, policy_version 1730604 (0.0008) [2023-12-27 17:38:06,217][20347] Updated weights for policy 0, policy_version 1730614 (0.0006) [2023-12-27 17:38:06,283][20347] Updated weights for policy 0, policy_version 1730624 (0.0005) [2023-12-27 17:38:06,576][20381] Updated weights for policy 1, policy_version 1729112 (0.0008) [2023-12-27 17:38:06,640][20381] Updated weights for policy 1, policy_version 1729122 (0.0008) [2023-12-27 17:38:06,711][20381] Updated weights for policy 1, policy_version 1729132 (0.0008) [2023-12-27 17:38:06,877][20347] Updated weights for policy 0, policy_version 1730634 (0.0006) [2023-12-27 17:38:06,936][20347] Updated weights for policy 0, policy_version 1730644 (0.0005) [2023-12-27 17:38:06,995][20347] Updated weights for policy 0, policy_version 1730654 (0.0008) [2023-12-27 17:38:07,042][20347] Updated weights for policy 0, policy_version 1730664 (0.0007) [2023-12-27 17:38:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 885833728. Throughput: 0: 10270.2, 1: 10401.3. Samples: 885803668. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:07,363][19170] Avg episode reward: [(0, '980.510'), (1, '1000.000')] [2023-12-27 17:38:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001729136_442703872.pth... [2023-12-27 17:38:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001730664_443129856.pth... [2023-12-27 17:38:07,370][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001727952_442400768.pth [2023-12-27 17:38:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001729448_442818560.pth [2023-12-27 17:38:07,445][20381] Updated weights for policy 1, policy_version 1729142 (0.0007) [2023-12-27 17:38:07,509][20381] Updated weights for policy 1, policy_version 1729152 (0.0008) [2023-12-27 17:38:07,557][20381] Updated weights for policy 1, policy_version 1729162 (0.0008) [2023-12-27 17:38:07,639][20347] Updated weights for policy 0, policy_version 1730674 (0.0006) [2023-12-27 17:38:07,691][20347] Updated weights for policy 0, policy_version 1730684 (0.0008) [2023-12-27 17:38:07,749][20347] Updated weights for policy 0, policy_version 1730694 (0.0006) [2023-12-27 17:38:08,271][20381] Updated weights for policy 1, policy_version 1729172 (0.0008) [2023-12-27 17:38:08,325][20381] Updated weights for policy 1, policy_version 1729182 (0.0007) [2023-12-27 17:38:08,376][20347] Updated weights for policy 0, policy_version 1730704 (0.0009) [2023-12-27 17:38:08,387][20381] Updated weights for policy 1, policy_version 1729192 (0.0008) [2023-12-27 17:38:08,429][20347] Updated weights for policy 0, policy_version 1730714 (0.0010) [2023-12-27 17:38:08,476][20347] Updated weights for policy 0, policy_version 1730724 (0.0010) [2023-12-27 17:38:09,100][20381] Updated weights for policy 1, policy_version 1729202 (0.0007) [2023-12-27 17:38:09,155][20381] Updated weights for policy 1, policy_version 1729212 (0.0006) [2023-12-27 17:38:09,156][20347] Updated weights for policy 0, policy_version 1730734 (0.0010) [2023-12-27 17:38:09,210][20381] Updated weights for policy 1, policy_version 1729222 (0.0006) [2023-12-27 17:38:09,215][20347] Updated weights for policy 0, policy_version 1730744 (0.0010) [2023-12-27 17:38:09,272][20381] Updated weights for policy 1, policy_version 1729232 (0.0007) [2023-12-27 17:38:09,278][20347] Updated weights for policy 0, policy_version 1730754 (0.0008) [2023-12-27 17:38:10,002][20347] Updated weights for policy 0, policy_version 1730764 (0.0009) [2023-12-27 17:38:10,006][20381] Updated weights for policy 1, policy_version 1729242 (0.0008) [2023-12-27 17:38:10,061][20381] Updated weights for policy 1, policy_version 1729252 (0.0006) [2023-12-27 17:38:10,061][20347] Updated weights for policy 0, policy_version 1730774 (0.0010) [2023-12-27 17:38:10,109][20381] Updated weights for policy 1, policy_version 1729262 (0.0006) [2023-12-27 17:38:10,122][20347] Updated weights for policy 0, policy_version 1730784 (0.0009) [2023-12-27 17:38:10,751][20347] Updated weights for policy 0, policy_version 1730794 (0.0010) [2023-12-27 17:38:10,807][20347] Updated weights for policy 0, policy_version 1730804 (0.0008) [2023-12-27 17:38:10,817][20381] Updated weights for policy 1, policy_version 1729272 (0.0007) [2023-12-27 17:38:10,860][20347] Updated weights for policy 0, policy_version 1730814 (0.0007) [2023-12-27 17:38:10,883][20381] Updated weights for policy 1, policy_version 1729282 (0.0007) [2023-12-27 17:38:10,909][20347] Updated weights for policy 0, policy_version 1730824 (0.0007) [2023-12-27 17:38:10,947][20381] Updated weights for policy 1, policy_version 1729292 (0.0008) [2023-12-27 17:38:11,674][20381] Updated weights for policy 1, policy_version 1729302 (0.0008) [2023-12-27 17:38:11,725][20347] Updated weights for policy 0, policy_version 1730834 (0.0006) [2023-12-27 17:38:11,737][20381] Updated weights for policy 1, policy_version 1729312 (0.0008) [2023-12-27 17:38:11,791][20347] Updated weights for policy 0, policy_version 1730844 (0.0010) [2023-12-27 17:38:11,795][20381] Updated weights for policy 1, policy_version 1729322 (0.0007) [2023-12-27 17:38:11,847][20347] Updated weights for policy 0, policy_version 1730854 (0.0009) [2023-12-27 17:38:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 885932032. Throughput: 0: 10315.0, 1: 10316.3. Samples: 885925968. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:12,363][19170] Avg episode reward: [(0, '961.000'), (1, '990.360')] [2023-12-27 17:38:12,380][20381] Updated weights for policy 1, policy_version 1729332 (0.0006) [2023-12-27 17:38:12,441][20381] Updated weights for policy 1, policy_version 1729342 (0.0008) [2023-12-27 17:38:12,499][20381] Updated weights for policy 1, policy_version 1729352 (0.0007) [2023-12-27 17:38:12,530][20347] Updated weights for policy 0, policy_version 1730864 (0.0010) [2023-12-27 17:38:12,593][20347] Updated weights for policy 0, policy_version 1730874 (0.0011) [2023-12-27 17:38:12,657][20347] Updated weights for policy 0, policy_version 1730884 (0.0010) [2023-12-27 17:38:13,140][20381] Updated weights for policy 1, policy_version 1729362 (0.0006) [2023-12-27 17:38:13,184][20381] Updated weights for policy 1, policy_version 1729372 (0.0007) [2023-12-27 17:38:13,236][20381] Updated weights for policy 1, policy_version 1729382 (0.0007) [2023-12-27 17:38:13,290][20381] Updated weights for policy 1, policy_version 1729392 (0.0007) [2023-12-27 17:38:13,329][20347] Updated weights for policy 0, policy_version 1730894 (0.0010) [2023-12-27 17:38:13,386][20347] Updated weights for policy 0, policy_version 1730904 (0.0010) [2023-12-27 17:38:13,443][20347] Updated weights for policy 0, policy_version 1730914 (0.0010) [2023-12-27 17:38:14,008][20381] Updated weights for policy 1, policy_version 1729402 (0.0008) [2023-12-27 17:38:14,057][20381] Updated weights for policy 1, policy_version 1729412 (0.0008) [2023-12-27 17:38:14,071][20347] Updated weights for policy 0, policy_version 1730924 (0.0010) [2023-12-27 17:38:14,114][20381] Updated weights for policy 1, policy_version 1729422 (0.0009) [2023-12-27 17:38:14,136][20347] Updated weights for policy 0, policy_version 1730934 (0.0008) [2023-12-27 17:38:14,192][20347] Updated weights for policy 0, policy_version 1730944 (0.0009) [2023-12-27 17:38:14,760][20381] Updated weights for policy 1, policy_version 1729432 (0.0006) [2023-12-27 17:38:14,829][20381] Updated weights for policy 1, policy_version 1729442 (0.0008) [2023-12-27 17:38:14,847][20347] Updated weights for policy 0, policy_version 1730954 (0.0007) [2023-12-27 17:38:14,890][20381] Updated weights for policy 1, policy_version 1729452 (0.0009) [2023-12-27 17:38:14,910][20347] Updated weights for policy 0, policy_version 1730964 (0.0006) [2023-12-27 17:38:14,973][20347] Updated weights for policy 0, policy_version 1730974 (0.0006) [2023-12-27 17:38:15,036][20347] Updated weights for policy 0, policy_version 1730984 (0.0006) [2023-12-27 17:38:15,587][20381] Updated weights for policy 1, policy_version 1729462 (0.0009) [2023-12-27 17:38:15,605][20347] Updated weights for policy 0, policy_version 1730994 (0.0011) [2023-12-27 17:38:15,644][20381] Updated weights for policy 1, policy_version 1729472 (0.0005) [2023-12-27 17:38:15,656][20347] Updated weights for policy 0, policy_version 1731004 (0.0010) [2023-12-27 17:38:15,701][20381] Updated weights for policy 1, policy_version 1729482 (0.0005) [2023-12-27 17:38:15,715][20347] Updated weights for policy 0, policy_version 1731014 (0.0010) [2023-12-27 17:38:16,323][20381] Updated weights for policy 1, policy_version 1729492 (0.0008) [2023-12-27 17:38:16,340][20347] Updated weights for policy 0, policy_version 1731024 (0.0008) [2023-12-27 17:38:16,372][20381] Updated weights for policy 1, policy_version 1729502 (0.0010) [2023-12-27 17:38:16,403][20347] Updated weights for policy 0, policy_version 1731034 (0.0008) [2023-12-27 17:38:16,420][20381] Updated weights for policy 1, policy_version 1729512 (0.0010) [2023-12-27 17:38:16,467][20347] Updated weights for policy 0, policy_version 1731044 (0.0008) [2023-12-27 17:38:17,091][20381] Updated weights for policy 1, policy_version 1729522 (0.0009) [2023-12-27 17:38:17,094][20347] Updated weights for policy 0, policy_version 1731054 (0.0006) [2023-12-27 17:38:17,145][20381] Updated weights for policy 1, policy_version 1729532 (0.0007) [2023-12-27 17:38:17,154][20347] Updated weights for policy 0, policy_version 1731064 (0.0007) [2023-12-27 17:38:17,197][20381] Updated weights for policy 1, policy_version 1729542 (0.0005) [2023-12-27 17:38:17,210][20347] Updated weights for policy 0, policy_version 1731074 (0.0010) [2023-12-27 17:38:17,250][20381] Updated weights for policy 1, policy_version 1729552 (0.0006) [2023-12-27 17:38:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.2, 300 sec: 20605.0). Total num frames: 886046720. Throughput: 0: 10342.8, 1: 10334.1. Samples: 886051464. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:17,363][19170] Avg episode reward: [(0, '921.900'), (1, '990.360')] [2023-12-27 17:38:17,827][20381] Updated weights for policy 1, policy_version 1729562 (0.0007) [2023-12-27 17:38:17,860][20347] Updated weights for policy 0, policy_version 1731084 (0.0010) [2023-12-27 17:38:17,877][20381] Updated weights for policy 1, policy_version 1729572 (0.0006) [2023-12-27 17:38:17,912][20347] Updated weights for policy 0, policy_version 1731094 (0.0007) [2023-12-27 17:38:17,930][20381] Updated weights for policy 1, policy_version 1729582 (0.0008) [2023-12-27 17:38:17,967][20347] Updated weights for policy 0, policy_version 1731104 (0.0006) [2023-12-27 17:38:18,609][20347] Updated weights for policy 0, policy_version 1731114 (0.0011) [2023-12-27 17:38:18,660][20381] Updated weights for policy 1, policy_version 1729592 (0.0007) [2023-12-27 17:38:18,668][20347] Updated weights for policy 0, policy_version 1731124 (0.0010) [2023-12-27 17:38:18,714][20381] Updated weights for policy 1, policy_version 1729602 (0.0007) [2023-12-27 17:38:18,723][20347] Updated weights for policy 0, policy_version 1731134 (0.0011) [2023-12-27 17:38:18,768][20381] Updated weights for policy 1, policy_version 1729612 (0.0009) [2023-12-27 17:38:18,779][20347] Updated weights for policy 0, policy_version 1731144 (0.0010) [2023-12-27 17:38:19,406][20381] Updated weights for policy 1, policy_version 1729622 (0.0007) [2023-12-27 17:38:19,463][20381] Updated weights for policy 1, policy_version 1729632 (0.0006) [2023-12-27 17:38:19,512][20347] Updated weights for policy 0, policy_version 1731154 (0.0007) [2023-12-27 17:38:19,527][20381] Updated weights for policy 1, policy_version 1729642 (0.0008) [2023-12-27 17:38:19,579][20347] Updated weights for policy 0, policy_version 1731164 (0.0007) [2023-12-27 17:38:19,643][20347] Updated weights for policy 0, policy_version 1731174 (0.0008) [2023-12-27 17:38:20,211][20381] Updated weights for policy 1, policy_version 1729652 (0.0006) [2023-12-27 17:38:20,274][20381] Updated weights for policy 1, policy_version 1729662 (0.0009) [2023-12-27 17:38:20,335][20381] Updated weights for policy 1, policy_version 1729672 (0.0008) [2023-12-27 17:38:20,418][20347] Updated weights for policy 0, policy_version 1731184 (0.0009) [2023-12-27 17:38:20,477][20347] Updated weights for policy 0, policy_version 1731194 (0.0009) [2023-12-27 17:38:20,533][20347] Updated weights for policy 0, policy_version 1731204 (0.0010) [2023-12-27 17:38:21,015][20381] Updated weights for policy 1, policy_version 1729682 (0.0007) [2023-12-27 17:38:21,085][20381] Updated weights for policy 1, policy_version 1729692 (0.0007) [2023-12-27 17:38:21,148][20381] Updated weights for policy 1, policy_version 1729702 (0.0008) [2023-12-27 17:38:21,209][20381] Updated weights for policy 1, policy_version 1729712 (0.0008) [2023-12-27 17:38:21,242][20347] Updated weights for policy 0, policy_version 1731214 (0.0008) [2023-12-27 17:38:21,304][20347] Updated weights for policy 0, policy_version 1731224 (0.0008) [2023-12-27 17:38:21,368][20347] Updated weights for policy 0, policy_version 1731234 (0.0008) [2023-12-27 17:38:21,925][20381] Updated weights for policy 1, policy_version 1729722 (0.0008) [2023-12-27 17:38:21,980][20381] Updated weights for policy 1, policy_version 1729732 (0.0008) [2023-12-27 17:38:22,039][20381] Updated weights for policy 1, policy_version 1729742 (0.0008) [2023-12-27 17:38:22,074][20347] Updated weights for policy 0, policy_version 1731244 (0.0008) [2023-12-27 17:38:22,135][20347] Updated weights for policy 0, policy_version 1731254 (0.0008) [2023-12-27 17:38:22,195][20347] Updated weights for policy 0, policy_version 1731264 (0.0008) [2023-12-27 17:38:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 886145024. Throughput: 0: 10294.6, 1: 10337.6. Samples: 886113012. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:22,363][19170] Avg episode reward: [(0, '921.730'), (1, '1000.000')] [2023-12-27 17:38:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001729744_442859520.pth... [2023-12-27 17:38:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001731272_443285504.pth... [2023-12-27 17:38:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001728560_442556416.pth [2023-12-27 17:38:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001730056_442974208.pth [2023-12-27 17:38:22,777][20381] Updated weights for policy 1, policy_version 1729752 (0.0007) [2023-12-27 17:38:22,840][20381] Updated weights for policy 1, policy_version 1729762 (0.0008) [2023-12-27 17:38:22,898][20381] Updated weights for policy 1, policy_version 1729772 (0.0008) [2023-12-27 17:38:22,928][20347] Updated weights for policy 0, policy_version 1731274 (0.0007) [2023-12-27 17:38:22,984][20347] Updated weights for policy 0, policy_version 1731284 (0.0007) [2023-12-27 17:38:23,039][20347] Updated weights for policy 0, policy_version 1731294 (0.0008) [2023-12-27 17:38:23,096][20347] Updated weights for policy 0, policy_version 1731304 (0.0005) [2023-12-27 17:38:23,522][20381] Updated weights for policy 1, policy_version 1729782 (0.0006) [2023-12-27 17:38:23,567][20381] Updated weights for policy 1, policy_version 1729792 (0.0005) [2023-12-27 17:38:23,615][20381] Updated weights for policy 1, policy_version 1729802 (0.0006) [2023-12-27 17:38:23,660][20347] Updated weights for policy 0, policy_version 1731314 (0.0010) [2023-12-27 17:38:23,711][20347] Updated weights for policy 0, policy_version 1731324 (0.0007) [2023-12-27 17:38:23,754][20347] Updated weights for policy 0, policy_version 1731334 (0.0005) [2023-12-27 17:38:24,205][20381] Updated weights for policy 1, policy_version 1729812 (0.0007) [2023-12-27 17:38:24,265][20381] Updated weights for policy 1, policy_version 1729822 (0.0007) [2023-12-27 17:38:24,323][20381] Updated weights for policy 1, policy_version 1729832 (0.0007) [2023-12-27 17:38:24,466][20347] Updated weights for policy 0, policy_version 1731344 (0.0008) [2023-12-27 17:38:24,529][20347] Updated weights for policy 0, policy_version 1731354 (0.0008) [2023-12-27 17:38:24,588][20347] Updated weights for policy 0, policy_version 1731364 (0.0008) [2023-12-27 17:38:24,917][20381] Updated weights for policy 1, policy_version 1729842 (0.0006) [2023-12-27 17:38:24,976][20381] Updated weights for policy 1, policy_version 1729852 (0.0006) [2023-12-27 17:38:25,032][20381] Updated weights for policy 1, policy_version 1729862 (0.0005) [2023-12-27 17:38:25,090][20381] Updated weights for policy 1, policy_version 1729872 (0.0005) [2023-12-27 17:38:25,301][20347] Updated weights for policy 0, policy_version 1731374 (0.0008) [2023-12-27 17:38:25,368][20347] Updated weights for policy 0, policy_version 1731384 (0.0008) [2023-12-27 17:38:25,428][20347] Updated weights for policy 0, policy_version 1731394 (0.0008) [2023-12-27 17:38:25,631][20381] Updated weights for policy 1, policy_version 1729882 (0.0008) [2023-12-27 17:38:25,695][20381] Updated weights for policy 1, policy_version 1729892 (0.0007) [2023-12-27 17:38:25,756][20381] Updated weights for policy 1, policy_version 1729902 (0.0006) [2023-12-27 17:38:26,000][20347] Updated weights for policy 0, policy_version 1731404 (0.0008) [2023-12-27 17:38:26,060][20347] Updated weights for policy 0, policy_version 1731414 (0.0008) [2023-12-27 17:38:26,119][20347] Updated weights for policy 0, policy_version 1731424 (0.0008) [2023-12-27 17:38:26,437][20381] Updated weights for policy 1, policy_version 1729912 (0.0008) [2023-12-27 17:38:26,494][20381] Updated weights for policy 1, policy_version 1729922 (0.0008) [2023-12-27 17:38:26,545][20381] Updated weights for policy 1, policy_version 1729932 (0.0009) [2023-12-27 17:38:26,802][20347] Updated weights for policy 0, policy_version 1731434 (0.0007) [2023-12-27 17:38:26,851][20347] Updated weights for policy 0, policy_version 1731444 (0.0005) [2023-12-27 17:38:26,911][20347] Updated weights for policy 0, policy_version 1731454 (0.0005) [2023-12-27 17:38:26,972][20347] Updated weights for policy 0, policy_version 1731464 (0.0005) [2023-12-27 17:38:27,245][20381] Updated weights for policy 1, policy_version 1729942 (0.0008) [2023-12-27 17:38:27,305][20381] Updated weights for policy 1, policy_version 1729952 (0.0008) [2023-12-27 17:38:27,358][20381] Updated weights for policy 1, policy_version 1729962 (0.0008) [2023-12-27 17:38:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 886243328. Throughput: 0: 10336.5, 1: 10335.8. Samples: 886237772. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:27,363][19170] Avg episode reward: [(0, '951.140'), (1, '1000.000')] [2023-12-27 17:38:27,545][20347] Updated weights for policy 0, policy_version 1731474 (0.0008) [2023-12-27 17:38:27,596][20347] Updated weights for policy 0, policy_version 1731484 (0.0008) [2023-12-27 17:38:27,648][20347] Updated weights for policy 0, policy_version 1731494 (0.0008) [2023-12-27 17:38:28,020][20381] Updated weights for policy 1, policy_version 1729972 (0.0007) [2023-12-27 17:38:28,070][20381] Updated weights for policy 1, policy_version 1729982 (0.0005) [2023-12-27 17:38:28,131][20381] Updated weights for policy 1, policy_version 1729992 (0.0005) [2023-12-27 17:38:28,383][20347] Updated weights for policy 0, policy_version 1731504 (0.0008) [2023-12-27 17:38:28,436][20347] Updated weights for policy 0, policy_version 1731514 (0.0007) [2023-12-27 17:38:28,496][20347] Updated weights for policy 0, policy_version 1731524 (0.0008) [2023-12-27 17:38:28,719][20381] Updated weights for policy 1, policy_version 1730002 (0.0006) [2023-12-27 17:38:28,780][20381] Updated weights for policy 1, policy_version 1730012 (0.0005) [2023-12-27 17:38:28,845][20381] Updated weights for policy 1, policy_version 1730022 (0.0007) [2023-12-27 17:38:28,903][20381] Updated weights for policy 1, policy_version 1730032 (0.0007) [2023-12-27 17:38:29,242][20347] Updated weights for policy 0, policy_version 1731534 (0.0007) [2023-12-27 17:38:29,297][20347] Updated weights for policy 0, policy_version 1731544 (0.0008) [2023-12-27 17:38:29,362][20347] Updated weights for policy 0, policy_version 1731554 (0.0009) [2023-12-27 17:38:29,572][20381] Updated weights for policy 1, policy_version 1730042 (0.0008) [2023-12-27 17:38:29,625][20381] Updated weights for policy 1, policy_version 1730052 (0.0008) [2023-12-27 17:38:29,672][20381] Updated weights for policy 1, policy_version 1730062 (0.0007) [2023-12-27 17:38:30,050][20347] Updated weights for policy 0, policy_version 1731564 (0.0008) [2023-12-27 17:38:30,119][20347] Updated weights for policy 0, policy_version 1731574 (0.0007) [2023-12-27 17:38:30,192][20347] Updated weights for policy 0, policy_version 1731584 (0.0006) [2023-12-27 17:38:30,400][20381] Updated weights for policy 1, policy_version 1730072 (0.0008) [2023-12-27 17:38:30,468][20381] Updated weights for policy 1, policy_version 1730082 (0.0007) [2023-12-27 17:38:30,525][20381] Updated weights for policy 1, policy_version 1730092 (0.0007) [2023-12-27 17:38:30,849][20347] Updated weights for policy 0, policy_version 1731594 (0.0006) [2023-12-27 17:38:30,903][20347] Updated weights for policy 0, policy_version 1731604 (0.0007) [2023-12-27 17:38:30,952][20347] Updated weights for policy 0, policy_version 1731614 (0.0007) [2023-12-27 17:38:31,006][20347] Updated weights for policy 0, policy_version 1731624 (0.0007) [2023-12-27 17:38:31,209][20381] Updated weights for policy 1, policy_version 1730102 (0.0008) [2023-12-27 17:38:31,271][20381] Updated weights for policy 1, policy_version 1730112 (0.0007) [2023-12-27 17:38:31,332][20381] Updated weights for policy 1, policy_version 1730122 (0.0006) [2023-12-27 17:38:31,812][20347] Updated weights for policy 0, policy_version 1731634 (0.0007) [2023-12-27 17:38:31,872][20347] Updated weights for policy 0, policy_version 1731644 (0.0007) [2023-12-27 17:38:31,931][20347] Updated weights for policy 0, policy_version 1731654 (0.0007) [2023-12-27 17:38:32,030][20381] Updated weights for policy 1, policy_version 1730132 (0.0008) [2023-12-27 17:38:32,084][20381] Updated weights for policy 1, policy_version 1730142 (0.0008) [2023-12-27 17:38:32,141][20381] Updated weights for policy 1, policy_version 1730152 (0.0008) [2023-12-27 17:38:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 886349824. Throughput: 0: 10252.3, 1: 10290.4. Samples: 886358260. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:32,363][19170] Avg episode reward: [(0, '980.520'), (1, '1000.000')] [2023-12-27 17:38:32,616][20347] Updated weights for policy 0, policy_version 1731664 (0.0006) [2023-12-27 17:38:32,665][20347] Updated weights for policy 0, policy_version 1731674 (0.0008) [2023-12-27 17:38:32,721][20347] Updated weights for policy 0, policy_version 1731684 (0.0008) [2023-12-27 17:38:32,802][20381] Updated weights for policy 1, policy_version 1730162 (0.0006) [2023-12-27 17:38:32,856][20381] Updated weights for policy 1, policy_version 1730172 (0.0005) [2023-12-27 17:38:32,910][20381] Updated weights for policy 1, policy_version 1730182 (0.0005) [2023-12-27 17:38:32,961][20381] Updated weights for policy 1, policy_version 1730192 (0.0005) [2023-12-27 17:38:33,471][20347] Updated weights for policy 0, policy_version 1731694 (0.0008) [2023-12-27 17:38:33,493][20381] Updated weights for policy 1, policy_version 1730202 (0.0006) [2023-12-27 17:38:33,520][20347] Updated weights for policy 0, policy_version 1731704 (0.0008) [2023-12-27 17:38:33,549][20381] Updated weights for policy 1, policy_version 1730212 (0.0005) [2023-12-27 17:38:33,569][20347] Updated weights for policy 0, policy_version 1731714 (0.0008) [2023-12-27 17:38:33,604][20381] Updated weights for policy 1, policy_version 1730222 (0.0005) [2023-12-27 17:38:34,191][20381] Updated weights for policy 1, policy_version 1730232 (0.0010) [2023-12-27 17:38:34,240][20381] Updated weights for policy 1, policy_version 1730242 (0.0010) [2023-12-27 17:38:34,291][20347] Updated weights for policy 0, policy_version 1731724 (0.0008) [2023-12-27 17:38:34,298][20381] Updated weights for policy 1, policy_version 1730252 (0.0011) [2023-12-27 17:38:34,352][20347] Updated weights for policy 0, policy_version 1731734 (0.0008) [2023-12-27 17:38:34,416][20347] Updated weights for policy 0, policy_version 1731744 (0.0007) [2023-12-27 17:38:34,928][20381] Updated weights for policy 1, policy_version 1730262 (0.0008) [2023-12-27 17:38:34,984][20381] Updated weights for policy 1, policy_version 1730272 (0.0005) [2023-12-27 17:38:35,044][20381] Updated weights for policy 1, policy_version 1730282 (0.0007) [2023-12-27 17:38:35,145][20347] Updated weights for policy 0, policy_version 1731754 (0.0008) [2023-12-27 17:38:35,205][20347] Updated weights for policy 0, policy_version 1731764 (0.0007) [2023-12-27 17:38:35,261][20347] Updated weights for policy 0, policy_version 1731774 (0.0007) [2023-12-27 17:38:35,315][20347] Updated weights for policy 0, policy_version 1731784 (0.0007) [2023-12-27 17:38:35,701][20381] Updated weights for policy 1, policy_version 1730292 (0.0008) [2023-12-27 17:38:35,762][20381] Updated weights for policy 1, policy_version 1730302 (0.0010) [2023-12-27 17:38:35,814][20381] Updated weights for policy 1, policy_version 1730312 (0.0010) [2023-12-27 17:38:36,015][20347] Updated weights for policy 0, policy_version 1731794 (0.0008) [2023-12-27 17:38:36,068][20347] Updated weights for policy 0, policy_version 1731804 (0.0008) [2023-12-27 17:38:36,143][20347] Updated weights for policy 0, policy_version 1731814 (0.0008) [2023-12-27 17:38:36,423][20381] Updated weights for policy 1, policy_version 1730322 (0.0008) [2023-12-27 17:38:36,487][20381] Updated weights for policy 1, policy_version 1730332 (0.0007) [2023-12-27 17:38:36,547][20381] Updated weights for policy 1, policy_version 1730342 (0.0007) [2023-12-27 17:38:36,608][20381] Updated weights for policy 1, policy_version 1730352 (0.0007) [2023-12-27 17:38:36,929][20347] Updated weights for policy 0, policy_version 1731824 (0.0008) [2023-12-27 17:38:36,993][20347] Updated weights for policy 0, policy_version 1731834 (0.0008) [2023-12-27 17:38:37,056][20347] Updated weights for policy 0, policy_version 1731844 (0.0008) [2023-12-27 17:38:37,177][20381] Updated weights for policy 1, policy_version 1730362 (0.0008) [2023-12-27 17:38:37,230][20381] Updated weights for policy 1, policy_version 1730372 (0.0008) [2023-12-27 17:38:37,280][20381] Updated weights for policy 1, policy_version 1730382 (0.0007) [2023-12-27 17:38:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 886456320. Throughput: 0: 10221.8, 1: 10397.0. Samples: 886421680. Policy #0 lag: (min: 5.0, avg: 11.2, max: 37.0) [2023-12-27 17:38:37,363][19170] Avg episode reward: [(0, '990.250'), (1, '990.300')] [2023-12-27 17:38:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001730384_443023360.pth... [2023-12-27 17:38:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001731848_443432960.pth... [2023-12-27 17:38:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001730664_443129856.pth [2023-12-27 17:38:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001729136_442703872.pth [2023-12-27 17:38:37,758][20347] Updated weights for policy 0, policy_version 1731854 (0.0008) [2023-12-27 17:38:37,817][20347] Updated weights for policy 0, policy_version 1731864 (0.0007) [2023-12-27 17:38:37,875][20347] Updated weights for policy 0, policy_version 1731874 (0.0007) [2023-12-27 17:38:37,968][20381] Updated weights for policy 1, policy_version 1730392 (0.0010) [2023-12-27 17:38:38,020][20381] Updated weights for policy 1, policy_version 1730402 (0.0010) [2023-12-27 17:38:38,078][20381] Updated weights for policy 1, policy_version 1730412 (0.0010) [2023-12-27 17:38:38,587][20347] Updated weights for policy 0, policy_version 1731884 (0.0007) [2023-12-27 17:38:38,646][20347] Updated weights for policy 0, policy_version 1731894 (0.0007) [2023-12-27 17:38:38,701][20347] Updated weights for policy 0, policy_version 1731904 (0.0008) [2023-12-27 17:38:38,757][20381] Updated weights for policy 1, policy_version 1730422 (0.0007) [2023-12-27 17:38:38,817][20381] Updated weights for policy 1, policy_version 1730432 (0.0006) [2023-12-27 17:38:38,878][20381] Updated weights for policy 1, policy_version 1730442 (0.0008) [2023-12-27 17:38:39,423][20347] Updated weights for policy 0, policy_version 1731914 (0.0007) [2023-12-27 17:38:39,488][20347] Updated weights for policy 0, policy_version 1731924 (0.0007) [2023-12-27 17:38:39,546][20347] Updated weights for policy 0, policy_version 1731934 (0.0007) [2023-12-27 17:38:39,551][20381] Updated weights for policy 1, policy_version 1730452 (0.0009) [2023-12-27 17:38:39,605][20347] Updated weights for policy 0, policy_version 1731944 (0.0006) [2023-12-27 17:38:39,611][20381] Updated weights for policy 1, policy_version 1730462 (0.0010) [2023-12-27 17:38:39,667][20381] Updated weights for policy 1, policy_version 1730472 (0.0011) [2023-12-27 17:38:40,359][20347] Updated weights for policy 0, policy_version 1731954 (0.0008) [2023-12-27 17:38:40,371][20381] Updated weights for policy 1, policy_version 1730482 (0.0010) [2023-12-27 17:38:40,421][20347] Updated weights for policy 0, policy_version 1731964 (0.0005) [2023-12-27 17:38:40,427][20381] Updated weights for policy 1, policy_version 1730492 (0.0011) [2023-12-27 17:38:40,477][20347] Updated weights for policy 0, policy_version 1731974 (0.0006) [2023-12-27 17:38:40,485][20381] Updated weights for policy 1, policy_version 1730502 (0.0010) [2023-12-27 17:38:40,541][20381] Updated weights for policy 1, policy_version 1730512 (0.0010) [2023-12-27 17:38:41,187][20347] Updated weights for policy 0, policy_version 1731984 (0.0007) [2023-12-27 17:38:41,249][20347] Updated weights for policy 0, policy_version 1731994 (0.0008) [2023-12-27 17:38:41,314][20381] Updated weights for policy 1, policy_version 1730522 (0.0011) [2023-12-27 17:38:41,314][20347] Updated weights for policy 0, policy_version 1732004 (0.0007) [2023-12-27 17:38:41,381][20381] Updated weights for policy 1, policy_version 1730532 (0.0010) [2023-12-27 17:38:41,446][20381] Updated weights for policy 1, policy_version 1730542 (0.0007) [2023-12-27 17:38:42,110][20347] Updated weights for policy 0, policy_version 1732014 (0.0007) [2023-12-27 17:38:42,151][20381] Updated weights for policy 1, policy_version 1730552 (0.0010) [2023-12-27 17:38:42,171][20347] Updated weights for policy 0, policy_version 1732024 (0.0007) [2023-12-27 17:38:42,214][20381] Updated weights for policy 1, policy_version 1730562 (0.0011) [2023-12-27 17:38:42,231][20347] Updated weights for policy 0, policy_version 1732034 (0.0008) [2023-12-27 17:38:42,281][20381] Updated weights for policy 1, policy_version 1730572 (0.0010) [2023-12-27 17:38:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 886554624. Throughput: 0: 10111.7, 1: 10337.8. Samples: 886541244. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:38:42,363][19170] Avg episode reward: [(0, '980.450'), (1, '990.300')] [2023-12-27 17:38:42,939][20347] Updated weights for policy 0, policy_version 1732044 (0.0008) [2023-12-27 17:38:42,963][20381] Updated weights for policy 1, policy_version 1730582 (0.0009) [2023-12-27 17:38:42,987][20347] Updated weights for policy 0, policy_version 1732054 (0.0005) [2023-12-27 17:38:43,020][20381] Updated weights for policy 1, policy_version 1730592 (0.0010) [2023-12-27 17:38:43,033][20347] Updated weights for policy 0, policy_version 1732064 (0.0008) [2023-12-27 17:38:43,071][20381] Updated weights for policy 1, policy_version 1730602 (0.0010) [2023-12-27 17:38:43,740][20347] Updated weights for policy 0, policy_version 1732074 (0.0006) [2023-12-27 17:38:43,758][20381] Updated weights for policy 1, policy_version 1730612 (0.0010) [2023-12-27 17:38:43,789][20347] Updated weights for policy 0, policy_version 1732084 (0.0006) [2023-12-27 17:38:43,813][20381] Updated weights for policy 1, policy_version 1730622 (0.0010) [2023-12-27 17:38:43,832][20347] Updated weights for policy 0, policy_version 1732094 (0.0007) [2023-12-27 17:38:43,870][20381] Updated weights for policy 1, policy_version 1730632 (0.0010) [2023-12-27 17:38:43,884][20347] Updated weights for policy 0, policy_version 1732104 (0.0005) [2023-12-27 17:38:44,533][20381] Updated weights for policy 1, policy_version 1730642 (0.0010) [2023-12-27 17:38:44,587][20381] Updated weights for policy 1, policy_version 1730652 (0.0010) [2023-12-27 17:38:44,611][20347] Updated weights for policy 0, policy_version 1732114 (0.0005) [2023-12-27 17:38:44,642][20381] Updated weights for policy 1, policy_version 1730662 (0.0010) [2023-12-27 17:38:44,662][20347] Updated weights for policy 0, policy_version 1732124 (0.0005) [2023-12-27 17:38:44,699][20381] Updated weights for policy 1, policy_version 1730672 (0.0010) [2023-12-27 17:38:44,716][20347] Updated weights for policy 0, policy_version 1732134 (0.0006) [2023-12-27 17:38:45,400][20381] Updated weights for policy 1, policy_version 1730682 (0.0010) [2023-12-27 17:38:45,434][20347] Updated weights for policy 0, policy_version 1732144 (0.0006) [2023-12-27 17:38:45,452][20381] Updated weights for policy 1, policy_version 1730692 (0.0010) [2023-12-27 17:38:45,486][20347] Updated weights for policy 0, policy_version 1732154 (0.0005) [2023-12-27 17:38:45,510][20381] Updated weights for policy 1, policy_version 1730702 (0.0010) [2023-12-27 17:38:45,546][20347] Updated weights for policy 0, policy_version 1732164 (0.0006) [2023-12-27 17:38:46,201][20381] Updated weights for policy 1, policy_version 1730712 (0.0010) [2023-12-27 17:38:46,259][20381] Updated weights for policy 1, policy_version 1730722 (0.0010) [2023-12-27 17:38:46,287][20347] Updated weights for policy 0, policy_version 1732174 (0.0008) [2023-12-27 17:38:46,313][20381] Updated weights for policy 1, policy_version 1730732 (0.0010) [2023-12-27 17:38:46,340][20347] Updated weights for policy 0, policy_version 1732184 (0.0008) [2023-12-27 17:38:46,391][20347] Updated weights for policy 0, policy_version 1732194 (0.0007) [2023-12-27 17:38:47,001][20381] Updated weights for policy 1, policy_version 1730742 (0.0010) [2023-12-27 17:38:47,055][20381] Updated weights for policy 1, policy_version 1730752 (0.0010) [2023-12-27 17:38:47,103][20381] Updated weights for policy 1, policy_version 1730762 (0.0010) [2023-12-27 17:38:47,110][20347] Updated weights for policy 0, policy_version 1732204 (0.0008) [2023-12-27 17:38:47,166][20347] Updated weights for policy 0, policy_version 1732214 (0.0008) [2023-12-27 17:38:47,223][20347] Updated weights for policy 0, policy_version 1732224 (0.0008) [2023-12-27 17:38:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 886652928. Throughput: 0: 10057.1, 1: 10308.6. Samples: 886658912. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:38:47,363][19170] Avg episode reward: [(0, '970.610'), (1, '1000.000')] [2023-12-27 17:38:47,710][20381] Updated weights for policy 1, policy_version 1730772 (0.0008) [2023-12-27 17:38:47,766][20381] Updated weights for policy 1, policy_version 1730782 (0.0007) [2023-12-27 17:38:47,825][20381] Updated weights for policy 1, policy_version 1730792 (0.0007) [2023-12-27 17:38:47,931][20347] Updated weights for policy 0, policy_version 1732234 (0.0007) [2023-12-27 17:38:47,981][20347] Updated weights for policy 0, policy_version 1732244 (0.0007) [2023-12-27 17:38:48,032][20347] Updated weights for policy 0, policy_version 1732254 (0.0007) [2023-12-27 17:38:48,086][20347] Updated weights for policy 0, policy_version 1732264 (0.0007) [2023-12-27 17:38:48,513][20381] Updated weights for policy 1, policy_version 1730802 (0.0009) [2023-12-27 17:38:48,568][20381] Updated weights for policy 1, policy_version 1730812 (0.0010) [2023-12-27 17:38:48,629][20381] Updated weights for policy 1, policy_version 1730822 (0.0011) [2023-12-27 17:38:48,690][20381] Updated weights for policy 1, policy_version 1730832 (0.0011) [2023-12-27 17:38:48,795][20347] Updated weights for policy 0, policy_version 1732274 (0.0008) [2023-12-27 17:38:48,847][20347] Updated weights for policy 0, policy_version 1732284 (0.0007) [2023-12-27 17:38:48,905][20347] Updated weights for policy 0, policy_version 1732294 (0.0007) [2023-12-27 17:38:49,376][20381] Updated weights for policy 1, policy_version 1730843 (0.0009) [2023-12-27 17:38:49,402][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-27 17:38:49,664][20347] Updated weights for policy 0, policy_version 1732304 (0.0007) [2023-12-27 17:38:49,716][20347] Updated weights for policy 0, policy_version 1732314 (0.0007) [2023-12-27 17:38:49,778][20347] Updated weights for policy 0, policy_version 1732324 (0.0008) [2023-12-27 17:38:50,113][20381] Updated weights for policy 1, policy_version 1730853 (0.0007) [2023-12-27 17:38:50,176][20381] Updated weights for policy 1, policy_version 1730863 (0.0008) [2023-12-27 17:38:50,240][20381] Updated weights for policy 1, policy_version 1730873 (0.0008) [2023-12-27 17:38:50,532][20347] Updated weights for policy 0, policy_version 1732334 (0.0008) [2023-12-27 17:38:50,606][20347] Updated weights for policy 0, policy_version 1732344 (0.0008) [2023-12-27 17:38:50,665][20347] Updated weights for policy 0, policy_version 1732354 (0.0007) [2023-12-27 17:38:50,926][20381] Updated weights for policy 1, policy_version 1730883 (0.0008) [2023-12-27 17:38:50,978][20381] Updated weights for policy 1, policy_version 1730893 (0.0008) [2023-12-27 17:38:51,040][20381] Updated weights for policy 1, policy_version 1730903 (0.0008) [2023-12-27 17:38:51,305][20347] Updated weights for policy 0, policy_version 1732364 (0.0007) [2023-12-27 17:38:51,377][20347] Updated weights for policy 0, policy_version 1732374 (0.0008) [2023-12-27 17:38:51,432][20347] Updated weights for policy 0, policy_version 1732384 (0.0008) [2023-12-27 17:38:51,823][20381] Updated weights for policy 1, policy_version 1730913 (0.0008) [2023-12-27 17:38:51,876][20381] Updated weights for policy 1, policy_version 1730923 (0.0008) [2023-12-27 17:38:51,924][20381] Updated weights for policy 1, policy_version 1730933 (0.0008) [2023-12-27 17:38:51,978][20381] Updated weights for policy 1, policy_version 1730943 (0.0008) [2023-12-27 17:38:52,189][20347] Updated weights for policy 0, policy_version 1732394 (0.0008) [2023-12-27 17:38:52,241][20347] Updated weights for policy 0, policy_version 1732404 (0.0008) [2023-12-27 17:38:52,307][20347] Updated weights for policy 0, policy_version 1732414 (0.0008) [2023-12-27 17:38:52,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 886743040. Throughput: 0: 10019.6, 1: 10313.7. Samples: 886718668. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:38:52,363][19170] Avg episode reward: [(0, '960.810'), (1, '1000.000')] [2023-12-27 17:38:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001730944_443170816.pth... [2023-12-27 17:38:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001732424_443580416.pth... [2023-12-27 17:38:52,372][20347] Updated weights for policy 0, policy_version 1732424 (0.0007) [2023-12-27 17:38:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001729744_442859520.pth [2023-12-27 17:38:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001731272_443285504.pth [2023-12-27 17:38:52,685][20381] Updated weights for policy 1, policy_version 1730953 (0.0006) [2023-12-27 17:38:52,752][20381] Updated weights for policy 1, policy_version 1730963 (0.0007) [2023-12-27 17:38:52,816][20381] Updated weights for policy 1, policy_version 1730973 (0.0007) [2023-12-27 17:38:53,105][20347] Updated weights for policy 0, policy_version 1732434 (0.0007) [2023-12-27 17:38:53,158][20347] Updated weights for policy 0, policy_version 1732444 (0.0007) [2023-12-27 17:38:53,215][20347] Updated weights for policy 0, policy_version 1732454 (0.0009) [2023-12-27 17:38:53,416][20381] Updated weights for policy 1, policy_version 1730983 (0.0007) [2023-12-27 17:38:53,474][20381] Updated weights for policy 1, policy_version 1730993 (0.0007) [2023-12-27 17:38:53,527][20381] Updated weights for policy 1, policy_version 1731003 (0.0008) [2023-12-27 17:38:53,898][20347] Updated weights for policy 0, policy_version 1732464 (0.0010) [2023-12-27 17:38:53,946][20347] Updated weights for policy 0, policy_version 1732474 (0.0008) [2023-12-27 17:38:54,004][20347] Updated weights for policy 0, policy_version 1732484 (0.0006) [2023-12-27 17:38:54,100][20381] Updated weights for policy 1, policy_version 1731013 (0.0007) [2023-12-27 17:38:54,157][20381] Updated weights for policy 1, policy_version 1731023 (0.0007) [2023-12-27 17:38:54,220][20381] Updated weights for policy 1, policy_version 1731033 (0.0008) [2023-12-27 17:38:54,666][20347] Updated weights for policy 0, policy_version 1732494 (0.0006) [2023-12-27 17:38:54,721][20347] Updated weights for policy 0, policy_version 1732504 (0.0010) [2023-12-27 17:38:54,777][20347] Updated weights for policy 0, policy_version 1732514 (0.0010) [2023-12-27 17:38:54,907][20381] Updated weights for policy 1, policy_version 1731043 (0.0007) [2023-12-27 17:38:54,955][20381] Updated weights for policy 1, policy_version 1731053 (0.0005) [2023-12-27 17:38:55,014][20381] Updated weights for policy 1, policy_version 1731063 (0.0007) [2023-12-27 17:38:55,433][20347] Updated weights for policy 0, policy_version 1732524 (0.0006) [2023-12-27 17:38:55,492][20347] Updated weights for policy 0, policy_version 1732534 (0.0009) [2023-12-27 17:38:55,552][20347] Updated weights for policy 0, policy_version 1732544 (0.0007) [2023-12-27 17:38:55,663][20381] Updated weights for policy 1, policy_version 1731073 (0.0008) [2023-12-27 17:38:55,723][20381] Updated weights for policy 1, policy_version 1731083 (0.0010) [2023-12-27 17:38:55,773][20381] Updated weights for policy 1, policy_version 1731093 (0.0010) [2023-12-27 17:38:55,822][20381] Updated weights for policy 1, policy_version 1731103 (0.0010) [2023-12-27 17:38:56,208][20347] Updated weights for policy 0, policy_version 1732554 (0.0010) [2023-12-27 17:38:56,268][20347] Updated weights for policy 0, policy_version 1732564 (0.0005) [2023-12-27 17:38:56,326][20347] Updated weights for policy 0, policy_version 1732574 (0.0009) [2023-12-27 17:38:56,371][20347] Updated weights for policy 0, policy_version 1732584 (0.0006) [2023-12-27 17:38:56,466][20381] Updated weights for policy 1, policy_version 1731113 (0.0007) [2023-12-27 17:38:56,515][20381] Updated weights for policy 1, policy_version 1731123 (0.0009) [2023-12-27 17:38:56,566][20381] Updated weights for policy 1, policy_version 1731133 (0.0010) [2023-12-27 17:38:56,905][20347] Updated weights for policy 0, policy_version 1732594 (0.0005) [2023-12-27 17:38:56,962][20347] Updated weights for policy 0, policy_version 1732604 (0.0005) [2023-12-27 17:38:57,013][20347] Updated weights for policy 0, policy_version 1732614 (0.0005) [2023-12-27 17:38:57,246][20381] Updated weights for policy 1, policy_version 1731143 (0.0009) [2023-12-27 17:38:57,310][20381] Updated weights for policy 1, policy_version 1731153 (0.0007) [2023-12-27 17:38:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 886849536. Throughput: 0: 9931.9, 1: 10405.3. Samples: 886841144. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:38:57,363][19170] Avg episode reward: [(0, '960.970'), (1, '1000.000')] [2023-12-27 17:38:57,368][20381] Updated weights for policy 1, policy_version 1731163 (0.0007) [2023-12-27 17:38:57,562][20347] Updated weights for policy 0, policy_version 1732624 (0.0005) [2023-12-27 17:38:57,614][20347] Updated weights for policy 0, policy_version 1732634 (0.0005) [2023-12-27 17:38:57,668][20347] Updated weights for policy 0, policy_version 1732644 (0.0005) [2023-12-27 17:38:57,982][20381] Updated weights for policy 1, policy_version 1731173 (0.0006) [2023-12-27 17:38:58,039][20381] Updated weights for policy 1, policy_version 1731183 (0.0007) [2023-12-27 17:38:58,092][20381] Updated weights for policy 1, policy_version 1731193 (0.0007) [2023-12-27 17:38:58,239][20347] Updated weights for policy 0, policy_version 1732654 (0.0008) [2023-12-27 17:38:58,300][20347] Updated weights for policy 0, policy_version 1732664 (0.0009) [2023-12-27 17:38:58,366][20347] Updated weights for policy 0, policy_version 1732674 (0.0008) [2023-12-27 17:38:58,869][20381] Updated weights for policy 1, policy_version 1731203 (0.0008) [2023-12-27 17:38:58,924][20381] Updated weights for policy 1, policy_version 1731213 (0.0008) [2023-12-27 17:38:58,990][20381] Updated weights for policy 1, policy_version 1731223 (0.0008) [2023-12-27 17:38:59,164][20347] Updated weights for policy 0, policy_version 1732684 (0.0009) [2023-12-27 17:38:59,223][20347] Updated weights for policy 0, policy_version 1732694 (0.0008) [2023-12-27 17:38:59,287][20347] Updated weights for policy 0, policy_version 1732704 (0.0009) [2023-12-27 17:38:59,712][20381] Updated weights for policy 1, policy_version 1731233 (0.0008) [2023-12-27 17:38:59,769][20381] Updated weights for policy 1, policy_version 1731243 (0.0007) [2023-12-27 17:38:59,821][20381] Updated weights for policy 1, policy_version 1731253 (0.0006) [2023-12-27 17:38:59,884][20381] Updated weights for policy 1, policy_version 1731263 (0.0007) [2023-12-27 17:39:00,015][20347] Updated weights for policy 0, policy_version 1732714 (0.0007) [2023-12-27 17:39:00,073][20347] Updated weights for policy 0, policy_version 1732724 (0.0005) [2023-12-27 17:39:00,130][20347] Updated weights for policy 0, policy_version 1732734 (0.0007) [2023-12-27 17:39:00,184][20347] Updated weights for policy 0, policy_version 1732744 (0.0007) [2023-12-27 17:39:00,530][20381] Updated weights for policy 1, policy_version 1731273 (0.0008) [2023-12-27 17:39:00,574][20381] Updated weights for policy 1, policy_version 1731283 (0.0007) [2023-12-27 17:39:00,626][20381] Updated weights for policy 1, policy_version 1731293 (0.0008) [2023-12-27 17:39:00,863][20347] Updated weights for policy 0, policy_version 1732754 (0.0008) [2023-12-27 17:39:00,920][20347] Updated weights for policy 0, policy_version 1732764 (0.0010) [2023-12-27 17:39:00,969][20347] Updated weights for policy 0, policy_version 1732774 (0.0011) [2023-12-27 17:39:01,398][20381] Updated weights for policy 1, policy_version 1731303 (0.0008) [2023-12-27 17:39:01,455][20381] Updated weights for policy 1, policy_version 1731313 (0.0007) [2023-12-27 17:39:01,509][20381] Updated weights for policy 1, policy_version 1731323 (0.0009) [2023-12-27 17:39:01,676][20347] Updated weights for policy 0, policy_version 1732784 (0.0009) [2023-12-27 17:39:01,745][20347] Updated weights for policy 0, policy_version 1732794 (0.0009) [2023-12-27 17:39:01,797][20347] Updated weights for policy 0, policy_version 1732804 (0.0006) [2023-12-27 17:39:02,172][20381] Updated weights for policy 1, policy_version 1731333 (0.0010) [2023-12-27 17:39:02,230][20381] Updated weights for policy 1, policy_version 1731343 (0.0010) [2023-12-27 17:39:02,291][20381] Updated weights for policy 1, policy_version 1731353 (0.0010) [2023-12-27 17:39:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 886956032. Throughput: 0: 9943.1, 1: 10356.1. Samples: 886964928. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:02,363][19170] Avg episode reward: [(0, '980.570'), (1, '1000.000')] [2023-12-27 17:39:02,484][20347] Updated weights for policy 0, policy_version 1732814 (0.0007) [2023-12-27 17:39:02,547][20347] Updated weights for policy 0, policy_version 1732824 (0.0008) [2023-12-27 17:39:02,603][20347] Updated weights for policy 0, policy_version 1732834 (0.0010) [2023-12-27 17:39:02,974][20381] Updated weights for policy 1, policy_version 1731363 (0.0009) [2023-12-27 17:39:03,033][20381] Updated weights for policy 1, policy_version 1731373 (0.0009) [2023-12-27 17:39:03,079][20381] Updated weights for policy 1, policy_version 1731383 (0.0009) [2023-12-27 17:39:03,199][20347] Updated weights for policy 0, policy_version 1732844 (0.0010) [2023-12-27 17:39:03,252][20347] Updated weights for policy 0, policy_version 1732854 (0.0010) [2023-12-27 17:39:03,303][20347] Updated weights for policy 0, policy_version 1732864 (0.0010) [2023-12-27 17:39:03,714][20381] Updated weights for policy 1, policy_version 1731393 (0.0008) [2023-12-27 17:39:03,767][20381] Updated weights for policy 1, policy_version 1731403 (0.0007) [2023-12-27 17:39:03,814][20381] Updated weights for policy 1, policy_version 1731413 (0.0007) [2023-12-27 17:39:03,895][20381] Updated weights for policy 1, policy_version 1731423 (0.0007) [2023-12-27 17:39:03,991][20347] Updated weights for policy 0, policy_version 1732874 (0.0010) [2023-12-27 17:39:04,056][20347] Updated weights for policy 0, policy_version 1732884 (0.0007) [2023-12-27 17:39:04,121][20347] Updated weights for policy 0, policy_version 1732894 (0.0007) [2023-12-27 17:39:04,188][20347] Updated weights for policy 0, policy_version 1732904 (0.0007) [2023-12-27 17:39:04,551][20381] Updated weights for policy 1, policy_version 1731433 (0.0008) [2023-12-27 17:39:04,611][20381] Updated weights for policy 1, policy_version 1731443 (0.0008) [2023-12-27 17:39:04,664][20381] Updated weights for policy 1, policy_version 1731453 (0.0007) [2023-12-27 17:39:04,792][20347] Updated weights for policy 0, policy_version 1732914 (0.0006) [2023-12-27 17:39:04,858][20347] Updated weights for policy 0, policy_version 1732924 (0.0006) [2023-12-27 17:39:04,912][20347] Updated weights for policy 0, policy_version 1732934 (0.0007) [2023-12-27 17:39:05,371][20381] Updated weights for policy 1, policy_version 1731463 (0.0007) [2023-12-27 17:39:05,432][20381] Updated weights for policy 1, policy_version 1731473 (0.0006) [2023-12-27 17:39:05,495][20381] Updated weights for policy 1, policy_version 1731483 (0.0007) [2023-12-27 17:39:05,530][20347] Updated weights for policy 0, policy_version 1732944 (0.0007) [2023-12-27 17:39:05,592][20347] Updated weights for policy 0, policy_version 1732954 (0.0005) [2023-12-27 17:39:05,645][20347] Updated weights for policy 0, policy_version 1732964 (0.0005) [2023-12-27 17:39:06,167][20381] Updated weights for policy 1, policy_version 1731493 (0.0009) [2023-12-27 17:39:06,227][20381] Updated weights for policy 1, policy_version 1731503 (0.0009) [2023-12-27 17:39:06,277][20347] Updated weights for policy 0, policy_version 1732974 (0.0009) [2023-12-27 17:39:06,290][20381] Updated weights for policy 1, policy_version 1731513 (0.0008) [2023-12-27 17:39:06,337][20347] Updated weights for policy 0, policy_version 1732984 (0.0010) [2023-12-27 17:39:06,395][20347] Updated weights for policy 0, policy_version 1732994 (0.0010) [2023-12-27 17:39:06,978][20381] Updated weights for policy 1, policy_version 1731523 (0.0011) [2023-12-27 17:39:07,042][20381] Updated weights for policy 1, policy_version 1731533 (0.0010) [2023-12-27 17:39:07,071][20347] Updated weights for policy 0, policy_version 1733004 (0.0009) [2023-12-27 17:39:07,100][20381] Updated weights for policy 1, policy_version 1731543 (0.0011) [2023-12-27 17:39:07,121][20347] Updated weights for policy 0, policy_version 1733014 (0.0005) [2023-12-27 17:39:07,181][20347] Updated weights for policy 0, policy_version 1733024 (0.0005) [2023-12-27 17:39:07,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20479.9, 300 sec: 20605.0). Total num frames: 887062528. Throughput: 0: 9974.9, 1: 10350.0. Samples: 887027636. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:07,363][19170] Avg episode reward: [(0, '990.240'), (1, '1000.000')] [2023-12-27 17:39:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001731552_443326464.pth... [2023-12-27 17:39:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001733032_443736064.pth... [2023-12-27 17:39:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001730384_443023360.pth [2023-12-27 17:39:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001731848_443432960.pth [2023-12-27 17:39:07,757][20347] Updated weights for policy 0, policy_version 1733034 (0.0006) [2023-12-27 17:39:07,789][20381] Updated weights for policy 1, policy_version 1731553 (0.0010) [2023-12-27 17:39:07,808][20347] Updated weights for policy 0, policy_version 1733044 (0.0006) [2023-12-27 17:39:07,859][20381] Updated weights for policy 1, policy_version 1731563 (0.0009) [2023-12-27 17:39:07,864][20347] Updated weights for policy 0, policy_version 1733054 (0.0005) [2023-12-27 17:39:07,915][20347] Updated weights for policy 0, policy_version 1733064 (0.0005) [2023-12-27 17:39:07,926][20381] Updated weights for policy 1, policy_version 1731573 (0.0008) [2023-12-27 17:39:07,984][20381] Updated weights for policy 1, policy_version 1731583 (0.0007) [2023-12-27 17:39:08,600][20381] Updated weights for policy 1, policy_version 1731593 (0.0006) [2023-12-27 17:39:08,603][20347] Updated weights for policy 0, policy_version 1733074 (0.0007) [2023-12-27 17:39:08,659][20381] Updated weights for policy 1, policy_version 1731603 (0.0007) [2023-12-27 17:39:08,662][20347] Updated weights for policy 0, policy_version 1733084 (0.0006) [2023-12-27 17:39:08,716][20381] Updated weights for policy 1, policy_version 1731613 (0.0008) [2023-12-27 17:39:08,722][20347] Updated weights for policy 0, policy_version 1733094 (0.0005) [2023-12-27 17:39:09,359][20347] Updated weights for policy 0, policy_version 1733104 (0.0007) [2023-12-27 17:39:09,426][20347] Updated weights for policy 0, policy_version 1733114 (0.0008) [2023-12-27 17:39:09,432][20381] Updated weights for policy 1, policy_version 1731623 (0.0007) [2023-12-27 17:39:09,486][20381] Updated weights for policy 1, policy_version 1731633 (0.0006) [2023-12-27 17:39:09,487][20347] Updated weights for policy 0, policy_version 1733124 (0.0006) [2023-12-27 17:39:09,534][20381] Updated weights for policy 1, policy_version 1731643 (0.0005) [2023-12-27 17:39:10,190][20381] Updated weights for policy 1, policy_version 1731653 (0.0006) [2023-12-27 17:39:10,202][20347] Updated weights for policy 0, policy_version 1733134 (0.0007) [2023-12-27 17:39:10,251][20381] Updated weights for policy 1, policy_version 1731663 (0.0006) [2023-12-27 17:39:10,255][20347] Updated weights for policy 0, policy_version 1733144 (0.0008) [2023-12-27 17:39:10,305][20381] Updated weights for policy 1, policy_version 1731673 (0.0006) [2023-12-27 17:39:10,308][20347] Updated weights for policy 0, policy_version 1733154 (0.0008) [2023-12-27 17:39:10,913][20381] Updated weights for policy 1, policy_version 1731683 (0.0007) [2023-12-27 17:39:10,963][20381] Updated weights for policy 1, policy_version 1731693 (0.0007) [2023-12-27 17:39:11,018][20381] Updated weights for policy 1, policy_version 1731703 (0.0007) [2023-12-27 17:39:11,023][20347] Updated weights for policy 0, policy_version 1733164 (0.0008) [2023-12-27 17:39:11,086][20347] Updated weights for policy 0, policy_version 1733174 (0.0008) [2023-12-27 17:39:11,148][20347] Updated weights for policy 0, policy_version 1733184 (0.0008) [2023-12-27 17:39:11,812][20381] Updated weights for policy 1, policy_version 1731713 (0.0007) [2023-12-27 17:39:11,876][20381] Updated weights for policy 1, policy_version 1731723 (0.0006) [2023-12-27 17:39:11,887][20347] Updated weights for policy 0, policy_version 1733194 (0.0008) [2023-12-27 17:39:11,937][20381] Updated weights for policy 1, policy_version 1731733 (0.0007) [2023-12-27 17:39:11,951][20347] Updated weights for policy 0, policy_version 1733204 (0.0006) [2023-12-27 17:39:11,997][20381] Updated weights for policy 1, policy_version 1731743 (0.0008) [2023-12-27 17:39:12,012][20347] Updated weights for policy 0, policy_version 1733214 (0.0010) [2023-12-27 17:39:12,072][20347] Updated weights for policy 0, policy_version 1733224 (0.0010) [2023-12-27 17:39:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 887160832. Throughput: 0: 10020.9, 1: 10282.2. Samples: 887151412. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:12,363][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 17:39:12,677][20381] Updated weights for policy 1, policy_version 1731753 (0.0007) [2023-12-27 17:39:12,737][20381] Updated weights for policy 1, policy_version 1731763 (0.0007) [2023-12-27 17:39:12,797][20347] Updated weights for policy 0, policy_version 1733234 (0.0009) [2023-12-27 17:39:12,804][20381] Updated weights for policy 1, policy_version 1731773 (0.0006) [2023-12-27 17:39:12,858][20347] Updated weights for policy 0, policy_version 1733244 (0.0010) [2023-12-27 17:39:12,917][20347] Updated weights for policy 0, policy_version 1733254 (0.0011) [2023-12-27 17:39:13,452][20381] Updated weights for policy 1, policy_version 1731783 (0.0007) [2023-12-27 17:39:13,510][20381] Updated weights for policy 1, policy_version 1731793 (0.0007) [2023-12-27 17:39:13,563][20381] Updated weights for policy 1, policy_version 1731803 (0.0008) [2023-12-27 17:39:13,576][20347] Updated weights for policy 0, policy_version 1733264 (0.0010) [2023-12-27 17:39:13,630][20347] Updated weights for policy 0, policy_version 1733274 (0.0010) [2023-12-27 17:39:13,684][20347] Updated weights for policy 0, policy_version 1733284 (0.0008) [2023-12-27 17:39:14,222][20381] Updated weights for policy 1, policy_version 1731813 (0.0006) [2023-12-27 17:39:14,275][20381] Updated weights for policy 1, policy_version 1731823 (0.0008) [2023-12-27 17:39:14,288][20347] Updated weights for policy 0, policy_version 1733294 (0.0008) [2023-12-27 17:39:14,330][20381] Updated weights for policy 1, policy_version 1731833 (0.0010) [2023-12-27 17:39:14,336][20347] Updated weights for policy 0, policy_version 1733304 (0.0010) [2023-12-27 17:39:14,384][20347] Updated weights for policy 0, policy_version 1733314 (0.0010) [2023-12-27 17:39:15,008][20381] Updated weights for policy 1, policy_version 1731843 (0.0010) [2023-12-27 17:39:15,064][20347] Updated weights for policy 0, policy_version 1733324 (0.0009) [2023-12-27 17:39:15,072][20381] Updated weights for policy 1, policy_version 1731853 (0.0006) [2023-12-27 17:39:15,124][20347] Updated weights for policy 0, policy_version 1733334 (0.0006) [2023-12-27 17:39:15,133][20381] Updated weights for policy 1, policy_version 1731863 (0.0009) [2023-12-27 17:39:15,178][20347] Updated weights for policy 0, policy_version 1733344 (0.0006) [2023-12-27 17:39:15,779][20381] Updated weights for policy 1, policy_version 1731873 (0.0010) [2023-12-27 17:39:15,828][20381] Updated weights for policy 1, policy_version 1731883 (0.0008) [2023-12-27 17:39:15,880][20347] Updated weights for policy 0, policy_version 1733354 (0.0008) [2023-12-27 17:39:15,886][20381] Updated weights for policy 1, policy_version 1731893 (0.0008) [2023-12-27 17:39:15,939][20347] Updated weights for policy 0, policy_version 1733364 (0.0009) [2023-12-27 17:39:15,941][20381] Updated weights for policy 1, policy_version 1731903 (0.0006) [2023-12-27 17:39:16,000][20347] Updated weights for policy 0, policy_version 1733374 (0.0007) [2023-12-27 17:39:16,054][20347] Updated weights for policy 0, policy_version 1733384 (0.0008) [2023-12-27 17:39:16,650][20381] Updated weights for policy 1, policy_version 1731913 (0.0005) [2023-12-27 17:39:16,703][20381] Updated weights for policy 1, policy_version 1731923 (0.0007) [2023-12-27 17:39:16,748][20347] Updated weights for policy 0, policy_version 1733394 (0.0007) [2023-12-27 17:39:16,758][20381] Updated weights for policy 1, policy_version 1731933 (0.0008) [2023-12-27 17:39:16,807][20347] Updated weights for policy 0, policy_version 1733404 (0.0005) [2023-12-27 17:39:16,864][20347] Updated weights for policy 0, policy_version 1733414 (0.0010) [2023-12-27 17:39:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20206.9, 300 sec: 20549.4). Total num frames: 887259136. Throughput: 0: 10043.0, 1: 10292.4. Samples: 887273352. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:17,363][19170] Avg episode reward: [(0, '990.230'), (1, '1000.000')] [2023-12-27 17:39:17,406][20381] Updated weights for policy 1, policy_version 1731943 (0.0007) [2023-12-27 17:39:17,457][20381] Updated weights for policy 1, policy_version 1731953 (0.0006) [2023-12-27 17:39:17,478][20347] Updated weights for policy 0, policy_version 1733424 (0.0009) [2023-12-27 17:39:17,511][20381] Updated weights for policy 1, policy_version 1731963 (0.0005) [2023-12-27 17:39:17,530][20347] Updated weights for policy 0, policy_version 1733434 (0.0010) [2023-12-27 17:39:17,582][20347] Updated weights for policy 0, policy_version 1733444 (0.0010) [2023-12-27 17:39:18,170][20381] Updated weights for policy 1, policy_version 1731973 (0.0007) [2023-12-27 17:39:18,204][20347] Updated weights for policy 0, policy_version 1733454 (0.0009) [2023-12-27 17:39:18,224][20381] Updated weights for policy 1, policy_version 1731983 (0.0007) [2023-12-27 17:39:18,259][20347] Updated weights for policy 0, policy_version 1733464 (0.0010) [2023-12-27 17:39:18,283][20381] Updated weights for policy 1, policy_version 1731993 (0.0005) [2023-12-27 17:39:18,311][20347] Updated weights for policy 0, policy_version 1733474 (0.0010) [2023-12-27 17:39:18,873][20381] Updated weights for policy 1, policy_version 1732003 (0.0006) [2023-12-27 17:39:18,927][20381] Updated weights for policy 1, policy_version 1732013 (0.0005) [2023-12-27 17:39:18,949][20347] Updated weights for policy 0, policy_version 1733484 (0.0009) [2023-12-27 17:39:18,973][20381] Updated weights for policy 1, policy_version 1732023 (0.0006) [2023-12-27 17:39:19,002][20347] Updated weights for policy 0, policy_version 1733494 (0.0006) [2023-12-27 17:39:19,056][20347] Updated weights for policy 0, policy_version 1733504 (0.0007) [2023-12-27 17:39:19,655][20381] Updated weights for policy 1, policy_version 1732033 (0.0007) [2023-12-27 17:39:19,712][20381] Updated weights for policy 1, policy_version 1732043 (0.0007) [2023-12-27 17:39:19,767][20381] Updated weights for policy 1, policy_version 1732053 (0.0005) [2023-12-27 17:39:19,793][20347] Updated weights for policy 0, policy_version 1733514 (0.0007) [2023-12-27 17:39:19,829][20381] Updated weights for policy 1, policy_version 1732063 (0.0008) [2023-12-27 17:39:19,855][20347] Updated weights for policy 0, policy_version 1733524 (0.0006) [2023-12-27 17:39:19,923][20347] Updated weights for policy 0, policy_version 1733534 (0.0007) [2023-12-27 17:39:19,987][20347] Updated weights for policy 0, policy_version 1733544 (0.0008) [2023-12-27 17:39:20,552][20381] Updated weights for policy 1, policy_version 1732073 (0.0008) [2023-12-27 17:39:20,602][20347] Updated weights for policy 0, policy_version 1733554 (0.0007) [2023-12-27 17:39:20,614][20381] Updated weights for policy 1, policy_version 1732083 (0.0008) [2023-12-27 17:39:20,659][20347] Updated weights for policy 0, policy_version 1733564 (0.0008) [2023-12-27 17:39:20,672][20381] Updated weights for policy 1, policy_version 1732093 (0.0008) [2023-12-27 17:39:20,711][20347] Updated weights for policy 0, policy_version 1733574 (0.0009) [2023-12-27 17:39:21,376][20381] Updated weights for policy 1, policy_version 1732103 (0.0007) [2023-12-27 17:39:21,440][20381] Updated weights for policy 1, policy_version 1732113 (0.0006) [2023-12-27 17:39:21,473][20347] Updated weights for policy 0, policy_version 1733584 (0.0007) [2023-12-27 17:39:21,496][20381] Updated weights for policy 1, policy_version 1732123 (0.0006) [2023-12-27 17:39:21,536][20347] Updated weights for policy 0, policy_version 1733594 (0.0006) [2023-12-27 17:39:21,593][20347] Updated weights for policy 0, policy_version 1733604 (0.0009) [2023-12-27 17:39:22,146][20381] Updated weights for policy 1, policy_version 1732133 (0.0008) [2023-12-27 17:39:22,207][20381] Updated weights for policy 1, policy_version 1732143 (0.0008) [2023-12-27 17:39:22,268][20381] Updated weights for policy 1, policy_version 1732153 (0.0008) [2023-12-27 17:39:22,332][20347] Updated weights for policy 0, policy_version 1733614 (0.0008) [2023-12-27 17:39:22,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 887365632. Throughput: 0: 10077.0, 1: 10242.2. Samples: 887336044. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:39:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001732160_443482112.pth... [2023-12-27 17:39:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001730944_443170816.pth [2023-12-27 17:39:22,400][20347] Updated weights for policy 0, policy_version 1733624 (0.0009) [2023-12-27 17:39:22,463][20347] Updated weights for policy 0, policy_version 1733634 (0.0007) [2023-12-27 17:39:22,490][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001733640_443891712.pth... [2023-12-27 17:39:22,493][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001732424_443580416.pth [2023-12-27 17:39:23,017][20381] Updated weights for policy 1, policy_version 1732163 (0.0008) [2023-12-27 17:39:23,079][20381] Updated weights for policy 1, policy_version 1732173 (0.0008) [2023-12-27 17:39:23,133][20381] Updated weights for policy 1, policy_version 1732183 (0.0008) [2023-12-27 17:39:23,178][20347] Updated weights for policy 0, policy_version 1733644 (0.0007) [2023-12-27 17:39:23,233][20347] Updated weights for policy 0, policy_version 1733654 (0.0007) [2023-12-27 17:39:23,293][20347] Updated weights for policy 0, policy_version 1733664 (0.0007) [2023-12-27 17:39:23,850][20381] Updated weights for policy 1, policy_version 1732193 (0.0007) [2023-12-27 17:39:23,910][20381] Updated weights for policy 1, policy_version 1732203 (0.0008) [2023-12-27 17:39:23,954][20347] Updated weights for policy 0, policy_version 1733674 (0.0008) [2023-12-27 17:39:23,968][20381] Updated weights for policy 1, policy_version 1732213 (0.0007) [2023-12-27 17:39:24,015][20347] Updated weights for policy 0, policy_version 1733684 (0.0010) [2023-12-27 17:39:24,017][20381] Updated weights for policy 1, policy_version 1732223 (0.0006) [2023-12-27 17:39:24,076][20347] Updated weights for policy 0, policy_version 1733694 (0.0010) [2023-12-27 17:39:24,139][20347] Updated weights for policy 0, policy_version 1733704 (0.0010) [2023-12-27 17:39:24,735][20381] Updated weights for policy 1, policy_version 1732233 (0.0007) [2023-12-27 17:39:24,794][20381] Updated weights for policy 1, policy_version 1732243 (0.0008) [2023-12-27 17:39:24,803][20347] Updated weights for policy 0, policy_version 1733714 (0.0005) [2023-12-27 17:39:24,847][20381] Updated weights for policy 1, policy_version 1732253 (0.0010) [2023-12-27 17:39:24,862][20347] Updated weights for policy 0, policy_version 1733724 (0.0006) [2023-12-27 17:39:24,898][20291] KL-divergence is very high: 142.7027 [2023-12-27 17:39:24,918][20347] Updated weights for policy 0, policy_version 1733734 (0.0010) [2023-12-27 17:39:25,506][20381] Updated weights for policy 1, policy_version 1732263 (0.0007) [2023-12-27 17:39:25,537][20291] KL-divergence is very high: 206.3077 [2023-12-27 17:39:25,558][20381] Updated weights for policy 1, policy_version 1732273 (0.0006) [2023-12-27 17:39:25,565][20347] Updated weights for policy 0, policy_version 1733744 (0.0010) [2023-12-27 17:39:25,580][20291] KL-divergence is very high: 150.3308 [2023-12-27 17:39:25,610][20381] Updated weights for policy 1, policy_version 1732283 (0.0005) [2023-12-27 17:39:25,619][20347] Updated weights for policy 0, policy_version 1733754 (0.0010) [2023-12-27 17:39:25,626][20291] KL-divergence is very high: 102.4359 [2023-12-27 17:39:25,683][20347] Updated weights for policy 0, policy_version 1733764 (0.0010) [2023-12-27 17:39:26,254][20381] Updated weights for policy 1, policy_version 1732293 (0.0008) [2023-12-27 17:39:26,299][20381] Updated weights for policy 1, policy_version 1732303 (0.0010) [2023-12-27 17:39:26,345][20347] Updated weights for policy 0, policy_version 1733774 (0.0010) [2023-12-27 17:39:26,348][20381] Updated weights for policy 1, policy_version 1732313 (0.0010) [2023-12-27 17:39:26,389][20347] Updated weights for policy 0, policy_version 1733784 (0.0010) [2023-12-27 17:39:26,440][20347] Updated weights for policy 0, policy_version 1733794 (0.0010) [2023-12-27 17:39:27,035][20347] Updated weights for policy 0, policy_version 1733804 (0.0010) [2023-12-27 17:39:27,038][20381] Updated weights for policy 1, policy_version 1732323 (0.0010) [2023-12-27 17:39:27,087][20347] Updated weights for policy 0, policy_version 1733814 (0.0010) [2023-12-27 17:39:27,096][20381] Updated weights for policy 1, policy_version 1732333 (0.0010) [2023-12-27 17:39:27,137][20347] Updated weights for policy 0, policy_version 1733824 (0.0010) [2023-12-27 17:39:27,148][20381] Updated weights for policy 1, policy_version 1732343 (0.0010) [2023-12-27 17:39:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 887472128. Throughput: 0: 10142.6, 1: 10190.7. Samples: 887456240. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:27,363][19170] Avg episode reward: [(0, '960.870'), (1, '1000.000')] [2023-12-27 17:39:27,762][20381] Updated weights for policy 1, policy_version 1732353 (0.0010) [2023-12-27 17:39:27,783][20347] Updated weights for policy 0, policy_version 1733834 (0.0010) [2023-12-27 17:39:27,821][20381] Updated weights for policy 1, policy_version 1732363 (0.0005) [2023-12-27 17:39:27,843][20347] Updated weights for policy 0, policy_version 1733844 (0.0007) [2023-12-27 17:39:27,878][20381] Updated weights for policy 1, policy_version 1732373 (0.0008) [2023-12-27 17:39:27,903][20347] Updated weights for policy 0, policy_version 1733854 (0.0005) [2023-12-27 17:39:27,934][20381] Updated weights for policy 1, policy_version 1732383 (0.0007) [2023-12-27 17:39:27,959][20347] Updated weights for policy 0, policy_version 1733864 (0.0006) [2023-12-27 17:39:28,576][20347] Updated weights for policy 0, policy_version 1733874 (0.0010) [2023-12-27 17:39:28,628][20381] Updated weights for policy 1, policy_version 1732393 (0.0006) [2023-12-27 17:39:28,640][20347] Updated weights for policy 0, policy_version 1733884 (0.0009) [2023-12-27 17:39:28,696][20347] Updated weights for policy 0, policy_version 1733894 (0.0007) [2023-12-27 17:39:28,698][20381] Updated weights for policy 1, policy_version 1732403 (0.0007) [2023-12-27 17:39:28,751][20381] Updated weights for policy 1, policy_version 1732413 (0.0009) [2023-12-27 17:39:29,317][20347] Updated weights for policy 0, policy_version 1733904 (0.0009) [2023-12-27 17:39:29,386][20347] Updated weights for policy 0, policy_version 1733914 (0.0010) [2023-12-27 17:39:29,425][20381] Updated weights for policy 1, policy_version 1732423 (0.0007) [2023-12-27 17:39:29,437][20347] Updated weights for policy 0, policy_version 1733924 (0.0008) [2023-12-27 17:39:29,486][20381] Updated weights for policy 1, policy_version 1732433 (0.0007) [2023-12-27 17:39:29,541][20381] Updated weights for policy 1, policy_version 1732443 (0.0007) [2023-12-27 17:39:30,091][20347] Updated weights for policy 0, policy_version 1733934 (0.0008) [2023-12-27 17:39:30,145][20347] Updated weights for policy 0, policy_version 1733944 (0.0010) [2023-12-27 17:39:30,201][20347] Updated weights for policy 0, policy_version 1733954 (0.0010) [2023-12-27 17:39:30,216][20381] Updated weights for policy 1, policy_version 1732453 (0.0007) [2023-12-27 17:39:30,272][20381] Updated weights for policy 1, policy_version 1732463 (0.0007) [2023-12-27 17:39:30,331][20381] Updated weights for policy 1, policy_version 1732473 (0.0006) [2023-12-27 17:39:30,890][20347] Updated weights for policy 0, policy_version 1733964 (0.0008) [2023-12-27 17:39:30,945][20347] Updated weights for policy 0, policy_version 1733974 (0.0005) [2023-12-27 17:39:31,008][20347] Updated weights for policy 0, policy_version 1733984 (0.0008) [2023-12-27 17:39:31,020][20381] Updated weights for policy 1, policy_version 1732483 (0.0008) [2023-12-27 17:39:31,085][20381] Updated weights for policy 1, policy_version 1732493 (0.0007) [2023-12-27 17:39:31,143][20381] Updated weights for policy 1, policy_version 1732503 (0.0007) [2023-12-27 17:39:31,635][20347] Updated weights for policy 0, policy_version 1733994 (0.0007) [2023-12-27 17:39:31,696][20347] Updated weights for policy 0, policy_version 1734004 (0.0008) [2023-12-27 17:39:31,763][20347] Updated weights for policy 0, policy_version 1734014 (0.0007) [2023-12-27 17:39:31,825][20347] Updated weights for policy 0, policy_version 1734024 (0.0009) [2023-12-27 17:39:31,902][20381] Updated weights for policy 1, policy_version 1732513 (0.0007) [2023-12-27 17:39:31,956][20381] Updated weights for policy 1, policy_version 1732523 (0.0008) [2023-12-27 17:39:32,017][20381] Updated weights for policy 1, policy_version 1732533 (0.0008) [2023-12-27 17:39:32,078][20381] Updated weights for policy 1, policy_version 1732543 (0.0008) [2023-12-27 17:39:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 887570432. Throughput: 0: 10279.2, 1: 10198.4. Samples: 887580404. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:32,363][19170] Avg episode reward: [(0, '960.870'), (1, '1000.000')] [2023-12-27 17:39:32,554][20347] Updated weights for policy 0, policy_version 1734034 (0.0008) [2023-12-27 17:39:32,607][20347] Updated weights for policy 0, policy_version 1734044 (0.0007) [2023-12-27 17:39:32,653][20347] Updated weights for policy 0, policy_version 1734054 (0.0007) [2023-12-27 17:39:32,790][20381] Updated weights for policy 1, policy_version 1732553 (0.0008) [2023-12-27 17:39:32,853][20381] Updated weights for policy 1, policy_version 1732563 (0.0005) [2023-12-27 17:39:32,901][20381] Updated weights for policy 1, policy_version 1732573 (0.0005) [2023-12-27 17:39:33,378][20347] Updated weights for policy 0, policy_version 1734064 (0.0007) [2023-12-27 17:39:33,433][20347] Updated weights for policy 0, policy_version 1734074 (0.0007) [2023-12-27 17:39:33,484][20381] Updated weights for policy 1, policy_version 1732583 (0.0005) [2023-12-27 17:39:33,486][20347] Updated weights for policy 0, policy_version 1734084 (0.0007) [2023-12-27 17:39:33,537][20381] Updated weights for policy 1, policy_version 1732593 (0.0006) [2023-12-27 17:39:33,592][20381] Updated weights for policy 1, policy_version 1732603 (0.0008) [2023-12-27 17:39:34,225][20381] Updated weights for policy 1, policy_version 1732613 (0.0007) [2023-12-27 17:39:34,234][20347] Updated weights for policy 0, policy_version 1734094 (0.0007) [2023-12-27 17:39:34,291][20347] Updated weights for policy 0, policy_version 1734104 (0.0007) [2023-12-27 17:39:34,293][20381] Updated weights for policy 1, policy_version 1732623 (0.0008) [2023-12-27 17:39:34,352][20381] Updated weights for policy 1, policy_version 1732633 (0.0007) [2023-12-27 17:39:34,353][20347] Updated weights for policy 0, policy_version 1734114 (0.0007) [2023-12-27 17:39:35,020][20381] Updated weights for policy 1, policy_version 1732643 (0.0007) [2023-12-27 17:39:35,068][20347] Updated weights for policy 0, policy_version 1734124 (0.0008) [2023-12-27 17:39:35,072][20381] Updated weights for policy 1, policy_version 1732653 (0.0008) [2023-12-27 17:39:35,119][20347] Updated weights for policy 0, policy_version 1734134 (0.0005) [2023-12-27 17:39:35,128][20381] Updated weights for policy 1, policy_version 1732663 (0.0006) [2023-12-27 17:39:35,178][20347] Updated weights for policy 0, policy_version 1734144 (0.0006) [2023-12-27 17:39:35,822][20381] Updated weights for policy 1, policy_version 1732673 (0.0007) [2023-12-27 17:39:35,836][20347] Updated weights for policy 0, policy_version 1734154 (0.0008) [2023-12-27 17:39:35,882][20381] Updated weights for policy 1, policy_version 1732683 (0.0006) [2023-12-27 17:39:35,895][20347] Updated weights for policy 0, policy_version 1734164 (0.0010) [2023-12-27 17:39:35,932][20381] Updated weights for policy 1, policy_version 1732693 (0.0006) [2023-12-27 17:39:35,952][20347] Updated weights for policy 0, policy_version 1734174 (0.0007) [2023-12-27 17:39:35,985][20381] Updated weights for policy 1, policy_version 1732703 (0.0008) [2023-12-27 17:39:36,001][20347] Updated weights for policy 0, policy_version 1734184 (0.0005) [2023-12-27 17:39:36,625][20381] Updated weights for policy 1, policy_version 1732713 (0.0006) [2023-12-27 17:39:36,674][20381] Updated weights for policy 1, policy_version 1732723 (0.0005) [2023-12-27 17:39:36,681][20347] Updated weights for policy 0, policy_version 1734194 (0.0011) [2023-12-27 17:39:36,732][20381] Updated weights for policy 1, policy_version 1732733 (0.0005) [2023-12-27 17:39:36,737][20347] Updated weights for policy 0, policy_version 1734204 (0.0010) [2023-12-27 17:39:36,800][20347] Updated weights for policy 0, policy_version 1734214 (0.0010) [2023-12-27 17:39:37,296][20381] Updated weights for policy 1, policy_version 1732743 (0.0007) [2023-12-27 17:39:37,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 887668736. Throughput: 0: 10302.0, 1: 10211.7. Samples: 887641784. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:37,363][20381] Updated weights for policy 1, policy_version 1732753 (0.0007) [2023-12-27 17:39:37,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:39:37,415][20347] Updated weights for policy 0, policy_version 1734224 (0.0006) [2023-12-27 17:39:37,418][20381] Updated weights for policy 1, policy_version 1732763 (0.0007) [2023-12-27 17:39:37,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001732768_443637760.pth... [2023-12-27 17:39:37,445][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001731552_443326464.pth [2023-12-27 17:39:37,471][20347] Updated weights for policy 0, policy_version 1734234 (0.0005) [2023-12-27 17:39:37,530][20347] Updated weights for policy 0, policy_version 1734244 (0.0005) [2023-12-27 17:39:37,555][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001734248_444047360.pth... [2023-12-27 17:39:37,559][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001733032_443736064.pth [2023-12-27 17:39:38,077][20381] Updated weights for policy 1, policy_version 1732773 (0.0006) [2023-12-27 17:39:38,131][20381] Updated weights for policy 1, policy_version 1732783 (0.0005) [2023-12-27 17:39:38,162][20347] Updated weights for policy 0, policy_version 1734254 (0.0005) [2023-12-27 17:39:38,190][20381] Updated weights for policy 1, policy_version 1732793 (0.0007) [2023-12-27 17:39:38,215][20347] Updated weights for policy 0, policy_version 1734264 (0.0006) [2023-12-27 17:39:38,267][20347] Updated weights for policy 0, policy_version 1734274 (0.0006) [2023-12-27 17:39:38,896][20381] Updated weights for policy 1, policy_version 1732803 (0.0007) [2023-12-27 17:39:38,910][20347] Updated weights for policy 0, policy_version 1734284 (0.0007) [2023-12-27 17:39:38,952][20381] Updated weights for policy 1, policy_version 1732813 (0.0006) [2023-12-27 17:39:38,967][20347] Updated weights for policy 0, policy_version 1734294 (0.0006) [2023-12-27 17:39:39,002][20381] Updated weights for policy 1, policy_version 1732823 (0.0005) [2023-12-27 17:39:39,023][20347] Updated weights for policy 0, policy_version 1734304 (0.0006) [2023-12-27 17:39:39,686][20381] Updated weights for policy 1, policy_version 1732833 (0.0005) [2023-12-27 17:39:39,746][20381] Updated weights for policy 1, policy_version 1732843 (0.0008) [2023-12-27 17:39:39,749][20347] Updated weights for policy 0, policy_version 1734314 (0.0008) [2023-12-27 17:39:39,803][20381] Updated weights for policy 1, policy_version 1732853 (0.0006) [2023-12-27 17:39:39,809][20347] Updated weights for policy 0, policy_version 1734324 (0.0006) [2023-12-27 17:39:39,871][20381] Updated weights for policy 1, policy_version 1732863 (0.0007) [2023-12-27 17:39:39,872][20347] Updated weights for policy 0, policy_version 1734334 (0.0008) [2023-12-27 17:39:39,934][20347] Updated weights for policy 0, policy_version 1734344 (0.0008) [2023-12-27 17:39:40,566][20347] Updated weights for policy 0, policy_version 1734354 (0.0006) [2023-12-27 17:39:40,582][20381] Updated weights for policy 1, policy_version 1732873 (0.0010) [2023-12-27 17:39:40,621][20347] Updated weights for policy 0, policy_version 1734364 (0.0006) [2023-12-27 17:39:40,641][20381] Updated weights for policy 1, policy_version 1732883 (0.0009) [2023-12-27 17:39:40,675][20347] Updated weights for policy 0, policy_version 1734374 (0.0005) [2023-12-27 17:39:40,692][20381] Updated weights for policy 1, policy_version 1732893 (0.0010) [2023-12-27 17:39:41,336][20347] Updated weights for policy 0, policy_version 1734384 (0.0009) [2023-12-27 17:39:41,405][20347] Updated weights for policy 0, policy_version 1734394 (0.0009) [2023-12-27 17:39:41,424][20381] Updated weights for policy 1, policy_version 1732903 (0.0009) [2023-12-27 17:39:41,461][20347] Updated weights for policy 0, policy_version 1734404 (0.0006) [2023-12-27 17:39:41,483][20381] Updated weights for policy 1, policy_version 1732913 (0.0010) [2023-12-27 17:39:41,548][20381] Updated weights for policy 1, policy_version 1732923 (0.0007) [2023-12-27 17:39:42,146][20347] Updated weights for policy 0, policy_version 1734414 (0.0008) [2023-12-27 17:39:42,205][20347] Updated weights for policy 0, policy_version 1734424 (0.0009) [2023-12-27 17:39:42,265][20347] Updated weights for policy 0, policy_version 1734434 (0.0008) [2023-12-27 17:39:42,277][20381] Updated weights for policy 1, policy_version 1732933 (0.0011) [2023-12-27 17:39:42,330][20381] Updated weights for policy 1, policy_version 1732943 (0.0010) [2023-12-27 17:39:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 887775232. Throughput: 0: 10374.4, 1: 10210.1. Samples: 887767444. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:42,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:39:42,390][20381] Updated weights for policy 1, policy_version 1732953 (0.0008) [2023-12-27 17:39:43,014][20381] Updated weights for policy 1, policy_version 1732963 (0.0007) [2023-12-27 17:39:43,023][20347] Updated weights for policy 0, policy_version 1734444 (0.0008) [2023-12-27 17:39:43,065][20381] Updated weights for policy 1, policy_version 1732973 (0.0006) [2023-12-27 17:39:43,082][20347] Updated weights for policy 0, policy_version 1734454 (0.0005) [2023-12-27 17:39:43,111][20381] Updated weights for policy 1, policy_version 1732983 (0.0010) [2023-12-27 17:39:43,143][20347] Updated weights for policy 0, policy_version 1734464 (0.0005) [2023-12-27 17:39:43,710][20347] Updated weights for policy 0, policy_version 1734474 (0.0007) [2023-12-27 17:39:43,765][20347] Updated weights for policy 0, policy_version 1734484 (0.0005) [2023-12-27 17:39:43,800][20381] Updated weights for policy 1, policy_version 1732993 (0.0011) [2023-12-27 17:39:43,822][20347] Updated weights for policy 0, policy_version 1734494 (0.0009) [2023-12-27 17:39:43,865][20381] Updated weights for policy 1, policy_version 1733003 (0.0011) [2023-12-27 17:39:43,877][20347] Updated weights for policy 0, policy_version 1734504 (0.0010) [2023-12-27 17:39:43,925][20381] Updated weights for policy 1, policy_version 1733013 (0.0010) [2023-12-27 17:39:43,987][20381] Updated weights for policy 1, policy_version 1733023 (0.0008) [2023-12-27 17:39:44,539][20347] Updated weights for policy 0, policy_version 1734514 (0.0008) [2023-12-27 17:39:44,595][20347] Updated weights for policy 0, policy_version 1734524 (0.0008) [2023-12-27 17:39:44,632][20381] Updated weights for policy 1, policy_version 1733033 (0.0011) [2023-12-27 17:39:44,648][20347] Updated weights for policy 0, policy_version 1734534 (0.0007) [2023-12-27 17:39:44,687][20381] Updated weights for policy 1, policy_version 1733043 (0.0006) [2023-12-27 17:39:44,744][20381] Updated weights for policy 1, policy_version 1733053 (0.0006) [2023-12-27 17:39:45,326][20347] Updated weights for policy 0, policy_version 1734544 (0.0007) [2023-12-27 17:39:45,375][20347] Updated weights for policy 0, policy_version 1734554 (0.0007) [2023-12-27 17:39:45,425][20347] Updated weights for policy 0, policy_version 1734564 (0.0006) [2023-12-27 17:39:45,430][20381] Updated weights for policy 1, policy_version 1733063 (0.0007) [2023-12-27 17:39:45,478][20381] Updated weights for policy 1, policy_version 1733073 (0.0007) [2023-12-27 17:39:45,524][20381] Updated weights for policy 1, policy_version 1733083 (0.0008) [2023-12-27 17:39:46,045][20347] Updated weights for policy 0, policy_version 1734574 (0.0007) [2023-12-27 17:39:46,119][20347] Updated weights for policy 0, policy_version 1734584 (0.0008) [2023-12-27 17:39:46,180][20347] Updated weights for policy 0, policy_version 1734594 (0.0008) [2023-12-27 17:39:46,212][20381] Updated weights for policy 1, policy_version 1733093 (0.0009) [2023-12-27 17:39:46,261][20381] Updated weights for policy 1, policy_version 1733103 (0.0010) [2023-12-27 17:39:46,316][20381] Updated weights for policy 1, policy_version 1733113 (0.0010) [2023-12-27 17:39:46,920][20347] Updated weights for policy 0, policy_version 1734604 (0.0008) [2023-12-27 17:39:46,935][20381] Updated weights for policy 1, policy_version 1733123 (0.0008) [2023-12-27 17:39:46,969][20347] Updated weights for policy 0, policy_version 1734614 (0.0008) [2023-12-27 17:39:46,980][20381] Updated weights for policy 1, policy_version 1733133 (0.0005) [2023-12-27 17:39:47,019][20347] Updated weights for policy 0, policy_version 1734624 (0.0008) [2023-12-27 17:39:47,029][20381] Updated weights for policy 1, policy_version 1733143 (0.0005) [2023-12-27 17:39:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 887881728. Throughput: 0: 10314.1, 1: 10257.8. Samples: 887890660. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:47,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:39:47,554][20381] Updated weights for policy 1, policy_version 1733153 (0.0006) [2023-12-27 17:39:47,602][20381] Updated weights for policy 1, policy_version 1733163 (0.0006) [2023-12-27 17:39:47,651][20381] Updated weights for policy 1, policy_version 1733173 (0.0008) [2023-12-27 17:39:47,699][20381] Updated weights for policy 1, policy_version 1733183 (0.0005) [2023-12-27 17:39:47,753][20347] Updated weights for policy 0, policy_version 1734634 (0.0007) [2023-12-27 17:39:47,818][20347] Updated weights for policy 0, policy_version 1734644 (0.0009) [2023-12-27 17:39:47,868][20347] Updated weights for policy 0, policy_version 1734654 (0.0010) [2023-12-27 17:39:47,922][20347] Updated weights for policy 0, policy_version 1734664 (0.0010) [2023-12-27 17:39:48,250][20381] Updated weights for policy 1, policy_version 1733193 (0.0010) [2023-12-27 17:39:48,302][20381] Updated weights for policy 1, policy_version 1733203 (0.0010) [2023-12-27 17:39:48,354][20381] Updated weights for policy 1, policy_version 1733213 (0.0010) [2023-12-27 17:39:48,599][20347] Updated weights for policy 0, policy_version 1734674 (0.0007) [2023-12-27 17:39:48,658][20347] Updated weights for policy 0, policy_version 1734684 (0.0007) [2023-12-27 17:39:48,719][20347] Updated weights for policy 0, policy_version 1734694 (0.0008) [2023-12-27 17:39:49,006][20381] Updated weights for policy 1, policy_version 1733223 (0.0011) [2023-12-27 17:39:49,064][20381] Updated weights for policy 1, policy_version 1733233 (0.0010) [2023-12-27 17:39:49,119][20381] Updated weights for policy 1, policy_version 1733243 (0.0010) [2023-12-27 17:39:49,359][20347] Updated weights for policy 0, policy_version 1734704 (0.0008) [2023-12-27 17:39:49,415][20347] Updated weights for policy 0, policy_version 1734714 (0.0008) [2023-12-27 17:39:49,471][20347] Updated weights for policy 0, policy_version 1734724 (0.0008) [2023-12-27 17:39:49,850][20381] Updated weights for policy 1, policy_version 1733253 (0.0009) [2023-12-27 17:39:49,901][20381] Updated weights for policy 1, policy_version 1733263 (0.0008) [2023-12-27 17:39:49,965][20381] Updated weights for policy 1, policy_version 1733273 (0.0008) [2023-12-27 17:39:50,216][20347] Updated weights for policy 0, policy_version 1734734 (0.0007) [2023-12-27 17:39:50,277][20347] Updated weights for policy 0, policy_version 1734744 (0.0008) [2023-12-27 17:39:50,333][20347] Updated weights for policy 0, policy_version 1734754 (0.0008) [2023-12-27 17:39:50,664][20381] Updated weights for policy 1, policy_version 1733283 (0.0008) [2023-12-27 17:39:50,721][20381] Updated weights for policy 1, policy_version 1733293 (0.0008) [2023-12-27 17:39:50,775][20381] Updated weights for policy 1, policy_version 1733303 (0.0009) [2023-12-27 17:39:50,956][20347] Updated weights for policy 0, policy_version 1734764 (0.0007) [2023-12-27 17:39:51,014][20347] Updated weights for policy 0, policy_version 1734774 (0.0007) [2023-12-27 17:39:51,077][20347] Updated weights for policy 0, policy_version 1734784 (0.0006) [2023-12-27 17:39:51,535][20381] Updated weights for policy 1, policy_version 1733313 (0.0009) [2023-12-27 17:39:51,598][20381] Updated weights for policy 1, policy_version 1733323 (0.0007) [2023-12-27 17:39:51,662][20381] Updated weights for policy 1, policy_version 1733333 (0.0008) [2023-12-27 17:39:51,720][20381] Updated weights for policy 1, policy_version 1733343 (0.0008) [2023-12-27 17:39:51,793][20347] Updated weights for policy 0, policy_version 1734794 (0.0009) [2023-12-27 17:39:51,841][20347] Updated weights for policy 0, policy_version 1734804 (0.0008) [2023-12-27 17:39:51,900][20347] Updated weights for policy 0, policy_version 1734814 (0.0008) [2023-12-27 17:39:51,954][20347] Updated weights for policy 0, policy_version 1734824 (0.0008) [2023-12-27 17:39:52,351][20381] Updated weights for policy 1, policy_version 1733353 (0.0007) [2023-12-27 17:39:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 887980032. Throughput: 0: 10281.8, 1: 10292.6. Samples: 887953480. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:52,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:39:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001734824_444194816.pth... [2023-12-27 17:39:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001733640_443891712.pth [2023-12-27 17:39:52,373][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001734824_444194816.pth [2023-12-27 17:39:52,412][20381] Updated weights for policy 1, policy_version 1733363 (0.0006) [2023-12-27 17:39:52,471][20381] Updated weights for policy 1, policy_version 1733373 (0.0005) [2023-12-27 17:39:52,485][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001733376_443793408.pth... [2023-12-27 17:39:52,488][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001732160_443482112.pth [2023-12-27 17:39:52,489][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001733376_443793408.pth [2023-12-27 17:39:52,696][20347] Updated weights for policy 0, policy_version 1734834 (0.0007) [2023-12-27 17:39:52,755][20347] Updated weights for policy 0, policy_version 1734844 (0.0006) [2023-12-27 17:39:52,814][20347] Updated weights for policy 0, policy_version 1734854 (0.0007) [2023-12-27 17:39:53,065][20381] Updated weights for policy 1, policy_version 1733383 (0.0006) [2023-12-27 17:39:53,112][20381] Updated weights for policy 1, policy_version 1733393 (0.0005) [2023-12-27 17:39:53,163][20381] Updated weights for policy 1, policy_version 1733403 (0.0005) [2023-12-27 17:39:53,470][20347] Updated weights for policy 0, policy_version 1734864 (0.0007) [2023-12-27 17:39:53,527][20347] Updated weights for policy 0, policy_version 1734874 (0.0007) [2023-12-27 17:39:53,589][20347] Updated weights for policy 0, policy_version 1734884 (0.0006) [2023-12-27 17:39:53,804][20381] Updated weights for policy 1, policy_version 1733413 (0.0007) [2023-12-27 17:39:53,866][20381] Updated weights for policy 1, policy_version 1733423 (0.0009) [2023-12-27 17:39:53,924][20381] Updated weights for policy 1, policy_version 1733433 (0.0010) [2023-12-27 17:39:54,229][20347] Updated weights for policy 0, policy_version 1734894 (0.0008) [2023-12-27 17:39:54,283][20347] Updated weights for policy 0, policy_version 1734904 (0.0008) [2023-12-27 17:39:54,342][20347] Updated weights for policy 0, policy_version 1734914 (0.0007) [2023-12-27 17:39:54,581][20381] Updated weights for policy 1, policy_version 1733443 (0.0010) [2023-12-27 17:39:54,633][20381] Updated weights for policy 1, policy_version 1733453 (0.0008) [2023-12-27 17:39:54,688][20381] Updated weights for policy 1, policy_version 1733463 (0.0007) [2023-12-27 17:39:55,039][20347] Updated weights for policy 0, policy_version 1734924 (0.0008) [2023-12-27 17:39:55,098][20347] Updated weights for policy 0, policy_version 1734934 (0.0007) [2023-12-27 17:39:55,160][20347] Updated weights for policy 0, policy_version 1734944 (0.0008) [2023-12-27 17:39:55,360][20381] Updated weights for policy 1, policy_version 1733473 (0.0007) [2023-12-27 17:39:55,418][20381] Updated weights for policy 1, policy_version 1733483 (0.0005) [2023-12-27 17:39:55,476][20381] Updated weights for policy 1, policy_version 1733493 (0.0005) [2023-12-27 17:39:55,538][20381] Updated weights for policy 1, policy_version 1733503 (0.0008) [2023-12-27 17:39:55,858][20347] Updated weights for policy 0, policy_version 1734954 (0.0009) [2023-12-27 17:39:55,919][20347] Updated weights for policy 0, policy_version 1734964 (0.0010) [2023-12-27 17:39:55,979][20347] Updated weights for policy 0, policy_version 1734974 (0.0010) [2023-12-27 17:39:56,041][20347] Updated weights for policy 0, policy_version 1734984 (0.0010) [2023-12-27 17:39:56,136][20381] Updated weights for policy 1, policy_version 1733513 (0.0005) [2023-12-27 17:39:56,193][20381] Updated weights for policy 1, policy_version 1733523 (0.0005) [2023-12-27 17:39:56,242][20381] Updated weights for policy 1, policy_version 1733533 (0.0005) [2023-12-27 17:39:56,665][20347] Updated weights for policy 0, policy_version 1734994 (0.0008) [2023-12-27 17:39:56,719][20347] Updated weights for policy 0, policy_version 1735004 (0.0007) [2023-12-27 17:39:56,771][20347] Updated weights for policy 0, policy_version 1735014 (0.0006) [2023-12-27 17:39:56,847][20381] Updated weights for policy 1, policy_version 1733543 (0.0007) [2023-12-27 17:39:56,892][20381] Updated weights for policy 1, policy_version 1733553 (0.0009) [2023-12-27 17:39:56,951][20381] Updated weights for policy 1, policy_version 1733563 (0.0011) [2023-12-27 17:39:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 888086528. Throughput: 0: 10215.8, 1: 10348.6. Samples: 888076812. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:39:57,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:39:57,408][20347] Updated weights for policy 0, policy_version 1735024 (0.0009) [2023-12-27 17:39:57,453][20347] Updated weights for policy 0, policy_version 1735034 (0.0010) [2023-12-27 17:39:57,504][20347] Updated weights for policy 0, policy_version 1735044 (0.0010) [2023-12-27 17:39:57,575][20381] Updated weights for policy 1, policy_version 1733573 (0.0009) [2023-12-27 17:39:57,630][20381] Updated weights for policy 1, policy_version 1733583 (0.0008) [2023-12-27 17:39:57,689][20381] Updated weights for policy 1, policy_version 1733593 (0.0008) [2023-12-27 17:39:58,070][20347] Updated weights for policy 0, policy_version 1735054 (0.0010) [2023-12-27 17:39:58,115][20347] Updated weights for policy 0, policy_version 1735064 (0.0010) [2023-12-27 17:39:58,159][20347] Updated weights for policy 0, policy_version 1735074 (0.0010) [2023-12-27 17:39:58,378][20381] Updated weights for policy 1, policy_version 1733603 (0.0008) [2023-12-27 17:39:58,444][20381] Updated weights for policy 1, policy_version 1733613 (0.0008) [2023-12-27 17:39:58,503][20381] Updated weights for policy 1, policy_version 1733623 (0.0008) [2023-12-27 17:39:58,944][20347] Updated weights for policy 0, policy_version 1735084 (0.0008) [2023-12-27 17:39:59,003][20347] Updated weights for policy 0, policy_version 1735094 (0.0008) [2023-12-27 17:39:59,060][20347] Updated weights for policy 0, policy_version 1735104 (0.0008) [2023-12-27 17:39:59,319][20381] Updated weights for policy 1, policy_version 1733633 (0.0007) [2023-12-27 17:39:59,392][20381] Updated weights for policy 1, policy_version 1733643 (0.0007) [2023-12-27 17:39:59,446][20381] Updated weights for policy 1, policy_version 1733653 (0.0006) [2023-12-27 17:39:59,495][20381] Updated weights for policy 1, policy_version 1733663 (0.0006) [2023-12-27 17:39:59,778][20347] Updated weights for policy 0, policy_version 1735114 (0.0008) [2023-12-27 17:39:59,839][20347] Updated weights for policy 0, policy_version 1735124 (0.0009) [2023-12-27 17:39:59,904][20347] Updated weights for policy 0, policy_version 1735134 (0.0009) [2023-12-27 17:39:59,965][20347] Updated weights for policy 0, policy_version 1735144 (0.0008) [2023-12-27 17:40:00,103][20381] Updated weights for policy 1, policy_version 1733673 (0.0006) [2023-12-27 17:40:00,160][20381] Updated weights for policy 1, policy_version 1733683 (0.0008) [2023-12-27 17:40:00,223][20381] Updated weights for policy 1, policy_version 1733693 (0.0007) [2023-12-27 17:40:00,626][20347] Updated weights for policy 0, policy_version 1735154 (0.0006) [2023-12-27 17:40:00,682][20347] Updated weights for policy 0, policy_version 1735164 (0.0008) [2023-12-27 17:40:00,748][20347] Updated weights for policy 0, policy_version 1735174 (0.0007) [2023-12-27 17:40:00,856][20381] Updated weights for policy 1, policy_version 1733703 (0.0008) [2023-12-27 17:40:00,905][20381] Updated weights for policy 1, policy_version 1733713 (0.0007) [2023-12-27 17:40:00,967][20381] Updated weights for policy 1, policy_version 1733723 (0.0008) [2023-12-27 17:40:01,469][20347] Updated weights for policy 0, policy_version 1735184 (0.0008) [2023-12-27 17:40:01,530][20347] Updated weights for policy 0, policy_version 1735194 (0.0007) [2023-12-27 17:40:01,589][20347] Updated weights for policy 0, policy_version 1735204 (0.0008) [2023-12-27 17:40:01,708][20381] Updated weights for policy 1, policy_version 1733733 (0.0008) [2023-12-27 17:40:01,776][20381] Updated weights for policy 1, policy_version 1733743 (0.0009) [2023-12-27 17:40:01,824][20381] Updated weights for policy 1, policy_version 1733753 (0.0008) [2023-12-27 17:40:02,289][20347] Updated weights for policy 0, policy_version 1735214 (0.0009) [2023-12-27 17:40:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 888184832. Throughput: 0: 10240.5, 1: 10332.6. Samples: 888199144. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:02,363][19170] Avg episode reward: [(0, '970.630'), (1, '1000.000')] [2023-12-27 17:40:02,381][20347] Updated weights for policy 0, policy_version 1735224 (0.0008) [2023-12-27 17:40:02,441][20347] Updated weights for policy 0, policy_version 1735234 (0.0008) [2023-12-27 17:40:02,525][20381] Updated weights for policy 1, policy_version 1733763 (0.0006) [2023-12-27 17:40:02,587][20381] Updated weights for policy 1, policy_version 1733773 (0.0006) [2023-12-27 17:40:02,651][20381] Updated weights for policy 1, policy_version 1733783 (0.0006) [2023-12-27 17:40:03,148][20347] Updated weights for policy 0, policy_version 1735244 (0.0008) [2023-12-27 17:40:03,207][20347] Updated weights for policy 0, policy_version 1735254 (0.0007) [2023-12-27 17:40:03,264][20347] Updated weights for policy 0, policy_version 1735264 (0.0008) [2023-12-27 17:40:03,287][20381] Updated weights for policy 1, policy_version 1733793 (0.0006) [2023-12-27 17:40:03,335][20381] Updated weights for policy 1, policy_version 1733803 (0.0008) [2023-12-27 17:40:03,377][20381] Updated weights for policy 1, policy_version 1733813 (0.0007) [2023-12-27 17:40:03,428][20381] Updated weights for policy 1, policy_version 1733823 (0.0005) [2023-12-27 17:40:04,000][20347] Updated weights for policy 0, policy_version 1735274 (0.0007) [2023-12-27 17:40:04,052][20347] Updated weights for policy 0, policy_version 1735284 (0.0005) [2023-12-27 17:40:04,053][20381] Updated weights for policy 1, policy_version 1733833 (0.0010) [2023-12-27 17:40:04,105][20381] Updated weights for policy 1, policy_version 1733843 (0.0010) [2023-12-27 17:40:04,110][20347] Updated weights for policy 0, policy_version 1735294 (0.0006) [2023-12-27 17:40:04,164][20381] Updated weights for policy 1, policy_version 1733853 (0.0010) [2023-12-27 17:40:04,173][20347] Updated weights for policy 0, policy_version 1735304 (0.0005) [2023-12-27 17:40:04,844][20347] Updated weights for policy 0, policy_version 1735314 (0.0010) [2023-12-27 17:40:04,845][20381] Updated weights for policy 1, policy_version 1733863 (0.0008) [2023-12-27 17:40:04,898][20347] Updated weights for policy 0, policy_version 1735324 (0.0010) [2023-12-27 17:40:04,900][20381] Updated weights for policy 1, policy_version 1733873 (0.0005) [2023-12-27 17:40:04,950][20347] Updated weights for policy 0, policy_version 1735334 (0.0010) [2023-12-27 17:40:04,952][20381] Updated weights for policy 1, policy_version 1733883 (0.0005) [2023-12-27 17:40:05,611][20347] Updated weights for policy 0, policy_version 1735344 (0.0006) [2023-12-27 17:40:05,645][20381] Updated weights for policy 1, policy_version 1733893 (0.0007) [2023-12-27 17:40:05,661][20347] Updated weights for policy 0, policy_version 1735354 (0.0005) [2023-12-27 17:40:05,704][20381] Updated weights for policy 1, policy_version 1733903 (0.0008) [2023-12-27 17:40:05,710][20347] Updated weights for policy 0, policy_version 1735364 (0.0006) [2023-12-27 17:40:05,751][20381] Updated weights for policy 1, policy_version 1733913 (0.0006) [2023-12-27 17:40:06,366][20347] Updated weights for policy 0, policy_version 1735374 (0.0005) [2023-12-27 17:40:06,416][20381] Updated weights for policy 1, policy_version 1733923 (0.0008) [2023-12-27 17:40:06,424][20347] Updated weights for policy 0, policy_version 1735384 (0.0005) [2023-12-27 17:40:06,474][20381] Updated weights for policy 1, policy_version 1733933 (0.0007) [2023-12-27 17:40:06,488][20347] Updated weights for policy 0, policy_version 1735394 (0.0005) [2023-12-27 17:40:06,532][20381] Updated weights for policy 1, policy_version 1733943 (0.0007) [2023-12-27 17:40:07,023][20347] Updated weights for policy 0, policy_version 1735404 (0.0007) [2023-12-27 17:40:07,086][20347] Updated weights for policy 0, policy_version 1735414 (0.0006) [2023-12-27 17:40:07,144][20347] Updated weights for policy 0, policy_version 1735424 (0.0007) [2023-12-27 17:40:07,233][20381] Updated weights for policy 1, policy_version 1733953 (0.0009) [2023-12-27 17:40:07,296][20381] Updated weights for policy 1, policy_version 1733963 (0.0005) [2023-12-27 17:40:07,359][20381] Updated weights for policy 1, policy_version 1733973 (0.0005) [2023-12-27 17:40:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 888291328. Throughput: 0: 10210.2, 1: 10325.5. Samples: 888260152. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:07,363][19170] Avg episode reward: [(0, '951.090'), (1, '1000.000')] [2023-12-27 17:40:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001735432_444350464.pth... [2023-12-27 17:40:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001734248_444047360.pth [2023-12-27 17:40:07,419][20381] Updated weights for policy 1, policy_version 1733983 (0.0007) [2023-12-27 17:40:07,424][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001733984_443949056.pth... [2023-12-27 17:40:07,428][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001732768_443637760.pth [2023-12-27 17:40:07,801][20347] Updated weights for policy 0, policy_version 1735434 (0.0005) [2023-12-27 17:40:07,850][20347] Updated weights for policy 0, policy_version 1735444 (0.0006) [2023-12-27 17:40:07,898][20347] Updated weights for policy 0, policy_version 1735454 (0.0007) [2023-12-27 17:40:07,955][20347] Updated weights for policy 0, policy_version 1735464 (0.0008) [2023-12-27 17:40:08,004][20381] Updated weights for policy 1, policy_version 1733993 (0.0010) [2023-12-27 17:40:08,050][20381] Updated weights for policy 1, policy_version 1734003 (0.0010) [2023-12-27 17:40:08,102][20381] Updated weights for policy 1, policy_version 1734013 (0.0011) [2023-12-27 17:40:08,695][20347] Updated weights for policy 0, policy_version 1735474 (0.0007) [2023-12-27 17:40:08,751][20347] Updated weights for policy 0, policy_version 1735484 (0.0008) [2023-12-27 17:40:08,805][20347] Updated weights for policy 0, policy_version 1735494 (0.0007) [2023-12-27 17:40:08,811][20381] Updated weights for policy 1, policy_version 1734023 (0.0011) [2023-12-27 17:40:08,860][20381] Updated weights for policy 1, policy_version 1734033 (0.0011) [2023-12-27 17:40:08,913][20381] Updated weights for policy 1, policy_version 1734043 (0.0011) [2023-12-27 17:40:09,520][20347] Updated weights for policy 0, policy_version 1735504 (0.0008) [2023-12-27 17:40:09,580][20347] Updated weights for policy 0, policy_version 1735514 (0.0007) [2023-12-27 17:40:09,639][20347] Updated weights for policy 0, policy_version 1735524 (0.0007) [2023-12-27 17:40:09,641][20381] Updated weights for policy 1, policy_version 1734053 (0.0011) [2023-12-27 17:40:09,701][20381] Updated weights for policy 1, policy_version 1734063 (0.0011) [2023-12-27 17:40:09,755][20381] Updated weights for policy 1, policy_version 1734073 (0.0011) [2023-12-27 17:40:10,368][20347] Updated weights for policy 0, policy_version 1735534 (0.0008) [2023-12-27 17:40:10,422][20347] Updated weights for policy 0, policy_version 1735544 (0.0006) [2023-12-27 17:40:10,460][20381] Updated weights for policy 1, policy_version 1734083 (0.0009) [2023-12-27 17:40:10,476][20347] Updated weights for policy 0, policy_version 1735554 (0.0008) [2023-12-27 17:40:10,514][20381] Updated weights for policy 1, policy_version 1734093 (0.0005) [2023-12-27 17:40:10,569][20381] Updated weights for policy 1, policy_version 1734103 (0.0005) [2023-12-27 17:40:11,185][20347] Updated weights for policy 0, policy_version 1735564 (0.0008) [2023-12-27 17:40:11,245][20347] Updated weights for policy 0, policy_version 1735574 (0.0008) [2023-12-27 17:40:11,290][20381] Updated weights for policy 1, policy_version 1734113 (0.0005) [2023-12-27 17:40:11,299][20347] Updated weights for policy 0, policy_version 1735584 (0.0009) [2023-12-27 17:40:11,349][20381] Updated weights for policy 1, policy_version 1734123 (0.0006) [2023-12-27 17:40:11,416][20381] Updated weights for policy 1, policy_version 1734133 (0.0008) [2023-12-27 17:40:11,477][20381] Updated weights for policy 1, policy_version 1734143 (0.0008) [2023-12-27 17:40:12,108][20347] Updated weights for policy 0, policy_version 1735594 (0.0008) [2023-12-27 17:40:12,144][20381] Updated weights for policy 1, policy_version 1734153 (0.0007) [2023-12-27 17:40:12,165][20347] Updated weights for policy 0, policy_version 1735604 (0.0007) [2023-12-27 17:40:12,201][20381] Updated weights for policy 1, policy_version 1734163 (0.0010) [2023-12-27 17:40:12,222][20347] Updated weights for policy 0, policy_version 1735614 (0.0006) [2023-12-27 17:40:12,255][20381] Updated weights for policy 1, policy_version 1734173 (0.0011) [2023-12-27 17:40:12,279][20347] Updated weights for policy 0, policy_version 1735624 (0.0006) [2023-12-27 17:40:12,367][19170] Fps is (10 sec: 21289.6, 60 sec: 20615.0, 300 sec: 20576.9). Total num frames: 888397824. Throughput: 0: 10242.8, 1: 10353.4. Samples: 888383164. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:12,368][19170] Avg episode reward: [(0, '970.680'), (1, '1000.000')] [2023-12-27 17:40:12,942][20381] Updated weights for policy 1, policy_version 1734183 (0.0007) [2023-12-27 17:40:12,994][20381] Updated weights for policy 1, policy_version 1734193 (0.0006) [2023-12-27 17:40:13,026][20347] Updated weights for policy 0, policy_version 1735634 (0.0005) [2023-12-27 17:40:13,043][20381] Updated weights for policy 1, policy_version 1734203 (0.0005) [2023-12-27 17:40:13,079][20347] Updated weights for policy 0, policy_version 1735644 (0.0006) [2023-12-27 17:40:13,137][20347] Updated weights for policy 0, policy_version 1735654 (0.0008) [2023-12-27 17:40:13,685][20381] Updated weights for policy 1, policy_version 1734213 (0.0005) [2023-12-27 17:40:13,733][20381] Updated weights for policy 1, policy_version 1734223 (0.0005) [2023-12-27 17:40:13,785][20381] Updated weights for policy 1, policy_version 1734233 (0.0005) [2023-12-27 17:40:13,848][20347] Updated weights for policy 0, policy_version 1735664 (0.0007) [2023-12-27 17:40:13,900][20347] Updated weights for policy 0, policy_version 1735674 (0.0007) [2023-12-27 17:40:13,949][20347] Updated weights for policy 0, policy_version 1735684 (0.0008) [2023-12-27 17:40:14,421][20381] Updated weights for policy 1, policy_version 1734243 (0.0007) [2023-12-27 17:40:14,477][20381] Updated weights for policy 1, policy_version 1734253 (0.0009) [2023-12-27 17:40:14,529][20381] Updated weights for policy 1, policy_version 1734263 (0.0008) [2023-12-27 17:40:14,557][20347] Updated weights for policy 0, policy_version 1735694 (0.0006) [2023-12-27 17:40:14,602][20347] Updated weights for policy 0, policy_version 1735704 (0.0005) [2023-12-27 17:40:14,649][20347] Updated weights for policy 0, policy_version 1735714 (0.0008) [2023-12-27 17:40:15,217][20381] Updated weights for policy 1, policy_version 1734273 (0.0008) [2023-12-27 17:40:15,276][20381] Updated weights for policy 1, policy_version 1734283 (0.0008) [2023-12-27 17:40:15,325][20347] Updated weights for policy 0, policy_version 1735724 (0.0007) [2023-12-27 17:40:15,331][20381] Updated weights for policy 1, policy_version 1734293 (0.0007) [2023-12-27 17:40:15,381][20347] Updated weights for policy 0, policy_version 1735734 (0.0006) [2023-12-27 17:40:15,394][20381] Updated weights for policy 1, policy_version 1734303 (0.0006) [2023-12-27 17:40:15,439][20347] Updated weights for policy 0, policy_version 1735744 (0.0007) [2023-12-27 17:40:15,998][20381] Updated weights for policy 1, policy_version 1734313 (0.0007) [2023-12-27 17:40:16,060][20381] Updated weights for policy 1, policy_version 1734323 (0.0008) [2023-12-27 17:40:16,111][20381] Updated weights for policy 1, policy_version 1734333 (0.0008) [2023-12-27 17:40:16,157][20347] Updated weights for policy 0, policy_version 1735754 (0.0008) [2023-12-27 17:40:16,208][20347] Updated weights for policy 0, policy_version 1735764 (0.0008) [2023-12-27 17:40:16,265][20347] Updated weights for policy 0, policy_version 1735774 (0.0008) [2023-12-27 17:40:16,326][20347] Updated weights for policy 0, policy_version 1735784 (0.0007) [2023-12-27 17:40:16,804][20381] Updated weights for policy 1, policy_version 1734343 (0.0008) [2023-12-27 17:40:16,863][20381] Updated weights for policy 1, policy_version 1734353 (0.0008) [2023-12-27 17:40:16,922][20381] Updated weights for policy 1, policy_version 1734363 (0.0008) [2023-12-27 17:40:16,978][20347] Updated weights for policy 0, policy_version 1735794 (0.0007) [2023-12-27 17:40:17,031][20347] Updated weights for policy 0, policy_version 1735804 (0.0008) [2023-12-27 17:40:17,083][20347] Updated weights for policy 0, policy_version 1735814 (0.0005) [2023-12-27 17:40:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 888496128. Throughput: 0: 10152.4, 1: 10402.4. Samples: 888505372. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:17,363][19170] Avg episode reward: [(0, '960.920'), (1, '1000.000')] [2023-12-27 17:40:17,593][20381] Updated weights for policy 1, policy_version 1734373 (0.0007) [2023-12-27 17:40:17,648][20381] Updated weights for policy 1, policy_version 1734383 (0.0007) [2023-12-27 17:40:17,693][20347] Updated weights for policy 0, policy_version 1735824 (0.0005) [2023-12-27 17:40:17,695][20381] Updated weights for policy 1, policy_version 1734393 (0.0008) [2023-12-27 17:40:17,754][20347] Updated weights for policy 0, policy_version 1735834 (0.0010) [2023-12-27 17:40:17,815][20347] Updated weights for policy 0, policy_version 1735844 (0.0008) [2023-12-27 17:40:18,348][20381] Updated weights for policy 1, policy_version 1734403 (0.0008) [2023-12-27 17:40:18,411][20381] Updated weights for policy 1, policy_version 1734413 (0.0007) [2023-12-27 17:40:18,429][20347] Updated weights for policy 0, policy_version 1735854 (0.0007) [2023-12-27 17:40:18,465][20381] Updated weights for policy 1, policy_version 1734423 (0.0007) [2023-12-27 17:40:18,487][20347] Updated weights for policy 0, policy_version 1735864 (0.0006) [2023-12-27 17:40:18,537][20347] Updated weights for policy 0, policy_version 1735874 (0.0007) [2023-12-27 17:40:19,125][20381] Updated weights for policy 1, policy_version 1734433 (0.0008) [2023-12-27 17:40:19,171][20381] Updated weights for policy 1, policy_version 1734443 (0.0008) [2023-12-27 17:40:19,232][20381] Updated weights for policy 1, policy_version 1734453 (0.0008) [2023-12-27 17:40:19,260][20347] Updated weights for policy 0, policy_version 1735884 (0.0007) [2023-12-27 17:40:19,293][20381] Updated weights for policy 1, policy_version 1734463 (0.0006) [2023-12-27 17:40:19,318][20347] Updated weights for policy 0, policy_version 1735894 (0.0007) [2023-12-27 17:40:19,383][20347] Updated weights for policy 0, policy_version 1735904 (0.0009) [2023-12-27 17:40:20,018][20381] Updated weights for policy 1, policy_version 1734473 (0.0008) [2023-12-27 17:40:20,067][20381] Updated weights for policy 1, policy_version 1734483 (0.0007) [2023-12-27 17:40:20,130][20381] Updated weights for policy 1, policy_version 1734493 (0.0008) [2023-12-27 17:40:20,164][20347] Updated weights for policy 0, policy_version 1735914 (0.0009) [2023-12-27 17:40:20,223][20347] Updated weights for policy 0, policy_version 1735924 (0.0007) [2023-12-27 17:40:20,278][20347] Updated weights for policy 0, policy_version 1735934 (0.0007) [2023-12-27 17:40:20,337][20347] Updated weights for policy 0, policy_version 1735944 (0.0007) [2023-12-27 17:40:20,824][20381] Updated weights for policy 1, policy_version 1734503 (0.0009) [2023-12-27 17:40:20,890][20381] Updated weights for policy 1, policy_version 1734513 (0.0011) [2023-12-27 17:40:20,942][20381] Updated weights for policy 1, policy_version 1734523 (0.0011) [2023-12-27 17:40:21,049][20347] Updated weights for policy 0, policy_version 1735954 (0.0007) [2023-12-27 17:40:21,113][20347] Updated weights for policy 0, policy_version 1735964 (0.0007) [2023-12-27 17:40:21,179][20347] Updated weights for policy 0, policy_version 1735974 (0.0007) [2023-12-27 17:40:21,659][20381] Updated weights for policy 1, policy_version 1734533 (0.0010) [2023-12-27 17:40:21,728][20381] Updated weights for policy 1, policy_version 1734543 (0.0011) [2023-12-27 17:40:21,797][20381] Updated weights for policy 1, policy_version 1734553 (0.0009) [2023-12-27 17:40:21,975][20347] Updated weights for policy 0, policy_version 1735984 (0.0008) [2023-12-27 17:40:22,032][20347] Updated weights for policy 0, policy_version 1735994 (0.0008) [2023-12-27 17:40:22,087][20347] Updated weights for policy 0, policy_version 1736004 (0.0008) [2023-12-27 17:40:22,362][19170] Fps is (10 sec: 19669.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 888594432. Throughput: 0: 10173.5, 1: 10379.2. Samples: 888566656. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:22,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:40:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001734560_444096512.pth... [2023-12-27 17:40:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001736008_444497920.pth... [2023-12-27 17:40:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001733376_443793408.pth [2023-12-27 17:40:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001734824_444194816.pth [2023-12-27 17:40:22,477][20381] Updated weights for policy 1, policy_version 1734563 (0.0010) [2023-12-27 17:40:22,529][20381] Updated weights for policy 1, policy_version 1734573 (0.0010) [2023-12-27 17:40:22,586][20381] Updated weights for policy 1, policy_version 1734583 (0.0011) [2023-12-27 17:40:22,816][20347] Updated weights for policy 0, policy_version 1736014 (0.0008) [2023-12-27 17:40:22,873][20347] Updated weights for policy 0, policy_version 1736024 (0.0007) [2023-12-27 17:40:22,931][20347] Updated weights for policy 0, policy_version 1736034 (0.0007) [2023-12-27 17:40:23,228][20381] Updated weights for policy 1, policy_version 1734593 (0.0011) [2023-12-27 17:40:23,280][20381] Updated weights for policy 1, policy_version 1734603 (0.0008) [2023-12-27 17:40:23,330][20381] Updated weights for policy 1, policy_version 1734613 (0.0008) [2023-12-27 17:40:23,384][20381] Updated weights for policy 1, policy_version 1734623 (0.0008) [2023-12-27 17:40:23,640][20347] Updated weights for policy 0, policy_version 1736044 (0.0008) [2023-12-27 17:40:23,688][20347] Updated weights for policy 0, policy_version 1736054 (0.0010) [2023-12-27 17:40:23,739][20347] Updated weights for policy 0, policy_version 1736064 (0.0010) [2023-12-27 17:40:24,059][20381] Updated weights for policy 1, policy_version 1734633 (0.0010) [2023-12-27 17:40:24,114][20381] Updated weights for policy 1, policy_version 1734643 (0.0010) [2023-12-27 17:40:24,172][20381] Updated weights for policy 1, policy_version 1734653 (0.0011) [2023-12-27 17:40:24,423][20347] Updated weights for policy 0, policy_version 1736074 (0.0010) [2023-12-27 17:40:24,485][20347] Updated weights for policy 0, policy_version 1736084 (0.0007) [2023-12-27 17:40:24,540][20347] Updated weights for policy 0, policy_version 1736094 (0.0008) [2023-12-27 17:40:24,588][20347] Updated weights for policy 0, policy_version 1736104 (0.0005) [2023-12-27 17:40:24,822][20381] Updated weights for policy 1, policy_version 1734663 (0.0007) [2023-12-27 17:40:24,875][20381] Updated weights for policy 1, policy_version 1734673 (0.0005) [2023-12-27 17:40:24,931][20381] Updated weights for policy 1, policy_version 1734683 (0.0006) [2023-12-27 17:40:25,170][20347] Updated weights for policy 0, policy_version 1736114 (0.0007) [2023-12-27 17:40:25,229][20347] Updated weights for policy 0, policy_version 1736124 (0.0008) [2023-12-27 17:40:25,289][20347] Updated weights for policy 0, policy_version 1736134 (0.0008) [2023-12-27 17:40:25,609][20381] Updated weights for policy 1, policy_version 1734693 (0.0008) [2023-12-27 17:40:25,665][20381] Updated weights for policy 1, policy_version 1734703 (0.0007) [2023-12-27 17:40:25,722][20381] Updated weights for policy 1, policy_version 1734713 (0.0008) [2023-12-27 17:40:25,951][20347] Updated weights for policy 0, policy_version 1736144 (0.0008) [2023-12-27 17:40:26,007][20347] Updated weights for policy 0, policy_version 1736154 (0.0005) [2023-12-27 17:40:26,067][20347] Updated weights for policy 0, policy_version 1736164 (0.0005) [2023-12-27 17:40:26,453][20381] Updated weights for policy 1, policy_version 1734723 (0.0007) [2023-12-27 17:40:26,519][20381] Updated weights for policy 1, policy_version 1734733 (0.0006) [2023-12-27 17:40:26,584][20381] Updated weights for policy 1, policy_version 1734743 (0.0008) [2023-12-27 17:40:26,586][20347] Updated weights for policy 0, policy_version 1736174 (0.0006) [2023-12-27 17:40:26,645][20347] Updated weights for policy 0, policy_version 1736184 (0.0009) [2023-12-27 17:40:26,700][20347] Updated weights for policy 0, policy_version 1736194 (0.0008) [2023-12-27 17:40:27,193][20381] Updated weights for policy 1, policy_version 1734753 (0.0006) [2023-12-27 17:40:27,246][20381] Updated weights for policy 1, policy_version 1734763 (0.0006) [2023-12-27 17:40:27,300][20381] Updated weights for policy 1, policy_version 1734773 (0.0005) [2023-12-27 17:40:27,324][20347] Updated weights for policy 0, policy_version 1736204 (0.0007) [2023-12-27 17:40:27,357][20381] Updated weights for policy 1, policy_version 1734783 (0.0008) [2023-12-27 17:40:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 888700928. Throughput: 0: 10088.7, 1: 10362.6. Samples: 888687752. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:27,363][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 17:40:27,382][20347] Updated weights for policy 0, policy_version 1736214 (0.0010) [2023-12-27 17:40:27,432][20347] Updated weights for policy 0, policy_version 1736224 (0.0010) [2023-12-27 17:40:27,996][20381] Updated weights for policy 1, policy_version 1734793 (0.0010) [2023-12-27 17:40:28,044][20381] Updated weights for policy 1, policy_version 1734803 (0.0010) [2023-12-27 17:40:28,091][20381] Updated weights for policy 1, policy_version 1734813 (0.0010) [2023-12-27 17:40:28,107][20347] Updated weights for policy 0, policy_version 1736234 (0.0009) [2023-12-27 17:40:28,162][20347] Updated weights for policy 0, policy_version 1736244 (0.0006) [2023-12-27 17:40:28,214][20347] Updated weights for policy 0, policy_version 1736254 (0.0007) [2023-12-27 17:40:28,263][20347] Updated weights for policy 0, policy_version 1736264 (0.0007) [2023-12-27 17:40:28,750][20381] Updated weights for policy 1, policy_version 1734823 (0.0010) [2023-12-27 17:40:28,811][20381] Updated weights for policy 1, policy_version 1734833 (0.0010) [2023-12-27 17:40:28,870][20381] Updated weights for policy 1, policy_version 1734843 (0.0011) [2023-12-27 17:40:28,928][20347] Updated weights for policy 0, policy_version 1736274 (0.0008) [2023-12-27 17:40:28,991][20347] Updated weights for policy 0, policy_version 1736284 (0.0006) [2023-12-27 17:40:29,054][20347] Updated weights for policy 0, policy_version 1736294 (0.0006) [2023-12-27 17:40:29,576][20381] Updated weights for policy 1, policy_version 1734853 (0.0009) [2023-12-27 17:40:29,640][20381] Updated weights for policy 1, policy_version 1734863 (0.0007) [2023-12-27 17:40:29,697][20381] Updated weights for policy 1, policy_version 1734873 (0.0007) [2023-12-27 17:40:29,705][20347] Updated weights for policy 0, policy_version 1736304 (0.0005) [2023-12-27 17:40:29,761][20347] Updated weights for policy 0, policy_version 1736314 (0.0005) [2023-12-27 17:40:29,823][20347] Updated weights for policy 0, policy_version 1736324 (0.0006) [2023-12-27 17:40:30,379][20381] Updated weights for policy 1, policy_version 1734883 (0.0008) [2023-12-27 17:40:30,435][20381] Updated weights for policy 1, policy_version 1734893 (0.0008) [2023-12-27 17:40:30,489][20381] Updated weights for policy 1, policy_version 1734903 (0.0005) [2023-12-27 17:40:30,524][20347] Updated weights for policy 0, policy_version 1736334 (0.0008) [2023-12-27 17:40:30,577][20347] Updated weights for policy 0, policy_version 1736344 (0.0008) [2023-12-27 17:40:30,623][20347] Updated weights for policy 0, policy_version 1736354 (0.0008) [2023-12-27 17:40:31,160][20381] Updated weights for policy 1, policy_version 1734913 (0.0006) [2023-12-27 17:40:31,219][20381] Updated weights for policy 1, policy_version 1734923 (0.0008) [2023-12-27 17:40:31,285][20381] Updated weights for policy 1, policy_version 1734933 (0.0007) [2023-12-27 17:40:31,299][20347] Updated weights for policy 0, policy_version 1736364 (0.0007) [2023-12-27 17:40:31,339][20381] Updated weights for policy 1, policy_version 1734943 (0.0006) [2023-12-27 17:40:31,357][20347] Updated weights for policy 0, policy_version 1736374 (0.0007) [2023-12-27 17:40:31,410][20291] KL-divergence is very high: 112.2244 [2023-12-27 17:40:31,422][20347] Updated weights for policy 0, policy_version 1736384 (0.0008) [2023-12-27 17:40:31,433][20291] KL-divergence is very high: 115.6090 [2023-12-27 17:40:31,461][20291] KL-divergence is very high: 106.1110 [2023-12-27 17:40:32,061][20381] Updated weights for policy 1, policy_version 1734953 (0.0007) [2023-12-27 17:40:32,120][20381] Updated weights for policy 1, policy_version 1734963 (0.0008) [2023-12-27 17:40:32,149][20347] Updated weights for policy 0, policy_version 1736394 (0.0007) [2023-12-27 17:40:32,180][20381] Updated weights for policy 1, policy_version 1734973 (0.0007) [2023-12-27 17:40:32,204][20347] Updated weights for policy 0, policy_version 1736404 (0.0005) [2023-12-27 17:40:32,255][20347] Updated weights for policy 0, policy_version 1736414 (0.0008) [2023-12-27 17:40:32,324][20347] Updated weights for policy 0, policy_version 1736424 (0.0010) [2023-12-27 17:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 888807424. Throughput: 0: 10136.7, 1: 10324.8. Samples: 888811428. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:32,363][19170] Avg episode reward: [(0, '980.450'), (1, '1000.000')] [2023-12-27 17:40:32,820][20381] Updated weights for policy 1, policy_version 1734983 (0.0006) [2023-12-27 17:40:32,880][20381] Updated weights for policy 1, policy_version 1734993 (0.0008) [2023-12-27 17:40:32,935][20381] Updated weights for policy 1, policy_version 1735003 (0.0006) [2023-12-27 17:40:32,937][20347] Updated weights for policy 0, policy_version 1736434 (0.0011) [2023-12-27 17:40:32,996][20347] Updated weights for policy 0, policy_version 1736444 (0.0010) [2023-12-27 17:40:33,058][20347] Updated weights for policy 0, policy_version 1736454 (0.0010) [2023-12-27 17:40:33,515][20381] Updated weights for policy 1, policy_version 1735013 (0.0006) [2023-12-27 17:40:33,566][20381] Updated weights for policy 1, policy_version 1735023 (0.0005) [2023-12-27 17:40:33,620][20381] Updated weights for policy 1, policy_version 1735033 (0.0005) [2023-12-27 17:40:33,653][20347] Updated weights for policy 0, policy_version 1736464 (0.0010) [2023-12-27 17:40:33,709][20347] Updated weights for policy 0, policy_version 1736474 (0.0010) [2023-12-27 17:40:33,760][20347] Updated weights for policy 0, policy_version 1736484 (0.0010) [2023-12-27 17:40:34,162][20381] Updated weights for policy 1, policy_version 1735043 (0.0006) [2023-12-27 17:40:34,225][20381] Updated weights for policy 1, policy_version 1735053 (0.0007) [2023-12-27 17:40:34,282][20381] Updated weights for policy 1, policy_version 1735063 (0.0008) [2023-12-27 17:40:34,482][20347] Updated weights for policy 0, policy_version 1736494 (0.0008) [2023-12-27 17:40:34,545][20347] Updated weights for policy 0, policy_version 1736504 (0.0009) [2023-12-27 17:40:34,603][20347] Updated weights for policy 0, policy_version 1736514 (0.0010) [2023-12-27 17:40:34,930][20381] Updated weights for policy 1, policy_version 1735073 (0.0007) [2023-12-27 17:40:34,985][20381] Updated weights for policy 1, policy_version 1735083 (0.0007) [2023-12-27 17:40:35,036][20381] Updated weights for policy 1, policy_version 1735093 (0.0006) [2023-12-27 17:40:35,082][20381] Updated weights for policy 1, policy_version 1735103 (0.0005) [2023-12-27 17:40:35,275][20347] Updated weights for policy 0, policy_version 1736524 (0.0006) [2023-12-27 17:40:35,328][20347] Updated weights for policy 0, policy_version 1736534 (0.0005) [2023-12-27 17:40:35,379][20347] Updated weights for policy 0, policy_version 1736544 (0.0005) [2023-12-27 17:40:35,807][20381] Updated weights for policy 1, policy_version 1735113 (0.0007) [2023-12-27 17:40:35,859][20381] Updated weights for policy 1, policy_version 1735123 (0.0007) [2023-12-27 17:40:35,909][20381] Updated weights for policy 1, policy_version 1735133 (0.0007) [2023-12-27 17:40:36,007][20347] Updated weights for policy 0, policy_version 1736554 (0.0007) [2023-12-27 17:40:36,073][20347] Updated weights for policy 0, policy_version 1736564 (0.0009) [2023-12-27 17:40:36,162][20347] Updated weights for policy 0, policy_version 1736574 (0.0011) [2023-12-27 17:40:36,220][20347] Updated weights for policy 0, policy_version 1736584 (0.0008) [2023-12-27 17:40:36,590][20381] Updated weights for policy 1, policy_version 1735143 (0.0008) [2023-12-27 17:40:36,655][20381] Updated weights for policy 1, policy_version 1735153 (0.0008) [2023-12-27 17:40:36,717][20381] Updated weights for policy 1, policy_version 1735163 (0.0005) [2023-12-27 17:40:36,780][20347] Updated weights for policy 0, policy_version 1736594 (0.0005) [2023-12-27 17:40:36,828][20347] Updated weights for policy 0, policy_version 1736604 (0.0005) [2023-12-27 17:40:36,875][20347] Updated weights for policy 0, policy_version 1736614 (0.0005) [2023-12-27 17:40:37,357][20381] Updated weights for policy 1, policy_version 1735173 (0.0007) [2023-12-27 17:40:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 888905728. Throughput: 0: 10160.5, 1: 10342.7. Samples: 888876120. Policy #0 lag: (min: 16.0, avg: 39.6, max: 48.0) [2023-12-27 17:40:37,363][19170] Avg episode reward: [(0, '970.660'), (1, '1000.000')] [2023-12-27 17:40:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001736616_444653568.pth... [2023-12-27 17:40:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001735432_444350464.pth [2023-12-27 17:40:37,412][20381] Updated weights for policy 1, policy_version 1735183 (0.0007) [2023-12-27 17:40:37,470][20381] Updated weights for policy 1, policy_version 1735193 (0.0005) [2023-12-27 17:40:37,499][20347] Updated weights for policy 0, policy_version 1736624 (0.0007) [2023-12-27 17:40:37,510][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001735200_444260352.pth... [2023-12-27 17:40:37,514][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001733984_443949056.pth [2023-12-27 17:40:37,564][20347] Updated weights for policy 0, policy_version 1736634 (0.0007) [2023-12-27 17:40:37,624][20347] Updated weights for policy 0, policy_version 1736644 (0.0007) [2023-12-27 17:40:38,068][20381] Updated weights for policy 1, policy_version 1735203 (0.0007) [2023-12-27 17:40:38,134][20381] Updated weights for policy 1, policy_version 1735213 (0.0008) [2023-12-27 17:40:38,198][20381] Updated weights for policy 1, policy_version 1735223 (0.0008) [2023-12-27 17:40:38,268][20347] Updated weights for policy 0, policy_version 1736654 (0.0008) [2023-12-27 17:40:38,324][20347] Updated weights for policy 0, policy_version 1736664 (0.0006) [2023-12-27 17:40:38,388][20347] Updated weights for policy 0, policy_version 1736674 (0.0007) [2023-12-27 17:40:38,938][20381] Updated weights for policy 1, policy_version 1735233 (0.0007) [2023-12-27 17:40:38,993][20381] Updated weights for policy 1, policy_version 1735243 (0.0006) [2023-12-27 17:40:39,048][20381] Updated weights for policy 1, policy_version 1735253 (0.0009) [2023-12-27 17:40:39,107][20347] Updated weights for policy 0, policy_version 1736684 (0.0006) [2023-12-27 17:40:39,119][20381] Updated weights for policy 1, policy_version 1735263 (0.0008) [2023-12-27 17:40:39,162][20347] Updated weights for policy 0, policy_version 1736694 (0.0008) [2023-12-27 17:40:39,215][20347] Updated weights for policy 0, policy_version 1736704 (0.0006) [2023-12-27 17:40:39,807][20381] Updated weights for policy 1, policy_version 1735273 (0.0007) [2023-12-27 17:40:39,875][20381] Updated weights for policy 1, policy_version 1735283 (0.0008) [2023-12-27 17:40:39,934][20381] Updated weights for policy 1, policy_version 1735293 (0.0008) [2023-12-27 17:40:39,992][20347] Updated weights for policy 0, policy_version 1736714 (0.0008) [2023-12-27 17:40:40,059][20347] Updated weights for policy 0, policy_version 1736724 (0.0008) [2023-12-27 17:40:40,126][20347] Updated weights for policy 0, policy_version 1736734 (0.0008) [2023-12-27 17:40:40,200][20347] Updated weights for policy 0, policy_version 1736744 (0.0005) [2023-12-27 17:40:40,626][20381] Updated weights for policy 1, policy_version 1735303 (0.0008) [2023-12-27 17:40:40,688][20381] Updated weights for policy 1, policy_version 1735313 (0.0006) [2023-12-27 17:40:40,741][20381] Updated weights for policy 1, policy_version 1735323 (0.0006) [2023-12-27 17:40:40,806][20347] Updated weights for policy 0, policy_version 1736754 (0.0007) [2023-12-27 17:40:40,865][20347] Updated weights for policy 0, policy_version 1736764 (0.0008) [2023-12-27 17:40:40,916][20347] Updated weights for policy 0, policy_version 1736774 (0.0007) [2023-12-27 17:40:41,551][20381] Updated weights for policy 1, policy_version 1735333 (0.0007) [2023-12-27 17:40:41,568][20347] Updated weights for policy 0, policy_version 1736784 (0.0007) [2023-12-27 17:40:41,610][20381] Updated weights for policy 1, policy_version 1735343 (0.0007) [2023-12-27 17:40:41,622][20347] Updated weights for policy 0, policy_version 1736794 (0.0008) [2023-12-27 17:40:41,674][20381] Updated weights for policy 1, policy_version 1735353 (0.0008) [2023-12-27 17:40:41,689][20347] Updated weights for policy 0, policy_version 1736804 (0.0007) [2023-12-27 17:40:42,285][20347] Updated weights for policy 0, policy_version 1736814 (0.0009) [2023-12-27 17:40:42,348][20347] Updated weights for policy 0, policy_version 1736824 (0.0010) [2023-12-27 17:40:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 889004032. Throughput: 0: 10235.6, 1: 10274.5. Samples: 888999764. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:40:42,363][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 17:40:42,387][20381] Updated weights for policy 1, policy_version 1735363 (0.0007) [2023-12-27 17:40:42,412][20347] Updated weights for policy 0, policy_version 1736834 (0.0010) [2023-12-27 17:40:42,449][20381] Updated weights for policy 1, policy_version 1735373 (0.0007) [2023-12-27 17:40:42,511][20381] Updated weights for policy 1, policy_version 1735383 (0.0008) [2023-12-27 17:40:43,056][20347] Updated weights for policy 0, policy_version 1736844 (0.0007) [2023-12-27 17:40:43,113][20347] Updated weights for policy 0, policy_version 1736854 (0.0010) [2023-12-27 17:40:43,174][20347] Updated weights for policy 0, policy_version 1736864 (0.0010) [2023-12-27 17:40:43,208][20381] Updated weights for policy 1, policy_version 1735393 (0.0008) [2023-12-27 17:40:43,264][20381] Updated weights for policy 1, policy_version 1735403 (0.0005) [2023-12-27 17:40:43,312][20381] Updated weights for policy 1, policy_version 1735413 (0.0005) [2023-12-27 17:40:43,367][20381] Updated weights for policy 1, policy_version 1735423 (0.0005) [2023-12-27 17:40:43,826][20347] Updated weights for policy 0, policy_version 1736874 (0.0011) [2023-12-27 17:40:43,881][20347] Updated weights for policy 0, policy_version 1736884 (0.0010) [2023-12-27 17:40:43,935][20347] Updated weights for policy 0, policy_version 1736894 (0.0010) [2023-12-27 17:40:43,984][20347] Updated weights for policy 0, policy_version 1736904 (0.0010) [2023-12-27 17:40:44,011][20381] Updated weights for policy 1, policy_version 1735433 (0.0009) [2023-12-27 17:40:44,065][20381] Updated weights for policy 1, policy_version 1735443 (0.0007) [2023-12-27 17:40:44,122][20381] Updated weights for policy 1, policy_version 1735453 (0.0007) [2023-12-27 17:40:44,646][20347] Updated weights for policy 0, policy_version 1736914 (0.0005) [2023-12-27 17:40:44,693][20347] Updated weights for policy 0, policy_version 1736924 (0.0005) [2023-12-27 17:40:44,724][20381] Updated weights for policy 1, policy_version 1735463 (0.0005) [2023-12-27 17:40:44,737][20347] Updated weights for policy 0, policy_version 1736934 (0.0005) [2023-12-27 17:40:44,787][20381] Updated weights for policy 1, policy_version 1735473 (0.0007) [2023-12-27 17:40:44,852][20381] Updated weights for policy 1, policy_version 1735483 (0.0009) [2023-12-27 17:40:45,393][20347] Updated weights for policy 0, policy_version 1736944 (0.0006) [2023-12-27 17:40:45,450][20347] Updated weights for policy 0, policy_version 1736954 (0.0005) [2023-12-27 17:40:45,470][20381] Updated weights for policy 1, policy_version 1735493 (0.0006) [2023-12-27 17:40:45,510][20347] Updated weights for policy 0, policy_version 1736964 (0.0006) [2023-12-27 17:40:45,539][20381] Updated weights for policy 1, policy_version 1735503 (0.0005) [2023-12-27 17:40:45,601][20381] Updated weights for policy 1, policy_version 1735513 (0.0007) [2023-12-27 17:40:46,077][20347] Updated weights for policy 0, policy_version 1736974 (0.0008) [2023-12-27 17:40:46,134][20347] Updated weights for policy 0, policy_version 1736984 (0.0010) [2023-12-27 17:40:46,135][20381] Updated weights for policy 1, policy_version 1735523 (0.0005) [2023-12-27 17:40:46,190][20347] Updated weights for policy 0, policy_version 1736994 (0.0007) [2023-12-27 17:40:46,195][20381] Updated weights for policy 1, policy_version 1735533 (0.0006) [2023-12-27 17:40:46,255][20381] Updated weights for policy 1, policy_version 1735543 (0.0007) [2023-12-27 17:40:46,852][20347] Updated weights for policy 0, policy_version 1737004 (0.0008) [2023-12-27 17:40:46,874][20381] Updated weights for policy 1, policy_version 1735553 (0.0007) [2023-12-27 17:40:46,908][20347] Updated weights for policy 0, policy_version 1737014 (0.0010) [2023-12-27 17:40:46,937][20381] Updated weights for policy 1, policy_version 1735563 (0.0006) [2023-12-27 17:40:46,965][20347] Updated weights for policy 0, policy_version 1737024 (0.0011) [2023-12-27 17:40:46,993][20381] Updated weights for policy 1, policy_version 1735573 (0.0005) [2023-12-27 17:40:47,044][20381] Updated weights for policy 1, policy_version 1735583 (0.0007) [2023-12-27 17:40:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 889118720. Throughput: 0: 10279.5, 1: 10343.7. Samples: 889127188. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:40:47,363][19170] Avg episode reward: [(0, '970.640'), (1, '1000.000')] [2023-12-27 17:40:47,649][20347] Updated weights for policy 0, policy_version 1737034 (0.0010) [2023-12-27 17:40:47,663][20381] Updated weights for policy 1, policy_version 1735593 (0.0007) [2023-12-27 17:40:47,699][20347] Updated weights for policy 0, policy_version 1737044 (0.0005) [2023-12-27 17:40:47,720][20381] Updated weights for policy 1, policy_version 1735603 (0.0008) [2023-12-27 17:40:47,751][20347] Updated weights for policy 0, policy_version 1737054 (0.0008) [2023-12-27 17:40:47,771][20381] Updated weights for policy 1, policy_version 1735613 (0.0005) [2023-12-27 17:40:47,805][20347] Updated weights for policy 0, policy_version 1737064 (0.0007) [2023-12-27 17:40:48,380][20381] Updated weights for policy 1, policy_version 1735623 (0.0007) [2023-12-27 17:40:48,438][20381] Updated weights for policy 1, policy_version 1735633 (0.0007) [2023-12-27 17:40:48,455][20347] Updated weights for policy 0, policy_version 1737074 (0.0009) [2023-12-27 17:40:48,494][20381] Updated weights for policy 1, policy_version 1735643 (0.0006) [2023-12-27 17:40:48,511][20347] Updated weights for policy 0, policy_version 1737084 (0.0010) [2023-12-27 17:40:48,573][20347] Updated weights for policy 0, policy_version 1737094 (0.0010) [2023-12-27 17:40:49,145][20381] Updated weights for policy 1, policy_version 1735653 (0.0007) [2023-12-27 17:40:49,201][20381] Updated weights for policy 1, policy_version 1735663 (0.0007) [2023-12-27 17:40:49,255][20347] Updated weights for policy 0, policy_version 1737104 (0.0011) [2023-12-27 17:40:49,265][20381] Updated weights for policy 1, policy_version 1735673 (0.0008) [2023-12-27 17:40:49,310][20347] Updated weights for policy 0, policy_version 1737114 (0.0008) [2023-12-27 17:40:49,370][20347] Updated weights for policy 0, policy_version 1737124 (0.0008) [2023-12-27 17:40:49,948][20381] Updated weights for policy 1, policy_version 1735683 (0.0008) [2023-12-27 17:40:50,013][20381] Updated weights for policy 1, policy_version 1735693 (0.0008) [2023-12-27 17:40:50,077][20381] Updated weights for policy 1, policy_version 1735703 (0.0006) [2023-12-27 17:40:50,129][20347] Updated weights for policy 0, policy_version 1737134 (0.0007) [2023-12-27 17:40:50,185][20347] Updated weights for policy 0, policy_version 1737144 (0.0007) [2023-12-27 17:40:50,242][20347] Updated weights for policy 0, policy_version 1737154 (0.0007) [2023-12-27 17:40:50,697][20381] Updated weights for policy 1, policy_version 1735713 (0.0006) [2023-12-27 17:40:50,748][20381] Updated weights for policy 1, policy_version 1735723 (0.0011) [2023-12-27 17:40:50,806][20381] Updated weights for policy 1, policy_version 1735733 (0.0008) [2023-12-27 17:40:50,863][20381] Updated weights for policy 1, policy_version 1735743 (0.0008) [2023-12-27 17:40:50,987][20347] Updated weights for policy 0, policy_version 1737164 (0.0010) [2023-12-27 17:40:51,051][20347] Updated weights for policy 0, policy_version 1737174 (0.0008) [2023-12-27 17:40:51,101][20347] Updated weights for policy 0, policy_version 1737184 (0.0009) [2023-12-27 17:40:51,599][20381] Updated weights for policy 1, policy_version 1735753 (0.0009) [2023-12-27 17:40:51,661][20381] Updated weights for policy 1, policy_version 1735763 (0.0007) [2023-12-27 17:40:51,722][20381] Updated weights for policy 1, policy_version 1735773 (0.0011) [2023-12-27 17:40:51,797][20347] Updated weights for policy 0, policy_version 1737194 (0.0010) [2023-12-27 17:40:51,861][20347] Updated weights for policy 0, policy_version 1737204 (0.0010) [2023-12-27 17:40:51,919][20347] Updated weights for policy 0, policy_version 1737214 (0.0010) [2023-12-27 17:40:51,981][20347] Updated weights for policy 0, policy_version 1737224 (0.0010) [2023-12-27 17:40:52,327][20381] Updated weights for policy 1, policy_version 1735783 (0.0009) [2023-12-27 17:40:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 889217024. Throughput: 0: 10310.8, 1: 10343.8. Samples: 889189612. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:40:52,363][19170] Avg episode reward: [(0, '960.900'), (1, '1000.000')] [2023-12-27 17:40:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001737224_444809216.pth... [2023-12-27 17:40:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001736008_444497920.pth [2023-12-27 17:40:52,389][20381] Updated weights for policy 1, policy_version 1735793 (0.0009) [2023-12-27 17:40:52,445][20381] Updated weights for policy 1, policy_version 1735803 (0.0008) [2023-12-27 17:40:52,466][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001735808_444416000.pth... [2023-12-27 17:40:52,469][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001734560_444096512.pth [2023-12-27 17:40:52,648][20347] Updated weights for policy 0, policy_version 1737234 (0.0010) [2023-12-27 17:40:52,706][20347] Updated weights for policy 0, policy_version 1737244 (0.0008) [2023-12-27 17:40:52,771][20347] Updated weights for policy 0, policy_version 1737254 (0.0005) [2023-12-27 17:40:53,168][20381] Updated weights for policy 1, policy_version 1735813 (0.0008) [2023-12-27 17:40:53,224][20381] Updated weights for policy 1, policy_version 1735823 (0.0007) [2023-12-27 17:40:53,279][20381] Updated weights for policy 1, policy_version 1735833 (0.0008) [2023-12-27 17:40:53,419][20347] Updated weights for policy 0, policy_version 1737264 (0.0007) [2023-12-27 17:40:53,476][20347] Updated weights for policy 0, policy_version 1737274 (0.0008) [2023-12-27 17:40:53,528][20347] Updated weights for policy 0, policy_version 1737284 (0.0007) [2023-12-27 17:40:53,993][20381] Updated weights for policy 1, policy_version 1735843 (0.0007) [2023-12-27 17:40:54,054][20381] Updated weights for policy 1, policy_version 1735853 (0.0005) [2023-12-27 17:40:54,118][20381] Updated weights for policy 1, policy_version 1735863 (0.0005) [2023-12-27 17:40:54,131][20347] Updated weights for policy 0, policy_version 1737294 (0.0006) [2023-12-27 17:40:54,187][20347] Updated weights for policy 0, policy_version 1737304 (0.0006) [2023-12-27 17:40:54,241][20347] Updated weights for policy 0, policy_version 1737314 (0.0008) [2023-12-27 17:40:54,723][20381] Updated weights for policy 1, policy_version 1735873 (0.0006) [2023-12-27 17:40:54,779][20381] Updated weights for policy 1, policy_version 1735883 (0.0008) [2023-12-27 17:40:54,834][20381] Updated weights for policy 1, policy_version 1735893 (0.0008) [2023-12-27 17:40:54,840][20347] Updated weights for policy 0, policy_version 1737324 (0.0006) [2023-12-27 17:40:54,892][20381] Updated weights for policy 1, policy_version 1735903 (0.0007) [2023-12-27 17:40:54,894][20347] Updated weights for policy 0, policy_version 1737334 (0.0006) [2023-12-27 17:40:54,951][20347] Updated weights for policy 0, policy_version 1737344 (0.0008) [2023-12-27 17:40:55,545][20347] Updated weights for policy 0, policy_version 1737354 (0.0008) [2023-12-27 17:40:55,601][20347] Updated weights for policy 0, policy_version 1737364 (0.0006) [2023-12-27 17:40:55,618][20381] Updated weights for policy 1, policy_version 1735913 (0.0006) [2023-12-27 17:40:55,650][20347] Updated weights for policy 0, policy_version 1737374 (0.0005) [2023-12-27 17:40:55,670][20381] Updated weights for policy 1, policy_version 1735923 (0.0006) [2023-12-27 17:40:55,698][20347] Updated weights for policy 0, policy_version 1737384 (0.0005) [2023-12-27 17:40:55,724][20381] Updated weights for policy 1, policy_version 1735933 (0.0006) [2023-12-27 17:40:56,372][20381] Updated weights for policy 1, policy_version 1735943 (0.0007) [2023-12-27 17:40:56,402][20347] Updated weights for policy 0, policy_version 1737394 (0.0006) [2023-12-27 17:40:56,430][20381] Updated weights for policy 1, policy_version 1735953 (0.0007) [2023-12-27 17:40:56,463][20347] Updated weights for policy 0, policy_version 1737404 (0.0008) [2023-12-27 17:40:56,483][20381] Updated weights for policy 1, policy_version 1735963 (0.0005) [2023-12-27 17:40:56,518][20347] Updated weights for policy 0, policy_version 1737414 (0.0006) [2023-12-27 17:40:57,114][20347] Updated weights for policy 0, policy_version 1737424 (0.0009) [2023-12-27 17:40:57,153][20381] Updated weights for policy 1, policy_version 1735973 (0.0005) [2023-12-27 17:40:57,166][20347] Updated weights for policy 0, policy_version 1737434 (0.0010) [2023-12-27 17:40:57,205][20381] Updated weights for policy 1, policy_version 1735983 (0.0007) [2023-12-27 17:40:57,214][20347] Updated weights for policy 0, policy_version 1737444 (0.0010) [2023-12-27 17:40:57,258][20381] Updated weights for policy 1, policy_version 1735993 (0.0006) [2023-12-27 17:40:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 889331712. Throughput: 0: 10333.8, 1: 10350.6. Samples: 889313868. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:40:57,363][19170] Avg episode reward: [(0, '960.900'), (1, '990.320')] [2023-12-27 17:40:57,870][20381] Updated weights for policy 1, policy_version 1736003 (0.0008) [2023-12-27 17:40:57,880][20347] Updated weights for policy 0, policy_version 1737454 (0.0009) [2023-12-27 17:40:57,926][20381] Updated weights for policy 1, policy_version 1736013 (0.0007) [2023-12-27 17:40:57,931][20347] Updated weights for policy 0, policy_version 1737464 (0.0010) [2023-12-27 17:40:57,974][20381] Updated weights for policy 1, policy_version 1736023 (0.0005) [2023-12-27 17:40:57,985][20347] Updated weights for policy 0, policy_version 1737474 (0.0010) [2023-12-27 17:40:58,751][20381] Updated weights for policy 1, policy_version 1736033 (0.0007) [2023-12-27 17:40:58,757][20347] Updated weights for policy 0, policy_version 1737484 (0.0009) [2023-12-27 17:40:58,820][20347] Updated weights for policy 0, policy_version 1737494 (0.0009) [2023-12-27 17:40:58,833][20381] Updated weights for policy 1, policy_version 1736043 (0.0009) [2023-12-27 17:40:58,884][20347] Updated weights for policy 0, policy_version 1737504 (0.0009) [2023-12-27 17:40:58,890][20381] Updated weights for policy 1, policy_version 1736053 (0.0009) [2023-12-27 17:40:58,946][20381] Updated weights for policy 1, policy_version 1736063 (0.0009) [2023-12-27 17:40:59,662][20347] Updated weights for policy 0, policy_version 1737514 (0.0009) [2023-12-27 17:40:59,721][20347] Updated weights for policy 0, policy_version 1737524 (0.0006) [2023-12-27 17:40:59,733][20381] Updated weights for policy 1, policy_version 1736073 (0.0008) [2023-12-27 17:40:59,776][20347] Updated weights for policy 0, policy_version 1737534 (0.0005) [2023-12-27 17:40:59,793][20381] Updated weights for policy 1, policy_version 1736083 (0.0007) [2023-12-27 17:40:59,835][20347] Updated weights for policy 0, policy_version 1737544 (0.0006) [2023-12-27 17:40:59,859][20381] Updated weights for policy 1, policy_version 1736093 (0.0008) [2023-12-27 17:41:00,517][20347] Updated weights for policy 0, policy_version 1737554 (0.0007) [2023-12-27 17:41:00,551][20381] Updated weights for policy 1, policy_version 1736103 (0.0007) [2023-12-27 17:41:00,568][20347] Updated weights for policy 0, policy_version 1737564 (0.0006) [2023-12-27 17:41:00,603][20381] Updated weights for policy 1, policy_version 1736113 (0.0006) [2023-12-27 17:41:00,619][20347] Updated weights for policy 0, policy_version 1737574 (0.0006) [2023-12-27 17:41:00,657][20381] Updated weights for policy 1, policy_version 1736123 (0.0007) [2023-12-27 17:41:01,251][20347] Updated weights for policy 0, policy_version 1737584 (0.0007) [2023-12-27 17:41:01,307][20347] Updated weights for policy 0, policy_version 1737594 (0.0007) [2023-12-27 17:41:01,369][20347] Updated weights for policy 0, policy_version 1737604 (0.0007) [2023-12-27 17:41:01,390][20381] Updated weights for policy 1, policy_version 1736133 (0.0011) [2023-12-27 17:41:01,451][20381] Updated weights for policy 1, policy_version 1736143 (0.0011) [2023-12-27 17:41:01,513][20381] Updated weights for policy 1, policy_version 1736153 (0.0011) [2023-12-27 17:41:02,091][20347] Updated weights for policy 0, policy_version 1737614 (0.0009) [2023-12-27 17:41:02,147][20347] Updated weights for policy 0, policy_version 1737624 (0.0008) [2023-12-27 17:41:02,198][20347] Updated weights for policy 0, policy_version 1737634 (0.0008) [2023-12-27 17:41:02,200][20381] Updated weights for policy 1, policy_version 1736163 (0.0009) [2023-12-27 17:41:02,259][20381] Updated weights for policy 1, policy_version 1736173 (0.0007) [2023-12-27 17:41:02,321][20381] Updated weights for policy 1, policy_version 1736183 (0.0008) [2023-12-27 17:41:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 889421824. Throughput: 0: 10354.7, 1: 10277.9. Samples: 889433836. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:02,363][19170] Avg episode reward: [(0, '960.910'), (1, '990.320')] [2023-12-27 17:41:02,944][20347] Updated weights for policy 0, policy_version 1737644 (0.0006) [2023-12-27 17:41:02,988][20347] Updated weights for policy 0, policy_version 1737654 (0.0007) [2023-12-27 17:41:02,993][20381] Updated weights for policy 1, policy_version 1736193 (0.0008) [2023-12-27 17:41:03,036][20347] Updated weights for policy 0, policy_version 1737664 (0.0005) [2023-12-27 17:41:03,039][20381] Updated weights for policy 1, policy_version 1736203 (0.0010) [2023-12-27 17:41:03,087][20381] Updated weights for policy 1, policy_version 1736213 (0.0007) [2023-12-27 17:41:03,136][20381] Updated weights for policy 1, policy_version 1736223 (0.0005) [2023-12-27 17:41:03,769][20347] Updated weights for policy 0, policy_version 1737674 (0.0006) [2023-12-27 17:41:03,826][20347] Updated weights for policy 0, policy_version 1737684 (0.0006) [2023-12-27 17:41:03,840][20381] Updated weights for policy 1, policy_version 1736233 (0.0006) [2023-12-27 17:41:03,894][20347] Updated weights for policy 0, policy_version 1737694 (0.0006) [2023-12-27 17:41:03,904][20381] Updated weights for policy 1, policy_version 1736243 (0.0007) [2023-12-27 17:41:03,957][20347] Updated weights for policy 0, policy_version 1737704 (0.0008) [2023-12-27 17:41:03,963][20381] Updated weights for policy 1, policy_version 1736253 (0.0006) [2023-12-27 17:41:04,617][20381] Updated weights for policy 1, policy_version 1736263 (0.0007) [2023-12-27 17:41:04,662][20347] Updated weights for policy 0, policy_version 1737714 (0.0007) [2023-12-27 17:41:04,675][20381] Updated weights for policy 1, policy_version 1736273 (0.0005) [2023-12-27 17:41:04,723][20347] Updated weights for policy 0, policy_version 1737724 (0.0006) [2023-12-27 17:41:04,729][20381] Updated weights for policy 1, policy_version 1736283 (0.0008) [2023-12-27 17:41:04,777][20347] Updated weights for policy 0, policy_version 1737734 (0.0007) [2023-12-27 17:41:05,317][20381] Updated weights for policy 1, policy_version 1736293 (0.0010) [2023-12-27 17:41:05,374][20381] Updated weights for policy 1, policy_version 1736303 (0.0008) [2023-12-27 17:41:05,432][20381] Updated weights for policy 1, policy_version 1736313 (0.0005) [2023-12-27 17:41:05,503][20347] Updated weights for policy 0, policy_version 1737744 (0.0007) [2023-12-27 17:41:05,551][20347] Updated weights for policy 0, policy_version 1737754 (0.0008) [2023-12-27 17:41:05,599][20347] Updated weights for policy 0, policy_version 1737764 (0.0008) [2023-12-27 17:41:05,972][20381] Updated weights for policy 1, policy_version 1736323 (0.0007) [2023-12-27 17:41:06,021][20381] Updated weights for policy 1, policy_version 1736333 (0.0011) [2023-12-27 17:41:06,077][20381] Updated weights for policy 1, policy_version 1736343 (0.0011) [2023-12-27 17:41:06,373][20347] Updated weights for policy 0, policy_version 1737774 (0.0008) [2023-12-27 17:41:06,435][20347] Updated weights for policy 0, policy_version 1737784 (0.0008) [2023-12-27 17:41:06,497][20347] Updated weights for policy 0, policy_version 1737794 (0.0007) [2023-12-27 17:41:06,777][20381] Updated weights for policy 1, policy_version 1736353 (0.0010) [2023-12-27 17:41:06,833][20381] Updated weights for policy 1, policy_version 1736363 (0.0006) [2023-12-27 17:41:06,897][20381] Updated weights for policy 1, policy_version 1736373 (0.0006) [2023-12-27 17:41:06,960][20381] Updated weights for policy 1, policy_version 1736383 (0.0006) [2023-12-27 17:41:07,215][20347] Updated weights for policy 0, policy_version 1737804 (0.0008) [2023-12-27 17:41:07,260][20347] Updated weights for policy 0, policy_version 1737814 (0.0008) [2023-12-27 17:41:07,316][20347] Updated weights for policy 0, policy_version 1737824 (0.0008) [2023-12-27 17:41:07,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20616.4, 300 sec: 20549.4). Total num frames: 889528320. Throughput: 0: 10319.1, 1: 10289.6. Samples: 889494052. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:07,364][19170] Avg episode reward: [(0, '990.270'), (1, '1000.000')] [2023-12-27 17:41:07,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001736384_444563456.pth... [2023-12-27 17:41:07,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001737832_444964864.pth... [2023-12-27 17:41:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001735200_444260352.pth [2023-12-27 17:41:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001736616_444653568.pth [2023-12-27 17:41:07,498][20381] Updated weights for policy 1, policy_version 1736393 (0.0008) [2023-12-27 17:41:07,548][20381] Updated weights for policy 1, policy_version 1736403 (0.0008) [2023-12-27 17:41:07,599][20381] Updated weights for policy 1, policy_version 1736413 (0.0007) [2023-12-27 17:41:08,012][20347] Updated weights for policy 0, policy_version 1737834 (0.0008) [2023-12-27 17:41:08,069][20347] Updated weights for policy 0, policy_version 1737844 (0.0005) [2023-12-27 17:41:08,128][20347] Updated weights for policy 0, policy_version 1737854 (0.0005) [2023-12-27 17:41:08,178][20347] Updated weights for policy 0, policy_version 1737864 (0.0005) [2023-12-27 17:41:08,242][20381] Updated weights for policy 1, policy_version 1736423 (0.0008) [2023-12-27 17:41:08,292][20381] Updated weights for policy 1, policy_version 1736433 (0.0010) [2023-12-27 17:41:08,342][20381] Updated weights for policy 1, policy_version 1736443 (0.0010) [2023-12-27 17:41:08,763][20347] Updated weights for policy 0, policy_version 1737874 (0.0008) [2023-12-27 17:41:08,827][20347] Updated weights for policy 0, policy_version 1737884 (0.0007) [2023-12-27 17:41:08,888][20347] Updated weights for policy 0, policy_version 1737894 (0.0009) [2023-12-27 17:41:09,075][20381] Updated weights for policy 1, policy_version 1736453 (0.0008) [2023-12-27 17:41:09,120][20381] Updated weights for policy 1, policy_version 1736463 (0.0007) [2023-12-27 17:41:09,176][20381] Updated weights for policy 1, policy_version 1736473 (0.0006) [2023-12-27 17:41:09,627][20347] Updated weights for policy 0, policy_version 1737904 (0.0008) [2023-12-27 17:41:09,688][20347] Updated weights for policy 0, policy_version 1737914 (0.0008) [2023-12-27 17:41:09,752][20347] Updated weights for policy 0, policy_version 1737924 (0.0008) [2023-12-27 17:41:09,883][20381] Updated weights for policy 1, policy_version 1736483 (0.0006) [2023-12-27 17:41:09,942][20381] Updated weights for policy 1, policy_version 1736493 (0.0008) [2023-12-27 17:41:10,003][20381] Updated weights for policy 1, policy_version 1736503 (0.0006) [2023-12-27 17:41:10,495][20347] Updated weights for policy 0, policy_version 1737934 (0.0008) [2023-12-27 17:41:10,557][20347] Updated weights for policy 0, policy_version 1737944 (0.0008) [2023-12-27 17:41:10,618][20347] Updated weights for policy 0, policy_version 1737954 (0.0008) [2023-12-27 17:41:10,667][20381] Updated weights for policy 1, policy_version 1736513 (0.0009) [2023-12-27 17:41:10,724][20381] Updated weights for policy 1, policy_version 1736523 (0.0010) [2023-12-27 17:41:10,786][20381] Updated weights for policy 1, policy_version 1736533 (0.0006) [2023-12-27 17:41:10,848][20381] Updated weights for policy 1, policy_version 1736543 (0.0008) [2023-12-27 17:41:11,346][20347] Updated weights for policy 0, policy_version 1737964 (0.0008) [2023-12-27 17:41:11,417][20347] Updated weights for policy 0, policy_version 1737974 (0.0008) [2023-12-27 17:41:11,473][20347] Updated weights for policy 0, policy_version 1737984 (0.0008) [2023-12-27 17:41:11,556][20381] Updated weights for policy 1, policy_version 1736553 (0.0011) [2023-12-27 17:41:11,617][20381] Updated weights for policy 1, policy_version 1736563 (0.0011) [2023-12-27 17:41:11,681][20381] Updated weights for policy 1, policy_version 1736573 (0.0008) [2023-12-27 17:41:12,210][20347] Updated weights for policy 0, policy_version 1737994 (0.0008) [2023-12-27 17:41:12,278][20347] Updated weights for policy 0, policy_version 1738004 (0.0007) [2023-12-27 17:41:12,342][20347] Updated weights for policy 0, policy_version 1738014 (0.0008) [2023-12-27 17:41:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20345.0, 300 sec: 20493.9). Total num frames: 889618432. Throughput: 0: 10294.6, 1: 10341.6. Samples: 889616388. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:12,363][19170] Avg episode reward: [(0, '970.650'), (1, '1000.000')] [2023-12-27 17:41:12,405][20347] Updated weights for policy 0, policy_version 1738024 (0.0007) [2023-12-27 17:41:12,424][20381] Updated weights for policy 1, policy_version 1736583 (0.0007) [2023-12-27 17:41:12,476][20381] Updated weights for policy 1, policy_version 1736593 (0.0007) [2023-12-27 17:41:12,526][20381] Updated weights for policy 1, policy_version 1736603 (0.0008) [2023-12-27 17:41:13,066][20347] Updated weights for policy 0, policy_version 1738034 (0.0005) [2023-12-27 17:41:13,125][20347] Updated weights for policy 0, policy_version 1738044 (0.0005) [2023-12-27 17:41:13,149][20381] Updated weights for policy 1, policy_version 1736613 (0.0008) [2023-12-27 17:41:13,186][20347] Updated weights for policy 0, policy_version 1738054 (0.0006) [2023-12-27 17:41:13,205][20381] Updated weights for policy 1, policy_version 1736623 (0.0006) [2023-12-27 17:41:13,267][20381] Updated weights for policy 1, policy_version 1736633 (0.0005) [2023-12-27 17:41:13,815][20347] Updated weights for policy 0, policy_version 1738064 (0.0006) [2023-12-27 17:41:13,866][20347] Updated weights for policy 0, policy_version 1738074 (0.0005) [2023-12-27 17:41:13,874][20381] Updated weights for policy 1, policy_version 1736643 (0.0008) [2023-12-27 17:41:13,923][20347] Updated weights for policy 0, policy_version 1738084 (0.0005) [2023-12-27 17:41:13,928][20381] Updated weights for policy 1, policy_version 1736653 (0.0006) [2023-12-27 17:41:13,986][20381] Updated weights for policy 1, policy_version 1736663 (0.0008) [2023-12-27 17:41:14,576][20347] Updated weights for policy 0, policy_version 1738094 (0.0007) [2023-12-27 17:41:14,623][20381] Updated weights for policy 1, policy_version 1736673 (0.0008) [2023-12-27 17:41:14,633][20347] Updated weights for policy 0, policy_version 1738104 (0.0007) [2023-12-27 17:41:14,675][20381] Updated weights for policy 1, policy_version 1736683 (0.0009) [2023-12-27 17:41:14,687][20347] Updated weights for policy 0, policy_version 1738114 (0.0007) [2023-12-27 17:41:14,725][20381] Updated weights for policy 1, policy_version 1736693 (0.0005) [2023-12-27 17:41:14,789][20381] Updated weights for policy 1, policy_version 1736703 (0.0007) [2023-12-27 17:41:15,417][20347] Updated weights for policy 0, policy_version 1738124 (0.0007) [2023-12-27 17:41:15,448][20381] Updated weights for policy 1, policy_version 1736713 (0.0007) [2023-12-27 17:41:15,462][20347] Updated weights for policy 0, policy_version 1738134 (0.0008) [2023-12-27 17:41:15,503][20381] Updated weights for policy 1, policy_version 1736723 (0.0006) [2023-12-27 17:41:15,523][20347] Updated weights for policy 0, policy_version 1738144 (0.0008) [2023-12-27 17:41:15,562][20381] Updated weights for policy 1, policy_version 1736733 (0.0006) [2023-12-27 17:41:16,180][20347] Updated weights for policy 0, policy_version 1738154 (0.0007) [2023-12-27 17:41:16,215][20381] Updated weights for policy 1, policy_version 1736743 (0.0007) [2023-12-27 17:41:16,237][20347] Updated weights for policy 0, policy_version 1738164 (0.0008) [2023-12-27 17:41:16,263][20381] Updated weights for policy 1, policy_version 1736753 (0.0010) [2023-12-27 17:41:16,294][20347] Updated weights for policy 0, policy_version 1738174 (0.0006) [2023-12-27 17:41:16,314][20381] Updated weights for policy 1, policy_version 1736763 (0.0011) [2023-12-27 17:41:16,348][20347] Updated weights for policy 0, policy_version 1738184 (0.0007) [2023-12-27 17:41:17,051][20347] Updated weights for policy 0, policy_version 1738194 (0.0008) [2023-12-27 17:41:17,060][20381] Updated weights for policy 1, policy_version 1736773 (0.0011) [2023-12-27 17:41:17,108][20347] Updated weights for policy 0, policy_version 1738204 (0.0006) [2023-12-27 17:41:17,110][20381] Updated weights for policy 1, policy_version 1736783 (0.0007) [2023-12-27 17:41:17,163][20381] Updated weights for policy 1, policy_version 1736793 (0.0008) [2023-12-27 17:41:17,165][20347] Updated weights for policy 0, policy_version 1738214 (0.0006) [2023-12-27 17:41:17,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 889733120. Throughput: 0: 10239.1, 1: 10373.9. Samples: 889739012. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:17,363][19170] Avg episode reward: [(0, '951.070'), (1, '1000.000')] [2023-12-27 17:41:17,851][20347] Updated weights for policy 0, policy_version 1738224 (0.0005) [2023-12-27 17:41:17,889][20381] Updated weights for policy 1, policy_version 1736803 (0.0009) [2023-12-27 17:41:17,905][20347] Updated weights for policy 0, policy_version 1738234 (0.0005) [2023-12-27 17:41:17,942][20381] Updated weights for policy 1, policy_version 1736813 (0.0009) [2023-12-27 17:41:17,963][20347] Updated weights for policy 0, policy_version 1738244 (0.0006) [2023-12-27 17:41:17,996][20381] Updated weights for policy 1, policy_version 1736823 (0.0007) [2023-12-27 17:41:18,568][20347] Updated weights for policy 0, policy_version 1738254 (0.0006) [2023-12-27 17:41:18,621][20347] Updated weights for policy 0, policy_version 1738264 (0.0006) [2023-12-27 17:41:18,660][20381] Updated weights for policy 1, policy_version 1736833 (0.0011) [2023-12-27 17:41:18,671][20347] Updated weights for policy 0, policy_version 1738274 (0.0007) [2023-12-27 17:41:18,712][20381] Updated weights for policy 1, policy_version 1736843 (0.0011) [2023-12-27 17:41:18,771][20381] Updated weights for policy 1, policy_version 1736853 (0.0011) [2023-12-27 17:41:18,830][20381] Updated weights for policy 1, policy_version 1736863 (0.0011) [2023-12-27 17:41:19,318][20347] Updated weights for policy 0, policy_version 1738284 (0.0009) [2023-12-27 17:41:19,383][20347] Updated weights for policy 0, policy_version 1738294 (0.0008) [2023-12-27 17:41:19,442][20347] Updated weights for policy 0, policy_version 1738304 (0.0008) [2023-12-27 17:41:19,564][20381] Updated weights for policy 1, policy_version 1736873 (0.0007) [2023-12-27 17:41:19,631][20381] Updated weights for policy 1, policy_version 1736883 (0.0008) [2023-12-27 17:41:19,693][20381] Updated weights for policy 1, policy_version 1736893 (0.0008) [2023-12-27 17:41:20,196][20347] Updated weights for policy 0, policy_version 1738314 (0.0009) [2023-12-27 17:41:20,248][20347] Updated weights for policy 0, policy_version 1738324 (0.0010) [2023-12-27 17:41:20,301][20347] Updated weights for policy 0, policy_version 1738334 (0.0010) [2023-12-27 17:41:20,360][20347] Updated weights for policy 0, policy_version 1738344 (0.0011) [2023-12-27 17:41:20,419][20381] Updated weights for policy 1, policy_version 1736903 (0.0009) [2023-12-27 17:41:20,476][20381] Updated weights for policy 1, policy_version 1736913 (0.0007) [2023-12-27 17:41:20,533][20381] Updated weights for policy 1, policy_version 1736923 (0.0006) [2023-12-27 17:41:21,079][20347] Updated weights for policy 0, policy_version 1738354 (0.0010) [2023-12-27 17:41:21,137][20347] Updated weights for policy 0, policy_version 1738364 (0.0010) [2023-12-27 17:41:21,201][20347] Updated weights for policy 0, policy_version 1738374 (0.0008) [2023-12-27 17:41:21,244][20381] Updated weights for policy 1, policy_version 1736933 (0.0009) [2023-12-27 17:41:21,309][20381] Updated weights for policy 1, policy_version 1736943 (0.0010) [2023-12-27 17:41:21,379][20381] Updated weights for policy 1, policy_version 1736953 (0.0008) [2023-12-27 17:41:21,932][20347] Updated weights for policy 0, policy_version 1738384 (0.0010) [2023-12-27 17:41:21,989][20347] Updated weights for policy 0, policy_version 1738394 (0.0010) [2023-12-27 17:41:22,054][20347] Updated weights for policy 0, policy_version 1738404 (0.0010) [2023-12-27 17:41:22,174][20381] Updated weights for policy 1, policy_version 1736963 (0.0009) [2023-12-27 17:41:22,223][20381] Updated weights for policy 1, policy_version 1736973 (0.0010) [2023-12-27 17:41:22,281][20381] Updated weights for policy 1, policy_version 1736983 (0.0011) [2023-12-27 17:41:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 889831424. Throughput: 0: 10236.2, 1: 10288.2. Samples: 889799716. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:22,363][19170] Avg episode reward: [(0, '980.400'), (1, '1000.000')] [2023-12-27 17:41:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001736992_444719104.pth... [2023-12-27 17:41:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001738408_445112320.pth... [2023-12-27 17:41:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001737224_444809216.pth [2023-12-27 17:41:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001735808_444416000.pth [2023-12-27 17:41:22,692][20347] Updated weights for policy 0, policy_version 1738414 (0.0008) [2023-12-27 17:41:22,751][20347] Updated weights for policy 0, policy_version 1738424 (0.0006) [2023-12-27 17:41:22,807][20347] Updated weights for policy 0, policy_version 1738434 (0.0007) [2023-12-27 17:41:22,983][20381] Updated weights for policy 1, policy_version 1736993 (0.0010) [2023-12-27 17:41:23,043][20381] Updated weights for policy 1, policy_version 1737003 (0.0008) [2023-12-27 17:41:23,095][20381] Updated weights for policy 1, policy_version 1737013 (0.0006) [2023-12-27 17:41:23,151][20381] Updated weights for policy 1, policy_version 1737023 (0.0006) [2023-12-27 17:41:23,452][20347] Updated weights for policy 0, policy_version 1738444 (0.0007) [2023-12-27 17:41:23,508][20347] Updated weights for policy 0, policy_version 1738454 (0.0007) [2023-12-27 17:41:23,558][20347] Updated weights for policy 0, policy_version 1738464 (0.0007) [2023-12-27 17:41:23,842][20381] Updated weights for policy 1, policy_version 1737033 (0.0010) [2023-12-27 17:41:23,887][20381] Updated weights for policy 1, policy_version 1737043 (0.0010) [2023-12-27 17:41:23,932][20381] Updated weights for policy 1, policy_version 1737053 (0.0010) [2023-12-27 17:41:24,263][20347] Updated weights for policy 0, policy_version 1738474 (0.0007) [2023-12-27 17:41:24,320][20347] Updated weights for policy 0, policy_version 1738484 (0.0009) [2023-12-27 17:41:24,380][20347] Updated weights for policy 0, policy_version 1738494 (0.0007) [2023-12-27 17:41:24,445][20347] Updated weights for policy 0, policy_version 1738504 (0.0007) [2023-12-27 17:41:24,590][20381] Updated weights for policy 1, policy_version 1737063 (0.0011) [2023-12-27 17:41:24,648][20381] Updated weights for policy 1, policy_version 1737073 (0.0011) [2023-12-27 17:41:24,695][20381] Updated weights for policy 1, policy_version 1737083 (0.0011) [2023-12-27 17:41:25,059][20347] Updated weights for policy 0, policy_version 1738514 (0.0007) [2023-12-27 17:41:25,106][20347] Updated weights for policy 0, policy_version 1738524 (0.0007) [2023-12-27 17:41:25,158][20347] Updated weights for policy 0, policy_version 1738534 (0.0008) [2023-12-27 17:41:25,395][20381] Updated weights for policy 1, policy_version 1737093 (0.0010) [2023-12-27 17:41:25,442][20381] Updated weights for policy 1, policy_version 1737103 (0.0010) [2023-12-27 17:41:25,485][20381] Updated weights for policy 1, policy_version 1737113 (0.0006) [2023-12-27 17:41:25,772][20347] Updated weights for policy 0, policy_version 1738544 (0.0007) [2023-12-27 17:41:25,822][20347] Updated weights for policy 0, policy_version 1738554 (0.0007) [2023-12-27 17:41:25,874][20347] Updated weights for policy 0, policy_version 1738564 (0.0007) [2023-12-27 17:41:26,158][20381] Updated weights for policy 1, policy_version 1737123 (0.0006) [2023-12-27 17:41:26,215][20381] Updated weights for policy 1, policy_version 1737133 (0.0008) [2023-12-27 17:41:26,270][20381] Updated weights for policy 1, policy_version 1737143 (0.0008) [2023-12-27 17:41:26,578][20347] Updated weights for policy 0, policy_version 1738574 (0.0006) [2023-12-27 17:41:26,624][20347] Updated weights for policy 0, policy_version 1738584 (0.0005) [2023-12-27 17:41:26,674][20347] Updated weights for policy 0, policy_version 1738594 (0.0005) [2023-12-27 17:41:26,993][20381] Updated weights for policy 1, policy_version 1737153 (0.0007) [2023-12-27 17:41:27,045][20381] Updated weights for policy 1, policy_version 1737163 (0.0006) [2023-12-27 17:41:27,095][20381] Updated weights for policy 1, policy_version 1737173 (0.0007) [2023-12-27 17:41:27,151][20381] Updated weights for policy 1, policy_version 1737183 (0.0008) [2023-12-27 17:41:27,229][20347] Updated weights for policy 0, policy_version 1738604 (0.0006) [2023-12-27 17:41:27,281][20347] Updated weights for policy 0, policy_version 1738614 (0.0008) [2023-12-27 17:41:27,337][20347] Updated weights for policy 0, policy_version 1738624 (0.0007) [2023-12-27 17:41:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 889929728. Throughput: 0: 10195.2, 1: 10291.0. Samples: 889921640. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:27,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:41:27,783][20381] Updated weights for policy 1, policy_version 1737193 (0.0007) [2023-12-27 17:41:27,831][20381] Updated weights for policy 1, policy_version 1737203 (0.0009) [2023-12-27 17:41:27,889][20381] Updated weights for policy 1, policy_version 1737213 (0.0008) [2023-12-27 17:41:27,971][20347] Updated weights for policy 0, policy_version 1738634 (0.0008) [2023-12-27 17:41:28,022][20347] Updated weights for policy 0, policy_version 1738644 (0.0006) [2023-12-27 17:41:28,077][20347] Updated weights for policy 0, policy_version 1738654 (0.0008) [2023-12-27 17:41:28,133][20347] Updated weights for policy 0, policy_version 1738664 (0.0006) [2023-12-27 17:41:28,553][20381] Updated weights for policy 1, policy_version 1737223 (0.0008) [2023-12-27 17:41:28,610][20381] Updated weights for policy 1, policy_version 1737233 (0.0007) [2023-12-27 17:41:28,663][20381] Updated weights for policy 1, policy_version 1737243 (0.0008) [2023-12-27 17:41:28,820][20291] KL-divergence is very high: 222.3978 [2023-12-27 17:41:28,825][20347] Updated weights for policy 0, policy_version 1738674 (0.0007) [2023-12-27 17:41:28,862][20291] KL-divergence is very high: 289.6713 [2023-12-27 17:41:28,878][20347] Updated weights for policy 0, policy_version 1738684 (0.0009) [2023-12-27 17:41:28,904][20291] KL-divergence is very high: 209.8388 [2023-12-27 17:41:28,929][20347] Updated weights for policy 0, policy_version 1738694 (0.0008) [2023-12-27 17:41:29,324][20381] Updated weights for policy 1, policy_version 1737253 (0.0010) [2023-12-27 17:41:29,390][20381] Updated weights for policy 1, policy_version 1737263 (0.0008) [2023-12-27 17:41:29,455][20381] Updated weights for policy 1, policy_version 1737273 (0.0008) [2023-12-27 17:41:29,673][20347] Updated weights for policy 0, policy_version 1738704 (0.0007) [2023-12-27 17:41:29,733][20347] Updated weights for policy 0, policy_version 1738714 (0.0008) [2023-12-27 17:41:29,790][20347] Updated weights for policy 0, policy_version 1738724 (0.0008) [2023-12-27 17:41:30,162][20381] Updated weights for policy 1, policy_version 1737283 (0.0007) [2023-12-27 17:41:30,215][20381] Updated weights for policy 1, policy_version 1737293 (0.0005) [2023-12-27 17:41:30,267][20381] Updated weights for policy 1, policy_version 1737303 (0.0005) [2023-12-27 17:41:30,525][20347] Updated weights for policy 0, policy_version 1738734 (0.0008) [2023-12-27 17:41:30,548][20291] KL-divergence is very high: 105.7967 [2023-12-27 17:41:30,581][20347] Updated weights for policy 0, policy_version 1738744 (0.0008) [2023-12-27 17:41:30,593][20291] KL-divergence is very high: 130.9007 [2023-12-27 17:41:30,637][20347] Updated weights for policy 0, policy_version 1738754 (0.0007) [2023-12-27 17:41:30,839][20381] Updated weights for policy 1, policy_version 1737313 (0.0006) [2023-12-27 17:41:30,893][20381] Updated weights for policy 1, policy_version 1737323 (0.0005) [2023-12-27 17:41:30,943][20381] Updated weights for policy 1, policy_version 1737333 (0.0005) [2023-12-27 17:41:30,988][20381] Updated weights for policy 1, policy_version 1737343 (0.0007) [2023-12-27 17:41:31,305][20347] Updated weights for policy 0, policy_version 1738764 (0.0009) [2023-12-27 17:41:31,371][20347] Updated weights for policy 0, policy_version 1738774 (0.0008) [2023-12-27 17:41:31,435][20347] Updated weights for policy 0, policy_version 1738784 (0.0006) [2023-12-27 17:41:31,701][20381] Updated weights for policy 1, policy_version 1737353 (0.0007) [2023-12-27 17:41:31,770][20381] Updated weights for policy 1, policy_version 1737363 (0.0008) [2023-12-27 17:41:31,824][20381] Updated weights for policy 1, policy_version 1737373 (0.0005) [2023-12-27 17:41:32,088][20347] Updated weights for policy 0, policy_version 1738794 (0.0008) [2023-12-27 17:41:32,149][20347] Updated weights for policy 0, policy_version 1738804 (0.0007) [2023-12-27 17:41:32,204][20347] Updated weights for policy 0, policy_version 1738814 (0.0008) [2023-12-27 17:41:32,266][20347] Updated weights for policy 0, policy_version 1738824 (0.0007) [2023-12-27 17:41:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 890036224. Throughput: 0: 10160.7, 1: 10270.1. Samples: 890046576. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:32,363][19170] Avg episode reward: [(0, '960.880'), (1, '1000.000')] [2023-12-27 17:41:32,477][20381] Updated weights for policy 1, policy_version 1737383 (0.0007) [2023-12-27 17:41:32,530][20381] Updated weights for policy 1, policy_version 1737393 (0.0007) [2023-12-27 17:41:32,591][20381] Updated weights for policy 1, policy_version 1737403 (0.0008) [2023-12-27 17:41:32,931][20347] Updated weights for policy 0, policy_version 1738834 (0.0007) [2023-12-27 17:41:32,988][20347] Updated weights for policy 0, policy_version 1738844 (0.0007) [2023-12-27 17:41:33,047][20347] Updated weights for policy 0, policy_version 1738854 (0.0007) [2023-12-27 17:41:33,224][20381] Updated weights for policy 1, policy_version 1737413 (0.0007) [2023-12-27 17:41:33,268][20381] Updated weights for policy 1, policy_version 1737423 (0.0008) [2023-12-27 17:41:33,313][20381] Updated weights for policy 1, policy_version 1737433 (0.0010) [2023-12-27 17:41:33,641][20347] Updated weights for policy 0, policy_version 1738864 (0.0006) [2023-12-27 17:41:33,687][20347] Updated weights for policy 0, policy_version 1738874 (0.0005) [2023-12-27 17:41:33,742][20347] Updated weights for policy 0, policy_version 1738884 (0.0005) [2023-12-27 17:41:33,940][20381] Updated weights for policy 1, policy_version 1737443 (0.0010) [2023-12-27 17:41:33,992][20381] Updated weights for policy 1, policy_version 1737453 (0.0011) [2023-12-27 17:41:34,045][20381] Updated weights for policy 1, policy_version 1737463 (0.0011) [2023-12-27 17:41:34,369][20347] Updated weights for policy 0, policy_version 1738894 (0.0007) [2023-12-27 17:41:34,431][20347] Updated weights for policy 0, policy_version 1738904 (0.0007) [2023-12-27 17:41:34,492][20347] Updated weights for policy 0, policy_version 1738914 (0.0005) [2023-12-27 17:41:34,817][20381] Updated weights for policy 1, policy_version 1737473 (0.0010) [2023-12-27 17:41:34,862][20381] Updated weights for policy 1, policy_version 1737483 (0.0010) [2023-12-27 17:41:34,907][20381] Updated weights for policy 1, policy_version 1737493 (0.0010) [2023-12-27 17:41:34,969][20381] Updated weights for policy 1, policy_version 1737503 (0.0011) [2023-12-27 17:41:35,138][20347] Updated weights for policy 0, policy_version 1738924 (0.0006) [2023-12-27 17:41:35,195][20347] Updated weights for policy 0, policy_version 1738934 (0.0007) [2023-12-27 17:41:35,255][20347] Updated weights for policy 0, policy_version 1738944 (0.0010) [2023-12-27 17:41:35,675][20381] Updated weights for policy 1, policy_version 1737513 (0.0010) [2023-12-27 17:41:35,729][20381] Updated weights for policy 1, policy_version 1737523 (0.0010) [2023-12-27 17:41:35,778][20381] Updated weights for policy 1, policy_version 1737533 (0.0010) [2023-12-27 17:41:35,800][20347] Updated weights for policy 0, policy_version 1738954 (0.0007) [2023-12-27 17:41:35,861][20347] Updated weights for policy 0, policy_version 1738964 (0.0008) [2023-12-27 17:41:35,915][20347] Updated weights for policy 0, policy_version 1738974 (0.0008) [2023-12-27 17:41:35,968][20347] Updated weights for policy 0, policy_version 1738984 (0.0009) [2023-12-27 17:41:36,537][20381] Updated weights for policy 1, policy_version 1737543 (0.0009) [2023-12-27 17:41:36,590][20381] Updated weights for policy 1, policy_version 1737553 (0.0008) [2023-12-27 17:41:36,643][20381] Updated weights for policy 1, policy_version 1737563 (0.0009) [2023-12-27 17:41:36,676][20347] Updated weights for policy 0, policy_version 1738994 (0.0008) [2023-12-27 17:41:36,725][20347] Updated weights for policy 0, policy_version 1739004 (0.0009) [2023-12-27 17:41:36,776][20347] Updated weights for policy 0, policy_version 1739014 (0.0008) [2023-12-27 17:41:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 890134528. Throughput: 0: 10197.4, 1: 10246.0. Samples: 890109564. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:37,363][19170] Avg episode reward: [(0, '970.680'), (1, '1000.000')] [2023-12-27 17:41:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001739016_445267968.pth... [2023-12-27 17:41:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001737832_444964864.pth [2023-12-27 17:41:37,388][20381] Updated weights for policy 1, policy_version 1737574 (0.0011) [2023-12-27 17:41:37,434][20381] Updated weights for policy 1, policy_version 1737584 (0.0010) [2023-12-27 17:41:37,484][20381] Updated weights for policy 1, policy_version 1737594 (0.0010) [2023-12-27 17:41:37,501][20347] Updated weights for policy 0, policy_version 1739024 (0.0007) [2023-12-27 17:41:37,513][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001737600_444874752.pth... [2023-12-27 17:41:37,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001736384_444563456.pth [2023-12-27 17:41:37,553][20347] Updated weights for policy 0, policy_version 1739034 (0.0007) [2023-12-27 17:41:37,613][20347] Updated weights for policy 0, policy_version 1739044 (0.0008) [2023-12-27 17:41:38,172][20381] Updated weights for policy 1, policy_version 1737604 (0.0008) [2023-12-27 17:41:38,221][20381] Updated weights for policy 1, policy_version 1737614 (0.0008) [2023-12-27 17:41:38,267][20381] Updated weights for policy 1, policy_version 1737624 (0.0007) [2023-12-27 17:41:38,341][20347] Updated weights for policy 0, policy_version 1739054 (0.0008) [2023-12-27 17:41:38,395][20347] Updated weights for policy 0, policy_version 1739064 (0.0006) [2023-12-27 17:41:38,456][20347] Updated weights for policy 0, policy_version 1739074 (0.0007) [2023-12-27 17:41:39,009][20381] Updated weights for policy 1, policy_version 1737634 (0.0007) [2023-12-27 17:41:39,068][20381] Updated weights for policy 1, policy_version 1737644 (0.0010) [2023-12-27 17:41:39,125][20381] Updated weights for policy 1, policy_version 1737654 (0.0010) [2023-12-27 17:41:39,131][20347] Updated weights for policy 0, policy_version 1739084 (0.0006) [2023-12-27 17:41:39,189][20381] Updated weights for policy 1, policy_version 1737664 (0.0010) [2023-12-27 17:41:39,191][20347] Updated weights for policy 0, policy_version 1739094 (0.0005) [2023-12-27 17:41:39,254][20347] Updated weights for policy 0, policy_version 1739104 (0.0008) [2023-12-27 17:41:39,876][20381] Updated weights for policy 1, policy_version 1737674 (0.0009) [2023-12-27 17:41:39,930][20381] Updated weights for policy 1, policy_version 1737684 (0.0008) [2023-12-27 17:41:39,957][20347] Updated weights for policy 0, policy_version 1739114 (0.0007) [2023-12-27 17:41:39,990][20381] Updated weights for policy 1, policy_version 1737694 (0.0008) [2023-12-27 17:41:40,016][20347] Updated weights for policy 0, policy_version 1739124 (0.0007) [2023-12-27 17:41:40,075][20347] Updated weights for policy 0, policy_version 1739134 (0.0008) [2023-12-27 17:41:40,135][20347] Updated weights for policy 0, policy_version 1739144 (0.0007) [2023-12-27 17:41:40,688][20381] Updated weights for policy 1, policy_version 1737704 (0.0010) [2023-12-27 17:41:40,743][20381] Updated weights for policy 1, policy_version 1737714 (0.0010) [2023-12-27 17:41:40,800][20381] Updated weights for policy 1, policy_version 1737724 (0.0011) [2023-12-27 17:41:40,854][20347] Updated weights for policy 0, policy_version 1739154 (0.0008) [2023-12-27 17:41:40,910][20347] Updated weights for policy 0, policy_version 1739164 (0.0007) [2023-12-27 17:41:40,967][20347] Updated weights for policy 0, policy_version 1739174 (0.0007) [2023-12-27 17:41:41,553][20381] Updated weights for policy 1, policy_version 1737734 (0.0009) [2023-12-27 17:41:41,632][20381] Updated weights for policy 1, policy_version 1737746 (0.0007) [2023-12-27 17:41:41,685][20381] Updated weights for policy 1, policy_version 1737756 (0.0008) [2023-12-27 17:41:41,742][20347] Updated weights for policy 0, policy_version 1739184 (0.0008) [2023-12-27 17:41:41,792][20347] Updated weights for policy 0, policy_version 1739194 (0.0008) [2023-12-27 17:41:41,850][20347] Updated weights for policy 0, policy_version 1739204 (0.0007) [2023-12-27 17:41:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 890232832. Throughput: 0: 10149.1, 1: 10194.5. Samples: 890229332. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:42,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:41:42,449][20381] Updated weights for policy 1, policy_version 1737766 (0.0006) [2023-12-27 17:41:42,506][20381] Updated weights for policy 1, policy_version 1737776 (0.0007) [2023-12-27 17:41:42,516][20347] Updated weights for policy 0, policy_version 1739214 (0.0007) [2023-12-27 17:41:42,569][20347] Updated weights for policy 0, policy_version 1739224 (0.0007) [2023-12-27 17:41:42,570][20381] Updated weights for policy 1, policy_version 1737786 (0.0008) [2023-12-27 17:41:42,625][20347] Updated weights for policy 0, policy_version 1739234 (0.0007) [2023-12-27 17:41:43,265][20347] Updated weights for policy 0, policy_version 1739244 (0.0007) [2023-12-27 17:41:43,275][20381] Updated weights for policy 1, policy_version 1737796 (0.0007) [2023-12-27 17:41:43,310][20347] Updated weights for policy 0, policy_version 1739254 (0.0005) [2023-12-27 17:41:43,323][20381] Updated weights for policy 1, policy_version 1737806 (0.0007) [2023-12-27 17:41:43,363][20347] Updated weights for policy 0, policy_version 1739264 (0.0006) [2023-12-27 17:41:43,369][20381] Updated weights for policy 1, policy_version 1737816 (0.0006) [2023-12-27 17:41:43,972][20381] Updated weights for policy 1, policy_version 1737826 (0.0007) [2023-12-27 17:41:44,020][20347] Updated weights for policy 0, policy_version 1739274 (0.0006) [2023-12-27 17:41:44,024][20381] Updated weights for policy 1, policy_version 1737836 (0.0006) [2023-12-27 17:41:44,074][20347] Updated weights for policy 0, policy_version 1739284 (0.0005) [2023-12-27 17:41:44,075][20381] Updated weights for policy 1, policy_version 1737846 (0.0005) [2023-12-27 17:41:44,127][20381] Updated weights for policy 1, policy_version 1737856 (0.0006) [2023-12-27 17:41:44,131][20347] Updated weights for policy 0, policy_version 1739294 (0.0008) [2023-12-27 17:41:44,186][20347] Updated weights for policy 0, policy_version 1739304 (0.0006) [2023-12-27 17:41:44,783][20347] Updated weights for policy 0, policy_version 1739314 (0.0009) [2023-12-27 17:41:44,784][20381] Updated weights for policy 1, policy_version 1737866 (0.0007) [2023-12-27 17:41:44,840][20381] Updated weights for policy 1, policy_version 1737876 (0.0006) [2023-12-27 17:41:44,846][20347] Updated weights for policy 0, policy_version 1739324 (0.0009) [2023-12-27 17:41:44,901][20381] Updated weights for policy 1, policy_version 1737886 (0.0006) [2023-12-27 17:41:44,910][20347] Updated weights for policy 0, policy_version 1739334 (0.0010) [2023-12-27 17:41:45,580][20381] Updated weights for policy 1, policy_version 1737896 (0.0006) [2023-12-27 17:41:45,629][20347] Updated weights for policy 0, policy_version 1739344 (0.0007) [2023-12-27 17:41:45,641][20381] Updated weights for policy 1, policy_version 1737906 (0.0006) [2023-12-27 17:41:45,686][20347] Updated weights for policy 0, policy_version 1739354 (0.0007) [2023-12-27 17:41:45,700][20381] Updated weights for policy 1, policy_version 1737916 (0.0005) [2023-12-27 17:41:45,746][20347] Updated weights for policy 0, policy_version 1739364 (0.0006) [2023-12-27 17:41:46,272][20381] Updated weights for policy 1, policy_version 1737926 (0.0006) [2023-12-27 17:41:46,328][20381] Updated weights for policy 1, policy_version 1737936 (0.0005) [2023-12-27 17:41:46,337][20347] Updated weights for policy 0, policy_version 1739374 (0.0006) [2023-12-27 17:41:46,383][20381] Updated weights for policy 1, policy_version 1737946 (0.0005) [2023-12-27 17:41:46,393][20347] Updated weights for policy 0, policy_version 1739384 (0.0007) [2023-12-27 17:41:46,447][20347] Updated weights for policy 0, policy_version 1739394 (0.0005) [2023-12-27 17:41:46,951][20381] Updated weights for policy 1, policy_version 1737956 (0.0007) [2023-12-27 17:41:47,006][20381] Updated weights for policy 1, policy_version 1737966 (0.0007) [2023-12-27 17:41:47,044][20347] Updated weights for policy 0, policy_version 1739404 (0.0006) [2023-12-27 17:41:47,061][20381] Updated weights for policy 1, policy_version 1737976 (0.0006) [2023-12-27 17:41:47,096][20347] Updated weights for policy 0, policy_version 1739414 (0.0006) [2023-12-27 17:41:47,152][20347] Updated weights for policy 0, policy_version 1739424 (0.0008) [2023-12-27 17:41:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 890347520. Throughput: 0: 10202.6, 1: 10284.1. Samples: 890355736. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:47,363][19170] Avg episode reward: [(0, '970.690'), (1, '1000.000')] [2023-12-27 17:41:47,768][20381] Updated weights for policy 1, policy_version 1737986 (0.0006) [2023-12-27 17:41:47,793][20347] Updated weights for policy 0, policy_version 1739434 (0.0008) [2023-12-27 17:41:47,832][20381] Updated weights for policy 1, policy_version 1737996 (0.0007) [2023-12-27 17:41:47,844][20347] Updated weights for policy 0, policy_version 1739444 (0.0007) [2023-12-27 17:41:47,891][20381] Updated weights for policy 1, policy_version 1738006 (0.0006) [2023-12-27 17:41:47,894][20347] Updated weights for policy 0, policy_version 1739454 (0.0005) [2023-12-27 17:41:47,950][20347] Updated weights for policy 0, policy_version 1739464 (0.0005) [2023-12-27 17:41:47,952][20381] Updated weights for policy 1, policy_version 1738016 (0.0006) [2023-12-27 17:41:48,581][20381] Updated weights for policy 1, policy_version 1738026 (0.0011) [2023-12-27 17:41:48,637][20381] Updated weights for policy 1, policy_version 1738036 (0.0011) [2023-12-27 17:41:48,697][20381] Updated weights for policy 1, policy_version 1738046 (0.0011) [2023-12-27 17:41:48,705][20347] Updated weights for policy 0, policy_version 1739474 (0.0006) [2023-12-27 17:41:48,761][20347] Updated weights for policy 0, policy_version 1739484 (0.0008) [2023-12-27 17:41:48,817][20347] Updated weights for policy 0, policy_version 1739494 (0.0007) [2023-12-27 17:41:49,410][20381] Updated weights for policy 1, policy_version 1738056 (0.0008) [2023-12-27 17:41:49,460][20381] Updated weights for policy 1, policy_version 1738066 (0.0006) [2023-12-27 17:41:49,520][20381] Updated weights for policy 1, policy_version 1738076 (0.0006) [2023-12-27 17:41:49,530][20347] Updated weights for policy 0, policy_version 1739504 (0.0007) [2023-12-27 17:41:49,591][20347] Updated weights for policy 0, policy_version 1739514 (0.0007) [2023-12-27 17:41:49,649][20347] Updated weights for policy 0, policy_version 1739524 (0.0007) [2023-12-27 17:41:50,178][20381] Updated weights for policy 1, policy_version 1738086 (0.0006) [2023-12-27 17:41:50,234][20381] Updated weights for policy 1, policy_version 1738096 (0.0006) [2023-12-27 17:41:50,283][20381] Updated weights for policy 1, policy_version 1738106 (0.0007) [2023-12-27 17:41:50,378][20347] Updated weights for policy 0, policy_version 1739534 (0.0008) [2023-12-27 17:41:50,435][20347] Updated weights for policy 0, policy_version 1739544 (0.0008) [2023-12-27 17:41:50,494][20347] Updated weights for policy 0, policy_version 1739554 (0.0008) [2023-12-27 17:41:50,962][20381] Updated weights for policy 1, policy_version 1738116 (0.0008) [2023-12-27 17:41:51,027][20381] Updated weights for policy 1, policy_version 1738126 (0.0008) [2023-12-27 17:41:51,096][20381] Updated weights for policy 1, policy_version 1738136 (0.0006) [2023-12-27 17:41:51,223][20347] Updated weights for policy 0, policy_version 1739564 (0.0008) [2023-12-27 17:41:51,282][20347] Updated weights for policy 0, policy_version 1739574 (0.0007) [2023-12-27 17:41:51,336][20347] Updated weights for policy 0, policy_version 1739584 (0.0008) [2023-12-27 17:41:51,786][20381] Updated weights for policy 1, policy_version 1738146 (0.0007) [2023-12-27 17:41:51,839][20381] Updated weights for policy 1, policy_version 1738156 (0.0008) [2023-12-27 17:41:51,895][20381] Updated weights for policy 1, policy_version 1738166 (0.0008) [2023-12-27 17:41:51,951][20381] Updated weights for policy 1, policy_version 1738176 (0.0007) [2023-12-27 17:41:52,089][20347] Updated weights for policy 0, policy_version 1739594 (0.0008) [2023-12-27 17:41:52,153][20347] Updated weights for policy 0, policy_version 1739604 (0.0008) [2023-12-27 17:41:52,211][20347] Updated weights for policy 0, policy_version 1739614 (0.0008) [2023-12-27 17:41:52,272][20347] Updated weights for policy 0, policy_version 1739624 (0.0007) [2023-12-27 17:41:52,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 890445824. Throughput: 0: 10211.5, 1: 10266.1. Samples: 890415540. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:52,363][19170] Avg episode reward: [(0, '980.460'), (1, '1000.000')] [2023-12-27 17:41:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001738176_445022208.pth... [2023-12-27 17:41:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001739624_445423616.pth... [2023-12-27 17:41:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001738408_445112320.pth [2023-12-27 17:41:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001736992_444719104.pth [2023-12-27 17:41:52,645][20381] Updated weights for policy 1, policy_version 1738186 (0.0008) [2023-12-27 17:41:52,692][20381] Updated weights for policy 1, policy_version 1738196 (0.0008) [2023-12-27 17:41:52,738][20381] Updated weights for policy 1, policy_version 1738206 (0.0007) [2023-12-27 17:41:52,946][20347] Updated weights for policy 0, policy_version 1739634 (0.0005) [2023-12-27 17:41:53,000][20347] Updated weights for policy 0, policy_version 1739644 (0.0008) [2023-12-27 17:41:53,053][20347] Updated weights for policy 0, policy_version 1739654 (0.0006) [2023-12-27 17:41:53,464][20381] Updated weights for policy 1, policy_version 1738216 (0.0007) [2023-12-27 17:41:53,525][20381] Updated weights for policy 1, policy_version 1738226 (0.0008) [2023-12-27 17:41:53,582][20381] Updated weights for policy 1, policy_version 1738236 (0.0007) [2023-12-27 17:41:53,606][20347] Updated weights for policy 0, policy_version 1739664 (0.0006) [2023-12-27 17:41:53,665][20347] Updated weights for policy 0, policy_version 1739674 (0.0006) [2023-12-27 17:41:53,722][20347] Updated weights for policy 0, policy_version 1739684 (0.0007) [2023-12-27 17:41:54,260][20381] Updated weights for policy 1, policy_version 1738246 (0.0005) [2023-12-27 17:41:54,322][20381] Updated weights for policy 1, policy_version 1738256 (0.0006) [2023-12-27 17:41:54,345][20347] Updated weights for policy 0, policy_version 1739694 (0.0008) [2023-12-27 17:41:54,378][20381] Updated weights for policy 1, policy_version 1738266 (0.0006) [2023-12-27 17:41:54,416][20347] Updated weights for policy 0, policy_version 1739704 (0.0008) [2023-12-27 17:41:54,479][20347] Updated weights for policy 0, policy_version 1739714 (0.0008) [2023-12-27 17:41:55,016][20381] Updated weights for policy 1, policy_version 1738276 (0.0008) [2023-12-27 17:41:55,066][20381] Updated weights for policy 1, policy_version 1738286 (0.0008) [2023-12-27 17:41:55,101][20347] Updated weights for policy 0, policy_version 1739724 (0.0008) [2023-12-27 17:41:55,118][20381] Updated weights for policy 1, policy_version 1738296 (0.0007) [2023-12-27 17:41:55,147][20347] Updated weights for policy 0, policy_version 1739734 (0.0010) [2023-12-27 17:41:55,198][20347] Updated weights for policy 0, policy_version 1739744 (0.0005) [2023-12-27 17:41:55,828][20381] Updated weights for policy 1, policy_version 1738306 (0.0005) [2023-12-27 17:41:55,862][20347] Updated weights for policy 0, policy_version 1739754 (0.0007) [2023-12-27 17:41:55,891][20381] Updated weights for policy 1, policy_version 1738316 (0.0006) [2023-12-27 17:41:55,920][20347] Updated weights for policy 0, policy_version 1739764 (0.0010) [2023-12-27 17:41:55,950][20381] Updated weights for policy 1, policy_version 1738326 (0.0005) [2023-12-27 17:41:55,984][20347] Updated weights for policy 0, policy_version 1739774 (0.0010) [2023-12-27 17:41:56,005][20381] Updated weights for policy 1, policy_version 1738336 (0.0005) [2023-12-27 17:41:56,040][20347] Updated weights for policy 0, policy_version 1739784 (0.0010) [2023-12-27 17:41:56,581][20381] Updated weights for policy 1, policy_version 1738346 (0.0005) [2023-12-27 17:41:56,622][20347] Updated weights for policy 0, policy_version 1739794 (0.0007) [2023-12-27 17:41:56,627][20381] Updated weights for policy 1, policy_version 1738356 (0.0005) [2023-12-27 17:41:56,669][20347] Updated weights for policy 0, policy_version 1739804 (0.0010) [2023-12-27 17:41:56,671][20381] Updated weights for policy 1, policy_version 1738366 (0.0005) [2023-12-27 17:41:56,720][20347] Updated weights for policy 0, policy_version 1739814 (0.0010) [2023-12-27 17:41:57,253][20381] Updated weights for policy 1, policy_version 1738376 (0.0007) [2023-12-27 17:41:57,303][20381] Updated weights for policy 1, policy_version 1738386 (0.0007) [2023-12-27 17:41:57,354][20381] Updated weights for policy 1, policy_version 1738396 (0.0008) [2023-12-27 17:41:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 890544128. Throughput: 0: 10286.9, 1: 10206.4. Samples: 890538584. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:41:57,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:41:57,389][20347] Updated weights for policy 0, policy_version 1739824 (0.0009) [2023-12-27 17:41:57,441][20347] Updated weights for policy 0, policy_version 1739834 (0.0010) [2023-12-27 17:41:57,488][20347] Updated weights for policy 0, policy_version 1739844 (0.0010) [2023-12-27 17:41:58,020][20381] Updated weights for policy 1, policy_version 1738406 (0.0007) [2023-12-27 17:41:58,071][20381] Updated weights for policy 1, policy_version 1738416 (0.0007) [2023-12-27 17:41:58,115][20381] Updated weights for policy 1, policy_version 1738426 (0.0007) [2023-12-27 17:41:58,166][20347] Updated weights for policy 0, policy_version 1739854 (0.0010) [2023-12-27 17:41:58,225][20347] Updated weights for policy 0, policy_version 1739864 (0.0009) [2023-12-27 17:41:58,284][20347] Updated weights for policy 0, policy_version 1739874 (0.0010) [2023-12-27 17:41:58,897][20381] Updated weights for policy 1, policy_version 1738436 (0.0008) [2023-12-27 17:41:58,963][20381] Updated weights for policy 1, policy_version 1738446 (0.0009) [2023-12-27 17:41:59,022][20381] Updated weights for policy 1, policy_version 1738456 (0.0011) [2023-12-27 17:41:59,062][20347] Updated weights for policy 0, policy_version 1739884 (0.0011) [2023-12-27 17:41:59,124][20347] Updated weights for policy 0, policy_version 1739894 (0.0010) [2023-12-27 17:41:59,183][20347] Updated weights for policy 0, policy_version 1739904 (0.0009) [2023-12-27 17:41:59,712][20381] Updated weights for policy 1, policy_version 1738466 (0.0011) [2023-12-27 17:41:59,765][20381] Updated weights for policy 1, policy_version 1738476 (0.0011) [2023-12-27 17:41:59,827][20381] Updated weights for policy 1, policy_version 1738486 (0.0011) [2023-12-27 17:41:59,892][20381] Updated weights for policy 1, policy_version 1738496 (0.0011) [2023-12-27 17:41:59,897][20347] Updated weights for policy 0, policy_version 1739914 (0.0010) [2023-12-27 17:41:59,969][20347] Updated weights for policy 0, policy_version 1739924 (0.0011) [2023-12-27 17:42:00,024][20347] Updated weights for policy 0, policy_version 1739934 (0.0008) [2023-12-27 17:42:00,069][20347] Updated weights for policy 0, policy_version 1739944 (0.0007) [2023-12-27 17:42:00,610][20381] Updated weights for policy 1, policy_version 1738506 (0.0010) [2023-12-27 17:42:00,658][20381] Updated weights for policy 1, policy_version 1738516 (0.0010) [2023-12-27 17:42:00,721][20381] Updated weights for policy 1, policy_version 1738526 (0.0010) [2023-12-27 17:42:00,791][20347] Updated weights for policy 0, policy_version 1739954 (0.0008) [2023-12-27 17:42:00,843][20347] Updated weights for policy 0, policy_version 1739964 (0.0007) [2023-12-27 17:42:00,891][20347] Updated weights for policy 0, policy_version 1739974 (0.0007) [2023-12-27 17:42:01,415][20381] Updated weights for policy 1, policy_version 1738536 (0.0010) [2023-12-27 17:42:01,471][20381] Updated weights for policy 1, policy_version 1738546 (0.0010) [2023-12-27 17:42:01,525][20381] Updated weights for policy 1, policy_version 1738556 (0.0011) [2023-12-27 17:42:01,656][20347] Updated weights for policy 0, policy_version 1739984 (0.0008) [2023-12-27 17:42:01,716][20347] Updated weights for policy 0, policy_version 1739994 (0.0007) [2023-12-27 17:42:01,778][20347] Updated weights for policy 0, policy_version 1740004 (0.0008) [2023-12-27 17:42:02,217][20381] Updated weights for policy 1, policy_version 1738566 (0.0009) [2023-12-27 17:42:02,278][20381] Updated weights for policy 1, policy_version 1738576 (0.0007) [2023-12-27 17:42:02,342][20381] Updated weights for policy 1, policy_version 1738586 (0.0008) [2023-12-27 17:42:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 890642432. Throughput: 0: 10268.8, 1: 10188.1. Samples: 890659572. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:02,363][19170] Avg episode reward: [(0, '980.400'), (1, '1000.000')] [2023-12-27 17:42:02,463][20347] Updated weights for policy 0, policy_version 1740014 (0.0006) [2023-12-27 17:42:02,529][20347] Updated weights for policy 0, policy_version 1740024 (0.0005) [2023-12-27 17:42:02,590][20347] Updated weights for policy 0, policy_version 1740034 (0.0005) [2023-12-27 17:42:03,051][20381] Updated weights for policy 1, policy_version 1738596 (0.0008) [2023-12-27 17:42:03,100][20381] Updated weights for policy 1, policy_version 1738606 (0.0008) [2023-12-27 17:42:03,141][20347] Updated weights for policy 0, policy_version 1740044 (0.0006) [2023-12-27 17:42:03,157][20381] Updated weights for policy 1, policy_version 1738616 (0.0005) [2023-12-27 17:42:03,185][20347] Updated weights for policy 0, policy_version 1740054 (0.0006) [2023-12-27 17:42:03,232][20347] Updated weights for policy 0, policy_version 1740064 (0.0009) [2023-12-27 17:42:03,793][20381] Updated weights for policy 1, policy_version 1738626 (0.0007) [2023-12-27 17:42:03,854][20381] Updated weights for policy 1, policy_version 1738636 (0.0007) [2023-12-27 17:42:03,920][20381] Updated weights for policy 1, policy_version 1738646 (0.0006) [2023-12-27 17:42:03,922][20347] Updated weights for policy 0, policy_version 1740074 (0.0008) [2023-12-27 17:42:03,984][20381] Updated weights for policy 1, policy_version 1738656 (0.0008) [2023-12-27 17:42:03,986][20347] Updated weights for policy 0, policy_version 1740084 (0.0008) [2023-12-27 17:42:04,043][20347] Updated weights for policy 0, policy_version 1740094 (0.0009) [2023-12-27 17:42:04,092][20347] Updated weights for policy 0, policy_version 1740104 (0.0010) [2023-12-27 17:42:04,665][20381] Updated weights for policy 1, policy_version 1738666 (0.0007) [2023-12-27 17:42:04,719][20381] Updated weights for policy 1, policy_version 1738676 (0.0008) [2023-12-27 17:42:04,783][20381] Updated weights for policy 1, policy_version 1738686 (0.0006) [2023-12-27 17:42:04,807][20347] Updated weights for policy 0, policy_version 1740114 (0.0010) [2023-12-27 17:42:04,868][20347] Updated weights for policy 0, policy_version 1740124 (0.0010) [2023-12-27 17:42:04,928][20347] Updated weights for policy 0, policy_version 1740134 (0.0010) [2023-12-27 17:42:05,460][20381] Updated weights for policy 1, policy_version 1738696 (0.0008) [2023-12-27 17:42:05,505][20381] Updated weights for policy 1, policy_version 1738706 (0.0005) [2023-12-27 17:42:05,550][20381] Updated weights for policy 1, policy_version 1738716 (0.0009) [2023-12-27 17:42:05,551][20347] Updated weights for policy 0, policy_version 1740144 (0.0010) [2023-12-27 17:42:05,602][20347] Updated weights for policy 0, policy_version 1740154 (0.0010) [2023-12-27 17:42:05,632][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-27 17:42:06,186][20381] Updated weights for policy 1, policy_version 1738726 (0.0007) [2023-12-27 17:42:06,247][20347] Updated weights for policy 0, policy_version 1740164 (0.0009) [2023-12-27 17:42:06,249][20381] Updated weights for policy 1, policy_version 1738736 (0.0007) [2023-12-27 17:42:06,304][20347] Updated weights for policy 0, policy_version 1740174 (0.0009) [2023-12-27 17:42:06,308][20381] Updated weights for policy 1, policy_version 1738746 (0.0011) [2023-12-27 17:42:06,361][20347] Updated weights for policy 0, policy_version 1740184 (0.0011) [2023-12-27 17:42:06,958][20381] Updated weights for policy 1, policy_version 1738756 (0.0010) [2023-12-27 17:42:07,013][20381] Updated weights for policy 1, policy_version 1738766 (0.0008) [2023-12-27 17:42:07,071][20381] Updated weights for policy 1, policy_version 1738776 (0.0009) [2023-12-27 17:42:07,076][20347] Updated weights for policy 0, policy_version 1740194 (0.0010) [2023-12-27 17:42:07,129][20347] Updated weights for policy 0, policy_version 1740204 (0.0007) [2023-12-27 17:42:07,189][20347] Updated weights for policy 0, policy_version 1740214 (0.0008) [2023-12-27 17:42:07,253][20347] Updated weights for policy 0, policy_version 1740224 (0.0008) [2023-12-27 17:42:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 890757120. Throughput: 0: 10279.8, 1: 10197.8. Samples: 890721204. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:07,363][19170] Avg episode reward: [(0, '990.200'), (1, '1000.000')] [2023-12-27 17:42:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001740224_445579264.pth... [2023-12-27 17:42:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001738784_445177856.pth... [2023-12-27 17:42:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001737600_444874752.pth [2023-12-27 17:42:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001739016_445267968.pth [2023-12-27 17:42:07,699][20381] Updated weights for policy 1, policy_version 1738786 (0.0006) [2023-12-27 17:42:07,764][20381] Updated weights for policy 1, policy_version 1738796 (0.0010) [2023-12-27 17:42:07,815][20381] Updated weights for policy 1, policy_version 1738806 (0.0010) [2023-12-27 17:42:07,876][20381] Updated weights for policy 1, policy_version 1738816 (0.0011) [2023-12-27 17:42:07,902][20347] Updated weights for policy 0, policy_version 1740234 (0.0009) [2023-12-27 17:42:07,960][20347] Updated weights for policy 0, policy_version 1740244 (0.0005) [2023-12-27 17:42:08,024][20347] Updated weights for policy 0, policy_version 1740254 (0.0007) [2023-12-27 17:42:08,545][20381] Updated weights for policy 1, policy_version 1738826 (0.0006) [2023-12-27 17:42:08,607][20381] Updated weights for policy 1, policy_version 1738836 (0.0009) [2023-12-27 17:42:08,630][20347] Updated weights for policy 0, policy_version 1740264 (0.0008) [2023-12-27 17:42:08,666][20381] Updated weights for policy 1, policy_version 1738846 (0.0011) [2023-12-27 17:42:08,679][20347] Updated weights for policy 0, policy_version 1740274 (0.0010) [2023-12-27 17:42:08,741][20347] Updated weights for policy 0, policy_version 1740284 (0.0010) [2023-12-27 17:42:09,283][20381] Updated weights for policy 1, policy_version 1738856 (0.0009) [2023-12-27 17:42:09,344][20381] Updated weights for policy 1, policy_version 1738866 (0.0011) [2023-12-27 17:42:09,412][20347] Updated weights for policy 0, policy_version 1740294 (0.0010) [2023-12-27 17:42:09,412][20381] Updated weights for policy 1, policy_version 1738876 (0.0010) [2023-12-27 17:42:09,477][20347] Updated weights for policy 0, policy_version 1740304 (0.0010) [2023-12-27 17:42:09,533][20347] Updated weights for policy 0, policy_version 1740314 (0.0009) [2023-12-27 17:42:10,046][20381] Updated weights for policy 1, policy_version 1738886 (0.0009) [2023-12-27 17:42:10,101][20381] Updated weights for policy 1, policy_version 1738896 (0.0008) [2023-12-27 17:42:10,164][20381] Updated weights for policy 1, policy_version 1738906 (0.0007) [2023-12-27 17:42:10,252][20347] Updated weights for policy 0, policy_version 1740324 (0.0008) [2023-12-27 17:42:10,312][20347] Updated weights for policy 0, policy_version 1740334 (0.0011) [2023-12-27 17:42:10,374][20347] Updated weights for policy 0, policy_version 1740344 (0.0010) [2023-12-27 17:42:10,828][20381] Updated weights for policy 1, policy_version 1738916 (0.0008) [2023-12-27 17:42:10,889][20381] Updated weights for policy 1, policy_version 1738926 (0.0007) [2023-12-27 17:42:10,944][20381] Updated weights for policy 1, policy_version 1738936 (0.0006) [2023-12-27 17:42:11,077][20347] Updated weights for policy 0, policy_version 1740354 (0.0010) [2023-12-27 17:42:11,138][20347] Updated weights for policy 0, policy_version 1740364 (0.0008) [2023-12-27 17:42:11,200][20347] Updated weights for policy 0, policy_version 1740374 (0.0008) [2023-12-27 17:42:11,262][20347] Updated weights for policy 0, policy_version 1740384 (0.0008) [2023-12-27 17:42:11,717][20381] Updated weights for policy 1, policy_version 1738946 (0.0008) [2023-12-27 17:42:11,784][20381] Updated weights for policy 1, policy_version 1738956 (0.0008) [2023-12-27 17:42:11,841][20381] Updated weights for policy 1, policy_version 1738966 (0.0008) [2023-12-27 17:42:11,895][20381] Updated weights for policy 1, policy_version 1738976 (0.0008) [2023-12-27 17:42:11,987][20347] Updated weights for policy 0, policy_version 1740394 (0.0011) [2023-12-27 17:42:12,049][20347] Updated weights for policy 0, policy_version 1740404 (0.0010) [2023-12-27 17:42:12,105][20347] Updated weights for policy 0, policy_version 1740414 (0.0010) [2023-12-27 17:42:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 890855424. Throughput: 0: 10301.2, 1: 10257.7. Samples: 890846792. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:12,363][19170] Avg episode reward: [(0, '980.490'), (1, '1000.000')] [2023-12-27 17:42:12,599][20381] Updated weights for policy 1, policy_version 1738986 (0.0008) [2023-12-27 17:42:12,657][20381] Updated weights for policy 1, policy_version 1738996 (0.0008) [2023-12-27 17:42:12,719][20381] Updated weights for policy 1, policy_version 1739006 (0.0007) [2023-12-27 17:42:12,815][20347] Updated weights for policy 0, policy_version 1740424 (0.0007) [2023-12-27 17:42:12,875][20347] Updated weights for policy 0, policy_version 1740434 (0.0005) [2023-12-27 17:42:12,929][20347] Updated weights for policy 0, policy_version 1740444 (0.0005) [2023-12-27 17:42:13,441][20381] Updated weights for policy 1, policy_version 1739016 (0.0006) [2023-12-27 17:42:13,486][20347] Updated weights for policy 0, policy_version 1740454 (0.0005) [2023-12-27 17:42:13,490][20381] Updated weights for policy 1, policy_version 1739026 (0.0005) [2023-12-27 17:42:13,539][20347] Updated weights for policy 0, policy_version 1740464 (0.0007) [2023-12-27 17:42:13,543][20381] Updated weights for policy 1, policy_version 1739036 (0.0005) [2023-12-27 17:42:13,594][20347] Updated weights for policy 0, policy_version 1740474 (0.0008) [2023-12-27 17:42:14,157][20381] Updated weights for policy 1, policy_version 1739046 (0.0006) [2023-12-27 17:42:14,180][20347] Updated weights for policy 0, policy_version 1740484 (0.0007) [2023-12-27 17:42:14,216][20381] Updated weights for policy 1, policy_version 1739056 (0.0009) [2023-12-27 17:42:14,232][20347] Updated weights for policy 0, policy_version 1740494 (0.0006) [2023-12-27 17:42:14,274][20381] Updated weights for policy 1, policy_version 1739066 (0.0010) [2023-12-27 17:42:14,287][20347] Updated weights for policy 0, policy_version 1740504 (0.0006) [2023-12-27 17:42:14,885][20381] Updated weights for policy 1, policy_version 1739076 (0.0011) [2023-12-27 17:42:14,916][20347] Updated weights for policy 0, policy_version 1740514 (0.0006) [2023-12-27 17:42:14,949][20381] Updated weights for policy 1, policy_version 1739086 (0.0011) [2023-12-27 17:42:14,974][20347] Updated weights for policy 0, policy_version 1740524 (0.0006) [2023-12-27 17:42:15,010][20381] Updated weights for policy 1, policy_version 1739096 (0.0010) [2023-12-27 17:42:15,032][20347] Updated weights for policy 0, policy_version 1740534 (0.0007) [2023-12-27 17:42:15,096][20347] Updated weights for policy 0, policy_version 1740544 (0.0007) [2023-12-27 17:42:15,650][20381] Updated weights for policy 1, policy_version 1739106 (0.0008) [2023-12-27 17:42:15,713][20381] Updated weights for policy 1, policy_version 1739116 (0.0008) [2023-12-27 17:42:15,728][20347] Updated weights for policy 0, policy_version 1740554 (0.0010) [2023-12-27 17:42:15,773][20381] Updated weights for policy 1, policy_version 1739126 (0.0005) [2023-12-27 17:42:15,782][20347] Updated weights for policy 0, policy_version 1740564 (0.0010) [2023-12-27 17:42:15,830][20381] Updated weights for policy 1, policy_version 1739136 (0.0005) [2023-12-27 17:42:15,840][20347] Updated weights for policy 0, policy_version 1740574 (0.0010) [2023-12-27 17:42:16,418][20381] Updated weights for policy 1, policy_version 1739146 (0.0008) [2023-12-27 17:42:16,463][20347] Updated weights for policy 0, policy_version 1740584 (0.0007) [2023-12-27 17:42:16,474][20381] Updated weights for policy 1, policy_version 1739156 (0.0007) [2023-12-27 17:42:16,518][20347] Updated weights for policy 0, policy_version 1740594 (0.0008) [2023-12-27 17:42:16,532][20381] Updated weights for policy 1, policy_version 1739166 (0.0010) [2023-12-27 17:42:16,575][20347] Updated weights for policy 0, policy_version 1740604 (0.0005) [2023-12-27 17:42:17,160][20381] Updated weights for policy 1, policy_version 1739176 (0.0008) [2023-12-27 17:42:17,170][20347] Updated weights for policy 0, policy_version 1740614 (0.0006) [2023-12-27 17:42:17,219][20381] Updated weights for policy 1, policy_version 1739186 (0.0005) [2023-12-27 17:42:17,229][20347] Updated weights for policy 0, policy_version 1740624 (0.0007) [2023-12-27 17:42:17,270][20381] Updated weights for policy 1, policy_version 1739196 (0.0005) [2023-12-27 17:42:17,291][20347] Updated weights for policy 0, policy_version 1740634 (0.0006) [2023-12-27 17:42:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 890970112. Throughput: 0: 10338.7, 1: 10260.4. Samples: 890973536. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:17,363][19170] Avg episode reward: [(0, '960.910'), (1, '1000.000')] [2023-12-27 17:42:17,937][20347] Updated weights for policy 0, policy_version 1740644 (0.0005) [2023-12-27 17:42:17,951][20381] Updated weights for policy 1, policy_version 1739206 (0.0007) [2023-12-27 17:42:17,988][20347] Updated weights for policy 0, policy_version 1740654 (0.0005) [2023-12-27 17:42:18,004][20381] Updated weights for policy 1, policy_version 1739216 (0.0008) [2023-12-27 17:42:18,040][20347] Updated weights for policy 0, policy_version 1740664 (0.0006) [2023-12-27 17:42:18,057][20381] Updated weights for policy 1, policy_version 1739226 (0.0008) [2023-12-27 17:42:18,700][20347] Updated weights for policy 0, policy_version 1740674 (0.0005) [2023-12-27 17:42:18,743][20381] Updated weights for policy 1, policy_version 1739236 (0.0006) [2023-12-27 17:42:18,760][20347] Updated weights for policy 0, policy_version 1740684 (0.0005) [2023-12-27 17:42:18,803][20381] Updated weights for policy 1, policy_version 1739246 (0.0005) [2023-12-27 17:42:18,817][20347] Updated weights for policy 0, policy_version 1740694 (0.0006) [2023-12-27 17:42:18,865][20381] Updated weights for policy 1, policy_version 1739256 (0.0006) [2023-12-27 17:42:18,872][20347] Updated weights for policy 0, policy_version 1740704 (0.0006) [2023-12-27 17:42:19,509][20381] Updated weights for policy 1, policy_version 1739266 (0.0007) [2023-12-27 17:42:19,567][20381] Updated weights for policy 1, policy_version 1739276 (0.0008) [2023-12-27 17:42:19,597][20347] Updated weights for policy 0, policy_version 1740714 (0.0007) [2023-12-27 17:42:19,627][20381] Updated weights for policy 1, policy_version 1739286 (0.0008) [2023-12-27 17:42:19,652][20347] Updated weights for policy 0, policy_version 1740724 (0.0007) [2023-12-27 17:42:19,682][20381] Updated weights for policy 1, policy_version 1739296 (0.0008) [2023-12-27 17:42:19,706][20347] Updated weights for policy 0, policy_version 1740734 (0.0006) [2023-12-27 17:42:20,394][20381] Updated weights for policy 1, policy_version 1739306 (0.0008) [2023-12-27 17:42:20,438][20381] Updated weights for policy 1, policy_version 1739316 (0.0006) [2023-12-27 17:42:20,440][20347] Updated weights for policy 0, policy_version 1740744 (0.0007) [2023-12-27 17:42:20,488][20381] Updated weights for policy 1, policy_version 1739326 (0.0006) [2023-12-27 17:42:20,493][20347] Updated weights for policy 0, policy_version 1740754 (0.0007) [2023-12-27 17:42:20,557][20347] Updated weights for policy 0, policy_version 1740764 (0.0008) [2023-12-27 17:42:21,229][20381] Updated weights for policy 1, policy_version 1739336 (0.0008) [2023-12-27 17:42:21,293][20347] Updated weights for policy 0, policy_version 1740774 (0.0007) [2023-12-27 17:42:21,297][20381] Updated weights for policy 1, policy_version 1739346 (0.0007) [2023-12-27 17:42:21,360][20347] Updated weights for policy 0, policy_version 1740784 (0.0006) [2023-12-27 17:42:21,361][20381] Updated weights for policy 1, policy_version 1739356 (0.0007) [2023-12-27 17:42:21,427][20347] Updated weights for policy 0, policy_version 1740794 (0.0008) [2023-12-27 17:42:22,072][20381] Updated weights for policy 1, policy_version 1739366 (0.0008) [2023-12-27 17:42:22,128][20381] Updated weights for policy 1, policy_version 1739376 (0.0008) [2023-12-27 17:42:22,152][20347] Updated weights for policy 0, policy_version 1740804 (0.0007) [2023-12-27 17:42:22,181][20381] Updated weights for policy 1, policy_version 1739386 (0.0007) [2023-12-27 17:42:22,208][20347] Updated weights for policy 0, policy_version 1740814 (0.0006) [2023-12-27 17:42:22,267][20347] Updated weights for policy 0, policy_version 1740824 (0.0006) [2023-12-27 17:42:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 891068416. Throughput: 0: 10301.8, 1: 10254.4. Samples: 891034596. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:22,363][19170] Avg episode reward: [(0, '990.210'), (1, '990.250')] [2023-12-27 17:42:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001739392_445333504.pth... [2023-12-27 17:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001740832_445734912.pth... [2023-12-27 17:42:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001738176_445022208.pth [2023-12-27 17:42:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001739624_445423616.pth [2023-12-27 17:42:22,857][20381] Updated weights for policy 1, policy_version 1739396 (0.0007) [2023-12-27 17:42:22,908][20381] Updated weights for policy 1, policy_version 1739406 (0.0008) [2023-12-27 17:42:22,967][20381] Updated weights for policy 1, policy_version 1739416 (0.0007) [2023-12-27 17:42:23,005][20347] Updated weights for policy 0, policy_version 1740834 (0.0007) [2023-12-27 17:42:23,057][20347] Updated weights for policy 0, policy_version 1740844 (0.0007) [2023-12-27 17:42:23,102][20347] Updated weights for policy 0, policy_version 1740854 (0.0007) [2023-12-27 17:42:23,147][20347] Updated weights for policy 0, policy_version 1740864 (0.0007) [2023-12-27 17:42:23,579][20381] Updated weights for policy 1, policy_version 1739426 (0.0007) [2023-12-27 17:42:23,627][20381] Updated weights for policy 1, policy_version 1739436 (0.0007) [2023-12-27 17:42:23,680][20381] Updated weights for policy 1, policy_version 1739446 (0.0008) [2023-12-27 17:42:23,733][20381] Updated weights for policy 1, policy_version 1739456 (0.0009) [2023-12-27 17:42:23,886][20347] Updated weights for policy 0, policy_version 1740874 (0.0008) [2023-12-27 17:42:23,948][20347] Updated weights for policy 0, policy_version 1740884 (0.0008) [2023-12-27 17:42:24,000][20347] Updated weights for policy 0, policy_version 1740894 (0.0007) [2023-12-27 17:42:24,399][20381] Updated weights for policy 1, policy_version 1739466 (0.0007) [2023-12-27 17:42:24,451][20381] Updated weights for policy 1, policy_version 1739476 (0.0008) [2023-12-27 17:42:24,513][20381] Updated weights for policy 1, policy_version 1739486 (0.0008) [2023-12-27 17:42:24,680][20347] Updated weights for policy 0, policy_version 1740904 (0.0006) [2023-12-27 17:42:24,738][20347] Updated weights for policy 0, policy_version 1740914 (0.0005) [2023-12-27 17:42:24,795][20347] Updated weights for policy 0, policy_version 1740924 (0.0006) [2023-12-27 17:42:25,177][20381] Updated weights for policy 1, policy_version 1739496 (0.0010) [2023-12-27 17:42:25,229][20381] Updated weights for policy 1, policy_version 1739506 (0.0010) [2023-12-27 17:42:25,281][20381] Updated weights for policy 1, policy_version 1739516 (0.0010) [2023-12-27 17:42:25,421][20347] Updated weights for policy 0, policy_version 1740934 (0.0009) [2023-12-27 17:42:25,469][20347] Updated weights for policy 0, policy_version 1740944 (0.0010) [2023-12-27 17:42:25,518][20347] Updated weights for policy 0, policy_version 1740954 (0.0010) [2023-12-27 17:42:25,958][20381] Updated weights for policy 1, policy_version 1739526 (0.0007) [2023-12-27 17:42:26,017][20381] Updated weights for policy 1, policy_version 1739536 (0.0005) [2023-12-27 17:42:26,077][20381] Updated weights for policy 1, policy_version 1739546 (0.0010) [2023-12-27 17:42:26,179][20347] Updated weights for policy 0, policy_version 1740964 (0.0008) [2023-12-27 17:42:26,223][20347] Updated weights for policy 0, policy_version 1740974 (0.0010) [2023-12-27 17:42:26,273][20347] Updated weights for policy 0, policy_version 1740984 (0.0007) [2023-12-27 17:42:26,726][20381] Updated weights for policy 1, policy_version 1739556 (0.0010) [2023-12-27 17:42:26,773][20381] Updated weights for policy 1, policy_version 1739566 (0.0010) [2023-12-27 17:42:26,821][20381] Updated weights for policy 1, policy_version 1739576 (0.0010) [2023-12-27 17:42:26,923][20347] Updated weights for policy 0, policy_version 1740994 (0.0006) [2023-12-27 17:42:26,968][20347] Updated weights for policy 0, policy_version 1741004 (0.0010) [2023-12-27 17:42:27,009][20347] Updated weights for policy 0, policy_version 1741014 (0.0010) [2023-12-27 17:42:27,053][20347] Updated weights for policy 0, policy_version 1741024 (0.0010) [2023-12-27 17:42:27,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 891166720. Throughput: 0: 10278.4, 1: 10314.1. Samples: 891155992. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:27,363][19170] Avg episode reward: [(0, '980.410'), (1, '990.250')] [2023-12-27 17:42:27,464][20381] Updated weights for policy 1, policy_version 1739586 (0.0010) [2023-12-27 17:42:27,515][20381] Updated weights for policy 1, policy_version 1739596 (0.0010) [2023-12-27 17:42:27,570][20381] Updated weights for policy 1, policy_version 1739606 (0.0011) [2023-12-27 17:42:27,624][20381] Updated weights for policy 1, policy_version 1739616 (0.0010) [2023-12-27 17:42:27,761][20347] Updated weights for policy 0, policy_version 1741034 (0.0010) [2023-12-27 17:42:27,809][20347] Updated weights for policy 0, policy_version 1741044 (0.0010) [2023-12-27 17:42:27,860][20347] Updated weights for policy 0, policy_version 1741054 (0.0010) [2023-12-27 17:42:28,255][20381] Updated weights for policy 1, policy_version 1739626 (0.0005) [2023-12-27 17:42:28,305][20381] Updated weights for policy 1, policy_version 1739636 (0.0005) [2023-12-27 17:42:28,364][20381] Updated weights for policy 1, policy_version 1739646 (0.0008) [2023-12-27 17:42:28,550][20347] Updated weights for policy 0, policy_version 1741064 (0.0008) [2023-12-27 17:42:28,605][20347] Updated weights for policy 0, policy_version 1741074 (0.0006) [2023-12-27 17:42:28,658][20347] Updated weights for policy 0, policy_version 1741084 (0.0008) [2023-12-27 17:42:29,073][20381] Updated weights for policy 1, policy_version 1739656 (0.0009) [2023-12-27 17:42:29,118][20381] Updated weights for policy 1, policy_version 1739666 (0.0007) [2023-12-27 17:42:29,165][20381] Updated weights for policy 1, policy_version 1739676 (0.0006) [2023-12-27 17:42:29,275][20347] Updated weights for policy 0, policy_version 1741094 (0.0008) [2023-12-27 17:42:29,330][20347] Updated weights for policy 0, policy_version 1741104 (0.0007) [2023-12-27 17:42:29,397][20347] Updated weights for policy 0, policy_version 1741114 (0.0006) [2023-12-27 17:42:29,411][20291] KL-divergence is very high: 109.4210 [2023-12-27 17:42:29,887][20381] Updated weights for policy 1, policy_version 1739686 (0.0010) [2023-12-27 17:42:29,949][20381] Updated weights for policy 1, policy_version 1739696 (0.0011) [2023-12-27 17:42:30,008][20381] Updated weights for policy 1, policy_version 1739706 (0.0006) [2023-12-27 17:42:30,133][20347] Updated weights for policy 0, policy_version 1741124 (0.0006) [2023-12-27 17:42:30,188][20347] Updated weights for policy 0, policy_version 1741134 (0.0006) [2023-12-27 17:42:30,243][20347] Updated weights for policy 0, policy_version 1741144 (0.0010) [2023-12-27 17:42:30,617][20381] Updated weights for policy 1, policy_version 1739716 (0.0005) [2023-12-27 17:42:30,669][20381] Updated weights for policy 1, policy_version 1739726 (0.0008) [2023-12-27 17:42:30,724][20381] Updated weights for policy 1, policy_version 1739736 (0.0010) [2023-12-27 17:42:30,887][20347] Updated weights for policy 0, policy_version 1741154 (0.0008) [2023-12-27 17:42:30,935][20347] Updated weights for policy 0, policy_version 1741164 (0.0005) [2023-12-27 17:42:30,991][20347] Updated weights for policy 0, policy_version 1741174 (0.0005) [2023-12-27 17:42:31,044][20347] Updated weights for policy 0, policy_version 1741184 (0.0007) [2023-12-27 17:42:31,387][20381] Updated weights for policy 1, policy_version 1739746 (0.0010) [2023-12-27 17:42:31,447][20381] Updated weights for policy 1, policy_version 1739756 (0.0007) [2023-12-27 17:42:31,500][20381] Updated weights for policy 1, policy_version 1739766 (0.0008) [2023-12-27 17:42:31,547][20381] Updated weights for policy 1, policy_version 1739776 (0.0006) [2023-12-27 17:42:31,804][20347] Updated weights for policy 0, policy_version 1741194 (0.0008) [2023-12-27 17:42:31,857][20347] Updated weights for policy 0, policy_version 1741204 (0.0008) [2023-12-27 17:42:31,911][20347] Updated weights for policy 0, policy_version 1741214 (0.0008) [2023-12-27 17:42:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 891265024. Throughput: 0: 10236.0, 1: 10285.0. Samples: 891279184. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:32,363][19170] Avg episode reward: [(0, '951.170'), (1, '1000.000')] [2023-12-27 17:42:32,370][20381] Updated weights for policy 1, policy_version 1739786 (0.0008) [2023-12-27 17:42:32,430][20381] Updated weights for policy 1, policy_version 1739796 (0.0007) [2023-12-27 17:42:32,480][20381] Updated weights for policy 1, policy_version 1739806 (0.0006) [2023-12-27 17:42:32,646][20347] Updated weights for policy 0, policy_version 1741224 (0.0010) [2023-12-27 17:42:32,708][20347] Updated weights for policy 0, policy_version 1741234 (0.0010) [2023-12-27 17:42:32,760][20347] Updated weights for policy 0, policy_version 1741244 (0.0010) [2023-12-27 17:42:33,185][20381] Updated weights for policy 1, policy_version 1739816 (0.0009) [2023-12-27 17:42:33,241][20381] Updated weights for policy 1, policy_version 1739826 (0.0008) [2023-12-27 17:42:33,295][20381] Updated weights for policy 1, policy_version 1739836 (0.0008) [2023-12-27 17:42:33,344][20347] Updated weights for policy 0, policy_version 1741254 (0.0007) [2023-12-27 17:42:33,400][20347] Updated weights for policy 0, policy_version 1741264 (0.0005) [2023-12-27 17:42:33,459][20347] Updated weights for policy 0, policy_version 1741274 (0.0005) [2023-12-27 17:42:34,008][20381] Updated weights for policy 1, policy_version 1739846 (0.0007) [2023-12-27 17:42:34,070][20381] Updated weights for policy 1, policy_version 1739856 (0.0005) [2023-12-27 17:42:34,077][20347] Updated weights for policy 0, policy_version 1741284 (0.0008) [2023-12-27 17:42:34,130][20381] Updated weights for policy 1, policy_version 1739866 (0.0006) [2023-12-27 17:42:34,140][20347] Updated weights for policy 0, policy_version 1741294 (0.0007) [2023-12-27 17:42:34,208][20347] Updated weights for policy 0, policy_version 1741304 (0.0010) [2023-12-27 17:42:34,801][20381] Updated weights for policy 1, policy_version 1739876 (0.0010) [2023-12-27 17:42:34,843][20347] Updated weights for policy 0, policy_version 1741314 (0.0007) [2023-12-27 17:42:34,858][20381] Updated weights for policy 1, policy_version 1739886 (0.0010) [2023-12-27 17:42:34,899][20347] Updated weights for policy 0, policy_version 1741324 (0.0010) [2023-12-27 17:42:34,914][20381] Updated weights for policy 1, policy_version 1739896 (0.0010) [2023-12-27 17:42:34,955][20347] Updated weights for policy 0, policy_version 1741334 (0.0010) [2023-12-27 17:42:35,015][20347] Updated weights for policy 0, policy_version 1741344 (0.0006) [2023-12-27 17:42:35,508][20381] Updated weights for policy 1, policy_version 1739906 (0.0006) [2023-12-27 17:42:35,564][20381] Updated weights for policy 1, policy_version 1739916 (0.0011) [2023-12-27 17:42:35,619][20381] Updated weights for policy 1, policy_version 1739926 (0.0011) [2023-12-27 17:42:35,665][20347] Updated weights for policy 0, policy_version 1741354 (0.0010) [2023-12-27 17:42:35,679][20381] Updated weights for policy 1, policy_version 1739936 (0.0011) [2023-12-27 17:42:35,722][20347] Updated weights for policy 0, policy_version 1741364 (0.0011) [2023-12-27 17:42:35,781][20347] Updated weights for policy 0, policy_version 1741374 (0.0011) [2023-12-27 17:42:36,352][20381] Updated weights for policy 1, policy_version 1739946 (0.0008) [2023-12-27 17:42:36,414][20381] Updated weights for policy 1, policy_version 1739956 (0.0008) [2023-12-27 17:42:36,477][20381] Updated weights for policy 1, policy_version 1739966 (0.0007) [2023-12-27 17:42:36,489][20347] Updated weights for policy 0, policy_version 1741384 (0.0010) [2023-12-27 17:42:36,548][20347] Updated weights for policy 0, policy_version 1741394 (0.0010) [2023-12-27 17:42:36,593][20347] Updated weights for policy 0, policy_version 1741404 (0.0010) [2023-12-27 17:42:37,157][20381] Updated weights for policy 1, policy_version 1739976 (0.0009) [2023-12-27 17:42:37,221][20381] Updated weights for policy 1, policy_version 1739986 (0.0006) [2023-12-27 17:42:37,241][20347] Updated weights for policy 0, policy_version 1741414 (0.0007) [2023-12-27 17:42:37,272][20381] Updated weights for policy 1, policy_version 1739996 (0.0008) [2023-12-27 17:42:37,297][20347] Updated weights for policy 0, policy_version 1741424 (0.0005) [2023-12-27 17:42:37,358][20347] Updated weights for policy 0, policy_version 1741434 (0.0007) [2023-12-27 17:42:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 891371520. Throughput: 0: 10299.4, 1: 10269.8. Samples: 891341152. Policy #0 lag: (min: 21.0, avg: 25.5, max: 53.0) [2023-12-27 17:42:37,363][19170] Avg episode reward: [(0, '961.020'), (1, '1000.000')] [2023-12-27 17:42:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001740000_445489152.pth... [2023-12-27 17:42:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001738784_445177856.pth [2023-12-27 17:42:37,391][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001741440_445890560.pth... [2023-12-27 17:42:37,394][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001740224_445579264.pth [2023-12-27 17:42:37,940][20381] Updated weights for policy 1, policy_version 1740006 (0.0006) [2023-12-27 17:42:37,995][20381] Updated weights for policy 1, policy_version 1740016 (0.0005) [2023-12-27 17:42:38,052][20381] Updated weights for policy 1, policy_version 1740026 (0.0006) [2023-12-27 17:42:38,062][20347] Updated weights for policy 0, policy_version 1741444 (0.0006) [2023-12-27 17:42:38,108][20347] Updated weights for policy 0, policy_version 1741454 (0.0008) [2023-12-27 17:42:38,162][20347] Updated weights for policy 0, policy_version 1741464 (0.0007) [2023-12-27 17:42:38,677][20381] Updated weights for policy 1, policy_version 1740036 (0.0007) [2023-12-27 17:42:38,741][20381] Updated weights for policy 1, policy_version 1740046 (0.0007) [2023-12-27 17:42:38,801][20381] Updated weights for policy 1, policy_version 1740056 (0.0007) [2023-12-27 17:42:38,853][20347] Updated weights for policy 0, policy_version 1741474 (0.0009) [2023-12-27 17:42:38,921][20347] Updated weights for policy 0, policy_version 1741484 (0.0008) [2023-12-27 17:42:38,984][20347] Updated weights for policy 0, policy_version 1741494 (0.0008) [2023-12-27 17:42:39,048][20347] Updated weights for policy 0, policy_version 1741504 (0.0008) [2023-12-27 17:42:39,539][20381] Updated weights for policy 1, policy_version 1740066 (0.0008) [2023-12-27 17:42:39,596][20381] Updated weights for policy 1, policy_version 1740076 (0.0007) [2023-12-27 17:42:39,656][20381] Updated weights for policy 1, policy_version 1740086 (0.0008) [2023-12-27 17:42:39,679][20347] Updated weights for policy 0, policy_version 1741514 (0.0006) [2023-12-27 17:42:39,720][20381] Updated weights for policy 1, policy_version 1740096 (0.0007) [2023-12-27 17:42:39,735][20347] Updated weights for policy 0, policy_version 1741524 (0.0007) [2023-12-27 17:42:39,799][20347] Updated weights for policy 0, policy_version 1741534 (0.0007) [2023-12-27 17:42:40,451][20347] Updated weights for policy 0, policy_version 1741544 (0.0007) [2023-12-27 17:42:40,472][20381] Updated weights for policy 1, policy_version 1740106 (0.0008) [2023-12-27 17:42:40,510][20347] Updated weights for policy 0, policy_version 1741554 (0.0006) [2023-12-27 17:42:40,530][20381] Updated weights for policy 1, policy_version 1740116 (0.0008) [2023-12-27 17:42:40,570][20347] Updated weights for policy 0, policy_version 1741564 (0.0008) [2023-12-27 17:42:40,589][20381] Updated weights for policy 1, policy_version 1740126 (0.0006) [2023-12-27 17:42:41,240][20347] Updated weights for policy 0, policy_version 1741574 (0.0007) [2023-12-27 17:42:41,292][20381] Updated weights for policy 1, policy_version 1740136 (0.0007) [2023-12-27 17:42:41,301][20347] Updated weights for policy 0, policy_version 1741584 (0.0008) [2023-12-27 17:42:41,346][20381] Updated weights for policy 1, policy_version 1740146 (0.0007) [2023-12-27 17:42:41,367][20347] Updated weights for policy 0, policy_version 1741594 (0.0007) [2023-12-27 17:42:41,420][20381] Updated weights for policy 1, policy_version 1740156 (0.0007) [2023-12-27 17:42:42,069][20347] Updated weights for policy 0, policy_version 1741604 (0.0008) [2023-12-27 17:42:42,124][20347] Updated weights for policy 0, policy_version 1741614 (0.0007) [2023-12-27 17:42:42,142][20381] Updated weights for policy 1, policy_version 1740166 (0.0006) [2023-12-27 17:42:42,186][20347] Updated weights for policy 0, policy_version 1741624 (0.0006) [2023-12-27 17:42:42,203][20381] Updated weights for policy 1, policy_version 1740176 (0.0006) [2023-12-27 17:42:42,266][20381] Updated weights for policy 1, policy_version 1740186 (0.0006) [2023-12-27 17:42:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 891478016. Throughput: 0: 10296.8, 1: 10266.7. Samples: 891463944. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:42:42,363][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 17:42:42,814][20347] Updated weights for policy 0, policy_version 1741634 (0.0007) [2023-12-27 17:42:42,872][20347] Updated weights for policy 0, policy_version 1741644 (0.0007) [2023-12-27 17:42:42,900][20381] Updated weights for policy 1, policy_version 1740196 (0.0006) [2023-12-27 17:42:42,922][20347] Updated weights for policy 0, policy_version 1741654 (0.0006) [2023-12-27 17:42:42,953][20381] Updated weights for policy 1, policy_version 1740206 (0.0006) [2023-12-27 17:42:42,973][20347] Updated weights for policy 0, policy_version 1741664 (0.0005) [2023-12-27 17:42:43,010][20381] Updated weights for policy 1, policy_version 1740216 (0.0007) [2023-12-27 17:42:43,612][20381] Updated weights for policy 1, policy_version 1740226 (0.0008) [2023-12-27 17:42:43,662][20381] Updated weights for policy 1, policy_version 1740236 (0.0006) [2023-12-27 17:42:43,695][20347] Updated weights for policy 0, policy_version 1741674 (0.0007) [2023-12-27 17:42:43,710][20381] Updated weights for policy 1, policy_version 1740246 (0.0005) [2023-12-27 17:42:43,749][20347] Updated weights for policy 0, policy_version 1741684 (0.0007) [2023-12-27 17:42:43,758][20381] Updated weights for policy 1, policy_version 1740256 (0.0005) [2023-12-27 17:42:43,807][20347] Updated weights for policy 0, policy_version 1741694 (0.0007) [2023-12-27 17:42:44,451][20381] Updated weights for policy 1, policy_version 1740266 (0.0006) [2023-12-27 17:42:44,499][20381] Updated weights for policy 1, policy_version 1740276 (0.0007) [2023-12-27 17:42:44,510][20347] Updated weights for policy 0, policy_version 1741704 (0.0007) [2023-12-27 17:42:44,554][20381] Updated weights for policy 1, policy_version 1740286 (0.0007) [2023-12-27 17:42:44,574][20347] Updated weights for policy 0, policy_version 1741714 (0.0006) [2023-12-27 17:42:44,638][20347] Updated weights for policy 0, policy_version 1741724 (0.0007) [2023-12-27 17:42:45,241][20347] Updated weights for policy 0, policy_version 1741734 (0.0007) [2023-12-27 17:42:45,271][20381] Updated weights for policy 1, policy_version 1740296 (0.0008) [2023-12-27 17:42:45,296][20347] Updated weights for policy 0, policy_version 1741744 (0.0008) [2023-12-27 17:42:45,328][20381] Updated weights for policy 1, policy_version 1740306 (0.0006) [2023-12-27 17:42:45,354][20347] Updated weights for policy 0, policy_version 1741754 (0.0005) [2023-12-27 17:42:45,393][20381] Updated weights for policy 1, policy_version 1740316 (0.0006) [2023-12-27 17:42:46,036][20381] Updated weights for policy 1, policy_version 1740326 (0.0008) [2023-12-27 17:42:46,043][20347] Updated weights for policy 0, policy_version 1741764 (0.0005) [2023-12-27 17:42:46,097][20347] Updated weights for policy 0, policy_version 1741774 (0.0005) [2023-12-27 17:42:46,098][20381] Updated weights for policy 1, policy_version 1740336 (0.0008) [2023-12-27 17:42:46,147][20347] Updated weights for policy 0, policy_version 1741784 (0.0008) [2023-12-27 17:42:46,157][20381] Updated weights for policy 1, policy_version 1740346 (0.0009) [2023-12-27 17:42:46,730][20347] Updated weights for policy 0, policy_version 1741794 (0.0006) [2023-12-27 17:42:46,785][20347] Updated weights for policy 0, policy_version 1741804 (0.0006) [2023-12-27 17:42:46,844][20347] Updated weights for policy 0, policy_version 1741814 (0.0006) [2023-12-27 17:42:46,851][20291] KL-divergence is very high: 124.1717 [2023-12-27 17:42:46,875][20381] Updated weights for policy 1, policy_version 1740356 (0.0006) [2023-12-27 17:42:46,894][20291] KL-divergence is very high: 169.2964 [2023-12-27 17:42:46,900][20347] Updated weights for policy 0, policy_version 1741824 (0.0007) [2023-12-27 17:42:46,927][20381] Updated weights for policy 1, policy_version 1740366 (0.0007) [2023-12-27 17:42:46,985][20381] Updated weights for policy 1, policy_version 1740376 (0.0007) [2023-12-27 17:42:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 891576320. Throughput: 0: 10347.8, 1: 10247.7. Samples: 891586372. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:42:47,363][19170] Avg episode reward: [(0, '951.150'), (1, '1000.000')] [2023-12-27 17:42:47,498][20347] Updated weights for policy 0, policy_version 1741834 (0.0005) [2023-12-27 17:42:47,551][20347] Updated weights for policy 0, policy_version 1741844 (0.0005) [2023-12-27 17:42:47,607][20347] Updated weights for policy 0, policy_version 1741854 (0.0005) [2023-12-27 17:42:47,625][20381] Updated weights for policy 1, policy_version 1740386 (0.0006) [2023-12-27 17:42:47,680][20381] Updated weights for policy 1, policy_version 1740396 (0.0008) [2023-12-27 17:42:47,750][20381] Updated weights for policy 1, policy_version 1740406 (0.0006) [2023-12-27 17:42:47,810][20381] Updated weights for policy 1, policy_version 1740416 (0.0007) [2023-12-27 17:42:48,280][20347] Updated weights for policy 0, policy_version 1741864 (0.0009) [2023-12-27 17:42:48,338][20347] Updated weights for policy 0, policy_version 1741874 (0.0010) [2023-12-27 17:42:48,408][20347] Updated weights for policy 0, policy_version 1741884 (0.0010) [2023-12-27 17:42:48,491][20381] Updated weights for policy 1, policy_version 1740426 (0.0007) [2023-12-27 17:42:48,539][20381] Updated weights for policy 1, policy_version 1740436 (0.0007) [2023-12-27 17:42:48,591][20381] Updated weights for policy 1, policy_version 1740446 (0.0008) [2023-12-27 17:42:49,123][20347] Updated weights for policy 0, policy_version 1741894 (0.0009) [2023-12-27 17:42:49,182][20347] Updated weights for policy 0, policy_version 1741905 (0.0009) [2023-12-27 17:42:49,241][20347] Updated weights for policy 0, policy_version 1741915 (0.0008) [2023-12-27 17:42:49,341][20381] Updated weights for policy 1, policy_version 1740456 (0.0010) [2023-12-27 17:42:49,404][20381] Updated weights for policy 1, policy_version 1740466 (0.0008) [2023-12-27 17:42:49,464][20381] Updated weights for policy 1, policy_version 1740476 (0.0009) [2023-12-27 17:42:49,944][20347] Updated weights for policy 0, policy_version 1741925 (0.0007) [2023-12-27 17:42:49,999][20347] Updated weights for policy 0, policy_version 1741935 (0.0008) [2023-12-27 17:42:50,053][20347] Updated weights for policy 0, policy_version 1741945 (0.0008) [2023-12-27 17:42:50,138][20381] Updated weights for policy 1, policy_version 1740486 (0.0007) [2023-12-27 17:42:50,195][20381] Updated weights for policy 1, policy_version 1740496 (0.0010) [2023-12-27 17:42:50,253][20381] Updated weights for policy 1, policy_version 1740506 (0.0008) [2023-12-27 17:42:50,778][20347] Updated weights for policy 0, policy_version 1741955 (0.0008) [2023-12-27 17:42:50,828][20347] Updated weights for policy 0, policy_version 1741965 (0.0007) [2023-12-27 17:42:50,883][20347] Updated weights for policy 0, policy_version 1741975 (0.0006) [2023-12-27 17:42:50,962][20381] Updated weights for policy 1, policy_version 1740516 (0.0009) [2023-12-27 17:42:51,019][20381] Updated weights for policy 1, policy_version 1740526 (0.0011) [2023-12-27 17:42:51,085][20381] Updated weights for policy 1, policy_version 1740536 (0.0011) [2023-12-27 17:42:51,647][20347] Updated weights for policy 0, policy_version 1741985 (0.0007) [2023-12-27 17:42:51,708][20347] Updated weights for policy 0, policy_version 1741995 (0.0008) [2023-12-27 17:42:51,774][20347] Updated weights for policy 0, policy_version 1742005 (0.0007) [2023-12-27 17:42:51,789][20381] Updated weights for policy 1, policy_version 1740546 (0.0009) [2023-12-27 17:42:51,841][20347] Updated weights for policy 0, policy_version 1742015 (0.0007) [2023-12-27 17:42:51,843][20381] Updated weights for policy 1, policy_version 1740556 (0.0008) [2023-12-27 17:42:51,898][20381] Updated weights for policy 1, policy_version 1740566 (0.0005) [2023-12-27 17:42:51,956][20381] Updated weights for policy 1, policy_version 1740576 (0.0007) [2023-12-27 17:42:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 891674624. Throughput: 0: 10324.0, 1: 10262.6. Samples: 891647604. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:42:52,363][19170] Avg episode reward: [(0, '970.670'), (1, '1000.000')] [2023-12-27 17:42:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001740576_445636608.pth... [2023-12-27 17:42:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001742016_446038016.pth... [2023-12-27 17:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001740832_445734912.pth [2023-12-27 17:42:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001739392_445333504.pth [2023-12-27 17:42:52,532][20347] Updated weights for policy 0, policy_version 1742025 (0.0009) [2023-12-27 17:42:52,585][20347] Updated weights for policy 0, policy_version 1742035 (0.0009) [2023-12-27 17:42:52,649][20347] Updated weights for policy 0, policy_version 1742045 (0.0008) [2023-12-27 17:42:52,716][20381] Updated weights for policy 1, policy_version 1740586 (0.0008) [2023-12-27 17:42:52,775][20381] Updated weights for policy 1, policy_version 1740596 (0.0006) [2023-12-27 17:42:52,827][20381] Updated weights for policy 1, policy_version 1740606 (0.0007) [2023-12-27 17:42:53,361][20347] Updated weights for policy 0, policy_version 1742055 (0.0006) [2023-12-27 17:42:53,413][20347] Updated weights for policy 0, policy_version 1742065 (0.0008) [2023-12-27 17:42:53,463][20347] Updated weights for policy 0, policy_version 1742075 (0.0007) [2023-12-27 17:42:53,497][20381] Updated weights for policy 1, policy_version 1740616 (0.0006) [2023-12-27 17:42:53,543][20381] Updated weights for policy 1, policy_version 1740626 (0.0005) [2023-12-27 17:42:53,592][20381] Updated weights for policy 1, policy_version 1740636 (0.0005) [2023-12-27 17:42:54,067][20347] Updated weights for policy 0, policy_version 1742085 (0.0009) [2023-12-27 17:42:54,126][20347] Updated weights for policy 0, policy_version 1742095 (0.0006) [2023-12-27 17:42:54,157][20381] Updated weights for policy 1, policy_version 1740646 (0.0008) [2023-12-27 17:42:54,184][20347] Updated weights for policy 0, policy_version 1742105 (0.0005) [2023-12-27 17:42:54,203][20381] Updated weights for policy 1, policy_version 1740656 (0.0010) [2023-12-27 17:42:54,262][20381] Updated weights for policy 1, policy_version 1740666 (0.0010) [2023-12-27 17:42:54,850][20347] Updated weights for policy 0, policy_version 1742115 (0.0008) [2023-12-27 17:42:54,913][20347] Updated weights for policy 0, policy_version 1742125 (0.0007) [2023-12-27 17:42:54,934][20381] Updated weights for policy 1, policy_version 1740676 (0.0010) [2023-12-27 17:42:54,958][20347] Updated weights for policy 0, policy_version 1742135 (0.0005) [2023-12-27 17:42:54,994][20381] Updated weights for policy 1, policy_version 1740686 (0.0008) [2023-12-27 17:42:55,049][20381] Updated weights for policy 1, policy_version 1740696 (0.0011) [2023-12-27 17:42:55,650][20347] Updated weights for policy 0, policy_version 1742145 (0.0008) [2023-12-27 17:42:55,699][20347] Updated weights for policy 0, policy_version 1742155 (0.0007) [2023-12-27 17:42:55,720][20381] Updated weights for policy 1, policy_version 1740706 (0.0010) [2023-12-27 17:42:55,754][20347] Updated weights for policy 0, policy_version 1742165 (0.0006) [2023-12-27 17:42:55,778][20381] Updated weights for policy 1, policy_version 1740716 (0.0010) [2023-12-27 17:42:55,809][20347] Updated weights for policy 0, policy_version 1742175 (0.0005) [2023-12-27 17:42:55,833][20381] Updated weights for policy 1, policy_version 1740726 (0.0010) [2023-12-27 17:42:55,884][20381] Updated weights for policy 1, policy_version 1740736 (0.0010) [2023-12-27 17:42:56,510][20347] Updated weights for policy 0, policy_version 1742185 (0.0007) [2023-12-27 17:42:56,516][20381] Updated weights for policy 1, policy_version 1740746 (0.0010) [2023-12-27 17:42:56,557][20347] Updated weights for policy 0, policy_version 1742195 (0.0005) [2023-12-27 17:42:56,563][20381] Updated weights for policy 1, policy_version 1740756 (0.0010) [2023-12-27 17:42:56,605][20347] Updated weights for policy 0, policy_version 1742205 (0.0005) [2023-12-27 17:42:56,611][20381] Updated weights for policy 1, policy_version 1740766 (0.0010) [2023-12-27 17:42:57,302][20347] Updated weights for policy 0, policy_version 1742215 (0.0006) [2023-12-27 17:42:57,311][20381] Updated weights for policy 1, policy_version 1740776 (0.0007) [2023-12-27 17:42:57,358][20347] Updated weights for policy 0, policy_version 1742225 (0.0006) [2023-12-27 17:42:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 891772928. Throughput: 0: 10279.5, 1: 10236.7. Samples: 891770024. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:42:57,363][19170] Avg episode reward: [(0, '960.940'), (1, '1000.000')] [2023-12-27 17:42:57,365][20381] Updated weights for policy 1, policy_version 1740786 (0.0008) [2023-12-27 17:42:57,418][20347] Updated weights for policy 0, policy_version 1742235 (0.0005) [2023-12-27 17:42:57,423][20381] Updated weights for policy 1, policy_version 1740796 (0.0007) [2023-12-27 17:42:58,046][20381] Updated weights for policy 1, policy_version 1740806 (0.0008) [2023-12-27 17:42:58,093][20347] Updated weights for policy 0, policy_version 1742245 (0.0006) [2023-12-27 17:42:58,103][20381] Updated weights for policy 1, policy_version 1740816 (0.0006) [2023-12-27 17:42:58,143][20347] Updated weights for policy 0, policy_version 1742255 (0.0007) [2023-12-27 17:42:58,151][20381] Updated weights for policy 1, policy_version 1740826 (0.0008) [2023-12-27 17:42:58,202][20347] Updated weights for policy 0, policy_version 1742265 (0.0007) [2023-12-27 17:42:58,937][20381] Updated weights for policy 1, policy_version 1740836 (0.0009) [2023-12-27 17:42:58,984][20347] Updated weights for policy 0, policy_version 1742275 (0.0007) [2023-12-27 17:42:58,989][20381] Updated weights for policy 1, policy_version 1740846 (0.0007) [2023-12-27 17:42:59,042][20347] Updated weights for policy 0, policy_version 1742285 (0.0008) [2023-12-27 17:42:59,046][20381] Updated weights for policy 1, policy_version 1740856 (0.0007) [2023-12-27 17:42:59,101][20347] Updated weights for policy 0, policy_version 1742295 (0.0007) [2023-12-27 17:42:59,672][20381] Updated weights for policy 1, policy_version 1740866 (0.0007) [2023-12-27 17:42:59,738][20381] Updated weights for policy 1, policy_version 1740876 (0.0006) [2023-12-27 17:42:59,795][20381] Updated weights for policy 1, policy_version 1740886 (0.0005) [2023-12-27 17:42:59,862][20347] Updated weights for policy 0, policy_version 1742305 (0.0005) [2023-12-27 17:42:59,864][20381] Updated weights for policy 1, policy_version 1740896 (0.0007) [2023-12-27 17:42:59,920][20347] Updated weights for policy 0, policy_version 1742315 (0.0007) [2023-12-27 17:42:59,983][20347] Updated weights for policy 0, policy_version 1742325 (0.0008) [2023-12-27 17:43:00,045][20347] Updated weights for policy 0, policy_version 1742335 (0.0008) [2023-12-27 17:43:00,498][20381] Updated weights for policy 1, policy_version 1740906 (0.0007) [2023-12-27 17:43:00,547][20381] Updated weights for policy 1, policy_version 1740916 (0.0007) [2023-12-27 17:43:00,599][20381] Updated weights for policy 1, policy_version 1740926 (0.0007) [2023-12-27 17:43:00,713][20347] Updated weights for policy 0, policy_version 1742345 (0.0007) [2023-12-27 17:43:00,761][20347] Updated weights for policy 0, policy_version 1742355 (0.0007) [2023-12-27 17:43:00,811][20347] Updated weights for policy 0, policy_version 1742365 (0.0005) [2023-12-27 17:43:01,317][20381] Updated weights for policy 1, policy_version 1740936 (0.0010) [2023-12-27 17:43:01,389][20381] Updated weights for policy 1, policy_version 1740946 (0.0010) [2023-12-27 17:43:01,451][20381] Updated weights for policy 1, policy_version 1740956 (0.0011) [2023-12-27 17:43:01,511][20347] Updated weights for policy 0, policy_version 1742375 (0.0008) [2023-12-27 17:43:01,568][20347] Updated weights for policy 0, policy_version 1742385 (0.0008) [2023-12-27 17:43:01,622][20347] Updated weights for policy 0, policy_version 1742395 (0.0009) [2023-12-27 17:43:02,126][20381] Updated weights for policy 1, policy_version 1740966 (0.0007) [2023-12-27 17:43:02,175][20381] Updated weights for policy 1, policy_version 1740976 (0.0007) [2023-12-27 17:43:02,223][20381] Updated weights for policy 1, policy_version 1740986 (0.0007) [2023-12-27 17:43:02,324][20347] Updated weights for policy 0, policy_version 1742405 (0.0009) [2023-12-27 17:43:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 891879424. Throughput: 0: 10170.1, 1: 10202.2. Samples: 891890288. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:02,363][19170] Avg episode reward: [(0, '960.940'), (1, '1000.000')] [2023-12-27 17:43:02,385][20347] Updated weights for policy 0, policy_version 1742415 (0.0007) [2023-12-27 17:43:02,438][20347] Updated weights for policy 0, policy_version 1742425 (0.0007) [2023-12-27 17:43:02,864][20381] Updated weights for policy 1, policy_version 1740996 (0.0007) [2023-12-27 17:43:02,921][20381] Updated weights for policy 1, policy_version 1741006 (0.0007) [2023-12-27 17:43:02,977][20381] Updated weights for policy 1, policy_version 1741016 (0.0007) [2023-12-27 17:43:03,118][20347] Updated weights for policy 0, policy_version 1742435 (0.0008) [2023-12-27 17:43:03,172][20347] Updated weights for policy 0, policy_version 1742445 (0.0008) [2023-12-27 17:43:03,230][20347] Updated weights for policy 0, policy_version 1742455 (0.0007) [2023-12-27 17:43:03,679][20381] Updated weights for policy 1, policy_version 1741026 (0.0008) [2023-12-27 17:43:03,733][20381] Updated weights for policy 1, policy_version 1741036 (0.0008) [2023-12-27 17:43:03,791][20381] Updated weights for policy 1, policy_version 1741046 (0.0008) [2023-12-27 17:43:03,860][20381] Updated weights for policy 1, policy_version 1741056 (0.0009) [2023-12-27 17:43:03,876][20347] Updated weights for policy 0, policy_version 1742465 (0.0007) [2023-12-27 17:43:03,933][20347] Updated weights for policy 0, policy_version 1742475 (0.0008) [2023-12-27 17:43:03,996][20347] Updated weights for policy 0, policy_version 1742485 (0.0008) [2023-12-27 17:43:04,060][20347] Updated weights for policy 0, policy_version 1742495 (0.0008) [2023-12-27 17:43:04,534][20381] Updated weights for policy 1, policy_version 1741066 (0.0005) [2023-12-27 17:43:04,585][20381] Updated weights for policy 1, policy_version 1741076 (0.0006) [2023-12-27 17:43:04,635][20381] Updated weights for policy 1, policy_version 1741086 (0.0007) [2023-12-27 17:43:04,736][20347] Updated weights for policy 0, policy_version 1742505 (0.0007) [2023-12-27 17:43:04,792][20347] Updated weights for policy 0, policy_version 1742515 (0.0008) [2023-12-27 17:43:04,846][20347] Updated weights for policy 0, policy_version 1742525 (0.0008) [2023-12-27 17:43:05,273][20381] Updated weights for policy 1, policy_version 1741096 (0.0008) [2023-12-27 17:43:05,330][20381] Updated weights for policy 1, policy_version 1741106 (0.0007) [2023-12-27 17:43:05,373][20381] Updated weights for policy 1, policy_version 1741116 (0.0007) [2023-12-27 17:43:05,569][20347] Updated weights for policy 0, policy_version 1742535 (0.0009) [2023-12-27 17:43:05,619][20347] Updated weights for policy 0, policy_version 1742545 (0.0010) [2023-12-27 17:43:05,680][20347] Updated weights for policy 0, policy_version 1742555 (0.0010) [2023-12-27 17:43:06,080][20381] Updated weights for policy 1, policy_version 1741126 (0.0009) [2023-12-27 17:43:06,143][20381] Updated weights for policy 1, policy_version 1741136 (0.0008) [2023-12-27 17:43:06,216][20381] Updated weights for policy 1, policy_version 1741146 (0.0008) [2023-12-27 17:43:06,376][20347] Updated weights for policy 0, policy_version 1742565 (0.0010) [2023-12-27 17:43:06,438][20347] Updated weights for policy 0, policy_version 1742575 (0.0010) [2023-12-27 17:43:06,497][20347] Updated weights for policy 0, policy_version 1742585 (0.0011) [2023-12-27 17:43:06,891][20381] Updated weights for policy 1, policy_version 1741156 (0.0007) [2023-12-27 17:43:06,940][20381] Updated weights for policy 1, policy_version 1741166 (0.0008) [2023-12-27 17:43:06,994][20381] Updated weights for policy 1, policy_version 1741176 (0.0008) [2023-12-27 17:43:07,137][20347] Updated weights for policy 0, policy_version 1742595 (0.0010) [2023-12-27 17:43:07,197][20347] Updated weights for policy 0, policy_version 1742605 (0.0010) [2023-12-27 17:43:07,258][20347] Updated weights for policy 0, policy_version 1742615 (0.0009) [2023-12-27 17:43:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 891985920. Throughput: 0: 10166.6, 1: 10225.5. Samples: 891952240. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:07,363][19170] Avg episode reward: [(0, '970.610'), (1, '1000.000')] [2023-12-27 17:43:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001741184_445792256.pth... [2023-12-27 17:43:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001742624_446193664.pth... [2023-12-27 17:43:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001740000_445489152.pth [2023-12-27 17:43:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001741440_445890560.pth [2023-12-27 17:43:07,656][20381] Updated weights for policy 1, policy_version 1741186 (0.0007) [2023-12-27 17:43:07,706][20381] Updated weights for policy 1, policy_version 1741196 (0.0008) [2023-12-27 17:43:07,755][20381] Updated weights for policy 1, policy_version 1741206 (0.0007) [2023-12-27 17:43:07,813][20381] Updated weights for policy 1, policy_version 1741216 (0.0007) [2023-12-27 17:43:07,827][20347] Updated weights for policy 0, policy_version 1742625 (0.0007) [2023-12-27 17:43:07,883][20347] Updated weights for policy 0, policy_version 1742635 (0.0005) [2023-12-27 17:43:07,936][20347] Updated weights for policy 0, policy_version 1742645 (0.0006) [2023-12-27 17:43:07,987][20347] Updated weights for policy 0, policy_version 1742655 (0.0005) [2023-12-27 17:43:08,551][20381] Updated weights for policy 1, policy_version 1741226 (0.0008) [2023-12-27 17:43:08,603][20381] Updated weights for policy 1, policy_version 1741236 (0.0007) [2023-12-27 17:43:08,639][20347] Updated weights for policy 0, policy_version 1742665 (0.0007) [2023-12-27 17:43:08,652][20381] Updated weights for policy 1, policy_version 1741246 (0.0005) [2023-12-27 17:43:08,692][20347] Updated weights for policy 0, policy_version 1742675 (0.0007) [2023-12-27 17:43:08,746][20347] Updated weights for policy 0, policy_version 1742685 (0.0007) [2023-12-27 17:43:09,371][20381] Updated weights for policy 1, policy_version 1741256 (0.0009) [2023-12-27 17:43:09,441][20381] Updated weights for policy 1, policy_version 1741266 (0.0008) [2023-12-27 17:43:09,443][20347] Updated weights for policy 0, policy_version 1742695 (0.0008) [2023-12-27 17:43:09,499][20347] Updated weights for policy 0, policy_version 1742705 (0.0008) [2023-12-27 17:43:09,505][20381] Updated weights for policy 1, policy_version 1741276 (0.0008) [2023-12-27 17:43:09,557][20347] Updated weights for policy 0, policy_version 1742715 (0.0006) [2023-12-27 17:43:10,244][20347] Updated weights for policy 0, policy_version 1742725 (0.0006) [2023-12-27 17:43:10,290][20381] Updated weights for policy 1, policy_version 1741286 (0.0008) [2023-12-27 17:43:10,308][20347] Updated weights for policy 0, policy_version 1742735 (0.0007) [2023-12-27 17:43:10,345][20381] Updated weights for policy 1, policy_version 1741296 (0.0006) [2023-12-27 17:43:10,370][20347] Updated weights for policy 0, policy_version 1742745 (0.0007) [2023-12-27 17:43:10,407][20381] Updated weights for policy 1, policy_version 1741306 (0.0008) [2023-12-27 17:43:11,033][20347] Updated weights for policy 0, policy_version 1742755 (0.0007) [2023-12-27 17:43:11,085][20381] Updated weights for policy 1, policy_version 1741316 (0.0007) [2023-12-27 17:43:11,099][20347] Updated weights for policy 0, policy_version 1742765 (0.0007) [2023-12-27 17:43:11,156][20381] Updated weights for policy 1, policy_version 1741326 (0.0009) [2023-12-27 17:43:11,173][20347] Updated weights for policy 0, policy_version 1742775 (0.0007) [2023-12-27 17:43:11,215][20381] Updated weights for policy 1, policy_version 1741336 (0.0008) [2023-12-27 17:43:11,918][20347] Updated weights for policy 0, policy_version 1742785 (0.0008) [2023-12-27 17:43:11,974][20381] Updated weights for policy 1, policy_version 1741346 (0.0009) [2023-12-27 17:43:11,975][20347] Updated weights for policy 0, policy_version 1742795 (0.0009) [2023-12-27 17:43:12,034][20381] Updated weights for policy 1, policy_version 1741356 (0.0011) [2023-12-27 17:43:12,035][20347] Updated weights for policy 0, policy_version 1742805 (0.0010) [2023-12-27 17:43:12,092][20347] Updated weights for policy 0, policy_version 1742815 (0.0010) [2023-12-27 17:43:12,093][20381] Updated weights for policy 1, policy_version 1741366 (0.0011) [2023-12-27 17:43:12,154][20381] Updated weights for policy 1, policy_version 1741376 (0.0011) [2023-12-27 17:43:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 892084224. Throughput: 0: 10232.7, 1: 10159.8. Samples: 892073656. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:12,363][19170] Avg episode reward: [(0, '951.080'), (1, '1000.000')] [2023-12-27 17:43:12,752][20347] Updated weights for policy 0, policy_version 1742825 (0.0011) [2023-12-27 17:43:12,812][20347] Updated weights for policy 0, policy_version 1742835 (0.0010) [2023-12-27 17:43:12,843][20381] Updated weights for policy 1, policy_version 1741386 (0.0006) [2023-12-27 17:43:12,874][20347] Updated weights for policy 0, policy_version 1742845 (0.0011) [2023-12-27 17:43:12,905][20381] Updated weights for policy 1, policy_version 1741396 (0.0010) [2023-12-27 17:43:12,964][20381] Updated weights for policy 1, policy_version 1741406 (0.0011) [2023-12-27 17:43:13,528][20381] Updated weights for policy 1, policy_version 1741416 (0.0011) [2023-12-27 17:43:13,564][20347] Updated weights for policy 0, policy_version 1742855 (0.0007) [2023-12-27 17:43:13,582][20381] Updated weights for policy 1, policy_version 1741426 (0.0010) [2023-12-27 17:43:13,619][20347] Updated weights for policy 0, policy_version 1742865 (0.0006) [2023-12-27 17:43:13,633][20381] Updated weights for policy 1, policy_version 1741436 (0.0009) [2023-12-27 17:43:13,665][20347] Updated weights for policy 0, policy_version 1742875 (0.0006) [2023-12-27 17:43:14,316][20347] Updated weights for policy 0, policy_version 1742885 (0.0006) [2023-12-27 17:43:14,318][20381] Updated weights for policy 1, policy_version 1741446 (0.0011) [2023-12-27 17:43:14,370][20347] Updated weights for policy 0, policy_version 1742895 (0.0005) [2023-12-27 17:43:14,372][20381] Updated weights for policy 1, policy_version 1741456 (0.0010) [2023-12-27 17:43:14,421][20347] Updated weights for policy 0, policy_version 1742905 (0.0005) [2023-12-27 17:43:14,430][20381] Updated weights for policy 1, policy_version 1741466 (0.0010) [2023-12-27 17:43:15,027][20347] Updated weights for policy 0, policy_version 1742915 (0.0006) [2023-12-27 17:43:15,091][20347] Updated weights for policy 0, policy_version 1742925 (0.0007) [2023-12-27 17:43:15,126][20381] Updated weights for policy 1, policy_version 1741476 (0.0009) [2023-12-27 17:43:15,149][20347] Updated weights for policy 0, policy_version 1742935 (0.0006) [2023-12-27 17:43:15,189][20381] Updated weights for policy 1, policy_version 1741486 (0.0006) [2023-12-27 17:43:15,245][20381] Updated weights for policy 1, policy_version 1741496 (0.0007) [2023-12-27 17:43:15,779][20347] Updated weights for policy 0, policy_version 1742945 (0.0006) [2023-12-27 17:43:15,829][20347] Updated weights for policy 0, policy_version 1742955 (0.0005) [2023-12-27 17:43:15,885][20347] Updated weights for policy 0, policy_version 1742965 (0.0005) [2023-12-27 17:43:15,946][20347] Updated weights for policy 0, policy_version 1742975 (0.0005) [2023-12-27 17:43:15,947][20381] Updated weights for policy 1, policy_version 1741506 (0.0008) [2023-12-27 17:43:15,996][20381] Updated weights for policy 1, policy_version 1741516 (0.0010) [2023-12-27 17:43:16,047][20381] Updated weights for policy 1, policy_version 1741526 (0.0009) [2023-12-27 17:43:16,102][20381] Updated weights for policy 1, policy_version 1741536 (0.0008) [2023-12-27 17:43:16,573][20347] Updated weights for policy 0, policy_version 1742985 (0.0005) [2023-12-27 17:43:16,634][20347] Updated weights for policy 0, policy_version 1742995 (0.0005) [2023-12-27 17:43:16,682][20347] Updated weights for policy 0, policy_version 1743005 (0.0005) [2023-12-27 17:43:16,738][20381] Updated weights for policy 1, policy_version 1741546 (0.0009) [2023-12-27 17:43:16,785][20381] Updated weights for policy 1, policy_version 1741556 (0.0009) [2023-12-27 17:43:16,842][20381] Updated weights for policy 1, policy_version 1741566 (0.0009) [2023-12-27 17:43:17,294][20347] Updated weights for policy 0, policy_version 1743015 (0.0007) [2023-12-27 17:43:17,352][20347] Updated weights for policy 0, policy_version 1743025 (0.0007) [2023-12-27 17:43:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 892182528. Throughput: 0: 10268.6, 1: 10148.0. Samples: 892197928. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:17,363][19170] Avg episode reward: [(0, '970.730'), (1, '1000.000')] [2023-12-27 17:43:17,411][20347] Updated weights for policy 0, policy_version 1743035 (0.0007) [2023-12-27 17:43:17,549][20381] Updated weights for policy 1, policy_version 1741576 (0.0010) [2023-12-27 17:43:17,603][20381] Updated weights for policy 1, policy_version 1741586 (0.0010) [2023-12-27 17:43:17,665][20381] Updated weights for policy 1, policy_version 1741596 (0.0010) [2023-12-27 17:43:18,068][20347] Updated weights for policy 0, policy_version 1743045 (0.0007) [2023-12-27 17:43:18,127][20347] Updated weights for policy 0, policy_version 1743055 (0.0007) [2023-12-27 17:43:18,190][20347] Updated weights for policy 0, policy_version 1743065 (0.0008) [2023-12-27 17:43:18,307][20381] Updated weights for policy 1, policy_version 1741606 (0.0007) [2023-12-27 17:43:18,375][20381] Updated weights for policy 1, policy_version 1741616 (0.0008) [2023-12-27 17:43:18,438][20381] Updated weights for policy 1, policy_version 1741626 (0.0008) [2023-12-27 17:43:18,916][20347] Updated weights for policy 0, policy_version 1743075 (0.0008) [2023-12-27 17:43:18,970][20347] Updated weights for policy 0, policy_version 1743085 (0.0007) [2023-12-27 17:43:19,035][20347] Updated weights for policy 0, policy_version 1743095 (0.0008) [2023-12-27 17:43:19,071][20381] Updated weights for policy 1, policy_version 1741636 (0.0007) [2023-12-27 17:43:19,121][20381] Updated weights for policy 1, policy_version 1741646 (0.0007) [2023-12-27 17:43:19,176][20381] Updated weights for policy 1, policy_version 1741656 (0.0008) [2023-12-27 17:43:19,733][20347] Updated weights for policy 0, policy_version 1743105 (0.0006) [2023-12-27 17:43:19,785][20347] Updated weights for policy 0, policy_version 1743115 (0.0007) [2023-12-27 17:43:19,847][20347] Updated weights for policy 0, policy_version 1743125 (0.0008) [2023-12-27 17:43:19,890][20381] Updated weights for policy 1, policy_version 1741666 (0.0007) [2023-12-27 17:43:19,903][20347] Updated weights for policy 0, policy_version 1743135 (0.0008) [2023-12-27 17:43:19,955][20381] Updated weights for policy 1, policy_version 1741676 (0.0006) [2023-12-27 17:43:20,014][20381] Updated weights for policy 1, policy_version 1741686 (0.0007) [2023-12-27 17:43:20,075][20381] Updated weights for policy 1, policy_version 1741696 (0.0006) [2023-12-27 17:43:20,493][20347] Updated weights for policy 0, policy_version 1743145 (0.0006) [2023-12-27 17:43:20,539][20347] Updated weights for policy 0, policy_version 1743155 (0.0005) [2023-12-27 17:43:20,595][20347] Updated weights for policy 0, policy_version 1743165 (0.0007) [2023-12-27 17:43:20,816][20381] Updated weights for policy 1, policy_version 1741706 (0.0006) [2023-12-27 17:43:20,884][20381] Updated weights for policy 1, policy_version 1741716 (0.0006) [2023-12-27 17:43:20,952][20381] Updated weights for policy 1, policy_version 1741726 (0.0006) [2023-12-27 17:43:21,325][20347] Updated weights for policy 0, policy_version 1743175 (0.0007) [2023-12-27 17:43:21,394][20347] Updated weights for policy 0, policy_version 1743185 (0.0008) [2023-12-27 17:43:21,458][20347] Updated weights for policy 0, policy_version 1743195 (0.0007) [2023-12-27 17:43:21,684][20381] Updated weights for policy 1, policy_version 1741736 (0.0008) [2023-12-27 17:43:21,750][20381] Updated weights for policy 1, policy_version 1741746 (0.0008) [2023-12-27 17:43:21,813][20381] Updated weights for policy 1, policy_version 1741756 (0.0008) [2023-12-27 17:43:22,183][20347] Updated weights for policy 0, policy_version 1743205 (0.0008) [2023-12-27 17:43:22,242][20347] Updated weights for policy 0, policy_version 1743215 (0.0008) [2023-12-27 17:43:22,305][20347] Updated weights for policy 0, policy_version 1743225 (0.0008) [2023-12-27 17:43:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 892289024. Throughput: 0: 10224.6, 1: 10149.6. Samples: 892257988. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:22,363][19170] Avg episode reward: [(0, '960.920'), (1, '1000.000')] [2023-12-27 17:43:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001741760_445939712.pth... [2023-12-27 17:43:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001743232_446349312.pth... [2023-12-27 17:43:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001740576_445636608.pth [2023-12-27 17:43:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001742016_446038016.pth [2023-12-27 17:43:22,546][20381] Updated weights for policy 1, policy_version 1741766 (0.0008) [2023-12-27 17:43:22,601][20381] Updated weights for policy 1, policy_version 1741776 (0.0008) [2023-12-27 17:43:22,650][20381] Updated weights for policy 1, policy_version 1741786 (0.0008) [2023-12-27 17:43:23,053][20347] Updated weights for policy 0, policy_version 1743235 (0.0008) [2023-12-27 17:43:23,098][20347] Updated weights for policy 0, policy_version 1743245 (0.0008) [2023-12-27 17:43:23,145][20347] Updated weights for policy 0, policy_version 1743255 (0.0007) [2023-12-27 17:43:23,315][20381] Updated weights for policy 1, policy_version 1741796 (0.0007) [2023-12-27 17:43:23,378][20381] Updated weights for policy 1, policy_version 1741806 (0.0005) [2023-12-27 17:43:23,431][20381] Updated weights for policy 1, policy_version 1741816 (0.0006) [2023-12-27 17:43:23,829][20347] Updated weights for policy 0, policy_version 1743265 (0.0006) [2023-12-27 17:43:23,882][20347] Updated weights for policy 0, policy_version 1743275 (0.0005) [2023-12-27 17:43:23,932][20347] Updated weights for policy 0, policy_version 1743285 (0.0008) [2023-12-27 17:43:23,975][20381] Updated weights for policy 1, policy_version 1741826 (0.0010) [2023-12-27 17:43:23,982][20347] Updated weights for policy 0, policy_version 1743295 (0.0008) [2023-12-27 17:43:24,024][20381] Updated weights for policy 1, policy_version 1741836 (0.0010) [2023-12-27 17:43:24,079][20381] Updated weights for policy 1, policy_version 1741846 (0.0010) [2023-12-27 17:43:24,138][20381] Updated weights for policy 1, policy_version 1741856 (0.0010) [2023-12-27 17:43:24,664][20347] Updated weights for policy 0, policy_version 1743305 (0.0007) [2023-12-27 17:43:24,711][20347] Updated weights for policy 0, policy_version 1743315 (0.0008) [2023-12-27 17:43:24,773][20347] Updated weights for policy 0, policy_version 1743325 (0.0008) [2023-12-27 17:43:24,809][20381] Updated weights for policy 1, policy_version 1741866 (0.0008) [2023-12-27 17:43:24,854][20381] Updated weights for policy 1, policy_version 1741876 (0.0005) [2023-12-27 17:43:24,902][20381] Updated weights for policy 1, policy_version 1741886 (0.0005) [2023-12-27 17:43:25,430][20381] Updated weights for policy 1, policy_version 1741896 (0.0005) [2023-12-27 17:43:25,488][20381] Updated weights for policy 1, policy_version 1741906 (0.0011) [2023-12-27 17:43:25,499][20347] Updated weights for policy 0, policy_version 1743335 (0.0008) [2023-12-27 17:43:25,543][20381] Updated weights for policy 1, policy_version 1741916 (0.0010) [2023-12-27 17:43:25,554][20347] Updated weights for policy 0, policy_version 1743345 (0.0005) [2023-12-27 17:43:25,604][20347] Updated weights for policy 0, policy_version 1743355 (0.0010) [2023-12-27 17:43:26,168][20381] Updated weights for policy 1, policy_version 1741926 (0.0011) [2023-12-27 17:43:26,223][20381] Updated weights for policy 1, policy_version 1741936 (0.0009) [2023-12-27 17:43:26,272][20381] Updated weights for policy 1, policy_version 1741946 (0.0005) [2023-12-27 17:43:26,281][20347] Updated weights for policy 0, policy_version 1743365 (0.0010) [2023-12-27 17:43:26,345][20347] Updated weights for policy 0, policy_version 1743375 (0.0010) [2023-12-27 17:43:26,410][20347] Updated weights for policy 0, policy_version 1743385 (0.0011) [2023-12-27 17:43:26,851][20381] Updated weights for policy 1, policy_version 1741956 (0.0007) [2023-12-27 17:43:26,898][20381] Updated weights for policy 1, policy_version 1741966 (0.0010) [2023-12-27 17:43:26,943][20381] Updated weights for policy 1, policy_version 1741976 (0.0010) [2023-12-27 17:43:27,053][20347] Updated weights for policy 0, policy_version 1743395 (0.0010) [2023-12-27 17:43:27,106][20347] Updated weights for policy 0, policy_version 1743405 (0.0010) [2023-12-27 17:43:27,159][20347] Updated weights for policy 0, policy_version 1743415 (0.0010) [2023-12-27 17:43:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 892395520. Throughput: 0: 10194.4, 1: 10219.0. Samples: 892382548. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:27,363][19170] Avg episode reward: [(0, '931.420'), (1, '1000.000')] [2023-12-27 17:43:27,597][20381] Updated weights for policy 1, policy_version 1741986 (0.0008) [2023-12-27 17:43:27,649][20381] Updated weights for policy 1, policy_version 1741996 (0.0008) [2023-12-27 17:43:27,703][20381] Updated weights for policy 1, policy_version 1742006 (0.0009) [2023-12-27 17:43:27,760][20381] Updated weights for policy 1, policy_version 1742016 (0.0008) [2023-12-27 17:43:27,819][20347] Updated weights for policy 0, policy_version 1743425 (0.0010) [2023-12-27 17:43:27,866][20347] Updated weights for policy 0, policy_version 1743435 (0.0010) [2023-12-27 17:43:27,920][20347] Updated weights for policy 0, policy_version 1743445 (0.0006) [2023-12-27 17:43:27,969][20347] Updated weights for policy 0, policy_version 1743455 (0.0008) [2023-12-27 17:43:28,389][20381] Updated weights for policy 1, policy_version 1742026 (0.0010) [2023-12-27 17:43:28,444][20381] Updated weights for policy 1, policy_version 1742036 (0.0010) [2023-12-27 17:43:28,503][20381] Updated weights for policy 1, policy_version 1742046 (0.0011) [2023-12-27 17:43:28,652][20347] Updated weights for policy 0, policy_version 1743465 (0.0006) [2023-12-27 17:43:28,713][20347] Updated weights for policy 0, policy_version 1743475 (0.0005) [2023-12-27 17:43:28,770][20347] Updated weights for policy 0, policy_version 1743485 (0.0005) [2023-12-27 17:43:29,144][20381] Updated weights for policy 1, policy_version 1742056 (0.0006) [2023-12-27 17:43:29,209][20381] Updated weights for policy 1, policy_version 1742066 (0.0006) [2023-12-27 17:43:29,272][20381] Updated weights for policy 1, policy_version 1742076 (0.0011) [2023-12-27 17:43:29,313][20347] Updated weights for policy 0, policy_version 1743495 (0.0008) [2023-12-27 17:43:29,380][20347] Updated weights for policy 0, policy_version 1743505 (0.0008) [2023-12-27 17:43:29,433][20347] Updated weights for policy 0, policy_version 1743515 (0.0008) [2023-12-27 17:43:29,926][20381] Updated weights for policy 1, policy_version 1742086 (0.0010) [2023-12-27 17:43:29,978][20381] Updated weights for policy 1, policy_version 1742096 (0.0010) [2023-12-27 17:43:30,031][20381] Updated weights for policy 1, policy_version 1742106 (0.0010) [2023-12-27 17:43:30,167][20347] Updated weights for policy 0, policy_version 1743525 (0.0007) [2023-12-27 17:43:30,221][20347] Updated weights for policy 0, policy_version 1743535 (0.0008) [2023-12-27 17:43:30,276][20347] Updated weights for policy 0, policy_version 1743545 (0.0007) [2023-12-27 17:43:30,717][20381] Updated weights for policy 1, policy_version 1742116 (0.0010) [2023-12-27 17:43:30,777][20381] Updated weights for policy 1, policy_version 1742126 (0.0010) [2023-12-27 17:43:30,840][20381] Updated weights for policy 1, policy_version 1742136 (0.0010) [2023-12-27 17:43:30,894][20347] Updated weights for policy 0, policy_version 1743555 (0.0007) [2023-12-27 17:43:30,950][20347] Updated weights for policy 0, policy_version 1743565 (0.0007) [2023-12-27 17:43:31,001][20347] Updated weights for policy 0, policy_version 1743575 (0.0010) [2023-12-27 17:43:31,521][20381] Updated weights for policy 1, policy_version 1742146 (0.0010) [2023-12-27 17:43:31,581][20381] Updated weights for policy 1, policy_version 1742156 (0.0009) [2023-12-27 17:43:31,645][20381] Updated weights for policy 1, policy_version 1742166 (0.0009) [2023-12-27 17:43:31,672][20347] Updated weights for policy 0, policy_version 1743585 (0.0009) [2023-12-27 17:43:31,706][20381] Updated weights for policy 1, policy_version 1742176 (0.0008) [2023-12-27 17:43:31,730][20347] Updated weights for policy 0, policy_version 1743595 (0.0007) [2023-12-27 17:43:31,793][20347] Updated weights for policy 0, policy_version 1743605 (0.0008) [2023-12-27 17:43:31,851][20347] Updated weights for policy 0, policy_version 1743615 (0.0006) [2023-12-27 17:43:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 892493824. Throughput: 0: 10224.8, 1: 10277.8. Samples: 892508988. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:32,363][19170] Avg episode reward: [(0, '931.520'), (1, '1000.000')] [2023-12-27 17:43:32,419][20381] Updated weights for policy 1, policy_version 1742186 (0.0009) [2023-12-27 17:43:32,472][20381] Updated weights for policy 1, policy_version 1742196 (0.0007) [2023-12-27 17:43:32,524][20381] Updated weights for policy 1, policy_version 1742206 (0.0008) [2023-12-27 17:43:32,532][20347] Updated weights for policy 0, policy_version 1743625 (0.0006) [2023-12-27 17:43:32,590][20347] Updated weights for policy 0, policy_version 1743635 (0.0006) [2023-12-27 17:43:32,646][20347] Updated weights for policy 0, policy_version 1743645 (0.0006) [2023-12-27 17:43:33,190][20381] Updated weights for policy 1, policy_version 1742216 (0.0008) [2023-12-27 17:43:33,249][20381] Updated weights for policy 1, policy_version 1742226 (0.0008) [2023-12-27 17:43:33,272][20347] Updated weights for policy 0, policy_version 1743655 (0.0006) [2023-12-27 17:43:33,298][20381] Updated weights for policy 1, policy_version 1742236 (0.0007) [2023-12-27 17:43:33,330][20347] Updated weights for policy 0, policy_version 1743665 (0.0005) [2023-12-27 17:43:33,386][20347] Updated weights for policy 0, policy_version 1743675 (0.0005) [2023-12-27 17:43:33,947][20347] Updated weights for policy 0, policy_version 1743685 (0.0005) [2023-12-27 17:43:33,998][20347] Updated weights for policy 0, policy_version 1743695 (0.0005) [2023-12-27 17:43:34,010][20381] Updated weights for policy 1, policy_version 1742246 (0.0008) [2023-12-27 17:43:34,046][20347] Updated weights for policy 0, policy_version 1743705 (0.0005) [2023-12-27 17:43:34,065][20381] Updated weights for policy 1, policy_version 1742256 (0.0008) [2023-12-27 17:43:34,126][20381] Updated weights for policy 1, policy_version 1742266 (0.0008) [2023-12-27 17:43:34,711][20347] Updated weights for policy 0, policy_version 1743715 (0.0007) [2023-12-27 17:43:34,764][20347] Updated weights for policy 0, policy_version 1743725 (0.0010) [2023-12-27 17:43:34,816][20347] Updated weights for policy 0, policy_version 1743735 (0.0010) [2023-12-27 17:43:34,888][20381] Updated weights for policy 1, policy_version 1742276 (0.0007) [2023-12-27 17:43:34,946][20381] Updated weights for policy 1, policy_version 1742286 (0.0007) [2023-12-27 17:43:34,994][20381] Updated weights for policy 1, policy_version 1742296 (0.0008) [2023-12-27 17:43:35,498][20347] Updated weights for policy 0, policy_version 1743745 (0.0010) [2023-12-27 17:43:35,550][20347] Updated weights for policy 0, policy_version 1743755 (0.0010) [2023-12-27 17:43:35,608][20347] Updated weights for policy 0, policy_version 1743765 (0.0011) [2023-12-27 17:43:35,668][20381] Updated weights for policy 1, policy_version 1742306 (0.0006) [2023-12-27 17:43:35,669][20347] Updated weights for policy 0, policy_version 1743775 (0.0010) [2023-12-27 17:43:35,730][20381] Updated weights for policy 1, policy_version 1742316 (0.0007) [2023-12-27 17:43:35,786][20381] Updated weights for policy 1, policy_version 1742326 (0.0008) [2023-12-27 17:43:35,838][20381] Updated weights for policy 1, policy_version 1742336 (0.0007) [2023-12-27 17:43:36,347][20347] Updated weights for policy 0, policy_version 1743785 (0.0010) [2023-12-27 17:43:36,409][20347] Updated weights for policy 0, policy_version 1743795 (0.0008) [2023-12-27 17:43:36,473][20347] Updated weights for policy 0, policy_version 1743805 (0.0011) [2023-12-27 17:43:36,575][20381] Updated weights for policy 1, policy_version 1742346 (0.0008) [2023-12-27 17:43:36,644][20381] Updated weights for policy 1, policy_version 1742356 (0.0008) [2023-12-27 17:43:36,707][20381] Updated weights for policy 1, policy_version 1742366 (0.0008) [2023-12-27 17:43:37,180][20347] Updated weights for policy 0, policy_version 1743815 (0.0008) [2023-12-27 17:43:37,241][20347] Updated weights for policy 0, policy_version 1743825 (0.0008) [2023-12-27 17:43:37,290][20347] Updated weights for policy 0, policy_version 1743835 (0.0007) [2023-12-27 17:43:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 892600320. Throughput: 0: 10261.5, 1: 10254.1. Samples: 892570804. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:37,363][19170] Avg episode reward: [(0, '951.150'), (1, '1000.000')] [2023-12-27 17:43:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001743840_446504960.pth... [2023-12-27 17:43:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001742624_446193664.pth [2023-12-27 17:43:37,402][20381] Updated weights for policy 1, policy_version 1742376 (0.0006) [2023-12-27 17:43:37,457][20381] Updated weights for policy 1, policy_version 1742386 (0.0005) [2023-12-27 17:43:37,512][20381] Updated weights for policy 1, policy_version 1742396 (0.0005) [2023-12-27 17:43:37,532][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001742400_446103552.pth... [2023-12-27 17:43:37,536][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001741184_445792256.pth [2023-12-27 17:43:37,917][20347] Updated weights for policy 0, policy_version 1743845 (0.0005) [2023-12-27 17:43:37,970][20347] Updated weights for policy 0, policy_version 1743855 (0.0006) [2023-12-27 17:43:38,020][20347] Updated weights for policy 0, policy_version 1743865 (0.0007) [2023-12-27 17:43:38,067][20381] Updated weights for policy 1, policy_version 1742406 (0.0007) [2023-12-27 17:43:38,122][20381] Updated weights for policy 1, policy_version 1742416 (0.0007) [2023-12-27 17:43:38,181][20381] Updated weights for policy 1, policy_version 1742426 (0.0008) [2023-12-27 17:43:38,662][20347] Updated weights for policy 0, policy_version 1743875 (0.0008) [2023-12-27 17:43:38,722][20347] Updated weights for policy 0, policy_version 1743885 (0.0007) [2023-12-27 17:43:38,780][20347] Updated weights for policy 0, policy_version 1743895 (0.0008) [2023-12-27 17:43:38,832][20381] Updated weights for policy 1, policy_version 1742436 (0.0006) [2023-12-27 17:43:38,895][20381] Updated weights for policy 1, policy_version 1742446 (0.0007) [2023-12-27 17:43:38,948][20381] Updated weights for policy 1, policy_version 1742456 (0.0008) [2023-12-27 17:43:39,550][20347] Updated weights for policy 0, policy_version 1743905 (0.0008) [2023-12-27 17:43:39,613][20347] Updated weights for policy 0, policy_version 1743915 (0.0006) [2023-12-27 17:43:39,639][20381] Updated weights for policy 1, policy_version 1742466 (0.0008) [2023-12-27 17:43:39,673][20347] Updated weights for policy 0, policy_version 1743925 (0.0007) [2023-12-27 17:43:39,703][20381] Updated weights for policy 1, policy_version 1742476 (0.0008) [2023-12-27 17:43:39,719][20347] Updated weights for policy 0, policy_version 1743935 (0.0007) [2023-12-27 17:43:39,759][20381] Updated weights for policy 1, policy_version 1742486 (0.0008) [2023-12-27 17:43:39,812][20381] Updated weights for policy 1, policy_version 1742496 (0.0008) [2023-12-27 17:43:40,462][20347] Updated weights for policy 0, policy_version 1743945 (0.0007) [2023-12-27 17:43:40,493][20381] Updated weights for policy 1, policy_version 1742506 (0.0005) [2023-12-27 17:43:40,515][20347] Updated weights for policy 0, policy_version 1743955 (0.0006) [2023-12-27 17:43:40,541][20381] Updated weights for policy 1, policy_version 1742516 (0.0005) [2023-12-27 17:43:40,565][20347] Updated weights for policy 0, policy_version 1743965 (0.0007) [2023-12-27 17:43:40,598][20381] Updated weights for policy 1, policy_version 1742526 (0.0005) [2023-12-27 17:43:41,238][20381] Updated weights for policy 1, policy_version 1742536 (0.0007) [2023-12-27 17:43:41,246][20347] Updated weights for policy 0, policy_version 1743975 (0.0008) [2023-12-27 17:43:41,300][20381] Updated weights for policy 1, policy_version 1742546 (0.0007) [2023-12-27 17:43:41,300][20347] Updated weights for policy 0, policy_version 1743985 (0.0008) [2023-12-27 17:43:41,358][20381] Updated weights for policy 1, policy_version 1742556 (0.0007) [2023-12-27 17:43:41,363][20347] Updated weights for policy 0, policy_version 1743995 (0.0008) [2023-12-27 17:43:42,043][20381] Updated weights for policy 1, policy_version 1742566 (0.0008) [2023-12-27 17:43:42,092][20381] Updated weights for policy 1, policy_version 1742576 (0.0007) [2023-12-27 17:43:42,146][20347] Updated weights for policy 0, policy_version 1744005 (0.0008) [2023-12-27 17:43:42,149][20381] Updated weights for policy 1, policy_version 1742586 (0.0008) [2023-12-27 17:43:42,200][20347] Updated weights for policy 0, policy_version 1744015 (0.0007) [2023-12-27 17:43:42,251][20347] Updated weights for policy 0, policy_version 1744025 (0.0007) [2023-12-27 17:43:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 892706816. Throughput: 0: 10262.0, 1: 10263.7. Samples: 892693680. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:42,363][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 17:43:42,872][20381] Updated weights for policy 1, policy_version 1742596 (0.0010) [2023-12-27 17:43:42,934][20381] Updated weights for policy 1, policy_version 1742606 (0.0007) [2023-12-27 17:43:42,992][20381] Updated weights for policy 1, policy_version 1742616 (0.0011) [2023-12-27 17:43:42,998][20347] Updated weights for policy 0, policy_version 1744035 (0.0006) [2023-12-27 17:43:43,050][20347] Updated weights for policy 0, policy_version 1744045 (0.0005) [2023-12-27 17:43:43,098][20347] Updated weights for policy 0, policy_version 1744055 (0.0005) [2023-12-27 17:43:43,650][20381] Updated weights for policy 1, policy_version 1742626 (0.0010) [2023-12-27 17:43:43,709][20381] Updated weights for policy 1, policy_version 1742636 (0.0010) [2023-12-27 17:43:43,723][20347] Updated weights for policy 0, policy_version 1744065 (0.0010) [2023-12-27 17:43:43,764][20381] Updated weights for policy 1, policy_version 1742646 (0.0010) [2023-12-27 17:43:43,771][20347] Updated weights for policy 0, policy_version 1744075 (0.0006) [2023-12-27 17:43:43,819][20381] Updated weights for policy 1, policy_version 1742656 (0.0010) [2023-12-27 17:43:43,822][20347] Updated weights for policy 0, policy_version 1744085 (0.0007) [2023-12-27 17:43:43,868][20347] Updated weights for policy 0, policy_version 1744095 (0.0006) [2023-12-27 17:43:44,461][20381] Updated weights for policy 1, policy_version 1742666 (0.0005) [2023-12-27 17:43:44,515][20381] Updated weights for policy 1, policy_version 1742676 (0.0005) [2023-12-27 17:43:44,552][20347] Updated weights for policy 0, policy_version 1744105 (0.0009) [2023-12-27 17:43:44,577][20381] Updated weights for policy 1, policy_version 1742686 (0.0006) [2023-12-27 17:43:44,607][20347] Updated weights for policy 0, policy_version 1744115 (0.0010) [2023-12-27 17:43:44,662][20347] Updated weights for policy 0, policy_version 1744125 (0.0009) [2023-12-27 17:43:45,198][20381] Updated weights for policy 1, policy_version 1742696 (0.0007) [2023-12-27 17:43:45,257][20381] Updated weights for policy 1, policy_version 1742706 (0.0007) [2023-12-27 17:43:45,308][20381] Updated weights for policy 1, policy_version 1742716 (0.0007) [2023-12-27 17:43:45,332][20347] Updated weights for policy 0, policy_version 1744135 (0.0009) [2023-12-27 17:43:45,387][20347] Updated weights for policy 0, policy_version 1744145 (0.0010) [2023-12-27 17:43:45,438][20347] Updated weights for policy 0, policy_version 1744155 (0.0010) [2023-12-27 17:43:45,994][20381] Updated weights for policy 1, policy_version 1742726 (0.0007) [2023-12-27 17:43:46,048][20381] Updated weights for policy 1, policy_version 1742736 (0.0007) [2023-12-27 17:43:46,095][20381] Updated weights for policy 1, policy_version 1742746 (0.0007) [2023-12-27 17:43:46,132][20347] Updated weights for policy 0, policy_version 1744165 (0.0010) [2023-12-27 17:43:46,177][20347] Updated weights for policy 0, policy_version 1744175 (0.0010) [2023-12-27 17:43:46,226][20347] Updated weights for policy 0, policy_version 1744185 (0.0010) [2023-12-27 17:43:46,796][20381] Updated weights for policy 1, policy_version 1742756 (0.0006) [2023-12-27 17:43:46,847][20381] Updated weights for policy 1, policy_version 1742766 (0.0005) [2023-12-27 17:43:46,899][20381] Updated weights for policy 1, policy_version 1742776 (0.0005) [2023-12-27 17:43:46,921][20347] Updated weights for policy 0, policy_version 1744195 (0.0010) [2023-12-27 17:43:46,973][20347] Updated weights for policy 0, policy_version 1744205 (0.0010) [2023-12-27 17:43:47,035][20347] Updated weights for policy 0, policy_version 1744215 (0.0010) [2023-12-27 17:43:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 892805120. Throughput: 0: 10286.1, 1: 10280.2. Samples: 892815768. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:47,363][19170] Avg episode reward: [(0, '980.470'), (1, '1000.000')] [2023-12-27 17:43:47,537][20381] Updated weights for policy 1, policy_version 1742786 (0.0005) [2023-12-27 17:43:47,598][20381] Updated weights for policy 1, policy_version 1742796 (0.0010) [2023-12-27 17:43:47,653][20381] Updated weights for policy 1, policy_version 1742806 (0.0011) [2023-12-27 17:43:47,700][20381] Updated weights for policy 1, policy_version 1742816 (0.0010) [2023-12-27 17:43:47,720][20347] Updated weights for policy 0, policy_version 1744225 (0.0010) [2023-12-27 17:43:47,775][20347] Updated weights for policy 0, policy_version 1744235 (0.0010) [2023-12-27 17:43:47,835][20347] Updated weights for policy 0, policy_version 1744245 (0.0010) [2023-12-27 17:43:47,892][20347] Updated weights for policy 0, policy_version 1744255 (0.0010) [2023-12-27 17:43:48,388][20381] Updated weights for policy 1, policy_version 1742826 (0.0011) [2023-12-27 17:43:48,440][20381] Updated weights for policy 1, policy_version 1742836 (0.0008) [2023-12-27 17:43:48,501][20381] Updated weights for policy 1, policy_version 1742846 (0.0006) [2023-12-27 17:43:48,576][20347] Updated weights for policy 0, policy_version 1744265 (0.0010) [2023-12-27 17:43:48,624][20347] Updated weights for policy 0, policy_version 1744275 (0.0010) [2023-12-27 17:43:48,682][20347] Updated weights for policy 0, policy_version 1744285 (0.0010) [2023-12-27 17:43:49,129][20381] Updated weights for policy 1, policy_version 1742856 (0.0005) [2023-12-27 17:43:49,177][20381] Updated weights for policy 1, policy_version 1742866 (0.0006) [2023-12-27 17:43:49,247][20381] Updated weights for policy 1, policy_version 1742876 (0.0007) [2023-12-27 17:43:49,410][20347] Updated weights for policy 0, policy_version 1744295 (0.0008) [2023-12-27 17:43:49,476][20347] Updated weights for policy 0, policy_version 1744305 (0.0008) [2023-12-27 17:43:49,537][20347] Updated weights for policy 0, policy_version 1744315 (0.0007) [2023-12-27 17:43:49,862][20381] Updated weights for policy 1, policy_version 1742886 (0.0009) [2023-12-27 17:43:49,931][20381] Updated weights for policy 1, policy_version 1742896 (0.0006) [2023-12-27 17:43:49,984][20381] Updated weights for policy 1, policy_version 1742906 (0.0007) [2023-12-27 17:43:50,284][20347] Updated weights for policy 0, policy_version 1744325 (0.0008) [2023-12-27 17:43:50,338][20347] Updated weights for policy 0, policy_version 1744335 (0.0008) [2023-12-27 17:43:50,391][20347] Updated weights for policy 0, policy_version 1744345 (0.0008) [2023-12-27 17:43:50,555][20381] Updated weights for policy 1, policy_version 1742916 (0.0007) [2023-12-27 17:43:50,612][20381] Updated weights for policy 1, policy_version 1742926 (0.0009) [2023-12-27 17:43:50,663][20381] Updated weights for policy 1, policy_version 1742936 (0.0009) [2023-12-27 17:43:51,082][20347] Updated weights for policy 0, policy_version 1744355 (0.0008) [2023-12-27 17:43:51,141][20347] Updated weights for policy 0, policy_version 1744365 (0.0008) [2023-12-27 17:43:51,210][20347] Updated weights for policy 0, policy_version 1744375 (0.0008) [2023-12-27 17:43:51,331][20381] Updated weights for policy 1, policy_version 1742946 (0.0007) [2023-12-27 17:43:51,404][20381] Updated weights for policy 1, policy_version 1742956 (0.0008) [2023-12-27 17:43:51,452][20381] Updated weights for policy 1, policy_version 1742966 (0.0008) [2023-12-27 17:43:51,501][20381] Updated weights for policy 1, policy_version 1742976 (0.0008) [2023-12-27 17:43:51,916][20347] Updated weights for policy 0, policy_version 1744385 (0.0008) [2023-12-27 17:43:51,979][20347] Updated weights for policy 0, policy_version 1744395 (0.0006) [2023-12-27 17:43:52,032][20347] Updated weights for policy 0, policy_version 1744405 (0.0005) [2023-12-27 17:43:52,087][20347] Updated weights for policy 0, policy_version 1744415 (0.0005) [2023-12-27 17:43:52,194][20381] Updated weights for policy 1, policy_version 1742986 (0.0007) [2023-12-27 17:43:52,242][20381] Updated weights for policy 1, policy_version 1742996 (0.0006) [2023-12-27 17:43:52,301][20381] Updated weights for policy 1, policy_version 1743006 (0.0007) [2023-12-27 17:43:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 892911616. Throughput: 0: 10266.8, 1: 10285.1. Samples: 892877072. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:52,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 17:43:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001743008_446259200.pth... [2023-12-27 17:43:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001744416_446652416.pth... [2023-12-27 17:43:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001741760_445939712.pth [2023-12-27 17:43:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001743232_446349312.pth [2023-12-27 17:43:52,755][20347] Updated weights for policy 0, policy_version 1744425 (0.0008) [2023-12-27 17:43:52,814][20347] Updated weights for policy 0, policy_version 1744435 (0.0010) [2023-12-27 17:43:52,875][20347] Updated weights for policy 0, policy_version 1744445 (0.0010) [2023-12-27 17:43:52,938][20381] Updated weights for policy 1, policy_version 1743016 (0.0007) [2023-12-27 17:43:52,993][20381] Updated weights for policy 1, policy_version 1743026 (0.0007) [2023-12-27 17:43:53,048][20381] Updated weights for policy 1, policy_version 1743036 (0.0007) [2023-12-27 17:43:53,520][20347] Updated weights for policy 0, policy_version 1744455 (0.0010) [2023-12-27 17:43:53,577][20347] Updated weights for policy 0, policy_version 1744465 (0.0006) [2023-12-27 17:43:53,630][20347] Updated weights for policy 0, policy_version 1744475 (0.0005) [2023-12-27 17:43:53,742][20381] Updated weights for policy 1, policy_version 1743046 (0.0008) [2023-12-27 17:43:53,790][20381] Updated weights for policy 1, policy_version 1743056 (0.0008) [2023-12-27 17:43:53,842][20381] Updated weights for policy 1, policy_version 1743066 (0.0008) [2023-12-27 17:43:54,205][20347] Updated weights for policy 0, policy_version 1744485 (0.0008) [2023-12-27 17:43:54,257][20347] Updated weights for policy 0, policy_version 1744495 (0.0010) [2023-12-27 17:43:54,308][20347] Updated weights for policy 0, policy_version 1744505 (0.0010) [2023-12-27 17:43:54,544][20381] Updated weights for policy 1, policy_version 1743076 (0.0008) [2023-12-27 17:43:54,601][20381] Updated weights for policy 1, policy_version 1743086 (0.0007) [2023-12-27 17:43:54,668][20381] Updated weights for policy 1, policy_version 1743096 (0.0007) [2023-12-27 17:43:54,984][20347] Updated weights for policy 0, policy_version 1744515 (0.0009) [2023-12-27 17:43:55,040][20347] Updated weights for policy 0, policy_version 1744525 (0.0005) [2023-12-27 17:43:55,097][20347] Updated weights for policy 0, policy_version 1744535 (0.0006) [2023-12-27 17:43:55,340][20381] Updated weights for policy 1, policy_version 1743106 (0.0007) [2023-12-27 17:43:55,389][20381] Updated weights for policy 1, policy_version 1743116 (0.0007) [2023-12-27 17:43:55,449][20381] Updated weights for policy 1, policy_version 1743126 (0.0009) [2023-12-27 17:43:55,500][20381] Updated weights for policy 1, policy_version 1743136 (0.0007) [2023-12-27 17:43:55,703][20347] Updated weights for policy 0, policy_version 1744545 (0.0010) [2023-12-27 17:43:55,754][20347] Updated weights for policy 0, policy_version 1744555 (0.0005) [2023-12-27 17:43:55,809][20347] Updated weights for policy 0, policy_version 1744565 (0.0007) [2023-12-27 17:43:55,857][20347] Updated weights for policy 0, policy_version 1744575 (0.0010) [2023-12-27 17:43:56,161][20381] Updated weights for policy 1, policy_version 1743146 (0.0006) [2023-12-27 17:43:56,207][20381] Updated weights for policy 1, policy_version 1743156 (0.0006) [2023-12-27 17:43:56,252][20381] Updated weights for policy 1, policy_version 1743166 (0.0005) [2023-12-27 17:43:56,538][20347] Updated weights for policy 0, policy_version 1744585 (0.0010) [2023-12-27 17:43:56,601][20347] Updated weights for policy 0, policy_version 1744595 (0.0010) [2023-12-27 17:43:56,668][20347] Updated weights for policy 0, policy_version 1744605 (0.0010) [2023-12-27 17:43:56,808][20381] Updated weights for policy 1, policy_version 1743176 (0.0006) [2023-12-27 17:43:56,858][20381] Updated weights for policy 1, policy_version 1743186 (0.0010) [2023-12-27 17:43:56,908][20381] Updated weights for policy 1, policy_version 1743196 (0.0009) [2023-12-27 17:43:57,220][20347] Updated weights for policy 0, policy_version 1744615 (0.0009) [2023-12-27 17:43:57,267][20347] Updated weights for policy 0, policy_version 1744625 (0.0010) [2023-12-27 17:43:57,321][20347] Updated weights for policy 0, policy_version 1744635 (0.0010) [2023-12-27 17:43:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 893018112. Throughput: 0: 10270.1, 1: 10373.7. Samples: 893002624. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:43:57,363][19170] Avg episode reward: [(0, '980.450'), (1, '1000.000')] [2023-12-27 17:43:57,629][20381] Updated weights for policy 1, policy_version 1743206 (0.0006) [2023-12-27 17:43:57,683][20381] Updated weights for policy 1, policy_version 1743216 (0.0005) [2023-12-27 17:43:57,739][20381] Updated weights for policy 1, policy_version 1743226 (0.0007) [2023-12-27 17:43:57,960][20347] Updated weights for policy 0, policy_version 1744645 (0.0007) [2023-12-27 17:43:58,005][20347] Updated weights for policy 0, policy_version 1744655 (0.0008) [2023-12-27 17:43:58,063][20347] Updated weights for policy 0, policy_version 1744665 (0.0008) [2023-12-27 17:43:58,413][20381] Updated weights for policy 1, policy_version 1743236 (0.0008) [2023-12-27 17:43:58,481][20381] Updated weights for policy 1, policy_version 1743246 (0.0007) [2023-12-27 17:43:58,539][20381] Updated weights for policy 1, policy_version 1743256 (0.0008) [2023-12-27 17:43:58,845][20347] Updated weights for policy 0, policy_version 1744675 (0.0007) [2023-12-27 17:43:58,911][20347] Updated weights for policy 0, policy_version 1744685 (0.0007) [2023-12-27 17:43:58,971][20347] Updated weights for policy 0, policy_version 1744695 (0.0007) [2023-12-27 17:43:59,392][20381] Updated weights for policy 1, policy_version 1743266 (0.0007) [2023-12-27 17:43:59,442][20381] Updated weights for policy 1, policy_version 1743276 (0.0006) [2023-12-27 17:43:59,490][20381] Updated weights for policy 1, policy_version 1743286 (0.0010) [2023-12-27 17:43:59,545][20381] Updated weights for policy 1, policy_version 1743296 (0.0010) [2023-12-27 17:43:59,661][20347] Updated weights for policy 0, policy_version 1744705 (0.0007) [2023-12-27 17:43:59,723][20347] Updated weights for policy 0, policy_version 1744715 (0.0008) [2023-12-27 17:43:59,779][20347] Updated weights for policy 0, policy_version 1744725 (0.0006) [2023-12-27 17:43:59,839][20347] Updated weights for policy 0, policy_version 1744735 (0.0008) [2023-12-27 17:44:00,184][20381] Updated weights for policy 1, policy_version 1743306 (0.0005) [2023-12-27 17:44:00,243][20381] Updated weights for policy 1, policy_version 1743316 (0.0006) [2023-12-27 17:44:00,296][20381] Updated weights for policy 1, policy_version 1743326 (0.0010) [2023-12-27 17:44:00,469][20347] Updated weights for policy 0, policy_version 1744745 (0.0007) [2023-12-27 17:44:00,524][20347] Updated weights for policy 0, policy_version 1744755 (0.0005) [2023-12-27 17:44:00,579][20347] Updated weights for policy 0, policy_version 1744765 (0.0007) [2023-12-27 17:44:00,914][20381] Updated weights for policy 1, policy_version 1743336 (0.0006) [2023-12-27 17:44:00,991][20381] Updated weights for policy 1, policy_version 1743346 (0.0006) [2023-12-27 17:44:01,041][20381] Updated weights for policy 1, policy_version 1743356 (0.0008) [2023-12-27 17:44:01,282][20347] Updated weights for policy 0, policy_version 1744775 (0.0007) [2023-12-27 17:44:01,334][20347] Updated weights for policy 0, policy_version 1744785 (0.0008) [2023-12-27 17:44:01,399][20347] Updated weights for policy 0, policy_version 1744795 (0.0007) [2023-12-27 17:44:01,753][20381] Updated weights for policy 1, policy_version 1743366 (0.0008) [2023-12-27 17:44:01,799][20381] Updated weights for policy 1, policy_version 1743376 (0.0008) [2023-12-27 17:44:01,851][20381] Updated weights for policy 1, policy_version 1743386 (0.0007) [2023-12-27 17:44:02,001][20347] Updated weights for policy 0, policy_version 1744805 (0.0006) [2023-12-27 17:44:02,057][20347] Updated weights for policy 0, policy_version 1744815 (0.0005) [2023-12-27 17:44:02,117][20347] Updated weights for policy 0, policy_version 1744825 (0.0005) [2023-12-27 17:44:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 893116416. Throughput: 0: 10261.5, 1: 10371.3. Samples: 893126404. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:02,363][19170] Avg episode reward: [(0, '980.420'), (1, '1000.000')] [2023-12-27 17:44:02,590][20381] Updated weights for policy 1, policy_version 1743396 (0.0008) [2023-12-27 17:44:02,656][20381] Updated weights for policy 1, policy_version 1743406 (0.0007) [2023-12-27 17:44:02,716][20381] Updated weights for policy 1, policy_version 1743416 (0.0008) [2023-12-27 17:44:02,735][20347] Updated weights for policy 0, policy_version 1744835 (0.0006) [2023-12-27 17:44:02,796][20347] Updated weights for policy 0, policy_version 1744845 (0.0007) [2023-12-27 17:44:02,852][20347] Updated weights for policy 0, policy_version 1744855 (0.0007) [2023-12-27 17:44:03,374][20381] Updated weights for policy 1, policy_version 1743426 (0.0008) [2023-12-27 17:44:03,443][20381] Updated weights for policy 1, policy_version 1743436 (0.0008) [2023-12-27 17:44:03,505][20381] Updated weights for policy 1, policy_version 1743446 (0.0008) [2023-12-27 17:44:03,536][20347] Updated weights for policy 0, policy_version 1744865 (0.0007) [2023-12-27 17:44:03,568][20381] Updated weights for policy 1, policy_version 1743456 (0.0008) [2023-12-27 17:44:03,586][20347] Updated weights for policy 0, policy_version 1744875 (0.0007) [2023-12-27 17:44:03,642][20347] Updated weights for policy 0, policy_version 1744885 (0.0008) [2023-12-27 17:44:03,704][20347] Updated weights for policy 0, policy_version 1744895 (0.0010) [2023-12-27 17:44:04,258][20381] Updated weights for policy 1, policy_version 1743466 (0.0011) [2023-12-27 17:44:04,309][20381] Updated weights for policy 1, policy_version 1743476 (0.0010) [2023-12-27 17:44:04,363][20347] Updated weights for policy 0, policy_version 1744905 (0.0006) [2023-12-27 17:44:04,365][20381] Updated weights for policy 1, policy_version 1743486 (0.0010) [2023-12-27 17:44:04,422][20347] Updated weights for policy 0, policy_version 1744915 (0.0007) [2023-12-27 17:44:04,479][20347] Updated weights for policy 0, policy_version 1744925 (0.0008) [2023-12-27 17:44:05,040][20381] Updated weights for policy 1, policy_version 1743496 (0.0007) [2023-12-27 17:44:05,091][20381] Updated weights for policy 1, policy_version 1743506 (0.0007) [2023-12-27 17:44:05,143][20381] Updated weights for policy 1, policy_version 1743516 (0.0006) [2023-12-27 17:44:05,198][20347] Updated weights for policy 0, policy_version 1744935 (0.0006) [2023-12-27 17:44:05,257][20347] Updated weights for policy 0, policy_version 1744945 (0.0005) [2023-12-27 17:44:05,313][20347] Updated weights for policy 0, policy_version 1744955 (0.0005) [2023-12-27 17:44:05,810][20381] Updated weights for policy 1, policy_version 1743526 (0.0006) [2023-12-27 17:44:05,828][20347] Updated weights for policy 0, policy_version 1744965 (0.0005) [2023-12-27 17:44:05,868][20381] Updated weights for policy 1, policy_version 1743536 (0.0006) [2023-12-27 17:44:05,892][20347] Updated weights for policy 0, policy_version 1744975 (0.0006) [2023-12-27 17:44:05,925][20381] Updated weights for policy 1, policy_version 1743546 (0.0006) [2023-12-27 17:44:05,945][20347] Updated weights for policy 0, policy_version 1744985 (0.0011) [2023-12-27 17:44:06,587][20347] Updated weights for policy 0, policy_version 1744995 (0.0009) [2023-12-27 17:44:06,620][20381] Updated weights for policy 1, policy_version 1743556 (0.0007) [2023-12-27 17:44:06,641][20347] Updated weights for policy 0, policy_version 1745005 (0.0006) [2023-12-27 17:44:06,679][20381] Updated weights for policy 1, policy_version 1743566 (0.0009) [2023-12-27 17:44:06,699][20347] Updated weights for policy 0, policy_version 1745015 (0.0008) [2023-12-27 17:44:06,735][20381] Updated weights for policy 1, policy_version 1743576 (0.0006) [2023-12-27 17:44:07,356][20347] Updated weights for policy 0, policy_version 1745025 (0.0006) [2023-12-27 17:44:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 893214720. Throughput: 0: 10278.4, 1: 10381.5. Samples: 893187684. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:07,363][19170] Avg episode reward: [(0, '960.950'), (1, '1000.000')] [2023-12-27 17:44:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001743584_446406656.pth... [2023-12-27 17:44:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001742400_446103552.pth [2023-12-27 17:44:07,409][20381] Updated weights for policy 1, policy_version 1743586 (0.0008) [2023-12-27 17:44:07,423][20347] Updated weights for policy 0, policy_version 1745035 (0.0008) [2023-12-27 17:44:07,463][20381] Updated weights for policy 1, policy_version 1743596 (0.0006) [2023-12-27 17:44:07,487][20347] Updated weights for policy 0, policy_version 1745045 (0.0006) [2023-12-27 17:44:07,519][20381] Updated weights for policy 1, policy_version 1743606 (0.0006) [2023-12-27 17:44:07,547][20347] Updated weights for policy 0, policy_version 1745055 (0.0007) [2023-12-27 17:44:07,552][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001745056_446816256.pth... [2023-12-27 17:44:07,555][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001743840_446504960.pth [2023-12-27 17:44:07,576][20381] Updated weights for policy 1, policy_version 1743616 (0.0007) [2023-12-27 17:44:08,194][20347] Updated weights for policy 0, policy_version 1745065 (0.0005) [2023-12-27 17:44:08,250][20347] Updated weights for policy 0, policy_version 1745075 (0.0008) [2023-12-27 17:44:08,303][20347] Updated weights for policy 0, policy_version 1745085 (0.0007) [2023-12-27 17:44:08,317][20381] Updated weights for policy 1, policy_version 1743626 (0.0005) [2023-12-27 17:44:08,380][20381] Updated weights for policy 1, policy_version 1743636 (0.0008) [2023-12-27 17:44:08,442][20381] Updated weights for policy 1, policy_version 1743646 (0.0008) [2023-12-27 17:44:08,983][20347] Updated weights for policy 0, policy_version 1745095 (0.0009) [2023-12-27 17:44:09,039][20347] Updated weights for policy 0, policy_version 1745105 (0.0006) [2023-12-27 17:44:09,099][20347] Updated weights for policy 0, policy_version 1745115 (0.0007) [2023-12-27 17:44:09,148][20381] Updated weights for policy 1, policy_version 1743656 (0.0007) [2023-12-27 17:44:09,201][20381] Updated weights for policy 1, policy_version 1743666 (0.0008) [2023-12-27 17:44:09,265][20381] Updated weights for policy 1, policy_version 1743676 (0.0007) [2023-12-27 17:44:09,786][20347] Updated weights for policy 0, policy_version 1745125 (0.0007) [2023-12-27 17:44:09,839][20347] Updated weights for policy 0, policy_version 1745135 (0.0008) [2023-12-27 17:44:09,902][20347] Updated weights for policy 0, policy_version 1745145 (0.0008) [2023-12-27 17:44:09,984][20381] Updated weights for policy 1, policy_version 1743686 (0.0007) [2023-12-27 17:44:10,047][20381] Updated weights for policy 1, policy_version 1743696 (0.0008) [2023-12-27 17:44:10,112][20381] Updated weights for policy 1, policy_version 1743706 (0.0008) [2023-12-27 17:44:10,623][20347] Updated weights for policy 0, policy_version 1745155 (0.0008) [2023-12-27 17:44:10,679][20347] Updated weights for policy 0, policy_version 1745165 (0.0009) [2023-12-27 17:44:10,738][20347] Updated weights for policy 0, policy_version 1745175 (0.0008) [2023-12-27 17:44:10,790][20381] Updated weights for policy 1, policy_version 1743716 (0.0007) [2023-12-27 17:44:10,836][20381] Updated weights for policy 1, policy_version 1743726 (0.0007) [2023-12-27 17:44:10,897][20381] Updated weights for policy 1, policy_version 1743736 (0.0008) [2023-12-27 17:44:11,434][20347] Updated weights for policy 0, policy_version 1745185 (0.0007) [2023-12-27 17:44:11,489][20347] Updated weights for policy 0, policy_version 1745195 (0.0008) [2023-12-27 17:44:11,545][20347] Updated weights for policy 0, policy_version 1745205 (0.0007) [2023-12-27 17:44:11,607][20347] Updated weights for policy 0, policy_version 1745215 (0.0007) [2023-12-27 17:44:11,712][20381] Updated weights for policy 1, policy_version 1743746 (0.0008) [2023-12-27 17:44:11,782][20381] Updated weights for policy 1, policy_version 1743756 (0.0009) [2023-12-27 17:44:11,843][20381] Updated weights for policy 1, policy_version 1743766 (0.0006) [2023-12-27 17:44:11,909][20381] Updated weights for policy 1, policy_version 1743776 (0.0005) [2023-12-27 17:44:12,318][20347] Updated weights for policy 0, policy_version 1745225 (0.0007) [2023-12-27 17:44:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 893313024. Throughput: 0: 10331.7, 1: 10253.8. Samples: 893308892. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:12,363][19170] Avg episode reward: [(0, '941.380'), (1, '990.270')] [2023-12-27 17:44:12,383][20347] Updated weights for policy 0, policy_version 1745235 (0.0007) [2023-12-27 17:44:12,432][20347] Updated weights for policy 0, policy_version 1745245 (0.0009) [2023-12-27 17:44:12,619][20381] Updated weights for policy 1, policy_version 1743786 (0.0006) [2023-12-27 17:44:12,682][20381] Updated weights for policy 1, policy_version 1743796 (0.0008) [2023-12-27 17:44:12,743][20381] Updated weights for policy 1, policy_version 1743806 (0.0007) [2023-12-27 17:44:13,067][20347] Updated weights for policy 0, policy_version 1745255 (0.0007) [2023-12-27 17:44:13,125][20347] Updated weights for policy 0, policy_version 1745265 (0.0006) [2023-12-27 17:44:13,182][20347] Updated weights for policy 0, policy_version 1745275 (0.0007) [2023-12-27 17:44:13,370][20381] Updated weights for policy 1, policy_version 1743816 (0.0009) [2023-12-27 17:44:13,415][20381] Updated weights for policy 1, policy_version 1743826 (0.0009) [2023-12-27 17:44:13,460][20381] Updated weights for policy 1, policy_version 1743836 (0.0008) [2023-12-27 17:44:13,802][20347] Updated weights for policy 0, policy_version 1745285 (0.0006) [2023-12-27 17:44:13,854][20347] Updated weights for policy 0, policy_version 1745295 (0.0005) [2023-12-27 17:44:13,906][20347] Updated weights for policy 0, policy_version 1745305 (0.0005) [2023-12-27 17:44:14,155][20381] Updated weights for policy 1, policy_version 1743846 (0.0008) [2023-12-27 17:44:14,205][20381] Updated weights for policy 1, policy_version 1743856 (0.0008) [2023-12-27 17:44:14,254][20381] Updated weights for policy 1, policy_version 1743866 (0.0007) [2023-12-27 17:44:14,533][20347] Updated weights for policy 0, policy_version 1745315 (0.0005) [2023-12-27 17:44:14,587][20347] Updated weights for policy 0, policy_version 1745325 (0.0005) [2023-12-27 17:44:14,636][20347] Updated weights for policy 0, policy_version 1745335 (0.0005) [2023-12-27 17:44:14,914][20381] Updated weights for policy 1, policy_version 1743876 (0.0006) [2023-12-27 17:44:14,977][20381] Updated weights for policy 1, policy_version 1743886 (0.0008) [2023-12-27 17:44:15,040][20381] Updated weights for policy 1, policy_version 1743896 (0.0008) [2023-12-27 17:44:15,260][20347] Updated weights for policy 0, policy_version 1745345 (0.0005) [2023-12-27 17:44:15,318][20347] Updated weights for policy 0, policy_version 1745355 (0.0006) [2023-12-27 17:44:15,379][20347] Updated weights for policy 0, policy_version 1745365 (0.0008) [2023-12-27 17:44:15,434][20347] Updated weights for policy 0, policy_version 1745375 (0.0008) [2023-12-27 17:44:15,684][20381] Updated weights for policy 1, policy_version 1743906 (0.0008) [2023-12-27 17:44:15,746][20381] Updated weights for policy 1, policy_version 1743916 (0.0006) [2023-12-27 17:44:15,800][20381] Updated weights for policy 1, policy_version 1743926 (0.0006) [2023-12-27 17:44:15,854][20381] Updated weights for policy 1, policy_version 1743936 (0.0006) [2023-12-27 17:44:16,057][20347] Updated weights for policy 0, policy_version 1745385 (0.0008) [2023-12-27 17:44:16,116][20347] Updated weights for policy 0, policy_version 1745395 (0.0008) [2023-12-27 17:44:16,173][20347] Updated weights for policy 0, policy_version 1745405 (0.0008) [2023-12-27 17:44:16,486][20381] Updated weights for policy 1, policy_version 1743946 (0.0008) [2023-12-27 17:44:16,544][20381] Updated weights for policy 1, policy_version 1743956 (0.0009) [2023-12-27 17:44:16,589][20381] Updated weights for policy 1, policy_version 1743966 (0.0008) [2023-12-27 17:44:16,815][20347] Updated weights for policy 0, policy_version 1745415 (0.0008) [2023-12-27 17:44:16,868][20347] Updated weights for policy 0, policy_version 1745425 (0.0008) [2023-12-27 17:44:16,928][20347] Updated weights for policy 0, policy_version 1745435 (0.0008) [2023-12-27 17:44:17,281][20381] Updated weights for policy 1, policy_version 1743976 (0.0007) [2023-12-27 17:44:17,334][20381] Updated weights for policy 1, policy_version 1743986 (0.0007) [2023-12-27 17:44:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 893419520. Throughput: 0: 10329.8, 1: 10219.6. Samples: 893433708. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:17,363][19170] Avg episode reward: [(0, '931.550'), (1, '990.270')] [2023-12-27 17:44:17,387][20381] Updated weights for policy 1, policy_version 1743996 (0.0008) [2023-12-27 17:44:17,576][20347] Updated weights for policy 0, policy_version 1745445 (0.0007) [2023-12-27 17:44:17,638][20347] Updated weights for policy 0, policy_version 1745455 (0.0008) [2023-12-27 17:44:17,689][20347] Updated weights for policy 0, policy_version 1745465 (0.0007) [2023-12-27 17:44:18,001][20381] Updated weights for policy 1, policy_version 1744006 (0.0007) [2023-12-27 17:44:18,059][20381] Updated weights for policy 1, policy_version 1744016 (0.0006) [2023-12-27 17:44:18,118][20381] Updated weights for policy 1, policy_version 1744026 (0.0007) [2023-12-27 17:44:18,425][20347] Updated weights for policy 0, policy_version 1745475 (0.0008) [2023-12-27 17:44:18,484][20347] Updated weights for policy 0, policy_version 1745485 (0.0008) [2023-12-27 17:44:18,539][20347] Updated weights for policy 0, policy_version 1745495 (0.0006) [2023-12-27 17:44:18,751][20381] Updated weights for policy 1, policy_version 1744036 (0.0007) [2023-12-27 17:44:18,809][20381] Updated weights for policy 1, policy_version 1744046 (0.0008) [2023-12-27 17:44:18,869][20381] Updated weights for policy 1, policy_version 1744056 (0.0008) [2023-12-27 17:44:19,260][20347] Updated weights for policy 0, policy_version 1745505 (0.0008) [2023-12-27 17:44:19,321][20347] Updated weights for policy 0, policy_version 1745515 (0.0008) [2023-12-27 17:44:19,385][20347] Updated weights for policy 0, policy_version 1745525 (0.0008) [2023-12-27 17:44:19,443][20347] Updated weights for policy 0, policy_version 1745535 (0.0006) [2023-12-27 17:44:19,672][20381] Updated weights for policy 1, policy_version 1744066 (0.0008) [2023-12-27 17:44:19,732][20381] Updated weights for policy 1, policy_version 1744076 (0.0008) [2023-12-27 17:44:19,794][20381] Updated weights for policy 1, policy_version 1744086 (0.0008) [2023-12-27 17:44:19,859][20381] Updated weights for policy 1, policy_version 1744096 (0.0006) [2023-12-27 17:44:20,110][20347] Updated weights for policy 0, policy_version 1745545 (0.0006) [2023-12-27 17:44:20,167][20347] Updated weights for policy 0, policy_version 1745555 (0.0006) [2023-12-27 17:44:20,230][20347] Updated weights for policy 0, policy_version 1745565 (0.0007) [2023-12-27 17:44:20,540][20381] Updated weights for policy 1, policy_version 1744106 (0.0008) [2023-12-27 17:44:20,600][20381] Updated weights for policy 1, policy_version 1744116 (0.0008) [2023-12-27 17:44:20,663][20381] Updated weights for policy 1, policy_version 1744126 (0.0008) [2023-12-27 17:44:20,955][20347] Updated weights for policy 0, policy_version 1745575 (0.0008) [2023-12-27 17:44:21,013][20347] Updated weights for policy 0, policy_version 1745585 (0.0008) [2023-12-27 17:44:21,082][20347] Updated weights for policy 0, policy_version 1745595 (0.0007) [2023-12-27 17:44:21,341][20381] Updated weights for policy 1, policy_version 1744136 (0.0007) [2023-12-27 17:44:21,410][20381] Updated weights for policy 1, policy_version 1744146 (0.0009) [2023-12-27 17:44:21,470][20381] Updated weights for policy 1, policy_version 1744156 (0.0008) [2023-12-27 17:44:21,832][20347] Updated weights for policy 0, policy_version 1745605 (0.0008) [2023-12-27 17:44:21,891][20347] Updated weights for policy 0, policy_version 1745615 (0.0007) [2023-12-27 17:44:21,954][20347] Updated weights for policy 0, policy_version 1745625 (0.0007) [2023-12-27 17:44:22,206][20381] Updated weights for policy 1, policy_version 1744166 (0.0006) [2023-12-27 17:44:22,264][20381] Updated weights for policy 1, policy_version 1744176 (0.0007) [2023-12-27 17:44:22,324][20381] Updated weights for policy 1, policy_version 1744186 (0.0010) [2023-12-27 17:44:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 893526016. Throughput: 0: 10290.3, 1: 10232.0. Samples: 893494308. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:22,363][19170] Avg episode reward: [(0, '941.680'), (1, '1000.000')] [2023-12-27 17:44:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001744192_446562304.pth... [2023-12-27 17:44:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001745632_446963712.pth... [2023-12-27 17:44:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001743008_446259200.pth [2023-12-27 17:44:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001744416_446652416.pth [2023-12-27 17:44:22,652][20347] Updated weights for policy 0, policy_version 1745635 (0.0008) [2023-12-27 17:44:22,709][20347] Updated weights for policy 0, policy_version 1745645 (0.0008) [2023-12-27 17:44:22,765][20347] Updated weights for policy 0, policy_version 1745655 (0.0008) [2023-12-27 17:44:23,016][20381] Updated weights for policy 1, policy_version 1744196 (0.0009) [2023-12-27 17:44:23,080][20381] Updated weights for policy 1, policy_version 1744206 (0.0008) [2023-12-27 17:44:23,138][20381] Updated weights for policy 1, policy_version 1744216 (0.0008) [2023-12-27 17:44:23,424][20347] Updated weights for policy 0, policy_version 1745665 (0.0008) [2023-12-27 17:44:23,478][20347] Updated weights for policy 0, policy_version 1745675 (0.0007) [2023-12-27 17:44:23,534][20347] Updated weights for policy 0, policy_version 1745685 (0.0005) [2023-12-27 17:44:23,588][20347] Updated weights for policy 0, policy_version 1745695 (0.0008) [2023-12-27 17:44:23,850][20381] Updated weights for policy 1, policy_version 1744226 (0.0008) [2023-12-27 17:44:23,905][20381] Updated weights for policy 1, policy_version 1744236 (0.0007) [2023-12-27 17:44:23,966][20381] Updated weights for policy 1, policy_version 1744246 (0.0008) [2023-12-27 17:44:24,016][20381] Updated weights for policy 1, policy_version 1744256 (0.0007) [2023-12-27 17:44:24,250][20347] Updated weights for policy 0, policy_version 1745705 (0.0010) [2023-12-27 17:44:24,302][20347] Updated weights for policy 0, policy_version 1745715 (0.0011) [2023-12-27 17:44:24,349][20347] Updated weights for policy 0, policy_version 1745725 (0.0008) [2023-12-27 17:44:24,717][20381] Updated weights for policy 1, policy_version 1744266 (0.0007) [2023-12-27 17:44:24,773][20381] Updated weights for policy 1, policy_version 1744276 (0.0009) [2023-12-27 17:44:24,822][20381] Updated weights for policy 1, policy_version 1744286 (0.0008) [2023-12-27 17:44:24,956][20347] Updated weights for policy 0, policy_version 1745735 (0.0007) [2023-12-27 17:44:25,016][20347] Updated weights for policy 0, policy_version 1745745 (0.0005) [2023-12-27 17:44:25,070][20347] Updated weights for policy 0, policy_version 1745755 (0.0005) [2023-12-27 17:44:25,584][20381] Updated weights for policy 1, policy_version 1744296 (0.0008) [2023-12-27 17:44:25,600][20347] Updated weights for policy 0, policy_version 1745765 (0.0005) [2023-12-27 17:44:25,640][20381] Updated weights for policy 1, policy_version 1744306 (0.0006) [2023-12-27 17:44:25,653][20347] Updated weights for policy 0, policy_version 1745775 (0.0005) [2023-12-27 17:44:25,692][20381] Updated weights for policy 1, policy_version 1744316 (0.0006) [2023-12-27 17:44:25,705][20347] Updated weights for policy 0, policy_version 1745785 (0.0006) [2023-12-27 17:44:26,324][20381] Updated weights for policy 1, policy_version 1744326 (0.0006) [2023-12-27 17:44:26,376][20347] Updated weights for policy 0, policy_version 1745795 (0.0006) [2023-12-27 17:44:26,377][20381] Updated weights for policy 1, policy_version 1744336 (0.0005) [2023-12-27 17:44:26,424][20381] Updated weights for policy 1, policy_version 1744346 (0.0006) [2023-12-27 17:44:26,433][20347] Updated weights for policy 0, policy_version 1745805 (0.0007) [2023-12-27 17:44:26,490][20347] Updated weights for policy 0, policy_version 1745815 (0.0007) [2023-12-27 17:44:27,026][20381] Updated weights for policy 1, policy_version 1744356 (0.0007) [2023-12-27 17:44:27,074][20381] Updated weights for policy 1, policy_version 1744366 (0.0008) [2023-12-27 17:44:27,119][20381] Updated weights for policy 1, policy_version 1744376 (0.0008) [2023-12-27 17:44:27,164][20347] Updated weights for policy 0, policy_version 1745825 (0.0007) [2023-12-27 17:44:27,211][20347] Updated weights for policy 0, policy_version 1745835 (0.0008) [2023-12-27 17:44:27,254][20347] Updated weights for policy 0, policy_version 1745845 (0.0007) [2023-12-27 17:44:27,307][20347] Updated weights for policy 0, policy_version 1745855 (0.0007) [2023-12-27 17:44:27,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 893632512. Throughput: 0: 10338.1, 1: 10171.8. Samples: 893616620. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:27,363][19170] Avg episode reward: [(0, '932.010'), (1, '1000.000')] [2023-12-27 17:44:27,825][20381] Updated weights for policy 1, policy_version 1744386 (0.0007) [2023-12-27 17:44:27,876][20381] Updated weights for policy 1, policy_version 1744396 (0.0008) [2023-12-27 17:44:27,924][20381] Updated weights for policy 1, policy_version 1744406 (0.0006) [2023-12-27 17:44:27,940][20347] Updated weights for policy 0, policy_version 1745865 (0.0008) [2023-12-27 17:44:27,978][20381] Updated weights for policy 1, policy_version 1744416 (0.0008) [2023-12-27 17:44:28,001][20347] Updated weights for policy 0, policy_version 1745875 (0.0007) [2023-12-27 17:44:28,044][20347] Updated weights for policy 0, policy_version 1745885 (0.0006) [2023-12-27 17:44:28,707][20381] Updated weights for policy 1, policy_version 1744426 (0.0007) [2023-12-27 17:44:28,710][20347] Updated weights for policy 0, policy_version 1745895 (0.0006) [2023-12-27 17:44:28,760][20381] Updated weights for policy 1, policy_version 1744436 (0.0006) [2023-12-27 17:44:28,769][20347] Updated weights for policy 0, policy_version 1745905 (0.0006) [2023-12-27 17:44:28,816][20381] Updated weights for policy 1, policy_version 1744446 (0.0006) [2023-12-27 17:44:28,825][20347] Updated weights for policy 0, policy_version 1745915 (0.0006) [2023-12-27 17:44:29,461][20347] Updated weights for policy 0, policy_version 1745925 (0.0006) [2023-12-27 17:44:29,520][20347] Updated weights for policy 0, policy_version 1745935 (0.0007) [2023-12-27 17:44:29,563][20381] Updated weights for policy 1, policy_version 1744456 (0.0007) [2023-12-27 17:44:29,574][20347] Updated weights for policy 0, policy_version 1745945 (0.0006) [2023-12-27 17:44:29,620][20381] Updated weights for policy 1, policy_version 1744466 (0.0008) [2023-12-27 17:44:29,674][20381] Updated weights for policy 1, policy_version 1744476 (0.0006) [2023-12-27 17:44:30,262][20347] Updated weights for policy 0, policy_version 1745955 (0.0006) [2023-12-27 17:44:30,324][20347] Updated weights for policy 0, policy_version 1745965 (0.0007) [2023-12-27 17:44:30,327][20381] Updated weights for policy 1, policy_version 1744486 (0.0008) [2023-12-27 17:44:30,387][20347] Updated weights for policy 0, policy_version 1745975 (0.0007) [2023-12-27 17:44:30,391][20381] Updated weights for policy 1, policy_version 1744496 (0.0007) [2023-12-27 17:44:30,449][20381] Updated weights for policy 1, policy_version 1744506 (0.0005) [2023-12-27 17:44:31,013][20347] Updated weights for policy 0, policy_version 1745985 (0.0008) [2023-12-27 17:44:31,055][20381] Updated weights for policy 1, policy_version 1744516 (0.0007) [2023-12-27 17:44:31,075][20347] Updated weights for policy 0, policy_version 1745995 (0.0007) [2023-12-27 17:44:31,111][20381] Updated weights for policy 1, policy_version 1744526 (0.0007) [2023-12-27 17:44:31,123][20347] Updated weights for policy 0, policy_version 1746005 (0.0006) [2023-12-27 17:44:31,175][20381] Updated weights for policy 1, policy_version 1744536 (0.0008) [2023-12-27 17:44:31,188][20347] Updated weights for policy 0, policy_version 1746015 (0.0007) [2023-12-27 17:44:31,820][20347] Updated weights for policy 0, policy_version 1746025 (0.0008) [2023-12-27 17:44:31,879][20347] Updated weights for policy 0, policy_version 1746035 (0.0008) [2023-12-27 17:44:31,921][20381] Updated weights for policy 1, policy_version 1744546 (0.0007) [2023-12-27 17:44:31,938][20347] Updated weights for policy 0, policy_version 1746045 (0.0006) [2023-12-27 17:44:31,980][20381] Updated weights for policy 1, policy_version 1744556 (0.0007) [2023-12-27 17:44:32,042][20381] Updated weights for policy 1, policy_version 1744566 (0.0008) [2023-12-27 17:44:32,103][20381] Updated weights for policy 1, policy_version 1744576 (0.0008) [2023-12-27 17:44:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 893730816. Throughput: 0: 10394.0, 1: 10155.1. Samples: 893740476. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:32,363][19170] Avg episode reward: [(0, '931.730'), (1, '1000.000')] [2023-12-27 17:44:32,595][20347] Updated weights for policy 0, policy_version 1746055 (0.0005) [2023-12-27 17:44:32,648][20347] Updated weights for policy 0, policy_version 1746065 (0.0007) [2023-12-27 17:44:32,696][20347] Updated weights for policy 0, policy_version 1746075 (0.0008) [2023-12-27 17:44:32,836][20381] Updated weights for policy 1, policy_version 1744586 (0.0006) [2023-12-27 17:44:32,897][20381] Updated weights for policy 1, policy_version 1744596 (0.0006) [2023-12-27 17:44:32,948][20381] Updated weights for policy 1, policy_version 1744606 (0.0005) [2023-12-27 17:44:33,340][20347] Updated weights for policy 0, policy_version 1746085 (0.0009) [2023-12-27 17:44:33,391][20347] Updated weights for policy 0, policy_version 1746095 (0.0011) [2023-12-27 17:44:33,443][20347] Updated weights for policy 0, policy_version 1746105 (0.0006) [2023-12-27 17:44:33,455][20381] Updated weights for policy 1, policy_version 1744616 (0.0005) [2023-12-27 17:44:33,502][20381] Updated weights for policy 1, policy_version 1744626 (0.0005) [2023-12-27 17:44:33,552][20381] Updated weights for policy 1, policy_version 1744636 (0.0005) [2023-12-27 17:44:34,127][20347] Updated weights for policy 0, policy_version 1746115 (0.0006) [2023-12-27 17:44:34,192][20347] Updated weights for policy 0, policy_version 1746125 (0.0008) [2023-12-27 17:44:34,196][20381] Updated weights for policy 1, policy_version 1744646 (0.0007) [2023-12-27 17:44:34,251][20347] Updated weights for policy 0, policy_version 1746135 (0.0006) [2023-12-27 17:44:34,259][20381] Updated weights for policy 1, policy_version 1744656 (0.0007) [2023-12-27 17:44:34,316][20381] Updated weights for policy 1, policy_version 1744666 (0.0008) [2023-12-27 17:44:34,875][20347] Updated weights for policy 0, policy_version 1746145 (0.0006) [2023-12-27 17:44:34,930][20347] Updated weights for policy 0, policy_version 1746155 (0.0006) [2023-12-27 17:44:34,992][20347] Updated weights for policy 0, policy_version 1746165 (0.0006) [2023-12-27 17:44:35,008][20381] Updated weights for policy 1, policy_version 1744676 (0.0008) [2023-12-27 17:44:35,043][20347] Updated weights for policy 0, policy_version 1746175 (0.0005) [2023-12-27 17:44:35,063][20381] Updated weights for policy 1, policy_version 1744686 (0.0010) [2023-12-27 17:44:35,121][20381] Updated weights for policy 1, policy_version 1744696 (0.0010) [2023-12-27 17:44:35,608][20347] Updated weights for policy 0, policy_version 1746185 (0.0005) [2023-12-27 17:44:35,655][20347] Updated weights for policy 0, policy_version 1746195 (0.0005) [2023-12-27 17:44:35,707][20347] Updated weights for policy 0, policy_version 1746205 (0.0005) [2023-12-27 17:44:35,775][20381] Updated weights for policy 1, policy_version 1744706 (0.0010) [2023-12-27 17:44:35,835][20381] Updated weights for policy 1, policy_version 1744716 (0.0010) [2023-12-27 17:44:35,892][20381] Updated weights for policy 1, policy_version 1744726 (0.0010) [2023-12-27 17:44:35,943][20381] Updated weights for policy 1, policy_version 1744736 (0.0010) [2023-12-27 17:44:36,306][20347] Updated weights for policy 0, policy_version 1746215 (0.0007) [2023-12-27 17:44:36,370][20347] Updated weights for policy 0, policy_version 1746225 (0.0007) [2023-12-27 17:44:36,438][20347] Updated weights for policy 0, policy_version 1746235 (0.0008) [2023-12-27 17:44:36,622][20381] Updated weights for policy 1, policy_version 1744746 (0.0011) [2023-12-27 17:44:36,683][20381] Updated weights for policy 1, policy_version 1744756 (0.0011) [2023-12-27 17:44:36,743][20381] Updated weights for policy 1, policy_version 1744766 (0.0011) [2023-12-27 17:44:37,104][20347] Updated weights for policy 0, policy_version 1746245 (0.0008) [2023-12-27 17:44:37,156][20347] Updated weights for policy 0, policy_version 1746255 (0.0007) [2023-12-27 17:44:37,212][20347] Updated weights for policy 0, policy_version 1746265 (0.0005) [2023-12-27 17:44:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 893837312. Throughput: 0: 10450.4, 1: 10157.2. Samples: 893804408. Policy #0 lag: (min: 11.0, avg: 16.5, max: 43.0) [2023-12-27 17:44:37,363][19170] Avg episode reward: [(0, '941.320'), (1, '1000.000')] [2023-12-27 17:44:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001746272_447127552.pth... [2023-12-27 17:44:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001745056_446816256.pth [2023-12-27 17:44:37,387][20381] Updated weights for policy 1, policy_version 1744776 (0.0011) [2023-12-27 17:44:37,442][20381] Updated weights for policy 1, policy_version 1744786 (0.0011) [2023-12-27 17:44:37,505][20381] Updated weights for policy 1, policy_version 1744796 (0.0011) [2023-12-27 17:44:37,527][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001744800_446717952.pth... [2023-12-27 17:44:37,530][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001743584_446406656.pth [2023-12-27 17:44:37,867][20347] Updated weights for policy 0, policy_version 1746275 (0.0006) [2023-12-27 17:44:37,926][20347] Updated weights for policy 0, policy_version 1746285 (0.0008) [2023-12-27 17:44:37,983][20347] Updated weights for policy 0, policy_version 1746295 (0.0007) [2023-12-27 17:44:38,198][20381] Updated weights for policy 1, policy_version 1744806 (0.0009) [2023-12-27 17:44:38,261][20381] Updated weights for policy 1, policy_version 1744816 (0.0007) [2023-12-27 17:44:38,320][20381] Updated weights for policy 1, policy_version 1744826 (0.0008) [2023-12-27 17:44:38,720][20347] Updated weights for policy 0, policy_version 1746305 (0.0007) [2023-12-27 17:44:38,785][20347] Updated weights for policy 0, policy_version 1746315 (0.0007) [2023-12-27 17:44:38,849][20347] Updated weights for policy 0, policy_version 1746325 (0.0009) [2023-12-27 17:44:38,908][20347] Updated weights for policy 0, policy_version 1746335 (0.0007) [2023-12-27 17:44:38,980][20381] Updated weights for policy 1, policy_version 1744836 (0.0009) [2023-12-27 17:44:39,043][20381] Updated weights for policy 1, policy_version 1744846 (0.0009) [2023-12-27 17:44:39,094][20381] Updated weights for policy 1, policy_version 1744856 (0.0005) [2023-12-27 17:44:39,600][20347] Updated weights for policy 0, policy_version 1746345 (0.0007) [2023-12-27 17:44:39,649][20347] Updated weights for policy 0, policy_version 1746355 (0.0009) [2023-12-27 17:44:39,702][20347] Updated weights for policy 0, policy_version 1746365 (0.0009) [2023-12-27 17:44:39,780][20381] Updated weights for policy 1, policy_version 1744866 (0.0006) [2023-12-27 17:44:39,846][20381] Updated weights for policy 1, policy_version 1744876 (0.0008) [2023-12-27 17:44:39,908][20381] Updated weights for policy 1, policy_version 1744886 (0.0008) [2023-12-27 17:44:39,972][20381] Updated weights for policy 1, policy_version 1744896 (0.0008) [2023-12-27 17:44:40,480][20347] Updated weights for policy 0, policy_version 1746375 (0.0008) [2023-12-27 17:44:40,530][20347] Updated weights for policy 0, policy_version 1746385 (0.0006) [2023-12-27 17:44:40,587][20347] Updated weights for policy 0, policy_version 1746395 (0.0008) [2023-12-27 17:44:40,647][20381] Updated weights for policy 1, policy_version 1744906 (0.0007) [2023-12-27 17:44:40,711][20381] Updated weights for policy 1, policy_version 1744916 (0.0008) [2023-12-27 17:44:40,771][20381] Updated weights for policy 1, policy_version 1744926 (0.0007) [2023-12-27 17:44:41,231][20347] Updated weights for policy 0, policy_version 1746405 (0.0007) [2023-12-27 17:44:41,295][20347] Updated weights for policy 0, policy_version 1746415 (0.0008) [2023-12-27 17:44:41,353][20347] Updated weights for policy 0, policy_version 1746425 (0.0008) [2023-12-27 17:44:41,540][20381] Updated weights for policy 1, policy_version 1744936 (0.0008) [2023-12-27 17:44:41,607][20381] Updated weights for policy 1, policy_version 1744946 (0.0009) [2023-12-27 17:44:41,669][20381] Updated weights for policy 1, policy_version 1744956 (0.0008) [2023-12-27 17:44:42,097][20347] Updated weights for policy 0, policy_version 1746435 (0.0008) [2023-12-27 17:44:42,151][20347] Updated weights for policy 0, policy_version 1746445 (0.0006) [2023-12-27 17:44:42,198][20347] Updated weights for policy 0, policy_version 1746455 (0.0006) [2023-12-27 17:44:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 893935616. Throughput: 0: 10444.9, 1: 10114.0. Samples: 893927772. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:44:42,363][19170] Avg episode reward: [(0, '951.110'), (1, '1000.000')] [2023-12-27 17:44:42,389][20381] Updated weights for policy 1, policy_version 1744966 (0.0008) [2023-12-27 17:44:42,447][20381] Updated weights for policy 1, policy_version 1744976 (0.0008) [2023-12-27 17:44:42,508][20381] Updated weights for policy 1, policy_version 1744986 (0.0007) [2023-12-27 17:44:42,836][20347] Updated weights for policy 0, policy_version 1746465 (0.0006) [2023-12-27 17:44:42,893][20347] Updated weights for policy 0, policy_version 1746475 (0.0010) [2023-12-27 17:44:42,951][20347] Updated weights for policy 0, policy_version 1746485 (0.0010) [2023-12-27 17:44:42,998][20347] Updated weights for policy 0, policy_version 1746495 (0.0010) [2023-12-27 17:44:43,232][20381] Updated weights for policy 1, policy_version 1744996 (0.0007) [2023-12-27 17:44:43,289][20381] Updated weights for policy 1, policy_version 1745006 (0.0008) [2023-12-27 17:44:43,351][20381] Updated weights for policy 1, policy_version 1745016 (0.0008) [2023-12-27 17:44:43,608][20347] Updated weights for policy 0, policy_version 1746505 (0.0006) [2023-12-27 17:44:43,657][20347] Updated weights for policy 0, policy_version 1746515 (0.0005) [2023-12-27 17:44:43,706][20347] Updated weights for policy 0, policy_version 1746525 (0.0005) [2023-12-27 17:44:44,092][20381] Updated weights for policy 1, policy_version 1745026 (0.0007) [2023-12-27 17:44:44,150][20381] Updated weights for policy 1, policy_version 1745036 (0.0007) [2023-12-27 17:44:44,194][20381] Updated weights for policy 1, policy_version 1745046 (0.0007) [2023-12-27 17:44:44,243][20381] Updated weights for policy 1, policy_version 1745056 (0.0007) [2023-12-27 17:44:44,317][20347] Updated weights for policy 0, policy_version 1746535 (0.0009) [2023-12-27 17:44:44,368][20347] Updated weights for policy 0, policy_version 1746545 (0.0010) [2023-12-27 17:44:44,416][20347] Updated weights for policy 0, policy_version 1746555 (0.0010) [2023-12-27 17:44:44,957][20381] Updated weights for policy 1, policy_version 1745066 (0.0010) [2023-12-27 17:44:45,018][20381] Updated weights for policy 1, policy_version 1745076 (0.0011) [2023-12-27 17:44:45,074][20381] Updated weights for policy 1, policy_version 1745086 (0.0011) [2023-12-27 17:44:45,122][20347] Updated weights for policy 0, policy_version 1746565 (0.0010) [2023-12-27 17:44:45,183][20347] Updated weights for policy 0, policy_version 1746575 (0.0011) [2023-12-27 17:44:45,244][20347] Updated weights for policy 0, policy_version 1746585 (0.0011) [2023-12-27 17:44:45,749][20381] Updated weights for policy 1, policy_version 1745096 (0.0008) [2023-12-27 17:44:45,795][20381] Updated weights for policy 1, policy_version 1745106 (0.0005) [2023-12-27 17:44:45,854][20381] Updated weights for policy 1, policy_version 1745116 (0.0007) [2023-12-27 17:44:45,930][20347] Updated weights for policy 0, policy_version 1746595 (0.0011) [2023-12-27 17:44:45,981][20347] Updated weights for policy 0, policy_version 1746605 (0.0010) [2023-12-27 17:44:46,032][20347] Updated weights for policy 0, policy_version 1746615 (0.0010) [2023-12-27 17:44:46,514][20381] Updated weights for policy 1, policy_version 1745126 (0.0009) [2023-12-27 17:44:46,572][20381] Updated weights for policy 1, policy_version 1745136 (0.0007) [2023-12-27 17:44:46,624][20381] Updated weights for policy 1, policy_version 1745146 (0.0008) [2023-12-27 17:44:46,737][20347] Updated weights for policy 0, policy_version 1746625 (0.0008) [2023-12-27 17:44:46,791][20347] Updated weights for policy 0, policy_version 1746635 (0.0009) [2023-12-27 17:44:46,855][20347] Updated weights for policy 0, policy_version 1746645 (0.0010) [2023-12-27 17:44:46,916][20347] Updated weights for policy 0, policy_version 1746655 (0.0005) [2023-12-27 17:44:47,219][20381] Updated weights for policy 1, policy_version 1745156 (0.0007) [2023-12-27 17:44:47,266][20381] Updated weights for policy 1, policy_version 1745166 (0.0007) [2023-12-27 17:44:47,311][20381] Updated weights for policy 1, policy_version 1745176 (0.0005) [2023-12-27 17:44:47,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 894042112. Throughput: 0: 10433.4, 1: 10094.6. Samples: 894050164. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:44:47,364][19170] Avg episode reward: [(0, '960.900'), (1, '1000.000')] [2023-12-27 17:44:47,547][20347] Updated weights for policy 0, policy_version 1746665 (0.0010) [2023-12-27 17:44:47,610][20347] Updated weights for policy 0, policy_version 1746675 (0.0010) [2023-12-27 17:44:47,668][20347] Updated weights for policy 0, policy_version 1746685 (0.0010) [2023-12-27 17:44:48,008][20381] Updated weights for policy 1, policy_version 1745186 (0.0007) [2023-12-27 17:44:48,058][20381] Updated weights for policy 1, policy_version 1745196 (0.0008) [2023-12-27 17:44:48,107][20381] Updated weights for policy 1, policy_version 1745206 (0.0008) [2023-12-27 17:44:48,156][20381] Updated weights for policy 1, policy_version 1745216 (0.0008) [2023-12-27 17:44:48,308][20347] Updated weights for policy 0, policy_version 1746695 (0.0010) [2023-12-27 17:44:48,373][20347] Updated weights for policy 0, policy_version 1746705 (0.0011) [2023-12-27 17:44:48,426][20347] Updated weights for policy 0, policy_version 1746715 (0.0010) [2023-12-27 17:44:48,869][20381] Updated weights for policy 1, policy_version 1745226 (0.0011) [2023-12-27 17:44:48,915][20381] Updated weights for policy 1, policy_version 1745236 (0.0010) [2023-12-27 17:44:48,975][20381] Updated weights for policy 1, policy_version 1745246 (0.0011) [2023-12-27 17:44:49,106][20347] Updated weights for policy 0, policy_version 1746725 (0.0008) [2023-12-27 17:44:49,165][20347] Updated weights for policy 0, policy_version 1746735 (0.0006) [2023-12-27 17:44:49,217][20347] Updated weights for policy 0, policy_version 1746745 (0.0006) [2023-12-27 17:44:49,731][20381] Updated weights for policy 1, policy_version 1745256 (0.0011) [2023-12-27 17:44:49,793][20381] Updated weights for policy 1, policy_version 1745266 (0.0011) [2023-12-27 17:44:49,858][20381] Updated weights for policy 1, policy_version 1745276 (0.0009) [2023-12-27 17:44:49,872][20347] Updated weights for policy 0, policy_version 1746755 (0.0008) [2023-12-27 17:44:49,931][20347] Updated weights for policy 0, policy_version 1746765 (0.0008) [2023-12-27 17:44:49,990][20347] Updated weights for policy 0, policy_version 1746775 (0.0008) [2023-12-27 17:44:50,460][20381] Updated weights for policy 1, policy_version 1745286 (0.0006) [2023-12-27 17:44:50,519][20381] Updated weights for policy 1, policy_version 1745296 (0.0008) [2023-12-27 17:44:50,581][20381] Updated weights for policy 1, policy_version 1745306 (0.0007) [2023-12-27 17:44:50,774][20347] Updated weights for policy 0, policy_version 1746785 (0.0007) [2023-12-27 17:44:50,834][20347] Updated weights for policy 0, policy_version 1746795 (0.0008) [2023-12-27 17:44:50,884][20347] Updated weights for policy 0, policy_version 1746805 (0.0006) [2023-12-27 17:44:50,938][20347] Updated weights for policy 0, policy_version 1746815 (0.0005) [2023-12-27 17:44:51,228][20381] Updated weights for policy 1, policy_version 1745316 (0.0007) [2023-12-27 17:44:51,296][20381] Updated weights for policy 1, policy_version 1745326 (0.0008) [2023-12-27 17:44:51,359][20381] Updated weights for policy 1, policy_version 1745336 (0.0008) [2023-12-27 17:44:51,638][20347] Updated weights for policy 0, policy_version 1746825 (0.0007) [2023-12-27 17:44:51,701][20347] Updated weights for policy 0, policy_version 1746835 (0.0008) [2023-12-27 17:44:51,778][20347] Updated weights for policy 0, policy_version 1746845 (0.0008) [2023-12-27 17:44:51,971][20381] Updated weights for policy 1, policy_version 1745346 (0.0006) [2023-12-27 17:44:52,032][20381] Updated weights for policy 1, policy_version 1745356 (0.0006) [2023-12-27 17:44:52,089][20381] Updated weights for policy 1, policy_version 1745366 (0.0008) [2023-12-27 17:44:52,145][20381] Updated weights for policy 1, policy_version 1745376 (0.0007) [2023-12-27 17:44:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 894140416. Throughput: 0: 10428.8, 1: 10084.0. Samples: 894110760. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:44:52,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 17:44:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001745376_446865408.pth... [2023-12-27 17:44:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001746848_447275008.pth... [2023-12-27 17:44:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001744192_446562304.pth [2023-12-27 17:44:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001745632_446963712.pth [2023-12-27 17:44:52,479][20347] Updated weights for policy 0, policy_version 1746855 (0.0008) [2023-12-27 17:44:52,539][20347] Updated weights for policy 0, policy_version 1746865 (0.0008) [2023-12-27 17:44:52,596][20347] Updated weights for policy 0, policy_version 1746875 (0.0007) [2023-12-27 17:44:52,777][20381] Updated weights for policy 1, policy_version 1745386 (0.0006) [2023-12-27 17:44:52,823][20381] Updated weights for policy 1, policy_version 1745396 (0.0005) [2023-12-27 17:44:52,870][20381] Updated weights for policy 1, policy_version 1745406 (0.0006) [2023-12-27 17:44:53,330][20347] Updated weights for policy 0, policy_version 1746885 (0.0007) [2023-12-27 17:44:53,388][20347] Updated weights for policy 0, policy_version 1746895 (0.0009) [2023-12-27 17:44:53,443][20347] Updated weights for policy 0, policy_version 1746905 (0.0010) [2023-12-27 17:44:53,542][20381] Updated weights for policy 1, policy_version 1745416 (0.0006) [2023-12-27 17:44:53,594][20381] Updated weights for policy 1, policy_version 1745426 (0.0007) [2023-12-27 17:44:53,658][20381] Updated weights for policy 1, policy_version 1745436 (0.0011) [2023-12-27 17:44:54,059][20347] Updated weights for policy 0, policy_version 1746915 (0.0009) [2023-12-27 17:44:54,114][20347] Updated weights for policy 0, policy_version 1746925 (0.0005) [2023-12-27 17:44:54,173][20347] Updated weights for policy 0, policy_version 1746935 (0.0005) [2023-12-27 17:44:54,297][20381] Updated weights for policy 1, policy_version 1745446 (0.0010) [2023-12-27 17:44:54,351][20381] Updated weights for policy 1, policy_version 1745456 (0.0005) [2023-12-27 17:44:54,404][20381] Updated weights for policy 1, policy_version 1745466 (0.0009) [2023-12-27 17:44:54,777][20347] Updated weights for policy 0, policy_version 1746945 (0.0007) [2023-12-27 17:44:54,831][20347] Updated weights for policy 0, policy_version 1746955 (0.0009) [2023-12-27 17:44:54,887][20347] Updated weights for policy 0, policy_version 1746965 (0.0005) [2023-12-27 17:44:54,937][20347] Updated weights for policy 0, policy_version 1746975 (0.0009) [2023-12-27 17:44:54,992][20381] Updated weights for policy 1, policy_version 1745476 (0.0009) [2023-12-27 17:44:55,048][20381] Updated weights for policy 1, policy_version 1745486 (0.0011) [2023-12-27 17:44:55,104][20381] Updated weights for policy 1, policy_version 1745496 (0.0008) [2023-12-27 17:44:55,536][20347] Updated weights for policy 0, policy_version 1746985 (0.0007) [2023-12-27 17:44:55,595][20347] Updated weights for policy 0, policy_version 1746995 (0.0006) [2023-12-27 17:44:55,652][20347] Updated weights for policy 0, policy_version 1747005 (0.0005) [2023-12-27 17:44:55,770][20381] Updated weights for policy 1, policy_version 1745506 (0.0011) [2023-12-27 17:44:55,831][20381] Updated weights for policy 1, policy_version 1745516 (0.0011) [2023-12-27 17:44:55,893][20381] Updated weights for policy 1, policy_version 1745526 (0.0010) [2023-12-27 17:44:55,951][20381] Updated weights for policy 1, policy_version 1745536 (0.0011) [2023-12-27 17:44:56,306][20347] Updated weights for policy 0, policy_version 1747015 (0.0006) [2023-12-27 17:44:56,359][20347] Updated weights for policy 0, policy_version 1747025 (0.0007) [2023-12-27 17:44:56,413][20347] Updated weights for policy 0, policy_version 1747035 (0.0007) [2023-12-27 17:44:56,610][20381] Updated weights for policy 1, policy_version 1745546 (0.0010) [2023-12-27 17:44:56,658][20381] Updated weights for policy 1, policy_version 1745556 (0.0010) [2023-12-27 17:44:56,709][20381] Updated weights for policy 1, policy_version 1745566 (0.0010) [2023-12-27 17:44:57,089][20347] Updated weights for policy 0, policy_version 1747045 (0.0006) [2023-12-27 17:44:57,134][20347] Updated weights for policy 0, policy_version 1747055 (0.0005) [2023-12-27 17:44:57,192][20347] Updated weights for policy 0, policy_version 1747065 (0.0005) [2023-12-27 17:44:57,279][20381] Updated weights for policy 1, policy_version 1745576 (0.0008) [2023-12-27 17:44:57,336][20381] Updated weights for policy 1, policy_version 1745586 (0.0006) [2023-12-27 17:44:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 894246912. Throughput: 0: 10402.3, 1: 10219.5. Samples: 894236872. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:44:57,363][19170] Avg episode reward: [(0, '970.720'), (1, '1000.000')] [2023-12-27 17:44:57,394][20381] Updated weights for policy 1, policy_version 1745596 (0.0005) [2023-12-27 17:44:57,767][20347] Updated weights for policy 0, policy_version 1747075 (0.0006) [2023-12-27 17:44:57,825][20347] Updated weights for policy 0, policy_version 1747085 (0.0006) [2023-12-27 17:44:57,884][20347] Updated weights for policy 0, policy_version 1747095 (0.0006) [2023-12-27 17:44:57,993][20381] Updated weights for policy 1, policy_version 1745606 (0.0005) [2023-12-27 17:44:58,048][20381] Updated weights for policy 1, policy_version 1745616 (0.0005) [2023-12-27 17:44:58,104][20381] Updated weights for policy 1, policy_version 1745626 (0.0005) [2023-12-27 17:44:58,519][20347] Updated weights for policy 0, policy_version 1747105 (0.0007) [2023-12-27 17:44:58,557][20291] KL-divergence is very high: 112.1436 [2023-12-27 17:44:58,563][20291] KL-divergence is very high: 107.9832 [2023-12-27 17:44:58,580][20347] Updated weights for policy 0, policy_version 1747115 (0.0008) [2023-12-27 17:44:58,604][20291] KL-divergence is very high: 179.5445 [2023-12-27 17:44:58,610][20291] KL-divergence is very high: 152.9594 [2023-12-27 17:44:58,643][20347] Updated weights for policy 0, policy_version 1747125 (0.0009) [2023-12-27 17:44:58,656][20291] KL-divergence is very high: 126.0880 [2023-12-27 17:44:58,712][20347] Updated weights for policy 0, policy_version 1747135 (0.0007) [2023-12-27 17:44:58,766][20381] Updated weights for policy 1, policy_version 1745636 (0.0006) [2023-12-27 17:44:58,841][20381] Updated weights for policy 1, policy_version 1745646 (0.0009) [2023-12-27 17:44:58,906][20381] Updated weights for policy 1, policy_version 1745656 (0.0006) [2023-12-27 17:44:59,495][20347] Updated weights for policy 0, policy_version 1747145 (0.0006) [2023-12-27 17:44:59,510][20381] Updated weights for policy 1, policy_version 1745666 (0.0006) [2023-12-27 17:44:59,550][20347] Updated weights for policy 0, policy_version 1747155 (0.0006) [2023-12-27 17:44:59,570][20381] Updated weights for policy 1, policy_version 1745676 (0.0007) [2023-12-27 17:44:59,603][20347] Updated weights for policy 0, policy_version 1747165 (0.0005) [2023-12-27 17:44:59,631][20381] Updated weights for policy 1, policy_version 1745686 (0.0008) [2023-12-27 17:44:59,687][20381] Updated weights for policy 1, policy_version 1745696 (0.0008) [2023-12-27 17:45:00,330][20381] Updated weights for policy 1, policy_version 1745706 (0.0008) [2023-12-27 17:45:00,346][20347] Updated weights for policy 0, policy_version 1747175 (0.0008) [2023-12-27 17:45:00,392][20381] Updated weights for policy 1, policy_version 1745716 (0.0006) [2023-12-27 17:45:00,402][20347] Updated weights for policy 0, policy_version 1747185 (0.0006) [2023-12-27 17:45:00,448][20347] Updated weights for policy 0, policy_version 1747195 (0.0007) [2023-12-27 17:45:00,453][20381] Updated weights for policy 1, policy_version 1745726 (0.0005) [2023-12-27 17:45:01,043][20381] Updated weights for policy 1, policy_version 1745736 (0.0006) [2023-12-27 17:45:01,103][20381] Updated weights for policy 1, policy_version 1745746 (0.0007) [2023-12-27 17:45:01,118][20347] Updated weights for policy 0, policy_version 1747205 (0.0009) [2023-12-27 17:45:01,163][20381] Updated weights for policy 1, policy_version 1745756 (0.0006) [2023-12-27 17:45:01,181][20347] Updated weights for policy 0, policy_version 1747215 (0.0008) [2023-12-27 17:45:01,239][20347] Updated weights for policy 0, policy_version 1747225 (0.0006) [2023-12-27 17:45:01,804][20381] Updated weights for policy 1, policy_version 1745766 (0.0007) [2023-12-27 17:45:01,864][20381] Updated weights for policy 1, policy_version 1745776 (0.0007) [2023-12-27 17:45:01,923][20381] Updated weights for policy 1, policy_version 1745786 (0.0008) [2023-12-27 17:45:01,937][20347] Updated weights for policy 0, policy_version 1747235 (0.0006) [2023-12-27 17:45:01,993][20347] Updated weights for policy 0, policy_version 1747245 (0.0005) [2023-12-27 17:45:02,051][20347] Updated weights for policy 0, policy_version 1747255 (0.0005) [2023-12-27 17:45:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 894353408. Throughput: 0: 10354.8, 1: 10291.7. Samples: 894362800. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:02,363][19170] Avg episode reward: [(0, '970.790'), (1, '990.240')] [2023-12-27 17:45:02,650][20381] Updated weights for policy 1, policy_version 1745796 (0.0006) [2023-12-27 17:45:02,655][20347] Updated weights for policy 0, policy_version 1747265 (0.0006) [2023-12-27 17:45:02,712][20347] Updated weights for policy 0, policy_version 1747275 (0.0005) [2023-12-27 17:45:02,713][20381] Updated weights for policy 1, policy_version 1745806 (0.0007) [2023-12-27 17:45:02,770][20347] Updated weights for policy 0, policy_version 1747285 (0.0005) [2023-12-27 17:45:02,771][20381] Updated weights for policy 1, policy_version 1745816 (0.0008) [2023-12-27 17:45:02,830][20347] Updated weights for policy 0, policy_version 1747295 (0.0006) [2023-12-27 17:45:03,393][20381] Updated weights for policy 1, policy_version 1745826 (0.0007) [2023-12-27 17:45:03,453][20381] Updated weights for policy 1, policy_version 1745836 (0.0007) [2023-12-27 17:45:03,494][20347] Updated weights for policy 0, policy_version 1747305 (0.0006) [2023-12-27 17:45:03,507][20381] Updated weights for policy 1, policy_version 1745846 (0.0006) [2023-12-27 17:45:03,538][20347] Updated weights for policy 0, policy_version 1747315 (0.0005) [2023-12-27 17:45:03,551][20381] Updated weights for policy 1, policy_version 1745856 (0.0005) [2023-12-27 17:45:03,584][20347] Updated weights for policy 0, policy_version 1747325 (0.0007) [2023-12-27 17:45:04,230][20381] Updated weights for policy 1, policy_version 1745866 (0.0011) [2023-12-27 17:45:04,287][20381] Updated weights for policy 1, policy_version 1745876 (0.0010) [2023-12-27 17:45:04,301][20347] Updated weights for policy 0, policy_version 1747335 (0.0007) [2023-12-27 17:45:04,347][20381] Updated weights for policy 1, policy_version 1745886 (0.0011) [2023-12-27 17:45:04,358][20347] Updated weights for policy 0, policy_version 1747345 (0.0008) [2023-12-27 17:45:04,418][20347] Updated weights for policy 0, policy_version 1747355 (0.0010) [2023-12-27 17:45:05,018][20381] Updated weights for policy 1, policy_version 1745896 (0.0010) [2023-12-27 17:45:05,070][20381] Updated weights for policy 1, policy_version 1745906 (0.0010) [2023-12-27 17:45:05,084][20347] Updated weights for policy 0, policy_version 1747365 (0.0010) [2023-12-27 17:45:05,125][20381] Updated weights for policy 1, policy_version 1745916 (0.0011) [2023-12-27 17:45:05,136][20347] Updated weights for policy 0, policy_version 1747375 (0.0010) [2023-12-27 17:45:05,193][20347] Updated weights for policy 0, policy_version 1747385 (0.0010) [2023-12-27 17:45:05,751][20381] Updated weights for policy 1, policy_version 1745926 (0.0010) [2023-12-27 17:45:05,812][20381] Updated weights for policy 1, policy_version 1745936 (0.0009) [2023-12-27 17:45:05,860][20347] Updated weights for policy 0, policy_version 1747395 (0.0010) [2023-12-27 17:45:05,868][20381] Updated weights for policy 1, policy_version 1745946 (0.0008) [2023-12-27 17:45:05,920][20347] Updated weights for policy 0, policy_version 1747405 (0.0006) [2023-12-27 17:45:05,974][20347] Updated weights for policy 0, policy_version 1747415 (0.0005) [2023-12-27 17:45:06,518][20381] Updated weights for policy 1, policy_version 1745956 (0.0011) [2023-12-27 17:45:06,575][20381] Updated weights for policy 1, policy_version 1745966 (0.0011) [2023-12-27 17:45:06,635][20381] Updated weights for policy 1, policy_version 1745976 (0.0010) [2023-12-27 17:45:06,692][20347] Updated weights for policy 0, policy_version 1747425 (0.0005) [2023-12-27 17:45:06,752][20347] Updated weights for policy 0, policy_version 1747435 (0.0007) [2023-12-27 17:45:06,810][20347] Updated weights for policy 0, policy_version 1747445 (0.0007) [2023-12-27 17:45:06,872][20347] Updated weights for policy 0, policy_version 1747455 (0.0007) [2023-12-27 17:45:07,283][20381] Updated weights for policy 1, policy_version 1745986 (0.0007) [2023-12-27 17:45:07,331][20381] Updated weights for policy 1, policy_version 1745996 (0.0010) [2023-12-27 17:45:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20522.0). Total num frames: 894451712. Throughput: 0: 10367.6, 1: 10309.0. Samples: 894424756. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:07,363][19170] Avg episode reward: [(0, '970.890'), (1, '990.240')] [2023-12-27 17:45:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001747456_447430656.pth... [2023-12-27 17:45:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001746272_447127552.pth [2023-12-27 17:45:07,377][20381] Updated weights for policy 1, policy_version 1746006 (0.0010) [2023-12-27 17:45:07,432][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001746016_447029248.pth... [2023-12-27 17:45:07,433][20381] Updated weights for policy 1, policy_version 1746016 (0.0011) [2023-12-27 17:45:07,436][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001744800_446717952.pth [2023-12-27 17:45:07,542][20347] Updated weights for policy 0, policy_version 1747465 (0.0008) [2023-12-27 17:45:07,586][20347] Updated weights for policy 0, policy_version 1747475 (0.0007) [2023-12-27 17:45:07,634][20347] Updated weights for policy 0, policy_version 1747485 (0.0008) [2023-12-27 17:45:08,142][20381] Updated weights for policy 1, policy_version 1746026 (0.0007) [2023-12-27 17:45:08,187][20381] Updated weights for policy 1, policy_version 1746036 (0.0005) [2023-12-27 17:45:08,236][20381] Updated weights for policy 1, policy_version 1746046 (0.0005) [2023-12-27 17:45:08,351][20347] Updated weights for policy 0, policy_version 1747495 (0.0008) [2023-12-27 17:45:08,412][20347] Updated weights for policy 0, policy_version 1747505 (0.0008) [2023-12-27 17:45:08,469][20347] Updated weights for policy 0, policy_version 1747515 (0.0007) [2023-12-27 17:45:08,925][20381] Updated weights for policy 1, policy_version 1746056 (0.0010) [2023-12-27 17:45:08,982][20381] Updated weights for policy 1, policy_version 1746066 (0.0011) [2023-12-27 17:45:09,043][20381] Updated weights for policy 1, policy_version 1746076 (0.0010) [2023-12-27 17:45:09,243][20347] Updated weights for policy 0, policy_version 1747525 (0.0008) [2023-12-27 17:45:09,298][20347] Updated weights for policy 0, policy_version 1747535 (0.0008) [2023-12-27 17:45:09,367][20347] Updated weights for policy 0, policy_version 1747545 (0.0008) [2023-12-27 17:45:09,779][20381] Updated weights for policy 1, policy_version 1746086 (0.0009) [2023-12-27 17:45:09,842][20381] Updated weights for policy 1, policy_version 1746096 (0.0007) [2023-12-27 17:45:09,904][20381] Updated weights for policy 1, policy_version 1746106 (0.0008) [2023-12-27 17:45:10,203][20347] Updated weights for policy 0, policy_version 1747555 (0.0007) [2023-12-27 17:45:10,262][20347] Updated weights for policy 0, policy_version 1747565 (0.0005) [2023-12-27 17:45:10,323][20347] Updated weights for policy 0, policy_version 1747575 (0.0009) [2023-12-27 17:45:10,550][20381] Updated weights for policy 1, policy_version 1746116 (0.0008) [2023-12-27 17:45:10,612][20381] Updated weights for policy 1, policy_version 1746126 (0.0006) [2023-12-27 17:45:10,671][20381] Updated weights for policy 1, policy_version 1746136 (0.0007) [2023-12-27 17:45:10,986][20347] Updated weights for policy 0, policy_version 1747585 (0.0010) [2023-12-27 17:45:11,066][20347] Updated weights for policy 0, policy_version 1747596 (0.0026) [2023-12-27 17:45:11,123][20347] Updated weights for policy 0, policy_version 1747606 (0.0006) [2023-12-27 17:45:11,190][20347] Updated weights for policy 0, policy_version 1747616 (0.0012) [2023-12-27 17:45:11,354][20381] Updated weights for policy 1, policy_version 1746146 (0.0007) [2023-12-27 17:45:11,422][20381] Updated weights for policy 1, policy_version 1746156 (0.0008) [2023-12-27 17:45:11,477][20381] Updated weights for policy 1, policy_version 1746166 (0.0007) [2023-12-27 17:45:11,527][20381] Updated weights for policy 1, policy_version 1746176 (0.0008) [2023-12-27 17:45:11,983][20347] Updated weights for policy 0, policy_version 1747626 (0.0009) [2023-12-27 17:45:12,033][20347] Updated weights for policy 0, policy_version 1747636 (0.0009) [2023-12-27 17:45:12,088][20347] Updated weights for policy 0, policy_version 1747646 (0.0008) [2023-12-27 17:45:12,342][20381] Updated weights for policy 1, policy_version 1746186 (0.0008) [2023-12-27 17:45:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 894550016. Throughput: 0: 10281.0, 1: 10360.9. Samples: 894545512. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:12,363][19170] Avg episode reward: [(0, '961.120'), (1, '1000.000')] [2023-12-27 17:45:12,405][20381] Updated weights for policy 1, policy_version 1746196 (0.0008) [2023-12-27 17:45:12,458][20381] Updated weights for policy 1, policy_version 1746206 (0.0007) [2023-12-27 17:45:12,831][20347] Updated weights for policy 0, policy_version 1747656 (0.0007) [2023-12-27 17:45:12,880][20347] Updated weights for policy 0, policy_version 1747666 (0.0007) [2023-12-27 17:45:12,944][20347] Updated weights for policy 0, policy_version 1747676 (0.0010) [2023-12-27 17:45:13,137][20381] Updated weights for policy 1, policy_version 1746216 (0.0009) [2023-12-27 17:45:13,201][20381] Updated weights for policy 1, policy_version 1746226 (0.0008) [2023-12-27 17:45:13,253][20381] Updated weights for policy 1, policy_version 1746236 (0.0008) [2023-12-27 17:45:13,536][20347] Updated weights for policy 0, policy_version 1747686 (0.0010) [2023-12-27 17:45:13,584][20347] Updated weights for policy 0, policy_version 1747696 (0.0010) [2023-12-27 17:45:13,630][20347] Updated weights for policy 0, policy_version 1747706 (0.0007) [2023-12-27 17:45:13,915][20381] Updated weights for policy 1, policy_version 1746246 (0.0008) [2023-12-27 17:45:13,967][20381] Updated weights for policy 1, policy_version 1746256 (0.0010) [2023-12-27 17:45:14,023][20381] Updated weights for policy 1, policy_version 1746266 (0.0011) [2023-12-27 17:45:14,266][20347] Updated weights for policy 0, policy_version 1747716 (0.0006) [2023-12-27 17:45:14,325][20347] Updated weights for policy 0, policy_version 1747726 (0.0008) [2023-12-27 17:45:14,377][20347] Updated weights for policy 0, policy_version 1747736 (0.0007) [2023-12-27 17:45:14,585][20381] Updated weights for policy 1, policy_version 1746276 (0.0009) [2023-12-27 17:45:14,632][20381] Updated weights for policy 1, policy_version 1746286 (0.0007) [2023-12-27 17:45:14,686][20381] Updated weights for policy 1, policy_version 1746296 (0.0007) [2023-12-27 17:45:15,082][20347] Updated weights for policy 0, policy_version 1747746 (0.0008) [2023-12-27 17:45:15,142][20347] Updated weights for policy 0, policy_version 1747756 (0.0009) [2023-12-27 17:45:15,204][20347] Updated weights for policy 0, policy_version 1747766 (0.0008) [2023-12-27 17:45:15,261][20347] Updated weights for policy 0, policy_version 1747776 (0.0008) [2023-12-27 17:45:15,356][20381] Updated weights for policy 1, policy_version 1746306 (0.0008) [2023-12-27 17:45:15,415][20381] Updated weights for policy 1, policy_version 1746316 (0.0005) [2023-12-27 17:45:15,469][20381] Updated weights for policy 1, policy_version 1746326 (0.0005) [2023-12-27 17:45:15,522][20381] Updated weights for policy 1, policy_version 1746336 (0.0005) [2023-12-27 17:45:16,017][20347] Updated weights for policy 0, policy_version 1747786 (0.0007) [2023-12-27 17:45:16,071][20381] Updated weights for policy 1, policy_version 1746346 (0.0005) [2023-12-27 17:45:16,072][20347] Updated weights for policy 0, policy_version 1747796 (0.0007) [2023-12-27 17:45:16,120][20381] Updated weights for policy 1, policy_version 1746356 (0.0005) [2023-12-27 17:45:16,128][20347] Updated weights for policy 0, policy_version 1747806 (0.0007) [2023-12-27 17:45:16,167][20381] Updated weights for policy 1, policy_version 1746366 (0.0005) [2023-12-27 17:45:16,800][20347] Updated weights for policy 0, policy_version 1747816 (0.0006) [2023-12-27 17:45:16,819][20381] Updated weights for policy 1, policy_version 1746376 (0.0005) [2023-12-27 17:45:16,861][20347] Updated weights for policy 0, policy_version 1747826 (0.0007) [2023-12-27 17:45:16,881][20381] Updated weights for policy 1, policy_version 1746386 (0.0005) [2023-12-27 17:45:16,908][20347] Updated weights for policy 0, policy_version 1747836 (0.0007) [2023-12-27 17:45:16,931][20381] Updated weights for policy 1, policy_version 1746396 (0.0006) [2023-12-27 17:45:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 894656512. Throughput: 0: 10205.3, 1: 10405.5. Samples: 894667964. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:17,363][19170] Avg episode reward: [(0, '960.930'), (1, '1000.000')] [2023-12-27 17:45:17,557][20347] Updated weights for policy 0, policy_version 1747846 (0.0006) [2023-12-27 17:45:17,597][20381] Updated weights for policy 1, policy_version 1746406 (0.0009) [2023-12-27 17:45:17,613][20347] Updated weights for policy 0, policy_version 1747856 (0.0005) [2023-12-27 17:45:17,646][20381] Updated weights for policy 1, policy_version 1746416 (0.0007) [2023-12-27 17:45:17,672][20347] Updated weights for policy 0, policy_version 1747866 (0.0005) [2023-12-27 17:45:17,701][20381] Updated weights for policy 1, policy_version 1746426 (0.0007) [2023-12-27 17:45:18,321][20347] Updated weights for policy 0, policy_version 1747876 (0.0008) [2023-12-27 17:45:18,383][20347] Updated weights for policy 0, policy_version 1747886 (0.0008) [2023-12-27 17:45:18,393][20381] Updated weights for policy 1, policy_version 1746436 (0.0008) [2023-12-27 17:45:18,444][20347] Updated weights for policy 0, policy_version 1747896 (0.0005) [2023-12-27 17:45:18,454][20381] Updated weights for policy 1, policy_version 1746446 (0.0007) [2023-12-27 17:45:18,506][20381] Updated weights for policy 1, policy_version 1746456 (0.0007) [2023-12-27 17:45:19,093][20347] Updated weights for policy 0, policy_version 1747906 (0.0006) [2023-12-27 17:45:19,154][20347] Updated weights for policy 0, policy_version 1747916 (0.0007) [2023-12-27 17:45:19,214][20347] Updated weights for policy 0, policy_version 1747926 (0.0008) [2023-12-27 17:45:19,220][20381] Updated weights for policy 1, policy_version 1746466 (0.0008) [2023-12-27 17:45:19,277][20347] Updated weights for policy 0, policy_version 1747936 (0.0008) [2023-12-27 17:45:19,281][20381] Updated weights for policy 1, policy_version 1746476 (0.0008) [2023-12-27 17:45:19,337][20381] Updated weights for policy 1, policy_version 1746486 (0.0008) [2023-12-27 17:45:19,403][20381] Updated weights for policy 1, policy_version 1746496 (0.0008) [2023-12-27 17:45:20,045][20347] Updated weights for policy 0, policy_version 1747946 (0.0006) [2023-12-27 17:45:20,107][20347] Updated weights for policy 0, policy_version 1747956 (0.0008) [2023-12-27 17:45:20,114][20381] Updated weights for policy 1, policy_version 1746506 (0.0008) [2023-12-27 17:45:20,173][20347] Updated weights for policy 0, policy_version 1747966 (0.0009) [2023-12-27 17:45:20,176][20381] Updated weights for policy 1, policy_version 1746516 (0.0007) [2023-12-27 17:45:20,243][20381] Updated weights for policy 1, policy_version 1746526 (0.0008) [2023-12-27 17:45:20,906][20347] Updated weights for policy 0, policy_version 1747976 (0.0009) [2023-12-27 17:45:20,947][20381] Updated weights for policy 1, policy_version 1746536 (0.0007) [2023-12-27 17:45:20,968][20347] Updated weights for policy 0, policy_version 1747986 (0.0006) [2023-12-27 17:45:21,007][20381] Updated weights for policy 1, policy_version 1746546 (0.0007) [2023-12-27 17:45:21,029][20347] Updated weights for policy 0, policy_version 1747996 (0.0007) [2023-12-27 17:45:21,069][20381] Updated weights for policy 1, policy_version 1746556 (0.0007) [2023-12-27 17:45:21,746][20347] Updated weights for policy 0, policy_version 1748006 (0.0007) [2023-12-27 17:45:21,807][20347] Updated weights for policy 0, policy_version 1748016 (0.0006) [2023-12-27 17:45:21,843][20381] Updated weights for policy 1, policy_version 1746566 (0.0010) [2023-12-27 17:45:21,870][20347] Updated weights for policy 0, policy_version 1748026 (0.0006) [2023-12-27 17:45:21,908][20381] Updated weights for policy 1, policy_version 1746576 (0.0008) [2023-12-27 17:45:21,968][20381] Updated weights for policy 1, policy_version 1746586 (0.0008) [2023-12-27 17:45:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 894754816. Throughput: 0: 10179.3, 1: 10357.7. Samples: 894728572. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:22,363][19170] Avg episode reward: [(0, '990.220'), (1, '1000.000')] [2023-12-27 17:45:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001748032_447578112.pth... [2023-12-27 17:45:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001746592_447176704.pth... [2023-12-27 17:45:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001746848_447275008.pth [2023-12-27 17:45:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001745376_446865408.pth [2023-12-27 17:45:22,549][20347] Updated weights for policy 0, policy_version 1748036 (0.0006) [2023-12-27 17:45:22,607][20347] Updated weights for policy 0, policy_version 1748046 (0.0006) [2023-12-27 17:45:22,667][20381] Updated weights for policy 1, policy_version 1746596 (0.0008) [2023-12-27 17:45:22,673][20347] Updated weights for policy 0, policy_version 1748056 (0.0006) [2023-12-27 17:45:22,725][20381] Updated weights for policy 1, policy_version 1746606 (0.0007) [2023-12-27 17:45:22,778][20381] Updated weights for policy 1, policy_version 1746616 (0.0008) [2023-12-27 17:45:23,252][20347] Updated weights for policy 0, policy_version 1748066 (0.0007) [2023-12-27 17:45:23,307][20347] Updated weights for policy 0, policy_version 1748076 (0.0008) [2023-12-27 17:45:23,356][20347] Updated weights for policy 0, policy_version 1748086 (0.0008) [2023-12-27 17:45:23,410][20347] Updated weights for policy 0, policy_version 1748096 (0.0008) [2023-12-27 17:45:23,413][20381] Updated weights for policy 1, policy_version 1746626 (0.0006) [2023-12-27 17:45:23,466][20381] Updated weights for policy 1, policy_version 1746636 (0.0005) [2023-12-27 17:45:23,524][20381] Updated weights for policy 1, policy_version 1746646 (0.0006) [2023-12-27 17:45:23,572][20381] Updated weights for policy 1, policy_version 1746656 (0.0007) [2023-12-27 17:45:24,123][20347] Updated weights for policy 0, policy_version 1748106 (0.0007) [2023-12-27 17:45:24,176][20347] Updated weights for policy 0, policy_version 1748116 (0.0006) [2023-12-27 17:45:24,182][20381] Updated weights for policy 1, policy_version 1746666 (0.0011) [2023-12-27 17:45:24,229][20347] Updated weights for policy 0, policy_version 1748126 (0.0008) [2023-12-27 17:45:24,233][20381] Updated weights for policy 1, policy_version 1746676 (0.0011) [2023-12-27 17:45:24,286][20381] Updated weights for policy 1, policy_version 1746686 (0.0011) [2023-12-27 17:45:24,850][20347] Updated weights for policy 0, policy_version 1748136 (0.0007) [2023-12-27 17:45:24,899][20347] Updated weights for policy 0, policy_version 1748146 (0.0007) [2023-12-27 17:45:24,946][20347] Updated weights for policy 0, policy_version 1748156 (0.0007) [2023-12-27 17:45:24,984][20381] Updated weights for policy 1, policy_version 1746696 (0.0008) [2023-12-27 17:45:25,033][20381] Updated weights for policy 1, policy_version 1746706 (0.0008) [2023-12-27 17:45:25,094][20381] Updated weights for policy 1, policy_version 1746716 (0.0007) [2023-12-27 17:45:25,602][20347] Updated weights for policy 0, policy_version 1748166 (0.0005) [2023-12-27 17:45:25,660][20347] Updated weights for policy 0, policy_version 1748176 (0.0006) [2023-12-27 17:45:25,714][20381] Updated weights for policy 1, policy_version 1746726 (0.0006) [2023-12-27 17:45:25,720][20347] Updated weights for policy 0, policy_version 1748186 (0.0009) [2023-12-27 17:45:25,769][20381] Updated weights for policy 1, policy_version 1746736 (0.0006) [2023-12-27 17:45:25,820][20381] Updated weights for policy 1, policy_version 1746746 (0.0005) [2023-12-27 17:45:26,331][20347] Updated weights for policy 0, policy_version 1748196 (0.0010) [2023-12-27 17:45:26,357][20381] Updated weights for policy 1, policy_version 1746756 (0.0005) [2023-12-27 17:45:26,377][20347] Updated weights for policy 0, policy_version 1748206 (0.0010) [2023-12-27 17:45:26,413][20381] Updated weights for policy 1, policy_version 1746766 (0.0006) [2023-12-27 17:45:26,436][20347] Updated weights for policy 0, policy_version 1748216 (0.0010) [2023-12-27 17:45:26,469][20381] Updated weights for policy 1, policy_version 1746776 (0.0008) [2023-12-27 17:45:26,997][20347] Updated weights for policy 0, policy_version 1748226 (0.0006) [2023-12-27 17:45:27,048][20347] Updated weights for policy 0, policy_version 1748236 (0.0008) [2023-12-27 17:45:27,076][20381] Updated weights for policy 1, policy_version 1746786 (0.0006) [2023-12-27 17:45:27,102][20347] Updated weights for policy 0, policy_version 1748246 (0.0010) [2023-12-27 17:45:27,125][20381] Updated weights for policy 1, policy_version 1746796 (0.0011) [2023-12-27 17:45:27,149][20347] Updated weights for policy 0, policy_version 1748256 (0.0010) [2023-12-27 17:45:27,176][20381] Updated weights for policy 1, policy_version 1746806 (0.0010) [2023-12-27 17:45:27,237][20381] Updated weights for policy 1, policy_version 1746816 (0.0010) [2023-12-27 17:45:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 894869504. Throughput: 0: 10173.6, 1: 10399.5. Samples: 894853560. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:45:27,753][20347] Updated weights for policy 0, policy_version 1748266 (0.0009) [2023-12-27 17:45:27,797][20347] Updated weights for policy 0, policy_version 1748276 (0.0009) [2023-12-27 17:45:27,842][20347] Updated weights for policy 0, policy_version 1748286 (0.0010) [2023-12-27 17:45:27,882][20381] Updated weights for policy 1, policy_version 1746826 (0.0007) [2023-12-27 17:45:27,937][20381] Updated weights for policy 1, policy_version 1746836 (0.0008) [2023-12-27 17:45:27,991][20381] Updated weights for policy 1, policy_version 1746846 (0.0007) [2023-12-27 17:45:28,436][20347] Updated weights for policy 0, policy_version 1748296 (0.0008) [2023-12-27 17:45:28,511][20347] Updated weights for policy 0, policy_version 1748306 (0.0008) [2023-12-27 17:45:28,562][20347] Updated weights for policy 0, policy_version 1748316 (0.0008) [2023-12-27 17:45:28,578][20381] Updated weights for policy 1, policy_version 1746856 (0.0007) [2023-12-27 17:45:28,636][20381] Updated weights for policy 1, policy_version 1746866 (0.0006) [2023-12-27 17:45:28,699][20381] Updated weights for policy 1, policy_version 1746876 (0.0008) [2023-12-27 17:45:29,182][20347] Updated weights for policy 0, policy_version 1748326 (0.0007) [2023-12-27 17:45:29,241][20347] Updated weights for policy 0, policy_version 1748336 (0.0009) [2023-12-27 17:45:29,298][20347] Updated weights for policy 0, policy_version 1748346 (0.0009) [2023-12-27 17:45:29,417][20381] Updated weights for policy 1, policy_version 1746886 (0.0007) [2023-12-27 17:45:29,466][20381] Updated weights for policy 1, policy_version 1746896 (0.0007) [2023-12-27 17:45:29,517][20381] Updated weights for policy 1, policy_version 1746906 (0.0007) [2023-12-27 17:45:30,019][20347] Updated weights for policy 0, policy_version 1748356 (0.0007) [2023-12-27 17:45:30,070][20347] Updated weights for policy 0, policy_version 1748366 (0.0007) [2023-12-27 17:45:30,120][20347] Updated weights for policy 0, policy_version 1748376 (0.0008) [2023-12-27 17:45:30,221][20381] Updated weights for policy 1, policy_version 1746916 (0.0008) [2023-12-27 17:45:30,279][20381] Updated weights for policy 1, policy_version 1746926 (0.0008) [2023-12-27 17:45:30,337][20381] Updated weights for policy 1, policy_version 1746936 (0.0008) [2023-12-27 17:45:30,732][20347] Updated weights for policy 0, policy_version 1748386 (0.0007) [2023-12-27 17:45:30,783][20347] Updated weights for policy 0, policy_version 1748396 (0.0005) [2023-12-27 17:45:30,836][20347] Updated weights for policy 0, policy_version 1748406 (0.0006) [2023-12-27 17:45:30,884][20347] Updated weights for policy 0, policy_version 1748416 (0.0007) [2023-12-27 17:45:31,039][20381] Updated weights for policy 1, policy_version 1746946 (0.0008) [2023-12-27 17:45:31,092][20381] Updated weights for policy 1, policy_version 1746956 (0.0007) [2023-12-27 17:45:31,153][20381] Updated weights for policy 1, policy_version 1746966 (0.0009) [2023-12-27 17:45:31,211][20381] Updated weights for policy 1, policy_version 1746976 (0.0008) [2023-12-27 17:45:31,590][20347] Updated weights for policy 0, policy_version 1748426 (0.0006) [2023-12-27 17:45:31,655][20347] Updated weights for policy 0, policy_version 1748436 (0.0008) [2023-12-27 17:45:31,715][20347] Updated weights for policy 0, policy_version 1748446 (0.0008) [2023-12-27 17:45:31,906][20381] Updated weights for policy 1, policy_version 1746986 (0.0008) [2023-12-27 17:45:31,961][20381] Updated weights for policy 1, policy_version 1746996 (0.0008) [2023-12-27 17:45:32,014][20381] Updated weights for policy 1, policy_version 1747006 (0.0008) [2023-12-27 17:45:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 894967808. Throughput: 0: 10223.2, 1: 10435.8. Samples: 894979820. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '1000.000')] [2023-12-27 17:45:32,422][20347] Updated weights for policy 0, policy_version 1748456 (0.0007) [2023-12-27 17:45:32,482][20347] Updated weights for policy 0, policy_version 1748466 (0.0007) [2023-12-27 17:45:32,534][20347] Updated weights for policy 0, policy_version 1748476 (0.0007) [2023-12-27 17:45:32,711][20381] Updated weights for policy 1, policy_version 1747016 (0.0006) [2023-12-27 17:45:32,772][20381] Updated weights for policy 1, policy_version 1747026 (0.0005) [2023-12-27 17:45:32,839][20381] Updated weights for policy 1, policy_version 1747036 (0.0007) [2023-12-27 17:45:33,162][20347] Updated weights for policy 0, policy_version 1748486 (0.0006) [2023-12-27 17:45:33,227][20347] Updated weights for policy 0, policy_version 1748496 (0.0006) [2023-12-27 17:45:33,289][20347] Updated weights for policy 0, policy_version 1748506 (0.0008) [2023-12-27 17:45:33,396][20381] Updated weights for policy 1, policy_version 1747046 (0.0007) [2023-12-27 17:45:33,442][20381] Updated weights for policy 1, policy_version 1747056 (0.0005) [2023-12-27 17:45:33,484][20381] Updated weights for policy 1, policy_version 1747066 (0.0005) [2023-12-27 17:45:33,508][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000009 [2023-12-27 17:45:33,952][20347] Updated weights for policy 0, policy_version 1748516 (0.0007) [2023-12-27 17:45:34,007][20347] Updated weights for policy 0, policy_version 1748526 (0.0007) [2023-12-27 17:45:34,058][20381] Updated weights for policy 1, policy_version 1747076 (0.0005) [2023-12-27 17:45:34,063][20347] Updated weights for policy 0, policy_version 1748536 (0.0007) [2023-12-27 17:45:34,116][20381] Updated weights for policy 1, policy_version 1747086 (0.0007) [2023-12-27 17:45:34,183][20381] Updated weights for policy 1, policy_version 1747096 (0.0007) [2023-12-27 17:45:34,782][20347] Updated weights for policy 0, policy_version 1748546 (0.0006) [2023-12-27 17:45:34,807][20381] Updated weights for policy 1, policy_version 1747106 (0.0007) [2023-12-27 17:45:34,831][20347] Updated weights for policy 0, policy_version 1748556 (0.0007) [2023-12-27 17:45:34,843][20291] KL-divergence is very high: 163.8148 [2023-12-27 17:45:34,858][20381] Updated weights for policy 1, policy_version 1747116 (0.0005) [2023-12-27 17:45:34,873][20347] Updated weights for policy 0, policy_version 1748566 (0.0007) [2023-12-27 17:45:34,878][20291] KL-divergence is very high: 238.7374 [2023-12-27 17:45:34,907][20381] Updated weights for policy 1, policy_version 1747126 (0.0006) [2023-12-27 17:45:34,916][20291] KL-divergence is very high: 167.1844 [2023-12-27 17:45:34,920][20347] Updated weights for policy 0, policy_version 1748576 (0.0007) [2023-12-27 17:45:34,956][20381] Updated weights for policy 1, policy_version 1747136 (0.0006) [2023-12-27 17:45:35,540][20381] Updated weights for policy 1, policy_version 1747146 (0.0005) [2023-12-27 17:45:35,593][20381] Updated weights for policy 1, policy_version 1747156 (0.0005) [2023-12-27 17:45:35,610][20347] Updated weights for policy 0, policy_version 1748586 (0.0005) [2023-12-27 17:45:35,644][20381] Updated weights for policy 1, policy_version 1747166 (0.0005) [2023-12-27 17:45:35,657][20347] Updated weights for policy 0, policy_version 1748596 (0.0005) [2023-12-27 17:45:35,708][20347] Updated weights for policy 0, policy_version 1748606 (0.0007) [2023-12-27 17:45:36,216][20381] Updated weights for policy 1, policy_version 1747176 (0.0008) [2023-12-27 17:45:36,285][20381] Updated weights for policy 1, policy_version 1747186 (0.0008) [2023-12-27 17:45:36,351][20381] Updated weights for policy 1, policy_version 1747196 (0.0008) [2023-12-27 17:45:36,354][20347] Updated weights for policy 0, policy_version 1748616 (0.0008) [2023-12-27 17:45:36,410][20347] Updated weights for policy 0, policy_version 1748626 (0.0008) [2023-12-27 17:45:36,469][20347] Updated weights for policy 0, policy_version 1748636 (0.0008) [2023-12-27 17:45:37,053][20381] Updated weights for policy 1, policy_version 1747206 (0.0009) [2023-12-27 17:45:37,102][20381] Updated weights for policy 1, policy_version 1747216 (0.0011) [2023-12-27 17:45:37,154][20381] Updated weights for policy 1, policy_version 1747226 (0.0011) [2023-12-27 17:45:37,181][20347] Updated weights for policy 0, policy_version 1748646 (0.0006) [2023-12-27 17:45:37,238][20347] Updated weights for policy 0, policy_version 1748656 (0.0005) [2023-12-27 17:45:37,294][20347] Updated weights for policy 0, policy_version 1748666 (0.0005) [2023-12-27 17:45:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 895082496. Throughput: 0: 10211.7, 1: 10526.8. Samples: 895043992. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:37,363][19170] Avg episode reward: [(0, '990.240'), (1, '990.230')] [2023-12-27 17:45:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001747232_447340544.pth... [2023-12-27 17:45:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001748672_447741952.pth... [2023-12-27 17:45:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001746016_447029248.pth [2023-12-27 17:45:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001747456_447430656.pth [2023-12-27 17:45:37,822][20381] Updated weights for policy 1, policy_version 1747236 (0.0011) [2023-12-27 17:45:37,874][20347] Updated weights for policy 0, policy_version 1748676 (0.0006) [2023-12-27 17:45:37,877][20381] Updated weights for policy 1, policy_version 1747246 (0.0011) [2023-12-27 17:45:37,930][20347] Updated weights for policy 0, policy_version 1748686 (0.0006) [2023-12-27 17:45:37,937][20381] Updated weights for policy 1, policy_version 1747256 (0.0011) [2023-12-27 17:45:37,979][20347] Updated weights for policy 0, policy_version 1748696 (0.0007) [2023-12-27 17:45:38,615][20381] Updated weights for policy 1, policy_version 1747266 (0.0011) [2023-12-27 17:45:38,666][20347] Updated weights for policy 0, policy_version 1748706 (0.0006) [2023-12-27 17:45:38,676][20381] Updated weights for policy 1, policy_version 1747276 (0.0010) [2023-12-27 17:45:38,715][20347] Updated weights for policy 0, policy_version 1748716 (0.0005) [2023-12-27 17:45:38,728][20381] Updated weights for policy 1, policy_version 1747286 (0.0011) [2023-12-27 17:45:38,774][20347] Updated weights for policy 0, policy_version 1748726 (0.0006) [2023-12-27 17:45:38,785][20381] Updated weights for policy 1, policy_version 1747296 (0.0009) [2023-12-27 17:45:38,838][20347] Updated weights for policy 0, policy_version 1748736 (0.0008) [2023-12-27 17:45:39,507][20381] Updated weights for policy 1, policy_version 1747306 (0.0007) [2023-12-27 17:45:39,521][20347] Updated weights for policy 0, policy_version 1748746 (0.0008) [2023-12-27 17:45:39,574][20381] Updated weights for policy 1, policy_version 1747316 (0.0008) [2023-12-27 17:45:39,584][20347] Updated weights for policy 0, policy_version 1748756 (0.0008) [2023-12-27 17:45:39,633][20381] Updated weights for policy 1, policy_version 1747326 (0.0006) [2023-12-27 17:45:39,642][20347] Updated weights for policy 0, policy_version 1748766 (0.0007) [2023-12-27 17:45:40,339][20347] Updated weights for policy 0, policy_version 1748776 (0.0007) [2023-12-27 17:45:40,404][20347] Updated weights for policy 0, policy_version 1748786 (0.0007) [2023-12-27 17:45:40,409][20381] Updated weights for policy 1, policy_version 1747336 (0.0007) [2023-12-27 17:45:40,461][20347] Updated weights for policy 0, policy_version 1748796 (0.0007) [2023-12-27 17:45:40,479][20381] Updated weights for policy 1, policy_version 1747346 (0.0006) [2023-12-27 17:45:40,545][20381] Updated weights for policy 1, policy_version 1747356 (0.0008) [2023-12-27 17:45:41,242][20347] Updated weights for policy 0, policy_version 1748806 (0.0007) [2023-12-27 17:45:41,269][20381] Updated weights for policy 1, policy_version 1747366 (0.0008) [2023-12-27 17:45:41,298][20347] Updated weights for policy 0, policy_version 1748816 (0.0008) [2023-12-27 17:45:41,326][20381] Updated weights for policy 1, policy_version 1747376 (0.0008) [2023-12-27 17:45:41,358][20347] Updated weights for policy 0, policy_version 1748826 (0.0008) [2023-12-27 17:45:41,399][20381] Updated weights for policy 1, policy_version 1747386 (0.0008) [2023-12-27 17:45:42,143][20381] Updated weights for policy 1, policy_version 1747396 (0.0007) [2023-12-27 17:45:42,157][20347] Updated weights for policy 0, policy_version 1748836 (0.0008) [2023-12-27 17:45:42,206][20381] Updated weights for policy 1, policy_version 1747406 (0.0008) [2023-12-27 17:45:42,215][20347] Updated weights for policy 0, policy_version 1748846 (0.0006) [2023-12-27 17:45:42,270][20381] Updated weights for policy 1, policy_version 1747416 (0.0007) [2023-12-27 17:45:42,275][20347] Updated weights for policy 0, policy_version 1748856 (0.0007) [2023-12-27 17:45:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 895180800. Throughput: 0: 10222.6, 1: 10448.1. Samples: 895167052. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:42,363][19170] Avg episode reward: [(0, '990.240'), (1, '990.230')] [2023-12-27 17:45:42,971][20347] Updated weights for policy 0, policy_version 1748866 (0.0007) [2023-12-27 17:45:42,995][20381] Updated weights for policy 1, policy_version 1747426 (0.0008) [2023-12-27 17:45:43,031][20347] Updated weights for policy 0, policy_version 1748876 (0.0007) [2023-12-27 17:45:43,061][20381] Updated weights for policy 1, policy_version 1747436 (0.0011) [2023-12-27 17:45:43,085][20347] Updated weights for policy 0, policy_version 1748886 (0.0006) [2023-12-27 17:45:43,127][20381] Updated weights for policy 1, policy_version 1747446 (0.0011) [2023-12-27 17:45:43,145][20347] Updated weights for policy 0, policy_version 1748896 (0.0006) [2023-12-27 17:45:43,190][20381] Updated weights for policy 1, policy_version 1747456 (0.0009) [2023-12-27 17:45:43,789][20347] Updated weights for policy 0, policy_version 1748906 (0.0008) [2023-12-27 17:45:43,817][20381] Updated weights for policy 1, policy_version 1747466 (0.0011) [2023-12-27 17:45:43,843][20347] Updated weights for policy 0, policy_version 1748916 (0.0006) [2023-12-27 17:45:43,863][20381] Updated weights for policy 1, policy_version 1747476 (0.0010) [2023-12-27 17:45:43,895][20347] Updated weights for policy 0, policy_version 1748926 (0.0005) [2023-12-27 17:45:43,912][20381] Updated weights for policy 1, policy_version 1747486 (0.0011) [2023-12-27 17:45:44,559][20347] Updated weights for policy 0, policy_version 1748936 (0.0006) [2023-12-27 17:45:44,613][20347] Updated weights for policy 0, policy_version 1748946 (0.0006) [2023-12-27 17:45:44,633][20381] Updated weights for policy 1, policy_version 1747496 (0.0008) [2023-12-27 17:45:44,661][20347] Updated weights for policy 0, policy_version 1748956 (0.0005) [2023-12-27 17:45:44,692][20381] Updated weights for policy 1, policy_version 1747506 (0.0008) [2023-12-27 17:45:44,738][20381] Updated weights for policy 1, policy_version 1747516 (0.0009) [2023-12-27 17:45:45,366][20347] Updated weights for policy 0, policy_version 1748966 (0.0007) [2023-12-27 17:45:45,427][20347] Updated weights for policy 0, policy_version 1748976 (0.0007) [2023-12-27 17:45:45,474][20381] Updated weights for policy 1, policy_version 1747526 (0.0007) [2023-12-27 17:45:45,484][20347] Updated weights for policy 0, policy_version 1748986 (0.0006) [2023-12-27 17:45:45,534][20381] Updated weights for policy 1, policy_version 1747536 (0.0008) [2023-12-27 17:45:45,591][20381] Updated weights for policy 1, policy_version 1747546 (0.0008) [2023-12-27 17:45:46,172][20347] Updated weights for policy 0, policy_version 1748996 (0.0007) [2023-12-27 17:45:46,218][20347] Updated weights for policy 0, policy_version 1749006 (0.0007) [2023-12-27 17:45:46,275][20347] Updated weights for policy 0, policy_version 1749016 (0.0007) [2023-12-27 17:45:46,300][20381] Updated weights for policy 1, policy_version 1747556 (0.0007) [2023-12-27 17:45:46,358][20381] Updated weights for policy 1, policy_version 1747566 (0.0007) [2023-12-27 17:45:46,419][20381] Updated weights for policy 1, policy_version 1747576 (0.0007) [2023-12-27 17:45:46,952][20347] Updated weights for policy 0, policy_version 1749026 (0.0007) [2023-12-27 17:45:47,013][20347] Updated weights for policy 0, policy_version 1749036 (0.0007) [2023-12-27 17:45:47,068][20347] Updated weights for policy 0, policy_version 1749046 (0.0008) [2023-12-27 17:45:47,126][20347] Updated weights for policy 0, policy_version 1749056 (0.0007) [2023-12-27 17:45:47,145][20381] Updated weights for policy 1, policy_version 1747586 (0.0008) [2023-12-27 17:45:47,201][20381] Updated weights for policy 1, policy_version 1747596 (0.0007) [2023-12-27 17:45:47,251][20381] Updated weights for policy 1, policy_version 1747606 (0.0008) [2023-12-27 17:45:47,302][20381] Updated weights for policy 1, policy_version 1747616 (0.0008) [2023-12-27 17:45:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 895279104. Throughput: 0: 10195.2, 1: 10302.7. Samples: 895285204. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:47,363][19170] Avg episode reward: [(0, '970.640'), (1, '1000.000')] [2023-12-27 17:45:47,849][20347] Updated weights for policy 0, policy_version 1749066 (0.0008) [2023-12-27 17:45:47,907][20347] Updated weights for policy 0, policy_version 1749076 (0.0008) [2023-12-27 17:45:47,926][20381] Updated weights for policy 1, policy_version 1747626 (0.0006) [2023-12-27 17:45:47,966][20347] Updated weights for policy 0, policy_version 1749086 (0.0008) [2023-12-27 17:45:47,993][20381] Updated weights for policy 1, policy_version 1747636 (0.0006) [2023-12-27 17:45:48,059][20381] Updated weights for policy 1, policy_version 1747646 (0.0006) [2023-12-27 17:45:48,686][20347] Updated weights for policy 0, policy_version 1749096 (0.0007) [2023-12-27 17:45:48,722][20381] Updated weights for policy 1, policy_version 1747656 (0.0008) [2023-12-27 17:45:48,750][20347] Updated weights for policy 0, policy_version 1749106 (0.0006) [2023-12-27 17:45:48,787][20381] Updated weights for policy 1, policy_version 1747666 (0.0006) [2023-12-27 17:45:48,808][20347] Updated weights for policy 0, policy_version 1749116 (0.0006) [2023-12-27 17:45:48,849][20381] Updated weights for policy 1, policy_version 1747676 (0.0006) [2023-12-27 17:45:49,539][20347] Updated weights for policy 0, policy_version 1749126 (0.0007) [2023-12-27 17:45:49,567][20381] Updated weights for policy 1, policy_version 1747686 (0.0007) [2023-12-27 17:45:49,603][20347] Updated weights for policy 0, policy_version 1749136 (0.0008) [2023-12-27 17:45:49,620][20381] Updated weights for policy 1, policy_version 1747696 (0.0005) [2023-12-27 17:45:49,659][20347] Updated weights for policy 0, policy_version 1749146 (0.0007) [2023-12-27 17:45:49,675][20381] Updated weights for policy 1, policy_version 1747706 (0.0006) [2023-12-27 17:45:50,330][20381] Updated weights for policy 1, policy_version 1747716 (0.0009) [2023-12-27 17:45:50,386][20381] Updated weights for policy 1, policy_version 1747726 (0.0008) [2023-12-27 17:45:50,408][20347] Updated weights for policy 0, policy_version 1749156 (0.0007) [2023-12-27 17:45:50,446][20381] Updated weights for policy 1, policy_version 1747736 (0.0008) [2023-12-27 17:45:50,465][20347] Updated weights for policy 0, policy_version 1749166 (0.0007) [2023-12-27 17:45:50,514][20347] Updated weights for policy 0, policy_version 1749176 (0.0008) [2023-12-27 17:45:51,072][20381] Updated weights for policy 1, policy_version 1747746 (0.0010) [2023-12-27 17:45:51,140][20381] Updated weights for policy 1, policy_version 1747756 (0.0008) [2023-12-27 17:45:51,200][20381] Updated weights for policy 1, policy_version 1747766 (0.0011) [2023-12-27 17:45:51,255][20381] Updated weights for policy 1, policy_version 1747776 (0.0009) [2023-12-27 17:45:51,295][20347] Updated weights for policy 0, policy_version 1749186 (0.0007) [2023-12-27 17:45:51,370][20347] Updated weights for policy 0, policy_version 1749196 (0.0008) [2023-12-27 17:45:51,427][20347] Updated weights for policy 0, policy_version 1749206 (0.0008) [2023-12-27 17:45:51,480][20347] Updated weights for policy 0, policy_version 1749216 (0.0009) [2023-12-27 17:45:51,960][20381] Updated weights for policy 1, policy_version 1747786 (0.0006) [2023-12-27 17:45:52,022][20381] Updated weights for policy 1, policy_version 1747796 (0.0007) [2023-12-27 17:45:52,081][20381] Updated weights for policy 1, policy_version 1747806 (0.0008) [2023-12-27 17:45:52,195][20347] Updated weights for policy 0, policy_version 1749226 (0.0005) [2023-12-27 17:45:52,261][20347] Updated weights for policy 0, policy_version 1749236 (0.0007) [2023-12-27 17:45:52,329][20347] Updated weights for policy 0, policy_version 1749246 (0.0010) [2023-12-27 17:45:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 895377408. Throughput: 0: 10135.6, 1: 10300.4. Samples: 895344376. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:52,363][19170] Avg episode reward: [(0, '951.080'), (1, '1000.000')] [2023-12-27 17:45:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001747808_447488000.pth... [2023-12-27 17:45:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001749248_447889408.pth... [2023-12-27 17:45:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001746592_447176704.pth [2023-12-27 17:45:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001748032_447578112.pth [2023-12-27 17:45:52,772][20381] Updated weights for policy 1, policy_version 1747816 (0.0006) [2023-12-27 17:45:52,832][20381] Updated weights for policy 1, policy_version 1747826 (0.0008) [2023-12-27 17:45:52,883][20381] Updated weights for policy 1, policy_version 1747836 (0.0010) [2023-12-27 17:45:52,969][20347] Updated weights for policy 0, policy_version 1749256 (0.0008) [2023-12-27 17:45:53,017][20347] Updated weights for policy 0, policy_version 1749266 (0.0008) [2023-12-27 17:45:53,071][20347] Updated weights for policy 0, policy_version 1749276 (0.0008) [2023-12-27 17:45:53,468][20381] Updated weights for policy 1, policy_version 1747846 (0.0010) [2023-12-27 17:45:53,511][20381] Updated weights for policy 1, policy_version 1747856 (0.0010) [2023-12-27 17:45:53,558][20381] Updated weights for policy 1, policy_version 1747866 (0.0005) [2023-12-27 17:45:53,712][20347] Updated weights for policy 0, policy_version 1749286 (0.0008) [2023-12-27 17:45:53,770][20347] Updated weights for policy 0, policy_version 1749296 (0.0008) [2023-12-27 17:45:53,823][20347] Updated weights for policy 0, policy_version 1749306 (0.0007) [2023-12-27 17:45:54,215][20381] Updated weights for policy 1, policy_version 1747876 (0.0006) [2023-12-27 17:45:54,269][20381] Updated weights for policy 1, policy_version 1747886 (0.0006) [2023-12-27 17:45:54,323][20381] Updated weights for policy 1, policy_version 1747896 (0.0008) [2023-12-27 17:45:54,464][20347] Updated weights for policy 0, policy_version 1749316 (0.0007) [2023-12-27 17:45:54,515][20347] Updated weights for policy 0, policy_version 1749326 (0.0007) [2023-12-27 17:45:54,571][20347] Updated weights for policy 0, policy_version 1749336 (0.0008) [2023-12-27 17:45:54,983][20381] Updated weights for policy 1, policy_version 1747906 (0.0008) [2023-12-27 17:45:55,036][20381] Updated weights for policy 1, policy_version 1747916 (0.0008) [2023-12-27 17:45:55,095][20381] Updated weights for policy 1, policy_version 1747926 (0.0008) [2023-12-27 17:45:55,154][20381] Updated weights for policy 1, policy_version 1747936 (0.0008) [2023-12-27 17:45:55,242][20347] Updated weights for policy 0, policy_version 1749346 (0.0007) [2023-12-27 17:45:55,299][20347] Updated weights for policy 0, policy_version 1749356 (0.0005) [2023-12-27 17:45:55,354][20347] Updated weights for policy 0, policy_version 1749366 (0.0005) [2023-12-27 17:45:55,414][20347] Updated weights for policy 0, policy_version 1749376 (0.0005) [2023-12-27 17:45:55,868][20381] Updated weights for policy 1, policy_version 1747946 (0.0007) [2023-12-27 17:45:55,902][20347] Updated weights for policy 0, policy_version 1749386 (0.0007) [2023-12-27 17:45:55,917][20381] Updated weights for policy 1, policy_version 1747956 (0.0005) [2023-12-27 17:45:55,957][20347] Updated weights for policy 0, policy_version 1749396 (0.0008) [2023-12-27 17:45:55,976][20381] Updated weights for policy 1, policy_version 1747966 (0.0006) [2023-12-27 17:45:56,006][20347] Updated weights for policy 0, policy_version 1749406 (0.0007) [2023-12-27 17:45:56,534][20381] Updated weights for policy 1, policy_version 1747976 (0.0007) [2023-12-27 17:45:56,592][20381] Updated weights for policy 1, policy_version 1747986 (0.0007) [2023-12-27 17:45:56,630][20347] Updated weights for policy 0, policy_version 1749416 (0.0007) [2023-12-27 17:45:56,642][20381] Updated weights for policy 1, policy_version 1747996 (0.0009) [2023-12-27 17:45:56,680][20347] Updated weights for policy 0, policy_version 1749426 (0.0008) [2023-12-27 17:45:56,737][20347] Updated weights for policy 0, policy_version 1749436 (0.0008) [2023-12-27 17:45:57,278][20381] Updated weights for policy 1, policy_version 1748006 (0.0007) [2023-12-27 17:45:57,330][20381] Updated weights for policy 1, policy_version 1748016 (0.0007) [2023-12-27 17:45:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 895475712. Throughput: 0: 10218.7, 1: 10318.9. Samples: 895469700. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:45:57,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:45:57,379][20381] Updated weights for policy 1, policy_version 1748026 (0.0006) [2023-12-27 17:45:57,393][20347] Updated weights for policy 0, policy_version 1749446 (0.0010) [2023-12-27 17:45:57,441][20347] Updated weights for policy 0, policy_version 1749456 (0.0010) [2023-12-27 17:45:57,493][20347] Updated weights for policy 0, policy_version 1749466 (0.0010) [2023-12-27 17:45:57,974][20381] Updated weights for policy 1, policy_version 1748036 (0.0006) [2023-12-27 17:45:58,020][20381] Updated weights for policy 1, policy_version 1748046 (0.0008) [2023-12-27 17:45:58,065][20381] Updated weights for policy 1, policy_version 1748056 (0.0007) [2023-12-27 17:45:58,134][20347] Updated weights for policy 0, policy_version 1749476 (0.0009) [2023-12-27 17:45:58,184][20347] Updated weights for policy 0, policy_version 1749486 (0.0010) [2023-12-27 17:45:58,251][20347] Updated weights for policy 0, policy_version 1749496 (0.0009) [2023-12-27 17:45:58,869][20381] Updated weights for policy 1, policy_version 1748066 (0.0008) [2023-12-27 17:45:58,935][20381] Updated weights for policy 1, policy_version 1748076 (0.0010) [2023-12-27 17:45:58,988][20381] Updated weights for policy 1, policy_version 1748086 (0.0009) [2023-12-27 17:45:59,032][20347] Updated weights for policy 0, policy_version 1749506 (0.0010) [2023-12-27 17:45:59,043][20381] Updated weights for policy 1, policy_version 1748096 (0.0007) [2023-12-27 17:45:59,088][20347] Updated weights for policy 0, policy_version 1749516 (0.0009) [2023-12-27 17:45:59,149][20347] Updated weights for policy 0, policy_version 1749526 (0.0010) [2023-12-27 17:45:59,205][20347] Updated weights for policy 0, policy_version 1749536 (0.0010) [2023-12-27 17:45:59,780][20381] Updated weights for policy 1, policy_version 1748106 (0.0006) [2023-12-27 17:45:59,842][20381] Updated weights for policy 1, policy_version 1748116 (0.0007) [2023-12-27 17:45:59,904][20381] Updated weights for policy 1, policy_version 1748126 (0.0009) [2023-12-27 17:45:59,919][20347] Updated weights for policy 0, policy_version 1749546 (0.0007) [2023-12-27 17:45:59,981][20347] Updated weights for policy 0, policy_version 1749556 (0.0007) [2023-12-27 17:46:00,037][20347] Updated weights for policy 0, policy_version 1749566 (0.0007) [2023-12-27 17:46:00,559][20381] Updated weights for policy 1, policy_version 1748136 (0.0008) [2023-12-27 17:46:00,612][20347] Updated weights for policy 0, policy_version 1749576 (0.0006) [2023-12-27 17:46:00,621][20381] Updated weights for policy 1, policy_version 1748146 (0.0006) [2023-12-27 17:46:00,661][20347] Updated weights for policy 0, policy_version 1749586 (0.0006) [2023-12-27 17:46:00,679][20381] Updated weights for policy 1, policy_version 1748156 (0.0005) [2023-12-27 17:46:00,719][20347] Updated weights for policy 0, policy_version 1749596 (0.0010) [2023-12-27 17:46:01,268][20381] Updated weights for policy 1, policy_version 1748166 (0.0006) [2023-12-27 17:46:01,323][20381] Updated weights for policy 1, policy_version 1748176 (0.0007) [2023-12-27 17:46:01,387][20381] Updated weights for policy 1, policy_version 1748186 (0.0008) [2023-12-27 17:46:01,438][20347] Updated weights for policy 0, policy_version 1749606 (0.0009) [2023-12-27 17:46:01,494][20347] Updated weights for policy 0, policy_version 1749616 (0.0007) [2023-12-27 17:46:01,553][20347] Updated weights for policy 0, policy_version 1749626 (0.0008) [2023-12-27 17:46:02,089][20381] Updated weights for policy 1, policy_version 1748196 (0.0008) [2023-12-27 17:46:02,142][20381] Updated weights for policy 1, policy_version 1748206 (0.0008) [2023-12-27 17:46:02,204][20381] Updated weights for policy 1, policy_version 1748216 (0.0007) [2023-12-27 17:46:02,222][20347] Updated weights for policy 0, policy_version 1749636 (0.0008) [2023-12-27 17:46:02,284][20347] Updated weights for policy 0, policy_version 1749646 (0.0007) [2023-12-27 17:46:02,343][20347] Updated weights for policy 0, policy_version 1749656 (0.0008) [2023-12-27 17:46:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 895582208. Throughput: 0: 10283.0, 1: 10281.9. Samples: 895593388. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:02,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:46:02,847][20381] Updated weights for policy 1, policy_version 1748226 (0.0007) [2023-12-27 17:46:02,902][20381] Updated weights for policy 1, policy_version 1748236 (0.0005) [2023-12-27 17:46:02,949][20381] Updated weights for policy 1, policy_version 1748246 (0.0005) [2023-12-27 17:46:03,003][20381] Updated weights for policy 1, policy_version 1748256 (0.0005) [2023-12-27 17:46:03,054][20347] Updated weights for policy 0, policy_version 1749666 (0.0008) [2023-12-27 17:46:03,100][20347] Updated weights for policy 0, policy_version 1749676 (0.0007) [2023-12-27 17:46:03,148][20347] Updated weights for policy 0, policy_version 1749686 (0.0007) [2023-12-27 17:46:03,203][20347] Updated weights for policy 0, policy_version 1749696 (0.0008) [2023-12-27 17:46:03,584][20381] Updated weights for policy 1, policy_version 1748266 (0.0008) [2023-12-27 17:46:03,641][20381] Updated weights for policy 1, policy_version 1748276 (0.0008) [2023-12-27 17:46:03,700][20381] Updated weights for policy 1, policy_version 1748286 (0.0006) [2023-12-27 17:46:03,808][20347] Updated weights for policy 0, policy_version 1749706 (0.0008) [2023-12-27 17:46:03,876][20347] Updated weights for policy 0, policy_version 1749716 (0.0008) [2023-12-27 17:46:03,934][20347] Updated weights for policy 0, policy_version 1749726 (0.0007) [2023-12-27 17:46:04,388][20381] Updated weights for policy 1, policy_version 1748296 (0.0008) [2023-12-27 17:46:04,457][20381] Updated weights for policy 1, policy_version 1748306 (0.0007) [2023-12-27 17:46:04,525][20381] Updated weights for policy 1, policy_version 1748316 (0.0008) [2023-12-27 17:46:04,618][20347] Updated weights for policy 0, policy_version 1749736 (0.0006) [2023-12-27 17:46:04,687][20347] Updated weights for policy 0, policy_version 1749746 (0.0006) [2023-12-27 17:46:04,746][20347] Updated weights for policy 0, policy_version 1749756 (0.0006) [2023-12-27 17:46:05,205][20381] Updated weights for policy 1, policy_version 1748326 (0.0006) [2023-12-27 17:46:05,254][20381] Updated weights for policy 1, policy_version 1748336 (0.0005) [2023-12-27 17:46:05,299][20381] Updated weights for policy 1, policy_version 1748346 (0.0005) [2023-12-27 17:46:05,402][20347] Updated weights for policy 0, policy_version 1749766 (0.0006) [2023-12-27 17:46:05,448][20347] Updated weights for policy 0, policy_version 1749776 (0.0005) [2023-12-27 17:46:05,493][20347] Updated weights for policy 0, policy_version 1749786 (0.0005) [2023-12-27 17:46:05,929][20381] Updated weights for policy 1, policy_version 1748356 (0.0007) [2023-12-27 17:46:05,990][20381] Updated weights for policy 1, policy_version 1748366 (0.0011) [2023-12-27 17:46:06,046][20347] Updated weights for policy 0, policy_version 1749796 (0.0005) [2023-12-27 17:46:06,055][20381] Updated weights for policy 1, policy_version 1748376 (0.0011) [2023-12-27 17:46:06,109][20347] Updated weights for policy 0, policy_version 1749806 (0.0006) [2023-12-27 17:46:06,172][20347] Updated weights for policy 0, policy_version 1749816 (0.0007) [2023-12-27 17:46:06,740][20381] Updated weights for policy 1, policy_version 1748386 (0.0009) [2023-12-27 17:46:06,805][20381] Updated weights for policy 1, policy_version 1748396 (0.0008) [2023-12-27 17:46:06,864][20381] Updated weights for policy 1, policy_version 1748406 (0.0007) [2023-12-27 17:46:06,898][20347] Updated weights for policy 0, policy_version 1749826 (0.0007) [2023-12-27 17:46:06,932][20381] Updated weights for policy 1, policy_version 1748416 (0.0006) [2023-12-27 17:46:06,955][20347] Updated weights for policy 0, policy_version 1749836 (0.0008) [2023-12-27 17:46:07,008][20347] Updated weights for policy 0, policy_version 1749846 (0.0008) [2023-12-27 17:46:07,053][20347] Updated weights for policy 0, policy_version 1749856 (0.0008) [2023-12-27 17:46:07,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 895688704. Throughput: 0: 10292.5, 1: 10338.6. Samples: 895656972. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:07,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 17:46:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001748416_447643648.pth... [2023-12-27 17:46:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001749856_448045056.pth... [2023-12-27 17:46:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001747232_447340544.pth [2023-12-27 17:46:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001748672_447741952.pth [2023-12-27 17:46:07,524][20381] Updated weights for policy 1, policy_version 1748426 (0.0007) [2023-12-27 17:46:07,583][20381] Updated weights for policy 1, policy_version 1748436 (0.0008) [2023-12-27 17:46:07,630][20381] Updated weights for policy 1, policy_version 1748446 (0.0010) [2023-12-27 17:46:07,763][20347] Updated weights for policy 0, policy_version 1749866 (0.0007) [2023-12-27 17:46:07,820][20347] Updated weights for policy 0, policy_version 1749876 (0.0008) [2023-12-27 17:46:07,879][20347] Updated weights for policy 0, policy_version 1749886 (0.0008) [2023-12-27 17:46:08,299][20381] Updated weights for policy 1, policy_version 1748456 (0.0008) [2023-12-27 17:46:08,367][20381] Updated weights for policy 1, policy_version 1748466 (0.0010) [2023-12-27 17:46:08,420][20381] Updated weights for policy 1, policy_version 1748476 (0.0010) [2023-12-27 17:46:08,541][20347] Updated weights for policy 0, policy_version 1749896 (0.0006) [2023-12-27 17:46:08,597][20347] Updated weights for policy 0, policy_version 1749906 (0.0007) [2023-12-27 17:46:08,656][20347] Updated weights for policy 0, policy_version 1749916 (0.0008) [2023-12-27 17:46:09,054][20381] Updated weights for policy 1, policy_version 1748486 (0.0008) [2023-12-27 17:46:09,108][20381] Updated weights for policy 1, policy_version 1748496 (0.0007) [2023-12-27 17:46:09,169][20381] Updated weights for policy 1, policy_version 1748506 (0.0007) [2023-12-27 17:46:09,399][20347] Updated weights for policy 0, policy_version 1749926 (0.0007) [2023-12-27 17:46:09,457][20347] Updated weights for policy 0, policy_version 1749936 (0.0007) [2023-12-27 17:46:09,520][20347] Updated weights for policy 0, policy_version 1749946 (0.0006) [2023-12-27 17:46:09,951][20381] Updated weights for policy 1, policy_version 1748516 (0.0009) [2023-12-27 17:46:10,012][20381] Updated weights for policy 1, policy_version 1748526 (0.0009) [2023-12-27 17:46:10,068][20381] Updated weights for policy 1, policy_version 1748536 (0.0008) [2023-12-27 17:46:10,195][20347] Updated weights for policy 0, policy_version 1749956 (0.0007) [2023-12-27 17:46:10,255][20347] Updated weights for policy 0, policy_version 1749966 (0.0007) [2023-12-27 17:46:10,315][20347] Updated weights for policy 0, policy_version 1749976 (0.0007) [2023-12-27 17:46:10,733][20381] Updated weights for policy 1, policy_version 1748546 (0.0010) [2023-12-27 17:46:10,800][20381] Updated weights for policy 1, policy_version 1748556 (0.0005) [2023-12-27 17:46:10,862][20381] Updated weights for policy 1, policy_version 1748566 (0.0006) [2023-12-27 17:46:10,922][20381] Updated weights for policy 1, policy_version 1748576 (0.0009) [2023-12-27 17:46:10,989][20347] Updated weights for policy 0, policy_version 1749986 (0.0007) [2023-12-27 17:46:11,047][20347] Updated weights for policy 0, policy_version 1749996 (0.0008) [2023-12-27 17:46:11,105][20347] Updated weights for policy 0, policy_version 1750006 (0.0008) [2023-12-27 17:46:11,173][20347] Updated weights for policy 0, policy_version 1750016 (0.0008) [2023-12-27 17:46:11,638][20381] Updated weights for policy 1, policy_version 1748586 (0.0010) [2023-12-27 17:46:11,699][20381] Updated weights for policy 1, policy_version 1748596 (0.0011) [2023-12-27 17:46:11,765][20381] Updated weights for policy 1, policy_version 1748606 (0.0011) [2023-12-27 17:46:11,948][20347] Updated weights for policy 0, policy_version 1750026 (0.0008) [2023-12-27 17:46:12,007][20347] Updated weights for policy 0, policy_version 1750036 (0.0007) [2023-12-27 17:46:12,071][20347] Updated weights for policy 0, policy_version 1750046 (0.0008) [2023-12-27 17:46:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 895787008. Throughput: 0: 10261.8, 1: 10326.1. Samples: 895780016. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:12,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 17:46:12,453][20381] Updated weights for policy 1, policy_version 1748616 (0.0009) [2023-12-27 17:46:12,515][20381] Updated weights for policy 1, policy_version 1748626 (0.0008) [2023-12-27 17:46:12,576][20381] Updated weights for policy 1, policy_version 1748636 (0.0009) [2023-12-27 17:46:12,814][20347] Updated weights for policy 0, policy_version 1750056 (0.0008) [2023-12-27 17:46:12,871][20347] Updated weights for policy 0, policy_version 1750066 (0.0007) [2023-12-27 17:46:12,932][20347] Updated weights for policy 0, policy_version 1750076 (0.0009) [2023-12-27 17:46:13,212][20381] Updated weights for policy 1, policy_version 1748646 (0.0007) [2023-12-27 17:46:13,266][20381] Updated weights for policy 1, policy_version 1748656 (0.0007) [2023-12-27 17:46:13,322][20381] Updated weights for policy 1, policy_version 1748666 (0.0007) [2023-12-27 17:46:13,591][20347] Updated weights for policy 0, policy_version 1750086 (0.0007) [2023-12-27 17:46:13,640][20347] Updated weights for policy 0, policy_version 1750096 (0.0005) [2023-12-27 17:46:13,694][20347] Updated weights for policy 0, policy_version 1750106 (0.0009) [2023-12-27 17:46:13,956][20381] Updated weights for policy 1, policy_version 1748676 (0.0009) [2023-12-27 17:46:14,018][20381] Updated weights for policy 1, policy_version 1748686 (0.0010) [2023-12-27 17:46:14,072][20381] Updated weights for policy 1, policy_version 1748696 (0.0010) [2023-12-27 17:46:14,244][20347] Updated weights for policy 0, policy_version 1750116 (0.0007) [2023-12-27 17:46:14,288][20347] Updated weights for policy 0, policy_version 1750126 (0.0010) [2023-12-27 17:46:14,342][20347] Updated weights for policy 0, policy_version 1750136 (0.0010) [2023-12-27 17:46:14,703][20381] Updated weights for policy 1, policy_version 1748706 (0.0009) [2023-12-27 17:46:14,759][20381] Updated weights for policy 1, policy_version 1748716 (0.0006) [2023-12-27 17:46:14,827][20381] Updated weights for policy 1, policy_version 1748726 (0.0008) [2023-12-27 17:46:14,883][20381] Updated weights for policy 1, policy_version 1748736 (0.0008) [2023-12-27 17:46:15,024][20347] Updated weights for policy 0, policy_version 1750146 (0.0009) [2023-12-27 17:46:15,084][20347] Updated weights for policy 0, policy_version 1750156 (0.0010) [2023-12-27 17:46:15,142][20347] Updated weights for policy 0, policy_version 1750166 (0.0010) [2023-12-27 17:46:15,196][20347] Updated weights for policy 0, policy_version 1750176 (0.0010) [2023-12-27 17:46:15,544][20381] Updated weights for policy 1, policy_version 1748746 (0.0007) [2023-12-27 17:46:15,599][20381] Updated weights for policy 1, policy_version 1748756 (0.0007) [2023-12-27 17:46:15,656][20381] Updated weights for policy 1, policy_version 1748766 (0.0007) [2023-12-27 17:46:15,836][20347] Updated weights for policy 0, policy_version 1750186 (0.0005) [2023-12-27 17:46:15,890][20347] Updated weights for policy 0, policy_version 1750196 (0.0005) [2023-12-27 17:46:15,940][20347] Updated weights for policy 0, policy_version 1750206 (0.0006) [2023-12-27 17:46:16,272][20381] Updated weights for policy 1, policy_version 1748776 (0.0005) [2023-12-27 17:46:16,322][20381] Updated weights for policy 1, policy_version 1748786 (0.0005) [2023-12-27 17:46:16,379][20381] Updated weights for policy 1, policy_version 1748796 (0.0005) [2023-12-27 17:46:16,529][20347] Updated weights for policy 0, policy_version 1750216 (0.0007) [2023-12-27 17:46:16,584][20347] Updated weights for policy 0, policy_version 1750226 (0.0007) [2023-12-27 17:46:16,641][20347] Updated weights for policy 0, policy_version 1750236 (0.0007) [2023-12-27 17:46:17,001][20381] Updated weights for policy 1, policy_version 1748806 (0.0008) [2023-12-27 17:46:17,050][20381] Updated weights for policy 1, policy_version 1748816 (0.0006) [2023-12-27 17:46:17,095][20381] Updated weights for policy 1, policy_version 1748826 (0.0005) [2023-12-27 17:46:17,347][20347] Updated weights for policy 0, policy_version 1750246 (0.0007) [2023-12-27 17:46:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 895893504. Throughput: 0: 10205.6, 1: 10360.1. Samples: 895905276. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:17,363][19170] Avg episode reward: [(0, '941.520'), (1, '1000.000')] [2023-12-27 17:46:17,408][20347] Updated weights for policy 0, policy_version 1750256 (0.0007) [2023-12-27 17:46:17,468][20347] Updated weights for policy 0, policy_version 1750266 (0.0007) [2023-12-27 17:46:17,722][20381] Updated weights for policy 1, policy_version 1748836 (0.0007) [2023-12-27 17:46:17,778][20381] Updated weights for policy 1, policy_version 1748846 (0.0007) [2023-12-27 17:46:17,840][20381] Updated weights for policy 1, policy_version 1748856 (0.0008) [2023-12-27 17:46:18,057][20347] Updated weights for policy 0, policy_version 1750276 (0.0008) [2023-12-27 17:46:18,117][20347] Updated weights for policy 0, policy_version 1750286 (0.0009) [2023-12-27 17:46:18,168][20347] Updated weights for policy 0, policy_version 1750296 (0.0010) [2023-12-27 17:46:18,479][20381] Updated weights for policy 1, policy_version 1748866 (0.0007) [2023-12-27 17:46:18,530][20381] Updated weights for policy 1, policy_version 1748876 (0.0008) [2023-12-27 17:46:18,583][20381] Updated weights for policy 1, policy_version 1748886 (0.0006) [2023-12-27 17:46:18,635][20381] Updated weights for policy 1, policy_version 1748896 (0.0008) [2023-12-27 17:46:18,866][20347] Updated weights for policy 0, policy_version 1750306 (0.0010) [2023-12-27 17:46:18,924][20347] Updated weights for policy 0, policy_version 1750316 (0.0006) [2023-12-27 17:46:18,985][20347] Updated weights for policy 0, policy_version 1750326 (0.0007) [2023-12-27 17:46:19,046][20347] Updated weights for policy 0, policy_version 1750336 (0.0008) [2023-12-27 17:46:19,372][20381] Updated weights for policy 1, policy_version 1748906 (0.0007) [2023-12-27 17:46:19,426][20381] Updated weights for policy 1, policy_version 1748916 (0.0007) [2023-12-27 17:46:19,477][20381] Updated weights for policy 1, policy_version 1748926 (0.0006) [2023-12-27 17:46:19,722][20347] Updated weights for policy 0, policy_version 1750346 (0.0010) [2023-12-27 17:46:19,781][20347] Updated weights for policy 0, policy_version 1750356 (0.0010) [2023-12-27 17:46:19,847][20347] Updated weights for policy 0, policy_version 1750366 (0.0008) [2023-12-27 17:46:20,237][20381] Updated weights for policy 1, policy_version 1748936 (0.0007) [2023-12-27 17:46:20,297][20381] Updated weights for policy 1, policy_version 1748946 (0.0008) [2023-12-27 17:46:20,359][20381] Updated weights for policy 1, policy_version 1748956 (0.0008) [2023-12-27 17:46:20,511][20347] Updated weights for policy 0, policy_version 1750376 (0.0006) [2023-12-27 17:46:20,573][20347] Updated weights for policy 0, policy_version 1750386 (0.0006) [2023-12-27 17:46:20,632][20347] Updated weights for policy 0, policy_version 1750396 (0.0009) [2023-12-27 17:46:21,016][20381] Updated weights for policy 1, policy_version 1748966 (0.0009) [2023-12-27 17:46:21,087][20381] Updated weights for policy 1, policy_version 1748976 (0.0010) [2023-12-27 17:46:21,148][20381] Updated weights for policy 1, policy_version 1748986 (0.0011) [2023-12-27 17:46:21,281][20347] Updated weights for policy 0, policy_version 1750406 (0.0011) [2023-12-27 17:46:21,351][20347] Updated weights for policy 0, policy_version 1750416 (0.0010) [2023-12-27 17:46:21,421][20347] Updated weights for policy 0, policy_version 1750426 (0.0010) [2023-12-27 17:46:21,852][20381] Updated weights for policy 1, policy_version 1748996 (0.0009) [2023-12-27 17:46:21,918][20381] Updated weights for policy 1, policy_version 1749006 (0.0006) [2023-12-27 17:46:21,981][20381] Updated weights for policy 1, policy_version 1749016 (0.0006) [2023-12-27 17:46:22,149][20347] Updated weights for policy 0, policy_version 1750436 (0.0009) [2023-12-27 17:46:22,205][20347] Updated weights for policy 0, policy_version 1750446 (0.0008) [2023-12-27 17:46:22,266][20347] Updated weights for policy 0, policy_version 1750456 (0.0007) [2023-12-27 17:46:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 896000000. Throughput: 0: 10233.1, 1: 10277.4. Samples: 895966964. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:22,363][19170] Avg episode reward: [(0, '951.250'), (1, '1000.000')] [2023-12-27 17:46:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001749024_447799296.pth... [2023-12-27 17:46:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001750464_448200704.pth... [2023-12-27 17:46:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001749248_447889408.pth [2023-12-27 17:46:22,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001747808_447488000.pth [2023-12-27 17:46:22,637][20381] Updated weights for policy 1, policy_version 1749026 (0.0007) [2023-12-27 17:46:22,703][20381] Updated weights for policy 1, policy_version 1749036 (0.0005) [2023-12-27 17:46:22,761][20381] Updated weights for policy 1, policy_version 1749046 (0.0006) [2023-12-27 17:46:22,821][20381] Updated weights for policy 1, policy_version 1749056 (0.0006) [2023-12-27 17:46:23,062][20347] Updated weights for policy 0, policy_version 1750466 (0.0007) [2023-12-27 17:46:23,115][20347] Updated weights for policy 0, policy_version 1750476 (0.0005) [2023-12-27 17:46:23,172][20347] Updated weights for policy 0, policy_version 1750486 (0.0005) [2023-12-27 17:46:23,230][20347] Updated weights for policy 0, policy_version 1750496 (0.0006) [2023-12-27 17:46:23,407][20381] Updated weights for policy 1, policy_version 1749066 (0.0011) [2023-12-27 17:46:23,458][20381] Updated weights for policy 1, policy_version 1749076 (0.0010) [2023-12-27 17:46:23,512][20381] Updated weights for policy 1, policy_version 1749086 (0.0010) [2023-12-27 17:46:23,891][20347] Updated weights for policy 0, policy_version 1750506 (0.0007) [2023-12-27 17:46:23,951][20347] Updated weights for policy 0, policy_version 1750516 (0.0008) [2023-12-27 17:46:24,000][20347] Updated weights for policy 0, policy_version 1750526 (0.0006) [2023-12-27 17:46:24,104][20381] Updated weights for policy 1, policy_version 1749096 (0.0011) [2023-12-27 17:46:24,164][20381] Updated weights for policy 1, policy_version 1749106 (0.0006) [2023-12-27 17:46:24,225][20381] Updated weights for policy 1, policy_version 1749116 (0.0007) [2023-12-27 17:46:24,724][20347] Updated weights for policy 0, policy_version 1750536 (0.0009) [2023-12-27 17:46:24,776][20347] Updated weights for policy 0, policy_version 1750546 (0.0006) [2023-12-27 17:46:24,814][20381] Updated weights for policy 1, policy_version 1749126 (0.0006) [2023-12-27 17:46:24,825][20347] Updated weights for policy 0, policy_version 1750556 (0.0007) [2023-12-27 17:46:24,869][20381] Updated weights for policy 1, policy_version 1749136 (0.0005) [2023-12-27 17:46:24,934][20381] Updated weights for policy 1, policy_version 1749146 (0.0005) [2023-12-27 17:46:25,469][20347] Updated weights for policy 0, policy_version 1750566 (0.0007) [2023-12-27 17:46:25,525][20347] Updated weights for policy 0, policy_version 1750576 (0.0008) [2023-12-27 17:46:25,548][20381] Updated weights for policy 1, policy_version 1749156 (0.0006) [2023-12-27 17:46:25,579][20347] Updated weights for policy 0, policy_version 1750586 (0.0009) [2023-12-27 17:46:25,607][20381] Updated weights for policy 1, policy_version 1749166 (0.0007) [2023-12-27 17:46:25,662][20381] Updated weights for policy 1, policy_version 1749176 (0.0007) [2023-12-27 17:46:26,265][20347] Updated weights for policy 0, policy_version 1750596 (0.0006) [2023-12-27 17:46:26,310][20347] Updated weights for policy 0, policy_version 1750606 (0.0007) [2023-12-27 17:46:26,357][20347] Updated weights for policy 0, policy_version 1750616 (0.0008) [2023-12-27 17:46:26,362][20381] Updated weights for policy 1, policy_version 1749186 (0.0008) [2023-12-27 17:46:26,417][20381] Updated weights for policy 1, policy_version 1749196 (0.0006) [2023-12-27 17:46:26,471][20381] Updated weights for policy 1, policy_version 1749206 (0.0007) [2023-12-27 17:46:26,527][20381] Updated weights for policy 1, policy_version 1749216 (0.0006) [2023-12-27 17:46:27,040][20347] Updated weights for policy 0, policy_version 1750626 (0.0006) [2023-12-27 17:46:27,094][20347] Updated weights for policy 0, policy_version 1750636 (0.0005) [2023-12-27 17:46:27,147][20347] Updated weights for policy 0, policy_version 1750646 (0.0005) [2023-12-27 17:46:27,159][20381] Updated weights for policy 1, policy_version 1749226 (0.0008) [2023-12-27 17:46:27,203][20347] Updated weights for policy 0, policy_version 1750656 (0.0008) [2023-12-27 17:46:27,214][20381] Updated weights for policy 1, policy_version 1749236 (0.0010) [2023-12-27 17:46:27,275][20381] Updated weights for policy 1, policy_version 1749246 (0.0010) [2023-12-27 17:46:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 896106496. Throughput: 0: 10179.0, 1: 10338.6. Samples: 896090344. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:27,363][19170] Avg episode reward: [(0, '970.650'), (1, '1000.000')] [2023-12-27 17:46:27,819][20347] Updated weights for policy 0, policy_version 1750666 (0.0005) [2023-12-27 17:46:27,868][20347] Updated weights for policy 0, policy_version 1750676 (0.0008) [2023-12-27 17:46:27,906][20381] Updated weights for policy 1, policy_version 1749256 (0.0011) [2023-12-27 17:46:27,916][20347] Updated weights for policy 0, policy_version 1750686 (0.0010) [2023-12-27 17:46:27,961][20381] Updated weights for policy 1, policy_version 1749266 (0.0010) [2023-12-27 17:46:28,015][20381] Updated weights for policy 1, policy_version 1749276 (0.0010) [2023-12-27 17:46:28,519][20347] Updated weights for policy 0, policy_version 1750696 (0.0009) [2023-12-27 17:46:28,581][20347] Updated weights for policy 0, policy_version 1750706 (0.0008) [2023-12-27 17:46:28,634][20347] Updated weights for policy 0, policy_version 1750716 (0.0009) [2023-12-27 17:46:28,671][20381] Updated weights for policy 1, policy_version 1749286 (0.0007) [2023-12-27 17:46:28,728][20381] Updated weights for policy 1, policy_version 1749296 (0.0006) [2023-12-27 17:46:28,816][20381] Updated weights for policy 1, policy_version 1749306 (0.0006) [2023-12-27 17:46:29,292][20347] Updated weights for policy 0, policy_version 1750726 (0.0007) [2023-12-27 17:46:29,353][20347] Updated weights for policy 0, policy_version 1750736 (0.0007) [2023-12-27 17:46:29,414][20347] Updated weights for policy 0, policy_version 1750746 (0.0007) [2023-12-27 17:46:29,415][20381] Updated weights for policy 1, policy_version 1749316 (0.0006) [2023-12-27 17:46:29,465][20381] Updated weights for policy 1, policy_version 1749326 (0.0007) [2023-12-27 17:46:29,521][20381] Updated weights for policy 1, policy_version 1749336 (0.0009) [2023-12-27 17:46:30,014][20347] Updated weights for policy 0, policy_version 1750756 (0.0007) [2023-12-27 17:46:30,070][20347] Updated weights for policy 0, policy_version 1750766 (0.0008) [2023-12-27 17:46:30,126][20347] Updated weights for policy 0, policy_version 1750776 (0.0006) [2023-12-27 17:46:30,249][20381] Updated weights for policy 1, policy_version 1749346 (0.0008) [2023-12-27 17:46:30,309][20381] Updated weights for policy 1, policy_version 1749356 (0.0008) [2023-12-27 17:46:30,368][20381] Updated weights for policy 1, policy_version 1749366 (0.0009) [2023-12-27 17:46:30,420][20381] Updated weights for policy 1, policy_version 1749376 (0.0008) [2023-12-27 17:46:30,725][20347] Updated weights for policy 0, policy_version 1750786 (0.0007) [2023-12-27 17:46:30,782][20347] Updated weights for policy 0, policy_version 1750796 (0.0008) [2023-12-27 17:46:30,838][20347] Updated weights for policy 0, policy_version 1750806 (0.0008) [2023-12-27 17:46:30,891][20347] Updated weights for policy 0, policy_version 1750816 (0.0007) [2023-12-27 17:46:31,115][20381] Updated weights for policy 1, policy_version 1749386 (0.0008) [2023-12-27 17:46:31,181][20381] Updated weights for policy 1, policy_version 1749396 (0.0008) [2023-12-27 17:46:31,236][20381] Updated weights for policy 1, policy_version 1749406 (0.0009) [2023-12-27 17:46:31,608][20347] Updated weights for policy 0, policy_version 1750826 (0.0008) [2023-12-27 17:46:31,674][20347] Updated weights for policy 0, policy_version 1750836 (0.0008) [2023-12-27 17:46:31,735][20347] Updated weights for policy 0, policy_version 1750846 (0.0008) [2023-12-27 17:46:31,960][20381] Updated weights for policy 1, policy_version 1749416 (0.0009) [2023-12-27 17:46:32,004][20381] Updated weights for policy 1, policy_version 1749426 (0.0007) [2023-12-27 17:46:32,051][20381] Updated weights for policy 1, policy_version 1749436 (0.0008) [2023-12-27 17:46:32,355][20347] Updated weights for policy 0, policy_version 1750856 (0.0007) [2023-12-27 17:46:32,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20616.4, 300 sec: 20577.2). Total num frames: 896204800. Throughput: 0: 10288.1, 1: 10397.9. Samples: 896216084. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:32,363][19170] Avg episode reward: [(0, '970.770'), (1, '1000.000')] [2023-12-27 17:46:32,419][20347] Updated weights for policy 0, policy_version 1750866 (0.0007) [2023-12-27 17:46:32,478][20347] Updated weights for policy 0, policy_version 1750876 (0.0007) [2023-12-27 17:46:32,787][20381] Updated weights for policy 1, policy_version 1749446 (0.0008) [2023-12-27 17:46:32,833][20381] Updated weights for policy 1, policy_version 1749456 (0.0008) [2023-12-27 17:46:32,883][20381] Updated weights for policy 1, policy_version 1749466 (0.0008) [2023-12-27 17:46:33,156][20347] Updated weights for policy 0, policy_version 1750886 (0.0007) [2023-12-27 17:46:33,210][20347] Updated weights for policy 0, policy_version 1750896 (0.0007) [2023-12-27 17:46:33,262][20347] Updated weights for policy 0, policy_version 1750906 (0.0007) [2023-12-27 17:46:33,630][20381] Updated weights for policy 1, policy_version 1749476 (0.0007) [2023-12-27 17:46:33,696][20381] Updated weights for policy 1, policy_version 1749486 (0.0005) [2023-12-27 17:46:33,755][20381] Updated weights for policy 1, policy_version 1749496 (0.0006) [2023-12-27 17:46:33,888][20347] Updated weights for policy 0, policy_version 1750916 (0.0007) [2023-12-27 17:46:33,944][20347] Updated weights for policy 0, policy_version 1750926 (0.0005) [2023-12-27 17:46:34,000][20347] Updated weights for policy 0, policy_version 1750936 (0.0005) [2023-12-27 17:46:34,430][20381] Updated weights for policy 1, policy_version 1749506 (0.0007) [2023-12-27 17:46:34,493][20381] Updated weights for policy 1, policy_version 1749516 (0.0008) [2023-12-27 17:46:34,554][20381] Updated weights for policy 1, policy_version 1749526 (0.0007) [2023-12-27 17:46:34,590][20347] Updated weights for policy 0, policy_version 1750946 (0.0006) [2023-12-27 17:46:34,613][20381] Updated weights for policy 1, policy_version 1749536 (0.0007) [2023-12-27 17:46:34,657][20347] Updated weights for policy 0, policy_version 1750956 (0.0007) [2023-12-27 17:46:34,718][20347] Updated weights for policy 0, policy_version 1750966 (0.0010) [2023-12-27 17:46:34,784][20347] Updated weights for policy 0, policy_version 1750976 (0.0011) [2023-12-27 17:46:35,324][20381] Updated weights for policy 1, policy_version 1749546 (0.0011) [2023-12-27 17:46:35,379][20381] Updated weights for policy 1, policy_version 1749556 (0.0010) [2023-12-27 17:46:35,425][20381] Updated weights for policy 1, policy_version 1749566 (0.0010) [2023-12-27 17:46:35,448][20347] Updated weights for policy 0, policy_version 1750986 (0.0010) [2023-12-27 17:46:35,499][20347] Updated weights for policy 0, policy_version 1750996 (0.0010) [2023-12-27 17:46:35,562][20347] Updated weights for policy 0, policy_version 1751006 (0.0010) [2023-12-27 17:46:35,996][20381] Updated weights for policy 1, policy_version 1749576 (0.0005) [2023-12-27 17:46:36,052][20381] Updated weights for policy 1, policy_version 1749586 (0.0010) [2023-12-27 17:46:36,119][20381] Updated weights for policy 1, policy_version 1749596 (0.0010) [2023-12-27 17:46:36,241][20347] Updated weights for policy 0, policy_version 1751016 (0.0008) [2023-12-27 17:46:36,302][20347] Updated weights for policy 0, policy_version 1751026 (0.0008) [2023-12-27 17:46:36,362][20347] Updated weights for policy 0, policy_version 1751036 (0.0008) [2023-12-27 17:46:36,813][20381] Updated weights for policy 1, policy_version 1749606 (0.0011) [2023-12-27 17:46:36,871][20381] Updated weights for policy 1, policy_version 1749616 (0.0011) [2023-12-27 17:46:36,921][20381] Updated weights for policy 1, policy_version 1749626 (0.0011) [2023-12-27 17:46:37,076][20347] Updated weights for policy 0, policy_version 1751046 (0.0007) [2023-12-27 17:46:37,133][20347] Updated weights for policy 0, policy_version 1751056 (0.0006) [2023-12-27 17:46:37,195][20347] Updated weights for policy 0, policy_version 1751066 (0.0010) [2023-12-27 17:46:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 896311296. Throughput: 0: 10363.6, 1: 10379.2. Samples: 896277804. Policy #0 lag: (min: 25.0, avg: 43.9, max: 57.0) [2023-12-27 17:46:37,363][19170] Avg episode reward: [(0, '961.070'), (1, '1000.000')] [2023-12-27 17:46:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001751072_448356352.pth... [2023-12-27 17:46:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001749632_447954944.pth... [2023-12-27 17:46:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001749856_448045056.pth [2023-12-27 17:46:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001748416_447643648.pth [2023-12-27 17:46:37,629][20381] Updated weights for policy 1, policy_version 1749636 (0.0011) [2023-12-27 17:46:37,674][20381] Updated weights for policy 1, policy_version 1749646 (0.0010) [2023-12-27 17:46:37,725][20381] Updated weights for policy 1, policy_version 1749656 (0.0011) [2023-12-27 17:46:37,805][20347] Updated weights for policy 0, policy_version 1751076 (0.0009) [2023-12-27 17:46:37,860][20347] Updated weights for policy 0, policy_version 1751086 (0.0010) [2023-12-27 17:46:37,919][20347] Updated weights for policy 0, policy_version 1751096 (0.0009) [2023-12-27 17:46:38,456][20381] Updated weights for policy 1, policy_version 1749666 (0.0010) [2023-12-27 17:46:38,523][20381] Updated weights for policy 1, policy_version 1749676 (0.0009) [2023-12-27 17:46:38,544][20347] Updated weights for policy 0, policy_version 1751106 (0.0006) [2023-12-27 17:46:38,583][20381] Updated weights for policy 1, policy_version 1749686 (0.0009) [2023-12-27 17:46:38,600][20347] Updated weights for policy 0, policy_version 1751116 (0.0011) [2023-12-27 17:46:38,644][20381] Updated weights for policy 1, policy_version 1749696 (0.0009) [2023-12-27 17:46:38,652][20347] Updated weights for policy 0, policy_version 1751126 (0.0010) [2023-12-27 17:46:38,709][20347] Updated weights for policy 0, policy_version 1751136 (0.0011) [2023-12-27 17:46:39,366][20347] Updated weights for policy 0, policy_version 1751146 (0.0011) [2023-12-27 17:46:39,386][20381] Updated weights for policy 1, policy_version 1749706 (0.0008) [2023-12-27 17:46:39,435][20347] Updated weights for policy 0, policy_version 1751156 (0.0009) [2023-12-27 17:46:39,457][20381] Updated weights for policy 1, policy_version 1749716 (0.0008) [2023-12-27 17:46:39,498][20347] Updated weights for policy 0, policy_version 1751166 (0.0008) [2023-12-27 17:46:39,519][20381] Updated weights for policy 1, policy_version 1749726 (0.0006) [2023-12-27 17:46:40,173][20347] Updated weights for policy 0, policy_version 1751176 (0.0008) [2023-12-27 17:46:40,208][20381] Updated weights for policy 1, policy_version 1749736 (0.0008) [2023-12-27 17:46:40,240][20347] Updated weights for policy 0, policy_version 1751186 (0.0008) [2023-12-27 17:46:40,270][20381] Updated weights for policy 1, policy_version 1749746 (0.0007) [2023-12-27 17:46:40,308][20347] Updated weights for policy 0, policy_version 1751196 (0.0008) [2023-12-27 17:46:40,326][20381] Updated weights for policy 1, policy_version 1749756 (0.0008) [2023-12-27 17:46:40,974][20347] Updated weights for policy 0, policy_version 1751206 (0.0006) [2023-12-27 17:46:41,001][20381] Updated weights for policy 1, policy_version 1749766 (0.0009) [2023-12-27 17:46:41,032][20347] Updated weights for policy 0, policy_version 1751216 (0.0007) [2023-12-27 17:46:41,065][20381] Updated weights for policy 1, policy_version 1749776 (0.0008) [2023-12-27 17:46:41,095][20347] Updated weights for policy 0, policy_version 1751226 (0.0006) [2023-12-27 17:46:41,129][20381] Updated weights for policy 1, policy_version 1749786 (0.0009) [2023-12-27 17:46:41,869][20347] Updated weights for policy 0, policy_version 1751236 (0.0006) [2023-12-27 17:46:41,880][20381] Updated weights for policy 1, policy_version 1749796 (0.0008) [2023-12-27 17:46:41,930][20347] Updated weights for policy 0, policy_version 1751246 (0.0007) [2023-12-27 17:46:41,936][20381] Updated weights for policy 1, policy_version 1749806 (0.0006) [2023-12-27 17:46:41,984][20347] Updated weights for policy 0, policy_version 1751256 (0.0006) [2023-12-27 17:46:41,997][20381] Updated weights for policy 1, policy_version 1749816 (0.0007) [2023-12-27 17:46:42,362][19170] Fps is (10 sec: 20480.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 896409600. Throughput: 0: 10356.0, 1: 10314.9. Samples: 896399892. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:46:42,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 17:46:42,668][20347] Updated weights for policy 0, policy_version 1751266 (0.0007) [2023-12-27 17:46:42,712][20381] Updated weights for policy 1, policy_version 1749826 (0.0006) [2023-12-27 17:46:42,732][20347] Updated weights for policy 0, policy_version 1751276 (0.0008) [2023-12-27 17:46:42,779][20381] Updated weights for policy 1, policy_version 1749836 (0.0007) [2023-12-27 17:46:42,790][20347] Updated weights for policy 0, policy_version 1751286 (0.0006) [2023-12-27 17:46:42,836][20381] Updated weights for policy 1, policy_version 1749846 (0.0007) [2023-12-27 17:46:42,849][20347] Updated weights for policy 0, policy_version 1751296 (0.0006) [2023-12-27 17:46:42,901][20381] Updated weights for policy 1, policy_version 1749856 (0.0007) [2023-12-27 17:46:43,548][20347] Updated weights for policy 0, policy_version 1751306 (0.0006) [2023-12-27 17:46:43,559][20381] Updated weights for policy 1, policy_version 1749866 (0.0007) [2023-12-27 17:46:43,604][20347] Updated weights for policy 0, policy_version 1751316 (0.0006) [2023-12-27 17:46:43,606][20381] Updated weights for policy 1, policy_version 1749876 (0.0007) [2023-12-27 17:46:43,656][20381] Updated weights for policy 1, policy_version 1749886 (0.0006) [2023-12-27 17:46:43,666][20347] Updated weights for policy 0, policy_version 1751326 (0.0006) [2023-12-27 17:46:44,258][20381] Updated weights for policy 1, policy_version 1749896 (0.0007) [2023-12-27 17:46:44,288][20347] Updated weights for policy 0, policy_version 1751336 (0.0007) [2023-12-27 17:46:44,307][20381] Updated weights for policy 1, policy_version 1749906 (0.0009) [2023-12-27 17:46:44,341][20347] Updated weights for policy 0, policy_version 1751346 (0.0006) [2023-12-27 17:46:44,364][20381] Updated weights for policy 1, policy_version 1749916 (0.0007) [2023-12-27 17:46:44,399][20347] Updated weights for policy 0, policy_version 1751356 (0.0008) [2023-12-27 17:46:45,001][20347] Updated weights for policy 0, policy_version 1751366 (0.0008) [2023-12-27 17:46:45,062][20347] Updated weights for policy 0, policy_version 1751376 (0.0007) [2023-12-27 17:46:45,118][20381] Updated weights for policy 1, policy_version 1749926 (0.0006) [2023-12-27 17:46:45,120][20347] Updated weights for policy 0, policy_version 1751386 (0.0007) [2023-12-27 17:46:45,177][20381] Updated weights for policy 1, policy_version 1749936 (0.0007) [2023-12-27 17:46:45,242][20381] Updated weights for policy 1, policy_version 1749946 (0.0007) [2023-12-27 17:46:45,805][20347] Updated weights for policy 0, policy_version 1751396 (0.0008) [2023-12-27 17:46:45,860][20347] Updated weights for policy 0, policy_version 1751406 (0.0010) [2023-12-27 17:46:45,914][20347] Updated weights for policy 0, policy_version 1751416 (0.0009) [2023-12-27 17:46:45,945][20381] Updated weights for policy 1, policy_version 1749956 (0.0008) [2023-12-27 17:46:46,004][20381] Updated weights for policy 1, policy_version 1749966 (0.0008) [2023-12-27 17:46:46,054][20381] Updated weights for policy 1, policy_version 1749976 (0.0009) [2023-12-27 17:46:46,497][20347] Updated weights for policy 0, policy_version 1751426 (0.0008) [2023-12-27 17:46:46,561][20347] Updated weights for policy 0, policy_version 1751436 (0.0010) [2023-12-27 17:46:46,620][20347] Updated weights for policy 0, policy_version 1751446 (0.0010) [2023-12-27 17:46:46,682][20347] Updated weights for policy 0, policy_version 1751456 (0.0010) [2023-12-27 17:46:46,697][20381] Updated weights for policy 1, policy_version 1749986 (0.0006) [2023-12-27 17:46:46,749][20381] Updated weights for policy 1, policy_version 1749996 (0.0007) [2023-12-27 17:46:46,800][20381] Updated weights for policy 1, policy_version 1750006 (0.0007) [2023-12-27 17:46:46,854][20381] Updated weights for policy 1, policy_version 1750016 (0.0008) [2023-12-27 17:46:47,341][20347] Updated weights for policy 0, policy_version 1751466 (0.0009) [2023-12-27 17:46:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 896507904. Throughput: 0: 10357.2, 1: 10293.2. Samples: 896522652. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:46:47,363][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 17:46:47,400][20347] Updated weights for policy 0, policy_version 1751476 (0.0010) [2023-12-27 17:46:47,455][20347] Updated weights for policy 0, policy_version 1751486 (0.0010) [2023-12-27 17:46:47,565][20381] Updated weights for policy 1, policy_version 1750026 (0.0006) [2023-12-27 17:46:47,614][20381] Updated weights for policy 1, policy_version 1750036 (0.0006) [2023-12-27 17:46:47,665][20381] Updated weights for policy 1, policy_version 1750046 (0.0005) [2023-12-27 17:46:48,073][20347] Updated weights for policy 0, policy_version 1751496 (0.0007) [2023-12-27 17:46:48,130][20347] Updated weights for policy 0, policy_version 1751506 (0.0005) [2023-12-27 17:46:48,185][20347] Updated weights for policy 0, policy_version 1751516 (0.0009) [2023-12-27 17:46:48,325][20381] Updated weights for policy 1, policy_version 1750056 (0.0007) [2023-12-27 17:46:48,386][20381] Updated weights for policy 1, policy_version 1750066 (0.0008) [2023-12-27 17:46:48,449][20381] Updated weights for policy 1, policy_version 1750076 (0.0007) [2023-12-27 17:46:48,850][20347] Updated weights for policy 0, policy_version 1751526 (0.0009) [2023-12-27 17:46:48,911][20347] Updated weights for policy 0, policy_version 1751536 (0.0008) [2023-12-27 17:46:48,969][20347] Updated weights for policy 0, policy_version 1751546 (0.0008) [2023-12-27 17:46:49,097][20381] Updated weights for policy 1, policy_version 1750086 (0.0008) [2023-12-27 17:46:49,153][20381] Updated weights for policy 1, policy_version 1750096 (0.0008) [2023-12-27 17:46:49,212][20381] Updated weights for policy 1, policy_version 1750106 (0.0008) [2023-12-27 17:46:49,579][20347] Updated weights for policy 0, policy_version 1751556 (0.0007) [2023-12-27 17:46:49,641][20347] Updated weights for policy 0, policy_version 1751566 (0.0005) [2023-12-27 17:46:49,700][20347] Updated weights for policy 0, policy_version 1751576 (0.0005) [2023-12-27 17:46:50,032][20381] Updated weights for policy 1, policy_version 1750116 (0.0007) [2023-12-27 17:46:50,089][20381] Updated weights for policy 1, policy_version 1750126 (0.0005) [2023-12-27 17:46:50,153][20381] Updated weights for policy 1, policy_version 1750136 (0.0007) [2023-12-27 17:46:50,378][20347] Updated weights for policy 0, policy_version 1751586 (0.0006) [2023-12-27 17:46:50,435][20347] Updated weights for policy 0, policy_version 1751596 (0.0008) [2023-12-27 17:46:50,480][20347] Updated weights for policy 0, policy_version 1751606 (0.0007) [2023-12-27 17:46:50,526][20347] Updated weights for policy 0, policy_version 1751616 (0.0009) [2023-12-27 17:46:50,751][20381] Updated weights for policy 1, policy_version 1750146 (0.0010) [2023-12-27 17:46:50,808][20381] Updated weights for policy 1, policy_version 1750156 (0.0008) [2023-12-27 17:46:50,866][20381] Updated weights for policy 1, policy_version 1750166 (0.0007) [2023-12-27 17:46:50,923][20381] Updated weights for policy 1, policy_version 1750176 (0.0009) [2023-12-27 17:46:51,246][20347] Updated weights for policy 0, policy_version 1751626 (0.0007) [2023-12-27 17:46:51,305][20347] Updated weights for policy 0, policy_version 1751636 (0.0006) [2023-12-27 17:46:51,313][20291] KL-divergence is very high: 103.1651 [2023-12-27 17:46:51,368][20347] Updated weights for policy 0, policy_version 1751646 (0.0008) [2023-12-27 17:46:51,664][20381] Updated weights for policy 1, policy_version 1750186 (0.0008) [2023-12-27 17:46:51,731][20381] Updated weights for policy 1, policy_version 1750196 (0.0008) [2023-12-27 17:46:51,785][20381] Updated weights for policy 1, policy_version 1750206 (0.0009) [2023-12-27 17:46:52,140][20347] Updated weights for policy 0, policy_version 1751656 (0.0009) [2023-12-27 17:46:52,201][20347] Updated weights for policy 0, policy_version 1751666 (0.0009) [2023-12-27 17:46:52,255][20347] Updated weights for policy 0, policy_version 1751676 (0.0009) [2023-12-27 17:46:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 896614400. Throughput: 0: 10367.6, 1: 10245.1. Samples: 896584544. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:46:52,363][19170] Avg episode reward: [(0, '941.530'), (1, '1000.000')] [2023-12-27 17:46:52,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001750208_448102400.pth... [2023-12-27 17:46:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001751680_448512000.pth... [2023-12-27 17:46:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001750464_448200704.pth [2023-12-27 17:46:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001749024_447799296.pth [2023-12-27 17:46:52,636][20381] Updated weights for policy 1, policy_version 1750216 (0.0009) [2023-12-27 17:46:52,696][20381] Updated weights for policy 1, policy_version 1750226 (0.0010) [2023-12-27 17:46:52,760][20381] Updated weights for policy 1, policy_version 1750236 (0.0006) [2023-12-27 17:46:53,045][20347] Updated weights for policy 0, policy_version 1751686 (0.0007) [2023-12-27 17:46:53,105][20347] Updated weights for policy 0, policy_version 1751696 (0.0006) [2023-12-27 17:46:53,171][20347] Updated weights for policy 0, policy_version 1751706 (0.0008) [2023-12-27 17:46:53,404][20381] Updated weights for policy 1, policy_version 1750246 (0.0006) [2023-12-27 17:46:53,459][20381] Updated weights for policy 1, policy_version 1750256 (0.0011) [2023-12-27 17:46:53,512][20381] Updated weights for policy 1, policy_version 1750266 (0.0006) [2023-12-27 17:46:53,743][20347] Updated weights for policy 0, policy_version 1751716 (0.0007) [2023-12-27 17:46:53,792][20347] Updated weights for policy 0, policy_version 1751726 (0.0005) [2023-12-27 17:46:53,848][20347] Updated weights for policy 0, policy_version 1751736 (0.0007) [2023-12-27 17:46:54,181][20381] Updated weights for policy 1, policy_version 1750276 (0.0008) [2023-12-27 17:46:54,233][20381] Updated weights for policy 1, policy_version 1750286 (0.0010) [2023-12-27 17:46:54,292][20381] Updated weights for policy 1, policy_version 1750296 (0.0011) [2023-12-27 17:46:54,491][20347] Updated weights for policy 0, policy_version 1751746 (0.0010) [2023-12-27 17:46:54,544][20347] Updated weights for policy 0, policy_version 1751756 (0.0009) [2023-12-27 17:46:54,556][20291] KL-divergence is very high: 134.2674 [2023-12-27 17:46:54,567][20291] KL-divergence is very high: 119.8339 [2023-12-27 17:46:54,597][20347] Updated weights for policy 0, policy_version 1751766 (0.0006) [2023-12-27 17:46:54,651][20347] Updated weights for policy 0, policy_version 1751776 (0.0005) [2023-12-27 17:46:54,968][20381] Updated weights for policy 1, policy_version 1750306 (0.0010) [2023-12-27 17:46:55,030][20381] Updated weights for policy 1, policy_version 1750316 (0.0008) [2023-12-27 17:46:55,094][20381] Updated weights for policy 1, policy_version 1750326 (0.0009) [2023-12-27 17:46:55,157][20381] Updated weights for policy 1, policy_version 1750336 (0.0008) [2023-12-27 17:46:55,253][20347] Updated weights for policy 0, policy_version 1751786 (0.0006) [2023-12-27 17:46:55,308][20347] Updated weights for policy 0, policy_version 1751796 (0.0006) [2023-12-27 17:46:55,363][20347] Updated weights for policy 0, policy_version 1751806 (0.0005) [2023-12-27 17:46:55,875][20381] Updated weights for policy 1, policy_version 1750346 (0.0006) [2023-12-27 17:46:55,936][20381] Updated weights for policy 1, policy_version 1750356 (0.0005) [2023-12-27 17:46:55,947][20347] Updated weights for policy 0, policy_version 1751816 (0.0009) [2023-12-27 17:46:55,992][20381] Updated weights for policy 1, policy_version 1750366 (0.0005) [2023-12-27 17:46:56,006][20347] Updated weights for policy 0, policy_version 1751826 (0.0011) [2023-12-27 17:46:56,063][20347] Updated weights for policy 0, policy_version 1751836 (0.0010) [2023-12-27 17:46:56,578][20381] Updated weights for policy 1, policy_version 1750376 (0.0009) [2023-12-27 17:46:56,625][20381] Updated weights for policy 1, policy_version 1750386 (0.0008) [2023-12-27 17:46:56,654][20347] Updated weights for policy 0, policy_version 1751846 (0.0010) [2023-12-27 17:46:56,667][20381] Updated weights for policy 1, policy_version 1750396 (0.0005) [2023-12-27 17:46:56,702][20347] Updated weights for policy 0, policy_version 1751856 (0.0010) [2023-12-27 17:46:56,746][20347] Updated weights for policy 0, policy_version 1751866 (0.0010) [2023-12-27 17:46:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 896712704. Throughput: 0: 10386.8, 1: 10199.5. Samples: 896706400. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:46:57,363][19170] Avg episode reward: [(0, '922.110'), (1, '1000.000')] [2023-12-27 17:46:57,389][20347] Updated weights for policy 0, policy_version 1751876 (0.0010) [2023-12-27 17:46:57,397][20381] Updated weights for policy 1, policy_version 1750406 (0.0007) [2023-12-27 17:46:57,441][20347] Updated weights for policy 0, policy_version 1751886 (0.0010) [2023-12-27 17:46:57,447][20381] Updated weights for policy 1, policy_version 1750416 (0.0005) [2023-12-27 17:46:57,490][20347] Updated weights for policy 0, policy_version 1751896 (0.0010) [2023-12-27 17:46:57,502][20381] Updated weights for policy 1, policy_version 1750426 (0.0005) [2023-12-27 17:46:58,084][20347] Updated weights for policy 0, policy_version 1751906 (0.0010) [2023-12-27 17:46:58,135][20347] Updated weights for policy 0, policy_version 1751916 (0.0010) [2023-12-27 17:46:58,189][20347] Updated weights for policy 0, policy_version 1751926 (0.0008) [2023-12-27 17:46:58,212][20381] Updated weights for policy 1, policy_version 1750436 (0.0006) [2023-12-27 17:46:58,249][20347] Updated weights for policy 0, policy_version 1751936 (0.0008) [2023-12-27 17:46:58,271][20381] Updated weights for policy 1, policy_version 1750446 (0.0007) [2023-12-27 17:46:58,331][20381] Updated weights for policy 1, policy_version 1750456 (0.0007) [2023-12-27 17:46:59,096][20381] Updated weights for policy 1, policy_version 1750466 (0.0008) [2023-12-27 17:46:59,107][20291] KL-divergence is very high: 196.9835 [2023-12-27 17:46:59,113][20347] Updated weights for policy 0, policy_version 1751946 (0.0007) [2023-12-27 17:46:59,146][20381] Updated weights for policy 1, policy_version 1750476 (0.0008) [2023-12-27 17:46:59,150][20291] KL-divergence is very high: 251.7657 [2023-12-27 17:46:59,159][20291] KL-divergence is very high: 485.6403 [2023-12-27 17:46:59,178][20347] Updated weights for policy 0, policy_version 1751956 (0.0007) [2023-12-27 17:46:59,203][20291] KL-divergence is very high: 280.8878 [2023-12-27 17:46:59,210][20291] KL-divergence is very high: 369.2557 [2023-12-27 17:46:59,211][20381] Updated weights for policy 1, policy_version 1750486 (0.0007) [2023-12-27 17:46:59,244][20347] Updated weights for policy 0, policy_version 1751966 (0.0008) [2023-12-27 17:46:59,274][20381] Updated weights for policy 1, policy_version 1750496 (0.0009) [2023-12-27 17:46:59,907][20291] KL-divergence is very high: 209.5551 [2023-12-27 17:46:59,952][20347] Updated weights for policy 0, policy_version 1751976 (0.0008) [2023-12-27 17:47:00,002][20381] Updated weights for policy 1, policy_version 1750506 (0.0006) [2023-12-27 17:47:00,006][20347] Updated weights for policy 0, policy_version 1751986 (0.0008) [2023-12-27 17:47:00,065][20381] Updated weights for policy 1, policy_version 1750516 (0.0005) [2023-12-27 17:47:00,066][20347] Updated weights for policy 0, policy_version 1751996 (0.0008) [2023-12-27 17:47:00,126][20381] Updated weights for policy 1, policy_version 1750526 (0.0005) [2023-12-27 17:47:00,720][20381] Updated weights for policy 1, policy_version 1750536 (0.0006) [2023-12-27 17:47:00,777][20381] Updated weights for policy 1, policy_version 1750546 (0.0008) [2023-12-27 17:47:00,778][20347] Updated weights for policy 0, policy_version 1752006 (0.0006) [2023-12-27 17:47:00,828][20381] Updated weights for policy 1, policy_version 1750556 (0.0010) [2023-12-27 17:47:00,829][20347] Updated weights for policy 0, policy_version 1752016 (0.0005) [2023-12-27 17:47:00,879][20347] Updated weights for policy 0, policy_version 1752026 (0.0007) [2023-12-27 17:47:01,450][20381] Updated weights for policy 1, policy_version 1750566 (0.0009) [2023-12-27 17:47:01,504][20381] Updated weights for policy 1, policy_version 1750576 (0.0008) [2023-12-27 17:47:01,557][20381] Updated weights for policy 1, policy_version 1750586 (0.0006) [2023-12-27 17:47:01,572][20347] Updated weights for policy 0, policy_version 1752036 (0.0006) [2023-12-27 17:47:01,637][20347] Updated weights for policy 0, policy_version 1752046 (0.0008) [2023-12-27 17:47:01,695][20347] Updated weights for policy 0, policy_version 1752056 (0.0007) [2023-12-27 17:47:02,277][20381] Updated weights for policy 1, policy_version 1750596 (0.0007) [2023-12-27 17:47:02,340][20381] Updated weights for policy 1, policy_version 1750606 (0.0008) [2023-12-27 17:47:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 896811008. Throughput: 0: 10372.8, 1: 10124.4. Samples: 896827648. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:02,363][19170] Avg episode reward: [(0, '921.970'), (1, '1000.000')] [2023-12-27 17:47:02,379][20347] Updated weights for policy 0, policy_version 1752066 (0.0008) [2023-12-27 17:47:02,403][20381] Updated weights for policy 1, policy_version 1750616 (0.0007) [2023-12-27 17:47:02,434][20347] Updated weights for policy 0, policy_version 1752076 (0.0007) [2023-12-27 17:47:02,483][20347] Updated weights for policy 0, policy_version 1752086 (0.0007) [2023-12-27 17:47:02,535][20347] Updated weights for policy 0, policy_version 1752096 (0.0007) [2023-12-27 17:47:03,010][20381] Updated weights for policy 1, policy_version 1750626 (0.0006) [2023-12-27 17:47:03,067][20381] Updated weights for policy 1, policy_version 1750636 (0.0005) [2023-12-27 17:47:03,118][20381] Updated weights for policy 1, policy_version 1750646 (0.0005) [2023-12-27 17:47:03,170][20381] Updated weights for policy 1, policy_version 1750656 (0.0005) [2023-12-27 17:47:03,265][20347] Updated weights for policy 0, policy_version 1752106 (0.0005) [2023-12-27 17:47:03,320][20347] Updated weights for policy 0, policy_version 1752116 (0.0005) [2023-12-27 17:47:03,378][20347] Updated weights for policy 0, policy_version 1752126 (0.0005) [2023-12-27 17:47:03,748][20381] Updated weights for policy 1, policy_version 1750666 (0.0005) [2023-12-27 17:47:03,802][20381] Updated weights for policy 1, policy_version 1750676 (0.0005) [2023-12-27 17:47:03,853][20381] Updated weights for policy 1, policy_version 1750686 (0.0006) [2023-12-27 17:47:03,997][20347] Updated weights for policy 0, policy_version 1752136 (0.0008) [2023-12-27 17:47:04,060][20347] Updated weights for policy 0, policy_version 1752146 (0.0008) [2023-12-27 17:47:04,125][20347] Updated weights for policy 0, policy_version 1752156 (0.0007) [2023-12-27 17:47:04,484][20381] Updated weights for policy 1, policy_version 1750696 (0.0009) [2023-12-27 17:47:04,532][20381] Updated weights for policy 1, policy_version 1750706 (0.0010) [2023-12-27 17:47:04,588][20381] Updated weights for policy 1, policy_version 1750716 (0.0006) [2023-12-27 17:47:04,808][20347] Updated weights for policy 0, policy_version 1752166 (0.0006) [2023-12-27 17:47:04,858][20347] Updated weights for policy 0, policy_version 1752176 (0.0005) [2023-12-27 17:47:04,932][20347] Updated weights for policy 0, policy_version 1752186 (0.0006) [2023-12-27 17:47:05,248][20381] Updated weights for policy 1, policy_version 1750726 (0.0007) [2023-12-27 17:47:05,303][20381] Updated weights for policy 1, policy_version 1750736 (0.0008) [2023-12-27 17:47:05,364][20381] Updated weights for policy 1, policy_version 1750746 (0.0007) [2023-12-27 17:47:05,596][20347] Updated weights for policy 0, policy_version 1752196 (0.0007) [2023-12-27 17:47:05,644][20347] Updated weights for policy 0, policy_version 1752206 (0.0005) [2023-12-27 17:47:05,696][20347] Updated weights for policy 0, policy_version 1752216 (0.0009) [2023-12-27 17:47:05,919][20381] Updated weights for policy 1, policy_version 1750756 (0.0006) [2023-12-27 17:47:05,983][20381] Updated weights for policy 1, policy_version 1750766 (0.0011) [2023-12-27 17:47:06,037][20381] Updated weights for policy 1, policy_version 1750776 (0.0009) [2023-12-27 17:47:06,456][20347] Updated weights for policy 0, policy_version 1752226 (0.0008) [2023-12-27 17:47:06,515][20347] Updated weights for policy 0, policy_version 1752236 (0.0007) [2023-12-27 17:47:06,577][20347] Updated weights for policy 0, policy_version 1752246 (0.0008) [2023-12-27 17:47:06,634][20347] Updated weights for policy 0, policy_version 1752256 (0.0007) [2023-12-27 17:47:06,635][20381] Updated weights for policy 1, policy_version 1750786 (0.0006) [2023-12-27 17:47:06,687][20381] Updated weights for policy 1, policy_version 1750796 (0.0008) [2023-12-27 17:47:06,738][20381] Updated weights for policy 1, policy_version 1750806 (0.0008) [2023-12-27 17:47:06,784][20381] Updated weights for policy 1, policy_version 1750816 (0.0008) [2023-12-27 17:47:07,321][20347] Updated weights for policy 0, policy_version 1752266 (0.0008) [2023-12-27 17:47:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 896917504. Throughput: 0: 10358.2, 1: 10182.3. Samples: 896891288. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:07,363][19170] Avg episode reward: [(0, '941.460'), (1, '990.260')] [2023-12-27 17:47:07,369][20347] Updated weights for policy 0, policy_version 1752276 (0.0007) [2023-12-27 17:47:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001750816_448258048.pth... [2023-12-27 17:47:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001749632_447954944.pth [2023-12-27 17:47:07,427][20347] Updated weights for policy 0, policy_version 1752286 (0.0007) [2023-12-27 17:47:07,436][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001752288_448667648.pth... [2023-12-27 17:47:07,440][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001751072_448356352.pth [2023-12-27 17:47:07,461][20381] Updated weights for policy 1, policy_version 1750826 (0.0008) [2023-12-27 17:47:07,513][20381] Updated weights for policy 1, policy_version 1750836 (0.0005) [2023-12-27 17:47:07,568][20381] Updated weights for policy 1, policy_version 1750846 (0.0005) [2023-12-27 17:47:08,120][20381] Updated weights for policy 1, policy_version 1750856 (0.0006) [2023-12-27 17:47:08,137][20347] Updated weights for policy 0, policy_version 1752296 (0.0007) [2023-12-27 17:47:08,172][20381] Updated weights for policy 1, policy_version 1750866 (0.0008) [2023-12-27 17:47:08,193][20347] Updated weights for policy 0, policy_version 1752306 (0.0005) [2023-12-27 17:47:08,229][20381] Updated weights for policy 1, policy_version 1750876 (0.0008) [2023-12-27 17:47:08,247][20347] Updated weights for policy 0, policy_version 1752316 (0.0005) [2023-12-27 17:47:08,926][20381] Updated weights for policy 1, policy_version 1750886 (0.0007) [2023-12-27 17:47:08,936][20347] Updated weights for policy 0, policy_version 1752326 (0.0007) [2023-12-27 17:47:08,978][20381] Updated weights for policy 1, policy_version 1750896 (0.0006) [2023-12-27 17:47:08,988][20347] Updated weights for policy 0, policy_version 1752336 (0.0007) [2023-12-27 17:47:09,040][20381] Updated weights for policy 1, policy_version 1750906 (0.0006) [2023-12-27 17:47:09,051][20347] Updated weights for policy 0, policy_version 1752346 (0.0007) [2023-12-27 17:47:09,726][20381] Updated weights for policy 1, policy_version 1750916 (0.0007) [2023-12-27 17:47:09,786][20381] Updated weights for policy 1, policy_version 1750926 (0.0008) [2023-12-27 17:47:09,792][20347] Updated weights for policy 0, policy_version 1752356 (0.0008) [2023-12-27 17:47:09,849][20381] Updated weights for policy 1, policy_version 1750936 (0.0008) [2023-12-27 17:47:09,860][20347] Updated weights for policy 0, policy_version 1752366 (0.0009) [2023-12-27 17:47:09,923][20347] Updated weights for policy 0, policy_version 1752376 (0.0007) [2023-12-27 17:47:10,505][20381] Updated weights for policy 1, policy_version 1750946 (0.0008) [2023-12-27 17:47:10,559][20381] Updated weights for policy 1, policy_version 1750956 (0.0008) [2023-12-27 17:47:10,598][20347] Updated weights for policy 0, policy_version 1752386 (0.0008) [2023-12-27 17:47:10,607][20381] Updated weights for policy 1, policy_version 1750966 (0.0006) [2023-12-27 17:47:10,653][20347] Updated weights for policy 0, policy_version 1752396 (0.0006) [2023-12-27 17:47:10,658][20381] Updated weights for policy 1, policy_version 1750976 (0.0007) [2023-12-27 17:47:10,712][20347] Updated weights for policy 0, policy_version 1752406 (0.0010) [2023-12-27 17:47:10,774][20347] Updated weights for policy 0, policy_version 1752416 (0.0010) [2023-12-27 17:47:11,466][20381] Updated weights for policy 1, policy_version 1750986 (0.0008) [2023-12-27 17:47:11,517][20347] Updated weights for policy 0, policy_version 1752426 (0.0007) [2023-12-27 17:47:11,537][20381] Updated weights for policy 1, policy_version 1750996 (0.0008) [2023-12-27 17:47:11,580][20347] Updated weights for policy 0, policy_version 1752436 (0.0007) [2023-12-27 17:47:11,603][20381] Updated weights for policy 1, policy_version 1751006 (0.0007) [2023-12-27 17:47:11,650][20347] Updated weights for policy 0, policy_version 1752446 (0.0008) [2023-12-27 17:47:12,305][20381] Updated weights for policy 1, policy_version 1751016 (0.0008) [2023-12-27 17:47:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 897015808. Throughput: 0: 10345.9, 1: 10202.4. Samples: 897015020. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:12,363][19170] Avg episode reward: [(0, '961.020'), (1, '990.260')] [2023-12-27 17:47:12,364][20381] Updated weights for policy 1, policy_version 1751026 (0.0008) [2023-12-27 17:47:12,372][20347] Updated weights for policy 0, policy_version 1752456 (0.0007) [2023-12-27 17:47:12,429][20381] Updated weights for policy 1, policy_version 1751036 (0.0007) [2023-12-27 17:47:12,432][20347] Updated weights for policy 0, policy_version 1752466 (0.0007) [2023-12-27 17:47:12,486][20347] Updated weights for policy 0, policy_version 1752476 (0.0007) [2023-12-27 17:47:13,114][20381] Updated weights for policy 1, policy_version 1751046 (0.0008) [2023-12-27 17:47:13,174][20381] Updated weights for policy 1, policy_version 1751056 (0.0008) [2023-12-27 17:47:13,200][20347] Updated weights for policy 0, policy_version 1752486 (0.0007) [2023-12-27 17:47:13,226][20381] Updated weights for policy 1, policy_version 1751066 (0.0008) [2023-12-27 17:47:13,249][20347] Updated weights for policy 0, policy_version 1752496 (0.0005) [2023-12-27 17:47:13,303][20347] Updated weights for policy 0, policy_version 1752506 (0.0007) [2023-12-27 17:47:13,846][20381] Updated weights for policy 1, policy_version 1751076 (0.0007) [2023-12-27 17:47:13,895][20381] Updated weights for policy 1, policy_version 1751086 (0.0005) [2023-12-27 17:47:13,943][20381] Updated weights for policy 1, policy_version 1751096 (0.0006) [2023-12-27 17:47:14,030][20347] Updated weights for policy 0, policy_version 1752516 (0.0008) [2023-12-27 17:47:14,080][20347] Updated weights for policy 0, policy_version 1752527 (0.0008) [2023-12-27 17:47:14,133][20347] Updated weights for policy 0, policy_version 1752537 (0.0008) [2023-12-27 17:47:14,579][20381] Updated weights for policy 1, policy_version 1751106 (0.0008) [2023-12-27 17:47:14,629][20381] Updated weights for policy 1, policy_version 1751116 (0.0009) [2023-12-27 17:47:14,679][20381] Updated weights for policy 1, policy_version 1751126 (0.0008) [2023-12-27 17:47:14,736][20381] Updated weights for policy 1, policy_version 1751136 (0.0008) [2023-12-27 17:47:14,776][20347] Updated weights for policy 0, policy_version 1752547 (0.0008) [2023-12-27 17:47:14,844][20347] Updated weights for policy 0, policy_version 1752557 (0.0008) [2023-12-27 17:47:14,902][20347] Updated weights for policy 0, policy_version 1752567 (0.0008) [2023-12-27 17:47:15,490][20381] Updated weights for policy 1, policy_version 1751146 (0.0006) [2023-12-27 17:47:15,542][20381] Updated weights for policy 1, policy_version 1751156 (0.0007) [2023-12-27 17:47:15,586][20347] Updated weights for policy 0, policy_version 1752577 (0.0007) [2023-12-27 17:47:15,593][20381] Updated weights for policy 1, policy_version 1751166 (0.0008) [2023-12-27 17:47:15,645][20347] Updated weights for policy 0, policy_version 1752587 (0.0008) [2023-12-27 17:47:15,697][20347] Updated weights for policy 0, policy_version 1752597 (0.0007) [2023-12-27 17:47:15,755][20347] Updated weights for policy 0, policy_version 1752607 (0.0005) [2023-12-27 17:47:16,292][20381] Updated weights for policy 1, policy_version 1751176 (0.0005) [2023-12-27 17:47:16,322][20347] Updated weights for policy 0, policy_version 1752617 (0.0010) [2023-12-27 17:47:16,343][20381] Updated weights for policy 1, policy_version 1751186 (0.0005) [2023-12-27 17:47:16,370][20347] Updated weights for policy 0, policy_version 1752627 (0.0010) [2023-12-27 17:47:16,391][20381] Updated weights for policy 1, policy_version 1751196 (0.0005) [2023-12-27 17:47:16,421][20347] Updated weights for policy 0, policy_version 1752637 (0.0010) [2023-12-27 17:47:17,024][20381] Updated weights for policy 1, policy_version 1751206 (0.0006) [2023-12-27 17:47:17,077][20381] Updated weights for policy 1, policy_version 1751216 (0.0007) [2023-12-27 17:47:17,121][20347] Updated weights for policy 0, policy_version 1752647 (0.0010) [2023-12-27 17:47:17,130][20381] Updated weights for policy 1, policy_version 1751226 (0.0006) [2023-12-27 17:47:17,178][20347] Updated weights for policy 0, policy_version 1752657 (0.0008) [2023-12-27 17:47:17,226][20347] Updated weights for policy 0, policy_version 1752667 (0.0005) [2023-12-27 17:47:17,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 897130496. Throughput: 0: 10257.0, 1: 10190.7. Samples: 897136220. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:17,363][19170] Avg episode reward: [(0, '951.170'), (1, '1000.000')] [2023-12-27 17:47:17,739][20381] Updated weights for policy 1, policy_version 1751236 (0.0007) [2023-12-27 17:47:17,773][20347] Updated weights for policy 0, policy_version 1752677 (0.0005) [2023-12-27 17:47:17,794][20381] Updated weights for policy 1, policy_version 1751246 (0.0008) [2023-12-27 17:47:17,827][20347] Updated weights for policy 0, policy_version 1752687 (0.0006) [2023-12-27 17:47:17,859][20381] Updated weights for policy 1, policy_version 1751256 (0.0007) [2023-12-27 17:47:17,880][20347] Updated weights for policy 0, policy_version 1752697 (0.0006) [2023-12-27 17:47:18,533][20381] Updated weights for policy 1, policy_version 1751266 (0.0006) [2023-12-27 17:47:18,583][20381] Updated weights for policy 1, policy_version 1751276 (0.0007) [2023-12-27 17:47:18,590][20347] Updated weights for policy 0, policy_version 1752707 (0.0007) [2023-12-27 17:47:18,640][20381] Updated weights for policy 1, policy_version 1751286 (0.0007) [2023-12-27 17:47:18,643][20347] Updated weights for policy 0, policy_version 1752717 (0.0008) [2023-12-27 17:47:18,692][20381] Updated weights for policy 1, policy_version 1751296 (0.0007) [2023-12-27 17:47:18,699][20347] Updated weights for policy 0, policy_version 1752727 (0.0008) [2023-12-27 17:47:19,412][20381] Updated weights for policy 1, policy_version 1751306 (0.0008) [2023-12-27 17:47:19,443][20347] Updated weights for policy 0, policy_version 1752737 (0.0008) [2023-12-27 17:47:19,469][20381] Updated weights for policy 1, policy_version 1751316 (0.0006) [2023-12-27 17:47:19,515][20347] Updated weights for policy 0, policy_version 1752747 (0.0008) [2023-12-27 17:47:19,530][20381] Updated weights for policy 1, policy_version 1751326 (0.0007) [2023-12-27 17:47:19,569][20347] Updated weights for policy 0, policy_version 1752757 (0.0008) [2023-12-27 17:47:19,628][20347] Updated weights for policy 0, policy_version 1752767 (0.0008) [2023-12-27 17:47:20,199][20381] Updated weights for policy 1, policy_version 1751336 (0.0007) [2023-12-27 17:47:20,258][20381] Updated weights for policy 1, policy_version 1751346 (0.0007) [2023-12-27 17:47:20,312][20381] Updated weights for policy 1, policy_version 1751356 (0.0008) [2023-12-27 17:47:20,400][20347] Updated weights for policy 0, policy_version 1752777 (0.0008) [2023-12-27 17:47:20,460][20347] Updated weights for policy 0, policy_version 1752787 (0.0008) [2023-12-27 17:47:20,512][20347] Updated weights for policy 0, policy_version 1752797 (0.0008) [2023-12-27 17:47:20,982][20381] Updated weights for policy 1, policy_version 1751366 (0.0007) [2023-12-27 17:47:21,047][20381] Updated weights for policy 1, policy_version 1751376 (0.0009) [2023-12-27 17:47:21,103][20381] Updated weights for policy 1, policy_version 1751386 (0.0008) [2023-12-27 17:47:21,255][20347] Updated weights for policy 0, policy_version 1752807 (0.0008) [2023-12-27 17:47:21,315][20347] Updated weights for policy 0, policy_version 1752817 (0.0007) [2023-12-27 17:47:21,377][20347] Updated weights for policy 0, policy_version 1752827 (0.0007) [2023-12-27 17:47:21,917][20381] Updated weights for policy 1, policy_version 1751396 (0.0009) [2023-12-27 17:47:21,969][20381] Updated weights for policy 1, policy_version 1751406 (0.0008) [2023-12-27 17:47:22,023][20381] Updated weights for policy 1, policy_version 1751416 (0.0008) [2023-12-27 17:47:22,072][20347] Updated weights for policy 0, policy_version 1752837 (0.0007) [2023-12-27 17:47:22,122][20347] Updated weights for policy 0, policy_version 1752847 (0.0008) [2023-12-27 17:47:22,181][20347] Updated weights for policy 0, policy_version 1752857 (0.0008) [2023-12-27 17:47:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 897228800. Throughput: 0: 10224.8, 1: 10218.5. Samples: 897197752. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:22,363][19170] Avg episode reward: [(0, '951.090'), (1, '1000.000')] [2023-12-27 17:47:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001751424_448413696.pth... [2023-12-27 17:47:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001752864_448815104.pth... [2023-12-27 17:47:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001750208_448102400.pth [2023-12-27 17:47:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001751680_448512000.pth [2023-12-27 17:47:22,756][20381] Updated weights for policy 1, policy_version 1751426 (0.0007) [2023-12-27 17:47:22,817][20381] Updated weights for policy 1, policy_version 1751436 (0.0010) [2023-12-27 17:47:22,874][20381] Updated weights for policy 1, policy_version 1751446 (0.0008) [2023-12-27 17:47:22,878][20347] Updated weights for policy 0, policy_version 1752867 (0.0007) [2023-12-27 17:47:22,928][20347] Updated weights for policy 0, policy_version 1752877 (0.0009) [2023-12-27 17:47:22,931][20381] Updated weights for policy 1, policy_version 1751456 (0.0008) [2023-12-27 17:47:22,988][20347] Updated weights for policy 0, policy_version 1752887 (0.0010) [2023-12-27 17:47:23,559][20381] Updated weights for policy 1, policy_version 1751466 (0.0005) [2023-12-27 17:47:23,616][20381] Updated weights for policy 1, policy_version 1751476 (0.0007) [2023-12-27 17:47:23,638][20347] Updated weights for policy 0, policy_version 1752897 (0.0010) [2023-12-27 17:47:23,665][20381] Updated weights for policy 1, policy_version 1751486 (0.0006) [2023-12-27 17:47:23,691][20347] Updated weights for policy 0, policy_version 1752907 (0.0009) [2023-12-27 17:47:23,746][20347] Updated weights for policy 0, policy_version 1752917 (0.0010) [2023-12-27 17:47:23,798][20347] Updated weights for policy 0, policy_version 1752927 (0.0005) [2023-12-27 17:47:24,311][20381] Updated weights for policy 1, policy_version 1751496 (0.0006) [2023-12-27 17:47:24,364][20381] Updated weights for policy 1, policy_version 1751506 (0.0007) [2023-12-27 17:47:24,375][20347] Updated weights for policy 0, policy_version 1752937 (0.0006) [2023-12-27 17:47:24,410][20381] Updated weights for policy 1, policy_version 1751516 (0.0006) [2023-12-27 17:47:24,430][20347] Updated weights for policy 0, policy_version 1752947 (0.0006) [2023-12-27 17:47:24,491][20347] Updated weights for policy 0, policy_version 1752957 (0.0007) [2023-12-27 17:47:25,023][20381] Updated weights for policy 1, policy_version 1751526 (0.0006) [2023-12-27 17:47:25,072][20381] Updated weights for policy 1, policy_version 1751536 (0.0007) [2023-12-27 17:47:25,120][20381] Updated weights for policy 1, policy_version 1751546 (0.0006) [2023-12-27 17:47:25,132][20347] Updated weights for policy 0, policy_version 1752967 (0.0007) [2023-12-27 17:47:25,190][20347] Updated weights for policy 0, policy_version 1752977 (0.0008) [2023-12-27 17:47:25,249][20347] Updated weights for policy 0, policy_version 1752987 (0.0008) [2023-12-27 17:47:25,784][20381] Updated weights for policy 1, policy_version 1751556 (0.0006) [2023-12-27 17:47:25,833][20381] Updated weights for policy 1, policy_version 1751566 (0.0005) [2023-12-27 17:47:25,884][20381] Updated weights for policy 1, policy_version 1751576 (0.0005) [2023-12-27 17:47:25,898][20347] Updated weights for policy 0, policy_version 1752997 (0.0007) [2023-12-27 17:47:25,956][20347] Updated weights for policy 0, policy_version 1753007 (0.0007) [2023-12-27 17:47:26,011][20347] Updated weights for policy 0, policy_version 1753017 (0.0006) [2023-12-27 17:47:26,541][20347] Updated weights for policy 0, policy_version 1753027 (0.0005) [2023-12-27 17:47:26,587][20381] Updated weights for policy 1, policy_version 1751586 (0.0007) [2023-12-27 17:47:26,599][20347] Updated weights for policy 0, policy_version 1753037 (0.0005) [2023-12-27 17:47:26,638][20381] Updated weights for policy 1, policy_version 1751596 (0.0008) [2023-12-27 17:47:26,656][20347] Updated weights for policy 0, policy_version 1753047 (0.0005) [2023-12-27 17:47:26,685][20381] Updated weights for policy 1, policy_version 1751606 (0.0007) [2023-12-27 17:47:26,731][20381] Updated weights for policy 1, policy_version 1751616 (0.0007) [2023-12-27 17:47:27,205][20347] Updated weights for policy 0, policy_version 1753057 (0.0005) [2023-12-27 17:47:27,247][20347] Updated weights for policy 0, policy_version 1753067 (0.0006) [2023-12-27 17:47:27,284][20291] KL-divergence is very high: 130.2228 [2023-12-27 17:47:27,295][20347] Updated weights for policy 0, policy_version 1753077 (0.0005) [2023-12-27 17:47:27,353][20347] Updated weights for policy 0, policy_version 1753087 (0.0006) [2023-12-27 17:47:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 897335296. Throughput: 0: 10211.8, 1: 10255.2. Samples: 897320908. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:27,363][19170] Avg episode reward: [(0, '960.830'), (1, '1000.000')] [2023-12-27 17:47:27,472][20381] Updated weights for policy 1, policy_version 1751626 (0.0007) [2023-12-27 17:47:27,524][20381] Updated weights for policy 1, policy_version 1751636 (0.0008) [2023-12-27 17:47:27,577][20381] Updated weights for policy 1, policy_version 1751646 (0.0007) [2023-12-27 17:47:27,947][20347] Updated weights for policy 0, policy_version 1753097 (0.0007) [2023-12-27 17:47:27,999][20347] Updated weights for policy 0, policy_version 1753107 (0.0007) [2023-12-27 17:47:28,052][20347] Updated weights for policy 0, policy_version 1753117 (0.0005) [2023-12-27 17:47:28,317][20381] Updated weights for policy 1, policy_version 1751656 (0.0009) [2023-12-27 17:47:28,382][20381] Updated weights for policy 1, policy_version 1751666 (0.0006) [2023-12-27 17:47:28,436][20381] Updated weights for policy 1, policy_version 1751676 (0.0007) [2023-12-27 17:47:28,612][20347] Updated weights for policy 0, policy_version 1753127 (0.0006) [2023-12-27 17:47:28,665][20347] Updated weights for policy 0, policy_version 1753137 (0.0008) [2023-12-27 17:47:28,726][20347] Updated weights for policy 0, policy_version 1753147 (0.0010) [2023-12-27 17:47:29,139][20381] Updated weights for policy 1, policy_version 1751686 (0.0006) [2023-12-27 17:47:29,190][20381] Updated weights for policy 1, policy_version 1751696 (0.0007) [2023-12-27 17:47:29,253][20381] Updated weights for policy 1, policy_version 1751706 (0.0008) [2023-12-27 17:47:29,347][20347] Updated weights for policy 0, policy_version 1753157 (0.0010) [2023-12-27 17:47:29,408][20347] Updated weights for policy 0, policy_version 1753167 (0.0008) [2023-12-27 17:47:29,460][20347] Updated weights for policy 0, policy_version 1753177 (0.0007) [2023-12-27 17:47:29,919][20381] Updated weights for policy 1, policy_version 1751716 (0.0007) [2023-12-27 17:47:29,977][20381] Updated weights for policy 1, policy_version 1751726 (0.0007) [2023-12-27 17:47:30,034][20381] Updated weights for policy 1, policy_version 1751736 (0.0006) [2023-12-27 17:47:30,188][20347] Updated weights for policy 0, policy_version 1753187 (0.0008) [2023-12-27 17:47:30,250][20347] Updated weights for policy 0, policy_version 1753197 (0.0006) [2023-12-27 17:47:30,310][20347] Updated weights for policy 0, policy_version 1753207 (0.0009) [2023-12-27 17:47:30,615][20381] Updated weights for policy 1, policy_version 1751746 (0.0006) [2023-12-27 17:47:30,668][20381] Updated weights for policy 1, policy_version 1751756 (0.0008) [2023-12-27 17:47:30,728][20381] Updated weights for policy 1, policy_version 1751766 (0.0007) [2023-12-27 17:47:30,772][20381] Updated weights for policy 1, policy_version 1751776 (0.0007) [2023-12-27 17:47:30,912][20347] Updated weights for policy 0, policy_version 1753217 (0.0009) [2023-12-27 17:47:30,959][20347] Updated weights for policy 0, policy_version 1753227 (0.0006) [2023-12-27 17:47:31,011][20347] Updated weights for policy 0, policy_version 1753237 (0.0007) [2023-12-27 17:47:31,070][20347] Updated weights for policy 0, policy_version 1753247 (0.0007) [2023-12-27 17:47:31,472][20381] Updated weights for policy 1, policy_version 1751786 (0.0008) [2023-12-27 17:47:31,514][20381] Updated weights for policy 1, policy_version 1751796 (0.0007) [2023-12-27 17:47:31,574][20381] Updated weights for policy 1, policy_version 1751806 (0.0008) [2023-12-27 17:47:31,786][20347] Updated weights for policy 0, policy_version 1753257 (0.0008) [2023-12-27 17:47:31,850][20347] Updated weights for policy 0, policy_version 1753267 (0.0005) [2023-12-27 17:47:31,910][20347] Updated weights for policy 0, policy_version 1753277 (0.0006) [2023-12-27 17:47:32,338][20381] Updated weights for policy 1, policy_version 1751816 (0.0008) [2023-12-27 17:47:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.2, 300 sec: 20549.4). Total num frames: 897433600. Throughput: 0: 10322.4, 1: 10240.5. Samples: 897447984. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:32,363][19170] Avg episode reward: [(0, '970.620'), (1, '1000.000')] [2023-12-27 17:47:32,394][20381] Updated weights for policy 1, policy_version 1751826 (0.0009) [2023-12-27 17:47:32,446][20381] Updated weights for policy 1, policy_version 1751836 (0.0007) [2023-12-27 17:47:32,489][20347] Updated weights for policy 0, policy_version 1753287 (0.0008) [2023-12-27 17:47:32,551][20347] Updated weights for policy 0, policy_version 1753297 (0.0011) [2023-12-27 17:47:32,611][20347] Updated weights for policy 0, policy_version 1753307 (0.0006) [2023-12-27 17:47:33,108][20381] Updated weights for policy 1, policy_version 1751846 (0.0006) [2023-12-27 17:47:33,162][20381] Updated weights for policy 1, policy_version 1751856 (0.0010) [2023-12-27 17:47:33,222][20381] Updated weights for policy 1, policy_version 1751866 (0.0010) [2023-12-27 17:47:33,245][20347] Updated weights for policy 0, policy_version 1753317 (0.0008) [2023-12-27 17:47:33,290][20347] Updated weights for policy 0, policy_version 1753327 (0.0010) [2023-12-27 17:47:33,341][20347] Updated weights for policy 0, policy_version 1753337 (0.0008) [2023-12-27 17:47:33,804][20381] Updated weights for policy 1, policy_version 1751876 (0.0008) [2023-12-27 17:47:33,847][20381] Updated weights for policy 1, policy_version 1751886 (0.0007) [2023-12-27 17:47:33,888][20381] Updated weights for policy 1, policy_version 1751896 (0.0007) [2023-12-27 17:47:34,010][20347] Updated weights for policy 0, policy_version 1753347 (0.0008) [2023-12-27 17:47:34,063][20347] Updated weights for policy 0, policy_version 1753357 (0.0010) [2023-12-27 17:47:34,114][20347] Updated weights for policy 0, policy_version 1753367 (0.0010) [2023-12-27 17:47:34,556][20381] Updated weights for policy 1, policy_version 1751906 (0.0007) [2023-12-27 17:47:34,611][20381] Updated weights for policy 1, policy_version 1751916 (0.0008) [2023-12-27 17:47:34,671][20381] Updated weights for policy 1, policy_version 1751926 (0.0007) [2023-12-27 17:47:34,733][20381] Updated weights for policy 1, policy_version 1751936 (0.0007) [2023-12-27 17:47:34,810][20347] Updated weights for policy 0, policy_version 1753377 (0.0010) [2023-12-27 17:47:34,863][20347] Updated weights for policy 0, policy_version 1753387 (0.0006) [2023-12-27 17:47:34,915][20347] Updated weights for policy 0, policy_version 1753397 (0.0005) [2023-12-27 17:47:34,974][20347] Updated weights for policy 0, policy_version 1753407 (0.0005) [2023-12-27 17:47:35,429][20381] Updated weights for policy 1, policy_version 1751946 (0.0010) [2023-12-27 17:47:35,486][20381] Updated weights for policy 1, policy_version 1751956 (0.0011) [2023-12-27 17:47:35,533][20381] Updated weights for policy 1, policy_version 1751966 (0.0010) [2023-12-27 17:47:35,630][20347] Updated weights for policy 0, policy_version 1753417 (0.0010) [2023-12-27 17:47:35,688][20347] Updated weights for policy 0, policy_version 1753427 (0.0010) [2023-12-27 17:47:35,741][20347] Updated weights for policy 0, policy_version 1753437 (0.0010) [2023-12-27 17:47:36,248][20381] Updated weights for policy 1, policy_version 1751976 (0.0008) [2023-12-27 17:47:36,300][20381] Updated weights for policy 1, policy_version 1751986 (0.0008) [2023-12-27 17:47:36,355][20381] Updated weights for policy 1, policy_version 1751996 (0.0008) [2023-12-27 17:47:36,465][20347] Updated weights for policy 0, policy_version 1753447 (0.0008) [2023-12-27 17:47:36,522][20347] Updated weights for policy 0, policy_version 1753457 (0.0007) [2023-12-27 17:47:36,578][20347] Updated weights for policy 0, policy_version 1753467 (0.0008) [2023-12-27 17:47:37,043][20381] Updated weights for policy 1, policy_version 1752006 (0.0009) [2023-12-27 17:47:37,091][20381] Updated weights for policy 1, policy_version 1752016 (0.0010) [2023-12-27 17:47:37,146][20381] Updated weights for policy 1, policy_version 1752026 (0.0006) [2023-12-27 17:47:37,253][20347] Updated weights for policy 0, policy_version 1753477 (0.0009) [2023-12-27 17:47:37,302][20347] Updated weights for policy 0, policy_version 1753487 (0.0010) [2023-12-27 17:47:37,357][20347] Updated weights for policy 0, policy_version 1753497 (0.0010) [2023-12-27 17:47:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 897540096. Throughput: 0: 10328.1, 1: 10280.7. Samples: 897511940. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:37,363][19170] Avg episode reward: [(0, '960.980'), (1, '1000.000')] [2023-12-27 17:47:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001752032_448569344.pth... [2023-12-27 17:47:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001750816_448258048.pth [2023-12-27 17:47:37,396][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001753504_448978944.pth... [2023-12-27 17:47:37,400][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001752288_448667648.pth [2023-12-27 17:47:37,815][20381] Updated weights for policy 1, policy_version 1752036 (0.0005) [2023-12-27 17:47:37,861][20381] Updated weights for policy 1, policy_version 1752046 (0.0008) [2023-12-27 17:47:37,908][20381] Updated weights for policy 1, policy_version 1752056 (0.0006) [2023-12-27 17:47:38,014][20347] Updated weights for policy 0, policy_version 1753507 (0.0009) [2023-12-27 17:47:38,064][20347] Updated weights for policy 0, policy_version 1753517 (0.0007) [2023-12-27 17:47:38,119][20347] Updated weights for policy 0, policy_version 1753527 (0.0008) [2023-12-27 17:47:38,565][20381] Updated weights for policy 1, policy_version 1752066 (0.0006) [2023-12-27 17:47:38,627][20381] Updated weights for policy 1, policy_version 1752076 (0.0007) [2023-12-27 17:47:38,684][20381] Updated weights for policy 1, policy_version 1752086 (0.0007) [2023-12-27 17:47:38,747][20381] Updated weights for policy 1, policy_version 1752096 (0.0007) [2023-12-27 17:47:38,822][20347] Updated weights for policy 0, policy_version 1753537 (0.0008) [2023-12-27 17:47:38,877][20347] Updated weights for policy 0, policy_version 1753547 (0.0010) [2023-12-27 17:47:38,929][20347] Updated weights for policy 0, policy_version 1753557 (0.0010) [2023-12-27 17:47:38,980][20347] Updated weights for policy 0, policy_version 1753567 (0.0010) [2023-12-27 17:47:39,429][20381] Updated weights for policy 1, policy_version 1752106 (0.0007) [2023-12-27 17:47:39,487][20381] Updated weights for policy 1, policy_version 1752116 (0.0007) [2023-12-27 17:47:39,547][20381] Updated weights for policy 1, policy_version 1752126 (0.0009) [2023-12-27 17:47:39,699][20347] Updated weights for policy 0, policy_version 1753577 (0.0008) [2023-12-27 17:47:39,766][20347] Updated weights for policy 0, policy_version 1753587 (0.0008) [2023-12-27 17:47:39,834][20347] Updated weights for policy 0, policy_version 1753597 (0.0008) [2023-12-27 17:47:40,316][20381] Updated weights for policy 1, policy_version 1752136 (0.0008) [2023-12-27 17:47:40,380][20381] Updated weights for policy 1, policy_version 1752146 (0.0008) [2023-12-27 17:47:40,436][20381] Updated weights for policy 1, policy_version 1752156 (0.0008) [2023-12-27 17:47:40,505][20347] Updated weights for policy 0, policy_version 1753607 (0.0008) [2023-12-27 17:47:40,557][20347] Updated weights for policy 0, policy_version 1753617 (0.0007) [2023-12-27 17:47:40,603][20347] Updated weights for policy 0, policy_version 1753627 (0.0008) [2023-12-27 17:47:41,188][20381] Updated weights for policy 1, policy_version 1752166 (0.0006) [2023-12-27 17:47:41,249][20381] Updated weights for policy 1, policy_version 1752176 (0.0007) [2023-12-27 17:47:41,312][20381] Updated weights for policy 1, policy_version 1752186 (0.0007) [2023-12-27 17:47:41,377][20347] Updated weights for policy 0, policy_version 1753637 (0.0008) [2023-12-27 17:47:41,442][20347] Updated weights for policy 0, policy_version 1753647 (0.0007) [2023-12-27 17:47:41,503][20347] Updated weights for policy 0, policy_version 1753657 (0.0007) [2023-12-27 17:47:42,025][20381] Updated weights for policy 1, policy_version 1752196 (0.0007) [2023-12-27 17:47:42,081][20381] Updated weights for policy 1, policy_version 1752206 (0.0008) [2023-12-27 17:47:42,134][20381] Updated weights for policy 1, policy_version 1752216 (0.0008) [2023-12-27 17:47:42,241][20347] Updated weights for policy 0, policy_version 1753667 (0.0007) [2023-12-27 17:47:42,303][20347] Updated weights for policy 0, policy_version 1753677 (0.0008) [2023-12-27 17:47:42,362][20347] Updated weights for policy 0, policy_version 1753687 (0.0008) [2023-12-27 17:47:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 897638400. Throughput: 0: 10288.1, 1: 10273.5. Samples: 897631672. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:42,363][19170] Avg episode reward: [(0, '921.920'), (1, '1000.000')] [2023-12-27 17:47:42,834][20381] Updated weights for policy 1, policy_version 1752226 (0.0008) [2023-12-27 17:47:42,887][20381] Updated weights for policy 1, policy_version 1752236 (0.0007) [2023-12-27 17:47:42,940][20381] Updated weights for policy 1, policy_version 1752246 (0.0008) [2023-12-27 17:47:42,992][20381] Updated weights for policy 1, policy_version 1752256 (0.0007) [2023-12-27 17:47:43,038][20347] Updated weights for policy 0, policy_version 1753697 (0.0008) [2023-12-27 17:47:43,091][20347] Updated weights for policy 0, policy_version 1753707 (0.0007) [2023-12-27 17:47:43,136][20347] Updated weights for policy 0, policy_version 1753717 (0.0007) [2023-12-27 17:47:43,180][20347] Updated weights for policy 0, policy_version 1753727 (0.0007) [2023-12-27 17:47:43,686][20381] Updated weights for policy 1, policy_version 1752266 (0.0005) [2023-12-27 17:47:43,746][20381] Updated weights for policy 1, policy_version 1752276 (0.0005) [2023-12-27 17:47:43,806][20381] Updated weights for policy 1, policy_version 1752286 (0.0008) [2023-12-27 17:47:43,870][20347] Updated weights for policy 0, policy_version 1753737 (0.0008) [2023-12-27 17:47:43,918][20347] Updated weights for policy 0, policy_version 1753747 (0.0006) [2023-12-27 17:47:43,971][20347] Updated weights for policy 0, policy_version 1753757 (0.0005) [2023-12-27 17:47:44,398][20381] Updated weights for policy 1, policy_version 1752296 (0.0006) [2023-12-27 17:47:44,444][20381] Updated weights for policy 1, policy_version 1752306 (0.0005) [2023-12-27 17:47:44,493][20381] Updated weights for policy 1, policy_version 1752316 (0.0005) [2023-12-27 17:47:44,540][20347] Updated weights for policy 0, policy_version 1753767 (0.0008) [2023-12-27 17:47:44,597][20347] Updated weights for policy 0, policy_version 1753777 (0.0010) [2023-12-27 17:47:44,652][20347] Updated weights for policy 0, policy_version 1753787 (0.0010) [2023-12-27 17:47:45,182][20381] Updated weights for policy 1, policy_version 1752326 (0.0007) [2023-12-27 17:47:45,244][20381] Updated weights for policy 1, policy_version 1752336 (0.0008) [2023-12-27 17:47:45,303][20381] Updated weights for policy 1, policy_version 1752346 (0.0008) [2023-12-27 17:47:45,335][20347] Updated weights for policy 0, policy_version 1753797 (0.0010) [2023-12-27 17:47:45,381][20347] Updated weights for policy 0, policy_version 1753807 (0.0010) [2023-12-27 17:47:45,439][20347] Updated weights for policy 0, policy_version 1753817 (0.0007) [2023-12-27 17:47:45,961][20381] Updated weights for policy 1, policy_version 1752356 (0.0009) [2023-12-27 17:47:46,023][20381] Updated weights for policy 1, policy_version 1752366 (0.0011) [2023-12-27 17:47:46,082][20381] Updated weights for policy 1, policy_version 1752376 (0.0011) [2023-12-27 17:47:46,125][20347] Updated weights for policy 0, policy_version 1753827 (0.0010) [2023-12-27 17:47:46,180][20347] Updated weights for policy 0, policy_version 1753837 (0.0010) [2023-12-27 17:47:46,230][20347] Updated weights for policy 0, policy_version 1753847 (0.0005) [2023-12-27 17:47:46,642][20381] Updated weights for policy 1, policy_version 1752386 (0.0007) [2023-12-27 17:47:46,694][20381] Updated weights for policy 1, policy_version 1752396 (0.0005) [2023-12-27 17:47:46,755][20381] Updated weights for policy 1, policy_version 1752406 (0.0005) [2023-12-27 17:47:46,806][20381] Updated weights for policy 1, policy_version 1752416 (0.0005) [2023-12-27 17:47:46,908][20347] Updated weights for policy 0, policy_version 1753857 (0.0006) [2023-12-27 17:47:46,968][20347] Updated weights for policy 0, policy_version 1753867 (0.0006) [2023-12-27 17:47:47,023][20347] Updated weights for policy 0, policy_version 1753877 (0.0007) [2023-12-27 17:47:47,086][20347] Updated weights for policy 0, policy_version 1753887 (0.0007) [2023-12-27 17:47:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 897744896. Throughput: 0: 10297.1, 1: 10346.7. Samples: 897756616. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:47,363][19170] Avg episode reward: [(0, '902.200'), (1, '1000.000')] [2023-12-27 17:47:47,419][20381] Updated weights for policy 1, policy_version 1752426 (0.0009) [2023-12-27 17:47:47,484][20381] Updated weights for policy 1, policy_version 1752436 (0.0011) [2023-12-27 17:47:47,545][20381] Updated weights for policy 1, policy_version 1752446 (0.0011) [2023-12-27 17:47:47,686][20347] Updated weights for policy 0, policy_version 1753897 (0.0010) [2023-12-27 17:47:47,740][20347] Updated weights for policy 0, policy_version 1753907 (0.0010) [2023-12-27 17:47:47,798][20347] Updated weights for policy 0, policy_version 1753917 (0.0007) [2023-12-27 17:47:48,182][20381] Updated weights for policy 1, policy_version 1752456 (0.0009) [2023-12-27 17:47:48,237][20381] Updated weights for policy 1, policy_version 1752466 (0.0011) [2023-12-27 17:47:48,286][20381] Updated weights for policy 1, policy_version 1752476 (0.0011) [2023-12-27 17:47:48,406][20347] Updated weights for policy 0, policy_version 1753927 (0.0009) [2023-12-27 17:47:48,465][20347] Updated weights for policy 0, policy_version 1753937 (0.0008) [2023-12-27 17:47:48,517][20347] Updated weights for policy 0, policy_version 1753947 (0.0008) [2023-12-27 17:47:48,963][20381] Updated weights for policy 1, policy_version 1752486 (0.0011) [2023-12-27 17:47:49,015][20381] Updated weights for policy 1, policy_version 1752496 (0.0011) [2023-12-27 17:47:49,063][20381] Updated weights for policy 1, policy_version 1752506 (0.0010) [2023-12-27 17:47:49,187][20347] Updated weights for policy 0, policy_version 1753957 (0.0006) [2023-12-27 17:47:49,251][20347] Updated weights for policy 0, policy_version 1753967 (0.0010) [2023-12-27 17:47:49,257][20291] KL-divergence is very high: 416.0125 [2023-12-27 17:47:49,306][20291] KL-divergence is very high: 540.3860 [2023-12-27 17:47:49,312][20347] Updated weights for policy 0, policy_version 1753977 (0.0010) [2023-12-27 17:47:49,746][20381] Updated weights for policy 1, policy_version 1752516 (0.0009) [2023-12-27 17:47:49,797][20381] Updated weights for policy 1, policy_version 1752526 (0.0007) [2023-12-27 17:47:49,860][20381] Updated weights for policy 1, policy_version 1752536 (0.0009) [2023-12-27 17:47:50,017][20347] Updated weights for policy 0, policy_version 1753987 (0.0009) [2023-12-27 17:47:50,076][20347] Updated weights for policy 0, policy_version 1753997 (0.0008) [2023-12-27 17:47:50,137][20347] Updated weights for policy 0, policy_version 1754007 (0.0008) [2023-12-27 17:47:50,588][20381] Updated weights for policy 1, policy_version 1752546 (0.0008) [2023-12-27 17:47:50,647][20381] Updated weights for policy 1, policy_version 1752556 (0.0008) [2023-12-27 17:47:50,706][20381] Updated weights for policy 1, policy_version 1752566 (0.0008) [2023-12-27 17:47:50,763][20381] Updated weights for policy 1, policy_version 1752576 (0.0006) [2023-12-27 17:47:50,857][20347] Updated weights for policy 0, policy_version 1754017 (0.0009) [2023-12-27 17:47:50,922][20347] Updated weights for policy 0, policy_version 1754027 (0.0010) [2023-12-27 17:47:50,984][20347] Updated weights for policy 0, policy_version 1754037 (0.0010) [2023-12-27 17:47:51,060][20347] Updated weights for policy 0, policy_version 1754047 (0.0009) [2023-12-27 17:47:51,396][20381] Updated weights for policy 1, policy_version 1752586 (0.0007) [2023-12-27 17:47:51,450][20381] Updated weights for policy 1, policy_version 1752596 (0.0006) [2023-12-27 17:47:51,509][20381] Updated weights for policy 1, policy_version 1752606 (0.0005) [2023-12-27 17:47:51,822][20347] Updated weights for policy 0, policy_version 1754057 (0.0009) [2023-12-27 17:47:51,884][20347] Updated weights for policy 0, policy_version 1754067 (0.0009) [2023-12-27 17:47:51,943][20347] Updated weights for policy 0, policy_version 1754077 (0.0010) [2023-12-27 17:47:52,171][20381] Updated weights for policy 1, policy_version 1752616 (0.0007) [2023-12-27 17:47:52,229][20381] Updated weights for policy 1, policy_version 1752626 (0.0007) [2023-12-27 17:47:52,288][20381] Updated weights for policy 1, policy_version 1752636 (0.0007) [2023-12-27 17:47:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 897851392. Throughput: 0: 10314.5, 1: 10290.0. Samples: 897818488. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:52,363][19170] Avg episode reward: [(0, '931.500'), (1, '1000.000')] [2023-12-27 17:47:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001752640_448724992.pth... [2023-12-27 17:47:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001754080_449126400.pth... [2023-12-27 17:47:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001751424_448413696.pth [2023-12-27 17:47:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001752864_448815104.pth [2023-12-27 17:47:52,685][20347] Updated weights for policy 0, policy_version 1754087 (0.0009) [2023-12-27 17:47:52,742][20347] Updated weights for policy 0, policy_version 1754097 (0.0008) [2023-12-27 17:47:52,801][20347] Updated weights for policy 0, policy_version 1754107 (0.0008) [2023-12-27 17:47:52,988][20381] Updated weights for policy 1, policy_version 1752646 (0.0008) [2023-12-27 17:47:53,046][20381] Updated weights for policy 1, policy_version 1752656 (0.0007) [2023-12-27 17:47:53,102][20381] Updated weights for policy 1, policy_version 1752666 (0.0008) [2023-12-27 17:47:53,512][20347] Updated weights for policy 0, policy_version 1754117 (0.0007) [2023-12-27 17:47:53,558][20347] Updated weights for policy 0, policy_version 1754127 (0.0005) [2023-12-27 17:47:53,616][20347] Updated weights for policy 0, policy_version 1754137 (0.0005) [2023-12-27 17:47:53,753][20381] Updated weights for policy 1, policy_version 1752676 (0.0008) [2023-12-27 17:47:53,813][20381] Updated weights for policy 1, policy_version 1752686 (0.0010) [2023-12-27 17:47:53,867][20381] Updated weights for policy 1, policy_version 1752696 (0.0010) [2023-12-27 17:47:54,281][20347] Updated weights for policy 0, policy_version 1754147 (0.0008) [2023-12-27 17:47:54,340][20347] Updated weights for policy 0, policy_version 1754157 (0.0010) [2023-12-27 17:47:54,397][20347] Updated weights for policy 0, policy_version 1754167 (0.0010) [2023-12-27 17:47:54,537][20381] Updated weights for policy 1, policy_version 1752706 (0.0010) [2023-12-27 17:47:54,592][20381] Updated weights for policy 1, policy_version 1752716 (0.0010) [2023-12-27 17:47:54,644][20381] Updated weights for policy 1, policy_version 1752726 (0.0010) [2023-12-27 17:47:54,698][20381] Updated weights for policy 1, policy_version 1752736 (0.0011) [2023-12-27 17:47:54,993][20347] Updated weights for policy 0, policy_version 1754177 (0.0006) [2023-12-27 17:47:55,046][20347] Updated weights for policy 0, policy_version 1754187 (0.0005) [2023-12-27 17:47:55,104][20347] Updated weights for policy 0, policy_version 1754197 (0.0008) [2023-12-27 17:47:55,159][20347] Updated weights for policy 0, policy_version 1754207 (0.0010) [2023-12-27 17:47:55,362][20381] Updated weights for policy 1, policy_version 1752746 (0.0010) [2023-12-27 17:47:55,425][20381] Updated weights for policy 1, policy_version 1752756 (0.0011) [2023-12-27 17:47:55,491][20381] Updated weights for policy 1, policy_version 1752766 (0.0011) [2023-12-27 17:47:55,790][20347] Updated weights for policy 0, policy_version 1754217 (0.0007) [2023-12-27 17:47:55,846][20347] Updated weights for policy 0, policy_version 1754227 (0.0010) [2023-12-27 17:47:55,905][20347] Updated weights for policy 0, policy_version 1754237 (0.0010) [2023-12-27 17:47:56,073][20381] Updated weights for policy 1, policy_version 1752776 (0.0006) [2023-12-27 17:47:56,123][20381] Updated weights for policy 1, policy_version 1752786 (0.0007) [2023-12-27 17:47:56,169][20381] Updated weights for policy 1, policy_version 1752796 (0.0008) [2023-12-27 17:47:56,532][20347] Updated weights for policy 0, policy_version 1754247 (0.0007) [2023-12-27 17:47:56,585][20347] Updated weights for policy 0, policy_version 1754257 (0.0006) [2023-12-27 17:47:56,661][20347] Updated weights for policy 0, policy_version 1754267 (0.0008) [2023-12-27 17:47:56,759][20381] Updated weights for policy 1, policy_version 1752806 (0.0006) [2023-12-27 17:47:56,815][20381] Updated weights for policy 1, policy_version 1752816 (0.0005) [2023-12-27 17:47:56,869][20381] Updated weights for policy 1, policy_version 1752826 (0.0008) [2023-12-27 17:47:57,250][20347] Updated weights for policy 0, policy_version 1754277 (0.0006) [2023-12-27 17:47:57,306][20347] Updated weights for policy 0, policy_version 1754287 (0.0005) [2023-12-27 17:47:57,362][20347] Updated weights for policy 0, policy_version 1754297 (0.0006) [2023-12-27 17:47:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 897949696. Throughput: 0: 10339.7, 1: 10257.3. Samples: 897941884. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:47:57,363][19170] Avg episode reward: [(0, '960.870'), (1, '990.240')] [2023-12-27 17:47:57,509][20381] Updated weights for policy 1, policy_version 1752836 (0.0006) [2023-12-27 17:47:57,559][20381] Updated weights for policy 1, policy_version 1752846 (0.0005) [2023-12-27 17:47:57,612][20381] Updated weights for policy 1, policy_version 1752856 (0.0006) [2023-12-27 17:47:57,993][20347] Updated weights for policy 0, policy_version 1754307 (0.0006) [2023-12-27 17:47:58,044][20347] Updated weights for policy 0, policy_version 1754317 (0.0008) [2023-12-27 17:47:58,105][20347] Updated weights for policy 0, policy_version 1754327 (0.0007) [2023-12-27 17:47:58,202][20381] Updated weights for policy 1, policy_version 1752866 (0.0008) [2023-12-27 17:47:58,264][20381] Updated weights for policy 1, policy_version 1752876 (0.0008) [2023-12-27 17:47:58,325][20381] Updated weights for policy 1, policy_version 1752886 (0.0007) [2023-12-27 17:47:58,386][20381] Updated weights for policy 1, policy_version 1752896 (0.0009) [2023-12-27 17:47:58,891][20347] Updated weights for policy 0, policy_version 1754337 (0.0006) [2023-12-27 17:47:58,944][20347] Updated weights for policy 0, policy_version 1754347 (0.0008) [2023-12-27 17:47:59,006][20347] Updated weights for policy 0, policy_version 1754357 (0.0007) [2023-12-27 17:47:59,063][20347] Updated weights for policy 0, policy_version 1754367 (0.0008) [2023-12-27 17:47:59,156][20381] Updated weights for policy 1, policy_version 1752906 (0.0008) [2023-12-27 17:47:59,212][20381] Updated weights for policy 1, policy_version 1752916 (0.0008) [2023-12-27 17:47:59,277][20381] Updated weights for policy 1, policy_version 1752926 (0.0009) [2023-12-27 17:47:59,788][20347] Updated weights for policy 0, policy_version 1754377 (0.0007) [2023-12-27 17:47:59,847][20347] Updated weights for policy 0, policy_version 1754387 (0.0007) [2023-12-27 17:47:59,906][20347] Updated weights for policy 0, policy_version 1754397 (0.0006) [2023-12-27 17:48:00,015][20381] Updated weights for policy 1, policy_version 1752936 (0.0008) [2023-12-27 17:48:00,066][20381] Updated weights for policy 1, policy_version 1752946 (0.0007) [2023-12-27 17:48:00,119][20381] Updated weights for policy 1, policy_version 1752956 (0.0008) [2023-12-27 17:48:00,558][20347] Updated weights for policy 0, policy_version 1754407 (0.0008) [2023-12-27 17:48:00,610][20347] Updated weights for policy 0, policy_version 1754417 (0.0008) [2023-12-27 17:48:00,648][20291] KL-divergence is very high: 472.1234 [2023-12-27 17:48:00,662][20347] Updated weights for policy 0, policy_version 1754427 (0.0008) [2023-12-27 17:48:00,820][20381] Updated weights for policy 1, policy_version 1752966 (0.0008) [2023-12-27 17:48:00,861][20381] Updated weights for policy 1, policy_version 1752976 (0.0006) [2023-12-27 17:48:00,904][20381] Updated weights for policy 1, policy_version 1752986 (0.0005) [2023-12-27 17:48:01,429][20347] Updated weights for policy 0, policy_version 1754437 (0.0007) [2023-12-27 17:48:01,479][20347] Updated weights for policy 0, policy_version 1754447 (0.0007) [2023-12-27 17:48:01,532][20347] Updated weights for policy 0, policy_version 1754457 (0.0006) [2023-12-27 17:48:01,617][20381] Updated weights for policy 1, policy_version 1752996 (0.0007) [2023-12-27 17:48:01,680][20381] Updated weights for policy 1, policy_version 1753006 (0.0011) [2023-12-27 17:48:01,757][20381] Updated weights for policy 1, policy_version 1753018 (0.0008) [2023-12-27 17:48:02,194][20347] Updated weights for policy 0, policy_version 1754467 (0.0007) [2023-12-27 17:48:02,253][20347] Updated weights for policy 0, policy_version 1754477 (0.0007) [2023-12-27 17:48:02,311][20347] Updated weights for policy 0, policy_version 1754487 (0.0008) [2023-12-27 17:48:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 898056192. Throughput: 0: 10341.9, 1: 10280.4. Samples: 898064224. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:02,363][19170] Avg episode reward: [(0, '941.300'), (1, '990.240')] [2023-12-27 17:48:02,485][20381] Updated weights for policy 1, policy_version 1753028 (0.0008) [2023-12-27 17:48:02,535][20381] Updated weights for policy 1, policy_version 1753038 (0.0006) [2023-12-27 17:48:02,591][20381] Updated weights for policy 1, policy_version 1753048 (0.0007) [2023-12-27 17:48:03,014][20347] Updated weights for policy 0, policy_version 1754497 (0.0008) [2023-12-27 17:48:03,071][20347] Updated weights for policy 0, policy_version 1754507 (0.0010) [2023-12-27 17:48:03,124][20347] Updated weights for policy 0, policy_version 1754517 (0.0010) [2023-12-27 17:48:03,169][20347] Updated weights for policy 0, policy_version 1754527 (0.0010) [2023-12-27 17:48:03,260][20381] Updated weights for policy 1, policy_version 1753058 (0.0007) [2023-12-27 17:48:03,317][20381] Updated weights for policy 1, policy_version 1753068 (0.0008) [2023-12-27 17:48:03,380][20381] Updated weights for policy 1, policy_version 1753078 (0.0008) [2023-12-27 17:48:03,434][20381] Updated weights for policy 1, policy_version 1753088 (0.0007) [2023-12-27 17:48:03,741][20347] Updated weights for policy 0, policy_version 1754537 (0.0010) [2023-12-27 17:48:03,789][20347] Updated weights for policy 0, policy_version 1754547 (0.0010) [2023-12-27 17:48:03,840][20347] Updated weights for policy 0, policy_version 1754557 (0.0010) [2023-12-27 17:48:04,140][20381] Updated weights for policy 1, policy_version 1753098 (0.0007) [2023-12-27 17:48:04,192][20381] Updated weights for policy 1, policy_version 1753108 (0.0007) [2023-12-27 17:48:04,249][20381] Updated weights for policy 1, policy_version 1753118 (0.0007) [2023-12-27 17:48:04,557][20347] Updated weights for policy 0, policy_version 1754567 (0.0008) [2023-12-27 17:48:04,620][20347] Updated weights for policy 0, policy_version 1754577 (0.0009) [2023-12-27 17:48:04,675][20347] Updated weights for policy 0, policy_version 1754587 (0.0010) [2023-12-27 17:48:04,906][20381] Updated weights for policy 1, policy_version 1753128 (0.0008) [2023-12-27 17:48:04,965][20381] Updated weights for policy 1, policy_version 1753138 (0.0008) [2023-12-27 17:48:05,025][20381] Updated weights for policy 1, policy_version 1753148 (0.0006) [2023-12-27 17:48:05,341][20347] Updated weights for policy 0, policy_version 1754597 (0.0010) [2023-12-27 17:48:05,386][20347] Updated weights for policy 0, policy_version 1754607 (0.0010) [2023-12-27 17:48:05,434][20347] Updated weights for policy 0, policy_version 1754617 (0.0010) [2023-12-27 17:48:05,477][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 17:48:05,616][20381] Updated weights for policy 1, policy_version 1753158 (0.0008) [2023-12-27 17:48:05,670][20381] Updated weights for policy 1, policy_version 1753168 (0.0009) [2023-12-27 17:48:05,728][20381] Updated weights for policy 1, policy_version 1753178 (0.0008) [2023-12-27 17:48:06,085][20347] Updated weights for policy 0, policy_version 1754627 (0.0010) [2023-12-27 17:48:06,153][20347] Updated weights for policy 0, policy_version 1754637 (0.0010) [2023-12-27 17:48:06,203][20347] Updated weights for policy 0, policy_version 1754647 (0.0008) [2023-12-27 17:48:06,494][20381] Updated weights for policy 1, policy_version 1753188 (0.0008) [2023-12-27 17:48:06,557][20381] Updated weights for policy 1, policy_version 1753198 (0.0008) [2023-12-27 17:48:06,607][20381] Updated weights for policy 1, policy_version 1753208 (0.0008) [2023-12-27 17:48:06,893][20347] Updated weights for policy 0, policy_version 1754657 (0.0008) [2023-12-27 17:48:06,957][20347] Updated weights for policy 0, policy_version 1754667 (0.0008) [2023-12-27 17:48:07,022][20347] Updated weights for policy 0, policy_version 1754677 (0.0011) [2023-12-27 17:48:07,084][20347] Updated weights for policy 0, policy_version 1754687 (0.0010) [2023-12-27 17:48:07,303][20381] Updated weights for policy 1, policy_version 1753218 (0.0007) [2023-12-27 17:48:07,350][20381] Updated weights for policy 1, policy_version 1753228 (0.0007) [2023-12-27 17:48:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 898154496. Throughput: 0: 10370.4, 1: 10270.2. Samples: 898126580. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:07,363][19170] Avg episode reward: [(0, '941.450'), (1, '1000.000')] [2023-12-27 17:48:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001754688_449282048.pth... [2023-12-27 17:48:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001753504_448978944.pth [2023-12-27 17:48:07,405][20381] Updated weights for policy 1, policy_version 1753238 (0.0007) [2023-12-27 17:48:07,459][20381] Updated weights for policy 1, policy_version 1753248 (0.0008) [2023-12-27 17:48:07,460][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001753248_448880640.pth... [2023-12-27 17:48:07,463][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001752032_448569344.pth [2023-12-27 17:48:07,743][20347] Updated weights for policy 0, policy_version 1754697 (0.0010) [2023-12-27 17:48:07,804][20347] Updated weights for policy 0, policy_version 1754707 (0.0010) [2023-12-27 17:48:07,864][20347] Updated weights for policy 0, policy_version 1754717 (0.0010) [2023-12-27 17:48:08,168][20381] Updated weights for policy 1, policy_version 1753258 (0.0007) [2023-12-27 17:48:08,224][20381] Updated weights for policy 1, policy_version 1753268 (0.0008) [2023-12-27 17:48:08,282][20381] Updated weights for policy 1, policy_version 1753278 (0.0008) [2023-12-27 17:48:08,518][20347] Updated weights for policy 0, policy_version 1754727 (0.0010) [2023-12-27 17:48:08,575][20347] Updated weights for policy 0, policy_version 1754737 (0.0010) [2023-12-27 17:48:08,628][20347] Updated weights for policy 0, policy_version 1754747 (0.0009) [2023-12-27 17:48:08,949][20381] Updated weights for policy 1, policy_version 1753288 (0.0010) [2023-12-27 17:48:08,994][20381] Updated weights for policy 1, policy_version 1753298 (0.0010) [2023-12-27 17:48:09,056][20381] Updated weights for policy 1, policy_version 1753308 (0.0010) [2023-12-27 17:48:09,317][20347] Updated weights for policy 0, policy_version 1754757 (0.0008) [2023-12-27 17:48:09,383][20347] Updated weights for policy 0, policy_version 1754767 (0.0011) [2023-12-27 17:48:09,446][20347] Updated weights for policy 0, policy_version 1754777 (0.0011) [2023-12-27 17:48:09,818][20381] Updated weights for policy 1, policy_version 1753318 (0.0009) [2023-12-27 17:48:09,876][20381] Updated weights for policy 1, policy_version 1753328 (0.0007) [2023-12-27 17:48:09,938][20381] Updated weights for policy 1, policy_version 1753338 (0.0007) [2023-12-27 17:48:10,212][20347] Updated weights for policy 0, policy_version 1754787 (0.0008) [2023-12-27 17:48:10,269][20347] Updated weights for policy 0, policy_version 1754797 (0.0008) [2023-12-27 17:48:10,326][20347] Updated weights for policy 0, policy_version 1754807 (0.0008) [2023-12-27 17:48:10,666][20381] Updated weights for policy 1, policy_version 1753348 (0.0007) [2023-12-27 17:48:10,723][20381] Updated weights for policy 1, policy_version 1753358 (0.0008) [2023-12-27 17:48:10,776][20381] Updated weights for policy 1, policy_version 1753368 (0.0008) [2023-12-27 17:48:11,043][20347] Updated weights for policy 0, policy_version 1754817 (0.0008) [2023-12-27 17:48:11,109][20347] Updated weights for policy 0, policy_version 1754827 (0.0006) [2023-12-27 17:48:11,176][20347] Updated weights for policy 0, policy_version 1754837 (0.0007) [2023-12-27 17:48:11,247][20347] Updated weights for policy 0, policy_version 1754847 (0.0008) [2023-12-27 17:48:11,527][20381] Updated weights for policy 1, policy_version 1753378 (0.0006) [2023-12-27 17:48:11,590][20381] Updated weights for policy 1, policy_version 1753388 (0.0006) [2023-12-27 17:48:11,657][20381] Updated weights for policy 1, policy_version 1753398 (0.0007) [2023-12-27 17:48:11,725][20381] Updated weights for policy 1, policy_version 1753408 (0.0008) [2023-12-27 17:48:11,988][20347] Updated weights for policy 0, policy_version 1754857 (0.0008) [2023-12-27 17:48:12,051][20347] Updated weights for policy 0, policy_version 1754867 (0.0008) [2023-12-27 17:48:12,110][20347] Updated weights for policy 0, policy_version 1754877 (0.0007) [2023-12-27 17:48:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 898252800. Throughput: 0: 10341.8, 1: 10205.8. Samples: 898245548. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:12,363][19170] Avg episode reward: [(0, '961.020'), (1, '1000.000')] [2023-12-27 17:48:12,401][20381] Updated weights for policy 1, policy_version 1753418 (0.0008) [2023-12-27 17:48:12,461][20381] Updated weights for policy 1, policy_version 1753428 (0.0008) [2023-12-27 17:48:12,527][20381] Updated weights for policy 1, policy_version 1753438 (0.0006) [2023-12-27 17:48:12,840][20347] Updated weights for policy 0, policy_version 1754887 (0.0008) [2023-12-27 17:48:12,893][20347] Updated weights for policy 0, policy_version 1754897 (0.0008) [2023-12-27 17:48:12,949][20347] Updated weights for policy 0, policy_version 1754907 (0.0008) [2023-12-27 17:48:13,222][20381] Updated weights for policy 1, policy_version 1753448 (0.0007) [2023-12-27 17:48:13,281][20381] Updated weights for policy 1, policy_version 1753458 (0.0008) [2023-12-27 17:48:13,336][20381] Updated weights for policy 1, policy_version 1753468 (0.0007) [2023-12-27 17:48:13,665][20347] Updated weights for policy 0, policy_version 1754917 (0.0008) [2023-12-27 17:48:13,722][20347] Updated weights for policy 0, policy_version 1754927 (0.0009) [2023-12-27 17:48:13,774][20347] Updated weights for policy 0, policy_version 1754937 (0.0007) [2023-12-27 17:48:13,916][20381] Updated weights for policy 1, policy_version 1753478 (0.0006) [2023-12-27 17:48:13,977][20381] Updated weights for policy 1, policy_version 1753488 (0.0005) [2023-12-27 17:48:14,037][20381] Updated weights for policy 1, policy_version 1753498 (0.0005) [2023-12-27 17:48:14,399][20347] Updated weights for policy 0, policy_version 1754947 (0.0007) [2023-12-27 17:48:14,457][20347] Updated weights for policy 0, policy_version 1754957 (0.0010) [2023-12-27 17:48:14,508][20347] Updated weights for policy 0, policy_version 1754967 (0.0007) [2023-12-27 17:48:14,671][20381] Updated weights for policy 1, policy_version 1753508 (0.0006) [2023-12-27 17:48:14,726][20381] Updated weights for policy 1, policy_version 1753518 (0.0010) [2023-12-27 17:48:14,788][20381] Updated weights for policy 1, policy_version 1753528 (0.0009) [2023-12-27 17:48:15,177][20347] Updated weights for policy 0, policy_version 1754977 (0.0006) [2023-12-27 17:48:15,238][20347] Updated weights for policy 0, policy_version 1754987 (0.0010) [2023-12-27 17:48:15,301][20347] Updated weights for policy 0, policy_version 1754997 (0.0008) [2023-12-27 17:48:15,363][20347] Updated weights for policy 0, policy_version 1755007 (0.0008) [2023-12-27 17:48:15,498][20381] Updated weights for policy 1, policy_version 1753538 (0.0010) [2023-12-27 17:48:15,552][20381] Updated weights for policy 1, policy_version 1753548 (0.0010) [2023-12-27 17:48:15,605][20381] Updated weights for policy 1, policy_version 1753558 (0.0008) [2023-12-27 17:48:15,657][20381] Updated weights for policy 1, policy_version 1753568 (0.0010) [2023-12-27 17:48:15,935][20347] Updated weights for policy 0, policy_version 1755017 (0.0010) [2023-12-27 17:48:15,996][20347] Updated weights for policy 0, policy_version 1755027 (0.0007) [2023-12-27 17:48:16,051][20347] Updated weights for policy 0, policy_version 1755037 (0.0009) [2023-12-27 17:48:16,334][20381] Updated weights for policy 1, policy_version 1753578 (0.0011) [2023-12-27 17:48:16,386][20381] Updated weights for policy 1, policy_version 1753588 (0.0010) [2023-12-27 17:48:16,434][20381] Updated weights for policy 1, policy_version 1753598 (0.0011) [2023-12-27 17:48:16,619][20347] Updated weights for policy 0, policy_version 1755047 (0.0006) [2023-12-27 17:48:16,673][20347] Updated weights for policy 0, policy_version 1755057 (0.0005) [2023-12-27 17:48:16,721][20347] Updated weights for policy 0, policy_version 1755067 (0.0005) [2023-12-27 17:48:17,148][20381] Updated weights for policy 1, policy_version 1753608 (0.0011) [2023-12-27 17:48:17,197][20381] Updated weights for policy 1, policy_version 1753618 (0.0011) [2023-12-27 17:48:17,251][20381] Updated weights for policy 1, policy_version 1753628 (0.0010) [2023-12-27 17:48:17,309][20347] Updated weights for policy 0, policy_version 1755077 (0.0009) [2023-12-27 17:48:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 898359296. Throughput: 0: 10234.5, 1: 10242.7. Samples: 898369460. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:17,363][19170] Avg episode reward: [(0, '980.440'), (1, '1000.000')] [2023-12-27 17:48:17,364][20347] Updated weights for policy 0, policy_version 1755087 (0.0010) [2023-12-27 17:48:17,410][20347] Updated weights for policy 0, policy_version 1755097 (0.0010) [2023-12-27 17:48:17,886][20381] Updated weights for policy 1, policy_version 1753638 (0.0008) [2023-12-27 17:48:17,934][20381] Updated weights for policy 1, policy_version 1753648 (0.0006) [2023-12-27 17:48:17,979][20381] Updated weights for policy 1, policy_version 1753658 (0.0005) [2023-12-27 17:48:18,099][20347] Updated weights for policy 0, policy_version 1755107 (0.0010) [2023-12-27 17:48:18,149][20347] Updated weights for policy 0, policy_version 1755117 (0.0008) [2023-12-27 17:48:18,202][20347] Updated weights for policy 0, policy_version 1755127 (0.0005) [2023-12-27 17:48:18,647][20381] Updated weights for policy 1, policy_version 1753668 (0.0007) [2023-12-27 17:48:18,695][20381] Updated weights for policy 1, policy_version 1753678 (0.0005) [2023-12-27 17:48:18,749][20381] Updated weights for policy 1, policy_version 1753688 (0.0006) [2023-12-27 17:48:18,813][20347] Updated weights for policy 0, policy_version 1755137 (0.0005) [2023-12-27 17:48:18,871][20347] Updated weights for policy 0, policy_version 1755147 (0.0006) [2023-12-27 17:48:18,930][20347] Updated weights for policy 0, policy_version 1755157 (0.0010) [2023-12-27 17:48:18,979][20347] Updated weights for policy 0, policy_version 1755167 (0.0010) [2023-12-27 17:48:19,467][20381] Updated weights for policy 1, policy_version 1753698 (0.0008) [2023-12-27 17:48:19,534][20381] Updated weights for policy 1, policy_version 1753708 (0.0008) [2023-12-27 17:48:19,602][20381] Updated weights for policy 1, policy_version 1753718 (0.0008) [2023-12-27 17:48:19,670][20381] Updated weights for policy 1, policy_version 1753728 (0.0008) [2023-12-27 17:48:19,686][20347] Updated weights for policy 0, policy_version 1755177 (0.0008) [2023-12-27 17:48:19,734][20347] Updated weights for policy 0, policy_version 1755187 (0.0008) [2023-12-27 17:48:19,789][20347] Updated weights for policy 0, policy_version 1755197 (0.0008) [2023-12-27 17:48:20,316][20381] Updated weights for policy 1, policy_version 1753738 (0.0008) [2023-12-27 17:48:20,377][20381] Updated weights for policy 1, policy_version 1753748 (0.0009) [2023-12-27 17:48:20,424][20381] Updated weights for policy 1, policy_version 1753758 (0.0005) [2023-12-27 17:48:20,565][20347] Updated weights for policy 0, policy_version 1755207 (0.0008) [2023-12-27 17:48:20,621][20347] Updated weights for policy 0, policy_version 1755217 (0.0007) [2023-12-27 17:48:20,681][20347] Updated weights for policy 0, policy_version 1755227 (0.0008) [2023-12-27 17:48:21,091][20381] Updated weights for policy 1, policy_version 1753768 (0.0007) [2023-12-27 17:48:21,160][20381] Updated weights for policy 1, policy_version 1753778 (0.0009) [2023-12-27 17:48:21,215][20381] Updated weights for policy 1, policy_version 1753788 (0.0007) [2023-12-27 17:48:21,429][20347] Updated weights for policy 0, policy_version 1755237 (0.0007) [2023-12-27 17:48:21,489][20347] Updated weights for policy 0, policy_version 1755247 (0.0005) [2023-12-27 17:48:21,549][20347] Updated weights for policy 0, policy_version 1755257 (0.0006) [2023-12-27 17:48:21,958][20381] Updated weights for policy 1, policy_version 1753798 (0.0010) [2023-12-27 17:48:22,017][20381] Updated weights for policy 1, policy_version 1753808 (0.0007) [2023-12-27 17:48:22,073][20381] Updated weights for policy 1, policy_version 1753818 (0.0007) [2023-12-27 17:48:22,280][20347] Updated weights for policy 0, policy_version 1755267 (0.0006) [2023-12-27 17:48:22,340][20347] Updated weights for policy 0, policy_version 1755277 (0.0007) [2023-12-27 17:48:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 898457600. Throughput: 0: 10206.3, 1: 10236.5. Samples: 898431868. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:22,363][20291] KL-divergence is very high: 111.9584 [2023-12-27 17:48:22,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 17:48:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001753824_449028096.pth... [2023-12-27 17:48:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001752640_448724992.pth [2023-12-27 17:48:22,404][20347] Updated weights for policy 0, policy_version 1755287 (0.0007) [2023-12-27 17:48:22,454][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001755296_449437696.pth... [2023-12-27 17:48:22,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001754080_449126400.pth [2023-12-27 17:48:22,757][20381] Updated weights for policy 1, policy_version 1753828 (0.0009) [2023-12-27 17:48:22,820][20381] Updated weights for policy 1, policy_version 1753838 (0.0011) [2023-12-27 17:48:22,883][20381] Updated weights for policy 1, policy_version 1753848 (0.0010) [2023-12-27 17:48:23,063][20347] Updated weights for policy 0, policy_version 1755297 (0.0006) [2023-12-27 17:48:23,125][20347] Updated weights for policy 0, policy_version 1755307 (0.0007) [2023-12-27 17:48:23,183][20347] Updated weights for policy 0, policy_version 1755317 (0.0008) [2023-12-27 17:48:23,234][20347] Updated weights for policy 0, policy_version 1755327 (0.0007) [2023-12-27 17:48:23,544][20381] Updated weights for policy 1, policy_version 1753858 (0.0010) [2023-12-27 17:48:23,604][20381] Updated weights for policy 1, policy_version 1753868 (0.0005) [2023-12-27 17:48:23,664][20381] Updated weights for policy 1, policy_version 1753878 (0.0005) [2023-12-27 17:48:23,718][20381] Updated weights for policy 1, policy_version 1753888 (0.0005) [2023-12-27 17:48:23,922][20347] Updated weights for policy 0, policy_version 1755337 (0.0006) [2023-12-27 17:48:23,977][20347] Updated weights for policy 0, policy_version 1755347 (0.0007) [2023-12-27 17:48:24,038][20347] Updated weights for policy 0, policy_version 1755357 (0.0007) [2023-12-27 17:48:24,250][20381] Updated weights for policy 1, policy_version 1753898 (0.0005) [2023-12-27 17:48:24,309][20381] Updated weights for policy 1, policy_version 1753908 (0.0006) [2023-12-27 17:48:24,358][20381] Updated weights for policy 1, policy_version 1753918 (0.0005) [2023-12-27 17:48:24,650][20347] Updated weights for policy 0, policy_version 1755367 (0.0008) [2023-12-27 17:48:24,705][20347] Updated weights for policy 0, policy_version 1755377 (0.0010) [2023-12-27 17:48:24,757][20347] Updated weights for policy 0, policy_version 1755387 (0.0010) [2023-12-27 17:48:24,968][20381] Updated weights for policy 1, policy_version 1753928 (0.0007) [2023-12-27 17:48:25,023][20381] Updated weights for policy 1, policy_version 1753938 (0.0006) [2023-12-27 17:48:25,086][20381] Updated weights for policy 1, policy_version 1753948 (0.0005) [2023-12-27 17:48:25,371][20347] Updated weights for policy 0, policy_version 1755397 (0.0006) [2023-12-27 17:48:25,419][20347] Updated weights for policy 0, policy_version 1755407 (0.0005) [2023-12-27 17:48:25,465][20347] Updated weights for policy 0, policy_version 1755417 (0.0005) [2023-12-27 17:48:25,728][20381] Updated weights for policy 1, policy_version 1753958 (0.0007) [2023-12-27 17:48:25,784][20381] Updated weights for policy 1, policy_version 1753968 (0.0008) [2023-12-27 17:48:25,846][20381] Updated weights for policy 1, policy_version 1753978 (0.0008) [2023-12-27 17:48:26,100][20347] Updated weights for policy 0, policy_version 1755427 (0.0007) [2023-12-27 17:48:26,161][20347] Updated weights for policy 0, policy_version 1755437 (0.0008) [2023-12-27 17:48:26,219][20347] Updated weights for policy 0, policy_version 1755447 (0.0008) [2023-12-27 17:48:26,504][20381] Updated weights for policy 1, policy_version 1753988 (0.0007) [2023-12-27 17:48:26,559][20381] Updated weights for policy 1, policy_version 1753998 (0.0008) [2023-12-27 17:48:26,604][20381] Updated weights for policy 1, policy_version 1754008 (0.0007) [2023-12-27 17:48:26,848][20347] Updated weights for policy 0, policy_version 1755457 (0.0007) [2023-12-27 17:48:26,914][20347] Updated weights for policy 0, policy_version 1755467 (0.0005) [2023-12-27 17:48:26,968][20347] Updated weights for policy 0, policy_version 1755477 (0.0005) [2023-12-27 17:48:27,026][20347] Updated weights for policy 0, policy_version 1755487 (0.0008) [2023-12-27 17:48:27,324][20381] Updated weights for policy 1, policy_version 1754018 (0.0006) [2023-12-27 17:48:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 898564096. Throughput: 0: 10249.1, 1: 10319.2. Samples: 898557244. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:27,363][19170] Avg episode reward: [(0, '960.850'), (1, '1000.000')] [2023-12-27 17:48:27,378][20381] Updated weights for policy 1, policy_version 1754028 (0.0005) [2023-12-27 17:48:27,430][20381] Updated weights for policy 1, policy_version 1754038 (0.0005) [2023-12-27 17:48:27,488][20381] Updated weights for policy 1, policy_version 1754048 (0.0006) [2023-12-27 17:48:27,675][20347] Updated weights for policy 0, policy_version 1755497 (0.0008) [2023-12-27 17:48:27,730][20347] Updated weights for policy 0, policy_version 1755507 (0.0007) [2023-12-27 17:48:27,790][20347] Updated weights for policy 0, policy_version 1755517 (0.0007) [2023-12-27 17:48:28,114][20381] Updated weights for policy 1, policy_version 1754058 (0.0008) [2023-12-27 17:48:28,167][20381] Updated weights for policy 1, policy_version 1754068 (0.0007) [2023-12-27 17:48:28,227][20381] Updated weights for policy 1, policy_version 1754078 (0.0005) [2023-12-27 17:48:28,440][20347] Updated weights for policy 0, policy_version 1755527 (0.0008) [2023-12-27 17:48:28,502][20347] Updated weights for policy 0, policy_version 1755537 (0.0008) [2023-12-27 17:48:28,566][20347] Updated weights for policy 0, policy_version 1755547 (0.0008) [2023-12-27 17:48:28,899][20381] Updated weights for policy 1, policy_version 1754088 (0.0005) [2023-12-27 17:48:28,946][20381] Updated weights for policy 1, policy_version 1754098 (0.0005) [2023-12-27 17:48:28,994][20381] Updated weights for policy 1, policy_version 1754108 (0.0005) [2023-12-27 17:48:29,226][20347] Updated weights for policy 0, policy_version 1755557 (0.0009) [2023-12-27 17:48:29,285][20347] Updated weights for policy 0, policy_version 1755567 (0.0008) [2023-12-27 17:48:29,346][20347] Updated weights for policy 0, policy_version 1755577 (0.0007) [2023-12-27 17:48:29,664][20381] Updated weights for policy 1, policy_version 1754118 (0.0007) [2023-12-27 17:48:29,727][20381] Updated weights for policy 1, policy_version 1754128 (0.0007) [2023-12-27 17:48:29,785][20381] Updated weights for policy 1, policy_version 1754138 (0.0007) [2023-12-27 17:48:30,025][20347] Updated weights for policy 0, policy_version 1755587 (0.0007) [2023-12-27 17:48:30,085][20347] Updated weights for policy 0, policy_version 1755597 (0.0006) [2023-12-27 17:48:30,140][20347] Updated weights for policy 0, policy_version 1755607 (0.0009) [2023-12-27 17:48:30,442][20381] Updated weights for policy 1, policy_version 1754148 (0.0007) [2023-12-27 17:48:30,493][20381] Updated weights for policy 1, policy_version 1754158 (0.0007) [2023-12-27 17:48:30,547][20381] Updated weights for policy 1, policy_version 1754168 (0.0007) [2023-12-27 17:48:30,843][20347] Updated weights for policy 0, policy_version 1755617 (0.0007) [2023-12-27 17:48:30,902][20347] Updated weights for policy 0, policy_version 1755627 (0.0005) [2023-12-27 17:48:30,959][20347] Updated weights for policy 0, policy_version 1755637 (0.0005) [2023-12-27 17:48:31,006][20347] Updated weights for policy 0, policy_version 1755647 (0.0005) [2023-12-27 17:48:31,192][20381] Updated weights for policy 1, policy_version 1754178 (0.0008) [2023-12-27 17:48:31,252][20381] Updated weights for policy 1, policy_version 1754188 (0.0011) [2023-12-27 17:48:31,315][20381] Updated weights for policy 1, policy_version 1754198 (0.0011) [2023-12-27 17:48:31,384][20381] Updated weights for policy 1, policy_version 1754208 (0.0010) [2023-12-27 17:48:31,706][20347] Updated weights for policy 0, policy_version 1755657 (0.0008) [2023-12-27 17:48:31,775][20347] Updated weights for policy 0, policy_version 1755667 (0.0007) [2023-12-27 17:48:31,834][20347] Updated weights for policy 0, policy_version 1755677 (0.0007) [2023-12-27 17:48:32,018][20381] Updated weights for policy 1, policy_version 1754218 (0.0011) [2023-12-27 17:48:32,075][20381] Updated weights for policy 1, policy_version 1754228 (0.0011) [2023-12-27 17:48:32,130][20381] Updated weights for policy 1, policy_version 1754238 (0.0011) [2023-12-27 17:48:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 898670592. Throughput: 0: 10242.8, 1: 10294.8. Samples: 898680812. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:32,363][19170] Avg episode reward: [(0, '980.420'), (1, '1000.000')] [2023-12-27 17:48:32,535][20347] Updated weights for policy 0, policy_version 1755687 (0.0006) [2023-12-27 17:48:32,588][20347] Updated weights for policy 0, policy_version 1755697 (0.0010) [2023-12-27 17:48:32,649][20347] Updated weights for policy 0, policy_version 1755707 (0.0009) [2023-12-27 17:48:32,866][20381] Updated weights for policy 1, policy_version 1754248 (0.0010) [2023-12-27 17:48:32,916][20381] Updated weights for policy 1, policy_version 1754258 (0.0010) [2023-12-27 17:48:32,964][20381] Updated weights for policy 1, policy_version 1754268 (0.0010) [2023-12-27 17:48:33,266][20347] Updated weights for policy 0, policy_version 1755717 (0.0008) [2023-12-27 17:48:33,313][20347] Updated weights for policy 0, policy_version 1755727 (0.0010) [2023-12-27 17:48:33,361][20347] Updated weights for policy 0, policy_version 1755737 (0.0010) [2023-12-27 17:48:33,563][20381] Updated weights for policy 1, policy_version 1754278 (0.0010) [2023-12-27 17:48:33,608][20381] Updated weights for policy 1, policy_version 1754288 (0.0010) [2023-12-27 17:48:33,666][20381] Updated weights for policy 1, policy_version 1754298 (0.0007) [2023-12-27 17:48:34,032][20347] Updated weights for policy 0, policy_version 1755747 (0.0009) [2023-12-27 17:48:34,079][20347] Updated weights for policy 0, policy_version 1755757 (0.0010) [2023-12-27 17:48:34,129][20347] Updated weights for policy 0, policy_version 1755767 (0.0010) [2023-12-27 17:48:34,271][20381] Updated weights for policy 1, policy_version 1754308 (0.0007) [2023-12-27 17:48:34,333][20381] Updated weights for policy 1, policy_version 1754318 (0.0006) [2023-12-27 17:48:34,392][20381] Updated weights for policy 1, policy_version 1754328 (0.0006) [2023-12-27 17:48:34,851][20347] Updated weights for policy 0, policy_version 1755777 (0.0010) [2023-12-27 17:48:34,907][20347] Updated weights for policy 0, policy_version 1755787 (0.0005) [2023-12-27 17:48:34,964][20347] Updated weights for policy 0, policy_version 1755797 (0.0005) [2023-12-27 17:48:35,022][20347] Updated weights for policy 0, policy_version 1755807 (0.0006) [2023-12-27 17:48:35,039][20381] Updated weights for policy 1, policy_version 1754338 (0.0007) [2023-12-27 17:48:35,091][20381] Updated weights for policy 1, policy_version 1754348 (0.0010) [2023-12-27 17:48:35,140][20381] Updated weights for policy 1, policy_version 1754358 (0.0011) [2023-12-27 17:48:35,185][20381] Updated weights for policy 1, policy_version 1754368 (0.0010) [2023-12-27 17:48:35,613][20347] Updated weights for policy 0, policy_version 1755817 (0.0005) [2023-12-27 17:48:35,668][20347] Updated weights for policy 0, policy_version 1755827 (0.0006) [2023-12-27 17:48:35,730][20347] Updated weights for policy 0, policy_version 1755837 (0.0007) [2023-12-27 17:48:35,908][20381] Updated weights for policy 1, policy_version 1754378 (0.0011) [2023-12-27 17:48:35,978][20381] Updated weights for policy 1, policy_version 1754388 (0.0008) [2023-12-27 17:48:36,043][20381] Updated weights for policy 1, policy_version 1754398 (0.0011) [2023-12-27 17:48:36,354][20347] Updated weights for policy 0, policy_version 1755847 (0.0008) [2023-12-27 17:48:36,417][20347] Updated weights for policy 0, policy_version 1755857 (0.0008) [2023-12-27 17:48:36,477][20347] Updated weights for policy 0, policy_version 1755867 (0.0008) [2023-12-27 17:48:36,705][20381] Updated weights for policy 1, policy_version 1754408 (0.0009) [2023-12-27 17:48:36,767][20381] Updated weights for policy 1, policy_version 1754418 (0.0008) [2023-12-27 17:48:36,821][20381] Updated weights for policy 1, policy_version 1754428 (0.0007) [2023-12-27 17:48:37,174][20347] Updated weights for policy 0, policy_version 1755877 (0.0009) [2023-12-27 17:48:37,233][20347] Updated weights for policy 0, policy_version 1755887 (0.0011) [2023-12-27 17:48:37,295][20347] Updated weights for policy 0, policy_version 1755897 (0.0009) [2023-12-27 17:48:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 898777088. Throughput: 0: 10242.9, 1: 10334.6. Samples: 898744472. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) [2023-12-27 17:48:37,363][19170] Avg episode reward: [(0, '990.340'), (1, '990.240')] [2023-12-27 17:48:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001754432_449183744.pth... [2023-12-27 17:48:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001755904_449593344.pth... [2023-12-27 17:48:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001754688_449282048.pth [2023-12-27 17:48:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001753248_448880640.pth [2023-12-27 17:48:37,432][20381] Updated weights for policy 1, policy_version 1754438 (0.0005) [2023-12-27 17:48:37,483][20381] Updated weights for policy 1, policy_version 1754448 (0.0005) [2023-12-27 17:48:37,529][20381] Updated weights for policy 1, policy_version 1754458 (0.0005) [2023-12-27 17:48:37,994][20347] Updated weights for policy 0, policy_version 1755907 (0.0008) [2023-12-27 17:48:38,053][20347] Updated weights for policy 0, policy_version 1755917 (0.0010) [2023-12-27 17:48:38,071][20381] Updated weights for policy 1, policy_version 1754468 (0.0006) [2023-12-27 17:48:38,109][20347] Updated weights for policy 0, policy_version 1755927 (0.0010) [2023-12-27 17:48:38,125][20381] Updated weights for policy 1, policy_version 1754478 (0.0010) [2023-12-27 17:48:38,176][20381] Updated weights for policy 1, policy_version 1754488 (0.0010) [2023-12-27 17:48:38,786][20347] Updated weights for policy 0, policy_version 1755937 (0.0010) [2023-12-27 17:48:38,847][20381] Updated weights for policy 1, policy_version 1754498 (0.0010) [2023-12-27 17:48:38,848][20347] Updated weights for policy 0, policy_version 1755947 (0.0010) [2023-12-27 17:48:38,906][20381] Updated weights for policy 1, policy_version 1754508 (0.0010) [2023-12-27 17:48:38,906][20347] Updated weights for policy 0, policy_version 1755957 (0.0010) [2023-12-27 17:48:38,964][20381] Updated weights for policy 1, policy_version 1754518 (0.0010) [2023-12-27 17:48:38,964][20347] Updated weights for policy 0, policy_version 1755967 (0.0010) [2023-12-27 17:48:39,012][20381] Updated weights for policy 1, policy_version 1754528 (0.0010) [2023-12-27 17:48:39,629][20347] Updated weights for policy 0, policy_version 1755977 (0.0007) [2023-12-27 17:48:39,692][20347] Updated weights for policy 0, policy_version 1755987 (0.0007) [2023-12-27 17:48:39,746][20347] Updated weights for policy 0, policy_version 1755997 (0.0006) [2023-12-27 17:48:39,748][20381] Updated weights for policy 1, policy_version 1754538 (0.0006) [2023-12-27 17:48:39,807][20381] Updated weights for policy 1, policy_version 1754548 (0.0006) [2023-12-27 17:48:39,874][20381] Updated weights for policy 1, policy_version 1754558 (0.0008) [2023-12-27 17:48:40,492][20347] Updated weights for policy 0, policy_version 1756007 (0.0009) [2023-12-27 17:48:40,513][20381] Updated weights for policy 1, policy_version 1754568 (0.0007) [2023-12-27 17:48:40,556][20347] Updated weights for policy 0, policy_version 1756017 (0.0011) [2023-12-27 17:48:40,572][20381] Updated weights for policy 1, policy_version 1754578 (0.0006) [2023-12-27 17:48:40,610][20347] Updated weights for policy 0, policy_version 1756027 (0.0007) [2023-12-27 17:48:40,633][20381] Updated weights for policy 1, policy_version 1754588 (0.0008) [2023-12-27 17:48:41,293][20347] Updated weights for policy 0, policy_version 1756037 (0.0006) [2023-12-27 17:48:41,354][20381] Updated weights for policy 1, policy_version 1754598 (0.0008) [2023-12-27 17:48:41,361][20347] Updated weights for policy 0, policy_version 1756047 (0.0007) [2023-12-27 17:48:41,422][20381] Updated weights for policy 1, policy_version 1754608 (0.0008) [2023-12-27 17:48:41,424][20347] Updated weights for policy 0, policy_version 1756057 (0.0007) [2023-12-27 17:48:41,486][20381] Updated weights for policy 1, policy_version 1754618 (0.0008) [2023-12-27 17:48:42,163][20347] Updated weights for policy 0, policy_version 1756067 (0.0007) [2023-12-27 17:48:42,228][20347] Updated weights for policy 0, policy_version 1756077 (0.0006) [2023-12-27 17:48:42,272][20381] Updated weights for policy 1, policy_version 1754628 (0.0008) [2023-12-27 17:48:42,287][20347] Updated weights for policy 0, policy_version 1756087 (0.0007) [2023-12-27 17:48:42,332][20381] Updated weights for policy 1, policy_version 1754638 (0.0008) [2023-12-27 17:48:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 898875392. Throughput: 0: 10260.1, 1: 10341.2. Samples: 898868944. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:48:42,363][19170] Avg episode reward: [(0, '921.850'), (1, '990.240')] [2023-12-27 17:48:42,395][20381] Updated weights for policy 1, policy_version 1754648 (0.0008) [2023-12-27 17:48:42,982][20347] Updated weights for policy 0, policy_version 1756097 (0.0007) [2023-12-27 17:48:43,040][20347] Updated weights for policy 0, policy_version 1756107 (0.0008) [2023-12-27 17:48:43,099][20347] Updated weights for policy 0, policy_version 1756117 (0.0007) [2023-12-27 17:48:43,129][20381] Updated weights for policy 1, policy_version 1754658 (0.0008) [2023-12-27 17:48:43,159][20347] Updated weights for policy 0, policy_version 1756127 (0.0006) [2023-12-27 17:48:43,179][20381] Updated weights for policy 1, policy_version 1754668 (0.0007) [2023-12-27 17:48:43,234][20381] Updated weights for policy 1, policy_version 1754678 (0.0006) [2023-12-27 17:48:43,293][20381] Updated weights for policy 1, policy_version 1754688 (0.0006) [2023-12-27 17:48:43,863][20347] Updated weights for policy 0, policy_version 1756137 (0.0008) [2023-12-27 17:48:43,877][20381] Updated weights for policy 1, policy_version 1754698 (0.0005) [2023-12-27 17:48:43,914][20347] Updated weights for policy 0, policy_version 1756147 (0.0007) [2023-12-27 17:48:43,933][20381] Updated weights for policy 1, policy_version 1754708 (0.0007) [2023-12-27 17:48:43,957][20347] Updated weights for policy 0, policy_version 1756157 (0.0007) [2023-12-27 17:48:43,988][20381] Updated weights for policy 1, policy_version 1754718 (0.0007) [2023-12-27 17:48:44,632][20347] Updated weights for policy 0, policy_version 1756167 (0.0007) [2023-12-27 17:48:44,685][20347] Updated weights for policy 0, policy_version 1756177 (0.0007) [2023-12-27 17:48:44,690][20381] Updated weights for policy 1, policy_version 1754728 (0.0006) [2023-12-27 17:48:44,739][20381] Updated weights for policy 1, policy_version 1754738 (0.0005) [2023-12-27 17:48:44,740][20347] Updated weights for policy 0, policy_version 1756187 (0.0007) [2023-12-27 17:48:44,789][20381] Updated weights for policy 1, policy_version 1754748 (0.0008) [2023-12-27 17:48:45,473][20347] Updated weights for policy 0, policy_version 1756197 (0.0009) [2023-12-27 17:48:45,507][20381] Updated weights for policy 1, policy_version 1754758 (0.0006) [2023-12-27 17:48:45,533][20347] Updated weights for policy 0, policy_version 1756207 (0.0010) [2023-12-27 17:48:45,557][20381] Updated weights for policy 1, policy_version 1754768 (0.0005) [2023-12-27 17:48:45,582][20347] Updated weights for policy 0, policy_version 1756217 (0.0010) [2023-12-27 17:48:45,606][20381] Updated weights for policy 1, policy_version 1754778 (0.0005) [2023-12-27 17:48:46,230][20347] Updated weights for policy 0, policy_version 1756227 (0.0009) [2023-12-27 17:48:46,275][20381] Updated weights for policy 1, policy_version 1754788 (0.0006) [2023-12-27 17:48:46,284][20347] Updated weights for policy 0, policy_version 1756237 (0.0007) [2023-12-27 17:48:46,327][20381] Updated weights for policy 1, policy_version 1754798 (0.0006) [2023-12-27 17:48:46,329][20347] Updated weights for policy 0, policy_version 1756247 (0.0010) [2023-12-27 17:48:46,375][20381] Updated weights for policy 1, policy_version 1754808 (0.0005) [2023-12-27 17:48:46,987][20347] Updated weights for policy 0, policy_version 1756257 (0.0010) [2023-12-27 17:48:47,047][20347] Updated weights for policy 0, policy_version 1756267 (0.0006) [2023-12-27 17:48:47,074][20381] Updated weights for policy 1, policy_version 1754818 (0.0008) [2023-12-27 17:48:47,100][20347] Updated weights for policy 0, policy_version 1756277 (0.0007) [2023-12-27 17:48:47,140][20381] Updated weights for policy 1, policy_version 1754828 (0.0011) [2023-12-27 17:48:47,148][20347] Updated weights for policy 0, policy_version 1756287 (0.0008) [2023-12-27 17:48:47,196][20381] Updated weights for policy 1, policy_version 1754838 (0.0011) [2023-12-27 17:48:47,251][20381] Updated weights for policy 1, policy_version 1754848 (0.0010) [2023-12-27 17:48:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 898981888. Throughput: 0: 10250.1, 1: 10294.1. Samples: 898988708. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:48:47,363][19170] Avg episode reward: [(0, '882.730'), (1, '1000.000')] [2023-12-27 17:48:47,770][20347] Updated weights for policy 0, policy_version 1756297 (0.0010) [2023-12-27 17:48:47,826][20347] Updated weights for policy 0, policy_version 1756307 (0.0010) [2023-12-27 17:48:47,828][20381] Updated weights for policy 1, policy_version 1754858 (0.0010) [2023-12-27 17:48:47,882][20381] Updated weights for policy 1, policy_version 1754868 (0.0010) [2023-12-27 17:48:47,882][20347] Updated weights for policy 0, policy_version 1756317 (0.0010) [2023-12-27 17:48:47,934][20381] Updated weights for policy 1, policy_version 1754878 (0.0011) [2023-12-27 17:48:48,505][20347] Updated weights for policy 0, policy_version 1756327 (0.0009) [2023-12-27 17:48:48,555][20347] Updated weights for policy 0, policy_version 1756337 (0.0010) [2023-12-27 17:48:48,611][20347] Updated weights for policy 0, policy_version 1756347 (0.0010) [2023-12-27 17:48:48,612][20381] Updated weights for policy 1, policy_version 1754888 (0.0007) [2023-12-27 17:48:48,672][20381] Updated weights for policy 1, policy_version 1754898 (0.0006) [2023-12-27 17:48:48,731][20381] Updated weights for policy 1, policy_version 1754908 (0.0005) [2023-12-27 17:48:49,322][20347] Updated weights for policy 0, policy_version 1756357 (0.0009) [2023-12-27 17:48:49,377][20381] Updated weights for policy 1, policy_version 1754918 (0.0007) [2023-12-27 17:48:49,395][20347] Updated weights for policy 0, policy_version 1756367 (0.0008) [2023-12-27 17:48:49,437][20381] Updated weights for policy 1, policy_version 1754928 (0.0007) [2023-12-27 17:48:49,460][20347] Updated weights for policy 0, policy_version 1756377 (0.0008) [2023-12-27 17:48:49,498][20381] Updated weights for policy 1, policy_version 1754938 (0.0007) [2023-12-27 17:48:50,230][20347] Updated weights for policy 0, policy_version 1756387 (0.0008) [2023-12-27 17:48:50,294][20381] Updated weights for policy 1, policy_version 1754948 (0.0008) [2023-12-27 17:48:50,295][20347] Updated weights for policy 0, policy_version 1756397 (0.0009) [2023-12-27 17:48:50,343][20381] Updated weights for policy 1, policy_version 1754958 (0.0009) [2023-12-27 17:48:50,355][20347] Updated weights for policy 0, policy_version 1756407 (0.0009) [2023-12-27 17:48:50,392][20381] Updated weights for policy 1, policy_version 1754968 (0.0009) [2023-12-27 17:48:51,031][20347] Updated weights for policy 0, policy_version 1756417 (0.0008) [2023-12-27 17:48:51,084][20347] Updated weights for policy 0, policy_version 1756427 (0.0008) [2023-12-27 17:48:51,125][20381] Updated weights for policy 1, policy_version 1754978 (0.0007) [2023-12-27 17:48:51,149][20347] Updated weights for policy 0, policy_version 1756437 (0.0008) [2023-12-27 17:48:51,178][20381] Updated weights for policy 1, policy_version 1754988 (0.0006) [2023-12-27 17:48:51,209][20347] Updated weights for policy 0, policy_version 1756447 (0.0008) [2023-12-27 17:48:51,237][20381] Updated weights for policy 1, policy_version 1754998 (0.0007) [2023-12-27 17:48:51,293][20381] Updated weights for policy 1, policy_version 1755008 (0.0008) [2023-12-27 17:48:51,888][20347] Updated weights for policy 0, policy_version 1756457 (0.0007) [2023-12-27 17:48:51,945][20347] Updated weights for policy 0, policy_version 1756467 (0.0007) [2023-12-27 17:48:52,007][20347] Updated weights for policy 0, policy_version 1756477 (0.0008) [2023-12-27 17:48:52,063][20381] Updated weights for policy 1, policy_version 1755018 (0.0008) [2023-12-27 17:48:52,117][20381] Updated weights for policy 1, policy_version 1755028 (0.0007) [2023-12-27 17:48:52,168][20381] Updated weights for policy 1, policy_version 1755038 (0.0007) [2023-12-27 17:48:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 899080192. Throughput: 0: 10225.7, 1: 10293.6. Samples: 899049952. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:48:52,363][19170] Avg episode reward: [(0, '892.380'), (1, '1000.000')] [2023-12-27 17:48:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001755040_449339392.pth... [2023-12-27 17:48:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001756480_449740800.pth... [2023-12-27 17:48:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001753824_449028096.pth [2023-12-27 17:48:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001755296_449437696.pth [2023-12-27 17:48:52,695][20347] Updated weights for policy 0, policy_version 1756487 (0.0008) [2023-12-27 17:48:52,758][20347] Updated weights for policy 0, policy_version 1756497 (0.0008) [2023-12-27 17:48:52,818][20347] Updated weights for policy 0, policy_version 1756507 (0.0007) [2023-12-27 17:48:52,904][20381] Updated weights for policy 1, policy_version 1755048 (0.0007) [2023-12-27 17:48:52,969][20381] Updated weights for policy 1, policy_version 1755058 (0.0008) [2023-12-27 17:48:53,034][20381] Updated weights for policy 1, policy_version 1755068 (0.0009) [2023-12-27 17:48:53,404][20347] Updated weights for policy 0, policy_version 1756517 (0.0008) [2023-12-27 17:48:53,455][20347] Updated weights for policy 0, policy_version 1756527 (0.0007) [2023-12-27 17:48:53,505][20347] Updated weights for policy 0, policy_version 1756537 (0.0007) [2023-12-27 17:48:53,755][20381] Updated weights for policy 1, policy_version 1755078 (0.0008) [2023-12-27 17:48:53,816][20381] Updated weights for policy 1, policy_version 1755088 (0.0009) [2023-12-27 17:48:53,861][20381] Updated weights for policy 1, policy_version 1755098 (0.0009) [2023-12-27 17:48:54,192][20347] Updated weights for policy 0, policy_version 1756547 (0.0007) [2023-12-27 17:48:54,241][20347] Updated weights for policy 0, policy_version 1756557 (0.0007) [2023-12-27 17:48:54,305][20347] Updated weights for policy 0, policy_version 1756567 (0.0008) [2023-12-27 17:48:54,573][20381] Updated weights for policy 1, policy_version 1755108 (0.0008) [2023-12-27 17:48:54,623][20381] Updated weights for policy 1, policy_version 1755118 (0.0008) [2023-12-27 17:48:54,667][20381] Updated weights for policy 1, policy_version 1755128 (0.0008) [2023-12-27 17:48:54,888][20347] Updated weights for policy 0, policy_version 1756577 (0.0006) [2023-12-27 17:48:54,943][20347] Updated weights for policy 0, policy_version 1756587 (0.0007) [2023-12-27 17:48:54,999][20347] Updated weights for policy 0, policy_version 1756597 (0.0008) [2023-12-27 17:48:55,058][20347] Updated weights for policy 0, policy_version 1756607 (0.0006) [2023-12-27 17:48:55,405][20381] Updated weights for policy 1, policy_version 1755138 (0.0008) [2023-12-27 17:48:55,457][20381] Updated weights for policy 1, policy_version 1755148 (0.0008) [2023-12-27 17:48:55,500][20381] Updated weights for policy 1, policy_version 1755158 (0.0007) [2023-12-27 17:48:55,550][20381] Updated weights for policy 1, policy_version 1755168 (0.0008) [2023-12-27 17:48:55,678][20347] Updated weights for policy 0, policy_version 1756617 (0.0005) [2023-12-27 17:48:55,743][20347] Updated weights for policy 0, policy_version 1756627 (0.0006) [2023-12-27 17:48:55,799][20347] Updated weights for policy 0, policy_version 1756637 (0.0008) [2023-12-27 17:48:56,322][20381] Updated weights for policy 1, policy_version 1755178 (0.0007) [2023-12-27 17:48:56,338][20347] Updated weights for policy 0, policy_version 1756647 (0.0008) [2023-12-27 17:48:56,371][20381] Updated weights for policy 1, policy_version 1755188 (0.0006) [2023-12-27 17:48:56,392][20347] Updated weights for policy 0, policy_version 1756657 (0.0008) [2023-12-27 17:48:56,427][20381] Updated weights for policy 1, policy_version 1755198 (0.0007) [2023-12-27 17:48:56,438][20347] Updated weights for policy 0, policy_version 1756667 (0.0010) [2023-12-27 17:48:56,966][20347] Updated weights for policy 0, policy_version 1756677 (0.0008) [2023-12-27 17:48:57,023][20347] Updated weights for policy 0, policy_version 1756687 (0.0010) [2023-12-27 17:48:57,070][20347] Updated weights for policy 0, policy_version 1756697 (0.0010) [2023-12-27 17:48:57,170][20381] Updated weights for policy 1, policy_version 1755208 (0.0007) [2023-12-27 17:48:57,223][20381] Updated weights for policy 1, policy_version 1755218 (0.0007) [2023-12-27 17:48:57,272][20381] Updated weights for policy 1, policy_version 1755228 (0.0008) [2023-12-27 17:48:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 899186688. Throughput: 0: 10306.1, 1: 10265.8. Samples: 899171280. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:48:57,363][19170] Avg episode reward: [(0, '941.290'), (1, '1000.000')] [2023-12-27 17:48:57,741][20347] Updated weights for policy 0, policy_version 1756707 (0.0010) [2023-12-27 17:48:57,797][20347] Updated weights for policy 0, policy_version 1756717 (0.0010) [2023-12-27 17:48:57,852][20347] Updated weights for policy 0, policy_version 1756727 (0.0010) [2023-12-27 17:48:57,956][20381] Updated weights for policy 1, policy_version 1755238 (0.0007) [2023-12-27 17:48:58,006][20381] Updated weights for policy 1, policy_version 1755248 (0.0007) [2023-12-27 17:48:58,061][20381] Updated weights for policy 1, policy_version 1755258 (0.0008) [2023-12-27 17:48:58,574][20347] Updated weights for policy 0, policy_version 1756737 (0.0010) [2023-12-27 17:48:58,632][20347] Updated weights for policy 0, policy_version 1756747 (0.0008) [2023-12-27 17:48:58,690][20347] Updated weights for policy 0, policy_version 1756757 (0.0008) [2023-12-27 17:48:58,749][20347] Updated weights for policy 0, policy_version 1756767 (0.0008) [2023-12-27 17:48:58,852][20381] Updated weights for policy 1, policy_version 1755268 (0.0008) [2023-12-27 17:48:58,945][20381] Updated weights for policy 1, policy_version 1755278 (0.0008) [2023-12-27 17:48:59,000][20381] Updated weights for policy 1, policy_version 1755288 (0.0008) [2023-12-27 17:48:59,551][20347] Updated weights for policy 0, policy_version 1756777 (0.0007) [2023-12-27 17:48:59,606][20347] Updated weights for policy 0, policy_version 1756787 (0.0007) [2023-12-27 17:48:59,665][20347] Updated weights for policy 0, policy_version 1756797 (0.0008) [2023-12-27 17:48:59,712][20381] Updated weights for policy 1, policy_version 1755298 (0.0008) [2023-12-27 17:48:59,768][20381] Updated weights for policy 1, policy_version 1755308 (0.0008) [2023-12-27 17:48:59,823][20381] Updated weights for policy 1, policy_version 1755318 (0.0007) [2023-12-27 17:48:59,881][20381] Updated weights for policy 1, policy_version 1755328 (0.0007) [2023-12-27 17:49:00,338][20347] Updated weights for policy 0, policy_version 1756807 (0.0008) [2023-12-27 17:49:00,400][20347] Updated weights for policy 0, policy_version 1756817 (0.0008) [2023-12-27 17:49:00,452][20347] Updated weights for policy 0, policy_version 1756827 (0.0007) [2023-12-27 17:49:00,574][20381] Updated weights for policy 1, policy_version 1755338 (0.0008) [2023-12-27 17:49:00,617][20381] Updated weights for policy 1, policy_version 1755348 (0.0005) [2023-12-27 17:49:00,670][20381] Updated weights for policy 1, policy_version 1755358 (0.0005) [2023-12-27 17:49:01,137][20347] Updated weights for policy 0, policy_version 1756837 (0.0007) [2023-12-27 17:49:01,196][20347] Updated weights for policy 0, policy_version 1756847 (0.0007) [2023-12-27 17:49:01,250][20381] Updated weights for policy 1, policy_version 1755368 (0.0006) [2023-12-27 17:49:01,251][20347] Updated weights for policy 0, policy_version 1756857 (0.0007) [2023-12-27 17:49:01,315][20381] Updated weights for policy 1, policy_version 1755378 (0.0007) [2023-12-27 17:49:01,376][20381] Updated weights for policy 1, policy_version 1755388 (0.0007) [2023-12-27 17:49:01,983][20347] Updated weights for policy 0, policy_version 1756867 (0.0007) [2023-12-27 17:49:02,017][20381] Updated weights for policy 1, policy_version 1755398 (0.0007) [2023-12-27 17:49:02,039][20347] Updated weights for policy 0, policy_version 1756877 (0.0006) [2023-12-27 17:49:02,072][20381] Updated weights for policy 1, policy_version 1755408 (0.0008) [2023-12-27 17:49:02,096][20347] Updated weights for policy 0, policy_version 1756887 (0.0006) [2023-12-27 17:49:02,123][20381] Updated weights for policy 1, policy_version 1755418 (0.0008) [2023-12-27 17:49:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 899284992. Throughput: 0: 10284.9, 1: 10223.2. Samples: 899292324. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:02,363][19170] Avg episode reward: [(0, '970.660'), (1, '1000.000')] [2023-12-27 17:49:02,753][20347] Updated weights for policy 0, policy_version 1756897 (0.0007) [2023-12-27 17:49:02,803][20347] Updated weights for policy 0, policy_version 1756907 (0.0008) [2023-12-27 17:49:02,846][20381] Updated weights for policy 1, policy_version 1755428 (0.0008) [2023-12-27 17:49:02,853][20347] Updated weights for policy 0, policy_version 1756917 (0.0005) [2023-12-27 17:49:02,903][20381] Updated weights for policy 1, policy_version 1755438 (0.0007) [2023-12-27 17:49:02,905][20347] Updated weights for policy 0, policy_version 1756927 (0.0008) [2023-12-27 17:49:02,959][20381] Updated weights for policy 1, policy_version 1755448 (0.0007) [2023-12-27 17:49:03,551][20347] Updated weights for policy 0, policy_version 1756937 (0.0009) [2023-12-27 17:49:03,595][20347] Updated weights for policy 0, policy_version 1756947 (0.0010) [2023-12-27 17:49:03,639][20347] Updated weights for policy 0, policy_version 1756957 (0.0010) [2023-12-27 17:49:03,661][20381] Updated weights for policy 1, policy_version 1755458 (0.0007) [2023-12-27 17:49:03,710][20381] Updated weights for policy 1, policy_version 1755468 (0.0007) [2023-12-27 17:49:03,765][20381] Updated weights for policy 1, policy_version 1755478 (0.0007) [2023-12-27 17:49:03,818][20381] Updated weights for policy 1, policy_version 1755488 (0.0007) [2023-12-27 17:49:04,352][20347] Updated weights for policy 0, policy_version 1756967 (0.0009) [2023-12-27 17:49:04,411][20347] Updated weights for policy 0, policy_version 1756977 (0.0008) [2023-12-27 17:49:04,460][20347] Updated weights for policy 0, policy_version 1756987 (0.0008) [2023-12-27 17:49:04,506][20381] Updated weights for policy 1, policy_version 1755498 (0.0007) [2023-12-27 17:49:04,554][20381] Updated weights for policy 1, policy_version 1755508 (0.0006) [2023-12-27 17:49:04,612][20381] Updated weights for policy 1, policy_version 1755518 (0.0006) [2023-12-27 17:49:05,150][20347] Updated weights for policy 0, policy_version 1756997 (0.0008) [2023-12-27 17:49:05,195][20347] Updated weights for policy 0, policy_version 1757007 (0.0006) [2023-12-27 17:49:05,237][20381] Updated weights for policy 1, policy_version 1755528 (0.0005) [2023-12-27 17:49:05,242][20347] Updated weights for policy 0, policy_version 1757017 (0.0005) [2023-12-27 17:49:05,287][20381] Updated weights for policy 1, policy_version 1755538 (0.0007) [2023-12-27 17:49:05,334][20381] Updated weights for policy 1, policy_version 1755548 (0.0008) [2023-12-27 17:49:05,802][20347] Updated weights for policy 0, policy_version 1757027 (0.0006) [2023-12-27 17:49:05,850][20347] Updated weights for policy 0, policy_version 1757037 (0.0008) [2023-12-27 17:49:05,894][20347] Updated weights for policy 0, policy_version 1757047 (0.0007) [2023-12-27 17:49:05,979][20381] Updated weights for policy 1, policy_version 1755558 (0.0007) [2023-12-27 17:49:06,030][20381] Updated weights for policy 1, policy_version 1755568 (0.0008) [2023-12-27 17:49:06,089][20381] Updated weights for policy 1, policy_version 1755578 (0.0008) [2023-12-27 17:49:06,609][20347] Updated weights for policy 0, policy_version 1757057 (0.0007) [2023-12-27 17:49:06,658][20347] Updated weights for policy 0, policy_version 1757067 (0.0008) [2023-12-27 17:49:06,704][20347] Updated weights for policy 0, policy_version 1757077 (0.0008) [2023-12-27 17:49:06,752][20347] Updated weights for policy 0, policy_version 1757087 (0.0007) [2023-12-27 17:49:06,822][20381] Updated weights for policy 1, policy_version 1755588 (0.0009) [2023-12-27 17:49:06,874][20381] Updated weights for policy 1, policy_version 1755598 (0.0009) [2023-12-27 17:49:06,934][20381] Updated weights for policy 1, policy_version 1755608 (0.0008) [2023-12-27 17:49:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 899383296. Throughput: 0: 10289.9, 1: 10208.4. Samples: 899354288. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:07,363][19170] Avg episode reward: [(0, '921.630'), (1, '1000.000')] [2023-12-27 17:49:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001755616_449486848.pth... [2023-12-27 17:49:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001757088_449896448.pth... [2023-12-27 17:49:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001755904_449593344.pth [2023-12-27 17:49:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001754432_449183744.pth [2023-12-27 17:49:07,465][20347] Updated weights for policy 0, policy_version 1757097 (0.0006) [2023-12-27 17:49:07,515][20347] Updated weights for policy 0, policy_version 1757107 (0.0005) [2023-12-27 17:49:07,572][20347] Updated weights for policy 0, policy_version 1757117 (0.0005) [2023-12-27 17:49:07,621][20381] Updated weights for policy 1, policy_version 1755618 (0.0007) [2023-12-27 17:49:07,677][20381] Updated weights for policy 1, policy_version 1755628 (0.0007) [2023-12-27 17:49:07,732][20381] Updated weights for policy 1, policy_version 1755638 (0.0007) [2023-12-27 17:49:07,780][20381] Updated weights for policy 1, policy_version 1755648 (0.0007) [2023-12-27 17:49:08,204][20347] Updated weights for policy 0, policy_version 1757127 (0.0007) [2023-12-27 17:49:08,261][20347] Updated weights for policy 0, policy_version 1757137 (0.0006) [2023-12-27 17:49:08,309][20347] Updated weights for policy 0, policy_version 1757147 (0.0006) [2023-12-27 17:49:08,412][20381] Updated weights for policy 1, policy_version 1755658 (0.0008) [2023-12-27 17:49:08,470][20381] Updated weights for policy 1, policy_version 1755668 (0.0007) [2023-12-27 17:49:08,520][20381] Updated weights for policy 1, policy_version 1755678 (0.0008) [2023-12-27 17:49:08,904][20347] Updated weights for policy 0, policy_version 1757157 (0.0007) [2023-12-27 17:49:08,949][20347] Updated weights for policy 0, policy_version 1757167 (0.0005) [2023-12-27 17:49:08,995][20347] Updated weights for policy 0, policy_version 1757177 (0.0007) [2023-12-27 17:49:09,311][20381] Updated weights for policy 1, policy_version 1755688 (0.0008) [2023-12-27 17:49:09,378][20381] Updated weights for policy 1, policy_version 1755698 (0.0008) [2023-12-27 17:49:09,446][20381] Updated weights for policy 1, policy_version 1755708 (0.0008) [2023-12-27 17:49:09,682][20347] Updated weights for policy 0, policy_version 1757187 (0.0008) [2023-12-27 17:49:09,732][20347] Updated weights for policy 0, policy_version 1757197 (0.0010) [2023-12-27 17:49:09,785][20347] Updated weights for policy 0, policy_version 1757207 (0.0010) [2023-12-27 17:49:10,168][20381] Updated weights for policy 1, policy_version 1755718 (0.0008) [2023-12-27 17:49:10,229][20381] Updated weights for policy 1, policy_version 1755728 (0.0008) [2023-12-27 17:49:10,283][20381] Updated weights for policy 1, policy_version 1755738 (0.0007) [2023-12-27 17:49:10,527][20347] Updated weights for policy 0, policy_version 1757217 (0.0007) [2023-12-27 17:49:10,583][20347] Updated weights for policy 0, policy_version 1757227 (0.0005) [2023-12-27 17:49:10,644][20347] Updated weights for policy 0, policy_version 1757237 (0.0007) [2023-12-27 17:49:10,698][20347] Updated weights for policy 0, policy_version 1757247 (0.0010) [2023-12-27 17:49:10,944][20381] Updated weights for policy 1, policy_version 1755748 (0.0009) [2023-12-27 17:49:11,000][20381] Updated weights for policy 1, policy_version 1755758 (0.0010) [2023-12-27 17:49:11,064][20381] Updated weights for policy 1, policy_version 1755768 (0.0010) [2023-12-27 17:49:11,348][20347] Updated weights for policy 0, policy_version 1757257 (0.0009) [2023-12-27 17:49:11,413][20347] Updated weights for policy 0, policy_version 1757267 (0.0010) [2023-12-27 17:49:11,474][20347] Updated weights for policy 0, policy_version 1757277 (0.0010) [2023-12-27 17:49:11,859][20381] Updated weights for policy 1, policy_version 1755778 (0.0009) [2023-12-27 17:49:11,915][20381] Updated weights for policy 1, policy_version 1755788 (0.0007) [2023-12-27 17:49:11,972][20381] Updated weights for policy 1, policy_version 1755798 (0.0005) [2023-12-27 17:49:12,032][20381] Updated weights for policy 1, policy_version 1755808 (0.0006) [2023-12-27 17:49:12,189][20347] Updated weights for policy 0, policy_version 1757287 (0.0010) [2023-12-27 17:49:12,256][20347] Updated weights for policy 0, policy_version 1757297 (0.0009) [2023-12-27 17:49:12,323][20347] Updated weights for policy 0, policy_version 1757307 (0.0007) [2023-12-27 17:49:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 899489792. Throughput: 0: 10330.8, 1: 10126.1. Samples: 899477800. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:12,363][19170] Avg episode reward: [(0, '882.690'), (1, '1000.000')] [2023-12-27 17:49:12,646][20381] Updated weights for policy 1, policy_version 1755818 (0.0006) [2023-12-27 17:49:12,696][20381] Updated weights for policy 1, policy_version 1755828 (0.0007) [2023-12-27 17:49:12,749][20381] Updated weights for policy 1, policy_version 1755838 (0.0008) [2023-12-27 17:49:13,004][20347] Updated weights for policy 0, policy_version 1757317 (0.0010) [2023-12-27 17:49:13,063][20347] Updated weights for policy 0, policy_version 1757327 (0.0011) [2023-12-27 17:49:13,131][20347] Updated weights for policy 0, policy_version 1757337 (0.0008) [2023-12-27 17:49:13,322][20381] Updated weights for policy 1, policy_version 1755848 (0.0007) [2023-12-27 17:49:13,379][20381] Updated weights for policy 1, policy_version 1755858 (0.0007) [2023-12-27 17:49:13,436][20381] Updated weights for policy 1, policy_version 1755868 (0.0007) [2023-12-27 17:49:13,780][20347] Updated weights for policy 0, policy_version 1757347 (0.0007) [2023-12-27 17:49:13,832][20347] Updated weights for policy 0, policy_version 1757357 (0.0005) [2023-12-27 17:49:13,885][20347] Updated weights for policy 0, policy_version 1757367 (0.0010) [2023-12-27 17:49:14,148][20381] Updated weights for policy 1, policy_version 1755878 (0.0007) [2023-12-27 17:49:14,207][20381] Updated weights for policy 1, policy_version 1755888 (0.0007) [2023-12-27 17:49:14,268][20381] Updated weights for policy 1, policy_version 1755898 (0.0008) [2023-12-27 17:49:14,535][20347] Updated weights for policy 0, policy_version 1757377 (0.0010) [2023-12-27 17:49:14,593][20347] Updated weights for policy 0, policy_version 1757387 (0.0006) [2023-12-27 17:49:14,645][20347] Updated weights for policy 0, policy_version 1757397 (0.0005) [2023-12-27 17:49:14,702][20347] Updated weights for policy 0, policy_version 1757407 (0.0010) [2023-12-27 17:49:14,973][20381] Updated weights for policy 1, policy_version 1755908 (0.0007) [2023-12-27 17:49:15,038][20381] Updated weights for policy 1, policy_version 1755918 (0.0008) [2023-12-27 17:49:15,105][20381] Updated weights for policy 1, policy_version 1755928 (0.0009) [2023-12-27 17:49:15,436][20347] Updated weights for policy 0, policy_version 1757417 (0.0008) [2023-12-27 17:49:15,489][20347] Updated weights for policy 0, policy_version 1757427 (0.0008) [2023-12-27 17:49:15,540][20347] Updated weights for policy 0, policy_version 1757437 (0.0007) [2023-12-27 17:49:15,791][20381] Updated weights for policy 1, policy_version 1755938 (0.0008) [2023-12-27 17:49:15,847][20381] Updated weights for policy 1, policy_version 1755948 (0.0009) [2023-12-27 17:49:15,894][20381] Updated weights for policy 1, policy_version 1755958 (0.0008) [2023-12-27 17:49:15,941][20381] Updated weights for policy 1, policy_version 1755968 (0.0009) [2023-12-27 17:49:16,207][20347] Updated weights for policy 0, policy_version 1757447 (0.0007) [2023-12-27 17:49:16,255][20347] Updated weights for policy 0, policy_version 1757457 (0.0005) [2023-12-27 17:49:16,310][20347] Updated weights for policy 0, policy_version 1757467 (0.0006) [2023-12-27 17:49:16,582][20381] Updated weights for policy 1, policy_version 1755978 (0.0010) [2023-12-27 17:49:16,631][20381] Updated weights for policy 1, policy_version 1755988 (0.0005) [2023-12-27 17:49:16,685][20381] Updated weights for policy 1, policy_version 1755998 (0.0009) [2023-12-27 17:49:17,019][20347] Updated weights for policy 0, policy_version 1757477 (0.0007) [2023-12-27 17:49:17,065][20347] Updated weights for policy 0, policy_version 1757487 (0.0007) [2023-12-27 17:49:17,110][20347] Updated weights for policy 0, policy_version 1757497 (0.0007) [2023-12-27 17:49:17,330][20381] Updated weights for policy 1, policy_version 1756008 (0.0007) [2023-12-27 17:49:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 899588096. Throughput: 0: 10302.6, 1: 10126.6. Samples: 899600124. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:17,363][19170] Avg episode reward: [(0, '872.840'), (1, '1000.000')] [2023-12-27 17:49:17,376][20381] Updated weights for policy 1, policy_version 1756018 (0.0008) [2023-12-27 17:49:17,428][20381] Updated weights for policy 1, policy_version 1756028 (0.0007) [2023-12-27 17:49:17,781][20347] Updated weights for policy 0, policy_version 1757507 (0.0008) [2023-12-27 17:49:17,834][20347] Updated weights for policy 0, policy_version 1757517 (0.0008) [2023-12-27 17:49:17,880][20347] Updated weights for policy 0, policy_version 1757527 (0.0006) [2023-12-27 17:49:18,168][20381] Updated weights for policy 1, policy_version 1756038 (0.0008) [2023-12-27 17:49:18,216][20381] Updated weights for policy 1, policy_version 1756048 (0.0007) [2023-12-27 17:49:18,259][20381] Updated weights for policy 1, policy_version 1756058 (0.0008) [2023-12-27 17:49:18,571][20347] Updated weights for policy 0, policy_version 1757537 (0.0005) [2023-12-27 17:49:18,631][20347] Updated weights for policy 0, policy_version 1757547 (0.0008) [2023-12-27 17:49:18,694][20347] Updated weights for policy 0, policy_version 1757557 (0.0006) [2023-12-27 17:49:18,757][20347] Updated weights for policy 0, policy_version 1757567 (0.0005) [2023-12-27 17:49:19,012][20381] Updated weights for policy 1, policy_version 1756068 (0.0008) [2023-12-27 17:49:19,064][20381] Updated weights for policy 1, policy_version 1756078 (0.0007) [2023-12-27 17:49:19,116][20381] Updated weights for policy 1, policy_version 1756088 (0.0008) [2023-12-27 17:49:19,325][20347] Updated weights for policy 0, policy_version 1757577 (0.0007) [2023-12-27 17:49:19,391][20347] Updated weights for policy 0, policy_version 1757587 (0.0008) [2023-12-27 17:49:19,450][20347] Updated weights for policy 0, policy_version 1757597 (0.0007) [2023-12-27 17:49:19,814][20381] Updated weights for policy 1, policy_version 1756098 (0.0009) [2023-12-27 17:49:19,881][20381] Updated weights for policy 1, policy_version 1756108 (0.0007) [2023-12-27 17:49:19,942][20381] Updated weights for policy 1, policy_version 1756118 (0.0007) [2023-12-27 17:49:20,003][20381] Updated weights for policy 1, policy_version 1756128 (0.0009) [2023-12-27 17:49:20,173][20347] Updated weights for policy 0, policy_version 1757607 (0.0008) [2023-12-27 17:49:20,230][20347] Updated weights for policy 0, policy_version 1757617 (0.0008) [2023-12-27 17:49:20,287][20347] Updated weights for policy 0, policy_version 1757627 (0.0008) [2023-12-27 17:49:20,780][20381] Updated weights for policy 1, policy_version 1756138 (0.0011) [2023-12-27 17:49:20,845][20381] Updated weights for policy 1, policy_version 1756148 (0.0011) [2023-12-27 17:49:20,902][20381] Updated weights for policy 1, policy_version 1756158 (0.0011) [2023-12-27 17:49:20,994][20347] Updated weights for policy 0, policy_version 1757637 (0.0008) [2023-12-27 17:49:21,063][20347] Updated weights for policy 0, policy_version 1757647 (0.0007) [2023-12-27 17:49:21,125][20347] Updated weights for policy 0, policy_version 1757657 (0.0008) [2023-12-27 17:49:21,711][20381] Updated weights for policy 1, policy_version 1756168 (0.0009) [2023-12-27 17:49:21,779][20381] Updated weights for policy 1, policy_version 1756178 (0.0009) [2023-12-27 17:49:21,834][20381] Updated weights for policy 1, policy_version 1756188 (0.0009) [2023-12-27 17:49:21,871][20347] Updated weights for policy 0, policy_version 1757667 (0.0007) [2023-12-27 17:49:21,929][20347] Updated weights for policy 0, policy_version 1757677 (0.0007) [2023-12-27 17:49:21,983][20347] Updated weights for policy 0, policy_version 1757687 (0.0008) [2023-12-27 17:49:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 899686400. Throughput: 0: 10305.1, 1: 10041.6. Samples: 899660072. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:22,363][19170] Avg episode reward: [(0, '853.250'), (1, '1000.000')] [2023-12-27 17:49:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001756192_449634304.pth... [2023-12-27 17:49:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001757696_450052096.pth... [2023-12-27 17:49:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001756480_449740800.pth [2023-12-27 17:49:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001755040_449339392.pth [2023-12-27 17:49:22,582][20381] Updated weights for policy 1, policy_version 1756198 (0.0008) [2023-12-27 17:49:22,633][20381] Updated weights for policy 1, policy_version 1756208 (0.0008) [2023-12-27 17:49:22,690][20381] Updated weights for policy 1, policy_version 1756218 (0.0011) [2023-12-27 17:49:22,726][20347] Updated weights for policy 0, policy_version 1757697 (0.0007) [2023-12-27 17:49:22,788][20347] Updated weights for policy 0, policy_version 1757707 (0.0007) [2023-12-27 17:49:22,849][20347] Updated weights for policy 0, policy_version 1757717 (0.0007) [2023-12-27 17:49:22,904][20347] Updated weights for policy 0, policy_version 1757727 (0.0007) [2023-12-27 17:49:23,361][20381] Updated weights for policy 1, policy_version 1756228 (0.0009) [2023-12-27 17:49:23,416][20381] Updated weights for policy 1, policy_version 1756238 (0.0006) [2023-12-27 17:49:23,474][20381] Updated weights for policy 1, policy_version 1756248 (0.0007) [2023-12-27 17:49:23,559][20347] Updated weights for policy 0, policy_version 1757737 (0.0007) [2023-12-27 17:49:23,603][20347] Updated weights for policy 0, policy_version 1757747 (0.0007) [2023-12-27 17:49:23,609][20291] KL-divergence is very high: 177.4016 [2023-12-27 17:49:23,647][20291] KL-divergence is very high: 196.9828 [2023-12-27 17:49:23,651][20347] Updated weights for policy 0, policy_version 1757757 (0.0007) [2023-12-27 17:49:24,083][20381] Updated weights for policy 1, policy_version 1756258 (0.0006) [2023-12-27 17:49:24,142][20381] Updated weights for policy 1, policy_version 1756268 (0.0008) [2023-12-27 17:49:24,197][20381] Updated weights for policy 1, policy_version 1756278 (0.0008) [2023-12-27 17:49:24,246][20381] Updated weights for policy 1, policy_version 1756288 (0.0009) [2023-12-27 17:49:24,301][20347] Updated weights for policy 0, policy_version 1757767 (0.0010) [2023-12-27 17:49:24,349][20347] Updated weights for policy 0, policy_version 1757777 (0.0010) [2023-12-27 17:49:24,402][20347] Updated weights for policy 0, policy_version 1757787 (0.0010) [2023-12-27 17:49:24,944][20381] Updated weights for policy 1, policy_version 1756298 (0.0007) [2023-12-27 17:49:24,997][20347] Updated weights for policy 0, policy_version 1757797 (0.0008) [2023-12-27 17:49:25,009][20381] Updated weights for policy 1, policy_version 1756308 (0.0007) [2023-12-27 17:49:25,049][20347] Updated weights for policy 0, policy_version 1757807 (0.0006) [2023-12-27 17:49:25,073][20381] Updated weights for policy 1, policy_version 1756318 (0.0008) [2023-12-27 17:49:25,098][20347] Updated weights for policy 0, policy_version 1757817 (0.0006) [2023-12-27 17:49:25,712][20347] Updated weights for policy 0, policy_version 1757827 (0.0008) [2023-12-27 17:49:25,744][20381] Updated weights for policy 1, policy_version 1756328 (0.0007) [2023-12-27 17:49:25,759][20347] Updated weights for policy 0, policy_version 1757837 (0.0006) [2023-12-27 17:49:25,792][20381] Updated weights for policy 1, policy_version 1756338 (0.0006) [2023-12-27 17:49:25,808][20347] Updated weights for policy 0, policy_version 1757847 (0.0006) [2023-12-27 17:49:25,844][20381] Updated weights for policy 1, policy_version 1756348 (0.0007) [2023-12-27 17:49:26,383][20381] Updated weights for policy 1, policy_version 1756358 (0.0007) [2023-12-27 17:49:26,431][20381] Updated weights for policy 1, policy_version 1756368 (0.0005) [2023-12-27 17:49:26,484][20381] Updated weights for policy 1, policy_version 1756378 (0.0005) [2023-12-27 17:49:26,508][20347] Updated weights for policy 0, policy_version 1757857 (0.0008) [2023-12-27 17:49:26,573][20347] Updated weights for policy 0, policy_version 1757867 (0.0007) [2023-12-27 17:49:26,632][20347] Updated weights for policy 0, policy_version 1757877 (0.0007) [2023-12-27 17:49:26,684][20347] Updated weights for policy 0, policy_version 1757887 (0.0007) [2023-12-27 17:49:27,090][20381] Updated weights for policy 1, policy_version 1756388 (0.0008) [2023-12-27 17:49:27,146][20381] Updated weights for policy 1, policy_version 1756398 (0.0005) [2023-12-27 17:49:27,201][20381] Updated weights for policy 1, policy_version 1756408 (0.0005) [2023-12-27 17:49:27,300][20347] Updated weights for policy 0, policy_version 1757897 (0.0007) [2023-12-27 17:49:27,358][20347] Updated weights for policy 0, policy_version 1757907 (0.0008) [2023-12-27 17:49:27,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 899792896. Throughput: 0: 10322.4, 1: 9980.5. Samples: 899782576. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:27,363][19170] Avg episode reward: [(0, '892.400'), (1, '1000.000')] [2023-12-27 17:49:27,415][20347] Updated weights for policy 0, policy_version 1757917 (0.0007) [2023-12-27 17:49:27,791][20381] Updated weights for policy 1, policy_version 1756418 (0.0005) [2023-12-27 17:49:27,841][20381] Updated weights for policy 1, policy_version 1756428 (0.0005) [2023-12-27 17:49:27,896][20381] Updated weights for policy 1, policy_version 1756438 (0.0010) [2023-12-27 17:49:27,949][20381] Updated weights for policy 1, policy_version 1756448 (0.0009) [2023-12-27 17:49:28,092][20347] Updated weights for policy 0, policy_version 1757927 (0.0007) [2023-12-27 17:49:28,136][20347] Updated weights for policy 0, policy_version 1757937 (0.0008) [2023-12-27 17:49:28,183][20347] Updated weights for policy 0, policy_version 1757947 (0.0007) [2023-12-27 17:49:28,567][20381] Updated weights for policy 1, policy_version 1756458 (0.0011) [2023-12-27 17:49:28,630][20381] Updated weights for policy 1, policy_version 1756468 (0.0011) [2023-12-27 17:49:28,684][20381] Updated weights for policy 1, policy_version 1756478 (0.0011) [2023-12-27 17:49:28,904][20347] Updated weights for policy 0, policy_version 1757957 (0.0008) [2023-12-27 17:49:28,957][20347] Updated weights for policy 0, policy_version 1757967 (0.0007) [2023-12-27 17:49:29,014][20347] Updated weights for policy 0, policy_version 1757977 (0.0007) [2023-12-27 17:49:29,387][20381] Updated weights for policy 1, policy_version 1756488 (0.0011) [2023-12-27 17:49:29,436][20381] Updated weights for policy 1, policy_version 1756498 (0.0011) [2023-12-27 17:49:29,491][20381] Updated weights for policy 1, policy_version 1756508 (0.0010) [2023-12-27 17:49:29,715][20347] Updated weights for policy 0, policy_version 1757987 (0.0007) [2023-12-27 17:49:29,774][20347] Updated weights for policy 0, policy_version 1757997 (0.0007) [2023-12-27 17:49:29,833][20347] Updated weights for policy 0, policy_version 1758007 (0.0007) [2023-12-27 17:49:30,214][20381] Updated weights for policy 1, policy_version 1756518 (0.0008) [2023-12-27 17:49:30,259][20381] Updated weights for policy 1, policy_version 1756528 (0.0007) [2023-12-27 17:49:30,311][20381] Updated weights for policy 1, policy_version 1756538 (0.0007) [2023-12-27 17:49:30,554][20347] Updated weights for policy 0, policy_version 1758017 (0.0007) [2023-12-27 17:49:30,610][20347] Updated weights for policy 0, policy_version 1758027 (0.0006) [2023-12-27 17:49:30,666][20347] Updated weights for policy 0, policy_version 1758037 (0.0007) [2023-12-27 17:49:30,714][20347] Updated weights for policy 0, policy_version 1758047 (0.0010) [2023-12-27 17:49:31,019][20381] Updated weights for policy 1, policy_version 1756548 (0.0007) [2023-12-27 17:49:31,083][20381] Updated weights for policy 1, policy_version 1756558 (0.0010) [2023-12-27 17:49:31,147][20381] Updated weights for policy 1, policy_version 1756568 (0.0011) [2023-12-27 17:49:31,350][20347] Updated weights for policy 0, policy_version 1758057 (0.0009) [2023-12-27 17:49:31,415][20347] Updated weights for policy 0, policy_version 1758067 (0.0012) [2023-12-27 17:49:31,470][20347] Updated weights for policy 0, policy_version 1758077 (0.0010) [2023-12-27 17:49:31,791][20381] Updated weights for policy 1, policy_version 1756578 (0.0007) [2023-12-27 17:49:31,854][20381] Updated weights for policy 1, policy_version 1756588 (0.0008) [2023-12-27 17:49:31,904][20381] Updated weights for policy 1, policy_version 1756598 (0.0007) [2023-12-27 17:49:31,956][20381] Updated weights for policy 1, policy_version 1756608 (0.0007) [2023-12-27 17:49:32,128][20347] Updated weights for policy 0, policy_version 1758087 (0.0008) [2023-12-27 17:49:32,180][20347] Updated weights for policy 0, policy_version 1758097 (0.0008) [2023-12-27 17:49:32,232][20347] Updated weights for policy 0, policy_version 1758107 (0.0006) [2023-12-27 17:49:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 899899392. Throughput: 0: 10348.4, 1: 10062.5. Samples: 899907200. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:32,363][19170] Avg episode reward: [(0, '911.860'), (1, '1000.000')] [2023-12-27 17:49:32,665][20381] Updated weights for policy 1, policy_version 1756618 (0.0007) [2023-12-27 17:49:32,723][20381] Updated weights for policy 1, policy_version 1756628 (0.0008) [2023-12-27 17:49:32,782][20381] Updated weights for policy 1, policy_version 1756638 (0.0008) [2023-12-27 17:49:32,904][20347] Updated weights for policy 0, policy_version 1758117 (0.0008) [2023-12-27 17:49:32,955][20347] Updated weights for policy 0, policy_version 1758127 (0.0007) [2023-12-27 17:49:33,000][20347] Updated weights for policy 0, policy_version 1758137 (0.0007) [2023-12-27 17:49:33,453][20381] Updated weights for policy 1, policy_version 1756648 (0.0006) [2023-12-27 17:49:33,507][20381] Updated weights for policy 1, policy_version 1756658 (0.0008) [2023-12-27 17:49:33,555][20381] Updated weights for policy 1, policy_version 1756668 (0.0008) [2023-12-27 17:49:33,737][20347] Updated weights for policy 0, policy_version 1758147 (0.0007) [2023-12-27 17:49:33,792][20347] Updated weights for policy 0, policy_version 1758157 (0.0007) [2023-12-27 17:49:33,850][20347] Updated weights for policy 0, policy_version 1758167 (0.0007) [2023-12-27 17:49:34,183][20381] Updated weights for policy 1, policy_version 1756678 (0.0007) [2023-12-27 17:49:34,241][20381] Updated weights for policy 1, policy_version 1756688 (0.0008) [2023-12-27 17:49:34,304][20381] Updated weights for policy 1, policy_version 1756698 (0.0009) [2023-12-27 17:49:34,487][20347] Updated weights for policy 0, policy_version 1758177 (0.0007) [2023-12-27 17:49:34,546][20347] Updated weights for policy 0, policy_version 1758187 (0.0007) [2023-12-27 17:49:34,606][20347] Updated weights for policy 0, policy_version 1758197 (0.0008) [2023-12-27 17:49:34,658][20347] Updated weights for policy 0, policy_version 1758207 (0.0007) [2023-12-27 17:49:34,970][20381] Updated weights for policy 1, policy_version 1756708 (0.0007) [2023-12-27 17:49:35,036][20381] Updated weights for policy 1, policy_version 1756718 (0.0008) [2023-12-27 17:49:35,082][20381] Updated weights for policy 1, policy_version 1756728 (0.0010) [2023-12-27 17:49:35,331][20347] Updated weights for policy 0, policy_version 1758217 (0.0005) [2023-12-27 17:49:35,350][20291] KL-divergence is very high: 130.1228 [2023-12-27 17:49:35,369][20291] KL-divergence is very high: 161.8705 [2023-12-27 17:49:35,376][20291] KL-divergence is very high: 153.1597 [2023-12-27 17:49:35,392][20347] Updated weights for policy 0, policy_version 1758227 (0.0005) [2023-12-27 17:49:35,399][20291] KL-divergence is very high: 136.3770 [2023-12-27 17:49:35,419][20291] KL-divergence is very high: 115.1581 [2023-12-27 17:49:35,455][20347] Updated weights for policy 0, policy_version 1758237 (0.0006) [2023-12-27 17:49:35,704][20381] Updated weights for policy 1, policy_version 1756738 (0.0009) [2023-12-27 17:49:35,755][20381] Updated weights for policy 1, policy_version 1756748 (0.0008) [2023-12-27 17:49:35,807][20381] Updated weights for policy 1, policy_version 1756758 (0.0006) [2023-12-27 17:49:35,858][20381] Updated weights for policy 1, policy_version 1756768 (0.0006) [2023-12-27 17:49:36,072][20347] Updated weights for policy 0, policy_version 1758247 (0.0008) [2023-12-27 17:49:36,128][20347] Updated weights for policy 0, policy_version 1758257 (0.0009) [2023-12-27 17:49:36,183][20347] Updated weights for policy 0, policy_version 1758267 (0.0008) [2023-12-27 17:49:36,521][20381] Updated weights for policy 1, policy_version 1756778 (0.0011) [2023-12-27 17:49:36,569][20381] Updated weights for policy 1, policy_version 1756788 (0.0005) [2023-12-27 17:49:36,616][20381] Updated weights for policy 1, policy_version 1756798 (0.0009) [2023-12-27 17:49:36,960][20347] Updated weights for policy 0, policy_version 1758277 (0.0008) [2023-12-27 17:49:37,008][20347] Updated weights for policy 0, policy_version 1758287 (0.0008) [2023-12-27 17:49:37,054][20347] Updated weights for policy 0, policy_version 1758297 (0.0008) [2023-12-27 17:49:37,177][20381] Updated weights for policy 1, policy_version 1756808 (0.0005) [2023-12-27 17:49:37,224][20381] Updated weights for policy 1, policy_version 1756818 (0.0005) [2023-12-27 17:49:37,285][20381] Updated weights for policy 1, policy_version 1756828 (0.0010) [2023-12-27 17:49:37,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 900005888. Throughput: 0: 10345.8, 1: 10069.4. Samples: 899968632. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:37,363][19170] Avg episode reward: [(0, '921.660'), (1, '1000.000')] [2023-12-27 17:49:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001756832_449798144.pth... [2023-12-27 17:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001758304_450207744.pth... [2023-12-27 17:49:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001755616_449486848.pth [2023-12-27 17:49:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001757088_449896448.pth [2023-12-27 17:49:37,831][20381] Updated weights for policy 1, policy_version 1756838 (0.0006) [2023-12-27 17:49:37,839][20347] Updated weights for policy 0, policy_version 1758307 (0.0008) [2023-12-27 17:49:37,886][20381] Updated weights for policy 1, policy_version 1756848 (0.0009) [2023-12-27 17:49:37,903][20347] Updated weights for policy 0, policy_version 1758317 (0.0008) [2023-12-27 17:49:37,946][20381] Updated weights for policy 1, policy_version 1756858 (0.0011) [2023-12-27 17:49:37,962][20347] Updated weights for policy 0, policy_version 1758327 (0.0006) [2023-12-27 17:49:38,570][20381] Updated weights for policy 1, policy_version 1756868 (0.0010) [2023-12-27 17:49:38,634][20381] Updated weights for policy 1, policy_version 1756878 (0.0008) [2023-12-27 17:49:38,681][20347] Updated weights for policy 0, policy_version 1758337 (0.0006) [2023-12-27 17:49:38,698][20381] Updated weights for policy 1, policy_version 1756888 (0.0011) [2023-12-27 17:49:38,738][20347] Updated weights for policy 0, policy_version 1758347 (0.0005) [2023-12-27 17:49:38,799][20347] Updated weights for policy 0, policy_version 1758357 (0.0007) [2023-12-27 17:49:38,859][20347] Updated weights for policy 0, policy_version 1758367 (0.0006) [2023-12-27 17:49:39,327][20381] Updated weights for policy 1, policy_version 1756898 (0.0011) [2023-12-27 17:49:39,393][20381] Updated weights for policy 1, policy_version 1756908 (0.0009) [2023-12-27 17:49:39,454][20381] Updated weights for policy 1, policy_version 1756918 (0.0008) [2023-12-27 17:49:39,498][20381] Updated weights for policy 1, policy_version 1756928 (0.0009) [2023-12-27 17:49:39,572][20347] Updated weights for policy 0, policy_version 1758377 (0.0008) [2023-12-27 17:49:39,628][20347] Updated weights for policy 0, policy_version 1758387 (0.0008) [2023-12-27 17:49:39,683][20347] Updated weights for policy 0, policy_version 1758397 (0.0008) [2023-12-27 17:49:40,218][20381] Updated weights for policy 1, policy_version 1756938 (0.0008) [2023-12-27 17:49:40,276][20381] Updated weights for policy 1, policy_version 1756948 (0.0008) [2023-12-27 17:49:40,332][20381] Updated weights for policy 1, policy_version 1756958 (0.0007) [2023-12-27 17:49:40,411][20347] Updated weights for policy 0, policy_version 1758407 (0.0007) [2023-12-27 17:49:40,467][20347] Updated weights for policy 0, policy_version 1758417 (0.0007) [2023-12-27 17:49:40,525][20347] Updated weights for policy 0, policy_version 1758427 (0.0007) [2023-12-27 17:49:41,070][20381] Updated weights for policy 1, policy_version 1756968 (0.0008) [2023-12-27 17:49:41,132][20381] Updated weights for policy 1, policy_version 1756978 (0.0009) [2023-12-27 17:49:41,197][20381] Updated weights for policy 1, policy_version 1756988 (0.0007) [2023-12-27 17:49:41,264][20347] Updated weights for policy 0, policy_version 1758437 (0.0008) [2023-12-27 17:49:41,321][20347] Updated weights for policy 0, policy_version 1758447 (0.0007) [2023-12-27 17:49:41,387][20347] Updated weights for policy 0, policy_version 1758457 (0.0007) [2023-12-27 17:49:41,932][20381] Updated weights for policy 1, policy_version 1756998 (0.0009) [2023-12-27 17:49:41,999][20381] Updated weights for policy 1, policy_version 1757008 (0.0007) [2023-12-27 17:49:42,056][20381] Updated weights for policy 1, policy_version 1757018 (0.0008) [2023-12-27 17:49:42,136][20347] Updated weights for policy 0, policy_version 1758467 (0.0008) [2023-12-27 17:49:42,193][20347] Updated weights for policy 0, policy_version 1758477 (0.0010) [2023-12-27 17:49:42,246][20347] Updated weights for policy 0, policy_version 1758487 (0.0009) [2023-12-27 17:49:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 900104192. Throughput: 0: 10223.8, 1: 10221.7. Samples: 900091328. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:42,363][19170] Avg episode reward: [(0, '911.910'), (1, '1000.000')] [2023-12-27 17:49:42,781][20381] Updated weights for policy 1, policy_version 1757028 (0.0008) [2023-12-27 17:49:42,839][20381] Updated weights for policy 1, policy_version 1757038 (0.0008) [2023-12-27 17:49:42,892][20381] Updated weights for policy 1, policy_version 1757048 (0.0005) [2023-12-27 17:49:42,939][20347] Updated weights for policy 0, policy_version 1758497 (0.0010) [2023-12-27 17:49:42,997][20347] Updated weights for policy 0, policy_version 1758507 (0.0009) [2023-12-27 17:49:43,051][20347] Updated weights for policy 0, policy_version 1758517 (0.0010) [2023-12-27 17:49:43,109][20347] Updated weights for policy 0, policy_version 1758527 (0.0010) [2023-12-27 17:49:43,492][20381] Updated weights for policy 1, policy_version 1757058 (0.0006) [2023-12-27 17:49:43,553][20381] Updated weights for policy 1, policy_version 1757068 (0.0010) [2023-12-27 17:49:43,603][20381] Updated weights for policy 1, policy_version 1757078 (0.0010) [2023-12-27 17:49:43,652][20381] Updated weights for policy 1, policy_version 1757088 (0.0009) [2023-12-27 17:49:43,750][20347] Updated weights for policy 0, policy_version 1758537 (0.0010) [2023-12-27 17:49:43,813][20347] Updated weights for policy 0, policy_version 1758547 (0.0009) [2023-12-27 17:49:43,873][20347] Updated weights for policy 0, policy_version 1758557 (0.0005) [2023-12-27 17:49:44,306][20381] Updated weights for policy 1, policy_version 1757098 (0.0008) [2023-12-27 17:49:44,368][20381] Updated weights for policy 1, policy_version 1757108 (0.0007) [2023-12-27 17:49:44,426][20381] Updated weights for policy 1, policy_version 1757118 (0.0007) [2023-12-27 17:49:44,509][20347] Updated weights for policy 0, policy_version 1758567 (0.0009) [2023-12-27 17:49:44,562][20347] Updated weights for policy 0, policy_version 1758577 (0.0010) [2023-12-27 17:49:44,612][20347] Updated weights for policy 0, policy_version 1758587 (0.0007) [2023-12-27 17:49:45,118][20381] Updated weights for policy 1, policy_version 1757128 (0.0008) [2023-12-27 17:49:45,174][20381] Updated weights for policy 1, policy_version 1757138 (0.0008) [2023-12-27 17:49:45,229][20381] Updated weights for policy 1, policy_version 1757148 (0.0008) [2023-12-27 17:49:45,315][20347] Updated weights for policy 0, policy_version 1758597 (0.0010) [2023-12-27 17:49:45,368][20347] Updated weights for policy 0, policy_version 1758607 (0.0010) [2023-12-27 17:49:45,416][20347] Updated weights for policy 0, policy_version 1758617 (0.0010) [2023-12-27 17:49:45,929][20381] Updated weights for policy 1, policy_version 1757158 (0.0008) [2023-12-27 17:49:45,984][20381] Updated weights for policy 1, policy_version 1757168 (0.0007) [2023-12-27 17:49:45,989][20347] Updated weights for policy 0, policy_version 1758627 (0.0009) [2023-12-27 17:49:46,040][20381] Updated weights for policy 1, policy_version 1757178 (0.0007) [2023-12-27 17:49:46,045][20347] Updated weights for policy 0, policy_version 1758637 (0.0006) [2023-12-27 17:49:46,101][20347] Updated weights for policy 0, policy_version 1758647 (0.0007) [2023-12-27 17:49:46,707][20347] Updated weights for policy 0, policy_version 1758657 (0.0005) [2023-12-27 17:49:46,726][20381] Updated weights for policy 1, policy_version 1757188 (0.0007) [2023-12-27 17:49:46,762][20347] Updated weights for policy 0, policy_version 1758667 (0.0005) [2023-12-27 17:49:46,782][20381] Updated weights for policy 1, policy_version 1757198 (0.0007) [2023-12-27 17:49:46,815][20347] Updated weights for policy 0, policy_version 1758677 (0.0006) [2023-12-27 17:49:46,839][20381] Updated weights for policy 1, policy_version 1757208 (0.0007) [2023-12-27 17:49:46,868][20347] Updated weights for policy 0, policy_version 1758687 (0.0005) [2023-12-27 17:49:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 900202496. Throughput: 0: 10247.2, 1: 10253.6. Samples: 900214860. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:47,363][19170] Avg episode reward: [(0, '863.040'), (1, '1000.000')] [2023-12-27 17:49:47,478][20347] Updated weights for policy 0, policy_version 1758697 (0.0006) [2023-12-27 17:49:47,539][20381] Updated weights for policy 1, policy_version 1757218 (0.0007) [2023-12-27 17:49:47,544][20347] Updated weights for policy 0, policy_version 1758707 (0.0007) [2023-12-27 17:49:47,590][20347] Updated weights for policy 0, policy_version 1758717 (0.0007) [2023-12-27 17:49:47,597][20381] Updated weights for policy 1, policy_version 1757228 (0.0007) [2023-12-27 17:49:47,654][20381] Updated weights for policy 1, policy_version 1757238 (0.0007) [2023-12-27 17:49:47,710][20381] Updated weights for policy 1, policy_version 1757248 (0.0008) [2023-12-27 17:49:48,231][20347] Updated weights for policy 0, policy_version 1758727 (0.0007) [2023-12-27 17:49:48,279][20347] Updated weights for policy 0, policy_version 1758737 (0.0008) [2023-12-27 17:49:48,334][20347] Updated weights for policy 0, policy_version 1758747 (0.0007) [2023-12-27 17:49:48,410][20381] Updated weights for policy 1, policy_version 1757258 (0.0008) [2023-12-27 17:49:48,468][20381] Updated weights for policy 1, policy_version 1757268 (0.0007) [2023-12-27 17:49:48,527][20381] Updated weights for policy 1, policy_version 1757278 (0.0007) [2023-12-27 17:49:49,071][20347] Updated weights for policy 0, policy_version 1758757 (0.0009) [2023-12-27 17:49:49,124][20347] Updated weights for policy 0, policy_version 1758767 (0.0007) [2023-12-27 17:49:49,183][20347] Updated weights for policy 0, policy_version 1758777 (0.0008) [2023-12-27 17:49:49,205][20381] Updated weights for policy 1, policy_version 1757288 (0.0006) [2023-12-27 17:49:49,274][20381] Updated weights for policy 1, policy_version 1757298 (0.0008) [2023-12-27 17:49:49,319][20381] Updated weights for policy 1, policy_version 1757308 (0.0007) [2023-12-27 17:49:49,816][20347] Updated weights for policy 0, policy_version 1758787 (0.0011) [2023-12-27 17:49:49,883][20347] Updated weights for policy 0, policy_version 1758797 (0.0007) [2023-12-27 17:49:49,944][20347] Updated weights for policy 0, policy_version 1758807 (0.0008) [2023-12-27 17:49:50,102][20381] Updated weights for policy 1, policy_version 1757318 (0.0007) [2023-12-27 17:49:50,166][20381] Updated weights for policy 1, policy_version 1757328 (0.0007) [2023-12-27 17:49:50,224][20381] Updated weights for policy 1, policy_version 1757338 (0.0009) [2023-12-27 17:49:50,557][20347] Updated weights for policy 0, policy_version 1758817 (0.0007) [2023-12-27 17:49:50,630][20347] Updated weights for policy 0, policy_version 1758827 (0.0008) [2023-12-27 17:49:50,685][20347] Updated weights for policy 0, policy_version 1758837 (0.0008) [2023-12-27 17:49:50,745][20347] Updated weights for policy 0, policy_version 1758847 (0.0008) [2023-12-27 17:49:50,932][20381] Updated weights for policy 1, policy_version 1757348 (0.0009) [2023-12-27 17:49:50,990][20381] Updated weights for policy 1, policy_version 1757358 (0.0008) [2023-12-27 17:49:51,053][20381] Updated weights for policy 1, policy_version 1757368 (0.0009) [2023-12-27 17:49:51,439][20347] Updated weights for policy 0, policy_version 1758857 (0.0008) [2023-12-27 17:49:51,508][20347] Updated weights for policy 0, policy_version 1758867 (0.0008) [2023-12-27 17:49:51,569][20347] Updated weights for policy 0, policy_version 1758877 (0.0008) [2023-12-27 17:49:51,794][20381] Updated weights for policy 1, policy_version 1757378 (0.0008) [2023-12-27 17:49:51,850][20381] Updated weights for policy 1, policy_version 1757388 (0.0008) [2023-12-27 17:49:51,907][20381] Updated weights for policy 1, policy_version 1757398 (0.0007) [2023-12-27 17:49:51,963][20381] Updated weights for policy 1, policy_version 1757408 (0.0008) [2023-12-27 17:49:52,312][20347] Updated weights for policy 0, policy_version 1758887 (0.0010) [2023-12-27 17:49:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 900300800. Throughput: 0: 10255.0, 1: 10202.2. Samples: 900274864. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:52,363][19170] Avg episode reward: [(0, '833.750'), (1, '1000.000')] [2023-12-27 17:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001757408_449945600.pth... [2023-12-27 17:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001756192_449634304.pth [2023-12-27 17:49:52,378][20347] Updated weights for policy 0, policy_version 1758897 (0.0009) [2023-12-27 17:49:52,439][20347] Updated weights for policy 0, policy_version 1758907 (0.0006) [2023-12-27 17:49:52,474][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001758912_450363392.pth... [2023-12-27 17:49:52,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001757696_450052096.pth [2023-12-27 17:49:52,617][20381] Updated weights for policy 1, policy_version 1757418 (0.0011) [2023-12-27 17:49:52,667][20381] Updated weights for policy 1, policy_version 1757428 (0.0011) [2023-12-27 17:49:52,717][20381] Updated weights for policy 1, policy_version 1757438 (0.0011) [2023-12-27 17:49:53,123][20347] Updated weights for policy 0, policy_version 1758917 (0.0009) [2023-12-27 17:49:53,182][20347] Updated weights for policy 0, policy_version 1758927 (0.0010) [2023-12-27 17:49:53,243][20347] Updated weights for policy 0, policy_version 1758937 (0.0010) [2023-12-27 17:49:53,418][20381] Updated weights for policy 1, policy_version 1757448 (0.0010) [2023-12-27 17:49:53,466][20381] Updated weights for policy 1, policy_version 1757458 (0.0010) [2023-12-27 17:49:53,522][20381] Updated weights for policy 1, policy_version 1757468 (0.0007) [2023-12-27 17:49:53,834][20347] Updated weights for policy 0, policy_version 1758947 (0.0009) [2023-12-27 17:49:53,893][20347] Updated weights for policy 0, policy_version 1758957 (0.0006) [2023-12-27 17:49:53,948][20347] Updated weights for policy 0, policy_version 1758967 (0.0010) [2023-12-27 17:49:54,138][20381] Updated weights for policy 1, policy_version 1757478 (0.0009) [2023-12-27 17:49:54,191][20381] Updated weights for policy 1, policy_version 1757488 (0.0010) [2023-12-27 17:49:54,248][20381] Updated weights for policy 1, policy_version 1757498 (0.0008) [2023-12-27 17:49:54,561][20347] Updated weights for policy 0, policy_version 1758977 (0.0009) [2023-12-27 17:49:54,617][20347] Updated weights for policy 0, policy_version 1758987 (0.0006) [2023-12-27 17:49:54,671][20347] Updated weights for policy 0, policy_version 1758997 (0.0005) [2023-12-27 17:49:54,731][20347] Updated weights for policy 0, policy_version 1759007 (0.0005) [2023-12-27 17:49:54,921][20381] Updated weights for policy 1, policy_version 1757508 (0.0009) [2023-12-27 17:49:54,983][20381] Updated weights for policy 1, policy_version 1757518 (0.0011) [2023-12-27 17:49:55,037][20381] Updated weights for policy 1, policy_version 1757528 (0.0011) [2023-12-27 17:49:55,304][20347] Updated weights for policy 0, policy_version 1759017 (0.0007) [2023-12-27 17:49:55,348][20347] Updated weights for policy 0, policy_version 1759027 (0.0007) [2023-12-27 17:49:55,405][20347] Updated weights for policy 0, policy_version 1759037 (0.0007) [2023-12-27 17:49:55,719][20381] Updated weights for policy 1, policy_version 1757538 (0.0010) [2023-12-27 17:49:55,768][20381] Updated weights for policy 1, policy_version 1757548 (0.0010) [2023-12-27 17:49:55,816][20381] Updated weights for policy 1, policy_version 1757558 (0.0010) [2023-12-27 17:49:55,864][20381] Updated weights for policy 1, policy_version 1757568 (0.0010) [2023-12-27 17:49:56,111][20347] Updated weights for policy 0, policy_version 1759047 (0.0006) [2023-12-27 17:49:56,172][20347] Updated weights for policy 0, policy_version 1759057 (0.0006) [2023-12-27 17:49:56,227][20347] Updated weights for policy 0, policy_version 1759067 (0.0007) [2023-12-27 17:49:56,572][20381] Updated weights for policy 1, policy_version 1757578 (0.0010) [2023-12-27 17:49:56,620][20381] Updated weights for policy 1, policy_version 1757588 (0.0010) [2023-12-27 17:49:56,664][20381] Updated weights for policy 1, policy_version 1757598 (0.0010) [2023-12-27 17:49:56,923][20347] Updated weights for policy 0, policy_version 1759077 (0.0006) [2023-12-27 17:49:56,990][20347] Updated weights for policy 0, policy_version 1759087 (0.0005) [2023-12-27 17:49:57,054][20347] Updated weights for policy 0, policy_version 1759097 (0.0005) [2023-12-27 17:49:57,348][20381] Updated weights for policy 1, policy_version 1757608 (0.0010) [2023-12-27 17:49:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 900407296. Throughput: 0: 10244.3, 1: 10229.9. Samples: 900399136. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:49:57,363][19170] Avg episode reward: [(0, '892.540'), (1, '1000.000')] [2023-12-27 17:49:57,403][20381] Updated weights for policy 1, policy_version 1757618 (0.0009) [2023-12-27 17:49:57,465][20381] Updated weights for policy 1, policy_version 1757628 (0.0010) [2023-12-27 17:49:57,569][20347] Updated weights for policy 0, policy_version 1759107 (0.0005) [2023-12-27 17:49:57,627][20347] Updated weights for policy 0, policy_version 1759117 (0.0005) [2023-12-27 17:49:57,685][20347] Updated weights for policy 0, policy_version 1759127 (0.0005) [2023-12-27 17:49:58,056][20381] Updated weights for policy 1, policy_version 1757638 (0.0008) [2023-12-27 17:49:58,106][20381] Updated weights for policy 1, policy_version 1757648 (0.0007) [2023-12-27 17:49:58,151][20381] Updated weights for policy 1, policy_version 1757658 (0.0007) [2023-12-27 17:49:58,242][20347] Updated weights for policy 0, policy_version 1759137 (0.0006) [2023-12-27 17:49:58,305][20347] Updated weights for policy 0, policy_version 1759147 (0.0008) [2023-12-27 17:49:58,374][20347] Updated weights for policy 0, policy_version 1759157 (0.0008) [2023-12-27 17:49:58,392][20291] KL-divergence is very high: 128.3741 [2023-12-27 17:49:58,435][20347] Updated weights for policy 0, policy_version 1759167 (0.0007) [2023-12-27 17:49:58,951][20381] Updated weights for policy 1, policy_version 1757668 (0.0007) [2023-12-27 17:49:59,006][20381] Updated weights for policy 1, policy_version 1757678 (0.0007) [2023-12-27 17:49:59,069][20381] Updated weights for policy 1, policy_version 1757688 (0.0008) [2023-12-27 17:49:59,227][20347] Updated weights for policy 0, policy_version 1759178 (0.0008) [2023-12-27 17:49:59,296][20347] Updated weights for policy 0, policy_version 1759188 (0.0011) [2023-12-27 17:49:59,359][20347] Updated weights for policy 0, policy_version 1759198 (0.0007) [2023-12-27 17:49:59,794][20381] Updated weights for policy 1, policy_version 1757698 (0.0008) [2023-12-27 17:49:59,863][20381] Updated weights for policy 1, policy_version 1757708 (0.0010) [2023-12-27 17:49:59,920][20381] Updated weights for policy 1, policy_version 1757718 (0.0010) [2023-12-27 17:49:59,980][20347] Updated weights for policy 0, policy_version 1759209 (0.0009) [2023-12-27 17:49:59,989][20381] Updated weights for policy 1, policy_version 1757728 (0.0010) [2023-12-27 17:50:00,044][20347] Updated weights for policy 0, policy_version 1759219 (0.0010) [2023-12-27 17:50:00,092][20347] Updated weights for policy 0, policy_version 1759229 (0.0010) [2023-12-27 17:50:00,682][20381] Updated weights for policy 1, policy_version 1757738 (0.0007) [2023-12-27 17:50:00,734][20381] Updated weights for policy 1, policy_version 1757748 (0.0005) [2023-12-27 17:50:00,742][20347] Updated weights for policy 0, policy_version 1759239 (0.0008) [2023-12-27 17:50:00,789][20381] Updated weights for policy 1, policy_version 1757758 (0.0005) [2023-12-27 17:50:00,796][20347] Updated weights for policy 0, policy_version 1759249 (0.0007) [2023-12-27 17:50:00,852][20347] Updated weights for policy 0, policy_version 1759259 (0.0009) [2023-12-27 17:50:01,365][20381] Updated weights for policy 1, policy_version 1757768 (0.0007) [2023-12-27 17:50:01,430][20381] Updated weights for policy 1, policy_version 1757778 (0.0008) [2023-12-27 17:50:01,492][20381] Updated weights for policy 1, policy_version 1757788 (0.0008) [2023-12-27 17:50:01,612][20347] Updated weights for policy 0, policy_version 1759269 (0.0009) [2023-12-27 17:50:01,676][20347] Updated weights for policy 0, policy_version 1759279 (0.0007) [2023-12-27 17:50:01,741][20347] Updated weights for policy 0, policy_version 1759289 (0.0007) [2023-12-27 17:50:02,176][20381] Updated weights for policy 1, policy_version 1757798 (0.0008) [2023-12-27 17:50:02,232][20381] Updated weights for policy 1, policy_version 1757808 (0.0007) [2023-12-27 17:50:02,290][20381] Updated weights for policy 1, policy_version 1757818 (0.0007) [2023-12-27 17:50:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 900513792. Throughput: 0: 10288.7, 1: 10214.5. Samples: 900522768. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:02,363][19170] Avg episode reward: [(0, '912.140'), (1, '1000.000')] [2023-12-27 17:50:02,440][20347] Updated weights for policy 0, policy_version 1759299 (0.0008) [2023-12-27 17:50:02,488][20347] Updated weights for policy 0, policy_version 1759309 (0.0008) [2023-12-27 17:50:02,536][20347] Updated weights for policy 0, policy_version 1759319 (0.0008) [2023-12-27 17:50:02,992][20381] Updated weights for policy 1, policy_version 1757828 (0.0008) [2023-12-27 17:50:03,036][20381] Updated weights for policy 1, policy_version 1757838 (0.0009) [2023-12-27 17:50:03,081][20381] Updated weights for policy 1, policy_version 1757848 (0.0005) [2023-12-27 17:50:03,232][20347] Updated weights for policy 0, policy_version 1759329 (0.0008) [2023-12-27 17:50:03,284][20347] Updated weights for policy 0, policy_version 1759339 (0.0009) [2023-12-27 17:50:03,332][20347] Updated weights for policy 0, policy_version 1759349 (0.0009) [2023-12-27 17:50:03,383][20347] Updated weights for policy 0, policy_version 1759359 (0.0008) [2023-12-27 17:50:03,659][20381] Updated weights for policy 1, policy_version 1757858 (0.0006) [2023-12-27 17:50:03,716][20381] Updated weights for policy 1, policy_version 1757868 (0.0007) [2023-12-27 17:50:03,777][20381] Updated weights for policy 1, policy_version 1757878 (0.0005) [2023-12-27 17:50:03,834][20381] Updated weights for policy 1, policy_version 1757888 (0.0007) [2023-12-27 17:50:04,091][20347] Updated weights for policy 0, policy_version 1759369 (0.0007) [2023-12-27 17:50:04,146][20347] Updated weights for policy 0, policy_version 1759379 (0.0008) [2023-12-27 17:50:04,199][20347] Updated weights for policy 0, policy_version 1759389 (0.0007) [2023-12-27 17:50:04,499][20381] Updated weights for policy 1, policy_version 1757898 (0.0010) [2023-12-27 17:50:04,564][20381] Updated weights for policy 1, policy_version 1757908 (0.0009) [2023-12-27 17:50:04,615][20381] Updated weights for policy 1, policy_version 1757918 (0.0008) [2023-12-27 17:50:04,905][20347] Updated weights for policy 0, policy_version 1759399 (0.0009) [2023-12-27 17:50:04,951][20347] Updated weights for policy 0, policy_version 1759409 (0.0009) [2023-12-27 17:50:05,003][20347] Updated weights for policy 0, policy_version 1759419 (0.0005) [2023-12-27 17:50:05,272][20381] Updated weights for policy 1, policy_version 1757928 (0.0010) [2023-12-27 17:50:05,333][20381] Updated weights for policy 1, policy_version 1757938 (0.0011) [2023-12-27 17:50:05,385][20381] Updated weights for policy 1, policy_version 1757948 (0.0011) [2023-12-27 17:50:05,607][20347] Updated weights for policy 0, policy_version 1759429 (0.0005) [2023-12-27 17:50:05,656][20347] Updated weights for policy 0, policy_version 1759439 (0.0005) [2023-12-27 17:50:05,712][20347] Updated weights for policy 0, policy_version 1759449 (0.0005) [2023-12-27 17:50:06,021][20381] Updated weights for policy 1, policy_version 1757958 (0.0007) [2023-12-27 17:50:06,083][20381] Updated weights for policy 1, policy_version 1757968 (0.0006) [2023-12-27 17:50:06,149][20381] Updated weights for policy 1, policy_version 1757978 (0.0011) [2023-12-27 17:50:06,329][20347] Updated weights for policy 0, policy_version 1759459 (0.0007) [2023-12-27 17:50:06,389][20347] Updated weights for policy 0, policy_version 1759469 (0.0010) [2023-12-27 17:50:06,450][20347] Updated weights for policy 0, policy_version 1759479 (0.0011) [2023-12-27 17:50:06,845][20381] Updated weights for policy 1, policy_version 1757988 (0.0010) [2023-12-27 17:50:06,894][20381] Updated weights for policy 1, policy_version 1757998 (0.0011) [2023-12-27 17:50:06,945][20381] Updated weights for policy 1, policy_version 1758008 (0.0009) [2023-12-27 17:50:07,164][20347] Updated weights for policy 0, policy_version 1759489 (0.0010) [2023-12-27 17:50:07,216][20347] Updated weights for policy 0, policy_version 1759499 (0.0009) [2023-12-27 17:50:07,280][20347] Updated weights for policy 0, policy_version 1759509 (0.0010) [2023-12-27 17:50:07,334][20347] Updated weights for policy 0, policy_version 1759519 (0.0010) [2023-12-27 17:50:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 900620288. Throughput: 0: 10253.5, 1: 10297.9. Samples: 900584888. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:07,363][19170] Avg episode reward: [(0, '902.360'), (1, '1000.000')] [2023-12-27 17:50:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001758016_450101248.pth... [2023-12-27 17:50:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001759520_450519040.pth... [2023-12-27 17:50:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001756832_449798144.pth [2023-12-27 17:50:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001758304_450207744.pth [2023-12-27 17:50:07,644][20381] Updated weights for policy 1, policy_version 1758018 (0.0010) [2023-12-27 17:50:07,702][20381] Updated weights for policy 1, policy_version 1758028 (0.0008) [2023-12-27 17:50:07,765][20381] Updated weights for policy 1, policy_version 1758038 (0.0009) [2023-12-27 17:50:07,824][20381] Updated weights for policy 1, policy_version 1758048 (0.0007) [2023-12-27 17:50:07,948][20347] Updated weights for policy 0, policy_version 1759529 (0.0006) [2023-12-27 17:50:08,000][20347] Updated weights for policy 0, policy_version 1759539 (0.0007) [2023-12-27 17:50:08,049][20347] Updated weights for policy 0, policy_version 1759549 (0.0008) [2023-12-27 17:50:08,481][20381] Updated weights for policy 1, policy_version 1758058 (0.0009) [2023-12-27 17:50:08,534][20381] Updated weights for policy 1, policy_version 1758068 (0.0009) [2023-12-27 17:50:08,594][20381] Updated weights for policy 1, policy_version 1758078 (0.0010) [2023-12-27 17:50:08,734][20347] Updated weights for policy 0, policy_version 1759559 (0.0006) [2023-12-27 17:50:08,779][20347] Updated weights for policy 0, policy_version 1759569 (0.0007) [2023-12-27 17:50:08,828][20347] Updated weights for policy 0, policy_version 1759579 (0.0008) [2023-12-27 17:50:09,303][20381] Updated weights for policy 1, policy_version 1758088 (0.0009) [2023-12-27 17:50:09,361][20381] Updated weights for policy 1, policy_version 1758098 (0.0007) [2023-12-27 17:50:09,429][20381] Updated weights for policy 1, policy_version 1758108 (0.0013) [2023-12-27 17:50:09,617][20347] Updated weights for policy 0, policy_version 1759589 (0.0006) [2023-12-27 17:50:09,674][20347] Updated weights for policy 0, policy_version 1759599 (0.0009) [2023-12-27 17:50:09,729][20347] Updated weights for policy 0, policy_version 1759609 (0.0007) [2023-12-27 17:50:10,056][20381] Updated weights for policy 1, policy_version 1758118 (0.0009) [2023-12-27 17:50:10,117][20381] Updated weights for policy 1, policy_version 1758128 (0.0007) [2023-12-27 17:50:10,182][20381] Updated weights for policy 1, policy_version 1758138 (0.0008) [2023-12-27 17:50:10,447][20347] Updated weights for policy 0, policy_version 1759619 (0.0007) [2023-12-27 17:50:10,498][20347] Updated weights for policy 0, policy_version 1759629 (0.0006) [2023-12-27 17:50:10,554][20347] Updated weights for policy 0, policy_version 1759639 (0.0008) [2023-12-27 17:50:10,845][20381] Updated weights for policy 1, policy_version 1758148 (0.0007) [2023-12-27 17:50:10,901][20381] Updated weights for policy 1, policy_version 1758158 (0.0011) [2023-12-27 17:50:10,953][20381] Updated weights for policy 1, policy_version 1758168 (0.0010) [2023-12-27 17:50:11,194][20347] Updated weights for policy 0, policy_version 1759649 (0.0006) [2023-12-27 17:50:11,254][20347] Updated weights for policy 0, policy_version 1759659 (0.0008) [2023-12-27 17:50:11,313][20347] Updated weights for policy 0, policy_version 1759669 (0.0006) [2023-12-27 17:50:11,382][20347] Updated weights for policy 0, policy_version 1759679 (0.0009) [2023-12-27 17:50:11,708][20381] Updated weights for policy 1, policy_version 1758178 (0.0010) [2023-12-27 17:50:11,772][20381] Updated weights for policy 1, policy_version 1758188 (0.0010) [2023-12-27 17:50:11,835][20381] Updated weights for policy 1, policy_version 1758198 (0.0005) [2023-12-27 17:50:11,895][20381] Updated weights for policy 1, policy_version 1758208 (0.0006) [2023-12-27 17:50:12,108][20347] Updated weights for policy 0, policy_version 1759689 (0.0008) [2023-12-27 17:50:12,158][20347] Updated weights for policy 0, policy_version 1759699 (0.0008) [2023-12-27 17:50:12,216][20347] Updated weights for policy 0, policy_version 1759709 (0.0007) [2023-12-27 17:50:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 900718592. Throughput: 0: 10262.3, 1: 10315.0. Samples: 900708548. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:12,363][19170] Avg episode reward: [(0, '912.000'), (1, '1000.000')] [2023-12-27 17:50:12,549][20381] Updated weights for policy 1, policy_version 1758218 (0.0009) [2023-12-27 17:50:12,613][20381] Updated weights for policy 1, policy_version 1758228 (0.0008) [2023-12-27 17:50:12,673][20381] Updated weights for policy 1, policy_version 1758238 (0.0009) [2023-12-27 17:50:12,945][20347] Updated weights for policy 0, policy_version 1759719 (0.0006) [2023-12-27 17:50:13,009][20347] Updated weights for policy 0, policy_version 1759729 (0.0007) [2023-12-27 17:50:13,075][20347] Updated weights for policy 0, policy_version 1759739 (0.0008) [2023-12-27 17:50:13,338][20381] Updated weights for policy 1, policy_version 1758248 (0.0010) [2023-12-27 17:50:13,391][20381] Updated weights for policy 1, policy_version 1758258 (0.0010) [2023-12-27 17:50:13,447][20381] Updated weights for policy 1, policy_version 1758268 (0.0010) [2023-12-27 17:50:13,747][20347] Updated weights for policy 0, policy_version 1759749 (0.0008) [2023-12-27 17:50:13,806][20347] Updated weights for policy 0, policy_version 1759759 (0.0007) [2023-12-27 17:50:13,868][20347] Updated weights for policy 0, policy_version 1759769 (0.0007) [2023-12-27 17:50:14,091][20381] Updated weights for policy 1, policy_version 1758278 (0.0010) [2023-12-27 17:50:14,139][20381] Updated weights for policy 1, policy_version 1758288 (0.0010) [2023-12-27 17:50:14,194][20381] Updated weights for policy 1, policy_version 1758298 (0.0010) [2023-12-27 17:50:14,559][20347] Updated weights for policy 0, policy_version 1759779 (0.0008) [2023-12-27 17:50:14,613][20347] Updated weights for policy 0, policy_version 1759789 (0.0008) [2023-12-27 17:50:14,671][20347] Updated weights for policy 0, policy_version 1759799 (0.0007) [2023-12-27 17:50:14,888][20381] Updated weights for policy 1, policy_version 1758308 (0.0009) [2023-12-27 17:50:14,954][20381] Updated weights for policy 1, policy_version 1758318 (0.0007) [2023-12-27 17:50:15,017][20381] Updated weights for policy 1, policy_version 1758328 (0.0006) [2023-12-27 17:50:15,451][20347] Updated weights for policy 0, policy_version 1759809 (0.0007) [2023-12-27 17:50:15,509][20347] Updated weights for policy 0, policy_version 1759819 (0.0007) [2023-12-27 17:50:15,554][20347] Updated weights for policy 0, policy_version 1759829 (0.0007) [2023-12-27 17:50:15,606][20347] Updated weights for policy 0, policy_version 1759839 (0.0008) [2023-12-27 17:50:15,699][20381] Updated weights for policy 1, policy_version 1758338 (0.0008) [2023-12-27 17:50:15,748][20381] Updated weights for policy 1, policy_version 1758348 (0.0008) [2023-12-27 17:50:15,802][20381] Updated weights for policy 1, policy_version 1758358 (0.0007) [2023-12-27 17:50:15,861][20381] Updated weights for policy 1, policy_version 1758368 (0.0008) [2023-12-27 17:50:16,250][20347] Updated weights for policy 0, policy_version 1759849 (0.0008) [2023-12-27 17:50:16,296][20347] Updated weights for policy 0, policy_version 1759859 (0.0008) [2023-12-27 17:50:16,341][20347] Updated weights for policy 0, policy_version 1759869 (0.0007) [2023-12-27 17:50:16,585][20381] Updated weights for policy 1, policy_version 1758378 (0.0007) [2023-12-27 17:50:16,642][20381] Updated weights for policy 1, policy_version 1758388 (0.0009) [2023-12-27 17:50:16,691][20381] Updated weights for policy 1, policy_version 1758398 (0.0010) [2023-12-27 17:50:17,013][20347] Updated weights for policy 0, policy_version 1759879 (0.0008) [2023-12-27 17:50:17,064][20347] Updated weights for policy 0, policy_version 1759889 (0.0007) [2023-12-27 17:50:17,114][20347] Updated weights for policy 0, policy_version 1759899 (0.0007) [2023-12-27 17:50:17,324][20381] Updated weights for policy 1, policy_version 1758408 (0.0010) [2023-12-27 17:50:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 900816896. Throughput: 0: 10227.0, 1: 10275.7. Samples: 900829824. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:17,364][19170] Avg episode reward: [(0, '823.950'), (1, '1000.000')] [2023-12-27 17:50:17,373][20381] Updated weights for policy 1, policy_version 1758418 (0.0008) [2023-12-27 17:50:17,429][20381] Updated weights for policy 1, policy_version 1758428 (0.0005) [2023-12-27 17:50:17,792][20347] Updated weights for policy 0, policy_version 1759909 (0.0009) [2023-12-27 17:50:17,837][20347] Updated weights for policy 0, policy_version 1759919 (0.0010) [2023-12-27 17:50:17,888][20347] Updated weights for policy 0, policy_version 1759929 (0.0010) [2023-12-27 17:50:18,007][20381] Updated weights for policy 1, policy_version 1758438 (0.0007) [2023-12-27 17:50:18,063][20381] Updated weights for policy 1, policy_version 1758448 (0.0010) [2023-12-27 17:50:18,123][20381] Updated weights for policy 1, policy_version 1758458 (0.0010) [2023-12-27 17:50:18,535][20347] Updated weights for policy 0, policy_version 1759939 (0.0009) [2023-12-27 17:50:18,563][20291] KL-divergence is very high: 102.4355 [2023-12-27 17:50:18,582][20291] KL-divergence is very high: 118.1465 [2023-12-27 17:50:18,583][20347] Updated weights for policy 0, policy_version 1759949 (0.0007) [2023-12-27 17:50:18,586][20291] KL-divergence is very high: 112.0798 [2023-12-27 17:50:18,628][20347] Updated weights for policy 0, policy_version 1759959 (0.0007) [2023-12-27 17:50:18,782][20381] Updated weights for policy 1, policy_version 1758468 (0.0010) [2023-12-27 17:50:18,842][20381] Updated weights for policy 1, policy_version 1758478 (0.0011) [2023-12-27 17:50:18,897][20381] Updated weights for policy 1, policy_version 1758488 (0.0011) [2023-12-27 17:50:19,341][20347] Updated weights for policy 0, policy_version 1759969 (0.0007) [2023-12-27 17:50:19,404][20347] Updated weights for policy 0, policy_version 1759979 (0.0012) [2023-12-27 17:50:19,461][20347] Updated weights for policy 0, policy_version 1759989 (0.0006) [2023-12-27 17:50:19,526][20347] Updated weights for policy 0, policy_version 1759999 (0.0008) [2023-12-27 17:50:19,627][20381] Updated weights for policy 1, policy_version 1758498 (0.0010) [2023-12-27 17:50:19,684][20381] Updated weights for policy 1, policy_version 1758508 (0.0006) [2023-12-27 17:50:19,746][20381] Updated weights for policy 1, policy_version 1758518 (0.0006) [2023-12-27 17:50:19,805][20381] Updated weights for policy 1, policy_version 1758528 (0.0005) [2023-12-27 17:50:20,295][20347] Updated weights for policy 0, policy_version 1760009 (0.0006) [2023-12-27 17:50:20,358][20347] Updated weights for policy 0, policy_version 1760019 (0.0007) [2023-12-27 17:50:20,416][20347] Updated weights for policy 0, policy_version 1760029 (0.0008) [2023-12-27 17:50:20,461][20381] Updated weights for policy 1, policy_version 1758538 (0.0008) [2023-12-27 17:50:20,524][20381] Updated weights for policy 1, policy_version 1758548 (0.0009) [2023-12-27 17:50:20,588][20381] Updated weights for policy 1, policy_version 1758558 (0.0008) [2023-12-27 17:50:21,102][20347] Updated weights for policy 0, policy_version 1760039 (0.0009) [2023-12-27 17:50:21,164][20347] Updated weights for policy 0, policy_version 1760049 (0.0009) [2023-12-27 17:50:21,214][20291] KL-divergence is very high: 115.8211 [2023-12-27 17:50:21,227][20347] Updated weights for policy 0, policy_version 1760059 (0.0009) [2023-12-27 17:50:21,286][20381] Updated weights for policy 1, policy_version 1758568 (0.0008) [2023-12-27 17:50:21,354][20381] Updated weights for policy 1, policy_version 1758578 (0.0010) [2023-12-27 17:50:21,415][20381] Updated weights for policy 1, policy_version 1758588 (0.0010) [2023-12-27 17:50:21,991][20347] Updated weights for policy 0, policy_version 1760069 (0.0009) [2023-12-27 17:50:22,050][20347] Updated weights for policy 0, policy_version 1760079 (0.0011) [2023-12-27 17:50:22,116][20347] Updated weights for policy 0, policy_version 1760089 (0.0010) [2023-12-27 17:50:22,141][20381] Updated weights for policy 1, policy_version 1758598 (0.0011) [2023-12-27 17:50:22,204][20381] Updated weights for policy 1, policy_version 1758608 (0.0011) [2023-12-27 17:50:22,254][20381] Updated weights for policy 1, policy_version 1758618 (0.0010) [2023-12-27 17:50:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 900923392. Throughput: 0: 10217.7, 1: 10297.0. Samples: 900891792. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:22,363][19170] Avg episode reward: [(0, '872.950'), (1, '1000.000')] [2023-12-27 17:50:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001758624_450256896.pth... [2023-12-27 17:50:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001760096_450666496.pth... [2023-12-27 17:50:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001757408_449945600.pth [2023-12-27 17:50:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001758912_450363392.pth [2023-12-27 17:50:22,777][20347] Updated weights for policy 0, policy_version 1760099 (0.0009) [2023-12-27 17:50:22,831][20347] Updated weights for policy 0, policy_version 1760109 (0.0007) [2023-12-27 17:50:22,893][20347] Updated weights for policy 0, policy_version 1760119 (0.0005) [2023-12-27 17:50:22,998][20381] Updated weights for policy 1, policy_version 1758628 (0.0009) [2023-12-27 17:50:23,061][20381] Updated weights for policy 1, policy_version 1758638 (0.0008) [2023-12-27 17:50:23,119][20381] Updated weights for policy 1, policy_version 1758648 (0.0007) [2023-12-27 17:50:23,505][20347] Updated weights for policy 0, policy_version 1760129 (0.0005) [2023-12-27 17:50:23,565][20347] Updated weights for policy 0, policy_version 1760139 (0.0008) [2023-12-27 17:50:23,626][20347] Updated weights for policy 0, policy_version 1760149 (0.0008) [2023-12-27 17:50:23,685][20347] Updated weights for policy 0, policy_version 1760159 (0.0010) [2023-12-27 17:50:23,740][20381] Updated weights for policy 1, policy_version 1758658 (0.0006) [2023-12-27 17:50:23,805][20381] Updated weights for policy 1, policy_version 1758668 (0.0006) [2023-12-27 17:50:23,864][20381] Updated weights for policy 1, policy_version 1758678 (0.0007) [2023-12-27 17:50:23,926][20381] Updated weights for policy 1, policy_version 1758688 (0.0011) [2023-12-27 17:50:24,311][20347] Updated weights for policy 0, policy_version 1760169 (0.0007) [2023-12-27 17:50:24,354][20347] Updated weights for policy 0, policy_version 1760179 (0.0007) [2023-12-27 17:50:24,407][20347] Updated weights for policy 0, policy_version 1760189 (0.0008) [2023-12-27 17:50:24,595][20381] Updated weights for policy 1, policy_version 1758698 (0.0011) [2023-12-27 17:50:24,654][20381] Updated weights for policy 1, policy_version 1758708 (0.0011) [2023-12-27 17:50:24,714][20381] Updated weights for policy 1, policy_version 1758718 (0.0010) [2023-12-27 17:50:25,082][20347] Updated weights for policy 0, policy_version 1760199 (0.0007) [2023-12-27 17:50:25,128][20347] Updated weights for policy 0, policy_version 1760209 (0.0008) [2023-12-27 17:50:25,180][20347] Updated weights for policy 0, policy_version 1760219 (0.0008) [2023-12-27 17:50:25,266][20381] Updated weights for policy 1, policy_version 1758728 (0.0011) [2023-12-27 17:50:25,321][20381] Updated weights for policy 1, policy_version 1758738 (0.0011) [2023-12-27 17:50:25,379][20381] Updated weights for policy 1, policy_version 1758748 (0.0011) [2023-12-27 17:50:25,856][20347] Updated weights for policy 0, policy_version 1760229 (0.0006) [2023-12-27 17:50:25,915][20347] Updated weights for policy 0, policy_version 1760239 (0.0006) [2023-12-27 17:50:25,964][20347] Updated weights for policy 0, policy_version 1760249 (0.0005) [2023-12-27 17:50:26,011][20381] Updated weights for policy 1, policy_version 1758758 (0.0009) [2023-12-27 17:50:26,072][20381] Updated weights for policy 1, policy_version 1758768 (0.0009) [2023-12-27 17:50:26,130][20381] Updated weights for policy 1, policy_version 1758778 (0.0011) [2023-12-27 17:50:26,566][20347] Updated weights for policy 0, policy_version 1760259 (0.0008) [2023-12-27 17:50:26,613][20347] Updated weights for policy 0, policy_version 1760269 (0.0010) [2023-12-27 17:50:26,665][20347] Updated weights for policy 0, policy_version 1760279 (0.0010) [2023-12-27 17:50:26,719][20381] Updated weights for policy 1, policy_version 1758788 (0.0008) [2023-12-27 17:50:26,770][20381] Updated weights for policy 1, policy_version 1758798 (0.0008) [2023-12-27 17:50:26,823][20381] Updated weights for policy 1, policy_version 1758808 (0.0007) [2023-12-27 17:50:27,291][20347] Updated weights for policy 0, policy_version 1760289 (0.0010) [2023-12-27 17:50:27,341][20381] Updated weights for policy 1, policy_version 1758818 (0.0006) [2023-12-27 17:50:27,346][20347] Updated weights for policy 0, policy_version 1760299 (0.0009) [2023-12-27 17:50:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 901021696. Throughput: 0: 10297.7, 1: 10223.8. Samples: 901014796. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:27,363][19170] Avg episode reward: [(0, '863.760'), (1, '1000.000')] [2023-12-27 17:50:27,400][20381] Updated weights for policy 1, policy_version 1758828 (0.0009) [2023-12-27 17:50:27,403][20347] Updated weights for policy 0, policy_version 1760310 (0.0009) [2023-12-27 17:50:27,458][20381] Updated weights for policy 1, policy_version 1758838 (0.0007) [2023-12-27 17:50:27,459][20347] Updated weights for policy 0, policy_version 1760320 (0.0006) [2023-12-27 17:50:27,516][20381] Updated weights for policy 1, policy_version 1758848 (0.0009) [2023-12-27 17:50:28,071][20347] Updated weights for policy 0, policy_version 1760330 (0.0007) [2023-12-27 17:50:28,126][20347] Updated weights for policy 0, policy_version 1760340 (0.0007) [2023-12-27 17:50:28,178][20347] Updated weights for policy 0, policy_version 1760350 (0.0006) [2023-12-27 17:50:28,205][20381] Updated weights for policy 1, policy_version 1758858 (0.0007) [2023-12-27 17:50:28,260][20381] Updated weights for policy 1, policy_version 1758868 (0.0008) [2023-12-27 17:50:28,315][20381] Updated weights for policy 1, policy_version 1758878 (0.0007) [2023-12-27 17:50:28,834][20347] Updated weights for policy 0, policy_version 1760360 (0.0006) [2023-12-27 17:50:28,891][20347] Updated weights for policy 0, policy_version 1760370 (0.0008) [2023-12-27 17:50:28,948][20347] Updated weights for policy 0, policy_version 1760380 (0.0007) [2023-12-27 17:50:29,019][20381] Updated weights for policy 1, policy_version 1758888 (0.0008) [2023-12-27 17:50:29,063][20381] Updated weights for policy 1, policy_version 1758898 (0.0008) [2023-12-27 17:50:29,112][20381] Updated weights for policy 1, policy_version 1758908 (0.0007) [2023-12-27 17:50:29,547][20347] Updated weights for policy 0, policy_version 1760390 (0.0009) [2023-12-27 17:50:29,611][20347] Updated weights for policy 0, policy_version 1760400 (0.0010) [2023-12-27 17:50:29,670][20347] Updated weights for policy 0, policy_version 1760410 (0.0011) [2023-12-27 17:50:29,836][20381] Updated weights for policy 1, policy_version 1758918 (0.0008) [2023-12-27 17:50:29,888][20381] Updated weights for policy 1, policy_version 1758928 (0.0008) [2023-12-27 17:50:29,948][20381] Updated weights for policy 1, policy_version 1758938 (0.0009) [2023-12-27 17:50:30,360][20347] Updated weights for policy 0, policy_version 1760420 (0.0008) [2023-12-27 17:50:30,417][20347] Updated weights for policy 0, policy_version 1760430 (0.0008) [2023-12-27 17:50:30,476][20347] Updated weights for policy 0, policy_version 1760440 (0.0008) [2023-12-27 17:50:30,663][20381] Updated weights for policy 1, policy_version 1758948 (0.0007) [2023-12-27 17:50:30,724][20381] Updated weights for policy 1, policy_version 1758958 (0.0007) [2023-12-27 17:50:30,777][20381] Updated weights for policy 1, policy_version 1758968 (0.0007) [2023-12-27 17:50:31,172][20347] Updated weights for policy 0, policy_version 1760450 (0.0007) [2023-12-27 17:50:31,234][20347] Updated weights for policy 0, policy_version 1760460 (0.0007) [2023-12-27 17:50:31,291][20347] Updated weights for policy 0, policy_version 1760470 (0.0008) [2023-12-27 17:50:31,352][20347] Updated weights for policy 0, policy_version 1760480 (0.0007) [2023-12-27 17:50:31,461][20381] Updated weights for policy 1, policy_version 1758978 (0.0006) [2023-12-27 17:50:31,516][20381] Updated weights for policy 1, policy_version 1758988 (0.0010) [2023-12-27 17:50:31,578][20381] Updated weights for policy 1, policy_version 1758998 (0.0010) [2023-12-27 17:50:31,648][20381] Updated weights for policy 1, policy_version 1759008 (0.0009) [2023-12-27 17:50:32,032][20347] Updated weights for policy 0, policy_version 1760490 (0.0007) [2023-12-27 17:50:32,083][20347] Updated weights for policy 0, policy_version 1760500 (0.0007) [2023-12-27 17:50:32,140][20347] Updated weights for policy 0, policy_version 1760510 (0.0007) [2023-12-27 17:50:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 901128192. Throughput: 0: 10321.6, 1: 10256.4. Samples: 901140872. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:32,363][19170] Avg episode reward: [(0, '833.390'), (1, '1000.000')] [2023-12-27 17:50:32,367][20381] Updated weights for policy 1, policy_version 1759018 (0.0007) [2023-12-27 17:50:32,425][20381] Updated weights for policy 1, policy_version 1759028 (0.0008) [2023-12-27 17:50:32,474][20381] Updated weights for policy 1, policy_version 1759038 (0.0008) [2023-12-27 17:50:32,855][20347] Updated weights for policy 0, policy_version 1760520 (0.0010) [2023-12-27 17:50:32,908][20347] Updated weights for policy 0, policy_version 1760530 (0.0010) [2023-12-27 17:50:32,960][20347] Updated weights for policy 0, policy_version 1760540 (0.0007) [2023-12-27 17:50:33,172][20381] Updated weights for policy 1, policy_version 1759048 (0.0006) [2023-12-27 17:50:33,224][20381] Updated weights for policy 1, policy_version 1759058 (0.0007) [2023-12-27 17:50:33,279][20381] Updated weights for policy 1, policy_version 1759068 (0.0008) [2023-12-27 17:50:33,545][20347] Updated weights for policy 0, policy_version 1760550 (0.0005) [2023-12-27 17:50:33,611][20347] Updated weights for policy 0, policy_version 1760560 (0.0005) [2023-12-27 17:50:33,678][20347] Updated weights for policy 0, policy_version 1760570 (0.0005) [2023-12-27 17:50:33,892][20381] Updated weights for policy 1, policy_version 1759078 (0.0008) [2023-12-27 17:50:33,941][20381] Updated weights for policy 1, policy_version 1759088 (0.0009) [2023-12-27 17:50:33,993][20381] Updated weights for policy 1, policy_version 1759098 (0.0009) [2023-12-27 17:50:34,228][20347] Updated weights for policy 0, policy_version 1760580 (0.0007) [2023-12-27 17:50:34,285][20347] Updated weights for policy 0, policy_version 1760590 (0.0006) [2023-12-27 17:50:34,353][20347] Updated weights for policy 0, policy_version 1760600 (0.0010) [2023-12-27 17:50:34,703][20381] Updated weights for policy 1, policy_version 1759108 (0.0007) [2023-12-27 17:50:34,755][20381] Updated weights for policy 1, policy_version 1759118 (0.0007) [2023-12-27 17:50:34,813][20381] Updated weights for policy 1, policy_version 1759128 (0.0006) [2023-12-27 17:50:34,956][20347] Updated weights for policy 0, policy_version 1760610 (0.0007) [2023-12-27 17:50:35,008][20347] Updated weights for policy 0, policy_version 1760620 (0.0010) [2023-12-27 17:50:35,059][20347] Updated weights for policy 0, policy_version 1760630 (0.0010) [2023-12-27 17:50:35,076][20291] KL-divergence is very high: 159.5624 [2023-12-27 17:50:35,082][20291] KL-divergence is very high: 123.1827 [2023-12-27 17:50:35,087][20291] KL-divergence is very high: 142.8774 [2023-12-27 17:50:35,092][20291] KL-divergence is very high: 143.3219 [2023-12-27 17:50:35,106][20291] KL-divergence is very high: 159.6130 [2023-12-27 17:50:35,110][20347] Updated weights for policy 0, policy_version 1760640 (0.0010) [2023-12-27 17:50:35,493][20381] Updated weights for policy 1, policy_version 1759138 (0.0007) [2023-12-27 17:50:35,555][20381] Updated weights for policy 1, policy_version 1759148 (0.0006) [2023-12-27 17:50:35,612][20381] Updated weights for policy 1, policy_version 1759158 (0.0007) [2023-12-27 17:50:35,672][20381] Updated weights for policy 1, policy_version 1759168 (0.0008) [2023-12-27 17:50:35,753][20347] Updated weights for policy 0, policy_version 1760650 (0.0006) [2023-12-27 17:50:35,804][20347] Updated weights for policy 0, policy_version 1760660 (0.0010) [2023-12-27 17:50:35,855][20347] Updated weights for policy 0, policy_version 1760670 (0.0009) [2023-12-27 17:50:36,350][20381] Updated weights for policy 1, policy_version 1759178 (0.0007) [2023-12-27 17:50:36,404][20381] Updated weights for policy 1, policy_version 1759188 (0.0007) [2023-12-27 17:50:36,458][20381] Updated weights for policy 1, policy_version 1759198 (0.0008) [2023-12-27 17:50:36,589][20347] Updated weights for policy 0, policy_version 1760680 (0.0007) [2023-12-27 17:50:36,651][20347] Updated weights for policy 0, policy_version 1760690 (0.0007) [2023-12-27 17:50:36,717][20347] Updated weights for policy 0, policy_version 1760700 (0.0008) [2023-12-27 17:50:37,185][20381] Updated weights for policy 1, policy_version 1759208 (0.0007) [2023-12-27 17:50:37,231][20381] Updated weights for policy 1, policy_version 1759218 (0.0008) [2023-12-27 17:50:37,279][20381] Updated weights for policy 1, policy_version 1759228 (0.0009) [2023-12-27 17:50:37,321][20347] Updated weights for policy 0, policy_version 1760710 (0.0007) [2023-12-27 17:50:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 901234688. Throughput: 0: 10358.0, 1: 10303.2. Samples: 901204616. Policy #0 lag: (min: 12.0, avg: 18.9, max: 44.0) [2023-12-27 17:50:37,363][19170] Avg episode reward: [(0, '855.490'), (1, '1000.000')] [2023-12-27 17:50:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001759232_450412544.pth... [2023-12-27 17:50:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001758016_450101248.pth [2023-12-27 17:50:37,383][20347] Updated weights for policy 0, policy_version 1760720 (0.0005) [2023-12-27 17:50:37,435][20347] Updated weights for policy 0, policy_version 1760730 (0.0005) [2023-12-27 17:50:37,462][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001760736_450830336.pth... [2023-12-27 17:50:37,465][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001759520_450519040.pth [2023-12-27 17:50:38,033][20381] Updated weights for policy 1, policy_version 1759238 (0.0007) [2023-12-27 17:50:38,082][20347] Updated weights for policy 0, policy_version 1760740 (0.0006) [2023-12-27 17:50:38,089][20381] Updated weights for policy 1, policy_version 1759248 (0.0008) [2023-12-27 17:50:38,142][20347] Updated weights for policy 0, policy_version 1760750 (0.0005) [2023-12-27 17:50:38,148][20381] Updated weights for policy 1, policy_version 1759258 (0.0008) [2023-12-27 17:50:38,193][20347] Updated weights for policy 0, policy_version 1760760 (0.0005) [2023-12-27 17:50:38,744][20347] Updated weights for policy 0, policy_version 1760770 (0.0005) [2023-12-27 17:50:38,798][20347] Updated weights for policy 0, policy_version 1760780 (0.0006) [2023-12-27 17:50:38,857][20347] Updated weights for policy 0, policy_version 1760790 (0.0008) [2023-12-27 17:50:38,900][20381] Updated weights for policy 1, policy_version 1759268 (0.0008) [2023-12-27 17:50:38,914][20347] Updated weights for policy 0, policy_version 1760800 (0.0006) [2023-12-27 17:50:38,949][20381] Updated weights for policy 1, policy_version 1759278 (0.0007) [2023-12-27 17:50:38,995][20381] Updated weights for policy 1, policy_version 1759288 (0.0007) [2023-12-27 17:50:39,651][20347] Updated weights for policy 0, policy_version 1760810 (0.0005) [2023-12-27 17:50:39,698][20381] Updated weights for policy 1, policy_version 1759298 (0.0007) [2023-12-27 17:50:39,713][20347] Updated weights for policy 0, policy_version 1760820 (0.0005) [2023-12-27 17:50:39,759][20381] Updated weights for policy 1, policy_version 1759308 (0.0008) [2023-12-27 17:50:39,774][20347] Updated weights for policy 0, policy_version 1760830 (0.0006) [2023-12-27 17:50:39,820][20381] Updated weights for policy 1, policy_version 1759318 (0.0008) [2023-12-27 17:50:39,884][20381] Updated weights for policy 1, policy_version 1759328 (0.0006) [2023-12-27 17:50:40,403][20347] Updated weights for policy 0, policy_version 1760840 (0.0007) [2023-12-27 17:50:40,464][20347] Updated weights for policy 0, policy_version 1760850 (0.0005) [2023-12-27 17:50:40,526][20347] Updated weights for policy 0, policy_version 1760860 (0.0006) [2023-12-27 17:50:40,531][20381] Updated weights for policy 1, policy_version 1759338 (0.0007) [2023-12-27 17:50:40,588][20381] Updated weights for policy 1, policy_version 1759348 (0.0007) [2023-12-27 17:50:40,635][20381] Updated weights for policy 1, policy_version 1759358 (0.0007) [2023-12-27 17:50:41,188][20347] Updated weights for policy 0, policy_version 1760870 (0.0006) [2023-12-27 17:50:41,240][20347] Updated weights for policy 0, policy_version 1760880 (0.0007) [2023-12-27 17:50:41,300][20347] Updated weights for policy 0, policy_version 1760890 (0.0007) [2023-12-27 17:50:41,303][20381] Updated weights for policy 1, policy_version 1759368 (0.0007) [2023-12-27 17:50:41,369][20381] Updated weights for policy 1, policy_version 1759378 (0.0008) [2023-12-27 17:50:41,432][20381] Updated weights for policy 1, policy_version 1759388 (0.0007) [2023-12-27 17:50:42,024][20347] Updated weights for policy 0, policy_version 1760900 (0.0008) [2023-12-27 17:50:42,079][20347] Updated weights for policy 0, policy_version 1760910 (0.0007) [2023-12-27 17:50:42,137][20347] Updated weights for policy 0, policy_version 1760920 (0.0007) [2023-12-27 17:50:42,157][20381] Updated weights for policy 1, policy_version 1759398 (0.0008) [2023-12-27 17:50:42,215][20381] Updated weights for policy 1, policy_version 1759408 (0.0008) [2023-12-27 17:50:42,277][20381] Updated weights for policy 1, policy_version 1759418 (0.0009) [2023-12-27 17:50:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 901341184. Throughput: 0: 10357.4, 1: 10262.2. Samples: 901327020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:50:42,363][19170] Avg episode reward: [(0, '841.020'), (1, '1000.000')] [2023-12-27 17:50:42,745][20347] Updated weights for policy 0, policy_version 1760930 (0.0005) [2023-12-27 17:50:42,797][20347] Updated weights for policy 0, policy_version 1760940 (0.0006) [2023-12-27 17:50:42,854][20347] Updated weights for policy 0, policy_version 1760950 (0.0006) [2023-12-27 17:50:42,900][20347] Updated weights for policy 0, policy_version 1760960 (0.0007) [2023-12-27 17:50:42,973][20381] Updated weights for policy 1, policy_version 1759428 (0.0007) [2023-12-27 17:50:43,030][20381] Updated weights for policy 1, policy_version 1759438 (0.0008) [2023-12-27 17:50:43,079][20381] Updated weights for policy 1, policy_version 1759448 (0.0008) [2023-12-27 17:50:43,529][20347] Updated weights for policy 0, policy_version 1760970 (0.0006) [2023-12-27 17:50:43,591][20347] Updated weights for policy 0, policy_version 1760980 (0.0006) [2023-12-27 17:50:43,648][20347] Updated weights for policy 0, policy_version 1760990 (0.0007) [2023-12-27 17:50:43,816][20381] Updated weights for policy 1, policy_version 1759458 (0.0007) [2023-12-27 17:50:43,870][20381] Updated weights for policy 1, policy_version 1759468 (0.0008) [2023-12-27 17:50:43,927][20381] Updated weights for policy 1, policy_version 1759478 (0.0008) [2023-12-27 17:50:43,982][20381] Updated weights for policy 1, policy_version 1759488 (0.0008) [2023-12-27 17:50:44,253][20347] Updated weights for policy 0, policy_version 1761000 (0.0007) [2023-12-27 17:50:44,305][20347] Updated weights for policy 0, policy_version 1761010 (0.0008) [2023-12-27 17:50:44,359][20347] Updated weights for policy 0, policy_version 1761020 (0.0007) [2023-12-27 17:50:44,672][20381] Updated weights for policy 1, policy_version 1759498 (0.0008) [2023-12-27 17:50:44,731][20381] Updated weights for policy 1, policy_version 1759508 (0.0008) [2023-12-27 17:50:44,790][20381] Updated weights for policy 1, policy_version 1759518 (0.0008) [2023-12-27 17:50:45,033][20347] Updated weights for policy 0, policy_version 1761030 (0.0007) [2023-12-27 17:50:45,094][20347] Updated weights for policy 0, policy_version 1761040 (0.0007) [2023-12-27 17:50:45,163][20347] Updated weights for policy 0, policy_version 1761050 (0.0007) [2023-12-27 17:50:45,539][20381] Updated weights for policy 1, policy_version 1759528 (0.0010) [2023-12-27 17:50:45,594][20381] Updated weights for policy 1, policy_version 1759538 (0.0011) [2023-12-27 17:50:45,650][20381] Updated weights for policy 1, policy_version 1759548 (0.0011) [2023-12-27 17:50:45,853][20347] Updated weights for policy 0, policy_version 1761060 (0.0007) [2023-12-27 17:50:45,919][20347] Updated weights for policy 0, policy_version 1761070 (0.0006) [2023-12-27 17:50:45,968][20347] Updated weights for policy 0, policy_version 1761080 (0.0006) [2023-12-27 17:50:46,262][20381] Updated weights for policy 1, policy_version 1759558 (0.0007) [2023-12-27 17:50:46,314][20381] Updated weights for policy 1, policy_version 1759568 (0.0009) [2023-12-27 17:50:46,371][20381] Updated weights for policy 1, policy_version 1759578 (0.0009) [2023-12-27 17:50:46,575][20347] Updated weights for policy 0, policy_version 1761090 (0.0008) [2023-12-27 17:50:46,633][20347] Updated weights for policy 0, policy_version 1761100 (0.0007) [2023-12-27 17:50:46,676][20347] Updated weights for policy 0, policy_version 1761110 (0.0007) [2023-12-27 17:50:46,731][20347] Updated weights for policy 0, policy_version 1761120 (0.0006) [2023-12-27 17:50:47,067][20381] Updated weights for policy 1, policy_version 1759588 (0.0008) [2023-12-27 17:50:47,111][20381] Updated weights for policy 1, policy_version 1759598 (0.0005) [2023-12-27 17:50:47,167][20381] Updated weights for policy 1, policy_version 1759608 (0.0005) [2023-12-27 17:50:47,339][20347] Updated weights for policy 0, policy_version 1761130 (0.0006) [2023-12-27 17:50:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 901439488. Throughput: 0: 10397.8, 1: 10214.8. Samples: 901450336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:50:47,363][19170] Avg episode reward: [(0, '814.660'), (1, '1000.000')] [2023-12-27 17:50:47,392][20347] Updated weights for policy 0, policy_version 1761140 (0.0005) [2023-12-27 17:50:47,441][20347] Updated weights for policy 0, policy_version 1761150 (0.0006) [2023-12-27 17:50:47,702][20381] Updated weights for policy 1, policy_version 1759618 (0.0005) [2023-12-27 17:50:47,770][20381] Updated weights for policy 1, policy_version 1759628 (0.0006) [2023-12-27 17:50:47,829][20381] Updated weights for policy 1, policy_version 1759638 (0.0010) [2023-12-27 17:50:47,886][20381] Updated weights for policy 1, policy_version 1759648 (0.0009) [2023-12-27 17:50:48,115][20347] Updated weights for policy 0, policy_version 1761160 (0.0006) [2023-12-27 17:50:48,165][20347] Updated weights for policy 0, policy_version 1761170 (0.0007) [2023-12-27 17:50:48,228][20347] Updated weights for policy 0, policy_version 1761180 (0.0008) [2023-12-27 17:50:48,483][20381] Updated weights for policy 1, policy_version 1759658 (0.0007) [2023-12-27 17:50:48,540][20381] Updated weights for policy 1, policy_version 1759668 (0.0007) [2023-12-27 17:50:48,596][20381] Updated weights for policy 1, policy_version 1759678 (0.0008) [2023-12-27 17:50:48,954][20347] Updated weights for policy 0, policy_version 1761190 (0.0007) [2023-12-27 17:50:49,003][20347] Updated weights for policy 0, policy_version 1761200 (0.0008) [2023-12-27 17:50:49,054][20347] Updated weights for policy 0, policy_version 1761210 (0.0007) [2023-12-27 17:50:49,287][20381] Updated weights for policy 1, policy_version 1759688 (0.0008) [2023-12-27 17:50:49,346][20381] Updated weights for policy 1, policy_version 1759698 (0.0009) [2023-12-27 17:50:49,414][20381] Updated weights for policy 1, policy_version 1759708 (0.0008) [2023-12-27 17:50:49,824][20347] Updated weights for policy 0, policy_version 1761220 (0.0007) [2023-12-27 17:50:49,893][20347] Updated weights for policy 0, policy_version 1761230 (0.0007) [2023-12-27 17:50:49,957][20347] Updated weights for policy 0, policy_version 1761240 (0.0008) [2023-12-27 17:50:50,152][20381] Updated weights for policy 1, policy_version 1759718 (0.0008) [2023-12-27 17:50:50,200][20381] Updated weights for policy 1, policy_version 1759728 (0.0008) [2023-12-27 17:50:50,252][20381] Updated weights for policy 1, policy_version 1759738 (0.0007) [2023-12-27 17:50:50,621][20347] Updated weights for policy 0, policy_version 1761250 (0.0008) [2023-12-27 17:50:50,672][20347] Updated weights for policy 0, policy_version 1761260 (0.0008) [2023-12-27 17:50:50,732][20347] Updated weights for policy 0, policy_version 1761270 (0.0008) [2023-12-27 17:50:50,792][20347] Updated weights for policy 0, policy_version 1761280 (0.0007) [2023-12-27 17:50:50,975][20381] Updated weights for policy 1, policy_version 1759748 (0.0008) [2023-12-27 17:50:51,039][20381] Updated weights for policy 1, policy_version 1759758 (0.0010) [2023-12-27 17:50:51,094][20381] Updated weights for policy 1, policy_version 1759768 (0.0008) [2023-12-27 17:50:51,524][20347] Updated weights for policy 0, policy_version 1761290 (0.0006) [2023-12-27 17:50:51,577][20347] Updated weights for policy 0, policy_version 1761300 (0.0006) [2023-12-27 17:50:51,642][20347] Updated weights for policy 0, policy_version 1761310 (0.0007) [2023-12-27 17:50:51,872][20381] Updated weights for policy 1, policy_version 1759778 (0.0007) [2023-12-27 17:50:51,924][20381] Updated weights for policy 1, policy_version 1759788 (0.0006) [2023-12-27 17:50:51,984][20381] Updated weights for policy 1, policy_version 1759798 (0.0006) [2023-12-27 17:50:52,037][20381] Updated weights for policy 1, policy_version 1759808 (0.0007) [2023-12-27 17:50:52,317][20347] Updated weights for policy 0, policy_version 1761320 (0.0008) [2023-12-27 17:50:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 901537792. Throughput: 0: 10380.3, 1: 10206.6. Samples: 901511296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:50:52,363][19170] Avg episode reward: [(0, '810.800'), (1, '1000.000')] [2023-12-27 17:50:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001759808_450560000.pth... [2023-12-27 17:50:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001758624_450256896.pth [2023-12-27 17:50:52,378][20347] Updated weights for policy 0, policy_version 1761330 (0.0009) [2023-12-27 17:50:52,434][20347] Updated weights for policy 0, policy_version 1761340 (0.0007) [2023-12-27 17:50:52,456][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001761344_450985984.pth... [2023-12-27 17:50:52,461][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001760096_450666496.pth [2023-12-27 17:50:52,723][20381] Updated weights for policy 1, policy_version 1759818 (0.0008) [2023-12-27 17:50:52,773][20381] Updated weights for policy 1, policy_version 1759828 (0.0008) [2023-12-27 17:50:52,825][20381] Updated weights for policy 1, policy_version 1759838 (0.0006) [2023-12-27 17:50:53,123][20347] Updated weights for policy 0, policy_version 1761350 (0.0007) [2023-12-27 17:50:53,183][20347] Updated weights for policy 0, policy_version 1761360 (0.0009) [2023-12-27 17:50:53,242][20347] Updated weights for policy 0, policy_version 1761370 (0.0008) [2023-12-27 17:50:53,382][20381] Updated weights for policy 1, policy_version 1759848 (0.0005) [2023-12-27 17:50:53,446][20381] Updated weights for policy 1, policy_version 1759858 (0.0005) [2023-12-27 17:50:53,508][20381] Updated weights for policy 1, policy_version 1759868 (0.0005) [2023-12-27 17:50:53,858][20347] Updated weights for policy 0, policy_version 1761380 (0.0007) [2023-12-27 17:50:53,917][20347] Updated weights for policy 0, policy_version 1761390 (0.0006) [2023-12-27 17:50:53,973][20347] Updated weights for policy 0, policy_version 1761400 (0.0007) [2023-12-27 17:50:54,107][20381] Updated weights for policy 1, policy_version 1759878 (0.0010) [2023-12-27 17:50:54,165][20381] Updated weights for policy 1, policy_version 1759888 (0.0011) [2023-12-27 17:50:54,230][20381] Updated weights for policy 1, policy_version 1759898 (0.0009) [2023-12-27 17:50:54,585][20347] Updated weights for policy 0, policy_version 1761410 (0.0007) [2023-12-27 17:50:54,632][20347] Updated weights for policy 0, policy_version 1761420 (0.0009) [2023-12-27 17:50:54,680][20347] Updated weights for policy 0, policy_version 1761430 (0.0007) [2023-12-27 17:50:54,735][20347] Updated weights for policy 0, policy_version 1761440 (0.0007) [2023-12-27 17:50:54,853][20381] Updated weights for policy 1, policy_version 1759908 (0.0008) [2023-12-27 17:50:54,912][20381] Updated weights for policy 1, policy_version 1759918 (0.0011) [2023-12-27 17:50:54,970][20381] Updated weights for policy 1, policy_version 1759928 (0.0007) [2023-12-27 17:50:55,424][20347] Updated weights for policy 0, policy_version 1761450 (0.0008) [2023-12-27 17:50:55,475][20347] Updated weights for policy 0, policy_version 1761460 (0.0007) [2023-12-27 17:50:55,532][20347] Updated weights for policy 0, policy_version 1761470 (0.0007) [2023-12-27 17:50:55,547][20381] Updated weights for policy 1, policy_version 1759938 (0.0008) [2023-12-27 17:50:55,607][20381] Updated weights for policy 1, policy_version 1759948 (0.0010) [2023-12-27 17:50:55,665][20381] Updated weights for policy 1, policy_version 1759958 (0.0009) [2023-12-27 17:50:55,727][20381] Updated weights for policy 1, policy_version 1759968 (0.0006) [2023-12-27 17:50:56,265][20347] Updated weights for policy 0, policy_version 1761480 (0.0007) [2023-12-27 17:50:56,304][20381] Updated weights for policy 1, policy_version 1759978 (0.0008) [2023-12-27 17:50:56,323][20347] Updated weights for policy 0, policy_version 1761490 (0.0006) [2023-12-27 17:50:56,371][20381] Updated weights for policy 1, policy_version 1759988 (0.0008) [2023-12-27 17:50:56,380][20347] Updated weights for policy 0, policy_version 1761500 (0.0005) [2023-12-27 17:50:56,431][20381] Updated weights for policy 1, policy_version 1759998 (0.0007) [2023-12-27 17:50:56,965][20347] Updated weights for policy 0, policy_version 1761510 (0.0007) [2023-12-27 17:50:57,019][20347] Updated weights for policy 0, policy_version 1761520 (0.0007) [2023-12-27 17:50:57,066][20347] Updated weights for policy 0, policy_version 1761530 (0.0007) [2023-12-27 17:50:57,116][20381] Updated weights for policy 1, policy_version 1760008 (0.0006) [2023-12-27 17:50:57,168][20381] Updated weights for policy 1, policy_version 1760018 (0.0007) [2023-12-27 17:50:57,227][20381] Updated weights for policy 1, policy_version 1760028 (0.0008) [2023-12-27 17:50:57,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 901652480. Throughput: 0: 10376.0, 1: 10293.8. Samples: 901638692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:50:57,364][19170] Avg episode reward: [(0, '770.300'), (1, '1000.000')] [2023-12-27 17:50:57,714][20347] Updated weights for policy 0, policy_version 1761540 (0.0007) [2023-12-27 17:50:57,763][20347] Updated weights for policy 0, policy_version 1761550 (0.0007) [2023-12-27 17:50:57,809][20347] Updated weights for policy 0, policy_version 1761560 (0.0008) [2023-12-27 17:50:57,929][20381] Updated weights for policy 1, policy_version 1760038 (0.0008) [2023-12-27 17:50:57,980][20381] Updated weights for policy 1, policy_version 1760048 (0.0008) [2023-12-27 17:50:58,028][20381] Updated weights for policy 1, policy_version 1760058 (0.0008) [2023-12-27 17:50:58,535][20347] Updated weights for policy 0, policy_version 1761570 (0.0008) [2023-12-27 17:50:58,598][20347] Updated weights for policy 0, policy_version 1761580 (0.0006) [2023-12-27 17:50:58,665][20347] Updated weights for policy 0, policy_version 1761591 (0.0007) [2023-12-27 17:50:58,775][20381] Updated weights for policy 1, policy_version 1760068 (0.0007) [2023-12-27 17:50:58,845][20381] Updated weights for policy 1, policy_version 1760078 (0.0010) [2023-12-27 17:50:58,901][20381] Updated weights for policy 1, policy_version 1760088 (0.0008) [2023-12-27 17:50:59,482][20347] Updated weights for policy 0, policy_version 1761601 (0.0008) [2023-12-27 17:50:59,543][20347] Updated weights for policy 0, policy_version 1761612 (0.0008) [2023-12-27 17:50:59,591][20347] Updated weights for policy 0, policy_version 1761622 (0.0007) [2023-12-27 17:50:59,647][20347] Updated weights for policy 0, policy_version 1761632 (0.0007) [2023-12-27 17:50:59,685][20381] Updated weights for policy 1, policy_version 1760098 (0.0008) [2023-12-27 17:50:59,745][20381] Updated weights for policy 1, policy_version 1760108 (0.0007) [2023-12-27 17:50:59,803][20381] Updated weights for policy 1, policy_version 1760118 (0.0007) [2023-12-27 17:50:59,868][20381] Updated weights for policy 1, policy_version 1760128 (0.0008) [2023-12-27 17:51:00,382][20347] Updated weights for policy 0, policy_version 1761642 (0.0008) [2023-12-27 17:51:00,441][20347] Updated weights for policy 0, policy_version 1761652 (0.0008) [2023-12-27 17:51:00,498][20347] Updated weights for policy 0, policy_version 1761662 (0.0007) [2023-12-27 17:51:00,555][20381] Updated weights for policy 1, policy_version 1760138 (0.0007) [2023-12-27 17:51:00,611][20381] Updated weights for policy 1, policy_version 1760148 (0.0008) [2023-12-27 17:51:00,665][20381] Updated weights for policy 1, policy_version 1760158 (0.0008) [2023-12-27 17:51:01,119][20347] Updated weights for policy 0, policy_version 1761672 (0.0009) [2023-12-27 17:51:01,176][20347] Updated weights for policy 0, policy_version 1761682 (0.0008) [2023-12-27 17:51:01,227][20347] Updated weights for policy 0, policy_version 1761692 (0.0008) [2023-12-27 17:51:01,394][20381] Updated weights for policy 1, policy_version 1760168 (0.0009) [2023-12-27 17:51:01,447][20381] Updated weights for policy 1, policy_version 1760178 (0.0007) [2023-12-27 17:51:01,512][20381] Updated weights for policy 1, policy_version 1760188 (0.0006) [2023-12-27 17:51:01,986][20347] Updated weights for policy 0, policy_version 1761702 (0.0010) [2023-12-27 17:51:02,049][20347] Updated weights for policy 0, policy_version 1761712 (0.0010) [2023-12-27 17:51:02,100][20347] Updated weights for policy 0, policy_version 1761722 (0.0009) [2023-12-27 17:51:02,184][20381] Updated weights for policy 1, policy_version 1760198 (0.0005) [2023-12-27 17:51:02,234][20381] Updated weights for policy 1, policy_version 1760208 (0.0005) [2023-12-27 17:51:02,294][20381] Updated weights for policy 1, policy_version 1760218 (0.0007) [2023-12-27 17:51:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 901750784. Throughput: 0: 10385.3, 1: 10231.2. Samples: 901757560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:02,363][19170] Avg episode reward: [(0, '780.570'), (1, '1000.000')] [2023-12-27 17:51:02,758][20347] Updated weights for policy 0, policy_version 1761732 (0.0006) [2023-12-27 17:51:02,814][20347] Updated weights for policy 0, policy_version 1761742 (0.0008) [2023-12-27 17:51:02,863][20347] Updated weights for policy 0, policy_version 1761752 (0.0010) [2023-12-27 17:51:02,873][20381] Updated weights for policy 1, policy_version 1760228 (0.0008) [2023-12-27 17:51:02,926][20381] Updated weights for policy 1, policy_version 1760238 (0.0006) [2023-12-27 17:51:02,980][20381] Updated weights for policy 1, policy_version 1760248 (0.0007) [2023-12-27 17:51:03,513][20347] Updated weights for policy 0, policy_version 1761762 (0.0009) [2023-12-27 17:51:03,569][20347] Updated weights for policy 0, policy_version 1761772 (0.0005) [2023-12-27 17:51:03,619][20347] Updated weights for policy 0, policy_version 1761782 (0.0005) [2023-12-27 17:51:03,661][20381] Updated weights for policy 1, policy_version 1760258 (0.0008) [2023-12-27 17:51:03,678][20347] Updated weights for policy 0, policy_version 1761792 (0.0005) [2023-12-27 17:51:03,714][20381] Updated weights for policy 1, policy_version 1760268 (0.0009) [2023-12-27 17:51:03,759][20381] Updated weights for policy 1, policy_version 1760278 (0.0009) [2023-12-27 17:51:03,805][20381] Updated weights for policy 1, policy_version 1760288 (0.0008) [2023-12-27 17:51:04,307][20347] Updated weights for policy 0, policy_version 1761802 (0.0008) [2023-12-27 17:51:04,359][20347] Updated weights for policy 0, policy_version 1761812 (0.0007) [2023-12-27 17:51:04,418][20347] Updated weights for policy 0, policy_version 1761822 (0.0007) [2023-12-27 17:51:04,561][20381] Updated weights for policy 1, policy_version 1760298 (0.0008) [2023-12-27 17:51:04,612][20381] Updated weights for policy 1, policy_version 1760308 (0.0006) [2023-12-27 17:51:04,663][20381] Updated weights for policy 1, policy_version 1760318 (0.0008) [2023-12-27 17:51:05,029][20347] Updated weights for policy 0, policy_version 1761832 (0.0006) [2023-12-27 17:51:05,088][20347] Updated weights for policy 0, policy_version 1761842 (0.0007) [2023-12-27 17:51:05,140][20347] Updated weights for policy 0, policy_version 1761852 (0.0007) [2023-12-27 17:51:05,358][20381] Updated weights for policy 1, policy_version 1760328 (0.0006) [2023-12-27 17:51:05,407][20381] Updated weights for policy 1, policy_version 1760338 (0.0005) [2023-12-27 17:51:05,467][20381] Updated weights for policy 1, policy_version 1760348 (0.0008) [2023-12-27 17:51:05,790][20347] Updated weights for policy 0, policy_version 1761862 (0.0008) [2023-12-27 17:51:05,845][20347] Updated weights for policy 0, policy_version 1761872 (0.0007) [2023-12-27 17:51:05,904][20347] Updated weights for policy 0, policy_version 1761882 (0.0005) [2023-12-27 17:51:06,032][20381] Updated weights for policy 1, policy_version 1760358 (0.0007) [2023-12-27 17:51:06,086][20381] Updated weights for policy 1, policy_version 1760368 (0.0008) [2023-12-27 17:51:06,147][20381] Updated weights for policy 1, policy_version 1760378 (0.0008) [2023-12-27 17:51:06,568][20347] Updated weights for policy 0, policy_version 1761892 (0.0006) [2023-12-27 17:51:06,623][20347] Updated weights for policy 0, policy_version 1761902 (0.0007) [2023-12-27 17:51:06,675][20347] Updated weights for policy 0, policy_version 1761912 (0.0008) [2023-12-27 17:51:06,849][20381] Updated weights for policy 1, policy_version 1760388 (0.0006) [2023-12-27 17:51:06,904][20381] Updated weights for policy 1, policy_version 1760398 (0.0007) [2023-12-27 17:51:06,955][20381] Updated weights for policy 1, policy_version 1760408 (0.0007) [2023-12-27 17:51:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 901849088. Throughput: 0: 10441.1, 1: 10205.0. Samples: 901820868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:07,363][19170] Avg episode reward: [(0, '818.670'), (1, '1000.000')] [2023-12-27 17:51:07,368][20347] Updated weights for policy 0, policy_version 1761922 (0.0008) [2023-12-27 17:51:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001760416_450715648.pth... [2023-12-27 17:51:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001759232_450412544.pth [2023-12-27 17:51:07,426][20347] Updated weights for policy 0, policy_version 1761932 (0.0005) [2023-12-27 17:51:07,481][20347] Updated weights for policy 0, policy_version 1761942 (0.0006) [2023-12-27 17:51:07,537][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001761952_451141632.pth... [2023-12-27 17:51:07,539][20347] Updated weights for policy 0, policy_version 1761952 (0.0005) [2023-12-27 17:51:07,541][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001760736_450830336.pth [2023-12-27 17:51:07,705][20381] Updated weights for policy 1, policy_version 1760418 (0.0007) [2023-12-27 17:51:07,760][20381] Updated weights for policy 1, policy_version 1760428 (0.0005) [2023-12-27 17:51:07,808][20381] Updated weights for policy 1, policy_version 1760438 (0.0005) [2023-12-27 17:51:07,864][20381] Updated weights for policy 1, policy_version 1760448 (0.0005) [2023-12-27 17:51:08,180][20347] Updated weights for policy 0, policy_version 1761962 (0.0007) [2023-12-27 17:51:08,234][20347] Updated weights for policy 0, policy_version 1761972 (0.0007) [2023-12-27 17:51:08,285][20347] Updated weights for policy 0, policy_version 1761982 (0.0007) [2023-12-27 17:51:08,495][20381] Updated weights for policy 1, policy_version 1760458 (0.0008) [2023-12-27 17:51:08,550][20381] Updated weights for policy 1, policy_version 1760468 (0.0009) [2023-12-27 17:51:08,598][20381] Updated weights for policy 1, policy_version 1760478 (0.0010) [2023-12-27 17:51:08,946][20347] Updated weights for policy 0, policy_version 1761992 (0.0007) [2023-12-27 17:51:09,003][20347] Updated weights for policy 0, policy_version 1762002 (0.0008) [2023-12-27 17:51:09,062][20347] Updated weights for policy 0, policy_version 1762012 (0.0006) [2023-12-27 17:51:09,350][20381] Updated weights for policy 1, policy_version 1760488 (0.0009) [2023-12-27 17:51:09,419][20381] Updated weights for policy 1, policy_version 1760498 (0.0009) [2023-12-27 17:51:09,476][20381] Updated weights for policy 1, policy_version 1760508 (0.0007) [2023-12-27 17:51:09,762][20347] Updated weights for policy 0, policy_version 1762022 (0.0007) [2023-12-27 17:51:09,824][20347] Updated weights for policy 0, policy_version 1762032 (0.0008) [2023-12-27 17:51:09,884][20347] Updated weights for policy 0, policy_version 1762042 (0.0006) [2023-12-27 17:51:10,164][20381] Updated weights for policy 1, policy_version 1760518 (0.0009) [2023-12-27 17:51:10,219][20381] Updated weights for policy 1, policy_version 1760528 (0.0010) [2023-12-27 17:51:10,280][20381] Updated weights for policy 1, policy_version 1760538 (0.0011) [2023-12-27 17:51:10,589][20347] Updated weights for policy 0, policy_version 1762052 (0.0007) [2023-12-27 17:51:10,641][20347] Updated weights for policy 0, policy_version 1762062 (0.0007) [2023-12-27 17:51:10,705][20347] Updated weights for policy 0, policy_version 1762072 (0.0007) [2023-12-27 17:51:10,951][20381] Updated weights for policy 1, policy_version 1760548 (0.0010) [2023-12-27 17:51:11,016][20381] Updated weights for policy 1, policy_version 1760558 (0.0006) [2023-12-27 17:51:11,082][20381] Updated weights for policy 1, policy_version 1760568 (0.0008) [2023-12-27 17:51:11,430][20347] Updated weights for policy 0, policy_version 1762082 (0.0007) [2023-12-27 17:51:11,490][20347] Updated weights for policy 0, policy_version 1762092 (0.0007) [2023-12-27 17:51:11,551][20347] Updated weights for policy 0, policy_version 1762102 (0.0008) [2023-12-27 17:51:11,610][20347] Updated weights for policy 0, policy_version 1762112 (0.0007) [2023-12-27 17:51:11,833][20381] Updated weights for policy 1, policy_version 1760578 (0.0008) [2023-12-27 17:51:11,885][20381] Updated weights for policy 1, policy_version 1760588 (0.0008) [2023-12-27 17:51:11,934][20381] Updated weights for policy 1, policy_version 1760598 (0.0008) [2023-12-27 17:51:11,986][20381] Updated weights for policy 1, policy_version 1760608 (0.0008) [2023-12-27 17:51:12,330][20347] Updated weights for policy 0, policy_version 1762122 (0.0007) [2023-12-27 17:51:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 901947392. Throughput: 0: 10424.2, 1: 10192.3. Samples: 901942536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:12,363][19170] Avg episode reward: [(0, '823.200'), (1, '1000.000')] [2023-12-27 17:51:12,400][20347] Updated weights for policy 0, policy_version 1762132 (0.0008) [2023-12-27 17:51:12,462][20347] Updated weights for policy 0, policy_version 1762142 (0.0007) [2023-12-27 17:51:12,672][20381] Updated weights for policy 1, policy_version 1760618 (0.0008) [2023-12-27 17:51:12,727][20381] Updated weights for policy 1, policy_version 1760628 (0.0008) [2023-12-27 17:51:12,783][20381] Updated weights for policy 1, policy_version 1760638 (0.0007) [2023-12-27 17:51:13,108][20347] Updated weights for policy 0, policy_version 1762152 (0.0010) [2023-12-27 17:51:13,153][20347] Updated weights for policy 0, policy_version 1762162 (0.0009) [2023-12-27 17:51:13,206][20347] Updated weights for policy 0, policy_version 1762172 (0.0008) [2023-12-27 17:51:13,418][20381] Updated weights for policy 1, policy_version 1760648 (0.0008) [2023-12-27 17:51:13,469][20381] Updated weights for policy 1, policy_version 1760658 (0.0007) [2023-12-27 17:51:13,525][20381] Updated weights for policy 1, policy_version 1760668 (0.0008) [2023-12-27 17:51:13,897][20347] Updated weights for policy 0, policy_version 1762182 (0.0007) [2023-12-27 17:51:13,943][20347] Updated weights for policy 0, policy_version 1762192 (0.0005) [2023-12-27 17:51:13,988][20347] Updated weights for policy 0, policy_version 1762202 (0.0005) [2023-12-27 17:51:14,206][20381] Updated weights for policy 1, policy_version 1760678 (0.0007) [2023-12-27 17:51:14,264][20381] Updated weights for policy 1, policy_version 1760688 (0.0008) [2023-12-27 17:51:14,320][20381] Updated weights for policy 1, policy_version 1760698 (0.0008) [2023-12-27 17:51:14,638][20347] Updated weights for policy 0, policy_version 1762212 (0.0005) [2023-12-27 17:51:14,695][20347] Updated weights for policy 0, policy_version 1762222 (0.0005) [2023-12-27 17:51:14,744][20347] Updated weights for policy 0, policy_version 1762232 (0.0005) [2023-12-27 17:51:15,063][20381] Updated weights for policy 1, policy_version 1760708 (0.0007) [2023-12-27 17:51:15,125][20381] Updated weights for policy 1, policy_version 1760718 (0.0008) [2023-12-27 17:51:15,183][20381] Updated weights for policy 1, policy_version 1760728 (0.0008) [2023-12-27 17:51:15,399][20347] Updated weights for policy 0, policy_version 1762242 (0.0009) [2023-12-27 17:51:15,455][20347] Updated weights for policy 0, policy_version 1762252 (0.0008) [2023-12-27 17:51:15,512][20347] Updated weights for policy 0, policy_version 1762262 (0.0008) [2023-12-27 17:51:15,567][20347] Updated weights for policy 0, policy_version 1762272 (0.0008) [2023-12-27 17:51:15,857][20381] Updated weights for policy 1, policy_version 1760738 (0.0008) [2023-12-27 17:51:15,911][20381] Updated weights for policy 1, policy_version 1760748 (0.0007) [2023-12-27 17:51:15,974][20381] Updated weights for policy 1, policy_version 1760758 (0.0008) [2023-12-27 17:51:16,029][20381] Updated weights for policy 1, policy_version 1760768 (0.0009) [2023-12-27 17:51:16,242][20347] Updated weights for policy 0, policy_version 1762282 (0.0006) [2023-12-27 17:51:16,301][20347] Updated weights for policy 0, policy_version 1762292 (0.0007) [2023-12-27 17:51:16,367][20347] Updated weights for policy 0, policy_version 1762302 (0.0007) [2023-12-27 17:51:16,738][20381] Updated weights for policy 1, policy_version 1760778 (0.0010) [2023-12-27 17:51:16,785][20381] Updated weights for policy 1, policy_version 1760788 (0.0009) [2023-12-27 17:51:16,832][20381] Updated weights for policy 1, policy_version 1760798 (0.0005) [2023-12-27 17:51:17,077][20347] Updated weights for policy 0, policy_version 1762312 (0.0008) [2023-12-27 17:51:17,122][20291] KL-divergence is very high: 198.7968 [2023-12-27 17:51:17,128][20347] Updated weights for policy 0, policy_version 1762322 (0.0007) [2023-12-27 17:51:17,132][20291] KL-divergence is very high: 222.1007 [2023-12-27 17:51:17,164][20291] KL-divergence is very high: 168.2218 [2023-12-27 17:51:17,174][20291] KL-divergence is very high: 132.9045 [2023-12-27 17:51:17,181][20347] Updated weights for policy 0, policy_version 1762332 (0.0007) [2023-12-27 17:51:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 902053888. Throughput: 0: 10372.1, 1: 10172.0. Samples: 902065356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:17,363][19170] Avg episode reward: [(0, '833.650'), (1, '1000.000')] [2023-12-27 17:51:17,388][20381] Updated weights for policy 1, policy_version 1760808 (0.0006) [2023-12-27 17:51:17,448][20381] Updated weights for policy 1, policy_version 1760818 (0.0008) [2023-12-27 17:51:17,502][20381] Updated weights for policy 1, policy_version 1760828 (0.0006) [2023-12-27 17:51:17,950][20347] Updated weights for policy 0, policy_version 1762342 (0.0007) [2023-12-27 17:51:17,962][20291] KL-divergence is very high: 159.0704 [2023-12-27 17:51:17,996][20347] Updated weights for policy 0, policy_version 1762352 (0.0007) [2023-12-27 17:51:18,001][20291] KL-divergence is very high: 162.8793 [2023-12-27 17:51:18,040][20291] KL-divergence is very high: 140.1870 [2023-12-27 17:51:18,044][20347] Updated weights for policy 0, policy_version 1762362 (0.0009) [2023-12-27 17:51:18,087][20381] Updated weights for policy 1, policy_version 1760838 (0.0009) [2023-12-27 17:51:18,136][20381] Updated weights for policy 1, policy_version 1760848 (0.0010) [2023-12-27 17:51:18,192][20381] Updated weights for policy 1, policy_version 1760858 (0.0010) [2023-12-27 17:51:18,758][20347] Updated weights for policy 0, policy_version 1762372 (0.0008) [2023-12-27 17:51:18,822][20347] Updated weights for policy 0, policy_version 1762382 (0.0007) [2023-12-27 17:51:18,883][20347] Updated weights for policy 0, policy_version 1762392 (0.0007) [2023-12-27 17:51:18,892][20381] Updated weights for policy 1, policy_version 1760868 (0.0010) [2023-12-27 17:51:18,949][20381] Updated weights for policy 1, policy_version 1760878 (0.0011) [2023-12-27 17:51:19,002][20381] Updated weights for policy 1, policy_version 1760888 (0.0010) [2023-12-27 17:51:19,589][20347] Updated weights for policy 0, policy_version 1762402 (0.0007) [2023-12-27 17:51:19,653][20347] Updated weights for policy 0, policy_version 1762412 (0.0008) [2023-12-27 17:51:19,703][20381] Updated weights for policy 1, policy_version 1760898 (0.0010) [2023-12-27 17:51:19,713][20347] Updated weights for policy 0, policy_version 1762422 (0.0007) [2023-12-27 17:51:19,762][20381] Updated weights for policy 1, policy_version 1760908 (0.0007) [2023-12-27 17:51:19,773][20347] Updated weights for policy 0, policy_version 1762432 (0.0009) [2023-12-27 17:51:19,826][20381] Updated weights for policy 1, policy_version 1760918 (0.0008) [2023-12-27 17:51:19,894][20381] Updated weights for policy 1, policy_version 1760928 (0.0008) [2023-12-27 17:51:20,407][20347] Updated weights for policy 0, policy_version 1762442 (0.0011) [2023-12-27 17:51:20,464][20347] Updated weights for policy 0, policy_version 1762452 (0.0011) [2023-12-27 17:51:20,477][20291] KL-divergence is very high: 159.7867 [2023-12-27 17:51:20,489][20291] KL-divergence is very high: 176.6676 [2023-12-27 17:51:20,495][20291] KL-divergence is very high: 163.9805 [2023-12-27 17:51:20,501][20291] KL-divergence is very high: 160.9420 [2023-12-27 17:51:20,507][20291] KL-divergence is very high: 198.9340 [2023-12-27 17:51:20,525][20291] KL-divergence is very high: 173.7767 [2023-12-27 17:51:20,527][20347] Updated weights for policy 0, policy_version 1762462 (0.0011) [2023-12-27 17:51:20,567][20381] Updated weights for policy 1, policy_version 1760938 (0.0007) [2023-12-27 17:51:20,629][20381] Updated weights for policy 1, policy_version 1760948 (0.0007) [2023-12-27 17:51:20,689][20381] Updated weights for policy 1, policy_version 1760958 (0.0007) [2023-12-27 17:51:21,238][20347] Updated weights for policy 0, policy_version 1762472 (0.0009) [2023-12-27 17:51:21,305][20347] Updated weights for policy 0, policy_version 1762482 (0.0007) [2023-12-27 17:51:21,386][20347] Updated weights for policy 0, policy_version 1762492 (0.0010) [2023-12-27 17:51:21,414][20381] Updated weights for policy 1, policy_version 1760968 (0.0007) [2023-12-27 17:51:21,477][20381] Updated weights for policy 1, policy_version 1760978 (0.0006) [2023-12-27 17:51:21,539][20381] Updated weights for policy 1, policy_version 1760988 (0.0005) [2023-12-27 17:51:22,029][20347] Updated weights for policy 0, policy_version 1762502 (0.0008) [2023-12-27 17:51:22,100][20347] Updated weights for policy 0, policy_version 1762512 (0.0006) [2023-12-27 17:51:22,166][20347] Updated weights for policy 0, policy_version 1762522 (0.0008) [2023-12-27 17:51:22,215][20381] Updated weights for policy 1, policy_version 1760998 (0.0007) [2023-12-27 17:51:22,272][20381] Updated weights for policy 1, policy_version 1761008 (0.0008) [2023-12-27 17:51:22,336][20381] Updated weights for policy 1, policy_version 1761018 (0.0007) [2023-12-27 17:51:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 902152192. Throughput: 0: 10271.5, 1: 10174.9. Samples: 902124708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:22,363][19170] Avg episode reward: [(0, '756.500'), (1, '1000.000')] [2023-12-27 17:51:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001762528_451289088.pth... [2023-12-27 17:51:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001761344_450985984.pth [2023-12-27 17:51:22,382][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001761024_450871296.pth... [2023-12-27 17:51:22,387][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001759808_450560000.pth [2023-12-27 17:51:22,853][20347] Updated weights for policy 0, policy_version 1762532 (0.0007) [2023-12-27 17:51:22,908][20347] Updated weights for policy 0, policy_version 1762542 (0.0008) [2023-12-27 17:51:22,961][20347] Updated weights for policy 0, policy_version 1762552 (0.0007) [2023-12-27 17:51:23,074][20381] Updated weights for policy 1, policy_version 1761028 (0.0008) [2023-12-27 17:51:23,131][20381] Updated weights for policy 1, policy_version 1761038 (0.0007) [2023-12-27 17:51:23,187][20381] Updated weights for policy 1, policy_version 1761048 (0.0008) [2023-12-27 17:51:23,672][20347] Updated weights for policy 0, policy_version 1762562 (0.0008) [2023-12-27 17:51:23,732][20347] Updated weights for policy 0, policy_version 1762572 (0.0008) [2023-12-27 17:51:23,785][20347] Updated weights for policy 0, policy_version 1762582 (0.0008) [2023-12-27 17:51:23,819][20381] Updated weights for policy 1, policy_version 1761058 (0.0007) [2023-12-27 17:51:23,842][20347] Updated weights for policy 0, policy_version 1762592 (0.0008) [2023-12-27 17:51:23,885][20381] Updated weights for policy 1, policy_version 1761068 (0.0009) [2023-12-27 17:51:23,932][20381] Updated weights for policy 1, policy_version 1761078 (0.0007) [2023-12-27 17:51:23,982][20381] Updated weights for policy 1, policy_version 1761088 (0.0009) [2023-12-27 17:51:24,507][20347] Updated weights for policy 0, policy_version 1762602 (0.0009) [2023-12-27 17:51:24,564][20347] Updated weights for policy 0, policy_version 1762612 (0.0008) [2023-12-27 17:51:24,620][20347] Updated weights for policy 0, policy_version 1762622 (0.0008) [2023-12-27 17:51:24,686][20381] Updated weights for policy 1, policy_version 1761098 (0.0007) [2023-12-27 17:51:24,731][20381] Updated weights for policy 1, policy_version 1761108 (0.0007) [2023-12-27 17:51:24,782][20381] Updated weights for policy 1, policy_version 1761118 (0.0008) [2023-12-27 17:51:25,276][20347] Updated weights for policy 0, policy_version 1762632 (0.0006) [2023-12-27 17:51:25,347][20347] Updated weights for policy 0, policy_version 1762642 (0.0006) [2023-12-27 17:51:25,400][20347] Updated weights for policy 0, policy_version 1762652 (0.0008) [2023-12-27 17:51:25,454][20381] Updated weights for policy 1, policy_version 1761128 (0.0006) [2023-12-27 17:51:25,510][20381] Updated weights for policy 1, policy_version 1761138 (0.0007) [2023-12-27 17:51:25,565][20381] Updated weights for policy 1, policy_version 1761148 (0.0011) [2023-12-27 17:51:26,055][20347] Updated weights for policy 0, policy_version 1762662 (0.0009) [2023-12-27 17:51:26,112][20347] Updated weights for policy 0, policy_version 1762672 (0.0008) [2023-12-27 17:51:26,172][20347] Updated weights for policy 0, policy_version 1762682 (0.0007) [2023-12-27 17:51:26,189][20381] Updated weights for policy 1, policy_version 1761158 (0.0007) [2023-12-27 17:51:26,242][20381] Updated weights for policy 1, policy_version 1761168 (0.0005) [2023-12-27 17:51:26,291][20381] Updated weights for policy 1, policy_version 1761178 (0.0005) [2023-12-27 17:51:26,858][20381] Updated weights for policy 1, policy_version 1761188 (0.0005) [2023-12-27 17:51:26,898][20347] Updated weights for policy 0, policy_version 1762692 (0.0007) [2023-12-27 17:51:26,916][20381] Updated weights for policy 1, policy_version 1761198 (0.0005) [2023-12-27 17:51:26,947][20347] Updated weights for policy 0, policy_version 1762702 (0.0006) [2023-12-27 17:51:26,971][20381] Updated weights for policy 1, policy_version 1761208 (0.0006) [2023-12-27 17:51:27,000][20347] Updated weights for policy 0, policy_version 1762712 (0.0006) [2023-12-27 17:51:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 902258688. Throughput: 0: 10217.5, 1: 10218.2. Samples: 902246628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:27,363][19170] Avg episode reward: [(0, '736.610'), (1, '1000.000')] [2023-12-27 17:51:27,512][20381] Updated weights for policy 1, policy_version 1761218 (0.0006) [2023-12-27 17:51:27,598][20381] Updated weights for policy 1, policy_version 1761228 (0.0007) [2023-12-27 17:51:27,658][20381] Updated weights for policy 1, policy_version 1761238 (0.0005) [2023-12-27 17:51:27,716][20381] Updated weights for policy 1, policy_version 1761248 (0.0005) [2023-12-27 17:51:27,719][20347] Updated weights for policy 0, policy_version 1762722 (0.0006) [2023-12-27 17:51:27,772][20347] Updated weights for policy 0, policy_version 1762732 (0.0005) [2023-12-27 17:51:27,831][20347] Updated weights for policy 0, policy_version 1762742 (0.0006) [2023-12-27 17:51:27,887][20347] Updated weights for policy 0, policy_version 1762752 (0.0005) [2023-12-27 17:51:28,296][20381] Updated weights for policy 1, policy_version 1761258 (0.0010) [2023-12-27 17:51:28,356][20381] Updated weights for policy 1, policy_version 1761268 (0.0009) [2023-12-27 17:51:28,396][20347] Updated weights for policy 0, policy_version 1762762 (0.0008) [2023-12-27 17:51:28,416][20381] Updated weights for policy 1, policy_version 1761278 (0.0006) [2023-12-27 17:51:28,446][20347] Updated weights for policy 0, policy_version 1762772 (0.0008) [2023-12-27 17:51:28,495][20347] Updated weights for policy 0, policy_version 1762782 (0.0009) [2023-12-27 17:51:29,089][20381] Updated weights for policy 1, policy_version 1761288 (0.0011) [2023-12-27 17:51:29,091][20347] Updated weights for policy 0, policy_version 1762792 (0.0007) [2023-12-27 17:51:29,143][20347] Updated weights for policy 0, policy_version 1762802 (0.0005) [2023-12-27 17:51:29,143][20381] Updated weights for policy 1, policy_version 1761298 (0.0010) [2023-12-27 17:51:29,191][20381] Updated weights for policy 1, policy_version 1761308 (0.0010) [2023-12-27 17:51:29,193][20347] Updated weights for policy 0, policy_version 1762812 (0.0006) [2023-12-27 17:51:29,851][20347] Updated weights for policy 0, policy_version 1762822 (0.0007) [2023-12-27 17:51:29,907][20381] Updated weights for policy 1, policy_version 1761318 (0.0007) [2023-12-27 17:51:29,912][20347] Updated weights for policy 0, policy_version 1762832 (0.0007) [2023-12-27 17:51:29,966][20381] Updated weights for policy 1, policy_version 1761328 (0.0008) [2023-12-27 17:51:29,972][20347] Updated weights for policy 0, policy_version 1762842 (0.0008) [2023-12-27 17:51:30,016][20381] Updated weights for policy 1, policy_version 1761338 (0.0007) [2023-12-27 17:51:30,664][20381] Updated weights for policy 1, policy_version 1761348 (0.0008) [2023-12-27 17:51:30,669][20347] Updated weights for policy 0, policy_version 1762852 (0.0008) [2023-12-27 17:51:30,719][20381] Updated weights for policy 1, policy_version 1761358 (0.0010) [2023-12-27 17:51:30,721][20347] Updated weights for policy 0, policy_version 1762862 (0.0007) [2023-12-27 17:51:30,765][20347] Updated weights for policy 0, policy_version 1762872 (0.0006) [2023-12-27 17:51:30,783][20381] Updated weights for policy 1, policy_version 1761368 (0.0010) [2023-12-27 17:51:31,433][20347] Updated weights for policy 0, policy_version 1762882 (0.0006) [2023-12-27 17:51:31,442][20381] Updated weights for policy 1, policy_version 1761378 (0.0011) [2023-12-27 17:51:31,477][20347] Updated weights for policy 0, policy_version 1762892 (0.0008) [2023-12-27 17:51:31,494][20381] Updated weights for policy 1, policy_version 1761388 (0.0010) [2023-12-27 17:51:31,526][20347] Updated weights for policy 0, policy_version 1762902 (0.0008) [2023-12-27 17:51:31,541][20381] Updated weights for policy 1, policy_version 1761398 (0.0007) [2023-12-27 17:51:31,578][20347] Updated weights for policy 0, policy_version 1762912 (0.0007) [2023-12-27 17:51:31,598][20381] Updated weights for policy 1, policy_version 1761408 (0.0008) [2023-12-27 17:51:32,322][20347] Updated weights for policy 0, policy_version 1762922 (0.0007) [2023-12-27 17:51:32,358][20381] Updated weights for policy 1, policy_version 1761418 (0.0008) [2023-12-27 17:51:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 902356992. Throughput: 0: 10197.6, 1: 10325.5. Samples: 902373876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:32,363][19170] Avg episode reward: [(0, '766.440'), (1, '1000.000')] [2023-12-27 17:51:32,385][20347] Updated weights for policy 0, policy_version 1762932 (0.0008) [2023-12-27 17:51:32,413][20381] Updated weights for policy 1, policy_version 1761428 (0.0006) [2023-12-27 17:51:32,417][20291] KL-divergence is very high: 130.0395 [2023-12-27 17:51:32,438][20347] Updated weights for policy 0, policy_version 1762942 (0.0007) [2023-12-27 17:51:32,469][20381] Updated weights for policy 1, policy_version 1761438 (0.0005) [2023-12-27 17:51:33,138][20347] Updated weights for policy 0, policy_version 1762952 (0.0006) [2023-12-27 17:51:33,156][20381] Updated weights for policy 1, policy_version 1761448 (0.0009) [2023-12-27 17:51:33,192][20347] Updated weights for policy 0, policy_version 1762962 (0.0006) [2023-12-27 17:51:33,211][20381] Updated weights for policy 1, policy_version 1761458 (0.0005) [2023-12-27 17:51:33,247][20347] Updated weights for policy 0, policy_version 1762972 (0.0007) [2023-12-27 17:51:33,266][20381] Updated weights for policy 1, policy_version 1761468 (0.0005) [2023-12-27 17:51:33,816][20381] Updated weights for policy 1, policy_version 1761478 (0.0007) [2023-12-27 17:51:33,867][20381] Updated weights for policy 1, policy_version 1761488 (0.0009) [2023-12-27 17:51:33,912][20347] Updated weights for policy 0, policy_version 1762982 (0.0009) [2023-12-27 17:51:33,912][20381] Updated weights for policy 1, policy_version 1761498 (0.0008) [2023-12-27 17:51:33,969][20347] Updated weights for policy 0, policy_version 1762992 (0.0010) [2023-12-27 17:51:34,025][20347] Updated weights for policy 0, policy_version 1763002 (0.0010) [2023-12-27 17:51:34,561][20381] Updated weights for policy 1, policy_version 1761508 (0.0006) [2023-12-27 17:51:34,619][20381] Updated weights for policy 1, policy_version 1761518 (0.0006) [2023-12-27 17:51:34,674][20381] Updated weights for policy 1, policy_version 1761528 (0.0008) [2023-12-27 17:51:34,759][20347] Updated weights for policy 0, policy_version 1763012 (0.0009) [2023-12-27 17:51:34,816][20347] Updated weights for policy 0, policy_version 1763022 (0.0007) [2023-12-27 17:51:34,873][20347] Updated weights for policy 0, policy_version 1763032 (0.0007) [2023-12-27 17:51:35,263][20381] Updated weights for policy 1, policy_version 1761538 (0.0007) [2023-12-27 17:51:35,328][20381] Updated weights for policy 1, policy_version 1761548 (0.0007) [2023-12-27 17:51:35,383][20381] Updated weights for policy 1, policy_version 1761558 (0.0008) [2023-12-27 17:51:35,442][20381] Updated weights for policy 1, policy_version 1761568 (0.0007) [2023-12-27 17:51:35,597][20347] Updated weights for policy 0, policy_version 1763042 (0.0007) [2023-12-27 17:51:35,653][20347] Updated weights for policy 0, policy_version 1763052 (0.0007) [2023-12-27 17:51:35,708][20347] Updated weights for policy 0, policy_version 1763062 (0.0009) [2023-12-27 17:51:35,759][20347] Updated weights for policy 0, policy_version 1763072 (0.0010) [2023-12-27 17:51:36,025][20381] Updated weights for policy 1, policy_version 1761578 (0.0007) [2023-12-27 17:51:36,069][20381] Updated weights for policy 1, policy_version 1761588 (0.0007) [2023-12-27 17:51:36,133][20381] Updated weights for policy 1, policy_version 1761598 (0.0008) [2023-12-27 17:51:36,501][20347] Updated weights for policy 0, policy_version 1763082 (0.0010) [2023-12-27 17:51:36,560][20347] Updated weights for policy 0, policy_version 1763092 (0.0011) [2023-12-27 17:51:36,620][20347] Updated weights for policy 0, policy_version 1763102 (0.0010) [2023-12-27 17:51:36,869][20381] Updated weights for policy 1, policy_version 1761608 (0.0008) [2023-12-27 17:51:36,917][20381] Updated weights for policy 1, policy_version 1761618 (0.0008) [2023-12-27 17:51:36,961][20381] Updated weights for policy 1, policy_version 1761628 (0.0007) [2023-12-27 17:51:37,274][20347] Updated weights for policy 0, policy_version 1763112 (0.0011) [2023-12-27 17:51:37,336][20347] Updated weights for policy 0, policy_version 1763122 (0.0010) [2023-12-27 17:51:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 902463488. Throughput: 0: 10212.4, 1: 10359.0. Samples: 902437012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:37,363][19170] Avg episode reward: [(0, '737.190'), (1, '1000.000')] [2023-12-27 17:51:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001761632_451026944.pth... [2023-12-27 17:51:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001760416_450715648.pth [2023-12-27 17:51:37,393][20347] Updated weights for policy 0, policy_version 1763132 (0.0011) [2023-12-27 17:51:37,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001763136_451444736.pth... [2023-12-27 17:51:37,420][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001761952_451141632.pth [2023-12-27 17:51:37,721][20381] Updated weights for policy 1, policy_version 1761638 (0.0008) [2023-12-27 17:51:37,782][20381] Updated weights for policy 1, policy_version 1761648 (0.0008) [2023-12-27 17:51:37,835][20381] Updated weights for policy 1, policy_version 1761658 (0.0007) [2023-12-27 17:51:38,047][20347] Updated weights for policy 0, policy_version 1763142 (0.0010) [2023-12-27 17:51:38,105][20347] Updated weights for policy 0, policy_version 1763152 (0.0010) [2023-12-27 17:51:38,171][20347] Updated weights for policy 0, policy_version 1763162 (0.0010) [2023-12-27 17:51:38,496][20381] Updated weights for policy 1, policy_version 1761668 (0.0008) [2023-12-27 17:51:38,555][20381] Updated weights for policy 1, policy_version 1761678 (0.0007) [2023-12-27 17:51:38,613][20381] Updated weights for policy 1, policy_version 1761688 (0.0007) [2023-12-27 17:51:38,809][20347] Updated weights for policy 0, policy_version 1763172 (0.0010) [2023-12-27 17:51:38,871][20347] Updated weights for policy 0, policy_version 1763182 (0.0007) [2023-12-27 17:51:38,929][20347] Updated weights for policy 0, policy_version 1763192 (0.0006) [2023-12-27 17:51:39,310][20381] Updated weights for policy 1, policy_version 1761698 (0.0007) [2023-12-27 17:51:39,377][20381] Updated weights for policy 1, policy_version 1761708 (0.0007) [2023-12-27 17:51:39,443][20381] Updated weights for policy 1, policy_version 1761718 (0.0007) [2023-12-27 17:51:39,506][20381] Updated weights for policy 1, policy_version 1761728 (0.0006) [2023-12-27 17:51:39,566][20347] Updated weights for policy 0, policy_version 1763202 (0.0006) [2023-12-27 17:51:39,631][20347] Updated weights for policy 0, policy_version 1763212 (0.0008) [2023-12-27 17:51:39,699][20347] Updated weights for policy 0, policy_version 1763222 (0.0008) [2023-12-27 17:51:39,758][20347] Updated weights for policy 0, policy_version 1763232 (0.0007) [2023-12-27 17:51:40,120][20381] Updated weights for policy 1, policy_version 1761738 (0.0008) [2023-12-27 17:51:40,182][20381] Updated weights for policy 1, policy_version 1761748 (0.0008) [2023-12-27 17:51:40,241][20381] Updated weights for policy 1, policy_version 1761758 (0.0008) [2023-12-27 17:51:40,450][20347] Updated weights for policy 0, policy_version 1763242 (0.0008) [2023-12-27 17:51:40,519][20347] Updated weights for policy 0, policy_version 1763252 (0.0008) [2023-12-27 17:51:40,586][20347] Updated weights for policy 0, policy_version 1763262 (0.0008) [2023-12-27 17:51:40,939][20381] Updated weights for policy 1, policy_version 1761768 (0.0006) [2023-12-27 17:51:40,994][20381] Updated weights for policy 1, policy_version 1761778 (0.0008) [2023-12-27 17:51:41,057][20381] Updated weights for policy 1, policy_version 1761788 (0.0007) [2023-12-27 17:51:41,259][20347] Updated weights for policy 0, policy_version 1763272 (0.0008) [2023-12-27 17:51:41,322][20347] Updated weights for policy 0, policy_version 1763282 (0.0007) [2023-12-27 17:51:41,393][20347] Updated weights for policy 0, policy_version 1763292 (0.0008) [2023-12-27 17:51:41,776][20381] Updated weights for policy 1, policy_version 1761798 (0.0008) [2023-12-27 17:51:41,827][20381] Updated weights for policy 1, policy_version 1761808 (0.0008) [2023-12-27 17:51:41,886][20381] Updated weights for policy 1, policy_version 1761818 (0.0008) [2023-12-27 17:51:42,121][20347] Updated weights for policy 0, policy_version 1763302 (0.0007) [2023-12-27 17:51:42,181][20347] Updated weights for policy 0, policy_version 1763312 (0.0006) [2023-12-27 17:51:42,243][20347] Updated weights for policy 0, policy_version 1763322 (0.0006) [2023-12-27 17:51:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 902569984. Throughput: 0: 10199.4, 1: 10247.9. Samples: 902558816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:42,363][19170] Avg episode reward: [(0, '795.300'), (1, '1000.000')] [2023-12-27 17:51:42,619][20381] Updated weights for policy 1, policy_version 1761828 (0.0009) [2023-12-27 17:51:42,678][20381] Updated weights for policy 1, policy_version 1761838 (0.0010) [2023-12-27 17:51:42,739][20381] Updated weights for policy 1, policy_version 1761848 (0.0010) [2023-12-27 17:51:42,925][20347] Updated weights for policy 0, policy_version 1763332 (0.0006) [2023-12-27 17:51:42,986][20347] Updated weights for policy 0, policy_version 1763342 (0.0007) [2023-12-27 17:51:43,044][20347] Updated weights for policy 0, policy_version 1763352 (0.0007) [2023-12-27 17:51:43,423][20381] Updated weights for policy 1, policy_version 1761858 (0.0008) [2023-12-27 17:51:43,484][20381] Updated weights for policy 1, policy_version 1761868 (0.0008) [2023-12-27 17:51:43,548][20381] Updated weights for policy 1, policy_version 1761878 (0.0008) [2023-12-27 17:51:43,610][20381] Updated weights for policy 1, policy_version 1761888 (0.0009) [2023-12-27 17:51:43,720][20347] Updated weights for policy 0, policy_version 1763362 (0.0007) [2023-12-27 17:51:43,768][20347] Updated weights for policy 0, policy_version 1763372 (0.0005) [2023-12-27 17:51:43,822][20347] Updated weights for policy 0, policy_version 1763382 (0.0005) [2023-12-27 17:51:43,873][20347] Updated weights for policy 0, policy_version 1763392 (0.0005) [2023-12-27 17:51:44,196][20381] Updated weights for policy 1, policy_version 1761898 (0.0011) [2023-12-27 17:51:44,248][20381] Updated weights for policy 1, policy_version 1761908 (0.0010) [2023-12-27 17:51:44,293][20381] Updated weights for policy 1, policy_version 1761918 (0.0010) [2023-12-27 17:51:44,433][20347] Updated weights for policy 0, policy_version 1763402 (0.0007) [2023-12-27 17:51:44,481][20347] Updated weights for policy 0, policy_version 1763412 (0.0007) [2023-12-27 17:51:44,534][20347] Updated weights for policy 0, policy_version 1763422 (0.0007) [2023-12-27 17:51:44,980][20381] Updated weights for policy 1, policy_version 1761928 (0.0009) [2023-12-27 17:51:45,047][20381] Updated weights for policy 1, policy_version 1761938 (0.0008) [2023-12-27 17:51:45,110][20381] Updated weights for policy 1, policy_version 1761948 (0.0008) [2023-12-27 17:51:45,272][20347] Updated weights for policy 0, policy_version 1763432 (0.0007) [2023-12-27 17:51:45,330][20347] Updated weights for policy 0, policy_version 1763442 (0.0007) [2023-12-27 17:51:45,385][20347] Updated weights for policy 0, policy_version 1763452 (0.0007) [2023-12-27 17:51:45,763][20381] Updated weights for policy 1, policy_version 1761958 (0.0006) [2023-12-27 17:51:45,817][20381] Updated weights for policy 1, policy_version 1761968 (0.0008) [2023-12-27 17:51:45,866][20381] Updated weights for policy 1, policy_version 1761978 (0.0010) [2023-12-27 17:51:46,018][20347] Updated weights for policy 0, policy_version 1763462 (0.0006) [2023-12-27 17:51:46,076][20347] Updated weights for policy 0, policy_version 1763472 (0.0009) [2023-12-27 17:51:46,132][20347] Updated weights for policy 0, policy_version 1763482 (0.0006) [2023-12-27 17:51:46,438][20381] Updated weights for policy 1, policy_version 1761988 (0.0008) [2023-12-27 17:51:46,495][20381] Updated weights for policy 1, policy_version 1761998 (0.0008) [2023-12-27 17:51:46,554][20381] Updated weights for policy 1, policy_version 1762008 (0.0010) [2023-12-27 17:51:46,694][20347] Updated weights for policy 0, policy_version 1763492 (0.0007) [2023-12-27 17:51:46,742][20347] Updated weights for policy 0, policy_version 1763502 (0.0005) [2023-12-27 17:51:46,792][20347] Updated weights for policy 0, policy_version 1763512 (0.0008) [2023-12-27 17:51:47,253][20381] Updated weights for policy 1, policy_version 1762018 (0.0010) [2023-12-27 17:51:47,304][20381] Updated weights for policy 1, policy_version 1762028 (0.0008) [2023-12-27 17:51:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 902668288. Throughput: 0: 10258.7, 1: 10339.7. Samples: 902684492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:47,363][20381] Updated weights for policy 1, policy_version 1762038 (0.0008) [2023-12-27 17:51:47,363][19170] Avg episode reward: [(0, '765.560'), (1, '1000.000')] [2023-12-27 17:51:47,428][20381] Updated weights for policy 1, policy_version 1762048 (0.0008) [2023-12-27 17:51:47,437][20347] Updated weights for policy 0, policy_version 1763522 (0.0010) [2023-12-27 17:51:47,504][20347] Updated weights for policy 0, policy_version 1763532 (0.0005) [2023-12-27 17:51:47,560][20347] Updated weights for policy 0, policy_version 1763542 (0.0007) [2023-12-27 17:51:47,621][20347] Updated weights for policy 0, policy_version 1763552 (0.0006) [2023-12-27 17:51:48,155][20381] Updated weights for policy 1, policy_version 1762058 (0.0011) [2023-12-27 17:51:48,181][20347] Updated weights for policy 0, policy_version 1763562 (0.0010) [2023-12-27 17:51:48,208][20381] Updated weights for policy 1, policy_version 1762068 (0.0011) [2023-12-27 17:51:48,238][20347] Updated weights for policy 0, policy_version 1763572 (0.0010) [2023-12-27 17:51:48,266][20381] Updated weights for policy 1, policy_version 1762078 (0.0011) [2023-12-27 17:51:48,295][20347] Updated weights for policy 0, policy_version 1763582 (0.0010) [2023-12-27 17:51:48,895][20347] Updated weights for policy 0, policy_version 1763592 (0.0007) [2023-12-27 17:51:48,949][20347] Updated weights for policy 0, policy_version 1763602 (0.0005) [2023-12-27 17:51:49,009][20347] Updated weights for policy 0, policy_version 1763612 (0.0006) [2023-12-27 17:51:49,041][20381] Updated weights for policy 1, policy_version 1762088 (0.0008) [2023-12-27 17:51:49,093][20381] Updated weights for policy 1, policy_version 1762098 (0.0006) [2023-12-27 17:51:49,143][20381] Updated weights for policy 1, policy_version 1762108 (0.0007) [2023-12-27 17:51:49,715][20347] Updated weights for policy 0, policy_version 1763622 (0.0008) [2023-12-27 17:51:49,754][20381] Updated weights for policy 1, policy_version 1762118 (0.0005) [2023-12-27 17:51:49,771][20347] Updated weights for policy 0, policy_version 1763632 (0.0008) [2023-12-27 17:51:49,811][20381] Updated weights for policy 1, policy_version 1762128 (0.0006) [2023-12-27 17:51:49,836][20347] Updated weights for policy 0, policy_version 1763642 (0.0007) [2023-12-27 17:51:49,877][20381] Updated weights for policy 1, policy_version 1762138 (0.0009) [2023-12-27 17:51:50,550][20347] Updated weights for policy 0, policy_version 1763652 (0.0008) [2023-12-27 17:51:50,572][20381] Updated weights for policy 1, policy_version 1762148 (0.0008) [2023-12-27 17:51:50,609][20347] Updated weights for policy 0, policy_version 1763662 (0.0008) [2023-12-27 17:51:50,636][20381] Updated weights for policy 1, policy_version 1762158 (0.0008) [2023-12-27 17:51:50,666][20347] Updated weights for policy 0, policy_version 1763672 (0.0007) [2023-12-27 17:51:50,698][20381] Updated weights for policy 1, policy_version 1762168 (0.0006) [2023-12-27 17:51:51,373][20381] Updated weights for policy 1, policy_version 1762178 (0.0007) [2023-12-27 17:51:51,380][20347] Updated weights for policy 0, policy_version 1763682 (0.0007) [2023-12-27 17:51:51,433][20381] Updated weights for policy 1, policy_version 1762188 (0.0008) [2023-12-27 17:51:51,443][20347] Updated weights for policy 0, policy_version 1763692 (0.0006) [2023-12-27 17:51:51,494][20381] Updated weights for policy 1, policy_version 1762198 (0.0008) [2023-12-27 17:51:51,500][20347] Updated weights for policy 0, policy_version 1763702 (0.0005) [2023-12-27 17:51:51,556][20381] Updated weights for policy 1, policy_version 1762208 (0.0008) [2023-12-27 17:51:51,558][20347] Updated weights for policy 0, policy_version 1763712 (0.0005) [2023-12-27 17:51:52,224][20347] Updated weights for policy 0, policy_version 1763722 (0.0007) [2023-12-27 17:51:52,228][20381] Updated weights for policy 1, policy_version 1762218 (0.0007) [2023-12-27 17:51:52,280][20347] Updated weights for policy 0, policy_version 1763732 (0.0008) [2023-12-27 17:51:52,286][20381] Updated weights for policy 1, policy_version 1762228 (0.0006) [2023-12-27 17:51:52,333][20347] Updated weights for policy 0, policy_version 1763742 (0.0006) [2023-12-27 17:51:52,346][20381] Updated weights for policy 1, policy_version 1762238 (0.0006) [2023-12-27 17:51:52,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 902782976. Throughput: 0: 10251.6, 1: 10310.1. Samples: 902746144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:52,363][19170] Avg episode reward: [(0, '696.980'), (1, '1000.000')] [2023-12-27 17:51:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001762240_451182592.pth... [2023-12-27 17:51:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001763744_451600384.pth... [2023-12-27 17:51:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001761024_450871296.pth [2023-12-27 17:51:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001762528_451289088.pth [2023-12-27 17:51:53,011][20347] Updated weights for policy 0, policy_version 1763752 (0.0007) [2023-12-27 17:51:53,060][20381] Updated weights for policy 1, policy_version 1762248 (0.0007) [2023-12-27 17:51:53,062][20347] Updated weights for policy 0, policy_version 1763762 (0.0006) [2023-12-27 17:51:53,116][20347] Updated weights for policy 0, policy_version 1763772 (0.0008) [2023-12-27 17:51:53,126][20381] Updated weights for policy 1, policy_version 1762258 (0.0007) [2023-12-27 17:51:53,189][20381] Updated weights for policy 1, policy_version 1762268 (0.0005) [2023-12-27 17:51:53,826][20347] Updated weights for policy 0, policy_version 1763782 (0.0006) [2023-12-27 17:51:53,846][20381] Updated weights for policy 1, policy_version 1762278 (0.0007) [2023-12-27 17:51:53,878][20347] Updated weights for policy 0, policy_version 1763792 (0.0006) [2023-12-27 17:51:53,893][20381] Updated weights for policy 1, policy_version 1762288 (0.0008) [2023-12-27 17:51:53,933][20347] Updated weights for policy 0, policy_version 1763802 (0.0006) [2023-12-27 17:51:53,942][20381] Updated weights for policy 1, policy_version 1762298 (0.0010) [2023-12-27 17:51:54,550][20347] Updated weights for policy 0, policy_version 1763812 (0.0007) [2023-12-27 17:51:54,606][20347] Updated weights for policy 0, policy_version 1763822 (0.0010) [2023-12-27 17:51:54,616][20381] Updated weights for policy 1, policy_version 1762308 (0.0009) [2023-12-27 17:51:54,670][20347] Updated weights for policy 0, policy_version 1763832 (0.0005) [2023-12-27 17:51:54,671][20381] Updated weights for policy 1, policy_version 1762318 (0.0010) [2023-12-27 17:51:54,725][20381] Updated weights for policy 1, policy_version 1762328 (0.0011) [2023-12-27 17:51:55,268][20347] Updated weights for policy 0, policy_version 1763842 (0.0008) [2023-12-27 17:51:55,317][20347] Updated weights for policy 0, policy_version 1763852 (0.0007) [2023-12-27 17:51:55,368][20347] Updated weights for policy 0, policy_version 1763862 (0.0007) [2023-12-27 17:51:55,414][20347] Updated weights for policy 0, policy_version 1763872 (0.0007) [2023-12-27 17:51:55,426][20381] Updated weights for policy 1, policy_version 1762338 (0.0011) [2023-12-27 17:51:55,480][20381] Updated weights for policy 1, policy_version 1762348 (0.0010) [2023-12-27 17:51:55,538][20381] Updated weights for policy 1, policy_version 1762358 (0.0010) [2023-12-27 17:51:55,592][20381] Updated weights for policy 1, policy_version 1762368 (0.0010) [2023-12-27 17:51:56,020][20347] Updated weights for policy 0, policy_version 1763882 (0.0008) [2023-12-27 17:51:56,073][20347] Updated weights for policy 0, policy_version 1763892 (0.0006) [2023-12-27 17:51:56,137][20347] Updated weights for policy 0, policy_version 1763902 (0.0006) [2023-12-27 17:51:56,271][20381] Updated weights for policy 1, policy_version 1762378 (0.0010) [2023-12-27 17:51:56,324][20381] Updated weights for policy 1, policy_version 1762388 (0.0010) [2023-12-27 17:51:56,381][20381] Updated weights for policy 1, policy_version 1762398 (0.0010) [2023-12-27 17:51:56,786][20347] Updated weights for policy 0, policy_version 1763912 (0.0007) [2023-12-27 17:51:56,839][20347] Updated weights for policy 0, policy_version 1763922 (0.0007) [2023-12-27 17:51:56,892][20347] Updated weights for policy 0, policy_version 1763932 (0.0007) [2023-12-27 17:51:57,026][20381] Updated weights for policy 1, policy_version 1762408 (0.0010) [2023-12-27 17:51:57,074][20381] Updated weights for policy 1, policy_version 1762418 (0.0010) [2023-12-27 17:51:57,130][20381] Updated weights for policy 1, policy_version 1762428 (0.0010) [2023-12-27 17:51:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 902881280. Throughput: 0: 10281.8, 1: 10314.4. Samples: 902869364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:51:57,363][19170] Avg episode reward: [(0, '824.910'), (1, '1000.000')] [2023-12-27 17:51:57,533][20347] Updated weights for policy 0, policy_version 1763942 (0.0007) [2023-12-27 17:51:57,593][20347] Updated weights for policy 0, policy_version 1763952 (0.0005) [2023-12-27 17:51:57,655][20347] Updated weights for policy 0, policy_version 1763962 (0.0006) [2023-12-27 17:51:57,808][20381] Updated weights for policy 1, policy_version 1762438 (0.0010) [2023-12-27 17:51:57,857][20381] Updated weights for policy 1, policy_version 1762448 (0.0010) [2023-12-27 17:51:57,902][20381] Updated weights for policy 1, policy_version 1762458 (0.0010) [2023-12-27 17:51:58,240][20347] Updated weights for policy 0, policy_version 1763972 (0.0008) [2023-12-27 17:51:58,303][20347] Updated weights for policy 0, policy_version 1763982 (0.0008) [2023-12-27 17:51:58,375][20347] Updated weights for policy 0, policy_version 1763992 (0.0009) [2023-12-27 17:51:58,730][20381] Updated weights for policy 1, policy_version 1762468 (0.0009) [2023-12-27 17:51:58,807][20381] Updated weights for policy 1, policy_version 1762478 (0.0008) [2023-12-27 17:51:58,875][20381] Updated weights for policy 1, policy_version 1762488 (0.0007) [2023-12-27 17:51:59,155][20347] Updated weights for policy 0, policy_version 1764002 (0.0008) [2023-12-27 17:51:59,214][20347] Updated weights for policy 0, policy_version 1764012 (0.0008) [2023-12-27 17:51:59,281][20347] Updated weights for policy 0, policy_version 1764022 (0.0009) [2023-12-27 17:51:59,349][20347] Updated weights for policy 0, policy_version 1764032 (0.0007) [2023-12-27 17:51:59,606][20381] Updated weights for policy 1, policy_version 1762498 (0.0008) [2023-12-27 17:51:59,665][20381] Updated weights for policy 1, policy_version 1762508 (0.0005) [2023-12-27 17:51:59,722][20381] Updated weights for policy 1, policy_version 1762518 (0.0005) [2023-12-27 17:51:59,779][20381] Updated weights for policy 1, policy_version 1762528 (0.0006) [2023-12-27 17:51:59,978][20347] Updated weights for policy 0, policy_version 1764042 (0.0007) [2023-12-27 17:52:00,017][20291] KL-divergence is very high: 136.2260 [2023-12-27 17:52:00,038][20347] Updated weights for policy 0, policy_version 1764052 (0.0006) [2023-12-27 17:52:00,095][20347] Updated weights for policy 0, policy_version 1764062 (0.0007) [2023-12-27 17:52:00,454][20381] Updated weights for policy 1, policy_version 1762538 (0.0007) [2023-12-27 17:52:00,503][20381] Updated weights for policy 1, policy_version 1762548 (0.0007) [2023-12-27 17:52:00,548][20381] Updated weights for policy 1, policy_version 1762558 (0.0007) [2023-12-27 17:52:00,735][20347] Updated weights for policy 0, policy_version 1764072 (0.0006) [2023-12-27 17:52:00,786][20347] Updated weights for policy 0, policy_version 1764082 (0.0005) [2023-12-27 17:52:00,839][20347] Updated weights for policy 0, policy_version 1764092 (0.0006) [2023-12-27 17:52:01,228][20381] Updated weights for policy 1, policy_version 1762568 (0.0010) [2023-12-27 17:52:01,286][20381] Updated weights for policy 1, policy_version 1762578 (0.0008) [2023-12-27 17:52:01,337][20381] Updated weights for policy 1, policy_version 1762588 (0.0007) [2023-12-27 17:52:01,557][20347] Updated weights for policy 0, policy_version 1764102 (0.0007) [2023-12-27 17:52:01,620][20347] Updated weights for policy 0, policy_version 1764112 (0.0006) [2023-12-27 17:52:01,682][20347] Updated weights for policy 0, policy_version 1764122 (0.0009) [2023-12-27 17:52:01,993][20381] Updated weights for policy 1, policy_version 1762598 (0.0008) [2023-12-27 17:52:02,045][20381] Updated weights for policy 1, policy_version 1762608 (0.0008) [2023-12-27 17:52:02,097][20381] Updated weights for policy 1, policy_version 1762618 (0.0008) [2023-12-27 17:52:02,264][20347] Updated weights for policy 0, policy_version 1764132 (0.0008) [2023-12-27 17:52:02,324][20347] Updated weights for policy 0, policy_version 1764142 (0.0006) [2023-12-27 17:52:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 902979584. Throughput: 0: 10296.2, 1: 10283.7. Samples: 902991452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:02,363][19170] Avg episode reward: [(0, '794.770'), (1, '1000.000')] [2023-12-27 17:52:02,392][20347] Updated weights for policy 0, policy_version 1764152 (0.0008) [2023-12-27 17:52:02,758][20381] Updated weights for policy 1, policy_version 1762628 (0.0008) [2023-12-27 17:52:02,813][20381] Updated weights for policy 1, policy_version 1762638 (0.0011) [2023-12-27 17:52:02,869][20381] Updated weights for policy 1, policy_version 1762648 (0.0011) [2023-12-27 17:52:03,073][20347] Updated weights for policy 0, policy_version 1764162 (0.0007) [2023-12-27 17:52:03,130][20347] Updated weights for policy 0, policy_version 1764172 (0.0010) [2023-12-27 17:52:03,188][20347] Updated weights for policy 0, policy_version 1764182 (0.0010) [2023-12-27 17:52:03,243][20347] Updated weights for policy 0, policy_version 1764192 (0.0010) [2023-12-27 17:52:03,498][20381] Updated weights for policy 1, policy_version 1762658 (0.0010) [2023-12-27 17:52:03,550][20381] Updated weights for policy 1, policy_version 1762668 (0.0011) [2023-12-27 17:52:03,604][20381] Updated weights for policy 1, policy_version 1762678 (0.0010) [2023-12-27 17:52:03,662][20381] Updated weights for policy 1, policy_version 1762688 (0.0010) [2023-12-27 17:52:03,897][20347] Updated weights for policy 0, policy_version 1764202 (0.0009) [2023-12-27 17:52:03,959][20347] Updated weights for policy 0, policy_version 1764212 (0.0007) [2023-12-27 17:52:04,023][20347] Updated weights for policy 0, policy_version 1764222 (0.0005) [2023-12-27 17:52:04,356][20381] Updated weights for policy 1, policy_version 1762698 (0.0008) [2023-12-27 17:52:04,409][20381] Updated weights for policy 1, policy_version 1762708 (0.0007) [2023-12-27 17:52:04,459][20381] Updated weights for policy 1, policy_version 1762718 (0.0008) [2023-12-27 17:52:04,705][20347] Updated weights for policy 0, policy_version 1764232 (0.0007) [2023-12-27 17:52:04,762][20347] Updated weights for policy 0, policy_version 1764242 (0.0010) [2023-12-27 17:52:04,813][20347] Updated weights for policy 0, policy_version 1764252 (0.0010) [2023-12-27 17:52:05,139][20381] Updated weights for policy 1, policy_version 1762728 (0.0006) [2023-12-27 17:52:05,191][20381] Updated weights for policy 1, policy_version 1762738 (0.0009) [2023-12-27 17:52:05,249][20381] Updated weights for policy 1, policy_version 1762748 (0.0007) [2023-12-27 17:52:05,459][20347] Updated weights for policy 0, policy_version 1764262 (0.0009) [2023-12-27 17:52:05,483][20291] KL-divergence is very high: 113.7952 [2023-12-27 17:52:05,495][20291] KL-divergence is very high: 150.0827 [2023-12-27 17:52:05,517][20347] Updated weights for policy 0, policy_version 1764272 (0.0007) [2023-12-27 17:52:05,532][20291] KL-divergence is very high: 171.6401 [2023-12-27 17:52:05,547][20291] KL-divergence is very high: 150.4487 [2023-12-27 17:52:05,578][20347] Updated weights for policy 0, policy_version 1764282 (0.0008) [2023-12-27 17:52:05,843][20381] Updated weights for policy 1, policy_version 1762758 (0.0007) [2023-12-27 17:52:05,892][20381] Updated weights for policy 1, policy_version 1762768 (0.0007) [2023-12-27 17:52:05,936][20381] Updated weights for policy 1, policy_version 1762778 (0.0005) [2023-12-27 17:52:06,223][20347] Updated weights for policy 0, policy_version 1764292 (0.0007) [2023-12-27 17:52:06,285][20347] Updated weights for policy 0, policy_version 1764302 (0.0008) [2023-12-27 17:52:06,337][20347] Updated weights for policy 0, policy_version 1764312 (0.0008) [2023-12-27 17:52:06,564][20381] Updated weights for policy 1, policy_version 1762788 (0.0005) [2023-12-27 17:52:06,622][20381] Updated weights for policy 1, policy_version 1762798 (0.0007) [2023-12-27 17:52:06,680][20381] Updated weights for policy 1, policy_version 1762808 (0.0007) [2023-12-27 17:52:07,009][20347] Updated weights for policy 0, policy_version 1764322 (0.0009) [2023-12-27 17:52:07,062][20347] Updated weights for policy 0, policy_version 1764332 (0.0009) [2023-12-27 17:52:07,112][20347] Updated weights for policy 0, policy_version 1764342 (0.0010) [2023-12-27 17:52:07,174][20347] Updated weights for policy 0, policy_version 1764352 (0.0010) [2023-12-27 17:52:07,312][20381] Updated weights for policy 1, policy_version 1762818 (0.0006) [2023-12-27 17:52:07,362][20381] Updated weights for policy 1, policy_version 1762828 (0.0005) [2023-12-27 17:52:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 903086080. Throughput: 0: 10357.3, 1: 10297.9. Samples: 903054192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:07,363][19170] Avg episode reward: [(0, '706.930'), (1, '1000.000')] [2023-12-27 17:52:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001764352_451756032.pth... [2023-12-27 17:52:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001763136_451444736.pth [2023-12-27 17:52:07,410][20381] Updated weights for policy 1, policy_version 1762838 (0.0008) [2023-12-27 17:52:07,473][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001762848_451338240.pth... [2023-12-27 17:52:07,474][20381] Updated weights for policy 1, policy_version 1762848 (0.0008) [2023-12-27 17:52:07,477][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001761632_451026944.pth [2023-12-27 17:52:07,758][20347] Updated weights for policy 0, policy_version 1764362 (0.0011) [2023-12-27 17:52:07,807][20347] Updated weights for policy 0, policy_version 1764372 (0.0010) [2023-12-27 17:52:07,864][20347] Updated weights for policy 0, policy_version 1764382 (0.0010) [2023-12-27 17:52:08,143][20381] Updated weights for policy 1, policy_version 1762858 (0.0006) [2023-12-27 17:52:08,199][20381] Updated weights for policy 1, policy_version 1762868 (0.0008) [2023-12-27 17:52:08,248][20381] Updated weights for policy 1, policy_version 1762878 (0.0008) [2023-12-27 17:52:08,604][20347] Updated weights for policy 0, policy_version 1764392 (0.0008) [2023-12-27 17:52:08,667][20347] Updated weights for policy 0, policy_version 1764402 (0.0008) [2023-12-27 17:52:08,726][20347] Updated weights for policy 0, policy_version 1764412 (0.0007) [2023-12-27 17:52:08,977][20381] Updated weights for policy 1, policy_version 1762888 (0.0008) [2023-12-27 17:52:09,025][20381] Updated weights for policy 1, policy_version 1762898 (0.0007) [2023-12-27 17:52:09,082][20381] Updated weights for policy 1, policy_version 1762908 (0.0007) [2023-12-27 17:52:09,398][20347] Updated weights for policy 0, policy_version 1764422 (0.0008) [2023-12-27 17:52:09,457][20347] Updated weights for policy 0, policy_version 1764432 (0.0011) [2023-12-27 17:52:09,513][20347] Updated weights for policy 0, policy_version 1764442 (0.0011) [2023-12-27 17:52:09,847][20381] Updated weights for policy 1, policy_version 1762918 (0.0007) [2023-12-27 17:52:09,907][20381] Updated weights for policy 1, policy_version 1762928 (0.0007) [2023-12-27 17:52:09,967][20381] Updated weights for policy 1, policy_version 1762938 (0.0008) [2023-12-27 17:52:10,228][20347] Updated weights for policy 0, policy_version 1764452 (0.0010) [2023-12-27 17:52:10,284][20347] Updated weights for policy 0, policy_version 1764462 (0.0010) [2023-12-27 17:52:10,348][20347] Updated weights for policy 0, policy_version 1764472 (0.0010) [2023-12-27 17:52:10,681][20381] Updated weights for policy 1, policy_version 1762948 (0.0007) [2023-12-27 17:52:10,743][20381] Updated weights for policy 1, policy_version 1762958 (0.0007) [2023-12-27 17:52:10,806][20381] Updated weights for policy 1, policy_version 1762968 (0.0007) [2023-12-27 17:52:10,990][20347] Updated weights for policy 0, policy_version 1764482 (0.0010) [2023-12-27 17:52:11,053][20347] Updated weights for policy 0, policy_version 1764492 (0.0007) [2023-12-27 17:52:11,116][20347] Updated weights for policy 0, policy_version 1764502 (0.0010) [2023-12-27 17:52:11,184][20347] Updated weights for policy 0, policy_version 1764512 (0.0008) [2023-12-27 17:52:11,534][20381] Updated weights for policy 1, policy_version 1762978 (0.0008) [2023-12-27 17:52:11,595][20381] Updated weights for policy 1, policy_version 1762988 (0.0009) [2023-12-27 17:52:11,660][20381] Updated weights for policy 1, policy_version 1762998 (0.0007) [2023-12-27 17:52:11,724][20381] Updated weights for policy 1, policy_version 1763008 (0.0008) [2023-12-27 17:52:11,885][20347] Updated weights for policy 0, policy_version 1764522 (0.0010) [2023-12-27 17:52:11,941][20347] Updated weights for policy 0, policy_version 1764532 (0.0011) [2023-12-27 17:52:11,994][20347] Updated weights for policy 0, policy_version 1764542 (0.0010) [2023-12-27 17:52:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 903184384. Throughput: 0: 10392.9, 1: 10300.9. Samples: 903177848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:12,363][19170] Avg episode reward: [(0, '687.850'), (1, '1000.000')] [2023-12-27 17:52:12,388][20381] Updated weights for policy 1, policy_version 1763018 (0.0008) [2023-12-27 17:52:12,446][20381] Updated weights for policy 1, policy_version 1763028 (0.0008) [2023-12-27 17:52:12,498][20381] Updated weights for policy 1, policy_version 1763038 (0.0008) [2023-12-27 17:52:12,698][20347] Updated weights for policy 0, policy_version 1764552 (0.0008) [2023-12-27 17:52:12,752][20347] Updated weights for policy 0, policy_version 1764562 (0.0010) [2023-12-27 17:52:12,812][20347] Updated weights for policy 0, policy_version 1764572 (0.0011) [2023-12-27 17:52:13,235][20381] Updated weights for policy 1, policy_version 1763048 (0.0008) [2023-12-27 17:52:13,291][20381] Updated weights for policy 1, policy_version 1763058 (0.0007) [2023-12-27 17:52:13,336][20381] Updated weights for policy 1, policy_version 1763068 (0.0007) [2023-12-27 17:52:13,433][20347] Updated weights for policy 0, policy_version 1764582 (0.0010) [2023-12-27 17:52:13,481][20347] Updated weights for policy 0, policy_version 1764592 (0.0010) [2023-12-27 17:52:13,532][20347] Updated weights for policy 0, policy_version 1764602 (0.0010) [2023-12-27 17:52:14,070][20381] Updated weights for policy 1, policy_version 1763078 (0.0008) [2023-12-27 17:52:14,130][20381] Updated weights for policy 1, policy_version 1763088 (0.0008) [2023-12-27 17:52:14,147][20347] Updated weights for policy 0, policy_version 1764612 (0.0010) [2023-12-27 17:52:14,187][20381] Updated weights for policy 1, policy_version 1763098 (0.0007) [2023-12-27 17:52:14,207][20347] Updated weights for policy 0, policy_version 1764622 (0.0011) [2023-12-27 17:52:14,259][20347] Updated weights for policy 0, policy_version 1764632 (0.0010) [2023-12-27 17:52:14,935][20381] Updated weights for policy 1, policy_version 1763108 (0.0008) [2023-12-27 17:52:14,955][20347] Updated weights for policy 0, policy_version 1764642 (0.0010) [2023-12-27 17:52:14,994][20381] Updated weights for policy 1, policy_version 1763118 (0.0007) [2023-12-27 17:52:15,017][20347] Updated weights for policy 0, policy_version 1764652 (0.0007) [2023-12-27 17:52:15,052][20381] Updated weights for policy 1, policy_version 1763128 (0.0008) [2023-12-27 17:52:15,082][20347] Updated weights for policy 0, policy_version 1764662 (0.0007) [2023-12-27 17:52:15,150][20347] Updated weights for policy 0, policy_version 1764672 (0.0007) [2023-12-27 17:52:15,739][20347] Updated weights for policy 0, policy_version 1764682 (0.0007) [2023-12-27 17:52:15,766][20381] Updated weights for policy 1, policy_version 1763138 (0.0007) [2023-12-27 17:52:15,790][20347] Updated weights for policy 0, policy_version 1764692 (0.0007) [2023-12-27 17:52:15,822][20381] Updated weights for policy 1, policy_version 1763148 (0.0006) [2023-12-27 17:52:15,843][20347] Updated weights for policy 0, policy_version 1764702 (0.0006) [2023-12-27 17:52:15,878][20381] Updated weights for policy 1, policy_version 1763158 (0.0006) [2023-12-27 17:52:15,931][20381] Updated weights for policy 1, policy_version 1763168 (0.0008) [2023-12-27 17:52:16,541][20347] Updated weights for policy 0, policy_version 1764712 (0.0007) [2023-12-27 17:52:16,587][20347] Updated weights for policy 0, policy_version 1764722 (0.0005) [2023-12-27 17:52:16,611][20381] Updated weights for policy 1, policy_version 1763178 (0.0008) [2023-12-27 17:52:16,641][20347] Updated weights for policy 0, policy_version 1764732 (0.0005) [2023-12-27 17:52:16,665][20381] Updated weights for policy 1, policy_version 1763188 (0.0008) [2023-12-27 17:52:16,723][20381] Updated weights for policy 1, policy_version 1763198 (0.0008) [2023-12-27 17:52:17,264][20347] Updated weights for policy 0, policy_version 1764742 (0.0006) [2023-12-27 17:52:17,323][20347] Updated weights for policy 0, policy_version 1764752 (0.0007) [2023-12-27 17:52:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 903282688. Throughput: 0: 10390.6, 1: 10170.2. Samples: 903299108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:17,363][19170] Avg episode reward: [(0, '795.000'), (1, '1000.000')] [2023-12-27 17:52:17,380][20347] Updated weights for policy 0, policy_version 1764762 (0.0007) [2023-12-27 17:52:17,417][20381] Updated weights for policy 1, policy_version 1763208 (0.0007) [2023-12-27 17:52:17,467][20381] Updated weights for policy 1, policy_version 1763218 (0.0008) [2023-12-27 17:52:17,519][20381] Updated weights for policy 1, policy_version 1763228 (0.0007) [2023-12-27 17:52:18,111][20347] Updated weights for policy 0, policy_version 1764772 (0.0006) [2023-12-27 17:52:18,116][20381] Updated weights for policy 1, policy_version 1763238 (0.0008) [2023-12-27 17:52:18,163][20347] Updated weights for policy 0, policy_version 1764782 (0.0005) [2023-12-27 17:52:18,169][20381] Updated weights for policy 1, policy_version 1763248 (0.0010) [2023-12-27 17:52:18,209][20347] Updated weights for policy 0, policy_version 1764792 (0.0005) [2023-12-27 17:52:18,213][20381] Updated weights for policy 1, policy_version 1763258 (0.0009) [2023-12-27 17:52:18,925][20381] Updated weights for policy 1, policy_version 1763268 (0.0007) [2023-12-27 17:52:18,930][20347] Updated weights for policy 0, policy_version 1764802 (0.0007) [2023-12-27 17:52:18,977][20347] Updated weights for policy 0, policy_version 1764812 (0.0006) [2023-12-27 17:52:18,982][20381] Updated weights for policy 1, policy_version 1763278 (0.0008) [2023-12-27 17:52:19,033][20347] Updated weights for policy 0, policy_version 1764822 (0.0007) [2023-12-27 17:52:19,036][20381] Updated weights for policy 1, policy_version 1763288 (0.0006) [2023-12-27 17:52:19,088][20347] Updated weights for policy 0, policy_version 1764832 (0.0007) [2023-12-27 17:52:19,709][20381] Updated weights for policy 1, policy_version 1763298 (0.0008) [2023-12-27 17:52:19,762][20381] Updated weights for policy 1, policy_version 1763308 (0.0007) [2023-12-27 17:52:19,814][20381] Updated weights for policy 1, policy_version 1763318 (0.0008) [2023-12-27 17:52:19,840][20347] Updated weights for policy 0, policy_version 1764842 (0.0006) [2023-12-27 17:52:19,880][20381] Updated weights for policy 1, policy_version 1763328 (0.0008) [2023-12-27 17:52:19,902][20347] Updated weights for policy 0, policy_version 1764852 (0.0008) [2023-12-27 17:52:19,964][20347] Updated weights for policy 0, policy_version 1764862 (0.0008) [2023-12-27 17:52:20,569][20381] Updated weights for policy 1, policy_version 1763338 (0.0007) [2023-12-27 17:52:20,631][20381] Updated weights for policy 1, policy_version 1763348 (0.0008) [2023-12-27 17:52:20,681][20347] Updated weights for policy 0, policy_version 1764872 (0.0008) [2023-12-27 17:52:20,690][20381] Updated weights for policy 1, policy_version 1763358 (0.0009) [2023-12-27 17:52:20,740][20347] Updated weights for policy 0, policy_version 1764882 (0.0007) [2023-12-27 17:52:20,797][20347] Updated weights for policy 0, policy_version 1764892 (0.0008) [2023-12-27 17:52:21,422][20381] Updated weights for policy 1, policy_version 1763368 (0.0008) [2023-12-27 17:52:21,477][20381] Updated weights for policy 1, policy_version 1763378 (0.0007) [2023-12-27 17:52:21,501][20347] Updated weights for policy 0, policy_version 1764902 (0.0007) [2023-12-27 17:52:21,537][20381] Updated weights for policy 1, policy_version 1763388 (0.0006) [2023-12-27 17:52:21,558][20347] Updated weights for policy 0, policy_version 1764912 (0.0006) [2023-12-27 17:52:21,623][20347] Updated weights for policy 0, policy_version 1764922 (0.0008) [2023-12-27 17:52:22,293][20381] Updated weights for policy 1, policy_version 1763398 (0.0010) [2023-12-27 17:52:22,331][20347] Updated weights for policy 0, policy_version 1764932 (0.0008) [2023-12-27 17:52:22,351][20381] Updated weights for policy 1, policy_version 1763408 (0.0011) [2023-12-27 17:52:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 903380992. Throughput: 0: 10375.0, 1: 10123.5. Samples: 903359444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:22,363][19170] Avg episode reward: [(0, '805.840'), (1, '1000.000')] [2023-12-27 17:52:22,395][20347] Updated weights for policy 0, policy_version 1764942 (0.0010) [2023-12-27 17:52:22,416][20381] Updated weights for policy 1, policy_version 1763418 (0.0009) [2023-12-27 17:52:22,445][20347] Updated weights for policy 0, policy_version 1764952 (0.0010) [2023-12-27 17:52:22,452][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001763424_451485696.pth... [2023-12-27 17:52:22,456][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001762240_451182592.pth [2023-12-27 17:52:22,484][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001764960_451911680.pth... [2023-12-27 17:52:22,488][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001763744_451600384.pth [2023-12-27 17:52:23,098][20381] Updated weights for policy 1, policy_version 1763428 (0.0007) [2023-12-27 17:52:23,157][20381] Updated weights for policy 1, policy_version 1763438 (0.0007) [2023-12-27 17:52:23,192][20347] Updated weights for policy 0, policy_version 1764962 (0.0010) [2023-12-27 17:52:23,213][20381] Updated weights for policy 1, policy_version 1763448 (0.0007) [2023-12-27 17:52:23,249][20347] Updated weights for policy 0, policy_version 1764972 (0.0010) [2023-12-27 17:52:23,307][20347] Updated weights for policy 0, policy_version 1764982 (0.0010) [2023-12-27 17:52:23,366][20347] Updated weights for policy 0, policy_version 1764992 (0.0010) [2023-12-27 17:52:23,844][20381] Updated weights for policy 1, policy_version 1763458 (0.0008) [2023-12-27 17:52:23,903][20381] Updated weights for policy 1, policy_version 1763468 (0.0011) [2023-12-27 17:52:23,962][20381] Updated weights for policy 1, policy_version 1763478 (0.0010) [2023-12-27 17:52:24,024][20381] Updated weights for policy 1, policy_version 1763488 (0.0011) [2023-12-27 17:52:24,037][20347] Updated weights for policy 0, policy_version 1765002 (0.0010) [2023-12-27 17:52:24,095][20347] Updated weights for policy 0, policy_version 1765012 (0.0010) [2023-12-27 17:52:24,150][20347] Updated weights for policy 0, policy_version 1765022 (0.0010) [2023-12-27 17:52:24,614][20381] Updated weights for policy 1, policy_version 1763498 (0.0007) [2023-12-27 17:52:24,672][20381] Updated weights for policy 1, policy_version 1763508 (0.0010) [2023-12-27 17:52:24,721][20381] Updated weights for policy 1, policy_version 1763518 (0.0010) [2023-12-27 17:52:24,846][20347] Updated weights for policy 0, policy_version 1765032 (0.0008) [2023-12-27 17:52:24,894][20347] Updated weights for policy 0, policy_version 1765042 (0.0006) [2023-12-27 17:52:24,946][20347] Updated weights for policy 0, policy_version 1765052 (0.0008) [2023-12-27 17:52:25,347][20381] Updated weights for policy 1, policy_version 1763528 (0.0008) [2023-12-27 17:52:25,407][20381] Updated weights for policy 1, policy_version 1763538 (0.0009) [2023-12-27 17:52:25,467][20381] Updated weights for policy 1, policy_version 1763548 (0.0008) [2023-12-27 17:52:25,633][20347] Updated weights for policy 0, policy_version 1765062 (0.0008) [2023-12-27 17:52:25,689][20347] Updated weights for policy 0, policy_version 1765072 (0.0007) [2023-12-27 17:52:25,748][20347] Updated weights for policy 0, policy_version 1765082 (0.0008) [2023-12-27 17:52:26,163][20381] Updated weights for policy 1, policy_version 1763558 (0.0006) [2023-12-27 17:52:26,213][20381] Updated weights for policy 1, policy_version 1763568 (0.0005) [2023-12-27 17:52:26,265][20381] Updated weights for policy 1, policy_version 1763578 (0.0005) [2023-12-27 17:52:26,388][20347] Updated weights for policy 0, policy_version 1765092 (0.0006) [2023-12-27 17:52:26,451][20347] Updated weights for policy 0, policy_version 1765102 (0.0006) [2023-12-27 17:52:26,506][20347] Updated weights for policy 0, policy_version 1765112 (0.0006) [2023-12-27 17:52:26,929][20381] Updated weights for policy 1, policy_version 1763588 (0.0006) [2023-12-27 17:52:26,980][20381] Updated weights for policy 1, policy_version 1763598 (0.0010) [2023-12-27 17:52:27,038][20381] Updated weights for policy 1, policy_version 1763608 (0.0010) [2023-12-27 17:52:27,041][20347] Updated weights for policy 0, policy_version 1765122 (0.0006) [2023-12-27 17:52:27,096][20347] Updated weights for policy 0, policy_version 1765132 (0.0008) [2023-12-27 17:52:27,149][20347] Updated weights for policy 0, policy_version 1765142 (0.0007) [2023-12-27 17:52:27,191][20347] Updated weights for policy 0, policy_version 1765152 (0.0006) [2023-12-27 17:52:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 903495680. Throughput: 0: 10335.6, 1: 10155.1. Samples: 903480900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:27,363][19170] Avg episode reward: [(0, '716.970'), (1, '1000.000')] [2023-12-27 17:52:27,697][20381] Updated weights for policy 1, policy_version 1763618 (0.0011) [2023-12-27 17:52:27,751][20381] Updated weights for policy 1, policy_version 1763628 (0.0010) [2023-12-27 17:52:27,796][20381] Updated weights for policy 1, policy_version 1763638 (0.0010) [2023-12-27 17:52:27,853][20381] Updated weights for policy 1, policy_version 1763648 (0.0010) [2023-12-27 17:52:27,889][20347] Updated weights for policy 0, policy_version 1765162 (0.0006) [2023-12-27 17:52:27,948][20347] Updated weights for policy 0, policy_version 1765172 (0.0007) [2023-12-27 17:52:27,996][20347] Updated weights for policy 0, policy_version 1765182 (0.0005) [2023-12-27 17:52:28,540][20381] Updated weights for policy 1, policy_version 1763658 (0.0011) [2023-12-27 17:52:28,588][20381] Updated weights for policy 1, policy_version 1763668 (0.0011) [2023-12-27 17:52:28,643][20381] Updated weights for policy 1, policy_version 1763678 (0.0008) [2023-12-27 17:52:28,709][20347] Updated weights for policy 0, policy_version 1765192 (0.0006) [2023-12-27 17:52:28,759][20347] Updated weights for policy 0, policy_version 1765202 (0.0007) [2023-12-27 17:52:28,792][20291] KL-divergence is very high: 136.5709 [2023-12-27 17:52:28,815][20347] Updated weights for policy 0, policy_version 1765212 (0.0007) [2023-12-27 17:52:29,298][20381] Updated weights for policy 1, policy_version 1763688 (0.0009) [2023-12-27 17:52:29,360][20381] Updated weights for policy 1, policy_version 1763698 (0.0009) [2023-12-27 17:52:29,428][20381] Updated weights for policy 1, policy_version 1763708 (0.0008) [2023-12-27 17:52:29,578][20347] Updated weights for policy 0, policy_version 1765222 (0.0007) [2023-12-27 17:52:29,641][20347] Updated weights for policy 0, policy_version 1765232 (0.0008) [2023-12-27 17:52:29,700][20347] Updated weights for policy 0, policy_version 1765242 (0.0008) [2023-12-27 17:52:30,119][20381] Updated weights for policy 1, policy_version 1763718 (0.0007) [2023-12-27 17:52:30,164][20381] Updated weights for policy 1, policy_version 1763728 (0.0007) [2023-12-27 17:52:30,221][20381] Updated weights for policy 1, policy_version 1763738 (0.0008) [2023-12-27 17:52:30,432][20347] Updated weights for policy 0, policy_version 1765252 (0.0008) [2023-12-27 17:52:30,487][20347] Updated weights for policy 0, policy_version 1765262 (0.0007) [2023-12-27 17:52:30,548][20347] Updated weights for policy 0, policy_version 1765272 (0.0007) [2023-12-27 17:52:30,941][20381] Updated weights for policy 1, policy_version 1763748 (0.0007) [2023-12-27 17:52:30,998][20381] Updated weights for policy 1, policy_version 1763758 (0.0005) [2023-12-27 17:52:31,060][20381] Updated weights for policy 1, policy_version 1763768 (0.0006) [2023-12-27 17:52:31,232][20347] Updated weights for policy 0, policy_version 1765282 (0.0007) [2023-12-27 17:52:31,293][20347] Updated weights for policy 0, policy_version 1765292 (0.0009) [2023-12-27 17:52:31,356][20347] Updated weights for policy 0, policy_version 1765302 (0.0010) [2023-12-27 17:52:31,418][20347] Updated weights for policy 0, policy_version 1765312 (0.0011) [2023-12-27 17:52:31,685][20381] Updated weights for policy 1, policy_version 1763778 (0.0007) [2023-12-27 17:52:31,750][20381] Updated weights for policy 1, policy_version 1763788 (0.0008) [2023-12-27 17:52:31,795][20381] Updated weights for policy 1, policy_version 1763798 (0.0008) [2023-12-27 17:52:31,847][20381] Updated weights for policy 1, policy_version 1763808 (0.0008) [2023-12-27 17:52:32,153][20347] Updated weights for policy 0, policy_version 1765322 (0.0010) [2023-12-27 17:52:32,214][20347] Updated weights for policy 0, policy_version 1765332 (0.0011) [2023-12-27 17:52:32,273][20347] Updated weights for policy 0, policy_version 1765342 (0.0010) [2023-12-27 17:52:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 903593984. Throughput: 0: 10288.4, 1: 10119.8. Samples: 903602860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:32,363][19170] Avg episode reward: [(0, '707.910'), (1, '1000.000')] [2023-12-27 17:52:32,530][20381] Updated weights for policy 1, policy_version 1763818 (0.0009) [2023-12-27 17:52:32,573][20381] Updated weights for policy 1, policy_version 1763828 (0.0006) [2023-12-27 17:52:32,627][20381] Updated weights for policy 1, policy_version 1763838 (0.0005) [2023-12-27 17:52:32,911][20347] Updated weights for policy 0, policy_version 1765352 (0.0007) [2023-12-27 17:52:32,956][20347] Updated weights for policy 0, policy_version 1765362 (0.0005) [2023-12-27 17:52:32,977][20291] KL-divergence is very high: 310.1994 [2023-12-27 17:52:32,981][20291] KL-divergence is very high: 344.8143 [2023-12-27 17:52:33,007][20347] Updated weights for policy 0, policy_version 1765372 (0.0008) [2023-12-27 17:52:33,016][20291] KL-divergence is very high: 425.5038 [2023-12-27 17:52:33,022][20291] KL-divergence is very high: 417.5082 [2023-12-27 17:52:33,232][20381] Updated weights for policy 1, policy_version 1763848 (0.0008) [2023-12-27 17:52:33,280][20381] Updated weights for policy 1, policy_version 1763858 (0.0007) [2023-12-27 17:52:33,333][20381] Updated weights for policy 1, policy_version 1763868 (0.0008) [2023-12-27 17:52:33,587][20347] Updated weights for policy 0, policy_version 1765382 (0.0008) [2023-12-27 17:52:33,594][20291] KL-divergence is very high: 217.3431 [2023-12-27 17:52:33,633][20291] KL-divergence is very high: 128.3205 [2023-12-27 17:52:33,640][20347] Updated weights for policy 0, policy_version 1765392 (0.0008) [2023-12-27 17:52:33,678][20291] KL-divergence is very high: 133.7440 [2023-12-27 17:52:33,696][20347] Updated weights for policy 0, policy_version 1765402 (0.0010) [2023-12-27 17:52:33,726][20291] KL-divergence is very high: 173.4814 [2023-12-27 17:52:33,938][20381] Updated weights for policy 1, policy_version 1763878 (0.0006) [2023-12-27 17:52:33,988][20381] Updated weights for policy 1, policy_version 1763888 (0.0005) [2023-12-27 17:52:34,040][20381] Updated weights for policy 1, policy_version 1763898 (0.0007) [2023-12-27 17:52:34,321][20347] Updated weights for policy 0, policy_version 1765412 (0.0009) [2023-12-27 17:52:34,388][20347] Updated weights for policy 0, policy_version 1765422 (0.0007) [2023-12-27 17:52:34,447][20347] Updated weights for policy 0, policy_version 1765432 (0.0008) [2023-12-27 17:52:34,690][20381] Updated weights for policy 1, policy_version 1763908 (0.0007) [2023-12-27 17:52:34,746][20381] Updated weights for policy 1, policy_version 1763918 (0.0006) [2023-12-27 17:52:34,798][20381] Updated weights for policy 1, policy_version 1763928 (0.0005) [2023-12-27 17:52:35,090][20347] Updated weights for policy 0, policy_version 1765442 (0.0010) [2023-12-27 17:52:35,144][20347] Updated weights for policy 0, policy_version 1765452 (0.0006) [2023-12-27 17:52:35,207][20347] Updated weights for policy 0, policy_version 1765462 (0.0008) [2023-12-27 17:52:35,269][20347] Updated weights for policy 0, policy_version 1765472 (0.0005) [2023-12-27 17:52:35,376][20381] Updated weights for policy 1, policy_version 1763938 (0.0006) [2023-12-27 17:52:35,426][20381] Updated weights for policy 1, policy_version 1763948 (0.0009) [2023-12-27 17:52:35,486][20381] Updated weights for policy 1, policy_version 1763958 (0.0008) [2023-12-27 17:52:35,537][20381] Updated weights for policy 1, policy_version 1763968 (0.0008) [2023-12-27 17:52:35,840][20347] Updated weights for policy 0, policy_version 1765482 (0.0005) [2023-12-27 17:52:35,896][20347] Updated weights for policy 0, policy_version 1765492 (0.0005) [2023-12-27 17:52:35,951][20347] Updated weights for policy 0, policy_version 1765502 (0.0005) [2023-12-27 17:52:36,269][20381] Updated weights for policy 1, policy_version 1763978 (0.0007) [2023-12-27 17:52:36,332][20381] Updated weights for policy 1, policy_version 1763988 (0.0011) [2023-12-27 17:52:36,388][20381] Updated weights for policy 1, policy_version 1763998 (0.0011) [2023-12-27 17:52:36,560][20347] Updated weights for policy 0, policy_version 1765512 (0.0007) [2023-12-27 17:52:36,621][20347] Updated weights for policy 0, policy_version 1765522 (0.0010) [2023-12-27 17:52:36,681][20347] Updated weights for policy 0, policy_version 1765532 (0.0011) [2023-12-27 17:52:37,067][20381] Updated weights for policy 1, policy_version 1764008 (0.0011) [2023-12-27 17:52:37,126][20381] Updated weights for policy 1, policy_version 1764018 (0.0011) [2023-12-27 17:52:37,187][20381] Updated weights for policy 1, policy_version 1764028 (0.0011) [2023-12-27 17:52:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 903700480. Throughput: 0: 10297.4, 1: 10222.0. Samples: 903669516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 17:52:37,363][19170] Avg episode reward: [(0, '726.950'), (1, '1000.000')] [2023-12-27 17:52:37,366][20347] Updated weights for policy 0, policy_version 1765542 (0.0010) [2023-12-27 17:52:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001764032_451641344.pth... [2023-12-27 17:52:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001762848_451338240.pth [2023-12-27 17:52:37,421][20347] Updated weights for policy 0, policy_version 1765552 (0.0010) [2023-12-27 17:52:37,478][20347] Updated weights for policy 0, policy_version 1765562 (0.0008) [2023-12-27 17:52:37,508][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001765568_452067328.pth... [2023-12-27 17:52:37,512][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001764352_451756032.pth [2023-12-27 17:52:37,854][20381] Updated weights for policy 1, policy_version 1764038 (0.0011) [2023-12-27 17:52:37,911][20381] Updated weights for policy 1, policy_version 1764048 (0.0011) [2023-12-27 17:52:37,970][20381] Updated weights for policy 1, policy_version 1764058 (0.0011) [2023-12-27 17:52:38,109][20347] Updated weights for policy 0, policy_version 1765572 (0.0009) [2023-12-27 17:52:38,167][20347] Updated weights for policy 0, policy_version 1765582 (0.0006) [2023-12-27 17:52:38,222][20347] Updated weights for policy 0, policy_version 1765592 (0.0005) [2023-12-27 17:52:38,685][20381] Updated weights for policy 1, policy_version 1764068 (0.0011) [2023-12-27 17:52:38,741][20381] Updated weights for policy 1, policy_version 1764078 (0.0011) [2023-12-27 17:52:38,796][20381] Updated weights for policy 1, policy_version 1764088 (0.0011) [2023-12-27 17:52:38,798][20347] Updated weights for policy 0, policy_version 1765602 (0.0006) [2023-12-27 17:52:38,863][20347] Updated weights for policy 0, policy_version 1765612 (0.0008) [2023-12-27 17:52:38,928][20347] Updated weights for policy 0, policy_version 1765622 (0.0008) [2023-12-27 17:52:38,989][20347] Updated weights for policy 0, policy_version 1765632 (0.0008) [2023-12-27 17:52:39,471][20381] Updated weights for policy 1, policy_version 1764098 (0.0010) [2023-12-27 17:52:39,535][20381] Updated weights for policy 1, policy_version 1764108 (0.0008) [2023-12-27 17:52:39,598][20381] Updated weights for policy 1, policy_version 1764118 (0.0008) [2023-12-27 17:52:39,639][20347] Updated weights for policy 0, policy_version 1765642 (0.0008) [2023-12-27 17:52:39,657][20381] Updated weights for policy 1, policy_version 1764128 (0.0006) [2023-12-27 17:52:39,703][20347] Updated weights for policy 0, policy_version 1765652 (0.0007) [2023-12-27 17:52:39,762][20347] Updated weights for policy 0, policy_version 1765662 (0.0007) [2023-12-27 17:52:40,329][20381] Updated weights for policy 1, policy_version 1764138 (0.0008) [2023-12-27 17:52:40,388][20381] Updated weights for policy 1, policy_version 1764148 (0.0006) [2023-12-27 17:52:40,447][20381] Updated weights for policy 1, policy_version 1764158 (0.0007) [2023-12-27 17:52:40,481][20347] Updated weights for policy 0, policy_version 1765672 (0.0007) [2023-12-27 17:52:40,541][20347] Updated weights for policy 0, policy_version 1765682 (0.0008) [2023-12-27 17:52:40,600][20347] Updated weights for policy 0, policy_version 1765692 (0.0007) [2023-12-27 17:52:41,152][20381] Updated weights for policy 1, policy_version 1764168 (0.0008) [2023-12-27 17:52:41,210][20381] Updated weights for policy 1, policy_version 1764178 (0.0008) [2023-12-27 17:52:41,286][20381] Updated weights for policy 1, policy_version 1764190 (0.0010) [2023-12-27 17:52:41,341][20347] Updated weights for policy 0, policy_version 1765702 (0.0009) [2023-12-27 17:52:41,407][20347] Updated weights for policy 0, policy_version 1765712 (0.0009) [2023-12-27 17:52:41,460][20347] Updated weights for policy 0, policy_version 1765722 (0.0008) [2023-12-27 17:52:42,080][20381] Updated weights for policy 1, policy_version 1764200 (0.0010) [2023-12-27 17:52:42,139][20381] Updated weights for policy 1, policy_version 1764210 (0.0007) [2023-12-27 17:52:42,177][20347] Updated weights for policy 0, policy_version 1765732 (0.0009) [2023-12-27 17:52:42,193][20381] Updated weights for policy 1, policy_version 1764220 (0.0010) [2023-12-27 17:52:42,229][20347] Updated weights for policy 0, policy_version 1765742 (0.0010) [2023-12-27 17:52:42,295][20347] Updated weights for policy 0, policy_version 1765752 (0.0010) [2023-12-27 17:52:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 903806976. Throughput: 0: 10301.5, 1: 10214.7. Samples: 903792596. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:52:42,363][19170] Avg episode reward: [(0, '765.290'), (1, '1000.000')] [2023-12-27 17:52:42,816][20381] Updated weights for policy 1, policy_version 1764230 (0.0008) [2023-12-27 17:52:42,865][20381] Updated weights for policy 1, policy_version 1764240 (0.0006) [2023-12-27 17:52:42,917][20381] Updated weights for policy 1, policy_version 1764250 (0.0005) [2023-12-27 17:52:43,069][20347] Updated weights for policy 0, policy_version 1765762 (0.0007) [2023-12-27 17:52:43,121][20347] Updated weights for policy 0, policy_version 1765772 (0.0009) [2023-12-27 17:52:43,164][20347] Updated weights for policy 0, policy_version 1765782 (0.0008) [2023-12-27 17:52:43,219][20347] Updated weights for policy 0, policy_version 1765792 (0.0007) [2023-12-27 17:52:43,470][20381] Updated weights for policy 1, policy_version 1764260 (0.0005) [2023-12-27 17:52:43,515][20381] Updated weights for policy 1, policy_version 1764270 (0.0005) [2023-12-27 17:52:43,576][20381] Updated weights for policy 1, policy_version 1764280 (0.0005) [2023-12-27 17:52:43,989][20347] Updated weights for policy 0, policy_version 1765802 (0.0007) [2023-12-27 17:52:44,051][20347] Updated weights for policy 0, policy_version 1765812 (0.0005) [2023-12-27 17:52:44,102][20347] Updated weights for policy 0, policy_version 1765822 (0.0007) [2023-12-27 17:52:44,114][20381] Updated weights for policy 1, policy_version 1764290 (0.0006) [2023-12-27 17:52:44,168][20381] Updated weights for policy 1, policy_version 1764300 (0.0007) [2023-12-27 17:52:44,224][20381] Updated weights for policy 1, policy_version 1764310 (0.0007) [2023-12-27 17:52:44,281][20381] Updated weights for policy 1, policy_version 1764320 (0.0008) [2023-12-27 17:52:44,785][20347] Updated weights for policy 0, policy_version 1765832 (0.0008) [2023-12-27 17:52:44,841][20347] Updated weights for policy 0, policy_version 1765842 (0.0008) [2023-12-27 17:52:44,897][20347] Updated weights for policy 0, policy_version 1765852 (0.0008) [2023-12-27 17:52:44,970][20381] Updated weights for policy 1, policy_version 1764330 (0.0006) [2023-12-27 17:52:45,034][20381] Updated weights for policy 1, policy_version 1764340 (0.0007) [2023-12-27 17:52:45,090][20381] Updated weights for policy 1, policy_version 1764350 (0.0008) [2023-12-27 17:52:45,669][20347] Updated weights for policy 0, policy_version 1765862 (0.0008) [2023-12-27 17:52:45,723][20347] Updated weights for policy 0, policy_version 1765872 (0.0008) [2023-12-27 17:52:45,732][20381] Updated weights for policy 1, policy_version 1764360 (0.0006) [2023-12-27 17:52:45,782][20347] Updated weights for policy 0, policy_version 1765882 (0.0006) [2023-12-27 17:52:45,785][20381] Updated weights for policy 1, policy_version 1764370 (0.0006) [2023-12-27 17:52:45,838][20381] Updated weights for policy 1, policy_version 1764380 (0.0007) [2023-12-27 17:52:46,503][20347] Updated weights for policy 0, policy_version 1765892 (0.0005) [2023-12-27 17:52:46,528][20381] Updated weights for policy 1, policy_version 1764390 (0.0007) [2023-12-27 17:52:46,561][20347] Updated weights for policy 0, policy_version 1765902 (0.0009) [2023-12-27 17:52:46,577][20381] Updated weights for policy 1, policy_version 1764400 (0.0006) [2023-12-27 17:52:46,616][20347] Updated weights for policy 0, policy_version 1765912 (0.0010) [2023-12-27 17:52:46,625][20381] Updated weights for policy 1, policy_version 1764410 (0.0005) [2023-12-27 17:52:47,200][20347] Updated weights for policy 0, policy_version 1765922 (0.0010) [2023-12-27 17:52:47,251][20347] Updated weights for policy 0, policy_version 1765932 (0.0007) [2023-12-27 17:52:47,309][20347] Updated weights for policy 0, policy_version 1765942 (0.0008) [2023-12-27 17:52:47,335][20291] KL-divergence is very high: 105.0747 [2023-12-27 17:52:47,345][20381] Updated weights for policy 1, policy_version 1764420 (0.0005) [2023-12-27 17:52:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 903897088. Throughput: 0: 10226.6, 1: 10284.0. Samples: 903914428. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:52:47,363][19170] Avg episode reward: [(0, '852.470'), (1, '1000.000')] [2023-12-27 17:52:47,370][20347] Updated weights for policy 0, policy_version 1765952 (0.0007) [2023-12-27 17:52:47,401][20381] Updated weights for policy 1, policy_version 1764430 (0.0005) [2023-12-27 17:52:47,460][20381] Updated weights for policy 1, policy_version 1764440 (0.0005) [2023-12-27 17:52:48,037][20381] Updated weights for policy 1, policy_version 1764450 (0.0007) [2023-12-27 17:52:48,041][20291] KL-divergence is very high: 104.3869 [2023-12-27 17:52:48,084][20381] Updated weights for policy 1, policy_version 1764460 (0.0005) [2023-12-27 17:52:48,096][20347] Updated weights for policy 0, policy_version 1765962 (0.0010) [2023-12-27 17:52:48,136][20381] Updated weights for policy 1, policy_version 1764470 (0.0006) [2023-12-27 17:52:48,152][20347] Updated weights for policy 0, policy_version 1765972 (0.0010) [2023-12-27 17:52:48,193][20381] Updated weights for policy 1, policy_version 1764480 (0.0006) [2023-12-27 17:52:48,204][20347] Updated weights for policy 0, policy_version 1765982 (0.0010) [2023-12-27 17:52:48,822][20347] Updated weights for policy 0, policy_version 1765992 (0.0009) [2023-12-27 17:52:48,861][20381] Updated weights for policy 1, policy_version 1764490 (0.0007) [2023-12-27 17:52:48,877][20347] Updated weights for policy 0, policy_version 1766002 (0.0010) [2023-12-27 17:52:48,917][20381] Updated weights for policy 1, policy_version 1764500 (0.0006) [2023-12-27 17:52:48,933][20347] Updated weights for policy 0, policy_version 1766012 (0.0010) [2023-12-27 17:52:48,970][20381] Updated weights for policy 1, policy_version 1764510 (0.0006) [2023-12-27 17:52:49,645][20347] Updated weights for policy 0, policy_version 1766022 (0.0009) [2023-12-27 17:52:49,708][20347] Updated weights for policy 0, policy_version 1766032 (0.0007) [2023-12-27 17:52:49,730][20381] Updated weights for policy 1, policy_version 1764520 (0.0006) [2023-12-27 17:52:49,767][20347] Updated weights for policy 0, policy_version 1766042 (0.0007) [2023-12-27 17:52:49,789][20381] Updated weights for policy 1, policy_version 1764530 (0.0006) [2023-12-27 17:52:49,850][20381] Updated weights for policy 1, policy_version 1764540 (0.0007) [2023-12-27 17:52:50,444][20347] Updated weights for policy 0, policy_version 1766052 (0.0006) [2023-12-27 17:52:50,505][20347] Updated weights for policy 0, policy_version 1766062 (0.0007) [2023-12-27 17:52:50,572][20347] Updated weights for policy 0, policy_version 1766072 (0.0008) [2023-12-27 17:52:50,590][20381] Updated weights for policy 1, policy_version 1764550 (0.0009) [2023-12-27 17:52:50,651][20381] Updated weights for policy 1, policy_version 1764560 (0.0010) [2023-12-27 17:52:50,711][20381] Updated weights for policy 1, policy_version 1764570 (0.0007) [2023-12-27 17:52:51,266][20347] Updated weights for policy 0, policy_version 1766082 (0.0007) [2023-12-27 17:52:51,329][20347] Updated weights for policy 0, policy_version 1766092 (0.0010) [2023-12-27 17:52:51,395][20347] Updated weights for policy 0, policy_version 1766102 (0.0008) [2023-12-27 17:52:51,454][20347] Updated weights for policy 0, policy_version 1766112 (0.0008) [2023-12-27 17:52:51,467][20381] Updated weights for policy 1, policy_version 1764580 (0.0009) [2023-12-27 17:52:51,522][20381] Updated weights for policy 1, policy_version 1764590 (0.0008) [2023-12-27 17:52:51,583][20381] Updated weights for policy 1, policy_version 1764600 (0.0009) [2023-12-27 17:52:52,133][20347] Updated weights for policy 0, policy_version 1766122 (0.0007) [2023-12-27 17:52:52,190][20347] Updated weights for policy 0, policy_version 1766132 (0.0007) [2023-12-27 17:52:52,244][20347] Updated weights for policy 0, policy_version 1766142 (0.0007) [2023-12-27 17:52:52,317][20381] Updated weights for policy 1, policy_version 1764610 (0.0008) [2023-12-27 17:52:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 904003584. Throughput: 0: 10202.4, 1: 10262.0. Samples: 903975092. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:52:52,363][19170] Avg episode reward: [(0, '784.940'), (1, '990.290')] [2023-12-27 17:52:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001766144_452214784.pth... [2023-12-27 17:52:52,374][20381] Updated weights for policy 1, policy_version 1764620 (0.0007) [2023-12-27 17:52:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001764960_451911680.pth [2023-12-27 17:52:52,435][20381] Updated weights for policy 1, policy_version 1764630 (0.0006) [2023-12-27 17:52:52,497][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001764640_451796992.pth... [2023-12-27 17:52:52,498][20381] Updated weights for policy 1, policy_version 1764640 (0.0005) [2023-12-27 17:52:52,501][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001763424_451485696.pth [2023-12-27 17:52:52,896][20347] Updated weights for policy 0, policy_version 1766152 (0.0008) [2023-12-27 17:52:52,953][20347] Updated weights for policy 0, policy_version 1766162 (0.0007) [2023-12-27 17:52:53,006][20347] Updated weights for policy 0, policy_version 1766172 (0.0009) [2023-12-27 17:52:53,072][20381] Updated weights for policy 1, policy_version 1764650 (0.0006) [2023-12-27 17:52:53,135][20381] Updated weights for policy 1, policy_version 1764660 (0.0006) [2023-12-27 17:52:53,191][20381] Updated weights for policy 1, policy_version 1764670 (0.0008) [2023-12-27 17:52:53,690][20347] Updated weights for policy 0, policy_version 1766182 (0.0007) [2023-12-27 17:52:53,771][20347] Updated weights for policy 0, policy_version 1766192 (0.0008) [2023-12-27 17:52:53,776][20381] Updated weights for policy 1, policy_version 1764680 (0.0008) [2023-12-27 17:52:53,827][20347] Updated weights for policy 0, policy_version 1766202 (0.0006) [2023-12-27 17:52:53,833][20381] Updated weights for policy 1, policy_version 1764690 (0.0010) [2023-12-27 17:52:53,882][20381] Updated weights for policy 1, policy_version 1764700 (0.0010) [2023-12-27 17:52:54,438][20381] Updated weights for policy 1, policy_version 1764710 (0.0011) [2023-12-27 17:52:54,499][20381] Updated weights for policy 1, policy_version 1764720 (0.0008) [2023-12-27 17:52:54,521][20347] Updated weights for policy 0, policy_version 1766212 (0.0005) [2023-12-27 17:52:54,565][20381] Updated weights for policy 1, policy_version 1764730 (0.0008) [2023-12-27 17:52:54,577][20347] Updated weights for policy 0, policy_version 1766222 (0.0006) [2023-12-27 17:52:54,638][20347] Updated weights for policy 0, policy_version 1766232 (0.0010) [2023-12-27 17:52:55,179][20381] Updated weights for policy 1, policy_version 1764740 (0.0008) [2023-12-27 17:52:55,230][20381] Updated weights for policy 1, policy_version 1764750 (0.0008) [2023-12-27 17:52:55,252][20347] Updated weights for policy 0, policy_version 1766242 (0.0009) [2023-12-27 17:52:55,280][20381] Updated weights for policy 1, policy_version 1764760 (0.0008) [2023-12-27 17:52:55,304][20347] Updated weights for policy 0, policy_version 1766252 (0.0005) [2023-12-27 17:52:55,362][20347] Updated weights for policy 0, policy_version 1766262 (0.0007) [2023-12-27 17:52:55,424][20347] Updated weights for policy 0, policy_version 1766272 (0.0009) [2023-12-27 17:52:55,926][20381] Updated weights for policy 1, policy_version 1764770 (0.0008) [2023-12-27 17:52:55,977][20381] Updated weights for policy 1, policy_version 1764780 (0.0005) [2023-12-27 17:52:56,030][20381] Updated weights for policy 1, policy_version 1764790 (0.0005) [2023-12-27 17:52:56,089][20381] Updated weights for policy 1, policy_version 1764800 (0.0005) [2023-12-27 17:52:56,133][20347] Updated weights for policy 0, policy_version 1766282 (0.0007) [2023-12-27 17:52:56,196][20347] Updated weights for policy 0, policy_version 1766292 (0.0005) [2023-12-27 17:52:56,256][20347] Updated weights for policy 0, policy_version 1766302 (0.0005) [2023-12-27 17:52:56,667][20381] Updated weights for policy 1, policy_version 1764810 (0.0010) [2023-12-27 17:52:56,711][20381] Updated weights for policy 1, policy_version 1764820 (0.0010) [2023-12-27 17:52:56,762][20381] Updated weights for policy 1, policy_version 1764830 (0.0010) [2023-12-27 17:52:56,819][20347] Updated weights for policy 0, policy_version 1766312 (0.0005) [2023-12-27 17:52:56,866][20347] Updated weights for policy 0, policy_version 1766322 (0.0009) [2023-12-27 17:52:56,921][20347] Updated weights for policy 0, policy_version 1766332 (0.0009) [2023-12-27 17:52:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 904110080. Throughput: 0: 10169.1, 1: 10335.4. Samples: 904100548. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:52:57,363][19170] Avg episode reward: [(0, '746.520'), (1, '980.550')] [2023-12-27 17:52:57,467][20381] Updated weights for policy 1, policy_version 1764840 (0.0010) [2023-12-27 17:52:57,525][20381] Updated weights for policy 1, policy_version 1764850 (0.0011) [2023-12-27 17:52:57,539][20347] Updated weights for policy 0, policy_version 1766342 (0.0010) [2023-12-27 17:52:57,580][20381] Updated weights for policy 1, policy_version 1764860 (0.0011) [2023-12-27 17:52:57,591][20347] Updated weights for policy 0, policy_version 1766352 (0.0010) [2023-12-27 17:52:57,635][20347] Updated weights for policy 0, policy_version 1766362 (0.0010) [2023-12-27 17:52:58,277][20381] Updated weights for policy 1, policy_version 1764870 (0.0011) [2023-12-27 17:52:58,316][20347] Updated weights for policy 0, policy_version 1766372 (0.0010) [2023-12-27 17:52:58,337][20381] Updated weights for policy 1, policy_version 1764880 (0.0010) [2023-12-27 17:52:58,385][20347] Updated weights for policy 0, policy_version 1766382 (0.0009) [2023-12-27 17:52:58,396][20381] Updated weights for policy 1, policy_version 1764890 (0.0008) [2023-12-27 17:52:58,452][20347] Updated weights for policy 0, policy_version 1766392 (0.0008) [2023-12-27 17:52:59,166][20381] Updated weights for policy 1, policy_version 1764900 (0.0007) [2023-12-27 17:52:59,224][20381] Updated weights for policy 1, policy_version 1764910 (0.0007) [2023-12-27 17:52:59,237][20347] Updated weights for policy 0, policy_version 1766402 (0.0008) [2023-12-27 17:52:59,289][20381] Updated weights for policy 1, policy_version 1764920 (0.0007) [2023-12-27 17:52:59,300][20347] Updated weights for policy 0, policy_version 1766412 (0.0010) [2023-12-27 17:52:59,356][20347] Updated weights for policy 0, policy_version 1766422 (0.0008) [2023-12-27 17:52:59,420][20347] Updated weights for policy 0, policy_version 1766432 (0.0008) [2023-12-27 17:53:00,008][20381] Updated weights for policy 1, policy_version 1764930 (0.0008) [2023-12-27 17:53:00,068][20381] Updated weights for policy 1, policy_version 1764940 (0.0011) [2023-12-27 17:53:00,128][20381] Updated weights for policy 1, policy_version 1764950 (0.0011) [2023-12-27 17:53:00,171][20347] Updated weights for policy 0, policy_version 1766442 (0.0008) [2023-12-27 17:53:00,183][20381] Updated weights for policy 1, policy_version 1764960 (0.0011) [2023-12-27 17:53:00,228][20347] Updated weights for policy 0, policy_version 1766452 (0.0007) [2023-12-27 17:53:00,288][20347] Updated weights for policy 0, policy_version 1766462 (0.0007) [2023-12-27 17:53:00,819][20381] Updated weights for policy 1, policy_version 1764970 (0.0010) [2023-12-27 17:53:00,875][20381] Updated weights for policy 1, policy_version 1764980 (0.0008) [2023-12-27 17:53:00,930][20381] Updated weights for policy 1, policy_version 1764990 (0.0010) [2023-12-27 17:53:00,951][20347] Updated weights for policy 0, policy_version 1766472 (0.0009) [2023-12-27 17:53:01,008][20347] Updated weights for policy 0, policy_version 1766482 (0.0010) [2023-12-27 17:53:01,070][20347] Updated weights for policy 0, policy_version 1766492 (0.0009) [2023-12-27 17:53:01,673][20381] Updated weights for policy 1, policy_version 1765000 (0.0009) [2023-12-27 17:53:01,731][20381] Updated weights for policy 1, policy_version 1765010 (0.0007) [2023-12-27 17:53:01,798][20381] Updated weights for policy 1, policy_version 1765020 (0.0009) [2023-12-27 17:53:01,801][20347] Updated weights for policy 0, policy_version 1766502 (0.0007) [2023-12-27 17:53:01,867][20347] Updated weights for policy 0, policy_version 1766512 (0.0005) [2023-12-27 17:53:01,927][20347] Updated weights for policy 0, policy_version 1766522 (0.0006) [2023-12-27 17:53:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 904208384. Throughput: 0: 10117.3, 1: 10349.9. Samples: 904220132. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:02,363][19170] Avg episode reward: [(0, '795.010'), (1, '1000.000')] [2023-12-27 17:53:02,485][20381] Updated weights for policy 1, policy_version 1765030 (0.0006) [2023-12-27 17:53:02,544][20381] Updated weights for policy 1, policy_version 1765040 (0.0005) [2023-12-27 17:53:02,600][20381] Updated weights for policy 1, policy_version 1765050 (0.0005) [2023-12-27 17:53:02,612][20347] Updated weights for policy 0, policy_version 1766532 (0.0008) [2023-12-27 17:53:02,665][20347] Updated weights for policy 0, policy_version 1766542 (0.0010) [2023-12-27 17:53:02,716][20347] Updated weights for policy 0, policy_version 1766552 (0.0010) [2023-12-27 17:53:03,271][20381] Updated weights for policy 1, policy_version 1765060 (0.0007) [2023-12-27 17:53:03,319][20347] Updated weights for policy 0, policy_version 1766562 (0.0009) [2023-12-27 17:53:03,322][20381] Updated weights for policy 1, policy_version 1765070 (0.0007) [2023-12-27 17:53:03,367][20381] Updated weights for policy 1, policy_version 1765080 (0.0008) [2023-12-27 17:53:03,379][20347] Updated weights for policy 0, policy_version 1766572 (0.0005) [2023-12-27 17:53:03,434][20347] Updated weights for policy 0, policy_version 1766582 (0.0005) [2023-12-27 17:53:03,493][20347] Updated weights for policy 0, policy_version 1766592 (0.0009) [2023-12-27 17:53:04,087][20381] Updated weights for policy 1, policy_version 1765090 (0.0007) [2023-12-27 17:53:04,122][20347] Updated weights for policy 0, policy_version 1766602 (0.0008) [2023-12-27 17:53:04,146][20381] Updated weights for policy 1, policy_version 1765100 (0.0007) [2023-12-27 17:53:04,179][20347] Updated weights for policy 0, policy_version 1766612 (0.0006) [2023-12-27 17:53:04,200][20381] Updated weights for policy 1, policy_version 1765110 (0.0005) [2023-12-27 17:53:04,236][20347] Updated weights for policy 0, policy_version 1766622 (0.0006) [2023-12-27 17:53:04,254][20381] Updated weights for policy 1, policy_version 1765120 (0.0006) [2023-12-27 17:53:04,881][20347] Updated weights for policy 0, policy_version 1766632 (0.0006) [2023-12-27 17:53:04,921][20381] Updated weights for policy 1, policy_version 1765130 (0.0005) [2023-12-27 17:53:04,940][20347] Updated weights for policy 0, policy_version 1766642 (0.0007) [2023-12-27 17:53:04,976][20381] Updated weights for policy 1, policy_version 1765140 (0.0005) [2023-12-27 17:53:04,996][20347] Updated weights for policy 0, policy_version 1766652 (0.0008) [2023-12-27 17:53:05,027][20381] Updated weights for policy 1, policy_version 1765150 (0.0010) [2023-12-27 17:53:05,582][20347] Updated weights for policy 0, policy_version 1766662 (0.0006) [2023-12-27 17:53:05,638][20347] Updated weights for policy 0, policy_version 1766672 (0.0006) [2023-12-27 17:53:05,664][20381] Updated weights for policy 1, policy_version 1765160 (0.0011) [2023-12-27 17:53:05,695][20347] Updated weights for policy 0, policy_version 1766682 (0.0005) [2023-12-27 17:53:05,719][20381] Updated weights for policy 1, policy_version 1765170 (0.0011) [2023-12-27 17:53:05,775][20381] Updated weights for policy 1, policy_version 1765180 (0.0011) [2023-12-27 17:53:06,338][20347] Updated weights for policy 0, policy_version 1766692 (0.0006) [2023-12-27 17:53:06,397][20347] Updated weights for policy 0, policy_version 1766702 (0.0007) [2023-12-27 17:53:06,459][20347] Updated weights for policy 0, policy_version 1766712 (0.0007) [2023-12-27 17:53:06,472][20381] Updated weights for policy 1, policy_version 1765190 (0.0011) [2023-12-27 17:53:06,538][20381] Updated weights for policy 1, policy_version 1765200 (0.0011) [2023-12-27 17:53:06,597][20381] Updated weights for policy 1, policy_version 1765210 (0.0011) [2023-12-27 17:53:07,151][20347] Updated weights for policy 0, policy_version 1766722 (0.0005) [2023-12-27 17:53:07,198][20381] Updated weights for policy 1, policy_version 1765220 (0.0008) [2023-12-27 17:53:07,215][20347] Updated weights for policy 0, policy_version 1766732 (0.0006) [2023-12-27 17:53:07,257][20381] Updated weights for policy 1, policy_version 1765230 (0.0007) [2023-12-27 17:53:07,270][20347] Updated weights for policy 0, policy_version 1766742 (0.0006) [2023-12-27 17:53:07,313][20381] Updated weights for policy 1, policy_version 1765240 (0.0005) [2023-12-27 17:53:07,330][20347] Updated weights for policy 0, policy_version 1766752 (0.0009) [2023-12-27 17:53:07,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 904323072. Throughput: 0: 10180.9, 1: 10349.8. Samples: 904283332. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:07,364][19170] Avg episode reward: [(0, '804.540'), (1, '1000.000')] [2023-12-27 17:53:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001765248_451952640.pth... [2023-12-27 17:53:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001766752_452370432.pth... [2023-12-27 17:53:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001764032_451641344.pth [2023-12-27 17:53:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001765568_452067328.pth [2023-12-27 17:53:07,856][20381] Updated weights for policy 1, policy_version 1765250 (0.0006) [2023-12-27 17:53:07,900][20347] Updated weights for policy 0, policy_version 1766762 (0.0009) [2023-12-27 17:53:07,909][20381] Updated weights for policy 1, policy_version 1765260 (0.0008) [2023-12-27 17:53:07,952][20347] Updated weights for policy 0, policy_version 1766772 (0.0006) [2023-12-27 17:53:07,961][20381] Updated weights for policy 1, policy_version 1765270 (0.0006) [2023-12-27 17:53:08,018][20347] Updated weights for policy 0, policy_version 1766782 (0.0008) [2023-12-27 17:53:08,030][20381] Updated weights for policy 1, policy_version 1765280 (0.0008) [2023-12-27 17:53:08,716][20381] Updated weights for policy 1, policy_version 1765290 (0.0006) [2023-12-27 17:53:08,721][20347] Updated weights for policy 0, policy_version 1766792 (0.0010) [2023-12-27 17:53:08,773][20381] Updated weights for policy 1, policy_version 1765300 (0.0006) [2023-12-27 17:53:08,786][20347] Updated weights for policy 0, policy_version 1766802 (0.0011) [2023-12-27 17:53:08,828][20381] Updated weights for policy 1, policy_version 1765310 (0.0005) [2023-12-27 17:53:08,841][20347] Updated weights for policy 0, policy_version 1766812 (0.0010) [2023-12-27 17:53:09,531][20347] Updated weights for policy 0, policy_version 1766822 (0.0008) [2023-12-27 17:53:09,543][20381] Updated weights for policy 1, policy_version 1765320 (0.0008) [2023-12-27 17:53:09,597][20347] Updated weights for policy 0, policy_version 1766832 (0.0007) [2023-12-27 17:53:09,599][20381] Updated weights for policy 1, policy_version 1765330 (0.0006) [2023-12-27 17:53:09,603][20291] KL-divergence is very high: 292.0986 [2023-12-27 17:53:09,610][20291] KL-divergence is very high: 319.3828 [2023-12-27 17:53:09,652][20291] KL-divergence is very high: 369.8688 [2023-12-27 17:53:09,657][20347] Updated weights for policy 0, policy_version 1766842 (0.0006) [2023-12-27 17:53:09,659][20291] KL-divergence is very high: 358.4008 [2023-12-27 17:53:09,665][20381] Updated weights for policy 1, policy_version 1765340 (0.0006) [2023-12-27 17:53:10,300][20381] Updated weights for policy 1, policy_version 1765350 (0.0008) [2023-12-27 17:53:10,305][20291] KL-divergence is very high: 151.8991 [2023-12-27 17:53:10,327][20347] Updated weights for policy 0, policy_version 1766852 (0.0007) [2023-12-27 17:53:10,361][20381] Updated weights for policy 1, policy_version 1765360 (0.0008) [2023-12-27 17:53:10,383][20347] Updated weights for policy 0, policy_version 1766862 (0.0007) [2023-12-27 17:53:10,423][20381] Updated weights for policy 1, policy_version 1765370 (0.0007) [2023-12-27 17:53:10,437][20347] Updated weights for policy 0, policy_version 1766872 (0.0006) [2023-12-27 17:53:11,113][20381] Updated weights for policy 1, policy_version 1765380 (0.0010) [2023-12-27 17:53:11,132][20347] Updated weights for policy 0, policy_version 1766882 (0.0007) [2023-12-27 17:53:11,180][20381] Updated weights for policy 1, policy_version 1765390 (0.0008) [2023-12-27 17:53:11,200][20347] Updated weights for policy 0, policy_version 1766892 (0.0007) [2023-12-27 17:53:11,248][20381] Updated weights for policy 1, policy_version 1765400 (0.0009) [2023-12-27 17:53:11,270][20347] Updated weights for policy 0, policy_version 1766902 (0.0007) [2023-12-27 17:53:11,327][20347] Updated weights for policy 0, policy_version 1766912 (0.0008) [2023-12-27 17:53:11,976][20381] Updated weights for policy 1, policy_version 1765410 (0.0008) [2023-12-27 17:53:12,010][20347] Updated weights for policy 0, policy_version 1766922 (0.0006) [2023-12-27 17:53:12,035][20381] Updated weights for policy 1, policy_version 1765420 (0.0011) [2023-12-27 17:53:12,068][20347] Updated weights for policy 0, policy_version 1766932 (0.0006) [2023-12-27 17:53:12,096][20381] Updated weights for policy 1, policy_version 1765430 (0.0011) [2023-12-27 17:53:12,129][20347] Updated weights for policy 0, policy_version 1766942 (0.0007) [2023-12-27 17:53:12,151][20381] Updated weights for policy 1, policy_version 1765440 (0.0011) [2023-12-27 17:53:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 904421376. Throughput: 0: 10247.8, 1: 10377.8. Samples: 904409056. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:12,363][19170] Avg episode reward: [(0, '735.450'), (1, '1000.000')] [2023-12-27 17:53:12,820][20347] Updated weights for policy 0, policy_version 1766952 (0.0006) [2023-12-27 17:53:12,875][20347] Updated weights for policy 0, policy_version 1766962 (0.0007) [2023-12-27 17:53:12,914][20381] Updated weights for policy 1, policy_version 1765450 (0.0007) [2023-12-27 17:53:12,930][20347] Updated weights for policy 0, policy_version 1766972 (0.0006) [2023-12-27 17:53:12,967][20381] Updated weights for policy 1, policy_version 1765460 (0.0007) [2023-12-27 17:53:13,020][20381] Updated weights for policy 1, policy_version 1765470 (0.0008) [2023-12-27 17:53:13,605][20347] Updated weights for policy 0, policy_version 1766982 (0.0006) [2023-12-27 17:53:13,655][20347] Updated weights for policy 0, policy_version 1766992 (0.0006) [2023-12-27 17:53:13,702][20347] Updated weights for policy 0, policy_version 1767002 (0.0008) [2023-12-27 17:53:13,730][20381] Updated weights for policy 1, policy_version 1765480 (0.0008) [2023-12-27 17:53:13,781][20381] Updated weights for policy 1, policy_version 1765490 (0.0007) [2023-12-27 17:53:13,840][20381] Updated weights for policy 1, policy_version 1765500 (0.0008) [2023-12-27 17:53:14,290][20347] Updated weights for policy 0, policy_version 1767012 (0.0008) [2023-12-27 17:53:14,353][20347] Updated weights for policy 0, policy_version 1767022 (0.0007) [2023-12-27 17:53:14,416][20347] Updated weights for policy 0, policy_version 1767032 (0.0005) [2023-12-27 17:53:14,560][20381] Updated weights for policy 1, policy_version 1765510 (0.0008) [2023-12-27 17:53:14,609][20381] Updated weights for policy 1, policy_version 1765520 (0.0008) [2023-12-27 17:53:14,660][20381] Updated weights for policy 1, policy_version 1765530 (0.0008) [2023-12-27 17:53:15,068][20347] Updated weights for policy 0, policy_version 1767042 (0.0007) [2023-12-27 17:53:15,128][20347] Updated weights for policy 0, policy_version 1767052 (0.0008) [2023-12-27 17:53:15,188][20347] Updated weights for policy 0, policy_version 1767062 (0.0008) [2023-12-27 17:53:15,237][20347] Updated weights for policy 0, policy_version 1767072 (0.0008) [2023-12-27 17:53:15,438][20381] Updated weights for policy 1, policy_version 1765540 (0.0007) [2023-12-27 17:53:15,503][20381] Updated weights for policy 1, policy_version 1765550 (0.0007) [2023-12-27 17:53:15,563][20381] Updated weights for policy 1, policy_version 1765560 (0.0008) [2023-12-27 17:53:15,904][20347] Updated weights for policy 0, policy_version 1767082 (0.0008) [2023-12-27 17:53:15,966][20347] Updated weights for policy 0, policy_version 1767092 (0.0008) [2023-12-27 17:53:16,024][20347] Updated weights for policy 0, policy_version 1767102 (0.0007) [2023-12-27 17:53:16,251][20381] Updated weights for policy 1, policy_version 1765570 (0.0008) [2023-12-27 17:53:16,312][20381] Updated weights for policy 1, policy_version 1765580 (0.0007) [2023-12-27 17:53:16,367][20381] Updated weights for policy 1, policy_version 1765590 (0.0008) [2023-12-27 17:53:16,427][20381] Updated weights for policy 1, policy_version 1765600 (0.0007) [2023-12-27 17:53:16,682][20347] Updated weights for policy 0, policy_version 1767112 (0.0008) [2023-12-27 17:53:16,738][20347] Updated weights for policy 0, policy_version 1767122 (0.0008) [2023-12-27 17:53:16,787][20347] Updated weights for policy 0, policy_version 1767132 (0.0009) [2023-12-27 17:53:17,057][20381] Updated weights for policy 1, policy_version 1765610 (0.0008) [2023-12-27 17:53:17,113][20381] Updated weights for policy 1, policy_version 1765620 (0.0008) [2023-12-27 17:53:17,180][20381] Updated weights for policy 1, policy_version 1765630 (0.0008) [2023-12-27 17:53:17,363][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 904519680. Throughput: 0: 10274.0, 1: 10320.6. Samples: 904529620. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:17,363][19170] Avg episode reward: [(0, '794.460'), (1, '990.290')] [2023-12-27 17:53:17,472][20347] Updated weights for policy 0, policy_version 1767142 (0.0007) [2023-12-27 17:53:17,522][20347] Updated weights for policy 0, policy_version 1767152 (0.0008) [2023-12-27 17:53:17,571][20347] Updated weights for policy 0, policy_version 1767162 (0.0008) [2023-12-27 17:53:17,891][20381] Updated weights for policy 1, policy_version 1765640 (0.0008) [2023-12-27 17:53:17,943][20381] Updated weights for policy 1, policy_version 1765650 (0.0008) [2023-12-27 17:53:17,992][20381] Updated weights for policy 1, policy_version 1765660 (0.0007) [2023-12-27 17:53:18,245][20347] Updated weights for policy 0, policy_version 1767172 (0.0008) [2023-12-27 17:53:18,291][20347] Updated weights for policy 0, policy_version 1767182 (0.0008) [2023-12-27 17:53:18,351][20347] Updated weights for policy 0, policy_version 1767192 (0.0008) [2023-12-27 17:53:18,650][20381] Updated weights for policy 1, policy_version 1765670 (0.0008) [2023-12-27 17:53:18,710][20381] Updated weights for policy 1, policy_version 1765680 (0.0008) [2023-12-27 17:53:18,770][20381] Updated weights for policy 1, policy_version 1765690 (0.0007) [2023-12-27 17:53:19,077][20347] Updated weights for policy 0, policy_version 1767202 (0.0006) [2023-12-27 17:53:19,134][20347] Updated weights for policy 0, policy_version 1767212 (0.0008) [2023-12-27 17:53:19,191][20347] Updated weights for policy 0, policy_version 1767222 (0.0008) [2023-12-27 17:53:19,254][20347] Updated weights for policy 0, policy_version 1767232 (0.0008) [2023-12-27 17:53:19,458][20381] Updated weights for policy 1, policy_version 1765700 (0.0008) [2023-12-27 17:53:19,526][20381] Updated weights for policy 1, policy_version 1765710 (0.0008) [2023-12-27 17:53:19,584][20381] Updated weights for policy 1, policy_version 1765720 (0.0008) [2023-12-27 17:53:19,946][20347] Updated weights for policy 0, policy_version 1767242 (0.0007) [2023-12-27 17:53:20,009][20347] Updated weights for policy 0, policy_version 1767252 (0.0007) [2023-12-27 17:53:20,072][20347] Updated weights for policy 0, policy_version 1767262 (0.0009) [2023-12-27 17:53:20,290][20381] Updated weights for policy 1, policy_version 1765730 (0.0007) [2023-12-27 17:53:20,350][20381] Updated weights for policy 1, policy_version 1765740 (0.0007) [2023-12-27 17:53:20,412][20381] Updated weights for policy 1, policy_version 1765750 (0.0011) [2023-12-27 17:53:20,472][20381] Updated weights for policy 1, policy_version 1765760 (0.0011) [2023-12-27 17:53:20,781][20347] Updated weights for policy 0, policy_version 1767272 (0.0007) [2023-12-27 17:53:20,841][20347] Updated weights for policy 0, policy_version 1767282 (0.0008) [2023-12-27 17:53:20,898][20347] Updated weights for policy 0, policy_version 1767292 (0.0007) [2023-12-27 17:53:21,219][20381] Updated weights for policy 1, policy_version 1765770 (0.0009) [2023-12-27 17:53:21,275][20381] Updated weights for policy 1, policy_version 1765780 (0.0009) [2023-12-27 17:53:21,337][20381] Updated weights for policy 1, policy_version 1765790 (0.0009) [2023-12-27 17:53:21,605][20347] Updated weights for policy 0, policy_version 1767302 (0.0007) [2023-12-27 17:53:21,674][20347] Updated weights for policy 0, policy_version 1767312 (0.0008) [2023-12-27 17:53:21,743][20347] Updated weights for policy 0, policy_version 1767322 (0.0007) [2023-12-27 17:53:22,047][20381] Updated weights for policy 1, policy_version 1765800 (0.0007) [2023-12-27 17:53:22,095][20381] Updated weights for policy 1, policy_version 1765810 (0.0006) [2023-12-27 17:53:22,143][20381] Updated weights for policy 1, policy_version 1765820 (0.0008) [2023-12-27 17:53:22,363][19170] Fps is (10 sec: 19659.0, 60 sec: 20616.2, 300 sec: 20521.6). Total num frames: 904617984. Throughput: 0: 10223.7, 1: 10238.5. Samples: 904590332. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:22,364][19170] Avg episode reward: [(0, '774.710'), (1, '990.290')] [2023-12-27 17:53:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001765824_452100096.pth... [2023-12-27 17:53:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001767328_452517888.pth... [2023-12-27 17:53:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001764640_451796992.pth [2023-12-27 17:53:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001766144_452214784.pth [2023-12-27 17:53:22,542][20347] Updated weights for policy 0, policy_version 1767332 (0.0008) [2023-12-27 17:53:22,602][20347] Updated weights for policy 0, policy_version 1767342 (0.0008) [2023-12-27 17:53:22,666][20347] Updated weights for policy 0, policy_version 1767352 (0.0007) [2023-12-27 17:53:22,779][20381] Updated weights for policy 1, policy_version 1765830 (0.0007) [2023-12-27 17:53:22,840][20381] Updated weights for policy 1, policy_version 1765840 (0.0007) [2023-12-27 17:53:22,894][20381] Updated weights for policy 1, policy_version 1765850 (0.0008) [2023-12-27 17:53:23,360][20347] Updated weights for policy 0, policy_version 1767362 (0.0007) [2023-12-27 17:53:23,426][20347] Updated weights for policy 0, policy_version 1767372 (0.0007) [2023-12-27 17:53:23,487][20347] Updated weights for policy 0, policy_version 1767382 (0.0008) [2023-12-27 17:53:23,547][20347] Updated weights for policy 0, policy_version 1767392 (0.0008) [2023-12-27 17:53:23,646][20381] Updated weights for policy 1, policy_version 1765860 (0.0007) [2023-12-27 17:53:23,706][20381] Updated weights for policy 1, policy_version 1765870 (0.0007) [2023-12-27 17:53:23,766][20381] Updated weights for policy 1, policy_version 1765880 (0.0006) [2023-12-27 17:53:24,178][20347] Updated weights for policy 0, policy_version 1767402 (0.0009) [2023-12-27 17:53:24,234][20347] Updated weights for policy 0, policy_version 1767412 (0.0008) [2023-12-27 17:53:24,296][20347] Updated weights for policy 0, policy_version 1767422 (0.0007) [2023-12-27 17:53:24,421][20381] Updated weights for policy 1, policy_version 1765890 (0.0009) [2023-12-27 17:53:24,474][20381] Updated weights for policy 1, policy_version 1765900 (0.0008) [2023-12-27 17:53:24,534][20381] Updated weights for policy 1, policy_version 1765910 (0.0008) [2023-12-27 17:53:24,595][20381] Updated weights for policy 1, policy_version 1765920 (0.0007) [2023-12-27 17:53:24,940][20347] Updated weights for policy 0, policy_version 1767432 (0.0006) [2023-12-27 17:53:25,003][20347] Updated weights for policy 0, policy_version 1767442 (0.0005) [2023-12-27 17:53:25,058][20347] Updated weights for policy 0, policy_version 1767452 (0.0006) [2023-12-27 17:53:25,302][20381] Updated weights for policy 1, policy_version 1765930 (0.0009) [2023-12-27 17:53:25,352][20381] Updated weights for policy 1, policy_version 1765940 (0.0008) [2023-12-27 17:53:25,406][20381] Updated weights for policy 1, policy_version 1765950 (0.0008) [2023-12-27 17:53:25,703][20347] Updated weights for policy 0, policy_version 1767462 (0.0008) [2023-12-27 17:53:25,743][20291] KL-divergence is very high: 171.5800 [2023-12-27 17:53:25,748][20347] Updated weights for policy 0, policy_version 1767472 (0.0007) [2023-12-27 17:53:25,757][20291] KL-divergence is very high: 226.6439 [2023-12-27 17:53:25,781][20291] KL-divergence is very high: 261.4168 [2023-12-27 17:53:25,798][20291] KL-divergence is very high: 250.8086 [2023-12-27 17:53:25,799][20347] Updated weights for policy 0, policy_version 1767482 (0.0007) [2023-12-27 17:53:25,828][20291] KL-divergence is very high: 206.5884 [2023-12-27 17:53:26,075][20381] Updated weights for policy 1, policy_version 1765960 (0.0008) [2023-12-27 17:53:26,122][20381] Updated weights for policy 1, policy_version 1765970 (0.0007) [2023-12-27 17:53:26,168][20381] Updated weights for policy 1, policy_version 1765980 (0.0007) [2023-12-27 17:53:26,466][20291] KL-divergence is very high: 167.2035 [2023-12-27 17:53:26,467][20347] Updated weights for policy 0, policy_version 1767492 (0.0010) [2023-12-27 17:53:26,478][20291] KL-divergence is very high: 110.7751 [2023-12-27 17:53:26,510][20291] KL-divergence is very high: 137.2375 [2023-12-27 17:53:26,521][20347] Updated weights for policy 0, policy_version 1767502 (0.0009) [2023-12-27 17:53:26,556][20291] KL-divergence is very high: 106.4859 [2023-12-27 17:53:26,575][20347] Updated weights for policy 0, policy_version 1767512 (0.0005) [2023-12-27 17:53:26,931][20381] Updated weights for policy 1, policy_version 1765990 (0.0008) [2023-12-27 17:53:26,985][20381] Updated weights for policy 1, policy_version 1766000 (0.0007) [2023-12-27 17:53:27,041][20381] Updated weights for policy 1, policy_version 1766010 (0.0008) [2023-12-27 17:53:27,129][20347] Updated weights for policy 0, policy_version 1767522 (0.0006) [2023-12-27 17:53:27,183][20347] Updated weights for policy 0, policy_version 1767532 (0.0005) [2023-12-27 17:53:27,243][20347] Updated weights for policy 0, policy_version 1767542 (0.0005) [2023-12-27 17:53:27,293][20347] Updated weights for policy 0, policy_version 1767552 (0.0005) [2023-12-27 17:53:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 904724480. Throughput: 0: 10150.3, 1: 10222.6. Samples: 904709384. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:27,364][19170] Avg episode reward: [(0, '786.240'), (1, '1000.000')] [2023-12-27 17:53:27,753][20381] Updated weights for policy 1, policy_version 1766020 (0.0007) [2023-12-27 17:53:27,801][20381] Updated weights for policy 1, policy_version 1766030 (0.0008) [2023-12-27 17:53:27,843][20381] Updated weights for policy 1, policy_version 1766040 (0.0008) [2023-12-27 17:53:27,894][20347] Updated weights for policy 0, policy_version 1767562 (0.0005) [2023-12-27 17:53:27,944][20347] Updated weights for policy 0, policy_version 1767572 (0.0007) [2023-12-27 17:53:27,990][20347] Updated weights for policy 0, policy_version 1767582 (0.0008) [2023-12-27 17:53:28,589][20381] Updated weights for policy 1, policy_version 1766050 (0.0007) [2023-12-27 17:53:28,645][20381] Updated weights for policy 1, policy_version 1766060 (0.0006) [2023-12-27 17:53:28,647][20347] Updated weights for policy 0, policy_version 1767592 (0.0007) [2023-12-27 17:53:28,695][20381] Updated weights for policy 1, policy_version 1766070 (0.0006) [2023-12-27 17:53:28,704][20347] Updated weights for policy 0, policy_version 1767602 (0.0007) [2023-12-27 17:53:28,749][20381] Updated weights for policy 1, policy_version 1766080 (0.0005) [2023-12-27 17:53:28,758][20347] Updated weights for policy 0, policy_version 1767612 (0.0007) [2023-12-27 17:53:29,438][20347] Updated weights for policy 0, policy_version 1767622 (0.0007) [2023-12-27 17:53:29,470][20381] Updated weights for policy 1, policy_version 1766090 (0.0008) [2023-12-27 17:53:29,494][20347] Updated weights for policy 0, policy_version 1767632 (0.0007) [2023-12-27 17:53:29,524][20381] Updated weights for policy 1, policy_version 1766100 (0.0008) [2023-12-27 17:53:29,555][20347] Updated weights for policy 0, policy_version 1767642 (0.0007) [2023-12-27 17:53:29,579][20381] Updated weights for policy 1, policy_version 1766110 (0.0008) [2023-12-27 17:53:30,265][20347] Updated weights for policy 0, policy_version 1767652 (0.0007) [2023-12-27 17:53:30,292][20381] Updated weights for policy 1, policy_version 1766120 (0.0008) [2023-12-27 17:53:30,323][20347] Updated weights for policy 0, policy_version 1767662 (0.0006) [2023-12-27 17:53:30,352][20381] Updated weights for policy 1, policy_version 1766130 (0.0006) [2023-12-27 17:53:30,385][20347] Updated weights for policy 0, policy_version 1767672 (0.0007) [2023-12-27 17:53:30,391][20291] KL-divergence is very high: 140.3584 [2023-12-27 17:53:30,403][20291] KL-divergence is very high: 142.0297 [2023-12-27 17:53:30,406][20381] Updated weights for policy 1, policy_version 1766140 (0.0007) [2023-12-27 17:53:30,414][20291] KL-divergence is very high: 163.7276 [2023-12-27 17:53:31,042][20347] Updated weights for policy 0, policy_version 1767682 (0.0007) [2023-12-27 17:53:31,082][20381] Updated weights for policy 1, policy_version 1766150 (0.0007) [2023-12-27 17:53:31,109][20347] Updated weights for policy 0, policy_version 1767692 (0.0007) [2023-12-27 17:53:31,145][20381] Updated weights for policy 1, policy_version 1766160 (0.0008) [2023-12-27 17:53:31,179][20347] Updated weights for policy 0, policy_version 1767702 (0.0008) [2023-12-27 17:53:31,222][20381] Updated weights for policy 1, policy_version 1766170 (0.0008) [2023-12-27 17:53:31,235][20347] Updated weights for policy 0, policy_version 1767712 (0.0009) [2023-12-27 17:53:31,932][20347] Updated weights for policy 0, policy_version 1767722 (0.0005) [2023-12-27 17:53:31,965][20381] Updated weights for policy 1, policy_version 1766180 (0.0008) [2023-12-27 17:53:31,992][20347] Updated weights for policy 0, policy_version 1767732 (0.0007) [2023-12-27 17:53:32,017][20381] Updated weights for policy 1, policy_version 1766190 (0.0006) [2023-12-27 17:53:32,050][20347] Updated weights for policy 0, policy_version 1767742 (0.0007) [2023-12-27 17:53:32,067][20381] Updated weights for policy 1, policy_version 1766200 (0.0006) [2023-12-27 17:53:32,362][19170] Fps is (10 sec: 20481.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 904822784. Throughput: 0: 10255.4, 1: 10107.1. Samples: 904830740. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:32,363][19170] Avg episode reward: [(0, '816.360'), (1, '990.330')] [2023-12-27 17:53:32,702][20347] Updated weights for policy 0, policy_version 1767752 (0.0007) [2023-12-27 17:53:32,758][20347] Updated weights for policy 0, policy_version 1767762 (0.0007) [2023-12-27 17:53:32,783][20381] Updated weights for policy 1, policy_version 1766210 (0.0007) [2023-12-27 17:53:32,818][20347] Updated weights for policy 0, policy_version 1767772 (0.0007) [2023-12-27 17:53:32,842][20381] Updated weights for policy 1, policy_version 1766220 (0.0006) [2023-12-27 17:53:32,904][20381] Updated weights for policy 1, policy_version 1766230 (0.0008) [2023-12-27 17:53:32,965][20381] Updated weights for policy 1, policy_version 1766240 (0.0007) [2023-12-27 17:53:33,516][20347] Updated weights for policy 0, policy_version 1767782 (0.0007) [2023-12-27 17:53:33,567][20347] Updated weights for policy 0, policy_version 1767792 (0.0007) [2023-12-27 17:53:33,617][20347] Updated weights for policy 0, policy_version 1767802 (0.0007) [2023-12-27 17:53:33,649][20381] Updated weights for policy 1, policy_version 1766250 (0.0005) [2023-12-27 17:53:33,700][20381] Updated weights for policy 1, policy_version 1766260 (0.0007) [2023-12-27 17:53:33,754][20381] Updated weights for policy 1, policy_version 1766270 (0.0007) [2023-12-27 17:53:34,306][20347] Updated weights for policy 0, policy_version 1767812 (0.0007) [2023-12-27 17:53:34,370][20347] Updated weights for policy 0, policy_version 1767822 (0.0008) [2023-12-27 17:53:34,426][20347] Updated weights for policy 0, policy_version 1767832 (0.0008) [2023-12-27 17:53:34,428][20381] Updated weights for policy 1, policy_version 1766280 (0.0006) [2023-12-27 17:53:34,488][20381] Updated weights for policy 1, policy_version 1766290 (0.0007) [2023-12-27 17:53:34,552][20381] Updated weights for policy 1, policy_version 1766300 (0.0008) [2023-12-27 17:53:35,142][20347] Updated weights for policy 0, policy_version 1767842 (0.0007) [2023-12-27 17:53:35,198][20347] Updated weights for policy 0, policy_version 1767852 (0.0007) [2023-12-27 17:53:35,218][20381] Updated weights for policy 1, policy_version 1766310 (0.0009) [2023-12-27 17:53:35,256][20347] Updated weights for policy 0, policy_version 1767862 (0.0007) [2023-12-27 17:53:35,267][20381] Updated weights for policy 1, policy_version 1766320 (0.0010) [2023-12-27 17:53:35,312][20381] Updated weights for policy 1, policy_version 1766330 (0.0010) [2023-12-27 17:53:35,312][20347] Updated weights for policy 0, policy_version 1767872 (0.0007) [2023-12-27 17:53:35,947][20381] Updated weights for policy 1, policy_version 1766340 (0.0010) [2023-12-27 17:53:35,992][20347] Updated weights for policy 0, policy_version 1767882 (0.0007) [2023-12-27 17:53:35,998][20381] Updated weights for policy 1, policy_version 1766350 (0.0010) [2023-12-27 17:53:36,043][20347] Updated weights for policy 0, policy_version 1767892 (0.0006) [2023-12-27 17:53:36,055][20381] Updated weights for policy 1, policy_version 1766360 (0.0007) [2023-12-27 17:53:36,100][20347] Updated weights for policy 0, policy_version 1767902 (0.0005) [2023-12-27 17:53:36,762][20381] Updated weights for policy 1, policy_version 1766370 (0.0007) [2023-12-27 17:53:36,779][20347] Updated weights for policy 0, policy_version 1767912 (0.0007) [2023-12-27 17:53:36,810][20381] Updated weights for policy 1, policy_version 1766380 (0.0008) [2023-12-27 17:53:36,834][20347] Updated weights for policy 0, policy_version 1767922 (0.0005) [2023-12-27 17:53:36,860][20381] Updated weights for policy 1, policy_version 1766390 (0.0008) [2023-12-27 17:53:36,892][20347] Updated weights for policy 0, policy_version 1767932 (0.0005) [2023-12-27 17:53:36,911][20381] Updated weights for policy 1, policy_version 1766400 (0.0008) [2023-12-27 17:53:37,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 904921088. Throughput: 0: 10258.1, 1: 10104.2. Samples: 904891392. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:37,363][19170] Avg episode reward: [(0, '835.020'), (1, '990.330')] [2023-12-27 17:53:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001766400_452247552.pth... [2023-12-27 17:53:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001767936_452673536.pth... [2023-12-27 17:53:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001766752_452370432.pth [2023-12-27 17:53:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001765248_451952640.pth [2023-12-27 17:53:37,517][20347] Updated weights for policy 0, policy_version 1767942 (0.0007) [2023-12-27 17:53:37,573][20347] Updated weights for policy 0, policy_version 1767952 (0.0008) [2023-12-27 17:53:37,633][20347] Updated weights for policy 0, policy_version 1767962 (0.0007) [2023-12-27 17:53:37,654][20381] Updated weights for policy 1, policy_version 1766410 (0.0006) [2023-12-27 17:53:37,703][20381] Updated weights for policy 1, policy_version 1766420 (0.0007) [2023-12-27 17:53:37,756][20381] Updated weights for policy 1, policy_version 1766430 (0.0008) [2023-12-27 17:53:38,274][20347] Updated weights for policy 0, policy_version 1767972 (0.0007) [2023-12-27 17:53:38,338][20347] Updated weights for policy 0, policy_version 1767982 (0.0007) [2023-12-27 17:53:38,404][20347] Updated weights for policy 0, policy_version 1767992 (0.0007) [2023-12-27 17:53:38,525][20381] Updated weights for policy 1, policy_version 1766440 (0.0007) [2023-12-27 17:53:38,590][20381] Updated weights for policy 1, policy_version 1766450 (0.0008) [2023-12-27 17:53:38,647][20381] Updated weights for policy 1, policy_version 1766460 (0.0008) [2023-12-27 17:53:39,126][20347] Updated weights for policy 0, policy_version 1768002 (0.0007) [2023-12-27 17:53:39,178][20347] Updated weights for policy 0, policy_version 1768012 (0.0007) [2023-12-27 17:53:39,221][20381] Updated weights for policy 1, policy_version 1766470 (0.0007) [2023-12-27 17:53:39,234][20347] Updated weights for policy 0, policy_version 1768022 (0.0008) [2023-12-27 17:53:39,283][20381] Updated weights for policy 1, policy_version 1766480 (0.0007) [2023-12-27 17:53:39,294][20347] Updated weights for policy 0, policy_version 1768032 (0.0008) [2023-12-27 17:53:39,349][20381] Updated weights for policy 1, policy_version 1766490 (0.0008) [2023-12-27 17:53:39,969][20347] Updated weights for policy 0, policy_version 1768042 (0.0007) [2023-12-27 17:53:40,029][20347] Updated weights for policy 0, policy_version 1768052 (0.0008) [2023-12-27 17:53:40,087][20347] Updated weights for policy 0, policy_version 1768062 (0.0007) [2023-12-27 17:53:40,103][20381] Updated weights for policy 1, policy_version 1766500 (0.0007) [2023-12-27 17:53:40,165][20381] Updated weights for policy 1, policy_version 1766510 (0.0008) [2023-12-27 17:53:40,224][20381] Updated weights for policy 1, policy_version 1766520 (0.0008) [2023-12-27 17:53:40,767][20347] Updated weights for policy 0, policy_version 1768072 (0.0009) [2023-12-27 17:53:40,815][20347] Updated weights for policy 0, policy_version 1768082 (0.0010) [2023-12-27 17:53:40,871][20347] Updated weights for policy 0, policy_version 1768092 (0.0010) [2023-12-27 17:53:40,948][20381] Updated weights for policy 1, policy_version 1766530 (0.0008) [2023-12-27 17:53:41,000][20381] Updated weights for policy 1, policy_version 1766540 (0.0007) [2023-12-27 17:53:41,064][20381] Updated weights for policy 1, policy_version 1766550 (0.0008) [2023-12-27 17:53:41,127][20381] Updated weights for policy 1, policy_version 1766560 (0.0008) [2023-12-27 17:53:41,661][20347] Updated weights for policy 0, policy_version 1768102 (0.0010) [2023-12-27 17:53:41,717][20347] Updated weights for policy 0, policy_version 1768112 (0.0010) [2023-12-27 17:53:41,780][20347] Updated weights for policy 0, policy_version 1768122 (0.0009) [2023-12-27 17:53:41,887][20381] Updated weights for policy 1, policy_version 1766570 (0.0008) [2023-12-27 17:53:41,947][20381] Updated weights for policy 1, policy_version 1766580 (0.0008) [2023-12-27 17:53:42,009][20381] Updated weights for policy 1, policy_version 1766590 (0.0007) [2023-12-27 17:53:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 905019392. Throughput: 0: 10266.6, 1: 9986.6. Samples: 905011940. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:42,363][19170] Avg episode reward: [(0, '863.500'), (1, '1000.000')] [2023-12-27 17:53:42,495][20347] Updated weights for policy 0, policy_version 1768132 (0.0009) [2023-12-27 17:53:42,554][20347] Updated weights for policy 0, policy_version 1768142 (0.0010) [2023-12-27 17:53:42,599][20347] Updated weights for policy 0, policy_version 1768152 (0.0010) [2023-12-27 17:53:42,746][20381] Updated weights for policy 1, policy_version 1766600 (0.0008) [2023-12-27 17:53:42,804][20381] Updated weights for policy 1, policy_version 1766610 (0.0007) [2023-12-27 17:53:42,860][20381] Updated weights for policy 1, policy_version 1766620 (0.0008) [2023-12-27 17:53:43,260][20347] Updated weights for policy 0, policy_version 1768162 (0.0009) [2023-12-27 17:53:43,317][20347] Updated weights for policy 0, policy_version 1768172 (0.0006) [2023-12-27 17:53:43,375][20347] Updated weights for policy 0, policy_version 1768182 (0.0008) [2023-12-27 17:53:43,429][20347] Updated weights for policy 0, policy_version 1768192 (0.0007) [2023-12-27 17:53:43,530][20381] Updated weights for policy 1, policy_version 1766630 (0.0007) [2023-12-27 17:53:43,578][20381] Updated weights for policy 1, policy_version 1766640 (0.0009) [2023-12-27 17:53:43,634][20381] Updated weights for policy 1, policy_version 1766650 (0.0007) [2023-12-27 17:53:44,085][20347] Updated weights for policy 0, policy_version 1768202 (0.0007) [2023-12-27 17:53:44,135][20347] Updated weights for policy 0, policy_version 1768212 (0.0008) [2023-12-27 17:53:44,183][20347] Updated weights for policy 0, policy_version 1768222 (0.0008) [2023-12-27 17:53:44,297][20381] Updated weights for policy 1, policy_version 1766660 (0.0008) [2023-12-27 17:53:44,348][20381] Updated weights for policy 1, policy_version 1766670 (0.0007) [2023-12-27 17:53:44,401][20381] Updated weights for policy 1, policy_version 1766680 (0.0008) [2023-12-27 17:53:44,905][20347] Updated weights for policy 0, policy_version 1768232 (0.0007) [2023-12-27 17:53:44,958][20347] Updated weights for policy 0, policy_version 1768242 (0.0008) [2023-12-27 17:53:45,018][20347] Updated weights for policy 0, policy_version 1768252 (0.0008) [2023-12-27 17:53:45,126][20381] Updated weights for policy 1, policy_version 1766690 (0.0008) [2023-12-27 17:53:45,189][20381] Updated weights for policy 1, policy_version 1766700 (0.0009) [2023-12-27 17:53:45,244][20381] Updated weights for policy 1, policy_version 1766710 (0.0008) [2023-12-27 17:53:45,301][20381] Updated weights for policy 1, policy_version 1766720 (0.0008) [2023-12-27 17:53:45,707][20347] Updated weights for policy 0, policy_version 1768262 (0.0007) [2023-12-27 17:53:45,762][20347] Updated weights for policy 0, policy_version 1768272 (0.0007) [2023-12-27 17:53:45,815][20347] Updated weights for policy 0, policy_version 1768282 (0.0005) [2023-12-27 17:53:46,001][20381] Updated weights for policy 1, policy_version 1766730 (0.0007) [2023-12-27 17:53:46,053][20381] Updated weights for policy 1, policy_version 1766740 (0.0008) [2023-12-27 17:53:46,108][20381] Updated weights for policy 1, policy_version 1766750 (0.0007) [2023-12-27 17:53:46,416][20347] Updated weights for policy 0, policy_version 1768292 (0.0006) [2023-12-27 17:53:46,462][20347] Updated weights for policy 0, policy_version 1768302 (0.0008) [2023-12-27 17:53:46,512][20347] Updated weights for policy 0, policy_version 1768312 (0.0007) [2023-12-27 17:53:46,761][20381] Updated weights for policy 1, policy_version 1766760 (0.0007) [2023-12-27 17:53:46,820][20381] Updated weights for policy 1, policy_version 1766770 (0.0008) [2023-12-27 17:53:46,876][20381] Updated weights for policy 1, policy_version 1766780 (0.0008) [2023-12-27 17:53:47,220][20347] Updated weights for policy 0, policy_version 1768322 (0.0008) [2023-12-27 17:53:47,279][20347] Updated weights for policy 0, policy_version 1768332 (0.0010) [2023-12-27 17:53:47,340][20347] Updated weights for policy 0, policy_version 1768342 (0.0010) [2023-12-27 17:53:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 905117696. Throughput: 0: 10277.4, 1: 10008.9. Samples: 905133016. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:47,363][19170] Avg episode reward: [(0, '804.520'), (1, '1000.000')] [2023-12-27 17:53:47,395][20347] Updated weights for policy 0, policy_version 1768352 (0.0010) [2023-12-27 17:53:47,553][20381] Updated weights for policy 1, policy_version 1766790 (0.0007) [2023-12-27 17:53:47,615][20381] Updated weights for policy 1, policy_version 1766800 (0.0009) [2023-12-27 17:53:47,672][20381] Updated weights for policy 1, policy_version 1766810 (0.0007) [2023-12-27 17:53:48,003][20347] Updated weights for policy 0, policy_version 1768362 (0.0010) [2023-12-27 17:53:48,055][20347] Updated weights for policy 0, policy_version 1768372 (0.0010) [2023-12-27 17:53:48,112][20347] Updated weights for policy 0, policy_version 1768382 (0.0010) [2023-12-27 17:53:48,343][20381] Updated weights for policy 1, policy_version 1766820 (0.0008) [2023-12-27 17:53:48,403][20381] Updated weights for policy 1, policy_version 1766830 (0.0008) [2023-12-27 17:53:48,454][20381] Updated weights for policy 1, policy_version 1766840 (0.0010) [2023-12-27 17:53:48,707][20347] Updated weights for policy 0, policy_version 1768392 (0.0009) [2023-12-27 17:53:48,757][20347] Updated weights for policy 0, policy_version 1768402 (0.0011) [2023-12-27 17:53:48,806][20347] Updated weights for policy 0, policy_version 1768412 (0.0011) [2023-12-27 17:53:49,198][20381] Updated weights for policy 1, policy_version 1766850 (0.0008) [2023-12-27 17:53:49,263][20381] Updated weights for policy 1, policy_version 1766860 (0.0007) [2023-12-27 17:53:49,321][20381] Updated weights for policy 1, policy_version 1766870 (0.0008) [2023-12-27 17:53:49,383][20381] Updated weights for policy 1, policy_version 1766880 (0.0008) [2023-12-27 17:53:49,529][20347] Updated weights for policy 0, policy_version 1768422 (0.0009) [2023-12-27 17:53:49,586][20347] Updated weights for policy 0, policy_version 1768432 (0.0009) [2023-12-27 17:53:49,639][20347] Updated weights for policy 0, policy_version 1768442 (0.0009) [2023-12-27 17:53:50,166][20381] Updated weights for policy 1, policy_version 1766890 (0.0005) [2023-12-27 17:53:50,219][20381] Updated weights for policy 1, policy_version 1766900 (0.0005) [2023-12-27 17:53:50,274][20381] Updated weights for policy 1, policy_version 1766910 (0.0007) [2023-12-27 17:53:50,440][20347] Updated weights for policy 0, policy_version 1768452 (0.0008) [2023-12-27 17:53:50,507][20347] Updated weights for policy 0, policy_version 1768462 (0.0006) [2023-12-27 17:53:50,565][20347] Updated weights for policy 0, policy_version 1768472 (0.0006) [2023-12-27 17:53:50,936][20381] Updated weights for policy 1, policy_version 1766920 (0.0008) [2023-12-27 17:53:51,000][20381] Updated weights for policy 1, policy_version 1766930 (0.0008) [2023-12-27 17:53:51,062][20381] Updated weights for policy 1, policy_version 1766940 (0.0007) [2023-12-27 17:53:51,267][20347] Updated weights for policy 0, policy_version 1768482 (0.0007) [2023-12-27 17:53:51,327][20347] Updated weights for policy 0, policy_version 1768492 (0.0006) [2023-12-27 17:53:51,398][20347] Updated weights for policy 0, policy_version 1768502 (0.0008) [2023-12-27 17:53:51,454][20347] Updated weights for policy 0, policy_version 1768512 (0.0007) [2023-12-27 17:53:51,817][20381] Updated weights for policy 1, policy_version 1766950 (0.0007) [2023-12-27 17:53:51,873][20381] Updated weights for policy 1, policy_version 1766960 (0.0008) [2023-12-27 17:53:51,918][20381] Updated weights for policy 1, policy_version 1766970 (0.0007) [2023-12-27 17:53:52,156][20347] Updated weights for policy 0, policy_version 1768522 (0.0008) [2023-12-27 17:53:52,202][20347] Updated weights for policy 0, policy_version 1768532 (0.0008) [2023-12-27 17:53:52,251][20347] Updated weights for policy 0, policy_version 1768542 (0.0008) [2023-12-27 17:53:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 905224192. Throughput: 0: 10252.0, 1: 9957.6. Samples: 905192760. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:52,363][19170] Avg episode reward: [(0, '822.980'), (1, '990.280')] [2023-12-27 17:53:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001766976_452395008.pth... [2023-12-27 17:53:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001768544_452829184.pth... [2023-12-27 17:53:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001765824_452100096.pth [2023-12-27 17:53:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001767328_452517888.pth [2023-12-27 17:53:52,650][20381] Updated weights for policy 1, policy_version 1766980 (0.0008) [2023-12-27 17:53:52,718][20381] Updated weights for policy 1, policy_version 1766990 (0.0007) [2023-12-27 17:53:52,785][20381] Updated weights for policy 1, policy_version 1767000 (0.0006) [2023-12-27 17:53:52,928][20347] Updated weights for policy 0, policy_version 1768552 (0.0005) [2023-12-27 17:53:52,985][20347] Updated weights for policy 0, policy_version 1768562 (0.0005) [2023-12-27 17:53:53,038][20347] Updated weights for policy 0, policy_version 1768572 (0.0005) [2023-12-27 17:53:53,432][20381] Updated weights for policy 1, policy_version 1767010 (0.0007) [2023-12-27 17:53:53,490][20381] Updated weights for policy 1, policy_version 1767020 (0.0007) [2023-12-27 17:53:53,543][20381] Updated weights for policy 1, policy_version 1767030 (0.0008) [2023-12-27 17:53:53,596][20381] Updated weights for policy 1, policy_version 1767040 (0.0007) [2023-12-27 17:53:53,696][20347] Updated weights for policy 0, policy_version 1768582 (0.0007) [2023-12-27 17:53:53,755][20347] Updated weights for policy 0, policy_version 1768592 (0.0008) [2023-12-27 17:53:53,811][20347] Updated weights for policy 0, policy_version 1768602 (0.0008) [2023-12-27 17:53:54,319][20381] Updated weights for policy 1, policy_version 1767050 (0.0008) [2023-12-27 17:53:54,372][20381] Updated weights for policy 1, policy_version 1767060 (0.0008) [2023-12-27 17:53:54,426][20381] Updated weights for policy 1, policy_version 1767070 (0.0007) [2023-12-27 17:53:54,461][20347] Updated weights for policy 0, policy_version 1768612 (0.0007) [2023-12-27 17:53:54,513][20347] Updated weights for policy 0, policy_version 1768622 (0.0009) [2023-12-27 17:53:54,565][20347] Updated weights for policy 0, policy_version 1768632 (0.0008) [2023-12-27 17:53:55,129][20381] Updated weights for policy 1, policy_version 1767080 (0.0007) [2023-12-27 17:53:55,183][20381] Updated weights for policy 1, policy_version 1767090 (0.0007) [2023-12-27 17:53:55,235][20381] Updated weights for policy 1, policy_version 1767100 (0.0008) [2023-12-27 17:53:55,264][20347] Updated weights for policy 0, policy_version 1768642 (0.0007) [2023-12-27 17:53:55,313][20347] Updated weights for policy 0, policy_version 1768652 (0.0007) [2023-12-27 17:53:55,364][20347] Updated weights for policy 0, policy_version 1768662 (0.0008) [2023-12-27 17:53:55,415][20347] Updated weights for policy 0, policy_version 1768672 (0.0008) [2023-12-27 17:53:55,956][20381] Updated weights for policy 1, policy_version 1767110 (0.0008) [2023-12-27 17:53:56,012][20381] Updated weights for policy 1, policy_version 1767120 (0.0008) [2023-12-27 17:53:56,062][20381] Updated weights for policy 1, policy_version 1767130 (0.0008) [2023-12-27 17:53:56,075][20347] Updated weights for policy 0, policy_version 1768682 (0.0005) [2023-12-27 17:53:56,131][20347] Updated weights for policy 0, policy_version 1768692 (0.0006) [2023-12-27 17:53:56,181][20347] Updated weights for policy 0, policy_version 1768702 (0.0007) [2023-12-27 17:53:56,744][20381] Updated weights for policy 1, policy_version 1767140 (0.0008) [2023-12-27 17:53:56,803][20381] Updated weights for policy 1, policy_version 1767150 (0.0006) [2023-12-27 17:53:56,835][20347] Updated weights for policy 0, policy_version 1768712 (0.0007) [2023-12-27 17:53:56,855][20381] Updated weights for policy 1, policy_version 1767160 (0.0005) [2023-12-27 17:53:56,880][20347] Updated weights for policy 0, policy_version 1768722 (0.0007) [2023-12-27 17:53:56,941][20347] Updated weights for policy 0, policy_version 1768732 (0.0010) [2023-12-27 17:53:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 905322496. Throughput: 0: 10219.1, 1: 9873.8. Samples: 905313236. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:53:57,363][19170] Avg episode reward: [(0, '811.330'), (1, '990.280')] [2023-12-27 17:53:57,441][20381] Updated weights for policy 1, policy_version 1767170 (0.0006) [2023-12-27 17:53:57,497][20381] Updated weights for policy 1, policy_version 1767180 (0.0007) [2023-12-27 17:53:57,558][20381] Updated weights for policy 1, policy_version 1767190 (0.0008) [2023-12-27 17:53:57,613][20381] Updated weights for policy 1, policy_version 1767200 (0.0006) [2023-12-27 17:53:57,618][20347] Updated weights for policy 0, policy_version 1768742 (0.0009) [2023-12-27 17:53:57,673][20347] Updated weights for policy 0, policy_version 1768752 (0.0007) [2023-12-27 17:53:57,727][20347] Updated weights for policy 0, policy_version 1768762 (0.0007) [2023-12-27 17:53:58,161][20381] Updated weights for policy 1, policy_version 1767210 (0.0005) [2023-12-27 17:53:58,225][20381] Updated weights for policy 1, policy_version 1767220 (0.0008) [2023-12-27 17:53:58,288][20381] Updated weights for policy 1, policy_version 1767230 (0.0008) [2023-12-27 17:53:58,412][20347] Updated weights for policy 0, policy_version 1768772 (0.0007) [2023-12-27 17:53:58,471][20347] Updated weights for policy 0, policy_version 1768782 (0.0011) [2023-12-27 17:53:58,534][20347] Updated weights for policy 0, policy_version 1768792 (0.0011) [2023-12-27 17:53:59,042][20381] Updated weights for policy 1, policy_version 1767240 (0.0009) [2023-12-27 17:53:59,095][20381] Updated weights for policy 1, policy_version 1767250 (0.0009) [2023-12-27 17:53:59,156][20381] Updated weights for policy 1, policy_version 1767260 (0.0009) [2023-12-27 17:53:59,323][20347] Updated weights for policy 0, policy_version 1768802 (0.0010) [2023-12-27 17:53:59,395][20347] Updated weights for policy 0, policy_version 1768812 (0.0008) [2023-12-27 17:53:59,460][20347] Updated weights for policy 0, policy_version 1768822 (0.0008) [2023-12-27 17:53:59,521][20347] Updated weights for policy 0, policy_version 1768832 (0.0008) [2023-12-27 17:53:59,835][20381] Updated weights for policy 1, policy_version 1767270 (0.0008) [2023-12-27 17:53:59,890][20381] Updated weights for policy 1, policy_version 1767280 (0.0008) [2023-12-27 17:53:59,944][20381] Updated weights for policy 1, policy_version 1767290 (0.0008) [2023-12-27 17:54:00,190][20347] Updated weights for policy 0, policy_version 1768842 (0.0007) [2023-12-27 17:54:00,242][20347] Updated weights for policy 0, policy_version 1768852 (0.0005) [2023-12-27 17:54:00,307][20347] Updated weights for policy 0, policy_version 1768862 (0.0005) [2023-12-27 17:54:00,608][20381] Updated weights for policy 1, policy_version 1767300 (0.0007) [2023-12-27 17:54:00,661][20381] Updated weights for policy 1, policy_version 1767310 (0.0005) [2023-12-27 17:54:00,719][20381] Updated weights for policy 1, policy_version 1767320 (0.0008) [2023-12-27 17:54:00,921][20347] Updated weights for policy 0, policy_version 1768872 (0.0007) [2023-12-27 17:54:00,966][20347] Updated weights for policy 0, policy_version 1768882 (0.0007) [2023-12-27 17:54:01,033][20347] Updated weights for policy 0, policy_version 1768892 (0.0008) [2023-12-27 17:54:01,330][20381] Updated weights for policy 1, policy_version 1767330 (0.0007) [2023-12-27 17:54:01,394][20381] Updated weights for policy 1, policy_version 1767340 (0.0008) [2023-12-27 17:54:01,450][20381] Updated weights for policy 1, policy_version 1767350 (0.0007) [2023-12-27 17:54:01,505][20381] Updated weights for policy 1, policy_version 1767360 (0.0008) [2023-12-27 17:54:01,747][20347] Updated weights for policy 0, policy_version 1768902 (0.0007) [2023-12-27 17:54:01,808][20347] Updated weights for policy 0, policy_version 1768912 (0.0008) [2023-12-27 17:54:01,864][20347] Updated weights for policy 0, policy_version 1768922 (0.0006) [2023-12-27 17:54:02,175][20381] Updated weights for policy 1, policy_version 1767370 (0.0006) [2023-12-27 17:54:02,230][20381] Updated weights for policy 1, policy_version 1767380 (0.0006) [2023-12-27 17:54:02,287][20381] Updated weights for policy 1, policy_version 1767390 (0.0006) [2023-12-27 17:54:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 905428992. Throughput: 0: 10170.4, 1: 9981.3. Samples: 905436444. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:02,363][19170] Avg episode reward: [(0, '852.280'), (1, '1000.000')] [2023-12-27 17:54:02,501][20291] KL-divergence is very high: 138.9671 [2023-12-27 17:54:02,517][20347] Updated weights for policy 0, policy_version 1768932 (0.0006) [2023-12-27 17:54:02,546][20291] KL-divergence is very high: 284.7157 [2023-12-27 17:54:02,575][20347] Updated weights for policy 0, policy_version 1768942 (0.0008) [2023-12-27 17:54:02,593][20291] KL-divergence is very high: 333.6866 [2023-12-27 17:54:02,633][20347] Updated weights for policy 0, policy_version 1768952 (0.0007) [2023-12-27 17:54:02,639][20291] KL-divergence is very high: 326.1761 [2023-12-27 17:54:02,868][20381] Updated weights for policy 1, policy_version 1767400 (0.0007) [2023-12-27 17:54:02,924][20381] Updated weights for policy 1, policy_version 1767410 (0.0007) [2023-12-27 17:54:02,983][20381] Updated weights for policy 1, policy_version 1767420 (0.0007) [2023-12-27 17:54:03,294][20347] Updated weights for policy 0, policy_version 1768962 (0.0007) [2023-12-27 17:54:03,349][20347] Updated weights for policy 0, policy_version 1768972 (0.0005) [2023-12-27 17:54:03,398][20347] Updated weights for policy 0, policy_version 1768982 (0.0005) [2023-12-27 17:54:03,448][20347] Updated weights for policy 0, policy_version 1768992 (0.0005) [2023-12-27 17:54:03,655][20381] Updated weights for policy 1, policy_version 1767430 (0.0005) [2023-12-27 17:54:03,712][20381] Updated weights for policy 1, policy_version 1767440 (0.0005) [2023-12-27 17:54:03,762][20381] Updated weights for policy 1, policy_version 1767450 (0.0005) [2023-12-27 17:54:03,998][20347] Updated weights for policy 0, policy_version 1769002 (0.0007) [2023-12-27 17:54:04,060][20347] Updated weights for policy 0, policy_version 1769012 (0.0007) [2023-12-27 17:54:04,125][20347] Updated weights for policy 0, policy_version 1769022 (0.0008) [2023-12-27 17:54:04,381][20381] Updated weights for policy 1, policy_version 1767460 (0.0006) [2023-12-27 17:54:04,430][20381] Updated weights for policy 1, policy_version 1767470 (0.0007) [2023-12-27 17:54:04,485][20381] Updated weights for policy 1, policy_version 1767480 (0.0008) [2023-12-27 17:54:04,733][20347] Updated weights for policy 0, policy_version 1769032 (0.0006) [2023-12-27 17:54:04,800][20347] Updated weights for policy 0, policy_version 1769042 (0.0005) [2023-12-27 17:54:04,867][20347] Updated weights for policy 0, policy_version 1769052 (0.0005) [2023-12-27 17:54:05,152][20381] Updated weights for policy 1, policy_version 1767490 (0.0007) [2023-12-27 17:54:05,210][20381] Updated weights for policy 1, policy_version 1767500 (0.0007) [2023-12-27 17:54:05,266][20381] Updated weights for policy 1, policy_version 1767510 (0.0008) [2023-12-27 17:54:05,322][20381] Updated weights for policy 1, policy_version 1767520 (0.0008) [2023-12-27 17:54:05,419][20347] Updated weights for policy 0, policy_version 1769062 (0.0007) [2023-12-27 17:54:05,468][20347] Updated weights for policy 0, policy_version 1769072 (0.0008) [2023-12-27 17:54:05,520][20347] Updated weights for policy 0, policy_version 1769082 (0.0007) [2023-12-27 17:54:06,024][20381] Updated weights for policy 1, policy_version 1767530 (0.0006) [2023-12-27 17:54:06,081][20381] Updated weights for policy 1, policy_version 1767540 (0.0007) [2023-12-27 17:54:06,146][20381] Updated weights for policy 1, policy_version 1767550 (0.0008) [2023-12-27 17:54:06,166][20347] Updated weights for policy 0, policy_version 1769092 (0.0008) [2023-12-27 17:54:06,227][20347] Updated weights for policy 0, policy_version 1769102 (0.0006) [2023-12-27 17:54:06,289][20347] Updated weights for policy 0, policy_version 1769112 (0.0006) [2023-12-27 17:54:06,905][20381] Updated weights for policy 1, policy_version 1767560 (0.0007) [2023-12-27 17:54:06,957][20347] Updated weights for policy 0, policy_version 1769122 (0.0006) [2023-12-27 17:54:06,959][20381] Updated weights for policy 1, policy_version 1767570 (0.0006) [2023-12-27 17:54:07,009][20347] Updated weights for policy 0, policy_version 1769132 (0.0008) [2023-12-27 17:54:07,011][20381] Updated weights for policy 1, policy_version 1767580 (0.0005) [2023-12-27 17:54:07,064][20347] Updated weights for policy 0, policy_version 1769142 (0.0007) [2023-12-27 17:54:07,126][20347] Updated weights for policy 0, policy_version 1769152 (0.0008) [2023-12-27 17:54:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 905535488. Throughput: 0: 10245.5, 1: 10022.1. Samples: 905502352. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:07,363][19170] Avg episode reward: [(0, '872.730'), (1, '1000.000')] [2023-12-27 17:54:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001767584_452550656.pth... [2023-12-27 17:54:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001769152_452984832.pth... [2023-12-27 17:54:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001766400_452247552.pth [2023-12-27 17:54:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001767936_452673536.pth [2023-12-27 17:54:07,690][20381] Updated weights for policy 1, policy_version 1767590 (0.0007) [2023-12-27 17:54:07,723][20347] Updated weights for policy 0, policy_version 1769162 (0.0005) [2023-12-27 17:54:07,748][20381] Updated weights for policy 1, policy_version 1767600 (0.0008) [2023-12-27 17:54:07,784][20347] Updated weights for policy 0, policy_version 1769172 (0.0005) [2023-12-27 17:54:07,801][20381] Updated weights for policy 1, policy_version 1767610 (0.0008) [2023-12-27 17:54:07,840][20347] Updated weights for policy 0, policy_version 1769182 (0.0005) [2023-12-27 17:54:08,415][20347] Updated weights for policy 0, policy_version 1769192 (0.0009) [2023-12-27 17:54:08,464][20347] Updated weights for policy 0, policy_version 1769202 (0.0010) [2023-12-27 17:54:08,530][20347] Updated weights for policy 0, policy_version 1769212 (0.0010) [2023-12-27 17:54:08,566][20381] Updated weights for policy 1, policy_version 1767620 (0.0007) [2023-12-27 17:54:08,629][20381] Updated weights for policy 1, policy_version 1767630 (0.0007) [2023-12-27 17:54:08,687][20381] Updated weights for policy 1, policy_version 1767640 (0.0007) [2023-12-27 17:54:09,139][20347] Updated weights for policy 0, policy_version 1769222 (0.0007) [2023-12-27 17:54:09,200][20347] Updated weights for policy 0, policy_version 1769232 (0.0009) [2023-12-27 17:54:09,267][20347] Updated weights for policy 0, policy_version 1769242 (0.0008) [2023-12-27 17:54:09,397][20381] Updated weights for policy 1, policy_version 1767650 (0.0007) [2023-12-27 17:54:09,458][20381] Updated weights for policy 1, policy_version 1767660 (0.0008) [2023-12-27 17:54:09,516][20381] Updated weights for policy 1, policy_version 1767670 (0.0007) [2023-12-27 17:54:09,575][20381] Updated weights for policy 1, policy_version 1767680 (0.0008) [2023-12-27 17:54:09,976][20347] Updated weights for policy 0, policy_version 1769252 (0.0007) [2023-12-27 17:54:10,032][20347] Updated weights for policy 0, policy_version 1769262 (0.0006) [2023-12-27 17:54:10,092][20347] Updated weights for policy 0, policy_version 1769272 (0.0006) [2023-12-27 17:54:10,321][20381] Updated weights for policy 1, policy_version 1767690 (0.0008) [2023-12-27 17:54:10,387][20381] Updated weights for policy 1, policy_version 1767700 (0.0008) [2023-12-27 17:54:10,444][20381] Updated weights for policy 1, policy_version 1767710 (0.0008) [2023-12-27 17:54:10,752][20347] Updated weights for policy 0, policy_version 1769282 (0.0007) [2023-12-27 17:54:10,800][20347] Updated weights for policy 0, policy_version 1769292 (0.0010) [2023-12-27 17:54:10,857][20347] Updated weights for policy 0, policy_version 1769302 (0.0010) [2023-12-27 17:54:10,915][20347] Updated weights for policy 0, policy_version 1769312 (0.0010) [2023-12-27 17:54:11,114][20381] Updated weights for policy 1, policy_version 1767720 (0.0008) [2023-12-27 17:54:11,179][20381] Updated weights for policy 1, policy_version 1767730 (0.0008) [2023-12-27 17:54:11,234][20381] Updated weights for policy 1, policy_version 1767740 (0.0007) [2023-12-27 17:54:11,640][20347] Updated weights for policy 0, policy_version 1769322 (0.0010) [2023-12-27 17:54:11,701][20347] Updated weights for policy 0, policy_version 1769332 (0.0008) [2023-12-27 17:54:11,763][20347] Updated weights for policy 0, policy_version 1769342 (0.0010) [2023-12-27 17:54:12,042][20381] Updated weights for policy 1, policy_version 1767750 (0.0008) [2023-12-27 17:54:12,094][20381] Updated weights for policy 1, policy_version 1767760 (0.0007) [2023-12-27 17:54:12,145][20381] Updated weights for policy 1, policy_version 1767770 (0.0007) [2023-12-27 17:54:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 905633792. Throughput: 0: 10348.6, 1: 9992.0. Samples: 905624704. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:12,363][19170] Avg episode reward: [(0, '872.110'), (1, '1000.000')] [2023-12-27 17:54:12,501][20347] Updated weights for policy 0, policy_version 1769352 (0.0007) [2023-12-27 17:54:12,563][20347] Updated weights for policy 0, policy_version 1769362 (0.0010) [2023-12-27 17:54:12,614][20347] Updated weights for policy 0, policy_version 1769372 (0.0008) [2023-12-27 17:54:12,868][20381] Updated weights for policy 1, policy_version 1767780 (0.0008) [2023-12-27 17:54:12,916][20381] Updated weights for policy 1, policy_version 1767790 (0.0007) [2023-12-27 17:54:12,967][20381] Updated weights for policy 1, policy_version 1767800 (0.0007) [2023-12-27 17:54:13,285][20347] Updated weights for policy 0, policy_version 1769382 (0.0009) [2023-12-27 17:54:13,343][20347] Updated weights for policy 0, policy_version 1769392 (0.0010) [2023-12-27 17:54:13,400][20347] Updated weights for policy 0, policy_version 1769402 (0.0010) [2023-12-27 17:54:13,686][20381] Updated weights for policy 1, policy_version 1767810 (0.0008) [2023-12-27 17:54:13,737][20381] Updated weights for policy 1, policy_version 1767820 (0.0007) [2023-12-27 17:54:13,788][20381] Updated weights for policy 1, policy_version 1767830 (0.0010) [2023-12-27 17:54:13,839][20381] Updated weights for policy 1, policy_version 1767840 (0.0010) [2023-12-27 17:54:14,040][20347] Updated weights for policy 0, policy_version 1769412 (0.0010) [2023-12-27 17:54:14,093][20347] Updated weights for policy 0, policy_version 1769422 (0.0010) [2023-12-27 17:54:14,148][20347] Updated weights for policy 0, policy_version 1769432 (0.0010) [2023-12-27 17:54:14,500][20381] Updated weights for policy 1, policy_version 1767850 (0.0011) [2023-12-27 17:54:14,555][20381] Updated weights for policy 1, policy_version 1767860 (0.0007) [2023-12-27 17:54:14,611][20381] Updated weights for policy 1, policy_version 1767870 (0.0005) [2023-12-27 17:54:14,839][20347] Updated weights for policy 0, policy_version 1769442 (0.0009) [2023-12-27 17:54:14,902][20347] Updated weights for policy 0, policy_version 1769452 (0.0007) [2023-12-27 17:54:14,963][20347] Updated weights for policy 0, policy_version 1769462 (0.0008) [2023-12-27 17:54:15,030][20347] Updated weights for policy 0, policy_version 1769472 (0.0007) [2023-12-27 17:54:15,245][20381] Updated weights for policy 1, policy_version 1767880 (0.0010) [2023-12-27 17:54:15,303][20381] Updated weights for policy 1, policy_version 1767890 (0.0011) [2023-12-27 17:54:15,363][20381] Updated weights for policy 1, policy_version 1767900 (0.0011) [2023-12-27 17:54:15,707][20347] Updated weights for policy 0, policy_version 1769482 (0.0007) [2023-12-27 17:54:15,754][20347] Updated weights for policy 0, policy_version 1769492 (0.0007) [2023-12-27 17:54:15,813][20347] Updated weights for policy 0, policy_version 1769502 (0.0007) [2023-12-27 17:54:16,049][20381] Updated weights for policy 1, policy_version 1767910 (0.0011) [2023-12-27 17:54:16,105][20381] Updated weights for policy 1, policy_version 1767920 (0.0011) [2023-12-27 17:54:16,159][20381] Updated weights for policy 1, policy_version 1767930 (0.0010) [2023-12-27 17:54:16,456][20347] Updated weights for policy 0, policy_version 1769512 (0.0005) [2023-12-27 17:54:16,512][20347] Updated weights for policy 0, policy_version 1769522 (0.0009) [2023-12-27 17:54:16,566][20347] Updated weights for policy 0, policy_version 1769532 (0.0010) [2023-12-27 17:54:16,795][20381] Updated weights for policy 1, policy_version 1767940 (0.0010) [2023-12-27 17:54:16,843][20381] Updated weights for policy 1, policy_version 1767950 (0.0010) [2023-12-27 17:54:16,888][20381] Updated weights for policy 1, policy_version 1767960 (0.0010) [2023-12-27 17:54:17,255][20347] Updated weights for policy 0, policy_version 1769542 (0.0009) [2023-12-27 17:54:17,308][20347] Updated weights for policy 0, policy_version 1769552 (0.0007) [2023-12-27 17:54:17,362][20347] Updated weights for policy 0, policy_version 1769562 (0.0007) [2023-12-27 17:54:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 905732096. Throughput: 0: 10307.6, 1: 10059.6. Samples: 905747268. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:17,363][19170] Avg episode reward: [(0, '859.670'), (1, '1000.000')] [2023-12-27 17:54:17,550][20381] Updated weights for policy 1, policy_version 1767970 (0.0010) [2023-12-27 17:54:17,600][20381] Updated weights for policy 1, policy_version 1767980 (0.0011) [2023-12-27 17:54:17,649][20381] Updated weights for policy 1, policy_version 1767990 (0.0010) [2023-12-27 17:54:17,708][20381] Updated weights for policy 1, policy_version 1768000 (0.0011) [2023-12-27 17:54:18,071][20347] Updated weights for policy 0, policy_version 1769572 (0.0008) [2023-12-27 17:54:18,134][20347] Updated weights for policy 0, policy_version 1769582 (0.0009) [2023-12-27 17:54:18,200][20347] Updated weights for policy 0, policy_version 1769592 (0.0005) [2023-12-27 17:54:18,391][20381] Updated weights for policy 1, policy_version 1768010 (0.0008) [2023-12-27 17:54:18,453][20381] Updated weights for policy 1, policy_version 1768020 (0.0007) [2023-12-27 17:54:18,506][20381] Updated weights for policy 1, policy_version 1768030 (0.0008) [2023-12-27 17:54:18,814][20347] Updated weights for policy 0, policy_version 1769602 (0.0006) [2023-12-27 17:54:18,882][20347] Updated weights for policy 0, policy_version 1769612 (0.0010) [2023-12-27 17:54:18,930][20347] Updated weights for policy 0, policy_version 1769622 (0.0010) [2023-12-27 17:54:18,982][20347] Updated weights for policy 0, policy_version 1769632 (0.0010) [2023-12-27 17:54:19,154][20381] Updated weights for policy 1, policy_version 1768040 (0.0008) [2023-12-27 17:54:19,214][20381] Updated weights for policy 1, policy_version 1768050 (0.0008) [2023-12-27 17:54:19,279][20381] Updated weights for policy 1, policy_version 1768060 (0.0008) [2023-12-27 17:54:19,681][20347] Updated weights for policy 0, policy_version 1769642 (0.0008) [2023-12-27 17:54:19,737][20347] Updated weights for policy 0, policy_version 1769652 (0.0008) [2023-12-27 17:54:19,801][20347] Updated weights for policy 0, policy_version 1769662 (0.0008) [2023-12-27 17:54:20,022][20381] Updated weights for policy 1, policy_version 1768070 (0.0007) [2023-12-27 17:54:20,081][20381] Updated weights for policy 1, policy_version 1768080 (0.0007) [2023-12-27 17:54:20,141][20381] Updated weights for policy 1, policy_version 1768090 (0.0007) [2023-12-27 17:54:20,581][20347] Updated weights for policy 0, policy_version 1769672 (0.0007) [2023-12-27 17:54:20,645][20347] Updated weights for policy 0, policy_version 1769682 (0.0007) [2023-12-27 17:54:20,707][20347] Updated weights for policy 0, policy_version 1769692 (0.0008) [2023-12-27 17:54:20,843][20381] Updated weights for policy 1, policy_version 1768100 (0.0007) [2023-12-27 17:54:20,914][20381] Updated weights for policy 1, policy_version 1768110 (0.0008) [2023-12-27 17:54:20,972][20381] Updated weights for policy 1, policy_version 1768120 (0.0008) [2023-12-27 17:54:21,391][20347] Updated weights for policy 0, policy_version 1769702 (0.0009) [2023-12-27 17:54:21,449][20347] Updated weights for policy 0, policy_version 1769712 (0.0009) [2023-12-27 17:54:21,499][20347] Updated weights for policy 0, policy_version 1769722 (0.0009) [2023-12-27 17:54:21,793][20381] Updated weights for policy 1, policy_version 1768130 (0.0008) [2023-12-27 17:54:21,853][20381] Updated weights for policy 1, policy_version 1768140 (0.0010) [2023-12-27 17:54:21,906][20381] Updated weights for policy 1, policy_version 1768150 (0.0009) [2023-12-27 17:54:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20207.2, 300 sec: 20466.1). Total num frames: 905830400. Throughput: 0: 10291.5, 1: 10048.2. Samples: 905806684. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:22,363][19170] Avg episode reward: [(0, '784.730'), (1, '1000.000')] [2023-12-27 17:54:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001768160_452698112.pth... [2023-12-27 17:54:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001769728_453132288.pth... [2023-12-27 17:54:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001766976_452395008.pth [2023-12-27 17:54:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001768544_452829184.pth [2023-12-27 17:54:22,438][20347] Updated weights for policy 0, policy_version 1769732 (0.0009) [2023-12-27 17:54:22,492][20347] Updated weights for policy 0, policy_version 1769742 (0.0009) [2023-12-27 17:54:22,541][20347] Updated weights for policy 0, policy_version 1769752 (0.0009) [2023-12-27 17:54:22,784][20381] Updated weights for policy 1, policy_version 1768161 (0.0009) [2023-12-27 17:54:22,839][20381] Updated weights for policy 1, policy_version 1768171 (0.0006) [2023-12-27 17:54:22,899][20381] Updated weights for policy 1, policy_version 1768181 (0.0005) [2023-12-27 17:54:22,965][20381] Updated weights for policy 1, policy_version 1768191 (0.0006) [2023-12-27 17:54:23,328][20347] Updated weights for policy 0, policy_version 1769762 (0.0009) [2023-12-27 17:54:23,380][20347] Updated weights for policy 0, policy_version 1769772 (0.0010) [2023-12-27 17:54:23,428][20347] Updated weights for policy 0, policy_version 1769782 (0.0010) [2023-12-27 17:54:23,481][20347] Updated weights for policy 0, policy_version 1769792 (0.0010) [2023-12-27 17:54:23,605][20381] Updated weights for policy 1, policy_version 1768201 (0.0007) [2023-12-27 17:54:23,666][20381] Updated weights for policy 1, policy_version 1768211 (0.0008) [2023-12-27 17:54:23,726][20381] Updated weights for policy 1, policy_version 1768221 (0.0007) [2023-12-27 17:54:24,222][20347] Updated weights for policy 0, policy_version 1769802 (0.0008) [2023-12-27 17:54:24,279][20347] Updated weights for policy 0, policy_version 1769812 (0.0007) [2023-12-27 17:54:24,327][20347] Updated weights for policy 0, policy_version 1769822 (0.0008) [2023-12-27 17:54:24,406][20381] Updated weights for policy 1, policy_version 1768231 (0.0007) [2023-12-27 17:54:24,459][20381] Updated weights for policy 1, policy_version 1768241 (0.0008) [2023-12-27 17:54:24,507][20381] Updated weights for policy 1, policy_version 1768251 (0.0005) [2023-12-27 17:54:24,982][20347] Updated weights for policy 0, policy_version 1769832 (0.0009) [2023-12-27 17:54:25,030][20347] Updated weights for policy 0, policy_version 1769842 (0.0010) [2023-12-27 17:54:25,081][20347] Updated weights for policy 0, policy_version 1769852 (0.0010) [2023-12-27 17:54:25,120][20381] Updated weights for policy 1, policy_version 1768261 (0.0007) [2023-12-27 17:54:25,175][20381] Updated weights for policy 1, policy_version 1768271 (0.0007) [2023-12-27 17:54:25,223][20381] Updated weights for policy 1, policy_version 1768281 (0.0007) [2023-12-27 17:54:25,737][20347] Updated weights for policy 0, policy_version 1769862 (0.0009) [2023-12-27 17:54:25,791][20347] Updated weights for policy 0, policy_version 1769872 (0.0007) [2023-12-27 17:54:25,847][20347] Updated weights for policy 0, policy_version 1769882 (0.0007) [2023-12-27 17:54:25,958][20381] Updated weights for policy 1, policy_version 1768291 (0.0008) [2023-12-27 17:54:26,009][20381] Updated weights for policy 1, policy_version 1768301 (0.0009) [2023-12-27 17:54:26,070][20381] Updated weights for policy 1, policy_version 1768311 (0.0008) [2023-12-27 17:54:26,456][20347] Updated weights for policy 0, policy_version 1769892 (0.0005) [2023-12-27 17:54:26,513][20347] Updated weights for policy 0, policy_version 1769902 (0.0005) [2023-12-27 17:54:26,573][20347] Updated weights for policy 0, policy_version 1769912 (0.0005) [2023-12-27 17:54:26,830][20381] Updated weights for policy 1, policy_version 1768321 (0.0008) [2023-12-27 17:54:26,882][20381] Updated weights for policy 1, policy_version 1768331 (0.0008) [2023-12-27 17:54:26,935][20381] Updated weights for policy 1, policy_version 1768341 (0.0008) [2023-12-27 17:54:26,991][20381] Updated weights for policy 1, policy_version 1768351 (0.0007) [2023-12-27 17:54:27,125][20347] Updated weights for policy 0, policy_version 1769922 (0.0006) [2023-12-27 17:54:27,184][20347] Updated weights for policy 0, policy_version 1769932 (0.0008) [2023-12-27 17:54:27,240][20347] Updated weights for policy 0, policy_version 1769942 (0.0008) [2023-12-27 17:54:27,299][20347] Updated weights for policy 0, policy_version 1769952 (0.0007) [2023-12-27 17:54:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20207.1, 300 sec: 20466.1). Total num frames: 905936896. Throughput: 0: 10212.2, 1: 10007.9. Samples: 905921844. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:27,363][19170] Avg episode reward: [(0, '826.760'), (1, '990.270')] [2023-12-27 17:54:27,726][20381] Updated weights for policy 1, policy_version 1768361 (0.0008) [2023-12-27 17:54:27,782][20381] Updated weights for policy 1, policy_version 1768371 (0.0009) [2023-12-27 17:54:27,839][20381] Updated weights for policy 1, policy_version 1768381 (0.0010) [2023-12-27 17:54:27,940][20347] Updated weights for policy 0, policy_version 1769962 (0.0008) [2023-12-27 17:54:27,997][20347] Updated weights for policy 0, policy_version 1769972 (0.0007) [2023-12-27 17:54:28,057][20347] Updated weights for policy 0, policy_version 1769982 (0.0008) [2023-12-27 17:54:28,520][20381] Updated weights for policy 1, policy_version 1768391 (0.0011) [2023-12-27 17:54:28,577][20381] Updated weights for policy 1, policy_version 1768401 (0.0011) [2023-12-27 17:54:28,633][20381] Updated weights for policy 1, policy_version 1768411 (0.0011) [2023-12-27 17:54:28,722][20347] Updated weights for policy 0, policy_version 1769992 (0.0008) [2023-12-27 17:54:28,778][20347] Updated weights for policy 0, policy_version 1770002 (0.0007) [2023-12-27 17:54:28,839][20347] Updated weights for policy 0, policy_version 1770012 (0.0007) [2023-12-27 17:54:29,319][20381] Updated weights for policy 1, policy_version 1768421 (0.0010) [2023-12-27 17:54:29,382][20381] Updated weights for policy 1, policy_version 1768431 (0.0009) [2023-12-27 17:54:29,447][20381] Updated weights for policy 1, policy_version 1768441 (0.0011) [2023-12-27 17:54:29,508][20347] Updated weights for policy 0, policy_version 1770022 (0.0007) [2023-12-27 17:54:29,563][20347] Updated weights for policy 0, policy_version 1770032 (0.0007) [2023-12-27 17:54:29,618][20347] Updated weights for policy 0, policy_version 1770042 (0.0008) [2023-12-27 17:54:30,154][20381] Updated weights for policy 1, policy_version 1768451 (0.0010) [2023-12-27 17:54:30,203][20381] Updated weights for policy 1, policy_version 1768461 (0.0007) [2023-12-27 17:54:30,262][20381] Updated weights for policy 1, policy_version 1768471 (0.0008) [2023-12-27 17:54:30,293][20347] Updated weights for policy 0, policy_version 1770052 (0.0006) [2023-12-27 17:54:30,360][20347] Updated weights for policy 0, policy_version 1770062 (0.0006) [2023-12-27 17:54:30,414][20347] Updated weights for policy 0, policy_version 1770072 (0.0008) [2023-12-27 17:54:30,923][20381] Updated weights for policy 1, policy_version 1768481 (0.0008) [2023-12-27 17:54:30,983][20381] Updated weights for policy 1, policy_version 1768491 (0.0006) [2023-12-27 17:54:31,039][20381] Updated weights for policy 1, policy_version 1768501 (0.0008) [2023-12-27 17:54:31,060][20347] Updated weights for policy 0, policy_version 1770082 (0.0006) [2023-12-27 17:54:31,099][20381] Updated weights for policy 1, policy_version 1768511 (0.0008) [2023-12-27 17:54:31,112][20347] Updated weights for policy 0, policy_version 1770092 (0.0006) [2023-12-27 17:54:31,174][20347] Updated weights for policy 0, policy_version 1770102 (0.0007) [2023-12-27 17:54:31,223][20347] Updated weights for policy 0, policy_version 1770112 (0.0005) [2023-12-27 17:54:31,847][20381] Updated weights for policy 1, policy_version 1768521 (0.0008) [2023-12-27 17:54:31,904][20381] Updated weights for policy 1, policy_version 1768531 (0.0008) [2023-12-27 17:54:31,925][20347] Updated weights for policy 0, policy_version 1770122 (0.0007) [2023-12-27 17:54:31,957][20381] Updated weights for policy 1, policy_version 1768541 (0.0006) [2023-12-27 17:54:31,977][20347] Updated weights for policy 0, policy_version 1770132 (0.0007) [2023-12-27 17:54:32,037][20347] Updated weights for policy 0, policy_version 1770142 (0.0007) [2023-12-27 17:54:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 906035200. Throughput: 0: 10266.3, 1: 9984.1. Samples: 906044284. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:32,364][19170] Avg episode reward: [(0, '860.800'), (1, '990.270')] [2023-12-27 17:54:32,702][20381] Updated weights for policy 1, policy_version 1768551 (0.0009) [2023-12-27 17:54:32,734][20347] Updated weights for policy 0, policy_version 1770152 (0.0007) [2023-12-27 17:54:32,760][20381] Updated weights for policy 1, policy_version 1768561 (0.0006) [2023-12-27 17:54:32,790][20347] Updated weights for policy 0, policy_version 1770162 (0.0006) [2023-12-27 17:54:32,817][20381] Updated weights for policy 1, policy_version 1768571 (0.0005) [2023-12-27 17:54:32,847][20347] Updated weights for policy 0, policy_version 1770172 (0.0006) [2023-12-27 17:54:33,363][20381] Updated weights for policy 1, policy_version 1768581 (0.0009) [2023-12-27 17:54:33,417][20381] Updated weights for policy 1, policy_version 1768591 (0.0010) [2023-12-27 17:54:33,471][20381] Updated weights for policy 1, policy_version 1768601 (0.0010) [2023-12-27 17:54:33,520][20347] Updated weights for policy 0, policy_version 1770182 (0.0005) [2023-12-27 17:54:33,570][20347] Updated weights for policy 0, policy_version 1770192 (0.0005) [2023-12-27 17:54:33,617][20347] Updated weights for policy 0, policy_version 1770202 (0.0005) [2023-12-27 17:54:34,045][20381] Updated weights for policy 1, policy_version 1768611 (0.0009) [2023-12-27 17:54:34,102][20381] Updated weights for policy 1, policy_version 1768621 (0.0007) [2023-12-27 17:54:34,169][20381] Updated weights for policy 1, policy_version 1768631 (0.0007) [2023-12-27 17:54:34,228][20347] Updated weights for policy 0, policy_version 1770212 (0.0005) [2023-12-27 17:54:34,281][20347] Updated weights for policy 0, policy_version 1770222 (0.0007) [2023-12-27 17:54:34,341][20347] Updated weights for policy 0, policy_version 1770232 (0.0008) [2023-12-27 17:54:34,805][20381] Updated weights for policy 1, policy_version 1768641 (0.0007) [2023-12-27 17:54:34,869][20381] Updated weights for policy 1, policy_version 1768651 (0.0006) [2023-12-27 17:54:34,932][20381] Updated weights for policy 1, policy_version 1768661 (0.0006) [2023-12-27 17:54:34,988][20381] Updated weights for policy 1, policy_version 1768671 (0.0008) [2023-12-27 17:54:35,051][20347] Updated weights for policy 0, policy_version 1770242 (0.0007) [2023-12-27 17:54:35,099][20347] Updated weights for policy 0, policy_version 1770252 (0.0007) [2023-12-27 17:54:35,147][20347] Updated weights for policy 0, policy_version 1770262 (0.0007) [2023-12-27 17:54:35,195][20347] Updated weights for policy 0, policy_version 1770272 (0.0007) [2023-12-27 17:54:35,623][20381] Updated weights for policy 1, policy_version 1768681 (0.0006) [2023-12-27 17:54:35,674][20381] Updated weights for policy 1, policy_version 1768691 (0.0007) [2023-12-27 17:54:35,730][20381] Updated weights for policy 1, policy_version 1768701 (0.0006) [2023-12-27 17:54:35,915][20347] Updated weights for policy 0, policy_version 1770282 (0.0005) [2023-12-27 17:54:35,965][20347] Updated weights for policy 0, policy_version 1770292 (0.0006) [2023-12-27 17:54:36,019][20347] Updated weights for policy 0, policy_version 1770302 (0.0005) [2023-12-27 17:54:36,315][20381] Updated weights for policy 1, policy_version 1768711 (0.0006) [2023-12-27 17:54:36,379][20381] Updated weights for policy 1, policy_version 1768721 (0.0006) [2023-12-27 17:54:36,444][20381] Updated weights for policy 1, policy_version 1768731 (0.0009) [2023-12-27 17:54:36,638][20347] Updated weights for policy 0, policy_version 1770312 (0.0008) [2023-12-27 17:54:36,694][20347] Updated weights for policy 0, policy_version 1770322 (0.0010) [2023-12-27 17:54:36,748][20347] Updated weights for policy 0, policy_version 1770332 (0.0006) [2023-12-27 17:54:37,114][20381] Updated weights for policy 1, policy_version 1768741 (0.0008) [2023-12-27 17:54:37,170][20381] Updated weights for policy 1, policy_version 1768751 (0.0005) [2023-12-27 17:54:37,222][20381] Updated weights for policy 1, policy_version 1768761 (0.0005) [2023-12-27 17:54:37,327][20347] Updated weights for policy 0, policy_version 1770342 (0.0005) [2023-12-27 17:54:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 906141696. Throughput: 0: 10275.5, 1: 10079.0. Samples: 906108712. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:37,363][19170] Avg episode reward: [(0, '863.200'), (1, '1000.000')] [2023-12-27 17:54:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001768768_452853760.pth... [2023-12-27 17:54:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001767584_452550656.pth [2023-12-27 17:54:37,385][20347] Updated weights for policy 0, policy_version 1770352 (0.0007) [2023-12-27 17:54:37,433][20347] Updated weights for policy 0, policy_version 1770362 (0.0010) [2023-12-27 17:54:37,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001770368_453296128.pth... [2023-12-27 17:54:37,463][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001769152_452984832.pth [2023-12-27 17:54:37,803][20381] Updated weights for policy 1, policy_version 1768771 (0.0005) [2023-12-27 17:54:37,864][20381] Updated weights for policy 1, policy_version 1768781 (0.0007) [2023-12-27 17:54:37,918][20381] Updated weights for policy 1, policy_version 1768791 (0.0010) [2023-12-27 17:54:38,106][20347] Updated weights for policy 0, policy_version 1770372 (0.0010) [2023-12-27 17:54:38,158][20347] Updated weights for policy 0, policy_version 1770382 (0.0010) [2023-12-27 17:54:38,206][20347] Updated weights for policy 0, policy_version 1770392 (0.0010) [2023-12-27 17:54:38,543][20381] Updated weights for policy 1, policy_version 1768801 (0.0010) [2023-12-27 17:54:38,597][20381] Updated weights for policy 1, policy_version 1768811 (0.0010) [2023-12-27 17:54:38,653][20381] Updated weights for policy 1, policy_version 1768821 (0.0010) [2023-12-27 17:54:38,707][20381] Updated weights for policy 1, policy_version 1768831 (0.0010) [2023-12-27 17:54:38,965][20347] Updated weights for policy 0, policy_version 1770402 (0.0009) [2023-12-27 17:54:39,024][20347] Updated weights for policy 0, policy_version 1770412 (0.0007) [2023-12-27 17:54:39,080][20347] Updated weights for policy 0, policy_version 1770422 (0.0009) [2023-12-27 17:54:39,129][20347] Updated weights for policy 0, policy_version 1770432 (0.0008) [2023-12-27 17:54:39,416][20381] Updated weights for policy 1, policy_version 1768841 (0.0009) [2023-12-27 17:54:39,474][20381] Updated weights for policy 1, policy_version 1768851 (0.0011) [2023-12-27 17:54:39,538][20381] Updated weights for policy 1, policy_version 1768861 (0.0011) [2023-12-27 17:54:39,794][20347] Updated weights for policy 0, policy_version 1770442 (0.0006) [2023-12-27 17:54:39,863][20347] Updated weights for policy 0, policy_version 1770452 (0.0007) [2023-12-27 17:54:39,919][20347] Updated weights for policy 0, policy_version 1770462 (0.0008) [2023-12-27 17:54:40,274][20381] Updated weights for policy 1, policy_version 1768871 (0.0009) [2023-12-27 17:54:40,323][20381] Updated weights for policy 1, policy_version 1768881 (0.0008) [2023-12-27 17:54:40,378][20381] Updated weights for policy 1, policy_version 1768891 (0.0008) [2023-12-27 17:54:40,584][20347] Updated weights for policy 0, policy_version 1770472 (0.0008) [2023-12-27 17:54:40,647][20347] Updated weights for policy 0, policy_version 1770482 (0.0008) [2023-12-27 17:54:40,696][20347] Updated weights for policy 0, policy_version 1770492 (0.0010) [2023-12-27 17:54:41,022][20381] Updated weights for policy 1, policy_version 1768901 (0.0009) [2023-12-27 17:54:41,092][20381] Updated weights for policy 1, policy_version 1768911 (0.0010) [2023-12-27 17:54:41,162][20381] Updated weights for policy 1, policy_version 1768921 (0.0011) [2023-12-27 17:54:41,432][20347] Updated weights for policy 0, policy_version 1770502 (0.0008) [2023-12-27 17:54:41,488][20347] Updated weights for policy 0, policy_version 1770512 (0.0009) [2023-12-27 17:54:41,546][20347] Updated weights for policy 0, policy_version 1770522 (0.0007) [2023-12-27 17:54:41,972][20381] Updated weights for policy 1, policy_version 1768931 (0.0010) [2023-12-27 17:54:42,033][20381] Updated weights for policy 1, policy_version 1768941 (0.0008) [2023-12-27 17:54:42,093][20381] Updated weights for policy 1, policy_version 1768951 (0.0008) [2023-12-27 17:54:42,223][20347] Updated weights for policy 0, policy_version 1770532 (0.0007) [2023-12-27 17:54:42,283][20347] Updated weights for policy 0, policy_version 1770542 (0.0007) [2023-12-27 17:54:42,345][20347] Updated weights for policy 0, policy_version 1770552 (0.0008) [2023-12-27 17:54:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 906240000. Throughput: 0: 10284.8, 1: 10161.2. Samples: 906233304. Policy #0 lag: (min: 25.0, avg: 38.0, max: 57.0) [2023-12-27 17:54:42,363][19170] Avg episode reward: [(0, '892.470'), (1, '1000.000')] [2023-12-27 17:54:42,813][20381] Updated weights for policy 1, policy_version 1768961 (0.0008) [2023-12-27 17:54:42,863][20381] Updated weights for policy 1, policy_version 1768971 (0.0006) [2023-12-27 17:54:42,915][20381] Updated weights for policy 1, policy_version 1768981 (0.0005) [2023-12-27 17:54:42,970][20381] Updated weights for policy 1, policy_version 1768991 (0.0005) [2023-12-27 17:54:42,984][20347] Updated weights for policy 0, policy_version 1770562 (0.0008) [2023-12-27 17:54:43,034][20347] Updated weights for policy 0, policy_version 1770572 (0.0005) [2023-12-27 17:54:43,086][20347] Updated weights for policy 0, policy_version 1770582 (0.0006) [2023-12-27 17:54:43,147][20347] Updated weights for policy 0, policy_version 1770592 (0.0005) [2023-12-27 17:54:43,578][20381] Updated weights for policy 1, policy_version 1769001 (0.0008) [2023-12-27 17:54:43,636][20381] Updated weights for policy 1, policy_version 1769011 (0.0006) [2023-12-27 17:54:43,701][20381] Updated weights for policy 1, policy_version 1769021 (0.0006) [2023-12-27 17:54:43,792][20347] Updated weights for policy 0, policy_version 1770602 (0.0007) [2023-12-27 17:54:43,847][20347] Updated weights for policy 0, policy_version 1770612 (0.0007) [2023-12-27 17:54:43,899][20347] Updated weights for policy 0, policy_version 1770622 (0.0008) [2023-12-27 17:54:44,341][20381] Updated weights for policy 1, policy_version 1769031 (0.0007) [2023-12-27 17:54:44,395][20381] Updated weights for policy 1, policy_version 1769041 (0.0007) [2023-12-27 17:54:44,441][20381] Updated weights for policy 1, policy_version 1769051 (0.0008) [2023-12-27 17:54:44,551][20347] Updated weights for policy 0, policy_version 1770632 (0.0007) [2023-12-27 17:54:44,603][20347] Updated weights for policy 0, policy_version 1770642 (0.0005) [2023-12-27 17:54:44,662][20347] Updated weights for policy 0, policy_version 1770652 (0.0005) [2023-12-27 17:54:45,083][20381] Updated weights for policy 1, policy_version 1769061 (0.0007) [2023-12-27 17:54:45,137][20381] Updated weights for policy 1, policy_version 1769071 (0.0008) [2023-12-27 17:54:45,186][20381] Updated weights for policy 1, policy_version 1769081 (0.0008) [2023-12-27 17:54:45,326][20347] Updated weights for policy 0, policy_version 1770662 (0.0007) [2023-12-27 17:54:45,385][20347] Updated weights for policy 0, policy_version 1770672 (0.0008) [2023-12-27 17:54:45,441][20347] Updated weights for policy 0, policy_version 1770682 (0.0008) [2023-12-27 17:54:45,822][20381] Updated weights for policy 1, policy_version 1769091 (0.0008) [2023-12-27 17:54:45,876][20381] Updated weights for policy 1, policy_version 1769101 (0.0008) [2023-12-27 17:54:45,933][20381] Updated weights for policy 1, policy_version 1769111 (0.0008) [2023-12-27 17:54:46,159][20347] Updated weights for policy 0, policy_version 1770692 (0.0008) [2023-12-27 17:54:46,208][20347] Updated weights for policy 0, policy_version 1770702 (0.0008) [2023-12-27 17:54:46,256][20347] Updated weights for policy 0, policy_version 1770712 (0.0007) [2023-12-27 17:54:46,569][20381] Updated weights for policy 1, policy_version 1769121 (0.0007) [2023-12-27 17:54:46,631][20381] Updated weights for policy 1, policy_version 1769131 (0.0008) [2023-12-27 17:54:46,689][20381] Updated weights for policy 1, policy_version 1769141 (0.0007) [2023-12-27 17:54:46,744][20381] Updated weights for policy 1, policy_version 1769151 (0.0007) [2023-12-27 17:54:46,946][20347] Updated weights for policy 0, policy_version 1770722 (0.0007) [2023-12-27 17:54:47,009][20347] Updated weights for policy 0, policy_version 1770732 (0.0009) [2023-12-27 17:54:47,067][20347] Updated weights for policy 0, policy_version 1770742 (0.0008) [2023-12-27 17:54:47,119][20347] Updated weights for policy 0, policy_version 1770752 (0.0008) [2023-12-27 17:54:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 906346496. Throughput: 0: 10324.4, 1: 10131.7. Samples: 906356968. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:54:47,363][19170] Avg episode reward: [(0, '873.110'), (1, '1000.000')] [2023-12-27 17:54:47,406][20381] Updated weights for policy 1, policy_version 1769161 (0.0005) [2023-12-27 17:54:47,456][20381] Updated weights for policy 1, policy_version 1769171 (0.0005) [2023-12-27 17:54:47,512][20381] Updated weights for policy 1, policy_version 1769181 (0.0006) [2023-12-27 17:54:47,821][20347] Updated weights for policy 0, policy_version 1770762 (0.0008) [2023-12-27 17:54:47,878][20347] Updated weights for policy 0, policy_version 1770772 (0.0008) [2023-12-27 17:54:47,934][20347] Updated weights for policy 0, policy_version 1770782 (0.0007) [2023-12-27 17:54:48,137][20381] Updated weights for policy 1, policy_version 1769191 (0.0007) [2023-12-27 17:54:48,194][20381] Updated weights for policy 1, policy_version 1769201 (0.0008) [2023-12-27 17:54:48,243][20381] Updated weights for policy 1, policy_version 1769211 (0.0007) [2023-12-27 17:54:48,603][20347] Updated weights for policy 0, policy_version 1770792 (0.0008) [2023-12-27 17:54:48,659][20347] Updated weights for policy 0, policy_version 1770802 (0.0008) [2023-12-27 17:54:48,719][20347] Updated weights for policy 0, policy_version 1770812 (0.0008) [2023-12-27 17:54:48,981][20381] Updated weights for policy 1, policy_version 1769221 (0.0008) [2023-12-27 17:54:49,042][20381] Updated weights for policy 1, policy_version 1769231 (0.0005) [2023-12-27 17:54:49,103][20381] Updated weights for policy 1, policy_version 1769241 (0.0005) [2023-12-27 17:54:49,343][20347] Updated weights for policy 0, policy_version 1770822 (0.0008) [2023-12-27 17:54:49,405][20347] Updated weights for policy 0, policy_version 1770832 (0.0008) [2023-12-27 17:54:49,461][20347] Updated weights for policy 0, policy_version 1770842 (0.0008) [2023-12-27 17:54:49,739][20381] Updated weights for policy 1, policy_version 1769251 (0.0006) [2023-12-27 17:54:49,795][20381] Updated weights for policy 1, policy_version 1769261 (0.0007) [2023-12-27 17:54:49,863][20381] Updated weights for policy 1, policy_version 1769271 (0.0008) [2023-12-27 17:54:50,179][20347] Updated weights for policy 0, policy_version 1770852 (0.0007) [2023-12-27 17:54:50,239][20347] Updated weights for policy 0, policy_version 1770862 (0.0006) [2023-12-27 17:54:50,296][20347] Updated weights for policy 0, policy_version 1770872 (0.0005) [2023-12-27 17:54:50,613][20381] Updated weights for policy 1, policy_version 1769281 (0.0007) [2023-12-27 17:54:50,669][20381] Updated weights for policy 1, policy_version 1769291 (0.0007) [2023-12-27 17:54:50,725][20381] Updated weights for policy 1, policy_version 1769301 (0.0008) [2023-12-27 17:54:50,778][20381] Updated weights for policy 1, policy_version 1769311 (0.0008) [2023-12-27 17:54:50,947][20347] Updated weights for policy 0, policy_version 1770882 (0.0007) [2023-12-27 17:54:51,006][20347] Updated weights for policy 0, policy_version 1770892 (0.0008) [2023-12-27 17:54:51,069][20347] Updated weights for policy 0, policy_version 1770902 (0.0007) [2023-12-27 17:54:51,123][20347] Updated weights for policy 0, policy_version 1770912 (0.0008) [2023-12-27 17:54:51,480][20381] Updated weights for policy 1, policy_version 1769321 (0.0008) [2023-12-27 17:54:51,538][20381] Updated weights for policy 1, policy_version 1769331 (0.0008) [2023-12-27 17:54:51,597][20381] Updated weights for policy 1, policy_version 1769341 (0.0009) [2023-12-27 17:54:51,831][20347] Updated weights for policy 0, policy_version 1770922 (0.0006) [2023-12-27 17:54:51,888][20347] Updated weights for policy 0, policy_version 1770932 (0.0008) [2023-12-27 17:54:51,948][20347] Updated weights for policy 0, policy_version 1770942 (0.0007) [2023-12-27 17:54:52,360][20381] Updated weights for policy 1, policy_version 1769351 (0.0010) [2023-12-27 17:54:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 906444800. Throughput: 0: 10252.9, 1: 10092.8. Samples: 906417908. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:54:52,363][19170] Avg episode reward: [(0, '863.130'), (1, '1000.000')] [2023-12-27 17:54:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001770944_453443584.pth... [2023-12-27 17:54:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001769728_453132288.pth [2023-12-27 17:54:52,421][20381] Updated weights for policy 1, policy_version 1769361 (0.0008) [2023-12-27 17:54:52,478][20381] Updated weights for policy 1, policy_version 1769371 (0.0008) [2023-12-27 17:54:52,504][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001769376_453009408.pth... [2023-12-27 17:54:52,507][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001768160_452698112.pth [2023-12-27 17:54:52,627][20347] Updated weights for policy 0, policy_version 1770952 (0.0007) [2023-12-27 17:54:52,682][20347] Updated weights for policy 0, policy_version 1770962 (0.0006) [2023-12-27 17:54:52,733][20347] Updated weights for policy 0, policy_version 1770972 (0.0007) [2023-12-27 17:54:53,194][20381] Updated weights for policy 1, policy_version 1769381 (0.0009) [2023-12-27 17:54:53,255][20381] Updated weights for policy 1, policy_version 1769391 (0.0007) [2023-12-27 17:54:53,318][20381] Updated weights for policy 1, policy_version 1769401 (0.0009) [2023-12-27 17:54:53,377][20347] Updated weights for policy 0, policy_version 1770982 (0.0007) [2023-12-27 17:54:53,423][20347] Updated weights for policy 0, policy_version 1770992 (0.0009) [2023-12-27 17:54:53,486][20347] Updated weights for policy 0, policy_version 1771002 (0.0008) [2023-12-27 17:54:53,835][20381] Updated weights for policy 1, policy_version 1769411 (0.0005) [2023-12-27 17:54:53,890][20381] Updated weights for policy 1, policy_version 1769421 (0.0005) [2023-12-27 17:54:53,952][20381] Updated weights for policy 1, policy_version 1769431 (0.0007) [2023-12-27 17:54:54,205][20347] Updated weights for policy 0, policy_version 1771012 (0.0008) [2023-12-27 17:54:54,270][20347] Updated weights for policy 0, policy_version 1771022 (0.0007) [2023-12-27 17:54:54,337][20347] Updated weights for policy 0, policy_version 1771032 (0.0006) [2023-12-27 17:54:54,628][20381] Updated weights for policy 1, policy_version 1769441 (0.0010) [2023-12-27 17:54:54,683][20381] Updated weights for policy 1, policy_version 1769451 (0.0010) [2023-12-27 17:54:54,738][20381] Updated weights for policy 1, policy_version 1769461 (0.0010) [2023-12-27 17:54:54,793][20381] Updated weights for policy 1, policy_version 1769471 (0.0010) [2023-12-27 17:54:54,982][20347] Updated weights for policy 0, policy_version 1771042 (0.0005) [2023-12-27 17:54:55,038][20347] Updated weights for policy 0, policy_version 1771052 (0.0008) [2023-12-27 17:54:55,086][20347] Updated weights for policy 0, policy_version 1771062 (0.0007) [2023-12-27 17:54:55,135][20347] Updated weights for policy 0, policy_version 1771072 (0.0005) [2023-12-27 17:54:55,466][20381] Updated weights for policy 1, policy_version 1769481 (0.0006) [2023-12-27 17:54:55,517][20381] Updated weights for policy 1, policy_version 1769491 (0.0005) [2023-12-27 17:54:55,566][20381] Updated weights for policy 1, policy_version 1769501 (0.0010) [2023-12-27 17:54:55,809][20347] Updated weights for policy 0, policy_version 1771082 (0.0005) [2023-12-27 17:54:55,861][20347] Updated weights for policy 0, policy_version 1771092 (0.0005) [2023-12-27 17:54:55,910][20347] Updated weights for policy 0, policy_version 1771102 (0.0006) [2023-12-27 17:54:56,164][20381] Updated weights for policy 1, policy_version 1769511 (0.0007) [2023-12-27 17:54:56,221][20381] Updated weights for policy 1, policy_version 1769521 (0.0006) [2023-12-27 17:54:56,276][20381] Updated weights for policy 1, policy_version 1769531 (0.0006) [2023-12-27 17:54:56,496][20347] Updated weights for policy 0, policy_version 1771112 (0.0009) [2023-12-27 17:54:56,544][20347] Updated weights for policy 0, policy_version 1771122 (0.0010) [2023-12-27 17:54:56,594][20347] Updated weights for policy 0, policy_version 1771132 (0.0010) [2023-12-27 17:54:56,841][20381] Updated weights for policy 1, policy_version 1769541 (0.0005) [2023-12-27 17:54:56,895][20381] Updated weights for policy 1, policy_version 1769551 (0.0005) [2023-12-27 17:54:56,957][20381] Updated weights for policy 1, policy_version 1769561 (0.0005) [2023-12-27 17:54:57,234][20347] Updated weights for policy 0, policy_version 1771142 (0.0009) [2023-12-27 17:54:57,287][20347] Updated weights for policy 0, policy_version 1771152 (0.0009) [2023-12-27 17:54:57,346][20347] Updated weights for policy 0, policy_version 1771162 (0.0010) [2023-12-27 17:54:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 906551296. Throughput: 0: 10199.4, 1: 10180.7. Samples: 906541808. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:54:57,363][19170] Avg episode reward: [(0, '882.660'), (1, '990.270')] [2023-12-27 17:54:57,514][20381] Updated weights for policy 1, policy_version 1769571 (0.0005) [2023-12-27 17:54:57,565][20381] Updated weights for policy 1, policy_version 1769581 (0.0005) [2023-12-27 17:54:57,611][20381] Updated weights for policy 1, policy_version 1769591 (0.0005) [2023-12-27 17:54:58,035][20347] Updated weights for policy 0, policy_version 1771172 (0.0010) [2023-12-27 17:54:58,092][20347] Updated weights for policy 0, policy_version 1771182 (0.0010) [2023-12-27 17:54:58,143][20347] Updated weights for policy 0, policy_version 1771192 (0.0010) [2023-12-27 17:54:58,261][20381] Updated weights for policy 1, policy_version 1769601 (0.0008) [2023-12-27 17:54:58,320][20381] Updated weights for policy 1, policy_version 1769611 (0.0010) [2023-12-27 17:54:58,392][20381] Updated weights for policy 1, policy_version 1769621 (0.0008) [2023-12-27 17:54:58,455][20381] Updated weights for policy 1, policy_version 1769631 (0.0010) [2023-12-27 17:54:59,061][20347] Updated weights for policy 0, policy_version 1771202 (0.0009) [2023-12-27 17:54:59,106][20291] KL-divergence is very high: 102.6227 [2023-12-27 17:54:59,112][20291] KL-divergence is very high: 100.7843 [2023-12-27 17:54:59,119][20347] Updated weights for policy 0, policy_version 1771212 (0.0007) [2023-12-27 17:54:59,152][20291] KL-divergence is very high: 100.8024 [2023-12-27 17:54:59,178][20347] Updated weights for policy 0, policy_version 1771222 (0.0008) [2023-12-27 17:54:59,192][20381] Updated weights for policy 1, policy_version 1769641 (0.0010) [2023-12-27 17:54:59,239][20347] Updated weights for policy 0, policy_version 1771232 (0.0008) [2023-12-27 17:54:59,256][20381] Updated weights for policy 1, policy_version 1769651 (0.0009) [2023-12-27 17:54:59,322][20381] Updated weights for policy 1, policy_version 1769661 (0.0008) [2023-12-27 17:54:59,983][20347] Updated weights for policy 0, policy_version 1771242 (0.0007) [2023-12-27 17:55:00,043][20347] Updated weights for policy 0, policy_version 1771252 (0.0007) [2023-12-27 17:55:00,102][20347] Updated weights for policy 0, policy_version 1771262 (0.0005) [2023-12-27 17:55:00,104][20381] Updated weights for policy 1, policy_version 1769671 (0.0008) [2023-12-27 17:55:00,160][20381] Updated weights for policy 1, policy_version 1769681 (0.0009) [2023-12-27 17:55:00,221][20381] Updated weights for policy 1, policy_version 1769691 (0.0008) [2023-12-27 17:55:00,725][20347] Updated weights for policy 0, policy_version 1771272 (0.0005) [2023-12-27 17:55:00,776][20347] Updated weights for policy 0, policy_version 1771282 (0.0005) [2023-12-27 17:55:00,822][20347] Updated weights for policy 0, policy_version 1771292 (0.0006) [2023-12-27 17:55:00,843][20381] Updated weights for policy 1, policy_version 1769701 (0.0006) [2023-12-27 17:55:00,906][20381] Updated weights for policy 1, policy_version 1769711 (0.0005) [2023-12-27 17:55:00,970][20381] Updated weights for policy 1, policy_version 1769721 (0.0005) [2023-12-27 17:55:01,485][20347] Updated weights for policy 0, policy_version 1771302 (0.0007) [2023-12-27 17:55:01,543][20347] Updated weights for policy 0, policy_version 1771312 (0.0008) [2023-12-27 17:55:01,604][20347] Updated weights for policy 0, policy_version 1771322 (0.0008) [2023-12-27 17:55:01,655][20381] Updated weights for policy 1, policy_version 1769731 (0.0006) [2023-12-27 17:55:01,711][20381] Updated weights for policy 1, policy_version 1769741 (0.0008) [2023-12-27 17:55:01,777][20381] Updated weights for policy 1, policy_version 1769751 (0.0008) [2023-12-27 17:55:02,318][20347] Updated weights for policy 0, policy_version 1771332 (0.0007) [2023-12-27 17:55:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 906649600. Throughput: 0: 10180.7, 1: 10207.7. Samples: 906664744. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:02,363][19170] Avg episode reward: [(0, '892.510'), (1, '990.270')] [2023-12-27 17:55:02,380][20347] Updated weights for policy 0, policy_version 1771342 (0.0008) [2023-12-27 17:55:02,440][20347] Updated weights for policy 0, policy_version 1771352 (0.0007) [2023-12-27 17:55:02,466][20381] Updated weights for policy 1, policy_version 1769761 (0.0007) [2023-12-27 17:55:02,522][20381] Updated weights for policy 1, policy_version 1769771 (0.0007) [2023-12-27 17:55:02,585][20381] Updated weights for policy 1, policy_version 1769781 (0.0010) [2023-12-27 17:55:02,643][20381] Updated weights for policy 1, policy_version 1769791 (0.0010) [2023-12-27 17:55:03,126][20347] Updated weights for policy 0, policy_version 1771362 (0.0008) [2023-12-27 17:55:03,184][20347] Updated weights for policy 0, policy_version 1771372 (0.0007) [2023-12-27 17:55:03,241][20347] Updated weights for policy 0, policy_version 1771382 (0.0007) [2023-12-27 17:55:03,298][20347] Updated weights for policy 0, policy_version 1771392 (0.0007) [2023-12-27 17:55:03,339][20381] Updated weights for policy 1, policy_version 1769801 (0.0011) [2023-12-27 17:55:03,393][20381] Updated weights for policy 1, policy_version 1769811 (0.0010) [2023-12-27 17:55:03,450][20381] Updated weights for policy 1, policy_version 1769821 (0.0010) [2023-12-27 17:55:03,865][20347] Updated weights for policy 0, policy_version 1771402 (0.0010) [2023-12-27 17:55:03,921][20347] Updated weights for policy 0, policy_version 1771412 (0.0009) [2023-12-27 17:55:03,987][20347] Updated weights for policy 0, policy_version 1771422 (0.0008) [2023-12-27 17:55:04,141][20381] Updated weights for policy 1, policy_version 1769831 (0.0011) [2023-12-27 17:55:04,195][20381] Updated weights for policy 1, policy_version 1769841 (0.0011) [2023-12-27 17:55:04,258][20381] Updated weights for policy 1, policy_version 1769851 (0.0011) [2023-12-27 17:55:04,721][20347] Updated weights for policy 0, policy_version 1771432 (0.0008) [2023-12-27 17:55:04,773][20347] Updated weights for policy 0, policy_version 1771442 (0.0007) [2023-12-27 17:55:04,821][20347] Updated weights for policy 0, policy_version 1771452 (0.0007) [2023-12-27 17:55:04,948][20381] Updated weights for policy 1, policy_version 1769861 (0.0010) [2023-12-27 17:55:05,004][20381] Updated weights for policy 1, policy_version 1769871 (0.0011) [2023-12-27 17:55:05,057][20381] Updated weights for policy 1, policy_version 1769881 (0.0011) [2023-12-27 17:55:05,499][20347] Updated weights for policy 0, policy_version 1771462 (0.0006) [2023-12-27 17:55:05,551][20347] Updated weights for policy 0, policy_version 1771472 (0.0006) [2023-12-27 17:55:05,610][20347] Updated weights for policy 0, policy_version 1771482 (0.0007) [2023-12-27 17:55:05,754][20381] Updated weights for policy 1, policy_version 1769891 (0.0011) [2023-12-27 17:55:05,814][20381] Updated weights for policy 1, policy_version 1769901 (0.0007) [2023-12-27 17:55:05,869][20381] Updated weights for policy 1, policy_version 1769911 (0.0011) [2023-12-27 17:55:06,150][20347] Updated weights for policy 0, policy_version 1771492 (0.0007) [2023-12-27 17:55:06,206][20347] Updated weights for policy 0, policy_version 1771502 (0.0007) [2023-12-27 17:55:06,265][20347] Updated weights for policy 0, policy_version 1771512 (0.0008) [2023-12-27 17:55:06,544][20381] Updated weights for policy 1, policy_version 1769921 (0.0010) [2023-12-27 17:55:06,606][20381] Updated weights for policy 1, policy_version 1769931 (0.0010) [2023-12-27 17:55:06,667][20381] Updated weights for policy 1, policy_version 1769941 (0.0010) [2023-12-27 17:55:06,723][20381] Updated weights for policy 1, policy_version 1769951 (0.0011) [2023-12-27 17:55:06,967][20347] Updated weights for policy 0, policy_version 1771522 (0.0008) [2023-12-27 17:55:07,023][20347] Updated weights for policy 0, policy_version 1771532 (0.0007) [2023-12-27 17:55:07,074][20347] Updated weights for policy 0, policy_version 1771542 (0.0007) [2023-12-27 17:55:07,126][20347] Updated weights for policy 0, policy_version 1771552 (0.0008) [2023-12-27 17:55:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 906756096. Throughput: 0: 10214.3, 1: 10205.6. Samples: 906725580. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:07,363][19170] Avg episode reward: [(0, '912.280'), (1, '1000.000')] [2023-12-27 17:55:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001771552_453599232.pth... [2023-12-27 17:55:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001770368_453296128.pth [2023-12-27 17:55:07,400][20381] Updated weights for policy 1, policy_version 1769961 (0.0011) [2023-12-27 17:55:07,455][20381] Updated weights for policy 1, policy_version 1769971 (0.0010) [2023-12-27 17:55:07,504][20381] Updated weights for policy 1, policy_version 1769981 (0.0010) [2023-12-27 17:55:07,518][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001769984_453165056.pth... [2023-12-27 17:55:07,522][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001768768_452853760.pth [2023-12-27 17:55:07,813][20347] Updated weights for policy 0, policy_version 1771562 (0.0008) [2023-12-27 17:55:07,862][20347] Updated weights for policy 0, policy_version 1771572 (0.0008) [2023-12-27 17:55:07,909][20347] Updated weights for policy 0, policy_version 1771582 (0.0007) [2023-12-27 17:55:08,157][20381] Updated weights for policy 1, policy_version 1769991 (0.0007) [2023-12-27 17:55:08,211][20381] Updated weights for policy 1, policy_version 1770001 (0.0006) [2023-12-27 17:55:08,268][20381] Updated weights for policy 1, policy_version 1770011 (0.0008) [2023-12-27 17:55:08,643][20347] Updated weights for policy 0, policy_version 1771592 (0.0008) [2023-12-27 17:55:08,706][20347] Updated weights for policy 0, policy_version 1771602 (0.0008) [2023-12-27 17:55:08,766][20347] Updated weights for policy 0, policy_version 1771612 (0.0008) [2023-12-27 17:55:08,998][20381] Updated weights for policy 1, policy_version 1770021 (0.0009) [2023-12-27 17:55:09,059][20381] Updated weights for policy 1, policy_version 1770031 (0.0010) [2023-12-27 17:55:09,123][20381] Updated weights for policy 1, policy_version 1770041 (0.0007) [2023-12-27 17:55:09,490][20347] Updated weights for policy 0, policy_version 1771622 (0.0007) [2023-12-27 17:55:09,550][20347] Updated weights for policy 0, policy_version 1771632 (0.0007) [2023-12-27 17:55:09,613][20347] Updated weights for policy 0, policy_version 1771642 (0.0008) [2023-12-27 17:55:09,881][20381] Updated weights for policy 1, policy_version 1770051 (0.0010) [2023-12-27 17:55:09,943][20381] Updated weights for policy 1, policy_version 1770061 (0.0007) [2023-12-27 17:55:10,011][20381] Updated weights for policy 1, policy_version 1770071 (0.0006) [2023-12-27 17:55:10,350][20347] Updated weights for policy 0, policy_version 1771652 (0.0008) [2023-12-27 17:55:10,401][20347] Updated weights for policy 0, policy_version 1771662 (0.0007) [2023-12-27 17:55:10,460][20347] Updated weights for policy 0, policy_version 1771672 (0.0006) [2023-12-27 17:55:10,677][20381] Updated weights for policy 1, policy_version 1770081 (0.0009) [2023-12-27 17:55:10,730][20381] Updated weights for policy 1, policy_version 1770091 (0.0006) [2023-12-27 17:55:10,799][20381] Updated weights for policy 1, policy_version 1770101 (0.0007) [2023-12-27 17:55:10,868][20381] Updated weights for policy 1, policy_version 1770111 (0.0008) [2023-12-27 17:55:11,116][20347] Updated weights for policy 0, policy_version 1771682 (0.0006) [2023-12-27 17:55:11,178][20347] Updated weights for policy 0, policy_version 1771692 (0.0008) [2023-12-27 17:55:11,240][20347] Updated weights for policy 0, policy_version 1771702 (0.0007) [2023-12-27 17:55:11,301][20347] Updated weights for policy 0, policy_version 1771712 (0.0008) [2023-12-27 17:55:11,576][20381] Updated weights for policy 1, policy_version 1770121 (0.0008) [2023-12-27 17:55:11,635][20381] Updated weights for policy 1, policy_version 1770131 (0.0007) [2023-12-27 17:55:11,698][20381] Updated weights for policy 1, policy_version 1770141 (0.0008) [2023-12-27 17:55:12,004][20347] Updated weights for policy 0, policy_version 1771722 (0.0008) [2023-12-27 17:55:12,062][20347] Updated weights for policy 0, policy_version 1771732 (0.0006) [2023-12-27 17:55:12,120][20347] Updated weights for policy 0, policy_version 1771742 (0.0006) [2023-12-27 17:55:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 906854400. Throughput: 0: 10291.0, 1: 10267.8. Samples: 906846992. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:12,363][19170] Avg episode reward: [(0, '922.140'), (1, '1000.000')] [2023-12-27 17:55:12,494][20381] Updated weights for policy 1, policy_version 1770151 (0.0008) [2023-12-27 17:55:12,553][20381] Updated weights for policy 1, policy_version 1770161 (0.0008) [2023-12-27 17:55:12,613][20381] Updated weights for policy 1, policy_version 1770171 (0.0007) [2023-12-27 17:55:12,779][20347] Updated weights for policy 0, policy_version 1771752 (0.0005) [2023-12-27 17:55:12,840][20347] Updated weights for policy 0, policy_version 1771762 (0.0005) [2023-12-27 17:55:12,906][20347] Updated weights for policy 0, policy_version 1771772 (0.0006) [2023-12-27 17:55:13,285][20381] Updated weights for policy 1, policy_version 1770181 (0.0008) [2023-12-27 17:55:13,345][20381] Updated weights for policy 1, policy_version 1770191 (0.0010) [2023-12-27 17:55:13,393][20381] Updated weights for policy 1, policy_version 1770201 (0.0010) [2023-12-27 17:55:13,555][20347] Updated weights for policy 0, policy_version 1771782 (0.0007) [2023-12-27 17:55:13,610][20347] Updated weights for policy 0, policy_version 1771792 (0.0007) [2023-12-27 17:55:13,669][20347] Updated weights for policy 0, policy_version 1771802 (0.0007) [2023-12-27 17:55:14,083][20381] Updated weights for policy 1, policy_version 1770211 (0.0011) [2023-12-27 17:55:14,140][20381] Updated weights for policy 1, policy_version 1770221 (0.0010) [2023-12-27 17:55:14,199][20381] Updated weights for policy 1, policy_version 1770231 (0.0010) [2023-12-27 17:55:14,373][20347] Updated weights for policy 0, policy_version 1771812 (0.0008) [2023-12-27 17:55:14,432][20347] Updated weights for policy 0, policy_version 1771822 (0.0008) [2023-12-27 17:55:14,481][20347] Updated weights for policy 0, policy_version 1771832 (0.0008) [2023-12-27 17:55:14,856][20381] Updated weights for policy 1, policy_version 1770241 (0.0010) [2023-12-27 17:55:14,915][20381] Updated weights for policy 1, policy_version 1770251 (0.0011) [2023-12-27 17:55:14,975][20381] Updated weights for policy 1, policy_version 1770261 (0.0011) [2023-12-27 17:55:15,030][20381] Updated weights for policy 1, policy_version 1770271 (0.0011) [2023-12-27 17:55:15,174][20347] Updated weights for policy 0, policy_version 1771842 (0.0007) [2023-12-27 17:55:15,230][20347] Updated weights for policy 0, policy_version 1771852 (0.0009) [2023-12-27 17:55:15,285][20347] Updated weights for policy 0, policy_version 1771862 (0.0007) [2023-12-27 17:55:15,344][20347] Updated weights for policy 0, policy_version 1771872 (0.0007) [2023-12-27 17:55:15,705][20381] Updated weights for policy 1, policy_version 1770281 (0.0008) [2023-12-27 17:55:15,764][20381] Updated weights for policy 1, policy_version 1770291 (0.0007) [2023-12-27 17:55:15,832][20381] Updated weights for policy 1, policy_version 1770301 (0.0008) [2023-12-27 17:55:15,987][20347] Updated weights for policy 0, policy_version 1771882 (0.0010) [2023-12-27 17:55:16,033][20347] Updated weights for policy 0, policy_version 1771892 (0.0010) [2023-12-27 17:55:16,089][20347] Updated weights for policy 0, policy_version 1771902 (0.0010) [2023-12-27 17:55:16,507][20381] Updated weights for policy 1, policy_version 1770311 (0.0010) [2023-12-27 17:55:16,553][20381] Updated weights for policy 1, policy_version 1770321 (0.0009) [2023-12-27 17:55:16,616][20381] Updated weights for policy 1, policy_version 1770331 (0.0010) [2023-12-27 17:55:16,756][20347] Updated weights for policy 0, policy_version 1771912 (0.0007) [2023-12-27 17:55:16,817][20347] Updated weights for policy 0, policy_version 1771922 (0.0005) [2023-12-27 17:55:16,866][20347] Updated weights for policy 0, policy_version 1771932 (0.0005) [2023-12-27 17:55:17,297][20381] Updated weights for policy 1, policy_version 1770341 (0.0009) [2023-12-27 17:55:17,360][20381] Updated weights for policy 1, policy_version 1770351 (0.0009) [2023-12-27 17:55:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 906952704. Throughput: 0: 10250.9, 1: 10276.5. Samples: 906968012. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:17,363][19170] Avg episode reward: [(0, '843.740'), (1, '990.300')] [2023-12-27 17:55:17,408][20381] Updated weights for policy 1, policy_version 1770361 (0.0009) [2023-12-27 17:55:17,432][20347] Updated weights for policy 0, policy_version 1771942 (0.0006) [2023-12-27 17:55:17,493][20347] Updated weights for policy 0, policy_version 1771952 (0.0008) [2023-12-27 17:55:17,546][20347] Updated weights for policy 0, policy_version 1771962 (0.0007) [2023-12-27 17:55:17,991][20381] Updated weights for policy 1, policy_version 1770371 (0.0009) [2023-12-27 17:55:18,049][20381] Updated weights for policy 1, policy_version 1770381 (0.0007) [2023-12-27 17:55:18,100][20381] Updated weights for policy 1, policy_version 1770391 (0.0010) [2023-12-27 17:55:18,201][20347] Updated weights for policy 0, policy_version 1771972 (0.0006) [2023-12-27 17:55:18,262][20347] Updated weights for policy 0, policy_version 1771982 (0.0009) [2023-12-27 17:55:18,307][20347] Updated weights for policy 0, policy_version 1771992 (0.0010) [2023-12-27 17:55:18,785][20381] Updated weights for policy 1, policy_version 1770401 (0.0010) [2023-12-27 17:55:18,847][20381] Updated weights for policy 1, policy_version 1770411 (0.0011) [2023-12-27 17:55:18,898][20381] Updated weights for policy 1, policy_version 1770421 (0.0011) [2023-12-27 17:55:18,949][20381] Updated weights for policy 1, policy_version 1770431 (0.0011) [2023-12-27 17:55:19,004][20347] Updated weights for policy 0, policy_version 1772002 (0.0011) [2023-12-27 17:55:19,060][20347] Updated weights for policy 0, policy_version 1772012 (0.0007) [2023-12-27 17:55:19,110][20347] Updated weights for policy 0, policy_version 1772022 (0.0007) [2023-12-27 17:55:19,161][20347] Updated weights for policy 0, policy_version 1772032 (0.0007) [2023-12-27 17:55:19,683][20381] Updated weights for policy 1, policy_version 1770441 (0.0011) [2023-12-27 17:55:19,743][20381] Updated weights for policy 1, policy_version 1770451 (0.0009) [2023-12-27 17:55:19,805][20381] Updated weights for policy 1, policy_version 1770461 (0.0006) [2023-12-27 17:55:19,920][20347] Updated weights for policy 0, policy_version 1772042 (0.0009) [2023-12-27 17:55:19,984][20347] Updated weights for policy 0, policy_version 1772052 (0.0008) [2023-12-27 17:55:20,046][20347] Updated weights for policy 0, policy_version 1772062 (0.0006) [2023-12-27 17:55:20,540][20381] Updated weights for policy 1, policy_version 1770471 (0.0008) [2023-12-27 17:55:20,614][20381] Updated weights for policy 1, policy_version 1770481 (0.0009) [2023-12-27 17:55:20,679][20381] Updated weights for policy 1, policy_version 1770491 (0.0009) [2023-12-27 17:55:20,728][20347] Updated weights for policy 0, policy_version 1772072 (0.0007) [2023-12-27 17:55:20,798][20347] Updated weights for policy 0, policy_version 1772082 (0.0008) [2023-12-27 17:55:20,859][20347] Updated weights for policy 0, policy_version 1772092 (0.0007) [2023-12-27 17:55:21,407][20381] Updated weights for policy 1, policy_version 1770501 (0.0009) [2023-12-27 17:55:21,468][20381] Updated weights for policy 1, policy_version 1770511 (0.0007) [2023-12-27 17:55:21,522][20381] Updated weights for policy 1, policy_version 1770521 (0.0007) [2023-12-27 17:55:21,619][20347] Updated weights for policy 0, policy_version 1772102 (0.0010) [2023-12-27 17:55:21,689][20347] Updated weights for policy 0, policy_version 1772112 (0.0011) [2023-12-27 17:55:21,759][20347] Updated weights for policy 0, policy_version 1772122 (0.0010) [2023-12-27 17:55:22,264][20381] Updated weights for policy 1, policy_version 1770531 (0.0007) [2023-12-27 17:55:22,323][20381] Updated weights for policy 1, policy_version 1770541 (0.0007) [2023-12-27 17:55:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 907051008. Throughput: 0: 10239.5, 1: 10228.8. Samples: 907029788. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:22,363][19170] Avg episode reward: [(0, '853.790'), (1, '1000.000')] [2023-12-27 17:55:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001772128_453746688.pth... [2023-12-27 17:55:22,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001770944_453443584.pth [2023-12-27 17:55:22,390][20381] Updated weights for policy 1, policy_version 1770551 (0.0008) [2023-12-27 17:55:22,440][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001770560_453312512.pth... [2023-12-27 17:55:22,444][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001769376_453009408.pth [2023-12-27 17:55:22,448][20347] Updated weights for policy 0, policy_version 1772132 (0.0007) [2023-12-27 17:55:22,517][20347] Updated weights for policy 0, policy_version 1772142 (0.0009) [2023-12-27 17:55:22,577][20347] Updated weights for policy 0, policy_version 1772152 (0.0011) [2023-12-27 17:55:23,116][20381] Updated weights for policy 1, policy_version 1770561 (0.0008) [2023-12-27 17:55:23,164][20381] Updated weights for policy 1, policy_version 1770571 (0.0008) [2023-12-27 17:55:23,211][20381] Updated weights for policy 1, policy_version 1770581 (0.0008) [2023-12-27 17:55:23,241][20347] Updated weights for policy 0, policy_version 1772162 (0.0009) [2023-12-27 17:55:23,258][20381] Updated weights for policy 1, policy_version 1770591 (0.0006) [2023-12-27 17:55:23,292][20347] Updated weights for policy 0, policy_version 1772172 (0.0007) [2023-12-27 17:55:23,342][20347] Updated weights for policy 0, policy_version 1772182 (0.0007) [2023-12-27 17:55:23,360][20291] KL-divergence is very high: 155.5920 [2023-12-27 17:55:23,406][20347] Updated weights for policy 0, policy_version 1772192 (0.0009) [2023-12-27 17:55:23,930][20381] Updated weights for policy 1, policy_version 1770601 (0.0008) [2023-12-27 17:55:23,988][20381] Updated weights for policy 1, policy_version 1770611 (0.0007) [2023-12-27 17:55:24,043][20381] Updated weights for policy 1, policy_version 1770621 (0.0008) [2023-12-27 17:55:24,061][20347] Updated weights for policy 0, policy_version 1772202 (0.0008) [2023-12-27 17:55:24,116][20347] Updated weights for policy 0, policy_version 1772212 (0.0010) [2023-12-27 17:55:24,175][20347] Updated weights for policy 0, policy_version 1772222 (0.0010) [2023-12-27 17:55:24,798][20381] Updated weights for policy 1, policy_version 1770631 (0.0008) [2023-12-27 17:55:24,816][20347] Updated weights for policy 0, policy_version 1772232 (0.0006) [2023-12-27 17:55:24,845][20381] Updated weights for policy 1, policy_version 1770641 (0.0007) [2023-12-27 17:55:24,872][20347] Updated weights for policy 0, policy_version 1772242 (0.0006) [2023-12-27 17:55:24,902][20381] Updated weights for policy 1, policy_version 1770651 (0.0005) [2023-12-27 17:55:24,929][20347] Updated weights for policy 0, policy_version 1772252 (0.0005) [2023-12-27 17:55:25,520][20381] Updated weights for policy 1, policy_version 1770661 (0.0010) [2023-12-27 17:55:25,564][20347] Updated weights for policy 0, policy_version 1772262 (0.0006) [2023-12-27 17:55:25,573][20381] Updated weights for policy 1, policy_version 1770671 (0.0010) [2023-12-27 17:55:25,616][20347] Updated weights for policy 0, policy_version 1772272 (0.0006) [2023-12-27 17:55:25,628][20381] Updated weights for policy 1, policy_version 1770681 (0.0009) [2023-12-27 17:55:25,675][20347] Updated weights for policy 0, policy_version 1772282 (0.0007) [2023-12-27 17:55:26,234][20381] Updated weights for policy 1, policy_version 1770691 (0.0005) [2023-12-27 17:55:26,290][20381] Updated weights for policy 1, policy_version 1770701 (0.0006) [2023-12-27 17:55:26,333][20381] Updated weights for policy 1, policy_version 1770711 (0.0007) [2023-12-27 17:55:26,400][20347] Updated weights for policy 0, policy_version 1772292 (0.0008) [2023-12-27 17:55:26,458][20347] Updated weights for policy 0, policy_version 1772302 (0.0010) [2023-12-27 17:55:26,509][20347] Updated weights for policy 0, policy_version 1772312 (0.0007) [2023-12-27 17:55:26,997][20381] Updated weights for policy 1, policy_version 1770721 (0.0008) [2023-12-27 17:55:27,041][20381] Updated weights for policy 1, policy_version 1770731 (0.0010) [2023-12-27 17:55:27,074][20347] Updated weights for policy 0, policy_version 1772322 (0.0006) [2023-12-27 17:55:27,091][20381] Updated weights for policy 1, policy_version 1770741 (0.0011) [2023-12-27 17:55:27,125][20347] Updated weights for policy 0, policy_version 1772332 (0.0010) [2023-12-27 17:55:27,136][20381] Updated weights for policy 1, policy_version 1770751 (0.0010) [2023-12-27 17:55:27,172][20347] Updated weights for policy 0, policy_version 1772342 (0.0008) [2023-12-27 17:55:27,223][20347] Updated weights for policy 0, policy_version 1772352 (0.0010) [2023-12-27 17:55:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 907165696. Throughput: 0: 10202.0, 1: 10186.5. Samples: 907150788. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:27,363][19170] Avg episode reward: [(0, '873.050'), (1, '1000.000')] [2023-12-27 17:55:27,819][20381] Updated weights for policy 1, policy_version 1770761 (0.0009) [2023-12-27 17:55:27,859][20347] Updated weights for policy 0, policy_version 1772362 (0.0005) [2023-12-27 17:55:27,868][20381] Updated weights for policy 1, policy_version 1770771 (0.0010) [2023-12-27 17:55:27,914][20347] Updated weights for policy 0, policy_version 1772372 (0.0008) [2023-12-27 17:55:27,922][20381] Updated weights for policy 1, policy_version 1770781 (0.0011) [2023-12-27 17:55:27,970][20347] Updated weights for policy 0, policy_version 1772382 (0.0010) [2023-12-27 17:55:28,600][20381] Updated weights for policy 1, policy_version 1770791 (0.0010) [2023-12-27 17:55:28,650][20347] Updated weights for policy 0, policy_version 1772392 (0.0009) [2023-12-27 17:55:28,652][20381] Updated weights for policy 1, policy_version 1770801 (0.0010) [2023-12-27 17:55:28,708][20381] Updated weights for policy 1, policy_version 1770811 (0.0009) [2023-12-27 17:55:28,711][20347] Updated weights for policy 0, policy_version 1772402 (0.0007) [2023-12-27 17:55:28,763][20347] Updated weights for policy 0, policy_version 1772412 (0.0008) [2023-12-27 17:55:29,399][20381] Updated weights for policy 1, policy_version 1770821 (0.0006) [2023-12-27 17:55:29,460][20381] Updated weights for policy 1, policy_version 1770831 (0.0005) [2023-12-27 17:55:29,473][20347] Updated weights for policy 0, policy_version 1772422 (0.0009) [2023-12-27 17:55:29,522][20381] Updated weights for policy 1, policy_version 1770841 (0.0007) [2023-12-27 17:55:29,530][20347] Updated weights for policy 0, policy_version 1772432 (0.0011) [2023-12-27 17:55:29,589][20347] Updated weights for policy 0, policy_version 1772442 (0.0010) [2023-12-27 17:55:30,136][20381] Updated weights for policy 1, policy_version 1770851 (0.0008) [2023-12-27 17:55:30,186][20381] Updated weights for policy 1, policy_version 1770861 (0.0007) [2023-12-27 17:55:30,241][20381] Updated weights for policy 1, policy_version 1770871 (0.0008) [2023-12-27 17:55:30,295][20347] Updated weights for policy 0, policy_version 1772452 (0.0009) [2023-12-27 17:55:30,356][20347] Updated weights for policy 0, policy_version 1772462 (0.0010) [2023-12-27 17:55:30,415][20347] Updated weights for policy 0, policy_version 1772472 (0.0010) [2023-12-27 17:55:30,850][20381] Updated weights for policy 1, policy_version 1770881 (0.0008) [2023-12-27 17:55:30,894][20381] Updated weights for policy 1, policy_version 1770891 (0.0007) [2023-12-27 17:55:30,941][20381] Updated weights for policy 1, policy_version 1770901 (0.0007) [2023-12-27 17:55:30,989][20381] Updated weights for policy 1, policy_version 1770911 (0.0008) [2023-12-27 17:55:31,083][20347] Updated weights for policy 0, policy_version 1772482 (0.0010) [2023-12-27 17:55:31,150][20347] Updated weights for policy 0, policy_version 1772492 (0.0010) [2023-12-27 17:55:31,217][20347] Updated weights for policy 0, policy_version 1772502 (0.0006) [2023-12-27 17:55:31,286][20347] Updated weights for policy 0, policy_version 1772512 (0.0007) [2023-12-27 17:55:31,698][20381] Updated weights for policy 1, policy_version 1770921 (0.0007) [2023-12-27 17:55:31,762][20381] Updated weights for policy 1, policy_version 1770931 (0.0008) [2023-12-27 17:55:31,815][20381] Updated weights for policy 1, policy_version 1770941 (0.0007) [2023-12-27 17:55:31,939][20347] Updated weights for policy 0, policy_version 1772522 (0.0007) [2023-12-27 17:55:31,997][20347] Updated weights for policy 0, policy_version 1772532 (0.0007) [2023-12-27 17:55:32,059][20347] Updated weights for policy 0, policy_version 1772542 (0.0006) [2023-12-27 17:55:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 907264000. Throughput: 0: 10217.1, 1: 10201.3. Samples: 907275796. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:32,363][19170] Avg episode reward: [(0, '882.790'), (1, '1000.000')] [2023-12-27 17:55:32,506][20381] Updated weights for policy 1, policy_version 1770951 (0.0007) [2023-12-27 17:55:32,564][20381] Updated weights for policy 1, policy_version 1770961 (0.0005) [2023-12-27 17:55:32,626][20381] Updated weights for policy 1, policy_version 1770971 (0.0005) [2023-12-27 17:55:32,767][20347] Updated weights for policy 0, policy_version 1772552 (0.0008) [2023-12-27 17:55:32,824][20347] Updated weights for policy 0, policy_version 1772562 (0.0007) [2023-12-27 17:55:32,879][20347] Updated weights for policy 0, policy_version 1772572 (0.0007) [2023-12-27 17:55:33,242][20381] Updated weights for policy 1, policy_version 1770981 (0.0005) [2023-12-27 17:55:33,296][20381] Updated weights for policy 1, policy_version 1770991 (0.0005) [2023-12-27 17:55:33,348][20381] Updated weights for policy 1, policy_version 1771001 (0.0005) [2023-12-27 17:55:33,520][20347] Updated weights for policy 0, policy_version 1772582 (0.0008) [2023-12-27 17:55:33,568][20347] Updated weights for policy 0, policy_version 1772592 (0.0008) [2023-12-27 17:55:33,615][20347] Updated weights for policy 0, policy_version 1772602 (0.0007) [2023-12-27 17:55:33,879][20381] Updated weights for policy 1, policy_version 1771011 (0.0005) [2023-12-27 17:55:33,929][20381] Updated weights for policy 1, policy_version 1771021 (0.0005) [2023-12-27 17:55:33,984][20381] Updated weights for policy 1, policy_version 1771031 (0.0005) [2023-12-27 17:55:34,278][20347] Updated weights for policy 0, policy_version 1772612 (0.0007) [2023-12-27 17:55:34,334][20347] Updated weights for policy 0, policy_version 1772622 (0.0007) [2023-12-27 17:55:34,394][20347] Updated weights for policy 0, policy_version 1772632 (0.0008) [2023-12-27 17:55:34,659][20381] Updated weights for policy 1, policy_version 1771041 (0.0007) [2023-12-27 17:55:34,729][20381] Updated weights for policy 1, policy_version 1771051 (0.0010) [2023-12-27 17:55:34,789][20381] Updated weights for policy 1, policy_version 1771061 (0.0007) [2023-12-27 17:55:34,838][20381] Updated weights for policy 1, policy_version 1771071 (0.0008) [2023-12-27 17:55:35,048][20347] Updated weights for policy 0, policy_version 1772642 (0.0006) [2023-12-27 17:55:35,100][20347] Updated weights for policy 0, policy_version 1772652 (0.0007) [2023-12-27 17:55:35,157][20347] Updated weights for policy 0, policy_version 1772662 (0.0007) [2023-12-27 17:55:35,208][20347] Updated weights for policy 0, policy_version 1772672 (0.0006) [2023-12-27 17:55:35,560][20381] Updated weights for policy 1, policy_version 1771081 (0.0007) [2023-12-27 17:55:35,621][20381] Updated weights for policy 1, policy_version 1771091 (0.0006) [2023-12-27 17:55:35,674][20381] Updated weights for policy 1, policy_version 1771101 (0.0008) [2023-12-27 17:55:35,785][20347] Updated weights for policy 0, policy_version 1772682 (0.0006) [2023-12-27 17:55:35,841][20347] Updated weights for policy 0, policy_version 1772692 (0.0005) [2023-12-27 17:55:35,892][20347] Updated weights for policy 0, policy_version 1772702 (0.0005) [2023-12-27 17:55:36,388][20381] Updated weights for policy 1, policy_version 1771111 (0.0008) [2023-12-27 17:55:36,448][20381] Updated weights for policy 1, policy_version 1771121 (0.0008) [2023-12-27 17:55:36,477][20347] Updated weights for policy 0, policy_version 1772712 (0.0005) [2023-12-27 17:55:36,505][20381] Updated weights for policy 1, policy_version 1771131 (0.0007) [2023-12-27 17:55:36,534][20347] Updated weights for policy 0, policy_version 1772722 (0.0005) [2023-12-27 17:55:36,591][20347] Updated weights for policy 0, policy_version 1772732 (0.0008) [2023-12-27 17:55:37,176][20381] Updated weights for policy 1, policy_version 1771141 (0.0006) [2023-12-27 17:55:37,235][20381] Updated weights for policy 1, policy_version 1771151 (0.0006) [2023-12-27 17:55:37,285][20347] Updated weights for policy 0, policy_version 1772742 (0.0009) [2023-12-27 17:55:37,297][20381] Updated weights for policy 1, policy_version 1771161 (0.0006) [2023-12-27 17:55:37,334][20347] Updated weights for policy 0, policy_version 1772752 (0.0010) [2023-12-27 17:55:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 907370496. Throughput: 0: 10236.6, 1: 10242.7. Samples: 907339472. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:37,363][19170] Avg episode reward: [(0, '892.810'), (1, '1000.000')] [2023-12-27 17:55:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001771168_453468160.pth... [2023-12-27 17:55:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001769984_453165056.pth [2023-12-27 17:55:37,389][20347] Updated weights for policy 0, policy_version 1772762 (0.0010) [2023-12-27 17:55:37,426][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001772768_453910528.pth... [2023-12-27 17:55:37,430][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001771552_453599232.pth [2023-12-27 17:55:37,888][20381] Updated weights for policy 1, policy_version 1771171 (0.0007) [2023-12-27 17:55:37,941][20381] Updated weights for policy 1, policy_version 1771181 (0.0006) [2023-12-27 17:55:37,999][20381] Updated weights for policy 1, policy_version 1771191 (0.0007) [2023-12-27 17:55:38,041][20347] Updated weights for policy 0, policy_version 1772772 (0.0010) [2023-12-27 17:55:38,096][20347] Updated weights for policy 0, policy_version 1772782 (0.0010) [2023-12-27 17:55:38,146][20347] Updated weights for policy 0, policy_version 1772792 (0.0009) [2023-12-27 17:55:38,719][20381] Updated weights for policy 1, policy_version 1771201 (0.0010) [2023-12-27 17:55:38,754][20347] Updated weights for policy 0, policy_version 1772802 (0.0010) [2023-12-27 17:55:38,783][20381] Updated weights for policy 1, policy_version 1771211 (0.0010) [2023-12-27 17:55:38,821][20347] Updated weights for policy 0, policy_version 1772812 (0.0007) [2023-12-27 17:55:38,850][20381] Updated weights for policy 1, policy_version 1771221 (0.0011) [2023-12-27 17:55:38,886][20347] Updated weights for policy 0, policy_version 1772822 (0.0007) [2023-12-27 17:55:38,907][20381] Updated weights for policy 1, policy_version 1771231 (0.0011) [2023-12-27 17:55:38,952][20347] Updated weights for policy 0, policy_version 1772832 (0.0007) [2023-12-27 17:55:39,607][20381] Updated weights for policy 1, policy_version 1771241 (0.0008) [2023-12-27 17:55:39,660][20381] Updated weights for policy 1, policy_version 1771251 (0.0008) [2023-12-27 17:55:39,675][20347] Updated weights for policy 0, policy_version 1772842 (0.0006) [2023-12-27 17:55:39,713][20381] Updated weights for policy 1, policy_version 1771261 (0.0008) [2023-12-27 17:55:39,735][20347] Updated weights for policy 0, policy_version 1772852 (0.0008) [2023-12-27 17:55:39,803][20347] Updated weights for policy 0, policy_version 1772862 (0.0006) [2023-12-27 17:55:40,498][20381] Updated weights for policy 1, policy_version 1771271 (0.0008) [2023-12-27 17:55:40,550][20381] Updated weights for policy 1, policy_version 1771281 (0.0007) [2023-12-27 17:55:40,563][20347] Updated weights for policy 0, policy_version 1772872 (0.0007) [2023-12-27 17:55:40,611][20381] Updated weights for policy 1, policy_version 1771291 (0.0006) [2023-12-27 17:55:40,620][20347] Updated weights for policy 0, policy_version 1772882 (0.0006) [2023-12-27 17:55:40,676][20347] Updated weights for policy 0, policy_version 1772892 (0.0007) [2023-12-27 17:55:41,338][20381] Updated weights for policy 1, policy_version 1771301 (0.0007) [2023-12-27 17:55:41,406][20381] Updated weights for policy 1, policy_version 1771311 (0.0008) [2023-12-27 17:55:41,419][20347] Updated weights for policy 0, policy_version 1772902 (0.0007) [2023-12-27 17:55:41,470][20381] Updated weights for policy 1, policy_version 1771321 (0.0007) [2023-12-27 17:55:41,481][20347] Updated weights for policy 0, policy_version 1772912 (0.0007) [2023-12-27 17:55:41,538][20347] Updated weights for policy 0, policy_version 1772922 (0.0007) [2023-12-27 17:55:42,174][20381] Updated weights for policy 1, policy_version 1771331 (0.0008) [2023-12-27 17:55:42,224][20381] Updated weights for policy 1, policy_version 1771341 (0.0011) [2023-12-27 17:55:42,263][20347] Updated weights for policy 0, policy_version 1772932 (0.0009) [2023-12-27 17:55:42,279][20381] Updated weights for policy 1, policy_version 1771351 (0.0009) [2023-12-27 17:55:42,332][20347] Updated weights for policy 0, policy_version 1772942 (0.0007) [2023-12-27 17:55:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 907468800. Throughput: 0: 10254.9, 1: 10172.1. Samples: 907461024. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:42,363][19170] Avg episode reward: [(0, '892.570'), (1, '1000.000')] [2023-12-27 17:55:42,394][20347] Updated weights for policy 0, policy_version 1772952 (0.0008) [2023-12-27 17:55:43,008][20381] Updated weights for policy 1, policy_version 1771361 (0.0007) [2023-12-27 17:55:43,068][20381] Updated weights for policy 1, policy_version 1771371 (0.0005) [2023-12-27 17:55:43,069][20347] Updated weights for policy 0, policy_version 1772962 (0.0007) [2023-12-27 17:55:43,122][20381] Updated weights for policy 1, policy_version 1771381 (0.0009) [2023-12-27 17:55:43,128][20347] Updated weights for policy 0, policy_version 1772972 (0.0007) [2023-12-27 17:55:43,172][20381] Updated weights for policy 1, policy_version 1771391 (0.0010) [2023-12-27 17:55:43,177][20347] Updated weights for policy 0, policy_version 1772982 (0.0007) [2023-12-27 17:55:43,235][20347] Updated weights for policy 0, policy_version 1772992 (0.0005) [2023-12-27 17:55:43,791][20381] Updated weights for policy 1, policy_version 1771401 (0.0006) [2023-12-27 17:55:43,842][20381] Updated weights for policy 1, policy_version 1771411 (0.0005) [2023-12-27 17:55:43,859][20347] Updated weights for policy 0, policy_version 1773002 (0.0010) [2023-12-27 17:55:43,893][20381] Updated weights for policy 1, policy_version 1771421 (0.0005) [2023-12-27 17:55:43,913][20347] Updated weights for policy 0, policy_version 1773012 (0.0010) [2023-12-27 17:55:43,967][20347] Updated weights for policy 0, policy_version 1773022 (0.0010) [2023-12-27 17:55:44,571][20381] Updated weights for policy 1, policy_version 1771431 (0.0007) [2023-12-27 17:55:44,630][20381] Updated weights for policy 1, policy_version 1771441 (0.0007) [2023-12-27 17:55:44,650][20347] Updated weights for policy 0, policy_version 1773032 (0.0010) [2023-12-27 17:55:44,684][20381] Updated weights for policy 1, policy_version 1771451 (0.0005) [2023-12-27 17:55:44,704][20347] Updated weights for policy 0, policy_version 1773042 (0.0010) [2023-12-27 17:55:44,756][20347] Updated weights for policy 0, policy_version 1773052 (0.0010) [2023-12-27 17:55:45,363][20381] Updated weights for policy 1, policy_version 1771461 (0.0005) [2023-12-27 17:55:45,433][20381] Updated weights for policy 1, policy_version 1771471 (0.0005) [2023-12-27 17:55:45,463][20347] Updated weights for policy 0, policy_version 1773062 (0.0008) [2023-12-27 17:55:45,494][20381] Updated weights for policy 1, policy_version 1771481 (0.0006) [2023-12-27 17:55:45,518][20347] Updated weights for policy 0, policy_version 1773072 (0.0007) [2023-12-27 17:55:45,576][20347] Updated weights for policy 0, policy_version 1773082 (0.0007) [2023-12-27 17:55:46,084][20381] Updated weights for policy 1, policy_version 1771491 (0.0007) [2023-12-27 17:55:46,135][20381] Updated weights for policy 1, policy_version 1771501 (0.0005) [2023-12-27 17:55:46,174][20347] Updated weights for policy 0, policy_version 1773092 (0.0007) [2023-12-27 17:55:46,191][20381] Updated weights for policy 1, policy_version 1771511 (0.0007) [2023-12-27 17:55:46,226][20347] Updated weights for policy 0, policy_version 1773102 (0.0005) [2023-12-27 17:55:46,296][20347] Updated weights for policy 0, policy_version 1773112 (0.0005) [2023-12-27 17:55:46,823][20381] Updated weights for policy 1, policy_version 1771521 (0.0007) [2023-12-27 17:55:46,869][20381] Updated weights for policy 1, policy_version 1771531 (0.0005) [2023-12-27 17:55:46,893][20347] Updated weights for policy 0, policy_version 1773122 (0.0006) [2023-12-27 17:55:46,917][20381] Updated weights for policy 1, policy_version 1771541 (0.0005) [2023-12-27 17:55:46,948][20347] Updated weights for policy 0, policy_version 1773132 (0.0010) [2023-12-27 17:55:46,968][20381] Updated weights for policy 1, policy_version 1771551 (0.0005) [2023-12-27 17:55:46,993][20347] Updated weights for policy 0, policy_version 1773142 (0.0010) [2023-12-27 17:55:47,039][20347] Updated weights for policy 0, policy_version 1773152 (0.0010) [2023-12-27 17:55:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 907575296. Throughput: 0: 10278.0, 1: 10186.9. Samples: 907585668. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:47,364][19170] Avg episode reward: [(0, '902.390'), (1, '1000.000')] [2023-12-27 17:55:47,569][20381] Updated weights for policy 1, policy_version 1771561 (0.0007) [2023-12-27 17:55:47,630][20381] Updated weights for policy 1, policy_version 1771571 (0.0005) [2023-12-27 17:55:47,687][20381] Updated weights for policy 1, policy_version 1771581 (0.0006) [2023-12-27 17:55:47,699][20347] Updated weights for policy 0, policy_version 1773162 (0.0009) [2023-12-27 17:55:47,760][20347] Updated weights for policy 0, policy_version 1773172 (0.0008) [2023-12-27 17:55:47,812][20347] Updated weights for policy 0, policy_version 1773182 (0.0008) [2023-12-27 17:55:48,262][20381] Updated weights for policy 1, policy_version 1771591 (0.0007) [2023-12-27 17:55:48,312][20381] Updated weights for policy 1, policy_version 1771601 (0.0007) [2023-12-27 17:55:48,369][20381] Updated weights for policy 1, policy_version 1771611 (0.0008) [2023-12-27 17:55:48,489][20347] Updated weights for policy 0, policy_version 1773192 (0.0008) [2023-12-27 17:55:48,540][20347] Updated weights for policy 0, policy_version 1773202 (0.0008) [2023-12-27 17:55:48,595][20347] Updated weights for policy 0, policy_version 1773212 (0.0008) [2023-12-27 17:55:49,077][20381] Updated weights for policy 1, policy_version 1771621 (0.0008) [2023-12-27 17:55:49,129][20381] Updated weights for policy 1, policy_version 1771631 (0.0007) [2023-12-27 17:55:49,184][20381] Updated weights for policy 1, policy_version 1771641 (0.0005) [2023-12-27 17:55:49,320][20347] Updated weights for policy 0, policy_version 1773222 (0.0008) [2023-12-27 17:55:49,400][20347] Updated weights for policy 0, policy_version 1773232 (0.0008) [2023-12-27 17:55:49,461][20347] Updated weights for policy 0, policy_version 1773242 (0.0007) [2023-12-27 17:55:49,920][20381] Updated weights for policy 1, policy_version 1771651 (0.0007) [2023-12-27 17:55:49,986][20381] Updated weights for policy 1, policy_version 1771661 (0.0009) [2023-12-27 17:55:50,047][20381] Updated weights for policy 1, policy_version 1771671 (0.0008) [2023-12-27 17:55:50,153][20347] Updated weights for policy 0, policy_version 1773252 (0.0008) [2023-12-27 17:55:50,205][20347] Updated weights for policy 0, policy_version 1773262 (0.0008) [2023-12-27 17:55:50,261][20347] Updated weights for policy 0, policy_version 1773272 (0.0007) [2023-12-27 17:55:50,701][20381] Updated weights for policy 1, policy_version 1771681 (0.0008) [2023-12-27 17:55:50,749][20381] Updated weights for policy 1, policy_version 1771691 (0.0008) [2023-12-27 17:55:50,795][20381] Updated weights for policy 1, policy_version 1771701 (0.0009) [2023-12-27 17:55:50,854][20381] Updated weights for policy 1, policy_version 1771711 (0.0007) [2023-12-27 17:55:50,943][20347] Updated weights for policy 0, policy_version 1773282 (0.0008) [2023-12-27 17:55:51,005][20347] Updated weights for policy 0, policy_version 1773292 (0.0007) [2023-12-27 17:55:51,070][20347] Updated weights for policy 0, policy_version 1773302 (0.0007) [2023-12-27 17:55:51,134][20347] Updated weights for policy 0, policy_version 1773312 (0.0008) [2023-12-27 17:55:51,569][20381] Updated weights for policy 1, policy_version 1771721 (0.0007) [2023-12-27 17:55:51,637][20381] Updated weights for policy 1, policy_version 1771731 (0.0007) [2023-12-27 17:55:51,709][20381] Updated weights for policy 1, policy_version 1771741 (0.0008) [2023-12-27 17:55:51,810][20347] Updated weights for policy 0, policy_version 1773322 (0.0009) [2023-12-27 17:55:51,868][20347] Updated weights for policy 0, policy_version 1773332 (0.0009) [2023-12-27 17:55:51,930][20347] Updated weights for policy 0, policy_version 1773342 (0.0008) [2023-12-27 17:55:52,358][20381] Updated weights for policy 1, policy_version 1771751 (0.0009) [2023-12-27 17:55:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 907673600. Throughput: 0: 10256.9, 1: 10205.8. Samples: 907646404. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:52,363][19170] Avg episode reward: [(0, '941.370'), (1, '1000.000')] [2023-12-27 17:55:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001773344_454057984.pth... [2023-12-27 17:55:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001772128_453746688.pth [2023-12-27 17:55:52,421][20381] Updated weights for policy 1, policy_version 1771761 (0.0008) [2023-12-27 17:55:52,481][20381] Updated weights for policy 1, policy_version 1771771 (0.0007) [2023-12-27 17:55:52,511][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001771776_453623808.pth... [2023-12-27 17:55:52,515][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001770560_453312512.pth [2023-12-27 17:55:52,683][20347] Updated weights for policy 0, policy_version 1773352 (0.0007) [2023-12-27 17:55:52,732][20347] Updated weights for policy 0, policy_version 1773362 (0.0007) [2023-12-27 17:55:52,783][20347] Updated weights for policy 0, policy_version 1773372 (0.0007) [2023-12-27 17:55:53,179][20381] Updated weights for policy 1, policy_version 1771781 (0.0009) [2023-12-27 17:55:53,226][20381] Updated weights for policy 1, policy_version 1771791 (0.0010) [2023-12-27 17:55:53,281][20381] Updated weights for policy 1, policy_version 1771801 (0.0011) [2023-12-27 17:55:53,501][20347] Updated weights for policy 0, policy_version 1773382 (0.0007) [2023-12-27 17:55:53,549][20347] Updated weights for policy 0, policy_version 1773392 (0.0008) [2023-12-27 17:55:53,606][20347] Updated weights for policy 0, policy_version 1773402 (0.0007) [2023-12-27 17:55:53,984][20381] Updated weights for policy 1, policy_version 1771811 (0.0011) [2023-12-27 17:55:54,038][20381] Updated weights for policy 1, policy_version 1771821 (0.0010) [2023-12-27 17:55:54,090][20381] Updated weights for policy 1, policy_version 1771831 (0.0010) [2023-12-27 17:55:54,230][20347] Updated weights for policy 0, policy_version 1773412 (0.0007) [2023-12-27 17:55:54,287][20347] Updated weights for policy 0, policy_version 1773422 (0.0005) [2023-12-27 17:55:54,334][20347] Updated weights for policy 0, policy_version 1773432 (0.0010) [2023-12-27 17:55:54,754][20381] Updated weights for policy 1, policy_version 1771841 (0.0010) [2023-12-27 17:55:54,803][20381] Updated weights for policy 1, policy_version 1771851 (0.0007) [2023-12-27 17:55:54,858][20381] Updated weights for policy 1, policy_version 1771861 (0.0007) [2023-12-27 17:55:54,909][20381] Updated weights for policy 1, policy_version 1771871 (0.0007) [2023-12-27 17:55:54,984][20347] Updated weights for policy 0, policy_version 1773442 (0.0010) [2023-12-27 17:55:55,044][20347] Updated weights for policy 0, policy_version 1773452 (0.0011) [2023-12-27 17:55:55,098][20347] Updated weights for policy 0, policy_version 1773462 (0.0010) [2023-12-27 17:55:55,153][20347] Updated weights for policy 0, policy_version 1773472 (0.0008) [2023-12-27 17:55:55,574][20381] Updated weights for policy 1, policy_version 1771881 (0.0006) [2023-12-27 17:55:55,626][20381] Updated weights for policy 1, policy_version 1771891 (0.0006) [2023-12-27 17:55:55,687][20381] Updated weights for policy 1, policy_version 1771901 (0.0006) [2023-12-27 17:55:55,712][20347] Updated weights for policy 0, policy_version 1773482 (0.0006) [2023-12-27 17:55:55,777][20347] Updated weights for policy 0, policy_version 1773492 (0.0006) [2023-12-27 17:55:55,831][20347] Updated weights for policy 0, policy_version 1773502 (0.0010) [2023-12-27 17:55:56,262][20381] Updated weights for policy 1, policy_version 1771911 (0.0006) [2023-12-27 17:55:56,315][20381] Updated weights for policy 1, policy_version 1771921 (0.0005) [2023-12-27 17:55:56,373][20381] Updated weights for policy 1, policy_version 1771931 (0.0007) [2023-12-27 17:55:56,450][20347] Updated weights for policy 0, policy_version 1773512 (0.0010) [2023-12-27 17:55:56,503][20347] Updated weights for policy 0, policy_version 1773522 (0.0009) [2023-12-27 17:55:56,556][20347] Updated weights for policy 0, policy_version 1773532 (0.0009) [2023-12-27 17:55:56,937][20381] Updated weights for policy 1, policy_version 1771941 (0.0006) [2023-12-27 17:55:56,982][20381] Updated weights for policy 1, policy_version 1771951 (0.0005) [2023-12-27 17:55:57,038][20381] Updated weights for policy 1, policy_version 1771961 (0.0005) [2023-12-27 17:55:57,171][20347] Updated weights for policy 0, policy_version 1773542 (0.0010) [2023-12-27 17:55:57,227][20347] Updated weights for policy 0, policy_version 1773552 (0.0010) [2023-12-27 17:55:57,277][20347] Updated weights for policy 0, policy_version 1773562 (0.0010) [2023-12-27 17:55:57,363][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 907788288. Throughput: 0: 10298.0, 1: 10250.5. Samples: 907771676. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:55:57,364][19170] Avg episode reward: [(0, '951.050'), (1, '1000.000')] [2023-12-27 17:55:57,646][20381] Updated weights for policy 1, policy_version 1771971 (0.0007) [2023-12-27 17:55:57,688][20381] Updated weights for policy 1, policy_version 1771981 (0.0009) [2023-12-27 17:55:57,739][20381] Updated weights for policy 1, policy_version 1771991 (0.0006) [2023-12-27 17:55:57,938][20347] Updated weights for policy 0, policy_version 1773572 (0.0010) [2023-12-27 17:55:57,986][20347] Updated weights for policy 0, policy_version 1773582 (0.0010) [2023-12-27 17:55:58,038][20347] Updated weights for policy 0, policy_version 1773592 (0.0006) [2023-12-27 17:55:58,405][20381] Updated weights for policy 1, policy_version 1772001 (0.0010) [2023-12-27 17:55:58,469][20381] Updated weights for policy 1, policy_version 1772011 (0.0010) [2023-12-27 17:55:58,529][20381] Updated weights for policy 1, policy_version 1772021 (0.0009) [2023-12-27 17:55:58,598][20381] Updated weights for policy 1, policy_version 1772031 (0.0007) [2023-12-27 17:55:58,712][20347] Updated weights for policy 0, policy_version 1773602 (0.0006) [2023-12-27 17:55:58,775][20347] Updated weights for policy 0, policy_version 1773612 (0.0007) [2023-12-27 17:55:58,844][20347] Updated weights for policy 0, policy_version 1773622 (0.0007) [2023-12-27 17:55:58,914][20347] Updated weights for policy 0, policy_version 1773632 (0.0007) [2023-12-27 17:55:59,412][20381] Updated weights for policy 1, policy_version 1772041 (0.0008) [2023-12-27 17:55:59,469][20381] Updated weights for policy 1, policy_version 1772051 (0.0008) [2023-12-27 17:55:59,522][20381] Updated weights for policy 1, policy_version 1772061 (0.0008) [2023-12-27 17:55:59,712][20347] Updated weights for policy 0, policy_version 1773642 (0.0007) [2023-12-27 17:55:59,763][20347] Updated weights for policy 0, policy_version 1773652 (0.0007) [2023-12-27 17:55:59,832][20347] Updated weights for policy 0, policy_version 1773662 (0.0008) [2023-12-27 17:56:00,219][20381] Updated weights for policy 1, policy_version 1772071 (0.0007) [2023-12-27 17:56:00,298][20381] Updated weights for policy 1, policy_version 1772081 (0.0006) [2023-12-27 17:56:00,350][20381] Updated weights for policy 1, policy_version 1772091 (0.0007) [2023-12-27 17:56:00,548][20347] Updated weights for policy 0, policy_version 1773672 (0.0007) [2023-12-27 17:56:00,603][20347] Updated weights for policy 0, policy_version 1773682 (0.0008) [2023-12-27 17:56:00,657][20347] Updated weights for policy 0, policy_version 1773692 (0.0006) [2023-12-27 17:56:00,971][20381] Updated weights for policy 1, policy_version 1772101 (0.0006) [2023-12-27 17:56:01,035][20381] Updated weights for policy 1, policy_version 1772111 (0.0007) [2023-12-27 17:56:01,104][20381] Updated weights for policy 1, policy_version 1772121 (0.0009) [2023-12-27 17:56:01,326][20347] Updated weights for policy 0, policy_version 1773702 (0.0007) [2023-12-27 17:56:01,397][20347] Updated weights for policy 0, policy_version 1773712 (0.0008) [2023-12-27 17:56:01,456][20347] Updated weights for policy 0, policy_version 1773722 (0.0009) [2023-12-27 17:56:01,782][20381] Updated weights for policy 1, policy_version 1772131 (0.0008) [2023-12-27 17:56:01,833][20381] Updated weights for policy 1, policy_version 1772141 (0.0008) [2023-12-27 17:56:01,891][20381] Updated weights for policy 1, policy_version 1772151 (0.0008) [2023-12-27 17:56:02,140][20347] Updated weights for policy 0, policy_version 1773732 (0.0008) [2023-12-27 17:56:02,196][20347] Updated weights for policy 0, policy_version 1773742 (0.0010) [2023-12-27 17:56:02,234][20291] KL-divergence is very high: 109.7086 [2023-12-27 17:56:02,244][20347] Updated weights for policy 0, policy_version 1773752 (0.0006) [2023-12-27 17:56:02,253][20291] KL-divergence is very high: 106.2116 [2023-12-27 17:56:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 907886592. Throughput: 0: 10273.7, 1: 10302.8. Samples: 907893956. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:02,363][19170] Avg episode reward: [(0, '912.200'), (1, '1000.000')] [2023-12-27 17:56:02,650][20381] Updated weights for policy 1, policy_version 1772161 (0.0008) [2023-12-27 17:56:02,709][20381] Updated weights for policy 1, policy_version 1772171 (0.0008) [2023-12-27 17:56:02,770][20381] Updated weights for policy 1, policy_version 1772181 (0.0006) [2023-12-27 17:56:02,834][20381] Updated weights for policy 1, policy_version 1772191 (0.0005) [2023-12-27 17:56:02,906][20347] Updated weights for policy 0, policy_version 1773762 (0.0009) [2023-12-27 17:56:02,954][20347] Updated weights for policy 0, policy_version 1773772 (0.0010) [2023-12-27 17:56:03,004][20347] Updated weights for policy 0, policy_version 1773782 (0.0010) [2023-12-27 17:56:03,055][20347] Updated weights for policy 0, policy_version 1773792 (0.0009) [2023-12-27 17:56:03,483][20381] Updated weights for policy 1, policy_version 1772201 (0.0007) [2023-12-27 17:56:03,539][20381] Updated weights for policy 1, policy_version 1772211 (0.0007) [2023-12-27 17:56:03,598][20381] Updated weights for policy 1, policy_version 1772221 (0.0007) [2023-12-27 17:56:03,702][20347] Updated weights for policy 0, policy_version 1773802 (0.0010) [2023-12-27 17:56:03,753][20347] Updated weights for policy 0, policy_version 1773812 (0.0010) [2023-12-27 17:56:03,806][20347] Updated weights for policy 0, policy_version 1773822 (0.0009) [2023-12-27 17:56:04,285][20381] Updated weights for policy 1, policy_version 1772231 (0.0007) [2023-12-27 17:56:04,345][20381] Updated weights for policy 1, policy_version 1772241 (0.0007) [2023-12-27 17:56:04,404][20381] Updated weights for policy 1, policy_version 1772251 (0.0006) [2023-12-27 17:56:04,512][20347] Updated weights for policy 0, policy_version 1773832 (0.0007) [2023-12-27 17:56:04,569][20347] Updated weights for policy 0, policy_version 1773842 (0.0008) [2023-12-27 17:56:04,624][20347] Updated weights for policy 0, policy_version 1773852 (0.0007) [2023-12-27 17:56:05,003][20381] Updated weights for policy 1, policy_version 1772261 (0.0007) [2023-12-27 17:56:05,060][20381] Updated weights for policy 1, policy_version 1772271 (0.0006) [2023-12-27 17:56:05,109][20381] Updated weights for policy 1, policy_version 1772281 (0.0005) [2023-12-27 17:56:05,320][20347] Updated weights for policy 0, policy_version 1773862 (0.0007) [2023-12-27 17:56:05,368][20347] Updated weights for policy 0, policy_version 1773872 (0.0007) [2023-12-27 17:56:05,416][20347] Updated weights for policy 0, policy_version 1773882 (0.0007) [2023-12-27 17:56:05,692][20381] Updated weights for policy 1, policy_version 1772291 (0.0005) [2023-12-27 17:56:05,745][20381] Updated weights for policy 1, policy_version 1772301 (0.0005) [2023-12-27 17:56:05,791][20381] Updated weights for policy 1, policy_version 1772311 (0.0006) [2023-12-27 17:56:06,174][20347] Updated weights for policy 0, policy_version 1773892 (0.0007) [2023-12-27 17:56:06,233][20347] Updated weights for policy 0, policy_version 1773902 (0.0007) [2023-12-27 17:56:06,299][20347] Updated weights for policy 0, policy_version 1773912 (0.0007) [2023-12-27 17:56:06,435][20381] Updated weights for policy 1, policy_version 1772321 (0.0006) [2023-12-27 17:56:06,493][20381] Updated weights for policy 1, policy_version 1772331 (0.0009) [2023-12-27 17:56:06,553][20381] Updated weights for policy 1, policy_version 1772341 (0.0011) [2023-12-27 17:56:06,613][20381] Updated weights for policy 1, policy_version 1772351 (0.0011) [2023-12-27 17:56:06,957][20347] Updated weights for policy 0, policy_version 1773922 (0.0006) [2023-12-27 17:56:07,016][20347] Updated weights for policy 0, policy_version 1773932 (0.0006) [2023-12-27 17:56:07,073][20347] Updated weights for policy 0, policy_version 1773942 (0.0005) [2023-12-27 17:56:07,126][20347] Updated weights for policy 0, policy_version 1773952 (0.0005) [2023-12-27 17:56:07,310][20381] Updated weights for policy 1, policy_version 1772361 (0.0009) [2023-12-27 17:56:07,361][20381] Updated weights for policy 1, policy_version 1772371 (0.0007) [2023-12-27 17:56:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 907984896. Throughput: 0: 10283.3, 1: 10319.7. Samples: 907956928. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:07,364][19170] Avg episode reward: [(0, '873.160'), (1, '1000.000')] [2023-12-27 17:56:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001773952_454213632.pth... [2023-12-27 17:56:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001772768_453910528.pth [2023-12-27 17:56:07,416][20381] Updated weights for policy 1, policy_version 1772381 (0.0010) [2023-12-27 17:56:07,431][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001772384_453779456.pth... [2023-12-27 17:56:07,434][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001771168_453468160.pth [2023-12-27 17:56:07,731][20347] Updated weights for policy 0, policy_version 1773962 (0.0010) [2023-12-27 17:56:07,789][20347] Updated weights for policy 0, policy_version 1773972 (0.0010) [2023-12-27 17:56:07,849][20347] Updated weights for policy 0, policy_version 1773982 (0.0010) [2023-12-27 17:56:08,019][20381] Updated weights for policy 1, policy_version 1772391 (0.0010) [2023-12-27 17:56:08,079][20381] Updated weights for policy 1, policy_version 1772401 (0.0011) [2023-12-27 17:56:08,137][20381] Updated weights for policy 1, policy_version 1772411 (0.0011) [2023-12-27 17:56:08,488][20347] Updated weights for policy 0, policy_version 1773992 (0.0010) [2023-12-27 17:56:08,553][20347] Updated weights for policy 0, policy_version 1774002 (0.0010) [2023-12-27 17:56:08,605][20347] Updated weights for policy 0, policy_version 1774012 (0.0010) [2023-12-27 17:56:08,730][20381] Updated weights for policy 1, policy_version 1772421 (0.0008) [2023-12-27 17:56:08,777][20381] Updated weights for policy 1, policy_version 1772431 (0.0005) [2023-12-27 17:56:08,840][20381] Updated weights for policy 1, policy_version 1772441 (0.0005) [2023-12-27 17:56:09,217][20347] Updated weights for policy 0, policy_version 1774022 (0.0008) [2023-12-27 17:56:09,278][20347] Updated weights for policy 0, policy_version 1774032 (0.0008) [2023-12-27 17:56:09,334][20347] Updated weights for policy 0, policy_version 1774042 (0.0008) [2023-12-27 17:56:09,508][20381] Updated weights for policy 1, policy_version 1772451 (0.0006) [2023-12-27 17:56:09,562][20381] Updated weights for policy 1, policy_version 1772461 (0.0007) [2023-12-27 17:56:09,622][20381] Updated weights for policy 1, policy_version 1772471 (0.0006) [2023-12-27 17:56:10,039][20347] Updated weights for policy 0, policy_version 1774052 (0.0008) [2023-12-27 17:56:10,092][20347] Updated weights for policy 0, policy_version 1774062 (0.0008) [2023-12-27 17:56:10,147][20347] Updated weights for policy 0, policy_version 1774072 (0.0008) [2023-12-27 17:56:10,301][20381] Updated weights for policy 1, policy_version 1772481 (0.0009) [2023-12-27 17:56:10,359][20381] Updated weights for policy 1, policy_version 1772491 (0.0008) [2023-12-27 17:56:10,410][20381] Updated weights for policy 1, policy_version 1772501 (0.0008) [2023-12-27 17:56:10,470][20381] Updated weights for policy 1, policy_version 1772511 (0.0008) [2023-12-27 17:56:10,833][20347] Updated weights for policy 0, policy_version 1774082 (0.0008) [2023-12-27 17:56:10,889][20347] Updated weights for policy 0, policy_version 1774092 (0.0011) [2023-12-27 17:56:10,942][20347] Updated weights for policy 0, policy_version 1774102 (0.0010) [2023-12-27 17:56:11,001][20347] Updated weights for policy 0, policy_version 1774112 (0.0010) [2023-12-27 17:56:11,200][20381] Updated weights for policy 1, policy_version 1772521 (0.0009) [2023-12-27 17:56:11,270][20381] Updated weights for policy 1, policy_version 1772531 (0.0008) [2023-12-27 17:56:11,322][20381] Updated weights for policy 1, policy_version 1772541 (0.0011) [2023-12-27 17:56:11,789][20347] Updated weights for policy 0, policy_version 1774122 (0.0008) [2023-12-27 17:56:11,836][20347] Updated weights for policy 0, policy_version 1774132 (0.0008) [2023-12-27 17:56:11,889][20347] Updated weights for policy 0, policy_version 1774142 (0.0007) [2023-12-27 17:56:12,078][20381] Updated weights for policy 1, policy_version 1772551 (0.0009) [2023-12-27 17:56:12,134][20381] Updated weights for policy 1, policy_version 1772561 (0.0008) [2023-12-27 17:56:12,190][20381] Updated weights for policy 1, policy_version 1772571 (0.0008) [2023-12-27 17:56:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 908091392. Throughput: 0: 10301.3, 1: 10367.6. Samples: 908080892. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:12,363][19170] Avg episode reward: [(0, '902.260'), (1, '980.560')] [2023-12-27 17:56:12,597][20347] Updated weights for policy 0, policy_version 1774152 (0.0008) [2023-12-27 17:56:12,656][20347] Updated weights for policy 0, policy_version 1774162 (0.0008) [2023-12-27 17:56:12,710][20347] Updated weights for policy 0, policy_version 1774172 (0.0007) [2023-12-27 17:56:12,940][20381] Updated weights for policy 1, policy_version 1772581 (0.0008) [2023-12-27 17:56:12,986][20381] Updated weights for policy 1, policy_version 1772591 (0.0007) [2023-12-27 17:56:13,039][20381] Updated weights for policy 1, policy_version 1772601 (0.0008) [2023-12-27 17:56:13,389][20347] Updated weights for policy 0, policy_version 1774182 (0.0006) [2023-12-27 17:56:13,442][20347] Updated weights for policy 0, policy_version 1774192 (0.0005) [2023-12-27 17:56:13,492][20347] Updated weights for policy 0, policy_version 1774202 (0.0008) [2023-12-27 17:56:13,757][20381] Updated weights for policy 1, policy_version 1772611 (0.0007) [2023-12-27 17:56:13,807][20381] Updated weights for policy 1, policy_version 1772621 (0.0008) [2023-12-27 17:56:13,858][20381] Updated weights for policy 1, policy_version 1772631 (0.0008) [2023-12-27 17:56:14,155][20347] Updated weights for policy 0, policy_version 1774212 (0.0008) [2023-12-27 17:56:14,199][20347] Updated weights for policy 0, policy_version 1774222 (0.0008) [2023-12-27 17:56:14,244][20347] Updated weights for policy 0, policy_version 1774232 (0.0007) [2023-12-27 17:56:14,568][20381] Updated weights for policy 1, policy_version 1772641 (0.0008) [2023-12-27 17:56:14,631][20381] Updated weights for policy 1, policy_version 1772651 (0.0008) [2023-12-27 17:56:14,692][20381] Updated weights for policy 1, policy_version 1772661 (0.0007) [2023-12-27 17:56:14,750][20381] Updated weights for policy 1, policy_version 1772671 (0.0008) [2023-12-27 17:56:14,909][20347] Updated weights for policy 0, policy_version 1774242 (0.0007) [2023-12-27 17:56:14,978][20347] Updated weights for policy 0, policy_version 1774252 (0.0007) [2023-12-27 17:56:14,999][20291] KL-divergence is very high: 210.0667 [2023-12-27 17:56:15,006][20291] KL-divergence is very high: 340.1475 [2023-12-27 17:56:15,043][20347] Updated weights for policy 0, policy_version 1774262 (0.0008) [2023-12-27 17:56:15,050][20291] KL-divergence is very high: 234.1826 [2023-12-27 17:56:15,057][20291] KL-divergence is very high: 326.4094 [2023-12-27 17:56:15,109][20347] Updated weights for policy 0, policy_version 1774272 (0.0007) [2023-12-27 17:56:15,367][20381] Updated weights for policy 1, policy_version 1772681 (0.0006) [2023-12-27 17:56:15,429][20381] Updated weights for policy 1, policy_version 1772691 (0.0005) [2023-12-27 17:56:15,493][20381] Updated weights for policy 1, policy_version 1772701 (0.0005) [2023-12-27 17:56:15,780][20347] Updated weights for policy 0, policy_version 1774282 (0.0006) [2023-12-27 17:56:15,833][20347] Updated weights for policy 0, policy_version 1774292 (0.0007) [2023-12-27 17:56:15,893][20347] Updated weights for policy 0, policy_version 1774302 (0.0008) [2023-12-27 17:56:16,086][20381] Updated weights for policy 1, policy_version 1772711 (0.0007) [2023-12-27 17:56:16,146][20381] Updated weights for policy 1, policy_version 1772721 (0.0008) [2023-12-27 17:56:16,205][20381] Updated weights for policy 1, policy_version 1772731 (0.0008) [2023-12-27 17:56:16,517][20347] Updated weights for policy 0, policy_version 1774312 (0.0010) [2023-12-27 17:56:16,576][20347] Updated weights for policy 0, policy_version 1774322 (0.0010) [2023-12-27 17:56:16,627][20347] Updated weights for policy 0, policy_version 1774332 (0.0008) [2023-12-27 17:56:16,848][20381] Updated weights for policy 1, policy_version 1772741 (0.0008) [2023-12-27 17:56:16,898][20381] Updated weights for policy 1, policy_version 1772751 (0.0007) [2023-12-27 17:56:16,949][20381] Updated weights for policy 1, policy_version 1772761 (0.0009) [2023-12-27 17:56:17,234][20347] Updated weights for policy 0, policy_version 1774342 (0.0009) [2023-12-27 17:56:17,291][20347] Updated weights for policy 0, policy_version 1774352 (0.0010) [2023-12-27 17:56:17,351][20347] Updated weights for policy 0, policy_version 1774362 (0.0010) [2023-12-27 17:56:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 908189696. Throughput: 0: 10287.8, 1: 10312.0. Samples: 908202788. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:17,363][19170] Avg episode reward: [(0, '863.480'), (1, '970.830')] [2023-12-27 17:56:17,595][20381] Updated weights for policy 1, policy_version 1772771 (0.0008) [2023-12-27 17:56:17,652][20381] Updated weights for policy 1, policy_version 1772781 (0.0005) [2023-12-27 17:56:17,703][20381] Updated weights for policy 1, policy_version 1772791 (0.0005) [2023-12-27 17:56:17,986][20347] Updated weights for policy 0, policy_version 1774372 (0.0007) [2023-12-27 17:56:18,044][20347] Updated weights for policy 0, policy_version 1774382 (0.0005) [2023-12-27 17:56:18,094][20347] Updated weights for policy 0, policy_version 1774392 (0.0006) [2023-12-27 17:56:18,298][20381] Updated weights for policy 1, policy_version 1772801 (0.0005) [2023-12-27 17:56:18,357][20381] Updated weights for policy 1, policy_version 1772811 (0.0008) [2023-12-27 17:56:18,407][20381] Updated weights for policy 1, policy_version 1772821 (0.0008) [2023-12-27 17:56:18,457][20381] Updated weights for policy 1, policy_version 1772831 (0.0008) [2023-12-27 17:56:18,696][20347] Updated weights for policy 0, policy_version 1774402 (0.0006) [2023-12-27 17:56:18,755][20347] Updated weights for policy 0, policy_version 1774412 (0.0010) [2023-12-27 17:56:18,814][20347] Updated weights for policy 0, policy_version 1774422 (0.0010) [2023-12-27 17:56:18,873][20347] Updated weights for policy 0, policy_version 1774432 (0.0010) [2023-12-27 17:56:19,143][20381] Updated weights for policy 1, policy_version 1772841 (0.0008) [2023-12-27 17:56:19,192][20381] Updated weights for policy 1, policy_version 1772851 (0.0008) [2023-12-27 17:56:19,259][20381] Updated weights for policy 1, policy_version 1772861 (0.0009) [2023-12-27 17:56:19,530][20347] Updated weights for policy 0, policy_version 1774442 (0.0008) [2023-12-27 17:56:19,581][20347] Updated weights for policy 0, policy_version 1774452 (0.0008) [2023-12-27 17:56:19,648][20347] Updated weights for policy 0, policy_version 1774462 (0.0008) [2023-12-27 17:56:20,094][20381] Updated weights for policy 1, policy_version 1772871 (0.0008) [2023-12-27 17:56:20,154][20381] Updated weights for policy 1, policy_version 1772881 (0.0008) [2023-12-27 17:56:20,215][20381] Updated weights for policy 1, policy_version 1772891 (0.0007) [2023-12-27 17:56:20,387][20347] Updated weights for policy 0, policy_version 1774472 (0.0010) [2023-12-27 17:56:20,441][20347] Updated weights for policy 0, policy_version 1774482 (0.0008) [2023-12-27 17:56:20,495][20347] Updated weights for policy 0, policy_version 1774492 (0.0010) [2023-12-27 17:56:20,953][20381] Updated weights for policy 1, policy_version 1772901 (0.0007) [2023-12-27 17:56:21,013][20381] Updated weights for policy 1, policy_version 1772911 (0.0008) [2023-12-27 17:56:21,077][20381] Updated weights for policy 1, policy_version 1772921 (0.0008) [2023-12-27 17:56:21,200][20347] Updated weights for policy 0, policy_version 1774502 (0.0010) [2023-12-27 17:56:21,258][20347] Updated weights for policy 0, policy_version 1774512 (0.0008) [2023-12-27 17:56:21,322][20347] Updated weights for policy 0, policy_version 1774522 (0.0009) [2023-12-27 17:56:21,790][20381] Updated weights for policy 1, policy_version 1772931 (0.0009) [2023-12-27 17:56:21,856][20381] Updated weights for policy 1, policy_version 1772941 (0.0008) [2023-12-27 17:56:21,919][20381] Updated weights for policy 1, policy_version 1772951 (0.0008) [2023-12-27 17:56:22,067][20347] Updated weights for policy 0, policy_version 1774532 (0.0011) [2023-12-27 17:56:22,129][20347] Updated weights for policy 0, policy_version 1774542 (0.0007) [2023-12-27 17:56:22,189][20347] Updated weights for policy 0, policy_version 1774552 (0.0005) [2023-12-27 17:56:22,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20466.1). Total num frames: 908296192. Throughput: 0: 10296.9, 1: 10286.5. Samples: 908265728. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:22,363][19170] Avg episode reward: [(0, '873.080'), (1, '990.270')] [2023-12-27 17:56:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001772960_453926912.pth... [2023-12-27 17:56:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001774560_454369280.pth... [2023-12-27 17:56:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001771776_453623808.pth [2023-12-27 17:56:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001773344_454057984.pth [2023-12-27 17:56:22,643][20381] Updated weights for policy 1, policy_version 1772961 (0.0008) [2023-12-27 17:56:22,703][20381] Updated weights for policy 1, policy_version 1772971 (0.0008) [2023-12-27 17:56:22,764][20381] Updated weights for policy 1, policy_version 1772981 (0.0008) [2023-12-27 17:56:22,824][20381] Updated weights for policy 1, policy_version 1772991 (0.0008) [2023-12-27 17:56:22,879][20347] Updated weights for policy 0, policy_version 1774562 (0.0006) [2023-12-27 17:56:22,935][20347] Updated weights for policy 0, policy_version 1774572 (0.0006) [2023-12-27 17:56:22,984][20347] Updated weights for policy 0, policy_version 1774582 (0.0007) [2023-12-27 17:56:23,036][20347] Updated weights for policy 0, policy_version 1774592 (0.0008) [2023-12-27 17:56:23,572][20381] Updated weights for policy 1, policy_version 1773001 (0.0007) [2023-12-27 17:56:23,590][20347] Updated weights for policy 0, policy_version 1774602 (0.0005) [2023-12-27 17:56:23,626][20381] Updated weights for policy 1, policy_version 1773011 (0.0008) [2023-12-27 17:56:23,645][20347] Updated weights for policy 0, policy_version 1774612 (0.0005) [2023-12-27 17:56:23,687][20381] Updated weights for policy 1, policy_version 1773021 (0.0008) [2023-12-27 17:56:23,691][20347] Updated weights for policy 0, policy_version 1774622 (0.0005) [2023-12-27 17:56:24,308][20381] Updated weights for policy 1, policy_version 1773031 (0.0006) [2023-12-27 17:56:24,338][20347] Updated weights for policy 0, policy_version 1774632 (0.0007) [2023-12-27 17:56:24,368][20381] Updated weights for policy 1, policy_version 1773041 (0.0006) [2023-12-27 17:56:24,384][20347] Updated weights for policy 0, policy_version 1774642 (0.0007) [2023-12-27 17:56:24,422][20381] Updated weights for policy 1, policy_version 1773051 (0.0006) [2023-12-27 17:56:24,430][20347] Updated weights for policy 0, policy_version 1774652 (0.0007) [2023-12-27 17:56:25,077][20381] Updated weights for policy 1, policy_version 1773061 (0.0007) [2023-12-27 17:56:25,126][20381] Updated weights for policy 1, policy_version 1773071 (0.0007) [2023-12-27 17:56:25,182][20347] Updated weights for policy 0, policy_version 1774662 (0.0008) [2023-12-27 17:56:25,187][20381] Updated weights for policy 1, policy_version 1773081 (0.0009) [2023-12-27 17:56:25,238][20347] Updated weights for policy 0, policy_version 1774672 (0.0006) [2023-12-27 17:56:25,294][20347] Updated weights for policy 0, policy_version 1774682 (0.0008) [2023-12-27 17:56:25,837][20381] Updated weights for policy 1, policy_version 1773091 (0.0006) [2023-12-27 17:56:25,896][20381] Updated weights for policy 1, policy_version 1773101 (0.0007) [2023-12-27 17:56:25,938][20347] Updated weights for policy 0, policy_version 1774692 (0.0008) [2023-12-27 17:56:25,947][20381] Updated weights for policy 1, policy_version 1773111 (0.0010) [2023-12-27 17:56:25,990][20347] Updated weights for policy 0, policy_version 1774702 (0.0005) [2023-12-27 17:56:26,046][20347] Updated weights for policy 0, policy_version 1774712 (0.0005) [2023-12-27 17:56:26,575][20347] Updated weights for policy 0, policy_version 1774722 (0.0005) [2023-12-27 17:56:26,576][20381] Updated weights for policy 1, policy_version 1773121 (0.0010) [2023-12-27 17:56:26,623][20347] Updated weights for policy 0, policy_version 1774732 (0.0005) [2023-12-27 17:56:26,639][20381] Updated weights for policy 1, policy_version 1773131 (0.0006) [2023-12-27 17:56:26,647][20291] KL-divergence is very high: 147.0002 [2023-12-27 17:56:26,657][20291] KL-divergence is very high: 118.3187 [2023-12-27 17:56:26,673][20347] Updated weights for policy 0, policy_version 1774742 (0.0005) [2023-12-27 17:56:26,691][20291] KL-divergence is very high: 265.7733 [2023-12-27 17:56:26,693][20381] Updated weights for policy 1, policy_version 1773141 (0.0006) [2023-12-27 17:56:26,704][20291] KL-divergence is very high: 174.4937 [2023-12-27 17:56:26,734][20347] Updated weights for policy 0, policy_version 1774752 (0.0005) [2023-12-27 17:56:26,744][20381] Updated weights for policy 1, policy_version 1773151 (0.0005) [2023-12-27 17:56:27,221][20291] KL-divergence is very high: 261.3443 [2023-12-27 17:56:27,272][20291] KL-divergence is very high: 193.6130 [2023-12-27 17:56:27,283][20347] Updated weights for policy 0, policy_version 1774762 (0.0008) [2023-12-27 17:56:27,323][20291] KL-divergence is very high: 141.4146 [2023-12-27 17:56:27,345][20347] Updated weights for policy 0, policy_version 1774772 (0.0008) [2023-12-27 17:56:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 908394496. Throughput: 0: 10278.4, 1: 10303.4. Samples: 908387204. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:27,364][19170] Avg episode reward: [(0, '882.750'), (1, '1000.000')] [2023-12-27 17:56:27,368][20291] KL-divergence is very high: 108.1611 [2023-12-27 17:56:27,399][20381] Updated weights for policy 1, policy_version 1773161 (0.0006) [2023-12-27 17:56:27,401][20347] Updated weights for policy 0, policy_version 1774782 (0.0006) [2023-12-27 17:56:27,445][20381] Updated weights for policy 1, policy_version 1773171 (0.0006) [2023-12-27 17:56:27,494][20381] Updated weights for policy 1, policy_version 1773181 (0.0005) [2023-12-27 17:56:28,012][20347] Updated weights for policy 0, policy_version 1774792 (0.0007) [2023-12-27 17:56:28,060][20347] Updated weights for policy 0, policy_version 1774802 (0.0005) [2023-12-27 17:56:28,094][20381] Updated weights for policy 1, policy_version 1773191 (0.0008) [2023-12-27 17:56:28,113][20347] Updated weights for policy 0, policy_version 1774812 (0.0005) [2023-12-27 17:56:28,144][20381] Updated weights for policy 1, policy_version 1773201 (0.0007) [2023-12-27 17:56:28,194][20381] Updated weights for policy 1, policy_version 1773211 (0.0009) [2023-12-27 17:56:28,742][20347] Updated weights for policy 0, policy_version 1774822 (0.0007) [2023-12-27 17:56:28,799][20347] Updated weights for policy 0, policy_version 1774832 (0.0008) [2023-12-27 17:56:28,851][20347] Updated weights for policy 0, policy_version 1774842 (0.0008) [2023-12-27 17:56:28,885][20381] Updated weights for policy 1, policy_version 1773221 (0.0008) [2023-12-27 17:56:28,939][20381] Updated weights for policy 1, policy_version 1773231 (0.0010) [2023-12-27 17:56:28,991][20381] Updated weights for policy 1, policy_version 1773241 (0.0010) [2023-12-27 17:56:29,594][20347] Updated weights for policy 0, policy_version 1774852 (0.0008) [2023-12-27 17:56:29,646][20347] Updated weights for policy 0, policy_version 1774862 (0.0008) [2023-12-27 17:56:29,682][20381] Updated weights for policy 1, policy_version 1773251 (0.0006) [2023-12-27 17:56:29,699][20347] Updated weights for policy 0, policy_version 1774872 (0.0006) [2023-12-27 17:56:29,739][20381] Updated weights for policy 1, policy_version 1773261 (0.0006) [2023-12-27 17:56:29,791][20381] Updated weights for policy 1, policy_version 1773271 (0.0008) [2023-12-27 17:56:30,389][20347] Updated weights for policy 0, policy_version 1774882 (0.0006) [2023-12-27 17:56:30,443][20347] Updated weights for policy 0, policy_version 1774892 (0.0007) [2023-12-27 17:56:30,503][20347] Updated weights for policy 0, policy_version 1774902 (0.0007) [2023-12-27 17:56:30,518][20381] Updated weights for policy 1, policy_version 1773281 (0.0009) [2023-12-27 17:56:30,558][20347] Updated weights for policy 0, policy_version 1774912 (0.0007) [2023-12-27 17:56:30,571][20381] Updated weights for policy 1, policy_version 1773291 (0.0006) [2023-12-27 17:56:30,627][20381] Updated weights for policy 1, policy_version 1773301 (0.0007) [2023-12-27 17:56:30,682][20381] Updated weights for policy 1, policy_version 1773311 (0.0006) [2023-12-27 17:56:31,277][20381] Updated weights for policy 1, policy_version 1773321 (0.0007) [2023-12-27 17:56:31,296][20347] Updated weights for policy 0, policy_version 1774922 (0.0008) [2023-12-27 17:56:31,328][20381] Updated weights for policy 1, policy_version 1773331 (0.0008) [2023-12-27 17:56:31,352][20347] Updated weights for policy 0, policy_version 1774932 (0.0007) [2023-12-27 17:56:31,389][20381] Updated weights for policy 1, policy_version 1773341 (0.0008) [2023-12-27 17:56:31,419][20347] Updated weights for policy 0, policy_version 1774942 (0.0009) [2023-12-27 17:56:32,120][20381] Updated weights for policy 1, policy_version 1773351 (0.0006) [2023-12-27 17:56:32,161][20347] Updated weights for policy 0, policy_version 1774952 (0.0008) [2023-12-27 17:56:32,177][20381] Updated weights for policy 1, policy_version 1773361 (0.0006) [2023-12-27 17:56:32,221][20347] Updated weights for policy 0, policy_version 1774962 (0.0009) [2023-12-27 17:56:32,234][20381] Updated weights for policy 1, policy_version 1773371 (0.0006) [2023-12-27 17:56:32,284][20347] Updated weights for policy 0, policy_version 1774972 (0.0011) [2023-12-27 17:56:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 908509184. Throughput: 0: 10326.2, 1: 10302.8. Samples: 908513968. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:32,363][19170] Avg episode reward: [(0, '892.420'), (1, '1000.000')] [2023-12-27 17:56:32,902][20381] Updated weights for policy 1, policy_version 1773381 (0.0009) [2023-12-27 17:56:32,909][20347] Updated weights for policy 0, policy_version 1774982 (0.0007) [2023-12-27 17:56:32,956][20381] Updated weights for policy 1, policy_version 1773391 (0.0010) [2023-12-27 17:56:32,964][20347] Updated weights for policy 0, policy_version 1774992 (0.0008) [2023-12-27 17:56:33,011][20381] Updated weights for policy 1, policy_version 1773401 (0.0010) [2023-12-27 17:56:33,019][20347] Updated weights for policy 0, policy_version 1775002 (0.0010) [2023-12-27 17:56:33,574][20381] Updated weights for policy 1, policy_version 1773411 (0.0009) [2023-12-27 17:56:33,620][20381] Updated weights for policy 1, policy_version 1773421 (0.0007) [2023-12-27 17:56:33,666][20381] Updated weights for policy 1, policy_version 1773431 (0.0007) [2023-12-27 17:56:33,672][20347] Updated weights for policy 0, policy_version 1775012 (0.0010) [2023-12-27 17:56:33,723][20347] Updated weights for policy 0, policy_version 1775022 (0.0010) [2023-12-27 17:56:33,783][20347] Updated weights for policy 0, policy_version 1775032 (0.0010) [2023-12-27 17:56:34,387][20381] Updated weights for policy 1, policy_version 1773441 (0.0006) [2023-12-27 17:56:34,445][20381] Updated weights for policy 1, policy_version 1773451 (0.0008) [2023-12-27 17:56:34,472][20347] Updated weights for policy 0, policy_version 1775042 (0.0010) [2023-12-27 17:56:34,508][20381] Updated weights for policy 1, policy_version 1773461 (0.0007) [2023-12-27 17:56:34,525][20347] Updated weights for policy 0, policy_version 1775052 (0.0010) [2023-12-27 17:56:34,564][20381] Updated weights for policy 1, policy_version 1773471 (0.0005) [2023-12-27 17:56:34,584][20347] Updated weights for policy 0, policy_version 1775062 (0.0010) [2023-12-27 17:56:34,643][20347] Updated weights for policy 0, policy_version 1775072 (0.0011) [2023-12-27 17:56:35,252][20381] Updated weights for policy 1, policy_version 1773481 (0.0007) [2023-12-27 17:56:35,308][20381] Updated weights for policy 1, policy_version 1773491 (0.0006) [2023-12-27 17:56:35,331][20347] Updated weights for policy 0, policy_version 1775082 (0.0010) [2023-12-27 17:56:35,359][20381] Updated weights for policy 1, policy_version 1773501 (0.0006) [2023-12-27 17:56:35,390][20347] Updated weights for policy 0, policy_version 1775092 (0.0011) [2023-12-27 17:56:35,448][20347] Updated weights for policy 0, policy_version 1775102 (0.0010) [2023-12-27 17:56:36,055][20381] Updated weights for policy 1, policy_version 1773511 (0.0007) [2023-12-27 17:56:36,113][20381] Updated weights for policy 1, policy_version 1773521 (0.0006) [2023-12-27 17:56:36,140][20347] Updated weights for policy 0, policy_version 1775112 (0.0010) [2023-12-27 17:56:36,177][20381] Updated weights for policy 1, policy_version 1773531 (0.0007) [2023-12-27 17:56:36,204][20347] Updated weights for policy 0, policy_version 1775122 (0.0008) [2023-12-27 17:56:36,269][20347] Updated weights for policy 0, policy_version 1775132 (0.0007) [2023-12-27 17:56:36,862][20381] Updated weights for policy 1, policy_version 1773541 (0.0007) [2023-12-27 17:56:36,919][20381] Updated weights for policy 1, policy_version 1773551 (0.0006) [2023-12-27 17:56:36,923][20347] Updated weights for policy 0, policy_version 1775142 (0.0009) [2023-12-27 17:56:36,980][20381] Updated weights for policy 1, policy_version 1773561 (0.0010) [2023-12-27 17:56:36,987][20347] Updated weights for policy 0, policy_version 1775152 (0.0010) [2023-12-27 17:56:37,044][20347] Updated weights for policy 0, policy_version 1775162 (0.0010) [2023-12-27 17:56:37,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 908607488. Throughput: 0: 10344.4, 1: 10318.5. Samples: 908576236. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:37,364][19170] Avg episode reward: [(0, '911.910'), (1, '1000.000')] [2023-12-27 17:56:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001773568_454082560.pth... [2023-12-27 17:56:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001775168_454524928.pth... [2023-12-27 17:56:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001772384_453779456.pth [2023-12-27 17:56:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001773952_454213632.pth [2023-12-27 17:56:37,547][20381] Updated weights for policy 1, policy_version 1773571 (0.0006) [2023-12-27 17:56:37,603][20381] Updated weights for policy 1, policy_version 1773581 (0.0005) [2023-12-27 17:56:37,659][20381] Updated weights for policy 1, policy_version 1773591 (0.0010) [2023-12-27 17:56:37,693][20347] Updated weights for policy 0, policy_version 1775172 (0.0009) [2023-12-27 17:56:37,759][20347] Updated weights for policy 0, policy_version 1775182 (0.0006) [2023-12-27 17:56:37,817][20347] Updated weights for policy 0, policy_version 1775192 (0.0006) [2023-12-27 17:56:38,221][20381] Updated weights for policy 1, policy_version 1773601 (0.0007) [2023-12-27 17:56:38,267][20381] Updated weights for policy 1, policy_version 1773611 (0.0005) [2023-12-27 17:56:38,316][20381] Updated weights for policy 1, policy_version 1773621 (0.0007) [2023-12-27 17:56:38,384][20381] Updated weights for policy 1, policy_version 1773631 (0.0008) [2023-12-27 17:56:38,510][20347] Updated weights for policy 0, policy_version 1775202 (0.0008) [2023-12-27 17:56:38,558][20347] Updated weights for policy 0, policy_version 1775212 (0.0010) [2023-12-27 17:56:38,608][20347] Updated weights for policy 0, policy_version 1775222 (0.0010) [2023-12-27 17:56:38,663][20347] Updated weights for policy 0, policy_version 1775232 (0.0010) [2023-12-27 17:56:39,066][20381] Updated weights for policy 1, policy_version 1773641 (0.0007) [2023-12-27 17:56:39,125][20381] Updated weights for policy 1, policy_version 1773651 (0.0005) [2023-12-27 17:56:39,175][20381] Updated weights for policy 1, policy_version 1773661 (0.0007) [2023-12-27 17:56:39,322][20347] Updated weights for policy 0, policy_version 1775242 (0.0007) [2023-12-27 17:56:39,388][20347] Updated weights for policy 0, policy_version 1775252 (0.0008) [2023-12-27 17:56:39,453][20347] Updated weights for policy 0, policy_version 1775262 (0.0008) [2023-12-27 17:56:39,808][20381] Updated weights for policy 1, policy_version 1773671 (0.0007) [2023-12-27 17:56:39,873][20381] Updated weights for policy 1, policy_version 1773681 (0.0008) [2023-12-27 17:56:39,937][20381] Updated weights for policy 1, policy_version 1773691 (0.0007) [2023-12-27 17:56:40,223][20347] Updated weights for policy 0, policy_version 1775272 (0.0007) [2023-12-27 17:56:40,283][20347] Updated weights for policy 0, policy_version 1775282 (0.0006) [2023-12-27 17:56:40,344][20347] Updated weights for policy 0, policy_version 1775292 (0.0008) [2023-12-27 17:56:40,671][20381] Updated weights for policy 1, policy_version 1773701 (0.0008) [2023-12-27 17:56:40,728][20381] Updated weights for policy 1, policy_version 1773711 (0.0008) [2023-12-27 17:56:40,784][20381] Updated weights for policy 1, policy_version 1773721 (0.0007) [2023-12-27 17:56:41,029][20347] Updated weights for policy 0, policy_version 1775302 (0.0007) [2023-12-27 17:56:41,094][20347] Updated weights for policy 0, policy_version 1775312 (0.0009) [2023-12-27 17:56:41,163][20347] Updated weights for policy 0, policy_version 1775322 (0.0011) [2023-12-27 17:56:41,517][20381] Updated weights for policy 1, policy_version 1773731 (0.0008) [2023-12-27 17:56:41,575][20381] Updated weights for policy 1, policy_version 1773741 (0.0008) [2023-12-27 17:56:41,638][20381] Updated weights for policy 1, policy_version 1773751 (0.0008) [2023-12-27 17:56:41,867][20347] Updated weights for policy 0, policy_version 1775332 (0.0007) [2023-12-27 17:56:41,916][20347] Updated weights for policy 0, policy_version 1775342 (0.0007) [2023-12-27 17:56:41,972][20347] Updated weights for policy 0, policy_version 1775352 (0.0008) [2023-12-27 17:56:42,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 908705792. Throughput: 0: 10286.3, 1: 10335.4. Samples: 908699652. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) [2023-12-27 17:56:42,363][19170] Avg episode reward: [(0, '931.670'), (1, '1000.000')] [2023-12-27 17:56:42,392][20381] Updated weights for policy 1, policy_version 1773761 (0.0008) [2023-12-27 17:56:42,452][20381] Updated weights for policy 1, policy_version 1773771 (0.0008) [2023-12-27 17:56:42,507][20381] Updated weights for policy 1, policy_version 1773781 (0.0009) [2023-12-27 17:56:42,556][20381] Updated weights for policy 1, policy_version 1773791 (0.0006) [2023-12-27 17:56:42,649][20347] Updated weights for policy 0, policy_version 1775362 (0.0008) [2023-12-27 17:56:42,707][20347] Updated weights for policy 0, policy_version 1775372 (0.0009) [2023-12-27 17:56:42,760][20347] Updated weights for policy 0, policy_version 1775382 (0.0010) [2023-12-27 17:56:42,824][20347] Updated weights for policy 0, policy_version 1775392 (0.0008) [2023-12-27 17:56:43,230][20381] Updated weights for policy 1, policy_version 1773801 (0.0007) [2023-12-27 17:56:43,285][20381] Updated weights for policy 1, policy_version 1773811 (0.0010) [2023-12-27 17:56:43,341][20381] Updated weights for policy 1, policy_version 1773821 (0.0010) [2023-12-27 17:56:43,479][20347] Updated weights for policy 0, policy_version 1775402 (0.0010) [2023-12-27 17:56:43,537][20347] Updated weights for policy 0, policy_version 1775412 (0.0010) [2023-12-27 17:56:43,596][20347] Updated weights for policy 0, policy_version 1775422 (0.0006) [2023-12-27 17:56:44,005][20381] Updated weights for policy 1, policy_version 1773831 (0.0009) [2023-12-27 17:56:44,053][20381] Updated weights for policy 1, policy_version 1773841 (0.0010) [2023-12-27 17:56:44,098][20381] Updated weights for policy 1, policy_version 1773851 (0.0010) [2023-12-27 17:56:44,250][20347] Updated weights for policy 0, policy_version 1775432 (0.0009) [2023-12-27 17:56:44,297][20347] Updated weights for policy 0, policy_version 1775442 (0.0010) [2023-12-27 17:56:44,354][20347] Updated weights for policy 0, policy_version 1775452 (0.0009) [2023-12-27 17:56:44,795][20381] Updated weights for policy 1, policy_version 1773861 (0.0010) [2023-12-27 17:56:44,854][20381] Updated weights for policy 1, policy_version 1773871 (0.0009) [2023-12-27 17:56:44,913][20381] Updated weights for policy 1, policy_version 1773881 (0.0007) [2023-12-27 17:56:45,034][20347] Updated weights for policy 0, policy_version 1775462 (0.0011) [2023-12-27 17:56:45,093][20347] Updated weights for policy 0, policy_version 1775472 (0.0010) [2023-12-27 17:56:45,158][20347] Updated weights for policy 0, policy_version 1775482 (0.0009) [2023-12-27 17:56:45,483][20381] Updated weights for policy 1, policy_version 1773891 (0.0009) [2023-12-27 17:56:45,548][20381] Updated weights for policy 1, policy_version 1773901 (0.0005) [2023-12-27 17:56:45,604][20381] Updated weights for policy 1, policy_version 1773911 (0.0005) [2023-12-27 17:56:45,839][20347] Updated weights for policy 0, policy_version 1775492 (0.0010) [2023-12-27 17:56:45,894][20347] Updated weights for policy 0, policy_version 1775502 (0.0010) [2023-12-27 17:56:45,948][20347] Updated weights for policy 0, policy_version 1775512 (0.0010) [2023-12-27 17:56:46,141][20381] Updated weights for policy 1, policy_version 1773921 (0.0007) [2023-12-27 17:56:46,192][20381] Updated weights for policy 1, policy_version 1773931 (0.0010) [2023-12-27 17:56:46,246][20381] Updated weights for policy 1, policy_version 1773941 (0.0010) [2023-12-27 17:56:46,300][20381] Updated weights for policy 1, policy_version 1773951 (0.0010) [2023-12-27 17:56:46,616][20347] Updated weights for policy 0, policy_version 1775522 (0.0010) [2023-12-27 17:56:46,672][20347] Updated weights for policy 0, policy_version 1775532 (0.0010) [2023-12-27 17:56:46,729][20347] Updated weights for policy 0, policy_version 1775542 (0.0010) [2023-12-27 17:56:46,774][20347] Updated weights for policy 0, policy_version 1775552 (0.0010) [2023-12-27 17:56:46,939][20381] Updated weights for policy 1, policy_version 1773961 (0.0008) [2023-12-27 17:56:46,989][20381] Updated weights for policy 1, policy_version 1773971 (0.0007) [2023-12-27 17:56:47,043][20381] Updated weights for policy 1, policy_version 1773981 (0.0008) [2023-12-27 17:56:47,363][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 908812288. Throughput: 0: 10301.9, 1: 10360.1. Samples: 908823748. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:56:47,364][19170] Avg episode reward: [(0, '912.410'), (1, '1000.000')] [2023-12-27 17:56:47,385][20347] Updated weights for policy 0, policy_version 1775562 (0.0007) [2023-12-27 17:56:47,433][20347] Updated weights for policy 0, policy_version 1775572 (0.0010) [2023-12-27 17:56:47,495][20347] Updated weights for policy 0, policy_version 1775582 (0.0010) [2023-12-27 17:56:47,737][20381] Updated weights for policy 1, policy_version 1773991 (0.0007) [2023-12-27 17:56:47,785][20381] Updated weights for policy 1, policy_version 1774001 (0.0007) [2023-12-27 17:56:47,843][20381] Updated weights for policy 1, policy_version 1774011 (0.0007) [2023-12-27 17:56:48,143][20347] Updated weights for policy 0, policy_version 1775592 (0.0010) [2023-12-27 17:56:48,203][20347] Updated weights for policy 0, policy_version 1775602 (0.0010) [2023-12-27 17:56:48,254][20347] Updated weights for policy 0, policy_version 1775612 (0.0010) [2023-12-27 17:56:48,573][20381] Updated weights for policy 1, policy_version 1774021 (0.0008) [2023-12-27 17:56:48,629][20381] Updated weights for policy 1, policy_version 1774031 (0.0008) [2023-12-27 17:56:48,679][20381] Updated weights for policy 1, policy_version 1774041 (0.0008) [2023-12-27 17:56:48,959][20347] Updated weights for policy 0, policy_version 1775622 (0.0010) [2023-12-27 17:56:49,020][20347] Updated weights for policy 0, policy_version 1775632 (0.0010) [2023-12-27 17:56:49,075][20347] Updated weights for policy 0, policy_version 1775642 (0.0010) [2023-12-27 17:56:49,394][20381] Updated weights for policy 1, policy_version 1774051 (0.0007) [2023-12-27 17:56:49,451][20381] Updated weights for policy 1, policy_version 1774061 (0.0006) [2023-12-27 17:56:49,509][20381] Updated weights for policy 1, policy_version 1774071 (0.0008) [2023-12-27 17:56:49,820][20347] Updated weights for policy 0, policy_version 1775652 (0.0010) [2023-12-27 17:56:49,879][20347] Updated weights for policy 0, policy_version 1775662 (0.0008) [2023-12-27 17:56:49,932][20347] Updated weights for policy 0, policy_version 1775672 (0.0007) [2023-12-27 17:56:50,173][20381] Updated weights for policy 1, policy_version 1774081 (0.0006) [2023-12-27 17:56:50,234][20381] Updated weights for policy 1, policy_version 1774091 (0.0008) [2023-12-27 17:56:50,283][20381] Updated weights for policy 1, policy_version 1774101 (0.0006) [2023-12-27 17:56:50,345][20381] Updated weights for policy 1, policy_version 1774111 (0.0006) [2023-12-27 17:56:50,684][20347] Updated weights for policy 0, policy_version 1775682 (0.0008) [2023-12-27 17:56:50,737][20347] Updated weights for policy 0, policy_version 1775692 (0.0007) [2023-12-27 17:56:50,787][20347] Updated weights for policy 0, policy_version 1775702 (0.0007) [2023-12-27 17:56:50,839][20347] Updated weights for policy 0, policy_version 1775712 (0.0007) [2023-12-27 17:56:51,055][20381] Updated weights for policy 1, policy_version 1774121 (0.0007) [2023-12-27 17:56:51,115][20381] Updated weights for policy 1, policy_version 1774131 (0.0008) [2023-12-27 17:56:51,174][20381] Updated weights for policy 1, policy_version 1774141 (0.0007) [2023-12-27 17:56:51,592][20347] Updated weights for policy 0, policy_version 1775722 (0.0007) [2023-12-27 17:56:51,653][20347] Updated weights for policy 0, policy_version 1775732 (0.0008) [2023-12-27 17:56:51,716][20347] Updated weights for policy 0, policy_version 1775742 (0.0008) [2023-12-27 17:56:51,913][20381] Updated weights for policy 1, policy_version 1774151 (0.0009) [2023-12-27 17:56:51,968][20381] Updated weights for policy 1, policy_version 1774161 (0.0006) [2023-12-27 17:56:52,017][20381] Updated weights for policy 1, policy_version 1774171 (0.0010) [2023-12-27 17:56:52,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 908910592. Throughput: 0: 10281.7, 1: 10315.6. Samples: 908883804. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:56:52,363][19170] Avg episode reward: [(0, '902.440'), (1, '990.250')] [2023-12-27 17:56:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001774176_454238208.pth... [2023-12-27 17:56:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001772960_453926912.pth [2023-12-27 17:56:52,415][20347] Updated weights for policy 0, policy_version 1775752 (0.0008) [2023-12-27 17:56:52,476][20347] Updated weights for policy 0, policy_version 1775762 (0.0006) [2023-12-27 17:56:52,536][20347] Updated weights for policy 0, policy_version 1775772 (0.0007) [2023-12-27 17:56:52,558][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001775776_454680576.pth... [2023-12-27 17:56:52,562][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001774560_454369280.pth [2023-12-27 17:56:52,688][20381] Updated weights for policy 1, policy_version 1774181 (0.0009) [2023-12-27 17:56:52,746][20381] Updated weights for policy 1, policy_version 1774191 (0.0008) [2023-12-27 17:56:52,806][20381] Updated weights for policy 1, policy_version 1774201 (0.0008) [2023-12-27 17:56:53,219][20347] Updated weights for policy 0, policy_version 1775782 (0.0007) [2023-12-27 17:56:53,268][20347] Updated weights for policy 0, policy_version 1775792 (0.0008) [2023-12-27 17:56:53,321][20347] Updated weights for policy 0, policy_version 1775802 (0.0008) [2023-12-27 17:56:53,467][20381] Updated weights for policy 1, policy_version 1774211 (0.0008) [2023-12-27 17:56:53,513][20381] Updated weights for policy 1, policy_version 1774221 (0.0009) [2023-12-27 17:56:53,569][20381] Updated weights for policy 1, policy_version 1774231 (0.0009) [2023-12-27 17:56:53,954][20347] Updated weights for policy 0, policy_version 1775812 (0.0007) [2023-12-27 17:56:54,003][20347] Updated weights for policy 0, policy_version 1775822 (0.0007) [2023-12-27 17:56:54,064][20347] Updated weights for policy 0, policy_version 1775832 (0.0008) [2023-12-27 17:56:54,284][20381] Updated weights for policy 1, policy_version 1774241 (0.0007) [2023-12-27 17:56:54,348][20381] Updated weights for policy 1, policy_version 1774251 (0.0008) [2023-12-27 17:56:54,408][20381] Updated weights for policy 1, policy_version 1774261 (0.0009) [2023-12-27 17:56:54,462][20381] Updated weights for policy 1, policy_version 1774271 (0.0008) [2023-12-27 17:56:54,705][20347] Updated weights for policy 0, policy_version 1775842 (0.0006) [2023-12-27 17:56:54,758][20347] Updated weights for policy 0, policy_version 1775852 (0.0005) [2023-12-27 17:56:54,813][20347] Updated weights for policy 0, policy_version 1775862 (0.0005) [2023-12-27 17:56:54,860][20347] Updated weights for policy 0, policy_version 1775872 (0.0005) [2023-12-27 17:56:55,208][20381] Updated weights for policy 1, policy_version 1774281 (0.0008) [2023-12-27 17:56:55,261][20381] Updated weights for policy 1, policy_version 1774291 (0.0008) [2023-12-27 17:56:55,307][20381] Updated weights for policy 1, policy_version 1774301 (0.0008) [2023-12-27 17:56:55,496][20347] Updated weights for policy 0, policy_version 1775882 (0.0006) [2023-12-27 17:56:55,551][20347] Updated weights for policy 0, policy_version 1775892 (0.0005) [2023-12-27 17:56:55,598][20347] Updated weights for policy 0, policy_version 1775902 (0.0006) [2023-12-27 17:56:56,080][20381] Updated weights for policy 1, policy_version 1774311 (0.0007) [2023-12-27 17:56:56,134][20381] Updated weights for policy 1, policy_version 1774321 (0.0005) [2023-12-27 17:56:56,153][20347] Updated weights for policy 0, policy_version 1775912 (0.0006) [2023-12-27 17:56:56,195][20381] Updated weights for policy 1, policy_version 1774331 (0.0007) [2023-12-27 17:56:56,212][20347] Updated weights for policy 0, policy_version 1775922 (0.0007) [2023-12-27 17:56:56,261][20347] Updated weights for policy 0, policy_version 1775932 (0.0005) [2023-12-27 17:56:56,725][20381] Updated weights for policy 1, policy_version 1774341 (0.0006) [2023-12-27 17:56:56,770][20381] Updated weights for policy 1, policy_version 1774351 (0.0005) [2023-12-27 17:56:56,819][20381] Updated weights for policy 1, policy_version 1774361 (0.0006) [2023-12-27 17:56:56,877][20347] Updated weights for policy 0, policy_version 1775942 (0.0007) [2023-12-27 17:56:56,924][20347] Updated weights for policy 0, policy_version 1775952 (0.0007) [2023-12-27 17:56:56,980][20347] Updated weights for policy 0, policy_version 1775962 (0.0007) [2023-12-27 17:56:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 909017088. Throughput: 0: 10327.1, 1: 10228.5. Samples: 909005892. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:56:57,363][19170] Avg episode reward: [(0, '921.770'), (1, '1000.000')] [2023-12-27 17:56:57,487][20381] Updated weights for policy 1, policy_version 1774371 (0.0008) [2023-12-27 17:56:57,532][20381] Updated weights for policy 1, policy_version 1774381 (0.0008) [2023-12-27 17:56:57,576][20381] Updated weights for policy 1, policy_version 1774391 (0.0009) [2023-12-27 17:56:57,620][20347] Updated weights for policy 0, policy_version 1775972 (0.0007) [2023-12-27 17:56:57,676][20347] Updated weights for policy 0, policy_version 1775982 (0.0005) [2023-12-27 17:56:57,733][20347] Updated weights for policy 0, policy_version 1775992 (0.0005) [2023-12-27 17:56:58,291][20381] Updated weights for policy 1, policy_version 1774401 (0.0007) [2023-12-27 17:56:58,365][20381] Updated weights for policy 1, policy_version 1774412 (0.0009) [2023-12-27 17:56:58,421][20347] Updated weights for policy 0, policy_version 1776002 (0.0007) [2023-12-27 17:56:58,434][20381] Updated weights for policy 1, policy_version 1774422 (0.0009) [2023-12-27 17:56:58,484][20347] Updated weights for policy 0, policy_version 1776012 (0.0008) [2023-12-27 17:56:58,491][20381] Updated weights for policy 1, policy_version 1774432 (0.0006) [2023-12-27 17:56:58,551][20347] Updated weights for policy 0, policy_version 1776022 (0.0008) [2023-12-27 17:56:58,618][20347] Updated weights for policy 0, policy_version 1776032 (0.0009) [2023-12-27 17:56:59,304][20381] Updated weights for policy 1, policy_version 1774442 (0.0008) [2023-12-27 17:56:59,361][20381] Updated weights for policy 1, policy_version 1774452 (0.0007) [2023-12-27 17:56:59,389][20347] Updated weights for policy 0, policy_version 1776042 (0.0009) [2023-12-27 17:56:59,424][20381] Updated weights for policy 1, policy_version 1774462 (0.0008) [2023-12-27 17:56:59,445][20347] Updated weights for policy 0, policy_version 1776052 (0.0008) [2023-12-27 17:56:59,501][20347] Updated weights for policy 0, policy_version 1776062 (0.0010) [2023-12-27 17:57:00,112][20381] Updated weights for policy 1, policy_version 1774472 (0.0005) [2023-12-27 17:57:00,169][20347] Updated weights for policy 0, policy_version 1776072 (0.0010) [2023-12-27 17:57:00,169][20381] Updated weights for policy 1, policy_version 1774482 (0.0005) [2023-12-27 17:57:00,222][20347] Updated weights for policy 0, policy_version 1776082 (0.0009) [2023-12-27 17:57:00,228][20381] Updated weights for policy 1, policy_version 1774492 (0.0007) [2023-12-27 17:57:00,277][20347] Updated weights for policy 0, policy_version 1776092 (0.0005) [2023-12-27 17:57:00,798][20381] Updated weights for policy 1, policy_version 1774502 (0.0006) [2023-12-27 17:57:00,842][20381] Updated weights for policy 1, policy_version 1774512 (0.0005) [2023-12-27 17:57:00,900][20381] Updated weights for policy 1, policy_version 1774522 (0.0005) [2023-12-27 17:57:00,922][20347] Updated weights for policy 0, policy_version 1776102 (0.0005) [2023-12-27 17:57:00,969][20347] Updated weights for policy 0, policy_version 1776112 (0.0005) [2023-12-27 17:57:01,017][20347] Updated weights for policy 0, policy_version 1776122 (0.0010) [2023-12-27 17:57:01,589][20381] Updated weights for policy 1, policy_version 1774532 (0.0006) [2023-12-27 17:57:01,645][20347] Updated weights for policy 0, policy_version 1776132 (0.0010) [2023-12-27 17:57:01,655][20381] Updated weights for policy 1, policy_version 1774542 (0.0008) [2023-12-27 17:57:01,707][20347] Updated weights for policy 0, policy_version 1776142 (0.0009) [2023-12-27 17:57:01,710][20381] Updated weights for policy 1, policy_version 1774552 (0.0006) [2023-12-27 17:57:01,769][20347] Updated weights for policy 0, policy_version 1776152 (0.0007) [2023-12-27 17:57:02,363][20381] Updated weights for policy 1, policy_version 1774562 (0.0007) [2023-12-27 17:57:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 909115392. Throughput: 0: 10335.2, 1: 10257.2. Samples: 909129448. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:02,363][19170] Avg episode reward: [(0, '941.270'), (1, '990.250')] [2023-12-27 17:57:02,424][20381] Updated weights for policy 1, policy_version 1774572 (0.0007) [2023-12-27 17:57:02,466][20347] Updated weights for policy 0, policy_version 1776162 (0.0010) [2023-12-27 17:57:02,475][20381] Updated weights for policy 1, policy_version 1774582 (0.0006) [2023-12-27 17:57:02,525][20347] Updated weights for policy 0, policy_version 1776172 (0.0010) [2023-12-27 17:57:02,532][20381] Updated weights for policy 1, policy_version 1774592 (0.0007) [2023-12-27 17:57:02,581][20347] Updated weights for policy 0, policy_version 1776182 (0.0011) [2023-12-27 17:57:02,643][20347] Updated weights for policy 0, policy_version 1776192 (0.0010) [2023-12-27 17:57:03,134][20381] Updated weights for policy 1, policy_version 1774602 (0.0007) [2023-12-27 17:57:03,182][20381] Updated weights for policy 1, policy_version 1774612 (0.0008) [2023-12-27 17:57:03,230][20381] Updated weights for policy 1, policy_version 1774622 (0.0007) [2023-12-27 17:57:03,309][20347] Updated weights for policy 0, policy_version 1776202 (0.0010) [2023-12-27 17:57:03,364][20347] Updated weights for policy 0, policy_version 1776212 (0.0009) [2023-12-27 17:57:03,422][20347] Updated weights for policy 0, policy_version 1776222 (0.0007) [2023-12-27 17:57:03,905][20381] Updated weights for policy 1, policy_version 1774632 (0.0007) [2023-12-27 17:57:03,965][20381] Updated weights for policy 1, policy_version 1774642 (0.0008) [2023-12-27 17:57:04,035][20347] Updated weights for policy 0, policy_version 1776232 (0.0007) [2023-12-27 17:57:04,036][20381] Updated weights for policy 1, policy_version 1774652 (0.0008) [2023-12-27 17:57:04,093][20347] Updated weights for policy 0, policy_version 1776242 (0.0008) [2023-12-27 17:57:04,161][20347] Updated weights for policy 0, policy_version 1776252 (0.0008) [2023-12-27 17:57:04,710][20381] Updated weights for policy 1, policy_version 1774662 (0.0008) [2023-12-27 17:57:04,766][20381] Updated weights for policy 1, policy_version 1774672 (0.0006) [2023-12-27 17:57:04,827][20381] Updated weights for policy 1, policy_version 1774682 (0.0007) [2023-12-27 17:57:04,862][20347] Updated weights for policy 0, policy_version 1776262 (0.0006) [2023-12-27 17:57:04,908][20347] Updated weights for policy 0, policy_version 1776272 (0.0005) [2023-12-27 17:57:04,958][20347] Updated weights for policy 0, policy_version 1776282 (0.0005) [2023-12-27 17:57:05,525][20347] Updated weights for policy 0, policy_version 1776292 (0.0005) [2023-12-27 17:57:05,548][20381] Updated weights for policy 1, policy_version 1774692 (0.0007) [2023-12-27 17:57:05,580][20347] Updated weights for policy 0, policy_version 1776302 (0.0006) [2023-12-27 17:57:05,603][20381] Updated weights for policy 1, policy_version 1774702 (0.0008) [2023-12-27 17:57:05,632][20347] Updated weights for policy 0, policy_version 1776312 (0.0005) [2023-12-27 17:57:05,657][20381] Updated weights for policy 1, policy_version 1774712 (0.0006) [2023-12-27 17:57:06,215][20347] Updated weights for policy 0, policy_version 1776322 (0.0005) [2023-12-27 17:57:06,276][20347] Updated weights for policy 0, policy_version 1776332 (0.0006) [2023-12-27 17:57:06,341][20347] Updated weights for policy 0, policy_version 1776342 (0.0008) [2023-12-27 17:57:06,400][20347] Updated weights for policy 0, policy_version 1776352 (0.0007) [2023-12-27 17:57:06,440][20381] Updated weights for policy 1, policy_version 1774722 (0.0007) [2023-12-27 17:57:06,499][20381] Updated weights for policy 1, policy_version 1774732 (0.0007) [2023-12-27 17:57:06,564][20381] Updated weights for policy 1, policy_version 1774742 (0.0007) [2023-12-27 17:57:06,627][20381] Updated weights for policy 1, policy_version 1774752 (0.0007) [2023-12-27 17:57:07,075][20347] Updated weights for policy 0, policy_version 1776362 (0.0008) [2023-12-27 17:57:07,138][20347] Updated weights for policy 0, policy_version 1776372 (0.0008) [2023-12-27 17:57:07,197][20347] Updated weights for policy 0, policy_version 1776382 (0.0007) [2023-12-27 17:57:07,296][20381] Updated weights for policy 1, policy_version 1774762 (0.0008) [2023-12-27 17:57:07,353][20381] Updated weights for policy 1, policy_version 1774772 (0.0007) [2023-12-27 17:57:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 909221888. Throughput: 0: 10329.0, 1: 10256.6. Samples: 909192080. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:07,363][19170] Avg episode reward: [(0, '892.370'), (1, '990.250')] [2023-12-27 17:57:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001776384_454836224.pth... [2023-12-27 17:57:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001775168_454524928.pth [2023-12-27 17:57:07,406][20381] Updated weights for policy 1, policy_version 1774782 (0.0008) [2023-12-27 17:57:07,416][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001774784_454393856.pth... [2023-12-27 17:57:07,420][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001773568_454082560.pth [2023-12-27 17:57:07,892][20347] Updated weights for policy 0, policy_version 1776392 (0.0007) [2023-12-27 17:57:07,941][20347] Updated weights for policy 0, policy_version 1776402 (0.0009) [2023-12-27 17:57:07,994][20347] Updated weights for policy 0, policy_version 1776412 (0.0010) [2023-12-27 17:57:08,025][20381] Updated weights for policy 1, policy_version 1774792 (0.0009) [2023-12-27 17:57:08,081][20381] Updated weights for policy 1, policy_version 1774802 (0.0008) [2023-12-27 17:57:08,139][20381] Updated weights for policy 1, policy_version 1774812 (0.0007) [2023-12-27 17:57:08,586][20347] Updated weights for policy 0, policy_version 1776422 (0.0007) [2023-12-27 17:57:08,643][20347] Updated weights for policy 0, policy_version 1776432 (0.0008) [2023-12-27 17:57:08,696][20347] Updated weights for policy 0, policy_version 1776442 (0.0008) [2023-12-27 17:57:08,865][20381] Updated weights for policy 1, policy_version 1774822 (0.0006) [2023-12-27 17:57:08,912][20381] Updated weights for policy 1, policy_version 1774832 (0.0008) [2023-12-27 17:57:08,964][20381] Updated weights for policy 1, policy_version 1774842 (0.0008) [2023-12-27 17:57:09,399][20347] Updated weights for policy 0, policy_version 1776452 (0.0008) [2023-12-27 17:57:09,462][20347] Updated weights for policy 0, policy_version 1776462 (0.0007) [2023-12-27 17:57:09,528][20347] Updated weights for policy 0, policy_version 1776472 (0.0008) [2023-12-27 17:57:09,727][20381] Updated weights for policy 1, policy_version 1774852 (0.0009) [2023-12-27 17:57:09,785][20381] Updated weights for policy 1, policy_version 1774862 (0.0010) [2023-12-27 17:57:09,847][20381] Updated weights for policy 1, policy_version 1774872 (0.0008) [2023-12-27 17:57:10,256][20347] Updated weights for policy 0, policy_version 1776482 (0.0007) [2023-12-27 17:57:10,324][20347] Updated weights for policy 0, policy_version 1776492 (0.0007) [2023-12-27 17:57:10,388][20347] Updated weights for policy 0, policy_version 1776502 (0.0006) [2023-12-27 17:57:10,453][20347] Updated weights for policy 0, policy_version 1776512 (0.0007) [2023-12-27 17:57:10,574][20381] Updated weights for policy 1, policy_version 1774882 (0.0007) [2023-12-27 17:57:10,630][20381] Updated weights for policy 1, policy_version 1774892 (0.0007) [2023-12-27 17:57:10,676][20292] KL-divergence is very high: 103.8063 [2023-12-27 17:57:10,686][20381] Updated weights for policy 1, policy_version 1774902 (0.0005) [2023-12-27 17:57:10,745][20381] Updated weights for policy 1, policy_version 1774912 (0.0006) [2023-12-27 17:57:11,148][20347] Updated weights for policy 0, policy_version 1776522 (0.0008) [2023-12-27 17:57:11,206][20347] Updated weights for policy 0, policy_version 1776532 (0.0008) [2023-12-27 17:57:11,266][20347] Updated weights for policy 0, policy_version 1776542 (0.0007) [2023-12-27 17:57:11,429][20381] Updated weights for policy 1, policy_version 1774922 (0.0008) [2023-12-27 17:57:11,491][20381] Updated weights for policy 1, policy_version 1774932 (0.0008) [2023-12-27 17:57:11,545][20381] Updated weights for policy 1, policy_version 1774942 (0.0007) [2023-12-27 17:57:11,973][20347] Updated weights for policy 0, policy_version 1776552 (0.0008) [2023-12-27 17:57:12,034][20347] Updated weights for policy 0, policy_version 1776562 (0.0008) [2023-12-27 17:57:12,092][20347] Updated weights for policy 0, policy_version 1776572 (0.0008) [2023-12-27 17:57:12,234][20381] Updated weights for policy 1, policy_version 1774952 (0.0007) [2023-12-27 17:57:12,293][20381] Updated weights for policy 1, policy_version 1774962 (0.0007) [2023-12-27 17:57:12,357][20381] Updated weights for policy 1, policy_version 1774972 (0.0008) [2023-12-27 17:57:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 909320192. Throughput: 0: 10344.8, 1: 10259.0. Samples: 909314372. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:12,363][19170] Avg episode reward: [(0, '853.190'), (1, '990.270')] [2023-12-27 17:57:12,806][20347] Updated weights for policy 0, policy_version 1776582 (0.0007) [2023-12-27 17:57:12,868][20347] Updated weights for policy 0, policy_version 1776592 (0.0010) [2023-12-27 17:57:12,928][20347] Updated weights for policy 0, policy_version 1776602 (0.0010) [2023-12-27 17:57:12,934][20381] Updated weights for policy 1, policy_version 1774982 (0.0008) [2023-12-27 17:57:12,988][20381] Updated weights for policy 1, policy_version 1774992 (0.0006) [2023-12-27 17:57:13,048][20381] Updated weights for policy 1, policy_version 1775002 (0.0008) [2023-12-27 17:57:13,562][20347] Updated weights for policy 0, policy_version 1776612 (0.0010) [2023-12-27 17:57:13,616][20347] Updated weights for policy 0, policy_version 1776622 (0.0010) [2023-12-27 17:57:13,669][20347] Updated weights for policy 0, policy_version 1776632 (0.0010) [2023-12-27 17:57:13,751][20381] Updated weights for policy 1, policy_version 1775012 (0.0008) [2023-12-27 17:57:13,802][20381] Updated weights for policy 1, policy_version 1775022 (0.0006) [2023-12-27 17:57:13,857][20381] Updated weights for policy 1, policy_version 1775032 (0.0005) [2023-12-27 17:57:14,336][20347] Updated weights for policy 0, policy_version 1776642 (0.0009) [2023-12-27 17:57:14,390][20347] Updated weights for policy 0, policy_version 1776652 (0.0006) [2023-12-27 17:57:14,440][20347] Updated weights for policy 0, policy_version 1776662 (0.0005) [2023-12-27 17:57:14,477][20381] Updated weights for policy 1, policy_version 1775042 (0.0006) [2023-12-27 17:57:14,498][20347] Updated weights for policy 0, policy_version 1776672 (0.0005) [2023-12-27 17:57:14,535][20381] Updated weights for policy 1, policy_version 1775052 (0.0008) [2023-12-27 17:57:14,583][20381] Updated weights for policy 1, policy_version 1775062 (0.0009) [2023-12-27 17:57:14,649][20381] Updated weights for policy 1, policy_version 1775072 (0.0007) [2023-12-27 17:57:15,148][20347] Updated weights for policy 0, policy_version 1776682 (0.0008) [2023-12-27 17:57:15,207][20347] Updated weights for policy 0, policy_version 1776692 (0.0008) [2023-12-27 17:57:15,270][20347] Updated weights for policy 0, policy_version 1776702 (0.0007) [2023-12-27 17:57:15,343][20381] Updated weights for policy 1, policy_version 1775082 (0.0011) [2023-12-27 17:57:15,394][20381] Updated weights for policy 1, policy_version 1775092 (0.0010) [2023-12-27 17:57:15,452][20381] Updated weights for policy 1, policy_version 1775102 (0.0010) [2023-12-27 17:57:15,950][20347] Updated weights for policy 0, policy_version 1776712 (0.0006) [2023-12-27 17:57:16,011][20347] Updated weights for policy 0, policy_version 1776722 (0.0006) [2023-12-27 17:57:16,039][20381] Updated weights for policy 1, policy_version 1775112 (0.0010) [2023-12-27 17:57:16,066][20347] Updated weights for policy 0, policy_version 1776732 (0.0006) [2023-12-27 17:57:16,094][20381] Updated weights for policy 1, policy_version 1775122 (0.0010) [2023-12-27 17:57:16,145][20381] Updated weights for policy 1, policy_version 1775132 (0.0010) [2023-12-27 17:57:16,699][20347] Updated weights for policy 0, policy_version 1776742 (0.0006) [2023-12-27 17:57:16,754][20347] Updated weights for policy 0, policy_version 1776752 (0.0006) [2023-12-27 17:57:16,779][20381] Updated weights for policy 1, policy_version 1775142 (0.0010) [2023-12-27 17:57:16,803][20347] Updated weights for policy 0, policy_version 1776762 (0.0006) [2023-12-27 17:57:16,840][20381] Updated weights for policy 1, policy_version 1775152 (0.0010) [2023-12-27 17:57:16,891][20381] Updated weights for policy 1, policy_version 1775162 (0.0010) [2023-12-27 17:57:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 909426688. Throughput: 0: 10275.8, 1: 10259.3. Samples: 909438048. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:17,363][19170] Avg episode reward: [(0, '892.510'), (1, '990.270')] [2023-12-27 17:57:17,513][20347] Updated weights for policy 0, policy_version 1776772 (0.0007) [2023-12-27 17:57:17,524][20381] Updated weights for policy 1, policy_version 1775172 (0.0009) [2023-12-27 17:57:17,562][20347] Updated weights for policy 0, policy_version 1776782 (0.0007) [2023-12-27 17:57:17,578][20381] Updated weights for policy 1, policy_version 1775182 (0.0005) [2023-12-27 17:57:17,611][20347] Updated weights for policy 0, policy_version 1776792 (0.0008) [2023-12-27 17:57:17,632][20381] Updated weights for policy 1, policy_version 1775192 (0.0005) [2023-12-27 17:57:18,242][20381] Updated weights for policy 1, policy_version 1775202 (0.0011) [2023-12-27 17:57:18,296][20381] Updated weights for policy 1, policy_version 1775212 (0.0010) [2023-12-27 17:57:18,298][20347] Updated weights for policy 0, policy_version 1776802 (0.0008) [2023-12-27 17:57:18,355][20347] Updated weights for policy 0, policy_version 1776812 (0.0009) [2023-12-27 17:57:18,361][20381] Updated weights for policy 1, policy_version 1775222 (0.0009) [2023-12-27 17:57:18,411][20381] Updated weights for policy 1, policy_version 1775232 (0.0010) [2023-12-27 17:57:18,415][20347] Updated weights for policy 0, policy_version 1776822 (0.0010) [2023-12-27 17:57:18,476][20347] Updated weights for policy 0, policy_version 1776832 (0.0011) [2023-12-27 17:57:19,091][20381] Updated weights for policy 1, policy_version 1775242 (0.0009) [2023-12-27 17:57:19,147][20381] Updated weights for policy 1, policy_version 1775252 (0.0008) [2023-12-27 17:57:19,153][20347] Updated weights for policy 0, policy_version 1776842 (0.0010) [2023-12-27 17:57:19,204][20347] Updated weights for policy 0, policy_version 1776852 (0.0010) [2023-12-27 17:57:19,209][20381] Updated weights for policy 1, policy_version 1775262 (0.0007) [2023-12-27 17:57:19,266][20347] Updated weights for policy 0, policy_version 1776862 (0.0010) [2023-12-27 17:57:19,937][20381] Updated weights for policy 1, policy_version 1775272 (0.0007) [2023-12-27 17:57:19,996][20381] Updated weights for policy 1, policy_version 1775282 (0.0006) [2023-12-27 17:57:20,002][20347] Updated weights for policy 0, policy_version 1776872 (0.0011) [2023-12-27 17:57:20,060][20381] Updated weights for policy 1, policy_version 1775292 (0.0006) [2023-12-27 17:57:20,066][20347] Updated weights for policy 0, policy_version 1776882 (0.0011) [2023-12-27 17:57:20,125][20347] Updated weights for policy 0, policy_version 1776892 (0.0011) [2023-12-27 17:57:20,786][20347] Updated weights for policy 0, policy_version 1776902 (0.0009) [2023-12-27 17:57:20,824][20381] Updated weights for policy 1, policy_version 1775302 (0.0006) [2023-12-27 17:57:20,846][20347] Updated weights for policy 0, policy_version 1776912 (0.0007) [2023-12-27 17:57:20,885][20381] Updated weights for policy 1, policy_version 1775312 (0.0008) [2023-12-27 17:57:20,908][20347] Updated weights for policy 0, policy_version 1776922 (0.0006) [2023-12-27 17:57:20,947][20381] Updated weights for policy 1, policy_version 1775322 (0.0007) [2023-12-27 17:57:21,627][20347] Updated weights for policy 0, policy_version 1776932 (0.0009) [2023-12-27 17:57:21,694][20347] Updated weights for policy 0, policy_version 1776942 (0.0007) [2023-12-27 17:57:21,713][20381] Updated weights for policy 1, policy_version 1775332 (0.0008) [2023-12-27 17:57:21,764][20347] Updated weights for policy 0, policy_version 1776952 (0.0009) [2023-12-27 17:57:21,783][20381] Updated weights for policy 1, policy_version 1775342 (0.0007) [2023-12-27 17:57:21,846][20381] Updated weights for policy 1, policy_version 1775352 (0.0006) [2023-12-27 17:57:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 909524992. Throughput: 0: 10254.9, 1: 10258.4. Samples: 909499332. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:22,363][19170] Avg episode reward: [(0, '912.180'), (1, '1000.000')] [2023-12-27 17:57:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001775360_454541312.pth... [2023-12-27 17:57:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001776960_454983680.pth... [2023-12-27 17:57:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001775776_454680576.pth [2023-12-27 17:57:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001774176_454238208.pth [2023-12-27 17:57:22,437][20381] Updated weights for policy 1, policy_version 1775362 (0.0006) [2023-12-27 17:57:22,463][20347] Updated weights for policy 0, policy_version 1776962 (0.0011) [2023-12-27 17:57:22,498][20381] Updated weights for policy 1, policy_version 1775372 (0.0007) [2023-12-27 17:57:22,520][20347] Updated weights for policy 0, policy_version 1776972 (0.0010) [2023-12-27 17:57:22,556][20381] Updated weights for policy 1, policy_version 1775382 (0.0006) [2023-12-27 17:57:22,580][20347] Updated weights for policy 0, policy_version 1776982 (0.0010) [2023-12-27 17:57:22,614][20381] Updated weights for policy 1, policy_version 1775392 (0.0007) [2023-12-27 17:57:22,635][20347] Updated weights for policy 0, policy_version 1776992 (0.0008) [2023-12-27 17:57:23,285][20347] Updated weights for policy 0, policy_version 1777002 (0.0007) [2023-12-27 17:57:23,307][20381] Updated weights for policy 1, policy_version 1775402 (0.0006) [2023-12-27 17:57:23,339][20347] Updated weights for policy 0, policy_version 1777012 (0.0007) [2023-12-27 17:57:23,364][20381] Updated weights for policy 1, policy_version 1775412 (0.0006) [2023-12-27 17:57:23,389][20347] Updated weights for policy 0, policy_version 1777022 (0.0007) [2023-12-27 17:57:23,416][20381] Updated weights for policy 1, policy_version 1775422 (0.0007) [2023-12-27 17:57:24,107][20347] Updated weights for policy 0, policy_version 1777032 (0.0007) [2023-12-27 17:57:24,110][20381] Updated weights for policy 1, policy_version 1775432 (0.0007) [2023-12-27 17:57:24,155][20381] Updated weights for policy 1, policy_version 1775442 (0.0005) [2023-12-27 17:57:24,168][20347] Updated weights for policy 0, policy_version 1777042 (0.0007) [2023-12-27 17:57:24,204][20381] Updated weights for policy 1, policy_version 1775452 (0.0005) [2023-12-27 17:57:24,216][20347] Updated weights for policy 0, policy_version 1777052 (0.0007) [2023-12-27 17:57:24,798][20381] Updated weights for policy 1, policy_version 1775462 (0.0005) [2023-12-27 17:57:24,803][20347] Updated weights for policy 0, policy_version 1777062 (0.0007) [2023-12-27 17:57:24,852][20381] Updated weights for policy 1, policy_version 1775472 (0.0008) [2023-12-27 17:57:24,857][20347] Updated weights for policy 0, policy_version 1777072 (0.0006) [2023-12-27 17:57:24,904][20381] Updated weights for policy 1, policy_version 1775482 (0.0010) [2023-12-27 17:57:24,918][20347] Updated weights for policy 0, policy_version 1777082 (0.0006) [2023-12-27 17:57:25,536][20381] Updated weights for policy 1, policy_version 1775492 (0.0007) [2023-12-27 17:57:25,592][20381] Updated weights for policy 1, policy_version 1775502 (0.0007) [2023-12-27 17:57:25,612][20347] Updated weights for policy 0, policy_version 1777092 (0.0006) [2023-12-27 17:57:25,649][20381] Updated weights for policy 1, policy_version 1775512 (0.0007) [2023-12-27 17:57:25,668][20347] Updated weights for policy 0, policy_version 1777102 (0.0007) [2023-12-27 17:57:25,719][20347] Updated weights for policy 0, policy_version 1777112 (0.0008) [2023-12-27 17:57:26,299][20381] Updated weights for policy 1, policy_version 1775522 (0.0006) [2023-12-27 17:57:26,355][20381] Updated weights for policy 1, policy_version 1775532 (0.0008) [2023-12-27 17:57:26,402][20381] Updated weights for policy 1, policy_version 1775542 (0.0007) [2023-12-27 17:57:26,423][20347] Updated weights for policy 0, policy_version 1777122 (0.0008) [2023-12-27 17:57:26,450][20381] Updated weights for policy 1, policy_version 1775552 (0.0006) [2023-12-27 17:57:26,478][20347] Updated weights for policy 0, policy_version 1777132 (0.0010) [2023-12-27 17:57:26,532][20347] Updated weights for policy 0, policy_version 1777142 (0.0010) [2023-12-27 17:57:26,591][20347] Updated weights for policy 0, policy_version 1777152 (0.0010) [2023-12-27 17:57:27,172][20381] Updated weights for policy 1, policy_version 1775562 (0.0006) [2023-12-27 17:57:27,222][20347] Updated weights for policy 0, policy_version 1777163 (0.0007) [2023-12-27 17:57:27,230][20381] Updated weights for policy 1, policy_version 1775572 (0.0006) [2023-12-27 17:57:27,276][20381] Updated weights for policy 1, policy_version 1775582 (0.0005) [2023-12-27 17:57:27,280][20347] Updated weights for policy 0, policy_version 1777174 (0.0007) [2023-12-27 17:57:27,342][20347] Updated weights for policy 0, policy_version 1777184 (0.0007) [2023-12-27 17:57:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 909639680. Throughput: 0: 10257.3, 1: 10234.8. Samples: 909621792. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:27,363][19170] Avg episode reward: [(0, '882.950'), (1, '990.250')] [2023-12-27 17:57:27,872][20381] Updated weights for policy 1, policy_version 1775592 (0.0007) [2023-12-27 17:57:27,932][20381] Updated weights for policy 1, policy_version 1775602 (0.0006) [2023-12-27 17:57:27,957][20347] Updated weights for policy 0, policy_version 1777194 (0.0006) [2023-12-27 17:57:27,993][20381] Updated weights for policy 1, policy_version 1775612 (0.0006) [2023-12-27 17:57:28,010][20347] Updated weights for policy 0, policy_version 1777204 (0.0007) [2023-12-27 17:57:28,065][20347] Updated weights for policy 0, policy_version 1777214 (0.0005) [2023-12-27 17:57:28,551][20381] Updated weights for policy 1, policy_version 1775622 (0.0007) [2023-12-27 17:57:28,610][20381] Updated weights for policy 1, policy_version 1775632 (0.0008) [2023-12-27 17:57:28,675][20381] Updated weights for policy 1, policy_version 1775642 (0.0007) [2023-12-27 17:57:28,708][20347] Updated weights for policy 0, policy_version 1777224 (0.0006) [2023-12-27 17:57:28,763][20347] Updated weights for policy 0, policy_version 1777234 (0.0006) [2023-12-27 17:57:28,815][20347] Updated weights for policy 0, policy_version 1777244 (0.0009) [2023-12-27 17:57:29,220][20381] Updated weights for policy 1, policy_version 1775652 (0.0006) [2023-12-27 17:57:29,284][20381] Updated weights for policy 1, policy_version 1775662 (0.0008) [2023-12-27 17:57:29,352][20381] Updated weights for policy 1, policy_version 1775672 (0.0007) [2023-12-27 17:57:29,571][20347] Updated weights for policy 0, policy_version 1777254 (0.0007) [2023-12-27 17:57:29,631][20347] Updated weights for policy 0, policy_version 1777264 (0.0008) [2023-12-27 17:57:29,679][20347] Updated weights for policy 0, policy_version 1777274 (0.0007) [2023-12-27 17:57:30,022][20381] Updated weights for policy 1, policy_version 1775682 (0.0008) [2023-12-27 17:57:30,083][20381] Updated weights for policy 1, policy_version 1775692 (0.0008) [2023-12-27 17:57:30,139][20381] Updated weights for policy 1, policy_version 1775702 (0.0008) [2023-12-27 17:57:30,192][20381] Updated weights for policy 1, policy_version 1775712 (0.0008) [2023-12-27 17:57:30,404][20347] Updated weights for policy 0, policy_version 1777284 (0.0007) [2023-12-27 17:57:30,458][20347] Updated weights for policy 0, policy_version 1777294 (0.0008) [2023-12-27 17:57:30,513][20347] Updated weights for policy 0, policy_version 1777304 (0.0007) [2023-12-27 17:57:30,863][20381] Updated weights for policy 1, policy_version 1775722 (0.0008) [2023-12-27 17:57:30,918][20381] Updated weights for policy 1, policy_version 1775732 (0.0008) [2023-12-27 17:57:30,968][20381] Updated weights for policy 1, policy_version 1775742 (0.0008) [2023-12-27 17:57:31,193][20347] Updated weights for policy 0, policy_version 1777314 (0.0008) [2023-12-27 17:57:31,249][20347] Updated weights for policy 0, policy_version 1777324 (0.0008) [2023-12-27 17:57:31,305][20347] Updated weights for policy 0, policy_version 1777334 (0.0008) [2023-12-27 17:57:31,367][20347] Updated weights for policy 0, policy_version 1777344 (0.0008) [2023-12-27 17:57:31,663][20381] Updated weights for policy 1, policy_version 1775752 (0.0007) [2023-12-27 17:57:31,714][20381] Updated weights for policy 1, policy_version 1775762 (0.0005) [2023-12-27 17:57:31,775][20381] Updated weights for policy 1, policy_version 1775772 (0.0006) [2023-12-27 17:57:32,128][20347] Updated weights for policy 0, policy_version 1777354 (0.0007) [2023-12-27 17:57:32,188][20347] Updated weights for policy 0, policy_version 1777364 (0.0007) [2023-12-27 17:57:32,254][20347] Updated weights for policy 0, policy_version 1777374 (0.0008) [2023-12-27 17:57:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 909737984. Throughput: 0: 10254.2, 1: 10269.6. Samples: 909747320. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:32,363][19170] Avg episode reward: [(0, '853.450'), (1, '990.250')] [2023-12-27 17:57:32,439][20381] Updated weights for policy 1, policy_version 1775782 (0.0006) [2023-12-27 17:57:32,499][20381] Updated weights for policy 1, policy_version 1775792 (0.0008) [2023-12-27 17:57:32,560][20381] Updated weights for policy 1, policy_version 1775802 (0.0010) [2023-12-27 17:57:32,965][20347] Updated weights for policy 0, policy_version 1777384 (0.0008) [2023-12-27 17:57:33,013][20347] Updated weights for policy 0, policy_version 1777394 (0.0007) [2023-12-27 17:57:33,059][20347] Updated weights for policy 0, policy_version 1777404 (0.0006) [2023-12-27 17:57:33,166][20381] Updated weights for policy 1, policy_version 1775812 (0.0010) [2023-12-27 17:57:33,220][20381] Updated weights for policy 1, policy_version 1775822 (0.0010) [2023-12-27 17:57:33,278][20381] Updated weights for policy 1, policy_version 1775832 (0.0010) [2023-12-27 17:57:33,655][20347] Updated weights for policy 0, policy_version 1777414 (0.0005) [2023-12-27 17:57:33,701][20347] Updated weights for policy 0, policy_version 1777424 (0.0005) [2023-12-27 17:57:33,746][20347] Updated weights for policy 0, policy_version 1777434 (0.0005) [2023-12-27 17:57:33,880][20381] Updated weights for policy 1, policy_version 1775842 (0.0009) [2023-12-27 17:57:33,927][20381] Updated weights for policy 1, policy_version 1775852 (0.0007) [2023-12-27 17:57:33,975][20381] Updated weights for policy 1, policy_version 1775862 (0.0010) [2023-12-27 17:57:34,026][20381] Updated weights for policy 1, policy_version 1775872 (0.0010) [2023-12-27 17:57:34,393][20347] Updated weights for policy 0, policy_version 1777444 (0.0007) [2023-12-27 17:57:34,454][20347] Updated weights for policy 0, policy_version 1777454 (0.0008) [2023-12-27 17:57:34,509][20347] Updated weights for policy 0, policy_version 1777464 (0.0008) [2023-12-27 17:57:34,746][20381] Updated weights for policy 1, policy_version 1775882 (0.0011) [2023-12-27 17:57:34,804][20381] Updated weights for policy 1, policy_version 1775892 (0.0011) [2023-12-27 17:57:34,857][20381] Updated weights for policy 1, policy_version 1775902 (0.0006) [2023-12-27 17:57:35,163][20347] Updated weights for policy 0, policy_version 1777474 (0.0008) [2023-12-27 17:57:35,224][20347] Updated weights for policy 0, policy_version 1777484 (0.0010) [2023-12-27 17:57:35,275][20347] Updated weights for policy 0, policy_version 1777494 (0.0009) [2023-12-27 17:57:35,281][20291] KL-divergence is very high: 116.7075 [2023-12-27 17:57:35,323][20291] KL-divergence is very high: 145.5055 [2023-12-27 17:57:35,329][20347] Updated weights for policy 0, policy_version 1777504 (0.0010) [2023-12-27 17:57:35,447][20381] Updated weights for policy 1, policy_version 1775912 (0.0010) [2023-12-27 17:57:35,503][20381] Updated weights for policy 1, policy_version 1775922 (0.0011) [2023-12-27 17:57:35,565][20381] Updated weights for policy 1, policy_version 1775932 (0.0011) [2023-12-27 17:57:35,903][20347] Updated weights for policy 0, policy_version 1777514 (0.0011) [2023-12-27 17:57:35,961][20347] Updated weights for policy 0, policy_version 1777524 (0.0010) [2023-12-27 17:57:36,017][20347] Updated weights for policy 0, policy_version 1777534 (0.0010) [2023-12-27 17:57:36,241][20381] Updated weights for policy 1, policy_version 1775942 (0.0007) [2023-12-27 17:57:36,303][20381] Updated weights for policy 1, policy_version 1775952 (0.0010) [2023-12-27 17:57:36,366][20381] Updated weights for policy 1, policy_version 1775962 (0.0009) [2023-12-27 17:57:36,690][20347] Updated weights for policy 0, policy_version 1777544 (0.0010) [2023-12-27 17:57:36,742][20347] Updated weights for policy 0, policy_version 1777554 (0.0010) [2023-12-27 17:57:36,804][20347] Updated weights for policy 0, policy_version 1777564 (0.0005) [2023-12-27 17:57:36,964][20381] Updated weights for policy 1, policy_version 1775972 (0.0007) [2023-12-27 17:57:37,029][20381] Updated weights for policy 1, policy_version 1775982 (0.0007) [2023-12-27 17:57:37,093][20381] Updated weights for policy 1, policy_version 1775992 (0.0008) [2023-12-27 17:57:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 909844480. Throughput: 0: 10296.6, 1: 10326.9. Samples: 909811864. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:37,363][19170] Avg episode reward: [(0, '921.730'), (1, '1000.000')] [2023-12-27 17:57:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001776000_454705152.pth... [2023-12-27 17:57:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001774784_454393856.pth [2023-12-27 17:57:37,415][20347] Updated weights for policy 0, policy_version 1777574 (0.0007) [2023-12-27 17:57:37,465][20347] Updated weights for policy 0, policy_version 1777584 (0.0007) [2023-12-27 17:57:37,519][20347] Updated weights for policy 0, policy_version 1777594 (0.0005) [2023-12-27 17:57:37,554][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001777600_455147520.pth... [2023-12-27 17:57:37,558][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001776384_454836224.pth [2023-12-27 17:57:37,737][20381] Updated weights for policy 1, policy_version 1776002 (0.0007) [2023-12-27 17:57:37,799][20381] Updated weights for policy 1, policy_version 1776012 (0.0005) [2023-12-27 17:57:37,849][20381] Updated weights for policy 1, policy_version 1776022 (0.0005) [2023-12-27 17:57:37,900][20381] Updated weights for policy 1, policy_version 1776032 (0.0006) [2023-12-27 17:57:38,163][20347] Updated weights for policy 0, policy_version 1777604 (0.0006) [2023-12-27 17:57:38,219][20347] Updated weights for policy 0, policy_version 1777614 (0.0005) [2023-12-27 17:57:38,274][20347] Updated weights for policy 0, policy_version 1777624 (0.0005) [2023-12-27 17:57:38,607][20381] Updated weights for policy 1, policy_version 1776042 (0.0007) [2023-12-27 17:57:38,664][20381] Updated weights for policy 1, policy_version 1776052 (0.0008) [2023-12-27 17:57:38,726][20381] Updated weights for policy 1, policy_version 1776062 (0.0008) [2023-12-27 17:57:38,880][20347] Updated weights for policy 0, policy_version 1777634 (0.0006) [2023-12-27 17:57:38,944][20347] Updated weights for policy 0, policy_version 1777644 (0.0005) [2023-12-27 17:57:38,998][20347] Updated weights for policy 0, policy_version 1777654 (0.0006) [2023-12-27 17:57:39,053][20347] Updated weights for policy 0, policy_version 1777664 (0.0008) [2023-12-27 17:57:39,473][20381] Updated weights for policy 1, policy_version 1776072 (0.0007) [2023-12-27 17:57:39,525][20381] Updated weights for policy 1, policy_version 1776082 (0.0007) [2023-12-27 17:57:39,573][20381] Updated weights for policy 1, policy_version 1776092 (0.0007) [2023-12-27 17:57:39,745][20347] Updated weights for policy 0, policy_version 1777674 (0.0011) [2023-12-27 17:57:39,804][20347] Updated weights for policy 0, policy_version 1777684 (0.0010) [2023-12-27 17:57:39,868][20347] Updated weights for policy 0, policy_version 1777695 (0.0009) [2023-12-27 17:57:40,308][20381] Updated weights for policy 1, policy_version 1776102 (0.0008) [2023-12-27 17:57:40,366][20381] Updated weights for policy 1, policy_version 1776112 (0.0008) [2023-12-27 17:57:40,434][20381] Updated weights for policy 1, policy_version 1776122 (0.0008) [2023-12-27 17:57:40,580][20347] Updated weights for policy 0, policy_version 1777705 (0.0008) [2023-12-27 17:57:40,636][20347] Updated weights for policy 0, policy_version 1777715 (0.0008) [2023-12-27 17:57:40,707][20347] Updated weights for policy 0, policy_version 1777725 (0.0006) [2023-12-27 17:57:41,109][20381] Updated weights for policy 1, policy_version 1776132 (0.0009) [2023-12-27 17:57:41,177][20381] Updated weights for policy 1, policy_version 1776142 (0.0011) [2023-12-27 17:57:41,236][20381] Updated weights for policy 1, policy_version 1776152 (0.0008) [2023-12-27 17:57:41,408][20347] Updated weights for policy 0, policy_version 1777735 (0.0008) [2023-12-27 17:57:41,462][20347] Updated weights for policy 0, policy_version 1777745 (0.0008) [2023-12-27 17:57:41,519][20347] Updated weights for policy 0, policy_version 1777755 (0.0008) [2023-12-27 17:57:41,899][20381] Updated weights for policy 1, policy_version 1776162 (0.0009) [2023-12-27 17:57:41,963][20381] Updated weights for policy 1, policy_version 1776172 (0.0008) [2023-12-27 17:57:42,017][20381] Updated weights for policy 1, policy_version 1776182 (0.0011) [2023-12-27 17:57:42,071][20381] Updated weights for policy 1, policy_version 1776192 (0.0011) [2023-12-27 17:57:42,255][20347] Updated weights for policy 0, policy_version 1777765 (0.0007) [2023-12-27 17:57:42,317][20347] Updated weights for policy 0, policy_version 1777775 (0.0007) [2023-12-27 17:57:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 909942784. Throughput: 0: 10316.0, 1: 10373.5. Samples: 909936920. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:42,363][19170] Avg episode reward: [(0, '902.450'), (1, '990.310')] [2023-12-27 17:57:42,383][20347] Updated weights for policy 0, policy_version 1777785 (0.0008) [2023-12-27 17:57:42,806][20381] Updated weights for policy 1, policy_version 1776202 (0.0008) [2023-12-27 17:57:42,868][20381] Updated weights for policy 1, policy_version 1776212 (0.0009) [2023-12-27 17:57:42,923][20381] Updated weights for policy 1, policy_version 1776222 (0.0008) [2023-12-27 17:57:43,010][20347] Updated weights for policy 0, policy_version 1777795 (0.0008) [2023-12-27 17:57:43,066][20347] Updated weights for policy 0, policy_version 1777805 (0.0005) [2023-12-27 17:57:43,120][20347] Updated weights for policy 0, policy_version 1777815 (0.0005) [2023-12-27 17:57:43,609][20381] Updated weights for policy 1, policy_version 1776232 (0.0009) [2023-12-27 17:57:43,671][20347] Updated weights for policy 0, policy_version 1777825 (0.0006) [2023-12-27 17:57:43,673][20381] Updated weights for policy 1, policy_version 1776242 (0.0007) [2023-12-27 17:57:43,716][20347] Updated weights for policy 0, policy_version 1777835 (0.0010) [2023-12-27 17:57:43,732][20381] Updated weights for policy 1, policy_version 1776252 (0.0007) [2023-12-27 17:57:43,767][20347] Updated weights for policy 0, policy_version 1777845 (0.0010) [2023-12-27 17:57:43,815][20347] Updated weights for policy 0, policy_version 1777855 (0.0010) [2023-12-27 17:57:44,365][20381] Updated weights for policy 1, policy_version 1776262 (0.0009) [2023-12-27 17:57:44,415][20381] Updated weights for policy 1, policy_version 1776272 (0.0011) [2023-12-27 17:57:44,476][20381] Updated weights for policy 1, policy_version 1776282 (0.0010) [2023-12-27 17:57:44,489][20347] Updated weights for policy 0, policy_version 1777865 (0.0007) [2023-12-27 17:57:44,552][20347] Updated weights for policy 0, policy_version 1777875 (0.0007) [2023-12-27 17:57:44,608][20347] Updated weights for policy 0, policy_version 1777885 (0.0007) [2023-12-27 17:57:45,187][20381] Updated weights for policy 1, policy_version 1776292 (0.0011) [2023-12-27 17:57:45,251][20381] Updated weights for policy 1, policy_version 1776302 (0.0011) [2023-12-27 17:57:45,318][20381] Updated weights for policy 1, policy_version 1776312 (0.0011) [2023-12-27 17:57:45,334][20347] Updated weights for policy 0, policy_version 1777895 (0.0009) [2023-12-27 17:57:45,392][20347] Updated weights for policy 0, policy_version 1777905 (0.0007) [2023-12-27 17:57:45,446][20347] Updated weights for policy 0, policy_version 1777915 (0.0006) [2023-12-27 17:57:45,992][20381] Updated weights for policy 1, policy_version 1776322 (0.0011) [2023-12-27 17:57:46,041][20381] Updated weights for policy 1, policy_version 1776332 (0.0011) [2023-12-27 17:57:46,042][20347] Updated weights for policy 0, policy_version 1777925 (0.0006) [2023-12-27 17:57:46,089][20347] Updated weights for policy 0, policy_version 1777935 (0.0005) [2023-12-27 17:57:46,090][20381] Updated weights for policy 1, policy_version 1776342 (0.0010) [2023-12-27 17:57:46,143][20347] Updated weights for policy 0, policy_version 1777945 (0.0005) [2023-12-27 17:57:46,149][20381] Updated weights for policy 1, policy_version 1776352 (0.0010) [2023-12-27 17:57:46,723][20347] Updated weights for policy 0, policy_version 1777955 (0.0006) [2023-12-27 17:57:46,777][20347] Updated weights for policy 0, policy_version 1777965 (0.0007) [2023-12-27 17:57:46,816][20381] Updated weights for policy 1, policy_version 1776362 (0.0011) [2023-12-27 17:57:46,829][20347] Updated weights for policy 0, policy_version 1777975 (0.0005) [2023-12-27 17:57:46,861][20381] Updated weights for policy 1, policy_version 1776372 (0.0010) [2023-12-27 17:57:46,913][20381] Updated weights for policy 1, policy_version 1776382 (0.0011) [2023-12-27 17:57:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 910049280. Throughput: 0: 10357.9, 1: 10349.1. Samples: 910061260. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:47,363][19170] Avg episode reward: [(0, '931.660'), (1, '990.310')] [2023-12-27 17:57:47,427][20347] Updated weights for policy 0, policy_version 1777985 (0.0006) [2023-12-27 17:57:47,484][20347] Updated weights for policy 0, policy_version 1777995 (0.0005) [2023-12-27 17:57:47,536][20347] Updated weights for policy 0, policy_version 1778005 (0.0005) [2023-12-27 17:57:47,585][20347] Updated weights for policy 0, policy_version 1778015 (0.0005) [2023-12-27 17:57:47,605][20381] Updated weights for policy 1, policy_version 1776392 (0.0008) [2023-12-27 17:57:47,651][20381] Updated weights for policy 1, policy_version 1776402 (0.0005) [2023-12-27 17:57:47,697][20381] Updated weights for policy 1, policy_version 1776412 (0.0005) [2023-12-27 17:57:48,157][20347] Updated weights for policy 0, policy_version 1778025 (0.0009) [2023-12-27 17:57:48,209][20347] Updated weights for policy 0, policy_version 1778035 (0.0008) [2023-12-27 17:57:48,260][20347] Updated weights for policy 0, policy_version 1778045 (0.0008) [2023-12-27 17:57:48,288][20381] Updated weights for policy 1, policy_version 1776422 (0.0007) [2023-12-27 17:57:48,351][20381] Updated weights for policy 1, policy_version 1776432 (0.0008) [2023-12-27 17:57:48,412][20381] Updated weights for policy 1, policy_version 1776442 (0.0006) [2023-12-27 17:57:48,989][20347] Updated weights for policy 0, policy_version 1778055 (0.0008) [2023-12-27 17:57:49,043][20347] Updated weights for policy 0, policy_version 1778065 (0.0007) [2023-12-27 17:57:49,093][20381] Updated weights for policy 1, policy_version 1776452 (0.0009) [2023-12-27 17:57:49,096][20347] Updated weights for policy 0, policy_version 1778075 (0.0007) [2023-12-27 17:57:49,139][20381] Updated weights for policy 1, policy_version 1776462 (0.0007) [2023-12-27 17:57:49,195][20381] Updated weights for policy 1, policy_version 1776472 (0.0008) [2023-12-27 17:57:49,838][20347] Updated weights for policy 0, policy_version 1778085 (0.0007) [2023-12-27 17:57:49,901][20347] Updated weights for policy 0, policy_version 1778095 (0.0010) [2023-12-27 17:57:49,909][20381] Updated weights for policy 1, policy_version 1776482 (0.0008) [2023-12-27 17:57:49,965][20347] Updated weights for policy 0, policy_version 1778105 (0.0011) [2023-12-27 17:57:49,976][20381] Updated weights for policy 1, policy_version 1776492 (0.0009) [2023-12-27 17:57:50,040][20381] Updated weights for policy 1, policy_version 1776502 (0.0006) [2023-12-27 17:57:50,102][20381] Updated weights for policy 1, policy_version 1776512 (0.0007) [2023-12-27 17:57:50,643][20347] Updated weights for policy 0, policy_version 1778115 (0.0010) [2023-12-27 17:57:50,714][20381] Updated weights for policy 1, policy_version 1776522 (0.0006) [2023-12-27 17:57:50,726][20347] Updated weights for policy 0, policy_version 1778125 (0.0011) [2023-12-27 17:57:50,770][20381] Updated weights for policy 1, policy_version 1776532 (0.0008) [2023-12-27 17:57:50,786][20347] Updated weights for policy 0, policy_version 1778135 (0.0011) [2023-12-27 17:57:50,825][20381] Updated weights for policy 1, policy_version 1776542 (0.0006) [2023-12-27 17:57:51,462][20347] Updated weights for policy 0, policy_version 1778145 (0.0010) [2023-12-27 17:57:51,516][20381] Updated weights for policy 1, policy_version 1776552 (0.0006) [2023-12-27 17:57:51,522][20347] Updated weights for policy 0, policy_version 1778155 (0.0006) [2023-12-27 17:57:51,574][20381] Updated weights for policy 1, policy_version 1776562 (0.0008) [2023-12-27 17:57:51,584][20347] Updated weights for policy 0, policy_version 1778165 (0.0006) [2023-12-27 17:57:51,641][20381] Updated weights for policy 1, policy_version 1776572 (0.0008) [2023-12-27 17:57:51,647][20347] Updated weights for policy 0, policy_version 1778175 (0.0008) [2023-12-27 17:57:52,355][20347] Updated weights for policy 0, policy_version 1778185 (0.0008) [2023-12-27 17:57:52,358][20381] Updated weights for policy 1, policy_version 1776582 (0.0007) [2023-12-27 17:57:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 910147584. Throughput: 0: 10352.3, 1: 10355.4. Samples: 910123924. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:52,363][19170] Avg episode reward: [(0, '912.200'), (1, '1000.000')] [2023-12-27 17:57:52,411][20347] Updated weights for policy 0, policy_version 1778195 (0.0006) [2023-12-27 17:57:52,413][20381] Updated weights for policy 1, policy_version 1776592 (0.0008) [2023-12-27 17:57:52,463][20381] Updated weights for policy 1, policy_version 1776602 (0.0007) [2023-12-27 17:57:52,471][20347] Updated weights for policy 0, policy_version 1778205 (0.0007) [2023-12-27 17:57:52,486][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001778208_455303168.pth... [2023-12-27 17:57:52,489][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001776960_454983680.pth [2023-12-27 17:57:52,496][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001776608_454860800.pth... [2023-12-27 17:57:52,519][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001775360_454541312.pth [2023-12-27 17:57:53,091][20347] Updated weights for policy 0, policy_version 1778215 (0.0008) [2023-12-27 17:57:53,118][20381] Updated weights for policy 1, policy_version 1776612 (0.0008) [2023-12-27 17:57:53,146][20347] Updated weights for policy 0, policy_version 1778225 (0.0006) [2023-12-27 17:57:53,169][20381] Updated weights for policy 1, policy_version 1776622 (0.0005) [2023-12-27 17:57:53,202][20347] Updated weights for policy 0, policy_version 1778235 (0.0006) [2023-12-27 17:57:53,224][20381] Updated weights for policy 1, policy_version 1776632 (0.0006) [2023-12-27 17:57:53,782][20381] Updated weights for policy 1, policy_version 1776642 (0.0005) [2023-12-27 17:57:53,841][20381] Updated weights for policy 1, policy_version 1776652 (0.0007) [2023-12-27 17:57:53,901][20381] Updated weights for policy 1, policy_version 1776662 (0.0008) [2023-12-27 17:57:53,951][20381] Updated weights for policy 1, policy_version 1776672 (0.0006) [2023-12-27 17:57:53,951][20347] Updated weights for policy 0, policy_version 1778245 (0.0007) [2023-12-27 17:57:54,002][20347] Updated weights for policy 0, policy_version 1778255 (0.0010) [2023-12-27 17:57:54,055][20347] Updated weights for policy 0, policy_version 1778265 (0.0008) [2023-12-27 17:57:54,574][20381] Updated weights for policy 1, policy_version 1776682 (0.0008) [2023-12-27 17:57:54,625][20381] Updated weights for policy 1, policy_version 1776692 (0.0007) [2023-12-27 17:57:54,675][20381] Updated weights for policy 1, policy_version 1776702 (0.0006) [2023-12-27 17:57:54,773][20347] Updated weights for policy 0, policy_version 1778275 (0.0007) [2023-12-27 17:57:54,828][20347] Updated weights for policy 0, policy_version 1778285 (0.0007) [2023-12-27 17:57:54,891][20347] Updated weights for policy 0, policy_version 1778295 (0.0008) [2023-12-27 17:57:55,292][20381] Updated weights for policy 1, policy_version 1776712 (0.0007) [2023-12-27 17:57:55,359][20381] Updated weights for policy 1, policy_version 1776722 (0.0007) [2023-12-27 17:57:55,415][20381] Updated weights for policy 1, policy_version 1776732 (0.0008) [2023-12-27 17:57:55,582][20347] Updated weights for policy 0, policy_version 1778305 (0.0008) [2023-12-27 17:57:55,648][20347] Updated weights for policy 0, policy_version 1778315 (0.0007) [2023-12-27 17:57:55,714][20347] Updated weights for policy 0, policy_version 1778325 (0.0008) [2023-12-27 17:57:55,779][20347] Updated weights for policy 0, policy_version 1778335 (0.0008) [2023-12-27 17:57:56,080][20381] Updated weights for policy 1, policy_version 1776742 (0.0007) [2023-12-27 17:57:56,131][20381] Updated weights for policy 1, policy_version 1776752 (0.0008) [2023-12-27 17:57:56,190][20381] Updated weights for policy 1, policy_version 1776762 (0.0008) [2023-12-27 17:57:56,452][20347] Updated weights for policy 0, policy_version 1778345 (0.0006) [2023-12-27 17:57:56,506][20347] Updated weights for policy 0, policy_version 1778355 (0.0006) [2023-12-27 17:57:56,555][20347] Updated weights for policy 0, policy_version 1778365 (0.0008) [2023-12-27 17:57:56,839][20381] Updated weights for policy 1, policy_version 1776772 (0.0007) [2023-12-27 17:57:56,889][20381] Updated weights for policy 1, policy_version 1776782 (0.0009) [2023-12-27 17:57:56,938][20381] Updated weights for policy 1, policy_version 1776792 (0.0009) [2023-12-27 17:57:57,138][20347] Updated weights for policy 0, policy_version 1778375 (0.0007) [2023-12-27 17:57:57,189][20347] Updated weights for policy 0, policy_version 1778385 (0.0005) [2023-12-27 17:57:57,238][20347] Updated weights for policy 0, policy_version 1778395 (0.0007) [2023-12-27 17:57:57,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20521.6). Total num frames: 910262272. Throughput: 0: 10299.7, 1: 10442.6. Samples: 910247780. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:57:57,364][19170] Avg episode reward: [(0, '921.840'), (1, '990.280')] [2023-12-27 17:57:57,636][20381] Updated weights for policy 1, policy_version 1776802 (0.0008) [2023-12-27 17:57:57,681][20381] Updated weights for policy 1, policy_version 1776812 (0.0007) [2023-12-27 17:57:57,739][20381] Updated weights for policy 1, policy_version 1776822 (0.0005) [2023-12-27 17:57:57,796][20381] Updated weights for policy 1, policy_version 1776832 (0.0005) [2023-12-27 17:57:57,859][20347] Updated weights for policy 0, policy_version 1778405 (0.0007) [2023-12-27 17:57:57,907][20347] Updated weights for policy 0, policy_version 1778415 (0.0007) [2023-12-27 17:57:57,965][20347] Updated weights for policy 0, policy_version 1778425 (0.0007) [2023-12-27 17:57:58,429][20381] Updated weights for policy 1, policy_version 1776842 (0.0009) [2023-12-27 17:57:58,483][20381] Updated weights for policy 1, policy_version 1776852 (0.0009) [2023-12-27 17:57:58,545][20381] Updated weights for policy 1, policy_version 1776862 (0.0009) [2023-12-27 17:57:58,701][20347] Updated weights for policy 0, policy_version 1778435 (0.0008) [2023-12-27 17:57:58,762][20347] Updated weights for policy 0, policy_version 1778445 (0.0009) [2023-12-27 17:57:58,827][20347] Updated weights for policy 0, policy_version 1778455 (0.0008) [2023-12-27 17:57:59,389][20381] Updated weights for policy 1, policy_version 1776872 (0.0010) [2023-12-27 17:57:59,444][20381] Updated weights for policy 1, policy_version 1776882 (0.0010) [2023-12-27 17:57:59,496][20381] Updated weights for policy 1, policy_version 1776892 (0.0010) [2023-12-27 17:57:59,565][20347] Updated weights for policy 0, policy_version 1778465 (0.0008) [2023-12-27 17:57:59,617][20347] Updated weights for policy 0, policy_version 1778475 (0.0007) [2023-12-27 17:57:59,673][20347] Updated weights for policy 0, policy_version 1778485 (0.0007) [2023-12-27 17:57:59,724][20347] Updated weights for policy 0, policy_version 1778495 (0.0007) [2023-12-27 17:58:00,203][20381] Updated weights for policy 1, policy_version 1776902 (0.0009) [2023-12-27 17:58:00,256][20381] Updated weights for policy 1, policy_version 1776912 (0.0008) [2023-12-27 17:58:00,303][20381] Updated weights for policy 1, policy_version 1776922 (0.0006) [2023-12-27 17:58:00,480][20347] Updated weights for policy 0, policy_version 1778505 (0.0006) [2023-12-27 17:58:00,530][20347] Updated weights for policy 0, policy_version 1778515 (0.0005) [2023-12-27 17:58:00,586][20347] Updated weights for policy 0, policy_version 1778525 (0.0007) [2023-12-27 17:58:00,975][20381] Updated weights for policy 1, policy_version 1776932 (0.0007) [2023-12-27 17:58:01,025][20381] Updated weights for policy 1, policy_version 1776942 (0.0010) [2023-12-27 17:58:01,086][20381] Updated weights for policy 1, policy_version 1776952 (0.0007) [2023-12-27 17:58:01,264][20347] Updated weights for policy 0, policy_version 1778535 (0.0008) [2023-12-27 17:58:01,311][20347] Updated weights for policy 0, policy_version 1778545 (0.0007) [2023-12-27 17:58:01,385][20347] Updated weights for policy 0, policy_version 1778555 (0.0009) [2023-12-27 17:58:01,829][20381] Updated weights for policy 1, policy_version 1776962 (0.0008) [2023-12-27 17:58:01,892][20381] Updated weights for policy 1, policy_version 1776972 (0.0007) [2023-12-27 17:58:01,949][20381] Updated weights for policy 1, policy_version 1776982 (0.0008) [2023-12-27 17:58:01,999][20381] Updated weights for policy 1, policy_version 1776992 (0.0007) [2023-12-27 17:58:02,059][20347] Updated weights for policy 0, policy_version 1778565 (0.0007) [2023-12-27 17:58:02,106][20347] Updated weights for policy 0, policy_version 1778575 (0.0007) [2023-12-27 17:58:02,171][20347] Updated weights for policy 0, policy_version 1778585 (0.0007) [2023-12-27 17:58:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 910360576. Throughput: 0: 10317.9, 1: 10374.5. Samples: 910369204. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:02,363][19170] Avg episode reward: [(0, '951.100'), (1, '990.280')] [2023-12-27 17:58:02,691][20381] Updated weights for policy 1, policy_version 1777002 (0.0008) [2023-12-27 17:58:02,744][20381] Updated weights for policy 1, policy_version 1777012 (0.0005) [2023-12-27 17:58:02,796][20381] Updated weights for policy 1, policy_version 1777022 (0.0005) [2023-12-27 17:58:02,881][20347] Updated weights for policy 0, policy_version 1778595 (0.0007) [2023-12-27 17:58:02,941][20347] Updated weights for policy 0, policy_version 1778605 (0.0005) [2023-12-27 17:58:02,999][20347] Updated weights for policy 0, policy_version 1778615 (0.0005) [2023-12-27 17:58:03,496][20381] Updated weights for policy 1, policy_version 1777032 (0.0005) [2023-12-27 17:58:03,551][20381] Updated weights for policy 1, policy_version 1777042 (0.0005) [2023-12-27 17:58:03,562][20347] Updated weights for policy 0, policy_version 1778625 (0.0005) [2023-12-27 17:58:03,610][20381] Updated weights for policy 1, policy_version 1777052 (0.0006) [2023-12-27 17:58:03,616][20347] Updated weights for policy 0, policy_version 1778635 (0.0007) [2023-12-27 17:58:03,672][20347] Updated weights for policy 0, policy_version 1778645 (0.0007) [2023-12-27 17:58:03,724][20347] Updated weights for policy 0, policy_version 1778655 (0.0008) [2023-12-27 17:58:04,235][20381] Updated weights for policy 1, policy_version 1777062 (0.0008) [2023-12-27 17:58:04,298][20381] Updated weights for policy 1, policy_version 1777072 (0.0009) [2023-12-27 17:58:04,358][20381] Updated weights for policy 1, policy_version 1777082 (0.0008) [2023-12-27 17:58:04,408][20347] Updated weights for policy 0, policy_version 1778665 (0.0007) [2023-12-27 17:58:04,463][20347] Updated weights for policy 0, policy_version 1778675 (0.0007) [2023-12-27 17:58:04,515][20347] Updated weights for policy 0, policy_version 1778685 (0.0007) [2023-12-27 17:58:05,059][20381] Updated weights for policy 1, policy_version 1777092 (0.0005) [2023-12-27 17:58:05,106][20381] Updated weights for policy 1, policy_version 1777102 (0.0005) [2023-12-27 17:58:05,165][20381] Updated weights for policy 1, policy_version 1777112 (0.0005) [2023-12-27 17:58:05,166][20347] Updated weights for policy 0, policy_version 1778695 (0.0009) [2023-12-27 17:58:05,233][20347] Updated weights for policy 0, policy_version 1778705 (0.0005) [2023-12-27 17:58:05,300][20347] Updated weights for policy 0, policy_version 1778715 (0.0005) [2023-12-27 17:58:05,847][20347] Updated weights for policy 0, policy_version 1778725 (0.0005) [2023-12-27 17:58:05,894][20381] Updated weights for policy 1, policy_version 1777122 (0.0008) [2023-12-27 17:58:05,905][20347] Updated weights for policy 0, policy_version 1778735 (0.0005) [2023-12-27 17:58:05,958][20347] Updated weights for policy 0, policy_version 1778745 (0.0005) [2023-12-27 17:58:05,960][20381] Updated weights for policy 1, policy_version 1777132 (0.0007) [2023-12-27 17:58:06,013][20381] Updated weights for policy 1, policy_version 1777142 (0.0006) [2023-12-27 17:58:06,069][20381] Updated weights for policy 1, policy_version 1777152 (0.0007) [2023-12-27 17:58:06,656][20347] Updated weights for policy 0, policy_version 1778755 (0.0008) [2023-12-27 17:58:06,714][20347] Updated weights for policy 0, policy_version 1778765 (0.0008) [2023-12-27 17:58:06,776][20381] Updated weights for policy 1, policy_version 1777162 (0.0006) [2023-12-27 17:58:06,785][20347] Updated weights for policy 0, policy_version 1778775 (0.0007) [2023-12-27 17:58:06,825][20381] Updated weights for policy 1, policy_version 1777172 (0.0007) [2023-12-27 17:58:06,880][20381] Updated weights for policy 1, policy_version 1777182 (0.0006) [2023-12-27 17:58:07,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 910458880. Throughput: 0: 10368.1, 1: 10363.0. Samples: 910432232. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:07,363][19170] Avg episode reward: [(0, '951.170'), (1, '970.780')] [2023-12-27 17:58:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001777184_455008256.pth... [2023-12-27 17:58:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001778784_455450624.pth... [2023-12-27 17:58:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001776000_454705152.pth [2023-12-27 17:58:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001777600_455147520.pth [2023-12-27 17:58:07,392][20347] Updated weights for policy 0, policy_version 1778785 (0.0006) [2023-12-27 17:58:07,451][20347] Updated weights for policy 0, policy_version 1778795 (0.0005) [2023-12-27 17:58:07,506][20347] Updated weights for policy 0, policy_version 1778805 (0.0005) [2023-12-27 17:58:07,552][20347] Updated weights for policy 0, policy_version 1778815 (0.0005) [2023-12-27 17:58:07,637][20381] Updated weights for policy 1, policy_version 1777192 (0.0008) [2023-12-27 17:58:07,699][20381] Updated weights for policy 1, policy_version 1777202 (0.0008) [2023-12-27 17:58:07,746][20381] Updated weights for policy 1, policy_version 1777212 (0.0007) [2023-12-27 17:58:08,143][20347] Updated weights for policy 0, policy_version 1778825 (0.0005) [2023-12-27 17:58:08,196][20347] Updated weights for policy 0, policy_version 1778835 (0.0005) [2023-12-27 17:58:08,250][20347] Updated weights for policy 0, policy_version 1778845 (0.0008) [2023-12-27 17:58:08,466][20381] Updated weights for policy 1, policy_version 1777222 (0.0008) [2023-12-27 17:58:08,523][20381] Updated weights for policy 1, policy_version 1777232 (0.0009) [2023-12-27 17:58:08,580][20381] Updated weights for policy 1, policy_version 1777242 (0.0007) [2023-12-27 17:58:08,883][20347] Updated weights for policy 0, policy_version 1778855 (0.0009) [2023-12-27 17:58:08,938][20347] Updated weights for policy 0, policy_version 1778865 (0.0010) [2023-12-27 17:58:08,996][20347] Updated weights for policy 0, policy_version 1778875 (0.0010) [2023-12-27 17:58:09,307][20381] Updated weights for policy 1, policy_version 1777252 (0.0007) [2023-12-27 17:58:09,374][20381] Updated weights for policy 1, policy_version 1777262 (0.0008) [2023-12-27 17:58:09,438][20381] Updated weights for policy 1, policy_version 1777272 (0.0009) [2023-12-27 17:58:09,692][20347] Updated weights for policy 0, policy_version 1778885 (0.0007) [2023-12-27 17:58:09,753][20347] Updated weights for policy 0, policy_version 1778895 (0.0008) [2023-12-27 17:58:09,816][20347] Updated weights for policy 0, policy_version 1778905 (0.0008) [2023-12-27 17:58:10,113][20381] Updated weights for policy 1, policy_version 1777282 (0.0009) [2023-12-27 17:58:10,173][20381] Updated weights for policy 1, policy_version 1777292 (0.0011) [2023-12-27 17:58:10,227][20381] Updated weights for policy 1, policy_version 1777302 (0.0011) [2023-12-27 17:58:10,286][20381] Updated weights for policy 1, policy_version 1777312 (0.0011) [2023-12-27 17:58:10,583][20347] Updated weights for policy 0, policy_version 1778915 (0.0008) [2023-12-27 17:58:10,634][20347] Updated weights for policy 0, policy_version 1778925 (0.0008) [2023-12-27 17:58:10,685][20347] Updated weights for policy 0, policy_version 1778935 (0.0007) [2023-12-27 17:58:10,984][20381] Updated weights for policy 1, policy_version 1777322 (0.0006) [2023-12-27 17:58:11,049][20381] Updated weights for policy 1, policy_version 1777332 (0.0007) [2023-12-27 17:58:11,107][20381] Updated weights for policy 1, policy_version 1777342 (0.0007) [2023-12-27 17:58:11,475][20347] Updated weights for policy 0, policy_version 1778945 (0.0008) [2023-12-27 17:58:11,536][20347] Updated weights for policy 0, policy_version 1778955 (0.0010) [2023-12-27 17:58:11,602][20347] Updated weights for policy 0, policy_version 1778965 (0.0010) [2023-12-27 17:58:11,665][20347] Updated weights for policy 0, policy_version 1778975 (0.0010) [2023-12-27 17:58:11,844][20381] Updated weights for policy 1, policy_version 1777352 (0.0007) [2023-12-27 17:58:11,901][20381] Updated weights for policy 1, policy_version 1777362 (0.0008) [2023-12-27 17:58:11,954][20381] Updated weights for policy 1, policy_version 1777372 (0.0008) [2023-12-27 17:58:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 910557184. Throughput: 0: 10405.7, 1: 10278.9. Samples: 910552600. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:12,363][19170] Avg episode reward: [(0, '980.420'), (1, '970.780')] [2023-12-27 17:58:12,396][20347] Updated weights for policy 0, policy_version 1778985 (0.0010) [2023-12-27 17:58:12,454][20347] Updated weights for policy 0, policy_version 1778995 (0.0010) [2023-12-27 17:58:12,513][20347] Updated weights for policy 0, policy_version 1779005 (0.0011) [2023-12-27 17:58:12,655][20381] Updated weights for policy 1, policy_version 1777382 (0.0007) [2023-12-27 17:58:12,717][20381] Updated weights for policy 1, policy_version 1777392 (0.0005) [2023-12-27 17:58:12,775][20381] Updated weights for policy 1, policy_version 1777402 (0.0006) [2023-12-27 17:58:13,151][20347] Updated weights for policy 0, policy_version 1779015 (0.0008) [2023-12-27 17:58:13,200][20347] Updated weights for policy 0, policy_version 1779025 (0.0005) [2023-12-27 17:58:13,253][20347] Updated weights for policy 0, policy_version 1779035 (0.0005) [2023-12-27 17:58:13,390][20381] Updated weights for policy 1, policy_version 1777412 (0.0008) [2023-12-27 17:58:13,439][20381] Updated weights for policy 1, policy_version 1777422 (0.0010) [2023-12-27 17:58:13,491][20381] Updated weights for policy 1, policy_version 1777432 (0.0010) [2023-12-27 17:58:13,873][20347] Updated weights for policy 0, policy_version 1779045 (0.0006) [2023-12-27 17:58:13,922][20347] Updated weights for policy 0, policy_version 1779055 (0.0007) [2023-12-27 17:58:13,970][20347] Updated weights for policy 0, policy_version 1779065 (0.0008) [2023-12-27 17:58:14,098][20381] Updated weights for policy 1, policy_version 1777442 (0.0010) [2023-12-27 17:58:14,154][20381] Updated weights for policy 1, policy_version 1777452 (0.0005) [2023-12-27 17:58:14,212][20381] Updated weights for policy 1, policy_version 1777462 (0.0006) [2023-12-27 17:58:14,270][20381] Updated weights for policy 1, policy_version 1777472 (0.0011) [2023-12-27 17:58:14,620][20347] Updated weights for policy 0, policy_version 1779075 (0.0007) [2023-12-27 17:58:14,675][20347] Updated weights for policy 0, policy_version 1779085 (0.0007) [2023-12-27 17:58:14,721][20347] Updated weights for policy 0, policy_version 1779095 (0.0007) [2023-12-27 17:58:14,891][20381] Updated weights for policy 1, policy_version 1777482 (0.0007) [2023-12-27 17:58:14,954][20381] Updated weights for policy 1, policy_version 1777492 (0.0006) [2023-12-27 17:58:15,019][20381] Updated weights for policy 1, policy_version 1777502 (0.0009) [2023-12-27 17:58:15,393][20347] Updated weights for policy 0, policy_version 1779105 (0.0007) [2023-12-27 17:58:15,444][20347] Updated weights for policy 0, policy_version 1779115 (0.0007) [2023-12-27 17:58:15,500][20347] Updated weights for policy 0, policy_version 1779125 (0.0007) [2023-12-27 17:58:15,556][20347] Updated weights for policy 0, policy_version 1779135 (0.0007) [2023-12-27 17:58:15,659][20381] Updated weights for policy 1, policy_version 1777512 (0.0010) [2023-12-27 17:58:15,713][20381] Updated weights for policy 1, policy_version 1777522 (0.0009) [2023-12-27 17:58:15,766][20381] Updated weights for policy 1, policy_version 1777532 (0.0008) [2023-12-27 17:58:16,256][20347] Updated weights for policy 0, policy_version 1779145 (0.0008) [2023-12-27 17:58:16,302][20347] Updated weights for policy 0, policy_version 1779155 (0.0008) [2023-12-27 17:58:16,351][20347] Updated weights for policy 0, policy_version 1779165 (0.0006) [2023-12-27 17:58:16,476][20381] Updated weights for policy 1, policy_version 1777542 (0.0008) [2023-12-27 17:58:16,534][20381] Updated weights for policy 1, policy_version 1777552 (0.0008) [2023-12-27 17:58:16,592][20381] Updated weights for policy 1, policy_version 1777562 (0.0009) [2023-12-27 17:58:16,971][20347] Updated weights for policy 0, policy_version 1779175 (0.0008) [2023-12-27 17:58:17,033][20347] Updated weights for policy 0, policy_version 1779185 (0.0006) [2023-12-27 17:58:17,093][20347] Updated weights for policy 0, policy_version 1779195 (0.0009) [2023-12-27 17:58:17,312][20381] Updated weights for policy 1, policy_version 1777572 (0.0007) [2023-12-27 17:58:17,360][20381] Updated weights for policy 1, policy_version 1777582 (0.0007) [2023-12-27 17:58:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 910663680. Throughput: 0: 10445.2, 1: 10219.6. Samples: 910677240. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:17,364][19170] Avg episode reward: [(0, '941.270'), (1, '1000.000')] [2023-12-27 17:58:17,405][20381] Updated weights for policy 1, policy_version 1777592 (0.0007) [2023-12-27 17:58:17,733][20347] Updated weights for policy 0, policy_version 1779205 (0.0010) [2023-12-27 17:58:17,792][20347] Updated weights for policy 0, policy_version 1779215 (0.0010) [2023-12-27 17:58:17,853][20347] Updated weights for policy 0, policy_version 1779225 (0.0010) [2023-12-27 17:58:18,028][20381] Updated weights for policy 1, policy_version 1777602 (0.0007) [2023-12-27 17:58:18,086][20381] Updated weights for policy 1, policy_version 1777612 (0.0005) [2023-12-27 17:58:18,148][20381] Updated weights for policy 1, policy_version 1777622 (0.0005) [2023-12-27 17:58:18,210][20381] Updated weights for policy 1, policy_version 1777632 (0.0006) [2023-12-27 17:58:18,551][20347] Updated weights for policy 0, policy_version 1779236 (0.0009) [2023-12-27 17:58:18,601][20347] Updated weights for policy 0, policy_version 1779246 (0.0007) [2023-12-27 17:58:18,651][20347] Updated weights for policy 0, policy_version 1779256 (0.0008) [2023-12-27 17:58:18,816][20381] Updated weights for policy 1, policy_version 1777642 (0.0008) [2023-12-27 17:58:18,879][20381] Updated weights for policy 1, policy_version 1777652 (0.0008) [2023-12-27 17:58:18,938][20381] Updated weights for policy 1, policy_version 1777662 (0.0010) [2023-12-27 17:58:19,387][20347] Updated weights for policy 0, policy_version 1779266 (0.0008) [2023-12-27 17:58:19,441][20347] Updated weights for policy 0, policy_version 1779276 (0.0007) [2023-12-27 17:58:19,504][20347] Updated weights for policy 0, policy_version 1779286 (0.0007) [2023-12-27 17:58:19,565][20347] Updated weights for policy 0, policy_version 1779296 (0.0007) [2023-12-27 17:58:19,665][20381] Updated weights for policy 1, policy_version 1777672 (0.0009) [2023-12-27 17:58:19,727][20381] Updated weights for policy 1, policy_version 1777682 (0.0008) [2023-12-27 17:58:19,791][20381] Updated weights for policy 1, policy_version 1777692 (0.0008) [2023-12-27 17:58:20,249][20347] Updated weights for policy 0, policy_version 1779306 (0.0007) [2023-12-27 17:58:20,301][20347] Updated weights for policy 0, policy_version 1779316 (0.0008) [2023-12-27 17:58:20,356][20347] Updated weights for policy 0, policy_version 1779326 (0.0007) [2023-12-27 17:58:20,548][20381] Updated weights for policy 1, policy_version 1777702 (0.0010) [2023-12-27 17:58:20,635][20381] Updated weights for policy 1, policy_version 1777712 (0.0011) [2023-12-27 17:58:20,695][20381] Updated weights for policy 1, policy_version 1777722 (0.0011) [2023-12-27 17:58:21,084][20347] Updated weights for policy 0, policy_version 1779336 (0.0007) [2023-12-27 17:58:21,150][20347] Updated weights for policy 0, policy_version 1779346 (0.0008) [2023-12-27 17:58:21,208][20347] Updated weights for policy 0, policy_version 1779356 (0.0007) [2023-12-27 17:58:21,422][20381] Updated weights for policy 1, policy_version 1777732 (0.0010) [2023-12-27 17:58:21,482][20381] Updated weights for policy 1, policy_version 1777742 (0.0011) [2023-12-27 17:58:21,542][20381] Updated weights for policy 1, policy_version 1777752 (0.0011) [2023-12-27 17:58:21,925][20347] Updated weights for policy 0, policy_version 1779366 (0.0007) [2023-12-27 17:58:21,987][20347] Updated weights for policy 0, policy_version 1779376 (0.0009) [2023-12-27 17:58:22,042][20347] Updated weights for policy 0, policy_version 1779386 (0.0006) [2023-12-27 17:58:22,258][20381] Updated weights for policy 1, policy_version 1777762 (0.0010) [2023-12-27 17:58:22,317][20381] Updated weights for policy 1, policy_version 1777772 (0.0008) [2023-12-27 17:58:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 910761984. Throughput: 0: 10389.5, 1: 10185.2. Samples: 910737724. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:22,364][19170] Avg episode reward: [(0, '921.750'), (1, '1000.000')] [2023-12-27 17:58:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001779392_455606272.pth... [2023-12-27 17:58:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001778208_455303168.pth [2023-12-27 17:58:22,383][20381] Updated weights for policy 1, policy_version 1777782 (0.0008) [2023-12-27 17:58:22,440][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001777792_455163904.pth... [2023-12-27 17:58:22,442][20381] Updated weights for policy 1, policy_version 1777792 (0.0008) [2023-12-27 17:58:22,443][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001776608_454860800.pth [2023-12-27 17:58:22,725][20347] Updated weights for policy 0, policy_version 1779396 (0.0007) [2023-12-27 17:58:22,769][20347] Updated weights for policy 0, policy_version 1779406 (0.0009) [2023-12-27 17:58:22,817][20347] Updated weights for policy 0, policy_version 1779416 (0.0010) [2023-12-27 17:58:23,089][20381] Updated weights for policy 1, policy_version 1777802 (0.0007) [2023-12-27 17:58:23,153][20381] Updated weights for policy 1, policy_version 1777812 (0.0005) [2023-12-27 17:58:23,214][20381] Updated weights for policy 1, policy_version 1777822 (0.0005) [2023-12-27 17:58:23,501][20347] Updated weights for policy 0, policy_version 1779426 (0.0009) [2023-12-27 17:58:23,558][20347] Updated weights for policy 0, policy_version 1779436 (0.0007) [2023-12-27 17:58:23,608][20347] Updated weights for policy 0, policy_version 1779446 (0.0006) [2023-12-27 17:58:23,669][20347] Updated weights for policy 0, policy_version 1779456 (0.0006) [2023-12-27 17:58:23,787][20381] Updated weights for policy 1, policy_version 1777832 (0.0005) [2023-12-27 17:58:23,836][20381] Updated weights for policy 1, policy_version 1777842 (0.0005) [2023-12-27 17:58:23,894][20381] Updated weights for policy 1, policy_version 1777852 (0.0005) [2023-12-27 17:58:24,294][20347] Updated weights for policy 0, policy_version 1779466 (0.0010) [2023-12-27 17:58:24,347][20347] Updated weights for policy 0, policy_version 1779476 (0.0009) [2023-12-27 17:58:24,399][20347] Updated weights for policy 0, policy_version 1779486 (0.0005) [2023-12-27 17:58:24,428][20381] Updated weights for policy 1, policy_version 1777862 (0.0006) [2023-12-27 17:58:24,477][20381] Updated weights for policy 1, policy_version 1777872 (0.0008) [2023-12-27 17:58:24,531][20381] Updated weights for policy 1, policy_version 1777882 (0.0006) [2023-12-27 17:58:25,005][20347] Updated weights for policy 0, policy_version 1779496 (0.0005) [2023-12-27 17:58:25,064][20347] Updated weights for policy 0, policy_version 1779506 (0.0008) [2023-12-27 17:58:25,125][20347] Updated weights for policy 0, policy_version 1779516 (0.0010) [2023-12-27 17:58:25,184][20381] Updated weights for policy 1, policy_version 1777892 (0.0006) [2023-12-27 17:58:25,247][20381] Updated weights for policy 1, policy_version 1777902 (0.0005) [2023-12-27 17:58:25,309][20381] Updated weights for policy 1, policy_version 1777912 (0.0005) [2023-12-27 17:58:25,672][20347] Updated weights for policy 0, policy_version 1779526 (0.0007) [2023-12-27 17:58:25,733][20347] Updated weights for policy 0, policy_version 1779536 (0.0008) [2023-12-27 17:58:25,791][20347] Updated weights for policy 0, policy_version 1779546 (0.0010) [2023-12-27 17:58:25,877][20381] Updated weights for policy 1, policy_version 1777922 (0.0005) [2023-12-27 17:58:25,936][20381] Updated weights for policy 1, policy_version 1777932 (0.0007) [2023-12-27 17:58:25,990][20381] Updated weights for policy 1, policy_version 1777942 (0.0007) [2023-12-27 17:58:26,038][20381] Updated weights for policy 1, policy_version 1777952 (0.0007) [2023-12-27 17:58:26,502][20347] Updated weights for policy 0, policy_version 1779556 (0.0008) [2023-12-27 17:58:26,561][20347] Updated weights for policy 0, policy_version 1779566 (0.0005) [2023-12-27 17:58:26,609][20347] Updated weights for policy 0, policy_version 1779576 (0.0005) [2023-12-27 17:58:26,714][20381] Updated weights for policy 1, policy_version 1777962 (0.0007) [2023-12-27 17:58:26,771][20381] Updated weights for policy 1, policy_version 1777972 (0.0006) [2023-12-27 17:58:26,827][20381] Updated weights for policy 1, policy_version 1777982 (0.0007) [2023-12-27 17:58:27,182][20347] Updated weights for policy 0, policy_version 1779586 (0.0006) [2023-12-27 17:58:27,235][20347] Updated weights for policy 0, policy_version 1779596 (0.0010) [2023-12-27 17:58:27,292][20347] Updated weights for policy 0, policy_version 1779606 (0.0010) [2023-12-27 17:58:27,343][20347] Updated weights for policy 0, policy_version 1779616 (0.0010) [2023-12-27 17:58:27,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 910876672. Throughput: 0: 10363.1, 1: 10236.5. Samples: 910863904. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:27,363][19170] Avg episode reward: [(0, '960.940'), (1, '990.280')] [2023-12-27 17:58:27,416][20381] Updated weights for policy 1, policy_version 1777992 (0.0006) [2023-12-27 17:58:27,469][20381] Updated weights for policy 1, policy_version 1778002 (0.0008) [2023-12-27 17:58:27,519][20381] Updated weights for policy 1, policy_version 1778012 (0.0007) [2023-12-27 17:58:28,003][20347] Updated weights for policy 0, policy_version 1779626 (0.0008) [2023-12-27 17:58:28,058][20347] Updated weights for policy 0, policy_version 1779636 (0.0007) [2023-12-27 17:58:28,116][20347] Updated weights for policy 0, policy_version 1779646 (0.0007) [2023-12-27 17:58:28,183][20381] Updated weights for policy 1, policy_version 1778022 (0.0008) [2023-12-27 17:58:28,239][20381] Updated weights for policy 1, policy_version 1778032 (0.0006) [2023-12-27 17:58:28,292][20381] Updated weights for policy 1, policy_version 1778042 (0.0007) [2023-12-27 17:58:28,801][20347] Updated weights for policy 0, policy_version 1779656 (0.0007) [2023-12-27 17:58:28,866][20347] Updated weights for policy 0, policy_version 1779666 (0.0007) [2023-12-27 17:58:28,925][20347] Updated weights for policy 0, policy_version 1779676 (0.0010) [2023-12-27 17:58:28,930][20381] Updated weights for policy 1, policy_version 1778052 (0.0007) [2023-12-27 17:58:28,991][20381] Updated weights for policy 1, policy_version 1778062 (0.0006) [2023-12-27 17:58:29,045][20381] Updated weights for policy 1, policy_version 1778072 (0.0007) [2023-12-27 17:58:29,635][20347] Updated weights for policy 0, policy_version 1779686 (0.0008) [2023-12-27 17:58:29,687][20347] Updated weights for policy 0, policy_version 1779696 (0.0008) [2023-12-27 17:58:29,738][20347] Updated weights for policy 0, policy_version 1779706 (0.0007) [2023-12-27 17:58:29,740][20381] Updated weights for policy 1, policy_version 1778082 (0.0007) [2023-12-27 17:58:29,794][20381] Updated weights for policy 1, policy_version 1778092 (0.0007) [2023-12-27 17:58:29,854][20381] Updated weights for policy 1, policy_version 1778102 (0.0009) [2023-12-27 17:58:29,911][20381] Updated weights for policy 1, policy_version 1778112 (0.0008) [2023-12-27 17:58:30,364][20347] Updated weights for policy 0, policy_version 1779716 (0.0007) [2023-12-27 17:58:30,422][20347] Updated weights for policy 0, policy_version 1779726 (0.0006) [2023-12-27 17:58:30,484][20347] Updated weights for policy 0, policy_version 1779736 (0.0006) [2023-12-27 17:58:30,629][20381] Updated weights for policy 1, policy_version 1778122 (0.0006) [2023-12-27 17:58:30,682][20381] Updated weights for policy 1, policy_version 1778132 (0.0008) [2023-12-27 17:58:30,726][20381] Updated weights for policy 1, policy_version 1778142 (0.0007) [2023-12-27 17:58:31,066][20347] Updated weights for policy 0, policy_version 1779746 (0.0007) [2023-12-27 17:58:31,125][20347] Updated weights for policy 0, policy_version 1779756 (0.0007) [2023-12-27 17:58:31,188][20347] Updated weights for policy 0, policy_version 1779766 (0.0008) [2023-12-27 17:58:31,246][20347] Updated weights for policy 0, policy_version 1779776 (0.0008) [2023-12-27 17:58:31,481][20381] Updated weights for policy 1, policy_version 1778152 (0.0008) [2023-12-27 17:58:31,544][20381] Updated weights for policy 1, policy_version 1778162 (0.0007) [2023-12-27 17:58:31,606][20381] Updated weights for policy 1, policy_version 1778172 (0.0006) [2023-12-27 17:58:31,882][20347] Updated weights for policy 0, policy_version 1779786 (0.0006) [2023-12-27 17:58:31,932][20347] Updated weights for policy 0, policy_version 1779796 (0.0005) [2023-12-27 17:58:31,985][20347] Updated weights for policy 0, policy_version 1779806 (0.0007) [2023-12-27 17:58:32,286][20381] Updated weights for policy 1, policy_version 1778182 (0.0008) [2023-12-27 17:58:32,352][20381] Updated weights for policy 1, policy_version 1778192 (0.0008) [2023-12-27 17:58:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 910974976. Throughput: 0: 10378.5, 1: 10255.6. Samples: 910989792. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:32,363][19170] Avg episode reward: [(0, '960.960'), (1, '990.280')] [2023-12-27 17:58:32,417][20381] Updated weights for policy 1, policy_version 1778202 (0.0007) [2023-12-27 17:58:32,636][20347] Updated weights for policy 0, policy_version 1779816 (0.0006) [2023-12-27 17:58:32,697][20347] Updated weights for policy 0, policy_version 1779826 (0.0005) [2023-12-27 17:58:32,761][20347] Updated weights for policy 0, policy_version 1779836 (0.0005) [2023-12-27 17:58:33,138][20381] Updated weights for policy 1, policy_version 1778212 (0.0008) [2023-12-27 17:58:33,195][20381] Updated weights for policy 1, policy_version 1778222 (0.0007) [2023-12-27 17:58:33,246][20381] Updated weights for policy 1, policy_version 1778232 (0.0009) [2023-12-27 17:58:33,271][20347] Updated weights for policy 0, policy_version 1779846 (0.0006) [2023-12-27 17:58:33,329][20347] Updated weights for policy 0, policy_version 1779856 (0.0006) [2023-12-27 17:58:33,380][20347] Updated weights for policy 0, policy_version 1779866 (0.0008) [2023-12-27 17:58:33,934][20381] Updated weights for policy 1, policy_version 1778242 (0.0006) [2023-12-27 17:58:33,985][20381] Updated weights for policy 1, policy_version 1778252 (0.0007) [2023-12-27 17:58:34,039][20347] Updated weights for policy 0, policy_version 1779876 (0.0006) [2023-12-27 17:58:34,040][20381] Updated weights for policy 1, policy_version 1778262 (0.0006) [2023-12-27 17:58:34,093][20381] Updated weights for policy 1, policy_version 1778272 (0.0007) [2023-12-27 17:58:34,097][20347] Updated weights for policy 0, policy_version 1779886 (0.0005) [2023-12-27 17:58:34,166][20347] Updated weights for policy 0, policy_version 1779896 (0.0006) [2023-12-27 17:58:34,798][20347] Updated weights for policy 0, policy_version 1779906 (0.0007) [2023-12-27 17:58:34,824][20381] Updated weights for policy 1, policy_version 1778282 (0.0006) [2023-12-27 17:58:34,856][20347] Updated weights for policy 0, policy_version 1779916 (0.0010) [2023-12-27 17:58:34,888][20381] Updated weights for policy 1, policy_version 1778292 (0.0005) [2023-12-27 17:58:34,914][20347] Updated weights for policy 0, policy_version 1779926 (0.0011) [2023-12-27 17:58:34,942][20381] Updated weights for policy 1, policy_version 1778302 (0.0005) [2023-12-27 17:58:34,972][20347] Updated weights for policy 0, policy_version 1779936 (0.0008) [2023-12-27 17:58:35,581][20347] Updated weights for policy 0, policy_version 1779946 (0.0006) [2023-12-27 17:58:35,596][20381] Updated weights for policy 1, policy_version 1778312 (0.0008) [2023-12-27 17:58:35,634][20347] Updated weights for policy 0, policy_version 1779956 (0.0005) [2023-12-27 17:58:35,644][20381] Updated weights for policy 1, policy_version 1778322 (0.0007) [2023-12-27 17:58:35,679][20347] Updated weights for policy 0, policy_version 1779966 (0.0009) [2023-12-27 17:58:35,692][20381] Updated weights for policy 1, policy_version 1778332 (0.0005) [2023-12-27 17:58:36,272][20347] Updated weights for policy 0, policy_version 1779976 (0.0010) [2023-12-27 17:58:36,325][20347] Updated weights for policy 0, policy_version 1779986 (0.0010) [2023-12-27 17:58:36,375][20347] Updated weights for policy 0, policy_version 1779996 (0.0010) [2023-12-27 17:58:36,409][20381] Updated weights for policy 1, policy_version 1778342 (0.0010) [2023-12-27 17:58:36,473][20381] Updated weights for policy 1, policy_version 1778352 (0.0011) [2023-12-27 17:58:36,533][20381] Updated weights for policy 1, policy_version 1778362 (0.0010) [2023-12-27 17:58:37,048][20347] Updated weights for policy 0, policy_version 1780006 (0.0010) [2023-12-27 17:58:37,105][20347] Updated weights for policy 0, policy_version 1780016 (0.0008) [2023-12-27 17:58:37,164][20347] Updated weights for policy 0, policy_version 1780026 (0.0009) [2023-12-27 17:58:37,197][20381] Updated weights for policy 1, policy_version 1778372 (0.0009) [2023-12-27 17:58:37,247][20381] Updated weights for policy 1, policy_version 1778382 (0.0009) [2023-12-27 17:58:37,296][20381] Updated weights for policy 1, policy_version 1778392 (0.0010) [2023-12-27 17:58:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 911089664. Throughput: 0: 10411.7, 1: 10220.6. Samples: 911052380. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:37,363][19170] Avg episode reward: [(0, '961.110'), (1, '990.280')] [2023-12-27 17:58:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001778400_455319552.pth... [2023-12-27 17:58:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001780032_455770112.pth... [2023-12-27 17:58:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001777184_455008256.pth [2023-12-27 17:58:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001778784_455450624.pth [2023-12-27 17:58:37,801][20347] Updated weights for policy 0, policy_version 1780036 (0.0007) [2023-12-27 17:58:37,859][20347] Updated weights for policy 0, policy_version 1780046 (0.0005) [2023-12-27 17:58:37,920][20347] Updated weights for policy 0, policy_version 1780056 (0.0007) [2023-12-27 17:58:38,005][20381] Updated weights for policy 1, policy_version 1778402 (0.0010) [2023-12-27 17:58:38,062][20381] Updated weights for policy 1, policy_version 1778412 (0.0008) [2023-12-27 17:58:38,117][20381] Updated weights for policy 1, policy_version 1778422 (0.0007) [2023-12-27 17:58:38,172][20381] Updated weights for policy 1, policy_version 1778432 (0.0008) [2023-12-27 17:58:38,541][20347] Updated weights for policy 0, policy_version 1780066 (0.0010) [2023-12-27 17:58:38,602][20347] Updated weights for policy 0, policy_version 1780076 (0.0008) [2023-12-27 17:58:38,664][20347] Updated weights for policy 0, policy_version 1780086 (0.0008) [2023-12-27 17:58:38,723][20347] Updated weights for policy 0, policy_version 1780096 (0.0008) [2023-12-27 17:58:38,830][20381] Updated weights for policy 1, policy_version 1778442 (0.0007) [2023-12-27 17:58:38,886][20381] Updated weights for policy 1, policy_version 1778452 (0.0008) [2023-12-27 17:58:38,945][20381] Updated weights for policy 1, policy_version 1778462 (0.0007) [2023-12-27 17:58:39,396][20347] Updated weights for policy 0, policy_version 1780106 (0.0011) [2023-12-27 17:58:39,461][20347] Updated weights for policy 0, policy_version 1780116 (0.0008) [2023-12-27 17:58:39,518][20347] Updated weights for policy 0, policy_version 1780126 (0.0007) [2023-12-27 17:58:39,736][20381] Updated weights for policy 1, policy_version 1778472 (0.0008) [2023-12-27 17:58:39,793][20381] Updated weights for policy 1, policy_version 1778482 (0.0007) [2023-12-27 17:58:39,861][20381] Updated weights for policy 1, policy_version 1778492 (0.0008) [2023-12-27 17:58:40,240][20347] Updated weights for policy 0, policy_version 1780136 (0.0007) [2023-12-27 17:58:40,301][20347] Updated weights for policy 0, policy_version 1780146 (0.0008) [2023-12-27 17:58:40,361][20347] Updated weights for policy 0, policy_version 1780156 (0.0007) [2023-12-27 17:58:40,597][20381] Updated weights for policy 1, policy_version 1778502 (0.0009) [2023-12-27 17:58:40,647][20381] Updated weights for policy 1, policy_version 1778512 (0.0008) [2023-12-27 17:58:40,703][20381] Updated weights for policy 1, policy_version 1778522 (0.0006) [2023-12-27 17:58:41,033][20347] Updated weights for policy 0, policy_version 1780166 (0.0007) [2023-12-27 17:58:41,098][20347] Updated weights for policy 0, policy_version 1780176 (0.0008) [2023-12-27 17:58:41,151][20347] Updated weights for policy 0, policy_version 1780186 (0.0006) [2023-12-27 17:58:41,439][20381] Updated weights for policy 1, policy_version 1778532 (0.0008) [2023-12-27 17:58:41,498][20381] Updated weights for policy 1, policy_version 1778542 (0.0008) [2023-12-27 17:58:41,560][20381] Updated weights for policy 1, policy_version 1778552 (0.0005) [2023-12-27 17:58:41,820][20347] Updated weights for policy 0, policy_version 1780196 (0.0007) [2023-12-27 17:58:41,881][20347] Updated weights for policy 0, policy_version 1780206 (0.0007) [2023-12-27 17:58:41,942][20347] Updated weights for policy 0, policy_version 1780216 (0.0007) [2023-12-27 17:58:42,289][20381] Updated weights for policy 1, policy_version 1778562 (0.0007) [2023-12-27 17:58:42,350][20381] Updated weights for policy 1, policy_version 1778572 (0.0007) [2023-12-27 17:58:42,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 911179776. Throughput: 0: 10487.6, 1: 10114.0. Samples: 911174848. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-12-27 17:58:42,365][19170] Avg episode reward: [(0, '961.070'), (1, '990.280')] [2023-12-27 17:58:42,414][20381] Updated weights for policy 1, policy_version 1778582 (0.0007) [2023-12-27 17:58:42,472][20381] Updated weights for policy 1, policy_version 1778592 (0.0006) [2023-12-27 17:58:42,633][20347] Updated weights for policy 0, policy_version 1780226 (0.0008) [2023-12-27 17:58:42,693][20347] Updated weights for policy 0, policy_version 1780236 (0.0008) [2023-12-27 17:58:42,749][20347] Updated weights for policy 0, policy_version 1780246 (0.0007) [2023-12-27 17:58:42,817][20347] Updated weights for policy 0, policy_version 1780256 (0.0007) [2023-12-27 17:58:43,111][20381] Updated weights for policy 1, policy_version 1778602 (0.0006) [2023-12-27 17:58:43,158][20381] Updated weights for policy 1, policy_version 1778612 (0.0009) [2023-12-27 17:58:43,217][20381] Updated weights for policy 1, policy_version 1778622 (0.0009) [2023-12-27 17:58:43,397][20347] Updated weights for policy 0, policy_version 1780266 (0.0006) [2023-12-27 17:58:43,462][20347] Updated weights for policy 0, policy_version 1780276 (0.0007) [2023-12-27 17:58:43,523][20347] Updated weights for policy 0, policy_version 1780286 (0.0007) [2023-12-27 17:58:43,850][20381] Updated weights for policy 1, policy_version 1778632 (0.0006) [2023-12-27 17:58:43,901][20381] Updated weights for policy 1, policy_version 1778642 (0.0005) [2023-12-27 17:58:43,963][20381] Updated weights for policy 1, policy_version 1778652 (0.0006) [2023-12-27 17:58:44,122][20347] Updated weights for policy 0, policy_version 1780296 (0.0007) [2023-12-27 17:58:44,180][20347] Updated weights for policy 0, policy_version 1780306 (0.0009) [2023-12-27 17:58:44,233][20347] Updated weights for policy 0, policy_version 1780316 (0.0007) [2023-12-27 17:58:44,494][20381] Updated weights for policy 1, policy_version 1778662 (0.0005) [2023-12-27 17:58:44,547][20381] Updated weights for policy 1, policy_version 1778672 (0.0006) [2023-12-27 17:58:44,601][20381] Updated weights for policy 1, policy_version 1778682 (0.0005) [2023-12-27 17:58:44,874][20347] Updated weights for policy 0, policy_version 1780326 (0.0008) [2023-12-27 17:58:44,929][20347] Updated weights for policy 0, policy_version 1780336 (0.0007) [2023-12-27 17:58:44,986][20347] Updated weights for policy 0, policy_version 1780346 (0.0007) [2023-12-27 17:58:45,240][20381] Updated weights for policy 1, policy_version 1778692 (0.0007) [2023-12-27 17:58:45,302][20381] Updated weights for policy 1, policy_version 1778702 (0.0008) [2023-12-27 17:58:45,360][20381] Updated weights for policy 1, policy_version 1778712 (0.0006) [2023-12-27 17:58:45,664][20347] Updated weights for policy 0, policy_version 1780356 (0.0007) [2023-12-27 17:58:45,714][20347] Updated weights for policy 0, policy_version 1780366 (0.0009) [2023-12-27 17:58:45,760][20347] Updated weights for policy 0, policy_version 1780376 (0.0010) [2023-12-27 17:58:45,885][20381] Updated weights for policy 1, policy_version 1778722 (0.0006) [2023-12-27 17:58:45,936][20381] Updated weights for policy 1, policy_version 1778732 (0.0010) [2023-12-27 17:58:45,993][20381] Updated weights for policy 1, policy_version 1778742 (0.0010) [2023-12-27 17:58:46,055][20381] Updated weights for policy 1, policy_version 1778752 (0.0010) [2023-12-27 17:58:46,483][20347] Updated weights for policy 0, policy_version 1780386 (0.0009) [2023-12-27 17:58:46,532][20347] Updated weights for policy 0, policy_version 1780396 (0.0007) [2023-12-27 17:58:46,582][20347] Updated weights for policy 0, policy_version 1780406 (0.0007) [2023-12-27 17:58:46,642][20347] Updated weights for policy 0, policy_version 1780416 (0.0008) [2023-12-27 17:58:46,717][20381] Updated weights for policy 1, policy_version 1778762 (0.0010) [2023-12-27 17:58:46,774][20381] Updated weights for policy 1, policy_version 1778772 (0.0010) [2023-12-27 17:58:46,830][20381] Updated weights for policy 1, policy_version 1778782 (0.0007) [2023-12-27 17:58:47,240][20347] Updated weights for policy 0, policy_version 1780426 (0.0009) [2023-12-27 17:58:47,298][20347] Updated weights for policy 0, policy_version 1780436 (0.0008) [2023-12-27 17:58:47,354][20347] Updated weights for policy 0, policy_version 1780446 (0.0010) [2023-12-27 17:58:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 911286272. Throughput: 0: 10513.8, 1: 10230.0. Samples: 911302672. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:58:47,363][19170] Avg episode reward: [(0, '970.670'), (1, '990.300')] [2023-12-27 17:58:47,413][20381] Updated weights for policy 1, policy_version 1778792 (0.0006) [2023-12-27 17:58:47,475][20381] Updated weights for policy 1, policy_version 1778802 (0.0006) [2023-12-27 17:58:47,540][20381] Updated weights for policy 1, policy_version 1778812 (0.0007) [2023-12-27 17:58:48,047][20347] Updated weights for policy 0, policy_version 1780456 (0.0008) [2023-12-27 17:58:48,101][20347] Updated weights for policy 0, policy_version 1780466 (0.0005) [2023-12-27 17:58:48,152][20347] Updated weights for policy 0, policy_version 1780476 (0.0010) [2023-12-27 17:58:48,199][20381] Updated weights for policy 1, policy_version 1778822 (0.0006) [2023-12-27 17:58:48,249][20381] Updated weights for policy 1, policy_version 1778832 (0.0005) [2023-12-27 17:58:48,301][20381] Updated weights for policy 1, policy_version 1778842 (0.0005) [2023-12-27 17:58:48,817][20347] Updated weights for policy 0, policy_version 1780486 (0.0010) [2023-12-27 17:58:48,868][20347] Updated weights for policy 0, policy_version 1780496 (0.0010) [2023-12-27 17:58:48,918][20381] Updated weights for policy 1, policy_version 1778852 (0.0007) [2023-12-27 17:58:48,924][20347] Updated weights for policy 0, policy_version 1780506 (0.0010) [2023-12-27 17:58:48,975][20381] Updated weights for policy 1, policy_version 1778862 (0.0006) [2023-12-27 17:58:49,030][20381] Updated weights for policy 1, policy_version 1778872 (0.0007) [2023-12-27 17:58:49,670][20347] Updated weights for policy 0, policy_version 1780516 (0.0009) [2023-12-27 17:58:49,690][20381] Updated weights for policy 1, policy_version 1778882 (0.0007) [2023-12-27 17:58:49,726][20347] Updated weights for policy 0, policy_version 1780526 (0.0009) [2023-12-27 17:58:49,747][20381] Updated weights for policy 1, policy_version 1778892 (0.0006) [2023-12-27 17:58:49,787][20347] Updated weights for policy 0, policy_version 1780536 (0.0007) [2023-12-27 17:58:49,796][20381] Updated weights for policy 1, policy_version 1778902 (0.0007) [2023-12-27 17:58:49,862][20381] Updated weights for policy 1, policy_version 1778912 (0.0006) [2023-12-27 17:58:50,480][20347] Updated weights for policy 0, policy_version 1780546 (0.0009) [2023-12-27 17:58:50,545][20347] Updated weights for policy 0, policy_version 1780556 (0.0010) [2023-12-27 17:58:50,595][20381] Updated weights for policy 1, policy_version 1778922 (0.0008) [2023-12-27 17:58:50,625][20347] Updated weights for policy 0, policy_version 1780566 (0.0008) [2023-12-27 17:58:50,652][20381] Updated weights for policy 1, policy_version 1778932 (0.0006) [2023-12-27 17:58:50,701][20381] Updated weights for policy 1, policy_version 1778942 (0.0008) [2023-12-27 17:58:51,288][20347] Updated weights for policy 0, policy_version 1780577 (0.0007) [2023-12-27 17:58:51,361][20347] Updated weights for policy 0, policy_version 1780587 (0.0007) [2023-12-27 17:58:51,425][20347] Updated weights for policy 0, policy_version 1780597 (0.0007) [2023-12-27 17:58:51,443][20381] Updated weights for policy 1, policy_version 1778952 (0.0007) [2023-12-27 17:58:51,482][20347] Updated weights for policy 0, policy_version 1780607 (0.0005) [2023-12-27 17:58:51,501][20381] Updated weights for policy 1, policy_version 1778962 (0.0008) [2023-12-27 17:58:51,564][20381] Updated weights for policy 1, policy_version 1778972 (0.0009) [2023-12-27 17:58:52,157][20347] Updated weights for policy 0, policy_version 1780617 (0.0006) [2023-12-27 17:58:52,218][20347] Updated weights for policy 0, policy_version 1780627 (0.0006) [2023-12-27 17:58:52,277][20347] Updated weights for policy 0, policy_version 1780637 (0.0009) [2023-12-27 17:58:52,320][20381] Updated weights for policy 1, policy_version 1778982 (0.0007) [2023-12-27 17:58:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 911392768. Throughput: 0: 10470.1, 1: 10243.4. Samples: 911364340. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:58:52,363][19170] Avg episode reward: [(0, '961.070'), (1, '990.300')] [2023-12-27 17:58:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001780640_455925760.pth... [2023-12-27 17:58:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001779392_455606272.pth [2023-12-27 17:58:52,384][20381] Updated weights for policy 1, policy_version 1778992 (0.0008) [2023-12-27 17:58:52,437][20381] Updated weights for policy 1, policy_version 1779002 (0.0006) [2023-12-27 17:58:52,465][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001779008_455475200.pth... [2023-12-27 17:58:52,468][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001777792_455163904.pth [2023-12-27 17:58:52,928][20347] Updated weights for policy 0, policy_version 1780647 (0.0010) [2023-12-27 17:58:52,980][20347] Updated weights for policy 0, policy_version 1780657 (0.0009) [2023-12-27 17:58:53,029][20347] Updated weights for policy 0, policy_version 1780667 (0.0006) [2023-12-27 17:58:53,112][20381] Updated weights for policy 1, policy_version 1779012 (0.0010) [2023-12-27 17:58:53,167][20381] Updated weights for policy 1, policy_version 1779022 (0.0011) [2023-12-27 17:58:53,222][20381] Updated weights for policy 1, policy_version 1779032 (0.0010) [2023-12-27 17:58:53,704][20347] Updated weights for policy 0, policy_version 1780677 (0.0008) [2023-12-27 17:58:53,754][20347] Updated weights for policy 0, policy_version 1780687 (0.0005) [2023-12-27 17:58:53,805][20347] Updated weights for policy 0, policy_version 1780697 (0.0005) [2023-12-27 17:58:53,890][20381] Updated weights for policy 1, policy_version 1779042 (0.0010) [2023-12-27 17:58:53,942][20381] Updated weights for policy 1, policy_version 1779052 (0.0008) [2023-12-27 17:58:53,988][20381] Updated weights for policy 1, policy_version 1779062 (0.0005) [2023-12-27 17:58:54,046][20381] Updated weights for policy 1, policy_version 1779072 (0.0005) [2023-12-27 17:58:54,398][20347] Updated weights for policy 0, policy_version 1780707 (0.0006) [2023-12-27 17:58:54,458][20347] Updated weights for policy 0, policy_version 1780717 (0.0007) [2023-12-27 17:58:54,520][20347] Updated weights for policy 0, policy_version 1780727 (0.0008) [2023-12-27 17:58:54,656][20381] Updated weights for policy 1, policy_version 1779082 (0.0005) [2023-12-27 17:58:54,704][20381] Updated weights for policy 1, policy_version 1779092 (0.0005) [2023-12-27 17:58:54,755][20381] Updated weights for policy 1, policy_version 1779102 (0.0005) [2023-12-27 17:58:55,254][20347] Updated weights for policy 0, policy_version 1780737 (0.0008) [2023-12-27 17:58:55,307][20347] Updated weights for policy 0, policy_version 1780747 (0.0008) [2023-12-27 17:58:55,367][20347] Updated weights for policy 0, policy_version 1780757 (0.0007) [2023-12-27 17:58:55,392][20381] Updated weights for policy 1, policy_version 1779112 (0.0007) [2023-12-27 17:58:55,424][20347] Updated weights for policy 0, policy_version 1780767 (0.0006) [2023-12-27 17:58:55,452][20381] Updated weights for policy 1, policy_version 1779122 (0.0006) [2023-12-27 17:58:55,506][20381] Updated weights for policy 1, policy_version 1779132 (0.0008) [2023-12-27 17:58:56,083][20347] Updated weights for policy 0, policy_version 1780777 (0.0005) [2023-12-27 17:58:56,092][20381] Updated weights for policy 1, policy_version 1779142 (0.0007) [2023-12-27 17:58:56,156][20347] Updated weights for policy 0, policy_version 1780787 (0.0007) [2023-12-27 17:58:56,160][20381] Updated weights for policy 1, policy_version 1779152 (0.0010) [2023-12-27 17:58:56,212][20347] Updated weights for policy 0, policy_version 1780797 (0.0008) [2023-12-27 17:58:56,217][20381] Updated weights for policy 1, policy_version 1779162 (0.0009) [2023-12-27 17:58:56,890][20381] Updated weights for policy 1, policy_version 1779172 (0.0011) [2023-12-27 17:58:56,906][20347] Updated weights for policy 0, policy_version 1780807 (0.0006) [2023-12-27 17:58:56,934][20381] Updated weights for policy 1, policy_version 1779182 (0.0010) [2023-12-27 17:58:56,962][20347] Updated weights for policy 0, policy_version 1780817 (0.0005) [2023-12-27 17:58:56,976][20381] Updated weights for policy 1, policy_version 1779192 (0.0010) [2023-12-27 17:58:57,015][20347] Updated weights for policy 0, policy_version 1780827 (0.0005) [2023-12-27 17:58:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 911499264. Throughput: 0: 10465.3, 1: 10332.0. Samples: 911488476. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:58:57,363][19170] Avg episode reward: [(0, '970.870'), (1, '1000.000')] [2023-12-27 17:58:57,601][20347] Updated weights for policy 0, policy_version 1780837 (0.0006) [2023-12-27 17:58:57,604][20381] Updated weights for policy 1, policy_version 1779202 (0.0009) [2023-12-27 17:58:57,651][20347] Updated weights for policy 0, policy_version 1780847 (0.0008) [2023-12-27 17:58:57,657][20381] Updated weights for policy 1, policy_version 1779212 (0.0005) [2023-12-27 17:58:57,702][20347] Updated weights for policy 0, policy_version 1780857 (0.0009) [2023-12-27 17:58:57,716][20381] Updated weights for policy 1, policy_version 1779222 (0.0009) [2023-12-27 17:58:57,767][20381] Updated weights for policy 1, policy_version 1779232 (0.0010) [2023-12-27 17:58:58,427][20381] Updated weights for policy 1, policy_version 1779242 (0.0010) [2023-12-27 17:58:58,457][20347] Updated weights for policy 0, policy_version 1780867 (0.0005) [2023-12-27 17:58:58,492][20381] Updated weights for policy 1, policy_version 1779252 (0.0009) [2023-12-27 17:58:58,520][20347] Updated weights for policy 0, policy_version 1780877 (0.0006) [2023-12-27 17:58:58,555][20381] Updated weights for policy 1, policy_version 1779262 (0.0006) [2023-12-27 17:58:58,588][20347] Updated weights for policy 0, policy_version 1780887 (0.0009) [2023-12-27 17:58:59,302][20381] Updated weights for policy 1, policy_version 1779272 (0.0010) [2023-12-27 17:58:59,305][20347] Updated weights for policy 0, policy_version 1780897 (0.0007) [2023-12-27 17:58:59,365][20381] Updated weights for policy 1, policy_version 1779282 (0.0011) [2023-12-27 17:58:59,371][20347] Updated weights for policy 0, policy_version 1780907 (0.0008) [2023-12-27 17:58:59,426][20347] Updated weights for policy 0, policy_version 1780917 (0.0006) [2023-12-27 17:58:59,428][20381] Updated weights for policy 1, policy_version 1779292 (0.0011) [2023-12-27 17:58:59,487][20347] Updated weights for policy 0, policy_version 1780927 (0.0006) [2023-12-27 17:59:00,101][20381] Updated weights for policy 1, policy_version 1779302 (0.0008) [2023-12-27 17:59:00,160][20381] Updated weights for policy 1, policy_version 1779312 (0.0011) [2023-12-27 17:59:00,188][20347] Updated weights for policy 0, policy_version 1780937 (0.0006) [2023-12-27 17:59:00,220][20381] Updated weights for policy 1, policy_version 1779322 (0.0011) [2023-12-27 17:59:00,245][20347] Updated weights for policy 0, policy_version 1780947 (0.0006) [2023-12-27 17:59:00,302][20347] Updated weights for policy 0, policy_version 1780957 (0.0007) [2023-12-27 17:59:00,814][20381] Updated weights for policy 1, policy_version 1779332 (0.0008) [2023-12-27 17:59:00,860][20381] Updated weights for policy 1, policy_version 1779342 (0.0005) [2023-12-27 17:59:00,905][20381] Updated weights for policy 1, policy_version 1779352 (0.0005) [2023-12-27 17:59:00,991][20347] Updated weights for policy 0, policy_version 1780967 (0.0009) [2023-12-27 17:59:01,067][20347] Updated weights for policy 0, policy_version 1780977 (0.0007) [2023-12-27 17:59:01,129][20347] Updated weights for policy 0, policy_version 1780987 (0.0007) [2023-12-27 17:59:01,546][20381] Updated weights for policy 1, policy_version 1779362 (0.0005) [2023-12-27 17:59:01,597][20381] Updated weights for policy 1, policy_version 1779372 (0.0005) [2023-12-27 17:59:01,660][20381] Updated weights for policy 1, policy_version 1779382 (0.0008) [2023-12-27 17:59:01,706][20381] Updated weights for policy 1, policy_version 1779392 (0.0005) [2023-12-27 17:59:01,853][20347] Updated weights for policy 0, policy_version 1780997 (0.0008) [2023-12-27 17:59:01,902][20347] Updated weights for policy 0, policy_version 1781007 (0.0010) [2023-12-27 17:59:01,951][20347] Updated weights for policy 0, policy_version 1781017 (0.0007) [2023-12-27 17:59:02,336][20381] Updated weights for policy 1, policy_version 1779402 (0.0007) [2023-12-27 17:59:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 911597568. Throughput: 0: 10376.3, 1: 10366.8. Samples: 911610676. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:02,363][19170] Avg episode reward: [(0, '980.530'), (1, '1000.000')] [2023-12-27 17:59:02,393][20381] Updated weights for policy 1, policy_version 1779412 (0.0007) [2023-12-27 17:59:02,438][20381] Updated weights for policy 1, policy_version 1779422 (0.0007) [2023-12-27 17:59:02,694][20347] Updated weights for policy 0, policy_version 1781027 (0.0009) [2023-12-27 17:59:02,751][20347] Updated weights for policy 0, policy_version 1781037 (0.0009) [2023-12-27 17:59:02,817][20347] Updated weights for policy 0, policy_version 1781047 (0.0006) [2023-12-27 17:59:03,080][20381] Updated weights for policy 1, policy_version 1779432 (0.0008) [2023-12-27 17:59:03,135][20381] Updated weights for policy 1, policy_version 1779442 (0.0008) [2023-12-27 17:59:03,183][20381] Updated weights for policy 1, policy_version 1779452 (0.0006) [2023-12-27 17:59:03,415][20347] Updated weights for policy 0, policy_version 1781057 (0.0008) [2023-12-27 17:59:03,469][20347] Updated weights for policy 0, policy_version 1781067 (0.0010) [2023-12-27 17:59:03,522][20347] Updated weights for policy 0, policy_version 1781077 (0.0010) [2023-12-27 17:59:03,573][20347] Updated weights for policy 0, policy_version 1781087 (0.0010) [2023-12-27 17:59:03,908][20381] Updated weights for policy 1, policy_version 1779462 (0.0007) [2023-12-27 17:59:03,968][20381] Updated weights for policy 1, policy_version 1779472 (0.0007) [2023-12-27 17:59:04,026][20381] Updated weights for policy 1, policy_version 1779482 (0.0009) [2023-12-27 17:59:04,224][20347] Updated weights for policy 0, policy_version 1781097 (0.0009) [2023-12-27 17:59:04,277][20347] Updated weights for policy 0, policy_version 1781107 (0.0007) [2023-12-27 17:59:04,329][20347] Updated weights for policy 0, policy_version 1781117 (0.0007) [2023-12-27 17:59:04,762][20381] Updated weights for policy 1, policy_version 1779492 (0.0008) [2023-12-27 17:59:04,808][20381] Updated weights for policy 1, policy_version 1779502 (0.0009) [2023-12-27 17:59:04,867][20381] Updated weights for policy 1, policy_version 1779512 (0.0007) [2023-12-27 17:59:04,950][20347] Updated weights for policy 0, policy_version 1781127 (0.0006) [2023-12-27 17:59:05,001][20347] Updated weights for policy 0, policy_version 1781137 (0.0005) [2023-12-27 17:59:05,054][20347] Updated weights for policy 0, policy_version 1781147 (0.0009) [2023-12-27 17:59:05,523][20381] Updated weights for policy 1, policy_version 1779522 (0.0006) [2023-12-27 17:59:05,574][20381] Updated weights for policy 1, policy_version 1779532 (0.0010) [2023-12-27 17:59:05,628][20347] Updated weights for policy 0, policy_version 1781157 (0.0006) [2023-12-27 17:59:05,628][20381] Updated weights for policy 1, policy_version 1779542 (0.0010) [2023-12-27 17:59:05,673][20347] Updated weights for policy 0, policy_version 1781167 (0.0005) [2023-12-27 17:59:05,683][20291] KL-divergence is very high: 106.2539 [2023-12-27 17:59:05,686][20381] Updated weights for policy 1, policy_version 1779552 (0.0010) [2023-12-27 17:59:05,731][20347] Updated weights for policy 0, policy_version 1781177 (0.0005) [2023-12-27 17:59:06,367][20347] Updated weights for policy 0, policy_version 1781187 (0.0005) [2023-12-27 17:59:06,388][20381] Updated weights for policy 1, policy_version 1779562 (0.0011) [2023-12-27 17:59:06,420][20347] Updated weights for policy 0, policy_version 1781197 (0.0006) [2023-12-27 17:59:06,443][20381] Updated weights for policy 1, policy_version 1779572 (0.0011) [2023-12-27 17:59:06,480][20347] Updated weights for policy 0, policy_version 1781207 (0.0006) [2023-12-27 17:59:06,497][20381] Updated weights for policy 1, policy_version 1779582 (0.0011) [2023-12-27 17:59:07,136][20347] Updated weights for policy 0, policy_version 1781217 (0.0006) [2023-12-27 17:59:07,156][20381] Updated weights for policy 1, policy_version 1779592 (0.0009) [2023-12-27 17:59:07,200][20347] Updated weights for policy 0, policy_version 1781227 (0.0009) [2023-12-27 17:59:07,212][20381] Updated weights for policy 1, policy_version 1779602 (0.0011) [2023-12-27 17:59:07,261][20347] Updated weights for policy 0, policy_version 1781237 (0.0011) [2023-12-27 17:59:07,269][20381] Updated weights for policy 1, policy_version 1779612 (0.0011) [2023-12-27 17:59:07,319][20347] Updated weights for policy 0, policy_version 1781247 (0.0011) [2023-12-27 17:59:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20605.0). Total num frames: 911712256. Throughput: 0: 10445.3, 1: 10357.2. Samples: 911673832. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:07,363][19170] Avg episode reward: [(0, '970.670'), (1, '990.290')] [2023-12-27 17:59:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001779616_455630848.pth... [2023-12-27 17:59:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001781248_456081408.pth... [2023-12-27 17:59:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001778400_455319552.pth [2023-12-27 17:59:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001780032_455770112.pth [2023-12-27 17:59:07,908][20381] Updated weights for policy 1, policy_version 1779622 (0.0007) [2023-12-27 17:59:07,959][20381] Updated weights for policy 1, policy_version 1779632 (0.0005) [2023-12-27 17:59:08,010][20381] Updated weights for policy 1, policy_version 1779642 (0.0005) [2023-12-27 17:59:08,017][20347] Updated weights for policy 0, policy_version 1781257 (0.0007) [2023-12-27 17:59:08,072][20347] Updated weights for policy 0, policy_version 1781267 (0.0008) [2023-12-27 17:59:08,124][20347] Updated weights for policy 0, policy_version 1781277 (0.0008) [2023-12-27 17:59:08,635][20381] Updated weights for policy 1, policy_version 1779652 (0.0006) [2023-12-27 17:59:08,680][20381] Updated weights for policy 1, policy_version 1779662 (0.0007) [2023-12-27 17:59:08,741][20381] Updated weights for policy 1, policy_version 1779672 (0.0009) [2023-12-27 17:59:08,794][20347] Updated weights for policy 0, policy_version 1781287 (0.0010) [2023-12-27 17:59:08,850][20347] Updated weights for policy 0, policy_version 1781297 (0.0008) [2023-12-27 17:59:08,912][20347] Updated weights for policy 0, policy_version 1781307 (0.0010) [2023-12-27 17:59:09,401][20381] Updated weights for policy 1, policy_version 1779682 (0.0008) [2023-12-27 17:59:09,464][20381] Updated weights for policy 1, policy_version 1779692 (0.0011) [2023-12-27 17:59:09,518][20381] Updated weights for policy 1, policy_version 1779702 (0.0011) [2023-12-27 17:59:09,577][20381] Updated weights for policy 1, policy_version 1779712 (0.0011) [2023-12-27 17:59:09,643][20347] Updated weights for policy 0, policy_version 1781317 (0.0009) [2023-12-27 17:59:09,704][20347] Updated weights for policy 0, policy_version 1781327 (0.0009) [2023-12-27 17:59:09,762][20347] Updated weights for policy 0, policy_version 1781337 (0.0009) [2023-12-27 17:59:10,298][20381] Updated weights for policy 1, policy_version 1779722 (0.0010) [2023-12-27 17:59:10,355][20381] Updated weights for policy 1, policy_version 1779732 (0.0009) [2023-12-27 17:59:10,410][20381] Updated weights for policy 1, policy_version 1779742 (0.0011) [2023-12-27 17:59:10,476][20347] Updated weights for policy 0, policy_version 1781347 (0.0008) [2023-12-27 17:59:10,534][20347] Updated weights for policy 0, policy_version 1781357 (0.0008) [2023-12-27 17:59:10,590][20347] Updated weights for policy 0, policy_version 1781367 (0.0010) [2023-12-27 17:59:11,047][20381] Updated weights for policy 1, policy_version 1779752 (0.0010) [2023-12-27 17:59:11,105][20381] Updated weights for policy 1, policy_version 1779762 (0.0011) [2023-12-27 17:59:11,164][20381] Updated weights for policy 1, policy_version 1779772 (0.0010) [2023-12-27 17:59:11,279][20347] Updated weights for policy 0, policy_version 1781377 (0.0010) [2023-12-27 17:59:11,347][20347] Updated weights for policy 0, policy_version 1781387 (0.0010) [2023-12-27 17:59:11,422][20347] Updated weights for policy 0, policy_version 1781397 (0.0007) [2023-12-27 17:59:11,482][20347] Updated weights for policy 0, policy_version 1781407 (0.0009) [2023-12-27 17:59:11,886][20381] Updated weights for policy 1, policy_version 1779782 (0.0005) [2023-12-27 17:59:11,942][20381] Updated weights for policy 1, policy_version 1779792 (0.0006) [2023-12-27 17:59:11,992][20381] Updated weights for policy 1, policy_version 1779802 (0.0005) [2023-12-27 17:59:12,238][20347] Updated weights for policy 0, policy_version 1781417 (0.0006) [2023-12-27 17:59:12,311][20347] Updated weights for policy 0, policy_version 1781427 (0.0007) [2023-12-27 17:59:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 911802368. Throughput: 0: 10424.5, 1: 10333.8. Samples: 911798032. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:12,363][19170] Avg episode reward: [(0, '921.760'), (1, '990.290')] [2023-12-27 17:59:12,370][20347] Updated weights for policy 0, policy_version 1781437 (0.0008) [2023-12-27 17:59:12,645][20381] Updated weights for policy 1, policy_version 1779812 (0.0006) [2023-12-27 17:59:12,692][20381] Updated weights for policy 1, policy_version 1779822 (0.0008) [2023-12-27 17:59:12,741][20381] Updated weights for policy 1, policy_version 1779832 (0.0007) [2023-12-27 17:59:13,006][20347] Updated weights for policy 0, policy_version 1781447 (0.0010) [2023-12-27 17:59:13,064][20347] Updated weights for policy 0, policy_version 1781457 (0.0007) [2023-12-27 17:59:13,116][20347] Updated weights for policy 0, policy_version 1781467 (0.0005) [2023-12-27 17:59:13,424][20381] Updated weights for policy 1, policy_version 1779842 (0.0008) [2023-12-27 17:59:13,478][20381] Updated weights for policy 1, policy_version 1779852 (0.0007) [2023-12-27 17:59:13,526][20381] Updated weights for policy 1, policy_version 1779862 (0.0008) [2023-12-27 17:59:13,582][20381] Updated weights for policy 1, policy_version 1779872 (0.0008) [2023-12-27 17:59:13,750][20347] Updated weights for policy 0, policy_version 1781477 (0.0007) [2023-12-27 17:59:13,808][20347] Updated weights for policy 0, policy_version 1781487 (0.0010) [2023-12-27 17:59:13,869][20347] Updated weights for policy 0, policy_version 1781497 (0.0007) [2023-12-27 17:59:14,223][20381] Updated weights for policy 1, policy_version 1779882 (0.0010) [2023-12-27 17:59:14,275][20381] Updated weights for policy 1, policy_version 1779892 (0.0010) [2023-12-27 17:59:14,323][20381] Updated weights for policy 1, policy_version 1779902 (0.0010) [2023-12-27 17:59:14,415][20347] Updated weights for policy 0, policy_version 1781507 (0.0006) [2023-12-27 17:59:14,465][20347] Updated weights for policy 0, policy_version 1781517 (0.0005) [2023-12-27 17:59:14,521][20347] Updated weights for policy 0, policy_version 1781527 (0.0005) [2023-12-27 17:59:15,013][20381] Updated weights for policy 1, policy_version 1779912 (0.0008) [2023-12-27 17:59:15,078][20381] Updated weights for policy 1, policy_version 1779922 (0.0006) [2023-12-27 17:59:15,142][20381] Updated weights for policy 1, policy_version 1779932 (0.0008) [2023-12-27 17:59:15,155][20347] Updated weights for policy 0, policy_version 1781537 (0.0005) [2023-12-27 17:59:15,218][20347] Updated weights for policy 0, policy_version 1781547 (0.0009) [2023-12-27 17:59:15,276][20347] Updated weights for policy 0, policy_version 1781557 (0.0007) [2023-12-27 17:59:15,333][20347] Updated weights for policy 0, policy_version 1781567 (0.0008) [2023-12-27 17:59:15,800][20381] Updated weights for policy 1, policy_version 1779942 (0.0007) [2023-12-27 17:59:15,856][20381] Updated weights for policy 1, policy_version 1779952 (0.0007) [2023-12-27 17:59:15,904][20381] Updated weights for policy 1, policy_version 1779962 (0.0008) [2023-12-27 17:59:15,968][20347] Updated weights for policy 0, policy_version 1781577 (0.0007) [2023-12-27 17:59:16,014][20347] Updated weights for policy 0, policy_version 1781587 (0.0005) [2023-12-27 17:59:16,057][20347] Updated weights for policy 0, policy_version 1781597 (0.0005) [2023-12-27 17:59:16,645][20347] Updated weights for policy 0, policy_version 1781607 (0.0006) [2023-12-27 17:59:16,662][20381] Updated weights for policy 1, policy_version 1779972 (0.0007) [2023-12-27 17:59:16,704][20347] Updated weights for policy 0, policy_version 1781617 (0.0006) [2023-12-27 17:59:16,723][20381] Updated weights for policy 1, policy_version 1779982 (0.0007) [2023-12-27 17:59:16,766][20347] Updated weights for policy 0, policy_version 1781627 (0.0005) [2023-12-27 17:59:16,786][20381] Updated weights for policy 1, policy_version 1779992 (0.0006) [2023-12-27 17:59:17,330][20347] Updated weights for policy 0, policy_version 1781637 (0.0006) [2023-12-27 17:59:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 911908864. Throughput: 0: 10410.0, 1: 10319.3. Samples: 911922612. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:17,363][19170] Avg episode reward: [(0, '921.810'), (1, '961.080')] [2023-12-27 17:59:17,386][20347] Updated weights for policy 0, policy_version 1781647 (0.0005) [2023-12-27 17:59:17,444][20347] Updated weights for policy 0, policy_version 1781657 (0.0007) [2023-12-27 17:59:17,448][20381] Updated weights for policy 1, policy_version 1780002 (0.0007) [2023-12-27 17:59:17,507][20381] Updated weights for policy 1, policy_version 1780012 (0.0007) [2023-12-27 17:59:17,560][20381] Updated weights for policy 1, policy_version 1780022 (0.0010) [2023-12-27 17:59:17,606][20381] Updated weights for policy 1, policy_version 1780032 (0.0010) [2023-12-27 17:59:18,038][20347] Updated weights for policy 0, policy_version 1781667 (0.0007) [2023-12-27 17:59:18,093][20347] Updated weights for policy 0, policy_version 1781677 (0.0008) [2023-12-27 17:59:18,150][20347] Updated weights for policy 0, policy_version 1781687 (0.0010) [2023-12-27 17:59:18,259][20381] Updated weights for policy 1, policy_version 1780042 (0.0010) [2023-12-27 17:59:18,307][20381] Updated weights for policy 1, policy_version 1780052 (0.0010) [2023-12-27 17:59:18,367][20381] Updated weights for policy 1, policy_version 1780062 (0.0009) [2023-12-27 17:59:18,831][20347] Updated weights for policy 0, policy_version 1781697 (0.0009) [2023-12-27 17:59:18,891][20347] Updated weights for policy 0, policy_version 1781707 (0.0011) [2023-12-27 17:59:18,950][20347] Updated weights for policy 0, policy_version 1781717 (0.0011) [2023-12-27 17:59:19,000][20347] Updated weights for policy 0, policy_version 1781727 (0.0010) [2023-12-27 17:59:19,078][20381] Updated weights for policy 1, policy_version 1780072 (0.0010) [2023-12-27 17:59:19,136][20381] Updated weights for policy 1, policy_version 1780082 (0.0010) [2023-12-27 17:59:19,184][20381] Updated weights for policy 1, policy_version 1780092 (0.0010) [2023-12-27 17:59:19,815][20347] Updated weights for policy 0, policy_version 1781737 (0.0009) [2023-12-27 17:59:19,887][20347] Updated weights for policy 0, policy_version 1781747 (0.0009) [2023-12-27 17:59:19,913][20381] Updated weights for policy 1, policy_version 1780102 (0.0009) [2023-12-27 17:59:19,950][20347] Updated weights for policy 0, policy_version 1781757 (0.0009) [2023-12-27 17:59:19,977][20381] Updated weights for policy 1, policy_version 1780112 (0.0008) [2023-12-27 17:59:20,030][20381] Updated weights for policy 1, policy_version 1780122 (0.0008) [2023-12-27 17:59:20,688][20347] Updated weights for policy 0, policy_version 1781767 (0.0008) [2023-12-27 17:59:20,751][20347] Updated weights for policy 0, policy_version 1781777 (0.0008) [2023-12-27 17:59:20,786][20381] Updated weights for policy 1, policy_version 1780132 (0.0007) [2023-12-27 17:59:20,815][20347] Updated weights for policy 0, policy_version 1781787 (0.0007) [2023-12-27 17:59:20,845][20381] Updated weights for policy 1, policy_version 1780142 (0.0008) [2023-12-27 17:59:20,908][20381] Updated weights for policy 1, policy_version 1780152 (0.0005) [2023-12-27 17:59:21,477][20347] Updated weights for policy 0, policy_version 1781797 (0.0007) [2023-12-27 17:59:21,531][20347] Updated weights for policy 0, policy_version 1781807 (0.0007) [2023-12-27 17:59:21,578][20381] Updated weights for policy 1, policy_version 1780162 (0.0009) [2023-12-27 17:59:21,588][20347] Updated weights for policy 0, policy_version 1781817 (0.0007) [2023-12-27 17:59:21,647][20381] Updated weights for policy 1, policy_version 1780172 (0.0008) [2023-12-27 17:59:21,698][20381] Updated weights for policy 1, policy_version 1780182 (0.0009) [2023-12-27 17:59:21,768][20381] Updated weights for policy 1, policy_version 1780192 (0.0007) [2023-12-27 17:59:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 912007168. Throughput: 0: 10345.2, 1: 10322.1. Samples: 911982408. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:22,363][19170] Avg episode reward: [(0, '951.250'), (1, '951.440')] [2023-12-27 17:59:22,365][20381] Updated weights for policy 1, policy_version 1780202 (0.0008) [2023-12-27 17:59:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001781824_456228864.pth... [2023-12-27 17:59:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001780640_455925760.pth [2023-12-27 17:59:22,421][20347] Updated weights for policy 0, policy_version 1781827 (0.0008) [2023-12-27 17:59:22,443][20381] Updated weights for policy 1, policy_version 1780212 (0.0009) [2023-12-27 17:59:22,479][20347] Updated weights for policy 0, policy_version 1781837 (0.0006) [2023-12-27 17:59:22,506][20381] Updated weights for policy 1, policy_version 1780222 (0.0007) [2023-12-27 17:59:22,517][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001780224_455786496.pth... [2023-12-27 17:59:22,521][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001779008_455475200.pth [2023-12-27 17:59:22,542][20347] Updated weights for policy 0, policy_version 1781847 (0.0006) [2023-12-27 17:59:23,116][20347] Updated weights for policy 0, policy_version 1781857 (0.0006) [2023-12-27 17:59:23,172][20347] Updated weights for policy 0, policy_version 1781867 (0.0007) [2023-12-27 17:59:23,212][20381] Updated weights for policy 1, policy_version 1780232 (0.0007) [2023-12-27 17:59:23,221][20347] Updated weights for policy 0, policy_version 1781877 (0.0006) [2023-12-27 17:59:23,269][20381] Updated weights for policy 1, policy_version 1780242 (0.0006) [2023-12-27 17:59:23,272][20347] Updated weights for policy 0, policy_version 1781887 (0.0007) [2023-12-27 17:59:23,322][20381] Updated weights for policy 1, policy_version 1780252 (0.0009) [2023-12-27 17:59:23,974][20381] Updated weights for policy 1, policy_version 1780262 (0.0007) [2023-12-27 17:59:23,994][20347] Updated weights for policy 0, policy_version 1781897 (0.0006) [2023-12-27 17:59:24,020][20381] Updated weights for policy 1, policy_version 1780272 (0.0005) [2023-12-27 17:59:24,040][20347] Updated weights for policy 0, policy_version 1781907 (0.0005) [2023-12-27 17:59:24,073][20381] Updated weights for policy 1, policy_version 1780282 (0.0006) [2023-12-27 17:59:24,086][20347] Updated weights for policy 0, policy_version 1781917 (0.0006) [2023-12-27 17:59:24,675][20381] Updated weights for policy 1, policy_version 1780292 (0.0005) [2023-12-27 17:59:24,739][20381] Updated weights for policy 1, policy_version 1780302 (0.0007) [2023-12-27 17:59:24,780][20347] Updated weights for policy 0, policy_version 1781927 (0.0006) [2023-12-27 17:59:24,798][20381] Updated weights for policy 1, policy_version 1780312 (0.0007) [2023-12-27 17:59:24,839][20347] Updated weights for policy 0, policy_version 1781937 (0.0009) [2023-12-27 17:59:24,894][20347] Updated weights for policy 0, policy_version 1781947 (0.0010) [2023-12-27 17:59:25,381][20381] Updated weights for policy 1, policy_version 1780322 (0.0006) [2023-12-27 17:59:25,425][20381] Updated weights for policy 1, policy_version 1780332 (0.0008) [2023-12-27 17:59:25,462][20347] Updated weights for policy 0, policy_version 1781957 (0.0010) [2023-12-27 17:59:25,482][20381] Updated weights for policy 1, policy_version 1780342 (0.0005) [2023-12-27 17:59:25,518][20347] Updated weights for policy 0, policy_version 1781967 (0.0010) [2023-12-27 17:59:25,532][20381] Updated weights for policy 1, policy_version 1780352 (0.0005) [2023-12-27 17:59:25,580][20347] Updated weights for policy 0, policy_version 1781977 (0.0007) [2023-12-27 17:59:26,143][20381] Updated weights for policy 1, policy_version 1780362 (0.0008) [2023-12-27 17:59:26,211][20381] Updated weights for policy 1, policy_version 1780372 (0.0007) [2023-12-27 17:59:26,232][20347] Updated weights for policy 0, policy_version 1781987 (0.0008) [2023-12-27 17:59:26,267][20381] Updated weights for policy 1, policy_version 1780382 (0.0006) [2023-12-27 17:59:26,287][20347] Updated weights for policy 0, policy_version 1781997 (0.0010) [2023-12-27 17:59:26,348][20347] Updated weights for policy 0, policy_version 1782007 (0.0007) [2023-12-27 17:59:26,964][20347] Updated weights for policy 0, policy_version 1782017 (0.0008) [2023-12-27 17:59:26,998][20381] Updated weights for policy 1, policy_version 1780392 (0.0008) [2023-12-27 17:59:27,029][20347] Updated weights for policy 0, policy_version 1782027 (0.0010) [2023-12-27 17:59:27,060][20381] Updated weights for policy 1, policy_version 1780402 (0.0006) [2023-12-27 17:59:27,086][20347] Updated weights for policy 0, policy_version 1782037 (0.0007) [2023-12-27 17:59:27,113][20381] Updated weights for policy 1, policy_version 1780412 (0.0008) [2023-12-27 17:59:27,137][20347] Updated weights for policy 0, policy_version 1782047 (0.0005) [2023-12-27 17:59:27,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20632.7). Total num frames: 912121856. Throughput: 0: 10305.5, 1: 10429.8. Samples: 912107936. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:27,364][19170] Avg episode reward: [(0, '970.800'), (1, '961.150')] [2023-12-27 17:59:27,762][20347] Updated weights for policy 0, policy_version 1782057 (0.0009) [2023-12-27 17:59:27,813][20347] Updated weights for policy 0, policy_version 1782067 (0.0010) [2023-12-27 17:59:27,822][20381] Updated weights for policy 1, policy_version 1780422 (0.0007) [2023-12-27 17:59:27,864][20381] Updated weights for policy 1, policy_version 1780432 (0.0008) [2023-12-27 17:59:27,866][20347] Updated weights for policy 0, policy_version 1782077 (0.0010) [2023-12-27 17:59:27,919][20381] Updated weights for policy 1, policy_version 1780442 (0.0006) [2023-12-27 17:59:28,515][20347] Updated weights for policy 0, policy_version 1782087 (0.0010) [2023-12-27 17:59:28,576][20347] Updated weights for policy 0, policy_version 1782097 (0.0010) [2023-12-27 17:59:28,636][20347] Updated weights for policy 0, policy_version 1782107 (0.0007) [2023-12-27 17:59:28,674][20381] Updated weights for policy 1, policy_version 1780452 (0.0007) [2023-12-27 17:59:28,725][20381] Updated weights for policy 1, policy_version 1780462 (0.0007) [2023-12-27 17:59:28,780][20381] Updated weights for policy 1, policy_version 1780472 (0.0006) [2023-12-27 17:59:29,278][20347] Updated weights for policy 0, policy_version 1782117 (0.0008) [2023-12-27 17:59:29,340][20347] Updated weights for policy 0, policy_version 1782127 (0.0007) [2023-12-27 17:59:29,405][20347] Updated weights for policy 0, policy_version 1782137 (0.0008) [2023-12-27 17:59:29,438][20381] Updated weights for policy 1, policy_version 1780482 (0.0006) [2023-12-27 17:59:29,496][20381] Updated weights for policy 1, policy_version 1780492 (0.0008) [2023-12-27 17:59:29,555][20381] Updated weights for policy 1, policy_version 1780502 (0.0007) [2023-12-27 17:59:29,619][20381] Updated weights for policy 1, policy_version 1780512 (0.0008) [2023-12-27 17:59:30,107][20347] Updated weights for policy 0, policy_version 1782147 (0.0006) [2023-12-27 17:59:30,166][20347] Updated weights for policy 0, policy_version 1782157 (0.0008) [2023-12-27 17:59:30,223][20347] Updated weights for policy 0, policy_version 1782167 (0.0007) [2023-12-27 17:59:30,296][20381] Updated weights for policy 1, policy_version 1780522 (0.0009) [2023-12-27 17:59:30,345][20381] Updated weights for policy 1, policy_version 1780532 (0.0008) [2023-12-27 17:59:30,402][20381] Updated weights for policy 1, policy_version 1780542 (0.0008) [2023-12-27 17:59:30,925][20347] Updated weights for policy 0, policy_version 1782177 (0.0007) [2023-12-27 17:59:30,972][20381] Updated weights for policy 1, policy_version 1780552 (0.0006) [2023-12-27 17:59:30,987][20347] Updated weights for policy 0, policy_version 1782187 (0.0007) [2023-12-27 17:59:31,034][20381] Updated weights for policy 1, policy_version 1780562 (0.0007) [2023-12-27 17:59:31,048][20347] Updated weights for policy 0, policy_version 1782197 (0.0007) [2023-12-27 17:59:31,097][20381] Updated weights for policy 1, policy_version 1780572 (0.0007) [2023-12-27 17:59:31,107][20347] Updated weights for policy 0, policy_version 1782207 (0.0009) [2023-12-27 17:59:31,784][20381] Updated weights for policy 1, policy_version 1780582 (0.0007) [2023-12-27 17:59:31,826][20347] Updated weights for policy 0, policy_version 1782217 (0.0010) [2023-12-27 17:59:31,843][20381] Updated weights for policy 1, policy_version 1780592 (0.0005) [2023-12-27 17:59:31,885][20347] Updated weights for policy 0, policy_version 1782227 (0.0010) [2023-12-27 17:59:31,901][20381] Updated weights for policy 1, policy_version 1780602 (0.0006) [2023-12-27 17:59:31,944][20347] Updated weights for policy 0, policy_version 1782237 (0.0010) [2023-12-27 17:59:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 912220160. Throughput: 0: 10301.7, 1: 10330.5. Samples: 912231124. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:32,363][19170] Avg episode reward: [(0, '960.900'), (1, '980.520')] [2023-12-27 17:59:32,615][20381] Updated weights for policy 1, policy_version 1780612 (0.0006) [2023-12-27 17:59:32,641][20347] Updated weights for policy 0, policy_version 1782247 (0.0010) [2023-12-27 17:59:32,666][20381] Updated weights for policy 1, policy_version 1780622 (0.0007) [2023-12-27 17:59:32,694][20347] Updated weights for policy 0, policy_version 1782257 (0.0011) [2023-12-27 17:59:32,723][20381] Updated weights for policy 1, policy_version 1780632 (0.0006) [2023-12-27 17:59:32,751][20347] Updated weights for policy 0, policy_version 1782267 (0.0008) [2023-12-27 17:59:33,291][20381] Updated weights for policy 1, policy_version 1780642 (0.0006) [2023-12-27 17:59:33,344][20381] Updated weights for policy 1, policy_version 1780652 (0.0007) [2023-12-27 17:59:33,404][20381] Updated weights for policy 1, policy_version 1780662 (0.0007) [2023-12-27 17:59:33,427][20347] Updated weights for policy 0, policy_version 1782277 (0.0010) [2023-12-27 17:59:33,463][20381] Updated weights for policy 1, policy_version 1780672 (0.0006) [2023-12-27 17:59:33,475][20347] Updated weights for policy 0, policy_version 1782287 (0.0010) [2023-12-27 17:59:33,524][20347] Updated weights for policy 0, policy_version 1782297 (0.0010) [2023-12-27 17:59:34,108][20381] Updated weights for policy 1, policy_version 1780682 (0.0007) [2023-12-27 17:59:34,169][20381] Updated weights for policy 1, policy_version 1780692 (0.0007) [2023-12-27 17:59:34,212][20347] Updated weights for policy 0, policy_version 1782307 (0.0009) [2023-12-27 17:59:34,226][20381] Updated weights for policy 1, policy_version 1780702 (0.0007) [2023-12-27 17:59:34,279][20347] Updated weights for policy 0, policy_version 1782317 (0.0007) [2023-12-27 17:59:34,346][20347] Updated weights for policy 0, policy_version 1782327 (0.0007) [2023-12-27 17:59:34,898][20381] Updated weights for policy 1, policy_version 1780712 (0.0008) [2023-12-27 17:59:34,963][20381] Updated weights for policy 1, policy_version 1780722 (0.0006) [2023-12-27 17:59:35,013][20347] Updated weights for policy 0, policy_version 1782337 (0.0007) [2023-12-27 17:59:35,015][20381] Updated weights for policy 1, policy_version 1780732 (0.0007) [2023-12-27 17:59:35,067][20347] Updated weights for policy 0, policy_version 1782347 (0.0005) [2023-12-27 17:59:35,117][20347] Updated weights for policy 0, policy_version 1782357 (0.0005) [2023-12-27 17:59:35,170][20347] Updated weights for policy 0, policy_version 1782367 (0.0005) [2023-12-27 17:59:35,624][20381] Updated weights for policy 1, policy_version 1780742 (0.0007) [2023-12-27 17:59:35,675][20381] Updated weights for policy 1, policy_version 1780752 (0.0005) [2023-12-27 17:59:35,727][20381] Updated weights for policy 1, policy_version 1780762 (0.0005) [2023-12-27 17:59:35,771][20347] Updated weights for policy 0, policy_version 1782377 (0.0005) [2023-12-27 17:59:35,831][20347] Updated weights for policy 0, policy_version 1782387 (0.0005) [2023-12-27 17:59:35,888][20347] Updated weights for policy 0, policy_version 1782397 (0.0009) [2023-12-27 17:59:36,280][20381] Updated weights for policy 1, policy_version 1780772 (0.0005) [2023-12-27 17:59:36,333][20381] Updated weights for policy 1, policy_version 1780782 (0.0005) [2023-12-27 17:59:36,388][20381] Updated weights for policy 1, policy_version 1780792 (0.0008) [2023-12-27 17:59:36,532][20347] Updated weights for policy 0, policy_version 1782407 (0.0010) [2023-12-27 17:59:36,597][20347] Updated weights for policy 0, policy_version 1782417 (0.0010) [2023-12-27 17:59:36,657][20347] Updated weights for policy 0, policy_version 1782427 (0.0009) [2023-12-27 17:59:36,979][20381] Updated weights for policy 1, policy_version 1780802 (0.0006) [2023-12-27 17:59:37,032][20381] Updated weights for policy 1, policy_version 1780812 (0.0007) [2023-12-27 17:59:37,085][20381] Updated weights for policy 1, policy_version 1780822 (0.0011) [2023-12-27 17:59:37,131][20381] Updated weights for policy 1, policy_version 1780832 (0.0011) [2023-12-27 17:59:37,251][20347] Updated weights for policy 0, policy_version 1782437 (0.0007) [2023-12-27 17:59:37,306][20347] Updated weights for policy 0, policy_version 1782447 (0.0006) [2023-12-27 17:59:37,362][20347] Updated weights for policy 0, policy_version 1782457 (0.0010) [2023-12-27 17:59:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 912326656. Throughput: 0: 10307.6, 1: 10339.3. Samples: 912293452. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:37,364][19170] Avg episode reward: [(0, '951.130'), (1, '990.400')] [2023-12-27 17:59:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001780832_455942144.pth... [2023-12-27 17:59:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001779616_455630848.pth [2023-12-27 17:59:37,401][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001782464_456392704.pth... [2023-12-27 17:59:37,405][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001781248_456081408.pth [2023-12-27 17:59:37,815][20381] Updated weights for policy 1, policy_version 1780842 (0.0011) [2023-12-27 17:59:37,872][20381] Updated weights for policy 1, policy_version 1780852 (0.0010) [2023-12-27 17:59:37,923][20381] Updated weights for policy 1, policy_version 1780862 (0.0005) [2023-12-27 17:59:38,022][20347] Updated weights for policy 0, policy_version 1782467 (0.0010) [2023-12-27 17:59:38,078][20347] Updated weights for policy 0, policy_version 1782477 (0.0010) [2023-12-27 17:59:38,136][20347] Updated weights for policy 0, policy_version 1782487 (0.0010) [2023-12-27 17:59:38,635][20381] Updated weights for policy 1, policy_version 1780872 (0.0009) [2023-12-27 17:59:38,685][20381] Updated weights for policy 1, policy_version 1780882 (0.0011) [2023-12-27 17:59:38,748][20381] Updated weights for policy 1, policy_version 1780892 (0.0011) [2023-12-27 17:59:38,792][20347] Updated weights for policy 0, policy_version 1782497 (0.0010) [2023-12-27 17:59:38,843][20347] Updated weights for policy 0, policy_version 1782507 (0.0010) [2023-12-27 17:59:38,900][20347] Updated weights for policy 0, policy_version 1782517 (0.0010) [2023-12-27 17:59:38,952][20347] Updated weights for policy 0, policy_version 1782527 (0.0010) [2023-12-27 17:59:39,454][20381] Updated weights for policy 1, policy_version 1780902 (0.0009) [2023-12-27 17:59:39,514][20381] Updated weights for policy 1, policy_version 1780912 (0.0007) [2023-12-27 17:59:39,574][20381] Updated weights for policy 1, policy_version 1780922 (0.0008) [2023-12-27 17:59:39,629][20347] Updated weights for policy 0, policy_version 1782537 (0.0008) [2023-12-27 17:59:39,689][20347] Updated weights for policy 0, policy_version 1782547 (0.0007) [2023-12-27 17:59:39,751][20347] Updated weights for policy 0, policy_version 1782557 (0.0007) [2023-12-27 17:59:40,246][20381] Updated weights for policy 1, policy_version 1780932 (0.0008) [2023-12-27 17:59:40,303][20381] Updated weights for policy 1, policy_version 1780942 (0.0008) [2023-12-27 17:59:40,364][20381] Updated weights for policy 1, policy_version 1780952 (0.0009) [2023-12-27 17:59:40,401][20347] Updated weights for policy 0, policy_version 1782567 (0.0006) [2023-12-27 17:59:40,451][20347] Updated weights for policy 0, policy_version 1782577 (0.0005) [2023-12-27 17:59:40,505][20347] Updated weights for policy 0, policy_version 1782587 (0.0008) [2023-12-27 17:59:41,018][20381] Updated weights for policy 1, policy_version 1780962 (0.0008) [2023-12-27 17:59:41,078][20381] Updated weights for policy 1, policy_version 1780972 (0.0008) [2023-12-27 17:59:41,139][20381] Updated weights for policy 1, policy_version 1780982 (0.0011) [2023-12-27 17:59:41,188][20347] Updated weights for policy 0, policy_version 1782597 (0.0008) [2023-12-27 17:59:41,198][20381] Updated weights for policy 1, policy_version 1780992 (0.0008) [2023-12-27 17:59:41,247][20347] Updated weights for policy 0, policy_version 1782607 (0.0008) [2023-12-27 17:59:41,309][20347] Updated weights for policy 0, policy_version 1782617 (0.0007) [2023-12-27 17:59:41,947][20381] Updated weights for policy 1, policy_version 1781002 (0.0008) [2023-12-27 17:59:42,011][20381] Updated weights for policy 1, policy_version 1781012 (0.0008) [2023-12-27 17:59:42,036][20347] Updated weights for policy 0, policy_version 1782627 (0.0008) [2023-12-27 17:59:42,069][20381] Updated weights for policy 1, policy_version 1781022 (0.0006) [2023-12-27 17:59:42,095][20347] Updated weights for policy 0, policy_version 1782637 (0.0007) [2023-12-27 17:59:42,150][20347] Updated weights for policy 0, policy_version 1782647 (0.0008) [2023-12-27 17:59:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 912433152. Throughput: 0: 10349.0, 1: 10373.1. Samples: 912420976. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:42,363][19170] Avg episode reward: [(0, '960.920'), (1, '990.400')] [2023-12-27 17:59:42,816][20381] Updated weights for policy 1, policy_version 1781032 (0.0008) [2023-12-27 17:59:42,821][20347] Updated weights for policy 0, policy_version 1782657 (0.0008) [2023-12-27 17:59:42,872][20381] Updated weights for policy 1, policy_version 1781042 (0.0007) [2023-12-27 17:59:42,877][20347] Updated weights for policy 0, policy_version 1782667 (0.0010) [2023-12-27 17:59:42,924][20381] Updated weights for policy 1, policy_version 1781052 (0.0006) [2023-12-27 17:59:42,936][20347] Updated weights for policy 0, policy_version 1782677 (0.0010) [2023-12-27 17:59:42,992][20347] Updated weights for policy 0, policy_version 1782687 (0.0009) [2023-12-27 17:59:43,560][20347] Updated weights for policy 0, policy_version 1782697 (0.0006) [2023-12-27 17:59:43,619][20347] Updated weights for policy 0, policy_version 1782707 (0.0008) [2023-12-27 17:59:43,642][20381] Updated weights for policy 1, policy_version 1781062 (0.0009) [2023-12-27 17:59:43,673][20347] Updated weights for policy 0, policy_version 1782717 (0.0006) [2023-12-27 17:59:43,698][20381] Updated weights for policy 1, policy_version 1781072 (0.0006) [2023-12-27 17:59:43,754][20381] Updated weights for policy 1, policy_version 1781082 (0.0008) [2023-12-27 17:59:44,354][20347] Updated weights for policy 0, policy_version 1782727 (0.0006) [2023-12-27 17:59:44,403][20347] Updated weights for policy 0, policy_version 1782737 (0.0008) [2023-12-27 17:59:44,449][20347] Updated weights for policy 0, policy_version 1782747 (0.0008) [2023-12-27 17:59:44,458][20381] Updated weights for policy 1, policy_version 1781092 (0.0008) [2023-12-27 17:59:44,509][20381] Updated weights for policy 1, policy_version 1781102 (0.0006) [2023-12-27 17:59:44,562][20381] Updated weights for policy 1, policy_version 1781112 (0.0006) [2023-12-27 17:59:45,079][20347] Updated weights for policy 0, policy_version 1782757 (0.0007) [2023-12-27 17:59:45,139][20347] Updated weights for policy 0, policy_version 1782767 (0.0006) [2023-12-27 17:59:45,195][20347] Updated weights for policy 0, policy_version 1782777 (0.0008) [2023-12-27 17:59:45,230][20381] Updated weights for policy 1, policy_version 1781122 (0.0007) [2023-12-27 17:59:45,289][20381] Updated weights for policy 1, policy_version 1781132 (0.0008) [2023-12-27 17:59:45,344][20381] Updated weights for policy 1, policy_version 1781142 (0.0008) [2023-12-27 17:59:45,404][20381] Updated weights for policy 1, policy_version 1781152 (0.0005) [2023-12-27 17:59:45,908][20347] Updated weights for policy 0, policy_version 1782787 (0.0007) [2023-12-27 17:59:45,963][20347] Updated weights for policy 0, policy_version 1782797 (0.0006) [2023-12-27 17:59:45,983][20381] Updated weights for policy 1, policy_version 1781162 (0.0008) [2023-12-27 17:59:46,021][20347] Updated weights for policy 0, policy_version 1782807 (0.0007) [2023-12-27 17:59:46,038][20381] Updated weights for policy 1, policy_version 1781172 (0.0005) [2023-12-27 17:59:46,092][20381] Updated weights for policy 1, policy_version 1781182 (0.0006) [2023-12-27 17:59:46,725][20347] Updated weights for policy 0, policy_version 1782817 (0.0005) [2023-12-27 17:59:46,772][20381] Updated weights for policy 1, policy_version 1781192 (0.0006) [2023-12-27 17:59:46,773][20347] Updated weights for policy 0, policy_version 1782827 (0.0006) [2023-12-27 17:59:46,821][20347] Updated weights for policy 0, policy_version 1782837 (0.0005) [2023-12-27 17:59:46,827][20381] Updated weights for policy 1, policy_version 1781202 (0.0005) [2023-12-27 17:59:46,889][20347] Updated weights for policy 0, policy_version 1782848 (0.0007) [2023-12-27 17:59:46,890][20381] Updated weights for policy 1, policy_version 1781212 (0.0006) [2023-12-27 17:59:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 912531456. Throughput: 0: 10410.4, 1: 10307.9. Samples: 912543000. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:47,363][19170] Avg episode reward: [(0, '931.650'), (1, '1000.000')] [2023-12-27 17:59:47,563][20347] Updated weights for policy 0, policy_version 1782858 (0.0006) [2023-12-27 17:59:47,598][20381] Updated weights for policy 1, policy_version 1781222 (0.0007) [2023-12-27 17:59:47,614][20347] Updated weights for policy 0, policy_version 1782868 (0.0007) [2023-12-27 17:59:47,647][20381] Updated weights for policy 1, policy_version 1781232 (0.0005) [2023-12-27 17:59:47,660][20347] Updated weights for policy 0, policy_version 1782878 (0.0008) [2023-12-27 17:59:47,700][20381] Updated weights for policy 1, policy_version 1781242 (0.0005) [2023-12-27 17:59:48,319][20347] Updated weights for policy 0, policy_version 1782888 (0.0007) [2023-12-27 17:59:48,345][20381] Updated weights for policy 1, policy_version 1781252 (0.0007) [2023-12-27 17:59:48,380][20347] Updated weights for policy 0, policy_version 1782898 (0.0008) [2023-12-27 17:59:48,416][20381] Updated weights for policy 1, policy_version 1781262 (0.0011) [2023-12-27 17:59:48,438][20347] Updated weights for policy 0, policy_version 1782908 (0.0007) [2023-12-27 17:59:48,487][20381] Updated weights for policy 1, policy_version 1781272 (0.0011) [2023-12-27 17:59:49,065][20381] Updated weights for policy 1, policy_version 1781282 (0.0009) [2023-12-27 17:59:49,118][20381] Updated weights for policy 1, policy_version 1781292 (0.0006) [2023-12-27 17:59:49,123][20347] Updated weights for policy 0, policy_version 1782918 (0.0006) [2023-12-27 17:59:49,169][20381] Updated weights for policy 1, policy_version 1781302 (0.0006) [2023-12-27 17:59:49,174][20347] Updated weights for policy 0, policy_version 1782928 (0.0005) [2023-12-27 17:59:49,228][20347] Updated weights for policy 0, policy_version 1782938 (0.0008) [2023-12-27 17:59:49,229][20381] Updated weights for policy 1, policy_version 1781312 (0.0008) [2023-12-27 17:59:49,918][20347] Updated weights for policy 0, policy_version 1782948 (0.0009) [2023-12-27 17:59:49,985][20347] Updated weights for policy 0, policy_version 1782958 (0.0007) [2023-12-27 17:59:49,988][20381] Updated weights for policy 1, policy_version 1781322 (0.0008) [2023-12-27 17:59:50,039][20347] Updated weights for policy 0, policy_version 1782968 (0.0006) [2023-12-27 17:59:50,041][20381] Updated weights for policy 1, policy_version 1781332 (0.0007) [2023-12-27 17:59:50,101][20381] Updated weights for policy 1, policy_version 1781342 (0.0006) [2023-12-27 17:59:50,704][20347] Updated weights for policy 0, policy_version 1782978 (0.0007) [2023-12-27 17:59:50,759][20347] Updated weights for policy 0, policy_version 1782988 (0.0008) [2023-12-27 17:59:50,809][20381] Updated weights for policy 1, policy_version 1781352 (0.0007) [2023-12-27 17:59:50,822][20347] Updated weights for policy 0, policy_version 1782998 (0.0006) [2023-12-27 17:59:50,869][20381] Updated weights for policy 1, policy_version 1781362 (0.0009) [2023-12-27 17:59:50,886][20347] Updated weights for policy 0, policy_version 1783008 (0.0007) [2023-12-27 17:59:50,925][20381] Updated weights for policy 1, policy_version 1781372 (0.0007) [2023-12-27 17:59:51,601][20347] Updated weights for policy 0, policy_version 1783018 (0.0007) [2023-12-27 17:59:51,657][20381] Updated weights for policy 1, policy_version 1781382 (0.0008) [2023-12-27 17:59:51,669][20347] Updated weights for policy 0, policy_version 1783028 (0.0008) [2023-12-27 17:59:51,712][20381] Updated weights for policy 1, policy_version 1781392 (0.0007) [2023-12-27 17:59:51,725][20347] Updated weights for policy 0, policy_version 1783038 (0.0006) [2023-12-27 17:59:51,779][20381] Updated weights for policy 1, policy_version 1781402 (0.0008) [2023-12-27 17:59:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 912629760. Throughput: 0: 10370.1, 1: 10323.7. Samples: 912605056. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:52,363][19170] Avg episode reward: [(0, '960.860'), (1, '990.300')] [2023-12-27 17:59:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001781408_456089600.pth... [2023-12-27 17:59:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001780224_455786496.pth [2023-12-27 17:59:52,372][20347] Updated weights for policy 0, policy_version 1783048 (0.0010) [2023-12-27 17:59:52,373][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001781408_456089600.pth [2023-12-27 17:59:52,423][20347] Updated weights for policy 0, policy_version 1783058 (0.0010) [2023-12-27 17:59:52,479][20347] Updated weights for policy 0, policy_version 1783068 (0.0010) [2023-12-27 17:59:52,503][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001783072_456548352.pth... [2023-12-27 17:59:52,507][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001781824_456228864.pth [2023-12-27 17:59:52,508][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001783072_456548352.pth [2023-12-27 17:59:52,565][20381] Updated weights for policy 1, policy_version 1781412 (0.0008) [2023-12-27 17:59:52,621][20381] Updated weights for policy 1, policy_version 1781422 (0.0008) [2023-12-27 17:59:52,677][20381] Updated weights for policy 1, policy_version 1781432 (0.0008) [2023-12-27 17:59:53,151][20347] Updated weights for policy 0, policy_version 1783078 (0.0009) [2023-12-27 17:59:53,199][20347] Updated weights for policy 0, policy_version 1783088 (0.0008) [2023-12-27 17:59:53,253][20347] Updated weights for policy 0, policy_version 1783098 (0.0007) [2023-12-27 17:59:53,351][20381] Updated weights for policy 1, policy_version 1781442 (0.0008) [2023-12-27 17:59:53,414][20381] Updated weights for policy 1, policy_version 1781452 (0.0009) [2023-12-27 17:59:53,477][20381] Updated weights for policy 1, policy_version 1781462 (0.0008) [2023-12-27 17:59:53,538][20381] Updated weights for policy 1, policy_version 1781472 (0.0007) [2023-12-27 17:59:53,934][20347] Updated weights for policy 0, policy_version 1783108 (0.0009) [2023-12-27 17:59:53,984][20347] Updated weights for policy 0, policy_version 1783118 (0.0010) [2023-12-27 17:59:54,043][20347] Updated weights for policy 0, policy_version 1783128 (0.0010) [2023-12-27 17:59:54,163][20381] Updated weights for policy 1, policy_version 1781482 (0.0011) [2023-12-27 17:59:54,223][20381] Updated weights for policy 1, policy_version 1781492 (0.0010) [2023-12-27 17:59:54,273][20381] Updated weights for policy 1, policy_version 1781502 (0.0005) [2023-12-27 17:59:54,746][20347] Updated weights for policy 0, policy_version 1783138 (0.0009) [2023-12-27 17:59:54,808][20347] Updated weights for policy 0, policy_version 1783148 (0.0005) [2023-12-27 17:59:54,866][20347] Updated weights for policy 0, policy_version 1783158 (0.0005) [2023-12-27 17:59:54,920][20347] Updated weights for policy 0, policy_version 1783168 (0.0005) [2023-12-27 17:59:54,946][20381] Updated weights for policy 1, policy_version 1781512 (0.0008) [2023-12-27 17:59:55,003][20381] Updated weights for policy 1, policy_version 1781522 (0.0006) [2023-12-27 17:59:55,066][20381] Updated weights for policy 1, policy_version 1781532 (0.0006) [2023-12-27 17:59:55,547][20347] Updated weights for policy 0, policy_version 1783178 (0.0010) [2023-12-27 17:59:55,605][20347] Updated weights for policy 0, policy_version 1783188 (0.0010) [2023-12-27 17:59:55,656][20347] Updated weights for policy 0, policy_version 1783198 (0.0010) [2023-12-27 17:59:55,688][20381] Updated weights for policy 1, policy_version 1781542 (0.0006) [2023-12-27 17:59:55,736][20381] Updated weights for policy 1, policy_version 1781552 (0.0005) [2023-12-27 17:59:55,791][20381] Updated weights for policy 1, policy_version 1781562 (0.0005) [2023-12-27 17:59:56,307][20347] Updated weights for policy 0, policy_version 1783208 (0.0006) [2023-12-27 17:59:56,356][20347] Updated weights for policy 0, policy_version 1783218 (0.0005) [2023-12-27 17:59:56,408][20347] Updated weights for policy 0, policy_version 1783228 (0.0007) [2023-12-27 17:59:56,426][20381] Updated weights for policy 1, policy_version 1781572 (0.0007) [2023-12-27 17:59:56,488][20381] Updated weights for policy 1, policy_version 1781582 (0.0011) [2023-12-27 17:59:56,544][20381] Updated weights for policy 1, policy_version 1781592 (0.0011) [2023-12-27 17:59:57,040][20347] Updated weights for policy 0, policy_version 1783238 (0.0006) [2023-12-27 17:59:57,095][20347] Updated weights for policy 0, policy_version 1783248 (0.0007) [2023-12-27 17:59:57,147][20347] Updated weights for policy 0, policy_version 1783258 (0.0008) [2023-12-27 17:59:57,163][20381] Updated weights for policy 1, policy_version 1781602 (0.0010) [2023-12-27 17:59:57,223][20381] Updated weights for policy 1, policy_version 1781612 (0.0011) [2023-12-27 17:59:57,284][20381] Updated weights for policy 1, policy_version 1781622 (0.0011) [2023-12-27 17:59:57,340][20381] Updated weights for policy 1, policy_version 1781632 (0.0011) [2023-12-27 17:59:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 912744448. Throughput: 0: 10360.9, 1: 10295.0. Samples: 912727548. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 17:59:57,363][19170] Avg episode reward: [(0, '960.890'), (1, '980.550')] [2023-12-27 17:59:57,847][20347] Updated weights for policy 0, policy_version 1783268 (0.0007) [2023-12-27 17:59:57,899][20347] Updated weights for policy 0, policy_version 1783278 (0.0007) [2023-12-27 17:59:57,946][20381] Updated weights for policy 1, policy_version 1781642 (0.0008) [2023-12-27 17:59:57,958][20347] Updated weights for policy 0, policy_version 1783288 (0.0005) [2023-12-27 17:59:58,004][20381] Updated weights for policy 1, policy_version 1781652 (0.0008) [2023-12-27 17:59:58,061][20381] Updated weights for policy 1, policy_version 1781662 (0.0009) [2023-12-27 17:59:58,661][20347] Updated weights for policy 0, policy_version 1783298 (0.0006) [2023-12-27 17:59:58,735][20347] Updated weights for policy 0, policy_version 1783308 (0.0007) [2023-12-27 17:59:58,805][20347] Updated weights for policy 0, policy_version 1783318 (0.0008) [2023-12-27 17:59:58,819][20381] Updated weights for policy 1, policy_version 1781672 (0.0009) [2023-12-27 17:59:58,864][20347] Updated weights for policy 0, policy_version 1783328 (0.0007) [2023-12-27 17:59:58,874][20381] Updated weights for policy 1, policy_version 1781682 (0.0007) [2023-12-27 17:59:58,935][20381] Updated weights for policy 1, policy_version 1781692 (0.0008) [2023-12-27 17:59:59,572][20347] Updated weights for policy 0, policy_version 1783338 (0.0005) [2023-12-27 17:59:59,616][20381] Updated weights for policy 1, policy_version 1781702 (0.0008) [2023-12-27 17:59:59,629][20347] Updated weights for policy 0, policy_version 1783348 (0.0006) [2023-12-27 17:59:59,667][20381] Updated weights for policy 1, policy_version 1781712 (0.0010) [2023-12-27 17:59:59,685][20347] Updated weights for policy 0, policy_version 1783358 (0.0006) [2023-12-27 17:59:59,725][20381] Updated weights for policy 1, policy_version 1781722 (0.0010) [2023-12-27 18:00:00,373][20347] Updated weights for policy 0, policy_version 1783368 (0.0006) [2023-12-27 18:00:00,425][20347] Updated weights for policy 0, policy_version 1783378 (0.0006) [2023-12-27 18:00:00,428][20381] Updated weights for policy 1, policy_version 1781732 (0.0011) [2023-12-27 18:00:00,479][20347] Updated weights for policy 0, policy_version 1783388 (0.0005) [2023-12-27 18:00:00,489][20381] Updated weights for policy 1, policy_version 1781742 (0.0011) [2023-12-27 18:00:00,544][20381] Updated weights for policy 1, policy_version 1781752 (0.0011) [2023-12-27 18:00:01,078][20347] Updated weights for policy 0, policy_version 1783398 (0.0008) [2023-12-27 18:00:01,143][20347] Updated weights for policy 0, policy_version 1783408 (0.0008) [2023-12-27 18:00:01,196][20381] Updated weights for policy 1, policy_version 1781762 (0.0009) [2023-12-27 18:00:01,203][20347] Updated weights for policy 0, policy_version 1783418 (0.0006) [2023-12-27 18:00:01,254][20381] Updated weights for policy 1, policy_version 1781772 (0.0006) [2023-12-27 18:00:01,309][20381] Updated weights for policy 1, policy_version 1781782 (0.0006) [2023-12-27 18:00:01,373][20381] Updated weights for policy 1, policy_version 1781792 (0.0007) [2023-12-27 18:00:01,855][20347] Updated weights for policy 0, policy_version 1783428 (0.0008) [2023-12-27 18:00:01,906][20347] Updated weights for policy 0, policy_version 1783438 (0.0007) [2023-12-27 18:00:01,955][20347] Updated weights for policy 0, policy_version 1783448 (0.0005) [2023-12-27 18:00:02,003][20381] Updated weights for policy 1, policy_version 1781802 (0.0007) [2023-12-27 18:00:02,052][20381] Updated weights for policy 1, policy_version 1781812 (0.0011) [2023-12-27 18:00:02,108][20381] Updated weights for policy 1, policy_version 1781822 (0.0011) [2023-12-27 18:00:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 912842752. Throughput: 0: 10308.7, 1: 10322.5. Samples: 912851016. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:02,363][19170] Avg episode reward: [(0, '941.450'), (1, '990.250')] [2023-12-27 18:00:02,600][20347] Updated weights for policy 0, policy_version 1783458 (0.0008) [2023-12-27 18:00:02,650][20347] Updated weights for policy 0, policy_version 1783468 (0.0007) [2023-12-27 18:00:02,708][20347] Updated weights for policy 0, policy_version 1783478 (0.0007) [2023-12-27 18:00:02,763][20347] Updated weights for policy 0, policy_version 1783488 (0.0008) [2023-12-27 18:00:02,794][20381] Updated weights for policy 1, policy_version 1781832 (0.0011) [2023-12-27 18:00:02,852][20381] Updated weights for policy 1, policy_version 1781842 (0.0010) [2023-12-27 18:00:02,910][20381] Updated weights for policy 1, policy_version 1781852 (0.0010) [2023-12-27 18:00:03,417][20347] Updated weights for policy 0, policy_version 1783498 (0.0007) [2023-12-27 18:00:03,464][20347] Updated weights for policy 0, policy_version 1783508 (0.0006) [2023-12-27 18:00:03,510][20347] Updated weights for policy 0, policy_version 1783518 (0.0006) [2023-12-27 18:00:03,581][20381] Updated weights for policy 1, policy_version 1781862 (0.0010) [2023-12-27 18:00:03,638][20381] Updated weights for policy 1, policy_version 1781872 (0.0010) [2023-12-27 18:00:03,693][20381] Updated weights for policy 1, policy_version 1781882 (0.0010) [2023-12-27 18:00:04,083][20347] Updated weights for policy 0, policy_version 1783528 (0.0009) [2023-12-27 18:00:04,151][20347] Updated weights for policy 0, policy_version 1783538 (0.0012) [2023-12-27 18:00:04,209][20347] Updated weights for policy 0, policy_version 1783548 (0.0010) [2023-12-27 18:00:04,387][20381] Updated weights for policy 1, policy_version 1781892 (0.0007) [2023-12-27 18:00:04,446][20381] Updated weights for policy 1, policy_version 1781902 (0.0007) [2023-12-27 18:00:04,507][20381] Updated weights for policy 1, policy_version 1781912 (0.0008) [2023-12-27 18:00:04,946][20347] Updated weights for policy 0, policy_version 1783558 (0.0010) [2023-12-27 18:00:04,997][20347] Updated weights for policy 0, policy_version 1783568 (0.0010) [2023-12-27 18:00:05,044][20347] Updated weights for policy 0, policy_version 1783578 (0.0010) [2023-12-27 18:00:05,140][20381] Updated weights for policy 1, policy_version 1781922 (0.0007) [2023-12-27 18:00:05,189][20381] Updated weights for policy 1, policy_version 1781932 (0.0007) [2023-12-27 18:00:05,239][20381] Updated weights for policy 1, policy_version 1781942 (0.0008) [2023-12-27 18:00:05,289][20381] Updated weights for policy 1, policy_version 1781952 (0.0008) [2023-12-27 18:00:05,695][20347] Updated weights for policy 0, policy_version 1783588 (0.0008) [2023-12-27 18:00:05,759][20347] Updated weights for policy 0, policy_version 1783598 (0.0005) [2023-12-27 18:00:05,820][20347] Updated weights for policy 0, policy_version 1783608 (0.0005) [2023-12-27 18:00:05,910][20381] Updated weights for policy 1, policy_version 1781962 (0.0006) [2023-12-27 18:00:05,960][20381] Updated weights for policy 1, policy_version 1781972 (0.0006) [2023-12-27 18:00:06,012][20381] Updated weights for policy 1, policy_version 1781982 (0.0009) [2023-12-27 18:00:06,401][20347] Updated weights for policy 0, policy_version 1783618 (0.0008) [2023-12-27 18:00:06,457][20347] Updated weights for policy 0, policy_version 1783628 (0.0010) [2023-12-27 18:00:06,515][20347] Updated weights for policy 0, policy_version 1783638 (0.0010) [2023-12-27 18:00:06,573][20347] Updated weights for policy 0, policy_version 1783648 (0.0010) [2023-12-27 18:00:06,725][20381] Updated weights for policy 1, policy_version 1781992 (0.0008) [2023-12-27 18:00:06,786][20381] Updated weights for policy 1, policy_version 1782002 (0.0007) [2023-12-27 18:00:06,851][20381] Updated weights for policy 1, policy_version 1782012 (0.0007) [2023-12-27 18:00:07,260][20347] Updated weights for policy 0, policy_version 1783658 (0.0010) [2023-12-27 18:00:07,313][20347] Updated weights for policy 0, policy_version 1783668 (0.0010) [2023-12-27 18:00:07,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 912941056. Throughput: 0: 10365.4, 1: 10335.7. Samples: 912913960. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:07,363][19170] Avg episode reward: [(0, '960.980'), (1, '1000.000')] [2023-12-27 18:00:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001782016_456245248.pth... [2023-12-27 18:00:07,370][20347] Updated weights for policy 0, policy_version 1783678 (0.0010) [2023-12-27 18:00:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001780832_455942144.pth [2023-12-27 18:00:07,381][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001783680_456704000.pth... [2023-12-27 18:00:07,385][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001782464_456392704.pth [2023-12-27 18:00:07,525][20381] Updated weights for policy 1, policy_version 1782022 (0.0008) [2023-12-27 18:00:07,583][20381] Updated weights for policy 1, policy_version 1782032 (0.0005) [2023-12-27 18:00:07,630][20381] Updated weights for policy 1, policy_version 1782042 (0.0007) [2023-12-27 18:00:08,039][20347] Updated weights for policy 0, policy_version 1783688 (0.0008) [2023-12-27 18:00:08,094][20347] Updated weights for policy 0, policy_version 1783698 (0.0006) [2023-12-27 18:00:08,153][20347] Updated weights for policy 0, policy_version 1783708 (0.0008) [2023-12-27 18:00:08,242][20381] Updated weights for policy 1, policy_version 1782052 (0.0010) [2023-12-27 18:00:08,291][20381] Updated weights for policy 1, policy_version 1782062 (0.0010) [2023-12-27 18:00:08,345][20381] Updated weights for policy 1, policy_version 1782072 (0.0009) [2023-12-27 18:00:08,863][20347] Updated weights for policy 0, policy_version 1783718 (0.0008) [2023-12-27 18:00:08,914][20347] Updated weights for policy 0, policy_version 1783728 (0.0010) [2023-12-27 18:00:08,932][20381] Updated weights for policy 1, policy_version 1782082 (0.0010) [2023-12-27 18:00:08,968][20347] Updated weights for policy 0, policy_version 1783738 (0.0007) [2023-12-27 18:00:08,983][20381] Updated weights for policy 1, policy_version 1782092 (0.0008) [2023-12-27 18:00:09,037][20381] Updated weights for policy 1, policy_version 1782102 (0.0006) [2023-12-27 18:00:09,094][20381] Updated weights for policy 1, policy_version 1782112 (0.0007) [2023-12-27 18:00:09,746][20347] Updated weights for policy 0, policy_version 1783748 (0.0008) [2023-12-27 18:00:09,803][20347] Updated weights for policy 0, policy_version 1783758 (0.0007) [2023-12-27 18:00:09,810][20381] Updated weights for policy 1, policy_version 1782122 (0.0006) [2023-12-27 18:00:09,872][20347] Updated weights for policy 0, policy_version 1783768 (0.0007) [2023-12-27 18:00:09,874][20381] Updated weights for policy 1, policy_version 1782132 (0.0007) [2023-12-27 18:00:09,934][20381] Updated weights for policy 1, policy_version 1782142 (0.0007) [2023-12-27 18:00:10,540][20347] Updated weights for policy 0, policy_version 1783778 (0.0007) [2023-12-27 18:00:10,595][20347] Updated weights for policy 0, policy_version 1783788 (0.0007) [2023-12-27 18:00:10,651][20347] Updated weights for policy 0, policy_version 1783798 (0.0007) [2023-12-27 18:00:10,672][20381] Updated weights for policy 1, policy_version 1782152 (0.0010) [2023-12-27 18:00:10,708][20347] Updated weights for policy 0, policy_version 1783808 (0.0007) [2023-12-27 18:00:10,723][20381] Updated weights for policy 1, policy_version 1782162 (0.0009) [2023-12-27 18:00:10,782][20381] Updated weights for policy 1, policy_version 1782172 (0.0007) [2023-12-27 18:00:11,456][20347] Updated weights for policy 0, policy_version 1783818 (0.0008) [2023-12-27 18:00:11,519][20347] Updated weights for policy 0, policy_version 1783828 (0.0006) [2023-12-27 18:00:11,529][20381] Updated weights for policy 1, policy_version 1782182 (0.0011) [2023-12-27 18:00:11,577][20347] Updated weights for policy 0, policy_version 1783838 (0.0006) [2023-12-27 18:00:11,587][20381] Updated weights for policy 1, policy_version 1782192 (0.0011) [2023-12-27 18:00:11,653][20381] Updated weights for policy 1, policy_version 1782202 (0.0009) [2023-12-27 18:00:12,303][20347] Updated weights for policy 0, policy_version 1783848 (0.0007) [2023-12-27 18:00:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 913039360. Throughput: 0: 10347.2, 1: 10304.5. Samples: 913037260. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:12,363][19170] Avg episode reward: [(0, '980.510'), (1, '990.310')] [2023-12-27 18:00:12,365][20347] Updated weights for policy 0, policy_version 1783858 (0.0008) [2023-12-27 18:00:12,403][20381] Updated weights for policy 1, policy_version 1782212 (0.0011) [2023-12-27 18:00:12,424][20347] Updated weights for policy 0, policy_version 1783868 (0.0006) [2023-12-27 18:00:12,459][20381] Updated weights for policy 1, policy_version 1782222 (0.0010) [2023-12-27 18:00:12,511][20381] Updated weights for policy 1, policy_version 1782232 (0.0011) [2023-12-27 18:00:13,122][20347] Updated weights for policy 0, policy_version 1783878 (0.0005) [2023-12-27 18:00:13,129][20381] Updated weights for policy 1, policy_version 1782242 (0.0010) [2023-12-27 18:00:13,178][20347] Updated weights for policy 0, policy_version 1783888 (0.0006) [2023-12-27 18:00:13,180][20381] Updated weights for policy 1, policy_version 1782252 (0.0007) [2023-12-27 18:00:13,233][20347] Updated weights for policy 0, policy_version 1783898 (0.0007) [2023-12-27 18:00:13,235][20381] Updated weights for policy 1, policy_version 1782262 (0.0006) [2023-12-27 18:00:13,291][20381] Updated weights for policy 1, policy_version 1782272 (0.0007) [2023-12-27 18:00:13,869][20347] Updated weights for policy 0, policy_version 1783908 (0.0007) [2023-12-27 18:00:13,926][20347] Updated weights for policy 0, policy_version 1783918 (0.0007) [2023-12-27 18:00:13,971][20381] Updated weights for policy 1, policy_version 1782282 (0.0006) [2023-12-27 18:00:13,985][20347] Updated weights for policy 0, policy_version 1783928 (0.0005) [2023-12-27 18:00:14,027][20381] Updated weights for policy 1, policy_version 1782292 (0.0008) [2023-12-27 18:00:14,093][20381] Updated weights for policy 1, policy_version 1782302 (0.0006) [2023-12-27 18:00:14,605][20347] Updated weights for policy 0, policy_version 1783938 (0.0006) [2023-12-27 18:00:14,664][20347] Updated weights for policy 0, policy_version 1783948 (0.0006) [2023-12-27 18:00:14,710][20381] Updated weights for policy 1, policy_version 1782312 (0.0006) [2023-12-27 18:00:14,720][20347] Updated weights for policy 0, policy_version 1783958 (0.0005) [2023-12-27 18:00:14,773][20381] Updated weights for policy 1, policy_version 1782322 (0.0008) [2023-12-27 18:00:14,784][20347] Updated weights for policy 0, policy_version 1783968 (0.0006) [2023-12-27 18:00:14,832][20381] Updated weights for policy 1, policy_version 1782332 (0.0011) [2023-12-27 18:00:15,425][20381] Updated weights for policy 1, policy_version 1782342 (0.0006) [2023-12-27 18:00:15,473][20381] Updated weights for policy 1, policy_version 1782352 (0.0005) [2023-12-27 18:00:15,523][20347] Updated weights for policy 0, policy_version 1783978 (0.0006) [2023-12-27 18:00:15,523][20381] Updated weights for policy 1, policy_version 1782362 (0.0007) [2023-12-27 18:00:15,577][20347] Updated weights for policy 0, policy_version 1783988 (0.0008) [2023-12-27 18:00:15,630][20347] Updated weights for policy 0, policy_version 1783998 (0.0009) [2023-12-27 18:00:16,191][20381] Updated weights for policy 1, policy_version 1782372 (0.0006) [2023-12-27 18:00:16,241][20347] Updated weights for policy 0, policy_version 1784008 (0.0006) [2023-12-27 18:00:16,253][20381] Updated weights for policy 1, policy_version 1782382 (0.0005) [2023-12-27 18:00:16,302][20347] Updated weights for policy 0, policy_version 1784018 (0.0006) [2023-12-27 18:00:16,303][20381] Updated weights for policy 1, policy_version 1782392 (0.0005) [2023-12-27 18:00:16,355][20347] Updated weights for policy 0, policy_version 1784028 (0.0008) [2023-12-27 18:00:16,819][20381] Updated weights for policy 1, policy_version 1782402 (0.0006) [2023-12-27 18:00:16,873][20381] Updated weights for policy 1, policy_version 1782412 (0.0008) [2023-12-27 18:00:16,933][20381] Updated weights for policy 1, policy_version 1782422 (0.0007) [2023-12-27 18:00:16,993][20381] Updated weights for policy 1, policy_version 1782432 (0.0007) [2023-12-27 18:00:17,073][20347] Updated weights for policy 0, policy_version 1784038 (0.0009) [2023-12-27 18:00:17,130][20347] Updated weights for policy 0, policy_version 1784048 (0.0010) [2023-12-27 18:00:17,182][20347] Updated weights for policy 0, policy_version 1784058 (0.0010) [2023-12-27 18:00:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20688.3). Total num frames: 913154048. Throughput: 0: 10315.0, 1: 10379.8. Samples: 913162392. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:17,363][19170] Avg episode reward: [(0, '970.730'), (1, '990.310')] [2023-12-27 18:00:17,628][20381] Updated weights for policy 1, policy_version 1782442 (0.0005) [2023-12-27 18:00:17,689][20381] Updated weights for policy 1, policy_version 1782452 (0.0010) [2023-12-27 18:00:17,737][20381] Updated weights for policy 1, policy_version 1782462 (0.0010) [2023-12-27 18:00:17,868][20347] Updated weights for policy 0, policy_version 1784068 (0.0010) [2023-12-27 18:00:17,929][20347] Updated weights for policy 0, policy_version 1784078 (0.0010) [2023-12-27 18:00:18,012][20347] Updated weights for policy 0, policy_version 1784088 (0.0011) [2023-12-27 18:00:18,367][20381] Updated weights for policy 1, policy_version 1782472 (0.0009) [2023-12-27 18:00:18,427][20381] Updated weights for policy 1, policy_version 1782482 (0.0008) [2023-12-27 18:00:18,483][20381] Updated weights for policy 1, policy_version 1782492 (0.0007) [2023-12-27 18:00:18,601][20347] Updated weights for policy 0, policy_version 1784098 (0.0009) [2023-12-27 18:00:18,664][20347] Updated weights for policy 0, policy_version 1784108 (0.0007) [2023-12-27 18:00:18,717][20347] Updated weights for policy 0, policy_version 1784118 (0.0010) [2023-12-27 18:00:18,770][20347] Updated weights for policy 0, policy_version 1784128 (0.0008) [2023-12-27 18:00:19,130][20381] Updated weights for policy 1, policy_version 1782502 (0.0008) [2023-12-27 18:00:19,180][20381] Updated weights for policy 1, policy_version 1782512 (0.0007) [2023-12-27 18:00:19,238][20381] Updated weights for policy 1, policy_version 1782522 (0.0007) [2023-12-27 18:00:19,504][20347] Updated weights for policy 0, policy_version 1784138 (0.0007) [2023-12-27 18:00:19,566][20347] Updated weights for policy 0, policy_version 1784148 (0.0008) [2023-12-27 18:00:19,630][20347] Updated weights for policy 0, policy_version 1784158 (0.0008) [2023-12-27 18:00:19,955][20381] Updated weights for policy 1, policy_version 1782532 (0.0008) [2023-12-27 18:00:20,015][20381] Updated weights for policy 1, policy_version 1782542 (0.0008) [2023-12-27 18:00:20,081][20381] Updated weights for policy 1, policy_version 1782552 (0.0008) [2023-12-27 18:00:20,319][20347] Updated weights for policy 0, policy_version 1784168 (0.0006) [2023-12-27 18:00:20,381][20347] Updated weights for policy 0, policy_version 1784178 (0.0006) [2023-12-27 18:00:20,441][20347] Updated weights for policy 0, policy_version 1784188 (0.0007) [2023-12-27 18:00:20,810][20381] Updated weights for policy 1, policy_version 1782562 (0.0007) [2023-12-27 18:00:20,873][20381] Updated weights for policy 1, policy_version 1782572 (0.0008) [2023-12-27 18:00:20,935][20381] Updated weights for policy 1, policy_version 1782582 (0.0008) [2023-12-27 18:00:21,000][20381] Updated weights for policy 1, policy_version 1782592 (0.0008) [2023-12-27 18:00:21,094][20347] Updated weights for policy 0, policy_version 1784198 (0.0009) [2023-12-27 18:00:21,159][20347] Updated weights for policy 0, policy_version 1784208 (0.0009) [2023-12-27 18:00:21,218][20347] Updated weights for policy 0, policy_version 1784218 (0.0008) [2023-12-27 18:00:21,746][20381] Updated weights for policy 1, policy_version 1782602 (0.0008) [2023-12-27 18:00:21,806][20381] Updated weights for policy 1, policy_version 1782612 (0.0008) [2023-12-27 18:00:21,855][20381] Updated weights for policy 1, policy_version 1782622 (0.0009) [2023-12-27 18:00:21,921][20347] Updated weights for policy 0, policy_version 1784228 (0.0008) [2023-12-27 18:00:21,985][20347] Updated weights for policy 0, policy_version 1784238 (0.0011) [2023-12-27 18:00:22,040][20347] Updated weights for policy 0, policy_version 1784248 (0.0011) [2023-12-27 18:00:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 913252352. Throughput: 0: 10304.1, 1: 10365.5. Samples: 913223580. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:22,363][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 18:00:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001782624_456400896.pth... [2023-12-27 18:00:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001784256_456851456.pth... [2023-12-27 18:00:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001781408_456089600.pth [2023-12-27 18:00:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001783072_456548352.pth [2023-12-27 18:00:22,649][20381] Updated weights for policy 1, policy_version 1782632 (0.0007) [2023-12-27 18:00:22,711][20381] Updated weights for policy 1, policy_version 1782642 (0.0007) [2023-12-27 18:00:22,756][20347] Updated weights for policy 0, policy_version 1784258 (0.0010) [2023-12-27 18:00:22,769][20381] Updated weights for policy 1, policy_version 1782652 (0.0007) [2023-12-27 18:00:22,815][20347] Updated weights for policy 0, policy_version 1784268 (0.0007) [2023-12-27 18:00:22,875][20347] Updated weights for policy 0, policy_version 1784278 (0.0006) [2023-12-27 18:00:22,923][20347] Updated weights for policy 0, policy_version 1784288 (0.0008) [2023-12-27 18:00:23,477][20381] Updated weights for policy 1, policy_version 1782662 (0.0007) [2023-12-27 18:00:23,531][20381] Updated weights for policy 1, policy_version 1782672 (0.0008) [2023-12-27 18:00:23,573][20347] Updated weights for policy 0, policy_version 1784298 (0.0008) [2023-12-27 18:00:23,582][20381] Updated weights for policy 1, policy_version 1782682 (0.0008) [2023-12-27 18:00:23,622][20347] Updated weights for policy 0, policy_version 1784308 (0.0007) [2023-12-27 18:00:23,674][20347] Updated weights for policy 0, policy_version 1784318 (0.0008) [2023-12-27 18:00:24,167][20381] Updated weights for policy 1, policy_version 1782692 (0.0006) [2023-12-27 18:00:24,231][20381] Updated weights for policy 1, policy_version 1782702 (0.0005) [2023-12-27 18:00:24,300][20381] Updated weights for policy 1, policy_version 1782712 (0.0005) [2023-12-27 18:00:24,341][20347] Updated weights for policy 0, policy_version 1784328 (0.0009) [2023-12-27 18:00:24,400][20347] Updated weights for policy 0, policy_version 1784338 (0.0008) [2023-12-27 18:00:24,455][20347] Updated weights for policy 0, policy_version 1784348 (0.0007) [2023-12-27 18:00:24,822][20381] Updated weights for policy 1, policy_version 1782722 (0.0005) [2023-12-27 18:00:24,885][20381] Updated weights for policy 1, policy_version 1782732 (0.0006) [2023-12-27 18:00:24,948][20381] Updated weights for policy 1, policy_version 1782742 (0.0005) [2023-12-27 18:00:25,009][20381] Updated weights for policy 1, policy_version 1782752 (0.0010) [2023-12-27 18:00:25,152][20347] Updated weights for policy 0, policy_version 1784358 (0.0006) [2023-12-27 18:00:25,217][20347] Updated weights for policy 0, policy_version 1784368 (0.0006) [2023-12-27 18:00:25,271][20347] Updated weights for policy 0, policy_version 1784378 (0.0007) [2023-12-27 18:00:25,625][20381] Updated weights for policy 1, policy_version 1782762 (0.0005) [2023-12-27 18:00:25,680][20381] Updated weights for policy 1, policy_version 1782772 (0.0005) [2023-12-27 18:00:25,739][20381] Updated weights for policy 1, policy_version 1782782 (0.0005) [2023-12-27 18:00:25,960][20347] Updated weights for policy 0, policy_version 1784388 (0.0006) [2023-12-27 18:00:26,009][20347] Updated weights for policy 0, policy_version 1784398 (0.0005) [2023-12-27 18:00:26,041][20291] KL-divergence is very high: 110.9217 [2023-12-27 18:00:26,070][20347] Updated weights for policy 0, policy_version 1784408 (0.0008) [2023-12-27 18:00:26,090][20291] KL-divergence is very high: 142.8872 [2023-12-27 18:00:26,261][20381] Updated weights for policy 1, policy_version 1782792 (0.0009) [2023-12-27 18:00:26,318][20381] Updated weights for policy 1, policy_version 1782802 (0.0010) [2023-12-27 18:00:26,369][20381] Updated weights for policy 1, policy_version 1782812 (0.0010) [2023-12-27 18:00:26,718][20347] Updated weights for policy 0, policy_version 1784418 (0.0007) [2023-12-27 18:00:26,764][20347] Updated weights for policy 0, policy_version 1784428 (0.0009) [2023-12-27 18:00:26,809][20347] Updated weights for policy 0, policy_version 1784438 (0.0009) [2023-12-27 18:00:26,867][20347] Updated weights for policy 0, policy_version 1784448 (0.0006) [2023-12-27 18:00:27,056][20381] Updated weights for policy 1, policy_version 1782822 (0.0011) [2023-12-27 18:00:27,113][20381] Updated weights for policy 1, policy_version 1782832 (0.0011) [2023-12-27 18:00:27,174][20381] Updated weights for policy 1, policy_version 1782842 (0.0010) [2023-12-27 18:00:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 913358848. Throughput: 0: 10232.7, 1: 10353.9. Samples: 913347372. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:27,363][19170] Avg episode reward: [(0, '980.440'), (1, '990.280')] [2023-12-27 18:00:27,570][20347] Updated weights for policy 0, policy_version 1784458 (0.0010) [2023-12-27 18:00:27,631][20347] Updated weights for policy 0, policy_version 1784468 (0.0010) [2023-12-27 18:00:27,688][20347] Updated weights for policy 0, policy_version 1784478 (0.0010) [2023-12-27 18:00:27,864][20381] Updated weights for policy 1, policy_version 1782852 (0.0009) [2023-12-27 18:00:27,921][20381] Updated weights for policy 1, policy_version 1782862 (0.0010) [2023-12-27 18:00:27,975][20381] Updated weights for policy 1, policy_version 1782872 (0.0010) [2023-12-27 18:00:28,372][20347] Updated weights for policy 0, policy_version 1784488 (0.0008) [2023-12-27 18:00:28,428][20347] Updated weights for policy 0, policy_version 1784498 (0.0009) [2023-12-27 18:00:28,488][20347] Updated weights for policy 0, policy_version 1784508 (0.0011) [2023-12-27 18:00:28,656][20381] Updated weights for policy 1, policy_version 1782882 (0.0009) [2023-12-27 18:00:28,711][20381] Updated weights for policy 1, policy_version 1782892 (0.0005) [2023-12-27 18:00:28,766][20381] Updated weights for policy 1, policy_version 1782902 (0.0008) [2023-12-27 18:00:28,821][20381] Updated weights for policy 1, policy_version 1782912 (0.0007) [2023-12-27 18:00:29,146][20347] Updated weights for policy 0, policy_version 1784518 (0.0007) [2023-12-27 18:00:29,194][20347] Updated weights for policy 0, policy_version 1784528 (0.0005) [2023-12-27 18:00:29,258][20347] Updated weights for policy 0, policy_version 1784538 (0.0010) [2023-12-27 18:00:29,467][20381] Updated weights for policy 1, policy_version 1782922 (0.0005) [2023-12-27 18:00:29,530][20381] Updated weights for policy 1, policy_version 1782932 (0.0006) [2023-12-27 18:00:29,592][20381] Updated weights for policy 1, policy_version 1782942 (0.0006) [2023-12-27 18:00:29,929][20347] Updated weights for policy 0, policy_version 1784548 (0.0008) [2023-12-27 18:00:29,991][20347] Updated weights for policy 0, policy_version 1784558 (0.0008) [2023-12-27 18:00:30,047][20347] Updated weights for policy 0, policy_version 1784568 (0.0008) [2023-12-27 18:00:30,246][20381] Updated weights for policy 1, policy_version 1782952 (0.0008) [2023-12-27 18:00:30,305][20381] Updated weights for policy 1, policy_version 1782962 (0.0008) [2023-12-27 18:00:30,364][20381] Updated weights for policy 1, policy_version 1782972 (0.0008) [2023-12-27 18:00:30,695][20347] Updated weights for policy 0, policy_version 1784578 (0.0008) [2023-12-27 18:00:30,747][20347] Updated weights for policy 0, policy_version 1784588 (0.0008) [2023-12-27 18:00:30,802][20347] Updated weights for policy 0, policy_version 1784598 (0.0005) [2023-12-27 18:00:30,859][20347] Updated weights for policy 0, policy_version 1784608 (0.0005) [2023-12-27 18:00:30,963][20381] Updated weights for policy 1, policy_version 1782982 (0.0007) [2023-12-27 18:00:31,011][20381] Updated weights for policy 1, policy_version 1782992 (0.0007) [2023-12-27 18:00:31,068][20381] Updated weights for policy 1, policy_version 1783002 (0.0008) [2023-12-27 18:00:31,536][20347] Updated weights for policy 0, policy_version 1784618 (0.0008) [2023-12-27 18:00:31,585][20347] Updated weights for policy 0, policy_version 1784628 (0.0007) [2023-12-27 18:00:31,644][20347] Updated weights for policy 0, policy_version 1784638 (0.0007) [2023-12-27 18:00:31,815][20381] Updated weights for policy 1, policy_version 1783012 (0.0008) [2023-12-27 18:00:31,879][20381] Updated weights for policy 1, policy_version 1783022 (0.0008) [2023-12-27 18:00:31,934][20381] Updated weights for policy 1, policy_version 1783032 (0.0007) [2023-12-27 18:00:32,356][20347] Updated weights for policy 0, policy_version 1784648 (0.0010) [2023-12-27 18:00:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 913457152. Throughput: 0: 10238.8, 1: 10377.1. Samples: 913470720. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:32,363][19170] Avg episode reward: [(0, '980.440'), (1, '990.280')] [2023-12-27 18:00:32,412][20347] Updated weights for policy 0, policy_version 1784658 (0.0010) [2023-12-27 18:00:32,468][20347] Updated weights for policy 0, policy_version 1784668 (0.0010) [2023-12-27 18:00:32,650][20381] Updated weights for policy 1, policy_version 1783042 (0.0008) [2023-12-27 18:00:32,699][20381] Updated weights for policy 1, policy_version 1783052 (0.0007) [2023-12-27 18:00:32,751][20381] Updated weights for policy 1, policy_version 1783062 (0.0008) [2023-12-27 18:00:32,802][20381] Updated weights for policy 1, policy_version 1783072 (0.0006) [2023-12-27 18:00:33,171][20347] Updated weights for policy 0, policy_version 1784678 (0.0008) [2023-12-27 18:00:33,216][20347] Updated weights for policy 0, policy_version 1784688 (0.0005) [2023-12-27 18:00:33,264][20347] Updated weights for policy 0, policy_version 1784698 (0.0006) [2023-12-27 18:00:33,451][20381] Updated weights for policy 1, policy_version 1783082 (0.0005) [2023-12-27 18:00:33,503][20381] Updated weights for policy 1, policy_version 1783092 (0.0008) [2023-12-27 18:00:33,558][20381] Updated weights for policy 1, policy_version 1783102 (0.0007) [2023-12-27 18:00:33,836][20347] Updated weights for policy 0, policy_version 1784708 (0.0010) [2023-12-27 18:00:33,891][20347] Updated weights for policy 0, policy_version 1784718 (0.0010) [2023-12-27 18:00:33,935][20347] Updated weights for policy 0, policy_version 1784728 (0.0010) [2023-12-27 18:00:34,273][20381] Updated weights for policy 1, policy_version 1783112 (0.0007) [2023-12-27 18:00:34,338][20381] Updated weights for policy 1, policy_version 1783122 (0.0008) [2023-12-27 18:00:34,400][20381] Updated weights for policy 1, policy_version 1783132 (0.0006) [2023-12-27 18:00:34,611][20347] Updated weights for policy 0, policy_version 1784738 (0.0009) [2023-12-27 18:00:34,667][20347] Updated weights for policy 0, policy_version 1784748 (0.0006) [2023-12-27 18:00:34,719][20347] Updated weights for policy 0, policy_version 1784758 (0.0008) [2023-12-27 18:00:34,778][20347] Updated weights for policy 0, policy_version 1784768 (0.0008) [2023-12-27 18:00:35,022][20381] Updated weights for policy 1, policy_version 1783142 (0.0007) [2023-12-27 18:00:35,073][20381] Updated weights for policy 1, policy_version 1783152 (0.0005) [2023-12-27 18:00:35,131][20381] Updated weights for policy 1, policy_version 1783162 (0.0006) [2023-12-27 18:00:35,476][20347] Updated weights for policy 0, policy_version 1784778 (0.0007) [2023-12-27 18:00:35,532][20347] Updated weights for policy 0, policy_version 1784788 (0.0005) [2023-12-27 18:00:35,582][20347] Updated weights for policy 0, policy_version 1784798 (0.0006) [2023-12-27 18:00:35,820][20381] Updated weights for policy 1, policy_version 1783172 (0.0008) [2023-12-27 18:00:35,878][20381] Updated weights for policy 1, policy_version 1783182 (0.0008) [2023-12-27 18:00:35,926][20381] Updated weights for policy 1, policy_version 1783192 (0.0007) [2023-12-27 18:00:36,235][20347] Updated weights for policy 0, policy_version 1784808 (0.0008) [2023-12-27 18:00:36,299][20347] Updated weights for policy 0, policy_version 1784818 (0.0008) [2023-12-27 18:00:36,358][20347] Updated weights for policy 0, policy_version 1784828 (0.0008) [2023-12-27 18:00:36,604][20381] Updated weights for policy 1, policy_version 1783202 (0.0008) [2023-12-27 18:00:36,663][20381] Updated weights for policy 1, policy_version 1783212 (0.0008) [2023-12-27 18:00:36,719][20381] Updated weights for policy 1, policy_version 1783222 (0.0008) [2023-12-27 18:00:36,774][20381] Updated weights for policy 1, policy_version 1783232 (0.0006) [2023-12-27 18:00:37,041][20347] Updated weights for policy 0, policy_version 1784838 (0.0009) [2023-12-27 18:00:37,089][20347] Updated weights for policy 0, policy_version 1784848 (0.0010) [2023-12-27 18:00:37,139][20347] Updated weights for policy 0, policy_version 1784858 (0.0007) [2023-12-27 18:00:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20660.5). Total num frames: 913563648. Throughput: 0: 10249.4, 1: 10382.0. Samples: 913533472. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:37,363][19170] Avg episode reward: [(0, '980.420'), (1, '990.290')] [2023-12-27 18:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001784864_457007104.pth... [2023-12-27 18:00:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001783232_456556544.pth... [2023-12-27 18:00:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001782016_456245248.pth [2023-12-27 18:00:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001783680_456704000.pth [2023-12-27 18:00:37,461][20381] Updated weights for policy 1, policy_version 1783242 (0.0006) [2023-12-27 18:00:37,520][20381] Updated weights for policy 1, policy_version 1783252 (0.0008) [2023-12-27 18:00:37,572][20381] Updated weights for policy 1, policy_version 1783262 (0.0007) [2023-12-27 18:00:37,753][20347] Updated weights for policy 0, policy_version 1784868 (0.0010) [2023-12-27 18:00:37,811][20347] Updated weights for policy 0, policy_version 1784878 (0.0010) [2023-12-27 18:00:37,862][20347] Updated weights for policy 0, policy_version 1784888 (0.0010) [2023-12-27 18:00:38,254][20381] Updated weights for policy 1, policy_version 1783272 (0.0009) [2023-12-27 18:00:38,306][20381] Updated weights for policy 1, policy_version 1783282 (0.0006) [2023-12-27 18:00:38,371][20381] Updated weights for policy 1, policy_version 1783292 (0.0008) [2023-12-27 18:00:38,559][20347] Updated weights for policy 0, policy_version 1784898 (0.0010) [2023-12-27 18:00:38,613][20347] Updated weights for policy 0, policy_version 1784908 (0.0010) [2023-12-27 18:00:38,673][20347] Updated weights for policy 0, policy_version 1784918 (0.0010) [2023-12-27 18:00:38,729][20347] Updated weights for policy 0, policy_version 1784928 (0.0010) [2023-12-27 18:00:39,097][20381] Updated weights for policy 1, policy_version 1783302 (0.0007) [2023-12-27 18:00:39,145][20381] Updated weights for policy 1, policy_version 1783312 (0.0007) [2023-12-27 18:00:39,203][20381] Updated weights for policy 1, policy_version 1783322 (0.0007) [2023-12-27 18:00:39,483][20347] Updated weights for policy 0, policy_version 1784938 (0.0008) [2023-12-27 18:00:39,534][20347] Updated weights for policy 0, policy_version 1784948 (0.0007) [2023-12-27 18:00:39,592][20347] Updated weights for policy 0, policy_version 1784958 (0.0007) [2023-12-27 18:00:39,966][20381] Updated weights for policy 1, policy_version 1783332 (0.0007) [2023-12-27 18:00:40,030][20381] Updated weights for policy 1, policy_version 1783342 (0.0008) [2023-12-27 18:00:40,092][20381] Updated weights for policy 1, policy_version 1783352 (0.0009) [2023-12-27 18:00:40,350][20347] Updated weights for policy 0, policy_version 1784968 (0.0006) [2023-12-27 18:00:40,420][20347] Updated weights for policy 0, policy_version 1784978 (0.0007) [2023-12-27 18:00:40,489][20347] Updated weights for policy 0, policy_version 1784988 (0.0006) [2023-12-27 18:00:40,776][20381] Updated weights for policy 1, policy_version 1783362 (0.0007) [2023-12-27 18:00:40,828][20381] Updated weights for policy 1, policy_version 1783372 (0.0007) [2023-12-27 18:00:40,890][20381] Updated weights for policy 1, policy_version 1783382 (0.0006) [2023-12-27 18:00:40,950][20381] Updated weights for policy 1, policy_version 1783392 (0.0007) [2023-12-27 18:00:41,140][20347] Updated weights for policy 0, policy_version 1784998 (0.0009) [2023-12-27 18:00:41,212][20347] Updated weights for policy 0, policy_version 1785008 (0.0007) [2023-12-27 18:00:41,287][20347] Updated weights for policy 0, policy_version 1785018 (0.0009) [2023-12-27 18:00:41,689][20381] Updated weights for policy 1, policy_version 1783402 (0.0008) [2023-12-27 18:00:41,757][20381] Updated weights for policy 1, policy_version 1783412 (0.0008) [2023-12-27 18:00:41,813][20381] Updated weights for policy 1, policy_version 1783422 (0.0008) [2023-12-27 18:00:42,040][20347] Updated weights for policy 0, policy_version 1785028 (0.0008) [2023-12-27 18:00:42,111][20347] Updated weights for policy 0, policy_version 1785038 (0.0007) [2023-12-27 18:00:42,172][20347] Updated weights for policy 0, policy_version 1785048 (0.0008) [2023-12-27 18:00:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 913661952. Throughput: 0: 10231.7, 1: 10342.8. Samples: 913653404. Policy #0 lag: (min: 30.0, avg: 34.2, max: 62.0) [2023-12-27 18:00:42,363][19170] Avg episode reward: [(0, '970.650'), (1, '990.290')] [2023-12-27 18:00:42,503][20381] Updated weights for policy 1, policy_version 1783432 (0.0010) [2023-12-27 18:00:42,565][20381] Updated weights for policy 1, policy_version 1783442 (0.0010) [2023-12-27 18:00:42,628][20381] Updated weights for policy 1, policy_version 1783452 (0.0005) [2023-12-27 18:00:42,922][20347] Updated weights for policy 0, policy_version 1785058 (0.0008) [2023-12-27 18:00:42,973][20347] Updated weights for policy 0, policy_version 1785068 (0.0005) [2023-12-27 18:00:43,028][20347] Updated weights for policy 0, policy_version 1785078 (0.0005) [2023-12-27 18:00:43,087][20347] Updated weights for policy 0, policy_version 1785088 (0.0006) [2023-12-27 18:00:43,200][20381] Updated weights for policy 1, policy_version 1783462 (0.0008) [2023-12-27 18:00:43,248][20381] Updated weights for policy 1, policy_version 1783472 (0.0008) [2023-12-27 18:00:43,301][20381] Updated weights for policy 1, policy_version 1783482 (0.0010) [2023-12-27 18:00:43,607][20347] Updated weights for policy 0, policy_version 1785098 (0.0008) [2023-12-27 18:00:43,655][20347] Updated weights for policy 0, policy_version 1785108 (0.0007) [2023-12-27 18:00:43,715][20347] Updated weights for policy 0, policy_version 1785118 (0.0008) [2023-12-27 18:00:43,955][20381] Updated weights for policy 1, policy_version 1783492 (0.0008) [2023-12-27 18:00:44,021][20381] Updated weights for policy 1, policy_version 1783502 (0.0005) [2023-12-27 18:00:44,082][20381] Updated weights for policy 1, policy_version 1783512 (0.0006) [2023-12-27 18:00:44,456][20347] Updated weights for policy 0, policy_version 1785128 (0.0007) [2023-12-27 18:00:44,505][20347] Updated weights for policy 0, policy_version 1785138 (0.0009) [2023-12-27 18:00:44,551][20347] Updated weights for policy 0, policy_version 1785148 (0.0007) [2023-12-27 18:00:44,710][20381] Updated weights for policy 1, policy_version 1783522 (0.0010) [2023-12-27 18:00:44,774][20381] Updated weights for policy 1, policy_version 1783532 (0.0010) [2023-12-27 18:00:44,842][20381] Updated weights for policy 1, policy_version 1783542 (0.0010) [2023-12-27 18:00:44,903][20381] Updated weights for policy 1, policy_version 1783552 (0.0011) [2023-12-27 18:00:45,261][20347] Updated weights for policy 0, policy_version 1785158 (0.0008) [2023-12-27 18:00:45,319][20347] Updated weights for policy 0, policy_version 1785168 (0.0007) [2023-12-27 18:00:45,375][20347] Updated weights for policy 0, policy_version 1785178 (0.0007) [2023-12-27 18:00:45,619][20381] Updated weights for policy 1, policy_version 1783562 (0.0011) [2023-12-27 18:00:45,677][20381] Updated weights for policy 1, policy_version 1783572 (0.0008) [2023-12-27 18:00:45,733][20381] Updated weights for policy 1, policy_version 1783582 (0.0010) [2023-12-27 18:00:46,080][20347] Updated weights for policy 0, policy_version 1785188 (0.0008) [2023-12-27 18:00:46,130][20347] Updated weights for policy 0, policy_version 1785198 (0.0008) [2023-12-27 18:00:46,190][20347] Updated weights for policy 0, policy_version 1785208 (0.0008) [2023-12-27 18:00:46,406][20381] Updated weights for policy 1, policy_version 1783592 (0.0011) [2023-12-27 18:00:46,467][20381] Updated weights for policy 1, policy_version 1783602 (0.0011) [2023-12-27 18:00:46,524][20381] Updated weights for policy 1, policy_version 1783612 (0.0011) [2023-12-27 18:00:46,906][20347] Updated weights for policy 0, policy_version 1785218 (0.0006) [2023-12-27 18:00:46,962][20347] Updated weights for policy 0, policy_version 1785228 (0.0008) [2023-12-27 18:00:47,017][20347] Updated weights for policy 0, policy_version 1785238 (0.0008) [2023-12-27 18:00:47,073][20347] Updated weights for policy 0, policy_version 1785248 (0.0007) [2023-12-27 18:00:47,165][20381] Updated weights for policy 1, policy_version 1783622 (0.0008) [2023-12-27 18:00:47,226][20381] Updated weights for policy 1, policy_version 1783632 (0.0005) [2023-12-27 18:00:47,281][20381] Updated weights for policy 1, policy_version 1783642 (0.0007) [2023-12-27 18:00:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20660.5). Total num frames: 913768448. Throughput: 0: 10176.5, 1: 10348.1. Samples: 913774620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:00:47,363][19170] Avg episode reward: [(0, '990.230'), (1, '990.330')] [2023-12-27 18:00:47,799][20347] Updated weights for policy 0, policy_version 1785258 (0.0008) [2023-12-27 18:00:47,858][20347] Updated weights for policy 0, policy_version 1785268 (0.0007) [2023-12-27 18:00:47,906][20347] Updated weights for policy 0, policy_version 1785278 (0.0009) [2023-12-27 18:00:47,929][20381] Updated weights for policy 1, policy_version 1783652 (0.0006) [2023-12-27 18:00:47,987][20381] Updated weights for policy 1, policy_version 1783662 (0.0005) [2023-12-27 18:00:48,047][20381] Updated weights for policy 1, policy_version 1783672 (0.0007) [2023-12-27 18:00:48,607][20347] Updated weights for policy 0, policy_version 1785288 (0.0007) [2023-12-27 18:00:48,663][20347] Updated weights for policy 0, policy_version 1785298 (0.0008) [2023-12-27 18:00:48,688][20381] Updated weights for policy 1, policy_version 1783682 (0.0007) [2023-12-27 18:00:48,725][20347] Updated weights for policy 0, policy_version 1785308 (0.0007) [2023-12-27 18:00:48,746][20381] Updated weights for policy 1, policy_version 1783692 (0.0006) [2023-12-27 18:00:48,811][20381] Updated weights for policy 1, policy_version 1783702 (0.0006) [2023-12-27 18:00:48,872][20381] Updated weights for policy 1, policy_version 1783712 (0.0007) [2023-12-27 18:00:49,446][20347] Updated weights for policy 0, policy_version 1785318 (0.0009) [2023-12-27 18:00:49,505][20347] Updated weights for policy 0, policy_version 1785328 (0.0011) [2023-12-27 18:00:49,565][20347] Updated weights for policy 0, policy_version 1785338 (0.0010) [2023-12-27 18:00:49,574][20381] Updated weights for policy 1, policy_version 1783722 (0.0005) [2023-12-27 18:00:49,623][20381] Updated weights for policy 1, policy_version 1783732 (0.0007) [2023-12-27 18:00:49,678][20381] Updated weights for policy 1, policy_version 1783742 (0.0007) [2023-12-27 18:00:50,304][20347] Updated weights for policy 0, policy_version 1785348 (0.0011) [2023-12-27 18:00:50,367][20347] Updated weights for policy 0, policy_version 1785358 (0.0010) [2023-12-27 18:00:50,400][20381] Updated weights for policy 1, policy_version 1783752 (0.0007) [2023-12-27 18:00:50,428][20347] Updated weights for policy 0, policy_version 1785368 (0.0011) [2023-12-27 18:00:50,464][20381] Updated weights for policy 1, policy_version 1783762 (0.0006) [2023-12-27 18:00:50,531][20381] Updated weights for policy 1, policy_version 1783772 (0.0008) [2023-12-27 18:00:51,039][20347] Updated weights for policy 0, policy_version 1785378 (0.0010) [2023-12-27 18:00:51,097][20347] Updated weights for policy 0, policy_version 1785388 (0.0006) [2023-12-27 18:00:51,164][20347] Updated weights for policy 0, policy_version 1785398 (0.0007) [2023-12-27 18:00:51,221][20381] Updated weights for policy 1, policy_version 1783782 (0.0007) [2023-12-27 18:00:51,222][20347] Updated weights for policy 0, policy_version 1785408 (0.0006) [2023-12-27 18:00:51,282][20381] Updated weights for policy 1, policy_version 1783792 (0.0007) [2023-12-27 18:00:51,343][20381] Updated weights for policy 1, policy_version 1783802 (0.0006) [2023-12-27 18:00:51,985][20347] Updated weights for policy 0, policy_version 1785418 (0.0007) [2023-12-27 18:00:52,035][20347] Updated weights for policy 0, policy_version 1785428 (0.0008) [2023-12-27 18:00:52,083][20381] Updated weights for policy 1, policy_version 1783812 (0.0007) [2023-12-27 18:00:52,089][20347] Updated weights for policy 0, policy_version 1785438 (0.0007) [2023-12-27 18:00:52,143][20381] Updated weights for policy 1, policy_version 1783822 (0.0006) [2023-12-27 18:00:52,205][20381] Updated weights for policy 1, policy_version 1783832 (0.0007) [2023-12-27 18:00:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 913866752. Throughput: 0: 10101.7, 1: 10330.0. Samples: 913833384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:00:52,363][19170] Avg episode reward: [(0, '990.250'), (1, '990.330')] [2023-12-27 18:00:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001785440_457154560.pth... [2023-12-27 18:00:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001783840_456712192.pth... [2023-12-27 18:00:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001784256_456851456.pth [2023-12-27 18:00:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001782624_456400896.pth [2023-12-27 18:00:52,782][20347] Updated weights for policy 0, policy_version 1785448 (0.0009) [2023-12-27 18:00:52,835][20347] Updated weights for policy 0, policy_version 1785458 (0.0010) [2023-12-27 18:00:52,884][20381] Updated weights for policy 1, policy_version 1783842 (0.0007) [2023-12-27 18:00:52,893][20347] Updated weights for policy 0, policy_version 1785468 (0.0009) [2023-12-27 18:00:52,940][20381] Updated weights for policy 1, policy_version 1783852 (0.0006) [2023-12-27 18:00:52,994][20381] Updated weights for policy 1, policy_version 1783862 (0.0007) [2023-12-27 18:00:53,052][20381] Updated weights for policy 1, policy_version 1783872 (0.0007) [2023-12-27 18:00:53,597][20347] Updated weights for policy 0, policy_version 1785478 (0.0010) [2023-12-27 18:00:53,649][20347] Updated weights for policy 0, policy_version 1785488 (0.0010) [2023-12-27 18:00:53,713][20347] Updated weights for policy 0, policy_version 1785498 (0.0010) [2023-12-27 18:00:53,747][20381] Updated weights for policy 1, policy_version 1783882 (0.0005) [2023-12-27 18:00:53,810][20381] Updated weights for policy 1, policy_version 1783892 (0.0007) [2023-12-27 18:00:53,867][20381] Updated weights for policy 1, policy_version 1783902 (0.0007) [2023-12-27 18:00:54,407][20347] Updated weights for policy 0, policy_version 1785508 (0.0010) [2023-12-27 18:00:54,463][20347] Updated weights for policy 0, policy_version 1785518 (0.0010) [2023-12-27 18:00:54,512][20347] Updated weights for policy 0, policy_version 1785528 (0.0010) [2023-12-27 18:00:54,562][20381] Updated weights for policy 1, policy_version 1783912 (0.0009) [2023-12-27 18:00:54,618][20381] Updated weights for policy 1, policy_version 1783922 (0.0005) [2023-12-27 18:00:54,667][20381] Updated weights for policy 1, policy_version 1783932 (0.0007) [2023-12-27 18:00:55,155][20347] Updated weights for policy 0, policy_version 1785538 (0.0010) [2023-12-27 18:00:55,209][20347] Updated weights for policy 0, policy_version 1785548 (0.0010) [2023-12-27 18:00:55,264][20347] Updated weights for policy 0, policy_version 1785558 (0.0010) [2023-12-27 18:00:55,318][20347] Updated weights for policy 0, policy_version 1785568 (0.0010) [2023-12-27 18:00:55,324][20381] Updated weights for policy 1, policy_version 1783942 (0.0011) [2023-12-27 18:00:55,381][20381] Updated weights for policy 1, policy_version 1783952 (0.0010) [2023-12-27 18:00:55,437][20381] Updated weights for policy 1, policy_version 1783962 (0.0011) [2023-12-27 18:00:56,008][20347] Updated weights for policy 0, policy_version 1785578 (0.0010) [2023-12-27 18:00:56,067][20347] Updated weights for policy 0, policy_version 1785588 (0.0010) [2023-12-27 18:00:56,075][20381] Updated weights for policy 1, policy_version 1783972 (0.0011) [2023-12-27 18:00:56,119][20347] Updated weights for policy 0, policy_version 1785598 (0.0010) [2023-12-27 18:00:56,128][20381] Updated weights for policy 1, policy_version 1783982 (0.0011) [2023-12-27 18:00:56,177][20381] Updated weights for policy 1, policy_version 1783992 (0.0010) [2023-12-27 18:00:56,787][20347] Updated weights for policy 0, policy_version 1785608 (0.0010) [2023-12-27 18:00:56,835][20347] Updated weights for policy 0, policy_version 1785618 (0.0010) [2023-12-27 18:00:56,859][20381] Updated weights for policy 1, policy_version 1784002 (0.0009) [2023-12-27 18:00:56,880][20347] Updated weights for policy 0, policy_version 1785628 (0.0010) [2023-12-27 18:00:56,921][20381] Updated weights for policy 1, policy_version 1784012 (0.0007) [2023-12-27 18:00:56,971][20381] Updated weights for policy 1, policy_version 1784022 (0.0007) [2023-12-27 18:00:57,015][20381] Updated weights for policy 1, policy_version 1784032 (0.0007) [2023-12-27 18:00:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 913965056. Throughput: 0: 10098.1, 1: 10281.6. Samples: 913954344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:00:57,363][19170] Avg episode reward: [(0, '980.450'), (1, '1000.000')] [2023-12-27 18:00:57,595][20347] Updated weights for policy 0, policy_version 1785638 (0.0010) [2023-12-27 18:00:57,655][20347] Updated weights for policy 0, policy_version 1785648 (0.0010) [2023-12-27 18:00:57,709][20381] Updated weights for policy 1, policy_version 1784042 (0.0007) [2023-12-27 18:00:57,714][20347] Updated weights for policy 0, policy_version 1785658 (0.0008) [2023-12-27 18:00:57,769][20381] Updated weights for policy 1, policy_version 1784052 (0.0007) [2023-12-27 18:00:57,827][20381] Updated weights for policy 1, policy_version 1784062 (0.0007) [2023-12-27 18:00:58,402][20347] Updated weights for policy 0, policy_version 1785668 (0.0011) [2023-12-27 18:00:58,456][20347] Updated weights for policy 0, policy_version 1785678 (0.0010) [2023-12-27 18:00:58,521][20347] Updated weights for policy 0, policy_version 1785688 (0.0009) [2023-12-27 18:00:58,568][20381] Updated weights for policy 1, policy_version 1784072 (0.0008) [2023-12-27 18:00:58,628][20381] Updated weights for policy 1, policy_version 1784082 (0.0006) [2023-12-27 18:00:58,689][20381] Updated weights for policy 1, policy_version 1784092 (0.0008) [2023-12-27 18:00:59,362][20347] Updated weights for policy 0, policy_version 1785698 (0.0008) [2023-12-27 18:00:59,435][20347] Updated weights for policy 0, policy_version 1785708 (0.0008) [2023-12-27 18:00:59,438][20381] Updated weights for policy 1, policy_version 1784102 (0.0007) [2023-12-27 18:00:59,491][20381] Updated weights for policy 1, policy_version 1784112 (0.0006) [2023-12-27 18:00:59,493][20347] Updated weights for policy 0, policy_version 1785718 (0.0007) [2023-12-27 18:00:59,547][20381] Updated weights for policy 1, policy_version 1784122 (0.0006) [2023-12-27 18:00:59,553][20347] Updated weights for policy 0, policy_version 1785728 (0.0006) [2023-12-27 18:01:00,159][20381] Updated weights for policy 1, policy_version 1784132 (0.0008) [2023-12-27 18:01:00,203][20381] Updated weights for policy 1, policy_version 1784142 (0.0006) [2023-12-27 18:01:00,259][20381] Updated weights for policy 1, policy_version 1784152 (0.0005) [2023-12-27 18:01:00,272][20347] Updated weights for policy 0, policy_version 1785738 (0.0007) [2023-12-27 18:01:00,328][20347] Updated weights for policy 0, policy_version 1785748 (0.0008) [2023-12-27 18:01:00,390][20347] Updated weights for policy 0, policy_version 1785758 (0.0007) [2023-12-27 18:01:00,889][20381] Updated weights for policy 1, policy_version 1784162 (0.0006) [2023-12-27 18:01:00,935][20381] Updated weights for policy 1, policy_version 1784172 (0.0007) [2023-12-27 18:01:00,982][20381] Updated weights for policy 1, policy_version 1784182 (0.0007) [2023-12-27 18:01:01,046][20381] Updated weights for policy 1, policy_version 1784192 (0.0008) [2023-12-27 18:01:01,095][20347] Updated weights for policy 0, policy_version 1785768 (0.0007) [2023-12-27 18:01:01,162][20347] Updated weights for policy 0, policy_version 1785778 (0.0007) [2023-12-27 18:01:01,223][20347] Updated weights for policy 0, policy_version 1785788 (0.0008) [2023-12-27 18:01:01,775][20381] Updated weights for policy 1, policy_version 1784202 (0.0008) [2023-12-27 18:01:01,839][20381] Updated weights for policy 1, policy_version 1784212 (0.0008) [2023-12-27 18:01:01,901][20381] Updated weights for policy 1, policy_version 1784222 (0.0008) [2023-12-27 18:01:01,908][20347] Updated weights for policy 0, policy_version 1785798 (0.0010) [2023-12-27 18:01:01,959][20347] Updated weights for policy 0, policy_version 1785808 (0.0010) [2023-12-27 18:01:02,012][20347] Updated weights for policy 0, policy_version 1785818 (0.0010) [2023-12-27 18:01:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 914063360. Throughput: 0: 10022.6, 1: 10198.7. Samples: 914072348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:02,363][19170] Avg episode reward: [(0, '980.430'), (1, '1000.000')] [2023-12-27 18:01:02,536][20381] Updated weights for policy 1, policy_version 1784232 (0.0007) [2023-12-27 18:01:02,584][20381] Updated weights for policy 1, policy_version 1784242 (0.0007) [2023-12-27 18:01:02,640][20381] Updated weights for policy 1, policy_version 1784252 (0.0007) [2023-12-27 18:01:02,740][20347] Updated weights for policy 0, policy_version 1785828 (0.0008) [2023-12-27 18:01:02,796][20347] Updated weights for policy 0, policy_version 1785838 (0.0006) [2023-12-27 18:01:02,856][20347] Updated weights for policy 0, policy_version 1785848 (0.0006) [2023-12-27 18:01:03,366][20381] Updated weights for policy 1, policy_version 1784262 (0.0006) [2023-12-27 18:01:03,422][20381] Updated weights for policy 1, policy_version 1784272 (0.0005) [2023-12-27 18:01:03,431][20347] Updated weights for policy 0, policy_version 1785858 (0.0006) [2023-12-27 18:01:03,474][20381] Updated weights for policy 1, policy_version 1784282 (0.0006) [2023-12-27 18:01:03,480][20347] Updated weights for policy 0, policy_version 1785868 (0.0007) [2023-12-27 18:01:03,534][20347] Updated weights for policy 0, policy_version 1785878 (0.0006) [2023-12-27 18:01:03,587][20347] Updated weights for policy 0, policy_version 1785888 (0.0006) [2023-12-27 18:01:04,103][20381] Updated weights for policy 1, policy_version 1784292 (0.0011) [2023-12-27 18:01:04,170][20381] Updated weights for policy 1, policy_version 1784302 (0.0008) [2023-12-27 18:01:04,206][20347] Updated weights for policy 0, policy_version 1785898 (0.0007) [2023-12-27 18:01:04,227][20381] Updated weights for policy 1, policy_version 1784312 (0.0007) [2023-12-27 18:01:04,264][20347] Updated weights for policy 0, policy_version 1785908 (0.0007) [2023-12-27 18:01:04,323][20347] Updated weights for policy 0, policy_version 1785918 (0.0006) [2023-12-27 18:01:04,923][20347] Updated weights for policy 0, policy_version 1785928 (0.0008) [2023-12-27 18:01:04,924][20381] Updated weights for policy 1, policy_version 1784322 (0.0008) [2023-12-27 18:01:04,979][20381] Updated weights for policy 1, policy_version 1784332 (0.0006) [2023-12-27 18:01:04,986][20347] Updated weights for policy 0, policy_version 1785938 (0.0005) [2023-12-27 18:01:05,030][20381] Updated weights for policy 1, policy_version 1784342 (0.0006) [2023-12-27 18:01:05,051][20347] Updated weights for policy 0, policy_version 1785948 (0.0008) [2023-12-27 18:01:05,081][20381] Updated weights for policy 1, policy_version 1784352 (0.0006) [2023-12-27 18:01:05,601][20347] Updated weights for policy 0, policy_version 1785958 (0.0007) [2023-12-27 18:01:05,666][20347] Updated weights for policy 0, policy_version 1785968 (0.0010) [2023-12-27 18:01:05,728][20381] Updated weights for policy 1, policy_version 1784362 (0.0006) [2023-12-27 18:01:05,729][20347] Updated weights for policy 0, policy_version 1785978 (0.0010) [2023-12-27 18:01:05,775][20381] Updated weights for policy 1, policy_version 1784372 (0.0006) [2023-12-27 18:01:05,830][20381] Updated weights for policy 1, policy_version 1784382 (0.0009) [2023-12-27 18:01:06,343][20347] Updated weights for policy 0, policy_version 1785988 (0.0009) [2023-12-27 18:01:06,403][20347] Updated weights for policy 0, policy_version 1785998 (0.0005) [2023-12-27 18:01:06,462][20347] Updated weights for policy 0, policy_version 1786008 (0.0006) [2023-12-27 18:01:06,491][20381] Updated weights for policy 1, policy_version 1784392 (0.0011) [2023-12-27 18:01:06,554][20381] Updated weights for policy 1, policy_version 1784402 (0.0011) [2023-12-27 18:01:06,617][20381] Updated weights for policy 1, policy_version 1784412 (0.0010) [2023-12-27 18:01:07,027][20347] Updated weights for policy 0, policy_version 1786018 (0.0007) [2023-12-27 18:01:07,093][20347] Updated weights for policy 0, policy_version 1786028 (0.0010) [2023-12-27 18:01:07,157][20347] Updated weights for policy 0, policy_version 1786038 (0.0008) [2023-12-27 18:01:07,167][20381] Updated weights for policy 1, policy_version 1784422 (0.0010) [2023-12-27 18:01:07,217][20347] Updated weights for policy 0, policy_version 1786048 (0.0005) [2023-12-27 18:01:07,226][20381] Updated weights for policy 1, policy_version 1784432 (0.0011) [2023-12-27 18:01:07,295][20381] Updated weights for policy 1, policy_version 1784442 (0.0011) [2023-12-27 18:01:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 914178048. Throughput: 0: 10096.6, 1: 10195.6. Samples: 914136724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:07,363][19170] Avg episode reward: [(0, '980.610'), (1, '1000.000')] [2023-12-27 18:01:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001784448_456867840.pth... [2023-12-27 18:01:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001786048_457310208.pth... [2023-12-27 18:01:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001783232_456556544.pth [2023-12-27 18:01:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001784864_457007104.pth [2023-12-27 18:01:07,857][20347] Updated weights for policy 0, policy_version 1786058 (0.0010) [2023-12-27 18:01:07,905][20347] Updated weights for policy 0, policy_version 1786068 (0.0010) [2023-12-27 18:01:07,953][20347] Updated weights for policy 0, policy_version 1786078 (0.0010) [2023-12-27 18:01:07,971][20381] Updated weights for policy 1, policy_version 1784452 (0.0011) [2023-12-27 18:01:08,027][20381] Updated weights for policy 1, policy_version 1784462 (0.0010) [2023-12-27 18:01:08,079][20381] Updated weights for policy 1, policy_version 1784472 (0.0010) [2023-12-27 18:01:08,657][20347] Updated weights for policy 0, policy_version 1786088 (0.0008) [2023-12-27 18:01:08,717][20347] Updated weights for policy 0, policy_version 1786098 (0.0006) [2023-12-27 18:01:08,762][20381] Updated weights for policy 1, policy_version 1784482 (0.0010) [2023-12-27 18:01:08,776][20347] Updated weights for policy 0, policy_version 1786108 (0.0007) [2023-12-27 18:01:08,819][20381] Updated weights for policy 1, policy_version 1784492 (0.0008) [2023-12-27 18:01:08,881][20381] Updated weights for policy 1, policy_version 1784502 (0.0010) [2023-12-27 18:01:08,940][20381] Updated weights for policy 1, policy_version 1784512 (0.0011) [2023-12-27 18:01:09,503][20347] Updated weights for policy 0, policy_version 1786118 (0.0008) [2023-12-27 18:01:09,556][20347] Updated weights for policy 0, policy_version 1786128 (0.0009) [2023-12-27 18:01:09,577][20381] Updated weights for policy 1, policy_version 1784522 (0.0009) [2023-12-27 18:01:09,609][20347] Updated weights for policy 0, policy_version 1786138 (0.0009) [2023-12-27 18:01:09,625][20381] Updated weights for policy 1, policy_version 1784532 (0.0009) [2023-12-27 18:01:09,678][20381] Updated weights for policy 1, policy_version 1784542 (0.0009) [2023-12-27 18:01:10,390][20347] Updated weights for policy 0, policy_version 1786148 (0.0009) [2023-12-27 18:01:10,417][20381] Updated weights for policy 1, policy_version 1784552 (0.0010) [2023-12-27 18:01:10,449][20347] Updated weights for policy 0, policy_version 1786158 (0.0010) [2023-12-27 18:01:10,471][20381] Updated weights for policy 1, policy_version 1784562 (0.0010) [2023-12-27 18:01:10,511][20347] Updated weights for policy 0, policy_version 1786168 (0.0010) [2023-12-27 18:01:10,525][20381] Updated weights for policy 1, policy_version 1784572 (0.0005) [2023-12-27 18:01:11,150][20347] Updated weights for policy 0, policy_version 1786178 (0.0010) [2023-12-27 18:01:11,211][20347] Updated weights for policy 0, policy_version 1786188 (0.0008) [2023-12-27 18:01:11,233][20381] Updated weights for policy 1, policy_version 1784582 (0.0008) [2023-12-27 18:01:11,271][20347] Updated weights for policy 0, policy_version 1786198 (0.0009) [2023-12-27 18:01:11,296][20381] Updated weights for policy 1, policy_version 1784592 (0.0009) [2023-12-27 18:01:11,332][20347] Updated weights for policy 0, policy_version 1786208 (0.0007) [2023-12-27 18:01:11,363][20381] Updated weights for policy 1, policy_version 1784603 (0.0007) [2023-12-27 18:01:12,082][20381] Updated weights for policy 1, policy_version 1784613 (0.0010) [2023-12-27 18:01:12,128][20347] Updated weights for policy 0, policy_version 1786218 (0.0007) [2023-12-27 18:01:12,148][20381] Updated weights for policy 1, policy_version 1784623 (0.0011) [2023-12-27 18:01:12,182][20347] Updated weights for policy 0, policy_version 1786228 (0.0006) [2023-12-27 18:01:12,211][20381] Updated weights for policy 1, policy_version 1784633 (0.0011) [2023-12-27 18:01:12,243][20347] Updated weights for policy 0, policy_version 1786238 (0.0007) [2023-12-27 18:01:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 914276352. Throughput: 0: 10140.6, 1: 10183.7. Samples: 914261964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:12,363][19170] Avg episode reward: [(0, '961.130'), (1, '990.270')] [2023-12-27 18:01:12,902][20381] Updated weights for policy 1, policy_version 1784643 (0.0011) [2023-12-27 18:01:12,946][20347] Updated weights for policy 0, policy_version 1786248 (0.0006) [2023-12-27 18:01:12,955][20381] Updated weights for policy 1, policy_version 1784653 (0.0010) [2023-12-27 18:01:13,004][20347] Updated weights for policy 0, policy_version 1786258 (0.0007) [2023-12-27 18:01:13,013][20381] Updated weights for policy 1, policy_version 1784663 (0.0010) [2023-12-27 18:01:13,056][20347] Updated weights for policy 0, policy_version 1786268 (0.0007) [2023-12-27 18:01:13,582][20381] Updated weights for policy 1, policy_version 1784673 (0.0006) [2023-12-27 18:01:13,636][20381] Updated weights for policy 1, policy_version 1784683 (0.0006) [2023-12-27 18:01:13,684][20381] Updated weights for policy 1, policy_version 1784693 (0.0005) [2023-12-27 18:01:13,737][20347] Updated weights for policy 0, policy_version 1786278 (0.0006) [2023-12-27 18:01:13,741][20381] Updated weights for policy 1, policy_version 1784703 (0.0007) [2023-12-27 18:01:13,791][20347] Updated weights for policy 0, policy_version 1786288 (0.0006) [2023-12-27 18:01:13,839][20347] Updated weights for policy 0, policy_version 1786298 (0.0006) [2023-12-27 18:01:14,386][20381] Updated weights for policy 1, policy_version 1784713 (0.0008) [2023-12-27 18:01:14,448][20381] Updated weights for policy 1, policy_version 1784723 (0.0008) [2023-12-27 18:01:14,448][20347] Updated weights for policy 0, policy_version 1786308 (0.0007) [2023-12-27 18:01:14,503][20347] Updated weights for policy 0, policy_version 1786318 (0.0008) [2023-12-27 18:01:14,512][20381] Updated weights for policy 1, policy_version 1784733 (0.0007) [2023-12-27 18:01:14,550][20347] Updated weights for policy 0, policy_version 1786328 (0.0007) [2023-12-27 18:01:15,227][20381] Updated weights for policy 1, policy_version 1784743 (0.0007) [2023-12-27 18:01:15,281][20381] Updated weights for policy 1, policy_version 1784753 (0.0006) [2023-12-27 18:01:15,286][20347] Updated weights for policy 0, policy_version 1786338 (0.0009) [2023-12-27 18:01:15,334][20381] Updated weights for policy 1, policy_version 1784763 (0.0007) [2023-12-27 18:01:15,348][20347] Updated weights for policy 0, policy_version 1786348 (0.0007) [2023-12-27 18:01:15,405][20347] Updated weights for policy 0, policy_version 1786358 (0.0007) [2023-12-27 18:01:15,462][20347] Updated weights for policy 0, policy_version 1786368 (0.0007) [2023-12-27 18:01:16,009][20381] Updated weights for policy 1, policy_version 1784774 (0.0008) [2023-12-27 18:01:16,055][20381] Updated weights for policy 1, policy_version 1784784 (0.0007) [2023-12-27 18:01:16,110][20381] Updated weights for policy 1, policy_version 1784794 (0.0006) [2023-12-27 18:01:16,132][20347] Updated weights for policy 0, policy_version 1786378 (0.0008) [2023-12-27 18:01:16,183][20347] Updated weights for policy 0, policy_version 1786388 (0.0008) [2023-12-27 18:01:16,233][20347] Updated weights for policy 0, policy_version 1786398 (0.0007) [2023-12-27 18:01:16,758][20381] Updated weights for policy 1, policy_version 1784804 (0.0007) [2023-12-27 18:01:16,815][20381] Updated weights for policy 1, policy_version 1784814 (0.0008) [2023-12-27 18:01:16,876][20381] Updated weights for policy 1, policy_version 1784824 (0.0011) [2023-12-27 18:01:16,957][20347] Updated weights for policy 0, policy_version 1786408 (0.0007) [2023-12-27 18:01:17,000][20347] Updated weights for policy 0, policy_version 1786418 (0.0006) [2023-12-27 18:01:17,052][20347] Updated weights for policy 0, policy_version 1786428 (0.0007) [2023-12-27 18:01:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 914374656. Throughput: 0: 10100.4, 1: 10193.5. Samples: 914383948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:17,363][19170] Avg episode reward: [(0, '970.760'), (1, '990.270')] [2023-12-27 18:01:17,503][20381] Updated weights for policy 1, policy_version 1784834 (0.0011) [2023-12-27 18:01:17,556][20381] Updated weights for policy 1, policy_version 1784844 (0.0010) [2023-12-27 18:01:17,605][20381] Updated weights for policy 1, policy_version 1784854 (0.0005) [2023-12-27 18:01:17,649][20381] Updated weights for policy 1, policy_version 1784864 (0.0005) [2023-12-27 18:01:17,775][20347] Updated weights for policy 0, policy_version 1786438 (0.0006) [2023-12-27 18:01:17,827][20347] Updated weights for policy 0, policy_version 1786448 (0.0005) [2023-12-27 18:01:17,871][20347] Updated weights for policy 0, policy_version 1786458 (0.0008) [2023-12-27 18:01:18,244][20381] Updated weights for policy 1, policy_version 1784874 (0.0008) [2023-12-27 18:01:18,294][20381] Updated weights for policy 1, policy_version 1784884 (0.0008) [2023-12-27 18:01:18,350][20381] Updated weights for policy 1, policy_version 1784894 (0.0007) [2023-12-27 18:01:18,567][20347] Updated weights for policy 0, policy_version 1786468 (0.0008) [2023-12-27 18:01:18,625][20347] Updated weights for policy 0, policy_version 1786478 (0.0007) [2023-12-27 18:01:18,687][20347] Updated weights for policy 0, policy_version 1786488 (0.0007) [2023-12-27 18:01:19,003][20381] Updated weights for policy 1, policy_version 1784904 (0.0006) [2023-12-27 18:01:19,074][20381] Updated weights for policy 1, policy_version 1784914 (0.0007) [2023-12-27 18:01:19,122][20381] Updated weights for policy 1, policy_version 1784924 (0.0007) [2023-12-27 18:01:19,315][20347] Updated weights for policy 0, policy_version 1786498 (0.0007) [2023-12-27 18:01:19,373][20347] Updated weights for policy 0, policy_version 1786508 (0.0008) [2023-12-27 18:01:19,422][20347] Updated weights for policy 0, policy_version 1786518 (0.0005) [2023-12-27 18:01:19,476][20347] Updated weights for policy 0, policy_version 1786528 (0.0007) [2023-12-27 18:01:19,875][20381] Updated weights for policy 1, policy_version 1784934 (0.0008) [2023-12-27 18:01:19,939][20381] Updated weights for policy 1, policy_version 1784944 (0.0008) [2023-12-27 18:01:19,997][20381] Updated weights for policy 1, policy_version 1784954 (0.0008) [2023-12-27 18:01:20,218][20347] Updated weights for policy 0, policy_version 1786538 (0.0007) [2023-12-27 18:01:20,278][20347] Updated weights for policy 0, policy_version 1786548 (0.0005) [2023-12-27 18:01:20,340][20347] Updated weights for policy 0, policy_version 1786558 (0.0007) [2023-12-27 18:01:20,683][20381] Updated weights for policy 1, policy_version 1784964 (0.0008) [2023-12-27 18:01:20,739][20381] Updated weights for policy 1, policy_version 1784974 (0.0008) [2023-12-27 18:01:20,790][20381] Updated weights for policy 1, policy_version 1784984 (0.0008) [2023-12-27 18:01:21,018][20347] Updated weights for policy 0, policy_version 1786568 (0.0008) [2023-12-27 18:01:21,084][20347] Updated weights for policy 0, policy_version 1786578 (0.0008) [2023-12-27 18:01:21,143][20347] Updated weights for policy 0, policy_version 1786588 (0.0008) [2023-12-27 18:01:21,545][20381] Updated weights for policy 1, policy_version 1784994 (0.0008) [2023-12-27 18:01:21,608][20381] Updated weights for policy 1, policy_version 1785004 (0.0008) [2023-12-27 18:01:21,676][20381] Updated weights for policy 1, policy_version 1785014 (0.0008) [2023-12-27 18:01:21,748][20381] Updated weights for policy 1, policy_version 1785024 (0.0009) [2023-12-27 18:01:21,960][20347] Updated weights for policy 0, policy_version 1786598 (0.0008) [2023-12-27 18:01:22,019][20347] Updated weights for policy 0, policy_version 1786608 (0.0008) [2023-12-27 18:01:22,073][20347] Updated weights for policy 0, policy_version 1786618 (0.0008) [2023-12-27 18:01:22,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 914472960. Throughput: 0: 10069.5, 1: 10197.3. Samples: 914445480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:22,363][19170] Avg episode reward: [(0, '960.930'), (1, '990.330')] [2023-12-27 18:01:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001786624_457457664.pth... [2023-12-27 18:01:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001785440_457154560.pth [2023-12-27 18:01:22,398][20381] Updated weights for policy 1, policy_version 1785034 (0.0010) [2023-12-27 18:01:22,462][20381] Updated weights for policy 1, policy_version 1785044 (0.0008) [2023-12-27 18:01:22,525][20381] Updated weights for policy 1, policy_version 1785054 (0.0008) [2023-12-27 18:01:22,535][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001785056_457023488.pth... [2023-12-27 18:01:22,538][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001783840_456712192.pth [2023-12-27 18:01:22,805][20347] Updated weights for policy 0, policy_version 1786628 (0.0006) [2023-12-27 18:01:22,864][20347] Updated weights for policy 0, policy_version 1786638 (0.0007) [2023-12-27 18:01:22,917][20347] Updated weights for policy 0, policy_version 1786648 (0.0010) [2023-12-27 18:01:23,259][20381] Updated weights for policy 1, policy_version 1785064 (0.0008) [2023-12-27 18:01:23,320][20381] Updated weights for policy 1, policy_version 1785074 (0.0008) [2023-12-27 18:01:23,371][20381] Updated weights for policy 1, policy_version 1785084 (0.0007) [2023-12-27 18:01:23,575][20347] Updated weights for policy 0, policy_version 1786658 (0.0007) [2023-12-27 18:01:23,631][20347] Updated weights for policy 0, policy_version 1786668 (0.0011) [2023-12-27 18:01:23,681][20347] Updated weights for policy 0, policy_version 1786678 (0.0010) [2023-12-27 18:01:23,730][20347] Updated weights for policy 0, policy_version 1786688 (0.0010) [2023-12-27 18:01:23,960][20381] Updated weights for policy 1, policy_version 1785094 (0.0007) [2023-12-27 18:01:24,003][20381] Updated weights for policy 1, policy_version 1785104 (0.0007) [2023-12-27 18:01:24,049][20381] Updated weights for policy 1, policy_version 1785114 (0.0010) [2023-12-27 18:01:24,427][20347] Updated weights for policy 0, policy_version 1786698 (0.0010) [2023-12-27 18:01:24,478][20347] Updated weights for policy 0, policy_version 1786708 (0.0010) [2023-12-27 18:01:24,538][20347] Updated weights for policy 0, policy_version 1786718 (0.0010) [2023-12-27 18:01:24,681][20381] Updated weights for policy 1, policy_version 1785124 (0.0010) [2023-12-27 18:01:24,740][20381] Updated weights for policy 1, policy_version 1785134 (0.0009) [2023-12-27 18:01:24,791][20381] Updated weights for policy 1, policy_version 1785144 (0.0010) [2023-12-27 18:01:25,228][20347] Updated weights for policy 0, policy_version 1786728 (0.0007) [2023-12-27 18:01:25,291][20347] Updated weights for policy 0, policy_version 1786738 (0.0008) [2023-12-27 18:01:25,350][20347] Updated weights for policy 0, policy_version 1786748 (0.0008) [2023-12-27 18:01:25,397][20381] Updated weights for policy 1, policy_version 1785154 (0.0011) [2023-12-27 18:01:25,455][20381] Updated weights for policy 1, policy_version 1785164 (0.0008) [2023-12-27 18:01:25,514][20381] Updated weights for policy 1, policy_version 1785174 (0.0005) [2023-12-27 18:01:25,563][20381] Updated weights for policy 1, policy_version 1785184 (0.0005) [2023-12-27 18:01:26,015][20347] Updated weights for policy 0, policy_version 1786758 (0.0007) [2023-12-27 18:01:26,066][20347] Updated weights for policy 0, policy_version 1786768 (0.0008) [2023-12-27 18:01:26,119][20347] Updated weights for policy 0, policy_version 1786778 (0.0008) [2023-12-27 18:01:26,150][20381] Updated weights for policy 1, policy_version 1785194 (0.0007) [2023-12-27 18:01:26,211][20381] Updated weights for policy 1, policy_version 1785204 (0.0008) [2023-12-27 18:01:26,274][20381] Updated weights for policy 1, policy_version 1785214 (0.0006) [2023-12-27 18:01:26,850][20347] Updated weights for policy 0, policy_version 1786788 (0.0007) [2023-12-27 18:01:26,885][20381] Updated weights for policy 1, policy_version 1785224 (0.0006) [2023-12-27 18:01:26,901][20347] Updated weights for policy 0, policy_version 1786798 (0.0006) [2023-12-27 18:01:26,937][20381] Updated weights for policy 1, policy_version 1785234 (0.0006) [2023-12-27 18:01:26,961][20347] Updated weights for policy 0, policy_version 1786808 (0.0008) [2023-12-27 18:01:26,983][20381] Updated weights for policy 1, policy_version 1785244 (0.0006) [2023-12-27 18:01:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 914579456. Throughput: 0: 10040.6, 1: 10281.9. Samples: 914567916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:27,363][19170] Avg episode reward: [(0, '951.240'), (1, '990.330')] [2023-12-27 18:01:27,651][20381] Updated weights for policy 1, policy_version 1785254 (0.0007) [2023-12-27 18:01:27,684][20347] Updated weights for policy 0, policy_version 1786818 (0.0007) [2023-12-27 18:01:27,710][20381] Updated weights for policy 1, policy_version 1785264 (0.0009) [2023-12-27 18:01:27,730][20347] Updated weights for policy 0, policy_version 1786828 (0.0008) [2023-12-27 18:01:27,770][20381] Updated weights for policy 1, policy_version 1785274 (0.0011) [2023-12-27 18:01:27,774][20347] Updated weights for policy 0, policy_version 1786838 (0.0006) [2023-12-27 18:01:27,819][20347] Updated weights for policy 0, policy_version 1786848 (0.0006) [2023-12-27 18:01:28,427][20381] Updated weights for policy 1, policy_version 1785284 (0.0009) [2023-12-27 18:01:28,430][20347] Updated weights for policy 0, policy_version 1786858 (0.0007) [2023-12-27 18:01:28,490][20381] Updated weights for policy 1, policy_version 1785294 (0.0005) [2023-12-27 18:01:28,494][20347] Updated weights for policy 0, policy_version 1786868 (0.0006) [2023-12-27 18:01:28,545][20291] KL-divergence is very high: 140.6391 [2023-12-27 18:01:28,548][20381] Updated weights for policy 1, policy_version 1785304 (0.0007) [2023-12-27 18:01:28,549][20347] Updated weights for policy 0, policy_version 1786878 (0.0006) [2023-12-27 18:01:28,551][20291] KL-divergence is very high: 145.8379 [2023-12-27 18:01:29,132][20381] Updated weights for policy 1, policy_version 1785314 (0.0006) [2023-12-27 18:01:29,189][20381] Updated weights for policy 1, policy_version 1785324 (0.0005) [2023-12-27 18:01:29,237][20347] Updated weights for policy 0, policy_version 1786888 (0.0007) [2023-12-27 18:01:29,251][20381] Updated weights for policy 1, policy_version 1785334 (0.0007) [2023-12-27 18:01:29,299][20347] Updated weights for policy 0, policy_version 1786898 (0.0008) [2023-12-27 18:01:29,310][20381] Updated weights for policy 1, policy_version 1785344 (0.0006) [2023-12-27 18:01:29,366][20347] Updated weights for policy 0, policy_version 1786908 (0.0008) [2023-12-27 18:01:29,987][20347] Updated weights for policy 0, policy_version 1786918 (0.0007) [2023-12-27 18:01:30,040][20347] Updated weights for policy 0, policy_version 1786928 (0.0006) [2023-12-27 18:01:30,042][20381] Updated weights for policy 1, policy_version 1785354 (0.0006) [2023-12-27 18:01:30,090][20347] Updated weights for policy 0, policy_version 1786938 (0.0005) [2023-12-27 18:01:30,102][20381] Updated weights for policy 1, policy_version 1785364 (0.0007) [2023-12-27 18:01:30,158][20381] Updated weights for policy 1, policy_version 1785374 (0.0007) [2023-12-27 18:01:30,803][20381] Updated weights for policy 1, policy_version 1785384 (0.0007) [2023-12-27 18:01:30,811][20347] Updated weights for policy 0, policy_version 1786948 (0.0007) [2023-12-27 18:01:30,858][20381] Updated weights for policy 1, policy_version 1785394 (0.0005) [2023-12-27 18:01:30,872][20347] Updated weights for policy 0, policy_version 1786958 (0.0008) [2023-12-27 18:01:30,913][20381] Updated weights for policy 1, policy_version 1785404 (0.0005) [2023-12-27 18:01:30,924][20347] Updated weights for policy 0, policy_version 1786968 (0.0007) [2023-12-27 18:01:31,527][20381] Updated weights for policy 1, policy_version 1785414 (0.0007) [2023-12-27 18:01:31,586][20381] Updated weights for policy 1, policy_version 1785424 (0.0008) [2023-12-27 18:01:31,644][20381] Updated weights for policy 1, policy_version 1785434 (0.0008) [2023-12-27 18:01:31,651][20347] Updated weights for policy 0, policy_version 1786978 (0.0007) [2023-12-27 18:01:31,714][20347] Updated weights for policy 0, policy_version 1786988 (0.0007) [2023-12-27 18:01:31,762][20347] Updated weights for policy 0, policy_version 1786998 (0.0006) [2023-12-27 18:01:31,809][20347] Updated weights for policy 0, policy_version 1787008 (0.0006) [2023-12-27 18:01:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 914677760. Throughput: 0: 10082.5, 1: 10304.2. Samples: 914692020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:32,363][19170] Avg episode reward: [(0, '970.810'), (1, '1000.000')] [2023-12-27 18:01:32,393][20381] Updated weights for policy 1, policy_version 1785444 (0.0008) [2023-12-27 18:01:32,446][20381] Updated weights for policy 1, policy_version 1785454 (0.0011) [2023-12-27 18:01:32,499][20381] Updated weights for policy 1, policy_version 1785464 (0.0010) [2023-12-27 18:01:32,504][20347] Updated weights for policy 0, policy_version 1787018 (0.0008) [2023-12-27 18:01:32,557][20347] Updated weights for policy 0, policy_version 1787028 (0.0006) [2023-12-27 18:01:32,609][20347] Updated weights for policy 0, policy_version 1787038 (0.0007) [2023-12-27 18:01:33,224][20381] Updated weights for policy 1, policy_version 1785474 (0.0010) [2023-12-27 18:01:33,226][20347] Updated weights for policy 0, policy_version 1787048 (0.0006) [2023-12-27 18:01:33,278][20381] Updated weights for policy 1, policy_version 1785484 (0.0007) [2023-12-27 18:01:33,281][20347] Updated weights for policy 0, policy_version 1787058 (0.0005) [2023-12-27 18:01:33,335][20347] Updated weights for policy 0, policy_version 1787068 (0.0005) [2023-12-27 18:01:33,335][20381] Updated weights for policy 1, policy_version 1785494 (0.0007) [2023-12-27 18:01:33,395][20381] Updated weights for policy 1, policy_version 1785504 (0.0009) [2023-12-27 18:01:33,877][20347] Updated weights for policy 0, policy_version 1787078 (0.0008) [2023-12-27 18:01:33,930][20347] Updated weights for policy 0, policy_version 1787088 (0.0006) [2023-12-27 18:01:33,992][20347] Updated weights for policy 0, policy_version 1787098 (0.0005) [2023-12-27 18:01:34,084][20381] Updated weights for policy 1, policy_version 1785514 (0.0008) [2023-12-27 18:01:34,134][20381] Updated weights for policy 1, policy_version 1785524 (0.0010) [2023-12-27 18:01:34,199][20381] Updated weights for policy 1, policy_version 1785534 (0.0011) [2023-12-27 18:01:34,679][20347] Updated weights for policy 0, policy_version 1787108 (0.0007) [2023-12-27 18:01:34,735][20347] Updated weights for policy 0, policy_version 1787118 (0.0010) [2023-12-27 18:01:34,790][20347] Updated weights for policy 0, policy_version 1787128 (0.0010) [2023-12-27 18:01:34,917][20381] Updated weights for policy 1, policy_version 1785544 (0.0007) [2023-12-27 18:01:34,981][20381] Updated weights for policy 1, policy_version 1785554 (0.0006) [2023-12-27 18:01:35,040][20381] Updated weights for policy 1, policy_version 1785564 (0.0010) [2023-12-27 18:01:35,428][20347] Updated weights for policy 0, policy_version 1787138 (0.0008) [2023-12-27 18:01:35,485][20347] Updated weights for policy 0, policy_version 1787148 (0.0009) [2023-12-27 18:01:35,534][20347] Updated weights for policy 0, policy_version 1787158 (0.0009) [2023-12-27 18:01:35,590][20347] Updated weights for policy 0, policy_version 1787168 (0.0009) [2023-12-27 18:01:35,614][20381] Updated weights for policy 1, policy_version 1785574 (0.0007) [2023-12-27 18:01:35,662][20381] Updated weights for policy 1, policy_version 1785584 (0.0005) [2023-12-27 18:01:35,706][20381] Updated weights for policy 1, policy_version 1785594 (0.0005) [2023-12-27 18:01:36,292][20381] Updated weights for policy 1, policy_version 1785604 (0.0006) [2023-12-27 18:01:36,305][20347] Updated weights for policy 0, policy_version 1787178 (0.0011) [2023-12-27 18:01:36,342][20381] Updated weights for policy 1, policy_version 1785614 (0.0008) [2023-12-27 18:01:36,366][20347] Updated weights for policy 0, policy_version 1787188 (0.0010) [2023-12-27 18:01:36,399][20381] Updated weights for policy 1, policy_version 1785624 (0.0008) [2023-12-27 18:01:36,429][20347] Updated weights for policy 0, policy_version 1787198 (0.0010) [2023-12-27 18:01:37,088][20381] Updated weights for policy 1, policy_version 1785634 (0.0008) [2023-12-27 18:01:37,105][20347] Updated weights for policy 0, policy_version 1787208 (0.0010) [2023-12-27 18:01:37,145][20381] Updated weights for policy 1, policy_version 1785644 (0.0005) [2023-12-27 18:01:37,166][20347] Updated weights for policy 0, policy_version 1787218 (0.0008) [2023-12-27 18:01:37,205][20381] Updated weights for policy 1, policy_version 1785654 (0.0005) [2023-12-27 18:01:37,224][20347] Updated weights for policy 0, policy_version 1787228 (0.0010) [2023-12-27 18:01:37,257][20381] Updated weights for policy 1, policy_version 1785664 (0.0005) [2023-12-27 18:01:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20632.7). Total num frames: 914792448. Throughput: 0: 10173.5, 1: 10299.3. Samples: 914754660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:37,363][19170] Avg episode reward: [(0, '970.940'), (1, '1000.000')] [2023-12-27 18:01:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001785664_457179136.pth... [2023-12-27 18:01:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001787232_457613312.pth... [2023-12-27 18:01:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001784448_456867840.pth [2023-12-27 18:01:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001786048_457310208.pth [2023-12-27 18:01:37,786][20347] Updated weights for policy 0, policy_version 1787238 (0.0007) [2023-12-27 18:01:37,821][20381] Updated weights for policy 1, policy_version 1785674 (0.0007) [2023-12-27 18:01:37,833][20347] Updated weights for policy 0, policy_version 1787248 (0.0005) [2023-12-27 18:01:37,871][20381] Updated weights for policy 1, policy_version 1785684 (0.0008) [2023-12-27 18:01:37,887][20347] Updated weights for policy 0, policy_version 1787258 (0.0006) [2023-12-27 18:01:37,930][20381] Updated weights for policy 1, policy_version 1785694 (0.0006) [2023-12-27 18:01:38,493][20347] Updated weights for policy 0, policy_version 1787268 (0.0006) [2023-12-27 18:01:38,553][20347] Updated weights for policy 0, policy_version 1787278 (0.0007) [2023-12-27 18:01:38,598][20347] Updated weights for policy 0, policy_version 1787288 (0.0007) [2023-12-27 18:01:38,658][20381] Updated weights for policy 1, policy_version 1785704 (0.0010) [2023-12-27 18:01:38,717][20381] Updated weights for policy 1, policy_version 1785714 (0.0010) [2023-12-27 18:01:38,780][20381] Updated weights for policy 1, policy_version 1785724 (0.0010) [2023-12-27 18:01:39,318][20347] Updated weights for policy 0, policy_version 1787298 (0.0007) [2023-12-27 18:01:39,384][20347] Updated weights for policy 0, policy_version 1787308 (0.0008) [2023-12-27 18:01:39,438][20381] Updated weights for policy 1, policy_version 1785734 (0.0009) [2023-12-27 18:01:39,452][20347] Updated weights for policy 0, policy_version 1787318 (0.0008) [2023-12-27 18:01:39,500][20381] Updated weights for policy 1, policy_version 1785744 (0.0007) [2023-12-27 18:01:39,519][20347] Updated weights for policy 0, policy_version 1787328 (0.0008) [2023-12-27 18:01:39,553][20381] Updated weights for policy 1, policy_version 1785754 (0.0008) [2023-12-27 18:01:40,266][20347] Updated weights for policy 0, policy_version 1787338 (0.0007) [2023-12-27 18:01:40,272][20381] Updated weights for policy 1, policy_version 1785764 (0.0007) [2023-12-27 18:01:40,329][20347] Updated weights for policy 0, policy_version 1787348 (0.0006) [2023-12-27 18:01:40,335][20381] Updated weights for policy 1, policy_version 1785774 (0.0007) [2023-12-27 18:01:40,388][20347] Updated weights for policy 0, policy_version 1787358 (0.0007) [2023-12-27 18:01:40,394][20381] Updated weights for policy 1, policy_version 1785784 (0.0006) [2023-12-27 18:01:41,049][20347] Updated weights for policy 0, policy_version 1787368 (0.0008) [2023-12-27 18:01:41,097][20347] Updated weights for policy 0, policy_version 1787378 (0.0008) [2023-12-27 18:01:41,107][20381] Updated weights for policy 1, policy_version 1785794 (0.0007) [2023-12-27 18:01:41,160][20347] Updated weights for policy 0, policy_version 1787388 (0.0007) [2023-12-27 18:01:41,176][20381] Updated weights for policy 1, policy_version 1785804 (0.0007) [2023-12-27 18:01:41,236][20381] Updated weights for policy 1, policy_version 1785814 (0.0008) [2023-12-27 18:01:41,303][20381] Updated weights for policy 1, policy_version 1785824 (0.0008) [2023-12-27 18:01:41,920][20347] Updated weights for policy 0, policy_version 1787398 (0.0007) [2023-12-27 18:01:41,982][20347] Updated weights for policy 0, policy_version 1787408 (0.0006) [2023-12-27 18:01:41,991][20381] Updated weights for policy 1, policy_version 1785834 (0.0006) [2023-12-27 18:01:42,038][20347] Updated weights for policy 0, policy_version 1787418 (0.0007) [2023-12-27 18:01:42,047][20381] Updated weights for policy 1, policy_version 1785844 (0.0006) [2023-12-27 18:01:42,102][20381] Updated weights for policy 1, policy_version 1785854 (0.0007) [2023-12-27 18:01:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 914890752. Throughput: 0: 10197.0, 1: 10362.3. Samples: 914879512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:42,363][19170] Avg episode reward: [(0, '931.970'), (1, '1000.000')] [2023-12-27 18:01:42,753][20347] Updated weights for policy 0, policy_version 1787428 (0.0007) [2023-12-27 18:01:42,787][20381] Updated weights for policy 1, policy_version 1785864 (0.0007) [2023-12-27 18:01:42,816][20347] Updated weights for policy 0, policy_version 1787438 (0.0009) [2023-12-27 18:01:42,846][20381] Updated weights for policy 1, policy_version 1785874 (0.0007) [2023-12-27 18:01:42,873][20347] Updated weights for policy 0, policy_version 1787448 (0.0006) [2023-12-27 18:01:42,904][20381] Updated weights for policy 1, policy_version 1785884 (0.0006) [2023-12-27 18:01:43,522][20347] Updated weights for policy 0, policy_version 1787458 (0.0007) [2023-12-27 18:01:43,577][20347] Updated weights for policy 0, policy_version 1787468 (0.0007) [2023-12-27 18:01:43,605][20381] Updated weights for policy 1, policy_version 1785894 (0.0006) [2023-12-27 18:01:43,630][20347] Updated weights for policy 0, policy_version 1787478 (0.0005) [2023-12-27 18:01:43,661][20381] Updated weights for policy 1, policy_version 1785904 (0.0006) [2023-12-27 18:01:43,675][20347] Updated weights for policy 0, policy_version 1787488 (0.0005) [2023-12-27 18:01:43,719][20381] Updated weights for policy 1, policy_version 1785914 (0.0007) [2023-12-27 18:01:44,369][20381] Updated weights for policy 1, policy_version 1785924 (0.0009) [2023-12-27 18:01:44,397][20347] Updated weights for policy 0, policy_version 1787498 (0.0007) [2023-12-27 18:01:44,425][20381] Updated weights for policy 1, policy_version 1785934 (0.0011) [2023-12-27 18:01:44,453][20347] Updated weights for policy 0, policy_version 1787508 (0.0006) [2023-12-27 18:01:44,475][20381] Updated weights for policy 1, policy_version 1785944 (0.0010) [2023-12-27 18:01:44,511][20347] Updated weights for policy 0, policy_version 1787518 (0.0006) [2023-12-27 18:01:45,146][20381] Updated weights for policy 1, policy_version 1785954 (0.0011) [2023-12-27 18:01:45,194][20381] Updated weights for policy 1, policy_version 1785964 (0.0009) [2023-12-27 18:01:45,234][20347] Updated weights for policy 0, policy_version 1787528 (0.0007) [2023-12-27 18:01:45,244][20381] Updated weights for policy 1, policy_version 1785974 (0.0006) [2023-12-27 18:01:45,297][20347] Updated weights for policy 0, policy_version 1787538 (0.0008) [2023-12-27 18:01:45,299][20381] Updated weights for policy 1, policy_version 1785984 (0.0007) [2023-12-27 18:01:45,359][20347] Updated weights for policy 0, policy_version 1787548 (0.0008) [2023-12-27 18:01:45,986][20347] Updated weights for policy 0, policy_version 1787558 (0.0009) [2023-12-27 18:01:46,036][20381] Updated weights for policy 1, policy_version 1785994 (0.0005) [2023-12-27 18:01:46,039][20347] Updated weights for policy 0, policy_version 1787568 (0.0009) [2023-12-27 18:01:46,082][20381] Updated weights for policy 1, policy_version 1786004 (0.0005) [2023-12-27 18:01:46,094][20347] Updated weights for policy 0, policy_version 1787578 (0.0006) [2023-12-27 18:01:46,130][20381] Updated weights for policy 1, policy_version 1786014 (0.0008) [2023-12-27 18:01:46,713][20347] Updated weights for policy 0, policy_version 1787588 (0.0007) [2023-12-27 18:01:46,769][20347] Updated weights for policy 0, policy_version 1787598 (0.0007) [2023-12-27 18:01:46,821][20347] Updated weights for policy 0, policy_version 1787608 (0.0007) [2023-12-27 18:01:46,876][20381] Updated weights for policy 1, policy_version 1786024 (0.0006) [2023-12-27 18:01:46,927][20381] Updated weights for policy 1, policy_version 1786034 (0.0005) [2023-12-27 18:01:46,972][20381] Updated weights for policy 1, policy_version 1786044 (0.0005) [2023-12-27 18:01:47,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20605.0). Total num frames: 914989056. Throughput: 0: 10253.9, 1: 10374.9. Samples: 915000648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:47,364][19170] Avg episode reward: [(0, '951.330'), (1, '1000.000')] [2023-12-27 18:01:47,508][20381] Updated weights for policy 1, policy_version 1786054 (0.0007) [2023-12-27 18:01:47,549][20347] Updated weights for policy 0, policy_version 1787618 (0.0005) [2023-12-27 18:01:47,562][20381] Updated weights for policy 1, policy_version 1786064 (0.0007) [2023-12-27 18:01:47,594][20347] Updated weights for policy 0, policy_version 1787628 (0.0005) [2023-12-27 18:01:47,614][20381] Updated weights for policy 1, policy_version 1786074 (0.0006) [2023-12-27 18:01:47,639][20347] Updated weights for policy 0, policy_version 1787638 (0.0006) [2023-12-27 18:01:47,696][20347] Updated weights for policy 0, policy_version 1787648 (0.0006) [2023-12-27 18:01:48,245][20347] Updated weights for policy 0, policy_version 1787658 (0.0005) [2023-12-27 18:01:48,292][20347] Updated weights for policy 0, policy_version 1787668 (0.0005) [2023-12-27 18:01:48,298][20381] Updated weights for policy 1, policy_version 1786084 (0.0007) [2023-12-27 18:01:48,361][20347] Updated weights for policy 0, policy_version 1787678 (0.0007) [2023-12-27 18:01:48,364][20381] Updated weights for policy 1, policy_version 1786094 (0.0007) [2023-12-27 18:01:48,419][20381] Updated weights for policy 1, policy_version 1786104 (0.0007) [2023-12-27 18:01:48,914][20347] Updated weights for policy 0, policy_version 1787688 (0.0007) [2023-12-27 18:01:48,974][20347] Updated weights for policy 0, policy_version 1787698 (0.0008) [2023-12-27 18:01:49,032][20347] Updated weights for policy 0, policy_version 1787708 (0.0007) [2023-12-27 18:01:49,134][20381] Updated weights for policy 1, policy_version 1786114 (0.0009) [2023-12-27 18:01:49,188][20381] Updated weights for policy 1, policy_version 1786124 (0.0010) [2023-12-27 18:01:49,253][20381] Updated weights for policy 1, policy_version 1786134 (0.0012) [2023-12-27 18:01:49,318][20381] Updated weights for policy 1, policy_version 1786144 (0.0008) [2023-12-27 18:01:49,832][20347] Updated weights for policy 0, policy_version 1787718 (0.0008) [2023-12-27 18:01:49,893][20347] Updated weights for policy 0, policy_version 1787728 (0.0009) [2023-12-27 18:01:49,958][20347] Updated weights for policy 0, policy_version 1787738 (0.0008) [2023-12-27 18:01:50,143][20381] Updated weights for policy 1, policy_version 1786154 (0.0008) [2023-12-27 18:01:50,191][20381] Updated weights for policy 1, policy_version 1786164 (0.0008) [2023-12-27 18:01:50,245][20381] Updated weights for policy 1, policy_version 1786174 (0.0008) [2023-12-27 18:01:50,670][20347] Updated weights for policy 0, policy_version 1787748 (0.0007) [2023-12-27 18:01:50,729][20347] Updated weights for policy 0, policy_version 1787758 (0.0007) [2023-12-27 18:01:50,791][20347] Updated weights for policy 0, policy_version 1787768 (0.0007) [2023-12-27 18:01:50,985][20381] Updated weights for policy 1, policy_version 1786184 (0.0008) [2023-12-27 18:01:51,047][20381] Updated weights for policy 1, policy_version 1786194 (0.0008) [2023-12-27 18:01:51,112][20381] Updated weights for policy 1, policy_version 1786204 (0.0008) [2023-12-27 18:01:51,449][20347] Updated weights for policy 0, policy_version 1787778 (0.0006) [2023-12-27 18:01:51,506][20347] Updated weights for policy 0, policy_version 1787788 (0.0008) [2023-12-27 18:01:51,563][20347] Updated weights for policy 0, policy_version 1787798 (0.0007) [2023-12-27 18:01:51,624][20347] Updated weights for policy 0, policy_version 1787808 (0.0008) [2023-12-27 18:01:51,826][20381] Updated weights for policy 1, policy_version 1786214 (0.0010) [2023-12-27 18:01:51,885][20381] Updated weights for policy 1, policy_version 1786224 (0.0011) [2023-12-27 18:01:51,947][20381] Updated weights for policy 1, policy_version 1786234 (0.0010) [2023-12-27 18:01:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 915087360. Throughput: 0: 10205.0, 1: 10332.8. Samples: 915060924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:52,363][19170] Avg episode reward: [(0, '980.590'), (1, '1000.000')] [2023-12-27 18:01:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001786240_457326592.pth... [2023-12-27 18:01:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001785056_457023488.pth [2023-12-27 18:01:52,372][20347] Updated weights for policy 0, policy_version 1787818 (0.0009) [2023-12-27 18:01:52,422][20347] Updated weights for policy 0, policy_version 1787828 (0.0009) [2023-12-27 18:01:52,469][20347] Updated weights for policy 0, policy_version 1787838 (0.0009) [2023-12-27 18:01:52,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001787840_457768960.pth... [2023-12-27 18:01:52,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001786624_457457664.pth [2023-12-27 18:01:52,521][20381] Updated weights for policy 1, policy_version 1786244 (0.0007) [2023-12-27 18:01:52,584][20381] Updated weights for policy 1, policy_version 1786254 (0.0011) [2023-12-27 18:01:52,649][20381] Updated weights for policy 1, policy_version 1786264 (0.0007) [2023-12-27 18:01:53,209][20347] Updated weights for policy 0, policy_version 1787848 (0.0008) [2023-12-27 18:01:53,254][20347] Updated weights for policy 0, policy_version 1787858 (0.0008) [2023-12-27 18:01:53,299][20381] Updated weights for policy 1, policy_version 1786274 (0.0007) [2023-12-27 18:01:53,306][20347] Updated weights for policy 0, policy_version 1787868 (0.0007) [2023-12-27 18:01:53,356][20381] Updated weights for policy 1, policy_version 1786284 (0.0005) [2023-12-27 18:01:53,410][20381] Updated weights for policy 1, policy_version 1786294 (0.0007) [2023-12-27 18:01:53,465][20381] Updated weights for policy 1, policy_version 1786304 (0.0008) [2023-12-27 18:01:54,012][20347] Updated weights for policy 0, policy_version 1787878 (0.0007) [2023-12-27 18:01:54,067][20347] Updated weights for policy 0, policy_version 1787888 (0.0008) [2023-12-27 18:01:54,124][20381] Updated weights for policy 1, policy_version 1786314 (0.0010) [2023-12-27 18:01:54,125][20347] Updated weights for policy 0, policy_version 1787898 (0.0005) [2023-12-27 18:01:54,173][20381] Updated weights for policy 1, policy_version 1786324 (0.0010) [2023-12-27 18:01:54,230][20381] Updated weights for policy 1, policy_version 1786334 (0.0010) [2023-12-27 18:01:54,789][20347] Updated weights for policy 0, policy_version 1787908 (0.0007) [2023-12-27 18:01:54,838][20347] Updated weights for policy 0, policy_version 1787918 (0.0008) [2023-12-27 18:01:54,885][20347] Updated weights for policy 0, policy_version 1787928 (0.0007) [2023-12-27 18:01:54,928][20381] Updated weights for policy 1, policy_version 1786344 (0.0006) [2023-12-27 18:01:54,992][20381] Updated weights for policy 1, policy_version 1786354 (0.0008) [2023-12-27 18:01:55,052][20381] Updated weights for policy 1, policy_version 1786364 (0.0007) [2023-12-27 18:01:55,508][20347] Updated weights for policy 0, policy_version 1787938 (0.0007) [2023-12-27 18:01:55,561][20347] Updated weights for policy 0, policy_version 1787948 (0.0006) [2023-12-27 18:01:55,610][20347] Updated weights for policy 0, policy_version 1787958 (0.0008) [2023-12-27 18:01:55,647][20381] Updated weights for policy 1, policy_version 1786374 (0.0006) [2023-12-27 18:01:55,665][20347] Updated weights for policy 0, policy_version 1787968 (0.0008) [2023-12-27 18:01:55,696][20381] Updated weights for policy 1, policy_version 1786384 (0.0005) [2023-12-27 18:01:55,744][20381] Updated weights for policy 1, policy_version 1786394 (0.0005) [2023-12-27 18:01:56,269][20347] Updated weights for policy 0, policy_version 1787978 (0.0007) [2023-12-27 18:01:56,332][20347] Updated weights for policy 0, policy_version 1787988 (0.0007) [2023-12-27 18:01:56,393][20347] Updated weights for policy 0, policy_version 1787998 (0.0006) [2023-12-27 18:01:56,445][20381] Updated weights for policy 1, policy_version 1786404 (0.0006) [2023-12-27 18:01:56,498][20381] Updated weights for policy 1, policy_version 1786414 (0.0005) [2023-12-27 18:01:56,553][20381] Updated weights for policy 1, policy_version 1786424 (0.0005) [2023-12-27 18:01:57,087][20347] Updated weights for policy 0, policy_version 1788008 (0.0008) [2023-12-27 18:01:57,149][20347] Updated weights for policy 0, policy_version 1788018 (0.0007) [2023-12-27 18:01:57,199][20381] Updated weights for policy 1, policy_version 1786434 (0.0008) [2023-12-27 18:01:57,207][20347] Updated weights for policy 0, policy_version 1788028 (0.0008) [2023-12-27 18:01:57,255][20381] Updated weights for policy 1, policy_version 1786444 (0.0005) [2023-12-27 18:01:57,316][20381] Updated weights for policy 1, policy_version 1786454 (0.0009) [2023-12-27 18:01:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 915193856. Throughput: 0: 10183.8, 1: 10328.7. Samples: 915185024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:01:57,363][19170] Avg episode reward: [(0, '990.290'), (1, '1000.000')] [2023-12-27 18:01:57,372][20381] Updated weights for policy 1, policy_version 1786464 (0.0008) [2023-12-27 18:01:57,918][20381] Updated weights for policy 1, policy_version 1786474 (0.0005) [2023-12-27 18:01:57,952][20347] Updated weights for policy 0, policy_version 1788038 (0.0007) [2023-12-27 18:01:57,978][20381] Updated weights for policy 1, policy_version 1786484 (0.0006) [2023-12-27 18:01:58,007][20347] Updated weights for policy 0, policy_version 1788048 (0.0006) [2023-12-27 18:01:58,035][20381] Updated weights for policy 1, policy_version 1786494 (0.0011) [2023-12-27 18:01:58,066][20347] Updated weights for policy 0, policy_version 1788058 (0.0006) [2023-12-27 18:01:58,789][20381] Updated weights for policy 1, policy_version 1786504 (0.0008) [2023-12-27 18:01:58,853][20381] Updated weights for policy 1, policy_version 1786514 (0.0008) [2023-12-27 18:01:58,872][20347] Updated weights for policy 0, policy_version 1788068 (0.0008) [2023-12-27 18:01:58,915][20381] Updated weights for policy 1, policy_version 1786524 (0.0010) [2023-12-27 18:01:58,932][20347] Updated weights for policy 0, policy_version 1788078 (0.0009) [2023-12-27 18:01:59,001][20347] Updated weights for policy 0, policy_version 1788088 (0.0007) [2023-12-27 18:01:59,647][20381] Updated weights for policy 1, policy_version 1786534 (0.0011) [2023-12-27 18:01:59,683][20347] Updated weights for policy 0, policy_version 1788098 (0.0009) [2023-12-27 18:01:59,695][20381] Updated weights for policy 1, policy_version 1786544 (0.0010) [2023-12-27 18:01:59,743][20347] Updated weights for policy 0, policy_version 1788108 (0.0007) [2023-12-27 18:01:59,747][20381] Updated weights for policy 1, policy_version 1786554 (0.0011) [2023-12-27 18:01:59,799][20347] Updated weights for policy 0, policy_version 1788118 (0.0006) [2023-12-27 18:01:59,858][20347] Updated weights for policy 0, policy_version 1788128 (0.0007) [2023-12-27 18:02:00,473][20381] Updated weights for policy 1, policy_version 1786564 (0.0011) [2023-12-27 18:02:00,527][20381] Updated weights for policy 1, policy_version 1786574 (0.0011) [2023-12-27 18:02:00,531][20347] Updated weights for policy 0, policy_version 1788138 (0.0007) [2023-12-27 18:02:00,580][20381] Updated weights for policy 1, policy_version 1786584 (0.0011) [2023-12-27 18:02:00,582][20347] Updated weights for policy 0, policy_version 1788148 (0.0007) [2023-12-27 18:02:00,638][20347] Updated weights for policy 0, policy_version 1788158 (0.0006) [2023-12-27 18:02:01,194][20381] Updated weights for policy 1, policy_version 1786594 (0.0010) [2023-12-27 18:02:01,251][20381] Updated weights for policy 1, policy_version 1786604 (0.0007) [2023-12-27 18:02:01,306][20381] Updated weights for policy 1, policy_version 1786614 (0.0007) [2023-12-27 18:02:01,368][20381] Updated weights for policy 1, policy_version 1786624 (0.0007) [2023-12-27 18:02:01,384][20347] Updated weights for policy 0, policy_version 1788168 (0.0010) [2023-12-27 18:02:01,443][20347] Updated weights for policy 0, policy_version 1788178 (0.0008) [2023-12-27 18:02:01,504][20347] Updated weights for policy 0, policy_version 1788188 (0.0008) [2023-12-27 18:02:02,039][20381] Updated weights for policy 1, policy_version 1786634 (0.0008) [2023-12-27 18:02:02,088][20381] Updated weights for policy 1, policy_version 1786644 (0.0008) [2023-12-27 18:02:02,139][20381] Updated weights for policy 1, policy_version 1786654 (0.0008) [2023-12-27 18:02:02,181][20347] Updated weights for policy 0, policy_version 1788198 (0.0006) [2023-12-27 18:02:02,240][20347] Updated weights for policy 0, policy_version 1788208 (0.0008) [2023-12-27 18:02:02,303][20347] Updated weights for policy 0, policy_version 1788218 (0.0007) [2023-12-27 18:02:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 915300352. Throughput: 0: 10141.7, 1: 10318.9. Samples: 915304676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:02,363][19170] Avg episode reward: [(0, '990.230'), (1, '1000.000')] [2023-12-27 18:02:02,864][20381] Updated weights for policy 1, policy_version 1786664 (0.0008) [2023-12-27 18:02:02,917][20381] Updated weights for policy 1, policy_version 1786674 (0.0006) [2023-12-27 18:02:02,965][20381] Updated weights for policy 1, policy_version 1786684 (0.0007) [2023-12-27 18:02:03,006][20347] Updated weights for policy 0, policy_version 1788228 (0.0007) [2023-12-27 18:02:03,063][20347] Updated weights for policy 0, policy_version 1788238 (0.0005) [2023-12-27 18:02:03,118][20347] Updated weights for policy 0, policy_version 1788248 (0.0005) [2023-12-27 18:02:03,589][20381] Updated weights for policy 1, policy_version 1786694 (0.0007) [2023-12-27 18:02:03,643][20381] Updated weights for policy 1, policy_version 1786704 (0.0007) [2023-12-27 18:02:03,682][20347] Updated weights for policy 0, policy_version 1788258 (0.0006) [2023-12-27 18:02:03,695][20381] Updated weights for policy 1, policy_version 1786714 (0.0007) [2023-12-27 18:02:03,727][20347] Updated weights for policy 0, policy_version 1788268 (0.0010) [2023-12-27 18:02:03,775][20347] Updated weights for policy 0, policy_version 1788278 (0.0010) [2023-12-27 18:02:03,819][20347] Updated weights for policy 0, policy_version 1788288 (0.0010) [2023-12-27 18:02:04,358][20381] Updated weights for policy 1, policy_version 1786724 (0.0005) [2023-12-27 18:02:04,419][20381] Updated weights for policy 1, policy_version 1786734 (0.0006) [2023-12-27 18:02:04,478][20381] Updated weights for policy 1, policy_version 1786744 (0.0006) [2023-12-27 18:02:04,521][20347] Updated weights for policy 0, policy_version 1788298 (0.0005) [2023-12-27 18:02:04,577][20347] Updated weights for policy 0, policy_version 1788308 (0.0006) [2023-12-27 18:02:04,635][20347] Updated weights for policy 0, policy_version 1788318 (0.0007) [2023-12-27 18:02:05,033][20381] Updated weights for policy 1, policy_version 1786754 (0.0007) [2023-12-27 18:02:05,091][20381] Updated weights for policy 1, policy_version 1786764 (0.0009) [2023-12-27 18:02:05,139][20381] Updated weights for policy 1, policy_version 1786774 (0.0007) [2023-12-27 18:02:05,170][20347] Updated weights for policy 0, policy_version 1788328 (0.0006) [2023-12-27 18:02:05,188][20381] Updated weights for policy 1, policy_version 1786784 (0.0007) [2023-12-27 18:02:05,224][20347] Updated weights for policy 0, policy_version 1788338 (0.0005) [2023-12-27 18:02:05,278][20347] Updated weights for policy 0, policy_version 1788348 (0.0006) [2023-12-27 18:02:05,847][20347] Updated weights for policy 0, policy_version 1788358 (0.0007) [2023-12-27 18:02:05,897][20347] Updated weights for policy 0, policy_version 1788368 (0.0007) [2023-12-27 18:02:05,946][20381] Updated weights for policy 1, policy_version 1786794 (0.0008) [2023-12-27 18:02:05,955][20347] Updated weights for policy 0, policy_version 1788378 (0.0007) [2023-12-27 18:02:05,996][20381] Updated weights for policy 1, policy_version 1786804 (0.0006) [2023-12-27 18:02:06,045][20381] Updated weights for policy 1, policy_version 1786814 (0.0008) [2023-12-27 18:02:06,717][20347] Updated weights for policy 0, policy_version 1788388 (0.0007) [2023-12-27 18:02:06,737][20381] Updated weights for policy 1, policy_version 1786824 (0.0007) [2023-12-27 18:02:06,780][20347] Updated weights for policy 0, policy_version 1788398 (0.0008) [2023-12-27 18:02:06,800][20381] Updated weights for policy 1, policy_version 1786834 (0.0007) [2023-12-27 18:02:06,845][20347] Updated weights for policy 0, policy_version 1788408 (0.0010) [2023-12-27 18:02:06,859][20381] Updated weights for policy 1, policy_version 1786844 (0.0007) [2023-12-27 18:02:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 915398656. Throughput: 0: 10195.0, 1: 10338.6. Samples: 915369488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:07,363][19170] Avg episode reward: [(0, '990.230'), (1, '1000.000')] [2023-12-27 18:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001786848_457482240.pth... [2023-12-27 18:02:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001788416_457916416.pth... [2023-12-27 18:02:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001785664_457179136.pth [2023-12-27 18:02:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001787232_457613312.pth [2023-12-27 18:02:07,498][20347] Updated weights for policy 0, policy_version 1788418 (0.0007) [2023-12-27 18:02:07,554][20347] Updated weights for policy 0, policy_version 1788428 (0.0007) [2023-12-27 18:02:07,571][20381] Updated weights for policy 1, policy_version 1786854 (0.0008) [2023-12-27 18:02:07,614][20347] Updated weights for policy 0, policy_version 1788438 (0.0006) [2023-12-27 18:02:07,631][20381] Updated weights for policy 1, policy_version 1786864 (0.0007) [2023-12-27 18:02:07,669][20347] Updated weights for policy 0, policy_version 1788448 (0.0008) [2023-12-27 18:02:07,683][20381] Updated weights for policy 1, policy_version 1786874 (0.0006) [2023-12-27 18:02:08,314][20347] Updated weights for policy 0, policy_version 1788458 (0.0007) [2023-12-27 18:02:08,379][20347] Updated weights for policy 0, policy_version 1788468 (0.0008) [2023-12-27 18:02:08,397][20381] Updated weights for policy 1, policy_version 1786884 (0.0007) [2023-12-27 18:02:08,439][20347] Updated weights for policy 0, policy_version 1788478 (0.0007) [2023-12-27 18:02:08,463][20381] Updated weights for policy 1, policy_version 1786894 (0.0008) [2023-12-27 18:02:08,513][20381] Updated weights for policy 1, policy_version 1786904 (0.0007) [2023-12-27 18:02:09,108][20381] Updated weights for policy 1, policy_version 1786914 (0.0008) [2023-12-27 18:02:09,150][20347] Updated weights for policy 0, policy_version 1788488 (0.0006) [2023-12-27 18:02:09,165][20381] Updated weights for policy 1, policy_version 1786924 (0.0009) [2023-12-27 18:02:09,208][20347] Updated weights for policy 0, policy_version 1788498 (0.0006) [2023-12-27 18:02:09,223][20381] Updated weights for policy 1, policy_version 1786934 (0.0008) [2023-12-27 18:02:09,273][20347] Updated weights for policy 0, policy_version 1788508 (0.0008) [2023-12-27 18:02:09,286][20381] Updated weights for policy 1, policy_version 1786944 (0.0007) [2023-12-27 18:02:10,014][20347] Updated weights for policy 0, policy_version 1788518 (0.0007) [2023-12-27 18:02:10,023][20381] Updated weights for policy 1, policy_version 1786954 (0.0008) [2023-12-27 18:02:10,071][20347] Updated weights for policy 0, policy_version 1788528 (0.0007) [2023-12-27 18:02:10,085][20381] Updated weights for policy 1, policy_version 1786964 (0.0006) [2023-12-27 18:02:10,127][20347] Updated weights for policy 0, policy_version 1788538 (0.0007) [2023-12-27 18:02:10,143][20381] Updated weights for policy 1, policy_version 1786974 (0.0005) [2023-12-27 18:02:10,795][20381] Updated weights for policy 1, policy_version 1786984 (0.0007) [2023-12-27 18:02:10,815][20347] Updated weights for policy 0, policy_version 1788548 (0.0009) [2023-12-27 18:02:10,846][20381] Updated weights for policy 1, policy_version 1786994 (0.0009) [2023-12-27 18:02:10,871][20347] Updated weights for policy 0, policy_version 1788558 (0.0011) [2023-12-27 18:02:10,901][20381] Updated weights for policy 1, policy_version 1787004 (0.0010) [2023-12-27 18:02:10,922][20347] Updated weights for policy 0, policy_version 1788568 (0.0010) [2023-12-27 18:02:11,644][20381] Updated weights for policy 1, policy_version 1787014 (0.0009) [2023-12-27 18:02:11,682][20347] Updated weights for policy 0, policy_version 1788578 (0.0006) [2023-12-27 18:02:11,708][20381] Updated weights for policy 1, policy_version 1787024 (0.0008) [2023-12-27 18:02:11,748][20347] Updated weights for policy 0, policy_version 1788588 (0.0008) [2023-12-27 18:02:11,766][20381] Updated weights for policy 1, policy_version 1787034 (0.0007) [2023-12-27 18:02:11,800][20347] Updated weights for policy 0, policy_version 1788598 (0.0006) [2023-12-27 18:02:11,861][20347] Updated weights for policy 0, policy_version 1788608 (0.0008) [2023-12-27 18:02:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 915496960. Throughput: 0: 10242.1, 1: 10275.7. Samples: 915491220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:12,363][19170] Avg episode reward: [(0, '990.280'), (1, '1000.000')] [2023-12-27 18:02:12,506][20347] Updated weights for policy 0, policy_version 1788618 (0.0007) [2023-12-27 18:02:12,542][20381] Updated weights for policy 1, policy_version 1787044 (0.0007) [2023-12-27 18:02:12,571][20347] Updated weights for policy 0, policy_version 1788628 (0.0006) [2023-12-27 18:02:12,604][20381] Updated weights for policy 1, policy_version 1787054 (0.0007) [2023-12-27 18:02:12,625][20347] Updated weights for policy 0, policy_version 1788638 (0.0006) [2023-12-27 18:02:12,665][20381] Updated weights for policy 1, policy_version 1787064 (0.0008) [2023-12-27 18:02:13,342][20381] Updated weights for policy 1, policy_version 1787074 (0.0005) [2023-12-27 18:02:13,350][20347] Updated weights for policy 0, policy_version 1788648 (0.0008) [2023-12-27 18:02:13,392][20381] Updated weights for policy 1, policy_version 1787084 (0.0008) [2023-12-27 18:02:13,408][20347] Updated weights for policy 0, policy_version 1788658 (0.0005) [2023-12-27 18:02:13,438][20381] Updated weights for policy 1, policy_version 1787094 (0.0008) [2023-12-27 18:02:13,466][20347] Updated weights for policy 0, policy_version 1788668 (0.0005) [2023-12-27 18:02:13,487][20381] Updated weights for policy 1, policy_version 1787104 (0.0008) [2023-12-27 18:02:14,129][20347] Updated weights for policy 0, policy_version 1788678 (0.0007) [2023-12-27 18:02:14,172][20381] Updated weights for policy 1, policy_version 1787114 (0.0005) [2023-12-27 18:02:14,182][20347] Updated weights for policy 0, policy_version 1788688 (0.0007) [2023-12-27 18:02:14,225][20381] Updated weights for policy 1, policy_version 1787124 (0.0005) [2023-12-27 18:02:14,234][20347] Updated weights for policy 0, policy_version 1788698 (0.0008) [2023-12-27 18:02:14,285][20381] Updated weights for policy 1, policy_version 1787134 (0.0006) [2023-12-27 18:02:14,929][20381] Updated weights for policy 1, policy_version 1787144 (0.0007) [2023-12-27 18:02:14,963][20347] Updated weights for policy 0, policy_version 1788708 (0.0007) [2023-12-27 18:02:14,987][20381] Updated weights for policy 1, policy_version 1787154 (0.0008) [2023-12-27 18:02:15,027][20347] Updated weights for policy 0, policy_version 1788718 (0.0007) [2023-12-27 18:02:15,053][20381] Updated weights for policy 1, policy_version 1787164 (0.0009) [2023-12-27 18:02:15,091][20347] Updated weights for policy 0, policy_version 1788728 (0.0008) [2023-12-27 18:02:15,655][20381] Updated weights for policy 1, policy_version 1787174 (0.0006) [2023-12-27 18:02:15,715][20381] Updated weights for policy 1, policy_version 1787184 (0.0005) [2023-12-27 18:02:15,773][20381] Updated weights for policy 1, policy_version 1787194 (0.0005) [2023-12-27 18:02:15,785][20347] Updated weights for policy 0, policy_version 1788738 (0.0007) [2023-12-27 18:02:15,834][20347] Updated weights for policy 0, policy_version 1788748 (0.0008) [2023-12-27 18:02:15,886][20347] Updated weights for policy 0, policy_version 1788758 (0.0006) [2023-12-27 18:02:15,941][20347] Updated weights for policy 0, policy_version 1788768 (0.0007) [2023-12-27 18:02:16,338][20381] Updated weights for policy 1, policy_version 1787204 (0.0005) [2023-12-27 18:02:16,403][20381] Updated weights for policy 1, policy_version 1787214 (0.0007) [2023-12-27 18:02:16,467][20381] Updated weights for policy 1, policy_version 1787224 (0.0005) [2023-12-27 18:02:16,585][20347] Updated weights for policy 0, policy_version 1788778 (0.0006) [2023-12-27 18:02:16,643][20347] Updated weights for policy 0, policy_version 1788788 (0.0008) [2023-12-27 18:02:16,707][20347] Updated weights for policy 0, policy_version 1788798 (0.0006) [2023-12-27 18:02:17,026][20381] Updated weights for policy 1, policy_version 1787234 (0.0006) [2023-12-27 18:02:17,081][20381] Updated weights for policy 1, policy_version 1787244 (0.0008) [2023-12-27 18:02:17,134][20381] Updated weights for policy 1, policy_version 1787254 (0.0007) [2023-12-27 18:02:17,189][20381] Updated weights for policy 1, policy_version 1787264 (0.0005) [2023-12-27 18:02:17,300][20347] Updated weights for policy 0, policy_version 1788808 (0.0008) [2023-12-27 18:02:17,358][20347] Updated weights for policy 0, policy_version 1788818 (0.0009) [2023-12-27 18:02:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 915603456. Throughput: 0: 10234.4, 1: 10277.4. Samples: 915615052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:17,363][19170] Avg episode reward: [(0, '961.010'), (1, '1000.000')] [2023-12-27 18:02:17,406][20347] Updated weights for policy 0, policy_version 1788828 (0.0009) [2023-12-27 18:02:17,808][20381] Updated weights for policy 1, policy_version 1787274 (0.0008) [2023-12-27 18:02:17,861][20381] Updated weights for policy 1, policy_version 1787284 (0.0009) [2023-12-27 18:02:17,903][20381] Updated weights for policy 1, policy_version 1787294 (0.0007) [2023-12-27 18:02:18,132][20347] Updated weights for policy 0, policy_version 1788838 (0.0008) [2023-12-27 18:02:18,177][20347] Updated weights for policy 0, policy_version 1788848 (0.0008) [2023-12-27 18:02:18,223][20347] Updated weights for policy 0, policy_version 1788858 (0.0008) [2023-12-27 18:02:18,561][20381] Updated weights for policy 1, policy_version 1787304 (0.0006) [2023-12-27 18:02:18,616][20381] Updated weights for policy 1, policy_version 1787314 (0.0007) [2023-12-27 18:02:18,675][20381] Updated weights for policy 1, policy_version 1787324 (0.0008) [2023-12-27 18:02:18,964][20347] Updated weights for policy 0, policy_version 1788868 (0.0008) [2023-12-27 18:02:19,023][20347] Updated weights for policy 0, policy_version 1788878 (0.0008) [2023-12-27 18:02:19,083][20347] Updated weights for policy 0, policy_version 1788888 (0.0007) [2023-12-27 18:02:19,338][20381] Updated weights for policy 1, policy_version 1787334 (0.0008) [2023-12-27 18:02:19,408][20381] Updated weights for policy 1, policy_version 1787344 (0.0009) [2023-12-27 18:02:19,478][20381] Updated weights for policy 1, policy_version 1787354 (0.0008) [2023-12-27 18:02:19,821][20347] Updated weights for policy 0, policy_version 1788898 (0.0008) [2023-12-27 18:02:19,886][20347] Updated weights for policy 0, policy_version 1788908 (0.0007) [2023-12-27 18:02:19,951][20347] Updated weights for policy 0, policy_version 1788918 (0.0008) [2023-12-27 18:02:20,011][20347] Updated weights for policy 0, policy_version 1788928 (0.0007) [2023-12-27 18:02:20,215][20381] Updated weights for policy 1, policy_version 1787364 (0.0008) [2023-12-27 18:02:20,277][20381] Updated weights for policy 1, policy_version 1787374 (0.0008) [2023-12-27 18:02:20,340][20381] Updated weights for policy 1, policy_version 1787384 (0.0007) [2023-12-27 18:02:20,694][20347] Updated weights for policy 0, policy_version 1788938 (0.0006) [2023-12-27 18:02:20,751][20347] Updated weights for policy 0, policy_version 1788948 (0.0006) [2023-12-27 18:02:20,811][20347] Updated weights for policy 0, policy_version 1788958 (0.0007) [2023-12-27 18:02:21,063][20381] Updated weights for policy 1, policy_version 1787394 (0.0008) [2023-12-27 18:02:21,123][20381] Updated weights for policy 1, policy_version 1787404 (0.0008) [2023-12-27 18:02:21,190][20381] Updated weights for policy 1, policy_version 1787414 (0.0009) [2023-12-27 18:02:21,497][20347] Updated weights for policy 0, policy_version 1788968 (0.0007) [2023-12-27 18:02:21,558][20347] Updated weights for policy 0, policy_version 1788978 (0.0008) [2023-12-27 18:02:21,622][20347] Updated weights for policy 0, policy_version 1788988 (0.0007) [2023-12-27 18:02:21,941][20381] Updated weights for policy 1, policy_version 1787425 (0.0009) [2023-12-27 18:02:22,004][20381] Updated weights for policy 1, policy_version 1787435 (0.0008) [2023-12-27 18:02:22,061][20381] Updated weights for policy 1, policy_version 1787445 (0.0006) [2023-12-27 18:02:22,118][20381] Updated weights for policy 1, policy_version 1787455 (0.0007) [2023-12-27 18:02:22,355][20347] Updated weights for policy 0, policy_version 1788998 (0.0008) [2023-12-27 18:02:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20549.4). Total num frames: 915701760. Throughput: 0: 10142.0, 1: 10308.7. Samples: 915674936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:22,363][19170] Avg episode reward: [(0, '970.770'), (1, '1000.000')] [2023-12-27 18:02:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001787456_457637888.pth... [2023-12-27 18:02:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001786240_457326592.pth [2023-12-27 18:02:22,423][20347] Updated weights for policy 0, policy_version 1789008 (0.0009) [2023-12-27 18:02:22,479][20347] Updated weights for policy 0, policy_version 1789018 (0.0007) [2023-12-27 18:02:22,512][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001789024_458072064.pth... [2023-12-27 18:02:22,517][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001787840_457768960.pth [2023-12-27 18:02:22,830][20381] Updated weights for policy 1, policy_version 1787465 (0.0008) [2023-12-27 18:02:22,887][20381] Updated weights for policy 1, policy_version 1787475 (0.0007) [2023-12-27 18:02:22,940][20381] Updated weights for policy 1, policy_version 1787485 (0.0008) [2023-12-27 18:02:23,185][20347] Updated weights for policy 0, policy_version 1789028 (0.0008) [2023-12-27 18:02:23,233][20347] Updated weights for policy 0, policy_version 1789038 (0.0007) [2023-12-27 18:02:23,281][20347] Updated weights for policy 0, policy_version 1789048 (0.0006) [2023-12-27 18:02:23,613][20381] Updated weights for policy 1, policy_version 1787495 (0.0007) [2023-12-27 18:02:23,662][20381] Updated weights for policy 1, policy_version 1787505 (0.0008) [2023-12-27 18:02:23,711][20381] Updated weights for policy 1, policy_version 1787515 (0.0008) [2023-12-27 18:02:23,984][20347] Updated weights for policy 0, policy_version 1789058 (0.0008) [2023-12-27 18:02:24,030][20347] Updated weights for policy 0, policy_version 1789068 (0.0008) [2023-12-27 18:02:24,076][20347] Updated weights for policy 0, policy_version 1789078 (0.0008) [2023-12-27 18:02:24,125][20347] Updated weights for policy 0, policy_version 1789088 (0.0007) [2023-12-27 18:02:24,409][20381] Updated weights for policy 1, policy_version 1787525 (0.0007) [2023-12-27 18:02:24,464][20381] Updated weights for policy 1, policy_version 1787535 (0.0006) [2023-12-27 18:02:24,523][20381] Updated weights for policy 1, policy_version 1787545 (0.0008) [2023-12-27 18:02:24,897][20347] Updated weights for policy 0, policy_version 1789098 (0.0008) [2023-12-27 18:02:24,956][20347] Updated weights for policy 0, policy_version 1789108 (0.0007) [2023-12-27 18:02:25,010][20347] Updated weights for policy 0, policy_version 1789118 (0.0007) [2023-12-27 18:02:25,128][20381] Updated weights for policy 1, policy_version 1787555 (0.0006) [2023-12-27 18:02:25,186][20381] Updated weights for policy 1, policy_version 1787565 (0.0007) [2023-12-27 18:02:25,242][20381] Updated weights for policy 1, policy_version 1787575 (0.0006) [2023-12-27 18:02:25,726][20347] Updated weights for policy 0, policy_version 1789128 (0.0008) [2023-12-27 18:02:25,775][20347] Updated weights for policy 0, policy_version 1789138 (0.0008) [2023-12-27 18:02:25,819][20347] Updated weights for policy 0, policy_version 1789148 (0.0007) [2023-12-27 18:02:25,852][20381] Updated weights for policy 1, policy_version 1787585 (0.0006) [2023-12-27 18:02:25,906][20381] Updated weights for policy 1, policy_version 1787595 (0.0007) [2023-12-27 18:02:25,962][20381] Updated weights for policy 1, policy_version 1787605 (0.0008) [2023-12-27 18:02:26,017][20381] Updated weights for policy 1, policy_version 1787615 (0.0008) [2023-12-27 18:02:26,544][20347] Updated weights for policy 0, policy_version 1789158 (0.0008) [2023-12-27 18:02:26,596][20347] Updated weights for policy 0, policy_version 1789168 (0.0007) [2023-12-27 18:02:26,644][20347] Updated weights for policy 0, policy_version 1789178 (0.0007) [2023-12-27 18:02:26,703][20381] Updated weights for policy 1, policy_version 1787625 (0.0007) [2023-12-27 18:02:26,748][20381] Updated weights for policy 1, policy_version 1787635 (0.0005) [2023-12-27 18:02:26,796][20381] Updated weights for policy 1, policy_version 1787645 (0.0006) [2023-12-27 18:02:27,298][20347] Updated weights for policy 0, policy_version 1789188 (0.0007) [2023-12-27 18:02:27,353][20347] Updated weights for policy 0, policy_version 1789198 (0.0008) [2023-12-27 18:02:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 915800064. Throughput: 0: 10075.5, 1: 10249.4. Samples: 915794132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:27,363][19170] Avg episode reward: [(0, '990.280'), (1, '1000.000')] [2023-12-27 18:02:27,402][20347] Updated weights for policy 0, policy_version 1789208 (0.0008) [2023-12-27 18:02:27,484][20381] Updated weights for policy 1, policy_version 1787655 (0.0007) [2023-12-27 18:02:27,546][20381] Updated weights for policy 1, policy_version 1787665 (0.0007) [2023-12-27 18:02:27,605][20381] Updated weights for policy 1, policy_version 1787675 (0.0007) [2023-12-27 18:02:28,119][20347] Updated weights for policy 0, policy_version 1789218 (0.0007) [2023-12-27 18:02:28,175][20347] Updated weights for policy 0, policy_version 1789228 (0.0007) [2023-12-27 18:02:28,233][20347] Updated weights for policy 0, policy_version 1789238 (0.0007) [2023-12-27 18:02:28,292][20347] Updated weights for policy 0, policy_version 1789248 (0.0007) [2023-12-27 18:02:28,296][20381] Updated weights for policy 1, policy_version 1787685 (0.0008) [2023-12-27 18:02:28,354][20381] Updated weights for policy 1, policy_version 1787695 (0.0008) [2023-12-27 18:02:28,413][20381] Updated weights for policy 1, policy_version 1787705 (0.0008) [2023-12-27 18:02:29,010][20347] Updated weights for policy 0, policy_version 1789258 (0.0008) [2023-12-27 18:02:29,072][20347] Updated weights for policy 0, policy_version 1789268 (0.0007) [2023-12-27 18:02:29,100][20381] Updated weights for policy 1, policy_version 1787715 (0.0008) [2023-12-27 18:02:29,121][20347] Updated weights for policy 0, policy_version 1789278 (0.0006) [2023-12-27 18:02:29,154][20381] Updated weights for policy 1, policy_version 1787725 (0.0006) [2023-12-27 18:02:29,206][20381] Updated weights for policy 1, policy_version 1787735 (0.0008) [2023-12-27 18:02:29,783][20347] Updated weights for policy 0, policy_version 1789288 (0.0007) [2023-12-27 18:02:29,847][20347] Updated weights for policy 0, policy_version 1789298 (0.0007) [2023-12-27 18:02:29,909][20347] Updated weights for policy 0, policy_version 1789308 (0.0008) [2023-12-27 18:02:29,987][20381] Updated weights for policy 1, policy_version 1787745 (0.0009) [2023-12-27 18:02:30,044][20381] Updated weights for policy 1, policy_version 1787755 (0.0008) [2023-12-27 18:02:30,094][20381] Updated weights for policy 1, policy_version 1787765 (0.0007) [2023-12-27 18:02:30,144][20381] Updated weights for policy 1, policy_version 1787775 (0.0007) [2023-12-27 18:02:30,610][20347] Updated weights for policy 0, policy_version 1789318 (0.0007) [2023-12-27 18:02:30,654][20347] Updated weights for policy 0, policy_version 1789328 (0.0008) [2023-12-27 18:02:30,700][20347] Updated weights for policy 0, policy_version 1789338 (0.0007) [2023-12-27 18:02:30,815][20381] Updated weights for policy 1, policy_version 1787785 (0.0010) [2023-12-27 18:02:30,871][20381] Updated weights for policy 1, policy_version 1787795 (0.0007) [2023-12-27 18:02:30,931][20381] Updated weights for policy 1, policy_version 1787805 (0.0005) [2023-12-27 18:02:31,459][20347] Updated weights for policy 0, policy_version 1789348 (0.0008) [2023-12-27 18:02:31,514][20347] Updated weights for policy 0, policy_version 1789358 (0.0010) [2023-12-27 18:02:31,572][20347] Updated weights for policy 0, policy_version 1789368 (0.0010) [2023-12-27 18:02:31,646][20381] Updated weights for policy 1, policy_version 1787815 (0.0006) [2023-12-27 18:02:31,704][20381] Updated weights for policy 1, policy_version 1787825 (0.0007) [2023-12-27 18:02:31,765][20381] Updated weights for policy 1, policy_version 1787835 (0.0006) [2023-12-27 18:02:32,241][20347] Updated weights for policy 0, policy_version 1789378 (0.0010) [2023-12-27 18:02:32,304][20347] Updated weights for policy 0, policy_version 1789388 (0.0008) [2023-12-27 18:02:32,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 915898368. Throughput: 0: 10059.7, 1: 10241.4. Samples: 915914192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:32,363][19170] Avg episode reward: [(0, '970.830'), (1, '1000.000')] [2023-12-27 18:02:32,373][20347] Updated weights for policy 0, policy_version 1789398 (0.0008) [2023-12-27 18:02:32,385][20381] Updated weights for policy 1, policy_version 1787845 (0.0007) [2023-12-27 18:02:32,434][20347] Updated weights for policy 0, policy_version 1789408 (0.0008) [2023-12-27 18:02:32,448][20381] Updated weights for policy 1, policy_version 1787855 (0.0008) [2023-12-27 18:02:32,507][20381] Updated weights for policy 1, policy_version 1787865 (0.0011) [2023-12-27 18:02:33,078][20347] Updated weights for policy 0, policy_version 1789418 (0.0010) [2023-12-27 18:02:33,125][20347] Updated weights for policy 0, policy_version 1789428 (0.0010) [2023-12-27 18:02:33,175][20381] Updated weights for policy 1, policy_version 1787875 (0.0011) [2023-12-27 18:02:33,184][20347] Updated weights for policy 0, policy_version 1789438 (0.0009) [2023-12-27 18:02:33,229][20381] Updated weights for policy 1, policy_version 1787885 (0.0011) [2023-12-27 18:02:33,280][20381] Updated weights for policy 1, policy_version 1787895 (0.0010) [2023-12-27 18:02:33,848][20347] Updated weights for policy 0, policy_version 1789448 (0.0010) [2023-12-27 18:02:33,880][20381] Updated weights for policy 1, policy_version 1787905 (0.0009) [2023-12-27 18:02:33,899][20347] Updated weights for policy 0, policy_version 1789458 (0.0010) [2023-12-27 18:02:33,931][20381] Updated weights for policy 1, policy_version 1787915 (0.0005) [2023-12-27 18:02:33,952][20347] Updated weights for policy 0, policy_version 1789468 (0.0010) [2023-12-27 18:02:33,979][20381] Updated weights for policy 1, policy_version 1787925 (0.0005) [2023-12-27 18:02:34,031][20381] Updated weights for policy 1, policy_version 1787935 (0.0007) [2023-12-27 18:02:34,635][20347] Updated weights for policy 0, policy_version 1789478 (0.0010) [2023-12-27 18:02:34,672][20381] Updated weights for policy 1, policy_version 1787945 (0.0011) [2023-12-27 18:02:34,691][20347] Updated weights for policy 0, policy_version 1789488 (0.0011) [2023-12-27 18:02:34,732][20381] Updated weights for policy 1, policy_version 1787955 (0.0011) [2023-12-27 18:02:34,753][20347] Updated weights for policy 0, policy_version 1789498 (0.0008) [2023-12-27 18:02:34,788][20381] Updated weights for policy 1, policy_version 1787965 (0.0011) [2023-12-27 18:02:35,389][20347] Updated weights for policy 0, policy_version 1789508 (0.0007) [2023-12-27 18:02:35,451][20347] Updated weights for policy 0, policy_version 1789518 (0.0010) [2023-12-27 18:02:35,471][20381] Updated weights for policy 1, policy_version 1787975 (0.0011) [2023-12-27 18:02:35,509][20347] Updated weights for policy 0, policy_version 1789528 (0.0009) [2023-12-27 18:02:35,526][20381] Updated weights for policy 1, policy_version 1787985 (0.0010) [2023-12-27 18:02:35,590][20381] Updated weights for policy 1, policy_version 1787995 (0.0010) [2023-12-27 18:02:36,136][20347] Updated weights for policy 0, policy_version 1789538 (0.0006) [2023-12-27 18:02:36,192][20347] Updated weights for policy 0, policy_version 1789548 (0.0007) [2023-12-27 18:02:36,247][20347] Updated weights for policy 0, policy_version 1789558 (0.0006) [2023-12-27 18:02:36,275][20381] Updated weights for policy 1, policy_version 1788005 (0.0009) [2023-12-27 18:02:36,308][20347] Updated weights for policy 0, policy_version 1789568 (0.0005) [2023-12-27 18:02:36,333][20381] Updated weights for policy 1, policy_version 1788015 (0.0006) [2023-12-27 18:02:36,393][20381] Updated weights for policy 1, policy_version 1788025 (0.0008) [2023-12-27 18:02:36,992][20347] Updated weights for policy 0, policy_version 1789578 (0.0007) [2023-12-27 18:02:37,047][20347] Updated weights for policy 0, policy_version 1789588 (0.0007) [2023-12-27 18:02:37,099][20381] Updated weights for policy 1, policy_version 1788035 (0.0007) [2023-12-27 18:02:37,112][20347] Updated weights for policy 0, policy_version 1789598 (0.0007) [2023-12-27 18:02:37,152][20381] Updated weights for policy 1, policy_version 1788045 (0.0006) [2023-12-27 18:02:37,205][20381] Updated weights for policy 1, policy_version 1788055 (0.0008) [2023-12-27 18:02:37,363][19170] Fps is (10 sec: 21298.4, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 916013056. Throughput: 0: 10068.3, 1: 10299.6. Samples: 915977484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:37,364][19170] Avg episode reward: [(0, '951.260'), (1, '1000.000')] [2023-12-27 18:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001789600_458219520.pth... [2023-12-27 18:02:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001788064_457793536.pth... [2023-12-27 18:02:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001788416_457916416.pth [2023-12-27 18:02:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001786848_457482240.pth [2023-12-27 18:02:37,839][20347] Updated weights for policy 0, policy_version 1789608 (0.0008) [2023-12-27 18:02:37,875][20381] Updated weights for policy 1, policy_version 1788065 (0.0007) [2023-12-27 18:02:37,895][20347] Updated weights for policy 0, policy_version 1789618 (0.0008) [2023-12-27 18:02:37,932][20381] Updated weights for policy 1, policy_version 1788075 (0.0006) [2023-12-27 18:02:37,949][20347] Updated weights for policy 0, policy_version 1789628 (0.0006) [2023-12-27 18:02:37,990][20381] Updated weights for policy 1, policy_version 1788085 (0.0006) [2023-12-27 18:02:38,043][20381] Updated weights for policy 1, policy_version 1788095 (0.0008) [2023-12-27 18:02:38,657][20381] Updated weights for policy 1, policy_version 1788105 (0.0007) [2023-12-27 18:02:38,716][20381] Updated weights for policy 1, policy_version 1788115 (0.0007) [2023-12-27 18:02:38,726][20347] Updated weights for policy 0, policy_version 1789638 (0.0006) [2023-12-27 18:02:38,773][20381] Updated weights for policy 1, policy_version 1788125 (0.0006) [2023-12-27 18:02:38,783][20347] Updated weights for policy 0, policy_version 1789648 (0.0006) [2023-12-27 18:02:38,845][20347] Updated weights for policy 0, policy_version 1789658 (0.0007) [2023-12-27 18:02:39,389][20381] Updated weights for policy 1, policy_version 1788135 (0.0008) [2023-12-27 18:02:39,457][20381] Updated weights for policy 1, policy_version 1788145 (0.0008) [2023-12-27 18:02:39,523][20381] Updated weights for policy 1, policy_version 1788155 (0.0007) [2023-12-27 18:02:39,578][20347] Updated weights for policy 0, policy_version 1789668 (0.0008) [2023-12-27 18:02:39,637][20347] Updated weights for policy 0, policy_version 1789678 (0.0008) [2023-12-27 18:02:39,694][20347] Updated weights for policy 0, policy_version 1789688 (0.0007) [2023-12-27 18:02:40,233][20381] Updated weights for policy 1, policy_version 1788165 (0.0008) [2023-12-27 18:02:40,294][20381] Updated weights for policy 1, policy_version 1788175 (0.0008) [2023-12-27 18:02:40,352][20381] Updated weights for policy 1, policy_version 1788185 (0.0007) [2023-12-27 18:02:40,453][20347] Updated weights for policy 0, policy_version 1789698 (0.0008) [2023-12-27 18:02:40,520][20347] Updated weights for policy 0, policy_version 1789708 (0.0005) [2023-12-27 18:02:40,585][20347] Updated weights for policy 0, policy_version 1789718 (0.0008) [2023-12-27 18:02:40,639][20347] Updated weights for policy 0, policy_version 1789728 (0.0007) [2023-12-27 18:02:40,996][20381] Updated weights for policy 1, policy_version 1788195 (0.0007) [2023-12-27 18:02:41,057][20381] Updated weights for policy 1, policy_version 1788205 (0.0010) [2023-12-27 18:02:41,118][20381] Updated weights for policy 1, policy_version 1788215 (0.0011) [2023-12-27 18:02:41,325][20347] Updated weights for policy 0, policy_version 1789738 (0.0007) [2023-12-27 18:02:41,392][20347] Updated weights for policy 0, policy_version 1789748 (0.0008) [2023-12-27 18:02:41,451][20347] Updated weights for policy 0, policy_version 1789758 (0.0006) [2023-12-27 18:02:41,907][20381] Updated weights for policy 1, policy_version 1788225 (0.0009) [2023-12-27 18:02:41,970][20381] Updated weights for policy 1, policy_version 1788235 (0.0006) [2023-12-27 18:02:42,030][20381] Updated weights for policy 1, policy_version 1788245 (0.0007) [2023-12-27 18:02:42,087][20381] Updated weights for policy 1, policy_version 1788255 (0.0008) [2023-12-27 18:02:42,186][20347] Updated weights for policy 0, policy_version 1789768 (0.0008) [2023-12-27 18:02:42,236][20347] Updated weights for policy 0, policy_version 1789778 (0.0008) [2023-12-27 18:02:42,296][20347] Updated weights for policy 0, policy_version 1789788 (0.0008) [2023-12-27 18:02:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 916111360. Throughput: 0: 10012.4, 1: 10294.8. Samples: 916098852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:02:42,363][19170] Avg episode reward: [(0, '960.970'), (1, '1000.000')] [2023-12-27 18:02:42,741][20381] Updated weights for policy 1, policy_version 1788265 (0.0008) [2023-12-27 18:02:42,789][20381] Updated weights for policy 1, policy_version 1788275 (0.0007) [2023-12-27 18:02:42,839][20381] Updated weights for policy 1, policy_version 1788285 (0.0007) [2023-12-27 18:02:43,017][20347] Updated weights for policy 0, policy_version 1789798 (0.0008) [2023-12-27 18:02:43,071][20347] Updated weights for policy 0, policy_version 1789808 (0.0007) [2023-12-27 18:02:43,124][20347] Updated weights for policy 0, policy_version 1789818 (0.0008) [2023-12-27 18:02:43,533][20381] Updated weights for policy 1, policy_version 1788295 (0.0009) [2023-12-27 18:02:43,582][20381] Updated weights for policy 1, policy_version 1788305 (0.0010) [2023-12-27 18:02:43,634][20381] Updated weights for policy 1, policy_version 1788315 (0.0011) [2023-12-27 18:02:43,832][20347] Updated weights for policy 0, policy_version 1789828 (0.0008) [2023-12-27 18:02:43,875][20347] Updated weights for policy 0, policy_version 1789838 (0.0007) [2023-12-27 18:02:43,918][20347] Updated weights for policy 0, policy_version 1789848 (0.0007) [2023-12-27 18:02:44,327][20381] Updated weights for policy 1, policy_version 1788325 (0.0008) [2023-12-27 18:02:44,382][20381] Updated weights for policy 1, policy_version 1788335 (0.0006) [2023-12-27 18:02:44,431][20381] Updated weights for policy 1, policy_version 1788345 (0.0010) [2023-12-27 18:02:44,638][20347] Updated weights for policy 0, policy_version 1789858 (0.0007) [2023-12-27 18:02:44,698][20347] Updated weights for policy 0, policy_version 1789868 (0.0007) [2023-12-27 18:02:44,759][20347] Updated weights for policy 0, policy_version 1789878 (0.0007) [2023-12-27 18:02:44,824][20347] Updated weights for policy 0, policy_version 1789888 (0.0007) [2023-12-27 18:02:45,115][20381] Updated weights for policy 1, policy_version 1788355 (0.0009) [2023-12-27 18:02:45,174][20381] Updated weights for policy 1, policy_version 1788365 (0.0010) [2023-12-27 18:02:45,237][20381] Updated weights for policy 1, policy_version 1788375 (0.0011) [2023-12-27 18:02:45,501][20347] Updated weights for policy 0, policy_version 1789898 (0.0007) [2023-12-27 18:02:45,557][20347] Updated weights for policy 0, policy_version 1789908 (0.0006) [2023-12-27 18:02:45,608][20347] Updated weights for policy 0, policy_version 1789918 (0.0005) [2023-12-27 18:02:45,915][20381] Updated weights for policy 1, policy_version 1788385 (0.0010) [2023-12-27 18:02:45,970][20381] Updated weights for policy 1, policy_version 1788395 (0.0005) [2023-12-27 18:02:46,037][20381] Updated weights for policy 1, policy_version 1788405 (0.0006) [2023-12-27 18:02:46,087][20381] Updated weights for policy 1, policy_version 1788415 (0.0005) [2023-12-27 18:02:46,164][20347] Updated weights for policy 0, policy_version 1789928 (0.0007) [2023-12-27 18:02:46,215][20347] Updated weights for policy 0, policy_version 1789938 (0.0008) [2023-12-27 18:02:46,266][20347] Updated weights for policy 0, policy_version 1789948 (0.0010) [2023-12-27 18:02:46,764][20381] Updated weights for policy 1, policy_version 1788425 (0.0005) [2023-12-27 18:02:46,823][20381] Updated weights for policy 1, policy_version 1788435 (0.0005) [2023-12-27 18:02:46,839][20347] Updated weights for policy 0, policy_version 1789958 (0.0009) [2023-12-27 18:02:46,880][20381] Updated weights for policy 1, policy_version 1788445 (0.0005) [2023-12-27 18:02:46,895][20347] Updated weights for policy 0, policy_version 1789968 (0.0008) [2023-12-27 18:02:46,949][20347] Updated weights for policy 0, policy_version 1789978 (0.0006) [2023-12-27 18:02:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 916209664. Throughput: 0: 10085.2, 1: 10293.9. Samples: 916221740. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:02:47,364][19170] Avg episode reward: [(0, '990.260'), (1, '1000.000')] [2023-12-27 18:02:47,434][20381] Updated weights for policy 1, policy_version 1788455 (0.0005) [2023-12-27 18:02:47,488][20381] Updated weights for policy 1, policy_version 1788465 (0.0010) [2023-12-27 18:02:47,541][20381] Updated weights for policy 1, policy_version 1788475 (0.0010) [2023-12-27 18:02:47,668][20347] Updated weights for policy 0, policy_version 1789988 (0.0008) [2023-12-27 18:02:47,715][20347] Updated weights for policy 0, policy_version 1789998 (0.0007) [2023-12-27 18:02:47,767][20347] Updated weights for policy 0, policy_version 1790008 (0.0007) [2023-12-27 18:02:48,139][20381] Updated weights for policy 1, policy_version 1788485 (0.0008) [2023-12-27 18:02:48,188][20381] Updated weights for policy 1, policy_version 1788495 (0.0010) [2023-12-27 18:02:48,236][20381] Updated weights for policy 1, policy_version 1788505 (0.0010) [2023-12-27 18:02:48,494][20347] Updated weights for policy 0, policy_version 1790018 (0.0008) [2023-12-27 18:02:48,550][20347] Updated weights for policy 0, policy_version 1790028 (0.0006) [2023-12-27 18:02:48,602][20347] Updated weights for policy 0, policy_version 1790038 (0.0007) [2023-12-27 18:02:48,664][20347] Updated weights for policy 0, policy_version 1790048 (0.0008) [2023-12-27 18:02:48,972][20381] Updated weights for policy 1, policy_version 1788515 (0.0010) [2023-12-27 18:02:49,018][20381] Updated weights for policy 1, policy_version 1788525 (0.0010) [2023-12-27 18:02:49,066][20381] Updated weights for policy 1, policy_version 1788535 (0.0010) [2023-12-27 18:02:49,314][20347] Updated weights for policy 0, policy_version 1790058 (0.0008) [2023-12-27 18:02:49,381][20347] Updated weights for policy 0, policy_version 1790068 (0.0008) [2023-12-27 18:02:49,440][20347] Updated weights for policy 0, policy_version 1790078 (0.0008) [2023-12-27 18:02:49,763][20381] Updated weights for policy 1, policy_version 1788545 (0.0006) [2023-12-27 18:02:49,817][20381] Updated weights for policy 1, policy_version 1788555 (0.0007) [2023-12-27 18:02:49,884][20381] Updated weights for policy 1, policy_version 1788565 (0.0009) [2023-12-27 18:02:49,942][20381] Updated weights for policy 1, policy_version 1788575 (0.0009) [2023-12-27 18:02:50,146][20347] Updated weights for policy 0, policy_version 1790088 (0.0007) [2023-12-27 18:02:50,193][20347] Updated weights for policy 0, policy_version 1790098 (0.0007) [2023-12-27 18:02:50,249][20347] Updated weights for policy 0, policy_version 1790108 (0.0006) [2023-12-27 18:02:50,649][20381] Updated weights for policy 1, policy_version 1788585 (0.0007) [2023-12-27 18:02:50,707][20381] Updated weights for policy 1, policy_version 1788595 (0.0009) [2023-12-27 18:02:50,768][20381] Updated weights for policy 1, policy_version 1788605 (0.0008) [2023-12-27 18:02:50,931][20347] Updated weights for policy 0, policy_version 1790118 (0.0007) [2023-12-27 18:02:50,994][20347] Updated weights for policy 0, policy_version 1790128 (0.0007) [2023-12-27 18:02:51,055][20347] Updated weights for policy 0, policy_version 1790138 (0.0008) [2023-12-27 18:02:51,485][20381] Updated weights for policy 1, policy_version 1788615 (0.0006) [2023-12-27 18:02:51,542][20381] Updated weights for policy 1, policy_version 1788625 (0.0007) [2023-12-27 18:02:51,609][20381] Updated weights for policy 1, policy_version 1788635 (0.0007) [2023-12-27 18:02:51,799][20347] Updated weights for policy 0, policy_version 1790148 (0.0008) [2023-12-27 18:02:51,868][20347] Updated weights for policy 0, policy_version 1790158 (0.0008) [2023-12-27 18:02:51,930][20347] Updated weights for policy 0, policy_version 1790168 (0.0008) [2023-12-27 18:02:52,284][20381] Updated weights for policy 1, policy_version 1788645 (0.0009) [2023-12-27 18:02:52,344][20381] Updated weights for policy 1, policy_version 1788655 (0.0008) [2023-12-27 18:02:52,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 916307968. Throughput: 0: 10022.4, 1: 10266.4. Samples: 916282488. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:02:52,363][19170] Avg episode reward: [(0, '990.380'), (1, '1000.000')] [2023-12-27 18:02:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001790176_458366976.pth... [2023-12-27 18:02:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001789024_458072064.pth [2023-12-27 18:02:52,411][20381] Updated weights for policy 1, policy_version 1788665 (0.0008) [2023-12-27 18:02:52,447][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001788672_457949184.pth... [2023-12-27 18:02:52,450][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001787456_457637888.pth [2023-12-27 18:02:52,625][20347] Updated weights for policy 0, policy_version 1790178 (0.0006) [2023-12-27 18:02:52,677][20347] Updated weights for policy 0, policy_version 1790188 (0.0007) [2023-12-27 18:02:52,733][20347] Updated weights for policy 0, policy_version 1790198 (0.0007) [2023-12-27 18:02:52,790][20347] Updated weights for policy 0, policy_version 1790208 (0.0008) [2023-12-27 18:02:53,095][20381] Updated weights for policy 1, policy_version 1788675 (0.0008) [2023-12-27 18:02:53,153][20381] Updated weights for policy 1, policy_version 1788685 (0.0006) [2023-12-27 18:02:53,210][20381] Updated weights for policy 1, policy_version 1788695 (0.0006) [2023-12-27 18:02:53,505][20347] Updated weights for policy 0, policy_version 1790218 (0.0005) [2023-12-27 18:02:53,561][20347] Updated weights for policy 0, policy_version 1790228 (0.0007) [2023-12-27 18:02:53,605][20347] Updated weights for policy 0, policy_version 1790238 (0.0007) [2023-12-27 18:02:53,811][20381] Updated weights for policy 1, policy_version 1788705 (0.0008) [2023-12-27 18:02:53,869][20381] Updated weights for policy 1, policy_version 1788715 (0.0011) [2023-12-27 18:02:53,921][20381] Updated weights for policy 1, policy_version 1788725 (0.0011) [2023-12-27 18:02:53,982][20381] Updated weights for policy 1, policy_version 1788735 (0.0008) [2023-12-27 18:02:54,210][20347] Updated weights for policy 0, policy_version 1790248 (0.0007) [2023-12-27 18:02:54,272][20347] Updated weights for policy 0, policy_version 1790258 (0.0005) [2023-12-27 18:02:54,331][20347] Updated weights for policy 0, policy_version 1790268 (0.0005) [2023-12-27 18:02:54,592][20381] Updated weights for policy 1, policy_version 1788745 (0.0008) [2023-12-27 18:02:54,649][20381] Updated weights for policy 1, policy_version 1788755 (0.0008) [2023-12-27 18:02:54,705][20381] Updated weights for policy 1, policy_version 1788765 (0.0007) [2023-12-27 18:02:54,970][20347] Updated weights for policy 0, policy_version 1790278 (0.0006) [2023-12-27 18:02:55,038][20347] Updated weights for policy 0, policy_version 1790288 (0.0006) [2023-12-27 18:02:55,102][20347] Updated weights for policy 0, policy_version 1790298 (0.0005) [2023-12-27 18:02:55,348][20381] Updated weights for policy 1, policy_version 1788775 (0.0008) [2023-12-27 18:02:55,400][20381] Updated weights for policy 1, policy_version 1788785 (0.0008) [2023-12-27 18:02:55,457][20381] Updated weights for policy 1, policy_version 1788795 (0.0007) [2023-12-27 18:02:55,699][20347] Updated weights for policy 0, policy_version 1790308 (0.0007) [2023-12-27 18:02:55,757][20347] Updated weights for policy 0, policy_version 1790318 (0.0006) [2023-12-27 18:02:55,816][20347] Updated weights for policy 0, policy_version 1790328 (0.0008) [2023-12-27 18:02:56,137][20381] Updated weights for policy 1, policy_version 1788805 (0.0007) [2023-12-27 18:02:56,191][20381] Updated weights for policy 1, policy_version 1788815 (0.0005) [2023-12-27 18:02:56,241][20381] Updated weights for policy 1, policy_version 1788825 (0.0005) [2023-12-27 18:02:56,534][20347] Updated weights for policy 0, policy_version 1790338 (0.0007) [2023-12-27 18:02:56,583][20347] Updated weights for policy 0, policy_version 1790348 (0.0007) [2023-12-27 18:02:56,635][20347] Updated weights for policy 0, policy_version 1790358 (0.0009) [2023-12-27 18:02:56,680][20347] Updated weights for policy 0, policy_version 1790368 (0.0008) [2023-12-27 18:02:56,806][20381] Updated weights for policy 1, policy_version 1788835 (0.0005) [2023-12-27 18:02:56,858][20381] Updated weights for policy 1, policy_version 1788845 (0.0005) [2023-12-27 18:02:56,914][20381] Updated weights for policy 1, policy_version 1788855 (0.0009) [2023-12-27 18:02:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 916414464. Throughput: 0: 10028.4, 1: 10311.2. Samples: 916406504. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:02:57,363][19170] Avg episode reward: [(0, '990.380'), (1, '1000.000')] [2023-12-27 18:02:57,420][20347] Updated weights for policy 0, policy_version 1790378 (0.0007) [2023-12-27 18:02:57,472][20381] Updated weights for policy 1, policy_version 1788865 (0.0010) [2023-12-27 18:02:57,474][20347] Updated weights for policy 0, policy_version 1790388 (0.0009) [2023-12-27 18:02:57,526][20381] Updated weights for policy 1, policy_version 1788875 (0.0007) [2023-12-27 18:02:57,531][20347] Updated weights for policy 0, policy_version 1790398 (0.0007) [2023-12-27 18:02:57,581][20381] Updated weights for policy 1, policy_version 1788885 (0.0010) [2023-12-27 18:02:57,636][20381] Updated weights for policy 1, policy_version 1788895 (0.0011) [2023-12-27 18:02:58,243][20347] Updated weights for policy 0, policy_version 1790408 (0.0007) [2023-12-27 18:02:58,310][20347] Updated weights for policy 0, policy_version 1790418 (0.0009) [2023-12-27 18:02:58,327][20381] Updated weights for policy 1, policy_version 1788905 (0.0009) [2023-12-27 18:02:58,379][20347] Updated weights for policy 0, policy_version 1790429 (0.0008) [2023-12-27 18:02:58,391][20381] Updated weights for policy 1, policy_version 1788915 (0.0009) [2023-12-27 18:02:58,453][20381] Updated weights for policy 1, policy_version 1788925 (0.0008) [2023-12-27 18:02:59,240][20347] Updated weights for policy 0, policy_version 1790439 (0.0008) [2023-12-27 18:02:59,302][20381] Updated weights for policy 1, policy_version 1788935 (0.0007) [2023-12-27 18:02:59,304][20347] Updated weights for policy 0, policy_version 1790449 (0.0006) [2023-12-27 18:02:59,373][20347] Updated weights for policy 0, policy_version 1790459 (0.0008) [2023-12-27 18:02:59,374][20381] Updated weights for policy 1, policy_version 1788945 (0.0008) [2023-12-27 18:02:59,433][20381] Updated weights for policy 1, policy_version 1788955 (0.0007) [2023-12-27 18:03:00,058][20347] Updated weights for policy 0, policy_version 1790469 (0.0007) [2023-12-27 18:03:00,123][20347] Updated weights for policy 0, policy_version 1790479 (0.0008) [2023-12-27 18:03:00,157][20381] Updated weights for policy 1, policy_version 1788965 (0.0006) [2023-12-27 18:03:00,187][20347] Updated weights for policy 0, policy_version 1790489 (0.0009) [2023-12-27 18:03:00,214][20381] Updated weights for policy 1, policy_version 1788975 (0.0008) [2023-12-27 18:03:00,273][20381] Updated weights for policy 1, policy_version 1788985 (0.0008) [2023-12-27 18:03:00,800][20347] Updated weights for policy 0, policy_version 1790499 (0.0007) [2023-12-27 18:03:00,853][20347] Updated weights for policy 0, policy_version 1790509 (0.0008) [2023-12-27 18:03:00,902][20347] Updated weights for policy 0, policy_version 1790519 (0.0007) [2023-12-27 18:03:00,982][20381] Updated weights for policy 1, policy_version 1788995 (0.0008) [2023-12-27 18:03:01,030][20381] Updated weights for policy 1, policy_version 1789005 (0.0007) [2023-12-27 18:03:01,090][20381] Updated weights for policy 1, policy_version 1789015 (0.0008) [2023-12-27 18:03:01,675][20347] Updated weights for policy 0, policy_version 1790529 (0.0007) [2023-12-27 18:03:01,740][20347] Updated weights for policy 0, policy_version 1790539 (0.0010) [2023-12-27 18:03:01,796][20347] Updated weights for policy 0, policy_version 1790549 (0.0010) [2023-12-27 18:03:01,809][20381] Updated weights for policy 1, policy_version 1789025 (0.0007) [2023-12-27 18:03:01,855][20347] Updated weights for policy 0, policy_version 1790559 (0.0010) [2023-12-27 18:03:01,872][20381] Updated weights for policy 1, policy_version 1789035 (0.0006) [2023-12-27 18:03:01,931][20381] Updated weights for policy 1, policy_version 1789045 (0.0008) [2023-12-27 18:03:01,988][20381] Updated weights for policy 1, policy_version 1789055 (0.0007) [2023-12-27 18:03:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 916512768. Throughput: 0: 9949.0, 1: 10254.5. Samples: 916524208. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '1000.000')] [2023-12-27 18:03:02,526][20347] Updated weights for policy 0, policy_version 1790569 (0.0008) [2023-12-27 18:03:02,581][20347] Updated weights for policy 0, policy_version 1790579 (0.0010) [2023-12-27 18:03:02,640][20347] Updated weights for policy 0, policy_version 1790589 (0.0010) [2023-12-27 18:03:02,708][20381] Updated weights for policy 1, policy_version 1789065 (0.0007) [2023-12-27 18:03:02,753][20381] Updated weights for policy 1, policy_version 1789075 (0.0008) [2023-12-27 18:03:02,801][20381] Updated weights for policy 1, policy_version 1789085 (0.0007) [2023-12-27 18:03:03,294][20347] Updated weights for policy 0, policy_version 1790599 (0.0007) [2023-12-27 18:03:03,355][20347] Updated weights for policy 0, policy_version 1790609 (0.0005) [2023-12-27 18:03:03,417][20347] Updated weights for policy 0, policy_version 1790619 (0.0006) [2023-12-27 18:03:03,535][20381] Updated weights for policy 1, policy_version 1789095 (0.0006) [2023-12-27 18:03:03,585][20381] Updated weights for policy 1, policy_version 1789105 (0.0005) [2023-12-27 18:03:03,641][20381] Updated weights for policy 1, policy_version 1789115 (0.0005) [2023-12-27 18:03:04,095][20347] Updated weights for policy 0, policy_version 1790629 (0.0008) [2023-12-27 18:03:04,158][20347] Updated weights for policy 0, policy_version 1790639 (0.0007) [2023-12-27 18:03:04,219][20347] Updated weights for policy 0, policy_version 1790649 (0.0007) [2023-12-27 18:03:04,268][20381] Updated weights for policy 1, policy_version 1789125 (0.0005) [2023-12-27 18:03:04,330][20381] Updated weights for policy 1, policy_version 1789135 (0.0008) [2023-12-27 18:03:04,391][20381] Updated weights for policy 1, policy_version 1789145 (0.0007) [2023-12-27 18:03:04,947][20347] Updated weights for policy 0, policy_version 1790659 (0.0007) [2023-12-27 18:03:04,998][20347] Updated weights for policy 0, policy_version 1790669 (0.0008) [2023-12-27 18:03:05,002][20381] Updated weights for policy 1, policy_version 1789155 (0.0006) [2023-12-27 18:03:05,054][20381] Updated weights for policy 1, policy_version 1789165 (0.0010) [2023-12-27 18:03:05,056][20347] Updated weights for policy 0, policy_version 1790679 (0.0006) [2023-12-27 18:03:05,103][20381] Updated weights for policy 1, policy_version 1789175 (0.0010) [2023-12-27 18:03:05,691][20381] Updated weights for policy 1, policy_version 1789185 (0.0010) [2023-12-27 18:03:05,742][20381] Updated weights for policy 1, policy_version 1789195 (0.0005) [2023-12-27 18:03:05,770][20347] Updated weights for policy 0, policy_version 1790689 (0.0005) [2023-12-27 18:03:05,791][20381] Updated weights for policy 1, policy_version 1789205 (0.0006) [2023-12-27 18:03:05,824][20347] Updated weights for policy 0, policy_version 1790699 (0.0008) [2023-12-27 18:03:05,852][20381] Updated weights for policy 1, policy_version 1789215 (0.0005) [2023-12-27 18:03:05,878][20347] Updated weights for policy 0, policy_version 1790709 (0.0008) [2023-12-27 18:03:05,937][20347] Updated weights for policy 0, policy_version 1790719 (0.0008) [2023-12-27 18:03:06,498][20381] Updated weights for policy 1, policy_version 1789225 (0.0008) [2023-12-27 18:03:06,563][20381] Updated weights for policy 1, policy_version 1789235 (0.0009) [2023-12-27 18:03:06,620][20381] Updated weights for policy 1, policy_version 1789245 (0.0008) [2023-12-27 18:03:06,663][20347] Updated weights for policy 0, policy_version 1790729 (0.0007) [2023-12-27 18:03:06,716][20347] Updated weights for policy 0, policy_version 1790739 (0.0008) [2023-12-27 18:03:06,765][20347] Updated weights for policy 0, policy_version 1790749 (0.0008) [2023-12-27 18:03:07,285][20381] Updated weights for policy 1, policy_version 1789255 (0.0011) [2023-12-27 18:03:07,334][20381] Updated weights for policy 1, policy_version 1789265 (0.0011) [2023-12-27 18:03:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20206.9, 300 sec: 20521.7). Total num frames: 916611072. Throughput: 0: 9980.9, 1: 10261.2. Samples: 916585828. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:07,363][19170] Avg episode reward: [(0, '980.500'), (1, '1000.000')] [2023-12-27 18:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001790752_458514432.pth... [2023-12-27 18:03:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001789600_458219520.pth [2023-12-27 18:03:07,388][20381] Updated weights for policy 1, policy_version 1789275 (0.0009) [2023-12-27 18:03:07,417][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001789280_458104832.pth... [2023-12-27 18:03:07,421][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001788064_457793536.pth [2023-12-27 18:03:07,431][20347] Updated weights for policy 0, policy_version 1790759 (0.0007) [2023-12-27 18:03:07,483][20347] Updated weights for policy 0, policy_version 1790769 (0.0007) [2023-12-27 18:03:07,538][20347] Updated weights for policy 0, policy_version 1790779 (0.0008) [2023-12-27 18:03:08,040][20381] Updated weights for policy 1, policy_version 1789285 (0.0006) [2023-12-27 18:03:08,098][20381] Updated weights for policy 1, policy_version 1789295 (0.0005) [2023-12-27 18:03:08,144][20381] Updated weights for policy 1, policy_version 1789305 (0.0007) [2023-12-27 18:03:08,250][20347] Updated weights for policy 0, policy_version 1790789 (0.0007) [2023-12-27 18:03:08,304][20347] Updated weights for policy 0, policy_version 1790799 (0.0007) [2023-12-27 18:03:08,375][20347] Updated weights for policy 0, policy_version 1790809 (0.0007) [2023-12-27 18:03:08,845][20381] Updated weights for policy 1, policy_version 1789315 (0.0009) [2023-12-27 18:03:08,904][20381] Updated weights for policy 1, policy_version 1789325 (0.0008) [2023-12-27 18:03:08,950][20381] Updated weights for policy 1, policy_version 1789335 (0.0008) [2023-12-27 18:03:09,063][20347] Updated weights for policy 0, policy_version 1790819 (0.0007) [2023-12-27 18:03:09,119][20347] Updated weights for policy 0, policy_version 1790829 (0.0009) [2023-12-27 18:03:09,170][20347] Updated weights for policy 0, policy_version 1790839 (0.0006) [2023-12-27 18:03:09,644][20381] Updated weights for policy 1, policy_version 1789345 (0.0008) [2023-12-27 18:03:09,706][20381] Updated weights for policy 1, policy_version 1789355 (0.0007) [2023-12-27 18:03:09,770][20381] Updated weights for policy 1, policy_version 1789365 (0.0007) [2023-12-27 18:03:09,828][20381] Updated weights for policy 1, policy_version 1789375 (0.0007) [2023-12-27 18:03:09,852][20347] Updated weights for policy 0, policy_version 1790849 (0.0006) [2023-12-27 18:03:09,923][20347] Updated weights for policy 0, policy_version 1790859 (0.0008) [2023-12-27 18:03:09,976][20347] Updated weights for policy 0, policy_version 1790869 (0.0009) [2023-12-27 18:03:10,040][20347] Updated weights for policy 0, policy_version 1790879 (0.0008) [2023-12-27 18:03:10,534][20381] Updated weights for policy 1, policy_version 1789385 (0.0010) [2023-12-27 18:03:10,596][20381] Updated weights for policy 1, policy_version 1789395 (0.0010) [2023-12-27 18:03:10,659][20381] Updated weights for policy 1, policy_version 1789405 (0.0011) [2023-12-27 18:03:10,714][20347] Updated weights for policy 0, policy_version 1790889 (0.0009) [2023-12-27 18:03:10,773][20347] Updated weights for policy 0, policy_version 1790899 (0.0010) [2023-12-27 18:03:10,827][20347] Updated weights for policy 0, policy_version 1790909 (0.0010) [2023-12-27 18:03:11,392][20381] Updated weights for policy 1, policy_version 1789415 (0.0009) [2023-12-27 18:03:11,455][20381] Updated weights for policy 1, policy_version 1789425 (0.0008) [2023-12-27 18:03:11,517][20381] Updated weights for policy 1, policy_version 1789435 (0.0007) [2023-12-27 18:03:11,593][20347] Updated weights for policy 0, policy_version 1790919 (0.0008) [2023-12-27 18:03:11,662][20347] Updated weights for policy 0, policy_version 1790929 (0.0009) [2023-12-27 18:03:11,725][20347] Updated weights for policy 0, policy_version 1790939 (0.0008) [2023-12-27 18:03:12,253][20381] Updated weights for policy 1, policy_version 1789445 (0.0008) [2023-12-27 18:03:12,318][20381] Updated weights for policy 1, policy_version 1789455 (0.0007) [2023-12-27 18:03:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 916709376. Throughput: 0: 10008.4, 1: 10298.0. Samples: 916707924. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:12,363][19170] Avg episode reward: [(0, '980.540'), (1, '1000.000')] [2023-12-27 18:03:12,383][20381] Updated weights for policy 1, policy_version 1789465 (0.0007) [2023-12-27 18:03:12,466][20347] Updated weights for policy 0, policy_version 1790949 (0.0006) [2023-12-27 18:03:12,526][20347] Updated weights for policy 0, policy_version 1790959 (0.0007) [2023-12-27 18:03:12,580][20347] Updated weights for policy 0, policy_version 1790969 (0.0007) [2023-12-27 18:03:13,108][20381] Updated weights for policy 1, policy_version 1789475 (0.0007) [2023-12-27 18:03:13,166][20381] Updated weights for policy 1, policy_version 1789485 (0.0007) [2023-12-27 18:03:13,210][20347] Updated weights for policy 0, policy_version 1790979 (0.0007) [2023-12-27 18:03:13,221][20381] Updated weights for policy 1, policy_version 1789495 (0.0007) [2023-12-27 18:03:13,259][20347] Updated weights for policy 0, policy_version 1790989 (0.0006) [2023-12-27 18:03:13,310][20347] Updated weights for policy 0, policy_version 1790999 (0.0007) [2023-12-27 18:03:13,908][20381] Updated weights for policy 1, policy_version 1789505 (0.0006) [2023-12-27 18:03:13,956][20347] Updated weights for policy 0, policy_version 1791009 (0.0007) [2023-12-27 18:03:13,964][20381] Updated weights for policy 1, policy_version 1789515 (0.0008) [2023-12-27 18:03:14,014][20347] Updated weights for policy 0, policy_version 1791019 (0.0006) [2023-12-27 18:03:14,021][20381] Updated weights for policy 1, policy_version 1789525 (0.0006) [2023-12-27 18:03:14,065][20381] Updated weights for policy 1, policy_version 1789535 (0.0006) [2023-12-27 18:03:14,067][20347] Updated weights for policy 0, policy_version 1791029 (0.0006) [2023-12-27 18:03:14,122][20347] Updated weights for policy 0, policy_version 1791039 (0.0008) [2023-12-27 18:03:14,789][20381] Updated weights for policy 1, policy_version 1789545 (0.0008) [2023-12-27 18:03:14,811][20347] Updated weights for policy 0, policy_version 1791049 (0.0006) [2023-12-27 18:03:14,852][20381] Updated weights for policy 1, policy_version 1789555 (0.0007) [2023-12-27 18:03:14,869][20347] Updated weights for policy 0, policy_version 1791059 (0.0007) [2023-12-27 18:03:14,912][20381] Updated weights for policy 1, policy_version 1789565 (0.0007) [2023-12-27 18:03:14,930][20347] Updated weights for policy 0, policy_version 1791069 (0.0006) [2023-12-27 18:03:15,589][20381] Updated weights for policy 1, policy_version 1789575 (0.0008) [2023-12-27 18:03:15,591][20347] Updated weights for policy 0, policy_version 1791079 (0.0007) [2023-12-27 18:03:15,639][20381] Updated weights for policy 1, policy_version 1789585 (0.0007) [2023-12-27 18:03:15,648][20347] Updated weights for policy 0, policy_version 1791089 (0.0007) [2023-12-27 18:03:15,691][20381] Updated weights for policy 1, policy_version 1789595 (0.0005) [2023-12-27 18:03:15,697][20347] Updated weights for policy 0, policy_version 1791099 (0.0005) [2023-12-27 18:03:16,256][20381] Updated weights for policy 1, policy_version 1789605 (0.0006) [2023-12-27 18:03:16,306][20381] Updated weights for policy 1, policy_version 1789615 (0.0005) [2023-12-27 18:03:16,353][20381] Updated weights for policy 1, policy_version 1789625 (0.0009) [2023-12-27 18:03:16,409][20347] Updated weights for policy 0, policy_version 1791109 (0.0007) [2023-12-27 18:03:16,467][20347] Updated weights for policy 0, policy_version 1791119 (0.0008) [2023-12-27 18:03:16,512][20347] Updated weights for policy 0, policy_version 1791129 (0.0008) [2023-12-27 18:03:16,977][20381] Updated weights for policy 1, policy_version 1789635 (0.0009) [2023-12-27 18:03:17,034][20381] Updated weights for policy 1, policy_version 1789645 (0.0008) [2023-12-27 18:03:17,079][20381] Updated weights for policy 1, policy_version 1789655 (0.0007) [2023-12-27 18:03:17,136][20347] Updated weights for policy 0, policy_version 1791139 (0.0009) [2023-12-27 18:03:17,195][20347] Updated weights for policy 0, policy_version 1791149 (0.0008) [2023-12-27 18:03:17,249][20347] Updated weights for policy 0, policy_version 1791159 (0.0007) [2023-12-27 18:03:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 916824064. Throughput: 0: 10047.2, 1: 10304.6. Samples: 916830024. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:17,363][19170] Avg episode reward: [(0, '961.020'), (1, '1000.000')] [2023-12-27 18:03:17,761][20381] Updated weights for policy 1, policy_version 1789665 (0.0007) [2023-12-27 18:03:17,808][20381] Updated weights for policy 1, policy_version 1789675 (0.0007) [2023-12-27 18:03:17,859][20381] Updated weights for policy 1, policy_version 1789685 (0.0007) [2023-12-27 18:03:17,910][20381] Updated weights for policy 1, policy_version 1789695 (0.0007) [2023-12-27 18:03:17,946][20347] Updated weights for policy 0, policy_version 1791169 (0.0007) [2023-12-27 18:03:17,998][20347] Updated weights for policy 0, policy_version 1791179 (0.0010) [2023-12-27 18:03:18,050][20347] Updated weights for policy 0, policy_version 1791189 (0.0010) [2023-12-27 18:03:18,094][20347] Updated weights for policy 0, policy_version 1791199 (0.0010) [2023-12-27 18:03:18,628][20381] Updated weights for policy 1, policy_version 1789705 (0.0008) [2023-12-27 18:03:18,689][20381] Updated weights for policy 1, policy_version 1789715 (0.0008) [2023-12-27 18:03:18,748][20381] Updated weights for policy 1, policy_version 1789725 (0.0008) [2023-12-27 18:03:18,764][20347] Updated weights for policy 0, policy_version 1791209 (0.0011) [2023-12-27 18:03:18,820][20347] Updated weights for policy 0, policy_version 1791219 (0.0010) [2023-12-27 18:03:18,877][20347] Updated weights for policy 0, policy_version 1791229 (0.0010) [2023-12-27 18:03:19,412][20381] Updated weights for policy 1, policy_version 1789735 (0.0010) [2023-12-27 18:03:19,470][20381] Updated weights for policy 1, policy_version 1789745 (0.0009) [2023-12-27 18:03:19,537][20381] Updated weights for policy 1, policy_version 1789755 (0.0008) [2023-12-27 18:03:19,613][20347] Updated weights for policy 0, policy_version 1791239 (0.0011) [2023-12-27 18:03:19,673][20347] Updated weights for policy 0, policy_version 1791249 (0.0011) [2023-12-27 18:03:19,733][20347] Updated weights for policy 0, policy_version 1791259 (0.0010) [2023-12-27 18:03:20,267][20381] Updated weights for policy 1, policy_version 1789765 (0.0008) [2023-12-27 18:03:20,324][20381] Updated weights for policy 1, policy_version 1789775 (0.0008) [2023-12-27 18:03:20,376][20381] Updated weights for policy 1, policy_version 1789785 (0.0007) [2023-12-27 18:03:20,440][20347] Updated weights for policy 0, policy_version 1791269 (0.0008) [2023-12-27 18:03:20,497][20347] Updated weights for policy 0, policy_version 1791279 (0.0006) [2023-12-27 18:03:20,553][20347] Updated weights for policy 0, policy_version 1791289 (0.0007) [2023-12-27 18:03:21,027][20381] Updated weights for policy 1, policy_version 1789795 (0.0008) [2023-12-27 18:03:21,092][20381] Updated weights for policy 1, policy_version 1789805 (0.0008) [2023-12-27 18:03:21,154][20381] Updated weights for policy 1, policy_version 1789815 (0.0008) [2023-12-27 18:03:21,311][20347] Updated weights for policy 0, policy_version 1791299 (0.0009) [2023-12-27 18:03:21,363][20347] Updated weights for policy 0, policy_version 1791309 (0.0010) [2023-12-27 18:03:21,435][20347] Updated weights for policy 0, policy_version 1791319 (0.0008) [2023-12-27 18:03:21,866][20381] Updated weights for policy 1, policy_version 1789825 (0.0008) [2023-12-27 18:03:21,923][20381] Updated weights for policy 1, policy_version 1789835 (0.0006) [2023-12-27 18:03:21,976][20381] Updated weights for policy 1, policy_version 1789845 (0.0008) [2023-12-27 18:03:22,029][20381] Updated weights for policy 1, policy_version 1789855 (0.0007) [2023-12-27 18:03:22,182][20347] Updated weights for policy 0, policy_version 1791329 (0.0005) [2023-12-27 18:03:22,237][20347] Updated weights for policy 0, policy_version 1791339 (0.0007) [2023-12-27 18:03:22,299][20347] Updated weights for policy 0, policy_version 1791349 (0.0007) [2023-12-27 18:03:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 916914176. Throughput: 0: 10013.7, 1: 10269.9. Samples: 916890240. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:22,363][19170] Avg episode reward: [(0, '960.980'), (1, '1000.000')] [2023-12-27 18:03:22,365][20347] Updated weights for policy 0, policy_version 1791359 (0.0007) [2023-12-27 18:03:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001789856_458252288.pth... [2023-12-27 18:03:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001791360_458670080.pth... [2023-12-27 18:03:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001788672_457949184.pth [2023-12-27 18:03:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001790176_458366976.pth [2023-12-27 18:03:22,739][20381] Updated weights for policy 1, policy_version 1789865 (0.0008) [2023-12-27 18:03:22,785][20381] Updated weights for policy 1, policy_version 1789875 (0.0008) [2023-12-27 18:03:22,839][20381] Updated weights for policy 1, policy_version 1789885 (0.0008) [2023-12-27 18:03:23,098][20347] Updated weights for policy 0, policy_version 1791369 (0.0008) [2023-12-27 18:03:23,154][20347] Updated weights for policy 0, policy_version 1791379 (0.0008) [2023-12-27 18:03:23,199][20347] Updated weights for policy 0, policy_version 1791389 (0.0007) [2023-12-27 18:03:23,551][20381] Updated weights for policy 1, policy_version 1789895 (0.0007) [2023-12-27 18:03:23,609][20381] Updated weights for policy 1, policy_version 1789905 (0.0008) [2023-12-27 18:03:23,667][20381] Updated weights for policy 1, policy_version 1789915 (0.0008) [2023-12-27 18:03:23,919][20347] Updated weights for policy 0, policy_version 1791399 (0.0008) [2023-12-27 18:03:23,980][20347] Updated weights for policy 0, policy_version 1791409 (0.0005) [2023-12-27 18:03:24,043][20347] Updated weights for policy 0, policy_version 1791419 (0.0005) [2023-12-27 18:03:24,338][20381] Updated weights for policy 1, policy_version 1789925 (0.0008) [2023-12-27 18:03:24,383][20381] Updated weights for policy 1, policy_version 1789935 (0.0008) [2023-12-27 18:03:24,430][20381] Updated weights for policy 1, policy_version 1789945 (0.0008) [2023-12-27 18:03:24,646][20347] Updated weights for policy 0, policy_version 1791429 (0.0009) [2023-12-27 18:03:24,703][20347] Updated weights for policy 0, policy_version 1791439 (0.0010) [2023-12-27 18:03:24,757][20347] Updated weights for policy 0, policy_version 1791449 (0.0010) [2023-12-27 18:03:25,106][20381] Updated weights for policy 1, policy_version 1789955 (0.0006) [2023-12-27 18:03:25,159][20381] Updated weights for policy 1, policy_version 1789965 (0.0006) [2023-12-27 18:03:25,213][20381] Updated weights for policy 1, policy_version 1789975 (0.0007) [2023-12-27 18:03:25,425][20347] Updated weights for policy 0, policy_version 1791459 (0.0009) [2023-12-27 18:03:25,477][20347] Updated weights for policy 0, policy_version 1791469 (0.0010) [2023-12-27 18:03:25,528][20347] Updated weights for policy 0, policy_version 1791479 (0.0010) [2023-12-27 18:03:25,808][20381] Updated weights for policy 1, policy_version 1789985 (0.0007) [2023-12-27 18:03:25,884][20381] Updated weights for policy 1, policy_version 1789995 (0.0008) [2023-12-27 18:03:25,936][20381] Updated weights for policy 1, policy_version 1790005 (0.0007) [2023-12-27 18:03:25,982][20381] Updated weights for policy 1, policy_version 1790015 (0.0008) [2023-12-27 18:03:26,139][20347] Updated weights for policy 0, policy_version 1791489 (0.0008) [2023-12-27 18:03:26,187][20347] Updated weights for policy 0, policy_version 1791499 (0.0008) [2023-12-27 18:03:26,236][20347] Updated weights for policy 0, policy_version 1791509 (0.0007) [2023-12-27 18:03:26,287][20347] Updated weights for policy 0, policy_version 1791519 (0.0007) [2023-12-27 18:03:26,634][20381] Updated weights for policy 1, policy_version 1790025 (0.0010) [2023-12-27 18:03:26,689][20381] Updated weights for policy 1, policy_version 1790035 (0.0009) [2023-12-27 18:03:26,748][20381] Updated weights for policy 1, policy_version 1790045 (0.0006) [2023-12-27 18:03:26,995][20347] Updated weights for policy 0, policy_version 1791529 (0.0007) [2023-12-27 18:03:27,049][20347] Updated weights for policy 0, policy_version 1791539 (0.0008) [2023-12-27 18:03:27,103][20347] Updated weights for policy 0, policy_version 1791549 (0.0008) [2023-12-27 18:03:27,337][20381] Updated weights for policy 1, policy_version 1790055 (0.0007) [2023-12-27 18:03:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 917020672. Throughput: 0: 10050.5, 1: 10243.9. Samples: 917012100. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:27,363][19170] Avg episode reward: [(0, '970.730'), (1, '1000.000')] [2023-12-27 18:03:27,394][20381] Updated weights for policy 1, policy_version 1790065 (0.0006) [2023-12-27 18:03:27,446][20381] Updated weights for policy 1, policy_version 1790075 (0.0005) [2023-12-27 18:03:27,746][20347] Updated weights for policy 0, policy_version 1791559 (0.0008) [2023-12-27 18:03:27,804][20347] Updated weights for policy 0, policy_version 1791569 (0.0008) [2023-12-27 18:03:27,850][20347] Updated weights for policy 0, policy_version 1791579 (0.0005) [2023-12-27 18:03:28,003][20381] Updated weights for policy 1, policy_version 1790085 (0.0005) [2023-12-27 18:03:28,063][20381] Updated weights for policy 1, policy_version 1790095 (0.0005) [2023-12-27 18:03:28,116][20381] Updated weights for policy 1, policy_version 1790105 (0.0007) [2023-12-27 18:03:28,508][20347] Updated weights for policy 0, policy_version 1791589 (0.0007) [2023-12-27 18:03:28,567][20347] Updated weights for policy 0, policy_version 1791599 (0.0008) [2023-12-27 18:03:28,622][20347] Updated weights for policy 0, policy_version 1791609 (0.0007) [2023-12-27 18:03:28,862][20381] Updated weights for policy 1, policy_version 1790115 (0.0007) [2023-12-27 18:03:28,918][20381] Updated weights for policy 1, policy_version 1790125 (0.0005) [2023-12-27 18:03:28,974][20381] Updated weights for policy 1, policy_version 1790135 (0.0005) [2023-12-27 18:03:29,349][20347] Updated weights for policy 0, policy_version 1791619 (0.0007) [2023-12-27 18:03:29,411][20347] Updated weights for policy 0, policy_version 1791629 (0.0008) [2023-12-27 18:03:29,468][20347] Updated weights for policy 0, policy_version 1791639 (0.0007) [2023-12-27 18:03:29,577][20381] Updated weights for policy 1, policy_version 1790145 (0.0005) [2023-12-27 18:03:29,640][20381] Updated weights for policy 1, policy_version 1790155 (0.0007) [2023-12-27 18:03:29,701][20381] Updated weights for policy 1, policy_version 1790165 (0.0008) [2023-12-27 18:03:29,766][20381] Updated weights for policy 1, policy_version 1790175 (0.0007) [2023-12-27 18:03:30,117][20347] Updated weights for policy 0, policy_version 1791649 (0.0009) [2023-12-27 18:03:30,179][20347] Updated weights for policy 0, policy_version 1791659 (0.0011) [2023-12-27 18:03:30,237][20347] Updated weights for policy 0, policy_version 1791669 (0.0010) [2023-12-27 18:03:30,292][20347] Updated weights for policy 0, policy_version 1791679 (0.0010) [2023-12-27 18:03:30,514][20381] Updated weights for policy 1, policy_version 1790185 (0.0007) [2023-12-27 18:03:30,570][20381] Updated weights for policy 1, policy_version 1790195 (0.0008) [2023-12-27 18:03:30,594][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000000 [2023-12-27 18:03:30,984][20347] Updated weights for policy 0, policy_version 1791689 (0.0010) [2023-12-27 18:03:31,049][20347] Updated weights for policy 0, policy_version 1791699 (0.0011) [2023-12-27 18:03:31,121][20347] Updated weights for policy 0, policy_version 1791709 (0.0009) [2023-12-27 18:03:31,380][20381] Updated weights for policy 1, policy_version 1790205 (0.0007) [2023-12-27 18:03:31,441][20381] Updated weights for policy 1, policy_version 1790215 (0.0008) [2023-12-27 18:03:31,501][20381] Updated weights for policy 1, policy_version 1790225 (0.0008) [2023-12-27 18:03:31,860][20347] Updated weights for policy 0, policy_version 1791719 (0.0006) [2023-12-27 18:03:31,914][20347] Updated weights for policy 0, policy_version 1791729 (0.0005) [2023-12-27 18:03:31,966][20347] Updated weights for policy 0, policy_version 1791739 (0.0007) [2023-12-27 18:03:32,261][20381] Updated weights for policy 1, policy_version 1790235 (0.0008) [2023-12-27 18:03:32,317][20381] Updated weights for policy 1, policy_version 1790245 (0.0008) [2023-12-27 18:03:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 917118976. Throughput: 0: 10043.4, 1: 10248.5. Samples: 917134872. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:32,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 18:03:32,383][20381] Updated weights for policy 1, policy_version 1790255 (0.0008) [2023-12-27 18:03:32,557][20347] Updated weights for policy 0, policy_version 1791749 (0.0007) [2023-12-27 18:03:32,611][20347] Updated weights for policy 0, policy_version 1791759 (0.0007) [2023-12-27 18:03:32,662][20347] Updated weights for policy 0, policy_version 1791769 (0.0006) [2023-12-27 18:03:33,060][20381] Updated weights for policy 1, policy_version 1790265 (0.0008) [2023-12-27 18:03:33,122][20381] Updated weights for policy 1, policy_version 1790275 (0.0008) [2023-12-27 18:03:33,181][20381] Updated weights for policy 1, policy_version 1790285 (0.0007) [2023-12-27 18:03:33,237][20381] Updated weights for policy 1, policy_version 1790295 (0.0008) [2023-12-27 18:03:33,321][20347] Updated weights for policy 0, policy_version 1791779 (0.0006) [2023-12-27 18:03:33,374][20347] Updated weights for policy 0, policy_version 1791789 (0.0005) [2023-12-27 18:03:33,427][20347] Updated weights for policy 0, policy_version 1791799 (0.0005) [2023-12-27 18:03:33,941][20347] Updated weights for policy 0, policy_version 1791809 (0.0006) [2023-12-27 18:03:33,963][20381] Updated weights for policy 1, policy_version 1790305 (0.0010) [2023-12-27 18:03:33,995][20347] Updated weights for policy 0, policy_version 1791819 (0.0010) [2023-12-27 18:03:34,017][20381] Updated weights for policy 1, policy_version 1790315 (0.0010) [2023-12-27 18:03:34,053][20347] Updated weights for policy 0, policy_version 1791829 (0.0010) [2023-12-27 18:03:34,070][20381] Updated weights for policy 1, policy_version 1790325 (0.0010) [2023-12-27 18:03:34,101][20347] Updated weights for policy 0, policy_version 1791839 (0.0010) [2023-12-27 18:03:34,739][20381] Updated weights for policy 1, policy_version 1790335 (0.0009) [2023-12-27 18:03:34,800][20347] Updated weights for policy 0, policy_version 1791849 (0.0011) [2023-12-27 18:03:34,803][20381] Updated weights for policy 1, policy_version 1790345 (0.0009) [2023-12-27 18:03:34,858][20347] Updated weights for policy 0, policy_version 1791859 (0.0010) [2023-12-27 18:03:34,859][20381] Updated weights for policy 1, policy_version 1790355 (0.0010) [2023-12-27 18:03:34,913][20347] Updated weights for policy 0, policy_version 1791869 (0.0010) [2023-12-27 18:03:35,503][20381] Updated weights for policy 1, policy_version 1790365 (0.0010) [2023-12-27 18:03:35,534][20347] Updated weights for policy 0, policy_version 1791879 (0.0010) [2023-12-27 18:03:35,557][20381] Updated weights for policy 1, policy_version 1790375 (0.0010) [2023-12-27 18:03:35,590][20347] Updated weights for policy 0, policy_version 1791889 (0.0010) [2023-12-27 18:03:35,611][20381] Updated weights for policy 1, policy_version 1790385 (0.0011) [2023-12-27 18:03:35,646][20347] Updated weights for policy 0, policy_version 1791899 (0.0010) [2023-12-27 18:03:36,285][20347] Updated weights for policy 0, policy_version 1791909 (0.0010) [2023-12-27 18:03:36,314][20381] Updated weights for policy 1, policy_version 1790395 (0.0011) [2023-12-27 18:03:36,345][20347] Updated weights for policy 0, policy_version 1791919 (0.0011) [2023-12-27 18:03:36,374][20381] Updated weights for policy 1, policy_version 1790405 (0.0011) [2023-12-27 18:03:36,401][20347] Updated weights for policy 0, policy_version 1791929 (0.0010) [2023-12-27 18:03:36,434][20381] Updated weights for policy 1, policy_version 1790415 (0.0011) [2023-12-27 18:03:37,102][20381] Updated weights for policy 1, policy_version 1790425 (0.0011) [2023-12-27 18:03:37,102][20347] Updated weights for policy 0, policy_version 1791939 (0.0010) [2023-12-27 18:03:37,152][20381] Updated weights for policy 1, policy_version 1790435 (0.0011) [2023-12-27 18:03:37,157][20347] Updated weights for policy 0, policy_version 1791949 (0.0007) [2023-12-27 18:03:37,205][20381] Updated weights for policy 1, policy_version 1790445 (0.0006) [2023-12-27 18:03:37,211][20347] Updated weights for policy 0, policy_version 1791959 (0.0008) [2023-12-27 18:03:37,262][20381] Updated weights for policy 1, policy_version 1790455 (0.0008) [2023-12-27 18:03:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 917233664. Throughput: 0: 10119.5, 1: 10234.8. Samples: 917198428. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:37,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 18:03:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001791968_458825728.pth... [2023-12-27 18:03:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001790456_458407936.pth... [2023-12-27 18:03:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001790752_458514432.pth [2023-12-27 18:03:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001789280_458104832.pth [2023-12-27 18:03:37,886][20381] Updated weights for policy 1, policy_version 1790465 (0.0006) [2023-12-27 18:03:37,908][20347] Updated weights for policy 0, policy_version 1791969 (0.0008) [2023-12-27 18:03:37,937][20381] Updated weights for policy 1, policy_version 1790475 (0.0005) [2023-12-27 18:03:37,963][20347] Updated weights for policy 0, policy_version 1791979 (0.0011) [2023-12-27 18:03:37,989][20381] Updated weights for policy 1, policy_version 1790485 (0.0006) [2023-12-27 18:03:38,019][20347] Updated weights for policy 0, policy_version 1791989 (0.0010) [2023-12-27 18:03:38,074][20347] Updated weights for policy 0, policy_version 1791999 (0.0010) [2023-12-27 18:03:38,587][20381] Updated weights for policy 1, policy_version 1790495 (0.0005) [2023-12-27 18:03:38,646][20381] Updated weights for policy 1, policy_version 1790505 (0.0010) [2023-12-27 18:03:38,713][20381] Updated weights for policy 1, policy_version 1790515 (0.0005) [2023-12-27 18:03:38,750][20347] Updated weights for policy 0, policy_version 1792009 (0.0009) [2023-12-27 18:03:38,816][20347] Updated weights for policy 0, policy_version 1792019 (0.0011) [2023-12-27 18:03:38,876][20347] Updated weights for policy 0, policy_version 1792029 (0.0010) [2023-12-27 18:03:39,332][20381] Updated weights for policy 1, policy_version 1790525 (0.0008) [2023-12-27 18:03:39,403][20381] Updated weights for policy 1, policy_version 1790535 (0.0011) [2023-12-27 18:03:39,465][20381] Updated weights for policy 1, policy_version 1790545 (0.0008) [2023-12-27 18:03:39,611][20347] Updated weights for policy 0, policy_version 1792039 (0.0009) [2023-12-27 18:03:39,674][20347] Updated weights for policy 0, policy_version 1792049 (0.0008) [2023-12-27 18:03:39,736][20347] Updated weights for policy 0, policy_version 1792059 (0.0007) [2023-12-27 18:03:40,189][20381] Updated weights for policy 1, policy_version 1790555 (0.0008) [2023-12-27 18:03:40,251][20381] Updated weights for policy 1, policy_version 1790565 (0.0010) [2023-12-27 18:03:40,311][20381] Updated weights for policy 1, policy_version 1790575 (0.0010) [2023-12-27 18:03:40,447][20347] Updated weights for policy 0, policy_version 1792069 (0.0007) [2023-12-27 18:03:40,503][20347] Updated weights for policy 0, policy_version 1792079 (0.0008) [2023-12-27 18:03:40,557][20347] Updated weights for policy 0, policy_version 1792089 (0.0007) [2023-12-27 18:03:40,981][20381] Updated weights for policy 1, policy_version 1790585 (0.0006) [2023-12-27 18:03:41,051][20381] Updated weights for policy 1, policy_version 1790595 (0.0008) [2023-12-27 18:03:41,116][20381] Updated weights for policy 1, policy_version 1790605 (0.0008) [2023-12-27 18:03:41,189][20381] Updated weights for policy 1, policy_version 1790615 (0.0008) [2023-12-27 18:03:41,317][20347] Updated weights for policy 0, policy_version 1792099 (0.0007) [2023-12-27 18:03:41,388][20347] Updated weights for policy 0, policy_version 1792109 (0.0008) [2023-12-27 18:03:41,441][20347] Updated weights for policy 0, policy_version 1792119 (0.0008) [2023-12-27 18:03:41,939][20381] Updated weights for policy 1, policy_version 1790625 (0.0011) [2023-12-27 18:03:41,996][20381] Updated weights for policy 1, policy_version 1790635 (0.0010) [2023-12-27 18:03:42,061][20381] Updated weights for policy 1, policy_version 1790645 (0.0008) [2023-12-27 18:03:42,140][20347] Updated weights for policy 0, policy_version 1792129 (0.0007) [2023-12-27 18:03:42,206][20347] Updated weights for policy 0, policy_version 1792139 (0.0007) [2023-12-27 18:03:42,273][20347] Updated weights for policy 0, policy_version 1792149 (0.0006) [2023-12-27 18:03:42,341][20347] Updated weights for policy 0, policy_version 1792159 (0.0007) [2023-12-27 18:03:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 917331968. Throughput: 0: 10087.7, 1: 10219.2. Samples: 917320312. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:42,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 18:03:42,750][20381] Updated weights for policy 1, policy_version 1790655 (0.0006) [2023-12-27 18:03:42,812][20381] Updated weights for policy 1, policy_version 1790665 (0.0005) [2023-12-27 18:03:42,866][20381] Updated weights for policy 1, policy_version 1790675 (0.0007) [2023-12-27 18:03:43,024][20347] Updated weights for policy 0, policy_version 1792169 (0.0008) [2023-12-27 18:03:43,076][20347] Updated weights for policy 0, policy_version 1792179 (0.0007) [2023-12-27 18:03:43,121][20347] Updated weights for policy 0, policy_version 1792189 (0.0007) [2023-12-27 18:03:43,497][20381] Updated weights for policy 1, policy_version 1790685 (0.0007) [2023-12-27 18:03:43,558][20381] Updated weights for policy 1, policy_version 1790695 (0.0006) [2023-12-27 18:03:43,616][20381] Updated weights for policy 1, policy_version 1790705 (0.0005) [2023-12-27 18:03:43,772][20347] Updated weights for policy 0, policy_version 1792199 (0.0008) [2023-12-27 18:03:43,831][20347] Updated weights for policy 0, policy_version 1792209 (0.0007) [2023-12-27 18:03:43,892][20347] Updated weights for policy 0, policy_version 1792219 (0.0009) [2023-12-27 18:03:44,225][20381] Updated weights for policy 1, policy_version 1790715 (0.0008) [2023-12-27 18:03:44,281][20381] Updated weights for policy 1, policy_version 1790725 (0.0010) [2023-12-27 18:03:44,336][20381] Updated weights for policy 1, policy_version 1790735 (0.0011) [2023-12-27 18:03:44,651][20347] Updated weights for policy 0, policy_version 1792229 (0.0007) [2023-12-27 18:03:44,706][20347] Updated weights for policy 0, policy_version 1792239 (0.0007) [2023-12-27 18:03:44,766][20347] Updated weights for policy 0, policy_version 1792249 (0.0007) [2023-12-27 18:03:44,964][20381] Updated weights for policy 1, policy_version 1790745 (0.0007) [2023-12-27 18:03:45,025][20381] Updated weights for policy 1, policy_version 1790755 (0.0011) [2023-12-27 18:03:45,085][20381] Updated weights for policy 1, policy_version 1790765 (0.0011) [2023-12-27 18:03:45,144][20381] Updated weights for policy 1, policy_version 1790775 (0.0011) [2023-12-27 18:03:45,467][20347] Updated weights for policy 0, policy_version 1792259 (0.0009) [2023-12-27 18:03:45,528][20347] Updated weights for policy 0, policy_version 1792269 (0.0010) [2023-12-27 18:03:45,587][20347] Updated weights for policy 0, policy_version 1792279 (0.0007) [2023-12-27 18:03:45,778][20381] Updated weights for policy 1, policy_version 1790785 (0.0008) [2023-12-27 18:03:45,826][20381] Updated weights for policy 1, policy_version 1790795 (0.0008) [2023-12-27 18:03:45,883][20381] Updated weights for policy 1, policy_version 1790805 (0.0008) [2023-12-27 18:03:46,138][20347] Updated weights for policy 0, policy_version 1792289 (0.0006) [2023-12-27 18:03:46,201][20347] Updated weights for policy 0, policy_version 1792299 (0.0006) [2023-12-27 18:03:46,259][20347] Updated weights for policy 0, policy_version 1792309 (0.0007) [2023-12-27 18:03:46,313][20347] Updated weights for policy 0, policy_version 1792319 (0.0008) [2023-12-27 18:03:46,643][20381] Updated weights for policy 1, policy_version 1790815 (0.0008) [2023-12-27 18:03:46,701][20381] Updated weights for policy 1, policy_version 1790825 (0.0007) [2023-12-27 18:03:46,767][20381] Updated weights for policy 1, policy_version 1790835 (0.0008) [2023-12-27 18:03:46,956][20347] Updated weights for policy 0, policy_version 1792329 (0.0006) [2023-12-27 18:03:47,013][20347] Updated weights for policy 0, policy_version 1792339 (0.0005) [2023-12-27 18:03:47,064][20347] Updated weights for policy 0, policy_version 1792349 (0.0005) [2023-12-27 18:03:47,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 917430272. Throughput: 0: 10183.1, 1: 10234.2. Samples: 917442988. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:47,364][19170] Avg episode reward: [(0, '961.130'), (1, '1000.000')] [2023-12-27 18:03:47,449][20381] Updated weights for policy 1, policy_version 1790845 (0.0007) [2023-12-27 18:03:47,510][20381] Updated weights for policy 1, policy_version 1790855 (0.0005) [2023-12-27 18:03:47,562][20381] Updated weights for policy 1, policy_version 1790865 (0.0006) [2023-12-27 18:03:47,717][20347] Updated weights for policy 0, policy_version 1792359 (0.0007) [2023-12-27 18:03:47,775][20347] Updated weights for policy 0, policy_version 1792369 (0.0007) [2023-12-27 18:03:47,830][20347] Updated weights for policy 0, policy_version 1792379 (0.0007) [2023-12-27 18:03:48,218][20381] Updated weights for policy 1, policy_version 1790875 (0.0006) [2023-12-27 18:03:48,266][20381] Updated weights for policy 1, policy_version 1790885 (0.0008) [2023-12-27 18:03:48,317][20381] Updated weights for policy 1, policy_version 1790895 (0.0009) [2023-12-27 18:03:48,479][20347] Updated weights for policy 0, policy_version 1792389 (0.0007) [2023-12-27 18:03:48,536][20347] Updated weights for policy 0, policy_version 1792399 (0.0008) [2023-12-27 18:03:48,598][20347] Updated weights for policy 0, policy_version 1792409 (0.0007) [2023-12-27 18:03:49,028][20381] Updated weights for policy 1, policy_version 1790905 (0.0008) [2023-12-27 18:03:49,082][20381] Updated weights for policy 1, policy_version 1790915 (0.0008) [2023-12-27 18:03:49,135][20381] Updated weights for policy 1, policy_version 1790925 (0.0008) [2023-12-27 18:03:49,187][20381] Updated weights for policy 1, policy_version 1790935 (0.0008) [2023-12-27 18:03:49,313][20347] Updated weights for policy 0, policy_version 1792419 (0.0008) [2023-12-27 18:03:49,378][20347] Updated weights for policy 0, policy_version 1792429 (0.0009) [2023-12-27 18:03:49,430][20347] Updated weights for policy 0, policy_version 1792439 (0.0008) [2023-12-27 18:03:49,857][20381] Updated weights for policy 1, policy_version 1790945 (0.0007) [2023-12-27 18:03:49,911][20381] Updated weights for policy 1, policy_version 1790955 (0.0007) [2023-12-27 18:03:49,979][20381] Updated weights for policy 1, policy_version 1790965 (0.0008) [2023-12-27 18:03:50,170][20347] Updated weights for policy 0, policy_version 1792449 (0.0007) [2023-12-27 18:03:50,228][20347] Updated weights for policy 0, policy_version 1792459 (0.0007) [2023-12-27 18:03:50,290][20347] Updated weights for policy 0, policy_version 1792469 (0.0006) [2023-12-27 18:03:50,352][20347] Updated weights for policy 0, policy_version 1792479 (0.0007) [2023-12-27 18:03:50,700][20381] Updated weights for policy 1, policy_version 1790975 (0.0008) [2023-12-27 18:03:50,762][20381] Updated weights for policy 1, policy_version 1790985 (0.0007) [2023-12-27 18:03:50,820][20381] Updated weights for policy 1, policy_version 1790995 (0.0008) [2023-12-27 18:03:50,953][20347] Updated weights for policy 0, policy_version 1792489 (0.0006) [2023-12-27 18:03:51,007][20347] Updated weights for policy 0, policy_version 1792499 (0.0008) [2023-12-27 18:03:51,064][20347] Updated weights for policy 0, policy_version 1792509 (0.0008) [2023-12-27 18:03:51,563][20381] Updated weights for policy 1, policy_version 1791005 (0.0008) [2023-12-27 18:03:51,626][20381] Updated weights for policy 1, policy_version 1791015 (0.0008) [2023-12-27 18:03:51,688][20381] Updated weights for policy 1, policy_version 1791025 (0.0008) [2023-12-27 18:03:51,779][20347] Updated weights for policy 0, policy_version 1792519 (0.0008) [2023-12-27 18:03:51,835][20347] Updated weights for policy 0, policy_version 1792529 (0.0008) [2023-12-27 18:03:51,892][20347] Updated weights for policy 0, policy_version 1792539 (0.0009) [2023-12-27 18:03:52,292][20381] Updated weights for policy 1, policy_version 1791035 (0.0007) [2023-12-27 18:03:52,349][20381] Updated weights for policy 1, policy_version 1791045 (0.0006) [2023-12-27 18:03:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 917528576. Throughput: 0: 10180.9, 1: 10216.6. Samples: 917503716. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:52,363][19170] Avg episode reward: [(0, '951.220'), (1, '1000.000')] [2023-12-27 18:03:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001792544_458973184.pth... [2023-12-27 18:03:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001791360_458670080.pth [2023-12-27 18:03:52,411][20381] Updated weights for policy 1, policy_version 1791055 (0.0007) [2023-12-27 18:03:52,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001791064_458563584.pth... [2023-12-27 18:03:52,460][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001789856_458252288.pth [2023-12-27 18:03:52,598][20347] Updated weights for policy 0, policy_version 1792549 (0.0009) [2023-12-27 18:03:52,657][20347] Updated weights for policy 0, policy_version 1792559 (0.0009) [2023-12-27 18:03:52,723][20347] Updated weights for policy 0, policy_version 1792569 (0.0007) [2023-12-27 18:03:53,035][20381] Updated weights for policy 1, policy_version 1791065 (0.0008) [2023-12-27 18:03:53,083][20381] Updated weights for policy 1, policy_version 1791075 (0.0005) [2023-12-27 18:03:53,151][20381] Updated weights for policy 1, policy_version 1791086 (0.0008) [2023-12-27 18:03:53,206][20381] Updated weights for policy 1, policy_version 1791096 (0.0009) [2023-12-27 18:03:53,365][20347] Updated weights for policy 0, policy_version 1792579 (0.0009) [2023-12-27 18:03:53,415][20347] Updated weights for policy 0, policy_version 1792589 (0.0008) [2023-12-27 18:03:53,468][20347] Updated weights for policy 0, policy_version 1792599 (0.0007) [2023-12-27 18:03:53,858][20381] Updated weights for policy 1, policy_version 1791106 (0.0011) [2023-12-27 18:03:53,903][20381] Updated weights for policy 1, policy_version 1791116 (0.0010) [2023-12-27 18:03:53,952][20381] Updated weights for policy 1, policy_version 1791126 (0.0010) [2023-12-27 18:03:54,120][20347] Updated weights for policy 0, policy_version 1792609 (0.0008) [2023-12-27 18:03:54,177][20347] Updated weights for policy 0, policy_version 1792619 (0.0007) [2023-12-27 18:03:54,229][20347] Updated weights for policy 0, policy_version 1792629 (0.0007) [2023-12-27 18:03:54,281][20347] Updated weights for policy 0, policy_version 1792639 (0.0007) [2023-12-27 18:03:54,661][20381] Updated weights for policy 1, policy_version 1791136 (0.0008) [2023-12-27 18:03:54,726][20381] Updated weights for policy 1, policy_version 1791146 (0.0008) [2023-12-27 18:03:54,774][20381] Updated weights for policy 1, policy_version 1791156 (0.0008) [2023-12-27 18:03:54,968][20347] Updated weights for policy 0, policy_version 1792649 (0.0010) [2023-12-27 18:03:55,028][20347] Updated weights for policy 0, policy_version 1792659 (0.0011) [2023-12-27 18:03:55,089][20347] Updated weights for policy 0, policy_version 1792669 (0.0010) [2023-12-27 18:03:55,456][20381] Updated weights for policy 1, policy_version 1791166 (0.0007) [2023-12-27 18:03:55,501][20381] Updated weights for policy 1, policy_version 1791176 (0.0007) [2023-12-27 18:03:55,560][20381] Updated weights for policy 1, policy_version 1791186 (0.0008) [2023-12-27 18:03:55,723][20347] Updated weights for policy 0, policy_version 1792679 (0.0010) [2023-12-27 18:03:55,776][20347] Updated weights for policy 0, policy_version 1792689 (0.0010) [2023-12-27 18:03:55,832][20347] Updated weights for policy 0, policy_version 1792699 (0.0010) [2023-12-27 18:03:56,290][20381] Updated weights for policy 1, policy_version 1791196 (0.0007) [2023-12-27 18:03:56,339][20381] Updated weights for policy 1, policy_version 1791206 (0.0007) [2023-12-27 18:03:56,388][20381] Updated weights for policy 1, policy_version 1791216 (0.0007) [2023-12-27 18:03:56,475][20347] Updated weights for policy 0, policy_version 1792709 (0.0010) [2023-12-27 18:03:56,535][20347] Updated weights for policy 0, policy_version 1792719 (0.0010) [2023-12-27 18:03:56,586][20347] Updated weights for policy 0, policy_version 1792729 (0.0010) [2023-12-27 18:03:56,956][20381] Updated weights for policy 1, policy_version 1791226 (0.0008) [2023-12-27 18:03:57,007][20381] Updated weights for policy 1, policy_version 1791236 (0.0010) [2023-12-27 18:03:57,075][20381] Updated weights for policy 1, policy_version 1791246 (0.0005) [2023-12-27 18:03:57,118][20381] Updated weights for policy 1, policy_version 1791256 (0.0005) [2023-12-27 18:03:57,258][20347] Updated weights for policy 0, policy_version 1792739 (0.0010) [2023-12-27 18:03:57,305][20347] Updated weights for policy 0, policy_version 1792749 (0.0010) [2023-12-27 18:03:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 917635072. Throughput: 0: 10212.0, 1: 10191.2. Samples: 917626068. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:03:57,363][19170] Avg episode reward: [(0, '970.700'), (1, '1000.000')] [2023-12-27 18:03:57,369][20347] Updated weights for policy 0, policy_version 1792759 (0.0010) [2023-12-27 18:03:57,644][20381] Updated weights for policy 1, policy_version 1791266 (0.0005) [2023-12-27 18:03:57,692][20381] Updated weights for policy 1, policy_version 1791276 (0.0005) [2023-12-27 18:03:57,743][20381] Updated weights for policy 1, policy_version 1791286 (0.0005) [2023-12-27 18:03:58,043][20347] Updated weights for policy 0, policy_version 1792769 (0.0010) [2023-12-27 18:03:58,091][20347] Updated weights for policy 0, policy_version 1792779 (0.0007) [2023-12-27 18:03:58,145][20347] Updated weights for policy 0, policy_version 1792789 (0.0005) [2023-12-27 18:03:58,209][20347] Updated weights for policy 0, policy_version 1792799 (0.0007) [2023-12-27 18:03:58,282][20381] Updated weights for policy 1, policy_version 1791296 (0.0010) [2023-12-27 18:03:58,360][20381] Updated weights for policy 1, policy_version 1791306 (0.0010) [2023-12-27 18:03:58,428][20381] Updated weights for policy 1, policy_version 1791316 (0.0009) [2023-12-27 18:03:59,037][20347] Updated weights for policy 0, policy_version 1792809 (0.0008) [2023-12-27 18:03:59,100][20347] Updated weights for policy 0, policy_version 1792819 (0.0008) [2023-12-27 18:03:59,158][20347] Updated weights for policy 0, policy_version 1792829 (0.0008) [2023-12-27 18:03:59,195][20381] Updated weights for policy 1, policy_version 1791326 (0.0007) [2023-12-27 18:03:59,268][20381] Updated weights for policy 1, policy_version 1791336 (0.0008) [2023-12-27 18:03:59,336][20381] Updated weights for policy 1, policy_version 1791346 (0.0008) [2023-12-27 18:03:59,941][20347] Updated weights for policy 0, policy_version 1792839 (0.0008) [2023-12-27 18:04:00,004][20347] Updated weights for policy 0, policy_version 1792849 (0.0007) [2023-12-27 18:04:00,039][20381] Updated weights for policy 1, policy_version 1791356 (0.0007) [2023-12-27 18:04:00,056][20347] Updated weights for policy 0, policy_version 1792859 (0.0007) [2023-12-27 18:04:00,094][20381] Updated weights for policy 1, policy_version 1791366 (0.0005) [2023-12-27 18:04:00,144][20381] Updated weights for policy 1, policy_version 1791376 (0.0009) [2023-12-27 18:04:00,726][20381] Updated weights for policy 1, policy_version 1791386 (0.0011) [2023-12-27 18:04:00,778][20347] Updated weights for policy 0, policy_version 1792869 (0.0007) [2023-12-27 18:04:00,781][20381] Updated weights for policy 1, policy_version 1791396 (0.0007) [2023-12-27 18:04:00,824][20347] Updated weights for policy 0, policy_version 1792879 (0.0006) [2023-12-27 18:04:00,834][20381] Updated weights for policy 1, policy_version 1791406 (0.0007) [2023-12-27 18:04:00,870][20347] Updated weights for policy 0, policy_version 1792889 (0.0005) [2023-12-27 18:04:00,887][20381] Updated weights for policy 1, policy_version 1791416 (0.0007) [2023-12-27 18:04:01,526][20347] Updated weights for policy 0, policy_version 1792899 (0.0005) [2023-12-27 18:04:01,586][20347] Updated weights for policy 0, policy_version 1792909 (0.0005) [2023-12-27 18:04:01,588][20381] Updated weights for policy 1, policy_version 1791426 (0.0011) [2023-12-27 18:04:01,650][20347] Updated weights for policy 0, policy_version 1792919 (0.0009) [2023-12-27 18:04:01,651][20381] Updated weights for policy 1, policy_version 1791436 (0.0010) [2023-12-27 18:04:01,713][20381] Updated weights for policy 1, policy_version 1791446 (0.0008) [2023-12-27 18:04:02,358][20347] Updated weights for policy 0, policy_version 1792929 (0.0009) [2023-12-27 18:04:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 917733376. Throughput: 0: 10167.8, 1: 10262.3. Samples: 917749380. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:02,363][19170] Avg episode reward: [(0, '960.960'), (1, '1000.000')] [2023-12-27 18:04:02,410][20347] Updated weights for policy 0, policy_version 1792939 (0.0008) [2023-12-27 18:04:02,459][20347] Updated weights for policy 0, policy_version 1792949 (0.0007) [2023-12-27 18:04:02,467][20381] Updated weights for policy 1, policy_version 1791456 (0.0010) [2023-12-27 18:04:02,514][20347] Updated weights for policy 0, policy_version 1792959 (0.0006) [2023-12-27 18:04:02,527][20381] Updated weights for policy 1, policy_version 1791466 (0.0011) [2023-12-27 18:04:02,581][20381] Updated weights for policy 1, policy_version 1791476 (0.0010) [2023-12-27 18:04:03,234][20347] Updated weights for policy 0, policy_version 1792969 (0.0007) [2023-12-27 18:04:03,286][20381] Updated weights for policy 1, policy_version 1791486 (0.0011) [2023-12-27 18:04:03,289][20347] Updated weights for policy 0, policy_version 1792979 (0.0006) [2023-12-27 18:04:03,340][20347] Updated weights for policy 0, policy_version 1792989 (0.0005) [2023-12-27 18:04:03,342][20381] Updated weights for policy 1, policy_version 1791496 (0.0011) [2023-12-27 18:04:03,402][20381] Updated weights for policy 1, policy_version 1791506 (0.0011) [2023-12-27 18:04:04,029][20381] Updated weights for policy 1, policy_version 1791516 (0.0011) [2023-12-27 18:04:04,034][20347] Updated weights for policy 0, policy_version 1792999 (0.0006) [2023-12-27 18:04:04,082][20381] Updated weights for policy 1, policy_version 1791526 (0.0011) [2023-12-27 18:04:04,088][20347] Updated weights for policy 0, policy_version 1793009 (0.0005) [2023-12-27 18:04:04,139][20381] Updated weights for policy 1, policy_version 1791536 (0.0011) [2023-12-27 18:04:04,145][20347] Updated weights for policy 0, policy_version 1793019 (0.0006) [2023-12-27 18:04:04,797][20347] Updated weights for policy 0, policy_version 1793029 (0.0007) [2023-12-27 18:04:04,850][20347] Updated weights for policy 0, policy_version 1793039 (0.0007) [2023-12-27 18:04:04,887][20381] Updated weights for policy 1, policy_version 1791546 (0.0009) [2023-12-27 18:04:04,907][20347] Updated weights for policy 0, policy_version 1793049 (0.0006) [2023-12-27 18:04:04,939][20381] Updated weights for policy 1, policy_version 1791556 (0.0010) [2023-12-27 18:04:04,993][20381] Updated weights for policy 1, policy_version 1791566 (0.0010) [2023-12-27 18:04:05,046][20381] Updated weights for policy 1, policy_version 1791576 (0.0010) [2023-12-27 18:04:05,595][20347] Updated weights for policy 0, policy_version 1793059 (0.0005) [2023-12-27 18:04:05,643][20347] Updated weights for policy 0, policy_version 1793069 (0.0005) [2023-12-27 18:04:05,692][20347] Updated weights for policy 0, policy_version 1793079 (0.0006) [2023-12-27 18:04:05,738][20381] Updated weights for policy 1, policy_version 1791586 (0.0011) [2023-12-27 18:04:05,790][20381] Updated weights for policy 1, policy_version 1791596 (0.0011) [2023-12-27 18:04:05,842][20381] Updated weights for policy 1, policy_version 1791606 (0.0010) [2023-12-27 18:04:06,277][20347] Updated weights for policy 0, policy_version 1793089 (0.0005) [2023-12-27 18:04:06,338][20347] Updated weights for policy 0, policy_version 1793099 (0.0010) [2023-12-27 18:04:06,396][20347] Updated weights for policy 0, policy_version 1793109 (0.0010) [2023-12-27 18:04:06,458][20347] Updated weights for policy 0, policy_version 1793119 (0.0010) [2023-12-27 18:04:06,555][20381] Updated weights for policy 1, policy_version 1791616 (0.0011) [2023-12-27 18:04:06,618][20381] Updated weights for policy 1, policy_version 1791626 (0.0011) [2023-12-27 18:04:06,678][20381] Updated weights for policy 1, policy_version 1791636 (0.0011) [2023-12-27 18:04:07,119][20347] Updated weights for policy 0, policy_version 1793129 (0.0011) [2023-12-27 18:04:07,179][20347] Updated weights for policy 0, policy_version 1793139 (0.0010) [2023-12-27 18:04:07,235][20347] Updated weights for policy 0, policy_version 1793149 (0.0011) [2023-12-27 18:04:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 917839872. Throughput: 0: 10168.7, 1: 10255.7. Samples: 917809336. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:07,363][19170] Avg episode reward: [(0, '951.190'), (1, '990.310')] [2023-12-27 18:04:07,364][20381] Updated weights for policy 1, policy_version 1791646 (0.0011) [2023-12-27 18:04:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001793152_459128832.pth... [2023-12-27 18:04:07,370][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001791968_458825728.pth [2023-12-27 18:04:07,432][20381] Updated weights for policy 1, policy_version 1791656 (0.0010) [2023-12-27 18:04:07,480][20381] Updated weights for policy 1, policy_version 1791666 (0.0010) [2023-12-27 18:04:07,512][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001791672_458719232.pth... [2023-12-27 18:04:07,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001790456_458407936.pth [2023-12-27 18:04:07,921][20347] Updated weights for policy 0, policy_version 1793159 (0.0008) [2023-12-27 18:04:07,974][20347] Updated weights for policy 0, policy_version 1793169 (0.0007) [2023-12-27 18:04:08,021][20347] Updated weights for policy 0, policy_version 1793179 (0.0008) [2023-12-27 18:04:08,124][20381] Updated weights for policy 1, policy_version 1791676 (0.0009) [2023-12-27 18:04:08,182][20381] Updated weights for policy 1, policy_version 1791686 (0.0010) [2023-12-27 18:04:08,235][20381] Updated weights for policy 1, policy_version 1791696 (0.0010) [2023-12-27 18:04:08,766][20347] Updated weights for policy 0, policy_version 1793190 (0.0008) [2023-12-27 18:04:08,825][20347] Updated weights for policy 0, policy_version 1793200 (0.0008) [2023-12-27 18:04:08,876][20347] Updated weights for policy 0, policy_version 1793210 (0.0007) [2023-12-27 18:04:08,919][20381] Updated weights for policy 1, policy_version 1791706 (0.0010) [2023-12-27 18:04:08,980][20381] Updated weights for policy 1, policy_version 1791716 (0.0007) [2023-12-27 18:04:09,034][20381] Updated weights for policy 1, policy_version 1791726 (0.0007) [2023-12-27 18:04:09,088][20381] Updated weights for policy 1, policy_version 1791736 (0.0008) [2023-12-27 18:04:09,576][20347] Updated weights for policy 0, policy_version 1793220 (0.0007) [2023-12-27 18:04:09,635][20347] Updated weights for policy 0, policy_version 1793230 (0.0007) [2023-12-27 18:04:09,701][20347] Updated weights for policy 0, policy_version 1793240 (0.0010) [2023-12-27 18:04:09,765][20381] Updated weights for policy 1, policy_version 1791746 (0.0007) [2023-12-27 18:04:09,831][20381] Updated weights for policy 1, policy_version 1791756 (0.0006) [2023-12-27 18:04:09,894][20381] Updated weights for policy 1, policy_version 1791766 (0.0007) [2023-12-27 18:04:10,450][20347] Updated weights for policy 0, policy_version 1793250 (0.0011) [2023-12-27 18:04:10,516][20347] Updated weights for policy 0, policy_version 1793260 (0.0011) [2023-12-27 18:04:10,573][20347] Updated weights for policy 0, policy_version 1793270 (0.0007) [2023-12-27 18:04:10,604][20381] Updated weights for policy 1, policy_version 1791776 (0.0007) [2023-12-27 18:04:10,634][20347] Updated weights for policy 0, policy_version 1793280 (0.0008) [2023-12-27 18:04:10,666][20381] Updated weights for policy 1, policy_version 1791786 (0.0007) [2023-12-27 18:04:10,723][20381] Updated weights for policy 1, policy_version 1791796 (0.0008) [2023-12-27 18:04:11,294][20347] Updated weights for policy 0, policy_version 1793290 (0.0009) [2023-12-27 18:04:11,358][20347] Updated weights for policy 0, policy_version 1793300 (0.0010) [2023-12-27 18:04:11,422][20347] Updated weights for policy 0, policy_version 1793310 (0.0009) [2023-12-27 18:04:11,470][20381] Updated weights for policy 1, policy_version 1791806 (0.0008) [2023-12-27 18:04:11,526][20381] Updated weights for policy 1, policy_version 1791816 (0.0008) [2023-12-27 18:04:11,594][20381] Updated weights for policy 1, policy_version 1791826 (0.0008) [2023-12-27 18:04:12,135][20347] Updated weights for policy 0, policy_version 1793320 (0.0008) [2023-12-27 18:04:12,186][20347] Updated weights for policy 0, policy_version 1793330 (0.0006) [2023-12-27 18:04:12,243][20347] Updated weights for policy 0, policy_version 1793340 (0.0007) [2023-12-27 18:04:12,251][20381] Updated weights for policy 1, policy_version 1791836 (0.0008) [2023-12-27 18:04:12,313][20381] Updated weights for policy 1, policy_version 1791846 (0.0007) [2023-12-27 18:04:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 917938176. Throughput: 0: 10181.1, 1: 10230.4. Samples: 917930616. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:12,363][19170] Avg episode reward: [(0, '960.930'), (1, '990.310')] [2023-12-27 18:04:12,375][20381] Updated weights for policy 1, policy_version 1791856 (0.0008) [2023-12-27 18:04:12,953][20347] Updated weights for policy 0, policy_version 1793350 (0.0007) [2023-12-27 18:04:13,004][20347] Updated weights for policy 0, policy_version 1793360 (0.0010) [2023-12-27 18:04:13,057][20347] Updated weights for policy 0, policy_version 1793370 (0.0010) [2023-12-27 18:04:13,115][20381] Updated weights for policy 1, policy_version 1791866 (0.0008) [2023-12-27 18:04:13,165][20381] Updated weights for policy 1, policy_version 1791876 (0.0007) [2023-12-27 18:04:13,226][20381] Updated weights for policy 1, policy_version 1791886 (0.0008) [2023-12-27 18:04:13,281][20381] Updated weights for policy 1, policy_version 1791896 (0.0007) [2023-12-27 18:04:13,678][20347] Updated weights for policy 0, policy_version 1793380 (0.0010) [2023-12-27 18:04:13,740][20347] Updated weights for policy 0, policy_version 1793390 (0.0010) [2023-12-27 18:04:13,795][20347] Updated weights for policy 0, policy_version 1793400 (0.0010) [2023-12-27 18:04:13,953][20381] Updated weights for policy 1, policy_version 1791906 (0.0005) [2023-12-27 18:04:14,016][20381] Updated weights for policy 1, policy_version 1791916 (0.0008) [2023-12-27 18:04:14,072][20381] Updated weights for policy 1, policy_version 1791926 (0.0010) [2023-12-27 18:04:14,479][20347] Updated weights for policy 0, policy_version 1793410 (0.0010) [2023-12-27 18:04:14,538][20347] Updated weights for policy 0, policy_version 1793420 (0.0007) [2023-12-27 18:04:14,587][20347] Updated weights for policy 0, policy_version 1793430 (0.0006) [2023-12-27 18:04:14,637][20347] Updated weights for policy 0, policy_version 1793440 (0.0006) [2023-12-27 18:04:14,735][20381] Updated weights for policy 1, policy_version 1791936 (0.0009) [2023-12-27 18:04:14,810][20381] Updated weights for policy 1, policy_version 1791946 (0.0010) [2023-12-27 18:04:14,860][20381] Updated weights for policy 1, policy_version 1791956 (0.0008) [2023-12-27 18:04:15,306][20347] Updated weights for policy 0, policy_version 1793450 (0.0008) [2023-12-27 18:04:15,365][20347] Updated weights for policy 0, policy_version 1793460 (0.0007) [2023-12-27 18:04:15,416][20347] Updated weights for policy 0, policy_version 1793470 (0.0007) [2023-12-27 18:04:15,561][20381] Updated weights for policy 1, policy_version 1791966 (0.0009) [2023-12-27 18:04:15,625][20381] Updated weights for policy 1, policy_version 1791976 (0.0008) [2023-12-27 18:04:15,679][20381] Updated weights for policy 1, policy_version 1791986 (0.0005) [2023-12-27 18:04:16,132][20347] Updated weights for policy 0, policy_version 1793480 (0.0008) [2023-12-27 18:04:16,192][20347] Updated weights for policy 0, policy_version 1793490 (0.0007) [2023-12-27 18:04:16,238][20381] Updated weights for policy 1, policy_version 1791996 (0.0008) [2023-12-27 18:04:16,239][20347] Updated weights for policy 0, policy_version 1793500 (0.0008) [2023-12-27 18:04:16,287][20381] Updated weights for policy 1, policy_version 1792006 (0.0010) [2023-12-27 18:04:16,336][20381] Updated weights for policy 1, policy_version 1792016 (0.0010) [2023-12-27 18:04:16,902][20347] Updated weights for policy 0, policy_version 1793510 (0.0007) [2023-12-27 18:04:16,949][20347] Updated weights for policy 0, policy_version 1793520 (0.0007) [2023-12-27 18:04:16,997][20347] Updated weights for policy 0, policy_version 1793530 (0.0007) [2023-12-27 18:04:17,039][20381] Updated weights for policy 1, policy_version 1792026 (0.0011) [2023-12-27 18:04:17,090][20381] Updated weights for policy 1, policy_version 1792036 (0.0011) [2023-12-27 18:04:17,142][20381] Updated weights for policy 1, policy_version 1792046 (0.0011) [2023-12-27 18:04:17,197][20381] Updated weights for policy 1, policy_version 1792056 (0.0010) [2023-12-27 18:04:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 918044672. Throughput: 0: 10161.4, 1: 10213.0. Samples: 918051720. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:17,363][19170] Avg episode reward: [(0, '970.670'), (1, '1000.000')] [2023-12-27 18:04:17,669][20347] Updated weights for policy 0, policy_version 1793540 (0.0007) [2023-12-27 18:04:17,732][20347] Updated weights for policy 0, policy_version 1793550 (0.0009) [2023-12-27 18:04:17,782][20347] Updated weights for policy 0, policy_version 1793560 (0.0010) [2023-12-27 18:04:17,888][20381] Updated weights for policy 1, policy_version 1792066 (0.0011) [2023-12-27 18:04:17,945][20381] Updated weights for policy 1, policy_version 1792076 (0.0010) [2023-12-27 18:04:17,992][20381] Updated weights for policy 1, policy_version 1792086 (0.0009) [2023-12-27 18:04:18,398][20347] Updated weights for policy 0, policy_version 1793570 (0.0009) [2023-12-27 18:04:18,450][20347] Updated weights for policy 0, policy_version 1793580 (0.0008) [2023-12-27 18:04:18,501][20347] Updated weights for policy 0, policy_version 1793590 (0.0008) [2023-12-27 18:04:18,559][20347] Updated weights for policy 0, policy_version 1793600 (0.0008) [2023-12-27 18:04:18,583][20381] Updated weights for policy 1, policy_version 1792096 (0.0005) [2023-12-27 18:04:18,638][20381] Updated weights for policy 1, policy_version 1792106 (0.0006) [2023-12-27 18:04:18,700][20381] Updated weights for policy 1, policy_version 1792116 (0.0005) [2023-12-27 18:04:19,236][20381] Updated weights for policy 1, policy_version 1792126 (0.0008) [2023-12-27 18:04:19,281][20347] Updated weights for policy 0, policy_version 1793610 (0.0011) [2023-12-27 18:04:19,292][20381] Updated weights for policy 1, policy_version 1792136 (0.0008) [2023-12-27 18:04:19,338][20347] Updated weights for policy 0, policy_version 1793620 (0.0011) [2023-12-27 18:04:19,350][20381] Updated weights for policy 1, policy_version 1792146 (0.0008) [2023-12-27 18:04:19,403][20347] Updated weights for policy 0, policy_version 1793630 (0.0009) [2023-12-27 18:04:20,056][20381] Updated weights for policy 1, policy_version 1792156 (0.0007) [2023-12-27 18:04:20,109][20381] Updated weights for policy 1, policy_version 1792166 (0.0008) [2023-12-27 18:04:20,168][20381] Updated weights for policy 1, policy_version 1792176 (0.0007) [2023-12-27 18:04:20,169][20347] Updated weights for policy 0, policy_version 1793640 (0.0011) [2023-12-27 18:04:20,229][20347] Updated weights for policy 0, policy_version 1793650 (0.0011) [2023-12-27 18:04:20,289][20347] Updated weights for policy 0, policy_version 1793660 (0.0009) [2023-12-27 18:04:20,910][20381] Updated weights for policy 1, policy_version 1792186 (0.0009) [2023-12-27 18:04:20,953][20347] Updated weights for policy 0, policy_version 1793670 (0.0009) [2023-12-27 18:04:20,971][20381] Updated weights for policy 1, policy_version 1792196 (0.0011) [2023-12-27 18:04:21,016][20347] Updated weights for policy 0, policy_version 1793680 (0.0011) [2023-12-27 18:04:21,052][20381] Updated weights for policy 1, policy_version 1792206 (0.0011) [2023-12-27 18:04:21,079][20347] Updated weights for policy 0, policy_version 1793690 (0.0008) [2023-12-27 18:04:21,113][20381] Updated weights for policy 1, policy_version 1792216 (0.0007) [2023-12-27 18:04:21,837][20347] Updated weights for policy 0, policy_version 1793700 (0.0007) [2023-12-27 18:04:21,878][20381] Updated weights for policy 1, policy_version 1792226 (0.0008) [2023-12-27 18:04:21,893][20347] Updated weights for policy 0, policy_version 1793710 (0.0009) [2023-12-27 18:04:21,938][20381] Updated weights for policy 1, policy_version 1792236 (0.0006) [2023-12-27 18:04:21,954][20347] Updated weights for policy 0, policy_version 1793720 (0.0010) [2023-12-27 18:04:21,991][20381] Updated weights for policy 1, policy_version 1792246 (0.0006) [2023-12-27 18:04:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 918142976. Throughput: 0: 10112.2, 1: 10263.4. Samples: 918115328. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:22,363][19170] Avg episode reward: [(0, '980.460'), (1, '1000.000')] [2023-12-27 18:04:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001792248_458866688.pth... [2023-12-27 18:04:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001793728_459276288.pth... [2023-12-27 18:04:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001791064_458563584.pth [2023-12-27 18:04:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001792544_458973184.pth [2023-12-27 18:04:22,572][20347] Updated weights for policy 0, policy_version 1793730 (0.0011) [2023-12-27 18:04:22,626][20347] Updated weights for policy 0, policy_version 1793740 (0.0011) [2023-12-27 18:04:22,685][20347] Updated weights for policy 0, policy_version 1793750 (0.0011) [2023-12-27 18:04:22,733][20381] Updated weights for policy 1, policy_version 1792256 (0.0006) [2023-12-27 18:04:22,745][20347] Updated weights for policy 0, policy_version 1793760 (0.0011) [2023-12-27 18:04:22,794][20381] Updated weights for policy 1, policy_version 1792266 (0.0007) [2023-12-27 18:04:22,846][20381] Updated weights for policy 1, policy_version 1792276 (0.0007) [2023-12-27 18:04:23,435][20347] Updated weights for policy 0, policy_version 1793770 (0.0008) [2023-12-27 18:04:23,489][20347] Updated weights for policy 0, policy_version 1793780 (0.0008) [2023-12-27 18:04:23,499][20381] Updated weights for policy 1, policy_version 1792286 (0.0005) [2023-12-27 18:04:23,539][20347] Updated weights for policy 0, policy_version 1793790 (0.0008) [2023-12-27 18:04:23,548][20381] Updated weights for policy 1, policy_version 1792296 (0.0005) [2023-12-27 18:04:23,602][20381] Updated weights for policy 1, policy_version 1792306 (0.0005) [2023-12-27 18:04:24,244][20381] Updated weights for policy 1, policy_version 1792316 (0.0007) [2023-12-27 18:04:24,271][20347] Updated weights for policy 0, policy_version 1793800 (0.0006) [2023-12-27 18:04:24,304][20381] Updated weights for policy 1, policy_version 1792326 (0.0006) [2023-12-27 18:04:24,323][20347] Updated weights for policy 0, policy_version 1793810 (0.0008) [2023-12-27 18:04:24,355][20381] Updated weights for policy 1, policy_version 1792336 (0.0007) [2023-12-27 18:04:24,379][20347] Updated weights for policy 0, policy_version 1793820 (0.0010) [2023-12-27 18:04:24,995][20347] Updated weights for policy 0, policy_version 1793830 (0.0010) [2023-12-27 18:04:25,024][20381] Updated weights for policy 1, policy_version 1792346 (0.0009) [2023-12-27 18:04:25,061][20347] Updated weights for policy 0, policy_version 1793840 (0.0008) [2023-12-27 18:04:25,091][20381] Updated weights for policy 1, policy_version 1792356 (0.0010) [2023-12-27 18:04:25,120][20347] Updated weights for policy 0, policy_version 1793850 (0.0006) [2023-12-27 18:04:25,155][20381] Updated weights for policy 1, policy_version 1792366 (0.0010) [2023-12-27 18:04:25,215][20381] Updated weights for policy 1, policy_version 1792376 (0.0010) [2023-12-27 18:04:25,688][20347] Updated weights for policy 0, policy_version 1793860 (0.0006) [2023-12-27 18:04:25,745][20347] Updated weights for policy 0, policy_version 1793870 (0.0008) [2023-12-27 18:04:25,784][20381] Updated weights for policy 1, policy_version 1792386 (0.0006) [2023-12-27 18:04:25,797][20347] Updated weights for policy 0, policy_version 1793880 (0.0010) [2023-12-27 18:04:25,840][20381] Updated weights for policy 1, policy_version 1792396 (0.0010) [2023-12-27 18:04:25,896][20381] Updated weights for policy 1, policy_version 1792406 (0.0007) [2023-12-27 18:04:26,392][20347] Updated weights for policy 0, policy_version 1793890 (0.0010) [2023-12-27 18:04:26,453][20347] Updated weights for policy 0, policy_version 1793900 (0.0010) [2023-12-27 18:04:26,512][20347] Updated weights for policy 0, policy_version 1793910 (0.0007) [2023-12-27 18:04:26,527][20381] Updated weights for policy 1, policy_version 1792416 (0.0007) [2023-12-27 18:04:26,571][20381] Updated weights for policy 1, policy_version 1792426 (0.0010) [2023-12-27 18:04:26,571][20347] Updated weights for policy 0, policy_version 1793920 (0.0005) [2023-12-27 18:04:26,613][20381] Updated weights for policy 1, policy_version 1792436 (0.0010) [2023-12-27 18:04:27,141][20347] Updated weights for policy 0, policy_version 1793930 (0.0010) [2023-12-27 18:04:27,189][20347] Updated weights for policy 0, policy_version 1793940 (0.0010) [2023-12-27 18:04:27,237][20347] Updated weights for policy 0, policy_version 1793950 (0.0010) [2023-12-27 18:04:27,292][20381] Updated weights for policy 1, policy_version 1792446 (0.0009) [2023-12-27 18:04:27,342][20381] Updated weights for policy 1, policy_version 1792456 (0.0006) [2023-12-27 18:04:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 918249472. Throughput: 0: 10145.8, 1: 10258.4. Samples: 918238500. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:27,363][19170] Avg episode reward: [(0, '970.760'), (1, '1000.000')] [2023-12-27 18:04:27,385][20381] Updated weights for policy 1, policy_version 1792466 (0.0007) [2023-12-27 18:04:27,916][20347] Updated weights for policy 0, policy_version 1793960 (0.0010) [2023-12-27 18:04:27,967][20347] Updated weights for policy 0, policy_version 1793970 (0.0010) [2023-12-27 18:04:27,988][20381] Updated weights for policy 1, policy_version 1792476 (0.0006) [2023-12-27 18:04:28,029][20347] Updated weights for policy 0, policy_version 1793980 (0.0010) [2023-12-27 18:04:28,040][20381] Updated weights for policy 1, policy_version 1792486 (0.0006) [2023-12-27 18:04:28,092][20381] Updated weights for policy 1, policy_version 1792496 (0.0005) [2023-12-27 18:04:28,656][20347] Updated weights for policy 0, policy_version 1793990 (0.0007) [2023-12-27 18:04:28,706][20347] Updated weights for policy 0, policy_version 1794000 (0.0005) [2023-12-27 18:04:28,763][20347] Updated weights for policy 0, policy_version 1794010 (0.0008) [2023-12-27 18:04:28,763][20381] Updated weights for policy 1, policy_version 1792506 (0.0008) [2023-12-27 18:04:28,824][20381] Updated weights for policy 1, policy_version 1792516 (0.0008) [2023-12-27 18:04:28,890][20381] Updated weights for policy 1, policy_version 1792526 (0.0007) [2023-12-27 18:04:28,958][20381] Updated weights for policy 1, policy_version 1792536 (0.0006) [2023-12-27 18:04:29,392][20347] Updated weights for policy 0, policy_version 1794020 (0.0007) [2023-12-27 18:04:29,441][20347] Updated weights for policy 0, policy_version 1794030 (0.0005) [2023-12-27 18:04:29,492][20347] Updated weights for policy 0, policy_version 1794040 (0.0008) [2023-12-27 18:04:29,613][20381] Updated weights for policy 1, policy_version 1792546 (0.0005) [2023-12-27 18:04:29,671][20381] Updated weights for policy 1, policy_version 1792556 (0.0007) [2023-12-27 18:04:29,728][20381] Updated weights for policy 1, policy_version 1792566 (0.0007) [2023-12-27 18:04:30,169][20347] Updated weights for policy 0, policy_version 1794050 (0.0009) [2023-12-27 18:04:30,225][20347] Updated weights for policy 0, policy_version 1794060 (0.0006) [2023-12-27 18:04:30,282][20347] Updated weights for policy 0, policy_version 1794070 (0.0006) [2023-12-27 18:04:30,330][20347] Updated weights for policy 0, policy_version 1794080 (0.0005) [2023-12-27 18:04:30,448][20381] Updated weights for policy 1, policy_version 1792576 (0.0007) [2023-12-27 18:04:30,503][20381] Updated weights for policy 1, policy_version 1792586 (0.0008) [2023-12-27 18:04:30,558][20381] Updated weights for policy 1, policy_version 1792596 (0.0007) [2023-12-27 18:04:30,934][20347] Updated weights for policy 0, policy_version 1794090 (0.0005) [2023-12-27 18:04:30,992][20347] Updated weights for policy 0, policy_version 1794100 (0.0009) [2023-12-27 18:04:31,052][20347] Updated weights for policy 0, policy_version 1794110 (0.0010) [2023-12-27 18:04:31,275][20381] Updated weights for policy 1, policy_version 1792606 (0.0007) [2023-12-27 18:04:31,334][20381] Updated weights for policy 1, policy_version 1792616 (0.0008) [2023-12-27 18:04:31,399][20381] Updated weights for policy 1, policy_version 1792626 (0.0008) [2023-12-27 18:04:31,672][20347] Updated weights for policy 0, policy_version 1794120 (0.0007) [2023-12-27 18:04:31,733][20347] Updated weights for policy 0, policy_version 1794130 (0.0007) [2023-12-27 18:04:31,791][20347] Updated weights for policy 0, policy_version 1794140 (0.0007) [2023-12-27 18:04:32,083][20381] Updated weights for policy 1, policy_version 1792636 (0.0008) [2023-12-27 18:04:32,142][20381] Updated weights for policy 1, policy_version 1792646 (0.0007) [2023-12-27 18:04:32,201][20381] Updated weights for policy 1, policy_version 1792656 (0.0008) [2023-12-27 18:04:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 918355968. Throughput: 0: 10249.6, 1: 10256.6. Samples: 918365764. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:32,363][19170] Avg episode reward: [(0, '951.100'), (1, '1000.000')] [2023-12-27 18:04:32,464][20347] Updated weights for policy 0, policy_version 1794150 (0.0007) [2023-12-27 18:04:32,521][20347] Updated weights for policy 0, policy_version 1794160 (0.0007) [2023-12-27 18:04:32,581][20347] Updated weights for policy 0, policy_version 1794170 (0.0007) [2023-12-27 18:04:33,021][20381] Updated weights for policy 1, policy_version 1792666 (0.0009) [2023-12-27 18:04:33,078][20381] Updated weights for policy 1, policy_version 1792676 (0.0008) [2023-12-27 18:04:33,131][20381] Updated weights for policy 1, policy_version 1792686 (0.0007) [2023-12-27 18:04:33,150][20347] Updated weights for policy 0, policy_version 1794180 (0.0009) [2023-12-27 18:04:33,185][20381] Updated weights for policy 1, policy_version 1792696 (0.0009) [2023-12-27 18:04:33,205][20347] Updated weights for policy 0, policy_version 1794190 (0.0010) [2023-12-27 18:04:33,255][20347] Updated weights for policy 0, policy_version 1794200 (0.0008) [2023-12-27 18:04:33,813][20347] Updated weights for policy 0, policy_version 1794210 (0.0005) [2023-12-27 18:04:33,864][20347] Updated weights for policy 0, policy_version 1794220 (0.0007) [2023-12-27 18:04:33,907][20381] Updated weights for policy 1, policy_version 1792706 (0.0006) [2023-12-27 18:04:33,913][20347] Updated weights for policy 0, policy_version 1794230 (0.0006) [2023-12-27 18:04:33,960][20381] Updated weights for policy 1, policy_version 1792716 (0.0006) [2023-12-27 18:04:33,976][20347] Updated weights for policy 0, policy_version 1794240 (0.0007) [2023-12-27 18:04:34,017][20381] Updated weights for policy 1, policy_version 1792726 (0.0007) [2023-12-27 18:04:34,694][20347] Updated weights for policy 0, policy_version 1794250 (0.0009) [2023-12-27 18:04:34,751][20347] Updated weights for policy 0, policy_version 1794260 (0.0007) [2023-12-27 18:04:34,754][20381] Updated weights for policy 1, policy_version 1792736 (0.0006) [2023-12-27 18:04:34,810][20347] Updated weights for policy 0, policy_version 1794270 (0.0007) [2023-12-27 18:04:34,814][20381] Updated weights for policy 1, policy_version 1792746 (0.0006) [2023-12-27 18:04:34,869][20381] Updated weights for policy 1, policy_version 1792756 (0.0008) [2023-12-27 18:04:35,447][20347] Updated weights for policy 0, policy_version 1794280 (0.0007) [2023-12-27 18:04:35,500][20347] Updated weights for policy 0, policy_version 1794290 (0.0007) [2023-12-27 18:04:35,516][20381] Updated weights for policy 1, policy_version 1792766 (0.0009) [2023-12-27 18:04:35,544][20347] Updated weights for policy 0, policy_version 1794300 (0.0005) [2023-12-27 18:04:35,568][20381] Updated weights for policy 1, policy_version 1792776 (0.0007) [2023-12-27 18:04:35,623][20381] Updated weights for policy 1, policy_version 1792786 (0.0007) [2023-12-27 18:04:36,156][20347] Updated weights for policy 0, policy_version 1794310 (0.0006) [2023-12-27 18:04:36,215][20347] Updated weights for policy 0, policy_version 1794320 (0.0007) [2023-12-27 18:04:36,274][20347] Updated weights for policy 0, policy_version 1794330 (0.0008) [2023-12-27 18:04:36,349][20381] Updated weights for policy 1, policy_version 1792796 (0.0008) [2023-12-27 18:04:36,398][20381] Updated weights for policy 1, policy_version 1792806 (0.0008) [2023-12-27 18:04:36,453][20381] Updated weights for policy 1, policy_version 1792816 (0.0008) [2023-12-27 18:04:36,993][20347] Updated weights for policy 0, policy_version 1794340 (0.0008) [2023-12-27 18:04:37,043][20347] Updated weights for policy 0, policy_version 1794350 (0.0009) [2023-12-27 18:04:37,099][20347] Updated weights for policy 0, policy_version 1794360 (0.0010) [2023-12-27 18:04:37,215][20381] Updated weights for policy 1, policy_version 1792826 (0.0008) [2023-12-27 18:04:37,267][20381] Updated weights for policy 1, policy_version 1792836 (0.0007) [2023-12-27 18:04:37,325][20381] Updated weights for policy 1, policy_version 1792846 (0.0007) [2023-12-27 18:04:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 918454272. Throughput: 0: 10308.3, 1: 10225.4. Samples: 918427736. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:37,364][19170] Avg episode reward: [(0, '941.350'), (1, '990.330')] [2023-12-27 18:04:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001794368_459440128.pth... [2023-12-27 18:04:37,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001792856_459022336.pth... [2023-12-27 18:04:37,375][20381] Updated weights for policy 1, policy_version 1792856 (0.0007) [2023-12-27 18:04:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001793152_459128832.pth [2023-12-27 18:04:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001791672_458719232.pth [2023-12-27 18:04:37,764][20347] Updated weights for policy 0, policy_version 1794370 (0.0006) [2023-12-27 18:04:37,825][20347] Updated weights for policy 0, policy_version 1794380 (0.0005) [2023-12-27 18:04:37,873][20347] Updated weights for policy 0, policy_version 1794390 (0.0005) [2023-12-27 18:04:37,919][20347] Updated weights for policy 0, policy_version 1794400 (0.0005) [2023-12-27 18:04:38,092][20381] Updated weights for policy 1, policy_version 1792866 (0.0008) [2023-12-27 18:04:38,160][20381] Updated weights for policy 1, policy_version 1792876 (0.0008) [2023-12-27 18:04:38,227][20381] Updated weights for policy 1, policy_version 1792886 (0.0008) [2023-12-27 18:04:38,500][20347] Updated weights for policy 0, policy_version 1794410 (0.0010) [2023-12-27 18:04:38,558][20347] Updated weights for policy 0, policy_version 1794420 (0.0010) [2023-12-27 18:04:38,613][20347] Updated weights for policy 0, policy_version 1794430 (0.0010) [2023-12-27 18:04:38,923][20381] Updated weights for policy 1, policy_version 1792896 (0.0010) [2023-12-27 18:04:38,979][20381] Updated weights for policy 1, policy_version 1792906 (0.0010) [2023-12-27 18:04:39,041][20381] Updated weights for policy 1, policy_version 1792916 (0.0010) [2023-12-27 18:04:39,311][20347] Updated weights for policy 0, policy_version 1794440 (0.0009) [2023-12-27 18:04:39,376][20347] Updated weights for policy 0, policy_version 1794450 (0.0008) [2023-12-27 18:04:39,441][20347] Updated weights for policy 0, policy_version 1794460 (0.0007) [2023-12-27 18:04:39,793][20381] Updated weights for policy 1, policy_version 1792926 (0.0009) [2023-12-27 18:04:39,855][20381] Updated weights for policy 1, policy_version 1792936 (0.0009) [2023-12-27 18:04:39,918][20381] Updated weights for policy 1, policy_version 1792946 (0.0006) [2023-12-27 18:04:40,096][20347] Updated weights for policy 0, policy_version 1794470 (0.0008) [2023-12-27 18:04:40,150][20347] Updated weights for policy 0, policy_version 1794480 (0.0007) [2023-12-27 18:04:40,204][20347] Updated weights for policy 0, policy_version 1794490 (0.0007) [2023-12-27 18:04:40,602][20381] Updated weights for policy 1, policy_version 1792956 (0.0007) [2023-12-27 18:04:40,657][20381] Updated weights for policy 1, policy_version 1792966 (0.0005) [2023-12-27 18:04:40,724][20381] Updated weights for policy 1, policy_version 1792976 (0.0005) [2023-12-27 18:04:40,940][20347] Updated weights for policy 0, policy_version 1794500 (0.0007) [2023-12-27 18:04:40,996][20347] Updated weights for policy 0, policy_version 1794510 (0.0008) [2023-12-27 18:04:41,055][20347] Updated weights for policy 0, policy_version 1794520 (0.0009) [2023-12-27 18:04:41,299][20381] Updated weights for policy 1, policy_version 1792986 (0.0006) [2023-12-27 18:04:41,375][20381] Updated weights for policy 1, policy_version 1792996 (0.0008) [2023-12-27 18:04:41,438][20381] Updated weights for policy 1, policy_version 1793006 (0.0006) [2023-12-27 18:04:41,491][20381] Updated weights for policy 1, policy_version 1793016 (0.0006) [2023-12-27 18:04:41,890][20347] Updated weights for policy 0, policy_version 1794530 (0.0007) [2023-12-27 18:04:41,945][20347] Updated weights for policy 0, policy_version 1794540 (0.0007) [2023-12-27 18:04:42,003][20347] Updated weights for policy 0, policy_version 1794550 (0.0008) [2023-12-27 18:04:42,056][20347] Updated weights for policy 0, policy_version 1794560 (0.0008) [2023-12-27 18:04:42,148][20381] Updated weights for policy 1, policy_version 1793026 (0.0006) [2023-12-27 18:04:42,208][20381] Updated weights for policy 1, policy_version 1793036 (0.0006) [2023-12-27 18:04:42,269][20381] Updated weights for policy 1, policy_version 1793046 (0.0008) [2023-12-27 18:04:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 918560768. Throughput: 0: 10317.1, 1: 10187.9. Samples: 918548796. Policy #0 lag: (min: 31.0, avg: 48.1, max: 63.0) [2023-12-27 18:04:42,363][19170] Avg episode reward: [(0, '951.160'), (1, '990.330')] [2023-12-27 18:04:42,751][20347] Updated weights for policy 0, policy_version 1794570 (0.0007) [2023-12-27 18:04:42,810][20347] Updated weights for policy 0, policy_version 1794580 (0.0005) [2023-12-27 18:04:42,868][20347] Updated weights for policy 0, policy_version 1794590 (0.0005) [2023-12-27 18:04:42,975][20381] Updated weights for policy 1, policy_version 1793056 (0.0007) [2023-12-27 18:04:43,030][20381] Updated weights for policy 1, policy_version 1793066 (0.0008) [2023-12-27 18:04:43,085][20381] Updated weights for policy 1, policy_version 1793076 (0.0007) [2023-12-27 18:04:43,463][20347] Updated weights for policy 0, policy_version 1794600 (0.0006) [2023-12-27 18:04:43,513][20347] Updated weights for policy 0, policy_version 1794610 (0.0007) [2023-12-27 18:04:43,558][20347] Updated weights for policy 0, policy_version 1794620 (0.0008) [2023-12-27 18:04:43,752][20381] Updated weights for policy 1, policy_version 1793086 (0.0008) [2023-12-27 18:04:43,799][20381] Updated weights for policy 1, policy_version 1793096 (0.0009) [2023-12-27 18:04:43,847][20381] Updated weights for policy 1, policy_version 1793106 (0.0007) [2023-12-27 18:04:44,238][20347] Updated weights for policy 0, policy_version 1794630 (0.0009) [2023-12-27 18:04:44,304][20347] Updated weights for policy 0, policy_version 1794640 (0.0009) [2023-12-27 18:04:44,357][20347] Updated weights for policy 0, policy_version 1794650 (0.0010) [2023-12-27 18:04:44,561][20381] Updated weights for policy 1, policy_version 1793116 (0.0008) [2023-12-27 18:04:44,618][20381] Updated weights for policy 1, policy_version 1793126 (0.0008) [2023-12-27 18:04:44,666][20381] Updated weights for policy 1, policy_version 1793136 (0.0008) [2023-12-27 18:04:45,095][20347] Updated weights for policy 0, policy_version 1794660 (0.0009) [2023-12-27 18:04:45,161][20347] Updated weights for policy 0, policy_version 1794670 (0.0006) [2023-12-27 18:04:45,213][20347] Updated weights for policy 0, policy_version 1794680 (0.0006) [2023-12-27 18:04:45,413][20381] Updated weights for policy 1, policy_version 1793146 (0.0008) [2023-12-27 18:04:45,464][20381] Updated weights for policy 1, policy_version 1793156 (0.0008) [2023-12-27 18:04:45,520][20381] Updated weights for policy 1, policy_version 1793166 (0.0005) [2023-12-27 18:04:45,575][20381] Updated weights for policy 1, policy_version 1793176 (0.0010) [2023-12-27 18:04:45,816][20347] Updated weights for policy 0, policy_version 1794690 (0.0007) [2023-12-27 18:04:45,868][20347] Updated weights for policy 0, policy_version 1794700 (0.0007) [2023-12-27 18:04:45,920][20347] Updated weights for policy 0, policy_version 1794710 (0.0009) [2023-12-27 18:04:45,966][20347] Updated weights for policy 0, policy_version 1794720 (0.0009) [2023-12-27 18:04:46,195][20381] Updated weights for policy 1, policy_version 1793186 (0.0010) [2023-12-27 18:04:46,243][20381] Updated weights for policy 1, policy_version 1793196 (0.0010) [2023-12-27 18:04:46,296][20381] Updated weights for policy 1, policy_version 1793206 (0.0010) [2023-12-27 18:04:46,677][20347] Updated weights for policy 0, policy_version 1794730 (0.0009) [2023-12-27 18:04:46,730][20347] Updated weights for policy 0, policy_version 1794740 (0.0007) [2023-12-27 18:04:46,782][20347] Updated weights for policy 0, policy_version 1794750 (0.0007) [2023-12-27 18:04:46,887][20381] Updated weights for policy 1, policy_version 1793216 (0.0010) [2023-12-27 18:04:46,932][20381] Updated weights for policy 1, policy_version 1793226 (0.0010) [2023-12-27 18:04:46,989][20381] Updated weights for policy 1, policy_version 1793236 (0.0010) [2023-12-27 18:04:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 918659072. Throughput: 0: 10348.4, 1: 10121.7. Samples: 918670536. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:04:47,364][19170] Avg episode reward: [(0, '960.970'), (1, '1000.000')] [2023-12-27 18:04:47,506][20347] Updated weights for policy 0, policy_version 1794760 (0.0008) [2023-12-27 18:04:47,567][20347] Updated weights for policy 0, policy_version 1794770 (0.0008) [2023-12-27 18:04:47,617][20381] Updated weights for policy 1, policy_version 1793246 (0.0009) [2023-12-27 18:04:47,633][20347] Updated weights for policy 0, policy_version 1794780 (0.0006) [2023-12-27 18:04:47,672][20381] Updated weights for policy 1, policy_version 1793256 (0.0010) [2023-12-27 18:04:47,728][20381] Updated weights for policy 1, policy_version 1793266 (0.0010) [2023-12-27 18:04:48,333][20347] Updated weights for policy 0, policy_version 1794790 (0.0007) [2023-12-27 18:04:48,393][20347] Updated weights for policy 0, policy_version 1794800 (0.0008) [2023-12-27 18:04:48,416][20381] Updated weights for policy 1, policy_version 1793276 (0.0011) [2023-12-27 18:04:48,450][20347] Updated weights for policy 0, policy_version 1794810 (0.0009) [2023-12-27 18:04:48,472][20381] Updated weights for policy 1, policy_version 1793286 (0.0011) [2023-12-27 18:04:48,529][20381] Updated weights for policy 1, policy_version 1793296 (0.0010) [2023-12-27 18:04:49,153][20347] Updated weights for policy 0, policy_version 1794820 (0.0007) [2023-12-27 18:04:49,206][20347] Updated weights for policy 0, policy_version 1794830 (0.0007) [2023-12-27 18:04:49,225][20381] Updated weights for policy 1, policy_version 1793306 (0.0008) [2023-12-27 18:04:49,270][20347] Updated weights for policy 0, policy_version 1794840 (0.0007) [2023-12-27 18:04:49,288][20381] Updated weights for policy 1, policy_version 1793316 (0.0008) [2023-12-27 18:04:49,348][20381] Updated weights for policy 1, policy_version 1793326 (0.0007) [2023-12-27 18:04:49,418][20381] Updated weights for policy 1, policy_version 1793336 (0.0009) [2023-12-27 18:04:49,970][20347] Updated weights for policy 0, policy_version 1794850 (0.0007) [2023-12-27 18:04:50,021][20347] Updated weights for policy 0, policy_version 1794860 (0.0006) [2023-12-27 18:04:50,071][20347] Updated weights for policy 0, policy_version 1794870 (0.0007) [2023-12-27 18:04:50,123][20381] Updated weights for policy 1, policy_version 1793346 (0.0008) [2023-12-27 18:04:50,129][20347] Updated weights for policy 0, policy_version 1794880 (0.0006) [2023-12-27 18:04:50,181][20381] Updated weights for policy 1, policy_version 1793356 (0.0009) [2023-12-27 18:04:50,250][20381] Updated weights for policy 1, policy_version 1793366 (0.0008) [2023-12-27 18:04:50,820][20347] Updated weights for policy 0, policy_version 1794890 (0.0006) [2023-12-27 18:04:50,887][20347] Updated weights for policy 0, policy_version 1794900 (0.0007) [2023-12-27 18:04:50,936][20381] Updated weights for policy 1, policy_version 1793376 (0.0007) [2023-12-27 18:04:50,946][20347] Updated weights for policy 0, policy_version 1794910 (0.0007) [2023-12-27 18:04:50,999][20381] Updated weights for policy 1, policy_version 1793386 (0.0007) [2023-12-27 18:04:51,070][20381] Updated weights for policy 1, policy_version 1793396 (0.0007) [2023-12-27 18:04:51,620][20347] Updated weights for policy 0, policy_version 1794920 (0.0007) [2023-12-27 18:04:51,677][20347] Updated weights for policy 0, policy_version 1794930 (0.0007) [2023-12-27 18:04:51,745][20347] Updated weights for policy 0, policy_version 1794940 (0.0007) [2023-12-27 18:04:51,814][20381] Updated weights for policy 1, policy_version 1793406 (0.0008) [2023-12-27 18:04:51,873][20381] Updated weights for policy 1, policy_version 1793416 (0.0006) [2023-12-27 18:04:51,933][20381] Updated weights for policy 1, policy_version 1793426 (0.0006) [2023-12-27 18:04:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 918757376. Throughput: 0: 10328.6, 1: 10132.2. Samples: 918730076. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:04:52,363][19170] Avg episode reward: [(0, '941.430'), (1, '990.270')] [2023-12-27 18:04:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001793432_459169792.pth... [2023-12-27 18:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001794944_459587584.pth... [2023-12-27 18:04:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001792248_458866688.pth [2023-12-27 18:04:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001793728_459276288.pth [2023-12-27 18:04:52,435][20347] Updated weights for policy 0, policy_version 1794950 (0.0008) [2023-12-27 18:04:52,494][20347] Updated weights for policy 0, policy_version 1794960 (0.0008) [2023-12-27 18:04:52,542][20347] Updated weights for policy 0, policy_version 1794970 (0.0008) [2023-12-27 18:04:52,574][20381] Updated weights for policy 1, policy_version 1793436 (0.0008) [2023-12-27 18:04:52,635][20381] Updated weights for policy 1, policy_version 1793446 (0.0005) [2023-12-27 18:04:52,694][20381] Updated weights for policy 1, policy_version 1793456 (0.0006) [2023-12-27 18:04:53,237][20347] Updated weights for policy 0, policy_version 1794980 (0.0008) [2023-12-27 18:04:53,303][20347] Updated weights for policy 0, policy_version 1794990 (0.0008) [2023-12-27 18:04:53,354][20381] Updated weights for policy 1, policy_version 1793466 (0.0008) [2023-12-27 18:04:53,357][20347] Updated weights for policy 0, policy_version 1795000 (0.0005) [2023-12-27 18:04:53,410][20381] Updated weights for policy 1, policy_version 1793476 (0.0007) [2023-12-27 18:04:53,461][20381] Updated weights for policy 1, policy_version 1793486 (0.0008) [2023-12-27 18:04:53,505][20381] Updated weights for policy 1, policy_version 1793496 (0.0007) [2023-12-27 18:04:53,921][20347] Updated weights for policy 0, policy_version 1795010 (0.0005) [2023-12-27 18:04:53,979][20347] Updated weights for policy 0, policy_version 1795020 (0.0006) [2023-12-27 18:04:54,040][20347] Updated weights for policy 0, policy_version 1795030 (0.0005) [2023-12-27 18:04:54,095][20347] Updated weights for policy 0, policy_version 1795040 (0.0006) [2023-12-27 18:04:54,191][20381] Updated weights for policy 1, policy_version 1793506 (0.0011) [2023-12-27 18:04:54,252][20381] Updated weights for policy 1, policy_version 1793516 (0.0011) [2023-12-27 18:04:54,314][20381] Updated weights for policy 1, policy_version 1793526 (0.0011) [2023-12-27 18:04:54,704][20347] Updated weights for policy 0, policy_version 1795050 (0.0011) [2023-12-27 18:04:54,757][20347] Updated weights for policy 0, policy_version 1795060 (0.0010) [2023-12-27 18:04:54,822][20347] Updated weights for policy 0, policy_version 1795070 (0.0010) [2023-12-27 18:04:54,896][20381] Updated weights for policy 1, policy_version 1793536 (0.0007) [2023-12-27 18:04:54,960][20381] Updated weights for policy 1, policy_version 1793546 (0.0008) [2023-12-27 18:04:55,018][20381] Updated weights for policy 1, policy_version 1793556 (0.0007) [2023-12-27 18:04:55,464][20347] Updated weights for policy 0, policy_version 1795080 (0.0009) [2023-12-27 18:04:55,528][20347] Updated weights for policy 0, policy_version 1795090 (0.0010) [2023-12-27 18:04:55,575][20347] Updated weights for policy 0, policy_version 1795100 (0.0008) [2023-12-27 18:04:55,670][20381] Updated weights for policy 1, policy_version 1793566 (0.0009) [2023-12-27 18:04:55,713][20381] Updated weights for policy 1, policy_version 1793576 (0.0008) [2023-12-27 18:04:55,759][20381] Updated weights for policy 1, policy_version 1793586 (0.0005) [2023-12-27 18:04:56,143][20347] Updated weights for policy 0, policy_version 1795110 (0.0007) [2023-12-27 18:04:56,207][20347] Updated weights for policy 0, policy_version 1795120 (0.0007) [2023-12-27 18:04:56,256][20347] Updated weights for policy 0, policy_version 1795130 (0.0005) [2023-12-27 18:04:56,446][20381] Updated weights for policy 1, policy_version 1793596 (0.0010) [2023-12-27 18:04:56,500][20381] Updated weights for policy 1, policy_version 1793606 (0.0010) [2023-12-27 18:04:56,551][20381] Updated weights for policy 1, policy_version 1793616 (0.0010) [2023-12-27 18:04:56,794][20347] Updated weights for policy 0, policy_version 1795140 (0.0006) [2023-12-27 18:04:56,850][20347] Updated weights for policy 0, policy_version 1795150 (0.0009) [2023-12-27 18:04:56,900][20347] Updated weights for policy 0, policy_version 1795160 (0.0005) [2023-12-27 18:04:57,238][20381] Updated weights for policy 1, policy_version 1793626 (0.0010) [2023-12-27 18:04:57,295][20381] Updated weights for policy 1, policy_version 1793636 (0.0010) [2023-12-27 18:04:57,343][20381] Updated weights for policy 1, policy_version 1793646 (0.0009) [2023-12-27 18:04:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 918863872. Throughput: 0: 10382.4, 1: 10196.6. Samples: 918856672. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:04:57,363][19170] Avg episode reward: [(0, '961.000'), (1, '990.270')] [2023-12-27 18:04:57,394][20381] Updated weights for policy 1, policy_version 1793656 (0.0011) [2023-12-27 18:04:57,445][20347] Updated weights for policy 0, policy_version 1795170 (0.0005) [2023-12-27 18:04:57,497][20347] Updated weights for policy 0, policy_version 1795180 (0.0009) [2023-12-27 18:04:57,548][20347] Updated weights for policy 0, policy_version 1795190 (0.0010) [2023-12-27 18:04:57,596][20347] Updated weights for policy 0, policy_version 1795200 (0.0010) [2023-12-27 18:04:58,082][20381] Updated weights for policy 1, policy_version 1793666 (0.0011) [2023-12-27 18:04:58,136][20381] Updated weights for policy 1, policy_version 1793676 (0.0010) [2023-12-27 18:04:58,195][20381] Updated weights for policy 1, policy_version 1793686 (0.0011) [2023-12-27 18:04:58,235][20347] Updated weights for policy 0, policy_version 1795210 (0.0009) [2023-12-27 18:04:58,295][20347] Updated weights for policy 0, policy_version 1795220 (0.0010) [2023-12-27 18:04:58,364][20347] Updated weights for policy 0, policy_version 1795230 (0.0009) [2023-12-27 18:04:58,984][20381] Updated weights for policy 1, policy_version 1793696 (0.0007) [2023-12-27 18:04:59,043][20381] Updated weights for policy 1, policy_version 1793706 (0.0007) [2023-12-27 18:04:59,098][20381] Updated weights for policy 1, policy_version 1793716 (0.0008) [2023-12-27 18:04:59,135][20347] Updated weights for policy 0, policy_version 1795240 (0.0008) [2023-12-27 18:04:59,204][20347] Updated weights for policy 0, policy_version 1795250 (0.0006) [2023-12-27 18:04:59,274][20347] Updated weights for policy 0, policy_version 1795260 (0.0008) [2023-12-27 18:04:59,881][20381] Updated weights for policy 1, policy_version 1793726 (0.0007) [2023-12-27 18:04:59,906][20347] Updated weights for policy 0, policy_version 1795270 (0.0007) [2023-12-27 18:04:59,948][20381] Updated weights for policy 1, policy_version 1793736 (0.0008) [2023-12-27 18:04:59,972][20347] Updated weights for policy 0, policy_version 1795280 (0.0010) [2023-12-27 18:05:00,008][20381] Updated weights for policy 1, policy_version 1793746 (0.0006) [2023-12-27 18:05:00,033][20347] Updated weights for policy 0, policy_version 1795290 (0.0011) [2023-12-27 18:05:00,641][20381] Updated weights for policy 1, policy_version 1793756 (0.0006) [2023-12-27 18:05:00,697][20381] Updated weights for policy 1, policy_version 1793766 (0.0005) [2023-12-27 18:05:00,705][20347] Updated weights for policy 0, policy_version 1795300 (0.0010) [2023-12-27 18:05:00,756][20347] Updated weights for policy 0, policy_version 1795310 (0.0010) [2023-12-27 18:05:00,758][20381] Updated weights for policy 1, policy_version 1793776 (0.0006) [2023-12-27 18:05:00,808][20347] Updated weights for policy 0, policy_version 1795320 (0.0010) [2023-12-27 18:05:01,344][20381] Updated weights for policy 1, policy_version 1793786 (0.0006) [2023-12-27 18:05:01,406][20381] Updated weights for policy 1, policy_version 1793796 (0.0009) [2023-12-27 18:05:01,455][20381] Updated weights for policy 1, policy_version 1793806 (0.0008) [2023-12-27 18:05:01,485][20347] Updated weights for policy 0, policy_version 1795330 (0.0009) [2023-12-27 18:05:01,506][20381] Updated weights for policy 1, policy_version 1793816 (0.0008) [2023-12-27 18:05:01,542][20347] Updated weights for policy 0, policy_version 1795340 (0.0005) [2023-12-27 18:05:01,608][20347] Updated weights for policy 0, policy_version 1795350 (0.0005) [2023-12-27 18:05:01,673][20347] Updated weights for policy 0, policy_version 1795360 (0.0008) [2023-12-27 18:05:02,190][20381] Updated weights for policy 1, policy_version 1793826 (0.0008) [2023-12-27 18:05:02,244][20381] Updated weights for policy 1, policy_version 1793836 (0.0008) [2023-12-27 18:05:02,306][20381] Updated weights for policy 1, policy_version 1793846 (0.0006) [2023-12-27 18:05:02,323][20347] Updated weights for policy 0, policy_version 1795370 (0.0008) [2023-12-27 18:05:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 918970368. Throughput: 0: 10473.5, 1: 10177.1. Samples: 918980996. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:02,363][19170] Avg episode reward: [(0, '970.840'), (1, '1000.000')] [2023-12-27 18:05:02,394][20347] Updated weights for policy 0, policy_version 1795380 (0.0009) [2023-12-27 18:05:02,461][20347] Updated weights for policy 0, policy_version 1795390 (0.0008) [2023-12-27 18:05:03,001][20381] Updated weights for policy 1, policy_version 1793856 (0.0007) [2023-12-27 18:05:03,050][20381] Updated weights for policy 1, policy_version 1793866 (0.0008) [2023-12-27 18:05:03,108][20381] Updated weights for policy 1, policy_version 1793876 (0.0008) [2023-12-27 18:05:03,113][20347] Updated weights for policy 0, policy_version 1795400 (0.0009) [2023-12-27 18:05:03,166][20347] Updated weights for policy 0, policy_version 1795410 (0.0009) [2023-12-27 18:05:03,220][20347] Updated weights for policy 0, policy_version 1795420 (0.0009) [2023-12-27 18:05:03,802][20381] Updated weights for policy 1, policy_version 1793886 (0.0007) [2023-12-27 18:05:03,860][20381] Updated weights for policy 1, policy_version 1793896 (0.0008) [2023-12-27 18:05:03,870][20347] Updated weights for policy 0, policy_version 1795430 (0.0008) [2023-12-27 18:05:03,916][20381] Updated weights for policy 1, policy_version 1793906 (0.0008) [2023-12-27 18:05:03,919][20347] Updated weights for policy 0, policy_version 1795440 (0.0006) [2023-12-27 18:05:03,980][20347] Updated weights for policy 0, policy_version 1795450 (0.0006) [2023-12-27 18:05:04,684][20347] Updated weights for policy 0, policy_version 1795460 (0.0007) [2023-12-27 18:05:04,694][20381] Updated weights for policy 1, policy_version 1793916 (0.0008) [2023-12-27 18:05:04,740][20347] Updated weights for policy 0, policy_version 1795470 (0.0007) [2023-12-27 18:05:04,743][20381] Updated weights for policy 1, policy_version 1793926 (0.0006) [2023-12-27 18:05:04,794][20347] Updated weights for policy 0, policy_version 1795480 (0.0008) [2023-12-27 18:05:04,796][20381] Updated weights for policy 1, policy_version 1793936 (0.0006) [2023-12-27 18:05:05,508][20347] Updated weights for policy 0, policy_version 1795490 (0.0006) [2023-12-27 18:05:05,510][20381] Updated weights for policy 1, policy_version 1793946 (0.0005) [2023-12-27 18:05:05,560][20347] Updated weights for policy 0, policy_version 1795500 (0.0006) [2023-12-27 18:05:05,565][20381] Updated weights for policy 1, policy_version 1793956 (0.0009) [2023-12-27 18:05:05,612][20347] Updated weights for policy 0, policy_version 1795510 (0.0005) [2023-12-27 18:05:05,617][20381] Updated weights for policy 1, policy_version 1793966 (0.0010) [2023-12-27 18:05:05,662][20347] Updated weights for policy 0, policy_version 1795520 (0.0006) [2023-12-27 18:05:05,669][20381] Updated weights for policy 1, policy_version 1793976 (0.0008) [2023-12-27 18:05:06,319][20381] Updated weights for policy 1, policy_version 1793986 (0.0008) [2023-12-27 18:05:06,378][20381] Updated weights for policy 1, policy_version 1793996 (0.0008) [2023-12-27 18:05:06,383][20347] Updated weights for policy 0, policy_version 1795530 (0.0006) [2023-12-27 18:05:06,438][20381] Updated weights for policy 1, policy_version 1794006 (0.0006) [2023-12-27 18:05:06,443][20347] Updated weights for policy 0, policy_version 1795540 (0.0007) [2023-12-27 18:05:06,505][20347] Updated weights for policy 0, policy_version 1795550 (0.0008) [2023-12-27 18:05:07,141][20381] Updated weights for policy 1, policy_version 1794016 (0.0007) [2023-12-27 18:05:07,192][20381] Updated weights for policy 1, policy_version 1794026 (0.0006) [2023-12-27 18:05:07,195][20347] Updated weights for policy 0, policy_version 1795560 (0.0006) [2023-12-27 18:05:07,239][20381] Updated weights for policy 1, policy_version 1794036 (0.0005) [2023-12-27 18:05:07,249][20347] Updated weights for policy 0, policy_version 1795570 (0.0006) [2023-12-27 18:05:07,293][20347] Updated weights for policy 0, policy_version 1795580 (0.0007) [2023-12-27 18:05:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 919076864. Throughput: 0: 10467.8, 1: 10108.0. Samples: 919041236. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:07,363][19170] Avg episode reward: [(0, '980.480'), (1, '1000.000')] [2023-12-27 18:05:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001794040_459325440.pth... [2023-12-27 18:05:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001795584_459751424.pth... [2023-12-27 18:05:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001792856_459022336.pth [2023-12-27 18:05:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001794368_459440128.pth [2023-12-27 18:05:07,873][20347] Updated weights for policy 0, policy_version 1795590 (0.0006) [2023-12-27 18:05:07,935][20347] Updated weights for policy 0, policy_version 1795600 (0.0005) [2023-12-27 18:05:07,945][20381] Updated weights for policy 1, policy_version 1794046 (0.0006) [2023-12-27 18:05:07,988][20347] Updated weights for policy 0, policy_version 1795610 (0.0006) [2023-12-27 18:05:07,990][20381] Updated weights for policy 1, policy_version 1794056 (0.0005) [2023-12-27 18:05:08,042][20381] Updated weights for policy 1, policy_version 1794066 (0.0011) [2023-12-27 18:05:08,568][20347] Updated weights for policy 0, policy_version 1795620 (0.0006) [2023-12-27 18:05:08,634][20347] Updated weights for policy 0, policy_version 1795630 (0.0008) [2023-12-27 18:05:08,689][20347] Updated weights for policy 0, policy_version 1795640 (0.0007) [2023-12-27 18:05:08,751][20381] Updated weights for policy 1, policy_version 1794076 (0.0011) [2023-12-27 18:05:08,811][20381] Updated weights for policy 1, policy_version 1794086 (0.0011) [2023-12-27 18:05:08,873][20381] Updated weights for policy 1, policy_version 1794096 (0.0011) [2023-12-27 18:05:09,386][20347] Updated weights for policy 0, policy_version 1795650 (0.0007) [2023-12-27 18:05:09,453][20347] Updated weights for policy 0, policy_version 1795660 (0.0008) [2023-12-27 18:05:09,510][20347] Updated weights for policy 0, policy_version 1795670 (0.0008) [2023-12-27 18:05:09,572][20347] Updated weights for policy 0, policy_version 1795680 (0.0010) [2023-12-27 18:05:09,577][20381] Updated weights for policy 1, policy_version 1794106 (0.0010) [2023-12-27 18:05:09,633][20381] Updated weights for policy 1, policy_version 1794116 (0.0008) [2023-12-27 18:05:09,695][20381] Updated weights for policy 1, policy_version 1794126 (0.0008) [2023-12-27 18:05:09,748][20381] Updated weights for policy 1, policy_version 1794136 (0.0008) [2023-12-27 18:05:10,250][20347] Updated weights for policy 0, policy_version 1795690 (0.0011) [2023-12-27 18:05:10,306][20347] Updated weights for policy 0, policy_version 1795700 (0.0010) [2023-12-27 18:05:10,362][20347] Updated weights for policy 0, policy_version 1795710 (0.0010) [2023-12-27 18:05:10,399][20381] Updated weights for policy 1, policy_version 1794146 (0.0006) [2023-12-27 18:05:10,463][20381] Updated weights for policy 1, policy_version 1794156 (0.0006) [2023-12-27 18:05:10,518][20381] Updated weights for policy 1, policy_version 1794166 (0.0006) [2023-12-27 18:05:11,060][20347] Updated weights for policy 0, policy_version 1795720 (0.0008) [2023-12-27 18:05:11,114][20381] Updated weights for policy 1, policy_version 1794176 (0.0006) [2023-12-27 18:05:11,126][20347] Updated weights for policy 0, policy_version 1795730 (0.0007) [2023-12-27 18:05:11,180][20381] Updated weights for policy 1, policy_version 1794186 (0.0007) [2023-12-27 18:05:11,189][20347] Updated weights for policy 0, policy_version 1795740 (0.0007) [2023-12-27 18:05:11,236][20381] Updated weights for policy 1, policy_version 1794196 (0.0007) [2023-12-27 18:05:11,949][20381] Updated weights for policy 1, policy_version 1794206 (0.0007) [2023-12-27 18:05:11,956][20347] Updated weights for policy 0, policy_version 1795750 (0.0008) [2023-12-27 18:05:12,010][20347] Updated weights for policy 0, policy_version 1795760 (0.0008) [2023-12-27 18:05:12,012][20381] Updated weights for policy 1, policy_version 1794216 (0.0006) [2023-12-27 18:05:12,064][20347] Updated weights for policy 0, policy_version 1795770 (0.0007) [2023-12-27 18:05:12,069][20381] Updated weights for policy 1, policy_version 1794226 (0.0007) [2023-12-27 18:05:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 919175168. Throughput: 0: 10468.0, 1: 10108.3. Samples: 919164436. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:12,363][19170] Avg episode reward: [(0, '980.470'), (1, '990.260')] [2023-12-27 18:05:12,690][20381] Updated weights for policy 1, policy_version 1794236 (0.0007) [2023-12-27 18:05:12,748][20347] Updated weights for policy 0, policy_version 1795780 (0.0005) [2023-12-27 18:05:12,755][20381] Updated weights for policy 1, policy_version 1794246 (0.0007) [2023-12-27 18:05:12,807][20347] Updated weights for policy 0, policy_version 1795790 (0.0007) [2023-12-27 18:05:12,813][20381] Updated weights for policy 1, policy_version 1794256 (0.0006) [2023-12-27 18:05:12,868][20347] Updated weights for policy 0, policy_version 1795800 (0.0006) [2023-12-27 18:05:13,519][20381] Updated weights for policy 1, policy_version 1794266 (0.0006) [2023-12-27 18:05:13,527][20347] Updated weights for policy 0, policy_version 1795810 (0.0007) [2023-12-27 18:05:13,577][20381] Updated weights for policy 1, policy_version 1794276 (0.0006) [2023-12-27 18:05:13,581][20347] Updated weights for policy 0, policy_version 1795820 (0.0007) [2023-12-27 18:05:13,629][20347] Updated weights for policy 0, policy_version 1795830 (0.0005) [2023-12-27 18:05:13,636][20381] Updated weights for policy 1, policy_version 1794286 (0.0007) [2023-12-27 18:05:13,680][20347] Updated weights for policy 0, policy_version 1795840 (0.0005) [2023-12-27 18:05:13,693][20381] Updated weights for policy 1, policy_version 1794296 (0.0007) [2023-12-27 18:05:14,259][20347] Updated weights for policy 0, policy_version 1795850 (0.0005) [2023-12-27 18:05:14,312][20347] Updated weights for policy 0, policy_version 1795860 (0.0006) [2023-12-27 18:05:14,363][20381] Updated weights for policy 1, policy_version 1794306 (0.0006) [2023-12-27 18:05:14,365][20347] Updated weights for policy 0, policy_version 1795870 (0.0007) [2023-12-27 18:05:14,420][20381] Updated weights for policy 1, policy_version 1794316 (0.0010) [2023-12-27 18:05:14,481][20381] Updated weights for policy 1, policy_version 1794326 (0.0009) [2023-12-27 18:05:15,081][20347] Updated weights for policy 0, policy_version 1795880 (0.0010) [2023-12-27 18:05:15,134][20347] Updated weights for policy 0, policy_version 1795890 (0.0010) [2023-12-27 18:05:15,170][20381] Updated weights for policy 1, policy_version 1794336 (0.0008) [2023-12-27 18:05:15,184][20347] Updated weights for policy 0, policy_version 1795900 (0.0010) [2023-12-27 18:05:15,226][20381] Updated weights for policy 1, policy_version 1794346 (0.0008) [2023-12-27 18:05:15,281][20381] Updated weights for policy 1, policy_version 1794356 (0.0007) [2023-12-27 18:05:15,796][20347] Updated weights for policy 0, policy_version 1795910 (0.0010) [2023-12-27 18:05:15,849][20347] Updated weights for policy 0, policy_version 1795920 (0.0010) [2023-12-27 18:05:15,898][20347] Updated weights for policy 0, policy_version 1795930 (0.0010) [2023-12-27 18:05:15,979][20381] Updated weights for policy 1, policy_version 1794366 (0.0009) [2023-12-27 18:05:16,027][20381] Updated weights for policy 1, policy_version 1794376 (0.0010) [2023-12-27 18:05:16,088][20381] Updated weights for policy 1, policy_version 1794386 (0.0011) [2023-12-27 18:05:16,586][20347] Updated weights for policy 0, policy_version 1795940 (0.0010) [2023-12-27 18:05:16,645][20347] Updated weights for policy 0, policy_version 1795950 (0.0010) [2023-12-27 18:05:16,699][20347] Updated weights for policy 0, policy_version 1795960 (0.0010) [2023-12-27 18:05:16,781][20381] Updated weights for policy 1, policy_version 1794396 (0.0010) [2023-12-27 18:05:16,835][20381] Updated weights for policy 1, policy_version 1794406 (0.0010) [2023-12-27 18:05:16,883][20381] Updated weights for policy 1, policy_version 1794416 (0.0010) [2023-12-27 18:05:17,354][20347] Updated weights for policy 0, policy_version 1795970 (0.0010) [2023-12-27 18:05:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 919273472. Throughput: 0: 10377.1, 1: 10090.7. Samples: 919286812. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:17,363][19170] Avg episode reward: [(0, '961.000'), (1, '970.810')] [2023-12-27 18:05:17,397][20347] Updated weights for policy 0, policy_version 1795980 (0.0007) [2023-12-27 18:05:17,449][20347] Updated weights for policy 0, policy_version 1795990 (0.0007) [2023-12-27 18:05:17,501][20347] Updated weights for policy 0, policy_version 1796000 (0.0008) [2023-12-27 18:05:17,568][20381] Updated weights for policy 1, policy_version 1794426 (0.0010) [2023-12-27 18:05:17,622][20381] Updated weights for policy 1, policy_version 1794436 (0.0005) [2023-12-27 18:05:17,686][20381] Updated weights for policy 1, policy_version 1794446 (0.0007) [2023-12-27 18:05:17,742][20381] Updated weights for policy 1, policy_version 1794456 (0.0007) [2023-12-27 18:05:18,136][20347] Updated weights for policy 0, policy_version 1796010 (0.0005) [2023-12-27 18:05:18,190][20347] Updated weights for policy 0, policy_version 1796020 (0.0005) [2023-12-27 18:05:18,244][20347] Updated weights for policy 0, policy_version 1796030 (0.0007) [2023-12-27 18:05:18,321][20381] Updated weights for policy 1, policy_version 1794466 (0.0006) [2023-12-27 18:05:18,377][20381] Updated weights for policy 1, policy_version 1794476 (0.0011) [2023-12-27 18:05:18,436][20381] Updated weights for policy 1, policy_version 1794486 (0.0011) [2023-12-27 18:05:18,855][20347] Updated weights for policy 0, policy_version 1796040 (0.0008) [2023-12-27 18:05:18,907][20347] Updated weights for policy 0, policy_version 1796050 (0.0007) [2023-12-27 18:05:18,957][20347] Updated weights for policy 0, policy_version 1796060 (0.0007) [2023-12-27 18:05:19,042][20381] Updated weights for policy 1, policy_version 1794496 (0.0011) [2023-12-27 18:05:19,091][20381] Updated weights for policy 1, policy_version 1794506 (0.0006) [2023-12-27 18:05:19,139][20381] Updated weights for policy 1, policy_version 1794516 (0.0005) [2023-12-27 18:05:19,689][20347] Updated weights for policy 0, policy_version 1796070 (0.0008) [2023-12-27 18:05:19,755][20347] Updated weights for policy 0, policy_version 1796080 (0.0009) [2023-12-27 18:05:19,814][20347] Updated weights for policy 0, policy_version 1796090 (0.0008) [2023-12-27 18:05:19,855][20381] Updated weights for policy 1, policy_version 1794526 (0.0008) [2023-12-27 18:05:19,917][20381] Updated weights for policy 1, policy_version 1794536 (0.0008) [2023-12-27 18:05:19,977][20381] Updated weights for policy 1, policy_version 1794546 (0.0009) [2023-12-27 18:05:20,509][20347] Updated weights for policy 0, policy_version 1796100 (0.0008) [2023-12-27 18:05:20,572][20347] Updated weights for policy 0, policy_version 1796110 (0.0008) [2023-12-27 18:05:20,640][20347] Updated weights for policy 0, policy_version 1796120 (0.0007) [2023-12-27 18:05:20,655][20381] Updated weights for policy 1, policy_version 1794556 (0.0007) [2023-12-27 18:05:20,707][20381] Updated weights for policy 1, policy_version 1794566 (0.0007) [2023-12-27 18:05:20,774][20381] Updated weights for policy 1, policy_version 1794576 (0.0006) [2023-12-27 18:05:21,329][20347] Updated weights for policy 0, policy_version 1796130 (0.0006) [2023-12-27 18:05:21,399][20347] Updated weights for policy 0, policy_version 1796140 (0.0008) [2023-12-27 18:05:21,459][20347] Updated weights for policy 0, policy_version 1796150 (0.0006) [2023-12-27 18:05:21,523][20347] Updated weights for policy 0, policy_version 1796160 (0.0007) [2023-12-27 18:05:21,553][20381] Updated weights for policy 1, policy_version 1794586 (0.0008) [2023-12-27 18:05:21,612][20381] Updated weights for policy 1, policy_version 1794596 (0.0008) [2023-12-27 18:05:21,679][20381] Updated weights for policy 1, policy_version 1794606 (0.0008) [2023-12-27 18:05:21,748][20381] Updated weights for policy 1, policy_version 1794616 (0.0008) [2023-12-27 18:05:22,201][20347] Updated weights for policy 0, policy_version 1796170 (0.0005) [2023-12-27 18:05:22,286][20347] Updated weights for policy 0, policy_version 1796180 (0.0006) [2023-12-27 18:05:22,350][20347] Updated weights for policy 0, policy_version 1796190 (0.0008) [2023-12-27 18:05:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 919371776. Throughput: 0: 10350.6, 1: 10160.5. Samples: 919350732. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:22,363][19170] Avg episode reward: [(0, '941.480'), (1, '990.270')] [2023-12-27 18:05:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001794616_459472896.pth... [2023-12-27 18:05:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001796192_459907072.pth... [2023-12-27 18:05:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001793432_459169792.pth [2023-12-27 18:05:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001794944_459587584.pth [2023-12-27 18:05:22,513][20381] Updated weights for policy 1, policy_version 1794626 (0.0008) [2023-12-27 18:05:22,571][20381] Updated weights for policy 1, policy_version 1794636 (0.0008) [2023-12-27 18:05:22,630][20381] Updated weights for policy 1, policy_version 1794646 (0.0007) [2023-12-27 18:05:22,997][20347] Updated weights for policy 0, policy_version 1796200 (0.0008) [2023-12-27 18:05:23,056][20347] Updated weights for policy 0, policy_version 1796210 (0.0008) [2023-12-27 18:05:23,111][20347] Updated weights for policy 0, policy_version 1796220 (0.0008) [2023-12-27 18:05:23,326][20381] Updated weights for policy 1, policy_version 1794656 (0.0008) [2023-12-27 18:05:23,375][20381] Updated weights for policy 1, policy_version 1794666 (0.0006) [2023-12-27 18:05:23,423][20381] Updated weights for policy 1, policy_version 1794676 (0.0005) [2023-12-27 18:05:23,807][20347] Updated weights for policy 0, policy_version 1796230 (0.0008) [2023-12-27 18:05:23,864][20347] Updated weights for policy 0, policy_version 1796240 (0.0007) [2023-12-27 18:05:23,922][20347] Updated weights for policy 0, policy_version 1796250 (0.0008) [2023-12-27 18:05:24,104][20381] Updated weights for policy 1, policy_version 1794686 (0.0007) [2023-12-27 18:05:24,155][20381] Updated weights for policy 1, policy_version 1794696 (0.0007) [2023-12-27 18:05:24,201][20381] Updated weights for policy 1, policy_version 1794706 (0.0007) [2023-12-27 18:05:24,545][20347] Updated weights for policy 0, policy_version 1796260 (0.0007) [2023-12-27 18:05:24,604][20347] Updated weights for policy 0, policy_version 1796270 (0.0008) [2023-12-27 18:05:24,662][20347] Updated weights for policy 0, policy_version 1796280 (0.0008) [2023-12-27 18:05:24,911][20381] Updated weights for policy 1, policy_version 1794716 (0.0008) [2023-12-27 18:05:24,957][20381] Updated weights for policy 1, policy_version 1794726 (0.0008) [2023-12-27 18:05:25,006][20381] Updated weights for policy 1, policy_version 1794736 (0.0008) [2023-12-27 18:05:25,350][20347] Updated weights for policy 0, policy_version 1796290 (0.0007) [2023-12-27 18:05:25,407][20347] Updated weights for policy 0, policy_version 1796300 (0.0008) [2023-12-27 18:05:25,457][20347] Updated weights for policy 0, policy_version 1796310 (0.0007) [2023-12-27 18:05:25,506][20347] Updated weights for policy 0, policy_version 1796320 (0.0008) [2023-12-27 18:05:25,685][20381] Updated weights for policy 1, policy_version 1794746 (0.0008) [2023-12-27 18:05:25,734][20381] Updated weights for policy 1, policy_version 1794756 (0.0008) [2023-12-27 18:05:25,779][20381] Updated weights for policy 1, policy_version 1794766 (0.0007) [2023-12-27 18:05:25,829][20381] Updated weights for policy 1, policy_version 1794776 (0.0008) [2023-12-27 18:05:26,186][20347] Updated weights for policy 0, policy_version 1796330 (0.0008) [2023-12-27 18:05:26,240][20347] Updated weights for policy 0, policy_version 1796340 (0.0007) [2023-12-27 18:05:26,294][20347] Updated weights for policy 0, policy_version 1796350 (0.0008) [2023-12-27 18:05:26,537][20381] Updated weights for policy 1, policy_version 1794786 (0.0008) [2023-12-27 18:05:26,599][20381] Updated weights for policy 1, policy_version 1794796 (0.0007) [2023-12-27 18:05:26,657][20381] Updated weights for policy 1, policy_version 1794806 (0.0005) [2023-12-27 18:05:26,946][20347] Updated weights for policy 0, policy_version 1796360 (0.0009) [2023-12-27 18:05:27,005][20347] Updated weights for policy 0, policy_version 1796370 (0.0008) [2023-12-27 18:05:27,049][20347] Updated weights for policy 0, policy_version 1796380 (0.0008) [2023-12-27 18:05:27,196][20381] Updated weights for policy 1, policy_version 1794816 (0.0007) [2023-12-27 18:05:27,254][20381] Updated weights for policy 1, policy_version 1794826 (0.0007) [2023-12-27 18:05:27,307][20381] Updated weights for policy 1, policy_version 1794836 (0.0008) [2023-12-27 18:05:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 919486464. Throughput: 0: 10325.2, 1: 10148.5. Samples: 919470108. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:27,363][19170] Avg episode reward: [(0, '951.190'), (1, '1000.000')] [2023-12-27 18:05:27,730][20347] Updated weights for policy 0, policy_version 1796390 (0.0008) [2023-12-27 18:05:27,778][20347] Updated weights for policy 0, policy_version 1796400 (0.0007) [2023-12-27 18:05:27,825][20347] Updated weights for policy 0, policy_version 1796410 (0.0008) [2023-12-27 18:05:27,990][20381] Updated weights for policy 1, policy_version 1794846 (0.0007) [2023-12-27 18:05:28,039][20381] Updated weights for policy 1, policy_version 1794856 (0.0008) [2023-12-27 18:05:28,087][20381] Updated weights for policy 1, policy_version 1794866 (0.0008) [2023-12-27 18:05:28,549][20347] Updated weights for policy 0, policy_version 1796420 (0.0007) [2023-12-27 18:05:28,602][20347] Updated weights for policy 0, policy_version 1796430 (0.0007) [2023-12-27 18:05:28,650][20347] Updated weights for policy 0, policy_version 1796440 (0.0009) [2023-12-27 18:05:28,773][20381] Updated weights for policy 1, policy_version 1794876 (0.0007) [2023-12-27 18:05:28,833][20381] Updated weights for policy 1, policy_version 1794886 (0.0007) [2023-12-27 18:05:28,883][20381] Updated weights for policy 1, policy_version 1794896 (0.0008) [2023-12-27 18:05:29,385][20347] Updated weights for policy 0, policy_version 1796450 (0.0007) [2023-12-27 18:05:29,444][20347] Updated weights for policy 0, policy_version 1796460 (0.0008) [2023-12-27 18:05:29,502][20347] Updated weights for policy 0, policy_version 1796470 (0.0005) [2023-12-27 18:05:29,556][20347] Updated weights for policy 0, policy_version 1796480 (0.0006) [2023-12-27 18:05:29,571][20381] Updated weights for policy 1, policy_version 1794906 (0.0008) [2023-12-27 18:05:29,638][20381] Updated weights for policy 1, policy_version 1794916 (0.0008) [2023-12-27 18:05:29,696][20381] Updated weights for policy 1, policy_version 1794926 (0.0008) [2023-12-27 18:05:29,750][20381] Updated weights for policy 1, policy_version 1794936 (0.0008) [2023-12-27 18:05:30,201][20347] Updated weights for policy 0, policy_version 1796490 (0.0007) [2023-12-27 18:05:30,261][20347] Updated weights for policy 0, policy_version 1796500 (0.0006) [2023-12-27 18:05:30,310][20347] Updated weights for policy 0, policy_version 1796510 (0.0005) [2023-12-27 18:05:30,465][20381] Updated weights for policy 1, policy_version 1794946 (0.0007) [2023-12-27 18:05:30,523][20381] Updated weights for policy 1, policy_version 1794956 (0.0008) [2023-12-27 18:05:30,570][20381] Updated weights for policy 1, policy_version 1794966 (0.0007) [2023-12-27 18:05:30,890][20347] Updated weights for policy 0, policy_version 1796520 (0.0005) [2023-12-27 18:05:30,941][20347] Updated weights for policy 0, policy_version 1796530 (0.0005) [2023-12-27 18:05:30,989][20347] Updated weights for policy 0, policy_version 1796540 (0.0006) [2023-12-27 18:05:31,278][20381] Updated weights for policy 1, policy_version 1794976 (0.0007) [2023-12-27 18:05:31,341][20381] Updated weights for policy 1, policy_version 1794986 (0.0007) [2023-12-27 18:05:31,406][20381] Updated weights for policy 1, policy_version 1794996 (0.0009) [2023-12-27 18:05:31,678][20347] Updated weights for policy 0, policy_version 1796550 (0.0008) [2023-12-27 18:05:31,742][20347] Updated weights for policy 0, policy_version 1796560 (0.0008) [2023-12-27 18:05:31,794][20347] Updated weights for policy 0, policy_version 1796570 (0.0007) [2023-12-27 18:05:32,156][20381] Updated weights for policy 1, policy_version 1795006 (0.0008) [2023-12-27 18:05:32,216][20381] Updated weights for policy 1, policy_version 1795016 (0.0005) [2023-12-27 18:05:32,277][20381] Updated weights for policy 1, policy_version 1795026 (0.0007) [2023-12-27 18:05:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 919584768. Throughput: 0: 10362.2, 1: 10144.9. Samples: 919593356. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:32,363][19170] Avg episode reward: [(0, '941.500'), (1, '1000.000')] [2023-12-27 18:05:32,468][20347] Updated weights for policy 0, policy_version 1796580 (0.0007) [2023-12-27 18:05:32,520][20347] Updated weights for policy 0, policy_version 1796590 (0.0007) [2023-12-27 18:05:32,572][20347] Updated weights for policy 0, policy_version 1796600 (0.0007) [2023-12-27 18:05:32,902][20381] Updated weights for policy 1, policy_version 1795036 (0.0006) [2023-12-27 18:05:32,963][20381] Updated weights for policy 1, policy_version 1795046 (0.0008) [2023-12-27 18:05:33,009][20381] Updated weights for policy 1, policy_version 1795056 (0.0007) [2023-12-27 18:05:33,240][20347] Updated weights for policy 0, policy_version 1796610 (0.0008) [2023-12-27 18:05:33,284][20347] Updated weights for policy 0, policy_version 1796620 (0.0007) [2023-12-27 18:05:33,331][20347] Updated weights for policy 0, policy_version 1796630 (0.0007) [2023-12-27 18:05:33,377][20347] Updated weights for policy 0, policy_version 1796640 (0.0010) [2023-12-27 18:05:33,621][20381] Updated weights for policy 1, policy_version 1795066 (0.0006) [2023-12-27 18:05:33,670][20381] Updated weights for policy 1, policy_version 1795076 (0.0008) [2023-12-27 18:05:33,721][20381] Updated weights for policy 1, policy_version 1795086 (0.0007) [2023-12-27 18:05:33,782][20381] Updated weights for policy 1, policy_version 1795096 (0.0008) [2023-12-27 18:05:34,078][20347] Updated weights for policy 0, policy_version 1796650 (0.0005) [2023-12-27 18:05:34,131][20347] Updated weights for policy 0, policy_version 1796660 (0.0007) [2023-12-27 18:05:34,198][20347] Updated weights for policy 0, policy_version 1796670 (0.0008) [2023-12-27 18:05:34,498][20381] Updated weights for policy 1, policy_version 1795106 (0.0007) [2023-12-27 18:05:34,558][20381] Updated weights for policy 1, policy_version 1795116 (0.0008) [2023-12-27 18:05:34,611][20381] Updated weights for policy 1, policy_version 1795126 (0.0008) [2023-12-27 18:05:34,893][20347] Updated weights for policy 0, policy_version 1796680 (0.0008) [2023-12-27 18:05:34,954][20347] Updated weights for policy 0, policy_version 1796690 (0.0008) [2023-12-27 18:05:35,012][20347] Updated weights for policy 0, policy_version 1796700 (0.0007) [2023-12-27 18:05:35,282][20381] Updated weights for policy 1, policy_version 1795136 (0.0008) [2023-12-27 18:05:35,327][20381] Updated weights for policy 1, policy_version 1795146 (0.0008) [2023-12-27 18:05:35,376][20381] Updated weights for policy 1, policy_version 1795156 (0.0007) [2023-12-27 18:05:35,700][20347] Updated weights for policy 0, policy_version 1796710 (0.0008) [2023-12-27 18:05:35,758][20347] Updated weights for policy 0, policy_version 1796720 (0.0007) [2023-12-27 18:05:35,815][20347] Updated weights for policy 0, policy_version 1796730 (0.0007) [2023-12-27 18:05:36,099][20381] Updated weights for policy 1, policy_version 1795166 (0.0008) [2023-12-27 18:05:36,165][20381] Updated weights for policy 1, policy_version 1795176 (0.0010) [2023-12-27 18:05:36,218][20381] Updated weights for policy 1, policy_version 1795186 (0.0008) [2023-12-27 18:05:36,431][20347] Updated weights for policy 0, policy_version 1796740 (0.0007) [2023-12-27 18:05:36,492][20347] Updated weights for policy 0, policy_version 1796750 (0.0006) [2023-12-27 18:05:36,551][20347] Updated weights for policy 0, policy_version 1796760 (0.0006) [2023-12-27 18:05:36,947][20381] Updated weights for policy 1, policy_version 1795196 (0.0008) [2023-12-27 18:05:37,004][20381] Updated weights for policy 1, policy_version 1795206 (0.0008) [2023-12-27 18:05:37,062][20381] Updated weights for policy 1, policy_version 1795216 (0.0008) [2023-12-27 18:05:37,245][20347] Updated weights for policy 0, policy_version 1796770 (0.0008) [2023-12-27 18:05:37,307][20347] Updated weights for policy 0, policy_version 1796780 (0.0008) [2023-12-27 18:05:37,362][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 919683072. Throughput: 0: 10392.1, 1: 10169.7. Samples: 919655356. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:37,363][19170] Avg episode reward: [(0, '951.230'), (1, '1000.000')] [2023-12-27 18:05:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001795224_459628544.pth... [2023-12-27 18:05:37,372][20347] Updated weights for policy 0, policy_version 1796790 (0.0008) [2023-12-27 18:05:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001794040_459325440.pth [2023-12-27 18:05:37,435][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001796800_460062720.pth... [2023-12-27 18:05:37,435][20347] Updated weights for policy 0, policy_version 1796800 (0.0008) [2023-12-27 18:05:37,439][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001795584_459751424.pth [2023-12-27 18:05:37,651][20381] Updated weights for policy 1, policy_version 1795226 (0.0006) [2023-12-27 18:05:37,710][20381] Updated weights for policy 1, policy_version 1795236 (0.0005) [2023-12-27 18:05:37,772][20381] Updated weights for policy 1, policy_version 1795246 (0.0007) [2023-12-27 18:05:37,827][20381] Updated weights for policy 1, policy_version 1795256 (0.0007) [2023-12-27 18:05:38,196][20347] Updated weights for policy 0, policy_version 1796810 (0.0007) [2023-12-27 18:05:38,249][20347] Updated weights for policy 0, policy_version 1796820 (0.0007) [2023-12-27 18:05:38,309][20347] Updated weights for policy 0, policy_version 1796830 (0.0007) [2023-12-27 18:05:38,367][20381] Updated weights for policy 1, policy_version 1795266 (0.0007) [2023-12-27 18:05:38,421][20381] Updated weights for policy 1, policy_version 1795276 (0.0008) [2023-12-27 18:05:38,481][20381] Updated weights for policy 1, policy_version 1795286 (0.0008) [2023-12-27 18:05:39,016][20347] Updated weights for policy 0, policy_version 1796840 (0.0006) [2023-12-27 18:05:39,065][20347] Updated weights for policy 0, policy_version 1796850 (0.0008) [2023-12-27 18:05:39,116][20347] Updated weights for policy 0, policy_version 1796860 (0.0007) [2023-12-27 18:05:39,166][20381] Updated weights for policy 1, policy_version 1795296 (0.0008) [2023-12-27 18:05:39,225][20381] Updated weights for policy 1, policy_version 1795306 (0.0008) [2023-12-27 18:05:39,291][20381] Updated weights for policy 1, policy_version 1795316 (0.0008) [2023-12-27 18:05:39,857][20347] Updated weights for policy 0, policy_version 1796870 (0.0008) [2023-12-27 18:05:39,917][20347] Updated weights for policy 0, policy_version 1796880 (0.0008) [2023-12-27 18:05:39,980][20347] Updated weights for policy 0, policy_version 1796890 (0.0008) [2023-12-27 18:05:40,076][20381] Updated weights for policy 1, policy_version 1795326 (0.0008) [2023-12-27 18:05:40,131][20381] Updated weights for policy 1, policy_version 1795336 (0.0007) [2023-12-27 18:05:40,195][20381] Updated weights for policy 1, policy_version 1795346 (0.0008) [2023-12-27 18:05:40,747][20347] Updated weights for policy 0, policy_version 1796900 (0.0008) [2023-12-27 18:05:40,799][20347] Updated weights for policy 0, policy_version 1796910 (0.0007) [2023-12-27 18:05:40,834][20381] Updated weights for policy 1, policy_version 1795356 (0.0006) [2023-12-27 18:05:40,848][20347] Updated weights for policy 0, policy_version 1796920 (0.0007) [2023-12-27 18:05:40,892][20381] Updated weights for policy 1, policy_version 1795366 (0.0005) [2023-12-27 18:05:40,954][20381] Updated weights for policy 1, policy_version 1795376 (0.0008) [2023-12-27 18:05:41,507][20347] Updated weights for policy 0, policy_version 1796930 (0.0007) [2023-12-27 18:05:41,570][20347] Updated weights for policy 0, policy_version 1796940 (0.0008) [2023-12-27 18:05:41,635][20347] Updated weights for policy 0, policy_version 1796950 (0.0008) [2023-12-27 18:05:41,694][20347] Updated weights for policy 0, policy_version 1796960 (0.0007) [2023-12-27 18:05:41,708][20381] Updated weights for policy 1, policy_version 1795386 (0.0008) [2023-12-27 18:05:41,778][20381] Updated weights for policy 1, policy_version 1795396 (0.0008) [2023-12-27 18:05:41,836][20381] Updated weights for policy 1, policy_version 1795406 (0.0008) [2023-12-27 18:05:41,893][20381] Updated weights for policy 1, policy_version 1795416 (0.0008) [2023-12-27 18:05:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20382.8). Total num frames: 919781376. Throughput: 0: 10278.7, 1: 10140.6. Samples: 919775544. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:42,363][19170] Avg episode reward: [(0, '970.800'), (1, '990.280')] [2023-12-27 18:05:42,410][20347] Updated weights for policy 0, policy_version 1796970 (0.0009) [2023-12-27 18:05:42,465][20347] Updated weights for policy 0, policy_version 1796980 (0.0011) [2023-12-27 18:05:42,519][20347] Updated weights for policy 0, policy_version 1796990 (0.0010) [2023-12-27 18:05:42,599][20381] Updated weights for policy 1, policy_version 1795426 (0.0008) [2023-12-27 18:05:42,661][20381] Updated weights for policy 1, policy_version 1795436 (0.0008) [2023-12-27 18:05:42,719][20381] Updated weights for policy 1, policy_version 1795446 (0.0008) [2023-12-27 18:05:43,182][20347] Updated weights for policy 0, policy_version 1797000 (0.0010) [2023-12-27 18:05:43,240][20347] Updated weights for policy 0, policy_version 1797010 (0.0010) [2023-12-27 18:05:43,289][20347] Updated weights for policy 0, policy_version 1797020 (0.0010) [2023-12-27 18:05:43,358][20381] Updated weights for policy 1, policy_version 1795456 (0.0006) [2023-12-27 18:05:43,412][20381] Updated weights for policy 1, policy_version 1795466 (0.0005) [2023-12-27 18:05:43,463][20381] Updated weights for policy 1, policy_version 1795476 (0.0005) [2023-12-27 18:05:43,895][20347] Updated weights for policy 0, policy_version 1797030 (0.0010) [2023-12-27 18:05:43,950][20347] Updated weights for policy 0, policy_version 1797040 (0.0010) [2023-12-27 18:05:44,007][20347] Updated weights for policy 0, policy_version 1797050 (0.0010) [2023-12-27 18:05:44,073][20381] Updated weights for policy 1, policy_version 1795486 (0.0008) [2023-12-27 18:05:44,128][20381] Updated weights for policy 1, policy_version 1795496 (0.0011) [2023-12-27 18:05:44,187][20381] Updated weights for policy 1, policy_version 1795506 (0.0010) [2023-12-27 18:05:44,662][20347] Updated weights for policy 0, policy_version 1797060 (0.0007) [2023-12-27 18:05:44,717][20347] Updated weights for policy 0, policy_version 1797070 (0.0007) [2023-12-27 18:05:44,780][20347] Updated weights for policy 0, policy_version 1797080 (0.0008) [2023-12-27 18:05:44,888][20381] Updated weights for policy 1, policy_version 1795516 (0.0010) [2023-12-27 18:05:44,945][20381] Updated weights for policy 1, policy_version 1795526 (0.0011) [2023-12-27 18:05:45,008][20381] Updated weights for policy 1, policy_version 1795536 (0.0011) [2023-12-27 18:05:45,451][20347] Updated weights for policy 0, policy_version 1797090 (0.0008) [2023-12-27 18:05:45,500][20347] Updated weights for policy 0, policy_version 1797100 (0.0010) [2023-12-27 18:05:45,548][20347] Updated weights for policy 0, policy_version 1797110 (0.0010) [2023-12-27 18:05:45,610][20347] Updated weights for policy 0, policy_version 1797120 (0.0010) [2023-12-27 18:05:45,676][20381] Updated weights for policy 1, policy_version 1795546 (0.0010) [2023-12-27 18:05:45,731][20381] Updated weights for policy 1, policy_version 1795556 (0.0011) [2023-12-27 18:05:45,783][20381] Updated weights for policy 1, policy_version 1795566 (0.0011) [2023-12-27 18:05:45,838][20381] Updated weights for policy 1, policy_version 1795576 (0.0011) [2023-12-27 18:05:46,309][20347] Updated weights for policy 0, policy_version 1797130 (0.0007) [2023-12-27 18:05:46,355][20347] Updated weights for policy 0, policy_version 1797140 (0.0007) [2023-12-27 18:05:46,404][20347] Updated weights for policy 0, policy_version 1797150 (0.0008) [2023-12-27 18:05:46,505][20381] Updated weights for policy 1, policy_version 1795586 (0.0005) [2023-12-27 18:05:46,556][20381] Updated weights for policy 1, policy_version 1795596 (0.0005) [2023-12-27 18:05:46,609][20381] Updated weights for policy 1, policy_version 1795606 (0.0011) [2023-12-27 18:05:47,032][20347] Updated weights for policy 0, policy_version 1797161 (0.0007) [2023-12-27 18:05:47,079][20347] Updated weights for policy 0, policy_version 1797171 (0.0008) [2023-12-27 18:05:47,128][20347] Updated weights for policy 0, policy_version 1797181 (0.0008) [2023-12-27 18:05:47,168][20381] Updated weights for policy 1, policy_version 1795616 (0.0009) [2023-12-27 18:05:47,217][20381] Updated weights for policy 1, policy_version 1795626 (0.0010) [2023-12-27 18:05:47,268][20381] Updated weights for policy 1, policy_version 1795636 (0.0009) [2023-12-27 18:05:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 919896064. Throughput: 0: 10239.6, 1: 10207.6. Samples: 919901116. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:47,363][19170] Avg episode reward: [(0, '970.800'), (1, '990.280')] [2023-12-27 18:05:47,852][20347] Updated weights for policy 0, policy_version 1797191 (0.0007) [2023-12-27 18:05:47,877][20381] Updated weights for policy 1, policy_version 1795646 (0.0005) [2023-12-27 18:05:47,900][20347] Updated weights for policy 0, policy_version 1797201 (0.0007) [2023-12-27 18:05:47,930][20381] Updated weights for policy 1, policy_version 1795656 (0.0005) [2023-12-27 18:05:47,952][20347] Updated weights for policy 0, policy_version 1797211 (0.0007) [2023-12-27 18:05:47,986][20381] Updated weights for policy 1, policy_version 1795666 (0.0005) [2023-12-27 18:05:48,557][20381] Updated weights for policy 1, policy_version 1795676 (0.0007) [2023-12-27 18:05:48,606][20381] Updated weights for policy 1, policy_version 1795686 (0.0007) [2023-12-27 18:05:48,626][20347] Updated weights for policy 0, policy_version 1797221 (0.0007) [2023-12-27 18:05:48,663][20381] Updated weights for policy 1, policy_version 1795696 (0.0006) [2023-12-27 18:05:48,684][20347] Updated weights for policy 0, policy_version 1797231 (0.0006) [2023-12-27 18:05:48,741][20347] Updated weights for policy 0, policy_version 1797241 (0.0007) [2023-12-27 18:05:49,393][20381] Updated weights for policy 1, policy_version 1795706 (0.0006) [2023-12-27 18:05:49,423][20347] Updated weights for policy 0, policy_version 1797251 (0.0008) [2023-12-27 18:05:49,448][20381] Updated weights for policy 1, policy_version 1795716 (0.0007) [2023-12-27 18:05:49,478][20347] Updated weights for policy 0, policy_version 1797261 (0.0008) [2023-12-27 18:05:49,506][20381] Updated weights for policy 1, policy_version 1795726 (0.0007) [2023-12-27 18:05:49,534][20347] Updated weights for policy 0, policy_version 1797271 (0.0008) [2023-12-27 18:05:49,561][20381] Updated weights for policy 1, policy_version 1795736 (0.0007) [2023-12-27 18:05:50,295][20381] Updated weights for policy 1, policy_version 1795746 (0.0011) [2023-12-27 18:05:50,302][20347] Updated weights for policy 0, policy_version 1797281 (0.0008) [2023-12-27 18:05:50,355][20381] Updated weights for policy 1, policy_version 1795756 (0.0010) [2023-12-27 18:05:50,362][20347] Updated weights for policy 0, policy_version 1797291 (0.0009) [2023-12-27 18:05:50,409][20381] Updated weights for policy 1, policy_version 1795766 (0.0011) [2023-12-27 18:05:50,422][20347] Updated weights for policy 0, policy_version 1797301 (0.0006) [2023-12-27 18:05:50,484][20347] Updated weights for policy 0, policy_version 1797311 (0.0006) [2023-12-27 18:05:51,053][20381] Updated weights for policy 1, policy_version 1795776 (0.0008) [2023-12-27 18:05:51,119][20381] Updated weights for policy 1, policy_version 1795786 (0.0008) [2023-12-27 18:05:51,178][20381] Updated weights for policy 1, policy_version 1795796 (0.0008) [2023-12-27 18:05:51,181][20347] Updated weights for policy 0, policy_version 1797321 (0.0006) [2023-12-27 18:05:51,241][20347] Updated weights for policy 0, policy_version 1797331 (0.0007) [2023-12-27 18:05:51,296][20347] Updated weights for policy 0, policy_version 1797341 (0.0008) [2023-12-27 18:05:51,940][20381] Updated weights for policy 1, policy_version 1795806 (0.0007) [2023-12-27 18:05:51,993][20347] Updated weights for policy 0, policy_version 1797351 (0.0006) [2023-12-27 18:05:52,004][20381] Updated weights for policy 1, policy_version 1795816 (0.0006) [2023-12-27 18:05:52,054][20347] Updated weights for policy 0, policy_version 1797361 (0.0007) [2023-12-27 18:05:52,066][20381] Updated weights for policy 1, policy_version 1795826 (0.0006) [2023-12-27 18:05:52,110][20347] Updated weights for policy 0, policy_version 1797371 (0.0006) [2023-12-27 18:05:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 919994368. Throughput: 0: 10218.4, 1: 10247.0. Samples: 919962180. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:52,363][19170] Avg episode reward: [(0, '980.450'), (1, '990.290')] [2023-12-27 18:05:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001797376_460210176.pth... [2023-12-27 18:05:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001795832_459784192.pth... [2023-12-27 18:05:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001796192_459907072.pth [2023-12-27 18:05:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001794616_459472896.pth [2023-12-27 18:05:52,688][20381] Updated weights for policy 1, policy_version 1795836 (0.0007) [2023-12-27 18:05:52,715][20347] Updated weights for policy 0, policy_version 1797381 (0.0006) [2023-12-27 18:05:52,744][20381] Updated weights for policy 1, policy_version 1795846 (0.0008) [2023-12-27 18:05:52,772][20347] Updated weights for policy 0, policy_version 1797391 (0.0007) [2023-12-27 18:05:52,806][20381] Updated weights for policy 1, policy_version 1795856 (0.0008) [2023-12-27 18:05:52,831][20347] Updated weights for policy 0, policy_version 1797401 (0.0006) [2023-12-27 18:05:53,502][20381] Updated weights for policy 1, policy_version 1795866 (0.0008) [2023-12-27 18:05:53,514][20347] Updated weights for policy 0, policy_version 1797411 (0.0007) [2023-12-27 18:05:53,560][20381] Updated weights for policy 1, policy_version 1795876 (0.0005) [2023-12-27 18:05:53,572][20347] Updated weights for policy 0, policy_version 1797421 (0.0006) [2023-12-27 18:05:53,623][20381] Updated weights for policy 1, policy_version 1795886 (0.0005) [2023-12-27 18:05:53,629][20347] Updated weights for policy 0, policy_version 1797431 (0.0006) [2023-12-27 18:05:53,680][20381] Updated weights for policy 1, policy_version 1795896 (0.0005) [2023-12-27 18:05:54,213][20381] Updated weights for policy 1, policy_version 1795906 (0.0007) [2023-12-27 18:05:54,270][20381] Updated weights for policy 1, policy_version 1795916 (0.0005) [2023-12-27 18:05:54,323][20381] Updated weights for policy 1, policy_version 1795926 (0.0009) [2023-12-27 18:05:54,326][20347] Updated weights for policy 0, policy_version 1797441 (0.0007) [2023-12-27 18:05:54,380][20347] Updated weights for policy 0, policy_version 1797451 (0.0007) [2023-12-27 18:05:54,429][20347] Updated weights for policy 0, policy_version 1797461 (0.0007) [2023-12-27 18:05:54,480][20347] Updated weights for policy 0, policy_version 1797471 (0.0007) [2023-12-27 18:05:55,008][20381] Updated weights for policy 1, policy_version 1795936 (0.0009) [2023-12-27 18:05:55,057][20381] Updated weights for policy 1, policy_version 1795946 (0.0008) [2023-12-27 18:05:55,106][20381] Updated weights for policy 1, policy_version 1795956 (0.0008) [2023-12-27 18:05:55,192][20347] Updated weights for policy 0, policy_version 1797481 (0.0007) [2023-12-27 18:05:55,243][20347] Updated weights for policy 0, policy_version 1797491 (0.0007) [2023-12-27 18:05:55,307][20347] Updated weights for policy 0, policy_version 1797501 (0.0006) [2023-12-27 18:05:55,759][20381] Updated weights for policy 1, policy_version 1795966 (0.0009) [2023-12-27 18:05:55,815][20381] Updated weights for policy 1, policy_version 1795976 (0.0008) [2023-12-27 18:05:55,871][20381] Updated weights for policy 1, policy_version 1795986 (0.0006) [2023-12-27 18:05:55,873][20347] Updated weights for policy 0, policy_version 1797511 (0.0006) [2023-12-27 18:05:55,930][20347] Updated weights for policy 0, policy_version 1797521 (0.0007) [2023-12-27 18:05:55,989][20347] Updated weights for policy 0, policy_version 1797531 (0.0008) [2023-12-27 18:05:56,531][20381] Updated weights for policy 1, policy_version 1795996 (0.0007) [2023-12-27 18:05:56,586][20381] Updated weights for policy 1, policy_version 1796006 (0.0008) [2023-12-27 18:05:56,643][20381] Updated weights for policy 1, policy_version 1796016 (0.0007) [2023-12-27 18:05:56,658][20347] Updated weights for policy 0, policy_version 1797541 (0.0009) [2023-12-27 18:05:56,708][20347] Updated weights for policy 0, policy_version 1797551 (0.0006) [2023-12-27 18:05:56,759][20347] Updated weights for policy 0, policy_version 1797561 (0.0007) [2023-12-27 18:05:57,314][20381] Updated weights for policy 1, policy_version 1796026 (0.0007) [2023-12-27 18:05:57,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 920092672. Throughput: 0: 10217.5, 1: 10269.9. Samples: 920086368. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:05:57,363][20381] Updated weights for policy 1, policy_version 1796036 (0.0007) [2023-12-27 18:05:57,363][19170] Avg episode reward: [(0, '941.350'), (1, '990.290')] [2023-12-27 18:05:57,417][20381] Updated weights for policy 1, policy_version 1796046 (0.0007) [2023-12-27 18:05:57,432][20347] Updated weights for policy 0, policy_version 1797571 (0.0008) [2023-12-27 18:05:57,475][20381] Updated weights for policy 1, policy_version 1796056 (0.0005) [2023-12-27 18:05:57,488][20347] Updated weights for policy 0, policy_version 1797581 (0.0006) [2023-12-27 18:05:57,543][20347] Updated weights for policy 0, policy_version 1797591 (0.0008) [2023-12-27 18:05:58,179][20381] Updated weights for policy 1, policy_version 1796066 (0.0008) [2023-12-27 18:05:58,230][20347] Updated weights for policy 0, policy_version 1797601 (0.0007) [2023-12-27 18:05:58,245][20381] Updated weights for policy 1, policy_version 1796076 (0.0008) [2023-12-27 18:05:58,290][20347] Updated weights for policy 0, policy_version 1797611 (0.0008) [2023-12-27 18:05:58,305][20381] Updated weights for policy 1, policy_version 1796086 (0.0007) [2023-12-27 18:05:58,354][20347] Updated weights for policy 0, policy_version 1797621 (0.0008) [2023-12-27 18:05:58,415][20347] Updated weights for policy 0, policy_version 1797631 (0.0009) [2023-12-27 18:05:59,073][20381] Updated weights for policy 1, policy_version 1796096 (0.0006) [2023-12-27 18:05:59,131][20381] Updated weights for policy 1, policy_version 1796106 (0.0007) [2023-12-27 18:05:59,179][20381] Updated weights for policy 1, policy_version 1796116 (0.0007) [2023-12-27 18:05:59,210][20347] Updated weights for policy 0, policy_version 1797641 (0.0007) [2023-12-27 18:05:59,279][20347] Updated weights for policy 0, policy_version 1797651 (0.0008) [2023-12-27 18:05:59,339][20347] Updated weights for policy 0, policy_version 1797661 (0.0006) [2023-12-27 18:05:59,884][20381] Updated weights for policy 1, policy_version 1796126 (0.0007) [2023-12-27 18:05:59,935][20381] Updated weights for policy 1, policy_version 1796136 (0.0006) [2023-12-27 18:06:00,005][20381] Updated weights for policy 1, policy_version 1796146 (0.0008) [2023-12-27 18:06:00,073][20347] Updated weights for policy 0, policy_version 1797671 (0.0007) [2023-12-27 18:06:00,141][20347] Updated weights for policy 0, policy_version 1797681 (0.0007) [2023-12-27 18:06:00,196][20347] Updated weights for policy 0, policy_version 1797691 (0.0007) [2023-12-27 18:06:00,685][20381] Updated weights for policy 1, policy_version 1796156 (0.0007) [2023-12-27 18:06:00,737][20381] Updated weights for policy 1, policy_version 1796166 (0.0008) [2023-12-27 18:06:00,786][20381] Updated weights for policy 1, policy_version 1796176 (0.0008) [2023-12-27 18:06:00,867][20347] Updated weights for policy 0, policy_version 1797701 (0.0007) [2023-12-27 18:06:00,923][20347] Updated weights for policy 0, policy_version 1797711 (0.0005) [2023-12-27 18:06:00,986][20347] Updated weights for policy 0, policy_version 1797721 (0.0006) [2023-12-27 18:06:01,518][20381] Updated weights for policy 1, policy_version 1796186 (0.0007) [2023-12-27 18:06:01,581][20381] Updated weights for policy 1, policy_version 1796196 (0.0008) [2023-12-27 18:06:01,649][20381] Updated weights for policy 1, policy_version 1796206 (0.0008) [2023-12-27 18:06:01,685][20347] Updated weights for policy 0, policy_version 1797731 (0.0007) [2023-12-27 18:06:01,716][20381] Updated weights for policy 1, policy_version 1796216 (0.0007) [2023-12-27 18:06:01,754][20347] Updated weights for policy 0, policy_version 1797741 (0.0008) [2023-12-27 18:06:01,815][20347] Updated weights for policy 0, policy_version 1797751 (0.0006) [2023-12-27 18:06:02,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20382.8). Total num frames: 920190976. Throughput: 0: 10157.5, 1: 10234.6. Samples: 920204460. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:02,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 18:06:02,436][20381] Updated weights for policy 1, policy_version 1796226 (0.0008) [2023-12-27 18:06:02,476][20347] Updated weights for policy 0, policy_version 1797761 (0.0008) [2023-12-27 18:06:02,490][20381] Updated weights for policy 1, policy_version 1796236 (0.0008) [2023-12-27 18:06:02,531][20347] Updated weights for policy 0, policy_version 1797771 (0.0007) [2023-12-27 18:06:02,547][20381] Updated weights for policy 1, policy_version 1796246 (0.0007) [2023-12-27 18:06:02,588][20347] Updated weights for policy 0, policy_version 1797781 (0.0006) [2023-12-27 18:06:02,643][20347] Updated weights for policy 0, policy_version 1797791 (0.0005) [2023-12-27 18:06:03,194][20381] Updated weights for policy 1, policy_version 1796256 (0.0008) [2023-12-27 18:06:03,243][20381] Updated weights for policy 1, policy_version 1796266 (0.0008) [2023-12-27 18:06:03,289][20381] Updated weights for policy 1, policy_version 1796276 (0.0007) [2023-12-27 18:06:03,319][20347] Updated weights for policy 0, policy_version 1797801 (0.0005) [2023-12-27 18:06:03,371][20347] Updated weights for policy 0, policy_version 1797811 (0.0005) [2023-12-27 18:06:03,427][20347] Updated weights for policy 0, policy_version 1797821 (0.0006) [2023-12-27 18:06:04,026][20347] Updated weights for policy 0, policy_version 1797831 (0.0007) [2023-12-27 18:06:04,065][20381] Updated weights for policy 1, policy_version 1796286 (0.0006) [2023-12-27 18:06:04,086][20347] Updated weights for policy 0, policy_version 1797841 (0.0007) [2023-12-27 18:06:04,127][20381] Updated weights for policy 1, policy_version 1796296 (0.0006) [2023-12-27 18:06:04,151][20347] Updated weights for policy 0, policy_version 1797851 (0.0007) [2023-12-27 18:06:04,184][20381] Updated weights for policy 1, policy_version 1796306 (0.0006) [2023-12-27 18:06:04,800][20347] Updated weights for policy 0, policy_version 1797861 (0.0007) [2023-12-27 18:06:04,854][20347] Updated weights for policy 0, policy_version 1797871 (0.0007) [2023-12-27 18:06:04,903][20347] Updated weights for policy 0, policy_version 1797881 (0.0007) [2023-12-27 18:06:04,939][20381] Updated weights for policy 1, policy_version 1796316 (0.0008) [2023-12-27 18:06:04,994][20381] Updated weights for policy 1, policy_version 1796326 (0.0008) [2023-12-27 18:06:05,059][20381] Updated weights for policy 1, policy_version 1796336 (0.0008) [2023-12-27 18:06:05,598][20347] Updated weights for policy 0, policy_version 1797891 (0.0007) [2023-12-27 18:06:05,661][20347] Updated weights for policy 0, policy_version 1797901 (0.0007) [2023-12-27 18:06:05,721][20381] Updated weights for policy 1, policy_version 1796346 (0.0008) [2023-12-27 18:06:05,725][20347] Updated weights for policy 0, policy_version 1797911 (0.0007) [2023-12-27 18:06:05,771][20381] Updated weights for policy 1, policy_version 1796356 (0.0008) [2023-12-27 18:06:05,826][20381] Updated weights for policy 1, policy_version 1796366 (0.0005) [2023-12-27 18:06:05,884][20381] Updated weights for policy 1, policy_version 1796376 (0.0006) [2023-12-27 18:06:06,390][20347] Updated weights for policy 0, policy_version 1797921 (0.0006) [2023-12-27 18:06:06,453][20347] Updated weights for policy 0, policy_version 1797931 (0.0007) [2023-12-27 18:06:06,515][20347] Updated weights for policy 0, policy_version 1797941 (0.0010) [2023-12-27 18:06:06,567][20347] Updated weights for policy 0, policy_version 1797951 (0.0010) [2023-12-27 18:06:06,614][20381] Updated weights for policy 1, policy_version 1796386 (0.0008) [2023-12-27 18:06:06,670][20381] Updated weights for policy 1, policy_version 1796396 (0.0007) [2023-12-27 18:06:06,729][20381] Updated weights for policy 1, policy_version 1796406 (0.0008) [2023-12-27 18:06:07,201][20347] Updated weights for policy 0, policy_version 1797961 (0.0010) [2023-12-27 18:06:07,256][20347] Updated weights for policy 0, policy_version 1797971 (0.0010) [2023-12-27 18:06:07,312][20347] Updated weights for policy 0, policy_version 1797981 (0.0010) [2023-12-27 18:06:07,338][20381] Updated weights for policy 1, policy_version 1796416 (0.0010) [2023-12-27 18:06:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 920297472. Throughput: 0: 10159.0, 1: 10166.6. Samples: 920265380. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:07,363][19170] Avg episode reward: [(0, '970.740'), (1, '1000.000')] [2023-12-27 18:06:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001797984_460365824.pth... [2023-12-27 18:06:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001796800_460062720.pth [2023-12-27 18:06:07,398][20381] Updated weights for policy 1, policy_version 1796426 (0.0011) [2023-12-27 18:06:07,457][20381] Updated weights for policy 1, policy_version 1796436 (0.0011) [2023-12-27 18:06:07,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001796440_459939840.pth... [2023-12-27 18:06:07,483][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001795224_459628544.pth [2023-12-27 18:06:07,938][20347] Updated weights for policy 0, policy_version 1797991 (0.0008) [2023-12-27 18:06:07,995][20347] Updated weights for policy 0, policy_version 1798001 (0.0007) [2023-12-27 18:06:08,051][20347] Updated weights for policy 0, policy_version 1798011 (0.0010) [2023-12-27 18:06:08,077][20381] Updated weights for policy 1, policy_version 1796446 (0.0011) [2023-12-27 18:06:08,124][20381] Updated weights for policy 1, policy_version 1796456 (0.0009) [2023-12-27 18:06:08,182][20381] Updated weights for policy 1, policy_version 1796466 (0.0006) [2023-12-27 18:06:08,725][20347] Updated weights for policy 0, policy_version 1798021 (0.0008) [2023-12-27 18:06:08,787][20347] Updated weights for policy 0, policy_version 1798031 (0.0006) [2023-12-27 18:06:08,845][20347] Updated weights for policy 0, policy_version 1798041 (0.0007) [2023-12-27 18:06:08,928][20381] Updated weights for policy 1, policy_version 1796476 (0.0006) [2023-12-27 18:06:08,977][20381] Updated weights for policy 1, policy_version 1796486 (0.0008) [2023-12-27 18:06:09,028][20381] Updated weights for policy 1, policy_version 1796496 (0.0008) [2023-12-27 18:06:09,522][20347] Updated weights for policy 0, policy_version 1798051 (0.0006) [2023-12-27 18:06:09,590][20347] Updated weights for policy 0, policy_version 1798061 (0.0007) [2023-12-27 18:06:09,650][20347] Updated weights for policy 0, policy_version 1798071 (0.0008) [2023-12-27 18:06:09,785][20381] Updated weights for policy 1, policy_version 1796506 (0.0008) [2023-12-27 18:06:09,850][20381] Updated weights for policy 1, policy_version 1796516 (0.0008) [2023-12-27 18:06:09,911][20381] Updated weights for policy 1, policy_version 1796526 (0.0008) [2023-12-27 18:06:09,984][20381] Updated weights for policy 1, policy_version 1796536 (0.0008) [2023-12-27 18:06:10,371][20347] Updated weights for policy 0, policy_version 1798081 (0.0008) [2023-12-27 18:06:10,441][20347] Updated weights for policy 0, policy_version 1798091 (0.0010) [2023-12-27 18:06:10,491][20347] Updated weights for policy 0, policy_version 1798101 (0.0010) [2023-12-27 18:06:10,542][20347] Updated weights for policy 0, policy_version 1798111 (0.0008) [2023-12-27 18:06:10,571][20381] Updated weights for policy 1, policy_version 1796546 (0.0007) [2023-12-27 18:06:10,630][20381] Updated weights for policy 1, policy_version 1796556 (0.0006) [2023-12-27 18:06:10,680][20381] Updated weights for policy 1, policy_version 1796566 (0.0005) [2023-12-27 18:06:11,172][20347] Updated weights for policy 0, policy_version 1798121 (0.0008) [2023-12-27 18:06:11,239][20347] Updated weights for policy 0, policy_version 1798131 (0.0008) [2023-12-27 18:06:11,294][20347] Updated weights for policy 0, policy_version 1798141 (0.0008) [2023-12-27 18:06:11,369][20381] Updated weights for policy 1, policy_version 1796576 (0.0010) [2023-12-27 18:06:11,433][20381] Updated weights for policy 1, policy_version 1796586 (0.0011) [2023-12-27 18:06:11,484][20381] Updated weights for policy 1, policy_version 1796596 (0.0011) [2023-12-27 18:06:12,056][20347] Updated weights for policy 0, policy_version 1798151 (0.0007) [2023-12-27 18:06:12,108][20347] Updated weights for policy 0, policy_version 1798161 (0.0007) [2023-12-27 18:06:12,166][20347] Updated weights for policy 0, policy_version 1798171 (0.0007) [2023-12-27 18:06:12,206][20381] Updated weights for policy 1, policy_version 1796606 (0.0009) [2023-12-27 18:06:12,266][20381] Updated weights for policy 1, policy_version 1796616 (0.0009) [2023-12-27 18:06:12,323][20381] Updated weights for policy 1, policy_version 1796626 (0.0007) [2023-12-27 18:06:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 920403968. Throughput: 0: 10195.4, 1: 10240.7. Samples: 920389732. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:12,363][19170] Avg episode reward: [(0, '951.120'), (1, '1000.000')] [2023-12-27 18:06:12,903][20347] Updated weights for policy 0, policy_version 1798181 (0.0008) [2023-12-27 18:06:12,968][20347] Updated weights for policy 0, policy_version 1798191 (0.0008) [2023-12-27 18:06:13,011][20381] Updated weights for policy 1, policy_version 1796636 (0.0008) [2023-12-27 18:06:13,020][20347] Updated weights for policy 0, policy_version 1798201 (0.0008) [2023-12-27 18:06:13,071][20381] Updated weights for policy 1, policy_version 1796646 (0.0005) [2023-12-27 18:06:13,122][20381] Updated weights for policy 1, policy_version 1796656 (0.0005) [2023-12-27 18:06:13,613][20347] Updated weights for policy 0, policy_version 1798211 (0.0008) [2023-12-27 18:06:13,667][20347] Updated weights for policy 0, policy_version 1798221 (0.0010) [2023-12-27 18:06:13,725][20347] Updated weights for policy 0, policy_version 1798231 (0.0010) [2023-12-27 18:06:13,741][20381] Updated weights for policy 1, policy_version 1796666 (0.0008) [2023-12-27 18:06:13,799][20381] Updated weights for policy 1, policy_version 1796676 (0.0008) [2023-12-27 18:06:13,862][20381] Updated weights for policy 1, policy_version 1796686 (0.0008) [2023-12-27 18:06:13,922][20381] Updated weights for policy 1, policy_version 1796696 (0.0007) [2023-12-27 18:06:14,391][20347] Updated weights for policy 0, policy_version 1798241 (0.0010) [2023-12-27 18:06:14,436][20347] Updated weights for policy 0, policy_version 1798251 (0.0007) [2023-12-27 18:06:14,489][20347] Updated weights for policy 0, policy_version 1798261 (0.0008) [2023-12-27 18:06:14,542][20347] Updated weights for policy 0, policy_version 1798271 (0.0008) [2023-12-27 18:06:14,598][20381] Updated weights for policy 1, policy_version 1796706 (0.0007) [2023-12-27 18:06:14,650][20381] Updated weights for policy 1, policy_version 1796716 (0.0008) [2023-12-27 18:06:14,708][20381] Updated weights for policy 1, policy_version 1796726 (0.0008) [2023-12-27 18:06:15,285][20347] Updated weights for policy 0, policy_version 1798281 (0.0007) [2023-12-27 18:06:15,334][20347] Updated weights for policy 0, policy_version 1798291 (0.0007) [2023-12-27 18:06:15,395][20347] Updated weights for policy 0, policy_version 1798301 (0.0007) [2023-12-27 18:06:15,429][20381] Updated weights for policy 1, policy_version 1796736 (0.0007) [2023-12-27 18:06:15,488][20381] Updated weights for policy 1, policy_version 1796746 (0.0005) [2023-12-27 18:06:15,553][20381] Updated weights for policy 1, policy_version 1796756 (0.0005) [2023-12-27 18:06:16,109][20347] Updated weights for policy 0, policy_version 1798311 (0.0005) [2023-12-27 18:06:16,164][20381] Updated weights for policy 1, policy_version 1796766 (0.0006) [2023-12-27 18:06:16,166][20347] Updated weights for policy 0, policy_version 1798321 (0.0005) [2023-12-27 18:06:16,216][20347] Updated weights for policy 0, policy_version 1798331 (0.0006) [2023-12-27 18:06:16,220][20381] Updated weights for policy 1, policy_version 1796776 (0.0007) [2023-12-27 18:06:16,267][20381] Updated weights for policy 1, policy_version 1796786 (0.0007) [2023-12-27 18:06:16,844][20347] Updated weights for policy 0, policy_version 1798341 (0.0007) [2023-12-27 18:06:16,890][20347] Updated weights for policy 0, policy_version 1798351 (0.0005) [2023-12-27 18:06:16,894][20381] Updated weights for policy 1, policy_version 1796796 (0.0007) [2023-12-27 18:06:16,941][20347] Updated weights for policy 0, policy_version 1798361 (0.0005) [2023-12-27 18:06:16,954][20381] Updated weights for policy 1, policy_version 1796806 (0.0008) [2023-12-27 18:06:17,008][20381] Updated weights for policy 1, policy_version 1796816 (0.0007) [2023-12-27 18:06:17,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20479.9, 300 sec: 20438.3). Total num frames: 920502272. Throughput: 0: 10156.5, 1: 10258.9. Samples: 920512052. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:17,364][19170] Avg episode reward: [(0, '911.880'), (1, '1000.000')] [2023-12-27 18:06:17,542][20347] Updated weights for policy 0, policy_version 1798371 (0.0006) [2023-12-27 18:06:17,596][20347] Updated weights for policy 0, policy_version 1798381 (0.0007) [2023-12-27 18:06:17,630][20381] Updated weights for policy 1, policy_version 1796826 (0.0009) [2023-12-27 18:06:17,657][20347] Updated weights for policy 0, policy_version 1798391 (0.0007) [2023-12-27 18:06:17,686][20381] Updated weights for policy 1, policy_version 1796836 (0.0011) [2023-12-27 18:06:17,742][20381] Updated weights for policy 1, policy_version 1796846 (0.0011) [2023-12-27 18:06:17,803][20381] Updated weights for policy 1, policy_version 1796856 (0.0010) [2023-12-27 18:06:18,314][20347] Updated weights for policy 0, policy_version 1798401 (0.0007) [2023-12-27 18:06:18,375][20347] Updated weights for policy 0, policy_version 1798411 (0.0011) [2023-12-27 18:06:18,434][20347] Updated weights for policy 0, policy_version 1798421 (0.0011) [2023-12-27 18:06:18,493][20381] Updated weights for policy 1, policy_version 1796866 (0.0011) [2023-12-27 18:06:18,493][20347] Updated weights for policy 0, policy_version 1798431 (0.0011) [2023-12-27 18:06:18,545][20381] Updated weights for policy 1, policy_version 1796876 (0.0010) [2023-12-27 18:06:18,605][20381] Updated weights for policy 1, policy_version 1796886 (0.0010) [2023-12-27 18:06:19,143][20347] Updated weights for policy 0, policy_version 1798441 (0.0007) [2023-12-27 18:06:19,205][20347] Updated weights for policy 0, policy_version 1798451 (0.0009) [2023-12-27 18:06:19,268][20347] Updated weights for policy 0, policy_version 1798461 (0.0008) [2023-12-27 18:06:19,361][20381] Updated weights for policy 1, policy_version 1796896 (0.0011) [2023-12-27 18:06:19,419][20381] Updated weights for policy 1, policy_version 1796906 (0.0010) [2023-12-27 18:06:19,468][20381] Updated weights for policy 1, policy_version 1796916 (0.0011) [2023-12-27 18:06:19,957][20347] Updated weights for policy 0, policy_version 1798471 (0.0008) [2023-12-27 18:06:20,007][20347] Updated weights for policy 0, policy_version 1798481 (0.0008) [2023-12-27 18:06:20,064][20347] Updated weights for policy 0, policy_version 1798491 (0.0008) [2023-12-27 18:06:20,163][20381] Updated weights for policy 1, policy_version 1796926 (0.0008) [2023-12-27 18:06:20,227][20381] Updated weights for policy 1, policy_version 1796936 (0.0006) [2023-12-27 18:06:20,284][20381] Updated weights for policy 1, policy_version 1796946 (0.0005) [2023-12-27 18:06:20,838][20347] Updated weights for policy 0, policy_version 1798501 (0.0008) [2023-12-27 18:06:20,895][20381] Updated weights for policy 1, policy_version 1796956 (0.0006) [2023-12-27 18:06:20,897][20347] Updated weights for policy 0, policy_version 1798511 (0.0007) [2023-12-27 18:06:20,954][20347] Updated weights for policy 0, policy_version 1798521 (0.0006) [2023-12-27 18:06:20,956][20381] Updated weights for policy 1, policy_version 1796966 (0.0007) [2023-12-27 18:06:21,025][20381] Updated weights for policy 1, policy_version 1796976 (0.0008) [2023-12-27 18:06:21,765][20347] Updated weights for policy 0, policy_version 1798531 (0.0006) [2023-12-27 18:06:21,790][20381] Updated weights for policy 1, policy_version 1796986 (0.0008) [2023-12-27 18:06:21,828][20347] Updated weights for policy 0, policy_version 1798541 (0.0006) [2023-12-27 18:06:21,849][20381] Updated weights for policy 1, policy_version 1796996 (0.0007) [2023-12-27 18:06:21,890][20347] Updated weights for policy 0, policy_version 1798551 (0.0006) [2023-12-27 18:06:21,911][20381] Updated weights for policy 1, policy_version 1797006 (0.0006) [2023-12-27 18:06:21,978][20381] Updated weights for policy 1, policy_version 1797016 (0.0007) [2023-12-27 18:06:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 920600576. Throughput: 0: 10173.9, 1: 10226.7. Samples: 920573380. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:22,363][19170] Avg episode reward: [(0, '921.820'), (1, '1000.000')] [2023-12-27 18:06:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001797016_460087296.pth... [2023-12-27 18:06:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001798560_460513280.pth... [2023-12-27 18:06:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001797376_460210176.pth [2023-12-27 18:06:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001795832_459784192.pth [2023-12-27 18:06:22,572][20347] Updated weights for policy 0, policy_version 1798561 (0.0006) [2023-12-27 18:06:22,631][20347] Updated weights for policy 0, policy_version 1798571 (0.0007) [2023-12-27 18:06:22,658][20381] Updated weights for policy 1, policy_version 1797026 (0.0007) [2023-12-27 18:06:22,695][20347] Updated weights for policy 0, policy_version 1798581 (0.0006) [2023-12-27 18:06:22,722][20381] Updated weights for policy 1, policy_version 1797036 (0.0008) [2023-12-27 18:06:22,758][20347] Updated weights for policy 0, policy_version 1798591 (0.0006) [2023-12-27 18:06:22,778][20381] Updated weights for policy 1, policy_version 1797046 (0.0007) [2023-12-27 18:06:23,326][20381] Updated weights for policy 1, policy_version 1797056 (0.0005) [2023-12-27 18:06:23,377][20381] Updated weights for policy 1, policy_version 1797066 (0.0005) [2023-12-27 18:06:23,430][20381] Updated weights for policy 1, policy_version 1797076 (0.0005) [2023-12-27 18:06:23,480][20347] Updated weights for policy 0, policy_version 1798601 (0.0006) [2023-12-27 18:06:23,532][20347] Updated weights for policy 0, policy_version 1798611 (0.0008) [2023-12-27 18:06:23,575][20347] Updated weights for policy 0, policy_version 1798621 (0.0008) [2023-12-27 18:06:23,963][20381] Updated weights for policy 1, policy_version 1797086 (0.0008) [2023-12-27 18:06:24,009][20381] Updated weights for policy 1, policy_version 1797096 (0.0010) [2023-12-27 18:06:24,064][20381] Updated weights for policy 1, policy_version 1797106 (0.0010) [2023-12-27 18:06:24,259][20347] Updated weights for policy 0, policy_version 1798631 (0.0009) [2023-12-27 18:06:24,316][20347] Updated weights for policy 0, policy_version 1798641 (0.0010) [2023-12-27 18:06:24,372][20347] Updated weights for policy 0, policy_version 1798651 (0.0010) [2023-12-27 18:06:24,721][20381] Updated weights for policy 1, policy_version 1797116 (0.0010) [2023-12-27 18:06:24,769][20381] Updated weights for policy 1, policy_version 1797126 (0.0010) [2023-12-27 18:06:24,811][20381] Updated weights for policy 1, policy_version 1797136 (0.0010) [2023-12-27 18:06:24,997][20347] Updated weights for policy 0, policy_version 1798661 (0.0009) [2023-12-27 18:06:25,058][20347] Updated weights for policy 0, policy_version 1798671 (0.0010) [2023-12-27 18:06:25,106][20347] Updated weights for policy 0, policy_version 1798681 (0.0010) [2023-12-27 18:06:25,538][20381] Updated weights for policy 1, policy_version 1797146 (0.0010) [2023-12-27 18:06:25,593][20381] Updated weights for policy 1, policy_version 1797156 (0.0006) [2023-12-27 18:06:25,645][20381] Updated weights for policy 1, policy_version 1797166 (0.0005) [2023-12-27 18:06:25,708][20381] Updated weights for policy 1, policy_version 1797176 (0.0005) [2023-12-27 18:06:25,778][20347] Updated weights for policy 0, policy_version 1798691 (0.0010) [2023-12-27 18:06:25,836][20347] Updated weights for policy 0, policy_version 1798701 (0.0010) [2023-12-27 18:06:25,891][20347] Updated weights for policy 0, policy_version 1798711 (0.0010) [2023-12-27 18:06:26,268][20381] Updated weights for policy 1, policy_version 1797186 (0.0007) [2023-12-27 18:06:26,318][20381] Updated weights for policy 1, policy_version 1797196 (0.0007) [2023-12-27 18:06:26,378][20381] Updated weights for policy 1, policy_version 1797206 (0.0007) [2023-12-27 18:06:26,567][20347] Updated weights for policy 0, policy_version 1798721 (0.0010) [2023-12-27 18:06:26,620][20347] Updated weights for policy 0, policy_version 1798731 (0.0010) [2023-12-27 18:06:26,679][20347] Updated weights for policy 0, policy_version 1798741 (0.0010) [2023-12-27 18:06:26,739][20347] Updated weights for policy 0, policy_version 1798751 (0.0010) [2023-12-27 18:06:27,025][20381] Updated weights for policy 1, policy_version 1797216 (0.0007) [2023-12-27 18:06:27,080][20381] Updated weights for policy 1, policy_version 1797226 (0.0008) [2023-12-27 18:06:27,141][20381] Updated weights for policy 1, policy_version 1797236 (0.0007) [2023-12-27 18:06:27,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 920707072. Throughput: 0: 10181.5, 1: 10305.9. Samples: 920697476. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:27,363][19170] Avg episode reward: [(0, '960.960'), (1, '1000.000')] [2023-12-27 18:06:27,404][20347] Updated weights for policy 0, policy_version 1798761 (0.0010) [2023-12-27 18:06:27,454][20347] Updated weights for policy 0, policy_version 1798771 (0.0010) [2023-12-27 18:06:27,504][20347] Updated weights for policy 0, policy_version 1798781 (0.0010) [2023-12-27 18:06:27,747][20381] Updated weights for policy 1, policy_version 1797246 (0.0007) [2023-12-27 18:06:27,808][20381] Updated weights for policy 1, policy_version 1797256 (0.0007) [2023-12-27 18:06:27,865][20381] Updated weights for policy 1, policy_version 1797266 (0.0007) [2023-12-27 18:06:28,181][20347] Updated weights for policy 0, policy_version 1798791 (0.0010) [2023-12-27 18:06:28,240][20347] Updated weights for policy 0, policy_version 1798801 (0.0009) [2023-12-27 18:06:28,296][20347] Updated weights for policy 0, policy_version 1798811 (0.0009) [2023-12-27 18:06:28,542][20381] Updated weights for policy 1, policy_version 1797276 (0.0008) [2023-12-27 18:06:28,603][20381] Updated weights for policy 1, policy_version 1797286 (0.0010) [2023-12-27 18:06:28,667][20381] Updated weights for policy 1, policy_version 1797296 (0.0011) [2023-12-27 18:06:28,963][20347] Updated weights for policy 0, policy_version 1798821 (0.0010) [2023-12-27 18:06:29,007][20347] Updated weights for policy 0, policy_version 1798831 (0.0010) [2023-12-27 18:06:29,053][20347] Updated weights for policy 0, policy_version 1798841 (0.0010) [2023-12-27 18:06:29,336][20381] Updated weights for policy 1, policy_version 1797306 (0.0010) [2023-12-27 18:06:29,391][20381] Updated weights for policy 1, policy_version 1797316 (0.0008) [2023-12-27 18:06:29,437][20381] Updated weights for policy 1, policy_version 1797326 (0.0009) [2023-12-27 18:06:29,484][20381] Updated weights for policy 1, policy_version 1797336 (0.0010) [2023-12-27 18:06:29,772][20347] Updated weights for policy 0, policy_version 1798851 (0.0010) [2023-12-27 18:06:29,833][20347] Updated weights for policy 0, policy_version 1798861 (0.0011) [2023-12-27 18:06:29,891][20347] Updated weights for policy 0, policy_version 1798871 (0.0007) [2023-12-27 18:06:30,194][20381] Updated weights for policy 1, policy_version 1797346 (0.0007) [2023-12-27 18:06:30,250][20381] Updated weights for policy 1, policy_version 1797356 (0.0008) [2023-12-27 18:06:30,307][20381] Updated weights for policy 1, policy_version 1797366 (0.0007) [2023-12-27 18:06:30,603][20347] Updated weights for policy 0, policy_version 1798881 (0.0009) [2023-12-27 18:06:30,650][20347] Updated weights for policy 0, policy_version 1798891 (0.0010) [2023-12-27 18:06:30,695][20347] Updated weights for policy 0, policy_version 1798901 (0.0006) [2023-12-27 18:06:30,761][20347] Updated weights for policy 0, policy_version 1798911 (0.0005) [2023-12-27 18:06:30,915][20381] Updated weights for policy 1, policy_version 1797376 (0.0010) [2023-12-27 18:06:30,964][20381] Updated weights for policy 1, policy_version 1797386 (0.0010) [2023-12-27 18:06:31,015][20381] Updated weights for policy 1, policy_version 1797396 (0.0010) [2023-12-27 18:06:31,380][20347] Updated weights for policy 0, policy_version 1798921 (0.0008) [2023-12-27 18:06:31,428][20347] Updated weights for policy 0, policy_version 1798931 (0.0007) [2023-12-27 18:06:31,478][20347] Updated weights for policy 0, policy_version 1798941 (0.0005) [2023-12-27 18:06:31,816][20381] Updated weights for policy 1, policy_version 1797406 (0.0008) [2023-12-27 18:06:31,878][20381] Updated weights for policy 1, policy_version 1797416 (0.0005) [2023-12-27 18:06:31,939][20381] Updated weights for policy 1, policy_version 1797426 (0.0008) [2023-12-27 18:06:32,108][20347] Updated weights for policy 0, policy_version 1798951 (0.0007) [2023-12-27 18:06:32,166][20347] Updated weights for policy 0, policy_version 1798961 (0.0007) [2023-12-27 18:06:32,229][20347] Updated weights for policy 0, policy_version 1798971 (0.0009) [2023-12-27 18:06:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 920813568. Throughput: 0: 10182.9, 1: 10274.2. Samples: 920821688. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:32,363][19170] Avg episode reward: [(0, '970.630'), (1, '1000.000')] [2023-12-27 18:06:32,658][20381] Updated weights for policy 1, policy_version 1797436 (0.0007) [2023-12-27 18:06:32,724][20381] Updated weights for policy 1, policy_version 1797446 (0.0005) [2023-12-27 18:06:32,780][20381] Updated weights for policy 1, policy_version 1797456 (0.0005) [2023-12-27 18:06:32,827][20347] Updated weights for policy 0, policy_version 1798981 (0.0008) [2023-12-27 18:06:32,878][20347] Updated weights for policy 0, policy_version 1798991 (0.0008) [2023-12-27 18:06:32,930][20347] Updated weights for policy 0, policy_version 1799001 (0.0007) [2023-12-27 18:06:33,377][20381] Updated weights for policy 1, policy_version 1797466 (0.0006) [2023-12-27 18:06:33,438][20381] Updated weights for policy 1, policy_version 1797476 (0.0011) [2023-12-27 18:06:33,500][20381] Updated weights for policy 1, policy_version 1797486 (0.0011) [2023-12-27 18:06:33,555][20347] Updated weights for policy 0, policy_version 1799011 (0.0007) [2023-12-27 18:06:33,556][20381] Updated weights for policy 1, policy_version 1797496 (0.0010) [2023-12-27 18:06:33,613][20347] Updated weights for policy 0, policy_version 1799021 (0.0006) [2023-12-27 18:06:33,670][20347] Updated weights for policy 0, policy_version 1799031 (0.0005) [2023-12-27 18:06:34,194][20381] Updated weights for policy 1, policy_version 1797508 (0.0009) [2023-12-27 18:06:34,250][20381] Updated weights for policy 1, policy_version 1797518 (0.0007) [2023-12-27 18:06:34,307][20381] Updated weights for policy 1, policy_version 1797528 (0.0007) [2023-12-27 18:06:34,333][20347] Updated weights for policy 0, policy_version 1799041 (0.0007) [2023-12-27 18:06:34,395][20347] Updated weights for policy 0, policy_version 1799051 (0.0007) [2023-12-27 18:06:34,449][20347] Updated weights for policy 0, policy_version 1799061 (0.0007) [2023-12-27 18:06:34,502][20347] Updated weights for policy 0, policy_version 1799071 (0.0008) [2023-12-27 18:06:35,006][20381] Updated weights for policy 1, policy_version 1797538 (0.0008) [2023-12-27 18:06:35,065][20381] Updated weights for policy 1, policy_version 1797548 (0.0007) [2023-12-27 18:06:35,126][20381] Updated weights for policy 1, policy_version 1797558 (0.0007) [2023-12-27 18:06:35,199][20347] Updated weights for policy 0, policy_version 1799081 (0.0006) [2023-12-27 18:06:35,255][20347] Updated weights for policy 0, policy_version 1799091 (0.0006) [2023-12-27 18:06:35,302][20347] Updated weights for policy 0, policy_version 1799101 (0.0005) [2023-12-27 18:06:35,756][20381] Updated weights for policy 1, policy_version 1797568 (0.0007) [2023-12-27 18:06:35,804][20381] Updated weights for policy 1, policy_version 1797578 (0.0010) [2023-12-27 18:06:35,856][20381] Updated weights for policy 1, policy_version 1797588 (0.0011) [2023-12-27 18:06:35,915][20347] Updated weights for policy 0, policy_version 1799111 (0.0007) [2023-12-27 18:06:35,980][20347] Updated weights for policy 0, policy_version 1799121 (0.0007) [2023-12-27 18:06:36,038][20347] Updated weights for policy 0, policy_version 1799131 (0.0006) [2023-12-27 18:06:36,556][20381] Updated weights for policy 1, policy_version 1797598 (0.0011) [2023-12-27 18:06:36,615][20381] Updated weights for policy 1, policy_version 1797608 (0.0009) [2023-12-27 18:06:36,660][20347] Updated weights for policy 0, policy_version 1799141 (0.0008) [2023-12-27 18:06:36,679][20381] Updated weights for policy 1, policy_version 1797618 (0.0010) [2023-12-27 18:06:36,718][20347] Updated weights for policy 0, policy_version 1799151 (0.0008) [2023-12-27 18:06:36,767][20347] Updated weights for policy 0, policy_version 1799161 (0.0010) [2023-12-27 18:06:37,311][20381] Updated weights for policy 1, policy_version 1797628 (0.0009) [2023-12-27 18:06:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 920911872. Throughput: 0: 10232.2, 1: 10270.2. Samples: 920884788. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:37,363][19170] Avg episode reward: [(0, '990.190'), (1, '1000.000')] [2023-12-27 18:06:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001799168_460668928.pth... [2023-12-27 18:06:37,372][20381] Updated weights for policy 1, policy_version 1797638 (0.0009) [2023-12-27 18:06:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001797984_460365824.pth [2023-12-27 18:06:37,424][20347] Updated weights for policy 0, policy_version 1799171 (0.0008) [2023-12-27 18:06:37,429][20381] Updated weights for policy 1, policy_version 1797648 (0.0008) [2023-12-27 18:06:37,471][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001797656_460251136.pth... [2023-12-27 18:06:37,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001796440_459939840.pth [2023-12-27 18:06:37,487][20347] Updated weights for policy 0, policy_version 1799181 (0.0009) [2023-12-27 18:06:37,542][20347] Updated weights for policy 0, policy_version 1799191 (0.0008) [2023-12-27 18:06:38,022][20381] Updated weights for policy 1, policy_version 1797658 (0.0007) [2023-12-27 18:06:38,078][20381] Updated weights for policy 1, policy_version 1797668 (0.0011) [2023-12-27 18:06:38,116][20347] Updated weights for policy 0, policy_version 1799201 (0.0010) [2023-12-27 18:06:38,140][20381] Updated weights for policy 1, policy_version 1797678 (0.0010) [2023-12-27 18:06:38,170][20347] Updated weights for policy 0, policy_version 1799211 (0.0006) [2023-12-27 18:06:38,196][20381] Updated weights for policy 1, policy_version 1797688 (0.0011) [2023-12-27 18:06:38,216][20347] Updated weights for policy 0, policy_version 1799221 (0.0007) [2023-12-27 18:06:38,264][20347] Updated weights for policy 0, policy_version 1799231 (0.0005) [2023-12-27 18:06:38,866][20347] Updated weights for policy 0, policy_version 1799241 (0.0006) [2023-12-27 18:06:38,866][20381] Updated weights for policy 1, policy_version 1797698 (0.0006) [2023-12-27 18:06:38,922][20347] Updated weights for policy 0, policy_version 1799251 (0.0006) [2023-12-27 18:06:38,923][20381] Updated weights for policy 1, policy_version 1797708 (0.0006) [2023-12-27 18:06:38,987][20381] Updated weights for policy 1, policy_version 1797718 (0.0006) [2023-12-27 18:06:38,988][20347] Updated weights for policy 0, policy_version 1799261 (0.0008) [2023-12-27 18:06:39,627][20381] Updated weights for policy 1, policy_version 1797728 (0.0007) [2023-12-27 18:06:39,660][20347] Updated weights for policy 0, policy_version 1799271 (0.0006) [2023-12-27 18:06:39,689][20381] Updated weights for policy 1, policy_version 1797738 (0.0007) [2023-12-27 18:06:39,719][20347] Updated weights for policy 0, policy_version 1799281 (0.0007) [2023-12-27 18:06:39,753][20381] Updated weights for policy 1, policy_version 1797748 (0.0006) [2023-12-27 18:06:39,778][20347] Updated weights for policy 0, policy_version 1799291 (0.0007) [2023-12-27 18:06:40,460][20381] Updated weights for policy 1, policy_version 1797758 (0.0007) [2023-12-27 18:06:40,495][20347] Updated weights for policy 0, policy_version 1799301 (0.0007) [2023-12-27 18:06:40,526][20381] Updated weights for policy 1, policy_version 1797768 (0.0008) [2023-12-27 18:06:40,550][20347] Updated weights for policy 0, policy_version 1799311 (0.0005) [2023-12-27 18:06:40,593][20381] Updated weights for policy 1, policy_version 1797778 (0.0008) [2023-12-27 18:06:40,602][20347] Updated weights for policy 0, policy_version 1799321 (0.0006) [2023-12-27 18:06:41,207][20347] Updated weights for policy 0, policy_version 1799331 (0.0006) [2023-12-27 18:06:41,273][20347] Updated weights for policy 0, policy_version 1799341 (0.0008) [2023-12-27 18:06:41,305][20381] Updated weights for policy 1, policy_version 1797788 (0.0007) [2023-12-27 18:06:41,336][20347] Updated weights for policy 0, policy_version 1799351 (0.0008) [2023-12-27 18:06:41,398][20381] Updated weights for policy 1, policy_version 1797798 (0.0008) [2023-12-27 18:06:41,454][20381] Updated weights for policy 1, policy_version 1797808 (0.0007) [2023-12-27 18:06:42,086][20347] Updated weights for policy 0, policy_version 1799361 (0.0008) [2023-12-27 18:06:42,125][20381] Updated weights for policy 1, policy_version 1797818 (0.0007) [2023-12-27 18:06:42,142][20347] Updated weights for policy 0, policy_version 1799371 (0.0006) [2023-12-27 18:06:42,184][20381] Updated weights for policy 1, policy_version 1797828 (0.0005) [2023-12-27 18:06:42,201][20347] Updated weights for policy 0, policy_version 1799381 (0.0009) [2023-12-27 18:06:42,241][20381] Updated weights for policy 1, policy_version 1797838 (0.0006) [2023-12-27 18:06:42,270][20347] Updated weights for policy 0, policy_version 1799391 (0.0007) [2023-12-27 18:06:42,302][20381] Updated weights for policy 1, policy_version 1797848 (0.0007) [2023-12-27 18:06:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 921026560. Throughput: 0: 10311.5, 1: 10261.8. Samples: 921012168. Policy #0 lag: (min: 31.0, avg: 42.2, max: 63.0) [2023-12-27 18:06:42,363][19170] Avg episode reward: [(0, '990.210'), (1, '1000.000')] [2023-12-27 18:06:42,959][20347] Updated weights for policy 0, policy_version 1799401 (0.0007) [2023-12-27 18:06:43,004][20381] Updated weights for policy 1, policy_version 1797858 (0.0007) [2023-12-27 18:06:43,014][20347] Updated weights for policy 0, policy_version 1799411 (0.0008) [2023-12-27 18:06:43,059][20381] Updated weights for policy 1, policy_version 1797868 (0.0006) [2023-12-27 18:06:43,064][20347] Updated weights for policy 0, policy_version 1799421 (0.0006) [2023-12-27 18:06:43,109][20381] Updated weights for policy 1, policy_version 1797878 (0.0005) [2023-12-27 18:06:43,628][20347] Updated weights for policy 0, policy_version 1799431 (0.0005) [2023-12-27 18:06:43,689][20347] Updated weights for policy 0, policy_version 1799441 (0.0005) [2023-12-27 18:06:43,724][20381] Updated weights for policy 1, policy_version 1797888 (0.0005) [2023-12-27 18:06:43,750][20347] Updated weights for policy 0, policy_version 1799451 (0.0007) [2023-12-27 18:06:43,775][20381] Updated weights for policy 1, policy_version 1797898 (0.0006) [2023-12-27 18:06:43,825][20381] Updated weights for policy 1, policy_version 1797908 (0.0006) [2023-12-27 18:06:44,303][20347] Updated weights for policy 0, policy_version 1799461 (0.0007) [2023-12-27 18:06:44,369][20347] Updated weights for policy 0, policy_version 1799471 (0.0006) [2023-12-27 18:06:44,406][20381] Updated weights for policy 1, policy_version 1797918 (0.0007) [2023-12-27 18:06:44,422][20347] Updated weights for policy 0, policy_version 1799481 (0.0007) [2023-12-27 18:06:44,460][20381] Updated weights for policy 1, policy_version 1797928 (0.0009) [2023-12-27 18:06:44,523][20381] Updated weights for policy 1, policy_version 1797938 (0.0010) [2023-12-27 18:06:45,071][20347] Updated weights for policy 0, policy_version 1799491 (0.0007) [2023-12-27 18:06:45,125][20347] Updated weights for policy 0, policy_version 1799501 (0.0009) [2023-12-27 18:06:45,174][20347] Updated weights for policy 0, policy_version 1799511 (0.0009) [2023-12-27 18:06:45,244][20381] Updated weights for policy 1, policy_version 1797948 (0.0010) [2023-12-27 18:06:45,307][20381] Updated weights for policy 1, policy_version 1797958 (0.0009) [2023-12-27 18:06:45,367][20381] Updated weights for policy 1, policy_version 1797968 (0.0007) [2023-12-27 18:06:45,981][20381] Updated weights for policy 1, policy_version 1797978 (0.0008) [2023-12-27 18:06:45,990][20347] Updated weights for policy 0, policy_version 1799521 (0.0008) [2023-12-27 18:06:46,038][20381] Updated weights for policy 1, policy_version 1797988 (0.0010) [2023-12-27 18:06:46,047][20347] Updated weights for policy 0, policy_version 1799531 (0.0007) [2023-12-27 18:06:46,083][20381] Updated weights for policy 1, policy_version 1797998 (0.0010) [2023-12-27 18:06:46,098][20347] Updated weights for policy 0, policy_version 1799541 (0.0008) [2023-12-27 18:06:46,134][20381] Updated weights for policy 1, policy_version 1798008 (0.0010) [2023-12-27 18:06:46,154][20347] Updated weights for policy 0, policy_version 1799551 (0.0008) [2023-12-27 18:06:46,769][20381] Updated weights for policy 1, policy_version 1798018 (0.0005) [2023-12-27 18:06:46,828][20381] Updated weights for policy 1, policy_version 1798028 (0.0005) [2023-12-27 18:06:46,837][20347] Updated weights for policy 0, policy_version 1799561 (0.0008) [2023-12-27 18:06:46,883][20381] Updated weights for policy 1, policy_version 1798038 (0.0006) [2023-12-27 18:06:46,892][20347] Updated weights for policy 0, policy_version 1799571 (0.0006) [2023-12-27 18:06:46,939][20347] Updated weights for policy 0, policy_version 1799581 (0.0007) [2023-12-27 18:06:47,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 921124864. Throughput: 0: 10363.2, 1: 10375.7. Samples: 921137708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:06:47,363][19170] Avg episode reward: [(0, '961.000'), (1, '1000.000')] [2023-12-27 18:06:47,444][20381] Updated weights for policy 1, policy_version 1798048 (0.0010) [2023-12-27 18:06:47,491][20381] Updated weights for policy 1, policy_version 1798058 (0.0009) [2023-12-27 18:06:47,537][20381] Updated weights for policy 1, policy_version 1798068 (0.0009) [2023-12-27 18:06:47,667][20347] Updated weights for policy 0, policy_version 1799591 (0.0007) [2023-12-27 18:06:47,727][20347] Updated weights for policy 0, policy_version 1799601 (0.0008) [2023-12-27 18:06:47,778][20347] Updated weights for policy 0, policy_version 1799611 (0.0007) [2023-12-27 18:06:48,211][20381] Updated weights for policy 1, policy_version 1798078 (0.0008) [2023-12-27 18:06:48,264][20381] Updated weights for policy 1, policy_version 1798088 (0.0008) [2023-12-27 18:06:48,322][20381] Updated weights for policy 1, policy_version 1798098 (0.0008) [2023-12-27 18:06:48,482][20347] Updated weights for policy 0, policy_version 1799621 (0.0009) [2023-12-27 18:06:48,544][20347] Updated weights for policy 0, policy_version 1799631 (0.0010) [2023-12-27 18:06:48,604][20347] Updated weights for policy 0, policy_version 1799641 (0.0010) [2023-12-27 18:06:49,032][20381] Updated weights for policy 1, policy_version 1798108 (0.0009) [2023-12-27 18:06:49,078][20381] Updated weights for policy 1, policy_version 1798118 (0.0010) [2023-12-27 18:06:49,125][20381] Updated weights for policy 1, policy_version 1798128 (0.0010) [2023-12-27 18:06:49,312][20347] Updated weights for policy 0, policy_version 1799651 (0.0009) [2023-12-27 18:06:49,375][20347] Updated weights for policy 0, policy_version 1799661 (0.0008) [2023-12-27 18:06:49,432][20347] Updated weights for policy 0, policy_version 1799671 (0.0006) [2023-12-27 18:06:49,804][20381] Updated weights for policy 1, policy_version 1798138 (0.0010) [2023-12-27 18:06:49,869][20381] Updated weights for policy 1, policy_version 1798148 (0.0008) [2023-12-27 18:06:49,928][20381] Updated weights for policy 1, policy_version 1798158 (0.0010) [2023-12-27 18:06:50,016][20381] Updated weights for policy 1, policy_version 1798168 (0.0007) [2023-12-27 18:06:50,074][20347] Updated weights for policy 0, policy_version 1799681 (0.0007) [2023-12-27 18:06:50,131][20347] Updated weights for policy 0, policy_version 1799691 (0.0006) [2023-12-27 18:06:50,192][20347] Updated weights for policy 0, policy_version 1799701 (0.0005) [2023-12-27 18:06:50,248][20347] Updated weights for policy 0, policy_version 1799711 (0.0006) [2023-12-27 18:06:50,687][20381] Updated weights for policy 1, policy_version 1798178 (0.0007) [2023-12-27 18:06:50,739][20381] Updated weights for policy 1, policy_version 1798188 (0.0008) [2023-12-27 18:06:50,798][20381] Updated weights for policy 1, policy_version 1798198 (0.0008) [2023-12-27 18:06:50,860][20347] Updated weights for policy 0, policy_version 1799721 (0.0006) [2023-12-27 18:06:50,908][20347] Updated weights for policy 0, policy_version 1799731 (0.0008) [2023-12-27 18:06:50,956][20347] Updated weights for policy 0, policy_version 1799741 (0.0007) [2023-12-27 18:06:51,545][20381] Updated weights for policy 1, policy_version 1798208 (0.0010) [2023-12-27 18:06:51,604][20381] Updated weights for policy 1, policy_version 1798218 (0.0010) [2023-12-27 18:06:51,671][20381] Updated weights for policy 1, policy_version 1798228 (0.0011) [2023-12-27 18:06:51,688][20347] Updated weights for policy 0, policy_version 1799751 (0.0008) [2023-12-27 18:06:51,754][20347] Updated weights for policy 0, policy_version 1799761 (0.0009) [2023-12-27 18:06:51,813][20347] Updated weights for policy 0, policy_version 1799771 (0.0008) [2023-12-27 18:06:52,362][20381] Updated weights for policy 1, policy_version 1798238 (0.0011) [2023-12-27 18:06:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 921223168. Throughput: 0: 10324.9, 1: 10409.0. Samples: 921198408. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:06:52,363][19170] Avg episode reward: [(0, '941.510'), (1, '1000.000')] [2023-12-27 18:06:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001799776_460824576.pth... [2023-12-27 18:06:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001798560_460513280.pth [2023-12-27 18:06:52,421][20381] Updated weights for policy 1, policy_version 1798248 (0.0011) [2023-12-27 18:06:52,487][20381] Updated weights for policy 1, policy_version 1798258 (0.0011) [2023-12-27 18:06:52,516][20347] Updated weights for policy 0, policy_version 1799781 (0.0007) [2023-12-27 18:06:52,526][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001798264_460406784.pth... [2023-12-27 18:06:52,531][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001797016_460087296.pth [2023-12-27 18:06:52,575][20347] Updated weights for policy 0, policy_version 1799791 (0.0006) [2023-12-27 18:06:52,637][20347] Updated weights for policy 0, policy_version 1799801 (0.0007) [2023-12-27 18:06:53,109][20381] Updated weights for policy 1, policy_version 1798268 (0.0011) [2023-12-27 18:06:53,164][20381] Updated weights for policy 1, policy_version 1798278 (0.0010) [2023-12-27 18:06:53,223][20381] Updated weights for policy 1, policy_version 1798288 (0.0010) [2023-12-27 18:06:53,238][20347] Updated weights for policy 0, policy_version 1799811 (0.0007) [2023-12-27 18:06:53,292][20347] Updated weights for policy 0, policy_version 1799821 (0.0005) [2023-12-27 18:06:53,340][20347] Updated weights for policy 0, policy_version 1799831 (0.0005) [2023-12-27 18:06:53,926][20381] Updated weights for policy 1, policy_version 1798298 (0.0009) [2023-12-27 18:06:53,929][20347] Updated weights for policy 0, policy_version 1799841 (0.0009) [2023-12-27 18:06:53,987][20347] Updated weights for policy 0, policy_version 1799851 (0.0006) [2023-12-27 18:06:53,999][20381] Updated weights for policy 1, policy_version 1798308 (0.0007) [2023-12-27 18:06:54,047][20347] Updated weights for policy 0, policy_version 1799861 (0.0006) [2023-12-27 18:06:54,065][20381] Updated weights for policy 1, policy_version 1798318 (0.0008) [2023-12-27 18:06:54,100][20347] Updated weights for policy 0, policy_version 1799871 (0.0008) [2023-12-27 18:06:54,127][20381] Updated weights for policy 1, policy_version 1798328 (0.0007) [2023-12-27 18:06:54,723][20347] Updated weights for policy 0, policy_version 1799881 (0.0006) [2023-12-27 18:06:54,762][20381] Updated weights for policy 1, policy_version 1798338 (0.0011) [2023-12-27 18:06:54,778][20347] Updated weights for policy 0, policy_version 1799891 (0.0005) [2023-12-27 18:06:54,812][20381] Updated weights for policy 1, policy_version 1798348 (0.0011) [2023-12-27 18:06:54,829][20347] Updated weights for policy 0, policy_version 1799901 (0.0005) [2023-12-27 18:06:54,861][20381] Updated weights for policy 1, policy_version 1798358 (0.0010) [2023-12-27 18:06:55,463][20381] Updated weights for policy 1, policy_version 1798368 (0.0010) [2023-12-27 18:06:55,507][20381] Updated weights for policy 1, policy_version 1798378 (0.0010) [2023-12-27 18:06:55,548][20347] Updated weights for policy 0, policy_version 1799911 (0.0008) [2023-12-27 18:06:55,553][20381] Updated weights for policy 1, policy_version 1798388 (0.0010) [2023-12-27 18:06:55,602][20347] Updated weights for policy 0, policy_version 1799921 (0.0007) [2023-12-27 18:06:55,659][20347] Updated weights for policy 0, policy_version 1799931 (0.0007) [2023-12-27 18:06:56,237][20381] Updated weights for policy 1, policy_version 1798398 (0.0010) [2023-12-27 18:06:56,289][20381] Updated weights for policy 1, policy_version 1798408 (0.0010) [2023-12-27 18:06:56,334][20381] Updated weights for policy 1, policy_version 1798418 (0.0009) [2023-12-27 18:06:56,343][20347] Updated weights for policy 0, policy_version 1799941 (0.0006) [2023-12-27 18:06:56,395][20347] Updated weights for policy 0, policy_version 1799951 (0.0006) [2023-12-27 18:06:56,451][20347] Updated weights for policy 0, policy_version 1799961 (0.0005) [2023-12-27 18:06:56,923][20381] Updated weights for policy 1, policy_version 1798428 (0.0006) [2023-12-27 18:06:56,971][20381] Updated weights for policy 1, policy_version 1798438 (0.0005) [2023-12-27 18:06:57,019][20381] Updated weights for policy 1, policy_version 1798448 (0.0006) [2023-12-27 18:06:57,032][20347] Updated weights for policy 0, policy_version 1799971 (0.0005) [2023-12-27 18:06:57,087][20347] Updated weights for policy 0, policy_version 1799981 (0.0005) [2023-12-27 18:06:57,135][20347] Updated weights for policy 0, policy_version 1799991 (0.0007) [2023-12-27 18:06:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 921337856. Throughput: 0: 10339.8, 1: 10395.0. Samples: 921322800. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:06:57,363][19170] Avg episode reward: [(0, '951.200'), (1, '1000.000')] [2023-12-27 18:06:57,661][20381] Updated weights for policy 1, policy_version 1798458 (0.0007) [2023-12-27 18:06:57,715][20381] Updated weights for policy 1, policy_version 1798468 (0.0008) [2023-12-27 18:06:57,725][20347] Updated weights for policy 0, policy_version 1800001 (0.0009) [2023-12-27 18:06:57,770][20381] Updated weights for policy 1, policy_version 1798478 (0.0010) [2023-12-27 18:06:57,782][20347] Updated weights for policy 0, policy_version 1800011 (0.0005) [2023-12-27 18:06:57,824][20381] Updated weights for policy 1, policy_version 1798488 (0.0010) [2023-12-27 18:06:57,835][20347] Updated weights for policy 0, policy_version 1800021 (0.0005) [2023-12-27 18:06:57,898][20347] Updated weights for policy 0, policy_version 1800031 (0.0005) [2023-12-27 18:06:58,467][20347] Updated weights for policy 0, policy_version 1800041 (0.0008) [2023-12-27 18:06:58,533][20347] Updated weights for policy 0, policy_version 1800051 (0.0008) [2023-12-27 18:06:58,558][20381] Updated weights for policy 1, policy_version 1798498 (0.0009) [2023-12-27 18:06:58,595][20347] Updated weights for policy 0, policy_version 1800061 (0.0008) [2023-12-27 18:06:58,619][20381] Updated weights for policy 1, policy_version 1798508 (0.0009) [2023-12-27 18:06:58,685][20381] Updated weights for policy 1, policy_version 1798518 (0.0007) [2023-12-27 18:06:59,403][20347] Updated weights for policy 0, policy_version 1800072 (0.0007) [2023-12-27 18:06:59,466][20381] Updated weights for policy 1, policy_version 1798528 (0.0008) [2023-12-27 18:06:59,468][20347] Updated weights for policy 0, policy_version 1800082 (0.0006) [2023-12-27 18:06:59,517][20381] Updated weights for policy 1, policy_version 1798538 (0.0007) [2023-12-27 18:06:59,531][20347] Updated weights for policy 0, policy_version 1800092 (0.0006) [2023-12-27 18:06:59,578][20381] Updated weights for policy 1, policy_version 1798548 (0.0007) [2023-12-27 18:07:00,145][20347] Updated weights for policy 0, policy_version 1800102 (0.0008) [2023-12-27 18:07:00,203][20347] Updated weights for policy 0, policy_version 1800112 (0.0010) [2023-12-27 18:07:00,238][20381] Updated weights for policy 1, policy_version 1798558 (0.0007) [2023-12-27 18:07:00,259][20347] Updated weights for policy 0, policy_version 1800122 (0.0010) [2023-12-27 18:07:00,289][20381] Updated weights for policy 1, policy_version 1798568 (0.0008) [2023-12-27 18:07:00,335][20381] Updated weights for policy 1, policy_version 1798578 (0.0008) [2023-12-27 18:07:00,877][20347] Updated weights for policy 0, policy_version 1800132 (0.0008) [2023-12-27 18:07:00,934][20347] Updated weights for policy 0, policy_version 1800142 (0.0005) [2023-12-27 18:07:00,998][20347] Updated weights for policy 0, policy_version 1800152 (0.0006) [2023-12-27 18:07:01,026][20381] Updated weights for policy 1, policy_version 1798588 (0.0007) [2023-12-27 18:07:01,090][20381] Updated weights for policy 1, policy_version 1798598 (0.0008) [2023-12-27 18:07:01,149][20381] Updated weights for policy 1, policy_version 1798608 (0.0007) [2023-12-27 18:07:01,691][20347] Updated weights for policy 0, policy_version 1800162 (0.0009) [2023-12-27 18:07:01,751][20347] Updated weights for policy 0, policy_version 1800172 (0.0009) [2023-12-27 18:07:01,802][20347] Updated weights for policy 0, policy_version 1800182 (0.0008) [2023-12-27 18:07:01,858][20347] Updated weights for policy 0, policy_version 1800192 (0.0008) [2023-12-27 18:07:01,877][20381] Updated weights for policy 1, policy_version 1798618 (0.0007) [2023-12-27 18:07:01,937][20381] Updated weights for policy 1, policy_version 1798628 (0.0008) [2023-12-27 18:07:01,991][20381] Updated weights for policy 1, policy_version 1798638 (0.0006) [2023-12-27 18:07:02,052][20381] Updated weights for policy 1, policy_version 1798648 (0.0005) [2023-12-27 18:07:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 921436160. Throughput: 0: 10395.8, 1: 10375.3. Samples: 921446744. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:02,363][19170] Avg episode reward: [(0, '941.340'), (1, '1000.000')] [2023-12-27 18:07:02,588][20347] Updated weights for policy 0, policy_version 1800202 (0.0008) [2023-12-27 18:07:02,632][20347] Updated weights for policy 0, policy_version 1800212 (0.0008) [2023-12-27 18:07:02,680][20381] Updated weights for policy 1, policy_version 1798658 (0.0006) [2023-12-27 18:07:02,689][20347] Updated weights for policy 0, policy_version 1800222 (0.0005) [2023-12-27 18:07:02,734][20381] Updated weights for policy 1, policy_version 1798668 (0.0007) [2023-12-27 18:07:02,794][20381] Updated weights for policy 1, policy_version 1798678 (0.0008) [2023-12-27 18:07:03,357][20347] Updated weights for policy 0, policy_version 1800232 (0.0007) [2023-12-27 18:07:03,419][20347] Updated weights for policy 0, policy_version 1800242 (0.0007) [2023-12-27 18:07:03,463][20381] Updated weights for policy 1, policy_version 1798688 (0.0006) [2023-12-27 18:07:03,477][20347] Updated weights for policy 0, policy_version 1800252 (0.0007) [2023-12-27 18:07:03,518][20381] Updated weights for policy 1, policy_version 1798698 (0.0005) [2023-12-27 18:07:03,574][20381] Updated weights for policy 1, policy_version 1798708 (0.0005) [2023-12-27 18:07:04,155][20381] Updated weights for policy 1, policy_version 1798718 (0.0007) [2023-12-27 18:07:04,196][20347] Updated weights for policy 0, policy_version 1800262 (0.0006) [2023-12-27 18:07:04,210][20381] Updated weights for policy 1, policy_version 1798728 (0.0006) [2023-12-27 18:07:04,253][20347] Updated weights for policy 0, policy_version 1800272 (0.0008) [2023-12-27 18:07:04,272][20381] Updated weights for policy 1, policy_version 1798738 (0.0006) [2023-12-27 18:07:04,311][20347] Updated weights for policy 0, policy_version 1800282 (0.0008) [2023-12-27 18:07:04,889][20381] Updated weights for policy 1, policy_version 1798748 (0.0006) [2023-12-27 18:07:04,946][20381] Updated weights for policy 1, policy_version 1798758 (0.0007) [2023-12-27 18:07:04,994][20381] Updated weights for policy 1, policy_version 1798768 (0.0005) [2023-12-27 18:07:05,017][20347] Updated weights for policy 0, policy_version 1800292 (0.0007) [2023-12-27 18:07:05,068][20347] Updated weights for policy 0, policy_version 1800302 (0.0007) [2023-12-27 18:07:05,119][20347] Updated weights for policy 0, policy_version 1800312 (0.0007) [2023-12-27 18:07:05,676][20381] Updated weights for policy 1, policy_version 1798778 (0.0006) [2023-12-27 18:07:05,724][20381] Updated weights for policy 1, policy_version 1798788 (0.0007) [2023-12-27 18:07:05,773][20381] Updated weights for policy 1, policy_version 1798798 (0.0007) [2023-12-27 18:07:05,813][20347] Updated weights for policy 0, policy_version 1800322 (0.0008) [2023-12-27 18:07:05,823][20381] Updated weights for policy 1, policy_version 1798808 (0.0008) [2023-12-27 18:07:05,866][20347] Updated weights for policy 0, policy_version 1800332 (0.0007) [2023-12-27 18:07:05,918][20347] Updated weights for policy 0, policy_version 1800342 (0.0008) [2023-12-27 18:07:05,971][20347] Updated weights for policy 0, policy_version 1800352 (0.0006) [2023-12-27 18:07:06,527][20381] Updated weights for policy 1, policy_version 1798818 (0.0011) [2023-12-27 18:07:06,580][20381] Updated weights for policy 1, policy_version 1798828 (0.0010) [2023-12-27 18:07:06,638][20381] Updated weights for policy 1, policy_version 1798838 (0.0011) [2023-12-27 18:07:06,706][20347] Updated weights for policy 0, policy_version 1800362 (0.0008) [2023-12-27 18:07:06,760][20347] Updated weights for policy 0, policy_version 1800372 (0.0008) [2023-12-27 18:07:06,820][20347] Updated weights for policy 0, policy_version 1800382 (0.0008) [2023-12-27 18:07:07,268][20381] Updated weights for policy 1, policy_version 1798848 (0.0006) [2023-12-27 18:07:07,329][20381] Updated weights for policy 1, policy_version 1798858 (0.0006) [2023-12-27 18:07:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 921534464. Throughput: 0: 10361.9, 1: 10438.2. Samples: 921509380. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:07,363][19170] Avg episode reward: [(0, '951.160'), (1, '1000.000')] [2023-12-27 18:07:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001800384_460980224.pth... [2023-12-27 18:07:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001799168_460668928.pth [2023-12-27 18:07:07,388][20381] Updated weights for policy 1, policy_version 1798868 (0.0006) [2023-12-27 18:07:07,406][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001798872_460562432.pth... [2023-12-27 18:07:07,409][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001797656_460251136.pth [2023-12-27 18:07:07,586][20347] Updated weights for policy 0, policy_version 1800392 (0.0009) [2023-12-27 18:07:07,631][20347] Updated weights for policy 0, policy_version 1800402 (0.0009) [2023-12-27 18:07:07,677][20347] Updated weights for policy 0, policy_version 1800412 (0.0007) [2023-12-27 18:07:07,948][20381] Updated weights for policy 1, policy_version 1798878 (0.0005) [2023-12-27 18:07:08,003][20381] Updated weights for policy 1, policy_version 1798888 (0.0005) [2023-12-27 18:07:08,051][20381] Updated weights for policy 1, policy_version 1798898 (0.0008) [2023-12-27 18:07:08,397][20347] Updated weights for policy 0, policy_version 1800422 (0.0008) [2023-12-27 18:07:08,450][20347] Updated weights for policy 0, policy_version 1800432 (0.0009) [2023-12-27 18:07:08,496][20347] Updated weights for policy 0, policy_version 1800442 (0.0009) [2023-12-27 18:07:08,722][20381] Updated weights for policy 1, policy_version 1798908 (0.0009) [2023-12-27 18:07:08,772][20381] Updated weights for policy 1, policy_version 1798918 (0.0011) [2023-12-27 18:07:08,828][20381] Updated weights for policy 1, policy_version 1798928 (0.0011) [2023-12-27 18:07:09,190][20347] Updated weights for policy 0, policy_version 1800452 (0.0007) [2023-12-27 18:07:09,252][20347] Updated weights for policy 0, policy_version 1800462 (0.0010) [2023-12-27 18:07:09,314][20347] Updated weights for policy 0, policy_version 1800472 (0.0008) [2023-12-27 18:07:09,432][20381] Updated weights for policy 1, policy_version 1798938 (0.0007) [2023-12-27 18:07:09,498][20381] Updated weights for policy 1, policy_version 1798948 (0.0008) [2023-12-27 18:07:09,559][20381] Updated weights for policy 1, policy_version 1798958 (0.0009) [2023-12-27 18:07:09,615][20381] Updated weights for policy 1, policy_version 1798968 (0.0011) [2023-12-27 18:07:10,090][20347] Updated weights for policy 0, policy_version 1800482 (0.0010) [2023-12-27 18:07:10,152][20347] Updated weights for policy 0, policy_version 1800492 (0.0007) [2023-12-27 18:07:10,214][20347] Updated weights for policy 0, policy_version 1800502 (0.0011) [2023-12-27 18:07:10,268][20381] Updated weights for policy 1, policy_version 1798978 (0.0011) [2023-12-27 18:07:10,270][20347] Updated weights for policy 0, policy_version 1800512 (0.0010) [2023-12-27 18:07:10,327][20381] Updated weights for policy 1, policy_version 1798988 (0.0011) [2023-12-27 18:07:10,387][20381] Updated weights for policy 1, policy_version 1798998 (0.0011) [2023-12-27 18:07:10,959][20347] Updated weights for policy 0, policy_version 1800522 (0.0010) [2023-12-27 18:07:11,018][20347] Updated weights for policy 0, policy_version 1800532 (0.0011) [2023-12-27 18:07:11,056][20381] Updated weights for policy 1, policy_version 1799008 (0.0009) [2023-12-27 18:07:11,084][20347] Updated weights for policy 0, policy_version 1800542 (0.0009) [2023-12-27 18:07:11,113][20381] Updated weights for policy 1, policy_version 1799018 (0.0008) [2023-12-27 18:07:11,173][20381] Updated weights for policy 1, policy_version 1799028 (0.0007) [2023-12-27 18:07:11,867][20347] Updated weights for policy 0, policy_version 1800552 (0.0007) [2023-12-27 18:07:11,905][20381] Updated weights for policy 1, policy_version 1799038 (0.0007) [2023-12-27 18:07:11,935][20347] Updated weights for policy 0, policy_version 1800562 (0.0007) [2023-12-27 18:07:11,967][20381] Updated weights for policy 1, policy_version 1799048 (0.0007) [2023-12-27 18:07:11,999][20347] Updated weights for policy 0, policy_version 1800572 (0.0008) [2023-12-27 18:07:12,031][20381] Updated weights for policy 1, policy_version 1799058 (0.0007) [2023-12-27 18:07:12,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 921640960. Throughput: 0: 10344.1, 1: 10404.4. Samples: 921631164. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:12,363][19170] Avg episode reward: [(0, '960.940'), (1, '1000.000')] [2023-12-27 18:07:12,681][20347] Updated weights for policy 0, policy_version 1800582 (0.0007) [2023-12-27 18:07:12,718][20381] Updated weights for policy 1, policy_version 1799068 (0.0008) [2023-12-27 18:07:12,742][20347] Updated weights for policy 0, policy_version 1800592 (0.0007) [2023-12-27 18:07:12,778][20381] Updated weights for policy 1, policy_version 1799078 (0.0008) [2023-12-27 18:07:12,805][20347] Updated weights for policy 0, policy_version 1800602 (0.0005) [2023-12-27 18:07:12,842][20381] Updated weights for policy 1, policy_version 1799088 (0.0008) [2023-12-27 18:07:13,442][20347] Updated weights for policy 0, policy_version 1800612 (0.0006) [2023-12-27 18:07:13,472][20381] Updated weights for policy 1, policy_version 1799098 (0.0009) [2023-12-27 18:07:13,497][20347] Updated weights for policy 0, policy_version 1800622 (0.0006) [2023-12-27 18:07:13,528][20381] Updated weights for policy 1, policy_version 1799108 (0.0008) [2023-12-27 18:07:13,550][20347] Updated weights for policy 0, policy_version 1800632 (0.0005) [2023-12-27 18:07:13,572][20381] Updated weights for policy 1, policy_version 1799118 (0.0008) [2023-12-27 18:07:13,626][20381] Updated weights for policy 1, policy_version 1799128 (0.0007) [2023-12-27 18:07:14,161][20347] Updated weights for policy 0, policy_version 1800642 (0.0006) [2023-12-27 18:07:14,216][20347] Updated weights for policy 0, policy_version 1800652 (0.0008) [2023-12-27 18:07:14,271][20347] Updated weights for policy 0, policy_version 1800662 (0.0007) [2023-12-27 18:07:14,324][20347] Updated weights for policy 0, policy_version 1800672 (0.0005) [2023-12-27 18:07:14,350][20381] Updated weights for policy 1, policy_version 1799138 (0.0005) [2023-12-27 18:07:14,404][20381] Updated weights for policy 1, policy_version 1799148 (0.0005) [2023-12-27 18:07:14,478][20381] Updated weights for policy 1, policy_version 1799158 (0.0005) [2023-12-27 18:07:14,959][20347] Updated weights for policy 0, policy_version 1800682 (0.0007) [2023-12-27 18:07:15,023][20347] Updated weights for policy 0, policy_version 1800692 (0.0008) [2023-12-27 18:07:15,088][20347] Updated weights for policy 0, policy_version 1800702 (0.0008) [2023-12-27 18:07:15,105][20381] Updated weights for policy 1, policy_version 1799168 (0.0008) [2023-12-27 18:07:15,166][20381] Updated weights for policy 1, policy_version 1799178 (0.0007) [2023-12-27 18:07:15,229][20381] Updated weights for policy 1, policy_version 1799188 (0.0008) [2023-12-27 18:07:15,799][20347] Updated weights for policy 0, policy_version 1800712 (0.0008) [2023-12-27 18:07:15,855][20347] Updated weights for policy 0, policy_version 1800722 (0.0008) [2023-12-27 18:07:15,897][20381] Updated weights for policy 1, policy_version 1799198 (0.0008) [2023-12-27 18:07:15,914][20347] Updated weights for policy 0, policy_version 1800732 (0.0006) [2023-12-27 18:07:15,958][20381] Updated weights for policy 1, policy_version 1799208 (0.0006) [2023-12-27 18:07:16,014][20381] Updated weights for policy 1, policy_version 1799218 (0.0007) [2023-12-27 18:07:16,606][20347] Updated weights for policy 0, policy_version 1800742 (0.0007) [2023-12-27 18:07:16,665][20347] Updated weights for policy 0, policy_version 1800752 (0.0007) [2023-12-27 18:07:16,684][20381] Updated weights for policy 1, policy_version 1799228 (0.0007) [2023-12-27 18:07:16,720][20347] Updated weights for policy 0, policy_version 1800762 (0.0006) [2023-12-27 18:07:16,740][20381] Updated weights for policy 1, policy_version 1799238 (0.0006) [2023-12-27 18:07:16,797][20381] Updated weights for policy 1, policy_version 1799248 (0.0007) [2023-12-27 18:07:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.7, 300 sec: 20466.1). Total num frames: 921739264. Throughput: 0: 10293.8, 1: 10398.1. Samples: 921752820. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:17,363][19170] Avg episode reward: [(0, '951.070'), (1, '1000.000')] [2023-12-27 18:07:17,408][20381] Updated weights for policy 1, policy_version 1799258 (0.0007) [2023-12-27 18:07:17,447][20347] Updated weights for policy 0, policy_version 1800772 (0.0006) [2023-12-27 18:07:17,464][20381] Updated weights for policy 1, policy_version 1799268 (0.0006) [2023-12-27 18:07:17,501][20347] Updated weights for policy 0, policy_version 1800782 (0.0006) [2023-12-27 18:07:17,523][20381] Updated weights for policy 1, policy_version 1799278 (0.0007) [2023-12-27 18:07:17,555][20347] Updated weights for policy 0, policy_version 1800792 (0.0006) [2023-12-27 18:07:17,582][20381] Updated weights for policy 1, policy_version 1799288 (0.0007) [2023-12-27 18:07:18,189][20347] Updated weights for policy 0, policy_version 1800802 (0.0007) [2023-12-27 18:07:18,219][20381] Updated weights for policy 1, policy_version 1799298 (0.0005) [2023-12-27 18:07:18,246][20347] Updated weights for policy 0, policy_version 1800812 (0.0006) [2023-12-27 18:07:18,276][20381] Updated weights for policy 1, policy_version 1799308 (0.0007) [2023-12-27 18:07:18,300][20347] Updated weights for policy 0, policy_version 1800822 (0.0005) [2023-12-27 18:07:18,325][20381] Updated weights for policy 1, policy_version 1799318 (0.0008) [2023-12-27 18:07:18,370][20347] Updated weights for policy 0, policy_version 1800832 (0.0007) [2023-12-27 18:07:18,964][20347] Updated weights for policy 0, policy_version 1800842 (0.0010) [2023-12-27 18:07:19,016][20347] Updated weights for policy 0, policy_version 1800852 (0.0010) [2023-12-27 18:07:19,022][20381] Updated weights for policy 1, policy_version 1799328 (0.0010) [2023-12-27 18:07:19,067][20381] Updated weights for policy 1, policy_version 1799338 (0.0010) [2023-12-27 18:07:19,067][20347] Updated weights for policy 0, policy_version 1800862 (0.0010) [2023-12-27 18:07:19,123][20381] Updated weights for policy 1, policy_version 1799348 (0.0010) [2023-12-27 18:07:19,786][20347] Updated weights for policy 0, policy_version 1800872 (0.0011) [2023-12-27 18:07:19,848][20347] Updated weights for policy 0, policy_version 1800882 (0.0010) [2023-12-27 18:07:19,893][20381] Updated weights for policy 1, policy_version 1799358 (0.0011) [2023-12-27 18:07:19,908][20347] Updated weights for policy 0, policy_version 1800892 (0.0010) [2023-12-27 18:07:19,960][20381] Updated weights for policy 1, policy_version 1799368 (0.0011) [2023-12-27 18:07:20,023][20381] Updated weights for policy 1, policy_version 1799378 (0.0011) [2023-12-27 18:07:20,627][20347] Updated weights for policy 0, policy_version 1800902 (0.0010) [2023-12-27 18:07:20,689][20347] Updated weights for policy 0, policy_version 1800912 (0.0010) [2023-12-27 18:07:20,697][20381] Updated weights for policy 1, policy_version 1799388 (0.0010) [2023-12-27 18:07:20,745][20347] Updated weights for policy 0, policy_version 1800922 (0.0010) [2023-12-27 18:07:20,750][20381] Updated weights for policy 1, policy_version 1799398 (0.0011) [2023-12-27 18:07:20,809][20381] Updated weights for policy 1, policy_version 1799408 (0.0010) [2023-12-27 18:07:21,461][20347] Updated weights for policy 0, policy_version 1800932 (0.0009) [2023-12-27 18:07:21,523][20347] Updated weights for policy 0, policy_version 1800942 (0.0007) [2023-12-27 18:07:21,525][20381] Updated weights for policy 1, policy_version 1799418 (0.0009) [2023-12-27 18:07:21,580][20347] Updated weights for policy 0, policy_version 1800952 (0.0008) [2023-12-27 18:07:21,584][20381] Updated weights for policy 1, policy_version 1799428 (0.0011) [2023-12-27 18:07:21,643][20381] Updated weights for policy 1, policy_version 1799438 (0.0009) [2023-12-27 18:07:21,696][20381] Updated weights for policy 1, policy_version 1799448 (0.0011) [2023-12-27 18:07:22,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 921837568. Throughput: 0: 10275.0, 1: 10370.1. Samples: 921813820. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:22,363][19170] Avg episode reward: [(0, '951.230'), (1, '1000.000')] [2023-12-27 18:07:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001799448_460709888.pth... [2023-12-27 18:07:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001800960_461127680.pth... [2023-12-27 18:07:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001798264_460406784.pth [2023-12-27 18:07:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001799776_460824576.pth [2023-12-27 18:07:22,453][20347] Updated weights for policy 0, policy_version 1800962 (0.0009) [2023-12-27 18:07:22,505][20347] Updated weights for policy 0, policy_version 1800972 (0.0007) [2023-12-27 18:07:22,564][20347] Updated weights for policy 0, policy_version 1800982 (0.0006) [2023-12-27 18:07:22,566][20381] Updated weights for policy 1, policy_version 1799458 (0.0008) [2023-12-27 18:07:22,628][20347] Updated weights for policy 0, policy_version 1800992 (0.0006) [2023-12-27 18:07:22,628][20381] Updated weights for policy 1, policy_version 1799468 (0.0007) [2023-12-27 18:07:22,686][20381] Updated weights for policy 1, policy_version 1799478 (0.0007) [2023-12-27 18:07:23,204][20347] Updated weights for policy 0, policy_version 1801002 (0.0006) [2023-12-27 18:07:23,252][20347] Updated weights for policy 0, policy_version 1801012 (0.0007) [2023-12-27 18:07:23,303][20347] Updated weights for policy 0, policy_version 1801022 (0.0007) [2023-12-27 18:07:23,444][20381] Updated weights for policy 1, policy_version 1799488 (0.0007) [2023-12-27 18:07:23,490][20381] Updated weights for policy 1, policy_version 1799498 (0.0010) [2023-12-27 18:07:23,543][20381] Updated weights for policy 1, policy_version 1799508 (0.0008) [2023-12-27 18:07:23,884][20347] Updated weights for policy 0, policy_version 1801032 (0.0006) [2023-12-27 18:07:23,941][20347] Updated weights for policy 0, policy_version 1801042 (0.0007) [2023-12-27 18:07:23,999][20347] Updated weights for policy 0, policy_version 1801052 (0.0007) [2023-12-27 18:07:24,221][20381] Updated weights for policy 1, policy_version 1799518 (0.0010) [2023-12-27 18:07:24,267][20381] Updated weights for policy 1, policy_version 1799528 (0.0010) [2023-12-27 18:07:24,315][20381] Updated weights for policy 1, policy_version 1799538 (0.0010) [2023-12-27 18:07:24,699][20347] Updated weights for policy 0, policy_version 1801062 (0.0009) [2023-12-27 18:07:24,757][20347] Updated weights for policy 0, policy_version 1801072 (0.0007) [2023-12-27 18:07:24,804][20347] Updated weights for policy 0, policy_version 1801082 (0.0007) [2023-12-27 18:07:24,937][20381] Updated weights for policy 1, policy_version 1799548 (0.0010) [2023-12-27 18:07:24,994][20381] Updated weights for policy 1, policy_version 1799558 (0.0010) [2023-12-27 18:07:25,051][20381] Updated weights for policy 1, policy_version 1799568 (0.0010) [2023-12-27 18:07:25,502][20347] Updated weights for policy 0, policy_version 1801092 (0.0007) [2023-12-27 18:07:25,548][20347] Updated weights for policy 0, policy_version 1801102 (0.0007) [2023-12-27 18:07:25,596][20347] Updated weights for policy 0, policy_version 1801112 (0.0007) [2023-12-27 18:07:25,731][20381] Updated weights for policy 1, policy_version 1799578 (0.0009) [2023-12-27 18:07:25,790][20381] Updated weights for policy 1, policy_version 1799588 (0.0005) [2023-12-27 18:07:25,852][20381] Updated weights for policy 1, policy_version 1799598 (0.0006) [2023-12-27 18:07:25,906][20381] Updated weights for policy 1, policy_version 1799608 (0.0007) [2023-12-27 18:07:26,283][20347] Updated weights for policy 0, policy_version 1801122 (0.0007) [2023-12-27 18:07:26,339][20347] Updated weights for policy 0, policy_version 1801132 (0.0007) [2023-12-27 18:07:26,395][20347] Updated weights for policy 0, policy_version 1801142 (0.0008) [2023-12-27 18:07:26,452][20347] Updated weights for policy 0, policy_version 1801152 (0.0008) [2023-12-27 18:07:26,474][20381] Updated weights for policy 1, policy_version 1799618 (0.0005) [2023-12-27 18:07:26,521][20381] Updated weights for policy 1, policy_version 1799628 (0.0008) [2023-12-27 18:07:26,566][20381] Updated weights for policy 1, policy_version 1799638 (0.0009) [2023-12-27 18:07:26,985][20347] Updated weights for policy 0, policy_version 1801162 (0.0005) [2023-12-27 18:07:27,031][20347] Updated weights for policy 0, policy_version 1801172 (0.0005) [2023-12-27 18:07:27,079][20347] Updated weights for policy 0, policy_version 1801182 (0.0005) [2023-12-27 18:07:27,293][20381] Updated weights for policy 1, policy_version 1799648 (0.0009) [2023-12-27 18:07:27,355][20381] Updated weights for policy 1, policy_version 1799658 (0.0007) [2023-12-27 18:07:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 921944064. Throughput: 0: 10175.7, 1: 10327.8. Samples: 921934824. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:27,363][19170] Avg episode reward: [(0, '970.830'), (1, '1000.000')] [2023-12-27 18:07:27,424][20381] Updated weights for policy 1, policy_version 1799668 (0.0005) [2023-12-27 18:07:27,621][20347] Updated weights for policy 0, policy_version 1801192 (0.0007) [2023-12-27 18:07:27,678][20347] Updated weights for policy 0, policy_version 1801202 (0.0008) [2023-12-27 18:07:27,734][20347] Updated weights for policy 0, policy_version 1801212 (0.0010) [2023-12-27 18:07:28,001][20381] Updated weights for policy 1, policy_version 1799678 (0.0007) [2023-12-27 18:07:28,056][20381] Updated weights for policy 1, policy_version 1799688 (0.0006) [2023-12-27 18:07:28,120][20381] Updated weights for policy 1, policy_version 1799698 (0.0006) [2023-12-27 18:07:28,393][20347] Updated weights for policy 0, policy_version 1801222 (0.0009) [2023-12-27 18:07:28,455][20347] Updated weights for policy 0, policy_version 1801232 (0.0008) [2023-12-27 18:07:28,520][20347] Updated weights for policy 0, policy_version 1801242 (0.0009) [2023-12-27 18:07:28,827][20381] Updated weights for policy 1, policy_version 1799708 (0.0008) [2023-12-27 18:07:28,875][20381] Updated weights for policy 1, policy_version 1799718 (0.0008) [2023-12-27 18:07:28,928][20381] Updated weights for policy 1, policy_version 1799728 (0.0006) [2023-12-27 18:07:29,118][20347] Updated weights for policy 0, policy_version 1801252 (0.0009) [2023-12-27 18:07:29,176][20347] Updated weights for policy 0, policy_version 1801262 (0.0005) [2023-12-27 18:07:29,236][20347] Updated weights for policy 0, policy_version 1801272 (0.0008) [2023-12-27 18:07:29,558][20381] Updated weights for policy 1, policy_version 1799738 (0.0005) [2023-12-27 18:07:29,619][20381] Updated weights for policy 1, policy_version 1799748 (0.0007) [2023-12-27 18:07:29,678][20381] Updated weights for policy 1, policy_version 1799758 (0.0011) [2023-12-27 18:07:29,737][20381] Updated weights for policy 1, policy_version 1799768 (0.0011) [2023-12-27 18:07:29,808][20347] Updated weights for policy 0, policy_version 1801282 (0.0006) [2023-12-27 18:07:29,872][20347] Updated weights for policy 0, policy_version 1801292 (0.0008) [2023-12-27 18:07:29,938][20347] Updated weights for policy 0, policy_version 1801302 (0.0007) [2023-12-27 18:07:29,998][20347] Updated weights for policy 0, policy_version 1801312 (0.0006) [2023-12-27 18:07:30,415][20381] Updated weights for policy 1, policy_version 1799778 (0.0007) [2023-12-27 18:07:30,463][20381] Updated weights for policy 1, policy_version 1799788 (0.0007) [2023-12-27 18:07:30,516][20381] Updated weights for policy 1, policy_version 1799798 (0.0007) [2023-12-27 18:07:30,625][20347] Updated weights for policy 0, policy_version 1801322 (0.0009) [2023-12-27 18:07:30,672][20347] Updated weights for policy 0, policy_version 1801332 (0.0006) [2023-12-27 18:07:30,735][20347] Updated weights for policy 0, policy_version 1801342 (0.0007) [2023-12-27 18:07:31,161][20381] Updated weights for policy 1, policy_version 1799808 (0.0007) [2023-12-27 18:07:31,215][20381] Updated weights for policy 1, policy_version 1799818 (0.0005) [2023-12-27 18:07:31,277][20381] Updated weights for policy 1, policy_version 1799828 (0.0009) [2023-12-27 18:07:31,475][20347] Updated weights for policy 0, policy_version 1801352 (0.0006) [2023-12-27 18:07:31,525][20347] Updated weights for policy 0, policy_version 1801362 (0.0006) [2023-12-27 18:07:31,583][20347] Updated weights for policy 0, policy_version 1801372 (0.0007) [2023-12-27 18:07:31,973][20381] Updated weights for policy 1, policy_version 1799838 (0.0008) [2023-12-27 18:07:32,034][20381] Updated weights for policy 1, policy_version 1799848 (0.0006) [2023-12-27 18:07:32,089][20381] Updated weights for policy 1, policy_version 1799858 (0.0006) [2023-12-27 18:07:32,308][20347] Updated weights for policy 0, policy_version 1801382 (0.0007) [2023-12-27 18:07:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 922050560. Throughput: 0: 10269.9, 1: 10295.1. Samples: 922063132. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:32,363][19170] Avg episode reward: [(0, '951.170'), (1, '1000.000')] [2023-12-27 18:07:32,371][20347] Updated weights for policy 0, policy_version 1801392 (0.0008) [2023-12-27 18:07:32,428][20347] Updated weights for policy 0, policy_version 1801402 (0.0007) [2023-12-27 18:07:32,731][20381] Updated weights for policy 1, policy_version 1799868 (0.0006) [2023-12-27 18:07:32,795][20381] Updated weights for policy 1, policy_version 1799878 (0.0008) [2023-12-27 18:07:32,860][20381] Updated weights for policy 1, policy_version 1799888 (0.0009) [2023-12-27 18:07:33,103][20347] Updated weights for policy 0, policy_version 1801412 (0.0007) [2023-12-27 18:07:33,162][20347] Updated weights for policy 0, policy_version 1801422 (0.0008) [2023-12-27 18:07:33,226][20347] Updated weights for policy 0, policy_version 1801432 (0.0008) [2023-12-27 18:07:33,448][20381] Updated weights for policy 1, policy_version 1799898 (0.0007) [2023-12-27 18:07:33,510][20381] Updated weights for policy 1, policy_version 1799908 (0.0005) [2023-12-27 18:07:33,566][20381] Updated weights for policy 1, policy_version 1799918 (0.0005) [2023-12-27 18:07:33,623][20381] Updated weights for policy 1, policy_version 1799928 (0.0005) [2023-12-27 18:07:33,913][20347] Updated weights for policy 0, policy_version 1801442 (0.0007) [2023-12-27 18:07:33,961][20347] Updated weights for policy 0, policy_version 1801452 (0.0008) [2023-12-27 18:07:34,005][20347] Updated weights for policy 0, policy_version 1801462 (0.0008) [2023-12-27 18:07:34,052][20347] Updated weights for policy 0, policy_version 1801472 (0.0007) [2023-12-27 18:07:34,121][20381] Updated weights for policy 1, policy_version 1799938 (0.0010) [2023-12-27 18:07:34,184][20381] Updated weights for policy 1, policy_version 1799948 (0.0009) [2023-12-27 18:07:34,244][20381] Updated weights for policy 1, policy_version 1799958 (0.0010) [2023-12-27 18:07:34,774][20347] Updated weights for policy 0, policy_version 1801482 (0.0008) [2023-12-27 18:07:34,831][20347] Updated weights for policy 0, policy_version 1801492 (0.0007) [2023-12-27 18:07:34,885][20347] Updated weights for policy 0, policy_version 1801502 (0.0007) [2023-12-27 18:07:34,959][20381] Updated weights for policy 1, policy_version 1799968 (0.0011) [2023-12-27 18:07:35,008][20381] Updated weights for policy 1, policy_version 1799978 (0.0011) [2023-12-27 18:07:35,059][20381] Updated weights for policy 1, policy_version 1799988 (0.0009) [2023-12-27 18:07:35,568][20347] Updated weights for policy 0, policy_version 1801512 (0.0009) [2023-12-27 18:07:35,626][20347] Updated weights for policy 0, policy_version 1801522 (0.0008) [2023-12-27 18:07:35,679][20347] Updated weights for policy 0, policy_version 1801532 (0.0005) [2023-12-27 18:07:35,744][20381] Updated weights for policy 1, policy_version 1799998 (0.0009) [2023-12-27 18:07:35,803][20381] Updated weights for policy 1, policy_version 1800008 (0.0011) [2023-12-27 18:07:35,851][20381] Updated weights for policy 1, policy_version 1800018 (0.0010) [2023-12-27 18:07:36,275][20347] Updated weights for policy 0, policy_version 1801542 (0.0007) [2023-12-27 18:07:36,335][20347] Updated weights for policy 0, policy_version 1801552 (0.0008) [2023-12-27 18:07:36,395][20347] Updated weights for policy 0, policy_version 1801562 (0.0008) [2023-12-27 18:07:36,548][20381] Updated weights for policy 1, policy_version 1800028 (0.0010) [2023-12-27 18:07:36,612][20381] Updated weights for policy 1, policy_version 1800038 (0.0011) [2023-12-27 18:07:36,672][20381] Updated weights for policy 1, policy_version 1800048 (0.0010) [2023-12-27 18:07:37,135][20347] Updated weights for policy 0, policy_version 1801572 (0.0009) [2023-12-27 18:07:37,196][20347] Updated weights for policy 0, policy_version 1801582 (0.0009) [2023-12-27 18:07:37,252][20347] Updated weights for policy 0, policy_version 1801592 (0.0009) [2023-12-27 18:07:37,320][20381] Updated weights for policy 1, policy_version 1800058 (0.0007) [2023-12-27 18:07:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 922157056. Throughput: 0: 10267.6, 1: 10348.0. Samples: 922126108. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:37,363][19170] Avg episode reward: [(0, '960.940'), (1, '990.400')] [2023-12-27 18:07:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001801600_461291520.pth... [2023-12-27 18:07:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001800384_460980224.pth [2023-12-27 18:07:37,385][20381] Updated weights for policy 1, policy_version 1800068 (0.0011) [2023-12-27 18:07:37,440][20381] Updated weights for policy 1, policy_version 1800078 (0.0011) [2023-12-27 18:07:37,495][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001800088_460873728.pth... [2023-12-27 18:07:37,496][20381] Updated weights for policy 1, policy_version 1800088 (0.0009) [2023-12-27 18:07:37,498][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001798872_460562432.pth [2023-12-27 18:07:37,913][20347] Updated weights for policy 0, policy_version 1801602 (0.0005) [2023-12-27 18:07:37,970][20347] Updated weights for policy 0, policy_version 1801612 (0.0007) [2023-12-27 18:07:38,029][20347] Updated weights for policy 0, policy_version 1801622 (0.0008) [2023-12-27 18:07:38,084][20347] Updated weights for policy 0, policy_version 1801632 (0.0007) [2023-12-27 18:07:38,146][20381] Updated weights for policy 1, policy_version 1800098 (0.0007) [2023-12-27 18:07:38,202][20381] Updated weights for policy 1, policy_version 1800108 (0.0006) [2023-12-27 18:07:38,253][20381] Updated weights for policy 1, policy_version 1800118 (0.0006) [2023-12-27 18:07:38,770][20347] Updated weights for policy 0, policy_version 1801642 (0.0011) [2023-12-27 18:07:38,832][20347] Updated weights for policy 0, policy_version 1801652 (0.0011) [2023-12-27 18:07:38,884][20347] Updated weights for policy 0, policy_version 1801662 (0.0008) [2023-12-27 18:07:38,896][20381] Updated weights for policy 1, policy_version 1800128 (0.0009) [2023-12-27 18:07:38,944][20381] Updated weights for policy 1, policy_version 1800138 (0.0010) [2023-12-27 18:07:38,992][20381] Updated weights for policy 1, policy_version 1800148 (0.0010) [2023-12-27 18:07:39,569][20347] Updated weights for policy 0, policy_version 1801672 (0.0007) [2023-12-27 18:07:39,624][20347] Updated weights for policy 0, policy_version 1801682 (0.0008) [2023-12-27 18:07:39,672][20347] Updated weights for policy 0, policy_version 1801692 (0.0008) [2023-12-27 18:07:39,747][20381] Updated weights for policy 1, policy_version 1800158 (0.0009) [2023-12-27 18:07:39,802][20381] Updated weights for policy 1, policy_version 1800168 (0.0007) [2023-12-27 18:07:39,870][20381] Updated weights for policy 1, policy_version 1800178 (0.0010) [2023-12-27 18:07:40,337][20347] Updated weights for policy 0, policy_version 1801702 (0.0007) [2023-12-27 18:07:40,399][20347] Updated weights for policy 0, policy_version 1801712 (0.0008) [2023-12-27 18:07:40,449][20347] Updated weights for policy 0, policy_version 1801722 (0.0007) [2023-12-27 18:07:40,612][20381] Updated weights for policy 1, policy_version 1800188 (0.0008) [2023-12-27 18:07:40,674][20381] Updated weights for policy 1, policy_version 1800198 (0.0007) [2023-12-27 18:07:40,737][20381] Updated weights for policy 1, policy_version 1800208 (0.0007) [2023-12-27 18:07:41,093][20347] Updated weights for policy 0, policy_version 1801732 (0.0007) [2023-12-27 18:07:41,161][20347] Updated weights for policy 0, policy_version 1801742 (0.0010) [2023-12-27 18:07:41,219][20347] Updated weights for policy 0, policy_version 1801752 (0.0008) [2023-12-27 18:07:41,529][20381] Updated weights for policy 1, policy_version 1800218 (0.0008) [2023-12-27 18:07:41,586][20381] Updated weights for policy 1, policy_version 1800228 (0.0007) [2023-12-27 18:07:41,663][20381] Updated weights for policy 1, policy_version 1800238 (0.0007) [2023-12-27 18:07:41,731][20381] Updated weights for policy 1, policy_version 1800248 (0.0007) [2023-12-27 18:07:41,866][20347] Updated weights for policy 0, policy_version 1801762 (0.0007) [2023-12-27 18:07:41,920][20347] Updated weights for policy 0, policy_version 1801772 (0.0006) [2023-12-27 18:07:41,974][20347] Updated weights for policy 0, policy_version 1801782 (0.0006) [2023-12-27 18:07:42,030][20347] Updated weights for policy 0, policy_version 1801792 (0.0006) [2023-12-27 18:07:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 922255360. Throughput: 0: 10247.4, 1: 10319.5. Samples: 922248312. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:42,363][19170] Avg episode reward: [(0, '961.020'), (1, '990.400')] [2023-12-27 18:07:42,497][20381] Updated weights for policy 1, policy_version 1800258 (0.0009) [2023-12-27 18:07:42,561][20381] Updated weights for policy 1, policy_version 1800268 (0.0008) [2023-12-27 18:07:42,623][20381] Updated weights for policy 1, policy_version 1800278 (0.0008) [2023-12-27 18:07:42,676][20347] Updated weights for policy 0, policy_version 1801802 (0.0007) [2023-12-27 18:07:42,732][20347] Updated weights for policy 0, policy_version 1801812 (0.0008) [2023-12-27 18:07:42,788][20347] Updated weights for policy 0, policy_version 1801822 (0.0007) [2023-12-27 18:07:43,318][20381] Updated weights for policy 1, policy_version 1800288 (0.0009) [2023-12-27 18:07:43,368][20381] Updated weights for policy 1, policy_version 1800298 (0.0008) [2023-12-27 18:07:43,428][20381] Updated weights for policy 1, policy_version 1800308 (0.0007) [2023-12-27 18:07:43,446][20347] Updated weights for policy 0, policy_version 1801832 (0.0006) [2023-12-27 18:07:43,501][20347] Updated weights for policy 0, policy_version 1801842 (0.0008) [2023-12-27 18:07:43,557][20347] Updated weights for policy 0, policy_version 1801852 (0.0008) [2023-12-27 18:07:44,169][20381] Updated weights for policy 1, policy_version 1800318 (0.0006) [2023-12-27 18:07:44,190][20347] Updated weights for policy 0, policy_version 1801862 (0.0007) [2023-12-27 18:07:44,219][20381] Updated weights for policy 1, policy_version 1800328 (0.0007) [2023-12-27 18:07:44,240][20347] Updated weights for policy 0, policy_version 1801872 (0.0006) [2023-12-27 18:07:44,268][20381] Updated weights for policy 1, policy_version 1800338 (0.0005) [2023-12-27 18:07:44,295][20347] Updated weights for policy 0, policy_version 1801882 (0.0006) [2023-12-27 18:07:44,938][20347] Updated weights for policy 0, policy_version 1801892 (0.0007) [2023-12-27 18:07:44,992][20381] Updated weights for policy 1, policy_version 1800348 (0.0008) [2023-12-27 18:07:45,000][20347] Updated weights for policy 0, policy_version 1801902 (0.0008) [2023-12-27 18:07:45,046][20381] Updated weights for policy 1, policy_version 1800358 (0.0007) [2023-12-27 18:07:45,063][20347] Updated weights for policy 0, policy_version 1801912 (0.0008) [2023-12-27 18:07:45,092][20381] Updated weights for policy 1, policy_version 1800368 (0.0006) [2023-12-27 18:07:45,730][20347] Updated weights for policy 0, policy_version 1801922 (0.0007) [2023-12-27 18:07:45,788][20347] Updated weights for policy 0, policy_version 1801932 (0.0007) [2023-12-27 18:07:45,824][20381] Updated weights for policy 1, policy_version 1800378 (0.0006) [2023-12-27 18:07:45,848][20347] Updated weights for policy 0, policy_version 1801942 (0.0008) [2023-12-27 18:07:45,889][20381] Updated weights for policy 1, policy_version 1800388 (0.0007) [2023-12-27 18:07:45,912][20347] Updated weights for policy 0, policy_version 1801952 (0.0006) [2023-12-27 18:07:45,940][20381] Updated weights for policy 1, policy_version 1800398 (0.0007) [2023-12-27 18:07:45,991][20381] Updated weights for policy 1, policy_version 1800408 (0.0007) [2023-12-27 18:07:46,621][20347] Updated weights for policy 0, policy_version 1801962 (0.0008) [2023-12-27 18:07:46,644][20381] Updated weights for policy 1, policy_version 1800418 (0.0006) [2023-12-27 18:07:46,674][20347] Updated weights for policy 0, policy_version 1801972 (0.0007) [2023-12-27 18:07:46,701][20381] Updated weights for policy 1, policy_version 1800428 (0.0005) [2023-12-27 18:07:46,726][20347] Updated weights for policy 0, policy_version 1801982 (0.0007) [2023-12-27 18:07:46,757][20381] Updated weights for policy 1, policy_version 1800438 (0.0005) [2023-12-27 18:07:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 922353664. Throughput: 0: 10219.5, 1: 10270.4. Samples: 922368792. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:47,363][19170] Avg episode reward: [(0, '941.580'), (1, '1000.000')] [2023-12-27 18:07:47,401][20347] Updated weights for policy 0, policy_version 1801992 (0.0006) [2023-12-27 18:07:47,403][20381] Updated weights for policy 1, policy_version 1800448 (0.0007) [2023-12-27 18:07:47,447][20347] Updated weights for policy 0, policy_version 1802002 (0.0007) [2023-12-27 18:07:47,463][20381] Updated weights for policy 1, policy_version 1800458 (0.0007) [2023-12-27 18:07:47,504][20347] Updated weights for policy 0, policy_version 1802012 (0.0005) [2023-12-27 18:07:47,516][20381] Updated weights for policy 1, policy_version 1800468 (0.0007) [2023-12-27 18:07:48,113][20347] Updated weights for policy 0, policy_version 1802022 (0.0006) [2023-12-27 18:07:48,161][20347] Updated weights for policy 0, policy_version 1802032 (0.0010) [2023-12-27 18:07:48,213][20347] Updated weights for policy 0, policy_version 1802042 (0.0010) [2023-12-27 18:07:48,237][20381] Updated weights for policy 1, policy_version 1800478 (0.0006) [2023-12-27 18:07:48,291][20381] Updated weights for policy 1, policy_version 1800488 (0.0007) [2023-12-27 18:07:48,350][20381] Updated weights for policy 1, policy_version 1800498 (0.0007) [2023-12-27 18:07:48,874][20347] Updated weights for policy 0, policy_version 1802052 (0.0008) [2023-12-27 18:07:48,927][20347] Updated weights for policy 0, policy_version 1802062 (0.0006) [2023-12-27 18:07:48,991][20347] Updated weights for policy 0, policy_version 1802072 (0.0008) [2023-12-27 18:07:49,036][20381] Updated weights for policy 1, policy_version 1800508 (0.0007) [2023-12-27 18:07:49,096][20381] Updated weights for policy 1, policy_version 1800518 (0.0008) [2023-12-27 18:07:49,145][20381] Updated weights for policy 1, policy_version 1800528 (0.0007) [2023-12-27 18:07:49,636][20347] Updated weights for policy 0, policy_version 1802082 (0.0006) [2023-12-27 18:07:49,690][20347] Updated weights for policy 0, policy_version 1802092 (0.0007) [2023-12-27 18:07:49,745][20347] Updated weights for policy 0, policy_version 1802102 (0.0007) [2023-12-27 18:07:49,793][20347] Updated weights for policy 0, policy_version 1802112 (0.0007) [2023-12-27 18:07:49,883][20381] Updated weights for policy 1, policy_version 1800538 (0.0010) [2023-12-27 18:07:49,939][20381] Updated weights for policy 1, policy_version 1800548 (0.0010) [2023-12-27 18:07:49,999][20381] Updated weights for policy 1, policy_version 1800558 (0.0009) [2023-12-27 18:07:50,052][20381] Updated weights for policy 1, policy_version 1800568 (0.0009) [2023-12-27 18:07:50,528][20347] Updated weights for policy 0, policy_version 1802122 (0.0007) [2023-12-27 18:07:50,593][20347] Updated weights for policy 0, policy_version 1802132 (0.0008) [2023-12-27 18:07:50,681][20347] Updated weights for policy 0, policy_version 1802142 (0.0007) [2023-12-27 18:07:50,789][20381] Updated weights for policy 1, policy_version 1800578 (0.0011) [2023-12-27 18:07:50,855][20381] Updated weights for policy 1, policy_version 1800588 (0.0011) [2023-12-27 18:07:50,914][20381] Updated weights for policy 1, policy_version 1800598 (0.0011) [2023-12-27 18:07:51,409][20347] Updated weights for policy 0, policy_version 1802152 (0.0008) [2023-12-27 18:07:51,468][20347] Updated weights for policy 0, policy_version 1802162 (0.0007) [2023-12-27 18:07:51,528][20347] Updated weights for policy 0, policy_version 1802172 (0.0008) [2023-12-27 18:07:51,637][20381] Updated weights for policy 1, policy_version 1800608 (0.0008) [2023-12-27 18:07:51,694][20381] Updated weights for policy 1, policy_version 1800618 (0.0008) [2023-12-27 18:07:51,756][20381] Updated weights for policy 1, policy_version 1800628 (0.0008) [2023-12-27 18:07:52,248][20347] Updated weights for policy 0, policy_version 1802182 (0.0007) [2023-12-27 18:07:52,307][20347] Updated weights for policy 0, policy_version 1802192 (0.0008) [2023-12-27 18:07:52,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 922451968. Throughput: 0: 10270.6, 1: 10186.0. Samples: 922429928. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:52,363][19170] Avg episode reward: [(0, '980.560'), (1, '970.900')] [2023-12-27 18:07:52,370][20347] Updated weights for policy 0, policy_version 1802202 (0.0006) [2023-12-27 18:07:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001800632_461012992.pth... [2023-12-27 18:07:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001799448_460709888.pth [2023-12-27 18:07:52,407][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001802208_461447168.pth... [2023-12-27 18:07:52,412][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001800960_461127680.pth [2023-12-27 18:07:52,482][20381] Updated weights for policy 1, policy_version 1800638 (0.0009) [2023-12-27 18:07:52,542][20381] Updated weights for policy 1, policy_version 1800648 (0.0009) [2023-12-27 18:07:52,610][20381] Updated weights for policy 1, policy_version 1800658 (0.0008) [2023-12-27 18:07:52,970][20347] Updated weights for policy 0, policy_version 1802212 (0.0007) [2023-12-27 18:07:53,022][20347] Updated weights for policy 0, policy_version 1802222 (0.0007) [2023-12-27 18:07:53,071][20347] Updated weights for policy 0, policy_version 1802232 (0.0007) [2023-12-27 18:07:53,234][20381] Updated weights for policy 1, policy_version 1800668 (0.0009) [2023-12-27 18:07:53,288][20381] Updated weights for policy 1, policy_version 1800678 (0.0007) [2023-12-27 18:07:53,334][20381] Updated weights for policy 1, policy_version 1800688 (0.0010) [2023-12-27 18:07:53,773][20347] Updated weights for policy 0, policy_version 1802242 (0.0008) [2023-12-27 18:07:53,824][20347] Updated weights for policy 0, policy_version 1802252 (0.0006) [2023-12-27 18:07:53,885][20347] Updated weights for policy 0, policy_version 1802262 (0.0010) [2023-12-27 18:07:53,941][20347] Updated weights for policy 0, policy_version 1802272 (0.0010) [2023-12-27 18:07:54,011][20381] Updated weights for policy 1, policy_version 1800698 (0.0011) [2023-12-27 18:07:54,065][20381] Updated weights for policy 1, policy_version 1800708 (0.0008) [2023-12-27 18:07:54,118][20381] Updated weights for policy 1, policy_version 1800718 (0.0009) [2023-12-27 18:07:54,170][20381] Updated weights for policy 1, policy_version 1800728 (0.0010) [2023-12-27 18:07:54,596][20347] Updated weights for policy 0, policy_version 1802282 (0.0010) [2023-12-27 18:07:54,648][20347] Updated weights for policy 0, policy_version 1802292 (0.0010) [2023-12-27 18:07:54,702][20347] Updated weights for policy 0, policy_version 1802302 (0.0010) [2023-12-27 18:07:54,842][20381] Updated weights for policy 1, policy_version 1800738 (0.0008) [2023-12-27 18:07:54,890][20381] Updated weights for policy 1, policy_version 1800748 (0.0010) [2023-12-27 18:07:54,949][20381] Updated weights for policy 1, policy_version 1800758 (0.0011) [2023-12-27 18:07:55,354][20347] Updated weights for policy 0, policy_version 1802312 (0.0010) [2023-12-27 18:07:55,402][20347] Updated weights for policy 0, policy_version 1802322 (0.0010) [2023-12-27 18:07:55,443][20347] Updated weights for policy 0, policy_version 1802332 (0.0010) [2023-12-27 18:07:55,623][20381] Updated weights for policy 1, policy_version 1800768 (0.0011) [2023-12-27 18:07:55,674][20381] Updated weights for policy 1, policy_version 1800778 (0.0010) [2023-12-27 18:07:55,731][20381] Updated weights for policy 1, policy_version 1800788 (0.0010) [2023-12-27 18:07:56,089][20347] Updated weights for policy 0, policy_version 1802342 (0.0010) [2023-12-27 18:07:56,146][20347] Updated weights for policy 0, policy_version 1802352 (0.0010) [2023-12-27 18:07:56,201][20347] Updated weights for policy 0, policy_version 1802362 (0.0010) [2023-12-27 18:07:56,408][20381] Updated weights for policy 1, policy_version 1800798 (0.0011) [2023-12-27 18:07:56,468][20381] Updated weights for policy 1, policy_version 1800808 (0.0010) [2023-12-27 18:07:56,528][20381] Updated weights for policy 1, policy_version 1800818 (0.0010) [2023-12-27 18:07:56,775][20347] Updated weights for policy 0, policy_version 1802372 (0.0008) [2023-12-27 18:07:56,822][20347] Updated weights for policy 0, policy_version 1802382 (0.0005) [2023-12-27 18:07:56,866][20347] Updated weights for policy 0, policy_version 1802392 (0.0005) [2023-12-27 18:07:57,210][20381] Updated weights for policy 1, policy_version 1800828 (0.0010) [2023-12-27 18:07:57,257][20381] Updated weights for policy 1, policy_version 1800838 (0.0010) [2023-12-27 18:07:57,305][20381] Updated weights for policy 1, policy_version 1800848 (0.0010) [2023-12-27 18:07:57,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 922566656. Throughput: 0: 10348.2, 1: 10125.3. Samples: 922552476. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:07:57,364][19170] Avg episode reward: [(0, '970.720'), (1, '961.150')] [2023-12-27 18:07:57,451][20347] Updated weights for policy 0, policy_version 1802402 (0.0006) [2023-12-27 18:07:57,501][20347] Updated weights for policy 0, policy_version 1802412 (0.0010) [2023-12-27 18:07:57,559][20347] Updated weights for policy 0, policy_version 1802422 (0.0009) [2023-12-27 18:07:57,615][20347] Updated weights for policy 0, policy_version 1802432 (0.0010) [2023-12-27 18:07:58,003][20381] Updated weights for policy 1, policy_version 1800858 (0.0010) [2023-12-27 18:07:58,049][20381] Updated weights for policy 1, policy_version 1800868 (0.0010) [2023-12-27 18:07:58,100][20381] Updated weights for policy 1, policy_version 1800878 (0.0010) [2023-12-27 18:07:58,158][20381] Updated weights for policy 1, policy_version 1800888 (0.0010) [2023-12-27 18:07:58,164][20347] Updated weights for policy 0, policy_version 1802442 (0.0010) [2023-12-27 18:07:58,225][20347] Updated weights for policy 0, policy_version 1802452 (0.0009) [2023-12-27 18:07:58,283][20347] Updated weights for policy 0, policy_version 1802462 (0.0007) [2023-12-27 18:07:58,915][20381] Updated weights for policy 1, policy_version 1800898 (0.0008) [2023-12-27 18:07:58,975][20381] Updated weights for policy 1, policy_version 1800908 (0.0009) [2023-12-27 18:07:59,000][20347] Updated weights for policy 0, policy_version 1802472 (0.0007) [2023-12-27 18:07:59,036][20381] Updated weights for policy 1, policy_version 1800918 (0.0011) [2023-12-27 18:07:59,059][20347] Updated weights for policy 0, policy_version 1802482 (0.0007) [2023-12-27 18:07:59,126][20347] Updated weights for policy 0, policy_version 1802492 (0.0009) [2023-12-27 18:07:59,726][20381] Updated weights for policy 1, policy_version 1800928 (0.0006) [2023-12-27 18:07:59,777][20381] Updated weights for policy 1, policy_version 1800938 (0.0005) [2023-12-27 18:07:59,812][20347] Updated weights for policy 0, policy_version 1802502 (0.0006) [2023-12-27 18:07:59,835][20381] Updated weights for policy 1, policy_version 1800948 (0.0009) [2023-12-27 18:07:59,880][20347] Updated weights for policy 0, policy_version 1802512 (0.0007) [2023-12-27 18:07:59,944][20347] Updated weights for policy 0, policy_version 1802522 (0.0007) [2023-12-27 18:08:00,475][20381] Updated weights for policy 1, policy_version 1800958 (0.0010) [2023-12-27 18:08:00,533][20381] Updated weights for policy 1, policy_version 1800968 (0.0011) [2023-12-27 18:08:00,596][20381] Updated weights for policy 1, policy_version 1800978 (0.0009) [2023-12-27 18:08:00,627][20347] Updated weights for policy 0, policy_version 1802532 (0.0006) [2023-12-27 18:08:00,689][20347] Updated weights for policy 0, policy_version 1802542 (0.0005) [2023-12-27 18:08:00,750][20347] Updated weights for policy 0, policy_version 1802552 (0.0005) [2023-12-27 18:08:01,281][20381] Updated weights for policy 1, policy_version 1800988 (0.0010) [2023-12-27 18:08:01,339][20381] Updated weights for policy 1, policy_version 1800998 (0.0008) [2023-12-27 18:08:01,409][20381] Updated weights for policy 1, policy_version 1801008 (0.0009) [2023-12-27 18:08:01,415][20347] Updated weights for policy 0, policy_version 1802562 (0.0005) [2023-12-27 18:08:01,476][20347] Updated weights for policy 0, policy_version 1802572 (0.0006) [2023-12-27 18:08:01,535][20347] Updated weights for policy 0, policy_version 1802582 (0.0005) [2023-12-27 18:08:01,592][20347] Updated weights for policy 0, policy_version 1802592 (0.0005) [2023-12-27 18:08:02,077][20381] Updated weights for policy 1, policy_version 1801018 (0.0008) [2023-12-27 18:08:02,131][20381] Updated weights for policy 1, policy_version 1801028 (0.0010) [2023-12-27 18:08:02,192][20381] Updated weights for policy 1, policy_version 1801038 (0.0007) [2023-12-27 18:08:02,219][20347] Updated weights for policy 0, policy_version 1802602 (0.0008) [2023-12-27 18:08:02,255][20381] Updated weights for policy 1, policy_version 1801048 (0.0006) [2023-12-27 18:08:02,278][20347] Updated weights for policy 0, policy_version 1802612 (0.0009) [2023-12-27 18:08:02,332][20347] Updated weights for policy 0, policy_version 1802622 (0.0008) [2023-12-27 18:08:02,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 922673152. Throughput: 0: 10441.2, 1: 10103.7. Samples: 922677344. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:02,363][19170] Avg episode reward: [(0, '951.120'), (1, '961.110')] [2023-12-27 18:08:02,969][20381] Updated weights for policy 1, policy_version 1801058 (0.0005) [2023-12-27 18:08:02,996][20347] Updated weights for policy 0, policy_version 1802632 (0.0007) [2023-12-27 18:08:03,031][20381] Updated weights for policy 1, policy_version 1801068 (0.0005) [2023-12-27 18:08:03,055][20347] Updated weights for policy 0, policy_version 1802642 (0.0006) [2023-12-27 18:08:03,091][20381] Updated weights for policy 1, policy_version 1801078 (0.0005) [2023-12-27 18:08:03,118][20347] Updated weights for policy 0, policy_version 1802652 (0.0008) [2023-12-27 18:08:03,731][20381] Updated weights for policy 1, policy_version 1801088 (0.0005) [2023-12-27 18:08:03,752][20347] Updated weights for policy 0, policy_version 1802662 (0.0007) [2023-12-27 18:08:03,777][20381] Updated weights for policy 1, policy_version 1801098 (0.0005) [2023-12-27 18:08:03,808][20347] Updated weights for policy 0, policy_version 1802672 (0.0006) [2023-12-27 18:08:03,825][20381] Updated weights for policy 1, policy_version 1801108 (0.0010) [2023-12-27 18:08:03,867][20347] Updated weights for policy 0, policy_version 1802682 (0.0006) [2023-12-27 18:08:04,448][20381] Updated weights for policy 1, policy_version 1801118 (0.0009) [2023-12-27 18:08:04,504][20381] Updated weights for policy 1, policy_version 1801128 (0.0007) [2023-12-27 18:08:04,571][20381] Updated weights for policy 1, policy_version 1801138 (0.0006) [2023-12-27 18:08:04,582][20347] Updated weights for policy 0, policy_version 1802692 (0.0008) [2023-12-27 18:08:04,634][20347] Updated weights for policy 0, policy_version 1802702 (0.0007) [2023-12-27 18:08:04,693][20347] Updated weights for policy 0, policy_version 1802712 (0.0006) [2023-12-27 18:08:05,183][20381] Updated weights for policy 1, policy_version 1801148 (0.0007) [2023-12-27 18:08:05,235][20381] Updated weights for policy 1, policy_version 1801158 (0.0008) [2023-12-27 18:08:05,298][20381] Updated weights for policy 1, policy_version 1801168 (0.0005) [2023-12-27 18:08:05,302][20347] Updated weights for policy 0, policy_version 1802722 (0.0006) [2023-12-27 18:08:05,353][20347] Updated weights for policy 0, policy_version 1802732 (0.0007) [2023-12-27 18:08:05,401][20347] Updated weights for policy 0, policy_version 1802742 (0.0007) [2023-12-27 18:08:05,447][20347] Updated weights for policy 0, policy_version 1802752 (0.0008) [2023-12-27 18:08:05,856][20381] Updated weights for policy 1, policy_version 1801178 (0.0006) [2023-12-27 18:08:05,910][20381] Updated weights for policy 1, policy_version 1801188 (0.0007) [2023-12-27 18:08:05,958][20381] Updated weights for policy 1, policy_version 1801198 (0.0007) [2023-12-27 18:08:06,015][20381] Updated weights for policy 1, policy_version 1801208 (0.0007) [2023-12-27 18:08:06,150][20347] Updated weights for policy 0, policy_version 1802762 (0.0008) [2023-12-27 18:08:06,217][20347] Updated weights for policy 0, policy_version 1802772 (0.0007) [2023-12-27 18:08:06,281][20347] Updated weights for policy 0, policy_version 1802782 (0.0006) [2023-12-27 18:08:06,750][20381] Updated weights for policy 1, policy_version 1801218 (0.0009) [2023-12-27 18:08:06,800][20381] Updated weights for policy 1, policy_version 1801228 (0.0009) [2023-12-27 18:08:06,853][20381] Updated weights for policy 1, policy_version 1801238 (0.0008) [2023-12-27 18:08:06,863][20347] Updated weights for policy 0, policy_version 1802792 (0.0006) [2023-12-27 18:08:06,916][20347] Updated weights for policy 0, policy_version 1802802 (0.0005) [2023-12-27 18:08:06,972][20347] Updated weights for policy 0, policy_version 1802812 (0.0008) [2023-12-27 18:08:07,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 922771456. Throughput: 0: 10447.3, 1: 10149.1. Samples: 922740664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:07,364][19170] Avg episode reward: [(0, '980.520'), (1, '961.140')] [2023-12-27 18:08:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001801240_461168640.pth... [2023-12-27 18:08:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001802816_461602816.pth... [2023-12-27 18:08:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001800088_460873728.pth [2023-12-27 18:08:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001801600_461291520.pth [2023-12-27 18:08:07,501][20381] Updated weights for policy 1, policy_version 1801248 (0.0007) [2023-12-27 18:08:07,549][20381] Updated weights for policy 1, policy_version 1801258 (0.0007) [2023-12-27 18:08:07,603][20381] Updated weights for policy 1, policy_version 1801268 (0.0007) [2023-12-27 18:08:07,623][20347] Updated weights for policy 0, policy_version 1802822 (0.0007) [2023-12-27 18:08:07,690][20347] Updated weights for policy 0, policy_version 1802832 (0.0008) [2023-12-27 18:08:07,747][20347] Updated weights for policy 0, policy_version 1802842 (0.0008) [2023-12-27 18:08:08,304][20381] Updated weights for policy 1, policy_version 1801278 (0.0007) [2023-12-27 18:08:08,368][20381] Updated weights for policy 1, policy_version 1801288 (0.0007) [2023-12-27 18:08:08,406][20347] Updated weights for policy 0, policy_version 1802852 (0.0006) [2023-12-27 18:08:08,436][20381] Updated weights for policy 1, policy_version 1801298 (0.0008) [2023-12-27 18:08:08,466][20347] Updated weights for policy 0, policy_version 1802862 (0.0007) [2023-12-27 18:08:08,522][20347] Updated weights for policy 0, policy_version 1802872 (0.0008) [2023-12-27 18:08:09,036][20381] Updated weights for policy 1, policy_version 1801308 (0.0009) [2023-12-27 18:08:09,085][20381] Updated weights for policy 1, policy_version 1801318 (0.0010) [2023-12-27 18:08:09,141][20381] Updated weights for policy 1, policy_version 1801328 (0.0011) [2023-12-27 18:08:09,263][20347] Updated weights for policy 0, policy_version 1802882 (0.0008) [2023-12-27 18:08:09,323][20347] Updated weights for policy 0, policy_version 1802892 (0.0007) [2023-12-27 18:08:09,395][20347] Updated weights for policy 0, policy_version 1802902 (0.0008) [2023-12-27 18:08:09,461][20347] Updated weights for policy 0, policy_version 1802912 (0.0007) [2023-12-27 18:08:09,909][20381] Updated weights for policy 1, policy_version 1801338 (0.0010) [2023-12-27 18:08:09,971][20381] Updated weights for policy 1, policy_version 1801348 (0.0008) [2023-12-27 18:08:10,031][20381] Updated weights for policy 1, policy_version 1801358 (0.0008) [2023-12-27 18:08:10,088][20381] Updated weights for policy 1, policy_version 1801368 (0.0008) [2023-12-27 18:08:10,163][20347] Updated weights for policy 0, policy_version 1802922 (0.0011) [2023-12-27 18:08:10,213][20347] Updated weights for policy 0, policy_version 1802932 (0.0011) [2023-12-27 18:08:10,270][20347] Updated weights for policy 0, policy_version 1802942 (0.0011) [2023-12-27 18:08:10,785][20381] Updated weights for policy 1, policy_version 1801378 (0.0007) [2023-12-27 18:08:10,840][20381] Updated weights for policy 1, policy_version 1801388 (0.0006) [2023-12-27 18:08:10,892][20381] Updated weights for policy 1, policy_version 1801398 (0.0006) [2023-12-27 18:08:10,988][20347] Updated weights for policy 0, policy_version 1802952 (0.0010) [2023-12-27 18:08:11,054][20347] Updated weights for policy 0, policy_version 1802962 (0.0009) [2023-12-27 18:08:11,115][20347] Updated weights for policy 0, policy_version 1802972 (0.0009) [2023-12-27 18:08:11,653][20381] Updated weights for policy 1, policy_version 1801408 (0.0008) [2023-12-27 18:08:11,709][20381] Updated weights for policy 1, policy_version 1801418 (0.0008) [2023-12-27 18:08:11,772][20381] Updated weights for policy 1, policy_version 1801428 (0.0006) [2023-12-27 18:08:11,892][20347] Updated weights for policy 0, policy_version 1802982 (0.0008) [2023-12-27 18:08:11,950][20347] Updated weights for policy 0, policy_version 1802992 (0.0008) [2023-12-27 18:08:12,008][20347] Updated weights for policy 0, policy_version 1803002 (0.0007) [2023-12-27 18:08:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 922869760. Throughput: 0: 10448.4, 1: 10176.5. Samples: 922862948. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:12,363][19170] Avg episode reward: [(0, '960.950'), (1, '990.280')] [2023-12-27 18:08:12,488][20381] Updated weights for policy 1, policy_version 1801438 (0.0006) [2023-12-27 18:08:12,555][20381] Updated weights for policy 1, policy_version 1801448 (0.0006) [2023-12-27 18:08:12,612][20381] Updated weights for policy 1, policy_version 1801458 (0.0007) [2023-12-27 18:08:12,732][20347] Updated weights for policy 0, policy_version 1803012 (0.0007) [2023-12-27 18:08:12,789][20347] Updated weights for policy 0, policy_version 1803022 (0.0007) [2023-12-27 18:08:12,847][20347] Updated weights for policy 0, policy_version 1803032 (0.0008) [2023-12-27 18:08:13,304][20381] Updated weights for policy 1, policy_version 1801468 (0.0008) [2023-12-27 18:08:13,358][20381] Updated weights for policy 1, policy_version 1801478 (0.0008) [2023-12-27 18:08:13,407][20347] Updated weights for policy 0, policy_version 1803042 (0.0007) [2023-12-27 18:08:13,408][20381] Updated weights for policy 1, policy_version 1801488 (0.0007) [2023-12-27 18:08:13,465][20347] Updated weights for policy 0, policy_version 1803052 (0.0005) [2023-12-27 18:08:13,525][20347] Updated weights for policy 0, policy_version 1803062 (0.0006) [2023-12-27 18:08:13,574][20347] Updated weights for policy 0, policy_version 1803072 (0.0006) [2023-12-27 18:08:14,150][20381] Updated weights for policy 1, policy_version 1801498 (0.0006) [2023-12-27 18:08:14,163][20347] Updated weights for policy 0, policy_version 1803082 (0.0009) [2023-12-27 18:08:14,202][20381] Updated weights for policy 1, policy_version 1801508 (0.0005) [2023-12-27 18:08:14,219][20347] Updated weights for policy 0, policy_version 1803092 (0.0010) [2023-12-27 18:08:14,263][20381] Updated weights for policy 1, policy_version 1801518 (0.0006) [2023-12-27 18:08:14,280][20347] Updated weights for policy 0, policy_version 1803102 (0.0010) [2023-12-27 18:08:14,319][20381] Updated weights for policy 1, policy_version 1801528 (0.0006) [2023-12-27 18:08:14,936][20347] Updated weights for policy 0, policy_version 1803112 (0.0010) [2023-12-27 18:08:14,984][20347] Updated weights for policy 0, policy_version 1803122 (0.0008) [2023-12-27 18:08:15,006][20381] Updated weights for policy 1, policy_version 1801538 (0.0007) [2023-12-27 18:08:15,047][20347] Updated weights for policy 0, policy_version 1803132 (0.0006) [2023-12-27 18:08:15,071][20381] Updated weights for policy 1, policy_version 1801548 (0.0007) [2023-12-27 18:08:15,141][20381] Updated weights for policy 1, policy_version 1801558 (0.0009) [2023-12-27 18:08:15,661][20347] Updated weights for policy 0, policy_version 1803142 (0.0007) [2023-12-27 18:08:15,704][20347] Updated weights for policy 0, policy_version 1803152 (0.0005) [2023-12-27 18:08:15,763][20347] Updated weights for policy 0, policy_version 1803162 (0.0006) [2023-12-27 18:08:15,834][20381] Updated weights for policy 1, policy_version 1801568 (0.0006) [2023-12-27 18:08:15,886][20381] Updated weights for policy 1, policy_version 1801578 (0.0006) [2023-12-27 18:08:15,945][20381] Updated weights for policy 1, policy_version 1801588 (0.0006) [2023-12-27 18:08:16,458][20347] Updated weights for policy 0, policy_version 1803172 (0.0008) [2023-12-27 18:08:16,517][20347] Updated weights for policy 0, policy_version 1803182 (0.0008) [2023-12-27 18:08:16,546][20381] Updated weights for policy 1, policy_version 1801598 (0.0006) [2023-12-27 18:08:16,574][20347] Updated weights for policy 0, policy_version 1803192 (0.0006) [2023-12-27 18:08:16,602][20381] Updated weights for policy 1, policy_version 1801608 (0.0005) [2023-12-27 18:08:16,660][20381] Updated weights for policy 1, policy_version 1801618 (0.0007) [2023-12-27 18:08:17,187][20347] Updated weights for policy 0, policy_version 1803202 (0.0005) [2023-12-27 18:08:17,241][20347] Updated weights for policy 0, policy_version 1803212 (0.0008) [2023-12-27 18:08:17,286][20347] Updated weights for policy 0, policy_version 1803222 (0.0008) [2023-12-27 18:08:17,303][20381] Updated weights for policy 1, policy_version 1801628 (0.0007) [2023-12-27 18:08:17,341][20347] Updated weights for policy 0, policy_version 1803232 (0.0008) [2023-12-27 18:08:17,353][20381] Updated weights for policy 1, policy_version 1801638 (0.0005) [2023-12-27 18:08:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 922976256. Throughput: 0: 10383.6, 1: 10112.3. Samples: 922985444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:17,363][19170] Avg episode reward: [(0, '960.850'), (1, '990.260')] [2023-12-27 18:08:17,407][20381] Updated weights for policy 1, policy_version 1801648 (0.0007) [2023-12-27 18:08:17,993][20347] Updated weights for policy 0, policy_version 1803242 (0.0007) [2023-12-27 18:08:18,052][20347] Updated weights for policy 0, policy_version 1803252 (0.0007) [2023-12-27 18:08:18,106][20381] Updated weights for policy 1, policy_version 1801658 (0.0008) [2023-12-27 18:08:18,110][20347] Updated weights for policy 0, policy_version 1803262 (0.0005) [2023-12-27 18:08:18,160][20381] Updated weights for policy 1, policy_version 1801668 (0.0009) [2023-12-27 18:08:18,212][20381] Updated weights for policy 1, policy_version 1801678 (0.0008) [2023-12-27 18:08:18,268][20381] Updated weights for policy 1, policy_version 1801688 (0.0008) [2023-12-27 18:08:18,721][20347] Updated weights for policy 0, policy_version 1803272 (0.0008) [2023-12-27 18:08:18,774][20347] Updated weights for policy 0, policy_version 1803282 (0.0007) [2023-12-27 18:08:18,830][20347] Updated weights for policy 0, policy_version 1803292 (0.0007) [2023-12-27 18:08:18,955][20381] Updated weights for policy 1, policy_version 1801698 (0.0008) [2023-12-27 18:08:19,005][20381] Updated weights for policy 1, policy_version 1801708 (0.0007) [2023-12-27 18:08:19,063][20381] Updated weights for policy 1, policy_version 1801718 (0.0008) [2023-12-27 18:08:19,581][20347] Updated weights for policy 0, policy_version 1803302 (0.0009) [2023-12-27 18:08:19,635][20347] Updated weights for policy 0, policy_version 1803312 (0.0009) [2023-12-27 18:08:19,697][20347] Updated weights for policy 0, policy_version 1803322 (0.0009) [2023-12-27 18:08:19,770][20381] Updated weights for policy 1, policy_version 1801728 (0.0008) [2023-12-27 18:08:19,827][20381] Updated weights for policy 1, policy_version 1801738 (0.0009) [2023-12-27 18:08:19,890][20381] Updated weights for policy 1, policy_version 1801748 (0.0007) [2023-12-27 18:08:20,446][20347] Updated weights for policy 0, policy_version 1803332 (0.0008) [2023-12-27 18:08:20,501][20347] Updated weights for policy 0, policy_version 1803342 (0.0009) [2023-12-27 18:08:20,551][20347] Updated weights for policy 0, policy_version 1803352 (0.0010) [2023-12-27 18:08:20,614][20381] Updated weights for policy 1, policy_version 1801758 (0.0007) [2023-12-27 18:08:20,675][20381] Updated weights for policy 1, policy_version 1801768 (0.0006) [2023-12-27 18:08:20,739][20381] Updated weights for policy 1, policy_version 1801778 (0.0006) [2023-12-27 18:08:21,287][20347] Updated weights for policy 0, policy_version 1803362 (0.0010) [2023-12-27 18:08:21,353][20347] Updated weights for policy 0, policy_version 1803372 (0.0010) [2023-12-27 18:08:21,428][20347] Updated weights for policy 0, policy_version 1803382 (0.0008) [2023-12-27 18:08:21,437][20381] Updated weights for policy 1, policy_version 1801788 (0.0006) [2023-12-27 18:08:21,486][20347] Updated weights for policy 0, policy_version 1803392 (0.0010) [2023-12-27 18:08:21,500][20381] Updated weights for policy 1, policy_version 1801798 (0.0006) [2023-12-27 18:08:21,561][20381] Updated weights for policy 1, policy_version 1801808 (0.0007) [2023-12-27 18:08:22,192][20347] Updated weights for policy 0, policy_version 1803402 (0.0007) [2023-12-27 18:08:22,230][20381] Updated weights for policy 1, policy_version 1801818 (0.0007) [2023-12-27 18:08:22,245][20347] Updated weights for policy 0, policy_version 1803412 (0.0007) [2023-12-27 18:08:22,291][20381] Updated weights for policy 1, policy_version 1801828 (0.0007) [2023-12-27 18:08:22,303][20347] Updated weights for policy 0, policy_version 1803422 (0.0007) [2023-12-27 18:08:22,351][20381] Updated weights for policy 1, policy_version 1801838 (0.0009) [2023-12-27 18:08:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 923074560. Throughput: 0: 10404.8, 1: 10050.0. Samples: 923046576. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:22,363][19170] Avg episode reward: [(0, '970.710'), (1, '970.860')] [2023-12-27 18:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001803424_461758464.pth... [2023-12-27 18:08:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001802208_461447168.pth [2023-12-27 18:08:22,410][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001801848_461324288.pth... [2023-12-27 18:08:22,412][20381] Updated weights for policy 1, policy_version 1801848 (0.0008) [2023-12-27 18:08:22,414][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001800632_461012992.pth [2023-12-27 18:08:22,967][20347] Updated weights for policy 0, policy_version 1803432 (0.0007) [2023-12-27 18:08:23,026][20347] Updated weights for policy 0, policy_version 1803442 (0.0008) [2023-12-27 18:08:23,086][20347] Updated weights for policy 0, policy_version 1803452 (0.0007) [2023-12-27 18:08:23,127][20381] Updated weights for policy 1, policy_version 1801858 (0.0009) [2023-12-27 18:08:23,181][20381] Updated weights for policy 1, policy_version 1801868 (0.0008) [2023-12-27 18:08:23,234][20381] Updated weights for policy 1, policy_version 1801878 (0.0008) [2023-12-27 18:08:23,709][20347] Updated weights for policy 0, policy_version 1803462 (0.0008) [2023-12-27 18:08:23,768][20347] Updated weights for policy 0, policy_version 1803472 (0.0007) [2023-12-27 18:08:23,826][20347] Updated weights for policy 0, policy_version 1803482 (0.0007) [2023-12-27 18:08:23,970][20381] Updated weights for policy 1, policy_version 1801888 (0.0009) [2023-12-27 18:08:24,030][20381] Updated weights for policy 1, policy_version 1801898 (0.0008) [2023-12-27 18:08:24,086][20381] Updated weights for policy 1, policy_version 1801908 (0.0008) [2023-12-27 18:08:24,423][20347] Updated weights for policy 0, policy_version 1803492 (0.0005) [2023-12-27 18:08:24,482][20347] Updated weights for policy 0, policy_version 1803502 (0.0006) [2023-12-27 18:08:24,538][20347] Updated weights for policy 0, policy_version 1803512 (0.0007) [2023-12-27 18:08:24,780][20381] Updated weights for policy 1, policy_version 1801918 (0.0007) [2023-12-27 18:08:24,836][20381] Updated weights for policy 1, policy_version 1801928 (0.0008) [2023-12-27 18:08:24,896][20381] Updated weights for policy 1, policy_version 1801938 (0.0009) [2023-12-27 18:08:25,206][20347] Updated weights for policy 0, policy_version 1803522 (0.0007) [2023-12-27 18:08:25,269][20347] Updated weights for policy 0, policy_version 1803532 (0.0008) [2023-12-27 18:08:25,332][20347] Updated weights for policy 0, policy_version 1803542 (0.0005) [2023-12-27 18:08:25,389][20347] Updated weights for policy 0, policy_version 1803552 (0.0006) [2023-12-27 18:08:25,620][20381] Updated weights for policy 1, policy_version 1801948 (0.0007) [2023-12-27 18:08:25,684][20381] Updated weights for policy 1, policy_version 1801958 (0.0006) [2023-12-27 18:08:25,740][20381] Updated weights for policy 1, policy_version 1801968 (0.0007) [2023-12-27 18:08:26,053][20347] Updated weights for policy 0, policy_version 1803562 (0.0010) [2023-12-27 18:08:26,109][20347] Updated weights for policy 0, policy_version 1803572 (0.0010) [2023-12-27 18:08:26,168][20347] Updated weights for policy 0, policy_version 1803582 (0.0010) [2023-12-27 18:08:26,412][20381] Updated weights for policy 1, policy_version 1801978 (0.0007) [2023-12-27 18:08:26,469][20381] Updated weights for policy 1, policy_version 1801988 (0.0007) [2023-12-27 18:08:26,520][20381] Updated weights for policy 1, policy_version 1801998 (0.0007) [2023-12-27 18:08:26,580][20381] Updated weights for policy 1, policy_version 1802008 (0.0007) [2023-12-27 18:08:26,842][20347] Updated weights for policy 0, policy_version 1803592 (0.0007) [2023-12-27 18:08:26,894][20347] Updated weights for policy 0, policy_version 1803602 (0.0005) [2023-12-27 18:08:26,943][20347] Updated weights for policy 0, policy_version 1803612 (0.0005) [2023-12-27 18:08:27,291][20381] Updated weights for policy 1, policy_version 1802018 (0.0007) [2023-12-27 18:08:27,343][20381] Updated weights for policy 1, policy_version 1802028 (0.0007) [2023-12-27 18:08:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 923172864. Throughput: 0: 10393.8, 1: 10024.9. Samples: 923167152. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:27,363][19170] Avg episode reward: [(0, '951.080'), (1, '970.840')] [2023-12-27 18:08:27,388][20381] Updated weights for policy 1, policy_version 1802038 (0.0008) [2023-12-27 18:08:27,534][20347] Updated weights for policy 0, policy_version 1803622 (0.0005) [2023-12-27 18:08:27,584][20347] Updated weights for policy 0, policy_version 1803632 (0.0005) [2023-12-27 18:08:27,640][20347] Updated weights for policy 0, policy_version 1803642 (0.0005) [2023-12-27 18:08:28,100][20381] Updated weights for policy 1, policy_version 1802048 (0.0007) [2023-12-27 18:08:28,149][20381] Updated weights for policy 1, policy_version 1802058 (0.0007) [2023-12-27 18:08:28,209][20381] Updated weights for policy 1, policy_version 1802068 (0.0006) [2023-12-27 18:08:28,253][20347] Updated weights for policy 0, policy_version 1803652 (0.0007) [2023-12-27 18:08:28,301][20347] Updated weights for policy 0, policy_version 1803662 (0.0010) [2023-12-27 18:08:28,363][20347] Updated weights for policy 0, policy_version 1803672 (0.0010) [2023-12-27 18:08:28,823][20381] Updated weights for policy 1, policy_version 1802078 (0.0006) [2023-12-27 18:08:28,881][20381] Updated weights for policy 1, policy_version 1802088 (0.0005) [2023-12-27 18:08:28,937][20381] Updated weights for policy 1, policy_version 1802098 (0.0005) [2023-12-27 18:08:29,048][20347] Updated weights for policy 0, policy_version 1803682 (0.0011) [2023-12-27 18:08:29,099][20347] Updated weights for policy 0, policy_version 1803692 (0.0010) [2023-12-27 18:08:29,157][20347] Updated weights for policy 0, policy_version 1803702 (0.0008) [2023-12-27 18:08:29,216][20347] Updated weights for policy 0, policy_version 1803712 (0.0008) [2023-12-27 18:08:29,618][20381] Updated weights for policy 1, policy_version 1802108 (0.0007) [2023-12-27 18:08:29,680][20381] Updated weights for policy 1, policy_version 1802118 (0.0008) [2023-12-27 18:08:29,735][20381] Updated weights for policy 1, policy_version 1802128 (0.0008) [2023-12-27 18:08:29,871][20347] Updated weights for policy 0, policy_version 1803722 (0.0007) [2023-12-27 18:08:29,949][20347] Updated weights for policy 0, policy_version 1803732 (0.0006) [2023-12-27 18:08:30,008][20347] Updated weights for policy 0, policy_version 1803742 (0.0006) [2023-12-27 18:08:30,473][20381] Updated weights for policy 1, policy_version 1802138 (0.0007) [2023-12-27 18:08:30,524][20381] Updated weights for policy 1, policy_version 1802148 (0.0008) [2023-12-27 18:08:30,580][20381] Updated weights for policy 1, policy_version 1802158 (0.0008) [2023-12-27 18:08:30,603][20347] Updated weights for policy 0, policy_version 1803752 (0.0005) [2023-12-27 18:08:30,631][20381] Updated weights for policy 1, policy_version 1802168 (0.0008) [2023-12-27 18:08:30,652][20347] Updated weights for policy 0, policy_version 1803762 (0.0005) [2023-12-27 18:08:30,710][20347] Updated weights for policy 0, policy_version 1803772 (0.0005) [2023-12-27 18:08:31,333][20381] Updated weights for policy 1, policy_version 1802178 (0.0008) [2023-12-27 18:08:31,398][20381] Updated weights for policy 1, policy_version 1802188 (0.0012) [2023-12-27 18:08:31,399][20347] Updated weights for policy 0, policy_version 1803782 (0.0007) [2023-12-27 18:08:31,457][20381] Updated weights for policy 1, policy_version 1802198 (0.0007) [2023-12-27 18:08:31,459][20347] Updated weights for policy 0, policy_version 1803792 (0.0008) [2023-12-27 18:08:31,508][20347] Updated weights for policy 0, policy_version 1803802 (0.0006) [2023-12-27 18:08:32,118][20347] Updated weights for policy 0, policy_version 1803812 (0.0005) [2023-12-27 18:08:32,150][20381] Updated weights for policy 1, policy_version 1802208 (0.0005) [2023-12-27 18:08:32,175][20347] Updated weights for policy 0, policy_version 1803822 (0.0006) [2023-12-27 18:08:32,207][20381] Updated weights for policy 1, policy_version 1802218 (0.0006) [2023-12-27 18:08:32,227][20347] Updated weights for policy 0, policy_version 1803832 (0.0005) [2023-12-27 18:08:32,264][20381] Updated weights for policy 1, policy_version 1802228 (0.0007) [2023-12-27 18:08:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 923287552. Throughput: 0: 10459.6, 1: 10067.1. Samples: 923292492. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:32,363][19170] Avg episode reward: [(0, '941.230'), (1, '980.480')] [2023-12-27 18:08:32,822][20347] Updated weights for policy 0, policy_version 1803842 (0.0006) [2023-12-27 18:08:32,877][20347] Updated weights for policy 0, policy_version 1803852 (0.0007) [2023-12-27 18:08:32,877][20381] Updated weights for policy 1, policy_version 1802238 (0.0007) [2023-12-27 18:08:32,935][20347] Updated weights for policy 0, policy_version 1803862 (0.0006) [2023-12-27 18:08:32,943][20381] Updated weights for policy 1, policy_version 1802248 (0.0009) [2023-12-27 18:08:32,995][20381] Updated weights for policy 1, policy_version 1802258 (0.0011) [2023-12-27 18:08:32,998][20347] Updated weights for policy 0, policy_version 1803872 (0.0005) [2023-12-27 18:08:33,607][20347] Updated weights for policy 0, policy_version 1803882 (0.0007) [2023-12-27 18:08:33,633][20381] Updated weights for policy 1, policy_version 1802268 (0.0009) [2023-12-27 18:08:33,670][20347] Updated weights for policy 0, policy_version 1803892 (0.0007) [2023-12-27 18:08:33,683][20381] Updated weights for policy 1, policy_version 1802278 (0.0005) [2023-12-27 18:08:33,727][20347] Updated weights for policy 0, policy_version 1803902 (0.0007) [2023-12-27 18:08:33,729][20381] Updated weights for policy 1, policy_version 1802288 (0.0005) [2023-12-27 18:08:34,410][20381] Updated weights for policy 1, policy_version 1802298 (0.0007) [2023-12-27 18:08:34,415][20347] Updated weights for policy 0, policy_version 1803912 (0.0008) [2023-12-27 18:08:34,471][20381] Updated weights for policy 1, policy_version 1802308 (0.0006) [2023-12-27 18:08:34,472][20347] Updated weights for policy 0, policy_version 1803922 (0.0006) [2023-12-27 18:08:34,528][20381] Updated weights for policy 1, policy_version 1802318 (0.0005) [2023-12-27 18:08:34,529][20347] Updated weights for policy 0, policy_version 1803932 (0.0007) [2023-12-27 18:08:34,586][20381] Updated weights for policy 1, policy_version 1802328 (0.0007) [2023-12-27 18:08:35,228][20347] Updated weights for policy 0, policy_version 1803942 (0.0007) [2023-12-27 18:08:35,234][20381] Updated weights for policy 1, policy_version 1802338 (0.0005) [2023-12-27 18:08:35,277][20347] Updated weights for policy 0, policy_version 1803952 (0.0006) [2023-12-27 18:08:35,284][20381] Updated weights for policy 1, policy_version 1802348 (0.0006) [2023-12-27 18:08:35,326][20347] Updated weights for policy 0, policy_version 1803962 (0.0006) [2023-12-27 18:08:35,336][20381] Updated weights for policy 1, policy_version 1802358 (0.0008) [2023-12-27 18:08:35,953][20347] Updated weights for policy 0, policy_version 1803972 (0.0007) [2023-12-27 18:08:35,991][20381] Updated weights for policy 1, policy_version 1802368 (0.0010) [2023-12-27 18:08:36,008][20347] Updated weights for policy 0, policy_version 1803982 (0.0005) [2023-12-27 18:08:36,046][20381] Updated weights for policy 1, policy_version 1802378 (0.0010) [2023-12-27 18:08:36,062][20347] Updated weights for policy 0, policy_version 1803992 (0.0005) [2023-12-27 18:08:36,095][20381] Updated weights for policy 1, policy_version 1802388 (0.0006) [2023-12-27 18:08:36,692][20347] Updated weights for policy 0, policy_version 1804002 (0.0006) [2023-12-27 18:08:36,739][20347] Updated weights for policy 0, policy_version 1804012 (0.0007) [2023-12-27 18:08:36,789][20347] Updated weights for policy 0, policy_version 1804022 (0.0007) [2023-12-27 18:08:36,803][20381] Updated weights for policy 1, policy_version 1802398 (0.0009) [2023-12-27 18:08:36,845][20347] Updated weights for policy 0, policy_version 1804032 (0.0005) [2023-12-27 18:08:36,855][20381] Updated weights for policy 1, policy_version 1802408 (0.0010) [2023-12-27 18:08:36,910][20381] Updated weights for policy 1, policy_version 1802418 (0.0010) [2023-12-27 18:08:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 923385856. Throughput: 0: 10451.6, 1: 10127.8. Samples: 923356004. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:37,363][19170] Avg episode reward: [(0, '970.660'), (1, '990.240')] [2023-12-27 18:08:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001802424_461471744.pth... [2023-12-27 18:08:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001804032_461914112.pth... [2023-12-27 18:08:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001802816_461602816.pth [2023-12-27 18:08:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001801240_461168640.pth [2023-12-27 18:08:37,556][20347] Updated weights for policy 0, policy_version 1804042 (0.0007) [2023-12-27 18:08:37,589][20381] Updated weights for policy 1, policy_version 1802428 (0.0011) [2023-12-27 18:08:37,613][20347] Updated weights for policy 0, policy_version 1804052 (0.0007) [2023-12-27 18:08:37,639][20381] Updated weights for policy 1, policy_version 1802438 (0.0010) [2023-12-27 18:08:37,675][20347] Updated weights for policy 0, policy_version 1804062 (0.0007) [2023-12-27 18:08:37,689][20381] Updated weights for policy 1, policy_version 1802448 (0.0011) [2023-12-27 18:08:38,371][20347] Updated weights for policy 0, policy_version 1804072 (0.0007) [2023-12-27 18:08:38,406][20381] Updated weights for policy 1, policy_version 1802458 (0.0010) [2023-12-27 18:08:38,425][20347] Updated weights for policy 0, policy_version 1804082 (0.0007) [2023-12-27 18:08:38,468][20381] Updated weights for policy 1, policy_version 1802468 (0.0006) [2023-12-27 18:08:38,477][20347] Updated weights for policy 0, policy_version 1804092 (0.0008) [2023-12-27 18:08:38,528][20381] Updated weights for policy 1, policy_version 1802478 (0.0008) [2023-12-27 18:08:38,590][20381] Updated weights for policy 1, policy_version 1802488 (0.0007) [2023-12-27 18:08:39,174][20347] Updated weights for policy 0, policy_version 1804102 (0.0007) [2023-12-27 18:08:39,235][20347] Updated weights for policy 0, policy_version 1804112 (0.0008) [2023-12-27 18:08:39,303][20347] Updated weights for policy 0, policy_version 1804122 (0.0009) [2023-12-27 18:08:39,313][20381] Updated weights for policy 1, policy_version 1802498 (0.0007) [2023-12-27 18:08:39,376][20381] Updated weights for policy 1, policy_version 1802508 (0.0008) [2023-12-27 18:08:39,445][20381] Updated weights for policy 1, policy_version 1802518 (0.0006) [2023-12-27 18:08:39,961][20347] Updated weights for policy 0, policy_version 1804132 (0.0011) [2023-12-27 18:08:40,023][20347] Updated weights for policy 0, policy_version 1804142 (0.0011) [2023-12-27 18:08:40,085][20347] Updated weights for policy 0, policy_version 1804152 (0.0011) [2023-12-27 18:08:40,091][20381] Updated weights for policy 1, policy_version 1802528 (0.0006) [2023-12-27 18:08:40,152][20381] Updated weights for policy 1, policy_version 1802538 (0.0006) [2023-12-27 18:08:40,210][20381] Updated weights for policy 1, policy_version 1802548 (0.0008) [2023-12-27 18:08:40,747][20347] Updated weights for policy 0, policy_version 1804162 (0.0011) [2023-12-27 18:08:40,801][20347] Updated weights for policy 0, policy_version 1804172 (0.0010) [2023-12-27 18:08:40,853][20347] Updated weights for policy 0, policy_version 1804182 (0.0009) [2023-12-27 18:08:40,902][20347] Updated weights for policy 0, policy_version 1804192 (0.0005) [2023-12-27 18:08:40,924][20381] Updated weights for policy 1, policy_version 1802558 (0.0009) [2023-12-27 18:08:40,975][20381] Updated weights for policy 1, policy_version 1802568 (0.0011) [2023-12-27 18:08:41,042][20381] Updated weights for policy 1, policy_version 1802578 (0.0010) [2023-12-27 18:08:41,675][20347] Updated weights for policy 0, policy_version 1804202 (0.0007) [2023-12-27 18:08:41,740][20347] Updated weights for policy 0, policy_version 1804212 (0.0007) [2023-12-27 18:08:41,802][20347] Updated weights for policy 0, policy_version 1804222 (0.0007) [2023-12-27 18:08:41,838][20381] Updated weights for policy 1, policy_version 1802588 (0.0010) [2023-12-27 18:08:41,894][20381] Updated weights for policy 1, policy_version 1802598 (0.0007) [2023-12-27 18:08:41,959][20381] Updated weights for policy 1, policy_version 1802608 (0.0008) [2023-12-27 18:08:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 923484160. Throughput: 0: 10455.5, 1: 10097.1. Samples: 923477336. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:08:42,363][19170] Avg episode reward: [(0, '970.600'), (1, '1000.000')] [2023-12-27 18:08:42,505][20347] Updated weights for policy 0, policy_version 1804232 (0.0008) [2023-12-27 18:08:42,558][20347] Updated weights for policy 0, policy_version 1804242 (0.0010) [2023-12-27 18:08:42,623][20347] Updated weights for policy 0, policy_version 1804252 (0.0008) [2023-12-27 18:08:42,686][20381] Updated weights for policy 1, policy_version 1802618 (0.0007) [2023-12-27 18:08:42,735][20381] Updated weights for policy 1, policy_version 1802628 (0.0007) [2023-12-27 18:08:42,795][20381] Updated weights for policy 1, policy_version 1802638 (0.0007) [2023-12-27 18:08:42,860][20381] Updated weights for policy 1, policy_version 1802648 (0.0008) [2023-12-27 18:08:43,205][20347] Updated weights for policy 0, policy_version 1804262 (0.0006) [2023-12-27 18:08:43,259][20347] Updated weights for policy 0, policy_version 1804272 (0.0007) [2023-12-27 18:08:43,308][20347] Updated weights for policy 0, policy_version 1804282 (0.0005) [2023-12-27 18:08:43,597][20381] Updated weights for policy 1, policy_version 1802658 (0.0008) [2023-12-27 18:08:43,652][20381] Updated weights for policy 1, policy_version 1802668 (0.0007) [2023-12-27 18:08:43,707][20381] Updated weights for policy 1, policy_version 1802678 (0.0008) [2023-12-27 18:08:43,858][20347] Updated weights for policy 0, policy_version 1804292 (0.0005) [2023-12-27 18:08:43,916][20347] Updated weights for policy 0, policy_version 1804302 (0.0005) [2023-12-27 18:08:43,974][20347] Updated weights for policy 0, policy_version 1804312 (0.0007) [2023-12-27 18:08:44,350][20381] Updated weights for policy 1, policy_version 1802688 (0.0006) [2023-12-27 18:08:44,409][20381] Updated weights for policy 1, policy_version 1802698 (0.0008) [2023-12-27 18:08:44,458][20381] Updated weights for policy 1, policy_version 1802708 (0.0005) [2023-12-27 18:08:44,505][20347] Updated weights for policy 0, policy_version 1804322 (0.0006) [2023-12-27 18:08:44,557][20347] Updated weights for policy 0, policy_version 1804332 (0.0006) [2023-12-27 18:08:44,609][20347] Updated weights for policy 0, policy_version 1804342 (0.0006) [2023-12-27 18:08:44,663][20347] Updated weights for policy 0, policy_version 1804352 (0.0006) [2023-12-27 18:08:45,155][20381] Updated weights for policy 1, policy_version 1802718 (0.0007) [2023-12-27 18:08:45,216][20381] Updated weights for policy 1, policy_version 1802728 (0.0008) [2023-12-27 18:08:45,282][20381] Updated weights for policy 1, policy_version 1802738 (0.0008) [2023-12-27 18:08:45,309][20347] Updated weights for policy 0, policy_version 1804362 (0.0011) [2023-12-27 18:08:45,366][20347] Updated weights for policy 0, policy_version 1804372 (0.0011) [2023-12-27 18:08:45,412][20347] Updated weights for policy 0, policy_version 1804382 (0.0010) [2023-12-27 18:08:45,960][20381] Updated weights for policy 1, policy_version 1802748 (0.0008) [2023-12-27 18:08:46,015][20381] Updated weights for policy 1, policy_version 1802758 (0.0008) [2023-12-27 18:08:46,068][20381] Updated weights for policy 1, policy_version 1802768 (0.0008) [2023-12-27 18:08:46,117][20347] Updated weights for policy 0, policy_version 1804392 (0.0010) [2023-12-27 18:08:46,172][20347] Updated weights for policy 0, policy_version 1804402 (0.0010) [2023-12-27 18:08:46,221][20347] Updated weights for policy 0, policy_version 1804412 (0.0010) [2023-12-27 18:08:46,752][20381] Updated weights for policy 1, policy_version 1802778 (0.0006) [2023-12-27 18:08:46,811][20381] Updated weights for policy 1, policy_version 1802788 (0.0007) [2023-12-27 18:08:46,871][20381] Updated weights for policy 1, policy_version 1802798 (0.0007) [2023-12-27 18:08:46,895][20347] Updated weights for policy 0, policy_version 1804422 (0.0006) [2023-12-27 18:08:46,931][20381] Updated weights for policy 1, policy_version 1802808 (0.0008) [2023-12-27 18:08:46,950][20347] Updated weights for policy 0, policy_version 1804432 (0.0008) [2023-12-27 18:08:47,011][20347] Updated weights for policy 0, policy_version 1804442 (0.0006) [2023-12-27 18:08:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 923590656. Throughput: 0: 10438.4, 1: 10065.4. Samples: 923600012. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:08:47,363][19170] Avg episode reward: [(0, '970.570'), (1, '1000.000')] [2023-12-27 18:08:47,633][20347] Updated weights for policy 0, policy_version 1804452 (0.0006) [2023-12-27 18:08:47,657][20381] Updated weights for policy 1, policy_version 1802818 (0.0007) [2023-12-27 18:08:47,686][20347] Updated weights for policy 0, policy_version 1804462 (0.0008) [2023-12-27 18:08:47,707][20381] Updated weights for policy 1, policy_version 1802828 (0.0006) [2023-12-27 18:08:47,741][20347] Updated weights for policy 0, policy_version 1804472 (0.0006) [2023-12-27 18:08:47,753][20381] Updated weights for policy 1, policy_version 1802838 (0.0007) [2023-12-27 18:08:48,386][20347] Updated weights for policy 0, policy_version 1804482 (0.0005) [2023-12-27 18:08:48,442][20347] Updated weights for policy 0, policy_version 1804492 (0.0008) [2023-12-27 18:08:48,495][20381] Updated weights for policy 1, policy_version 1802848 (0.0009) [2023-12-27 18:08:48,502][20347] Updated weights for policy 0, policy_version 1804502 (0.0008) [2023-12-27 18:08:48,554][20381] Updated weights for policy 1, policy_version 1802858 (0.0006) [2023-12-27 18:08:48,555][20347] Updated weights for policy 0, policy_version 1804512 (0.0006) [2023-12-27 18:08:48,616][20381] Updated weights for policy 1, policy_version 1802868 (0.0008) [2023-12-27 18:08:49,152][20347] Updated weights for policy 0, policy_version 1804522 (0.0005) [2023-12-27 18:08:49,212][20347] Updated weights for policy 0, policy_version 1804532 (0.0005) [2023-12-27 18:08:49,276][20347] Updated weights for policy 0, policy_version 1804542 (0.0008) [2023-12-27 18:08:49,388][20381] Updated weights for policy 1, policy_version 1802878 (0.0009) [2023-12-27 18:08:49,438][20381] Updated weights for policy 1, policy_version 1802888 (0.0009) [2023-12-27 18:08:49,492][20381] Updated weights for policy 1, policy_version 1802898 (0.0007) [2023-12-27 18:08:49,924][20347] Updated weights for policy 0, policy_version 1804552 (0.0007) [2023-12-27 18:08:49,990][20347] Updated weights for policy 0, policy_version 1804562 (0.0006) [2023-12-27 18:08:50,054][20347] Updated weights for policy 0, policy_version 1804572 (0.0006) [2023-12-27 18:08:50,200][20381] Updated weights for policy 1, policy_version 1802908 (0.0007) [2023-12-27 18:08:50,261][20381] Updated weights for policy 1, policy_version 1802918 (0.0006) [2023-12-27 18:08:50,327][20381] Updated weights for policy 1, policy_version 1802928 (0.0005) [2023-12-27 18:08:50,678][20347] Updated weights for policy 0, policy_version 1804582 (0.0007) [2023-12-27 18:08:50,737][20347] Updated weights for policy 0, policy_version 1804592 (0.0008) [2023-12-27 18:08:50,798][20347] Updated weights for policy 0, policy_version 1804602 (0.0006) [2023-12-27 18:08:50,872][20381] Updated weights for policy 1, policy_version 1802938 (0.0006) [2023-12-27 18:08:50,924][20381] Updated weights for policy 1, policy_version 1802948 (0.0008) [2023-12-27 18:08:50,976][20381] Updated weights for policy 1, policy_version 1802958 (0.0008) [2023-12-27 18:08:51,028][20381] Updated weights for policy 1, policy_version 1802968 (0.0008) [2023-12-27 18:08:51,474][20347] Updated weights for policy 0, policy_version 1804612 (0.0006) [2023-12-27 18:08:51,540][20347] Updated weights for policy 0, policy_version 1804622 (0.0006) [2023-12-27 18:08:51,604][20347] Updated weights for policy 0, policy_version 1804632 (0.0006) [2023-12-27 18:08:51,826][20381] Updated weights for policy 1, policy_version 1802978 (0.0008) [2023-12-27 18:08:51,879][20381] Updated weights for policy 1, policy_version 1802988 (0.0008) [2023-12-27 18:08:51,938][20381] Updated weights for policy 1, policy_version 1802998 (0.0007) [2023-12-27 18:08:52,238][20347] Updated weights for policy 0, policy_version 1804642 (0.0007) [2023-12-27 18:08:52,300][20347] Updated weights for policy 0, policy_version 1804652 (0.0008) [2023-12-27 18:08:52,357][20347] Updated weights for policy 0, policy_version 1804662 (0.0008) [2023-12-27 18:08:52,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 923688960. Throughput: 0: 10468.0, 1: 10001.7. Samples: 923661800. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:08:52,363][19170] Avg episode reward: [(0, '960.980'), (1, '990.240')] [2023-12-27 18:08:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001803000_461619200.pth... [2023-12-27 18:08:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001801848_461324288.pth [2023-12-27 18:08:52,417][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001804672_462077952.pth... [2023-12-27 18:08:52,420][20347] Updated weights for policy 0, policy_version 1804672 (0.0008) [2023-12-27 18:08:52,420][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001803424_461758464.pth [2023-12-27 18:08:52,709][20381] Updated weights for policy 1, policy_version 1803008 (0.0008) [2023-12-27 18:08:52,778][20381] Updated weights for policy 1, policy_version 1803018 (0.0008) [2023-12-27 18:08:52,840][20381] Updated weights for policy 1, policy_version 1803028 (0.0007) [2023-12-27 18:08:53,151][20347] Updated weights for policy 0, policy_version 1804682 (0.0008) [2023-12-27 18:08:53,205][20347] Updated weights for policy 0, policy_version 1804692 (0.0007) [2023-12-27 18:08:53,256][20347] Updated weights for policy 0, policy_version 1804702 (0.0007) [2023-12-27 18:08:53,436][20381] Updated weights for policy 1, policy_version 1803038 (0.0008) [2023-12-27 18:08:53,494][20381] Updated weights for policy 1, policy_version 1803048 (0.0008) [2023-12-27 18:08:53,547][20381] Updated weights for policy 1, policy_version 1803058 (0.0008) [2023-12-27 18:08:53,953][20347] Updated weights for policy 0, policy_version 1804712 (0.0007) [2023-12-27 18:08:54,001][20347] Updated weights for policy 0, policy_version 1804722 (0.0008) [2023-12-27 18:08:54,056][20347] Updated weights for policy 0, policy_version 1804732 (0.0007) [2023-12-27 18:08:54,218][20381] Updated weights for policy 1, policy_version 1803068 (0.0007) [2023-12-27 18:08:54,274][20381] Updated weights for policy 1, policy_version 1803078 (0.0008) [2023-12-27 18:08:54,326][20381] Updated weights for policy 1, policy_version 1803088 (0.0008) [2023-12-27 18:08:54,771][20347] Updated weights for policy 0, policy_version 1804742 (0.0007) [2023-12-27 18:08:54,825][20347] Updated weights for policy 0, policy_version 1804752 (0.0008) [2023-12-27 18:08:54,882][20347] Updated weights for policy 0, policy_version 1804762 (0.0007) [2023-12-27 18:08:55,024][20381] Updated weights for policy 1, policy_version 1803098 (0.0008) [2023-12-27 18:08:55,081][20381] Updated weights for policy 1, policy_version 1803108 (0.0008) [2023-12-27 18:08:55,140][20381] Updated weights for policy 1, policy_version 1803118 (0.0006) [2023-12-27 18:08:55,193][20381] Updated weights for policy 1, policy_version 1803128 (0.0008) [2023-12-27 18:08:55,593][20347] Updated weights for policy 0, policy_version 1804772 (0.0008) [2023-12-27 18:08:55,654][20347] Updated weights for policy 0, policy_version 1804782 (0.0008) [2023-12-27 18:08:55,712][20347] Updated weights for policy 0, policy_version 1804792 (0.0007) [2023-12-27 18:08:55,798][20381] Updated weights for policy 1, policy_version 1803138 (0.0011) [2023-12-27 18:08:55,856][20381] Updated weights for policy 1, policy_version 1803148 (0.0010) [2023-12-27 18:08:55,908][20381] Updated weights for policy 1, policy_version 1803158 (0.0010) [2023-12-27 18:08:56,442][20347] Updated weights for policy 0, policy_version 1804802 (0.0006) [2023-12-27 18:08:56,498][20347] Updated weights for policy 0, policy_version 1804812 (0.0007) [2023-12-27 18:08:56,498][20381] Updated weights for policy 1, policy_version 1803168 (0.0006) [2023-12-27 18:08:56,556][20347] Updated weights for policy 0, policy_version 1804822 (0.0008) [2023-12-27 18:08:56,557][20381] Updated weights for policy 1, policy_version 1803178 (0.0006) [2023-12-27 18:08:56,602][20347] Updated weights for policy 0, policy_version 1804832 (0.0007) [2023-12-27 18:08:56,604][20381] Updated weights for policy 1, policy_version 1803188 (0.0010) [2023-12-27 18:08:57,214][20381] Updated weights for policy 1, policy_version 1803198 (0.0009) [2023-12-27 18:08:57,277][20381] Updated weights for policy 1, policy_version 1803208 (0.0010) [2023-12-27 18:08:57,316][20347] Updated weights for policy 0, policy_version 1804842 (0.0006) [2023-12-27 18:08:57,332][20381] Updated weights for policy 1, policy_version 1803218 (0.0011) [2023-12-27 18:08:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 923787264. Throughput: 0: 10449.6, 1: 10018.8. Samples: 923784024. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:08:57,363][19170] Avg episode reward: [(0, '961.020'), (1, '990.240')] [2023-12-27 18:08:57,372][20347] Updated weights for policy 0, policy_version 1804852 (0.0006) [2023-12-27 18:08:57,426][20347] Updated weights for policy 0, policy_version 1804862 (0.0007) [2023-12-27 18:08:57,993][20381] Updated weights for policy 1, policy_version 1803228 (0.0011) [2023-12-27 18:08:58,046][20347] Updated weights for policy 0, policy_version 1804872 (0.0006) [2023-12-27 18:08:58,051][20381] Updated weights for policy 1, policy_version 1803238 (0.0006) [2023-12-27 18:08:58,097][20347] Updated weights for policy 0, policy_version 1804882 (0.0005) [2023-12-27 18:08:58,104][20381] Updated weights for policy 1, policy_version 1803248 (0.0005) [2023-12-27 18:08:58,149][20347] Updated weights for policy 0, policy_version 1804892 (0.0009) [2023-12-27 18:08:58,776][20381] Updated weights for policy 1, policy_version 1803258 (0.0006) [2023-12-27 18:08:58,842][20381] Updated weights for policy 1, policy_version 1803268 (0.0010) [2023-12-27 18:08:58,909][20381] Updated weights for policy 1, policy_version 1803278 (0.0009) [2023-12-27 18:08:58,948][20347] Updated weights for policy 0, policy_version 1804902 (0.0008) [2023-12-27 18:08:58,969][20381] Updated weights for policy 1, policy_version 1803288 (0.0009) [2023-12-27 18:08:59,002][20347] Updated weights for policy 0, policy_version 1804912 (0.0007) [2023-12-27 18:08:59,058][20347] Updated weights for policy 0, policy_version 1804922 (0.0007) [2023-12-27 18:08:59,664][20381] Updated weights for policy 1, policy_version 1803298 (0.0011) [2023-12-27 18:08:59,725][20381] Updated weights for policy 1, policy_version 1803308 (0.0011) [2023-12-27 18:08:59,784][20381] Updated weights for policy 1, policy_version 1803318 (0.0010) [2023-12-27 18:08:59,790][20347] Updated weights for policy 0, policy_version 1804932 (0.0008) [2023-12-27 18:08:59,847][20347] Updated weights for policy 0, policy_version 1804942 (0.0007) [2023-12-27 18:08:59,915][20347] Updated weights for policy 0, policy_version 1804952 (0.0008) [2023-12-27 18:09:00,457][20381] Updated weights for policy 1, policy_version 1803328 (0.0010) [2023-12-27 18:09:00,516][20381] Updated weights for policy 1, policy_version 1803338 (0.0010) [2023-12-27 18:09:00,579][20381] Updated weights for policy 1, policy_version 1803348 (0.0010) [2023-12-27 18:09:00,632][20347] Updated weights for policy 0, policy_version 1804962 (0.0007) [2023-12-27 18:09:00,683][20347] Updated weights for policy 0, policy_version 1804972 (0.0007) [2023-12-27 18:09:00,733][20347] Updated weights for policy 0, policy_version 1804982 (0.0007) [2023-12-27 18:09:00,787][20347] Updated weights for policy 0, policy_version 1804992 (0.0007) [2023-12-27 18:09:01,297][20381] Updated weights for policy 1, policy_version 1803358 (0.0010) [2023-12-27 18:09:01,357][20381] Updated weights for policy 1, policy_version 1803368 (0.0010) [2023-12-27 18:09:01,424][20381] Updated weights for policy 1, policy_version 1803378 (0.0010) [2023-12-27 18:09:01,460][20347] Updated weights for policy 0, policy_version 1805002 (0.0005) [2023-12-27 18:09:01,509][20347] Updated weights for policy 0, policy_version 1805012 (0.0005) [2023-12-27 18:09:01,577][20347] Updated weights for policy 0, policy_version 1805022 (0.0007) [2023-12-27 18:09:02,086][20381] Updated weights for policy 1, policy_version 1803388 (0.0008) [2023-12-27 18:09:02,153][20381] Updated weights for policy 1, policy_version 1803398 (0.0005) [2023-12-27 18:09:02,209][20381] Updated weights for policy 1, policy_version 1803408 (0.0007) [2023-12-27 18:09:02,311][20347] Updated weights for policy 0, policy_version 1805032 (0.0008) [2023-12-27 18:09:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 923893760. Throughput: 0: 10375.1, 1: 10064.7. Samples: 923905236. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:02,363][19170] Avg episode reward: [(0, '970.610'), (1, '1000.000')] [2023-12-27 18:09:02,373][20347] Updated weights for policy 0, policy_version 1805042 (0.0008) [2023-12-27 18:09:02,437][20347] Updated weights for policy 0, policy_version 1805052 (0.0005) [2023-12-27 18:09:02,787][20381] Updated weights for policy 1, policy_version 1803418 (0.0006) [2023-12-27 18:09:02,848][20381] Updated weights for policy 1, policy_version 1803428 (0.0011) [2023-12-27 18:09:02,900][20381] Updated weights for policy 1, policy_version 1803438 (0.0010) [2023-12-27 18:09:02,953][20381] Updated weights for policy 1, policy_version 1803448 (0.0008) [2023-12-27 18:09:03,128][20347] Updated weights for policy 0, policy_version 1805062 (0.0007) [2023-12-27 18:09:03,174][20347] Updated weights for policy 0, policy_version 1805072 (0.0007) [2023-12-27 18:09:03,225][20347] Updated weights for policy 0, policy_version 1805082 (0.0006) [2023-12-27 18:09:03,530][20381] Updated weights for policy 1, policy_version 1803458 (0.0008) [2023-12-27 18:09:03,585][20381] Updated weights for policy 1, policy_version 1803468 (0.0008) [2023-12-27 18:09:03,648][20381] Updated weights for policy 1, policy_version 1803478 (0.0008) [2023-12-27 18:09:03,904][20347] Updated weights for policy 0, policy_version 1805092 (0.0007) [2023-12-27 18:09:03,964][20347] Updated weights for policy 0, policy_version 1805102 (0.0011) [2023-12-27 18:09:04,033][20347] Updated weights for policy 0, policy_version 1805112 (0.0011) [2023-12-27 18:09:04,246][20381] Updated weights for policy 1, policy_version 1803488 (0.0008) [2023-12-27 18:09:04,314][20381] Updated weights for policy 1, policy_version 1803498 (0.0007) [2023-12-27 18:09:04,383][20381] Updated weights for policy 1, policy_version 1803508 (0.0007) [2023-12-27 18:09:04,718][20347] Updated weights for policy 0, policy_version 1805122 (0.0010) [2023-12-27 18:09:04,772][20347] Updated weights for policy 0, policy_version 1805132 (0.0010) [2023-12-27 18:09:04,829][20347] Updated weights for policy 0, policy_version 1805142 (0.0010) [2023-12-27 18:09:04,883][20347] Updated weights for policy 0, policy_version 1805152 (0.0010) [2023-12-27 18:09:04,930][20381] Updated weights for policy 1, policy_version 1803518 (0.0008) [2023-12-27 18:09:04,993][20381] Updated weights for policy 1, policy_version 1803528 (0.0007) [2023-12-27 18:09:05,049][20381] Updated weights for policy 1, policy_version 1803538 (0.0009) [2023-12-27 18:09:05,562][20347] Updated weights for policy 0, policy_version 1805162 (0.0010) [2023-12-27 18:09:05,609][20347] Updated weights for policy 0, policy_version 1805172 (0.0010) [2023-12-27 18:09:05,663][20347] Updated weights for policy 0, policy_version 1805182 (0.0010) [2023-12-27 18:09:05,701][20381] Updated weights for policy 1, policy_version 1803548 (0.0009) [2023-12-27 18:09:05,749][20381] Updated weights for policy 1, policy_version 1803558 (0.0007) [2023-12-27 18:09:05,809][20381] Updated weights for policy 1, policy_version 1803568 (0.0007) [2023-12-27 18:09:06,368][20347] Updated weights for policy 0, policy_version 1805192 (0.0011) [2023-12-27 18:09:06,430][20347] Updated weights for policy 0, policy_version 1805202 (0.0011) [2023-12-27 18:09:06,443][20381] Updated weights for policy 1, policy_version 1803578 (0.0007) [2023-12-27 18:09:06,486][20347] Updated weights for policy 0, policy_version 1805212 (0.0010) [2023-12-27 18:09:06,504][20381] Updated weights for policy 1, policy_version 1803588 (0.0006) [2023-12-27 18:09:06,561][20381] Updated weights for policy 1, policy_version 1803598 (0.0008) [2023-12-27 18:09:06,618][20381] Updated weights for policy 1, policy_version 1803608 (0.0008) [2023-12-27 18:09:07,180][20347] Updated weights for policy 0, policy_version 1805222 (0.0007) [2023-12-27 18:09:07,242][20347] Updated weights for policy 0, policy_version 1805232 (0.0010) [2023-12-27 18:09:07,295][20381] Updated weights for policy 1, policy_version 1803618 (0.0006) [2023-12-27 18:09:07,299][20347] Updated weights for policy 0, policy_version 1805242 (0.0010) [2023-12-27 18:09:07,356][20381] Updated weights for policy 1, policy_version 1803628 (0.0006) [2023-12-27 18:09:07,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 924000256. Throughput: 0: 10354.8, 1: 10150.5. Samples: 923969312. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:07,363][19170] Avg episode reward: [(0, '931.430'), (1, '1000.000')] [2023-12-27 18:09:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001805248_462225408.pth... [2023-12-27 18:09:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001804032_461914112.pth [2023-12-27 18:09:07,421][20381] Updated weights for policy 1, policy_version 1803638 (0.0006) [2023-12-27 18:09:07,434][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001803640_461783040.pth... [2023-12-27 18:09:07,438][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001802424_461471744.pth [2023-12-27 18:09:07,905][20347] Updated weights for policy 0, policy_version 1805252 (0.0009) [2023-12-27 18:09:07,965][20347] Updated weights for policy 0, policy_version 1805262 (0.0006) [2023-12-27 18:09:08,034][20347] Updated weights for policy 0, policy_version 1805272 (0.0009) [2023-12-27 18:09:08,121][20381] Updated weights for policy 1, policy_version 1803648 (0.0007) [2023-12-27 18:09:08,173][20381] Updated weights for policy 1, policy_version 1803658 (0.0008) [2023-12-27 18:09:08,231][20381] Updated weights for policy 1, policy_version 1803668 (0.0006) [2023-12-27 18:09:08,700][20347] Updated weights for policy 0, policy_version 1805282 (0.0010) [2023-12-27 18:09:08,761][20347] Updated weights for policy 0, policy_version 1805292 (0.0006) [2023-12-27 18:09:08,822][20347] Updated weights for policy 0, policy_version 1805302 (0.0006) [2023-12-27 18:09:08,883][20347] Updated weights for policy 0, policy_version 1805312 (0.0007) [2023-12-27 18:09:08,929][20381] Updated weights for policy 1, policy_version 1803678 (0.0006) [2023-12-27 18:09:08,990][20381] Updated weights for policy 1, policy_version 1803688 (0.0009) [2023-12-27 18:09:09,047][20381] Updated weights for policy 1, policy_version 1803698 (0.0011) [2023-12-27 18:09:09,593][20347] Updated weights for policy 0, policy_version 1805322 (0.0008) [2023-12-27 18:09:09,655][20347] Updated weights for policy 0, policy_version 1805332 (0.0008) [2023-12-27 18:09:09,714][20347] Updated weights for policy 0, policy_version 1805342 (0.0008) [2023-12-27 18:09:09,780][20381] Updated weights for policy 1, policy_version 1803708 (0.0011) [2023-12-27 18:09:09,840][20381] Updated weights for policy 1, policy_version 1803718 (0.0010) [2023-12-27 18:09:09,905][20381] Updated weights for policy 1, policy_version 1803729 (0.0009) [2023-12-27 18:09:10,448][20347] Updated weights for policy 0, policy_version 1805352 (0.0008) [2023-12-27 18:09:10,510][20347] Updated weights for policy 0, policy_version 1805362 (0.0007) [2023-12-27 18:09:10,566][20347] Updated weights for policy 0, policy_version 1805372 (0.0007) [2023-12-27 18:09:10,638][20381] Updated weights for policy 1, policy_version 1803739 (0.0008) [2023-12-27 18:09:10,694][20381] Updated weights for policy 1, policy_version 1803749 (0.0005) [2023-12-27 18:09:10,724][20292] KL-divergence is very high: 154.0723 [2023-12-27 18:09:10,755][20381] Updated weights for policy 1, policy_version 1803759 (0.0008) [2023-12-27 18:09:10,773][20292] KL-divergence is very high: 142.3770 [2023-12-27 18:09:11,341][20347] Updated weights for policy 0, policy_version 1805382 (0.0008) [2023-12-27 18:09:11,416][20347] Updated weights for policy 0, policy_version 1805392 (0.0008) [2023-12-27 18:09:11,475][20347] Updated weights for policy 0, policy_version 1805402 (0.0007) [2023-12-27 18:09:11,482][20381] Updated weights for policy 1, policy_version 1803769 (0.0007) [2023-12-27 18:09:11,542][20381] Updated weights for policy 1, policy_version 1803779 (0.0008) [2023-12-27 18:09:11,597][20381] Updated weights for policy 1, policy_version 1803789 (0.0008) [2023-12-27 18:09:11,658][20381] Updated weights for policy 1, policy_version 1803799 (0.0008) [2023-12-27 18:09:12,174][20347] Updated weights for policy 0, policy_version 1805412 (0.0007) [2023-12-27 18:09:12,231][20347] Updated weights for policy 0, policy_version 1805422 (0.0008) [2023-12-27 18:09:12,297][20347] Updated weights for policy 0, policy_version 1805432 (0.0008) [2023-12-27 18:09:12,360][20381] Updated weights for policy 1, policy_version 1803809 (0.0007) [2023-12-27 18:09:12,370][19170] Fps is (10 sec: 20464.6, 60 sec: 20477.5, 300 sec: 20521.1). Total num frames: 924098560. Throughput: 0: 10319.7, 1: 10180.7. Samples: 924089824. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:12,371][19170] Avg episode reward: [(0, '951.000'), (1, '990.240')] [2023-12-27 18:09:12,422][20381] Updated weights for policy 1, policy_version 1803819 (0.0008) [2023-12-27 18:09:12,474][20381] Updated weights for policy 1, policy_version 1803829 (0.0007) [2023-12-27 18:09:12,963][20347] Updated weights for policy 0, policy_version 1805442 (0.0007) [2023-12-27 18:09:13,012][20347] Updated weights for policy 0, policy_version 1805452 (0.0005) [2023-12-27 18:09:13,061][20347] Updated weights for policy 0, policy_version 1805462 (0.0007) [2023-12-27 18:09:13,096][20381] Updated weights for policy 1, policy_version 1803839 (0.0006) [2023-12-27 18:09:13,113][20347] Updated weights for policy 0, policy_version 1805472 (0.0006) [2023-12-27 18:09:13,154][20381] Updated weights for policy 1, policy_version 1803849 (0.0006) [2023-12-27 18:09:13,217][20381] Updated weights for policy 1, policy_version 1803859 (0.0008) [2023-12-27 18:09:13,755][20347] Updated weights for policy 0, policy_version 1805482 (0.0005) [2023-12-27 18:09:13,797][20381] Updated weights for policy 1, policy_version 1803869 (0.0006) [2023-12-27 18:09:13,814][20347] Updated weights for policy 0, policy_version 1805492 (0.0005) [2023-12-27 18:09:13,848][20381] Updated weights for policy 1, policy_version 1803879 (0.0005) [2023-12-27 18:09:13,870][20347] Updated weights for policy 0, policy_version 1805502 (0.0005) [2023-12-27 18:09:13,896][20381] Updated weights for policy 1, policy_version 1803889 (0.0005) [2023-12-27 18:09:14,427][20347] Updated weights for policy 0, policy_version 1805512 (0.0005) [2023-12-27 18:09:14,479][20347] Updated weights for policy 0, policy_version 1805522 (0.0007) [2023-12-27 18:09:14,524][20347] Updated weights for policy 0, policy_version 1805532 (0.0008) [2023-12-27 18:09:14,534][20381] Updated weights for policy 1, policy_version 1803899 (0.0007) [2023-12-27 18:09:14,589][20381] Updated weights for policy 1, policy_version 1803909 (0.0010) [2023-12-27 18:09:14,644][20381] Updated weights for policy 1, policy_version 1803919 (0.0009) [2023-12-27 18:09:15,146][20347] Updated weights for policy 0, policy_version 1805542 (0.0009) [2023-12-27 18:09:15,207][20347] Updated weights for policy 0, policy_version 1805552 (0.0010) [2023-12-27 18:09:15,272][20347] Updated weights for policy 0, policy_version 1805562 (0.0011) [2023-12-27 18:09:15,340][20381] Updated weights for policy 1, policy_version 1803929 (0.0008) [2023-12-27 18:09:15,395][20381] Updated weights for policy 1, policy_version 1803939 (0.0011) [2023-12-27 18:09:15,445][20381] Updated weights for policy 1, policy_version 1803949 (0.0009) [2023-12-27 18:09:15,501][20381] Updated weights for policy 1, policy_version 1803959 (0.0006) [2023-12-27 18:09:15,889][20347] Updated weights for policy 0, policy_version 1805572 (0.0010) [2023-12-27 18:09:15,948][20347] Updated weights for policy 0, policy_version 1805582 (0.0009) [2023-12-27 18:09:16,006][20347] Updated weights for policy 0, policy_version 1805592 (0.0010) [2023-12-27 18:09:16,109][20381] Updated weights for policy 1, policy_version 1803969 (0.0006) [2023-12-27 18:09:16,161][20381] Updated weights for policy 1, policy_version 1803979 (0.0007) [2023-12-27 18:09:16,209][20381] Updated weights for policy 1, policy_version 1803989 (0.0009) [2023-12-27 18:09:16,639][20347] Updated weights for policy 0, policy_version 1805602 (0.0010) [2023-12-27 18:09:16,695][20347] Updated weights for policy 0, policy_version 1805612 (0.0010) [2023-12-27 18:09:16,754][20347] Updated weights for policy 0, policy_version 1805622 (0.0009) [2023-12-27 18:09:16,805][20347] Updated weights for policy 0, policy_version 1805632 (0.0008) [2023-12-27 18:09:16,925][20381] Updated weights for policy 1, policy_version 1803999 (0.0007) [2023-12-27 18:09:16,984][20381] Updated weights for policy 1, policy_version 1804009 (0.0006) [2023-12-27 18:09:17,045][20381] Updated weights for policy 1, policy_version 1804019 (0.0006) [2023-12-27 18:09:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 924205056. Throughput: 0: 10287.8, 1: 10233.8. Samples: 924215964. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:17,363][19170] Avg episode reward: [(0, '970.560'), (1, '970.750')] [2023-12-27 18:09:17,494][20347] Updated weights for policy 0, policy_version 1805642 (0.0010) [2023-12-27 18:09:17,552][20347] Updated weights for policy 0, policy_version 1805652 (0.0010) [2023-12-27 18:09:17,604][20347] Updated weights for policy 0, policy_version 1805662 (0.0007) [2023-12-27 18:09:17,714][20381] Updated weights for policy 1, policy_version 1804029 (0.0008) [2023-12-27 18:09:17,772][20381] Updated weights for policy 1, policy_version 1804039 (0.0008) [2023-12-27 18:09:17,819][20381] Updated weights for policy 1, policy_version 1804049 (0.0007) [2023-12-27 18:09:18,287][20347] Updated weights for policy 0, policy_version 1805672 (0.0005) [2023-12-27 18:09:18,363][20347] Updated weights for policy 0, policy_version 1805682 (0.0006) [2023-12-27 18:09:18,430][20347] Updated weights for policy 0, policy_version 1805692 (0.0007) [2023-12-27 18:09:18,499][20381] Updated weights for policy 1, policy_version 1804059 (0.0007) [2023-12-27 18:09:18,558][20381] Updated weights for policy 1, policy_version 1804069 (0.0008) [2023-12-27 18:09:18,611][20381] Updated weights for policy 1, policy_version 1804079 (0.0008) [2023-12-27 18:09:19,057][20347] Updated weights for policy 0, policy_version 1805702 (0.0007) [2023-12-27 18:09:19,106][20347] Updated weights for policy 0, policy_version 1805712 (0.0007) [2023-12-27 18:09:19,159][20347] Updated weights for policy 0, policy_version 1805722 (0.0005) [2023-12-27 18:09:19,343][20381] Updated weights for policy 1, policy_version 1804089 (0.0008) [2023-12-27 18:09:19,404][20381] Updated weights for policy 1, policy_version 1804099 (0.0008) [2023-12-27 18:09:19,467][20381] Updated weights for policy 1, policy_version 1804109 (0.0008) [2023-12-27 18:09:19,532][20381] Updated weights for policy 1, policy_version 1804119 (0.0008) [2023-12-27 18:09:19,847][20347] Updated weights for policy 0, policy_version 1805732 (0.0006) [2023-12-27 18:09:19,911][20347] Updated weights for policy 0, policy_version 1805742 (0.0007) [2023-12-27 18:09:19,978][20347] Updated weights for policy 0, policy_version 1805752 (0.0007) [2023-12-27 18:09:20,272][20381] Updated weights for policy 1, policy_version 1804129 (0.0006) [2023-12-27 18:09:20,341][20381] Updated weights for policy 1, policy_version 1804139 (0.0007) [2023-12-27 18:09:20,409][20381] Updated weights for policy 1, policy_version 1804149 (0.0008) [2023-12-27 18:09:20,645][20347] Updated weights for policy 0, policy_version 1805762 (0.0006) [2023-12-27 18:09:20,700][20347] Updated weights for policy 0, policy_version 1805772 (0.0007) [2023-12-27 18:09:20,757][20347] Updated weights for policy 0, policy_version 1805782 (0.0007) [2023-12-27 18:09:20,810][20347] Updated weights for policy 0, policy_version 1805792 (0.0008) [2023-12-27 18:09:21,028][20381] Updated weights for policy 1, policy_version 1804159 (0.0008) [2023-12-27 18:09:21,090][20381] Updated weights for policy 1, policy_version 1804169 (0.0007) [2023-12-27 18:09:21,155][20381] Updated weights for policy 1, policy_version 1804179 (0.0009) [2023-12-27 18:09:21,546][20347] Updated weights for policy 0, policy_version 1805802 (0.0006) [2023-12-27 18:09:21,606][20347] Updated weights for policy 0, policy_version 1805812 (0.0008) [2023-12-27 18:09:21,675][20347] Updated weights for policy 0, policy_version 1805822 (0.0008) [2023-12-27 18:09:21,879][20381] Updated weights for policy 1, policy_version 1804189 (0.0008) [2023-12-27 18:09:21,942][20381] Updated weights for policy 1, policy_version 1804199 (0.0007) [2023-12-27 18:09:22,000][20381] Updated weights for policy 1, policy_version 1804209 (0.0007) [2023-12-27 18:09:22,362][19170] Fps is (10 sec: 20495.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 924303360. Throughput: 0: 10263.4, 1: 10184.4. Samples: 924276152. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:22,363][19170] Avg episode reward: [(0, '951.190'), (1, '980.510')] [2023-12-27 18:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001804216_461930496.pth... [2023-12-27 18:09:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001803000_461619200.pth [2023-12-27 18:09:22,388][20347] Updated weights for policy 0, policy_version 1805832 (0.0010) [2023-12-27 18:09:22,455][20347] Updated weights for policy 0, policy_version 1805842 (0.0009) [2023-12-27 18:09:22,521][20347] Updated weights for policy 0, policy_version 1805852 (0.0008) [2023-12-27 18:09:22,543][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001805856_462381056.pth... [2023-12-27 18:09:22,547][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001804672_462077952.pth [2023-12-27 18:09:22,628][20381] Updated weights for policy 1, policy_version 1804219 (0.0007) [2023-12-27 18:09:22,681][20381] Updated weights for policy 1, policy_version 1804229 (0.0007) [2023-12-27 18:09:22,735][20381] Updated weights for policy 1, policy_version 1804239 (0.0008) [2023-12-27 18:09:23,235][20347] Updated weights for policy 0, policy_version 1805862 (0.0008) [2023-12-27 18:09:23,296][20347] Updated weights for policy 0, policy_version 1805872 (0.0007) [2023-12-27 18:09:23,360][20347] Updated weights for policy 0, policy_version 1805882 (0.0008) [2023-12-27 18:09:23,427][20381] Updated weights for policy 1, policy_version 1804249 (0.0007) [2023-12-27 18:09:23,476][20381] Updated weights for policy 1, policy_version 1804259 (0.0008) [2023-12-27 18:09:23,526][20381] Updated weights for policy 1, policy_version 1804269 (0.0008) [2023-12-27 18:09:23,585][20381] Updated weights for policy 1, policy_version 1804279 (0.0008) [2023-12-27 18:09:24,024][20347] Updated weights for policy 0, policy_version 1805892 (0.0006) [2023-12-27 18:09:24,074][20347] Updated weights for policy 0, policy_version 1805902 (0.0008) [2023-12-27 18:09:24,130][20347] Updated weights for policy 0, policy_version 1805912 (0.0008) [2023-12-27 18:09:24,283][20381] Updated weights for policy 1, policy_version 1804289 (0.0008) [2023-12-27 18:09:24,342][20381] Updated weights for policy 1, policy_version 1804299 (0.0008) [2023-12-27 18:09:24,390][20381] Updated weights for policy 1, policy_version 1804309 (0.0008) [2023-12-27 18:09:24,748][20347] Updated weights for policy 0, policy_version 1805922 (0.0008) [2023-12-27 18:09:24,802][20347] Updated weights for policy 0, policy_version 1805932 (0.0005) [2023-12-27 18:09:24,859][20347] Updated weights for policy 0, policy_version 1805942 (0.0005) [2023-12-27 18:09:24,903][20347] Updated weights for policy 0, policy_version 1805952 (0.0005) [2023-12-27 18:09:25,153][20381] Updated weights for policy 1, policy_version 1804319 (0.0009) [2023-12-27 18:09:25,211][20381] Updated weights for policy 1, policy_version 1804329 (0.0008) [2023-12-27 18:09:25,273][20381] Updated weights for policy 1, policy_version 1804339 (0.0008) [2023-12-27 18:09:25,474][20347] Updated weights for policy 0, policy_version 1805962 (0.0005) [2023-12-27 18:09:25,527][20347] Updated weights for policy 0, policy_version 1805972 (0.0006) [2023-12-27 18:09:25,575][20347] Updated weights for policy 0, policy_version 1805982 (0.0005) [2023-12-27 18:09:25,995][20381] Updated weights for policy 1, policy_version 1804349 (0.0007) [2023-12-27 18:09:26,057][20381] Updated weights for policy 1, policy_version 1804359 (0.0009) [2023-12-27 18:09:26,114][20381] Updated weights for policy 1, policy_version 1804369 (0.0008) [2023-12-27 18:09:26,169][20347] Updated weights for policy 0, policy_version 1805992 (0.0006) [2023-12-27 18:09:26,216][20347] Updated weights for policy 0, policy_version 1806002 (0.0005) [2023-12-27 18:09:26,265][20347] Updated weights for policy 0, policy_version 1806012 (0.0006) [2023-12-27 18:09:26,818][20381] Updated weights for policy 1, policy_version 1804379 (0.0008) [2023-12-27 18:09:26,872][20381] Updated weights for policy 1, policy_version 1804389 (0.0009) [2023-12-27 18:09:26,920][20347] Updated weights for policy 0, policy_version 1806022 (0.0009) [2023-12-27 18:09:26,931][20381] Updated weights for policy 1, policy_version 1804399 (0.0009) [2023-12-27 18:09:26,974][20347] Updated weights for policy 0, policy_version 1806032 (0.0010) [2023-12-27 18:09:27,019][20347] Updated weights for policy 0, policy_version 1806042 (0.0010) [2023-12-27 18:09:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 924409856. Throughput: 0: 10288.4, 1: 10173.8. Samples: 924398136. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:27,363][19170] Avg episode reward: [(0, '961.000'), (1, '1000.000')] [2023-12-27 18:09:27,631][20381] Updated weights for policy 1, policy_version 1804409 (0.0006) [2023-12-27 18:09:27,647][20347] Updated weights for policy 0, policy_version 1806052 (0.0009) [2023-12-27 18:09:27,679][20381] Updated weights for policy 1, policy_version 1804419 (0.0005) [2023-12-27 18:09:27,699][20347] Updated weights for policy 0, policy_version 1806062 (0.0007) [2023-12-27 18:09:27,731][20381] Updated weights for policy 1, policy_version 1804429 (0.0006) [2023-12-27 18:09:27,747][20347] Updated weights for policy 0, policy_version 1806072 (0.0007) [2023-12-27 18:09:27,782][20381] Updated weights for policy 1, policy_version 1804439 (0.0007) [2023-12-27 18:09:28,357][20347] Updated weights for policy 0, policy_version 1806082 (0.0006) [2023-12-27 18:09:28,411][20347] Updated weights for policy 0, policy_version 1806092 (0.0006) [2023-12-27 18:09:28,447][20381] Updated weights for policy 1, policy_version 1804449 (0.0006) [2023-12-27 18:09:28,464][20347] Updated weights for policy 0, policy_version 1806102 (0.0006) [2023-12-27 18:09:28,503][20381] Updated weights for policy 1, policy_version 1804459 (0.0006) [2023-12-27 18:09:28,525][20347] Updated weights for policy 0, policy_version 1806112 (0.0006) [2023-12-27 18:09:28,563][20381] Updated weights for policy 1, policy_version 1804469 (0.0009) [2023-12-27 18:09:29,190][20347] Updated weights for policy 0, policy_version 1806122 (0.0008) [2023-12-27 18:09:29,253][20347] Updated weights for policy 0, policy_version 1806132 (0.0008) [2023-12-27 18:09:29,259][20381] Updated weights for policy 1, policy_version 1804479 (0.0008) [2023-12-27 18:09:29,306][20347] Updated weights for policy 0, policy_version 1806142 (0.0006) [2023-12-27 18:09:29,318][20381] Updated weights for policy 1, policy_version 1804489 (0.0008) [2023-12-27 18:09:29,384][20381] Updated weights for policy 1, policy_version 1804499 (0.0008) [2023-12-27 18:09:30,003][20347] Updated weights for policy 0, policy_version 1806152 (0.0007) [2023-12-27 18:09:30,059][20347] Updated weights for policy 0, policy_version 1806162 (0.0007) [2023-12-27 18:09:30,072][20381] Updated weights for policy 1, policy_version 1804509 (0.0009) [2023-12-27 18:09:30,116][20347] Updated weights for policy 0, policy_version 1806172 (0.0007) [2023-12-27 18:09:30,121][20381] Updated weights for policy 1, policy_version 1804519 (0.0009) [2023-12-27 18:09:30,172][20381] Updated weights for policy 1, policy_version 1804529 (0.0009) [2023-12-27 18:09:30,796][20347] Updated weights for policy 0, policy_version 1806182 (0.0007) [2023-12-27 18:09:30,849][20347] Updated weights for policy 0, policy_version 1806192 (0.0007) [2023-12-27 18:09:30,874][20381] Updated weights for policy 1, policy_version 1804539 (0.0011) [2023-12-27 18:09:30,901][20347] Updated weights for policy 0, policy_version 1806202 (0.0005) [2023-12-27 18:09:30,929][20381] Updated weights for policy 1, policy_version 1804549 (0.0010) [2023-12-27 18:09:30,974][20381] Updated weights for policy 1, policy_version 1804559 (0.0010) [2023-12-27 18:09:31,594][20347] Updated weights for policy 0, policy_version 1806212 (0.0006) [2023-12-27 18:09:31,655][20347] Updated weights for policy 0, policy_version 1806222 (0.0008) [2023-12-27 18:09:31,713][20347] Updated weights for policy 0, policy_version 1806232 (0.0007) [2023-12-27 18:09:31,745][20381] Updated weights for policy 1, policy_version 1804569 (0.0009) [2023-12-27 18:09:31,799][20381] Updated weights for policy 1, policy_version 1804579 (0.0008) [2023-12-27 18:09:31,858][20381] Updated weights for policy 1, policy_version 1804589 (0.0008) [2023-12-27 18:09:31,916][20381] Updated weights for policy 1, policy_version 1804599 (0.0008) [2023-12-27 18:09:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 924508160. Throughput: 0: 10252.9, 1: 10192.6. Samples: 924520060. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:32,363][19170] Avg episode reward: [(0, '951.190'), (1, '1000.000')] [2023-12-27 18:09:32,401][20347] Updated weights for policy 0, policy_version 1806242 (0.0007) [2023-12-27 18:09:32,456][20347] Updated weights for policy 0, policy_version 1806252 (0.0007) [2023-12-27 18:09:32,509][20347] Updated weights for policy 0, policy_version 1806262 (0.0008) [2023-12-27 18:09:32,555][20347] Updated weights for policy 0, policy_version 1806272 (0.0007) [2023-12-27 18:09:32,652][20381] Updated weights for policy 1, policy_version 1804609 (0.0008) [2023-12-27 18:09:32,708][20381] Updated weights for policy 1, policy_version 1804619 (0.0008) [2023-12-27 18:09:32,760][20381] Updated weights for policy 1, policy_version 1804629 (0.0009) [2023-12-27 18:09:33,239][20347] Updated weights for policy 0, policy_version 1806282 (0.0007) [2023-12-27 18:09:33,298][20347] Updated weights for policy 0, policy_version 1806292 (0.0008) [2023-12-27 18:09:33,350][20347] Updated weights for policy 0, policy_version 1806302 (0.0006) [2023-12-27 18:09:33,460][20381] Updated weights for policy 1, policy_version 1804639 (0.0008) [2023-12-27 18:09:33,517][20381] Updated weights for policy 1, policy_version 1804649 (0.0007) [2023-12-27 18:09:33,572][20381] Updated weights for policy 1, policy_version 1804659 (0.0008) [2023-12-27 18:09:33,902][20347] Updated weights for policy 0, policy_version 1806312 (0.0006) [2023-12-27 18:09:33,947][20347] Updated weights for policy 0, policy_version 1806322 (0.0005) [2023-12-27 18:09:33,992][20347] Updated weights for policy 0, policy_version 1806332 (0.0005) [2023-12-27 18:09:34,192][20381] Updated weights for policy 1, policy_version 1804669 (0.0008) [2023-12-27 18:09:34,249][20381] Updated weights for policy 1, policy_version 1804679 (0.0007) [2023-12-27 18:09:34,303][20381] Updated weights for policy 1, policy_version 1804689 (0.0009) [2023-12-27 18:09:34,622][20347] Updated weights for policy 0, policy_version 1806342 (0.0007) [2023-12-27 18:09:34,683][20347] Updated weights for policy 0, policy_version 1806352 (0.0008) [2023-12-27 18:09:34,743][20347] Updated weights for policy 0, policy_version 1806362 (0.0007) [2023-12-27 18:09:34,985][20381] Updated weights for policy 1, policy_version 1804699 (0.0007) [2023-12-27 18:09:35,040][20381] Updated weights for policy 1, policy_version 1804709 (0.0007) [2023-12-27 18:09:35,100][20381] Updated weights for policy 1, policy_version 1804719 (0.0008) [2023-12-27 18:09:35,393][20347] Updated weights for policy 0, policy_version 1806372 (0.0009) [2023-12-27 18:09:35,440][20347] Updated weights for policy 0, policy_version 1806382 (0.0007) [2023-12-27 18:09:35,493][20347] Updated weights for policy 0, policy_version 1806392 (0.0008) [2023-12-27 18:09:35,756][20381] Updated weights for policy 1, policy_version 1804729 (0.0008) [2023-12-27 18:09:35,810][20381] Updated weights for policy 1, policy_version 1804739 (0.0005) [2023-12-27 18:09:35,870][20381] Updated weights for policy 1, policy_version 1804749 (0.0006) [2023-12-27 18:09:35,926][20381] Updated weights for policy 1, policy_version 1804759 (0.0006) [2023-12-27 18:09:36,244][20347] Updated weights for policy 0, policy_version 1806402 (0.0008) [2023-12-27 18:09:36,301][20347] Updated weights for policy 0, policy_version 1806412 (0.0008) [2023-12-27 18:09:36,362][20347] Updated weights for policy 0, policy_version 1806422 (0.0007) [2023-12-27 18:09:36,424][20347] Updated weights for policy 0, policy_version 1806432 (0.0006) [2023-12-27 18:09:36,544][20381] Updated weights for policy 1, policy_version 1804769 (0.0008) [2023-12-27 18:09:36,592][20381] Updated weights for policy 1, policy_version 1804779 (0.0008) [2023-12-27 18:09:36,650][20381] Updated weights for policy 1, policy_version 1804789 (0.0008) [2023-12-27 18:09:37,055][20347] Updated weights for policy 0, policy_version 1806442 (0.0008) [2023-12-27 18:09:37,110][20347] Updated weights for policy 0, policy_version 1806452 (0.0007) [2023-12-27 18:09:37,164][20347] Updated weights for policy 0, policy_version 1806462 (0.0006) [2023-12-27 18:09:37,323][20381] Updated weights for policy 1, policy_version 1804799 (0.0010) [2023-12-27 18:09:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 924614656. Throughput: 0: 10249.9, 1: 10224.7. Samples: 924583152. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:37,363][19170] Avg episode reward: [(0, '941.320'), (1, '980.510')] [2023-12-27 18:09:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001806464_462536704.pth... [2023-12-27 18:09:37,373][20381] Updated weights for policy 1, policy_version 1804809 (0.0011) [2023-12-27 18:09:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001805248_462225408.pth [2023-12-27 18:09:37,419][20381] Updated weights for policy 1, policy_version 1804819 (0.0010) [2023-12-27 18:09:37,445][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001804824_462086144.pth... [2023-12-27 18:09:37,448][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001803640_461783040.pth [2023-12-27 18:09:37,879][20347] Updated weights for policy 0, policy_version 1806472 (0.0007) [2023-12-27 18:09:37,925][20347] Updated weights for policy 0, policy_version 1806482 (0.0007) [2023-12-27 18:09:37,982][20347] Updated weights for policy 0, policy_version 1806492 (0.0008) [2023-12-27 18:09:38,055][20381] Updated weights for policy 1, policy_version 1804829 (0.0011) [2023-12-27 18:09:38,105][20381] Updated weights for policy 1, policy_version 1804839 (0.0010) [2023-12-27 18:09:38,148][20381] Updated weights for policy 1, policy_version 1804849 (0.0008) [2023-12-27 18:09:38,702][20347] Updated weights for policy 0, policy_version 1806502 (0.0008) [2023-12-27 18:09:38,769][20347] Updated weights for policy 0, policy_version 1806512 (0.0008) [2023-12-27 18:09:38,824][20347] Updated weights for policy 0, policy_version 1806522 (0.0008) [2023-12-27 18:09:38,856][20381] Updated weights for policy 1, policy_version 1804859 (0.0007) [2023-12-27 18:09:38,913][20381] Updated weights for policy 1, policy_version 1804869 (0.0010) [2023-12-27 18:09:38,969][20381] Updated weights for policy 1, policy_version 1804879 (0.0011) [2023-12-27 18:09:39,551][20347] Updated weights for policy 0, policy_version 1806532 (0.0010) [2023-12-27 18:09:39,615][20347] Updated weights for policy 0, policy_version 1806542 (0.0007) [2023-12-27 18:09:39,684][20347] Updated weights for policy 0, policy_version 1806552 (0.0008) [2023-12-27 18:09:39,697][20381] Updated weights for policy 1, policy_version 1804889 (0.0011) [2023-12-27 18:09:39,763][20381] Updated weights for policy 1, policy_version 1804899 (0.0007) [2023-12-27 18:09:39,827][20381] Updated weights for policy 1, policy_version 1804909 (0.0007) [2023-12-27 18:09:39,902][20381] Updated weights for policy 1, policy_version 1804919 (0.0010) [2023-12-27 18:09:40,399][20347] Updated weights for policy 0, policy_version 1806562 (0.0008) [2023-12-27 18:09:40,455][20347] Updated weights for policy 0, policy_version 1806572 (0.0007) [2023-12-27 18:09:40,508][20347] Updated weights for policy 0, policy_version 1806582 (0.0008) [2023-12-27 18:09:40,560][20347] Updated weights for policy 0, policy_version 1806592 (0.0008) [2023-12-27 18:09:40,604][20381] Updated weights for policy 1, policy_version 1804929 (0.0009) [2023-12-27 18:09:40,658][20381] Updated weights for policy 1, policy_version 1804939 (0.0007) [2023-12-27 18:09:40,705][20381] Updated weights for policy 1, policy_version 1804949 (0.0008) [2023-12-27 18:09:41,318][20347] Updated weights for policy 0, policy_version 1806602 (0.0007) [2023-12-27 18:09:41,388][20347] Updated weights for policy 0, policy_version 1806612 (0.0009) [2023-12-27 18:09:41,444][20347] Updated weights for policy 0, policy_version 1806622 (0.0009) [2023-12-27 18:09:41,507][20381] Updated weights for policy 1, policy_version 1804959 (0.0009) [2023-12-27 18:09:41,564][20381] Updated weights for policy 1, policy_version 1804969 (0.0008) [2023-12-27 18:09:41,619][20381] Updated weights for policy 1, policy_version 1804979 (0.0007) [2023-12-27 18:09:42,215][20347] Updated weights for policy 0, policy_version 1806632 (0.0008) [2023-12-27 18:09:42,279][20347] Updated weights for policy 0, policy_version 1806642 (0.0009) [2023-12-27 18:09:42,338][20347] Updated weights for policy 0, policy_version 1806652 (0.0007) [2023-12-27 18:09:42,350][20381] Updated weights for policy 1, policy_version 1804989 (0.0009) [2023-12-27 18:09:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 924704768. Throughput: 0: 10239.2, 1: 10211.5. Samples: 924704304. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:42,363][19170] Avg episode reward: [(0, '980.380'), (1, '980.510')] [2023-12-27 18:09:42,417][20381] Updated weights for policy 1, policy_version 1804999 (0.0008) [2023-12-27 18:09:42,478][20381] Updated weights for policy 1, policy_version 1805009 (0.0008) [2023-12-27 18:09:43,009][20347] Updated weights for policy 0, policy_version 1806662 (0.0008) [2023-12-27 18:09:43,056][20347] Updated weights for policy 0, policy_version 1806672 (0.0007) [2023-12-27 18:09:43,108][20347] Updated weights for policy 0, policy_version 1806682 (0.0008) [2023-12-27 18:09:43,188][20381] Updated weights for policy 1, policy_version 1805019 (0.0009) [2023-12-27 18:09:43,235][20381] Updated weights for policy 1, policy_version 1805029 (0.0007) [2023-12-27 18:09:43,287][20381] Updated weights for policy 1, policy_version 1805039 (0.0007) [2023-12-27 18:09:43,726][20347] Updated weights for policy 0, policy_version 1806692 (0.0007) [2023-12-27 18:09:43,778][20347] Updated weights for policy 0, policy_version 1806702 (0.0005) [2023-12-27 18:09:43,838][20347] Updated weights for policy 0, policy_version 1806712 (0.0005) [2023-12-27 18:09:43,987][20381] Updated weights for policy 1, policy_version 1805049 (0.0007) [2023-12-27 18:09:44,044][20381] Updated weights for policy 1, policy_version 1805059 (0.0008) [2023-12-27 18:09:44,095][20381] Updated weights for policy 1, policy_version 1805069 (0.0008) [2023-12-27 18:09:44,150][20381] Updated weights for policy 1, policy_version 1805079 (0.0007) [2023-12-27 18:09:44,478][20347] Updated weights for policy 0, policy_version 1806722 (0.0006) [2023-12-27 18:09:44,535][20347] Updated weights for policy 0, policy_version 1806732 (0.0008) [2023-12-27 18:09:44,591][20347] Updated weights for policy 0, policy_version 1806742 (0.0007) [2023-12-27 18:09:44,645][20347] Updated weights for policy 0, policy_version 1806752 (0.0005) [2023-12-27 18:09:44,843][20381] Updated weights for policy 1, policy_version 1805089 (0.0006) [2023-12-27 18:09:44,912][20381] Updated weights for policy 1, policy_version 1805099 (0.0006) [2023-12-27 18:09:44,975][20381] Updated weights for policy 1, policy_version 1805109 (0.0007) [2023-12-27 18:09:45,348][20347] Updated weights for policy 0, policy_version 1806762 (0.0008) [2023-12-27 18:09:45,402][20347] Updated weights for policy 0, policy_version 1806772 (0.0007) [2023-12-27 18:09:45,457][20347] Updated weights for policy 0, policy_version 1806782 (0.0007) [2023-12-27 18:09:45,615][20381] Updated weights for policy 1, policy_version 1805119 (0.0007) [2023-12-27 18:09:45,670][20381] Updated weights for policy 1, policy_version 1805129 (0.0009) [2023-12-27 18:09:45,726][20381] Updated weights for policy 1, policy_version 1805139 (0.0008) [2023-12-27 18:09:46,037][20347] Updated weights for policy 0, policy_version 1806792 (0.0005) [2023-12-27 18:09:46,090][20347] Updated weights for policy 0, policy_version 1806802 (0.0005) [2023-12-27 18:09:46,154][20347] Updated weights for policy 0, policy_version 1806812 (0.0005) [2023-12-27 18:09:46,429][20381] Updated weights for policy 1, policy_version 1805149 (0.0007) [2023-12-27 18:09:46,480][20381] Updated weights for policy 1, policy_version 1805159 (0.0005) [2023-12-27 18:09:46,541][20381] Updated weights for policy 1, policy_version 1805169 (0.0007) [2023-12-27 18:09:46,682][20347] Updated weights for policy 0, policy_version 1806822 (0.0007) [2023-12-27 18:09:46,739][20347] Updated weights for policy 0, policy_version 1806832 (0.0007) [2023-12-27 18:09:46,785][20347] Updated weights for policy 0, policy_version 1806842 (0.0005) [2023-12-27 18:09:47,201][20381] Updated weights for policy 1, policy_version 1805179 (0.0008) [2023-12-27 18:09:47,261][20381] Updated weights for policy 1, policy_version 1805189 (0.0008) [2023-12-27 18:09:47,320][20381] Updated weights for policy 1, policy_version 1805199 (0.0008) [2023-12-27 18:09:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 924811264. Throughput: 0: 10328.5, 1: 10163.3. Samples: 924827368. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:47,363][19170] Avg episode reward: [(0, '990.230'), (1, '980.530')] [2023-12-27 18:09:47,461][20347] Updated weights for policy 0, policy_version 1806852 (0.0006) [2023-12-27 18:09:47,519][20347] Updated weights for policy 0, policy_version 1806862 (0.0007) [2023-12-27 18:09:47,570][20347] Updated weights for policy 0, policy_version 1806872 (0.0007) [2023-12-27 18:09:48,034][20381] Updated weights for policy 1, policy_version 1805209 (0.0007) [2023-12-27 18:09:48,080][20381] Updated weights for policy 1, policy_version 1805219 (0.0008) [2023-12-27 18:09:48,128][20381] Updated weights for policy 1, policy_version 1805229 (0.0007) [2023-12-27 18:09:48,189][20381] Updated weights for policy 1, policy_version 1805239 (0.0007) [2023-12-27 18:09:48,191][20347] Updated weights for policy 0, policy_version 1806882 (0.0007) [2023-12-27 18:09:48,252][20347] Updated weights for policy 0, policy_version 1806892 (0.0010) [2023-12-27 18:09:48,310][20347] Updated weights for policy 0, policy_version 1806902 (0.0010) [2023-12-27 18:09:48,375][20347] Updated weights for policy 0, policy_version 1806912 (0.0009) [2023-12-27 18:09:48,964][20381] Updated weights for policy 1, policy_version 1805249 (0.0010) [2023-12-27 18:09:49,025][20381] Updated weights for policy 1, policy_version 1805259 (0.0010) [2023-12-27 18:09:49,051][20347] Updated weights for policy 0, policy_version 1806922 (0.0010) [2023-12-27 18:09:49,076][20381] Updated weights for policy 1, policy_version 1805269 (0.0010) [2023-12-27 18:09:49,097][20347] Updated weights for policy 0, policy_version 1806932 (0.0010) [2023-12-27 18:09:49,148][20347] Updated weights for policy 0, policy_version 1806942 (0.0010) [2023-12-27 18:09:49,791][20381] Updated weights for policy 1, policy_version 1805279 (0.0010) [2023-12-27 18:09:49,833][20347] Updated weights for policy 0, policy_version 1806952 (0.0011) [2023-12-27 18:09:49,853][20381] Updated weights for policy 1, policy_version 1805289 (0.0011) [2023-12-27 18:09:49,896][20347] Updated weights for policy 0, policy_version 1806962 (0.0011) [2023-12-27 18:09:49,914][20381] Updated weights for policy 1, policy_version 1805299 (0.0010) [2023-12-27 18:09:49,962][20347] Updated weights for policy 0, policy_version 1806972 (0.0008) [2023-12-27 18:09:50,652][20347] Updated weights for policy 0, policy_version 1806982 (0.0007) [2023-12-27 18:09:50,657][20381] Updated weights for policy 1, policy_version 1805309 (0.0008) [2023-12-27 18:09:50,710][20381] Updated weights for policy 1, policy_version 1805319 (0.0006) [2023-12-27 18:09:50,714][20347] Updated weights for policy 0, policy_version 1806992 (0.0009) [2023-12-27 18:09:50,767][20381] Updated weights for policy 1, policy_version 1805329 (0.0007) [2023-12-27 18:09:50,774][20347] Updated weights for policy 0, policy_version 1807002 (0.0007) [2023-12-27 18:09:51,466][20347] Updated weights for policy 0, policy_version 1807012 (0.0007) [2023-12-27 18:09:51,479][20381] Updated weights for policy 1, policy_version 1805339 (0.0008) [2023-12-27 18:09:51,530][20347] Updated weights for policy 0, policy_version 1807022 (0.0006) [2023-12-27 18:09:51,536][20381] Updated weights for policy 1, policy_version 1805349 (0.0006) [2023-12-27 18:09:51,592][20347] Updated weights for policy 0, policy_version 1807032 (0.0009) [2023-12-27 18:09:51,594][20381] Updated weights for policy 1, policy_version 1805359 (0.0009) [2023-12-27 18:09:52,274][20347] Updated weights for policy 0, policy_version 1807042 (0.0008) [2023-12-27 18:09:52,293][20381] Updated weights for policy 1, policy_version 1805369 (0.0008) [2023-12-27 18:09:52,330][20347] Updated weights for policy 0, policy_version 1807052 (0.0010) [2023-12-27 18:09:52,342][20381] Updated weights for policy 1, policy_version 1805379 (0.0005) [2023-12-27 18:09:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 924909568. Throughput: 0: 10347.6, 1: 10044.0. Samples: 924886932. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:52,363][19170] Avg episode reward: [(0, '951.080'), (1, '980.530')] [2023-12-27 18:09:52,392][20347] Updated weights for policy 0, policy_version 1807062 (0.0009) [2023-12-27 18:09:52,408][20381] Updated weights for policy 1, policy_version 1805389 (0.0007) [2023-12-27 18:09:52,453][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001807072_462692352.pth... [2023-12-27 18:09:52,453][20347] Updated weights for policy 0, policy_version 1807072 (0.0007) [2023-12-27 18:09:52,457][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001805856_462381056.pth [2023-12-27 18:09:52,463][20381] Updated weights for policy 1, policy_version 1805399 (0.0005) [2023-12-27 18:09:52,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001805400_462233600.pth... [2023-12-27 18:09:52,472][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001804216_461930496.pth [2023-12-27 18:09:53,046][20347] Updated weights for policy 0, policy_version 1807082 (0.0005) [2023-12-27 18:09:53,096][20347] Updated weights for policy 0, policy_version 1807092 (0.0007) [2023-12-27 18:09:53,152][20347] Updated weights for policy 0, policy_version 1807102 (0.0006) [2023-12-27 18:09:53,158][20381] Updated weights for policy 1, policy_version 1805409 (0.0005) [2023-12-27 18:09:53,216][20381] Updated weights for policy 1, policy_version 1805419 (0.0007) [2023-12-27 18:09:53,265][20381] Updated weights for policy 1, policy_version 1805429 (0.0008) [2023-12-27 18:09:53,802][20347] Updated weights for policy 0, policy_version 1807112 (0.0006) [2023-12-27 18:09:53,857][20347] Updated weights for policy 0, policy_version 1807122 (0.0006) [2023-12-27 18:09:53,909][20347] Updated weights for policy 0, policy_version 1807132 (0.0005) [2023-12-27 18:09:53,990][20381] Updated weights for policy 1, policy_version 1805439 (0.0008) [2023-12-27 18:09:54,047][20381] Updated weights for policy 1, policy_version 1805449 (0.0008) [2023-12-27 18:09:54,096][20381] Updated weights for policy 1, policy_version 1805459 (0.0008) [2023-12-27 18:09:54,564][20347] Updated weights for policy 0, policy_version 1807142 (0.0007) [2023-12-27 18:09:54,616][20347] Updated weights for policy 0, policy_version 1807152 (0.0005) [2023-12-27 18:09:54,675][20347] Updated weights for policy 0, policy_version 1807162 (0.0005) [2023-12-27 18:09:54,797][20381] Updated weights for policy 1, policy_version 1805469 (0.0009) [2023-12-27 18:09:54,847][20381] Updated weights for policy 1, policy_version 1805479 (0.0010) [2023-12-27 18:09:54,891][20381] Updated weights for policy 1, policy_version 1805489 (0.0010) [2023-12-27 18:09:55,259][20347] Updated weights for policy 0, policy_version 1807172 (0.0007) [2023-12-27 18:09:55,309][20347] Updated weights for policy 0, policy_version 1807182 (0.0007) [2023-12-27 18:09:55,356][20347] Updated weights for policy 0, policy_version 1807192 (0.0007) [2023-12-27 18:09:55,596][20381] Updated weights for policy 1, policy_version 1805499 (0.0010) [2023-12-27 18:09:55,648][20381] Updated weights for policy 1, policy_version 1805509 (0.0011) [2023-12-27 18:09:55,704][20381] Updated weights for policy 1, policy_version 1805519 (0.0011) [2023-12-27 18:09:56,043][20347] Updated weights for policy 0, policy_version 1807202 (0.0006) [2023-12-27 18:09:56,097][20347] Updated weights for policy 0, policy_version 1807212 (0.0006) [2023-12-27 18:09:56,152][20347] Updated weights for policy 0, policy_version 1807222 (0.0005) [2023-12-27 18:09:56,207][20347] Updated weights for policy 0, policy_version 1807232 (0.0006) [2023-12-27 18:09:56,408][20381] Updated weights for policy 1, policy_version 1805529 (0.0011) [2023-12-27 18:09:56,459][20381] Updated weights for policy 1, policy_version 1805539 (0.0010) [2023-12-27 18:09:56,517][20381] Updated weights for policy 1, policy_version 1805549 (0.0008) [2023-12-27 18:09:56,572][20381] Updated weights for policy 1, policy_version 1805559 (0.0005) [2023-12-27 18:09:56,776][20347] Updated weights for policy 0, policy_version 1807242 (0.0008) [2023-12-27 18:09:56,835][20347] Updated weights for policy 0, policy_version 1807252 (0.0006) [2023-12-27 18:09:56,885][20347] Updated weights for policy 0, policy_version 1807262 (0.0005) [2023-12-27 18:09:57,204][20381] Updated weights for policy 1, policy_version 1805569 (0.0010) [2023-12-27 18:09:57,263][20381] Updated weights for policy 1, policy_version 1805579 (0.0011) [2023-12-27 18:09:57,326][20381] Updated weights for policy 1, policy_version 1805589 (0.0011) [2023-12-27 18:09:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 925024256. Throughput: 0: 10430.4, 1: 10033.1. Samples: 925010532. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:09:57,363][19170] Avg episode reward: [(0, '921.700'), (1, '990.270')] [2023-12-27 18:09:57,460][20347] Updated weights for policy 0, policy_version 1807272 (0.0005) [2023-12-27 18:09:57,521][20347] Updated weights for policy 0, policy_version 1807282 (0.0006) [2023-12-27 18:09:57,575][20347] Updated weights for policy 0, policy_version 1807292 (0.0005) [2023-12-27 18:09:57,944][20381] Updated weights for policy 1, policy_version 1805599 (0.0008) [2023-12-27 18:09:57,989][20381] Updated weights for policy 1, policy_version 1805609 (0.0011) [2023-12-27 18:09:58,049][20381] Updated weights for policy 1, policy_version 1805619 (0.0011) [2023-12-27 18:09:58,180][20347] Updated weights for policy 0, policy_version 1807302 (0.0006) [2023-12-27 18:09:58,247][20347] Updated weights for policy 0, policy_version 1807312 (0.0008) [2023-12-27 18:09:58,305][20347] Updated weights for policy 0, policy_version 1807322 (0.0008) [2023-12-27 18:09:58,801][20381] Updated weights for policy 1, policy_version 1805629 (0.0009) [2023-12-27 18:09:58,873][20381] Updated weights for policy 1, policy_version 1805639 (0.0008) [2023-12-27 18:09:58,934][20381] Updated weights for policy 1, policy_version 1805649 (0.0006) [2023-12-27 18:09:59,102][20347] Updated weights for policy 0, policy_version 1807332 (0.0008) [2023-12-27 18:09:59,159][20347] Updated weights for policy 0, policy_version 1807342 (0.0007) [2023-12-27 18:09:59,221][20347] Updated weights for policy 0, policy_version 1807352 (0.0007) [2023-12-27 18:09:59,698][20381] Updated weights for policy 1, policy_version 1805659 (0.0009) [2023-12-27 18:09:59,758][20381] Updated weights for policy 1, policy_version 1805669 (0.0011) [2023-12-27 18:09:59,819][20381] Updated weights for policy 1, policy_version 1805679 (0.0011) [2023-12-27 18:09:59,876][20347] Updated weights for policy 0, policy_version 1807362 (0.0009) [2023-12-27 18:09:59,930][20347] Updated weights for policy 0, policy_version 1807372 (0.0006) [2023-12-27 18:09:59,991][20347] Updated weights for policy 0, policy_version 1807382 (0.0008) [2023-12-27 18:10:00,046][20347] Updated weights for policy 0, policy_version 1807392 (0.0007) [2023-12-27 18:10:00,551][20381] Updated weights for policy 1, policy_version 1805689 (0.0009) [2023-12-27 18:10:00,607][20381] Updated weights for policy 1, policy_version 1805699 (0.0005) [2023-12-27 18:10:00,666][20381] Updated weights for policy 1, policy_version 1805709 (0.0006) [2023-12-27 18:10:00,674][20347] Updated weights for policy 0, policy_version 1807402 (0.0005) [2023-12-27 18:10:00,721][20381] Updated weights for policy 1, policy_version 1805719 (0.0006) [2023-12-27 18:10:00,730][20347] Updated weights for policy 0, policy_version 1807412 (0.0005) [2023-12-27 18:10:00,788][20347] Updated weights for policy 0, policy_version 1807422 (0.0005) [2023-12-27 18:10:01,378][20381] Updated weights for policy 1, policy_version 1805729 (0.0009) [2023-12-27 18:10:01,426][20347] Updated weights for policy 0, policy_version 1807432 (0.0010) [2023-12-27 18:10:01,437][20381] Updated weights for policy 1, policy_version 1805739 (0.0011) [2023-12-27 18:10:01,487][20347] Updated weights for policy 0, policy_version 1807442 (0.0010) [2023-12-27 18:10:01,499][20381] Updated weights for policy 1, policy_version 1805749 (0.0011) [2023-12-27 18:10:01,546][20347] Updated weights for policy 0, policy_version 1807452 (0.0011) [2023-12-27 18:10:02,189][20381] Updated weights for policy 1, policy_version 1805759 (0.0007) [2023-12-27 18:10:02,251][20381] Updated weights for policy 1, policy_version 1805769 (0.0007) [2023-12-27 18:10:02,280][20347] Updated weights for policy 0, policy_version 1807462 (0.0009) [2023-12-27 18:10:02,306][20381] Updated weights for policy 1, policy_version 1805779 (0.0007) [2023-12-27 18:10:02,328][20347] Updated weights for policy 0, policy_version 1807472 (0.0007) [2023-12-27 18:10:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 925122560. Throughput: 0: 10425.8, 1: 9974.0. Samples: 925133952. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:02,363][19170] Avg episode reward: [(0, '951.140'), (1, '990.270')] [2023-12-27 18:10:02,385][20347] Updated weights for policy 0, policy_version 1807482 (0.0007) [2023-12-27 18:10:02,903][20381] Updated weights for policy 1, policy_version 1805789 (0.0006) [2023-12-27 18:10:02,959][20381] Updated weights for policy 1, policy_version 1805799 (0.0008) [2023-12-27 18:10:03,006][20347] Updated weights for policy 0, policy_version 1807492 (0.0007) [2023-12-27 18:10:03,012][20381] Updated weights for policy 1, policy_version 1805809 (0.0006) [2023-12-27 18:10:03,064][20347] Updated weights for policy 0, policy_version 1807502 (0.0007) [2023-12-27 18:10:03,122][20347] Updated weights for policy 0, policy_version 1807512 (0.0005) [2023-12-27 18:10:03,630][20381] Updated weights for policy 1, policy_version 1805819 (0.0006) [2023-12-27 18:10:03,688][20381] Updated weights for policy 1, policy_version 1805829 (0.0006) [2023-12-27 18:10:03,712][20347] Updated weights for policy 0, policy_version 1807522 (0.0006) [2023-12-27 18:10:03,736][20381] Updated weights for policy 1, policy_version 1805839 (0.0006) [2023-12-27 18:10:03,764][20347] Updated weights for policy 0, policy_version 1807532 (0.0006) [2023-12-27 18:10:03,818][20347] Updated weights for policy 0, policy_version 1807542 (0.0007) [2023-12-27 18:10:03,886][20347] Updated weights for policy 0, policy_version 1807552 (0.0008) [2023-12-27 18:10:04,388][20381] Updated weights for policy 1, policy_version 1805849 (0.0005) [2023-12-27 18:10:04,438][20381] Updated weights for policy 1, policy_version 1805859 (0.0007) [2023-12-27 18:10:04,480][20381] Updated weights for policy 1, policy_version 1805869 (0.0007) [2023-12-27 18:10:04,527][20381] Updated weights for policy 1, policy_version 1805879 (0.0008) [2023-12-27 18:10:04,548][20347] Updated weights for policy 0, policy_version 1807562 (0.0008) [2023-12-27 18:10:04,604][20347] Updated weights for policy 0, policy_version 1807572 (0.0007) [2023-12-27 18:10:04,656][20347] Updated weights for policy 0, policy_version 1807582 (0.0008) [2023-12-27 18:10:05,177][20381] Updated weights for policy 1, policy_version 1805889 (0.0007) [2023-12-27 18:10:05,220][20381] Updated weights for policy 1, policy_version 1805899 (0.0007) [2023-12-27 18:10:05,266][20381] Updated weights for policy 1, policy_version 1805909 (0.0008) [2023-12-27 18:10:05,366][20347] Updated weights for policy 0, policy_version 1807592 (0.0007) [2023-12-27 18:10:05,411][20347] Updated weights for policy 0, policy_version 1807602 (0.0008) [2023-12-27 18:10:05,463][20347] Updated weights for policy 0, policy_version 1807612 (0.0008) [2023-12-27 18:10:06,002][20381] Updated weights for policy 1, policy_version 1805919 (0.0008) [2023-12-27 18:10:06,060][20381] Updated weights for policy 1, policy_version 1805929 (0.0008) [2023-12-27 18:10:06,133][20381] Updated weights for policy 1, policy_version 1805939 (0.0009) [2023-12-27 18:10:06,186][20347] Updated weights for policy 0, policy_version 1807622 (0.0007) [2023-12-27 18:10:06,245][20347] Updated weights for policy 0, policy_version 1807632 (0.0008) [2023-12-27 18:10:06,304][20347] Updated weights for policy 0, policy_version 1807642 (0.0006) [2023-12-27 18:10:06,806][20381] Updated weights for policy 1, policy_version 1805949 (0.0007) [2023-12-27 18:10:06,871][20381] Updated weights for policy 1, policy_version 1805959 (0.0006) [2023-12-27 18:10:06,933][20347] Updated weights for policy 0, policy_version 1807652 (0.0008) [2023-12-27 18:10:06,949][20381] Updated weights for policy 1, policy_version 1805969 (0.0007) [2023-12-27 18:10:06,989][20347] Updated weights for policy 0, policy_version 1807662 (0.0006) [2023-12-27 18:10:07,047][20347] Updated weights for policy 0, policy_version 1807672 (0.0006) [2023-12-27 18:10:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 925229056. Throughput: 0: 10455.2, 1: 10040.4. Samples: 925198456. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:07,363][19170] Avg episode reward: [(0, '941.330'), (1, '1000.000')] [2023-12-27 18:10:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001805976_462381056.pth... [2023-12-27 18:10:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001807680_462848000.pth... [2023-12-27 18:10:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001806464_462536704.pth [2023-12-27 18:10:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001804824_462086144.pth [2023-12-27 18:10:07,522][20381] Updated weights for policy 1, policy_version 1805979 (0.0008) [2023-12-27 18:10:07,573][20381] Updated weights for policy 1, policy_version 1805989 (0.0008) [2023-12-27 18:10:07,621][20381] Updated weights for policy 1, policy_version 1805999 (0.0007) [2023-12-27 18:10:07,707][20347] Updated weights for policy 0, policy_version 1807682 (0.0007) [2023-12-27 18:10:07,762][20347] Updated weights for policy 0, policy_version 1807692 (0.0007) [2023-12-27 18:10:07,820][20347] Updated weights for policy 0, policy_version 1807702 (0.0007) [2023-12-27 18:10:07,879][20347] Updated weights for policy 0, policy_version 1807712 (0.0007) [2023-12-27 18:10:08,269][20381] Updated weights for policy 1, policy_version 1806009 (0.0007) [2023-12-27 18:10:08,336][20381] Updated weights for policy 1, policy_version 1806019 (0.0013) [2023-12-27 18:10:08,392][20381] Updated weights for policy 1, policy_version 1806030 (0.0007) [2023-12-27 18:10:08,455][20381] Updated weights for policy 1, policy_version 1806040 (0.0009) [2023-12-27 18:10:08,514][20347] Updated weights for policy 0, policy_version 1807722 (0.0008) [2023-12-27 18:10:08,573][20347] Updated weights for policy 0, policy_version 1807732 (0.0008) [2023-12-27 18:10:08,628][20347] Updated weights for policy 0, policy_version 1807742 (0.0008) [2023-12-27 18:10:09,143][20381] Updated weights for policy 1, policy_version 1806050 (0.0009) [2023-12-27 18:10:09,196][20381] Updated weights for policy 1, policy_version 1806060 (0.0005) [2023-12-27 18:10:09,262][20381] Updated weights for policy 1, policy_version 1806070 (0.0008) [2023-12-27 18:10:09,346][20347] Updated weights for policy 0, policy_version 1807752 (0.0008) [2023-12-27 18:10:09,413][20347] Updated weights for policy 0, policy_version 1807762 (0.0009) [2023-12-27 18:10:09,473][20347] Updated weights for policy 0, policy_version 1807772 (0.0008) [2023-12-27 18:10:10,010][20381] Updated weights for policy 1, policy_version 1806080 (0.0008) [2023-12-27 18:10:10,072][20381] Updated weights for policy 1, policy_version 1806090 (0.0008) [2023-12-27 18:10:10,132][20381] Updated weights for policy 1, policy_version 1806100 (0.0008) [2023-12-27 18:10:10,208][20347] Updated weights for policy 0, policy_version 1807782 (0.0008) [2023-12-27 18:10:10,270][20347] Updated weights for policy 0, policy_version 1807792 (0.0008) [2023-12-27 18:10:10,341][20347] Updated weights for policy 0, policy_version 1807802 (0.0008) [2023-12-27 18:10:10,842][20381] Updated weights for policy 1, policy_version 1806110 (0.0006) [2023-12-27 18:10:10,905][20381] Updated weights for policy 1, policy_version 1806120 (0.0006) [2023-12-27 18:10:10,966][20381] Updated weights for policy 1, policy_version 1806130 (0.0006) [2023-12-27 18:10:11,033][20347] Updated weights for policy 0, policy_version 1807812 (0.0007) [2023-12-27 18:10:11,095][20347] Updated weights for policy 0, policy_version 1807822 (0.0008) [2023-12-27 18:10:11,155][20347] Updated weights for policy 0, policy_version 1807832 (0.0008) [2023-12-27 18:10:11,700][20381] Updated weights for policy 1, policy_version 1806140 (0.0007) [2023-12-27 18:10:11,767][20381] Updated weights for policy 1, policy_version 1806150 (0.0008) [2023-12-27 18:10:11,828][20381] Updated weights for policy 1, policy_version 1806160 (0.0008) [2023-12-27 18:10:11,903][20347] Updated weights for policy 0, policy_version 1807842 (0.0008) [2023-12-27 18:10:11,966][20347] Updated weights for policy 0, policy_version 1807852 (0.0007) [2023-12-27 18:10:12,026][20347] Updated weights for policy 0, policy_version 1807862 (0.0007) [2023-12-27 18:10:12,082][20347] Updated weights for policy 0, policy_version 1807872 (0.0008) [2023-12-27 18:10:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20482.5, 300 sec: 20521.6). Total num frames: 925327360. Throughput: 0: 10398.5, 1: 10083.3. Samples: 925319816. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:12,363][19170] Avg episode reward: [(0, '931.550'), (1, '990.380')] [2023-12-27 18:10:12,521][20381] Updated weights for policy 1, policy_version 1806170 (0.0008) [2023-12-27 18:10:12,581][20381] Updated weights for policy 1, policy_version 1806180 (0.0007) [2023-12-27 18:10:12,646][20381] Updated weights for policy 1, policy_version 1806190 (0.0006) [2023-12-27 18:10:12,709][20381] Updated weights for policy 1, policy_version 1806200 (0.0006) [2023-12-27 18:10:12,722][20347] Updated weights for policy 0, policy_version 1807882 (0.0006) [2023-12-27 18:10:12,785][20347] Updated weights for policy 0, policy_version 1807892 (0.0007) [2023-12-27 18:10:12,848][20347] Updated weights for policy 0, policy_version 1807902 (0.0008) [2023-12-27 18:10:13,383][20381] Updated weights for policy 1, policy_version 1806210 (0.0010) [2023-12-27 18:10:13,433][20381] Updated weights for policy 1, policy_version 1806220 (0.0010) [2023-12-27 18:10:13,471][20347] Updated weights for policy 0, policy_version 1807912 (0.0010) [2023-12-27 18:10:13,482][20381] Updated weights for policy 1, policy_version 1806230 (0.0010) [2023-12-27 18:10:13,528][20347] Updated weights for policy 0, policy_version 1807922 (0.0010) [2023-12-27 18:10:13,586][20347] Updated weights for policy 0, policy_version 1807932 (0.0010) [2023-12-27 18:10:14,090][20381] Updated weights for policy 1, policy_version 1806240 (0.0007) [2023-12-27 18:10:14,154][20381] Updated weights for policy 1, policy_version 1806250 (0.0006) [2023-12-27 18:10:14,215][20381] Updated weights for policy 1, policy_version 1806260 (0.0005) [2023-12-27 18:10:14,230][20347] Updated weights for policy 0, policy_version 1807942 (0.0008) [2023-12-27 18:10:14,288][20347] Updated weights for policy 0, policy_version 1807952 (0.0006) [2023-12-27 18:10:14,349][20347] Updated weights for policy 0, policy_version 1807962 (0.0010) [2023-12-27 18:10:14,848][20381] Updated weights for policy 1, policy_version 1806270 (0.0008) [2023-12-27 18:10:14,904][20381] Updated weights for policy 1, policy_version 1806280 (0.0010) [2023-12-27 18:10:14,965][20381] Updated weights for policy 1, policy_version 1806290 (0.0007) [2023-12-27 18:10:15,037][20347] Updated weights for policy 0, policy_version 1807972 (0.0010) [2023-12-27 18:10:15,090][20347] Updated weights for policy 0, policy_version 1807982 (0.0008) [2023-12-27 18:10:15,150][20347] Updated weights for policy 0, policy_version 1807992 (0.0008) [2023-12-27 18:10:15,593][20381] Updated weights for policy 1, policy_version 1806300 (0.0009) [2023-12-27 18:10:15,656][20381] Updated weights for policy 1, policy_version 1806310 (0.0005) [2023-12-27 18:10:15,714][20381] Updated weights for policy 1, policy_version 1806320 (0.0005) [2023-12-27 18:10:15,830][20347] Updated weights for policy 0, policy_version 1808002 (0.0008) [2023-12-27 18:10:15,874][20347] Updated weights for policy 0, policy_version 1808012 (0.0007) [2023-12-27 18:10:15,922][20347] Updated weights for policy 0, policy_version 1808022 (0.0008) [2023-12-27 18:10:15,976][20347] Updated weights for policy 0, policy_version 1808032 (0.0009) [2023-12-27 18:10:16,270][20381] Updated weights for policy 1, policy_version 1806330 (0.0006) [2023-12-27 18:10:16,324][20381] Updated weights for policy 1, policy_version 1806340 (0.0011) [2023-12-27 18:10:16,378][20381] Updated weights for policy 1, policy_version 1806350 (0.0011) [2023-12-27 18:10:16,437][20381] Updated weights for policy 1, policy_version 1806360 (0.0011) [2023-12-27 18:10:16,680][20347] Updated weights for policy 0, policy_version 1808042 (0.0007) [2023-12-27 18:10:16,738][20347] Updated weights for policy 0, policy_version 1808052 (0.0007) [2023-12-27 18:10:16,784][20347] Updated weights for policy 0, policy_version 1808062 (0.0008) [2023-12-27 18:10:17,088][20381] Updated weights for policy 1, policy_version 1806370 (0.0007) [2023-12-27 18:10:17,143][20381] Updated weights for policy 1, policy_version 1806380 (0.0007) [2023-12-27 18:10:17,200][20381] Updated weights for policy 1, policy_version 1806390 (0.0007) [2023-12-27 18:10:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 925433856. Throughput: 0: 10375.6, 1: 10163.2. Samples: 925444308. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:17,363][19170] Avg episode reward: [(0, '980.470'), (1, '990.380')] [2023-12-27 18:10:17,508][20347] Updated weights for policy 0, policy_version 1808072 (0.0007) [2023-12-27 18:10:17,563][20347] Updated weights for policy 0, policy_version 1808082 (0.0007) [2023-12-27 18:10:17,620][20347] Updated weights for policy 0, policy_version 1808092 (0.0008) [2023-12-27 18:10:17,873][20381] Updated weights for policy 1, policy_version 1806400 (0.0006) [2023-12-27 18:10:17,928][20381] Updated weights for policy 1, policy_version 1806410 (0.0006) [2023-12-27 18:10:17,987][20381] Updated weights for policy 1, policy_version 1806420 (0.0007) [2023-12-27 18:10:18,358][20347] Updated weights for policy 0, policy_version 1808102 (0.0009) [2023-12-27 18:10:18,413][20347] Updated weights for policy 0, policy_version 1808112 (0.0010) [2023-12-27 18:10:18,469][20347] Updated weights for policy 0, policy_version 1808122 (0.0011) [2023-12-27 18:10:18,642][20381] Updated weights for policy 1, policy_version 1806430 (0.0008) [2023-12-27 18:10:18,693][20381] Updated weights for policy 1, policy_version 1806440 (0.0007) [2023-12-27 18:10:18,746][20381] Updated weights for policy 1, policy_version 1806450 (0.0008) [2023-12-27 18:10:19,144][20347] Updated weights for policy 0, policy_version 1808132 (0.0010) [2023-12-27 18:10:19,198][20347] Updated weights for policy 0, policy_version 1808142 (0.0010) [2023-12-27 18:10:19,262][20347] Updated weights for policy 0, policy_version 1808152 (0.0013) [2023-12-27 18:10:19,545][20381] Updated weights for policy 1, policy_version 1806460 (0.0007) [2023-12-27 18:10:19,607][20381] Updated weights for policy 1, policy_version 1806470 (0.0008) [2023-12-27 18:10:19,666][20381] Updated weights for policy 1, policy_version 1806480 (0.0008) [2023-12-27 18:10:20,066][20347] Updated weights for policy 0, policy_version 1808162 (0.0009) [2023-12-27 18:10:20,132][20347] Updated weights for policy 0, policy_version 1808172 (0.0007) [2023-12-27 18:10:20,188][20347] Updated weights for policy 0, policy_version 1808182 (0.0008) [2023-12-27 18:10:20,238][20347] Updated weights for policy 0, policy_version 1808192 (0.0009) [2023-12-27 18:10:20,393][20381] Updated weights for policy 1, policy_version 1806490 (0.0008) [2023-12-27 18:10:20,455][20381] Updated weights for policy 1, policy_version 1806500 (0.0008) [2023-12-27 18:10:20,506][20381] Updated weights for policy 1, policy_version 1806510 (0.0007) [2023-12-27 18:10:20,556][20381] Updated weights for policy 1, policy_version 1806520 (0.0010) [2023-12-27 18:10:20,906][20347] Updated weights for policy 0, policy_version 1808202 (0.0011) [2023-12-27 18:10:20,967][20347] Updated weights for policy 0, policy_version 1808212 (0.0006) [2023-12-27 18:10:21,037][20347] Updated weights for policy 0, policy_version 1808222 (0.0009) [2023-12-27 18:10:21,370][20381] Updated weights for policy 1, policy_version 1806530 (0.0008) [2023-12-27 18:10:21,429][20381] Updated weights for policy 1, policy_version 1806540 (0.0008) [2023-12-27 18:10:21,492][20381] Updated weights for policy 1, policy_version 1806550 (0.0007) [2023-12-27 18:10:21,764][20347] Updated weights for policy 0, policy_version 1808232 (0.0011) [2023-12-27 18:10:21,827][20347] Updated weights for policy 0, policy_version 1808242 (0.0010) [2023-12-27 18:10:21,884][20347] Updated weights for policy 0, policy_version 1808252 (0.0011) [2023-12-27 18:10:22,281][20381] Updated weights for policy 1, policy_version 1806560 (0.0007) [2023-12-27 18:10:22,345][20381] Updated weights for policy 1, policy_version 1806570 (0.0008) [2023-12-27 18:10:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 925523968. Throughput: 0: 10292.3, 1: 10138.8. Samples: 925502556. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:22,363][19170] Avg episode reward: [(0, '941.260'), (1, '980.550')] [2023-12-27 18:10:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001808256_462995456.pth... [2023-12-27 18:10:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001807072_462692352.pth [2023-12-27 18:10:22,416][20381] Updated weights for policy 1, policy_version 1806580 (0.0007) [2023-12-27 18:10:22,442][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001806584_462536704.pth... [2023-12-27 18:10:22,447][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001805400_462233600.pth [2023-12-27 18:10:22,563][20347] Updated weights for policy 0, policy_version 1808262 (0.0010) [2023-12-27 18:10:22,623][20347] Updated weights for policy 0, policy_version 1808272 (0.0010) [2023-12-27 18:10:22,682][20347] Updated weights for policy 0, policy_version 1808282 (0.0010) [2023-12-27 18:10:23,073][20381] Updated weights for policy 1, policy_version 1806590 (0.0007) [2023-12-27 18:10:23,143][20381] Updated weights for policy 1, policy_version 1806600 (0.0006) [2023-12-27 18:10:23,212][20381] Updated weights for policy 1, policy_version 1806610 (0.0006) [2023-12-27 18:10:23,304][20347] Updated weights for policy 0, policy_version 1808292 (0.0008) [2023-12-27 18:10:23,353][20347] Updated weights for policy 0, policy_version 1808302 (0.0010) [2023-12-27 18:10:23,411][20347] Updated weights for policy 0, policy_version 1808312 (0.0010) [2023-12-27 18:10:23,794][20381] Updated weights for policy 1, policy_version 1806620 (0.0006) [2023-12-27 18:10:23,862][20381] Updated weights for policy 1, policy_version 1806630 (0.0007) [2023-12-27 18:10:23,925][20381] Updated weights for policy 1, policy_version 1806640 (0.0007) [2023-12-27 18:10:24,057][20347] Updated weights for policy 0, policy_version 1808322 (0.0010) [2023-12-27 18:10:24,110][20347] Updated weights for policy 0, policy_version 1808332 (0.0008) [2023-12-27 18:10:24,160][20347] Updated weights for policy 0, policy_version 1808342 (0.0008) [2023-12-27 18:10:24,215][20347] Updated weights for policy 0, policy_version 1808352 (0.0007) [2023-12-27 18:10:24,508][20381] Updated weights for policy 1, policy_version 1806650 (0.0008) [2023-12-27 18:10:24,567][20381] Updated weights for policy 1, policy_version 1806660 (0.0008) [2023-12-27 18:10:24,623][20381] Updated weights for policy 1, policy_version 1806670 (0.0008) [2023-12-27 18:10:24,678][20381] Updated weights for policy 1, policy_version 1806680 (0.0007) [2023-12-27 18:10:24,876][20347] Updated weights for policy 0, policy_version 1808362 (0.0006) [2023-12-27 18:10:24,936][20347] Updated weights for policy 0, policy_version 1808372 (0.0007) [2023-12-27 18:10:24,987][20347] Updated weights for policy 0, policy_version 1808382 (0.0008) [2023-12-27 18:10:25,388][20381] Updated weights for policy 1, policy_version 1806690 (0.0005) [2023-12-27 18:10:25,449][20381] Updated weights for policy 1, policy_version 1806700 (0.0010) [2023-12-27 18:10:25,509][20381] Updated weights for policy 1, policy_version 1806710 (0.0010) [2023-12-27 18:10:25,651][20347] Updated weights for policy 0, policy_version 1808392 (0.0007) [2023-12-27 18:10:25,704][20347] Updated weights for policy 0, policy_version 1808402 (0.0007) [2023-12-27 18:10:25,771][20347] Updated weights for policy 0, policy_version 1808412 (0.0008) [2023-12-27 18:10:26,058][20381] Updated weights for policy 1, policy_version 1806720 (0.0006) [2023-12-27 18:10:26,113][20381] Updated weights for policy 1, policy_version 1806730 (0.0010) [2023-12-27 18:10:26,165][20381] Updated weights for policy 1, policy_version 1806740 (0.0011) [2023-12-27 18:10:26,493][20347] Updated weights for policy 0, policy_version 1808422 (0.0008) [2023-12-27 18:10:26,539][20347] Updated weights for policy 0, policy_version 1808432 (0.0007) [2023-12-27 18:10:26,587][20347] Updated weights for policy 0, policy_version 1808442 (0.0007) [2023-12-27 18:10:26,845][20381] Updated weights for policy 1, policy_version 1806750 (0.0009) [2023-12-27 18:10:26,904][20381] Updated weights for policy 1, policy_version 1806760 (0.0009) [2023-12-27 18:10:26,950][20381] Updated weights for policy 1, policy_version 1806770 (0.0010) [2023-12-27 18:10:27,223][20347] Updated weights for policy 0, policy_version 1808452 (0.0007) [2023-12-27 18:10:27,274][20347] Updated weights for policy 0, policy_version 1808462 (0.0007) [2023-12-27 18:10:27,328][20347] Updated weights for policy 0, policy_version 1808472 (0.0007) [2023-12-27 18:10:27,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 925630464. Throughput: 0: 10320.0, 1: 10128.3. Samples: 925624480. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:27,364][19170] Avg episode reward: [(0, '921.790'), (1, '970.870')] [2023-12-27 18:10:27,641][20381] Updated weights for policy 1, policy_version 1806780 (0.0008) [2023-12-27 18:10:27,692][20381] Updated weights for policy 1, policy_version 1806790 (0.0010) [2023-12-27 18:10:27,749][20381] Updated weights for policy 1, policy_version 1806800 (0.0011) [2023-12-27 18:10:28,000][20347] Updated weights for policy 0, policy_version 1808482 (0.0007) [2023-12-27 18:10:28,057][20347] Updated weights for policy 0, policy_version 1808492 (0.0007) [2023-12-27 18:10:28,108][20347] Updated weights for policy 0, policy_version 1808502 (0.0007) [2023-12-27 18:10:28,157][20347] Updated weights for policy 0, policy_version 1808512 (0.0005) [2023-12-27 18:10:28,407][20381] Updated weights for policy 1, policy_version 1806810 (0.0010) [2023-12-27 18:10:28,466][20381] Updated weights for policy 1, policy_version 1806820 (0.0011) [2023-12-27 18:10:28,527][20381] Updated weights for policy 1, policy_version 1806830 (0.0011) [2023-12-27 18:10:28,578][20381] Updated weights for policy 1, policy_version 1806840 (0.0008) [2023-12-27 18:10:28,779][20347] Updated weights for policy 0, policy_version 1808522 (0.0006) [2023-12-27 18:10:28,848][20347] Updated weights for policy 0, policy_version 1808532 (0.0006) [2023-12-27 18:10:28,908][20347] Updated weights for policy 0, policy_version 1808542 (0.0006) [2023-12-27 18:10:29,216][20381] Updated weights for policy 1, policy_version 1806850 (0.0010) [2023-12-27 18:10:29,277][20381] Updated weights for policy 1, policy_version 1806860 (0.0008) [2023-12-27 18:10:29,337][20381] Updated weights for policy 1, policy_version 1806870 (0.0006) [2023-12-27 18:10:29,495][20347] Updated weights for policy 0, policy_version 1808552 (0.0007) [2023-12-27 18:10:29,543][20347] Updated weights for policy 0, policy_version 1808562 (0.0008) [2023-12-27 18:10:29,590][20347] Updated weights for policy 0, policy_version 1808572 (0.0008) [2023-12-27 18:10:29,986][20381] Updated weights for policy 1, policy_version 1806880 (0.0008) [2023-12-27 18:10:30,042][20381] Updated weights for policy 1, policy_version 1806890 (0.0008) [2023-12-27 18:10:30,105][20381] Updated weights for policy 1, policy_version 1806900 (0.0007) [2023-12-27 18:10:30,389][20347] Updated weights for policy 0, policy_version 1808582 (0.0008) [2023-12-27 18:10:30,441][20347] Updated weights for policy 0, policy_version 1808592 (0.0007) [2023-12-27 18:10:30,490][20347] Updated weights for policy 0, policy_version 1808602 (0.0008) [2023-12-27 18:10:30,715][20381] Updated weights for policy 1, policy_version 1806910 (0.0006) [2023-12-27 18:10:30,771][20381] Updated weights for policy 1, policy_version 1806920 (0.0005) [2023-12-27 18:10:30,826][20381] Updated weights for policy 1, policy_version 1806930 (0.0005) [2023-12-27 18:10:31,190][20347] Updated weights for policy 0, policy_version 1808612 (0.0007) [2023-12-27 18:10:31,254][20347] Updated weights for policy 0, policy_version 1808622 (0.0008) [2023-12-27 18:10:31,318][20347] Updated weights for policy 0, policy_version 1808632 (0.0007) [2023-12-27 18:10:31,451][20381] Updated weights for policy 1, policy_version 1806940 (0.0006) [2023-12-27 18:10:31,505][20381] Updated weights for policy 1, policy_version 1806950 (0.0008) [2023-12-27 18:10:31,558][20381] Updated weights for policy 1, policy_version 1806960 (0.0008) [2023-12-27 18:10:31,996][20347] Updated weights for policy 0, policy_version 1808642 (0.0007) [2023-12-27 18:10:32,043][20347] Updated weights for policy 0, policy_version 1808652 (0.0008) [2023-12-27 18:10:32,097][20347] Updated weights for policy 0, policy_version 1808662 (0.0008) [2023-12-27 18:10:32,154][20347] Updated weights for policy 0, policy_version 1808672 (0.0007) [2023-12-27 18:10:32,197][20381] Updated weights for policy 1, policy_version 1806970 (0.0007) [2023-12-27 18:10:32,262][20381] Updated weights for policy 1, policy_version 1806980 (0.0008) [2023-12-27 18:10:32,322][20381] Updated weights for policy 1, policy_version 1806990 (0.0008) [2023-12-27 18:10:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 925736960. Throughput: 0: 10290.6, 1: 10212.0. Samples: 925749984. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:32,363][19170] Avg episode reward: [(0, '921.800'), (1, '970.910')] [2023-12-27 18:10:32,388][20381] Updated weights for policy 1, policy_version 1807000 (0.0009) [2023-12-27 18:10:32,881][20347] Updated weights for policy 0, policy_version 1808682 (0.0008) [2023-12-27 18:10:32,934][20347] Updated weights for policy 0, policy_version 1808692 (0.0007) [2023-12-27 18:10:32,994][20347] Updated weights for policy 0, policy_version 1808702 (0.0007) [2023-12-27 18:10:33,067][20381] Updated weights for policy 1, policy_version 1807010 (0.0008) [2023-12-27 18:10:33,126][20381] Updated weights for policy 1, policy_version 1807020 (0.0008) [2023-12-27 18:10:33,184][20381] Updated weights for policy 1, policy_version 1807030 (0.0008) [2023-12-27 18:10:33,688][20347] Updated weights for policy 0, policy_version 1808712 (0.0010) [2023-12-27 18:10:33,746][20347] Updated weights for policy 0, policy_version 1808722 (0.0010) [2023-12-27 18:10:33,770][20381] Updated weights for policy 1, policy_version 1807040 (0.0007) [2023-12-27 18:10:33,800][20347] Updated weights for policy 0, policy_version 1808732 (0.0010) [2023-12-27 18:10:33,828][20381] Updated weights for policy 1, policy_version 1807050 (0.0005) [2023-12-27 18:10:33,887][20381] Updated weights for policy 1, policy_version 1807060 (0.0006) [2023-12-27 18:10:34,475][20347] Updated weights for policy 0, policy_version 1808742 (0.0010) [2023-12-27 18:10:34,535][20347] Updated weights for policy 0, policy_version 1808752 (0.0011) [2023-12-27 18:10:34,553][20381] Updated weights for policy 1, policy_version 1807070 (0.0007) [2023-12-27 18:10:34,595][20347] Updated weights for policy 0, policy_version 1808762 (0.0008) [2023-12-27 18:10:34,615][20381] Updated weights for policy 1, policy_version 1807080 (0.0008) [2023-12-27 18:10:34,671][20381] Updated weights for policy 1, policy_version 1807090 (0.0008) [2023-12-27 18:10:35,225][20381] Updated weights for policy 1, policy_version 1807100 (0.0007) [2023-12-27 18:10:35,280][20347] Updated weights for policy 0, policy_version 1808772 (0.0010) [2023-12-27 18:10:35,288][20381] Updated weights for policy 1, policy_version 1807110 (0.0005) [2023-12-27 18:10:35,335][20347] Updated weights for policy 0, policy_version 1808782 (0.0010) [2023-12-27 18:10:35,344][20381] Updated weights for policy 1, policy_version 1807120 (0.0005) [2023-12-27 18:10:35,391][20347] Updated weights for policy 0, policy_version 1808792 (0.0010) [2023-12-27 18:10:35,958][20381] Updated weights for policy 1, policy_version 1807130 (0.0006) [2023-12-27 18:10:36,008][20381] Updated weights for policy 1, policy_version 1807140 (0.0008) [2023-12-27 18:10:36,059][20381] Updated weights for policy 1, policy_version 1807150 (0.0007) [2023-12-27 18:10:36,065][20347] Updated weights for policy 0, policy_version 1808802 (0.0010) [2023-12-27 18:10:36,118][20381] Updated weights for policy 1, policy_version 1807160 (0.0007) [2023-12-27 18:10:36,128][20347] Updated weights for policy 0, policy_version 1808812 (0.0010) [2023-12-27 18:10:36,183][20347] Updated weights for policy 0, policy_version 1808822 (0.0008) [2023-12-27 18:10:36,251][20347] Updated weights for policy 0, policy_version 1808832 (0.0009) [2023-12-27 18:10:36,776][20381] Updated weights for policy 1, policy_version 1807170 (0.0010) [2023-12-27 18:10:36,820][20347] Updated weights for policy 0, policy_version 1808842 (0.0007) [2023-12-27 18:10:36,833][20381] Updated weights for policy 1, policy_version 1807180 (0.0011) [2023-12-27 18:10:36,867][20347] Updated weights for policy 0, policy_version 1808852 (0.0008) [2023-12-27 18:10:36,896][20381] Updated weights for policy 1, policy_version 1807190 (0.0011) [2023-12-27 18:10:36,930][20347] Updated weights for policy 0, policy_version 1808862 (0.0008) [2023-12-27 18:10:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 925843456. Throughput: 0: 10276.5, 1: 10293.3. Samples: 925812572. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:37,363][19170] Avg episode reward: [(0, '960.910'), (1, '980.570')] [2023-12-27 18:10:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001808864_463151104.pth... [2023-12-27 18:10:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001807192_462692352.pth... [2023-12-27 18:10:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001807680_462848000.pth [2023-12-27 18:10:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001805976_462381056.pth [2023-12-27 18:10:37,529][20381] Updated weights for policy 1, policy_version 1807200 (0.0008) [2023-12-27 18:10:37,545][20347] Updated weights for policy 0, policy_version 1808872 (0.0006) [2023-12-27 18:10:37,585][20381] Updated weights for policy 1, policy_version 1807210 (0.0006) [2023-12-27 18:10:37,606][20347] Updated weights for policy 0, policy_version 1808882 (0.0010) [2023-12-27 18:10:37,642][20381] Updated weights for policy 1, policy_version 1807220 (0.0006) [2023-12-27 18:10:37,659][20347] Updated weights for policy 0, policy_version 1808892 (0.0006) [2023-12-27 18:10:38,273][20381] Updated weights for policy 1, policy_version 1807230 (0.0006) [2023-12-27 18:10:38,310][20347] Updated weights for policy 0, policy_version 1808902 (0.0007) [2023-12-27 18:10:38,323][20381] Updated weights for policy 1, policy_version 1807240 (0.0006) [2023-12-27 18:10:38,373][20347] Updated weights for policy 0, policy_version 1808912 (0.0009) [2023-12-27 18:10:38,387][20381] Updated weights for policy 1, policy_version 1807250 (0.0007) [2023-12-27 18:10:38,434][20347] Updated weights for policy 0, policy_version 1808922 (0.0007) [2023-12-27 18:10:39,068][20381] Updated weights for policy 1, policy_version 1807260 (0.0006) [2023-12-27 18:10:39,131][20381] Updated weights for policy 1, policy_version 1807270 (0.0008) [2023-12-27 18:10:39,174][20347] Updated weights for policy 0, policy_version 1808932 (0.0006) [2023-12-27 18:10:39,190][20381] Updated weights for policy 1, policy_version 1807280 (0.0006) [2023-12-27 18:10:39,229][20347] Updated weights for policy 0, policy_version 1808942 (0.0009) [2023-12-27 18:10:39,290][20347] Updated weights for policy 0, policy_version 1808952 (0.0009) [2023-12-27 18:10:39,985][20381] Updated weights for policy 1, policy_version 1807290 (0.0010) [2023-12-27 18:10:40,039][20381] Updated weights for policy 1, policy_version 1807300 (0.0009) [2023-12-27 18:10:40,093][20381] Updated weights for policy 1, policy_version 1807310 (0.0009) [2023-12-27 18:10:40,096][20347] Updated weights for policy 0, policy_version 1808962 (0.0008) [2023-12-27 18:10:40,146][20381] Updated weights for policy 1, policy_version 1807320 (0.0009) [2023-12-27 18:10:40,151][20347] Updated weights for policy 0, policy_version 1808972 (0.0009) [2023-12-27 18:10:40,206][20347] Updated weights for policy 0, policy_version 1808982 (0.0008) [2023-12-27 18:10:40,273][20347] Updated weights for policy 0, policy_version 1808992 (0.0008) [2023-12-27 18:10:40,903][20381] Updated weights for policy 1, policy_version 1807330 (0.0011) [2023-12-27 18:10:40,962][20381] Updated weights for policy 1, policy_version 1807340 (0.0010) [2023-12-27 18:10:41,000][20347] Updated weights for policy 0, policy_version 1809002 (0.0009) [2023-12-27 18:10:41,015][20381] Updated weights for policy 1, policy_version 1807350 (0.0011) [2023-12-27 18:10:41,062][20347] Updated weights for policy 0, policy_version 1809012 (0.0007) [2023-12-27 18:10:41,122][20347] Updated weights for policy 0, policy_version 1809022 (0.0007) [2023-12-27 18:10:41,756][20381] Updated weights for policy 1, policy_version 1807360 (0.0008) [2023-12-27 18:10:41,809][20381] Updated weights for policy 1, policy_version 1807370 (0.0007) [2023-12-27 18:10:41,867][20347] Updated weights for policy 0, policy_version 1809032 (0.0007) [2023-12-27 18:10:41,871][20381] Updated weights for policy 1, policy_version 1807380 (0.0006) [2023-12-27 18:10:41,925][20347] Updated weights for policy 0, policy_version 1809042 (0.0006) [2023-12-27 18:10:41,977][20347] Updated weights for policy 0, policy_version 1809052 (0.0005) [2023-12-27 18:10:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 925941760. Throughput: 0: 10216.5, 1: 10298.0. Samples: 925933680. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:42,363][19170] Avg episode reward: [(0, '970.640'), (1, '970.810')] [2023-12-27 18:10:42,546][20381] Updated weights for policy 1, policy_version 1807390 (0.0006) [2023-12-27 18:10:42,601][20347] Updated weights for policy 0, policy_version 1809062 (0.0007) [2023-12-27 18:10:42,604][20381] Updated weights for policy 1, policy_version 1807400 (0.0006) [2023-12-27 18:10:42,660][20347] Updated weights for policy 0, policy_version 1809072 (0.0007) [2023-12-27 18:10:42,663][20381] Updated weights for policy 1, policy_version 1807410 (0.0006) [2023-12-27 18:10:42,717][20347] Updated weights for policy 0, policy_version 1809082 (0.0006) [2023-12-27 18:10:43,334][20381] Updated weights for policy 1, policy_version 1807420 (0.0008) [2023-12-27 18:10:43,392][20381] Updated weights for policy 1, policy_version 1807430 (0.0008) [2023-12-27 18:10:43,439][20347] Updated weights for policy 0, policy_version 1809092 (0.0008) [2023-12-27 18:10:43,451][20381] Updated weights for policy 1, policy_version 1807440 (0.0009) [2023-12-27 18:10:43,494][20347] Updated weights for policy 0, policy_version 1809102 (0.0006) [2023-12-27 18:10:43,547][20347] Updated weights for policy 0, policy_version 1809112 (0.0008) [2023-12-27 18:10:44,113][20381] Updated weights for policy 1, policy_version 1807450 (0.0006) [2023-12-27 18:10:44,167][20381] Updated weights for policy 1, policy_version 1807460 (0.0008) [2023-12-27 18:10:44,217][20381] Updated weights for policy 1, policy_version 1807470 (0.0008) [2023-12-27 18:10:44,235][20347] Updated weights for policy 0, policy_version 1809122 (0.0008) [2023-12-27 18:10:44,274][20381] Updated weights for policy 1, policy_version 1807480 (0.0009) [2023-12-27 18:10:44,290][20347] Updated weights for policy 0, policy_version 1809132 (0.0009) [2023-12-27 18:10:44,347][20347] Updated weights for policy 0, policy_version 1809142 (0.0006) [2023-12-27 18:10:44,391][20347] Updated weights for policy 0, policy_version 1809152 (0.0005) [2023-12-27 18:10:44,954][20381] Updated weights for policy 1, policy_version 1807490 (0.0008) [2023-12-27 18:10:45,013][20381] Updated weights for policy 1, policy_version 1807500 (0.0007) [2023-12-27 18:10:45,015][20347] Updated weights for policy 0, policy_version 1809162 (0.0006) [2023-12-27 18:10:45,073][20381] Updated weights for policy 1, policy_version 1807510 (0.0006) [2023-12-27 18:10:45,082][20347] Updated weights for policy 0, policy_version 1809172 (0.0008) [2023-12-27 18:10:45,145][20347] Updated weights for policy 0, policy_version 1809182 (0.0007) [2023-12-27 18:10:45,725][20381] Updated weights for policy 1, policy_version 1807520 (0.0007) [2023-12-27 18:10:45,777][20381] Updated weights for policy 1, policy_version 1807530 (0.0008) [2023-12-27 18:10:45,824][20381] Updated weights for policy 1, policy_version 1807540 (0.0007) [2023-12-27 18:10:45,840][20347] Updated weights for policy 0, policy_version 1809192 (0.0008) [2023-12-27 18:10:45,897][20347] Updated weights for policy 0, policy_version 1809202 (0.0008) [2023-12-27 18:10:45,954][20347] Updated weights for policy 0, policy_version 1809212 (0.0008) [2023-12-27 18:10:46,503][20381] Updated weights for policy 1, policy_version 1807550 (0.0008) [2023-12-27 18:10:46,559][20381] Updated weights for policy 1, policy_version 1807560 (0.0008) [2023-12-27 18:10:46,618][20381] Updated weights for policy 1, policy_version 1807570 (0.0008) [2023-12-27 18:10:46,640][20347] Updated weights for policy 0, policy_version 1809222 (0.0007) [2023-12-27 18:10:46,702][20347] Updated weights for policy 0, policy_version 1809232 (0.0007) [2023-12-27 18:10:46,760][20347] Updated weights for policy 0, policy_version 1809242 (0.0005) [2023-12-27 18:10:47,234][20381] Updated weights for policy 1, policy_version 1807580 (0.0008) [2023-12-27 18:10:47,298][20381] Updated weights for policy 1, policy_version 1807590 (0.0008) [2023-12-27 18:10:47,312][20347] Updated weights for policy 0, policy_version 1809252 (0.0005) [2023-12-27 18:10:47,357][20381] Updated weights for policy 1, policy_version 1807600 (0.0008) [2023-12-27 18:10:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 926040064. Throughput: 0: 10173.5, 1: 10344.4. Samples: 926057260. Policy #0 lag: (min: 23.0, avg: 23.7, max: 45.0) [2023-12-27 18:10:47,363][19170] Avg episode reward: [(0, '970.640'), (1, '1000.000')] [2023-12-27 18:10:47,366][20347] Updated weights for policy 0, policy_version 1809262 (0.0005) [2023-12-27 18:10:47,422][20347] Updated weights for policy 0, policy_version 1809272 (0.0005) [2023-12-27 18:10:47,997][20347] Updated weights for policy 0, policy_version 1809282 (0.0006) [2023-12-27 18:10:48,045][20381] Updated weights for policy 1, policy_version 1807610 (0.0007) [2023-12-27 18:10:48,055][20347] Updated weights for policy 0, policy_version 1809292 (0.0006) [2023-12-27 18:10:48,111][20381] Updated weights for policy 1, policy_version 1807620 (0.0008) [2023-12-27 18:10:48,113][20347] Updated weights for policy 0, policy_version 1809302 (0.0006) [2023-12-27 18:10:48,171][20347] Updated weights for policy 0, policy_version 1809312 (0.0006) [2023-12-27 18:10:48,173][20381] Updated weights for policy 1, policy_version 1807630 (0.0008) [2023-12-27 18:10:48,231][20381] Updated weights for policy 1, policy_version 1807640 (0.0008) [2023-12-27 18:10:48,837][20381] Updated weights for policy 1, policy_version 1807650 (0.0006) [2023-12-27 18:10:48,842][20347] Updated weights for policy 0, policy_version 1809322 (0.0006) [2023-12-27 18:10:48,893][20381] Updated weights for policy 1, policy_version 1807660 (0.0007) [2023-12-27 18:10:48,898][20347] Updated weights for policy 0, policy_version 1809332 (0.0006) [2023-12-27 18:10:48,951][20381] Updated weights for policy 1, policy_version 1807670 (0.0006) [2023-12-27 18:10:48,956][20347] Updated weights for policy 0, policy_version 1809342 (0.0007) [2023-12-27 18:10:49,642][20347] Updated weights for policy 0, policy_version 1809352 (0.0007) [2023-12-27 18:10:49,685][20381] Updated weights for policy 1, policy_version 1807680 (0.0006) [2023-12-27 18:10:49,695][20347] Updated weights for policy 0, policy_version 1809362 (0.0006) [2023-12-27 18:10:49,735][20381] Updated weights for policy 1, policy_version 1807690 (0.0005) [2023-12-27 18:10:49,744][20347] Updated weights for policy 0, policy_version 1809372 (0.0006) [2023-12-27 18:10:49,786][20381] Updated weights for policy 1, policy_version 1807700 (0.0006) [2023-12-27 18:10:50,494][20347] Updated weights for policy 0, policy_version 1809382 (0.0006) [2023-12-27 18:10:50,518][20381] Updated weights for policy 1, policy_version 1807710 (0.0006) [2023-12-27 18:10:50,560][20347] Updated weights for policy 0, policy_version 1809392 (0.0007) [2023-12-27 18:10:50,577][20381] Updated weights for policy 1, policy_version 1807720 (0.0007) [2023-12-27 18:10:50,634][20347] Updated weights for policy 0, policy_version 1809402 (0.0006) [2023-12-27 18:10:50,636][20381] Updated weights for policy 1, policy_version 1807730 (0.0008) [2023-12-27 18:10:51,275][20347] Updated weights for policy 0, policy_version 1809412 (0.0008) [2023-12-27 18:10:51,341][20347] Updated weights for policy 0, policy_version 1809422 (0.0008) [2023-12-27 18:10:51,410][20347] Updated weights for policy 0, policy_version 1809432 (0.0008) [2023-12-27 18:10:51,449][20381] Updated weights for policy 1, policy_version 1807740 (0.0008) [2023-12-27 18:10:51,514][20381] Updated weights for policy 1, policy_version 1807750 (0.0007) [2023-12-27 18:10:51,576][20381] Updated weights for policy 1, policy_version 1807760 (0.0007) [2023-12-27 18:10:52,141][20347] Updated weights for policy 0, policy_version 1809442 (0.0008) [2023-12-27 18:10:52,201][20347] Updated weights for policy 0, policy_version 1809452 (0.0007) [2023-12-27 18:10:52,261][20347] Updated weights for policy 0, policy_version 1809462 (0.0008) [2023-12-27 18:10:52,300][20381] Updated weights for policy 1, policy_version 1807770 (0.0007) [2023-12-27 18:10:52,323][20347] Updated weights for policy 0, policy_version 1809472 (0.0007) [2023-12-27 18:10:52,361][20381] Updated weights for policy 1, policy_version 1807780 (0.0008) [2023-12-27 18:10:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 926146560. Throughput: 0: 10170.9, 1: 10282.5. Samples: 926118860. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:10:52,363][19170] Avg episode reward: [(0, '970.690'), (1, '980.500')] [2023-12-27 18:10:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001809472_463306752.pth... [2023-12-27 18:10:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001808256_462995456.pth [2023-12-27 18:10:52,426][20381] Updated weights for policy 1, policy_version 1807790 (0.0008) [2023-12-27 18:10:52,481][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001807800_462848000.pth... [2023-12-27 18:10:52,483][20381] Updated weights for policy 1, policy_version 1807800 (0.0007) [2023-12-27 18:10:52,485][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001806584_462536704.pth [2023-12-27 18:10:53,063][20381] Updated weights for policy 1, policy_version 1807810 (0.0006) [2023-12-27 18:10:53,076][20347] Updated weights for policy 0, policy_version 1809482 (0.0007) [2023-12-27 18:10:53,109][20381] Updated weights for policy 1, policy_version 1807820 (0.0006) [2023-12-27 18:10:53,132][20347] Updated weights for policy 0, policy_version 1809492 (0.0007) [2023-12-27 18:10:53,156][20381] Updated weights for policy 1, policy_version 1807830 (0.0007) [2023-12-27 18:10:53,184][20347] Updated weights for policy 0, policy_version 1809502 (0.0006) [2023-12-27 18:10:53,772][20381] Updated weights for policy 1, policy_version 1807840 (0.0006) [2023-12-27 18:10:53,832][20381] Updated weights for policy 1, policy_version 1807850 (0.0007) [2023-12-27 18:10:53,884][20381] Updated weights for policy 1, policy_version 1807860 (0.0007) [2023-12-27 18:10:53,912][20347] Updated weights for policy 0, policy_version 1809512 (0.0007) [2023-12-27 18:10:53,968][20347] Updated weights for policy 0, policy_version 1809522 (0.0006) [2023-12-27 18:10:54,028][20347] Updated weights for policy 0, policy_version 1809532 (0.0005) [2023-12-27 18:10:54,583][20381] Updated weights for policy 1, policy_version 1807870 (0.0006) [2023-12-27 18:10:54,642][20381] Updated weights for policy 1, policy_version 1807880 (0.0006) [2023-12-27 18:10:54,682][20347] Updated weights for policy 0, policy_version 1809542 (0.0006) [2023-12-27 18:10:54,695][20381] Updated weights for policy 1, policy_version 1807890 (0.0006) [2023-12-27 18:10:54,734][20347] Updated weights for policy 0, policy_version 1809552 (0.0006) [2023-12-27 18:10:54,780][20347] Updated weights for policy 0, policy_version 1809562 (0.0007) [2023-12-27 18:10:55,327][20381] Updated weights for policy 1, policy_version 1807900 (0.0006) [2023-12-27 18:10:55,378][20381] Updated weights for policy 1, policy_version 1807910 (0.0008) [2023-12-27 18:10:55,431][20381] Updated weights for policy 1, policy_version 1807920 (0.0008) [2023-12-27 18:10:55,503][20347] Updated weights for policy 0, policy_version 1809572 (0.0008) [2023-12-27 18:10:55,556][20347] Updated weights for policy 0, policy_version 1809582 (0.0007) [2023-12-27 18:10:55,607][20347] Updated weights for policy 0, policy_version 1809592 (0.0007) [2023-12-27 18:10:56,059][20381] Updated weights for policy 1, policy_version 1807930 (0.0007) [2023-12-27 18:10:56,107][20381] Updated weights for policy 1, policy_version 1807940 (0.0006) [2023-12-27 18:10:56,152][20381] Updated weights for policy 1, policy_version 1807950 (0.0008) [2023-12-27 18:10:56,203][20381] Updated weights for policy 1, policy_version 1807960 (0.0008) [2023-12-27 18:10:56,339][20347] Updated weights for policy 0, policy_version 1809602 (0.0008) [2023-12-27 18:10:56,393][20347] Updated weights for policy 0, policy_version 1809612 (0.0007) [2023-12-27 18:10:56,450][20347] Updated weights for policy 0, policy_version 1809622 (0.0007) [2023-12-27 18:10:56,508][20347] Updated weights for policy 0, policy_version 1809632 (0.0008) [2023-12-27 18:10:56,842][20381] Updated weights for policy 1, policy_version 1807970 (0.0008) [2023-12-27 18:10:56,893][20381] Updated weights for policy 1, policy_version 1807980 (0.0007) [2023-12-27 18:10:56,956][20381] Updated weights for policy 1, policy_version 1807990 (0.0005) [2023-12-27 18:10:57,218][20347] Updated weights for policy 0, policy_version 1809642 (0.0007) [2023-12-27 18:10:57,272][20347] Updated weights for policy 0, policy_version 1809652 (0.0007) [2023-12-27 18:10:57,333][20347] Updated weights for policy 0, policy_version 1809662 (0.0007) [2023-12-27 18:10:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 926253056. Throughput: 0: 10125.1, 1: 10307.1. Samples: 926239264. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:10:57,363][19170] Avg episode reward: [(0, '961.040'), (1, '980.500')] [2023-12-27 18:10:57,567][20381] Updated weights for policy 1, policy_version 1808000 (0.0006) [2023-12-27 18:10:57,619][20381] Updated weights for policy 1, policy_version 1808010 (0.0008) [2023-12-27 18:10:57,674][20381] Updated weights for policy 1, policy_version 1808020 (0.0008) [2023-12-27 18:10:58,003][20347] Updated weights for policy 0, policy_version 1809672 (0.0006) [2023-12-27 18:10:58,047][20347] Updated weights for policy 0, policy_version 1809682 (0.0007) [2023-12-27 18:10:58,094][20347] Updated weights for policy 0, policy_version 1809692 (0.0007) [2023-12-27 18:10:58,377][20381] Updated weights for policy 1, policy_version 1808030 (0.0008) [2023-12-27 18:10:58,446][20381] Updated weights for policy 1, policy_version 1808040 (0.0007) [2023-12-27 18:10:58,513][20381] Updated weights for policy 1, policy_version 1808050 (0.0008) [2023-12-27 18:10:58,904][20347] Updated weights for policy 0, policy_version 1809702 (0.0008) [2023-12-27 18:10:58,960][20347] Updated weights for policy 0, policy_version 1809712 (0.0007) [2023-12-27 18:10:59,019][20347] Updated weights for policy 0, policy_version 1809722 (0.0007) [2023-12-27 18:10:59,207][20381] Updated weights for policy 1, policy_version 1808060 (0.0009) [2023-12-27 18:10:59,275][20381] Updated weights for policy 1, policy_version 1808070 (0.0008) [2023-12-27 18:10:59,346][20381] Updated weights for policy 1, policy_version 1808080 (0.0010) [2023-12-27 18:10:59,743][20347] Updated weights for policy 0, policy_version 1809732 (0.0008) [2023-12-27 18:10:59,810][20347] Updated weights for policy 0, policy_version 1809742 (0.0007) [2023-12-27 18:10:59,868][20347] Updated weights for policy 0, policy_version 1809752 (0.0008) [2023-12-27 18:11:00,009][20381] Updated weights for policy 1, policy_version 1808090 (0.0007) [2023-12-27 18:11:00,069][20381] Updated weights for policy 1, policy_version 1808100 (0.0010) [2023-12-27 18:11:00,128][20381] Updated weights for policy 1, policy_version 1808110 (0.0011) [2023-12-27 18:11:00,178][20381] Updated weights for policy 1, policy_version 1808120 (0.0007) [2023-12-27 18:11:00,523][20347] Updated weights for policy 0, policy_version 1809762 (0.0009) [2023-12-27 18:11:00,576][20347] Updated weights for policy 0, policy_version 1809772 (0.0007) [2023-12-27 18:11:00,626][20347] Updated weights for policy 0, policy_version 1809782 (0.0010) [2023-12-27 18:11:00,670][20347] Updated weights for policy 0, policy_version 1809792 (0.0010) [2023-12-27 18:11:00,837][20381] Updated weights for policy 1, policy_version 1808130 (0.0006) [2023-12-27 18:11:00,889][20381] Updated weights for policy 1, policy_version 1808140 (0.0009) [2023-12-27 18:11:00,932][20381] Updated weights for policy 1, policy_version 1808150 (0.0007) [2023-12-27 18:11:01,317][20347] Updated weights for policy 0, policy_version 1809802 (0.0010) [2023-12-27 18:11:01,379][20347] Updated weights for policy 0, policy_version 1809812 (0.0009) [2023-12-27 18:11:01,443][20347] Updated weights for policy 0, policy_version 1809822 (0.0010) [2023-12-27 18:11:01,607][20381] Updated weights for policy 1, policy_version 1808160 (0.0006) [2023-12-27 18:11:01,672][20381] Updated weights for policy 1, policy_version 1808170 (0.0009) [2023-12-27 18:11:01,731][20381] Updated weights for policy 1, policy_version 1808180 (0.0006) [2023-12-27 18:11:02,165][20347] Updated weights for policy 0, policy_version 1809832 (0.0006) [2023-12-27 18:11:02,223][20347] Updated weights for policy 0, policy_version 1809842 (0.0006) [2023-12-27 18:11:02,282][20347] Updated weights for policy 0, policy_version 1809852 (0.0008) [2023-12-27 18:11:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 926351360. Throughput: 0: 10093.2, 1: 10268.5. Samples: 926360580. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:02,363][19170] Avg episode reward: [(0, '961.090'), (1, '990.270')] [2023-12-27 18:11:02,428][20381] Updated weights for policy 1, policy_version 1808190 (0.0008) [2023-12-27 18:11:02,479][20381] Updated weights for policy 1, policy_version 1808200 (0.0007) [2023-12-27 18:11:02,532][20381] Updated weights for policy 1, policy_version 1808210 (0.0008) [2023-12-27 18:11:02,852][20347] Updated weights for policy 0, policy_version 1809862 (0.0007) [2023-12-27 18:11:02,912][20347] Updated weights for policy 0, policy_version 1809872 (0.0005) [2023-12-27 18:11:02,970][20347] Updated weights for policy 0, policy_version 1809882 (0.0006) [2023-12-27 18:11:03,281][20381] Updated weights for policy 1, policy_version 1808220 (0.0008) [2023-12-27 18:11:03,337][20381] Updated weights for policy 1, policy_version 1808230 (0.0008) [2023-12-27 18:11:03,389][20381] Updated weights for policy 1, policy_version 1808240 (0.0008) [2023-12-27 18:11:03,535][20347] Updated weights for policy 0, policy_version 1809892 (0.0007) [2023-12-27 18:11:03,586][20347] Updated weights for policy 0, policy_version 1809902 (0.0010) [2023-12-27 18:11:03,637][20347] Updated weights for policy 0, policy_version 1809912 (0.0010) [2023-12-27 18:11:04,111][20381] Updated weights for policy 1, policy_version 1808250 (0.0007) [2023-12-27 18:11:04,171][20381] Updated weights for policy 1, policy_version 1808260 (0.0007) [2023-12-27 18:11:04,239][20381] Updated weights for policy 1, policy_version 1808270 (0.0008) [2023-12-27 18:11:04,279][20347] Updated weights for policy 0, policy_version 1809922 (0.0009) [2023-12-27 18:11:04,302][20381] Updated weights for policy 1, policy_version 1808280 (0.0008) [2023-12-27 18:11:04,338][20347] Updated weights for policy 0, policy_version 1809932 (0.0006) [2023-12-27 18:11:04,397][20347] Updated weights for policy 0, policy_version 1809942 (0.0008) [2023-12-27 18:11:04,445][20347] Updated weights for policy 0, policy_version 1809952 (0.0010) [2023-12-27 18:11:05,006][20347] Updated weights for policy 0, policy_version 1809962 (0.0005) [2023-12-27 18:11:05,049][20381] Updated weights for policy 1, policy_version 1808290 (0.0007) [2023-12-27 18:11:05,061][20347] Updated weights for policy 0, policy_version 1809972 (0.0005) [2023-12-27 18:11:05,105][20381] Updated weights for policy 1, policy_version 1808300 (0.0009) [2023-12-27 18:11:05,116][20347] Updated weights for policy 0, policy_version 1809982 (0.0006) [2023-12-27 18:11:05,162][20381] Updated weights for policy 1, policy_version 1808310 (0.0007) [2023-12-27 18:11:05,724][20347] Updated weights for policy 0, policy_version 1809992 (0.0010) [2023-12-27 18:11:05,780][20347] Updated weights for policy 0, policy_version 1810002 (0.0010) [2023-12-27 18:11:05,831][20347] Updated weights for policy 0, policy_version 1810012 (0.0010) [2023-12-27 18:11:05,880][20381] Updated weights for policy 1, policy_version 1808320 (0.0006) [2023-12-27 18:11:05,934][20381] Updated weights for policy 1, policy_version 1808330 (0.0007) [2023-12-27 18:11:05,987][20381] Updated weights for policy 1, policy_version 1808340 (0.0007) [2023-12-27 18:11:06,488][20347] Updated weights for policy 0, policy_version 1810022 (0.0008) [2023-12-27 18:11:06,537][20347] Updated weights for policy 0, policy_version 1810032 (0.0008) [2023-12-27 18:11:06,589][20347] Updated weights for policy 0, policy_version 1810042 (0.0007) [2023-12-27 18:11:06,703][20381] Updated weights for policy 1, policy_version 1808350 (0.0007) [2023-12-27 18:11:06,723][20292] KL-divergence is very high: 108.4366 [2023-12-27 18:11:06,755][20381] Updated weights for policy 1, policy_version 1808360 (0.0007) [2023-12-27 18:11:06,766][20292] KL-divergence is very high: 172.4201 [2023-12-27 18:11:06,811][20381] Updated weights for policy 1, policy_version 1808370 (0.0009) [2023-12-27 18:11:06,812][20292] KL-divergence is very high: 133.0758 [2023-12-27 18:11:07,237][20347] Updated weights for policy 0, policy_version 1810052 (0.0007) [2023-12-27 18:11:07,288][20347] Updated weights for policy 0, policy_version 1810062 (0.0008) [2023-12-27 18:11:07,341][20347] Updated weights for policy 0, policy_version 1810072 (0.0008) [2023-12-27 18:11:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 926449664. Throughput: 0: 10225.4, 1: 10239.5. Samples: 926423476. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:07,363][19170] Avg episode reward: [(0, '970.790'), (1, '980.610')] [2023-12-27 18:11:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001808376_462995456.pth... [2023-12-27 18:11:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001807192_462692352.pth [2023-12-27 18:11:07,384][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001810080_463462400.pth... [2023-12-27 18:11:07,387][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001808864_463151104.pth [2023-12-27 18:11:07,537][20381] Updated weights for policy 1, policy_version 1808380 (0.0008) [2023-12-27 18:11:07,586][20381] Updated weights for policy 1, policy_version 1808390 (0.0008) [2023-12-27 18:11:07,631][20381] Updated weights for policy 1, policy_version 1808400 (0.0008) [2023-12-27 18:11:08,003][20347] Updated weights for policy 0, policy_version 1810082 (0.0008) [2023-12-27 18:11:08,058][20347] Updated weights for policy 0, policy_version 1810092 (0.0007) [2023-12-27 18:11:08,113][20347] Updated weights for policy 0, policy_version 1810102 (0.0008) [2023-12-27 18:11:08,165][20347] Updated weights for policy 0, policy_version 1810112 (0.0007) [2023-12-27 18:11:08,346][20381] Updated weights for policy 1, policy_version 1808410 (0.0007) [2023-12-27 18:11:08,403][20381] Updated weights for policy 1, policy_version 1808420 (0.0008) [2023-12-27 18:11:08,467][20381] Updated weights for policy 1, policy_version 1808430 (0.0008) [2023-12-27 18:11:08,523][20381] Updated weights for policy 1, policy_version 1808440 (0.0008) [2023-12-27 18:11:08,872][20347] Updated weights for policy 0, policy_version 1810122 (0.0008) [2023-12-27 18:11:08,936][20347] Updated weights for policy 0, policy_version 1810132 (0.0008) [2023-12-27 18:11:08,991][20347] Updated weights for policy 0, policy_version 1810142 (0.0008) [2023-12-27 18:11:09,198][20381] Updated weights for policy 1, policy_version 1808450 (0.0008) [2023-12-27 18:11:09,264][20381] Updated weights for policy 1, policy_version 1808460 (0.0010) [2023-12-27 18:11:09,330][20381] Updated weights for policy 1, policy_version 1808470 (0.0011) [2023-12-27 18:11:09,696][20347] Updated weights for policy 0, policy_version 1810152 (0.0008) [2023-12-27 18:11:09,747][20347] Updated weights for policy 0, policy_version 1810162 (0.0008) [2023-12-27 18:11:09,799][20347] Updated weights for policy 0, policy_version 1810172 (0.0007) [2023-12-27 18:11:10,143][20381] Updated weights for policy 1, policy_version 1808480 (0.0009) [2023-12-27 18:11:10,199][20381] Updated weights for policy 1, policy_version 1808490 (0.0011) [2023-12-27 18:11:10,253][20381] Updated weights for policy 1, policy_version 1808500 (0.0011) [2023-12-27 18:11:10,498][20347] Updated weights for policy 0, policy_version 1810182 (0.0008) [2023-12-27 18:11:10,554][20347] Updated weights for policy 0, policy_version 1810192 (0.0008) [2023-12-27 18:11:10,612][20347] Updated weights for policy 0, policy_version 1810202 (0.0007) [2023-12-27 18:11:10,885][20381] Updated weights for policy 1, policy_version 1808510 (0.0007) [2023-12-27 18:11:10,950][20381] Updated weights for policy 1, policy_version 1808520 (0.0006) [2023-12-27 18:11:11,012][20381] Updated weights for policy 1, policy_version 1808530 (0.0007) [2023-12-27 18:11:11,337][20347] Updated weights for policy 0, policy_version 1810212 (0.0009) [2023-12-27 18:11:11,405][20347] Updated weights for policy 0, policy_version 1810222 (0.0009) [2023-12-27 18:11:11,465][20347] Updated weights for policy 0, policy_version 1810232 (0.0009) [2023-12-27 18:11:11,771][20381] Updated weights for policy 1, policy_version 1808540 (0.0007) [2023-12-27 18:11:11,829][20381] Updated weights for policy 1, policy_version 1808550 (0.0008) [2023-12-27 18:11:11,883][20381] Updated weights for policy 1, policy_version 1808560 (0.0009) [2023-12-27 18:11:12,119][20347] Updated weights for policy 0, policy_version 1810242 (0.0010) [2023-12-27 18:11:12,177][20347] Updated weights for policy 0, policy_version 1810252 (0.0007) [2023-12-27 18:11:12,237][20347] Updated weights for policy 0, policy_version 1810262 (0.0008) [2023-12-27 18:11:12,308][20347] Updated weights for policy 0, policy_version 1810272 (0.0008) [2023-12-27 18:11:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 926556160. Throughput: 0: 10246.6, 1: 10192.3. Samples: 926544228. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:12,363][19170] Avg episode reward: [(0, '980.480'), (1, '980.610')] [2023-12-27 18:11:12,582][20381] Updated weights for policy 1, policy_version 1808570 (0.0008) [2023-12-27 18:11:12,644][20381] Updated weights for policy 1, policy_version 1808580 (0.0006) [2023-12-27 18:11:12,691][20381] Updated weights for policy 1, policy_version 1808590 (0.0005) [2023-12-27 18:11:12,746][20381] Updated weights for policy 1, policy_version 1808600 (0.0007) [2023-12-27 18:11:13,074][20347] Updated weights for policy 0, policy_version 1810282 (0.0008) [2023-12-27 18:11:13,140][20347] Updated weights for policy 0, policy_version 1810292 (0.0008) [2023-12-27 18:11:13,205][20347] Updated weights for policy 0, policy_version 1810302 (0.0007) [2023-12-27 18:11:13,340][20381] Updated weights for policy 1, policy_version 1808610 (0.0010) [2023-12-27 18:11:13,383][20381] Updated weights for policy 1, policy_version 1808620 (0.0010) [2023-12-27 18:11:13,431][20381] Updated weights for policy 1, policy_version 1808630 (0.0010) [2023-12-27 18:11:13,842][20347] Updated weights for policy 0, policy_version 1810312 (0.0010) [2023-12-27 18:11:13,900][20347] Updated weights for policy 0, policy_version 1810322 (0.0010) [2023-12-27 18:11:13,958][20347] Updated weights for policy 0, policy_version 1810332 (0.0010) [2023-12-27 18:11:14,121][20381] Updated weights for policy 1, policy_version 1808640 (0.0009) [2023-12-27 18:11:14,174][20381] Updated weights for policy 1, policy_version 1808650 (0.0006) [2023-12-27 18:11:14,230][20381] Updated weights for policy 1, policy_version 1808660 (0.0005) [2023-12-27 18:11:14,547][20347] Updated weights for policy 0, policy_version 1810342 (0.0008) [2023-12-27 18:11:14,607][20347] Updated weights for policy 0, policy_version 1810352 (0.0008) [2023-12-27 18:11:14,668][20347] Updated weights for policy 0, policy_version 1810362 (0.0007) [2023-12-27 18:11:14,876][20381] Updated weights for policy 1, policy_version 1808670 (0.0006) [2023-12-27 18:11:14,929][20381] Updated weights for policy 1, policy_version 1808680 (0.0008) [2023-12-27 18:11:14,985][20381] Updated weights for policy 1, policy_version 1808690 (0.0007) [2023-12-27 18:11:15,375][20347] Updated weights for policy 0, policy_version 1810372 (0.0011) [2023-12-27 18:11:15,431][20347] Updated weights for policy 0, policy_version 1810382 (0.0009) [2023-12-27 18:11:15,494][20347] Updated weights for policy 0, policy_version 1810392 (0.0010) [2023-12-27 18:11:15,705][20381] Updated weights for policy 1, policy_version 1808700 (0.0007) [2023-12-27 18:11:15,752][20381] Updated weights for policy 1, policy_version 1808710 (0.0007) [2023-12-27 18:11:15,813][20381] Updated weights for policy 1, policy_version 1808720 (0.0007) [2023-12-27 18:11:16,161][20347] Updated weights for policy 0, policy_version 1810402 (0.0010) [2023-12-27 18:11:16,240][20347] Updated weights for policy 0, policy_version 1810412 (0.0008) [2023-12-27 18:11:16,298][20347] Updated weights for policy 0, policy_version 1810422 (0.0008) [2023-12-27 18:11:16,348][20347] Updated weights for policy 0, policy_version 1810432 (0.0008) [2023-12-27 18:11:16,481][20381] Updated weights for policy 1, policy_version 1808730 (0.0007) [2023-12-27 18:11:16,524][20381] Updated weights for policy 1, policy_version 1808740 (0.0008) [2023-12-27 18:11:16,569][20381] Updated weights for policy 1, policy_version 1808750 (0.0008) [2023-12-27 18:11:16,615][20381] Updated weights for policy 1, policy_version 1808760 (0.0006) [2023-12-27 18:11:17,044][20347] Updated weights for policy 0, policy_version 1810442 (0.0009) [2023-12-27 18:11:17,091][20347] Updated weights for policy 0, policy_version 1810452 (0.0007) [2023-12-27 18:11:17,139][20347] Updated weights for policy 0, policy_version 1810462 (0.0006) [2023-12-27 18:11:17,224][20381] Updated weights for policy 1, policy_version 1808770 (0.0010) [2023-12-27 18:11:17,284][20381] Updated weights for policy 1, policy_version 1808780 (0.0008) [2023-12-27 18:11:17,335][20381] Updated weights for policy 1, policy_version 1808790 (0.0007) [2023-12-27 18:11:17,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 926662656. Throughput: 0: 10207.9, 1: 10184.6. Samples: 926667648. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:17,363][19170] Avg episode reward: [(0, '941.380'), (1, '1000.000')] [2023-12-27 18:11:17,735][20347] Updated weights for policy 0, policy_version 1810472 (0.0006) [2023-12-27 18:11:17,781][20347] Updated weights for policy 0, policy_version 1810482 (0.0007) [2023-12-27 18:11:17,843][20347] Updated weights for policy 0, policy_version 1810492 (0.0008) [2023-12-27 18:11:18,044][20381] Updated weights for policy 1, policy_version 1808800 (0.0007) [2023-12-27 18:11:18,102][20381] Updated weights for policy 1, policy_version 1808810 (0.0008) [2023-12-27 18:11:18,162][20381] Updated weights for policy 1, policy_version 1808820 (0.0007) [2023-12-27 18:11:18,592][20347] Updated weights for policy 0, policy_version 1810502 (0.0006) [2023-12-27 18:11:18,649][20347] Updated weights for policy 0, policy_version 1810512 (0.0005) [2023-12-27 18:11:18,704][20347] Updated weights for policy 0, policy_version 1810522 (0.0007) [2023-12-27 18:11:18,837][20381] Updated weights for policy 1, policy_version 1808830 (0.0008) [2023-12-27 18:11:18,885][20381] Updated weights for policy 1, policy_version 1808840 (0.0010) [2023-12-27 18:11:18,934][20381] Updated weights for policy 1, policy_version 1808850 (0.0010) [2023-12-27 18:11:19,405][20347] Updated weights for policy 0, policy_version 1810532 (0.0008) [2023-12-27 18:11:19,461][20347] Updated weights for policy 0, policy_version 1810542 (0.0008) [2023-12-27 18:11:19,526][20347] Updated weights for policy 0, policy_version 1810552 (0.0007) [2023-12-27 18:11:19,669][20381] Updated weights for policy 1, policy_version 1808860 (0.0011) [2023-12-27 18:11:19,731][20381] Updated weights for policy 1, policy_version 1808870 (0.0010) [2023-12-27 18:11:19,787][20381] Updated weights for policy 1, policy_version 1808880 (0.0011) [2023-12-27 18:11:20,253][20347] Updated weights for policy 0, policy_version 1810562 (0.0007) [2023-12-27 18:11:20,308][20347] Updated weights for policy 0, policy_version 1810572 (0.0007) [2023-12-27 18:11:20,365][20347] Updated weights for policy 0, policy_version 1810582 (0.0008) [2023-12-27 18:11:20,423][20347] Updated weights for policy 0, policy_version 1810592 (0.0007) [2023-12-27 18:11:20,510][20381] Updated weights for policy 1, policy_version 1808890 (0.0011) [2023-12-27 18:11:20,567][20381] Updated weights for policy 1, policy_version 1808900 (0.0008) [2023-12-27 18:11:20,626][20381] Updated weights for policy 1, policy_version 1808910 (0.0009) [2023-12-27 18:11:20,682][20381] Updated weights for policy 1, policy_version 1808920 (0.0007) [2023-12-27 18:11:21,162][20347] Updated weights for policy 0, policy_version 1810602 (0.0009) [2023-12-27 18:11:21,218][20347] Updated weights for policy 0, policy_version 1810612 (0.0007) [2023-12-27 18:11:21,277][20347] Updated weights for policy 0, policy_version 1810622 (0.0009) [2023-12-27 18:11:21,411][20381] Updated weights for policy 1, policy_version 1808930 (0.0009) [2023-12-27 18:11:21,471][20381] Updated weights for policy 1, policy_version 1808940 (0.0008) [2023-12-27 18:11:21,529][20381] Updated weights for policy 1, policy_version 1808950 (0.0008) [2023-12-27 18:11:22,041][20347] Updated weights for policy 0, policy_version 1810632 (0.0007) [2023-12-27 18:11:22,101][20347] Updated weights for policy 0, policy_version 1810642 (0.0008) [2023-12-27 18:11:22,160][20347] Updated weights for policy 0, policy_version 1810652 (0.0008) [2023-12-27 18:11:22,265][20381] Updated weights for policy 1, policy_version 1808960 (0.0007) [2023-12-27 18:11:22,334][20381] Updated weights for policy 1, policy_version 1808970 (0.0007) [2023-12-27 18:11:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 926752768. Throughput: 0: 10210.6, 1: 10122.6. Samples: 926727568. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:22,363][19170] Avg episode reward: [(0, '931.660'), (1, '990.330')] [2023-12-27 18:11:22,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001810656_463609856.pth... [2023-12-27 18:11:22,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001809472_463306752.pth [2023-12-27 18:11:22,402][20381] Updated weights for policy 1, policy_version 1808980 (0.0007) [2023-12-27 18:11:22,427][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001808984_463151104.pth... [2023-12-27 18:11:22,431][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001807800_462848000.pth [2023-12-27 18:11:22,893][20347] Updated weights for policy 0, policy_version 1810662 (0.0008) [2023-12-27 18:11:22,953][20347] Updated weights for policy 0, policy_version 1810672 (0.0008) [2023-12-27 18:11:23,009][20347] Updated weights for policy 0, policy_version 1810682 (0.0008) [2023-12-27 18:11:23,058][20381] Updated weights for policy 1, policy_version 1808990 (0.0007) [2023-12-27 18:11:23,120][20381] Updated weights for policy 1, policy_version 1809000 (0.0007) [2023-12-27 18:11:23,187][20381] Updated weights for policy 1, policy_version 1809010 (0.0006) [2023-12-27 18:11:23,702][20347] Updated weights for policy 0, policy_version 1810692 (0.0007) [2023-12-27 18:11:23,749][20347] Updated weights for policy 0, policy_version 1810702 (0.0007) [2023-12-27 18:11:23,804][20381] Updated weights for policy 1, policy_version 1809020 (0.0010) [2023-12-27 18:11:23,809][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-27 18:11:23,810][20347] Updated weights for policy 0, policy_version 1810712 (0.0007) [2023-12-27 18:11:23,853][20381] Updated weights for policy 1, policy_version 1809030 (0.0010) [2023-12-27 18:11:23,903][20381] Updated weights for policy 1, policy_version 1809040 (0.0008) [2023-12-27 18:11:24,433][20347] Updated weights for policy 0, policy_version 1810722 (0.0008) [2023-12-27 18:11:24,494][20347] Updated weights for policy 0, policy_version 1810732 (0.0006) [2023-12-27 18:11:24,557][20347] Updated weights for policy 0, policy_version 1810742 (0.0007) [2023-12-27 18:11:24,581][20381] Updated weights for policy 1, policy_version 1809050 (0.0007) [2023-12-27 18:11:24,631][20381] Updated weights for policy 1, policy_version 1809060 (0.0006) [2023-12-27 18:11:24,689][20381] Updated weights for policy 1, policy_version 1809070 (0.0008) [2023-12-27 18:11:24,750][20381] Updated weights for policy 1, policy_version 1809080 (0.0007) [2023-12-27 18:11:25,194][20347] Updated weights for policy 0, policy_version 1810752 (0.0010) [2023-12-27 18:11:25,246][20347] Updated weights for policy 0, policy_version 1810762 (0.0010) [2023-12-27 18:11:25,308][20347] Updated weights for policy 0, policy_version 1810772 (0.0011) [2023-12-27 18:11:25,368][20381] Updated weights for policy 1, policy_version 1809090 (0.0007) [2023-12-27 18:11:25,425][20381] Updated weights for policy 1, policy_version 1809100 (0.0009) [2023-12-27 18:11:25,476][20381] Updated weights for policy 1, policy_version 1809110 (0.0005) [2023-12-27 18:11:25,904][20347] Updated weights for policy 0, policy_version 1810782 (0.0007) [2023-12-27 18:11:25,961][20347] Updated weights for policy 0, policy_version 1810792 (0.0005) [2023-12-27 18:11:26,025][20347] Updated weights for policy 0, policy_version 1810802 (0.0009) [2023-12-27 18:11:26,074][20381] Updated weights for policy 1, policy_version 1809120 (0.0009) [2023-12-27 18:11:26,125][20381] Updated weights for policy 1, policy_version 1809130 (0.0010) [2023-12-27 18:11:26,183][20381] Updated weights for policy 1, policy_version 1809140 (0.0010) [2023-12-27 18:11:26,624][20347] Updated weights for policy 0, policy_version 1810812 (0.0007) [2023-12-27 18:11:26,676][20347] Updated weights for policy 0, policy_version 1810822 (0.0010) [2023-12-27 18:11:26,730][20347] Updated weights for policy 0, policy_version 1810832 (0.0010) [2023-12-27 18:11:26,867][20381] Updated weights for policy 1, policy_version 1809150 (0.0008) [2023-12-27 18:11:26,922][20381] Updated weights for policy 1, policy_version 1809160 (0.0006) [2023-12-27 18:11:26,981][20381] Updated weights for policy 1, policy_version 1809170 (0.0005) [2023-12-27 18:11:27,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 926859264. Throughput: 0: 10215.3, 1: 10162.0. Samples: 926850656. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:27,363][19170] Avg episode reward: [(0, '970.710'), (1, '980.580')] [2023-12-27 18:11:27,407][20347] Updated weights for policy 0, policy_version 1810842 (0.0010) [2023-12-27 18:11:27,451][20347] Updated weights for policy 0, policy_version 1810852 (0.0010) [2023-12-27 18:11:27,505][20347] Updated weights for policy 0, policy_version 1810862 (0.0010) [2023-12-27 18:11:27,527][20381] Updated weights for policy 1, policy_version 1809180 (0.0007) [2023-12-27 18:11:27,556][20347] Updated weights for policy 0, policy_version 1810872 (0.0010) [2023-12-27 18:11:27,577][20381] Updated weights for policy 1, policy_version 1809190 (0.0010) [2023-12-27 18:11:27,638][20381] Updated weights for policy 1, policy_version 1809200 (0.0010) [2023-12-27 18:11:28,179][20347] Updated weights for policy 0, policy_version 1810882 (0.0005) [2023-12-27 18:11:28,233][20347] Updated weights for policy 0, policy_version 1810892 (0.0005) [2023-12-27 18:11:28,291][20347] Updated weights for policy 0, policy_version 1810902 (0.0005) [2023-12-27 18:11:28,316][20381] Updated weights for policy 1, policy_version 1809210 (0.0010) [2023-12-27 18:11:28,378][20381] Updated weights for policy 1, policy_version 1809220 (0.0008) [2023-12-27 18:11:28,440][20381] Updated weights for policy 1, policy_version 1809230 (0.0008) [2023-12-27 18:11:28,500][20381] Updated weights for policy 1, policy_version 1809240 (0.0008) [2023-12-27 18:11:28,909][20347] Updated weights for policy 0, policy_version 1810912 (0.0010) [2023-12-27 18:11:28,963][20347] Updated weights for policy 0, policy_version 1810922 (0.0010) [2023-12-27 18:11:29,014][20347] Updated weights for policy 0, policy_version 1810932 (0.0010) [2023-12-27 18:11:29,106][20381] Updated weights for policy 1, policy_version 1809250 (0.0010) [2023-12-27 18:11:29,150][20381] Updated weights for policy 1, policy_version 1809260 (0.0010) [2023-12-27 18:11:29,195][20381] Updated weights for policy 1, policy_version 1809270 (0.0010) [2023-12-27 18:11:29,698][20347] Updated weights for policy 0, policy_version 1810942 (0.0010) [2023-12-27 18:11:29,759][20347] Updated weights for policy 0, policy_version 1810952 (0.0010) [2023-12-27 18:11:29,813][20347] Updated weights for policy 0, policy_version 1810962 (0.0008) [2023-12-27 18:11:29,849][20381] Updated weights for policy 1, policy_version 1809280 (0.0009) [2023-12-27 18:11:29,902][20381] Updated weights for policy 1, policy_version 1809290 (0.0009) [2023-12-27 18:11:29,957][20381] Updated weights for policy 1, policy_version 1809300 (0.0009) [2023-12-27 18:11:30,528][20347] Updated weights for policy 0, policy_version 1810972 (0.0007) [2023-12-27 18:11:30,579][20347] Updated weights for policy 0, policy_version 1810982 (0.0006) [2023-12-27 18:11:30,630][20381] Updated weights for policy 1, policy_version 1809310 (0.0008) [2023-12-27 18:11:30,638][20347] Updated weights for policy 0, policy_version 1810992 (0.0007) [2023-12-27 18:11:30,684][20381] Updated weights for policy 1, policy_version 1809320 (0.0005) [2023-12-27 18:11:30,739][20381] Updated weights for policy 1, policy_version 1809330 (0.0005) [2023-12-27 18:11:31,307][20347] Updated weights for policy 0, policy_version 1811002 (0.0008) [2023-12-27 18:11:31,375][20347] Updated weights for policy 0, policy_version 1811012 (0.0007) [2023-12-27 18:11:31,384][20381] Updated weights for policy 1, policy_version 1809340 (0.0007) [2023-12-27 18:11:31,438][20347] Updated weights for policy 0, policy_version 1811022 (0.0009) [2023-12-27 18:11:31,447][20381] Updated weights for policy 1, policy_version 1809350 (0.0007) [2023-12-27 18:11:31,493][20347] Updated weights for policy 0, policy_version 1811032 (0.0007) [2023-12-27 18:11:31,503][20381] Updated weights for policy 1, policy_version 1809360 (0.0006) [2023-12-27 18:11:32,212][20347] Updated weights for policy 0, policy_version 1811042 (0.0007) [2023-12-27 18:11:32,227][20381] Updated weights for policy 1, policy_version 1809370 (0.0008) [2023-12-27 18:11:32,272][20347] Updated weights for policy 0, policy_version 1811052 (0.0007) [2023-12-27 18:11:32,292][20381] Updated weights for policy 1, policy_version 1809380 (0.0011) [2023-12-27 18:11:32,323][20347] Updated weights for policy 0, policy_version 1811062 (0.0007) [2023-12-27 18:11:32,353][20381] Updated weights for policy 1, policy_version 1809390 (0.0009) [2023-12-27 18:11:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 926965760. Throughput: 0: 10238.6, 1: 10195.0. Samples: 926976772. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:32,363][19170] Avg episode reward: [(0, '921.730'), (1, '980.550')] [2023-12-27 18:11:32,410][20381] Updated weights for policy 1, policy_version 1809400 (0.0011) [2023-12-27 18:11:33,030][20347] Updated weights for policy 0, policy_version 1811072 (0.0007) [2023-12-27 18:11:33,083][20347] Updated weights for policy 0, policy_version 1811082 (0.0006) [2023-12-27 18:11:33,108][20381] Updated weights for policy 1, policy_version 1809410 (0.0007) [2023-12-27 18:11:33,133][20347] Updated weights for policy 0, policy_version 1811092 (0.0005) [2023-12-27 18:11:33,166][20381] Updated weights for policy 1, policy_version 1809420 (0.0007) [2023-12-27 18:11:33,226][20381] Updated weights for policy 1, policy_version 1809430 (0.0008) [2023-12-27 18:11:33,826][20347] Updated weights for policy 0, policy_version 1811102 (0.0006) [2023-12-27 18:11:33,847][20381] Updated weights for policy 1, policy_version 1809440 (0.0010) [2023-12-27 18:11:33,871][20347] Updated weights for policy 0, policy_version 1811112 (0.0005) [2023-12-27 18:11:33,902][20381] Updated weights for policy 1, policy_version 1809450 (0.0010) [2023-12-27 18:11:33,922][20347] Updated weights for policy 0, policy_version 1811122 (0.0005) [2023-12-27 18:11:33,954][20381] Updated weights for policy 1, policy_version 1809460 (0.0010) [2023-12-27 18:11:34,639][20347] Updated weights for policy 0, policy_version 1811132 (0.0005) [2023-12-27 18:11:34,643][20381] Updated weights for policy 1, policy_version 1809470 (0.0011) [2023-12-27 18:11:34,700][20347] Updated weights for policy 0, policy_version 1811142 (0.0008) [2023-12-27 18:11:34,703][20381] Updated weights for policy 1, policy_version 1809480 (0.0011) [2023-12-27 18:11:34,756][20381] Updated weights for policy 1, policy_version 1809490 (0.0010) [2023-12-27 18:11:34,761][20347] Updated weights for policy 0, policy_version 1811152 (0.0005) [2023-12-27 18:11:35,335][20347] Updated weights for policy 0, policy_version 1811162 (0.0006) [2023-12-27 18:11:35,389][20347] Updated weights for policy 0, policy_version 1811172 (0.0005) [2023-12-27 18:11:35,438][20347] Updated weights for policy 0, policy_version 1811182 (0.0005) [2023-12-27 18:11:35,449][20381] Updated weights for policy 1, policy_version 1809500 (0.0011) [2023-12-27 18:11:35,488][20347] Updated weights for policy 0, policy_version 1811192 (0.0005) [2023-12-27 18:11:35,498][20381] Updated weights for policy 1, policy_version 1809510 (0.0010) [2023-12-27 18:11:35,553][20381] Updated weights for policy 1, policy_version 1809520 (0.0010) [2023-12-27 18:11:36,081][20347] Updated weights for policy 0, policy_version 1811202 (0.0007) [2023-12-27 18:11:36,145][20347] Updated weights for policy 0, policy_version 1811212 (0.0008) [2023-12-27 18:11:36,205][20347] Updated weights for policy 0, policy_version 1811222 (0.0007) [2023-12-27 18:11:36,231][20381] Updated weights for policy 1, policy_version 1809530 (0.0010) [2023-12-27 18:11:36,294][20381] Updated weights for policy 1, policy_version 1809540 (0.0007) [2023-12-27 18:11:36,351][20381] Updated weights for policy 1, policy_version 1809550 (0.0007) [2023-12-27 18:11:36,409][20381] Updated weights for policy 1, policy_version 1809560 (0.0006) [2023-12-27 18:11:36,949][20347] Updated weights for policy 0, policy_version 1811232 (0.0008) [2023-12-27 18:11:37,004][20347] Updated weights for policy 0, policy_version 1811242 (0.0007) [2023-12-27 18:11:37,047][20381] Updated weights for policy 1, policy_version 1809570 (0.0008) [2023-12-27 18:11:37,053][20347] Updated weights for policy 0, policy_version 1811252 (0.0006) [2023-12-27 18:11:37,098][20381] Updated weights for policy 1, policy_version 1809580 (0.0007) [2023-12-27 18:11:37,151][20381] Updated weights for policy 1, policy_version 1809590 (0.0008) [2023-12-27 18:11:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 927072256. Throughput: 0: 10202.5, 1: 10209.0. Samples: 927037380. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:37,363][19170] Avg episode reward: [(0, '882.630'), (1, '951.390')] [2023-12-27 18:11:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001809592_463306752.pth... [2023-12-27 18:11:37,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001811256_463765504.pth... [2023-12-27 18:11:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001808376_462995456.pth [2023-12-27 18:11:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001810080_463462400.pth [2023-12-27 18:11:37,775][20347] Updated weights for policy 0, policy_version 1811262 (0.0008) [2023-12-27 18:11:37,803][20381] Updated weights for policy 1, policy_version 1809600 (0.0006) [2023-12-27 18:11:37,840][20347] Updated weights for policy 0, policy_version 1811272 (0.0010) [2023-12-27 18:11:37,851][20381] Updated weights for policy 1, policy_version 1809610 (0.0006) [2023-12-27 18:11:37,894][20347] Updated weights for policy 0, policy_version 1811282 (0.0009) [2023-12-27 18:11:37,902][20381] Updated weights for policy 1, policy_version 1809620 (0.0006) [2023-12-27 18:11:38,475][20347] Updated weights for policy 0, policy_version 1811292 (0.0008) [2023-12-27 18:11:38,527][20347] Updated weights for policy 0, policy_version 1811302 (0.0010) [2023-12-27 18:11:38,559][20381] Updated weights for policy 1, policy_version 1809630 (0.0008) [2023-12-27 18:11:38,582][20347] Updated weights for policy 0, policy_version 1811312 (0.0010) [2023-12-27 18:11:38,616][20381] Updated weights for policy 1, policy_version 1809640 (0.0008) [2023-12-27 18:11:38,672][20381] Updated weights for policy 1, policy_version 1809650 (0.0007) [2023-12-27 18:11:39,208][20347] Updated weights for policy 0, policy_version 1811322 (0.0009) [2023-12-27 18:11:39,237][20381] Updated weights for policy 1, policy_version 1809660 (0.0006) [2023-12-27 18:11:39,268][20347] Updated weights for policy 0, policy_version 1811332 (0.0009) [2023-12-27 18:11:39,299][20381] Updated weights for policy 1, policy_version 1809670 (0.0006) [2023-12-27 18:11:39,326][20347] Updated weights for policy 0, policy_version 1811342 (0.0011) [2023-12-27 18:11:39,363][20381] Updated weights for policy 1, policy_version 1809680 (0.0008) [2023-12-27 18:11:39,392][20347] Updated weights for policy 0, policy_version 1811352 (0.0011) [2023-12-27 18:11:40,082][20347] Updated weights for policy 0, policy_version 1811362 (0.0010) [2023-12-27 18:11:40,089][20381] Updated weights for policy 1, policy_version 1809690 (0.0008) [2023-12-27 18:11:40,146][20347] Updated weights for policy 0, policy_version 1811372 (0.0008) [2023-12-27 18:11:40,152][20381] Updated weights for policy 1, policy_version 1809700 (0.0007) [2023-12-27 18:11:40,212][20381] Updated weights for policy 1, policy_version 1809710 (0.0007) [2023-12-27 18:11:40,214][20347] Updated weights for policy 0, policy_version 1811382 (0.0011) [2023-12-27 18:11:40,271][20381] Updated weights for policy 1, policy_version 1809720 (0.0007) [2023-12-27 18:11:40,868][20381] Updated weights for policy 1, policy_version 1809730 (0.0009) [2023-12-27 18:11:40,879][20347] Updated weights for policy 0, policy_version 1811392 (0.0009) [2023-12-27 18:11:40,920][20381] Updated weights for policy 1, policy_version 1809740 (0.0006) [2023-12-27 18:11:40,929][20347] Updated weights for policy 0, policy_version 1811402 (0.0006) [2023-12-27 18:11:40,967][20381] Updated weights for policy 1, policy_version 1809750 (0.0005) [2023-12-27 18:11:40,979][20347] Updated weights for policy 0, policy_version 1811412 (0.0005) [2023-12-27 18:11:41,648][20347] Updated weights for policy 0, policy_version 1811423 (0.0007) [2023-12-27 18:11:41,707][20347] Updated weights for policy 0, policy_version 1811433 (0.0011) [2023-12-27 18:11:41,740][20381] Updated weights for policy 1, policy_version 1809760 (0.0007) [2023-12-27 18:11:41,770][20347] Updated weights for policy 0, policy_version 1811443 (0.0010) [2023-12-27 18:11:41,800][20381] Updated weights for policy 1, policy_version 1809770 (0.0007) [2023-12-27 18:11:41,852][20381] Updated weights for policy 1, policy_version 1809780 (0.0007) [2023-12-27 18:11:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 927170560. Throughput: 0: 10306.0, 1: 10235.4. Samples: 927163632. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:42,363][19170] Avg episode reward: [(0, '912.020'), (1, '951.450')] [2023-12-27 18:11:42,498][20381] Updated weights for policy 1, policy_version 1809790 (0.0007) [2023-12-27 18:11:42,533][20347] Updated weights for policy 0, policy_version 1811453 (0.0009) [2023-12-27 18:11:42,557][20381] Updated weights for policy 1, policy_version 1809800 (0.0009) [2023-12-27 18:11:42,593][20347] Updated weights for policy 0, policy_version 1811463 (0.0008) [2023-12-27 18:11:42,618][20381] Updated weights for policy 1, policy_version 1809810 (0.0007) [2023-12-27 18:11:42,653][20347] Updated weights for policy 0, policy_version 1811473 (0.0008) [2023-12-27 18:11:43,239][20381] Updated weights for policy 1, policy_version 1809820 (0.0006) [2023-12-27 18:11:43,297][20381] Updated weights for policy 1, policy_version 1809830 (0.0005) [2023-12-27 18:11:43,340][20347] Updated weights for policy 0, policy_version 1811483 (0.0008) [2023-12-27 18:11:43,354][20381] Updated weights for policy 1, policy_version 1809840 (0.0006) [2023-12-27 18:11:43,390][20347] Updated weights for policy 0, policy_version 1811493 (0.0006) [2023-12-27 18:11:43,443][20347] Updated weights for policy 0, policy_version 1811503 (0.0009) [2023-12-27 18:11:43,996][20381] Updated weights for policy 1, policy_version 1809850 (0.0006) [2023-12-27 18:11:44,054][20381] Updated weights for policy 1, policy_version 1809860 (0.0005) [2023-12-27 18:11:44,113][20381] Updated weights for policy 1, policy_version 1809870 (0.0007) [2023-12-27 18:11:44,160][20347] Updated weights for policy 0, policy_version 1811513 (0.0008) [2023-12-27 18:11:44,167][20381] Updated weights for policy 1, policy_version 1809880 (0.0007) [2023-12-27 18:11:44,210][20347] Updated weights for policy 0, policy_version 1811523 (0.0008) [2023-12-27 18:11:44,261][20347] Updated weights for policy 0, policy_version 1811533 (0.0007) [2023-12-27 18:11:44,319][20347] Updated weights for policy 0, policy_version 1811543 (0.0008) [2023-12-27 18:11:44,783][20381] Updated weights for policy 1, policy_version 1809890 (0.0007) [2023-12-27 18:11:44,847][20381] Updated weights for policy 1, policy_version 1809900 (0.0008) [2023-12-27 18:11:44,903][20381] Updated weights for policy 1, policy_version 1809910 (0.0008) [2023-12-27 18:11:45,020][20347] Updated weights for policy 0, policy_version 1811553 (0.0009) [2023-12-27 18:11:45,077][20347] Updated weights for policy 0, policy_version 1811563 (0.0007) [2023-12-27 18:11:45,132][20347] Updated weights for policy 0, policy_version 1811573 (0.0008) [2023-12-27 18:11:45,631][20381] Updated weights for policy 1, policy_version 1809920 (0.0007) [2023-12-27 18:11:45,687][20381] Updated weights for policy 1, policy_version 1809930 (0.0008) [2023-12-27 18:11:45,741][20381] Updated weights for policy 1, policy_version 1809940 (0.0008) [2023-12-27 18:11:45,825][20347] Updated weights for policy 0, policy_version 1811583 (0.0008) [2023-12-27 18:11:45,880][20347] Updated weights for policy 0, policy_version 1811593 (0.0010) [2023-12-27 18:11:45,935][20347] Updated weights for policy 0, policy_version 1811603 (0.0010) [2023-12-27 18:11:46,399][20381] Updated weights for policy 1, policy_version 1809950 (0.0007) [2023-12-27 18:11:46,460][20381] Updated weights for policy 1, policy_version 1809960 (0.0008) [2023-12-27 18:11:46,510][20381] Updated weights for policy 1, policy_version 1809970 (0.0007) [2023-12-27 18:11:46,576][20347] Updated weights for policy 0, policy_version 1811613 (0.0010) [2023-12-27 18:11:46,641][20347] Updated weights for policy 0, policy_version 1811623 (0.0009) [2023-12-27 18:11:46,707][20347] Updated weights for policy 0, policy_version 1811633 (0.0005) [2023-12-27 18:11:47,153][20381] Updated weights for policy 1, policy_version 1809980 (0.0007) [2023-12-27 18:11:47,218][20381] Updated weights for policy 1, policy_version 1809990 (0.0008) [2023-12-27 18:11:47,271][20347] Updated weights for policy 0, policy_version 1811643 (0.0007) [2023-12-27 18:11:47,273][20381] Updated weights for policy 1, policy_version 1810000 (0.0005) [2023-12-27 18:11:47,330][20347] Updated weights for policy 0, policy_version 1811653 (0.0010) [2023-12-27 18:11:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 927277056. Throughput: 0: 10333.9, 1: 10248.4. Samples: 927286784. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:47,363][19170] Avg episode reward: [(0, '941.410'), (1, '980.610')] [2023-12-27 18:11:47,382][20347] Updated weights for policy 0, policy_version 1811663 (0.0009) [2023-12-27 18:11:47,948][20347] Updated weights for policy 0, policy_version 1811673 (0.0006) [2023-12-27 18:11:47,949][20381] Updated weights for policy 1, policy_version 1810010 (0.0005) [2023-12-27 18:11:48,006][20347] Updated weights for policy 0, policy_version 1811683 (0.0005) [2023-12-27 18:11:48,019][20381] Updated weights for policy 1, policy_version 1810020 (0.0006) [2023-12-27 18:11:48,063][20347] Updated weights for policy 0, policy_version 1811693 (0.0006) [2023-12-27 18:11:48,068][20381] Updated weights for policy 1, policy_version 1810030 (0.0007) [2023-12-27 18:11:48,117][20381] Updated weights for policy 1, policy_version 1810040 (0.0007) [2023-12-27 18:11:48,120][20347] Updated weights for policy 0, policy_version 1811703 (0.0005) [2023-12-27 18:11:48,762][20347] Updated weights for policy 0, policy_version 1811713 (0.0010) [2023-12-27 18:11:48,776][20381] Updated weights for policy 1, policy_version 1810050 (0.0006) [2023-12-27 18:11:48,828][20347] Updated weights for policy 0, policy_version 1811723 (0.0006) [2023-12-27 18:11:48,840][20381] Updated weights for policy 1, policy_version 1810060 (0.0005) [2023-12-27 18:11:48,892][20347] Updated weights for policy 0, policy_version 1811733 (0.0011) [2023-12-27 18:11:48,904][20381] Updated weights for policy 1, policy_version 1810070 (0.0011) [2023-12-27 18:11:49,478][20347] Updated weights for policy 0, policy_version 1811743 (0.0010) [2023-12-27 18:11:49,536][20347] Updated weights for policy 0, policy_version 1811753 (0.0010) [2023-12-27 18:11:49,546][20381] Updated weights for policy 1, policy_version 1810080 (0.0011) [2023-12-27 18:11:49,596][20347] Updated weights for policy 0, policy_version 1811763 (0.0010) [2023-12-27 18:11:49,636][20381] Updated weights for policy 1, policy_version 1810090 (0.0010) [2023-12-27 18:11:49,698][20381] Updated weights for policy 1, policy_version 1810100 (0.0011) [2023-12-27 18:11:50,288][20347] Updated weights for policy 0, policy_version 1811773 (0.0009) [2023-12-27 18:11:50,343][20347] Updated weights for policy 0, policy_version 1811783 (0.0010) [2023-12-27 18:11:50,402][20347] Updated weights for policy 0, policy_version 1811793 (0.0010) [2023-12-27 18:11:50,421][20381] Updated weights for policy 1, policy_version 1810110 (0.0011) [2023-12-27 18:11:50,473][20381] Updated weights for policy 1, policy_version 1810120 (0.0011) [2023-12-27 18:11:50,521][20381] Updated weights for policy 1, policy_version 1810130 (0.0010) [2023-12-27 18:11:51,106][20347] Updated weights for policy 0, policy_version 1811803 (0.0009) [2023-12-27 18:11:51,169][20347] Updated weights for policy 0, policy_version 1811813 (0.0006) [2023-12-27 18:11:51,189][20381] Updated weights for policy 1, policy_version 1810140 (0.0006) [2023-12-27 18:11:51,229][20347] Updated weights for policy 0, policy_version 1811823 (0.0009) [2023-12-27 18:11:51,247][20381] Updated weights for policy 1, policy_version 1810150 (0.0008) [2023-12-27 18:11:51,314][20381] Updated weights for policy 1, policy_version 1810160 (0.0007) [2023-12-27 18:11:51,952][20347] Updated weights for policy 0, policy_version 1811833 (0.0010) [2023-12-27 18:11:51,990][20381] Updated weights for policy 1, policy_version 1810170 (0.0007) [2023-12-27 18:11:52,008][20347] Updated weights for policy 0, policy_version 1811843 (0.0007) [2023-12-27 18:11:52,037][20381] Updated weights for policy 1, policy_version 1810180 (0.0006) [2023-12-27 18:11:52,064][20347] Updated weights for policy 0, policy_version 1811853 (0.0007) [2023-12-27 18:11:52,090][20381] Updated weights for policy 1, policy_version 1810190 (0.0006) [2023-12-27 18:11:52,120][20347] Updated weights for policy 0, policy_version 1811863 (0.0007) [2023-12-27 18:11:52,147][20381] Updated weights for policy 1, policy_version 1810200 (0.0006) [2023-12-27 18:11:52,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 927383552. Throughput: 0: 10290.9, 1: 10300.8. Samples: 927350108. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:52,363][19170] Avg episode reward: [(0, '912.000'), (1, '990.240')] [2023-12-27 18:11:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001810200_463462400.pth... [2023-12-27 18:11:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001811864_463921152.pth... [2023-12-27 18:11:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001808984_463151104.pth [2023-12-27 18:11:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001810656_463609856.pth [2023-12-27 18:11:52,787][20381] Updated weights for policy 1, policy_version 1810210 (0.0009) [2023-12-27 18:11:52,837][20381] Updated weights for policy 1, policy_version 1810220 (0.0007) [2023-12-27 18:11:52,839][20347] Updated weights for policy 0, policy_version 1811873 (0.0005) [2023-12-27 18:11:52,892][20381] Updated weights for policy 1, policy_version 1810230 (0.0007) [2023-12-27 18:11:52,893][20347] Updated weights for policy 0, policy_version 1811883 (0.0005) [2023-12-27 18:11:52,941][20347] Updated weights for policy 0, policy_version 1811893 (0.0005) [2023-12-27 18:11:53,472][20347] Updated weights for policy 0, policy_version 1811903 (0.0006) [2023-12-27 18:11:53,524][20347] Updated weights for policy 0, policy_version 1811913 (0.0006) [2023-12-27 18:11:53,578][20347] Updated weights for policy 0, policy_version 1811923 (0.0006) [2023-12-27 18:11:53,666][20381] Updated weights for policy 1, policy_version 1810240 (0.0007) [2023-12-27 18:11:53,724][20381] Updated weights for policy 1, policy_version 1810250 (0.0008) [2023-12-27 18:11:53,788][20381] Updated weights for policy 1, policy_version 1810260 (0.0008) [2023-12-27 18:11:54,155][20347] Updated weights for policy 0, policy_version 1811933 (0.0010) [2023-12-27 18:11:54,207][20347] Updated weights for policy 0, policy_version 1811943 (0.0009) [2023-12-27 18:11:54,263][20347] Updated weights for policy 0, policy_version 1811953 (0.0011) [2023-12-27 18:11:54,446][20381] Updated weights for policy 1, policy_version 1810270 (0.0008) [2023-12-27 18:11:54,502][20381] Updated weights for policy 1, policy_version 1810280 (0.0008) [2023-12-27 18:11:54,562][20381] Updated weights for policy 1, policy_version 1810290 (0.0007) [2023-12-27 18:11:54,968][20347] Updated weights for policy 0, policy_version 1811963 (0.0010) [2023-12-27 18:11:55,028][20347] Updated weights for policy 0, policy_version 1811973 (0.0010) [2023-12-27 18:11:55,085][20347] Updated weights for policy 0, policy_version 1811983 (0.0010) [2023-12-27 18:11:55,282][20381] Updated weights for policy 1, policy_version 1810300 (0.0008) [2023-12-27 18:11:55,329][20381] Updated weights for policy 1, policy_version 1810310 (0.0007) [2023-12-27 18:11:55,378][20381] Updated weights for policy 1, policy_version 1810320 (0.0007) [2023-12-27 18:11:55,717][20347] Updated weights for policy 0, policy_version 1811993 (0.0010) [2023-12-27 18:11:55,769][20347] Updated weights for policy 0, policy_version 1812003 (0.0007) [2023-12-27 18:11:55,817][20347] Updated weights for policy 0, policy_version 1812013 (0.0007) [2023-12-27 18:11:55,861][20347] Updated weights for policy 0, policy_version 1812023 (0.0007) [2023-12-27 18:11:56,038][20381] Updated weights for policy 1, policy_version 1810330 (0.0005) [2023-12-27 18:11:56,088][20381] Updated weights for policy 1, policy_version 1810340 (0.0008) [2023-12-27 18:11:56,134][20381] Updated weights for policy 1, policy_version 1810350 (0.0008) [2023-12-27 18:11:56,191][20381] Updated weights for policy 1, policy_version 1810360 (0.0008) [2023-12-27 18:11:56,536][20347] Updated weights for policy 0, policy_version 1812033 (0.0006) [2023-12-27 18:11:56,587][20347] Updated weights for policy 0, policy_version 1812043 (0.0009) [2023-12-27 18:11:56,631][20347] Updated weights for policy 0, policy_version 1812053 (0.0009) [2023-12-27 18:11:56,845][20381] Updated weights for policy 1, policy_version 1810370 (0.0005) [2023-12-27 18:11:56,893][20381] Updated weights for policy 1, policy_version 1810380 (0.0005) [2023-12-27 18:11:56,944][20381] Updated weights for policy 1, policy_version 1810390 (0.0007) [2023-12-27 18:11:57,260][20347] Updated weights for policy 0, policy_version 1812063 (0.0008) [2023-12-27 18:11:57,309][20347] Updated weights for policy 0, policy_version 1812073 (0.0008) [2023-12-27 18:11:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 927481856. Throughput: 0: 10304.8, 1: 10330.1. Samples: 927472796. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:11:57,363][19170] Avg episode reward: [(0, '941.310'), (1, '1000.000')] [2023-12-27 18:11:57,364][20347] Updated weights for policy 0, policy_version 1812083 (0.0008) [2023-12-27 18:11:57,615][20381] Updated weights for policy 1, policy_version 1810400 (0.0006) [2023-12-27 18:11:57,673][20381] Updated weights for policy 1, policy_version 1810410 (0.0005) [2023-12-27 18:11:57,729][20381] Updated weights for policy 1, policy_version 1810420 (0.0005) [2023-12-27 18:11:58,090][20347] Updated weights for policy 0, policy_version 1812093 (0.0006) [2023-12-27 18:11:58,145][20347] Updated weights for policy 0, policy_version 1812103 (0.0005) [2023-12-27 18:11:58,209][20347] Updated weights for policy 0, policy_version 1812113 (0.0008) [2023-12-27 18:11:58,363][20381] Updated weights for policy 1, policy_version 1810430 (0.0007) [2023-12-27 18:11:58,429][20381] Updated weights for policy 1, policy_version 1810440 (0.0007) [2023-12-27 18:11:58,490][20381] Updated weights for policy 1, policy_version 1810450 (0.0009) [2023-12-27 18:11:58,984][20347] Updated weights for policy 0, policy_version 1812123 (0.0006) [2023-12-27 18:11:59,037][20347] Updated weights for policy 0, policy_version 1812133 (0.0008) [2023-12-27 18:11:59,095][20347] Updated weights for policy 0, policy_version 1812143 (0.0007) [2023-12-27 18:11:59,294][20381] Updated weights for policy 1, policy_version 1810460 (0.0008) [2023-12-27 18:11:59,362][20381] Updated weights for policy 1, policy_version 1810470 (0.0007) [2023-12-27 18:11:59,422][20381] Updated weights for policy 1, policy_version 1810480 (0.0006) [2023-12-27 18:11:59,789][20347] Updated weights for policy 0, policy_version 1812153 (0.0007) [2023-12-27 18:11:59,850][20347] Updated weights for policy 0, policy_version 1812163 (0.0007) [2023-12-27 18:11:59,909][20347] Updated weights for policy 0, policy_version 1812173 (0.0008) [2023-12-27 18:11:59,971][20347] Updated weights for policy 0, policy_version 1812183 (0.0008) [2023-12-27 18:12:00,029][20381] Updated weights for policy 1, policy_version 1810490 (0.0006) [2023-12-27 18:12:00,080][20381] Updated weights for policy 1, policy_version 1810500 (0.0008) [2023-12-27 18:12:00,134][20381] Updated weights for policy 1, policy_version 1810510 (0.0007) [2023-12-27 18:12:00,190][20381] Updated weights for policy 1, policy_version 1810520 (0.0008) [2023-12-27 18:12:00,594][20347] Updated weights for policy 0, policy_version 1812193 (0.0008) [2023-12-27 18:12:00,640][20347] Updated weights for policy 0, policy_version 1812203 (0.0008) [2023-12-27 18:12:00,691][20347] Updated weights for policy 0, policy_version 1812213 (0.0007) [2023-12-27 18:12:00,870][20381] Updated weights for policy 1, policy_version 1810530 (0.0008) [2023-12-27 18:12:00,914][20381] Updated weights for policy 1, policy_version 1810540 (0.0009) [2023-12-27 18:12:00,959][20381] Updated weights for policy 1, policy_version 1810550 (0.0009) [2023-12-27 18:12:01,429][20347] Updated weights for policy 0, policy_version 1812223 (0.0008) [2023-12-27 18:12:01,482][20347] Updated weights for policy 0, policy_version 1812233 (0.0005) [2023-12-27 18:12:01,541][20347] Updated weights for policy 0, policy_version 1812243 (0.0011) [2023-12-27 18:12:01,745][20381] Updated weights for policy 1, policy_version 1810560 (0.0008) [2023-12-27 18:12:01,793][20381] Updated weights for policy 1, policy_version 1810570 (0.0007) [2023-12-27 18:12:01,840][20381] Updated weights for policy 1, policy_version 1810580 (0.0007) [2023-12-27 18:12:02,207][20347] Updated weights for policy 0, policy_version 1812253 (0.0009) [2023-12-27 18:12:02,269][20347] Updated weights for policy 0, policy_version 1812263 (0.0007) [2023-12-27 18:12:02,324][20347] Updated weights for policy 0, policy_version 1812273 (0.0010) [2023-12-27 18:12:02,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 927580160. Throughput: 0: 10323.6, 1: 10293.2. Samples: 927595404. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:02,363][19170] Avg episode reward: [(0, '960.890'), (1, '990.250')] [2023-12-27 18:12:02,548][20381] Updated weights for policy 1, policy_version 1810590 (0.0008) [2023-12-27 18:12:02,606][20381] Updated weights for policy 1, policy_version 1810600 (0.0007) [2023-12-27 18:12:02,660][20381] Updated weights for policy 1, policy_version 1810610 (0.0007) [2023-12-27 18:12:03,009][20347] Updated weights for policy 0, policy_version 1812283 (0.0010) [2023-12-27 18:12:03,068][20347] Updated weights for policy 0, policy_version 1812293 (0.0008) [2023-12-27 18:12:03,123][20347] Updated weights for policy 0, policy_version 1812303 (0.0010) [2023-12-27 18:12:03,373][20381] Updated weights for policy 1, policy_version 1810620 (0.0007) [2023-12-27 18:12:03,423][20381] Updated weights for policy 1, policy_version 1810630 (0.0008) [2023-12-27 18:12:03,474][20381] Updated weights for policy 1, policy_version 1810640 (0.0007) [2023-12-27 18:12:03,778][20347] Updated weights for policy 0, policy_version 1812313 (0.0010) [2023-12-27 18:12:03,838][20347] Updated weights for policy 0, policy_version 1812323 (0.0007) [2023-12-27 18:12:03,899][20347] Updated weights for policy 0, policy_version 1812333 (0.0007) [2023-12-27 18:12:03,957][20347] Updated weights for policy 0, policy_version 1812343 (0.0009) [2023-12-27 18:12:04,215][20381] Updated weights for policy 1, policy_version 1810650 (0.0007) [2023-12-27 18:12:04,281][20381] Updated weights for policy 1, policy_version 1810660 (0.0005) [2023-12-27 18:12:04,329][20381] Updated weights for policy 1, policy_version 1810670 (0.0006) [2023-12-27 18:12:04,380][20381] Updated weights for policy 1, policy_version 1810680 (0.0006) [2023-12-27 18:12:04,606][20347] Updated weights for policy 0, policy_version 1812353 (0.0010) [2023-12-27 18:12:04,662][20347] Updated weights for policy 0, policy_version 1812363 (0.0010) [2023-12-27 18:12:04,726][20347] Updated weights for policy 0, policy_version 1812373 (0.0010) [2023-12-27 18:12:04,963][20381] Updated weights for policy 1, policy_version 1810690 (0.0009) [2023-12-27 18:12:05,019][20381] Updated weights for policy 1, policy_version 1810700 (0.0011) [2023-12-27 18:12:05,074][20381] Updated weights for policy 1, policy_version 1810710 (0.0011) [2023-12-27 18:12:05,371][20347] Updated weights for policy 0, policy_version 1812383 (0.0008) [2023-12-27 18:12:05,421][20347] Updated weights for policy 0, policy_version 1812393 (0.0006) [2023-12-27 18:12:05,479][20347] Updated weights for policy 0, policy_version 1812403 (0.0009) [2023-12-27 18:12:05,667][20381] Updated weights for policy 1, policy_version 1810720 (0.0007) [2023-12-27 18:12:05,724][20381] Updated weights for policy 1, policy_version 1810730 (0.0006) [2023-12-27 18:12:05,790][20381] Updated weights for policy 1, policy_version 1810740 (0.0007) [2023-12-27 18:12:06,136][20347] Updated weights for policy 0, policy_version 1812413 (0.0010) [2023-12-27 18:12:06,201][20347] Updated weights for policy 0, policy_version 1812423 (0.0007) [2023-12-27 18:12:06,268][20347] Updated weights for policy 0, policy_version 1812433 (0.0007) [2023-12-27 18:12:06,410][20381] Updated weights for policy 1, policy_version 1810750 (0.0005) [2023-12-27 18:12:06,468][20381] Updated weights for policy 1, policy_version 1810760 (0.0007) [2023-12-27 18:12:06,528][20381] Updated weights for policy 1, policy_version 1810770 (0.0006) [2023-12-27 18:12:06,947][20347] Updated weights for policy 0, policy_version 1812443 (0.0009) [2023-12-27 18:12:07,009][20347] Updated weights for policy 0, policy_version 1812453 (0.0010) [2023-12-27 18:12:07,071][20347] Updated weights for policy 0, policy_version 1812463 (0.0010) [2023-12-27 18:12:07,194][20381] Updated weights for policy 1, policy_version 1810780 (0.0010) [2023-12-27 18:12:07,246][20381] Updated weights for policy 1, policy_version 1810790 (0.0010) [2023-12-27 18:12:07,295][20381] Updated weights for policy 1, policy_version 1810800 (0.0010) [2023-12-27 18:12:07,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 927694848. Throughput: 0: 10333.0, 1: 10317.1. Samples: 927656824. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:07,363][19170] Avg episode reward: [(0, '941.440'), (1, '961.170')] [2023-12-27 18:12:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001812472_464076800.pth... [2023-12-27 18:12:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001810808_463618048.pth... [2023-12-27 18:12:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001811256_463765504.pth [2023-12-27 18:12:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001809592_463306752.pth [2023-12-27 18:12:07,673][20347] Updated weights for policy 0, policy_version 1812473 (0.0010) [2023-12-27 18:12:07,745][20347] Updated weights for policy 0, policy_version 1812483 (0.0010) [2023-12-27 18:12:07,803][20347] Updated weights for policy 0, policy_version 1812493 (0.0010) [2023-12-27 18:12:07,866][20347] Updated weights for policy 0, policy_version 1812503 (0.0010) [2023-12-27 18:12:07,934][20381] Updated weights for policy 1, policy_version 1810810 (0.0010) [2023-12-27 18:12:07,990][20381] Updated weights for policy 1, policy_version 1810820 (0.0010) [2023-12-27 18:12:08,044][20381] Updated weights for policy 1, policy_version 1810830 (0.0011) [2023-12-27 18:12:08,095][20381] Updated weights for policy 1, policy_version 1810840 (0.0010) [2023-12-27 18:12:08,526][20347] Updated weights for policy 0, policy_version 1812513 (0.0008) [2023-12-27 18:12:08,586][20347] Updated weights for policy 0, policy_version 1812523 (0.0007) [2023-12-27 18:12:08,643][20347] Updated weights for policy 0, policy_version 1812533 (0.0008) [2023-12-27 18:12:08,777][20381] Updated weights for policy 1, policy_version 1810850 (0.0008) [2023-12-27 18:12:08,841][20381] Updated weights for policy 1, policy_version 1810860 (0.0005) [2023-12-27 18:12:08,897][20381] Updated weights for policy 1, policy_version 1810870 (0.0006) [2023-12-27 18:12:09,298][20347] Updated weights for policy 0, policy_version 1812543 (0.0008) [2023-12-27 18:12:09,364][20347] Updated weights for policy 0, policy_version 1812553 (0.0009) [2023-12-27 18:12:09,431][20347] Updated weights for policy 0, policy_version 1812563 (0.0008) [2023-12-27 18:12:09,535][20381] Updated weights for policy 1, policy_version 1810880 (0.0010) [2023-12-27 18:12:09,594][20381] Updated weights for policy 1, policy_version 1810890 (0.0010) [2023-12-27 18:12:09,653][20381] Updated weights for policy 1, policy_version 1810900 (0.0010) [2023-12-27 18:12:10,150][20347] Updated weights for policy 0, policy_version 1812573 (0.0008) [2023-12-27 18:12:10,213][20347] Updated weights for policy 0, policy_version 1812583 (0.0008) [2023-12-27 18:12:10,271][20347] Updated weights for policy 0, policy_version 1812593 (0.0008) [2023-12-27 18:12:10,326][20381] Updated weights for policy 1, policy_version 1810910 (0.0011) [2023-12-27 18:12:10,380][20381] Updated weights for policy 1, policy_version 1810920 (0.0010) [2023-12-27 18:12:10,401][20292] KL-divergence is very high: 130.6551 [2023-12-27 18:12:10,431][20381] Updated weights for policy 1, policy_version 1810930 (0.0006) [2023-12-27 18:12:10,443][20292] KL-divergence is very high: 116.3160 [2023-12-27 18:12:11,004][20347] Updated weights for policy 0, policy_version 1812603 (0.0008) [2023-12-27 18:12:11,070][20347] Updated weights for policy 0, policy_version 1812613 (0.0008) [2023-12-27 18:12:11,123][20347] Updated weights for policy 0, policy_version 1812623 (0.0008) [2023-12-27 18:12:11,165][20381] Updated weights for policy 1, policy_version 1810940 (0.0006) [2023-12-27 18:12:11,223][20381] Updated weights for policy 1, policy_version 1810950 (0.0007) [2023-12-27 18:12:11,285][20381] Updated weights for policy 1, policy_version 1810960 (0.0009) [2023-12-27 18:12:11,882][20347] Updated weights for policy 0, policy_version 1812633 (0.0008) [2023-12-27 18:12:11,947][20347] Updated weights for policy 0, policy_version 1812643 (0.0007) [2023-12-27 18:12:12,002][20347] Updated weights for policy 0, policy_version 1812653 (0.0007) [2023-12-27 18:12:12,030][20381] Updated weights for policy 1, policy_version 1810970 (0.0010) [2023-12-27 18:12:12,061][20347] Updated weights for policy 0, policy_version 1812663 (0.0007) [2023-12-27 18:12:12,083][20381] Updated weights for policy 1, policy_version 1810980 (0.0009) [2023-12-27 18:12:12,140][20381] Updated weights for policy 1, policy_version 1810990 (0.0007) [2023-12-27 18:12:12,198][20381] Updated weights for policy 1, policy_version 1811000 (0.0011) [2023-12-27 18:12:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 927793152. Throughput: 0: 10322.3, 1: 10346.6. Samples: 927780756. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:12,363][19170] Avg episode reward: [(0, '941.360'), (1, '961.170')] [2023-12-27 18:12:12,710][20347] Updated weights for policy 0, policy_version 1812673 (0.0008) [2023-12-27 18:12:12,758][20347] Updated weights for policy 0, policy_version 1812683 (0.0007) [2023-12-27 18:12:12,805][20347] Updated weights for policy 0, policy_version 1812693 (0.0006) [2023-12-27 18:12:12,944][20381] Updated weights for policy 1, policy_version 1811010 (0.0011) [2023-12-27 18:12:13,005][20381] Updated weights for policy 1, policy_version 1811020 (0.0011) [2023-12-27 18:12:13,066][20381] Updated weights for policy 1, policy_version 1811030 (0.0011) [2023-12-27 18:12:13,499][20347] Updated weights for policy 0, policy_version 1812703 (0.0009) [2023-12-27 18:12:13,557][20347] Updated weights for policy 0, policy_version 1812713 (0.0007) [2023-12-27 18:12:13,617][20347] Updated weights for policy 0, policy_version 1812723 (0.0009) [2023-12-27 18:12:13,700][20381] Updated weights for policy 1, policy_version 1811040 (0.0010) [2023-12-27 18:12:13,746][20381] Updated weights for policy 1, policy_version 1811050 (0.0009) [2023-12-27 18:12:13,793][20381] Updated weights for policy 1, policy_version 1811060 (0.0008) [2023-12-27 18:12:14,331][20347] Updated weights for policy 0, policy_version 1812733 (0.0008) [2023-12-27 18:12:14,388][20347] Updated weights for policy 0, policy_version 1812743 (0.0008) [2023-12-27 18:12:14,445][20381] Updated weights for policy 1, policy_version 1811070 (0.0009) [2023-12-27 18:12:14,446][20347] Updated weights for policy 0, policy_version 1812753 (0.0005) [2023-12-27 18:12:14,499][20381] Updated weights for policy 1, policy_version 1811080 (0.0007) [2023-12-27 18:12:14,560][20381] Updated weights for policy 1, policy_version 1811090 (0.0007) [2023-12-27 18:12:15,156][20381] Updated weights for policy 1, policy_version 1811100 (0.0007) [2023-12-27 18:12:15,161][20347] Updated weights for policy 0, policy_version 1812763 (0.0006) [2023-12-27 18:12:15,214][20381] Updated weights for policy 1, policy_version 1811110 (0.0006) [2023-12-27 18:12:15,216][20347] Updated weights for policy 0, policy_version 1812773 (0.0007) [2023-12-27 18:12:15,269][20347] Updated weights for policy 0, policy_version 1812783 (0.0006) [2023-12-27 18:12:15,271][20381] Updated weights for policy 1, policy_version 1811120 (0.0006) [2023-12-27 18:12:15,941][20381] Updated weights for policy 1, policy_version 1811130 (0.0006) [2023-12-27 18:12:15,972][20347] Updated weights for policy 0, policy_version 1812793 (0.0006) [2023-12-27 18:12:15,998][20381] Updated weights for policy 1, policy_version 1811140 (0.0008) [2023-12-27 18:12:16,032][20347] Updated weights for policy 0, policy_version 1812803 (0.0006) [2023-12-27 18:12:16,051][20381] Updated weights for policy 1, policy_version 1811150 (0.0010) [2023-12-27 18:12:16,083][20347] Updated weights for policy 0, policy_version 1812813 (0.0005) [2023-12-27 18:12:16,111][20381] Updated weights for policy 1, policy_version 1811160 (0.0009) [2023-12-27 18:12:16,136][20347] Updated weights for policy 0, policy_version 1812823 (0.0007) [2023-12-27 18:12:16,715][20381] Updated weights for policy 1, policy_version 1811170 (0.0006) [2023-12-27 18:12:16,760][20347] Updated weights for policy 0, policy_version 1812833 (0.0007) [2023-12-27 18:12:16,764][20381] Updated weights for policy 1, policy_version 1811180 (0.0006) [2023-12-27 18:12:16,809][20381] Updated weights for policy 1, policy_version 1811190 (0.0005) [2023-12-27 18:12:16,822][20347] Updated weights for policy 0, policy_version 1812843 (0.0007) [2023-12-27 18:12:16,881][20347] Updated weights for policy 0, policy_version 1812853 (0.0007) [2023-12-27 18:12:17,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 927891456. Throughput: 0: 10269.7, 1: 10328.6. Samples: 927903700. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:17,364][19170] Avg episode reward: [(0, '921.800'), (1, '970.740')] [2023-12-27 18:12:17,434][20381] Updated weights for policy 1, policy_version 1811200 (0.0009) [2023-12-27 18:12:17,491][20381] Updated weights for policy 1, policy_version 1811210 (0.0010) [2023-12-27 18:12:17,548][20381] Updated weights for policy 1, policy_version 1811220 (0.0006) [2023-12-27 18:12:17,556][20347] Updated weights for policy 0, policy_version 1812863 (0.0008) [2023-12-27 18:12:17,620][20347] Updated weights for policy 0, policy_version 1812873 (0.0005) [2023-12-27 18:12:17,669][20347] Updated weights for policy 0, policy_version 1812883 (0.0005) [2023-12-27 18:12:18,126][20381] Updated weights for policy 1, policy_version 1811230 (0.0009) [2023-12-27 18:12:18,181][20381] Updated weights for policy 1, policy_version 1811240 (0.0008) [2023-12-27 18:12:18,235][20381] Updated weights for policy 1, policy_version 1811250 (0.0008) [2023-12-27 18:12:18,249][20347] Updated weights for policy 0, policy_version 1812893 (0.0006) [2023-12-27 18:12:18,307][20347] Updated weights for policy 0, policy_version 1812903 (0.0010) [2023-12-27 18:12:18,375][20347] Updated weights for policy 0, policy_version 1812913 (0.0010) [2023-12-27 18:12:19,007][20381] Updated weights for policy 1, policy_version 1811260 (0.0007) [2023-12-27 18:12:19,063][20381] Updated weights for policy 1, policy_version 1811270 (0.0007) [2023-12-27 18:12:19,082][20347] Updated weights for policy 0, policy_version 1812923 (0.0011) [2023-12-27 18:12:19,117][20381] Updated weights for policy 1, policy_version 1811280 (0.0007) [2023-12-27 18:12:19,137][20347] Updated weights for policy 0, policy_version 1812933 (0.0010) [2023-12-27 18:12:19,196][20347] Updated weights for policy 0, policy_version 1812943 (0.0011) [2023-12-27 18:12:19,828][20381] Updated weights for policy 1, policy_version 1811290 (0.0006) [2023-12-27 18:12:19,891][20381] Updated weights for policy 1, policy_version 1811300 (0.0009) [2023-12-27 18:12:19,920][20347] Updated weights for policy 0, policy_version 1812953 (0.0011) [2023-12-27 18:12:19,957][20381] Updated weights for policy 1, policy_version 1811310 (0.0008) [2023-12-27 18:12:19,975][20347] Updated weights for policy 0, policy_version 1812963 (0.0010) [2023-12-27 18:12:20,015][20381] Updated weights for policy 1, policy_version 1811320 (0.0006) [2023-12-27 18:12:20,033][20347] Updated weights for policy 0, policy_version 1812973 (0.0010) [2023-12-27 18:12:20,092][20347] Updated weights for policy 0, policy_version 1812983 (0.0011) [2023-12-27 18:12:20,667][20381] Updated weights for policy 1, policy_version 1811330 (0.0007) [2023-12-27 18:12:20,725][20381] Updated weights for policy 1, policy_version 1811340 (0.0007) [2023-12-27 18:12:20,740][20347] Updated weights for policy 0, policy_version 1812993 (0.0008) [2023-12-27 18:12:20,778][20381] Updated weights for policy 1, policy_version 1811350 (0.0007) [2023-12-27 18:12:20,796][20347] Updated weights for policy 0, policy_version 1813003 (0.0008) [2023-12-27 18:12:20,852][20347] Updated weights for policy 0, policy_version 1813013 (0.0008) [2023-12-27 18:12:21,465][20381] Updated weights for policy 1, policy_version 1811360 (0.0007) [2023-12-27 18:12:21,517][20381] Updated weights for policy 1, policy_version 1811370 (0.0007) [2023-12-27 18:12:21,576][20381] Updated weights for policy 1, policy_version 1811380 (0.0006) [2023-12-27 18:12:21,663][20347] Updated weights for policy 0, policy_version 1813023 (0.0008) [2023-12-27 18:12:21,732][20347] Updated weights for policy 0, policy_version 1813033 (0.0009) [2023-12-27 18:12:21,795][20347] Updated weights for policy 0, policy_version 1813043 (0.0007) [2023-12-27 18:12:22,255][20381] Updated weights for policy 1, policy_version 1811390 (0.0007) [2023-12-27 18:12:22,316][20381] Updated weights for policy 1, policy_version 1811400 (0.0007) [2023-12-27 18:12:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 927989760. Throughput: 0: 10286.0, 1: 10319.4. Samples: 927964620. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:22,364][19170] Avg episode reward: [(0, '882.710'), (1, '951.220')] [2023-12-27 18:12:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001813048_464224256.pth... [2023-12-27 18:12:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001811864_463921152.pth [2023-12-27 18:12:22,380][20381] Updated weights for policy 1, policy_version 1811410 (0.0007) [2023-12-27 18:12:22,414][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001811416_463773696.pth... [2023-12-27 18:12:22,418][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001810200_463462400.pth [2023-12-27 18:12:22,514][20347] Updated weights for policy 0, policy_version 1813053 (0.0009) [2023-12-27 18:12:22,574][20347] Updated weights for policy 0, policy_version 1813063 (0.0010) [2023-12-27 18:12:22,638][20347] Updated weights for policy 0, policy_version 1813073 (0.0011) [2023-12-27 18:12:23,069][20381] Updated weights for policy 1, policy_version 1811420 (0.0008) [2023-12-27 18:12:23,116][20381] Updated weights for policy 1, policy_version 1811430 (0.0009) [2023-12-27 18:12:23,168][20381] Updated weights for policy 1, policy_version 1811440 (0.0008) [2023-12-27 18:12:23,224][20347] Updated weights for policy 0, policy_version 1813083 (0.0010) [2023-12-27 18:12:23,275][20347] Updated weights for policy 0, policy_version 1813093 (0.0010) [2023-12-27 18:12:23,323][20347] Updated weights for policy 0, policy_version 1813103 (0.0010) [2023-12-27 18:12:23,856][20381] Updated weights for policy 1, policy_version 1811450 (0.0007) [2023-12-27 18:12:23,910][20381] Updated weights for policy 1, policy_version 1811460 (0.0007) [2023-12-27 18:12:23,966][20381] Updated weights for policy 1, policy_version 1811470 (0.0007) [2023-12-27 18:12:24,015][20347] Updated weights for policy 0, policy_version 1813113 (0.0010) [2023-12-27 18:12:24,023][20381] Updated weights for policy 1, policy_version 1811480 (0.0007) [2023-12-27 18:12:24,066][20347] Updated weights for policy 0, policy_version 1813123 (0.0010) [2023-12-27 18:12:24,121][20347] Updated weights for policy 0, policy_version 1813133 (0.0010) [2023-12-27 18:12:24,175][20347] Updated weights for policy 0, policy_version 1813143 (0.0010) [2023-12-27 18:12:24,679][20381] Updated weights for policy 1, policy_version 1811490 (0.0008) [2023-12-27 18:12:24,736][20381] Updated weights for policy 1, policy_version 1811500 (0.0007) [2023-12-27 18:12:24,797][20381] Updated weights for policy 1, policy_version 1811510 (0.0007) [2023-12-27 18:12:24,862][20347] Updated weights for policy 0, policy_version 1813153 (0.0010) [2023-12-27 18:12:24,920][20347] Updated weights for policy 0, policy_version 1813163 (0.0010) [2023-12-27 18:12:24,972][20347] Updated weights for policy 0, policy_version 1813173 (0.0010) [2023-12-27 18:12:25,398][20381] Updated weights for policy 1, policy_version 1811520 (0.0010) [2023-12-27 18:12:25,454][20381] Updated weights for policy 1, policy_version 1811530 (0.0011) [2023-12-27 18:12:25,511][20381] Updated weights for policy 1, policy_version 1811540 (0.0009) [2023-12-27 18:12:25,592][20347] Updated weights for policy 0, policy_version 1813183 (0.0007) [2023-12-27 18:12:25,652][20347] Updated weights for policy 0, policy_version 1813193 (0.0005) [2023-12-27 18:12:25,708][20347] Updated weights for policy 0, policy_version 1813203 (0.0005) [2023-12-27 18:12:26,134][20381] Updated weights for policy 1, policy_version 1811550 (0.0011) [2023-12-27 18:12:26,190][20381] Updated weights for policy 1, policy_version 1811560 (0.0011) [2023-12-27 18:12:26,251][20381] Updated weights for policy 1, policy_version 1811570 (0.0010) [2023-12-27 18:12:26,336][20347] Updated weights for policy 0, policy_version 1813213 (0.0008) [2023-12-27 18:12:26,390][20347] Updated weights for policy 0, policy_version 1813223 (0.0010) [2023-12-27 18:12:26,445][20347] Updated weights for policy 0, policy_version 1813233 (0.0010) [2023-12-27 18:12:26,880][20381] Updated weights for policy 1, policy_version 1811580 (0.0009) [2023-12-27 18:12:26,928][20381] Updated weights for policy 1, policy_version 1811590 (0.0010) [2023-12-27 18:12:26,973][20381] Updated weights for policy 1, policy_version 1811600 (0.0010) [2023-12-27 18:12:27,129][20347] Updated weights for policy 0, policy_version 1813243 (0.0010) [2023-12-27 18:12:27,185][20347] Updated weights for policy 0, policy_version 1813253 (0.0010) [2023-12-27 18:12:27,241][20347] Updated weights for policy 0, policy_version 1813263 (0.0010) [2023-12-27 18:12:27,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 928104448. Throughput: 0: 10252.9, 1: 10305.1. Samples: 928088740. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:27,363][19170] Avg episode reward: [(0, '960.940'), (1, '960.980')] [2023-12-27 18:12:27,556][20381] Updated weights for policy 1, policy_version 1811610 (0.0009) [2023-12-27 18:12:27,606][20381] Updated weights for policy 1, policy_version 1811620 (0.0005) [2023-12-27 18:12:27,658][20381] Updated weights for policy 1, policy_version 1811630 (0.0005) [2023-12-27 18:12:27,713][20381] Updated weights for policy 1, policy_version 1811640 (0.0005) [2023-12-27 18:12:27,894][20347] Updated weights for policy 0, policy_version 1813273 (0.0010) [2023-12-27 18:12:27,943][20347] Updated weights for policy 0, policy_version 1813283 (0.0009) [2023-12-27 18:12:27,994][20347] Updated weights for policy 0, policy_version 1813293 (0.0010) [2023-12-27 18:12:28,048][20347] Updated weights for policy 0, policy_version 1813303 (0.0010) [2023-12-27 18:12:28,316][20381] Updated weights for policy 1, policy_version 1811650 (0.0008) [2023-12-27 18:12:28,375][20381] Updated weights for policy 1, policy_version 1811660 (0.0009) [2023-12-27 18:12:28,427][20381] Updated weights for policy 1, policy_version 1811670 (0.0011) [2023-12-27 18:12:28,771][20347] Updated weights for policy 0, policy_version 1813313 (0.0010) [2023-12-27 18:12:28,821][20347] Updated weights for policy 0, policy_version 1813323 (0.0010) [2023-12-27 18:12:28,872][20347] Updated weights for policy 0, policy_version 1813333 (0.0010) [2023-12-27 18:12:29,137][20381] Updated weights for policy 1, policy_version 1811680 (0.0010) [2023-12-27 18:12:29,197][20381] Updated weights for policy 1, policy_version 1811690 (0.0010) [2023-12-27 18:12:29,262][20381] Updated weights for policy 1, policy_version 1811700 (0.0010) [2023-12-27 18:12:29,539][20347] Updated weights for policy 0, policy_version 1813343 (0.0010) [2023-12-27 18:12:29,596][20347] Updated weights for policy 0, policy_version 1813353 (0.0010) [2023-12-27 18:12:29,650][20347] Updated weights for policy 0, policy_version 1813363 (0.0010) [2023-12-27 18:12:29,976][20381] Updated weights for policy 1, policy_version 1811710 (0.0009) [2023-12-27 18:12:30,037][20381] Updated weights for policy 1, policy_version 1811720 (0.0010) [2023-12-27 18:12:30,097][20381] Updated weights for policy 1, policy_version 1811730 (0.0007) [2023-12-27 18:12:30,333][20347] Updated weights for policy 0, policy_version 1813373 (0.0009) [2023-12-27 18:12:30,394][20347] Updated weights for policy 0, policy_version 1813383 (0.0008) [2023-12-27 18:12:30,450][20347] Updated weights for policy 0, policy_version 1813393 (0.0007) [2023-12-27 18:12:30,737][20381] Updated weights for policy 1, policy_version 1811740 (0.0007) [2023-12-27 18:12:30,780][20381] Updated weights for policy 1, policy_version 1811750 (0.0007) [2023-12-27 18:12:30,840][20381] Updated weights for policy 1, policy_version 1811760 (0.0010) [2023-12-27 18:12:31,044][20347] Updated weights for policy 0, policy_version 1813403 (0.0006) [2023-12-27 18:12:31,099][20347] Updated weights for policy 0, policy_version 1813413 (0.0006) [2023-12-27 18:12:31,167][20347] Updated weights for policy 0, policy_version 1813423 (0.0007) [2023-12-27 18:12:31,513][20381] Updated weights for policy 1, policy_version 1811770 (0.0007) [2023-12-27 18:12:31,562][20381] Updated weights for policy 1, policy_version 1811780 (0.0007) [2023-12-27 18:12:31,623][20381] Updated weights for policy 1, policy_version 1811790 (0.0008) [2023-12-27 18:12:31,686][20381] Updated weights for policy 1, policy_version 1811800 (0.0008) [2023-12-27 18:12:31,868][20347] Updated weights for policy 0, policy_version 1813433 (0.0007) [2023-12-27 18:12:31,938][20347] Updated weights for policy 0, policy_version 1813443 (0.0008) [2023-12-27 18:12:31,994][20347] Updated weights for policy 0, policy_version 1813453 (0.0008) [2023-12-27 18:12:32,050][20347] Updated weights for policy 0, policy_version 1813463 (0.0007) [2023-12-27 18:12:32,350][20381] Updated weights for policy 1, policy_version 1811810 (0.0008) [2023-12-27 18:12:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 928202752. Throughput: 0: 10265.2, 1: 10345.2. Samples: 928214256. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:32,363][19170] Avg episode reward: [(0, '961.020'), (1, '980.550')] [2023-12-27 18:12:32,416][20381] Updated weights for policy 1, policy_version 1811820 (0.0007) [2023-12-27 18:12:32,475][20381] Updated weights for policy 1, policy_version 1811830 (0.0006) [2023-12-27 18:12:32,768][20347] Updated weights for policy 0, policy_version 1813473 (0.0008) [2023-12-27 18:12:32,815][20347] Updated weights for policy 0, policy_version 1813483 (0.0007) [2023-12-27 18:12:32,864][20347] Updated weights for policy 0, policy_version 1813493 (0.0008) [2023-12-27 18:12:33,085][20381] Updated weights for policy 1, policy_version 1811840 (0.0008) [2023-12-27 18:12:33,135][20381] Updated weights for policy 1, policy_version 1811850 (0.0009) [2023-12-27 18:12:33,186][20381] Updated weights for policy 1, policy_version 1811860 (0.0007) [2023-12-27 18:12:33,515][20347] Updated weights for policy 0, policy_version 1813503 (0.0007) [2023-12-27 18:12:33,568][20347] Updated weights for policy 0, policy_version 1813513 (0.0007) [2023-12-27 18:12:33,621][20347] Updated weights for policy 0, policy_version 1813523 (0.0005) [2023-12-27 18:12:33,832][20381] Updated weights for policy 1, policy_version 1811870 (0.0008) [2023-12-27 18:12:33,889][20381] Updated weights for policy 1, policy_version 1811880 (0.0008) [2023-12-27 18:12:33,953][20381] Updated weights for policy 1, policy_version 1811890 (0.0009) [2023-12-27 18:12:34,267][20347] Updated weights for policy 0, policy_version 1813533 (0.0007) [2023-12-27 18:12:34,322][20347] Updated weights for policy 0, policy_version 1813543 (0.0007) [2023-12-27 18:12:34,385][20347] Updated weights for policy 0, policy_version 1813553 (0.0007) [2023-12-27 18:12:34,661][20381] Updated weights for policy 1, policy_version 1811900 (0.0008) [2023-12-27 18:12:34,717][20381] Updated weights for policy 1, policy_version 1811910 (0.0008) [2023-12-27 18:12:34,774][20381] Updated weights for policy 1, policy_version 1811920 (0.0008) [2023-12-27 18:12:35,082][20347] Updated weights for policy 0, policy_version 1813563 (0.0007) [2023-12-27 18:12:35,129][20347] Updated weights for policy 0, policy_version 1813573 (0.0005) [2023-12-27 18:12:35,185][20347] Updated weights for policy 0, policy_version 1813583 (0.0008) [2023-12-27 18:12:35,469][20381] Updated weights for policy 1, policy_version 1811930 (0.0008) [2023-12-27 18:12:35,528][20381] Updated weights for policy 1, policy_version 1811940 (0.0007) [2023-12-27 18:12:35,576][20381] Updated weights for policy 1, policy_version 1811950 (0.0008) [2023-12-27 18:12:35,626][20381] Updated weights for policy 1, policy_version 1811960 (0.0008) [2023-12-27 18:12:35,901][20347] Updated weights for policy 0, policy_version 1813593 (0.0008) [2023-12-27 18:12:35,946][20347] Updated weights for policy 0, policy_version 1813603 (0.0008) [2023-12-27 18:12:35,998][20347] Updated weights for policy 0, policy_version 1813613 (0.0007) [2023-12-27 18:12:36,048][20347] Updated weights for policy 0, policy_version 1813623 (0.0010) [2023-12-27 18:12:36,198][20381] Updated weights for policy 1, policy_version 1811970 (0.0008) [2023-12-27 18:12:36,265][20381] Updated weights for policy 1, policy_version 1811980 (0.0008) [2023-12-27 18:12:36,330][20381] Updated weights for policy 1, policy_version 1811990 (0.0008) [2023-12-27 18:12:36,815][20347] Updated weights for policy 0, policy_version 1813633 (0.0010) [2023-12-27 18:12:36,869][20347] Updated weights for policy 0, policy_version 1813643 (0.0010) [2023-12-27 18:12:36,924][20347] Updated weights for policy 0, policy_version 1813653 (0.0010) [2023-12-27 18:12:37,021][20381] Updated weights for policy 1, policy_version 1812000 (0.0008) [2023-12-27 18:12:37,079][20381] Updated weights for policy 1, policy_version 1812010 (0.0007) [2023-12-27 18:12:37,137][20381] Updated weights for policy 1, policy_version 1812020 (0.0007) [2023-12-27 18:12:37,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 928309248. Throughput: 0: 10224.6, 1: 10358.0. Samples: 928276324. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:37,363][19170] Avg episode reward: [(0, '960.950'), (1, '970.880')] [2023-12-27 18:12:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001812024_463929344.pth... [2023-12-27 18:12:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001813656_464379904.pth... [2023-12-27 18:12:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001810808_463618048.pth [2023-12-27 18:12:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001812472_464076800.pth [2023-12-27 18:12:37,599][20347] Updated weights for policy 0, policy_version 1813663 (0.0008) [2023-12-27 18:12:37,653][20347] Updated weights for policy 0, policy_version 1813673 (0.0007) [2023-12-27 18:12:37,715][20347] Updated weights for policy 0, policy_version 1813683 (0.0006) [2023-12-27 18:12:37,789][20381] Updated weights for policy 1, policy_version 1812030 (0.0008) [2023-12-27 18:12:37,843][20381] Updated weights for policy 1, policy_version 1812040 (0.0007) [2023-12-27 18:12:37,900][20381] Updated weights for policy 1, policy_version 1812050 (0.0008) [2023-12-27 18:12:38,393][20347] Updated weights for policy 0, policy_version 1813693 (0.0007) [2023-12-27 18:12:38,450][20347] Updated weights for policy 0, policy_version 1813703 (0.0007) [2023-12-27 18:12:38,515][20347] Updated weights for policy 0, policy_version 1813713 (0.0008) [2023-12-27 18:12:38,561][20381] Updated weights for policy 1, policy_version 1812060 (0.0009) [2023-12-27 18:12:38,617][20381] Updated weights for policy 1, policy_version 1812070 (0.0010) [2023-12-27 18:12:38,676][20381] Updated weights for policy 1, policy_version 1812080 (0.0011) [2023-12-27 18:12:39,226][20347] Updated weights for policy 0, policy_version 1813723 (0.0008) [2023-12-27 18:12:39,283][20347] Updated weights for policy 0, policy_version 1813733 (0.0007) [2023-12-27 18:12:39,346][20347] Updated weights for policy 0, policy_version 1813743 (0.0008) [2023-12-27 18:12:39,390][20381] Updated weights for policy 1, policy_version 1812090 (0.0010) [2023-12-27 18:12:39,457][20381] Updated weights for policy 1, policy_version 1812100 (0.0008) [2023-12-27 18:12:39,516][20381] Updated weights for policy 1, policy_version 1812110 (0.0007) [2023-12-27 18:12:39,575][20381] Updated weights for policy 1, policy_version 1812120 (0.0008) [2023-12-27 18:12:40,050][20347] Updated weights for policy 0, policy_version 1813753 (0.0007) [2023-12-27 18:12:40,123][20347] Updated weights for policy 0, policy_version 1813763 (0.0006) [2023-12-27 18:12:40,190][20347] Updated weights for policy 0, policy_version 1813773 (0.0008) [2023-12-27 18:12:40,252][20381] Updated weights for policy 1, policy_version 1812130 (0.0008) [2023-12-27 18:12:40,259][20347] Updated weights for policy 0, policy_version 1813783 (0.0007) [2023-12-27 18:12:40,312][20381] Updated weights for policy 1, policy_version 1812140 (0.0008) [2023-12-27 18:12:40,373][20381] Updated weights for policy 1, policy_version 1812150 (0.0007) [2023-12-27 18:12:40,919][20347] Updated weights for policy 0, policy_version 1813793 (0.0007) [2023-12-27 18:12:40,979][20347] Updated weights for policy 0, policy_version 1813803 (0.0008) [2023-12-27 18:12:41,040][20347] Updated weights for policy 0, policy_version 1813813 (0.0007) [2023-12-27 18:12:41,052][20381] Updated weights for policy 1, policy_version 1812160 (0.0006) [2023-12-27 18:12:41,111][20381] Updated weights for policy 1, policy_version 1812170 (0.0007) [2023-12-27 18:12:41,172][20381] Updated weights for policy 1, policy_version 1812180 (0.0009) [2023-12-27 18:12:41,793][20347] Updated weights for policy 0, policy_version 1813823 (0.0009) [2023-12-27 18:12:41,849][20347] Updated weights for policy 0, policy_version 1813833 (0.0008) [2023-12-27 18:12:41,856][20381] Updated weights for policy 1, policy_version 1812190 (0.0007) [2023-12-27 18:12:41,905][20347] Updated weights for policy 0, policy_version 1813843 (0.0007) [2023-12-27 18:12:41,918][20381] Updated weights for policy 1, policy_version 1812200 (0.0008) [2023-12-27 18:12:41,974][20381] Updated weights for policy 1, policy_version 1812210 (0.0007) [2023-12-27 18:12:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 928407552. Throughput: 0: 10155.7, 1: 10380.3. Samples: 928396920. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:42,363][19170] Avg episode reward: [(0, '941.270'), (1, '931.800')] [2023-12-27 18:12:42,607][20381] Updated weights for policy 1, policy_version 1812220 (0.0006) [2023-12-27 18:12:42,610][20347] Updated weights for policy 0, policy_version 1813853 (0.0007) [2023-12-27 18:12:42,656][20347] Updated weights for policy 0, policy_version 1813863 (0.0007) [2023-12-27 18:12:42,670][20381] Updated weights for policy 1, policy_version 1812230 (0.0008) [2023-12-27 18:12:42,711][20347] Updated weights for policy 0, policy_version 1813873 (0.0007) [2023-12-27 18:12:42,724][20381] Updated weights for policy 1, policy_version 1812240 (0.0009) [2023-12-27 18:12:43,349][20347] Updated weights for policy 0, policy_version 1813883 (0.0007) [2023-12-27 18:12:43,404][20347] Updated weights for policy 0, policy_version 1813893 (0.0008) [2023-12-27 18:12:43,437][20381] Updated weights for policy 1, policy_version 1812250 (0.0007) [2023-12-27 18:12:43,459][20347] Updated weights for policy 0, policy_version 1813903 (0.0007) [2023-12-27 18:12:43,487][20381] Updated weights for policy 1, policy_version 1812260 (0.0009) [2023-12-27 18:12:43,539][20381] Updated weights for policy 1, policy_version 1812270 (0.0006) [2023-12-27 18:12:43,602][20381] Updated weights for policy 1, policy_version 1812280 (0.0006) [2023-12-27 18:12:44,100][20347] Updated weights for policy 0, policy_version 1813913 (0.0006) [2023-12-27 18:12:44,153][20347] Updated weights for policy 0, policy_version 1813923 (0.0005) [2023-12-27 18:12:44,209][20347] Updated weights for policy 0, policy_version 1813933 (0.0007) [2023-12-27 18:12:44,238][20381] Updated weights for policy 1, policy_version 1812290 (0.0007) [2023-12-27 18:12:44,272][20347] Updated weights for policy 0, policy_version 1813943 (0.0007) [2023-12-27 18:12:44,287][20381] Updated weights for policy 1, policy_version 1812300 (0.0008) [2023-12-27 18:12:44,343][20381] Updated weights for policy 1, policy_version 1812310 (0.0008) [2023-12-27 18:12:44,897][20347] Updated weights for policy 0, policy_version 1813953 (0.0010) [2023-12-27 18:12:44,957][20347] Updated weights for policy 0, policy_version 1813963 (0.0011) [2023-12-27 18:12:45,014][20347] Updated weights for policy 0, policy_version 1813973 (0.0010) [2023-12-27 18:12:45,039][20381] Updated weights for policy 1, policy_version 1812320 (0.0007) [2023-12-27 18:12:45,088][20381] Updated weights for policy 1, policy_version 1812330 (0.0008) [2023-12-27 18:12:45,139][20381] Updated weights for policy 1, policy_version 1812340 (0.0007) [2023-12-27 18:12:45,719][20347] Updated weights for policy 0, policy_version 1813983 (0.0010) [2023-12-27 18:12:45,777][20347] Updated weights for policy 0, policy_version 1813993 (0.0010) [2023-12-27 18:12:45,835][20347] Updated weights for policy 0, policy_version 1814003 (0.0010) [2023-12-27 18:12:45,854][20381] Updated weights for policy 1, policy_version 1812350 (0.0009) [2023-12-27 18:12:45,901][20381] Updated weights for policy 1, policy_version 1812360 (0.0007) [2023-12-27 18:12:45,950][20381] Updated weights for policy 1, policy_version 1812370 (0.0007) [2023-12-27 18:12:46,518][20347] Updated weights for policy 0, policy_version 1814013 (0.0010) [2023-12-27 18:12:46,569][20347] Updated weights for policy 0, policy_version 1814023 (0.0010) [2023-12-27 18:12:46,624][20347] Updated weights for policy 0, policy_version 1814033 (0.0010) [2023-12-27 18:12:46,673][20381] Updated weights for policy 1, policy_version 1812380 (0.0008) [2023-12-27 18:12:46,735][20381] Updated weights for policy 1, policy_version 1812390 (0.0007) [2023-12-27 18:12:46,789][20381] Updated weights for policy 1, policy_version 1812400 (0.0007) [2023-12-27 18:12:47,276][20347] Updated weights for policy 0, policy_version 1814043 (0.0007) [2023-12-27 18:12:47,330][20347] Updated weights for policy 0, policy_version 1814053 (0.0010) [2023-12-27 18:12:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 928505856. Throughput: 0: 10160.4, 1: 10380.5. Samples: 928519748. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) [2023-12-27 18:12:47,363][19170] Avg episode reward: [(0, '931.640'), (1, '980.530')] [2023-12-27 18:12:47,387][20347] Updated weights for policy 0, policy_version 1814063 (0.0006) [2023-12-27 18:12:47,473][20381] Updated weights for policy 1, policy_version 1812410 (0.0007) [2023-12-27 18:12:47,532][20381] Updated weights for policy 1, policy_version 1812420 (0.0008) [2023-12-27 18:12:47,577][20381] Updated weights for policy 1, policy_version 1812430 (0.0007) [2023-12-27 18:12:47,630][20381] Updated weights for policy 1, policy_version 1812440 (0.0008) [2023-12-27 18:12:48,036][20347] Updated weights for policy 0, policy_version 1814073 (0.0008) [2023-12-27 18:12:48,084][20347] Updated weights for policy 0, policy_version 1814083 (0.0010) [2023-12-27 18:12:48,133][20347] Updated weights for policy 0, policy_version 1814093 (0.0010) [2023-12-27 18:12:48,180][20347] Updated weights for policy 0, policy_version 1814103 (0.0010) [2023-12-27 18:12:48,303][20381] Updated weights for policy 1, policy_version 1812450 (0.0007) [2023-12-27 18:12:48,367][20381] Updated weights for policy 1, policy_version 1812460 (0.0007) [2023-12-27 18:12:48,429][20381] Updated weights for policy 1, policy_version 1812470 (0.0007) [2023-12-27 18:12:48,903][20347] Updated weights for policy 0, policy_version 1814113 (0.0006) [2023-12-27 18:12:48,967][20347] Updated weights for policy 0, policy_version 1814123 (0.0006) [2023-12-27 18:12:49,035][20347] Updated weights for policy 0, policy_version 1814133 (0.0007) [2023-12-27 18:12:49,082][20381] Updated weights for policy 1, policy_version 1812480 (0.0009) [2023-12-27 18:12:49,134][20381] Updated weights for policy 1, policy_version 1812490 (0.0010) [2023-12-27 18:12:49,188][20381] Updated weights for policy 1, policy_version 1812500 (0.0010) [2023-12-27 18:12:49,680][20347] Updated weights for policy 0, policy_version 1814143 (0.0007) [2023-12-27 18:12:49,734][20347] Updated weights for policy 0, policy_version 1814153 (0.0007) [2023-12-27 18:12:49,793][20347] Updated weights for policy 0, policy_version 1814163 (0.0008) [2023-12-27 18:12:49,989][20381] Updated weights for policy 1, policy_version 1812510 (0.0009) [2023-12-27 18:12:50,047][20381] Updated weights for policy 1, policy_version 1812520 (0.0006) [2023-12-27 18:12:50,107][20381] Updated weights for policy 1, policy_version 1812530 (0.0005) [2023-12-27 18:12:50,445][20347] Updated weights for policy 0, policy_version 1814173 (0.0008) [2023-12-27 18:12:50,490][20347] Updated weights for policy 0, policy_version 1814183 (0.0008) [2023-12-27 18:12:50,537][20347] Updated weights for policy 0, policy_version 1814193 (0.0007) [2023-12-27 18:12:50,800][20381] Updated weights for policy 1, policy_version 1812540 (0.0008) [2023-12-27 18:12:50,849][20381] Updated weights for policy 1, policy_version 1812550 (0.0009) [2023-12-27 18:12:50,906][20381] Updated weights for policy 1, policy_version 1812560 (0.0008) [2023-12-27 18:12:51,203][20347] Updated weights for policy 0, policy_version 1814203 (0.0006) [2023-12-27 18:12:51,262][20347] Updated weights for policy 0, policy_version 1814213 (0.0008) [2023-12-27 18:12:51,314][20347] Updated weights for policy 0, policy_version 1814223 (0.0008) [2023-12-27 18:12:51,685][20381] Updated weights for policy 1, policy_version 1812570 (0.0008) [2023-12-27 18:12:51,750][20381] Updated weights for policy 1, policy_version 1812580 (0.0008) [2023-12-27 18:12:51,805][20381] Updated weights for policy 1, policy_version 1812590 (0.0007) [2023-12-27 18:12:51,867][20381] Updated weights for policy 1, policy_version 1812600 (0.0005) [2023-12-27 18:12:52,012][20347] Updated weights for policy 0, policy_version 1814233 (0.0008) [2023-12-27 18:12:52,080][20347] Updated weights for policy 0, policy_version 1814243 (0.0007) [2023-12-27 18:12:52,142][20347] Updated weights for policy 0, policy_version 1814253 (0.0007) [2023-12-27 18:12:52,207][20347] Updated weights for policy 0, policy_version 1814263 (0.0008) [2023-12-27 18:12:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 928612352. Throughput: 0: 10157.2, 1: 10358.6. Samples: 928580036. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:12:52,363][19170] Avg episode reward: [(0, '960.990'), (1, '990.230')] [2023-12-27 18:12:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001812600_464076800.pth... [2023-12-27 18:12:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001814264_464535552.pth... [2023-12-27 18:12:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001811416_463773696.pth [2023-12-27 18:12:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001813048_464224256.pth [2023-12-27 18:12:52,553][20381] Updated weights for policy 1, policy_version 1812610 (0.0008) [2023-12-27 18:12:52,614][20381] Updated weights for policy 1, policy_version 1812620 (0.0009) [2023-12-27 18:12:52,671][20381] Updated weights for policy 1, policy_version 1812630 (0.0008) [2023-12-27 18:12:52,847][20347] Updated weights for policy 0, policy_version 1814273 (0.0007) [2023-12-27 18:12:52,908][20347] Updated weights for policy 0, policy_version 1814283 (0.0007) [2023-12-27 18:12:52,966][20347] Updated weights for policy 0, policy_version 1814293 (0.0007) [2023-12-27 18:12:53,312][20381] Updated weights for policy 1, policy_version 1812640 (0.0010) [2023-12-27 18:12:53,370][20381] Updated weights for policy 1, policy_version 1812650 (0.0011) [2023-12-27 18:12:53,415][20381] Updated weights for policy 1, policy_version 1812660 (0.0010) [2023-12-27 18:12:53,615][20347] Updated weights for policy 0, policy_version 1814303 (0.0006) [2023-12-27 18:12:53,674][20347] Updated weights for policy 0, policy_version 1814313 (0.0005) [2023-12-27 18:12:53,732][20347] Updated weights for policy 0, policy_version 1814323 (0.0005) [2023-12-27 18:12:54,060][20381] Updated weights for policy 1, policy_version 1812670 (0.0010) [2023-12-27 18:12:54,114][20381] Updated weights for policy 1, policy_version 1812680 (0.0010) [2023-12-27 18:12:54,167][20381] Updated weights for policy 1, policy_version 1812690 (0.0009) [2023-12-27 18:12:54,280][20347] Updated weights for policy 0, policy_version 1814333 (0.0007) [2023-12-27 18:12:54,332][20347] Updated weights for policy 0, policy_version 1814343 (0.0007) [2023-12-27 18:12:54,397][20347] Updated weights for policy 0, policy_version 1814353 (0.0008) [2023-12-27 18:12:54,736][20381] Updated weights for policy 1, policy_version 1812700 (0.0005) [2023-12-27 18:12:54,802][20381] Updated weights for policy 1, policy_version 1812710 (0.0006) [2023-12-27 18:12:54,866][20381] Updated weights for policy 1, policy_version 1812720 (0.0005) [2023-12-27 18:12:55,087][20347] Updated weights for policy 0, policy_version 1814363 (0.0007) [2023-12-27 18:12:55,139][20347] Updated weights for policy 0, policy_version 1814373 (0.0010) [2023-12-27 18:12:55,194][20347] Updated weights for policy 0, policy_version 1814383 (0.0007) [2023-12-27 18:12:55,418][20381] Updated weights for policy 1, policy_version 1812730 (0.0006) [2023-12-27 18:12:55,478][20381] Updated weights for policy 1, policy_version 1812740 (0.0011) [2023-12-27 18:12:55,536][20381] Updated weights for policy 1, policy_version 1812750 (0.0011) [2023-12-27 18:12:55,591][20381] Updated weights for policy 1, policy_version 1812760 (0.0010) [2023-12-27 18:12:55,816][20347] Updated weights for policy 0, policy_version 1814393 (0.0007) [2023-12-27 18:12:55,864][20347] Updated weights for policy 0, policy_version 1814403 (0.0005) [2023-12-27 18:12:55,910][20347] Updated weights for policy 0, policy_version 1814413 (0.0009) [2023-12-27 18:12:55,954][20347] Updated weights for policy 0, policy_version 1814423 (0.0005) [2023-12-27 18:12:56,234][20381] Updated weights for policy 1, policy_version 1812770 (0.0010) [2023-12-27 18:12:56,285][20381] Updated weights for policy 1, policy_version 1812780 (0.0010) [2023-12-27 18:12:56,336][20381] Updated weights for policy 1, policy_version 1812790 (0.0010) [2023-12-27 18:12:56,551][20347] Updated weights for policy 0, policy_version 1814433 (0.0005) [2023-12-27 18:12:56,599][20347] Updated weights for policy 0, policy_version 1814443 (0.0005) [2023-12-27 18:12:56,645][20347] Updated weights for policy 0, policy_version 1814453 (0.0005) [2023-12-27 18:12:57,029][20381] Updated weights for policy 1, policy_version 1812800 (0.0011) [2023-12-27 18:12:57,073][20381] Updated weights for policy 1, policy_version 1812810 (0.0010) [2023-12-27 18:12:57,128][20381] Updated weights for policy 1, policy_version 1812820 (0.0010) [2023-12-27 18:12:57,208][20347] Updated weights for policy 0, policy_version 1814463 (0.0008) [2023-12-27 18:12:57,271][20347] Updated weights for policy 0, policy_version 1814473 (0.0008) [2023-12-27 18:12:57,316][20347] Updated weights for policy 0, policy_version 1814483 (0.0007) [2023-12-27 18:12:57,362][19170] Fps is (10 sec: 22118.5, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 928727040. Throughput: 0: 10233.0, 1: 10364.6. Samples: 928707648. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:12:57,363][19170] Avg episode reward: [(0, '941.400'), (1, '970.700')] [2023-12-27 18:12:57,834][20381] Updated weights for policy 1, policy_version 1812830 (0.0011) [2023-12-27 18:12:57,885][20381] Updated weights for policy 1, policy_version 1812840 (0.0010) [2023-12-27 18:12:57,940][20381] Updated weights for policy 1, policy_version 1812850 (0.0010) [2023-12-27 18:12:57,974][20347] Updated weights for policy 0, policy_version 1814493 (0.0009) [2023-12-27 18:12:58,023][20347] Updated weights for policy 0, policy_version 1814503 (0.0010) [2023-12-27 18:12:58,073][20347] Updated weights for policy 0, policy_version 1814513 (0.0009) [2023-12-27 18:12:58,754][20381] Updated weights for policy 1, policy_version 1812860 (0.0010) [2023-12-27 18:12:58,823][20381] Updated weights for policy 1, policy_version 1812870 (0.0007) [2023-12-27 18:12:58,868][20347] Updated weights for policy 0, policy_version 1814523 (0.0005) [2023-12-27 18:12:58,885][20381] Updated weights for policy 1, policy_version 1812880 (0.0008) [2023-12-27 18:12:58,932][20347] Updated weights for policy 0, policy_version 1814533 (0.0007) [2023-12-27 18:12:58,989][20347] Updated weights for policy 0, policy_version 1814543 (0.0007) [2023-12-27 18:12:59,615][20381] Updated weights for policy 1, policy_version 1812890 (0.0007) [2023-12-27 18:12:59,678][20381] Updated weights for policy 1, policy_version 1812900 (0.0007) [2023-12-27 18:12:59,678][20347] Updated weights for policy 0, policy_version 1814553 (0.0007) [2023-12-27 18:12:59,728][20347] Updated weights for policy 0, policy_version 1814563 (0.0007) [2023-12-27 18:12:59,737][20381] Updated weights for policy 1, policy_version 1812910 (0.0005) [2023-12-27 18:12:59,773][20347] Updated weights for policy 0, policy_version 1814573 (0.0007) [2023-12-27 18:12:59,797][20381] Updated weights for policy 1, policy_version 1812920 (0.0006) [2023-12-27 18:12:59,828][20347] Updated weights for policy 0, policy_version 1814583 (0.0007) [2023-12-27 18:13:00,440][20381] Updated weights for policy 1, policy_version 1812930 (0.0007) [2023-12-27 18:13:00,495][20381] Updated weights for policy 1, policy_version 1812940 (0.0008) [2023-12-27 18:13:00,528][20347] Updated weights for policy 0, policy_version 1814593 (0.0006) [2023-12-27 18:13:00,548][20381] Updated weights for policy 1, policy_version 1812950 (0.0008) [2023-12-27 18:13:00,585][20347] Updated weights for policy 0, policy_version 1814603 (0.0006) [2023-12-27 18:13:00,643][20347] Updated weights for policy 0, policy_version 1814613 (0.0007) [2023-12-27 18:13:01,265][20381] Updated weights for policy 1, policy_version 1812960 (0.0007) [2023-12-27 18:13:01,296][20347] Updated weights for policy 0, policy_version 1814623 (0.0007) [2023-12-27 18:13:01,320][20381] Updated weights for policy 1, policy_version 1812970 (0.0006) [2023-12-27 18:13:01,355][20347] Updated weights for policy 0, policy_version 1814633 (0.0008) [2023-12-27 18:13:01,387][20381] Updated weights for policy 1, policy_version 1812980 (0.0008) [2023-12-27 18:13:01,418][20347] Updated weights for policy 0, policy_version 1814643 (0.0008) [2023-12-27 18:13:02,048][20381] Updated weights for policy 1, policy_version 1812990 (0.0006) [2023-12-27 18:13:02,099][20381] Updated weights for policy 1, policy_version 1813000 (0.0007) [2023-12-27 18:13:02,144][20347] Updated weights for policy 0, policy_version 1814653 (0.0008) [2023-12-27 18:13:02,154][20381] Updated weights for policy 1, policy_version 1813010 (0.0007) [2023-12-27 18:13:02,195][20347] Updated weights for policy 0, policy_version 1814663 (0.0008) [2023-12-27 18:13:02,247][20347] Updated weights for policy 0, policy_version 1814673 (0.0007) [2023-12-27 18:13:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 928825344. Throughput: 0: 10291.3, 1: 10280.5. Samples: 928829428. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:02,363][19170] Avg episode reward: [(0, '951.220'), (1, '961.070')] [2023-12-27 18:13:02,795][20381] Updated weights for policy 1, policy_version 1813020 (0.0007) [2023-12-27 18:13:02,849][20381] Updated weights for policy 1, policy_version 1813030 (0.0006) [2023-12-27 18:13:02,899][20347] Updated weights for policy 0, policy_version 1814683 (0.0009) [2023-12-27 18:13:02,915][20381] Updated weights for policy 1, policy_version 1813040 (0.0006) [2023-12-27 18:13:02,950][20347] Updated weights for policy 0, policy_version 1814693 (0.0010) [2023-12-27 18:13:02,997][20347] Updated weights for policy 0, policy_version 1814703 (0.0010) [2023-12-27 18:13:03,532][20381] Updated weights for policy 1, policy_version 1813050 (0.0008) [2023-12-27 18:13:03,589][20381] Updated weights for policy 1, policy_version 1813060 (0.0007) [2023-12-27 18:13:03,601][20347] Updated weights for policy 0, policy_version 1814713 (0.0008) [2023-12-27 18:13:03,648][20347] Updated weights for policy 0, policy_version 1814723 (0.0010) [2023-12-27 18:13:03,650][20381] Updated weights for policy 1, policy_version 1813070 (0.0006) [2023-12-27 18:13:03,697][20381] Updated weights for policy 1, policy_version 1813080 (0.0005) [2023-12-27 18:13:03,702][20347] Updated weights for policy 0, policy_version 1814733 (0.0010) [2023-12-27 18:13:03,766][20347] Updated weights for policy 0, policy_version 1814743 (0.0010) [2023-12-27 18:13:04,362][20381] Updated weights for policy 1, policy_version 1813090 (0.0006) [2023-12-27 18:13:04,414][20381] Updated weights for policy 1, policy_version 1813100 (0.0006) [2023-12-27 18:13:04,459][20347] Updated weights for policy 0, policy_version 1814753 (0.0010) [2023-12-27 18:13:04,468][20381] Updated weights for policy 1, policy_version 1813110 (0.0008) [2023-12-27 18:13:04,514][20347] Updated weights for policy 0, policy_version 1814763 (0.0010) [2023-12-27 18:13:04,565][20347] Updated weights for policy 0, policy_version 1814773 (0.0009) [2023-12-27 18:13:05,047][20381] Updated weights for policy 1, policy_version 1813120 (0.0006) [2023-12-27 18:13:05,104][20381] Updated weights for policy 1, policy_version 1813130 (0.0005) [2023-12-27 18:13:05,158][20381] Updated weights for policy 1, policy_version 1813140 (0.0005) [2023-12-27 18:13:05,205][20347] Updated weights for policy 0, policy_version 1814783 (0.0009) [2023-12-27 18:13:05,260][20347] Updated weights for policy 0, policy_version 1814793 (0.0010) [2023-12-27 18:13:05,309][20347] Updated weights for policy 0, policy_version 1814803 (0.0010) [2023-12-27 18:13:05,682][20381] Updated weights for policy 1, policy_version 1813150 (0.0007) [2023-12-27 18:13:05,732][20381] Updated weights for policy 1, policy_version 1813160 (0.0006) [2023-12-27 18:13:05,789][20381] Updated weights for policy 1, policy_version 1813170 (0.0008) [2023-12-27 18:13:05,973][20347] Updated weights for policy 0, policy_version 1814813 (0.0010) [2023-12-27 18:13:06,021][20347] Updated weights for policy 0, policy_version 1814823 (0.0010) [2023-12-27 18:13:06,073][20347] Updated weights for policy 0, policy_version 1814833 (0.0010) [2023-12-27 18:13:06,486][20381] Updated weights for policy 1, policy_version 1813180 (0.0008) [2023-12-27 18:13:06,550][20381] Updated weights for policy 1, policy_version 1813190 (0.0006) [2023-12-27 18:13:06,616][20381] Updated weights for policy 1, policy_version 1813200 (0.0007) [2023-12-27 18:13:06,835][20347] Updated weights for policy 0, policy_version 1814843 (0.0008) [2023-12-27 18:13:06,903][20347] Updated weights for policy 0, policy_version 1814853 (0.0006) [2023-12-27 18:13:06,970][20347] Updated weights for policy 0, policy_version 1814863 (0.0005) [2023-12-27 18:13:07,271][20381] Updated weights for policy 1, policy_version 1813210 (0.0006) [2023-12-27 18:13:07,326][20381] Updated weights for policy 1, policy_version 1813220 (0.0008) [2023-12-27 18:13:07,363][19170] Fps is (10 sec: 19659.6, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 928923648. Throughput: 0: 10317.3, 1: 10352.2. Samples: 928894756. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:07,364][19170] Avg episode reward: [(0, '970.680'), (1, '922.280')] [2023-12-27 18:13:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001814872_464691200.pth... [2023-12-27 18:13:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001813656_464379904.pth [2023-12-27 18:13:07,380][20381] Updated weights for policy 1, policy_version 1813230 (0.0008) [2023-12-27 18:13:07,424][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001813240_464240640.pth... [2023-12-27 18:13:07,426][20381] Updated weights for policy 1, policy_version 1813240 (0.0011) [2023-12-27 18:13:07,427][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001812024_463929344.pth [2023-12-27 18:13:07,598][20347] Updated weights for policy 0, policy_version 1814873 (0.0006) [2023-12-27 18:13:07,652][20347] Updated weights for policy 0, policy_version 1814883 (0.0007) [2023-12-27 18:13:07,705][20347] Updated weights for policy 0, policy_version 1814893 (0.0006) [2023-12-27 18:13:07,759][20347] Updated weights for policy 0, policy_version 1814903 (0.0008) [2023-12-27 18:13:08,062][20381] Updated weights for policy 1, policy_version 1813250 (0.0005) [2023-12-27 18:13:08,108][20381] Updated weights for policy 1, policy_version 1813260 (0.0005) [2023-12-27 18:13:08,156][20381] Updated weights for policy 1, policy_version 1813270 (0.0005) [2023-12-27 18:13:08,513][20347] Updated weights for policy 0, policy_version 1814913 (0.0007) [2023-12-27 18:13:08,577][20347] Updated weights for policy 0, policy_version 1814923 (0.0007) [2023-12-27 18:13:08,635][20347] Updated weights for policy 0, policy_version 1814933 (0.0007) [2023-12-27 18:13:08,779][20381] Updated weights for policy 1, policy_version 1813280 (0.0009) [2023-12-27 18:13:08,841][20381] Updated weights for policy 1, policy_version 1813290 (0.0011) [2023-12-27 18:13:08,904][20381] Updated weights for policy 1, policy_version 1813300 (0.0010) [2023-12-27 18:13:09,311][20347] Updated weights for policy 0, policy_version 1814943 (0.0007) [2023-12-27 18:13:09,376][20347] Updated weights for policy 0, policy_version 1814953 (0.0007) [2023-12-27 18:13:09,446][20347] Updated weights for policy 0, policy_version 1814963 (0.0008) [2023-12-27 18:13:09,639][20381] Updated weights for policy 1, policy_version 1813310 (0.0007) [2023-12-27 18:13:09,699][20381] Updated weights for policy 1, policy_version 1813320 (0.0006) [2023-12-27 18:13:09,762][20381] Updated weights for policy 1, policy_version 1813330 (0.0005) [2023-12-27 18:13:10,203][20347] Updated weights for policy 0, policy_version 1814973 (0.0008) [2023-12-27 18:13:10,256][20347] Updated weights for policy 0, policy_version 1814983 (0.0007) [2023-12-27 18:13:10,308][20347] Updated weights for policy 0, policy_version 1814993 (0.0008) [2023-12-27 18:13:10,406][20381] Updated weights for policy 1, policy_version 1813340 (0.0006) [2023-12-27 18:13:10,461][20381] Updated weights for policy 1, policy_version 1813350 (0.0008) [2023-12-27 18:13:10,518][20381] Updated weights for policy 1, policy_version 1813360 (0.0008) [2023-12-27 18:13:10,999][20347] Updated weights for policy 0, policy_version 1815003 (0.0008) [2023-12-27 18:13:11,065][20347] Updated weights for policy 0, policy_version 1815013 (0.0009) [2023-12-27 18:13:11,128][20347] Updated weights for policy 0, policy_version 1815023 (0.0008) [2023-12-27 18:13:11,278][20381] Updated weights for policy 1, policy_version 1813370 (0.0007) [2023-12-27 18:13:11,341][20381] Updated weights for policy 1, policy_version 1813380 (0.0008) [2023-12-27 18:13:11,415][20381] Updated weights for policy 1, policy_version 1813390 (0.0007) [2023-12-27 18:13:11,468][20381] Updated weights for policy 1, policy_version 1813400 (0.0009) [2023-12-27 18:13:11,899][20347] Updated weights for policy 0, policy_version 1815033 (0.0009) [2023-12-27 18:13:11,945][20347] Updated weights for policy 0, policy_version 1815043 (0.0007) [2023-12-27 18:13:11,994][20347] Updated weights for policy 0, policy_version 1815053 (0.0006) [2023-12-27 18:13:12,056][20347] Updated weights for policy 0, policy_version 1815063 (0.0006) [2023-12-27 18:13:12,141][20381] Updated weights for policy 1, policy_version 1813410 (0.0006) [2023-12-27 18:13:12,204][20381] Updated weights for policy 1, policy_version 1813420 (0.0006) [2023-12-27 18:13:12,265][20381] Updated weights for policy 1, policy_version 1813430 (0.0007) [2023-12-27 18:13:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 929030144. Throughput: 0: 10266.1, 1: 10356.7. Samples: 929016768. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:12,363][19170] Avg episode reward: [(0, '970.700'), (1, '902.610')] [2023-12-27 18:13:12,714][20347] Updated weights for policy 0, policy_version 1815073 (0.0008) [2023-12-27 18:13:12,778][20347] Updated weights for policy 0, policy_version 1815083 (0.0009) [2023-12-27 18:13:12,840][20347] Updated weights for policy 0, policy_version 1815093 (0.0007) [2023-12-27 18:13:12,925][20381] Updated weights for policy 1, policy_version 1813440 (0.0008) [2023-12-27 18:13:12,973][20381] Updated weights for policy 1, policy_version 1813450 (0.0008) [2023-12-27 18:13:13,037][20381] Updated weights for policy 1, policy_version 1813460 (0.0007) [2023-12-27 18:13:13,515][20347] Updated weights for policy 0, policy_version 1815103 (0.0009) [2023-12-27 18:13:13,560][20347] Updated weights for policy 0, policy_version 1815113 (0.0005) [2023-12-27 18:13:13,606][20347] Updated weights for policy 0, policy_version 1815123 (0.0005) [2023-12-27 18:13:13,675][20381] Updated weights for policy 1, policy_version 1813470 (0.0009) [2023-12-27 18:13:13,729][20381] Updated weights for policy 1, policy_version 1813480 (0.0009) [2023-12-27 18:13:13,789][20381] Updated weights for policy 1, policy_version 1813490 (0.0006) [2023-12-27 18:13:14,257][20347] Updated weights for policy 0, policy_version 1815133 (0.0008) [2023-12-27 18:13:14,314][20347] Updated weights for policy 0, policy_version 1815143 (0.0010) [2023-12-27 18:13:14,354][20381] Updated weights for policy 1, policy_version 1813500 (0.0010) [2023-12-27 18:13:14,372][20347] Updated weights for policy 0, policy_version 1815153 (0.0010) [2023-12-27 18:13:14,417][20381] Updated weights for policy 1, policy_version 1813510 (0.0007) [2023-12-27 18:13:14,478][20381] Updated weights for policy 1, policy_version 1813520 (0.0007) [2023-12-27 18:13:15,046][20347] Updated weights for policy 0, policy_version 1815163 (0.0006) [2023-12-27 18:13:15,114][20347] Updated weights for policy 0, policy_version 1815173 (0.0008) [2023-12-27 18:13:15,170][20381] Updated weights for policy 1, policy_version 1813530 (0.0007) [2023-12-27 18:13:15,179][20347] Updated weights for policy 0, policy_version 1815183 (0.0007) [2023-12-27 18:13:15,232][20381] Updated weights for policy 1, policy_version 1813540 (0.0007) [2023-12-27 18:13:15,286][20381] Updated weights for policy 1, policy_version 1813550 (0.0008) [2023-12-27 18:13:15,345][20381] Updated weights for policy 1, policy_version 1813560 (0.0008) [2023-12-27 18:13:15,759][20347] Updated weights for policy 0, policy_version 1815193 (0.0008) [2023-12-27 18:13:15,808][20347] Updated weights for policy 0, policy_version 1815203 (0.0008) [2023-12-27 18:13:15,863][20347] Updated weights for policy 0, policy_version 1815213 (0.0008) [2023-12-27 18:13:15,922][20347] Updated weights for policy 0, policy_version 1815223 (0.0005) [2023-12-27 18:13:16,017][20381] Updated weights for policy 1, policy_version 1813570 (0.0008) [2023-12-27 18:13:16,074][20381] Updated weights for policy 1, policy_version 1813580 (0.0009) [2023-12-27 18:13:16,135][20381] Updated weights for policy 1, policy_version 1813590 (0.0008) [2023-12-27 18:13:16,500][20347] Updated weights for policy 0, policy_version 1815233 (0.0008) [2023-12-27 18:13:16,550][20347] Updated weights for policy 0, policy_version 1815243 (0.0007) [2023-12-27 18:13:16,603][20347] Updated weights for policy 0, policy_version 1815253 (0.0005) [2023-12-27 18:13:16,855][20381] Updated weights for policy 1, policy_version 1813600 (0.0006) [2023-12-27 18:13:16,917][20381] Updated weights for policy 1, policy_version 1813610 (0.0007) [2023-12-27 18:13:16,982][20381] Updated weights for policy 1, policy_version 1813620 (0.0011) [2023-12-27 18:13:17,234][20347] Updated weights for policy 0, policy_version 1815263 (0.0007) [2023-12-27 18:13:17,289][20347] Updated weights for policy 0, policy_version 1815273 (0.0005) [2023-12-27 18:13:17,340][20347] Updated weights for policy 0, policy_version 1815283 (0.0006) [2023-12-27 18:13:17,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 929128448. Throughput: 0: 10314.2, 1: 10319.7. Samples: 929142784. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:17,363][19170] Avg episode reward: [(0, '980.480'), (1, '951.140')] [2023-12-27 18:13:17,591][20381] Updated weights for policy 1, policy_version 1813630 (0.0011) [2023-12-27 18:13:17,643][20381] Updated weights for policy 1, policy_version 1813640 (0.0010) [2023-12-27 18:13:17,701][20381] Updated weights for policy 1, policy_version 1813650 (0.0011) [2023-12-27 18:13:17,909][20347] Updated weights for policy 0, policy_version 1815293 (0.0008) [2023-12-27 18:13:17,970][20347] Updated weights for policy 0, policy_version 1815303 (0.0010) [2023-12-27 18:13:18,038][20347] Updated weights for policy 0, policy_version 1815313 (0.0010) [2023-12-27 18:13:18,375][20381] Updated weights for policy 1, policy_version 1813660 (0.0011) [2023-12-27 18:13:18,438][20381] Updated weights for policy 1, policy_version 1813670 (0.0011) [2023-12-27 18:13:18,498][20381] Updated weights for policy 1, policy_version 1813680 (0.0011) [2023-12-27 18:13:18,725][20347] Updated weights for policy 0, policy_version 1815323 (0.0010) [2023-12-27 18:13:18,770][20347] Updated weights for policy 0, policy_version 1815333 (0.0007) [2023-12-27 18:13:18,819][20347] Updated weights for policy 0, policy_version 1815343 (0.0007) [2023-12-27 18:13:19,194][20381] Updated weights for policy 1, policy_version 1813690 (0.0010) [2023-12-27 18:13:19,256][20381] Updated weights for policy 1, policy_version 1813700 (0.0008) [2023-12-27 18:13:19,314][20381] Updated weights for policy 1, policy_version 1813710 (0.0006) [2023-12-27 18:13:19,381][20381] Updated weights for policy 1, policy_version 1813720 (0.0008) [2023-12-27 18:13:19,571][20347] Updated weights for policy 0, policy_version 1815353 (0.0008) [2023-12-27 18:13:19,644][20347] Updated weights for policy 0, policy_version 1815363 (0.0011) [2023-12-27 18:13:19,712][20347] Updated weights for policy 0, policy_version 1815373 (0.0010) [2023-12-27 18:13:19,769][20347] Updated weights for policy 0, policy_version 1815383 (0.0011) [2023-12-27 18:13:20,051][20381] Updated weights for policy 1, policy_version 1813730 (0.0006) [2023-12-27 18:13:20,110][20381] Updated weights for policy 1, policy_version 1813740 (0.0006) [2023-12-27 18:13:20,165][20381] Updated weights for policy 1, policy_version 1813750 (0.0006) [2023-12-27 18:13:20,479][20347] Updated weights for policy 0, policy_version 1815393 (0.0006) [2023-12-27 18:13:20,538][20347] Updated weights for policy 0, policy_version 1815403 (0.0006) [2023-12-27 18:13:20,606][20347] Updated weights for policy 0, policy_version 1815413 (0.0008) [2023-12-27 18:13:20,781][20381] Updated weights for policy 1, policy_version 1813760 (0.0010) [2023-12-27 18:13:20,838][20381] Updated weights for policy 1, policy_version 1813770 (0.0011) [2023-12-27 18:13:20,893][20381] Updated weights for policy 1, policy_version 1813780 (0.0011) [2023-12-27 18:13:21,293][20347] Updated weights for policy 0, policy_version 1815423 (0.0007) [2023-12-27 18:13:21,353][20347] Updated weights for policy 0, policy_version 1815433 (0.0007) [2023-12-27 18:13:21,424][20347] Updated weights for policy 0, policy_version 1815443 (0.0008) [2023-12-27 18:13:21,681][20381] Updated weights for policy 1, policy_version 1813790 (0.0011) [2023-12-27 18:13:21,748][20381] Updated weights for policy 1, policy_version 1813800 (0.0011) [2023-12-27 18:13:21,809][20381] Updated weights for policy 1, policy_version 1813810 (0.0011) [2023-12-27 18:13:22,190][20347] Updated weights for policy 0, policy_version 1815453 (0.0007) [2023-12-27 18:13:22,250][20347] Updated weights for policy 0, policy_version 1815463 (0.0008) [2023-12-27 18:13:22,314][20347] Updated weights for policy 0, policy_version 1815473 (0.0008) [2023-12-27 18:13:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 929234944. Throughput: 0: 10303.6, 1: 10312.0. Samples: 929204024. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:22,363][19170] Avg episode reward: [(0, '980.440'), (1, '941.420')] [2023-12-27 18:13:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001813816_464388096.pth... [2023-12-27 18:13:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001815480_464846848.pth... [2023-12-27 18:13:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001812600_464076800.pth [2023-12-27 18:13:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001814264_464535552.pth [2023-12-27 18:13:22,525][20381] Updated weights for policy 1, policy_version 1813820 (0.0009) [2023-12-27 18:13:22,584][20381] Updated weights for policy 1, policy_version 1813830 (0.0008) [2023-12-27 18:13:22,643][20381] Updated weights for policy 1, policy_version 1813840 (0.0008) [2023-12-27 18:13:23,059][20347] Updated weights for policy 0, policy_version 1815483 (0.0008) [2023-12-27 18:13:23,111][20347] Updated weights for policy 0, policy_version 1815493 (0.0010) [2023-12-27 18:13:23,172][20347] Updated weights for policy 0, policy_version 1815503 (0.0010) [2023-12-27 18:13:23,347][20381] Updated weights for policy 1, policy_version 1813850 (0.0006) [2023-12-27 18:13:23,411][20381] Updated weights for policy 1, policy_version 1813860 (0.0005) [2023-12-27 18:13:23,465][20381] Updated weights for policy 1, policy_version 1813870 (0.0005) [2023-12-27 18:13:23,526][20381] Updated weights for policy 1, policy_version 1813880 (0.0005) [2023-12-27 18:13:23,786][20347] Updated weights for policy 0, policy_version 1815513 (0.0005) [2023-12-27 18:13:23,845][20347] Updated weights for policy 0, policy_version 1815523 (0.0010) [2023-12-27 18:13:23,906][20347] Updated weights for policy 0, policy_version 1815533 (0.0006) [2023-12-27 18:13:23,966][20347] Updated weights for policy 0, policy_version 1815543 (0.0005) [2023-12-27 18:13:24,051][20381] Updated weights for policy 1, policy_version 1813890 (0.0006) [2023-12-27 18:13:24,112][20381] Updated weights for policy 1, policy_version 1813900 (0.0008) [2023-12-27 18:13:24,173][20381] Updated weights for policy 1, policy_version 1813910 (0.0010) [2023-12-27 18:13:24,509][20347] Updated weights for policy 0, policy_version 1815553 (0.0005) [2023-12-27 18:13:24,564][20347] Updated weights for policy 0, policy_version 1815563 (0.0009) [2023-12-27 18:13:24,623][20347] Updated weights for policy 0, policy_version 1815573 (0.0010) [2023-12-27 18:13:24,690][20381] Updated weights for policy 1, policy_version 1813920 (0.0007) [2023-12-27 18:13:24,739][20381] Updated weights for policy 1, policy_version 1813930 (0.0007) [2023-12-27 18:13:24,786][20381] Updated weights for policy 1, policy_version 1813940 (0.0007) [2023-12-27 18:13:25,231][20347] Updated weights for policy 0, policy_version 1815583 (0.0007) [2023-12-27 18:13:25,281][20347] Updated weights for policy 0, policy_version 1815593 (0.0010) [2023-12-27 18:13:25,336][20347] Updated weights for policy 0, policy_version 1815603 (0.0010) [2023-12-27 18:13:25,472][20381] Updated weights for policy 1, policy_version 1813950 (0.0007) [2023-12-27 18:13:25,522][20381] Updated weights for policy 1, policy_version 1813960 (0.0005) [2023-12-27 18:13:25,578][20381] Updated weights for policy 1, policy_version 1813970 (0.0005) [2023-12-27 18:13:25,976][20347] Updated weights for policy 0, policy_version 1815613 (0.0010) [2023-12-27 18:13:26,032][20347] Updated weights for policy 0, policy_version 1815623 (0.0010) [2023-12-27 18:13:26,082][20347] Updated weights for policy 0, policy_version 1815633 (0.0010) [2023-12-27 18:13:26,166][20381] Updated weights for policy 1, policy_version 1813980 (0.0008) [2023-12-27 18:13:26,229][20381] Updated weights for policy 1, policy_version 1813990 (0.0007) [2023-12-27 18:13:26,285][20381] Updated weights for policy 1, policy_version 1814000 (0.0005) [2023-12-27 18:13:26,761][20347] Updated weights for policy 0, policy_version 1815643 (0.0010) [2023-12-27 18:13:26,807][20347] Updated weights for policy 0, policy_version 1815653 (0.0010) [2023-12-27 18:13:26,856][20347] Updated weights for policy 0, policy_version 1815663 (0.0010) [2023-12-27 18:13:26,879][20381] Updated weights for policy 1, policy_version 1814010 (0.0007) [2023-12-27 18:13:26,937][20381] Updated weights for policy 1, policy_version 1814020 (0.0010) [2023-12-27 18:13:26,988][20381] Updated weights for policy 1, policy_version 1814030 (0.0010) [2023-12-27 18:13:27,043][20381] Updated weights for policy 1, policy_version 1814040 (0.0010) [2023-12-27 18:13:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 929341440. Throughput: 0: 10369.7, 1: 10364.3. Samples: 929329948. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:27,363][19170] Avg episode reward: [(0, '970.640'), (1, '931.830')] [2023-12-27 18:13:27,510][20347] Updated weights for policy 0, policy_version 1815673 (0.0010) [2023-12-27 18:13:27,558][20347] Updated weights for policy 0, policy_version 1815683 (0.0005) [2023-12-27 18:13:27,607][20347] Updated weights for policy 0, policy_version 1815693 (0.0006) [2023-12-27 18:13:27,651][20347] Updated weights for policy 0, policy_version 1815703 (0.0010) [2023-12-27 18:13:27,726][20381] Updated weights for policy 1, policy_version 1814050 (0.0005) [2023-12-27 18:13:27,769][20381] Updated weights for policy 1, policy_version 1814060 (0.0005) [2023-12-27 18:13:27,820][20381] Updated weights for policy 1, policy_version 1814070 (0.0006) [2023-12-27 18:13:28,270][20347] Updated weights for policy 0, policy_version 1815713 (0.0010) [2023-12-27 18:13:28,335][20347] Updated weights for policy 0, policy_version 1815723 (0.0012) [2023-12-27 18:13:28,378][20381] Updated weights for policy 1, policy_version 1814080 (0.0007) [2023-12-27 18:13:28,394][20347] Updated weights for policy 0, policy_version 1815733 (0.0007) [2023-12-27 18:13:28,438][20381] Updated weights for policy 1, policy_version 1814090 (0.0009) [2023-12-27 18:13:28,502][20381] Updated weights for policy 1, policy_version 1814100 (0.0008) [2023-12-27 18:13:29,018][20347] Updated weights for policy 0, policy_version 1815743 (0.0009) [2023-12-27 18:13:29,069][20347] Updated weights for policy 0, policy_version 1815753 (0.0007) [2023-12-27 18:13:29,116][20381] Updated weights for policy 1, policy_version 1814110 (0.0007) [2023-12-27 18:13:29,118][20347] Updated weights for policy 0, policy_version 1815763 (0.0009) [2023-12-27 18:13:29,167][20381] Updated weights for policy 1, policy_version 1814120 (0.0006) [2023-12-27 18:13:29,213][20381] Updated weights for policy 1, policy_version 1814130 (0.0007) [2023-12-27 18:13:29,846][20347] Updated weights for policy 0, policy_version 1815773 (0.0009) [2023-12-27 18:13:29,910][20347] Updated weights for policy 0, policy_version 1815783 (0.0006) [2023-12-27 18:13:29,941][20381] Updated weights for policy 1, policy_version 1814140 (0.0007) [2023-12-27 18:13:29,978][20347] Updated weights for policy 0, policy_version 1815793 (0.0009) [2023-12-27 18:13:30,006][20381] Updated weights for policy 1, policy_version 1814150 (0.0007) [2023-12-27 18:13:30,067][20381] Updated weights for policy 1, policy_version 1814160 (0.0008) [2023-12-27 18:13:30,649][20347] Updated weights for policy 0, policy_version 1815803 (0.0009) [2023-12-27 18:13:30,698][20347] Updated weights for policy 0, policy_version 1815813 (0.0007) [2023-12-27 18:13:30,746][20347] Updated weights for policy 0, policy_version 1815823 (0.0007) [2023-12-27 18:13:30,748][20381] Updated weights for policy 1, policy_version 1814170 (0.0008) [2023-12-27 18:13:30,802][20381] Updated weights for policy 1, policy_version 1814180 (0.0010) [2023-12-27 18:13:30,853][20381] Updated weights for policy 1, policy_version 1814190 (0.0010) [2023-12-27 18:13:30,907][20381] Updated weights for policy 1, policy_version 1814200 (0.0010) [2023-12-27 18:13:31,474][20347] Updated weights for policy 0, policy_version 1815833 (0.0005) [2023-12-27 18:13:31,529][20347] Updated weights for policy 0, policy_version 1815843 (0.0005) [2023-12-27 18:13:31,589][20347] Updated weights for policy 0, policy_version 1815853 (0.0007) [2023-12-27 18:13:31,593][20381] Updated weights for policy 1, policy_version 1814210 (0.0007) [2023-12-27 18:13:31,648][20347] Updated weights for policy 0, policy_version 1815863 (0.0007) [2023-12-27 18:13:31,651][20381] Updated weights for policy 1, policy_version 1814220 (0.0007) [2023-12-27 18:13:31,697][20381] Updated weights for policy 1, policy_version 1814230 (0.0005) [2023-12-27 18:13:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 929439744. Throughput: 0: 10382.6, 1: 10415.3. Samples: 929455652. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:32,363][19170] Avg episode reward: [(0, '970.650'), (1, '951.320')] [2023-12-27 18:13:32,402][20347] Updated weights for policy 0, policy_version 1815873 (0.0008) [2023-12-27 18:13:32,415][20381] Updated weights for policy 1, policy_version 1814240 (0.0008) [2023-12-27 18:13:32,460][20347] Updated weights for policy 0, policy_version 1815883 (0.0006) [2023-12-27 18:13:32,465][20381] Updated weights for policy 1, policy_version 1814250 (0.0011) [2023-12-27 18:13:32,507][20381] Updated weights for policy 1, policy_version 1814260 (0.0010) [2023-12-27 18:13:32,520][20347] Updated weights for policy 0, policy_version 1815893 (0.0006) [2023-12-27 18:13:33,188][20381] Updated weights for policy 1, policy_version 1814270 (0.0008) [2023-12-27 18:13:33,201][20347] Updated weights for policy 0, policy_version 1815903 (0.0006) [2023-12-27 18:13:33,239][20381] Updated weights for policy 1, policy_version 1814280 (0.0005) [2023-12-27 18:13:33,256][20347] Updated weights for policy 0, policy_version 1815913 (0.0007) [2023-12-27 18:13:33,291][20381] Updated weights for policy 1, policy_version 1814290 (0.0006) [2023-12-27 18:13:33,307][20347] Updated weights for policy 0, policy_version 1815923 (0.0005) [2023-12-27 18:13:33,949][20381] Updated weights for policy 1, policy_version 1814300 (0.0005) [2023-12-27 18:13:33,965][20347] Updated weights for policy 0, policy_version 1815933 (0.0008) [2023-12-27 18:13:34,005][20381] Updated weights for policy 1, policy_version 1814310 (0.0005) [2023-12-27 18:13:34,013][20347] Updated weights for policy 0, policy_version 1815943 (0.0010) [2023-12-27 18:13:34,062][20381] Updated weights for policy 1, policy_version 1814320 (0.0008) [2023-12-27 18:13:34,078][20347] Updated weights for policy 0, policy_version 1815953 (0.0009) [2023-12-27 18:13:34,746][20381] Updated weights for policy 1, policy_version 1814330 (0.0007) [2023-12-27 18:13:34,791][20347] Updated weights for policy 0, policy_version 1815963 (0.0006) [2023-12-27 18:13:34,804][20381] Updated weights for policy 1, policy_version 1814340 (0.0007) [2023-12-27 18:13:34,845][20347] Updated weights for policy 0, policy_version 1815973 (0.0006) [2023-12-27 18:13:34,862][20381] Updated weights for policy 1, policy_version 1814350 (0.0005) [2023-12-27 18:13:34,903][20347] Updated weights for policy 0, policy_version 1815983 (0.0008) [2023-12-27 18:13:34,916][20381] Updated weights for policy 1, policy_version 1814360 (0.0006) [2023-12-27 18:13:35,452][20347] Updated weights for policy 0, policy_version 1815993 (0.0008) [2023-12-27 18:13:35,505][20347] Updated weights for policy 0, policy_version 1816003 (0.0005) [2023-12-27 18:13:35,551][20347] Updated weights for policy 0, policy_version 1816013 (0.0005) [2023-12-27 18:13:35,602][20347] Updated weights for policy 0, policy_version 1816023 (0.0009) [2023-12-27 18:13:35,627][20381] Updated weights for policy 1, policy_version 1814370 (0.0006) [2023-12-27 18:13:35,688][20381] Updated weights for policy 1, policy_version 1814380 (0.0007) [2023-12-27 18:13:35,750][20381] Updated weights for policy 1, policy_version 1814390 (0.0007) [2023-12-27 18:13:36,203][20347] Updated weights for policy 0, policy_version 1816033 (0.0006) [2023-12-27 18:13:36,258][20347] Updated weights for policy 0, policy_version 1816043 (0.0006) [2023-12-27 18:13:36,310][20347] Updated weights for policy 0, policy_version 1816053 (0.0006) [2023-12-27 18:13:36,473][20381] Updated weights for policy 1, policy_version 1814400 (0.0008) [2023-12-27 18:13:36,530][20381] Updated weights for policy 1, policy_version 1814410 (0.0008) [2023-12-27 18:13:36,592][20381] Updated weights for policy 1, policy_version 1814420 (0.0009) [2023-12-27 18:13:36,968][20347] Updated weights for policy 0, policy_version 1816063 (0.0007) [2023-12-27 18:13:37,027][20347] Updated weights for policy 0, policy_version 1816073 (0.0008) [2023-12-27 18:13:37,080][20347] Updated weights for policy 0, policy_version 1816083 (0.0008) [2023-12-27 18:13:37,299][20381] Updated weights for policy 1, policy_version 1814430 (0.0007) [2023-12-27 18:13:37,349][20381] Updated weights for policy 1, policy_version 1814440 (0.0008) [2023-12-27 18:13:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 929546240. Throughput: 0: 10373.3, 1: 10432.0. Samples: 929516272. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:37,363][19170] Avg episode reward: [(0, '970.630'), (1, '961.060')] [2023-12-27 18:13:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001816088_465002496.pth... [2023-12-27 18:13:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001814872_464691200.pth [2023-12-27 18:13:37,403][20381] Updated weights for policy 1, policy_version 1814450 (0.0008) [2023-12-27 18:13:37,430][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001814456_464551936.pth... [2023-12-27 18:13:37,433][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001813240_464240640.pth [2023-12-27 18:13:37,781][20347] Updated weights for policy 0, policy_version 1816093 (0.0008) [2023-12-27 18:13:37,836][20347] Updated weights for policy 0, policy_version 1816103 (0.0007) [2023-12-27 18:13:37,890][20347] Updated weights for policy 0, policy_version 1816113 (0.0007) [2023-12-27 18:13:38,109][20381] Updated weights for policy 1, policy_version 1814460 (0.0007) [2023-12-27 18:13:38,159][20381] Updated weights for policy 1, policy_version 1814470 (0.0006) [2023-12-27 18:13:38,216][20381] Updated weights for policy 1, policy_version 1814480 (0.0008) [2023-12-27 18:13:38,603][20347] Updated weights for policy 0, policy_version 1816123 (0.0008) [2023-12-27 18:13:38,647][20347] Updated weights for policy 0, policy_version 1816133 (0.0007) [2023-12-27 18:13:38,692][20347] Updated weights for policy 0, policy_version 1816143 (0.0007) [2023-12-27 18:13:38,932][20381] Updated weights for policy 1, policy_version 1814490 (0.0007) [2023-12-27 18:13:38,989][20381] Updated weights for policy 1, policy_version 1814500 (0.0007) [2023-12-27 18:13:39,037][20381] Updated weights for policy 1, policy_version 1814510 (0.0008) [2023-12-27 18:13:39,093][20381] Updated weights for policy 1, policy_version 1814520 (0.0007) [2023-12-27 18:13:39,396][20347] Updated weights for policy 0, policy_version 1816153 (0.0008) [2023-12-27 18:13:39,462][20347] Updated weights for policy 0, policy_version 1816163 (0.0006) [2023-12-27 18:13:39,514][20347] Updated weights for policy 0, policy_version 1816173 (0.0005) [2023-12-27 18:13:39,576][20347] Updated weights for policy 0, policy_version 1816183 (0.0006) [2023-12-27 18:13:39,924][20381] Updated weights for policy 1, policy_version 1814530 (0.0008) [2023-12-27 18:13:39,989][20381] Updated weights for policy 1, policy_version 1814540 (0.0007) [2023-12-27 18:13:40,044][20381] Updated weights for policy 1, policy_version 1814550 (0.0007) [2023-12-27 18:13:40,323][20347] Updated weights for policy 0, policy_version 1816193 (0.0006) [2023-12-27 18:13:40,375][20347] Updated weights for policy 0, policy_version 1816203 (0.0007) [2023-12-27 18:13:40,438][20347] Updated weights for policy 0, policy_version 1816213 (0.0005) [2023-12-27 18:13:40,764][20381] Updated weights for policy 1, policy_version 1814560 (0.0007) [2023-12-27 18:13:40,819][20381] Updated weights for policy 1, policy_version 1814570 (0.0007) [2023-12-27 18:13:40,879][20381] Updated weights for policy 1, policy_version 1814580 (0.0008) [2023-12-27 18:13:41,065][20347] Updated weights for policy 0, policy_version 1816223 (0.0007) [2023-12-27 18:13:41,132][20347] Updated weights for policy 0, policy_version 1816233 (0.0008) [2023-12-27 18:13:41,193][20347] Updated weights for policy 0, policy_version 1816243 (0.0008) [2023-12-27 18:13:41,577][20381] Updated weights for policy 1, policy_version 1814590 (0.0008) [2023-12-27 18:13:41,642][20381] Updated weights for policy 1, policy_version 1814600 (0.0008) [2023-12-27 18:13:41,695][20381] Updated weights for policy 1, policy_version 1814610 (0.0006) [2023-12-27 18:13:42,012][20347] Updated weights for policy 0, policy_version 1816253 (0.0007) [2023-12-27 18:13:42,058][20347] Updated weights for policy 0, policy_version 1816263 (0.0008) [2023-12-27 18:13:42,108][20347] Updated weights for policy 0, policy_version 1816273 (0.0008) [2023-12-27 18:13:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 929644544. Throughput: 0: 10341.3, 1: 10299.6. Samples: 929636492. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:42,363][19170] Avg episode reward: [(0, '970.620'), (1, '931.930')] [2023-12-27 18:13:42,370][20381] Updated weights for policy 1, policy_version 1814620 (0.0008) [2023-12-27 18:13:42,434][20381] Updated weights for policy 1, policy_version 1814630 (0.0007) [2023-12-27 18:13:42,492][20381] Updated weights for policy 1, policy_version 1814640 (0.0008) [2023-12-27 18:13:42,846][20347] Updated weights for policy 0, policy_version 1816283 (0.0008) [2023-12-27 18:13:42,894][20347] Updated weights for policy 0, policy_version 1816293 (0.0007) [2023-12-27 18:13:42,947][20347] Updated weights for policy 0, policy_version 1816303 (0.0006) [2023-12-27 18:13:43,150][20381] Updated weights for policy 1, policy_version 1814650 (0.0008) [2023-12-27 18:13:43,207][20381] Updated weights for policy 1, policy_version 1814660 (0.0005) [2023-12-27 18:13:43,263][20381] Updated weights for policy 1, policy_version 1814670 (0.0007) [2023-12-27 18:13:43,325][20381] Updated weights for policy 1, policy_version 1814680 (0.0008) [2023-12-27 18:13:43,656][20347] Updated weights for policy 0, policy_version 1816313 (0.0008) [2023-12-27 18:13:43,704][20347] Updated weights for policy 0, policy_version 1816323 (0.0007) [2023-12-27 18:13:43,749][20347] Updated weights for policy 0, policy_version 1816333 (0.0007) [2023-12-27 18:13:43,807][20347] Updated weights for policy 0, policy_version 1816343 (0.0007) [2023-12-27 18:13:43,961][20381] Updated weights for policy 1, policy_version 1814690 (0.0006) [2023-12-27 18:13:44,016][20381] Updated weights for policy 1, policy_version 1814700 (0.0005) [2023-12-27 18:13:44,061][20381] Updated weights for policy 1, policy_version 1814710 (0.0007) [2023-12-27 18:13:44,534][20347] Updated weights for policy 0, policy_version 1816353 (0.0007) [2023-12-27 18:13:44,583][20347] Updated weights for policy 0, policy_version 1816363 (0.0007) [2023-12-27 18:13:44,656][20347] Updated weights for policy 0, policy_version 1816373 (0.0008) [2023-12-27 18:13:44,721][20381] Updated weights for policy 1, policy_version 1814720 (0.0007) [2023-12-27 18:13:44,791][20381] Updated weights for policy 1, policy_version 1814732 (0.0009) [2023-12-27 18:13:44,854][20381] Updated weights for policy 1, policy_version 1814742 (0.0007) [2023-12-27 18:13:45,318][20347] Updated weights for policy 0, policy_version 1816383 (0.0009) [2023-12-27 18:13:45,364][20347] Updated weights for policy 0, policy_version 1816393 (0.0010) [2023-12-27 18:13:45,411][20347] Updated weights for policy 0, policy_version 1816403 (0.0010) [2023-12-27 18:13:45,566][20381] Updated weights for policy 1, policy_version 1814752 (0.0007) [2023-12-27 18:13:45,624][20381] Updated weights for policy 1, policy_version 1814762 (0.0005) [2023-12-27 18:13:45,689][20381] Updated weights for policy 1, policy_version 1814772 (0.0005) [2023-12-27 18:13:46,114][20347] Updated weights for policy 0, policy_version 1816413 (0.0008) [2023-12-27 18:13:46,157][20347] Updated weights for policy 0, policy_version 1816423 (0.0008) [2023-12-27 18:13:46,205][20347] Updated weights for policy 0, policy_version 1816433 (0.0007) [2023-12-27 18:13:46,233][20381] Updated weights for policy 1, policy_version 1814782 (0.0006) [2023-12-27 18:13:46,293][20381] Updated weights for policy 1, policy_version 1814792 (0.0007) [2023-12-27 18:13:46,353][20381] Updated weights for policy 1, policy_version 1814802 (0.0005) [2023-12-27 18:13:46,901][20347] Updated weights for policy 0, policy_version 1816443 (0.0006) [2023-12-27 18:13:46,941][20381] Updated weights for policy 1, policy_version 1814812 (0.0007) [2023-12-27 18:13:46,969][20347] Updated weights for policy 0, policy_version 1816453 (0.0007) [2023-12-27 18:13:46,990][20381] Updated weights for policy 1, policy_version 1814822 (0.0010) [2023-12-27 18:13:47,015][20347] Updated weights for policy 0, policy_version 1816463 (0.0005) [2023-12-27 18:13:47,047][20381] Updated weights for policy 1, policy_version 1814832 (0.0010) [2023-12-27 18:13:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 929751040. Throughput: 0: 10257.9, 1: 10394.1. Samples: 929758764. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:47,363][19170] Avg episode reward: [(0, '970.690'), (1, '951.300')] [2023-12-27 18:13:47,700][20381] Updated weights for policy 1, policy_version 1814842 (0.0010) [2023-12-27 18:13:47,720][20347] Updated weights for policy 0, policy_version 1816473 (0.0006) [2023-12-27 18:13:47,760][20381] Updated weights for policy 1, policy_version 1814852 (0.0010) [2023-12-27 18:13:47,773][20347] Updated weights for policy 0, policy_version 1816483 (0.0005) [2023-12-27 18:13:47,810][20381] Updated weights for policy 1, policy_version 1814862 (0.0011) [2023-12-27 18:13:47,824][20347] Updated weights for policy 0, policy_version 1816493 (0.0005) [2023-12-27 18:13:47,869][20381] Updated weights for policy 1, policy_version 1814872 (0.0008) [2023-12-27 18:13:47,882][20347] Updated weights for policy 0, policy_version 1816503 (0.0006) [2023-12-27 18:13:48,504][20381] Updated weights for policy 1, policy_version 1814882 (0.0005) [2023-12-27 18:13:48,566][20381] Updated weights for policy 1, policy_version 1814892 (0.0007) [2023-12-27 18:13:48,603][20347] Updated weights for policy 0, policy_version 1816513 (0.0007) [2023-12-27 18:13:48,622][20381] Updated weights for policy 1, policy_version 1814902 (0.0005) [2023-12-27 18:13:48,660][20347] Updated weights for policy 0, policy_version 1816523 (0.0007) [2023-12-27 18:13:48,718][20347] Updated weights for policy 0, policy_version 1816533 (0.0008) [2023-12-27 18:13:49,247][20381] Updated weights for policy 1, policy_version 1814912 (0.0008) [2023-12-27 18:13:49,307][20381] Updated weights for policy 1, policy_version 1814922 (0.0008) [2023-12-27 18:13:49,365][20381] Updated weights for policy 1, policy_version 1814932 (0.0008) [2023-12-27 18:13:49,444][20347] Updated weights for policy 0, policy_version 1816543 (0.0008) [2023-12-27 18:13:49,505][20347] Updated weights for policy 0, policy_version 1816553 (0.0007) [2023-12-27 18:13:49,564][20347] Updated weights for policy 0, policy_version 1816563 (0.0007) [2023-12-27 18:13:50,047][20381] Updated weights for policy 1, policy_version 1814942 (0.0008) [2023-12-27 18:13:50,090][20381] Updated weights for policy 1, policy_version 1814952 (0.0008) [2023-12-27 18:13:50,143][20381] Updated weights for policy 1, policy_version 1814962 (0.0008) [2023-12-27 18:13:50,261][20347] Updated weights for policy 0, policy_version 1816573 (0.0008) [2023-12-27 18:13:50,314][20347] Updated weights for policy 0, policy_version 1816583 (0.0008) [2023-12-27 18:13:50,361][20347] Updated weights for policy 0, policy_version 1816593 (0.0008) [2023-12-27 18:13:50,840][20381] Updated weights for policy 1, policy_version 1814972 (0.0008) [2023-12-27 18:13:50,890][20381] Updated weights for policy 1, policy_version 1814982 (0.0008) [2023-12-27 18:13:50,939][20381] Updated weights for policy 1, policy_version 1814992 (0.0008) [2023-12-27 18:13:51,103][20347] Updated weights for policy 0, policy_version 1816603 (0.0008) [2023-12-27 18:13:51,157][20347] Updated weights for policy 0, policy_version 1816613 (0.0010) [2023-12-27 18:13:51,212][20347] Updated weights for policy 0, policy_version 1816623 (0.0008) [2023-12-27 18:13:51,751][20381] Updated weights for policy 1, policy_version 1815002 (0.0007) [2023-12-27 18:13:51,814][20381] Updated weights for policy 1, policy_version 1815012 (0.0007) [2023-12-27 18:13:51,878][20381] Updated weights for policy 1, policy_version 1815022 (0.0006) [2023-12-27 18:13:51,940][20381] Updated weights for policy 1, policy_version 1815032 (0.0006) [2023-12-27 18:13:51,984][20347] Updated weights for policy 0, policy_version 1816633 (0.0008) [2023-12-27 18:13:52,039][20347] Updated weights for policy 0, policy_version 1816643 (0.0010) [2023-12-27 18:13:52,098][20347] Updated weights for policy 0, policy_version 1816653 (0.0010) [2023-12-27 18:13:52,158][20347] Updated weights for policy 0, policy_version 1816663 (0.0010) [2023-12-27 18:13:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 929849344. Throughput: 0: 10191.9, 1: 10355.7. Samples: 929819388. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:52,363][19170] Avg episode reward: [(0, '970.690'), (1, '970.700')] [2023-12-27 18:13:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001815032_464699392.pth... [2023-12-27 18:13:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001816664_465149952.pth... [2023-12-27 18:13:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001813816_464388096.pth [2023-12-27 18:13:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001815480_464846848.pth [2023-12-27 18:13:52,570][20381] Updated weights for policy 1, policy_version 1815042 (0.0011) [2023-12-27 18:13:52,630][20381] Updated weights for policy 1, policy_version 1815052 (0.0011) [2023-12-27 18:13:52,688][20381] Updated weights for policy 1, policy_version 1815062 (0.0010) [2023-12-27 18:13:52,823][20347] Updated weights for policy 0, policy_version 1816673 (0.0010) [2023-12-27 18:13:52,874][20347] Updated weights for policy 0, policy_version 1816683 (0.0010) [2023-12-27 18:13:52,922][20347] Updated weights for policy 0, policy_version 1816693 (0.0010) [2023-12-27 18:13:53,297][20381] Updated weights for policy 1, policy_version 1815072 (0.0007) [2023-12-27 18:13:53,351][20381] Updated weights for policy 1, policy_version 1815082 (0.0007) [2023-12-27 18:13:53,397][20381] Updated weights for policy 1, policy_version 1815092 (0.0011) [2023-12-27 18:13:53,500][20347] Updated weights for policy 0, policy_version 1816703 (0.0006) [2023-12-27 18:13:53,560][20347] Updated weights for policy 0, policy_version 1816713 (0.0005) [2023-12-27 18:13:53,620][20347] Updated weights for policy 0, policy_version 1816723 (0.0005) [2023-12-27 18:13:54,038][20381] Updated weights for policy 1, policy_version 1815102 (0.0008) [2023-12-27 18:13:54,097][20381] Updated weights for policy 1, policy_version 1815112 (0.0007) [2023-12-27 18:13:54,153][20381] Updated weights for policy 1, policy_version 1815122 (0.0008) [2023-12-27 18:13:54,204][20347] Updated weights for policy 0, policy_version 1816733 (0.0006) [2023-12-27 18:13:54,253][20347] Updated weights for policy 0, policy_version 1816743 (0.0008) [2023-12-27 18:13:54,319][20347] Updated weights for policy 0, policy_version 1816753 (0.0006) [2023-12-27 18:13:54,786][20381] Updated weights for policy 1, policy_version 1815132 (0.0006) [2023-12-27 18:13:54,844][20381] Updated weights for policy 1, policy_version 1815142 (0.0006) [2023-12-27 18:13:54,898][20381] Updated weights for policy 1, policy_version 1815152 (0.0007) [2023-12-27 18:13:55,026][20347] Updated weights for policy 0, policy_version 1816763 (0.0008) [2023-12-27 18:13:55,087][20347] Updated weights for policy 0, policy_version 1816773 (0.0008) [2023-12-27 18:13:55,134][20347] Updated weights for policy 0, policy_version 1816783 (0.0010) [2023-12-27 18:13:55,477][20381] Updated weights for policy 1, policy_version 1815162 (0.0007) [2023-12-27 18:13:55,529][20381] Updated weights for policy 1, policy_version 1815172 (0.0009) [2023-12-27 18:13:55,583][20381] Updated weights for policy 1, policy_version 1815182 (0.0010) [2023-12-27 18:13:55,635][20381] Updated weights for policy 1, policy_version 1815192 (0.0007) [2023-12-27 18:13:55,775][20347] Updated weights for policy 0, policy_version 1816793 (0.0010) [2023-12-27 18:13:55,832][20347] Updated weights for policy 0, policy_version 1816803 (0.0007) [2023-12-27 18:13:55,892][20347] Updated weights for policy 0, policy_version 1816813 (0.0010) [2023-12-27 18:13:55,950][20347] Updated weights for policy 0, policy_version 1816823 (0.0010) [2023-12-27 18:13:56,236][20381] Updated weights for policy 1, policy_version 1815202 (0.0011) [2023-12-27 18:13:56,281][20381] Updated weights for policy 1, policy_version 1815212 (0.0010) [2023-12-27 18:13:56,334][20381] Updated weights for policy 1, policy_version 1815222 (0.0006) [2023-12-27 18:13:56,568][20347] Updated weights for policy 0, policy_version 1816833 (0.0006) [2023-12-27 18:13:56,624][20347] Updated weights for policy 0, policy_version 1816843 (0.0005) [2023-12-27 18:13:56,679][20347] Updated weights for policy 0, policy_version 1816853 (0.0005) [2023-12-27 18:13:56,913][20381] Updated weights for policy 1, policy_version 1815232 (0.0007) [2023-12-27 18:13:56,966][20381] Updated weights for policy 1, policy_version 1815242 (0.0009) [2023-12-27 18:13:57,023][20381] Updated weights for policy 1, policy_version 1815252 (0.0009) [2023-12-27 18:13:57,200][20347] Updated weights for policy 0, policy_version 1816863 (0.0005) [2023-12-27 18:13:57,256][20347] Updated weights for policy 0, policy_version 1816873 (0.0006) [2023-12-27 18:13:57,310][20347] Updated weights for policy 0, policy_version 1816883 (0.0007) [2023-12-27 18:13:57,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 929964032. Throughput: 0: 10256.1, 1: 10398.7. Samples: 929946236. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:13:57,363][19170] Avg episode reward: [(0, '931.470'), (1, '990.240')] [2023-12-27 18:13:57,662][20381] Updated weights for policy 1, policy_version 1815262 (0.0006) [2023-12-27 18:13:57,708][20381] Updated weights for policy 1, policy_version 1815272 (0.0005) [2023-12-27 18:13:57,764][20381] Updated weights for policy 1, policy_version 1815282 (0.0005) [2023-12-27 18:13:57,975][20347] Updated weights for policy 0, policy_version 1816893 (0.0009) [2023-12-27 18:13:58,025][20347] Updated weights for policy 0, policy_version 1816903 (0.0007) [2023-12-27 18:13:58,073][20347] Updated weights for policy 0, policy_version 1816913 (0.0007) [2023-12-27 18:13:58,409][20381] Updated weights for policy 1, policy_version 1815292 (0.0009) [2023-12-27 18:13:58,472][20381] Updated weights for policy 1, policy_version 1815302 (0.0008) [2023-12-27 18:13:58,503][20292] KL-divergence is very high: 108.3926 [2023-12-27 18:13:58,510][20292] KL-divergence is very high: 114.3789 [2023-12-27 18:13:58,534][20381] Updated weights for policy 1, policy_version 1815312 (0.0008) [2023-12-27 18:13:58,552][20292] KL-divergence is very high: 110.9221 [2023-12-27 18:13:58,558][20292] KL-divergence is very high: 103.3602 [2023-12-27 18:13:58,816][20347] Updated weights for policy 0, policy_version 1816923 (0.0007) [2023-12-27 18:13:58,883][20347] Updated weights for policy 0, policy_version 1816933 (0.0007) [2023-12-27 18:13:58,939][20347] Updated weights for policy 0, policy_version 1816943 (0.0007) [2023-12-27 18:13:59,303][20381] Updated weights for policy 1, policy_version 1815322 (0.0008) [2023-12-27 18:13:59,366][20381] Updated weights for policy 1, policy_version 1815332 (0.0008) [2023-12-27 18:13:59,428][20381] Updated weights for policy 1, policy_version 1815342 (0.0007) [2023-12-27 18:13:59,485][20381] Updated weights for policy 1, policy_version 1815352 (0.0007) [2023-12-27 18:13:59,656][20347] Updated weights for policy 0, policy_version 1816953 (0.0006) [2023-12-27 18:13:59,721][20347] Updated weights for policy 0, policy_version 1816963 (0.0006) [2023-12-27 18:13:59,780][20347] Updated weights for policy 0, policy_version 1816973 (0.0010) [2023-12-27 18:13:59,845][20347] Updated weights for policy 0, policy_version 1816983 (0.0009) [2023-12-27 18:14:00,222][20381] Updated weights for policy 1, policy_version 1815362 (0.0006) [2023-12-27 18:14:00,287][20381] Updated weights for policy 1, policy_version 1815372 (0.0006) [2023-12-27 18:14:00,337][20381] Updated weights for policy 1, policy_version 1815382 (0.0007) [2023-12-27 18:14:00,496][20347] Updated weights for policy 0, policy_version 1816993 (0.0010) [2023-12-27 18:14:00,558][20347] Updated weights for policy 0, policy_version 1817003 (0.0010) [2023-12-27 18:14:00,619][20347] Updated weights for policy 0, policy_version 1817013 (0.0009) [2023-12-27 18:14:00,915][20381] Updated weights for policy 1, policy_version 1815392 (0.0008) [2023-12-27 18:14:00,969][20381] Updated weights for policy 1, policy_version 1815402 (0.0008) [2023-12-27 18:14:01,025][20381] Updated weights for policy 1, policy_version 1815412 (0.0007) [2023-12-27 18:14:01,300][20347] Updated weights for policy 0, policy_version 1817023 (0.0011) [2023-12-27 18:14:01,359][20347] Updated weights for policy 0, policy_version 1817033 (0.0010) [2023-12-27 18:14:01,425][20347] Updated weights for policy 0, policy_version 1817043 (0.0010) [2023-12-27 18:14:01,752][20381] Updated weights for policy 1, policy_version 1815422 (0.0012) [2023-12-27 18:14:01,803][20381] Updated weights for policy 1, policy_version 1815432 (0.0005) [2023-12-27 18:14:01,873][20381] Updated weights for policy 1, policy_version 1815442 (0.0007) [2023-12-27 18:14:02,147][20347] Updated weights for policy 0, policy_version 1817053 (0.0010) [2023-12-27 18:14:02,206][20347] Updated weights for policy 0, policy_version 1817063 (0.0010) [2023-12-27 18:14:02,255][20347] Updated weights for policy 0, policy_version 1817073 (0.0010) [2023-12-27 18:14:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 930062336. Throughput: 0: 10208.7, 1: 10393.8. Samples: 930069896. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:02,363][19170] Avg episode reward: [(0, '931.550'), (1, '951.300')] [2023-12-27 18:14:02,505][20381] Updated weights for policy 1, policy_version 1815452 (0.0007) [2023-12-27 18:14:02,559][20381] Updated weights for policy 1, policy_version 1815462 (0.0008) [2023-12-27 18:14:02,617][20381] Updated weights for policy 1, policy_version 1815472 (0.0008) [2023-12-27 18:14:02,924][20347] Updated weights for policy 0, policy_version 1817083 (0.0009) [2023-12-27 18:14:02,977][20347] Updated weights for policy 0, policy_version 1817093 (0.0005) [2023-12-27 18:14:03,033][20347] Updated weights for policy 0, policy_version 1817103 (0.0005) [2023-12-27 18:14:03,277][20381] Updated weights for policy 1, policy_version 1815482 (0.0008) [2023-12-27 18:14:03,330][20381] Updated weights for policy 1, policy_version 1815492 (0.0005) [2023-12-27 18:14:03,388][20381] Updated weights for policy 1, policy_version 1815502 (0.0005) [2023-12-27 18:14:03,445][20381] Updated weights for policy 1, policy_version 1815512 (0.0005) [2023-12-27 18:14:03,665][20347] Updated weights for policy 0, policy_version 1817113 (0.0007) [2023-12-27 18:14:03,720][20347] Updated weights for policy 0, policy_version 1817123 (0.0010) [2023-12-27 18:14:03,765][20347] Updated weights for policy 0, policy_version 1817133 (0.0010) [2023-12-27 18:14:03,810][20347] Updated weights for policy 0, policy_version 1817143 (0.0010) [2023-12-27 18:14:04,003][20381] Updated weights for policy 1, policy_version 1815522 (0.0011) [2023-12-27 18:14:04,068][20381] Updated weights for policy 1, policy_version 1815532 (0.0008) [2023-12-27 18:14:04,129][20381] Updated weights for policy 1, policy_version 1815542 (0.0011) [2023-12-27 18:14:04,564][20347] Updated weights for policy 0, policy_version 1817153 (0.0011) [2023-12-27 18:14:04,622][20347] Updated weights for policy 0, policy_version 1817163 (0.0011) [2023-12-27 18:14:04,682][20347] Updated weights for policy 0, policy_version 1817173 (0.0010) [2023-12-27 18:14:04,791][20381] Updated weights for policy 1, policy_version 1815552 (0.0011) [2023-12-27 18:14:04,846][20381] Updated weights for policy 1, policy_version 1815562 (0.0011) [2023-12-27 18:14:04,901][20381] Updated weights for policy 1, policy_version 1815572 (0.0010) [2023-12-27 18:14:05,329][20347] Updated weights for policy 0, policy_version 1817183 (0.0007) [2023-12-27 18:14:05,390][20347] Updated weights for policy 0, policy_version 1817193 (0.0005) [2023-12-27 18:14:05,445][20347] Updated weights for policy 0, policy_version 1817203 (0.0006) [2023-12-27 18:14:05,574][20381] Updated weights for policy 1, policy_version 1815582 (0.0008) [2023-12-27 18:14:05,631][20381] Updated weights for policy 1, policy_version 1815592 (0.0005) [2023-12-27 18:14:05,683][20381] Updated weights for policy 1, policy_version 1815602 (0.0005) [2023-12-27 18:14:06,147][20347] Updated weights for policy 0, policy_version 1817213 (0.0008) [2023-12-27 18:14:06,209][20347] Updated weights for policy 0, policy_version 1817223 (0.0006) [2023-12-27 18:14:06,256][20381] Updated weights for policy 1, policy_version 1815612 (0.0006) [2023-12-27 18:14:06,265][20347] Updated weights for policy 0, policy_version 1817233 (0.0006) [2023-12-27 18:14:06,313][20381] Updated weights for policy 1, policy_version 1815622 (0.0007) [2023-12-27 18:14:06,377][20381] Updated weights for policy 1, policy_version 1815632 (0.0007) [2023-12-27 18:14:06,871][20347] Updated weights for policy 0, policy_version 1817243 (0.0007) [2023-12-27 18:14:06,927][20347] Updated weights for policy 0, policy_version 1817253 (0.0010) [2023-12-27 18:14:06,992][20347] Updated weights for policy 0, policy_version 1817263 (0.0010) [2023-12-27 18:14:07,082][20381] Updated weights for policy 1, policy_version 1815642 (0.0008) [2023-12-27 18:14:07,145][20381] Updated weights for policy 1, policy_version 1815652 (0.0009) [2023-12-27 18:14:07,198][20381] Updated weights for policy 1, policy_version 1815662 (0.0008) [2023-12-27 18:14:07,252][20381] Updated weights for policy 1, policy_version 1815672 (0.0008) [2023-12-27 18:14:07,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.3, 300 sec: 20577.7). Total num frames: 930168832. Throughput: 0: 10216.4, 1: 10417.8. Samples: 930132560. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:07,363][19170] Avg episode reward: [(0, '912.150'), (1, '922.090')] [2023-12-27 18:14:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001815672_464863232.pth... [2023-12-27 18:14:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001817272_465305600.pth... [2023-12-27 18:14:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001816088_465002496.pth [2023-12-27 18:14:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001814456_464551936.pth [2023-12-27 18:14:07,587][20347] Updated weights for policy 0, policy_version 1817273 (0.0010) [2023-12-27 18:14:07,632][20347] Updated weights for policy 0, policy_version 1817283 (0.0010) [2023-12-27 18:14:07,677][20347] Updated weights for policy 0, policy_version 1817293 (0.0010) [2023-12-27 18:14:07,735][20347] Updated weights for policy 0, policy_version 1817303 (0.0010) [2023-12-27 18:14:07,888][20381] Updated weights for policy 1, policy_version 1815682 (0.0010) [2023-12-27 18:14:07,948][20381] Updated weights for policy 1, policy_version 1815692 (0.0010) [2023-12-27 18:14:08,002][20381] Updated weights for policy 1, policy_version 1815702 (0.0010) [2023-12-27 18:14:08,441][20347] Updated weights for policy 0, policy_version 1817313 (0.0010) [2023-12-27 18:14:08,502][20347] Updated weights for policy 0, policy_version 1817323 (0.0009) [2023-12-27 18:14:08,564][20347] Updated weights for policy 0, policy_version 1817333 (0.0006) [2023-12-27 18:14:08,666][20381] Updated weights for policy 1, policy_version 1815712 (0.0010) [2023-12-27 18:14:08,725][20381] Updated weights for policy 1, policy_version 1815722 (0.0008) [2023-12-27 18:14:08,787][20381] Updated weights for policy 1, policy_version 1815732 (0.0008) [2023-12-27 18:14:09,230][20347] Updated weights for policy 0, policy_version 1817343 (0.0010) [2023-12-27 18:14:09,284][20347] Updated weights for policy 0, policy_version 1817353 (0.0009) [2023-12-27 18:14:09,342][20347] Updated weights for policy 0, policy_version 1817363 (0.0008) [2023-12-27 18:14:09,535][20381] Updated weights for policy 1, policy_version 1815742 (0.0009) [2023-12-27 18:14:09,584][20381] Updated weights for policy 1, policy_version 1815752 (0.0009) [2023-12-27 18:14:09,643][20381] Updated weights for policy 1, policy_version 1815762 (0.0009) [2023-12-27 18:14:10,177][20347] Updated weights for policy 0, policy_version 1817373 (0.0008) [2023-12-27 18:14:10,244][20347] Updated weights for policy 0, policy_version 1817383 (0.0009) [2023-12-27 18:14:10,307][20347] Updated weights for policy 0, policy_version 1817393 (0.0008) [2023-12-27 18:14:10,329][20381] Updated weights for policy 1, policy_version 1815772 (0.0008) [2023-12-27 18:14:10,387][20381] Updated weights for policy 1, policy_version 1815782 (0.0007) [2023-12-27 18:14:10,447][20381] Updated weights for policy 1, policy_version 1815792 (0.0005) [2023-12-27 18:14:11,016][20381] Updated weights for policy 1, policy_version 1815802 (0.0006) [2023-12-27 18:14:11,055][20347] Updated weights for policy 0, policy_version 1817403 (0.0008) [2023-12-27 18:14:11,083][20381] Updated weights for policy 1, policy_version 1815812 (0.0007) [2023-12-27 18:14:11,115][20347] Updated weights for policy 0, policy_version 1817413 (0.0007) [2023-12-27 18:14:11,146][20381] Updated weights for policy 1, policy_version 1815822 (0.0007) [2023-12-27 18:14:11,182][20347] Updated weights for policy 0, policy_version 1817423 (0.0007) [2023-12-27 18:14:11,215][20381] Updated weights for policy 1, policy_version 1815832 (0.0008) [2023-12-27 18:14:11,906][20381] Updated weights for policy 1, policy_version 1815842 (0.0006) [2023-12-27 18:14:11,950][20347] Updated weights for policy 0, policy_version 1817433 (0.0008) [2023-12-27 18:14:11,958][20381] Updated weights for policy 1, policy_version 1815852 (0.0007) [2023-12-27 18:14:12,007][20347] Updated weights for policy 0, policy_version 1817443 (0.0006) [2023-12-27 18:14:12,012][20381] Updated weights for policy 1, policy_version 1815862 (0.0006) [2023-12-27 18:14:12,065][20347] Updated weights for policy 0, policy_version 1817453 (0.0007) [2023-12-27 18:14:12,122][20347] Updated weights for policy 0, policy_version 1817463 (0.0008) [2023-12-27 18:14:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 930267136. Throughput: 0: 10177.9, 1: 10413.7. Samples: 930256568. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:12,363][19170] Avg episode reward: [(0, '921.940'), (1, '912.320')] [2023-12-27 18:14:12,653][20381] Updated weights for policy 1, policy_version 1815872 (0.0007) [2023-12-27 18:14:12,713][20381] Updated weights for policy 1, policy_version 1815882 (0.0005) [2023-12-27 18:14:12,766][20381] Updated weights for policy 1, policy_version 1815892 (0.0007) [2023-12-27 18:14:12,862][20347] Updated weights for policy 0, policy_version 1817473 (0.0009) [2023-12-27 18:14:12,910][20347] Updated weights for policy 0, policy_version 1817483 (0.0008) [2023-12-27 18:14:12,963][20347] Updated weights for policy 0, policy_version 1817493 (0.0007) [2023-12-27 18:14:13,372][20381] Updated weights for policy 1, policy_version 1815902 (0.0007) [2023-12-27 18:14:13,425][20381] Updated weights for policy 1, policy_version 1815912 (0.0008) [2023-12-27 18:14:13,484][20381] Updated weights for policy 1, policy_version 1815922 (0.0007) [2023-12-27 18:14:13,611][20347] Updated weights for policy 0, policy_version 1817503 (0.0006) [2023-12-27 18:14:13,670][20347] Updated weights for policy 0, policy_version 1817513 (0.0005) [2023-12-27 18:14:13,736][20347] Updated weights for policy 0, policy_version 1817523 (0.0005) [2023-12-27 18:14:14,212][20381] Updated weights for policy 1, policy_version 1815932 (0.0008) [2023-12-27 18:14:14,268][20381] Updated weights for policy 1, policy_version 1815942 (0.0008) [2023-12-27 18:14:14,321][20381] Updated weights for policy 1, policy_version 1815952 (0.0008) [2023-12-27 18:14:14,339][20347] Updated weights for policy 0, policy_version 1817533 (0.0006) [2023-12-27 18:14:14,392][20347] Updated weights for policy 0, policy_version 1817543 (0.0006) [2023-12-27 18:14:14,447][20347] Updated weights for policy 0, policy_version 1817553 (0.0005) [2023-12-27 18:14:14,996][20381] Updated weights for policy 1, policy_version 1815962 (0.0007) [2023-12-27 18:14:15,058][20381] Updated weights for policy 1, policy_version 1815972 (0.0007) [2023-12-27 18:14:15,116][20381] Updated weights for policy 1, policy_version 1815982 (0.0008) [2023-12-27 18:14:15,151][20347] Updated weights for policy 0, policy_version 1817563 (0.0006) [2023-12-27 18:14:15,176][20381] Updated weights for policy 1, policy_version 1815992 (0.0007) [2023-12-27 18:14:15,215][20347] Updated weights for policy 0, policy_version 1817573 (0.0007) [2023-12-27 18:14:15,280][20347] Updated weights for policy 0, policy_version 1817583 (0.0007) [2023-12-27 18:14:15,865][20347] Updated weights for policy 0, policy_version 1817593 (0.0006) [2023-12-27 18:14:15,915][20381] Updated weights for policy 1, policy_version 1816002 (0.0006) [2023-12-27 18:14:15,926][20347] Updated weights for policy 0, policy_version 1817603 (0.0008) [2023-12-27 18:14:15,971][20381] Updated weights for policy 1, policy_version 1816012 (0.0006) [2023-12-27 18:14:15,992][20347] Updated weights for policy 0, policy_version 1817613 (0.0006) [2023-12-27 18:14:16,030][20381] Updated weights for policy 1, policy_version 1816022 (0.0006) [2023-12-27 18:14:16,056][20347] Updated weights for policy 0, policy_version 1817623 (0.0006) [2023-12-27 18:14:16,573][20381] Updated weights for policy 1, policy_version 1816032 (0.0005) [2023-12-27 18:14:16,643][20347] Updated weights for policy 0, policy_version 1817633 (0.0007) [2023-12-27 18:14:16,647][20381] Updated weights for policy 1, policy_version 1816042 (0.0005) [2023-12-27 18:14:16,690][20347] Updated weights for policy 0, policy_version 1817643 (0.0006) [2023-12-27 18:14:16,706][20381] Updated weights for policy 1, policy_version 1816052 (0.0007) [2023-12-27 18:14:16,735][20347] Updated weights for policy 0, policy_version 1817653 (0.0006) [2023-12-27 18:14:17,335][20381] Updated weights for policy 1, policy_version 1816062 (0.0008) [2023-12-27 18:14:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 930365440. Throughput: 0: 10164.6, 1: 10387.8. Samples: 930380508. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:17,363][19170] Avg episode reward: [(0, '960.880'), (1, '941.520')] [2023-12-27 18:14:17,392][20381] Updated weights for policy 1, policy_version 1816072 (0.0007) [2023-12-27 18:14:17,450][20381] Updated weights for policy 1, policy_version 1816082 (0.0007) [2023-12-27 18:14:17,451][20347] Updated weights for policy 0, policy_version 1817663 (0.0006) [2023-12-27 18:14:17,505][20347] Updated weights for policy 0, policy_version 1817673 (0.0006) [2023-12-27 18:14:17,557][20347] Updated weights for policy 0, policy_version 1817683 (0.0008) [2023-12-27 18:14:18,134][20381] Updated weights for policy 1, policy_version 1816092 (0.0007) [2023-12-27 18:14:18,199][20381] Updated weights for policy 1, policy_version 1816102 (0.0008) [2023-12-27 18:14:18,254][20381] Updated weights for policy 1, policy_version 1816112 (0.0007) [2023-12-27 18:14:18,275][20347] Updated weights for policy 0, policy_version 1817693 (0.0007) [2023-12-27 18:14:18,331][20347] Updated weights for policy 0, policy_version 1817703 (0.0006) [2023-12-27 18:14:18,397][20347] Updated weights for policy 0, policy_version 1817713 (0.0009) [2023-12-27 18:14:18,900][20381] Updated weights for policy 1, policy_version 1816122 (0.0006) [2023-12-27 18:14:18,962][20381] Updated weights for policy 1, policy_version 1816132 (0.0007) [2023-12-27 18:14:19,017][20381] Updated weights for policy 1, policy_version 1816142 (0.0008) [2023-12-27 18:14:19,079][20381] Updated weights for policy 1, policy_version 1816152 (0.0008) [2023-12-27 18:14:19,130][20347] Updated weights for policy 0, policy_version 1817723 (0.0007) [2023-12-27 18:14:19,190][20347] Updated weights for policy 0, policy_version 1817733 (0.0008) [2023-12-27 18:14:19,264][20347] Updated weights for policy 0, policy_version 1817743 (0.0009) [2023-12-27 18:14:19,835][20381] Updated weights for policy 1, policy_version 1816162 (0.0007) [2023-12-27 18:14:19,892][20381] Updated weights for policy 1, policy_version 1816172 (0.0008) [2023-12-27 18:14:19,960][20381] Updated weights for policy 1, policy_version 1816182 (0.0007) [2023-12-27 18:14:19,987][20347] Updated weights for policy 0, policy_version 1817753 (0.0007) [2023-12-27 18:14:20,048][20347] Updated weights for policy 0, policy_version 1817763 (0.0009) [2023-12-27 18:14:20,110][20347] Updated weights for policy 0, policy_version 1817773 (0.0008) [2023-12-27 18:14:20,169][20347] Updated weights for policy 0, policy_version 1817783 (0.0008) [2023-12-27 18:14:20,619][20381] Updated weights for policy 1, policy_version 1816192 (0.0007) [2023-12-27 18:14:20,684][20381] Updated weights for policy 1, policy_version 1816202 (0.0007) [2023-12-27 18:14:20,748][20381] Updated weights for policy 1, policy_version 1816212 (0.0007) [2023-12-27 18:14:20,899][20347] Updated weights for policy 0, policy_version 1817793 (0.0008) [2023-12-27 18:14:20,964][20347] Updated weights for policy 0, policy_version 1817803 (0.0006) [2023-12-27 18:14:21,026][20347] Updated weights for policy 0, policy_version 1817813 (0.0007) [2023-12-27 18:14:21,425][20381] Updated weights for policy 1, policy_version 1816222 (0.0006) [2023-12-27 18:14:21,480][20381] Updated weights for policy 1, policy_version 1816232 (0.0007) [2023-12-27 18:14:21,543][20381] Updated weights for policy 1, policy_version 1816242 (0.0006) [2023-12-27 18:14:21,715][20347] Updated weights for policy 0, policy_version 1817823 (0.0007) [2023-12-27 18:14:21,792][20347] Updated weights for policy 0, policy_version 1817833 (0.0006) [2023-12-27 18:14:21,851][20347] Updated weights for policy 0, policy_version 1817843 (0.0007) [2023-12-27 18:14:22,299][20381] Updated weights for policy 1, policy_version 1816252 (0.0007) [2023-12-27 18:14:22,360][20381] Updated weights for policy 1, policy_version 1816262 (0.0007) [2023-12-27 18:14:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 930463744. Throughput: 0: 10137.9, 1: 10376.4. Samples: 930439420. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:22,363][19170] Avg episode reward: [(0, '980.390'), (1, '960.990')] [2023-12-27 18:14:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001817848_465453056.pth... [2023-12-27 18:14:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001816664_465149952.pth [2023-12-27 18:14:22,429][20381] Updated weights for policy 1, policy_version 1816272 (0.0008) [2023-12-27 18:14:22,483][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001816280_465018880.pth... [2023-12-27 18:14:22,487][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001815032_464699392.pth [2023-12-27 18:14:22,523][20347] Updated weights for policy 0, policy_version 1817853 (0.0008) [2023-12-27 18:14:22,587][20347] Updated weights for policy 0, policy_version 1817863 (0.0008) [2023-12-27 18:14:22,644][20347] Updated weights for policy 0, policy_version 1817873 (0.0008) [2023-12-27 18:14:23,114][20381] Updated weights for policy 1, policy_version 1816282 (0.0007) [2023-12-27 18:14:23,166][20381] Updated weights for policy 1, policy_version 1816292 (0.0005) [2023-12-27 18:14:23,217][20381] Updated weights for policy 1, policy_version 1816302 (0.0005) [2023-12-27 18:14:23,273][20381] Updated weights for policy 1, policy_version 1816312 (0.0007) [2023-12-27 18:14:23,349][20347] Updated weights for policy 0, policy_version 1817883 (0.0007) [2023-12-27 18:14:23,402][20347] Updated weights for policy 0, policy_version 1817893 (0.0006) [2023-12-27 18:14:23,451][20347] Updated weights for policy 0, policy_version 1817903 (0.0005) [2023-12-27 18:14:23,856][20381] Updated weights for policy 1, policy_version 1816322 (0.0006) [2023-12-27 18:14:23,906][20381] Updated weights for policy 1, policy_version 1816332 (0.0007) [2023-12-27 18:14:23,959][20381] Updated weights for policy 1, policy_version 1816342 (0.0006) [2023-12-27 18:14:24,095][20347] Updated weights for policy 0, policy_version 1817913 (0.0005) [2023-12-27 18:14:24,151][20347] Updated weights for policy 0, policy_version 1817923 (0.0007) [2023-12-27 18:14:24,200][20347] Updated weights for policy 0, policy_version 1817933 (0.0007) [2023-12-27 18:14:24,246][20347] Updated weights for policy 0, policy_version 1817943 (0.0008) [2023-12-27 18:14:24,591][20381] Updated weights for policy 1, policy_version 1816352 (0.0007) [2023-12-27 18:14:24,640][20381] Updated weights for policy 1, policy_version 1816362 (0.0008) [2023-12-27 18:14:24,696][20381] Updated weights for policy 1, policy_version 1816372 (0.0007) [2023-12-27 18:14:24,883][20347] Updated weights for policy 0, policy_version 1817953 (0.0007) [2023-12-27 18:14:24,946][20347] Updated weights for policy 0, policy_version 1817963 (0.0008) [2023-12-27 18:14:24,999][20347] Updated weights for policy 0, policy_version 1817973 (0.0006) [2023-12-27 18:14:25,368][20381] Updated weights for policy 1, policy_version 1816382 (0.0007) [2023-12-27 18:14:25,421][20381] Updated weights for policy 1, policy_version 1816392 (0.0007) [2023-12-27 18:14:25,468][20381] Updated weights for policy 1, policy_version 1816402 (0.0007) [2023-12-27 18:14:25,576][20347] Updated weights for policy 0, policy_version 1817983 (0.0007) [2023-12-27 18:14:25,634][20347] Updated weights for policy 0, policy_version 1817993 (0.0008) [2023-12-27 18:14:25,689][20347] Updated weights for policy 0, policy_version 1818003 (0.0007) [2023-12-27 18:14:26,141][20381] Updated weights for policy 1, policy_version 1816412 (0.0007) [2023-12-27 18:14:26,200][20381] Updated weights for policy 1, policy_version 1816422 (0.0008) [2023-12-27 18:14:26,256][20381] Updated weights for policy 1, policy_version 1816432 (0.0006) [2023-12-27 18:14:26,400][20347] Updated weights for policy 0, policy_version 1818013 (0.0008) [2023-12-27 18:14:26,450][20347] Updated weights for policy 0, policy_version 1818023 (0.0007) [2023-12-27 18:14:26,498][20347] Updated weights for policy 0, policy_version 1818033 (0.0007) [2023-12-27 18:14:26,791][20381] Updated weights for policy 1, policy_version 1816442 (0.0006) [2023-12-27 18:14:26,842][20381] Updated weights for policy 1, policy_version 1816452 (0.0009) [2023-12-27 18:14:26,897][20381] Updated weights for policy 1, policy_version 1816462 (0.0005) [2023-12-27 18:14:26,956][20381] Updated weights for policy 1, policy_version 1816472 (0.0006) [2023-12-27 18:14:27,188][20347] Updated weights for policy 0, policy_version 1818043 (0.0009) [2023-12-27 18:14:27,238][20347] Updated weights for policy 0, policy_version 1818053 (0.0007) [2023-12-27 18:14:27,288][20347] Updated weights for policy 0, policy_version 1818063 (0.0005) [2023-12-27 18:14:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 930578432. Throughput: 0: 10123.2, 1: 10486.4. Samples: 930563924. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.710')] [2023-12-27 18:14:27,498][20381] Updated weights for policy 1, policy_version 1816482 (0.0005) [2023-12-27 18:14:27,546][20381] Updated weights for policy 1, policy_version 1816492 (0.0005) [2023-12-27 18:14:27,594][20381] Updated weights for policy 1, policy_version 1816502 (0.0007) [2023-12-27 18:14:27,823][20347] Updated weights for policy 0, policy_version 1818073 (0.0006) [2023-12-27 18:14:27,883][20347] Updated weights for policy 0, policy_version 1818083 (0.0005) [2023-12-27 18:14:27,937][20347] Updated weights for policy 0, policy_version 1818093 (0.0005) [2023-12-27 18:14:27,988][20347] Updated weights for policy 0, policy_version 1818103 (0.0007) [2023-12-27 18:14:28,176][20381] Updated weights for policy 1, policy_version 1816512 (0.0008) [2023-12-27 18:14:28,228][20381] Updated weights for policy 1, policy_version 1816522 (0.0009) [2023-12-27 18:14:28,281][20381] Updated weights for policy 1, policy_version 1816532 (0.0009) [2023-12-27 18:14:28,566][20347] Updated weights for policy 0, policy_version 1818113 (0.0006) [2023-12-27 18:14:28,631][20347] Updated weights for policy 0, policy_version 1818123 (0.0007) [2023-12-27 18:14:28,696][20347] Updated weights for policy 0, policy_version 1818133 (0.0007) [2023-12-27 18:14:28,976][20381] Updated weights for policy 1, policy_version 1816542 (0.0008) [2023-12-27 18:14:29,042][20381] Updated weights for policy 1, policy_version 1816552 (0.0007) [2023-12-27 18:14:29,102][20381] Updated weights for policy 1, policy_version 1816562 (0.0007) [2023-12-27 18:14:29,350][20347] Updated weights for policy 0, policy_version 1818143 (0.0007) [2023-12-27 18:14:29,411][20347] Updated weights for policy 0, policy_version 1818153 (0.0008) [2023-12-27 18:14:29,459][20347] Updated weights for policy 0, policy_version 1818163 (0.0008) [2023-12-27 18:14:29,797][20381] Updated weights for policy 1, policy_version 1816572 (0.0007) [2023-12-27 18:14:29,851][20381] Updated weights for policy 1, policy_version 1816582 (0.0008) [2023-12-27 18:14:29,906][20381] Updated weights for policy 1, policy_version 1816592 (0.0007) [2023-12-27 18:14:30,183][20347] Updated weights for policy 0, policy_version 1818173 (0.0007) [2023-12-27 18:14:30,240][20347] Updated weights for policy 0, policy_version 1818183 (0.0008) [2023-12-27 18:14:30,300][20347] Updated weights for policy 0, policy_version 1818193 (0.0008) [2023-12-27 18:14:30,580][20381] Updated weights for policy 1, policy_version 1816602 (0.0007) [2023-12-27 18:14:30,628][20381] Updated weights for policy 1, policy_version 1816612 (0.0005) [2023-12-27 18:14:30,681][20381] Updated weights for policy 1, policy_version 1816622 (0.0005) [2023-12-27 18:14:30,728][20381] Updated weights for policy 1, policy_version 1816632 (0.0005) [2023-12-27 18:14:30,997][20347] Updated weights for policy 0, policy_version 1818203 (0.0009) [2023-12-27 18:14:31,055][20347] Updated weights for policy 0, policy_version 1818213 (0.0010) [2023-12-27 18:14:31,107][20347] Updated weights for policy 0, policy_version 1818223 (0.0010) [2023-12-27 18:14:31,366][20381] Updated weights for policy 1, policy_version 1816642 (0.0008) [2023-12-27 18:14:31,419][20381] Updated weights for policy 1, policy_version 1816652 (0.0009) [2023-12-27 18:14:31,469][20381] Updated weights for policy 1, policy_version 1816662 (0.0007) [2023-12-27 18:14:31,813][20347] Updated weights for policy 0, policy_version 1818233 (0.0009) [2023-12-27 18:14:31,877][20347] Updated weights for policy 0, policy_version 1818243 (0.0007) [2023-12-27 18:14:31,940][20347] Updated weights for policy 0, policy_version 1818253 (0.0009) [2023-12-27 18:14:31,993][20347] Updated weights for policy 0, policy_version 1818263 (0.0011) [2023-12-27 18:14:32,165][20381] Updated weights for policy 1, policy_version 1816672 (0.0006) [2023-12-27 18:14:32,221][20381] Updated weights for policy 1, policy_version 1816682 (0.0007) [2023-12-27 18:14:32,281][20381] Updated weights for policy 1, policy_version 1816692 (0.0008) [2023-12-27 18:14:32,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 930684928. Throughput: 0: 10212.0, 1: 10532.9. Samples: 930692284. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '961.000')] [2023-12-27 18:14:32,645][20347] Updated weights for policy 0, policy_version 1818273 (0.0009) [2023-12-27 18:14:32,700][20347] Updated weights for policy 0, policy_version 1818283 (0.0010) [2023-12-27 18:14:32,765][20347] Updated weights for policy 0, policy_version 1818293 (0.0010) [2023-12-27 18:14:32,902][20381] Updated weights for policy 1, policy_version 1816702 (0.0006) [2023-12-27 18:14:32,950][20381] Updated weights for policy 1, policy_version 1816712 (0.0005) [2023-12-27 18:14:33,004][20381] Updated weights for policy 1, policy_version 1816722 (0.0005) [2023-12-27 18:14:33,363][20347] Updated weights for policy 0, policy_version 1818303 (0.0007) [2023-12-27 18:14:33,423][20347] Updated weights for policy 0, policy_version 1818313 (0.0009) [2023-12-27 18:14:33,477][20347] Updated weights for policy 0, policy_version 1818323 (0.0010) [2023-12-27 18:14:33,553][20381] Updated weights for policy 1, policy_version 1816732 (0.0005) [2023-12-27 18:14:33,609][20381] Updated weights for policy 1, policy_version 1816742 (0.0005) [2023-12-27 18:14:33,668][20381] Updated weights for policy 1, policy_version 1816752 (0.0006) [2023-12-27 18:14:34,122][20347] Updated weights for policy 0, policy_version 1818333 (0.0007) [2023-12-27 18:14:34,186][20347] Updated weights for policy 0, policy_version 1818343 (0.0006) [2023-12-27 18:14:34,201][20381] Updated weights for policy 1, policy_version 1816762 (0.0006) [2023-12-27 18:14:34,244][20347] Updated weights for policy 0, policy_version 1818353 (0.0006) [2023-12-27 18:14:34,261][20381] Updated weights for policy 1, policy_version 1816772 (0.0008) [2023-12-27 18:14:34,323][20381] Updated weights for policy 1, policy_version 1816782 (0.0007) [2023-12-27 18:14:34,386][20381] Updated weights for policy 1, policy_version 1816792 (0.0007) [2023-12-27 18:14:34,915][20347] Updated weights for policy 0, policy_version 1818363 (0.0006) [2023-12-27 18:14:34,967][20347] Updated weights for policy 0, policy_version 1818373 (0.0007) [2023-12-27 18:14:35,023][20347] Updated weights for policy 0, policy_version 1818383 (0.0006) [2023-12-27 18:14:35,045][20381] Updated weights for policy 1, policy_version 1816802 (0.0006) [2023-12-27 18:14:35,098][20381] Updated weights for policy 1, policy_version 1816812 (0.0007) [2023-12-27 18:14:35,153][20381] Updated weights for policy 1, policy_version 1816822 (0.0005) [2023-12-27 18:14:35,716][20347] Updated weights for policy 0, policy_version 1818393 (0.0006) [2023-12-27 18:14:35,747][20381] Updated weights for policy 1, policy_version 1816832 (0.0007) [2023-12-27 18:14:35,774][20347] Updated weights for policy 0, policy_version 1818403 (0.0010) [2023-12-27 18:14:35,794][20381] Updated weights for policy 1, policy_version 1816842 (0.0006) [2023-12-27 18:14:35,832][20347] Updated weights for policy 0, policy_version 1818413 (0.0010) [2023-12-27 18:14:35,846][20381] Updated weights for policy 1, policy_version 1816852 (0.0008) [2023-12-27 18:14:35,888][20347] Updated weights for policy 0, policy_version 1818423 (0.0010) [2023-12-27 18:14:36,492][20381] Updated weights for policy 1, policy_version 1816862 (0.0009) [2023-12-27 18:14:36,554][20381] Updated weights for policy 1, policy_version 1816872 (0.0008) [2023-12-27 18:14:36,559][20347] Updated weights for policy 0, policy_version 1818433 (0.0006) [2023-12-27 18:14:36,615][20381] Updated weights for policy 1, policy_version 1816882 (0.0007) [2023-12-27 18:14:36,618][20347] Updated weights for policy 0, policy_version 1818443 (0.0007) [2023-12-27 18:14:36,674][20347] Updated weights for policy 0, policy_version 1818453 (0.0011) [2023-12-27 18:14:37,265][20347] Updated weights for policy 0, policy_version 1818463 (0.0010) [2023-12-27 18:14:37,278][20381] Updated weights for policy 1, policy_version 1816892 (0.0006) [2023-12-27 18:14:37,311][20347] Updated weights for policy 0, policy_version 1818473 (0.0010) [2023-12-27 18:14:37,327][20381] Updated weights for policy 1, policy_version 1816902 (0.0005) [2023-12-27 18:14:37,363][20347] Updated weights for policy 0, policy_version 1818483 (0.0010) [2023-12-27 18:14:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 930783232. Throughput: 0: 10268.0, 1: 10581.1. Samples: 930757600. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.700')] [2023-12-27 18:14:37,380][20381] Updated weights for policy 1, policy_version 1816912 (0.0006) [2023-12-27 18:14:37,385][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001818488_465616896.pth... [2023-12-27 18:14:37,388][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001817272_465305600.pth [2023-12-27 18:14:37,423][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001816920_465182720.pth... [2023-12-27 18:14:37,426][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001815672_464863232.pth [2023-12-27 18:14:38,049][20347] Updated weights for policy 0, policy_version 1818493 (0.0010) [2023-12-27 18:14:38,098][20381] Updated weights for policy 1, policy_version 1816922 (0.0007) [2023-12-27 18:14:38,103][20347] Updated weights for policy 0, policy_version 1818503 (0.0010) [2023-12-27 18:14:38,153][20381] Updated weights for policy 1, policy_version 1816932 (0.0005) [2023-12-27 18:14:38,158][20347] Updated weights for policy 0, policy_version 1818513 (0.0011) [2023-12-27 18:14:38,211][20381] Updated weights for policy 1, policy_version 1816942 (0.0006) [2023-12-27 18:14:38,272][20381] Updated weights for policy 1, policy_version 1816952 (0.0007) [2023-12-27 18:14:38,864][20347] Updated weights for policy 0, policy_version 1818523 (0.0009) [2023-12-27 18:14:38,919][20347] Updated weights for policy 0, policy_version 1818533 (0.0007) [2023-12-27 18:14:38,966][20347] Updated weights for policy 0, policy_version 1818543 (0.0007) [2023-12-27 18:14:38,981][20381] Updated weights for policy 1, policy_version 1816962 (0.0005) [2023-12-27 18:14:39,034][20381] Updated weights for policy 1, policy_version 1816972 (0.0006) [2023-12-27 18:14:39,082][20381] Updated weights for policy 1, policy_version 1816982 (0.0010) [2023-12-27 18:14:39,734][20381] Updated weights for policy 1, policy_version 1816992 (0.0010) [2023-12-27 18:14:39,750][20347] Updated weights for policy 0, policy_version 1818553 (0.0008) [2023-12-27 18:14:39,788][20381] Updated weights for policy 1, policy_version 1817002 (0.0011) [2023-12-27 18:14:39,810][20347] Updated weights for policy 0, policy_version 1818563 (0.0007) [2023-12-27 18:14:39,850][20381] Updated weights for policy 1, policy_version 1817012 (0.0009) [2023-12-27 18:14:39,885][20347] Updated weights for policy 0, policy_version 1818573 (0.0009) [2023-12-27 18:14:39,949][20347] Updated weights for policy 0, policy_version 1818583 (0.0011) [2023-12-27 18:14:40,578][20381] Updated weights for policy 1, policy_version 1817022 (0.0009) [2023-12-27 18:14:40,631][20381] Updated weights for policy 1, policy_version 1817032 (0.0006) [2023-12-27 18:14:40,633][20347] Updated weights for policy 0, policy_version 1818593 (0.0007) [2023-12-27 18:14:40,685][20381] Updated weights for policy 1, policy_version 1817042 (0.0005) [2023-12-27 18:14:40,690][20347] Updated weights for policy 0, policy_version 1818603 (0.0007) [2023-12-27 18:14:40,748][20347] Updated weights for policy 0, policy_version 1818613 (0.0007) [2023-12-27 18:14:41,428][20347] Updated weights for policy 0, policy_version 1818623 (0.0008) [2023-12-27 18:14:41,438][20381] Updated weights for policy 1, policy_version 1817052 (0.0006) [2023-12-27 18:14:41,484][20347] Updated weights for policy 0, policy_version 1818633 (0.0008) [2023-12-27 18:14:41,498][20381] Updated weights for policy 1, policy_version 1817062 (0.0006) [2023-12-27 18:14:41,543][20347] Updated weights for policy 0, policy_version 1818643 (0.0006) [2023-12-27 18:14:41,560][20381] Updated weights for policy 1, policy_version 1817072 (0.0007) [2023-12-27 18:14:42,276][20381] Updated weights for policy 1, policy_version 1817082 (0.0008) [2023-12-27 18:14:42,301][20347] Updated weights for policy 0, policy_version 1818653 (0.0007) [2023-12-27 18:14:42,334][20381] Updated weights for policy 1, policy_version 1817092 (0.0006) [2023-12-27 18:14:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 930881536. Throughput: 0: 10231.5, 1: 10508.4. Samples: 930879528. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:42,364][20347] Updated weights for policy 0, policy_version 1818663 (0.0007) [2023-12-27 18:14:42,364][19170] Avg episode reward: [(0, '970.810'), (1, '922.040')] [2023-12-27 18:14:42,401][20381] Updated weights for policy 1, policy_version 1817102 (0.0007) [2023-12-27 18:14:42,426][20347] Updated weights for policy 0, policy_version 1818673 (0.0007) [2023-12-27 18:14:42,455][20381] Updated weights for policy 1, policy_version 1817112 (0.0005) [2023-12-27 18:14:43,096][20347] Updated weights for policy 0, policy_version 1818683 (0.0007) [2023-12-27 18:14:43,147][20347] Updated weights for policy 0, policy_version 1818693 (0.0006) [2023-12-27 18:14:43,171][20381] Updated weights for policy 1, policy_version 1817122 (0.0007) [2023-12-27 18:14:43,198][20347] Updated weights for policy 0, policy_version 1818703 (0.0006) [2023-12-27 18:14:43,229][20381] Updated weights for policy 1, policy_version 1817132 (0.0007) [2023-12-27 18:14:43,281][20381] Updated weights for policy 1, policy_version 1817142 (0.0008) [2023-12-27 18:14:43,848][20347] Updated weights for policy 0, policy_version 1818713 (0.0005) [2023-12-27 18:14:43,910][20347] Updated weights for policy 0, policy_version 1818723 (0.0006) [2023-12-27 18:14:43,963][20347] Updated weights for policy 0, policy_version 1818733 (0.0006) [2023-12-27 18:14:43,976][20381] Updated weights for policy 1, policy_version 1817152 (0.0006) [2023-12-27 18:14:44,018][20347] Updated weights for policy 0, policy_version 1818743 (0.0006) [2023-12-27 18:14:44,031][20381] Updated weights for policy 1, policy_version 1817162 (0.0005) [2023-12-27 18:14:44,086][20381] Updated weights for policy 1, policy_version 1817172 (0.0007) [2023-12-27 18:14:44,655][20381] Updated weights for policy 1, policy_version 1817182 (0.0006) [2023-12-27 18:14:44,682][20347] Updated weights for policy 0, policy_version 1818753 (0.0006) [2023-12-27 18:14:44,712][20381] Updated weights for policy 1, policy_version 1817192 (0.0007) [2023-12-27 18:14:44,738][20347] Updated weights for policy 0, policy_version 1818763 (0.0005) [2023-12-27 18:14:44,772][20381] Updated weights for policy 1, policy_version 1817202 (0.0007) [2023-12-27 18:14:44,800][20347] Updated weights for policy 0, policy_version 1818773 (0.0007) [2023-12-27 18:14:45,458][20347] Updated weights for policy 0, policy_version 1818783 (0.0006) [2023-12-27 18:14:45,482][20381] Updated weights for policy 1, policy_version 1817212 (0.0008) [2023-12-27 18:14:45,511][20347] Updated weights for policy 0, policy_version 1818793 (0.0005) [2023-12-27 18:14:45,534][20381] Updated weights for policy 1, policy_version 1817222 (0.0008) [2023-12-27 18:14:45,567][20347] Updated weights for policy 0, policy_version 1818803 (0.0006) [2023-12-27 18:14:45,587][20381] Updated weights for policy 1, policy_version 1817232 (0.0006) [2023-12-27 18:14:46,163][20347] Updated weights for policy 0, policy_version 1818813 (0.0007) [2023-12-27 18:14:46,209][20347] Updated weights for policy 0, policy_version 1818823 (0.0008) [2023-12-27 18:14:46,261][20381] Updated weights for policy 1, policy_version 1817242 (0.0007) [2023-12-27 18:14:46,263][20347] Updated weights for policy 0, policy_version 1818833 (0.0007) [2023-12-27 18:14:46,310][20381] Updated weights for policy 1, policy_version 1817252 (0.0006) [2023-12-27 18:14:46,362][20381] Updated weights for policy 1, policy_version 1817262 (0.0008) [2023-12-27 18:14:46,422][20381] Updated weights for policy 1, policy_version 1817272 (0.0008) [2023-12-27 18:14:46,996][20347] Updated weights for policy 0, policy_version 1818843 (0.0007) [2023-12-27 18:14:47,048][20347] Updated weights for policy 0, policy_version 1818853 (0.0007) [2023-12-27 18:14:47,108][20347] Updated weights for policy 0, policy_version 1818863 (0.0007) [2023-12-27 18:14:47,142][20381] Updated weights for policy 1, policy_version 1817282 (0.0006) [2023-12-27 18:14:47,197][20381] Updated weights for policy 1, policy_version 1817292 (0.0008) [2023-12-27 18:14:47,263][20381] Updated weights for policy 1, policy_version 1817302 (0.0008) [2023-12-27 18:14:47,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 930996224. Throughput: 0: 10228.3, 1: 10468.8. Samples: 931001264. Policy #0 lag: (min: 31.0, avg: 31.4, max: 47.0) [2023-12-27 18:14:47,363][19170] Avg episode reward: [(0, '941.480'), (1, '922.120')] [2023-12-27 18:14:47,724][20347] Updated weights for policy 0, policy_version 1818873 (0.0006) [2023-12-27 18:14:47,791][20347] Updated weights for policy 0, policy_version 1818883 (0.0008) [2023-12-27 18:14:47,855][20347] Updated weights for policy 0, policy_version 1818893 (0.0008) [2023-12-27 18:14:47,909][20347] Updated weights for policy 0, policy_version 1818903 (0.0005) [2023-12-27 18:14:47,917][20381] Updated weights for policy 1, policy_version 1817312 (0.0007) [2023-12-27 18:14:47,966][20381] Updated weights for policy 1, policy_version 1817322 (0.0009) [2023-12-27 18:14:48,019][20381] Updated weights for policy 1, policy_version 1817332 (0.0009) [2023-12-27 18:14:48,490][20347] Updated weights for policy 0, policy_version 1818913 (0.0008) [2023-12-27 18:14:48,546][20347] Updated weights for policy 0, policy_version 1818923 (0.0008) [2023-12-27 18:14:48,599][20347] Updated weights for policy 0, policy_version 1818933 (0.0008) [2023-12-27 18:14:48,694][20381] Updated weights for policy 1, policy_version 1817342 (0.0007) [2023-12-27 18:14:48,745][20381] Updated weights for policy 1, policy_version 1817352 (0.0007) [2023-12-27 18:14:48,801][20381] Updated weights for policy 1, policy_version 1817362 (0.0005) [2023-12-27 18:14:49,318][20347] Updated weights for policy 0, policy_version 1818943 (0.0007) [2023-12-27 18:14:49,385][20347] Updated weights for policy 0, policy_version 1818953 (0.0008) [2023-12-27 18:14:49,444][20347] Updated weights for policy 0, policy_version 1818963 (0.0007) [2023-12-27 18:14:49,483][20381] Updated weights for policy 1, policy_version 1817372 (0.0006) [2023-12-27 18:14:49,542][20381] Updated weights for policy 1, policy_version 1817382 (0.0008) [2023-12-27 18:14:49,603][20381] Updated weights for policy 1, policy_version 1817392 (0.0008) [2023-12-27 18:14:50,127][20347] Updated weights for policy 0, policy_version 1818973 (0.0006) [2023-12-27 18:14:50,177][20347] Updated weights for policy 0, policy_version 1818983 (0.0006) [2023-12-27 18:14:50,241][20347] Updated weights for policy 0, policy_version 1818993 (0.0007) [2023-12-27 18:14:50,336][20381] Updated weights for policy 1, policy_version 1817402 (0.0008) [2023-12-27 18:14:50,384][20381] Updated weights for policy 1, policy_version 1817412 (0.0008) [2023-12-27 18:14:50,440][20381] Updated weights for policy 1, policy_version 1817422 (0.0008) [2023-12-27 18:14:50,486][20381] Updated weights for policy 1, policy_version 1817432 (0.0007) [2023-12-27 18:14:50,877][20347] Updated weights for policy 0, policy_version 1819003 (0.0008) [2023-12-27 18:14:50,942][20347] Updated weights for policy 0, policy_version 1819013 (0.0011) [2023-12-27 18:14:51,007][20347] Updated weights for policy 0, policy_version 1819023 (0.0010) [2023-12-27 18:14:51,144][20381] Updated weights for policy 1, policy_version 1817442 (0.0007) [2023-12-27 18:14:51,203][20381] Updated weights for policy 1, policy_version 1817452 (0.0007) [2023-12-27 18:14:51,267][20381] Updated weights for policy 1, policy_version 1817462 (0.0007) [2023-12-27 18:14:51,685][20347] Updated weights for policy 0, policy_version 1819033 (0.0009) [2023-12-27 18:14:51,754][20347] Updated weights for policy 0, policy_version 1819043 (0.0007) [2023-12-27 18:14:51,804][20347] Updated weights for policy 0, policy_version 1819053 (0.0008) [2023-12-27 18:14:51,858][20347] Updated weights for policy 0, policy_version 1819063 (0.0007) [2023-12-27 18:14:51,999][20381] Updated weights for policy 1, policy_version 1817472 (0.0009) [2023-12-27 18:14:52,051][20381] Updated weights for policy 1, policy_version 1817482 (0.0010) [2023-12-27 18:14:52,116][20381] Updated weights for policy 1, policy_version 1817492 (0.0011) [2023-12-27 18:14:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 931094528. Throughput: 0: 10245.8, 1: 10432.2. Samples: 931063068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:14:52,363][19170] Avg episode reward: [(0, '941.410'), (1, '941.500')] [2023-12-27 18:14:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001817496_465330176.pth... [2023-12-27 18:14:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001819064_465764352.pth... [2023-12-27 18:14:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001816280_465018880.pth [2023-12-27 18:14:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001817848_465453056.pth [2023-12-27 18:14:52,550][20347] Updated weights for policy 0, policy_version 1819073 (0.0008) [2023-12-27 18:14:52,614][20347] Updated weights for policy 0, policy_version 1819083 (0.0008) [2023-12-27 18:14:52,673][20347] Updated weights for policy 0, policy_version 1819093 (0.0007) [2023-12-27 18:14:52,805][20381] Updated weights for policy 1, policy_version 1817502 (0.0011) [2023-12-27 18:14:52,858][20381] Updated weights for policy 1, policy_version 1817512 (0.0011) [2023-12-27 18:14:52,916][20381] Updated weights for policy 1, policy_version 1817522 (0.0010) [2023-12-27 18:14:53,283][20347] Updated weights for policy 0, policy_version 1819103 (0.0006) [2023-12-27 18:14:53,339][20347] Updated weights for policy 0, policy_version 1819113 (0.0007) [2023-12-27 18:14:53,397][20347] Updated weights for policy 0, policy_version 1819123 (0.0008) [2023-12-27 18:14:53,601][20381] Updated weights for policy 1, policy_version 1817532 (0.0010) [2023-12-27 18:14:53,656][20381] Updated weights for policy 1, policy_version 1817542 (0.0010) [2023-12-27 18:14:53,704][20381] Updated weights for policy 1, policy_version 1817552 (0.0010) [2023-12-27 18:14:54,063][20347] Updated weights for policy 0, policy_version 1819133 (0.0008) [2023-12-27 18:14:54,128][20347] Updated weights for policy 0, policy_version 1819143 (0.0008) [2023-12-27 18:14:54,138][20291] KL-divergence is very high: 108.7476 [2023-12-27 18:14:54,175][20291] KL-divergence is very high: 146.1696 [2023-12-27 18:14:54,188][20291] KL-divergence is very high: 135.5067 [2023-12-27 18:14:54,188][20347] Updated weights for policy 0, policy_version 1819153 (0.0007) [2023-12-27 18:14:54,222][20291] KL-divergence is very high: 119.6465 [2023-12-27 18:14:54,369][20381] Updated weights for policy 1, policy_version 1817562 (0.0009) [2023-12-27 18:14:54,419][20381] Updated weights for policy 1, policy_version 1817572 (0.0008) [2023-12-27 18:14:54,467][20381] Updated weights for policy 1, policy_version 1817582 (0.0008) [2023-12-27 18:14:54,510][20381] Updated weights for policy 1, policy_version 1817592 (0.0007) [2023-12-27 18:14:54,840][20291] KL-divergence is very high: 101.5734 [2023-12-27 18:14:54,845][20347] Updated weights for policy 0, policy_version 1819163 (0.0007) [2023-12-27 18:14:54,915][20347] Updated weights for policy 0, policy_version 1819173 (0.0008) [2023-12-27 18:14:54,981][20347] Updated weights for policy 0, policy_version 1819183 (0.0008) [2023-12-27 18:14:55,178][20381] Updated weights for policy 1, policy_version 1817602 (0.0011) [2023-12-27 18:14:55,227][20381] Updated weights for policy 1, policy_version 1817612 (0.0010) [2023-12-27 18:14:55,276][20381] Updated weights for policy 1, policy_version 1817622 (0.0008) [2023-12-27 18:14:55,639][20347] Updated weights for policy 0, policy_version 1819193 (0.0006) [2023-12-27 18:14:55,687][20347] Updated weights for policy 0, policy_version 1819203 (0.0007) [2023-12-27 18:14:55,737][20347] Updated weights for policy 0, policy_version 1819213 (0.0007) [2023-12-27 18:14:55,790][20347] Updated weights for policy 0, policy_version 1819223 (0.0007) [2023-12-27 18:14:55,879][20381] Updated weights for policy 1, policy_version 1817632 (0.0010) [2023-12-27 18:14:55,941][20381] Updated weights for policy 1, policy_version 1817642 (0.0011) [2023-12-27 18:14:55,989][20381] Updated weights for policy 1, policy_version 1817652 (0.0010) [2023-12-27 18:14:56,446][20347] Updated weights for policy 0, policy_version 1819233 (0.0008) [2023-12-27 18:14:56,504][20347] Updated weights for policy 0, policy_version 1819243 (0.0007) [2023-12-27 18:14:56,561][20347] Updated weights for policy 0, policy_version 1819253 (0.0007) [2023-12-27 18:14:56,682][20381] Updated weights for policy 1, policy_version 1817662 (0.0010) [2023-12-27 18:14:56,746][20381] Updated weights for policy 1, policy_version 1817672 (0.0010) [2023-12-27 18:14:56,794][20381] Updated weights for policy 1, policy_version 1817683 (0.0010) [2023-12-27 18:14:57,134][20347] Updated weights for policy 0, policy_version 1819263 (0.0006) [2023-12-27 18:14:57,182][20347] Updated weights for policy 0, policy_version 1819273 (0.0005) [2023-12-27 18:14:57,234][20347] Updated weights for policy 0, policy_version 1819283 (0.0005) [2023-12-27 18:14:57,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 931201024. Throughput: 0: 10279.3, 1: 10419.8. Samples: 931188028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:14:57,363][19170] Avg episode reward: [(0, '902.210'), (1, '951.190')] [2023-12-27 18:14:57,483][20381] Updated weights for policy 1, policy_version 1817693 (0.0011) [2023-12-27 18:14:57,528][20381] Updated weights for policy 1, policy_version 1817703 (0.0010) [2023-12-27 18:14:57,574][20381] Updated weights for policy 1, policy_version 1817713 (0.0011) [2023-12-27 18:14:57,828][20347] Updated weights for policy 0, policy_version 1819293 (0.0006) [2023-12-27 18:14:57,880][20347] Updated weights for policy 0, policy_version 1819303 (0.0007) [2023-12-27 18:14:57,930][20347] Updated weights for policy 0, policy_version 1819313 (0.0007) [2023-12-27 18:14:58,283][20381] Updated weights for policy 1, policy_version 1817723 (0.0009) [2023-12-27 18:14:58,348][20381] Updated weights for policy 1, policy_version 1817733 (0.0007) [2023-12-27 18:14:58,411][20381] Updated weights for policy 1, policy_version 1817743 (0.0008) [2023-12-27 18:14:58,687][20347] Updated weights for policy 0, policy_version 1819323 (0.0007) [2023-12-27 18:14:58,759][20347] Updated weights for policy 0, policy_version 1819333 (0.0007) [2023-12-27 18:14:58,828][20347] Updated weights for policy 0, policy_version 1819343 (0.0008) [2023-12-27 18:14:59,205][20381] Updated weights for policy 1, policy_version 1817753 (0.0009) [2023-12-27 18:14:59,274][20381] Updated weights for policy 1, policy_version 1817763 (0.0008) [2023-12-27 18:14:59,337][20381] Updated weights for policy 1, policy_version 1817773 (0.0006) [2023-12-27 18:14:59,396][20381] Updated weights for policy 1, policy_version 1817783 (0.0007) [2023-12-27 18:14:59,571][20347] Updated weights for policy 0, policy_version 1819353 (0.0007) [2023-12-27 18:14:59,633][20347] Updated weights for policy 0, policy_version 1819363 (0.0007) [2023-12-27 18:14:59,688][20347] Updated weights for policy 0, policy_version 1819373 (0.0007) [2023-12-27 18:14:59,743][20347] Updated weights for policy 0, policy_version 1819383 (0.0008) [2023-12-27 18:15:00,098][20381] Updated weights for policy 1, policy_version 1817793 (0.0009) [2023-12-27 18:15:00,153][20381] Updated weights for policy 1, policy_version 1817803 (0.0008) [2023-12-27 18:15:00,201][20381] Updated weights for policy 1, policy_version 1817813 (0.0008) [2023-12-27 18:15:00,437][20347] Updated weights for policy 0, policy_version 1819393 (0.0007) [2023-12-27 18:15:00,485][20347] Updated weights for policy 0, policy_version 1819403 (0.0007) [2023-12-27 18:15:00,535][20347] Updated weights for policy 0, policy_version 1819413 (0.0007) [2023-12-27 18:15:00,837][20381] Updated weights for policy 1, policy_version 1817823 (0.0006) [2023-12-27 18:15:00,880][20381] Updated weights for policy 1, policy_version 1817833 (0.0005) [2023-12-27 18:15:00,930][20381] Updated weights for policy 1, policy_version 1817843 (0.0005) [2023-12-27 18:15:01,264][20347] Updated weights for policy 0, policy_version 1819423 (0.0007) [2023-12-27 18:15:01,325][20347] Updated weights for policy 0, policy_version 1819433 (0.0007) [2023-12-27 18:15:01,388][20347] Updated weights for policy 0, policy_version 1819443 (0.0008) [2023-12-27 18:15:01,571][20381] Updated weights for policy 1, policy_version 1817853 (0.0005) [2023-12-27 18:15:01,630][20381] Updated weights for policy 1, policy_version 1817863 (0.0007) [2023-12-27 18:15:01,696][20381] Updated weights for policy 1, policy_version 1817873 (0.0007) [2023-12-27 18:15:02,089][20347] Updated weights for policy 0, policy_version 1819453 (0.0008) [2023-12-27 18:15:02,138][20347] Updated weights for policy 0, policy_version 1819463 (0.0008) [2023-12-27 18:15:02,194][20347] Updated weights for policy 0, policy_version 1819473 (0.0008) [2023-12-27 18:15:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 931299328. Throughput: 0: 10276.2, 1: 10373.0. Samples: 931309724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:02,363][19170] Avg episode reward: [(0, '912.010'), (1, '990.220')] [2023-12-27 18:15:02,377][20381] Updated weights for policy 1, policy_version 1817883 (0.0008) [2023-12-27 18:15:02,436][20381] Updated weights for policy 1, policy_version 1817893 (0.0008) [2023-12-27 18:15:02,495][20381] Updated weights for policy 1, policy_version 1817903 (0.0008) [2023-12-27 18:15:02,914][20347] Updated weights for policy 0, policy_version 1819483 (0.0007) [2023-12-27 18:15:02,966][20347] Updated weights for policy 0, policy_version 1819493 (0.0008) [2023-12-27 18:15:03,022][20347] Updated weights for policy 0, policy_version 1819503 (0.0009) [2023-12-27 18:15:03,179][20381] Updated weights for policy 1, policy_version 1817913 (0.0007) [2023-12-27 18:15:03,242][20381] Updated weights for policy 1, policy_version 1817923 (0.0007) [2023-12-27 18:15:03,301][20381] Updated weights for policy 1, policy_version 1817933 (0.0007) [2023-12-27 18:15:03,358][20381] Updated weights for policy 1, policy_version 1817943 (0.0008) [2023-12-27 18:15:03,675][20347] Updated weights for policy 0, policy_version 1819513 (0.0007) [2023-12-27 18:15:03,728][20347] Updated weights for policy 0, policy_version 1819523 (0.0005) [2023-12-27 18:15:03,773][20347] Updated weights for policy 0, policy_version 1819533 (0.0005) [2023-12-27 18:15:03,829][20347] Updated weights for policy 0, policy_version 1819543 (0.0005) [2023-12-27 18:15:04,078][20381] Updated weights for policy 1, policy_version 1817953 (0.0010) [2023-12-27 18:15:04,138][20381] Updated weights for policy 1, policy_version 1817963 (0.0007) [2023-12-27 18:15:04,196][20381] Updated weights for policy 1, policy_version 1817973 (0.0007) [2023-12-27 18:15:04,443][20347] Updated weights for policy 0, policy_version 1819553 (0.0009) [2023-12-27 18:15:04,499][20347] Updated weights for policy 0, policy_version 1819563 (0.0008) [2023-12-27 18:15:04,554][20347] Updated weights for policy 0, policy_version 1819573 (0.0008) [2023-12-27 18:15:04,841][20381] Updated weights for policy 1, policy_version 1817983 (0.0006) [2023-12-27 18:15:04,898][20381] Updated weights for policy 1, policy_version 1817993 (0.0006) [2023-12-27 18:15:04,960][20381] Updated weights for policy 1, policy_version 1818003 (0.0007) [2023-12-27 18:15:05,235][20347] Updated weights for policy 0, policy_version 1819583 (0.0010) [2023-12-27 18:15:05,291][20347] Updated weights for policy 0, policy_version 1819593 (0.0011) [2023-12-27 18:15:05,342][20347] Updated weights for policy 0, policy_version 1819603 (0.0010) [2023-12-27 18:15:05,619][20381] Updated weights for policy 1, policy_version 1818013 (0.0006) [2023-12-27 18:15:05,669][20381] Updated weights for policy 1, policy_version 1818023 (0.0006) [2023-12-27 18:15:05,716][20381] Updated weights for policy 1, policy_version 1818033 (0.0007) [2023-12-27 18:15:05,987][20347] Updated weights for policy 0, policy_version 1819613 (0.0008) [2023-12-27 18:15:06,044][20347] Updated weights for policy 0, policy_version 1819623 (0.0005) [2023-12-27 18:15:06,113][20347] Updated weights for policy 0, policy_version 1819633 (0.0006) [2023-12-27 18:15:06,379][20381] Updated weights for policy 1, policy_version 1818043 (0.0008) [2023-12-27 18:15:06,435][20381] Updated weights for policy 1, policy_version 1818053 (0.0011) [2023-12-27 18:15:06,493][20381] Updated weights for policy 1, policy_version 1818063 (0.0011) [2023-12-27 18:15:06,770][20347] Updated weights for policy 0, policy_version 1819643 (0.0008) [2023-12-27 18:15:06,822][20347] Updated weights for policy 0, policy_version 1819653 (0.0010) [2023-12-27 18:15:06,875][20347] Updated weights for policy 0, policy_version 1819663 (0.0010) [2023-12-27 18:15:07,156][20381] Updated weights for policy 1, policy_version 1818073 (0.0010) [2023-12-27 18:15:07,209][20381] Updated weights for policy 1, policy_version 1818083 (0.0009) [2023-12-27 18:15:07,262][20381] Updated weights for policy 1, policy_version 1818093 (0.0011) [2023-12-27 18:15:07,314][20381] Updated weights for policy 1, policy_version 1818103 (0.0010) [2023-12-27 18:15:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 931405824. Throughput: 0: 10325.8, 1: 10385.6. Samples: 931371436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:07,364][19170] Avg episode reward: [(0, '921.930'), (1, '970.760')] [2023-12-27 18:15:07,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001818104_465485824.pth... [2023-12-27 18:15:07,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001819672_465920000.pth... [2023-12-27 18:15:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001816920_465182720.pth [2023-12-27 18:15:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001818488_465616896.pth [2023-12-27 18:15:07,480][20347] Updated weights for policy 0, policy_version 1819673 (0.0010) [2023-12-27 18:15:07,528][20347] Updated weights for policy 0, policy_version 1819683 (0.0006) [2023-12-27 18:15:07,583][20347] Updated weights for policy 0, policy_version 1819693 (0.0006) [2023-12-27 18:15:07,641][20347] Updated weights for policy 0, policy_version 1819703 (0.0005) [2023-12-27 18:15:08,000][20381] Updated weights for policy 1, policy_version 1818113 (0.0008) [2023-12-27 18:15:08,058][20381] Updated weights for policy 1, policy_version 1818123 (0.0008) [2023-12-27 18:15:08,103][20381] Updated weights for policy 1, policy_version 1818133 (0.0008) [2023-12-27 18:15:08,302][20347] Updated weights for policy 0, policy_version 1819713 (0.0010) [2023-12-27 18:15:08,366][20347] Updated weights for policy 0, policy_version 1819723 (0.0008) [2023-12-27 18:15:08,427][20347] Updated weights for policy 0, policy_version 1819733 (0.0008) [2023-12-27 18:15:08,817][20381] Updated weights for policy 1, policy_version 1818143 (0.0006) [2023-12-27 18:15:08,866][20381] Updated weights for policy 1, policy_version 1818153 (0.0005) [2023-12-27 18:15:08,917][20381] Updated weights for policy 1, policy_version 1818163 (0.0005) [2023-12-27 18:15:09,145][20347] Updated weights for policy 0, policy_version 1819743 (0.0007) [2023-12-27 18:15:09,209][20347] Updated weights for policy 0, policy_version 1819753 (0.0007) [2023-12-27 18:15:09,273][20347] Updated weights for policy 0, policy_version 1819763 (0.0008) [2023-12-27 18:15:09,539][20381] Updated weights for policy 1, policy_version 1818173 (0.0006) [2023-12-27 18:15:09,602][20381] Updated weights for policy 1, policy_version 1818183 (0.0008) [2023-12-27 18:15:09,665][20381] Updated weights for policy 1, policy_version 1818193 (0.0008) [2023-12-27 18:15:10,031][20347] Updated weights for policy 0, policy_version 1819773 (0.0009) [2023-12-27 18:15:10,094][20347] Updated weights for policy 0, policy_version 1819783 (0.0008) [2023-12-27 18:15:10,154][20347] Updated weights for policy 0, policy_version 1819793 (0.0008) [2023-12-27 18:15:10,332][20381] Updated weights for policy 1, policy_version 1818203 (0.0008) [2023-12-27 18:15:10,394][20381] Updated weights for policy 1, policy_version 1818213 (0.0008) [2023-12-27 18:15:10,450][20381] Updated weights for policy 1, policy_version 1818223 (0.0007) [2023-12-27 18:15:10,858][20347] Updated weights for policy 0, policy_version 1819803 (0.0008) [2023-12-27 18:15:10,917][20347] Updated weights for policy 0, policy_version 1819813 (0.0008) [2023-12-27 18:15:10,968][20347] Updated weights for policy 0, policy_version 1819823 (0.0007) [2023-12-27 18:15:11,176][20381] Updated weights for policy 1, policy_version 1818233 (0.0008) [2023-12-27 18:15:11,235][20381] Updated weights for policy 1, policy_version 1818243 (0.0008) [2023-12-27 18:15:11,297][20381] Updated weights for policy 1, policy_version 1818253 (0.0008) [2023-12-27 18:15:11,357][20381] Updated weights for policy 1, policy_version 1818263 (0.0007) [2023-12-27 18:15:11,694][20347] Updated weights for policy 0, policy_version 1819833 (0.0006) [2023-12-27 18:15:11,769][20347] Updated weights for policy 0, policy_version 1819843 (0.0008) [2023-12-27 18:15:11,829][20347] Updated weights for policy 0, policy_version 1819853 (0.0008) [2023-12-27 18:15:11,893][20347] Updated weights for policy 0, policy_version 1819863 (0.0009) [2023-12-27 18:15:12,001][20381] Updated weights for policy 1, policy_version 1818273 (0.0007) [2023-12-27 18:15:12,051][20381] Updated weights for policy 1, policy_version 1818283 (0.0008) [2023-12-27 18:15:12,105][20381] Updated weights for policy 1, policy_version 1818294 (0.0008) [2023-12-27 18:15:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 931504128. Throughput: 0: 10298.8, 1: 10366.8. Samples: 931493880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:12,363][19170] Avg episode reward: [(0, '941.430'), (1, '961.000')] [2023-12-27 18:15:12,629][20347] Updated weights for policy 0, policy_version 1819873 (0.0008) [2023-12-27 18:15:12,692][20347] Updated weights for policy 0, policy_version 1819883 (0.0008) [2023-12-27 18:15:12,759][20347] Updated weights for policy 0, policy_version 1819893 (0.0008) [2023-12-27 18:15:12,819][20381] Updated weights for policy 1, policy_version 1818304 (0.0006) [2023-12-27 18:15:12,874][20381] Updated weights for policy 1, policy_version 1818314 (0.0008) [2023-12-27 18:15:12,932][20381] Updated weights for policy 1, policy_version 1818324 (0.0007) [2023-12-27 18:15:13,396][20347] Updated weights for policy 0, policy_version 1819903 (0.0008) [2023-12-27 18:15:13,446][20347] Updated weights for policy 0, policy_version 1819913 (0.0007) [2023-12-27 18:15:13,509][20347] Updated weights for policy 0, policy_version 1819923 (0.0007) [2023-12-27 18:15:13,595][20381] Updated weights for policy 1, policy_version 1818334 (0.0007) [2023-12-27 18:15:13,646][20381] Updated weights for policy 1, policy_version 1818344 (0.0008) [2023-12-27 18:15:13,700][20381] Updated weights for policy 1, policy_version 1818354 (0.0008) [2023-12-27 18:15:14,210][20347] Updated weights for policy 0, policy_version 1819933 (0.0007) [2023-12-27 18:15:14,268][20347] Updated weights for policy 0, policy_version 1819943 (0.0007) [2023-12-27 18:15:14,317][20347] Updated weights for policy 0, policy_version 1819953 (0.0006) [2023-12-27 18:15:14,402][20381] Updated weights for policy 1, policy_version 1818364 (0.0007) [2023-12-27 18:15:14,450][20381] Updated weights for policy 1, policy_version 1818374 (0.0005) [2023-12-27 18:15:14,508][20381] Updated weights for policy 1, policy_version 1818384 (0.0006) [2023-12-27 18:15:14,997][20347] Updated weights for policy 0, policy_version 1819963 (0.0007) [2023-12-27 18:15:15,054][20347] Updated weights for policy 0, policy_version 1819973 (0.0008) [2023-12-27 18:15:15,109][20347] Updated weights for policy 0, policy_version 1819983 (0.0006) [2023-12-27 18:15:15,194][20381] Updated weights for policy 1, policy_version 1818394 (0.0006) [2023-12-27 18:15:15,257][20381] Updated weights for policy 1, policy_version 1818404 (0.0006) [2023-12-27 18:15:15,311][20381] Updated weights for policy 1, policy_version 1818414 (0.0005) [2023-12-27 18:15:15,358][20381] Updated weights for policy 1, policy_version 1818424 (0.0005) [2023-12-27 18:15:15,717][20347] Updated weights for policy 0, policy_version 1819993 (0.0006) [2023-12-27 18:15:15,788][20347] Updated weights for policy 0, policy_version 1820003 (0.0008) [2023-12-27 18:15:15,852][20347] Updated weights for policy 0, policy_version 1820013 (0.0007) [2023-12-27 18:15:15,906][20347] Updated weights for policy 0, policy_version 1820023 (0.0006) [2023-12-27 18:15:15,934][20381] Updated weights for policy 1, policy_version 1818434 (0.0007) [2023-12-27 18:15:15,986][20381] Updated weights for policy 1, policy_version 1818444 (0.0007) [2023-12-27 18:15:16,044][20381] Updated weights for policy 1, policy_version 1818454 (0.0007) [2023-12-27 18:15:16,572][20347] Updated weights for policy 0, policy_version 1820033 (0.0005) [2023-12-27 18:15:16,635][20347] Updated weights for policy 0, policy_version 1820043 (0.0005) [2023-12-27 18:15:16,677][20381] Updated weights for policy 1, policy_version 1818464 (0.0005) [2023-12-27 18:15:16,694][20347] Updated weights for policy 0, policy_version 1820053 (0.0005) [2023-12-27 18:15:16,729][20381] Updated weights for policy 1, policy_version 1818474 (0.0008) [2023-12-27 18:15:16,788][20381] Updated weights for policy 1, policy_version 1818484 (0.0007) [2023-12-27 18:15:17,326][20347] Updated weights for policy 0, policy_version 1820063 (0.0007) [2023-12-27 18:15:17,363][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 931602432. Throughput: 0: 10258.3, 1: 10298.8. Samples: 931617356. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:17,364][19170] Avg episode reward: [(0, '970.620'), (1, '970.760')] [2023-12-27 18:15:17,377][20347] Updated weights for policy 0, policy_version 1820073 (0.0005) [2023-12-27 18:15:17,430][20347] Updated weights for policy 0, policy_version 1820083 (0.0005) [2023-12-27 18:15:17,471][20381] Updated weights for policy 1, policy_version 1818494 (0.0010) [2023-12-27 18:15:17,533][20381] Updated weights for policy 1, policy_version 1818504 (0.0010) [2023-12-27 18:15:17,585][20381] Updated weights for policy 1, policy_version 1818514 (0.0007) [2023-12-27 18:15:18,059][20347] Updated weights for policy 0, policy_version 1820093 (0.0009) [2023-12-27 18:15:18,107][20347] Updated weights for policy 0, policy_version 1820103 (0.0010) [2023-12-27 18:15:18,157][20347] Updated weights for policy 0, policy_version 1820113 (0.0010) [2023-12-27 18:15:18,291][20381] Updated weights for policy 1, policy_version 1818524 (0.0006) [2023-12-27 18:15:18,356][20381] Updated weights for policy 1, policy_version 1818534 (0.0007) [2023-12-27 18:15:18,420][20381] Updated weights for policy 1, policy_version 1818544 (0.0008) [2023-12-27 18:15:18,798][20347] Updated weights for policy 0, policy_version 1820123 (0.0010) [2023-12-27 18:15:18,846][20347] Updated weights for policy 0, policy_version 1820133 (0.0010) [2023-12-27 18:15:18,898][20347] Updated weights for policy 0, policy_version 1820143 (0.0010) [2023-12-27 18:15:19,142][20381] Updated weights for policy 1, policy_version 1818554 (0.0008) [2023-12-27 18:15:19,203][20381] Updated weights for policy 1, policy_version 1818564 (0.0007) [2023-12-27 18:15:19,267][20381] Updated weights for policy 1, policy_version 1818574 (0.0007) [2023-12-27 18:15:19,326][20381] Updated weights for policy 1, policy_version 1818584 (0.0007) [2023-12-27 18:15:19,649][20347] Updated weights for policy 0, policy_version 1820153 (0.0010) [2023-12-27 18:15:19,704][20347] Updated weights for policy 0, policy_version 1820163 (0.0008) [2023-12-27 18:15:19,768][20347] Updated weights for policy 0, policy_version 1820173 (0.0009) [2023-12-27 18:15:19,833][20347] Updated weights for policy 0, policy_version 1820183 (0.0008) [2023-12-27 18:15:20,050][20381] Updated weights for policy 1, policy_version 1818594 (0.0008) [2023-12-27 18:15:20,111][20381] Updated weights for policy 1, policy_version 1818604 (0.0006) [2023-12-27 18:15:20,165][20381] Updated weights for policy 1, policy_version 1818614 (0.0005) [2023-12-27 18:15:20,552][20347] Updated weights for policy 0, policy_version 1820193 (0.0009) [2023-12-27 18:15:20,617][20347] Updated weights for policy 0, policy_version 1820203 (0.0007) [2023-12-27 18:15:20,677][20347] Updated weights for policy 0, policy_version 1820213 (0.0008) [2023-12-27 18:15:20,864][20381] Updated weights for policy 1, policy_version 1818624 (0.0008) [2023-12-27 18:15:20,921][20381] Updated weights for policy 1, policy_version 1818634 (0.0009) [2023-12-27 18:15:20,977][20381] Updated weights for policy 1, policy_version 1818644 (0.0008) [2023-12-27 18:15:21,384][20347] Updated weights for policy 0, policy_version 1820223 (0.0008) [2023-12-27 18:15:21,445][20347] Updated weights for policy 0, policy_version 1820233 (0.0007) [2023-12-27 18:15:21,512][20347] Updated weights for policy 0, policy_version 1820243 (0.0008) [2023-12-27 18:15:21,777][20381] Updated weights for policy 1, policy_version 1818654 (0.0008) [2023-12-27 18:15:21,834][20381] Updated weights for policy 1, policy_version 1818664 (0.0008) [2023-12-27 18:15:21,897][20381] Updated weights for policy 1, policy_version 1818674 (0.0008) [2023-12-27 18:15:22,135][20347] Updated weights for policy 0, policy_version 1820253 (0.0006) [2023-12-27 18:15:22,191][20347] Updated weights for policy 0, policy_version 1820263 (0.0005) [2023-12-27 18:15:22,249][20347] Updated weights for policy 0, policy_version 1820273 (0.0007) [2023-12-27 18:15:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 931708928. Throughput: 0: 10234.7, 1: 10195.6. Samples: 931676964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:22,363][19170] Avg episode reward: [(0, '980.370'), (1, '941.480')] [2023-12-27 18:15:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001818680_465633280.pth... [2023-12-27 18:15:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001820280_466075648.pth... [2023-12-27 18:15:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001817496_465330176.pth [2023-12-27 18:15:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001819064_465764352.pth [2023-12-27 18:15:22,650][20381] Updated weights for policy 1, policy_version 1818684 (0.0008) [2023-12-27 18:15:22,712][20381] Updated weights for policy 1, policy_version 1818694 (0.0007) [2023-12-27 18:15:22,774][20381] Updated weights for policy 1, policy_version 1818704 (0.0006) [2023-12-27 18:15:22,895][20347] Updated weights for policy 0, policy_version 1820283 (0.0009) [2023-12-27 18:15:22,950][20347] Updated weights for policy 0, policy_version 1820293 (0.0009) [2023-12-27 18:15:23,007][20347] Updated weights for policy 0, policy_version 1820303 (0.0010) [2023-12-27 18:15:23,456][20381] Updated weights for policy 1, policy_version 1818714 (0.0006) [2023-12-27 18:15:23,507][20381] Updated weights for policy 1, policy_version 1818724 (0.0007) [2023-12-27 18:15:23,552][20381] Updated weights for policy 1, policy_version 1818734 (0.0007) [2023-12-27 18:15:23,597][20381] Updated weights for policy 1, policy_version 1818744 (0.0008) [2023-12-27 18:15:23,669][20347] Updated weights for policy 0, policy_version 1820313 (0.0009) [2023-12-27 18:15:23,727][20347] Updated weights for policy 0, policy_version 1820323 (0.0010) [2023-12-27 18:15:23,778][20347] Updated weights for policy 0, policy_version 1820333 (0.0010) [2023-12-27 18:15:23,827][20347] Updated weights for policy 0, policy_version 1820343 (0.0010) [2023-12-27 18:15:24,288][20381] Updated weights for policy 1, policy_version 1818754 (0.0009) [2023-12-27 18:15:24,346][20381] Updated weights for policy 1, policy_version 1818764 (0.0007) [2023-12-27 18:15:24,401][20381] Updated weights for policy 1, policy_version 1818774 (0.0007) [2023-12-27 18:15:24,500][20347] Updated weights for policy 0, policy_version 1820353 (0.0010) [2023-12-27 18:15:24,559][20347] Updated weights for policy 0, policy_version 1820363 (0.0007) [2023-12-27 18:15:24,607][20347] Updated weights for policy 0, policy_version 1820373 (0.0010) [2023-12-27 18:15:25,105][20381] Updated weights for policy 1, policy_version 1818784 (0.0009) [2023-12-27 18:15:25,154][20381] Updated weights for policy 1, policy_version 1818794 (0.0010) [2023-12-27 18:15:25,155][20347] Updated weights for policy 0, policy_version 1820383 (0.0007) [2023-12-27 18:15:25,200][20381] Updated weights for policy 1, policy_version 1818804 (0.0010) [2023-12-27 18:15:25,215][20347] Updated weights for policy 0, policy_version 1820393 (0.0006) [2023-12-27 18:15:25,266][20347] Updated weights for policy 0, policy_version 1820403 (0.0010) [2023-12-27 18:15:25,873][20347] Updated weights for policy 0, policy_version 1820413 (0.0009) [2023-12-27 18:15:25,898][20381] Updated weights for policy 1, policy_version 1818814 (0.0009) [2023-12-27 18:15:25,923][20347] Updated weights for policy 0, policy_version 1820423 (0.0006) [2023-12-27 18:15:25,953][20381] Updated weights for policy 1, policy_version 1818824 (0.0007) [2023-12-27 18:15:25,978][20347] Updated weights for policy 0, policy_version 1820433 (0.0006) [2023-12-27 18:15:26,008][20381] Updated weights for policy 1, policy_version 1818834 (0.0007) [2023-12-27 18:15:26,638][20381] Updated weights for policy 1, policy_version 1818844 (0.0009) [2023-12-27 18:15:26,669][20347] Updated weights for policy 0, policy_version 1820443 (0.0005) [2023-12-27 18:15:26,693][20381] Updated weights for policy 1, policy_version 1818854 (0.0010) [2023-12-27 18:15:26,717][20347] Updated weights for policy 0, policy_version 1820453 (0.0005) [2023-12-27 18:15:26,748][20381] Updated weights for policy 1, policy_version 1818864 (0.0010) [2023-12-27 18:15:26,768][20347] Updated weights for policy 0, policy_version 1820463 (0.0005) [2023-12-27 18:15:27,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 931807232. Throughput: 0: 10304.7, 1: 10137.3. Samples: 931799420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:27,364][19170] Avg episode reward: [(0, '941.430'), (1, '970.600')] [2023-12-27 18:15:27,400][20381] Updated weights for policy 1, policy_version 1818874 (0.0010) [2023-12-27 18:15:27,452][20381] Updated weights for policy 1, policy_version 1818884 (0.0010) [2023-12-27 18:15:27,461][20347] Updated weights for policy 0, policy_version 1820473 (0.0007) [2023-12-27 18:15:27,506][20381] Updated weights for policy 1, policy_version 1818894 (0.0010) [2023-12-27 18:15:27,515][20347] Updated weights for policy 0, policy_version 1820483 (0.0005) [2023-12-27 18:15:27,567][20381] Updated weights for policy 1, policy_version 1818904 (0.0010) [2023-12-27 18:15:27,572][20347] Updated weights for policy 0, policy_version 1820493 (0.0006) [2023-12-27 18:15:27,625][20347] Updated weights for policy 0, policy_version 1820503 (0.0007) [2023-12-27 18:15:28,190][20381] Updated weights for policy 1, policy_version 1818914 (0.0007) [2023-12-27 18:15:28,243][20381] Updated weights for policy 1, policy_version 1818924 (0.0010) [2023-12-27 18:15:28,293][20381] Updated weights for policy 1, policy_version 1818934 (0.0009) [2023-12-27 18:15:28,311][20347] Updated weights for policy 0, policy_version 1820513 (0.0006) [2023-12-27 18:15:28,378][20347] Updated weights for policy 0, policy_version 1820523 (0.0008) [2023-12-27 18:15:28,429][20347] Updated weights for policy 0, policy_version 1820533 (0.0008) [2023-12-27 18:15:28,893][20381] Updated weights for policy 1, policy_version 1818944 (0.0009) [2023-12-27 18:15:28,952][20381] Updated weights for policy 1, policy_version 1818954 (0.0009) [2023-12-27 18:15:29,009][20381] Updated weights for policy 1, policy_version 1818964 (0.0010) [2023-12-27 18:15:29,144][20347] Updated weights for policy 0, policy_version 1820543 (0.0006) [2023-12-27 18:15:29,201][20347] Updated weights for policy 0, policy_version 1820553 (0.0005) [2023-12-27 18:15:29,267][20347] Updated weights for policy 0, policy_version 1820563 (0.0007) [2023-12-27 18:15:29,751][20381] Updated weights for policy 1, policy_version 1818974 (0.0010) [2023-12-27 18:15:29,806][20381] Updated weights for policy 1, policy_version 1818984 (0.0010) [2023-12-27 18:15:29,867][20381] Updated weights for policy 1, policy_version 1818994 (0.0009) [2023-12-27 18:15:29,943][20347] Updated weights for policy 0, policy_version 1820573 (0.0007) [2023-12-27 18:15:30,007][20347] Updated weights for policy 0, policy_version 1820583 (0.0008) [2023-12-27 18:15:30,067][20347] Updated weights for policy 0, policy_version 1820593 (0.0005) [2023-12-27 18:15:30,538][20381] Updated weights for policy 1, policy_version 1819004 (0.0007) [2023-12-27 18:15:30,589][20381] Updated weights for policy 1, policy_version 1819014 (0.0005) [2023-12-27 18:15:30,646][20381] Updated weights for policy 1, policy_version 1819024 (0.0005) [2023-12-27 18:15:30,715][20347] Updated weights for policy 0, policy_version 1820603 (0.0007) [2023-12-27 18:15:30,777][20347] Updated weights for policy 0, policy_version 1820613 (0.0005) [2023-12-27 18:15:30,827][20347] Updated weights for policy 0, policy_version 1820623 (0.0006) [2023-12-27 18:15:31,244][20381] Updated weights for policy 1, policy_version 1819034 (0.0007) [2023-12-27 18:15:31,307][20381] Updated weights for policy 1, policy_version 1819044 (0.0008) [2023-12-27 18:15:31,369][20381] Updated weights for policy 1, policy_version 1819054 (0.0007) [2023-12-27 18:15:31,426][20381] Updated weights for policy 1, policy_version 1819064 (0.0007) [2023-12-27 18:15:31,520][20347] Updated weights for policy 0, policy_version 1820633 (0.0006) [2023-12-27 18:15:31,583][20347] Updated weights for policy 0, policy_version 1820643 (0.0008) [2023-12-27 18:15:31,638][20347] Updated weights for policy 0, policy_version 1820653 (0.0008) [2023-12-27 18:15:31,697][20347] Updated weights for policy 0, policy_version 1820663 (0.0009) [2023-12-27 18:15:32,107][20381] Updated weights for policy 1, policy_version 1819074 (0.0008) [2023-12-27 18:15:32,156][20381] Updated weights for policy 1, policy_version 1819084 (0.0009) [2023-12-27 18:15:32,206][20381] Updated weights for policy 1, policy_version 1819094 (0.0009) [2023-12-27 18:15:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 931913728. Throughput: 0: 10280.8, 1: 10193.3. Samples: 931922600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:32,363][19170] Avg episode reward: [(0, '931.620'), (1, '959.390')] [2023-12-27 18:15:32,388][20347] Updated weights for policy 0, policy_version 1820673 (0.0008) [2023-12-27 18:15:32,443][20347] Updated weights for policy 0, policy_version 1820683 (0.0007) [2023-12-27 18:15:32,490][20347] Updated weights for policy 0, policy_version 1820693 (0.0006) [2023-12-27 18:15:32,957][20381] Updated weights for policy 1, policy_version 1819104 (0.0008) [2023-12-27 18:15:33,006][20381] Updated weights for policy 1, policy_version 1819114 (0.0008) [2023-12-27 18:15:33,058][20381] Updated weights for policy 1, policy_version 1819124 (0.0008) [2023-12-27 18:15:33,138][20347] Updated weights for policy 0, policy_version 1820703 (0.0007) [2023-12-27 18:15:33,188][20347] Updated weights for policy 0, policy_version 1820713 (0.0007) [2023-12-27 18:15:33,242][20347] Updated weights for policy 0, policy_version 1820723 (0.0008) [2023-12-27 18:15:33,770][20381] Updated weights for policy 1, policy_version 1819134 (0.0008) [2023-12-27 18:15:33,823][20381] Updated weights for policy 1, policy_version 1819144 (0.0007) [2023-12-27 18:15:33,837][20292] KL-divergence is very high: 185.5715 [2023-12-27 18:15:33,848][20292] KL-divergence is very high: 221.5945 [2023-12-27 18:15:33,857][20347] Updated weights for policy 0, policy_version 1820733 (0.0006) [2023-12-27 18:15:33,881][20381] Updated weights for policy 1, policy_version 1819154 (0.0005) [2023-12-27 18:15:33,881][20292] KL-divergence is very high: 203.2478 [2023-12-27 18:15:33,892][20292] KL-divergence is very high: 167.3050 [2023-12-27 18:15:33,912][20347] Updated weights for policy 0, policy_version 1820743 (0.0005) [2023-12-27 18:15:33,972][20347] Updated weights for policy 0, policy_version 1820753 (0.0007) [2023-12-27 18:15:34,630][20381] Updated weights for policy 1, policy_version 1819164 (0.0006) [2023-12-27 18:15:34,654][20347] Updated weights for policy 0, policy_version 1820763 (0.0008) [2023-12-27 18:15:34,694][20381] Updated weights for policy 1, policy_version 1819174 (0.0006) [2023-12-27 18:15:34,714][20347] Updated weights for policy 0, policy_version 1820773 (0.0010) [2023-12-27 18:15:34,757][20381] Updated weights for policy 1, policy_version 1819184 (0.0006) [2023-12-27 18:15:34,771][20347] Updated weights for policy 0, policy_version 1820783 (0.0011) [2023-12-27 18:15:35,423][20347] Updated weights for policy 0, policy_version 1820793 (0.0010) [2023-12-27 18:15:35,436][20381] Updated weights for policy 1, policy_version 1819194 (0.0006) [2023-12-27 18:15:35,475][20347] Updated weights for policy 0, policy_version 1820803 (0.0007) [2023-12-27 18:15:35,496][20381] Updated weights for policy 1, policy_version 1819204 (0.0007) [2023-12-27 18:15:35,525][20347] Updated weights for policy 0, policy_version 1820813 (0.0005) [2023-12-27 18:15:35,556][20381] Updated weights for policy 1, policy_version 1819214 (0.0008) [2023-12-27 18:15:35,578][20347] Updated weights for policy 0, policy_version 1820823 (0.0006) [2023-12-27 18:15:35,621][20381] Updated weights for policy 1, policy_version 1819224 (0.0007) [2023-12-27 18:15:36,168][20347] Updated weights for policy 0, policy_version 1820833 (0.0007) [2023-12-27 18:15:36,229][20347] Updated weights for policy 0, policy_version 1820843 (0.0007) [2023-12-27 18:15:36,289][20347] Updated weights for policy 0, policy_version 1820853 (0.0006) [2023-12-27 18:15:36,379][20381] Updated weights for policy 1, policy_version 1819234 (0.0009) [2023-12-27 18:15:36,436][20381] Updated weights for policy 1, policy_version 1819244 (0.0008) [2023-12-27 18:15:36,497][20381] Updated weights for policy 1, policy_version 1819254 (0.0009) [2023-12-27 18:15:36,932][20347] Updated weights for policy 0, policy_version 1820863 (0.0006) [2023-12-27 18:15:36,995][20347] Updated weights for policy 0, policy_version 1820873 (0.0006) [2023-12-27 18:15:37,054][20347] Updated weights for policy 0, policy_version 1820883 (0.0005) [2023-12-27 18:15:37,219][20381] Updated weights for policy 1, policy_version 1819264 (0.0007) [2023-12-27 18:15:37,278][20381] Updated weights for policy 1, policy_version 1819274 (0.0008) [2023-12-27 18:15:37,337][20381] Updated weights for policy 1, policy_version 1819284 (0.0008) [2023-12-27 18:15:37,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 932020224. Throughput: 0: 10282.2, 1: 10176.3. Samples: 931983696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:37,363][19170] Avg episode reward: [(0, '882.440'), (1, '928.780')] [2023-12-27 18:15:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001820888_466231296.pth... [2023-12-27 18:15:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001819288_465788928.pth... [2023-12-27 18:15:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001819672_465920000.pth [2023-12-27 18:15:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001818104_465485824.pth [2023-12-27 18:15:37,654][20347] Updated weights for policy 0, policy_version 1820893 (0.0006) [2023-12-27 18:15:37,705][20347] Updated weights for policy 0, policy_version 1820903 (0.0007) [2023-12-27 18:15:37,767][20347] Updated weights for policy 0, policy_version 1820913 (0.0006) [2023-12-27 18:15:38,046][20381] Updated weights for policy 1, policy_version 1819294 (0.0008) [2023-12-27 18:15:38,111][20381] Updated weights for policy 1, policy_version 1819304 (0.0008) [2023-12-27 18:15:38,169][20381] Updated weights for policy 1, policy_version 1819314 (0.0007) [2023-12-27 18:15:38,414][20347] Updated weights for policy 0, policy_version 1820923 (0.0008) [2023-12-27 18:15:38,481][20347] Updated weights for policy 0, policy_version 1820933 (0.0008) [2023-12-27 18:15:38,549][20347] Updated weights for policy 0, policy_version 1820943 (0.0007) [2023-12-27 18:15:38,898][20381] Updated weights for policy 1, policy_version 1819324 (0.0008) [2023-12-27 18:15:38,960][20381] Updated weights for policy 1, policy_version 1819334 (0.0008) [2023-12-27 18:15:39,010][20381] Updated weights for policy 1, policy_version 1819344 (0.0008) [2023-12-27 18:15:39,231][20347] Updated weights for policy 0, policy_version 1820953 (0.0008) [2023-12-27 18:15:39,290][20347] Updated weights for policy 0, policy_version 1820963 (0.0007) [2023-12-27 18:15:39,359][20347] Updated weights for policy 0, policy_version 1820973 (0.0008) [2023-12-27 18:15:39,415][20347] Updated weights for policy 0, policy_version 1820983 (0.0007) [2023-12-27 18:15:39,696][20381] Updated weights for policy 1, policy_version 1819354 (0.0009) [2023-12-27 18:15:39,746][20381] Updated weights for policy 1, policy_version 1819364 (0.0011) [2023-12-27 18:15:39,801][20381] Updated weights for policy 1, policy_version 1819374 (0.0011) [2023-12-27 18:15:39,866][20381] Updated weights for policy 1, policy_version 1819384 (0.0008) [2023-12-27 18:15:40,159][20347] Updated weights for policy 0, policy_version 1820993 (0.0007) [2023-12-27 18:15:40,216][20347] Updated weights for policy 0, policy_version 1821003 (0.0008) [2023-12-27 18:15:40,269][20347] Updated weights for policy 0, policy_version 1821013 (0.0007) [2023-12-27 18:15:40,546][20381] Updated weights for policy 1, policy_version 1819394 (0.0009) [2023-12-27 18:15:40,604][20381] Updated weights for policy 1, policy_version 1819404 (0.0011) [2023-12-27 18:15:40,651][20381] Updated weights for policy 1, policy_version 1819414 (0.0011) [2023-12-27 18:15:40,997][20347] Updated weights for policy 0, policy_version 1821023 (0.0008) [2023-12-27 18:15:41,065][20347] Updated weights for policy 0, policy_version 1821033 (0.0008) [2023-12-27 18:15:41,125][20347] Updated weights for policy 0, policy_version 1821043 (0.0007) [2023-12-27 18:15:41,429][20381] Updated weights for policy 1, policy_version 1819424 (0.0009) [2023-12-27 18:15:41,493][20381] Updated weights for policy 1, policy_version 1819434 (0.0007) [2023-12-27 18:15:41,551][20381] Updated weights for policy 1, policy_version 1819444 (0.0008) [2023-12-27 18:15:41,903][20347] Updated weights for policy 0, policy_version 1821053 (0.0008) [2023-12-27 18:15:41,963][20347] Updated weights for policy 0, policy_version 1821063 (0.0008) [2023-12-27 18:15:41,980][20291] KL-divergence is very high: 107.3560 [2023-12-27 18:15:41,992][20291] KL-divergence is very high: 113.6068 [2023-12-27 18:15:42,019][20347] Updated weights for policy 0, policy_version 1821073 (0.0008) [2023-12-27 18:15:42,201][20381] Updated weights for policy 1, policy_version 1819454 (0.0007) [2023-12-27 18:15:42,266][20381] Updated weights for policy 1, policy_version 1819464 (0.0007) [2023-12-27 18:15:42,326][20381] Updated weights for policy 1, policy_version 1819474 (0.0008) [2023-12-27 18:15:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 932110336. Throughput: 0: 10288.3, 1: 10065.5. Samples: 932103952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:42,363][19170] Avg episode reward: [(0, '892.210'), (1, '959.020')] [2023-12-27 18:15:42,836][20347] Updated weights for policy 0, policy_version 1821083 (0.0008) [2023-12-27 18:15:42,894][20347] Updated weights for policy 0, policy_version 1821093 (0.0008) [2023-12-27 18:15:42,952][20347] Updated weights for policy 0, policy_version 1821103 (0.0008) [2023-12-27 18:15:42,997][20381] Updated weights for policy 1, policy_version 1819484 (0.0007) [2023-12-27 18:15:43,049][20381] Updated weights for policy 1, policy_version 1819494 (0.0007) [2023-12-27 18:15:43,101][20381] Updated weights for policy 1, policy_version 1819504 (0.0008) [2023-12-27 18:15:43,673][20347] Updated weights for policy 0, policy_version 1821113 (0.0007) [2023-12-27 18:15:43,723][20347] Updated weights for policy 0, policy_version 1821123 (0.0006) [2023-12-27 18:15:43,773][20347] Updated weights for policy 0, policy_version 1821133 (0.0008) [2023-12-27 18:15:43,802][20381] Updated weights for policy 1, policy_version 1819514 (0.0007) [2023-12-27 18:15:43,823][20347] Updated weights for policy 0, policy_version 1821143 (0.0006) [2023-12-27 18:15:43,848][20381] Updated weights for policy 1, policy_version 1819524 (0.0006) [2023-12-27 18:15:43,901][20381] Updated weights for policy 1, policy_version 1819534 (0.0008) [2023-12-27 18:15:43,953][20381] Updated weights for policy 1, policy_version 1819544 (0.0007) [2023-12-27 18:15:44,531][20347] Updated weights for policy 0, policy_version 1821153 (0.0007) [2023-12-27 18:15:44,579][20347] Updated weights for policy 0, policy_version 1821163 (0.0007) [2023-12-27 18:15:44,631][20347] Updated weights for policy 0, policy_version 1821173 (0.0005) [2023-12-27 18:15:44,688][20381] Updated weights for policy 1, policy_version 1819554 (0.0007) [2023-12-27 18:15:44,750][20381] Updated weights for policy 1, policy_version 1819564 (0.0007) [2023-12-27 18:15:44,812][20381] Updated weights for policy 1, policy_version 1819574 (0.0008) [2023-12-27 18:15:45,378][20347] Updated weights for policy 0, policy_version 1821183 (0.0007) [2023-12-27 18:15:45,425][20347] Updated weights for policy 0, policy_version 1821193 (0.0008) [2023-12-27 18:15:45,474][20347] Updated weights for policy 0, policy_version 1821203 (0.0007) [2023-12-27 18:15:45,517][20381] Updated weights for policy 1, policy_version 1819584 (0.0006) [2023-12-27 18:15:45,577][20381] Updated weights for policy 1, policy_version 1819594 (0.0006) [2023-12-27 18:15:45,634][20381] Updated weights for policy 1, policy_version 1819604 (0.0006) [2023-12-27 18:15:46,126][20347] Updated weights for policy 0, policy_version 1821213 (0.0008) [2023-12-27 18:15:46,181][20347] Updated weights for policy 0, policy_version 1821223 (0.0010) [2023-12-27 18:15:46,233][20347] Updated weights for policy 0, policy_version 1821233 (0.0010) [2023-12-27 18:15:46,253][20381] Updated weights for policy 1, policy_version 1819614 (0.0006) [2023-12-27 18:15:46,305][20381] Updated weights for policy 1, policy_version 1819624 (0.0006) [2023-12-27 18:15:46,354][20381] Updated weights for policy 1, policy_version 1819634 (0.0005) [2023-12-27 18:15:46,980][20381] Updated weights for policy 1, policy_version 1819644 (0.0006) [2023-12-27 18:15:46,988][20347] Updated weights for policy 0, policy_version 1821243 (0.0010) [2023-12-27 18:15:47,038][20381] Updated weights for policy 1, policy_version 1819654 (0.0007) [2023-12-27 18:15:47,041][20347] Updated weights for policy 0, policy_version 1821253 (0.0005) [2023-12-27 18:15:47,098][20381] Updated weights for policy 1, policy_version 1819664 (0.0006) [2023-12-27 18:15:47,099][20347] Updated weights for policy 0, policy_version 1821263 (0.0006) [2023-12-27 18:15:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 932216832. Throughput: 0: 10197.8, 1: 10077.9. Samples: 932222132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:47,364][19170] Avg episode reward: [(0, '960.810'), (1, '979.210')] [2023-12-27 18:15:47,730][20347] Updated weights for policy 0, policy_version 1821273 (0.0006) [2023-12-27 18:15:47,774][20381] Updated weights for policy 1, policy_version 1819674 (0.0007) [2023-12-27 18:15:47,793][20347] Updated weights for policy 0, policy_version 1821283 (0.0011) [2023-12-27 18:15:47,831][20381] Updated weights for policy 1, policy_version 1819684 (0.0009) [2023-12-27 18:15:47,852][20347] Updated weights for policy 0, policy_version 1821293 (0.0010) [2023-12-27 18:15:47,885][20381] Updated weights for policy 1, policy_version 1819694 (0.0011) [2023-12-27 18:15:47,901][20347] Updated weights for policy 0, policy_version 1821303 (0.0010) [2023-12-27 18:15:47,938][20381] Updated weights for policy 1, policy_version 1819704 (0.0010) [2023-12-27 18:15:48,582][20347] Updated weights for policy 0, policy_version 1821313 (0.0011) [2023-12-27 18:15:48,592][20381] Updated weights for policy 1, policy_version 1819714 (0.0010) [2023-12-27 18:15:48,635][20347] Updated weights for policy 0, policy_version 1821323 (0.0010) [2023-12-27 18:15:48,649][20381] Updated weights for policy 1, policy_version 1819724 (0.0008) [2023-12-27 18:15:48,694][20347] Updated weights for policy 0, policy_version 1821333 (0.0010) [2023-12-27 18:15:48,707][20381] Updated weights for policy 1, policy_version 1819734 (0.0006) [2023-12-27 18:15:49,387][20381] Updated weights for policy 1, policy_version 1819744 (0.0008) [2023-12-27 18:15:49,408][20347] Updated weights for policy 0, policy_version 1821343 (0.0010) [2023-12-27 18:15:49,444][20381] Updated weights for policy 1, policy_version 1819754 (0.0010) [2023-12-27 18:15:49,467][20347] Updated weights for policy 0, policy_version 1821353 (0.0010) [2023-12-27 18:15:49,499][20381] Updated weights for policy 1, policy_version 1819764 (0.0011) [2023-12-27 18:15:49,523][20347] Updated weights for policy 0, policy_version 1821363 (0.0010) [2023-12-27 18:15:50,201][20381] Updated weights for policy 1, policy_version 1819774 (0.0010) [2023-12-27 18:15:50,263][20381] Updated weights for policy 1, policy_version 1819784 (0.0010) [2023-12-27 18:15:50,269][20347] Updated weights for policy 0, policy_version 1821373 (0.0008) [2023-12-27 18:15:50,321][20381] Updated weights for policy 1, policy_version 1819794 (0.0011) [2023-12-27 18:15:50,327][20347] Updated weights for policy 0, policy_version 1821383 (0.0006) [2023-12-27 18:15:50,384][20347] Updated weights for policy 0, policy_version 1821393 (0.0009) [2023-12-27 18:15:50,976][20381] Updated weights for policy 1, policy_version 1819804 (0.0008) [2023-12-27 18:15:51,036][20381] Updated weights for policy 1, policy_version 1819814 (0.0011) [2023-12-27 18:15:51,042][20347] Updated weights for policy 0, policy_version 1821403 (0.0007) [2023-12-27 18:15:51,093][20381] Updated weights for policy 1, policy_version 1819824 (0.0008) [2023-12-27 18:15:51,099][20347] Updated weights for policy 0, policy_version 1821413 (0.0006) [2023-12-27 18:15:51,164][20347] Updated weights for policy 0, policy_version 1821423 (0.0007) [2023-12-27 18:15:51,818][20381] Updated weights for policy 1, policy_version 1819834 (0.0009) [2023-12-27 18:15:51,869][20381] Updated weights for policy 1, policy_version 1819844 (0.0007) [2023-12-27 18:15:51,896][20347] Updated weights for policy 0, policy_version 1821433 (0.0007) [2023-12-27 18:15:51,925][20381] Updated weights for policy 1, policy_version 1819854 (0.0007) [2023-12-27 18:15:51,950][20347] Updated weights for policy 0, policy_version 1821443 (0.0006) [2023-12-27 18:15:51,979][20381] Updated weights for policy 1, policy_version 1819864 (0.0005) [2023-12-27 18:15:52,009][20347] Updated weights for policy 0, policy_version 1821453 (0.0007) [2023-12-27 18:15:52,075][20347] Updated weights for policy 0, policy_version 1821463 (0.0008) [2023-12-27 18:15:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 932315136. Throughput: 0: 10168.9, 1: 10084.3. Samples: 932282828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:52,363][19170] Avg episode reward: [(0, '980.430'), (1, '939.430')] [2023-12-27 18:15:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001819864_465936384.pth... [2023-12-27 18:15:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001821464_466378752.pth... [2023-12-27 18:15:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001818680_465633280.pth [2023-12-27 18:15:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001820280_466075648.pth [2023-12-27 18:15:52,658][20381] Updated weights for policy 1, policy_version 1819874 (0.0007) [2023-12-27 18:15:52,717][20381] Updated weights for policy 1, policy_version 1819884 (0.0008) [2023-12-27 18:15:52,779][20381] Updated weights for policy 1, policy_version 1819894 (0.0008) [2023-12-27 18:15:52,811][20347] Updated weights for policy 0, policy_version 1821473 (0.0010) [2023-12-27 18:15:52,869][20347] Updated weights for policy 0, policy_version 1821483 (0.0010) [2023-12-27 18:15:52,928][20347] Updated weights for policy 0, policy_version 1821493 (0.0011) [2023-12-27 18:15:53,441][20381] Updated weights for policy 1, policy_version 1819904 (0.0006) [2023-12-27 18:15:53,494][20381] Updated weights for policy 1, policy_version 1819914 (0.0007) [2023-12-27 18:15:53,514][20347] Updated weights for policy 0, policy_version 1821503 (0.0008) [2023-12-27 18:15:53,549][20381] Updated weights for policy 1, policy_version 1819924 (0.0007) [2023-12-27 18:15:53,575][20347] Updated weights for policy 0, policy_version 1821513 (0.0008) [2023-12-27 18:15:53,634][20347] Updated weights for policy 0, policy_version 1821523 (0.0006) [2023-12-27 18:15:54,152][20381] Updated weights for policy 1, policy_version 1819934 (0.0008) [2023-12-27 18:15:54,209][20381] Updated weights for policy 1, policy_version 1819944 (0.0008) [2023-12-27 18:15:54,257][20347] Updated weights for policy 0, policy_version 1821533 (0.0006) [2023-12-27 18:15:54,261][20381] Updated weights for policy 1, policy_version 1819954 (0.0007) [2023-12-27 18:15:54,320][20347] Updated weights for policy 0, policy_version 1821543 (0.0006) [2023-12-27 18:15:54,376][20347] Updated weights for policy 0, policy_version 1821553 (0.0006) [2023-12-27 18:15:54,841][20381] Updated weights for policy 1, policy_version 1819964 (0.0006) [2023-12-27 18:15:54,903][20381] Updated weights for policy 1, policy_version 1819974 (0.0005) [2023-12-27 18:15:54,966][20381] Updated weights for policy 1, policy_version 1819984 (0.0006) [2023-12-27 18:15:55,036][20347] Updated weights for policy 0, policy_version 1821563 (0.0006) [2023-12-27 18:15:55,095][20347] Updated weights for policy 0, policy_version 1821573 (0.0008) [2023-12-27 18:15:55,158][20347] Updated weights for policy 0, policy_version 1821583 (0.0007) [2023-12-27 18:15:55,556][20381] Updated weights for policy 1, policy_version 1819994 (0.0008) [2023-12-27 18:15:55,601][20381] Updated weights for policy 1, policy_version 1820004 (0.0007) [2023-12-27 18:15:55,650][20381] Updated weights for policy 1, policy_version 1820014 (0.0007) [2023-12-27 18:15:55,703][20381] Updated weights for policy 1, policy_version 1820024 (0.0007) [2023-12-27 18:15:55,859][20347] Updated weights for policy 0, policy_version 1821593 (0.0007) [2023-12-27 18:15:55,912][20347] Updated weights for policy 0, policy_version 1821603 (0.0008) [2023-12-27 18:15:55,965][20347] Updated weights for policy 0, policy_version 1821613 (0.0007) [2023-12-27 18:15:56,020][20347] Updated weights for policy 0, policy_version 1821623 (0.0006) [2023-12-27 18:15:56,370][20381] Updated weights for policy 1, policy_version 1820034 (0.0008) [2023-12-27 18:15:56,414][20381] Updated weights for policy 1, policy_version 1820044 (0.0007) [2023-12-27 18:15:56,461][20381] Updated weights for policy 1, policy_version 1820054 (0.0007) [2023-12-27 18:15:56,635][20347] Updated weights for policy 0, policy_version 1821633 (0.0007) [2023-12-27 18:15:56,690][20347] Updated weights for policy 0, policy_version 1821643 (0.0007) [2023-12-27 18:15:56,755][20347] Updated weights for policy 0, policy_version 1821653 (0.0005) [2023-12-27 18:15:57,130][20381] Updated weights for policy 1, policy_version 1820064 (0.0005) [2023-12-27 18:15:57,178][20381] Updated weights for policy 1, policy_version 1820074 (0.0005) [2023-12-27 18:15:57,233][20381] Updated weights for policy 1, policy_version 1820084 (0.0007) [2023-12-27 18:15:57,349][20347] Updated weights for policy 0, policy_version 1821663 (0.0009) [2023-12-27 18:15:57,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 932421632. Throughput: 0: 10173.7, 1: 10144.5. Samples: 932408196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:15:57,363][19170] Avg episode reward: [(0, '921.710'), (1, '969.040')] [2023-12-27 18:15:57,397][20347] Updated weights for policy 0, policy_version 1821673 (0.0010) [2023-12-27 18:15:57,444][20347] Updated weights for policy 0, policy_version 1821683 (0.0010) [2023-12-27 18:15:57,873][20381] Updated weights for policy 1, policy_version 1820094 (0.0010) [2023-12-27 18:15:57,929][20381] Updated weights for policy 1, policy_version 1820104 (0.0009) [2023-12-27 18:15:57,988][20381] Updated weights for policy 1, policy_version 1820114 (0.0006) [2023-12-27 18:15:58,107][20347] Updated weights for policy 0, policy_version 1821693 (0.0010) [2023-12-27 18:15:58,155][20347] Updated weights for policy 0, policy_version 1821703 (0.0010) [2023-12-27 18:15:58,213][20347] Updated weights for policy 0, policy_version 1821713 (0.0007) [2023-12-27 18:15:58,717][20381] Updated weights for policy 1, policy_version 1820124 (0.0008) [2023-12-27 18:15:58,781][20381] Updated weights for policy 1, policy_version 1820134 (0.0007) [2023-12-27 18:15:58,851][20381] Updated weights for policy 1, policy_version 1820144 (0.0007) [2023-12-27 18:15:59,035][20347] Updated weights for policy 0, policy_version 1821723 (0.0010) [2023-12-27 18:15:59,094][20347] Updated weights for policy 0, policy_version 1821733 (0.0010) [2023-12-27 18:15:59,154][20347] Updated weights for policy 0, policy_version 1821743 (0.0011) [2023-12-27 18:15:59,654][20381] Updated weights for policy 1, policy_version 1820154 (0.0009) [2023-12-27 18:15:59,703][20381] Updated weights for policy 1, policy_version 1820164 (0.0011) [2023-12-27 18:15:59,755][20381] Updated weights for policy 1, policy_version 1820174 (0.0008) [2023-12-27 18:15:59,806][20381] Updated weights for policy 1, policy_version 1820184 (0.0007) [2023-12-27 18:15:59,887][20347] Updated weights for policy 0, policy_version 1821753 (0.0010) [2023-12-27 18:15:59,949][20347] Updated weights for policy 0, policy_version 1821763 (0.0010) [2023-12-27 18:16:00,010][20347] Updated weights for policy 0, policy_version 1821773 (0.0011) [2023-12-27 18:16:00,078][20347] Updated weights for policy 0, policy_version 1821783 (0.0011) [2023-12-27 18:16:00,472][20381] Updated weights for policy 1, policy_version 1820194 (0.0006) [2023-12-27 18:16:00,517][20381] Updated weights for policy 1, policy_version 1820204 (0.0005) [2023-12-27 18:16:00,560][20381] Updated weights for policy 1, policy_version 1820214 (0.0007) [2023-12-27 18:16:00,738][20347] Updated weights for policy 0, policy_version 1821793 (0.0006) [2023-12-27 18:16:00,799][20347] Updated weights for policy 0, policy_version 1821803 (0.0007) [2023-12-27 18:16:00,849][20347] Updated weights for policy 0, policy_version 1821813 (0.0006) [2023-12-27 18:16:01,212][20381] Updated weights for policy 1, policy_version 1820224 (0.0010) [2023-12-27 18:16:01,272][20381] Updated weights for policy 1, policy_version 1820234 (0.0009) [2023-12-27 18:16:01,327][20381] Updated weights for policy 1, policy_version 1820244 (0.0005) [2023-12-27 18:16:01,492][20347] Updated weights for policy 0, policy_version 1821823 (0.0009) [2023-12-27 18:16:01,552][20347] Updated weights for policy 0, policy_version 1821833 (0.0010) [2023-12-27 18:16:01,610][20347] Updated weights for policy 0, policy_version 1821843 (0.0011) [2023-12-27 18:16:01,940][20381] Updated weights for policy 1, policy_version 1820254 (0.0006) [2023-12-27 18:16:02,007][20381] Updated weights for policy 1, policy_version 1820264 (0.0008) [2023-12-27 18:16:02,078][20381] Updated weights for policy 1, policy_version 1820274 (0.0009) [2023-12-27 18:16:02,270][20347] Updated weights for policy 0, policy_version 1821853 (0.0009) [2023-12-27 18:16:02,329][20347] Updated weights for policy 0, policy_version 1821863 (0.0010) [2023-12-27 18:16:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 932519936. Throughput: 0: 10196.7, 1: 10117.4. Samples: 932531484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:02,363][19170] Avg episode reward: [(0, '872.770'), (1, '921.950')] [2023-12-27 18:16:02,385][20347] Updated weights for policy 0, policy_version 1821873 (0.0010) [2023-12-27 18:16:02,702][20381] Updated weights for policy 1, policy_version 1820284 (0.0007) [2023-12-27 18:16:02,756][20381] Updated weights for policy 1, policy_version 1820294 (0.0007) [2023-12-27 18:16:02,811][20381] Updated weights for policy 1, policy_version 1820304 (0.0008) [2023-12-27 18:16:03,015][20347] Updated weights for policy 0, policy_version 1821883 (0.0006) [2023-12-27 18:16:03,074][20347] Updated weights for policy 0, policy_version 1821893 (0.0010) [2023-12-27 18:16:03,129][20347] Updated weights for policy 0, policy_version 1821903 (0.0010) [2023-12-27 18:16:03,516][20381] Updated weights for policy 1, policy_version 1820314 (0.0007) [2023-12-27 18:16:03,572][20381] Updated weights for policy 1, policy_version 1820324 (0.0007) [2023-12-27 18:16:03,623][20381] Updated weights for policy 1, policy_version 1820334 (0.0007) [2023-12-27 18:16:03,676][20381] Updated weights for policy 1, policy_version 1820344 (0.0007) [2023-12-27 18:16:03,810][20347] Updated weights for policy 0, policy_version 1821913 (0.0010) [2023-12-27 18:16:03,878][20347] Updated weights for policy 0, policy_version 1821923 (0.0011) [2023-12-27 18:16:03,934][20347] Updated weights for policy 0, policy_version 1821933 (0.0008) [2023-12-27 18:16:03,990][20347] Updated weights for policy 0, policy_version 1821943 (0.0007) [2023-12-27 18:16:04,423][20381] Updated weights for policy 1, policy_version 1820354 (0.0008) [2023-12-27 18:16:04,473][20381] Updated weights for policy 1, policy_version 1820364 (0.0010) [2023-12-27 18:16:04,529][20381] Updated weights for policy 1, policy_version 1820374 (0.0010) [2023-12-27 18:16:04,679][20347] Updated weights for policy 0, policy_version 1821953 (0.0010) [2023-12-27 18:16:04,734][20347] Updated weights for policy 0, policy_version 1821963 (0.0010) [2023-12-27 18:16:04,783][20347] Updated weights for policy 0, policy_version 1821973 (0.0010) [2023-12-27 18:16:05,231][20381] Updated weights for policy 1, policy_version 1820384 (0.0010) [2023-12-27 18:16:05,290][20381] Updated weights for policy 1, policy_version 1820394 (0.0011) [2023-12-27 18:16:05,348][20381] Updated weights for policy 1, policy_version 1820404 (0.0010) [2023-12-27 18:16:05,426][20347] Updated weights for policy 0, policy_version 1821983 (0.0010) [2023-12-27 18:16:05,486][20347] Updated weights for policy 0, policy_version 1821993 (0.0009) [2023-12-27 18:16:05,545][20347] Updated weights for policy 0, policy_version 1822003 (0.0006) [2023-12-27 18:16:06,062][20381] Updated weights for policy 1, policy_version 1820414 (0.0009) [2023-12-27 18:16:06,136][20381] Updated weights for policy 1, policy_version 1820424 (0.0008) [2023-12-27 18:16:06,162][20347] Updated weights for policy 0, policy_version 1822013 (0.0008) [2023-12-27 18:16:06,198][20381] Updated weights for policy 1, policy_version 1820434 (0.0007) [2023-12-27 18:16:06,224][20347] Updated weights for policy 0, policy_version 1822023 (0.0008) [2023-12-27 18:16:06,283][20347] Updated weights for policy 0, policy_version 1822033 (0.0006) [2023-12-27 18:16:06,833][20381] Updated weights for policy 1, policy_version 1820444 (0.0006) [2023-12-27 18:16:06,900][20381] Updated weights for policy 1, policy_version 1820454 (0.0005) [2023-12-27 18:16:06,967][20381] Updated weights for policy 1, policy_version 1820464 (0.0006) [2023-12-27 18:16:07,005][20347] Updated weights for policy 0, policy_version 1822043 (0.0006) [2023-12-27 18:16:07,066][20347] Updated weights for policy 0, policy_version 1822053 (0.0007) [2023-12-27 18:16:07,129][20347] Updated weights for policy 0, policy_version 1822063 (0.0005) [2023-12-27 18:16:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 932626432. Throughput: 0: 10199.4, 1: 10140.6. Samples: 932592268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:07,364][19170] Avg episode reward: [(0, '902.090'), (1, '931.670')] [2023-12-27 18:16:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001820472_466092032.pth... [2023-12-27 18:16:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001822072_466534400.pth... [2023-12-27 18:16:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001820888_466231296.pth [2023-12-27 18:16:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001819288_465788928.pth [2023-12-27 18:16:07,485][20381] Updated weights for policy 1, policy_version 1820474 (0.0006) [2023-12-27 18:16:07,531][20381] Updated weights for policy 1, policy_version 1820484 (0.0008) [2023-12-27 18:16:07,581][20381] Updated weights for policy 1, policy_version 1820494 (0.0008) [2023-12-27 18:16:07,643][20381] Updated weights for policy 1, policy_version 1820504 (0.0007) [2023-12-27 18:16:07,754][20347] Updated weights for policy 0, policy_version 1822073 (0.0007) [2023-12-27 18:16:07,811][20347] Updated weights for policy 0, policy_version 1822083 (0.0005) [2023-12-27 18:16:07,876][20347] Updated weights for policy 0, policy_version 1822093 (0.0005) [2023-12-27 18:16:07,938][20347] Updated weights for policy 0, policy_version 1822103 (0.0005) [2023-12-27 18:16:08,312][20381] Updated weights for policy 1, policy_version 1820514 (0.0009) [2023-12-27 18:16:08,375][20381] Updated weights for policy 1, policy_version 1820524 (0.0008) [2023-12-27 18:16:08,430][20381] Updated weights for policy 1, policy_version 1820534 (0.0007) [2023-12-27 18:16:08,471][20347] Updated weights for policy 0, policy_version 1822113 (0.0006) [2023-12-27 18:16:08,531][20347] Updated weights for policy 0, policy_version 1822123 (0.0005) [2023-12-27 18:16:08,580][20347] Updated weights for policy 0, policy_version 1822133 (0.0006) [2023-12-27 18:16:09,169][20381] Updated weights for policy 1, policy_version 1820544 (0.0008) [2023-12-27 18:16:09,223][20347] Updated weights for policy 0, policy_version 1822143 (0.0008) [2023-12-27 18:16:09,244][20381] Updated weights for policy 1, policy_version 1820554 (0.0008) [2023-12-27 18:16:09,285][20347] Updated weights for policy 0, policy_version 1822153 (0.0007) [2023-12-27 18:16:09,299][20381] Updated weights for policy 1, policy_version 1820564 (0.0006) [2023-12-27 18:16:09,346][20347] Updated weights for policy 0, policy_version 1822163 (0.0007) [2023-12-27 18:16:10,034][20381] Updated weights for policy 1, policy_version 1820574 (0.0006) [2023-12-27 18:16:10,074][20347] Updated weights for policy 0, policy_version 1822173 (0.0007) [2023-12-27 18:16:10,091][20381] Updated weights for policy 1, policy_version 1820584 (0.0007) [2023-12-27 18:16:10,131][20347] Updated weights for policy 0, policy_version 1822183 (0.0006) [2023-12-27 18:16:10,152][20381] Updated weights for policy 1, policy_version 1820594 (0.0007) [2023-12-27 18:16:10,185][20347] Updated weights for policy 0, policy_version 1822193 (0.0006) [2023-12-27 18:16:10,863][20381] Updated weights for policy 1, policy_version 1820604 (0.0008) [2023-12-27 18:16:10,902][20347] Updated weights for policy 0, policy_version 1822203 (0.0007) [2023-12-27 18:16:10,919][20381] Updated weights for policy 1, policy_version 1820614 (0.0008) [2023-12-27 18:16:10,953][20347] Updated weights for policy 0, policy_version 1822213 (0.0006) [2023-12-27 18:16:10,978][20381] Updated weights for policy 1, policy_version 1820624 (0.0007) [2023-12-27 18:16:11,001][20347] Updated weights for policy 0, policy_version 1822223 (0.0005) [2023-12-27 18:16:11,748][20381] Updated weights for policy 1, policy_version 1820634 (0.0008) [2023-12-27 18:16:11,798][20347] Updated weights for policy 0, policy_version 1822233 (0.0008) [2023-12-27 18:16:11,812][20381] Updated weights for policy 1, policy_version 1820644 (0.0007) [2023-12-27 18:16:11,850][20347] Updated weights for policy 0, policy_version 1822243 (0.0006) [2023-12-27 18:16:11,870][20381] Updated weights for policy 1, policy_version 1820654 (0.0007) [2023-12-27 18:16:11,906][20347] Updated weights for policy 0, policy_version 1822253 (0.0006) [2023-12-27 18:16:11,933][20381] Updated weights for policy 1, policy_version 1820664 (0.0007) [2023-12-27 18:16:11,968][20347] Updated weights for policy 0, policy_version 1822263 (0.0008) [2023-12-27 18:16:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 932724736. Throughput: 0: 10188.2, 1: 10184.3. Samples: 932716180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:12,363][19170] Avg episode reward: [(0, '960.930'), (1, '970.760')] [2023-12-27 18:16:12,635][20381] Updated weights for policy 1, policy_version 1820674 (0.0007) [2023-12-27 18:16:12,693][20381] Updated weights for policy 1, policy_version 1820684 (0.0006) [2023-12-27 18:16:12,727][20347] Updated weights for policy 0, policy_version 1822273 (0.0010) [2023-12-27 18:16:12,754][20381] Updated weights for policy 1, policy_version 1820694 (0.0008) [2023-12-27 18:16:12,783][20347] Updated weights for policy 0, policy_version 1822283 (0.0010) [2023-12-27 18:16:12,831][20347] Updated weights for policy 0, policy_version 1822293 (0.0010) [2023-12-27 18:16:13,332][20381] Updated weights for policy 1, policy_version 1820704 (0.0008) [2023-12-27 18:16:13,381][20381] Updated weights for policy 1, policy_version 1820714 (0.0005) [2023-12-27 18:16:13,435][20381] Updated weights for policy 1, policy_version 1820724 (0.0005) [2023-12-27 18:16:13,499][20347] Updated weights for policy 0, policy_version 1822303 (0.0008) [2023-12-27 18:16:13,549][20347] Updated weights for policy 0, policy_version 1822313 (0.0007) [2023-12-27 18:16:13,610][20347] Updated weights for policy 0, policy_version 1822323 (0.0008) [2023-12-27 18:16:14,138][20381] Updated weights for policy 1, policy_version 1820734 (0.0008) [2023-12-27 18:16:14,186][20381] Updated weights for policy 1, policy_version 1820744 (0.0008) [2023-12-27 18:16:14,221][20347] Updated weights for policy 0, policy_version 1822333 (0.0007) [2023-12-27 18:16:14,239][20381] Updated weights for policy 1, policy_version 1820754 (0.0008) [2023-12-27 18:16:14,274][20347] Updated weights for policy 0, policy_version 1822343 (0.0007) [2023-12-27 18:16:14,330][20347] Updated weights for policy 0, policy_version 1822353 (0.0008) [2023-12-27 18:16:14,977][20381] Updated weights for policy 1, policy_version 1820764 (0.0007) [2023-12-27 18:16:15,015][20347] Updated weights for policy 0, policy_version 1822363 (0.0006) [2023-12-27 18:16:15,034][20381] Updated weights for policy 1, policy_version 1820774 (0.0008) [2023-12-27 18:16:15,070][20347] Updated weights for policy 0, policy_version 1822373 (0.0005) [2023-12-27 18:16:15,086][20381] Updated weights for policy 1, policy_version 1820784 (0.0006) [2023-12-27 18:16:15,133][20347] Updated weights for policy 0, policy_version 1822383 (0.0006) [2023-12-27 18:16:15,725][20347] Updated weights for policy 0, policy_version 1822393 (0.0006) [2023-12-27 18:16:15,784][20347] Updated weights for policy 0, policy_version 1822403 (0.0006) [2023-12-27 18:16:15,811][20381] Updated weights for policy 1, policy_version 1820794 (0.0008) [2023-12-27 18:16:15,847][20347] Updated weights for policy 0, policy_version 1822413 (0.0007) [2023-12-27 18:16:15,874][20381] Updated weights for policy 1, policy_version 1820804 (0.0007) [2023-12-27 18:16:15,903][20347] Updated weights for policy 0, policy_version 1822423 (0.0007) [2023-12-27 18:16:15,934][20381] Updated weights for policy 1, policy_version 1820814 (0.0007) [2023-12-27 18:16:15,997][20381] Updated weights for policy 1, policy_version 1820824 (0.0007) [2023-12-27 18:16:16,524][20347] Updated weights for policy 0, policy_version 1822433 (0.0007) [2023-12-27 18:16:16,578][20347] Updated weights for policy 0, policy_version 1822443 (0.0008) [2023-12-27 18:16:16,641][20347] Updated weights for policy 0, policy_version 1822453 (0.0007) [2023-12-27 18:16:16,689][20381] Updated weights for policy 1, policy_version 1820834 (0.0007) [2023-12-27 18:16:16,744][20381] Updated weights for policy 1, policy_version 1820844 (0.0008) [2023-12-27 18:16:16,796][20381] Updated weights for policy 1, policy_version 1820854 (0.0007) [2023-12-27 18:16:17,276][20347] Updated weights for policy 0, policy_version 1822463 (0.0007) [2023-12-27 18:16:17,328][20347] Updated weights for policy 0, policy_version 1822473 (0.0008) [2023-12-27 18:16:17,362][19170] Fps is (10 sec: 19661.6, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 932823040. Throughput: 0: 10233.6, 1: 10120.2. Samples: 932838520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:17,363][19170] Avg episode reward: [(0, '960.940'), (1, '951.290')] [2023-12-27 18:16:17,385][20347] Updated weights for policy 0, policy_version 1822483 (0.0008) [2023-12-27 18:16:17,485][20381] Updated weights for policy 1, policy_version 1820864 (0.0008) [2023-12-27 18:16:17,543][20381] Updated weights for policy 1, policy_version 1820874 (0.0007) [2023-12-27 18:16:17,605][20381] Updated weights for policy 1, policy_version 1820884 (0.0008) [2023-12-27 18:16:17,969][20347] Updated weights for policy 0, policy_version 1822493 (0.0007) [2023-12-27 18:16:18,016][20347] Updated weights for policy 0, policy_version 1822503 (0.0006) [2023-12-27 18:16:18,070][20347] Updated weights for policy 0, policy_version 1822513 (0.0007) [2023-12-27 18:16:18,357][20381] Updated weights for policy 1, policy_version 1820894 (0.0007) [2023-12-27 18:16:18,413][20381] Updated weights for policy 1, policy_version 1820904 (0.0009) [2023-12-27 18:16:18,469][20381] Updated weights for policy 1, policy_version 1820914 (0.0010) [2023-12-27 18:16:18,668][20347] Updated weights for policy 0, policy_version 1822523 (0.0006) [2023-12-27 18:16:18,735][20347] Updated weights for policy 0, policy_version 1822533 (0.0008) [2023-12-27 18:16:18,794][20347] Updated weights for policy 0, policy_version 1822543 (0.0007) [2023-12-27 18:16:19,140][20381] Updated weights for policy 1, policy_version 1820924 (0.0008) [2023-12-27 18:16:19,187][20381] Updated weights for policy 1, policy_version 1820934 (0.0007) [2023-12-27 18:16:19,250][20381] Updated weights for policy 1, policy_version 1820944 (0.0008) [2023-12-27 18:16:19,407][20347] Updated weights for policy 0, policy_version 1822553 (0.0006) [2023-12-27 18:16:19,463][20347] Updated weights for policy 0, policy_version 1822563 (0.0008) [2023-12-27 18:16:19,525][20347] Updated weights for policy 0, policy_version 1822573 (0.0008) [2023-12-27 18:16:19,582][20347] Updated weights for policy 0, policy_version 1822583 (0.0007) [2023-12-27 18:16:19,919][20381] Updated weights for policy 1, policy_version 1820954 (0.0007) [2023-12-27 18:16:19,985][20381] Updated weights for policy 1, policy_version 1820964 (0.0008) [2023-12-27 18:16:20,045][20381] Updated weights for policy 1, policy_version 1820974 (0.0008) [2023-12-27 18:16:20,102][20381] Updated weights for policy 1, policy_version 1820984 (0.0008) [2023-12-27 18:16:20,310][20347] Updated weights for policy 0, policy_version 1822593 (0.0008) [2023-12-27 18:16:20,371][20347] Updated weights for policy 0, policy_version 1822603 (0.0007) [2023-12-27 18:16:20,430][20347] Updated weights for policy 0, policy_version 1822613 (0.0007) [2023-12-27 18:16:20,867][20381] Updated weights for policy 1, policy_version 1820994 (0.0005) [2023-12-27 18:16:20,931][20381] Updated weights for policy 1, policy_version 1821004 (0.0007) [2023-12-27 18:16:20,996][20381] Updated weights for policy 1, policy_version 1821014 (0.0008) [2023-12-27 18:16:21,296][20347] Updated weights for policy 0, policy_version 1822623 (0.0008) [2023-12-27 18:16:21,361][20347] Updated weights for policy 0, policy_version 1822633 (0.0007) [2023-12-27 18:16:21,426][20347] Updated weights for policy 0, policy_version 1822643 (0.0008) [2023-12-27 18:16:21,739][20381] Updated weights for policy 1, policy_version 1821024 (0.0008) [2023-12-27 18:16:21,802][20381] Updated weights for policy 1, policy_version 1821034 (0.0006) [2023-12-27 18:16:21,865][20381] Updated weights for policy 1, policy_version 1821044 (0.0007) [2023-12-27 18:16:22,133][20347] Updated weights for policy 0, policy_version 1822653 (0.0008) [2023-12-27 18:16:22,199][20347] Updated weights for policy 0, policy_version 1822663 (0.0008) [2023-12-27 18:16:22,267][20347] Updated weights for policy 0, policy_version 1822673 (0.0008) [2023-12-27 18:16:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20577.2). Total num frames: 932929536. Throughput: 0: 10251.9, 1: 10118.5. Samples: 932900364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:22,363][19170] Avg episode reward: [(0, '960.940'), (1, '970.840')] [2023-12-27 18:16:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001821048_466239488.pth... [2023-12-27 18:16:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001822680_466690048.pth... [2023-12-27 18:16:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001819864_465936384.pth [2023-12-27 18:16:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001821464_466378752.pth [2023-12-27 18:16:22,574][20381] Updated weights for policy 1, policy_version 1821054 (0.0008) [2023-12-27 18:16:22,638][20381] Updated weights for policy 1, policy_version 1821064 (0.0008) [2023-12-27 18:16:22,697][20381] Updated weights for policy 1, policy_version 1821074 (0.0008) [2023-12-27 18:16:22,961][20347] Updated weights for policy 0, policy_version 1822683 (0.0008) [2023-12-27 18:16:23,020][20347] Updated weights for policy 0, policy_version 1822693 (0.0008) [2023-12-27 18:16:23,081][20347] Updated weights for policy 0, policy_version 1822703 (0.0006) [2023-12-27 18:16:23,404][20381] Updated weights for policy 1, policy_version 1821084 (0.0008) [2023-12-27 18:16:23,454][20381] Updated weights for policy 1, policy_version 1821094 (0.0007) [2023-12-27 18:16:23,498][20381] Updated weights for policy 1, policy_version 1821104 (0.0007) [2023-12-27 18:16:23,695][20347] Updated weights for policy 0, policy_version 1822713 (0.0005) [2023-12-27 18:16:23,752][20347] Updated weights for policy 0, policy_version 1822723 (0.0007) [2023-12-27 18:16:23,813][20347] Updated weights for policy 0, policy_version 1822733 (0.0007) [2023-12-27 18:16:23,858][20347] Updated weights for policy 0, policy_version 1822743 (0.0005) [2023-12-27 18:16:24,167][20381] Updated weights for policy 1, policy_version 1821114 (0.0008) [2023-12-27 18:16:24,217][20381] Updated weights for policy 1, policy_version 1821124 (0.0007) [2023-12-27 18:16:24,265][20381] Updated weights for policy 1, policy_version 1821134 (0.0005) [2023-12-27 18:16:24,324][20381] Updated weights for policy 1, policy_version 1821144 (0.0005) [2023-12-27 18:16:24,422][20347] Updated weights for policy 0, policy_version 1822753 (0.0005) [2023-12-27 18:16:24,487][20347] Updated weights for policy 0, policy_version 1822763 (0.0006) [2023-12-27 18:16:24,544][20347] Updated weights for policy 0, policy_version 1822773 (0.0006) [2023-12-27 18:16:24,895][20381] Updated weights for policy 1, policy_version 1821154 (0.0009) [2023-12-27 18:16:24,943][20381] Updated weights for policy 1, policy_version 1821164 (0.0008) [2023-12-27 18:16:24,992][20381] Updated weights for policy 1, policy_version 1821174 (0.0007) [2023-12-27 18:16:25,118][20347] Updated weights for policy 0, policy_version 1822783 (0.0005) [2023-12-27 18:16:25,181][20347] Updated weights for policy 0, policy_version 1822793 (0.0007) [2023-12-27 18:16:25,235][20347] Updated weights for policy 0, policy_version 1822803 (0.0007) [2023-12-27 18:16:25,708][20381] Updated weights for policy 1, policy_version 1821184 (0.0008) [2023-12-27 18:16:25,760][20381] Updated weights for policy 1, policy_version 1821194 (0.0008) [2023-12-27 18:16:25,805][20381] Updated weights for policy 1, policy_version 1821204 (0.0007) [2023-12-27 18:16:25,857][20347] Updated weights for policy 0, policy_version 1822813 (0.0006) [2023-12-27 18:16:25,910][20347] Updated weights for policy 0, policy_version 1822823 (0.0008) [2023-12-27 18:16:25,969][20347] Updated weights for policy 0, policy_version 1822834 (0.0007) [2023-12-27 18:16:26,553][20381] Updated weights for policy 1, policy_version 1821214 (0.0008) [2023-12-27 18:16:26,591][20347] Updated weights for policy 0, policy_version 1822844 (0.0005) [2023-12-27 18:16:26,612][20381] Updated weights for policy 1, policy_version 1821224 (0.0008) [2023-12-27 18:16:26,647][20347] Updated weights for policy 0, policy_version 1822854 (0.0005) [2023-12-27 18:16:26,671][20381] Updated weights for policy 1, policy_version 1821234 (0.0007) [2023-12-27 18:16:26,710][20347] Updated weights for policy 0, policy_version 1822864 (0.0006) [2023-12-27 18:16:27,335][20381] Updated weights for policy 1, policy_version 1821244 (0.0007) [2023-12-27 18:16:27,362][20347] Updated weights for policy 0, policy_version 1822874 (0.0008) [2023-12-27 18:16:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 933027840. Throughput: 0: 10250.1, 1: 10161.4. Samples: 933022472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:27,363][19170] Avg episode reward: [(0, '960.890'), (1, '980.560')] [2023-12-27 18:16:27,391][20381] Updated weights for policy 1, policy_version 1821254 (0.0005) [2023-12-27 18:16:27,417][20347] Updated weights for policy 0, policy_version 1822884 (0.0007) [2023-12-27 18:16:27,446][20381] Updated weights for policy 1, policy_version 1821264 (0.0005) [2023-12-27 18:16:27,479][20347] Updated weights for policy 0, policy_version 1822894 (0.0007) [2023-12-27 18:16:27,532][20347] Updated weights for policy 0, policy_version 1822904 (0.0008) [2023-12-27 18:16:27,970][20381] Updated weights for policy 1, policy_version 1821274 (0.0005) [2023-12-27 18:16:28,021][20381] Updated weights for policy 1, policy_version 1821284 (0.0005) [2023-12-27 18:16:28,076][20381] Updated weights for policy 1, policy_version 1821294 (0.0005) [2023-12-27 18:16:28,129][20381] Updated weights for policy 1, policy_version 1821304 (0.0005) [2023-12-27 18:16:28,280][20347] Updated weights for policy 0, policy_version 1822914 (0.0008) [2023-12-27 18:16:28,325][20347] Updated weights for policy 0, policy_version 1822924 (0.0009) [2023-12-27 18:16:28,387][20347] Updated weights for policy 0, policy_version 1822934 (0.0009) [2023-12-27 18:16:28,687][20381] Updated weights for policy 1, policy_version 1821314 (0.0010) [2023-12-27 18:16:28,746][20381] Updated weights for policy 1, policy_version 1821324 (0.0007) [2023-12-27 18:16:28,799][20381] Updated weights for policy 1, policy_version 1821334 (0.0005) [2023-12-27 18:16:29,153][20347] Updated weights for policy 0, policy_version 1822944 (0.0007) [2023-12-27 18:16:29,211][20347] Updated weights for policy 0, policy_version 1822954 (0.0007) [2023-12-27 18:16:29,272][20347] Updated weights for policy 0, policy_version 1822964 (0.0008) [2023-12-27 18:16:29,404][20381] Updated weights for policy 1, policy_version 1821344 (0.0008) [2023-12-27 18:16:29,461][20381] Updated weights for policy 1, policy_version 1821354 (0.0007) [2023-12-27 18:16:29,519][20381] Updated weights for policy 1, policy_version 1821364 (0.0007) [2023-12-27 18:16:30,000][20347] Updated weights for policy 0, policy_version 1822974 (0.0009) [2023-12-27 18:16:30,056][20347] Updated weights for policy 0, policy_version 1822984 (0.0010) [2023-12-27 18:16:30,106][20347] Updated weights for policy 0, policy_version 1822994 (0.0008) [2023-12-27 18:16:30,213][20381] Updated weights for policy 1, policy_version 1821374 (0.0008) [2023-12-27 18:16:30,270][20381] Updated weights for policy 1, policy_version 1821384 (0.0008) [2023-12-27 18:16:30,325][20381] Updated weights for policy 1, policy_version 1821394 (0.0008) [2023-12-27 18:16:30,786][20347] Updated weights for policy 0, policy_version 1823004 (0.0006) [2023-12-27 18:16:30,832][20347] Updated weights for policy 0, policy_version 1823014 (0.0007) [2023-12-27 18:16:30,884][20347] Updated weights for policy 0, policy_version 1823024 (0.0005) [2023-12-27 18:16:30,972][20381] Updated weights for policy 1, policy_version 1821404 (0.0008) [2023-12-27 18:16:31,028][20381] Updated weights for policy 1, policy_version 1821414 (0.0008) [2023-12-27 18:16:31,091][20381] Updated weights for policy 1, policy_version 1821424 (0.0007) [2023-12-27 18:16:31,547][20347] Updated weights for policy 0, policy_version 1823034 (0.0006) [2023-12-27 18:16:31,606][20347] Updated weights for policy 0, policy_version 1823044 (0.0011) [2023-12-27 18:16:31,670][20347] Updated weights for policy 0, policy_version 1823054 (0.0008) [2023-12-27 18:16:31,741][20347] Updated weights for policy 0, policy_version 1823064 (0.0010) [2023-12-27 18:16:31,800][20381] Updated weights for policy 1, policy_version 1821434 (0.0009) [2023-12-27 18:16:31,858][20381] Updated weights for policy 1, policy_version 1821444 (0.0010) [2023-12-27 18:16:31,917][20381] Updated weights for policy 1, policy_version 1821454 (0.0010) [2023-12-27 18:16:31,973][20381] Updated weights for policy 1, policy_version 1821464 (0.0005) [2023-12-27 18:16:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 933134336. Throughput: 0: 10302.4, 1: 10251.0. Samples: 933147032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:32,363][19170] Avg episode reward: [(0, '921.860'), (1, '941.410')] [2023-12-27 18:16:32,424][20347] Updated weights for policy 0, policy_version 1823074 (0.0006) [2023-12-27 18:16:32,481][20347] Updated weights for policy 0, policy_version 1823084 (0.0005) [2023-12-27 18:16:32,535][20347] Updated weights for policy 0, policy_version 1823094 (0.0008) [2023-12-27 18:16:32,628][20381] Updated weights for policy 1, policy_version 1821474 (0.0006) [2023-12-27 18:16:32,687][20381] Updated weights for policy 1, policy_version 1821484 (0.0006) [2023-12-27 18:16:32,705][20292] KL-divergence is very high: 136.5462 [2023-12-27 18:16:32,744][20381] Updated weights for policy 1, policy_version 1821494 (0.0008) [2023-12-27 18:16:32,749][20292] KL-divergence is very high: 136.1155 [2023-12-27 18:16:33,207][20347] Updated weights for policy 0, policy_version 1823104 (0.0010) [2023-12-27 18:16:33,262][20347] Updated weights for policy 0, policy_version 1823114 (0.0010) [2023-12-27 18:16:33,310][20347] Updated weights for policy 0, policy_version 1823124 (0.0007) [2023-12-27 18:16:33,404][20381] Updated weights for policy 1, policy_version 1821504 (0.0006) [2023-12-27 18:16:33,464][20381] Updated weights for policy 1, policy_version 1821514 (0.0005) [2023-12-27 18:16:33,524][20381] Updated weights for policy 1, policy_version 1821524 (0.0005) [2023-12-27 18:16:33,957][20347] Updated weights for policy 0, policy_version 1823134 (0.0007) [2023-12-27 18:16:34,012][20347] Updated weights for policy 0, policy_version 1823144 (0.0005) [2023-12-27 18:16:34,071][20347] Updated weights for policy 0, policy_version 1823154 (0.0005) [2023-12-27 18:16:34,131][20381] Updated weights for policy 1, policy_version 1821534 (0.0010) [2023-12-27 18:16:34,196][20381] Updated weights for policy 1, policy_version 1821544 (0.0011) [2023-12-27 18:16:34,258][20381] Updated weights for policy 1, policy_version 1821554 (0.0007) [2023-12-27 18:16:34,731][20347] Updated weights for policy 0, policy_version 1823164 (0.0007) [2023-12-27 18:16:34,786][20347] Updated weights for policy 0, policy_version 1823174 (0.0008) [2023-12-27 18:16:34,836][20347] Updated weights for policy 0, policy_version 1823184 (0.0009) [2023-12-27 18:16:34,923][20381] Updated weights for policy 1, policy_version 1821564 (0.0006) [2023-12-27 18:16:34,969][20381] Updated weights for policy 1, policy_version 1821574 (0.0008) [2023-12-27 18:16:35,015][20381] Updated weights for policy 1, policy_version 1821584 (0.0009) [2023-12-27 18:16:35,449][20347] Updated weights for policy 0, policy_version 1823194 (0.0007) [2023-12-27 18:16:35,508][20347] Updated weights for policy 0, policy_version 1823204 (0.0007) [2023-12-27 18:16:35,561][20347] Updated weights for policy 0, policy_version 1823214 (0.0005) [2023-12-27 18:16:35,623][20347] Updated weights for policy 0, policy_version 1823224 (0.0006) [2023-12-27 18:16:35,751][20381] Updated weights for policy 1, policy_version 1821594 (0.0007) [2023-12-27 18:16:35,808][20381] Updated weights for policy 1, policy_version 1821604 (0.0008) [2023-12-27 18:16:35,866][20381] Updated weights for policy 1, policy_version 1821614 (0.0006) [2023-12-27 18:16:35,925][20381] Updated weights for policy 1, policy_version 1821624 (0.0005) [2023-12-27 18:16:36,284][20347] Updated weights for policy 0, policy_version 1823234 (0.0006) [2023-12-27 18:16:36,351][20347] Updated weights for policy 0, policy_version 1823244 (0.0005) [2023-12-27 18:16:36,421][20347] Updated weights for policy 0, policy_version 1823254 (0.0006) [2023-12-27 18:16:36,632][20381] Updated weights for policy 1, policy_version 1821634 (0.0007) [2023-12-27 18:16:36,681][20381] Updated weights for policy 1, policy_version 1821644 (0.0007) [2023-12-27 18:16:36,728][20381] Updated weights for policy 1, policy_version 1821654 (0.0005) [2023-12-27 18:16:37,053][20347] Updated weights for policy 0, policy_version 1823264 (0.0006) [2023-12-27 18:16:37,116][20347] Updated weights for policy 0, policy_version 1823274 (0.0006) [2023-12-27 18:16:37,182][20347] Updated weights for policy 0, policy_version 1823284 (0.0007) [2023-12-27 18:16:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 933240832. Throughput: 0: 10344.7, 1: 10247.6. Samples: 933209476. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:37,363][19170] Avg episode reward: [(0, '921.860'), (1, '941.360')] [2023-12-27 18:16:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001823288_466845696.pth... [2023-12-27 18:16:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001822072_466534400.pth [2023-12-27 18:16:37,409][20381] Updated weights for policy 1, policy_version 1821664 (0.0007) [2023-12-27 18:16:37,459][20381] Updated weights for policy 1, policy_version 1821674 (0.0008) [2023-12-27 18:16:37,508][20381] Updated weights for policy 1, policy_version 1821684 (0.0007) [2023-12-27 18:16:37,527][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001821688_466403328.pth... [2023-12-27 18:16:37,532][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001820472_466092032.pth [2023-12-27 18:16:37,824][20347] Updated weights for policy 0, policy_version 1823294 (0.0008) [2023-12-27 18:16:37,877][20347] Updated weights for policy 0, policy_version 1823304 (0.0008) [2023-12-27 18:16:37,925][20347] Updated weights for policy 0, policy_version 1823314 (0.0007) [2023-12-27 18:16:38,247][20381] Updated weights for policy 1, policy_version 1821694 (0.0007) [2023-12-27 18:16:38,311][20381] Updated weights for policy 1, policy_version 1821704 (0.0008) [2023-12-27 18:16:38,375][20381] Updated weights for policy 1, policy_version 1821714 (0.0008) [2023-12-27 18:16:38,621][20347] Updated weights for policy 0, policy_version 1823324 (0.0007) [2023-12-27 18:16:38,670][20347] Updated weights for policy 0, policy_version 1823334 (0.0010) [2023-12-27 18:16:38,722][20347] Updated weights for policy 0, policy_version 1823344 (0.0010) [2023-12-27 18:16:39,093][20381] Updated weights for policy 1, policy_version 1821724 (0.0008) [2023-12-27 18:16:39,154][20381] Updated weights for policy 1, policy_version 1821734 (0.0007) [2023-12-27 18:16:39,219][20381] Updated weights for policy 1, policy_version 1821744 (0.0008) [2023-12-27 18:16:39,345][20347] Updated weights for policy 0, policy_version 1823354 (0.0009) [2023-12-27 18:16:39,409][20347] Updated weights for policy 0, policy_version 1823364 (0.0007) [2023-12-27 18:16:39,475][20347] Updated weights for policy 0, policy_version 1823374 (0.0007) [2023-12-27 18:16:39,539][20347] Updated weights for policy 0, policy_version 1823384 (0.0008) [2023-12-27 18:16:39,995][20381] Updated weights for policy 1, policy_version 1821754 (0.0008) [2023-12-27 18:16:40,059][20381] Updated weights for policy 1, policy_version 1821764 (0.0006) [2023-12-27 18:16:40,123][20381] Updated weights for policy 1, policy_version 1821774 (0.0005) [2023-12-27 18:16:40,175][20381] Updated weights for policy 1, policy_version 1821784 (0.0007) [2023-12-27 18:16:40,268][20347] Updated weights for policy 0, policy_version 1823394 (0.0007) [2023-12-27 18:16:40,320][20347] Updated weights for policy 0, policy_version 1823404 (0.0008) [2023-12-27 18:16:40,378][20347] Updated weights for policy 0, policy_version 1823414 (0.0008) [2023-12-27 18:16:40,825][20381] Updated weights for policy 1, policy_version 1821794 (0.0005) [2023-12-27 18:16:40,882][20381] Updated weights for policy 1, policy_version 1821804 (0.0005) [2023-12-27 18:16:40,944][20381] Updated weights for policy 1, policy_version 1821814 (0.0005) [2023-12-27 18:16:41,048][20347] Updated weights for policy 0, policy_version 1823424 (0.0007) [2023-12-27 18:16:41,111][20347] Updated weights for policy 0, policy_version 1823434 (0.0008) [2023-12-27 18:16:41,182][20347] Updated weights for policy 0, policy_version 1823444 (0.0011) [2023-12-27 18:16:41,611][20381] Updated weights for policy 1, policy_version 1821824 (0.0008) [2023-12-27 18:16:41,667][20381] Updated weights for policy 1, policy_version 1821834 (0.0007) [2023-12-27 18:16:41,723][20381] Updated weights for policy 1, policy_version 1821844 (0.0008) [2023-12-27 18:16:41,892][20347] Updated weights for policy 0, policy_version 1823454 (0.0008) [2023-12-27 18:16:41,952][20347] Updated weights for policy 0, policy_version 1823464 (0.0009) [2023-12-27 18:16:42,009][20347] Updated weights for policy 0, policy_version 1823474 (0.0008) [2023-12-27 18:16:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 933339136. Throughput: 0: 10370.0, 1: 10151.0. Samples: 933331640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:42,363][19170] Avg episode reward: [(0, '970.620'), (1, '960.900')] [2023-12-27 18:16:42,489][20381] Updated weights for policy 1, policy_version 1821854 (0.0009) [2023-12-27 18:16:42,542][20381] Updated weights for policy 1, policy_version 1821864 (0.0011) [2023-12-27 18:16:42,601][20381] Updated weights for policy 1, policy_version 1821874 (0.0011) [2023-12-27 18:16:42,707][20347] Updated weights for policy 0, policy_version 1823484 (0.0007) [2023-12-27 18:16:42,762][20347] Updated weights for policy 0, policy_version 1823494 (0.0010) [2023-12-27 18:16:42,823][20347] Updated weights for policy 0, policy_version 1823504 (0.0010) [2023-12-27 18:16:43,292][20381] Updated weights for policy 1, policy_version 1821884 (0.0011) [2023-12-27 18:16:43,343][20381] Updated weights for policy 1, policy_version 1821894 (0.0010) [2023-12-27 18:16:43,395][20381] Updated weights for policy 1, policy_version 1821904 (0.0011) [2023-12-27 18:16:43,467][20347] Updated weights for policy 0, policy_version 1823514 (0.0010) [2023-12-27 18:16:43,525][20347] Updated weights for policy 0, policy_version 1823524 (0.0011) [2023-12-27 18:16:43,570][20347] Updated weights for policy 0, policy_version 1823534 (0.0010) [2023-12-27 18:16:43,618][20347] Updated weights for policy 0, policy_version 1823544 (0.0010) [2023-12-27 18:16:44,097][20381] Updated weights for policy 1, policy_version 1821914 (0.0010) [2023-12-27 18:16:44,155][20381] Updated weights for policy 1, policy_version 1821924 (0.0008) [2023-12-27 18:16:44,216][20381] Updated weights for policy 1, policy_version 1821934 (0.0008) [2023-12-27 18:16:44,278][20381] Updated weights for policy 1, policy_version 1821944 (0.0006) [2023-12-27 18:16:44,305][20347] Updated weights for policy 0, policy_version 1823554 (0.0010) [2023-12-27 18:16:44,357][20347] Updated weights for policy 0, policy_version 1823564 (0.0010) [2023-12-27 18:16:44,405][20347] Updated weights for policy 0, policy_version 1823574 (0.0010) [2023-12-27 18:16:44,960][20381] Updated weights for policy 1, policy_version 1821954 (0.0008) [2023-12-27 18:16:45,006][20381] Updated weights for policy 1, policy_version 1821964 (0.0008) [2023-12-27 18:16:45,061][20381] Updated weights for policy 1, policy_version 1821974 (0.0008) [2023-12-27 18:16:45,145][20347] Updated weights for policy 0, policy_version 1823584 (0.0010) [2023-12-27 18:16:45,194][20347] Updated weights for policy 0, policy_version 1823594 (0.0010) [2023-12-27 18:16:45,245][20347] Updated weights for policy 0, policy_version 1823604 (0.0011) [2023-12-27 18:16:45,803][20381] Updated weights for policy 1, policy_version 1821984 (0.0007) [2023-12-27 18:16:45,855][20381] Updated weights for policy 1, policy_version 1821994 (0.0007) [2023-12-27 18:16:45,933][20381] Updated weights for policy 1, policy_version 1822004 (0.0007) [2023-12-27 18:16:45,950][20347] Updated weights for policy 0, policy_version 1823614 (0.0011) [2023-12-27 18:16:45,999][20347] Updated weights for policy 0, policy_version 1823624 (0.0010) [2023-12-27 18:16:46,057][20347] Updated weights for policy 0, policy_version 1823634 (0.0010) [2023-12-27 18:16:46,629][20381] Updated weights for policy 1, policy_version 1822014 (0.0007) [2023-12-27 18:16:46,677][20381] Updated weights for policy 1, policy_version 1822024 (0.0007) [2023-12-27 18:16:46,702][20347] Updated weights for policy 0, policy_version 1823644 (0.0007) [2023-12-27 18:16:46,736][20381] Updated weights for policy 1, policy_version 1822034 (0.0006) [2023-12-27 18:16:46,751][20347] Updated weights for policy 0, policy_version 1823654 (0.0007) [2023-12-27 18:16:46,801][20347] Updated weights for policy 0, policy_version 1823664 (0.0006) [2023-12-27 18:16:47,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 933437440. Throughput: 0: 10347.4, 1: 10082.5. Samples: 933450832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:47,363][19170] Avg episode reward: [(0, '951.260'), (1, '960.970')] [2023-12-27 18:16:47,395][20347] Updated weights for policy 0, policy_version 1823674 (0.0008) [2023-12-27 18:16:47,449][20347] Updated weights for policy 0, policy_version 1823684 (0.0008) [2023-12-27 18:16:47,466][20381] Updated weights for policy 1, policy_version 1822044 (0.0007) [2023-12-27 18:16:47,504][20347] Updated weights for policy 0, policy_version 1823694 (0.0006) [2023-12-27 18:16:47,517][20381] Updated weights for policy 1, policy_version 1822054 (0.0005) [2023-12-27 18:16:47,561][20347] Updated weights for policy 0, policy_version 1823704 (0.0008) [2023-12-27 18:16:47,566][20381] Updated weights for policy 1, policy_version 1822064 (0.0005) [2023-12-27 18:16:48,131][20381] Updated weights for policy 1, policy_version 1822074 (0.0006) [2023-12-27 18:16:48,195][20381] Updated weights for policy 1, policy_version 1822084 (0.0005) [2023-12-27 18:16:48,253][20381] Updated weights for policy 1, policy_version 1822094 (0.0006) [2023-12-27 18:16:48,255][20347] Updated weights for policy 0, policy_version 1823714 (0.0005) [2023-12-27 18:16:48,314][20347] Updated weights for policy 0, policy_version 1823724 (0.0005) [2023-12-27 18:16:48,316][20381] Updated weights for policy 1, policy_version 1822104 (0.0006) [2023-12-27 18:16:48,381][20347] Updated weights for policy 0, policy_version 1823734 (0.0008) [2023-12-27 18:16:48,882][20381] Updated weights for policy 1, policy_version 1822114 (0.0009) [2023-12-27 18:16:48,942][20381] Updated weights for policy 1, policy_version 1822124 (0.0008) [2023-12-27 18:16:49,000][20381] Updated weights for policy 1, policy_version 1822134 (0.0008) [2023-12-27 18:16:49,081][20347] Updated weights for policy 0, policy_version 1823744 (0.0008) [2023-12-27 18:16:49,138][20347] Updated weights for policy 0, policy_version 1823754 (0.0008) [2023-12-27 18:16:49,193][20347] Updated weights for policy 0, policy_version 1823764 (0.0007) [2023-12-27 18:16:49,821][20381] Updated weights for policy 1, policy_version 1822144 (0.0010) [2023-12-27 18:16:49,885][20381] Updated weights for policy 1, policy_version 1822154 (0.0009) [2023-12-27 18:16:49,952][20381] Updated weights for policy 1, policy_version 1822164 (0.0008) [2023-12-27 18:16:49,958][20347] Updated weights for policy 0, policy_version 1823774 (0.0007) [2023-12-27 18:16:50,010][20347] Updated weights for policy 0, policy_version 1823784 (0.0007) [2023-12-27 18:16:50,064][20347] Updated weights for policy 0, policy_version 1823794 (0.0006) [2023-12-27 18:16:50,601][20381] Updated weights for policy 1, policy_version 1822174 (0.0008) [2023-12-27 18:16:50,655][20381] Updated weights for policy 1, policy_version 1822184 (0.0005) [2023-12-27 18:16:50,718][20381] Updated weights for policy 1, policy_version 1822194 (0.0007) [2023-12-27 18:16:50,721][20347] Updated weights for policy 0, policy_version 1823804 (0.0006) [2023-12-27 18:16:50,778][20347] Updated weights for policy 0, policy_version 1823814 (0.0006) [2023-12-27 18:16:50,833][20347] Updated weights for policy 0, policy_version 1823824 (0.0006) [2023-12-27 18:16:51,460][20381] Updated weights for policy 1, policy_version 1822204 (0.0007) [2023-12-27 18:16:51,513][20347] Updated weights for policy 0, policy_version 1823834 (0.0007) [2023-12-27 18:16:51,522][20381] Updated weights for policy 1, policy_version 1822214 (0.0006) [2023-12-27 18:16:51,580][20347] Updated weights for policy 0, policy_version 1823844 (0.0008) [2023-12-27 18:16:51,583][20381] Updated weights for policy 1, policy_version 1822224 (0.0006) [2023-12-27 18:16:51,647][20347] Updated weights for policy 0, policy_version 1823854 (0.0008) [2023-12-27 18:16:51,707][20347] Updated weights for policy 0, policy_version 1823864 (0.0007) [2023-12-27 18:16:52,300][20381] Updated weights for policy 1, policy_version 1822234 (0.0009) [2023-12-27 18:16:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 933535744. Throughput: 0: 10325.9, 1: 10100.4. Samples: 933511448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:52,363][19170] Avg episode reward: [(0, '961.060'), (1, '951.230')] [2023-12-27 18:16:52,365][20381] Updated weights for policy 1, policy_version 1822244 (0.0009) [2023-12-27 18:16:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001823864_466993152.pth... [2023-12-27 18:16:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001822680_466690048.pth [2023-12-27 18:16:52,422][20381] Updated weights for policy 1, policy_version 1822254 (0.0008) [2023-12-27 18:16:52,437][20347] Updated weights for policy 0, policy_version 1823874 (0.0005) [2023-12-27 18:16:52,471][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001822264_466550784.pth... [2023-12-27 18:16:52,472][20381] Updated weights for policy 1, policy_version 1822264 (0.0007) [2023-12-27 18:16:52,475][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001821048_466239488.pth [2023-12-27 18:16:52,488][20347] Updated weights for policy 0, policy_version 1823884 (0.0005) [2023-12-27 18:16:52,536][20347] Updated weights for policy 0, policy_version 1823894 (0.0006) [2023-12-27 18:16:53,172][20381] Updated weights for policy 1, policy_version 1822274 (0.0007) [2023-12-27 18:16:53,203][20347] Updated weights for policy 0, policy_version 1823904 (0.0006) [2023-12-27 18:16:53,224][20381] Updated weights for policy 1, policy_version 1822284 (0.0007) [2023-12-27 18:16:53,255][20347] Updated weights for policy 0, policy_version 1823914 (0.0006) [2023-12-27 18:16:53,278][20381] Updated weights for policy 1, policy_version 1822294 (0.0006) [2023-12-27 18:16:53,311][20347] Updated weights for policy 0, policy_version 1823924 (0.0006) [2023-12-27 18:16:53,869][20381] Updated weights for policy 1, policy_version 1822304 (0.0008) [2023-12-27 18:16:53,932][20381] Updated weights for policy 1, policy_version 1822314 (0.0008) [2023-12-27 18:16:53,990][20381] Updated weights for policy 1, policy_version 1822324 (0.0007) [2023-12-27 18:16:54,066][20347] Updated weights for policy 0, policy_version 1823934 (0.0008) [2023-12-27 18:16:54,120][20347] Updated weights for policy 0, policy_version 1823944 (0.0008) [2023-12-27 18:16:54,170][20347] Updated weights for policy 0, policy_version 1823954 (0.0008) [2023-12-27 18:16:54,666][20381] Updated weights for policy 1, policy_version 1822334 (0.0007) [2023-12-27 18:16:54,718][20381] Updated weights for policy 1, policy_version 1822344 (0.0007) [2023-12-27 18:16:54,773][20381] Updated weights for policy 1, policy_version 1822354 (0.0008) [2023-12-27 18:16:54,865][20347] Updated weights for policy 0, policy_version 1823964 (0.0008) [2023-12-27 18:16:54,917][20347] Updated weights for policy 0, policy_version 1823974 (0.0005) [2023-12-27 18:16:54,961][20347] Updated weights for policy 0, policy_version 1823984 (0.0005) [2023-12-27 18:16:55,435][20381] Updated weights for policy 1, policy_version 1822364 (0.0008) [2023-12-27 18:16:55,483][20381] Updated weights for policy 1, policy_version 1822374 (0.0007) [2023-12-27 18:16:55,530][20381] Updated weights for policy 1, policy_version 1822384 (0.0007) [2023-12-27 18:16:55,548][20347] Updated weights for policy 0, policy_version 1823994 (0.0010) [2023-12-27 18:16:55,600][20347] Updated weights for policy 0, policy_version 1824004 (0.0010) [2023-12-27 18:16:55,651][20347] Updated weights for policy 0, policy_version 1824014 (0.0010) [2023-12-27 18:16:55,704][20347] Updated weights for policy 0, policy_version 1824024 (0.0008) [2023-12-27 18:16:56,261][20381] Updated weights for policy 1, policy_version 1822394 (0.0006) [2023-12-27 18:16:56,318][20381] Updated weights for policy 1, policy_version 1822404 (0.0008) [2023-12-27 18:16:56,347][20347] Updated weights for policy 0, policy_version 1824034 (0.0010) [2023-12-27 18:16:56,373][20381] Updated weights for policy 1, policy_version 1822414 (0.0005) [2023-12-27 18:16:56,401][20347] Updated weights for policy 0, policy_version 1824044 (0.0010) [2023-12-27 18:16:56,421][20381] Updated weights for policy 1, policy_version 1822424 (0.0005) [2023-12-27 18:16:56,455][20347] Updated weights for policy 0, policy_version 1824054 (0.0010) [2023-12-27 18:16:57,087][20347] Updated weights for policy 0, policy_version 1824064 (0.0007) [2023-12-27 18:16:57,135][20381] Updated weights for policy 1, policy_version 1822434 (0.0006) [2023-12-27 18:16:57,148][20347] Updated weights for policy 0, policy_version 1824074 (0.0005) [2023-12-27 18:16:57,190][20381] Updated weights for policy 1, policy_version 1822444 (0.0007) [2023-12-27 18:16:57,196][20347] Updated weights for policy 0, policy_version 1824084 (0.0006) [2023-12-27 18:16:57,235][20381] Updated weights for policy 1, policy_version 1822454 (0.0006) [2023-12-27 18:16:57,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 933650432. Throughput: 0: 10308.9, 1: 10123.8. Samples: 933635652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:16:57,364][19170] Avg episode reward: [(0, '951.150'), (1, '931.760')] [2023-12-27 18:16:57,804][20381] Updated weights for policy 1, policy_version 1822464 (0.0005) [2023-12-27 18:16:57,852][20381] Updated weights for policy 1, policy_version 1822474 (0.0006) [2023-12-27 18:16:57,906][20381] Updated weights for policy 1, policy_version 1822484 (0.0010) [2023-12-27 18:16:57,919][20347] Updated weights for policy 0, policy_version 1824094 (0.0005) [2023-12-27 18:16:57,978][20347] Updated weights for policy 0, policy_version 1824104 (0.0007) [2023-12-27 18:16:58,036][20347] Updated weights for policy 0, policy_version 1824114 (0.0008) [2023-12-27 18:16:58,603][20381] Updated weights for policy 1, policy_version 1822494 (0.0011) [2023-12-27 18:16:58,669][20381] Updated weights for policy 1, policy_version 1822504 (0.0011) [2023-12-27 18:16:58,733][20381] Updated weights for policy 1, policy_version 1822514 (0.0013) [2023-12-27 18:16:58,818][20347] Updated weights for policy 0, policy_version 1824124 (0.0007) [2023-12-27 18:16:58,886][20347] Updated weights for policy 0, policy_version 1824134 (0.0009) [2023-12-27 18:16:58,948][20347] Updated weights for policy 0, policy_version 1824144 (0.0008) [2023-12-27 18:16:59,470][20381] Updated weights for policy 1, policy_version 1822524 (0.0007) [2023-12-27 18:16:59,528][20381] Updated weights for policy 1, policy_version 1822534 (0.0007) [2023-12-27 18:16:59,584][20381] Updated weights for policy 1, policy_version 1822544 (0.0006) [2023-12-27 18:16:59,691][20347] Updated weights for policy 0, policy_version 1824154 (0.0007) [2023-12-27 18:16:59,750][20347] Updated weights for policy 0, policy_version 1824164 (0.0005) [2023-12-27 18:16:59,810][20347] Updated weights for policy 0, policy_version 1824174 (0.0005) [2023-12-27 18:16:59,868][20347] Updated weights for policy 0, policy_version 1824184 (0.0008) [2023-12-27 18:17:00,258][20381] Updated weights for policy 1, policy_version 1822554 (0.0009) [2023-12-27 18:17:00,313][20381] Updated weights for policy 1, policy_version 1822564 (0.0010) [2023-12-27 18:17:00,373][20381] Updated weights for policy 1, policy_version 1822574 (0.0008) [2023-12-27 18:17:00,434][20381] Updated weights for policy 1, policy_version 1822584 (0.0008) [2023-12-27 18:17:00,459][20347] Updated weights for policy 0, policy_version 1824194 (0.0010) [2023-12-27 18:17:00,510][20347] Updated weights for policy 0, policy_version 1824204 (0.0010) [2023-12-27 18:17:00,568][20347] Updated weights for policy 0, policy_version 1824214 (0.0010) [2023-12-27 18:17:00,981][20381] Updated weights for policy 1, policy_version 1822594 (0.0006) [2023-12-27 18:17:01,049][20381] Updated weights for policy 1, policy_version 1822604 (0.0006) [2023-12-27 18:17:01,105][20381] Updated weights for policy 1, policy_version 1822614 (0.0008) [2023-12-27 18:17:01,272][20347] Updated weights for policy 0, policy_version 1824224 (0.0007) [2023-12-27 18:17:01,332][20347] Updated weights for policy 0, policy_version 1824234 (0.0008) [2023-12-27 18:17:01,401][20347] Updated weights for policy 0, policy_version 1824244 (0.0008) [2023-12-27 18:17:01,879][20381] Updated weights for policy 1, policy_version 1822624 (0.0008) [2023-12-27 18:17:01,939][20381] Updated weights for policy 1, policy_version 1822634 (0.0008) [2023-12-27 18:17:01,992][20381] Updated weights for policy 1, policy_version 1822644 (0.0007) [2023-12-27 18:17:02,081][20347] Updated weights for policy 0, policy_version 1824254 (0.0008) [2023-12-27 18:17:02,139][20347] Updated weights for policy 0, policy_version 1824264 (0.0008) [2023-12-27 18:17:02,195][20347] Updated weights for policy 0, policy_version 1824274 (0.0007) [2023-12-27 18:17:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 933748736. Throughput: 0: 10251.9, 1: 10149.2. Samples: 933756568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:02,363][19170] Avg episode reward: [(0, '951.140'), (1, '951.200')] [2023-12-27 18:17:02,698][20381] Updated weights for policy 1, policy_version 1822654 (0.0008) [2023-12-27 18:17:02,747][20381] Updated weights for policy 1, policy_version 1822664 (0.0008) [2023-12-27 18:17:02,804][20381] Updated weights for policy 1, policy_version 1822674 (0.0008) [2023-12-27 18:17:02,907][20347] Updated weights for policy 0, policy_version 1824284 (0.0008) [2023-12-27 18:17:02,961][20347] Updated weights for policy 0, policy_version 1824294 (0.0008) [2023-12-27 18:17:03,019][20347] Updated weights for policy 0, policy_version 1824304 (0.0008) [2023-12-27 18:17:03,489][20381] Updated weights for policy 1, policy_version 1822684 (0.0007) [2023-12-27 18:17:03,550][20381] Updated weights for policy 1, policy_version 1822694 (0.0005) [2023-12-27 18:17:03,610][20381] Updated weights for policy 1, policy_version 1822704 (0.0005) [2023-12-27 18:17:03,670][20347] Updated weights for policy 0, policy_version 1824314 (0.0007) [2023-12-27 18:17:03,714][20347] Updated weights for policy 0, policy_version 1824324 (0.0009) [2023-12-27 18:17:03,759][20347] Updated weights for policy 0, policy_version 1824334 (0.0008) [2023-12-27 18:17:03,806][20347] Updated weights for policy 0, policy_version 1824344 (0.0007) [2023-12-27 18:17:04,170][20381] Updated weights for policy 1, policy_version 1822714 (0.0006) [2023-12-27 18:17:04,223][20381] Updated weights for policy 1, policy_version 1822724 (0.0011) [2023-12-27 18:17:04,282][20381] Updated weights for policy 1, policy_version 1822734 (0.0011) [2023-12-27 18:17:04,348][20381] Updated weights for policy 1, policy_version 1822744 (0.0011) [2023-12-27 18:17:04,549][20347] Updated weights for policy 0, policy_version 1824354 (0.0008) [2023-12-27 18:17:04,596][20347] Updated weights for policy 0, policy_version 1824364 (0.0007) [2023-12-27 18:17:04,647][20347] Updated weights for policy 0, policy_version 1824374 (0.0008) [2023-12-27 18:17:05,025][20381] Updated weights for policy 1, policy_version 1822754 (0.0011) [2023-12-27 18:17:05,084][20381] Updated weights for policy 1, policy_version 1822764 (0.0010) [2023-12-27 18:17:05,141][20381] Updated weights for policy 1, policy_version 1822774 (0.0010) [2023-12-27 18:17:05,336][20347] Updated weights for policy 0, policy_version 1824384 (0.0007) [2023-12-27 18:17:05,385][20347] Updated weights for policy 0, policy_version 1824394 (0.0008) [2023-12-27 18:17:05,427][20347] Updated weights for policy 0, policy_version 1824404 (0.0007) [2023-12-27 18:17:05,795][20381] Updated weights for policy 1, policy_version 1822784 (0.0010) [2023-12-27 18:17:05,846][20381] Updated weights for policy 1, policy_version 1822794 (0.0010) [2023-12-27 18:17:05,891][20381] Updated weights for policy 1, policy_version 1822804 (0.0010) [2023-12-27 18:17:06,178][20347] Updated weights for policy 0, policy_version 1824414 (0.0008) [2023-12-27 18:17:06,238][20347] Updated weights for policy 0, policy_version 1824424 (0.0008) [2023-12-27 18:17:06,293][20347] Updated weights for policy 0, policy_version 1824434 (0.0007) [2023-12-27 18:17:06,593][20381] Updated weights for policy 1, policy_version 1822814 (0.0009) [2023-12-27 18:17:06,645][20381] Updated weights for policy 1, policy_version 1822824 (0.0008) [2023-12-27 18:17:06,697][20381] Updated weights for policy 1, policy_version 1822834 (0.0006) [2023-12-27 18:17:07,007][20347] Updated weights for policy 0, policy_version 1824444 (0.0006) [2023-12-27 18:17:07,072][20347] Updated weights for policy 0, policy_version 1824454 (0.0008) [2023-12-27 18:17:07,136][20347] Updated weights for policy 0, policy_version 1824464 (0.0008) [2023-12-27 18:17:07,354][20381] Updated weights for policy 1, policy_version 1822844 (0.0006) [2023-12-27 18:17:07,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 933847040. Throughput: 0: 10198.6, 1: 10190.6. Samples: 933817876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:07,363][19170] Avg episode reward: [(0, '960.840'), (1, '941.390')] [2023-12-27 18:17:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001824472_467148800.pth... [2023-12-27 18:17:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001823288_466845696.pth [2023-12-27 18:17:07,407][20381] Updated weights for policy 1, policy_version 1822854 (0.0007) [2023-12-27 18:17:07,462][20381] Updated weights for policy 1, policy_version 1822864 (0.0008) [2023-12-27 18:17:07,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001822872_466706432.pth... [2023-12-27 18:17:07,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001821688_466403328.pth [2023-12-27 18:17:07,769][20347] Updated weights for policy 0, policy_version 1824474 (0.0008) [2023-12-27 18:17:07,815][20347] Updated weights for policy 0, policy_version 1824484 (0.0005) [2023-12-27 18:17:07,867][20347] Updated weights for policy 0, policy_version 1824494 (0.0005) [2023-12-27 18:17:07,926][20347] Updated weights for policy 0, policy_version 1824504 (0.0005) [2023-12-27 18:17:08,220][20381] Updated weights for policy 1, policy_version 1822874 (0.0008) [2023-12-27 18:17:08,273][20381] Updated weights for policy 1, policy_version 1822884 (0.0008) [2023-12-27 18:17:08,328][20381] Updated weights for policy 1, policy_version 1822894 (0.0008) [2023-12-27 18:17:08,386][20381] Updated weights for policy 1, policy_version 1822904 (0.0007) [2023-12-27 18:17:08,463][20347] Updated weights for policy 0, policy_version 1824514 (0.0008) [2023-12-27 18:17:08,526][20347] Updated weights for policy 0, policy_version 1824524 (0.0008) [2023-12-27 18:17:08,589][20347] Updated weights for policy 0, policy_version 1824534 (0.0008) [2023-12-27 18:17:09,098][20381] Updated weights for policy 1, policy_version 1822914 (0.0006) [2023-12-27 18:17:09,164][20381] Updated weights for policy 1, policy_version 1822924 (0.0006) [2023-12-27 18:17:09,211][20381] Updated weights for policy 1, policy_version 1822934 (0.0007) [2023-12-27 18:17:09,311][20347] Updated weights for policy 0, policy_version 1824544 (0.0007) [2023-12-27 18:17:09,377][20347] Updated weights for policy 0, policy_version 1824554 (0.0009) [2023-12-27 18:17:09,446][20347] Updated weights for policy 0, policy_version 1824564 (0.0008) [2023-12-27 18:17:09,962][20381] Updated weights for policy 1, policy_version 1822944 (0.0007) [2023-12-27 18:17:10,023][20381] Updated weights for policy 1, policy_version 1822954 (0.0007) [2023-12-27 18:17:10,078][20381] Updated weights for policy 1, policy_version 1822964 (0.0007) [2023-12-27 18:17:10,148][20347] Updated weights for policy 0, policy_version 1824574 (0.0008) [2023-12-27 18:17:10,204][20347] Updated weights for policy 0, policy_version 1824584 (0.0008) [2023-12-27 18:17:10,259][20347] Updated weights for policy 0, policy_version 1824594 (0.0008) [2023-12-27 18:17:10,690][20381] Updated weights for policy 1, policy_version 1822974 (0.0006) [2023-12-27 18:17:10,738][20381] Updated weights for policy 1, policy_version 1822984 (0.0005) [2023-12-27 18:17:10,787][20381] Updated weights for policy 1, policy_version 1822994 (0.0007) [2023-12-27 18:17:10,929][20347] Updated weights for policy 0, policy_version 1824604 (0.0008) [2023-12-27 18:17:10,977][20347] Updated weights for policy 0, policy_version 1824614 (0.0007) [2023-12-27 18:17:11,038][20347] Updated weights for policy 0, policy_version 1824624 (0.0009) [2023-12-27 18:17:11,482][20381] Updated weights for policy 1, policy_version 1823004 (0.0008) [2023-12-27 18:17:11,542][20381] Updated weights for policy 1, policy_version 1823014 (0.0008) [2023-12-27 18:17:11,602][20381] Updated weights for policy 1, policy_version 1823024 (0.0008) [2023-12-27 18:17:11,784][20347] Updated weights for policy 0, policy_version 1824634 (0.0008) [2023-12-27 18:17:11,841][20347] Updated weights for policy 0, policy_version 1824644 (0.0007) [2023-12-27 18:17:11,891][20347] Updated weights for policy 0, policy_version 1824654 (0.0007) [2023-12-27 18:17:11,951][20347] Updated weights for policy 0, policy_version 1824664 (0.0007) [2023-12-27 18:17:12,347][20381] Updated weights for policy 1, policy_version 1823034 (0.0008) [2023-12-27 18:17:12,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 933945344. Throughput: 0: 10169.3, 1: 10207.7. Samples: 933939436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:12,363][19170] Avg episode reward: [(0, '951.050'), (1, '931.640')] [2023-12-27 18:17:12,407][20381] Updated weights for policy 1, policy_version 1823044 (0.0007) [2023-12-27 18:17:12,465][20381] Updated weights for policy 1, policy_version 1823054 (0.0007) [2023-12-27 18:17:12,517][20381] Updated weights for policy 1, policy_version 1823064 (0.0005) [2023-12-27 18:17:12,644][20347] Updated weights for policy 0, policy_version 1824674 (0.0008) [2023-12-27 18:17:12,702][20347] Updated weights for policy 0, policy_version 1824684 (0.0009) [2023-12-27 18:17:12,758][20347] Updated weights for policy 0, policy_version 1824694 (0.0009) [2023-12-27 18:17:13,184][20381] Updated weights for policy 1, policy_version 1823074 (0.0011) [2023-12-27 18:17:13,230][20381] Updated weights for policy 1, policy_version 1823084 (0.0010) [2023-12-27 18:17:13,282][20381] Updated weights for policy 1, policy_version 1823094 (0.0011) [2023-12-27 18:17:13,369][20347] Updated weights for policy 0, policy_version 1824704 (0.0007) [2023-12-27 18:17:13,424][20347] Updated weights for policy 0, policy_version 1824714 (0.0007) [2023-12-27 18:17:13,484][20347] Updated weights for policy 0, policy_version 1824724 (0.0009) [2023-12-27 18:17:13,979][20381] Updated weights for policy 1, policy_version 1823104 (0.0008) [2023-12-27 18:17:14,026][20381] Updated weights for policy 1, policy_version 1823114 (0.0007) [2023-12-27 18:17:14,074][20381] Updated weights for policy 1, policy_version 1823124 (0.0007) [2023-12-27 18:17:14,147][20347] Updated weights for policy 0, policy_version 1824734 (0.0010) [2023-12-27 18:17:14,200][20347] Updated weights for policy 0, policy_version 1824744 (0.0010) [2023-12-27 18:17:14,255][20347] Updated weights for policy 0, policy_version 1824754 (0.0010) [2023-12-27 18:17:14,802][20381] Updated weights for policy 1, policy_version 1823134 (0.0007) [2023-12-27 18:17:14,865][20381] Updated weights for policy 1, policy_version 1823144 (0.0008) [2023-12-27 18:17:14,922][20381] Updated weights for policy 1, policy_version 1823154 (0.0011) [2023-12-27 18:17:14,939][20347] Updated weights for policy 0, policy_version 1824764 (0.0010) [2023-12-27 18:17:14,998][20347] Updated weights for policy 0, policy_version 1824774 (0.0010) [2023-12-27 18:17:15,048][20347] Updated weights for policy 0, policy_version 1824784 (0.0010) [2023-12-27 18:17:15,598][20381] Updated weights for policy 1, policy_version 1823164 (0.0010) [2023-12-27 18:17:15,646][20381] Updated weights for policy 1, policy_version 1823174 (0.0008) [2023-12-27 18:17:15,705][20381] Updated weights for policy 1, policy_version 1823184 (0.0006) [2023-12-27 18:17:15,737][20347] Updated weights for policy 0, policy_version 1824794 (0.0010) [2023-12-27 18:17:15,796][20347] Updated weights for policy 0, policy_version 1824804 (0.0009) [2023-12-27 18:17:15,854][20347] Updated weights for policy 0, policy_version 1824814 (0.0010) [2023-12-27 18:17:15,902][20347] Updated weights for policy 0, policy_version 1824824 (0.0010) [2023-12-27 18:17:16,307][20381] Updated weights for policy 1, policy_version 1823194 (0.0010) [2023-12-27 18:17:16,364][20381] Updated weights for policy 1, policy_version 1823204 (0.0007) [2023-12-27 18:17:16,423][20381] Updated weights for policy 1, policy_version 1823214 (0.0007) [2023-12-27 18:17:16,477][20381] Updated weights for policy 1, policy_version 1823224 (0.0006) [2023-12-27 18:17:16,570][20347] Updated weights for policy 0, policy_version 1824834 (0.0010) [2023-12-27 18:17:16,622][20347] Updated weights for policy 0, policy_version 1824844 (0.0007) [2023-12-27 18:17:16,675][20347] Updated weights for policy 0, policy_version 1824854 (0.0006) [2023-12-27 18:17:17,152][20381] Updated weights for policy 1, policy_version 1823234 (0.0008) [2023-12-27 18:17:17,202][20381] Updated weights for policy 1, policy_version 1823244 (0.0007) [2023-12-27 18:17:17,263][20381] Updated weights for policy 1, policy_version 1823254 (0.0007) [2023-12-27 18:17:17,356][20347] Updated weights for policy 0, policy_version 1824864 (0.0009) [2023-12-27 18:17:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 934051840. Throughput: 0: 10195.5, 1: 10120.5. Samples: 934061252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:17,363][19170] Avg episode reward: [(0, '951.160'), (1, '960.730')] [2023-12-27 18:17:17,411][20347] Updated weights for policy 0, policy_version 1824874 (0.0010) [2023-12-27 18:17:17,462][20347] Updated weights for policy 0, policy_version 1824884 (0.0007) [2023-12-27 18:17:17,905][20381] Updated weights for policy 1, policy_version 1823264 (0.0005) [2023-12-27 18:17:17,963][20381] Updated weights for policy 1, policy_version 1823274 (0.0005) [2023-12-27 18:17:18,009][20381] Updated weights for policy 1, policy_version 1823284 (0.0005) [2023-12-27 18:17:18,070][20347] Updated weights for policy 0, policy_version 1824894 (0.0005) [2023-12-27 18:17:18,121][20347] Updated weights for policy 0, policy_version 1824904 (0.0005) [2023-12-27 18:17:18,186][20347] Updated weights for policy 0, policy_version 1824914 (0.0005) [2023-12-27 18:17:18,549][20381] Updated weights for policy 1, policy_version 1823294 (0.0006) [2023-12-27 18:17:18,604][20381] Updated weights for policy 1, policy_version 1823304 (0.0007) [2023-12-27 18:17:18,660][20381] Updated weights for policy 1, policy_version 1823314 (0.0008) [2023-12-27 18:17:18,795][20347] Updated weights for policy 0, policy_version 1824924 (0.0006) [2023-12-27 18:17:18,856][20347] Updated weights for policy 0, policy_version 1824934 (0.0007) [2023-12-27 18:17:18,918][20347] Updated weights for policy 0, policy_version 1824944 (0.0008) [2023-12-27 18:17:19,385][20381] Updated weights for policy 1, policy_version 1823326 (0.0009) [2023-12-27 18:17:19,455][20381] Updated weights for policy 1, policy_version 1823337 (0.0007) [2023-12-27 18:17:19,517][20381] Updated weights for policy 1, policy_version 1823347 (0.0006) [2023-12-27 18:17:19,662][20347] Updated weights for policy 0, policy_version 1824954 (0.0007) [2023-12-27 18:17:19,726][20347] Updated weights for policy 0, policy_version 1824964 (0.0006) [2023-12-27 18:17:19,783][20347] Updated weights for policy 0, policy_version 1824974 (0.0007) [2023-12-27 18:17:19,848][20347] Updated weights for policy 0, policy_version 1824984 (0.0008) [2023-12-27 18:17:20,269][20381] Updated weights for policy 1, policy_version 1823357 (0.0009) [2023-12-27 18:17:20,328][20381] Updated weights for policy 1, policy_version 1823367 (0.0011) [2023-12-27 18:17:20,386][20381] Updated weights for policy 1, policy_version 1823377 (0.0011) [2023-12-27 18:17:20,589][20347] Updated weights for policy 0, policy_version 1824994 (0.0011) [2023-12-27 18:17:20,645][20347] Updated weights for policy 0, policy_version 1825004 (0.0010) [2023-12-27 18:17:20,709][20347] Updated weights for policy 0, policy_version 1825014 (0.0010) [2023-12-27 18:17:21,083][20381] Updated weights for policy 1, policy_version 1823387 (0.0009) [2023-12-27 18:17:21,147][20381] Updated weights for policy 1, policy_version 1823397 (0.0009) [2023-12-27 18:17:21,202][20381] Updated weights for policy 1, policy_version 1823407 (0.0007) [2023-12-27 18:17:21,377][20347] Updated weights for policy 0, policy_version 1825024 (0.0008) [2023-12-27 18:17:21,429][20347] Updated weights for policy 0, policy_version 1825034 (0.0007) [2023-12-27 18:17:21,480][20347] Updated weights for policy 0, policy_version 1825044 (0.0007) [2023-12-27 18:17:21,991][20381] Updated weights for policy 1, policy_version 1823417 (0.0006) [2023-12-27 18:17:22,041][20381] Updated weights for policy 1, policy_version 1823427 (0.0008) [2023-12-27 18:17:22,102][20381] Updated weights for policy 1, policy_version 1823437 (0.0008) [2023-12-27 18:17:22,165][20381] Updated weights for policy 1, policy_version 1823447 (0.0008) [2023-12-27 18:17:22,171][20347] Updated weights for policy 0, policy_version 1825054 (0.0007) [2023-12-27 18:17:22,228][20347] Updated weights for policy 0, policy_version 1825064 (0.0007) [2023-12-27 18:17:22,290][20347] Updated weights for policy 0, policy_version 1825074 (0.0009) [2023-12-27 18:17:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 934158336. Throughput: 0: 10183.1, 1: 10139.9. Samples: 934124012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:22,363][19170] Avg episode reward: [(0, '961.210'), (1, '979.080')] [2023-12-27 18:17:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001823448_466853888.pth... [2023-12-27 18:17:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001825080_467304448.pth... [2023-12-27 18:17:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001822264_466550784.pth [2023-12-27 18:17:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001823864_466993152.pth [2023-12-27 18:17:22,934][20381] Updated weights for policy 1, policy_version 1823457 (0.0007) [2023-12-27 18:17:22,989][20381] Updated weights for policy 1, policy_version 1823467 (0.0007) [2023-12-27 18:17:23,006][20347] Updated weights for policy 0, policy_version 1825084 (0.0009) [2023-12-27 18:17:23,049][20381] Updated weights for policy 1, policy_version 1823477 (0.0007) [2023-12-27 18:17:23,064][20347] Updated weights for policy 0, policy_version 1825094 (0.0010) [2023-12-27 18:17:23,118][20347] Updated weights for policy 0, policy_version 1825104 (0.0010) [2023-12-27 18:17:23,722][20381] Updated weights for policy 1, policy_version 1823487 (0.0006) [2023-12-27 18:17:23,742][20347] Updated weights for policy 0, policy_version 1825114 (0.0010) [2023-12-27 18:17:23,773][20381] Updated weights for policy 1, policy_version 1823497 (0.0006) [2023-12-27 18:17:23,780][20291] KL-divergence is very high: 123.8961 [2023-12-27 18:17:23,797][20347] Updated weights for policy 0, policy_version 1825124 (0.0011) [2023-12-27 18:17:23,823][20291] KL-divergence is very high: 214.6864 [2023-12-27 18:17:23,828][20381] Updated weights for policy 1, policy_version 1823507 (0.0006) [2023-12-27 18:17:23,852][20347] Updated weights for policy 0, policy_version 1825134 (0.0010) [2023-12-27 18:17:23,870][20291] KL-divergence is very high: 200.8441 [2023-12-27 18:17:23,910][20347] Updated weights for policy 0, policy_version 1825144 (0.0010) [2023-12-27 18:17:24,514][20347] Updated weights for policy 0, policy_version 1825154 (0.0008) [2023-12-27 18:17:24,546][20381] Updated weights for policy 1, policy_version 1823517 (0.0009) [2023-12-27 18:17:24,568][20347] Updated weights for policy 0, policy_version 1825164 (0.0006) [2023-12-27 18:17:24,595][20381] Updated weights for policy 1, policy_version 1823527 (0.0008) [2023-12-27 18:17:24,624][20347] Updated weights for policy 0, policy_version 1825174 (0.0007) [2023-12-27 18:17:24,657][20381] Updated weights for policy 1, policy_version 1823537 (0.0006) [2023-12-27 18:17:25,194][20347] Updated weights for policy 0, policy_version 1825184 (0.0006) [2023-12-27 18:17:25,227][20381] Updated weights for policy 1, policy_version 1823547 (0.0007) [2023-12-27 18:17:25,250][20347] Updated weights for policy 0, policy_version 1825194 (0.0005) [2023-12-27 18:17:25,281][20381] Updated weights for policy 1, policy_version 1823557 (0.0006) [2023-12-27 18:17:25,301][20347] Updated weights for policy 0, policy_version 1825204 (0.0005) [2023-12-27 18:17:25,328][20381] Updated weights for policy 1, policy_version 1823567 (0.0007) [2023-12-27 18:17:25,864][20347] Updated weights for policy 0, policy_version 1825214 (0.0008) [2023-12-27 18:17:25,926][20347] Updated weights for policy 0, policy_version 1825224 (0.0008) [2023-12-27 18:17:25,983][20347] Updated weights for policy 0, policy_version 1825234 (0.0008) [2023-12-27 18:17:26,009][20381] Updated weights for policy 1, policy_version 1823577 (0.0007) [2023-12-27 18:17:26,066][20381] Updated weights for policy 1, policy_version 1823587 (0.0005) [2023-12-27 18:17:26,127][20381] Updated weights for policy 1, policy_version 1823597 (0.0007) [2023-12-27 18:17:26,190][20381] Updated weights for policy 1, policy_version 1823607 (0.0007) [2023-12-27 18:17:26,663][20347] Updated weights for policy 0, policy_version 1825244 (0.0008) [2023-12-27 18:17:26,705][20381] Updated weights for policy 1, policy_version 1823617 (0.0006) [2023-12-27 18:17:26,706][20347] Updated weights for policy 0, policy_version 1825254 (0.0007) [2023-12-27 18:17:26,752][20347] Updated weights for policy 0, policy_version 1825264 (0.0006) [2023-12-27 18:17:26,763][20381] Updated weights for policy 1, policy_version 1823627 (0.0005) [2023-12-27 18:17:26,820][20381] Updated weights for policy 1, policy_version 1823637 (0.0005) [2023-12-27 18:17:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 934256640. Throughput: 0: 10223.2, 1: 10138.8. Samples: 934247928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:27,363][19170] Avg episode reward: [(0, '961.150'), (1, '968.860')] [2023-12-27 18:17:27,406][20381] Updated weights for policy 1, policy_version 1823647 (0.0007) [2023-12-27 18:17:27,423][20347] Updated weights for policy 0, policy_version 1825274 (0.0008) [2023-12-27 18:17:27,454][20381] Updated weights for policy 1, policy_version 1823657 (0.0006) [2023-12-27 18:17:27,471][20347] Updated weights for policy 0, policy_version 1825284 (0.0010) [2023-12-27 18:17:27,501][20381] Updated weights for policy 1, policy_version 1823667 (0.0005) [2023-12-27 18:17:27,521][20347] Updated weights for policy 0, policy_version 1825294 (0.0010) [2023-12-27 18:17:27,572][20347] Updated weights for policy 0, policy_version 1825304 (0.0010) [2023-12-27 18:17:28,101][20381] Updated weights for policy 1, policy_version 1823677 (0.0006) [2023-12-27 18:17:28,155][20381] Updated weights for policy 1, policy_version 1823687 (0.0006) [2023-12-27 18:17:28,210][20381] Updated weights for policy 1, policy_version 1823697 (0.0006) [2023-12-27 18:17:28,250][20347] Updated weights for policy 0, policy_version 1825314 (0.0010) [2023-12-27 18:17:28,294][20347] Updated weights for policy 0, policy_version 1825324 (0.0010) [2023-12-27 18:17:28,345][20347] Updated weights for policy 0, policy_version 1825334 (0.0009) [2023-12-27 18:17:28,851][20381] Updated weights for policy 1, policy_version 1823707 (0.0008) [2023-12-27 18:17:28,911][20381] Updated weights for policy 1, policy_version 1823717 (0.0008) [2023-12-27 18:17:28,962][20381] Updated weights for policy 1, policy_version 1823727 (0.0005) [2023-12-27 18:17:29,046][20347] Updated weights for policy 0, policy_version 1825344 (0.0009) [2023-12-27 18:17:29,103][20347] Updated weights for policy 0, policy_version 1825354 (0.0010) [2023-12-27 18:17:29,163][20347] Updated weights for policy 0, policy_version 1825364 (0.0010) [2023-12-27 18:17:29,586][20381] Updated weights for policy 1, policy_version 1823737 (0.0006) [2023-12-27 18:17:29,641][20381] Updated weights for policy 1, policy_version 1823747 (0.0009) [2023-12-27 18:17:29,700][20381] Updated weights for policy 1, policy_version 1823757 (0.0007) [2023-12-27 18:17:29,757][20381] Updated weights for policy 1, policy_version 1823767 (0.0006) [2023-12-27 18:17:29,889][20347] Updated weights for policy 0, policy_version 1825374 (0.0010) [2023-12-27 18:17:29,950][20347] Updated weights for policy 0, policy_version 1825384 (0.0007) [2023-12-27 18:17:30,009][20347] Updated weights for policy 0, policy_version 1825394 (0.0010) [2023-12-27 18:17:30,402][20381] Updated weights for policy 1, policy_version 1823777 (0.0008) [2023-12-27 18:17:30,460][20381] Updated weights for policy 1, policy_version 1823787 (0.0006) [2023-12-27 18:17:30,512][20381] Updated weights for policy 1, policy_version 1823797 (0.0005) [2023-12-27 18:17:30,684][20347] Updated weights for policy 0, policy_version 1825404 (0.0010) [2023-12-27 18:17:30,735][20347] Updated weights for policy 0, policy_version 1825414 (0.0010) [2023-12-27 18:17:30,779][20347] Updated weights for policy 0, policy_version 1825424 (0.0007) [2023-12-27 18:17:31,075][20381] Updated weights for policy 1, policy_version 1823807 (0.0007) [2023-12-27 18:17:31,152][20381] Updated weights for policy 1, policy_version 1823817 (0.0007) [2023-12-27 18:17:31,214][20381] Updated weights for policy 1, policy_version 1823827 (0.0008) [2023-12-27 18:17:31,469][20347] Updated weights for policy 0, policy_version 1825434 (0.0006) [2023-12-27 18:17:31,528][20347] Updated weights for policy 0, policy_version 1825444 (0.0008) [2023-12-27 18:17:31,584][20347] Updated weights for policy 0, policy_version 1825454 (0.0008) [2023-12-27 18:17:31,639][20347] Updated weights for policy 0, policy_version 1825464 (0.0006) [2023-12-27 18:17:31,832][20381] Updated weights for policy 1, policy_version 1823837 (0.0009) [2023-12-27 18:17:31,900][20381] Updated weights for policy 1, policy_version 1823847 (0.0008) [2023-12-27 18:17:31,962][20381] Updated weights for policy 1, policy_version 1823857 (0.0007) [2023-12-27 18:17:32,316][20347] Updated weights for policy 0, policy_version 1825474 (0.0006) [2023-12-27 18:17:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 934363136. Throughput: 0: 10225.0, 1: 10321.7. Samples: 934375432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:32,363][19170] Avg episode reward: [(0, '990.250'), (1, '950.020')] [2023-12-27 18:17:32,375][20347] Updated weights for policy 0, policy_version 1825484 (0.0007) [2023-12-27 18:17:32,433][20347] Updated weights for policy 0, policy_version 1825494 (0.0006) [2023-12-27 18:17:32,729][20381] Updated weights for policy 1, policy_version 1823867 (0.0009) [2023-12-27 18:17:32,787][20381] Updated weights for policy 1, policy_version 1823877 (0.0010) [2023-12-27 18:17:32,840][20381] Updated weights for policy 1, policy_version 1823887 (0.0007) [2023-12-27 18:17:33,000][20347] Updated weights for policy 0, policy_version 1825504 (0.0005) [2023-12-27 18:17:33,060][20347] Updated weights for policy 0, policy_version 1825514 (0.0005) [2023-12-27 18:17:33,120][20347] Updated weights for policy 0, policy_version 1825524 (0.0005) [2023-12-27 18:17:33,408][20381] Updated weights for policy 1, policy_version 1823897 (0.0007) [2023-12-27 18:17:33,454][20381] Updated weights for policy 1, policy_version 1823907 (0.0009) [2023-12-27 18:17:33,498][20381] Updated weights for policy 1, policy_version 1823917 (0.0005) [2023-12-27 18:17:33,546][20381] Updated weights for policy 1, policy_version 1823927 (0.0005) [2023-12-27 18:17:33,707][20347] Updated weights for policy 0, policy_version 1825534 (0.0007) [2023-12-27 18:17:33,764][20347] Updated weights for policy 0, policy_version 1825544 (0.0008) [2023-12-27 18:17:33,818][20347] Updated weights for policy 0, policy_version 1825554 (0.0006) [2023-12-27 18:17:34,182][20381] Updated weights for policy 1, policy_version 1823937 (0.0007) [2023-12-27 18:17:34,228][20381] Updated weights for policy 1, policy_version 1823947 (0.0008) [2023-12-27 18:17:34,281][20381] Updated weights for policy 1, policy_version 1823957 (0.0008) [2023-12-27 18:17:34,537][20347] Updated weights for policy 0, policy_version 1825564 (0.0007) [2023-12-27 18:17:34,599][20347] Updated weights for policy 0, policy_version 1825574 (0.0007) [2023-12-27 18:17:34,653][20347] Updated weights for policy 0, policy_version 1825584 (0.0008) [2023-12-27 18:17:34,984][20381] Updated weights for policy 1, policy_version 1823967 (0.0007) [2023-12-27 18:17:35,032][20381] Updated weights for policy 1, policy_version 1823977 (0.0009) [2023-12-27 18:17:35,077][20381] Updated weights for policy 1, policy_version 1823987 (0.0010) [2023-12-27 18:17:35,316][20347] Updated weights for policy 0, policy_version 1825594 (0.0007) [2023-12-27 18:17:35,367][20347] Updated weights for policy 0, policy_version 1825604 (0.0010) [2023-12-27 18:17:35,421][20347] Updated weights for policy 0, policy_version 1825614 (0.0010) [2023-12-27 18:17:35,477][20347] Updated weights for policy 0, policy_version 1825624 (0.0010) [2023-12-27 18:17:35,752][20381] Updated weights for policy 1, policy_version 1823997 (0.0011) [2023-12-27 18:17:35,807][20381] Updated weights for policy 1, policy_version 1824007 (0.0010) [2023-12-27 18:17:35,865][20381] Updated weights for policy 1, policy_version 1824017 (0.0010) [2023-12-27 18:17:36,079][20347] Updated weights for policy 0, policy_version 1825634 (0.0009) [2023-12-27 18:17:36,137][20347] Updated weights for policy 0, policy_version 1825644 (0.0009) [2023-12-27 18:17:36,193][20347] Updated weights for policy 0, policy_version 1825654 (0.0008) [2023-12-27 18:17:36,532][20381] Updated weights for policy 1, policy_version 1824027 (0.0010) [2023-12-27 18:17:36,598][20381] Updated weights for policy 1, policy_version 1824037 (0.0011) [2023-12-27 18:17:36,661][20381] Updated weights for policy 1, policy_version 1824047 (0.0011) [2023-12-27 18:17:36,963][20347] Updated weights for policy 0, policy_version 1825664 (0.0007) [2023-12-27 18:17:37,020][20347] Updated weights for policy 0, policy_version 1825674 (0.0007) [2023-12-27 18:17:37,080][20347] Updated weights for policy 0, policy_version 1825684 (0.0006) [2023-12-27 18:17:37,353][20381] Updated weights for policy 1, policy_version 1824057 (0.0011) [2023-12-27 18:17:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 934469632. Throughput: 0: 10268.5, 1: 10349.5. Samples: 934439252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.340')] [2023-12-27 18:17:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001825688_467460096.pth... [2023-12-27 18:17:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001824472_467148800.pth [2023-12-27 18:17:37,404][20381] Updated weights for policy 1, policy_version 1824067 (0.0007) [2023-12-27 18:17:37,458][20381] Updated weights for policy 1, policy_version 1824077 (0.0005) [2023-12-27 18:17:37,514][20381] Updated weights for policy 1, policy_version 1824087 (0.0005) [2023-12-27 18:17:37,517][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001824088_467017728.pth... [2023-12-27 18:17:37,520][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001822872_466706432.pth [2023-12-27 18:17:37,701][20347] Updated weights for policy 0, policy_version 1825694 (0.0007) [2023-12-27 18:17:37,749][20347] Updated weights for policy 0, policy_version 1825704 (0.0007) [2023-12-27 18:17:37,800][20347] Updated weights for policy 0, policy_version 1825714 (0.0007) [2023-12-27 18:17:38,121][20381] Updated weights for policy 1, policy_version 1824097 (0.0005) [2023-12-27 18:17:38,175][20381] Updated weights for policy 1, policy_version 1824107 (0.0010) [2023-12-27 18:17:38,231][20381] Updated weights for policy 1, policy_version 1824117 (0.0010) [2023-12-27 18:17:38,500][20347] Updated weights for policy 0, policy_version 1825724 (0.0008) [2023-12-27 18:17:38,564][20347] Updated weights for policy 0, policy_version 1825734 (0.0011) [2023-12-27 18:17:38,621][20347] Updated weights for policy 0, policy_version 1825744 (0.0010) [2023-12-27 18:17:38,902][20381] Updated weights for policy 1, policy_version 1824127 (0.0008) [2023-12-27 18:17:38,959][20381] Updated weights for policy 1, policy_version 1824137 (0.0007) [2023-12-27 18:17:39,014][20381] Updated weights for policy 1, policy_version 1824147 (0.0007) [2023-12-27 18:17:39,286][20347] Updated weights for policy 0, policy_version 1825754 (0.0011) [2023-12-27 18:17:39,342][20347] Updated weights for policy 0, policy_version 1825764 (0.0009) [2023-12-27 18:17:39,413][20347] Updated weights for policy 0, policy_version 1825774 (0.0009) [2023-12-27 18:17:39,473][20347] Updated weights for policy 0, policy_version 1825784 (0.0007) [2023-12-27 18:17:39,774][20381] Updated weights for policy 1, policy_version 1824157 (0.0006) [2023-12-27 18:17:39,842][20381] Updated weights for policy 1, policy_version 1824167 (0.0008) [2023-12-27 18:17:39,910][20381] Updated weights for policy 1, policy_version 1824177 (0.0011) [2023-12-27 18:17:40,183][20347] Updated weights for policy 0, policy_version 1825794 (0.0011) [2023-12-27 18:17:40,251][20347] Updated weights for policy 0, policy_version 1825804 (0.0011) [2023-12-27 18:17:40,313][20347] Updated weights for policy 0, policy_version 1825814 (0.0011) [2023-12-27 18:17:40,614][20381] Updated weights for policy 1, policy_version 1824187 (0.0009) [2023-12-27 18:17:40,668][20381] Updated weights for policy 1, policy_version 1824197 (0.0008) [2023-12-27 18:17:40,720][20381] Updated weights for policy 1, policy_version 1824207 (0.0006) [2023-12-27 18:17:40,955][20347] Updated weights for policy 0, policy_version 1825824 (0.0007) [2023-12-27 18:17:41,012][20347] Updated weights for policy 0, policy_version 1825834 (0.0005) [2023-12-27 18:17:41,079][20347] Updated weights for policy 0, policy_version 1825844 (0.0007) [2023-12-27 18:17:41,449][20381] Updated weights for policy 1, policy_version 1824217 (0.0006) [2023-12-27 18:17:41,512][20381] Updated weights for policy 1, policy_version 1824227 (0.0009) [2023-12-27 18:17:41,578][20381] Updated weights for policy 1, policy_version 1824237 (0.0008) [2023-12-27 18:17:41,645][20381] Updated weights for policy 1, policy_version 1824247 (0.0011) [2023-12-27 18:17:41,783][20347] Updated weights for policy 0, policy_version 1825854 (0.0009) [2023-12-27 18:17:41,846][20347] Updated weights for policy 0, policy_version 1825864 (0.0009) [2023-12-27 18:17:41,905][20347] Updated weights for policy 0, policy_version 1825874 (0.0010) [2023-12-27 18:17:42,356][20381] Updated weights for policy 1, policy_version 1824257 (0.0011) [2023-12-27 18:17:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 934567936. Throughput: 0: 10264.9, 1: 10323.5. Samples: 934562128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:42,363][19170] Avg episode reward: [(0, '990.280'), (1, '970.570')] [2023-12-27 18:17:42,420][20381] Updated weights for policy 1, policy_version 1824267 (0.0008) [2023-12-27 18:17:42,480][20381] Updated weights for policy 1, policy_version 1824277 (0.0008) [2023-12-27 18:17:42,602][20347] Updated weights for policy 0, policy_version 1825884 (0.0006) [2023-12-27 18:17:42,651][20347] Updated weights for policy 0, policy_version 1825894 (0.0007) [2023-12-27 18:17:42,703][20347] Updated weights for policy 0, policy_version 1825904 (0.0008) [2023-12-27 18:17:43,165][20381] Updated weights for policy 1, policy_version 1824287 (0.0011) [2023-12-27 18:17:43,221][20381] Updated weights for policy 1, policy_version 1824297 (0.0010) [2023-12-27 18:17:43,278][20381] Updated weights for policy 1, policy_version 1824307 (0.0010) [2023-12-27 18:17:43,404][20347] Updated weights for policy 0, policy_version 1825914 (0.0007) [2023-12-27 18:17:43,451][20347] Updated weights for policy 0, policy_version 1825924 (0.0007) [2023-12-27 18:17:43,494][20347] Updated weights for policy 0, policy_version 1825934 (0.0007) [2023-12-27 18:17:43,554][20347] Updated weights for policy 0, policy_version 1825944 (0.0007) [2023-12-27 18:17:43,966][20381] Updated weights for policy 1, policy_version 1824317 (0.0011) [2023-12-27 18:17:44,025][20381] Updated weights for policy 1, policy_version 1824327 (0.0009) [2023-12-27 18:17:44,080][20381] Updated weights for policy 1, policy_version 1824337 (0.0005) [2023-12-27 18:17:44,158][20347] Updated weights for policy 0, policy_version 1825954 (0.0006) [2023-12-27 18:17:44,204][20347] Updated weights for policy 0, policy_version 1825964 (0.0007) [2023-12-27 18:17:44,258][20347] Updated weights for policy 0, policy_version 1825974 (0.0007) [2023-12-27 18:17:44,751][20381] Updated weights for policy 1, policy_version 1824347 (0.0010) [2023-12-27 18:17:44,815][20381] Updated weights for policy 1, policy_version 1824357 (0.0009) [2023-12-27 18:17:44,869][20381] Updated weights for policy 1, policy_version 1824367 (0.0011) [2023-12-27 18:17:44,891][20347] Updated weights for policy 0, policy_version 1825984 (0.0008) [2023-12-27 18:17:44,951][20347] Updated weights for policy 0, policy_version 1825994 (0.0006) [2023-12-27 18:17:45,017][20347] Updated weights for policy 0, policy_version 1826004 (0.0007) [2023-12-27 18:17:45,555][20381] Updated weights for policy 1, policy_version 1824377 (0.0011) [2023-12-27 18:17:45,606][20381] Updated weights for policy 1, policy_version 1824387 (0.0009) [2023-12-27 18:17:45,662][20381] Updated weights for policy 1, policy_version 1824397 (0.0009) [2023-12-27 18:17:45,671][20347] Updated weights for policy 0, policy_version 1826014 (0.0006) [2023-12-27 18:17:45,717][20381] Updated weights for policy 1, policy_version 1824407 (0.0008) [2023-12-27 18:17:45,718][20347] Updated weights for policy 0, policy_version 1826024 (0.0007) [2023-12-27 18:17:45,773][20347] Updated weights for policy 0, policy_version 1826034 (0.0008) [2023-12-27 18:17:46,338][20381] Updated weights for policy 1, policy_version 1824417 (0.0006) [2023-12-27 18:17:46,387][20381] Updated weights for policy 1, policy_version 1824427 (0.0005) [2023-12-27 18:17:46,437][20381] Updated weights for policy 1, policy_version 1824437 (0.0006) [2023-12-27 18:17:46,470][20347] Updated weights for policy 0, policy_version 1826044 (0.0007) [2023-12-27 18:17:46,530][20347] Updated weights for policy 0, policy_version 1826054 (0.0007) [2023-12-27 18:17:46,578][20347] Updated weights for policy 0, policy_version 1826064 (0.0007) [2023-12-27 18:17:47,104][20381] Updated weights for policy 1, policy_version 1824447 (0.0008) [2023-12-27 18:17:47,158][20381] Updated weights for policy 1, policy_version 1824457 (0.0008) [2023-12-27 18:17:47,215][20381] Updated weights for policy 1, policy_version 1824467 (0.0008) [2023-12-27 18:17:47,245][20347] Updated weights for policy 0, policy_version 1826074 (0.0008) [2023-12-27 18:17:47,299][20347] Updated weights for policy 0, policy_version 1826084 (0.0007) [2023-12-27 18:17:47,350][20347] Updated weights for policy 0, policy_version 1826094 (0.0006) [2023-12-27 18:17:47,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 934674432. Throughput: 0: 10313.0, 1: 10307.2. Samples: 934684484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:47,364][19170] Avg episode reward: [(0, '980.560'), (1, '931.670')] [2023-12-27 18:17:47,394][20347] Updated weights for policy 0, policy_version 1826104 (0.0005) [2023-12-27 18:17:47,856][20381] Updated weights for policy 1, policy_version 1824477 (0.0007) [2023-12-27 18:17:47,928][20381] Updated weights for policy 1, policy_version 1824487 (0.0005) [2023-12-27 18:17:47,987][20381] Updated weights for policy 1, policy_version 1824497 (0.0006) [2023-12-27 18:17:48,003][20347] Updated weights for policy 0, policy_version 1826114 (0.0009) [2023-12-27 18:17:48,056][20347] Updated weights for policy 0, policy_version 1826124 (0.0007) [2023-12-27 18:17:48,110][20347] Updated weights for policy 0, policy_version 1826134 (0.0007) [2023-12-27 18:17:48,582][20381] Updated weights for policy 1, policy_version 1824507 (0.0007) [2023-12-27 18:17:48,639][20381] Updated weights for policy 1, policy_version 1824517 (0.0008) [2023-12-27 18:17:48,695][20381] Updated weights for policy 1, policy_version 1824527 (0.0008) [2023-12-27 18:17:48,763][20347] Updated weights for policy 0, policy_version 1826144 (0.0008) [2023-12-27 18:17:48,819][20347] Updated weights for policy 0, policy_version 1826154 (0.0008) [2023-12-27 18:17:48,871][20347] Updated weights for policy 0, policy_version 1826164 (0.0007) [2023-12-27 18:17:49,404][20381] Updated weights for policy 1, policy_version 1824537 (0.0006) [2023-12-27 18:17:49,454][20381] Updated weights for policy 1, policy_version 1824547 (0.0011) [2023-12-27 18:17:49,509][20381] Updated weights for policy 1, policy_version 1824557 (0.0010) [2023-12-27 18:17:49,570][20381] Updated weights for policy 1, policy_version 1824567 (0.0008) [2023-12-27 18:17:49,596][20347] Updated weights for policy 0, policy_version 1826174 (0.0008) [2023-12-27 18:17:49,648][20347] Updated weights for policy 0, policy_version 1826184 (0.0008) [2023-12-27 18:17:49,704][20347] Updated weights for policy 0, policy_version 1826194 (0.0008) [2023-12-27 18:17:50,252][20381] Updated weights for policy 1, policy_version 1824577 (0.0008) [2023-12-27 18:17:50,301][20381] Updated weights for policy 1, policy_version 1824587 (0.0007) [2023-12-27 18:17:50,359][20381] Updated weights for policy 1, policy_version 1824597 (0.0008) [2023-12-27 18:17:50,468][20347] Updated weights for policy 0, policy_version 1826204 (0.0006) [2023-12-27 18:17:50,516][20347] Updated weights for policy 0, policy_version 1826214 (0.0008) [2023-12-27 18:17:50,560][20347] Updated weights for policy 0, policy_version 1826224 (0.0007) [2023-12-27 18:17:51,059][20381] Updated weights for policy 1, policy_version 1824607 (0.0007) [2023-12-27 18:17:51,124][20381] Updated weights for policy 1, policy_version 1824617 (0.0008) [2023-12-27 18:17:51,186][20381] Updated weights for policy 1, policy_version 1824627 (0.0006) [2023-12-27 18:17:51,338][20347] Updated weights for policy 0, policy_version 1826234 (0.0008) [2023-12-27 18:17:51,410][20347] Updated weights for policy 0, policy_version 1826244 (0.0008) [2023-12-27 18:17:51,466][20347] Updated weights for policy 0, policy_version 1826254 (0.0007) [2023-12-27 18:17:51,524][20347] Updated weights for policy 0, policy_version 1826264 (0.0008) [2023-12-27 18:17:51,867][20381] Updated weights for policy 1, policy_version 1824637 (0.0006) [2023-12-27 18:17:51,922][20381] Updated weights for policy 1, policy_version 1824647 (0.0009) [2023-12-27 18:17:51,986][20381] Updated weights for policy 1, policy_version 1824657 (0.0007) [2023-12-27 18:17:52,164][20347] Updated weights for policy 0, policy_version 1826274 (0.0006) [2023-12-27 18:17:52,219][20347] Updated weights for policy 0, policy_version 1826284 (0.0005) [2023-12-27 18:17:52,282][20347] Updated weights for policy 0, policy_version 1826294 (0.0006) [2023-12-27 18:17:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 934780928. Throughput: 0: 10333.3, 1: 10317.2. Samples: 934747148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:52,363][19170] Avg episode reward: [(0, '990.280'), (1, '921.970')] [2023-12-27 18:17:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001826296_467615744.pth... [2023-12-27 18:17:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001824664_467165184.pth... [2023-12-27 18:17:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001823448_466853888.pth [2023-12-27 18:17:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001825080_467304448.pth [2023-12-27 18:17:52,659][20381] Updated weights for policy 1, policy_version 1824667 (0.0007) [2023-12-27 18:17:52,713][20381] Updated weights for policy 1, policy_version 1824677 (0.0006) [2023-12-27 18:17:52,771][20381] Updated weights for policy 1, policy_version 1824687 (0.0007) [2023-12-27 18:17:52,912][20347] Updated weights for policy 0, policy_version 1826304 (0.0007) [2023-12-27 18:17:52,973][20347] Updated weights for policy 0, policy_version 1826314 (0.0007) [2023-12-27 18:17:53,035][20347] Updated weights for policy 0, policy_version 1826324 (0.0008) [2023-12-27 18:17:53,468][20381] Updated weights for policy 1, policy_version 1824697 (0.0007) [2023-12-27 18:17:53,517][20381] Updated weights for policy 1, policy_version 1824707 (0.0007) [2023-12-27 18:17:53,564][20381] Updated weights for policy 1, policy_version 1824717 (0.0008) [2023-12-27 18:17:53,622][20381] Updated weights for policy 1, policy_version 1824727 (0.0008) [2023-12-27 18:17:53,675][20347] Updated weights for policy 0, policy_version 1826334 (0.0008) [2023-12-27 18:17:53,730][20347] Updated weights for policy 0, policy_version 1826344 (0.0008) [2023-12-27 18:17:53,789][20347] Updated weights for policy 0, policy_version 1826354 (0.0008) [2023-12-27 18:17:54,295][20381] Updated weights for policy 1, policy_version 1824737 (0.0008) [2023-12-27 18:17:54,346][20381] Updated weights for policy 1, policy_version 1824747 (0.0007) [2023-12-27 18:17:54,399][20381] Updated weights for policy 1, policy_version 1824757 (0.0008) [2023-12-27 18:17:54,474][20347] Updated weights for policy 0, policy_version 1826364 (0.0007) [2023-12-27 18:17:54,529][20347] Updated weights for policy 0, policy_version 1826374 (0.0005) [2023-12-27 18:17:54,588][20347] Updated weights for policy 0, policy_version 1826384 (0.0005) [2023-12-27 18:17:55,115][20381] Updated weights for policy 1, policy_version 1824767 (0.0007) [2023-12-27 18:17:55,170][20381] Updated weights for policy 1, policy_version 1824777 (0.0008) [2023-12-27 18:17:55,201][20347] Updated weights for policy 0, policy_version 1826394 (0.0006) [2023-12-27 18:17:55,226][20381] Updated weights for policy 1, policy_version 1824787 (0.0007) [2023-12-27 18:17:55,261][20347] Updated weights for policy 0, policy_version 1826404 (0.0008) [2023-12-27 18:17:55,305][20347] Updated weights for policy 0, policy_version 1826414 (0.0007) [2023-12-27 18:17:55,356][20347] Updated weights for policy 0, policy_version 1826424 (0.0008) [2023-12-27 18:17:55,864][20381] Updated weights for policy 1, policy_version 1824797 (0.0006) [2023-12-27 18:17:55,919][20381] Updated weights for policy 1, policy_version 1824807 (0.0007) [2023-12-27 18:17:55,971][20381] Updated weights for policy 1, policy_version 1824817 (0.0007) [2023-12-27 18:17:56,051][20347] Updated weights for policy 0, policy_version 1826434 (0.0007) [2023-12-27 18:17:56,107][20347] Updated weights for policy 0, policy_version 1826444 (0.0007) [2023-12-27 18:17:56,165][20347] Updated weights for policy 0, policy_version 1826454 (0.0007) [2023-12-27 18:17:56,658][20381] Updated weights for policy 1, policy_version 1824827 (0.0007) [2023-12-27 18:17:56,716][20381] Updated weights for policy 1, policy_version 1824837 (0.0005) [2023-12-27 18:17:56,768][20381] Updated weights for policy 1, policy_version 1824847 (0.0007) [2023-12-27 18:17:56,808][20347] Updated weights for policy 0, policy_version 1826464 (0.0009) [2023-12-27 18:17:56,859][20347] Updated weights for policy 0, policy_version 1826474 (0.0007) [2023-12-27 18:17:56,903][20347] Updated weights for policy 0, policy_version 1826484 (0.0008) [2023-12-27 18:17:57,344][20381] Updated weights for policy 1, policy_version 1824857 (0.0008) [2023-12-27 18:17:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 934879232. Throughput: 0: 10356.5, 1: 10323.9. Samples: 934870056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:17:57,363][19170] Avg episode reward: [(0, '980.480'), (1, '931.780')] [2023-12-27 18:17:57,408][20381] Updated weights for policy 1, policy_version 1824867 (0.0005) [2023-12-27 18:17:57,463][20381] Updated weights for policy 1, policy_version 1824877 (0.0005) [2023-12-27 18:17:57,513][20381] Updated weights for policy 1, policy_version 1824887 (0.0005) [2023-12-27 18:17:57,571][20347] Updated weights for policy 0, policy_version 1826494 (0.0008) [2023-12-27 18:17:57,619][20347] Updated weights for policy 0, policy_version 1826504 (0.0007) [2023-12-27 18:17:57,670][20347] Updated weights for policy 0, policy_version 1826514 (0.0007) [2023-12-27 18:17:58,060][20381] Updated weights for policy 1, policy_version 1824897 (0.0007) [2023-12-27 18:17:58,122][20381] Updated weights for policy 1, policy_version 1824907 (0.0008) [2023-12-27 18:17:58,177][20381] Updated weights for policy 1, policy_version 1824917 (0.0005) [2023-12-27 18:17:58,399][20347] Updated weights for policy 0, policy_version 1826524 (0.0008) [2023-12-27 18:17:58,461][20347] Updated weights for policy 0, policy_version 1826534 (0.0010) [2023-12-27 18:17:58,527][20347] Updated weights for policy 0, policy_version 1826544 (0.0009) [2023-12-27 18:17:58,926][20381] Updated weights for policy 1, policy_version 1824927 (0.0009) [2023-12-27 18:17:58,986][20381] Updated weights for policy 1, policy_version 1824937 (0.0008) [2023-12-27 18:17:59,043][20381] Updated weights for policy 1, policy_version 1824947 (0.0007) [2023-12-27 18:17:59,246][20347] Updated weights for policy 0, policy_version 1826554 (0.0008) [2023-12-27 18:17:59,305][20347] Updated weights for policy 0, policy_version 1826564 (0.0008) [2023-12-27 18:17:59,368][20347] Updated weights for policy 0, policy_version 1826574 (0.0008) [2023-12-27 18:17:59,421][20347] Updated weights for policy 0, policy_version 1826584 (0.0010) [2023-12-27 18:17:59,781][20381] Updated weights for policy 1, policy_version 1824957 (0.0006) [2023-12-27 18:17:59,847][20381] Updated weights for policy 1, policy_version 1824967 (0.0008) [2023-12-27 18:17:59,902][20381] Updated weights for policy 1, policy_version 1824977 (0.0008) [2023-12-27 18:18:00,142][20347] Updated weights for policy 0, policy_version 1826594 (0.0010) [2023-12-27 18:18:00,197][20347] Updated weights for policy 0, policy_version 1826604 (0.0010) [2023-12-27 18:18:00,248][20347] Updated weights for policy 0, policy_version 1826614 (0.0010) [2023-12-27 18:18:00,619][20381] Updated weights for policy 1, policy_version 1824987 (0.0008) [2023-12-27 18:18:00,679][20381] Updated weights for policy 1, policy_version 1824997 (0.0008) [2023-12-27 18:18:00,724][20381] Updated weights for policy 1, policy_version 1825007 (0.0009) [2023-12-27 18:18:00,869][20347] Updated weights for policy 0, policy_version 1826624 (0.0006) [2023-12-27 18:18:00,933][20347] Updated weights for policy 0, policy_version 1826634 (0.0007) [2023-12-27 18:18:00,987][20347] Updated weights for policy 0, policy_version 1826644 (0.0010) [2023-12-27 18:18:01,471][20381] Updated weights for policy 1, policy_version 1825017 (0.0007) [2023-12-27 18:18:01,526][20381] Updated weights for policy 1, policy_version 1825027 (0.0007) [2023-12-27 18:18:01,583][20381] Updated weights for policy 1, policy_version 1825037 (0.0007) [2023-12-27 18:18:01,648][20381] Updated weights for policy 1, policy_version 1825047 (0.0007) [2023-12-27 18:18:01,673][20347] Updated weights for policy 0, policy_version 1826654 (0.0010) [2023-12-27 18:18:01,733][20347] Updated weights for policy 0, policy_version 1826664 (0.0009) [2023-12-27 18:18:01,793][20347] Updated weights for policy 0, policy_version 1826674 (0.0005) [2023-12-27 18:18:02,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 934977536. Throughput: 0: 10357.1, 1: 10322.0. Samples: 934991808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:02,363][19170] Avg episode reward: [(0, '980.480'), (1, '951.360')] [2023-12-27 18:18:02,391][20381] Updated weights for policy 1, policy_version 1825057 (0.0008) [2023-12-27 18:18:02,434][20347] Updated weights for policy 0, policy_version 1826684 (0.0006) [2023-12-27 18:18:02,450][20381] Updated weights for policy 1, policy_version 1825067 (0.0006) [2023-12-27 18:18:02,486][20347] Updated weights for policy 0, policy_version 1826694 (0.0008) [2023-12-27 18:18:02,501][20381] Updated weights for policy 1, policy_version 1825077 (0.0007) [2023-12-27 18:18:02,538][20347] Updated weights for policy 0, policy_version 1826704 (0.0008) [2023-12-27 18:18:03,176][20381] Updated weights for policy 1, policy_version 1825087 (0.0008) [2023-12-27 18:18:03,224][20347] Updated weights for policy 0, policy_version 1826714 (0.0008) [2023-12-27 18:18:03,233][20381] Updated weights for policy 1, policy_version 1825097 (0.0007) [2023-12-27 18:18:03,282][20347] Updated weights for policy 0, policy_version 1826724 (0.0007) [2023-12-27 18:18:03,294][20381] Updated weights for policy 1, policy_version 1825107 (0.0006) [2023-12-27 18:18:03,330][20347] Updated weights for policy 0, policy_version 1826734 (0.0006) [2023-12-27 18:18:03,378][20347] Updated weights for policy 0, policy_version 1826744 (0.0009) [2023-12-27 18:18:03,893][20381] Updated weights for policy 1, policy_version 1825117 (0.0007) [2023-12-27 18:18:03,951][20381] Updated weights for policy 1, policy_version 1825127 (0.0008) [2023-12-27 18:18:04,007][20381] Updated weights for policy 1, policy_version 1825137 (0.0008) [2023-12-27 18:18:04,029][20347] Updated weights for policy 0, policy_version 1826754 (0.0010) [2023-12-27 18:18:04,076][20347] Updated weights for policy 0, policy_version 1826764 (0.0010) [2023-12-27 18:18:04,125][20347] Updated weights for policy 0, policy_version 1826774 (0.0010) [2023-12-27 18:18:04,649][20381] Updated weights for policy 1, policy_version 1825147 (0.0008) [2023-12-27 18:18:04,701][20381] Updated weights for policy 1, policy_version 1825157 (0.0008) [2023-12-27 18:18:04,755][20381] Updated weights for policy 1, policy_version 1825167 (0.0007) [2023-12-27 18:18:04,828][20347] Updated weights for policy 0, policy_version 1826784 (0.0010) [2023-12-27 18:18:04,878][20347] Updated weights for policy 0, policy_version 1826794 (0.0010) [2023-12-27 18:18:04,930][20347] Updated weights for policy 0, policy_version 1826804 (0.0010) [2023-12-27 18:18:05,411][20381] Updated weights for policy 1, policy_version 1825177 (0.0008) [2023-12-27 18:18:05,459][20381] Updated weights for policy 1, policy_version 1825187 (0.0007) [2023-12-27 18:18:05,510][20381] Updated weights for policy 1, policy_version 1825197 (0.0008) [2023-12-27 18:18:05,531][20347] Updated weights for policy 0, policy_version 1826814 (0.0009) [2023-12-27 18:18:05,562][20381] Updated weights for policy 1, policy_version 1825207 (0.0006) [2023-12-27 18:18:05,585][20347] Updated weights for policy 0, policy_version 1826824 (0.0010) [2023-12-27 18:18:05,642][20347] Updated weights for policy 0, policy_version 1826834 (0.0010) [2023-12-27 18:18:06,238][20381] Updated weights for policy 1, policy_version 1825217 (0.0008) [2023-12-27 18:18:06,280][20347] Updated weights for policy 0, policy_version 1826844 (0.0010) [2023-12-27 18:18:06,290][20381] Updated weights for policy 1, policy_version 1825227 (0.0007) [2023-12-27 18:18:06,340][20347] Updated weights for policy 0, policy_version 1826854 (0.0011) [2023-12-27 18:18:06,353][20381] Updated weights for policy 1, policy_version 1825237 (0.0006) [2023-12-27 18:18:06,404][20347] Updated weights for policy 0, policy_version 1826864 (0.0010) [2023-12-27 18:18:07,029][20381] Updated weights for policy 1, policy_version 1825247 (0.0006) [2023-12-27 18:18:07,035][20347] Updated weights for policy 0, policy_version 1826874 (0.0009) [2023-12-27 18:18:07,078][20381] Updated weights for policy 1, policy_version 1825257 (0.0005) [2023-12-27 18:18:07,080][20347] Updated weights for policy 0, policy_version 1826884 (0.0010) [2023-12-27 18:18:07,127][20381] Updated weights for policy 1, policy_version 1825267 (0.0006) [2023-12-27 18:18:07,132][20347] Updated weights for policy 0, policy_version 1826894 (0.0010) [2023-12-27 18:18:07,193][20347] Updated weights for policy 0, policy_version 1826904 (0.0005) [2023-12-27 18:18:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 935092224. Throughput: 0: 10353.1, 1: 10332.0. Samples: 935054844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:07,363][19170] Avg episode reward: [(0, '990.240'), (1, '970.830')] [2023-12-27 18:18:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001825272_467320832.pth... [2023-12-27 18:18:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001826904_467771392.pth... [2023-12-27 18:18:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001824088_467017728.pth [2023-12-27 18:18:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001825688_467460096.pth [2023-12-27 18:18:07,723][20381] Updated weights for policy 1, policy_version 1825277 (0.0007) [2023-12-27 18:18:07,773][20381] Updated weights for policy 1, policy_version 1825287 (0.0008) [2023-12-27 18:18:07,821][20381] Updated weights for policy 1, policy_version 1825297 (0.0007) [2023-12-27 18:18:07,890][20347] Updated weights for policy 0, policy_version 1826914 (0.0008) [2023-12-27 18:18:07,947][20347] Updated weights for policy 0, policy_version 1826924 (0.0008) [2023-12-27 18:18:08,008][20347] Updated weights for policy 0, policy_version 1826934 (0.0006) [2023-12-27 18:18:08,503][20381] Updated weights for policy 1, policy_version 1825307 (0.0007) [2023-12-27 18:18:08,556][20381] Updated weights for policy 1, policy_version 1825317 (0.0008) [2023-12-27 18:18:08,616][20381] Updated weights for policy 1, policy_version 1825327 (0.0005) [2023-12-27 18:18:08,658][20347] Updated weights for policy 0, policy_version 1826944 (0.0008) [2023-12-27 18:18:08,708][20347] Updated weights for policy 0, policy_version 1826954 (0.0008) [2023-12-27 18:18:08,758][20347] Updated weights for policy 0, policy_version 1826964 (0.0007) [2023-12-27 18:18:09,222][20381] Updated weights for policy 1, policy_version 1825337 (0.0006) [2023-12-27 18:18:09,282][20381] Updated weights for policy 1, policy_version 1825347 (0.0007) [2023-12-27 18:18:09,347][20381] Updated weights for policy 1, policy_version 1825357 (0.0007) [2023-12-27 18:18:09,411][20381] Updated weights for policy 1, policy_version 1825367 (0.0008) [2023-12-27 18:18:09,503][20347] Updated weights for policy 0, policy_version 1826974 (0.0006) [2023-12-27 18:18:09,550][20347] Updated weights for policy 0, policy_version 1826984 (0.0006) [2023-12-27 18:18:09,595][20347] Updated weights for policy 0, policy_version 1826994 (0.0007) [2023-12-27 18:18:10,134][20381] Updated weights for policy 1, policy_version 1825377 (0.0008) [2023-12-27 18:18:10,200][20381] Updated weights for policy 1, policy_version 1825387 (0.0006) [2023-12-27 18:18:10,260][20381] Updated weights for policy 1, policy_version 1825397 (0.0008) [2023-12-27 18:18:10,303][20347] Updated weights for policy 0, policy_version 1827004 (0.0007) [2023-12-27 18:18:10,366][20347] Updated weights for policy 0, policy_version 1827014 (0.0008) [2023-12-27 18:18:10,423][20347] Updated weights for policy 0, policy_version 1827024 (0.0008) [2023-12-27 18:18:10,948][20381] Updated weights for policy 1, policy_version 1825407 (0.0007) [2023-12-27 18:18:11,014][20381] Updated weights for policy 1, policy_version 1825417 (0.0008) [2023-12-27 18:18:11,075][20381] Updated weights for policy 1, policy_version 1825427 (0.0008) [2023-12-27 18:18:11,101][20347] Updated weights for policy 0, policy_version 1827034 (0.0007) [2023-12-27 18:18:11,166][20347] Updated weights for policy 0, policy_version 1827044 (0.0007) [2023-12-27 18:18:11,229][20347] Updated weights for policy 0, policy_version 1827054 (0.0007) [2023-12-27 18:18:11,296][20347] Updated weights for policy 0, policy_version 1827064 (0.0007) [2023-12-27 18:18:11,877][20381] Updated weights for policy 1, policy_version 1825437 (0.0008) [2023-12-27 18:18:11,942][20381] Updated weights for policy 1, policy_version 1825447 (0.0008) [2023-12-27 18:18:11,998][20381] Updated weights for policy 1, policy_version 1825457 (0.0008) [2023-12-27 18:18:12,072][20347] Updated weights for policy 0, policy_version 1827074 (0.0007) [2023-12-27 18:18:12,139][20347] Updated weights for policy 0, policy_version 1827084 (0.0008) [2023-12-27 18:18:12,207][20347] Updated weights for policy 0, policy_version 1827094 (0.0007) [2023-12-27 18:18:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 935190528. Throughput: 0: 10330.1, 1: 10361.6. Samples: 935179056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:12,363][19170] Avg episode reward: [(0, '980.480'), (1, '951.250')] [2023-12-27 18:18:12,726][20381] Updated weights for policy 1, policy_version 1825467 (0.0007) [2023-12-27 18:18:12,774][20381] Updated weights for policy 1, policy_version 1825477 (0.0008) [2023-12-27 18:18:12,836][20381] Updated weights for policy 1, policy_version 1825487 (0.0007) [2023-12-27 18:18:12,867][20347] Updated weights for policy 0, policy_version 1827104 (0.0007) [2023-12-27 18:18:12,926][20347] Updated weights for policy 0, policy_version 1827114 (0.0007) [2023-12-27 18:18:12,984][20347] Updated weights for policy 0, policy_version 1827124 (0.0007) [2023-12-27 18:18:13,588][20381] Updated weights for policy 1, policy_version 1825497 (0.0006) [2023-12-27 18:18:13,643][20381] Updated weights for policy 1, policy_version 1825507 (0.0007) [2023-12-27 18:18:13,669][20347] Updated weights for policy 0, policy_version 1827134 (0.0007) [2023-12-27 18:18:13,709][20381] Updated weights for policy 1, policy_version 1825517 (0.0007) [2023-12-27 18:18:13,724][20347] Updated weights for policy 0, policy_version 1827144 (0.0005) [2023-12-27 18:18:13,763][20381] Updated weights for policy 1, policy_version 1825527 (0.0006) [2023-12-27 18:18:13,778][20347] Updated weights for policy 0, policy_version 1827154 (0.0005) [2023-12-27 18:18:14,401][20381] Updated weights for policy 1, policy_version 1825537 (0.0010) [2023-12-27 18:18:14,454][20347] Updated weights for policy 0, policy_version 1827164 (0.0007) [2023-12-27 18:18:14,459][20381] Updated weights for policy 1, policy_version 1825547 (0.0010) [2023-12-27 18:18:14,508][20347] Updated weights for policy 0, policy_version 1827174 (0.0005) [2023-12-27 18:18:14,517][20381] Updated weights for policy 1, policy_version 1825557 (0.0010) [2023-12-27 18:18:14,567][20347] Updated weights for policy 0, policy_version 1827184 (0.0007) [2023-12-27 18:18:15,197][20381] Updated weights for policy 1, policy_version 1825567 (0.0010) [2023-12-27 18:18:15,257][20381] Updated weights for policy 1, policy_version 1825577 (0.0011) [2023-12-27 18:18:15,295][20347] Updated weights for policy 0, policy_version 1827194 (0.0008) [2023-12-27 18:18:15,317][20381] Updated weights for policy 1, policy_version 1825587 (0.0011) [2023-12-27 18:18:15,353][20347] Updated weights for policy 0, policy_version 1827204 (0.0007) [2023-12-27 18:18:15,409][20347] Updated weights for policy 0, policy_version 1827214 (0.0005) [2023-12-27 18:18:15,464][20347] Updated weights for policy 0, policy_version 1827224 (0.0006) [2023-12-27 18:18:15,992][20381] Updated weights for policy 1, policy_version 1825597 (0.0010) [2023-12-27 18:18:16,046][20381] Updated weights for policy 1, policy_version 1825607 (0.0010) [2023-12-27 18:18:16,091][20381] Updated weights for policy 1, policy_version 1825617 (0.0010) [2023-12-27 18:18:16,101][20347] Updated weights for policy 0, policy_version 1827234 (0.0006) [2023-12-27 18:18:16,148][20347] Updated weights for policy 0, policy_version 1827244 (0.0006) [2023-12-27 18:18:16,198][20347] Updated weights for policy 0, policy_version 1827254 (0.0007) [2023-12-27 18:18:16,789][20381] Updated weights for policy 1, policy_version 1825627 (0.0009) [2023-12-27 18:18:16,843][20381] Updated weights for policy 1, policy_version 1825637 (0.0005) [2023-12-27 18:18:16,896][20381] Updated weights for policy 1, policy_version 1825647 (0.0006) [2023-12-27 18:18:16,915][20347] Updated weights for policy 0, policy_version 1827264 (0.0006) [2023-12-27 18:18:16,967][20347] Updated weights for policy 0, policy_version 1827274 (0.0007) [2023-12-27 18:18:17,018][20347] Updated weights for policy 0, policy_version 1827284 (0.0007) [2023-12-27 18:18:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 935288832. Throughput: 0: 10298.3, 1: 10238.1. Samples: 935299572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:17,363][19170] Avg episode reward: [(0, '990.240'), (1, '960.980')] [2023-12-27 18:18:17,495][20381] Updated weights for policy 1, policy_version 1825657 (0.0006) [2023-12-27 18:18:17,553][20381] Updated weights for policy 1, policy_version 1825667 (0.0010) [2023-12-27 18:18:17,614][20381] Updated weights for policy 1, policy_version 1825677 (0.0008) [2023-12-27 18:18:17,671][20381] Updated weights for policy 1, policy_version 1825687 (0.0007) [2023-12-27 18:18:17,693][20347] Updated weights for policy 0, policy_version 1827294 (0.0007) [2023-12-27 18:18:17,745][20347] Updated weights for policy 0, policy_version 1827304 (0.0007) [2023-12-27 18:18:17,811][20347] Updated weights for policy 0, policy_version 1827314 (0.0005) [2023-12-27 18:18:18,402][20347] Updated weights for policy 0, policy_version 1827324 (0.0006) [2023-12-27 18:18:18,405][20381] Updated weights for policy 1, policy_version 1825697 (0.0006) [2023-12-27 18:18:18,455][20347] Updated weights for policy 0, policy_version 1827334 (0.0009) [2023-12-27 18:18:18,461][20381] Updated weights for policy 1, policy_version 1825707 (0.0007) [2023-12-27 18:18:18,508][20347] Updated weights for policy 0, policy_version 1827344 (0.0006) [2023-12-27 18:18:18,518][20381] Updated weights for policy 1, policy_version 1825717 (0.0006) [2023-12-27 18:18:19,178][20381] Updated weights for policy 1, policy_version 1825727 (0.0007) [2023-12-27 18:18:19,209][20347] Updated weights for policy 0, policy_version 1827354 (0.0007) [2023-12-27 18:18:19,234][20381] Updated weights for policy 1, policy_version 1825737 (0.0007) [2023-12-27 18:18:19,271][20347] Updated weights for policy 0, policy_version 1827364 (0.0007) [2023-12-27 18:18:19,291][20381] Updated weights for policy 1, policy_version 1825747 (0.0007) [2023-12-27 18:18:19,333][20347] Updated weights for policy 0, policy_version 1827374 (0.0006) [2023-12-27 18:18:19,399][20347] Updated weights for policy 0, policy_version 1827384 (0.0007) [2023-12-27 18:18:20,018][20381] Updated weights for policy 1, policy_version 1825757 (0.0007) [2023-12-27 18:18:20,079][20381] Updated weights for policy 1, policy_version 1825767 (0.0007) [2023-12-27 18:18:20,137][20381] Updated weights for policy 1, policy_version 1825777 (0.0006) [2023-12-27 18:18:20,161][20347] Updated weights for policy 0, policy_version 1827394 (0.0007) [2023-12-27 18:18:20,221][20347] Updated weights for policy 0, policy_version 1827404 (0.0007) [2023-12-27 18:18:20,278][20347] Updated weights for policy 0, policy_version 1827414 (0.0009) [2023-12-27 18:18:20,854][20381] Updated weights for policy 1, policy_version 1825787 (0.0006) [2023-12-27 18:18:20,914][20381] Updated weights for policy 1, policy_version 1825797 (0.0011) [2023-12-27 18:18:20,976][20381] Updated weights for policy 1, policy_version 1825807 (0.0011) [2023-12-27 18:18:21,000][20347] Updated weights for policy 0, policy_version 1827424 (0.0006) [2023-12-27 18:18:21,066][20347] Updated weights for policy 0, policy_version 1827434 (0.0007) [2023-12-27 18:18:21,127][20347] Updated weights for policy 0, policy_version 1827444 (0.0007) [2023-12-27 18:18:21,758][20381] Updated weights for policy 1, policy_version 1825817 (0.0010) [2023-12-27 18:18:21,823][20381] Updated weights for policy 1, policy_version 1825827 (0.0009) [2023-12-27 18:18:21,831][20347] Updated weights for policy 0, policy_version 1827454 (0.0007) [2023-12-27 18:18:21,882][20347] Updated weights for policy 0, policy_version 1827464 (0.0006) [2023-12-27 18:18:21,883][20381] Updated weights for policy 1, policy_version 1825837 (0.0009) [2023-12-27 18:18:21,937][20347] Updated weights for policy 0, policy_version 1827474 (0.0006) [2023-12-27 18:18:21,945][20381] Updated weights for policy 1, policy_version 1825847 (0.0008) [2023-12-27 18:18:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 935387136. Throughput: 0: 10285.1, 1: 10176.9. Samples: 935360044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.530')] [2023-12-27 18:18:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001825848_467468288.pth... [2023-12-27 18:18:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001827480_467918848.pth... [2023-12-27 18:18:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001824664_467165184.pth [2023-12-27 18:18:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001826296_467615744.pth [2023-12-27 18:18:22,667][20347] Updated weights for policy 0, policy_version 1827484 (0.0006) [2023-12-27 18:18:22,690][20381] Updated weights for policy 1, policy_version 1825857 (0.0008) [2023-12-27 18:18:22,734][20347] Updated weights for policy 0, policy_version 1827494 (0.0006) [2023-12-27 18:18:22,748][20381] Updated weights for policy 1, policy_version 1825867 (0.0007) [2023-12-27 18:18:22,795][20347] Updated weights for policy 0, policy_version 1827504 (0.0007) [2023-12-27 18:18:22,807][20381] Updated weights for policy 1, policy_version 1825877 (0.0007) [2023-12-27 18:18:23,451][20381] Updated weights for policy 1, policy_version 1825887 (0.0006) [2023-12-27 18:18:23,479][20347] Updated weights for policy 0, policy_version 1827514 (0.0007) [2023-12-27 18:18:23,512][20381] Updated weights for policy 1, policy_version 1825897 (0.0005) [2023-12-27 18:18:23,542][20347] Updated weights for policy 0, policy_version 1827524 (0.0008) [2023-12-27 18:18:23,570][20381] Updated weights for policy 1, policy_version 1825907 (0.0006) [2023-12-27 18:18:23,608][20347] Updated weights for policy 0, policy_version 1827534 (0.0008) [2023-12-27 18:18:23,655][20347] Updated weights for policy 0, policy_version 1827544 (0.0010) [2023-12-27 18:18:24,118][20381] Updated weights for policy 1, policy_version 1825917 (0.0008) [2023-12-27 18:18:24,168][20381] Updated weights for policy 1, policy_version 1825927 (0.0008) [2023-12-27 18:18:24,219][20381] Updated weights for policy 1, policy_version 1825937 (0.0008) [2023-12-27 18:18:24,340][20347] Updated weights for policy 0, policy_version 1827554 (0.0007) [2023-12-27 18:18:24,392][20347] Updated weights for policy 0, policy_version 1827564 (0.0007) [2023-12-27 18:18:24,447][20347] Updated weights for policy 0, policy_version 1827574 (0.0008) [2023-12-27 18:18:24,852][20381] Updated weights for policy 1, policy_version 1825947 (0.0007) [2023-12-27 18:18:24,908][20381] Updated weights for policy 1, policy_version 1825957 (0.0005) [2023-12-27 18:18:24,966][20381] Updated weights for policy 1, policy_version 1825967 (0.0005) [2023-12-27 18:18:25,085][20347] Updated weights for policy 0, policy_version 1827584 (0.0008) [2023-12-27 18:18:25,140][20347] Updated weights for policy 0, policy_version 1827594 (0.0008) [2023-12-27 18:18:25,198][20347] Updated weights for policy 0, policy_version 1827604 (0.0008) [2023-12-27 18:18:25,575][20381] Updated weights for policy 1, policy_version 1825977 (0.0006) [2023-12-27 18:18:25,631][20381] Updated weights for policy 1, policy_version 1825987 (0.0005) [2023-12-27 18:18:25,690][20381] Updated weights for policy 1, policy_version 1825997 (0.0005) [2023-12-27 18:18:25,750][20381] Updated weights for policy 1, policy_version 1826007 (0.0005) [2023-12-27 18:18:25,839][20347] Updated weights for policy 0, policy_version 1827614 (0.0007) [2023-12-27 18:18:25,894][20347] Updated weights for policy 0, policy_version 1827624 (0.0007) [2023-12-27 18:18:25,947][20347] Updated weights for policy 0, policy_version 1827634 (0.0007) [2023-12-27 18:18:26,314][20381] Updated weights for policy 1, policy_version 1826017 (0.0006) [2023-12-27 18:18:26,364][20381] Updated weights for policy 1, policy_version 1826027 (0.0008) [2023-12-27 18:18:26,412][20381] Updated weights for policy 1, policy_version 1826037 (0.0007) [2023-12-27 18:18:26,665][20347] Updated weights for policy 0, policy_version 1827644 (0.0007) [2023-12-27 18:18:26,715][20347] Updated weights for policy 0, policy_version 1827654 (0.0005) [2023-12-27 18:18:26,766][20347] Updated weights for policy 0, policy_version 1827664 (0.0006) [2023-12-27 18:18:27,009][20381] Updated weights for policy 1, policy_version 1826047 (0.0008) [2023-12-27 18:18:27,055][20381] Updated weights for policy 1, policy_version 1826057 (0.0010) [2023-12-27 18:18:27,105][20381] Updated weights for policy 1, policy_version 1826067 (0.0011) [2023-12-27 18:18:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 935493632. Throughput: 0: 10229.4, 1: 10233.8. Samples: 935482968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.440')] [2023-12-27 18:18:27,449][20347] Updated weights for policy 0, policy_version 1827674 (0.0008) [2023-12-27 18:18:27,507][20347] Updated weights for policy 0, policy_version 1827684 (0.0005) [2023-12-27 18:18:27,566][20347] Updated weights for policy 0, policy_version 1827694 (0.0007) [2023-12-27 18:18:27,618][20347] Updated weights for policy 0, policy_version 1827704 (0.0008) [2023-12-27 18:18:27,694][20381] Updated weights for policy 1, policy_version 1826077 (0.0008) [2023-12-27 18:18:27,743][20381] Updated weights for policy 1, policy_version 1826087 (0.0005) [2023-12-27 18:18:27,792][20381] Updated weights for policy 1, policy_version 1826097 (0.0005) [2023-12-27 18:18:28,209][20347] Updated weights for policy 0, policy_version 1827714 (0.0005) [2023-12-27 18:18:28,255][20347] Updated weights for policy 0, policy_version 1827724 (0.0005) [2023-12-27 18:18:28,303][20347] Updated weights for policy 0, policy_version 1827734 (0.0007) [2023-12-27 18:18:28,410][20381] Updated weights for policy 1, policy_version 1826107 (0.0008) [2023-12-27 18:18:28,469][20381] Updated weights for policy 1, policy_version 1826117 (0.0009) [2023-12-27 18:18:28,529][20381] Updated weights for policy 1, policy_version 1826127 (0.0007) [2023-12-27 18:18:28,923][20347] Updated weights for policy 0, policy_version 1827744 (0.0010) [2023-12-27 18:18:28,965][20347] Updated weights for policy 0, policy_version 1827754 (0.0010) [2023-12-27 18:18:29,013][20347] Updated weights for policy 0, policy_version 1827764 (0.0010) [2023-12-27 18:18:29,224][20381] Updated weights for policy 1, policy_version 1826137 (0.0008) [2023-12-27 18:18:29,283][20381] Updated weights for policy 1, policy_version 1826147 (0.0008) [2023-12-27 18:18:29,338][20381] Updated weights for policy 1, policy_version 1826157 (0.0007) [2023-12-27 18:18:29,401][20381] Updated weights for policy 1, policy_version 1826167 (0.0008) [2023-12-27 18:18:29,719][20347] Updated weights for policy 0, policy_version 1827774 (0.0010) [2023-12-27 18:18:29,777][20347] Updated weights for policy 0, policy_version 1827784 (0.0010) [2023-12-27 18:18:29,841][20347] Updated weights for policy 0, policy_version 1827794 (0.0010) [2023-12-27 18:18:30,139][20381] Updated weights for policy 1, policy_version 1826177 (0.0008) [2023-12-27 18:18:30,189][20381] Updated weights for policy 1, policy_version 1826187 (0.0008) [2023-12-27 18:18:30,245][20381] Updated weights for policy 1, policy_version 1826197 (0.0008) [2023-12-27 18:18:30,474][20347] Updated weights for policy 0, policy_version 1827804 (0.0009) [2023-12-27 18:18:30,528][20347] Updated weights for policy 0, policy_version 1827814 (0.0008) [2023-12-27 18:18:30,577][20347] Updated weights for policy 0, policy_version 1827824 (0.0007) [2023-12-27 18:18:30,958][20381] Updated weights for policy 1, policy_version 1826207 (0.0007) [2023-12-27 18:18:31,007][20381] Updated weights for policy 1, policy_version 1826217 (0.0008) [2023-12-27 18:18:31,067][20381] Updated weights for policy 1, policy_version 1826227 (0.0006) [2023-12-27 18:18:31,264][20347] Updated weights for policy 0, policy_version 1827834 (0.0007) [2023-12-27 18:18:31,323][20347] Updated weights for policy 0, policy_version 1827844 (0.0008) [2023-12-27 18:18:31,383][20347] Updated weights for policy 0, policy_version 1827854 (0.0008) [2023-12-27 18:18:31,444][20347] Updated weights for policy 0, policy_version 1827864 (0.0009) [2023-12-27 18:18:31,768][20381] Updated weights for policy 1, policy_version 1826237 (0.0008) [2023-12-27 18:18:31,827][20381] Updated weights for policy 1, policy_version 1826247 (0.0008) [2023-12-27 18:18:31,884][20381] Updated weights for policy 1, policy_version 1826257 (0.0008) [2023-12-27 18:18:32,150][20347] Updated weights for policy 0, policy_version 1827874 (0.0007) [2023-12-27 18:18:32,203][20347] Updated weights for policy 0, policy_version 1827884 (0.0006) [2023-12-27 18:18:32,261][20347] Updated weights for policy 0, policy_version 1827894 (0.0007) [2023-12-27 18:18:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 935600128. Throughput: 0: 10249.0, 1: 10268.4. Samples: 935607764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.730')] [2023-12-27 18:18:32,625][20381] Updated weights for policy 1, policy_version 1826267 (0.0007) [2023-12-27 18:18:32,678][20381] Updated weights for policy 1, policy_version 1826277 (0.0007) [2023-12-27 18:18:32,727][20381] Updated weights for policy 1, policy_version 1826287 (0.0008) [2023-12-27 18:18:32,909][20347] Updated weights for policy 0, policy_version 1827904 (0.0008) [2023-12-27 18:18:32,971][20347] Updated weights for policy 0, policy_version 1827914 (0.0008) [2023-12-27 18:18:33,030][20347] Updated weights for policy 0, policy_version 1827924 (0.0007) [2023-12-27 18:18:33,430][20381] Updated weights for policy 1, policy_version 1826297 (0.0010) [2023-12-27 18:18:33,480][20381] Updated weights for policy 1, policy_version 1826307 (0.0007) [2023-12-27 18:18:33,533][20381] Updated weights for policy 1, policy_version 1826317 (0.0005) [2023-12-27 18:18:33,579][20381] Updated weights for policy 1, policy_version 1826327 (0.0005) [2023-12-27 18:18:33,678][20347] Updated weights for policy 0, policy_version 1827934 (0.0007) [2023-12-27 18:18:33,739][20347] Updated weights for policy 0, policy_version 1827944 (0.0005) [2023-12-27 18:18:33,795][20347] Updated weights for policy 0, policy_version 1827954 (0.0009) [2023-12-27 18:18:34,165][20381] Updated weights for policy 1, policy_version 1826337 (0.0007) [2023-12-27 18:18:34,228][20381] Updated weights for policy 1, policy_version 1826347 (0.0008) [2023-12-27 18:18:34,292][20381] Updated weights for policy 1, policy_version 1826357 (0.0011) [2023-12-27 18:18:34,459][20347] Updated weights for policy 0, policy_version 1827964 (0.0009) [2023-12-27 18:18:34,504][20347] Updated weights for policy 0, policy_version 1827974 (0.0007) [2023-12-27 18:18:34,552][20347] Updated weights for policy 0, policy_version 1827984 (0.0007) [2023-12-27 18:18:34,917][20381] Updated weights for policy 1, policy_version 1826367 (0.0007) [2023-12-27 18:18:34,971][20381] Updated weights for policy 1, policy_version 1826377 (0.0005) [2023-12-27 18:18:35,031][20381] Updated weights for policy 1, policy_version 1826387 (0.0005) [2023-12-27 18:18:35,184][20347] Updated weights for policy 0, policy_version 1827994 (0.0007) [2023-12-27 18:18:35,231][20347] Updated weights for policy 0, policy_version 1828004 (0.0009) [2023-12-27 18:18:35,275][20347] Updated weights for policy 0, policy_version 1828014 (0.0010) [2023-12-27 18:18:35,324][20347] Updated weights for policy 0, policy_version 1828024 (0.0010) [2023-12-27 18:18:35,685][20381] Updated weights for policy 1, policy_version 1826397 (0.0007) [2023-12-27 18:18:35,728][20381] Updated weights for policy 1, policy_version 1826407 (0.0008) [2023-12-27 18:18:35,776][20381] Updated weights for policy 1, policy_version 1826417 (0.0007) [2023-12-27 18:18:35,900][20347] Updated weights for policy 0, policy_version 1828034 (0.0010) [2023-12-27 18:18:35,954][20347] Updated weights for policy 0, policy_version 1828044 (0.0007) [2023-12-27 18:18:36,002][20347] Updated weights for policy 0, policy_version 1828054 (0.0008) [2023-12-27 18:18:36,524][20381] Updated weights for policy 1, policy_version 1826427 (0.0009) [2023-12-27 18:18:36,578][20381] Updated weights for policy 1, policy_version 1826437 (0.0008) [2023-12-27 18:18:36,641][20381] Updated weights for policy 1, policy_version 1826447 (0.0007) [2023-12-27 18:18:36,659][20347] Updated weights for policy 0, policy_version 1828064 (0.0009) [2023-12-27 18:18:36,720][20347] Updated weights for policy 0, policy_version 1828074 (0.0010) [2023-12-27 18:18:36,776][20347] Updated weights for policy 0, policy_version 1828084 (0.0010) [2023-12-27 18:18:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 935698432. Throughput: 0: 10255.2, 1: 10285.7. Samples: 935671488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:37,363][19170] Avg episode reward: [(0, '990.330'), (1, '921.750')] [2023-12-27 18:18:37,367][20381] Updated weights for policy 1, policy_version 1826457 (0.0008) [2023-12-27 18:18:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001828088_468074496.pth... [2023-12-27 18:18:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001826904_467771392.pth [2023-12-27 18:18:37,429][20381] Updated weights for policy 1, policy_version 1826467 (0.0011) [2023-12-27 18:18:37,447][20347] Updated weights for policy 0, policy_version 1828094 (0.0006) [2023-12-27 18:18:37,481][20381] Updated weights for policy 1, policy_version 1826477 (0.0010) [2023-12-27 18:18:37,501][20347] Updated weights for policy 0, policy_version 1828104 (0.0005) [2023-12-27 18:18:37,532][20381] Updated weights for policy 1, policy_version 1826487 (0.0010) [2023-12-27 18:18:37,537][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001826488_467632128.pth... [2023-12-27 18:18:37,540][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001825272_467320832.pth [2023-12-27 18:18:37,557][20347] Updated weights for policy 0, policy_version 1828114 (0.0007) [2023-12-27 18:18:38,124][20381] Updated weights for policy 1, policy_version 1826497 (0.0007) [2023-12-27 18:18:38,180][20381] Updated weights for policy 1, policy_version 1826507 (0.0006) [2023-12-27 18:18:38,181][20347] Updated weights for policy 0, policy_version 1828124 (0.0007) [2023-12-27 18:18:38,237][20381] Updated weights for policy 1, policy_version 1826517 (0.0006) [2023-12-27 18:18:38,245][20347] Updated weights for policy 0, policy_version 1828134 (0.0008) [2023-12-27 18:18:38,312][20347] Updated weights for policy 0, policy_version 1828144 (0.0007) [2023-12-27 18:18:38,895][20381] Updated weights for policy 1, policy_version 1826527 (0.0006) [2023-12-27 18:18:38,952][20381] Updated weights for policy 1, policy_version 1826537 (0.0005) [2023-12-27 18:18:38,995][20347] Updated weights for policy 0, policy_version 1828154 (0.0008) [2023-12-27 18:18:39,014][20381] Updated weights for policy 1, policy_version 1826547 (0.0005) [2023-12-27 18:18:39,054][20347] Updated weights for policy 0, policy_version 1828164 (0.0006) [2023-12-27 18:18:39,103][20347] Updated weights for policy 0, policy_version 1828174 (0.0007) [2023-12-27 18:18:39,153][20347] Updated weights for policy 0, policy_version 1828184 (0.0008) [2023-12-27 18:18:39,684][20381] Updated weights for policy 1, policy_version 1826557 (0.0007) [2023-12-27 18:18:39,747][20381] Updated weights for policy 1, policy_version 1826567 (0.0008) [2023-12-27 18:18:39,807][20381] Updated weights for policy 1, policy_version 1826577 (0.0008) [2023-12-27 18:18:39,862][20347] Updated weights for policy 0, policy_version 1828194 (0.0008) [2023-12-27 18:18:39,922][20347] Updated weights for policy 0, policy_version 1828204 (0.0008) [2023-12-27 18:18:39,978][20347] Updated weights for policy 0, policy_version 1828214 (0.0008) [2023-12-27 18:18:40,538][20381] Updated weights for policy 1, policy_version 1826587 (0.0008) [2023-12-27 18:18:40,591][20381] Updated weights for policy 1, policy_version 1826597 (0.0008) [2023-12-27 18:18:40,643][20381] Updated weights for policy 1, policy_version 1826607 (0.0008) [2023-12-27 18:18:40,683][20347] Updated weights for policy 0, policy_version 1828224 (0.0006) [2023-12-27 18:18:40,739][20347] Updated weights for policy 0, policy_version 1828234 (0.0007) [2023-12-27 18:18:40,794][20347] Updated weights for policy 0, policy_version 1828244 (0.0007) [2023-12-27 18:18:41,397][20381] Updated weights for policy 1, policy_version 1826617 (0.0006) [2023-12-27 18:18:41,456][20381] Updated weights for policy 1, policy_version 1826627 (0.0008) [2023-12-27 18:18:41,516][20381] Updated weights for policy 1, policy_version 1826637 (0.0008) [2023-12-27 18:18:41,572][20381] Updated weights for policy 1, policy_version 1826647 (0.0008) [2023-12-27 18:18:41,606][20347] Updated weights for policy 0, policy_version 1828254 (0.0007) [2023-12-27 18:18:41,670][20347] Updated weights for policy 0, policy_version 1828264 (0.0008) [2023-12-27 18:18:41,732][20347] Updated weights for policy 0, policy_version 1828274 (0.0010) [2023-12-27 18:18:42,304][20381] Updated weights for policy 1, policy_version 1826657 (0.0007) [2023-12-27 18:18:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 935796736. Throughput: 0: 10260.6, 1: 10271.3. Samples: 935793992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:42,363][19170] Avg episode reward: [(0, '990.330'), (1, '931.660')] [2023-12-27 18:18:42,373][20381] Updated weights for policy 1, policy_version 1826667 (0.0009) [2023-12-27 18:18:42,438][20381] Updated weights for policy 1, policy_version 1826677 (0.0009) [2023-12-27 18:18:42,470][20347] Updated weights for policy 0, policy_version 1828284 (0.0008) [2023-12-27 18:18:42,559][20347] Updated weights for policy 0, policy_version 1828294 (0.0006) [2023-12-27 18:18:42,622][20347] Updated weights for policy 0, policy_version 1828304 (0.0011) [2023-12-27 18:18:43,143][20381] Updated weights for policy 1, policy_version 1826687 (0.0008) [2023-12-27 18:18:43,200][20381] Updated weights for policy 1, policy_version 1826697 (0.0008) [2023-12-27 18:18:43,253][20381] Updated weights for policy 1, policy_version 1826707 (0.0007) [2023-12-27 18:18:43,258][20347] Updated weights for policy 0, policy_version 1828314 (0.0011) [2023-12-27 18:18:43,312][20347] Updated weights for policy 0, policy_version 1828324 (0.0007) [2023-12-27 18:18:43,367][20347] Updated weights for policy 0, policy_version 1828334 (0.0006) [2023-12-27 18:18:43,427][20347] Updated weights for policy 0, policy_version 1828344 (0.0008) [2023-12-27 18:18:44,005][20347] Updated weights for policy 0, policy_version 1828354 (0.0006) [2023-12-27 18:18:44,017][20381] Updated weights for policy 1, policy_version 1826717 (0.0007) [2023-12-27 18:18:44,055][20347] Updated weights for policy 0, policy_version 1828364 (0.0009) [2023-12-27 18:18:44,068][20381] Updated weights for policy 1, policy_version 1826727 (0.0006) [2023-12-27 18:18:44,101][20347] Updated weights for policy 0, policy_version 1828374 (0.0010) [2023-12-27 18:18:44,124][20381] Updated weights for policy 1, policy_version 1826737 (0.0006) [2023-12-27 18:18:44,678][20347] Updated weights for policy 0, policy_version 1828384 (0.0008) [2023-12-27 18:18:44,737][20347] Updated weights for policy 0, policy_version 1828394 (0.0007) [2023-12-27 18:18:44,802][20347] Updated weights for policy 0, policy_version 1828404 (0.0008) [2023-12-27 18:18:44,838][20381] Updated weights for policy 1, policy_version 1826747 (0.0007) [2023-12-27 18:18:44,899][20381] Updated weights for policy 1, policy_version 1826757 (0.0006) [2023-12-27 18:18:44,954][20381] Updated weights for policy 1, policy_version 1826767 (0.0006) [2023-12-27 18:18:45,461][20347] Updated weights for policy 0, policy_version 1828414 (0.0006) [2023-12-27 18:18:45,512][20347] Updated weights for policy 0, policy_version 1828424 (0.0005) [2023-12-27 18:18:45,564][20347] Updated weights for policy 0, policy_version 1828434 (0.0005) [2023-12-27 18:18:45,597][20381] Updated weights for policy 1, policy_version 1826777 (0.0008) [2023-12-27 18:18:45,643][20381] Updated weights for policy 1, policy_version 1826787 (0.0007) [2023-12-27 18:18:45,696][20381] Updated weights for policy 1, policy_version 1826797 (0.0005) [2023-12-27 18:18:45,759][20381] Updated weights for policy 1, policy_version 1826807 (0.0005) [2023-12-27 18:18:46,226][20347] Updated weights for policy 0, policy_version 1828444 (0.0007) [2023-12-27 18:18:46,287][20347] Updated weights for policy 0, policy_version 1828454 (0.0010) [2023-12-27 18:18:46,347][20347] Updated weights for policy 0, policy_version 1828464 (0.0010) [2023-12-27 18:18:46,349][20381] Updated weights for policy 1, policy_version 1826817 (0.0006) [2023-12-27 18:18:46,403][20381] Updated weights for policy 1, policy_version 1826827 (0.0006) [2023-12-27 18:18:46,463][20381] Updated weights for policy 1, policy_version 1826837 (0.0007) [2023-12-27 18:18:46,998][20347] Updated weights for policy 0, policy_version 1828474 (0.0010) [2023-12-27 18:18:47,060][20347] Updated weights for policy 0, policy_version 1828484 (0.0010) [2023-12-27 18:18:47,114][20347] Updated weights for policy 0, policy_version 1828494 (0.0010) [2023-12-27 18:18:47,141][20381] Updated weights for policy 1, policy_version 1826847 (0.0006) [2023-12-27 18:18:47,175][20347] Updated weights for policy 0, policy_version 1828504 (0.0010) [2023-12-27 18:18:47,189][20381] Updated weights for policy 1, policy_version 1826857 (0.0005) [2023-12-27 18:18:47,242][20381] Updated weights for policy 1, policy_version 1826867 (0.0005) [2023-12-27 18:18:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 935911424. Throughput: 0: 10307.8, 1: 10257.1. Samples: 935917224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:18:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.980')] [2023-12-27 18:18:47,836][20347] Updated weights for policy 0, policy_version 1828514 (0.0010) [2023-12-27 18:18:47,889][20381] Updated weights for policy 1, policy_version 1826877 (0.0005) [2023-12-27 18:18:47,891][20347] Updated weights for policy 0, policy_version 1828524 (0.0010) [2023-12-27 18:18:47,949][20347] Updated weights for policy 0, policy_version 1828534 (0.0010) [2023-12-27 18:18:47,956][20381] Updated weights for policy 1, policy_version 1826887 (0.0006) [2023-12-27 18:18:48,015][20381] Updated weights for policy 1, policy_version 1826897 (0.0007) [2023-12-27 18:18:48,658][20347] Updated weights for policy 0, policy_version 1828544 (0.0009) [2023-12-27 18:18:48,700][20381] Updated weights for policy 1, policy_version 1826907 (0.0007) [2023-12-27 18:18:48,713][20347] Updated weights for policy 0, policy_version 1828554 (0.0007) [2023-12-27 18:18:48,760][20381] Updated weights for policy 1, policy_version 1826917 (0.0006) [2023-12-27 18:18:48,774][20347] Updated weights for policy 0, policy_version 1828564 (0.0008) [2023-12-27 18:18:48,818][20381] Updated weights for policy 1, policy_version 1826927 (0.0008) [2023-12-27 18:18:49,466][20347] Updated weights for policy 0, policy_version 1828574 (0.0009) [2023-12-27 18:18:49,524][20347] Updated weights for policy 0, policy_version 1828584 (0.0010) [2023-12-27 18:18:49,545][20381] Updated weights for policy 1, policy_version 1826937 (0.0008) [2023-12-27 18:18:49,581][20347] Updated weights for policy 0, policy_version 1828594 (0.0010) [2023-12-27 18:18:49,592][20381] Updated weights for policy 1, policy_version 1826947 (0.0011) [2023-12-27 18:18:49,639][20381] Updated weights for policy 1, policy_version 1826957 (0.0010) [2023-12-27 18:18:49,688][20381] Updated weights for policy 1, policy_version 1826967 (0.0011) [2023-12-27 18:18:50,286][20347] Updated weights for policy 0, policy_version 1828604 (0.0011) [2023-12-27 18:18:50,349][20347] Updated weights for policy 0, policy_version 1828614 (0.0011) [2023-12-27 18:18:50,412][20347] Updated weights for policy 0, policy_version 1828624 (0.0010) [2023-12-27 18:18:50,417][20381] Updated weights for policy 1, policy_version 1826977 (0.0010) [2023-12-27 18:18:50,472][20381] Updated weights for policy 1, policy_version 1826987 (0.0010) [2023-12-27 18:18:50,530][20381] Updated weights for policy 1, policy_version 1826997 (0.0009) [2023-12-27 18:18:51,108][20347] Updated weights for policy 0, policy_version 1828634 (0.0009) [2023-12-27 18:18:51,166][20347] Updated weights for policy 0, policy_version 1828644 (0.0006) [2023-12-27 18:18:51,225][20347] Updated weights for policy 0, policy_version 1828654 (0.0006) [2023-12-27 18:18:51,261][20381] Updated weights for policy 1, policy_version 1827007 (0.0008) [2023-12-27 18:18:51,289][20347] Updated weights for policy 0, policy_version 1828664 (0.0007) [2023-12-27 18:18:51,327][20381] Updated weights for policy 1, policy_version 1827017 (0.0007) [2023-12-27 18:18:51,398][20381] Updated weights for policy 1, policy_version 1827027 (0.0007) [2023-12-27 18:18:51,996][20347] Updated weights for policy 0, policy_version 1828674 (0.0007) [2023-12-27 18:18:52,059][20347] Updated weights for policy 0, policy_version 1828684 (0.0008) [2023-12-27 18:18:52,123][20347] Updated weights for policy 0, policy_version 1828694 (0.0008) [2023-12-27 18:18:52,149][20381] Updated weights for policy 1, policy_version 1827037 (0.0007) [2023-12-27 18:18:52,198][20381] Updated weights for policy 1, policy_version 1827047 (0.0007) [2023-12-27 18:18:52,260][20381] Updated weights for policy 1, policy_version 1827057 (0.0006) [2023-12-27 18:18:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 936009728. Throughput: 0: 10275.4, 1: 10207.7. Samples: 935976584. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:18:52,364][19170] Avg episode reward: [(0, '1000.000'), (1, '882.620')] [2023-12-27 18:18:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001827064_467779584.pth... [2023-12-27 18:18:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001828696_468230144.pth... [2023-12-27 18:18:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001825848_467468288.pth [2023-12-27 18:18:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001827480_467918848.pth [2023-12-27 18:18:52,782][20347] Updated weights for policy 0, policy_version 1828704 (0.0007) [2023-12-27 18:18:52,832][20347] Updated weights for policy 0, policy_version 1828714 (0.0008) [2023-12-27 18:18:52,883][20347] Updated weights for policy 0, policy_version 1828724 (0.0009) [2023-12-27 18:18:52,911][20381] Updated weights for policy 1, policy_version 1827067 (0.0008) [2023-12-27 18:18:52,971][20381] Updated weights for policy 1, policy_version 1827077 (0.0006) [2023-12-27 18:18:53,029][20381] Updated weights for policy 1, policy_version 1827087 (0.0006) [2023-12-27 18:18:53,498][20347] Updated weights for policy 0, policy_version 1828734 (0.0005) [2023-12-27 18:18:53,545][20347] Updated weights for policy 0, policy_version 1828744 (0.0005) [2023-12-27 18:18:53,564][20381] Updated weights for policy 1, policy_version 1827097 (0.0008) [2023-12-27 18:18:53,600][20347] Updated weights for policy 0, policy_version 1828754 (0.0006) [2023-12-27 18:18:53,629][20381] Updated weights for policy 1, policy_version 1827107 (0.0006) [2023-12-27 18:18:53,690][20381] Updated weights for policy 1, policy_version 1827117 (0.0005) [2023-12-27 18:18:53,749][20381] Updated weights for policy 1, policy_version 1827127 (0.0005) [2023-12-27 18:18:54,299][20381] Updated weights for policy 1, policy_version 1827137 (0.0010) [2023-12-27 18:18:54,315][20347] Updated weights for policy 0, policy_version 1828764 (0.0008) [2023-12-27 18:18:54,355][20381] Updated weights for policy 1, policy_version 1827147 (0.0009) [2023-12-27 18:18:54,364][20347] Updated weights for policy 0, policy_version 1828774 (0.0007) [2023-12-27 18:18:54,414][20381] Updated weights for policy 1, policy_version 1827157 (0.0005) [2023-12-27 18:18:54,424][20347] Updated weights for policy 0, policy_version 1828784 (0.0008) [2023-12-27 18:18:55,062][20381] Updated weights for policy 1, policy_version 1827167 (0.0009) [2023-12-27 18:18:55,111][20347] Updated weights for policy 0, policy_version 1828794 (0.0008) [2023-12-27 18:18:55,117][20381] Updated weights for policy 1, policy_version 1827177 (0.0010) [2023-12-27 18:18:55,164][20347] Updated weights for policy 0, policy_version 1828804 (0.0005) [2023-12-27 18:18:55,166][20381] Updated weights for policy 1, policy_version 1827187 (0.0010) [2023-12-27 18:18:55,217][20347] Updated weights for policy 0, policy_version 1828814 (0.0006) [2023-12-27 18:18:55,265][20347] Updated weights for policy 0, policy_version 1828824 (0.0007) [2023-12-27 18:18:55,814][20381] Updated weights for policy 1, policy_version 1827197 (0.0009) [2023-12-27 18:18:55,860][20381] Updated weights for policy 1, policy_version 1827207 (0.0008) [2023-12-27 18:18:55,917][20381] Updated weights for policy 1, policy_version 1827217 (0.0007) [2023-12-27 18:18:55,977][20347] Updated weights for policy 0, policy_version 1828834 (0.0006) [2023-12-27 18:18:56,034][20347] Updated weights for policy 0, policy_version 1828844 (0.0005) [2023-12-27 18:18:56,084][20347] Updated weights for policy 0, policy_version 1828854 (0.0007) [2023-12-27 18:18:56,518][20381] Updated weights for policy 1, policy_version 1827227 (0.0006) [2023-12-27 18:18:56,570][20381] Updated weights for policy 1, policy_version 1827237 (0.0005) [2023-12-27 18:18:56,618][20381] Updated weights for policy 1, policy_version 1827247 (0.0005) [2023-12-27 18:18:56,712][20347] Updated weights for policy 0, policy_version 1828864 (0.0007) [2023-12-27 18:18:56,760][20347] Updated weights for policy 0, policy_version 1828874 (0.0008) [2023-12-27 18:18:56,807][20347] Updated weights for policy 0, policy_version 1828884 (0.0007) [2023-12-27 18:18:57,251][20381] Updated weights for policy 1, policy_version 1827257 (0.0006) [2023-12-27 18:18:57,306][20381] Updated weights for policy 1, policy_version 1827267 (0.0008) [2023-12-27 18:18:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 936108032. Throughput: 0: 10227.9, 1: 10255.2. Samples: 936100796. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:18:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '969.440')] [2023-12-27 18:18:57,365][20381] Updated weights for policy 1, policy_version 1827277 (0.0009) [2023-12-27 18:18:57,412][20381] Updated weights for policy 1, policy_version 1827287 (0.0008) [2023-12-27 18:18:57,493][20347] Updated weights for policy 0, policy_version 1828894 (0.0006) [2023-12-27 18:18:57,554][20347] Updated weights for policy 0, policy_version 1828904 (0.0006) [2023-12-27 18:18:57,604][20347] Updated weights for policy 0, policy_version 1828914 (0.0007) [2023-12-27 18:18:58,130][20381] Updated weights for policy 1, policy_version 1827297 (0.0008) [2023-12-27 18:18:58,185][20381] Updated weights for policy 1, policy_version 1827307 (0.0007) [2023-12-27 18:18:58,247][20381] Updated weights for policy 1, policy_version 1827317 (0.0007) [2023-12-27 18:18:58,256][20347] Updated weights for policy 0, policy_version 1828924 (0.0007) [2023-12-27 18:18:58,308][20347] Updated weights for policy 0, policy_version 1828934 (0.0007) [2023-12-27 18:18:58,373][20347] Updated weights for policy 0, policy_version 1828944 (0.0008) [2023-12-27 18:18:59,025][20381] Updated weights for policy 1, policy_version 1827327 (0.0006) [2023-12-27 18:18:59,082][20381] Updated weights for policy 1, policy_version 1827337 (0.0008) [2023-12-27 18:18:59,104][20347] Updated weights for policy 0, policy_version 1828954 (0.0009) [2023-12-27 18:18:59,137][20381] Updated weights for policy 1, policy_version 1827347 (0.0008) [2023-12-27 18:18:59,160][20347] Updated weights for policy 0, policy_version 1828964 (0.0010) [2023-12-27 18:18:59,226][20347] Updated weights for policy 0, policy_version 1828974 (0.0010) [2023-12-27 18:18:59,299][20347] Updated weights for policy 0, policy_version 1828984 (0.0009) [2023-12-27 18:18:59,840][20381] Updated weights for policy 1, policy_version 1827357 (0.0008) [2023-12-27 18:18:59,892][20381] Updated weights for policy 1, policy_version 1827367 (0.0007) [2023-12-27 18:18:59,952][20381] Updated weights for policy 1, policy_version 1827377 (0.0008) [2023-12-27 18:19:00,007][20347] Updated weights for policy 0, policy_version 1828994 (0.0006) [2023-12-27 18:19:00,059][20347] Updated weights for policy 0, policy_version 1829004 (0.0006) [2023-12-27 18:19:00,114][20347] Updated weights for policy 0, policy_version 1829014 (0.0005) [2023-12-27 18:19:00,689][20381] Updated weights for policy 1, policy_version 1827387 (0.0008) [2023-12-27 18:19:00,714][20347] Updated weights for policy 0, policy_version 1829024 (0.0005) [2023-12-27 18:19:00,750][20381] Updated weights for policy 1, policy_version 1827397 (0.0007) [2023-12-27 18:19:00,771][20347] Updated weights for policy 0, policy_version 1829034 (0.0006) [2023-12-27 18:19:00,795][20381] Updated weights for policy 1, policy_version 1827407 (0.0008) [2023-12-27 18:19:00,826][20347] Updated weights for policy 0, policy_version 1829044 (0.0005) [2023-12-27 18:19:01,464][20347] Updated weights for policy 0, policy_version 1829054 (0.0005) [2023-12-27 18:19:01,476][20381] Updated weights for policy 1, policy_version 1827417 (0.0007) [2023-12-27 18:19:01,515][20347] Updated weights for policy 0, policy_version 1829064 (0.0006) [2023-12-27 18:19:01,539][20381] Updated weights for policy 1, policy_version 1827427 (0.0006) [2023-12-27 18:19:01,562][20347] Updated weights for policy 0, policy_version 1829074 (0.0006) [2023-12-27 18:19:01,595][20381] Updated weights for policy 1, policy_version 1827437 (0.0007) [2023-12-27 18:19:01,660][20381] Updated weights for policy 1, policy_version 1827447 (0.0006) [2023-12-27 18:19:02,273][20381] Updated weights for policy 1, policy_version 1827457 (0.0009) [2023-12-27 18:19:02,303][20347] Updated weights for policy 0, policy_version 1829084 (0.0005) [2023-12-27 18:19:02,330][20381] Updated weights for policy 1, policy_version 1827467 (0.0007) [2023-12-27 18:19:02,358][20347] Updated weights for policy 0, policy_version 1829094 (0.0006) [2023-12-27 18:19:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 936206336. Throughput: 0: 10276.0, 1: 10268.8. Samples: 936224088. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '918.570')] [2023-12-27 18:19:02,397][20381] Updated weights for policy 1, policy_version 1827477 (0.0007) [2023-12-27 18:19:02,420][20347] Updated weights for policy 0, policy_version 1829104 (0.0008) [2023-12-27 18:19:03,087][20347] Updated weights for policy 0, policy_version 1829114 (0.0007) [2023-12-27 18:19:03,111][20381] Updated weights for policy 1, policy_version 1827487 (0.0008) [2023-12-27 18:19:03,142][20347] Updated weights for policy 0, policy_version 1829124 (0.0005) [2023-12-27 18:19:03,158][20381] Updated weights for policy 1, policy_version 1827497 (0.0007) [2023-12-27 18:19:03,194][20347] Updated weights for policy 0, policy_version 1829134 (0.0006) [2023-12-27 18:19:03,212][20381] Updated weights for policy 1, policy_version 1827507 (0.0007) [2023-12-27 18:19:03,256][20347] Updated weights for policy 0, policy_version 1829144 (0.0006) [2023-12-27 18:19:03,833][20381] Updated weights for policy 1, policy_version 1827517 (0.0008) [2023-12-27 18:19:03,848][20347] Updated weights for policy 0, policy_version 1829154 (0.0008) [2023-12-27 18:19:03,906][20347] Updated weights for policy 0, policy_version 1829164 (0.0008) [2023-12-27 18:19:03,912][20381] Updated weights for policy 1, policy_version 1827527 (0.0007) [2023-12-27 18:19:03,965][20347] Updated weights for policy 0, policy_version 1829174 (0.0008) [2023-12-27 18:19:03,971][20381] Updated weights for policy 1, policy_version 1827537 (0.0006) [2023-12-27 18:19:04,661][20381] Updated weights for policy 1, policy_version 1827547 (0.0007) [2023-12-27 18:19:04,679][20347] Updated weights for policy 0, policy_version 1829184 (0.0008) [2023-12-27 18:19:04,719][20381] Updated weights for policy 1, policy_version 1827557 (0.0006) [2023-12-27 18:19:04,740][20347] Updated weights for policy 0, policy_version 1829194 (0.0005) [2023-12-27 18:19:04,776][20381] Updated weights for policy 1, policy_version 1827567 (0.0006) [2023-12-27 18:19:04,797][20347] Updated weights for policy 0, policy_version 1829204 (0.0008) [2023-12-27 18:19:05,415][20381] Updated weights for policy 1, policy_version 1827577 (0.0005) [2023-12-27 18:19:05,470][20381] Updated weights for policy 1, policy_version 1827587 (0.0008) [2023-12-27 18:19:05,498][20347] Updated weights for policy 0, policy_version 1829214 (0.0006) [2023-12-27 18:19:05,525][20381] Updated weights for policy 1, policy_version 1827597 (0.0007) [2023-12-27 18:19:05,544][20347] Updated weights for policy 0, policy_version 1829224 (0.0005) [2023-12-27 18:19:05,575][20381] Updated weights for policy 1, policy_version 1827607 (0.0006) [2023-12-27 18:19:05,596][20347] Updated weights for policy 0, policy_version 1829234 (0.0007) [2023-12-27 18:19:06,206][20381] Updated weights for policy 1, policy_version 1827617 (0.0007) [2023-12-27 18:19:06,257][20381] Updated weights for policy 1, policy_version 1827627 (0.0007) [2023-12-27 18:19:06,286][20292] KL-divergence is very high: 102.8957 [2023-12-27 18:19:06,309][20381] Updated weights for policy 1, policy_version 1827638 (0.0008) [2023-12-27 18:19:06,318][20347] Updated weights for policy 0, policy_version 1829244 (0.0008) [2023-12-27 18:19:06,378][20347] Updated weights for policy 0, policy_version 1829254 (0.0008) [2023-12-27 18:19:06,437][20347] Updated weights for policy 0, policy_version 1829264 (0.0009) [2023-12-27 18:19:07,052][20381] Updated weights for policy 1, policy_version 1827648 (0.0008) [2023-12-27 18:19:07,116][20381] Updated weights for policy 1, policy_version 1827658 (0.0008) [2023-12-27 18:19:07,135][20347] Updated weights for policy 0, policy_version 1829274 (0.0007) [2023-12-27 18:19:07,172][20381] Updated weights for policy 1, policy_version 1827668 (0.0007) [2023-12-27 18:19:07,193][20347] Updated weights for policy 0, policy_version 1829284 (0.0007) [2023-12-27 18:19:07,251][20347] Updated weights for policy 0, policy_version 1829294 (0.0008) [2023-12-27 18:19:07,311][20347] Updated weights for policy 0, policy_version 1829304 (0.0007) [2023-12-27 18:19:07,363][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 936321024. Throughput: 0: 10270.0, 1: 10298.0. Samples: 936285608. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '868.930')] [2023-12-27 18:19:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001827672_467935232.pth... [2023-12-27 18:19:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001829304_468385792.pth... [2023-12-27 18:19:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001828088_468074496.pth [2023-12-27 18:19:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001826488_467632128.pth [2023-12-27 18:19:07,801][20381] Updated weights for policy 1, policy_version 1827678 (0.0007) [2023-12-27 18:19:07,850][20381] Updated weights for policy 1, policy_version 1827688 (0.0007) [2023-12-27 18:19:07,898][20381] Updated weights for policy 1, policy_version 1827698 (0.0008) [2023-12-27 18:19:07,976][20347] Updated weights for policy 0, policy_version 1829314 (0.0008) [2023-12-27 18:19:08,030][20347] Updated weights for policy 0, policy_version 1829324 (0.0007) [2023-12-27 18:19:08,089][20347] Updated weights for policy 0, policy_version 1829334 (0.0006) [2023-12-27 18:19:08,589][20381] Updated weights for policy 1, policy_version 1827708 (0.0008) [2023-12-27 18:19:08,649][20381] Updated weights for policy 1, policy_version 1827718 (0.0008) [2023-12-27 18:19:08,708][20381] Updated weights for policy 1, policy_version 1827728 (0.0008) [2023-12-27 18:19:08,808][20347] Updated weights for policy 0, policy_version 1829344 (0.0007) [2023-12-27 18:19:08,864][20347] Updated weights for policy 0, policy_version 1829354 (0.0008) [2023-12-27 18:19:08,919][20347] Updated weights for policy 0, policy_version 1829364 (0.0007) [2023-12-27 18:19:09,399][20381] Updated weights for policy 1, policy_version 1827738 (0.0007) [2023-12-27 18:19:09,463][20381] Updated weights for policy 1, policy_version 1827748 (0.0006) [2023-12-27 18:19:09,526][20381] Updated weights for policy 1, policy_version 1827758 (0.0006) [2023-12-27 18:19:09,589][20381] Updated weights for policy 1, policy_version 1827768 (0.0006) [2023-12-27 18:19:09,601][20347] Updated weights for policy 0, policy_version 1829374 (0.0008) [2023-12-27 18:19:09,661][20347] Updated weights for policy 0, policy_version 1829384 (0.0010) [2023-12-27 18:19:09,718][20347] Updated weights for policy 0, policy_version 1829394 (0.0011) [2023-12-27 18:19:10,203][20381] Updated weights for policy 1, policy_version 1827778 (0.0007) [2023-12-27 18:19:10,266][20381] Updated weights for policy 1, policy_version 1827788 (0.0008) [2023-12-27 18:19:10,328][20381] Updated weights for policy 1, policy_version 1827798 (0.0008) [2023-12-27 18:19:10,454][20347] Updated weights for policy 0, policy_version 1829404 (0.0010) [2023-12-27 18:19:10,509][20347] Updated weights for policy 0, policy_version 1829414 (0.0007) [2023-12-27 18:19:10,568][20347] Updated weights for policy 0, policy_version 1829424 (0.0007) [2023-12-27 18:19:11,011][20381] Updated weights for policy 1, policy_version 1827808 (0.0008) [2023-12-27 18:19:11,073][20381] Updated weights for policy 1, policy_version 1827818 (0.0006) [2023-12-27 18:19:11,091][20292] KL-divergence is very high: 103.9392 [2023-12-27 18:19:11,136][20381] Updated weights for policy 1, policy_version 1827828 (0.0008) [2023-12-27 18:19:11,144][20292] KL-divergence is very high: 127.3035 [2023-12-27 18:19:11,189][20347] Updated weights for policy 0, policy_version 1829434 (0.0006) [2023-12-27 18:19:11,249][20347] Updated weights for policy 0, policy_version 1829444 (0.0008) [2023-12-27 18:19:11,308][20347] Updated weights for policy 0, policy_version 1829454 (0.0008) [2023-12-27 18:19:11,371][20347] Updated weights for policy 0, policy_version 1829464 (0.0008) [2023-12-27 18:19:11,935][20381] Updated weights for policy 1, policy_version 1827838 (0.0009) [2023-12-27 18:19:11,990][20381] Updated weights for policy 1, policy_version 1827848 (0.0009) [2023-12-27 18:19:12,044][20381] Updated weights for policy 1, policy_version 1827858 (0.0007) [2023-12-27 18:19:12,108][20347] Updated weights for policy 0, policy_version 1829474 (0.0006) [2023-12-27 18:19:12,172][20347] Updated weights for policy 0, policy_version 1829484 (0.0007) [2023-12-27 18:19:12,227][20347] Updated weights for policy 0, policy_version 1829494 (0.0007) [2023-12-27 18:19:12,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 936419328. Throughput: 0: 10288.0, 1: 10252.7. Samples: 936407300. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:12,365][19170] Avg episode reward: [(0, '1000.000'), (1, '926.540')] [2023-12-27 18:19:12,720][20381] Updated weights for policy 1, policy_version 1827868 (0.0008) [2023-12-27 18:19:12,779][20381] Updated weights for policy 1, policy_version 1827878 (0.0009) [2023-12-27 18:19:12,835][20381] Updated weights for policy 1, policy_version 1827888 (0.0010) [2023-12-27 18:19:12,893][20347] Updated weights for policy 0, policy_version 1829504 (0.0007) [2023-12-27 18:19:12,944][20347] Updated weights for policy 0, policy_version 1829514 (0.0007) [2023-12-27 18:19:13,006][20347] Updated weights for policy 0, policy_version 1829524 (0.0007) [2023-12-27 18:19:13,436][20381] Updated weights for policy 1, policy_version 1827898 (0.0010) [2023-12-27 18:19:13,491][20381] Updated weights for policy 1, policy_version 1827908 (0.0006) [2023-12-27 18:19:13,545][20381] Updated weights for policy 1, policy_version 1827918 (0.0009) [2023-12-27 18:19:13,601][20381] Updated weights for policy 1, policy_version 1827928 (0.0008) [2023-12-27 18:19:13,606][20347] Updated weights for policy 0, policy_version 1829534 (0.0007) [2023-12-27 18:19:13,657][20347] Updated weights for policy 0, policy_version 1829544 (0.0005) [2023-12-27 18:19:13,709][20347] Updated weights for policy 0, policy_version 1829554 (0.0005) [2023-12-27 18:19:14,259][20347] Updated weights for policy 0, policy_version 1829564 (0.0006) [2023-12-27 18:19:14,306][20381] Updated weights for policy 1, policy_version 1827938 (0.0005) [2023-12-27 18:19:14,314][20347] Updated weights for policy 0, policy_version 1829574 (0.0007) [2023-12-27 18:19:14,373][20381] Updated weights for policy 1, policy_version 1827948 (0.0008) [2023-12-27 18:19:14,379][20347] Updated weights for policy 0, policy_version 1829584 (0.0006) [2023-12-27 18:19:14,438][20381] Updated weights for policy 1, policy_version 1827958 (0.0007) [2023-12-27 18:19:15,053][20347] Updated weights for policy 0, policy_version 1829594 (0.0006) [2023-12-27 18:19:15,110][20347] Updated weights for policy 0, policy_version 1829604 (0.0007) [2023-12-27 18:19:15,112][20381] Updated weights for policy 1, policy_version 1827968 (0.0006) [2023-12-27 18:19:15,160][20347] Updated weights for policy 0, policy_version 1829614 (0.0006) [2023-12-27 18:19:15,177][20381] Updated weights for policy 1, policy_version 1827978 (0.0008) [2023-12-27 18:19:15,223][20347] Updated weights for policy 0, policy_version 1829624 (0.0006) [2023-12-27 18:19:15,242][20381] Updated weights for policy 1, policy_version 1827988 (0.0008) [2023-12-27 18:19:15,858][20381] Updated weights for policy 1, policy_version 1827998 (0.0007) [2023-12-27 18:19:15,911][20381] Updated weights for policy 1, policy_version 1828008 (0.0006) [2023-12-27 18:19:15,925][20347] Updated weights for policy 0, policy_version 1829634 (0.0009) [2023-12-27 18:19:15,965][20381] Updated weights for policy 1, policy_version 1828018 (0.0008) [2023-12-27 18:19:15,979][20347] Updated weights for policy 0, policy_version 1829644 (0.0006) [2023-12-27 18:19:16,028][20347] Updated weights for policy 0, policy_version 1829654 (0.0009) [2023-12-27 18:19:16,589][20381] Updated weights for policy 1, policy_version 1828028 (0.0005) [2023-12-27 18:19:16,646][20381] Updated weights for policy 1, policy_version 1828038 (0.0007) [2023-12-27 18:19:16,681][20347] Updated weights for policy 0, policy_version 1829664 (0.0008) [2023-12-27 18:19:16,695][20381] Updated weights for policy 1, policy_version 1828048 (0.0009) [2023-12-27 18:19:16,730][20347] Updated weights for policy 0, policy_version 1829674 (0.0008) [2023-12-27 18:19:16,778][20347] Updated weights for policy 0, policy_version 1829684 (0.0007) [2023-12-27 18:19:17,352][20381] Updated weights for policy 1, policy_version 1828058 (0.0006) [2023-12-27 18:19:17,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 936517632. Throughput: 0: 10281.1, 1: 10264.5. Samples: 936532316. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '958.380')] [2023-12-27 18:19:17,403][20381] Updated weights for policy 1, policy_version 1828068 (0.0007) [2023-12-27 18:19:17,449][20381] Updated weights for policy 1, policy_version 1828078 (0.0007) [2023-12-27 18:19:17,469][20347] Updated weights for policy 0, policy_version 1829694 (0.0009) [2023-12-27 18:19:17,504][20381] Updated weights for policy 1, policy_version 1828088 (0.0005) [2023-12-27 18:19:17,533][20347] Updated weights for policy 0, policy_version 1829704 (0.0010) [2023-12-27 18:19:17,594][20347] Updated weights for policy 0, policy_version 1829714 (0.0010) [2023-12-27 18:19:18,154][20381] Updated weights for policy 1, policy_version 1828098 (0.0006) [2023-12-27 18:19:18,207][20381] Updated weights for policy 1, policy_version 1828108 (0.0005) [2023-12-27 18:19:18,217][20347] Updated weights for policy 0, policy_version 1829724 (0.0011) [2023-12-27 18:19:18,259][20381] Updated weights for policy 1, policy_version 1828118 (0.0006) [2023-12-27 18:19:18,273][20347] Updated weights for policy 0, policy_version 1829734 (0.0010) [2023-12-27 18:19:18,325][20347] Updated weights for policy 0, policy_version 1829744 (0.0010) [2023-12-27 18:19:18,954][20381] Updated weights for policy 1, policy_version 1828128 (0.0007) [2023-12-27 18:19:18,987][20347] Updated weights for policy 0, policy_version 1829754 (0.0007) [2023-12-27 18:19:19,012][20381] Updated weights for policy 1, policy_version 1828138 (0.0008) [2023-12-27 18:19:19,045][20347] Updated weights for policy 0, policy_version 1829764 (0.0005) [2023-12-27 18:19:19,061][20381] Updated weights for policy 1, policy_version 1828148 (0.0007) [2023-12-27 18:19:19,102][20347] Updated weights for policy 0, policy_version 1829774 (0.0005) [2023-12-27 18:19:19,166][20347] Updated weights for policy 0, policy_version 1829784 (0.0005) [2023-12-27 18:19:19,758][20381] Updated weights for policy 1, policy_version 1828158 (0.0006) [2023-12-27 18:19:19,773][20347] Updated weights for policy 0, policy_version 1829794 (0.0007) [2023-12-27 18:19:19,814][20381] Updated weights for policy 1, policy_version 1828168 (0.0006) [2023-12-27 18:19:19,825][20347] Updated weights for policy 0, policy_version 1829804 (0.0007) [2023-12-27 18:19:19,880][20381] Updated weights for policy 1, policy_version 1828178 (0.0007) [2023-12-27 18:19:19,888][20347] Updated weights for policy 0, policy_version 1829814 (0.0006) [2023-12-27 18:19:20,584][20381] Updated weights for policy 1, policy_version 1828188 (0.0007) [2023-12-27 18:19:20,644][20381] Updated weights for policy 1, policy_version 1828198 (0.0007) [2023-12-27 18:19:20,671][20347] Updated weights for policy 0, policy_version 1829824 (0.0008) [2023-12-27 18:19:20,708][20381] Updated weights for policy 1, policy_version 1828208 (0.0006) [2023-12-27 18:19:20,734][20347] Updated weights for policy 0, policy_version 1829834 (0.0007) [2023-12-27 18:19:20,800][20347] Updated weights for policy 0, policy_version 1829844 (0.0008) [2023-12-27 18:19:21,419][20381] Updated weights for policy 1, policy_version 1828218 (0.0006) [2023-12-27 18:19:21,477][20381] Updated weights for policy 1, policy_version 1828228 (0.0008) [2023-12-27 18:19:21,499][20347] Updated weights for policy 0, policy_version 1829854 (0.0006) [2023-12-27 18:19:21,537][20381] Updated weights for policy 1, policy_version 1828238 (0.0008) [2023-12-27 18:19:21,552][20347] Updated weights for policy 0, policy_version 1829864 (0.0006) [2023-12-27 18:19:21,593][20381] Updated weights for policy 1, policy_version 1828248 (0.0007) [2023-12-27 18:19:21,606][20347] Updated weights for policy 0, policy_version 1829874 (0.0005) [2023-12-27 18:19:22,333][20347] Updated weights for policy 0, policy_version 1829884 (0.0008) [2023-12-27 18:19:22,338][20381] Updated weights for policy 1, policy_version 1828258 (0.0007) [2023-12-27 18:19:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 936615936. Throughput: 0: 10280.4, 1: 10220.6. Samples: 936594036. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.460')] [2023-12-27 18:19:22,402][20381] Updated weights for policy 1, policy_version 1828268 (0.0008) [2023-12-27 18:19:22,408][20347] Updated weights for policy 0, policy_version 1829894 (0.0007) [2023-12-27 18:19:22,466][20381] Updated weights for policy 1, policy_version 1828278 (0.0008) [2023-12-27 18:19:22,469][20347] Updated weights for policy 0, policy_version 1829904 (0.0006) [2023-12-27 18:19:22,476][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001828280_468090880.pth... [2023-12-27 18:19:22,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001827064_467779584.pth [2023-12-27 18:19:22,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001829912_468541440.pth... [2023-12-27 18:19:22,516][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001828696_468230144.pth [2023-12-27 18:19:23,111][20381] Updated weights for policy 1, policy_version 1828288 (0.0006) [2023-12-27 18:19:23,138][20347] Updated weights for policy 0, policy_version 1829914 (0.0006) [2023-12-27 18:19:23,167][20381] Updated weights for policy 1, policy_version 1828298 (0.0005) [2023-12-27 18:19:23,198][20347] Updated weights for policy 0, policy_version 1829924 (0.0007) [2023-12-27 18:19:23,222][20381] Updated weights for policy 1, policy_version 1828308 (0.0005) [2023-12-27 18:19:23,254][20347] Updated weights for policy 0, policy_version 1829934 (0.0007) [2023-12-27 18:19:23,309][20347] Updated weights for policy 0, policy_version 1829944 (0.0008) [2023-12-27 18:19:23,814][20381] Updated weights for policy 1, policy_version 1828318 (0.0008) [2023-12-27 18:19:23,864][20381] Updated weights for policy 1, policy_version 1828328 (0.0009) [2023-12-27 18:19:23,912][20381] Updated weights for policy 1, policy_version 1828338 (0.0010) [2023-12-27 18:19:23,999][20347] Updated weights for policy 0, policy_version 1829954 (0.0007) [2023-12-27 18:19:24,050][20347] Updated weights for policy 0, policy_version 1829964 (0.0007) [2023-12-27 18:19:24,101][20347] Updated weights for policy 0, policy_version 1829974 (0.0007) [2023-12-27 18:19:24,574][20381] Updated weights for policy 1, policy_version 1828348 (0.0010) [2023-12-27 18:19:24,630][20381] Updated weights for policy 1, policy_version 1828358 (0.0011) [2023-12-27 18:19:24,688][20381] Updated weights for policy 1, policy_version 1828368 (0.0010) [2023-12-27 18:19:24,804][20347] Updated weights for policy 0, policy_version 1829984 (0.0007) [2023-12-27 18:19:24,850][20347] Updated weights for policy 0, policy_version 1829994 (0.0007) [2023-12-27 18:19:24,903][20347] Updated weights for policy 0, policy_version 1830004 (0.0007) [2023-12-27 18:19:25,268][20381] Updated weights for policy 1, policy_version 1828378 (0.0010) [2023-12-27 18:19:25,323][20381] Updated weights for policy 1, policy_version 1828388 (0.0005) [2023-12-27 18:19:25,382][20381] Updated weights for policy 1, policy_version 1828398 (0.0008) [2023-12-27 18:19:25,444][20381] Updated weights for policy 1, policy_version 1828408 (0.0006) [2023-12-27 18:19:25,647][20347] Updated weights for policy 0, policy_version 1830014 (0.0007) [2023-12-27 18:19:25,694][20347] Updated weights for policy 0, policy_version 1830024 (0.0007) [2023-12-27 18:19:25,747][20347] Updated weights for policy 0, policy_version 1830034 (0.0007) [2023-12-27 18:19:26,073][20381] Updated weights for policy 1, policy_version 1828418 (0.0010) [2023-12-27 18:19:26,137][20381] Updated weights for policy 1, policy_version 1828428 (0.0009) [2023-12-27 18:19:26,201][20381] Updated weights for policy 1, policy_version 1828438 (0.0007) [2023-12-27 18:19:26,424][20347] Updated weights for policy 0, policy_version 1830044 (0.0007) [2023-12-27 18:19:26,469][20347] Updated weights for policy 0, policy_version 1830054 (0.0007) [2023-12-27 18:19:26,516][20347] Updated weights for policy 0, policy_version 1830064 (0.0007) [2023-12-27 18:19:26,806][20381] Updated weights for policy 1, policy_version 1828448 (0.0007) [2023-12-27 18:19:26,862][20381] Updated weights for policy 1, policy_version 1828458 (0.0006) [2023-12-27 18:19:26,924][20381] Updated weights for policy 1, policy_version 1828468 (0.0005) [2023-12-27 18:19:27,236][20347] Updated weights for policy 0, policy_version 1830074 (0.0007) [2023-12-27 18:19:27,284][20347] Updated weights for policy 0, policy_version 1830084 (0.0007) [2023-12-27 18:19:27,337][20347] Updated weights for policy 0, policy_version 1830094 (0.0007) [2023-12-27 18:19:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 936722432. Throughput: 0: 10205.2, 1: 10257.5. Samples: 936714812. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '990.000')] [2023-12-27 18:19:27,391][20347] Updated weights for policy 0, policy_version 1830104 (0.0007) [2023-12-27 18:19:27,480][20381] Updated weights for policy 1, policy_version 1828478 (0.0005) [2023-12-27 18:19:27,525][20381] Updated weights for policy 1, policy_version 1828488 (0.0005) [2023-12-27 18:19:27,571][20381] Updated weights for policy 1, policy_version 1828498 (0.0007) [2023-12-27 18:19:28,082][20347] Updated weights for policy 0, policy_version 1830114 (0.0007) [2023-12-27 18:19:28,129][20347] Updated weights for policy 0, policy_version 1830124 (0.0007) [2023-12-27 18:19:28,181][20381] Updated weights for policy 1, policy_version 1828508 (0.0008) [2023-12-27 18:19:28,181][20347] Updated weights for policy 0, policy_version 1830134 (0.0007) [2023-12-27 18:19:28,230][20381] Updated weights for policy 1, policy_version 1828518 (0.0005) [2023-12-27 18:19:28,279][20381] Updated weights for policy 1, policy_version 1828528 (0.0005) [2023-12-27 18:19:28,906][20347] Updated weights for policy 0, policy_version 1830144 (0.0007) [2023-12-27 18:19:28,945][20381] Updated weights for policy 1, policy_version 1828538 (0.0006) [2023-12-27 18:19:28,956][20347] Updated weights for policy 0, policy_version 1830154 (0.0007) [2023-12-27 18:19:28,991][20381] Updated weights for policy 1, policy_version 1828548 (0.0011) [2023-12-27 18:19:29,011][20347] Updated weights for policy 0, policy_version 1830164 (0.0005) [2023-12-27 18:19:29,038][20381] Updated weights for policy 1, policy_version 1828558 (0.0009) [2023-12-27 18:19:29,093][20381] Updated weights for policy 1, policy_version 1828568 (0.0006) [2023-12-27 18:19:29,674][20347] Updated weights for policy 0, policy_version 1830174 (0.0007) [2023-12-27 18:19:29,727][20347] Updated weights for policy 0, policy_version 1830184 (0.0008) [2023-12-27 18:19:29,752][20381] Updated weights for policy 1, policy_version 1828578 (0.0009) [2023-12-27 18:19:29,785][20347] Updated weights for policy 0, policy_version 1830194 (0.0008) [2023-12-27 18:19:29,809][20381] Updated weights for policy 1, policy_version 1828588 (0.0008) [2023-12-27 18:19:29,870][20381] Updated weights for policy 1, policy_version 1828598 (0.0008) [2023-12-27 18:19:30,461][20381] Updated weights for policy 1, policy_version 1828608 (0.0005) [2023-12-27 18:19:30,512][20381] Updated weights for policy 1, policy_version 1828618 (0.0005) [2023-12-27 18:19:30,565][20381] Updated weights for policy 1, policy_version 1828628 (0.0005) [2023-12-27 18:19:30,566][20347] Updated weights for policy 0, policy_version 1830204 (0.0007) [2023-12-27 18:19:30,619][20347] Updated weights for policy 0, policy_version 1830214 (0.0006) [2023-12-27 18:19:30,666][20347] Updated weights for policy 0, policy_version 1830224 (0.0008) [2023-12-27 18:19:31,205][20381] Updated weights for policy 1, policy_version 1828638 (0.0008) [2023-12-27 18:19:31,270][20381] Updated weights for policy 1, policy_version 1828648 (0.0008) [2023-12-27 18:19:31,321][20347] Updated weights for policy 0, policy_version 1830234 (0.0007) [2023-12-27 18:19:31,328][20381] Updated weights for policy 1, policy_version 1828658 (0.0007) [2023-12-27 18:19:31,385][20347] Updated weights for policy 0, policy_version 1830244 (0.0007) [2023-12-27 18:19:31,442][20347] Updated weights for policy 0, policy_version 1830254 (0.0008) [2023-12-27 18:19:31,500][20347] Updated weights for policy 0, policy_version 1830264 (0.0007) [2023-12-27 18:19:32,060][20381] Updated weights for policy 1, policy_version 1828668 (0.0008) [2023-12-27 18:19:32,115][20381] Updated weights for policy 1, policy_version 1828678 (0.0007) [2023-12-27 18:19:32,172][20381] Updated weights for policy 1, policy_version 1828688 (0.0006) [2023-12-27 18:19:32,181][20347] Updated weights for policy 0, policy_version 1830274 (0.0007) [2023-12-27 18:19:32,239][20347] Updated weights for policy 0, policy_version 1830284 (0.0006) [2023-12-27 18:19:32,300][20347] Updated weights for policy 0, policy_version 1830294 (0.0008) [2023-12-27 18:19:32,362][19170] Fps is (10 sec: 22118.6, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 936837120. Throughput: 0: 10157.9, 1: 10366.4. Samples: 936840816. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.740')] [2023-12-27 18:19:32,870][20381] Updated weights for policy 1, policy_version 1828698 (0.0007) [2023-12-27 18:19:32,928][20381] Updated weights for policy 1, policy_version 1828708 (0.0010) [2023-12-27 18:19:32,984][20381] Updated weights for policy 1, policy_version 1828718 (0.0010) [2023-12-27 18:19:32,994][20347] Updated weights for policy 0, policy_version 1830304 (0.0006) [2023-12-27 18:19:33,038][20381] Updated weights for policy 1, policy_version 1828728 (0.0009) [2023-12-27 18:19:33,043][20347] Updated weights for policy 0, policy_version 1830314 (0.0006) [2023-12-27 18:19:33,098][20347] Updated weights for policy 0, policy_version 1830324 (0.0007) [2023-12-27 18:19:33,677][20381] Updated weights for policy 1, policy_version 1828738 (0.0009) [2023-12-27 18:19:33,722][20381] Updated weights for policy 1, policy_version 1828748 (0.0010) [2023-12-27 18:19:33,768][20347] Updated weights for policy 0, policy_version 1830334 (0.0006) [2023-12-27 18:19:33,774][20381] Updated weights for policy 1, policy_version 1828758 (0.0010) [2023-12-27 18:19:33,822][20347] Updated weights for policy 0, policy_version 1830344 (0.0007) [2023-12-27 18:19:33,872][20347] Updated weights for policy 0, policy_version 1830354 (0.0007) [2023-12-27 18:19:34,460][20381] Updated weights for policy 1, policy_version 1828768 (0.0010) [2023-12-27 18:19:34,508][20381] Updated weights for policy 1, policy_version 1828778 (0.0007) [2023-12-27 18:19:34,563][20381] Updated weights for policy 1, policy_version 1828788 (0.0008) [2023-12-27 18:19:34,610][20347] Updated weights for policy 0, policy_version 1830364 (0.0008) [2023-12-27 18:19:34,674][20347] Updated weights for policy 0, policy_version 1830374 (0.0005) [2023-12-27 18:19:34,733][20347] Updated weights for policy 0, policy_version 1830384 (0.0006) [2023-12-27 18:19:35,226][20381] Updated weights for policy 1, policy_version 1828798 (0.0008) [2023-12-27 18:19:35,264][20347] Updated weights for policy 0, policy_version 1830394 (0.0006) [2023-12-27 18:19:35,282][20381] Updated weights for policy 1, policy_version 1828808 (0.0010) [2023-12-27 18:19:35,318][20347] Updated weights for policy 0, policy_version 1830404 (0.0008) [2023-12-27 18:19:35,330][20381] Updated weights for policy 1, policy_version 1828818 (0.0010) [2023-12-27 18:19:35,373][20347] Updated weights for policy 0, policy_version 1830414 (0.0008) [2023-12-27 18:19:35,424][20347] Updated weights for policy 0, policy_version 1830424 (0.0010) [2023-12-27 18:19:35,984][20381] Updated weights for policy 1, policy_version 1828828 (0.0011) [2023-12-27 18:19:36,003][20347] Updated weights for policy 0, policy_version 1830434 (0.0009) [2023-12-27 18:19:36,043][20381] Updated weights for policy 1, policy_version 1828838 (0.0011) [2023-12-27 18:19:36,057][20347] Updated weights for policy 0, policy_version 1830444 (0.0010) [2023-12-27 18:19:36,098][20381] Updated weights for policy 1, policy_version 1828848 (0.0010) [2023-12-27 18:19:36,103][20347] Updated weights for policy 0, policy_version 1830454 (0.0010) [2023-12-27 18:19:36,798][20381] Updated weights for policy 1, policy_version 1828858 (0.0010) [2023-12-27 18:19:36,804][20347] Updated weights for policy 0, policy_version 1830464 (0.0006) [2023-12-27 18:19:36,854][20381] Updated weights for policy 1, policy_version 1828868 (0.0011) [2023-12-27 18:19:36,859][20347] Updated weights for policy 0, policy_version 1830474 (0.0009) [2023-12-27 18:19:36,909][20381] Updated weights for policy 1, policy_version 1828878 (0.0011) [2023-12-27 18:19:36,916][20347] Updated weights for policy 0, policy_version 1830484 (0.0010) [2023-12-27 18:19:36,965][20381] Updated weights for policy 1, policy_version 1828888 (0.0011) [2023-12-27 18:19:37,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 936935424. Throughput: 0: 10185.5, 1: 10411.5. Samples: 936903452. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.990')] [2023-12-27 18:19:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001828888_468246528.pth... [2023-12-27 18:19:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001830488_468688896.pth... [2023-12-27 18:19:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001829304_468385792.pth [2023-12-27 18:19:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001827672_467935232.pth [2023-12-27 18:19:37,591][20347] Updated weights for policy 0, policy_version 1830494 (0.0008) [2023-12-27 18:19:37,645][20347] Updated weights for policy 0, policy_version 1830504 (0.0008) [2023-12-27 18:19:37,689][20381] Updated weights for policy 1, policy_version 1828898 (0.0007) [2023-12-27 18:19:37,699][20347] Updated weights for policy 0, policy_version 1830514 (0.0007) [2023-12-27 18:19:37,748][20381] Updated weights for policy 1, policy_version 1828908 (0.0007) [2023-12-27 18:19:37,803][20381] Updated weights for policy 1, policy_version 1828918 (0.0007) [2023-12-27 18:19:38,336][20347] Updated weights for policy 0, policy_version 1830524 (0.0007) [2023-12-27 18:19:38,392][20347] Updated weights for policy 0, policy_version 1830534 (0.0008) [2023-12-27 18:19:38,442][20347] Updated weights for policy 0, policy_version 1830544 (0.0010) [2023-12-27 18:19:38,536][20381] Updated weights for policy 1, policy_version 1828928 (0.0008) [2023-12-27 18:19:38,601][20381] Updated weights for policy 1, policy_version 1828938 (0.0005) [2023-12-27 18:19:38,661][20381] Updated weights for policy 1, policy_version 1828948 (0.0008) [2023-12-27 18:19:39,137][20347] Updated weights for policy 0, policy_version 1830554 (0.0010) [2023-12-27 18:19:39,192][20347] Updated weights for policy 0, policy_version 1830564 (0.0007) [2023-12-27 18:19:39,257][20347] Updated weights for policy 0, policy_version 1830574 (0.0007) [2023-12-27 18:19:39,319][20347] Updated weights for policy 0, policy_version 1830584 (0.0006) [2023-12-27 18:19:39,332][20381] Updated weights for policy 1, policy_version 1828958 (0.0008) [2023-12-27 18:19:39,395][20381] Updated weights for policy 1, policy_version 1828968 (0.0008) [2023-12-27 18:19:39,461][20381] Updated weights for policy 1, policy_version 1828978 (0.0007) [2023-12-27 18:19:40,073][20347] Updated weights for policy 0, policy_version 1830594 (0.0008) [2023-12-27 18:19:40,140][20347] Updated weights for policy 0, policy_version 1830604 (0.0008) [2023-12-27 18:19:40,158][20381] Updated weights for policy 1, policy_version 1828988 (0.0006) [2023-12-27 18:19:40,194][20347] Updated weights for policy 0, policy_version 1830614 (0.0007) [2023-12-27 18:19:40,219][20381] Updated weights for policy 1, policy_version 1828998 (0.0010) [2023-12-27 18:19:40,266][20381] Updated weights for policy 1, policy_version 1829008 (0.0011) [2023-12-27 18:19:40,883][20347] Updated weights for policy 0, policy_version 1830624 (0.0005) [2023-12-27 18:19:40,944][20347] Updated weights for policy 0, policy_version 1830634 (0.0009) [2023-12-27 18:19:40,977][20381] Updated weights for policy 1, policy_version 1829018 (0.0010) [2023-12-27 18:19:40,998][20347] Updated weights for policy 0, policy_version 1830644 (0.0007) [2023-12-27 18:19:41,036][20381] Updated weights for policy 1, policy_version 1829028 (0.0008) [2023-12-27 18:19:41,093][20381] Updated weights for policy 1, policy_version 1829038 (0.0006) [2023-12-27 18:19:41,154][20381] Updated weights for policy 1, policy_version 1829048 (0.0008) [2023-12-27 18:19:41,775][20347] Updated weights for policy 0, policy_version 1830654 (0.0008) [2023-12-27 18:19:41,833][20347] Updated weights for policy 0, policy_version 1830664 (0.0011) [2023-12-27 18:19:41,889][20381] Updated weights for policy 1, policy_version 1829058 (0.0006) [2023-12-27 18:19:41,894][20347] Updated weights for policy 0, policy_version 1830674 (0.0011) [2023-12-27 18:19:41,951][20381] Updated weights for policy 1, policy_version 1829068 (0.0006) [2023-12-27 18:19:42,007][20381] Updated weights for policy 1, policy_version 1829078 (0.0008) [2023-12-27 18:19:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 937033728. Throughput: 0: 10209.3, 1: 10314.8. Samples: 937024376. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:42,363][19170] Avg episode reward: [(0, '990.350'), (1, '961.010')] [2023-12-27 18:19:42,595][20347] Updated weights for policy 0, policy_version 1830684 (0.0009) [2023-12-27 18:19:42,655][20347] Updated weights for policy 0, policy_version 1830694 (0.0007) [2023-12-27 18:19:42,706][20381] Updated weights for policy 1, policy_version 1829088 (0.0007) [2023-12-27 18:19:42,717][20347] Updated weights for policy 0, policy_version 1830704 (0.0006) [2023-12-27 18:19:42,765][20381] Updated weights for policy 1, policy_version 1829098 (0.0007) [2023-12-27 18:19:42,828][20381] Updated weights for policy 1, policy_version 1829108 (0.0008) [2023-12-27 18:19:43,364][20347] Updated weights for policy 0, policy_version 1830714 (0.0006) [2023-12-27 18:19:43,414][20347] Updated weights for policy 0, policy_version 1830724 (0.0009) [2023-12-27 18:19:43,458][20347] Updated weights for policy 0, policy_version 1830734 (0.0008) [2023-12-27 18:19:43,471][20381] Updated weights for policy 1, policy_version 1829118 (0.0006) [2023-12-27 18:19:43,508][20347] Updated weights for policy 0, policy_version 1830744 (0.0007) [2023-12-27 18:19:43,526][20381] Updated weights for policy 1, policy_version 1829128 (0.0006) [2023-12-27 18:19:43,580][20381] Updated weights for policy 1, policy_version 1829138 (0.0008) [2023-12-27 18:19:44,118][20347] Updated weights for policy 0, policy_version 1830754 (0.0005) [2023-12-27 18:19:44,144][20381] Updated weights for policy 1, policy_version 1829148 (0.0008) [2023-12-27 18:19:44,178][20347] Updated weights for policy 0, policy_version 1830764 (0.0006) [2023-12-27 18:19:44,198][20381] Updated weights for policy 1, policy_version 1829158 (0.0007) [2023-12-27 18:19:44,230][20347] Updated weights for policy 0, policy_version 1830774 (0.0005) [2023-12-27 18:19:44,249][20381] Updated weights for policy 1, policy_version 1829168 (0.0008) [2023-12-27 18:19:44,820][20347] Updated weights for policy 0, policy_version 1830784 (0.0007) [2023-12-27 18:19:44,880][20347] Updated weights for policy 0, policy_version 1830794 (0.0007) [2023-12-27 18:19:44,944][20347] Updated weights for policy 0, policy_version 1830804 (0.0007) [2023-12-27 18:19:44,996][20381] Updated weights for policy 1, policy_version 1829178 (0.0009) [2023-12-27 18:19:45,061][20381] Updated weights for policy 1, policy_version 1829188 (0.0008) [2023-12-27 18:19:45,125][20381] Updated weights for policy 1, policy_version 1829198 (0.0007) [2023-12-27 18:19:45,186][20381] Updated weights for policy 1, policy_version 1829208 (0.0006) [2023-12-27 18:19:45,561][20347] Updated weights for policy 0, policy_version 1830814 (0.0006) [2023-12-27 18:19:45,606][20347] Updated weights for policy 0, policy_version 1830824 (0.0005) [2023-12-27 18:19:45,657][20347] Updated weights for policy 0, policy_version 1830834 (0.0005) [2023-12-27 18:19:45,874][20381] Updated weights for policy 1, policy_version 1829218 (0.0007) [2023-12-27 18:19:45,919][20381] Updated weights for policy 1, policy_version 1829228 (0.0008) [2023-12-27 18:19:45,968][20381] Updated weights for policy 1, policy_version 1829238 (0.0008) [2023-12-27 18:19:46,293][20347] Updated weights for policy 0, policy_version 1830844 (0.0005) [2023-12-27 18:19:46,349][20347] Updated weights for policy 0, policy_version 1830854 (0.0005) [2023-12-27 18:19:46,408][20347] Updated weights for policy 0, policy_version 1830864 (0.0005) [2023-12-27 18:19:46,706][20381] Updated weights for policy 1, policy_version 1829248 (0.0007) [2023-12-27 18:19:46,766][20381] Updated weights for policy 1, policy_version 1829258 (0.0008) [2023-12-27 18:19:46,824][20381] Updated weights for policy 1, policy_version 1829268 (0.0008) [2023-12-27 18:19:46,956][20347] Updated weights for policy 0, policy_version 1830874 (0.0006) [2023-12-27 18:19:47,011][20347] Updated weights for policy 0, policy_version 1830884 (0.0005) [2023-12-27 18:19:47,073][20347] Updated weights for policy 0, policy_version 1830894 (0.0006) [2023-12-27 18:19:47,131][20347] Updated weights for policy 0, policy_version 1830904 (0.0007) [2023-12-27 18:19:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 937140224. Throughput: 0: 10268.0, 1: 10298.0. Samples: 937149556. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:47,363][19170] Avg episode reward: [(0, '980.610'), (1, '902.430')] [2023-12-27 18:19:47,580][20381] Updated weights for policy 1, policy_version 1829278 (0.0008) [2023-12-27 18:19:47,630][20381] Updated weights for policy 1, policy_version 1829288 (0.0008) [2023-12-27 18:19:47,678][20381] Updated weights for policy 1, policy_version 1829298 (0.0007) [2023-12-27 18:19:47,683][20347] Updated weights for policy 0, policy_version 1830914 (0.0006) [2023-12-27 18:19:47,737][20347] Updated weights for policy 0, policy_version 1830924 (0.0007) [2023-12-27 18:19:47,793][20347] Updated weights for policy 0, policy_version 1830934 (0.0008) [2023-12-27 18:19:48,323][20381] Updated weights for policy 1, policy_version 1829308 (0.0007) [2023-12-27 18:19:48,386][20381] Updated weights for policy 1, policy_version 1829318 (0.0007) [2023-12-27 18:19:48,454][20381] Updated weights for policy 1, policy_version 1829328 (0.0008) [2023-12-27 18:19:48,502][20347] Updated weights for policy 0, policy_version 1830944 (0.0006) [2023-12-27 18:19:48,566][20347] Updated weights for policy 0, policy_version 1830954 (0.0006) [2023-12-27 18:19:48,625][20347] Updated weights for policy 0, policy_version 1830964 (0.0008) [2023-12-27 18:19:49,138][20381] Updated weights for policy 1, policy_version 1829338 (0.0008) [2023-12-27 18:19:49,189][20381] Updated weights for policy 1, policy_version 1829348 (0.0008) [2023-12-27 18:19:49,209][20292] KL-divergence is very high: 129.3417 [2023-12-27 18:19:49,229][20292] KL-divergence is very high: 162.0603 [2023-12-27 18:19:49,256][20381] Updated weights for policy 1, policy_version 1829358 (0.0007) [2023-12-27 18:19:49,262][20347] Updated weights for policy 0, policy_version 1830974 (0.0007) [2023-12-27 18:19:49,263][20292] KL-divergence is very high: 173.6645 [2023-12-27 18:19:49,280][20292] KL-divergence is very high: 164.8994 [2023-12-27 18:19:49,309][20292] KL-divergence is very high: 130.0704 [2023-12-27 18:19:49,315][20381] Updated weights for policy 1, policy_version 1829368 (0.0007) [2023-12-27 18:19:49,321][20347] Updated weights for policy 0, policy_version 1830984 (0.0009) [2023-12-27 18:19:49,388][20347] Updated weights for policy 0, policy_version 1830994 (0.0009) [2023-12-27 18:19:50,001][20347] Updated weights for policy 0, policy_version 1831004 (0.0006) [2023-12-27 18:19:50,028][20381] Updated weights for policy 1, policy_version 1829378 (0.0007) [2023-12-27 18:19:50,062][20347] Updated weights for policy 0, policy_version 1831014 (0.0006) [2023-12-27 18:19:50,083][20381] Updated weights for policy 1, policy_version 1829388 (0.0007) [2023-12-27 18:19:50,125][20347] Updated weights for policy 0, policy_version 1831024 (0.0006) [2023-12-27 18:19:50,141][20381] Updated weights for policy 1, policy_version 1829398 (0.0006) [2023-12-27 18:19:50,724][20381] Updated weights for policy 1, policy_version 1829408 (0.0007) [2023-12-27 18:19:50,758][20347] Updated weights for policy 0, policy_version 1831034 (0.0007) [2023-12-27 18:19:50,788][20381] Updated weights for policy 1, policy_version 1829418 (0.0008) [2023-12-27 18:19:50,816][20347] Updated weights for policy 0, policy_version 1831044 (0.0008) [2023-12-27 18:19:50,847][20381] Updated weights for policy 1, policy_version 1829428 (0.0008) [2023-12-27 18:19:50,879][20347] Updated weights for policy 0, policy_version 1831054 (0.0005) [2023-12-27 18:19:50,943][20347] Updated weights for policy 0, policy_version 1831064 (0.0005) [2023-12-27 18:19:51,567][20381] Updated weights for policy 1, policy_version 1829438 (0.0009) [2023-12-27 18:19:51,628][20381] Updated weights for policy 1, policy_version 1829448 (0.0007) [2023-12-27 18:19:51,643][20347] Updated weights for policy 0, policy_version 1831074 (0.0008) [2023-12-27 18:19:51,696][20381] Updated weights for policy 1, policy_version 1829458 (0.0006) [2023-12-27 18:19:51,701][20347] Updated weights for policy 0, policy_version 1831084 (0.0008) [2023-12-27 18:19:51,765][20347] Updated weights for policy 0, policy_version 1831094 (0.0009) [2023-12-27 18:19:52,353][20381] Updated weights for policy 1, policy_version 1829468 (0.0007) [2023-12-27 18:19:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 937238528. Throughput: 0: 10308.8, 1: 10290.1. Samples: 937212560. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.490')] [2023-12-27 18:19:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001831096_468844544.pth... [2023-12-27 18:19:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001829912_468541440.pth [2023-12-27 18:19:52,375][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001831096_468844544.pth [2023-12-27 18:19:52,422][20381] Updated weights for policy 1, policy_version 1829478 (0.0008) [2023-12-27 18:19:52,486][20381] Updated weights for policy 1, policy_version 1829488 (0.0006) [2023-12-27 18:19:52,529][20347] Updated weights for policy 0, policy_version 1831104 (0.0007) [2023-12-27 18:19:52,533][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001829496_468402176.pth... [2023-12-27 18:19:52,537][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001828280_468090880.pth [2023-12-27 18:19:52,538][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001829496_468402176.pth [2023-12-27 18:19:52,587][20347] Updated weights for policy 0, policy_version 1831114 (0.0007) [2023-12-27 18:19:52,653][20347] Updated weights for policy 0, policy_version 1831124 (0.0006) [2023-12-27 18:19:53,170][20381] Updated weights for policy 1, policy_version 1829498 (0.0006) [2023-12-27 18:19:53,228][20381] Updated weights for policy 1, policy_version 1829508 (0.0005) [2023-12-27 18:19:53,284][20381] Updated weights for policy 1, policy_version 1829518 (0.0005) [2023-12-27 18:19:53,342][20381] Updated weights for policy 1, policy_version 1829528 (0.0006) [2023-12-27 18:19:53,347][20347] Updated weights for policy 0, policy_version 1831134 (0.0008) [2023-12-27 18:19:53,407][20347] Updated weights for policy 0, policy_version 1831144 (0.0005) [2023-12-27 18:19:53,467][20347] Updated weights for policy 0, policy_version 1831154 (0.0005) [2023-12-27 18:19:53,964][20381] Updated weights for policy 1, policy_version 1829538 (0.0007) [2023-12-27 18:19:54,017][20381] Updated weights for policy 1, policy_version 1829548 (0.0009) [2023-12-27 18:19:54,034][20347] Updated weights for policy 0, policy_version 1831164 (0.0005) [2023-12-27 18:19:54,070][20381] Updated weights for policy 1, policy_version 1829558 (0.0005) [2023-12-27 18:19:54,082][20347] Updated weights for policy 0, policy_version 1831174 (0.0007) [2023-12-27 18:19:54,131][20347] Updated weights for policy 0, policy_version 1831184 (0.0008) [2023-12-27 18:19:54,775][20347] Updated weights for policy 0, policy_version 1831194 (0.0007) [2023-12-27 18:19:54,777][20381] Updated weights for policy 1, policy_version 1829568 (0.0007) [2023-12-27 18:19:54,834][20381] Updated weights for policy 1, policy_version 1829578 (0.0006) [2023-12-27 18:19:54,834][20347] Updated weights for policy 0, policy_version 1831204 (0.0007) [2023-12-27 18:19:54,890][20381] Updated weights for policy 1, policy_version 1829588 (0.0009) [2023-12-27 18:19:54,892][20347] Updated weights for policy 0, policy_version 1831214 (0.0009) [2023-12-27 18:19:54,947][20347] Updated weights for policy 0, policy_version 1831224 (0.0006) [2023-12-27 18:19:55,537][20381] Updated weights for policy 1, policy_version 1829598 (0.0005) [2023-12-27 18:19:55,594][20381] Updated weights for policy 1, policy_version 1829608 (0.0005) [2023-12-27 18:19:55,648][20347] Updated weights for policy 0, policy_version 1831234 (0.0006) [2023-12-27 18:19:55,650][20381] Updated weights for policy 1, policy_version 1829618 (0.0005) [2023-12-27 18:19:55,699][20347] Updated weights for policy 0, policy_version 1831244 (0.0005) [2023-12-27 18:19:55,751][20347] Updated weights for policy 0, policy_version 1831254 (0.0005) [2023-12-27 18:19:56,217][20381] Updated weights for policy 1, policy_version 1829628 (0.0007) [2023-12-27 18:19:56,267][20381] Updated weights for policy 1, policy_version 1829638 (0.0008) [2023-12-27 18:19:56,283][20347] Updated weights for policy 0, policy_version 1831264 (0.0005) [2023-12-27 18:19:56,318][20381] Updated weights for policy 1, policy_version 1829648 (0.0008) [2023-12-27 18:19:56,338][20347] Updated weights for policy 0, policy_version 1831274 (0.0007) [2023-12-27 18:19:56,400][20347] Updated weights for policy 0, policy_version 1831284 (0.0007) [2023-12-27 18:19:56,941][20347] Updated weights for policy 0, policy_version 1831294 (0.0006) [2023-12-27 18:19:56,991][20347] Updated weights for policy 0, policy_version 1831304 (0.0005) [2023-12-27 18:19:57,023][20381] Updated weights for policy 1, policy_version 1829658 (0.0008) [2023-12-27 18:19:57,049][20347] Updated weights for policy 0, policy_version 1831314 (0.0005) [2023-12-27 18:19:57,077][20381] Updated weights for policy 1, policy_version 1829668 (0.0010) [2023-12-27 18:19:57,131][20381] Updated weights for policy 1, policy_version 1829678 (0.0010) [2023-12-27 18:19:57,176][20381] Updated weights for policy 1, policy_version 1829688 (0.0010) [2023-12-27 18:19:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 937353216. Throughput: 0: 10350.4, 1: 10333.4. Samples: 937338072. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:19:57,363][19170] Avg episode reward: [(0, '990.370'), (1, '931.730')] [2023-12-27 18:19:57,632][20347] Updated weights for policy 0, policy_version 1831324 (0.0006) [2023-12-27 18:19:57,683][20347] Updated weights for policy 0, policy_version 1831334 (0.0007) [2023-12-27 18:19:57,731][20347] Updated weights for policy 0, policy_version 1831344 (0.0007) [2023-12-27 18:19:57,864][20381] Updated weights for policy 1, policy_version 1829698 (0.0010) [2023-12-27 18:19:57,908][20381] Updated weights for policy 1, policy_version 1829708 (0.0010) [2023-12-27 18:19:57,973][20381] Updated weights for policy 1, policy_version 1829718 (0.0008) [2023-12-27 18:19:58,410][20347] Updated weights for policy 0, policy_version 1831354 (0.0007) [2023-12-27 18:19:58,476][20347] Updated weights for policy 0, policy_version 1831364 (0.0007) [2023-12-27 18:19:58,541][20347] Updated weights for policy 0, policy_version 1831374 (0.0008) [2023-12-27 18:19:58,604][20347] Updated weights for policy 0, policy_version 1831384 (0.0007) [2023-12-27 18:19:58,766][20381] Updated weights for policy 1, policy_version 1829728 (0.0007) [2023-12-27 18:19:58,829][20381] Updated weights for policy 1, policy_version 1829738 (0.0007) [2023-12-27 18:19:58,903][20381] Updated weights for policy 1, policy_version 1829748 (0.0007) [2023-12-27 18:19:59,415][20347] Updated weights for policy 0, policy_version 1831394 (0.0007) [2023-12-27 18:19:59,472][20347] Updated weights for policy 0, policy_version 1831404 (0.0007) [2023-12-27 18:19:59,524][20347] Updated weights for policy 0, policy_version 1831414 (0.0008) [2023-12-27 18:19:59,570][20381] Updated weights for policy 1, policy_version 1829758 (0.0005) [2023-12-27 18:19:59,625][20381] Updated weights for policy 1, policy_version 1829768 (0.0007) [2023-12-27 18:19:59,674][20381] Updated weights for policy 1, policy_version 1829778 (0.0008) [2023-12-27 18:20:00,248][20347] Updated weights for policy 0, policy_version 1831424 (0.0006) [2023-12-27 18:20:00,310][20347] Updated weights for policy 0, policy_version 1831434 (0.0005) [2023-12-27 18:20:00,310][20381] Updated weights for policy 1, policy_version 1829788 (0.0008) [2023-12-27 18:20:00,367][20347] Updated weights for policy 0, policy_version 1831444 (0.0005) [2023-12-27 18:20:00,372][20381] Updated weights for policy 1, policy_version 1829798 (0.0007) [2023-12-27 18:20:00,438][20381] Updated weights for policy 1, policy_version 1829808 (0.0007) [2023-12-27 18:20:00,937][20347] Updated weights for policy 0, policy_version 1831454 (0.0005) [2023-12-27 18:20:00,983][20347] Updated weights for policy 0, policy_version 1831464 (0.0005) [2023-12-27 18:20:01,006][20381] Updated weights for policy 1, policy_version 1829818 (0.0006) [2023-12-27 18:20:01,038][20347] Updated weights for policy 0, policy_version 1831474 (0.0006) [2023-12-27 18:20:01,067][20381] Updated weights for policy 1, policy_version 1829828 (0.0009) [2023-12-27 18:20:01,131][20381] Updated weights for policy 1, policy_version 1829838 (0.0006) [2023-12-27 18:20:01,195][20381] Updated weights for policy 1, policy_version 1829848 (0.0009) [2023-12-27 18:20:01,743][20347] Updated weights for policy 0, policy_version 1831484 (0.0007) [2023-12-27 18:20:01,798][20347] Updated weights for policy 0, policy_version 1831494 (0.0007) [2023-12-27 18:20:01,857][20347] Updated weights for policy 0, policy_version 1831504 (0.0007) [2023-12-27 18:20:01,914][20381] Updated weights for policy 1, policy_version 1829858 (0.0006) [2023-12-27 18:20:01,978][20381] Updated weights for policy 1, policy_version 1829868 (0.0008) [2023-12-27 18:20:02,035][20381] Updated weights for policy 1, policy_version 1829878 (0.0007) [2023-12-27 18:20:02,362][19170] Fps is (10 sec: 21299.8, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 937451520. Throughput: 0: 10367.4, 1: 10298.3. Samples: 937462272. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:02,363][19170] Avg episode reward: [(0, '990.370'), (1, '941.470')] [2023-12-27 18:20:02,568][20347] Updated weights for policy 0, policy_version 1831514 (0.0006) [2023-12-27 18:20:02,619][20347] Updated weights for policy 0, policy_version 1831524 (0.0007) [2023-12-27 18:20:02,677][20347] Updated weights for policy 0, policy_version 1831534 (0.0008) [2023-12-27 18:20:02,683][20381] Updated weights for policy 1, policy_version 1829888 (0.0008) [2023-12-27 18:20:02,731][20347] Updated weights for policy 0, policy_version 1831544 (0.0008) [2023-12-27 18:20:02,736][20381] Updated weights for policy 1, policy_version 1829898 (0.0008) [2023-12-27 18:20:02,786][20381] Updated weights for policy 1, policy_version 1829908 (0.0008) [2023-12-27 18:20:03,412][20347] Updated weights for policy 0, policy_version 1831554 (0.0005) [2023-12-27 18:20:03,464][20347] Updated weights for policy 0, policy_version 1831564 (0.0007) [2023-12-27 18:20:03,492][20381] Updated weights for policy 1, policy_version 1829918 (0.0007) [2023-12-27 18:20:03,517][20347] Updated weights for policy 0, policy_version 1831574 (0.0007) [2023-12-27 18:20:03,541][20381] Updated weights for policy 1, policy_version 1829928 (0.0007) [2023-12-27 18:20:03,592][20381] Updated weights for policy 1, policy_version 1829938 (0.0007) [2023-12-27 18:20:04,136][20347] Updated weights for policy 0, policy_version 1831584 (0.0007) [2023-12-27 18:20:04,196][20347] Updated weights for policy 0, policy_version 1831594 (0.0006) [2023-12-27 18:20:04,250][20347] Updated weights for policy 0, policy_version 1831604 (0.0007) [2023-12-27 18:20:04,325][20381] Updated weights for policy 1, policy_version 1829948 (0.0010) [2023-12-27 18:20:04,384][20381] Updated weights for policy 1, policy_version 1829958 (0.0007) [2023-12-27 18:20:04,443][20381] Updated weights for policy 1, policy_version 1829968 (0.0010) [2023-12-27 18:20:04,942][20347] Updated weights for policy 0, policy_version 1831614 (0.0005) [2023-12-27 18:20:04,996][20347] Updated weights for policy 0, policy_version 1831624 (0.0007) [2023-12-27 18:20:05,055][20347] Updated weights for policy 0, policy_version 1831634 (0.0007) [2023-12-27 18:20:05,098][20381] Updated weights for policy 1, policy_version 1829978 (0.0008) [2023-12-27 18:20:05,149][20381] Updated weights for policy 1, policy_version 1829988 (0.0010) [2023-12-27 18:20:05,207][20381] Updated weights for policy 1, policy_version 1829998 (0.0010) [2023-12-27 18:20:05,263][20381] Updated weights for policy 1, policy_version 1830008 (0.0010) [2023-12-27 18:20:05,669][20347] Updated weights for policy 0, policy_version 1831644 (0.0006) [2023-12-27 18:20:05,719][20347] Updated weights for policy 0, policy_version 1831654 (0.0007) [2023-12-27 18:20:05,774][20347] Updated weights for policy 0, policy_version 1831664 (0.0008) [2023-12-27 18:20:05,910][20381] Updated weights for policy 1, policy_version 1830018 (0.0006) [2023-12-27 18:20:05,965][20381] Updated weights for policy 1, policy_version 1830028 (0.0006) [2023-12-27 18:20:06,026][20381] Updated weights for policy 1, policy_version 1830038 (0.0006) [2023-12-27 18:20:06,407][20347] Updated weights for policy 0, policy_version 1831674 (0.0008) [2023-12-27 18:20:06,464][20347] Updated weights for policy 0, policy_version 1831684 (0.0006) [2023-12-27 18:20:06,520][20347] Updated weights for policy 0, policy_version 1831694 (0.0006) [2023-12-27 18:20:06,582][20347] Updated weights for policy 0, policy_version 1831704 (0.0007) [2023-12-27 18:20:06,694][20381] Updated weights for policy 1, policy_version 1830048 (0.0008) [2023-12-27 18:20:06,744][20381] Updated weights for policy 1, policy_version 1830058 (0.0008) [2023-12-27 18:20:06,799][20381] Updated weights for policy 1, policy_version 1830068 (0.0005) [2023-12-27 18:20:07,136][20347] Updated weights for policy 0, policy_version 1831714 (0.0007) [2023-12-27 18:20:07,191][20347] Updated weights for policy 0, policy_version 1831724 (0.0007) [2023-12-27 18:20:07,249][20347] Updated weights for policy 0, policy_version 1831734 (0.0007) [2023-12-27 18:20:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 937558016. Throughput: 0: 10361.1, 1: 10300.3. Samples: 937523804. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '922.020')] [2023-12-27 18:20:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001831736_469008384.pth... [2023-12-27 18:20:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001830072_468549632.pth... [2023-12-27 18:20:07,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001830488_468688896.pth [2023-12-27 18:20:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001828888_468246528.pth [2023-12-27 18:20:07,475][20381] Updated weights for policy 1, policy_version 1830078 (0.0009) [2023-12-27 18:20:07,527][20381] Updated weights for policy 1, policy_version 1830088 (0.0011) [2023-12-27 18:20:07,581][20381] Updated weights for policy 1, policy_version 1830098 (0.0011) [2023-12-27 18:20:07,899][20347] Updated weights for policy 0, policy_version 1831744 (0.0007) [2023-12-27 18:20:07,961][20347] Updated weights for policy 0, policy_version 1831754 (0.0007) [2023-12-27 18:20:08,021][20347] Updated weights for policy 0, policy_version 1831764 (0.0007) [2023-12-27 18:20:08,259][20381] Updated weights for policy 1, policy_version 1830108 (0.0006) [2023-12-27 18:20:08,322][20381] Updated weights for policy 1, policy_version 1830118 (0.0006) [2023-12-27 18:20:08,385][20381] Updated weights for policy 1, policy_version 1830128 (0.0008) [2023-12-27 18:20:08,733][20347] Updated weights for policy 0, policy_version 1831774 (0.0007) [2023-12-27 18:20:08,795][20347] Updated weights for policy 0, policy_version 1831784 (0.0007) [2023-12-27 18:20:08,852][20347] Updated weights for policy 0, policy_version 1831794 (0.0007) [2023-12-27 18:20:09,039][20381] Updated weights for policy 1, policy_version 1830138 (0.0007) [2023-12-27 18:20:09,097][20381] Updated weights for policy 1, policy_version 1830148 (0.0005) [2023-12-27 18:20:09,151][20381] Updated weights for policy 1, policy_version 1830158 (0.0006) [2023-12-27 18:20:09,209][20381] Updated weights for policy 1, policy_version 1830168 (0.0010) [2023-12-27 18:20:09,573][20347] Updated weights for policy 0, policy_version 1831804 (0.0007) [2023-12-27 18:20:09,623][20347] Updated weights for policy 0, policy_version 1831814 (0.0008) [2023-12-27 18:20:09,669][20347] Updated weights for policy 0, policy_version 1831824 (0.0007) [2023-12-27 18:20:09,953][20381] Updated weights for policy 1, policy_version 1830178 (0.0008) [2023-12-27 18:20:10,012][20381] Updated weights for policy 1, policy_version 1830188 (0.0008) [2023-12-27 18:20:10,061][20381] Updated weights for policy 1, policy_version 1830198 (0.0008) [2023-12-27 18:20:10,402][20347] Updated weights for policy 0, policy_version 1831834 (0.0008) [2023-12-27 18:20:10,459][20347] Updated weights for policy 0, policy_version 1831844 (0.0008) [2023-12-27 18:20:10,518][20347] Updated weights for policy 0, policy_version 1831854 (0.0008) [2023-12-27 18:20:10,580][20347] Updated weights for policy 0, policy_version 1831864 (0.0007) [2023-12-27 18:20:10,777][20381] Updated weights for policy 1, policy_version 1830208 (0.0008) [2023-12-27 18:20:10,828][20381] Updated weights for policy 1, policy_version 1830218 (0.0007) [2023-12-27 18:20:10,877][20381] Updated weights for policy 1, policy_version 1830228 (0.0006) [2023-12-27 18:20:11,288][20347] Updated weights for policy 0, policy_version 1831874 (0.0008) [2023-12-27 18:20:11,357][20347] Updated weights for policy 0, policy_version 1831884 (0.0008) [2023-12-27 18:20:11,423][20347] Updated weights for policy 0, policy_version 1831894 (0.0008) [2023-12-27 18:20:11,594][20381] Updated weights for policy 1, policy_version 1830238 (0.0007) [2023-12-27 18:20:11,654][20381] Updated weights for policy 1, policy_version 1830248 (0.0008) [2023-12-27 18:20:11,722][20381] Updated weights for policy 1, policy_version 1830258 (0.0009) [2023-12-27 18:20:12,099][20347] Updated weights for policy 0, policy_version 1831904 (0.0008) [2023-12-27 18:20:12,157][20347] Updated weights for policy 0, policy_version 1831914 (0.0008) [2023-12-27 18:20:12,217][20347] Updated weights for policy 0, policy_version 1831924 (0.0008) [2023-12-27 18:20:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 937656320. Throughput: 0: 10433.2, 1: 10261.2. Samples: 937646064. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:12,363][19170] Avg episode reward: [(0, '990.260'), (1, '941.500')] [2023-12-27 18:20:12,462][20381] Updated weights for policy 1, policy_version 1830268 (0.0009) [2023-12-27 18:20:12,521][20381] Updated weights for policy 1, policy_version 1830278 (0.0007) [2023-12-27 18:20:12,585][20381] Updated weights for policy 1, policy_version 1830288 (0.0007) [2023-12-27 18:20:12,895][20347] Updated weights for policy 0, policy_version 1831934 (0.0007) [2023-12-27 18:20:12,955][20347] Updated weights for policy 0, policy_version 1831944 (0.0007) [2023-12-27 18:20:13,010][20347] Updated weights for policy 0, policy_version 1831954 (0.0008) [2023-12-27 18:20:13,257][20381] Updated weights for policy 1, policy_version 1830298 (0.0007) [2023-12-27 18:20:13,315][20381] Updated weights for policy 1, policy_version 1830308 (0.0005) [2023-12-27 18:20:13,376][20381] Updated weights for policy 1, policy_version 1830318 (0.0007) [2023-12-27 18:20:13,428][20381] Updated weights for policy 1, policy_version 1830328 (0.0011) [2023-12-27 18:20:13,750][20347] Updated weights for policy 0, policy_version 1831964 (0.0008) [2023-12-27 18:20:13,807][20347] Updated weights for policy 0, policy_version 1831974 (0.0008) [2023-12-27 18:20:13,860][20347] Updated weights for policy 0, policy_version 1831984 (0.0007) [2023-12-27 18:20:13,973][20381] Updated weights for policy 1, policy_version 1830338 (0.0010) [2023-12-27 18:20:14,027][20381] Updated weights for policy 1, policy_version 1830348 (0.0010) [2023-12-27 18:20:14,082][20381] Updated weights for policy 1, policy_version 1830358 (0.0010) [2023-12-27 18:20:14,555][20347] Updated weights for policy 0, policy_version 1831994 (0.0007) [2023-12-27 18:20:14,605][20347] Updated weights for policy 0, policy_version 1832004 (0.0007) [2023-12-27 18:20:14,656][20347] Updated weights for policy 0, policy_version 1832014 (0.0007) [2023-12-27 18:20:14,712][20347] Updated weights for policy 0, policy_version 1832024 (0.0007) [2023-12-27 18:20:14,752][20381] Updated weights for policy 1, policy_version 1830368 (0.0009) [2023-12-27 18:20:14,814][20381] Updated weights for policy 1, policy_version 1830378 (0.0008) [2023-12-27 18:20:14,871][20381] Updated weights for policy 1, policy_version 1830388 (0.0009) [2023-12-27 18:20:15,500][20347] Updated weights for policy 0, policy_version 1832034 (0.0007) [2023-12-27 18:20:15,540][20381] Updated weights for policy 1, policy_version 1830398 (0.0009) [2023-12-27 18:20:15,557][20347] Updated weights for policy 0, policy_version 1832044 (0.0008) [2023-12-27 18:20:15,597][20381] Updated weights for policy 1, policy_version 1830408 (0.0011) [2023-12-27 18:20:15,615][20347] Updated weights for policy 0, policy_version 1832054 (0.0006) [2023-12-27 18:20:15,651][20381] Updated weights for policy 1, policy_version 1830418 (0.0011) [2023-12-27 18:20:16,298][20347] Updated weights for policy 0, policy_version 1832064 (0.0006) [2023-12-27 18:20:16,318][20381] Updated weights for policy 1, policy_version 1830428 (0.0011) [2023-12-27 18:20:16,342][20347] Updated weights for policy 0, policy_version 1832074 (0.0005) [2023-12-27 18:20:16,367][20381] Updated weights for policy 1, policy_version 1830438 (0.0010) [2023-12-27 18:20:16,390][20347] Updated weights for policy 0, policy_version 1832084 (0.0006) [2023-12-27 18:20:16,419][20381] Updated weights for policy 1, policy_version 1830448 (0.0011) [2023-12-27 18:20:17,097][20381] Updated weights for policy 1, policy_version 1830458 (0.0009) [2023-12-27 18:20:17,119][20347] Updated weights for policy 0, policy_version 1832094 (0.0007) [2023-12-27 18:20:17,151][20381] Updated weights for policy 1, policy_version 1830468 (0.0005) [2023-12-27 18:20:17,168][20347] Updated weights for policy 0, policy_version 1832104 (0.0008) [2023-12-27 18:20:17,204][20381] Updated weights for policy 1, policy_version 1830478 (0.0007) [2023-12-27 18:20:17,215][20347] Updated weights for policy 0, policy_version 1832114 (0.0007) [2023-12-27 18:20:17,254][20292] KL-divergence is very high: 120.1338 [2023-12-27 18:20:17,259][20381] Updated weights for policy 1, policy_version 1830488 (0.0007) [2023-12-27 18:20:17,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 937762816. Throughput: 0: 10390.8, 1: 10199.4. Samples: 937767380. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:17,364][19170] Avg episode reward: [(0, '990.260'), (1, '941.230')] [2023-12-27 18:20:17,895][20292] KL-divergence is very high: 116.5769 [2023-12-27 18:20:17,897][20347] Updated weights for policy 0, policy_version 1832124 (0.0007) [2023-12-27 18:20:17,943][20381] Updated weights for policy 1, policy_version 1830498 (0.0006) [2023-12-27 18:20:17,945][20347] Updated weights for policy 0, policy_version 1832134 (0.0005) [2023-12-27 18:20:18,001][20347] Updated weights for policy 0, policy_version 1832144 (0.0005) [2023-12-27 18:20:18,003][20381] Updated weights for policy 1, policy_version 1830508 (0.0006) [2023-12-27 18:20:18,060][20381] Updated weights for policy 1, policy_version 1830518 (0.0006) [2023-12-27 18:20:18,760][20381] Updated weights for policy 1, policy_version 1830528 (0.0007) [2023-12-27 18:20:18,798][20347] Updated weights for policy 0, policy_version 1832154 (0.0007) [2023-12-27 18:20:18,822][20381] Updated weights for policy 1, policy_version 1830538 (0.0007) [2023-12-27 18:20:18,855][20347] Updated weights for policy 0, policy_version 1832164 (0.0008) [2023-12-27 18:20:18,884][20381] Updated weights for policy 1, policy_version 1830548 (0.0007) [2023-12-27 18:20:18,914][20347] Updated weights for policy 0, policy_version 1832174 (0.0008) [2023-12-27 18:20:18,966][20347] Updated weights for policy 0, policy_version 1832184 (0.0008) [2023-12-27 18:20:19,533][20381] Updated weights for policy 1, policy_version 1830558 (0.0007) [2023-12-27 18:20:19,595][20381] Updated weights for policy 1, policy_version 1830568 (0.0008) [2023-12-27 18:20:19,659][20381] Updated weights for policy 1, policy_version 1830578 (0.0006) [2023-12-27 18:20:19,716][20347] Updated weights for policy 0, policy_version 1832194 (0.0008) [2023-12-27 18:20:19,780][20347] Updated weights for policy 0, policy_version 1832204 (0.0008) [2023-12-27 18:20:19,845][20347] Updated weights for policy 0, policy_version 1832214 (0.0007) [2023-12-27 18:20:20,321][20381] Updated weights for policy 1, policy_version 1830588 (0.0006) [2023-12-27 18:20:20,381][20381] Updated weights for policy 1, policy_version 1830598 (0.0007) [2023-12-27 18:20:20,441][20381] Updated weights for policy 1, policy_version 1830608 (0.0008) [2023-12-27 18:20:20,612][20347] Updated weights for policy 0, policy_version 1832224 (0.0008) [2023-12-27 18:20:20,671][20347] Updated weights for policy 0, policy_version 1832234 (0.0007) [2023-12-27 18:20:20,728][20347] Updated weights for policy 0, policy_version 1832244 (0.0007) [2023-12-27 18:20:21,216][20381] Updated weights for policy 1, policy_version 1830618 (0.0008) [2023-12-27 18:20:21,281][20381] Updated weights for policy 1, policy_version 1830628 (0.0007) [2023-12-27 18:20:21,335][20381] Updated weights for policy 1, policy_version 1830638 (0.0007) [2023-12-27 18:20:21,405][20381] Updated weights for policy 1, policy_version 1830648 (0.0007) [2023-12-27 18:20:21,502][20347] Updated weights for policy 0, policy_version 1832254 (0.0010) [2023-12-27 18:20:21,554][20347] Updated weights for policy 0, policy_version 1832264 (0.0009) [2023-12-27 18:20:21,602][20291] KL-divergence is very high: 104.0204 [2023-12-27 18:20:21,615][20347] Updated weights for policy 0, policy_version 1832274 (0.0011) [2023-12-27 18:20:22,073][20381] Updated weights for policy 1, policy_version 1830658 (0.0008) [2023-12-27 18:20:22,139][20381] Updated weights for policy 1, policy_version 1830668 (0.0008) [2023-12-27 18:20:22,198][20381] Updated weights for policy 1, policy_version 1830678 (0.0008) [2023-12-27 18:20:22,324][20347] Updated weights for policy 0, policy_version 1832284 (0.0009) [2023-12-27 18:20:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 937852928. Throughput: 0: 10332.0, 1: 10163.0. Samples: 937825728. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:22,363][19170] Avg episode reward: [(0, '990.260'), (1, '911.930')] [2023-12-27 18:20:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001830680_468705280.pth... [2023-12-27 18:20:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001829496_468402176.pth [2023-12-27 18:20:22,396][20347] Updated weights for policy 0, policy_version 1832294 (0.0011) [2023-12-27 18:20:22,459][20347] Updated weights for policy 0, policy_version 1832304 (0.0010) [2023-12-27 18:20:22,505][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001832312_469155840.pth... [2023-12-27 18:20:22,509][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001831096_468844544.pth [2023-12-27 18:20:22,924][20381] Updated weights for policy 1, policy_version 1830688 (0.0007) [2023-12-27 18:20:22,979][20381] Updated weights for policy 1, policy_version 1830698 (0.0008) [2023-12-27 18:20:23,042][20381] Updated weights for policy 1, policy_version 1830708 (0.0007) [2023-12-27 18:20:23,149][20347] Updated weights for policy 0, policy_version 1832314 (0.0010) [2023-12-27 18:20:23,207][20347] Updated weights for policy 0, policy_version 1832324 (0.0007) [2023-12-27 18:20:23,268][20347] Updated weights for policy 0, policy_version 1832334 (0.0006) [2023-12-27 18:20:23,324][20347] Updated weights for policy 0, policy_version 1832344 (0.0006) [2023-12-27 18:20:23,754][20381] Updated weights for policy 1, policy_version 1830718 (0.0009) [2023-12-27 18:20:23,809][20381] Updated weights for policy 1, policy_version 1830728 (0.0008) [2023-12-27 18:20:23,868][20381] Updated weights for policy 1, policy_version 1830738 (0.0007) [2023-12-27 18:20:23,967][20347] Updated weights for policy 0, policy_version 1832354 (0.0008) [2023-12-27 18:20:24,027][20347] Updated weights for policy 0, policy_version 1832364 (0.0007) [2023-12-27 18:20:24,090][20347] Updated weights for policy 0, policy_version 1832374 (0.0008) [2023-12-27 18:20:24,465][20381] Updated weights for policy 1, policy_version 1830748 (0.0006) [2023-12-27 18:20:24,514][20381] Updated weights for policy 1, policy_version 1830758 (0.0007) [2023-12-27 18:20:24,558][20381] Updated weights for policy 1, policy_version 1830768 (0.0005) [2023-12-27 18:20:24,784][20347] Updated weights for policy 0, policy_version 1832384 (0.0008) [2023-12-27 18:20:24,848][20347] Updated weights for policy 0, policy_version 1832394 (0.0008) [2023-12-27 18:20:24,896][20347] Updated weights for policy 0, policy_version 1832404 (0.0005) [2023-12-27 18:20:25,218][20381] Updated weights for policy 1, policy_version 1830778 (0.0007) [2023-12-27 18:20:25,279][20381] Updated weights for policy 1, policy_version 1830788 (0.0005) [2023-12-27 18:20:25,324][20381] Updated weights for policy 1, policy_version 1830798 (0.0005) [2023-12-27 18:20:25,385][20381] Updated weights for policy 1, policy_version 1830808 (0.0007) [2023-12-27 18:20:25,522][20347] Updated weights for policy 0, policy_version 1832414 (0.0005) [2023-12-27 18:20:25,581][20347] Updated weights for policy 0, policy_version 1832424 (0.0005) [2023-12-27 18:20:25,633][20347] Updated weights for policy 0, policy_version 1832434 (0.0006) [2023-12-27 18:20:26,052][20381] Updated weights for policy 1, policy_version 1830818 (0.0006) [2023-12-27 18:20:26,129][20381] Updated weights for policy 1, policy_version 1830828 (0.0008) [2023-12-27 18:20:26,187][20381] Updated weights for policy 1, policy_version 1830838 (0.0007) [2023-12-27 18:20:26,293][20347] Updated weights for policy 0, policy_version 1832444 (0.0007) [2023-12-27 18:20:26,341][20347] Updated weights for policy 0, policy_version 1832454 (0.0007) [2023-12-27 18:20:26,392][20347] Updated weights for policy 0, policy_version 1832464 (0.0007) [2023-12-27 18:20:26,827][20381] Updated weights for policy 1, policy_version 1830848 (0.0008) [2023-12-27 18:20:26,879][20381] Updated weights for policy 1, policy_version 1830858 (0.0008) [2023-12-27 18:20:26,926][20381] Updated weights for policy 1, policy_version 1830868 (0.0007) [2023-12-27 18:20:27,089][20347] Updated weights for policy 0, policy_version 1832474 (0.0007) [2023-12-27 18:20:27,146][20347] Updated weights for policy 0, policy_version 1832484 (0.0007) [2023-12-27 18:20:27,200][20347] Updated weights for policy 0, policy_version 1832494 (0.0007) [2023-12-27 18:20:27,258][20347] Updated weights for policy 0, policy_version 1832504 (0.0008) [2023-12-27 18:20:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 937959424. Throughput: 0: 10298.7, 1: 10190.7. Samples: 937946396. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:27,363][19170] Avg episode reward: [(0, '980.570'), (1, '950.970')] [2023-12-27 18:20:27,576][20381] Updated weights for policy 1, policy_version 1830878 (0.0007) [2023-12-27 18:20:27,620][20381] Updated weights for policy 1, policy_version 1830888 (0.0006) [2023-12-27 18:20:27,675][20381] Updated weights for policy 1, policy_version 1830898 (0.0006) [2023-12-27 18:20:27,985][20347] Updated weights for policy 0, policy_version 1832514 (0.0005) [2023-12-27 18:20:28,033][20347] Updated weights for policy 0, policy_version 1832524 (0.0005) [2023-12-27 18:20:28,096][20347] Updated weights for policy 0, policy_version 1832534 (0.0006) [2023-12-27 18:20:28,296][20381] Updated weights for policy 1, policy_version 1830908 (0.0006) [2023-12-27 18:20:28,352][20381] Updated weights for policy 1, policy_version 1830918 (0.0007) [2023-12-27 18:20:28,409][20381] Updated weights for policy 1, policy_version 1830928 (0.0005) [2023-12-27 18:20:28,710][20347] Updated weights for policy 0, policy_version 1832544 (0.0009) [2023-12-27 18:20:28,765][20347] Updated weights for policy 0, policy_version 1832554 (0.0010) [2023-12-27 18:20:28,821][20347] Updated weights for policy 0, policy_version 1832564 (0.0010) [2023-12-27 18:20:29,031][20381] Updated weights for policy 1, policy_version 1830938 (0.0005) [2023-12-27 18:20:29,078][20381] Updated weights for policy 1, policy_version 1830948 (0.0007) [2023-12-27 18:20:29,132][20381] Updated weights for policy 1, policy_version 1830958 (0.0007) [2023-12-27 18:20:29,190][20381] Updated weights for policy 1, policy_version 1830968 (0.0006) [2023-12-27 18:20:29,539][20347] Updated weights for policy 0, policy_version 1832574 (0.0010) [2023-12-27 18:20:29,588][20347] Updated weights for policy 0, policy_version 1832584 (0.0010) [2023-12-27 18:20:29,639][20347] Updated weights for policy 0, policy_version 1832594 (0.0010) [2023-12-27 18:20:29,855][20381] Updated weights for policy 1, policy_version 1830978 (0.0009) [2023-12-27 18:20:29,913][20381] Updated weights for policy 1, policy_version 1830988 (0.0007) [2023-12-27 18:20:29,973][20381] Updated weights for policy 1, policy_version 1830998 (0.0007) [2023-12-27 18:20:30,337][20347] Updated weights for policy 0, policy_version 1832604 (0.0009) [2023-12-27 18:20:30,392][20347] Updated weights for policy 0, policy_version 1832614 (0.0008) [2023-12-27 18:20:30,444][20347] Updated weights for policy 0, policy_version 1832624 (0.0008) [2023-12-27 18:20:30,613][20381] Updated weights for policy 1, policy_version 1831008 (0.0008) [2023-12-27 18:20:30,677][20381] Updated weights for policy 1, policy_version 1831018 (0.0008) [2023-12-27 18:20:30,732][20381] Updated weights for policy 1, policy_version 1831028 (0.0008) [2023-12-27 18:20:31,192][20347] Updated weights for policy 0, policy_version 1832634 (0.0009) [2023-12-27 18:20:31,258][20347] Updated weights for policy 0, policy_version 1832644 (0.0008) [2023-12-27 18:20:31,312][20347] Updated weights for policy 0, policy_version 1832654 (0.0008) [2023-12-27 18:20:31,376][20347] Updated weights for policy 0, policy_version 1832664 (0.0008) [2023-12-27 18:20:31,435][20381] Updated weights for policy 1, policy_version 1831038 (0.0007) [2023-12-27 18:20:31,495][20381] Updated weights for policy 1, policy_version 1831048 (0.0007) [2023-12-27 18:20:31,554][20381] Updated weights for policy 1, policy_version 1831058 (0.0007) [2023-12-27 18:20:32,020][20347] Updated weights for policy 0, policy_version 1832674 (0.0007) [2023-12-27 18:20:32,082][20347] Updated weights for policy 0, policy_version 1832684 (0.0006) [2023-12-27 18:20:32,141][20347] Updated weights for policy 0, policy_version 1832694 (0.0007) [2023-12-27 18:20:32,220][20381] Updated weights for policy 1, policy_version 1831068 (0.0008) [2023-12-27 18:20:32,274][20381] Updated weights for policy 1, policy_version 1831078 (0.0008) [2023-12-27 18:20:32,336][20381] Updated weights for policy 1, policy_version 1831088 (0.0008) [2023-12-27 18:20:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 938057728. Throughput: 0: 10209.5, 1: 10256.9. Samples: 938070544. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:32,363][19170] Avg episode reward: [(0, '980.530'), (1, '960.570')] [2023-12-27 18:20:32,765][20347] Updated weights for policy 0, policy_version 1832704 (0.0006) [2023-12-27 18:20:32,825][20347] Updated weights for policy 0, policy_version 1832714 (0.0006) [2023-12-27 18:20:32,883][20347] Updated weights for policy 0, policy_version 1832724 (0.0008) [2023-12-27 18:20:33,043][20381] Updated weights for policy 1, policy_version 1831098 (0.0009) [2023-12-27 18:20:33,098][20381] Updated weights for policy 1, policy_version 1831108 (0.0010) [2023-12-27 18:20:33,154][20381] Updated weights for policy 1, policy_version 1831118 (0.0008) [2023-12-27 18:20:33,209][20381] Updated weights for policy 1, policy_version 1831128 (0.0005) [2023-12-27 18:20:33,410][20347] Updated weights for policy 0, policy_version 1832734 (0.0006) [2023-12-27 18:20:33,458][20347] Updated weights for policy 0, policy_version 1832744 (0.0005) [2023-12-27 18:20:33,505][20347] Updated weights for policy 0, policy_version 1832754 (0.0007) [2023-12-27 18:20:33,831][20381] Updated weights for policy 1, policy_version 1831138 (0.0007) [2023-12-27 18:20:33,881][20381] Updated weights for policy 1, policy_version 1831148 (0.0009) [2023-12-27 18:20:33,927][20381] Updated weights for policy 1, policy_version 1831158 (0.0005) [2023-12-27 18:20:34,063][20347] Updated weights for policy 0, policy_version 1832764 (0.0008) [2023-12-27 18:20:34,119][20347] Updated weights for policy 0, policy_version 1832774 (0.0005) [2023-12-27 18:20:34,181][20347] Updated weights for policy 0, policy_version 1832784 (0.0007) [2023-12-27 18:20:34,554][20381] Updated weights for policy 1, policy_version 1831168 (0.0010) [2023-12-27 18:20:34,622][20381] Updated weights for policy 1, policy_version 1831178 (0.0011) [2023-12-27 18:20:34,682][20381] Updated weights for policy 1, policy_version 1831188 (0.0011) [2023-12-27 18:20:34,895][20347] Updated weights for policy 0, policy_version 1832794 (0.0008) [2023-12-27 18:20:34,948][20347] Updated weights for policy 0, policy_version 1832804 (0.0008) [2023-12-27 18:20:35,006][20347] Updated weights for policy 0, policy_version 1832814 (0.0007) [2023-12-27 18:20:35,065][20347] Updated weights for policy 0, policy_version 1832824 (0.0007) [2023-12-27 18:20:35,344][20381] Updated weights for policy 1, policy_version 1831198 (0.0011) [2023-12-27 18:20:35,392][20381] Updated weights for policy 1, policy_version 1831208 (0.0010) [2023-12-27 18:20:35,444][20381] Updated weights for policy 1, policy_version 1831218 (0.0010) [2023-12-27 18:20:35,706][20347] Updated weights for policy 0, policy_version 1832834 (0.0007) [2023-12-27 18:20:35,768][20347] Updated weights for policy 0, policy_version 1832844 (0.0008) [2023-12-27 18:20:35,815][20347] Updated weights for policy 0, policy_version 1832854 (0.0007) [2023-12-27 18:20:36,022][20381] Updated weights for policy 1, policy_version 1831228 (0.0009) [2023-12-27 18:20:36,076][20381] Updated weights for policy 1, policy_version 1831238 (0.0010) [2023-12-27 18:20:36,143][20381] Updated weights for policy 1, policy_version 1831248 (0.0009) [2023-12-27 18:20:36,464][20347] Updated weights for policy 0, policy_version 1832864 (0.0008) [2023-12-27 18:20:36,526][20347] Updated weights for policy 0, policy_version 1832874 (0.0008) [2023-12-27 18:20:36,587][20347] Updated weights for policy 0, policy_version 1832884 (0.0008) [2023-12-27 18:20:36,854][20381] Updated weights for policy 1, policy_version 1831258 (0.0009) [2023-12-27 18:20:36,912][20381] Updated weights for policy 1, policy_version 1831268 (0.0006) [2023-12-27 18:20:36,970][20381] Updated weights for policy 1, policy_version 1831278 (0.0011) [2023-12-27 18:20:37,024][20381] Updated weights for policy 1, policy_version 1831288 (0.0007) [2023-12-27 18:20:37,233][20347] Updated weights for policy 0, policy_version 1832894 (0.0006) [2023-12-27 18:20:37,284][20347] Updated weights for policy 0, policy_version 1832904 (0.0005) [2023-12-27 18:20:37,331][20347] Updated weights for policy 0, policy_version 1832914 (0.0008) [2023-12-27 18:20:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 938172416. Throughput: 0: 10219.3, 1: 10286.4. Samples: 938135316. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:37,363][19170] Avg episode reward: [(0, '980.450'), (1, '970.240')] [2023-12-27 18:20:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001831288_468860928.pth... [2023-12-27 18:20:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001832920_469311488.pth... [2023-12-27 18:20:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001830072_468549632.pth [2023-12-27 18:20:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001831736_469008384.pth [2023-12-27 18:20:37,624][20381] Updated weights for policy 1, policy_version 1831298 (0.0006) [2023-12-27 18:20:37,680][20381] Updated weights for policy 1, policy_version 1831308 (0.0006) [2023-12-27 18:20:37,748][20381] Updated weights for policy 1, policy_version 1831318 (0.0009) [2023-12-27 18:20:38,059][20347] Updated weights for policy 0, policy_version 1832924 (0.0008) [2023-12-27 18:20:38,117][20347] Updated weights for policy 0, policy_version 1832934 (0.0008) [2023-12-27 18:20:38,180][20347] Updated weights for policy 0, policy_version 1832944 (0.0007) [2023-12-27 18:20:38,381][20381] Updated weights for policy 1, policy_version 1831328 (0.0008) [2023-12-27 18:20:38,434][20381] Updated weights for policy 1, policy_version 1831338 (0.0009) [2023-12-27 18:20:38,485][20381] Updated weights for policy 1, policy_version 1831348 (0.0011) [2023-12-27 18:20:38,935][20347] Updated weights for policy 0, policy_version 1832954 (0.0007) [2023-12-27 18:20:38,990][20347] Updated weights for policy 0, policy_version 1832964 (0.0008) [2023-12-27 18:20:39,040][20347] Updated weights for policy 0, policy_version 1832974 (0.0007) [2023-12-27 18:20:39,095][20347] Updated weights for policy 0, policy_version 1832984 (0.0008) [2023-12-27 18:20:39,170][20381] Updated weights for policy 1, policy_version 1831358 (0.0007) [2023-12-27 18:20:39,230][20381] Updated weights for policy 1, policy_version 1831368 (0.0008) [2023-12-27 18:20:39,290][20381] Updated weights for policy 1, policy_version 1831378 (0.0008) [2023-12-27 18:20:39,833][20291] KL-divergence is very high: 118.8297 [2023-12-27 18:20:39,877][20347] Updated weights for policy 0, policy_version 1832994 (0.0007) [2023-12-27 18:20:39,886][20291] KL-divergence is very high: 113.8723 [2023-12-27 18:20:39,910][20381] Updated weights for policy 1, policy_version 1831388 (0.0008) [2023-12-27 18:20:39,941][20347] Updated weights for policy 0, policy_version 1833004 (0.0007) [2023-12-27 18:20:39,979][20381] Updated weights for policy 1, policy_version 1831398 (0.0009) [2023-12-27 18:20:40,004][20347] Updated weights for policy 0, policy_version 1833014 (0.0006) [2023-12-27 18:20:40,038][20381] Updated weights for policy 1, policy_version 1831408 (0.0008) [2023-12-27 18:20:40,671][20347] Updated weights for policy 0, policy_version 1833024 (0.0007) [2023-12-27 18:20:40,718][20381] Updated weights for policy 1, policy_version 1831418 (0.0008) [2023-12-27 18:20:40,731][20347] Updated weights for policy 0, policy_version 1833034 (0.0007) [2023-12-27 18:20:40,769][20381] Updated weights for policy 1, policy_version 1831428 (0.0007) [2023-12-27 18:20:40,783][20347] Updated weights for policy 0, policy_version 1833044 (0.0005) [2023-12-27 18:20:40,814][20381] Updated weights for policy 1, policy_version 1831438 (0.0008) [2023-12-27 18:20:40,866][20381] Updated weights for policy 1, policy_version 1831448 (0.0007) [2023-12-27 18:20:41,500][20347] Updated weights for policy 0, policy_version 1833054 (0.0007) [2023-12-27 18:20:41,554][20347] Updated weights for policy 0, policy_version 1833064 (0.0009) [2023-12-27 18:20:41,603][20347] Updated weights for policy 0, policy_version 1833074 (0.0011) [2023-12-27 18:20:41,656][20381] Updated weights for policy 1, policy_version 1831458 (0.0008) [2023-12-27 18:20:41,714][20381] Updated weights for policy 1, policy_version 1831468 (0.0007) [2023-12-27 18:20:41,778][20381] Updated weights for policy 1, policy_version 1831478 (0.0008) [2023-12-27 18:20:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 938262528. Throughput: 0: 10164.2, 1: 10288.7. Samples: 938258456. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:42,363][19170] Avg episode reward: [(0, '980.480'), (1, '921.320')] [2023-12-27 18:20:42,412][20347] Updated weights for policy 0, policy_version 1833084 (0.0009) [2023-12-27 18:20:42,432][20381] Updated weights for policy 1, policy_version 1831488 (0.0008) [2023-12-27 18:20:42,468][20347] Updated weights for policy 0, policy_version 1833094 (0.0006) [2023-12-27 18:20:42,488][20381] Updated weights for policy 1, policy_version 1831498 (0.0008) [2023-12-27 18:20:42,526][20347] Updated weights for policy 0, policy_version 1833104 (0.0006) [2023-12-27 18:20:42,536][20381] Updated weights for policy 1, policy_version 1831508 (0.0007) [2023-12-27 18:20:43,091][20347] Updated weights for policy 0, policy_version 1833114 (0.0007) [2023-12-27 18:20:43,146][20347] Updated weights for policy 0, policy_version 1833124 (0.0005) [2023-12-27 18:20:43,206][20347] Updated weights for policy 0, policy_version 1833134 (0.0005) [2023-12-27 18:20:43,264][20347] Updated weights for policy 0, policy_version 1833144 (0.0007) [2023-12-27 18:20:43,276][20381] Updated weights for policy 1, policy_version 1831518 (0.0006) [2023-12-27 18:20:43,342][20381] Updated weights for policy 1, policy_version 1831528 (0.0007) [2023-12-27 18:20:43,402][20381] Updated weights for policy 1, policy_version 1831538 (0.0005) [2023-12-27 18:20:43,944][20347] Updated weights for policy 0, policy_version 1833154 (0.0008) [2023-12-27 18:20:43,955][20381] Updated weights for policy 1, policy_version 1831548 (0.0005) [2023-12-27 18:20:43,997][20347] Updated weights for policy 0, policy_version 1833164 (0.0008) [2023-12-27 18:20:44,015][20381] Updated weights for policy 1, policy_version 1831558 (0.0005) [2023-12-27 18:20:44,054][20347] Updated weights for policy 0, policy_version 1833174 (0.0008) [2023-12-27 18:20:44,077][20381] Updated weights for policy 1, policy_version 1831568 (0.0006) [2023-12-27 18:20:44,659][20347] Updated weights for policy 0, policy_version 1833184 (0.0009) [2023-12-27 18:20:44,708][20347] Updated weights for policy 0, policy_version 1833194 (0.0008) [2023-12-27 18:20:44,731][20381] Updated weights for policy 1, policy_version 1831578 (0.0008) [2023-12-27 18:20:44,763][20347] Updated weights for policy 0, policy_version 1833204 (0.0007) [2023-12-27 18:20:44,790][20381] Updated weights for policy 1, policy_version 1831588 (0.0007) [2023-12-27 18:20:44,848][20381] Updated weights for policy 1, policy_version 1831598 (0.0008) [2023-12-27 18:20:44,910][20381] Updated weights for policy 1, policy_version 1831608 (0.0007) [2023-12-27 18:20:45,446][20347] Updated weights for policy 0, policy_version 1833214 (0.0008) [2023-12-27 18:20:45,506][20347] Updated weights for policy 0, policy_version 1833224 (0.0008) [2023-12-27 18:20:45,554][20347] Updated weights for policy 0, policy_version 1833234 (0.0007) [2023-12-27 18:20:45,587][20381] Updated weights for policy 1, policy_version 1831618 (0.0005) [2023-12-27 18:20:45,636][20381] Updated weights for policy 1, policy_version 1831628 (0.0006) [2023-12-27 18:20:45,686][20381] Updated weights for policy 1, policy_version 1831638 (0.0008) [2023-12-27 18:20:46,186][20347] Updated weights for policy 0, policy_version 1833244 (0.0006) [2023-12-27 18:20:46,244][20347] Updated weights for policy 0, policy_version 1833254 (0.0006) [2023-12-27 18:20:46,304][20347] Updated weights for policy 0, policy_version 1833264 (0.0007) [2023-12-27 18:20:46,416][20381] Updated weights for policy 1, policy_version 1831648 (0.0008) [2023-12-27 18:20:46,463][20381] Updated weights for policy 1, policy_version 1831658 (0.0009) [2023-12-27 18:20:46,515][20381] Updated weights for policy 1, policy_version 1831668 (0.0006) [2023-12-27 18:20:46,938][20347] Updated weights for policy 0, policy_version 1833274 (0.0008) [2023-12-27 18:20:46,993][20347] Updated weights for policy 0, policy_version 1833284 (0.0006) [2023-12-27 18:20:47,049][20347] Updated weights for policy 0, policy_version 1833294 (0.0007) [2023-12-27 18:20:47,103][20347] Updated weights for policy 0, policy_version 1833304 (0.0009) [2023-12-27 18:20:47,134][20381] Updated weights for policy 1, policy_version 1831678 (0.0006) [2023-12-27 18:20:47,192][20381] Updated weights for policy 1, policy_version 1831688 (0.0005) [2023-12-27 18:20:47,244][20381] Updated weights for policy 1, policy_version 1831698 (0.0006) [2023-12-27 18:20:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 938377216. Throughput: 0: 10150.3, 1: 10313.4. Samples: 938383140. Policy #0 lag: (min: 26.0, avg: 36.9, max: 58.0) [2023-12-27 18:20:47,363][19170] Avg episode reward: [(0, '990.250'), (1, '920.880')] [2023-12-27 18:20:47,815][20381] Updated weights for policy 1, policy_version 1831708 (0.0006) [2023-12-27 18:20:47,853][20347] Updated weights for policy 0, policy_version 1833314 (0.0006) [2023-12-27 18:20:47,867][20381] Updated weights for policy 1, policy_version 1831718 (0.0007) [2023-12-27 18:20:47,908][20347] Updated weights for policy 0, policy_version 1833324 (0.0007) [2023-12-27 18:20:47,912][20381] Updated weights for policy 1, policy_version 1831728 (0.0005) [2023-12-27 18:20:47,960][20347] Updated weights for policy 0, policy_version 1833334 (0.0008) [2023-12-27 18:20:48,551][20381] Updated weights for policy 1, policy_version 1831738 (0.0005) [2023-12-27 18:20:48,602][20381] Updated weights for policy 1, policy_version 1831748 (0.0007) [2023-12-27 18:20:48,662][20381] Updated weights for policy 1, policy_version 1831758 (0.0008) [2023-12-27 18:20:48,709][20347] Updated weights for policy 0, policy_version 1833344 (0.0006) [2023-12-27 18:20:48,719][20381] Updated weights for policy 1, policy_version 1831768 (0.0007) [2023-12-27 18:20:48,757][20347] Updated weights for policy 0, policy_version 1833354 (0.0007) [2023-12-27 18:20:48,822][20347] Updated weights for policy 0, policy_version 1833364 (0.0008) [2023-12-27 18:20:49,357][20381] Updated weights for policy 1, policy_version 1831778 (0.0009) [2023-12-27 18:20:49,419][20381] Updated weights for policy 1, policy_version 1831788 (0.0008) [2023-12-27 18:20:49,469][20381] Updated weights for policy 1, policy_version 1831798 (0.0007) [2023-12-27 18:20:49,569][20347] Updated weights for policy 0, policy_version 1833374 (0.0008) [2023-12-27 18:20:49,622][20347] Updated weights for policy 0, policy_version 1833384 (0.0007) [2023-12-27 18:20:49,675][20347] Updated weights for policy 0, policy_version 1833394 (0.0008) [2023-12-27 18:20:50,206][20381] Updated weights for policy 1, policy_version 1831808 (0.0008) [2023-12-27 18:20:50,271][20381] Updated weights for policy 1, policy_version 1831818 (0.0008) [2023-12-27 18:20:50,331][20381] Updated weights for policy 1, policy_version 1831828 (0.0008) [2023-12-27 18:20:50,423][20347] Updated weights for policy 0, policy_version 1833404 (0.0008) [2023-12-27 18:20:50,478][20347] Updated weights for policy 0, policy_version 1833414 (0.0008) [2023-12-27 18:20:50,524][20347] Updated weights for policy 0, policy_version 1833424 (0.0008) [2023-12-27 18:20:51,040][20381] Updated weights for policy 1, policy_version 1831838 (0.0008) [2023-12-27 18:20:51,102][20381] Updated weights for policy 1, policy_version 1831848 (0.0007) [2023-12-27 18:20:51,162][20381] Updated weights for policy 1, policy_version 1831858 (0.0010) [2023-12-27 18:20:51,187][20347] Updated weights for policy 0, policy_version 1833434 (0.0007) [2023-12-27 18:20:51,244][20347] Updated weights for policy 0, policy_version 1833444 (0.0007) [2023-12-27 18:20:51,304][20347] Updated weights for policy 0, policy_version 1833454 (0.0008) [2023-12-27 18:20:51,366][20347] Updated weights for policy 0, policy_version 1833464 (0.0008) [2023-12-27 18:20:51,901][20381] Updated weights for policy 1, policy_version 1831868 (0.0011) [2023-12-27 18:20:51,953][20381] Updated weights for policy 1, policy_version 1831878 (0.0011) [2023-12-27 18:20:52,012][20381] Updated weights for policy 1, policy_version 1831888 (0.0011) [2023-12-27 18:20:52,062][20347] Updated weights for policy 0, policy_version 1833474 (0.0006) [2023-12-27 18:20:52,117][20347] Updated weights for policy 0, policy_version 1833484 (0.0008) [2023-12-27 18:20:52,171][20347] Updated weights for policy 0, policy_version 1833494 (0.0007) [2023-12-27 18:20:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 938475520. Throughput: 0: 10078.2, 1: 10350.0. Samples: 938443072. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:20:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '920.400')] [2023-12-27 18:20:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001831896_469016576.pth... [2023-12-27 18:20:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001833496_469458944.pth... [2023-12-27 18:20:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001832312_469155840.pth [2023-12-27 18:20:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001830680_468705280.pth [2023-12-27 18:20:52,705][20381] Updated weights for policy 1, policy_version 1831898 (0.0010) [2023-12-27 18:20:52,757][20381] Updated weights for policy 1, policy_version 1831908 (0.0010) [2023-12-27 18:20:52,807][20381] Updated weights for policy 1, policy_version 1831918 (0.0011) [2023-12-27 18:20:52,869][20381] Updated weights for policy 1, policy_version 1831928 (0.0011) [2023-12-27 18:20:52,886][20347] Updated weights for policy 0, policy_version 1833504 (0.0008) [2023-12-27 18:20:52,945][20347] Updated weights for policy 0, policy_version 1833514 (0.0007) [2023-12-27 18:20:53,003][20347] Updated weights for policy 0, policy_version 1833524 (0.0007) [2023-12-27 18:20:53,581][20381] Updated weights for policy 1, policy_version 1831938 (0.0010) [2023-12-27 18:20:53,636][20381] Updated weights for policy 1, policy_version 1831948 (0.0010) [2023-12-27 18:20:53,663][20347] Updated weights for policy 0, policy_version 1833534 (0.0007) [2023-12-27 18:20:53,694][20381] Updated weights for policy 1, policy_version 1831958 (0.0011) [2023-12-27 18:20:53,712][20347] Updated weights for policy 0, policy_version 1833544 (0.0005) [2023-12-27 18:20:53,764][20347] Updated weights for policy 0, policy_version 1833554 (0.0005) [2023-12-27 18:20:54,377][20381] Updated weights for policy 1, policy_version 1831968 (0.0011) [2023-12-27 18:20:54,418][20347] Updated weights for policy 0, policy_version 1833564 (0.0005) [2023-12-27 18:20:54,434][20381] Updated weights for policy 1, policy_version 1831978 (0.0010) [2023-12-27 18:20:54,466][20347] Updated weights for policy 0, policy_version 1833574 (0.0006) [2023-12-27 18:20:54,482][20381] Updated weights for policy 1, policy_version 1831988 (0.0010) [2023-12-27 18:20:54,518][20347] Updated weights for policy 0, policy_version 1833584 (0.0006) [2023-12-27 18:20:55,140][20347] Updated weights for policy 0, policy_version 1833594 (0.0008) [2023-12-27 18:20:55,190][20381] Updated weights for policy 1, policy_version 1831998 (0.0010) [2023-12-27 18:20:55,196][20347] Updated weights for policy 0, policy_version 1833604 (0.0006) [2023-12-27 18:20:55,236][20381] Updated weights for policy 1, policy_version 1832008 (0.0010) [2023-12-27 18:20:55,243][20347] Updated weights for policy 0, policy_version 1833614 (0.0006) [2023-12-27 18:20:55,287][20381] Updated weights for policy 1, policy_version 1832018 (0.0010) [2023-12-27 18:20:55,292][20347] Updated weights for policy 0, policy_version 1833624 (0.0005) [2023-12-27 18:20:55,888][20381] Updated weights for policy 1, policy_version 1832028 (0.0011) [2023-12-27 18:20:55,948][20381] Updated weights for policy 1, policy_version 1832038 (0.0011) [2023-12-27 18:20:55,997][20381] Updated weights for policy 1, policy_version 1832048 (0.0011) [2023-12-27 18:20:56,019][20347] Updated weights for policy 0, policy_version 1833634 (0.0006) [2023-12-27 18:20:56,071][20347] Updated weights for policy 0, policy_version 1833644 (0.0007) [2023-12-27 18:20:56,120][20347] Updated weights for policy 0, policy_version 1833654 (0.0007) [2023-12-27 18:20:56,710][20381] Updated weights for policy 1, policy_version 1832058 (0.0010) [2023-12-27 18:20:56,762][20347] Updated weights for policy 0, policy_version 1833664 (0.0007) [2023-12-27 18:20:56,768][20381] Updated weights for policy 1, policy_version 1832068 (0.0005) [2023-12-27 18:20:56,817][20347] Updated weights for policy 0, policy_version 1833674 (0.0008) [2023-12-27 18:20:56,818][20381] Updated weights for policy 1, policy_version 1832078 (0.0007) [2023-12-27 18:20:56,873][20347] Updated weights for policy 0, policy_version 1833684 (0.0005) [2023-12-27 18:20:56,879][20381] Updated weights for policy 1, policy_version 1832088 (0.0011) [2023-12-27 18:20:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 938573824. Throughput: 0: 10076.0, 1: 10360.8. Samples: 938565720. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:20:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '910.050')] [2023-12-27 18:20:57,518][20381] Updated weights for policy 1, policy_version 1832098 (0.0010) [2023-12-27 18:20:57,567][20347] Updated weights for policy 0, policy_version 1833694 (0.0006) [2023-12-27 18:20:57,569][20381] Updated weights for policy 1, policy_version 1832108 (0.0010) [2023-12-27 18:20:57,617][20381] Updated weights for policy 1, policy_version 1832118 (0.0010) [2023-12-27 18:20:57,624][20347] Updated weights for policy 0, policy_version 1833704 (0.0006) [2023-12-27 18:20:57,669][20347] Updated weights for policy 0, policy_version 1833714 (0.0007) [2023-12-27 18:20:58,282][20381] Updated weights for policy 1, policy_version 1832128 (0.0008) [2023-12-27 18:20:58,347][20381] Updated weights for policy 1, policy_version 1832138 (0.0008) [2023-12-27 18:20:58,410][20381] Updated weights for policy 1, policy_version 1832148 (0.0008) [2023-12-27 18:20:58,418][20347] Updated weights for policy 0, policy_version 1833724 (0.0007) [2023-12-27 18:20:58,483][20347] Updated weights for policy 0, policy_version 1833734 (0.0008) [2023-12-27 18:20:58,550][20347] Updated weights for policy 0, policy_version 1833744 (0.0008) [2023-12-27 18:20:59,200][20381] Updated weights for policy 1, policy_version 1832158 (0.0009) [2023-12-27 18:20:59,267][20381] Updated weights for policy 1, policy_version 1832168 (0.0008) [2023-12-27 18:20:59,312][20347] Updated weights for policy 0, policy_version 1833754 (0.0008) [2023-12-27 18:20:59,328][20381] Updated weights for policy 1, policy_version 1832178 (0.0010) [2023-12-27 18:20:59,375][20347] Updated weights for policy 0, policy_version 1833764 (0.0008) [2023-12-27 18:20:59,437][20347] Updated weights for policy 0, policy_version 1833774 (0.0008) [2023-12-27 18:20:59,496][20347] Updated weights for policy 0, policy_version 1833784 (0.0008) [2023-12-27 18:21:00,070][20381] Updated weights for policy 1, policy_version 1832188 (0.0010) [2023-12-27 18:21:00,121][20381] Updated weights for policy 1, policy_version 1832198 (0.0007) [2023-12-27 18:21:00,171][20381] Updated weights for policy 1, policy_version 1832208 (0.0007) [2023-12-27 18:21:00,177][20347] Updated weights for policy 0, policy_version 1833794 (0.0007) [2023-12-27 18:21:00,234][20347] Updated weights for policy 0, policy_version 1833804 (0.0006) [2023-12-27 18:21:00,287][20347] Updated weights for policy 0, policy_version 1833814 (0.0007) [2023-12-27 18:21:00,773][20381] Updated weights for policy 1, policy_version 1832218 (0.0006) [2023-12-27 18:21:00,823][20381] Updated weights for policy 1, policy_version 1832228 (0.0005) [2023-12-27 18:21:00,881][20381] Updated weights for policy 1, policy_version 1832238 (0.0005) [2023-12-27 18:21:00,937][20381] Updated weights for policy 1, policy_version 1832248 (0.0006) [2023-12-27 18:21:00,993][20347] Updated weights for policy 0, policy_version 1833824 (0.0010) [2023-12-27 18:21:01,056][20347] Updated weights for policy 0, policy_version 1833834 (0.0009) [2023-12-27 18:21:01,106][20347] Updated weights for policy 0, policy_version 1833844 (0.0010) [2023-12-27 18:21:01,637][20381] Updated weights for policy 1, policy_version 1832258 (0.0008) [2023-12-27 18:21:01,699][20381] Updated weights for policy 1, policy_version 1832268 (0.0008) [2023-12-27 18:21:01,770][20381] Updated weights for policy 1, policy_version 1832278 (0.0008) [2023-12-27 18:21:01,858][20347] Updated weights for policy 0, policy_version 1833854 (0.0009) [2023-12-27 18:21:01,919][20347] Updated weights for policy 0, policy_version 1833864 (0.0008) [2023-12-27 18:21:01,983][20347] Updated weights for policy 0, policy_version 1833874 (0.0008) [2023-12-27 18:21:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 938672128. Throughput: 0: 10057.7, 1: 10336.3. Samples: 938685108. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '930.320')] [2023-12-27 18:21:02,424][20381] Updated weights for policy 1, policy_version 1832288 (0.0006) [2023-12-27 18:21:02,471][20381] Updated weights for policy 1, policy_version 1832298 (0.0005) [2023-12-27 18:21:02,522][20381] Updated weights for policy 1, policy_version 1832308 (0.0005) [2023-12-27 18:21:02,632][20347] Updated weights for policy 0, policy_version 1833884 (0.0007) [2023-12-27 18:21:02,683][20347] Updated weights for policy 0, policy_version 1833894 (0.0005) [2023-12-27 18:21:02,740][20347] Updated weights for policy 0, policy_version 1833904 (0.0006) [2023-12-27 18:21:03,198][20381] Updated weights for policy 1, policy_version 1832318 (0.0007) [2023-12-27 18:21:03,249][20381] Updated weights for policy 1, policy_version 1832328 (0.0006) [2023-12-27 18:21:03,295][20381] Updated weights for policy 1, policy_version 1832338 (0.0005) [2023-12-27 18:21:03,331][20347] Updated weights for policy 0, policy_version 1833914 (0.0006) [2023-12-27 18:21:03,380][20347] Updated weights for policy 0, policy_version 1833924 (0.0008) [2023-12-27 18:21:03,428][20347] Updated weights for policy 0, policy_version 1833934 (0.0008) [2023-12-27 18:21:03,474][20347] Updated weights for policy 0, policy_version 1833944 (0.0007) [2023-12-27 18:21:03,926][20381] Updated weights for policy 1, policy_version 1832348 (0.0006) [2023-12-27 18:21:03,975][20381] Updated weights for policy 1, policy_version 1832358 (0.0011) [2023-12-27 18:21:04,026][20381] Updated weights for policy 1, policy_version 1832368 (0.0011) [2023-12-27 18:21:04,240][20347] Updated weights for policy 0, policy_version 1833954 (0.0008) [2023-12-27 18:21:04,307][20347] Updated weights for policy 0, policy_version 1833964 (0.0006) [2023-12-27 18:21:04,376][20347] Updated weights for policy 0, policy_version 1833974 (0.0008) [2023-12-27 18:21:04,701][20381] Updated weights for policy 1, policy_version 1832378 (0.0011) [2023-12-27 18:21:04,759][20381] Updated weights for policy 1, policy_version 1832388 (0.0010) [2023-12-27 18:21:04,815][20381] Updated weights for policy 1, policy_version 1832398 (0.0011) [2023-12-27 18:21:04,873][20381] Updated weights for policy 1, policy_version 1832408 (0.0010) [2023-12-27 18:21:05,020][20347] Updated weights for policy 0, policy_version 1833984 (0.0008) [2023-12-27 18:21:05,075][20347] Updated weights for policy 0, policy_version 1833994 (0.0008) [2023-12-27 18:21:05,127][20347] Updated weights for policy 0, policy_version 1834004 (0.0008) [2023-12-27 18:21:05,508][20381] Updated weights for policy 1, policy_version 1832418 (0.0008) [2023-12-27 18:21:05,556][20381] Updated weights for policy 1, policy_version 1832428 (0.0008) [2023-12-27 18:21:05,613][20381] Updated weights for policy 1, policy_version 1832438 (0.0010) [2023-12-27 18:21:05,805][20347] Updated weights for policy 0, policy_version 1834014 (0.0008) [2023-12-27 18:21:05,852][20347] Updated weights for policy 0, policy_version 1834024 (0.0007) [2023-12-27 18:21:05,899][20347] Updated weights for policy 0, policy_version 1834034 (0.0007) [2023-12-27 18:21:06,345][20381] Updated weights for policy 1, policy_version 1832448 (0.0011) [2023-12-27 18:21:06,408][20381] Updated weights for policy 1, policy_version 1832458 (0.0011) [2023-12-27 18:21:06,469][20381] Updated weights for policy 1, policy_version 1832468 (0.0011) [2023-12-27 18:21:06,666][20347] Updated weights for policy 0, policy_version 1834044 (0.0007) [2023-12-27 18:21:06,724][20347] Updated weights for policy 0, policy_version 1834054 (0.0006) [2023-12-27 18:21:06,775][20347] Updated weights for policy 0, policy_version 1834064 (0.0007) [2023-12-27 18:21:07,156][20381] Updated weights for policy 1, policy_version 1832478 (0.0011) [2023-12-27 18:21:07,208][20381] Updated weights for policy 1, policy_version 1832488 (0.0011) [2023-12-27 18:21:07,261][20381] Updated weights for policy 1, policy_version 1832498 (0.0011) [2023-12-27 18:21:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 938778624. Throughput: 0: 10140.4, 1: 10366.4. Samples: 938748532. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:07,363][19170] Avg episode reward: [(0, '990.260'), (1, '960.320')] [2023-12-27 18:21:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001834072_469606400.pth... [2023-12-27 18:21:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001832504_469172224.pth... [2023-12-27 18:21:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001832920_469311488.pth [2023-12-27 18:21:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001831288_468860928.pth [2023-12-27 18:21:07,471][20347] Updated weights for policy 0, policy_version 1834074 (0.0008) [2023-12-27 18:21:07,524][20347] Updated weights for policy 0, policy_version 1834084 (0.0007) [2023-12-27 18:21:07,579][20347] Updated weights for policy 0, policy_version 1834094 (0.0008) [2023-12-27 18:21:07,633][20347] Updated weights for policy 0, policy_version 1834104 (0.0007) [2023-12-27 18:21:07,890][20381] Updated weights for policy 1, policy_version 1832508 (0.0009) [2023-12-27 18:21:07,947][20381] Updated weights for policy 1, policy_version 1832518 (0.0005) [2023-12-27 18:21:08,000][20381] Updated weights for policy 1, policy_version 1832528 (0.0007) [2023-12-27 18:21:08,326][20347] Updated weights for policy 0, policy_version 1834114 (0.0007) [2023-12-27 18:21:08,386][20347] Updated weights for policy 0, policy_version 1834124 (0.0008) [2023-12-27 18:21:08,437][20347] Updated weights for policy 0, policy_version 1834134 (0.0006) [2023-12-27 18:21:08,735][20381] Updated weights for policy 1, policy_version 1832538 (0.0008) [2023-12-27 18:21:08,787][20381] Updated weights for policy 1, policy_version 1832548 (0.0007) [2023-12-27 18:21:08,840][20381] Updated weights for policy 1, policy_version 1832558 (0.0008) [2023-12-27 18:21:08,862][20292] KL-divergence is very high: 102.5998 [2023-12-27 18:21:08,893][20381] Updated weights for policy 1, policy_version 1832568 (0.0008) [2023-12-27 18:21:09,098][20347] Updated weights for policy 0, policy_version 1834144 (0.0007) [2023-12-27 18:21:09,157][20347] Updated weights for policy 0, policy_version 1834154 (0.0008) [2023-12-27 18:21:09,207][20347] Updated weights for policy 0, policy_version 1834164 (0.0007) [2023-12-27 18:21:09,628][20381] Updated weights for policy 1, policy_version 1832578 (0.0008) [2023-12-27 18:21:09,696][20381] Updated weights for policy 1, policy_version 1832588 (0.0007) [2023-12-27 18:21:09,758][20381] Updated weights for policy 1, policy_version 1832598 (0.0007) [2023-12-27 18:21:09,952][20347] Updated weights for policy 0, policy_version 1834174 (0.0008) [2023-12-27 18:21:10,013][20347] Updated weights for policy 0, policy_version 1834184 (0.0007) [2023-12-27 18:21:10,073][20347] Updated weights for policy 0, policy_version 1834194 (0.0008) [2023-12-27 18:21:10,459][20381] Updated weights for policy 1, policy_version 1832608 (0.0007) [2023-12-27 18:21:10,511][20381] Updated weights for policy 1, policy_version 1832618 (0.0008) [2023-12-27 18:21:10,565][20381] Updated weights for policy 1, policy_version 1832628 (0.0008) [2023-12-27 18:21:10,759][20347] Updated weights for policy 0, policy_version 1834204 (0.0007) [2023-12-27 18:21:10,816][20347] Updated weights for policy 0, policy_version 1834214 (0.0008) [2023-12-27 18:21:10,874][20347] Updated weights for policy 0, policy_version 1834224 (0.0006) [2023-12-27 18:21:11,363][20381] Updated weights for policy 1, policy_version 1832638 (0.0007) [2023-12-27 18:21:11,435][20381] Updated weights for policy 1, policy_version 1832648 (0.0007) [2023-12-27 18:21:11,497][20381] Updated weights for policy 1, policy_version 1832658 (0.0006) [2023-12-27 18:21:11,550][20347] Updated weights for policy 0, policy_version 1834234 (0.0007) [2023-12-27 18:21:11,612][20347] Updated weights for policy 0, policy_version 1834244 (0.0010) [2023-12-27 18:21:11,693][20347] Updated weights for policy 0, policy_version 1834254 (0.0009) [2023-12-27 18:21:11,755][20347] Updated weights for policy 0, policy_version 1834264 (0.0012) [2023-12-27 18:21:12,116][20381] Updated weights for policy 1, policy_version 1832668 (0.0007) [2023-12-27 18:21:12,180][20381] Updated weights for policy 1, policy_version 1832678 (0.0007) [2023-12-27 18:21:12,236][20381] Updated weights for policy 1, policy_version 1832688 (0.0008) [2023-12-27 18:21:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 938876928. Throughput: 0: 10138.6, 1: 10336.3. Samples: 938867768. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:12,363][19170] Avg episode reward: [(0, '990.260'), (1, '902.370')] [2023-12-27 18:21:12,474][20347] Updated weights for policy 0, policy_version 1834274 (0.0007) [2023-12-27 18:21:12,529][20347] Updated weights for policy 0, policy_version 1834284 (0.0007) [2023-12-27 18:21:12,587][20347] Updated weights for policy 0, policy_version 1834294 (0.0007) [2023-12-27 18:21:13,028][20381] Updated weights for policy 1, policy_version 1832698 (0.0007) [2023-12-27 18:21:13,092][20381] Updated weights for policy 1, policy_version 1832708 (0.0006) [2023-12-27 18:21:13,156][20347] Updated weights for policy 0, policy_version 1834304 (0.0006) [2023-12-27 18:21:13,161][20381] Updated weights for policy 1, policy_version 1832718 (0.0007) [2023-12-27 18:21:13,211][20347] Updated weights for policy 0, policy_version 1834314 (0.0007) [2023-12-27 18:21:13,222][20381] Updated weights for policy 1, policy_version 1832728 (0.0007) [2023-12-27 18:21:13,276][20347] Updated weights for policy 0, policy_version 1834324 (0.0005) [2023-12-27 18:21:13,868][20381] Updated weights for policy 1, policy_version 1832738 (0.0007) [2023-12-27 18:21:13,906][20347] Updated weights for policy 0, policy_version 1834334 (0.0009) [2023-12-27 18:21:13,926][20381] Updated weights for policy 1, policy_version 1832748 (0.0006) [2023-12-27 18:21:13,960][20347] Updated weights for policy 0, policy_version 1834344 (0.0010) [2023-12-27 18:21:13,975][20381] Updated weights for policy 1, policy_version 1832758 (0.0006) [2023-12-27 18:21:14,013][20347] Updated weights for policy 0, policy_version 1834354 (0.0005) [2023-12-27 18:21:14,623][20381] Updated weights for policy 1, policy_version 1832768 (0.0007) [2023-12-27 18:21:14,649][20347] Updated weights for policy 0, policy_version 1834364 (0.0007) [2023-12-27 18:21:14,682][20381] Updated weights for policy 1, policy_version 1832778 (0.0007) [2023-12-27 18:21:14,693][20347] Updated weights for policy 0, policy_version 1834374 (0.0010) [2023-12-27 18:21:14,733][20381] Updated weights for policy 1, policy_version 1832788 (0.0010) [2023-12-27 18:21:14,744][20347] Updated weights for policy 0, policy_version 1834384 (0.0010) [2023-12-27 18:21:15,418][20381] Updated weights for policy 1, policy_version 1832798 (0.0011) [2023-12-27 18:21:15,426][20347] Updated weights for policy 0, policy_version 1834394 (0.0008) [2023-12-27 18:21:15,476][20381] Updated weights for policy 1, policy_version 1832808 (0.0010) [2023-12-27 18:21:15,482][20347] Updated weights for policy 0, policy_version 1834404 (0.0008) [2023-12-27 18:21:15,516][20292] KL-divergence is very high: 110.8833 [2023-12-27 18:21:15,531][20381] Updated weights for policy 1, policy_version 1832818 (0.0010) [2023-12-27 18:21:15,537][20347] Updated weights for policy 0, policy_version 1834414 (0.0005) [2023-12-27 18:21:15,558][20292] KL-divergence is very high: 138.3681 [2023-12-27 18:21:15,593][20347] Updated weights for policy 0, policy_version 1834424 (0.0005) [2023-12-27 18:21:16,113][20381] Updated weights for policy 1, policy_version 1832828 (0.0006) [2023-12-27 18:21:16,164][20381] Updated weights for policy 1, policy_version 1832838 (0.0006) [2023-12-27 18:21:16,213][20381] Updated weights for policy 1, policy_version 1832848 (0.0005) [2023-12-27 18:21:16,274][20347] Updated weights for policy 0, policy_version 1834434 (0.0007) [2023-12-27 18:21:16,326][20347] Updated weights for policy 0, policy_version 1834444 (0.0008) [2023-12-27 18:21:16,376][20347] Updated weights for policy 0, policy_version 1834454 (0.0008) [2023-12-27 18:21:16,817][20381] Updated weights for policy 1, policy_version 1832858 (0.0006) [2023-12-27 18:21:16,869][20381] Updated weights for policy 1, policy_version 1832868 (0.0009) [2023-12-27 18:21:16,916][20381] Updated weights for policy 1, policy_version 1832878 (0.0010) [2023-12-27 18:21:16,961][20381] Updated weights for policy 1, policy_version 1832888 (0.0008) [2023-12-27 18:21:17,016][20347] Updated weights for policy 0, policy_version 1834464 (0.0006) [2023-12-27 18:21:17,076][20347] Updated weights for policy 0, policy_version 1834474 (0.0008) [2023-12-27 18:21:17,126][20347] Updated weights for policy 0, policy_version 1834484 (0.0008) [2023-12-27 18:21:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 938983424. Throughput: 0: 10192.4, 1: 10316.0. Samples: 938993420. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.830')] [2023-12-27 18:21:17,621][20381] Updated weights for policy 1, policy_version 1832898 (0.0005) [2023-12-27 18:21:17,671][20381] Updated weights for policy 1, policy_version 1832908 (0.0005) [2023-12-27 18:21:17,726][20381] Updated weights for policy 1, policy_version 1832918 (0.0005) [2023-12-27 18:21:17,822][20347] Updated weights for policy 0, policy_version 1834494 (0.0009) [2023-12-27 18:21:17,883][20347] Updated weights for policy 0, policy_version 1834504 (0.0008) [2023-12-27 18:21:17,934][20347] Updated weights for policy 0, policy_version 1834514 (0.0008) [2023-12-27 18:21:18,334][20381] Updated weights for policy 1, policy_version 1832928 (0.0010) [2023-12-27 18:21:18,399][20381] Updated weights for policy 1, policy_version 1832938 (0.0008) [2023-12-27 18:21:18,459][20381] Updated weights for policy 1, policy_version 1832948 (0.0010) [2023-12-27 18:21:18,602][20347] Updated weights for policy 0, policy_version 1834524 (0.0009) [2023-12-27 18:21:18,659][20347] Updated weights for policy 0, policy_version 1834534 (0.0008) [2023-12-27 18:21:18,721][20347] Updated weights for policy 0, policy_version 1834544 (0.0007) [2023-12-27 18:21:19,130][20381] Updated weights for policy 1, policy_version 1832958 (0.0011) [2023-12-27 18:21:19,182][20381] Updated weights for policy 1, policy_version 1832968 (0.0011) [2023-12-27 18:21:19,240][20381] Updated weights for policy 1, policy_version 1832978 (0.0011) [2023-12-27 18:21:19,448][20347] Updated weights for policy 0, policy_version 1834554 (0.0008) [2023-12-27 18:21:19,509][20347] Updated weights for policy 0, policy_version 1834564 (0.0007) [2023-12-27 18:21:19,573][20347] Updated weights for policy 0, policy_version 1834574 (0.0007) [2023-12-27 18:21:19,641][20347] Updated weights for policy 0, policy_version 1834584 (0.0008) [2023-12-27 18:21:19,973][20381] Updated weights for policy 1, policy_version 1832988 (0.0007) [2023-12-27 18:21:20,034][20381] Updated weights for policy 1, policy_version 1832998 (0.0005) [2023-12-27 18:21:20,094][20381] Updated weights for policy 1, policy_version 1833008 (0.0006) [2023-12-27 18:21:20,361][20347] Updated weights for policy 0, policy_version 1834594 (0.0007) [2023-12-27 18:21:20,431][20347] Updated weights for policy 0, policy_version 1834604 (0.0008) [2023-12-27 18:21:20,496][20347] Updated weights for policy 0, policy_version 1834614 (0.0008) [2023-12-27 18:21:20,694][20381] Updated weights for policy 1, policy_version 1833018 (0.0007) [2023-12-27 18:21:20,756][20381] Updated weights for policy 1, policy_version 1833028 (0.0008) [2023-12-27 18:21:20,817][20381] Updated weights for policy 1, policy_version 1833038 (0.0008) [2023-12-27 18:21:20,878][20381] Updated weights for policy 1, policy_version 1833048 (0.0008) [2023-12-27 18:21:21,228][20347] Updated weights for policy 0, policy_version 1834624 (0.0007) [2023-12-27 18:21:21,290][20347] Updated weights for policy 0, policy_version 1834634 (0.0007) [2023-12-27 18:21:21,367][20347] Updated weights for policy 0, policy_version 1834644 (0.0009) [2023-12-27 18:21:21,565][20381] Updated weights for policy 1, policy_version 1833058 (0.0008) [2023-12-27 18:21:21,631][20381] Updated weights for policy 1, policy_version 1833068 (0.0007) [2023-12-27 18:21:21,701][20381] Updated weights for policy 1, policy_version 1833078 (0.0008) [2023-12-27 18:21:22,032][20347] Updated weights for policy 0, policy_version 1834654 (0.0007) [2023-12-27 18:21:22,088][20347] Updated weights for policy 0, policy_version 1834664 (0.0006) [2023-12-27 18:21:22,149][20347] Updated weights for policy 0, policy_version 1834674 (0.0008) [2023-12-27 18:21:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 939081728. Throughput: 0: 10105.6, 1: 10312.1. Samples: 939054112. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.910')] [2023-12-27 18:21:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001833080_469319680.pth... [2023-12-27 18:21:22,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001834680_469762048.pth... [2023-12-27 18:21:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001831896_469016576.pth [2023-12-27 18:21:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001833496_469458944.pth [2023-12-27 18:21:22,457][20381] Updated weights for policy 1, policy_version 1833088 (0.0008) [2023-12-27 18:21:22,517][20381] Updated weights for policy 1, policy_version 1833098 (0.0008) [2023-12-27 18:21:22,577][20381] Updated weights for policy 1, policy_version 1833108 (0.0008) [2023-12-27 18:21:22,797][20347] Updated weights for policy 0, policy_version 1834684 (0.0007) [2023-12-27 18:21:22,849][20347] Updated weights for policy 0, policy_version 1834694 (0.0007) [2023-12-27 18:21:22,901][20347] Updated weights for policy 0, policy_version 1834704 (0.0007) [2023-12-27 18:21:23,276][20381] Updated weights for policy 1, policy_version 1833118 (0.0008) [2023-12-27 18:21:23,339][20381] Updated weights for policy 1, policy_version 1833128 (0.0008) [2023-12-27 18:21:23,397][20381] Updated weights for policy 1, policy_version 1833138 (0.0011) [2023-12-27 18:21:23,638][20347] Updated weights for policy 0, policy_version 1834714 (0.0008) [2023-12-27 18:21:23,701][20347] Updated weights for policy 0, policy_version 1834724 (0.0008) [2023-12-27 18:21:23,750][20347] Updated weights for policy 0, policy_version 1834734 (0.0010) [2023-12-27 18:21:23,802][20347] Updated weights for policy 0, policy_version 1834744 (0.0010) [2023-12-27 18:21:23,995][20381] Updated weights for policy 1, policy_version 1833148 (0.0009) [2023-12-27 18:21:24,054][20381] Updated weights for policy 1, policy_version 1833158 (0.0005) [2023-12-27 18:21:24,109][20381] Updated weights for policy 1, policy_version 1833168 (0.0006) [2023-12-27 18:21:24,412][20347] Updated weights for policy 0, policy_version 1834754 (0.0007) [2023-12-27 18:21:24,465][20347] Updated weights for policy 0, policy_version 1834764 (0.0005) [2023-12-27 18:21:24,518][20347] Updated weights for policy 0, policy_version 1834774 (0.0007) [2023-12-27 18:21:24,739][20381] Updated weights for policy 1, policy_version 1833178 (0.0011) [2023-12-27 18:21:24,794][20381] Updated weights for policy 1, policy_version 1833188 (0.0010) [2023-12-27 18:21:24,844][20381] Updated weights for policy 1, policy_version 1833198 (0.0010) [2023-12-27 18:21:24,891][20381] Updated weights for policy 1, policy_version 1833208 (0.0010) [2023-12-27 18:21:25,072][20347] Updated weights for policy 0, policy_version 1834784 (0.0006) [2023-12-27 18:21:25,128][20347] Updated weights for policy 0, policy_version 1834794 (0.0005) [2023-12-27 18:21:25,184][20347] Updated weights for policy 0, policy_version 1834804 (0.0005) [2023-12-27 18:21:25,593][20381] Updated weights for policy 1, policy_version 1833218 (0.0010) [2023-12-27 18:21:25,647][20381] Updated weights for policy 1, policy_version 1833228 (0.0010) [2023-12-27 18:21:25,707][20381] Updated weights for policy 1, policy_version 1833238 (0.0010) [2023-12-27 18:21:25,791][20347] Updated weights for policy 0, policy_version 1834814 (0.0005) [2023-12-27 18:21:25,846][20347] Updated weights for policy 0, policy_version 1834824 (0.0006) [2023-12-27 18:21:25,896][20347] Updated weights for policy 0, policy_version 1834834 (0.0010) [2023-12-27 18:21:26,346][20381] Updated weights for policy 1, policy_version 1833248 (0.0006) [2023-12-27 18:21:26,400][20381] Updated weights for policy 1, policy_version 1833258 (0.0007) [2023-12-27 18:21:26,427][20347] Updated weights for policy 0, policy_version 1834844 (0.0007) [2023-12-27 18:21:26,462][20381] Updated weights for policy 1, policy_version 1833268 (0.0009) [2023-12-27 18:21:26,486][20347] Updated weights for policy 0, policy_version 1834854 (0.0010) [2023-12-27 18:21:26,531][20347] Updated weights for policy 0, policy_version 1834864 (0.0008) [2023-12-27 18:21:27,082][20347] Updated weights for policy 0, policy_version 1834874 (0.0005) [2023-12-27 18:21:27,116][20381] Updated weights for policy 1, policy_version 1833278 (0.0010) [2023-12-27 18:21:27,132][20347] Updated weights for policy 0, policy_version 1834884 (0.0006) [2023-12-27 18:21:27,174][20381] Updated weights for policy 1, policy_version 1833288 (0.0007) [2023-12-27 18:21:27,186][20347] Updated weights for policy 0, policy_version 1834894 (0.0010) [2023-12-27 18:21:27,221][20381] Updated weights for policy 1, policy_version 1833298 (0.0005) [2023-12-27 18:21:27,238][20347] Updated weights for policy 0, policy_version 1834904 (0.0008) [2023-12-27 18:21:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 939196416. Throughput: 0: 10171.4, 1: 10268.2. Samples: 939178236. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:27,363][19170] Avg episode reward: [(0, '990.290'), (1, '882.780')] [2023-12-27 18:21:27,871][20381] Updated weights for policy 1, policy_version 1833308 (0.0007) [2023-12-27 18:21:27,883][20347] Updated weights for policy 0, policy_version 1834914 (0.0010) [2023-12-27 18:21:27,920][20381] Updated weights for policy 1, policy_version 1833318 (0.0005) [2023-12-27 18:21:27,931][20347] Updated weights for policy 0, policy_version 1834924 (0.0010) [2023-12-27 18:21:27,975][20381] Updated weights for policy 1, policy_version 1833328 (0.0005) [2023-12-27 18:21:27,975][20347] Updated weights for policy 0, policy_version 1834934 (0.0008) [2023-12-27 18:21:28,628][20381] Updated weights for policy 1, policy_version 1833338 (0.0006) [2023-12-27 18:21:28,668][20347] Updated weights for policy 0, policy_version 1834944 (0.0006) [2023-12-27 18:21:28,677][20381] Updated weights for policy 1, policy_version 1833348 (0.0010) [2023-12-27 18:21:28,726][20381] Updated weights for policy 1, policy_version 1833358 (0.0011) [2023-12-27 18:21:28,729][20347] Updated weights for policy 0, policy_version 1834954 (0.0005) [2023-12-27 18:21:28,778][20381] Updated weights for policy 1, policy_version 1833368 (0.0010) [2023-12-27 18:21:28,797][20347] Updated weights for policy 0, policy_version 1834964 (0.0006) [2023-12-27 18:21:29,371][20347] Updated weights for policy 0, policy_version 1834974 (0.0007) [2023-12-27 18:21:29,436][20347] Updated weights for policy 0, policy_version 1834984 (0.0007) [2023-12-27 18:21:29,451][20381] Updated weights for policy 1, policy_version 1833378 (0.0007) [2023-12-27 18:21:29,493][20347] Updated weights for policy 0, policy_version 1834994 (0.0009) [2023-12-27 18:21:29,504][20381] Updated weights for policy 1, policy_version 1833388 (0.0011) [2023-12-27 18:21:29,563][20381] Updated weights for policy 1, policy_version 1833398 (0.0011) [2023-12-27 18:21:30,119][20347] Updated weights for policy 0, policy_version 1835004 (0.0007) [2023-12-27 18:21:30,178][20347] Updated weights for policy 0, policy_version 1835014 (0.0006) [2023-12-27 18:21:30,236][20347] Updated weights for policy 0, policy_version 1835024 (0.0010) [2023-12-27 18:21:30,240][20381] Updated weights for policy 1, policy_version 1833408 (0.0011) [2023-12-27 18:21:30,291][20381] Updated weights for policy 1, policy_version 1833418 (0.0010) [2023-12-27 18:21:30,348][20381] Updated weights for policy 1, policy_version 1833428 (0.0008) [2023-12-27 18:21:30,867][20347] Updated weights for policy 0, policy_version 1835034 (0.0010) [2023-12-27 18:21:30,922][20347] Updated weights for policy 0, policy_version 1835044 (0.0010) [2023-12-27 18:21:30,963][20381] Updated weights for policy 1, policy_version 1833438 (0.0008) [2023-12-27 18:21:30,981][20347] Updated weights for policy 0, policy_version 1835054 (0.0010) [2023-12-27 18:21:31,005][20381] Updated weights for policy 1, policy_version 1833448 (0.0007) [2023-12-27 18:21:31,043][20347] Updated weights for policy 0, policy_version 1835064 (0.0010) [2023-12-27 18:21:31,075][20381] Updated weights for policy 1, policy_version 1833458 (0.0006) [2023-12-27 18:21:31,746][20347] Updated weights for policy 0, policy_version 1835074 (0.0008) [2023-12-27 18:21:31,802][20381] Updated weights for policy 1, policy_version 1833468 (0.0007) [2023-12-27 18:21:31,804][20347] Updated weights for policy 0, policy_version 1835084 (0.0007) [2023-12-27 18:21:31,855][20381] Updated weights for policy 1, policy_version 1833478 (0.0005) [2023-12-27 18:21:31,861][20347] Updated weights for policy 0, policy_version 1835094 (0.0006) [2023-12-27 18:21:31,905][20381] Updated weights for policy 1, policy_version 1833488 (0.0007) [2023-12-27 18:21:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 939294720. Throughput: 0: 10228.7, 1: 10275.9. Samples: 939305848. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:32,363][19170] Avg episode reward: [(0, '990.290'), (1, '892.510')] [2023-12-27 18:21:32,523][20347] Updated weights for policy 0, policy_version 1835104 (0.0007) [2023-12-27 18:21:32,572][20347] Updated weights for policy 0, policy_version 1835114 (0.0008) [2023-12-27 18:21:32,629][20381] Updated weights for policy 1, policy_version 1833498 (0.0007) [2023-12-27 18:21:32,638][20347] Updated weights for policy 0, policy_version 1835124 (0.0008) [2023-12-27 18:21:32,684][20381] Updated weights for policy 1, policy_version 1833508 (0.0006) [2023-12-27 18:21:32,733][20381] Updated weights for policy 1, policy_version 1833518 (0.0008) [2023-12-27 18:21:32,782][20381] Updated weights for policy 1, policy_version 1833528 (0.0008) [2023-12-27 18:21:33,323][20347] Updated weights for policy 0, policy_version 1835134 (0.0007) [2023-12-27 18:21:33,380][20347] Updated weights for policy 0, policy_version 1835144 (0.0006) [2023-12-27 18:21:33,433][20347] Updated weights for policy 0, policy_version 1835154 (0.0007) [2023-12-27 18:21:33,477][20381] Updated weights for policy 1, policy_version 1833538 (0.0010) [2023-12-27 18:21:33,538][20381] Updated weights for policy 1, policy_version 1833548 (0.0010) [2023-12-27 18:21:33,591][20381] Updated weights for policy 1, policy_version 1833558 (0.0010) [2023-12-27 18:21:33,984][20347] Updated weights for policy 0, policy_version 1835164 (0.0006) [2023-12-27 18:21:34,037][20347] Updated weights for policy 0, policy_version 1835174 (0.0007) [2023-12-27 18:21:34,087][20347] Updated weights for policy 0, policy_version 1835184 (0.0008) [2023-12-27 18:21:34,289][20381] Updated weights for policy 1, policy_version 1833568 (0.0008) [2023-12-27 18:21:34,350][20381] Updated weights for policy 1, policy_version 1833578 (0.0007) [2023-12-27 18:21:34,410][20381] Updated weights for policy 1, policy_version 1833588 (0.0007) [2023-12-27 18:21:34,737][20347] Updated weights for policy 0, policy_version 1835194 (0.0007) [2023-12-27 18:21:34,796][20347] Updated weights for policy 0, policy_version 1835204 (0.0006) [2023-12-27 18:21:34,848][20347] Updated weights for policy 0, policy_version 1835214 (0.0005) [2023-12-27 18:21:34,897][20347] Updated weights for policy 0, policy_version 1835224 (0.0006) [2023-12-27 18:21:35,108][20381] Updated weights for policy 1, policy_version 1833598 (0.0009) [2023-12-27 18:21:35,168][20381] Updated weights for policy 1, policy_version 1833608 (0.0010) [2023-12-27 18:21:35,225][20381] Updated weights for policy 1, policy_version 1833618 (0.0010) [2023-12-27 18:21:35,491][20347] Updated weights for policy 0, policy_version 1835234 (0.0005) [2023-12-27 18:21:35,540][20347] Updated weights for policy 0, policy_version 1835244 (0.0005) [2023-12-27 18:21:35,598][20347] Updated weights for policy 0, policy_version 1835254 (0.0005) [2023-12-27 18:21:35,858][20381] Updated weights for policy 1, policy_version 1833628 (0.0010) [2023-12-27 18:21:35,913][20381] Updated weights for policy 1, policy_version 1833638 (0.0010) [2023-12-27 18:21:35,971][20381] Updated weights for policy 1, policy_version 1833648 (0.0010) [2023-12-27 18:21:36,131][20347] Updated weights for policy 0, policy_version 1835264 (0.0007) [2023-12-27 18:21:36,201][20347] Updated weights for policy 0, policy_version 1835274 (0.0006) [2023-12-27 18:21:36,271][20347] Updated weights for policy 0, policy_version 1835284 (0.0006) [2023-12-27 18:21:36,614][20381] Updated weights for policy 1, policy_version 1833658 (0.0010) [2023-12-27 18:21:36,676][20381] Updated weights for policy 1, policy_version 1833668 (0.0006) [2023-12-27 18:21:36,732][20381] Updated weights for policy 1, policy_version 1833678 (0.0005) [2023-12-27 18:21:36,787][20381] Updated weights for policy 1, policy_version 1833688 (0.0006) [2023-12-27 18:21:36,983][20347] Updated weights for policy 0, policy_version 1835294 (0.0008) [2023-12-27 18:21:37,039][20347] Updated weights for policy 0, policy_version 1835304 (0.0007) [2023-12-27 18:21:37,089][20347] Updated weights for policy 0, policy_version 1835314 (0.0007) [2023-12-27 18:21:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 939401216. Throughput: 0: 10340.4, 1: 10228.6. Samples: 939368672. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '910.920')] [2023-12-27 18:21:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001835320_469925888.pth... [2023-12-27 18:21:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001834072_469606400.pth [2023-12-27 18:21:37,431][20381] Updated weights for policy 1, policy_version 1833698 (0.0011) [2023-12-27 18:21:37,478][20381] Updated weights for policy 1, policy_version 1833708 (0.0011) [2023-12-27 18:21:37,525][20381] Updated weights for policy 1, policy_version 1833718 (0.0011) [2023-12-27 18:21:37,534][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001833720_469483520.pth... [2023-12-27 18:21:37,537][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001832504_469172224.pth [2023-12-27 18:21:37,766][20347] Updated weights for policy 0, policy_version 1835324 (0.0007) [2023-12-27 18:21:37,830][20347] Updated weights for policy 0, policy_version 1835334 (0.0007) [2023-12-27 18:21:37,882][20347] Updated weights for policy 0, policy_version 1835344 (0.0007) [2023-12-27 18:21:38,184][20381] Updated weights for policy 1, policy_version 1833728 (0.0007) [2023-12-27 18:21:38,236][20381] Updated weights for policy 1, policy_version 1833738 (0.0009) [2023-12-27 18:21:38,305][20381] Updated weights for policy 1, policy_version 1833748 (0.0010) [2023-12-27 18:21:38,584][20347] Updated weights for policy 0, policy_version 1835354 (0.0007) [2023-12-27 18:21:38,641][20347] Updated weights for policy 0, policy_version 1835364 (0.0007) [2023-12-27 18:21:38,707][20347] Updated weights for policy 0, policy_version 1835374 (0.0005) [2023-12-27 18:21:38,769][20347] Updated weights for policy 0, policy_version 1835384 (0.0007) [2023-12-27 18:21:38,971][20381] Updated weights for policy 1, policy_version 1833758 (0.0008) [2023-12-27 18:21:39,020][20381] Updated weights for policy 1, policy_version 1833768 (0.0009) [2023-12-27 18:21:39,076][20381] Updated weights for policy 1, policy_version 1833778 (0.0006) [2023-12-27 18:21:39,378][20347] Updated weights for policy 0, policy_version 1835394 (0.0009) [2023-12-27 18:21:39,442][20347] Updated weights for policy 0, policy_version 1835404 (0.0009) [2023-12-27 18:21:39,493][20347] Updated weights for policy 0, policy_version 1835414 (0.0008) [2023-12-27 18:21:39,851][20381] Updated weights for policy 1, policy_version 1833788 (0.0009) [2023-12-27 18:21:39,913][20381] Updated weights for policy 1, policy_version 1833798 (0.0011) [2023-12-27 18:21:39,980][20381] Updated weights for policy 1, policy_version 1833808 (0.0007) [2023-12-27 18:21:40,271][20347] Updated weights for policy 0, policy_version 1835424 (0.0010) [2023-12-27 18:21:40,337][20347] Updated weights for policy 0, policy_version 1835434 (0.0011) [2023-12-27 18:21:40,401][20347] Updated weights for policy 0, policy_version 1835444 (0.0011) [2023-12-27 18:21:40,594][20381] Updated weights for policy 1, policy_version 1833818 (0.0006) [2023-12-27 18:21:40,644][20381] Updated weights for policy 1, policy_version 1833828 (0.0005) [2023-12-27 18:21:40,695][20381] Updated weights for policy 1, policy_version 1833838 (0.0007) [2023-12-27 18:21:40,751][20381] Updated weights for policy 1, policy_version 1833848 (0.0011) [2023-12-27 18:21:41,062][20347] Updated weights for policy 0, policy_version 1835454 (0.0009) [2023-12-27 18:21:41,118][20347] Updated weights for policy 0, policy_version 1835464 (0.0007) [2023-12-27 18:21:41,188][20347] Updated weights for policy 0, policy_version 1835474 (0.0008) [2023-12-27 18:21:41,381][20381] Updated weights for policy 1, policy_version 1833858 (0.0008) [2023-12-27 18:21:41,443][20381] Updated weights for policy 1, policy_version 1833868 (0.0009) [2023-12-27 18:21:41,502][20381] Updated weights for policy 1, policy_version 1833878 (0.0008) [2023-12-27 18:21:41,941][20347] Updated weights for policy 0, policy_version 1835484 (0.0007) [2023-12-27 18:21:42,002][20347] Updated weights for policy 0, policy_version 1835494 (0.0008) [2023-12-27 18:21:42,057][20347] Updated weights for policy 0, policy_version 1835504 (0.0008) [2023-12-27 18:21:42,191][20381] Updated weights for policy 1, policy_version 1833888 (0.0010) [2023-12-27 18:21:42,256][20381] Updated weights for policy 1, policy_version 1833898 (0.0010) [2023-12-27 18:21:42,326][20381] Updated weights for policy 1, policy_version 1833908 (0.0007) [2023-12-27 18:21:42,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 939507712. Throughput: 0: 10352.1, 1: 10278.6. Samples: 939494100. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '919.580')] [2023-12-27 18:21:42,684][20347] Updated weights for policy 0, policy_version 1835514 (0.0008) [2023-12-27 18:21:42,742][20347] Updated weights for policy 0, policy_version 1835524 (0.0007) [2023-12-27 18:21:42,795][20347] Updated weights for policy 0, policy_version 1835534 (0.0008) [2023-12-27 18:21:42,854][20347] Updated weights for policy 0, policy_version 1835544 (0.0008) [2023-12-27 18:21:43,003][20381] Updated weights for policy 1, policy_version 1833918 (0.0008) [2023-12-27 18:21:43,062][20381] Updated weights for policy 1, policy_version 1833928 (0.0008) [2023-12-27 18:21:43,119][20381] Updated weights for policy 1, policy_version 1833938 (0.0008) [2023-12-27 18:21:43,539][20347] Updated weights for policy 0, policy_version 1835554 (0.0006) [2023-12-27 18:21:43,590][20347] Updated weights for policy 0, policy_version 1835564 (0.0007) [2023-12-27 18:21:43,648][20347] Updated weights for policy 0, policy_version 1835574 (0.0007) [2023-12-27 18:21:43,805][20381] Updated weights for policy 1, policy_version 1833948 (0.0008) [2023-12-27 18:21:43,859][20381] Updated weights for policy 1, policy_version 1833958 (0.0008) [2023-12-27 18:21:43,914][20381] Updated weights for policy 1, policy_version 1833968 (0.0008) [2023-12-27 18:21:44,338][20347] Updated weights for policy 0, policy_version 1835584 (0.0007) [2023-12-27 18:21:44,392][20347] Updated weights for policy 0, policy_version 1835594 (0.0008) [2023-12-27 18:21:44,445][20347] Updated weights for policy 0, policy_version 1835604 (0.0008) [2023-12-27 18:21:44,604][20381] Updated weights for policy 1, policy_version 1833978 (0.0008) [2023-12-27 18:21:44,657][20381] Updated weights for policy 1, policy_version 1833988 (0.0008) [2023-12-27 18:21:44,703][20381] Updated weights for policy 1, policy_version 1833998 (0.0009) [2023-12-27 18:21:44,758][20381] Updated weights for policy 1, policy_version 1834008 (0.0008) [2023-12-27 18:21:45,181][20347] Updated weights for policy 0, policy_version 1835614 (0.0007) [2023-12-27 18:21:45,232][20347] Updated weights for policy 0, policy_version 1835624 (0.0008) [2023-12-27 18:21:45,286][20347] Updated weights for policy 0, policy_version 1835634 (0.0007) [2023-12-27 18:21:45,449][20381] Updated weights for policy 1, policy_version 1834018 (0.0008) [2023-12-27 18:21:45,498][20381] Updated weights for policy 1, policy_version 1834028 (0.0008) [2023-12-27 18:21:45,546][20381] Updated weights for policy 1, policy_version 1834038 (0.0007) [2023-12-27 18:21:45,977][20347] Updated weights for policy 0, policy_version 1835644 (0.0006) [2023-12-27 18:21:46,040][20347] Updated weights for policy 0, policy_version 1835654 (0.0008) [2023-12-27 18:21:46,092][20347] Updated weights for policy 0, policy_version 1835664 (0.0007) [2023-12-27 18:21:46,212][20381] Updated weights for policy 1, policy_version 1834048 (0.0007) [2023-12-27 18:21:46,259][20381] Updated weights for policy 1, policy_version 1834058 (0.0006) [2023-12-27 18:21:46,313][20381] Updated weights for policy 1, policy_version 1834068 (0.0007) [2023-12-27 18:21:46,775][20347] Updated weights for policy 0, policy_version 1835674 (0.0007) [2023-12-27 18:21:46,831][20347] Updated weights for policy 0, policy_version 1835684 (0.0006) [2023-12-27 18:21:46,882][20347] Updated weights for policy 0, policy_version 1835694 (0.0005) [2023-12-27 18:21:46,939][20347] Updated weights for policy 0, policy_version 1835704 (0.0007) [2023-12-27 18:21:47,021][20381] Updated weights for policy 1, policy_version 1834078 (0.0008) [2023-12-27 18:21:47,079][20381] Updated weights for policy 1, policy_version 1834088 (0.0008) [2023-12-27 18:21:47,138][20381] Updated weights for policy 1, policy_version 1834098 (0.0007) [2023-12-27 18:21:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 939606016. Throughput: 0: 10405.1, 1: 10249.1. Samples: 939614548. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:47,363][19170] Avg episode reward: [(0, '990.230'), (1, '930.690')] [2023-12-27 18:21:47,528][20347] Updated weights for policy 0, policy_version 1835714 (0.0006) [2023-12-27 18:21:47,576][20347] Updated weights for policy 0, policy_version 1835724 (0.0006) [2023-12-27 18:21:47,628][20347] Updated weights for policy 0, policy_version 1835734 (0.0005) [2023-12-27 18:21:47,792][20381] Updated weights for policy 1, policy_version 1834108 (0.0006) [2023-12-27 18:21:47,860][20381] Updated weights for policy 1, policy_version 1834118 (0.0005) [2023-12-27 18:21:47,918][20381] Updated weights for policy 1, policy_version 1834128 (0.0005) [2023-12-27 18:21:48,254][20347] Updated weights for policy 0, policy_version 1835744 (0.0007) [2023-12-27 18:21:48,320][20347] Updated weights for policy 0, policy_version 1835754 (0.0009) [2023-12-27 18:21:48,385][20347] Updated weights for policy 0, policy_version 1835764 (0.0010) [2023-12-27 18:21:48,602][20381] Updated weights for policy 1, policy_version 1834138 (0.0006) [2023-12-27 18:21:48,655][20381] Updated weights for policy 1, policy_version 1834148 (0.0007) [2023-12-27 18:21:48,701][20381] Updated weights for policy 1, policy_version 1834158 (0.0008) [2023-12-27 18:21:48,749][20381] Updated weights for policy 1, policy_version 1834168 (0.0008) [2023-12-27 18:21:49,082][20347] Updated weights for policy 0, policy_version 1835774 (0.0007) [2023-12-27 18:21:49,138][20347] Updated weights for policy 0, policy_version 1835784 (0.0008) [2023-12-27 18:21:49,193][20347] Updated weights for policy 0, policy_version 1835794 (0.0007) [2023-12-27 18:21:49,502][20381] Updated weights for policy 1, policy_version 1834178 (0.0007) [2023-12-27 18:21:49,561][20381] Updated weights for policy 1, policy_version 1834188 (0.0007) [2023-12-27 18:21:49,613][20381] Updated weights for policy 1, policy_version 1834198 (0.0008) [2023-12-27 18:21:49,909][20347] Updated weights for policy 0, policy_version 1835804 (0.0008) [2023-12-27 18:21:49,968][20347] Updated weights for policy 0, policy_version 1835814 (0.0010) [2023-12-27 18:21:50,018][20347] Updated weights for policy 0, policy_version 1835824 (0.0010) [2023-12-27 18:21:50,392][20381] Updated weights for policy 1, policy_version 1834208 (0.0008) [2023-12-27 18:21:50,447][20381] Updated weights for policy 1, policy_version 1834218 (0.0007) [2023-12-27 18:21:50,498][20381] Updated weights for policy 1, policy_version 1834228 (0.0008) [2023-12-27 18:21:50,722][20347] Updated weights for policy 0, policy_version 1835834 (0.0010) [2023-12-27 18:21:50,788][20347] Updated weights for policy 0, policy_version 1835844 (0.0010) [2023-12-27 18:21:50,853][20347] Updated weights for policy 0, policy_version 1835854 (0.0010) [2023-12-27 18:21:50,912][20347] Updated weights for policy 0, policy_version 1835864 (0.0010) [2023-12-27 18:21:51,200][20381] Updated weights for policy 1, policy_version 1834238 (0.0007) [2023-12-27 18:21:51,261][20381] Updated weights for policy 1, policy_version 1834248 (0.0007) [2023-12-27 18:21:51,324][20381] Updated weights for policy 1, policy_version 1834258 (0.0008) [2023-12-27 18:21:51,642][20347] Updated weights for policy 0, policy_version 1835874 (0.0009) [2023-12-27 18:21:51,711][20347] Updated weights for policy 0, policy_version 1835884 (0.0007) [2023-12-27 18:21:51,775][20347] Updated weights for policy 0, policy_version 1835894 (0.0008) [2023-12-27 18:21:52,011][20381] Updated weights for policy 1, policy_version 1834268 (0.0010) [2023-12-27 18:21:52,066][20381] Updated weights for policy 1, policy_version 1834278 (0.0006) [2023-12-27 18:21:52,135][20381] Updated weights for policy 1, policy_version 1834288 (0.0008) [2023-12-27 18:21:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 939704320. Throughput: 0: 10386.5, 1: 10217.1. Samples: 939675700. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:52,363][19170] Avg episode reward: [(0, '990.230'), (1, '930.340')] [2023-12-27 18:21:52,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001834296_469630976.pth... [2023-12-27 18:21:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001835896_470073344.pth... [2023-12-27 18:21:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001834680_469762048.pth [2023-12-27 18:21:52,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001833080_469319680.pth [2023-12-27 18:21:52,453][20347] Updated weights for policy 0, policy_version 1835904 (0.0008) [2023-12-27 18:21:52,500][20347] Updated weights for policy 0, policy_version 1835914 (0.0008) [2023-12-27 18:21:52,559][20347] Updated weights for policy 0, policy_version 1835924 (0.0008) [2023-12-27 18:21:52,833][20381] Updated weights for policy 1, policy_version 1834298 (0.0007) [2023-12-27 18:21:52,896][20381] Updated weights for policy 1, policy_version 1834308 (0.0007) [2023-12-27 18:21:52,950][20381] Updated weights for policy 1, policy_version 1834318 (0.0005) [2023-12-27 18:21:53,010][20381] Updated weights for policy 1, policy_version 1834328 (0.0008) [2023-12-27 18:21:53,258][20347] Updated weights for policy 0, policy_version 1835934 (0.0007) [2023-12-27 18:21:53,309][20347] Updated weights for policy 0, policy_version 1835944 (0.0005) [2023-12-27 18:21:53,374][20347] Updated weights for policy 0, policy_version 1835954 (0.0006) [2023-12-27 18:21:53,701][20381] Updated weights for policy 1, policy_version 1834338 (0.0008) [2023-12-27 18:21:53,763][20381] Updated weights for policy 1, policy_version 1834348 (0.0007) [2023-12-27 18:21:53,828][20381] Updated weights for policy 1, policy_version 1834358 (0.0008) [2023-12-27 18:21:54,004][20347] Updated weights for policy 0, policy_version 1835964 (0.0007) [2023-12-27 18:21:54,063][20347] Updated weights for policy 0, policy_version 1835974 (0.0008) [2023-12-27 18:21:54,110][20347] Updated weights for policy 0, policy_version 1835984 (0.0008) [2023-12-27 18:21:54,487][20381] Updated weights for policy 1, policy_version 1834368 (0.0007) [2023-12-27 18:21:54,553][20381] Updated weights for policy 1, policy_version 1834378 (0.0006) [2023-12-27 18:21:54,604][20381] Updated weights for policy 1, policy_version 1834388 (0.0008) [2023-12-27 18:21:54,730][20347] Updated weights for policy 0, policy_version 1835994 (0.0007) [2023-12-27 18:21:54,788][20347] Updated weights for policy 0, policy_version 1836004 (0.0007) [2023-12-27 18:21:54,849][20347] Updated weights for policy 0, policy_version 1836014 (0.0008) [2023-12-27 18:21:54,902][20347] Updated weights for policy 0, policy_version 1836024 (0.0007) [2023-12-27 18:21:55,245][20381] Updated weights for policy 1, policy_version 1834398 (0.0008) [2023-12-27 18:21:55,297][20381] Updated weights for policy 1, policy_version 1834408 (0.0007) [2023-12-27 18:21:55,346][20381] Updated weights for policy 1, policy_version 1834418 (0.0006) [2023-12-27 18:21:55,589][20347] Updated weights for policy 0, policy_version 1836034 (0.0007) [2023-12-27 18:21:55,647][20347] Updated weights for policy 0, policy_version 1836044 (0.0007) [2023-12-27 18:21:55,704][20347] Updated weights for policy 0, policy_version 1836054 (0.0007) [2023-12-27 18:21:55,997][20381] Updated weights for policy 1, policy_version 1834428 (0.0007) [2023-12-27 18:21:56,057][20381] Updated weights for policy 1, policy_version 1834438 (0.0005) [2023-12-27 18:21:56,109][20381] Updated weights for policy 1, policy_version 1834448 (0.0006) [2023-12-27 18:21:56,380][20347] Updated weights for policy 0, policy_version 1836064 (0.0006) [2023-12-27 18:21:56,428][20347] Updated weights for policy 0, policy_version 1836074 (0.0007) [2023-12-27 18:21:56,479][20347] Updated weights for policy 0, policy_version 1836084 (0.0007) [2023-12-27 18:21:56,788][20381] Updated weights for policy 1, policy_version 1834458 (0.0007) [2023-12-27 18:21:56,844][20381] Updated weights for policy 1, policy_version 1834468 (0.0007) [2023-12-27 18:21:56,896][20381] Updated weights for policy 1, policy_version 1834478 (0.0008) [2023-12-27 18:21:56,945][20381] Updated weights for policy 1, policy_version 1834488 (0.0007) [2023-12-27 18:21:57,140][20347] Updated weights for policy 0, policy_version 1836094 (0.0008) [2023-12-27 18:21:57,197][20347] Updated weights for policy 0, policy_version 1836104 (0.0008) [2023-12-27 18:21:57,256][20347] Updated weights for policy 0, policy_version 1836114 (0.0008) [2023-12-27 18:21:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 939810816. Throughput: 0: 10398.9, 1: 10265.7. Samples: 939797676. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:21:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.440')] [2023-12-27 18:21:57,649][20381] Updated weights for policy 1, policy_version 1834498 (0.0007) [2023-12-27 18:21:57,698][20381] Updated weights for policy 1, policy_version 1834508 (0.0008) [2023-12-27 18:21:57,749][20381] Updated weights for policy 1, policy_version 1834518 (0.0007) [2023-12-27 18:21:57,943][20347] Updated weights for policy 0, policy_version 1836124 (0.0009) [2023-12-27 18:21:57,989][20347] Updated weights for policy 0, policy_version 1836134 (0.0010) [2023-12-27 18:21:58,042][20347] Updated weights for policy 0, policy_version 1836144 (0.0008) [2023-12-27 18:21:58,451][20381] Updated weights for policy 1, policy_version 1834528 (0.0008) [2023-12-27 18:21:58,513][20381] Updated weights for policy 1, policy_version 1834538 (0.0009) [2023-12-27 18:21:58,574][20381] Updated weights for policy 1, policy_version 1834548 (0.0009) [2023-12-27 18:21:58,945][20347] Updated weights for policy 0, policy_version 1836154 (0.0009) [2023-12-27 18:21:59,006][20347] Updated weights for policy 0, policy_version 1836164 (0.0008) [2023-12-27 18:21:59,064][20347] Updated weights for policy 0, policy_version 1836174 (0.0008) [2023-12-27 18:21:59,125][20347] Updated weights for policy 0, policy_version 1836184 (0.0007) [2023-12-27 18:21:59,397][20381] Updated weights for policy 1, policy_version 1834558 (0.0007) [2023-12-27 18:21:59,459][20381] Updated weights for policy 1, policy_version 1834568 (0.0008) [2023-12-27 18:21:59,524][20381] Updated weights for policy 1, policy_version 1834578 (0.0007) [2023-12-27 18:21:59,878][20347] Updated weights for policy 0, policy_version 1836194 (0.0009) [2023-12-27 18:21:59,932][20347] Updated weights for policy 0, policy_version 1836204 (0.0008) [2023-12-27 18:21:59,995][20347] Updated weights for policy 0, policy_version 1836214 (0.0008) [2023-12-27 18:22:00,150][20381] Updated weights for policy 1, policy_version 1834588 (0.0008) [2023-12-27 18:22:00,204][20381] Updated weights for policy 1, policy_version 1834598 (0.0007) [2023-12-27 18:22:00,259][20381] Updated weights for policy 1, policy_version 1834608 (0.0005) [2023-12-27 18:22:00,704][20347] Updated weights for policy 0, policy_version 1836224 (0.0006) [2023-12-27 18:22:00,759][20347] Updated weights for policy 0, policy_version 1836234 (0.0005) [2023-12-27 18:22:00,817][20347] Updated weights for policy 0, policy_version 1836244 (0.0005) [2023-12-27 18:22:00,828][20381] Updated weights for policy 1, policy_version 1834618 (0.0006) [2023-12-27 18:22:00,889][20381] Updated weights for policy 1, policy_version 1834628 (0.0008) [2023-12-27 18:22:00,949][20381] Updated weights for policy 1, policy_version 1834638 (0.0008) [2023-12-27 18:22:01,004][20381] Updated weights for policy 1, policy_version 1834648 (0.0008) [2023-12-27 18:22:01,380][20347] Updated weights for policy 0, policy_version 1836254 (0.0006) [2023-12-27 18:22:01,441][20347] Updated weights for policy 0, policy_version 1836264 (0.0006) [2023-12-27 18:22:01,500][20347] Updated weights for policy 0, policy_version 1836274 (0.0006) [2023-12-27 18:22:01,731][20381] Updated weights for policy 1, policy_version 1834658 (0.0008) [2023-12-27 18:22:01,787][20381] Updated weights for policy 1, policy_version 1834668 (0.0008) [2023-12-27 18:22:01,850][20381] Updated weights for policy 1, policy_version 1834678 (0.0007) [2023-12-27 18:22:02,117][20347] Updated weights for policy 0, policy_version 1836284 (0.0006) [2023-12-27 18:22:02,175][20347] Updated weights for policy 0, policy_version 1836294 (0.0006) [2023-12-27 18:22:02,241][20347] Updated weights for policy 0, policy_version 1836304 (0.0006) [2023-12-27 18:22:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 939909120. Throughput: 0: 10346.7, 1: 10205.4. Samples: 939918264. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.030')] [2023-12-27 18:22:02,530][20381] Updated weights for policy 1, policy_version 1834688 (0.0006) [2023-12-27 18:22:02,586][20381] Updated weights for policy 1, policy_version 1834698 (0.0005) [2023-12-27 18:22:02,643][20381] Updated weights for policy 1, policy_version 1834708 (0.0005) [2023-12-27 18:22:02,842][20347] Updated weights for policy 0, policy_version 1836314 (0.0007) [2023-12-27 18:22:02,894][20347] Updated weights for policy 0, policy_version 1836324 (0.0007) [2023-12-27 18:22:02,941][20347] Updated weights for policy 0, policy_version 1836334 (0.0009) [2023-12-27 18:22:02,986][20347] Updated weights for policy 0, policy_version 1836344 (0.0008) [2023-12-27 18:22:03,239][20381] Updated weights for policy 1, policy_version 1834718 (0.0005) [2023-12-27 18:22:03,287][20381] Updated weights for policy 1, policy_version 1834728 (0.0005) [2023-12-27 18:22:03,335][20381] Updated weights for policy 1, policy_version 1834738 (0.0005) [2023-12-27 18:22:03,768][20347] Updated weights for policy 0, policy_version 1836354 (0.0007) [2023-12-27 18:22:03,817][20347] Updated weights for policy 0, policy_version 1836364 (0.0009) [2023-12-27 18:22:03,872][20347] Updated weights for policy 0, policy_version 1836374 (0.0008) [2023-12-27 18:22:03,882][20381] Updated weights for policy 1, policy_version 1834748 (0.0007) [2023-12-27 18:22:03,934][20381] Updated weights for policy 1, policy_version 1834758 (0.0008) [2023-12-27 18:22:03,992][20381] Updated weights for policy 1, policy_version 1834768 (0.0007) [2023-12-27 18:22:04,623][20347] Updated weights for policy 0, policy_version 1836384 (0.0007) [2023-12-27 18:22:04,678][20347] Updated weights for policy 0, policy_version 1836394 (0.0008) [2023-12-27 18:22:04,688][20381] Updated weights for policy 1, policy_version 1834778 (0.0007) [2023-12-27 18:22:04,737][20347] Updated weights for policy 0, policy_version 1836404 (0.0005) [2023-12-27 18:22:04,746][20381] Updated weights for policy 1, policy_version 1834788 (0.0011) [2023-12-27 18:22:04,807][20381] Updated weights for policy 1, policy_version 1834798 (0.0011) [2023-12-27 18:22:04,872][20381] Updated weights for policy 1, policy_version 1834808 (0.0011) [2023-12-27 18:22:05,457][20347] Updated weights for policy 0, policy_version 1836414 (0.0007) [2023-12-27 18:22:05,464][20381] Updated weights for policy 1, policy_version 1834818 (0.0008) [2023-12-27 18:22:05,504][20347] Updated weights for policy 0, policy_version 1836424 (0.0006) [2023-12-27 18:22:05,525][20381] Updated weights for policy 1, policy_version 1834828 (0.0008) [2023-12-27 18:22:05,549][20347] Updated weights for policy 0, policy_version 1836434 (0.0007) [2023-12-27 18:22:05,586][20381] Updated weights for policy 1, policy_version 1834838 (0.0008) [2023-12-27 18:22:06,124][20381] Updated weights for policy 1, policy_version 1834848 (0.0008) [2023-12-27 18:22:06,188][20381] Updated weights for policy 1, policy_version 1834858 (0.0011) [2023-12-27 18:22:06,248][20381] Updated weights for policy 1, policy_version 1834868 (0.0011) [2023-12-27 18:22:06,311][20347] Updated weights for policy 0, policy_version 1836444 (0.0010) [2023-12-27 18:22:06,366][20347] Updated weights for policy 0, policy_version 1836454 (0.0010) [2023-12-27 18:22:06,429][20347] Updated weights for policy 0, policy_version 1836464 (0.0010) [2023-12-27 18:22:06,932][20381] Updated weights for policy 1, policy_version 1834878 (0.0010) [2023-12-27 18:22:06,992][20381] Updated weights for policy 1, policy_version 1834888 (0.0011) [2023-12-27 18:22:07,049][20381] Updated weights for policy 1, policy_version 1834898 (0.0011) [2023-12-27 18:22:07,107][20347] Updated weights for policy 0, policy_version 1836474 (0.0011) [2023-12-27 18:22:07,163][20347] Updated weights for policy 0, policy_version 1836484 (0.0010) [2023-12-27 18:22:07,209][20347] Updated weights for policy 0, policy_version 1836494 (0.0007) [2023-12-27 18:22:07,270][20347] Updated weights for policy 0, policy_version 1836504 (0.0006) [2023-12-27 18:22:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 940015616. Throughput: 0: 10358.8, 1: 10248.1. Samples: 939981420. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.690')] [2023-12-27 18:22:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001834904_469786624.pth... [2023-12-27 18:22:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001836504_470228992.pth... [2023-12-27 18:22:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001833720_469483520.pth [2023-12-27 18:22:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001835320_469925888.pth [2023-12-27 18:22:07,733][20381] Updated weights for policy 1, policy_version 1834908 (0.0010) [2023-12-27 18:22:07,784][20381] Updated weights for policy 1, policy_version 1834918 (0.0007) [2023-12-27 18:22:07,839][20381] Updated weights for policy 1, policy_version 1834928 (0.0007) [2023-12-27 18:22:07,916][20347] Updated weights for policy 0, policy_version 1836514 (0.0010) [2023-12-27 18:22:07,973][20347] Updated weights for policy 0, policy_version 1836524 (0.0010) [2023-12-27 18:22:08,025][20347] Updated weights for policy 0, policy_version 1836534 (0.0010) [2023-12-27 18:22:08,526][20381] Updated weights for policy 1, policy_version 1834938 (0.0008) [2023-12-27 18:22:08,587][20381] Updated weights for policy 1, policy_version 1834948 (0.0011) [2023-12-27 18:22:08,646][20381] Updated weights for policy 1, policy_version 1834958 (0.0011) [2023-12-27 18:22:08,709][20381] Updated weights for policy 1, policy_version 1834968 (0.0011) [2023-12-27 18:22:08,742][20347] Updated weights for policy 0, policy_version 1836544 (0.0010) [2023-12-27 18:22:08,800][20347] Updated weights for policy 0, policy_version 1836554 (0.0010) [2023-12-27 18:22:08,858][20347] Updated weights for policy 0, policy_version 1836564 (0.0007) [2023-12-27 18:22:09,403][20381] Updated weights for policy 1, policy_version 1834978 (0.0009) [2023-12-27 18:22:09,467][20381] Updated weights for policy 1, policy_version 1834988 (0.0007) [2023-12-27 18:22:09,530][20381] Updated weights for policy 1, policy_version 1834998 (0.0006) [2023-12-27 18:22:09,563][20347] Updated weights for policy 0, policy_version 1836574 (0.0010) [2023-12-27 18:22:09,618][20347] Updated weights for policy 0, policy_version 1836584 (0.0007) [2023-12-27 18:22:09,682][20347] Updated weights for policy 0, policy_version 1836594 (0.0007) [2023-12-27 18:22:10,279][20381] Updated weights for policy 1, policy_version 1835008 (0.0007) [2023-12-27 18:22:10,336][20381] Updated weights for policy 1, policy_version 1835018 (0.0007) [2023-12-27 18:22:10,345][20347] Updated weights for policy 0, policy_version 1836604 (0.0007) [2023-12-27 18:22:10,393][20381] Updated weights for policy 1, policy_version 1835028 (0.0006) [2023-12-27 18:22:10,406][20347] Updated weights for policy 0, policy_version 1836614 (0.0007) [2023-12-27 18:22:10,465][20347] Updated weights for policy 0, policy_version 1836624 (0.0007) [2023-12-27 18:22:11,073][20347] Updated weights for policy 0, policy_version 1836634 (0.0008) [2023-12-27 18:22:11,094][20381] Updated weights for policy 1, policy_version 1835038 (0.0006) [2023-12-27 18:22:11,134][20347] Updated weights for policy 0, policy_version 1836644 (0.0006) [2023-12-27 18:22:11,163][20381] Updated weights for policy 1, policy_version 1835048 (0.0009) [2023-12-27 18:22:11,200][20347] Updated weights for policy 0, policy_version 1836654 (0.0007) [2023-12-27 18:22:11,223][20381] Updated weights for policy 1, policy_version 1835058 (0.0007) [2023-12-27 18:22:11,268][20347] Updated weights for policy 0, policy_version 1836664 (0.0007) [2023-12-27 18:22:11,922][20381] Updated weights for policy 1, policy_version 1835068 (0.0008) [2023-12-27 18:22:11,935][20347] Updated weights for policy 0, policy_version 1836674 (0.0007) [2023-12-27 18:22:11,979][20381] Updated weights for policy 1, policy_version 1835078 (0.0008) [2023-12-27 18:22:11,997][20347] Updated weights for policy 0, policy_version 1836684 (0.0007) [2023-12-27 18:22:12,042][20381] Updated weights for policy 1, policy_version 1835088 (0.0007) [2023-12-27 18:22:12,051][20347] Updated weights for policy 0, policy_version 1836694 (0.0007) [2023-12-27 18:22:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 940113920. Throughput: 0: 10317.6, 1: 10240.7. Samples: 940103360. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '852.990')] [2023-12-27 18:22:12,675][20381] Updated weights for policy 1, policy_version 1835098 (0.0006) [2023-12-27 18:22:12,744][20381] Updated weights for policy 1, policy_version 1835108 (0.0008) [2023-12-27 18:22:12,788][20347] Updated weights for policy 0, policy_version 1836704 (0.0006) [2023-12-27 18:22:12,805][20381] Updated weights for policy 1, policy_version 1835118 (0.0008) [2023-12-27 18:22:12,848][20347] Updated weights for policy 0, policy_version 1836714 (0.0007) [2023-12-27 18:22:12,861][20381] Updated weights for policy 1, policy_version 1835128 (0.0005) [2023-12-27 18:22:12,903][20347] Updated weights for policy 0, policy_version 1836724 (0.0007) [2023-12-27 18:22:13,459][20381] Updated weights for policy 1, policy_version 1835138 (0.0007) [2023-12-27 18:22:13,519][20381] Updated weights for policy 1, policy_version 1835148 (0.0008) [2023-12-27 18:22:13,570][20381] Updated weights for policy 1, policy_version 1835158 (0.0005) [2023-12-27 18:22:13,602][20347] Updated weights for policy 0, policy_version 1836734 (0.0008) [2023-12-27 18:22:13,648][20347] Updated weights for policy 0, policy_version 1836744 (0.0007) [2023-12-27 18:22:13,705][20347] Updated weights for policy 0, policy_version 1836754 (0.0008) [2023-12-27 18:22:14,179][20381] Updated weights for policy 1, policy_version 1835168 (0.0007) [2023-12-27 18:22:14,236][20381] Updated weights for policy 1, policy_version 1835178 (0.0008) [2023-12-27 18:22:14,300][20381] Updated weights for policy 1, policy_version 1835188 (0.0008) [2023-12-27 18:22:14,426][20347] Updated weights for policy 0, policy_version 1836764 (0.0008) [2023-12-27 18:22:14,477][20347] Updated weights for policy 0, policy_version 1836774 (0.0007) [2023-12-27 18:22:14,522][20347] Updated weights for policy 0, policy_version 1836784 (0.0007) [2023-12-27 18:22:14,950][20381] Updated weights for policy 1, policy_version 1835198 (0.0008) [2023-12-27 18:22:15,004][20381] Updated weights for policy 1, policy_version 1835208 (0.0007) [2023-12-27 18:22:15,066][20381] Updated weights for policy 1, policy_version 1835218 (0.0008) [2023-12-27 18:22:15,257][20347] Updated weights for policy 0, policy_version 1836794 (0.0006) [2023-12-27 18:22:15,311][20347] Updated weights for policy 0, policy_version 1836804 (0.0008) [2023-12-27 18:22:15,363][20347] Updated weights for policy 0, policy_version 1836814 (0.0007) [2023-12-27 18:22:15,416][20347] Updated weights for policy 0, policy_version 1836824 (0.0007) [2023-12-27 18:22:15,759][20381] Updated weights for policy 1, policy_version 1835228 (0.0008) [2023-12-27 18:22:15,823][20381] Updated weights for policy 1, policy_version 1835238 (0.0007) [2023-12-27 18:22:15,876][20381] Updated weights for policy 1, policy_version 1835248 (0.0007) [2023-12-27 18:22:16,121][20347] Updated weights for policy 0, policy_version 1836834 (0.0008) [2023-12-27 18:22:16,185][20347] Updated weights for policy 0, policy_version 1836844 (0.0008) [2023-12-27 18:22:16,243][20347] Updated weights for policy 0, policy_version 1836854 (0.0008) [2023-12-27 18:22:16,469][20381] Updated weights for policy 1, policy_version 1835258 (0.0007) [2023-12-27 18:22:16,524][20381] Updated weights for policy 1, policy_version 1835268 (0.0007) [2023-12-27 18:22:16,584][20381] Updated weights for policy 1, policy_version 1835278 (0.0008) [2023-12-27 18:22:16,647][20381] Updated weights for policy 1, policy_version 1835288 (0.0006) [2023-12-27 18:22:16,857][20347] Updated weights for policy 0, policy_version 1836864 (0.0006) [2023-12-27 18:22:16,909][20347] Updated weights for policy 0, policy_version 1836874 (0.0008) [2023-12-27 18:22:16,960][20347] Updated weights for policy 0, policy_version 1836884 (0.0008) [2023-12-27 18:22:17,225][20381] Updated weights for policy 1, policy_version 1835298 (0.0006) [2023-12-27 18:22:17,275][20381] Updated weights for policy 1, policy_version 1835308 (0.0005) [2023-12-27 18:22:17,324][20381] Updated weights for policy 1, policy_version 1835318 (0.0005) [2023-12-27 18:22:17,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 940220416. Throughput: 0: 10184.2, 1: 10283.7. Samples: 940226908. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '882.790')] [2023-12-27 18:22:17,646][20347] Updated weights for policy 0, policy_version 1836894 (0.0006) [2023-12-27 18:22:17,706][20347] Updated weights for policy 0, policy_version 1836904 (0.0006) [2023-12-27 18:22:17,768][20347] Updated weights for policy 0, policy_version 1836914 (0.0007) [2023-12-27 18:22:17,894][20381] Updated weights for policy 1, policy_version 1835328 (0.0005) [2023-12-27 18:22:17,949][20381] Updated weights for policy 1, policy_version 1835338 (0.0007) [2023-12-27 18:22:18,011][20381] Updated weights for policy 1, policy_version 1835348 (0.0008) [2023-12-27 18:22:18,440][20347] Updated weights for policy 0, policy_version 1836924 (0.0008) [2023-12-27 18:22:18,500][20347] Updated weights for policy 0, policy_version 1836934 (0.0008) [2023-12-27 18:22:18,552][20347] Updated weights for policy 0, policy_version 1836944 (0.0008) [2023-12-27 18:22:18,718][20381] Updated weights for policy 1, policy_version 1835358 (0.0008) [2023-12-27 18:22:18,782][20381] Updated weights for policy 1, policy_version 1835368 (0.0008) [2023-12-27 18:22:18,845][20381] Updated weights for policy 1, policy_version 1835378 (0.0008) [2023-12-27 18:22:19,225][20347] Updated weights for policy 0, policy_version 1836954 (0.0007) [2023-12-27 18:22:19,286][20347] Updated weights for policy 0, policy_version 1836964 (0.0007) [2023-12-27 18:22:19,357][20347] Updated weights for policy 0, policy_version 1836974 (0.0008) [2023-12-27 18:22:19,412][20347] Updated weights for policy 0, policy_version 1836984 (0.0008) [2023-12-27 18:22:19,573][20381] Updated weights for policy 1, policy_version 1835388 (0.0008) [2023-12-27 18:22:19,631][20381] Updated weights for policy 1, policy_version 1835398 (0.0007) [2023-12-27 18:22:19,682][20381] Updated weights for policy 1, policy_version 1835408 (0.0007) [2023-12-27 18:22:20,240][20347] Updated weights for policy 0, policy_version 1836994 (0.0008) [2023-12-27 18:22:20,301][20347] Updated weights for policy 0, policy_version 1837004 (0.0007) [2023-12-27 18:22:20,359][20347] Updated weights for policy 0, policy_version 1837014 (0.0007) [2023-12-27 18:22:20,445][20381] Updated weights for policy 1, policy_version 1835418 (0.0008) [2023-12-27 18:22:20,509][20381] Updated weights for policy 1, policy_version 1835428 (0.0008) [2023-12-27 18:22:20,563][20381] Updated weights for policy 1, policy_version 1835438 (0.0006) [2023-12-27 18:22:20,630][20381] Updated weights for policy 1, policy_version 1835448 (0.0007) [2023-12-27 18:22:21,014][20347] Updated weights for policy 0, policy_version 1837024 (0.0007) [2023-12-27 18:22:21,080][20347] Updated weights for policy 0, policy_version 1837034 (0.0008) [2023-12-27 18:22:21,139][20347] Updated weights for policy 0, policy_version 1837044 (0.0007) [2023-12-27 18:22:21,318][20381] Updated weights for policy 1, policy_version 1835458 (0.0007) [2023-12-27 18:22:21,385][20381] Updated weights for policy 1, policy_version 1835468 (0.0009) [2023-12-27 18:22:21,451][20381] Updated weights for policy 1, policy_version 1835478 (0.0008) [2023-12-27 18:22:21,914][20347] Updated weights for policy 0, policy_version 1837054 (0.0008) [2023-12-27 18:22:21,974][20347] Updated weights for policy 0, policy_version 1837064 (0.0007) [2023-12-27 18:22:22,026][20347] Updated weights for policy 0, policy_version 1837074 (0.0007) [2023-12-27 18:22:22,195][20381] Updated weights for policy 1, policy_version 1835488 (0.0009) [2023-12-27 18:22:22,262][20381] Updated weights for policy 1, policy_version 1835498 (0.0009) [2023-12-27 18:22:22,310][20381] Updated weights for policy 1, policy_version 1835508 (0.0008) [2023-12-27 18:22:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 940318720. Throughput: 0: 10108.2, 1: 10286.4. Samples: 940286428. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.970')] [2023-12-27 18:22:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001835512_469942272.pth... [2023-12-27 18:22:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001837080_470376448.pth... [2023-12-27 18:22:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001834296_469630976.pth [2023-12-27 18:22:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001835896_470073344.pth [2023-12-27 18:22:22,738][20347] Updated weights for policy 0, policy_version 1837084 (0.0008) [2023-12-27 18:22:22,801][20347] Updated weights for policy 0, policy_version 1837094 (0.0008) [2023-12-27 18:22:22,860][20347] Updated weights for policy 0, policy_version 1837104 (0.0007) [2023-12-27 18:22:23,049][20381] Updated weights for policy 1, policy_version 1835518 (0.0008) [2023-12-27 18:22:23,111][20381] Updated weights for policy 1, policy_version 1835528 (0.0008) [2023-12-27 18:22:23,165][20381] Updated weights for policy 1, policy_version 1835538 (0.0008) [2023-12-27 18:22:23,571][20347] Updated weights for policy 0, policy_version 1837114 (0.0007) [2023-12-27 18:22:23,620][20347] Updated weights for policy 0, policy_version 1837124 (0.0007) [2023-12-27 18:22:23,681][20347] Updated weights for policy 0, policy_version 1837134 (0.0007) [2023-12-27 18:22:23,740][20347] Updated weights for policy 0, policy_version 1837144 (0.0008) [2023-12-27 18:22:23,826][20381] Updated weights for policy 1, policy_version 1835548 (0.0008) [2023-12-27 18:22:23,886][20381] Updated weights for policy 1, policy_version 1835558 (0.0009) [2023-12-27 18:22:23,946][20381] Updated weights for policy 1, policy_version 1835568 (0.0008) [2023-12-27 18:22:23,972][20292] KL-divergence is very high: 153.7489 [2023-12-27 18:22:24,441][20347] Updated weights for policy 0, policy_version 1837154 (0.0005) [2023-12-27 18:22:24,502][20347] Updated weights for policy 0, policy_version 1837164 (0.0005) [2023-12-27 18:22:24,563][20347] Updated weights for policy 0, policy_version 1837174 (0.0008) [2023-12-27 18:22:24,688][20381] Updated weights for policy 1, policy_version 1835578 (0.0007) [2023-12-27 18:22:24,740][20381] Updated weights for policy 1, policy_version 1835588 (0.0008) [2023-12-27 18:22:24,791][20381] Updated weights for policy 1, policy_version 1835598 (0.0007) [2023-12-27 18:22:24,854][20381] Updated weights for policy 1, policy_version 1835608 (0.0005) [2023-12-27 18:22:25,196][20347] Updated weights for policy 0, policy_version 1837184 (0.0006) [2023-12-27 18:22:25,263][20347] Updated weights for policy 0, policy_version 1837194 (0.0005) [2023-12-27 18:22:25,317][20347] Updated weights for policy 0, policy_version 1837204 (0.0008) [2023-12-27 18:22:25,554][20381] Updated weights for policy 1, policy_version 1835618 (0.0008) [2023-12-27 18:22:25,611][20381] Updated weights for policy 1, policy_version 1835628 (0.0008) [2023-12-27 18:22:25,670][20381] Updated weights for policy 1, policy_version 1835638 (0.0007) [2023-12-27 18:22:25,937][20347] Updated weights for policy 0, policy_version 1837214 (0.0007) [2023-12-27 18:22:25,985][20347] Updated weights for policy 0, policy_version 1837224 (0.0008) [2023-12-27 18:22:26,043][20347] Updated weights for policy 0, policy_version 1837234 (0.0008) [2023-12-27 18:22:26,375][20381] Updated weights for policy 1, policy_version 1835648 (0.0008) [2023-12-27 18:22:26,427][20381] Updated weights for policy 1, policy_version 1835658 (0.0008) [2023-12-27 18:22:26,476][20381] Updated weights for policy 1, policy_version 1835668 (0.0007) [2023-12-27 18:22:26,712][20347] Updated weights for policy 0, policy_version 1837244 (0.0007) [2023-12-27 18:22:26,757][20347] Updated weights for policy 0, policy_version 1837254 (0.0005) [2023-12-27 18:22:26,812][20347] Updated weights for policy 0, policy_version 1837264 (0.0007) [2023-12-27 18:22:27,208][20381] Updated weights for policy 1, policy_version 1835678 (0.0007) [2023-12-27 18:22:27,257][20381] Updated weights for policy 1, policy_version 1835688 (0.0008) [2023-12-27 18:22:27,322][20381] Updated weights for policy 1, policy_version 1835698 (0.0007) [2023-12-27 18:22:27,363][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 940417024. Throughput: 0: 10037.3, 1: 10190.9. Samples: 940404372. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '911.230')] [2023-12-27 18:22:27,461][20347] Updated weights for policy 0, policy_version 1837274 (0.0007) [2023-12-27 18:22:27,513][20347] Updated weights for policy 0, policy_version 1837284 (0.0007) [2023-12-27 18:22:27,558][20347] Updated weights for policy 0, policy_version 1837294 (0.0008) [2023-12-27 18:22:27,602][20347] Updated weights for policy 0, policy_version 1837304 (0.0007) [2023-12-27 18:22:28,016][20381] Updated weights for policy 1, policy_version 1835708 (0.0009) [2023-12-27 18:22:28,066][20381] Updated weights for policy 1, policy_version 1835718 (0.0011) [2023-12-27 18:22:28,115][20381] Updated weights for policy 1, policy_version 1835728 (0.0010) [2023-12-27 18:22:28,292][20347] Updated weights for policy 0, policy_version 1837314 (0.0005) [2023-12-27 18:22:28,356][20347] Updated weights for policy 0, policy_version 1837324 (0.0006) [2023-12-27 18:22:28,409][20347] Updated weights for policy 0, policy_version 1837334 (0.0005) [2023-12-27 18:22:28,760][20381] Updated weights for policy 1, policy_version 1835738 (0.0009) [2023-12-27 18:22:28,814][20381] Updated weights for policy 1, policy_version 1835748 (0.0005) [2023-12-27 18:22:28,869][20381] Updated weights for policy 1, policy_version 1835758 (0.0005) [2023-12-27 18:22:28,924][20381] Updated weights for policy 1, policy_version 1835768 (0.0007) [2023-12-27 18:22:29,132][20347] Updated weights for policy 0, policy_version 1837344 (0.0007) [2023-12-27 18:22:29,181][20347] Updated weights for policy 0, policy_version 1837354 (0.0007) [2023-12-27 18:22:29,243][20347] Updated weights for policy 0, policy_version 1837364 (0.0008) [2023-12-27 18:22:29,570][20381] Updated weights for policy 1, policy_version 1835778 (0.0011) [2023-12-27 18:22:29,625][20381] Updated weights for policy 1, policy_version 1835788 (0.0011) [2023-12-27 18:22:29,683][20381] Updated weights for policy 1, policy_version 1835798 (0.0011) [2023-12-27 18:22:29,962][20347] Updated weights for policy 0, policy_version 1837374 (0.0008) [2023-12-27 18:22:30,017][20347] Updated weights for policy 0, policy_version 1837384 (0.0006) [2023-12-27 18:22:30,074][20347] Updated weights for policy 0, policy_version 1837394 (0.0007) [2023-12-27 18:22:30,377][20381] Updated weights for policy 1, policy_version 1835808 (0.0011) [2023-12-27 18:22:30,439][20381] Updated weights for policy 1, policy_version 1835818 (0.0011) [2023-12-27 18:22:30,494][20381] Updated weights for policy 1, policy_version 1835828 (0.0010) [2023-12-27 18:22:30,774][20347] Updated weights for policy 0, policy_version 1837404 (0.0008) [2023-12-27 18:22:30,827][20347] Updated weights for policy 0, policy_version 1837414 (0.0008) [2023-12-27 18:22:30,871][20347] Updated weights for policy 0, policy_version 1837424 (0.0008) [2023-12-27 18:22:31,068][20381] Updated weights for policy 1, policy_version 1835838 (0.0008) [2023-12-27 18:22:31,125][20381] Updated weights for policy 1, policy_version 1835848 (0.0008) [2023-12-27 18:22:31,193][20381] Updated weights for policy 1, policy_version 1835858 (0.0008) [2023-12-27 18:22:31,627][20347] Updated weights for policy 0, policy_version 1837434 (0.0008) [2023-12-27 18:22:31,695][20347] Updated weights for policy 0, policy_version 1837444 (0.0009) [2023-12-27 18:22:31,757][20347] Updated weights for policy 0, policy_version 1837454 (0.0009) [2023-12-27 18:22:31,816][20347] Updated weights for policy 0, policy_version 1837464 (0.0008) [2023-12-27 18:22:31,848][20381] Updated weights for policy 1, policy_version 1835868 (0.0007) [2023-12-27 18:22:31,905][20381] Updated weights for policy 1, policy_version 1835878 (0.0006) [2023-12-27 18:22:31,957][20381] Updated weights for policy 1, policy_version 1835888 (0.0005) [2023-12-27 18:22:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 940515328. Throughput: 0: 10023.5, 1: 10266.5. Samples: 940527596. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.570')] [2023-12-27 18:22:32,557][20347] Updated weights for policy 0, policy_version 1837474 (0.0007) [2023-12-27 18:22:32,606][20347] Updated weights for policy 0, policy_version 1837484 (0.0007) [2023-12-27 18:22:32,624][20381] Updated weights for policy 1, policy_version 1835898 (0.0006) [2023-12-27 18:22:32,656][20347] Updated weights for policy 0, policy_version 1837494 (0.0006) [2023-12-27 18:22:32,674][20381] Updated weights for policy 1, policy_version 1835908 (0.0006) [2023-12-27 18:22:32,701][20292] KL-divergence is very high: 142.5181 [2023-12-27 18:22:32,720][20381] Updated weights for policy 1, policy_version 1835918 (0.0008) [2023-12-27 18:22:32,741][20292] KL-divergence is very high: 136.6455 [2023-12-27 18:22:32,770][20381] Updated weights for policy 1, policy_version 1835928 (0.0007) [2023-12-27 18:22:33,266][20347] Updated weights for policy 0, policy_version 1837504 (0.0005) [2023-12-27 18:22:33,311][20347] Updated weights for policy 0, policy_version 1837514 (0.0005) [2023-12-27 18:22:33,373][20347] Updated weights for policy 0, policy_version 1837524 (0.0005) [2023-12-27 18:22:33,412][20381] Updated weights for policy 1, policy_version 1835938 (0.0006) [2023-12-27 18:22:33,467][20381] Updated weights for policy 1, policy_version 1835948 (0.0005) [2023-12-27 18:22:33,528][20381] Updated weights for policy 1, policy_version 1835958 (0.0006) [2023-12-27 18:22:34,025][20347] Updated weights for policy 0, policy_version 1837534 (0.0008) [2023-12-27 18:22:34,075][20347] Updated weights for policy 0, policy_version 1837544 (0.0007) [2023-12-27 18:22:34,120][20381] Updated weights for policy 1, policy_version 1835968 (0.0009) [2023-12-27 18:22:34,130][20347] Updated weights for policy 0, policy_version 1837554 (0.0009) [2023-12-27 18:22:34,184][20381] Updated weights for policy 1, policy_version 1835978 (0.0011) [2023-12-27 18:22:34,242][20381] Updated weights for policy 1, policy_version 1835988 (0.0010) [2023-12-27 18:22:34,796][20347] Updated weights for policy 0, policy_version 1837564 (0.0007) [2023-12-27 18:22:34,847][20347] Updated weights for policy 0, policy_version 1837574 (0.0007) [2023-12-27 18:22:34,902][20347] Updated weights for policy 0, policy_version 1837584 (0.0008) [2023-12-27 18:22:34,924][20381] Updated weights for policy 1, policy_version 1835998 (0.0008) [2023-12-27 18:22:34,987][20381] Updated weights for policy 1, policy_version 1836008 (0.0005) [2023-12-27 18:22:35,050][20381] Updated weights for policy 1, policy_version 1836018 (0.0005) [2023-12-27 18:22:35,569][20381] Updated weights for policy 1, policy_version 1836028 (0.0005) [2023-12-27 18:22:35,624][20381] Updated weights for policy 1, policy_version 1836038 (0.0005) [2023-12-27 18:22:35,664][20347] Updated weights for policy 0, policy_version 1837594 (0.0007) [2023-12-27 18:22:35,670][20381] Updated weights for policy 1, policy_version 1836048 (0.0005) [2023-12-27 18:22:35,723][20347] Updated weights for policy 0, policy_version 1837604 (0.0006) [2023-12-27 18:22:35,777][20347] Updated weights for policy 0, policy_version 1837614 (0.0007) [2023-12-27 18:22:35,838][20347] Updated weights for policy 0, policy_version 1837624 (0.0007) [2023-12-27 18:22:36,331][20381] Updated weights for policy 1, policy_version 1836058 (0.0009) [2023-12-27 18:22:36,394][20381] Updated weights for policy 1, policy_version 1836068 (0.0006) [2023-12-27 18:22:36,455][20381] Updated weights for policy 1, policy_version 1836078 (0.0006) [2023-12-27 18:22:36,514][20381] Updated weights for policy 1, policy_version 1836088 (0.0006) [2023-12-27 18:22:36,543][20347] Updated weights for policy 0, policy_version 1837634 (0.0008) [2023-12-27 18:22:36,605][20347] Updated weights for policy 0, policy_version 1837644 (0.0008) [2023-12-27 18:22:36,661][20347] Updated weights for policy 0, policy_version 1837654 (0.0007) [2023-12-27 18:22:37,111][20381] Updated weights for policy 1, policy_version 1836098 (0.0007) [2023-12-27 18:22:37,173][20381] Updated weights for policy 1, policy_version 1836108 (0.0008) [2023-12-27 18:22:37,237][20381] Updated weights for policy 1, policy_version 1836118 (0.0007) [2023-12-27 18:22:37,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 940621824. Throughput: 0: 10027.0, 1: 10322.5. Samples: 940591424. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.840')] [2023-12-27 18:22:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001836120_470097920.pth... [2023-12-27 18:22:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001834904_469786624.pth [2023-12-27 18:22:37,384][20347] Updated weights for policy 0, policy_version 1837664 (0.0007) [2023-12-27 18:22:37,438][20347] Updated weights for policy 0, policy_version 1837674 (0.0010) [2023-12-27 18:22:37,496][20347] Updated weights for policy 0, policy_version 1837684 (0.0008) [2023-12-27 18:22:37,513][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001837688_470532096.pth... [2023-12-27 18:22:37,516][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001836504_470228992.pth [2023-12-27 18:22:37,878][20381] Updated weights for policy 1, policy_version 1836128 (0.0007) [2023-12-27 18:22:37,937][20381] Updated weights for policy 1, policy_version 1836138 (0.0007) [2023-12-27 18:22:37,995][20381] Updated weights for policy 1, policy_version 1836148 (0.0007) [2023-12-27 18:22:38,216][20347] Updated weights for policy 0, policy_version 1837694 (0.0009) [2023-12-27 18:22:38,262][20347] Updated weights for policy 0, policy_version 1837704 (0.0010) [2023-12-27 18:22:38,310][20347] Updated weights for policy 0, policy_version 1837714 (0.0009) [2023-12-27 18:22:38,590][20381] Updated weights for policy 1, policy_version 1836158 (0.0007) [2023-12-27 18:22:38,651][20381] Updated weights for policy 1, policy_version 1836168 (0.0008) [2023-12-27 18:22:38,711][20381] Updated weights for policy 1, policy_version 1836178 (0.0008) [2023-12-27 18:22:39,133][20347] Updated weights for policy 0, policy_version 1837724 (0.0008) [2023-12-27 18:22:39,195][20347] Updated weights for policy 0, policy_version 1837734 (0.0008) [2023-12-27 18:22:39,259][20347] Updated weights for policy 0, policy_version 1837744 (0.0008) [2023-12-27 18:22:39,427][20381] Updated weights for policy 1, policy_version 1836188 (0.0008) [2023-12-27 18:22:39,483][20381] Updated weights for policy 1, policy_version 1836198 (0.0007) [2023-12-27 18:22:39,551][20381] Updated weights for policy 1, policy_version 1836208 (0.0008) [2023-12-27 18:22:39,961][20347] Updated weights for policy 0, policy_version 1837754 (0.0008) [2023-12-27 18:22:40,013][20347] Updated weights for policy 0, policy_version 1837764 (0.0008) [2023-12-27 18:22:40,058][20347] Updated weights for policy 0, policy_version 1837774 (0.0008) [2023-12-27 18:22:40,102][20347] Updated weights for policy 0, policy_version 1837784 (0.0008) [2023-12-27 18:22:40,273][20381] Updated weights for policy 1, policy_version 1836218 (0.0007) [2023-12-27 18:22:40,332][20381] Updated weights for policy 1, policy_version 1836228 (0.0005) [2023-12-27 18:22:40,399][20381] Updated weights for policy 1, policy_version 1836238 (0.0009) [2023-12-27 18:22:40,461][20381] Updated weights for policy 1, policy_version 1836248 (0.0009) [2023-12-27 18:22:40,815][20347] Updated weights for policy 0, policy_version 1837794 (0.0005) [2023-12-27 18:22:40,867][20347] Updated weights for policy 0, policy_version 1837804 (0.0006) [2023-12-27 18:22:40,921][20347] Updated weights for policy 0, policy_version 1837814 (0.0008) [2023-12-27 18:22:41,140][20381] Updated weights for policy 1, policy_version 1836258 (0.0008) [2023-12-27 18:22:41,206][20381] Updated weights for policy 1, policy_version 1836268 (0.0008) [2023-12-27 18:22:41,270][20381] Updated weights for policy 1, policy_version 1836278 (0.0007) [2023-12-27 18:22:41,629][20347] Updated weights for policy 0, policy_version 1837824 (0.0008) [2023-12-27 18:22:41,697][20347] Updated weights for policy 0, policy_version 1837834 (0.0008) [2023-12-27 18:22:41,769][20347] Updated weights for policy 0, policy_version 1837844 (0.0009) [2023-12-27 18:22:41,920][20381] Updated weights for policy 1, policy_version 1836288 (0.0006) [2023-12-27 18:22:41,980][20381] Updated weights for policy 1, policy_version 1836298 (0.0007) [2023-12-27 18:22:42,040][20381] Updated weights for policy 1, policy_version 1836308 (0.0008) [2023-12-27 18:22:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 940720128. Throughput: 0: 9963.4, 1: 10349.0. Samples: 940711740. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '908.910')] [2023-12-27 18:22:42,560][20347] Updated weights for policy 0, policy_version 1837854 (0.0007) [2023-12-27 18:22:42,610][20347] Updated weights for policy 0, policy_version 1837864 (0.0008) [2023-12-27 18:22:42,665][20347] Updated weights for policy 0, policy_version 1837874 (0.0007) [2023-12-27 18:22:42,706][20381] Updated weights for policy 1, policy_version 1836318 (0.0007) [2023-12-27 18:22:42,762][20381] Updated weights for policy 1, policy_version 1836328 (0.0006) [2023-12-27 18:22:42,824][20381] Updated weights for policy 1, policy_version 1836338 (0.0005) [2023-12-27 18:22:43,387][20381] Updated weights for policy 1, policy_version 1836348 (0.0005) [2023-12-27 18:22:43,410][20347] Updated weights for policy 0, policy_version 1837884 (0.0007) [2023-12-27 18:22:43,438][20381] Updated weights for policy 1, policy_version 1836358 (0.0005) [2023-12-27 18:22:43,465][20347] Updated weights for policy 0, policy_version 1837894 (0.0007) [2023-12-27 18:22:43,486][20381] Updated weights for policy 1, policy_version 1836368 (0.0007) [2023-12-27 18:22:43,520][20347] Updated weights for policy 0, policy_version 1837904 (0.0006) [2023-12-27 18:22:44,170][20381] Updated weights for policy 1, policy_version 1836378 (0.0007) [2023-12-27 18:22:44,187][20347] Updated weights for policy 0, policy_version 1837914 (0.0005) [2023-12-27 18:22:44,222][20381] Updated weights for policy 1, policy_version 1836388 (0.0007) [2023-12-27 18:22:44,243][20347] Updated weights for policy 0, policy_version 1837924 (0.0008) [2023-12-27 18:22:44,279][20381] Updated weights for policy 1, policy_version 1836398 (0.0006) [2023-12-27 18:22:44,301][20347] Updated weights for policy 0, policy_version 1837934 (0.0006) [2023-12-27 18:22:44,333][20381] Updated weights for policy 1, policy_version 1836408 (0.0008) [2023-12-27 18:22:44,354][20347] Updated weights for policy 0, policy_version 1837944 (0.0006) [2023-12-27 18:22:44,953][20381] Updated weights for policy 1, policy_version 1836418 (0.0006) [2023-12-27 18:22:45,017][20381] Updated weights for policy 1, policy_version 1836428 (0.0008) [2023-12-27 18:22:45,077][20381] Updated weights for policy 1, policy_version 1836438 (0.0009) [2023-12-27 18:22:45,103][20347] Updated weights for policy 0, policy_version 1837954 (0.0006) [2023-12-27 18:22:45,169][20347] Updated weights for policy 0, policy_version 1837964 (0.0007) [2023-12-27 18:22:45,231][20347] Updated weights for policy 0, policy_version 1837974 (0.0007) [2023-12-27 18:22:45,738][20381] Updated weights for policy 1, policy_version 1836448 (0.0011) [2023-12-27 18:22:45,796][20381] Updated weights for policy 1, policy_version 1836458 (0.0008) [2023-12-27 18:22:45,842][20381] Updated weights for policy 1, policy_version 1836468 (0.0008) [2023-12-27 18:22:45,843][20347] Updated weights for policy 0, policy_version 1837984 (0.0005) [2023-12-27 18:22:45,893][20347] Updated weights for policy 0, policy_version 1837994 (0.0005) [2023-12-27 18:22:45,952][20347] Updated weights for policy 0, policy_version 1838004 (0.0007) [2023-12-27 18:22:46,507][20381] Updated weights for policy 1, policy_version 1836478 (0.0007) [2023-12-27 18:22:46,563][20381] Updated weights for policy 1, policy_version 1836488 (0.0006) [2023-12-27 18:22:46,570][20347] Updated weights for policy 0, policy_version 1838014 (0.0010) [2023-12-27 18:22:46,616][20347] Updated weights for policy 0, policy_version 1838024 (0.0010) [2023-12-27 18:22:46,622][20381] Updated weights for policy 1, policy_version 1836498 (0.0008) [2023-12-27 18:22:46,662][20347] Updated weights for policy 0, policy_version 1838034 (0.0010) [2023-12-27 18:22:47,246][20381] Updated weights for policy 1, policy_version 1836508 (0.0008) [2023-12-27 18:22:47,260][20347] Updated weights for policy 0, policy_version 1838044 (0.0010) [2023-12-27 18:22:47,305][20381] Updated weights for policy 1, policy_version 1836518 (0.0008) [2023-12-27 18:22:47,316][20347] Updated weights for policy 0, policy_version 1838054 (0.0010) [2023-12-27 18:22:47,351][20381] Updated weights for policy 1, policy_version 1836528 (0.0006) [2023-12-27 18:22:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 940818432. Throughput: 0: 9966.9, 1: 10442.4. Samples: 940836684. Policy #0 lag: (min: 13.0, avg: 26.5, max: 45.0) [2023-12-27 18:22:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '908.420')] [2023-12-27 18:22:47,368][20347] Updated weights for policy 0, policy_version 1838064 (0.0010) [2023-12-27 18:22:47,901][20381] Updated weights for policy 1, policy_version 1836538 (0.0005) [2023-12-27 18:22:47,959][20381] Updated weights for policy 1, policy_version 1836548 (0.0010) [2023-12-27 18:22:47,999][20347] Updated weights for policy 0, policy_version 1838074 (0.0009) [2023-12-27 18:22:48,012][20381] Updated weights for policy 1, policy_version 1836558 (0.0010) [2023-12-27 18:22:48,056][20347] Updated weights for policy 0, policy_version 1838084 (0.0008) [2023-12-27 18:22:48,063][20381] Updated weights for policy 1, policy_version 1836568 (0.0011) [2023-12-27 18:22:48,109][20347] Updated weights for policy 0, policy_version 1838094 (0.0006) [2023-12-27 18:22:48,158][20347] Updated weights for policy 0, policy_version 1838104 (0.0006) [2023-12-27 18:22:48,730][20381] Updated weights for policy 1, policy_version 1836578 (0.0007) [2023-12-27 18:22:48,790][20381] Updated weights for policy 1, policy_version 1836588 (0.0007) [2023-12-27 18:22:48,842][20381] Updated weights for policy 1, policy_version 1836598 (0.0008) [2023-12-27 18:22:48,884][20347] Updated weights for policy 0, policy_version 1838114 (0.0010) [2023-12-27 18:22:48,944][20347] Updated weights for policy 0, policy_version 1838124 (0.0009) [2023-12-27 18:22:48,997][20347] Updated weights for policy 0, policy_version 1838134 (0.0010) [2023-12-27 18:22:49,613][20381] Updated weights for policy 1, policy_version 1836608 (0.0006) [2023-12-27 18:22:49,669][20381] Updated weights for policy 1, policy_version 1836618 (0.0006) [2023-12-27 18:22:49,678][20347] Updated weights for policy 0, policy_version 1838144 (0.0007) [2023-12-27 18:22:49,722][20381] Updated weights for policy 1, policy_version 1836628 (0.0005) [2023-12-27 18:22:49,733][20347] Updated weights for policy 0, policy_version 1838154 (0.0007) [2023-12-27 18:22:49,789][20347] Updated weights for policy 0, policy_version 1838164 (0.0008) [2023-12-27 18:22:50,438][20381] Updated weights for policy 1, policy_version 1836638 (0.0006) [2023-12-27 18:22:50,495][20381] Updated weights for policy 1, policy_version 1836648 (0.0007) [2023-12-27 18:22:50,519][20347] Updated weights for policy 0, policy_version 1838174 (0.0007) [2023-12-27 18:22:50,552][20381] Updated weights for policy 1, policy_version 1836658 (0.0006) [2023-12-27 18:22:50,575][20347] Updated weights for policy 0, policy_version 1838184 (0.0007) [2023-12-27 18:22:50,635][20347] Updated weights for policy 0, policy_version 1838194 (0.0008) [2023-12-27 18:22:51,302][20381] Updated weights for policy 1, policy_version 1836668 (0.0008) [2023-12-27 18:22:51,355][20381] Updated weights for policy 1, policy_version 1836678 (0.0008) [2023-12-27 18:22:51,382][20347] Updated weights for policy 0, policy_version 1838204 (0.0008) [2023-12-27 18:22:51,422][20381] Updated weights for policy 1, policy_version 1836688 (0.0009) [2023-12-27 18:22:51,447][20347] Updated weights for policy 0, policy_version 1838214 (0.0009) [2023-12-27 18:22:51,507][20347] Updated weights for policy 0, policy_version 1838224 (0.0008) [2023-12-27 18:22:52,184][20381] Updated weights for policy 1, policy_version 1836698 (0.0007) [2023-12-27 18:22:52,215][20347] Updated weights for policy 0, policy_version 1838234 (0.0007) [2023-12-27 18:22:52,240][20381] Updated weights for policy 1, policy_version 1836708 (0.0007) [2023-12-27 18:22:52,278][20347] Updated weights for policy 0, policy_version 1838244 (0.0007) [2023-12-27 18:22:52,305][20381] Updated weights for policy 1, policy_version 1836718 (0.0007) [2023-12-27 18:22:52,342][20347] Updated weights for policy 0, policy_version 1838254 (0.0007) [2023-12-27 18:22:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 940916736. Throughput: 0: 9980.2, 1: 10394.6. Samples: 940898284. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:22:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '909.330')] [2023-12-27 18:22:52,366][20381] Updated weights for policy 1, policy_version 1836728 (0.0007) [2023-12-27 18:22:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001836728_470253568.pth... [2023-12-27 18:22:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001835512_469942272.pth [2023-12-27 18:22:52,397][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001838264_470679552.pth... [2023-12-27 18:22:52,399][20347] Updated weights for policy 0, policy_version 1838264 (0.0008) [2023-12-27 18:22:52,400][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001837080_470376448.pth [2023-12-27 18:22:53,055][20347] Updated weights for policy 0, policy_version 1838274 (0.0008) [2023-12-27 18:22:53,110][20347] Updated weights for policy 0, policy_version 1838284 (0.0005) [2023-12-27 18:22:53,111][20381] Updated weights for policy 1, policy_version 1836738 (0.0008) [2023-12-27 18:22:53,161][20381] Updated weights for policy 1, policy_version 1836748 (0.0007) [2023-12-27 18:22:53,167][20347] Updated weights for policy 0, policy_version 1838294 (0.0005) [2023-12-27 18:22:53,212][20381] Updated weights for policy 1, policy_version 1836758 (0.0008) [2023-12-27 18:22:53,744][20347] Updated weights for policy 0, policy_version 1838304 (0.0009) [2023-12-27 18:22:53,804][20347] Updated weights for policy 0, policy_version 1838314 (0.0010) [2023-12-27 18:22:53,859][20347] Updated weights for policy 0, policy_version 1838324 (0.0008) [2023-12-27 18:22:53,871][20381] Updated weights for policy 1, policy_version 1836768 (0.0006) [2023-12-27 18:22:53,931][20381] Updated weights for policy 1, policy_version 1836778 (0.0008) [2023-12-27 18:22:54,019][20381] Updated weights for policy 1, policy_version 1836788 (0.0007) [2023-12-27 18:22:54,613][20347] Updated weights for policy 0, policy_version 1838334 (0.0008) [2023-12-27 18:22:54,660][20347] Updated weights for policy 0, policy_version 1838344 (0.0006) [2023-12-27 18:22:54,662][20381] Updated weights for policy 1, policy_version 1836798 (0.0006) [2023-12-27 18:22:54,709][20347] Updated weights for policy 0, policy_version 1838354 (0.0005) [2023-12-27 18:22:54,711][20381] Updated weights for policy 1, policy_version 1836808 (0.0006) [2023-12-27 18:22:54,766][20381] Updated weights for policy 1, policy_version 1836818 (0.0006) [2023-12-27 18:22:55,385][20347] Updated weights for policy 0, policy_version 1838364 (0.0006) [2023-12-27 18:22:55,443][20347] Updated weights for policy 0, policy_version 1838374 (0.0006) [2023-12-27 18:22:55,486][20381] Updated weights for policy 1, policy_version 1836828 (0.0007) [2023-12-27 18:22:55,503][20347] Updated weights for policy 0, policy_version 1838384 (0.0006) [2023-12-27 18:22:55,546][20381] Updated weights for policy 1, policy_version 1836838 (0.0006) [2023-12-27 18:22:55,598][20381] Updated weights for policy 1, policy_version 1836848 (0.0007) [2023-12-27 18:22:56,144][20347] Updated weights for policy 0, policy_version 1838394 (0.0006) [2023-12-27 18:22:56,196][20347] Updated weights for policy 0, policy_version 1838404 (0.0008) [2023-12-27 18:22:56,255][20347] Updated weights for policy 0, policy_version 1838414 (0.0007) [2023-12-27 18:22:56,272][20381] Updated weights for policy 1, policy_version 1836858 (0.0008) [2023-12-27 18:22:56,316][20347] Updated weights for policy 0, policy_version 1838424 (0.0006) [2023-12-27 18:22:56,333][20381] Updated weights for policy 1, policy_version 1836868 (0.0006) [2023-12-27 18:22:56,401][20381] Updated weights for policy 1, policy_version 1836878 (0.0008) [2023-12-27 18:22:56,458][20381] Updated weights for policy 1, policy_version 1836888 (0.0008) [2023-12-27 18:22:56,952][20347] Updated weights for policy 0, policy_version 1838434 (0.0007) [2023-12-27 18:22:57,015][20347] Updated weights for policy 0, policy_version 1838444 (0.0008) [2023-12-27 18:22:57,063][20347] Updated weights for policy 0, policy_version 1838454 (0.0007) [2023-12-27 18:22:57,160][20381] Updated weights for policy 1, policy_version 1836898 (0.0007) [2023-12-27 18:22:57,210][20381] Updated weights for policy 1, policy_version 1836908 (0.0008) [2023-12-27 18:22:57,256][20381] Updated weights for policy 1, policy_version 1836918 (0.0008) [2023-12-27 18:22:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 941031424. Throughput: 0: 9982.2, 1: 10337.0. Samples: 941017724. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:22:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '910.130')] [2023-12-27 18:22:57,685][20347] Updated weights for policy 0, policy_version 1838464 (0.0006) [2023-12-27 18:22:57,741][20347] Updated weights for policy 0, policy_version 1838474 (0.0005) [2023-12-27 18:22:57,794][20347] Updated weights for policy 0, policy_version 1838484 (0.0007) [2023-12-27 18:22:57,866][20381] Updated weights for policy 1, policy_version 1836928 (0.0009) [2023-12-27 18:22:57,916][20381] Updated weights for policy 1, policy_version 1836938 (0.0008) [2023-12-27 18:22:57,962][20381] Updated weights for policy 1, policy_version 1836948 (0.0009) [2023-12-27 18:22:58,406][20347] Updated weights for policy 0, policy_version 1838494 (0.0007) [2023-12-27 18:22:58,469][20347] Updated weights for policy 0, policy_version 1838504 (0.0008) [2023-12-27 18:22:58,532][20347] Updated weights for policy 0, policy_version 1838514 (0.0009) [2023-12-27 18:22:58,787][20381] Updated weights for policy 1, policy_version 1836958 (0.0008) [2023-12-27 18:22:58,862][20381] Updated weights for policy 1, policy_version 1836968 (0.0009) [2023-12-27 18:22:58,921][20381] Updated weights for policy 1, policy_version 1836978 (0.0008) [2023-12-27 18:22:59,293][20347] Updated weights for policy 0, policy_version 1838524 (0.0009) [2023-12-27 18:22:59,351][20347] Updated weights for policy 0, policy_version 1838534 (0.0009) [2023-12-27 18:22:59,416][20347] Updated weights for policy 0, policy_version 1838544 (0.0008) [2023-12-27 18:22:59,630][20381] Updated weights for policy 1, policy_version 1836988 (0.0007) [2023-12-27 18:22:59,686][20381] Updated weights for policy 1, policy_version 1836998 (0.0005) [2023-12-27 18:22:59,744][20381] Updated weights for policy 1, policy_version 1837008 (0.0005) [2023-12-27 18:23:00,121][20347] Updated weights for policy 0, policy_version 1838554 (0.0007) [2023-12-27 18:23:00,178][20347] Updated weights for policy 0, policy_version 1838564 (0.0005) [2023-12-27 18:23:00,235][20347] Updated weights for policy 0, policy_version 1838574 (0.0006) [2023-12-27 18:23:00,293][20347] Updated weights for policy 0, policy_version 1838584 (0.0005) [2023-12-27 18:23:00,392][20381] Updated weights for policy 1, policy_version 1837018 (0.0007) [2023-12-27 18:23:00,442][20381] Updated weights for policy 1, policy_version 1837028 (0.0005) [2023-12-27 18:23:00,492][20381] Updated weights for policy 1, policy_version 1837038 (0.0006) [2023-12-27 18:23:00,535][20381] Updated weights for policy 1, policy_version 1837048 (0.0007) [2023-12-27 18:23:00,825][20347] Updated weights for policy 0, policy_version 1838594 (0.0005) [2023-12-27 18:23:00,874][20347] Updated weights for policy 0, policy_version 1838604 (0.0005) [2023-12-27 18:23:00,920][20347] Updated weights for policy 0, policy_version 1838614 (0.0006) [2023-12-27 18:23:01,136][20381] Updated weights for policy 1, policy_version 1837058 (0.0006) [2023-12-27 18:23:01,200][20381] Updated weights for policy 1, policy_version 1837068 (0.0010) [2023-12-27 18:23:01,256][20381] Updated weights for policy 1, policy_version 1837078 (0.0008) [2023-12-27 18:23:01,555][20347] Updated weights for policy 0, policy_version 1838624 (0.0008) [2023-12-27 18:23:01,605][20347] Updated weights for policy 0, policy_version 1838634 (0.0007) [2023-12-27 18:23:01,668][20347] Updated weights for policy 0, policy_version 1838644 (0.0007) [2023-12-27 18:23:01,931][20381] Updated weights for policy 1, policy_version 1837088 (0.0010) [2023-12-27 18:23:01,984][20381] Updated weights for policy 1, policy_version 1837098 (0.0011) [2023-12-27 18:23:02,052][20381] Updated weights for policy 1, policy_version 1837108 (0.0009) [2023-12-27 18:23:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 941129728. Throughput: 0: 10080.5, 1: 10268.8. Samples: 941142624. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.850')] [2023-12-27 18:23:02,398][20347] Updated weights for policy 0, policy_version 1838654 (0.0008) [2023-12-27 18:23:02,455][20347] Updated weights for policy 0, policy_version 1838664 (0.0007) [2023-12-27 18:23:02,510][20347] Updated weights for policy 0, policy_version 1838674 (0.0007) [2023-12-27 18:23:02,694][20381] Updated weights for policy 1, policy_version 1837118 (0.0008) [2023-12-27 18:23:02,745][20381] Updated weights for policy 1, policy_version 1837128 (0.0010) [2023-12-27 18:23:02,803][20381] Updated weights for policy 1, policy_version 1837138 (0.0011) [2023-12-27 18:23:03,207][20347] Updated weights for policy 0, policy_version 1838684 (0.0007) [2023-12-27 18:23:03,257][20347] Updated weights for policy 0, policy_version 1838694 (0.0006) [2023-12-27 18:23:03,310][20347] Updated weights for policy 0, policy_version 1838704 (0.0006) [2023-12-27 18:23:03,478][20381] Updated weights for policy 1, policy_version 1837148 (0.0008) [2023-12-27 18:23:03,530][20381] Updated weights for policy 1, policy_version 1837158 (0.0007) [2023-12-27 18:23:03,584][20381] Updated weights for policy 1, policy_version 1837168 (0.0010) [2023-12-27 18:23:03,987][20347] Updated weights for policy 0, policy_version 1838714 (0.0006) [2023-12-27 18:23:04,045][20347] Updated weights for policy 0, policy_version 1838724 (0.0007) [2023-12-27 18:23:04,104][20347] Updated weights for policy 0, policy_version 1838734 (0.0008) [2023-12-27 18:23:04,167][20347] Updated weights for policy 0, policy_version 1838744 (0.0011) [2023-12-27 18:23:04,267][20381] Updated weights for policy 1, policy_version 1837178 (0.0010) [2023-12-27 18:23:04,323][20381] Updated weights for policy 1, policy_version 1837188 (0.0010) [2023-12-27 18:23:04,383][20381] Updated weights for policy 1, policy_version 1837198 (0.0010) [2023-12-27 18:23:04,437][20381] Updated weights for policy 1, policy_version 1837208 (0.0010) [2023-12-27 18:23:04,798][20347] Updated weights for policy 0, policy_version 1838754 (0.0010) [2023-12-27 18:23:04,852][20347] Updated weights for policy 0, policy_version 1838764 (0.0010) [2023-12-27 18:23:04,907][20347] Updated weights for policy 0, policy_version 1838774 (0.0010) [2023-12-27 18:23:05,131][20381] Updated weights for policy 1, policy_version 1837218 (0.0010) [2023-12-27 18:23:05,182][20381] Updated weights for policy 1, policy_version 1837228 (0.0010) [2023-12-27 18:23:05,234][20381] Updated weights for policy 1, policy_version 1837238 (0.0010) [2023-12-27 18:23:05,540][20347] Updated weights for policy 0, policy_version 1838784 (0.0010) [2023-12-27 18:23:05,598][20347] Updated weights for policy 0, policy_version 1838794 (0.0010) [2023-12-27 18:23:05,655][20347] Updated weights for policy 0, policy_version 1838804 (0.0010) [2023-12-27 18:23:05,906][20381] Updated weights for policy 1, policy_version 1837248 (0.0011) [2023-12-27 18:23:05,956][20381] Updated weights for policy 1, policy_version 1837258 (0.0008) [2023-12-27 18:23:06,019][20381] Updated weights for policy 1, policy_version 1837268 (0.0007) [2023-12-27 18:23:06,353][20347] Updated weights for policy 0, policy_version 1838814 (0.0010) [2023-12-27 18:23:06,405][20347] Updated weights for policy 0, policy_version 1838824 (0.0010) [2023-12-27 18:23:06,464][20347] Updated weights for policy 0, policy_version 1838834 (0.0010) [2023-12-27 18:23:06,658][20381] Updated weights for policy 1, policy_version 1837278 (0.0009) [2023-12-27 18:23:06,717][20381] Updated weights for policy 1, policy_version 1837288 (0.0011) [2023-12-27 18:23:06,773][20381] Updated weights for policy 1, policy_version 1837298 (0.0011) [2023-12-27 18:23:07,104][20347] Updated weights for policy 0, policy_version 1838844 (0.0010) [2023-12-27 18:23:07,163][20347] Updated weights for policy 0, policy_version 1838854 (0.0010) [2023-12-27 18:23:07,223][20347] Updated weights for policy 0, policy_version 1838864 (0.0010) [2023-12-27 18:23:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 941236224. Throughput: 0: 10106.7, 1: 10277.1. Samples: 941203696. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.270')] [2023-12-27 18:23:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001837304_470401024.pth... [2023-12-27 18:23:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001838872_470835200.pth... [2023-12-27 18:23:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001836120_470097920.pth [2023-12-27 18:23:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001837688_470532096.pth [2023-12-27 18:23:07,464][20381] Updated weights for policy 1, policy_version 1837308 (0.0010) [2023-12-27 18:23:07,526][20381] Updated weights for policy 1, policy_version 1837318 (0.0011) [2023-12-27 18:23:07,581][20381] Updated weights for policy 1, policy_version 1837328 (0.0011) [2023-12-27 18:23:07,815][20347] Updated weights for policy 0, policy_version 1838874 (0.0009) [2023-12-27 18:23:07,870][20347] Updated weights for policy 0, policy_version 1838884 (0.0008) [2023-12-27 18:23:07,927][20347] Updated weights for policy 0, policy_version 1838894 (0.0010) [2023-12-27 18:23:07,987][20347] Updated weights for policy 0, policy_version 1838904 (0.0010) [2023-12-27 18:23:08,236][20381] Updated weights for policy 1, policy_version 1837338 (0.0010) [2023-12-27 18:23:08,294][20381] Updated weights for policy 1, policy_version 1837348 (0.0005) [2023-12-27 18:23:08,358][20381] Updated weights for policy 1, policy_version 1837358 (0.0007) [2023-12-27 18:23:08,416][20381] Updated weights for policy 1, policy_version 1837368 (0.0008) [2023-12-27 18:23:08,670][20347] Updated weights for policy 0, policy_version 1838914 (0.0011) [2023-12-27 18:23:08,728][20347] Updated weights for policy 0, policy_version 1838924 (0.0010) [2023-12-27 18:23:08,786][20347] Updated weights for policy 0, policy_version 1838934 (0.0010) [2023-12-27 18:23:09,072][20381] Updated weights for policy 1, policy_version 1837378 (0.0008) [2023-12-27 18:23:09,133][20381] Updated weights for policy 1, policy_version 1837388 (0.0006) [2023-12-27 18:23:09,201][20381] Updated weights for policy 1, policy_version 1837398 (0.0006) [2023-12-27 18:23:09,516][20347] Updated weights for policy 0, policy_version 1838944 (0.0007) [2023-12-27 18:23:09,576][20347] Updated weights for policy 0, policy_version 1838954 (0.0009) [2023-12-27 18:23:09,633][20347] Updated weights for policy 0, policy_version 1838964 (0.0011) [2023-12-27 18:23:09,946][20381] Updated weights for policy 1, policy_version 1837408 (0.0007) [2023-12-27 18:23:10,006][20381] Updated weights for policy 1, policy_version 1837418 (0.0009) [2023-12-27 18:23:10,066][20381] Updated weights for policy 1, policy_version 1837428 (0.0008) [2023-12-27 18:23:10,343][20347] Updated weights for policy 0, policy_version 1838974 (0.0009) [2023-12-27 18:23:10,401][20347] Updated weights for policy 0, policy_version 1838984 (0.0006) [2023-12-27 18:23:10,448][20347] Updated weights for policy 0, policy_version 1838994 (0.0005) [2023-12-27 18:23:10,705][20381] Updated weights for policy 1, policy_version 1837438 (0.0010) [2023-12-27 18:23:10,764][20381] Updated weights for policy 1, policy_version 1837448 (0.0010) [2023-12-27 18:23:10,816][20381] Updated weights for policy 1, policy_version 1837458 (0.0010) [2023-12-27 18:23:11,135][20347] Updated weights for policy 0, policy_version 1839004 (0.0007) [2023-12-27 18:23:11,187][20347] Updated weights for policy 0, policy_version 1839014 (0.0009) [2023-12-27 18:23:11,241][20347] Updated weights for policy 0, policy_version 1839024 (0.0008) [2023-12-27 18:23:11,545][20381] Updated weights for policy 1, policy_version 1837468 (0.0009) [2023-12-27 18:23:11,610][20381] Updated weights for policy 1, policy_version 1837478 (0.0008) [2023-12-27 18:23:11,673][20381] Updated weights for policy 1, policy_version 1837488 (0.0007) [2023-12-27 18:23:11,933][20347] Updated weights for policy 0, policy_version 1839034 (0.0008) [2023-12-27 18:23:11,984][20347] Updated weights for policy 0, policy_version 1839044 (0.0011) [2023-12-27 18:23:12,044][20347] Updated weights for policy 0, policy_version 1839054 (0.0010) [2023-12-27 18:23:12,107][20347] Updated weights for policy 0, policy_version 1839064 (0.0008) [2023-12-27 18:23:12,328][20381] Updated weights for policy 1, policy_version 1837498 (0.0008) [2023-12-27 18:23:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 941334528. Throughput: 0: 10173.0, 1: 10344.6. Samples: 941327660. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.450')] [2023-12-27 18:23:12,392][20381] Updated weights for policy 1, policy_version 1837508 (0.0007) [2023-12-27 18:23:12,454][20381] Updated weights for policy 1, policy_version 1837518 (0.0007) [2023-12-27 18:23:12,509][20381] Updated weights for policy 1, policy_version 1837528 (0.0008) [2023-12-27 18:23:12,755][20347] Updated weights for policy 0, policy_version 1839074 (0.0009) [2023-12-27 18:23:12,808][20347] Updated weights for policy 0, policy_version 1839084 (0.0011) [2023-12-27 18:23:12,868][20347] Updated weights for policy 0, policy_version 1839094 (0.0011) [2023-12-27 18:23:13,137][20381] Updated weights for policy 1, policy_version 1837538 (0.0008) [2023-12-27 18:23:13,185][20381] Updated weights for policy 1, policy_version 1837548 (0.0007) [2023-12-27 18:23:13,234][20381] Updated weights for policy 1, policy_version 1837558 (0.0007) [2023-12-27 18:23:13,547][20347] Updated weights for policy 0, policy_version 1839104 (0.0010) [2023-12-27 18:23:13,598][20347] Updated weights for policy 0, policy_version 1839114 (0.0010) [2023-12-27 18:23:13,646][20347] Updated weights for policy 0, policy_version 1839124 (0.0010) [2023-12-27 18:23:13,944][20381] Updated weights for policy 1, policy_version 1837568 (0.0006) [2023-12-27 18:23:13,989][20381] Updated weights for policy 1, policy_version 1837578 (0.0005) [2023-12-27 18:23:14,042][20381] Updated weights for policy 1, policy_version 1837588 (0.0005) [2023-12-27 18:23:14,327][20347] Updated weights for policy 0, policy_version 1839134 (0.0009) [2023-12-27 18:23:14,382][20347] Updated weights for policy 0, policy_version 1839144 (0.0007) [2023-12-27 18:23:14,439][20347] Updated weights for policy 0, policy_version 1839154 (0.0007) [2023-12-27 18:23:14,701][20381] Updated weights for policy 1, policy_version 1837598 (0.0007) [2023-12-27 18:23:14,759][20381] Updated weights for policy 1, policy_version 1837608 (0.0007) [2023-12-27 18:23:14,822][20381] Updated weights for policy 1, policy_version 1837618 (0.0006) [2023-12-27 18:23:15,171][20347] Updated weights for policy 0, policy_version 1839164 (0.0008) [2023-12-27 18:23:15,223][20347] Updated weights for policy 0, policy_version 1839174 (0.0008) [2023-12-27 18:23:15,279][20347] Updated weights for policy 0, policy_version 1839184 (0.0008) [2023-12-27 18:23:15,407][20381] Updated weights for policy 1, policy_version 1837628 (0.0007) [2023-12-27 18:23:15,458][20381] Updated weights for policy 1, policy_version 1837638 (0.0007) [2023-12-27 18:23:15,505][20381] Updated weights for policy 1, policy_version 1837648 (0.0007) [2023-12-27 18:23:15,949][20347] Updated weights for policy 0, policy_version 1839194 (0.0007) [2023-12-27 18:23:16,006][20347] Updated weights for policy 0, policy_version 1839204 (0.0007) [2023-12-27 18:23:16,067][20347] Updated weights for policy 0, policy_version 1839214 (0.0010) [2023-12-27 18:23:16,125][20347] Updated weights for policy 0, policy_version 1839224 (0.0011) [2023-12-27 18:23:16,219][20381] Updated weights for policy 1, policy_version 1837658 (0.0007) [2023-12-27 18:23:16,266][20381] Updated weights for policy 1, policy_version 1837668 (0.0007) [2023-12-27 18:23:16,312][20381] Updated weights for policy 1, policy_version 1837678 (0.0007) [2023-12-27 18:23:16,361][20381] Updated weights for policy 1, policy_version 1837688 (0.0007) [2023-12-27 18:23:16,719][20347] Updated weights for policy 0, policy_version 1839234 (0.0010) [2023-12-27 18:23:16,779][20347] Updated weights for policy 0, policy_version 1839244 (0.0010) [2023-12-27 18:23:16,837][20347] Updated weights for policy 0, policy_version 1839254 (0.0010) [2023-12-27 18:23:17,095][20381] Updated weights for policy 1, policy_version 1837698 (0.0007) [2023-12-27 18:23:17,156][20381] Updated weights for policy 1, policy_version 1837708 (0.0005) [2023-12-27 18:23:17,217][20381] Updated weights for policy 1, policy_version 1837718 (0.0005) [2023-12-27 18:23:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 941441024. Throughput: 0: 10207.1, 1: 10308.8. Samples: 941450812. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.750')] [2023-12-27 18:23:17,476][20347] Updated weights for policy 0, policy_version 1839264 (0.0008) [2023-12-27 18:23:17,527][20347] Updated weights for policy 0, policy_version 1839274 (0.0007) [2023-12-27 18:23:17,577][20347] Updated weights for policy 0, policy_version 1839284 (0.0007) [2023-12-27 18:23:17,812][20381] Updated weights for policy 1, policy_version 1837728 (0.0007) [2023-12-27 18:23:17,860][20381] Updated weights for policy 1, policy_version 1837738 (0.0007) [2023-12-27 18:23:17,907][20381] Updated weights for policy 1, policy_version 1837748 (0.0005) [2023-12-27 18:23:18,295][20347] Updated weights for policy 0, policy_version 1839294 (0.0007) [2023-12-27 18:23:18,350][20347] Updated weights for policy 0, policy_version 1839304 (0.0008) [2023-12-27 18:23:18,409][20347] Updated weights for policy 0, policy_version 1839314 (0.0007) [2023-12-27 18:23:18,572][20381] Updated weights for policy 1, policy_version 1837758 (0.0009) [2023-12-27 18:23:18,636][20381] Updated weights for policy 1, policy_version 1837768 (0.0011) [2023-12-27 18:23:18,686][20381] Updated weights for policy 1, policy_version 1837778 (0.0011) [2023-12-27 18:23:19,124][20347] Updated weights for policy 0, policy_version 1839324 (0.0006) [2023-12-27 18:23:19,180][20347] Updated weights for policy 0, policy_version 1839334 (0.0008) [2023-12-27 18:23:19,236][20347] Updated weights for policy 0, policy_version 1839344 (0.0008) [2023-12-27 18:23:19,415][20381] Updated weights for policy 1, policy_version 1837788 (0.0009) [2023-12-27 18:23:19,477][20381] Updated weights for policy 1, policy_version 1837798 (0.0006) [2023-12-27 18:23:19,539][20381] Updated weights for policy 1, policy_version 1837808 (0.0008) [2023-12-27 18:23:19,954][20347] Updated weights for policy 0, policy_version 1839354 (0.0008) [2023-12-27 18:23:20,008][20347] Updated weights for policy 0, policy_version 1839364 (0.0007) [2023-12-27 18:23:20,064][20347] Updated weights for policy 0, policy_version 1839374 (0.0007) [2023-12-27 18:23:20,118][20347] Updated weights for policy 0, policy_version 1839384 (0.0006) [2023-12-27 18:23:20,288][20381] Updated weights for policy 1, policy_version 1837818 (0.0009) [2023-12-27 18:23:20,350][20381] Updated weights for policy 1, policy_version 1837828 (0.0008) [2023-12-27 18:23:20,410][20381] Updated weights for policy 1, policy_version 1837838 (0.0007) [2023-12-27 18:23:20,476][20381] Updated weights for policy 1, policy_version 1837848 (0.0011) [2023-12-27 18:23:20,782][20347] Updated weights for policy 0, policy_version 1839394 (0.0008) [2023-12-27 18:23:20,843][20347] Updated weights for policy 0, policy_version 1839404 (0.0005) [2023-12-27 18:23:20,905][20347] Updated weights for policy 0, policy_version 1839414 (0.0008) [2023-12-27 18:23:21,180][20381] Updated weights for policy 1, policy_version 1837858 (0.0006) [2023-12-27 18:23:21,245][20381] Updated weights for policy 1, policy_version 1837868 (0.0009) [2023-12-27 18:23:21,304][20381] Updated weights for policy 1, policy_version 1837878 (0.0008) [2023-12-27 18:23:21,709][20347] Updated weights for policy 0, policy_version 1839424 (0.0009) [2023-12-27 18:23:21,775][20347] Updated weights for policy 0, policy_version 1839434 (0.0008) [2023-12-27 18:23:21,832][20347] Updated weights for policy 0, policy_version 1839444 (0.0006) [2023-12-27 18:23:22,005][20381] Updated weights for policy 1, policy_version 1837888 (0.0008) [2023-12-27 18:23:22,068][20381] Updated weights for policy 1, policy_version 1837898 (0.0009) [2023-12-27 18:23:22,133][20381] Updated weights for policy 1, policy_version 1837908 (0.0008) [2023-12-27 18:23:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 941539328. Throughput: 0: 10170.6, 1: 10269.3. Samples: 941511216. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.380')] [2023-12-27 18:23:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001837912_470556672.pth... [2023-12-27 18:23:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001839448_470982656.pth... [2023-12-27 18:23:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001838264_470679552.pth [2023-12-27 18:23:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001836728_470253568.pth [2023-12-27 18:23:22,531][20347] Updated weights for policy 0, policy_version 1839454 (0.0008) [2023-12-27 18:23:22,584][20347] Updated weights for policy 0, policy_version 1839464 (0.0008) [2023-12-27 18:23:22,646][20347] Updated weights for policy 0, policy_version 1839474 (0.0008) [2023-12-27 18:23:22,800][20381] Updated weights for policy 1, policy_version 1837918 (0.0008) [2023-12-27 18:23:22,861][20381] Updated weights for policy 1, policy_version 1837928 (0.0008) [2023-12-27 18:23:22,921][20381] Updated weights for policy 1, policy_version 1837938 (0.0007) [2023-12-27 18:23:23,327][20347] Updated weights for policy 0, policy_version 1839484 (0.0008) [2023-12-27 18:23:23,392][20347] Updated weights for policy 0, policy_version 1839494 (0.0008) [2023-12-27 18:23:23,446][20347] Updated weights for policy 0, policy_version 1839504 (0.0005) [2023-12-27 18:23:23,666][20381] Updated weights for policy 1, policy_version 1837948 (0.0008) [2023-12-27 18:23:23,717][20381] Updated weights for policy 1, policy_version 1837958 (0.0010) [2023-12-27 18:23:23,779][20381] Updated weights for policy 1, policy_version 1837968 (0.0008) [2023-12-27 18:23:24,006][20347] Updated weights for policy 0, policy_version 1839514 (0.0006) [2023-12-27 18:23:24,067][20347] Updated weights for policy 0, policy_version 1839524 (0.0007) [2023-12-27 18:23:24,120][20347] Updated weights for policy 0, policy_version 1839534 (0.0008) [2023-12-27 18:23:24,181][20347] Updated weights for policy 0, policy_version 1839544 (0.0008) [2023-12-27 18:23:24,484][20381] Updated weights for policy 1, policy_version 1837978 (0.0007) [2023-12-27 18:23:24,538][20381] Updated weights for policy 1, policy_version 1837988 (0.0008) [2023-12-27 18:23:24,588][20381] Updated weights for policy 1, policy_version 1837998 (0.0008) [2023-12-27 18:23:24,644][20381] Updated weights for policy 1, policy_version 1838008 (0.0008) [2023-12-27 18:23:24,906][20347] Updated weights for policy 0, policy_version 1839554 (0.0009) [2023-12-27 18:23:24,955][20347] Updated weights for policy 0, policy_version 1839564 (0.0009) [2023-12-27 18:23:25,011][20347] Updated weights for policy 0, policy_version 1839574 (0.0010) [2023-12-27 18:23:25,339][20381] Updated weights for policy 1, policy_version 1838018 (0.0008) [2023-12-27 18:23:25,388][20381] Updated weights for policy 1, policy_version 1838028 (0.0006) [2023-12-27 18:23:25,436][20381] Updated weights for policy 1, policy_version 1838038 (0.0007) [2023-12-27 18:23:25,622][20347] Updated weights for policy 0, policy_version 1839584 (0.0006) [2023-12-27 18:23:25,676][20347] Updated weights for policy 0, policy_version 1839594 (0.0006) [2023-12-27 18:23:25,737][20347] Updated weights for policy 0, policy_version 1839604 (0.0007) [2023-12-27 18:23:26,128][20381] Updated weights for policy 1, policy_version 1838048 (0.0008) [2023-12-27 18:23:26,187][20381] Updated weights for policy 1, policy_version 1838058 (0.0008) [2023-12-27 18:23:26,237][20381] Updated weights for policy 1, policy_version 1838068 (0.0008) [2023-12-27 18:23:26,283][20347] Updated weights for policy 0, policy_version 1839614 (0.0007) [2023-12-27 18:23:26,332][20347] Updated weights for policy 0, policy_version 1839624 (0.0007) [2023-12-27 18:23:26,379][20347] Updated weights for policy 0, policy_version 1839634 (0.0007) [2023-12-27 18:23:26,839][20381] Updated weights for policy 1, policy_version 1838078 (0.0008) [2023-12-27 18:23:26,889][20381] Updated weights for policy 1, policy_version 1838088 (0.0007) [2023-12-27 18:23:26,938][20381] Updated weights for policy 1, policy_version 1838098 (0.0006) [2023-12-27 18:23:27,081][20347] Updated weights for policy 0, policy_version 1839644 (0.0007) [2023-12-27 18:23:27,144][20347] Updated weights for policy 0, policy_version 1839654 (0.0006) [2023-12-27 18:23:27,204][20347] Updated weights for policy 0, policy_version 1839664 (0.0007) [2023-12-27 18:23:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 941645824. Throughput: 0: 10263.7, 1: 10195.5. Samples: 941632400. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.950')] [2023-12-27 18:23:27,608][20381] Updated weights for policy 1, policy_version 1838108 (0.0006) [2023-12-27 18:23:27,657][20381] Updated weights for policy 1, policy_version 1838118 (0.0007) [2023-12-27 18:23:27,710][20381] Updated weights for policy 1, policy_version 1838128 (0.0007) [2023-12-27 18:23:27,829][20347] Updated weights for policy 0, policy_version 1839674 (0.0009) [2023-12-27 18:23:27,879][20347] Updated weights for policy 0, policy_version 1839684 (0.0005) [2023-12-27 18:23:27,940][20347] Updated weights for policy 0, policy_version 1839694 (0.0005) [2023-12-27 18:23:27,995][20347] Updated weights for policy 0, policy_version 1839704 (0.0005) [2023-12-27 18:23:28,427][20381] Updated weights for policy 1, policy_version 1838138 (0.0007) [2023-12-27 18:23:28,478][20381] Updated weights for policy 1, policy_version 1838148 (0.0007) [2023-12-27 18:23:28,534][20381] Updated weights for policy 1, policy_version 1838158 (0.0007) [2023-12-27 18:23:28,573][20347] Updated weights for policy 0, policy_version 1839714 (0.0010) [2023-12-27 18:23:28,590][20381] Updated weights for policy 1, policy_version 1838168 (0.0005) [2023-12-27 18:23:28,626][20347] Updated weights for policy 0, policy_version 1839724 (0.0010) [2023-12-27 18:23:28,687][20347] Updated weights for policy 0, policy_version 1839734 (0.0006) [2023-12-27 18:23:29,312][20381] Updated weights for policy 1, policy_version 1838178 (0.0008) [2023-12-27 18:23:29,347][20347] Updated weights for policy 0, policy_version 1839744 (0.0007) [2023-12-27 18:23:29,372][20381] Updated weights for policy 1, policy_version 1838188 (0.0007) [2023-12-27 18:23:29,406][20347] Updated weights for policy 0, policy_version 1839754 (0.0008) [2023-12-27 18:23:29,434][20381] Updated weights for policy 1, policy_version 1838198 (0.0008) [2023-12-27 18:23:29,465][20347] Updated weights for policy 0, policy_version 1839764 (0.0006) [2023-12-27 18:23:30,125][20347] Updated weights for policy 0, policy_version 1839774 (0.0005) [2023-12-27 18:23:30,177][20347] Updated weights for policy 0, policy_version 1839784 (0.0007) [2023-12-27 18:23:30,207][20381] Updated weights for policy 1, policy_version 1838208 (0.0010) [2023-12-27 18:23:30,233][20347] Updated weights for policy 0, policy_version 1839794 (0.0006) [2023-12-27 18:23:30,266][20381] Updated weights for policy 1, policy_version 1838218 (0.0011) [2023-12-27 18:23:30,322][20381] Updated weights for policy 1, policy_version 1838228 (0.0011) [2023-12-27 18:23:30,834][20347] Updated weights for policy 0, policy_version 1839804 (0.0009) [2023-12-27 18:23:30,878][20347] Updated weights for policy 0, policy_version 1839814 (0.0009) [2023-12-27 18:23:30,925][20347] Updated weights for policy 0, policy_version 1839824 (0.0007) [2023-12-27 18:23:30,940][20381] Updated weights for policy 1, policy_version 1838238 (0.0008) [2023-12-27 18:23:31,004][20381] Updated weights for policy 1, policy_version 1838248 (0.0006) [2023-12-27 18:23:31,068][20381] Updated weights for policy 1, policy_version 1838258 (0.0008) [2023-12-27 18:23:31,684][20347] Updated weights for policy 0, policy_version 1839834 (0.0007) [2023-12-27 18:23:31,705][20381] Updated weights for policy 1, policy_version 1838268 (0.0007) [2023-12-27 18:23:31,749][20347] Updated weights for policy 0, policy_version 1839844 (0.0009) [2023-12-27 18:23:31,773][20381] Updated weights for policy 1, policy_version 1838278 (0.0007) [2023-12-27 18:23:31,808][20347] Updated weights for policy 0, policy_version 1839854 (0.0010) [2023-12-27 18:23:31,830][20381] Updated weights for policy 1, policy_version 1838288 (0.0005) [2023-12-27 18:23:31,863][20347] Updated weights for policy 0, policy_version 1839864 (0.0010) [2023-12-27 18:23:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 941744128. Throughput: 0: 10321.8, 1: 10130.8. Samples: 941757048. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.780')] [2023-12-27 18:23:32,492][20381] Updated weights for policy 1, policy_version 1838298 (0.0006) [2023-12-27 18:23:32,552][20381] Updated weights for policy 1, policy_version 1838308 (0.0009) [2023-12-27 18:23:32,558][20347] Updated weights for policy 0, policy_version 1839874 (0.0011) [2023-12-27 18:23:32,606][20381] Updated weights for policy 1, policy_version 1838318 (0.0007) [2023-12-27 18:23:32,620][20347] Updated weights for policy 0, policy_version 1839884 (0.0009) [2023-12-27 18:23:32,653][20381] Updated weights for policy 1, policy_version 1838328 (0.0008) [2023-12-27 18:23:32,672][20347] Updated weights for policy 0, policy_version 1839894 (0.0009) [2023-12-27 18:23:33,321][20381] Updated weights for policy 1, policy_version 1838338 (0.0008) [2023-12-27 18:23:33,357][20347] Updated weights for policy 0, policy_version 1839904 (0.0008) [2023-12-27 18:23:33,375][20381] Updated weights for policy 1, policy_version 1838348 (0.0010) [2023-12-27 18:23:33,413][20347] Updated weights for policy 0, policy_version 1839914 (0.0008) [2023-12-27 18:23:33,429][20381] Updated weights for policy 1, policy_version 1838358 (0.0010) [2023-12-27 18:23:33,463][20347] Updated weights for policy 0, policy_version 1839924 (0.0008) [2023-12-27 18:23:34,106][20347] Updated weights for policy 0, policy_version 1839934 (0.0009) [2023-12-27 18:23:34,107][20381] Updated weights for policy 1, policy_version 1838368 (0.0011) [2023-12-27 18:23:34,165][20347] Updated weights for policy 0, policy_version 1839944 (0.0009) [2023-12-27 18:23:34,168][20381] Updated weights for policy 1, policy_version 1838378 (0.0011) [2023-12-27 18:23:34,223][20347] Updated weights for policy 0, policy_version 1839954 (0.0008) [2023-12-27 18:23:34,226][20381] Updated weights for policy 1, policy_version 1838388 (0.0011) [2023-12-27 18:23:34,896][20347] Updated weights for policy 0, policy_version 1839964 (0.0009) [2023-12-27 18:23:34,921][20381] Updated weights for policy 1, policy_version 1838398 (0.0011) [2023-12-27 18:23:34,949][20347] Updated weights for policy 0, policy_version 1839974 (0.0011) [2023-12-27 18:23:34,973][20381] Updated weights for policy 1, policy_version 1838408 (0.0005) [2023-12-27 18:23:35,001][20347] Updated weights for policy 0, policy_version 1839984 (0.0010) [2023-12-27 18:23:35,031][20381] Updated weights for policy 1, policy_version 1838418 (0.0006) [2023-12-27 18:23:35,659][20381] Updated weights for policy 1, policy_version 1838428 (0.0005) [2023-12-27 18:23:35,691][20347] Updated weights for policy 0, policy_version 1839994 (0.0011) [2023-12-27 18:23:35,722][20381] Updated weights for policy 1, policy_version 1838438 (0.0005) [2023-12-27 18:23:35,745][20347] Updated weights for policy 0, policy_version 1840004 (0.0009) [2023-12-27 18:23:35,779][20381] Updated weights for policy 1, policy_version 1838448 (0.0005) [2023-12-27 18:23:35,794][20347] Updated weights for policy 0, policy_version 1840014 (0.0010) [2023-12-27 18:23:35,839][20347] Updated weights for policy 0, policy_version 1840024 (0.0010) [2023-12-27 18:23:36,325][20381] Updated weights for policy 1, policy_version 1838458 (0.0006) [2023-12-27 18:23:36,375][20381] Updated weights for policy 1, policy_version 1838468 (0.0010) [2023-12-27 18:23:36,429][20381] Updated weights for policy 1, policy_version 1838478 (0.0010) [2023-12-27 18:23:36,491][20381] Updated weights for policy 1, policy_version 1838488 (0.0009) [2023-12-27 18:23:36,511][20347] Updated weights for policy 0, policy_version 1840034 (0.0008) [2023-12-27 18:23:36,576][20347] Updated weights for policy 0, policy_version 1840044 (0.0008) [2023-12-27 18:23:36,637][20347] Updated weights for policy 0, policy_version 1840054 (0.0010) [2023-12-27 18:23:37,166][20381] Updated weights for policy 1, policy_version 1838498 (0.0011) [2023-12-27 18:23:37,221][20381] Updated weights for policy 1, policy_version 1838508 (0.0010) [2023-12-27 18:23:37,256][20347] Updated weights for policy 0, policy_version 1840064 (0.0010) [2023-12-27 18:23:37,280][20381] Updated weights for policy 1, policy_version 1838518 (0.0011) [2023-12-27 18:23:37,317][20347] Updated weights for policy 0, policy_version 1840074 (0.0010) [2023-12-27 18:23:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 941850624. Throughput: 0: 10334.7, 1: 10124.2. Samples: 941818932. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.910')] [2023-12-27 18:23:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001838520_470712320.pth... [2023-12-27 18:23:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001837304_470401024.pth [2023-12-27 18:23:37,375][20347] Updated weights for policy 0, policy_version 1840084 (0.0010) [2023-12-27 18:23:37,399][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001840088_471146496.pth... [2023-12-27 18:23:37,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001838872_470835200.pth [2023-12-27 18:23:37,893][20381] Updated weights for policy 1, policy_version 1838528 (0.0007) [2023-12-27 18:23:37,942][20381] Updated weights for policy 1, policy_version 1838538 (0.0007) [2023-12-27 18:23:37,995][20381] Updated weights for policy 1, policy_version 1838548 (0.0009) [2023-12-27 18:23:38,049][20347] Updated weights for policy 0, policy_version 1840094 (0.0010) [2023-12-27 18:23:38,107][20347] Updated weights for policy 0, policy_version 1840104 (0.0010) [2023-12-27 18:23:38,164][20347] Updated weights for policy 0, policy_version 1840114 (0.0010) [2023-12-27 18:23:38,605][20381] Updated weights for policy 1, policy_version 1838558 (0.0009) [2023-12-27 18:23:38,661][20381] Updated weights for policy 1, policy_version 1838568 (0.0006) [2023-12-27 18:23:38,708][20381] Updated weights for policy 1, policy_version 1838578 (0.0005) [2023-12-27 18:23:38,853][20347] Updated weights for policy 0, policy_version 1840124 (0.0010) [2023-12-27 18:23:38,911][20347] Updated weights for policy 0, policy_version 1840134 (0.0010) [2023-12-27 18:23:38,963][20347] Updated weights for policy 0, policy_version 1840144 (0.0010) [2023-12-27 18:23:39,320][20381] Updated weights for policy 1, policy_version 1838588 (0.0009) [2023-12-27 18:23:39,393][20381] Updated weights for policy 1, policy_version 1838598 (0.0009) [2023-12-27 18:23:39,463][20381] Updated weights for policy 1, policy_version 1838608 (0.0008) [2023-12-27 18:23:39,701][20347] Updated weights for policy 0, policy_version 1840154 (0.0010) [2023-12-27 18:23:39,754][20347] Updated weights for policy 0, policy_version 1840164 (0.0008) [2023-12-27 18:23:39,808][20347] Updated weights for policy 0, policy_version 1840174 (0.0007) [2023-12-27 18:23:39,872][20347] Updated weights for policy 0, policy_version 1840184 (0.0008) [2023-12-27 18:23:40,111][20381] Updated weights for policy 1, policy_version 1838618 (0.0007) [2023-12-27 18:23:40,170][20381] Updated weights for policy 1, policy_version 1838628 (0.0007) [2023-12-27 18:23:40,235][20381] Updated weights for policy 1, policy_version 1838638 (0.0006) [2023-12-27 18:23:40,297][20381] Updated weights for policy 1, policy_version 1838648 (0.0007) [2023-12-27 18:23:40,648][20347] Updated weights for policy 0, policy_version 1840194 (0.0008) [2023-12-27 18:23:40,698][20347] Updated weights for policy 0, policy_version 1840204 (0.0006) [2023-12-27 18:23:40,754][20347] Updated weights for policy 0, policy_version 1840214 (0.0007) [2023-12-27 18:23:40,899][20381] Updated weights for policy 1, policy_version 1838658 (0.0007) [2023-12-27 18:23:40,955][20381] Updated weights for policy 1, policy_version 1838668 (0.0007) [2023-12-27 18:23:41,015][20381] Updated weights for policy 1, policy_version 1838678 (0.0007) [2023-12-27 18:23:41,483][20347] Updated weights for policy 0, policy_version 1840224 (0.0009) [2023-12-27 18:23:41,546][20347] Updated weights for policy 0, policy_version 1840234 (0.0007) [2023-12-27 18:23:41,606][20347] Updated weights for policy 0, policy_version 1840244 (0.0007) [2023-12-27 18:23:41,779][20381] Updated weights for policy 1, policy_version 1838688 (0.0009) [2023-12-27 18:23:41,845][20381] Updated weights for policy 1, policy_version 1838698 (0.0007) [2023-12-27 18:23:41,910][20381] Updated weights for policy 1, policy_version 1838708 (0.0005) [2023-12-27 18:23:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 941948928. Throughput: 0: 10317.5, 1: 10279.5. Samples: 941944588. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.430')] [2023-12-27 18:23:42,368][20347] Updated weights for policy 0, policy_version 1840254 (0.0008) [2023-12-27 18:23:42,430][20347] Updated weights for policy 0, policy_version 1840264 (0.0008) [2023-12-27 18:23:42,486][20347] Updated weights for policy 0, policy_version 1840274 (0.0008) [2023-12-27 18:23:42,541][20381] Updated weights for policy 1, policy_version 1838718 (0.0007) [2023-12-27 18:23:42,607][20381] Updated weights for policy 1, policy_version 1838728 (0.0008) [2023-12-27 18:23:42,664][20381] Updated weights for policy 1, policy_version 1838738 (0.0007) [2023-12-27 18:23:43,217][20347] Updated weights for policy 0, policy_version 1840284 (0.0006) [2023-12-27 18:23:43,247][20381] Updated weights for policy 1, policy_version 1838748 (0.0007) [2023-12-27 18:23:43,266][20347] Updated weights for policy 0, policy_version 1840294 (0.0007) [2023-12-27 18:23:43,292][20381] Updated weights for policy 1, policy_version 1838758 (0.0008) [2023-12-27 18:23:43,323][20347] Updated weights for policy 0, policy_version 1840304 (0.0007) [2023-12-27 18:23:43,348][20381] Updated weights for policy 1, policy_version 1838768 (0.0007) [2023-12-27 18:23:43,924][20347] Updated weights for policy 0, policy_version 1840314 (0.0006) [2023-12-27 18:23:43,978][20347] Updated weights for policy 0, policy_version 1840324 (0.0010) [2023-12-27 18:23:44,030][20347] Updated weights for policy 0, policy_version 1840334 (0.0010) [2023-12-27 18:23:44,045][20381] Updated weights for policy 1, policy_version 1838778 (0.0006) [2023-12-27 18:23:44,072][20347] Updated weights for policy 0, policy_version 1840344 (0.0010) [2023-12-27 18:23:44,100][20381] Updated weights for policy 1, policy_version 1838788 (0.0007) [2023-12-27 18:23:44,148][20381] Updated weights for policy 1, policy_version 1838798 (0.0007) [2023-12-27 18:23:44,197][20381] Updated weights for policy 1, policy_version 1838808 (0.0007) [2023-12-27 18:23:44,780][20347] Updated weights for policy 0, policy_version 1840354 (0.0008) [2023-12-27 18:23:44,837][20347] Updated weights for policy 0, policy_version 1840364 (0.0008) [2023-12-27 18:23:44,878][20381] Updated weights for policy 1, policy_version 1838818 (0.0008) [2023-12-27 18:23:44,902][20347] Updated weights for policy 0, policy_version 1840374 (0.0007) [2023-12-27 18:23:44,942][20381] Updated weights for policy 1, policy_version 1838828 (0.0007) [2023-12-27 18:23:45,007][20381] Updated weights for policy 1, policy_version 1838838 (0.0008) [2023-12-27 18:23:45,598][20347] Updated weights for policy 0, policy_version 1840384 (0.0006) [2023-12-27 18:23:45,653][20347] Updated weights for policy 0, policy_version 1840394 (0.0005) [2023-12-27 18:23:45,706][20381] Updated weights for policy 1, policy_version 1838848 (0.0008) [2023-12-27 18:23:45,714][20347] Updated weights for policy 0, policy_version 1840404 (0.0005) [2023-12-27 18:23:45,763][20381] Updated weights for policy 1, policy_version 1838858 (0.0008) [2023-12-27 18:23:45,826][20381] Updated weights for policy 1, policy_version 1838868 (0.0005) [2023-12-27 18:23:46,355][20347] Updated weights for policy 0, policy_version 1840414 (0.0007) [2023-12-27 18:23:46,371][20381] Updated weights for policy 1, policy_version 1838878 (0.0006) [2023-12-27 18:23:46,410][20347] Updated weights for policy 0, policy_version 1840424 (0.0007) [2023-12-27 18:23:46,424][20381] Updated weights for policy 1, policy_version 1838888 (0.0006) [2023-12-27 18:23:46,463][20347] Updated weights for policy 0, policy_version 1840434 (0.0007) [2023-12-27 18:23:46,481][20381] Updated weights for policy 1, policy_version 1838898 (0.0008) [2023-12-27 18:23:47,029][20347] Updated weights for policy 0, policy_version 1840444 (0.0006) [2023-12-27 18:23:47,044][20381] Updated weights for policy 1, policy_version 1838908 (0.0007) [2023-12-27 18:23:47,075][20347] Updated weights for policy 0, policy_version 1840454 (0.0006) [2023-12-27 18:23:47,106][20381] Updated weights for policy 1, policy_version 1838918 (0.0005) [2023-12-27 18:23:47,125][20347] Updated weights for policy 0, policy_version 1840464 (0.0008) [2023-12-27 18:23:47,168][20381] Updated weights for policy 1, policy_version 1838928 (0.0005) [2023-12-27 18:23:47,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 942063616. Throughput: 0: 10266.8, 1: 10348.9. Samples: 942070328. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.620')] [2023-12-27 18:23:47,672][20381] Updated weights for policy 1, policy_version 1838938 (0.0005) [2023-12-27 18:23:47,739][20381] Updated weights for policy 1, policy_version 1838948 (0.0005) [2023-12-27 18:23:47,787][20381] Updated weights for policy 1, policy_version 1838958 (0.0005) [2023-12-27 18:23:47,835][20381] Updated weights for policy 1, policy_version 1838968 (0.0006) [2023-12-27 18:23:47,859][20347] Updated weights for policy 0, policy_version 1840474 (0.0007) [2023-12-27 18:23:47,911][20347] Updated weights for policy 0, policy_version 1840484 (0.0007) [2023-12-27 18:23:47,971][20347] Updated weights for policy 0, policy_version 1840495 (0.0009) [2023-12-27 18:23:48,445][20381] Updated weights for policy 1, policy_version 1838978 (0.0007) [2023-12-27 18:23:48,505][20381] Updated weights for policy 1, policy_version 1838988 (0.0008) [2023-12-27 18:23:48,566][20381] Updated weights for policy 1, policy_version 1838998 (0.0008) [2023-12-27 18:23:48,671][20347] Updated weights for policy 0, policy_version 1840505 (0.0006) [2023-12-27 18:23:48,730][20347] Updated weights for policy 0, policy_version 1840515 (0.0007) [2023-12-27 18:23:48,785][20347] Updated weights for policy 0, policy_version 1840525 (0.0008) [2023-12-27 18:23:48,847][20347] Updated weights for policy 0, policy_version 1840535 (0.0007) [2023-12-27 18:23:49,270][20381] Updated weights for policy 1, policy_version 1839008 (0.0009) [2023-12-27 18:23:49,331][20381] Updated weights for policy 1, policy_version 1839018 (0.0007) [2023-12-27 18:23:49,398][20381] Updated weights for policy 1, policy_version 1839028 (0.0007) [2023-12-27 18:23:49,591][20347] Updated weights for policy 0, policy_version 1840545 (0.0008) [2023-12-27 18:23:49,650][20347] Updated weights for policy 0, policy_version 1840555 (0.0007) [2023-12-27 18:23:49,709][20347] Updated weights for policy 0, policy_version 1840565 (0.0008) [2023-12-27 18:23:50,124][20381] Updated weights for policy 1, policy_version 1839038 (0.0005) [2023-12-27 18:23:50,191][20381] Updated weights for policy 1, policy_version 1839048 (0.0005) [2023-12-27 18:23:50,259][20381] Updated weights for policy 1, policy_version 1839058 (0.0006) [2023-12-27 18:23:50,421][20347] Updated weights for policy 0, policy_version 1840575 (0.0006) [2023-12-27 18:23:50,485][20347] Updated weights for policy 0, policy_version 1840585 (0.0009) [2023-12-27 18:23:50,532][20347] Updated weights for policy 0, policy_version 1840595 (0.0007) [2023-12-27 18:23:50,877][20381] Updated weights for policy 1, policy_version 1839068 (0.0007) [2023-12-27 18:23:50,937][20381] Updated weights for policy 1, policy_version 1839078 (0.0008) [2023-12-27 18:23:50,994][20381] Updated weights for policy 1, policy_version 1839088 (0.0008) [2023-12-27 18:23:51,181][20347] Updated weights for policy 0, policy_version 1840605 (0.0007) [2023-12-27 18:23:51,245][20347] Updated weights for policy 0, policy_version 1840615 (0.0008) [2023-12-27 18:23:51,302][20347] Updated weights for policy 0, policy_version 1840625 (0.0007) [2023-12-27 18:23:51,742][20381] Updated weights for policy 1, policy_version 1839098 (0.0008) [2023-12-27 18:23:51,794][20381] Updated weights for policy 1, policy_version 1839108 (0.0006) [2023-12-27 18:23:51,844][20381] Updated weights for policy 1, policy_version 1839118 (0.0005) [2023-12-27 18:23:51,902][20381] Updated weights for policy 1, policy_version 1839128 (0.0007) [2023-12-27 18:23:51,972][20347] Updated weights for policy 0, policy_version 1840635 (0.0005) [2023-12-27 18:23:52,040][20347] Updated weights for policy 0, policy_version 1840645 (0.0005) [2023-12-27 18:23:52,103][20347] Updated weights for policy 0, policy_version 1840655 (0.0006) [2023-12-27 18:23:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 942161920. Throughput: 0: 10229.5, 1: 10372.5. Samples: 942130788. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.630')] [2023-12-27 18:23:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001839128_470867968.pth... [2023-12-27 18:23:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001840664_471293952.pth... [2023-12-27 18:23:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001837912_470556672.pth [2023-12-27 18:23:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001839448_470982656.pth [2023-12-27 18:23:52,610][20381] Updated weights for policy 1, policy_version 1839138 (0.0007) [2023-12-27 18:23:52,669][20347] Updated weights for policy 0, policy_version 1840665 (0.0006) [2023-12-27 18:23:52,675][20381] Updated weights for policy 1, policy_version 1839148 (0.0008) [2023-12-27 18:23:52,726][20347] Updated weights for policy 0, policy_version 1840675 (0.0006) [2023-12-27 18:23:52,732][20381] Updated weights for policy 1, policy_version 1839158 (0.0006) [2023-12-27 18:23:52,775][20347] Updated weights for policy 0, policy_version 1840685 (0.0007) [2023-12-27 18:23:52,829][20347] Updated weights for policy 0, policy_version 1840695 (0.0008) [2023-12-27 18:23:53,333][20381] Updated weights for policy 1, policy_version 1839168 (0.0005) [2023-12-27 18:23:53,385][20381] Updated weights for policy 1, policy_version 1839178 (0.0005) [2023-12-27 18:23:53,443][20381] Updated weights for policy 1, policy_version 1839188 (0.0005) [2023-12-27 18:23:53,490][20347] Updated weights for policy 0, policy_version 1840705 (0.0006) [2023-12-27 18:23:53,545][20347] Updated weights for policy 0, policy_version 1840715 (0.0005) [2023-12-27 18:23:53,597][20347] Updated weights for policy 0, policy_version 1840725 (0.0005) [2023-12-27 18:23:53,977][20381] Updated weights for policy 1, policy_version 1839198 (0.0007) [2023-12-27 18:23:54,037][20381] Updated weights for policy 1, policy_version 1839208 (0.0006) [2023-12-27 18:23:54,098][20381] Updated weights for policy 1, policy_version 1839218 (0.0005) [2023-12-27 18:23:54,118][20347] Updated weights for policy 0, policy_version 1840735 (0.0006) [2023-12-27 18:23:54,173][20347] Updated weights for policy 0, policy_version 1840745 (0.0005) [2023-12-27 18:23:54,227][20347] Updated weights for policy 0, policy_version 1840755 (0.0006) [2023-12-27 18:23:54,701][20381] Updated weights for policy 1, policy_version 1839228 (0.0007) [2023-12-27 18:23:54,761][20381] Updated weights for policy 1, policy_version 1839238 (0.0011) [2023-12-27 18:23:54,819][20381] Updated weights for policy 1, policy_version 1839248 (0.0011) [2023-12-27 18:23:54,883][20347] Updated weights for policy 0, policy_version 1840765 (0.0008) [2023-12-27 18:23:54,945][20347] Updated weights for policy 0, policy_version 1840775 (0.0007) [2023-12-27 18:23:55,006][20347] Updated weights for policy 0, policy_version 1840785 (0.0008) [2023-12-27 18:23:55,467][20381] Updated weights for policy 1, policy_version 1839258 (0.0010) [2023-12-27 18:23:55,518][20381] Updated weights for policy 1, policy_version 1839268 (0.0007) [2023-12-27 18:23:55,558][20381] Updated weights for policy 1, policy_version 1839278 (0.0006) [2023-12-27 18:23:55,609][20381] Updated weights for policy 1, policy_version 1839288 (0.0005) [2023-12-27 18:23:55,642][20347] Updated weights for policy 0, policy_version 1840795 (0.0007) [2023-12-27 18:23:55,685][20347] Updated weights for policy 0, policy_version 1840805 (0.0005) [2023-12-27 18:23:55,732][20347] Updated weights for policy 0, policy_version 1840815 (0.0007) [2023-12-27 18:23:56,272][20381] Updated weights for policy 1, policy_version 1839298 (0.0011) [2023-12-27 18:23:56,328][20381] Updated weights for policy 1, policy_version 1839308 (0.0011) [2023-12-27 18:23:56,378][20381] Updated weights for policy 1, policy_version 1839318 (0.0010) [2023-12-27 18:23:56,385][20347] Updated weights for policy 0, policy_version 1840825 (0.0007) [2023-12-27 18:23:56,441][20347] Updated weights for policy 0, policy_version 1840835 (0.0007) [2023-12-27 18:23:56,503][20347] Updated weights for policy 0, policy_version 1840845 (0.0006) [2023-12-27 18:23:56,561][20347] Updated weights for policy 0, policy_version 1840855 (0.0008) [2023-12-27 18:23:57,073][20381] Updated weights for policy 1, policy_version 1839328 (0.0007) [2023-12-27 18:23:57,128][20381] Updated weights for policy 1, policy_version 1839338 (0.0010) [2023-12-27 18:23:57,179][20381] Updated weights for policy 1, policy_version 1839348 (0.0010) [2023-12-27 18:23:57,186][20347] Updated weights for policy 0, policy_version 1840865 (0.0005) [2023-12-27 18:23:57,241][20347] Updated weights for policy 0, policy_version 1840875 (0.0007) [2023-12-27 18:23:57,294][20347] Updated weights for policy 0, policy_version 1840885 (0.0007) [2023-12-27 18:23:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 942276608. Throughput: 0: 10301.7, 1: 10426.7. Samples: 942260436. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:23:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.080')] [2023-12-27 18:23:57,771][20381] Updated weights for policy 1, policy_version 1839358 (0.0009) [2023-12-27 18:23:57,819][20381] Updated weights for policy 1, policy_version 1839368 (0.0010) [2023-12-27 18:23:57,876][20381] Updated weights for policy 1, policy_version 1839378 (0.0011) [2023-12-27 18:23:57,956][20347] Updated weights for policy 0, policy_version 1840895 (0.0009) [2023-12-27 18:23:58,005][20347] Updated weights for policy 0, policy_version 1840905 (0.0010) [2023-12-27 18:23:58,060][20347] Updated weights for policy 0, policy_version 1840915 (0.0010) [2023-12-27 18:23:58,647][20381] Updated weights for policy 1, policy_version 1839388 (0.0009) [2023-12-27 18:23:58,712][20381] Updated weights for policy 1, policy_version 1839398 (0.0008) [2023-12-27 18:23:58,781][20381] Updated weights for policy 1, policy_version 1839408 (0.0014) [2023-12-27 18:23:58,877][20347] Updated weights for policy 0, policy_version 1840925 (0.0009) [2023-12-27 18:23:58,929][20347] Updated weights for policy 0, policy_version 1840935 (0.0009) [2023-12-27 18:23:58,990][20347] Updated weights for policy 0, policy_version 1840945 (0.0010) [2023-12-27 18:23:59,593][20381] Updated weights for policy 1, policy_version 1839418 (0.0008) [2023-12-27 18:23:59,645][20381] Updated weights for policy 1, policy_version 1839428 (0.0005) [2023-12-27 18:23:59,698][20381] Updated weights for policy 1, policy_version 1839438 (0.0008) [2023-12-27 18:23:59,717][20347] Updated weights for policy 0, policy_version 1840955 (0.0008) [2023-12-27 18:23:59,755][20381] Updated weights for policy 1, policy_version 1839448 (0.0006) [2023-12-27 18:23:59,777][20347] Updated weights for policy 0, policy_version 1840965 (0.0007) [2023-12-27 18:23:59,853][20347] Updated weights for policy 0, policy_version 1840975 (0.0007) [2023-12-27 18:24:00,432][20381] Updated weights for policy 1, policy_version 1839458 (0.0008) [2023-12-27 18:24:00,486][20381] Updated weights for policy 1, policy_version 1839468 (0.0008) [2023-12-27 18:24:00,534][20347] Updated weights for policy 0, policy_version 1840985 (0.0009) [2023-12-27 18:24:00,540][20381] Updated weights for policy 1, policy_version 1839478 (0.0007) [2023-12-27 18:24:00,586][20347] Updated weights for policy 0, policy_version 1840995 (0.0006) [2023-12-27 18:24:00,644][20347] Updated weights for policy 0, policy_version 1841005 (0.0007) [2023-12-27 18:24:00,706][20347] Updated weights for policy 0, policy_version 1841015 (0.0009) [2023-12-27 18:24:01,156][20381] Updated weights for policy 1, policy_version 1839488 (0.0008) [2023-12-27 18:24:01,208][20381] Updated weights for policy 1, policy_version 1839498 (0.0005) [2023-12-27 18:24:01,271][20381] Updated weights for policy 1, policy_version 1839508 (0.0007) [2023-12-27 18:24:01,460][20347] Updated weights for policy 0, policy_version 1841025 (0.0006) [2023-12-27 18:24:01,515][20347] Updated weights for policy 0, policy_version 1841035 (0.0008) [2023-12-27 18:24:01,572][20347] Updated weights for policy 0, policy_version 1841045 (0.0007) [2023-12-27 18:24:02,004][20381] Updated weights for policy 1, policy_version 1839518 (0.0008) [2023-12-27 18:24:02,067][20381] Updated weights for policy 1, policy_version 1839528 (0.0009) [2023-12-27 18:24:02,127][20381] Updated weights for policy 1, policy_version 1839538 (0.0008) [2023-12-27 18:24:02,280][20347] Updated weights for policy 0, policy_version 1841055 (0.0009) [2023-12-27 18:24:02,335][20347] Updated weights for policy 0, policy_version 1841065 (0.0008) [2023-12-27 18:24:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 942366720. Throughput: 0: 10242.8, 1: 10392.8. Samples: 942379416. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '889.620')] [2023-12-27 18:24:02,393][20347] Updated weights for policy 0, policy_version 1841075 (0.0007) [2023-12-27 18:24:02,850][20381] Updated weights for policy 1, policy_version 1839548 (0.0009) [2023-12-27 18:24:02,899][20381] Updated weights for policy 1, policy_version 1839558 (0.0008) [2023-12-27 18:24:02,950][20381] Updated weights for policy 1, policy_version 1839568 (0.0007) [2023-12-27 18:24:03,042][20347] Updated weights for policy 0, policy_version 1841085 (0.0006) [2023-12-27 18:24:03,100][20347] Updated weights for policy 0, policy_version 1841095 (0.0005) [2023-12-27 18:24:03,146][20347] Updated weights for policy 0, policy_version 1841105 (0.0005) [2023-12-27 18:24:03,671][20347] Updated weights for policy 0, policy_version 1841115 (0.0006) [2023-12-27 18:24:03,717][20381] Updated weights for policy 1, policy_version 1839578 (0.0007) [2023-12-27 18:24:03,734][20347] Updated weights for policy 0, policy_version 1841125 (0.0006) [2023-12-27 18:24:03,772][20381] Updated weights for policy 1, policy_version 1839588 (0.0006) [2023-12-27 18:24:03,785][20347] Updated weights for policy 0, policy_version 1841135 (0.0005) [2023-12-27 18:24:03,828][20381] Updated weights for policy 1, policy_version 1839598 (0.0005) [2023-12-27 18:24:03,889][20381] Updated weights for policy 1, policy_version 1839608 (0.0009) [2023-12-27 18:24:04,529][20347] Updated weights for policy 0, policy_version 1841145 (0.0006) [2023-12-27 18:24:04,547][20381] Updated weights for policy 1, policy_version 1839618 (0.0007) [2023-12-27 18:24:04,582][20347] Updated weights for policy 0, policy_version 1841155 (0.0008) [2023-12-27 18:24:04,600][20381] Updated weights for policy 1, policy_version 1839628 (0.0007) [2023-12-27 18:24:04,629][20347] Updated weights for policy 0, policy_version 1841165 (0.0005) [2023-12-27 18:24:04,656][20381] Updated weights for policy 1, policy_version 1839638 (0.0008) [2023-12-27 18:24:04,679][20347] Updated weights for policy 0, policy_version 1841175 (0.0005) [2023-12-27 18:24:05,273][20347] Updated weights for policy 0, policy_version 1841185 (0.0005) [2023-12-27 18:24:05,324][20347] Updated weights for policy 0, policy_version 1841195 (0.0006) [2023-12-27 18:24:05,376][20381] Updated weights for policy 1, policy_version 1839648 (0.0008) [2023-12-27 18:24:05,379][20347] Updated weights for policy 0, policy_version 1841205 (0.0005) [2023-12-27 18:24:05,431][20381] Updated weights for policy 1, policy_version 1839658 (0.0009) [2023-12-27 18:24:05,493][20381] Updated weights for policy 1, policy_version 1839668 (0.0008) [2023-12-27 18:24:05,991][20347] Updated weights for policy 0, policy_version 1841215 (0.0007) [2023-12-27 18:24:06,037][20347] Updated weights for policy 0, policy_version 1841225 (0.0008) [2023-12-27 18:24:06,083][20347] Updated weights for policy 0, policy_version 1841235 (0.0008) [2023-12-27 18:24:06,207][20381] Updated weights for policy 1, policy_version 1839678 (0.0008) [2023-12-27 18:24:06,262][20381] Updated weights for policy 1, policy_version 1839688 (0.0007) [2023-12-27 18:24:06,318][20381] Updated weights for policy 1, policy_version 1839698 (0.0007) [2023-12-27 18:24:06,847][20347] Updated weights for policy 0, policy_version 1841245 (0.0023) [2023-12-27 18:24:06,902][20347] Updated weights for policy 0, policy_version 1841255 (0.0010) [2023-12-27 18:24:06,960][20347] Updated weights for policy 0, policy_version 1841265 (0.0009) [2023-12-27 18:24:07,005][20381] Updated weights for policy 1, policy_version 1839708 (0.0007) [2023-12-27 18:24:07,052][20381] Updated weights for policy 1, policy_version 1839718 (0.0008) [2023-12-27 18:24:07,090][20292] KL-divergence is very high: 122.4157 [2023-12-27 18:24:07,095][20292] KL-divergence is very high: 125.8653 [2023-12-27 18:24:07,105][20381] Updated weights for policy 1, policy_version 1839728 (0.0009) [2023-12-27 18:24:07,107][20292] KL-divergence is very high: 125.3571 [2023-12-27 18:24:07,135][20292] KL-divergence is very high: 108.6536 [2023-12-27 18:24:07,142][20292] KL-divergence is very high: 106.1043 [2023-12-27 18:24:07,363][19170] Fps is (10 sec: 19660.3, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 942473216. Throughput: 0: 10319.3, 1: 10370.3. Samples: 942442252. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '898.850')] [2023-12-27 18:24:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001841272_471449600.pth... [2023-12-27 18:24:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001839736_471023616.pth... [2023-12-27 18:24:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001840088_471146496.pth [2023-12-27 18:24:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001838520_470712320.pth [2023-12-27 18:24:07,546][20347] Updated weights for policy 0, policy_version 1841275 (0.0006) [2023-12-27 18:24:07,614][20347] Updated weights for policy 0, policy_version 1841285 (0.0008) [2023-12-27 18:24:07,669][20347] Updated weights for policy 0, policy_version 1841295 (0.0010) [2023-12-27 18:24:07,824][20381] Updated weights for policy 1, policy_version 1839738 (0.0008) [2023-12-27 18:24:07,880][20381] Updated weights for policy 1, policy_version 1839748 (0.0009) [2023-12-27 18:24:07,942][20381] Updated weights for policy 1, policy_version 1839758 (0.0007) [2023-12-27 18:24:08,004][20381] Updated weights for policy 1, policy_version 1839768 (0.0007) [2023-12-27 18:24:08,299][20347] Updated weights for policy 0, policy_version 1841305 (0.0008) [2023-12-27 18:24:08,360][20347] Updated weights for policy 0, policy_version 1841315 (0.0010) [2023-12-27 18:24:08,416][20347] Updated weights for policy 0, policy_version 1841325 (0.0009) [2023-12-27 18:24:08,469][20347] Updated weights for policy 0, policy_version 1841335 (0.0008) [2023-12-27 18:24:08,707][20381] Updated weights for policy 1, policy_version 1839778 (0.0007) [2023-12-27 18:24:08,763][20381] Updated weights for policy 1, policy_version 1839788 (0.0008) [2023-12-27 18:24:08,818][20381] Updated weights for policy 1, policy_version 1839798 (0.0007) [2023-12-27 18:24:09,170][20347] Updated weights for policy 0, policy_version 1841345 (0.0010) [2023-12-27 18:24:09,231][20347] Updated weights for policy 0, policy_version 1841355 (0.0011) [2023-12-27 18:24:09,290][20347] Updated weights for policy 0, policy_version 1841365 (0.0009) [2023-12-27 18:24:09,506][20381] Updated weights for policy 1, policy_version 1839808 (0.0007) [2023-12-27 18:24:09,572][20381] Updated weights for policy 1, policy_version 1839818 (0.0008) [2023-12-27 18:24:09,626][20381] Updated weights for policy 1, policy_version 1839828 (0.0007) [2023-12-27 18:24:10,037][20347] Updated weights for policy 0, policy_version 1841375 (0.0007) [2023-12-27 18:24:10,094][20347] Updated weights for policy 0, policy_version 1841385 (0.0006) [2023-12-27 18:24:10,145][20347] Updated weights for policy 0, policy_version 1841395 (0.0006) [2023-12-27 18:24:10,401][20381] Updated weights for policy 1, policy_version 1839838 (0.0008) [2023-12-27 18:24:10,454][20381] Updated weights for policy 1, policy_version 1839848 (0.0008) [2023-12-27 18:24:10,504][20381] Updated weights for policy 1, policy_version 1839858 (0.0008) [2023-12-27 18:24:10,804][20347] Updated weights for policy 0, policy_version 1841405 (0.0007) [2023-12-27 18:24:10,859][20347] Updated weights for policy 0, policy_version 1841415 (0.0007) [2023-12-27 18:24:10,922][20347] Updated weights for policy 0, policy_version 1841425 (0.0009) [2023-12-27 18:24:11,210][20381] Updated weights for policy 1, policy_version 1839868 (0.0007) [2023-12-27 18:24:11,271][20381] Updated weights for policy 1, policy_version 1839878 (0.0008) [2023-12-27 18:24:11,335][20381] Updated weights for policy 1, policy_version 1839888 (0.0008) [2023-12-27 18:24:11,698][20347] Updated weights for policy 0, policy_version 1841435 (0.0008) [2023-12-27 18:24:11,777][20347] Updated weights for policy 0, policy_version 1841445 (0.0009) [2023-12-27 18:24:11,841][20347] Updated weights for policy 0, policy_version 1841455 (0.0007) [2023-12-27 18:24:12,020][20381] Updated weights for policy 1, policy_version 1839898 (0.0008) [2023-12-27 18:24:12,069][20381] Updated weights for policy 1, policy_version 1839908 (0.0007) [2023-12-27 18:24:12,125][20381] Updated weights for policy 1, policy_version 1839918 (0.0008) [2023-12-27 18:24:12,187][20381] Updated weights for policy 1, policy_version 1839928 (0.0008) [2023-12-27 18:24:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 942571520. Throughput: 0: 10310.4, 1: 10353.5. Samples: 942562276. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '928.060')] [2023-12-27 18:24:12,520][20347] Updated weights for policy 0, policy_version 1841465 (0.0007) [2023-12-27 18:24:12,580][20347] Updated weights for policy 0, policy_version 1841475 (0.0007) [2023-12-27 18:24:12,636][20347] Updated weights for policy 0, policy_version 1841485 (0.0008) [2023-12-27 18:24:12,700][20347] Updated weights for policy 0, policy_version 1841495 (0.0008) [2023-12-27 18:24:12,931][20381] Updated weights for policy 1, policy_version 1839938 (0.0007) [2023-12-27 18:24:12,980][20381] Updated weights for policy 1, policy_version 1839948 (0.0008) [2023-12-27 18:24:13,029][20381] Updated weights for policy 1, policy_version 1839958 (0.0008) [2023-12-27 18:24:13,381][20347] Updated weights for policy 0, policy_version 1841505 (0.0010) [2023-12-27 18:24:13,432][20347] Updated weights for policy 0, policy_version 1841515 (0.0005) [2023-12-27 18:24:13,485][20347] Updated weights for policy 0, policy_version 1841525 (0.0005) [2023-12-27 18:24:13,704][20381] Updated weights for policy 1, policy_version 1839968 (0.0010) [2023-12-27 18:24:13,761][20381] Updated weights for policy 1, policy_version 1839978 (0.0010) [2023-12-27 18:24:13,818][20381] Updated weights for policy 1, policy_version 1839988 (0.0010) [2023-12-27 18:24:14,134][20347] Updated weights for policy 0, policy_version 1841535 (0.0008) [2023-12-27 18:24:14,183][20347] Updated weights for policy 0, policy_version 1841545 (0.0010) [2023-12-27 18:24:14,234][20347] Updated weights for policy 0, policy_version 1841555 (0.0010) [2023-12-27 18:24:14,447][20381] Updated weights for policy 1, policy_version 1839998 (0.0010) [2023-12-27 18:24:14,499][20381] Updated weights for policy 1, policy_version 1840008 (0.0010) [2023-12-27 18:24:14,547][20381] Updated weights for policy 1, policy_version 1840018 (0.0010) [2023-12-27 18:24:14,886][20347] Updated weights for policy 0, policy_version 1841565 (0.0008) [2023-12-27 18:24:14,950][20347] Updated weights for policy 0, policy_version 1841575 (0.0006) [2023-12-27 18:24:15,012][20347] Updated weights for policy 0, policy_version 1841585 (0.0007) [2023-12-27 18:24:15,211][20381] Updated weights for policy 1, policy_version 1840028 (0.0010) [2023-12-27 18:24:15,263][20381] Updated weights for policy 1, policy_version 1840038 (0.0010) [2023-12-27 18:24:15,316][20381] Updated weights for policy 1, policy_version 1840048 (0.0010) [2023-12-27 18:24:15,680][20347] Updated weights for policy 0, policy_version 1841595 (0.0007) [2023-12-27 18:24:15,741][20347] Updated weights for policy 0, policy_version 1841605 (0.0010) [2023-12-27 18:24:15,801][20347] Updated weights for policy 0, policy_version 1841615 (0.0010) [2023-12-27 18:24:15,934][20381] Updated weights for policy 1, policy_version 1840058 (0.0006) [2023-12-27 18:24:15,999][20381] Updated weights for policy 1, policy_version 1840068 (0.0006) [2023-12-27 18:24:16,063][20381] Updated weights for policy 1, policy_version 1840078 (0.0006) [2023-12-27 18:24:16,115][20381] Updated weights for policy 1, policy_version 1840088 (0.0007) [2023-12-27 18:24:16,466][20347] Updated weights for policy 0, policy_version 1841625 (0.0010) [2023-12-27 18:24:16,530][20347] Updated weights for policy 0, policy_version 1841635 (0.0010) [2023-12-27 18:24:16,588][20347] Updated weights for policy 0, policy_version 1841645 (0.0010) [2023-12-27 18:24:16,646][20347] Updated weights for policy 0, policy_version 1841655 (0.0006) [2023-12-27 18:24:16,751][20381] Updated weights for policy 1, policy_version 1840098 (0.0011) [2023-12-27 18:24:16,805][20381] Updated weights for policy 1, policy_version 1840108 (0.0010) [2023-12-27 18:24:16,863][20381] Updated weights for policy 1, policy_version 1840118 (0.0010) [2023-12-27 18:24:17,233][20347] Updated weights for policy 0, policy_version 1841665 (0.0005) [2023-12-27 18:24:17,288][20347] Updated weights for policy 0, policy_version 1841675 (0.0005) [2023-12-27 18:24:17,351][20347] Updated weights for policy 0, policy_version 1841685 (0.0006) [2023-12-27 18:24:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 942669824. Throughput: 0: 10268.9, 1: 10395.8. Samples: 942686960. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '899.150')] [2023-12-27 18:24:17,477][20381] Updated weights for policy 1, policy_version 1840128 (0.0009) [2023-12-27 18:24:17,533][20381] Updated weights for policy 1, policy_version 1840138 (0.0008) [2023-12-27 18:24:17,589][20381] Updated weights for policy 1, policy_version 1840148 (0.0007) [2023-12-27 18:24:17,928][20347] Updated weights for policy 0, policy_version 1841695 (0.0005) [2023-12-27 18:24:17,978][20347] Updated weights for policy 0, policy_version 1841705 (0.0005) [2023-12-27 18:24:18,031][20347] Updated weights for policy 0, policy_version 1841715 (0.0005) [2023-12-27 18:24:18,268][20381] Updated weights for policy 1, policy_version 1840158 (0.0007) [2023-12-27 18:24:18,336][20381] Updated weights for policy 1, policy_version 1840170 (0.0006) [2023-12-27 18:24:18,386][20381] Updated weights for policy 1, policy_version 1840180 (0.0007) [2023-12-27 18:24:18,625][20347] Updated weights for policy 0, policy_version 1841725 (0.0008) [2023-12-27 18:24:18,687][20347] Updated weights for policy 0, policy_version 1841735 (0.0010) [2023-12-27 18:24:18,748][20347] Updated weights for policy 0, policy_version 1841745 (0.0010) [2023-12-27 18:24:18,984][20381] Updated weights for policy 1, policy_version 1840190 (0.0005) [2023-12-27 18:24:19,039][20381] Updated weights for policy 1, policy_version 1840200 (0.0005) [2023-12-27 18:24:19,088][20381] Updated weights for policy 1, policy_version 1840210 (0.0005) [2023-12-27 18:24:19,395][20347] Updated weights for policy 0, policy_version 1841755 (0.0010) [2023-12-27 18:24:19,453][20347] Updated weights for policy 0, policy_version 1841765 (0.0009) [2023-12-27 18:24:19,516][20347] Updated weights for policy 0, policy_version 1841775 (0.0008) [2023-12-27 18:24:19,691][20381] Updated weights for policy 1, policy_version 1840220 (0.0006) [2023-12-27 18:24:19,758][20381] Updated weights for policy 1, policy_version 1840230 (0.0008) [2023-12-27 18:24:19,829][20381] Updated weights for policy 1, policy_version 1840240 (0.0008) [2023-12-27 18:24:20,183][20347] Updated weights for policy 0, policy_version 1841785 (0.0006) [2023-12-27 18:24:20,241][20347] Updated weights for policy 0, policy_version 1841795 (0.0008) [2023-12-27 18:24:20,306][20347] Updated weights for policy 0, policy_version 1841805 (0.0008) [2023-12-27 18:24:20,372][20347] Updated weights for policy 0, policy_version 1841815 (0.0008) [2023-12-27 18:24:20,435][20381] Updated weights for policy 1, policy_version 1840250 (0.0007) [2023-12-27 18:24:20,500][20381] Updated weights for policy 1, policy_version 1840260 (0.0008) [2023-12-27 18:24:20,559][20381] Updated weights for policy 1, policy_version 1840270 (0.0008) [2023-12-27 18:24:20,627][20381] Updated weights for policy 1, policy_version 1840280 (0.0007) [2023-12-27 18:24:21,064][20347] Updated weights for policy 0, policy_version 1841825 (0.0008) [2023-12-27 18:24:21,115][20347] Updated weights for policy 0, policy_version 1841835 (0.0007) [2023-12-27 18:24:21,180][20347] Updated weights for policy 0, policy_version 1841845 (0.0007) [2023-12-27 18:24:21,319][20381] Updated weights for policy 1, policy_version 1840290 (0.0008) [2023-12-27 18:24:21,386][20381] Updated weights for policy 1, policy_version 1840300 (0.0007) [2023-12-27 18:24:21,452][20381] Updated weights for policy 1, policy_version 1840310 (0.0007) [2023-12-27 18:24:21,993][20347] Updated weights for policy 0, policy_version 1841855 (0.0007) [2023-12-27 18:24:22,049][20347] Updated weights for policy 0, policy_version 1841865 (0.0006) [2023-12-27 18:24:22,084][20381] Updated weights for policy 1, policy_version 1840320 (0.0006) [2023-12-27 18:24:22,107][20347] Updated weights for policy 0, policy_version 1841875 (0.0007) [2023-12-27 18:24:22,140][20381] Updated weights for policy 1, policy_version 1840330 (0.0006) [2023-12-27 18:24:22,196][20381] Updated weights for policy 1, policy_version 1840340 (0.0008) [2023-12-27 18:24:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 942784512. Throughput: 0: 10306.6, 1: 10429.4. Samples: 942752056. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '868.880')] [2023-12-27 18:24:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001840344_471179264.pth... [2023-12-27 18:24:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001841880_471605248.pth... [2023-12-27 18:24:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001839128_470867968.pth [2023-12-27 18:24:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001840664_471293952.pth [2023-12-27 18:24:22,810][20347] Updated weights for policy 0, policy_version 1841885 (0.0006) [2023-12-27 18:24:22,869][20347] Updated weights for policy 0, policy_version 1841895 (0.0007) [2023-12-27 18:24:22,890][20381] Updated weights for policy 1, policy_version 1840350 (0.0008) [2023-12-27 18:24:22,922][20347] Updated weights for policy 0, policy_version 1841905 (0.0006) [2023-12-27 18:24:22,948][20381] Updated weights for policy 1, policy_version 1840360 (0.0008) [2023-12-27 18:24:23,001][20381] Updated weights for policy 1, policy_version 1840370 (0.0010) [2023-12-27 18:24:23,573][20347] Updated weights for policy 0, policy_version 1841915 (0.0008) [2023-12-27 18:24:23,641][20347] Updated weights for policy 0, policy_version 1841925 (0.0007) [2023-12-27 18:24:23,668][20381] Updated weights for policy 1, policy_version 1840380 (0.0008) [2023-12-27 18:24:23,699][20347] Updated weights for policy 0, policy_version 1841935 (0.0007) [2023-12-27 18:24:23,720][20381] Updated weights for policy 1, policy_version 1840390 (0.0005) [2023-12-27 18:24:23,773][20381] Updated weights for policy 1, policy_version 1840400 (0.0006) [2023-12-27 18:24:24,326][20381] Updated weights for policy 1, policy_version 1840410 (0.0005) [2023-12-27 18:24:24,339][20347] Updated weights for policy 0, policy_version 1841945 (0.0007) [2023-12-27 18:24:24,382][20381] Updated weights for policy 1, policy_version 1840420 (0.0006) [2023-12-27 18:24:24,391][20347] Updated weights for policy 0, policy_version 1841955 (0.0007) [2023-12-27 18:24:24,445][20381] Updated weights for policy 1, policy_version 1840431 (0.0012) [2023-12-27 18:24:24,451][20347] Updated weights for policy 0, policy_version 1841965 (0.0006) [2023-12-27 18:24:24,507][20347] Updated weights for policy 0, policy_version 1841975 (0.0008) [2023-12-27 18:24:25,081][20381] Updated weights for policy 1, policy_version 1840441 (0.0008) [2023-12-27 18:24:25,142][20381] Updated weights for policy 1, policy_version 1840451 (0.0008) [2023-12-27 18:24:25,195][20381] Updated weights for policy 1, policy_version 1840461 (0.0006) [2023-12-27 18:24:25,212][20347] Updated weights for policy 0, policy_version 1841985 (0.0007) [2023-12-27 18:24:25,251][20381] Updated weights for policy 1, policy_version 1840471 (0.0005) [2023-12-27 18:24:25,267][20347] Updated weights for policy 0, policy_version 1841995 (0.0008) [2023-12-27 18:24:25,318][20347] Updated weights for policy 0, policy_version 1842005 (0.0009) [2023-12-27 18:24:25,872][20381] Updated weights for policy 1, policy_version 1840481 (0.0010) [2023-12-27 18:24:25,896][20347] Updated weights for policy 0, policy_version 1842015 (0.0006) [2023-12-27 18:24:25,917][20381] Updated weights for policy 1, policy_version 1840491 (0.0010) [2023-12-27 18:24:25,948][20347] Updated weights for policy 0, policy_version 1842025 (0.0005) [2023-12-27 18:24:25,961][20381] Updated weights for policy 1, policy_version 1840501 (0.0010) [2023-12-27 18:24:26,003][20347] Updated weights for policy 0, policy_version 1842035 (0.0009) [2023-12-27 18:24:26,653][20381] Updated weights for policy 1, policy_version 1840511 (0.0009) [2023-12-27 18:24:26,675][20347] Updated weights for policy 0, policy_version 1842045 (0.0009) [2023-12-27 18:24:26,703][20381] Updated weights for policy 1, policy_version 1840521 (0.0010) [2023-12-27 18:24:26,726][20347] Updated weights for policy 0, policy_version 1842055 (0.0005) [2023-12-27 18:24:26,751][20381] Updated weights for policy 1, policy_version 1840531 (0.0010) [2023-12-27 18:24:26,787][20347] Updated weights for policy 0, policy_version 1842065 (0.0006) [2023-12-27 18:24:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 942882816. Throughput: 0: 10302.8, 1: 10390.8. Samples: 942875804. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '854.660')] [2023-12-27 18:24:27,405][20347] Updated weights for policy 0, policy_version 1842075 (0.0006) [2023-12-27 18:24:27,416][20381] Updated weights for policy 1, policy_version 1840541 (0.0009) [2023-12-27 18:24:27,461][20347] Updated weights for policy 0, policy_version 1842085 (0.0005) [2023-12-27 18:24:27,464][20381] Updated weights for policy 1, policy_version 1840551 (0.0010) [2023-12-27 18:24:27,516][20381] Updated weights for policy 1, policy_version 1840561 (0.0010) [2023-12-27 18:24:27,517][20347] Updated weights for policy 0, policy_version 1842095 (0.0005) [2023-12-27 18:24:28,086][20347] Updated weights for policy 0, policy_version 1842105 (0.0005) [2023-12-27 18:24:28,150][20347] Updated weights for policy 0, policy_version 1842115 (0.0007) [2023-12-27 18:24:28,204][20381] Updated weights for policy 1, policy_version 1840571 (0.0010) [2023-12-27 18:24:28,214][20347] Updated weights for policy 0, policy_version 1842125 (0.0006) [2023-12-27 18:24:28,258][20381] Updated weights for policy 1, policy_version 1840581 (0.0010) [2023-12-27 18:24:28,269][20347] Updated weights for policy 0, policy_version 1842135 (0.0007) [2023-12-27 18:24:28,275][20292] KL-divergence is very high: 128.2466 [2023-12-27 18:24:28,286][20292] KL-divergence is very high: 125.5155 [2023-12-27 18:24:28,298][20292] KL-divergence is very high: 141.1273 [2023-12-27 18:24:28,304][20292] KL-divergence is very high: 155.6532 [2023-12-27 18:24:28,323][20381] Updated weights for policy 1, policy_version 1840591 (0.0010) [2023-12-27 18:24:28,332][20292] KL-divergence is very high: 123.8513 [2023-12-27 18:24:28,971][20347] Updated weights for policy 0, policy_version 1842145 (0.0008) [2023-12-27 18:24:29,005][20381] Updated weights for policy 1, policy_version 1840601 (0.0009) [2023-12-27 18:24:29,022][20347] Updated weights for policy 0, policy_version 1842155 (0.0007) [2023-12-27 18:24:29,054][20381] Updated weights for policy 1, policy_version 1840611 (0.0005) [2023-12-27 18:24:29,075][20347] Updated weights for policy 0, policy_version 1842165 (0.0007) [2023-12-27 18:24:29,105][20381] Updated weights for policy 1, policy_version 1840621 (0.0005) [2023-12-27 18:24:29,160][20381] Updated weights for policy 1, policy_version 1840631 (0.0005) [2023-12-27 18:24:29,743][20347] Updated weights for policy 0, policy_version 1842175 (0.0007) [2023-12-27 18:24:29,800][20347] Updated weights for policy 0, policy_version 1842185 (0.0006) [2023-12-27 18:24:29,845][20381] Updated weights for policy 1, policy_version 1840641 (0.0008) [2023-12-27 18:24:29,864][20347] Updated weights for policy 0, policy_version 1842195 (0.0007) [2023-12-27 18:24:29,909][20381] Updated weights for policy 1, policy_version 1840651 (0.0006) [2023-12-27 18:24:29,977][20381] Updated weights for policy 1, policy_version 1840661 (0.0008) [2023-12-27 18:24:30,519][20347] Updated weights for policy 0, policy_version 1842205 (0.0007) [2023-12-27 18:24:30,571][20347] Updated weights for policy 0, policy_version 1842215 (0.0008) [2023-12-27 18:24:30,623][20347] Updated weights for policy 0, policy_version 1842225 (0.0007) [2023-12-27 18:24:30,667][20381] Updated weights for policy 1, policy_version 1840671 (0.0006) [2023-12-27 18:24:30,711][20381] Updated weights for policy 1, policy_version 1840681 (0.0006) [2023-12-27 18:24:30,756][20381] Updated weights for policy 1, policy_version 1840691 (0.0005) [2023-12-27 18:24:31,330][20347] Updated weights for policy 0, policy_version 1842235 (0.0007) [2023-12-27 18:24:31,387][20381] Updated weights for policy 1, policy_version 1840701 (0.0007) [2023-12-27 18:24:31,401][20347] Updated weights for policy 0, policy_version 1842245 (0.0008) [2023-12-27 18:24:31,451][20381] Updated weights for policy 1, policy_version 1840711 (0.0008) [2023-12-27 18:24:31,468][20347] Updated weights for policy 0, policy_version 1842255 (0.0008) [2023-12-27 18:24:31,510][20381] Updated weights for policy 1, policy_version 1840721 (0.0007) [2023-12-27 18:24:32,166][20381] Updated weights for policy 1, policy_version 1840731 (0.0007) [2023-12-27 18:24:32,195][20347] Updated weights for policy 0, policy_version 1842265 (0.0008) [2023-12-27 18:24:32,215][20381] Updated weights for policy 1, policy_version 1840741 (0.0009) [2023-12-27 18:24:32,253][20347] Updated weights for policy 0, policy_version 1842275 (0.0005) [2023-12-27 18:24:32,270][20381] Updated weights for policy 1, policy_version 1840751 (0.0006) [2023-12-27 18:24:32,310][20347] Updated weights for policy 0, policy_version 1842285 (0.0007) [2023-12-27 18:24:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20521.7). Total num frames: 942989312. Throughput: 0: 10312.8, 1: 10337.6. Samples: 942999596. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.210')] [2023-12-27 18:24:32,386][20347] Updated weights for policy 0, policy_version 1842295 (0.0008) [2023-12-27 18:24:32,948][20381] Updated weights for policy 1, policy_version 1840761 (0.0007) [2023-12-27 18:24:32,963][20347] Updated weights for policy 0, policy_version 1842305 (0.0008) [2023-12-27 18:24:32,994][20381] Updated weights for policy 1, policy_version 1840771 (0.0006) [2023-12-27 18:24:33,018][20347] Updated weights for policy 0, policy_version 1842315 (0.0005) [2023-12-27 18:24:33,037][20381] Updated weights for policy 1, policy_version 1840781 (0.0006) [2023-12-27 18:24:33,070][20347] Updated weights for policy 0, policy_version 1842325 (0.0006) [2023-12-27 18:24:33,090][20381] Updated weights for policy 1, policy_version 1840791 (0.0006) [2023-12-27 18:24:33,673][20381] Updated weights for policy 1, policy_version 1840801 (0.0007) [2023-12-27 18:24:33,700][20347] Updated weights for policy 0, policy_version 1842335 (0.0006) [2023-12-27 18:24:33,721][20381] Updated weights for policy 1, policy_version 1840811 (0.0006) [2023-12-27 18:24:33,752][20347] Updated weights for policy 0, policy_version 1842345 (0.0006) [2023-12-27 18:24:33,776][20381] Updated weights for policy 1, policy_version 1840821 (0.0005) [2023-12-27 18:24:33,809][20347] Updated weights for policy 0, policy_version 1842355 (0.0006) [2023-12-27 18:24:34,468][20347] Updated weights for policy 0, policy_version 1842365 (0.0008) [2023-12-27 18:24:34,474][20381] Updated weights for policy 1, policy_version 1840831 (0.0006) [2023-12-27 18:24:34,521][20347] Updated weights for policy 0, policy_version 1842375 (0.0010) [2023-12-27 18:24:34,534][20381] Updated weights for policy 1, policy_version 1840841 (0.0006) [2023-12-27 18:24:34,585][20347] Updated weights for policy 0, policy_version 1842385 (0.0011) [2023-12-27 18:24:34,591][20381] Updated weights for policy 1, policy_version 1840851 (0.0006) [2023-12-27 18:24:35,187][20347] Updated weights for policy 0, policy_version 1842395 (0.0009) [2023-12-27 18:24:35,243][20347] Updated weights for policy 0, policy_version 1842405 (0.0007) [2023-12-27 18:24:35,251][20381] Updated weights for policy 1, policy_version 1840861 (0.0006) [2023-12-27 18:24:35,288][20347] Updated weights for policy 0, policy_version 1842415 (0.0010) [2023-12-27 18:24:35,312][20381] Updated weights for policy 1, policy_version 1840871 (0.0007) [2023-12-27 18:24:35,368][20381] Updated weights for policy 1, policy_version 1840881 (0.0008) [2023-12-27 18:24:35,911][20347] Updated weights for policy 0, policy_version 1842425 (0.0010) [2023-12-27 18:24:35,967][20347] Updated weights for policy 0, policy_version 1842435 (0.0005) [2023-12-27 18:24:36,012][20347] Updated weights for policy 0, policy_version 1842445 (0.0005) [2023-12-27 18:24:36,023][20381] Updated weights for policy 1, policy_version 1840891 (0.0006) [2023-12-27 18:24:36,064][20347] Updated weights for policy 0, policy_version 1842455 (0.0005) [2023-12-27 18:24:36,084][20381] Updated weights for policy 1, policy_version 1840901 (0.0005) [2023-12-27 18:24:36,157][20381] Updated weights for policy 1, policy_version 1840911 (0.0007) [2023-12-27 18:24:36,656][20347] Updated weights for policy 0, policy_version 1842465 (0.0010) [2023-12-27 18:24:36,709][20347] Updated weights for policy 0, policy_version 1842475 (0.0010) [2023-12-27 18:24:36,764][20347] Updated weights for policy 0, policy_version 1842485 (0.0010) [2023-12-27 18:24:36,786][20381] Updated weights for policy 1, policy_version 1840921 (0.0006) [2023-12-27 18:24:36,842][20381] Updated weights for policy 1, policy_version 1840931 (0.0008) [2023-12-27 18:24:36,901][20381] Updated weights for policy 1, policy_version 1840941 (0.0008) [2023-12-27 18:24:36,959][20381] Updated weights for policy 1, policy_version 1840951 (0.0010) [2023-12-27 18:24:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20753.0, 300 sec: 20549.4). Total num frames: 943095808. Throughput: 0: 10406.9, 1: 10360.5. Samples: 943065324. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '926.760')] [2023-12-27 18:24:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001840952_471334912.pth... [2023-12-27 18:24:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001842488_471760896.pth... [2023-12-27 18:24:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001839736_471023616.pth [2023-12-27 18:24:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001841272_471449600.pth [2023-12-27 18:24:37,454][20347] Updated weights for policy 0, policy_version 1842495 (0.0010) [2023-12-27 18:24:37,502][20347] Updated weights for policy 0, policy_version 1842505 (0.0010) [2023-12-27 18:24:37,560][20347] Updated weights for policy 0, policy_version 1842515 (0.0010) [2023-12-27 18:24:37,643][20381] Updated weights for policy 1, policy_version 1840961 (0.0010) [2023-12-27 18:24:37,700][20381] Updated weights for policy 1, policy_version 1840971 (0.0011) [2023-12-27 18:24:37,752][20381] Updated weights for policy 1, policy_version 1840981 (0.0010) [2023-12-27 18:24:38,246][20347] Updated weights for policy 0, policy_version 1842525 (0.0010) [2023-12-27 18:24:38,301][20347] Updated weights for policy 0, policy_version 1842535 (0.0010) [2023-12-27 18:24:38,363][20347] Updated weights for policy 0, policy_version 1842545 (0.0009) [2023-12-27 18:24:38,457][20381] Updated weights for policy 1, policy_version 1840991 (0.0007) [2023-12-27 18:24:38,511][20381] Updated weights for policy 1, policy_version 1841001 (0.0006) [2023-12-27 18:24:38,569][20381] Updated weights for policy 1, policy_version 1841011 (0.0008) [2023-12-27 18:24:39,027][20347] Updated weights for policy 0, policy_version 1842555 (0.0007) [2023-12-27 18:24:39,083][20347] Updated weights for policy 0, policy_version 1842565 (0.0008) [2023-12-27 18:24:39,148][20347] Updated weights for policy 0, policy_version 1842575 (0.0008) [2023-12-27 18:24:39,276][20381] Updated weights for policy 1, policy_version 1841021 (0.0008) [2023-12-27 18:24:39,336][20381] Updated weights for policy 1, policy_version 1841031 (0.0007) [2023-12-27 18:24:39,409][20381] Updated weights for policy 1, policy_version 1841041 (0.0009) [2023-12-27 18:24:39,920][20347] Updated weights for policy 0, policy_version 1842585 (0.0009) [2023-12-27 18:24:39,974][20347] Updated weights for policy 0, policy_version 1842595 (0.0008) [2023-12-27 18:24:40,037][20347] Updated weights for policy 0, policy_version 1842605 (0.0009) [2023-12-27 18:24:40,089][20381] Updated weights for policy 1, policy_version 1841051 (0.0007) [2023-12-27 18:24:40,102][20347] Updated weights for policy 0, policy_version 1842615 (0.0007) [2023-12-27 18:24:40,146][20381] Updated weights for policy 1, policy_version 1841061 (0.0008) [2023-12-27 18:24:40,199][20381] Updated weights for policy 1, policy_version 1841071 (0.0008) [2023-12-27 18:24:40,801][20347] Updated weights for policy 0, policy_version 1842625 (0.0008) [2023-12-27 18:24:40,867][20347] Updated weights for policy 0, policy_version 1842635 (0.0008) [2023-12-27 18:24:40,906][20381] Updated weights for policy 1, policy_version 1841081 (0.0008) [2023-12-27 18:24:40,931][20347] Updated weights for policy 0, policy_version 1842645 (0.0006) [2023-12-27 18:24:40,959][20381] Updated weights for policy 1, policy_version 1841091 (0.0008) [2023-12-27 18:24:41,023][20381] Updated weights for policy 1, policy_version 1841101 (0.0008) [2023-12-27 18:24:41,080][20381] Updated weights for policy 1, policy_version 1841111 (0.0009) [2023-12-27 18:24:41,643][20347] Updated weights for policy 0, policy_version 1842655 (0.0007) [2023-12-27 18:24:41,709][20347] Updated weights for policy 0, policy_version 1842665 (0.0009) [2023-12-27 18:24:41,779][20347] Updated weights for policy 0, policy_version 1842675 (0.0011) [2023-12-27 18:24:41,848][20381] Updated weights for policy 1, policy_version 1841121 (0.0007) [2023-12-27 18:24:41,908][20381] Updated weights for policy 1, policy_version 1841131 (0.0007) [2023-12-27 18:24:41,964][20381] Updated weights for policy 1, policy_version 1841141 (0.0008) [2023-12-27 18:24:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 943194112. Throughput: 0: 10328.1, 1: 10267.8. Samples: 943187252. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '917.530')] [2023-12-27 18:24:42,470][20347] Updated weights for policy 0, policy_version 1842685 (0.0011) [2023-12-27 18:24:42,534][20347] Updated weights for policy 0, policy_version 1842695 (0.0009) [2023-12-27 18:24:42,596][20347] Updated weights for policy 0, policy_version 1842705 (0.0007) [2023-12-27 18:24:42,660][20381] Updated weights for policy 1, policy_version 1841151 (0.0007) [2023-12-27 18:24:42,723][20381] Updated weights for policy 1, policy_version 1841161 (0.0009) [2023-12-27 18:24:42,773][20381] Updated weights for policy 1, policy_version 1841171 (0.0011) [2023-12-27 18:24:43,265][20347] Updated weights for policy 0, policy_version 1842715 (0.0010) [2023-12-27 18:24:43,319][20347] Updated weights for policy 0, policy_version 1842725 (0.0010) [2023-12-27 18:24:43,379][20347] Updated weights for policy 0, policy_version 1842735 (0.0010) [2023-12-27 18:24:43,427][20381] Updated weights for policy 1, policy_version 1841181 (0.0008) [2023-12-27 18:24:43,486][20381] Updated weights for policy 1, policy_version 1841191 (0.0007) [2023-12-27 18:24:43,529][20381] Updated weights for policy 1, policy_version 1841201 (0.0005) [2023-12-27 18:24:44,048][20347] Updated weights for policy 0, policy_version 1842745 (0.0010) [2023-12-27 18:24:44,072][20381] Updated weights for policy 1, policy_version 1841211 (0.0005) [2023-12-27 18:24:44,104][20347] Updated weights for policy 0, policy_version 1842755 (0.0010) [2023-12-27 18:24:44,128][20381] Updated weights for policy 1, policy_version 1841221 (0.0006) [2023-12-27 18:24:44,158][20347] Updated weights for policy 0, policy_version 1842765 (0.0010) [2023-12-27 18:24:44,185][20381] Updated weights for policy 1, policy_version 1841231 (0.0005) [2023-12-27 18:24:44,216][20347] Updated weights for policy 0, policy_version 1842775 (0.0010) [2023-12-27 18:24:44,878][20381] Updated weights for policy 1, policy_version 1841241 (0.0008) [2023-12-27 18:24:44,894][20347] Updated weights for policy 0, policy_version 1842785 (0.0010) [2023-12-27 18:24:44,939][20381] Updated weights for policy 1, policy_version 1841251 (0.0010) [2023-12-27 18:24:44,954][20347] Updated weights for policy 0, policy_version 1842795 (0.0010) [2023-12-27 18:24:45,003][20381] Updated weights for policy 1, policy_version 1841261 (0.0007) [2023-12-27 18:24:45,013][20347] Updated weights for policy 0, policy_version 1842805 (0.0008) [2023-12-27 18:24:45,073][20381] Updated weights for policy 1, policy_version 1841271 (0.0007) [2023-12-27 18:24:45,702][20347] Updated weights for policy 0, policy_version 1842815 (0.0010) [2023-12-27 18:24:45,726][20381] Updated weights for policy 1, policy_version 1841281 (0.0005) [2023-12-27 18:24:45,750][20347] Updated weights for policy 0, policy_version 1842825 (0.0010) [2023-12-27 18:24:45,785][20381] Updated weights for policy 1, policy_version 1841291 (0.0006) [2023-12-27 18:24:45,805][20347] Updated weights for policy 0, policy_version 1842835 (0.0010) [2023-12-27 18:24:45,841][20381] Updated weights for policy 1, policy_version 1841301 (0.0006) [2023-12-27 18:24:46,495][20347] Updated weights for policy 0, policy_version 1842845 (0.0010) [2023-12-27 18:24:46,537][20381] Updated weights for policy 1, policy_version 1841311 (0.0006) [2023-12-27 18:24:46,549][20347] Updated weights for policy 0, policy_version 1842855 (0.0010) [2023-12-27 18:24:46,594][20381] Updated weights for policy 1, policy_version 1841321 (0.0006) [2023-12-27 18:24:46,619][20347] Updated weights for policy 0, policy_version 1842865 (0.0010) [2023-12-27 18:24:46,645][20381] Updated weights for policy 1, policy_version 1841331 (0.0006) [2023-12-27 18:24:47,213][20347] Updated weights for policy 0, policy_version 1842875 (0.0009) [2023-12-27 18:24:47,229][20381] Updated weights for policy 1, policy_version 1841341 (0.0005) [2023-12-27 18:24:47,269][20347] Updated weights for policy 0, policy_version 1842885 (0.0006) [2023-12-27 18:24:47,286][20381] Updated weights for policy 1, policy_version 1841351 (0.0005) [2023-12-27 18:24:47,327][20347] Updated weights for policy 0, policy_version 1842895 (0.0008) [2023-12-27 18:24:47,343][20381] Updated weights for policy 1, policy_version 1841361 (0.0010) [2023-12-27 18:24:47,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 943292416. Throughput: 0: 10364.5, 1: 10327.7. Samples: 943310560. Policy #0 lag: (min: 31.0, avg: 33.5, max: 63.0) [2023-12-27 18:24:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '918.970')] [2023-12-27 18:24:47,977][20347] Updated weights for policy 0, policy_version 1842905 (0.0010) [2023-12-27 18:24:47,996][20381] Updated weights for policy 1, policy_version 1841371 (0.0010) [2023-12-27 18:24:48,031][20347] Updated weights for policy 0, policy_version 1842915 (0.0006) [2023-12-27 18:24:48,050][20381] Updated weights for policy 1, policy_version 1841381 (0.0011) [2023-12-27 18:24:48,080][20347] Updated weights for policy 0, policy_version 1842925 (0.0006) [2023-12-27 18:24:48,109][20381] Updated weights for policy 1, policy_version 1841391 (0.0007) [2023-12-27 18:24:48,137][20347] Updated weights for policy 0, policy_version 1842935 (0.0005) [2023-12-27 18:24:48,770][20381] Updated weights for policy 1, policy_version 1841401 (0.0006) [2023-12-27 18:24:48,824][20381] Updated weights for policy 1, policy_version 1841411 (0.0006) [2023-12-27 18:24:48,860][20347] Updated weights for policy 0, policy_version 1842945 (0.0006) [2023-12-27 18:24:48,891][20381] Updated weights for policy 1, policy_version 1841421 (0.0006) [2023-12-27 18:24:48,915][20347] Updated weights for policy 0, policy_version 1842955 (0.0006) [2023-12-27 18:24:48,941][20381] Updated weights for policy 1, policy_version 1841431 (0.0006) [2023-12-27 18:24:48,976][20347] Updated weights for policy 0, policy_version 1842965 (0.0005) [2023-12-27 18:24:49,574][20381] Updated weights for policy 1, policy_version 1841441 (0.0006) [2023-12-27 18:24:49,636][20381] Updated weights for policy 1, policy_version 1841451 (0.0007) [2023-12-27 18:24:49,683][20347] Updated weights for policy 0, policy_version 1842975 (0.0008) [2023-12-27 18:24:49,701][20381] Updated weights for policy 1, policy_version 1841461 (0.0006) [2023-12-27 18:24:49,744][20347] Updated weights for policy 0, policy_version 1842985 (0.0007) [2023-12-27 18:24:49,810][20347] Updated weights for policy 0, policy_version 1842995 (0.0007) [2023-12-27 18:24:50,335][20381] Updated weights for policy 1, policy_version 1841471 (0.0008) [2023-12-27 18:24:50,392][20381] Updated weights for policy 1, policy_version 1841481 (0.0008) [2023-12-27 18:24:50,447][20381] Updated weights for policy 1, policy_version 1841491 (0.0008) [2023-12-27 18:24:50,568][20347] Updated weights for policy 0, policy_version 1843005 (0.0007) [2023-12-27 18:24:50,627][20347] Updated weights for policy 0, policy_version 1843015 (0.0008) [2023-12-27 18:24:50,682][20347] Updated weights for policy 0, policy_version 1843025 (0.0007) [2023-12-27 18:24:51,175][20381] Updated weights for policy 1, policy_version 1841501 (0.0006) [2023-12-27 18:24:51,239][20381] Updated weights for policy 1, policy_version 1841511 (0.0008) [2023-12-27 18:24:51,303][20381] Updated weights for policy 1, policy_version 1841521 (0.0008) [2023-12-27 18:24:51,411][20347] Updated weights for policy 0, policy_version 1843035 (0.0007) [2023-12-27 18:24:51,470][20347] Updated weights for policy 0, policy_version 1843045 (0.0005) [2023-12-27 18:24:51,523][20347] Updated weights for policy 0, policy_version 1843055 (0.0005) [2023-12-27 18:24:52,013][20381] Updated weights for policy 1, policy_version 1841531 (0.0013) [2023-12-27 18:24:52,077][20381] Updated weights for policy 1, policy_version 1841541 (0.0005) [2023-12-27 18:24:52,139][20381] Updated weights for policy 1, policy_version 1841551 (0.0007) [2023-12-27 18:24:52,154][20347] Updated weights for policy 0, policy_version 1843065 (0.0006) [2023-12-27 18:24:52,214][20347] Updated weights for policy 0, policy_version 1843075 (0.0008) [2023-12-27 18:24:52,276][20347] Updated weights for policy 0, policy_version 1843085 (0.0008) [2023-12-27 18:24:52,327][20347] Updated weights for policy 0, policy_version 1843095 (0.0008) [2023-12-27 18:24:52,370][19170] Fps is (10 sec: 21283.1, 60 sec: 20750.5, 300 sec: 20521.1). Total num frames: 943407104. Throughput: 0: 10304.9, 1: 10370.2. Samples: 943372784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:24:52,371][19170] Avg episode reward: [(0, '1000.000'), (1, '900.670')] [2023-12-27 18:24:52,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001843096_471916544.pth... [2023-12-27 18:24:52,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001841560_471490560.pth... [2023-12-27 18:24:52,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001840344_471179264.pth [2023-12-27 18:24:52,382][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001841880_471605248.pth [2023-12-27 18:24:52,761][20381] Updated weights for policy 1, policy_version 1841561 (0.0008) [2023-12-27 18:24:52,822][20381] Updated weights for policy 1, policy_version 1841571 (0.0007) [2023-12-27 18:24:52,882][20381] Updated weights for policy 1, policy_version 1841581 (0.0007) [2023-12-27 18:24:52,941][20381] Updated weights for policy 1, policy_version 1841591 (0.0007) [2023-12-27 18:24:53,025][20347] Updated weights for policy 0, policy_version 1843105 (0.0010) [2023-12-27 18:24:53,083][20347] Updated weights for policy 0, policy_version 1843115 (0.0006) [2023-12-27 18:24:53,136][20347] Updated weights for policy 0, policy_version 1843125 (0.0007) [2023-12-27 18:24:53,536][20381] Updated weights for policy 1, policy_version 1841601 (0.0005) [2023-12-27 18:24:53,591][20381] Updated weights for policy 1, policy_version 1841611 (0.0005) [2023-12-27 18:24:53,645][20381] Updated weights for policy 1, policy_version 1841621 (0.0006) [2023-12-27 18:24:53,730][20347] Updated weights for policy 0, policy_version 1843135 (0.0007) [2023-12-27 18:24:53,786][20347] Updated weights for policy 0, policy_version 1843145 (0.0005) [2023-12-27 18:24:53,842][20347] Updated weights for policy 0, policy_version 1843155 (0.0005) [2023-12-27 18:24:54,225][20381] Updated weights for policy 1, policy_version 1841631 (0.0007) [2023-12-27 18:24:54,285][20381] Updated weights for policy 1, policy_version 1841641 (0.0005) [2023-12-27 18:24:54,341][20381] Updated weights for policy 1, policy_version 1841651 (0.0005) [2023-12-27 18:24:54,436][20347] Updated weights for policy 0, policy_version 1843165 (0.0008) [2023-12-27 18:24:54,481][20347] Updated weights for policy 0, policy_version 1843175 (0.0010) [2023-12-27 18:24:54,530][20347] Updated weights for policy 0, policy_version 1843185 (0.0010) [2023-12-27 18:24:54,886][20381] Updated weights for policy 1, policy_version 1841661 (0.0005) [2023-12-27 18:24:54,948][20381] Updated weights for policy 1, policy_version 1841671 (0.0007) [2023-12-27 18:24:55,011][20381] Updated weights for policy 1, policy_version 1841681 (0.0007) [2023-12-27 18:24:55,275][20347] Updated weights for policy 0, policy_version 1843195 (0.0010) [2023-12-27 18:24:55,334][20347] Updated weights for policy 0, policy_version 1843205 (0.0010) [2023-12-27 18:24:55,385][20347] Updated weights for policy 0, policy_version 1843215 (0.0007) [2023-12-27 18:24:55,701][20381] Updated weights for policy 1, policy_version 1841691 (0.0007) [2023-12-27 18:24:55,754][20381] Updated weights for policy 1, policy_version 1841701 (0.0007) [2023-12-27 18:24:55,819][20381] Updated weights for policy 1, policy_version 1841711 (0.0007) [2023-12-27 18:24:56,057][20347] Updated weights for policy 0, policy_version 1843225 (0.0007) [2023-12-27 18:24:56,113][20347] Updated weights for policy 0, policy_version 1843235 (0.0005) [2023-12-27 18:24:56,163][20347] Updated weights for policy 0, policy_version 1843245 (0.0009) [2023-12-27 18:24:56,213][20347] Updated weights for policy 0, policy_version 1843255 (0.0009) [2023-12-27 18:24:56,420][20381] Updated weights for policy 1, policy_version 1841721 (0.0007) [2023-12-27 18:24:56,480][20381] Updated weights for policy 1, policy_version 1841731 (0.0010) [2023-12-27 18:24:56,536][20381] Updated weights for policy 1, policy_version 1841741 (0.0010) [2023-12-27 18:24:56,586][20381] Updated weights for policy 1, policy_version 1841751 (0.0009) [2023-12-27 18:24:56,869][20347] Updated weights for policy 0, policy_version 1843265 (0.0006) [2023-12-27 18:24:56,928][20347] Updated weights for policy 0, policy_version 1843275 (0.0008) [2023-12-27 18:24:56,975][20347] Updated weights for policy 0, policy_version 1843285 (0.0008) [2023-12-27 18:24:57,146][20381] Updated weights for policy 1, policy_version 1841761 (0.0007) [2023-12-27 18:24:57,193][20381] Updated weights for policy 1, policy_version 1841771 (0.0007) [2023-12-27 18:24:57,251][20381] Updated weights for policy 1, policy_version 1841781 (0.0008) [2023-12-27 18:24:57,362][19170] Fps is (10 sec: 22118.4, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 943513600. Throughput: 0: 10309.1, 1: 10493.5. Samples: 943498392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:24:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.930')] [2023-12-27 18:24:57,624][20347] Updated weights for policy 0, policy_version 1843295 (0.0006) [2023-12-27 18:24:57,674][20347] Updated weights for policy 0, policy_version 1843305 (0.0005) [2023-12-27 18:24:57,722][20347] Updated weights for policy 0, policy_version 1843315 (0.0005) [2023-12-27 18:24:57,902][20381] Updated weights for policy 1, policy_version 1841791 (0.0007) [2023-12-27 18:24:57,948][20381] Updated weights for policy 1, policy_version 1841801 (0.0007) [2023-12-27 18:24:57,992][20381] Updated weights for policy 1, policy_version 1841811 (0.0008) [2023-12-27 18:24:58,343][20347] Updated weights for policy 0, policy_version 1843325 (0.0008) [2023-12-27 18:24:58,417][20347] Updated weights for policy 0, policy_version 1843335 (0.0009) [2023-12-27 18:24:58,488][20347] Updated weights for policy 0, policy_version 1843345 (0.0008) [2023-12-27 18:24:58,756][20381] Updated weights for policy 1, policy_version 1841821 (0.0007) [2023-12-27 18:24:58,818][20381] Updated weights for policy 1, policy_version 1841831 (0.0009) [2023-12-27 18:24:58,886][20381] Updated weights for policy 1, policy_version 1841841 (0.0009) [2023-12-27 18:24:59,365][20347] Updated weights for policy 0, policy_version 1843355 (0.0009) [2023-12-27 18:24:59,434][20347] Updated weights for policy 0, policy_version 1843365 (0.0010) [2023-12-27 18:24:59,483][20347] Updated weights for policy 0, policy_version 1843375 (0.0007) [2023-12-27 18:24:59,764][20381] Updated weights for policy 1, policy_version 1841851 (0.0008) [2023-12-27 18:24:59,817][20381] Updated weights for policy 1, policy_version 1841861 (0.0006) [2023-12-27 18:24:59,881][20381] Updated weights for policy 1, policy_version 1841871 (0.0006) [2023-12-27 18:25:00,185][20347] Updated weights for policy 0, policy_version 1843385 (0.0007) [2023-12-27 18:25:00,243][20347] Updated weights for policy 0, policy_version 1843395 (0.0007) [2023-12-27 18:25:00,293][20347] Updated weights for policy 0, policy_version 1843405 (0.0009) [2023-12-27 18:25:00,350][20347] Updated weights for policy 0, policy_version 1843415 (0.0008) [2023-12-27 18:25:00,436][20381] Updated weights for policy 1, policy_version 1841881 (0.0006) [2023-12-27 18:25:00,498][20381] Updated weights for policy 1, policy_version 1841891 (0.0005) [2023-12-27 18:25:00,552][20381] Updated weights for policy 1, policy_version 1841901 (0.0007) [2023-12-27 18:25:00,608][20381] Updated weights for policy 1, policy_version 1841911 (0.0008) [2023-12-27 18:25:01,077][20347] Updated weights for policy 0, policy_version 1843425 (0.0008) [2023-12-27 18:25:01,137][20347] Updated weights for policy 0, policy_version 1843435 (0.0007) [2023-12-27 18:25:01,192][20347] Updated weights for policy 0, policy_version 1843445 (0.0006) [2023-12-27 18:25:01,228][20381] Updated weights for policy 1, policy_version 1841921 (0.0008) [2023-12-27 18:25:01,287][20381] Updated weights for policy 1, policy_version 1841931 (0.0008) [2023-12-27 18:25:01,340][20381] Updated weights for policy 1, policy_version 1841941 (0.0008) [2023-12-27 18:25:01,904][20347] Updated weights for policy 0, policy_version 1843455 (0.0007) [2023-12-27 18:25:01,965][20347] Updated weights for policy 0, policy_version 1843465 (0.0006) [2023-12-27 18:25:02,030][20347] Updated weights for policy 0, policy_version 1843475 (0.0006) [2023-12-27 18:25:02,118][20381] Updated weights for policy 1, policy_version 1841951 (0.0008) [2023-12-27 18:25:02,167][20381] Updated weights for policy 1, policy_version 1841961 (0.0007) [2023-12-27 18:25:02,227][20381] Updated weights for policy 1, policy_version 1841971 (0.0006) [2023-12-27 18:25:02,362][19170] Fps is (10 sec: 20495.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 943611904. Throughput: 0: 10261.2, 1: 10443.4. Samples: 943618668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.980')] [2023-12-27 18:25:02,687][20347] Updated weights for policy 0, policy_version 1843485 (0.0006) [2023-12-27 18:25:02,777][20347] Updated weights for policy 0, policy_version 1843495 (0.0008) [2023-12-27 18:25:02,843][20347] Updated weights for policy 0, policy_version 1843505 (0.0008) [2023-12-27 18:25:02,961][20381] Updated weights for policy 1, policy_version 1841981 (0.0010) [2023-12-27 18:25:03,016][20381] Updated weights for policy 1, policy_version 1841991 (0.0009) [2023-12-27 18:25:03,073][20381] Updated weights for policy 1, policy_version 1842001 (0.0008) [2023-12-27 18:25:03,414][20347] Updated weights for policy 0, policy_version 1843515 (0.0007) [2023-12-27 18:25:03,458][20347] Updated weights for policy 0, policy_version 1843525 (0.0007) [2023-12-27 18:25:03,508][20347] Updated weights for policy 0, policy_version 1843535 (0.0009) [2023-12-27 18:25:03,729][20381] Updated weights for policy 1, policy_version 1842011 (0.0008) [2023-12-27 18:25:03,788][20381] Updated weights for policy 1, policy_version 1842021 (0.0008) [2023-12-27 18:25:03,845][20381] Updated weights for policy 1, policy_version 1842031 (0.0007) [2023-12-27 18:25:04,158][20347] Updated weights for policy 0, policy_version 1843545 (0.0007) [2023-12-27 18:25:04,218][20347] Updated weights for policy 0, policy_version 1843555 (0.0006) [2023-12-27 18:25:04,269][20347] Updated weights for policy 0, policy_version 1843565 (0.0005) [2023-12-27 18:25:04,332][20347] Updated weights for policy 0, policy_version 1843575 (0.0005) [2023-12-27 18:25:04,577][20381] Updated weights for policy 1, policy_version 1842041 (0.0007) [2023-12-27 18:25:04,628][20381] Updated weights for policy 1, policy_version 1842051 (0.0008) [2023-12-27 18:25:04,679][20381] Updated weights for policy 1, policy_version 1842061 (0.0008) [2023-12-27 18:25:04,730][20381] Updated weights for policy 1, policy_version 1842071 (0.0006) [2023-12-27 18:25:04,920][20347] Updated weights for policy 0, policy_version 1843585 (0.0005) [2023-12-27 18:25:04,978][20347] Updated weights for policy 0, policy_version 1843595 (0.0005) [2023-12-27 18:25:05,028][20347] Updated weights for policy 0, policy_version 1843605 (0.0005) [2023-12-27 18:25:05,316][20381] Updated weights for policy 1, policy_version 1842081 (0.0009) [2023-12-27 18:25:05,365][20381] Updated weights for policy 1, policy_version 1842091 (0.0011) [2023-12-27 18:25:05,420][20381] Updated weights for policy 1, policy_version 1842101 (0.0010) [2023-12-27 18:25:05,670][20347] Updated weights for policy 0, policy_version 1843615 (0.0008) [2023-12-27 18:25:05,719][20347] Updated weights for policy 0, policy_version 1843625 (0.0007) [2023-12-27 18:25:05,769][20347] Updated weights for policy 0, policy_version 1843635 (0.0007) [2023-12-27 18:25:06,098][20381] Updated weights for policy 1, policy_version 1842111 (0.0009) [2023-12-27 18:25:06,152][20381] Updated weights for policy 1, policy_version 1842121 (0.0011) [2023-12-27 18:25:06,207][20381] Updated weights for policy 1, policy_version 1842131 (0.0011) [2023-12-27 18:25:06,478][20347] Updated weights for policy 0, policy_version 1843646 (0.0008) [2023-12-27 18:25:06,535][20347] Updated weights for policy 0, policy_version 1843656 (0.0007) [2023-12-27 18:25:06,590][20347] Updated weights for policy 0, policy_version 1843666 (0.0009) [2023-12-27 18:25:06,910][20381] Updated weights for policy 1, policy_version 1842141 (0.0011) [2023-12-27 18:25:06,975][20381] Updated weights for policy 1, policy_version 1842151 (0.0011) [2023-12-27 18:25:07,031][20381] Updated weights for policy 1, policy_version 1842161 (0.0009) [2023-12-27 18:25:07,251][20347] Updated weights for policy 0, policy_version 1843676 (0.0007) [2023-12-27 18:25:07,302][20347] Updated weights for policy 0, policy_version 1843686 (0.0005) [2023-12-27 18:25:07,355][20347] Updated weights for policy 0, policy_version 1843696 (0.0005) [2023-12-27 18:25:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 943710208. Throughput: 0: 10272.8, 1: 10410.3. Samples: 943682788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '850.320')] [2023-12-27 18:25:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001842168_471646208.pth... [2023-12-27 18:25:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001840952_471334912.pth [2023-12-27 18:25:07,400][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001843704_472072192.pth... [2023-12-27 18:25:07,403][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001842488_471760896.pth [2023-12-27 18:25:07,694][20381] Updated weights for policy 1, policy_version 1842171 (0.0006) [2023-12-27 18:25:07,747][20381] Updated weights for policy 1, policy_version 1842181 (0.0008) [2023-12-27 18:25:07,799][20381] Updated weights for policy 1, policy_version 1842191 (0.0008) [2023-12-27 18:25:07,910][20347] Updated weights for policy 0, policy_version 1843706 (0.0005) [2023-12-27 18:25:07,965][20347] Updated weights for policy 0, policy_version 1843716 (0.0006) [2023-12-27 18:25:08,022][20347] Updated weights for policy 0, policy_version 1843726 (0.0005) [2023-12-27 18:25:08,082][20347] Updated weights for policy 0, policy_version 1843736 (0.0005) [2023-12-27 18:25:08,494][20381] Updated weights for policy 1, policy_version 1842201 (0.0010) [2023-12-27 18:25:08,558][20381] Updated weights for policy 1, policy_version 1842211 (0.0006) [2023-12-27 18:25:08,616][20381] Updated weights for policy 1, policy_version 1842221 (0.0010) [2023-12-27 18:25:08,670][20381] Updated weights for policy 1, policy_version 1842231 (0.0007) [2023-12-27 18:25:08,710][20347] Updated weights for policy 0, policy_version 1843746 (0.0010) [2023-12-27 18:25:08,761][20347] Updated weights for policy 0, policy_version 1843756 (0.0010) [2023-12-27 18:25:08,813][20347] Updated weights for policy 0, policy_version 1843766 (0.0010) [2023-12-27 18:25:09,391][20381] Updated weights for policy 1, policy_version 1842241 (0.0009) [2023-12-27 18:25:09,454][20381] Updated weights for policy 1, policy_version 1842251 (0.0009) [2023-12-27 18:25:09,514][20381] Updated weights for policy 1, policy_version 1842261 (0.0008) [2023-12-27 18:25:09,534][20347] Updated weights for policy 0, policy_version 1843776 (0.0007) [2023-12-27 18:25:09,599][20347] Updated weights for policy 0, policy_version 1843786 (0.0008) [2023-12-27 18:25:09,662][20347] Updated weights for policy 0, policy_version 1843796 (0.0007) [2023-12-27 18:25:10,322][20381] Updated weights for policy 1, policy_version 1842271 (0.0010) [2023-12-27 18:25:10,382][20347] Updated weights for policy 0, policy_version 1843806 (0.0008) [2023-12-27 18:25:10,387][20381] Updated weights for policy 1, policy_version 1842281 (0.0010) [2023-12-27 18:25:10,444][20347] Updated weights for policy 0, policy_version 1843816 (0.0007) [2023-12-27 18:25:10,456][20381] Updated weights for policy 1, policy_version 1842291 (0.0009) [2023-12-27 18:25:10,508][20347] Updated weights for policy 0, policy_version 1843826 (0.0007) [2023-12-27 18:25:11,180][20347] Updated weights for policy 0, policy_version 1843836 (0.0007) [2023-12-27 18:25:11,190][20381] Updated weights for policy 1, policy_version 1842301 (0.0009) [2023-12-27 18:25:11,251][20347] Updated weights for policy 0, policy_version 1843846 (0.0008) [2023-12-27 18:25:11,263][20381] Updated weights for policy 1, policy_version 1842311 (0.0009) [2023-12-27 18:25:11,324][20381] Updated weights for policy 1, policy_version 1842321 (0.0010) [2023-12-27 18:25:11,325][20347] Updated weights for policy 0, policy_version 1843856 (0.0007) [2023-12-27 18:25:12,135][20347] Updated weights for policy 0, policy_version 1843866 (0.0008) [2023-12-27 18:25:12,202][20347] Updated weights for policy 0, policy_version 1843876 (0.0009) [2023-12-27 18:25:12,227][20381] Updated weights for policy 1, policy_version 1842331 (0.0010) [2023-12-27 18:25:12,268][20347] Updated weights for policy 0, policy_version 1843886 (0.0008) [2023-12-27 18:25:12,295][20381] Updated weights for policy 1, policy_version 1842341 (0.0008) [2023-12-27 18:25:12,339][20347] Updated weights for policy 0, policy_version 1843896 (0.0007) [2023-12-27 18:25:12,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 943808512. Throughput: 0: 10322.0, 1: 10290.6. Samples: 943803368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '878.310')] [2023-12-27 18:25:12,378][20381] Updated weights for policy 1, policy_version 1842351 (0.0008) [2023-12-27 18:25:13,118][20347] Updated weights for policy 0, policy_version 1843906 (0.0009) [2023-12-27 18:25:13,120][20381] Updated weights for policy 1, policy_version 1842361 (0.0013) [2023-12-27 18:25:13,185][20347] Updated weights for policy 0, policy_version 1843916 (0.0008) [2023-12-27 18:25:13,191][20381] Updated weights for policy 1, policy_version 1842371 (0.0008) [2023-12-27 18:25:13,245][20347] Updated weights for policy 0, policy_version 1843926 (0.0008) [2023-12-27 18:25:13,260][20381] Updated weights for policy 1, policy_version 1842381 (0.0008) [2023-12-27 18:25:13,328][20381] Updated weights for policy 1, policy_version 1842391 (0.0008) [2023-12-27 18:25:14,011][20347] Updated weights for policy 0, policy_version 1843936 (0.0010) [2023-12-27 18:25:14,080][20347] Updated weights for policy 0, policy_version 1843946 (0.0010) [2023-12-27 18:25:14,083][20381] Updated weights for policy 1, policy_version 1842401 (0.0008) [2023-12-27 18:25:14,153][20381] Updated weights for policy 1, policy_version 1842411 (0.0009) [2023-12-27 18:25:14,155][20347] Updated weights for policy 0, policy_version 1843956 (0.0010) [2023-12-27 18:25:14,227][20381] Updated weights for policy 1, policy_version 1842421 (0.0008) [2023-12-27 18:25:14,919][20347] Updated weights for policy 0, policy_version 1843966 (0.0008) [2023-12-27 18:25:14,978][20347] Updated weights for policy 0, policy_version 1843976 (0.0007) [2023-12-27 18:25:14,996][20381] Updated weights for policy 1, policy_version 1842431 (0.0007) [2023-12-27 18:25:15,033][20347] Updated weights for policy 0, policy_version 1843986 (0.0006) [2023-12-27 18:25:15,055][20381] Updated weights for policy 1, policy_version 1842441 (0.0006) [2023-12-27 18:25:15,104][20381] Updated weights for policy 1, policy_version 1842451 (0.0007) [2023-12-27 18:25:15,778][20381] Updated weights for policy 1, policy_version 1842461 (0.0007) [2023-12-27 18:25:15,834][20381] Updated weights for policy 1, policy_version 1842471 (0.0007) [2023-12-27 18:25:15,860][20347] Updated weights for policy 0, policy_version 1843996 (0.0008) [2023-12-27 18:25:15,892][20381] Updated weights for policy 1, policy_version 1842481 (0.0007) [2023-12-27 18:25:15,913][20347] Updated weights for policy 0, policy_version 1844006 (0.0006) [2023-12-27 18:25:15,966][20347] Updated weights for policy 0, policy_version 1844016 (0.0008) [2023-12-27 18:25:16,560][20381] Updated weights for policy 1, policy_version 1842491 (0.0007) [2023-12-27 18:25:16,608][20381] Updated weights for policy 1, policy_version 1842501 (0.0010) [2023-12-27 18:25:16,657][20381] Updated weights for policy 1, policy_version 1842511 (0.0010) [2023-12-27 18:25:16,704][20347] Updated weights for policy 0, policy_version 1844026 (0.0008) [2023-12-27 18:25:16,761][20347] Updated weights for policy 0, policy_version 1844036 (0.0008) [2023-12-27 18:25:16,822][20347] Updated weights for policy 0, policy_version 1844046 (0.0008) [2023-12-27 18:25:16,879][20347] Updated weights for policy 0, policy_version 1844056 (0.0007) [2023-12-27 18:25:17,362][19170] Fps is (10 sec: 18841.4, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 943898624. Throughput: 0: 10156.8, 1: 10148.6. Samples: 943913340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '927.140')] [2023-12-27 18:25:17,391][20381] Updated weights for policy 1, policy_version 1842521 (0.0011) [2023-12-27 18:25:17,455][20381] Updated weights for policy 1, policy_version 1842531 (0.0007) [2023-12-27 18:25:17,511][20381] Updated weights for policy 1, policy_version 1842541 (0.0008) [2023-12-27 18:25:17,513][20347] Updated weights for policy 0, policy_version 1844066 (0.0010) [2023-12-27 18:25:17,567][20347] Updated weights for policy 0, policy_version 1844076 (0.0010) [2023-12-27 18:25:17,567][20381] Updated weights for policy 1, policy_version 1842551 (0.0008) [2023-12-27 18:25:17,624][20347] Updated weights for policy 0, policy_version 1844086 (0.0010) [2023-12-27 18:25:18,255][20381] Updated weights for policy 1, policy_version 1842561 (0.0008) [2023-12-27 18:25:18,309][20347] Updated weights for policy 0, policy_version 1844096 (0.0010) [2023-12-27 18:25:18,314][20381] Updated weights for policy 1, policy_version 1842571 (0.0007) [2023-12-27 18:25:18,372][20347] Updated weights for policy 0, policy_version 1844106 (0.0009) [2023-12-27 18:25:18,390][20381] Updated weights for policy 1, policy_version 1842582 (0.0008) [2023-12-27 18:25:18,433][20347] Updated weights for policy 0, policy_version 1844116 (0.0011) [2023-12-27 18:25:19,124][20347] Updated weights for policy 0, policy_version 1844126 (0.0007) [2023-12-27 18:25:19,136][20381] Updated weights for policy 1, policy_version 1842592 (0.0007) [2023-12-27 18:25:19,180][20347] Updated weights for policy 0, policy_version 1844136 (0.0006) [2023-12-27 18:25:19,203][20381] Updated weights for policy 1, policy_version 1842602 (0.0007) [2023-12-27 18:25:19,244][20347] Updated weights for policy 0, policy_version 1844146 (0.0008) [2023-12-27 18:25:19,273][20381] Updated weights for policy 1, policy_version 1842612 (0.0007) [2023-12-27 18:25:19,975][20347] Updated weights for policy 0, policy_version 1844156 (0.0008) [2023-12-27 18:25:20,050][20347] Updated weights for policy 0, policy_version 1844166 (0.0007) [2023-12-27 18:25:20,087][20381] Updated weights for policy 1, policy_version 1842622 (0.0008) [2023-12-27 18:25:20,121][20347] Updated weights for policy 0, policy_version 1844176 (0.0008) [2023-12-27 18:25:20,158][20381] Updated weights for policy 1, policy_version 1842632 (0.0008) [2023-12-27 18:25:20,223][20381] Updated weights for policy 1, policy_version 1842642 (0.0008) [2023-12-27 18:25:20,928][20347] Updated weights for policy 0, policy_version 1844186 (0.0007) [2023-12-27 18:25:20,999][20347] Updated weights for policy 0, policy_version 1844196 (0.0007) [2023-12-27 18:25:21,035][20381] Updated weights for policy 1, policy_version 1842652 (0.0009) [2023-12-27 18:25:21,073][20347] Updated weights for policy 0, policy_version 1844206 (0.0008) [2023-12-27 18:25:21,112][20381] Updated weights for policy 1, policy_version 1842662 (0.0009) [2023-12-27 18:25:21,146][20347] Updated weights for policy 0, policy_version 1844216 (0.0007) [2023-12-27 18:25:21,181][20381] Updated weights for policy 1, policy_version 1842672 (0.0008) [2023-12-27 18:25:21,934][20347] Updated weights for policy 0, policy_version 1844226 (0.0008) [2023-12-27 18:25:21,996][20347] Updated weights for policy 0, policy_version 1844236 (0.0009) [2023-12-27 18:25:22,013][20381] Updated weights for policy 1, policy_version 1842682 (0.0008) [2023-12-27 18:25:22,060][20347] Updated weights for policy 0, policy_version 1844246 (0.0010) [2023-12-27 18:25:22,076][20381] Updated weights for policy 1, policy_version 1842692 (0.0006) [2023-12-27 18:25:22,137][20381] Updated weights for policy 1, policy_version 1842702 (0.0008) [2023-12-27 18:25:22,198][20381] Updated weights for policy 1, policy_version 1842712 (0.0008) [2023-12-27 18:25:22,362][19170] Fps is (10 sec: 18841.4, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 943996928. Throughput: 0: 10075.8, 1: 10036.5. Samples: 943970372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '879.310')] [2023-12-27 18:25:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001842712_471785472.pth... [2023-12-27 18:25:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001844248_472211456.pth... [2023-12-27 18:25:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001841560_471490560.pth [2023-12-27 18:25:22,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001843096_471916544.pth [2023-12-27 18:25:22,809][20347] Updated weights for policy 0, policy_version 1844256 (0.0008) [2023-12-27 18:25:22,871][20347] Updated weights for policy 0, policy_version 1844266 (0.0008) [2023-12-27 18:25:22,926][20347] Updated weights for policy 0, policy_version 1844276 (0.0008) [2023-12-27 18:25:22,978][20381] Updated weights for policy 1, policy_version 1842722 (0.0008) [2023-12-27 18:25:23,045][20381] Updated weights for policy 1, policy_version 1842732 (0.0008) [2023-12-27 18:25:23,108][20381] Updated weights for policy 1, policy_version 1842742 (0.0007) [2023-12-27 18:25:23,654][20347] Updated weights for policy 0, policy_version 1844286 (0.0009) [2023-12-27 18:25:23,713][20347] Updated weights for policy 0, policy_version 1844296 (0.0007) [2023-12-27 18:25:23,768][20347] Updated weights for policy 0, policy_version 1844306 (0.0008) [2023-12-27 18:25:23,831][20381] Updated weights for policy 1, policy_version 1842752 (0.0007) [2023-12-27 18:25:23,895][20381] Updated weights for policy 1, policy_version 1842762 (0.0009) [2023-12-27 18:25:23,965][20381] Updated weights for policy 1, policy_version 1842772 (0.0008) [2023-12-27 18:25:24,557][20347] Updated weights for policy 0, policy_version 1844316 (0.0008) [2023-12-27 18:25:24,621][20347] Updated weights for policy 0, policy_version 1844326 (0.0009) [2023-12-27 18:25:24,685][20347] Updated weights for policy 0, policy_version 1844336 (0.0008) [2023-12-27 18:25:24,702][20381] Updated weights for policy 1, policy_version 1842782 (0.0007) [2023-12-27 18:25:24,768][20381] Updated weights for policy 1, policy_version 1842792 (0.0008) [2023-12-27 18:25:24,830][20381] Updated weights for policy 1, policy_version 1842802 (0.0009) [2023-12-27 18:25:25,477][20347] Updated weights for policy 0, policy_version 1844346 (0.0007) [2023-12-27 18:25:25,541][20347] Updated weights for policy 0, policy_version 1844356 (0.0007) [2023-12-27 18:25:25,577][20381] Updated weights for policy 1, policy_version 1842812 (0.0009) [2023-12-27 18:25:25,609][20347] Updated weights for policy 0, policy_version 1844366 (0.0009) [2023-12-27 18:25:25,644][20381] Updated weights for policy 1, policy_version 1842822 (0.0007) [2023-12-27 18:25:25,673][20347] Updated weights for policy 0, policy_version 1844376 (0.0009) [2023-12-27 18:25:25,706][20381] Updated weights for policy 1, policy_version 1842832 (0.0006) [2023-12-27 18:25:26,533][20347] Updated weights for policy 0, policy_version 1844386 (0.0007) [2023-12-27 18:25:26,555][20381] Updated weights for policy 1, policy_version 1842842 (0.0008) [2023-12-27 18:25:26,589][20347] Updated weights for policy 0, policy_version 1844396 (0.0007) [2023-12-27 18:25:26,618][20381] Updated weights for policy 1, policy_version 1842852 (0.0008) [2023-12-27 18:25:26,651][20347] Updated weights for policy 0, policy_version 1844406 (0.0006) [2023-12-27 18:25:26,677][20381] Updated weights for policy 1, policy_version 1842862 (0.0007) [2023-12-27 18:25:26,735][20381] Updated weights for policy 1, policy_version 1842872 (0.0008) [2023-12-27 18:25:27,263][20347] Updated weights for policy 0, policy_version 1844416 (0.0007) [2023-12-27 18:25:27,331][20347] Updated weights for policy 0, policy_version 1844426 (0.0008) [2023-12-27 18:25:27,362][19170] Fps is (10 sec: 18022.6, 60 sec: 19933.9, 300 sec: 20410.6). Total num frames: 944078848. Throughput: 0: 9863.6, 1: 9895.8. Samples: 944076428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '878.410')] [2023-12-27 18:25:27,402][20347] Updated weights for policy 0, policy_version 1844436 (0.0006) [2023-12-27 18:25:27,493][20381] Updated weights for policy 1, policy_version 1842882 (0.0007) [2023-12-27 18:25:27,551][20381] Updated weights for policy 1, policy_version 1842892 (0.0008) [2023-12-27 18:25:27,606][20381] Updated weights for policy 1, policy_version 1842902 (0.0008) [2023-12-27 18:25:28,106][20347] Updated weights for policy 0, policy_version 1844446 (0.0008) [2023-12-27 18:25:28,168][20347] Updated weights for policy 0, policy_version 1844456 (0.0008) [2023-12-27 18:25:28,231][20347] Updated weights for policy 0, policy_version 1844466 (0.0008) [2023-12-27 18:25:28,324][20381] Updated weights for policy 1, policy_version 1842912 (0.0010) [2023-12-27 18:25:28,397][20381] Updated weights for policy 1, policy_version 1842922 (0.0011) [2023-12-27 18:25:28,470][20381] Updated weights for policy 1, policy_version 1842932 (0.0009) [2023-12-27 18:25:29,105][20347] Updated weights for policy 0, policy_version 1844476 (0.0008) [2023-12-27 18:25:29,174][20347] Updated weights for policy 0, policy_version 1844486 (0.0008) [2023-12-27 18:25:29,242][20347] Updated weights for policy 0, policy_version 1844496 (0.0008) [2023-12-27 18:25:29,301][20381] Updated weights for policy 1, policy_version 1842942 (0.0009) [2023-12-27 18:25:29,396][20381] Updated weights for policy 1, policy_version 1842952 (0.0009) [2023-12-27 18:25:29,465][20381] Updated weights for policy 1, policy_version 1842962 (0.0009) [2023-12-27 18:25:30,105][20347] Updated weights for policy 0, policy_version 1844506 (0.0008) [2023-12-27 18:25:30,171][20347] Updated weights for policy 0, policy_version 1844516 (0.0008) [2023-12-27 18:25:30,229][20347] Updated weights for policy 0, policy_version 1844526 (0.0007) [2023-12-27 18:25:30,259][20381] Updated weights for policy 1, policy_version 1842972 (0.0008) [2023-12-27 18:25:30,296][20347] Updated weights for policy 0, policy_version 1844536 (0.0007) [2023-12-27 18:25:30,325][20381] Updated weights for policy 1, policy_version 1842982 (0.0007) [2023-12-27 18:25:30,395][20381] Updated weights for policy 1, policy_version 1842992 (0.0008) [2023-12-27 18:25:31,036][20347] Updated weights for policy 0, policy_version 1844546 (0.0008) [2023-12-27 18:25:31,111][20347] Updated weights for policy 0, policy_version 1844556 (0.0007) [2023-12-27 18:25:31,168][20381] Updated weights for policy 1, policy_version 1843002 (0.0009) [2023-12-27 18:25:31,187][20347] Updated weights for policy 0, policy_version 1844566 (0.0008) [2023-12-27 18:25:31,239][20381] Updated weights for policy 1, policy_version 1843013 (0.0009) [2023-12-27 18:25:31,310][20381] Updated weights for policy 1, policy_version 1843023 (0.0007) [2023-12-27 18:25:32,032][20347] Updated weights for policy 0, policy_version 1844576 (0.0008) [2023-12-27 18:25:32,088][20347] Updated weights for policy 0, policy_version 1844586 (0.0007) [2023-12-27 18:25:32,106][20381] Updated weights for policy 1, policy_version 1843034 (0.0009) [2023-12-27 18:25:32,143][20347] Updated weights for policy 0, policy_version 1844596 (0.0006) [2023-12-27 18:25:32,165][20381] Updated weights for policy 1, policy_version 1843044 (0.0006) [2023-12-27 18:25:32,227][20381] Updated weights for policy 1, policy_version 1843054 (0.0008) [2023-12-27 18:25:32,295][20381] Updated weights for policy 1, policy_version 1843064 (0.0008) [2023-12-27 18:25:32,362][19170] Fps is (10 sec: 18022.5, 60 sec: 19797.3, 300 sec: 20355.0). Total num frames: 944177152. Throughput: 0: 9716.6, 1: 9666.6. Samples: 944182808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '907.470')] [2023-12-27 18:25:32,885][20347] Updated weights for policy 0, policy_version 1844606 (0.0005) [2023-12-27 18:25:32,947][20347] Updated weights for policy 0, policy_version 1844616 (0.0007) [2023-12-27 18:25:33,013][20347] Updated weights for policy 0, policy_version 1844626 (0.0007) [2023-12-27 18:25:33,056][20381] Updated weights for policy 1, policy_version 1843074 (0.0008) [2023-12-27 18:25:33,119][20381] Updated weights for policy 1, policy_version 1843084 (0.0008) [2023-12-27 18:25:33,177][20381] Updated weights for policy 1, policy_version 1843094 (0.0008) [2023-12-27 18:25:33,711][20347] Updated weights for policy 0, policy_version 1844636 (0.0007) [2023-12-27 18:25:33,772][20347] Updated weights for policy 0, policy_version 1844646 (0.0008) [2023-12-27 18:25:33,834][20347] Updated weights for policy 0, policy_version 1844656 (0.0008) [2023-12-27 18:25:33,918][20381] Updated weights for policy 1, policy_version 1843104 (0.0007) [2023-12-27 18:25:33,979][20381] Updated weights for policy 1, policy_version 1843114 (0.0007) [2023-12-27 18:25:34,041][20381] Updated weights for policy 1, policy_version 1843124 (0.0007) [2023-12-27 18:25:34,638][20347] Updated weights for policy 0, policy_version 1844666 (0.0008) [2023-12-27 18:25:34,715][20347] Updated weights for policy 0, policy_version 1844676 (0.0007) [2023-12-27 18:25:34,784][20347] Updated weights for policy 0, policy_version 1844686 (0.0007) [2023-12-27 18:25:34,812][20381] Updated weights for policy 1, policy_version 1843134 (0.0009) [2023-12-27 18:25:34,849][20347] Updated weights for policy 0, policy_version 1844696 (0.0008) [2023-12-27 18:25:34,883][20381] Updated weights for policy 1, policy_version 1843144 (0.0009) [2023-12-27 18:25:34,952][20381] Updated weights for policy 1, policy_version 1843154 (0.0008) [2023-12-27 18:25:35,515][20347] Updated weights for policy 0, policy_version 1844706 (0.0006) [2023-12-27 18:25:35,575][20347] Updated weights for policy 0, policy_version 1844716 (0.0008) [2023-12-27 18:25:35,628][20347] Updated weights for policy 0, policy_version 1844726 (0.0008) [2023-12-27 18:25:35,673][20381] Updated weights for policy 1, policy_version 1843164 (0.0008) [2023-12-27 18:25:35,732][20381] Updated weights for policy 1, policy_version 1843174 (0.0008) [2023-12-27 18:25:35,792][20381] Updated weights for policy 1, policy_version 1843184 (0.0008) [2023-12-27 18:25:36,351][20347] Updated weights for policy 0, policy_version 1844736 (0.0007) [2023-12-27 18:25:36,413][20347] Updated weights for policy 0, policy_version 1844746 (0.0008) [2023-12-27 18:25:36,479][20347] Updated weights for policy 0, policy_version 1844756 (0.0008) [2023-12-27 18:25:36,579][20381] Updated weights for policy 1, policy_version 1843194 (0.0008) [2023-12-27 18:25:36,641][20381] Updated weights for policy 1, policy_version 1843204 (0.0008) [2023-12-27 18:25:36,703][20381] Updated weights for policy 1, policy_version 1843214 (0.0008) [2023-12-27 18:25:36,770][20381] Updated weights for policy 1, policy_version 1843224 (0.0007) [2023-12-27 18:25:37,243][20347] Updated weights for policy 0, policy_version 1844766 (0.0007) [2023-12-27 18:25:37,304][20347] Updated weights for policy 0, policy_version 1844776 (0.0008) [2023-12-27 18:25:37,362][19170] Fps is (10 sec: 18022.2, 60 sec: 19387.8, 300 sec: 20327.3). Total num frames: 944259072. Throughput: 0: 9657.4, 1: 9565.3. Samples: 944237664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '829.080')] [2023-12-27 18:25:37,365][20347] Updated weights for policy 0, policy_version 1844786 (0.0007) [2023-12-27 18:25:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001843224_471916544.pth... [2023-12-27 18:25:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001842168_471646208.pth [2023-12-27 18:25:37,396][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001844792_472350720.pth... [2023-12-27 18:25:37,399][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001843704_472072192.pth [2023-12-27 18:25:37,470][20381] Updated weights for policy 1, policy_version 1843234 (0.0008) [2023-12-27 18:25:37,524][20381] Updated weights for policy 1, policy_version 1843244 (0.0008) [2023-12-27 18:25:37,591][20381] Updated weights for policy 1, policy_version 1843254 (0.0008) [2023-12-27 18:25:38,061][20347] Updated weights for policy 0, policy_version 1844796 (0.0009) [2023-12-27 18:25:38,115][20347] Updated weights for policy 0, policy_version 1844806 (0.0011) [2023-12-27 18:25:38,173][20347] Updated weights for policy 0, policy_version 1844816 (0.0011) [2023-12-27 18:25:38,333][20381] Updated weights for policy 1, policy_version 1843264 (0.0010) [2023-12-27 18:25:38,393][20381] Updated weights for policy 1, policy_version 1843274 (0.0009) [2023-12-27 18:25:38,459][20381] Updated weights for policy 1, policy_version 1843284 (0.0007) [2023-12-27 18:25:38,966][20347] Updated weights for policy 0, policy_version 1844826 (0.0009) [2023-12-27 18:25:39,021][20347] Updated weights for policy 0, policy_version 1844836 (0.0008) [2023-12-27 18:25:39,066][20347] Updated weights for policy 0, policy_version 1844846 (0.0007) [2023-12-27 18:25:39,113][20347] Updated weights for policy 0, policy_version 1844856 (0.0005) [2023-12-27 18:25:39,250][20381] Updated weights for policy 1, policy_version 1843295 (0.0008) [2023-12-27 18:25:39,314][20381] Updated weights for policy 1, policy_version 1843305 (0.0008) [2023-12-27 18:25:39,383][20381] Updated weights for policy 1, policy_version 1843315 (0.0007) [2023-12-27 18:25:39,936][20347] Updated weights for policy 0, policy_version 1844866 (0.0008) [2023-12-27 18:25:40,005][20347] Updated weights for policy 0, policy_version 1844876 (0.0008) [2023-12-27 18:25:40,065][20347] Updated weights for policy 0, policy_version 1844886 (0.0008) [2023-12-27 18:25:40,182][20381] Updated weights for policy 1, policy_version 1843325 (0.0009) [2023-12-27 18:25:40,250][20381] Updated weights for policy 1, policy_version 1843335 (0.0008) [2023-12-27 18:25:40,315][20381] Updated weights for policy 1, policy_version 1843345 (0.0008) [2023-12-27 18:25:40,881][20347] Updated weights for policy 0, policy_version 1844896 (0.0007) [2023-12-27 18:25:40,943][20347] Updated weights for policy 0, policy_version 1844906 (0.0007) [2023-12-27 18:25:41,010][20347] Updated weights for policy 0, policy_version 1844916 (0.0009) [2023-12-27 18:25:41,069][20381] Updated weights for policy 1, policy_version 1843355 (0.0008) [2023-12-27 18:25:41,135][20381] Updated weights for policy 1, policy_version 1843365 (0.0007) [2023-12-27 18:25:41,194][20381] Updated weights for policy 1, policy_version 1843375 (0.0008) [2023-12-27 18:25:41,876][20347] Updated weights for policy 0, policy_version 1844926 (0.0009) [2023-12-27 18:25:41,940][20347] Updated weights for policy 0, policy_version 1844936 (0.0009) [2023-12-27 18:25:41,999][20347] Updated weights for policy 0, policy_version 1844946 (0.0009) [2023-12-27 18:25:42,005][20381] Updated weights for policy 1, policy_version 1843385 (0.0008) [2023-12-27 18:25:42,067][20381] Updated weights for policy 1, policy_version 1843395 (0.0007) [2023-12-27 18:25:42,120][20381] Updated weights for policy 1, policy_version 1843405 (0.0008) [2023-12-27 18:25:42,174][20381] Updated weights for policy 1, policy_version 1843415 (0.0006) [2023-12-27 18:25:42,362][19170] Fps is (10 sec: 18022.4, 60 sec: 19387.7, 300 sec: 20271.7). Total num frames: 944357376. Throughput: 0: 9515.3, 1: 9367.7. Samples: 944348128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '847.010')] [2023-12-27 18:25:42,845][20347] Updated weights for policy 0, policy_version 1844956 (0.0009) [2023-12-27 18:25:42,910][20347] Updated weights for policy 0, policy_version 1844966 (0.0008) [2023-12-27 18:25:42,969][20347] Updated weights for policy 0, policy_version 1844976 (0.0006) [2023-12-27 18:25:42,989][20381] Updated weights for policy 1, policy_version 1843425 (0.0008) [2023-12-27 18:25:43,053][20381] Updated weights for policy 1, policy_version 1843435 (0.0006) [2023-12-27 18:25:43,107][20381] Updated weights for policy 1, policy_version 1843445 (0.0006) [2023-12-27 18:25:43,673][20347] Updated weights for policy 0, policy_version 1844986 (0.0007) [2023-12-27 18:25:43,733][20347] Updated weights for policy 0, policy_version 1844996 (0.0008) [2023-12-27 18:25:43,771][20381] Updated weights for policy 1, policy_version 1843455 (0.0007) [2023-12-27 18:25:43,796][20347] Updated weights for policy 0, policy_version 1845006 (0.0008) [2023-12-27 18:25:43,840][20381] Updated weights for policy 1, policy_version 1843465 (0.0007) [2023-12-27 18:25:43,857][20347] Updated weights for policy 0, policy_version 1845016 (0.0006) [2023-12-27 18:25:43,900][20381] Updated weights for policy 1, policy_version 1843475 (0.0007) [2023-12-27 18:25:44,557][20381] Updated weights for policy 1, policy_version 1843485 (0.0007) [2023-12-27 18:25:44,599][20347] Updated weights for policy 0, policy_version 1845026 (0.0007) [2023-12-27 18:25:44,621][20381] Updated weights for policy 1, policy_version 1843495 (0.0008) [2023-12-27 18:25:44,664][20347] Updated weights for policy 0, policy_version 1845036 (0.0008) [2023-12-27 18:25:44,683][20381] Updated weights for policy 1, policy_version 1843505 (0.0008) [2023-12-27 18:25:44,720][20347] Updated weights for policy 0, policy_version 1845046 (0.0006) [2023-12-27 18:25:45,479][20347] Updated weights for policy 0, policy_version 1845056 (0.0007) [2023-12-27 18:25:45,514][20381] Updated weights for policy 1, policy_version 1843515 (0.0008) [2023-12-27 18:25:45,535][20347] Updated weights for policy 0, policy_version 1845066 (0.0006) [2023-12-27 18:25:45,570][20381] Updated weights for policy 1, policy_version 1843525 (0.0010) [2023-12-27 18:25:45,589][20347] Updated weights for policy 0, policy_version 1845076 (0.0009) [2023-12-27 18:25:45,629][20381] Updated weights for policy 1, policy_version 1843535 (0.0011) [2023-12-27 18:25:46,257][20381] Updated weights for policy 1, policy_version 1843545 (0.0011) [2023-12-27 18:25:46,300][20347] Updated weights for policy 0, policy_version 1845086 (0.0006) [2023-12-27 18:25:46,315][20381] Updated weights for policy 1, policy_version 1843555 (0.0010) [2023-12-27 18:25:46,362][20347] Updated weights for policy 0, policy_version 1845096 (0.0009) [2023-12-27 18:25:46,369][20381] Updated weights for policy 1, policy_version 1843565 (0.0010) [2023-12-27 18:25:46,412][20347] Updated weights for policy 0, policy_version 1845106 (0.0007) [2023-12-27 18:25:46,434][20381] Updated weights for policy 1, policy_version 1843575 (0.0010) [2023-12-27 18:25:47,072][20347] Updated weights for policy 0, policy_version 1845116 (0.0007) [2023-12-27 18:25:47,094][20381] Updated weights for policy 1, policy_version 1843585 (0.0008) [2023-12-27 18:25:47,130][20347] Updated weights for policy 0, policy_version 1845126 (0.0010) [2023-12-27 18:25:47,159][20381] Updated weights for policy 1, policy_version 1843595 (0.0007) [2023-12-27 18:25:47,194][20347] Updated weights for policy 0, policy_version 1845136 (0.0011) [2023-12-27 18:25:47,213][20381] Updated weights for policy 1, policy_version 1843605 (0.0006) [2023-12-27 18:25:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 19387.7, 300 sec: 20271.7). Total num frames: 944455680. Throughput: 0: 9423.1, 1: 9298.2. Samples: 944461128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '886.100')] [2023-12-27 18:25:47,946][20381] Updated weights for policy 1, policy_version 1843615 (0.0008) [2023-12-27 18:25:47,964][20347] Updated weights for policy 0, policy_version 1845146 (0.0010) [2023-12-27 18:25:48,008][20381] Updated weights for policy 1, policy_version 1843625 (0.0007) [2023-12-27 18:25:48,033][20347] Updated weights for policy 0, policy_version 1845156 (0.0009) [2023-12-27 18:25:48,075][20381] Updated weights for policy 1, policy_version 1843635 (0.0008) [2023-12-27 18:25:48,097][20347] Updated weights for policy 0, policy_version 1845166 (0.0010) [2023-12-27 18:25:48,157][20347] Updated weights for policy 0, policy_version 1845176 (0.0010) [2023-12-27 18:25:48,868][20381] Updated weights for policy 1, policy_version 1843645 (0.0007) [2023-12-27 18:25:48,940][20381] Updated weights for policy 1, policy_version 1843655 (0.0008) [2023-12-27 18:25:49,008][20381] Updated weights for policy 1, policy_version 1843665 (0.0007) [2023-12-27 18:25:49,036][20347] Updated weights for policy 0, policy_version 1845186 (0.0008) [2023-12-27 18:25:49,095][20347] Updated weights for policy 0, policy_version 1845196 (0.0008) [2023-12-27 18:25:49,157][20347] Updated weights for policy 0, policy_version 1845206 (0.0008) [2023-12-27 18:25:49,860][20381] Updated weights for policy 1, policy_version 1843675 (0.0009) [2023-12-27 18:25:49,921][20381] Updated weights for policy 1, policy_version 1843685 (0.0012) [2023-12-27 18:25:49,999][20381] Updated weights for policy 1, policy_version 1843695 (0.0009) [2023-12-27 18:25:50,067][20347] Updated weights for policy 0, policy_version 1845216 (0.0008) [2023-12-27 18:25:50,130][20347] Updated weights for policy 0, policy_version 1845226 (0.0008) [2023-12-27 18:25:50,194][20347] Updated weights for policy 0, policy_version 1845236 (0.0007) [2023-12-27 18:25:50,758][20381] Updated weights for policy 1, policy_version 1843705 (0.0009) [2023-12-27 18:25:50,823][20381] Updated weights for policy 1, policy_version 1843715 (0.0008) [2023-12-27 18:25:50,885][20381] Updated weights for policy 1, policy_version 1843725 (0.0007) [2023-12-27 18:25:50,942][20347] Updated weights for policy 0, policy_version 1845246 (0.0006) [2023-12-27 18:25:50,949][20381] Updated weights for policy 1, policy_version 1843735 (0.0007) [2023-12-27 18:25:51,000][20347] Updated weights for policy 0, policy_version 1845256 (0.0006) [2023-12-27 18:25:51,070][20347] Updated weights for policy 0, policy_version 1845266 (0.0008) [2023-12-27 18:25:51,660][20381] Updated weights for policy 1, policy_version 1843745 (0.0008) [2023-12-27 18:25:51,725][20381] Updated weights for policy 1, policy_version 1843755 (0.0008) [2023-12-27 18:25:51,787][20381] Updated weights for policy 1, policy_version 1843765 (0.0006) [2023-12-27 18:25:51,795][20347] Updated weights for policy 0, policy_version 1845276 (0.0007) [2023-12-27 18:25:51,846][20347] Updated weights for policy 0, policy_version 1845286 (0.0008) [2023-12-27 18:25:51,899][20347] Updated weights for policy 0, policy_version 1845296 (0.0008) [2023-12-27 18:25:52,362][19170] Fps is (10 sec: 18022.5, 60 sec: 18844.0, 300 sec: 20216.2). Total num frames: 944537600. Throughput: 0: 9244.4, 1: 9185.2. Samples: 944512124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '849.670')] [2023-12-27 18:25:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001845304_472481792.pth... [2023-12-27 18:25:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001843768_472055808.pth... [2023-12-27 18:25:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001842712_471785472.pth [2023-12-27 18:25:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001844248_472211456.pth [2023-12-27 18:25:52,471][20381] Updated weights for policy 1, policy_version 1843775 (0.0008) [2023-12-27 18:25:52,532][20381] Updated weights for policy 1, policy_version 1843785 (0.0008) [2023-12-27 18:25:52,592][20347] Updated weights for policy 0, policy_version 1845306 (0.0008) [2023-12-27 18:25:52,593][20381] Updated weights for policy 1, policy_version 1843795 (0.0008) [2023-12-27 18:25:52,648][20347] Updated weights for policy 0, policy_version 1845316 (0.0008) [2023-12-27 18:25:52,704][20347] Updated weights for policy 0, policy_version 1845326 (0.0008) [2023-12-27 18:25:52,752][20347] Updated weights for policy 0, policy_version 1845336 (0.0008) [2023-12-27 18:25:53,280][20381] Updated weights for policy 1, policy_version 1843805 (0.0008) [2023-12-27 18:25:53,328][20381] Updated weights for policy 1, policy_version 1843815 (0.0007) [2023-12-27 18:25:53,383][20381] Updated weights for policy 1, policy_version 1843825 (0.0007) [2023-12-27 18:25:53,457][20347] Updated weights for policy 0, policy_version 1845346 (0.0010) [2023-12-27 18:25:53,505][20347] Updated weights for policy 0, policy_version 1845356 (0.0010) [2023-12-27 18:25:53,556][20347] Updated weights for policy 0, policy_version 1845366 (0.0010) [2023-12-27 18:25:54,104][20381] Updated weights for policy 1, policy_version 1843835 (0.0006) [2023-12-27 18:25:54,161][20381] Updated weights for policy 1, policy_version 1843845 (0.0007) [2023-12-27 18:25:54,207][20347] Updated weights for policy 0, policy_version 1845376 (0.0008) [2023-12-27 18:25:54,211][20381] Updated weights for policy 1, policy_version 1843855 (0.0009) [2023-12-27 18:25:54,253][20347] Updated weights for policy 0, policy_version 1845386 (0.0009) [2023-12-27 18:25:54,297][20347] Updated weights for policy 0, policy_version 1845396 (0.0010) [2023-12-27 18:25:54,885][20381] Updated weights for policy 1, policy_version 1843865 (0.0011) [2023-12-27 18:25:54,910][20347] Updated weights for policy 0, policy_version 1845406 (0.0007) [2023-12-27 18:25:54,945][20381] Updated weights for policy 1, policy_version 1843875 (0.0011) [2023-12-27 18:25:54,970][20347] Updated weights for policy 0, policy_version 1845416 (0.0006) [2023-12-27 18:25:55,000][20381] Updated weights for policy 1, policy_version 1843885 (0.0011) [2023-12-27 18:25:55,023][20347] Updated weights for policy 0, policy_version 1845426 (0.0010) [2023-12-27 18:25:55,053][20381] Updated weights for policy 1, policy_version 1843895 (0.0011) [2023-12-27 18:25:55,629][20347] Updated weights for policy 0, policy_version 1845436 (0.0009) [2023-12-27 18:25:55,683][20347] Updated weights for policy 0, policy_version 1845446 (0.0005) [2023-12-27 18:25:55,739][20347] Updated weights for policy 0, policy_version 1845456 (0.0006) [2023-12-27 18:25:55,748][20381] Updated weights for policy 1, policy_version 1843905 (0.0008) [2023-12-27 18:25:55,799][20381] Updated weights for policy 1, policy_version 1843915 (0.0008) [2023-12-27 18:25:55,860][20381] Updated weights for policy 1, policy_version 1843925 (0.0007) [2023-12-27 18:25:56,391][20347] Updated weights for policy 0, policy_version 1845466 (0.0006) [2023-12-27 18:25:56,442][20347] Updated weights for policy 0, policy_version 1845476 (0.0005) [2023-12-27 18:25:56,496][20347] Updated weights for policy 0, policy_version 1845486 (0.0007) [2023-12-27 18:25:56,497][20381] Updated weights for policy 1, policy_version 1843935 (0.0009) [2023-12-27 18:25:56,542][20381] Updated weights for policy 1, policy_version 1843945 (0.0010) [2023-12-27 18:25:56,547][20347] Updated weights for policy 0, policy_version 1845496 (0.0006) [2023-12-27 18:25:56,593][20381] Updated weights for policy 1, policy_version 1843955 (0.0010) [2023-12-27 18:25:57,178][20347] Updated weights for policy 0, policy_version 1845506 (0.0005) [2023-12-27 18:25:57,232][20347] Updated weights for policy 0, policy_version 1845516 (0.0005) [2023-12-27 18:25:57,287][20347] Updated weights for policy 0, policy_version 1845526 (0.0005) [2023-12-27 18:25:57,308][20381] Updated weights for policy 1, policy_version 1843965 (0.0009) [2023-12-27 18:25:57,362][20381] Updated weights for policy 1, policy_version 1843975 (0.0009) [2023-12-27 18:25:57,362][19170] Fps is (10 sec: 18841.8, 60 sec: 18841.6, 300 sec: 20244.0). Total num frames: 944644096. Throughput: 0: 9226.4, 1: 9225.4. Samples: 944633700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:25:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '849.650')] [2023-12-27 18:25:57,414][20381] Updated weights for policy 1, policy_version 1843985 (0.0008) [2023-12-27 18:25:57,846][20347] Updated weights for policy 0, policy_version 1845536 (0.0006) [2023-12-27 18:25:57,890][20347] Updated weights for policy 0, policy_version 1845546 (0.0007) [2023-12-27 18:25:57,942][20347] Updated weights for policy 0, policy_version 1845556 (0.0008) [2023-12-27 18:25:57,979][20381] Updated weights for policy 1, policy_version 1843995 (0.0009) [2023-12-27 18:25:58,036][20381] Updated weights for policy 1, policy_version 1844005 (0.0010) [2023-12-27 18:25:58,091][20381] Updated weights for policy 1, policy_version 1844015 (0.0010) [2023-12-27 18:25:58,671][20347] Updated weights for policy 0, policy_version 1845566 (0.0007) [2023-12-27 18:25:58,738][20347] Updated weights for policy 0, policy_version 1845576 (0.0008) [2023-12-27 18:25:58,815][20347] Updated weights for policy 0, policy_version 1845586 (0.0008) [2023-12-27 18:25:58,919][20381] Updated weights for policy 1, policy_version 1844025 (0.0010) [2023-12-27 18:25:58,982][20381] Updated weights for policy 1, policy_version 1844035 (0.0008) [2023-12-27 18:25:59,044][20381] Updated weights for policy 1, policy_version 1844045 (0.0008) [2023-12-27 18:25:59,102][20381] Updated weights for policy 1, policy_version 1844055 (0.0008) [2023-12-27 18:25:59,570][20347] Updated weights for policy 0, policy_version 1845596 (0.0007) [2023-12-27 18:25:59,633][20347] Updated weights for policy 0, policy_version 1845606 (0.0005) [2023-12-27 18:25:59,695][20347] Updated weights for policy 0, policy_version 1845616 (0.0006) [2023-12-27 18:25:59,812][20381] Updated weights for policy 1, policy_version 1844065 (0.0007) [2023-12-27 18:25:59,873][20381] Updated weights for policy 1, policy_version 1844075 (0.0008) [2023-12-27 18:25:59,926][20381] Updated weights for policy 1, policy_version 1844085 (0.0008) [2023-12-27 18:26:00,345][20347] Updated weights for policy 0, policy_version 1845626 (0.0010) [2023-12-27 18:26:00,404][20347] Updated weights for policy 0, policy_version 1845636 (0.0008) [2023-12-27 18:26:00,463][20347] Updated weights for policy 0, policy_version 1845646 (0.0007) [2023-12-27 18:26:00,530][20347] Updated weights for policy 0, policy_version 1845656 (0.0008) [2023-12-27 18:26:00,614][20381] Updated weights for policy 1, policy_version 1844095 (0.0007) [2023-12-27 18:26:00,668][20381] Updated weights for policy 1, policy_version 1844105 (0.0010) [2023-12-27 18:26:00,733][20381] Updated weights for policy 1, policy_version 1844115 (0.0010) [2023-12-27 18:26:01,173][20347] Updated weights for policy 0, policy_version 1845666 (0.0008) [2023-12-27 18:26:01,228][20347] Updated weights for policy 0, policy_version 1845676 (0.0008) [2023-12-27 18:26:01,282][20347] Updated weights for policy 0, policy_version 1845686 (0.0006) [2023-12-27 18:26:01,426][20381] Updated weights for policy 1, policy_version 1844125 (0.0009) [2023-12-27 18:26:01,481][20381] Updated weights for policy 1, policy_version 1844135 (0.0008) [2023-12-27 18:26:01,540][20381] Updated weights for policy 1, policy_version 1844145 (0.0008) [2023-12-27 18:26:01,981][20347] Updated weights for policy 0, policy_version 1845696 (0.0009) [2023-12-27 18:26:02,041][20347] Updated weights for policy 0, policy_version 1845706 (0.0007) [2023-12-27 18:26:02,100][20347] Updated weights for policy 0, policy_version 1845716 (0.0007) [2023-12-27 18:26:02,216][20381] Updated weights for policy 1, policy_version 1844155 (0.0008) [2023-12-27 18:26:02,277][20381] Updated weights for policy 1, policy_version 1844165 (0.0007) [2023-12-27 18:26:02,330][20381] Updated weights for policy 1, policy_version 1844175 (0.0006) [2023-12-27 18:26:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 18841.6, 300 sec: 20216.2). Total num frames: 944742400. Throughput: 0: 9391.9, 1: 9317.0. Samples: 944755240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '908.520')] [2023-12-27 18:26:02,719][20347] Updated weights for policy 0, policy_version 1845726 (0.0006) [2023-12-27 18:26:02,777][20347] Updated weights for policy 0, policy_version 1845736 (0.0006) [2023-12-27 18:26:02,839][20347] Updated weights for policy 0, policy_version 1845746 (0.0008) [2023-12-27 18:26:03,017][20381] Updated weights for policy 1, policy_version 1844185 (0.0007) [2023-12-27 18:26:03,066][20381] Updated weights for policy 1, policy_version 1844195 (0.0005) [2023-12-27 18:26:03,114][20381] Updated weights for policy 1, policy_version 1844205 (0.0005) [2023-12-27 18:26:03,158][20381] Updated weights for policy 1, policy_version 1844215 (0.0005) [2023-12-27 18:26:03,457][20347] Updated weights for policy 0, policy_version 1845756 (0.0009) [2023-12-27 18:26:03,508][20347] Updated weights for policy 0, policy_version 1845766 (0.0010) [2023-12-27 18:26:03,568][20347] Updated weights for policy 0, policy_version 1845776 (0.0010) [2023-12-27 18:26:03,764][20381] Updated weights for policy 1, policy_version 1844225 (0.0006) [2023-12-27 18:26:03,815][20381] Updated weights for policy 1, policy_version 1844235 (0.0010) [2023-12-27 18:26:03,877][20381] Updated weights for policy 1, policy_version 1844245 (0.0010) [2023-12-27 18:26:04,332][20347] Updated weights for policy 0, policy_version 1845786 (0.0010) [2023-12-27 18:26:04,395][20347] Updated weights for policy 0, policy_version 1845796 (0.0009) [2023-12-27 18:26:04,454][20347] Updated weights for policy 0, policy_version 1845806 (0.0008) [2023-12-27 18:26:04,513][20347] Updated weights for policy 0, policy_version 1845816 (0.0008) [2023-12-27 18:26:04,584][20381] Updated weights for policy 1, policy_version 1844255 (0.0010) [2023-12-27 18:26:04,642][20381] Updated weights for policy 1, policy_version 1844265 (0.0011) [2023-12-27 18:26:04,701][20381] Updated weights for policy 1, policy_version 1844275 (0.0011) [2023-12-27 18:26:05,161][20347] Updated weights for policy 0, policy_version 1845826 (0.0005) [2023-12-27 18:26:05,220][20347] Updated weights for policy 0, policy_version 1845836 (0.0006) [2023-12-27 18:26:05,272][20347] Updated weights for policy 0, policy_version 1845846 (0.0005) [2023-12-27 18:26:05,387][20381] Updated weights for policy 1, policy_version 1844285 (0.0011) [2023-12-27 18:26:05,448][20381] Updated weights for policy 1, policy_version 1844295 (0.0011) [2023-12-27 18:26:05,503][20381] Updated weights for policy 1, policy_version 1844305 (0.0010) [2023-12-27 18:26:05,905][20347] Updated weights for policy 0, policy_version 1845856 (0.0007) [2023-12-27 18:26:05,950][20347] Updated weights for policy 0, policy_version 1845866 (0.0007) [2023-12-27 18:26:06,004][20347] Updated weights for policy 0, policy_version 1845876 (0.0007) [2023-12-27 18:26:06,184][20381] Updated weights for policy 1, policy_version 1844315 (0.0010) [2023-12-27 18:26:06,244][20381] Updated weights for policy 1, policy_version 1844325 (0.0011) [2023-12-27 18:26:06,307][20381] Updated weights for policy 1, policy_version 1844335 (0.0011) [2023-12-27 18:26:06,711][20347] Updated weights for policy 0, policy_version 1845886 (0.0007) [2023-12-27 18:26:06,774][20347] Updated weights for policy 0, policy_version 1845896 (0.0007) [2023-12-27 18:26:06,826][20347] Updated weights for policy 0, policy_version 1845906 (0.0008) [2023-12-27 18:26:06,984][20381] Updated weights for policy 1, policy_version 1844345 (0.0010) [2023-12-27 18:26:07,038][20381] Updated weights for policy 1, policy_version 1844355 (0.0008) [2023-12-27 18:26:07,096][20381] Updated weights for policy 1, policy_version 1844365 (0.0008) [2023-12-27 18:26:07,153][20381] Updated weights for policy 1, policy_version 1844375 (0.0006) [2023-12-27 18:26:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 18978.1, 300 sec: 20244.0). Total num frames: 944848896. Throughput: 0: 9423.5, 1: 9395.8. Samples: 944817240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '846.070')] [2023-12-27 18:26:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001844376_472211456.pth... [2023-12-27 18:26:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001845912_472637440.pth... [2023-12-27 18:26:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001843224_471916544.pth [2023-12-27 18:26:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001844792_472350720.pth [2023-12-27 18:26:07,523][20347] Updated weights for policy 0, policy_version 1845916 (0.0007) [2023-12-27 18:26:07,582][20347] Updated weights for policy 0, policy_version 1845926 (0.0007) [2023-12-27 18:26:07,640][20347] Updated weights for policy 0, policy_version 1845936 (0.0007) [2023-12-27 18:26:07,829][20381] Updated weights for policy 1, policy_version 1844385 (0.0005) [2023-12-27 18:26:07,890][20381] Updated weights for policy 1, policy_version 1844395 (0.0006) [2023-12-27 18:26:07,936][20381] Updated weights for policy 1, policy_version 1844405 (0.0008) [2023-12-27 18:26:08,291][20347] Updated weights for policy 0, policy_version 1845946 (0.0007) [2023-12-27 18:26:08,345][20347] Updated weights for policy 0, policy_version 1845956 (0.0007) [2023-12-27 18:26:08,409][20347] Updated weights for policy 0, policy_version 1845966 (0.0008) [2023-12-27 18:26:08,467][20347] Updated weights for policy 0, policy_version 1845976 (0.0006) [2023-12-27 18:26:08,583][20381] Updated weights for policy 1, policy_version 1844415 (0.0008) [2023-12-27 18:26:08,640][20381] Updated weights for policy 1, policy_version 1844425 (0.0008) [2023-12-27 18:26:08,688][20381] Updated weights for policy 1, policy_version 1844435 (0.0009) [2023-12-27 18:26:09,120][20347] Updated weights for policy 0, policy_version 1845986 (0.0010) [2023-12-27 18:26:09,185][20347] Updated weights for policy 0, policy_version 1845996 (0.0008) [2023-12-27 18:26:09,246][20347] Updated weights for policy 0, policy_version 1846006 (0.0008) [2023-12-27 18:26:09,406][20381] Updated weights for policy 1, policy_version 1844445 (0.0009) [2023-12-27 18:26:09,472][20381] Updated weights for policy 1, policy_version 1844455 (0.0008) [2023-12-27 18:26:09,534][20381] Updated weights for policy 1, policy_version 1844465 (0.0008) [2023-12-27 18:26:10,009][20347] Updated weights for policy 0, policy_version 1846016 (0.0010) [2023-12-27 18:26:10,058][20347] Updated weights for policy 0, policy_version 1846026 (0.0010) [2023-12-27 18:26:10,108][20347] Updated weights for policy 0, policy_version 1846036 (0.0010) [2023-12-27 18:26:10,289][20381] Updated weights for policy 1, policy_version 1844475 (0.0008) [2023-12-27 18:26:10,353][20381] Updated weights for policy 1, policy_version 1844485 (0.0008) [2023-12-27 18:26:10,407][20381] Updated weights for policy 1, policy_version 1844495 (0.0006) [2023-12-27 18:26:10,755][20347] Updated weights for policy 0, policy_version 1846046 (0.0009) [2023-12-27 18:26:10,813][20347] Updated weights for policy 0, policy_version 1846056 (0.0006) [2023-12-27 18:26:10,872][20347] Updated weights for policy 0, policy_version 1846066 (0.0007) [2023-12-27 18:26:11,104][20381] Updated weights for policy 1, policy_version 1844505 (0.0006) [2023-12-27 18:26:11,173][20381] Updated weights for policy 1, policy_version 1844515 (0.0007) [2023-12-27 18:26:11,238][20381] Updated weights for policy 1, policy_version 1844525 (0.0006) [2023-12-27 18:26:11,300][20381] Updated weights for policy 1, policy_version 1844535 (0.0007) [2023-12-27 18:26:11,601][20347] Updated weights for policy 0, policy_version 1846076 (0.0008) [2023-12-27 18:26:11,672][20347] Updated weights for policy 0, policy_version 1846086 (0.0008) [2023-12-27 18:26:11,742][20347] Updated weights for policy 0, policy_version 1846096 (0.0007) [2023-12-27 18:26:12,067][20381] Updated weights for policy 1, policy_version 1844545 (0.0008) [2023-12-27 18:26:12,130][20381] Updated weights for policy 1, policy_version 1844555 (0.0007) [2023-12-27 18:26:12,193][20381] Updated weights for policy 1, policy_version 1844565 (0.0008) [2023-12-27 18:26:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 18978.1, 300 sec: 20216.2). Total num frames: 944947200. Throughput: 0: 9612.0, 1: 9533.4. Samples: 944937972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '848.780')] [2023-12-27 18:26:12,521][20347] Updated weights for policy 0, policy_version 1846106 (0.0008) [2023-12-27 18:26:12,579][20347] Updated weights for policy 0, policy_version 1846116 (0.0007) [2023-12-27 18:26:12,637][20347] Updated weights for policy 0, policy_version 1846126 (0.0007) [2023-12-27 18:26:12,693][20347] Updated weights for policy 0, policy_version 1846136 (0.0006) [2023-12-27 18:26:12,892][20381] Updated weights for policy 1, policy_version 1844575 (0.0010) [2023-12-27 18:26:12,939][20381] Updated weights for policy 1, policy_version 1844585 (0.0009) [2023-12-27 18:26:12,984][20381] Updated weights for policy 1, policy_version 1844595 (0.0006) [2023-12-27 18:26:13,313][20347] Updated weights for policy 0, policy_version 1846146 (0.0008) [2023-12-27 18:26:13,374][20347] Updated weights for policy 0, policy_version 1846156 (0.0006) [2023-12-27 18:26:13,430][20347] Updated weights for policy 0, policy_version 1846166 (0.0007) [2023-12-27 18:26:13,599][20381] Updated weights for policy 1, policy_version 1844605 (0.0008) [2023-12-27 18:26:13,655][20381] Updated weights for policy 1, policy_version 1844615 (0.0005) [2023-12-27 18:26:13,714][20381] Updated weights for policy 1, policy_version 1844625 (0.0006) [2023-12-27 18:26:14,113][20347] Updated weights for policy 0, policy_version 1846176 (0.0007) [2023-12-27 18:26:14,171][20347] Updated weights for policy 0, policy_version 1846186 (0.0007) [2023-12-27 18:26:14,223][20347] Updated weights for policy 0, policy_version 1846196 (0.0007) [2023-12-27 18:26:14,306][20381] Updated weights for policy 1, policy_version 1844635 (0.0010) [2023-12-27 18:26:14,351][20381] Updated weights for policy 1, policy_version 1844645 (0.0008) [2023-12-27 18:26:14,398][20381] Updated weights for policy 1, policy_version 1844655 (0.0006) [2023-12-27 18:26:14,918][20347] Updated weights for policy 0, policy_version 1846206 (0.0009) [2023-12-27 18:26:14,978][20347] Updated weights for policy 0, policy_version 1846216 (0.0007) [2023-12-27 18:26:15,038][20347] Updated weights for policy 0, policy_version 1846226 (0.0006) [2023-12-27 18:26:15,103][20381] Updated weights for policy 1, policy_version 1844665 (0.0008) [2023-12-27 18:26:15,164][20381] Updated weights for policy 1, policy_version 1844675 (0.0008) [2023-12-27 18:26:15,221][20381] Updated weights for policy 1, policy_version 1844685 (0.0008) [2023-12-27 18:26:15,281][20381] Updated weights for policy 1, policy_version 1844695 (0.0009) [2023-12-27 18:26:15,644][20347] Updated weights for policy 0, policy_version 1846236 (0.0006) [2023-12-27 18:26:15,687][20347] Updated weights for policy 0, policy_version 1846246 (0.0008) [2023-12-27 18:26:15,744][20347] Updated weights for policy 0, policy_version 1846256 (0.0005) [2023-12-27 18:26:15,928][20381] Updated weights for policy 1, policy_version 1844705 (0.0005) [2023-12-27 18:26:15,991][20381] Updated weights for policy 1, policy_version 1844715 (0.0006) [2023-12-27 18:26:16,044][20381] Updated weights for policy 1, policy_version 1844725 (0.0008) [2023-12-27 18:26:16,300][20347] Updated weights for policy 0, policy_version 1846266 (0.0005) [2023-12-27 18:26:16,362][20347] Updated weights for policy 0, policy_version 1846276 (0.0007) [2023-12-27 18:26:16,414][20347] Updated weights for policy 0, policy_version 1846286 (0.0006) [2023-12-27 18:26:16,470][20347] Updated weights for policy 0, policy_version 1846296 (0.0005) [2023-12-27 18:26:16,754][20381] Updated weights for policy 1, policy_version 1844735 (0.0010) [2023-12-27 18:26:16,802][20381] Updated weights for policy 1, policy_version 1844745 (0.0010) [2023-12-27 18:26:16,847][20381] Updated weights for policy 1, policy_version 1844755 (0.0010) [2023-12-27 18:26:17,094][20347] Updated weights for policy 0, policy_version 1846306 (0.0005) [2023-12-27 18:26:17,145][20347] Updated weights for policy 0, policy_version 1846316 (0.0005) [2023-12-27 18:26:17,194][20347] Updated weights for policy 0, policy_version 1846326 (0.0005) [2023-12-27 18:26:17,363][19170] Fps is (10 sec: 20479.7, 60 sec: 19251.2, 300 sec: 20244.0). Total num frames: 945053696. Throughput: 0: 9820.5, 1: 9737.1. Samples: 945062904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '867.940')] [2023-12-27 18:26:17,526][20381] Updated weights for policy 1, policy_version 1844765 (0.0008) [2023-12-27 18:26:17,591][20381] Updated weights for policy 1, policy_version 1844775 (0.0008) [2023-12-27 18:26:17,644][20381] Updated weights for policy 1, policy_version 1844785 (0.0011) [2023-12-27 18:26:17,748][20347] Updated weights for policy 0, policy_version 1846336 (0.0006) [2023-12-27 18:26:17,805][20347] Updated weights for policy 0, policy_version 1846346 (0.0007) [2023-12-27 18:26:17,859][20347] Updated weights for policy 0, policy_version 1846356 (0.0007) [2023-12-27 18:26:18,326][20381] Updated weights for policy 1, policy_version 1844795 (0.0011) [2023-12-27 18:26:18,387][20381] Updated weights for policy 1, policy_version 1844805 (0.0008) [2023-12-27 18:26:18,443][20381] Updated weights for policy 1, policy_version 1844815 (0.0007) [2023-12-27 18:26:18,544][20347] Updated weights for policy 0, policy_version 1846366 (0.0008) [2023-12-27 18:26:18,597][20347] Updated weights for policy 0, policy_version 1846376 (0.0008) [2023-12-27 18:26:18,658][20347] Updated weights for policy 0, policy_version 1846386 (0.0006) [2023-12-27 18:26:19,144][20381] Updated weights for policy 1, policy_version 1844825 (0.0007) [2023-12-27 18:26:19,204][20381] Updated weights for policy 1, policy_version 1844835 (0.0011) [2023-12-27 18:26:19,269][20381] Updated weights for policy 1, policy_version 1844845 (0.0011) [2023-12-27 18:26:19,308][20347] Updated weights for policy 0, policy_version 1846396 (0.0006) [2023-12-27 18:26:19,329][20381] Updated weights for policy 1, policy_version 1844855 (0.0011) [2023-12-27 18:26:19,371][20347] Updated weights for policy 0, policy_version 1846406 (0.0007) [2023-12-27 18:26:19,433][20347] Updated weights for policy 0, policy_version 1846416 (0.0007) [2023-12-27 18:26:20,060][20381] Updated weights for policy 1, policy_version 1844865 (0.0010) [2023-12-27 18:26:20,110][20381] Updated weights for policy 1, policy_version 1844875 (0.0011) [2023-12-27 18:26:20,143][20347] Updated weights for policy 0, policy_version 1846426 (0.0007) [2023-12-27 18:26:20,164][20381] Updated weights for policy 1, policy_version 1844885 (0.0011) [2023-12-27 18:26:20,201][20347] Updated weights for policy 0, policy_version 1846436 (0.0006) [2023-12-27 18:26:20,254][20347] Updated weights for policy 0, policy_version 1846446 (0.0008) [2023-12-27 18:26:20,319][20347] Updated weights for policy 0, policy_version 1846456 (0.0006) [2023-12-27 18:26:20,908][20381] Updated weights for policy 1, policy_version 1844895 (0.0007) [2023-12-27 18:26:20,967][20381] Updated weights for policy 1, policy_version 1844905 (0.0006) [2023-12-27 18:26:21,016][20347] Updated weights for policy 0, policy_version 1846466 (0.0008) [2023-12-27 18:26:21,030][20381] Updated weights for policy 1, policy_version 1844915 (0.0008) [2023-12-27 18:26:21,083][20347] Updated weights for policy 0, policy_version 1846476 (0.0007) [2023-12-27 18:26:21,148][20347] Updated weights for policy 0, policy_version 1846486 (0.0007) [2023-12-27 18:26:21,799][20381] Updated weights for policy 1, policy_version 1844925 (0.0007) [2023-12-27 18:26:21,860][20381] Updated weights for policy 1, policy_version 1844935 (0.0007) [2023-12-27 18:26:21,896][20347] Updated weights for policy 0, policy_version 1846496 (0.0008) [2023-12-27 18:26:21,918][20381] Updated weights for policy 1, policy_version 1844945 (0.0008) [2023-12-27 18:26:21,951][20347] Updated weights for policy 0, policy_version 1846506 (0.0006) [2023-12-27 18:26:21,999][20347] Updated weights for policy 0, policy_version 1846516 (0.0007) [2023-12-27 18:26:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 19251.2, 300 sec: 20188.4). Total num frames: 945152000. Throughput: 0: 9908.8, 1: 9795.8. Samples: 945124372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '906.570')] [2023-12-27 18:26:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001844952_472358912.pth... [2023-12-27 18:26:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001846520_472793088.pth... [2023-12-27 18:26:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001843768_472055808.pth [2023-12-27 18:26:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001845304_472481792.pth [2023-12-27 18:26:22,565][20381] Updated weights for policy 1, policy_version 1844955 (0.0007) [2023-12-27 18:26:22,623][20381] Updated weights for policy 1, policy_version 1844965 (0.0007) [2023-12-27 18:26:22,679][20381] Updated weights for policy 1, policy_version 1844975 (0.0006) [2023-12-27 18:26:22,783][20347] Updated weights for policy 0, policy_version 1846526 (0.0008) [2023-12-27 18:26:22,841][20347] Updated weights for policy 0, policy_version 1846536 (0.0008) [2023-12-27 18:26:22,893][20347] Updated weights for policy 0, policy_version 1846546 (0.0008) [2023-12-27 18:26:23,325][20381] Updated weights for policy 1, policy_version 1844985 (0.0005) [2023-12-27 18:26:23,375][20381] Updated weights for policy 1, policy_version 1844995 (0.0008) [2023-12-27 18:26:23,429][20381] Updated weights for policy 1, policy_version 1845005 (0.0008) [2023-12-27 18:26:23,481][20381] Updated weights for policy 1, policy_version 1845015 (0.0008) [2023-12-27 18:26:23,600][20347] Updated weights for policy 0, policy_version 1846556 (0.0007) [2023-12-27 18:26:23,645][20347] Updated weights for policy 0, policy_version 1846566 (0.0005) [2023-12-27 18:26:23,692][20347] Updated weights for policy 0, policy_version 1846576 (0.0005) [2023-12-27 18:26:24,227][20381] Updated weights for policy 1, policy_version 1845025 (0.0008) [2023-12-27 18:26:24,232][20347] Updated weights for policy 0, policy_version 1846586 (0.0005) [2023-12-27 18:26:24,280][20347] Updated weights for policy 0, policy_version 1846596 (0.0005) [2023-12-27 18:26:24,282][20381] Updated weights for policy 1, policy_version 1845035 (0.0006) [2023-12-27 18:26:24,337][20347] Updated weights for policy 0, policy_version 1846606 (0.0006) [2023-12-27 18:26:24,337][20381] Updated weights for policy 1, policy_version 1845045 (0.0008) [2023-12-27 18:26:24,397][20347] Updated weights for policy 0, policy_version 1846616 (0.0010) [2023-12-27 18:26:25,001][20347] Updated weights for policy 0, policy_version 1846626 (0.0010) [2023-12-27 18:26:25,047][20381] Updated weights for policy 1, policy_version 1845055 (0.0008) [2023-12-27 18:26:25,067][20347] Updated weights for policy 0, policy_version 1846636 (0.0010) [2023-12-27 18:26:25,098][20381] Updated weights for policy 1, policy_version 1845065 (0.0009) [2023-12-27 18:26:25,116][20347] Updated weights for policy 0, policy_version 1846646 (0.0009) [2023-12-27 18:26:25,149][20381] Updated weights for policy 1, policy_version 1845075 (0.0007) [2023-12-27 18:26:25,796][20347] Updated weights for policy 0, policy_version 1846656 (0.0010) [2023-12-27 18:26:25,860][20381] Updated weights for policy 1, policy_version 1845085 (0.0008) [2023-12-27 18:26:25,860][20347] Updated weights for policy 0, policy_version 1846666 (0.0009) [2023-12-27 18:26:25,922][20347] Updated weights for policy 0, policy_version 1846676 (0.0011) [2023-12-27 18:26:25,929][20381] Updated weights for policy 1, policy_version 1845095 (0.0008) [2023-12-27 18:26:25,980][20381] Updated weights for policy 1, policy_version 1845105 (0.0008) [2023-12-27 18:26:26,624][20347] Updated weights for policy 0, policy_version 1846686 (0.0009) [2023-12-27 18:26:26,636][20381] Updated weights for policy 1, policy_version 1845115 (0.0007) [2023-12-27 18:26:26,679][20347] Updated weights for policy 0, policy_version 1846696 (0.0007) [2023-12-27 18:26:26,687][20381] Updated weights for policy 1, policy_version 1845125 (0.0005) [2023-12-27 18:26:26,736][20381] Updated weights for policy 1, policy_version 1845135 (0.0006) [2023-12-27 18:26:26,737][20347] Updated weights for policy 0, policy_version 1846706 (0.0006) [2023-12-27 18:26:27,293][20347] Updated weights for policy 0, policy_version 1846716 (0.0007) [2023-12-27 18:26:27,348][20347] Updated weights for policy 0, policy_version 1846726 (0.0006) [2023-12-27 18:26:27,357][20381] Updated weights for policy 1, policy_version 1845145 (0.0008) [2023-12-27 18:26:27,362][19170] Fps is (10 sec: 19661.1, 60 sec: 19524.3, 300 sec: 20188.4). Total num frames: 945250304. Throughput: 0: 10027.6, 1: 9883.7. Samples: 945244132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '895.010')] [2023-12-27 18:26:27,408][20347] Updated weights for policy 0, policy_version 1846736 (0.0007) [2023-12-27 18:26:27,413][20381] Updated weights for policy 1, policy_version 1845155 (0.0010) [2023-12-27 18:26:27,467][20381] Updated weights for policy 1, policy_version 1845165 (0.0010) [2023-12-27 18:26:27,520][20381] Updated weights for policy 1, policy_version 1845175 (0.0010) [2023-12-27 18:26:28,046][20347] Updated weights for policy 0, policy_version 1846746 (0.0006) [2023-12-27 18:26:28,103][20347] Updated weights for policy 0, policy_version 1846756 (0.0011) [2023-12-27 18:26:28,127][20381] Updated weights for policy 1, policy_version 1845185 (0.0006) [2023-12-27 18:26:28,165][20347] Updated weights for policy 0, policy_version 1846766 (0.0010) [2023-12-27 18:26:28,188][20381] Updated weights for policy 1, policy_version 1845195 (0.0011) [2023-12-27 18:26:28,220][20347] Updated weights for policy 0, policy_version 1846776 (0.0010) [2023-12-27 18:26:28,253][20381] Updated weights for policy 1, policy_version 1845205 (0.0011) [2023-12-27 18:26:28,839][20381] Updated weights for policy 1, policy_version 1845215 (0.0007) [2023-12-27 18:26:28,904][20381] Updated weights for policy 1, policy_version 1845225 (0.0006) [2023-12-27 18:26:28,919][20347] Updated weights for policy 0, policy_version 1846786 (0.0010) [2023-12-27 18:26:28,968][20381] Updated weights for policy 1, policy_version 1845235 (0.0005) [2023-12-27 18:26:28,976][20347] Updated weights for policy 0, policy_version 1846796 (0.0007) [2023-12-27 18:26:29,038][20347] Updated weights for policy 0, policy_version 1846806 (0.0005) [2023-12-27 18:26:29,508][20381] Updated weights for policy 1, policy_version 1845245 (0.0005) [2023-12-27 18:26:29,562][20381] Updated weights for policy 1, policy_version 1845255 (0.0005) [2023-12-27 18:26:29,621][20381] Updated weights for policy 1, policy_version 1845265 (0.0007) [2023-12-27 18:26:29,661][20347] Updated weights for policy 0, policy_version 1846816 (0.0010) [2023-12-27 18:26:29,707][20347] Updated weights for policy 0, policy_version 1846826 (0.0006) [2023-12-27 18:26:29,761][20347] Updated weights for policy 0, policy_version 1846836 (0.0006) [2023-12-27 18:26:30,225][20381] Updated weights for policy 1, policy_version 1845275 (0.0007) [2023-12-27 18:26:30,278][20381] Updated weights for policy 1, policy_version 1845285 (0.0005) [2023-12-27 18:26:30,333][20381] Updated weights for policy 1, policy_version 1845295 (0.0005) [2023-12-27 18:26:30,436][20347] Updated weights for policy 0, policy_version 1846846 (0.0007) [2023-12-27 18:26:30,491][20347] Updated weights for policy 0, policy_version 1846856 (0.0005) [2023-12-27 18:26:30,541][20347] Updated weights for policy 0, policy_version 1846866 (0.0005) [2023-12-27 18:26:30,926][20381] Updated weights for policy 1, policy_version 1845305 (0.0006) [2023-12-27 18:26:30,974][20381] Updated weights for policy 1, policy_version 1845315 (0.0005) [2023-12-27 18:26:31,030][20381] Updated weights for policy 1, policy_version 1845325 (0.0006) [2023-12-27 18:26:31,094][20381] Updated weights for policy 1, policy_version 1845335 (0.0007) [2023-12-27 18:26:31,190][20347] Updated weights for policy 0, policy_version 1846876 (0.0006) [2023-12-27 18:26:31,258][20347] Updated weights for policy 0, policy_version 1846886 (0.0008) [2023-12-27 18:26:31,314][20347] Updated weights for policy 0, policy_version 1846896 (0.0007) [2023-12-27 18:26:31,820][20381] Updated weights for policy 1, policy_version 1845345 (0.0007) [2023-12-27 18:26:31,879][20381] Updated weights for policy 1, policy_version 1845355 (0.0007) [2023-12-27 18:26:31,938][20381] Updated weights for policy 1, policy_version 1845365 (0.0008) [2023-12-27 18:26:32,086][20347] Updated weights for policy 0, policy_version 1846906 (0.0008) [2023-12-27 18:26:32,144][20347] Updated weights for policy 0, policy_version 1846916 (0.0008) [2023-12-27 18:26:32,206][20347] Updated weights for policy 0, policy_version 1846926 (0.0006) [2023-12-27 18:26:32,271][20347] Updated weights for policy 0, policy_version 1846936 (0.0006) [2023-12-27 18:26:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 19797.3, 300 sec: 20216.2). Total num frames: 945364992. Throughput: 0: 10202.9, 1: 10071.3. Samples: 945373464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '906.690')] [2023-12-27 18:26:32,650][20381] Updated weights for policy 1, policy_version 1845375 (0.0008) [2023-12-27 18:26:32,706][20381] Updated weights for policy 1, policy_version 1845385 (0.0008) [2023-12-27 18:26:32,762][20381] Updated weights for policy 1, policy_version 1845395 (0.0007) [2023-12-27 18:26:32,896][20347] Updated weights for policy 0, policy_version 1846946 (0.0007) [2023-12-27 18:26:32,949][20347] Updated weights for policy 0, policy_version 1846956 (0.0007) [2023-12-27 18:26:33,000][20347] Updated weights for policy 0, policy_version 1846966 (0.0005) [2023-12-27 18:26:33,417][20381] Updated weights for policy 1, policy_version 1845405 (0.0008) [2023-12-27 18:26:33,466][20381] Updated weights for policy 1, policy_version 1845415 (0.0007) [2023-12-27 18:26:33,521][20381] Updated weights for policy 1, policy_version 1845425 (0.0005) [2023-12-27 18:26:33,638][20347] Updated weights for policy 0, policy_version 1846976 (0.0005) [2023-12-27 18:26:33,701][20347] Updated weights for policy 0, policy_version 1846986 (0.0005) [2023-12-27 18:26:33,763][20347] Updated weights for policy 0, policy_version 1846996 (0.0007) [2023-12-27 18:26:34,041][20381] Updated weights for policy 1, policy_version 1845435 (0.0005) [2023-12-27 18:26:34,094][20381] Updated weights for policy 1, policy_version 1845445 (0.0008) [2023-12-27 18:26:34,146][20381] Updated weights for policy 1, policy_version 1845455 (0.0007) [2023-12-27 18:26:34,342][20347] Updated weights for policy 0, policy_version 1847006 (0.0006) [2023-12-27 18:26:34,401][20347] Updated weights for policy 0, policy_version 1847016 (0.0006) [2023-12-27 18:26:34,464][20347] Updated weights for policy 0, policy_version 1847026 (0.0007) [2023-12-27 18:26:34,850][20381] Updated weights for policy 1, policy_version 1845465 (0.0007) [2023-12-27 18:26:34,909][20381] Updated weights for policy 1, policy_version 1845475 (0.0005) [2023-12-27 18:26:34,956][20381] Updated weights for policy 1, policy_version 1845485 (0.0005) [2023-12-27 18:26:35,006][20381] Updated weights for policy 1, policy_version 1845495 (0.0005) [2023-12-27 18:26:35,129][20347] Updated weights for policy 0, policy_version 1847036 (0.0009) [2023-12-27 18:26:35,184][20347] Updated weights for policy 0, policy_version 1847046 (0.0010) [2023-12-27 18:26:35,235][20347] Updated weights for policy 0, policy_version 1847056 (0.0010) [2023-12-27 18:26:35,668][20381] Updated weights for policy 1, policy_version 1845505 (0.0007) [2023-12-27 18:26:35,722][20381] Updated weights for policy 1, policy_version 1845515 (0.0007) [2023-12-27 18:26:35,779][20381] Updated weights for policy 1, policy_version 1845525 (0.0007) [2023-12-27 18:26:35,906][20347] Updated weights for policy 0, policy_version 1847066 (0.0010) [2023-12-27 18:26:35,960][20347] Updated weights for policy 0, policy_version 1847076 (0.0010) [2023-12-27 18:26:36,012][20347] Updated weights for policy 0, policy_version 1847086 (0.0010) [2023-12-27 18:26:36,067][20347] Updated weights for policy 0, policy_version 1847096 (0.0010) [2023-12-27 18:26:36,453][20381] Updated weights for policy 1, policy_version 1845535 (0.0007) [2023-12-27 18:26:36,513][20381] Updated weights for policy 1, policy_version 1845545 (0.0008) [2023-12-27 18:26:36,576][20381] Updated weights for policy 1, policy_version 1845555 (0.0008) [2023-12-27 18:26:36,790][20347] Updated weights for policy 0, policy_version 1847106 (0.0008) [2023-12-27 18:26:36,852][20347] Updated weights for policy 0, policy_version 1847116 (0.0007) [2023-12-27 18:26:36,905][20347] Updated weights for policy 0, policy_version 1847126 (0.0005) [2023-12-27 18:26:37,314][20381] Updated weights for policy 1, policy_version 1845565 (0.0007) [2023-12-27 18:26:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20070.4, 300 sec: 20188.4). Total num frames: 945463296. Throughput: 0: 10363.4, 1: 10214.1. Samples: 945438112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '899.970')] [2023-12-27 18:26:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001847128_472948736.pth... [2023-12-27 18:26:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001845912_472637440.pth [2023-12-27 18:26:37,371][20381] Updated weights for policy 1, policy_version 1845575 (0.0007) [2023-12-27 18:26:37,430][20381] Updated weights for policy 1, policy_version 1845585 (0.0007) [2023-12-27 18:26:37,470][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001845592_472522752.pth... [2023-12-27 18:26:37,474][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001844376_472211456.pth [2023-12-27 18:26:37,498][20347] Updated weights for policy 0, policy_version 1847136 (0.0009) [2023-12-27 18:26:37,546][20347] Updated weights for policy 0, policy_version 1847146 (0.0010) [2023-12-27 18:26:37,598][20347] Updated weights for policy 0, policy_version 1847156 (0.0007) [2023-12-27 18:26:38,054][20381] Updated weights for policy 1, policy_version 1845595 (0.0007) [2023-12-27 18:26:38,099][20381] Updated weights for policy 1, policy_version 1845605 (0.0008) [2023-12-27 18:26:38,145][20381] Updated weights for policy 1, policy_version 1845615 (0.0010) [2023-12-27 18:26:38,209][20347] Updated weights for policy 0, policy_version 1847166 (0.0005) [2023-12-27 18:26:38,262][20347] Updated weights for policy 0, policy_version 1847176 (0.0005) [2023-12-27 18:26:38,312][20347] Updated weights for policy 0, policy_version 1847186 (0.0009) [2023-12-27 18:26:38,905][20381] Updated weights for policy 1, policy_version 1845625 (0.0009) [2023-12-27 18:26:38,969][20381] Updated weights for policy 1, policy_version 1845635 (0.0011) [2023-12-27 18:26:38,976][20347] Updated weights for policy 0, policy_version 1847196 (0.0009) [2023-12-27 18:26:39,019][20381] Updated weights for policy 1, policy_version 1845645 (0.0011) [2023-12-27 18:26:39,028][20347] Updated weights for policy 0, policy_version 1847206 (0.0007) [2023-12-27 18:26:39,079][20381] Updated weights for policy 1, policy_version 1845655 (0.0011) [2023-12-27 18:26:39,082][20347] Updated weights for policy 0, policy_version 1847216 (0.0010) [2023-12-27 18:26:39,785][20347] Updated weights for policy 0, policy_version 1847226 (0.0011) [2023-12-27 18:26:39,846][20381] Updated weights for policy 1, policy_version 1845665 (0.0010) [2023-12-27 18:26:39,848][20347] Updated weights for policy 0, policy_version 1847236 (0.0010) [2023-12-27 18:26:39,906][20381] Updated weights for policy 1, policy_version 1845675 (0.0009) [2023-12-27 18:26:39,907][20347] Updated weights for policy 0, policy_version 1847246 (0.0007) [2023-12-27 18:26:39,970][20381] Updated weights for policy 1, policy_version 1845685 (0.0009) [2023-12-27 18:26:39,970][20347] Updated weights for policy 0, policy_version 1847256 (0.0008) [2023-12-27 18:26:40,605][20381] Updated weights for policy 1, policy_version 1845695 (0.0006) [2023-12-27 18:26:40,655][20381] Updated weights for policy 1, policy_version 1845705 (0.0006) [2023-12-27 18:26:40,703][20381] Updated weights for policy 1, policy_version 1845715 (0.0007) [2023-12-27 18:26:40,720][20347] Updated weights for policy 0, policy_version 1847266 (0.0011) [2023-12-27 18:26:40,775][20347] Updated weights for policy 0, policy_version 1847276 (0.0010) [2023-12-27 18:26:40,824][20347] Updated weights for policy 0, policy_version 1847286 (0.0011) [2023-12-27 18:26:41,366][20381] Updated weights for policy 1, policy_version 1845725 (0.0008) [2023-12-27 18:26:41,426][20381] Updated weights for policy 1, policy_version 1845735 (0.0007) [2023-12-27 18:26:41,488][20381] Updated weights for policy 1, policy_version 1845745 (0.0008) [2023-12-27 18:26:41,611][20347] Updated weights for policy 0, policy_version 1847296 (0.0007) [2023-12-27 18:26:41,679][20347] Updated weights for policy 0, policy_version 1847306 (0.0007) [2023-12-27 18:26:41,743][20347] Updated weights for policy 0, policy_version 1847316 (0.0007) [2023-12-27 18:26:42,210][20381] Updated weights for policy 1, policy_version 1845755 (0.0007) [2023-12-27 18:26:42,262][20381] Updated weights for policy 1, policy_version 1845765 (0.0007) [2023-12-27 18:26:42,324][20381] Updated weights for policy 1, policy_version 1845775 (0.0006) [2023-12-27 18:26:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20070.4, 300 sec: 20188.4). Total num frames: 945561600. Throughput: 0: 10360.5, 1: 10219.7. Samples: 945559812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '904.850')] [2023-12-27 18:26:42,428][20347] Updated weights for policy 0, policy_version 1847326 (0.0008) [2023-12-27 18:26:42,487][20347] Updated weights for policy 0, policy_version 1847336 (0.0008) [2023-12-27 18:26:42,533][20347] Updated weights for policy 0, policy_version 1847346 (0.0010) [2023-12-27 18:26:43,000][20381] Updated weights for policy 1, policy_version 1845785 (0.0008) [2023-12-27 18:26:43,062][20381] Updated weights for policy 1, policy_version 1845795 (0.0006) [2023-12-27 18:26:43,117][20381] Updated weights for policy 1, policy_version 1845805 (0.0008) [2023-12-27 18:26:43,163][20381] Updated weights for policy 1, policy_version 1845815 (0.0007) [2023-12-27 18:26:43,237][20347] Updated weights for policy 0, policy_version 1847356 (0.0007) [2023-12-27 18:26:43,286][20347] Updated weights for policy 0, policy_version 1847366 (0.0007) [2023-12-27 18:26:43,332][20347] Updated weights for policy 0, policy_version 1847376 (0.0007) [2023-12-27 18:26:43,731][20381] Updated weights for policy 1, policy_version 1845825 (0.0006) [2023-12-27 18:26:43,787][20381] Updated weights for policy 1, policy_version 1845835 (0.0007) [2023-12-27 18:26:43,842][20381] Updated weights for policy 1, policy_version 1845845 (0.0008) [2023-12-27 18:26:44,093][20347] Updated weights for policy 0, policy_version 1847386 (0.0007) [2023-12-27 18:26:44,148][20347] Updated weights for policy 0, policy_version 1847396 (0.0007) [2023-12-27 18:26:44,204][20347] Updated weights for policy 0, policy_version 1847406 (0.0008) [2023-12-27 18:26:44,260][20347] Updated weights for policy 0, policy_version 1847416 (0.0007) [2023-12-27 18:26:44,490][20381] Updated weights for policy 1, policy_version 1845855 (0.0007) [2023-12-27 18:26:44,534][20381] Updated weights for policy 1, policy_version 1845865 (0.0007) [2023-12-27 18:26:44,582][20381] Updated weights for policy 1, policy_version 1845875 (0.0007) [2023-12-27 18:26:44,958][20347] Updated weights for policy 0, policy_version 1847426 (0.0010) [2023-12-27 18:26:45,027][20347] Updated weights for policy 0, policy_version 1847436 (0.0009) [2023-12-27 18:26:45,095][20347] Updated weights for policy 0, policy_version 1847446 (0.0006) [2023-12-27 18:26:45,311][20381] Updated weights for policy 1, policy_version 1845885 (0.0008) [2023-12-27 18:26:45,365][20381] Updated weights for policy 1, policy_version 1845895 (0.0008) [2023-12-27 18:26:45,429][20381] Updated weights for policy 1, policy_version 1845905 (0.0008) [2023-12-27 18:26:45,744][20347] Updated weights for policy 0, policy_version 1847456 (0.0010) [2023-12-27 18:26:45,799][20347] Updated weights for policy 0, policy_version 1847466 (0.0010) [2023-12-27 18:26:45,849][20347] Updated weights for policy 0, policy_version 1847476 (0.0010) [2023-12-27 18:26:46,136][20381] Updated weights for policy 1, policy_version 1845915 (0.0008) [2023-12-27 18:26:46,182][20381] Updated weights for policy 1, policy_version 1845925 (0.0007) [2023-12-27 18:26:46,228][20381] Updated weights for policy 1, policy_version 1845935 (0.0010) [2023-12-27 18:26:46,427][20347] Updated weights for policy 0, policy_version 1847486 (0.0008) [2023-12-27 18:26:46,484][20347] Updated weights for policy 0, policy_version 1847496 (0.0008) [2023-12-27 18:26:46,539][20347] Updated weights for policy 0, policy_version 1847506 (0.0007) [2023-12-27 18:26:46,925][20381] Updated weights for policy 1, policy_version 1845945 (0.0009) [2023-12-27 18:26:46,980][20381] Updated weights for policy 1, policy_version 1845955 (0.0007) [2023-12-27 18:26:47,035][20381] Updated weights for policy 1, policy_version 1845965 (0.0007) [2023-12-27 18:26:47,090][20381] Updated weights for policy 1, policy_version 1845975 (0.0007) [2023-12-27 18:26:47,152][20347] Updated weights for policy 0, policy_version 1847516 (0.0007) [2023-12-27 18:26:47,206][20347] Updated weights for policy 0, policy_version 1847526 (0.0006) [2023-12-27 18:26:47,261][20347] Updated weights for policy 0, policy_version 1847536 (0.0007) [2023-12-27 18:26:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20244.0). Total num frames: 945676288. Throughput: 0: 10342.1, 1: 10257.8. Samples: 945682236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 18:26:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '947.900')] [2023-12-27 18:26:47,811][20381] Updated weights for policy 1, policy_version 1845985 (0.0008) [2023-12-27 18:26:47,869][20381] Updated weights for policy 1, policy_version 1845995 (0.0008) [2023-12-27 18:26:47,927][20381] Updated weights for policy 1, policy_version 1846005 (0.0007) [2023-12-27 18:26:47,929][20347] Updated weights for policy 0, policy_version 1847546 (0.0007) [2023-12-27 18:26:47,977][20347] Updated weights for policy 0, policy_version 1847556 (0.0007) [2023-12-27 18:26:48,040][20347] Updated weights for policy 0, policy_version 1847566 (0.0007) [2023-12-27 18:26:48,093][20347] Updated weights for policy 0, policy_version 1847576 (0.0008) [2023-12-27 18:26:48,589][20381] Updated weights for policy 1, policy_version 1846015 (0.0009) [2023-12-27 18:26:48,650][20381] Updated weights for policy 1, policy_version 1846025 (0.0011) [2023-12-27 18:26:48,710][20381] Updated weights for policy 1, policy_version 1846035 (0.0007) [2023-12-27 18:26:48,786][20347] Updated weights for policy 0, policy_version 1847586 (0.0005) [2023-12-27 18:26:48,849][20347] Updated weights for policy 0, policy_version 1847596 (0.0005) [2023-12-27 18:26:48,910][20347] Updated weights for policy 0, policy_version 1847606 (0.0006) [2023-12-27 18:26:49,335][20381] Updated weights for policy 1, policy_version 1846045 (0.0011) [2023-12-27 18:26:49,397][20381] Updated weights for policy 1, policy_version 1846055 (0.0009) [2023-12-27 18:26:49,451][20381] Updated weights for policy 1, policy_version 1846065 (0.0009) [2023-12-27 18:26:49,537][20347] Updated weights for policy 0, policy_version 1847616 (0.0006) [2023-12-27 18:26:49,597][20347] Updated weights for policy 0, policy_version 1847626 (0.0007) [2023-12-27 18:26:49,655][20347] Updated weights for policy 0, policy_version 1847636 (0.0007) [2023-12-27 18:26:50,206][20381] Updated weights for policy 1, policy_version 1846075 (0.0009) [2023-12-27 18:26:50,260][20381] Updated weights for policy 1, policy_version 1846085 (0.0010) [2023-12-27 18:26:50,316][20381] Updated weights for policy 1, policy_version 1846095 (0.0011) [2023-12-27 18:26:50,327][20347] Updated weights for policy 0, policy_version 1847646 (0.0007) [2023-12-27 18:26:50,385][20347] Updated weights for policy 0, policy_version 1847656 (0.0009) [2023-12-27 18:26:50,440][20347] Updated weights for policy 0, policy_version 1847666 (0.0007) [2023-12-27 18:26:50,985][20381] Updated weights for policy 1, policy_version 1846105 (0.0011) [2023-12-27 18:26:51,046][20381] Updated weights for policy 1, policy_version 1846115 (0.0011) [2023-12-27 18:26:51,107][20381] Updated weights for policy 1, policy_version 1846125 (0.0011) [2023-12-27 18:26:51,170][20381] Updated weights for policy 1, policy_version 1846135 (0.0008) [2023-12-27 18:26:51,173][20347] Updated weights for policy 0, policy_version 1847676 (0.0007) [2023-12-27 18:26:51,229][20347] Updated weights for policy 0, policy_version 1847686 (0.0008) [2023-12-27 18:26:51,291][20347] Updated weights for policy 0, policy_version 1847696 (0.0006) [2023-12-27 18:26:51,860][20381] Updated weights for policy 1, policy_version 1846145 (0.0007) [2023-12-27 18:26:51,921][20381] Updated weights for policy 1, policy_version 1846155 (0.0008) [2023-12-27 18:26:51,929][20347] Updated weights for policy 0, policy_version 1847706 (0.0007) [2023-12-27 18:26:51,981][20381] Updated weights for policy 1, policy_version 1846165 (0.0009) [2023-12-27 18:26:51,983][20347] Updated weights for policy 0, policy_version 1847716 (0.0005) [2023-12-27 18:26:52,038][20347] Updated weights for policy 0, policy_version 1847726 (0.0006) [2023-12-27 18:26:52,102][20347] Updated weights for policy 0, policy_version 1847736 (0.0006) [2023-12-27 18:26:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20216.2). Total num frames: 945774592. Throughput: 0: 10347.3, 1: 10239.4. Samples: 945743640. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:26:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '899.580')] [2023-12-27 18:26:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001847736_473104384.pth... [2023-12-27 18:26:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001846168_472670208.pth... [2023-12-27 18:26:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001846520_472793088.pth [2023-12-27 18:26:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001844952_472358912.pth [2023-12-27 18:26:52,689][20381] Updated weights for policy 1, policy_version 1846175 (0.0007) [2023-12-27 18:26:52,740][20381] Updated weights for policy 1, policy_version 1846185 (0.0006) [2023-12-27 18:26:52,745][20347] Updated weights for policy 0, policy_version 1847746 (0.0007) [2023-12-27 18:26:52,790][20381] Updated weights for policy 1, policy_version 1846195 (0.0009) [2023-12-27 18:26:52,799][20347] Updated weights for policy 0, policy_version 1847756 (0.0008) [2023-12-27 18:26:52,858][20347] Updated weights for policy 0, policy_version 1847766 (0.0006) [2023-12-27 18:26:53,476][20347] Updated weights for policy 0, policy_version 1847776 (0.0005) [2023-12-27 18:26:53,481][20381] Updated weights for policy 1, policy_version 1846205 (0.0006) [2023-12-27 18:26:53,523][20347] Updated weights for policy 0, policy_version 1847786 (0.0010) [2023-12-27 18:26:53,539][20381] Updated weights for policy 1, policy_version 1846215 (0.0007) [2023-12-27 18:26:53,578][20347] Updated weights for policy 0, policy_version 1847796 (0.0010) [2023-12-27 18:26:53,598][20381] Updated weights for policy 1, policy_version 1846225 (0.0006) [2023-12-27 18:26:54,175][20347] Updated weights for policy 0, policy_version 1847806 (0.0007) [2023-12-27 18:26:54,229][20347] Updated weights for policy 0, policy_version 1847816 (0.0005) [2023-12-27 18:26:54,275][20381] Updated weights for policy 1, policy_version 1846235 (0.0007) [2023-12-27 18:26:54,292][20347] Updated weights for policy 0, policy_version 1847826 (0.0005) [2023-12-27 18:26:54,328][20381] Updated weights for policy 1, policy_version 1846245 (0.0007) [2023-12-27 18:26:54,384][20381] Updated weights for policy 1, policy_version 1846255 (0.0007) [2023-12-27 18:26:54,872][20347] Updated weights for policy 0, policy_version 1847836 (0.0005) [2023-12-27 18:26:54,931][20347] Updated weights for policy 0, policy_version 1847846 (0.0007) [2023-12-27 18:26:54,994][20347] Updated weights for policy 0, policy_version 1847856 (0.0007) [2023-12-27 18:26:55,121][20381] Updated weights for policy 1, policy_version 1846265 (0.0008) [2023-12-27 18:26:55,177][20381] Updated weights for policy 1, policy_version 1846275 (0.0008) [2023-12-27 18:26:55,229][20381] Updated weights for policy 1, policy_version 1846285 (0.0008) [2023-12-27 18:26:55,283][20381] Updated weights for policy 1, policy_version 1846295 (0.0007) [2023-12-27 18:26:55,590][20347] Updated weights for policy 0, policy_version 1847866 (0.0008) [2023-12-27 18:26:55,652][20347] Updated weights for policy 0, policy_version 1847876 (0.0008) [2023-12-27 18:26:55,703][20347] Updated weights for policy 0, policy_version 1847886 (0.0006) [2023-12-27 18:26:55,752][20347] Updated weights for policy 0, policy_version 1847896 (0.0005) [2023-12-27 18:26:55,944][20381] Updated weights for policy 1, policy_version 1846305 (0.0006) [2023-12-27 18:26:55,992][20381] Updated weights for policy 1, policy_version 1846315 (0.0005) [2023-12-27 18:26:56,035][20381] Updated weights for policy 1, policy_version 1846325 (0.0005) [2023-12-27 18:26:56,418][20347] Updated weights for policy 0, policy_version 1847906 (0.0007) [2023-12-27 18:26:56,475][20347] Updated weights for policy 0, policy_version 1847916 (0.0008) [2023-12-27 18:26:56,534][20347] Updated weights for policy 0, policy_version 1847926 (0.0008) [2023-12-27 18:26:56,706][20381] Updated weights for policy 1, policy_version 1846335 (0.0007) [2023-12-27 18:26:56,757][20381] Updated weights for policy 1, policy_version 1846345 (0.0007) [2023-12-27 18:26:56,812][20381] Updated weights for policy 1, policy_version 1846355 (0.0008) [2023-12-27 18:26:57,213][20347] Updated weights for policy 0, policy_version 1847936 (0.0008) [2023-12-27 18:26:57,267][20347] Updated weights for policy 0, policy_version 1847946 (0.0007) [2023-12-27 18:26:57,310][20347] Updated weights for policy 0, policy_version 1847956 (0.0008) [2023-12-27 18:26:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20243.9). Total num frames: 945881088. Throughput: 0: 10421.8, 1: 10254.8. Samples: 945868420. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:26:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.790')] [2023-12-27 18:26:57,479][20381] Updated weights for policy 1, policy_version 1846365 (0.0006) [2023-12-27 18:26:57,533][20381] Updated weights for policy 1, policy_version 1846375 (0.0007) [2023-12-27 18:26:57,585][20381] Updated weights for policy 1, policy_version 1846385 (0.0010) [2023-12-27 18:26:58,012][20347] Updated weights for policy 0, policy_version 1847966 (0.0009) [2023-12-27 18:26:58,070][20347] Updated weights for policy 0, policy_version 1847976 (0.0007) [2023-12-27 18:26:58,127][20347] Updated weights for policy 0, policy_version 1847986 (0.0009) [2023-12-27 18:26:58,210][20381] Updated weights for policy 1, policy_version 1846395 (0.0010) [2023-12-27 18:26:58,266][20381] Updated weights for policy 1, policy_version 1846405 (0.0008) [2023-12-27 18:26:58,324][20381] Updated weights for policy 1, policy_version 1846415 (0.0008) [2023-12-27 18:26:58,850][20347] Updated weights for policy 0, policy_version 1847996 (0.0009) [2023-12-27 18:26:58,923][20347] Updated weights for policy 0, policy_version 1848006 (0.0008) [2023-12-27 18:26:58,986][20347] Updated weights for policy 0, policy_version 1848016 (0.0006) [2023-12-27 18:26:59,099][20381] Updated weights for policy 1, policy_version 1846425 (0.0008) [2023-12-27 18:26:59,152][20381] Updated weights for policy 1, policy_version 1846435 (0.0007) [2023-12-27 18:26:59,211][20381] Updated weights for policy 1, policy_version 1846445 (0.0007) [2023-12-27 18:26:59,282][20381] Updated weights for policy 1, policy_version 1846455 (0.0009) [2023-12-27 18:26:59,646][20347] Updated weights for policy 0, policy_version 1848026 (0.0007) [2023-12-27 18:26:59,704][20347] Updated weights for policy 0, policy_version 1848036 (0.0006) [2023-12-27 18:26:59,767][20347] Updated weights for policy 0, policy_version 1848046 (0.0007) [2023-12-27 18:26:59,824][20347] Updated weights for policy 0, policy_version 1848056 (0.0007) [2023-12-27 18:26:59,971][20381] Updated weights for policy 1, policy_version 1846465 (0.0009) [2023-12-27 18:27:00,036][20381] Updated weights for policy 1, policy_version 1846475 (0.0008) [2023-12-27 18:27:00,100][20381] Updated weights for policy 1, policy_version 1846485 (0.0008) [2023-12-27 18:27:00,484][20347] Updated weights for policy 0, policy_version 1848066 (0.0008) [2023-12-27 18:27:00,541][20347] Updated weights for policy 0, policy_version 1848076 (0.0007) [2023-12-27 18:27:00,593][20347] Updated weights for policy 0, policy_version 1848086 (0.0005) [2023-12-27 18:27:00,721][20381] Updated weights for policy 1, policy_version 1846495 (0.0007) [2023-12-27 18:27:00,773][20381] Updated weights for policy 1, policy_version 1846505 (0.0008) [2023-12-27 18:27:00,816][20381] Updated weights for policy 1, policy_version 1846515 (0.0008) [2023-12-27 18:27:00,838][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000005 [2023-12-27 18:27:01,300][20347] Updated weights for policy 0, policy_version 1848096 (0.0008) [2023-12-27 18:27:01,371][20347] Updated weights for policy 0, policy_version 1848106 (0.0008) [2023-12-27 18:27:01,432][20347] Updated weights for policy 0, policy_version 1848116 (0.0007) [2023-12-27 18:27:01,527][20381] Updated weights for policy 1, policy_version 1846525 (0.0009) [2023-12-27 18:27:01,580][20381] Updated weights for policy 1, policy_version 1846535 (0.0009) [2023-12-27 18:27:01,645][20381] Updated weights for policy 1, policy_version 1846545 (0.0007) [2023-12-27 18:27:02,152][20347] Updated weights for policy 0, policy_version 1848126 (0.0008) [2023-12-27 18:27:02,199][20347] Updated weights for policy 0, policy_version 1848136 (0.0007) [2023-12-27 18:27:02,259][20347] Updated weights for policy 0, policy_version 1848146 (0.0005) [2023-12-27 18:27:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20216.2). Total num frames: 945979392. Throughput: 0: 10362.1, 1: 10223.8. Samples: 945989264. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:02,363][19170] Avg episode reward: [(0, '990.250'), (1, '881.670')] [2023-12-27 18:27:02,378][20381] Updated weights for policy 1, policy_version 1846555 (0.0007) [2023-12-27 18:27:02,438][20381] Updated weights for policy 1, policy_version 1846565 (0.0008) [2023-12-27 18:27:02,498][20381] Updated weights for policy 1, policy_version 1846575 (0.0007) [2023-12-27 18:27:02,950][20347] Updated weights for policy 0, policy_version 1848156 (0.0006) [2023-12-27 18:27:03,002][20347] Updated weights for policy 0, policy_version 1848166 (0.0008) [2023-12-27 18:27:03,050][20347] Updated weights for policy 0, policy_version 1848176 (0.0008) [2023-12-27 18:27:03,224][20381] Updated weights for policy 1, policy_version 1846585 (0.0008) [2023-12-27 18:27:03,269][20381] Updated weights for policy 1, policy_version 1846595 (0.0007) [2023-12-27 18:27:03,326][20381] Updated weights for policy 1, policy_version 1846605 (0.0007) [2023-12-27 18:27:03,380][20381] Updated weights for policy 1, policy_version 1846615 (0.0007) [2023-12-27 18:27:03,663][20347] Updated weights for policy 0, policy_version 1848186 (0.0006) [2023-12-27 18:27:03,713][20347] Updated weights for policy 0, policy_version 1848196 (0.0007) [2023-12-27 18:27:03,767][20347] Updated weights for policy 0, policy_version 1848206 (0.0007) [2023-12-27 18:27:03,811][20347] Updated weights for policy 0, policy_version 1848216 (0.0007) [2023-12-27 18:27:04,096][20381] Updated weights for policy 1, policy_version 1846625 (0.0008) [2023-12-27 18:27:04,154][20381] Updated weights for policy 1, policy_version 1846635 (0.0007) [2023-12-27 18:27:04,218][20381] Updated weights for policy 1, policy_version 1846645 (0.0008) [2023-12-27 18:27:04,536][20347] Updated weights for policy 0, policy_version 1848226 (0.0008) [2023-12-27 18:27:04,593][20347] Updated weights for policy 0, policy_version 1848236 (0.0007) [2023-12-27 18:27:04,644][20347] Updated weights for policy 0, policy_version 1848246 (0.0007) [2023-12-27 18:27:04,932][20381] Updated weights for policy 1, policy_version 1846655 (0.0008) [2023-12-27 18:27:04,985][20381] Updated weights for policy 1, policy_version 1846665 (0.0008) [2023-12-27 18:27:05,043][20381] Updated weights for policy 1, policy_version 1846675 (0.0007) [2023-12-27 18:27:05,268][20347] Updated weights for policy 0, policy_version 1848256 (0.0010) [2023-12-27 18:27:05,320][20347] Updated weights for policy 0, policy_version 1848266 (0.0010) [2023-12-27 18:27:05,379][20347] Updated weights for policy 0, policy_version 1848276 (0.0010) [2023-12-27 18:27:05,781][20381] Updated weights for policy 1, policy_version 1846685 (0.0007) [2023-12-27 18:27:05,827][20381] Updated weights for policy 1, policy_version 1846695 (0.0010) [2023-12-27 18:27:05,877][20381] Updated weights for policy 1, policy_version 1846705 (0.0008) [2023-12-27 18:27:05,971][20347] Updated weights for policy 0, policy_version 1848286 (0.0007) [2023-12-27 18:27:06,023][20347] Updated weights for policy 0, policy_version 1848296 (0.0009) [2023-12-27 18:27:06,084][20347] Updated weights for policy 0, policy_version 1848306 (0.0010) [2023-12-27 18:27:06,610][20381] Updated weights for policy 1, policy_version 1846715 (0.0007) [2023-12-27 18:27:06,669][20381] Updated weights for policy 1, policy_version 1846725 (0.0006) [2023-12-27 18:27:06,728][20381] Updated weights for policy 1, policy_version 1846735 (0.0007) [2023-12-27 18:27:06,780][20347] Updated weights for policy 0, policy_version 1848316 (0.0009) [2023-12-27 18:27:06,836][20347] Updated weights for policy 0, policy_version 1848326 (0.0011) [2023-12-27 18:27:06,882][20347] Updated weights for policy 0, policy_version 1848336 (0.0010) [2023-12-27 18:27:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 946077696. Throughput: 0: 10348.8, 1: 10206.8. Samples: 946049376. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:07,363][19170] Avg episode reward: [(0, '990.250'), (1, '910.380')] [2023-12-27 18:27:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001846744_472817664.pth... [2023-12-27 18:27:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001848344_473260032.pth... [2023-12-27 18:27:07,380][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001845592_472522752.pth [2023-12-27 18:27:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001847128_472948736.pth [2023-12-27 18:27:07,410][20381] Updated weights for policy 1, policy_version 1846745 (0.0007) [2023-12-27 18:27:07,477][20381] Updated weights for policy 1, policy_version 1846755 (0.0007) [2023-12-27 18:27:07,529][20381] Updated weights for policy 1, policy_version 1846765 (0.0008) [2023-12-27 18:27:07,577][20381] Updated weights for policy 1, policy_version 1846775 (0.0008) [2023-12-27 18:27:07,587][20347] Updated weights for policy 0, policy_version 1848346 (0.0011) [2023-12-27 18:27:07,642][20347] Updated weights for policy 0, policy_version 1848356 (0.0006) [2023-12-27 18:27:07,700][20347] Updated weights for policy 0, policy_version 1848366 (0.0007) [2023-12-27 18:27:07,758][20347] Updated weights for policy 0, policy_version 1848376 (0.0011) [2023-12-27 18:27:08,312][20381] Updated weights for policy 1, policy_version 1846785 (0.0007) [2023-12-27 18:27:08,368][20347] Updated weights for policy 0, policy_version 1848386 (0.0010) [2023-12-27 18:27:08,377][20381] Updated weights for policy 1, policy_version 1846795 (0.0008) [2023-12-27 18:27:08,426][20347] Updated weights for policy 0, policy_version 1848396 (0.0010) [2023-12-27 18:27:08,443][20381] Updated weights for policy 1, policy_version 1846805 (0.0006) [2023-12-27 18:27:08,481][20347] Updated weights for policy 0, policy_version 1848406 (0.0010) [2023-12-27 18:27:09,122][20347] Updated weights for policy 0, policy_version 1848416 (0.0006) [2023-12-27 18:27:09,169][20381] Updated weights for policy 1, policy_version 1846815 (0.0007) [2023-12-27 18:27:09,179][20347] Updated weights for policy 0, policy_version 1848426 (0.0007) [2023-12-27 18:27:09,235][20381] Updated weights for policy 1, policy_version 1846825 (0.0007) [2023-12-27 18:27:09,237][20347] Updated weights for policy 0, policy_version 1848436 (0.0009) [2023-12-27 18:27:09,300][20381] Updated weights for policy 1, policy_version 1846835 (0.0008) [2023-12-27 18:27:09,907][20347] Updated weights for policy 0, policy_version 1848446 (0.0010) [2023-12-27 18:27:09,968][20347] Updated weights for policy 0, policy_version 1848456 (0.0010) [2023-12-27 18:27:10,031][20347] Updated weights for policy 0, policy_version 1848466 (0.0010) [2023-12-27 18:27:10,074][20381] Updated weights for policy 1, policy_version 1846845 (0.0007) [2023-12-27 18:27:10,130][20381] Updated weights for policy 1, policy_version 1846855 (0.0007) [2023-12-27 18:27:10,193][20381] Updated weights for policy 1, policy_version 1846865 (0.0009) [2023-12-27 18:27:10,699][20347] Updated weights for policy 0, policy_version 1848476 (0.0008) [2023-12-27 18:27:10,748][20347] Updated weights for policy 0, policy_version 1848486 (0.0006) [2023-12-27 18:27:10,792][20347] Updated weights for policy 0, policy_version 1848496 (0.0007) [2023-12-27 18:27:10,899][20381] Updated weights for policy 1, policy_version 1846875 (0.0008) [2023-12-27 18:27:10,946][20381] Updated weights for policy 1, policy_version 1846885 (0.0007) [2023-12-27 18:27:10,994][20381] Updated weights for policy 1, policy_version 1846895 (0.0007) [2023-12-27 18:27:11,470][20347] Updated weights for policy 0, policy_version 1848506 (0.0008) [2023-12-27 18:27:11,529][20347] Updated weights for policy 0, policy_version 1848516 (0.0010) [2023-12-27 18:27:11,591][20347] Updated weights for policy 0, policy_version 1848526 (0.0010) [2023-12-27 18:27:11,657][20347] Updated weights for policy 0, policy_version 1848536 (0.0009) [2023-12-27 18:27:11,824][20381] Updated weights for policy 1, policy_version 1846905 (0.0008) [2023-12-27 18:27:11,880][20381] Updated weights for policy 1, policy_version 1846915 (0.0008) [2023-12-27 18:27:11,932][20381] Updated weights for policy 1, policy_version 1846925 (0.0010) [2023-12-27 18:27:11,979][20381] Updated weights for policy 1, policy_version 1846935 (0.0010) [2023-12-27 18:27:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20188.4). Total num frames: 946176000. Throughput: 0: 10401.1, 1: 10170.8. Samples: 946169872. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '920.830')] [2023-12-27 18:27:12,374][20347] Updated weights for policy 0, policy_version 1848546 (0.0007) [2023-12-27 18:27:12,433][20347] Updated weights for policy 0, policy_version 1848556 (0.0005) [2023-12-27 18:27:12,492][20347] Updated weights for policy 0, policy_version 1848566 (0.0007) [2023-12-27 18:27:12,708][20381] Updated weights for policy 1, policy_version 1846945 (0.0010) [2023-12-27 18:27:12,757][20381] Updated weights for policy 1, policy_version 1846955 (0.0011) [2023-12-27 18:27:12,807][20381] Updated weights for policy 1, policy_version 1846965 (0.0011) [2023-12-27 18:27:13,158][20347] Updated weights for policy 0, policy_version 1848576 (0.0009) [2023-12-27 18:27:13,209][20347] Updated weights for policy 0, policy_version 1848586 (0.0007) [2023-12-27 18:27:13,260][20347] Updated weights for policy 0, policy_version 1848596 (0.0006) [2023-12-27 18:27:13,454][20381] Updated weights for policy 1, policy_version 1846975 (0.0011) [2023-12-27 18:27:13,509][20381] Updated weights for policy 1, policy_version 1846985 (0.0011) [2023-12-27 18:27:13,567][20381] Updated weights for policy 1, policy_version 1846995 (0.0011) [2023-12-27 18:27:13,998][20347] Updated weights for policy 0, policy_version 1848606 (0.0007) [2023-12-27 18:27:14,058][20347] Updated weights for policy 0, policy_version 1848616 (0.0008) [2023-12-27 18:27:14,121][20347] Updated weights for policy 0, policy_version 1848626 (0.0008) [2023-12-27 18:27:14,167][20381] Updated weights for policy 1, policy_version 1847005 (0.0007) [2023-12-27 18:27:14,218][20381] Updated weights for policy 1, policy_version 1847015 (0.0008) [2023-12-27 18:27:14,263][20381] Updated weights for policy 1, policy_version 1847025 (0.0008) [2023-12-27 18:27:14,821][20347] Updated weights for policy 0, policy_version 1848636 (0.0007) [2023-12-27 18:27:14,886][20347] Updated weights for policy 0, policy_version 1848646 (0.0007) [2023-12-27 18:27:14,886][20381] Updated weights for policy 1, policy_version 1847035 (0.0008) [2023-12-27 18:27:14,948][20381] Updated weights for policy 1, policy_version 1847045 (0.0007) [2023-12-27 18:27:14,950][20347] Updated weights for policy 0, policy_version 1848656 (0.0006) [2023-12-27 18:27:14,998][20381] Updated weights for policy 1, policy_version 1847055 (0.0006) [2023-12-27 18:27:15,614][20347] Updated weights for policy 0, policy_version 1848666 (0.0009) [2023-12-27 18:27:15,682][20347] Updated weights for policy 0, policy_version 1848676 (0.0007) [2023-12-27 18:27:15,709][20381] Updated weights for policy 1, policy_version 1847065 (0.0008) [2023-12-27 18:27:15,737][20347] Updated weights for policy 0, policy_version 1848686 (0.0007) [2023-12-27 18:27:15,773][20381] Updated weights for policy 1, policy_version 1847075 (0.0006) [2023-12-27 18:27:15,785][20347] Updated weights for policy 0, policy_version 1848696 (0.0005) [2023-12-27 18:27:15,825][20381] Updated weights for policy 1, policy_version 1847085 (0.0005) [2023-12-27 18:27:15,881][20381] Updated weights for policy 1, policy_version 1847095 (0.0007) [2023-12-27 18:27:16,376][20347] Updated weights for policy 0, policy_version 1848706 (0.0008) [2023-12-27 18:27:16,433][20347] Updated weights for policy 0, policy_version 1848716 (0.0008) [2023-12-27 18:27:16,488][20347] Updated weights for policy 0, policy_version 1848726 (0.0007) [2023-12-27 18:27:16,557][20381] Updated weights for policy 1, policy_version 1847105 (0.0007) [2023-12-27 18:27:16,606][20381] Updated weights for policy 1, policy_version 1847115 (0.0008) [2023-12-27 18:27:16,661][20381] Updated weights for policy 1, policy_version 1847125 (0.0009) [2023-12-27 18:27:17,125][20347] Updated weights for policy 0, policy_version 1848736 (0.0008) [2023-12-27 18:27:17,183][20347] Updated weights for policy 0, policy_version 1848746 (0.0008) [2023-12-27 18:27:17,243][20347] Updated weights for policy 0, policy_version 1848756 (0.0005) [2023-12-27 18:27:17,296][20381] Updated weights for policy 1, policy_version 1847135 (0.0008) [2023-12-27 18:27:17,360][20381] Updated weights for policy 1, policy_version 1847145 (0.0008) [2023-12-27 18:27:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 946282496. Throughput: 0: 10353.1, 1: 10084.4. Samples: 946293152. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.240')] [2023-12-27 18:27:17,411][20381] Updated weights for policy 1, policy_version 1847155 (0.0009) [2023-12-27 18:27:17,865][20347] Updated weights for policy 0, policy_version 1848766 (0.0007) [2023-12-27 18:27:17,914][20347] Updated weights for policy 0, policy_version 1848776 (0.0010) [2023-12-27 18:27:17,968][20347] Updated weights for policy 0, policy_version 1848786 (0.0009) [2023-12-27 18:27:18,057][20381] Updated weights for policy 1, policy_version 1847165 (0.0007) [2023-12-27 18:27:18,106][20381] Updated weights for policy 1, policy_version 1847175 (0.0007) [2023-12-27 18:27:18,157][20381] Updated weights for policy 1, policy_version 1847185 (0.0008) [2023-12-27 18:27:18,651][20347] Updated weights for policy 0, policy_version 1848796 (0.0008) [2023-12-27 18:27:18,717][20347] Updated weights for policy 0, policy_version 1848806 (0.0008) [2023-12-27 18:27:18,780][20347] Updated weights for policy 0, policy_version 1848816 (0.0006) [2023-12-27 18:27:18,918][20381] Updated weights for policy 1, policy_version 1847195 (0.0008) [2023-12-27 18:27:18,969][20381] Updated weights for policy 1, policy_version 1847205 (0.0006) [2023-12-27 18:27:19,014][20381] Updated weights for policy 1, policy_version 1847215 (0.0005) [2023-12-27 18:27:19,413][20347] Updated weights for policy 0, policy_version 1848826 (0.0005) [2023-12-27 18:27:19,475][20347] Updated weights for policy 0, policy_version 1848836 (0.0006) [2023-12-27 18:27:19,537][20347] Updated weights for policy 0, policy_version 1848846 (0.0009) [2023-12-27 18:27:19,594][20347] Updated weights for policy 0, policy_version 1848856 (0.0009) [2023-12-27 18:27:19,700][20381] Updated weights for policy 1, policy_version 1847225 (0.0006) [2023-12-27 18:27:19,760][20381] Updated weights for policy 1, policy_version 1847235 (0.0009) [2023-12-27 18:27:19,828][20381] Updated weights for policy 1, policy_version 1847245 (0.0008) [2023-12-27 18:27:19,896][20381] Updated weights for policy 1, policy_version 1847255 (0.0007) [2023-12-27 18:27:20,360][20347] Updated weights for policy 0, policy_version 1848866 (0.0008) [2023-12-27 18:27:20,417][20347] Updated weights for policy 0, policy_version 1848876 (0.0008) [2023-12-27 18:27:20,479][20347] Updated weights for policy 0, policy_version 1848886 (0.0008) [2023-12-27 18:27:20,548][20381] Updated weights for policy 1, policy_version 1847265 (0.0006) [2023-12-27 18:27:20,609][20381] Updated weights for policy 1, policy_version 1847275 (0.0008) [2023-12-27 18:27:20,670][20381] Updated weights for policy 1, policy_version 1847285 (0.0006) [2023-12-27 18:27:21,262][20347] Updated weights for policy 0, policy_version 1848896 (0.0007) [2023-12-27 18:27:21,329][20347] Updated weights for policy 0, policy_version 1848906 (0.0007) [2023-12-27 18:27:21,350][20381] Updated weights for policy 1, policy_version 1847295 (0.0006) [2023-12-27 18:27:21,405][20347] Updated weights for policy 0, policy_version 1848916 (0.0008) [2023-12-27 18:27:21,423][20381] Updated weights for policy 1, policy_version 1847305 (0.0007) [2023-12-27 18:27:21,485][20381] Updated weights for policy 1, policy_version 1847315 (0.0006) [2023-12-27 18:27:22,083][20347] Updated weights for policy 0, policy_version 1848926 (0.0007) [2023-12-27 18:27:22,149][20347] Updated weights for policy 0, policy_version 1848936 (0.0006) [2023-12-27 18:27:22,158][20381] Updated weights for policy 1, policy_version 1847325 (0.0005) [2023-12-27 18:27:22,214][20347] Updated weights for policy 0, policy_version 1848946 (0.0008) [2023-12-27 18:27:22,223][20381] Updated weights for policy 1, policy_version 1847335 (0.0005) [2023-12-27 18:27:22,296][20381] Updated weights for policy 1, policy_version 1847345 (0.0007) [2023-12-27 18:27:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 946388992. Throughput: 0: 10320.6, 1: 10034.3. Samples: 946354084. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '940.990')] [2023-12-27 18:27:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001847352_472973312.pth... [2023-12-27 18:27:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001848952_473415680.pth... [2023-12-27 18:27:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001846168_472670208.pth [2023-12-27 18:27:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001847736_473104384.pth [2023-12-27 18:27:22,898][20381] Updated weights for policy 1, policy_version 1847355 (0.0008) [2023-12-27 18:27:22,949][20347] Updated weights for policy 0, policy_version 1848956 (0.0008) [2023-12-27 18:27:22,958][20381] Updated weights for policy 1, policy_version 1847365 (0.0006) [2023-12-27 18:27:23,004][20347] Updated weights for policy 0, policy_version 1848966 (0.0008) [2023-12-27 18:27:23,020][20381] Updated weights for policy 1, policy_version 1847375 (0.0006) [2023-12-27 18:27:23,066][20347] Updated weights for policy 0, policy_version 1848976 (0.0008) [2023-12-27 18:27:23,554][20381] Updated weights for policy 1, policy_version 1847385 (0.0006) [2023-12-27 18:27:23,607][20381] Updated weights for policy 1, policy_version 1847395 (0.0005) [2023-12-27 18:27:23,659][20381] Updated weights for policy 1, policy_version 1847405 (0.0007) [2023-12-27 18:27:23,708][20381] Updated weights for policy 1, policy_version 1847415 (0.0008) [2023-12-27 18:27:23,751][20347] Updated weights for policy 0, policy_version 1848986 (0.0008) [2023-12-27 18:27:23,806][20347] Updated weights for policy 0, policy_version 1848996 (0.0006) [2023-12-27 18:27:23,861][20347] Updated weights for policy 0, policy_version 1849006 (0.0005) [2023-12-27 18:27:23,921][20347] Updated weights for policy 0, policy_version 1849016 (0.0007) [2023-12-27 18:27:24,402][20381] Updated weights for policy 1, policy_version 1847425 (0.0008) [2023-12-27 18:27:24,459][20381] Updated weights for policy 1, policy_version 1847435 (0.0008) [2023-12-27 18:27:24,507][20381] Updated weights for policy 1, policy_version 1847445 (0.0007) [2023-12-27 18:27:24,556][20347] Updated weights for policy 0, policy_version 1849026 (0.0007) [2023-12-27 18:27:24,609][20347] Updated weights for policy 0, policy_version 1849036 (0.0007) [2023-12-27 18:27:24,665][20347] Updated weights for policy 0, policy_version 1849046 (0.0007) [2023-12-27 18:27:25,135][20381] Updated weights for policy 1, policy_version 1847455 (0.0005) [2023-12-27 18:27:25,183][20381] Updated weights for policy 1, policy_version 1847465 (0.0005) [2023-12-27 18:27:25,240][20381] Updated weights for policy 1, policy_version 1847475 (0.0006) [2023-12-27 18:27:25,388][20347] Updated weights for policy 0, policy_version 1849056 (0.0007) [2023-12-27 18:27:25,436][20347] Updated weights for policy 0, policy_version 1849066 (0.0008) [2023-12-27 18:27:25,499][20347] Updated weights for policy 0, policy_version 1849076 (0.0006) [2023-12-27 18:27:25,859][20381] Updated weights for policy 1, policy_version 1847485 (0.0007) [2023-12-27 18:27:25,921][20381] Updated weights for policy 1, policy_version 1847495 (0.0008) [2023-12-27 18:27:25,980][20381] Updated weights for policy 1, policy_version 1847505 (0.0007) [2023-12-27 18:27:26,211][20347] Updated weights for policy 0, policy_version 1849086 (0.0007) [2023-12-27 18:27:26,262][20347] Updated weights for policy 0, policy_version 1849096 (0.0006) [2023-12-27 18:27:26,313][20347] Updated weights for policy 0, policy_version 1849106 (0.0007) [2023-12-27 18:27:26,657][20381] Updated weights for policy 1, policy_version 1847515 (0.0008) [2023-12-27 18:27:26,712][20381] Updated weights for policy 1, policy_version 1847525 (0.0008) [2023-12-27 18:27:26,767][20381] Updated weights for policy 1, policy_version 1847535 (0.0007) [2023-12-27 18:27:27,027][20347] Updated weights for policy 0, policy_version 1849116 (0.0008) [2023-12-27 18:27:27,080][20347] Updated weights for policy 0, policy_version 1849126 (0.0007) [2023-12-27 18:27:27,134][20347] Updated weights for policy 0, policy_version 1849136 (0.0007) [2023-12-27 18:27:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 946487296. Throughput: 0: 10247.4, 1: 10129.8. Samples: 946476784. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.140')] [2023-12-27 18:27:27,455][20381] Updated weights for policy 1, policy_version 1847545 (0.0007) [2023-12-27 18:27:27,513][20381] Updated weights for policy 1, policy_version 1847555 (0.0008) [2023-12-27 18:27:27,564][20381] Updated weights for policy 1, policy_version 1847565 (0.0007) [2023-12-27 18:27:27,616][20381] Updated weights for policy 1, policy_version 1847575 (0.0008) [2023-12-27 18:27:27,824][20347] Updated weights for policy 0, policy_version 1849146 (0.0007) [2023-12-27 18:27:27,869][20347] Updated weights for policy 0, policy_version 1849156 (0.0007) [2023-12-27 18:27:27,917][20347] Updated weights for policy 0, policy_version 1849166 (0.0008) [2023-12-27 18:27:27,962][20347] Updated weights for policy 0, policy_version 1849176 (0.0007) [2023-12-27 18:27:28,281][20381] Updated weights for policy 1, policy_version 1847585 (0.0006) [2023-12-27 18:27:28,338][20381] Updated weights for policy 1, policy_version 1847595 (0.0008) [2023-12-27 18:27:28,399][20381] Updated weights for policy 1, policy_version 1847605 (0.0007) [2023-12-27 18:27:28,703][20347] Updated weights for policy 0, policy_version 1849186 (0.0008) [2023-12-27 18:27:28,758][20347] Updated weights for policy 0, policy_version 1849196 (0.0007) [2023-12-27 18:27:28,810][20347] Updated weights for policy 0, policy_version 1849206 (0.0006) [2023-12-27 18:27:29,089][20381] Updated weights for policy 1, policy_version 1847615 (0.0007) [2023-12-27 18:27:29,149][20381] Updated weights for policy 1, policy_version 1847625 (0.0005) [2023-12-27 18:27:29,204][20381] Updated weights for policy 1, policy_version 1847635 (0.0006) [2023-12-27 18:27:29,471][20347] Updated weights for policy 0, policy_version 1849216 (0.0007) [2023-12-27 18:27:29,526][20347] Updated weights for policy 0, policy_version 1849226 (0.0009) [2023-12-27 18:27:29,581][20347] Updated weights for policy 0, policy_version 1849236 (0.0008) [2023-12-27 18:27:29,766][20381] Updated weights for policy 1, policy_version 1847645 (0.0007) [2023-12-27 18:27:29,815][20381] Updated weights for policy 1, policy_version 1847655 (0.0006) [2023-12-27 18:27:29,879][20381] Updated weights for policy 1, policy_version 1847665 (0.0008) [2023-12-27 18:27:30,289][20347] Updated weights for policy 0, policy_version 1849246 (0.0008) [2023-12-27 18:27:30,346][20347] Updated weights for policy 0, policy_version 1849256 (0.0008) [2023-12-27 18:27:30,399][20347] Updated weights for policy 0, policy_version 1849266 (0.0008) [2023-12-27 18:27:30,517][20381] Updated weights for policy 1, policy_version 1847675 (0.0007) [2023-12-27 18:27:30,592][20381] Updated weights for policy 1, policy_version 1847685 (0.0006) [2023-12-27 18:27:30,653][20381] Updated weights for policy 1, policy_version 1847695 (0.0005) [2023-12-27 18:27:31,084][20347] Updated weights for policy 0, policy_version 1849276 (0.0007) [2023-12-27 18:27:31,150][20347] Updated weights for policy 0, policy_version 1849287 (0.0010) [2023-12-27 18:27:31,205][20347] Updated weights for policy 0, policy_version 1849297 (0.0007) [2023-12-27 18:27:31,339][20381] Updated weights for policy 1, policy_version 1847705 (0.0006) [2023-12-27 18:27:31,404][20381] Updated weights for policy 1, policy_version 1847715 (0.0007) [2023-12-27 18:27:31,462][20381] Updated weights for policy 1, policy_version 1847725 (0.0005) [2023-12-27 18:27:31,519][20381] Updated weights for policy 1, policy_version 1847735 (0.0005) [2023-12-27 18:27:31,890][20347] Updated weights for policy 0, policy_version 1849307 (0.0006) [2023-12-27 18:27:31,946][20347] Updated weights for policy 0, policy_version 1849317 (0.0006) [2023-12-27 18:27:32,006][20347] Updated weights for policy 0, policy_version 1849327 (0.0005) [2023-12-27 18:27:32,207][20381] Updated weights for policy 1, policy_version 1847745 (0.0007) [2023-12-27 18:27:32,259][20381] Updated weights for policy 1, policy_version 1847755 (0.0005) [2023-12-27 18:27:32,324][20381] Updated weights for policy 1, policy_version 1847765 (0.0006) [2023-12-27 18:27:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 946593792. Throughput: 0: 10233.9, 1: 10144.1. Samples: 946599244. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.730')] [2023-12-27 18:27:32,677][20347] Updated weights for policy 0, policy_version 1849337 (0.0006) [2023-12-27 18:27:32,738][20347] Updated weights for policy 0, policy_version 1849347 (0.0005) [2023-12-27 18:27:32,792][20347] Updated weights for policy 0, policy_version 1849357 (0.0005) [2023-12-27 18:27:32,843][20347] Updated weights for policy 0, policy_version 1849367 (0.0007) [2023-12-27 18:27:32,913][20381] Updated weights for policy 1, policy_version 1847775 (0.0006) [2023-12-27 18:27:32,970][20381] Updated weights for policy 1, policy_version 1847785 (0.0005) [2023-12-27 18:27:33,021][20381] Updated weights for policy 1, policy_version 1847795 (0.0005) [2023-12-27 18:27:33,542][20347] Updated weights for policy 0, policy_version 1849377 (0.0010) [2023-12-27 18:27:33,573][20381] Updated weights for policy 1, policy_version 1847805 (0.0007) [2023-12-27 18:27:33,597][20347] Updated weights for policy 0, policy_version 1849387 (0.0010) [2023-12-27 18:27:33,621][20381] Updated weights for policy 1, policy_version 1847815 (0.0005) [2023-12-27 18:27:33,652][20347] Updated weights for policy 0, policy_version 1849397 (0.0010) [2023-12-27 18:27:33,674][20381] Updated weights for policy 1, policy_version 1847825 (0.0005) [2023-12-27 18:27:34,254][20347] Updated weights for policy 0, policy_version 1849407 (0.0009) [2023-12-27 18:27:34,286][20381] Updated weights for policy 1, policy_version 1847835 (0.0006) [2023-12-27 18:27:34,317][20347] Updated weights for policy 0, policy_version 1849417 (0.0011) [2023-12-27 18:27:34,343][20381] Updated weights for policy 1, policy_version 1847845 (0.0007) [2023-12-27 18:27:34,376][20347] Updated weights for policy 0, policy_version 1849427 (0.0010) [2023-12-27 18:27:34,401][20381] Updated weights for policy 1, policy_version 1847855 (0.0006) [2023-12-27 18:27:34,982][20347] Updated weights for policy 0, policy_version 1849437 (0.0005) [2023-12-27 18:27:35,018][20381] Updated weights for policy 1, policy_version 1847865 (0.0008) [2023-12-27 18:27:35,038][20347] Updated weights for policy 0, policy_version 1849447 (0.0008) [2023-12-27 18:27:35,069][20381] Updated weights for policy 1, policy_version 1847875 (0.0005) [2023-12-27 18:27:35,084][20347] Updated weights for policy 0, policy_version 1849457 (0.0007) [2023-12-27 18:27:35,121][20381] Updated weights for policy 1, policy_version 1847885 (0.0006) [2023-12-27 18:27:35,177][20381] Updated weights for policy 1, policy_version 1847895 (0.0008) [2023-12-27 18:27:35,697][20347] Updated weights for policy 0, policy_version 1849467 (0.0007) [2023-12-27 18:27:35,747][20347] Updated weights for policy 0, policy_version 1849477 (0.0007) [2023-12-27 18:27:35,809][20347] Updated weights for policy 0, policy_version 1849487 (0.0007) [2023-12-27 18:27:35,872][20381] Updated weights for policy 1, policy_version 1847905 (0.0007) [2023-12-27 18:27:35,917][20381] Updated weights for policy 1, policy_version 1847915 (0.0007) [2023-12-27 18:27:35,965][20381] Updated weights for policy 1, policy_version 1847925 (0.0007) [2023-12-27 18:27:36,469][20347] Updated weights for policy 0, policy_version 1849497 (0.0006) [2023-12-27 18:27:36,533][20347] Updated weights for policy 0, policy_version 1849507 (0.0007) [2023-12-27 18:27:36,596][20347] Updated weights for policy 0, policy_version 1849517 (0.0008) [2023-12-27 18:27:36,656][20347] Updated weights for policy 0, policy_version 1849527 (0.0007) [2023-12-27 18:27:36,666][20381] Updated weights for policy 1, policy_version 1847935 (0.0007) [2023-12-27 18:27:36,731][20381] Updated weights for policy 1, policy_version 1847945 (0.0008) [2023-12-27 18:27:36,795][20381] Updated weights for policy 1, policy_version 1847955 (0.0008) [2023-12-27 18:27:37,334][20347] Updated weights for policy 0, policy_version 1849537 (0.0007) [2023-12-27 18:27:37,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20243.9). Total num frames: 946692096. Throughput: 0: 10251.6, 1: 10226.9. Samples: 946665176. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '882.730')] [2023-12-27 18:27:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001847960_473128960.pth... [2023-12-27 18:27:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001846744_472817664.pth [2023-12-27 18:27:37,384][20347] Updated weights for policy 0, policy_version 1849547 (0.0007) [2023-12-27 18:27:37,430][20347] Updated weights for policy 0, policy_version 1849557 (0.0008) [2023-12-27 18:27:37,441][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001849560_473571328.pth... [2023-12-27 18:27:37,444][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001848344_473260032.pth [2023-12-27 18:27:37,491][20381] Updated weights for policy 1, policy_version 1847965 (0.0009) [2023-12-27 18:27:37,538][20381] Updated weights for policy 1, policy_version 1847975 (0.0007) [2023-12-27 18:27:37,591][20381] Updated weights for policy 1, policy_version 1847985 (0.0007) [2023-12-27 18:27:38,122][20347] Updated weights for policy 0, policy_version 1849567 (0.0006) [2023-12-27 18:27:38,173][20347] Updated weights for policy 0, policy_version 1849577 (0.0008) [2023-12-27 18:27:38,221][20347] Updated weights for policy 0, policy_version 1849587 (0.0008) [2023-12-27 18:27:38,289][20381] Updated weights for policy 1, policy_version 1847995 (0.0006) [2023-12-27 18:27:38,350][20381] Updated weights for policy 1, policy_version 1848005 (0.0008) [2023-12-27 18:27:38,407][20381] Updated weights for policy 1, policy_version 1848015 (0.0008) [2023-12-27 18:27:38,959][20347] Updated weights for policy 0, policy_version 1849597 (0.0008) [2023-12-27 18:27:39,018][20347] Updated weights for policy 0, policy_version 1849607 (0.0007) [2023-12-27 18:27:39,076][20347] Updated weights for policy 0, policy_version 1849617 (0.0008) [2023-12-27 18:27:39,097][20381] Updated weights for policy 1, policy_version 1848025 (0.0008) [2023-12-27 18:27:39,146][20381] Updated weights for policy 1, policy_version 1848035 (0.0010) [2023-12-27 18:27:39,202][20381] Updated weights for policy 1, policy_version 1848045 (0.0010) [2023-12-27 18:27:39,262][20381] Updated weights for policy 1, policy_version 1848055 (0.0009) [2023-12-27 18:27:39,814][20347] Updated weights for policy 0, policy_version 1849627 (0.0006) [2023-12-27 18:27:39,875][20347] Updated weights for policy 0, policy_version 1849637 (0.0009) [2023-12-27 18:27:39,929][20347] Updated weights for policy 0, policy_version 1849647 (0.0008) [2023-12-27 18:27:40,068][20381] Updated weights for policy 1, policy_version 1848065 (0.0008) [2023-12-27 18:27:40,122][20381] Updated weights for policy 1, policy_version 1848075 (0.0009) [2023-12-27 18:27:40,175][20381] Updated weights for policy 1, policy_version 1848085 (0.0009) [2023-12-27 18:27:40,607][20347] Updated weights for policy 0, policy_version 1849657 (0.0008) [2023-12-27 18:27:40,653][20347] Updated weights for policy 0, policy_version 1849667 (0.0008) [2023-12-27 18:27:40,702][20347] Updated weights for policy 0, policy_version 1849677 (0.0006) [2023-12-27 18:27:40,748][20347] Updated weights for policy 0, policy_version 1849687 (0.0005) [2023-12-27 18:27:40,883][20381] Updated weights for policy 1, policy_version 1848095 (0.0006) [2023-12-27 18:27:40,941][20381] Updated weights for policy 1, policy_version 1848105 (0.0005) [2023-12-27 18:27:40,992][20381] Updated weights for policy 1, policy_version 1848115 (0.0007) [2023-12-27 18:27:41,491][20347] Updated weights for policy 0, policy_version 1849697 (0.0008) [2023-12-27 18:27:41,553][20347] Updated weights for policy 0, policy_version 1849707 (0.0008) [2023-12-27 18:27:41,616][20347] Updated weights for policy 0, policy_version 1849717 (0.0008) [2023-12-27 18:27:41,782][20381] Updated weights for policy 1, policy_version 1848125 (0.0007) [2023-12-27 18:27:41,848][20381] Updated weights for policy 1, policy_version 1848135 (0.0007) [2023-12-27 18:27:41,909][20381] Updated weights for policy 1, policy_version 1848145 (0.0008) [2023-12-27 18:27:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 946790400. Throughput: 0: 10172.6, 1: 10198.6. Samples: 946785120. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.650')] [2023-12-27 18:27:42,385][20347] Updated weights for policy 0, policy_version 1849727 (0.0008) [2023-12-27 18:27:42,439][20347] Updated weights for policy 0, policy_version 1849737 (0.0007) [2023-12-27 18:27:42,498][20347] Updated weights for policy 0, policy_version 1849747 (0.0007) [2023-12-27 18:27:42,590][20381] Updated weights for policy 1, policy_version 1848155 (0.0008) [2023-12-27 18:27:42,646][20381] Updated weights for policy 1, policy_version 1848165 (0.0006) [2023-12-27 18:27:42,704][20381] Updated weights for policy 1, policy_version 1848175 (0.0006) [2023-12-27 18:27:43,096][20347] Updated weights for policy 0, policy_version 1849757 (0.0007) [2023-12-27 18:27:43,141][20347] Updated weights for policy 0, policy_version 1849767 (0.0008) [2023-12-27 18:27:43,191][20347] Updated weights for policy 0, policy_version 1849777 (0.0007) [2023-12-27 18:27:43,390][20381] Updated weights for policy 1, policy_version 1848185 (0.0008) [2023-12-27 18:27:43,444][20381] Updated weights for policy 1, policy_version 1848195 (0.0008) [2023-12-27 18:27:43,507][20381] Updated weights for policy 1, policy_version 1848205 (0.0008) [2023-12-27 18:27:43,563][20381] Updated weights for policy 1, policy_version 1848215 (0.0008) [2023-12-27 18:27:43,877][20347] Updated weights for policy 0, policy_version 1849787 (0.0007) [2023-12-27 18:27:43,930][20347] Updated weights for policy 0, policy_version 1849797 (0.0005) [2023-12-27 18:27:43,983][20347] Updated weights for policy 0, policy_version 1849807 (0.0006) [2023-12-27 18:27:44,210][20381] Updated weights for policy 1, policy_version 1848225 (0.0006) [2023-12-27 18:27:44,259][20381] Updated weights for policy 1, policy_version 1848235 (0.0005) [2023-12-27 18:27:44,314][20381] Updated weights for policy 1, policy_version 1848245 (0.0007) [2023-12-27 18:27:44,592][20347] Updated weights for policy 0, policy_version 1849817 (0.0006) [2023-12-27 18:27:44,652][20347] Updated weights for policy 0, policy_version 1849827 (0.0010) [2023-12-27 18:27:44,708][20347] Updated weights for policy 0, policy_version 1849837 (0.0010) [2023-12-27 18:27:44,774][20347] Updated weights for policy 0, policy_version 1849847 (0.0010) [2023-12-27 18:27:44,930][20381] Updated weights for policy 1, policy_version 1848255 (0.0011) [2023-12-27 18:27:44,990][20381] Updated weights for policy 1, policy_version 1848265 (0.0011) [2023-12-27 18:27:45,049][20381] Updated weights for policy 1, policy_version 1848275 (0.0009) [2023-12-27 18:27:45,452][20347] Updated weights for policy 0, policy_version 1849857 (0.0010) [2023-12-27 18:27:45,510][20347] Updated weights for policy 0, policy_version 1849867 (0.0010) [2023-12-27 18:27:45,568][20347] Updated weights for policy 0, policy_version 1849877 (0.0010) [2023-12-27 18:27:45,669][20381] Updated weights for policy 1, policy_version 1848285 (0.0005) [2023-12-27 18:27:45,725][20381] Updated weights for policy 1, policy_version 1848295 (0.0007) [2023-12-27 18:27:45,781][20381] Updated weights for policy 1, policy_version 1848305 (0.0008) [2023-12-27 18:27:46,164][20347] Updated weights for policy 0, policy_version 1849887 (0.0010) [2023-12-27 18:27:46,221][20347] Updated weights for policy 0, policy_version 1849897 (0.0011) [2023-12-27 18:27:46,278][20347] Updated weights for policy 0, policy_version 1849907 (0.0012) [2023-12-27 18:27:46,432][20381] Updated weights for policy 1, policy_version 1848315 (0.0008) [2023-12-27 18:27:46,486][20381] Updated weights for policy 1, policy_version 1848325 (0.0007) [2023-12-27 18:27:46,547][20381] Updated weights for policy 1, policy_version 1848335 (0.0007) [2023-12-27 18:27:46,867][20347] Updated weights for policy 0, policy_version 1849917 (0.0008) [2023-12-27 18:27:46,933][20347] Updated weights for policy 0, policy_version 1849927 (0.0008) [2023-12-27 18:27:46,989][20347] Updated weights for policy 0, policy_version 1849937 (0.0010) [2023-12-27 18:27:47,275][20381] Updated weights for policy 1, policy_version 1848345 (0.0007) [2023-12-27 18:27:47,338][20381] Updated weights for policy 1, policy_version 1848355 (0.0008) [2023-12-27 18:27:47,362][19170] Fps is (10 sec: 20480.7, 60 sec: 20343.5, 300 sec: 20271.7). Total num frames: 946896896. Throughput: 0: 10231.6, 1: 10220.4. Samples: 946909600. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:47,363][19170] Avg episode reward: [(0, '980.490'), (1, '872.990')] [2023-12-27 18:27:47,395][20381] Updated weights for policy 1, policy_version 1848365 (0.0007) [2023-12-27 18:27:47,453][20381] Updated weights for policy 1, policy_version 1848375 (0.0008) [2023-12-27 18:27:47,662][20347] Updated weights for policy 0, policy_version 1849947 (0.0011) [2023-12-27 18:27:47,717][20347] Updated weights for policy 0, policy_version 1849957 (0.0010) [2023-12-27 18:27:47,775][20347] Updated weights for policy 0, policy_version 1849967 (0.0010) [2023-12-27 18:27:48,166][20381] Updated weights for policy 1, policy_version 1848385 (0.0007) [2023-12-27 18:27:48,218][20381] Updated weights for policy 1, policy_version 1848395 (0.0008) [2023-12-27 18:27:48,276][20381] Updated weights for policy 1, policy_version 1848405 (0.0008) [2023-12-27 18:27:48,420][20347] Updated weights for policy 0, policy_version 1849977 (0.0010) [2023-12-27 18:27:48,485][20347] Updated weights for policy 0, policy_version 1849987 (0.0008) [2023-12-27 18:27:48,534][20347] Updated weights for policy 0, policy_version 1849997 (0.0008) [2023-12-27 18:27:48,589][20347] Updated weights for policy 0, policy_version 1850007 (0.0008) [2023-12-27 18:27:48,963][20381] Updated weights for policy 1, policy_version 1848415 (0.0007) [2023-12-27 18:27:49,017][20381] Updated weights for policy 1, policy_version 1848425 (0.0008) [2023-12-27 18:27:49,069][20381] Updated weights for policy 1, policy_version 1848435 (0.0006) [2023-12-27 18:27:49,315][20347] Updated weights for policy 0, policy_version 1850017 (0.0007) [2023-12-27 18:27:49,379][20347] Updated weights for policy 0, policy_version 1850027 (0.0009) [2023-12-27 18:27:49,430][20347] Updated weights for policy 0, policy_version 1850037 (0.0007) [2023-12-27 18:27:49,709][20381] Updated weights for policy 1, policy_version 1848445 (0.0007) [2023-12-27 18:27:49,769][20381] Updated weights for policy 1, policy_version 1848455 (0.0008) [2023-12-27 18:27:49,829][20381] Updated weights for policy 1, policy_version 1848465 (0.0008) [2023-12-27 18:27:50,149][20347] Updated weights for policy 0, policy_version 1850047 (0.0006) [2023-12-27 18:27:50,206][20347] Updated weights for policy 0, policy_version 1850057 (0.0006) [2023-12-27 18:27:50,261][20347] Updated weights for policy 0, policy_version 1850067 (0.0007) [2023-12-27 18:27:50,576][20381] Updated weights for policy 1, policy_version 1848475 (0.0008) [2023-12-27 18:27:50,642][20381] Updated weights for policy 1, policy_version 1848485 (0.0008) [2023-12-27 18:27:50,699][20381] Updated weights for policy 1, policy_version 1848495 (0.0008) [2023-12-27 18:27:50,939][20347] Updated weights for policy 0, policy_version 1850077 (0.0008) [2023-12-27 18:27:50,983][20347] Updated weights for policy 0, policy_version 1850087 (0.0007) [2023-12-27 18:27:51,045][20347] Updated weights for policy 0, policy_version 1850097 (0.0007) [2023-12-27 18:27:51,438][20381] Updated weights for policy 1, policy_version 1848505 (0.0008) [2023-12-27 18:27:51,502][20381] Updated weights for policy 1, policy_version 1848515 (0.0009) [2023-12-27 18:27:51,559][20381] Updated weights for policy 1, policy_version 1848525 (0.0005) [2023-12-27 18:27:51,623][20381] Updated weights for policy 1, policy_version 1848535 (0.0007) [2023-12-27 18:27:51,778][20347] Updated weights for policy 0, policy_version 1850107 (0.0008) [2023-12-27 18:27:51,837][20347] Updated weights for policy 0, policy_version 1850117 (0.0007) [2023-12-27 18:27:51,896][20347] Updated weights for policy 0, policy_version 1850127 (0.0008) [2023-12-27 18:27:52,325][20381] Updated weights for policy 1, policy_version 1848545 (0.0008) [2023-12-27 18:27:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20216.2). Total num frames: 946995200. Throughput: 0: 10214.6, 1: 10241.4. Samples: 946969896. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:52,363][19170] Avg episode reward: [(0, '980.490'), (1, '872.420')] [2023-12-27 18:27:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001850136_473718784.pth... [2023-12-27 18:27:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001848952_473415680.pth [2023-12-27 18:27:52,390][20381] Updated weights for policy 1, policy_version 1848555 (0.0008) [2023-12-27 18:27:52,448][20381] Updated weights for policy 1, policy_version 1848565 (0.0008) [2023-12-27 18:27:52,464][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001848568_473284608.pth... [2023-12-27 18:27:52,467][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001847352_472973312.pth [2023-12-27 18:27:52,539][20347] Updated weights for policy 0, policy_version 1850137 (0.0007) [2023-12-27 18:27:52,604][20347] Updated weights for policy 0, policy_version 1850147 (0.0006) [2023-12-27 18:27:52,663][20347] Updated weights for policy 0, policy_version 1850157 (0.0007) [2023-12-27 18:27:52,732][20347] Updated weights for policy 0, policy_version 1850167 (0.0008) [2023-12-27 18:27:53,134][20381] Updated weights for policy 1, policy_version 1848575 (0.0006) [2023-12-27 18:27:53,194][20381] Updated weights for policy 1, policy_version 1848585 (0.0008) [2023-12-27 18:27:53,258][20381] Updated weights for policy 1, policy_version 1848595 (0.0007) [2023-12-27 18:27:53,315][20347] Updated weights for policy 0, policy_version 1850177 (0.0007) [2023-12-27 18:27:53,370][20347] Updated weights for policy 0, policy_version 1850187 (0.0007) [2023-12-27 18:27:53,425][20347] Updated weights for policy 0, policy_version 1850197 (0.0008) [2023-12-27 18:27:53,942][20381] Updated weights for policy 1, policy_version 1848605 (0.0008) [2023-12-27 18:27:54,001][20381] Updated weights for policy 1, policy_version 1848615 (0.0006) [2023-12-27 18:27:54,061][20381] Updated weights for policy 1, policy_version 1848625 (0.0008) [2023-12-27 18:27:54,126][20347] Updated weights for policy 0, policy_version 1850207 (0.0007) [2023-12-27 18:27:54,173][20347] Updated weights for policy 0, policy_version 1850217 (0.0007) [2023-12-27 18:27:54,225][20347] Updated weights for policy 0, policy_version 1850227 (0.0007) [2023-12-27 18:27:54,699][20381] Updated weights for policy 1, policy_version 1848635 (0.0007) [2023-12-27 18:27:54,744][20381] Updated weights for policy 1, policy_version 1848645 (0.0007) [2023-12-27 18:27:54,793][20381] Updated weights for policy 1, policy_version 1848656 (0.0008) [2023-12-27 18:27:54,912][20347] Updated weights for policy 0, policy_version 1850237 (0.0010) [2023-12-27 18:27:54,969][20347] Updated weights for policy 0, policy_version 1850247 (0.0006) [2023-12-27 18:27:55,024][20347] Updated weights for policy 0, policy_version 1850257 (0.0006) [2023-12-27 18:27:55,456][20381] Updated weights for policy 1, policy_version 1848666 (0.0005) [2023-12-27 18:27:55,516][20381] Updated weights for policy 1, policy_version 1848676 (0.0007) [2023-12-27 18:27:55,571][20381] Updated weights for policy 1, policy_version 1848686 (0.0006) [2023-12-27 18:27:55,599][20347] Updated weights for policy 0, policy_version 1850267 (0.0007) [2023-12-27 18:27:55,625][20381] Updated weights for policy 1, policy_version 1848696 (0.0005) [2023-12-27 18:27:55,651][20347] Updated weights for policy 0, policy_version 1850277 (0.0010) [2023-12-27 18:27:55,699][20347] Updated weights for policy 0, policy_version 1850287 (0.0010) [2023-12-27 18:27:56,240][20381] Updated weights for policy 1, policy_version 1848706 (0.0007) [2023-12-27 18:27:56,298][20381] Updated weights for policy 1, policy_version 1848716 (0.0005) [2023-12-27 18:27:56,341][20347] Updated weights for policy 0, policy_version 1850297 (0.0010) [2023-12-27 18:27:56,349][20381] Updated weights for policy 1, policy_version 1848726 (0.0005) [2023-12-27 18:27:56,395][20347] Updated weights for policy 0, policy_version 1850307 (0.0010) [2023-12-27 18:27:56,453][20347] Updated weights for policy 0, policy_version 1850317 (0.0010) [2023-12-27 18:27:56,514][20347] Updated weights for policy 0, policy_version 1850327 (0.0010) [2023-12-27 18:27:56,856][20381] Updated weights for policy 1, policy_version 1848736 (0.0005) [2023-12-27 18:27:56,910][20381] Updated weights for policy 1, policy_version 1848746 (0.0006) [2023-12-27 18:27:56,960][20381] Updated weights for policy 1, policy_version 1848756 (0.0008) [2023-12-27 18:27:57,163][20347] Updated weights for policy 0, policy_version 1850337 (0.0010) [2023-12-27 18:27:57,228][20347] Updated weights for policy 0, policy_version 1850347 (0.0010) [2023-12-27 18:27:57,279][20347] Updated weights for policy 0, policy_version 1850357 (0.0010) [2023-12-27 18:27:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20480.0, 300 sec: 20271.7). Total num frames: 947109888. Throughput: 0: 10204.3, 1: 10335.9. Samples: 947094180. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:27:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '891.450')] [2023-12-27 18:27:57,608][20381] Updated weights for policy 1, policy_version 1848766 (0.0007) [2023-12-27 18:27:57,658][20381] Updated weights for policy 1, policy_version 1848776 (0.0007) [2023-12-27 18:27:57,716][20381] Updated weights for policy 1, policy_version 1848786 (0.0008) [2023-12-27 18:27:57,959][20347] Updated weights for policy 0, policy_version 1850367 (0.0010) [2023-12-27 18:27:58,019][20347] Updated weights for policy 0, policy_version 1850377 (0.0010) [2023-12-27 18:27:58,076][20347] Updated weights for policy 0, policy_version 1850387 (0.0010) [2023-12-27 18:27:58,401][20381] Updated weights for policy 1, policy_version 1848796 (0.0007) [2023-12-27 18:27:58,465][20381] Updated weights for policy 1, policy_version 1848806 (0.0008) [2023-12-27 18:27:58,529][20381] Updated weights for policy 1, policy_version 1848816 (0.0008) [2023-12-27 18:27:58,830][20347] Updated weights for policy 0, policy_version 1850397 (0.0010) [2023-12-27 18:27:58,882][20347] Updated weights for policy 0, policy_version 1850407 (0.0010) [2023-12-27 18:27:58,948][20347] Updated weights for policy 0, policy_version 1850418 (0.0008) [2023-12-27 18:27:59,333][20381] Updated weights for policy 1, policy_version 1848826 (0.0008) [2023-12-27 18:27:59,402][20381] Updated weights for policy 1, policy_version 1848836 (0.0008) [2023-12-27 18:27:59,460][20381] Updated weights for policy 1, policy_version 1848846 (0.0008) [2023-12-27 18:27:59,521][20381] Updated weights for policy 1, policy_version 1848856 (0.0009) [2023-12-27 18:27:59,645][20347] Updated weights for policy 0, policy_version 1850428 (0.0008) [2023-12-27 18:27:59,699][20347] Updated weights for policy 0, policy_version 1850438 (0.0005) [2023-12-27 18:27:59,752][20347] Updated weights for policy 0, policy_version 1850448 (0.0007) [2023-12-27 18:28:00,220][20381] Updated weights for policy 1, policy_version 1848866 (0.0008) [2023-12-27 18:28:00,276][20381] Updated weights for policy 1, policy_version 1848876 (0.0008) [2023-12-27 18:28:00,331][20381] Updated weights for policy 1, policy_version 1848886 (0.0008) [2023-12-27 18:28:00,403][20347] Updated weights for policy 0, policy_version 1850458 (0.0007) [2023-12-27 18:28:00,450][20347] Updated weights for policy 0, policy_version 1850468 (0.0007) [2023-12-27 18:28:00,499][20347] Updated weights for policy 0, policy_version 1850478 (0.0007) [2023-12-27 18:28:00,548][20347] Updated weights for policy 0, policy_version 1850488 (0.0005) [2023-12-27 18:28:01,001][20381] Updated weights for policy 1, policy_version 1848896 (0.0008) [2023-12-27 18:28:01,064][20381] Updated weights for policy 1, policy_version 1848906 (0.0009) [2023-12-27 18:28:01,116][20347] Updated weights for policy 0, policy_version 1850498 (0.0007) [2023-12-27 18:28:01,120][20381] Updated weights for policy 1, policy_version 1848916 (0.0007) [2023-12-27 18:28:01,178][20347] Updated weights for policy 0, policy_version 1850508 (0.0007) [2023-12-27 18:28:01,242][20347] Updated weights for policy 0, policy_version 1850518 (0.0006) [2023-12-27 18:28:01,819][20381] Updated weights for policy 1, policy_version 1848926 (0.0006) [2023-12-27 18:28:01,881][20381] Updated weights for policy 1, policy_version 1848936 (0.0005) [2023-12-27 18:28:01,938][20381] Updated weights for policy 1, policy_version 1848946 (0.0005) [2023-12-27 18:28:01,990][20347] Updated weights for policy 0, policy_version 1850528 (0.0008) [2023-12-27 18:28:02,051][20347] Updated weights for policy 0, policy_version 1850538 (0.0009) [2023-12-27 18:28:02,111][20347] Updated weights for policy 0, policy_version 1850548 (0.0008) [2023-12-27 18:28:02,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 947208192. Throughput: 0: 10225.4, 1: 10319.4. Samples: 947217668. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '909.990')] [2023-12-27 18:28:02,549][20381] Updated weights for policy 1, policy_version 1848956 (0.0006) [2023-12-27 18:28:02,602][20381] Updated weights for policy 1, policy_version 1848966 (0.0008) [2023-12-27 18:28:02,654][20381] Updated weights for policy 1, policy_version 1848976 (0.0008) [2023-12-27 18:28:02,767][20347] Updated weights for policy 0, policy_version 1850558 (0.0006) [2023-12-27 18:28:02,822][20347] Updated weights for policy 0, policy_version 1850568 (0.0008) [2023-12-27 18:28:02,873][20347] Updated weights for policy 0, policy_version 1850578 (0.0007) [2023-12-27 18:28:03,337][20381] Updated weights for policy 1, policy_version 1848986 (0.0008) [2023-12-27 18:28:03,390][20381] Updated weights for policy 1, policy_version 1848996 (0.0008) [2023-12-27 18:28:03,441][20381] Updated weights for policy 1, policy_version 1849006 (0.0008) [2023-12-27 18:28:03,471][20347] Updated weights for policy 0, policy_version 1850588 (0.0007) [2023-12-27 18:28:03,499][20381] Updated weights for policy 1, policy_version 1849016 (0.0008) [2023-12-27 18:28:03,521][20347] Updated weights for policy 0, policy_version 1850598 (0.0007) [2023-12-27 18:28:03,574][20347] Updated weights for policy 0, policy_version 1850608 (0.0005) [2023-12-27 18:28:04,224][20347] Updated weights for policy 0, policy_version 1850618 (0.0005) [2023-12-27 18:28:04,267][20381] Updated weights for policy 1, policy_version 1849026 (0.0006) [2023-12-27 18:28:04,274][20347] Updated weights for policy 0, policy_version 1850628 (0.0006) [2023-12-27 18:28:04,322][20347] Updated weights for policy 0, policy_version 1850638 (0.0006) [2023-12-27 18:28:04,327][20381] Updated weights for policy 1, policy_version 1849036 (0.0008) [2023-12-27 18:28:04,371][20347] Updated weights for policy 0, policy_version 1850648 (0.0008) [2023-12-27 18:28:04,387][20381] Updated weights for policy 1, policy_version 1849046 (0.0008) [2023-12-27 18:28:04,981][20381] Updated weights for policy 1, policy_version 1849056 (0.0006) [2023-12-27 18:28:05,029][20381] Updated weights for policy 1, policy_version 1849066 (0.0008) [2023-12-27 18:28:05,071][20381] Updated weights for policy 1, policy_version 1849076 (0.0005) [2023-12-27 18:28:05,086][20347] Updated weights for policy 0, policy_version 1850658 (0.0008) [2023-12-27 18:28:05,133][20347] Updated weights for policy 0, policy_version 1850668 (0.0008) [2023-12-27 18:28:05,181][20347] Updated weights for policy 0, policy_version 1850678 (0.0007) [2023-12-27 18:28:05,678][20381] Updated weights for policy 1, policy_version 1849086 (0.0007) [2023-12-27 18:28:05,742][20381] Updated weights for policy 1, policy_version 1849096 (0.0009) [2023-12-27 18:28:05,792][20381] Updated weights for policy 1, policy_version 1849106 (0.0008) [2023-12-27 18:28:05,916][20347] Updated weights for policy 0, policy_version 1850688 (0.0008) [2023-12-27 18:28:05,978][20347] Updated weights for policy 0, policy_version 1850698 (0.0008) [2023-12-27 18:28:06,026][20347] Updated weights for policy 0, policy_version 1850708 (0.0007) [2023-12-27 18:28:06,462][20381] Updated weights for policy 1, policy_version 1849116 (0.0007) [2023-12-27 18:28:06,517][20381] Updated weights for policy 1, policy_version 1849126 (0.0007) [2023-12-27 18:28:06,577][20381] Updated weights for policy 1, policy_version 1849136 (0.0008) [2023-12-27 18:28:06,681][20347] Updated weights for policy 0, policy_version 1850718 (0.0008) [2023-12-27 18:28:06,734][20347] Updated weights for policy 0, policy_version 1850728 (0.0008) [2023-12-27 18:28:06,786][20347] Updated weights for policy 0, policy_version 1850738 (0.0007) [2023-12-27 18:28:07,245][20381] Updated weights for policy 1, policy_version 1849146 (0.0008) [2023-12-27 18:28:07,301][20381] Updated weights for policy 1, policy_version 1849156 (0.0011) [2023-12-27 18:28:07,347][20381] Updated weights for policy 1, policy_version 1849166 (0.0010) [2023-12-27 18:28:07,363][19170] Fps is (10 sec: 19660.1, 60 sec: 20479.9, 300 sec: 20243.9). Total num frames: 947306496. Throughput: 0: 10248.4, 1: 10333.5. Samples: 947280280. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '900.200')] [2023-12-27 18:28:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001850744_473874432.pth... [2023-12-27 18:28:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001849560_473571328.pth [2023-12-27 18:28:07,394][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001849176_473440256.pth... [2023-12-27 18:28:07,397][20381] Updated weights for policy 1, policy_version 1849176 (0.0010) [2023-12-27 18:28:07,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001847960_473128960.pth [2023-12-27 18:28:07,512][20347] Updated weights for policy 0, policy_version 1850748 (0.0008) [2023-12-27 18:28:07,567][20347] Updated weights for policy 0, policy_version 1850758 (0.0005) [2023-12-27 18:28:07,618][20347] Updated weights for policy 0, policy_version 1850768 (0.0005) [2023-12-27 18:28:08,022][20381] Updated weights for policy 1, policy_version 1849186 (0.0005) [2023-12-27 18:28:08,083][20381] Updated weights for policy 1, policy_version 1849196 (0.0008) [2023-12-27 18:28:08,147][20381] Updated weights for policy 1, policy_version 1849206 (0.0011) [2023-12-27 18:28:08,247][20347] Updated weights for policy 0, policy_version 1850778 (0.0008) [2023-12-27 18:28:08,304][20347] Updated weights for policy 0, policy_version 1850788 (0.0007) [2023-12-27 18:28:08,365][20347] Updated weights for policy 0, policy_version 1850798 (0.0007) [2023-12-27 18:28:08,427][20347] Updated weights for policy 0, policy_version 1850808 (0.0008) [2023-12-27 18:28:08,762][20381] Updated weights for policy 1, policy_version 1849216 (0.0011) [2023-12-27 18:28:08,821][20381] Updated weights for policy 1, policy_version 1849226 (0.0011) [2023-12-27 18:28:08,879][20381] Updated weights for policy 1, policy_version 1849236 (0.0011) [2023-12-27 18:28:09,124][20347] Updated weights for policy 0, policy_version 1850818 (0.0007) [2023-12-27 18:28:09,182][20347] Updated weights for policy 0, policy_version 1850828 (0.0008) [2023-12-27 18:28:09,246][20347] Updated weights for policy 0, policy_version 1850838 (0.0008) [2023-12-27 18:28:09,537][20381] Updated weights for policy 1, policy_version 1849246 (0.0009) [2023-12-27 18:28:09,593][20381] Updated weights for policy 1, policy_version 1849256 (0.0008) [2023-12-27 18:28:09,644][20381] Updated weights for policy 1, policy_version 1849266 (0.0008) [2023-12-27 18:28:09,987][20347] Updated weights for policy 0, policy_version 1850848 (0.0008) [2023-12-27 18:28:10,046][20347] Updated weights for policy 0, policy_version 1850858 (0.0007) [2023-12-27 18:28:10,113][20347] Updated weights for policy 0, policy_version 1850868 (0.0006) [2023-12-27 18:28:10,391][20381] Updated weights for policy 1, policy_version 1849276 (0.0008) [2023-12-27 18:28:10,447][20381] Updated weights for policy 1, policy_version 1849286 (0.0007) [2023-12-27 18:28:10,497][20381] Updated weights for policy 1, policy_version 1849296 (0.0010) [2023-12-27 18:28:10,861][20347] Updated weights for policy 0, policy_version 1850878 (0.0009) [2023-12-27 18:28:10,916][20347] Updated weights for policy 0, policy_version 1850888 (0.0010) [2023-12-27 18:28:10,970][20347] Updated weights for policy 0, policy_version 1850898 (0.0010) [2023-12-27 18:28:11,143][20381] Updated weights for policy 1, policy_version 1849306 (0.0006) [2023-12-27 18:28:11,204][20381] Updated weights for policy 1, policy_version 1849316 (0.0008) [2023-12-27 18:28:11,267][20381] Updated weights for policy 1, policy_version 1849326 (0.0008) [2023-12-27 18:28:11,332][20381] Updated weights for policy 1, policy_version 1849336 (0.0007) [2023-12-27 18:28:11,749][20347] Updated weights for policy 0, policy_version 1850908 (0.0010) [2023-12-27 18:28:11,817][20347] Updated weights for policy 0, policy_version 1850918 (0.0007) [2023-12-27 18:28:11,873][20347] Updated weights for policy 0, policy_version 1850928 (0.0008) [2023-12-27 18:28:12,063][20381] Updated weights for policy 1, policy_version 1849346 (0.0008) [2023-12-27 18:28:12,112][20381] Updated weights for policy 1, policy_version 1849356 (0.0007) [2023-12-27 18:28:12,162][20381] Updated weights for policy 1, policy_version 1849366 (0.0007) [2023-12-27 18:28:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 947412992. Throughput: 0: 10280.5, 1: 10293.4. Samples: 947402612. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '890.540')] [2023-12-27 18:28:12,612][20347] Updated weights for policy 0, policy_version 1850938 (0.0007) [2023-12-27 18:28:12,665][20347] Updated weights for policy 0, policy_version 1850948 (0.0007) [2023-12-27 18:28:12,723][20347] Updated weights for policy 0, policy_version 1850958 (0.0007) [2023-12-27 18:28:12,784][20347] Updated weights for policy 0, policy_version 1850968 (0.0007) [2023-12-27 18:28:12,871][20381] Updated weights for policy 1, policy_version 1849376 (0.0007) [2023-12-27 18:28:12,926][20381] Updated weights for policy 1, policy_version 1849386 (0.0005) [2023-12-27 18:28:12,982][20381] Updated weights for policy 1, policy_version 1849396 (0.0007) [2023-12-27 18:28:13,409][20347] Updated weights for policy 0, policy_version 1850978 (0.0007) [2023-12-27 18:28:13,466][20347] Updated weights for policy 0, policy_version 1850988 (0.0006) [2023-12-27 18:28:13,528][20347] Updated weights for policy 0, policy_version 1850998 (0.0007) [2023-12-27 18:28:13,626][20381] Updated weights for policy 1, policy_version 1849406 (0.0010) [2023-12-27 18:28:13,688][20381] Updated weights for policy 1, policy_version 1849416 (0.0008) [2023-12-27 18:28:13,741][20381] Updated weights for policy 1, policy_version 1849426 (0.0008) [2023-12-27 18:28:14,198][20347] Updated weights for policy 0, policy_version 1851008 (0.0008) [2023-12-27 18:28:14,253][20347] Updated weights for policy 0, policy_version 1851018 (0.0007) [2023-12-27 18:28:14,304][20347] Updated weights for policy 0, policy_version 1851028 (0.0005) [2023-12-27 18:28:14,426][20381] Updated weights for policy 1, policy_version 1849436 (0.0008) [2023-12-27 18:28:14,472][20381] Updated weights for policy 1, policy_version 1849446 (0.0008) [2023-12-27 18:28:14,517][20381] Updated weights for policy 1, policy_version 1849456 (0.0007) [2023-12-27 18:28:14,953][20347] Updated weights for policy 0, policy_version 1851038 (0.0007) [2023-12-27 18:28:15,006][20347] Updated weights for policy 0, policy_version 1851048 (0.0006) [2023-12-27 18:28:15,059][20347] Updated weights for policy 0, policy_version 1851058 (0.0006) [2023-12-27 18:28:15,243][20381] Updated weights for policy 1, policy_version 1849466 (0.0008) [2023-12-27 18:28:15,300][20381] Updated weights for policy 1, policy_version 1849476 (0.0008) [2023-12-27 18:28:15,360][20381] Updated weights for policy 1, policy_version 1849486 (0.0008) [2023-12-27 18:28:15,409][20381] Updated weights for policy 1, policy_version 1849496 (0.0008) [2023-12-27 18:28:15,699][20347] Updated weights for policy 0, policy_version 1851068 (0.0006) [2023-12-27 18:28:15,751][20347] Updated weights for policy 0, policy_version 1851078 (0.0007) [2023-12-27 18:28:15,803][20347] Updated weights for policy 0, policy_version 1851088 (0.0008) [2023-12-27 18:28:16,105][20381] Updated weights for policy 1, policy_version 1849506 (0.0008) [2023-12-27 18:28:16,163][20381] Updated weights for policy 1, policy_version 1849516 (0.0007) [2023-12-27 18:28:16,217][20381] Updated weights for policy 1, policy_version 1849526 (0.0006) [2023-12-27 18:28:16,486][20347] Updated weights for policy 0, policy_version 1851098 (0.0007) [2023-12-27 18:28:16,545][20347] Updated weights for policy 0, policy_version 1851108 (0.0006) [2023-12-27 18:28:16,593][20347] Updated weights for policy 0, policy_version 1851118 (0.0005) [2023-12-27 18:28:16,651][20347] Updated weights for policy 0, policy_version 1851128 (0.0007) [2023-12-27 18:28:16,914][20381] Updated weights for policy 1, policy_version 1849536 (0.0008) [2023-12-27 18:28:16,969][20381] Updated weights for policy 1, policy_version 1849546 (0.0008) [2023-12-27 18:28:17,033][20381] Updated weights for policy 1, policy_version 1849556 (0.0008) [2023-12-27 18:28:17,227][20347] Updated weights for policy 0, policy_version 1851138 (0.0006) [2023-12-27 18:28:17,272][20347] Updated weights for policy 0, policy_version 1851148 (0.0005) [2023-12-27 18:28:17,322][20347] Updated weights for policy 0, policy_version 1851158 (0.0006) [2023-12-27 18:28:17,362][19170] Fps is (10 sec: 21300.3, 60 sec: 20616.6, 300 sec: 20271.7). Total num frames: 947519488. Throughput: 0: 10320.4, 1: 10253.7. Samples: 947525076. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '890.620')] [2023-12-27 18:28:17,647][20381] Updated weights for policy 1, policy_version 1849566 (0.0006) [2023-12-27 18:28:17,708][20381] Updated weights for policy 1, policy_version 1849576 (0.0009) [2023-12-27 18:28:17,770][20381] Updated weights for policy 1, policy_version 1849586 (0.0008) [2023-12-27 18:28:17,884][20347] Updated weights for policy 0, policy_version 1851168 (0.0006) [2023-12-27 18:28:17,944][20347] Updated weights for policy 0, policy_version 1851178 (0.0006) [2023-12-27 18:28:18,001][20347] Updated weights for policy 0, policy_version 1851188 (0.0006) [2023-12-27 18:28:18,426][20381] Updated weights for policy 1, policy_version 1849596 (0.0008) [2023-12-27 18:28:18,485][20381] Updated weights for policy 1, policy_version 1849606 (0.0008) [2023-12-27 18:28:18,544][20381] Updated weights for policy 1, policy_version 1849616 (0.0008) [2023-12-27 18:28:18,683][20347] Updated weights for policy 0, policy_version 1851198 (0.0007) [2023-12-27 18:28:18,744][20347] Updated weights for policy 0, policy_version 1851208 (0.0008) [2023-12-27 18:28:18,805][20347] Updated weights for policy 0, policy_version 1851218 (0.0008) [2023-12-27 18:28:19,232][20381] Updated weights for policy 1, policy_version 1849626 (0.0008) [2023-12-27 18:28:19,296][20381] Updated weights for policy 1, policy_version 1849636 (0.0008) [2023-12-27 18:28:19,360][20381] Updated weights for policy 1, policy_version 1849646 (0.0008) [2023-12-27 18:28:19,420][20381] Updated weights for policy 1, policy_version 1849656 (0.0008) [2023-12-27 18:28:19,571][20347] Updated weights for policy 0, policy_version 1851228 (0.0007) [2023-12-27 18:28:19,630][20347] Updated weights for policy 0, policy_version 1851238 (0.0008) [2023-12-27 18:28:19,690][20347] Updated weights for policy 0, policy_version 1851248 (0.0008) [2023-12-27 18:28:20,150][20381] Updated weights for policy 1, policy_version 1849666 (0.0008) [2023-12-27 18:28:20,201][20381] Updated weights for policy 1, policy_version 1849676 (0.0007) [2023-12-27 18:28:20,264][20381] Updated weights for policy 1, policy_version 1849686 (0.0006) [2023-12-27 18:28:20,416][20347] Updated weights for policy 0, policy_version 1851258 (0.0008) [2023-12-27 18:28:20,475][20347] Updated weights for policy 0, policy_version 1851268 (0.0007) [2023-12-27 18:28:20,528][20347] Updated weights for policy 0, policy_version 1851278 (0.0008) [2023-12-27 18:28:20,585][20347] Updated weights for policy 0, policy_version 1851288 (0.0007) [2023-12-27 18:28:20,934][20292] KL-divergence is very high: 102.0591 [2023-12-27 18:28:20,942][20381] Updated weights for policy 1, policy_version 1849696 (0.0007) [2023-12-27 18:28:20,984][20292] KL-divergence is very high: 181.7766 [2023-12-27 18:28:20,990][20292] KL-divergence is very high: 144.0988 [2023-12-27 18:28:21,001][20381] Updated weights for policy 1, policy_version 1849706 (0.0008) [2023-12-27 18:28:21,033][20292] KL-divergence is very high: 161.5701 [2023-12-27 18:28:21,040][20292] KL-divergence is very high: 121.0385 [2023-12-27 18:28:21,070][20381] Updated weights for policy 1, policy_version 1849716 (0.0008) [2023-12-27 18:28:21,087][20292] KL-divergence is very high: 112.2857 [2023-12-27 18:28:21,271][20347] Updated weights for policy 0, policy_version 1851298 (0.0009) [2023-12-27 18:28:21,335][20347] Updated weights for policy 0, policy_version 1851308 (0.0008) [2023-12-27 18:28:21,409][20347] Updated weights for policy 0, policy_version 1851318 (0.0009) [2023-12-27 18:28:21,776][20381] Updated weights for policy 1, policy_version 1849726 (0.0008) [2023-12-27 18:28:21,832][20381] Updated weights for policy 1, policy_version 1849736 (0.0007) [2023-12-27 18:28:21,900][20381] Updated weights for policy 1, policy_version 1849746 (0.0006) [2023-12-27 18:28:22,158][20347] Updated weights for policy 0, policy_version 1851328 (0.0009) [2023-12-27 18:28:22,225][20347] Updated weights for policy 0, policy_version 1851338 (0.0008) [2023-12-27 18:28:22,288][20347] Updated weights for policy 0, policy_version 1851348 (0.0007) [2023-12-27 18:28:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 947617792. Throughput: 0: 10300.1, 1: 10171.0. Samples: 947586368. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '947.900')] [2023-12-27 18:28:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001849752_473587712.pth... [2023-12-27 18:28:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001851352_474030080.pth... [2023-12-27 18:28:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001850136_473718784.pth [2023-12-27 18:28:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001848568_473284608.pth [2023-12-27 18:28:22,555][20381] Updated weights for policy 1, policy_version 1849756 (0.0006) [2023-12-27 18:28:22,617][20381] Updated weights for policy 1, policy_version 1849766 (0.0007) [2023-12-27 18:28:22,670][20381] Updated weights for policy 1, policy_version 1849776 (0.0007) [2023-12-27 18:28:23,043][20347] Updated weights for policy 0, policy_version 1851358 (0.0008) [2023-12-27 18:28:23,099][20347] Updated weights for policy 0, policy_version 1851368 (0.0007) [2023-12-27 18:28:23,158][20347] Updated weights for policy 0, policy_version 1851378 (0.0007) [2023-12-27 18:28:23,351][20381] Updated weights for policy 1, policy_version 1849786 (0.0007) [2023-12-27 18:28:23,404][20381] Updated weights for policy 1, policy_version 1849796 (0.0008) [2023-12-27 18:28:23,452][20381] Updated weights for policy 1, policy_version 1849806 (0.0008) [2023-12-27 18:28:23,507][20381] Updated weights for policy 1, policy_version 1849816 (0.0007) [2023-12-27 18:28:23,849][20347] Updated weights for policy 0, policy_version 1851388 (0.0007) [2023-12-27 18:28:23,907][20347] Updated weights for policy 0, policy_version 1851398 (0.0007) [2023-12-27 18:28:23,964][20347] Updated weights for policy 0, policy_version 1851408 (0.0007) [2023-12-27 18:28:24,157][20381] Updated weights for policy 1, policy_version 1849826 (0.0008) [2023-12-27 18:28:24,206][20381] Updated weights for policy 1, policy_version 1849836 (0.0006) [2023-12-27 18:28:24,256][20381] Updated weights for policy 1, policy_version 1849846 (0.0007) [2023-12-27 18:28:24,642][20347] Updated weights for policy 0, policy_version 1851418 (0.0008) [2023-12-27 18:28:24,700][20347] Updated weights for policy 0, policy_version 1851428 (0.0010) [2023-12-27 18:28:24,754][20347] Updated weights for policy 0, policy_version 1851438 (0.0010) [2023-12-27 18:28:24,809][20347] Updated weights for policy 0, policy_version 1851448 (0.0010) [2023-12-27 18:28:24,876][20381] Updated weights for policy 1, policy_version 1849856 (0.0005) [2023-12-27 18:28:24,933][20381] Updated weights for policy 1, policy_version 1849866 (0.0006) [2023-12-27 18:28:24,994][20381] Updated weights for policy 1, policy_version 1849876 (0.0007) [2023-12-27 18:28:25,462][20347] Updated weights for policy 0, policy_version 1851458 (0.0009) [2023-12-27 18:28:25,507][20347] Updated weights for policy 0, policy_version 1851468 (0.0006) [2023-12-27 18:28:25,555][20347] Updated weights for policy 0, policy_version 1851478 (0.0010) [2023-12-27 18:28:25,632][20381] Updated weights for policy 1, policy_version 1849886 (0.0008) [2023-12-27 18:28:25,681][20381] Updated weights for policy 1, policy_version 1849896 (0.0009) [2023-12-27 18:28:25,728][20381] Updated weights for policy 1, policy_version 1849906 (0.0007) [2023-12-27 18:28:26,222][20347] Updated weights for policy 0, policy_version 1851488 (0.0010) [2023-12-27 18:28:26,270][20347] Updated weights for policy 0, policy_version 1851498 (0.0010) [2023-12-27 18:28:26,325][20347] Updated weights for policy 0, policy_version 1851508 (0.0010) [2023-12-27 18:28:26,346][20381] Updated weights for policy 1, policy_version 1849916 (0.0009) [2023-12-27 18:28:26,405][20381] Updated weights for policy 1, policy_version 1849926 (0.0010) [2023-12-27 18:28:26,459][20381] Updated weights for policy 1, policy_version 1849936 (0.0010) [2023-12-27 18:28:26,943][20347] Updated weights for policy 0, policy_version 1851518 (0.0009) [2023-12-27 18:28:26,998][20347] Updated weights for policy 0, policy_version 1851528 (0.0008) [2023-12-27 18:28:27,048][20347] Updated weights for policy 0, policy_version 1851538 (0.0008) [2023-12-27 18:28:27,141][20381] Updated weights for policy 1, policy_version 1849946 (0.0010) [2023-12-27 18:28:27,195][20381] Updated weights for policy 1, policy_version 1849956 (0.0010) [2023-12-27 18:28:27,259][20381] Updated weights for policy 1, policy_version 1849966 (0.0010) [2023-12-27 18:28:27,314][20381] Updated weights for policy 1, policy_version 1849976 (0.0009) [2023-12-27 18:28:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 947724288. Throughput: 0: 10272.3, 1: 10265.1. Samples: 947709304. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '878.830')] [2023-12-27 18:28:27,729][20347] Updated weights for policy 0, policy_version 1851548 (0.0009) [2023-12-27 18:28:27,779][20347] Updated weights for policy 0, policy_version 1851558 (0.0010) [2023-12-27 18:28:27,826][20347] Updated weights for policy 0, policy_version 1851568 (0.0010) [2023-12-27 18:28:27,915][20381] Updated weights for policy 1, policy_version 1849986 (0.0010) [2023-12-27 18:28:27,966][20381] Updated weights for policy 1, policy_version 1849996 (0.0010) [2023-12-27 18:28:28,014][20381] Updated weights for policy 1, policy_version 1850006 (0.0010) [2023-12-27 18:28:28,457][20347] Updated weights for policy 0, policy_version 1851578 (0.0010) [2023-12-27 18:28:28,505][20347] Updated weights for policy 0, policy_version 1851588 (0.0010) [2023-12-27 18:28:28,565][20347] Updated weights for policy 0, policy_version 1851598 (0.0010) [2023-12-27 18:28:28,623][20347] Updated weights for policy 0, policy_version 1851608 (0.0010) [2023-12-27 18:28:28,693][20381] Updated weights for policy 1, policy_version 1850016 (0.0010) [2023-12-27 18:28:28,752][20381] Updated weights for policy 1, policy_version 1850026 (0.0010) [2023-12-27 18:28:28,807][20381] Updated weights for policy 1, policy_version 1850036 (0.0010) [2023-12-27 18:28:29,330][20347] Updated weights for policy 0, policy_version 1851618 (0.0006) [2023-12-27 18:28:29,400][20347] Updated weights for policy 0, policy_version 1851628 (0.0008) [2023-12-27 18:28:29,459][20347] Updated weights for policy 0, policy_version 1851638 (0.0007) [2023-12-27 18:28:29,546][20381] Updated weights for policy 1, policy_version 1850046 (0.0011) [2023-12-27 18:28:29,595][20381] Updated weights for policy 1, policy_version 1850056 (0.0010) [2023-12-27 18:28:29,660][20381] Updated weights for policy 1, policy_version 1850066 (0.0011) [2023-12-27 18:28:30,134][20347] Updated weights for policy 0, policy_version 1851648 (0.0007) [2023-12-27 18:28:30,186][20347] Updated weights for policy 0, policy_version 1851658 (0.0008) [2023-12-27 18:28:30,242][20347] Updated weights for policy 0, policy_version 1851668 (0.0008) [2023-12-27 18:28:30,351][20381] Updated weights for policy 1, policy_version 1850076 (0.0009) [2023-12-27 18:28:30,412][20381] Updated weights for policy 1, policy_version 1850086 (0.0008) [2023-12-27 18:28:30,455][20381] Updated weights for policy 1, policy_version 1850096 (0.0010) [2023-12-27 18:28:30,883][20347] Updated weights for policy 0, policy_version 1851678 (0.0009) [2023-12-27 18:28:30,931][20347] Updated weights for policy 0, policy_version 1851688 (0.0005) [2023-12-27 18:28:30,979][20347] Updated weights for policy 0, policy_version 1851698 (0.0009) [2023-12-27 18:28:31,190][20381] Updated weights for policy 1, policy_version 1850106 (0.0009) [2023-12-27 18:28:31,246][20381] Updated weights for policy 1, policy_version 1850116 (0.0008) [2023-12-27 18:28:31,300][20381] Updated weights for policy 1, policy_version 1850126 (0.0008) [2023-12-27 18:28:31,368][20381] Updated weights for policy 1, policy_version 1850136 (0.0008) [2023-12-27 18:28:31,614][20347] Updated weights for policy 0, policy_version 1851708 (0.0009) [2023-12-27 18:28:31,673][20347] Updated weights for policy 0, policy_version 1851718 (0.0006) [2023-12-27 18:28:31,736][20347] Updated weights for policy 0, policy_version 1851728 (0.0008) [2023-12-27 18:28:32,069][20381] Updated weights for policy 1, policy_version 1850146 (0.0006) [2023-12-27 18:28:32,120][20381] Updated weights for policy 1, policy_version 1850156 (0.0006) [2023-12-27 18:28:32,175][20381] Updated weights for policy 1, policy_version 1850166 (0.0006) [2023-12-27 18:28:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 947822592. Throughput: 0: 10261.0, 1: 10246.6. Samples: 947832448. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.250')] [2023-12-27 18:28:32,471][20347] Updated weights for policy 0, policy_version 1851738 (0.0008) [2023-12-27 18:28:32,534][20347] Updated weights for policy 0, policy_version 1851748 (0.0008) [2023-12-27 18:28:32,602][20347] Updated weights for policy 0, policy_version 1851758 (0.0008) [2023-12-27 18:28:32,664][20347] Updated weights for policy 0, policy_version 1851768 (0.0008) [2023-12-27 18:28:32,865][20381] Updated weights for policy 1, policy_version 1850176 (0.0008) [2023-12-27 18:28:32,921][20381] Updated weights for policy 1, policy_version 1850186 (0.0007) [2023-12-27 18:28:32,980][20381] Updated weights for policy 1, policy_version 1850196 (0.0007) [2023-12-27 18:28:33,329][20347] Updated weights for policy 0, policy_version 1851778 (0.0007) [2023-12-27 18:28:33,373][20347] Updated weights for policy 0, policy_version 1851788 (0.0005) [2023-12-27 18:28:33,419][20347] Updated weights for policy 0, policy_version 1851798 (0.0005) [2023-12-27 18:28:33,691][20381] Updated weights for policy 1, policy_version 1850206 (0.0007) [2023-12-27 18:28:33,737][20381] Updated weights for policy 1, policy_version 1850216 (0.0008) [2023-12-27 18:28:33,784][20381] Updated weights for policy 1, policy_version 1850226 (0.0010) [2023-12-27 18:28:34,047][20347] Updated weights for policy 0, policy_version 1851808 (0.0005) [2023-12-27 18:28:34,100][20347] Updated weights for policy 0, policy_version 1851818 (0.0006) [2023-12-27 18:28:34,151][20347] Updated weights for policy 0, policy_version 1851828 (0.0008) [2023-12-27 18:28:34,515][20381] Updated weights for policy 1, policy_version 1850236 (0.0009) [2023-12-27 18:28:34,585][20381] Updated weights for policy 1, policy_version 1850246 (0.0008) [2023-12-27 18:28:34,646][20381] Updated weights for policy 1, policy_version 1850256 (0.0009) [2023-12-27 18:28:34,843][20347] Updated weights for policy 0, policy_version 1851838 (0.0010) [2023-12-27 18:28:34,895][20347] Updated weights for policy 0, policy_version 1851848 (0.0010) [2023-12-27 18:28:34,954][20347] Updated weights for policy 0, policy_version 1851858 (0.0011) [2023-12-27 18:28:35,316][20381] Updated weights for policy 1, policy_version 1850266 (0.0008) [2023-12-27 18:28:35,381][20381] Updated weights for policy 1, policy_version 1850276 (0.0008) [2023-12-27 18:28:35,430][20381] Updated weights for policy 1, policy_version 1850286 (0.0007) [2023-12-27 18:28:35,486][20381] Updated weights for policy 1, policy_version 1850296 (0.0007) [2023-12-27 18:28:35,628][20347] Updated weights for policy 0, policy_version 1851868 (0.0010) [2023-12-27 18:28:35,679][20347] Updated weights for policy 0, policy_version 1851878 (0.0010) [2023-12-27 18:28:35,736][20347] Updated weights for policy 0, policy_version 1851888 (0.0010) [2023-12-27 18:28:36,223][20381] Updated weights for policy 1, policy_version 1850306 (0.0007) [2023-12-27 18:28:36,287][20381] Updated weights for policy 1, policy_version 1850316 (0.0008) [2023-12-27 18:28:36,350][20381] Updated weights for policy 1, policy_version 1850326 (0.0007) [2023-12-27 18:28:36,428][20347] Updated weights for policy 0, policy_version 1851898 (0.0011) [2023-12-27 18:28:36,485][20347] Updated weights for policy 0, policy_version 1851908 (0.0011) [2023-12-27 18:28:36,540][20347] Updated weights for policy 0, policy_version 1851918 (0.0010) [2023-12-27 18:28:36,599][20347] Updated weights for policy 0, policy_version 1851928 (0.0010) [2023-12-27 18:28:36,933][20381] Updated weights for policy 1, policy_version 1850336 (0.0007) [2023-12-27 18:28:36,995][20381] Updated weights for policy 1, policy_version 1850346 (0.0008) [2023-12-27 18:28:37,044][20381] Updated weights for policy 1, policy_version 1850356 (0.0006) [2023-12-27 18:28:37,223][20347] Updated weights for policy 0, policy_version 1851938 (0.0010) [2023-12-27 18:28:37,270][20347] Updated weights for policy 0, policy_version 1851948 (0.0009) [2023-12-27 18:28:37,322][20347] Updated weights for policy 0, policy_version 1851958 (0.0007) [2023-12-27 18:28:37,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20271.7). Total num frames: 947929088. Throughput: 0: 10266.8, 1: 10251.3. Samples: 947893216. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.140')] [2023-12-27 18:28:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001850360_473743360.pth... [2023-12-27 18:28:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001851960_474185728.pth... [2023-12-27 18:28:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001849176_473440256.pth [2023-12-27 18:28:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001850744_473874432.pth [2023-12-27 18:28:37,651][20381] Updated weights for policy 1, policy_version 1850366 (0.0007) [2023-12-27 18:28:37,703][20381] Updated weights for policy 1, policy_version 1850376 (0.0007) [2023-12-27 18:28:37,762][20381] Updated weights for policy 1, policy_version 1850386 (0.0007) [2023-12-27 18:28:38,045][20347] Updated weights for policy 0, policy_version 1851968 (0.0007) [2023-12-27 18:28:38,106][20347] Updated weights for policy 0, policy_version 1851978 (0.0006) [2023-12-27 18:28:38,162][20347] Updated weights for policy 0, policy_version 1851988 (0.0005) [2023-12-27 18:28:38,492][20381] Updated weights for policy 1, policy_version 1850396 (0.0007) [2023-12-27 18:28:38,555][20381] Updated weights for policy 1, policy_version 1850406 (0.0009) [2023-12-27 18:28:38,621][20381] Updated weights for policy 1, policy_version 1850416 (0.0008) [2023-12-27 18:28:38,729][20347] Updated weights for policy 0, policy_version 1851998 (0.0007) [2023-12-27 18:28:38,788][20347] Updated weights for policy 0, policy_version 1852008 (0.0008) [2023-12-27 18:28:38,850][20347] Updated weights for policy 0, policy_version 1852018 (0.0007) [2023-12-27 18:28:39,341][20381] Updated weights for policy 1, policy_version 1850426 (0.0010) [2023-12-27 18:28:39,409][20381] Updated weights for policy 1, policy_version 1850436 (0.0011) [2023-12-27 18:28:39,471][20381] Updated weights for policy 1, policy_version 1850446 (0.0007) [2023-12-27 18:28:39,528][20381] Updated weights for policy 1, policy_version 1850456 (0.0008) [2023-12-27 18:28:39,626][20347] Updated weights for policy 0, policy_version 1852028 (0.0008) [2023-12-27 18:28:39,693][20347] Updated weights for policy 0, policy_version 1852038 (0.0008) [2023-12-27 18:28:39,755][20347] Updated weights for policy 0, policy_version 1852048 (0.0008) [2023-12-27 18:28:40,217][20381] Updated weights for policy 1, policy_version 1850466 (0.0006) [2023-12-27 18:28:40,274][20381] Updated weights for policy 1, policy_version 1850476 (0.0007) [2023-12-27 18:28:40,328][20381] Updated weights for policy 1, policy_version 1850486 (0.0008) [2023-12-27 18:28:40,471][20347] Updated weights for policy 0, policy_version 1852058 (0.0008) [2023-12-27 18:28:40,518][20347] Updated weights for policy 0, policy_version 1852068 (0.0007) [2023-12-27 18:28:40,565][20347] Updated weights for policy 0, policy_version 1852078 (0.0009) [2023-12-27 18:28:40,622][20347] Updated weights for policy 0, policy_version 1852088 (0.0006) [2023-12-27 18:28:40,964][20381] Updated weights for policy 1, policy_version 1850496 (0.0006) [2023-12-27 18:28:41,016][20381] Updated weights for policy 1, policy_version 1850506 (0.0005) [2023-12-27 18:28:41,071][20381] Updated weights for policy 1, policy_version 1850516 (0.0008) [2023-12-27 18:28:41,314][20347] Updated weights for policy 0, policy_version 1852098 (0.0007) [2023-12-27 18:28:41,388][20347] Updated weights for policy 0, policy_version 1852108 (0.0008) [2023-12-27 18:28:41,451][20347] Updated weights for policy 0, policy_version 1852118 (0.0008) [2023-12-27 18:28:41,817][20381] Updated weights for policy 1, policy_version 1850526 (0.0009) [2023-12-27 18:28:41,874][20381] Updated weights for policy 1, policy_version 1850536 (0.0007) [2023-12-27 18:28:41,935][20381] Updated weights for policy 1, policy_version 1850546 (0.0008) [2023-12-27 18:28:42,136][20347] Updated weights for policy 0, policy_version 1852128 (0.0008) [2023-12-27 18:28:42,197][20347] Updated weights for policy 0, policy_version 1852138 (0.0008) [2023-12-27 18:28:42,253][20347] Updated weights for policy 0, policy_version 1852148 (0.0007) [2023-12-27 18:28:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20216.2). Total num frames: 948027392. Throughput: 0: 10238.3, 1: 10240.5. Samples: 948015728. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.080')] [2023-12-27 18:28:42,696][20381] Updated weights for policy 1, policy_version 1850556 (0.0008) [2023-12-27 18:28:42,749][20381] Updated weights for policy 1, policy_version 1850566 (0.0009) [2023-12-27 18:28:42,805][20381] Updated weights for policy 1, policy_version 1850576 (0.0006) [2023-12-27 18:28:42,962][20347] Updated weights for policy 0, policy_version 1852158 (0.0009) [2023-12-27 18:28:43,009][20347] Updated weights for policy 0, policy_version 1852168 (0.0010) [2023-12-27 18:28:43,060][20347] Updated weights for policy 0, policy_version 1852178 (0.0006) [2023-12-27 18:28:43,442][20381] Updated weights for policy 1, policy_version 1850586 (0.0009) [2023-12-27 18:28:43,503][20381] Updated weights for policy 1, policy_version 1850596 (0.0005) [2023-12-27 18:28:43,562][20381] Updated weights for policy 1, policy_version 1850606 (0.0006) [2023-12-27 18:28:43,615][20381] Updated weights for policy 1, policy_version 1850616 (0.0006) [2023-12-27 18:28:43,670][20347] Updated weights for policy 0, policy_version 1852188 (0.0006) [2023-12-27 18:28:43,724][20347] Updated weights for policy 0, policy_version 1852198 (0.0006) [2023-12-27 18:28:43,776][20347] Updated weights for policy 0, policy_version 1852208 (0.0005) [2023-12-27 18:28:44,165][20381] Updated weights for policy 1, policy_version 1850626 (0.0005) [2023-12-27 18:28:44,213][20381] Updated weights for policy 1, policy_version 1850636 (0.0005) [2023-12-27 18:28:44,263][20381] Updated weights for policy 1, policy_version 1850646 (0.0005) [2023-12-27 18:28:44,383][20347] Updated weights for policy 0, policy_version 1852218 (0.0007) [2023-12-27 18:28:44,432][20347] Updated weights for policy 0, policy_version 1852228 (0.0010) [2023-12-27 18:28:44,498][20347] Updated weights for policy 0, policy_version 1852238 (0.0010) [2023-12-27 18:28:44,557][20347] Updated weights for policy 0, policy_version 1852248 (0.0010) [2023-12-27 18:28:44,946][20381] Updated weights for policy 1, policy_version 1850656 (0.0007) [2023-12-27 18:28:45,009][20381] Updated weights for policy 1, policy_version 1850666 (0.0007) [2023-12-27 18:28:45,074][20381] Updated weights for policy 1, policy_version 1850676 (0.0006) [2023-12-27 18:28:45,223][20347] Updated weights for policy 0, policy_version 1852258 (0.0011) [2023-12-27 18:28:45,280][20347] Updated weights for policy 0, policy_version 1852268 (0.0011) [2023-12-27 18:28:45,344][20347] Updated weights for policy 0, policy_version 1852278 (0.0010) [2023-12-27 18:28:45,702][20381] Updated weights for policy 1, policy_version 1850686 (0.0006) [2023-12-27 18:28:45,759][20381] Updated weights for policy 1, policy_version 1850696 (0.0005) [2023-12-27 18:28:45,822][20381] Updated weights for policy 1, policy_version 1850706 (0.0006) [2023-12-27 18:28:45,990][20347] Updated weights for policy 0, policy_version 1852288 (0.0008) [2023-12-27 18:28:46,048][20347] Updated weights for policy 0, policy_version 1852298 (0.0007) [2023-12-27 18:28:46,103][20347] Updated weights for policy 0, policy_version 1852308 (0.0007) [2023-12-27 18:28:46,469][20381] Updated weights for policy 1, policy_version 1850716 (0.0008) [2023-12-27 18:28:46,521][20381] Updated weights for policy 1, policy_version 1850726 (0.0007) [2023-12-27 18:28:46,580][20381] Updated weights for policy 1, policy_version 1850736 (0.0006) [2023-12-27 18:28:46,783][20347] Updated weights for policy 0, policy_version 1852318 (0.0007) [2023-12-27 18:28:46,841][20347] Updated weights for policy 0, policy_version 1852328 (0.0007) [2023-12-27 18:28:46,882][20347] Updated weights for policy 0, policy_version 1852338 (0.0007) [2023-12-27 18:28:47,225][20381] Updated weights for policy 1, policy_version 1850746 (0.0007) [2023-12-27 18:28:47,275][20381] Updated weights for policy 1, policy_version 1850756 (0.0007) [2023-12-27 18:28:47,329][20381] Updated weights for policy 1, policy_version 1850766 (0.0006) [2023-12-27 18:28:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 948125696. Throughput: 0: 10246.1, 1: 10267.1. Samples: 948140760. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '900.880')] [2023-12-27 18:28:47,375][20381] Updated weights for policy 1, policy_version 1850776 (0.0005) [2023-12-27 18:28:47,604][20347] Updated weights for policy 0, policy_version 1852348 (0.0007) [2023-12-27 18:28:47,665][20347] Updated weights for policy 0, policy_version 1852358 (0.0007) [2023-12-27 18:28:47,720][20347] Updated weights for policy 0, policy_version 1852368 (0.0007) [2023-12-27 18:28:48,004][20381] Updated weights for policy 1, policy_version 1850786 (0.0008) [2023-12-27 18:28:48,056][20381] Updated weights for policy 1, policy_version 1850796 (0.0007) [2023-12-27 18:28:48,106][20381] Updated weights for policy 1, policy_version 1850806 (0.0007) [2023-12-27 18:28:48,425][20347] Updated weights for policy 0, policy_version 1852378 (0.0008) [2023-12-27 18:28:48,488][20347] Updated weights for policy 0, policy_version 1852388 (0.0011) [2023-12-27 18:28:48,550][20347] Updated weights for policy 0, policy_version 1852398 (0.0011) [2023-12-27 18:28:48,603][20347] Updated weights for policy 0, policy_version 1852408 (0.0011) [2023-12-27 18:28:48,833][20381] Updated weights for policy 1, policy_version 1850816 (0.0008) [2023-12-27 18:28:48,895][20381] Updated weights for policy 1, policy_version 1850826 (0.0008) [2023-12-27 18:28:48,952][20381] Updated weights for policy 1, policy_version 1850836 (0.0009) [2023-12-27 18:28:49,182][20347] Updated weights for policy 0, policy_version 1852418 (0.0005) [2023-12-27 18:28:49,246][20347] Updated weights for policy 0, policy_version 1852428 (0.0008) [2023-12-27 18:28:49,313][20347] Updated weights for policy 0, policy_version 1852438 (0.0010) [2023-12-27 18:28:49,661][20381] Updated weights for policy 1, policy_version 1850846 (0.0008) [2023-12-27 18:28:49,716][20381] Updated weights for policy 1, policy_version 1850856 (0.0007) [2023-12-27 18:28:49,771][20381] Updated weights for policy 1, policy_version 1850866 (0.0007) [2023-12-27 18:28:50,014][20347] Updated weights for policy 0, policy_version 1852448 (0.0010) [2023-12-27 18:28:50,077][20347] Updated weights for policy 0, policy_version 1852458 (0.0010) [2023-12-27 18:28:50,142][20347] Updated weights for policy 0, policy_version 1852468 (0.0010) [2023-12-27 18:28:50,438][20381] Updated weights for policy 1, policy_version 1850876 (0.0008) [2023-12-27 18:28:50,490][20381] Updated weights for policy 1, policy_version 1850886 (0.0009) [2023-12-27 18:28:50,550][20381] Updated weights for policy 1, policy_version 1850896 (0.0008) [2023-12-27 18:28:50,802][20347] Updated weights for policy 0, policy_version 1852478 (0.0009) [2023-12-27 18:28:50,864][20347] Updated weights for policy 0, policy_version 1852488 (0.0008) [2023-12-27 18:28:50,924][20347] Updated weights for policy 0, policy_version 1852498 (0.0008) [2023-12-27 18:28:51,253][20381] Updated weights for policy 1, policy_version 1850906 (0.0008) [2023-12-27 18:28:51,316][20381] Updated weights for policy 1, policy_version 1850916 (0.0009) [2023-12-27 18:28:51,381][20381] Updated weights for policy 1, policy_version 1850926 (0.0009) [2023-12-27 18:28:51,659][20347] Updated weights for policy 0, policy_version 1852508 (0.0007) [2023-12-27 18:28:51,714][20347] Updated weights for policy 0, policy_version 1852518 (0.0007) [2023-12-27 18:28:51,782][20347] Updated weights for policy 0, policy_version 1852529 (0.0011) [2023-12-27 18:28:52,065][20381] Updated weights for policy 1, policy_version 1850937 (0.0008) [2023-12-27 18:28:52,131][20381] Updated weights for policy 1, policy_version 1850947 (0.0006) [2023-12-27 18:28:52,190][20381] Updated weights for policy 1, policy_version 1850957 (0.0006) [2023-12-27 18:28:52,245][20381] Updated weights for policy 1, policy_version 1850967 (0.0006) [2023-12-27 18:28:52,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20188.4). Total num frames: 948232192. Throughput: 0: 10216.8, 1: 10267.5. Samples: 948202068. Policy #0 lag: (min: 14.0, avg: 38.2, max: 40.0) [2023-12-27 18:28:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '813.790')] [2023-12-27 18:28:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001850968_473899008.pth... [2023-12-27 18:28:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001852536_474333184.pth... [2023-12-27 18:28:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001849752_473587712.pth [2023-12-27 18:28:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001851352_474030080.pth [2023-12-27 18:28:52,544][20347] Updated weights for policy 0, policy_version 1852539 (0.0008) [2023-12-27 18:28:52,596][20347] Updated weights for policy 0, policy_version 1852549 (0.0008) [2023-12-27 18:28:52,646][20347] Updated weights for policy 0, policy_version 1852559 (0.0009) [2023-12-27 18:28:52,852][20381] Updated weights for policy 1, policy_version 1850977 (0.0006) [2023-12-27 18:28:52,907][20381] Updated weights for policy 1, policy_version 1850987 (0.0010) [2023-12-27 18:28:52,968][20381] Updated weights for policy 1, policy_version 1850997 (0.0011) [2023-12-27 18:28:53,327][20347] Updated weights for policy 0, policy_version 1852569 (0.0009) [2023-12-27 18:28:53,388][20347] Updated weights for policy 0, policy_version 1852579 (0.0008) [2023-12-27 18:28:53,434][20347] Updated weights for policy 0, policy_version 1852589 (0.0010) [2023-12-27 18:28:53,479][20347] Updated weights for policy 0, policy_version 1852599 (0.0009) [2023-12-27 18:28:53,574][20381] Updated weights for policy 1, policy_version 1851007 (0.0009) [2023-12-27 18:28:53,629][20381] Updated weights for policy 1, policy_version 1851017 (0.0011) [2023-12-27 18:28:53,683][20381] Updated weights for policy 1, policy_version 1851027 (0.0010) [2023-12-27 18:28:54,218][20347] Updated weights for policy 0, policy_version 1852609 (0.0007) [2023-12-27 18:28:54,265][20347] Updated weights for policy 0, policy_version 1852619 (0.0007) [2023-12-27 18:28:54,309][20381] Updated weights for policy 1, policy_version 1851037 (0.0011) [2023-12-27 18:28:54,318][20347] Updated weights for policy 0, policy_version 1852629 (0.0006) [2023-12-27 18:28:54,364][20381] Updated weights for policy 1, policy_version 1851047 (0.0011) [2023-12-27 18:28:54,419][20381] Updated weights for policy 1, policy_version 1851057 (0.0010) [2023-12-27 18:28:54,969][20347] Updated weights for policy 0, policy_version 1852639 (0.0006) [2023-12-27 18:28:55,022][20347] Updated weights for policy 0, policy_version 1852649 (0.0007) [2023-12-27 18:28:55,074][20347] Updated weights for policy 0, policy_version 1852659 (0.0006) [2023-12-27 18:28:55,086][20381] Updated weights for policy 1, policy_version 1851067 (0.0011) [2023-12-27 18:28:55,144][20381] Updated weights for policy 1, policy_version 1851077 (0.0009) [2023-12-27 18:28:55,199][20381] Updated weights for policy 1, policy_version 1851087 (0.0006) [2023-12-27 18:28:55,659][20347] Updated weights for policy 0, policy_version 1852669 (0.0005) [2023-12-27 18:28:55,707][20347] Updated weights for policy 0, policy_version 1852679 (0.0007) [2023-12-27 18:28:55,757][20347] Updated weights for policy 0, policy_version 1852689 (0.0007) [2023-12-27 18:28:55,872][20381] Updated weights for policy 1, policy_version 1851097 (0.0007) [2023-12-27 18:28:55,916][20381] Updated weights for policy 1, policy_version 1851107 (0.0008) [2023-12-27 18:28:55,966][20381] Updated weights for policy 1, policy_version 1851117 (0.0009) [2023-12-27 18:28:56,024][20381] Updated weights for policy 1, policy_version 1851127 (0.0005) [2023-12-27 18:28:56,452][20347] Updated weights for policy 0, policy_version 1852699 (0.0009) [2023-12-27 18:28:56,503][20347] Updated weights for policy 0, policy_version 1852709 (0.0010) [2023-12-27 18:28:56,550][20347] Updated weights for policy 0, policy_version 1852719 (0.0010) [2023-12-27 18:28:56,598][20381] Updated weights for policy 1, policy_version 1851137 (0.0007) [2023-12-27 18:28:56,646][20381] Updated weights for policy 1, policy_version 1851147 (0.0007) [2023-12-27 18:28:56,702][20381] Updated weights for policy 1, policy_version 1851157 (0.0006) [2023-12-27 18:28:57,264][20347] Updated weights for policy 0, policy_version 1852729 (0.0008) [2023-12-27 18:28:57,304][20381] Updated weights for policy 1, policy_version 1851167 (0.0006) [2023-12-27 18:28:57,323][20347] Updated weights for policy 0, policy_version 1852739 (0.0010) [2023-12-27 18:28:57,362][20381] Updated weights for policy 1, policy_version 1851177 (0.0007) [2023-12-27 18:28:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20216.2). Total num frames: 948330496. Throughput: 0: 10243.2, 1: 10287.7. Samples: 948326500. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:28:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '832.430')] [2023-12-27 18:28:57,377][20347] Updated weights for policy 0, policy_version 1852749 (0.0010) [2023-12-27 18:28:57,420][20381] Updated weights for policy 1, policy_version 1851187 (0.0007) [2023-12-27 18:28:57,438][20347] Updated weights for policy 0, policy_version 1852759 (0.0010) [2023-12-27 18:28:58,084][20381] Updated weights for policy 1, policy_version 1851197 (0.0009) [2023-12-27 18:28:58,103][20347] Updated weights for policy 0, policy_version 1852769 (0.0010) [2023-12-27 18:28:58,129][20381] Updated weights for policy 1, policy_version 1851207 (0.0010) [2023-12-27 18:28:58,161][20347] Updated weights for policy 0, policy_version 1852779 (0.0010) [2023-12-27 18:28:58,183][20381] Updated weights for policy 1, policy_version 1851217 (0.0010) [2023-12-27 18:28:58,215][20347] Updated weights for policy 0, policy_version 1852789 (0.0009) [2023-12-27 18:28:58,951][20381] Updated weights for policy 1, policy_version 1851227 (0.0006) [2023-12-27 18:28:58,970][20347] Updated weights for policy 0, policy_version 1852799 (0.0006) [2023-12-27 18:28:59,009][20381] Updated weights for policy 1, policy_version 1851237 (0.0008) [2023-12-27 18:28:59,026][20347] Updated weights for policy 0, policy_version 1852809 (0.0008) [2023-12-27 18:28:59,068][20381] Updated weights for policy 1, policy_version 1851247 (0.0006) [2023-12-27 18:28:59,082][20347] Updated weights for policy 0, policy_version 1852819 (0.0006) [2023-12-27 18:28:59,695][20347] Updated weights for policy 0, policy_version 1852829 (0.0006) [2023-12-27 18:28:59,745][20347] Updated weights for policy 0, policy_version 1852839 (0.0005) [2023-12-27 18:28:59,797][20347] Updated weights for policy 0, policy_version 1852849 (0.0007) [2023-12-27 18:28:59,850][20381] Updated weights for policy 1, policy_version 1851257 (0.0007) [2023-12-27 18:28:59,909][20381] Updated weights for policy 1, policy_version 1851267 (0.0008) [2023-12-27 18:28:59,972][20381] Updated weights for policy 1, policy_version 1851277 (0.0007) [2023-12-27 18:29:00,032][20381] Updated weights for policy 1, policy_version 1851287 (0.0006) [2023-12-27 18:29:00,462][20347] Updated weights for policy 0, policy_version 1852859 (0.0009) [2023-12-27 18:29:00,511][20347] Updated weights for policy 0, policy_version 1852869 (0.0010) [2023-12-27 18:29:00,556][20347] Updated weights for policy 0, policy_version 1852879 (0.0010) [2023-12-27 18:29:00,641][20381] Updated weights for policy 1, policy_version 1851297 (0.0007) [2023-12-27 18:29:00,688][20381] Updated weights for policy 1, policy_version 1851307 (0.0007) [2023-12-27 18:29:00,741][20381] Updated weights for policy 1, policy_version 1851317 (0.0007) [2023-12-27 18:29:01,317][20347] Updated weights for policy 0, policy_version 1852889 (0.0008) [2023-12-27 18:29:01,406][20347] Updated weights for policy 0, policy_version 1852899 (0.0037) [2023-12-27 18:29:01,438][20381] Updated weights for policy 1, policy_version 1851327 (0.0006) [2023-12-27 18:29:01,454][20347] Updated weights for policy 0, policy_version 1852909 (0.0008) [2023-12-27 18:29:01,495][20381] Updated weights for policy 1, policy_version 1851337 (0.0010) [2023-12-27 18:29:01,506][20347] Updated weights for policy 0, policy_version 1852919 (0.0008) [2023-12-27 18:29:01,551][20381] Updated weights for policy 1, policy_version 1851347 (0.0010) [2023-12-27 18:29:02,156][20381] Updated weights for policy 1, policy_version 1851357 (0.0011) [2023-12-27 18:29:02,202][20381] Updated weights for policy 1, policy_version 1851367 (0.0008) [2023-12-27 18:29:02,250][20381] Updated weights for policy 1, policy_version 1851377 (0.0006) [2023-12-27 18:29:02,276][20347] Updated weights for policy 0, policy_version 1852929 (0.0007) [2023-12-27 18:29:02,334][20347] Updated weights for policy 0, policy_version 1852939 (0.0008) [2023-12-27 18:29:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 948436992. Throughput: 0: 10194.6, 1: 10331.9. Samples: 948448768. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '841.210')] [2023-12-27 18:29:02,396][20347] Updated weights for policy 0, policy_version 1852949 (0.0008) [2023-12-27 18:29:02,859][20381] Updated weights for policy 1, policy_version 1851387 (0.0007) [2023-12-27 18:29:02,903][20381] Updated weights for policy 1, policy_version 1851397 (0.0007) [2023-12-27 18:29:02,950][20381] Updated weights for policy 1, policy_version 1851407 (0.0007) [2023-12-27 18:29:03,101][20347] Updated weights for policy 0, policy_version 1852959 (0.0010) [2023-12-27 18:29:03,159][20347] Updated weights for policy 0, policy_version 1852969 (0.0010) [2023-12-27 18:29:03,211][20347] Updated weights for policy 0, policy_version 1852979 (0.0010) [2023-12-27 18:29:03,642][20381] Updated weights for policy 1, policy_version 1851417 (0.0008) [2023-12-27 18:29:03,706][20381] Updated weights for policy 1, policy_version 1851427 (0.0011) [2023-12-27 18:29:03,764][20381] Updated weights for policy 1, policy_version 1851437 (0.0010) [2023-12-27 18:29:03,793][20347] Updated weights for policy 0, policy_version 1852989 (0.0005) [2023-12-27 18:29:03,824][20381] Updated weights for policy 1, policy_version 1851447 (0.0007) [2023-12-27 18:29:03,853][20347] Updated weights for policy 0, policy_version 1852999 (0.0007) [2023-12-27 18:29:03,907][20347] Updated weights for policy 0, policy_version 1853009 (0.0010) [2023-12-27 18:29:04,403][20381] Updated weights for policy 1, policy_version 1851457 (0.0006) [2023-12-27 18:29:04,454][20381] Updated weights for policy 1, policy_version 1851467 (0.0006) [2023-12-27 18:29:04,511][20381] Updated weights for policy 1, policy_version 1851477 (0.0008) [2023-12-27 18:29:04,576][20347] Updated weights for policy 0, policy_version 1853019 (0.0009) [2023-12-27 18:29:04,637][20347] Updated weights for policy 0, policy_version 1853029 (0.0007) [2023-12-27 18:29:04,700][20347] Updated weights for policy 0, policy_version 1853039 (0.0006) [2023-12-27 18:29:05,163][20381] Updated weights for policy 1, policy_version 1851487 (0.0008) [2023-12-27 18:29:05,218][20381] Updated weights for policy 1, policy_version 1851497 (0.0007) [2023-12-27 18:29:05,262][20381] Updated weights for policy 1, policy_version 1851507 (0.0008) [2023-12-27 18:29:05,344][20347] Updated weights for policy 0, policy_version 1853049 (0.0006) [2023-12-27 18:29:05,407][20347] Updated weights for policy 0, policy_version 1853059 (0.0009) [2023-12-27 18:29:05,470][20347] Updated weights for policy 0, policy_version 1853069 (0.0006) [2023-12-27 18:29:05,528][20347] Updated weights for policy 0, policy_version 1853079 (0.0006) [2023-12-27 18:29:06,011][20381] Updated weights for policy 1, policy_version 1851517 (0.0006) [2023-12-27 18:29:06,063][20381] Updated weights for policy 1, policy_version 1851527 (0.0005) [2023-12-27 18:29:06,119][20347] Updated weights for policy 0, policy_version 1853089 (0.0007) [2023-12-27 18:29:06,125][20381] Updated weights for policy 1, policy_version 1851537 (0.0007) [2023-12-27 18:29:06,180][20347] Updated weights for policy 0, policy_version 1853099 (0.0006) [2023-12-27 18:29:06,240][20347] Updated weights for policy 0, policy_version 1853109 (0.0007) [2023-12-27 18:29:06,811][20381] Updated weights for policy 1, policy_version 1851547 (0.0006) [2023-12-27 18:29:06,875][20381] Updated weights for policy 1, policy_version 1851557 (0.0007) [2023-12-27 18:29:06,895][20347] Updated weights for policy 0, policy_version 1853119 (0.0006) [2023-12-27 18:29:06,931][20381] Updated weights for policy 1, policy_version 1851567 (0.0011) [2023-12-27 18:29:06,951][20347] Updated weights for policy 0, policy_version 1853129 (0.0005) [2023-12-27 18:29:07,006][20347] Updated weights for policy 0, policy_version 1853139 (0.0006) [2023-12-27 18:29:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.7, 300 sec: 20244.0). Total num frames: 948543488. Throughput: 0: 10213.1, 1: 10389.8. Samples: 948513496. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '889.790')] [2023-12-27 18:29:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001851576_474054656.pth... [2023-12-27 18:29:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001853144_474488832.pth... [2023-12-27 18:29:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001851960_474185728.pth [2023-12-27 18:29:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001850360_473743360.pth [2023-12-27 18:29:07,574][20381] Updated weights for policy 1, policy_version 1851577 (0.0011) [2023-12-27 18:29:07,629][20381] Updated weights for policy 1, policy_version 1851587 (0.0011) [2023-12-27 18:29:07,679][20347] Updated weights for policy 0, policy_version 1853149 (0.0009) [2023-12-27 18:29:07,681][20381] Updated weights for policy 1, policy_version 1851597 (0.0010) [2023-12-27 18:29:07,736][20381] Updated weights for policy 1, policy_version 1851607 (0.0010) [2023-12-27 18:29:07,741][20347] Updated weights for policy 0, policy_version 1853159 (0.0007) [2023-12-27 18:29:07,795][20347] Updated weights for policy 0, policy_version 1853169 (0.0010) [2023-12-27 18:29:08,368][20381] Updated weights for policy 1, policy_version 1851617 (0.0008) [2023-12-27 18:29:08,427][20381] Updated weights for policy 1, policy_version 1851627 (0.0008) [2023-12-27 18:29:08,466][20347] Updated weights for policy 0, policy_version 1853179 (0.0010) [2023-12-27 18:29:08,485][20381] Updated weights for policy 1, policy_version 1851637 (0.0008) [2023-12-27 18:29:08,528][20347] Updated weights for policy 0, policy_version 1853189 (0.0010) [2023-12-27 18:29:08,589][20347] Updated weights for policy 0, policy_version 1853199 (0.0010) [2023-12-27 18:29:09,182][20381] Updated weights for policy 1, policy_version 1851647 (0.0008) [2023-12-27 18:29:09,242][20381] Updated weights for policy 1, policy_version 1851657 (0.0008) [2023-12-27 18:29:09,248][20347] Updated weights for policy 0, policy_version 1853209 (0.0010) [2023-12-27 18:29:09,303][20381] Updated weights for policy 1, policy_version 1851667 (0.0007) [2023-12-27 18:29:09,316][20347] Updated weights for policy 0, policy_version 1853219 (0.0008) [2023-12-27 18:29:09,390][20347] Updated weights for policy 0, policy_version 1853229 (0.0009) [2023-12-27 18:29:09,454][20347] Updated weights for policy 0, policy_version 1853239 (0.0007) [2023-12-27 18:29:09,980][20381] Updated weights for policy 1, policy_version 1851677 (0.0006) [2023-12-27 18:29:10,037][20381] Updated weights for policy 1, policy_version 1851687 (0.0007) [2023-12-27 18:29:10,099][20381] Updated weights for policy 1, policy_version 1851697 (0.0007) [2023-12-27 18:29:10,180][20347] Updated weights for policy 0, policy_version 1853249 (0.0008) [2023-12-27 18:29:10,239][20347] Updated weights for policy 0, policy_version 1853259 (0.0006) [2023-12-27 18:29:10,290][20347] Updated weights for policy 0, policy_version 1853269 (0.0007) [2023-12-27 18:29:10,759][20381] Updated weights for policy 1, policy_version 1851707 (0.0009) [2023-12-27 18:29:10,825][20381] Updated weights for policy 1, policy_version 1851717 (0.0011) [2023-12-27 18:29:10,883][20381] Updated weights for policy 1, policy_version 1851727 (0.0010) [2023-12-27 18:29:11,035][20347] Updated weights for policy 0, policy_version 1853280 (0.0009) [2023-12-27 18:29:11,094][20347] Updated weights for policy 0, policy_version 1853290 (0.0008) [2023-12-27 18:29:11,147][20347] Updated weights for policy 0, policy_version 1853300 (0.0008) [2023-12-27 18:29:11,585][20381] Updated weights for policy 1, policy_version 1851737 (0.0006) [2023-12-27 18:29:11,656][20381] Updated weights for policy 1, policy_version 1851747 (0.0008) [2023-12-27 18:29:11,717][20381] Updated weights for policy 1, policy_version 1851757 (0.0006) [2023-12-27 18:29:11,786][20381] Updated weights for policy 1, policy_version 1851767 (0.0007) [2023-12-27 18:29:11,903][20347] Updated weights for policy 0, policy_version 1853310 (0.0008) [2023-12-27 18:29:11,966][20347] Updated weights for policy 0, policy_version 1853320 (0.0008) [2023-12-27 18:29:12,025][20347] Updated weights for policy 0, policy_version 1853330 (0.0006) [2023-12-27 18:29:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20244.0). Total num frames: 948641792. Throughput: 0: 10242.0, 1: 10339.6. Samples: 948635472. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '928.380')] [2023-12-27 18:29:12,470][20381] Updated weights for policy 1, policy_version 1851777 (0.0006) [2023-12-27 18:29:12,530][20381] Updated weights for policy 1, policy_version 1851787 (0.0005) [2023-12-27 18:29:12,593][20381] Updated weights for policy 1, policy_version 1851797 (0.0006) [2023-12-27 18:29:12,655][20347] Updated weights for policy 0, policy_version 1853340 (0.0006) [2023-12-27 18:29:12,708][20347] Updated weights for policy 0, policy_version 1853350 (0.0007) [2023-12-27 18:29:12,759][20347] Updated weights for policy 0, policy_version 1853360 (0.0007) [2023-12-27 18:29:13,246][20381] Updated weights for policy 1, policy_version 1851807 (0.0010) [2023-12-27 18:29:13,304][20381] Updated weights for policy 1, policy_version 1851817 (0.0011) [2023-12-27 18:29:13,354][20381] Updated weights for policy 1, policy_version 1851827 (0.0007) [2023-12-27 18:29:13,416][20347] Updated weights for policy 0, policy_version 1853370 (0.0008) [2023-12-27 18:29:13,496][20347] Updated weights for policy 0, policy_version 1853380 (0.0007) [2023-12-27 18:29:13,553][20347] Updated weights for policy 0, policy_version 1853390 (0.0007) [2023-12-27 18:29:13,611][20347] Updated weights for policy 0, policy_version 1853400 (0.0007) [2023-12-27 18:29:14,022][20381] Updated weights for policy 1, policy_version 1851837 (0.0009) [2023-12-27 18:29:14,080][20381] Updated weights for policy 1, policy_version 1851847 (0.0010) [2023-12-27 18:29:14,142][20381] Updated weights for policy 1, policy_version 1851857 (0.0010) [2023-12-27 18:29:14,291][20347] Updated weights for policy 0, policy_version 1853410 (0.0007) [2023-12-27 18:29:14,346][20347] Updated weights for policy 0, policy_version 1853420 (0.0008) [2023-12-27 18:29:14,405][20347] Updated weights for policy 0, policy_version 1853430 (0.0007) [2023-12-27 18:29:14,816][20381] Updated weights for policy 1, policy_version 1851867 (0.0009) [2023-12-27 18:29:14,868][20381] Updated weights for policy 1, policy_version 1851877 (0.0007) [2023-12-27 18:29:14,927][20381] Updated weights for policy 1, policy_version 1851887 (0.0008) [2023-12-27 18:29:15,044][20347] Updated weights for policy 0, policy_version 1853440 (0.0007) [2023-12-27 18:29:15,096][20347] Updated weights for policy 0, policy_version 1853450 (0.0007) [2023-12-27 18:29:15,152][20347] Updated weights for policy 0, policy_version 1853460 (0.0007) [2023-12-27 18:29:15,638][20381] Updated weights for policy 1, policy_version 1851897 (0.0008) [2023-12-27 18:29:15,691][20381] Updated weights for policy 1, policy_version 1851907 (0.0009) [2023-12-27 18:29:15,740][20381] Updated weights for policy 1, policy_version 1851917 (0.0007) [2023-12-27 18:29:15,789][20381] Updated weights for policy 1, policy_version 1851927 (0.0009) [2023-12-27 18:29:15,830][20347] Updated weights for policy 0, policy_version 1853470 (0.0010) [2023-12-27 18:29:15,894][20347] Updated weights for policy 0, policy_version 1853480 (0.0010) [2023-12-27 18:29:15,948][20347] Updated weights for policy 0, policy_version 1853490 (0.0010) [2023-12-27 18:29:16,480][20381] Updated weights for policy 1, policy_version 1851937 (0.0006) [2023-12-27 18:29:16,535][20381] Updated weights for policy 1, policy_version 1851947 (0.0005) [2023-12-27 18:29:16,552][20347] Updated weights for policy 0, policy_version 1853500 (0.0010) [2023-12-27 18:29:16,588][20381] Updated weights for policy 1, policy_version 1851957 (0.0005) [2023-12-27 18:29:16,598][20347] Updated weights for policy 0, policy_version 1853510 (0.0010) [2023-12-27 18:29:16,650][20347] Updated weights for policy 0, policy_version 1853520 (0.0010) [2023-12-27 18:29:17,229][20381] Updated weights for policy 1, policy_version 1851967 (0.0005) [2023-12-27 18:29:17,273][20347] Updated weights for policy 0, policy_version 1853530 (0.0010) [2023-12-27 18:29:17,292][20381] Updated weights for policy 1, policy_version 1851977 (0.0006) [2023-12-27 18:29:17,333][20347] Updated weights for policy 0, policy_version 1853540 (0.0010) [2023-12-27 18:29:17,352][20381] Updated weights for policy 1, policy_version 1851987 (0.0006) [2023-12-27 18:29:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20188.4). Total num frames: 948740096. Throughput: 0: 10236.5, 1: 10347.9. Samples: 948758748. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '877.230')] [2023-12-27 18:29:17,383][20347] Updated weights for policy 0, policy_version 1853550 (0.0008) [2023-12-27 18:29:17,439][20347] Updated weights for policy 0, policy_version 1853560 (0.0005) [2023-12-27 18:29:17,953][20381] Updated weights for policy 1, policy_version 1851997 (0.0007) [2023-12-27 18:29:18,019][20381] Updated weights for policy 1, policy_version 1852007 (0.0005) [2023-12-27 18:29:18,027][20347] Updated weights for policy 0, policy_version 1853570 (0.0010) [2023-12-27 18:29:18,077][20381] Updated weights for policy 1, policy_version 1852017 (0.0007) [2023-12-27 18:29:18,082][20347] Updated weights for policy 0, policy_version 1853580 (0.0010) [2023-12-27 18:29:18,137][20347] Updated weights for policy 0, policy_version 1853590 (0.0010) [2023-12-27 18:29:18,698][20381] Updated weights for policy 1, policy_version 1852027 (0.0008) [2023-12-27 18:29:18,761][20347] Updated weights for policy 0, policy_version 1853600 (0.0007) [2023-12-27 18:29:18,763][20381] Updated weights for policy 1, policy_version 1852037 (0.0007) [2023-12-27 18:29:18,818][20347] Updated weights for policy 0, policy_version 1853610 (0.0007) [2023-12-27 18:29:18,822][20381] Updated weights for policy 1, policy_version 1852047 (0.0008) [2023-12-27 18:29:18,882][20347] Updated weights for policy 0, policy_version 1853620 (0.0006) [2023-12-27 18:29:19,542][20381] Updated weights for policy 1, policy_version 1852057 (0.0007) [2023-12-27 18:29:19,605][20381] Updated weights for policy 1, policy_version 1852067 (0.0009) [2023-12-27 18:29:19,656][20347] Updated weights for policy 0, policy_version 1853630 (0.0006) [2023-12-27 18:29:19,658][20381] Updated weights for policy 1, policy_version 1852077 (0.0009) [2023-12-27 18:29:19,714][20381] Updated weights for policy 1, policy_version 1852087 (0.0009) [2023-12-27 18:29:19,716][20347] Updated weights for policy 0, policy_version 1853640 (0.0006) [2023-12-27 18:29:19,772][20347] Updated weights for policy 0, policy_version 1853650 (0.0007) [2023-12-27 18:29:20,469][20347] Updated weights for policy 0, policy_version 1853660 (0.0007) [2023-12-27 18:29:20,487][20381] Updated weights for policy 1, policy_version 1852097 (0.0007) [2023-12-27 18:29:20,528][20347] Updated weights for policy 0, policy_version 1853670 (0.0006) [2023-12-27 18:29:20,540][20381] Updated weights for policy 1, policy_version 1852107 (0.0007) [2023-12-27 18:29:20,588][20347] Updated weights for policy 0, policy_version 1853680 (0.0009) [2023-12-27 18:29:20,600][20381] Updated weights for policy 1, policy_version 1852117 (0.0008) [2023-12-27 18:29:21,203][20347] Updated weights for policy 0, policy_version 1853690 (0.0006) [2023-12-27 18:29:21,269][20347] Updated weights for policy 0, policy_version 1853700 (0.0008) [2023-12-27 18:29:21,314][20381] Updated weights for policy 1, policy_version 1852127 (0.0007) [2023-12-27 18:29:21,331][20347] Updated weights for policy 0, policy_version 1853710 (0.0006) [2023-12-27 18:29:21,384][20381] Updated weights for policy 1, policy_version 1852137 (0.0009) [2023-12-27 18:29:21,407][20347] Updated weights for policy 0, policy_version 1853720 (0.0007) [2023-12-27 18:29:21,447][20381] Updated weights for policy 1, policy_version 1852147 (0.0010) [2023-12-27 18:29:22,091][20347] Updated weights for policy 0, policy_version 1853730 (0.0008) [2023-12-27 18:29:22,124][20381] Updated weights for policy 1, policy_version 1852157 (0.0011) [2023-12-27 18:29:22,156][20347] Updated weights for policy 0, policy_version 1853740 (0.0007) [2023-12-27 18:29:22,178][20381] Updated weights for policy 1, policy_version 1852167 (0.0011) [2023-12-27 18:29:22,213][20347] Updated weights for policy 0, policy_version 1853750 (0.0006) [2023-12-27 18:29:22,239][20381] Updated weights for policy 1, policy_version 1852177 (0.0011) [2023-12-27 18:29:22,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 948854784. Throughput: 0: 10260.4, 1: 10350.3. Samples: 948820696. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '861.140')] [2023-12-27 18:29:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001852184_474210304.pth... [2023-12-27 18:29:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001853752_474644480.pth... [2023-12-27 18:29:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001850968_473899008.pth [2023-12-27 18:29:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001852536_474333184.pth [2023-12-27 18:29:22,928][20347] Updated weights for policy 0, policy_version 1853760 (0.0008) [2023-12-27 18:29:22,941][20381] Updated weights for policy 1, policy_version 1852187 (0.0010) [2023-12-27 18:29:22,984][20347] Updated weights for policy 0, policy_version 1853770 (0.0005) [2023-12-27 18:29:23,000][20381] Updated weights for policy 1, policy_version 1852197 (0.0010) [2023-12-27 18:29:23,039][20347] Updated weights for policy 0, policy_version 1853780 (0.0005) [2023-12-27 18:29:23,064][20381] Updated weights for policy 1, policy_version 1852207 (0.0011) [2023-12-27 18:29:23,657][20347] Updated weights for policy 0, policy_version 1853790 (0.0005) [2023-12-27 18:29:23,716][20347] Updated weights for policy 0, policy_version 1853800 (0.0005) [2023-12-27 18:29:23,755][20381] Updated weights for policy 1, policy_version 1852217 (0.0011) [2023-12-27 18:29:23,772][20347] Updated weights for policy 0, policy_version 1853810 (0.0005) [2023-12-27 18:29:23,811][20381] Updated weights for policy 1, policy_version 1852227 (0.0011) [2023-12-27 18:29:23,866][20381] Updated weights for policy 1, policy_version 1852237 (0.0010) [2023-12-27 18:29:23,932][20381] Updated weights for policy 1, policy_version 1852247 (0.0009) [2023-12-27 18:29:24,293][20347] Updated weights for policy 0, policy_version 1853820 (0.0005) [2023-12-27 18:29:24,353][20347] Updated weights for policy 0, policy_version 1853830 (0.0007) [2023-12-27 18:29:24,407][20347] Updated weights for policy 0, policy_version 1853840 (0.0008) [2023-12-27 18:29:24,540][20381] Updated weights for policy 1, policy_version 1852257 (0.0006) [2023-12-27 18:29:24,589][20381] Updated weights for policy 1, policy_version 1852267 (0.0006) [2023-12-27 18:29:24,646][20381] Updated weights for policy 1, policy_version 1852277 (0.0005) [2023-12-27 18:29:25,068][20347] Updated weights for policy 0, policy_version 1853850 (0.0007) [2023-12-27 18:29:25,126][20347] Updated weights for policy 0, policy_version 1853860 (0.0006) [2023-12-27 18:29:25,180][20347] Updated weights for policy 0, policy_version 1853870 (0.0008) [2023-12-27 18:29:25,226][20347] Updated weights for policy 0, policy_version 1853880 (0.0007) [2023-12-27 18:29:25,230][20381] Updated weights for policy 1, policy_version 1852287 (0.0005) [2023-12-27 18:29:25,274][20381] Updated weights for policy 1, policy_version 1852297 (0.0005) [2023-12-27 18:29:25,329][20381] Updated weights for policy 1, policy_version 1852307 (0.0006) [2023-12-27 18:29:25,927][20347] Updated weights for policy 0, policy_version 1853890 (0.0006) [2023-12-27 18:29:25,944][20381] Updated weights for policy 1, policy_version 1852317 (0.0008) [2023-12-27 18:29:25,983][20347] Updated weights for policy 0, policy_version 1853900 (0.0005) [2023-12-27 18:29:26,006][20381] Updated weights for policy 1, policy_version 1852327 (0.0011) [2023-12-27 18:29:26,041][20347] Updated weights for policy 0, policy_version 1853910 (0.0006) [2023-12-27 18:29:26,058][20381] Updated weights for policy 1, policy_version 1852337 (0.0010) [2023-12-27 18:29:26,562][20347] Updated weights for policy 0, policy_version 1853920 (0.0007) [2023-12-27 18:29:26,607][20347] Updated weights for policy 0, policy_version 1853930 (0.0010) [2023-12-27 18:29:26,671][20347] Updated weights for policy 0, policy_version 1853940 (0.0010) [2023-12-27 18:29:26,693][20381] Updated weights for policy 1, policy_version 1852347 (0.0009) [2023-12-27 18:29:26,746][20381] Updated weights for policy 1, policy_version 1852357 (0.0005) [2023-12-27 18:29:26,794][20381] Updated weights for policy 1, policy_version 1852367 (0.0006) [2023-12-27 18:29:27,321][20347] Updated weights for policy 0, policy_version 1853950 (0.0009) [2023-12-27 18:29:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 948953088. Throughput: 0: 10288.6, 1: 10371.4. Samples: 948945428. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:27,363][19170] Avg episode reward: [(0, '990.270'), (1, '871.760')] [2023-12-27 18:29:27,376][20347] Updated weights for policy 0, policy_version 1853960 (0.0007) [2023-12-27 18:29:27,435][20347] Updated weights for policy 0, policy_version 1853970 (0.0010) [2023-12-27 18:29:27,482][20381] Updated weights for policy 1, policy_version 1852377 (0.0007) [2023-12-27 18:29:27,542][20381] Updated weights for policy 1, policy_version 1852387 (0.0008) [2023-12-27 18:29:27,602][20381] Updated weights for policy 1, policy_version 1852397 (0.0008) [2023-12-27 18:29:27,658][20381] Updated weights for policy 1, policy_version 1852407 (0.0008) [2023-12-27 18:29:28,065][20347] Updated weights for policy 0, policy_version 1853980 (0.0008) [2023-12-27 18:29:28,111][20347] Updated weights for policy 0, policy_version 1853990 (0.0005) [2023-12-27 18:29:28,155][20347] Updated weights for policy 0, policy_version 1854000 (0.0005) [2023-12-27 18:29:28,367][20381] Updated weights for policy 1, policy_version 1852417 (0.0008) [2023-12-27 18:29:28,426][20381] Updated weights for policy 1, policy_version 1852427 (0.0007) [2023-12-27 18:29:28,479][20381] Updated weights for policy 1, policy_version 1852437 (0.0008) [2023-12-27 18:29:28,789][20347] Updated weights for policy 0, policy_version 1854010 (0.0006) [2023-12-27 18:29:28,846][20347] Updated weights for policy 0, policy_version 1854020 (0.0008) [2023-12-27 18:29:28,899][20347] Updated weights for policy 0, policy_version 1854030 (0.0008) [2023-12-27 18:29:28,954][20347] Updated weights for policy 0, policy_version 1854040 (0.0008) [2023-12-27 18:29:29,209][20381] Updated weights for policy 1, policy_version 1852447 (0.0008) [2023-12-27 18:29:29,274][20381] Updated weights for policy 1, policy_version 1852457 (0.0007) [2023-12-27 18:29:29,332][20381] Updated weights for policy 1, policy_version 1852467 (0.0007) [2023-12-27 18:29:29,678][20347] Updated weights for policy 0, policy_version 1854050 (0.0007) [2023-12-27 18:29:29,729][20347] Updated weights for policy 0, policy_version 1854060 (0.0007) [2023-12-27 18:29:29,791][20347] Updated weights for policy 0, policy_version 1854070 (0.0007) [2023-12-27 18:29:30,065][20381] Updated weights for policy 1, policy_version 1852477 (0.0008) [2023-12-27 18:29:30,127][20381] Updated weights for policy 1, policy_version 1852487 (0.0008) [2023-12-27 18:29:30,189][20381] Updated weights for policy 1, policy_version 1852497 (0.0005) [2023-12-27 18:29:30,487][20347] Updated weights for policy 0, policy_version 1854080 (0.0008) [2023-12-27 18:29:30,537][20347] Updated weights for policy 0, policy_version 1854090 (0.0007) [2023-12-27 18:29:30,593][20347] Updated weights for policy 0, policy_version 1854100 (0.0008) [2023-12-27 18:29:30,725][20381] Updated weights for policy 1, policy_version 1852507 (0.0005) [2023-12-27 18:29:30,771][20381] Updated weights for policy 1, policy_version 1852517 (0.0005) [2023-12-27 18:29:30,817][20381] Updated weights for policy 1, policy_version 1852527 (0.0005) [2023-12-27 18:29:31,383][20347] Updated weights for policy 0, policy_version 1854110 (0.0009) [2023-12-27 18:29:31,443][20347] Updated weights for policy 0, policy_version 1854120 (0.0008) [2023-12-27 18:29:31,495][20347] Updated weights for policy 0, policy_version 1854130 (0.0008) [2023-12-27 18:29:31,545][20381] Updated weights for policy 1, policy_version 1852537 (0.0007) [2023-12-27 18:29:31,615][20381] Updated weights for policy 1, policy_version 1852547 (0.0008) [2023-12-27 18:29:31,680][20381] Updated weights for policy 1, policy_version 1852557 (0.0008) [2023-12-27 18:29:31,741][20381] Updated weights for policy 1, policy_version 1852567 (0.0008) [2023-12-27 18:29:32,188][20347] Updated weights for policy 0, policy_version 1854140 (0.0008) [2023-12-27 18:29:32,242][20347] Updated weights for policy 0, policy_version 1854150 (0.0007) [2023-12-27 18:29:32,302][20347] Updated weights for policy 0, policy_version 1854160 (0.0008) [2023-12-27 18:29:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20216.2). Total num frames: 949059584. Throughput: 0: 10297.9, 1: 10337.8. Samples: 949069364. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:32,363][19170] Avg episode reward: [(0, '990.270'), (1, '822.930')] [2023-12-27 18:29:32,453][20381] Updated weights for policy 1, policy_version 1852577 (0.0008) [2023-12-27 18:29:32,507][20381] Updated weights for policy 1, policy_version 1852587 (0.0008) [2023-12-27 18:29:32,564][20381] Updated weights for policy 1, policy_version 1852597 (0.0008) [2023-12-27 18:29:32,972][20347] Updated weights for policy 0, policy_version 1854170 (0.0007) [2023-12-27 18:29:33,026][20347] Updated weights for policy 0, policy_version 1854180 (0.0006) [2023-12-27 18:29:33,074][20347] Updated weights for policy 0, policy_version 1854190 (0.0005) [2023-12-27 18:29:33,133][20347] Updated weights for policy 0, policy_version 1854200 (0.0005) [2023-12-27 18:29:33,296][20381] Updated weights for policy 1, policy_version 1852607 (0.0007) [2023-12-27 18:29:33,346][20381] Updated weights for policy 1, policy_version 1852617 (0.0008) [2023-12-27 18:29:33,392][20381] Updated weights for policy 1, policy_version 1852628 (0.0009) [2023-12-27 18:29:33,707][20347] Updated weights for policy 0, policy_version 1854210 (0.0008) [2023-12-27 18:29:33,752][20347] Updated weights for policy 0, policy_version 1854220 (0.0007) [2023-12-27 18:29:33,797][20347] Updated weights for policy 0, policy_version 1854230 (0.0010) [2023-12-27 18:29:34,046][20381] Updated weights for policy 1, policy_version 1852638 (0.0008) [2023-12-27 18:29:34,094][20381] Updated weights for policy 1, policy_version 1852648 (0.0007) [2023-12-27 18:29:34,144][20381] Updated weights for policy 1, policy_version 1852658 (0.0007) [2023-12-27 18:29:34,500][20347] Updated weights for policy 0, policy_version 1854240 (0.0010) [2023-12-27 18:29:34,563][20347] Updated weights for policy 0, policy_version 1854250 (0.0010) [2023-12-27 18:29:34,624][20347] Updated weights for policy 0, policy_version 1854260 (0.0011) [2023-12-27 18:29:34,883][20381] Updated weights for policy 1, policy_version 1852668 (0.0008) [2023-12-27 18:29:34,948][20381] Updated weights for policy 1, policy_version 1852678 (0.0008) [2023-12-27 18:29:34,997][20381] Updated weights for policy 1, policy_version 1852688 (0.0009) [2023-12-27 18:29:35,217][20347] Updated weights for policy 0, policy_version 1854270 (0.0007) [2023-12-27 18:29:35,268][20347] Updated weights for policy 0, policy_version 1854280 (0.0006) [2023-12-27 18:29:35,313][20347] Updated weights for policy 0, policy_version 1854290 (0.0010) [2023-12-27 18:29:35,766][20381] Updated weights for policy 1, policy_version 1852698 (0.0008) [2023-12-27 18:29:35,827][20381] Updated weights for policy 1, policy_version 1852708 (0.0008) [2023-12-27 18:29:35,869][20347] Updated weights for policy 0, policy_version 1854300 (0.0010) [2023-12-27 18:29:35,885][20381] Updated weights for policy 1, policy_version 1852718 (0.0007) [2023-12-27 18:29:35,925][20347] Updated weights for policy 0, policy_version 1854310 (0.0008) [2023-12-27 18:29:35,941][20381] Updated weights for policy 1, policy_version 1852728 (0.0005) [2023-12-27 18:29:35,974][20347] Updated weights for policy 0, policy_version 1854320 (0.0009) [2023-12-27 18:29:36,596][20347] Updated weights for policy 0, policy_version 1854330 (0.0005) [2023-12-27 18:29:36,657][20347] Updated weights for policy 0, policy_version 1854340 (0.0007) [2023-12-27 18:29:36,675][20381] Updated weights for policy 1, policy_version 1852738 (0.0011) [2023-12-27 18:29:36,718][20347] Updated weights for policy 0, policy_version 1854350 (0.0007) [2023-12-27 18:29:36,735][20381] Updated weights for policy 1, policy_version 1852748 (0.0011) [2023-12-27 18:29:36,775][20347] Updated weights for policy 0, policy_version 1854360 (0.0007) [2023-12-27 18:29:36,793][20381] Updated weights for policy 1, policy_version 1852758 (0.0011) [2023-12-27 18:29:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.1, 300 sec: 20216.2). Total num frames: 949157888. Throughput: 0: 10337.9, 1: 10307.2. Samples: 949131096. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '832.490')] [2023-12-27 18:29:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001852760_474357760.pth... [2023-12-27 18:29:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001851576_474054656.pth [2023-12-27 18:29:37,427][20347] Updated weights for policy 0, policy_version 1854370 (0.0010) [2023-12-27 18:29:37,489][20381] Updated weights for policy 1, policy_version 1852768 (0.0011) [2023-12-27 18:29:37,494][20347] Updated weights for policy 0, policy_version 1854380 (0.0009) [2023-12-27 18:29:37,540][20381] Updated weights for policy 1, policy_version 1852778 (0.0011) [2023-12-27 18:29:37,554][20347] Updated weights for policy 0, policy_version 1854390 (0.0007) [2023-12-27 18:29:37,561][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001854392_474808320.pth... [2023-12-27 18:29:37,564][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001853144_474488832.pth [2023-12-27 18:29:37,587][20381] Updated weights for policy 1, policy_version 1852788 (0.0011) [2023-12-27 18:29:38,231][20347] Updated weights for policy 0, policy_version 1854400 (0.0007) [2023-12-27 18:29:38,285][20347] Updated weights for policy 0, policy_version 1854410 (0.0007) [2023-12-27 18:29:38,319][20381] Updated weights for policy 1, policy_version 1852798 (0.0011) [2023-12-27 18:29:38,339][20347] Updated weights for policy 0, policy_version 1854420 (0.0007) [2023-12-27 18:29:38,380][20381] Updated weights for policy 1, policy_version 1852808 (0.0010) [2023-12-27 18:29:38,437][20381] Updated weights for policy 1, policy_version 1852818 (0.0011) [2023-12-27 18:29:39,048][20347] Updated weights for policy 0, policy_version 1854430 (0.0008) [2023-12-27 18:29:39,085][20381] Updated weights for policy 1, policy_version 1852828 (0.0009) [2023-12-27 18:29:39,095][20347] Updated weights for policy 0, policy_version 1854440 (0.0007) [2023-12-27 18:29:39,142][20381] Updated weights for policy 1, policy_version 1852838 (0.0005) [2023-12-27 18:29:39,154][20347] Updated weights for policy 0, policy_version 1854450 (0.0006) [2023-12-27 18:29:39,199][20381] Updated weights for policy 1, policy_version 1852848 (0.0007) [2023-12-27 18:29:39,931][20347] Updated weights for policy 0, policy_version 1854460 (0.0008) [2023-12-27 18:29:39,964][20381] Updated weights for policy 1, policy_version 1852858 (0.0008) [2023-12-27 18:29:39,997][20347] Updated weights for policy 0, policy_version 1854470 (0.0007) [2023-12-27 18:29:40,028][20381] Updated weights for policy 1, policy_version 1852868 (0.0006) [2023-12-27 18:29:40,057][20347] Updated weights for policy 0, policy_version 1854480 (0.0007) [2023-12-27 18:29:40,086][20381] Updated weights for policy 1, policy_version 1852878 (0.0006) [2023-12-27 18:29:40,148][20381] Updated weights for policy 1, policy_version 1852888 (0.0009) [2023-12-27 18:29:40,736][20347] Updated weights for policy 0, policy_version 1854490 (0.0007) [2023-12-27 18:29:40,797][20347] Updated weights for policy 0, policy_version 1854500 (0.0007) [2023-12-27 18:29:40,850][20347] Updated weights for policy 0, policy_version 1854510 (0.0006) [2023-12-27 18:29:40,867][20381] Updated weights for policy 1, policy_version 1852898 (0.0006) [2023-12-27 18:29:40,907][20347] Updated weights for policy 0, policy_version 1854520 (0.0007) [2023-12-27 18:29:40,915][20381] Updated weights for policy 1, policy_version 1852908 (0.0007) [2023-12-27 18:29:40,966][20381] Updated weights for policy 1, policy_version 1852918 (0.0009) [2023-12-27 18:29:41,655][20347] Updated weights for policy 0, policy_version 1854530 (0.0008) [2023-12-27 18:29:41,719][20347] Updated weights for policy 0, policy_version 1854540 (0.0008) [2023-12-27 18:29:41,765][20381] Updated weights for policy 1, policy_version 1852928 (0.0008) [2023-12-27 18:29:41,787][20347] Updated weights for policy 0, policy_version 1854550 (0.0008) [2023-12-27 18:29:41,818][20381] Updated weights for policy 1, policy_version 1852938 (0.0008) [2023-12-27 18:29:41,878][20381] Updated weights for policy 1, policy_version 1852948 (0.0008) [2023-12-27 18:29:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20216.2). Total num frames: 949256192. Throughput: 0: 10368.3, 1: 10174.0. Samples: 949250908. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '784.680')] [2023-12-27 18:29:42,469][20347] Updated weights for policy 0, policy_version 1854560 (0.0008) [2023-12-27 18:29:42,529][20347] Updated weights for policy 0, policy_version 1854570 (0.0009) [2023-12-27 18:29:42,560][20381] Updated weights for policy 1, policy_version 1852958 (0.0007) [2023-12-27 18:29:42,591][20347] Updated weights for policy 0, policy_version 1854580 (0.0006) [2023-12-27 18:29:42,617][20381] Updated weights for policy 1, policy_version 1852968 (0.0007) [2023-12-27 18:29:42,666][20381] Updated weights for policy 1, policy_version 1852978 (0.0008) [2023-12-27 18:29:43,292][20381] Updated weights for policy 1, policy_version 1852988 (0.0008) [2023-12-27 18:29:43,317][20347] Updated weights for policy 0, policy_version 1854590 (0.0006) [2023-12-27 18:29:43,350][20381] Updated weights for policy 1, policy_version 1852998 (0.0007) [2023-12-27 18:29:43,374][20347] Updated weights for policy 0, policy_version 1854600 (0.0006) [2023-12-27 18:29:43,406][20381] Updated weights for policy 1, policy_version 1853008 (0.0006) [2023-12-27 18:29:43,427][20347] Updated weights for policy 0, policy_version 1854610 (0.0006) [2023-12-27 18:29:44,028][20381] Updated weights for policy 1, policy_version 1853018 (0.0006) [2023-12-27 18:29:44,083][20381] Updated weights for policy 1, policy_version 1853028 (0.0008) [2023-12-27 18:29:44,098][20347] Updated weights for policy 0, policy_version 1854620 (0.0006) [2023-12-27 18:29:44,140][20381] Updated weights for policy 1, policy_version 1853038 (0.0009) [2023-12-27 18:29:44,158][20347] Updated weights for policy 0, policy_version 1854630 (0.0007) [2023-12-27 18:29:44,193][20381] Updated weights for policy 1, policy_version 1853048 (0.0008) [2023-12-27 18:29:44,207][20347] Updated weights for policy 0, policy_version 1854640 (0.0005) [2023-12-27 18:29:44,855][20347] Updated weights for policy 0, policy_version 1854650 (0.0006) [2023-12-27 18:29:44,911][20381] Updated weights for policy 1, policy_version 1853058 (0.0006) [2023-12-27 18:29:44,912][20347] Updated weights for policy 0, policy_version 1854660 (0.0006) [2023-12-27 18:29:44,963][20347] Updated weights for policy 0, policy_version 1854670 (0.0006) [2023-12-27 18:29:44,973][20381] Updated weights for policy 1, policy_version 1853068 (0.0006) [2023-12-27 18:29:45,022][20347] Updated weights for policy 0, policy_version 1854680 (0.0006) [2023-12-27 18:29:45,027][20381] Updated weights for policy 1, policy_version 1853078 (0.0006) [2023-12-27 18:29:45,689][20381] Updated weights for policy 1, policy_version 1853088 (0.0008) [2023-12-27 18:29:45,731][20347] Updated weights for policy 0, policy_version 1854690 (0.0010) [2023-12-27 18:29:45,742][20381] Updated weights for policy 1, policy_version 1853098 (0.0009) [2023-12-27 18:29:45,779][20347] Updated weights for policy 0, policy_version 1854700 (0.0010) [2023-12-27 18:29:45,796][20381] Updated weights for policy 1, policy_version 1853108 (0.0007) [2023-12-27 18:29:45,837][20347] Updated weights for policy 0, policy_version 1854710 (0.0010) [2023-12-27 18:29:46,448][20381] Updated weights for policy 1, policy_version 1853118 (0.0005) [2023-12-27 18:29:46,496][20381] Updated weights for policy 1, policy_version 1853128 (0.0007) [2023-12-27 18:29:46,527][20347] Updated weights for policy 0, policy_version 1854720 (0.0010) [2023-12-27 18:29:46,547][20381] Updated weights for policy 1, policy_version 1853138 (0.0005) [2023-12-27 18:29:46,581][20347] Updated weights for policy 0, policy_version 1854730 (0.0010) [2023-12-27 18:29:46,637][20347] Updated weights for policy 0, policy_version 1854740 (0.0010) [2023-12-27 18:29:47,206][20381] Updated weights for policy 1, policy_version 1853148 (0.0007) [2023-12-27 18:29:47,258][20381] Updated weights for policy 1, policy_version 1853158 (0.0007) [2023-12-27 18:29:47,304][20347] Updated weights for policy 0, policy_version 1854750 (0.0007) [2023-12-27 18:29:47,320][20381] Updated weights for policy 1, policy_version 1853169 (0.0008) [2023-12-27 18:29:47,362][20347] Updated weights for policy 0, policy_version 1854760 (0.0007) [2023-12-27 18:29:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20188.9). Total num frames: 949362688. Throughput: 0: 10359.5, 1: 10184.4. Samples: 949373240. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '832.490')] [2023-12-27 18:29:47,413][20347] Updated weights for policy 0, policy_version 1854770 (0.0008) [2023-12-27 18:29:47,915][20381] Updated weights for policy 1, policy_version 1853179 (0.0006) [2023-12-27 18:29:47,974][20381] Updated weights for policy 1, policy_version 1853189 (0.0005) [2023-12-27 18:29:48,024][20381] Updated weights for policy 1, policy_version 1853199 (0.0006) [2023-12-27 18:29:48,030][20347] Updated weights for policy 0, policy_version 1854780 (0.0007) [2023-12-27 18:29:48,081][20347] Updated weights for policy 0, policy_version 1854790 (0.0007) [2023-12-27 18:29:48,128][20347] Updated weights for policy 0, policy_version 1854800 (0.0007) [2023-12-27 18:29:48,669][20381] Updated weights for policy 1, policy_version 1853209 (0.0007) [2023-12-27 18:29:48,727][20381] Updated weights for policy 1, policy_version 1853219 (0.0007) [2023-12-27 18:29:48,779][20381] Updated weights for policy 1, policy_version 1853229 (0.0008) [2023-12-27 18:29:48,825][20347] Updated weights for policy 0, policy_version 1854810 (0.0007) [2023-12-27 18:29:48,831][20381] Updated weights for policy 1, policy_version 1853239 (0.0007) [2023-12-27 18:29:48,871][20347] Updated weights for policy 0, policy_version 1854820 (0.0005) [2023-12-27 18:29:48,928][20347] Updated weights for policy 0, policy_version 1854830 (0.0005) [2023-12-27 18:29:48,992][20347] Updated weights for policy 0, policy_version 1854840 (0.0005) [2023-12-27 18:29:49,535][20381] Updated weights for policy 1, policy_version 1853249 (0.0011) [2023-12-27 18:29:49,593][20381] Updated weights for policy 1, policy_version 1853259 (0.0011) [2023-12-27 18:29:49,628][20347] Updated weights for policy 0, policy_version 1854850 (0.0006) [2023-12-27 18:29:49,653][20381] Updated weights for policy 1, policy_version 1853269 (0.0011) [2023-12-27 18:29:49,686][20347] Updated weights for policy 0, policy_version 1854860 (0.0006) [2023-12-27 18:29:49,734][20347] Updated weights for policy 0, policy_version 1854870 (0.0007) [2023-12-27 18:29:50,362][20381] Updated weights for policy 1, policy_version 1853279 (0.0009) [2023-12-27 18:29:50,418][20381] Updated weights for policy 1, policy_version 1853289 (0.0010) [2023-12-27 18:29:50,474][20381] Updated weights for policy 1, policy_version 1853299 (0.0011) [2023-12-27 18:29:50,484][20347] Updated weights for policy 0, policy_version 1854880 (0.0006) [2023-12-27 18:29:50,537][20347] Updated weights for policy 0, policy_version 1854890 (0.0005) [2023-12-27 18:29:50,603][20347] Updated weights for policy 0, policy_version 1854900 (0.0007) [2023-12-27 18:29:51,113][20381] Updated weights for policy 1, policy_version 1853309 (0.0008) [2023-12-27 18:29:51,178][20381] Updated weights for policy 1, policy_version 1853319 (0.0008) [2023-12-27 18:29:51,231][20381] Updated weights for policy 1, policy_version 1853329 (0.0006) [2023-12-27 18:29:51,360][20347] Updated weights for policy 0, policy_version 1854910 (0.0008) [2023-12-27 18:29:51,425][20347] Updated weights for policy 0, policy_version 1854920 (0.0008) [2023-12-27 18:29:51,487][20347] Updated weights for policy 0, policy_version 1854930 (0.0007) [2023-12-27 18:29:51,916][20381] Updated weights for policy 1, policy_version 1853339 (0.0008) [2023-12-27 18:29:51,969][20381] Updated weights for policy 1, policy_version 1853349 (0.0008) [2023-12-27 18:29:52,016][20381] Updated weights for policy 1, policy_version 1853359 (0.0007) [2023-12-27 18:29:52,244][20347] Updated weights for policy 0, policy_version 1854940 (0.0008) [2023-12-27 18:29:52,303][20347] Updated weights for policy 0, policy_version 1854950 (0.0007) [2023-12-27 18:29:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20160.6). Total num frames: 949460992. Throughput: 0: 10334.7, 1: 10143.2. Samples: 949435004. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:52,364][19170] Avg episode reward: [(0, '990.280'), (1, '812.580')] [2023-12-27 18:29:52,365][20347] Updated weights for policy 0, policy_version 1854960 (0.0009) [2023-12-27 18:29:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001853368_474513408.pth... [2023-12-27 18:29:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001852184_474210304.pth [2023-12-27 18:29:52,407][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001854968_474955776.pth... [2023-12-27 18:29:52,411][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001853752_474644480.pth [2023-12-27 18:29:52,622][20381] Updated weights for policy 1, policy_version 1853369 (0.0008) [2023-12-27 18:29:52,679][20381] Updated weights for policy 1, policy_version 1853379 (0.0007) [2023-12-27 18:29:52,739][20381] Updated weights for policy 1, policy_version 1853389 (0.0008) [2023-12-27 18:29:52,785][20381] Updated weights for policy 1, policy_version 1853399 (0.0008) [2023-12-27 18:29:53,042][20347] Updated weights for policy 0, policy_version 1854970 (0.0008) [2023-12-27 18:29:53,103][20347] Updated weights for policy 0, policy_version 1854980 (0.0010) [2023-12-27 18:29:53,158][20347] Updated weights for policy 0, policy_version 1854990 (0.0005) [2023-12-27 18:29:53,212][20347] Updated weights for policy 0, policy_version 1855000 (0.0005) [2023-12-27 18:29:53,464][20381] Updated weights for policy 1, policy_version 1853409 (0.0010) [2023-12-27 18:29:53,512][20381] Updated weights for policy 1, policy_version 1853419 (0.0010) [2023-12-27 18:29:53,553][20381] Updated weights for policy 1, policy_version 1853429 (0.0010) [2023-12-27 18:29:53,794][20347] Updated weights for policy 0, policy_version 1855010 (0.0007) [2023-12-27 18:29:53,848][20347] Updated weights for policy 0, policy_version 1855020 (0.0007) [2023-12-27 18:29:53,898][20347] Updated weights for policy 0, policy_version 1855030 (0.0005) [2023-12-27 18:29:54,247][20381] Updated weights for policy 1, policy_version 1853439 (0.0009) [2023-12-27 18:29:54,311][20381] Updated weights for policy 1, policy_version 1853449 (0.0010) [2023-12-27 18:29:54,371][20381] Updated weights for policy 1, policy_version 1853459 (0.0008) [2023-12-27 18:29:54,511][20347] Updated weights for policy 0, policy_version 1855040 (0.0009) [2023-12-27 18:29:54,573][20347] Updated weights for policy 0, policy_version 1855050 (0.0010) [2023-12-27 18:29:54,628][20347] Updated weights for policy 0, policy_version 1855060 (0.0010) [2023-12-27 18:29:55,001][20381] Updated weights for policy 1, policy_version 1853469 (0.0011) [2023-12-27 18:29:55,056][20381] Updated weights for policy 1, policy_version 1853479 (0.0010) [2023-12-27 18:29:55,108][20381] Updated weights for policy 1, policy_version 1853489 (0.0010) [2023-12-27 18:29:55,296][20347] Updated weights for policy 0, policy_version 1855070 (0.0010) [2023-12-27 18:29:55,345][20347] Updated weights for policy 0, policy_version 1855080 (0.0010) [2023-12-27 18:29:55,390][20347] Updated weights for policy 0, policy_version 1855090 (0.0010) [2023-12-27 18:29:55,764][20381] Updated weights for policy 1, policy_version 1853499 (0.0011) [2023-12-27 18:29:55,815][20381] Updated weights for policy 1, policy_version 1853509 (0.0011) [2023-12-27 18:29:55,867][20381] Updated weights for policy 1, policy_version 1853519 (0.0011) [2023-12-27 18:29:56,082][20347] Updated weights for policy 0, policy_version 1855100 (0.0011) [2023-12-27 18:29:56,139][20347] Updated weights for policy 0, policy_version 1855110 (0.0010) [2023-12-27 18:29:56,201][20347] Updated weights for policy 0, policy_version 1855120 (0.0007) [2023-12-27 18:29:56,450][20381] Updated weights for policy 1, policy_version 1853529 (0.0011) [2023-12-27 18:29:56,498][20381] Updated weights for policy 1, policy_version 1853539 (0.0011) [2023-12-27 18:29:56,548][20381] Updated weights for policy 1, policy_version 1853549 (0.0010) [2023-12-27 18:29:56,609][20381] Updated weights for policy 1, policy_version 1853559 (0.0011) [2023-12-27 18:29:56,807][20347] Updated weights for policy 0, policy_version 1855130 (0.0010) [2023-12-27 18:29:56,851][20347] Updated weights for policy 0, policy_version 1855140 (0.0010) [2023-12-27 18:29:56,895][20347] Updated weights for policy 0, policy_version 1855150 (0.0010) [2023-12-27 18:29:56,943][20347] Updated weights for policy 0, policy_version 1855160 (0.0010) [2023-12-27 18:29:57,310][20381] Updated weights for policy 1, policy_version 1853569 (0.0010) [2023-12-27 18:29:57,362][20381] Updated weights for policy 1, policy_version 1853579 (0.0011) [2023-12-27 18:29:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20188.4). Total num frames: 949567488. Throughput: 0: 10350.1, 1: 10186.4. Samples: 949559616. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:29:57,363][19170] Avg episode reward: [(0, '990.280'), (1, '830.090')] [2023-12-27 18:29:57,416][20381] Updated weights for policy 1, policy_version 1853589 (0.0010) [2023-12-27 18:29:57,628][20347] Updated weights for policy 0, policy_version 1855170 (0.0006) [2023-12-27 18:29:57,692][20347] Updated weights for policy 0, policy_version 1855180 (0.0007) [2023-12-27 18:29:57,751][20347] Updated weights for policy 0, policy_version 1855190 (0.0006) [2023-12-27 18:29:58,124][20381] Updated weights for policy 1, policy_version 1853599 (0.0011) [2023-12-27 18:29:58,185][20381] Updated weights for policy 1, policy_version 1853609 (0.0011) [2023-12-27 18:29:58,255][20381] Updated weights for policy 1, policy_version 1853619 (0.0008) [2023-12-27 18:29:58,386][20347] Updated weights for policy 0, policy_version 1855200 (0.0007) [2023-12-27 18:29:58,458][20347] Updated weights for policy 0, policy_version 1855210 (0.0008) [2023-12-27 18:29:58,521][20347] Updated weights for policy 0, policy_version 1855220 (0.0008) [2023-12-27 18:29:59,043][20381] Updated weights for policy 1, policy_version 1853629 (0.0008) [2023-12-27 18:29:59,104][20381] Updated weights for policy 1, policy_version 1853639 (0.0008) [2023-12-27 18:29:59,164][20381] Updated weights for policy 1, policy_version 1853649 (0.0008) [2023-12-27 18:29:59,222][20347] Updated weights for policy 0, policy_version 1855230 (0.0008) [2023-12-27 18:29:59,282][20347] Updated weights for policy 0, policy_version 1855240 (0.0010) [2023-12-27 18:29:59,345][20347] Updated weights for policy 0, policy_version 1855250 (0.0010) [2023-12-27 18:29:59,863][20381] Updated weights for policy 1, policy_version 1853659 (0.0006) [2023-12-27 18:29:59,921][20381] Updated weights for policy 1, policy_version 1853669 (0.0009) [2023-12-27 18:29:59,973][20381] Updated weights for policy 1, policy_version 1853679 (0.0010) [2023-12-27 18:30:00,021][20347] Updated weights for policy 0, policy_version 1855260 (0.0010) [2023-12-27 18:30:00,069][20347] Updated weights for policy 0, policy_version 1855270 (0.0010) [2023-12-27 18:30:00,124][20347] Updated weights for policy 0, policy_version 1855280 (0.0010) [2023-12-27 18:30:00,636][20381] Updated weights for policy 1, policy_version 1853689 (0.0010) [2023-12-27 18:30:00,691][20381] Updated weights for policy 1, policy_version 1853699 (0.0008) [2023-12-27 18:30:00,743][20381] Updated weights for policy 1, policy_version 1853709 (0.0006) [2023-12-27 18:30:00,794][20381] Updated weights for policy 1, policy_version 1853719 (0.0007) [2023-12-27 18:30:00,804][20347] Updated weights for policy 0, policy_version 1855290 (0.0010) [2023-12-27 18:30:00,852][20347] Updated weights for policy 0, policy_version 1855300 (0.0010) [2023-12-27 18:30:00,900][20347] Updated weights for policy 0, policy_version 1855310 (0.0010) [2023-12-27 18:30:00,945][20347] Updated weights for policy 0, policy_version 1855320 (0.0010) [2023-12-27 18:30:01,459][20381] Updated weights for policy 1, policy_version 1853729 (0.0006) [2023-12-27 18:30:01,514][20381] Updated weights for policy 1, policy_version 1853739 (0.0005) [2023-12-27 18:30:01,572][20381] Updated weights for policy 1, policy_version 1853749 (0.0006) [2023-12-27 18:30:01,767][20347] Updated weights for policy 0, policy_version 1855330 (0.0008) [2023-12-27 18:30:01,824][20347] Updated weights for policy 0, policy_version 1855340 (0.0007) [2023-12-27 18:30:01,876][20347] Updated weights for policy 0, policy_version 1855350 (0.0007) [2023-12-27 18:30:02,306][20381] Updated weights for policy 1, policy_version 1853759 (0.0008) [2023-12-27 18:30:02,367][20381] Updated weights for policy 1, policy_version 1853769 (0.0008) [2023-12-27 18:30:02,372][19170] Fps is (10 sec: 20460.5, 60 sec: 20476.7, 300 sec: 20187.8). Total num frames: 949665792. Throughput: 0: 10313.0, 1: 10163.0. Samples: 949680364. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:02,374][19170] Avg episode reward: [(0, '1000.000'), (1, '811.090')] [2023-12-27 18:30:02,428][20381] Updated weights for policy 1, policy_version 1853779 (0.0007) [2023-12-27 18:30:02,494][20347] Updated weights for policy 0, policy_version 1855360 (0.0008) [2023-12-27 18:30:02,553][20347] Updated weights for policy 0, policy_version 1855370 (0.0007) [2023-12-27 18:30:02,609][20347] Updated weights for policy 0, policy_version 1855380 (0.0007) [2023-12-27 18:30:03,052][20381] Updated weights for policy 1, policy_version 1853789 (0.0007) [2023-12-27 18:30:03,109][20381] Updated weights for policy 1, policy_version 1853799 (0.0010) [2023-12-27 18:30:03,170][20381] Updated weights for policy 1, policy_version 1853809 (0.0010) [2023-12-27 18:30:03,260][20347] Updated weights for policy 0, policy_version 1855390 (0.0007) [2023-12-27 18:30:03,306][20347] Updated weights for policy 0, policy_version 1855400 (0.0007) [2023-12-27 18:30:03,359][20347] Updated weights for policy 0, policy_version 1855410 (0.0006) [2023-12-27 18:30:03,839][20381] Updated weights for policy 1, policy_version 1853819 (0.0010) [2023-12-27 18:30:03,904][20381] Updated weights for policy 1, policy_version 1853829 (0.0012) [2023-12-27 18:30:03,964][20381] Updated weights for policy 1, policy_version 1853839 (0.0008) [2023-12-27 18:30:03,998][20347] Updated weights for policy 0, policy_version 1855420 (0.0007) [2023-12-27 18:30:04,049][20347] Updated weights for policy 0, policy_version 1855430 (0.0007) [2023-12-27 18:30:04,097][20347] Updated weights for policy 0, policy_version 1855440 (0.0007) [2023-12-27 18:30:04,693][20381] Updated weights for policy 1, policy_version 1853849 (0.0008) [2023-12-27 18:30:04,749][20381] Updated weights for policy 1, policy_version 1853859 (0.0007) [2023-12-27 18:30:04,799][20347] Updated weights for policy 0, policy_version 1855450 (0.0007) [2023-12-27 18:30:04,806][20381] Updated weights for policy 1, policy_version 1853869 (0.0008) [2023-12-27 18:30:04,848][20347] Updated weights for policy 0, policy_version 1855460 (0.0007) [2023-12-27 18:30:04,861][20381] Updated weights for policy 1, policy_version 1853879 (0.0006) [2023-12-27 18:30:04,895][20347] Updated weights for policy 0, policy_version 1855470 (0.0006) [2023-12-27 18:30:04,948][20347] Updated weights for policy 0, policy_version 1855480 (0.0006) [2023-12-27 18:30:05,495][20381] Updated weights for policy 1, policy_version 1853889 (0.0007) [2023-12-27 18:30:05,540][20381] Updated weights for policy 1, policy_version 1853899 (0.0008) [2023-12-27 18:30:05,584][20381] Updated weights for policy 1, policy_version 1853909 (0.0006) [2023-12-27 18:30:05,607][20347] Updated weights for policy 0, policy_version 1855490 (0.0008) [2023-12-27 18:30:05,655][20347] Updated weights for policy 0, policy_version 1855500 (0.0008) [2023-12-27 18:30:05,707][20347] Updated weights for policy 0, policy_version 1855510 (0.0008) [2023-12-27 18:30:06,168][20381] Updated weights for policy 1, policy_version 1853919 (0.0008) [2023-12-27 18:30:06,229][20381] Updated weights for policy 1, policy_version 1853929 (0.0006) [2023-12-27 18:30:06,286][20381] Updated weights for policy 1, policy_version 1853939 (0.0007) [2023-12-27 18:30:06,344][20347] Updated weights for policy 0, policy_version 1855520 (0.0009) [2023-12-27 18:30:06,402][20347] Updated weights for policy 0, policy_version 1855530 (0.0006) [2023-12-27 18:30:06,457][20347] Updated weights for policy 0, policy_version 1855540 (0.0005) [2023-12-27 18:30:06,921][20381] Updated weights for policy 1, policy_version 1853949 (0.0005) [2023-12-27 18:30:06,980][20381] Updated weights for policy 1, policy_version 1853959 (0.0007) [2023-12-27 18:30:07,037][20381] Updated weights for policy 1, policy_version 1853969 (0.0009) [2023-12-27 18:30:07,057][20347] Updated weights for policy 0, policy_version 1855550 (0.0006) [2023-12-27 18:30:07,120][20347] Updated weights for policy 0, policy_version 1855560 (0.0007) [2023-12-27 18:30:07,180][20347] Updated weights for policy 0, policy_version 1855570 (0.0008) [2023-12-27 18:30:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20244.0). Total num frames: 949780480. Throughput: 0: 10322.2, 1: 10172.8. Samples: 949742972. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '773.380')] [2023-12-27 18:30:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001855576_475111424.pth... [2023-12-27 18:30:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001853976_474669056.pth... [2023-12-27 18:30:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001854392_474808320.pth [2023-12-27 18:30:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001852760_474357760.pth [2023-12-27 18:30:07,728][20381] Updated weights for policy 1, policy_version 1853979 (0.0007) [2023-12-27 18:30:07,783][20381] Updated weights for policy 1, policy_version 1853989 (0.0008) [2023-12-27 18:30:07,837][20381] Updated weights for policy 1, policy_version 1853999 (0.0007) [2023-12-27 18:30:07,859][20347] Updated weights for policy 0, policy_version 1855580 (0.0007) [2023-12-27 18:30:07,911][20347] Updated weights for policy 0, policy_version 1855590 (0.0005) [2023-12-27 18:30:07,958][20347] Updated weights for policy 0, policy_version 1855600 (0.0007) [2023-12-27 18:30:08,544][20381] Updated weights for policy 1, policy_version 1854009 (0.0006) [2023-12-27 18:30:08,604][20347] Updated weights for policy 0, policy_version 1855610 (0.0007) [2023-12-27 18:30:08,606][20381] Updated weights for policy 1, policy_version 1854019 (0.0007) [2023-12-27 18:30:08,663][20347] Updated weights for policy 0, policy_version 1855620 (0.0010) [2023-12-27 18:30:08,663][20381] Updated weights for policy 1, policy_version 1854029 (0.0008) [2023-12-27 18:30:08,717][20381] Updated weights for policy 1, policy_version 1854039 (0.0007) [2023-12-27 18:30:08,722][20347] Updated weights for policy 0, policy_version 1855630 (0.0010) [2023-12-27 18:30:08,783][20347] Updated weights for policy 0, policy_version 1855640 (0.0010) [2023-12-27 18:30:09,445][20381] Updated weights for policy 1, policy_version 1854049 (0.0008) [2023-12-27 18:30:09,453][20347] Updated weights for policy 0, policy_version 1855650 (0.0009) [2023-12-27 18:30:09,504][20347] Updated weights for policy 0, policy_version 1855660 (0.0007) [2023-12-27 18:30:09,506][20381] Updated weights for policy 1, policy_version 1854059 (0.0007) [2023-12-27 18:30:09,553][20347] Updated weights for policy 0, policy_version 1855670 (0.0006) [2023-12-27 18:30:09,569][20381] Updated weights for policy 1, policy_version 1854069 (0.0008) [2023-12-27 18:30:10,238][20347] Updated weights for policy 0, policy_version 1855680 (0.0007) [2023-12-27 18:30:10,292][20347] Updated weights for policy 0, policy_version 1855690 (0.0006) [2023-12-27 18:30:10,333][20381] Updated weights for policy 1, policy_version 1854079 (0.0008) [2023-12-27 18:30:10,350][20347] Updated weights for policy 0, policy_version 1855700 (0.0007) [2023-12-27 18:30:10,393][20381] Updated weights for policy 1, policy_version 1854089 (0.0008) [2023-12-27 18:30:10,450][20381] Updated weights for policy 1, policy_version 1854099 (0.0007) [2023-12-27 18:30:11,056][20347] Updated weights for policy 0, policy_version 1855710 (0.0007) [2023-12-27 18:30:11,123][20347] Updated weights for policy 0, policy_version 1855720 (0.0007) [2023-12-27 18:30:11,133][20381] Updated weights for policy 1, policy_version 1854109 (0.0007) [2023-12-27 18:30:11,191][20347] Updated weights for policy 0, policy_version 1855730 (0.0007) [2023-12-27 18:30:11,197][20381] Updated weights for policy 1, policy_version 1854119 (0.0007) [2023-12-27 18:30:11,252][20381] Updated weights for policy 1, policy_version 1854129 (0.0008) [2023-12-27 18:30:11,957][20347] Updated weights for policy 0, policy_version 1855740 (0.0008) [2023-12-27 18:30:11,979][20381] Updated weights for policy 1, policy_version 1854139 (0.0009) [2023-12-27 18:30:12,018][20347] Updated weights for policy 0, policy_version 1855750 (0.0009) [2023-12-27 18:30:12,039][20381] Updated weights for policy 1, policy_version 1854149 (0.0008) [2023-12-27 18:30:12,078][20347] Updated weights for policy 0, policy_version 1855760 (0.0007) [2023-12-27 18:30:12,097][20381] Updated weights for policy 1, policy_version 1854159 (0.0006) [2023-12-27 18:30:12,362][19170] Fps is (10 sec: 21319.7, 60 sec: 20616.5, 300 sec: 20271.7). Total num frames: 949878784. Throughput: 0: 10328.3, 1: 10154.1. Samples: 949867132. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '802.350')] [2023-12-27 18:30:12,754][20347] Updated weights for policy 0, policy_version 1855770 (0.0006) [2023-12-27 18:30:12,817][20347] Updated weights for policy 0, policy_version 1855780 (0.0008) [2023-12-27 18:30:12,839][20381] Updated weights for policy 1, policy_version 1854169 (0.0008) [2023-12-27 18:30:12,878][20347] Updated weights for policy 0, policy_version 1855790 (0.0008) [2023-12-27 18:30:12,892][20381] Updated weights for policy 1, policy_version 1854179 (0.0006) [2023-12-27 18:30:12,933][20347] Updated weights for policy 0, policy_version 1855800 (0.0006) [2023-12-27 18:30:12,953][20381] Updated weights for policy 1, policy_version 1854189 (0.0006) [2023-12-27 18:30:13,006][20381] Updated weights for policy 1, policy_version 1854199 (0.0009) [2023-12-27 18:30:13,689][20347] Updated weights for policy 0, policy_version 1855810 (0.0011) [2023-12-27 18:30:13,702][20381] Updated weights for policy 1, policy_version 1854209 (0.0006) [2023-12-27 18:30:13,754][20347] Updated weights for policy 0, policy_version 1855820 (0.0010) [2023-12-27 18:30:13,757][20381] Updated weights for policy 1, policy_version 1854219 (0.0009) [2023-12-27 18:30:13,809][20381] Updated weights for policy 1, policy_version 1854229 (0.0010) [2023-12-27 18:30:13,809][20347] Updated weights for policy 0, policy_version 1855830 (0.0010) [2023-12-27 18:30:14,456][20381] Updated weights for policy 1, policy_version 1854239 (0.0011) [2023-12-27 18:30:14,460][20347] Updated weights for policy 0, policy_version 1855840 (0.0010) [2023-12-27 18:30:14,505][20347] Updated weights for policy 0, policy_version 1855850 (0.0010) [2023-12-27 18:30:14,517][20381] Updated weights for policy 1, policy_version 1854249 (0.0010) [2023-12-27 18:30:14,554][20347] Updated weights for policy 0, policy_version 1855860 (0.0008) [2023-12-27 18:30:14,573][20381] Updated weights for policy 1, policy_version 1854259 (0.0010) [2023-12-27 18:30:15,194][20347] Updated weights for policy 0, policy_version 1855870 (0.0008) [2023-12-27 18:30:15,254][20381] Updated weights for policy 1, policy_version 1854269 (0.0010) [2023-12-27 18:30:15,257][20347] Updated weights for policy 0, policy_version 1855880 (0.0010) [2023-12-27 18:30:15,310][20381] Updated weights for policy 1, policy_version 1854279 (0.0011) [2023-12-27 18:30:15,316][20347] Updated weights for policy 0, policy_version 1855890 (0.0011) [2023-12-27 18:30:15,362][20381] Updated weights for policy 1, policy_version 1854289 (0.0010) [2023-12-27 18:30:15,929][20347] Updated weights for policy 0, policy_version 1855900 (0.0009) [2023-12-27 18:30:15,995][20347] Updated weights for policy 0, policy_version 1855910 (0.0008) [2023-12-27 18:30:16,061][20347] Updated weights for policy 0, policy_version 1855920 (0.0008) [2023-12-27 18:30:16,065][20381] Updated weights for policy 1, policy_version 1854299 (0.0011) [2023-12-27 18:30:16,121][20381] Updated weights for policy 1, policy_version 1854309 (0.0011) [2023-12-27 18:30:16,182][20381] Updated weights for policy 1, policy_version 1854319 (0.0011) [2023-12-27 18:30:16,757][20347] Updated weights for policy 0, policy_version 1855930 (0.0009) [2023-12-27 18:30:16,808][20381] Updated weights for policy 1, policy_version 1854329 (0.0010) [2023-12-27 18:30:16,811][20347] Updated weights for policy 0, policy_version 1855940 (0.0005) [2023-12-27 18:30:16,857][20381] Updated weights for policy 1, policy_version 1854339 (0.0007) [2023-12-27 18:30:16,867][20347] Updated weights for policy 0, policy_version 1855950 (0.0006) [2023-12-27 18:30:16,911][20381] Updated weights for policy 1, policy_version 1854349 (0.0007) [2023-12-27 18:30:16,924][20347] Updated weights for policy 0, policy_version 1855960 (0.0007) [2023-12-27 18:30:16,961][20381] Updated weights for policy 1, policy_version 1854359 (0.0005) [2023-12-27 18:30:17,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20271.7). Total num frames: 949977088. Throughput: 0: 10267.1, 1: 10145.9. Samples: 949987948. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.150')] [2023-12-27 18:30:17,589][20347] Updated weights for policy 0, policy_version 1855970 (0.0007) [2023-12-27 18:30:17,608][20381] Updated weights for policy 1, policy_version 1854369 (0.0005) [2023-12-27 18:30:17,643][20347] Updated weights for policy 0, policy_version 1855980 (0.0007) [2023-12-27 18:30:17,656][20381] Updated weights for policy 1, policy_version 1854379 (0.0005) [2023-12-27 18:30:17,700][20347] Updated weights for policy 0, policy_version 1855990 (0.0008) [2023-12-27 18:30:17,707][20381] Updated weights for policy 1, policy_version 1854389 (0.0006) [2023-12-27 18:30:18,310][20381] Updated weights for policy 1, policy_version 1854399 (0.0007) [2023-12-27 18:30:18,372][20381] Updated weights for policy 1, policy_version 1854409 (0.0007) [2023-12-27 18:30:18,425][20347] Updated weights for policy 0, policy_version 1856000 (0.0006) [2023-12-27 18:30:18,430][20381] Updated weights for policy 1, policy_version 1854419 (0.0007) [2023-12-27 18:30:18,489][20347] Updated weights for policy 0, policy_version 1856010 (0.0006) [2023-12-27 18:30:18,557][20347] Updated weights for policy 0, policy_version 1856020 (0.0008) [2023-12-27 18:30:19,114][20381] Updated weights for policy 1, policy_version 1854429 (0.0006) [2023-12-27 18:30:19,158][20381] Updated weights for policy 1, policy_version 1854439 (0.0007) [2023-12-27 18:30:19,203][20381] Updated weights for policy 1, policy_version 1854449 (0.0008) [2023-12-27 18:30:19,273][20347] Updated weights for policy 0, policy_version 1856030 (0.0007) [2023-12-27 18:30:19,333][20347] Updated weights for policy 0, policy_version 1856040 (0.0008) [2023-12-27 18:30:19,391][20347] Updated weights for policy 0, policy_version 1856050 (0.0007) [2023-12-27 18:30:19,918][20381] Updated weights for policy 1, policy_version 1854459 (0.0007) [2023-12-27 18:30:19,982][20381] Updated weights for policy 1, policy_version 1854469 (0.0008) [2023-12-27 18:30:20,045][20381] Updated weights for policy 1, policy_version 1854479 (0.0008) [2023-12-27 18:30:20,084][20347] Updated weights for policy 0, policy_version 1856060 (0.0006) [2023-12-27 18:30:20,141][20347] Updated weights for policy 0, policy_version 1856070 (0.0007) [2023-12-27 18:30:20,201][20347] Updated weights for policy 0, policy_version 1856080 (0.0008) [2023-12-27 18:30:20,735][20381] Updated weights for policy 1, policy_version 1854489 (0.0006) [2023-12-27 18:30:20,791][20381] Updated weights for policy 1, policy_version 1854499 (0.0008) [2023-12-27 18:30:20,848][20381] Updated weights for policy 1, policy_version 1854509 (0.0008) [2023-12-27 18:30:20,903][20381] Updated weights for policy 1, policy_version 1854519 (0.0008) [2023-12-27 18:30:20,917][20347] Updated weights for policy 0, policy_version 1856090 (0.0007) [2023-12-27 18:30:20,982][20347] Updated weights for policy 0, policy_version 1856100 (0.0006) [2023-12-27 18:30:21,054][20347] Updated weights for policy 0, policy_version 1856110 (0.0007) [2023-12-27 18:30:21,110][20347] Updated weights for policy 0, policy_version 1856120 (0.0008) [2023-12-27 18:30:21,752][20381] Updated weights for policy 1, policy_version 1854529 (0.0008) [2023-12-27 18:30:21,817][20381] Updated weights for policy 1, policy_version 1854539 (0.0008) [2023-12-27 18:30:21,869][20347] Updated weights for policy 0, policy_version 1856130 (0.0006) [2023-12-27 18:30:21,874][20381] Updated weights for policy 1, policy_version 1854549 (0.0007) [2023-12-27 18:30:21,928][20347] Updated weights for policy 0, policy_version 1856140 (0.0008) [2023-12-27 18:30:21,983][20347] Updated weights for policy 0, policy_version 1856150 (0.0007) [2023-12-27 18:30:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20327.3). Total num frames: 950075392. Throughput: 0: 10207.2, 1: 10174.7. Samples: 950048284. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.570')] [2023-12-27 18:30:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001854552_474816512.pth... [2023-12-27 18:30:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001856152_475258880.pth... [2023-12-27 18:30:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001853368_474513408.pth [2023-12-27 18:30:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001854968_474955776.pth [2023-12-27 18:30:22,598][20381] Updated weights for policy 1, policy_version 1854559 (0.0006) [2023-12-27 18:30:22,665][20381] Updated weights for policy 1, policy_version 1854569 (0.0007) [2023-12-27 18:30:22,693][20347] Updated weights for policy 0, policy_version 1856160 (0.0007) [2023-12-27 18:30:22,723][20381] Updated weights for policy 1, policy_version 1854579 (0.0005) [2023-12-27 18:30:22,747][20347] Updated weights for policy 0, policy_version 1856170 (0.0006) [2023-12-27 18:30:22,806][20347] Updated weights for policy 0, policy_version 1856180 (0.0007) [2023-12-27 18:30:23,421][20381] Updated weights for policy 1, policy_version 1854589 (0.0007) [2023-12-27 18:30:23,473][20381] Updated weights for policy 1, policy_version 1854599 (0.0008) [2023-12-27 18:30:23,505][20347] Updated weights for policy 0, policy_version 1856190 (0.0007) [2023-12-27 18:30:23,529][20381] Updated weights for policy 1, policy_version 1854609 (0.0005) [2023-12-27 18:30:23,560][20347] Updated weights for policy 0, policy_version 1856200 (0.0005) [2023-12-27 18:30:23,621][20347] Updated weights for policy 0, policy_version 1856210 (0.0007) [2023-12-27 18:30:24,231][20381] Updated weights for policy 1, policy_version 1854619 (0.0006) [2023-12-27 18:30:24,285][20347] Updated weights for policy 0, policy_version 1856220 (0.0007) [2023-12-27 18:30:24,292][20381] Updated weights for policy 1, policy_version 1854629 (0.0008) [2023-12-27 18:30:24,336][20347] Updated weights for policy 0, policy_version 1856230 (0.0006) [2023-12-27 18:30:24,358][20381] Updated weights for policy 1, policy_version 1854639 (0.0008) [2023-12-27 18:30:24,386][20347] Updated weights for policy 0, policy_version 1856240 (0.0006) [2023-12-27 18:30:24,961][20381] Updated weights for policy 1, policy_version 1854649 (0.0009) [2023-12-27 18:30:24,999][20347] Updated weights for policy 0, policy_version 1856250 (0.0006) [2023-12-27 18:30:25,014][20381] Updated weights for policy 1, policy_version 1854659 (0.0010) [2023-12-27 18:30:25,057][20347] Updated weights for policy 0, policy_version 1856260 (0.0006) [2023-12-27 18:30:25,093][20381] Updated weights for policy 1, policy_version 1854669 (0.0009) [2023-12-27 18:30:25,111][20347] Updated weights for policy 0, policy_version 1856270 (0.0007) [2023-12-27 18:30:25,146][20381] Updated weights for policy 1, policy_version 1854679 (0.0010) [2023-12-27 18:30:25,161][20347] Updated weights for policy 0, policy_version 1856280 (0.0006) [2023-12-27 18:30:25,789][20381] Updated weights for policy 1, policy_version 1854689 (0.0005) [2023-12-27 18:30:25,792][20347] Updated weights for policy 0, policy_version 1856290 (0.0011) [2023-12-27 18:30:25,846][20381] Updated weights for policy 1, policy_version 1854699 (0.0005) [2023-12-27 18:30:25,855][20347] Updated weights for policy 0, policy_version 1856300 (0.0010) [2023-12-27 18:30:25,897][20381] Updated weights for policy 1, policy_version 1854709 (0.0006) [2023-12-27 18:30:25,921][20347] Updated weights for policy 0, policy_version 1856310 (0.0008) [2023-12-27 18:30:26,465][20381] Updated weights for policy 1, policy_version 1854719 (0.0005) [2023-12-27 18:30:26,506][20347] Updated weights for policy 0, policy_version 1856320 (0.0009) [2023-12-27 18:30:26,517][20381] Updated weights for policy 1, policy_version 1854729 (0.0005) [2023-12-27 18:30:26,563][20347] Updated weights for policy 0, policy_version 1856330 (0.0010) [2023-12-27 18:30:26,575][20381] Updated weights for policy 1, policy_version 1854739 (0.0005) [2023-12-27 18:30:26,617][20347] Updated weights for policy 0, policy_version 1856340 (0.0010) [2023-12-27 18:30:27,174][20381] Updated weights for policy 1, policy_version 1854749 (0.0005) [2023-12-27 18:30:27,223][20381] Updated weights for policy 1, policy_version 1854759 (0.0008) [2023-12-27 18:30:27,269][20381] Updated weights for policy 1, policy_version 1854769 (0.0010) [2023-12-27 18:30:27,294][20347] Updated weights for policy 0, policy_version 1856350 (0.0010) [2023-12-27 18:30:27,344][20347] Updated weights for policy 0, policy_version 1856360 (0.0009) [2023-12-27 18:30:27,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20355.0). Total num frames: 950181888. Throughput: 0: 10176.1, 1: 10230.0. Samples: 950169188. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:27,364][19170] Avg episode reward: [(0, '990.320'), (1, '872.650')] [2023-12-27 18:30:27,392][20347] Updated weights for policy 0, policy_version 1856370 (0.0006) [2023-12-27 18:30:27,821][20381] Updated weights for policy 1, policy_version 1854779 (0.0009) [2023-12-27 18:30:27,871][20381] Updated weights for policy 1, policy_version 1854789 (0.0005) [2023-12-27 18:30:27,925][20381] Updated weights for policy 1, policy_version 1854799 (0.0008) [2023-12-27 18:30:28,038][20347] Updated weights for policy 0, policy_version 1856380 (0.0010) [2023-12-27 18:30:28,094][20347] Updated weights for policy 0, policy_version 1856390 (0.0009) [2023-12-27 18:30:28,149][20347] Updated weights for policy 0, policy_version 1856400 (0.0008) [2023-12-27 18:30:28,595][20381] Updated weights for policy 1, policy_version 1854809 (0.0010) [2023-12-27 18:30:28,656][20381] Updated weights for policy 1, policy_version 1854819 (0.0005) [2023-12-27 18:30:28,715][20381] Updated weights for policy 1, policy_version 1854829 (0.0005) [2023-12-27 18:30:28,774][20381] Updated weights for policy 1, policy_version 1854839 (0.0005) [2023-12-27 18:30:28,822][20347] Updated weights for policy 0, policy_version 1856410 (0.0008) [2023-12-27 18:30:28,873][20347] Updated weights for policy 0, policy_version 1856420 (0.0010) [2023-12-27 18:30:28,920][20347] Updated weights for policy 0, policy_version 1856430 (0.0009) [2023-12-27 18:30:28,978][20347] Updated weights for policy 0, policy_version 1856440 (0.0007) [2023-12-27 18:30:29,284][20381] Updated weights for policy 1, policy_version 1854849 (0.0010) [2023-12-27 18:30:29,333][20381] Updated weights for policy 1, policy_version 1854859 (0.0010) [2023-12-27 18:30:29,391][20381] Updated weights for policy 1, policy_version 1854869 (0.0010) [2023-12-27 18:30:29,618][20347] Updated weights for policy 0, policy_version 1856450 (0.0010) [2023-12-27 18:30:29,682][20347] Updated weights for policy 0, policy_version 1856460 (0.0010) [2023-12-27 18:30:29,734][20347] Updated weights for policy 0, policy_version 1856470 (0.0010) [2023-12-27 18:30:30,090][20381] Updated weights for policy 1, policy_version 1854879 (0.0010) [2023-12-27 18:30:30,138][20381] Updated weights for policy 1, policy_version 1854889 (0.0006) [2023-12-27 18:30:30,184][20381] Updated weights for policy 1, policy_version 1854899 (0.0008) [2023-12-27 18:30:30,454][20347] Updated weights for policy 0, policy_version 1856480 (0.0010) [2023-12-27 18:30:30,505][20347] Updated weights for policy 0, policy_version 1856490 (0.0010) [2023-12-27 18:30:30,549][20347] Updated weights for policy 0, policy_version 1856500 (0.0010) [2023-12-27 18:30:30,869][20381] Updated weights for policy 1, policy_version 1854909 (0.0007) [2023-12-27 18:30:30,920][20381] Updated weights for policy 1, policy_version 1854919 (0.0005) [2023-12-27 18:30:30,977][20381] Updated weights for policy 1, policy_version 1854929 (0.0007) [2023-12-27 18:30:31,227][20347] Updated weights for policy 0, policy_version 1856510 (0.0008) [2023-12-27 18:30:31,296][20347] Updated weights for policy 0, policy_version 1856520 (0.0008) [2023-12-27 18:30:31,358][20347] Updated weights for policy 0, policy_version 1856530 (0.0007) [2023-12-27 18:30:31,663][20381] Updated weights for policy 1, policy_version 1854939 (0.0007) [2023-12-27 18:30:31,723][20381] Updated weights for policy 1, policy_version 1854949 (0.0007) [2023-12-27 18:30:31,792][20381] Updated weights for policy 1, policy_version 1854959 (0.0007) [2023-12-27 18:30:31,960][20347] Updated weights for policy 0, policy_version 1856540 (0.0008) [2023-12-27 18:30:32,025][20347] Updated weights for policy 0, policy_version 1856550 (0.0007) [2023-12-27 18:30:32,084][20347] Updated weights for policy 0, policy_version 1856560 (0.0006) [2023-12-27 18:30:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 950288384. Throughput: 0: 10257.3, 1: 10313.6. Samples: 950298932. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:32,363][19170] Avg episode reward: [(0, '990.320'), (1, '843.610')] [2023-12-27 18:30:32,414][20381] Updated weights for policy 1, policy_version 1854969 (0.0006) [2023-12-27 18:30:32,464][20381] Updated weights for policy 1, policy_version 1854979 (0.0007) [2023-12-27 18:30:32,523][20381] Updated weights for policy 1, policy_version 1854989 (0.0007) [2023-12-27 18:30:32,581][20381] Updated weights for policy 1, policy_version 1854999 (0.0008) [2023-12-27 18:30:32,746][20347] Updated weights for policy 0, policy_version 1856570 (0.0008) [2023-12-27 18:30:32,797][20347] Updated weights for policy 0, policy_version 1856580 (0.0008) [2023-12-27 18:30:32,849][20347] Updated weights for policy 0, policy_version 1856590 (0.0007) [2023-12-27 18:30:32,900][20347] Updated weights for policy 0, policy_version 1856600 (0.0008) [2023-12-27 18:30:33,231][20381] Updated weights for policy 1, policy_version 1855009 (0.0010) [2023-12-27 18:30:33,279][20381] Updated weights for policy 1, policy_version 1855019 (0.0010) [2023-12-27 18:30:33,327][20381] Updated weights for policy 1, policy_version 1855029 (0.0010) [2023-12-27 18:30:33,542][20347] Updated weights for policy 0, policy_version 1856610 (0.0010) [2023-12-27 18:30:33,597][20347] Updated weights for policy 0, policy_version 1856620 (0.0006) [2023-12-27 18:30:33,660][20347] Updated weights for policy 0, policy_version 1856630 (0.0005) [2023-12-27 18:30:33,969][20381] Updated weights for policy 1, policy_version 1855039 (0.0010) [2023-12-27 18:30:34,024][20381] Updated weights for policy 1, policy_version 1855049 (0.0010) [2023-12-27 18:30:34,078][20381] Updated weights for policy 1, policy_version 1855059 (0.0010) [2023-12-27 18:30:34,269][20347] Updated weights for policy 0, policy_version 1856640 (0.0007) [2023-12-27 18:30:34,324][20347] Updated weights for policy 0, policy_version 1856650 (0.0008) [2023-12-27 18:30:34,370][20347] Updated weights for policy 0, policy_version 1856660 (0.0008) [2023-12-27 18:30:34,794][20381] Updated weights for policy 1, policy_version 1855069 (0.0008) [2023-12-27 18:30:34,852][20381] Updated weights for policy 1, policy_version 1855079 (0.0005) [2023-12-27 18:30:34,901][20381] Updated weights for policy 1, policy_version 1855089 (0.0008) [2023-12-27 18:30:35,101][20347] Updated weights for policy 0, policy_version 1856670 (0.0006) [2023-12-27 18:30:35,155][20347] Updated weights for policy 0, policy_version 1856680 (0.0005) [2023-12-27 18:30:35,206][20347] Updated weights for policy 0, policy_version 1856690 (0.0005) [2023-12-27 18:30:35,508][20381] Updated weights for policy 1, policy_version 1855099 (0.0009) [2023-12-27 18:30:35,554][20381] Updated weights for policy 1, policy_version 1855109 (0.0005) [2023-12-27 18:30:35,601][20381] Updated weights for policy 1, policy_version 1855119 (0.0005) [2023-12-27 18:30:35,748][20347] Updated weights for policy 0, policy_version 1856700 (0.0007) [2023-12-27 18:30:35,807][20347] Updated weights for policy 0, policy_version 1856710 (0.0010) [2023-12-27 18:30:35,865][20347] Updated weights for policy 0, policy_version 1856720 (0.0010) [2023-12-27 18:30:36,204][20381] Updated weights for policy 1, policy_version 1855129 (0.0006) [2023-12-27 18:30:36,261][20381] Updated weights for policy 1, policy_version 1855139 (0.0010) [2023-12-27 18:30:36,314][20381] Updated weights for policy 1, policy_version 1855149 (0.0011) [2023-12-27 18:30:36,367][20381] Updated weights for policy 1, policy_version 1855159 (0.0011) [2023-12-27 18:30:36,612][20347] Updated weights for policy 0, policy_version 1856730 (0.0010) [2023-12-27 18:30:36,670][20347] Updated weights for policy 0, policy_version 1856740 (0.0006) [2023-12-27 18:30:36,732][20347] Updated weights for policy 0, policy_version 1856750 (0.0011) [2023-12-27 18:30:36,786][20347] Updated weights for policy 0, policy_version 1856760 (0.0009) [2023-12-27 18:30:37,085][20381] Updated weights for policy 1, policy_version 1855169 (0.0008) [2023-12-27 18:30:37,134][20381] Updated weights for policy 1, policy_version 1855179 (0.0008) [2023-12-27 18:30:37,186][20381] Updated weights for policy 1, policy_version 1855189 (0.0008) [2023-12-27 18:30:37,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 950394880. Throughput: 0: 10269.8, 1: 10310.6. Samples: 950361120. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.320')] [2023-12-27 18:30:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001855192_474980352.pth... [2023-12-27 18:30:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001856760_475414528.pth... [2023-12-27 18:30:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001855576_475111424.pth [2023-12-27 18:30:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001853976_474669056.pth [2023-12-27 18:30:37,458][20347] Updated weights for policy 0, policy_version 1856770 (0.0010) [2023-12-27 18:30:37,519][20347] Updated weights for policy 0, policy_version 1856780 (0.0010) [2023-12-27 18:30:37,574][20347] Updated weights for policy 0, policy_version 1856790 (0.0010) [2023-12-27 18:30:37,918][20381] Updated weights for policy 1, policy_version 1855199 (0.0008) [2023-12-27 18:30:37,974][20381] Updated weights for policy 1, policy_version 1855209 (0.0008) [2023-12-27 18:30:38,030][20381] Updated weights for policy 1, policy_version 1855219 (0.0007) [2023-12-27 18:30:38,257][20347] Updated weights for policy 0, policy_version 1856800 (0.0010) [2023-12-27 18:30:38,305][20347] Updated weights for policy 0, policy_version 1856810 (0.0010) [2023-12-27 18:30:38,369][20347] Updated weights for policy 0, policy_version 1856820 (0.0009) [2023-12-27 18:30:38,667][20381] Updated weights for policy 1, policy_version 1855229 (0.0006) [2023-12-27 18:30:38,723][20381] Updated weights for policy 1, policy_version 1855239 (0.0007) [2023-12-27 18:30:38,772][20381] Updated weights for policy 1, policy_version 1855249 (0.0007) [2023-12-27 18:30:39,084][20347] Updated weights for policy 0, policy_version 1856830 (0.0010) [2023-12-27 18:30:39,139][20347] Updated weights for policy 0, policy_version 1856840 (0.0010) [2023-12-27 18:30:39,200][20347] Updated weights for policy 0, policy_version 1856850 (0.0010) [2023-12-27 18:30:39,473][20381] Updated weights for policy 1, policy_version 1855259 (0.0007) [2023-12-27 18:30:39,537][20381] Updated weights for policy 1, policy_version 1855269 (0.0008) [2023-12-27 18:30:39,595][20381] Updated weights for policy 1, policy_version 1855279 (0.0007) [2023-12-27 18:30:39,938][20347] Updated weights for policy 0, policy_version 1856860 (0.0009) [2023-12-27 18:30:40,005][20347] Updated weights for policy 0, policy_version 1856870 (0.0010) [2023-12-27 18:30:40,072][20347] Updated weights for policy 0, policy_version 1856880 (0.0011) [2023-12-27 18:30:40,341][20381] Updated weights for policy 1, policy_version 1855289 (0.0006) [2023-12-27 18:30:40,399][20381] Updated weights for policy 1, policy_version 1855299 (0.0008) [2023-12-27 18:30:40,460][20381] Updated weights for policy 1, policy_version 1855309 (0.0009) [2023-12-27 18:30:40,514][20381] Updated weights for policy 1, policy_version 1855319 (0.0009) [2023-12-27 18:30:40,776][20347] Updated weights for policy 0, policy_version 1856890 (0.0011) [2023-12-27 18:30:40,825][20347] Updated weights for policy 0, policy_version 1856900 (0.0010) [2023-12-27 18:30:40,890][20347] Updated weights for policy 0, policy_version 1856910 (0.0010) [2023-12-27 18:30:40,945][20347] Updated weights for policy 0, policy_version 1856920 (0.0010) [2023-12-27 18:30:41,376][20381] Updated weights for policy 1, policy_version 1855329 (0.0011) [2023-12-27 18:30:41,438][20381] Updated weights for policy 1, policy_version 1855339 (0.0007) [2023-12-27 18:30:41,504][20381] Updated weights for policy 1, policy_version 1855349 (0.0008) [2023-12-27 18:30:41,778][20347] Updated weights for policy 0, policy_version 1856930 (0.0008) [2023-12-27 18:30:41,842][20347] Updated weights for policy 0, policy_version 1856940 (0.0008) [2023-12-27 18:30:41,904][20347] Updated weights for policy 0, policy_version 1856950 (0.0007) [2023-12-27 18:30:42,252][20381] Updated weights for policy 1, policy_version 1855359 (0.0009) [2023-12-27 18:30:42,313][20381] Updated weights for policy 1, policy_version 1855369 (0.0008) [2023-12-27 18:30:42,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 950484992. Throughput: 0: 10245.3, 1: 10238.8. Samples: 950481404. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:42,363][19170] Avg episode reward: [(0, '990.230'), (1, '880.790')] [2023-12-27 18:30:42,378][20381] Updated weights for policy 1, policy_version 1855379 (0.0007) [2023-12-27 18:30:42,680][20347] Updated weights for policy 0, policy_version 1856960 (0.0008) [2023-12-27 18:30:42,733][20347] Updated weights for policy 0, policy_version 1856970 (0.0007) [2023-12-27 18:30:42,791][20347] Updated weights for policy 0, policy_version 1856980 (0.0008) [2023-12-27 18:30:43,022][20381] Updated weights for policy 1, policy_version 1855389 (0.0007) [2023-12-27 18:30:43,079][20381] Updated weights for policy 1, policy_version 1855399 (0.0008) [2023-12-27 18:30:43,138][20381] Updated weights for policy 1, policy_version 1855409 (0.0007) [2023-12-27 18:30:43,480][20347] Updated weights for policy 0, policy_version 1856990 (0.0008) [2023-12-27 18:30:43,535][20347] Updated weights for policy 0, policy_version 1857000 (0.0008) [2023-12-27 18:30:43,590][20347] Updated weights for policy 0, policy_version 1857010 (0.0008) [2023-12-27 18:30:43,801][20381] Updated weights for policy 1, policy_version 1855419 (0.0008) [2023-12-27 18:30:43,864][20381] Updated weights for policy 1, policy_version 1855429 (0.0008) [2023-12-27 18:30:43,926][20381] Updated weights for policy 1, policy_version 1855439 (0.0006) [2023-12-27 18:30:44,231][20347] Updated weights for policy 0, policy_version 1857020 (0.0009) [2023-12-27 18:30:44,286][20347] Updated weights for policy 0, policy_version 1857030 (0.0010) [2023-12-27 18:30:44,344][20347] Updated weights for policy 0, policy_version 1857040 (0.0010) [2023-12-27 18:30:44,601][20381] Updated weights for policy 1, policy_version 1855449 (0.0008) [2023-12-27 18:30:44,665][20381] Updated weights for policy 1, policy_version 1855459 (0.0010) [2023-12-27 18:30:44,718][20381] Updated weights for policy 1, policy_version 1855469 (0.0010) [2023-12-27 18:30:44,797][20381] Updated weights for policy 1, policy_version 1855479 (0.0010) [2023-12-27 18:30:45,000][20347] Updated weights for policy 0, policy_version 1857050 (0.0010) [2023-12-27 18:30:45,061][20347] Updated weights for policy 0, policy_version 1857060 (0.0007) [2023-12-27 18:30:45,115][20347] Updated weights for policy 0, policy_version 1857070 (0.0008) [2023-12-27 18:30:45,178][20347] Updated weights for policy 0, policy_version 1857080 (0.0008) [2023-12-27 18:30:45,486][20381] Updated weights for policy 1, policy_version 1855489 (0.0011) [2023-12-27 18:30:45,538][20381] Updated weights for policy 1, policy_version 1855499 (0.0010) [2023-12-27 18:30:45,587][20381] Updated weights for policy 1, policy_version 1855509 (0.0010) [2023-12-27 18:30:45,875][20347] Updated weights for policy 0, policy_version 1857090 (0.0007) [2023-12-27 18:30:45,931][20347] Updated weights for policy 0, policy_version 1857100 (0.0008) [2023-12-27 18:30:45,990][20347] Updated weights for policy 0, policy_version 1857110 (0.0008) [2023-12-27 18:30:46,234][20381] Updated weights for policy 1, policy_version 1855519 (0.0007) [2023-12-27 18:30:46,294][20381] Updated weights for policy 1, policy_version 1855529 (0.0010) [2023-12-27 18:30:46,346][20381] Updated weights for policy 1, policy_version 1855539 (0.0010) [2023-12-27 18:30:46,731][20347] Updated weights for policy 0, policy_version 1857120 (0.0006) [2023-12-27 18:30:46,794][20347] Updated weights for policy 0, policy_version 1857130 (0.0008) [2023-12-27 18:30:46,859][20347] Updated weights for policy 0, policy_version 1857140 (0.0007) [2023-12-27 18:30:47,004][20381] Updated weights for policy 1, policy_version 1855549 (0.0008) [2023-12-27 18:30:47,061][20381] Updated weights for policy 1, policy_version 1855559 (0.0007) [2023-12-27 18:30:47,114][20381] Updated weights for policy 1, policy_version 1855569 (0.0008) [2023-12-27 18:30:47,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 950591488. Throughput: 0: 10192.7, 1: 10250.4. Samples: 950600112. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:47,364][19170] Avg episode reward: [(0, '990.230'), (1, '831.380')] [2023-12-27 18:30:47,535][20347] Updated weights for policy 0, policy_version 1857150 (0.0006) [2023-12-27 18:30:47,583][20347] Updated weights for policy 0, policy_version 1857160 (0.0005) [2023-12-27 18:30:47,631][20347] Updated weights for policy 0, policy_version 1857170 (0.0006) [2023-12-27 18:30:47,744][20381] Updated weights for policy 1, policy_version 1855579 (0.0009) [2023-12-27 18:30:47,807][20381] Updated weights for policy 1, policy_version 1855589 (0.0010) [2023-12-27 18:30:47,869][20381] Updated weights for policy 1, policy_version 1855599 (0.0011) [2023-12-27 18:30:48,208][20347] Updated weights for policy 0, policy_version 1857180 (0.0007) [2023-12-27 18:30:48,269][20347] Updated weights for policy 0, policy_version 1857190 (0.0006) [2023-12-27 18:30:48,346][20347] Updated weights for policy 0, policy_version 1857200 (0.0009) [2023-12-27 18:30:48,556][20381] Updated weights for policy 1, policy_version 1855609 (0.0010) [2023-12-27 18:30:48,619][20381] Updated weights for policy 1, policy_version 1855619 (0.0006) [2023-12-27 18:30:48,682][20381] Updated weights for policy 1, policy_version 1855629 (0.0008) [2023-12-27 18:30:48,746][20381] Updated weights for policy 1, policy_version 1855639 (0.0011) [2023-12-27 18:30:49,041][20347] Updated weights for policy 0, policy_version 1857210 (0.0011) [2023-12-27 18:30:49,093][20347] Updated weights for policy 0, policy_version 1857220 (0.0010) [2023-12-27 18:30:49,139][20347] Updated weights for policy 0, policy_version 1857230 (0.0010) [2023-12-27 18:30:49,188][20347] Updated weights for policy 0, policy_version 1857240 (0.0010) [2023-12-27 18:30:49,429][20381] Updated weights for policy 1, policy_version 1855649 (0.0006) [2023-12-27 18:30:49,494][20381] Updated weights for policy 1, policy_version 1855659 (0.0007) [2023-12-27 18:30:49,557][20381] Updated weights for policy 1, policy_version 1855669 (0.0006) [2023-12-27 18:30:49,978][20347] Updated weights for policy 0, policy_version 1857250 (0.0006) [2023-12-27 18:30:50,039][20347] Updated weights for policy 0, policy_version 1857260 (0.0006) [2023-12-27 18:30:50,100][20347] Updated weights for policy 0, policy_version 1857270 (0.0006) [2023-12-27 18:30:50,260][20381] Updated weights for policy 1, policy_version 1855679 (0.0008) [2023-12-27 18:30:50,311][20381] Updated weights for policy 1, policy_version 1855689 (0.0007) [2023-12-27 18:30:50,372][20381] Updated weights for policy 1, policy_version 1855699 (0.0008) [2023-12-27 18:30:50,836][20347] Updated weights for policy 0, policy_version 1857280 (0.0007) [2023-12-27 18:30:50,895][20347] Updated weights for policy 0, policy_version 1857290 (0.0009) [2023-12-27 18:30:50,952][20347] Updated weights for policy 0, policy_version 1857300 (0.0008) [2023-12-27 18:30:51,141][20381] Updated weights for policy 1, policy_version 1855709 (0.0007) [2023-12-27 18:30:51,199][20381] Updated weights for policy 1, policy_version 1855719 (0.0008) [2023-12-27 18:30:51,258][20381] Updated weights for policy 1, policy_version 1855729 (0.0007) [2023-12-27 18:30:51,688][20347] Updated weights for policy 0, policy_version 1857310 (0.0007) [2023-12-27 18:30:51,756][20347] Updated weights for policy 0, policy_version 1857320 (0.0010) [2023-12-27 18:30:51,812][20347] Updated weights for policy 0, policy_version 1857330 (0.0008) [2023-12-27 18:30:51,966][20381] Updated weights for policy 1, policy_version 1855739 (0.0007) [2023-12-27 18:30:52,029][20381] Updated weights for policy 1, policy_version 1855749 (0.0009) [2023-12-27 18:30:52,090][20381] Updated weights for policy 1, policy_version 1855759 (0.0008) [2023-12-27 18:30:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20493.9). Total num frames: 950689792. Throughput: 0: 10161.0, 1: 10248.7. Samples: 950661412. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) [2023-12-27 18:30:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '841.320')] [2023-12-27 18:30:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001857336_475561984.pth... [2023-12-27 18:30:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001855768_475127808.pth... [2023-12-27 18:30:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001856152_475258880.pth [2023-12-27 18:30:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001854552_474816512.pth [2023-12-27 18:30:52,579][20347] Updated weights for policy 0, policy_version 1857340 (0.0008) [2023-12-27 18:30:52,639][20347] Updated weights for policy 0, policy_version 1857350 (0.0008) [2023-12-27 18:30:52,702][20347] Updated weights for policy 0, policy_version 1857360 (0.0008) [2023-12-27 18:30:52,802][20381] Updated weights for policy 1, policy_version 1855769 (0.0007) [2023-12-27 18:30:52,861][20381] Updated weights for policy 1, policy_version 1855779 (0.0008) [2023-12-27 18:30:52,912][20381] Updated weights for policy 1, policy_version 1855789 (0.0006) [2023-12-27 18:30:52,960][20381] Updated weights for policy 1, policy_version 1855799 (0.0009) [2023-12-27 18:30:53,342][20347] Updated weights for policy 0, policy_version 1857370 (0.0007) [2023-12-27 18:30:53,410][20347] Updated weights for policy 0, policy_version 1857380 (0.0005) [2023-12-27 18:30:53,480][20347] Updated weights for policy 0, policy_version 1857390 (0.0007) [2023-12-27 18:30:53,524][20347] Updated weights for policy 0, policy_version 1857400 (0.0008) [2023-12-27 18:30:53,638][20381] Updated weights for policy 1, policy_version 1855809 (0.0009) [2023-12-27 18:30:53,683][20381] Updated weights for policy 1, policy_version 1855819 (0.0010) [2023-12-27 18:30:53,728][20381] Updated weights for policy 1, policy_version 1855829 (0.0011) [2023-12-27 18:30:54,169][20347] Updated weights for policy 0, policy_version 1857410 (0.0005) [2023-12-27 18:30:54,219][20347] Updated weights for policy 0, policy_version 1857420 (0.0006) [2023-12-27 18:30:54,274][20347] Updated weights for policy 0, policy_version 1857430 (0.0006) [2023-12-27 18:30:54,401][20381] Updated weights for policy 1, policy_version 1855839 (0.0010) [2023-12-27 18:30:54,462][20381] Updated weights for policy 1, policy_version 1855849 (0.0011) [2023-12-27 18:30:54,515][20381] Updated weights for policy 1, policy_version 1855859 (0.0010) [2023-12-27 18:30:54,822][20347] Updated weights for policy 0, policy_version 1857440 (0.0006) [2023-12-27 18:30:54,882][20347] Updated weights for policy 0, policy_version 1857450 (0.0009) [2023-12-27 18:30:54,930][20347] Updated weights for policy 0, policy_version 1857460 (0.0010) [2023-12-27 18:30:55,136][20381] Updated weights for policy 1, policy_version 1855869 (0.0010) [2023-12-27 18:30:55,203][20381] Updated weights for policy 1, policy_version 1855879 (0.0005) [2023-12-27 18:30:55,259][20381] Updated weights for policy 1, policy_version 1855889 (0.0007) [2023-12-27 18:30:55,584][20347] Updated weights for policy 0, policy_version 1857470 (0.0008) [2023-12-27 18:30:55,635][20347] Updated weights for policy 0, policy_version 1857480 (0.0007) [2023-12-27 18:30:55,685][20347] Updated weights for policy 0, policy_version 1857490 (0.0007) [2023-12-27 18:30:55,927][20381] Updated weights for policy 1, policy_version 1855899 (0.0008) [2023-12-27 18:30:55,978][20381] Updated weights for policy 1, policy_version 1855909 (0.0007) [2023-12-27 18:30:56,037][20381] Updated weights for policy 1, policy_version 1855919 (0.0008) [2023-12-27 18:30:56,360][20347] Updated weights for policy 0, policy_version 1857500 (0.0007) [2023-12-27 18:30:56,420][20347] Updated weights for policy 0, policy_version 1857510 (0.0007) [2023-12-27 18:30:56,478][20347] Updated weights for policy 0, policy_version 1857520 (0.0007) [2023-12-27 18:30:56,749][20381] Updated weights for policy 1, policy_version 1855929 (0.0007) [2023-12-27 18:30:56,798][20381] Updated weights for policy 1, policy_version 1855939 (0.0008) [2023-12-27 18:30:56,842][20381] Updated weights for policy 1, policy_version 1855949 (0.0008) [2023-12-27 18:30:56,893][20381] Updated weights for policy 1, policy_version 1855959 (0.0008) [2023-12-27 18:30:57,138][20347] Updated weights for policy 0, policy_version 1857530 (0.0007) [2023-12-27 18:30:57,191][20347] Updated weights for policy 0, policy_version 1857540 (0.0010) [2023-12-27 18:30:57,249][20347] Updated weights for policy 0, policy_version 1857550 (0.0010) [2023-12-27 18:30:57,309][20347] Updated weights for policy 0, policy_version 1857560 (0.0009) [2023-12-27 18:30:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 950796288. Throughput: 0: 10121.7, 1: 10230.4. Samples: 950782976. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:30:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.790')] [2023-12-27 18:30:57,616][20381] Updated weights for policy 1, policy_version 1855969 (0.0007) [2023-12-27 18:30:57,672][20381] Updated weights for policy 1, policy_version 1855979 (0.0007) [2023-12-27 18:30:57,727][20381] Updated weights for policy 1, policy_version 1855989 (0.0008) [2023-12-27 18:30:57,951][20347] Updated weights for policy 0, policy_version 1857570 (0.0010) [2023-12-27 18:30:58,011][20347] Updated weights for policy 0, policy_version 1857580 (0.0010) [2023-12-27 18:30:58,065][20347] Updated weights for policy 0, policy_version 1857590 (0.0010) [2023-12-27 18:30:58,388][20381] Updated weights for policy 1, policy_version 1855999 (0.0008) [2023-12-27 18:30:58,451][20381] Updated weights for policy 1, policy_version 1856009 (0.0007) [2023-12-27 18:30:58,519][20381] Updated weights for policy 1, policy_version 1856019 (0.0010) [2023-12-27 18:30:58,827][20347] Updated weights for policy 0, policy_version 1857600 (0.0008) [2023-12-27 18:30:58,893][20347] Updated weights for policy 0, policy_version 1857610 (0.0008) [2023-12-27 18:30:58,953][20347] Updated weights for policy 0, policy_version 1857620 (0.0008) [2023-12-27 18:30:59,304][20381] Updated weights for policy 1, policy_version 1856029 (0.0009) [2023-12-27 18:30:59,378][20381] Updated weights for policy 1, policy_version 1856039 (0.0008) [2023-12-27 18:30:59,441][20381] Updated weights for policy 1, policy_version 1856049 (0.0006) [2023-12-27 18:30:59,801][20347] Updated weights for policy 0, policy_version 1857630 (0.0008) [2023-12-27 18:30:59,859][20347] Updated weights for policy 0, policy_version 1857640 (0.0008) [2023-12-27 18:30:59,909][20347] Updated weights for policy 0, policy_version 1857650 (0.0008) [2023-12-27 18:31:00,145][20381] Updated weights for policy 1, policy_version 1856059 (0.0006) [2023-12-27 18:31:00,200][20381] Updated weights for policy 1, policy_version 1856069 (0.0007) [2023-12-27 18:31:00,258][20381] Updated weights for policy 1, policy_version 1856079 (0.0007) [2023-12-27 18:31:00,560][20347] Updated weights for policy 0, policy_version 1857660 (0.0008) [2023-12-27 18:31:00,617][20347] Updated weights for policy 0, policy_version 1857670 (0.0008) [2023-12-27 18:31:00,678][20347] Updated weights for policy 0, policy_version 1857680 (0.0006) [2023-12-27 18:31:00,971][20381] Updated weights for policy 1, policy_version 1856089 (0.0007) [2023-12-27 18:31:01,021][20381] Updated weights for policy 1, policy_version 1856099 (0.0006) [2023-12-27 18:31:01,089][20381] Updated weights for policy 1, policy_version 1856109 (0.0008) [2023-12-27 18:31:01,160][20381] Updated weights for policy 1, policy_version 1856119 (0.0007) [2023-12-27 18:31:01,376][20347] Updated weights for policy 0, policy_version 1857690 (0.0006) [2023-12-27 18:31:01,431][20347] Updated weights for policy 0, policy_version 1857700 (0.0008) [2023-12-27 18:31:01,482][20347] Updated weights for policy 0, policy_version 1857710 (0.0008) [2023-12-27 18:31:01,540][20347] Updated weights for policy 0, policy_version 1857720 (0.0008) [2023-12-27 18:31:01,881][20381] Updated weights for policy 1, policy_version 1856129 (0.0006) [2023-12-27 18:31:01,935][20381] Updated weights for policy 1, policy_version 1856139 (0.0006) [2023-12-27 18:31:01,991][20381] Updated weights for policy 1, policy_version 1856149 (0.0007) [2023-12-27 18:31:02,221][20347] Updated weights for policy 0, policy_version 1857730 (0.0006) [2023-12-27 18:31:02,285][20347] Updated weights for policy 0, policy_version 1857740 (0.0010) [2023-12-27 18:31:02,352][20347] Updated weights for policy 0, policy_version 1857750 (0.0007) [2023-12-27 18:31:02,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20346.7, 300 sec: 20466.1). Total num frames: 950886400. Throughput: 0: 10123.2, 1: 10177.6. Samples: 950901484. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.410')] [2023-12-27 18:31:02,681][20381] Updated weights for policy 1, policy_version 1856159 (0.0006) [2023-12-27 18:31:02,740][20381] Updated weights for policy 1, policy_version 1856169 (0.0006) [2023-12-27 18:31:02,798][20381] Updated weights for policy 1, policy_version 1856179 (0.0007) [2023-12-27 18:31:02,977][20347] Updated weights for policy 0, policy_version 1857760 (0.0008) [2023-12-27 18:31:03,033][20347] Updated weights for policy 0, policy_version 1857770 (0.0007) [2023-12-27 18:31:03,084][20347] Updated weights for policy 0, policy_version 1857780 (0.0008) [2023-12-27 18:31:03,478][20381] Updated weights for policy 1, policy_version 1856189 (0.0007) [2023-12-27 18:31:03,530][20381] Updated weights for policy 1, policy_version 1856199 (0.0006) [2023-12-27 18:31:03,595][20381] Updated weights for policy 1, policy_version 1856209 (0.0005) [2023-12-27 18:31:03,786][20347] Updated weights for policy 0, policy_version 1857790 (0.0007) [2023-12-27 18:31:03,843][20347] Updated weights for policy 0, policy_version 1857800 (0.0008) [2023-12-27 18:31:03,908][20347] Updated weights for policy 0, policy_version 1857810 (0.0007) [2023-12-27 18:31:04,250][20381] Updated weights for policy 1, policy_version 1856219 (0.0007) [2023-12-27 18:31:04,306][20381] Updated weights for policy 1, policy_version 1856229 (0.0009) [2023-12-27 18:31:04,361][20381] Updated weights for policy 1, policy_version 1856239 (0.0009) [2023-12-27 18:31:04,634][20347] Updated weights for policy 0, policy_version 1857820 (0.0006) [2023-12-27 18:31:04,691][20347] Updated weights for policy 0, policy_version 1857830 (0.0008) [2023-12-27 18:31:04,742][20347] Updated weights for policy 0, policy_version 1857840 (0.0008) [2023-12-27 18:31:05,055][20381] Updated weights for policy 1, policy_version 1856249 (0.0009) [2023-12-27 18:31:05,110][20381] Updated weights for policy 1, policy_version 1856259 (0.0008) [2023-12-27 18:31:05,163][20381] Updated weights for policy 1, policy_version 1856269 (0.0008) [2023-12-27 18:31:05,210][20381] Updated weights for policy 1, policy_version 1856279 (0.0007) [2023-12-27 18:31:05,431][20347] Updated weights for policy 0, policy_version 1857850 (0.0008) [2023-12-27 18:31:05,482][20347] Updated weights for policy 0, policy_version 1857860 (0.0007) [2023-12-27 18:31:05,534][20347] Updated weights for policy 0, policy_version 1857870 (0.0008) [2023-12-27 18:31:05,580][20347] Updated weights for policy 0, policy_version 1857880 (0.0010) [2023-12-27 18:31:05,858][20381] Updated weights for policy 1, policy_version 1856289 (0.0008) [2023-12-27 18:31:05,920][20381] Updated weights for policy 1, policy_version 1856299 (0.0008) [2023-12-27 18:31:05,977][20381] Updated weights for policy 1, policy_version 1856309 (0.0007) [2023-12-27 18:31:06,337][20347] Updated weights for policy 0, policy_version 1857890 (0.0007) [2023-12-27 18:31:06,397][20347] Updated weights for policy 0, policy_version 1857900 (0.0008) [2023-12-27 18:31:06,447][20347] Updated weights for policy 0, policy_version 1857910 (0.0008) [2023-12-27 18:31:06,725][20381] Updated weights for policy 1, policy_version 1856319 (0.0008) [2023-12-27 18:31:06,785][20381] Updated weights for policy 1, policy_version 1856329 (0.0008) [2023-12-27 18:31:06,838][20381] Updated weights for policy 1, policy_version 1856339 (0.0008) [2023-12-27 18:31:07,143][20347] Updated weights for policy 0, policy_version 1857920 (0.0007) [2023-12-27 18:31:07,202][20347] Updated weights for policy 0, policy_version 1857930 (0.0007) [2023-12-27 18:31:07,262][20347] Updated weights for policy 0, policy_version 1857940 (0.0005) [2023-12-27 18:31:07,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 950992896. Throughput: 0: 10130.9, 1: 10168.5. Samples: 950961760. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '848.440')] [2023-12-27 18:31:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001857944_475717632.pth... [2023-12-27 18:31:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001856344_475275264.pth... [2023-12-27 18:31:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001856760_475414528.pth [2023-12-27 18:31:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001855192_474980352.pth [2023-12-27 18:31:07,565][20381] Updated weights for policy 1, policy_version 1856349 (0.0008) [2023-12-27 18:31:07,615][20381] Updated weights for policy 1, policy_version 1856359 (0.0008) [2023-12-27 18:31:07,674][20381] Updated weights for policy 1, policy_version 1856369 (0.0007) [2023-12-27 18:31:07,915][20347] Updated weights for policy 0, policy_version 1857950 (0.0006) [2023-12-27 18:31:07,967][20347] Updated weights for policy 0, policy_version 1857960 (0.0007) [2023-12-27 18:31:08,016][20347] Updated weights for policy 0, policy_version 1857970 (0.0007) [2023-12-27 18:31:08,419][20381] Updated weights for policy 1, policy_version 1856379 (0.0008) [2023-12-27 18:31:08,489][20381] Updated weights for policy 1, policy_version 1856389 (0.0008) [2023-12-27 18:31:08,555][20381] Updated weights for policy 1, policy_version 1856399 (0.0007) [2023-12-27 18:31:08,728][20347] Updated weights for policy 0, policy_version 1857980 (0.0007) [2023-12-27 18:31:08,792][20347] Updated weights for policy 0, policy_version 1857990 (0.0009) [2023-12-27 18:31:08,852][20347] Updated weights for policy 0, policy_version 1858000 (0.0010) [2023-12-27 18:31:09,284][20381] Updated weights for policy 1, policy_version 1856409 (0.0007) [2023-12-27 18:31:09,349][20381] Updated weights for policy 1, policy_version 1856419 (0.0008) [2023-12-27 18:31:09,414][20381] Updated weights for policy 1, policy_version 1856429 (0.0009) [2023-12-27 18:31:09,479][20381] Updated weights for policy 1, policy_version 1856439 (0.0008) [2023-12-27 18:31:09,507][20347] Updated weights for policy 0, policy_version 1858010 (0.0010) [2023-12-27 18:31:09,564][20347] Updated weights for policy 0, policy_version 1858020 (0.0008) [2023-12-27 18:31:09,621][20347] Updated weights for policy 0, policy_version 1858030 (0.0007) [2023-12-27 18:31:09,677][20347] Updated weights for policy 0, policy_version 1858040 (0.0007) [2023-12-27 18:31:10,230][20381] Updated weights for policy 1, policy_version 1856449 (0.0008) [2023-12-27 18:31:10,280][20381] Updated weights for policy 1, policy_version 1856459 (0.0009) [2023-12-27 18:31:10,337][20381] Updated weights for policy 1, policy_version 1856469 (0.0011) [2023-12-27 18:31:10,437][20347] Updated weights for policy 0, policy_version 1858050 (0.0008) [2023-12-27 18:31:10,492][20347] Updated weights for policy 0, policy_version 1858060 (0.0007) [2023-12-27 18:31:10,552][20347] Updated weights for policy 0, policy_version 1858070 (0.0008) [2023-12-27 18:31:11,068][20381] Updated weights for policy 1, policy_version 1856479 (0.0010) [2023-12-27 18:31:11,151][20381] Updated weights for policy 1, policy_version 1856491 (0.0011) [2023-12-27 18:31:11,217][20381] Updated weights for policy 1, policy_version 1856501 (0.0010) [2023-12-27 18:31:11,251][20347] Updated weights for policy 0, policy_version 1858080 (0.0008) [2023-12-27 18:31:11,306][20347] Updated weights for policy 0, policy_version 1858090 (0.0007) [2023-12-27 18:31:11,374][20347] Updated weights for policy 0, policy_version 1858100 (0.0008) [2023-12-27 18:31:11,966][20381] Updated weights for policy 1, policy_version 1856511 (0.0009) [2023-12-27 18:31:12,023][20381] Updated weights for policy 1, policy_version 1856521 (0.0008) [2023-12-27 18:31:12,084][20381] Updated weights for policy 1, policy_version 1856531 (0.0008) [2023-12-27 18:31:12,143][20347] Updated weights for policy 0, policy_version 1858110 (0.0008) [2023-12-27 18:31:12,207][20347] Updated weights for policy 0, policy_version 1858120 (0.0007) [2023-12-27 18:31:12,271][20347] Updated weights for policy 0, policy_version 1858130 (0.0008) [2023-12-27 18:31:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 951091200. Throughput: 0: 10115.6, 1: 10112.6. Samples: 951079452. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '759.740')] [2023-12-27 18:31:12,816][20381] Updated weights for policy 1, policy_version 1856541 (0.0008) [2023-12-27 18:31:12,865][20381] Updated weights for policy 1, policy_version 1856551 (0.0008) [2023-12-27 18:31:12,920][20381] Updated weights for policy 1, policy_version 1856561 (0.0007) [2023-12-27 18:31:12,992][20347] Updated weights for policy 0, policy_version 1858140 (0.0008) [2023-12-27 18:31:13,042][20347] Updated weights for policy 0, policy_version 1858150 (0.0011) [2023-12-27 18:31:13,098][20347] Updated weights for policy 0, policy_version 1858160 (0.0010) [2023-12-27 18:31:13,682][20381] Updated weights for policy 1, policy_version 1856571 (0.0006) [2023-12-27 18:31:13,733][20381] Updated weights for policy 1, policy_version 1856581 (0.0007) [2023-12-27 18:31:13,783][20381] Updated weights for policy 1, policy_version 1856591 (0.0007) [2023-12-27 18:31:13,809][20347] Updated weights for policy 0, policy_version 1858170 (0.0010) [2023-12-27 18:31:13,873][20347] Updated weights for policy 0, policy_version 1858180 (0.0006) [2023-12-27 18:31:13,932][20347] Updated weights for policy 0, policy_version 1858190 (0.0007) [2023-12-27 18:31:13,990][20347] Updated weights for policy 0, policy_version 1858200 (0.0008) [2023-12-27 18:31:14,488][20381] Updated weights for policy 1, policy_version 1856601 (0.0007) [2023-12-27 18:31:14,551][20381] Updated weights for policy 1, policy_version 1856611 (0.0010) [2023-12-27 18:31:14,599][20381] Updated weights for policy 1, policy_version 1856621 (0.0010) [2023-12-27 18:31:14,621][20347] Updated weights for policy 0, policy_version 1858210 (0.0006) [2023-12-27 18:31:14,644][20381] Updated weights for policy 1, policy_version 1856631 (0.0010) [2023-12-27 18:31:14,681][20347] Updated weights for policy 0, policy_version 1858220 (0.0007) [2023-12-27 18:31:14,728][20347] Updated weights for policy 0, policy_version 1858230 (0.0007) [2023-12-27 18:31:15,396][20381] Updated weights for policy 1, policy_version 1856641 (0.0008) [2023-12-27 18:31:15,447][20381] Updated weights for policy 1, policy_version 1856651 (0.0007) [2023-12-27 18:31:15,458][20347] Updated weights for policy 0, policy_version 1858240 (0.0007) [2023-12-27 18:31:15,496][20381] Updated weights for policy 1, policy_version 1856661 (0.0007) [2023-12-27 18:31:15,521][20347] Updated weights for policy 0, policy_version 1858250 (0.0006) [2023-12-27 18:31:15,584][20347] Updated weights for policy 0, policy_version 1858260 (0.0007) [2023-12-27 18:31:16,193][20347] Updated weights for policy 0, policy_version 1858270 (0.0006) [2023-12-27 18:31:16,195][20381] Updated weights for policy 1, policy_version 1856671 (0.0009) [2023-12-27 18:31:16,246][20347] Updated weights for policy 0, policy_version 1858280 (0.0006) [2023-12-27 18:31:16,255][20381] Updated weights for policy 1, policy_version 1856681 (0.0011) [2023-12-27 18:31:16,305][20347] Updated weights for policy 0, policy_version 1858290 (0.0006) [2023-12-27 18:31:16,313][20381] Updated weights for policy 1, policy_version 1856691 (0.0011) [2023-12-27 18:31:16,907][20347] Updated weights for policy 0, policy_version 1858300 (0.0007) [2023-12-27 18:31:16,958][20381] Updated weights for policy 1, policy_version 1856701 (0.0011) [2023-12-27 18:31:16,960][20347] Updated weights for policy 0, policy_version 1858310 (0.0009) [2023-12-27 18:31:17,009][20381] Updated weights for policy 1, policy_version 1856711 (0.0010) [2023-12-27 18:31:17,015][20347] Updated weights for policy 0, policy_version 1858320 (0.0010) [2023-12-27 18:31:17,068][20381] Updated weights for policy 1, policy_version 1856721 (0.0009) [2023-12-27 18:31:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 951189504. Throughput: 0: 10047.7, 1: 9938.5. Samples: 951198312. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '820.160')] [2023-12-27 18:31:17,707][20347] Updated weights for policy 0, policy_version 1858330 (0.0010) [2023-12-27 18:31:17,743][20381] Updated weights for policy 1, policy_version 1856731 (0.0007) [2023-12-27 18:31:17,771][20347] Updated weights for policy 0, policy_version 1858340 (0.0010) [2023-12-27 18:31:17,796][20381] Updated weights for policy 1, policy_version 1856741 (0.0005) [2023-12-27 18:31:17,822][20347] Updated weights for policy 0, policy_version 1858350 (0.0009) [2023-12-27 18:31:17,853][20381] Updated weights for policy 1, policy_version 1856751 (0.0005) [2023-12-27 18:31:17,870][20347] Updated weights for policy 0, policy_version 1858360 (0.0007) [2023-12-27 18:31:18,451][20347] Updated weights for policy 0, policy_version 1858370 (0.0008) [2023-12-27 18:31:18,463][20381] Updated weights for policy 1, policy_version 1856761 (0.0006) [2023-12-27 18:31:18,515][20347] Updated weights for policy 0, policy_version 1858380 (0.0008) [2023-12-27 18:31:18,530][20381] Updated weights for policy 1, policy_version 1856771 (0.0006) [2023-12-27 18:31:18,573][20347] Updated weights for policy 0, policy_version 1858390 (0.0007) [2023-12-27 18:31:18,597][20381] Updated weights for policy 1, policy_version 1856781 (0.0006) [2023-12-27 18:31:18,654][20381] Updated weights for policy 1, policy_version 1856791 (0.0008) [2023-12-27 18:31:19,241][20347] Updated weights for policy 0, policy_version 1858400 (0.0008) [2023-12-27 18:31:19,300][20347] Updated weights for policy 0, policy_version 1858410 (0.0008) [2023-12-27 18:31:19,323][20381] Updated weights for policy 1, policy_version 1856801 (0.0008) [2023-12-27 18:31:19,368][20347] Updated weights for policy 0, policy_version 1858420 (0.0008) [2023-12-27 18:31:19,387][20381] Updated weights for policy 1, policy_version 1856811 (0.0008) [2023-12-27 18:31:19,447][20381] Updated weights for policy 1, policy_version 1856821 (0.0007) [2023-12-27 18:31:20,097][20347] Updated weights for policy 0, policy_version 1858430 (0.0007) [2023-12-27 18:31:20,156][20347] Updated weights for policy 0, policy_version 1858440 (0.0007) [2023-12-27 18:31:20,215][20347] Updated weights for policy 0, policy_version 1858450 (0.0008) [2023-12-27 18:31:20,218][20381] Updated weights for policy 1, policy_version 1856831 (0.0008) [2023-12-27 18:31:20,264][20381] Updated weights for policy 1, policy_version 1856841 (0.0008) [2023-12-27 18:31:20,315][20381] Updated weights for policy 1, policy_version 1856851 (0.0008) [2023-12-27 18:31:20,942][20347] Updated weights for policy 0, policy_version 1858460 (0.0008) [2023-12-27 18:31:20,996][20381] Updated weights for policy 1, policy_version 1856861 (0.0007) [2023-12-27 18:31:20,999][20347] Updated weights for policy 0, policy_version 1858470 (0.0007) [2023-12-27 18:31:21,061][20381] Updated weights for policy 1, policy_version 1856871 (0.0008) [2023-12-27 18:31:21,066][20347] Updated weights for policy 0, policy_version 1858480 (0.0008) [2023-12-27 18:31:21,124][20381] Updated weights for policy 1, policy_version 1856881 (0.0007) [2023-12-27 18:31:21,746][20347] Updated weights for policy 0, policy_version 1858490 (0.0007) [2023-12-27 18:31:21,816][20347] Updated weights for policy 0, policy_version 1858500 (0.0009) [2023-12-27 18:31:21,825][20381] Updated weights for policy 1, policy_version 1856891 (0.0008) [2023-12-27 18:31:21,873][20347] Updated weights for policy 0, policy_version 1858510 (0.0011) [2023-12-27 18:31:21,882][20381] Updated weights for policy 1, policy_version 1856901 (0.0006) [2023-12-27 18:31:21,931][20347] Updated weights for policy 0, policy_version 1858520 (0.0010) [2023-12-27 18:31:21,933][20381] Updated weights for policy 1, policy_version 1856911 (0.0007) [2023-12-27 18:31:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 951287808. Throughput: 0: 10033.8, 1: 9926.6. Samples: 951259340. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:22,363][19170] Avg episode reward: [(0, '990.310'), (1, '838.080')] [2023-12-27 18:31:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001856920_475422720.pth... [2023-12-27 18:31:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001858520_475865088.pth... [2023-12-27 18:31:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001855768_475127808.pth [2023-12-27 18:31:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001857336_475561984.pth [2023-12-27 18:31:22,620][20381] Updated weights for policy 1, policy_version 1856921 (0.0007) [2023-12-27 18:31:22,634][20347] Updated weights for policy 0, policy_version 1858530 (0.0010) [2023-12-27 18:31:22,681][20381] Updated weights for policy 1, policy_version 1856931 (0.0006) [2023-12-27 18:31:22,684][20347] Updated weights for policy 0, policy_version 1858540 (0.0011) [2023-12-27 18:31:22,741][20381] Updated weights for policy 1, policy_version 1856941 (0.0006) [2023-12-27 18:31:22,744][20347] Updated weights for policy 0, policy_version 1858550 (0.0011) [2023-12-27 18:31:22,802][20381] Updated weights for policy 1, policy_version 1856951 (0.0007) [2023-12-27 18:31:23,438][20347] Updated weights for policy 0, policy_version 1858560 (0.0006) [2023-12-27 18:31:23,490][20347] Updated weights for policy 0, policy_version 1858570 (0.0010) [2023-12-27 18:31:23,510][20381] Updated weights for policy 1, policy_version 1856961 (0.0006) [2023-12-27 18:31:23,537][20347] Updated weights for policy 0, policy_version 1858580 (0.0009) [2023-12-27 18:31:23,566][20381] Updated weights for policy 1, policy_version 1856971 (0.0007) [2023-12-27 18:31:23,613][20381] Updated weights for policy 1, policy_version 1856981 (0.0005) [2023-12-27 18:31:24,136][20347] Updated weights for policy 0, policy_version 1858590 (0.0008) [2023-12-27 18:31:24,189][20347] Updated weights for policy 0, policy_version 1858600 (0.0010) [2023-12-27 18:31:24,238][20347] Updated weights for policy 0, policy_version 1858610 (0.0010) [2023-12-27 18:31:24,270][20381] Updated weights for policy 1, policy_version 1856991 (0.0006) [2023-12-27 18:31:24,319][20381] Updated weights for policy 1, policy_version 1857001 (0.0009) [2023-12-27 18:31:24,363][20381] Updated weights for policy 1, policy_version 1857011 (0.0008) [2023-12-27 18:31:24,883][20347] Updated weights for policy 0, policy_version 1858620 (0.0008) [2023-12-27 18:31:24,943][20347] Updated weights for policy 0, policy_version 1858630 (0.0006) [2023-12-27 18:31:24,992][20347] Updated weights for policy 0, policy_version 1858640 (0.0007) [2023-12-27 18:31:25,089][20381] Updated weights for policy 1, policy_version 1857021 (0.0007) [2023-12-27 18:31:25,135][20381] Updated weights for policy 1, policy_version 1857031 (0.0007) [2023-12-27 18:31:25,190][20381] Updated weights for policy 1, policy_version 1857041 (0.0007) [2023-12-27 18:31:25,591][20347] Updated weights for policy 0, policy_version 1858650 (0.0006) [2023-12-27 18:31:25,644][20347] Updated weights for policy 0, policy_version 1858660 (0.0006) [2023-12-27 18:31:25,696][20347] Updated weights for policy 0, policy_version 1858670 (0.0006) [2023-12-27 18:31:25,742][20347] Updated weights for policy 0, policy_version 1858680 (0.0008) [2023-12-27 18:31:25,743][20381] Updated weights for policy 1, policy_version 1857051 (0.0005) [2023-12-27 18:31:25,801][20381] Updated weights for policy 1, policy_version 1857061 (0.0006) [2023-12-27 18:31:25,862][20381] Updated weights for policy 1, policy_version 1857071 (0.0005) [2023-12-27 18:31:26,388][20347] Updated weights for policy 0, policy_version 1858690 (0.0010) [2023-12-27 18:31:26,434][20347] Updated weights for policy 0, policy_version 1858700 (0.0010) [2023-12-27 18:31:26,468][20381] Updated weights for policy 1, policy_version 1857081 (0.0007) [2023-12-27 18:31:26,486][20347] Updated weights for policy 0, policy_version 1858710 (0.0006) [2023-12-27 18:31:26,520][20381] Updated weights for policy 1, policy_version 1857091 (0.0007) [2023-12-27 18:31:26,571][20381] Updated weights for policy 1, policy_version 1857101 (0.0008) [2023-12-27 18:31:26,629][20381] Updated weights for policy 1, policy_version 1857111 (0.0010) [2023-12-27 18:31:27,112][20347] Updated weights for policy 0, policy_version 1858720 (0.0009) [2023-12-27 18:31:27,160][20347] Updated weights for policy 0, policy_version 1858730 (0.0010) [2023-12-27 18:31:27,217][20347] Updated weights for policy 0, policy_version 1858740 (0.0010) [2023-12-27 18:31:27,332][20381] Updated weights for policy 1, policy_version 1857121 (0.0009) [2023-12-27 18:31:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20207.0, 300 sec: 20438.4). Total num frames: 951394304. Throughput: 0: 10080.9, 1: 9979.0. Samples: 951384096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:27,363][19170] Avg episode reward: [(0, '990.310'), (1, '750.480')] [2023-12-27 18:31:27,389][20381] Updated weights for policy 1, policy_version 1857131 (0.0006) [2023-12-27 18:31:27,444][20381] Updated weights for policy 1, policy_version 1857141 (0.0010) [2023-12-27 18:31:27,886][20347] Updated weights for policy 0, policy_version 1858750 (0.0010) [2023-12-27 18:31:27,931][20347] Updated weights for policy 0, policy_version 1858760 (0.0010) [2023-12-27 18:31:27,975][20347] Updated weights for policy 0, policy_version 1858770 (0.0010) [2023-12-27 18:31:28,094][20381] Updated weights for policy 1, policy_version 1857151 (0.0010) [2023-12-27 18:31:28,141][20381] Updated weights for policy 1, policy_version 1857161 (0.0010) [2023-12-27 18:31:28,192][20381] Updated weights for policy 1, policy_version 1857171 (0.0010) [2023-12-27 18:31:28,681][20347] Updated weights for policy 0, policy_version 1858780 (0.0010) [2023-12-27 18:31:28,736][20347] Updated weights for policy 0, policy_version 1858790 (0.0010) [2023-12-27 18:31:28,794][20347] Updated weights for policy 0, policy_version 1858800 (0.0010) [2023-12-27 18:31:28,878][20381] Updated weights for policy 1, policy_version 1857181 (0.0009) [2023-12-27 18:31:28,935][20381] Updated weights for policy 1, policy_version 1857191 (0.0008) [2023-12-27 18:31:28,987][20381] Updated weights for policy 1, policy_version 1857201 (0.0007) [2023-12-27 18:31:29,476][20347] Updated weights for policy 0, policy_version 1858810 (0.0010) [2023-12-27 18:31:29,532][20347] Updated weights for policy 0, policy_version 1858820 (0.0010) [2023-12-27 18:31:29,587][20347] Updated weights for policy 0, policy_version 1858830 (0.0010) [2023-12-27 18:31:29,644][20347] Updated weights for policy 0, policy_version 1858840 (0.0010) [2023-12-27 18:31:29,671][20381] Updated weights for policy 1, policy_version 1857211 (0.0007) [2023-12-27 18:31:29,727][20381] Updated weights for policy 1, policy_version 1857221 (0.0007) [2023-12-27 18:31:29,773][20381] Updated weights for policy 1, policy_version 1857231 (0.0007) [2023-12-27 18:31:30,317][20347] Updated weights for policy 0, policy_version 1858850 (0.0010) [2023-12-27 18:31:30,377][20347] Updated weights for policy 0, policy_version 1858860 (0.0006) [2023-12-27 18:31:30,435][20347] Updated weights for policy 0, policy_version 1858870 (0.0005) [2023-12-27 18:31:30,469][20381] Updated weights for policy 1, policy_version 1857241 (0.0007) [2023-12-27 18:31:30,523][20381] Updated weights for policy 1, policy_version 1857251 (0.0008) [2023-12-27 18:31:30,583][20381] Updated weights for policy 1, policy_version 1857261 (0.0006) [2023-12-27 18:31:30,641][20381] Updated weights for policy 1, policy_version 1857271 (0.0005) [2023-12-27 18:31:31,085][20347] Updated weights for policy 0, policy_version 1858880 (0.0007) [2023-12-27 18:31:31,148][20347] Updated weights for policy 0, policy_version 1858890 (0.0010) [2023-12-27 18:31:31,197][20381] Updated weights for policy 1, policy_version 1857281 (0.0008) [2023-12-27 18:31:31,203][20347] Updated weights for policy 0, policy_version 1858900 (0.0011) [2023-12-27 18:31:31,248][20381] Updated weights for policy 1, policy_version 1857291 (0.0008) [2023-12-27 18:31:31,299][20381] Updated weights for policy 1, policy_version 1857301 (0.0008) [2023-12-27 18:31:31,965][20347] Updated weights for policy 0, policy_version 1858910 (0.0009) [2023-12-27 18:31:32,016][20347] Updated weights for policy 0, policy_version 1858920 (0.0007) [2023-12-27 18:31:32,067][20347] Updated weights for policy 0, policy_version 1858930 (0.0006) [2023-12-27 18:31:32,081][20381] Updated weights for policy 1, policy_version 1857311 (0.0010) [2023-12-27 18:31:32,131][20381] Updated weights for policy 1, policy_version 1857321 (0.0011) [2023-12-27 18:31:32,186][20381] Updated weights for policy 1, policy_version 1857331 (0.0011) [2023-12-27 18:31:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 951500800. Throughput: 0: 10154.4, 1: 10012.7. Samples: 951507628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:32,363][19170] Avg episode reward: [(0, '990.270'), (1, '742.800')] [2023-12-27 18:31:32,772][20347] Updated weights for policy 0, policy_version 1858940 (0.0006) [2023-12-27 18:31:32,826][20347] Updated weights for policy 0, policy_version 1858950 (0.0007) [2023-12-27 18:31:32,877][20347] Updated weights for policy 0, policy_version 1858960 (0.0008) [2023-12-27 18:31:32,881][20381] Updated weights for policy 1, policy_version 1857341 (0.0010) [2023-12-27 18:31:32,935][20381] Updated weights for policy 1, policy_version 1857351 (0.0010) [2023-12-27 18:31:32,988][20381] Updated weights for policy 1, policy_version 1857361 (0.0010) [2023-12-27 18:31:33,605][20347] Updated weights for policy 0, policy_version 1858970 (0.0007) [2023-12-27 18:31:33,614][20381] Updated weights for policy 1, policy_version 1857371 (0.0009) [2023-12-27 18:31:33,650][20347] Updated weights for policy 0, policy_version 1858980 (0.0007) [2023-12-27 18:31:33,662][20381] Updated weights for policy 1, policy_version 1857381 (0.0010) [2023-12-27 18:31:33,700][20347] Updated weights for policy 0, policy_version 1858990 (0.0005) [2023-12-27 18:31:33,720][20381] Updated weights for policy 1, policy_version 1857391 (0.0010) [2023-12-27 18:31:33,755][20347] Updated weights for policy 0, policy_version 1859000 (0.0006) [2023-12-27 18:31:34,387][20381] Updated weights for policy 1, policy_version 1857401 (0.0010) [2023-12-27 18:31:34,445][20381] Updated weights for policy 1, policy_version 1857411 (0.0011) [2023-12-27 18:31:34,457][20347] Updated weights for policy 0, policy_version 1859010 (0.0011) [2023-12-27 18:31:34,498][20381] Updated weights for policy 1, policy_version 1857421 (0.0011) [2023-12-27 18:31:34,513][20347] Updated weights for policy 0, policy_version 1859020 (0.0010) [2023-12-27 18:31:34,554][20381] Updated weights for policy 1, policy_version 1857431 (0.0011) [2023-12-27 18:31:34,566][20347] Updated weights for policy 0, policy_version 1859030 (0.0011) [2023-12-27 18:31:35,180][20347] Updated weights for policy 0, policy_version 1859040 (0.0006) [2023-12-27 18:31:35,234][20347] Updated weights for policy 0, policy_version 1859050 (0.0006) [2023-12-27 18:31:35,265][20381] Updated weights for policy 1, policy_version 1857441 (0.0007) [2023-12-27 18:31:35,289][20347] Updated weights for policy 0, policy_version 1859060 (0.0006) [2023-12-27 18:31:35,311][20381] Updated weights for policy 1, policy_version 1857451 (0.0006) [2023-12-27 18:31:35,361][20381] Updated weights for policy 1, policy_version 1857461 (0.0009) [2023-12-27 18:31:35,867][20347] Updated weights for policy 0, policy_version 1859070 (0.0006) [2023-12-27 18:31:35,921][20347] Updated weights for policy 0, policy_version 1859080 (0.0006) [2023-12-27 18:31:35,976][20347] Updated weights for policy 0, policy_version 1859090 (0.0005) [2023-12-27 18:31:36,074][20381] Updated weights for policy 1, policy_version 1857471 (0.0008) [2023-12-27 18:31:36,134][20381] Updated weights for policy 1, policy_version 1857481 (0.0007) [2023-12-27 18:31:36,196][20381] Updated weights for policy 1, policy_version 1857491 (0.0007) [2023-12-27 18:31:36,675][20347] Updated weights for policy 0, policy_version 1859100 (0.0007) [2023-12-27 18:31:36,734][20347] Updated weights for policy 0, policy_version 1859110 (0.0007) [2023-12-27 18:31:36,789][20347] Updated weights for policy 0, policy_version 1859120 (0.0007) [2023-12-27 18:31:36,873][20381] Updated weights for policy 1, policy_version 1857501 (0.0008) [2023-12-27 18:31:36,922][20381] Updated weights for policy 1, policy_version 1857511 (0.0008) [2023-12-27 18:31:36,975][20381] Updated weights for policy 1, policy_version 1857521 (0.0006) [2023-12-27 18:31:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20070.3, 300 sec: 20466.1). Total num frames: 951599104. Throughput: 0: 10145.2, 1: 10023.7. Samples: 951569016. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:37,364][19170] Avg episode reward: [(0, '990.270'), (1, '731.230')] [2023-12-27 18:31:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001859128_476020736.pth... [2023-12-27 18:31:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001857528_475578368.pth... [2023-12-27 18:31:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001857944_475717632.pth [2023-12-27 18:31:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001856344_475275264.pth [2023-12-27 18:31:37,521][20347] Updated weights for policy 0, policy_version 1859130 (0.0008) [2023-12-27 18:31:37,574][20347] Updated weights for policy 0, policy_version 1859140 (0.0008) [2023-12-27 18:31:37,598][20381] Updated weights for policy 1, policy_version 1857531 (0.0007) [2023-12-27 18:31:37,636][20347] Updated weights for policy 0, policy_version 1859150 (0.0008) [2023-12-27 18:31:37,649][20381] Updated weights for policy 1, policy_version 1857541 (0.0005) [2023-12-27 18:31:37,690][20347] Updated weights for policy 0, policy_version 1859160 (0.0007) [2023-12-27 18:31:37,701][20381] Updated weights for policy 1, policy_version 1857551 (0.0006) [2023-12-27 18:31:38,327][20381] Updated weights for policy 1, policy_version 1857561 (0.0008) [2023-12-27 18:31:38,390][20381] Updated weights for policy 1, policy_version 1857571 (0.0007) [2023-12-27 18:31:38,396][20347] Updated weights for policy 0, policy_version 1859170 (0.0007) [2023-12-27 18:31:38,445][20381] Updated weights for policy 1, policy_version 1857581 (0.0007) [2023-12-27 18:31:38,453][20347] Updated weights for policy 0, policy_version 1859180 (0.0008) [2023-12-27 18:31:38,500][20381] Updated weights for policy 1, policy_version 1857591 (0.0009) [2023-12-27 18:31:38,517][20347] Updated weights for policy 0, policy_version 1859190 (0.0007) [2023-12-27 18:31:39,215][20381] Updated weights for policy 1, policy_version 1857601 (0.0006) [2023-12-27 18:31:39,228][20347] Updated weights for policy 0, policy_version 1859200 (0.0007) [2023-12-27 18:31:39,295][20347] Updated weights for policy 0, policy_version 1859210 (0.0008) [2023-12-27 18:31:39,297][20381] Updated weights for policy 1, policy_version 1857611 (0.0008) [2023-12-27 18:31:39,357][20381] Updated weights for policy 1, policy_version 1857621 (0.0008) [2023-12-27 18:31:39,358][20347] Updated weights for policy 0, policy_version 1859220 (0.0008) [2023-12-27 18:31:40,102][20347] Updated weights for policy 0, policy_version 1859230 (0.0010) [2023-12-27 18:31:40,104][20381] Updated weights for policy 1, policy_version 1857631 (0.0008) [2023-12-27 18:31:40,155][20381] Updated weights for policy 1, policy_version 1857641 (0.0007) [2023-12-27 18:31:40,164][20347] Updated weights for policy 0, policy_version 1859240 (0.0011) [2023-12-27 18:31:40,211][20381] Updated weights for policy 1, policy_version 1857651 (0.0005) [2023-12-27 18:31:40,225][20347] Updated weights for policy 0, policy_version 1859250 (0.0011) [2023-12-27 18:31:40,894][20381] Updated weights for policy 1, policy_version 1857661 (0.0007) [2023-12-27 18:31:40,911][20347] Updated weights for policy 0, policy_version 1859260 (0.0008) [2023-12-27 18:31:40,950][20381] Updated weights for policy 1, policy_version 1857671 (0.0005) [2023-12-27 18:31:40,969][20347] Updated weights for policy 0, policy_version 1859270 (0.0005) [2023-12-27 18:31:41,006][20381] Updated weights for policy 1, policy_version 1857681 (0.0006) [2023-12-27 18:31:41,030][20347] Updated weights for policy 0, policy_version 1859280 (0.0006) [2023-12-27 18:31:41,666][20347] Updated weights for policy 0, policy_version 1859290 (0.0009) [2023-12-27 18:31:41,723][20347] Updated weights for policy 0, policy_version 1859300 (0.0007) [2023-12-27 18:31:41,769][20381] Updated weights for policy 1, policy_version 1857691 (0.0008) [2023-12-27 18:31:41,794][20347] Updated weights for policy 0, policy_version 1859310 (0.0008) [2023-12-27 18:31:41,840][20381] Updated weights for policy 1, policy_version 1857701 (0.0007) [2023-12-27 18:31:41,854][20347] Updated weights for policy 0, policy_version 1859320 (0.0006) [2023-12-27 18:31:41,900][20381] Updated weights for policy 1, policy_version 1857711 (0.0006) [2023-12-27 18:31:42,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 951697408. Throughput: 0: 10122.1, 1: 10033.1. Samples: 951689960. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '829.900')] [2023-12-27 18:31:42,583][20347] Updated weights for policy 0, policy_version 1859330 (0.0008) [2023-12-27 18:31:42,587][20381] Updated weights for policy 1, policy_version 1857721 (0.0007) [2023-12-27 18:31:42,647][20381] Updated weights for policy 1, policy_version 1857731 (0.0006) [2023-12-27 18:31:42,667][20347] Updated weights for policy 0, policy_version 1859340 (0.0008) [2023-12-27 18:31:42,704][20381] Updated weights for policy 1, policy_version 1857741 (0.0007) [2023-12-27 18:31:42,727][20347] Updated weights for policy 0, policy_version 1859350 (0.0008) [2023-12-27 18:31:42,764][20381] Updated weights for policy 1, policy_version 1857751 (0.0006) [2023-12-27 18:31:43,434][20347] Updated weights for policy 0, policy_version 1859360 (0.0007) [2023-12-27 18:31:43,457][20381] Updated weights for policy 1, policy_version 1857761 (0.0011) [2023-12-27 18:31:43,488][20347] Updated weights for policy 0, policy_version 1859370 (0.0005) [2023-12-27 18:31:43,505][20381] Updated weights for policy 1, policy_version 1857771 (0.0010) [2023-12-27 18:31:43,540][20347] Updated weights for policy 0, policy_version 1859380 (0.0005) [2023-12-27 18:31:43,550][20381] Updated weights for policy 1, policy_version 1857781 (0.0010) [2023-12-27 18:31:44,171][20381] Updated weights for policy 1, policy_version 1857791 (0.0011) [2023-12-27 18:31:44,222][20381] Updated weights for policy 1, policy_version 1857801 (0.0010) [2023-12-27 18:31:44,255][20347] Updated weights for policy 0, policy_version 1859390 (0.0006) [2023-12-27 18:31:44,279][20381] Updated weights for policy 1, policy_version 1857811 (0.0009) [2023-12-27 18:31:44,310][20347] Updated weights for policy 0, policy_version 1859400 (0.0007) [2023-12-27 18:31:44,360][20347] Updated weights for policy 0, policy_version 1859410 (0.0007) [2023-12-27 18:31:44,961][20381] Updated weights for policy 1, policy_version 1857821 (0.0007) [2023-12-27 18:31:45,019][20381] Updated weights for policy 1, policy_version 1857831 (0.0007) [2023-12-27 18:31:45,086][20381] Updated weights for policy 1, policy_version 1857841 (0.0007) [2023-12-27 18:31:45,111][20347] Updated weights for policy 0, policy_version 1859420 (0.0007) [2023-12-27 18:31:45,174][20347] Updated weights for policy 0, policy_version 1859430 (0.0008) [2023-12-27 18:31:45,234][20347] Updated weights for policy 0, policy_version 1859440 (0.0009) [2023-12-27 18:31:45,734][20381] Updated weights for policy 1, policy_version 1857851 (0.0007) [2023-12-27 18:31:45,790][20381] Updated weights for policy 1, policy_version 1857861 (0.0007) [2023-12-27 18:31:45,843][20347] Updated weights for policy 0, policy_version 1859450 (0.0008) [2023-12-27 18:31:45,847][20381] Updated weights for policy 1, policy_version 1857871 (0.0006) [2023-12-27 18:31:45,893][20347] Updated weights for policy 0, policy_version 1859460 (0.0006) [2023-12-27 18:31:45,938][20347] Updated weights for policy 0, policy_version 1859470 (0.0007) [2023-12-27 18:31:45,979][20347] Updated weights for policy 0, policy_version 1859480 (0.0008) [2023-12-27 18:31:46,489][20381] Updated weights for policy 1, policy_version 1857881 (0.0005) [2023-12-27 18:31:46,548][20381] Updated weights for policy 1, policy_version 1857891 (0.0006) [2023-12-27 18:31:46,611][20381] Updated weights for policy 1, policy_version 1857901 (0.0007) [2023-12-27 18:31:46,658][20347] Updated weights for policy 0, policy_version 1859490 (0.0007) [2023-12-27 18:31:46,664][20381] Updated weights for policy 1, policy_version 1857911 (0.0007) [2023-12-27 18:31:46,708][20347] Updated weights for policy 0, policy_version 1859500 (0.0007) [2023-12-27 18:31:46,752][20347] Updated weights for policy 0, policy_version 1859510 (0.0008) [2023-12-27 18:31:47,276][20381] Updated weights for policy 1, policy_version 1857921 (0.0008) [2023-12-27 18:31:47,332][20381] Updated weights for policy 1, policy_version 1857931 (0.0007) [2023-12-27 18:31:47,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20070.5, 300 sec: 20410.6). Total num frames: 951795712. Throughput: 0: 10140.7, 1: 10110.0. Samples: 951812768. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '840.290')] [2023-12-27 18:31:47,367][20347] Updated weights for policy 0, policy_version 1859520 (0.0008) [2023-12-27 18:31:47,384][20381] Updated weights for policy 1, policy_version 1857941 (0.0006) [2023-12-27 18:31:47,430][20347] Updated weights for policy 0, policy_version 1859530 (0.0008) [2023-12-27 18:31:47,506][20347] Updated weights for policy 0, policy_version 1859540 (0.0007) [2023-12-27 18:31:48,081][20381] Updated weights for policy 1, policy_version 1857951 (0.0005) [2023-12-27 18:31:48,135][20381] Updated weights for policy 1, policy_version 1857961 (0.0007) [2023-12-27 18:31:48,145][20347] Updated weights for policy 0, policy_version 1859550 (0.0007) [2023-12-27 18:31:48,183][20381] Updated weights for policy 1, policy_version 1857971 (0.0006) [2023-12-27 18:31:48,199][20347] Updated weights for policy 0, policy_version 1859560 (0.0006) [2023-12-27 18:31:48,246][20347] Updated weights for policy 0, policy_version 1859570 (0.0007) [2023-12-27 18:31:48,856][20347] Updated weights for policy 0, policy_version 1859580 (0.0007) [2023-12-27 18:31:48,890][20381] Updated weights for policy 1, policy_version 1857981 (0.0006) [2023-12-27 18:31:48,907][20347] Updated weights for policy 0, policy_version 1859590 (0.0006) [2023-12-27 18:31:48,942][20381] Updated weights for policy 1, policy_version 1857991 (0.0008) [2023-12-27 18:31:48,957][20347] Updated weights for policy 0, policy_version 1859600 (0.0005) [2023-12-27 18:31:48,988][20381] Updated weights for policy 1, policy_version 1858001 (0.0006) [2023-12-27 18:31:49,618][20347] Updated weights for policy 0, policy_version 1859610 (0.0007) [2023-12-27 18:31:49,673][20347] Updated weights for policy 0, policy_version 1859620 (0.0007) [2023-12-27 18:31:49,718][20381] Updated weights for policy 1, policy_version 1858011 (0.0007) [2023-12-27 18:31:49,725][20347] Updated weights for policy 0, policy_version 1859630 (0.0007) [2023-12-27 18:31:49,775][20381] Updated weights for policy 1, policy_version 1858021 (0.0010) [2023-12-27 18:31:49,777][20347] Updated weights for policy 0, policy_version 1859640 (0.0006) [2023-12-27 18:31:49,827][20381] Updated weights for policy 1, policy_version 1858031 (0.0010) [2023-12-27 18:31:50,515][20347] Updated weights for policy 0, policy_version 1859650 (0.0007) [2023-12-27 18:31:50,541][20381] Updated weights for policy 1, policy_version 1858041 (0.0007) [2023-12-27 18:31:50,591][20347] Updated weights for policy 0, policy_version 1859660 (0.0006) [2023-12-27 18:31:50,606][20381] Updated weights for policy 1, policy_version 1858051 (0.0009) [2023-12-27 18:31:50,648][20347] Updated weights for policy 0, policy_version 1859670 (0.0006) [2023-12-27 18:31:50,659][20381] Updated weights for policy 1, policy_version 1858061 (0.0008) [2023-12-27 18:31:50,718][20381] Updated weights for policy 1, policy_version 1858071 (0.0011) [2023-12-27 18:31:51,384][20381] Updated weights for policy 1, policy_version 1858081 (0.0010) [2023-12-27 18:31:51,438][20347] Updated weights for policy 0, policy_version 1859680 (0.0008) [2023-12-27 18:31:51,442][20381] Updated weights for policy 1, policy_version 1858091 (0.0009) [2023-12-27 18:31:51,492][20381] Updated weights for policy 1, policy_version 1858101 (0.0009) [2023-12-27 18:31:51,496][20347] Updated weights for policy 0, policy_version 1859690 (0.0009) [2023-12-27 18:31:51,549][20347] Updated weights for policy 0, policy_version 1859700 (0.0006) [2023-12-27 18:31:52,185][20347] Updated weights for policy 0, policy_version 1859710 (0.0009) [2023-12-27 18:31:52,238][20381] Updated weights for policy 1, policy_version 1858111 (0.0010) [2023-12-27 18:31:52,238][20347] Updated weights for policy 0, policy_version 1859720 (0.0010) [2023-12-27 18:31:52,298][20347] Updated weights for policy 0, policy_version 1859730 (0.0011) [2023-12-27 18:31:52,299][20381] Updated weights for policy 1, policy_version 1858121 (0.0011) [2023-12-27 18:31:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 951902208. Throughput: 0: 10183.0, 1: 10089.2. Samples: 951874008. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:52,363][20381] Updated weights for policy 1, policy_version 1858131 (0.0010) [2023-12-27 18:31:52,363][19170] Avg episode reward: [(0, '990.300'), (1, '869.750')] [2023-12-27 18:31:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001859736_476176384.pth... [2023-12-27 18:31:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001858520_475865088.pth [2023-12-27 18:31:52,393][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001858136_475734016.pth... [2023-12-27 18:31:52,397][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001856920_475422720.pth [2023-12-27 18:31:53,004][20347] Updated weights for policy 0, policy_version 1859740 (0.0009) [2023-12-27 18:31:53,065][20347] Updated weights for policy 0, policy_version 1859750 (0.0006) [2023-12-27 18:31:53,087][20381] Updated weights for policy 1, policy_version 1858141 (0.0007) [2023-12-27 18:31:53,115][20347] Updated weights for policy 0, policy_version 1859760 (0.0005) [2023-12-27 18:31:53,133][20381] Updated weights for policy 1, policy_version 1858151 (0.0005) [2023-12-27 18:31:53,190][20381] Updated weights for policy 1, policy_version 1858161 (0.0005) [2023-12-27 18:31:53,686][20347] Updated weights for policy 0, policy_version 1859770 (0.0006) [2023-12-27 18:31:53,732][20347] Updated weights for policy 0, policy_version 1859780 (0.0007) [2023-12-27 18:31:53,781][20347] Updated weights for policy 0, policy_version 1859790 (0.0008) [2023-12-27 18:31:53,798][20381] Updated weights for policy 1, policy_version 1858171 (0.0007) [2023-12-27 18:31:53,830][20347] Updated weights for policy 0, policy_version 1859800 (0.0006) [2023-12-27 18:31:53,845][20381] Updated weights for policy 1, policy_version 1858181 (0.0007) [2023-12-27 18:31:53,899][20381] Updated weights for policy 1, policy_version 1858191 (0.0007) [2023-12-27 18:31:54,442][20347] Updated weights for policy 0, policy_version 1859810 (0.0005) [2023-12-27 18:31:54,503][20347] Updated weights for policy 0, policy_version 1859820 (0.0008) [2023-12-27 18:31:54,555][20347] Updated weights for policy 0, policy_version 1859830 (0.0008) [2023-12-27 18:31:54,638][20381] Updated weights for policy 1, policy_version 1858201 (0.0008) [2023-12-27 18:31:54,690][20381] Updated weights for policy 1, policy_version 1858211 (0.0008) [2023-12-27 18:31:54,746][20381] Updated weights for policy 1, policy_version 1858221 (0.0008) [2023-12-27 18:31:54,806][20381] Updated weights for policy 1, policy_version 1858231 (0.0006) [2023-12-27 18:31:55,227][20347] Updated weights for policy 0, policy_version 1859840 (0.0008) [2023-12-27 18:31:55,278][20347] Updated weights for policy 0, policy_version 1859850 (0.0007) [2023-12-27 18:31:55,337][20347] Updated weights for policy 0, policy_version 1859860 (0.0008) [2023-12-27 18:31:55,433][20381] Updated weights for policy 1, policy_version 1858241 (0.0006) [2023-12-27 18:31:55,487][20381] Updated weights for policy 1, policy_version 1858251 (0.0006) [2023-12-27 18:31:55,541][20381] Updated weights for policy 1, policy_version 1858261 (0.0006) [2023-12-27 18:31:55,969][20347] Updated weights for policy 0, policy_version 1859870 (0.0007) [2023-12-27 18:31:56,025][20347] Updated weights for policy 0, policy_version 1859880 (0.0006) [2023-12-27 18:31:56,075][20347] Updated weights for policy 0, policy_version 1859890 (0.0008) [2023-12-27 18:31:56,146][20381] Updated weights for policy 1, policy_version 1858271 (0.0007) [2023-12-27 18:31:56,210][20381] Updated weights for policy 1, policy_version 1858281 (0.0008) [2023-12-27 18:31:56,269][20381] Updated weights for policy 1, policy_version 1858291 (0.0008) [2023-12-27 18:31:56,783][20347] Updated weights for policy 0, policy_version 1859900 (0.0007) [2023-12-27 18:31:56,834][20347] Updated weights for policy 0, policy_version 1859910 (0.0008) [2023-12-27 18:31:56,886][20347] Updated weights for policy 0, policy_version 1859920 (0.0007) [2023-12-27 18:31:56,931][20381] Updated weights for policy 1, policy_version 1858301 (0.0008) [2023-12-27 18:31:56,988][20381] Updated weights for policy 1, policy_version 1858311 (0.0008) [2023-12-27 18:31:57,043][20381] Updated weights for policy 1, policy_version 1858321 (0.0007) [2023-12-27 18:31:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 952008704. Throughput: 0: 10232.3, 1: 10181.0. Samples: 951998048. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:31:57,363][19170] Avg episode reward: [(0, '990.300'), (1, '821.790')] [2023-12-27 18:31:57,491][20347] Updated weights for policy 0, policy_version 1859930 (0.0008) [2023-12-27 18:31:57,542][20347] Updated weights for policy 0, policy_version 1859940 (0.0006) [2023-12-27 18:31:57,602][20347] Updated weights for policy 0, policy_version 1859950 (0.0006) [2023-12-27 18:31:57,660][20347] Updated weights for policy 0, policy_version 1859960 (0.0007) [2023-12-27 18:31:57,790][20381] Updated weights for policy 1, policy_version 1858331 (0.0008) [2023-12-27 18:31:57,847][20381] Updated weights for policy 1, policy_version 1858341 (0.0010) [2023-12-27 18:31:57,906][20381] Updated weights for policy 1, policy_version 1858352 (0.0007) [2023-12-27 18:31:58,362][20347] Updated weights for policy 0, policy_version 1859970 (0.0008) [2023-12-27 18:31:58,423][20347] Updated weights for policy 0, policy_version 1859980 (0.0009) [2023-12-27 18:31:58,481][20347] Updated weights for policy 0, policy_version 1859990 (0.0008) [2023-12-27 18:31:58,550][20381] Updated weights for policy 1, policy_version 1858362 (0.0006) [2023-12-27 18:31:58,610][20381] Updated weights for policy 1, policy_version 1858372 (0.0008) [2023-12-27 18:31:58,668][20381] Updated weights for policy 1, policy_version 1858382 (0.0007) [2023-12-27 18:31:58,734][20381] Updated weights for policy 1, policy_version 1858392 (0.0008) [2023-12-27 18:31:59,262][20347] Updated weights for policy 0, policy_version 1860000 (0.0008) [2023-12-27 18:31:59,318][20347] Updated weights for policy 0, policy_version 1860010 (0.0008) [2023-12-27 18:31:59,384][20347] Updated weights for policy 0, policy_version 1860020 (0.0008) [2023-12-27 18:31:59,474][20381] Updated weights for policy 1, policy_version 1858402 (0.0008) [2023-12-27 18:31:59,533][20381] Updated weights for policy 1, policy_version 1858412 (0.0006) [2023-12-27 18:31:59,595][20381] Updated weights for policy 1, policy_version 1858422 (0.0006) [2023-12-27 18:32:00,085][20347] Updated weights for policy 0, policy_version 1860030 (0.0007) [2023-12-27 18:32:00,142][20347] Updated weights for policy 0, policy_version 1860040 (0.0005) [2023-12-27 18:32:00,205][20347] Updated weights for policy 0, policy_version 1860050 (0.0006) [2023-12-27 18:32:00,263][20381] Updated weights for policy 1, policy_version 1858432 (0.0009) [2023-12-27 18:32:00,325][20381] Updated weights for policy 1, policy_version 1858442 (0.0008) [2023-12-27 18:32:00,377][20381] Updated weights for policy 1, policy_version 1858452 (0.0010) [2023-12-27 18:32:00,857][20347] Updated weights for policy 0, policy_version 1860060 (0.0007) [2023-12-27 18:32:00,899][20347] Updated weights for policy 0, policy_version 1860070 (0.0007) [2023-12-27 18:32:00,945][20347] Updated weights for policy 0, policy_version 1860080 (0.0007) [2023-12-27 18:32:01,061][20381] Updated weights for policy 1, policy_version 1858462 (0.0010) [2023-12-27 18:32:01,122][20381] Updated weights for policy 1, policy_version 1858472 (0.0011) [2023-12-27 18:32:01,182][20381] Updated weights for policy 1, policy_version 1858482 (0.0006) [2023-12-27 18:32:01,683][20347] Updated weights for policy 0, policy_version 1860090 (0.0007) [2023-12-27 18:32:01,746][20347] Updated weights for policy 0, policy_version 1860100 (0.0008) [2023-12-27 18:32:01,805][20347] Updated weights for policy 0, policy_version 1860110 (0.0008) [2023-12-27 18:32:01,857][20381] Updated weights for policy 1, policy_version 1858492 (0.0006) [2023-12-27 18:32:01,863][20347] Updated weights for policy 0, policy_version 1860120 (0.0006) [2023-12-27 18:32:01,907][20381] Updated weights for policy 1, policy_version 1858502 (0.0005) [2023-12-27 18:32:01,966][20381] Updated weights for policy 1, policy_version 1858512 (0.0007) [2023-12-27 18:32:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 952107008. Throughput: 0: 10225.1, 1: 10205.7. Samples: 952117700. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '831.010')] [2023-12-27 18:32:02,625][20381] Updated weights for policy 1, policy_version 1858522 (0.0007) [2023-12-27 18:32:02,630][20347] Updated weights for policy 0, policy_version 1860130 (0.0007) [2023-12-27 18:32:02,680][20381] Updated weights for policy 1, policy_version 1858532 (0.0006) [2023-12-27 18:32:02,692][20347] Updated weights for policy 0, policy_version 1860140 (0.0008) [2023-12-27 18:32:02,742][20381] Updated weights for policy 1, policy_version 1858542 (0.0006) [2023-12-27 18:32:02,748][20347] Updated weights for policy 0, policy_version 1860150 (0.0007) [2023-12-27 18:32:02,803][20381] Updated weights for policy 1, policy_version 1858552 (0.0006) [2023-12-27 18:32:03,398][20381] Updated weights for policy 1, policy_version 1858562 (0.0007) [2023-12-27 18:32:03,442][20347] Updated weights for policy 0, policy_version 1860160 (0.0009) [2023-12-27 18:32:03,444][20381] Updated weights for policy 1, policy_version 1858572 (0.0007) [2023-12-27 18:32:03,485][20381] Updated weights for policy 1, policy_version 1858582 (0.0005) [2023-12-27 18:32:03,497][20347] Updated weights for policy 0, policy_version 1860170 (0.0010) [2023-12-27 18:32:03,550][20347] Updated weights for policy 0, policy_version 1860180 (0.0010) [2023-12-27 18:32:04,138][20381] Updated weights for policy 1, policy_version 1858592 (0.0008) [2023-12-27 18:32:04,200][20381] Updated weights for policy 1, policy_version 1858602 (0.0007) [2023-12-27 18:32:04,247][20347] Updated weights for policy 0, policy_version 1860190 (0.0009) [2023-12-27 18:32:04,257][20381] Updated weights for policy 1, policy_version 1858612 (0.0006) [2023-12-27 18:32:04,306][20347] Updated weights for policy 0, policy_version 1860200 (0.0007) [2023-12-27 18:32:04,362][20347] Updated weights for policy 0, policy_version 1860210 (0.0006) [2023-12-27 18:32:04,867][20381] Updated weights for policy 1, policy_version 1858622 (0.0007) [2023-12-27 18:32:04,923][20381] Updated weights for policy 1, policy_version 1858632 (0.0010) [2023-12-27 18:32:04,982][20381] Updated weights for policy 1, policy_version 1858642 (0.0010) [2023-12-27 18:32:05,089][20347] Updated weights for policy 0, policy_version 1860220 (0.0009) [2023-12-27 18:32:05,143][20347] Updated weights for policy 0, policy_version 1860230 (0.0008) [2023-12-27 18:32:05,188][20347] Updated weights for policy 0, policy_version 1860240 (0.0007) [2023-12-27 18:32:05,648][20381] Updated weights for policy 1, policy_version 1858652 (0.0008) [2023-12-27 18:32:05,704][20381] Updated weights for policy 1, policy_version 1858662 (0.0008) [2023-12-27 18:32:05,752][20381] Updated weights for policy 1, policy_version 1858672 (0.0007) [2023-12-27 18:32:05,893][20347] Updated weights for policy 0, policy_version 1860250 (0.0007) [2023-12-27 18:32:05,954][20347] Updated weights for policy 0, policy_version 1860260 (0.0007) [2023-12-27 18:32:06,010][20347] Updated weights for policy 0, policy_version 1860270 (0.0008) [2023-12-27 18:32:06,069][20347] Updated weights for policy 0, policy_version 1860280 (0.0008) [2023-12-27 18:32:06,481][20381] Updated weights for policy 1, policy_version 1858682 (0.0008) [2023-12-27 18:32:06,541][20381] Updated weights for policy 1, policy_version 1858692 (0.0008) [2023-12-27 18:32:06,594][20381] Updated weights for policy 1, policy_version 1858702 (0.0006) [2023-12-27 18:32:06,654][20381] Updated weights for policy 1, policy_version 1858712 (0.0007) [2023-12-27 18:32:06,796][20347] Updated weights for policy 0, policy_version 1860290 (0.0007) [2023-12-27 18:32:06,849][20347] Updated weights for policy 0, policy_version 1860300 (0.0008) [2023-12-27 18:32:06,894][20347] Updated weights for policy 0, policy_version 1860310 (0.0008) [2023-12-27 18:32:07,218][20381] Updated weights for policy 1, policy_version 1858722 (0.0009) [2023-12-27 18:32:07,268][20381] Updated weights for policy 1, policy_version 1858732 (0.0009) [2023-12-27 18:32:07,321][20381] Updated weights for policy 1, policy_version 1858742 (0.0009) [2023-12-27 18:32:07,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 952213504. Throughput: 0: 10194.8, 1: 10263.5. Samples: 952179964. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '838.070')] [2023-12-27 18:32:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001860312_476323840.pth... [2023-12-27 18:32:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001858744_475889664.pth... [2023-12-27 18:32:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001859128_476020736.pth [2023-12-27 18:32:07,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001857528_475578368.pth [2023-12-27 18:32:07,609][20347] Updated weights for policy 0, policy_version 1860320 (0.0007) [2023-12-27 18:32:07,661][20347] Updated weights for policy 0, policy_version 1860330 (0.0007) [2023-12-27 18:32:07,710][20347] Updated weights for policy 0, policy_version 1860340 (0.0007) [2023-12-27 18:32:08,011][20381] Updated weights for policy 1, policy_version 1858752 (0.0006) [2023-12-27 18:32:08,066][20381] Updated weights for policy 1, policy_version 1858762 (0.0007) [2023-12-27 18:32:08,125][20381] Updated weights for policy 1, policy_version 1858772 (0.0008) [2023-12-27 18:32:08,400][20347] Updated weights for policy 0, policy_version 1860350 (0.0008) [2023-12-27 18:32:08,467][20347] Updated weights for policy 0, policy_version 1860360 (0.0011) [2023-12-27 18:32:08,523][20347] Updated weights for policy 0, policy_version 1860370 (0.0007) [2023-12-27 18:32:08,831][20381] Updated weights for policy 1, policy_version 1858782 (0.0008) [2023-12-27 18:32:08,891][20381] Updated weights for policy 1, policy_version 1858792 (0.0008) [2023-12-27 18:32:08,948][20381] Updated weights for policy 1, policy_version 1858802 (0.0008) [2023-12-27 18:32:09,229][20347] Updated weights for policy 0, policy_version 1860380 (0.0007) [2023-12-27 18:32:09,290][20347] Updated weights for policy 0, policy_version 1860390 (0.0007) [2023-12-27 18:32:09,354][20347] Updated weights for policy 0, policy_version 1860400 (0.0009) [2023-12-27 18:32:09,705][20381] Updated weights for policy 1, policy_version 1858812 (0.0008) [2023-12-27 18:32:09,773][20381] Updated weights for policy 1, policy_version 1858822 (0.0007) [2023-12-27 18:32:09,840][20381] Updated weights for policy 1, policy_version 1858832 (0.0008) [2023-12-27 18:32:10,099][20347] Updated weights for policy 0, policy_version 1860410 (0.0008) [2023-12-27 18:32:10,158][20347] Updated weights for policy 0, policy_version 1860420 (0.0007) [2023-12-27 18:32:10,213][20347] Updated weights for policy 0, policy_version 1860430 (0.0006) [2023-12-27 18:32:10,273][20347] Updated weights for policy 0, policy_version 1860440 (0.0006) [2023-12-27 18:32:10,518][20381] Updated weights for policy 1, policy_version 1858842 (0.0008) [2023-12-27 18:32:10,571][20381] Updated weights for policy 1, policy_version 1858852 (0.0011) [2023-12-27 18:32:10,631][20381] Updated weights for policy 1, policy_version 1858862 (0.0010) [2023-12-27 18:32:10,684][20381] Updated weights for policy 1, policy_version 1858872 (0.0010) [2023-12-27 18:32:10,945][20347] Updated weights for policy 0, policy_version 1860450 (0.0007) [2023-12-27 18:32:10,993][20347] Updated weights for policy 0, policy_version 1860460 (0.0007) [2023-12-27 18:32:11,051][20347] Updated weights for policy 0, policy_version 1860470 (0.0007) [2023-12-27 18:32:11,367][20381] Updated weights for policy 1, policy_version 1858882 (0.0008) [2023-12-27 18:32:11,428][20381] Updated weights for policy 1, policy_version 1858892 (0.0009) [2023-12-27 18:32:11,482][20381] Updated weights for policy 1, policy_version 1858902 (0.0011) [2023-12-27 18:32:11,846][20347] Updated weights for policy 0, policy_version 1860480 (0.0007) [2023-12-27 18:32:11,912][20347] Updated weights for policy 0, policy_version 1860490 (0.0007) [2023-12-27 18:32:11,971][20347] Updated weights for policy 0, policy_version 1860500 (0.0007) [2023-12-27 18:32:12,196][20381] Updated weights for policy 1, policy_version 1858912 (0.0011) [2023-12-27 18:32:12,258][20381] Updated weights for policy 1, policy_version 1858922 (0.0008) [2023-12-27 18:32:12,318][20381] Updated weights for policy 1, policy_version 1858932 (0.0009) [2023-12-27 18:32:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 952311808. Throughput: 0: 10117.3, 1: 10236.9. Samples: 952300036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '848.490')] [2023-12-27 18:32:12,672][20347] Updated weights for policy 0, policy_version 1860510 (0.0006) [2023-12-27 18:32:12,733][20347] Updated weights for policy 0, policy_version 1860520 (0.0007) [2023-12-27 18:32:12,792][20347] Updated weights for policy 0, policy_version 1860530 (0.0007) [2023-12-27 18:32:12,965][20381] Updated weights for policy 1, policy_version 1858942 (0.0008) [2023-12-27 18:32:13,019][20381] Updated weights for policy 1, policy_version 1858952 (0.0005) [2023-12-27 18:32:13,078][20381] Updated weights for policy 1, policy_version 1858962 (0.0005) [2023-12-27 18:32:13,475][20347] Updated weights for policy 0, policy_version 1860540 (0.0007) [2023-12-27 18:32:13,534][20347] Updated weights for policy 0, policy_version 1860550 (0.0007) [2023-12-27 18:32:13,597][20347] Updated weights for policy 0, policy_version 1860560 (0.0008) [2023-12-27 18:32:13,697][20381] Updated weights for policy 1, policy_version 1858972 (0.0008) [2023-12-27 18:32:13,751][20381] Updated weights for policy 1, policy_version 1858982 (0.0008) [2023-12-27 18:32:13,806][20381] Updated weights for policy 1, policy_version 1858992 (0.0008) [2023-12-27 18:32:14,276][20347] Updated weights for policy 0, policy_version 1860570 (0.0007) [2023-12-27 18:32:14,336][20347] Updated weights for policy 0, policy_version 1860580 (0.0008) [2023-12-27 18:32:14,391][20347] Updated weights for policy 0, policy_version 1860590 (0.0008) [2023-12-27 18:32:14,449][20347] Updated weights for policy 0, policy_version 1860600 (0.0007) [2023-12-27 18:32:14,539][20381] Updated weights for policy 1, policy_version 1859002 (0.0008) [2023-12-27 18:32:14,596][20381] Updated weights for policy 1, policy_version 1859012 (0.0007) [2023-12-27 18:32:14,646][20381] Updated weights for policy 1, policy_version 1859022 (0.0008) [2023-12-27 18:32:14,695][20381] Updated weights for policy 1, policy_version 1859032 (0.0007) [2023-12-27 18:32:15,200][20347] Updated weights for policy 0, policy_version 1860610 (0.0008) [2023-12-27 18:32:15,262][20347] Updated weights for policy 0, policy_version 1860620 (0.0006) [2023-12-27 18:32:15,323][20347] Updated weights for policy 0, policy_version 1860630 (0.0006) [2023-12-27 18:32:15,360][20381] Updated weights for policy 1, policy_version 1859042 (0.0008) [2023-12-27 18:32:15,422][20381] Updated weights for policy 1, policy_version 1859052 (0.0008) [2023-12-27 18:32:15,488][20381] Updated weights for policy 1, policy_version 1859062 (0.0008) [2023-12-27 18:32:15,985][20347] Updated weights for policy 0, policy_version 1860640 (0.0008) [2023-12-27 18:32:16,044][20347] Updated weights for policy 0, policy_version 1860650 (0.0006) [2023-12-27 18:32:16,098][20347] Updated weights for policy 0, policy_version 1860660 (0.0007) [2023-12-27 18:32:16,105][20381] Updated weights for policy 1, policy_version 1859072 (0.0007) [2023-12-27 18:32:16,178][20381] Updated weights for policy 1, policy_version 1859082 (0.0007) [2023-12-27 18:32:16,244][20381] Updated weights for policy 1, policy_version 1859092 (0.0006) [2023-12-27 18:32:16,680][20347] Updated weights for policy 0, policy_version 1860670 (0.0008) [2023-12-27 18:32:16,735][20347] Updated weights for policy 0, policy_version 1860680 (0.0010) [2023-12-27 18:32:16,792][20347] Updated weights for policy 0, policy_version 1860690 (0.0010) [2023-12-27 18:32:16,923][20381] Updated weights for policy 1, policy_version 1859102 (0.0009) [2023-12-27 18:32:16,969][20381] Updated weights for policy 1, policy_version 1859112 (0.0008) [2023-12-27 18:32:17,022][20381] Updated weights for policy 1, policy_version 1859122 (0.0007) [2023-12-27 18:32:17,349][20347] Updated weights for policy 0, policy_version 1860700 (0.0009) [2023-12-27 18:32:17,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 952410112. Throughput: 0: 10093.2, 1: 10209.6. Samples: 952421256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '850.000')] [2023-12-27 18:32:17,410][20347] Updated weights for policy 0, policy_version 1860710 (0.0007) [2023-12-27 18:32:17,467][20347] Updated weights for policy 0, policy_version 1860720 (0.0010) [2023-12-27 18:32:17,790][20381] Updated weights for policy 1, policy_version 1859132 (0.0007) [2023-12-27 18:32:17,848][20381] Updated weights for policy 1, policy_version 1859142 (0.0007) [2023-12-27 18:32:17,902][20381] Updated weights for policy 1, policy_version 1859152 (0.0007) [2023-12-27 18:32:18,068][20347] Updated weights for policy 0, policy_version 1860730 (0.0009) [2023-12-27 18:32:18,125][20347] Updated weights for policy 0, policy_version 1860740 (0.0006) [2023-12-27 18:32:18,173][20347] Updated weights for policy 0, policy_version 1860750 (0.0008) [2023-12-27 18:32:18,221][20347] Updated weights for policy 0, policy_version 1860760 (0.0010) [2023-12-27 18:32:18,610][20381] Updated weights for policy 1, policy_version 1859162 (0.0007) [2023-12-27 18:32:18,672][20381] Updated weights for policy 1, policy_version 1859172 (0.0005) [2023-12-27 18:32:18,729][20381] Updated weights for policy 1, policy_version 1859182 (0.0005) [2023-12-27 18:32:18,792][20381] Updated weights for policy 1, policy_version 1859192 (0.0006) [2023-12-27 18:32:18,898][20347] Updated weights for policy 0, policy_version 1860770 (0.0011) [2023-12-27 18:32:18,959][20347] Updated weights for policy 0, policy_version 1860780 (0.0010) [2023-12-27 18:32:19,007][20347] Updated weights for policy 0, policy_version 1860790 (0.0010) [2023-12-27 18:32:19,344][20381] Updated weights for policy 1, policy_version 1859202 (0.0007) [2023-12-27 18:32:19,402][20381] Updated weights for policy 1, policy_version 1859212 (0.0007) [2023-12-27 18:32:19,456][20381] Updated weights for policy 1, policy_version 1859222 (0.0007) [2023-12-27 18:32:19,752][20347] Updated weights for policy 0, policy_version 1860800 (0.0007) [2023-12-27 18:32:19,812][20347] Updated weights for policy 0, policy_version 1860810 (0.0007) [2023-12-27 18:32:19,876][20347] Updated weights for policy 0, policy_version 1860820 (0.0010) [2023-12-27 18:32:20,157][20381] Updated weights for policy 1, policy_version 1859232 (0.0006) [2023-12-27 18:32:20,212][20381] Updated weights for policy 1, policy_version 1859242 (0.0006) [2023-12-27 18:32:20,269][20381] Updated weights for policy 1, policy_version 1859252 (0.0007) [2023-12-27 18:32:20,640][20347] Updated weights for policy 0, policy_version 1860830 (0.0008) [2023-12-27 18:32:20,695][20347] Updated weights for policy 0, policy_version 1860840 (0.0008) [2023-12-27 18:32:20,755][20347] Updated weights for policy 0, policy_version 1860850 (0.0008) [2023-12-27 18:32:20,913][20381] Updated weights for policy 1, policy_version 1859262 (0.0007) [2023-12-27 18:32:20,975][20381] Updated weights for policy 1, policy_version 1859272 (0.0007) [2023-12-27 18:32:21,030][20381] Updated weights for policy 1, policy_version 1859282 (0.0007) [2023-12-27 18:32:21,560][20347] Updated weights for policy 0, policy_version 1860860 (0.0009) [2023-12-27 18:32:21,630][20347] Updated weights for policy 0, policy_version 1860870 (0.0007) [2023-12-27 18:32:21,691][20347] Updated weights for policy 0, policy_version 1860880 (0.0008) [2023-12-27 18:32:21,706][20381] Updated weights for policy 1, policy_version 1859292 (0.0007) [2023-12-27 18:32:21,781][20381] Updated weights for policy 1, policy_version 1859302 (0.0009) [2023-12-27 18:32:21,849][20381] Updated weights for policy 1, policy_version 1859312 (0.0008) [2023-12-27 18:32:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 952508416. Throughput: 0: 10114.3, 1: 10209.9. Samples: 952483600. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:22,363][19170] Avg episode reward: [(0, '990.230'), (1, '790.700')] [2023-12-27 18:32:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001859320_476037120.pth... [2023-12-27 18:32:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001860888_476471296.pth... [2023-12-27 18:32:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001858136_475734016.pth [2023-12-27 18:32:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001859736_476176384.pth [2023-12-27 18:32:22,496][20347] Updated weights for policy 0, policy_version 1860890 (0.0008) [2023-12-27 18:32:22,520][20381] Updated weights for policy 1, policy_version 1859322 (0.0006) [2023-12-27 18:32:22,559][20347] Updated weights for policy 0, policy_version 1860900 (0.0008) [2023-12-27 18:32:22,582][20381] Updated weights for policy 1, policy_version 1859332 (0.0006) [2023-12-27 18:32:22,625][20347] Updated weights for policy 0, policy_version 1860910 (0.0008) [2023-12-27 18:32:22,643][20381] Updated weights for policy 1, policy_version 1859342 (0.0007) [2023-12-27 18:32:22,689][20347] Updated weights for policy 0, policy_version 1860920 (0.0008) [2023-12-27 18:32:22,698][20381] Updated weights for policy 1, policy_version 1859352 (0.0009) [2023-12-27 18:32:23,321][20381] Updated weights for policy 1, policy_version 1859362 (0.0010) [2023-12-27 18:32:23,366][20381] Updated weights for policy 1, policy_version 1859372 (0.0008) [2023-12-27 18:32:23,402][20347] Updated weights for policy 0, policy_version 1860930 (0.0009) [2023-12-27 18:32:23,411][20381] Updated weights for policy 1, policy_version 1859382 (0.0005) [2023-12-27 18:32:23,457][20347] Updated weights for policy 0, policy_version 1860940 (0.0010) [2023-12-27 18:32:23,508][20347] Updated weights for policy 0, policy_version 1860950 (0.0010) [2023-12-27 18:32:24,076][20381] Updated weights for policy 1, policy_version 1859392 (0.0008) [2023-12-27 18:32:24,125][20381] Updated weights for policy 1, policy_version 1859402 (0.0007) [2023-12-27 18:32:24,179][20347] Updated weights for policy 0, policy_version 1860960 (0.0008) [2023-12-27 18:32:24,185][20381] Updated weights for policy 1, policy_version 1859412 (0.0007) [2023-12-27 18:32:24,231][20347] Updated weights for policy 0, policy_version 1860970 (0.0005) [2023-12-27 18:32:24,281][20347] Updated weights for policy 0, policy_version 1860980 (0.0005) [2023-12-27 18:32:24,830][20381] Updated weights for policy 1, policy_version 1859422 (0.0006) [2023-12-27 18:32:24,894][20381] Updated weights for policy 1, policy_version 1859432 (0.0005) [2023-12-27 18:32:24,910][20347] Updated weights for policy 0, policy_version 1860990 (0.0006) [2023-12-27 18:32:24,956][20381] Updated weights for policy 1, policy_version 1859442 (0.0005) [2023-12-27 18:32:24,962][20347] Updated weights for policy 0, policy_version 1861000 (0.0007) [2023-12-27 18:32:25,012][20347] Updated weights for policy 0, policy_version 1861010 (0.0006) [2023-12-27 18:32:25,614][20381] Updated weights for policy 1, policy_version 1859452 (0.0008) [2023-12-27 18:32:25,643][20347] Updated weights for policy 0, policy_version 1861020 (0.0008) [2023-12-27 18:32:25,669][20381] Updated weights for policy 1, policy_version 1859462 (0.0010) [2023-12-27 18:32:25,699][20347] Updated weights for policy 0, policy_version 1861030 (0.0008) [2023-12-27 18:32:25,723][20381] Updated weights for policy 1, policy_version 1859472 (0.0006) [2023-12-27 18:32:25,754][20347] Updated weights for policy 0, policy_version 1861040 (0.0008) [2023-12-27 18:32:26,385][20381] Updated weights for policy 1, policy_version 1859482 (0.0008) [2023-12-27 18:32:26,439][20381] Updated weights for policy 1, policy_version 1859492 (0.0010) [2023-12-27 18:32:26,445][20347] Updated weights for policy 0, policy_version 1861050 (0.0008) [2023-12-27 18:32:26,491][20381] Updated weights for policy 1, policy_version 1859502 (0.0010) [2023-12-27 18:32:26,492][20347] Updated weights for policy 0, policy_version 1861060 (0.0005) [2023-12-27 18:32:26,540][20347] Updated weights for policy 0, policy_version 1861070 (0.0005) [2023-12-27 18:32:26,545][20381] Updated weights for policy 1, policy_version 1859512 (0.0010) [2023-12-27 18:32:26,586][20347] Updated weights for policy 0, policy_version 1861080 (0.0007) [2023-12-27 18:32:27,238][20381] Updated weights for policy 1, policy_version 1859522 (0.0011) [2023-12-27 18:32:27,294][20347] Updated weights for policy 0, policy_version 1861090 (0.0005) [2023-12-27 18:32:27,296][20381] Updated weights for policy 1, policy_version 1859532 (0.0010) [2023-12-27 18:32:27,343][20347] Updated weights for policy 0, policy_version 1861100 (0.0007) [2023-12-27 18:32:27,347][20381] Updated weights for policy 1, policy_version 1859542 (0.0010) [2023-12-27 18:32:27,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 952614912. Throughput: 0: 10077.6, 1: 10257.1. Samples: 952605020. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:27,363][19170] Avg episode reward: [(0, '990.230'), (1, '802.540')] [2023-12-27 18:32:27,389][20347] Updated weights for policy 0, policy_version 1861110 (0.0007) [2023-12-27 18:32:28,026][20347] Updated weights for policy 0, policy_version 1861120 (0.0009) [2023-12-27 18:32:28,030][20381] Updated weights for policy 1, policy_version 1859552 (0.0010) [2023-12-27 18:32:28,078][20381] Updated weights for policy 1, policy_version 1859562 (0.0010) [2023-12-27 18:32:28,083][20347] Updated weights for policy 0, policy_version 1861130 (0.0006) [2023-12-27 18:32:28,133][20381] Updated weights for policy 1, policy_version 1859572 (0.0010) [2023-12-27 18:32:28,137][20347] Updated weights for policy 0, policy_version 1861140 (0.0005) [2023-12-27 18:32:28,774][20347] Updated weights for policy 0, policy_version 1861150 (0.0006) [2023-12-27 18:32:28,830][20347] Updated weights for policy 0, policy_version 1861160 (0.0006) [2023-12-27 18:32:28,832][20381] Updated weights for policy 1, policy_version 1859582 (0.0010) [2023-12-27 18:32:28,882][20347] Updated weights for policy 0, policy_version 1861170 (0.0005) [2023-12-27 18:32:28,884][20381] Updated weights for policy 1, policy_version 1859592 (0.0010) [2023-12-27 18:32:28,937][20381] Updated weights for policy 1, policy_version 1859602 (0.0010) [2023-12-27 18:32:29,587][20347] Updated weights for policy 0, policy_version 1861180 (0.0006) [2023-12-27 18:32:29,643][20347] Updated weights for policy 0, policy_version 1861190 (0.0007) [2023-12-27 18:32:29,652][20381] Updated weights for policy 1, policy_version 1859612 (0.0009) [2023-12-27 18:32:29,701][20347] Updated weights for policy 0, policy_version 1861200 (0.0009) [2023-12-27 18:32:29,707][20381] Updated weights for policy 1, policy_version 1859622 (0.0007) [2023-12-27 18:32:29,757][20381] Updated weights for policy 1, policy_version 1859632 (0.0007) [2023-12-27 18:32:30,355][20347] Updated weights for policy 0, policy_version 1861210 (0.0005) [2023-12-27 18:32:30,416][20347] Updated weights for policy 0, policy_version 1861220 (0.0005) [2023-12-27 18:32:30,432][20381] Updated weights for policy 1, policy_version 1859642 (0.0008) [2023-12-27 18:32:30,463][20347] Updated weights for policy 0, policy_version 1861230 (0.0006) [2023-12-27 18:32:30,484][20381] Updated weights for policy 1, policy_version 1859652 (0.0007) [2023-12-27 18:32:30,510][20347] Updated weights for policy 0, policy_version 1861240 (0.0008) [2023-12-27 18:32:30,537][20381] Updated weights for policy 1, policy_version 1859662 (0.0009) [2023-12-27 18:32:30,586][20381] Updated weights for policy 1, policy_version 1859672 (0.0008) [2023-12-27 18:32:31,167][20347] Updated weights for policy 0, policy_version 1861250 (0.0007) [2023-12-27 18:32:31,214][20347] Updated weights for policy 0, policy_version 1861260 (0.0005) [2023-12-27 18:32:31,266][20347] Updated weights for policy 0, policy_version 1861270 (0.0008) [2023-12-27 18:32:31,277][20381] Updated weights for policy 1, policy_version 1859682 (0.0007) [2023-12-27 18:32:31,339][20381] Updated weights for policy 1, policy_version 1859692 (0.0007) [2023-12-27 18:32:31,405][20381] Updated weights for policy 1, policy_version 1859702 (0.0008) [2023-12-27 18:32:32,011][20381] Updated weights for policy 1, policy_version 1859712 (0.0007) [2023-12-27 18:32:32,013][20347] Updated weights for policy 0, policy_version 1861280 (0.0006) [2023-12-27 18:32:32,070][20381] Updated weights for policy 1, policy_version 1859722 (0.0006) [2023-12-27 18:32:32,073][20347] Updated weights for policy 0, policy_version 1861290 (0.0007) [2023-12-27 18:32:32,129][20347] Updated weights for policy 0, policy_version 1861300 (0.0007) [2023-12-27 18:32:32,138][20381] Updated weights for policy 1, policy_version 1859732 (0.0007) [2023-12-27 18:32:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 952721408. Throughput: 0: 10109.5, 1: 10234.8. Samples: 952728264. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '781.560')] [2023-12-27 18:32:32,759][20381] Updated weights for policy 1, policy_version 1859742 (0.0006) [2023-12-27 18:32:32,811][20381] Updated weights for policy 1, policy_version 1859752 (0.0008) [2023-12-27 18:32:32,858][20347] Updated weights for policy 0, policy_version 1861310 (0.0006) [2023-12-27 18:32:32,864][20381] Updated weights for policy 1, policy_version 1859762 (0.0007) [2023-12-27 18:32:32,912][20347] Updated weights for policy 0, policy_version 1861320 (0.0007) [2023-12-27 18:32:32,961][20347] Updated weights for policy 0, policy_version 1861330 (0.0008) [2023-12-27 18:32:33,493][20381] Updated weights for policy 1, policy_version 1859772 (0.0005) [2023-12-27 18:32:33,548][20381] Updated weights for policy 1, policy_version 1859782 (0.0005) [2023-12-27 18:32:33,602][20381] Updated weights for policy 1, policy_version 1859792 (0.0008) [2023-12-27 18:32:33,687][20347] Updated weights for policy 0, policy_version 1861340 (0.0008) [2023-12-27 18:32:33,751][20347] Updated weights for policy 0, policy_version 1861350 (0.0007) [2023-12-27 18:32:33,814][20347] Updated weights for policy 0, policy_version 1861360 (0.0009) [2023-12-27 18:32:34,171][20381] Updated weights for policy 1, policy_version 1859802 (0.0008) [2023-12-27 18:32:34,228][20381] Updated weights for policy 1, policy_version 1859812 (0.0007) [2023-12-27 18:32:34,284][20381] Updated weights for policy 1, policy_version 1859822 (0.0007) [2023-12-27 18:32:34,345][20381] Updated weights for policy 1, policy_version 1859832 (0.0007) [2023-12-27 18:32:34,465][20347] Updated weights for policy 0, policy_version 1861370 (0.0009) [2023-12-27 18:32:34,520][20347] Updated weights for policy 0, policy_version 1861380 (0.0005) [2023-12-27 18:32:34,585][20347] Updated weights for policy 0, policy_version 1861390 (0.0005) [2023-12-27 18:32:34,649][20347] Updated weights for policy 0, policy_version 1861400 (0.0006) [2023-12-27 18:32:35,047][20381] Updated weights for policy 1, policy_version 1859842 (0.0006) [2023-12-27 18:32:35,110][20381] Updated weights for policy 1, policy_version 1859852 (0.0005) [2023-12-27 18:32:35,167][20381] Updated weights for policy 1, policy_version 1859862 (0.0005) [2023-12-27 18:32:35,238][20347] Updated weights for policy 0, policy_version 1861410 (0.0010) [2023-12-27 18:32:35,288][20347] Updated weights for policy 0, policy_version 1861420 (0.0010) [2023-12-27 18:32:35,333][20347] Updated weights for policy 0, policy_version 1861430 (0.0010) [2023-12-27 18:32:35,756][20381] Updated weights for policy 1, policy_version 1859872 (0.0010) [2023-12-27 18:32:35,805][20381] Updated weights for policy 1, policy_version 1859882 (0.0010) [2023-12-27 18:32:35,864][20381] Updated weights for policy 1, policy_version 1859892 (0.0011) [2023-12-27 18:32:35,997][20347] Updated weights for policy 0, policy_version 1861440 (0.0008) [2023-12-27 18:32:36,057][20347] Updated weights for policy 0, policy_version 1861450 (0.0007) [2023-12-27 18:32:36,129][20347] Updated weights for policy 0, policy_version 1861460 (0.0008) [2023-12-27 18:32:36,586][20381] Updated weights for policy 1, policy_version 1859902 (0.0009) [2023-12-27 18:32:36,642][20381] Updated weights for policy 1, policy_version 1859912 (0.0008) [2023-12-27 18:32:36,691][20381] Updated weights for policy 1, policy_version 1859922 (0.0007) [2023-12-27 18:32:36,812][20347] Updated weights for policy 0, policy_version 1861470 (0.0009) [2023-12-27 18:32:36,880][20347] Updated weights for policy 0, policy_version 1861480 (0.0010) [2023-12-27 18:32:36,931][20347] Updated weights for policy 0, policy_version 1861490 (0.0010) [2023-12-27 18:32:37,342][20381] Updated weights for policy 1, policy_version 1859932 (0.0006) [2023-12-27 18:32:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.6, 300 sec: 20438.3). Total num frames: 952819712. Throughput: 0: 10081.1, 1: 10322.2. Samples: 952792156. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '743.230')] [2023-12-27 18:32:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001861496_476626944.pth... [2023-12-27 18:32:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001860312_476323840.pth [2023-12-27 18:32:37,397][20381] Updated weights for policy 1, policy_version 1859942 (0.0005) [2023-12-27 18:32:37,451][20381] Updated weights for policy 1, policy_version 1859952 (0.0009) [2023-12-27 18:32:37,491][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001859960_476200960.pth... [2023-12-27 18:32:37,494][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001858744_475889664.pth [2023-12-27 18:32:37,586][20347] Updated weights for policy 0, policy_version 1861500 (0.0009) [2023-12-27 18:32:37,646][20347] Updated weights for policy 0, policy_version 1861510 (0.0006) [2023-12-27 18:32:37,707][20347] Updated weights for policy 0, policy_version 1861520 (0.0010) [2023-12-27 18:32:38,065][20381] Updated weights for policy 1, policy_version 1859962 (0.0008) [2023-12-27 18:32:38,122][20381] Updated weights for policy 1, policy_version 1859972 (0.0007) [2023-12-27 18:32:38,170][20381] Updated weights for policy 1, policy_version 1859982 (0.0007) [2023-12-27 18:32:38,220][20381] Updated weights for policy 1, policy_version 1859992 (0.0007) [2023-12-27 18:32:38,317][20347] Updated weights for policy 0, policy_version 1861530 (0.0010) [2023-12-27 18:32:38,373][20347] Updated weights for policy 0, policy_version 1861540 (0.0008) [2023-12-27 18:32:38,436][20347] Updated weights for policy 0, policy_version 1861550 (0.0007) [2023-12-27 18:32:38,490][20347] Updated weights for policy 0, policy_version 1861560 (0.0006) [2023-12-27 18:32:38,985][20381] Updated weights for policy 1, policy_version 1860003 (0.0009) [2023-12-27 18:32:39,038][20381] Updated weights for policy 1, policy_version 1860013 (0.0009) [2023-12-27 18:32:39,095][20381] Updated weights for policy 1, policy_version 1860023 (0.0007) [2023-12-27 18:32:39,125][20347] Updated weights for policy 0, policy_version 1861570 (0.0006) [2023-12-27 18:32:39,179][20347] Updated weights for policy 0, policy_version 1861580 (0.0005) [2023-12-27 18:32:39,229][20347] Updated weights for policy 0, policy_version 1861590 (0.0008) [2023-12-27 18:32:39,846][20381] Updated weights for policy 1, policy_version 1860033 (0.0010) [2023-12-27 18:32:39,904][20347] Updated weights for policy 0, policy_version 1861600 (0.0008) [2023-12-27 18:32:39,911][20381] Updated weights for policy 1, policy_version 1860043 (0.0008) [2023-12-27 18:32:39,967][20347] Updated weights for policy 0, policy_version 1861610 (0.0010) [2023-12-27 18:32:39,981][20381] Updated weights for policy 1, policy_version 1860053 (0.0008) [2023-12-27 18:32:40,027][20347] Updated weights for policy 0, policy_version 1861620 (0.0010) [2023-12-27 18:32:40,696][20347] Updated weights for policy 0, policy_version 1861630 (0.0008) [2023-12-27 18:32:40,746][20381] Updated weights for policy 1, policy_version 1860063 (0.0007) [2023-12-27 18:32:40,752][20347] Updated weights for policy 0, policy_version 1861640 (0.0006) [2023-12-27 18:32:40,806][20381] Updated weights for policy 1, policy_version 1860073 (0.0006) [2023-12-27 18:32:40,816][20347] Updated weights for policy 0, policy_version 1861650 (0.0007) [2023-12-27 18:32:40,867][20381] Updated weights for policy 1, policy_version 1860083 (0.0007) [2023-12-27 18:32:41,557][20347] Updated weights for policy 0, policy_version 1861660 (0.0006) [2023-12-27 18:32:41,577][20381] Updated weights for policy 1, policy_version 1860093 (0.0007) [2023-12-27 18:32:41,612][20347] Updated weights for policy 0, policy_version 1861670 (0.0006) [2023-12-27 18:32:41,644][20381] Updated weights for policy 1, policy_version 1860103 (0.0008) [2023-12-27 18:32:41,674][20347] Updated weights for policy 0, policy_version 1861680 (0.0007) [2023-12-27 18:32:41,703][20381] Updated weights for policy 1, policy_version 1860113 (0.0006) [2023-12-27 18:32:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 952918016. Throughput: 0: 10083.3, 1: 10268.4. Samples: 952913876. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '781.940')] [2023-12-27 18:32:42,409][20347] Updated weights for policy 0, policy_version 1861690 (0.0007) [2023-12-27 18:32:42,425][20381] Updated weights for policy 1, policy_version 1860123 (0.0007) [2023-12-27 18:32:42,465][20347] Updated weights for policy 0, policy_version 1861700 (0.0006) [2023-12-27 18:32:42,482][20381] Updated weights for policy 1, policy_version 1860133 (0.0008) [2023-12-27 18:32:42,520][20347] Updated weights for policy 0, policy_version 1861710 (0.0005) [2023-12-27 18:32:42,537][20381] Updated weights for policy 1, policy_version 1860143 (0.0006) [2023-12-27 18:32:42,583][20347] Updated weights for policy 0, policy_version 1861720 (0.0006) [2023-12-27 18:32:43,240][20347] Updated weights for policy 0, policy_version 1861730 (0.0009) [2023-12-27 18:32:43,245][20381] Updated weights for policy 1, policy_version 1860153 (0.0006) [2023-12-27 18:32:43,287][20381] Updated weights for policy 1, policy_version 1860163 (0.0007) [2023-12-27 18:32:43,289][20347] Updated weights for policy 0, policy_version 1861740 (0.0005) [2023-12-27 18:32:43,340][20381] Updated weights for policy 1, policy_version 1860173 (0.0005) [2023-12-27 18:32:43,342][20347] Updated weights for policy 0, policy_version 1861750 (0.0005) [2023-12-27 18:32:43,401][20381] Updated weights for policy 1, policy_version 1860183 (0.0005) [2023-12-27 18:32:43,908][20347] Updated weights for policy 0, policy_version 1861760 (0.0005) [2023-12-27 18:32:43,977][20347] Updated weights for policy 0, policy_version 1861770 (0.0005) [2023-12-27 18:32:44,027][20381] Updated weights for policy 1, policy_version 1860193 (0.0007) [2023-12-27 18:32:44,046][20347] Updated weights for policy 0, policy_version 1861780 (0.0008) [2023-12-27 18:32:44,096][20381] Updated weights for policy 1, policy_version 1860203 (0.0009) [2023-12-27 18:32:44,162][20381] Updated weights for policy 1, policy_version 1860213 (0.0008) [2023-12-27 18:32:44,619][20347] Updated weights for policy 0, policy_version 1861790 (0.0008) [2023-12-27 18:32:44,677][20347] Updated weights for policy 0, policy_version 1861800 (0.0007) [2023-12-27 18:32:44,734][20347] Updated weights for policy 0, policy_version 1861810 (0.0007) [2023-12-27 18:32:44,882][20381] Updated weights for policy 1, policy_version 1860223 (0.0007) [2023-12-27 18:32:44,945][20381] Updated weights for policy 1, policy_version 1860233 (0.0010) [2023-12-27 18:32:45,007][20381] Updated weights for policy 1, policy_version 1860243 (0.0010) [2023-12-27 18:32:45,397][20347] Updated weights for policy 0, policy_version 1861820 (0.0009) [2023-12-27 18:32:45,455][20347] Updated weights for policy 0, policy_version 1861830 (0.0009) [2023-12-27 18:32:45,511][20347] Updated weights for policy 0, policy_version 1861840 (0.0007) [2023-12-27 18:32:45,697][20381] Updated weights for policy 1, policy_version 1860253 (0.0009) [2023-12-27 18:32:45,743][20381] Updated weights for policy 1, policy_version 1860263 (0.0007) [2023-12-27 18:32:45,791][20381] Updated weights for policy 1, policy_version 1860273 (0.0008) [2023-12-27 18:32:46,104][20347] Updated weights for policy 0, policy_version 1861850 (0.0006) [2023-12-27 18:32:46,166][20347] Updated weights for policy 0, policy_version 1861860 (0.0011) [2023-12-27 18:32:46,223][20347] Updated weights for policy 0, policy_version 1861870 (0.0010) [2023-12-27 18:32:46,285][20347] Updated weights for policy 0, policy_version 1861880 (0.0009) [2023-12-27 18:32:46,386][20381] Updated weights for policy 1, policy_version 1860283 (0.0007) [2023-12-27 18:32:46,450][20381] Updated weights for policy 1, policy_version 1860293 (0.0005) [2023-12-27 18:32:46,500][20381] Updated weights for policy 1, policy_version 1860303 (0.0010) [2023-12-27 18:32:46,904][20347] Updated weights for policy 0, policy_version 1861890 (0.0010) [2023-12-27 18:32:46,957][20347] Updated weights for policy 0, policy_version 1861900 (0.0010) [2023-12-27 18:32:47,013][20347] Updated weights for policy 0, policy_version 1861910 (0.0005) [2023-12-27 18:32:47,158][20381] Updated weights for policy 1, policy_version 1860313 (0.0010) [2023-12-27 18:32:47,210][20381] Updated weights for policy 1, policy_version 1860323 (0.0010) [2023-12-27 18:32:47,258][20381] Updated weights for policy 1, policy_version 1860333 (0.0010) [2023-12-27 18:32:47,315][20381] Updated weights for policy 1, policy_version 1860343 (0.0010) [2023-12-27 18:32:47,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 953032704. Throughput: 0: 10159.7, 1: 10314.3. Samples: 953039032. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '833.520')] [2023-12-27 18:32:47,586][20347] Updated weights for policy 0, policy_version 1861920 (0.0009) [2023-12-27 18:32:47,646][20347] Updated weights for policy 0, policy_version 1861930 (0.0008) [2023-12-27 18:32:47,706][20347] Updated weights for policy 0, policy_version 1861940 (0.0005) [2023-12-27 18:32:47,976][20381] Updated weights for policy 1, policy_version 1860353 (0.0006) [2023-12-27 18:32:48,036][20381] Updated weights for policy 1, policy_version 1860363 (0.0006) [2023-12-27 18:32:48,102][20381] Updated weights for policy 1, policy_version 1860373 (0.0007) [2023-12-27 18:32:48,288][20347] Updated weights for policy 0, policy_version 1861950 (0.0009) [2023-12-27 18:32:48,360][20347] Updated weights for policy 0, policy_version 1861960 (0.0008) [2023-12-27 18:32:48,418][20347] Updated weights for policy 0, policy_version 1861970 (0.0008) [2023-12-27 18:32:48,757][20381] Updated weights for policy 1, policy_version 1860383 (0.0007) [2023-12-27 18:32:48,818][20381] Updated weights for policy 1, policy_version 1860393 (0.0008) [2023-12-27 18:32:48,873][20381] Updated weights for policy 1, policy_version 1860403 (0.0008) [2023-12-27 18:32:48,998][20347] Updated weights for policy 0, policy_version 1861980 (0.0008) [2023-12-27 18:32:49,056][20347] Updated weights for policy 0, policy_version 1861990 (0.0007) [2023-12-27 18:32:49,114][20347] Updated weights for policy 0, policy_version 1862000 (0.0008) [2023-12-27 18:32:49,576][20381] Updated weights for policy 1, policy_version 1860413 (0.0008) [2023-12-27 18:32:49,628][20381] Updated weights for policy 1, policy_version 1860423 (0.0008) [2023-12-27 18:32:49,681][20381] Updated weights for policy 1, policy_version 1860433 (0.0007) [2023-12-27 18:32:49,827][20347] Updated weights for policy 0, policy_version 1862010 (0.0007) [2023-12-27 18:32:49,892][20347] Updated weights for policy 0, policy_version 1862020 (0.0007) [2023-12-27 18:32:49,962][20347] Updated weights for policy 0, policy_version 1862030 (0.0007) [2023-12-27 18:32:50,416][20381] Updated weights for policy 1, policy_version 1860443 (0.0007) [2023-12-27 18:32:50,474][20381] Updated weights for policy 1, policy_version 1860453 (0.0007) [2023-12-27 18:32:50,526][20381] Updated weights for policy 1, policy_version 1860463 (0.0007) [2023-12-27 18:32:50,659][20347] Updated weights for policy 0, policy_version 1862041 (0.0007) [2023-12-27 18:32:50,712][20347] Updated weights for policy 0, policy_version 1862051 (0.0007) [2023-12-27 18:32:50,764][20347] Updated weights for policy 0, policy_version 1862061 (0.0008) [2023-12-27 18:32:50,821][20347] Updated weights for policy 0, policy_version 1862071 (0.0008) [2023-12-27 18:32:51,253][20381] Updated weights for policy 1, policy_version 1860473 (0.0006) [2023-12-27 18:32:51,316][20381] Updated weights for policy 1, policy_version 1860483 (0.0008) [2023-12-27 18:32:51,379][20381] Updated weights for policy 1, policy_version 1860493 (0.0008) [2023-12-27 18:32:51,446][20381] Updated weights for policy 1, policy_version 1860503 (0.0008) [2023-12-27 18:32:51,554][20347] Updated weights for policy 0, policy_version 1862081 (0.0008) [2023-12-27 18:32:51,620][20347] Updated weights for policy 0, policy_version 1862091 (0.0008) [2023-12-27 18:32:51,686][20347] Updated weights for policy 0, policy_version 1862101 (0.0008) [2023-12-27 18:32:52,126][20381] Updated weights for policy 1, policy_version 1860513 (0.0008) [2023-12-27 18:32:52,175][20381] Updated weights for policy 1, policy_version 1860523 (0.0008) [2023-12-27 18:32:52,221][20381] Updated weights for policy 1, policy_version 1860533 (0.0008) [2023-12-27 18:32:52,331][20347] Updated weights for policy 0, policy_version 1862111 (0.0008) [2023-12-27 18:32:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 953131008. Throughput: 0: 10237.2, 1: 10247.9. Samples: 953101796. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:32:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '868.810')] [2023-12-27 18:32:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001860536_476348416.pth... [2023-12-27 18:32:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001859320_476037120.pth [2023-12-27 18:32:52,389][20347] Updated weights for policy 0, policy_version 1862121 (0.0007) [2023-12-27 18:32:52,451][20347] Updated weights for policy 0, policy_version 1862131 (0.0006) [2023-12-27 18:32:52,481][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001862136_476790784.pth... [2023-12-27 18:32:52,484][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001860888_476471296.pth [2023-12-27 18:32:52,938][20381] Updated weights for policy 1, policy_version 1860543 (0.0008) [2023-12-27 18:32:52,997][20381] Updated weights for policy 1, policy_version 1860553 (0.0007) [2023-12-27 18:32:53,054][20381] Updated weights for policy 1, policy_version 1860563 (0.0005) [2023-12-27 18:32:53,068][20347] Updated weights for policy 0, policy_version 1862141 (0.0005) [2023-12-27 18:32:53,120][20347] Updated weights for policy 0, policy_version 1862151 (0.0005) [2023-12-27 18:32:53,170][20347] Updated weights for policy 0, policy_version 1862161 (0.0006) [2023-12-27 18:32:53,688][20381] Updated weights for policy 1, policy_version 1860573 (0.0005) [2023-12-27 18:32:53,739][20381] Updated weights for policy 1, policy_version 1860583 (0.0008) [2023-12-27 18:32:53,793][20381] Updated weights for policy 1, policy_version 1860593 (0.0011) [2023-12-27 18:32:53,834][20347] Updated weights for policy 0, policy_version 1862171 (0.0006) [2023-12-27 18:32:53,897][20347] Updated weights for policy 0, policy_version 1862181 (0.0007) [2023-12-27 18:32:53,959][20347] Updated weights for policy 0, policy_version 1862191 (0.0006) [2023-12-27 18:32:54,469][20381] Updated weights for policy 1, policy_version 1860603 (0.0009) [2023-12-27 18:32:54,522][20381] Updated weights for policy 1, policy_version 1860613 (0.0005) [2023-12-27 18:32:54,571][20381] Updated weights for policy 1, policy_version 1860623 (0.0005) [2023-12-27 18:32:54,599][20347] Updated weights for policy 0, policy_version 1862201 (0.0006) [2023-12-27 18:32:54,655][20347] Updated weights for policy 0, policy_version 1862211 (0.0005) [2023-12-27 18:32:54,709][20347] Updated weights for policy 0, policy_version 1862221 (0.0005) [2023-12-27 18:32:54,769][20347] Updated weights for policy 0, policy_version 1862231 (0.0007) [2023-12-27 18:32:55,125][20381] Updated weights for policy 1, policy_version 1860633 (0.0008) [2023-12-27 18:32:55,181][20381] Updated weights for policy 1, policy_version 1860643 (0.0011) [2023-12-27 18:32:55,232][20381] Updated weights for policy 1, policy_version 1860653 (0.0008) [2023-12-27 18:32:55,284][20381] Updated weights for policy 1, policy_version 1860663 (0.0005) [2023-12-27 18:32:55,289][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000007 [2023-12-27 18:32:55,461][20347] Updated weights for policy 0, policy_version 1862241 (0.0008) [2023-12-27 18:32:55,512][20347] Updated weights for policy 0, policy_version 1862251 (0.0008) [2023-12-27 18:32:55,567][20347] Updated weights for policy 0, policy_version 1862261 (0.0008) [2023-12-27 18:32:55,848][20381] Updated weights for policy 1, policy_version 1860673 (0.0006) [2023-12-27 18:32:55,904][20381] Updated weights for policy 1, policy_version 1860683 (0.0007) [2023-12-27 18:32:55,961][20381] Updated weights for policy 1, policy_version 1860693 (0.0005) [2023-12-27 18:32:56,251][20347] Updated weights for policy 0, policy_version 1862271 (0.0010) [2023-12-27 18:32:56,302][20347] Updated weights for policy 0, policy_version 1862281 (0.0010) [2023-12-27 18:32:56,361][20347] Updated weights for policy 0, policy_version 1862291 (0.0006) [2023-12-27 18:32:56,585][20381] Updated weights for policy 1, policy_version 1860703 (0.0007) [2023-12-27 18:32:56,628][20381] Updated weights for policy 1, policy_version 1860713 (0.0009) [2023-12-27 18:32:56,673][20381] Updated weights for policy 1, policy_version 1860723 (0.0007) [2023-12-27 18:32:57,021][20347] Updated weights for policy 0, policy_version 1862301 (0.0007) [2023-12-27 18:32:57,076][20347] Updated weights for policy 0, policy_version 1862311 (0.0007) [2023-12-27 18:32:57,131][20347] Updated weights for policy 0, policy_version 1862321 (0.0008) [2023-12-27 18:32:57,322][20381] Updated weights for policy 1, policy_version 1860733 (0.0008) [2023-12-27 18:32:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 953237504. Throughput: 0: 10288.4, 1: 10306.2. Samples: 953226796. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:32:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.170')] [2023-12-27 18:32:57,373][20381] Updated weights for policy 1, policy_version 1860743 (0.0006) [2023-12-27 18:32:57,430][20381] Updated weights for policy 1, policy_version 1860753 (0.0005) [2023-12-27 18:32:57,733][20347] Updated weights for policy 0, policy_version 1862331 (0.0007) [2023-12-27 18:32:57,777][20347] Updated weights for policy 0, policy_version 1862341 (0.0007) [2023-12-27 18:32:57,830][20347] Updated weights for policy 0, policy_version 1862351 (0.0007) [2023-12-27 18:32:58,113][20381] Updated weights for policy 1, policy_version 1860763 (0.0006) [2023-12-27 18:32:58,171][20381] Updated weights for policy 1, policy_version 1860773 (0.0007) [2023-12-27 18:32:58,235][20381] Updated weights for policy 1, policy_version 1860783 (0.0008) [2023-12-27 18:32:58,450][20347] Updated weights for policy 0, policy_version 1862361 (0.0005) [2023-12-27 18:32:58,505][20347] Updated weights for policy 0, policy_version 1862371 (0.0007) [2023-12-27 18:32:58,566][20347] Updated weights for policy 0, policy_version 1862381 (0.0008) [2023-12-27 18:32:58,629][20347] Updated weights for policy 0, policy_version 1862391 (0.0008) [2023-12-27 18:32:59,044][20381] Updated weights for policy 1, policy_version 1860793 (0.0007) [2023-12-27 18:32:59,093][20381] Updated weights for policy 1, policy_version 1860803 (0.0008) [2023-12-27 18:32:59,145][20381] Updated weights for policy 1, policy_version 1860813 (0.0008) [2023-12-27 18:32:59,203][20381] Updated weights for policy 1, policy_version 1860823 (0.0008) [2023-12-27 18:32:59,440][20347] Updated weights for policy 0, policy_version 1862401 (0.0008) [2023-12-27 18:32:59,501][20347] Updated weights for policy 0, policy_version 1862411 (0.0008) [2023-12-27 18:32:59,562][20347] Updated weights for policy 0, policy_version 1862421 (0.0010) [2023-12-27 18:32:59,916][20381] Updated weights for policy 1, policy_version 1860833 (0.0008) [2023-12-27 18:32:59,971][20381] Updated weights for policy 1, policy_version 1860843 (0.0007) [2023-12-27 18:33:00,026][20381] Updated weights for policy 1, policy_version 1860853 (0.0008) [2023-12-27 18:33:00,250][20347] Updated weights for policy 0, policy_version 1862431 (0.0011) [2023-12-27 18:33:00,311][20347] Updated weights for policy 0, policy_version 1862441 (0.0010) [2023-12-27 18:33:00,370][20347] Updated weights for policy 0, policy_version 1862451 (0.0011) [2023-12-27 18:33:00,699][20381] Updated weights for policy 1, policy_version 1860863 (0.0008) [2023-12-27 18:33:00,752][20381] Updated weights for policy 1, policy_version 1860873 (0.0007) [2023-12-27 18:33:00,799][20381] Updated weights for policy 1, policy_version 1860883 (0.0008) [2023-12-27 18:33:00,922][20347] Updated weights for policy 0, policy_version 1862461 (0.0008) [2023-12-27 18:33:00,970][20347] Updated weights for policy 0, policy_version 1862471 (0.0005) [2023-12-27 18:33:01,017][20347] Updated weights for policy 0, policy_version 1862481 (0.0005) [2023-12-27 18:33:01,551][20381] Updated weights for policy 1, policy_version 1860893 (0.0007) [2023-12-27 18:33:01,600][20381] Updated weights for policy 1, policy_version 1860903 (0.0007) [2023-12-27 18:33:01,666][20381] Updated weights for policy 1, policy_version 1860913 (0.0008) [2023-12-27 18:33:01,683][20347] Updated weights for policy 0, policy_version 1862491 (0.0006) [2023-12-27 18:33:01,744][20347] Updated weights for policy 0, policy_version 1862501 (0.0008) [2023-12-27 18:33:01,796][20347] Updated weights for policy 0, policy_version 1862511 (0.0007) [2023-12-27 18:33:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 953335808. Throughput: 0: 10321.6, 1: 10288.8. Samples: 953348716. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '779.710')] [2023-12-27 18:33:02,372][20381] Updated weights for policy 1, policy_version 1860923 (0.0008) [2023-12-27 18:33:02,435][20381] Updated weights for policy 1, policy_version 1860933 (0.0007) [2023-12-27 18:33:02,470][20347] Updated weights for policy 0, policy_version 1862521 (0.0010) [2023-12-27 18:33:02,503][20381] Updated weights for policy 1, policy_version 1860943 (0.0005) [2023-12-27 18:33:02,526][20347] Updated weights for policy 0, policy_version 1862531 (0.0007) [2023-12-27 18:33:02,582][20347] Updated weights for policy 0, policy_version 1862541 (0.0007) [2023-12-27 18:33:02,638][20347] Updated weights for policy 0, policy_version 1862551 (0.0008) [2023-12-27 18:33:03,116][20381] Updated weights for policy 1, policy_version 1860953 (0.0006) [2023-12-27 18:33:03,173][20381] Updated weights for policy 1, policy_version 1860963 (0.0008) [2023-12-27 18:33:03,228][20381] Updated weights for policy 1, policy_version 1860973 (0.0008) [2023-12-27 18:33:03,267][20347] Updated weights for policy 0, policy_version 1862561 (0.0007) [2023-12-27 18:33:03,280][20381] Updated weights for policy 1, policy_version 1860983 (0.0006) [2023-12-27 18:33:03,320][20347] Updated weights for policy 0, policy_version 1862571 (0.0006) [2023-12-27 18:33:03,375][20347] Updated weights for policy 0, policy_version 1862581 (0.0009) [2023-12-27 18:33:03,982][20347] Updated weights for policy 0, policy_version 1862591 (0.0010) [2023-12-27 18:33:04,027][20381] Updated weights for policy 1, policy_version 1860993 (0.0008) [2023-12-27 18:33:04,041][20347] Updated weights for policy 0, policy_version 1862601 (0.0010) [2023-12-27 18:33:04,086][20381] Updated weights for policy 1, policy_version 1861003 (0.0010) [2023-12-27 18:33:04,097][20347] Updated weights for policy 0, policy_version 1862611 (0.0010) [2023-12-27 18:33:04,143][20381] Updated weights for policy 1, policy_version 1861013 (0.0008) [2023-12-27 18:33:04,773][20347] Updated weights for policy 0, policy_version 1862621 (0.0009) [2023-12-27 18:33:04,820][20347] Updated weights for policy 0, policy_version 1862631 (0.0007) [2023-12-27 18:33:04,845][20381] Updated weights for policy 1, policy_version 1861023 (0.0011) [2023-12-27 18:33:04,869][20347] Updated weights for policy 0, policy_version 1862641 (0.0005) [2023-12-27 18:33:04,901][20381] Updated weights for policy 1, policy_version 1861033 (0.0010) [2023-12-27 18:33:04,952][20381] Updated weights for policy 1, policy_version 1861043 (0.0009) [2023-12-27 18:33:05,490][20347] Updated weights for policy 0, policy_version 1862651 (0.0005) [2023-12-27 18:33:05,541][20347] Updated weights for policy 0, policy_version 1862661 (0.0005) [2023-12-27 18:33:05,599][20347] Updated weights for policy 0, policy_version 1862671 (0.0006) [2023-12-27 18:33:05,607][20381] Updated weights for policy 1, policy_version 1861053 (0.0009) [2023-12-27 18:33:05,661][20381] Updated weights for policy 1, policy_version 1861063 (0.0006) [2023-12-27 18:33:05,721][20381] Updated weights for policy 1, policy_version 1861073 (0.0007) [2023-12-27 18:33:06,258][20347] Updated weights for policy 0, policy_version 1862681 (0.0007) [2023-12-27 18:33:06,297][20381] Updated weights for policy 1, policy_version 1861083 (0.0006) [2023-12-27 18:33:06,319][20347] Updated weights for policy 0, policy_version 1862691 (0.0007) [2023-12-27 18:33:06,352][20381] Updated weights for policy 1, policy_version 1861093 (0.0006) [2023-12-27 18:33:06,377][20347] Updated weights for policy 0, policy_version 1862701 (0.0006) [2023-12-27 18:33:06,412][20381] Updated weights for policy 1, policy_version 1861103 (0.0006) [2023-12-27 18:33:06,433][20347] Updated weights for policy 0, policy_version 1862711 (0.0007) [2023-12-27 18:33:07,121][20347] Updated weights for policy 0, policy_version 1862721 (0.0006) [2023-12-27 18:33:07,129][20381] Updated weights for policy 1, policy_version 1861113 (0.0006) [2023-12-27 18:33:07,181][20347] Updated weights for policy 0, policy_version 1862731 (0.0005) [2023-12-27 18:33:07,183][20381] Updated weights for policy 1, policy_version 1861123 (0.0008) [2023-12-27 18:33:07,240][20381] Updated weights for policy 1, policy_version 1861133 (0.0007) [2023-12-27 18:33:07,241][20347] Updated weights for policy 0, policy_version 1862741 (0.0005) [2023-12-27 18:33:07,299][20381] Updated weights for policy 1, policy_version 1861143 (0.0007) [2023-12-27 18:33:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 953450496. Throughput: 0: 10362.5, 1: 10271.3. Samples: 953412120. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '840.210')] [2023-12-27 18:33:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001862744_476946432.pth... [2023-12-27 18:33:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001861144_476504064.pth... [2023-12-27 18:33:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001861496_476626944.pth [2023-12-27 18:33:07,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001859960_476200960.pth [2023-12-27 18:33:07,867][20347] Updated weights for policy 0, policy_version 1862751 (0.0007) [2023-12-27 18:33:07,929][20347] Updated weights for policy 0, policy_version 1862761 (0.0005) [2023-12-27 18:33:07,970][20381] Updated weights for policy 1, policy_version 1861153 (0.0007) [2023-12-27 18:33:07,983][20347] Updated weights for policy 0, policy_version 1862771 (0.0006) [2023-12-27 18:33:08,023][20381] Updated weights for policy 1, policy_version 1861163 (0.0007) [2023-12-27 18:33:08,085][20381] Updated weights for policy 1, policy_version 1861173 (0.0007) [2023-12-27 18:33:08,582][20347] Updated weights for policy 0, policy_version 1862781 (0.0006) [2023-12-27 18:33:08,638][20347] Updated weights for policy 0, policy_version 1862791 (0.0008) [2023-12-27 18:33:08,690][20347] Updated weights for policy 0, policy_version 1862801 (0.0007) [2023-12-27 18:33:08,734][20381] Updated weights for policy 1, policy_version 1861183 (0.0009) [2023-12-27 18:33:08,789][20381] Updated weights for policy 1, policy_version 1861193 (0.0007) [2023-12-27 18:33:08,848][20381] Updated weights for policy 1, policy_version 1861203 (0.0008) [2023-12-27 18:33:09,310][20347] Updated weights for policy 0, policy_version 1862811 (0.0008) [2023-12-27 18:33:09,378][20347] Updated weights for policy 0, policy_version 1862821 (0.0007) [2023-12-27 18:33:09,443][20347] Updated weights for policy 0, policy_version 1862831 (0.0008) [2023-12-27 18:33:09,518][20381] Updated weights for policy 1, policy_version 1861213 (0.0008) [2023-12-27 18:33:09,573][20381] Updated weights for policy 1, policy_version 1861223 (0.0008) [2023-12-27 18:33:09,630][20381] Updated weights for policy 1, policy_version 1861233 (0.0006) [2023-12-27 18:33:10,166][20347] Updated weights for policy 0, policy_version 1862841 (0.0008) [2023-12-27 18:33:10,230][20347] Updated weights for policy 0, policy_version 1862851 (0.0006) [2023-12-27 18:33:10,278][20381] Updated weights for policy 1, policy_version 1861243 (0.0008) [2023-12-27 18:33:10,286][20347] Updated weights for policy 0, policy_version 1862861 (0.0005) [2023-12-27 18:33:10,337][20381] Updated weights for policy 1, policy_version 1861253 (0.0008) [2023-12-27 18:33:10,343][20347] Updated weights for policy 0, policy_version 1862871 (0.0006) [2023-12-27 18:33:10,395][20381] Updated weights for policy 1, policy_version 1861263 (0.0008) [2023-12-27 18:33:10,913][20347] Updated weights for policy 0, policy_version 1862881 (0.0008) [2023-12-27 18:33:10,973][20347] Updated weights for policy 0, policy_version 1862891 (0.0008) [2023-12-27 18:33:11,031][20347] Updated weights for policy 0, policy_version 1862901 (0.0008) [2023-12-27 18:33:11,078][20381] Updated weights for policy 1, policy_version 1861273 (0.0008) [2023-12-27 18:33:11,136][20381] Updated weights for policy 1, policy_version 1861283 (0.0008) [2023-12-27 18:33:11,204][20381] Updated weights for policy 1, policy_version 1861293 (0.0008) [2023-12-27 18:33:11,267][20381] Updated weights for policy 1, policy_version 1861303 (0.0008) [2023-12-27 18:33:11,806][20347] Updated weights for policy 0, policy_version 1862911 (0.0008) [2023-12-27 18:33:11,861][20347] Updated weights for policy 0, policy_version 1862921 (0.0008) [2023-12-27 18:33:11,918][20347] Updated weights for policy 0, policy_version 1862931 (0.0009) [2023-12-27 18:33:11,990][20381] Updated weights for policy 1, policy_version 1861313 (0.0006) [2023-12-27 18:33:12,055][20381] Updated weights for policy 1, policy_version 1861323 (0.0006) [2023-12-27 18:33:12,119][20381] Updated weights for policy 1, policy_version 1861333 (0.0006) [2023-12-27 18:33:12,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 953548800. Throughput: 0: 10480.1, 1: 10259.0. Samples: 953538280. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '860.360')] [2023-12-27 18:33:12,580][20347] Updated weights for policy 0, policy_version 1862941 (0.0010) [2023-12-27 18:33:12,636][20347] Updated weights for policy 0, policy_version 1862951 (0.0010) [2023-12-27 18:33:12,697][20347] Updated weights for policy 0, policy_version 1862961 (0.0009) [2023-12-27 18:33:12,720][20381] Updated weights for policy 1, policy_version 1861343 (0.0007) [2023-12-27 18:33:12,774][20381] Updated weights for policy 1, policy_version 1861353 (0.0007) [2023-12-27 18:33:12,832][20381] Updated weights for policy 1, policy_version 1861363 (0.0007) [2023-12-27 18:33:13,373][20347] Updated weights for policy 0, policy_version 1862971 (0.0008) [2023-12-27 18:33:13,442][20347] Updated weights for policy 0, policy_version 1862981 (0.0005) [2023-12-27 18:33:13,473][20381] Updated weights for policy 1, policy_version 1861373 (0.0007) [2023-12-27 18:33:13,498][20347] Updated weights for policy 0, policy_version 1862991 (0.0006) [2023-12-27 18:33:13,518][20381] Updated weights for policy 1, policy_version 1861383 (0.0007) [2023-12-27 18:33:13,574][20381] Updated weights for policy 1, policy_version 1861393 (0.0007) [2023-12-27 18:33:14,133][20347] Updated weights for policy 0, policy_version 1863001 (0.0007) [2023-12-27 18:33:14,182][20347] Updated weights for policy 0, policy_version 1863011 (0.0008) [2023-12-27 18:33:14,219][20381] Updated weights for policy 1, policy_version 1861403 (0.0008) [2023-12-27 18:33:14,232][20347] Updated weights for policy 0, policy_version 1863021 (0.0006) [2023-12-27 18:33:14,275][20381] Updated weights for policy 1, policy_version 1861413 (0.0007) [2023-12-27 18:33:14,278][20347] Updated weights for policy 0, policy_version 1863031 (0.0005) [2023-12-27 18:33:14,336][20381] Updated weights for policy 1, policy_version 1861423 (0.0009) [2023-12-27 18:33:15,003][20347] Updated weights for policy 0, policy_version 1863041 (0.0007) [2023-12-27 18:33:15,069][20347] Updated weights for policy 0, policy_version 1863051 (0.0007) [2023-12-27 18:33:15,108][20381] Updated weights for policy 1, policy_version 1861433 (0.0009) [2023-12-27 18:33:15,134][20347] Updated weights for policy 0, policy_version 1863061 (0.0007) [2023-12-27 18:33:15,166][20381] Updated weights for policy 1, policy_version 1861443 (0.0008) [2023-12-27 18:33:15,226][20381] Updated weights for policy 1, policy_version 1861453 (0.0007) [2023-12-27 18:33:15,291][20381] Updated weights for policy 1, policy_version 1861463 (0.0006) [2023-12-27 18:33:15,776][20347] Updated weights for policy 0, policy_version 1863071 (0.0010) [2023-12-27 18:33:15,834][20347] Updated weights for policy 0, policy_version 1863081 (0.0010) [2023-12-27 18:33:15,880][20381] Updated weights for policy 1, policy_version 1861473 (0.0007) [2023-12-27 18:33:15,890][20347] Updated weights for policy 0, policy_version 1863091 (0.0009) [2023-12-27 18:33:15,939][20381] Updated weights for policy 1, policy_version 1861483 (0.0006) [2023-12-27 18:33:15,997][20381] Updated weights for policy 1, policy_version 1861493 (0.0007) [2023-12-27 18:33:16,549][20347] Updated weights for policy 0, policy_version 1863101 (0.0010) [2023-12-27 18:33:16,616][20347] Updated weights for policy 0, policy_version 1863111 (0.0010) [2023-12-27 18:33:16,673][20347] Updated weights for policy 0, policy_version 1863121 (0.0005) [2023-12-27 18:33:16,676][20381] Updated weights for policy 1, policy_version 1861503 (0.0005) [2023-12-27 18:33:16,733][20381] Updated weights for policy 1, policy_version 1861513 (0.0005) [2023-12-27 18:33:16,786][20381] Updated weights for policy 1, policy_version 1861523 (0.0005) [2023-12-27 18:33:17,275][20347] Updated weights for policy 0, policy_version 1863131 (0.0007) [2023-12-27 18:33:17,326][20347] Updated weights for policy 0, policy_version 1863141 (0.0010) [2023-12-27 18:33:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 953647104. Throughput: 0: 10467.3, 1: 10282.0. Samples: 953661980. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.240')] [2023-12-27 18:33:17,375][20347] Updated weights for policy 0, policy_version 1863151 (0.0010) [2023-12-27 18:33:17,435][20381] Updated weights for policy 1, policy_version 1861533 (0.0009) [2023-12-27 18:33:17,495][20381] Updated weights for policy 1, policy_version 1861543 (0.0006) [2023-12-27 18:33:17,541][20381] Updated weights for policy 1, policy_version 1861553 (0.0005) [2023-12-27 18:33:18,089][20347] Updated weights for policy 0, policy_version 1863161 (0.0010) [2023-12-27 18:33:18,146][20347] Updated weights for policy 0, policy_version 1863171 (0.0006) [2023-12-27 18:33:18,162][20381] Updated weights for policy 1, policy_version 1861563 (0.0007) [2023-12-27 18:33:18,197][20347] Updated weights for policy 0, policy_version 1863181 (0.0010) [2023-12-27 18:33:18,217][20381] Updated weights for policy 1, policy_version 1861573 (0.0011) [2023-12-27 18:33:18,246][20347] Updated weights for policy 0, policy_version 1863191 (0.0010) [2023-12-27 18:33:18,266][20381] Updated weights for policy 1, policy_version 1861583 (0.0010) [2023-12-27 18:33:18,887][20347] Updated weights for policy 0, policy_version 1863201 (0.0009) [2023-12-27 18:33:18,939][20347] Updated weights for policy 0, policy_version 1863211 (0.0009) [2023-12-27 18:33:18,976][20381] Updated weights for policy 1, policy_version 1861593 (0.0011) [2023-12-27 18:33:18,990][20347] Updated weights for policy 0, policy_version 1863221 (0.0010) [2023-12-27 18:33:19,021][20381] Updated weights for policy 1, policy_version 1861603 (0.0010) [2023-12-27 18:33:19,066][20381] Updated weights for policy 1, policy_version 1861613 (0.0010) [2023-12-27 18:33:19,106][20381] Updated weights for policy 1, policy_version 1861623 (0.0009) [2023-12-27 18:33:19,795][20347] Updated weights for policy 0, policy_version 1863231 (0.0007) [2023-12-27 18:33:19,803][20381] Updated weights for policy 1, policy_version 1861633 (0.0007) [2023-12-27 18:33:19,866][20347] Updated weights for policy 0, policy_version 1863241 (0.0007) [2023-12-27 18:33:19,867][20381] Updated weights for policy 1, policy_version 1861643 (0.0007) [2023-12-27 18:33:19,928][20347] Updated weights for policy 0, policy_version 1863251 (0.0008) [2023-12-27 18:33:19,932][20381] Updated weights for policy 1, policy_version 1861653 (0.0008) [2023-12-27 18:33:20,575][20347] Updated weights for policy 0, policy_version 1863261 (0.0008) [2023-12-27 18:33:20,609][20381] Updated weights for policy 1, policy_version 1861663 (0.0008) [2023-12-27 18:33:20,634][20347] Updated weights for policy 0, policy_version 1863271 (0.0009) [2023-12-27 18:33:20,661][20381] Updated weights for policy 1, policy_version 1861673 (0.0007) [2023-12-27 18:33:20,689][20347] Updated weights for policy 0, policy_version 1863281 (0.0008) [2023-12-27 18:33:20,712][20381] Updated weights for policy 1, policy_version 1861683 (0.0007) [2023-12-27 18:33:21,391][20381] Updated weights for policy 1, policy_version 1861693 (0.0008) [2023-12-27 18:33:21,444][20381] Updated weights for policy 1, policy_version 1861703 (0.0007) [2023-12-27 18:33:21,488][20347] Updated weights for policy 0, policy_version 1863291 (0.0007) [2023-12-27 18:33:21,501][20381] Updated weights for policy 1, policy_version 1861713 (0.0006) [2023-12-27 18:33:21,560][20347] Updated weights for policy 0, policy_version 1863301 (0.0007) [2023-12-27 18:33:21,627][20347] Updated weights for policy 0, policy_version 1863311 (0.0008) [2023-12-27 18:33:22,214][20381] Updated weights for policy 1, policy_version 1861723 (0.0006) [2023-12-27 18:33:22,236][20347] Updated weights for policy 0, policy_version 1863321 (0.0007) [2023-12-27 18:33:22,270][20381] Updated weights for policy 1, policy_version 1861733 (0.0008) [2023-12-27 18:33:22,295][20347] Updated weights for policy 0, policy_version 1863331 (0.0006) [2023-12-27 18:33:22,334][20381] Updated weights for policy 1, policy_version 1861743 (0.0007) [2023-12-27 18:33:22,346][20347] Updated weights for policy 0, policy_version 1863341 (0.0007) [2023-12-27 18:33:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 953745408. Throughput: 0: 10452.0, 1: 10223.1. Samples: 953722536. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.700')] [2023-12-27 18:33:22,391][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001861752_476659712.pth... [2023-12-27 18:33:22,394][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001860536_476348416.pth [2023-12-27 18:33:22,411][20347] Updated weights for policy 0, policy_version 1863351 (0.0007) [2023-12-27 18:33:22,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001863352_477102080.pth... [2023-12-27 18:33:22,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001862136_476790784.pth [2023-12-27 18:33:23,010][20381] Updated weights for policy 1, policy_version 1861753 (0.0008) [2023-12-27 18:33:23,052][20347] Updated weights for policy 0, policy_version 1863361 (0.0008) [2023-12-27 18:33:23,065][20381] Updated weights for policy 1, policy_version 1861763 (0.0006) [2023-12-27 18:33:23,108][20347] Updated weights for policy 0, policy_version 1863371 (0.0005) [2023-12-27 18:33:23,123][20381] Updated weights for policy 1, policy_version 1861773 (0.0006) [2023-12-27 18:33:23,162][20347] Updated weights for policy 0, policy_version 1863381 (0.0005) [2023-12-27 18:33:23,184][20381] Updated weights for policy 1, policy_version 1861783 (0.0007) [2023-12-27 18:33:23,763][20381] Updated weights for policy 1, policy_version 1861793 (0.0007) [2023-12-27 18:33:23,809][20381] Updated weights for policy 1, policy_version 1861803 (0.0008) [2023-12-27 18:33:23,815][20347] Updated weights for policy 0, policy_version 1863391 (0.0006) [2023-12-27 18:33:23,862][20381] Updated weights for policy 1, policy_version 1861813 (0.0006) [2023-12-27 18:33:23,871][20347] Updated weights for policy 0, policy_version 1863401 (0.0006) [2023-12-27 18:33:23,925][20347] Updated weights for policy 0, policy_version 1863411 (0.0007) [2023-12-27 18:33:24,473][20381] Updated weights for policy 1, policy_version 1861823 (0.0006) [2023-12-27 18:33:24,519][20381] Updated weights for policy 1, policy_version 1861833 (0.0005) [2023-12-27 18:33:24,571][20381] Updated weights for policy 1, policy_version 1861843 (0.0006) [2023-12-27 18:33:24,639][20347] Updated weights for policy 0, policy_version 1863421 (0.0008) [2023-12-27 18:33:24,693][20347] Updated weights for policy 0, policy_version 1863431 (0.0009) [2023-12-27 18:33:24,746][20347] Updated weights for policy 0, policy_version 1863441 (0.0008) [2023-12-27 18:33:25,173][20381] Updated weights for policy 1, policy_version 1861853 (0.0006) [2023-12-27 18:33:25,240][20381] Updated weights for policy 1, policy_version 1861863 (0.0006) [2023-12-27 18:33:25,308][20381] Updated weights for policy 1, policy_version 1861873 (0.0006) [2023-12-27 18:33:25,527][20347] Updated weights for policy 0, policy_version 1863451 (0.0007) [2023-12-27 18:33:25,584][20347] Updated weights for policy 0, policy_version 1863461 (0.0007) [2023-12-27 18:33:25,636][20347] Updated weights for policy 0, policy_version 1863471 (0.0008) [2023-12-27 18:33:25,991][20381] Updated weights for policy 1, policy_version 1861883 (0.0009) [2023-12-27 18:33:26,042][20381] Updated weights for policy 1, policy_version 1861893 (0.0010) [2023-12-27 18:33:26,084][20381] Updated weights for policy 1, policy_version 1861903 (0.0010) [2023-12-27 18:33:26,339][20347] Updated weights for policy 0, policy_version 1863481 (0.0007) [2023-12-27 18:33:26,394][20347] Updated weights for policy 0, policy_version 1863491 (0.0007) [2023-12-27 18:33:26,457][20347] Updated weights for policy 0, policy_version 1863501 (0.0007) [2023-12-27 18:33:26,515][20347] Updated weights for policy 0, policy_version 1863511 (0.0006) [2023-12-27 18:33:26,785][20381] Updated weights for policy 1, policy_version 1861913 (0.0010) [2023-12-27 18:33:26,833][20381] Updated weights for policy 1, policy_version 1861923 (0.0010) [2023-12-27 18:33:26,878][20381] Updated weights for policy 1, policy_version 1861933 (0.0010) [2023-12-27 18:33:26,926][20381] Updated weights for policy 1, policy_version 1861943 (0.0010) [2023-12-27 18:33:27,198][20347] Updated weights for policy 0, policy_version 1863521 (0.0007) [2023-12-27 18:33:27,247][20347] Updated weights for policy 0, policy_version 1863531 (0.0005) [2023-12-27 18:33:27,307][20347] Updated weights for policy 0, policy_version 1863541 (0.0006) [2023-12-27 18:33:27,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 953860096. Throughput: 0: 10399.0, 1: 10318.1. Samples: 953846140. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '852.850')] [2023-12-27 18:33:27,543][20381] Updated weights for policy 1, policy_version 1861953 (0.0010) [2023-12-27 18:33:27,597][20381] Updated weights for policy 1, policy_version 1861963 (0.0010) [2023-12-27 18:33:27,653][20381] Updated weights for policy 1, policy_version 1861973 (0.0010) [2023-12-27 18:33:27,957][20347] Updated weights for policy 0, policy_version 1863551 (0.0007) [2023-12-27 18:33:28,004][20347] Updated weights for policy 0, policy_version 1863561 (0.0007) [2023-12-27 18:33:28,048][20347] Updated weights for policy 0, policy_version 1863571 (0.0007) [2023-12-27 18:33:28,342][20381] Updated weights for policy 1, policy_version 1861983 (0.0010) [2023-12-27 18:33:28,407][20381] Updated weights for policy 1, policy_version 1861993 (0.0008) [2023-12-27 18:33:28,466][20381] Updated weights for policy 1, policy_version 1862003 (0.0008) [2023-12-27 18:33:28,676][20347] Updated weights for policy 0, policy_version 1863581 (0.0007) [2023-12-27 18:33:28,728][20347] Updated weights for policy 0, policy_version 1863591 (0.0009) [2023-12-27 18:33:28,790][20347] Updated weights for policy 0, policy_version 1863601 (0.0008) [2023-12-27 18:33:29,064][20381] Updated weights for policy 1, policy_version 1862013 (0.0007) [2023-12-27 18:33:29,124][20381] Updated weights for policy 1, policy_version 1862023 (0.0008) [2023-12-27 18:33:29,186][20381] Updated weights for policy 1, policy_version 1862033 (0.0007) [2023-12-27 18:33:29,504][20347] Updated weights for policy 0, policy_version 1863611 (0.0009) [2023-12-27 18:33:29,567][20347] Updated weights for policy 0, policy_version 1863621 (0.0011) [2023-12-27 18:33:29,641][20347] Updated weights for policy 0, policy_version 1863631 (0.0010) [2023-12-27 18:33:29,846][20381] Updated weights for policy 1, policy_version 1862043 (0.0007) [2023-12-27 18:33:29,904][20381] Updated weights for policy 1, policy_version 1862053 (0.0005) [2023-12-27 18:33:29,967][20381] Updated weights for policy 1, policy_version 1862063 (0.0006) [2023-12-27 18:33:30,369][20347] Updated weights for policy 0, policy_version 1863641 (0.0010) [2023-12-27 18:33:30,426][20347] Updated weights for policy 0, policy_version 1863651 (0.0008) [2023-12-27 18:33:30,482][20347] Updated weights for policy 0, policy_version 1863661 (0.0007) [2023-12-27 18:33:30,538][20347] Updated weights for policy 0, policy_version 1863671 (0.0008) [2023-12-27 18:33:30,565][20381] Updated weights for policy 1, policy_version 1862073 (0.0007) [2023-12-27 18:33:30,623][20381] Updated weights for policy 1, policy_version 1862083 (0.0008) [2023-12-27 18:33:30,682][20381] Updated weights for policy 1, policy_version 1862093 (0.0008) [2023-12-27 18:33:30,734][20381] Updated weights for policy 1, policy_version 1862103 (0.0008) [2023-12-27 18:33:31,271][20347] Updated weights for policy 0, policy_version 1863681 (0.0008) [2023-12-27 18:33:31,324][20347] Updated weights for policy 0, policy_version 1863691 (0.0006) [2023-12-27 18:33:31,390][20347] Updated weights for policy 0, policy_version 1863701 (0.0007) [2023-12-27 18:33:31,480][20381] Updated weights for policy 1, policy_version 1862113 (0.0008) [2023-12-27 18:33:31,540][20381] Updated weights for policy 1, policy_version 1862123 (0.0007) [2023-12-27 18:33:31,604][20381] Updated weights for policy 1, policy_version 1862133 (0.0008) [2023-12-27 18:33:32,133][20347] Updated weights for policy 0, policy_version 1863711 (0.0009) [2023-12-27 18:33:32,187][20347] Updated weights for policy 0, policy_version 1863721 (0.0006) [2023-12-27 18:33:32,238][20347] Updated weights for policy 0, policy_version 1863731 (0.0007) [2023-12-27 18:33:32,294][20381] Updated weights for policy 1, policy_version 1862143 (0.0007) [2023-12-27 18:33:32,352][20381] Updated weights for policy 1, policy_version 1862153 (0.0007) [2023-12-27 18:33:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 953958400. Throughput: 0: 10331.5, 1: 10337.9. Samples: 953969160. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '802.670')] [2023-12-27 18:33:32,413][20381] Updated weights for policy 1, policy_version 1862163 (0.0008) [2023-12-27 18:33:32,945][20347] Updated weights for policy 0, policy_version 1863741 (0.0007) [2023-12-27 18:33:32,999][20347] Updated weights for policy 0, policy_version 1863751 (0.0006) [2023-12-27 18:33:33,047][20347] Updated weights for policy 0, policy_version 1863761 (0.0007) [2023-12-27 18:33:33,080][20381] Updated weights for policy 1, policy_version 1862173 (0.0007) [2023-12-27 18:33:33,125][20381] Updated weights for policy 1, policy_version 1862183 (0.0008) [2023-12-27 18:33:33,184][20381] Updated weights for policy 1, policy_version 1862193 (0.0008) [2023-12-27 18:33:33,609][20347] Updated weights for policy 0, policy_version 1863771 (0.0006) [2023-12-27 18:33:33,657][20347] Updated weights for policy 0, policy_version 1863781 (0.0008) [2023-12-27 18:33:33,707][20347] Updated weights for policy 0, policy_version 1863791 (0.0007) [2023-12-27 18:33:33,922][20381] Updated weights for policy 1, policy_version 1862203 (0.0007) [2023-12-27 18:33:33,968][20381] Updated weights for policy 1, policy_version 1862213 (0.0008) [2023-12-27 18:33:34,013][20381] Updated weights for policy 1, policy_version 1862223 (0.0008) [2023-12-27 18:33:34,355][20347] Updated weights for policy 0, policy_version 1863801 (0.0007) [2023-12-27 18:33:34,419][20347] Updated weights for policy 0, policy_version 1863811 (0.0005) [2023-12-27 18:33:34,480][20347] Updated weights for policy 0, policy_version 1863821 (0.0006) [2023-12-27 18:33:34,538][20347] Updated weights for policy 0, policy_version 1863831 (0.0005) [2023-12-27 18:33:34,785][20381] Updated weights for policy 1, policy_version 1862233 (0.0008) [2023-12-27 18:33:34,839][20381] Updated weights for policy 1, policy_version 1862243 (0.0008) [2023-12-27 18:33:34,885][20381] Updated weights for policy 1, policy_version 1862253 (0.0008) [2023-12-27 18:33:34,941][20381] Updated weights for policy 1, policy_version 1862263 (0.0007) [2023-12-27 18:33:35,116][20347] Updated weights for policy 0, policy_version 1863841 (0.0006) [2023-12-27 18:33:35,162][20347] Updated weights for policy 0, policy_version 1863851 (0.0007) [2023-12-27 18:33:35,215][20347] Updated weights for policy 0, policy_version 1863861 (0.0007) [2023-12-27 18:33:35,566][20381] Updated weights for policy 1, policy_version 1862273 (0.0010) [2023-12-27 18:33:35,615][20381] Updated weights for policy 1, policy_version 1862283 (0.0010) [2023-12-27 18:33:35,670][20381] Updated weights for policy 1, policy_version 1862293 (0.0010) [2023-12-27 18:33:35,886][20347] Updated weights for policy 0, policy_version 1863871 (0.0007) [2023-12-27 18:33:35,943][20347] Updated weights for policy 0, policy_version 1863881 (0.0008) [2023-12-27 18:33:36,000][20347] Updated weights for policy 0, policy_version 1863891 (0.0007) [2023-12-27 18:33:36,350][20381] Updated weights for policy 1, policy_version 1862303 (0.0011) [2023-12-27 18:33:36,400][20381] Updated weights for policy 1, policy_version 1862313 (0.0011) [2023-12-27 18:33:36,464][20381] Updated weights for policy 1, policy_version 1862323 (0.0011) [2023-12-27 18:33:36,689][20347] Updated weights for policy 0, policy_version 1863901 (0.0006) [2023-12-27 18:33:36,748][20347] Updated weights for policy 0, policy_version 1863911 (0.0007) [2023-12-27 18:33:36,815][20347] Updated weights for policy 0, policy_version 1863921 (0.0007) [2023-12-27 18:33:37,218][20381] Updated weights for policy 1, policy_version 1862333 (0.0011) [2023-12-27 18:33:37,273][20381] Updated weights for policy 1, policy_version 1862343 (0.0011) [2023-12-27 18:33:37,329][20381] Updated weights for policy 1, policy_version 1862353 (0.0011) [2023-12-27 18:33:37,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.5, 300 sec: 20438.4). Total num frames: 954056704. Throughput: 0: 10342.4, 1: 10323.9. Samples: 954031780. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '841.920')] [2023-12-27 18:33:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001863928_477249536.pth... [2023-12-27 18:33:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001862744_476946432.pth [2023-12-27 18:33:37,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001862360_476815360.pth... [2023-12-27 18:33:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001861144_476504064.pth [2023-12-27 18:33:37,450][20347] Updated weights for policy 0, policy_version 1863931 (0.0006) [2023-12-27 18:33:37,502][20347] Updated weights for policy 0, policy_version 1863941 (0.0007) [2023-12-27 18:33:37,560][20347] Updated weights for policy 0, policy_version 1863951 (0.0007) [2023-12-27 18:33:38,024][20381] Updated weights for policy 1, policy_version 1862363 (0.0011) [2023-12-27 18:33:38,073][20381] Updated weights for policy 1, policy_version 1862373 (0.0011) [2023-12-27 18:33:38,122][20381] Updated weights for policy 1, policy_version 1862383 (0.0010) [2023-12-27 18:33:38,258][20347] Updated weights for policy 0, policy_version 1863961 (0.0008) [2023-12-27 18:33:38,321][20347] Updated weights for policy 0, policy_version 1863971 (0.0007) [2023-12-27 18:33:38,378][20347] Updated weights for policy 0, policy_version 1863981 (0.0008) [2023-12-27 18:33:38,426][20347] Updated weights for policy 0, policy_version 1863991 (0.0008) [2023-12-27 18:33:38,850][20381] Updated weights for policy 1, policy_version 1862393 (0.0010) [2023-12-27 18:33:38,906][20381] Updated weights for policy 1, policy_version 1862403 (0.0007) [2023-12-27 18:33:38,953][20381] Updated weights for policy 1, policy_version 1862413 (0.0005) [2023-12-27 18:33:39,001][20381] Updated weights for policy 1, policy_version 1862423 (0.0005) [2023-12-27 18:33:39,152][20347] Updated weights for policy 0, policy_version 1864001 (0.0008) [2023-12-27 18:33:39,213][20347] Updated weights for policy 0, policy_version 1864011 (0.0008) [2023-12-27 18:33:39,276][20347] Updated weights for policy 0, policy_version 1864021 (0.0009) [2023-12-27 18:33:39,642][20381] Updated weights for policy 1, policy_version 1862433 (0.0007) [2023-12-27 18:33:39,700][20381] Updated weights for policy 1, policy_version 1862443 (0.0008) [2023-12-27 18:33:39,762][20381] Updated weights for policy 1, policy_version 1862453 (0.0008) [2023-12-27 18:33:40,033][20347] Updated weights for policy 0, policy_version 1864031 (0.0008) [2023-12-27 18:33:40,092][20347] Updated weights for policy 0, policy_version 1864041 (0.0006) [2023-12-27 18:33:40,140][20347] Updated weights for policy 0, policy_version 1864051 (0.0005) [2023-12-27 18:33:40,484][20381] Updated weights for policy 1, policy_version 1862463 (0.0006) [2023-12-27 18:33:40,541][20381] Updated weights for policy 1, policy_version 1862473 (0.0006) [2023-12-27 18:33:40,603][20381] Updated weights for policy 1, policy_version 1862483 (0.0005) [2023-12-27 18:33:40,858][20347] Updated weights for policy 0, policy_version 1864061 (0.0008) [2023-12-27 18:33:40,904][20347] Updated weights for policy 0, policy_version 1864071 (0.0008) [2023-12-27 18:33:40,951][20347] Updated weights for policy 0, policy_version 1864081 (0.0008) [2023-12-27 18:33:41,165][20381] Updated weights for policy 1, policy_version 1862493 (0.0007) [2023-12-27 18:33:41,226][20381] Updated weights for policy 1, policy_version 1862503 (0.0008) [2023-12-27 18:33:41,288][20381] Updated weights for policy 1, policy_version 1862513 (0.0009) [2023-12-27 18:33:41,786][20347] Updated weights for policy 0, policy_version 1864091 (0.0007) [2023-12-27 18:33:41,844][20347] Updated weights for policy 0, policy_version 1864101 (0.0009) [2023-12-27 18:33:41,898][20347] Updated weights for policy 0, policy_version 1864111 (0.0006) [2023-12-27 18:33:42,080][20381] Updated weights for policy 1, policy_version 1862523 (0.0007) [2023-12-27 18:33:42,143][20381] Updated weights for policy 1, policy_version 1862533 (0.0007) [2023-12-27 18:33:42,201][20381] Updated weights for policy 1, policy_version 1862543 (0.0007) [2023-12-27 18:33:42,364][19170] Fps is (10 sec: 20476.2, 60 sec: 20752.5, 300 sec: 20466.0). Total num frames: 954163200. Throughput: 0: 10291.4, 1: 10283.6. Samples: 954152708. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:42,365][19170] Avg episode reward: [(0, '1000.000'), (1, '831.520')] [2023-12-27 18:33:42,649][20347] Updated weights for policy 0, policy_version 1864121 (0.0007) [2023-12-27 18:33:42,715][20347] Updated weights for policy 0, policy_version 1864131 (0.0007) [2023-12-27 18:33:42,780][20347] Updated weights for policy 0, policy_version 1864141 (0.0007) [2023-12-27 18:33:42,827][20381] Updated weights for policy 1, policy_version 1862553 (0.0006) [2023-12-27 18:33:42,843][20347] Updated weights for policy 0, policy_version 1864151 (0.0007) [2023-12-27 18:33:42,891][20381] Updated weights for policy 1, policy_version 1862563 (0.0007) [2023-12-27 18:33:42,954][20381] Updated weights for policy 1, policy_version 1862573 (0.0007) [2023-12-27 18:33:43,010][20381] Updated weights for policy 1, policy_version 1862583 (0.0008) [2023-12-27 18:33:43,553][20347] Updated weights for policy 0, policy_version 1864161 (0.0007) [2023-12-27 18:33:43,606][20347] Updated weights for policy 0, policy_version 1864171 (0.0006) [2023-12-27 18:33:43,611][20381] Updated weights for policy 1, policy_version 1862593 (0.0006) [2023-12-27 18:33:43,651][20347] Updated weights for policy 0, policy_version 1864181 (0.0005) [2023-12-27 18:33:43,663][20381] Updated weights for policy 1, policy_version 1862603 (0.0006) [2023-12-27 18:33:43,722][20381] Updated weights for policy 1, policy_version 1862613 (0.0008) [2023-12-27 18:33:44,363][20381] Updated weights for policy 1, policy_version 1862623 (0.0006) [2023-12-27 18:33:44,368][20347] Updated weights for policy 0, policy_version 1864191 (0.0006) [2023-12-27 18:33:44,415][20381] Updated weights for policy 1, policy_version 1862633 (0.0006) [2023-12-27 18:33:44,424][20347] Updated weights for policy 0, policy_version 1864201 (0.0006) [2023-12-27 18:33:44,463][20381] Updated weights for policy 1, policy_version 1862643 (0.0006) [2023-12-27 18:33:44,480][20347] Updated weights for policy 0, policy_version 1864211 (0.0007) [2023-12-27 18:33:45,173][20347] Updated weights for policy 0, policy_version 1864221 (0.0007) [2023-12-27 18:33:45,207][20381] Updated weights for policy 1, policy_version 1862653 (0.0005) [2023-12-27 18:33:45,232][20347] Updated weights for policy 0, policy_version 1864231 (0.0006) [2023-12-27 18:33:45,269][20381] Updated weights for policy 1, policy_version 1862663 (0.0007) [2023-12-27 18:33:45,294][20347] Updated weights for policy 0, policy_version 1864241 (0.0007) [2023-12-27 18:33:45,327][20381] Updated weights for policy 1, policy_version 1862673 (0.0007) [2023-12-27 18:33:45,938][20381] Updated weights for policy 1, policy_version 1862683 (0.0007) [2023-12-27 18:33:45,992][20381] Updated weights for policy 1, policy_version 1862693 (0.0006) [2023-12-27 18:33:46,051][20347] Updated weights for policy 0, policy_version 1864251 (0.0005) [2023-12-27 18:33:46,053][20381] Updated weights for policy 1, policy_version 1862703 (0.0006) [2023-12-27 18:33:46,116][20347] Updated weights for policy 0, policy_version 1864261 (0.0008) [2023-12-27 18:33:46,170][20347] Updated weights for policy 0, policy_version 1864271 (0.0008) [2023-12-27 18:33:46,608][20381] Updated weights for policy 1, policy_version 1862713 (0.0007) [2023-12-27 18:33:46,666][20381] Updated weights for policy 1, policy_version 1862723 (0.0009) [2023-12-27 18:33:46,719][20381] Updated weights for policy 1, policy_version 1862733 (0.0010) [2023-12-27 18:33:46,774][20381] Updated weights for policy 1, policy_version 1862743 (0.0010) [2023-12-27 18:33:46,908][20347] Updated weights for policy 0, policy_version 1864281 (0.0010) [2023-12-27 18:33:46,956][20347] Updated weights for policy 0, policy_version 1864291 (0.0008) [2023-12-27 18:33:47,003][20347] Updated weights for policy 0, policy_version 1864301 (0.0010) [2023-12-27 18:33:47,058][20347] Updated weights for policy 0, policy_version 1864311 (0.0008) [2023-12-27 18:33:47,354][20381] Updated weights for policy 1, policy_version 1862753 (0.0006) [2023-12-27 18:33:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 954261504. Throughput: 0: 10176.8, 1: 10370.1. Samples: 954273328. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '822.680')] [2023-12-27 18:33:47,409][20381] Updated weights for policy 1, policy_version 1862763 (0.0005) [2023-12-27 18:33:47,470][20381] Updated weights for policy 1, policy_version 1862773 (0.0010) [2023-12-27 18:33:47,768][20347] Updated weights for policy 0, policy_version 1864321 (0.0006) [2023-12-27 18:33:47,827][20347] Updated weights for policy 0, policy_version 1864331 (0.0009) [2023-12-27 18:33:47,889][20347] Updated weights for policy 0, policy_version 1864341 (0.0010) [2023-12-27 18:33:48,069][20381] Updated weights for policy 1, policy_version 1862783 (0.0010) [2023-12-27 18:33:48,126][20381] Updated weights for policy 1, policy_version 1862793 (0.0009) [2023-12-27 18:33:48,177][20381] Updated weights for policy 1, policy_version 1862803 (0.0005) [2023-12-27 18:33:48,540][20347] Updated weights for policy 0, policy_version 1864351 (0.0010) [2023-12-27 18:33:48,603][20347] Updated weights for policy 0, policy_version 1864361 (0.0010) [2023-12-27 18:33:48,662][20347] Updated weights for policy 0, policy_version 1864371 (0.0010) [2023-12-27 18:33:48,861][20381] Updated weights for policy 1, policy_version 1862813 (0.0006) [2023-12-27 18:33:48,917][20381] Updated weights for policy 1, policy_version 1862823 (0.0007) [2023-12-27 18:33:48,978][20381] Updated weights for policy 1, policy_version 1862833 (0.0006) [2023-12-27 18:33:49,300][20347] Updated weights for policy 0, policy_version 1864381 (0.0009) [2023-12-27 18:33:49,361][20347] Updated weights for policy 0, policy_version 1864391 (0.0008) [2023-12-27 18:33:49,427][20347] Updated weights for policy 0, policy_version 1864401 (0.0009) [2023-12-27 18:33:49,703][20381] Updated weights for policy 1, policy_version 1862843 (0.0007) [2023-12-27 18:33:49,762][20381] Updated weights for policy 1, policy_version 1862853 (0.0007) [2023-12-27 18:33:49,817][20381] Updated weights for policy 1, policy_version 1862863 (0.0008) [2023-12-27 18:33:50,122][20347] Updated weights for policy 0, policy_version 1864411 (0.0008) [2023-12-27 18:33:50,180][20347] Updated weights for policy 0, policy_version 1864421 (0.0011) [2023-12-27 18:33:50,237][20347] Updated weights for policy 0, policy_version 1864431 (0.0010) [2023-12-27 18:33:50,564][20381] Updated weights for policy 1, policy_version 1862873 (0.0008) [2023-12-27 18:33:50,623][20381] Updated weights for policy 1, policy_version 1862883 (0.0007) [2023-12-27 18:33:50,683][20381] Updated weights for policy 1, policy_version 1862893 (0.0008) [2023-12-27 18:33:50,734][20381] Updated weights for policy 1, policy_version 1862903 (0.0007) [2023-12-27 18:33:50,894][20347] Updated weights for policy 0, policy_version 1864441 (0.0010) [2023-12-27 18:33:50,950][20347] Updated weights for policy 0, policy_version 1864451 (0.0011) [2023-12-27 18:33:50,999][20347] Updated weights for policy 0, policy_version 1864461 (0.0010) [2023-12-27 18:33:51,062][20347] Updated weights for policy 0, policy_version 1864471 (0.0010) [2023-12-27 18:33:51,488][20381] Updated weights for policy 1, policy_version 1862913 (0.0010) [2023-12-27 18:33:51,542][20381] Updated weights for policy 1, policy_version 1862923 (0.0011) [2023-12-27 18:33:51,607][20381] Updated weights for policy 1, policy_version 1862933 (0.0011) [2023-12-27 18:33:51,813][20347] Updated weights for policy 0, policy_version 1864481 (0.0007) [2023-12-27 18:33:51,879][20347] Updated weights for policy 0, policy_version 1864491 (0.0006) [2023-12-27 18:33:51,937][20347] Updated weights for policy 0, policy_version 1864501 (0.0005) [2023-12-27 18:33:52,353][20381] Updated weights for policy 1, policy_version 1862943 (0.0008) [2023-12-27 18:33:52,362][19170] Fps is (10 sec: 19664.3, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 954359808. Throughput: 0: 10130.4, 1: 10374.4. Samples: 954334840. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '772.810')] [2023-12-27 18:33:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001864504_477396992.pth... [2023-12-27 18:33:52,394][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001863352_477102080.pth [2023-12-27 18:33:52,420][20381] Updated weights for policy 1, policy_version 1862953 (0.0009) [2023-12-27 18:33:52,482][20381] Updated weights for policy 1, policy_version 1862963 (0.0011) [2023-12-27 18:33:52,508][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001862968_476971008.pth... [2023-12-27 18:33:52,511][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001861752_476659712.pth [2023-12-27 18:33:52,552][20347] Updated weights for policy 0, policy_version 1864511 (0.0006) [2023-12-27 18:33:52,608][20347] Updated weights for policy 0, policy_version 1864521 (0.0007) [2023-12-27 18:33:52,657][20347] Updated weights for policy 0, policy_version 1864531 (0.0008) [2023-12-27 18:33:53,204][20381] Updated weights for policy 1, policy_version 1862973 (0.0011) [2023-12-27 18:33:53,259][20381] Updated weights for policy 1, policy_version 1862983 (0.0010) [2023-12-27 18:33:53,314][20381] Updated weights for policy 1, policy_version 1862993 (0.0010) [2023-12-27 18:33:53,380][20347] Updated weights for policy 0, policy_version 1864541 (0.0009) [2023-12-27 18:33:53,428][20347] Updated weights for policy 0, policy_version 1864551 (0.0010) [2023-12-27 18:33:53,482][20347] Updated weights for policy 0, policy_version 1864561 (0.0010) [2023-12-27 18:33:53,972][20381] Updated weights for policy 1, policy_version 1863003 (0.0010) [2023-12-27 18:33:54,023][20381] Updated weights for policy 1, policy_version 1863013 (0.0011) [2023-12-27 18:33:54,091][20381] Updated weights for policy 1, policy_version 1863023 (0.0011) [2023-12-27 18:33:54,113][20347] Updated weights for policy 0, policy_version 1864571 (0.0010) [2023-12-27 18:33:54,162][20347] Updated weights for policy 0, policy_version 1864581 (0.0010) [2023-12-27 18:33:54,214][20347] Updated weights for policy 0, policy_version 1864591 (0.0010) [2023-12-27 18:33:54,786][20381] Updated weights for policy 1, policy_version 1863033 (0.0011) [2023-12-27 18:33:54,818][20347] Updated weights for policy 0, policy_version 1864601 (0.0010) [2023-12-27 18:33:54,845][20381] Updated weights for policy 1, policy_version 1863043 (0.0011) [2023-12-27 18:33:54,872][20347] Updated weights for policy 0, policy_version 1864611 (0.0005) [2023-12-27 18:33:54,896][20381] Updated weights for policy 1, policy_version 1863053 (0.0010) [2023-12-27 18:33:54,929][20347] Updated weights for policy 0, policy_version 1864621 (0.0007) [2023-12-27 18:33:54,947][20381] Updated weights for policy 1, policy_version 1863063 (0.0009) [2023-12-27 18:33:54,984][20347] Updated weights for policy 0, policy_version 1864631 (0.0008) [2023-12-27 18:33:55,564][20347] Updated weights for policy 0, policy_version 1864641 (0.0010) [2023-12-27 18:33:55,597][20381] Updated weights for policy 1, policy_version 1863073 (0.0007) [2023-12-27 18:33:55,608][20347] Updated weights for policy 0, policy_version 1864651 (0.0008) [2023-12-27 18:33:55,657][20381] Updated weights for policy 1, policy_version 1863083 (0.0006) [2023-12-27 18:33:55,659][20347] Updated weights for policy 0, policy_version 1864661 (0.0010) [2023-12-27 18:33:55,711][20381] Updated weights for policy 1, policy_version 1863093 (0.0007) [2023-12-27 18:33:56,363][20347] Updated weights for policy 0, policy_version 1864671 (0.0011) [2023-12-27 18:33:56,375][20381] Updated weights for policy 1, policy_version 1863103 (0.0008) [2023-12-27 18:33:56,419][20347] Updated weights for policy 0, policy_version 1864681 (0.0010) [2023-12-27 18:33:56,424][20381] Updated weights for policy 1, policy_version 1863113 (0.0007) [2023-12-27 18:33:56,474][20347] Updated weights for policy 0, policy_version 1864691 (0.0010) [2023-12-27 18:33:56,479][20381] Updated weights for policy 1, policy_version 1863123 (0.0008) [2023-12-27 18:33:57,083][20347] Updated weights for policy 0, policy_version 1864701 (0.0010) [2023-12-27 18:33:57,131][20347] Updated weights for policy 0, policy_version 1864711 (0.0010) [2023-12-27 18:33:57,182][20347] Updated weights for policy 0, policy_version 1864721 (0.0010) [2023-12-27 18:33:57,196][20381] Updated weights for policy 1, policy_version 1863133 (0.0007) [2023-12-27 18:33:57,256][20381] Updated weights for policy 1, policy_version 1863143 (0.0008) [2023-12-27 18:33:57,310][20381] Updated weights for policy 1, policy_version 1863153 (0.0007) [2023-12-27 18:33:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 954474496. Throughput: 0: 10121.9, 1: 10322.2. Samples: 954458264. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:33:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '831.470')] [2023-12-27 18:33:57,804][20347] Updated weights for policy 0, policy_version 1864731 (0.0009) [2023-12-27 18:33:57,858][20347] Updated weights for policy 0, policy_version 1864741 (0.0006) [2023-12-27 18:33:57,904][20347] Updated weights for policy 0, policy_version 1864751 (0.0007) [2023-12-27 18:33:58,023][20381] Updated weights for policy 1, policy_version 1863163 (0.0007) [2023-12-27 18:33:58,071][20381] Updated weights for policy 1, policy_version 1863173 (0.0008) [2023-12-27 18:33:58,121][20381] Updated weights for policy 1, policy_version 1863183 (0.0007) [2023-12-27 18:33:58,551][20347] Updated weights for policy 0, policy_version 1864761 (0.0007) [2023-12-27 18:33:58,617][20347] Updated weights for policy 0, policy_version 1864771 (0.0009) [2023-12-27 18:33:58,679][20347] Updated weights for policy 0, policy_version 1864781 (0.0007) [2023-12-27 18:33:58,749][20347] Updated weights for policy 0, policy_version 1864792 (0.0008) [2023-12-27 18:33:58,948][20381] Updated weights for policy 1, policy_version 1863193 (0.0006) [2023-12-27 18:33:59,006][20381] Updated weights for policy 1, policy_version 1863203 (0.0007) [2023-12-27 18:33:59,061][20381] Updated weights for policy 1, policy_version 1863213 (0.0008) [2023-12-27 18:33:59,120][20381] Updated weights for policy 1, policy_version 1863223 (0.0007) [2023-12-27 18:33:59,519][20347] Updated weights for policy 0, policy_version 1864802 (0.0006) [2023-12-27 18:33:59,578][20347] Updated weights for policy 0, policy_version 1864812 (0.0008) [2023-12-27 18:33:59,637][20347] Updated weights for policy 0, policy_version 1864822 (0.0007) [2023-12-27 18:33:59,850][20381] Updated weights for policy 1, policy_version 1863233 (0.0007) [2023-12-27 18:33:59,918][20381] Updated weights for policy 1, policy_version 1863243 (0.0008) [2023-12-27 18:33:59,980][20381] Updated weights for policy 1, policy_version 1863253 (0.0008) [2023-12-27 18:34:00,261][20347] Updated weights for policy 0, policy_version 1864832 (0.0008) [2023-12-27 18:34:00,320][20347] Updated weights for policy 0, policy_version 1864842 (0.0007) [2023-12-27 18:34:00,377][20347] Updated weights for policy 0, policy_version 1864852 (0.0007) [2023-12-27 18:34:00,671][20381] Updated weights for policy 1, policy_version 1863263 (0.0006) [2023-12-27 18:34:00,728][20381] Updated weights for policy 1, policy_version 1863273 (0.0005) [2023-12-27 18:34:00,776][20381] Updated weights for policy 1, policy_version 1863283 (0.0005) [2023-12-27 18:34:01,011][20347] Updated weights for policy 0, policy_version 1864862 (0.0007) [2023-12-27 18:34:01,067][20347] Updated weights for policy 0, policy_version 1864872 (0.0007) [2023-12-27 18:34:01,122][20347] Updated weights for policy 0, policy_version 1864882 (0.0007) [2023-12-27 18:34:01,431][20381] Updated weights for policy 1, policy_version 1863293 (0.0008) [2023-12-27 18:34:01,484][20381] Updated weights for policy 1, policy_version 1863303 (0.0012) [2023-12-27 18:34:01,533][20381] Updated weights for policy 1, policy_version 1863313 (0.0010) [2023-12-27 18:34:01,804][20347] Updated weights for policy 0, policy_version 1864892 (0.0008) [2023-12-27 18:34:01,852][20347] Updated weights for policy 0, policy_version 1864902 (0.0007) [2023-12-27 18:34:01,899][20347] Updated weights for policy 0, policy_version 1864912 (0.0006) [2023-12-27 18:34:02,250][20381] Updated weights for policy 1, policy_version 1863323 (0.0010) [2023-12-27 18:34:02,313][20381] Updated weights for policy 1, policy_version 1863333 (0.0008) [2023-12-27 18:34:02,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20479.9, 300 sec: 20410.6). Total num frames: 954564608. Throughput: 0: 10133.1, 1: 10241.8. Samples: 954578856. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '800.740')] [2023-12-27 18:34:02,370][20381] Updated weights for policy 1, policy_version 1863343 (0.0008) [2023-12-27 18:34:02,538][20347] Updated weights for policy 0, policy_version 1864922 (0.0008) [2023-12-27 18:34:02,587][20347] Updated weights for policy 0, policy_version 1864932 (0.0008) [2023-12-27 18:34:02,634][20347] Updated weights for policy 0, policy_version 1864942 (0.0007) [2023-12-27 18:34:02,683][20347] Updated weights for policy 0, policy_version 1864952 (0.0008) [2023-12-27 18:34:03,054][20381] Updated weights for policy 1, policy_version 1863353 (0.0008) [2023-12-27 18:34:03,116][20381] Updated weights for policy 1, policy_version 1863363 (0.0007) [2023-12-27 18:34:03,178][20381] Updated weights for policy 1, policy_version 1863373 (0.0008) [2023-12-27 18:34:03,234][20381] Updated weights for policy 1, policy_version 1863383 (0.0007) [2023-12-27 18:34:03,317][20347] Updated weights for policy 0, policy_version 1864962 (0.0005) [2023-12-27 18:34:03,375][20347] Updated weights for policy 0, policy_version 1864972 (0.0005) [2023-12-27 18:34:03,428][20347] Updated weights for policy 0, policy_version 1864982 (0.0010) [2023-12-27 18:34:03,934][20381] Updated weights for policy 1, policy_version 1863393 (0.0008) [2023-12-27 18:34:03,992][20381] Updated weights for policy 1, policy_version 1863403 (0.0007) [2023-12-27 18:34:04,049][20381] Updated weights for policy 1, policy_version 1863413 (0.0007) [2023-12-27 18:34:04,082][20347] Updated weights for policy 0, policy_version 1864992 (0.0009) [2023-12-27 18:34:04,141][20347] Updated weights for policy 0, policy_version 1865002 (0.0010) [2023-12-27 18:34:04,204][20347] Updated weights for policy 0, policy_version 1865012 (0.0010) [2023-12-27 18:34:04,765][20381] Updated weights for policy 1, policy_version 1863423 (0.0008) [2023-12-27 18:34:04,815][20381] Updated weights for policy 1, policy_version 1863433 (0.0007) [2023-12-27 18:34:04,859][20347] Updated weights for policy 0, policy_version 1865022 (0.0010) [2023-12-27 18:34:04,864][20381] Updated weights for policy 1, policy_version 1863443 (0.0006) [2023-12-27 18:34:04,907][20347] Updated weights for policy 0, policy_version 1865032 (0.0010) [2023-12-27 18:34:04,956][20347] Updated weights for policy 0, policy_version 1865042 (0.0009) [2023-12-27 18:34:05,437][20381] Updated weights for policy 1, policy_version 1863453 (0.0006) [2023-12-27 18:34:05,483][20381] Updated weights for policy 1, policy_version 1863463 (0.0005) [2023-12-27 18:34:05,531][20381] Updated weights for policy 1, policy_version 1863473 (0.0005) [2023-12-27 18:34:05,593][20347] Updated weights for policy 0, policy_version 1865052 (0.0009) [2023-12-27 18:34:05,642][20347] Updated weights for policy 0, policy_version 1865062 (0.0007) [2023-12-27 18:34:05,699][20347] Updated weights for policy 0, policy_version 1865072 (0.0006) [2023-12-27 18:34:06,081][20381] Updated weights for policy 1, policy_version 1863483 (0.0005) [2023-12-27 18:34:06,146][20381] Updated weights for policy 1, policy_version 1863493 (0.0008) [2023-12-27 18:34:06,205][20381] Updated weights for policy 1, policy_version 1863503 (0.0006) [2023-12-27 18:34:06,451][20347] Updated weights for policy 0, policy_version 1865082 (0.0009) [2023-12-27 18:34:06,508][20347] Updated weights for policy 0, policy_version 1865092 (0.0007) [2023-12-27 18:34:06,560][20347] Updated weights for policy 0, policy_version 1865102 (0.0007) [2023-12-27 18:34:06,610][20347] Updated weights for policy 0, policy_version 1865112 (0.0008) [2023-12-27 18:34:06,860][20381] Updated weights for policy 1, policy_version 1863513 (0.0007) [2023-12-27 18:34:06,916][20381] Updated weights for policy 1, policy_version 1863523 (0.0008) [2023-12-27 18:34:06,971][20381] Updated weights for policy 1, policy_version 1863533 (0.0007) [2023-12-27 18:34:07,021][20381] Updated weights for policy 1, policy_version 1863543 (0.0008) [2023-12-27 18:34:07,345][20347] Updated weights for policy 0, policy_version 1865122 (0.0005) [2023-12-27 18:34:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 954671104. Throughput: 0: 10187.9, 1: 10230.0. Samples: 954641336. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '799.530')] [2023-12-27 18:34:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001863544_477118464.pth... [2023-12-27 18:34:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001862360_476815360.pth [2023-12-27 18:34:07,401][20347] Updated weights for policy 0, policy_version 1865132 (0.0009) [2023-12-27 18:34:07,457][20347] Updated weights for policy 0, policy_version 1865142 (0.0011) [2023-12-27 18:34:07,466][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001865144_477560832.pth... [2023-12-27 18:34:07,470][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001863928_477249536.pth [2023-12-27 18:34:07,752][20381] Updated weights for policy 1, policy_version 1863553 (0.0007) [2023-12-27 18:34:07,796][20381] Updated weights for policy 1, policy_version 1863563 (0.0007) [2023-12-27 18:34:07,851][20381] Updated weights for policy 1, policy_version 1863573 (0.0007) [2023-12-27 18:34:08,085][20347] Updated weights for policy 0, policy_version 1865152 (0.0008) [2023-12-27 18:34:08,139][20347] Updated weights for policy 0, policy_version 1865162 (0.0008) [2023-12-27 18:34:08,183][20347] Updated weights for policy 0, policy_version 1865172 (0.0007) [2023-12-27 18:34:08,516][20381] Updated weights for policy 1, policy_version 1863583 (0.0008) [2023-12-27 18:34:08,566][20381] Updated weights for policy 1, policy_version 1863593 (0.0007) [2023-12-27 18:34:08,628][20381] Updated weights for policy 1, policy_version 1863603 (0.0007) [2023-12-27 18:34:08,886][20347] Updated weights for policy 0, policy_version 1865182 (0.0007) [2023-12-27 18:34:08,935][20347] Updated weights for policy 0, policy_version 1865192 (0.0007) [2023-12-27 18:34:08,984][20347] Updated weights for policy 0, policy_version 1865202 (0.0007) [2023-12-27 18:34:09,263][20381] Updated weights for policy 1, policy_version 1863613 (0.0006) [2023-12-27 18:34:09,325][20381] Updated weights for policy 1, policy_version 1863623 (0.0008) [2023-12-27 18:34:09,386][20381] Updated weights for policy 1, policy_version 1863633 (0.0009) [2023-12-27 18:34:09,770][20347] Updated weights for policy 0, policy_version 1865212 (0.0007) [2023-12-27 18:34:09,834][20347] Updated weights for policy 0, policy_version 1865222 (0.0008) [2023-12-27 18:34:09,900][20347] Updated weights for policy 0, policy_version 1865232 (0.0008) [2023-12-27 18:34:10,024][20381] Updated weights for policy 1, policy_version 1863643 (0.0007) [2023-12-27 18:34:10,087][20381] Updated weights for policy 1, policy_version 1863653 (0.0005) [2023-12-27 18:34:10,149][20381] Updated weights for policy 1, policy_version 1863663 (0.0008) [2023-12-27 18:34:10,511][20347] Updated weights for policy 0, policy_version 1865242 (0.0008) [2023-12-27 18:34:10,573][20347] Updated weights for policy 0, policy_version 1865252 (0.0006) [2023-12-27 18:34:10,634][20347] Updated weights for policy 0, policy_version 1865262 (0.0005) [2023-12-27 18:34:10,697][20347] Updated weights for policy 0, policy_version 1865272 (0.0006) [2023-12-27 18:34:10,804][20381] Updated weights for policy 1, policy_version 1863673 (0.0009) [2023-12-27 18:34:10,864][20381] Updated weights for policy 1, policy_version 1863683 (0.0009) [2023-12-27 18:34:10,923][20381] Updated weights for policy 1, policy_version 1863693 (0.0006) [2023-12-27 18:34:10,982][20381] Updated weights for policy 1, policy_version 1863703 (0.0009) [2023-12-27 18:34:11,360][20347] Updated weights for policy 0, policy_version 1865282 (0.0008) [2023-12-27 18:34:11,424][20347] Updated weights for policy 0, policy_version 1865292 (0.0007) [2023-12-27 18:34:11,489][20347] Updated weights for policy 0, policy_version 1865302 (0.0005) [2023-12-27 18:34:11,732][20381] Updated weights for policy 1, policy_version 1863713 (0.0008) [2023-12-27 18:34:11,796][20381] Updated weights for policy 1, policy_version 1863723 (0.0007) [2023-12-27 18:34:11,852][20381] Updated weights for policy 1, policy_version 1863733 (0.0008) [2023-12-27 18:34:12,145][20347] Updated weights for policy 0, policy_version 1865312 (0.0010) [2023-12-27 18:34:12,204][20347] Updated weights for policy 0, policy_version 1865322 (0.0010) [2023-12-27 18:34:12,265][20347] Updated weights for policy 0, policy_version 1865332 (0.0006) [2023-12-27 18:34:12,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 954777600. Throughput: 0: 10209.7, 1: 10230.1. Samples: 954765932. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '859.760')] [2023-12-27 18:34:12,557][20381] Updated weights for policy 1, policy_version 1863743 (0.0008) [2023-12-27 18:34:12,624][20381] Updated weights for policy 1, policy_version 1863753 (0.0008) [2023-12-27 18:34:12,684][20381] Updated weights for policy 1, policy_version 1863763 (0.0008) [2023-12-27 18:34:12,973][20347] Updated weights for policy 0, policy_version 1865342 (0.0007) [2023-12-27 18:34:13,027][20347] Updated weights for policy 0, policy_version 1865352 (0.0006) [2023-12-27 18:34:13,077][20347] Updated weights for policy 0, policy_version 1865362 (0.0009) [2023-12-27 18:34:13,371][20381] Updated weights for policy 1, policy_version 1863773 (0.0007) [2023-12-27 18:34:13,414][20381] Updated weights for policy 1, policy_version 1863783 (0.0011) [2023-12-27 18:34:13,463][20381] Updated weights for policy 1, policy_version 1863793 (0.0010) [2023-12-27 18:34:13,665][20347] Updated weights for policy 0, policy_version 1865372 (0.0009) [2023-12-27 18:34:13,723][20347] Updated weights for policy 0, policy_version 1865382 (0.0010) [2023-12-27 18:34:13,781][20347] Updated weights for policy 0, policy_version 1865392 (0.0010) [2023-12-27 18:34:14,123][20381] Updated weights for policy 1, policy_version 1863803 (0.0009) [2023-12-27 18:34:14,167][20381] Updated weights for policy 1, policy_version 1863813 (0.0007) [2023-12-27 18:34:14,221][20381] Updated weights for policy 1, policy_version 1863823 (0.0007) [2023-12-27 18:34:14,461][20347] Updated weights for policy 0, policy_version 1865402 (0.0010) [2023-12-27 18:34:14,506][20347] Updated weights for policy 0, policy_version 1865412 (0.0010) [2023-12-27 18:34:14,557][20347] Updated weights for policy 0, policy_version 1865422 (0.0010) [2023-12-27 18:34:14,611][20347] Updated weights for policy 0, policy_version 1865432 (0.0010) [2023-12-27 18:34:14,900][20381] Updated weights for policy 1, policy_version 1863833 (0.0005) [2023-12-27 18:34:14,960][20381] Updated weights for policy 1, policy_version 1863843 (0.0007) [2023-12-27 18:34:15,023][20381] Updated weights for policy 1, policy_version 1863853 (0.0007) [2023-12-27 18:34:15,088][20381] Updated weights for policy 1, policy_version 1863863 (0.0008) [2023-12-27 18:34:15,347][20347] Updated weights for policy 0, policy_version 1865442 (0.0010) [2023-12-27 18:34:15,396][20347] Updated weights for policy 0, policy_version 1865452 (0.0010) [2023-12-27 18:34:15,447][20347] Updated weights for policy 0, policy_version 1865462 (0.0010) [2023-12-27 18:34:15,764][20381] Updated weights for policy 1, policy_version 1863873 (0.0009) [2023-12-27 18:34:15,815][20381] Updated weights for policy 1, policy_version 1863883 (0.0008) [2023-12-27 18:34:15,866][20381] Updated weights for policy 1, policy_version 1863893 (0.0006) [2023-12-27 18:34:16,062][20347] Updated weights for policy 0, policy_version 1865472 (0.0008) [2023-12-27 18:34:16,115][20347] Updated weights for policy 0, policy_version 1865482 (0.0008) [2023-12-27 18:34:16,165][20347] Updated weights for policy 0, policy_version 1865492 (0.0007) [2023-12-27 18:34:16,569][20381] Updated weights for policy 1, policy_version 1863903 (0.0008) [2023-12-27 18:34:16,627][20381] Updated weights for policy 1, policy_version 1863913 (0.0005) [2023-12-27 18:34:16,688][20381] Updated weights for policy 1, policy_version 1863923 (0.0006) [2023-12-27 18:34:16,874][20347] Updated weights for policy 0, policy_version 1865502 (0.0009) [2023-12-27 18:34:16,928][20347] Updated weights for policy 0, policy_version 1865512 (0.0010) [2023-12-27 18:34:16,978][20347] Updated weights for policy 0, policy_version 1865522 (0.0010) [2023-12-27 18:34:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 954875904. Throughput: 0: 10238.6, 1: 10181.6. Samples: 954888068. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.920')] [2023-12-27 18:34:17,368][20381] Updated weights for policy 1, policy_version 1863933 (0.0007) [2023-12-27 18:34:17,413][20381] Updated weights for policy 1, policy_version 1863943 (0.0007) [2023-12-27 18:34:17,466][20381] Updated weights for policy 1, policy_version 1863953 (0.0008) [2023-12-27 18:34:17,661][20347] Updated weights for policy 0, policy_version 1865532 (0.0009) [2023-12-27 18:34:17,715][20347] Updated weights for policy 0, policy_version 1865542 (0.0008) [2023-12-27 18:34:17,760][20347] Updated weights for policy 0, policy_version 1865552 (0.0008) [2023-12-27 18:34:18,067][20381] Updated weights for policy 1, policy_version 1863963 (0.0006) [2023-12-27 18:34:18,127][20381] Updated weights for policy 1, policy_version 1863973 (0.0005) [2023-12-27 18:34:18,176][20381] Updated weights for policy 1, policy_version 1863983 (0.0005) [2023-12-27 18:34:18,449][20347] Updated weights for policy 0, policy_version 1865562 (0.0009) [2023-12-27 18:34:18,515][20347] Updated weights for policy 0, policy_version 1865572 (0.0008) [2023-12-27 18:34:18,583][20347] Updated weights for policy 0, policy_version 1865582 (0.0008) [2023-12-27 18:34:18,639][20347] Updated weights for policy 0, policy_version 1865592 (0.0008) [2023-12-27 18:34:18,769][20381] Updated weights for policy 1, policy_version 1863993 (0.0006) [2023-12-27 18:34:18,827][20381] Updated weights for policy 1, policy_version 1864003 (0.0007) [2023-12-27 18:34:18,890][20381] Updated weights for policy 1, policy_version 1864013 (0.0006) [2023-12-27 18:34:18,952][20381] Updated weights for policy 1, policy_version 1864023 (0.0005) [2023-12-27 18:34:19,349][20347] Updated weights for policy 0, policy_version 1865602 (0.0008) [2023-12-27 18:34:19,415][20347] Updated weights for policy 0, policy_version 1865612 (0.0007) [2023-12-27 18:34:19,465][20347] Updated weights for policy 0, policy_version 1865622 (0.0006) [2023-12-27 18:34:19,655][20381] Updated weights for policy 1, policy_version 1864033 (0.0009) [2023-12-27 18:34:19,719][20381] Updated weights for policy 1, policy_version 1864043 (0.0009) [2023-12-27 18:34:19,787][20381] Updated weights for policy 1, policy_version 1864053 (0.0007) [2023-12-27 18:34:20,104][20347] Updated weights for policy 0, policy_version 1865632 (0.0008) [2023-12-27 18:34:20,163][20347] Updated weights for policy 0, policy_version 1865642 (0.0008) [2023-12-27 18:34:20,220][20347] Updated weights for policy 0, policy_version 1865652 (0.0007) [2023-12-27 18:34:20,524][20381] Updated weights for policy 1, policy_version 1864063 (0.0008) [2023-12-27 18:34:20,578][20381] Updated weights for policy 1, policy_version 1864073 (0.0008) [2023-12-27 18:34:20,633][20381] Updated weights for policy 1, policy_version 1864083 (0.0007) [2023-12-27 18:34:20,909][20347] Updated weights for policy 0, policy_version 1865662 (0.0008) [2023-12-27 18:34:20,968][20347] Updated weights for policy 0, policy_version 1865672 (0.0007) [2023-12-27 18:34:21,021][20347] Updated weights for policy 0, policy_version 1865682 (0.0008) [2023-12-27 18:34:21,289][20381] Updated weights for policy 1, policy_version 1864093 (0.0008) [2023-12-27 18:34:21,347][20381] Updated weights for policy 1, policy_version 1864103 (0.0006) [2023-12-27 18:34:21,415][20381] Updated weights for policy 1, policy_version 1864113 (0.0008) [2023-12-27 18:34:21,808][20347] Updated weights for policy 0, policy_version 1865692 (0.0008) [2023-12-27 18:34:21,874][20347] Updated weights for policy 0, policy_version 1865702 (0.0009) [2023-12-27 18:34:21,933][20347] Updated weights for policy 0, policy_version 1865712 (0.0007) [2023-12-27 18:34:22,098][20381] Updated weights for policy 1, policy_version 1864123 (0.0008) [2023-12-27 18:34:22,153][20381] Updated weights for policy 1, policy_version 1864133 (0.0007) [2023-12-27 18:34:22,207][20381] Updated weights for policy 1, policy_version 1864143 (0.0007) [2023-12-27 18:34:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 954982400. Throughput: 0: 10176.7, 1: 10232.3. Samples: 954950188. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '805.350')] [2023-12-27 18:34:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001864152_477274112.pth... [2023-12-27 18:34:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001865720_477708288.pth... [2023-12-27 18:34:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001862968_476971008.pth [2023-12-27 18:34:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001864504_477396992.pth [2023-12-27 18:34:22,664][20347] Updated weights for policy 0, policy_version 1865722 (0.0007) [2023-12-27 18:34:22,728][20347] Updated weights for policy 0, policy_version 1865732 (0.0006) [2023-12-27 18:34:22,789][20347] Updated weights for policy 0, policy_version 1865742 (0.0006) [2023-12-27 18:34:22,851][20347] Updated weights for policy 0, policy_version 1865752 (0.0005) [2023-12-27 18:34:22,927][20381] Updated weights for policy 1, policy_version 1864153 (0.0007) [2023-12-27 18:34:22,979][20381] Updated weights for policy 1, policy_version 1864163 (0.0009) [2023-12-27 18:34:23,028][20381] Updated weights for policy 1, policy_version 1864173 (0.0008) [2023-12-27 18:34:23,081][20381] Updated weights for policy 1, policy_version 1864183 (0.0009) [2023-12-27 18:34:23,405][20347] Updated weights for policy 0, policy_version 1865762 (0.0008) [2023-12-27 18:34:23,469][20347] Updated weights for policy 0, policy_version 1865772 (0.0007) [2023-12-27 18:34:23,526][20347] Updated weights for policy 0, policy_version 1865782 (0.0007) [2023-12-27 18:34:23,803][20381] Updated weights for policy 1, policy_version 1864193 (0.0008) [2023-12-27 18:34:23,848][20381] Updated weights for policy 1, policy_version 1864203 (0.0008) [2023-12-27 18:34:23,898][20381] Updated weights for policy 1, policy_version 1864213 (0.0008) [2023-12-27 18:34:24,200][20347] Updated weights for policy 0, policy_version 1865792 (0.0008) [2023-12-27 18:34:24,255][20347] Updated weights for policy 0, policy_version 1865802 (0.0008) [2023-12-27 18:34:24,317][20347] Updated weights for policy 0, policy_version 1865812 (0.0007) [2023-12-27 18:34:24,587][20381] Updated weights for policy 1, policy_version 1864223 (0.0008) [2023-12-27 18:34:24,636][20381] Updated weights for policy 1, policy_version 1864233 (0.0008) [2023-12-27 18:34:24,682][20381] Updated weights for policy 1, policy_version 1864243 (0.0007) [2023-12-27 18:34:24,997][20347] Updated weights for policy 0, policy_version 1865822 (0.0008) [2023-12-27 18:34:25,055][20347] Updated weights for policy 0, policy_version 1865832 (0.0008) [2023-12-27 18:34:25,114][20347] Updated weights for policy 0, policy_version 1865842 (0.0008) [2023-12-27 18:34:25,322][20381] Updated weights for policy 1, policy_version 1864253 (0.0008) [2023-12-27 18:34:25,367][20381] Updated weights for policy 1, policy_version 1864263 (0.0007) [2023-12-27 18:34:25,412][20381] Updated weights for policy 1, policy_version 1864273 (0.0007) [2023-12-27 18:34:25,815][20347] Updated weights for policy 0, policy_version 1865852 (0.0008) [2023-12-27 18:34:25,875][20347] Updated weights for policy 0, policy_version 1865862 (0.0009) [2023-12-27 18:34:25,935][20347] Updated weights for policy 0, policy_version 1865872 (0.0008) [2023-12-27 18:34:26,053][20381] Updated weights for policy 1, policy_version 1864283 (0.0008) [2023-12-27 18:34:26,110][20381] Updated weights for policy 1, policy_version 1864293 (0.0005) [2023-12-27 18:34:26,165][20381] Updated weights for policy 1, policy_version 1864303 (0.0006) [2023-12-27 18:34:26,657][20347] Updated weights for policy 0, policy_version 1865882 (0.0007) [2023-12-27 18:34:26,709][20347] Updated weights for policy 0, policy_version 1865892 (0.0007) [2023-12-27 18:34:26,733][20381] Updated weights for policy 1, policy_version 1864313 (0.0007) [2023-12-27 18:34:26,762][20347] Updated weights for policy 0, policy_version 1865902 (0.0008) [2023-12-27 18:34:26,791][20381] Updated weights for policy 1, policy_version 1864323 (0.0005) [2023-12-27 18:34:26,818][20347] Updated weights for policy 0, policy_version 1865912 (0.0008) [2023-12-27 18:34:26,845][20381] Updated weights for policy 1, policy_version 1864333 (0.0005) [2023-12-27 18:34:26,894][20381] Updated weights for policy 1, policy_version 1864343 (0.0005) [2023-12-27 18:34:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 955080704. Throughput: 0: 10189.4, 1: 10222.5. Samples: 955071204. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '836.980')] [2023-12-27 18:34:27,484][20381] Updated weights for policy 1, policy_version 1864353 (0.0006) [2023-12-27 18:34:27,490][20347] Updated weights for policy 0, policy_version 1865922 (0.0006) [2023-12-27 18:34:27,536][20381] Updated weights for policy 1, policy_version 1864363 (0.0006) [2023-12-27 18:34:27,550][20347] Updated weights for policy 0, policy_version 1865932 (0.0007) [2023-12-27 18:34:27,598][20381] Updated weights for policy 1, policy_version 1864373 (0.0007) [2023-12-27 18:34:27,603][20347] Updated weights for policy 0, policy_version 1865942 (0.0006) [2023-12-27 18:34:28,243][20381] Updated weights for policy 1, policy_version 1864383 (0.0008) [2023-12-27 18:34:28,269][20347] Updated weights for policy 0, policy_version 1865952 (0.0007) [2023-12-27 18:34:28,310][20381] Updated weights for policy 1, policy_version 1864393 (0.0008) [2023-12-27 18:34:28,323][20347] Updated weights for policy 0, policy_version 1865962 (0.0006) [2023-12-27 18:34:28,376][20381] Updated weights for policy 1, policy_version 1864403 (0.0008) [2023-12-27 18:34:28,382][20347] Updated weights for policy 0, policy_version 1865972 (0.0008) [2023-12-27 18:34:29,034][20347] Updated weights for policy 0, policy_version 1865982 (0.0007) [2023-12-27 18:34:29,084][20347] Updated weights for policy 0, policy_version 1865992 (0.0007) [2023-12-27 18:34:29,112][20381] Updated weights for policy 1, policy_version 1864413 (0.0008) [2023-12-27 18:34:29,134][20347] Updated weights for policy 0, policy_version 1866002 (0.0006) [2023-12-27 18:34:29,166][20381] Updated weights for policy 1, policy_version 1864423 (0.0005) [2023-12-27 18:34:29,214][20381] Updated weights for policy 1, policy_version 1864433 (0.0006) [2023-12-27 18:34:29,847][20347] Updated weights for policy 0, policy_version 1866012 (0.0008) [2023-12-27 18:34:29,905][20347] Updated weights for policy 0, policy_version 1866022 (0.0007) [2023-12-27 18:34:29,912][20381] Updated weights for policy 1, policy_version 1864443 (0.0009) [2023-12-27 18:34:29,972][20347] Updated weights for policy 0, policy_version 1866032 (0.0007) [2023-12-27 18:34:29,974][20381] Updated weights for policy 1, policy_version 1864453 (0.0007) [2023-12-27 18:34:30,034][20381] Updated weights for policy 1, policy_version 1864463 (0.0008) [2023-12-27 18:34:30,614][20347] Updated weights for policy 0, policy_version 1866042 (0.0006) [2023-12-27 18:34:30,664][20347] Updated weights for policy 0, policy_version 1866052 (0.0007) [2023-12-27 18:34:30,712][20347] Updated weights for policy 0, policy_version 1866062 (0.0006) [2023-12-27 18:34:30,729][20381] Updated weights for policy 1, policy_version 1864473 (0.0009) [2023-12-27 18:34:30,765][20347] Updated weights for policy 0, policy_version 1866072 (0.0006) [2023-12-27 18:34:30,779][20381] Updated weights for policy 1, policy_version 1864483 (0.0006) [2023-12-27 18:34:30,823][20381] Updated weights for policy 1, policy_version 1864493 (0.0007) [2023-12-27 18:34:30,874][20381] Updated weights for policy 1, policy_version 1864503 (0.0009) [2023-12-27 18:34:31,508][20381] Updated weights for policy 1, policy_version 1864513 (0.0006) [2023-12-27 18:34:31,509][20347] Updated weights for policy 0, policy_version 1866082 (0.0006) [2023-12-27 18:34:31,568][20381] Updated weights for policy 1, policy_version 1864523 (0.0007) [2023-12-27 18:34:31,570][20347] Updated weights for policy 0, policy_version 1866092 (0.0007) [2023-12-27 18:34:31,630][20381] Updated weights for policy 1, policy_version 1864533 (0.0006) [2023-12-27 18:34:31,633][20347] Updated weights for policy 0, policy_version 1866102 (0.0007) [2023-12-27 18:34:32,308][20347] Updated weights for policy 0, policy_version 1866112 (0.0008) [2023-12-27 18:34:32,342][20381] Updated weights for policy 1, policy_version 1864543 (0.0007) [2023-12-27 18:34:32,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 955179008. Throughput: 0: 10282.0, 1: 10208.7. Samples: 955195408. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '835.350')] [2023-12-27 18:34:32,364][20347] Updated weights for policy 0, policy_version 1866122 (0.0007) [2023-12-27 18:34:32,406][20381] Updated weights for policy 1, policy_version 1864553 (0.0007) [2023-12-27 18:34:32,421][20347] Updated weights for policy 0, policy_version 1866133 (0.0010) [2023-12-27 18:34:32,462][20381] Updated weights for policy 1, policy_version 1864563 (0.0006) [2023-12-27 18:34:33,103][20347] Updated weights for policy 0, policy_version 1866143 (0.0006) [2023-12-27 18:34:33,141][20381] Updated weights for policy 1, policy_version 1864573 (0.0007) [2023-12-27 18:34:33,147][20347] Updated weights for policy 0, policy_version 1866153 (0.0006) [2023-12-27 18:34:33,186][20381] Updated weights for policy 1, policy_version 1864583 (0.0005) [2023-12-27 18:34:33,191][20347] Updated weights for policy 0, policy_version 1866163 (0.0007) [2023-12-27 18:34:33,242][20381] Updated weights for policy 1, policy_version 1864593 (0.0006) [2023-12-27 18:34:33,861][20381] Updated weights for policy 1, policy_version 1864603 (0.0008) [2023-12-27 18:34:33,862][20347] Updated weights for policy 0, policy_version 1866173 (0.0007) [2023-12-27 18:34:33,915][20381] Updated weights for policy 1, policy_version 1864613 (0.0007) [2023-12-27 18:34:33,920][20347] Updated weights for policy 0, policy_version 1866183 (0.0008) [2023-12-27 18:34:33,972][20381] Updated weights for policy 1, policy_version 1864623 (0.0008) [2023-12-27 18:34:33,976][20347] Updated weights for policy 0, policy_version 1866193 (0.0005) [2023-12-27 18:34:34,601][20347] Updated weights for policy 0, policy_version 1866203 (0.0006) [2023-12-27 18:34:34,638][20381] Updated weights for policy 1, policy_version 1864633 (0.0007) [2023-12-27 18:34:34,657][20347] Updated weights for policy 0, policy_version 1866213 (0.0008) [2023-12-27 18:34:34,693][20381] Updated weights for policy 1, policy_version 1864643 (0.0006) [2023-12-27 18:34:34,712][20347] Updated weights for policy 0, policy_version 1866223 (0.0008) [2023-12-27 18:34:34,754][20381] Updated weights for policy 1, policy_version 1864653 (0.0007) [2023-12-27 18:34:34,810][20381] Updated weights for policy 1, policy_version 1864663 (0.0008) [2023-12-27 18:34:35,333][20347] Updated weights for policy 0, policy_version 1866233 (0.0008) [2023-12-27 18:34:35,381][20347] Updated weights for policy 0, policy_version 1866243 (0.0008) [2023-12-27 18:34:35,435][20347] Updated weights for policy 0, policy_version 1866253 (0.0007) [2023-12-27 18:34:35,455][20381] Updated weights for policy 1, policy_version 1864673 (0.0007) [2023-12-27 18:34:35,484][20347] Updated weights for policy 0, policy_version 1866263 (0.0007) [2023-12-27 18:34:35,515][20381] Updated weights for policy 1, policy_version 1864683 (0.0005) [2023-12-27 18:34:35,571][20381] Updated weights for policy 1, policy_version 1864693 (0.0005) [2023-12-27 18:34:36,204][20347] Updated weights for policy 0, policy_version 1866273 (0.0006) [2023-12-27 18:34:36,226][20381] Updated weights for policy 1, policy_version 1864703 (0.0008) [2023-12-27 18:34:36,263][20347] Updated weights for policy 0, policy_version 1866283 (0.0008) [2023-12-27 18:34:36,275][20381] Updated weights for policy 1, policy_version 1864713 (0.0007) [2023-12-27 18:34:36,322][20381] Updated weights for policy 1, policy_version 1864723 (0.0007) [2023-12-27 18:34:36,328][20347] Updated weights for policy 0, policy_version 1866293 (0.0009) [2023-12-27 18:34:36,969][20347] Updated weights for policy 0, policy_version 1866303 (0.0007) [2023-12-27 18:34:36,981][20381] Updated weights for policy 1, policy_version 1864733 (0.0007) [2023-12-27 18:34:37,029][20347] Updated weights for policy 0, policy_version 1866313 (0.0006) [2023-12-27 18:34:37,035][20381] Updated weights for policy 1, policy_version 1864743 (0.0009) [2023-12-27 18:34:37,082][20381] Updated weights for policy 1, policy_version 1864753 (0.0011) [2023-12-27 18:34:37,088][20347] Updated weights for policy 0, policy_version 1866323 (0.0006) [2023-12-27 18:34:37,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 955293696. Throughput: 0: 10311.9, 1: 10228.8. Samples: 955259168. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '797.840')] [2023-12-27 18:34:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001864760_477429760.pth... [2023-12-27 18:34:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001866328_477863936.pth... [2023-12-27 18:34:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001863544_477118464.pth [2023-12-27 18:34:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001865144_477560832.pth [2023-12-27 18:34:37,697][20347] Updated weights for policy 0, policy_version 1866333 (0.0006) [2023-12-27 18:34:37,725][20381] Updated weights for policy 1, policy_version 1864763 (0.0011) [2023-12-27 18:34:37,750][20347] Updated weights for policy 0, policy_version 1866343 (0.0010) [2023-12-27 18:34:37,781][20381] Updated weights for policy 1, policy_version 1864773 (0.0011) [2023-12-27 18:34:37,806][20347] Updated weights for policy 0, policy_version 1866353 (0.0007) [2023-12-27 18:34:37,841][20381] Updated weights for policy 1, policy_version 1864783 (0.0011) [2023-12-27 18:34:38,481][20381] Updated weights for policy 1, policy_version 1864793 (0.0008) [2023-12-27 18:34:38,528][20347] Updated weights for policy 0, policy_version 1866363 (0.0008) [2023-12-27 18:34:38,549][20381] Updated weights for policy 1, policy_version 1864803 (0.0009) [2023-12-27 18:34:38,584][20347] Updated weights for policy 0, policy_version 1866373 (0.0009) [2023-12-27 18:34:38,602][20381] Updated weights for policy 1, policy_version 1864813 (0.0009) [2023-12-27 18:34:38,632][20347] Updated weights for policy 0, policy_version 1866383 (0.0008) [2023-12-27 18:34:38,659][20381] Updated weights for policy 1, policy_version 1864823 (0.0009) [2023-12-27 18:34:39,424][20381] Updated weights for policy 1, policy_version 1864833 (0.0007) [2023-12-27 18:34:39,457][20347] Updated weights for policy 0, policy_version 1866394 (0.0009) [2023-12-27 18:34:39,488][20381] Updated weights for policy 1, policy_version 1864843 (0.0006) [2023-12-27 18:34:39,511][20347] Updated weights for policy 0, policy_version 1866404 (0.0007) [2023-12-27 18:34:39,550][20381] Updated weights for policy 1, policy_version 1864853 (0.0006) [2023-12-27 18:34:39,572][20347] Updated weights for policy 0, policy_version 1866414 (0.0008) [2023-12-27 18:34:39,631][20347] Updated weights for policy 0, policy_version 1866424 (0.0007) [2023-12-27 18:34:40,247][20381] Updated weights for policy 1, policy_version 1864863 (0.0007) [2023-12-27 18:34:40,300][20347] Updated weights for policy 0, policy_version 1866434 (0.0007) [2023-12-27 18:34:40,303][20381] Updated weights for policy 1, policy_version 1864873 (0.0008) [2023-12-27 18:34:40,361][20347] Updated weights for policy 0, policy_version 1866444 (0.0007) [2023-12-27 18:34:40,362][20381] Updated weights for policy 1, policy_version 1864883 (0.0008) [2023-12-27 18:34:40,418][20347] Updated weights for policy 0, policy_version 1866454 (0.0005) [2023-12-27 18:34:41,025][20347] Updated weights for policy 0, policy_version 1866464 (0.0006) [2023-12-27 18:34:41,088][20347] Updated weights for policy 0, policy_version 1866474 (0.0008) [2023-12-27 18:34:41,110][20381] Updated weights for policy 1, policy_version 1864893 (0.0007) [2023-12-27 18:34:41,151][20347] Updated weights for policy 0, policy_version 1866484 (0.0007) [2023-12-27 18:34:41,176][20381] Updated weights for policy 1, policy_version 1864903 (0.0008) [2023-12-27 18:34:41,237][20381] Updated weights for policy 1, policy_version 1864913 (0.0008) [2023-12-27 18:34:41,920][20347] Updated weights for policy 0, policy_version 1866494 (0.0008) [2023-12-27 18:34:41,978][20347] Updated weights for policy 0, policy_version 1866504 (0.0007) [2023-12-27 18:34:42,003][20381] Updated weights for policy 1, policy_version 1864923 (0.0008) [2023-12-27 18:34:42,040][20347] Updated weights for policy 0, policy_version 1866514 (0.0008) [2023-12-27 18:34:42,064][20381] Updated weights for policy 1, policy_version 1864933 (0.0006) [2023-12-27 18:34:42,116][20381] Updated weights for policy 1, policy_version 1864943 (0.0007) [2023-12-27 18:34:42,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.6, 300 sec: 20438.3). Total num frames: 955392000. Throughput: 0: 10262.6, 1: 10245.1. Samples: 955381108. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '806.470')] [2023-12-27 18:34:42,742][20347] Updated weights for policy 0, policy_version 1866524 (0.0008) [2023-12-27 18:34:42,801][20347] Updated weights for policy 0, policy_version 1866534 (0.0007) [2023-12-27 18:34:42,822][20381] Updated weights for policy 1, policy_version 1864953 (0.0008) [2023-12-27 18:34:42,855][20347] Updated weights for policy 0, policy_version 1866544 (0.0006) [2023-12-27 18:34:42,889][20381] Updated weights for policy 1, policy_version 1864963 (0.0008) [2023-12-27 18:34:42,951][20381] Updated weights for policy 1, policy_version 1864973 (0.0008) [2023-12-27 18:34:43,003][20381] Updated weights for policy 1, policy_version 1864983 (0.0007) [2023-12-27 18:34:43,534][20347] Updated weights for policy 0, policy_version 1866554 (0.0006) [2023-12-27 18:34:43,594][20347] Updated weights for policy 0, policy_version 1866564 (0.0008) [2023-12-27 18:34:43,651][20347] Updated weights for policy 0, policy_version 1866574 (0.0007) [2023-12-27 18:34:43,694][20381] Updated weights for policy 1, policy_version 1864993 (0.0007) [2023-12-27 18:34:43,707][20347] Updated weights for policy 0, policy_version 1866584 (0.0006) [2023-12-27 18:34:43,751][20381] Updated weights for policy 1, policy_version 1865003 (0.0007) [2023-12-27 18:34:43,806][20381] Updated weights for policy 1, policy_version 1865013 (0.0008) [2023-12-27 18:34:44,375][20347] Updated weights for policy 0, policy_version 1866594 (0.0008) [2023-12-27 18:34:44,432][20347] Updated weights for policy 0, policy_version 1866604 (0.0007) [2023-12-27 18:34:44,460][20381] Updated weights for policy 1, policy_version 1865023 (0.0006) [2023-12-27 18:34:44,485][20347] Updated weights for policy 0, policy_version 1866614 (0.0007) [2023-12-27 18:34:44,508][20381] Updated weights for policy 1, policy_version 1865033 (0.0006) [2023-12-27 18:34:44,553][20381] Updated weights for policy 1, policy_version 1865043 (0.0007) [2023-12-27 18:34:45,210][20347] Updated weights for policy 0, policy_version 1866624 (0.0008) [2023-12-27 18:34:45,266][20347] Updated weights for policy 0, policy_version 1866634 (0.0008) [2023-12-27 18:34:45,278][20381] Updated weights for policy 1, policy_version 1865053 (0.0007) [2023-12-27 18:34:45,330][20347] Updated weights for policy 0, policy_version 1866644 (0.0007) [2023-12-27 18:34:45,340][20381] Updated weights for policy 1, policy_version 1865063 (0.0006) [2023-12-27 18:34:45,393][20381] Updated weights for policy 1, policy_version 1865073 (0.0007) [2023-12-27 18:34:46,006][20347] Updated weights for policy 0, policy_version 1866654 (0.0007) [2023-12-27 18:34:46,057][20347] Updated weights for policy 0, policy_version 1866664 (0.0008) [2023-12-27 18:34:46,117][20347] Updated weights for policy 0, policy_version 1866674 (0.0007) [2023-12-27 18:34:46,120][20381] Updated weights for policy 1, policy_version 1865083 (0.0008) [2023-12-27 18:34:46,179][20381] Updated weights for policy 1, policy_version 1865093 (0.0011) [2023-12-27 18:34:46,240][20381] Updated weights for policy 1, policy_version 1865103 (0.0011) [2023-12-27 18:34:46,761][20347] Updated weights for policy 0, policy_version 1866684 (0.0008) [2023-12-27 18:34:46,813][20347] Updated weights for policy 0, policy_version 1866694 (0.0010) [2023-12-27 18:34:46,867][20347] Updated weights for policy 0, policy_version 1866704 (0.0010) [2023-12-27 18:34:46,930][20381] Updated weights for policy 1, policy_version 1865113 (0.0011) [2023-12-27 18:34:46,990][20381] Updated weights for policy 1, policy_version 1865123 (0.0011) [2023-12-27 18:34:47,046][20381] Updated weights for policy 1, policy_version 1865133 (0.0011) [2023-12-27 18:34:47,106][20381] Updated weights for policy 1, policy_version 1865143 (0.0011) [2023-12-27 18:34:47,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 955490304. Throughput: 0: 10213.1, 1: 10256.9. Samples: 955500004. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '902.800')] [2023-12-27 18:34:47,525][20347] Updated weights for policy 0, policy_version 1866714 (0.0009) [2023-12-27 18:34:47,574][20347] Updated weights for policy 0, policy_version 1866724 (0.0005) [2023-12-27 18:34:47,630][20347] Updated weights for policy 0, policy_version 1866734 (0.0005) [2023-12-27 18:34:47,686][20347] Updated weights for policy 0, policy_version 1866744 (0.0007) [2023-12-27 18:34:47,795][20381] Updated weights for policy 1, policy_version 1865153 (0.0008) [2023-12-27 18:34:47,855][20381] Updated weights for policy 1, policy_version 1865163 (0.0008) [2023-12-27 18:34:47,916][20381] Updated weights for policy 1, policy_version 1865173 (0.0007) [2023-12-27 18:34:48,368][20347] Updated weights for policy 0, policy_version 1866754 (0.0010) [2023-12-27 18:34:48,420][20347] Updated weights for policy 0, policy_version 1866764 (0.0010) [2023-12-27 18:34:48,483][20347] Updated weights for policy 0, policy_version 1866774 (0.0009) [2023-12-27 18:34:48,572][20381] Updated weights for policy 1, policy_version 1865183 (0.0006) [2023-12-27 18:34:48,623][20381] Updated weights for policy 1, policy_version 1865193 (0.0006) [2023-12-27 18:34:48,675][20381] Updated weights for policy 1, policy_version 1865203 (0.0011) [2023-12-27 18:34:49,189][20347] Updated weights for policy 0, policy_version 1866784 (0.0007) [2023-12-27 18:34:49,253][20347] Updated weights for policy 0, policy_version 1866794 (0.0007) [2023-12-27 18:34:49,315][20347] Updated weights for policy 0, policy_version 1866804 (0.0006) [2023-12-27 18:34:49,335][20381] Updated weights for policy 1, policy_version 1865213 (0.0010) [2023-12-27 18:34:49,393][20381] Updated weights for policy 1, policy_version 1865223 (0.0010) [2023-12-27 18:34:49,455][20381] Updated weights for policy 1, policy_version 1865233 (0.0010) [2023-12-27 18:34:50,042][20347] Updated weights for policy 0, policy_version 1866814 (0.0008) [2023-12-27 18:34:50,106][20347] Updated weights for policy 0, policy_version 1866824 (0.0007) [2023-12-27 18:34:50,164][20347] Updated weights for policy 0, policy_version 1866834 (0.0007) [2023-12-27 18:34:50,190][20381] Updated weights for policy 1, policy_version 1865243 (0.0010) [2023-12-27 18:34:50,253][20381] Updated weights for policy 1, policy_version 1865253 (0.0008) [2023-12-27 18:34:50,313][20381] Updated weights for policy 1, policy_version 1865263 (0.0005) [2023-12-27 18:34:50,764][20347] Updated weights for policy 0, policy_version 1866844 (0.0007) [2023-12-27 18:34:50,818][20347] Updated weights for policy 0, policy_version 1866854 (0.0006) [2023-12-27 18:34:50,870][20347] Updated weights for policy 0, policy_version 1866864 (0.0006) [2023-12-27 18:34:50,984][20381] Updated weights for policy 1, policy_version 1865273 (0.0008) [2023-12-27 18:34:51,050][20381] Updated weights for policy 1, policy_version 1865283 (0.0008) [2023-12-27 18:34:51,102][20381] Updated weights for policy 1, policy_version 1865293 (0.0006) [2023-12-27 18:34:51,162][20381] Updated weights for policy 1, policy_version 1865303 (0.0007) [2023-12-27 18:34:51,554][20347] Updated weights for policy 0, policy_version 1866874 (0.0006) [2023-12-27 18:34:51,607][20347] Updated weights for policy 0, policy_version 1866884 (0.0007) [2023-12-27 18:34:51,673][20347] Updated weights for policy 0, policy_version 1866894 (0.0008) [2023-12-27 18:34:51,735][20347] Updated weights for policy 0, policy_version 1866904 (0.0007) [2023-12-27 18:34:51,868][20381] Updated weights for policy 1, policy_version 1865313 (0.0008) [2023-12-27 18:34:51,923][20381] Updated weights for policy 1, policy_version 1865323 (0.0008) [2023-12-27 18:34:51,980][20381] Updated weights for policy 1, policy_version 1865333 (0.0008) [2023-12-27 18:34:52,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 955588608. Throughput: 0: 10166.9, 1: 10259.2. Samples: 955560512. Policy #0 lag: (min: 12.0, avg: 12.3, max: 23.0) [2023-12-27 18:34:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '788.390')] [2023-12-27 18:34:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001865336_477577216.pth... [2023-12-27 18:34:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001864152_477274112.pth [2023-12-27 18:34:52,400][20347] Updated weights for policy 0, policy_version 1866914 (0.0007) [2023-12-27 18:34:52,457][20347] Updated weights for policy 0, policy_version 1866924 (0.0007) [2023-12-27 18:34:52,513][20347] Updated weights for policy 0, policy_version 1866934 (0.0007) [2023-12-27 18:34:52,525][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001866936_478019584.pth... [2023-12-27 18:34:52,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001865720_477708288.pth [2023-12-27 18:34:52,723][20381] Updated weights for policy 1, policy_version 1865343 (0.0007) [2023-12-27 18:34:52,782][20381] Updated weights for policy 1, policy_version 1865353 (0.0007) [2023-12-27 18:34:52,838][20381] Updated weights for policy 1, policy_version 1865363 (0.0008) [2023-12-27 18:34:53,230][20347] Updated weights for policy 0, policy_version 1866944 (0.0007) [2023-12-27 18:34:53,281][20347] Updated weights for policy 0, policy_version 1866954 (0.0007) [2023-12-27 18:34:53,342][20347] Updated weights for policy 0, policy_version 1866964 (0.0008) [2023-12-27 18:34:53,448][20381] Updated weights for policy 1, policy_version 1865373 (0.0006) [2023-12-27 18:34:53,496][20381] Updated weights for policy 1, policy_version 1865383 (0.0005) [2023-12-27 18:34:53,541][20381] Updated weights for policy 1, policy_version 1865393 (0.0005) [2023-12-27 18:34:54,068][20347] Updated weights for policy 0, policy_version 1866974 (0.0009) [2023-12-27 18:34:54,121][20381] Updated weights for policy 1, policy_version 1865403 (0.0006) [2023-12-27 18:34:54,133][20347] Updated weights for policy 0, policy_version 1866984 (0.0010) [2023-12-27 18:34:54,174][20381] Updated weights for policy 1, policy_version 1865413 (0.0006) [2023-12-27 18:34:54,194][20347] Updated weights for policy 0, policy_version 1866994 (0.0010) [2023-12-27 18:34:54,236][20381] Updated weights for policy 1, policy_version 1865423 (0.0005) [2023-12-27 18:34:54,844][20347] Updated weights for policy 0, policy_version 1867004 (0.0010) [2023-12-27 18:34:54,896][20347] Updated weights for policy 0, policy_version 1867014 (0.0010) [2023-12-27 18:34:54,924][20381] Updated weights for policy 1, policy_version 1865433 (0.0008) [2023-12-27 18:34:54,945][20347] Updated weights for policy 0, policy_version 1867024 (0.0010) [2023-12-27 18:34:54,972][20381] Updated weights for policy 1, policy_version 1865443 (0.0010) [2023-12-27 18:34:55,033][20381] Updated weights for policy 1, policy_version 1865453 (0.0008) [2023-12-27 18:34:55,098][20381] Updated weights for policy 1, policy_version 1865463 (0.0008) [2023-12-27 18:34:55,602][20347] Updated weights for policy 0, policy_version 1867034 (0.0008) [2023-12-27 18:34:55,651][20347] Updated weights for policy 0, policy_version 1867044 (0.0010) [2023-12-27 18:34:55,706][20347] Updated weights for policy 0, policy_version 1867054 (0.0010) [2023-12-27 18:34:55,719][20381] Updated weights for policy 1, policy_version 1865473 (0.0005) [2023-12-27 18:34:55,761][20347] Updated weights for policy 0, policy_version 1867064 (0.0010) [2023-12-27 18:34:55,772][20381] Updated weights for policy 1, policy_version 1865483 (0.0006) [2023-12-27 18:34:55,820][20381] Updated weights for policy 1, policy_version 1865493 (0.0007) [2023-12-27 18:34:56,398][20381] Updated weights for policy 1, policy_version 1865503 (0.0006) [2023-12-27 18:34:56,414][20347] Updated weights for policy 0, policy_version 1867074 (0.0010) [2023-12-27 18:34:56,454][20381] Updated weights for policy 1, policy_version 1865513 (0.0005) [2023-12-27 18:34:56,471][20347] Updated weights for policy 0, policy_version 1867084 (0.0010) [2023-12-27 18:34:56,509][20381] Updated weights for policy 1, policy_version 1865523 (0.0005) [2023-12-27 18:34:56,513][20347] Updated weights for policy 0, policy_version 1867094 (0.0010) [2023-12-27 18:34:56,994][20381] Updated weights for policy 1, policy_version 1865533 (0.0005) [2023-12-27 18:34:57,042][20381] Updated weights for policy 1, policy_version 1865543 (0.0008) [2023-12-27 18:34:57,087][20381] Updated weights for policy 1, policy_version 1865553 (0.0010) [2023-12-27 18:34:57,185][20347] Updated weights for policy 0, policy_version 1867104 (0.0010) [2023-12-27 18:34:57,229][20347] Updated weights for policy 0, policy_version 1867114 (0.0010) [2023-12-27 18:34:57,273][20347] Updated weights for policy 0, policy_version 1867124 (0.0010) [2023-12-27 18:34:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20466.8). Total num frames: 955703296. Throughput: 0: 10164.7, 1: 10254.6. Samples: 955684800. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:34:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '778.490')] [2023-12-27 18:34:57,765][20381] Updated weights for policy 1, policy_version 1865563 (0.0010) [2023-12-27 18:34:57,812][20381] Updated weights for policy 1, policy_version 1865573 (0.0008) [2023-12-27 18:34:57,857][20381] Updated weights for policy 1, policy_version 1865583 (0.0006) [2023-12-27 18:34:57,923][20347] Updated weights for policy 0, policy_version 1867134 (0.0007) [2023-12-27 18:34:57,989][20347] Updated weights for policy 0, policy_version 1867144 (0.0006) [2023-12-27 18:34:58,047][20347] Updated weights for policy 0, policy_version 1867154 (0.0006) [2023-12-27 18:34:58,520][20381] Updated weights for policy 1, policy_version 1865593 (0.0005) [2023-12-27 18:34:58,580][20381] Updated weights for policy 1, policy_version 1865603 (0.0007) [2023-12-27 18:34:58,642][20381] Updated weights for policy 1, policy_version 1865613 (0.0007) [2023-12-27 18:34:58,695][20381] Updated weights for policy 1, policy_version 1865623 (0.0005) [2023-12-27 18:34:58,745][20347] Updated weights for policy 0, policy_version 1867164 (0.0009) [2023-12-27 18:34:58,804][20347] Updated weights for policy 0, policy_version 1867174 (0.0009) [2023-12-27 18:34:58,871][20347] Updated weights for policy 0, policy_version 1867184 (0.0010) [2023-12-27 18:34:59,571][20381] Updated weights for policy 1, policy_version 1865633 (0.0007) [2023-12-27 18:34:59,622][20347] Updated weights for policy 0, policy_version 1867194 (0.0008) [2023-12-27 18:34:59,628][20381] Updated weights for policy 1, policy_version 1865643 (0.0007) [2023-12-27 18:34:59,677][20347] Updated weights for policy 0, policy_version 1867204 (0.0006) [2023-12-27 18:34:59,689][20381] Updated weights for policy 1, policy_version 1865653 (0.0008) [2023-12-27 18:34:59,734][20347] Updated weights for policy 0, policy_version 1867214 (0.0006) [2023-12-27 18:34:59,789][20347] Updated weights for policy 0, policy_version 1867224 (0.0005) [2023-12-27 18:35:00,402][20347] Updated weights for policy 0, policy_version 1867234 (0.0010) [2023-12-27 18:35:00,461][20347] Updated weights for policy 0, policy_version 1867244 (0.0011) [2023-12-27 18:35:00,503][20381] Updated weights for policy 1, policy_version 1865663 (0.0009) [2023-12-27 18:35:00,514][20347] Updated weights for policy 0, policy_version 1867254 (0.0010) [2023-12-27 18:35:00,554][20381] Updated weights for policy 1, policy_version 1865673 (0.0009) [2023-12-27 18:35:00,611][20381] Updated weights for policy 1, policy_version 1865683 (0.0009) [2023-12-27 18:35:01,138][20347] Updated weights for policy 0, policy_version 1867264 (0.0011) [2023-12-27 18:35:01,197][20347] Updated weights for policy 0, policy_version 1867274 (0.0010) [2023-12-27 18:35:01,259][20347] Updated weights for policy 0, policy_version 1867284 (0.0010) [2023-12-27 18:35:01,354][20381] Updated weights for policy 1, policy_version 1865693 (0.0009) [2023-12-27 18:35:01,420][20381] Updated weights for policy 1, policy_version 1865703 (0.0007) [2023-12-27 18:35:01,477][20381] Updated weights for policy 1, policy_version 1865713 (0.0007) [2023-12-27 18:35:01,977][20347] Updated weights for policy 0, policy_version 1867294 (0.0009) [2023-12-27 18:35:02,032][20347] Updated weights for policy 0, policy_version 1867304 (0.0007) [2023-12-27 18:35:02,084][20347] Updated weights for policy 0, policy_version 1867314 (0.0005) [2023-12-27 18:35:02,142][20381] Updated weights for policy 1, policy_version 1865723 (0.0007) [2023-12-27 18:35:02,213][20381] Updated weights for policy 1, policy_version 1865733 (0.0008) [2023-12-27 18:35:02,272][20381] Updated weights for policy 1, policy_version 1865743 (0.0008) [2023-12-27 18:35:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20410.6). Total num frames: 955801600. Throughput: 0: 10167.9, 1: 10250.3. Samples: 955806888. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '797.050')] [2023-12-27 18:35:02,721][20347] Updated weights for policy 0, policy_version 1867324 (0.0009) [2023-12-27 18:35:02,778][20347] Updated weights for policy 0, policy_version 1867334 (0.0010) [2023-12-27 18:35:02,833][20347] Updated weights for policy 0, policy_version 1867344 (0.0010) [2023-12-27 18:35:02,982][20381] Updated weights for policy 1, policy_version 1865753 (0.0008) [2023-12-27 18:35:03,037][20381] Updated weights for policy 1, policy_version 1865763 (0.0011) [2023-12-27 18:35:03,093][20381] Updated weights for policy 1, policy_version 1865773 (0.0011) [2023-12-27 18:35:03,150][20381] Updated weights for policy 1, policy_version 1865783 (0.0010) [2023-12-27 18:35:03,434][20347] Updated weights for policy 0, policy_version 1867354 (0.0009) [2023-12-27 18:35:03,483][20347] Updated weights for policy 0, policy_version 1867364 (0.0005) [2023-12-27 18:35:03,533][20347] Updated weights for policy 0, policy_version 1867374 (0.0008) [2023-12-27 18:35:03,582][20347] Updated weights for policy 0, policy_version 1867384 (0.0010) [2023-12-27 18:35:03,778][20381] Updated weights for policy 1, policy_version 1865793 (0.0010) [2023-12-27 18:35:03,826][20381] Updated weights for policy 1, policy_version 1865803 (0.0010) [2023-12-27 18:35:03,885][20381] Updated weights for policy 1, policy_version 1865813 (0.0010) [2023-12-27 18:35:04,206][20347] Updated weights for policy 0, policy_version 1867394 (0.0008) [2023-12-27 18:35:04,263][20347] Updated weights for policy 0, policy_version 1867404 (0.0006) [2023-12-27 18:35:04,320][20347] Updated weights for policy 0, policy_version 1867414 (0.0005) [2023-12-27 18:35:04,590][20381] Updated weights for policy 1, policy_version 1865823 (0.0010) [2023-12-27 18:35:04,652][20381] Updated weights for policy 1, policy_version 1865833 (0.0007) [2023-12-27 18:35:04,703][20381] Updated weights for policy 1, policy_version 1865843 (0.0010) [2023-12-27 18:35:04,877][20347] Updated weights for policy 0, policy_version 1867424 (0.0005) [2023-12-27 18:35:04,921][20347] Updated weights for policy 0, policy_version 1867434 (0.0006) [2023-12-27 18:35:04,966][20347] Updated weights for policy 0, policy_version 1867444 (0.0006) [2023-12-27 18:35:05,355][20381] Updated weights for policy 1, policy_version 1865853 (0.0008) [2023-12-27 18:35:05,410][20381] Updated weights for policy 1, policy_version 1865863 (0.0007) [2023-12-27 18:35:05,465][20381] Updated weights for policy 1, policy_version 1865873 (0.0011) [2023-12-27 18:35:05,602][20347] Updated weights for policy 0, policy_version 1867454 (0.0006) [2023-12-27 18:35:05,657][20347] Updated weights for policy 0, policy_version 1867464 (0.0007) [2023-12-27 18:35:05,703][20347] Updated weights for policy 0, policy_version 1867474 (0.0007) [2023-12-27 18:35:06,124][20381] Updated weights for policy 1, policy_version 1865883 (0.0011) [2023-12-27 18:35:06,184][20381] Updated weights for policy 1, policy_version 1865893 (0.0011) [2023-12-27 18:35:06,241][20381] Updated weights for policy 1, policy_version 1865903 (0.0011) [2023-12-27 18:35:06,393][20347] Updated weights for policy 0, policy_version 1867484 (0.0008) [2023-12-27 18:35:06,448][20347] Updated weights for policy 0, policy_version 1867494 (0.0007) [2023-12-27 18:35:06,506][20347] Updated weights for policy 0, policy_version 1867504 (0.0007) [2023-12-27 18:35:06,935][20381] Updated weights for policy 1, policy_version 1865913 (0.0007) [2023-12-27 18:35:06,989][20381] Updated weights for policy 1, policy_version 1865923 (0.0010) [2023-12-27 18:35:07,050][20381] Updated weights for policy 1, policy_version 1865933 (0.0007) [2023-12-27 18:35:07,103][20381] Updated weights for policy 1, policy_version 1865943 (0.0007) [2023-12-27 18:35:07,186][20347] Updated weights for policy 0, policy_version 1867514 (0.0007) [2023-12-27 18:35:07,246][20347] Updated weights for policy 0, policy_version 1867524 (0.0007) [2023-12-27 18:35:07,305][20347] Updated weights for policy 0, policy_version 1867534 (0.0008) [2023-12-27 18:35:07,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.4, 300 sec: 20438.3). Total num frames: 955908096. Throughput: 0: 10245.7, 1: 10231.6. Samples: 955871668. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:07,363][20347] Updated weights for policy 0, policy_version 1867544 (0.0007) [2023-12-27 18:35:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '782.460')] [2023-12-27 18:35:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001865944_477732864.pth... [2023-12-27 18:35:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001867544_478175232.pth... [2023-12-27 18:35:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001866328_477863936.pth [2023-12-27 18:35:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001864760_477429760.pth [2023-12-27 18:35:07,796][20381] Updated weights for policy 1, policy_version 1865953 (0.0006) [2023-12-27 18:35:07,847][20381] Updated weights for policy 1, policy_version 1865963 (0.0006) [2023-12-27 18:35:07,904][20381] Updated weights for policy 1, policy_version 1865973 (0.0008) [2023-12-27 18:35:07,989][20347] Updated weights for policy 0, policy_version 1867554 (0.0007) [2023-12-27 18:35:08,040][20347] Updated weights for policy 0, policy_version 1867564 (0.0007) [2023-12-27 18:35:08,089][20347] Updated weights for policy 0, policy_version 1867574 (0.0008) [2023-12-27 18:35:08,549][20381] Updated weights for policy 1, policy_version 1865983 (0.0007) [2023-12-27 18:35:08,605][20381] Updated weights for policy 1, policy_version 1865993 (0.0008) [2023-12-27 18:35:08,665][20381] Updated weights for policy 1, policy_version 1866003 (0.0008) [2023-12-27 18:35:08,836][20347] Updated weights for policy 0, policy_version 1867584 (0.0007) [2023-12-27 18:35:08,899][20347] Updated weights for policy 0, policy_version 1867594 (0.0007) [2023-12-27 18:35:08,960][20347] Updated weights for policy 0, policy_version 1867604 (0.0006) [2023-12-27 18:35:09,299][20381] Updated weights for policy 1, policy_version 1866013 (0.0008) [2023-12-27 18:35:09,359][20381] Updated weights for policy 1, policy_version 1866023 (0.0008) [2023-12-27 18:35:09,426][20381] Updated weights for policy 1, policy_version 1866033 (0.0008) [2023-12-27 18:35:09,679][20347] Updated weights for policy 0, policy_version 1867614 (0.0006) [2023-12-27 18:35:09,739][20347] Updated weights for policy 0, policy_version 1867624 (0.0007) [2023-12-27 18:35:09,796][20347] Updated weights for policy 0, policy_version 1867634 (0.0008) [2023-12-27 18:35:10,115][20381] Updated weights for policy 1, policy_version 1866043 (0.0007) [2023-12-27 18:35:10,162][20381] Updated weights for policy 1, policy_version 1866053 (0.0008) [2023-12-27 18:35:10,213][20381] Updated weights for policy 1, policy_version 1866063 (0.0008) [2023-12-27 18:35:10,528][20347] Updated weights for policy 0, policy_version 1867644 (0.0008) [2023-12-27 18:35:10,581][20347] Updated weights for policy 0, policy_version 1867654 (0.0008) [2023-12-27 18:35:10,636][20347] Updated weights for policy 0, policy_version 1867664 (0.0008) [2023-12-27 18:35:10,936][20381] Updated weights for policy 1, policy_version 1866073 (0.0008) [2023-12-27 18:35:10,994][20381] Updated weights for policy 1, policy_version 1866083 (0.0007) [2023-12-27 18:35:11,058][20381] Updated weights for policy 1, policy_version 1866093 (0.0008) [2023-12-27 18:35:11,119][20381] Updated weights for policy 1, policy_version 1866103 (0.0008) [2023-12-27 18:35:11,374][20347] Updated weights for policy 0, policy_version 1867674 (0.0008) [2023-12-27 18:35:11,435][20347] Updated weights for policy 0, policy_version 1867684 (0.0008) [2023-12-27 18:35:11,481][20347] Updated weights for policy 0, policy_version 1867694 (0.0008) [2023-12-27 18:35:11,533][20347] Updated weights for policy 0, policy_version 1867704 (0.0007) [2023-12-27 18:35:11,815][20381] Updated weights for policy 1, policy_version 1866113 (0.0008) [2023-12-27 18:35:11,871][20381] Updated weights for policy 1, policy_version 1866123 (0.0008) [2023-12-27 18:35:11,925][20381] Updated weights for policy 1, policy_version 1866133 (0.0007) [2023-12-27 18:35:12,279][20347] Updated weights for policy 0, policy_version 1867714 (0.0008) [2023-12-27 18:35:12,347][20347] Updated weights for policy 0, policy_version 1867724 (0.0008) [2023-12-27 18:35:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20410.6). Total num frames: 955998208. Throughput: 0: 10263.7, 1: 10218.1. Samples: 955992888. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '878.940')] [2023-12-27 18:35:12,413][20347] Updated weights for policy 0, policy_version 1867734 (0.0009) [2023-12-27 18:35:12,647][20381] Updated weights for policy 1, policy_version 1866143 (0.0007) [2023-12-27 18:35:12,704][20381] Updated weights for policy 1, policy_version 1866153 (0.0008) [2023-12-27 18:35:12,764][20381] Updated weights for policy 1, policy_version 1866163 (0.0006) [2023-12-27 18:35:13,138][20347] Updated weights for policy 0, policy_version 1867744 (0.0008) [2023-12-27 18:35:13,195][20347] Updated weights for policy 0, policy_version 1867754 (0.0005) [2023-12-27 18:35:13,247][20347] Updated weights for policy 0, policy_version 1867764 (0.0005) [2023-12-27 18:35:13,422][20381] Updated weights for policy 1, policy_version 1866173 (0.0006) [2023-12-27 18:35:13,480][20381] Updated weights for policy 1, policy_version 1866183 (0.0007) [2023-12-27 18:35:13,531][20381] Updated weights for policy 1, policy_version 1866193 (0.0007) [2023-12-27 18:35:13,899][20347] Updated weights for policy 0, policy_version 1867774 (0.0008) [2023-12-27 18:35:13,957][20347] Updated weights for policy 0, policy_version 1867785 (0.0006) [2023-12-27 18:35:14,010][20347] Updated weights for policy 0, policy_version 1867795 (0.0005) [2023-12-27 18:35:14,188][20381] Updated weights for policy 1, policy_version 1866203 (0.0007) [2023-12-27 18:35:14,244][20381] Updated weights for policy 1, policy_version 1866213 (0.0008) [2023-12-27 18:35:14,308][20381] Updated weights for policy 1, policy_version 1866223 (0.0007) [2023-12-27 18:35:14,661][20347] Updated weights for policy 0, policy_version 1867805 (0.0006) [2023-12-27 18:35:14,720][20347] Updated weights for policy 0, policy_version 1867815 (0.0006) [2023-12-27 18:35:14,773][20347] Updated weights for policy 0, policy_version 1867825 (0.0006) [2023-12-27 18:35:14,974][20381] Updated weights for policy 1, policy_version 1866233 (0.0007) [2023-12-27 18:35:15,040][20381] Updated weights for policy 1, policy_version 1866243 (0.0006) [2023-12-27 18:35:15,100][20381] Updated weights for policy 1, policy_version 1866253 (0.0010) [2023-12-27 18:35:15,163][20381] Updated weights for policy 1, policy_version 1866263 (0.0011) [2023-12-27 18:35:15,411][20347] Updated weights for policy 0, policy_version 1867835 (0.0008) [2023-12-27 18:35:15,466][20347] Updated weights for policy 0, policy_version 1867845 (0.0010) [2023-12-27 18:35:15,515][20347] Updated weights for policy 0, policy_version 1867855 (0.0010) [2023-12-27 18:35:15,836][20381] Updated weights for policy 1, policy_version 1866273 (0.0007) [2023-12-27 18:35:15,895][20381] Updated weights for policy 1, policy_version 1866283 (0.0008) [2023-12-27 18:35:15,960][20381] Updated weights for policy 1, policy_version 1866293 (0.0006) [2023-12-27 18:35:16,269][20347] Updated weights for policy 0, policy_version 1867865 (0.0009) [2023-12-27 18:35:16,327][20347] Updated weights for policy 0, policy_version 1867875 (0.0007) [2023-12-27 18:35:16,380][20347] Updated weights for policy 0, policy_version 1867885 (0.0007) [2023-12-27 18:35:16,444][20347] Updated weights for policy 0, policy_version 1867895 (0.0006) [2023-12-27 18:35:16,636][20381] Updated weights for policy 1, policy_version 1866303 (0.0009) [2023-12-27 18:35:16,696][20381] Updated weights for policy 1, policy_version 1866313 (0.0010) [2023-12-27 18:35:16,751][20381] Updated weights for policy 1, policy_version 1866323 (0.0010) [2023-12-27 18:35:17,085][20347] Updated weights for policy 0, policy_version 1867905 (0.0010) [2023-12-27 18:35:17,146][20347] Updated weights for policy 0, policy_version 1867915 (0.0010) [2023-12-27 18:35:17,197][20347] Updated weights for policy 0, policy_version 1867925 (0.0007) [2023-12-27 18:35:17,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 956104704. Throughput: 0: 10261.4, 1: 10169.6. Samples: 956114804. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '874.870')] [2023-12-27 18:35:17,368][20381] Updated weights for policy 1, policy_version 1866333 (0.0008) [2023-12-27 18:35:17,432][20381] Updated weights for policy 1, policy_version 1866343 (0.0005) [2023-12-27 18:35:17,496][20381] Updated weights for policy 1, policy_version 1866353 (0.0005) [2023-12-27 18:35:17,798][20347] Updated weights for policy 0, policy_version 1867935 (0.0006) [2023-12-27 18:35:17,853][20347] Updated weights for policy 0, policy_version 1867945 (0.0008) [2023-12-27 18:35:17,900][20347] Updated weights for policy 0, policy_version 1867955 (0.0009) [2023-12-27 18:35:18,040][20381] Updated weights for policy 1, policy_version 1866363 (0.0006) [2023-12-27 18:35:18,107][20381] Updated weights for policy 1, policy_version 1866373 (0.0005) [2023-12-27 18:35:18,166][20381] Updated weights for policy 1, policy_version 1866383 (0.0007) [2023-12-27 18:35:18,618][20347] Updated weights for policy 0, policy_version 1867965 (0.0009) [2023-12-27 18:35:18,677][20347] Updated weights for policy 0, policy_version 1867975 (0.0010) [2023-12-27 18:35:18,728][20347] Updated weights for policy 0, policy_version 1867985 (0.0010) [2023-12-27 18:35:18,805][20381] Updated weights for policy 1, policy_version 1866393 (0.0010) [2023-12-27 18:35:18,854][20381] Updated weights for policy 1, policy_version 1866403 (0.0008) [2023-12-27 18:35:18,899][20381] Updated weights for policy 1, policy_version 1866413 (0.0007) [2023-12-27 18:35:18,951][20381] Updated weights for policy 1, policy_version 1866423 (0.0007) [2023-12-27 18:35:19,414][20347] Updated weights for policy 0, policy_version 1867995 (0.0009) [2023-12-27 18:35:19,480][20347] Updated weights for policy 0, policy_version 1868005 (0.0007) [2023-12-27 18:35:19,544][20347] Updated weights for policy 0, policy_version 1868015 (0.0010) [2023-12-27 18:35:19,633][20381] Updated weights for policy 1, policy_version 1866433 (0.0007) [2023-12-27 18:35:19,689][20381] Updated weights for policy 1, policy_version 1866443 (0.0007) [2023-12-27 18:35:19,751][20381] Updated weights for policy 1, policy_version 1866453 (0.0007) [2023-12-27 18:35:20,197][20347] Updated weights for policy 0, policy_version 1868025 (0.0010) [2023-12-27 18:35:20,252][20347] Updated weights for policy 0, policy_version 1868035 (0.0010) [2023-12-27 18:35:20,305][20347] Updated weights for policy 0, policy_version 1868045 (0.0011) [2023-12-27 18:35:20,365][20347] Updated weights for policy 0, policy_version 1868055 (0.0010) [2023-12-27 18:35:20,434][20381] Updated weights for policy 1, policy_version 1866463 (0.0008) [2023-12-27 18:35:20,496][20381] Updated weights for policy 1, policy_version 1866473 (0.0006) [2023-12-27 18:35:20,556][20381] Updated weights for policy 1, policy_version 1866483 (0.0011) [2023-12-27 18:35:21,037][20347] Updated weights for policy 0, policy_version 1868065 (0.0008) [2023-12-27 18:35:21,100][20347] Updated weights for policy 0, policy_version 1868075 (0.0008) [2023-12-27 18:35:21,175][20347] Updated weights for policy 0, policy_version 1868085 (0.0008) [2023-12-27 18:35:21,328][20381] Updated weights for policy 1, policy_version 1866493 (0.0008) [2023-12-27 18:35:21,398][20381] Updated weights for policy 1, policy_version 1866503 (0.0008) [2023-12-27 18:35:21,453][20381] Updated weights for policy 1, policy_version 1866513 (0.0008) [2023-12-27 18:35:21,903][20347] Updated weights for policy 0, policy_version 1868095 (0.0007) [2023-12-27 18:35:21,964][20347] Updated weights for policy 0, policy_version 1868105 (0.0006) [2023-12-27 18:35:22,024][20347] Updated weights for policy 0, policy_version 1868115 (0.0008) [2023-12-27 18:35:22,147][20381] Updated weights for policy 1, policy_version 1866523 (0.0008) [2023-12-27 18:35:22,211][20381] Updated weights for policy 1, policy_version 1866533 (0.0007) [2023-12-27 18:35:22,270][20381] Updated weights for policy 1, policy_version 1866543 (0.0008) [2023-12-27 18:35:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 956211200. Throughput: 0: 10225.9, 1: 10192.1. Samples: 956177980. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:22,363][19170] Avg episode reward: [(0, '990.240'), (1, '759.060')] [2023-12-27 18:35:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001866552_477888512.pth... [2023-12-27 18:35:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001868120_478322688.pth... [2023-12-27 18:35:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001865336_477577216.pth [2023-12-27 18:35:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001866936_478019584.pth [2023-12-27 18:35:22,707][20347] Updated weights for policy 0, policy_version 1868125 (0.0007) [2023-12-27 18:35:22,769][20347] Updated weights for policy 0, policy_version 1868135 (0.0007) [2023-12-27 18:35:22,830][20347] Updated weights for policy 0, policy_version 1868145 (0.0008) [2023-12-27 18:35:22,945][20381] Updated weights for policy 1, policy_version 1866553 (0.0008) [2023-12-27 18:35:22,993][20381] Updated weights for policy 1, policy_version 1866563 (0.0007) [2023-12-27 18:35:23,046][20381] Updated weights for policy 1, policy_version 1866573 (0.0007) [2023-12-27 18:35:23,105][20381] Updated weights for policy 1, policy_version 1866583 (0.0009) [2023-12-27 18:35:23,497][20347] Updated weights for policy 0, policy_version 1868155 (0.0009) [2023-12-27 18:35:23,558][20347] Updated weights for policy 0, policy_version 1868165 (0.0007) [2023-12-27 18:35:23,605][20347] Updated weights for policy 0, policy_version 1868175 (0.0006) [2023-12-27 18:35:23,778][20381] Updated weights for policy 1, policy_version 1866593 (0.0006) [2023-12-27 18:35:23,828][20381] Updated weights for policy 1, policy_version 1866603 (0.0005) [2023-12-27 18:35:23,877][20381] Updated weights for policy 1, policy_version 1866613 (0.0005) [2023-12-27 18:35:24,256][20347] Updated weights for policy 0, policy_version 1868185 (0.0006) [2023-12-27 18:35:24,314][20347] Updated weights for policy 0, policy_version 1868195 (0.0008) [2023-12-27 18:35:24,365][20347] Updated weights for policy 0, policy_version 1868205 (0.0008) [2023-12-27 18:35:24,412][20347] Updated weights for policy 0, policy_version 1868215 (0.0006) [2023-12-27 18:35:24,444][20381] Updated weights for policy 1, policy_version 1866623 (0.0008) [2023-12-27 18:35:24,499][20381] Updated weights for policy 1, policy_version 1866633 (0.0005) [2023-12-27 18:35:24,552][20381] Updated weights for policy 1, policy_version 1866643 (0.0005) [2023-12-27 18:35:25,108][20381] Updated weights for policy 1, policy_version 1866653 (0.0008) [2023-12-27 18:35:25,127][20347] Updated weights for policy 0, policy_version 1868225 (0.0009) [2023-12-27 18:35:25,166][20381] Updated weights for policy 1, policy_version 1866663 (0.0006) [2023-12-27 18:35:25,182][20347] Updated weights for policy 0, policy_version 1868235 (0.0007) [2023-12-27 18:35:25,222][20381] Updated weights for policy 1, policy_version 1866673 (0.0011) [2023-12-27 18:35:25,229][20347] Updated weights for policy 0, policy_version 1868245 (0.0005) [2023-12-27 18:35:25,840][20381] Updated weights for policy 1, policy_version 1866683 (0.0011) [2023-12-27 18:35:25,887][20347] Updated weights for policy 0, policy_version 1868255 (0.0005) [2023-12-27 18:35:25,895][20381] Updated weights for policy 1, policy_version 1866693 (0.0010) [2023-12-27 18:35:25,936][20347] Updated weights for policy 0, policy_version 1868265 (0.0005) [2023-12-27 18:35:25,947][20381] Updated weights for policy 1, policy_version 1866703 (0.0010) [2023-12-27 18:35:25,985][20347] Updated weights for policy 0, policy_version 1868275 (0.0005) [2023-12-27 18:35:26,578][20347] Updated weights for policy 0, policy_version 1868285 (0.0006) [2023-12-27 18:35:26,630][20347] Updated weights for policy 0, policy_version 1868295 (0.0007) [2023-12-27 18:35:26,633][20381] Updated weights for policy 1, policy_version 1866713 (0.0011) [2023-12-27 18:35:26,686][20347] Updated weights for policy 0, policy_version 1868305 (0.0006) [2023-12-27 18:35:26,694][20381] Updated weights for policy 1, policy_version 1866723 (0.0011) [2023-12-27 18:35:26,748][20381] Updated weights for policy 1, policy_version 1866733 (0.0010) [2023-12-27 18:35:26,805][20381] Updated weights for policy 1, policy_version 1866743 (0.0008) [2023-12-27 18:35:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 956309504. Throughput: 0: 10220.6, 1: 10244.9. Samples: 956302056. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '813.990')] [2023-12-27 18:35:27,369][20347] Updated weights for policy 0, policy_version 1868315 (0.0005) [2023-12-27 18:35:27,389][20381] Updated weights for policy 1, policy_version 1866753 (0.0008) [2023-12-27 18:35:27,425][20347] Updated weights for policy 0, policy_version 1868325 (0.0006) [2023-12-27 18:35:27,441][20381] Updated weights for policy 1, policy_version 1866763 (0.0006) [2023-12-27 18:35:27,476][20347] Updated weights for policy 0, policy_version 1868335 (0.0005) [2023-12-27 18:35:27,493][20381] Updated weights for policy 1, policy_version 1866773 (0.0006) [2023-12-27 18:35:28,070][20347] Updated weights for policy 0, policy_version 1868345 (0.0006) [2023-12-27 18:35:28,119][20347] Updated weights for policy 0, policy_version 1868355 (0.0007) [2023-12-27 18:35:28,139][20381] Updated weights for policy 1, policy_version 1866783 (0.0009) [2023-12-27 18:35:28,164][20347] Updated weights for policy 0, policy_version 1868365 (0.0005) [2023-12-27 18:35:28,187][20381] Updated weights for policy 1, policy_version 1866793 (0.0010) [2023-12-27 18:35:28,209][20347] Updated weights for policy 0, policy_version 1868375 (0.0006) [2023-12-27 18:35:28,241][20381] Updated weights for policy 1, policy_version 1866803 (0.0010) [2023-12-27 18:35:28,866][20381] Updated weights for policy 1, policy_version 1866813 (0.0008) [2023-12-27 18:35:28,905][20347] Updated weights for policy 0, policy_version 1868385 (0.0005) [2023-12-27 18:35:28,920][20381] Updated weights for policy 1, policy_version 1866823 (0.0008) [2023-12-27 18:35:28,951][20347] Updated weights for policy 0, policy_version 1868395 (0.0005) [2023-12-27 18:35:28,971][20381] Updated weights for policy 1, policy_version 1866833 (0.0006) [2023-12-27 18:35:29,007][20347] Updated weights for policy 0, policy_version 1868405 (0.0005) [2023-12-27 18:35:29,634][20381] Updated weights for policy 1, policy_version 1866843 (0.0005) [2023-12-27 18:35:29,659][20347] Updated weights for policy 0, policy_version 1868415 (0.0007) [2023-12-27 18:35:29,693][20381] Updated weights for policy 1, policy_version 1866853 (0.0007) [2023-12-27 18:35:29,719][20347] Updated weights for policy 0, policy_version 1868425 (0.0005) [2023-12-27 18:35:29,753][20381] Updated weights for policy 1, policy_version 1866863 (0.0007) [2023-12-27 18:35:29,778][20347] Updated weights for policy 0, policy_version 1868435 (0.0006) [2023-12-27 18:35:30,386][20347] Updated weights for policy 0, policy_version 1868445 (0.0008) [2023-12-27 18:35:30,445][20347] Updated weights for policy 0, policy_version 1868455 (0.0008) [2023-12-27 18:35:30,475][20381] Updated weights for policy 1, policy_version 1866873 (0.0008) [2023-12-27 18:35:30,496][20347] Updated weights for policy 0, policy_version 1868465 (0.0007) [2023-12-27 18:35:30,521][20381] Updated weights for policy 1, policy_version 1866883 (0.0005) [2023-12-27 18:35:30,564][20381] Updated weights for policy 1, policy_version 1866893 (0.0006) [2023-12-27 18:35:30,616][20381] Updated weights for policy 1, policy_version 1866903 (0.0007) [2023-12-27 18:35:31,230][20347] Updated weights for policy 0, policy_version 1868475 (0.0007) [2023-12-27 18:35:31,297][20347] Updated weights for policy 0, policy_version 1868485 (0.0009) [2023-12-27 18:35:31,361][20381] Updated weights for policy 1, policy_version 1866913 (0.0007) [2023-12-27 18:35:31,364][20347] Updated weights for policy 0, policy_version 1868495 (0.0007) [2023-12-27 18:35:31,431][20381] Updated weights for policy 1, policy_version 1866923 (0.0007) [2023-12-27 18:35:31,495][20381] Updated weights for policy 1, policy_version 1866933 (0.0008) [2023-12-27 18:35:32,064][20347] Updated weights for policy 0, policy_version 1868505 (0.0008) [2023-12-27 18:35:32,126][20347] Updated weights for policy 0, policy_version 1868515 (0.0008) [2023-12-27 18:35:32,182][20347] Updated weights for policy 0, policy_version 1868525 (0.0010) [2023-12-27 18:35:32,236][20347] Updated weights for policy 0, policy_version 1868535 (0.0010) [2023-12-27 18:35:32,283][20381] Updated weights for policy 1, policy_version 1866943 (0.0007) [2023-12-27 18:35:32,337][20381] Updated weights for policy 1, policy_version 1866953 (0.0009) [2023-12-27 18:35:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 956416000. Throughput: 0: 10290.0, 1: 10311.4. Samples: 956427064. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.750')] [2023-12-27 18:35:32,417][20381] Updated weights for policy 1, policy_version 1866963 (0.0008) [2023-12-27 18:35:32,914][20347] Updated weights for policy 0, policy_version 1868545 (0.0008) [2023-12-27 18:35:32,968][20347] Updated weights for policy 0, policy_version 1868555 (0.0007) [2023-12-27 18:35:33,020][20347] Updated weights for policy 0, policy_version 1868565 (0.0008) [2023-12-27 18:35:33,048][20381] Updated weights for policy 1, policy_version 1866973 (0.0007) [2023-12-27 18:35:33,108][20381] Updated weights for policy 1, policy_version 1866983 (0.0005) [2023-12-27 18:35:33,165][20381] Updated weights for policy 1, policy_version 1866993 (0.0005) [2023-12-27 18:35:33,677][20347] Updated weights for policy 0, policy_version 1868575 (0.0007) [2023-12-27 18:35:33,682][20381] Updated weights for policy 1, policy_version 1867003 (0.0008) [2023-12-27 18:35:33,727][20347] Updated weights for policy 0, policy_version 1868585 (0.0005) [2023-12-27 18:35:33,735][20381] Updated weights for policy 1, policy_version 1867013 (0.0010) [2023-12-27 18:35:33,773][20347] Updated weights for policy 0, policy_version 1868595 (0.0009) [2023-12-27 18:35:33,793][20381] Updated weights for policy 1, policy_version 1867023 (0.0007) [2023-12-27 18:35:34,379][20381] Updated weights for policy 1, policy_version 1867033 (0.0010) [2023-12-27 18:35:34,442][20381] Updated weights for policy 1, policy_version 1867043 (0.0008) [2023-12-27 18:35:34,452][20347] Updated weights for policy 0, policy_version 1868605 (0.0008) [2023-12-27 18:35:34,497][20381] Updated weights for policy 1, policy_version 1867053 (0.0006) [2023-12-27 18:35:34,514][20347] Updated weights for policy 0, policy_version 1868615 (0.0007) [2023-12-27 18:35:34,556][20381] Updated weights for policy 1, policy_version 1867063 (0.0006) [2023-12-27 18:35:34,574][20347] Updated weights for policy 0, policy_version 1868625 (0.0007) [2023-12-27 18:35:35,199][20347] Updated weights for policy 0, policy_version 1868635 (0.0008) [2023-12-27 18:35:35,252][20381] Updated weights for policy 1, policy_version 1867073 (0.0006) [2023-12-27 18:35:35,256][20347] Updated weights for policy 0, policy_version 1868645 (0.0006) [2023-12-27 18:35:35,302][20347] Updated weights for policy 0, policy_version 1868655 (0.0005) [2023-12-27 18:35:35,307][20381] Updated weights for policy 1, policy_version 1867083 (0.0007) [2023-12-27 18:35:35,357][20381] Updated weights for policy 1, policy_version 1867093 (0.0006) [2023-12-27 18:35:35,970][20381] Updated weights for policy 1, policy_version 1867103 (0.0005) [2023-12-27 18:35:36,008][20347] Updated weights for policy 0, policy_version 1868665 (0.0006) [2023-12-27 18:35:36,026][20381] Updated weights for policy 1, policy_version 1867113 (0.0006) [2023-12-27 18:35:36,058][20347] Updated weights for policy 0, policy_version 1868675 (0.0007) [2023-12-27 18:35:36,078][20381] Updated weights for policy 1, policy_version 1867123 (0.0006) [2023-12-27 18:35:36,117][20347] Updated weights for policy 0, policy_version 1868685 (0.0007) [2023-12-27 18:35:36,183][20347] Updated weights for policy 0, policy_version 1868695 (0.0008) [2023-12-27 18:35:36,736][20381] Updated weights for policy 1, policy_version 1867133 (0.0007) [2023-12-27 18:35:36,781][20381] Updated weights for policy 1, policy_version 1867143 (0.0008) [2023-12-27 18:35:36,840][20381] Updated weights for policy 1, policy_version 1867153 (0.0008) [2023-12-27 18:35:36,906][20347] Updated weights for policy 0, policy_version 1868705 (0.0010) [2023-12-27 18:35:36,965][20347] Updated weights for policy 0, policy_version 1868715 (0.0010) [2023-12-27 18:35:37,025][20347] Updated weights for policy 0, policy_version 1868725 (0.0010) [2023-12-27 18:35:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 956522496. Throughput: 0: 10308.8, 1: 10374.9. Samples: 956491280. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.740')] [2023-12-27 18:35:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001867160_478044160.pth... [2023-12-27 18:35:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001868728_478478336.pth... [2023-12-27 18:35:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001867544_478175232.pth [2023-12-27 18:35:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001865944_477732864.pth [2023-12-27 18:35:37,582][20381] Updated weights for policy 1, policy_version 1867163 (0.0006) [2023-12-27 18:35:37,632][20381] Updated weights for policy 1, policy_version 1867173 (0.0008) [2023-12-27 18:35:37,641][20347] Updated weights for policy 0, policy_version 1868735 (0.0007) [2023-12-27 18:35:37,687][20381] Updated weights for policy 1, policy_version 1867183 (0.0007) [2023-12-27 18:35:37,694][20347] Updated weights for policy 0, policy_version 1868745 (0.0006) [2023-12-27 18:35:37,746][20347] Updated weights for policy 0, policy_version 1868755 (0.0010) [2023-12-27 18:35:38,374][20381] Updated weights for policy 1, policy_version 1867193 (0.0008) [2023-12-27 18:35:38,395][20347] Updated weights for policy 0, policy_version 1868765 (0.0009) [2023-12-27 18:35:38,430][20381] Updated weights for policy 1, policy_version 1867203 (0.0006) [2023-12-27 18:35:38,458][20347] Updated weights for policy 0, policy_version 1868775 (0.0010) [2023-12-27 18:35:38,485][20381] Updated weights for policy 1, policy_version 1867213 (0.0005) [2023-12-27 18:35:38,517][20347] Updated weights for policy 0, policy_version 1868785 (0.0010) [2023-12-27 18:35:38,545][20381] Updated weights for policy 1, policy_version 1867223 (0.0006) [2023-12-27 18:35:39,173][20347] Updated weights for policy 0, policy_version 1868795 (0.0008) [2023-12-27 18:35:39,203][20381] Updated weights for policy 1, policy_version 1867233 (0.0005) [2023-12-27 18:35:39,240][20347] Updated weights for policy 0, policy_version 1868805 (0.0008) [2023-12-27 18:35:39,267][20381] Updated weights for policy 1, policy_version 1867243 (0.0007) [2023-12-27 18:35:39,305][20347] Updated weights for policy 0, policy_version 1868815 (0.0010) [2023-12-27 18:35:39,324][20381] Updated weights for policy 1, policy_version 1867253 (0.0008) [2023-12-27 18:35:40,018][20381] Updated weights for policy 1, policy_version 1867263 (0.0007) [2023-12-27 18:35:40,062][20347] Updated weights for policy 0, policy_version 1868825 (0.0010) [2023-12-27 18:35:40,085][20381] Updated weights for policy 1, policy_version 1867273 (0.0007) [2023-12-27 18:35:40,116][20347] Updated weights for policy 0, policy_version 1868835 (0.0008) [2023-12-27 18:35:40,147][20381] Updated weights for policy 1, policy_version 1867283 (0.0008) [2023-12-27 18:35:40,168][20347] Updated weights for policy 0, policy_version 1868845 (0.0006) [2023-12-27 18:35:40,223][20347] Updated weights for policy 0, policy_version 1868855 (0.0007) [2023-12-27 18:35:40,829][20381] Updated weights for policy 1, policy_version 1867293 (0.0008) [2023-12-27 18:35:40,886][20381] Updated weights for policy 1, policy_version 1867303 (0.0006) [2023-12-27 18:35:40,909][20347] Updated weights for policy 0, policy_version 1868865 (0.0007) [2023-12-27 18:35:40,943][20381] Updated weights for policy 1, policy_version 1867313 (0.0006) [2023-12-27 18:35:40,961][20347] Updated weights for policy 0, policy_version 1868875 (0.0007) [2023-12-27 18:35:41,014][20347] Updated weights for policy 0, policy_version 1868885 (0.0008) [2023-12-27 18:35:41,663][20381] Updated weights for policy 1, policy_version 1867323 (0.0008) [2023-12-27 18:35:41,724][20381] Updated weights for policy 1, policy_version 1867333 (0.0007) [2023-12-27 18:35:41,759][20347] Updated weights for policy 0, policy_version 1868895 (0.0008) [2023-12-27 18:35:41,795][20381] Updated weights for policy 1, policy_version 1867343 (0.0006) [2023-12-27 18:35:41,819][20347] Updated weights for policy 0, policy_version 1868905 (0.0006) [2023-12-27 18:35:41,879][20347] Updated weights for policy 0, policy_version 1868915 (0.0005) [2023-12-27 18:35:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 956620800. Throughput: 0: 10306.3, 1: 10329.0. Samples: 956613388. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '869.490')] [2023-12-27 18:35:42,515][20347] Updated weights for policy 0, policy_version 1868925 (0.0006) [2023-12-27 18:35:42,563][20381] Updated weights for policy 1, policy_version 1867353 (0.0007) [2023-12-27 18:35:42,576][20347] Updated weights for policy 0, policy_version 1868935 (0.0006) [2023-12-27 18:35:42,624][20381] Updated weights for policy 1, policy_version 1867363 (0.0006) [2023-12-27 18:35:42,634][20347] Updated weights for policy 0, policy_version 1868945 (0.0006) [2023-12-27 18:35:42,683][20381] Updated weights for policy 1, policy_version 1867373 (0.0007) [2023-12-27 18:35:42,737][20381] Updated weights for policy 1, policy_version 1867383 (0.0008) [2023-12-27 18:35:43,246][20347] Updated weights for policy 0, policy_version 1868955 (0.0006) [2023-12-27 18:35:43,304][20347] Updated weights for policy 0, policy_version 1868965 (0.0006) [2023-12-27 18:35:43,359][20347] Updated weights for policy 0, policy_version 1868975 (0.0005) [2023-12-27 18:35:43,459][20381] Updated weights for policy 1, policy_version 1867393 (0.0006) [2023-12-27 18:35:43,508][20381] Updated weights for policy 1, policy_version 1867403 (0.0005) [2023-12-27 18:35:43,558][20381] Updated weights for policy 1, policy_version 1867413 (0.0005) [2023-12-27 18:35:43,991][20347] Updated weights for policy 0, policy_version 1868985 (0.0005) [2023-12-27 18:35:44,048][20347] Updated weights for policy 0, policy_version 1868995 (0.0005) [2023-12-27 18:35:44,093][20381] Updated weights for policy 1, policy_version 1867423 (0.0009) [2023-12-27 18:35:44,103][20347] Updated weights for policy 0, policy_version 1869005 (0.0006) [2023-12-27 18:35:44,141][20381] Updated weights for policy 1, policy_version 1867433 (0.0010) [2023-12-27 18:35:44,147][20347] Updated weights for policy 0, policy_version 1869015 (0.0005) [2023-12-27 18:35:44,187][20381] Updated weights for policy 1, policy_version 1867443 (0.0010) [2023-12-27 18:35:44,767][20347] Updated weights for policy 0, policy_version 1869025 (0.0009) [2023-12-27 18:35:44,839][20347] Updated weights for policy 0, policy_version 1869036 (0.0006) [2023-12-27 18:35:44,849][20381] Updated weights for policy 1, policy_version 1867453 (0.0009) [2023-12-27 18:35:44,902][20347] Updated weights for policy 0, policy_version 1869046 (0.0010) [2023-12-27 18:35:44,915][20381] Updated weights for policy 1, policy_version 1867463 (0.0008) [2023-12-27 18:35:44,980][20381] Updated weights for policy 1, policy_version 1867473 (0.0011) [2023-12-27 18:35:45,542][20347] Updated weights for policy 0, policy_version 1869056 (0.0006) [2023-12-27 18:35:45,605][20347] Updated weights for policy 0, policy_version 1869066 (0.0006) [2023-12-27 18:35:45,650][20347] Updated weights for policy 0, policy_version 1869076 (0.0007) [2023-12-27 18:35:45,669][20381] Updated weights for policy 1, policy_version 1867483 (0.0008) [2023-12-27 18:35:45,722][20381] Updated weights for policy 1, policy_version 1867493 (0.0006) [2023-12-27 18:35:45,775][20381] Updated weights for policy 1, policy_version 1867503 (0.0005) [2023-12-27 18:35:46,343][20347] Updated weights for policy 0, policy_version 1869086 (0.0006) [2023-12-27 18:35:46,398][20347] Updated weights for policy 0, policy_version 1869096 (0.0007) [2023-12-27 18:35:46,408][20381] Updated weights for policy 1, policy_version 1867514 (0.0007) [2023-12-27 18:35:46,459][20347] Updated weights for policy 0, policy_version 1869106 (0.0006) [2023-12-27 18:35:46,467][20381] Updated weights for policy 1, policy_version 1867524 (0.0006) [2023-12-27 18:35:46,521][20381] Updated weights for policy 1, policy_version 1867534 (0.0007) [2023-12-27 18:35:46,582][20381] Updated weights for policy 1, policy_version 1867544 (0.0009) [2023-12-27 18:35:47,069][20347] Updated weights for policy 0, policy_version 1869116 (0.0007) [2023-12-27 18:35:47,124][20347] Updated weights for policy 0, policy_version 1869126 (0.0007) [2023-12-27 18:35:47,168][20347] Updated weights for policy 0, policy_version 1869136 (0.0007) [2023-12-27 18:35:47,296][20381] Updated weights for policy 1, policy_version 1867554 (0.0009) [2023-12-27 18:35:47,354][20381] Updated weights for policy 1, policy_version 1867564 (0.0005) [2023-12-27 18:35:47,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 956727296. Throughput: 0: 10349.0, 1: 10347.2. Samples: 956738220. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '798.640')] [2023-12-27 18:35:47,408][20381] Updated weights for policy 1, policy_version 1867574 (0.0007) [2023-12-27 18:35:47,876][20347] Updated weights for policy 0, policy_version 1869146 (0.0009) [2023-12-27 18:35:47,933][20347] Updated weights for policy 0, policy_version 1869156 (0.0010) [2023-12-27 18:35:47,990][20347] Updated weights for policy 0, policy_version 1869166 (0.0010) [2023-12-27 18:35:48,009][20381] Updated weights for policy 1, policy_version 1867584 (0.0010) [2023-12-27 18:35:48,052][20347] Updated weights for policy 0, policy_version 1869176 (0.0010) [2023-12-27 18:35:48,064][20381] Updated weights for policy 1, policy_version 1867594 (0.0010) [2023-12-27 18:35:48,112][20381] Updated weights for policy 1, policy_version 1867604 (0.0010) [2023-12-27 18:35:48,714][20347] Updated weights for policy 0, policy_version 1869186 (0.0008) [2023-12-27 18:35:48,775][20347] Updated weights for policy 0, policy_version 1869196 (0.0010) [2023-12-27 18:35:48,831][20347] Updated weights for policy 0, policy_version 1869206 (0.0010) [2023-12-27 18:35:48,850][20381] Updated weights for policy 1, policy_version 1867614 (0.0009) [2023-12-27 18:35:48,908][20381] Updated weights for policy 1, policy_version 1867624 (0.0007) [2023-12-27 18:35:48,962][20381] Updated weights for policy 1, policy_version 1867634 (0.0007) [2023-12-27 18:35:49,544][20347] Updated weights for policy 0, policy_version 1869216 (0.0008) [2023-12-27 18:35:49,606][20347] Updated weights for policy 0, policy_version 1869226 (0.0008) [2023-12-27 18:35:49,667][20347] Updated weights for policy 0, policy_version 1869236 (0.0007) [2023-12-27 18:35:49,673][20381] Updated weights for policy 1, policy_version 1867644 (0.0007) [2023-12-27 18:35:49,724][20381] Updated weights for policy 1, policy_version 1867654 (0.0007) [2023-12-27 18:35:49,776][20381] Updated weights for policy 1, policy_version 1867664 (0.0007) [2023-12-27 18:35:50,375][20347] Updated weights for policy 0, policy_version 1869246 (0.0007) [2023-12-27 18:35:50,422][20347] Updated weights for policy 0, policy_version 1869256 (0.0005) [2023-12-27 18:35:50,469][20347] Updated weights for policy 0, policy_version 1869266 (0.0005) [2023-12-27 18:35:50,546][20381] Updated weights for policy 1, policy_version 1867674 (0.0008) [2023-12-27 18:35:50,608][20381] Updated weights for policy 1, policy_version 1867684 (0.0008) [2023-12-27 18:35:50,674][20381] Updated weights for policy 1, policy_version 1867694 (0.0006) [2023-12-27 18:35:50,735][20381] Updated weights for policy 1, policy_version 1867704 (0.0009) [2023-12-27 18:35:51,099][20347] Updated weights for policy 0, policy_version 1869276 (0.0005) [2023-12-27 18:35:51,169][20347] Updated weights for policy 0, policy_version 1869286 (0.0008) [2023-12-27 18:35:51,228][20347] Updated weights for policy 0, policy_version 1869296 (0.0007) [2023-12-27 18:35:51,406][20381] Updated weights for policy 1, policy_version 1867714 (0.0007) [2023-12-27 18:35:51,464][20381] Updated weights for policy 1, policy_version 1867724 (0.0008) [2023-12-27 18:35:51,520][20381] Updated weights for policy 1, policy_version 1867734 (0.0007) [2023-12-27 18:35:51,951][20347] Updated weights for policy 0, policy_version 1869306 (0.0006) [2023-12-27 18:35:52,006][20347] Updated weights for policy 0, policy_version 1869316 (0.0008) [2023-12-27 18:35:52,064][20347] Updated weights for policy 0, policy_version 1869326 (0.0007) [2023-12-27 18:35:52,130][20347] Updated weights for policy 0, policy_version 1869336 (0.0005) [2023-12-27 18:35:52,227][20381] Updated weights for policy 1, policy_version 1867744 (0.0008) [2023-12-27 18:35:52,291][20381] Updated weights for policy 1, policy_version 1867754 (0.0008) [2023-12-27 18:35:52,354][20381] Updated weights for policy 1, policy_version 1867764 (0.0007) [2023-12-27 18:35:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 956825600. Throughput: 0: 10264.7, 1: 10329.4. Samples: 956798400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '876.710')] [2023-12-27 18:35:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001869336_478633984.pth... [2023-12-27 18:35:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001868120_478322688.pth [2023-12-27 18:35:52,378][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001867768_478199808.pth... [2023-12-27 18:35:52,382][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001866552_477888512.pth [2023-12-27 18:35:52,774][20347] Updated weights for policy 0, policy_version 1869346 (0.0008) [2023-12-27 18:35:52,837][20347] Updated weights for policy 0, policy_version 1869356 (0.0007) [2023-12-27 18:35:52,901][20347] Updated weights for policy 0, policy_version 1869366 (0.0008) [2023-12-27 18:35:53,099][20381] Updated weights for policy 1, policy_version 1867774 (0.0008) [2023-12-27 18:35:53,144][20381] Updated weights for policy 1, policy_version 1867784 (0.0007) [2023-12-27 18:35:53,190][20381] Updated weights for policy 1, policy_version 1867794 (0.0008) [2023-12-27 18:35:53,557][20347] Updated weights for policy 0, policy_version 1869376 (0.0007) [2023-12-27 18:35:53,610][20347] Updated weights for policy 0, policy_version 1869386 (0.0008) [2023-12-27 18:35:53,659][20347] Updated weights for policy 0, policy_version 1869396 (0.0008) [2023-12-27 18:35:53,887][20381] Updated weights for policy 1, policy_version 1867804 (0.0008) [2023-12-27 18:35:53,941][20381] Updated weights for policy 1, policy_version 1867814 (0.0007) [2023-12-27 18:35:54,003][20381] Updated weights for policy 1, policy_version 1867824 (0.0008) [2023-12-27 18:35:54,325][20347] Updated weights for policy 0, policy_version 1869406 (0.0007) [2023-12-27 18:35:54,377][20347] Updated weights for policy 0, policy_version 1869416 (0.0008) [2023-12-27 18:35:54,430][20347] Updated weights for policy 0, policy_version 1869426 (0.0007) [2023-12-27 18:35:54,680][20381] Updated weights for policy 1, policy_version 1867834 (0.0007) [2023-12-27 18:35:54,734][20381] Updated weights for policy 1, policy_version 1867844 (0.0008) [2023-12-27 18:35:54,783][20381] Updated weights for policy 1, policy_version 1867854 (0.0008) [2023-12-27 18:35:54,828][20381] Updated weights for policy 1, policy_version 1867864 (0.0008) [2023-12-27 18:35:55,117][20347] Updated weights for policy 0, policy_version 1869436 (0.0007) [2023-12-27 18:35:55,172][20347] Updated weights for policy 0, policy_version 1869446 (0.0005) [2023-12-27 18:35:55,226][20347] Updated weights for policy 0, policy_version 1869456 (0.0005) [2023-12-27 18:35:55,509][20381] Updated weights for policy 1, policy_version 1867874 (0.0006) [2023-12-27 18:35:55,559][20381] Updated weights for policy 1, policy_version 1867884 (0.0005) [2023-12-27 18:35:55,609][20381] Updated weights for policy 1, policy_version 1867894 (0.0008) [2023-12-27 18:35:55,852][20347] Updated weights for policy 0, policy_version 1869466 (0.0006) [2023-12-27 18:35:55,909][20347] Updated weights for policy 0, policy_version 1869476 (0.0007) [2023-12-27 18:35:55,969][20347] Updated weights for policy 0, policy_version 1869486 (0.0005) [2023-12-27 18:35:56,033][20347] Updated weights for policy 0, policy_version 1869496 (0.0005) [2023-12-27 18:35:56,251][20381] Updated weights for policy 1, policy_version 1867904 (0.0010) [2023-12-27 18:35:56,309][20381] Updated weights for policy 1, policy_version 1867914 (0.0010) [2023-12-27 18:35:56,363][20381] Updated weights for policy 1, policy_version 1867924 (0.0010) [2023-12-27 18:35:56,589][20347] Updated weights for policy 0, policy_version 1869506 (0.0005) [2023-12-27 18:35:56,651][20347] Updated weights for policy 0, policy_version 1869516 (0.0005) [2023-12-27 18:35:56,704][20347] Updated weights for policy 0, policy_version 1869526 (0.0005) [2023-12-27 18:35:56,965][20381] Updated weights for policy 1, policy_version 1867934 (0.0007) [2023-12-27 18:35:57,015][20381] Updated weights for policy 1, policy_version 1867944 (0.0005) [2023-12-27 18:35:57,061][20381] Updated weights for policy 1, policy_version 1867954 (0.0005) [2023-12-27 18:35:57,285][20347] Updated weights for policy 0, policy_version 1869536 (0.0006) [2023-12-27 18:35:57,333][20347] Updated weights for policy 0, policy_version 1869546 (0.0008) [2023-12-27 18:35:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 956932096. Throughput: 0: 10312.3, 1: 10332.1. Samples: 956921884. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:35:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '852.400')] [2023-12-27 18:35:57,378][20347] Updated weights for policy 0, policy_version 1869556 (0.0010) [2023-12-27 18:35:57,679][20381] Updated weights for policy 1, policy_version 1867964 (0.0007) [2023-12-27 18:35:57,731][20381] Updated weights for policy 1, policy_version 1867974 (0.0010) [2023-12-27 18:35:57,795][20381] Updated weights for policy 1, policy_version 1867984 (0.0010) [2023-12-27 18:35:58,017][20347] Updated weights for policy 0, policy_version 1869566 (0.0010) [2023-12-27 18:35:58,070][20347] Updated weights for policy 0, policy_version 1869576 (0.0010) [2023-12-27 18:35:58,129][20347] Updated weights for policy 0, policy_version 1869586 (0.0009) [2023-12-27 18:35:58,466][20381] Updated weights for policy 1, policy_version 1867994 (0.0011) [2023-12-27 18:35:58,529][20381] Updated weights for policy 1, policy_version 1868004 (0.0011) [2023-12-27 18:35:58,589][20381] Updated weights for policy 1, policy_version 1868014 (0.0011) [2023-12-27 18:35:58,645][20381] Updated weights for policy 1, policy_version 1868024 (0.0009) [2023-12-27 18:35:58,941][20347] Updated weights for policy 0, policy_version 1869596 (0.0006) [2023-12-27 18:35:59,005][20347] Updated weights for policy 0, policy_version 1869606 (0.0007) [2023-12-27 18:35:59,070][20347] Updated weights for policy 0, policy_version 1869616 (0.0008) [2023-12-27 18:35:59,441][20381] Updated weights for policy 1, policy_version 1868034 (0.0008) [2023-12-27 18:35:59,498][20381] Updated weights for policy 1, policy_version 1868044 (0.0008) [2023-12-27 18:35:59,558][20381] Updated weights for policy 1, policy_version 1868054 (0.0008) [2023-12-27 18:35:59,819][20347] Updated weights for policy 0, policy_version 1869626 (0.0007) [2023-12-27 18:35:59,876][20347] Updated weights for policy 0, policy_version 1869636 (0.0008) [2023-12-27 18:35:59,936][20347] Updated weights for policy 0, policy_version 1869646 (0.0008) [2023-12-27 18:35:59,982][20347] Updated weights for policy 0, policy_version 1869656 (0.0006) [2023-12-27 18:36:00,273][20381] Updated weights for policy 1, policy_version 1868064 (0.0007) [2023-12-27 18:36:00,330][20381] Updated weights for policy 1, policy_version 1868074 (0.0008) [2023-12-27 18:36:00,380][20381] Updated weights for policy 1, policy_version 1868084 (0.0007) [2023-12-27 18:36:00,692][20347] Updated weights for policy 0, policy_version 1869666 (0.0007) [2023-12-27 18:36:00,750][20347] Updated weights for policy 0, policy_version 1869676 (0.0005) [2023-12-27 18:36:00,801][20347] Updated weights for policy 0, policy_version 1869686 (0.0005) [2023-12-27 18:36:01,125][20381] Updated weights for policy 1, policy_version 1868094 (0.0008) [2023-12-27 18:36:01,189][20381] Updated weights for policy 1, policy_version 1868104 (0.0008) [2023-12-27 18:36:01,246][20381] Updated weights for policy 1, policy_version 1868114 (0.0008) [2023-12-27 18:36:01,399][20347] Updated weights for policy 0, policy_version 1869696 (0.0006) [2023-12-27 18:36:01,450][20347] Updated weights for policy 0, policy_version 1869706 (0.0005) [2023-12-27 18:36:01,502][20347] Updated weights for policy 0, policy_version 1869716 (0.0007) [2023-12-27 18:36:01,995][20381] Updated weights for policy 1, policy_version 1868124 (0.0007) [2023-12-27 18:36:02,048][20381] Updated weights for policy 1, policy_version 1868134 (0.0006) [2023-12-27 18:36:02,104][20381] Updated weights for policy 1, policy_version 1868144 (0.0005) [2023-12-27 18:36:02,147][20347] Updated weights for policy 0, policy_version 1869726 (0.0007) [2023-12-27 18:36:02,196][20347] Updated weights for policy 0, policy_version 1869736 (0.0009) [2023-12-27 18:36:02,253][20347] Updated weights for policy 0, policy_version 1869746 (0.0009) [2023-12-27 18:36:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 957038592. Throughput: 0: 10356.0, 1: 10308.7. Samples: 957044716. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '849.600')] [2023-12-27 18:36:02,813][20381] Updated weights for policy 1, policy_version 1868154 (0.0006) [2023-12-27 18:36:02,872][20381] Updated weights for policy 1, policy_version 1868164 (0.0009) [2023-12-27 18:36:02,904][20347] Updated weights for policy 0, policy_version 1869756 (0.0006) [2023-12-27 18:36:02,922][20381] Updated weights for policy 1, policy_version 1868174 (0.0008) [2023-12-27 18:36:02,959][20347] Updated weights for policy 0, policy_version 1869766 (0.0006) [2023-12-27 18:36:02,975][20381] Updated weights for policy 1, policy_version 1868184 (0.0008) [2023-12-27 18:36:03,013][20347] Updated weights for policy 0, policy_version 1869776 (0.0005) [2023-12-27 18:36:03,645][20347] Updated weights for policy 0, policy_version 1869786 (0.0006) [2023-12-27 18:36:03,652][20381] Updated weights for policy 1, policy_version 1868194 (0.0007) [2023-12-27 18:36:03,698][20347] Updated weights for policy 0, policy_version 1869796 (0.0009) [2023-12-27 18:36:03,708][20381] Updated weights for policy 1, policy_version 1868204 (0.0008) [2023-12-27 18:36:03,755][20347] Updated weights for policy 0, policy_version 1869806 (0.0007) [2023-12-27 18:36:03,758][20381] Updated weights for policy 1, policy_version 1868214 (0.0007) [2023-12-27 18:36:03,802][20347] Updated weights for policy 0, policy_version 1869816 (0.0008) [2023-12-27 18:36:04,467][20381] Updated weights for policy 1, policy_version 1868224 (0.0006) [2023-12-27 18:36:04,506][20347] Updated weights for policy 0, policy_version 1869826 (0.0011) [2023-12-27 18:36:04,522][20381] Updated weights for policy 1, policy_version 1868234 (0.0006) [2023-12-27 18:36:04,564][20347] Updated weights for policy 0, policy_version 1869836 (0.0010) [2023-12-27 18:36:04,580][20381] Updated weights for policy 1, policy_version 1868244 (0.0005) [2023-12-27 18:36:04,618][20347] Updated weights for policy 0, policy_version 1869846 (0.0010) [2023-12-27 18:36:05,241][20347] Updated weights for policy 0, policy_version 1869856 (0.0011) [2023-12-27 18:36:05,272][20381] Updated weights for policy 1, policy_version 1868254 (0.0006) [2023-12-27 18:36:05,289][20347] Updated weights for policy 0, policy_version 1869866 (0.0010) [2023-12-27 18:36:05,331][20381] Updated weights for policy 1, policy_version 1868264 (0.0005) [2023-12-27 18:36:05,344][20347] Updated weights for policy 0, policy_version 1869876 (0.0010) [2023-12-27 18:36:05,386][20381] Updated weights for policy 1, policy_version 1868274 (0.0006) [2023-12-27 18:36:05,995][20381] Updated weights for policy 1, policy_version 1868284 (0.0007) [2023-12-27 18:36:06,031][20347] Updated weights for policy 0, policy_version 1869886 (0.0007) [2023-12-27 18:36:06,050][20381] Updated weights for policy 1, policy_version 1868294 (0.0007) [2023-12-27 18:36:06,091][20347] Updated weights for policy 0, policy_version 1869896 (0.0007) [2023-12-27 18:36:06,109][20381] Updated weights for policy 1, policy_version 1868304 (0.0006) [2023-12-27 18:36:06,150][20347] Updated weights for policy 0, policy_version 1869906 (0.0008) [2023-12-27 18:36:06,823][20381] Updated weights for policy 1, policy_version 1868314 (0.0008) [2023-12-27 18:36:06,864][20347] Updated weights for policy 0, policy_version 1869916 (0.0008) [2023-12-27 18:36:06,881][20381] Updated weights for policy 1, policy_version 1868324 (0.0006) [2023-12-27 18:36:06,914][20347] Updated weights for policy 0, policy_version 1869926 (0.0006) [2023-12-27 18:36:06,937][20381] Updated weights for policy 1, policy_version 1868334 (0.0006) [2023-12-27 18:36:06,964][20347] Updated weights for policy 0, policy_version 1869936 (0.0008) [2023-12-27 18:36:06,998][20381] Updated weights for policy 1, policy_version 1868344 (0.0005) [2023-12-27 18:36:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 957136896. Throughput: 0: 10367.6, 1: 10251.4. Samples: 957105832. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '854.550')] [2023-12-27 18:36:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001868344_478347264.pth... [2023-12-27 18:36:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001869944_478789632.pth... [2023-12-27 18:36:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001867160_478044160.pth [2023-12-27 18:36:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001868728_478478336.pth [2023-12-27 18:36:07,592][20381] Updated weights for policy 1, policy_version 1868354 (0.0008) [2023-12-27 18:36:07,617][20347] Updated weights for policy 0, policy_version 1869946 (0.0008) [2023-12-27 18:36:07,654][20381] Updated weights for policy 1, policy_version 1868364 (0.0007) [2023-12-27 18:36:07,668][20347] Updated weights for policy 0, policy_version 1869956 (0.0007) [2023-12-27 18:36:07,718][20381] Updated weights for policy 1, policy_version 1868374 (0.0007) [2023-12-27 18:36:07,724][20347] Updated weights for policy 0, policy_version 1869966 (0.0005) [2023-12-27 18:36:07,782][20347] Updated weights for policy 0, policy_version 1869976 (0.0008) [2023-12-27 18:36:08,400][20347] Updated weights for policy 0, policy_version 1869986 (0.0007) [2023-12-27 18:36:08,429][20381] Updated weights for policy 1, policy_version 1868384 (0.0008) [2023-12-27 18:36:08,458][20347] Updated weights for policy 0, policy_version 1869996 (0.0005) [2023-12-27 18:36:08,484][20381] Updated weights for policy 1, policy_version 1868394 (0.0008) [2023-12-27 18:36:08,518][20347] Updated weights for policy 0, policy_version 1870006 (0.0005) [2023-12-27 18:36:08,543][20381] Updated weights for policy 1, policy_version 1868404 (0.0007) [2023-12-27 18:36:09,082][20347] Updated weights for policy 0, policy_version 1870016 (0.0006) [2023-12-27 18:36:09,134][20347] Updated weights for policy 0, policy_version 1870026 (0.0008) [2023-12-27 18:36:09,182][20347] Updated weights for policy 0, policy_version 1870036 (0.0007) [2023-12-27 18:36:09,231][20381] Updated weights for policy 1, policy_version 1868414 (0.0009) [2023-12-27 18:36:09,290][20381] Updated weights for policy 1, policy_version 1868424 (0.0006) [2023-12-27 18:36:09,356][20381] Updated weights for policy 1, policy_version 1868434 (0.0008) [2023-12-27 18:36:09,935][20347] Updated weights for policy 0, policy_version 1870046 (0.0008) [2023-12-27 18:36:09,994][20347] Updated weights for policy 0, policy_version 1870056 (0.0007) [2023-12-27 18:36:10,055][20347] Updated weights for policy 0, policy_version 1870066 (0.0008) [2023-12-27 18:36:10,074][20381] Updated weights for policy 1, policy_version 1868444 (0.0008) [2023-12-27 18:36:10,140][20381] Updated weights for policy 1, policy_version 1868454 (0.0008) [2023-12-27 18:36:10,200][20381] Updated weights for policy 1, policy_version 1868464 (0.0007) [2023-12-27 18:36:10,778][20347] Updated weights for policy 0, policy_version 1870076 (0.0008) [2023-12-27 18:36:10,828][20347] Updated weights for policy 0, policy_version 1870086 (0.0005) [2023-12-27 18:36:10,862][20381] Updated weights for policy 1, policy_version 1868474 (0.0008) [2023-12-27 18:36:10,883][20347] Updated weights for policy 0, policy_version 1870096 (0.0006) [2023-12-27 18:36:10,919][20381] Updated weights for policy 1, policy_version 1868484 (0.0008) [2023-12-27 18:36:10,974][20381] Updated weights for policy 1, policy_version 1868494 (0.0007) [2023-12-27 18:36:11,020][20381] Updated weights for policy 1, policy_version 1868504 (0.0009) [2023-12-27 18:36:11,582][20347] Updated weights for policy 0, policy_version 1870106 (0.0005) [2023-12-27 18:36:11,651][20347] Updated weights for policy 0, policy_version 1870116 (0.0007) [2023-12-27 18:36:11,715][20347] Updated weights for policy 0, policy_version 1870126 (0.0008) [2023-12-27 18:36:11,744][20381] Updated weights for policy 1, policy_version 1868514 (0.0008) [2023-12-27 18:36:11,780][20347] Updated weights for policy 0, policy_version 1870136 (0.0007) [2023-12-27 18:36:11,803][20381] Updated weights for policy 1, policy_version 1868524 (0.0007) [2023-12-27 18:36:11,854][20381] Updated weights for policy 1, policy_version 1868534 (0.0006) [2023-12-27 18:36:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 957235200. Throughput: 0: 10402.7, 1: 10202.9. Samples: 957229308. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.270')] [2023-12-27 18:36:12,465][20347] Updated weights for policy 0, policy_version 1870146 (0.0008) [2023-12-27 18:36:12,525][20347] Updated weights for policy 0, policy_version 1870156 (0.0008) [2023-12-27 18:36:12,542][20381] Updated weights for policy 1, policy_version 1868544 (0.0007) [2023-12-27 18:36:12,585][20347] Updated weights for policy 0, policy_version 1870166 (0.0007) [2023-12-27 18:36:12,606][20381] Updated weights for policy 1, policy_version 1868554 (0.0007) [2023-12-27 18:36:12,664][20381] Updated weights for policy 1, policy_version 1868564 (0.0008) [2023-12-27 18:36:13,243][20347] Updated weights for policy 0, policy_version 1870176 (0.0007) [2023-12-27 18:36:13,289][20347] Updated weights for policy 0, policy_version 1870186 (0.0008) [2023-12-27 18:36:13,336][20347] Updated weights for policy 0, policy_version 1870196 (0.0006) [2023-12-27 18:36:13,341][20381] Updated weights for policy 1, policy_version 1868574 (0.0007) [2023-12-27 18:36:13,399][20381] Updated weights for policy 1, policy_version 1868584 (0.0007) [2023-12-27 18:36:13,445][20381] Updated weights for policy 1, policy_version 1868594 (0.0005) [2023-12-27 18:36:13,982][20381] Updated weights for policy 1, policy_version 1868604 (0.0006) [2023-12-27 18:36:14,033][20381] Updated weights for policy 1, policy_version 1868614 (0.0005) [2023-12-27 18:36:14,081][20347] Updated weights for policy 0, policy_version 1870206 (0.0005) [2023-12-27 18:36:14,090][20381] Updated weights for policy 1, policy_version 1868624 (0.0006) [2023-12-27 18:36:14,140][20347] Updated weights for policy 0, policy_version 1870216 (0.0006) [2023-12-27 18:36:14,202][20347] Updated weights for policy 0, policy_version 1870226 (0.0007) [2023-12-27 18:36:14,717][20381] Updated weights for policy 1, policy_version 1868634 (0.0008) [2023-12-27 18:36:14,770][20381] Updated weights for policy 1, policy_version 1868644 (0.0008) [2023-12-27 18:36:14,831][20381] Updated weights for policy 1, policy_version 1868654 (0.0008) [2023-12-27 18:36:14,887][20381] Updated weights for policy 1, policy_version 1868664 (0.0009) [2023-12-27 18:36:14,891][20347] Updated weights for policy 0, policy_version 1870236 (0.0007) [2023-12-27 18:36:14,957][20347] Updated weights for policy 0, policy_version 1870246 (0.0007) [2023-12-27 18:36:15,022][20347] Updated weights for policy 0, policy_version 1870256 (0.0009) [2023-12-27 18:36:15,545][20381] Updated weights for policy 1, policy_version 1868674 (0.0008) [2023-12-27 18:36:15,602][20381] Updated weights for policy 1, policy_version 1868684 (0.0008) [2023-12-27 18:36:15,661][20381] Updated weights for policy 1, policy_version 1868694 (0.0008) [2023-12-27 18:36:15,727][20347] Updated weights for policy 0, policy_version 1870266 (0.0008) [2023-12-27 18:36:15,786][20347] Updated weights for policy 0, policy_version 1870276 (0.0007) [2023-12-27 18:36:15,838][20347] Updated weights for policy 0, policy_version 1870286 (0.0008) [2023-12-27 18:36:15,885][20347] Updated weights for policy 0, policy_version 1870296 (0.0007) [2023-12-27 18:36:16,368][20381] Updated weights for policy 1, policy_version 1868704 (0.0006) [2023-12-27 18:36:16,432][20381] Updated weights for policy 1, policy_version 1868714 (0.0007) [2023-12-27 18:36:16,489][20381] Updated weights for policy 1, policy_version 1868724 (0.0008) [2023-12-27 18:36:16,619][20347] Updated weights for policy 0, policy_version 1870306 (0.0007) [2023-12-27 18:36:16,675][20347] Updated weights for policy 0, policy_version 1870316 (0.0008) [2023-12-27 18:36:16,730][20347] Updated weights for policy 0, policy_version 1870326 (0.0008) [2023-12-27 18:36:17,068][20381] Updated weights for policy 1, policy_version 1868734 (0.0007) [2023-12-27 18:36:17,123][20381] Updated weights for policy 1, policy_version 1868744 (0.0006) [2023-12-27 18:36:17,179][20381] Updated weights for policy 1, policy_version 1868754 (0.0006) [2023-12-27 18:36:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 957341696. Throughput: 0: 10310.6, 1: 10246.4. Samples: 957352124. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.500')] [2023-12-27 18:36:17,424][20347] Updated weights for policy 0, policy_version 1870336 (0.0006) [2023-12-27 18:36:17,482][20347] Updated weights for policy 0, policy_version 1870346 (0.0005) [2023-12-27 18:36:17,534][20347] Updated weights for policy 0, policy_version 1870356 (0.0007) [2023-12-27 18:36:17,859][20381] Updated weights for policy 1, policy_version 1868764 (0.0009) [2023-12-27 18:36:17,911][20381] Updated weights for policy 1, policy_version 1868774 (0.0011) [2023-12-27 18:36:17,962][20381] Updated weights for policy 1, policy_version 1868784 (0.0010) [2023-12-27 18:36:18,101][20347] Updated weights for policy 0, policy_version 1870366 (0.0006) [2023-12-27 18:36:18,155][20347] Updated weights for policy 0, policy_version 1870376 (0.0007) [2023-12-27 18:36:18,205][20347] Updated weights for policy 0, policy_version 1870386 (0.0008) [2023-12-27 18:36:18,577][20381] Updated weights for policy 1, policy_version 1868794 (0.0007) [2023-12-27 18:36:18,642][20381] Updated weights for policy 1, policy_version 1868804 (0.0008) [2023-12-27 18:36:18,704][20381] Updated weights for policy 1, policy_version 1868814 (0.0010) [2023-12-27 18:36:18,767][20381] Updated weights for policy 1, policy_version 1868824 (0.0011) [2023-12-27 18:36:18,924][20347] Updated weights for policy 0, policy_version 1870396 (0.0006) [2023-12-27 18:36:18,976][20347] Updated weights for policy 0, policy_version 1870406 (0.0008) [2023-12-27 18:36:19,029][20347] Updated weights for policy 0, policy_version 1870416 (0.0005) [2023-12-27 18:36:19,445][20381] Updated weights for policy 1, policy_version 1868834 (0.0007) [2023-12-27 18:36:19,515][20381] Updated weights for policy 1, policy_version 1868844 (0.0011) [2023-12-27 18:36:19,578][20381] Updated weights for policy 1, policy_version 1868854 (0.0007) [2023-12-27 18:36:19,672][20347] Updated weights for policy 0, policy_version 1870426 (0.0005) [2023-12-27 18:36:19,726][20347] Updated weights for policy 0, policy_version 1870436 (0.0008) [2023-12-27 18:36:19,788][20347] Updated weights for policy 0, policy_version 1870446 (0.0008) [2023-12-27 18:36:19,850][20347] Updated weights for policy 0, policy_version 1870456 (0.0009) [2023-12-27 18:36:20,219][20381] Updated weights for policy 1, policy_version 1868864 (0.0010) [2023-12-27 18:36:20,279][20381] Updated weights for policy 1, policy_version 1868874 (0.0011) [2023-12-27 18:36:20,336][20381] Updated weights for policy 1, policy_version 1868884 (0.0011) [2023-12-27 18:36:20,593][20347] Updated weights for policy 0, policy_version 1870466 (0.0011) [2023-12-27 18:36:20,677][20347] Updated weights for policy 0, policy_version 1870476 (0.0010) [2023-12-27 18:36:20,730][20347] Updated weights for policy 0, policy_version 1870486 (0.0010) [2023-12-27 18:36:21,069][20381] Updated weights for policy 1, policy_version 1868894 (0.0010) [2023-12-27 18:36:21,129][20381] Updated weights for policy 1, policy_version 1868904 (0.0010) [2023-12-27 18:36:21,193][20381] Updated weights for policy 1, policy_version 1868914 (0.0007) [2023-12-27 18:36:21,469][20347] Updated weights for policy 0, policy_version 1870496 (0.0009) [2023-12-27 18:36:21,525][20347] Updated weights for policy 0, policy_version 1870506 (0.0007) [2023-12-27 18:36:21,581][20347] Updated weights for policy 0, policy_version 1870516 (0.0007) [2023-12-27 18:36:21,878][20381] Updated weights for policy 1, policy_version 1868924 (0.0007) [2023-12-27 18:36:21,937][20381] Updated weights for policy 1, policy_version 1868934 (0.0005) [2023-12-27 18:36:21,996][20381] Updated weights for policy 1, policy_version 1868944 (0.0006) [2023-12-27 18:36:22,306][20347] Updated weights for policy 0, policy_version 1870526 (0.0007) [2023-12-27 18:36:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 957440000. Throughput: 0: 10321.4, 1: 10206.2. Samples: 957415024. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '831.940')] [2023-12-27 18:36:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001868952_478502912.pth... [2023-12-27 18:36:22,375][20347] Updated weights for policy 0, policy_version 1870536 (0.0007) [2023-12-27 18:36:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001867768_478199808.pth [2023-12-27 18:36:22,439][20347] Updated weights for policy 0, policy_version 1870546 (0.0007) [2023-12-27 18:36:22,475][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001870552_478945280.pth... [2023-12-27 18:36:22,479][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001869336_478633984.pth [2023-12-27 18:36:22,714][20381] Updated weights for policy 1, policy_version 1868954 (0.0006) [2023-12-27 18:36:22,783][20381] Updated weights for policy 1, policy_version 1868964 (0.0005) [2023-12-27 18:36:22,843][20381] Updated weights for policy 1, policy_version 1868974 (0.0008) [2023-12-27 18:36:22,907][20381] Updated weights for policy 1, policy_version 1868984 (0.0008) [2023-12-27 18:36:23,059][20347] Updated weights for policy 0, policy_version 1870556 (0.0006) [2023-12-27 18:36:23,114][20347] Updated weights for policy 0, policy_version 1870566 (0.0008) [2023-12-27 18:36:23,172][20347] Updated weights for policy 0, policy_version 1870576 (0.0008) [2023-12-27 18:36:23,548][20381] Updated weights for policy 1, policy_version 1868994 (0.0005) [2023-12-27 18:36:23,606][20381] Updated weights for policy 1, policy_version 1869004 (0.0006) [2023-12-27 18:36:23,670][20381] Updated weights for policy 1, policy_version 1869014 (0.0005) [2023-12-27 18:36:23,808][20347] Updated weights for policy 0, policy_version 1870586 (0.0006) [2023-12-27 18:36:23,854][20347] Updated weights for policy 0, policy_version 1870596 (0.0006) [2023-12-27 18:36:23,905][20347] Updated weights for policy 0, policy_version 1870606 (0.0007) [2023-12-27 18:36:23,966][20347] Updated weights for policy 0, policy_version 1870616 (0.0008) [2023-12-27 18:36:24,215][20381] Updated weights for policy 1, policy_version 1869024 (0.0007) [2023-12-27 18:36:24,274][20381] Updated weights for policy 1, policy_version 1869034 (0.0007) [2023-12-27 18:36:24,323][20381] Updated weights for policy 1, policy_version 1869044 (0.0008) [2023-12-27 18:36:24,566][20347] Updated weights for policy 0, policy_version 1870626 (0.0005) [2023-12-27 18:36:24,623][20347] Updated weights for policy 0, policy_version 1870636 (0.0006) [2023-12-27 18:36:24,673][20347] Updated weights for policy 0, policy_version 1870646 (0.0008) [2023-12-27 18:36:25,059][20381] Updated weights for policy 1, policy_version 1869054 (0.0008) [2023-12-27 18:36:25,104][20381] Updated weights for policy 1, policy_version 1869064 (0.0008) [2023-12-27 18:36:25,152][20381] Updated weights for policy 1, policy_version 1869074 (0.0008) [2023-12-27 18:36:25,317][20347] Updated weights for policy 0, policy_version 1870656 (0.0008) [2023-12-27 18:36:25,371][20347] Updated weights for policy 0, policy_version 1870666 (0.0008) [2023-12-27 18:36:25,419][20347] Updated weights for policy 0, policy_version 1870676 (0.0008) [2023-12-27 18:36:25,748][20381] Updated weights for policy 1, policy_version 1869084 (0.0007) [2023-12-27 18:36:25,797][20381] Updated weights for policy 1, policy_version 1869094 (0.0005) [2023-12-27 18:36:25,840][20381] Updated weights for policy 1, policy_version 1869104 (0.0005) [2023-12-27 18:36:26,088][20347] Updated weights for policy 0, policy_version 1870686 (0.0007) [2023-12-27 18:36:26,142][20347] Updated weights for policy 0, policy_version 1870696 (0.0005) [2023-12-27 18:36:26,192][20347] Updated weights for policy 0, policy_version 1870706 (0.0005) [2023-12-27 18:36:26,394][20381] Updated weights for policy 1, policy_version 1869114 (0.0006) [2023-12-27 18:36:26,446][20381] Updated weights for policy 1, policy_version 1869124 (0.0008) [2023-12-27 18:36:26,501][20381] Updated weights for policy 1, policy_version 1869134 (0.0008) [2023-12-27 18:36:26,556][20381] Updated weights for policy 1, policy_version 1869144 (0.0007) [2023-12-27 18:36:26,831][20347] Updated weights for policy 0, policy_version 1870716 (0.0006) [2023-12-27 18:36:26,885][20347] Updated weights for policy 0, policy_version 1870726 (0.0007) [2023-12-27 18:36:26,948][20347] Updated weights for policy 0, policy_version 1870736 (0.0008) [2023-12-27 18:36:27,194][20381] Updated weights for policy 1, policy_version 1869154 (0.0006) [2023-12-27 18:36:27,240][20381] Updated weights for policy 1, policy_version 1869164 (0.0005) [2023-12-27 18:36:27,296][20381] Updated weights for policy 1, policy_version 1869174 (0.0006) [2023-12-27 18:36:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 957554688. Throughput: 0: 10339.7, 1: 10258.4. Samples: 957540300. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '831.940')] [2023-12-27 18:36:27,640][20347] Updated weights for policy 0, policy_version 1870746 (0.0006) [2023-12-27 18:36:27,695][20347] Updated weights for policy 0, policy_version 1870756 (0.0009) [2023-12-27 18:36:27,746][20347] Updated weights for policy 0, policy_version 1870766 (0.0009) [2023-12-27 18:36:27,797][20347] Updated weights for policy 0, policy_version 1870776 (0.0008) [2023-12-27 18:36:27,890][20381] Updated weights for policy 1, policy_version 1869184 (0.0006) [2023-12-27 18:36:27,935][20381] Updated weights for policy 1, policy_version 1869194 (0.0007) [2023-12-27 18:36:27,981][20381] Updated weights for policy 1, policy_version 1869204 (0.0007) [2023-12-27 18:36:28,508][20347] Updated weights for policy 0, policy_version 1870786 (0.0005) [2023-12-27 18:36:28,569][20347] Updated weights for policy 0, policy_version 1870796 (0.0005) [2023-12-27 18:36:28,627][20347] Updated weights for policy 0, policy_version 1870806 (0.0005) [2023-12-27 18:36:28,671][20381] Updated weights for policy 1, policy_version 1869214 (0.0009) [2023-12-27 18:36:28,732][20381] Updated weights for policy 1, policy_version 1869224 (0.0010) [2023-12-27 18:36:28,785][20381] Updated weights for policy 1, policy_version 1869234 (0.0010) [2023-12-27 18:36:29,169][20347] Updated weights for policy 0, policy_version 1870816 (0.0005) [2023-12-27 18:36:29,224][20347] Updated weights for policy 0, policy_version 1870826 (0.0006) [2023-12-27 18:36:29,290][20347] Updated weights for policy 0, policy_version 1870836 (0.0009) [2023-12-27 18:36:29,488][20381] Updated weights for policy 1, policy_version 1869244 (0.0010) [2023-12-27 18:36:29,537][20381] Updated weights for policy 1, policy_version 1869254 (0.0007) [2023-12-27 18:36:29,598][20381] Updated weights for policy 1, policy_version 1869264 (0.0006) [2023-12-27 18:36:30,022][20347] Updated weights for policy 0, policy_version 1870846 (0.0007) [2023-12-27 18:36:30,085][20347] Updated weights for policy 0, policy_version 1870856 (0.0005) [2023-12-27 18:36:30,146][20347] Updated weights for policy 0, policy_version 1870866 (0.0005) [2023-12-27 18:36:30,264][20381] Updated weights for policy 1, policy_version 1869274 (0.0006) [2023-12-27 18:36:30,321][20381] Updated weights for policy 1, policy_version 1869284 (0.0008) [2023-12-27 18:36:30,377][20381] Updated weights for policy 1, policy_version 1869294 (0.0008) [2023-12-27 18:36:30,430][20381] Updated weights for policy 1, policy_version 1869304 (0.0008) [2023-12-27 18:36:30,793][20347] Updated weights for policy 0, policy_version 1870876 (0.0006) [2023-12-27 18:36:30,852][20347] Updated weights for policy 0, policy_version 1870886 (0.0008) [2023-12-27 18:36:30,898][20347] Updated weights for policy 0, policy_version 1870896 (0.0007) [2023-12-27 18:36:31,139][20381] Updated weights for policy 1, policy_version 1869314 (0.0008) [2023-12-27 18:36:31,203][20381] Updated weights for policy 1, policy_version 1869324 (0.0008) [2023-12-27 18:36:31,260][20381] Updated weights for policy 1, policy_version 1869334 (0.0006) [2023-12-27 18:36:31,628][20347] Updated weights for policy 0, policy_version 1870906 (0.0008) [2023-12-27 18:36:31,687][20347] Updated weights for policy 0, policy_version 1870916 (0.0008) [2023-12-27 18:36:31,748][20347] Updated weights for policy 0, policy_version 1870926 (0.0007) [2023-12-27 18:36:31,802][20347] Updated weights for policy 0, policy_version 1870936 (0.0008) [2023-12-27 18:36:31,926][20381] Updated weights for policy 1, policy_version 1869344 (0.0008) [2023-12-27 18:36:31,974][20381] Updated weights for policy 1, policy_version 1869354 (0.0008) [2023-12-27 18:36:32,027][20381] Updated weights for policy 1, policy_version 1869364 (0.0008) [2023-12-27 18:36:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 957652992. Throughput: 0: 10292.1, 1: 10277.8. Samples: 957663864. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '914.330')] [2023-12-27 18:36:32,472][20347] Updated weights for policy 0, policy_version 1870946 (0.0006) [2023-12-27 18:36:32,530][20347] Updated weights for policy 0, policy_version 1870956 (0.0005) [2023-12-27 18:36:32,588][20347] Updated weights for policy 0, policy_version 1870966 (0.0005) [2023-12-27 18:36:32,762][20381] Updated weights for policy 1, policy_version 1869374 (0.0007) [2023-12-27 18:36:32,810][20381] Updated weights for policy 1, policy_version 1869384 (0.0007) [2023-12-27 18:36:32,862][20381] Updated weights for policy 1, policy_version 1869394 (0.0008) [2023-12-27 18:36:33,232][20347] Updated weights for policy 0, policy_version 1870976 (0.0008) [2023-12-27 18:36:33,290][20347] Updated weights for policy 0, policy_version 1870986 (0.0008) [2023-12-27 18:36:33,343][20347] Updated weights for policy 0, policy_version 1870996 (0.0008) [2023-12-27 18:36:33,497][20381] Updated weights for policy 1, policy_version 1869404 (0.0008) [2023-12-27 18:36:33,559][20381] Updated weights for policy 1, policy_version 1869414 (0.0005) [2023-12-27 18:36:33,615][20381] Updated weights for policy 1, policy_version 1869424 (0.0005) [2023-12-27 18:36:33,970][20347] Updated weights for policy 0, policy_version 1871006 (0.0009) [2023-12-27 18:36:34,024][20347] Updated weights for policy 0, policy_version 1871016 (0.0007) [2023-12-27 18:36:34,073][20347] Updated weights for policy 0, policy_version 1871026 (0.0008) [2023-12-27 18:36:34,188][20381] Updated weights for policy 1, policy_version 1869434 (0.0006) [2023-12-27 18:36:34,248][20381] Updated weights for policy 1, policy_version 1869444 (0.0008) [2023-12-27 18:36:34,307][20381] Updated weights for policy 1, policy_version 1869454 (0.0009) [2023-12-27 18:36:34,365][20381] Updated weights for policy 1, policy_version 1869464 (0.0008) [2023-12-27 18:36:34,738][20347] Updated weights for policy 0, policy_version 1871036 (0.0007) [2023-12-27 18:36:34,794][20347] Updated weights for policy 0, policy_version 1871046 (0.0005) [2023-12-27 18:36:34,849][20347] Updated weights for policy 0, policy_version 1871056 (0.0006) [2023-12-27 18:36:35,107][20381] Updated weights for policy 1, policy_version 1869474 (0.0007) [2023-12-27 18:36:35,159][20381] Updated weights for policy 1, policy_version 1869484 (0.0008) [2023-12-27 18:36:35,213][20381] Updated weights for policy 1, policy_version 1869494 (0.0006) [2023-12-27 18:36:35,459][20347] Updated weights for policy 0, policy_version 1871066 (0.0007) [2023-12-27 18:36:35,509][20347] Updated weights for policy 0, policy_version 1871076 (0.0006) [2023-12-27 18:36:35,558][20347] Updated weights for policy 0, policy_version 1871086 (0.0007) [2023-12-27 18:36:35,604][20347] Updated weights for policy 0, policy_version 1871096 (0.0008) [2023-12-27 18:36:35,931][20381] Updated weights for policy 1, policy_version 1869504 (0.0008) [2023-12-27 18:36:35,987][20381] Updated weights for policy 1, policy_version 1869514 (0.0007) [2023-12-27 18:36:36,046][20381] Updated weights for policy 1, policy_version 1869524 (0.0006) [2023-12-27 18:36:36,249][20347] Updated weights for policy 0, policy_version 1871106 (0.0006) [2023-12-27 18:36:36,313][20347] Updated weights for policy 0, policy_version 1871116 (0.0006) [2023-12-27 18:36:36,375][20347] Updated weights for policy 0, policy_version 1871126 (0.0007) [2023-12-27 18:36:36,757][20381] Updated weights for policy 1, policy_version 1869534 (0.0007) [2023-12-27 18:36:36,819][20381] Updated weights for policy 1, policy_version 1869544 (0.0007) [2023-12-27 18:36:36,878][20381] Updated weights for policy 1, policy_version 1869554 (0.0008) [2023-12-27 18:36:37,026][20347] Updated weights for policy 0, policy_version 1871136 (0.0010) [2023-12-27 18:36:37,079][20347] Updated weights for policy 0, policy_version 1871146 (0.0010) [2023-12-27 18:36:37,136][20347] Updated weights for policy 0, policy_version 1871156 (0.0008) [2023-12-27 18:36:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 957759488. Throughput: 0: 10331.4, 1: 10311.2. Samples: 957727316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '879.340')] [2023-12-27 18:36:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001869560_478658560.pth... [2023-12-27 18:36:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001871160_479100928.pth... [2023-12-27 18:36:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001869944_478789632.pth [2023-12-27 18:36:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001868344_478347264.pth [2023-12-27 18:36:37,605][20381] Updated weights for policy 1, policy_version 1869564 (0.0008) [2023-12-27 18:36:37,665][20381] Updated weights for policy 1, policy_version 1869574 (0.0007) [2023-12-27 18:36:37,723][20381] Updated weights for policy 1, policy_version 1869584 (0.0007) [2023-12-27 18:36:37,769][20347] Updated weights for policy 0, policy_version 1871166 (0.0010) [2023-12-27 18:36:37,826][20347] Updated weights for policy 0, policy_version 1871176 (0.0010) [2023-12-27 18:36:37,884][20347] Updated weights for policy 0, policy_version 1871186 (0.0010) [2023-12-27 18:36:38,447][20381] Updated weights for policy 1, policy_version 1869594 (0.0006) [2023-12-27 18:36:38,488][20347] Updated weights for policy 0, policy_version 1871196 (0.0008) [2023-12-27 18:36:38,506][20381] Updated weights for policy 1, policy_version 1869604 (0.0008) [2023-12-27 18:36:38,551][20347] Updated weights for policy 0, policy_version 1871206 (0.0007) [2023-12-27 18:36:38,560][20381] Updated weights for policy 1, policy_version 1869614 (0.0007) [2023-12-27 18:36:38,613][20347] Updated weights for policy 0, policy_version 1871216 (0.0011) [2023-12-27 18:36:38,619][20381] Updated weights for policy 1, policy_version 1869624 (0.0006) [2023-12-27 18:36:39,253][20347] Updated weights for policy 0, policy_version 1871226 (0.0010) [2023-12-27 18:36:39,253][20381] Updated weights for policy 1, policy_version 1869634 (0.0009) [2023-12-27 18:36:39,310][20381] Updated weights for policy 1, policy_version 1869644 (0.0006) [2023-12-27 18:36:39,310][20347] Updated weights for policy 0, policy_version 1871236 (0.0011) [2023-12-27 18:36:39,371][20381] Updated weights for policy 1, policy_version 1869654 (0.0007) [2023-12-27 18:36:39,377][20347] Updated weights for policy 0, policy_version 1871246 (0.0009) [2023-12-27 18:36:39,439][20347] Updated weights for policy 0, policy_version 1871256 (0.0008) [2023-12-27 18:36:40,061][20381] Updated weights for policy 1, policy_version 1869664 (0.0009) [2023-12-27 18:36:40,117][20381] Updated weights for policy 1, policy_version 1869674 (0.0008) [2023-12-27 18:36:40,180][20381] Updated weights for policy 1, policy_version 1869684 (0.0007) [2023-12-27 18:36:40,195][20347] Updated weights for policy 0, policy_version 1871266 (0.0007) [2023-12-27 18:36:40,254][20347] Updated weights for policy 0, policy_version 1871276 (0.0010) [2023-12-27 18:36:40,309][20347] Updated weights for policy 0, policy_version 1871286 (0.0010) [2023-12-27 18:36:40,918][20381] Updated weights for policy 1, policy_version 1869694 (0.0006) [2023-12-27 18:36:40,961][20347] Updated weights for policy 0, policy_version 1871296 (0.0005) [2023-12-27 18:36:40,973][20381] Updated weights for policy 1, policy_version 1869704 (0.0005) [2023-12-27 18:36:41,019][20347] Updated weights for policy 0, policy_version 1871306 (0.0006) [2023-12-27 18:36:41,032][20381] Updated weights for policy 1, policy_version 1869714 (0.0006) [2023-12-27 18:36:41,086][20347] Updated weights for policy 0, policy_version 1871316 (0.0008) [2023-12-27 18:36:41,711][20381] Updated weights for policy 1, policy_version 1869724 (0.0007) [2023-12-27 18:36:41,776][20381] Updated weights for policy 1, policy_version 1869734 (0.0007) [2023-12-27 18:36:41,815][20347] Updated weights for policy 0, policy_version 1871326 (0.0008) [2023-12-27 18:36:41,842][20381] Updated weights for policy 1, policy_version 1869744 (0.0006) [2023-12-27 18:36:41,876][20347] Updated weights for policy 0, policy_version 1871336 (0.0008) [2023-12-27 18:36:41,940][20347] Updated weights for policy 0, policy_version 1871346 (0.0009) [2023-12-27 18:36:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 957857792. Throughput: 0: 10354.2, 1: 10267.9. Samples: 957849880. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '826.280')] [2023-12-27 18:36:42,465][20381] Updated weights for policy 1, policy_version 1869754 (0.0007) [2023-12-27 18:36:42,525][20381] Updated weights for policy 1, policy_version 1869764 (0.0011) [2023-12-27 18:36:42,587][20381] Updated weights for policy 1, policy_version 1869774 (0.0011) [2023-12-27 18:36:42,637][20347] Updated weights for policy 0, policy_version 1871356 (0.0007) [2023-12-27 18:36:42,646][20381] Updated weights for policy 1, policy_version 1869784 (0.0011) [2023-12-27 18:36:42,692][20347] Updated weights for policy 0, policy_version 1871366 (0.0006) [2023-12-27 18:36:42,739][20347] Updated weights for policy 0, policy_version 1871376 (0.0007) [2023-12-27 18:36:43,249][20381] Updated weights for policy 1, policy_version 1869794 (0.0008) [2023-12-27 18:36:43,298][20381] Updated weights for policy 1, policy_version 1869804 (0.0007) [2023-12-27 18:36:43,350][20381] Updated weights for policy 1, policy_version 1869814 (0.0008) [2023-12-27 18:36:43,449][20347] Updated weights for policy 0, policy_version 1871386 (0.0008) [2023-12-27 18:36:43,506][20347] Updated weights for policy 0, policy_version 1871396 (0.0008) [2023-12-27 18:36:43,560][20347] Updated weights for policy 0, policy_version 1871406 (0.0007) [2023-12-27 18:36:43,622][20347] Updated weights for policy 0, policy_version 1871416 (0.0008) [2023-12-27 18:36:43,990][20381] Updated weights for policy 1, policy_version 1869824 (0.0009) [2023-12-27 18:36:44,042][20381] Updated weights for policy 1, policy_version 1869834 (0.0010) [2023-12-27 18:36:44,093][20381] Updated weights for policy 1, policy_version 1869844 (0.0010) [2023-12-27 18:36:44,316][20347] Updated weights for policy 0, policy_version 1871426 (0.0010) [2023-12-27 18:36:44,365][20347] Updated weights for policy 0, policy_version 1871436 (0.0007) [2023-12-27 18:36:44,419][20347] Updated weights for policy 0, policy_version 1871446 (0.0006) [2023-12-27 18:36:44,781][20381] Updated weights for policy 1, policy_version 1869854 (0.0009) [2023-12-27 18:36:44,839][20381] Updated weights for policy 1, policy_version 1869864 (0.0007) [2023-12-27 18:36:44,898][20381] Updated weights for policy 1, policy_version 1869874 (0.0008) [2023-12-27 18:36:45,115][20347] Updated weights for policy 0, policy_version 1871456 (0.0010) [2023-12-27 18:36:45,174][20347] Updated weights for policy 0, policy_version 1871466 (0.0011) [2023-12-27 18:36:45,233][20347] Updated weights for policy 0, policy_version 1871476 (0.0010) [2023-12-27 18:36:45,590][20381] Updated weights for policy 1, policy_version 1869884 (0.0006) [2023-12-27 18:36:45,648][20381] Updated weights for policy 1, policy_version 1869894 (0.0008) [2023-12-27 18:36:45,709][20381] Updated weights for policy 1, policy_version 1869904 (0.0010) [2023-12-27 18:36:45,844][20347] Updated weights for policy 0, policy_version 1871486 (0.0010) [2023-12-27 18:36:45,897][20347] Updated weights for policy 0, policy_version 1871496 (0.0010) [2023-12-27 18:36:45,946][20347] Updated weights for policy 0, policy_version 1871506 (0.0010) [2023-12-27 18:36:46,337][20381] Updated weights for policy 1, policy_version 1869914 (0.0010) [2023-12-27 18:36:46,394][20381] Updated weights for policy 1, policy_version 1869924 (0.0009) [2023-12-27 18:36:46,450][20381] Updated weights for policy 1, policy_version 1869934 (0.0010) [2023-12-27 18:36:46,509][20381] Updated weights for policy 1, policy_version 1869944 (0.0011) [2023-12-27 18:36:46,667][20347] Updated weights for policy 0, policy_version 1871516 (0.0010) [2023-12-27 18:36:46,729][20347] Updated weights for policy 0, policy_version 1871526 (0.0010) [2023-12-27 18:36:46,787][20347] Updated weights for policy 0, policy_version 1871536 (0.0010) [2023-12-27 18:36:47,200][20381] Updated weights for policy 1, policy_version 1869954 (0.0010) [2023-12-27 18:36:47,262][20381] Updated weights for policy 1, policy_version 1869964 (0.0010) [2023-12-27 18:36:47,313][20381] Updated weights for policy 1, policy_version 1869974 (0.0010) [2023-12-27 18:36:47,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 957964288. Throughput: 0: 10290.2, 1: 10331.8. Samples: 957972708. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '866.660')] [2023-12-27 18:36:47,468][20347] Updated weights for policy 0, policy_version 1871546 (0.0011) [2023-12-27 18:36:47,543][20347] Updated weights for policy 0, policy_version 1871556 (0.0011) [2023-12-27 18:36:47,596][20347] Updated weights for policy 0, policy_version 1871566 (0.0010) [2023-12-27 18:36:47,654][20347] Updated weights for policy 0, policy_version 1871576 (0.0009) [2023-12-27 18:36:47,987][20381] Updated weights for policy 1, policy_version 1869984 (0.0009) [2023-12-27 18:36:48,040][20381] Updated weights for policy 1, policy_version 1869994 (0.0008) [2023-12-27 18:36:48,089][20381] Updated weights for policy 1, policy_version 1870004 (0.0008) [2023-12-27 18:36:48,285][20347] Updated weights for policy 0, policy_version 1871586 (0.0007) [2023-12-27 18:36:48,350][20347] Updated weights for policy 0, policy_version 1871596 (0.0008) [2023-12-27 18:36:48,411][20347] Updated weights for policy 0, policy_version 1871606 (0.0007) [2023-12-27 18:36:48,801][20381] Updated weights for policy 1, policy_version 1870014 (0.0010) [2023-12-27 18:36:48,864][20381] Updated weights for policy 1, policy_version 1870024 (0.0011) [2023-12-27 18:36:48,932][20381] Updated weights for policy 1, policy_version 1870034 (0.0010) [2023-12-27 18:36:49,101][20347] Updated weights for policy 0, policy_version 1871616 (0.0009) [2023-12-27 18:36:49,152][20347] Updated weights for policy 0, policy_version 1871626 (0.0009) [2023-12-27 18:36:49,218][20347] Updated weights for policy 0, policy_version 1871636 (0.0009) [2023-12-27 18:36:49,647][20381] Updated weights for policy 1, policy_version 1870044 (0.0010) [2023-12-27 18:36:49,702][20381] Updated weights for policy 1, policy_version 1870054 (0.0007) [2023-12-27 18:36:49,759][20381] Updated weights for policy 1, policy_version 1870064 (0.0007) [2023-12-27 18:36:49,975][20347] Updated weights for policy 0, policy_version 1871646 (0.0010) [2023-12-27 18:36:50,035][20347] Updated weights for policy 0, policy_version 1871656 (0.0010) [2023-12-27 18:36:50,092][20347] Updated weights for policy 0, policy_version 1871666 (0.0007) [2023-12-27 18:36:50,437][20381] Updated weights for policy 1, policy_version 1870074 (0.0007) [2023-12-27 18:36:50,490][20381] Updated weights for policy 1, policy_version 1870084 (0.0006) [2023-12-27 18:36:50,545][20381] Updated weights for policy 1, policy_version 1870094 (0.0006) [2023-12-27 18:36:50,606][20381] Updated weights for policy 1, policy_version 1870104 (0.0006) [2023-12-27 18:36:50,724][20347] Updated weights for policy 0, policy_version 1871676 (0.0006) [2023-12-27 18:36:50,777][20347] Updated weights for policy 0, policy_version 1871686 (0.0008) [2023-12-27 18:36:50,829][20347] Updated weights for policy 0, policy_version 1871696 (0.0007) [2023-12-27 18:36:51,237][20381] Updated weights for policy 1, policy_version 1870114 (0.0006) [2023-12-27 18:36:51,298][20381] Updated weights for policy 1, policy_version 1870124 (0.0007) [2023-12-27 18:36:51,357][20381] Updated weights for policy 1, policy_version 1870134 (0.0007) [2023-12-27 18:36:51,625][20347] Updated weights for policy 0, policy_version 1871706 (0.0009) [2023-12-27 18:36:51,678][20347] Updated weights for policy 0, policy_version 1871716 (0.0008) [2023-12-27 18:36:51,725][20347] Updated weights for policy 0, policy_version 1871726 (0.0008) [2023-12-27 18:36:51,785][20347] Updated weights for policy 0, policy_version 1871736 (0.0008) [2023-12-27 18:36:52,014][20381] Updated weights for policy 1, policy_version 1870144 (0.0007) [2023-12-27 18:36:52,080][20381] Updated weights for policy 1, policy_version 1870154 (0.0007) [2023-12-27 18:36:52,145][20381] Updated weights for policy 1, policy_version 1870164 (0.0008) [2023-12-27 18:36:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 958062592. Throughput: 0: 10270.6, 1: 10324.4. Samples: 958032612. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:36:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.120')] [2023-12-27 18:36:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001870168_478814208.pth... [2023-12-27 18:36:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001871736_479248384.pth... [2023-12-27 18:36:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001868952_478502912.pth [2023-12-27 18:36:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001870552_478945280.pth [2023-12-27 18:36:52,532][20347] Updated weights for policy 0, policy_version 1871746 (0.0007) [2023-12-27 18:36:52,591][20347] Updated weights for policy 0, policy_version 1871756 (0.0007) [2023-12-27 18:36:52,644][20347] Updated weights for policy 0, policy_version 1871766 (0.0007) [2023-12-27 18:36:52,833][20381] Updated weights for policy 1, policy_version 1870174 (0.0008) [2023-12-27 18:36:52,897][20381] Updated weights for policy 1, policy_version 1870184 (0.0008) [2023-12-27 18:36:52,963][20381] Updated weights for policy 1, policy_version 1870194 (0.0007) [2023-12-27 18:36:53,369][20347] Updated weights for policy 0, policy_version 1871776 (0.0008) [2023-12-27 18:36:53,418][20347] Updated weights for policy 0, policy_version 1871786 (0.0010) [2023-12-27 18:36:53,473][20347] Updated weights for policy 0, policy_version 1871796 (0.0007) [2023-12-27 18:36:53,528][20381] Updated weights for policy 1, policy_version 1870204 (0.0006) [2023-12-27 18:36:53,591][20381] Updated weights for policy 1, policy_version 1870214 (0.0006) [2023-12-27 18:36:53,653][20381] Updated weights for policy 1, policy_version 1870224 (0.0008) [2023-12-27 18:36:54,214][20347] Updated weights for policy 0, policy_version 1871806 (0.0009) [2023-12-27 18:36:54,243][20381] Updated weights for policy 1, policy_version 1870234 (0.0008) [2023-12-27 18:36:54,266][20347] Updated weights for policy 0, policy_version 1871816 (0.0006) [2023-12-27 18:36:54,301][20381] Updated weights for policy 1, policy_version 1870244 (0.0011) [2023-12-27 18:36:54,314][20347] Updated weights for policy 0, policy_version 1871826 (0.0005) [2023-12-27 18:36:54,356][20381] Updated weights for policy 1, policy_version 1870254 (0.0010) [2023-12-27 18:36:54,413][20381] Updated weights for policy 1, policy_version 1870264 (0.0011) [2023-12-27 18:36:54,956][20347] Updated weights for policy 0, policy_version 1871836 (0.0006) [2023-12-27 18:36:55,016][20347] Updated weights for policy 0, policy_version 1871846 (0.0007) [2023-12-27 18:36:55,071][20347] Updated weights for policy 0, policy_version 1871856 (0.0010) [2023-12-27 18:36:55,095][20381] Updated weights for policy 1, policy_version 1870274 (0.0005) [2023-12-27 18:36:55,142][20381] Updated weights for policy 1, policy_version 1870284 (0.0006) [2023-12-27 18:36:55,189][20381] Updated weights for policy 1, policy_version 1870294 (0.0009) [2023-12-27 18:36:55,732][20347] Updated weights for policy 0, policy_version 1871866 (0.0010) [2023-12-27 18:36:55,780][20347] Updated weights for policy 0, policy_version 1871876 (0.0010) [2023-12-27 18:36:55,828][20347] Updated weights for policy 0, policy_version 1871886 (0.0010) [2023-12-27 18:36:55,832][20381] Updated weights for policy 1, policy_version 1870304 (0.0007) [2023-12-27 18:36:55,878][20347] Updated weights for policy 0, policy_version 1871896 (0.0010) [2023-12-27 18:36:55,883][20381] Updated weights for policy 1, policy_version 1870314 (0.0006) [2023-12-27 18:36:55,938][20381] Updated weights for policy 1, policy_version 1870324 (0.0005) [2023-12-27 18:36:56,490][20381] Updated weights for policy 1, policy_version 1870334 (0.0007) [2023-12-27 18:36:56,493][20347] Updated weights for policy 0, policy_version 1871906 (0.0005) [2023-12-27 18:36:56,549][20381] Updated weights for policy 1, policy_version 1870344 (0.0007) [2023-12-27 18:36:56,553][20347] Updated weights for policy 0, policy_version 1871916 (0.0005) [2023-12-27 18:36:56,604][20381] Updated weights for policy 1, policy_version 1870354 (0.0005) [2023-12-27 18:36:56,610][20347] Updated weights for policy 0, policy_version 1871926 (0.0005) [2023-12-27 18:36:57,241][20347] Updated weights for policy 0, policy_version 1871936 (0.0005) [2023-12-27 18:36:57,289][20381] Updated weights for policy 1, policy_version 1870364 (0.0007) [2023-12-27 18:36:57,299][20347] Updated weights for policy 0, policy_version 1871946 (0.0005) [2023-12-27 18:36:57,342][20381] Updated weights for policy 1, policy_version 1870374 (0.0008) [2023-12-27 18:36:57,354][20347] Updated weights for policy 0, policy_version 1871956 (0.0005) [2023-12-27 18:36:57,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 958160896. Throughput: 0: 10199.7, 1: 10417.6. Samples: 958157088. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:36:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.840')] [2023-12-27 18:36:57,396][20381] Updated weights for policy 1, policy_version 1870384 (0.0008) [2023-12-27 18:36:57,878][20347] Updated weights for policy 0, policy_version 1871966 (0.0007) [2023-12-27 18:36:57,925][20347] Updated weights for policy 0, policy_version 1871976 (0.0007) [2023-12-27 18:36:57,979][20347] Updated weights for policy 0, policy_version 1871986 (0.0006) [2023-12-27 18:36:58,128][20381] Updated weights for policy 1, policy_version 1870394 (0.0009) [2023-12-27 18:36:58,197][20381] Updated weights for policy 1, policy_version 1870404 (0.0006) [2023-12-27 18:36:58,259][20381] Updated weights for policy 1, policy_version 1870414 (0.0008) [2023-12-27 18:36:58,319][20381] Updated weights for policy 1, policy_version 1870424 (0.0007) [2023-12-27 18:36:58,721][20347] Updated weights for policy 0, policy_version 1871996 (0.0007) [2023-12-27 18:36:58,789][20347] Updated weights for policy 0, policy_version 1872006 (0.0008) [2023-12-27 18:36:58,860][20347] Updated weights for policy 0, policy_version 1872016 (0.0008) [2023-12-27 18:36:59,116][20381] Updated weights for policy 1, policy_version 1870434 (0.0008) [2023-12-27 18:36:59,173][20381] Updated weights for policy 1, policy_version 1870444 (0.0007) [2023-12-27 18:36:59,240][20381] Updated weights for policy 1, policy_version 1870454 (0.0008) [2023-12-27 18:36:59,560][20347] Updated weights for policy 0, policy_version 1872026 (0.0006) [2023-12-27 18:36:59,611][20347] Updated weights for policy 0, policy_version 1872036 (0.0006) [2023-12-27 18:36:59,655][20347] Updated weights for policy 0, policy_version 1872046 (0.0006) [2023-12-27 18:36:59,710][20347] Updated weights for policy 0, policy_version 1872056 (0.0007) [2023-12-27 18:36:59,962][20381] Updated weights for policy 1, policy_version 1870464 (0.0007) [2023-12-27 18:37:00,018][20381] Updated weights for policy 1, policy_version 1870474 (0.0007) [2023-12-27 18:37:00,073][20381] Updated weights for policy 1, policy_version 1870484 (0.0005) [2023-12-27 18:37:00,468][20347] Updated weights for policy 0, policy_version 1872066 (0.0008) [2023-12-27 18:37:00,528][20347] Updated weights for policy 0, policy_version 1872076 (0.0007) [2023-12-27 18:37:00,587][20347] Updated weights for policy 0, policy_version 1872086 (0.0008) [2023-12-27 18:37:00,626][20381] Updated weights for policy 1, policy_version 1870494 (0.0007) [2023-12-27 18:37:00,685][20381] Updated weights for policy 1, policy_version 1870504 (0.0006) [2023-12-27 18:37:00,747][20381] Updated weights for policy 1, policy_version 1870514 (0.0005) [2023-12-27 18:37:01,352][20347] Updated weights for policy 0, policy_version 1872096 (0.0007) [2023-12-27 18:37:01,361][20381] Updated weights for policy 1, policy_version 1870524 (0.0006) [2023-12-27 18:37:01,411][20347] Updated weights for policy 0, policy_version 1872106 (0.0008) [2023-12-27 18:37:01,415][20381] Updated weights for policy 1, policy_version 1870534 (0.0007) [2023-12-27 18:37:01,471][20347] Updated weights for policy 0, policy_version 1872116 (0.0007) [2023-12-27 18:37:01,480][20381] Updated weights for policy 1, policy_version 1870544 (0.0007) [2023-12-27 18:37:02,123][20347] Updated weights for policy 0, policy_version 1872126 (0.0006) [2023-12-27 18:37:02,185][20347] Updated weights for policy 0, policy_version 1872136 (0.0005) [2023-12-27 18:37:02,193][20381] Updated weights for policy 1, policy_version 1870554 (0.0007) [2023-12-27 18:37:02,242][20347] Updated weights for policy 0, policy_version 1872146 (0.0006) [2023-12-27 18:37:02,253][20381] Updated weights for policy 1, policy_version 1870564 (0.0009) [2023-12-27 18:37:02,315][20381] Updated weights for policy 1, policy_version 1870574 (0.0007) [2023-12-27 18:37:02,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 958267392. Throughput: 0: 10273.4, 1: 10339.4. Samples: 958279700. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.180')] [2023-12-27 18:37:02,380][20381] Updated weights for policy 1, policy_version 1870584 (0.0007) [2023-12-27 18:37:02,850][20347] Updated weights for policy 0, policy_version 1872156 (0.0010) [2023-12-27 18:37:02,906][20347] Updated weights for policy 0, policy_version 1872166 (0.0007) [2023-12-27 18:37:02,958][20347] Updated weights for policy 0, policy_version 1872176 (0.0008) [2023-12-27 18:37:03,018][20381] Updated weights for policy 1, policy_version 1870594 (0.0006) [2023-12-27 18:37:03,068][20381] Updated weights for policy 1, policy_version 1870604 (0.0007) [2023-12-27 18:37:03,113][20381] Updated weights for policy 1, policy_version 1870614 (0.0010) [2023-12-27 18:37:03,553][20347] Updated weights for policy 0, policy_version 1872186 (0.0007) [2023-12-27 18:37:03,610][20347] Updated weights for policy 0, policy_version 1872196 (0.0005) [2023-12-27 18:37:03,670][20347] Updated weights for policy 0, policy_version 1872206 (0.0005) [2023-12-27 18:37:03,741][20347] Updated weights for policy 0, policy_version 1872216 (0.0005) [2023-12-27 18:37:03,767][20381] Updated weights for policy 1, policy_version 1870624 (0.0010) [2023-12-27 18:37:03,826][20381] Updated weights for policy 1, policy_version 1870634 (0.0010) [2023-12-27 18:37:03,883][20381] Updated weights for policy 1, policy_version 1870644 (0.0008) [2023-12-27 18:37:04,319][20347] Updated weights for policy 0, policy_version 1872226 (0.0011) [2023-12-27 18:37:04,383][20347] Updated weights for policy 0, policy_version 1872236 (0.0010) [2023-12-27 18:37:04,449][20347] Updated weights for policy 0, policy_version 1872246 (0.0008) [2023-12-27 18:37:04,557][20381] Updated weights for policy 1, policy_version 1870654 (0.0007) [2023-12-27 18:37:04,625][20381] Updated weights for policy 1, policy_version 1870664 (0.0008) [2023-12-27 18:37:04,690][20381] Updated weights for policy 1, policy_version 1870674 (0.0008) [2023-12-27 18:37:05,131][20347] Updated weights for policy 0, policy_version 1872256 (0.0006) [2023-12-27 18:37:05,179][20347] Updated weights for policy 0, policy_version 1872266 (0.0008) [2023-12-27 18:37:05,226][20347] Updated weights for policy 0, policy_version 1872276 (0.0007) [2023-12-27 18:37:05,281][20381] Updated weights for policy 1, policy_version 1870684 (0.0007) [2023-12-27 18:37:05,350][20381] Updated weights for policy 1, policy_version 1870694 (0.0005) [2023-12-27 18:37:05,411][20381] Updated weights for policy 1, policy_version 1870704 (0.0005) [2023-12-27 18:37:05,915][20381] Updated weights for policy 1, policy_version 1870714 (0.0007) [2023-12-27 18:37:05,940][20347] Updated weights for policy 0, policy_version 1872286 (0.0006) [2023-12-27 18:37:05,971][20381] Updated weights for policy 1, policy_version 1870724 (0.0011) [2023-12-27 18:37:05,996][20347] Updated weights for policy 0, policy_version 1872296 (0.0006) [2023-12-27 18:37:06,027][20381] Updated weights for policy 1, policy_version 1870734 (0.0011) [2023-12-27 18:37:06,047][20347] Updated weights for policy 0, policy_version 1872306 (0.0005) [2023-12-27 18:37:06,088][20381] Updated weights for policy 1, policy_version 1870744 (0.0010) [2023-12-27 18:37:06,774][20347] Updated weights for policy 0, policy_version 1872316 (0.0007) [2023-12-27 18:37:06,804][20381] Updated weights for policy 1, policy_version 1870754 (0.0010) [2023-12-27 18:37:06,837][20347] Updated weights for policy 0, policy_version 1872326 (0.0007) [2023-12-27 18:37:06,869][20381] Updated weights for policy 1, policy_version 1870764 (0.0011) [2023-12-27 18:37:06,897][20347] Updated weights for policy 0, policy_version 1872336 (0.0009) [2023-12-27 18:37:06,931][20381] Updated weights for policy 1, policy_version 1870774 (0.0011) [2023-12-27 18:37:07,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 958373888. Throughput: 0: 10298.4, 1: 10355.2. Samples: 958344444. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '800.420')] [2023-12-27 18:37:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001870776_478969856.pth... [2023-12-27 18:37:07,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001872344_479404032.pth... [2023-12-27 18:37:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001871160_479100928.pth [2023-12-27 18:37:07,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001869560_478658560.pth [2023-12-27 18:37:07,549][20347] Updated weights for policy 0, policy_version 1872346 (0.0008) [2023-12-27 18:37:07,618][20347] Updated weights for policy 0, policy_version 1872356 (0.0005) [2023-12-27 18:37:07,636][20381] Updated weights for policy 1, policy_version 1870784 (0.0008) [2023-12-27 18:37:07,688][20347] Updated weights for policy 0, policy_version 1872366 (0.0005) [2023-12-27 18:37:07,691][20381] Updated weights for policy 1, policy_version 1870794 (0.0007) [2023-12-27 18:37:07,742][20347] Updated weights for policy 0, policy_version 1872376 (0.0006) [2023-12-27 18:37:07,744][20381] Updated weights for policy 1, policy_version 1870804 (0.0006) [2023-12-27 18:37:08,413][20347] Updated weights for policy 0, policy_version 1872386 (0.0008) [2023-12-27 18:37:08,463][20381] Updated weights for policy 1, policy_version 1870814 (0.0007) [2023-12-27 18:37:08,473][20347] Updated weights for policy 0, policy_version 1872396 (0.0006) [2023-12-27 18:37:08,524][20381] Updated weights for policy 1, policy_version 1870824 (0.0009) [2023-12-27 18:37:08,538][20347] Updated weights for policy 0, policy_version 1872406 (0.0006) [2023-12-27 18:37:08,582][20381] Updated weights for policy 1, policy_version 1870834 (0.0008) [2023-12-27 18:37:09,157][20347] Updated weights for policy 0, policy_version 1872416 (0.0008) [2023-12-27 18:37:09,203][20347] Updated weights for policy 0, policy_version 1872426 (0.0007) [2023-12-27 18:37:09,253][20381] Updated weights for policy 1, policy_version 1870844 (0.0009) [2023-12-27 18:37:09,269][20347] Updated weights for policy 0, policy_version 1872436 (0.0007) [2023-12-27 18:37:09,311][20381] Updated weights for policy 1, policy_version 1870854 (0.0008) [2023-12-27 18:37:09,378][20381] Updated weights for policy 1, policy_version 1870864 (0.0013) [2023-12-27 18:37:10,027][20347] Updated weights for policy 0, policy_version 1872446 (0.0006) [2023-12-27 18:37:10,091][20381] Updated weights for policy 1, policy_version 1870874 (0.0009) [2023-12-27 18:37:10,094][20347] Updated weights for policy 0, policy_version 1872456 (0.0006) [2023-12-27 18:37:10,146][20381] Updated weights for policy 1, policy_version 1870884 (0.0006) [2023-12-27 18:37:10,161][20347] Updated weights for policy 0, policy_version 1872466 (0.0006) [2023-12-27 18:37:10,198][20381] Updated weights for policy 1, policy_version 1870894 (0.0007) [2023-12-27 18:37:10,255][20381] Updated weights for policy 1, policy_version 1870904 (0.0009) [2023-12-27 18:37:10,676][20347] Updated weights for policy 0, policy_version 1872476 (0.0008) [2023-12-27 18:37:10,736][20347] Updated weights for policy 0, policy_version 1872486 (0.0010) [2023-12-27 18:37:10,786][20347] Updated weights for policy 0, policy_version 1872496 (0.0010) [2023-12-27 18:37:10,928][20381] Updated weights for policy 1, policy_version 1870914 (0.0009) [2023-12-27 18:37:10,976][20381] Updated weights for policy 1, policy_version 1870924 (0.0009) [2023-12-27 18:37:11,031][20381] Updated weights for policy 1, policy_version 1870934 (0.0007) [2023-12-27 18:37:11,508][20347] Updated weights for policy 0, policy_version 1872506 (0.0010) [2023-12-27 18:37:11,570][20347] Updated weights for policy 0, policy_version 1872516 (0.0006) [2023-12-27 18:37:11,632][20347] Updated weights for policy 0, policy_version 1872526 (0.0009) [2023-12-27 18:37:11,691][20347] Updated weights for policy 0, policy_version 1872536 (0.0008) [2023-12-27 18:37:11,807][20381] Updated weights for policy 1, policy_version 1870944 (0.0008) [2023-12-27 18:37:11,868][20381] Updated weights for policy 1, policy_version 1870954 (0.0007) [2023-12-27 18:37:11,927][20381] Updated weights for policy 1, policy_version 1870964 (0.0006) [2023-12-27 18:37:12,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 958472192. Throughput: 0: 10301.5, 1: 10293.8. Samples: 958467092. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '817.690')] [2023-12-27 18:37:12,434][20347] Updated weights for policy 0, policy_version 1872546 (0.0007) [2023-12-27 18:37:12,493][20347] Updated weights for policy 0, policy_version 1872556 (0.0010) [2023-12-27 18:37:12,548][20347] Updated weights for policy 0, policy_version 1872566 (0.0010) [2023-12-27 18:37:12,633][20381] Updated weights for policy 1, policy_version 1870974 (0.0007) [2023-12-27 18:37:12,694][20381] Updated weights for policy 1, policy_version 1870984 (0.0005) [2023-12-27 18:37:12,757][20381] Updated weights for policy 1, policy_version 1870994 (0.0006) [2023-12-27 18:37:13,189][20347] Updated weights for policy 0, policy_version 1872576 (0.0006) [2023-12-27 18:37:13,243][20347] Updated weights for policy 0, policy_version 1872586 (0.0005) [2023-12-27 18:37:13,295][20347] Updated weights for policy 0, policy_version 1872596 (0.0005) [2023-12-27 18:37:13,346][20381] Updated weights for policy 1, policy_version 1871004 (0.0006) [2023-12-27 18:37:13,407][20381] Updated weights for policy 1, policy_version 1871014 (0.0008) [2023-12-27 18:37:13,462][20381] Updated weights for policy 1, policy_version 1871024 (0.0010) [2023-12-27 18:37:13,873][20347] Updated weights for policy 0, policy_version 1872606 (0.0007) [2023-12-27 18:37:13,921][20347] Updated weights for policy 0, policy_version 1872616 (0.0005) [2023-12-27 18:37:13,978][20347] Updated weights for policy 0, policy_version 1872626 (0.0007) [2023-12-27 18:37:14,128][20381] Updated weights for policy 1, policy_version 1871034 (0.0010) [2023-12-27 18:37:14,180][20381] Updated weights for policy 1, policy_version 1871044 (0.0009) [2023-12-27 18:37:14,235][20381] Updated weights for policy 1, policy_version 1871054 (0.0006) [2023-12-27 18:37:14,241][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000004 [2023-12-27 18:37:14,652][20347] Updated weights for policy 0, policy_version 1872636 (0.0007) [2023-12-27 18:37:14,711][20347] Updated weights for policy 0, policy_version 1872646 (0.0008) [2023-12-27 18:37:14,773][20347] Updated weights for policy 0, policy_version 1872656 (0.0006) [2023-12-27 18:37:14,955][20381] Updated weights for policy 1, policy_version 1871064 (0.0008) [2023-12-27 18:37:15,012][20381] Updated weights for policy 1, policy_version 1871074 (0.0008) [2023-12-27 18:37:15,069][20381] Updated weights for policy 1, policy_version 1871084 (0.0007) [2023-12-27 18:37:15,478][20347] Updated weights for policy 0, policy_version 1872666 (0.0006) [2023-12-27 18:37:15,529][20347] Updated weights for policy 0, policy_version 1872676 (0.0005) [2023-12-27 18:37:15,574][20347] Updated weights for policy 0, policy_version 1872686 (0.0006) [2023-12-27 18:37:15,623][20347] Updated weights for policy 0, policy_version 1872696 (0.0007) [2023-12-27 18:37:15,712][20381] Updated weights for policy 1, policy_version 1871094 (0.0006) [2023-12-27 18:37:15,768][20381] Updated weights for policy 1, policy_version 1871104 (0.0005) [2023-12-27 18:37:15,831][20381] Updated weights for policy 1, policy_version 1871114 (0.0006) [2023-12-27 18:37:16,262][20347] Updated weights for policy 0, policy_version 1872706 (0.0007) [2023-12-27 18:37:16,320][20347] Updated weights for policy 0, policy_version 1872716 (0.0008) [2023-12-27 18:37:16,382][20347] Updated weights for policy 0, policy_version 1872726 (0.0007) [2023-12-27 18:37:16,402][20381] Updated weights for policy 1, policy_version 1871124 (0.0008) [2023-12-27 18:37:16,460][20381] Updated weights for policy 1, policy_version 1871134 (0.0008) [2023-12-27 18:37:16,509][20381] Updated weights for policy 1, policy_version 1871144 (0.0010) [2023-12-27 18:37:17,079][20347] Updated weights for policy 0, policy_version 1872736 (0.0008) [2023-12-27 18:37:17,134][20347] Updated weights for policy 0, policy_version 1872746 (0.0008) [2023-12-27 18:37:17,190][20347] Updated weights for policy 0, policy_version 1872756 (0.0008) [2023-12-27 18:37:17,236][20381] Updated weights for policy 1, policy_version 1871154 (0.0010) [2023-12-27 18:37:17,290][20381] Updated weights for policy 1, policy_version 1871164 (0.0008) [2023-12-27 18:37:17,336][20381] Updated weights for policy 1, policy_version 1871174 (0.0008) [2023-12-27 18:37:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 958578688. Throughput: 0: 10303.7, 1: 10317.6. Samples: 958591820. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '857.720')] [2023-12-27 18:37:17,390][20381] Updated weights for policy 1, policy_version 1871184 (0.0010) [2023-12-27 18:37:17,808][20347] Updated weights for policy 0, policy_version 1872766 (0.0006) [2023-12-27 18:37:17,855][20347] Updated weights for policy 0, policy_version 1872776 (0.0005) [2023-12-27 18:37:17,905][20347] Updated weights for policy 0, policy_version 1872786 (0.0005) [2023-12-27 18:37:18,143][20381] Updated weights for policy 1, policy_version 1871194 (0.0007) [2023-12-27 18:37:18,189][20381] Updated weights for policy 1, policy_version 1871204 (0.0005) [2023-12-27 18:37:18,235][20381] Updated weights for policy 1, policy_version 1871214 (0.0005) [2023-12-27 18:37:18,522][20347] Updated weights for policy 0, policy_version 1872796 (0.0007) [2023-12-27 18:37:18,580][20347] Updated weights for policy 0, policy_version 1872806 (0.0010) [2023-12-27 18:37:18,628][20347] Updated weights for policy 0, policy_version 1872816 (0.0006) [2023-12-27 18:37:18,889][20381] Updated weights for policy 1, policy_version 1871224 (0.0007) [2023-12-27 18:37:18,937][20381] Updated weights for policy 1, policy_version 1871234 (0.0005) [2023-12-27 18:37:18,988][20381] Updated weights for policy 1, policy_version 1871244 (0.0005) [2023-12-27 18:37:19,291][20347] Updated weights for policy 0, policy_version 1872826 (0.0008) [2023-12-27 18:37:19,349][20347] Updated weights for policy 0, policy_version 1872836 (0.0007) [2023-12-27 18:37:19,422][20347] Updated weights for policy 0, policy_version 1872846 (0.0009) [2023-12-27 18:37:19,482][20347] Updated weights for policy 0, policy_version 1872856 (0.0008) [2023-12-27 18:37:19,638][20381] Updated weights for policy 1, policy_version 1871254 (0.0008) [2023-12-27 18:37:19,697][20381] Updated weights for policy 1, policy_version 1871264 (0.0009) [2023-12-27 18:37:19,755][20381] Updated weights for policy 1, policy_version 1871274 (0.0009) [2023-12-27 18:37:20,229][20347] Updated weights for policy 0, policy_version 1872866 (0.0006) [2023-12-27 18:37:20,280][20347] Updated weights for policy 0, policy_version 1872876 (0.0006) [2023-12-27 18:37:20,328][20347] Updated weights for policy 0, policy_version 1872886 (0.0005) [2023-12-27 18:37:20,569][20381] Updated weights for policy 1, policy_version 1871284 (0.0008) [2023-12-27 18:37:20,632][20381] Updated weights for policy 1, policy_version 1871294 (0.0008) [2023-12-27 18:37:20,683][20381] Updated weights for policy 1, policy_version 1871304 (0.0008) [2023-12-27 18:37:21,014][20347] Updated weights for policy 0, policy_version 1872896 (0.0010) [2023-12-27 18:37:21,085][20347] Updated weights for policy 0, policy_version 1872906 (0.0011) [2023-12-27 18:37:21,141][20347] Updated weights for policy 0, policy_version 1872916 (0.0009) [2023-12-27 18:37:21,427][20381] Updated weights for policy 1, policy_version 1871314 (0.0008) [2023-12-27 18:37:21,479][20381] Updated weights for policy 1, policy_version 1871324 (0.0008) [2023-12-27 18:37:21,539][20381] Updated weights for policy 1, policy_version 1871334 (0.0008) [2023-12-27 18:37:21,610][20381] Updated weights for policy 1, policy_version 1871344 (0.0006) [2023-12-27 18:37:21,902][20347] Updated weights for policy 0, policy_version 1872926 (0.0008) [2023-12-27 18:37:21,966][20347] Updated weights for policy 0, policy_version 1872936 (0.0008) [2023-12-27 18:37:22,028][20347] Updated weights for policy 0, policy_version 1872946 (0.0008) [2023-12-27 18:37:22,344][20381] Updated weights for policy 1, policy_version 1871354 (0.0008) [2023-12-27 18:37:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 958676992. Throughput: 0: 10281.8, 1: 10271.1. Samples: 958652200. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '848.850')] [2023-12-27 18:37:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001872952_479559680.pth... [2023-12-27 18:37:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001871736_479248384.pth [2023-12-27 18:37:22,409][20381] Updated weights for policy 1, policy_version 1871364 (0.0009) [2023-12-27 18:37:22,470][20381] Updated weights for policy 1, policy_version 1871374 (0.0008) [2023-12-27 18:37:22,480][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001871376_479125504.pth... [2023-12-27 18:37:22,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001870168_478814208.pth [2023-12-27 18:37:22,721][20347] Updated weights for policy 0, policy_version 1872956 (0.0008) [2023-12-27 18:37:22,785][20347] Updated weights for policy 0, policy_version 1872966 (0.0008) [2023-12-27 18:37:22,841][20347] Updated weights for policy 0, policy_version 1872976 (0.0008) [2023-12-27 18:37:23,196][20381] Updated weights for policy 1, policy_version 1871384 (0.0007) [2023-12-27 18:37:23,253][20381] Updated weights for policy 1, policy_version 1871394 (0.0007) [2023-12-27 18:37:23,306][20381] Updated weights for policy 1, policy_version 1871404 (0.0008) [2023-12-27 18:37:23,546][20347] Updated weights for policy 0, policy_version 1872986 (0.0007) [2023-12-27 18:37:23,598][20347] Updated weights for policy 0, policy_version 1872996 (0.0006) [2023-12-27 18:37:23,649][20347] Updated weights for policy 0, policy_version 1873006 (0.0005) [2023-12-27 18:37:23,708][20347] Updated weights for policy 0, policy_version 1873016 (0.0009) [2023-12-27 18:37:24,024][20381] Updated weights for policy 1, policy_version 1871414 (0.0006) [2023-12-27 18:37:24,070][20381] Updated weights for policy 1, policy_version 1871424 (0.0005) [2023-12-27 18:37:24,119][20381] Updated weights for policy 1, policy_version 1871434 (0.0005) [2023-12-27 18:37:24,323][20347] Updated weights for policy 0, policy_version 1873026 (0.0010) [2023-12-27 18:37:24,381][20347] Updated weights for policy 0, policy_version 1873036 (0.0010) [2023-12-27 18:37:24,433][20347] Updated weights for policy 0, policy_version 1873046 (0.0010) [2023-12-27 18:37:24,727][20381] Updated weights for policy 1, policy_version 1871444 (0.0006) [2023-12-27 18:37:24,785][20381] Updated weights for policy 1, policy_version 1871454 (0.0007) [2023-12-27 18:37:24,846][20381] Updated weights for policy 1, policy_version 1871464 (0.0007) [2023-12-27 18:37:25,078][20347] Updated weights for policy 0, policy_version 1873056 (0.0007) [2023-12-27 18:37:25,129][20347] Updated weights for policy 0, policy_version 1873066 (0.0007) [2023-12-27 18:37:25,173][20347] Updated weights for policy 0, policy_version 1873076 (0.0007) [2023-12-27 18:37:25,565][20381] Updated weights for policy 1, policy_version 1871474 (0.0007) [2023-12-27 18:37:25,625][20381] Updated weights for policy 1, policy_version 1871484 (0.0007) [2023-12-27 18:37:25,677][20381] Updated weights for policy 1, policy_version 1871494 (0.0008) [2023-12-27 18:37:25,733][20381] Updated weights for policy 1, policy_version 1871504 (0.0007) [2023-12-27 18:37:25,765][20347] Updated weights for policy 0, policy_version 1873086 (0.0006) [2023-12-27 18:37:25,826][20347] Updated weights for policy 0, policy_version 1873096 (0.0005) [2023-12-27 18:37:25,884][20347] Updated weights for policy 0, policy_version 1873106 (0.0007) [2023-12-27 18:37:26,432][20381] Updated weights for policy 1, policy_version 1871514 (0.0007) [2023-12-27 18:37:26,471][20347] Updated weights for policy 0, policy_version 1873116 (0.0007) [2023-12-27 18:37:26,490][20381] Updated weights for policy 1, policy_version 1871524 (0.0005) [2023-12-27 18:37:26,520][20347] Updated weights for policy 0, policy_version 1873126 (0.0005) [2023-12-27 18:37:26,553][20381] Updated weights for policy 1, policy_version 1871534 (0.0008) [2023-12-27 18:37:26,565][20347] Updated weights for policy 0, policy_version 1873136 (0.0006) [2023-12-27 18:37:27,164][20381] Updated weights for policy 1, policy_version 1871544 (0.0005) [2023-12-27 18:37:27,207][20381] Updated weights for policy 1, policy_version 1871554 (0.0005) [2023-12-27 18:37:27,250][20381] Updated weights for policy 1, policy_version 1871564 (0.0007) [2023-12-27 18:37:27,256][20347] Updated weights for policy 0, policy_version 1873146 (0.0006) [2023-12-27 18:37:27,305][20347] Updated weights for policy 0, policy_version 1873156 (0.0007) [2023-12-27 18:37:27,360][20347] Updated weights for policy 0, policy_version 1873166 (0.0007) [2023-12-27 18:37:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 958783488. Throughput: 0: 10249.3, 1: 10268.2. Samples: 958773164. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '849.640')] [2023-12-27 18:37:27,420][20347] Updated weights for policy 0, policy_version 1873176 (0.0008) [2023-12-27 18:37:27,808][20381] Updated weights for policy 1, policy_version 1871574 (0.0008) [2023-12-27 18:37:27,866][20381] Updated weights for policy 1, policy_version 1871584 (0.0010) [2023-12-27 18:37:27,921][20381] Updated weights for policy 1, policy_version 1871594 (0.0010) [2023-12-27 18:37:28,131][20347] Updated weights for policy 0, policy_version 1873186 (0.0005) [2023-12-27 18:37:28,193][20347] Updated weights for policy 0, policy_version 1873196 (0.0005) [2023-12-27 18:37:28,252][20347] Updated weights for policy 0, policy_version 1873206 (0.0006) [2023-12-27 18:37:28,567][20381] Updated weights for policy 1, policy_version 1871604 (0.0008) [2023-12-27 18:37:28,628][20381] Updated weights for policy 1, policy_version 1871614 (0.0010) [2023-12-27 18:37:28,689][20381] Updated weights for policy 1, policy_version 1871624 (0.0011) [2023-12-27 18:37:28,875][20347] Updated weights for policy 0, policy_version 1873216 (0.0007) [2023-12-27 18:37:28,929][20347] Updated weights for policy 0, policy_version 1873226 (0.0008) [2023-12-27 18:37:28,978][20347] Updated weights for policy 0, policy_version 1873236 (0.0008) [2023-12-27 18:37:29,355][20381] Updated weights for policy 1, policy_version 1871634 (0.0011) [2023-12-27 18:37:29,415][20381] Updated weights for policy 1, policy_version 1871644 (0.0008) [2023-12-27 18:37:29,471][20381] Updated weights for policy 1, policy_version 1871654 (0.0008) [2023-12-27 18:37:29,529][20381] Updated weights for policy 1, policy_version 1871664 (0.0006) [2023-12-27 18:37:29,697][20347] Updated weights for policy 0, policy_version 1873246 (0.0007) [2023-12-27 18:37:29,758][20347] Updated weights for policy 0, policy_version 1873256 (0.0008) [2023-12-27 18:37:29,804][20347] Updated weights for policy 0, policy_version 1873266 (0.0010) [2023-12-27 18:37:30,183][20381] Updated weights for policy 1, policy_version 1871674 (0.0008) [2023-12-27 18:37:30,248][20381] Updated weights for policy 1, policy_version 1871684 (0.0006) [2023-12-27 18:37:30,314][20381] Updated weights for policy 1, policy_version 1871694 (0.0005) [2023-12-27 18:37:30,578][20347] Updated weights for policy 0, policy_version 1873276 (0.0008) [2023-12-27 18:37:30,636][20347] Updated weights for policy 0, policy_version 1873286 (0.0008) [2023-12-27 18:37:30,693][20347] Updated weights for policy 0, policy_version 1873296 (0.0007) [2023-12-27 18:37:30,954][20381] Updated weights for policy 1, policy_version 1871704 (0.0007) [2023-12-27 18:37:31,008][20381] Updated weights for policy 1, policy_version 1871714 (0.0007) [2023-12-27 18:37:31,069][20381] Updated weights for policy 1, policy_version 1871724 (0.0008) [2023-12-27 18:37:31,355][20347] Updated weights for policy 0, policy_version 1873306 (0.0008) [2023-12-27 18:37:31,422][20347] Updated weights for policy 0, policy_version 1873316 (0.0008) [2023-12-27 18:37:31,479][20347] Updated weights for policy 0, policy_version 1873326 (0.0008) [2023-12-27 18:37:31,544][20347] Updated weights for policy 0, policy_version 1873336 (0.0008) [2023-12-27 18:37:31,759][20381] Updated weights for policy 1, policy_version 1871734 (0.0007) [2023-12-27 18:37:31,815][20381] Updated weights for policy 1, policy_version 1871744 (0.0007) [2023-12-27 18:37:31,860][20381] Updated weights for policy 1, policy_version 1871754 (0.0008) [2023-12-27 18:37:32,237][20347] Updated weights for policy 0, policy_version 1873346 (0.0011) [2023-12-27 18:37:32,299][20347] Updated weights for policy 0, policy_version 1873356 (0.0010) [2023-12-27 18:37:32,359][20347] Updated weights for policy 0, policy_version 1873366 (0.0010) [2023-12-27 18:37:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 958881792. Throughput: 0: 10267.1, 1: 10279.8. Samples: 958897312. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '860.470')] [2023-12-27 18:37:32,551][20381] Updated weights for policy 1, policy_version 1871764 (0.0007) [2023-12-27 18:37:32,610][20381] Updated weights for policy 1, policy_version 1871774 (0.0006) [2023-12-27 18:37:32,659][20381] Updated weights for policy 1, policy_version 1871784 (0.0007) [2023-12-27 18:37:33,027][20347] Updated weights for policy 0, policy_version 1873376 (0.0010) [2023-12-27 18:37:33,075][20347] Updated weights for policy 0, policy_version 1873386 (0.0010) [2023-12-27 18:37:33,135][20347] Updated weights for policy 0, policy_version 1873396 (0.0007) [2023-12-27 18:37:33,235][20381] Updated weights for policy 1, policy_version 1871794 (0.0009) [2023-12-27 18:37:33,287][20381] Updated weights for policy 1, policy_version 1871804 (0.0009) [2023-12-27 18:37:33,338][20381] Updated weights for policy 1, policy_version 1871814 (0.0008) [2023-12-27 18:37:33,386][20381] Updated weights for policy 1, policy_version 1871824 (0.0007) [2023-12-27 18:37:33,698][20347] Updated weights for policy 0, policy_version 1873406 (0.0006) [2023-12-27 18:37:33,748][20347] Updated weights for policy 0, policy_version 1873416 (0.0005) [2023-12-27 18:37:33,791][20347] Updated weights for policy 0, policy_version 1873426 (0.0005) [2023-12-27 18:37:34,140][20381] Updated weights for policy 1, policy_version 1871834 (0.0006) [2023-12-27 18:37:34,201][20381] Updated weights for policy 1, policy_version 1871844 (0.0008) [2023-12-27 18:37:34,254][20381] Updated weights for policy 1, policy_version 1871854 (0.0008) [2023-12-27 18:37:34,424][20347] Updated weights for policy 0, policy_version 1873436 (0.0007) [2023-12-27 18:37:34,488][20347] Updated weights for policy 0, policy_version 1873446 (0.0010) [2023-12-27 18:37:34,547][20347] Updated weights for policy 0, policy_version 1873456 (0.0009) [2023-12-27 18:37:35,001][20381] Updated weights for policy 1, policy_version 1871864 (0.0006) [2023-12-27 18:37:35,060][20381] Updated weights for policy 1, policy_version 1871874 (0.0005) [2023-12-27 18:37:35,116][20381] Updated weights for policy 1, policy_version 1871884 (0.0005) [2023-12-27 18:37:35,128][20347] Updated weights for policy 0, policy_version 1873466 (0.0006) [2023-12-27 18:37:35,172][20347] Updated weights for policy 0, policy_version 1873476 (0.0005) [2023-12-27 18:37:35,220][20347] Updated weights for policy 0, policy_version 1873486 (0.0009) [2023-12-27 18:37:35,273][20347] Updated weights for policy 0, policy_version 1873496 (0.0007) [2023-12-27 18:37:35,733][20381] Updated weights for policy 1, policy_version 1871894 (0.0011) [2023-12-27 18:37:35,784][20381] Updated weights for policy 1, policy_version 1871904 (0.0010) [2023-12-27 18:37:35,832][20381] Updated weights for policy 1, policy_version 1871914 (0.0010) [2023-12-27 18:37:35,910][20347] Updated weights for policy 0, policy_version 1873506 (0.0009) [2023-12-27 18:37:35,967][20347] Updated weights for policy 0, policy_version 1873516 (0.0010) [2023-12-27 18:37:36,028][20347] Updated weights for policy 0, policy_version 1873526 (0.0010) [2023-12-27 18:37:36,036][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000002 [2023-12-27 18:37:36,471][20381] Updated weights for policy 1, policy_version 1871924 (0.0009) [2023-12-27 18:37:36,530][20381] Updated weights for policy 1, policy_version 1871934 (0.0006) [2023-12-27 18:37:36,587][20381] Updated weights for policy 1, policy_version 1871944 (0.0007) [2023-12-27 18:37:36,719][20347] Updated weights for policy 0, policy_version 1873536 (0.0009) [2023-12-27 18:37:36,787][20347] Updated weights for policy 0, policy_version 1873546 (0.0008) [2023-12-27 18:37:36,855][20347] Updated weights for policy 0, policy_version 1873556 (0.0008) [2023-12-27 18:37:37,177][20381] Updated weights for policy 1, policy_version 1871954 (0.0007) [2023-12-27 18:37:37,235][20381] Updated weights for policy 1, policy_version 1871964 (0.0006) [2023-12-27 18:37:37,292][20381] Updated weights for policy 1, policy_version 1871974 (0.0006) [2023-12-27 18:37:37,352][20381] Updated weights for policy 1, policy_version 1871984 (0.0006) [2023-12-27 18:37:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 958996480. Throughput: 0: 10327.9, 1: 10316.3. Samples: 958961596. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.430')] [2023-12-27 18:37:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001871984_479281152.pth... [2023-12-27 18:37:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001873560_479715328.pth... [2023-12-27 18:37:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001872344_479404032.pth [2023-12-27 18:37:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001870776_478969856.pth [2023-12-27 18:37:37,524][20347] Updated weights for policy 0, policy_version 1873566 (0.0009) [2023-12-27 18:37:37,576][20347] Updated weights for policy 0, policy_version 1873576 (0.0010) [2023-12-27 18:37:37,626][20347] Updated weights for policy 0, policy_version 1873586 (0.0010) [2023-12-27 18:37:37,943][20381] Updated weights for policy 1, policy_version 1871994 (0.0010) [2023-12-27 18:37:38,001][20381] Updated weights for policy 1, policy_version 1872004 (0.0006) [2023-12-27 18:37:38,055][20381] Updated weights for policy 1, policy_version 1872014 (0.0005) [2023-12-27 18:37:38,269][20347] Updated weights for policy 0, policy_version 1873596 (0.0008) [2023-12-27 18:37:38,325][20347] Updated weights for policy 0, policy_version 1873606 (0.0006) [2023-12-27 18:37:38,383][20347] Updated weights for policy 0, policy_version 1873616 (0.0007) [2023-12-27 18:37:38,659][20381] Updated weights for policy 1, policy_version 1872024 (0.0005) [2023-12-27 18:37:38,714][20381] Updated weights for policy 1, policy_version 1872034 (0.0006) [2023-12-27 18:37:38,783][20381] Updated weights for policy 1, policy_version 1872044 (0.0008) [2023-12-27 18:37:39,059][20347] Updated weights for policy 0, policy_version 1873626 (0.0007) [2023-12-27 18:37:39,113][20347] Updated weights for policy 0, policy_version 1873636 (0.0007) [2023-12-27 18:37:39,158][20347] Updated weights for policy 0, policy_version 1873646 (0.0008) [2023-12-27 18:37:39,215][20347] Updated weights for policy 0, policy_version 1873656 (0.0008) [2023-12-27 18:37:39,368][20381] Updated weights for policy 1, policy_version 1872054 (0.0007) [2023-12-27 18:37:39,438][20381] Updated weights for policy 1, policy_version 1872064 (0.0008) [2023-12-27 18:37:39,497][20381] Updated weights for policy 1, policy_version 1872074 (0.0008) [2023-12-27 18:37:39,925][20347] Updated weights for policy 0, policy_version 1873666 (0.0006) [2023-12-27 18:37:39,982][20347] Updated weights for policy 0, policy_version 1873676 (0.0007) [2023-12-27 18:37:40,042][20347] Updated weights for policy 0, policy_version 1873686 (0.0007) [2023-12-27 18:37:40,228][20381] Updated weights for policy 1, policy_version 1872084 (0.0007) [2023-12-27 18:37:40,284][20381] Updated weights for policy 1, policy_version 1872094 (0.0007) [2023-12-27 18:37:40,341][20381] Updated weights for policy 1, policy_version 1872104 (0.0008) [2023-12-27 18:37:40,698][20347] Updated weights for policy 0, policy_version 1873696 (0.0006) [2023-12-27 18:37:40,759][20347] Updated weights for policy 0, policy_version 1873706 (0.0007) [2023-12-27 18:37:40,817][20347] Updated weights for policy 0, policy_version 1873716 (0.0008) [2023-12-27 18:37:41,074][20381] Updated weights for policy 1, policy_version 1872114 (0.0008) [2023-12-27 18:37:41,136][20381] Updated weights for policy 1, policy_version 1872124 (0.0007) [2023-12-27 18:37:41,198][20381] Updated weights for policy 1, policy_version 1872134 (0.0008) [2023-12-27 18:37:41,258][20381] Updated weights for policy 1, policy_version 1872144 (0.0008) [2023-12-27 18:37:41,524][20347] Updated weights for policy 0, policy_version 1873726 (0.0007) [2023-12-27 18:37:41,596][20347] Updated weights for policy 0, policy_version 1873736 (0.0008) [2023-12-27 18:37:41,664][20347] Updated weights for policy 0, policy_version 1873746 (0.0007) [2023-12-27 18:37:42,003][20381] Updated weights for policy 1, policy_version 1872154 (0.0008) [2023-12-27 18:37:42,054][20381] Updated weights for policy 1, policy_version 1872164 (0.0007) [2023-12-27 18:37:42,112][20381] Updated weights for policy 1, policy_version 1872174 (0.0008) [2023-12-27 18:37:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 959094784. Throughput: 0: 10387.9, 1: 10290.6. Samples: 959087620. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '877.470')] [2023-12-27 18:37:42,363][20347] Updated weights for policy 0, policy_version 1873756 (0.0009) [2023-12-27 18:37:42,425][20347] Updated weights for policy 0, policy_version 1873766 (0.0009) [2023-12-27 18:37:42,473][20347] Updated weights for policy 0, policy_version 1873776 (0.0009) [2023-12-27 18:37:42,789][20381] Updated weights for policy 1, policy_version 1872184 (0.0009) [2023-12-27 18:37:42,837][20381] Updated weights for policy 1, policy_version 1872194 (0.0008) [2023-12-27 18:37:42,886][20381] Updated weights for policy 1, policy_version 1872204 (0.0009) [2023-12-27 18:37:43,095][20347] Updated weights for policy 0, policy_version 1873786 (0.0007) [2023-12-27 18:37:43,146][20347] Updated weights for policy 0, policy_version 1873796 (0.0005) [2023-12-27 18:37:43,194][20347] Updated weights for policy 0, policy_version 1873806 (0.0007) [2023-12-27 18:37:43,249][20347] Updated weights for policy 0, policy_version 1873816 (0.0008) [2023-12-27 18:37:43,580][20381] Updated weights for policy 1, policy_version 1872214 (0.0007) [2023-12-27 18:37:43,646][20381] Updated weights for policy 1, policy_version 1872224 (0.0008) [2023-12-27 18:37:43,714][20381] Updated weights for policy 1, policy_version 1872234 (0.0008) [2023-12-27 18:37:43,865][20347] Updated weights for policy 0, policy_version 1873826 (0.0007) [2023-12-27 18:37:43,921][20347] Updated weights for policy 0, policy_version 1873836 (0.0008) [2023-12-27 18:37:43,974][20347] Updated weights for policy 0, policy_version 1873846 (0.0006) [2023-12-27 18:37:44,421][20381] Updated weights for policy 1, policy_version 1872244 (0.0007) [2023-12-27 18:37:44,478][20381] Updated weights for policy 1, policy_version 1872254 (0.0007) [2023-12-27 18:37:44,538][20381] Updated weights for policy 1, policy_version 1872264 (0.0008) [2023-12-27 18:37:44,573][20347] Updated weights for policy 0, policy_version 1873856 (0.0009) [2023-12-27 18:37:44,633][20347] Updated weights for policy 0, policy_version 1873866 (0.0006) [2023-12-27 18:37:44,693][20347] Updated weights for policy 0, policy_version 1873876 (0.0006) [2023-12-27 18:37:45,233][20381] Updated weights for policy 1, policy_version 1872274 (0.0008) [2023-12-27 18:37:45,287][20381] Updated weights for policy 1, policy_version 1872284 (0.0008) [2023-12-27 18:37:45,335][20381] Updated weights for policy 1, policy_version 1872294 (0.0006) [2023-12-27 18:37:45,350][20347] Updated weights for policy 0, policy_version 1873886 (0.0007) [2023-12-27 18:37:45,389][20381] Updated weights for policy 1, policy_version 1872304 (0.0006) [2023-12-27 18:37:45,408][20347] Updated weights for policy 0, policy_version 1873896 (0.0007) [2023-12-27 18:37:45,461][20347] Updated weights for policy 0, policy_version 1873906 (0.0008) [2023-12-27 18:37:46,068][20347] Updated weights for policy 0, policy_version 1873916 (0.0007) [2023-12-27 18:37:46,116][20381] Updated weights for policy 1, policy_version 1872314 (0.0006) [2023-12-27 18:37:46,125][20347] Updated weights for policy 0, policy_version 1873926 (0.0007) [2023-12-27 18:37:46,168][20381] Updated weights for policy 1, policy_version 1872324 (0.0005) [2023-12-27 18:37:46,181][20347] Updated weights for policy 0, policy_version 1873936 (0.0006) [2023-12-27 18:37:46,223][20381] Updated weights for policy 1, policy_version 1872334 (0.0006) [2023-12-27 18:37:46,875][20347] Updated weights for policy 0, policy_version 1873946 (0.0006) [2023-12-27 18:37:46,934][20381] Updated weights for policy 1, policy_version 1872344 (0.0008) [2023-12-27 18:37:46,941][20347] Updated weights for policy 0, policy_version 1873956 (0.0007) [2023-12-27 18:37:46,988][20381] Updated weights for policy 1, policy_version 1872354 (0.0007) [2023-12-27 18:37:47,002][20347] Updated weights for policy 0, policy_version 1873966 (0.0006) [2023-12-27 18:37:47,047][20381] Updated weights for policy 1, policy_version 1872364 (0.0008) [2023-12-27 18:37:47,059][20347] Updated weights for policy 0, policy_version 1873976 (0.0006) [2023-12-27 18:37:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 959201280. Throughput: 0: 10408.8, 1: 10259.4. Samples: 959209772. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '919.220')] [2023-12-27 18:37:47,674][20347] Updated weights for policy 0, policy_version 1873986 (0.0008) [2023-12-27 18:37:47,739][20347] Updated weights for policy 0, policy_version 1873996 (0.0009) [2023-12-27 18:37:47,770][20381] Updated weights for policy 1, policy_version 1872374 (0.0007) [2023-12-27 18:37:47,802][20347] Updated weights for policy 0, policy_version 1874006 (0.0008) [2023-12-27 18:37:47,830][20381] Updated weights for policy 1, policy_version 1872384 (0.0007) [2023-12-27 18:37:47,890][20381] Updated weights for policy 1, policy_version 1872394 (0.0006) [2023-12-27 18:37:48,416][20347] Updated weights for policy 0, policy_version 1874016 (0.0010) [2023-12-27 18:37:48,475][20347] Updated weights for policy 0, policy_version 1874026 (0.0010) [2023-12-27 18:37:48,527][20347] Updated weights for policy 0, policy_version 1874036 (0.0010) [2023-12-27 18:37:48,536][20381] Updated weights for policy 1, policy_version 1872404 (0.0006) [2023-12-27 18:37:48,591][20381] Updated weights for policy 1, policy_version 1872414 (0.0007) [2023-12-27 18:37:48,640][20381] Updated weights for policy 1, policy_version 1872424 (0.0008) [2023-12-27 18:37:49,178][20347] Updated weights for policy 0, policy_version 1874046 (0.0007) [2023-12-27 18:37:49,248][20347] Updated weights for policy 0, policy_version 1874057 (0.0010) [2023-12-27 18:37:49,298][20347] Updated weights for policy 0, policy_version 1874067 (0.0008) [2023-12-27 18:37:49,331][20381] Updated weights for policy 1, policy_version 1872434 (0.0008) [2023-12-27 18:37:49,397][20381] Updated weights for policy 1, policy_version 1872444 (0.0007) [2023-12-27 18:37:49,448][20381] Updated weights for policy 1, policy_version 1872454 (0.0006) [2023-12-27 18:37:49,505][20381] Updated weights for policy 1, policy_version 1872464 (0.0007) [2023-12-27 18:37:49,979][20347] Updated weights for policy 0, policy_version 1874077 (0.0008) [2023-12-27 18:37:50,036][20347] Updated weights for policy 0, policy_version 1874087 (0.0010) [2023-12-27 18:37:50,082][20347] Updated weights for policy 0, policy_version 1874097 (0.0010) [2023-12-27 18:37:50,104][20381] Updated weights for policy 1, policy_version 1872474 (0.0006) [2023-12-27 18:37:50,161][20381] Updated weights for policy 1, policy_version 1872484 (0.0007) [2023-12-27 18:37:50,220][20381] Updated weights for policy 1, policy_version 1872494 (0.0007) [2023-12-27 18:37:50,809][20347] Updated weights for policy 0, policy_version 1874107 (0.0011) [2023-12-27 18:37:50,862][20347] Updated weights for policy 0, policy_version 1874117 (0.0010) [2023-12-27 18:37:50,917][20347] Updated weights for policy 0, policy_version 1874127 (0.0010) [2023-12-27 18:37:50,973][20381] Updated weights for policy 1, policy_version 1872504 (0.0008) [2023-12-27 18:37:51,032][20381] Updated weights for policy 1, policy_version 1872514 (0.0008) [2023-12-27 18:37:51,090][20381] Updated weights for policy 1, policy_version 1872524 (0.0008) [2023-12-27 18:37:51,599][20347] Updated weights for policy 0, policy_version 1874137 (0.0010) [2023-12-27 18:37:51,667][20347] Updated weights for policy 0, policy_version 1874147 (0.0009) [2023-12-27 18:37:51,734][20347] Updated weights for policy 0, policy_version 1874157 (0.0008) [2023-12-27 18:37:51,788][20347] Updated weights for policy 0, policy_version 1874167 (0.0007) [2023-12-27 18:37:51,848][20381] Updated weights for policy 1, policy_version 1872534 (0.0009) [2023-12-27 18:37:51,906][20381] Updated weights for policy 1, policy_version 1872544 (0.0008) [2023-12-27 18:37:51,964][20381] Updated weights for policy 1, policy_version 1872554 (0.0007) [2023-12-27 18:37:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 959299584. Throughput: 0: 10382.6, 1: 10245.4. Samples: 959272700. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '917.620')] [2023-12-27 18:37:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001874168_479870976.pth... [2023-12-27 18:37:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001872560_479428608.pth... [2023-12-27 18:37:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001872952_479559680.pth [2023-12-27 18:37:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001871376_479125504.pth [2023-12-27 18:37:52,477][20347] Updated weights for policy 0, policy_version 1874177 (0.0007) [2023-12-27 18:37:52,537][20347] Updated weights for policy 0, policy_version 1874187 (0.0008) [2023-12-27 18:37:52,594][20347] Updated weights for policy 0, policy_version 1874197 (0.0008) [2023-12-27 18:37:52,700][20381] Updated weights for policy 1, policy_version 1872564 (0.0007) [2023-12-27 18:37:52,751][20381] Updated weights for policy 1, policy_version 1872574 (0.0008) [2023-12-27 18:37:52,804][20381] Updated weights for policy 1, policy_version 1872584 (0.0007) [2023-12-27 18:37:53,309][20347] Updated weights for policy 0, policy_version 1874207 (0.0007) [2023-12-27 18:37:53,363][20347] Updated weights for policy 0, policy_version 1874217 (0.0008) [2023-12-27 18:37:53,414][20347] Updated weights for policy 0, policy_version 1874227 (0.0007) [2023-12-27 18:37:53,500][20381] Updated weights for policy 1, policy_version 1872594 (0.0008) [2023-12-27 18:37:53,542][20381] Updated weights for policy 1, policy_version 1872604 (0.0007) [2023-12-27 18:37:53,591][20381] Updated weights for policy 1, policy_version 1872614 (0.0009) [2023-12-27 18:37:53,646][20381] Updated weights for policy 1, policy_version 1872624 (0.0008) [2023-12-27 18:37:54,030][20347] Updated weights for policy 0, policy_version 1874237 (0.0006) [2023-12-27 18:37:54,089][20347] Updated weights for policy 0, policy_version 1874247 (0.0006) [2023-12-27 18:37:54,158][20347] Updated weights for policy 0, policy_version 1874257 (0.0005) [2023-12-27 18:37:54,416][20381] Updated weights for policy 1, policy_version 1872634 (0.0007) [2023-12-27 18:37:54,481][20381] Updated weights for policy 1, policy_version 1872644 (0.0007) [2023-12-27 18:37:54,533][20381] Updated weights for policy 1, policy_version 1872654 (0.0007) [2023-12-27 18:37:54,747][20347] Updated weights for policy 0, policy_version 1874267 (0.0006) [2023-12-27 18:37:54,803][20347] Updated weights for policy 0, policy_version 1874277 (0.0008) [2023-12-27 18:37:54,857][20347] Updated weights for policy 0, policy_version 1874287 (0.0005) [2023-12-27 18:37:55,280][20381] Updated weights for policy 1, policy_version 1872664 (0.0007) [2023-12-27 18:37:55,330][20381] Updated weights for policy 1, policy_version 1872674 (0.0007) [2023-12-27 18:37:55,383][20381] Updated weights for policy 1, policy_version 1872684 (0.0007) [2023-12-27 18:37:55,441][20347] Updated weights for policy 0, policy_version 1874297 (0.0006) [2023-12-27 18:37:55,490][20347] Updated weights for policy 0, policy_version 1874307 (0.0005) [2023-12-27 18:37:55,540][20347] Updated weights for policy 0, policy_version 1874317 (0.0006) [2023-12-27 18:37:55,592][20347] Updated weights for policy 0, policy_version 1874327 (0.0006) [2023-12-27 18:37:56,119][20381] Updated weights for policy 1, policy_version 1872694 (0.0009) [2023-12-27 18:37:56,133][20347] Updated weights for policy 0, policy_version 1874337 (0.0005) [2023-12-27 18:37:56,168][20381] Updated weights for policy 1, policy_version 1872704 (0.0007) [2023-12-27 18:37:56,182][20347] Updated weights for policy 0, policy_version 1874347 (0.0005) [2023-12-27 18:37:56,226][20381] Updated weights for policy 1, policy_version 1872714 (0.0006) [2023-12-27 18:37:56,227][20347] Updated weights for policy 0, policy_version 1874357 (0.0006) [2023-12-27 18:37:56,861][20347] Updated weights for policy 0, policy_version 1874367 (0.0009) [2023-12-27 18:37:56,905][20347] Updated weights for policy 0, policy_version 1874377 (0.0006) [2023-12-27 18:37:56,906][20381] Updated weights for policy 1, policy_version 1872724 (0.0007) [2023-12-27 18:37:56,953][20347] Updated weights for policy 0, policy_version 1874387 (0.0006) [2023-12-27 18:37:56,962][20381] Updated weights for policy 1, policy_version 1872734 (0.0007) [2023-12-27 18:37:57,012][20381] Updated weights for policy 1, policy_version 1872744 (0.0006) [2023-12-27 18:37:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 959406080. Throughput: 0: 10439.9, 1: 10165.0. Samples: 959394312. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:37:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '865.970')] [2023-12-27 18:37:57,582][20347] Updated weights for policy 0, policy_version 1874397 (0.0010) [2023-12-27 18:37:57,643][20347] Updated weights for policy 0, policy_version 1874407 (0.0010) [2023-12-27 18:37:57,686][20381] Updated weights for policy 1, policy_version 1872754 (0.0008) [2023-12-27 18:37:57,700][20347] Updated weights for policy 0, policy_version 1874417 (0.0010) [2023-12-27 18:37:57,742][20381] Updated weights for policy 1, policy_version 1872764 (0.0006) [2023-12-27 18:37:57,797][20381] Updated weights for policy 1, policy_version 1872774 (0.0007) [2023-12-27 18:37:57,851][20381] Updated weights for policy 1, policy_version 1872784 (0.0006) [2023-12-27 18:37:58,306][20347] Updated weights for policy 0, policy_version 1874427 (0.0010) [2023-12-27 18:37:58,370][20347] Updated weights for policy 0, policy_version 1874437 (0.0010) [2023-12-27 18:37:58,430][20347] Updated weights for policy 0, policy_version 1874447 (0.0007) [2023-12-27 18:37:58,567][20381] Updated weights for policy 1, policy_version 1872794 (0.0006) [2023-12-27 18:37:58,633][20381] Updated weights for policy 1, policy_version 1872804 (0.0009) [2023-12-27 18:37:58,697][20381] Updated weights for policy 1, policy_version 1872814 (0.0008) [2023-12-27 18:37:59,207][20347] Updated weights for policy 0, policy_version 1874457 (0.0010) [2023-12-27 18:37:59,275][20347] Updated weights for policy 0, policy_version 1874467 (0.0008) [2023-12-27 18:37:59,328][20347] Updated weights for policy 0, policy_version 1874477 (0.0007) [2023-12-27 18:37:59,396][20347] Updated weights for policy 0, policy_version 1874487 (0.0007) [2023-12-27 18:37:59,405][20381] Updated weights for policy 1, policy_version 1872824 (0.0007) [2023-12-27 18:37:59,457][20381] Updated weights for policy 1, policy_version 1872834 (0.0008) [2023-12-27 18:37:59,508][20381] Updated weights for policy 1, policy_version 1872844 (0.0008) [2023-12-27 18:38:00,161][20347] Updated weights for policy 0, policy_version 1874497 (0.0007) [2023-12-27 18:38:00,164][20381] Updated weights for policy 1, policy_version 1872854 (0.0007) [2023-12-27 18:38:00,211][20347] Updated weights for policy 0, policy_version 1874507 (0.0006) [2023-12-27 18:38:00,220][20381] Updated weights for policy 1, policy_version 1872864 (0.0006) [2023-12-27 18:38:00,262][20347] Updated weights for policy 0, policy_version 1874517 (0.0007) [2023-12-27 18:38:00,276][20381] Updated weights for policy 1, policy_version 1872874 (0.0005) [2023-12-27 18:38:00,881][20347] Updated weights for policy 0, policy_version 1874527 (0.0007) [2023-12-27 18:38:00,931][20347] Updated weights for policy 0, policy_version 1874537 (0.0007) [2023-12-27 18:38:00,937][20381] Updated weights for policy 1, policy_version 1872884 (0.0008) [2023-12-27 18:38:00,980][20347] Updated weights for policy 0, policy_version 1874547 (0.0005) [2023-12-27 18:38:00,983][20381] Updated weights for policy 1, policy_version 1872894 (0.0008) [2023-12-27 18:38:01,042][20381] Updated weights for policy 1, policy_version 1872904 (0.0008) [2023-12-27 18:38:01,672][20347] Updated weights for policy 0, policy_version 1874557 (0.0007) [2023-12-27 18:38:01,738][20347] Updated weights for policy 0, policy_version 1874567 (0.0009) [2023-12-27 18:38:01,760][20381] Updated weights for policy 1, policy_version 1872914 (0.0008) [2023-12-27 18:38:01,809][20347] Updated weights for policy 0, policy_version 1874577 (0.0008) [2023-12-27 18:38:01,821][20381] Updated weights for policy 1, policy_version 1872924 (0.0007) [2023-12-27 18:38:01,875][20381] Updated weights for policy 1, policy_version 1872934 (0.0006) [2023-12-27 18:38:01,932][20381] Updated weights for policy 1, policy_version 1872944 (0.0006) [2023-12-27 18:38:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 959504384. Throughput: 0: 10430.9, 1: 10105.8. Samples: 959515972. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.190')] [2023-12-27 18:38:02,503][20347] Updated weights for policy 0, policy_version 1874587 (0.0009) [2023-12-27 18:38:02,558][20347] Updated weights for policy 0, policy_version 1874597 (0.0010) [2023-12-27 18:38:02,603][20381] Updated weights for policy 1, policy_version 1872954 (0.0005) [2023-12-27 18:38:02,616][20347] Updated weights for policy 0, policy_version 1874607 (0.0010) [2023-12-27 18:38:02,664][20381] Updated weights for policy 1, policy_version 1872964 (0.0007) [2023-12-27 18:38:02,722][20381] Updated weights for policy 1, policy_version 1872974 (0.0008) [2023-12-27 18:38:03,251][20347] Updated weights for policy 0, policy_version 1874617 (0.0008) [2023-12-27 18:38:03,298][20347] Updated weights for policy 0, policy_version 1874627 (0.0010) [2023-12-27 18:38:03,352][20347] Updated weights for policy 0, policy_version 1874637 (0.0010) [2023-12-27 18:38:03,398][20347] Updated weights for policy 0, policy_version 1874647 (0.0006) [2023-12-27 18:38:03,412][20381] Updated weights for policy 1, policy_version 1872984 (0.0008) [2023-12-27 18:38:03,462][20381] Updated weights for policy 1, policy_version 1872994 (0.0006) [2023-12-27 18:38:03,507][20381] Updated weights for policy 1, policy_version 1873004 (0.0005) [2023-12-27 18:38:04,061][20347] Updated weights for policy 0, policy_version 1874657 (0.0009) [2023-12-27 18:38:04,093][20381] Updated weights for policy 1, policy_version 1873014 (0.0005) [2023-12-27 18:38:04,126][20347] Updated weights for policy 0, policy_version 1874667 (0.0011) [2023-12-27 18:38:04,150][20381] Updated weights for policy 1, policy_version 1873024 (0.0008) [2023-12-27 18:38:04,186][20347] Updated weights for policy 0, policy_version 1874677 (0.0011) [2023-12-27 18:38:04,207][20381] Updated weights for policy 1, policy_version 1873034 (0.0006) [2023-12-27 18:38:04,844][20347] Updated weights for policy 0, policy_version 1874687 (0.0010) [2023-12-27 18:38:04,860][20381] Updated weights for policy 1, policy_version 1873044 (0.0007) [2023-12-27 18:38:04,903][20347] Updated weights for policy 0, policy_version 1874697 (0.0010) [2023-12-27 18:38:04,915][20381] Updated weights for policy 1, policy_version 1873054 (0.0006) [2023-12-27 18:38:04,961][20347] Updated weights for policy 0, policy_version 1874707 (0.0010) [2023-12-27 18:38:04,962][20381] Updated weights for policy 1, policy_version 1873064 (0.0005) [2023-12-27 18:38:05,590][20347] Updated weights for policy 0, policy_version 1874717 (0.0010) [2023-12-27 18:38:05,645][20381] Updated weights for policy 1, policy_version 1873074 (0.0006) [2023-12-27 18:38:05,655][20347] Updated weights for policy 0, policy_version 1874727 (0.0011) [2023-12-27 18:38:05,703][20381] Updated weights for policy 1, policy_version 1873084 (0.0006) [2023-12-27 18:38:05,709][20347] Updated weights for policy 0, policy_version 1874737 (0.0010) [2023-12-27 18:38:05,758][20381] Updated weights for policy 1, policy_version 1873094 (0.0007) [2023-12-27 18:38:05,806][20381] Updated weights for policy 1, policy_version 1873104 (0.0009) [2023-12-27 18:38:06,392][20347] Updated weights for policy 0, policy_version 1874747 (0.0010) [2023-12-27 18:38:06,402][20381] Updated weights for policy 1, policy_version 1873114 (0.0011) [2023-12-27 18:38:06,451][20347] Updated weights for policy 0, policy_version 1874757 (0.0010) [2023-12-27 18:38:06,463][20381] Updated weights for policy 1, policy_version 1873124 (0.0011) [2023-12-27 18:38:06,508][20347] Updated weights for policy 0, policy_version 1874767 (0.0010) [2023-12-27 18:38:06,521][20381] Updated weights for policy 1, policy_version 1873134 (0.0011) [2023-12-27 18:38:07,168][20381] Updated weights for policy 1, policy_version 1873144 (0.0007) [2023-12-27 18:38:07,204][20347] Updated weights for policy 0, policy_version 1874777 (0.0010) [2023-12-27 18:38:07,216][20381] Updated weights for policy 1, policy_version 1873154 (0.0005) [2023-12-27 18:38:07,266][20347] Updated weights for policy 0, policy_version 1874787 (0.0010) [2023-12-27 18:38:07,275][20381] Updated weights for policy 1, policy_version 1873164 (0.0010) [2023-12-27 18:38:07,331][20347] Updated weights for policy 0, policy_version 1874797 (0.0010) [2023-12-27 18:38:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 959610880. Throughput: 0: 10441.2, 1: 10163.9. Samples: 959579428. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '783.940')] [2023-12-27 18:38:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001873168_479584256.pth... [2023-12-27 18:38:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001871984_479281152.pth [2023-12-27 18:38:07,393][20347] Updated weights for policy 0, policy_version 1874807 (0.0010) [2023-12-27 18:38:07,395][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001874808_480034816.pth... [2023-12-27 18:38:07,399][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001873560_479715328.pth [2023-12-27 18:38:07,954][20381] Updated weights for policy 1, policy_version 1873174 (0.0011) [2023-12-27 18:38:08,014][20381] Updated weights for policy 1, policy_version 1873184 (0.0011) [2023-12-27 18:38:08,057][20347] Updated weights for policy 0, policy_version 1874817 (0.0010) [2023-12-27 18:38:08,067][20381] Updated weights for policy 1, policy_version 1873194 (0.0011) [2023-12-27 18:38:08,113][20347] Updated weights for policy 0, policy_version 1874827 (0.0010) [2023-12-27 18:38:08,161][20347] Updated weights for policy 0, policy_version 1874837 (0.0010) [2023-12-27 18:38:08,725][20381] Updated weights for policy 1, policy_version 1873204 (0.0010) [2023-12-27 18:38:08,787][20381] Updated weights for policy 1, policy_version 1873214 (0.0007) [2023-12-27 18:38:08,846][20381] Updated weights for policy 1, policy_version 1873224 (0.0007) [2023-12-27 18:38:08,892][20347] Updated weights for policy 0, policy_version 1874847 (0.0011) [2023-12-27 18:38:08,958][20347] Updated weights for policy 0, policy_version 1874857 (0.0012) [2023-12-27 18:38:09,022][20347] Updated weights for policy 0, policy_version 1874867 (0.0010) [2023-12-27 18:38:09,558][20381] Updated weights for policy 1, policy_version 1873234 (0.0007) [2023-12-27 18:38:09,617][20381] Updated weights for policy 1, policy_version 1873244 (0.0008) [2023-12-27 18:38:09,680][20381] Updated weights for policy 1, policy_version 1873254 (0.0007) [2023-12-27 18:38:09,743][20381] Updated weights for policy 1, policy_version 1873264 (0.0007) [2023-12-27 18:38:09,752][20347] Updated weights for policy 0, policy_version 1874877 (0.0009) [2023-12-27 18:38:09,813][20347] Updated weights for policy 0, policy_version 1874887 (0.0008) [2023-12-27 18:38:09,884][20347] Updated weights for policy 0, policy_version 1874897 (0.0010) [2023-12-27 18:38:10,460][20381] Updated weights for policy 1, policy_version 1873274 (0.0008) [2023-12-27 18:38:10,525][20381] Updated weights for policy 1, policy_version 1873284 (0.0006) [2023-12-27 18:38:10,578][20381] Updated weights for policy 1, policy_version 1873294 (0.0005) [2023-12-27 18:38:10,650][20347] Updated weights for policy 0, policy_version 1874907 (0.0009) [2023-12-27 18:38:10,711][20347] Updated weights for policy 0, policy_version 1874917 (0.0008) [2023-12-27 18:38:10,766][20347] Updated weights for policy 0, policy_version 1874927 (0.0008) [2023-12-27 18:38:11,188][20381] Updated weights for policy 1, policy_version 1873304 (0.0008) [2023-12-27 18:38:11,248][20381] Updated weights for policy 1, policy_version 1873314 (0.0008) [2023-12-27 18:38:11,305][20381] Updated weights for policy 1, policy_version 1873324 (0.0008) [2023-12-27 18:38:11,490][20347] Updated weights for policy 0, policy_version 1874937 (0.0008) [2023-12-27 18:38:11,548][20347] Updated weights for policy 0, policy_version 1874947 (0.0007) [2023-12-27 18:38:11,611][20347] Updated weights for policy 0, policy_version 1874957 (0.0008) [2023-12-27 18:38:11,671][20347] Updated weights for policy 0, policy_version 1874967 (0.0008) [2023-12-27 18:38:12,118][20381] Updated weights for policy 1, policy_version 1873334 (0.0011) [2023-12-27 18:38:12,174][20381] Updated weights for policy 1, policy_version 1873344 (0.0011) [2023-12-27 18:38:12,234][20381] Updated weights for policy 1, policy_version 1873354 (0.0011) [2023-12-27 18:38:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 959709184. Throughput: 0: 10374.3, 1: 10239.5. Samples: 959700788. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '844.850')] [2023-12-27 18:38:12,436][20347] Updated weights for policy 0, policy_version 1874977 (0.0008) [2023-12-27 18:38:12,493][20347] Updated weights for policy 0, policy_version 1874987 (0.0007) [2023-12-27 18:38:12,556][20347] Updated weights for policy 0, policy_version 1874997 (0.0007) [2023-12-27 18:38:12,909][20381] Updated weights for policy 1, policy_version 1873364 (0.0009) [2023-12-27 18:38:12,954][20381] Updated weights for policy 1, policy_version 1873374 (0.0008) [2023-12-27 18:38:13,010][20381] Updated weights for policy 1, policy_version 1873384 (0.0008) [2023-12-27 18:38:13,166][20347] Updated weights for policy 0, policy_version 1875007 (0.0010) [2023-12-27 18:38:13,223][20347] Updated weights for policy 0, policy_version 1875017 (0.0010) [2023-12-27 18:38:13,280][20347] Updated weights for policy 0, policy_version 1875027 (0.0010) [2023-12-27 18:38:13,759][20381] Updated weights for policy 1, policy_version 1873394 (0.0007) [2023-12-27 18:38:13,813][20381] Updated weights for policy 1, policy_version 1873404 (0.0008) [2023-12-27 18:38:13,872][20381] Updated weights for policy 1, policy_version 1873414 (0.0006) [2023-12-27 18:38:13,874][20347] Updated weights for policy 0, policy_version 1875037 (0.0008) [2023-12-27 18:38:13,921][20381] Updated weights for policy 1, policy_version 1873424 (0.0005) [2023-12-27 18:38:13,922][20347] Updated weights for policy 0, policy_version 1875047 (0.0010) [2023-12-27 18:38:13,967][20347] Updated weights for policy 0, policy_version 1875057 (0.0010) [2023-12-27 18:38:14,620][20381] Updated weights for policy 1, policy_version 1873434 (0.0007) [2023-12-27 18:38:14,658][20347] Updated weights for policy 0, policy_version 1875067 (0.0010) [2023-12-27 18:38:14,681][20381] Updated weights for policy 1, policy_version 1873444 (0.0006) [2023-12-27 18:38:14,709][20347] Updated weights for policy 0, policy_version 1875077 (0.0010) [2023-12-27 18:38:14,736][20381] Updated weights for policy 1, policy_version 1873454 (0.0005) [2023-12-27 18:38:14,772][20347] Updated weights for policy 0, policy_version 1875087 (0.0009) [2023-12-27 18:38:15,434][20381] Updated weights for policy 1, policy_version 1873464 (0.0009) [2023-12-27 18:38:15,477][20347] Updated weights for policy 0, policy_version 1875097 (0.0008) [2023-12-27 18:38:15,487][20381] Updated weights for policy 1, policy_version 1873474 (0.0008) [2023-12-27 18:38:15,531][20347] Updated weights for policy 0, policy_version 1875107 (0.0005) [2023-12-27 18:38:15,547][20381] Updated weights for policy 1, policy_version 1873484 (0.0009) [2023-12-27 18:38:15,590][20347] Updated weights for policy 0, policy_version 1875117 (0.0005) [2023-12-27 18:38:15,651][20347] Updated weights for policy 0, policy_version 1875127 (0.0006) [2023-12-27 18:38:16,203][20381] Updated weights for policy 1, policy_version 1873494 (0.0009) [2023-12-27 18:38:16,208][20347] Updated weights for policy 0, policy_version 1875137 (0.0006) [2023-12-27 18:38:16,255][20381] Updated weights for policy 1, policy_version 1873504 (0.0005) [2023-12-27 18:38:16,261][20347] Updated weights for policy 0, policy_version 1875147 (0.0007) [2023-12-27 18:38:16,312][20381] Updated weights for policy 1, policy_version 1873514 (0.0009) [2023-12-27 18:38:16,313][20347] Updated weights for policy 0, policy_version 1875157 (0.0006) [2023-12-27 18:38:16,963][20381] Updated weights for policy 1, policy_version 1873524 (0.0011) [2023-12-27 18:38:16,998][20347] Updated weights for policy 0, policy_version 1875167 (0.0005) [2023-12-27 18:38:17,011][20381] Updated weights for policy 1, policy_version 1873534 (0.0010) [2023-12-27 18:38:17,045][20347] Updated weights for policy 0, policy_version 1875177 (0.0007) [2023-12-27 18:38:17,070][20381] Updated weights for policy 1, policy_version 1873544 (0.0011) [2023-12-27 18:38:17,106][20347] Updated weights for policy 0, policy_version 1875187 (0.0007) [2023-12-27 18:38:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 959815680. Throughput: 0: 10406.1, 1: 10151.7. Samples: 959822408. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '818.020')] [2023-12-27 18:38:17,842][20347] Updated weights for policy 0, policy_version 1875197 (0.0007) [2023-12-27 18:38:17,847][20381] Updated weights for policy 1, policy_version 1873555 (0.0007) [2023-12-27 18:38:17,896][20347] Updated weights for policy 0, policy_version 1875207 (0.0006) [2023-12-27 18:38:17,901][20381] Updated weights for policy 1, policy_version 1873565 (0.0006) [2023-12-27 18:38:17,951][20347] Updated weights for policy 0, policy_version 1875217 (0.0005) [2023-12-27 18:38:17,956][20381] Updated weights for policy 1, policy_version 1873575 (0.0006) [2023-12-27 18:38:18,633][20347] Updated weights for policy 0, policy_version 1875227 (0.0007) [2023-12-27 18:38:18,646][20381] Updated weights for policy 1, policy_version 1873585 (0.0006) [2023-12-27 18:38:18,685][20347] Updated weights for policy 0, policy_version 1875237 (0.0010) [2023-12-27 18:38:18,702][20381] Updated weights for policy 1, policy_version 1873595 (0.0005) [2023-12-27 18:38:18,741][20347] Updated weights for policy 0, policy_version 1875247 (0.0010) [2023-12-27 18:38:18,761][20381] Updated weights for policy 1, policy_version 1873605 (0.0005) [2023-12-27 18:38:18,817][20381] Updated weights for policy 1, policy_version 1873615 (0.0006) [2023-12-27 18:38:19,392][20347] Updated weights for policy 0, policy_version 1875257 (0.0010) [2023-12-27 18:38:19,443][20347] Updated weights for policy 0, policy_version 1875267 (0.0006) [2023-12-27 18:38:19,484][20381] Updated weights for policy 1, policy_version 1873625 (0.0008) [2023-12-27 18:38:19,502][20347] Updated weights for policy 0, policy_version 1875277 (0.0007) [2023-12-27 18:38:19,548][20381] Updated weights for policy 1, policy_version 1873635 (0.0007) [2023-12-27 18:38:19,564][20347] Updated weights for policy 0, policy_version 1875287 (0.0010) [2023-12-27 18:38:19,610][20381] Updated weights for policy 1, policy_version 1873645 (0.0008) [2023-12-27 18:38:20,253][20347] Updated weights for policy 0, policy_version 1875297 (0.0007) [2023-12-27 18:38:20,302][20381] Updated weights for policy 1, policy_version 1873655 (0.0005) [2023-12-27 18:38:20,315][20347] Updated weights for policy 0, policy_version 1875307 (0.0006) [2023-12-27 18:38:20,362][20381] Updated weights for policy 1, policy_version 1873665 (0.0006) [2023-12-27 18:38:20,378][20347] Updated weights for policy 0, policy_version 1875317 (0.0006) [2023-12-27 18:38:20,423][20381] Updated weights for policy 1, policy_version 1873675 (0.0007) [2023-12-27 18:38:21,115][20381] Updated weights for policy 1, policy_version 1873685 (0.0007) [2023-12-27 18:38:21,135][20347] Updated weights for policy 0, policy_version 1875327 (0.0007) [2023-12-27 18:38:21,179][20381] Updated weights for policy 1, policy_version 1873695 (0.0007) [2023-12-27 18:38:21,200][20347] Updated weights for policy 0, policy_version 1875337 (0.0008) [2023-12-27 18:38:21,235][20381] Updated weights for policy 1, policy_version 1873705 (0.0007) [2023-12-27 18:38:21,259][20347] Updated weights for policy 0, policy_version 1875347 (0.0007) [2023-12-27 18:38:21,971][20381] Updated weights for policy 1, policy_version 1873715 (0.0007) [2023-12-27 18:38:22,015][20347] Updated weights for policy 0, policy_version 1875357 (0.0008) [2023-12-27 18:38:22,030][20381] Updated weights for policy 1, policy_version 1873725 (0.0006) [2023-12-27 18:38:22,084][20347] Updated weights for policy 0, policy_version 1875367 (0.0006) [2023-12-27 18:38:22,093][20381] Updated weights for policy 1, policy_version 1873735 (0.0007) [2023-12-27 18:38:22,146][20347] Updated weights for policy 0, policy_version 1875377 (0.0007) [2023-12-27 18:38:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 959913984. Throughput: 0: 10347.6, 1: 10117.5. Samples: 959882528. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '818.960')] [2023-12-27 18:38:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001873744_479731712.pth... [2023-12-27 18:38:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001875384_480182272.pth... [2023-12-27 18:38:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001872560_479428608.pth [2023-12-27 18:38:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001874168_479870976.pth [2023-12-27 18:38:22,750][20381] Updated weights for policy 1, policy_version 1873745 (0.0007) [2023-12-27 18:38:22,812][20381] Updated weights for policy 1, policy_version 1873755 (0.0007) [2023-12-27 18:38:22,833][20347] Updated weights for policy 0, policy_version 1875387 (0.0009) [2023-12-27 18:38:22,892][20347] Updated weights for policy 0, policy_version 1875397 (0.0010) [2023-12-27 18:38:22,894][20381] Updated weights for policy 1, policy_version 1873765 (0.0005) [2023-12-27 18:38:22,950][20381] Updated weights for policy 1, policy_version 1873775 (0.0007) [2023-12-27 18:38:22,951][20347] Updated weights for policy 0, policy_version 1875407 (0.0010) [2023-12-27 18:38:23,572][20381] Updated weights for policy 1, policy_version 1873785 (0.0009) [2023-12-27 18:38:23,611][20347] Updated weights for policy 0, policy_version 1875417 (0.0008) [2023-12-27 18:38:23,622][20381] Updated weights for policy 1, policy_version 1873795 (0.0008) [2023-12-27 18:38:23,660][20347] Updated weights for policy 0, policy_version 1875427 (0.0005) [2023-12-27 18:38:23,674][20381] Updated weights for policy 1, policy_version 1873805 (0.0008) [2023-12-27 18:38:23,708][20347] Updated weights for policy 0, policy_version 1875437 (0.0007) [2023-12-27 18:38:23,759][20347] Updated weights for policy 0, policy_version 1875447 (0.0007) [2023-12-27 18:38:24,341][20347] Updated weights for policy 0, policy_version 1875457 (0.0006) [2023-12-27 18:38:24,389][20347] Updated weights for policy 0, policy_version 1875467 (0.0005) [2023-12-27 18:38:24,439][20347] Updated weights for policy 0, policy_version 1875477 (0.0006) [2023-12-27 18:38:24,440][20381] Updated weights for policy 1, policy_version 1873815 (0.0007) [2023-12-27 18:38:24,491][20381] Updated weights for policy 1, policy_version 1873825 (0.0007) [2023-12-27 18:38:24,544][20381] Updated weights for policy 1, policy_version 1873835 (0.0009) [2023-12-27 18:38:25,028][20347] Updated weights for policy 0, policy_version 1875487 (0.0007) [2023-12-27 18:38:25,075][20347] Updated weights for policy 0, policy_version 1875497 (0.0007) [2023-12-27 18:38:25,118][20347] Updated weights for policy 0, policy_version 1875507 (0.0008) [2023-12-27 18:38:25,260][20381] Updated weights for policy 1, policy_version 1873845 (0.0007) [2023-12-27 18:38:25,309][20381] Updated weights for policy 1, policy_version 1873855 (0.0008) [2023-12-27 18:38:25,368][20381] Updated weights for policy 1, policy_version 1873865 (0.0008) [2023-12-27 18:38:25,764][20347] Updated weights for policy 0, policy_version 1875517 (0.0008) [2023-12-27 18:38:25,814][20347] Updated weights for policy 0, policy_version 1875527 (0.0007) [2023-12-27 18:38:25,865][20347] Updated weights for policy 0, policy_version 1875537 (0.0008) [2023-12-27 18:38:26,041][20381] Updated weights for policy 1, policy_version 1873875 (0.0008) [2023-12-27 18:38:26,103][20381] Updated weights for policy 1, policy_version 1873885 (0.0007) [2023-12-27 18:38:26,162][20381] Updated weights for policy 1, policy_version 1873895 (0.0008) [2023-12-27 18:38:26,579][20347] Updated weights for policy 0, policy_version 1875547 (0.0007) [2023-12-27 18:38:26,631][20347] Updated weights for policy 0, policy_version 1875557 (0.0005) [2023-12-27 18:38:26,679][20347] Updated weights for policy 0, policy_version 1875567 (0.0005) [2023-12-27 18:38:26,794][20381] Updated weights for policy 1, policy_version 1873905 (0.0007) [2023-12-27 18:38:26,840][20381] Updated weights for policy 1, policy_version 1873915 (0.0008) [2023-12-27 18:38:26,890][20381] Updated weights for policy 1, policy_version 1873925 (0.0008) [2023-12-27 18:38:26,942][20381] Updated weights for policy 1, policy_version 1873935 (0.0008) [2023-12-27 18:38:27,275][20347] Updated weights for policy 0, policy_version 1875577 (0.0006) [2023-12-27 18:38:27,329][20347] Updated weights for policy 0, policy_version 1875587 (0.0009) [2023-12-27 18:38:27,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 960012288. Throughput: 0: 10334.9, 1: 10014.0. Samples: 960003316. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '866.080')] [2023-12-27 18:38:27,384][20347] Updated weights for policy 0, policy_version 1875597 (0.0008) [2023-12-27 18:38:27,438][20347] Updated weights for policy 0, policy_version 1875607 (0.0009) [2023-12-27 18:38:27,598][20381] Updated weights for policy 1, policy_version 1873945 (0.0006) [2023-12-27 18:38:27,647][20381] Updated weights for policy 1, policy_version 1873955 (0.0008) [2023-12-27 18:38:27,695][20381] Updated weights for policy 1, policy_version 1873965 (0.0006) [2023-12-27 18:38:28,055][20347] Updated weights for policy 0, policy_version 1875617 (0.0007) [2023-12-27 18:38:28,113][20347] Updated weights for policy 0, policy_version 1875627 (0.0007) [2023-12-27 18:38:28,161][20347] Updated weights for policy 0, policy_version 1875637 (0.0006) [2023-12-27 18:38:28,358][20381] Updated weights for policy 1, policy_version 1873975 (0.0009) [2023-12-27 18:38:28,413][20381] Updated weights for policy 1, policy_version 1873985 (0.0011) [2023-12-27 18:38:28,469][20381] Updated weights for policy 1, policy_version 1873995 (0.0006) [2023-12-27 18:38:28,829][20347] Updated weights for policy 0, policy_version 1875647 (0.0006) [2023-12-27 18:38:28,883][20347] Updated weights for policy 0, policy_version 1875657 (0.0006) [2023-12-27 18:38:28,941][20347] Updated weights for policy 0, policy_version 1875667 (0.0006) [2023-12-27 18:38:29,060][20381] Updated weights for policy 1, policy_version 1874005 (0.0008) [2023-12-27 18:38:29,108][20381] Updated weights for policy 1, policy_version 1874015 (0.0008) [2023-12-27 18:38:29,151][20381] Updated weights for policy 1, policy_version 1874025 (0.0005) [2023-12-27 18:38:29,682][20347] Updated weights for policy 0, policy_version 1875677 (0.0007) [2023-12-27 18:38:29,733][20347] Updated weights for policy 0, policy_version 1875687 (0.0007) [2023-12-27 18:38:29,780][20347] Updated weights for policy 0, policy_version 1875697 (0.0008) [2023-12-27 18:38:29,827][20381] Updated weights for policy 1, policy_version 1874035 (0.0006) [2023-12-27 18:38:29,884][20381] Updated weights for policy 1, policy_version 1874045 (0.0007) [2023-12-27 18:38:29,948][20381] Updated weights for policy 1, policy_version 1874055 (0.0007) [2023-12-27 18:38:30,479][20347] Updated weights for policy 0, policy_version 1875707 (0.0008) [2023-12-27 18:38:30,538][20347] Updated weights for policy 0, policy_version 1875717 (0.0005) [2023-12-27 18:38:30,566][20381] Updated weights for policy 1, policy_version 1874065 (0.0006) [2023-12-27 18:38:30,589][20347] Updated weights for policy 0, policy_version 1875727 (0.0005) [2023-12-27 18:38:30,620][20381] Updated weights for policy 1, policy_version 1874075 (0.0010) [2023-12-27 18:38:30,673][20381] Updated weights for policy 1, policy_version 1874085 (0.0010) [2023-12-27 18:38:30,723][20381] Updated weights for policy 1, policy_version 1874095 (0.0010) [2023-12-27 18:38:31,232][20347] Updated weights for policy 0, policy_version 1875737 (0.0008) [2023-12-27 18:38:31,290][20347] Updated weights for policy 0, policy_version 1875747 (0.0008) [2023-12-27 18:38:31,349][20347] Updated weights for policy 0, policy_version 1875757 (0.0007) [2023-12-27 18:38:31,405][20347] Updated weights for policy 0, policy_version 1875767 (0.0008) [2023-12-27 18:38:31,454][20381] Updated weights for policy 1, policy_version 1874105 (0.0008) [2023-12-27 18:38:31,511][20381] Updated weights for policy 1, policy_version 1874115 (0.0008) [2023-12-27 18:38:31,568][20381] Updated weights for policy 1, policy_version 1874125 (0.0008) [2023-12-27 18:38:32,168][20347] Updated weights for policy 0, policy_version 1875777 (0.0007) [2023-12-27 18:38:32,226][20347] Updated weights for policy 0, policy_version 1875787 (0.0008) [2023-12-27 18:38:32,275][20381] Updated weights for policy 1, policy_version 1874135 (0.0008) [2023-12-27 18:38:32,288][20347] Updated weights for policy 0, policy_version 1875797 (0.0008) [2023-12-27 18:38:32,335][20381] Updated weights for policy 1, policy_version 1874145 (0.0007) [2023-12-27 18:38:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 960118784. Throughput: 0: 10322.9, 1: 10117.0. Samples: 960129564. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.740')] [2023-12-27 18:38:32,397][20381] Updated weights for policy 1, policy_version 1874155 (0.0006) [2023-12-27 18:38:32,974][20381] Updated weights for policy 1, policy_version 1874165 (0.0007) [2023-12-27 18:38:33,007][20347] Updated weights for policy 0, policy_version 1875807 (0.0005) [2023-12-27 18:38:33,025][20381] Updated weights for policy 1, policy_version 1874175 (0.0007) [2023-12-27 18:38:33,058][20347] Updated weights for policy 0, policy_version 1875817 (0.0005) [2023-12-27 18:38:33,081][20381] Updated weights for policy 1, policy_version 1874185 (0.0006) [2023-12-27 18:38:33,121][20347] Updated weights for policy 0, policy_version 1875827 (0.0009) [2023-12-27 18:38:33,653][20381] Updated weights for policy 1, policy_version 1874195 (0.0006) [2023-12-27 18:38:33,710][20381] Updated weights for policy 1, policy_version 1874205 (0.0006) [2023-12-27 18:38:33,767][20381] Updated weights for policy 1, policy_version 1874215 (0.0008) [2023-12-27 18:38:33,787][20347] Updated weights for policy 0, policy_version 1875837 (0.0010) [2023-12-27 18:38:33,842][20347] Updated weights for policy 0, policy_version 1875847 (0.0010) [2023-12-27 18:38:33,890][20347] Updated weights for policy 0, policy_version 1875857 (0.0010) [2023-12-27 18:38:34,372][20381] Updated weights for policy 1, policy_version 1874225 (0.0006) [2023-12-27 18:38:34,429][20381] Updated weights for policy 1, policy_version 1874235 (0.0008) [2023-12-27 18:38:34,480][20381] Updated weights for policy 1, policy_version 1874245 (0.0008) [2023-12-27 18:38:34,536][20381] Updated weights for policy 1, policy_version 1874255 (0.0006) [2023-12-27 18:38:34,562][20347] Updated weights for policy 0, policy_version 1875867 (0.0010) [2023-12-27 18:38:34,611][20347] Updated weights for policy 0, policy_version 1875877 (0.0006) [2023-12-27 18:38:34,662][20347] Updated weights for policy 0, policy_version 1875887 (0.0007) [2023-12-27 18:38:35,261][20381] Updated weights for policy 1, policy_version 1874265 (0.0006) [2023-12-27 18:38:35,312][20381] Updated weights for policy 1, policy_version 1874275 (0.0007) [2023-12-27 18:38:35,333][20347] Updated weights for policy 0, policy_version 1875897 (0.0006) [2023-12-27 18:38:35,362][20381] Updated weights for policy 1, policy_version 1874285 (0.0007) [2023-12-27 18:38:35,389][20347] Updated weights for policy 0, policy_version 1875907 (0.0007) [2023-12-27 18:38:35,440][20347] Updated weights for policy 0, policy_version 1875917 (0.0005) [2023-12-27 18:38:35,499][20347] Updated weights for policy 0, policy_version 1875927 (0.0006) [2023-12-27 18:38:35,987][20381] Updated weights for policy 1, policy_version 1874295 (0.0007) [2023-12-27 18:38:36,046][20381] Updated weights for policy 1, policy_version 1874305 (0.0005) [2023-12-27 18:38:36,104][20381] Updated weights for policy 1, policy_version 1874315 (0.0006) [2023-12-27 18:38:36,121][20347] Updated weights for policy 0, policy_version 1875937 (0.0009) [2023-12-27 18:38:36,174][20347] Updated weights for policy 0, policy_version 1875947 (0.0010) [2023-12-27 18:38:36,233][20347] Updated weights for policy 0, policy_version 1875957 (0.0007) [2023-12-27 18:38:36,809][20381] Updated weights for policy 1, policy_version 1874325 (0.0009) [2023-12-27 18:38:36,864][20381] Updated weights for policy 1, policy_version 1874335 (0.0010) [2023-12-27 18:38:36,913][20381] Updated weights for policy 1, policy_version 1874345 (0.0010) [2023-12-27 18:38:36,919][20347] Updated weights for policy 0, policy_version 1875967 (0.0007) [2023-12-27 18:38:36,969][20347] Updated weights for policy 0, policy_version 1875977 (0.0007) [2023-12-27 18:38:37,014][20347] Updated weights for policy 0, policy_version 1875987 (0.0008) [2023-12-27 18:38:37,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20480.0, 300 sec: 20549.5). Total num frames: 960225280. Throughput: 0: 10289.1, 1: 10154.3. Samples: 960192656. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.960')] [2023-12-27 18:38:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001874352_479887360.pth... [2023-12-27 18:38:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001875992_480337920.pth... [2023-12-27 18:38:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001873168_479584256.pth [2023-12-27 18:38:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001874808_480034816.pth [2023-12-27 18:38:37,498][20381] Updated weights for policy 1, policy_version 1874355 (0.0009) [2023-12-27 18:38:37,550][20381] Updated weights for policy 1, policy_version 1874365 (0.0005) [2023-12-27 18:38:37,603][20381] Updated weights for policy 1, policy_version 1874375 (0.0008) [2023-12-27 18:38:37,781][20347] Updated weights for policy 0, policy_version 1875997 (0.0006) [2023-12-27 18:38:37,839][20347] Updated weights for policy 0, policy_version 1876007 (0.0007) [2023-12-27 18:38:37,891][20347] Updated weights for policy 0, policy_version 1876017 (0.0010) [2023-12-27 18:38:38,244][20381] Updated weights for policy 1, policy_version 1874385 (0.0011) [2023-12-27 18:38:38,304][20381] Updated weights for policy 1, policy_version 1874395 (0.0010) [2023-12-27 18:38:38,361][20381] Updated weights for policy 1, policy_version 1874405 (0.0011) [2023-12-27 18:38:38,420][20381] Updated weights for policy 1, policy_version 1874415 (0.0010) [2023-12-27 18:38:38,496][20347] Updated weights for policy 0, policy_version 1876027 (0.0009) [2023-12-27 18:38:38,557][20347] Updated weights for policy 0, policy_version 1876037 (0.0011) [2023-12-27 18:38:38,618][20347] Updated weights for policy 0, policy_version 1876047 (0.0010) [2023-12-27 18:38:39,091][20381] Updated weights for policy 1, policy_version 1874425 (0.0009) [2023-12-27 18:38:39,145][20381] Updated weights for policy 1, policy_version 1874435 (0.0007) [2023-12-27 18:38:39,203][20381] Updated weights for policy 1, policy_version 1874445 (0.0008) [2023-12-27 18:38:39,228][20347] Updated weights for policy 0, policy_version 1876057 (0.0010) [2023-12-27 18:38:39,292][20347] Updated weights for policy 0, policy_version 1876067 (0.0007) [2023-12-27 18:38:39,356][20347] Updated weights for policy 0, policy_version 1876077 (0.0008) [2023-12-27 18:38:39,426][20347] Updated weights for policy 0, policy_version 1876087 (0.0008) [2023-12-27 18:38:40,004][20381] Updated weights for policy 1, policy_version 1874455 (0.0008) [2023-12-27 18:38:40,064][20381] Updated weights for policy 1, policy_version 1874465 (0.0007) [2023-12-27 18:38:40,129][20381] Updated weights for policy 1, policy_version 1874475 (0.0008) [2023-12-27 18:38:40,197][20347] Updated weights for policy 0, policy_version 1876097 (0.0007) [2023-12-27 18:38:40,258][20347] Updated weights for policy 0, policy_version 1876107 (0.0007) [2023-12-27 18:38:40,315][20347] Updated weights for policy 0, policy_version 1876117 (0.0007) [2023-12-27 18:38:40,810][20381] Updated weights for policy 1, policy_version 1874485 (0.0008) [2023-12-27 18:38:40,865][20381] Updated weights for policy 1, policy_version 1874495 (0.0005) [2023-12-27 18:38:40,920][20381] Updated weights for policy 1, policy_version 1874505 (0.0006) [2023-12-27 18:38:41,048][20347] Updated weights for policy 0, policy_version 1876127 (0.0008) [2023-12-27 18:38:41,103][20347] Updated weights for policy 0, policy_version 1876137 (0.0007) [2023-12-27 18:38:41,177][20347] Updated weights for policy 0, policy_version 1876147 (0.0008) [2023-12-27 18:38:41,579][20381] Updated weights for policy 1, policy_version 1874515 (0.0007) [2023-12-27 18:38:41,665][20381] Updated weights for policy 1, policy_version 1874525 (0.0008) [2023-12-27 18:38:41,716][20381] Updated weights for policy 1, policy_version 1874535 (0.0008) [2023-12-27 18:38:41,991][20347] Updated weights for policy 0, policy_version 1876157 (0.0008) [2023-12-27 18:38:42,052][20347] Updated weights for policy 0, policy_version 1876167 (0.0009) [2023-12-27 18:38:42,105][20347] Updated weights for policy 0, policy_version 1876177 (0.0009) [2023-12-27 18:38:42,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 960323584. Throughput: 0: 10217.1, 1: 10268.8. Samples: 960316176. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.040')] [2023-12-27 18:38:42,521][20381] Updated weights for policy 1, policy_version 1874545 (0.0008) [2023-12-27 18:38:42,565][20381] Updated weights for policy 1, policy_version 1874555 (0.0008) [2023-12-27 18:38:42,609][20381] Updated weights for policy 1, policy_version 1874565 (0.0008) [2023-12-27 18:38:42,657][20381] Updated weights for policy 1, policy_version 1874575 (0.0008) [2023-12-27 18:38:42,852][20347] Updated weights for policy 0, policy_version 1876187 (0.0007) [2023-12-27 18:38:42,905][20347] Updated weights for policy 0, policy_version 1876197 (0.0010) [2023-12-27 18:38:42,956][20347] Updated weights for policy 0, policy_version 1876207 (0.0010) [2023-12-27 18:38:43,345][20381] Updated weights for policy 1, policy_version 1874585 (0.0007) [2023-12-27 18:38:43,402][20381] Updated weights for policy 1, policy_version 1874595 (0.0005) [2023-12-27 18:38:43,460][20381] Updated weights for policy 1, policy_version 1874605 (0.0007) [2023-12-27 18:38:43,602][20347] Updated weights for policy 0, policy_version 1876217 (0.0010) [2023-12-27 18:38:43,646][20347] Updated weights for policy 0, policy_version 1876227 (0.0005) [2023-12-27 18:38:43,695][20347] Updated weights for policy 0, policy_version 1876237 (0.0009) [2023-12-27 18:38:43,743][20347] Updated weights for policy 0, policy_version 1876247 (0.0010) [2023-12-27 18:38:44,051][20381] Updated weights for policy 1, policy_version 1874615 (0.0011) [2023-12-27 18:38:44,097][20381] Updated weights for policy 1, policy_version 1874625 (0.0011) [2023-12-27 18:38:44,159][20381] Updated weights for policy 1, policy_version 1874635 (0.0011) [2023-12-27 18:38:44,346][20347] Updated weights for policy 0, policy_version 1876257 (0.0007) [2023-12-27 18:38:44,407][20347] Updated weights for policy 0, policy_version 1876267 (0.0007) [2023-12-27 18:38:44,461][20347] Updated weights for policy 0, policy_version 1876277 (0.0010) [2023-12-27 18:38:44,760][20381] Updated weights for policy 1, policy_version 1874645 (0.0009) [2023-12-27 18:38:44,822][20381] Updated weights for policy 1, policy_version 1874655 (0.0007) [2023-12-27 18:38:44,872][20381] Updated weights for policy 1, policy_version 1874665 (0.0011) [2023-12-27 18:38:45,105][20347] Updated weights for policy 0, policy_version 1876287 (0.0010) [2023-12-27 18:38:45,165][20347] Updated weights for policy 0, policy_version 1876297 (0.0011) [2023-12-27 18:38:45,227][20347] Updated weights for policy 0, policy_version 1876307 (0.0010) [2023-12-27 18:38:45,571][20381] Updated weights for policy 1, policy_version 1874675 (0.0010) [2023-12-27 18:38:45,620][20381] Updated weights for policy 1, policy_version 1874685 (0.0007) [2023-12-27 18:38:45,679][20381] Updated weights for policy 1, policy_version 1874695 (0.0008) [2023-12-27 18:38:45,821][20347] Updated weights for policy 0, policy_version 1876317 (0.0010) [2023-12-27 18:38:45,869][20347] Updated weights for policy 0, policy_version 1876327 (0.0010) [2023-12-27 18:38:45,929][20347] Updated weights for policy 0, policy_version 1876337 (0.0010) [2023-12-27 18:38:46,319][20381] Updated weights for policy 1, policy_version 1874705 (0.0008) [2023-12-27 18:38:46,367][20381] Updated weights for policy 1, policy_version 1874715 (0.0010) [2023-12-27 18:38:46,416][20381] Updated weights for policy 1, policy_version 1874725 (0.0010) [2023-12-27 18:38:46,486][20381] Updated weights for policy 1, policy_version 1874735 (0.0009) [2023-12-27 18:38:46,547][20347] Updated weights for policy 0, policy_version 1876347 (0.0009) [2023-12-27 18:38:46,590][20347] Updated weights for policy 0, policy_version 1876357 (0.0007) [2023-12-27 18:38:46,634][20347] Updated weights for policy 0, policy_version 1876367 (0.0007) [2023-12-27 18:38:47,163][20381] Updated weights for policy 1, policy_version 1874745 (0.0010) [2023-12-27 18:38:47,207][20381] Updated weights for policy 1, policy_version 1874755 (0.0007) [2023-12-27 18:38:47,258][20381] Updated weights for policy 1, policy_version 1874765 (0.0007) [2023-12-27 18:38:47,355][20347] Updated weights for policy 0, policy_version 1876377 (0.0007) [2023-12-27 18:38:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 960430080. Throughput: 0: 10225.9, 1: 10297.4. Samples: 960439524. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.500')] [2023-12-27 18:38:47,417][20347] Updated weights for policy 0, policy_version 1876387 (0.0007) [2023-12-27 18:38:47,470][20347] Updated weights for policy 0, policy_version 1876397 (0.0005) [2023-12-27 18:38:47,518][20347] Updated weights for policy 0, policy_version 1876407 (0.0006) [2023-12-27 18:38:47,954][20381] Updated weights for policy 1, policy_version 1874775 (0.0006) [2023-12-27 18:38:48,022][20381] Updated weights for policy 1, policy_version 1874785 (0.0007) [2023-12-27 18:38:48,077][20381] Updated weights for policy 1, policy_version 1874795 (0.0008) [2023-12-27 18:38:48,233][20347] Updated weights for policy 0, policy_version 1876417 (0.0007) [2023-12-27 18:38:48,287][20347] Updated weights for policy 0, policy_version 1876427 (0.0009) [2023-12-27 18:38:48,379][20347] Updated weights for policy 0, policy_version 1876437 (0.0008) [2023-12-27 18:38:48,742][20381] Updated weights for policy 1, policy_version 1874805 (0.0007) [2023-12-27 18:38:48,794][20381] Updated weights for policy 1, policy_version 1874815 (0.0007) [2023-12-27 18:38:48,856][20381] Updated weights for policy 1, policy_version 1874825 (0.0008) [2023-12-27 18:38:49,013][20347] Updated weights for policy 0, policy_version 1876447 (0.0006) [2023-12-27 18:38:49,070][20347] Updated weights for policy 0, policy_version 1876457 (0.0008) [2023-12-27 18:38:49,129][20347] Updated weights for policy 0, policy_version 1876467 (0.0008) [2023-12-27 18:38:49,509][20381] Updated weights for policy 1, policy_version 1874835 (0.0007) [2023-12-27 18:38:49,561][20381] Updated weights for policy 1, policy_version 1874845 (0.0008) [2023-12-27 18:38:49,622][20381] Updated weights for policy 1, policy_version 1874855 (0.0008) [2023-12-27 18:38:49,855][20347] Updated weights for policy 0, policy_version 1876477 (0.0010) [2023-12-27 18:38:49,916][20347] Updated weights for policy 0, policy_version 1876487 (0.0010) [2023-12-27 18:38:49,984][20347] Updated weights for policy 0, policy_version 1876497 (0.0006) [2023-12-27 18:38:50,360][20381] Updated weights for policy 1, policy_version 1874865 (0.0008) [2023-12-27 18:38:50,409][20381] Updated weights for policy 1, policy_version 1874875 (0.0011) [2023-12-27 18:38:50,463][20381] Updated weights for policy 1, policy_version 1874885 (0.0010) [2023-12-27 18:38:50,518][20381] Updated weights for policy 1, policy_version 1874895 (0.0010) [2023-12-27 18:38:50,682][20347] Updated weights for policy 0, policy_version 1876507 (0.0007) [2023-12-27 18:38:50,740][20347] Updated weights for policy 0, policy_version 1876517 (0.0008) [2023-12-27 18:38:50,803][20347] Updated weights for policy 0, policy_version 1876527 (0.0007) [2023-12-27 18:38:51,167][20381] Updated weights for policy 1, policy_version 1874905 (0.0011) [2023-12-27 18:38:51,222][20381] Updated weights for policy 1, policy_version 1874915 (0.0010) [2023-12-27 18:38:51,289][20381] Updated weights for policy 1, policy_version 1874925 (0.0009) [2023-12-27 18:38:51,572][20347] Updated weights for policy 0, policy_version 1876537 (0.0007) [2023-12-27 18:38:51,643][20347] Updated weights for policy 0, policy_version 1876547 (0.0007) [2023-12-27 18:38:51,703][20347] Updated weights for policy 0, policy_version 1876557 (0.0009) [2023-12-27 18:38:51,769][20347] Updated weights for policy 0, policy_version 1876567 (0.0007) [2023-12-27 18:38:52,026][20381] Updated weights for policy 1, policy_version 1874935 (0.0008) [2023-12-27 18:38:52,081][20381] Updated weights for policy 1, policy_version 1874945 (0.0007) [2023-12-27 18:38:52,142][20381] Updated weights for policy 1, policy_version 1874955 (0.0008) [2023-12-27 18:38:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 960528384. Throughput: 0: 10192.4, 1: 10260.2. Samples: 960499796. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-12-27 18:38:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '857.460')] [2023-12-27 18:38:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001874960_480043008.pth... [2023-12-27 18:38:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001876568_480485376.pth... [2023-12-27 18:38:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001873744_479731712.pth [2023-12-27 18:38:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001875384_480182272.pth [2023-12-27 18:38:52,456][20347] Updated weights for policy 0, policy_version 1876577 (0.0008) [2023-12-27 18:38:52,515][20347] Updated weights for policy 0, policy_version 1876587 (0.0008) [2023-12-27 18:38:52,575][20347] Updated weights for policy 0, policy_version 1876597 (0.0008) [2023-12-27 18:38:52,860][20381] Updated weights for policy 1, policy_version 1874965 (0.0008) [2023-12-27 18:38:52,926][20381] Updated weights for policy 1, policy_version 1874975 (0.0011) [2023-12-27 18:38:52,988][20381] Updated weights for policy 1, policy_version 1874985 (0.0011) [2023-12-27 18:38:53,263][20347] Updated weights for policy 0, policy_version 1876607 (0.0008) [2023-12-27 18:38:53,327][20347] Updated weights for policy 0, policy_version 1876617 (0.0007) [2023-12-27 18:38:53,375][20347] Updated weights for policy 0, policy_version 1876627 (0.0007) [2023-12-27 18:38:53,666][20381] Updated weights for policy 1, policy_version 1874995 (0.0011) [2023-12-27 18:38:53,724][20381] Updated weights for policy 1, policy_version 1875005 (0.0011) [2023-12-27 18:38:53,775][20381] Updated weights for policy 1, policy_version 1875015 (0.0010) [2023-12-27 18:38:54,039][20347] Updated weights for policy 0, policy_version 1876637 (0.0009) [2023-12-27 18:38:54,096][20347] Updated weights for policy 0, policy_version 1876647 (0.0009) [2023-12-27 18:38:54,150][20347] Updated weights for policy 0, policy_version 1876657 (0.0005) [2023-12-27 18:38:54,358][20381] Updated weights for policy 1, policy_version 1875025 (0.0009) [2023-12-27 18:38:54,414][20381] Updated weights for policy 1, policy_version 1875035 (0.0008) [2023-12-27 18:38:54,472][20381] Updated weights for policy 1, policy_version 1875045 (0.0009) [2023-12-27 18:38:54,527][20381] Updated weights for policy 1, policy_version 1875055 (0.0007) [2023-12-27 18:38:54,765][20347] Updated weights for policy 0, policy_version 1876667 (0.0006) [2023-12-27 18:38:54,819][20347] Updated weights for policy 0, policy_version 1876677 (0.0007) [2023-12-27 18:38:54,876][20347] Updated weights for policy 0, policy_version 1876687 (0.0008) [2023-12-27 18:38:55,129][20381] Updated weights for policy 1, policy_version 1875065 (0.0005) [2023-12-27 18:38:55,189][20381] Updated weights for policy 1, policy_version 1875075 (0.0005) [2023-12-27 18:38:55,251][20381] Updated weights for policy 1, policy_version 1875085 (0.0005) [2023-12-27 18:38:55,562][20347] Updated weights for policy 0, policy_version 1876697 (0.0010) [2023-12-27 18:38:55,625][20347] Updated weights for policy 0, policy_version 1876707 (0.0010) [2023-12-27 18:38:55,690][20347] Updated weights for policy 0, policy_version 1876717 (0.0007) [2023-12-27 18:38:55,736][20347] Updated weights for policy 0, policy_version 1876727 (0.0009) [2023-12-27 18:38:55,807][20381] Updated weights for policy 1, policy_version 1875095 (0.0007) [2023-12-27 18:38:55,861][20381] Updated weights for policy 1, policy_version 1875105 (0.0007) [2023-12-27 18:38:55,920][20381] Updated weights for policy 1, policy_version 1875115 (0.0008) [2023-12-27 18:38:56,400][20347] Updated weights for policy 0, policy_version 1876737 (0.0008) [2023-12-27 18:38:56,452][20347] Updated weights for policy 0, policy_version 1876747 (0.0007) [2023-12-27 18:38:56,506][20347] Updated weights for policy 0, policy_version 1876757 (0.0007) [2023-12-27 18:38:56,559][20381] Updated weights for policy 1, policy_version 1875125 (0.0007) [2023-12-27 18:38:56,617][20381] Updated weights for policy 1, policy_version 1875135 (0.0008) [2023-12-27 18:38:56,665][20381] Updated weights for policy 1, policy_version 1875145 (0.0008) [2023-12-27 18:38:57,228][20347] Updated weights for policy 0, policy_version 1876767 (0.0007) [2023-12-27 18:38:57,264][20381] Updated weights for policy 1, policy_version 1875155 (0.0007) [2023-12-27 18:38:57,279][20347] Updated weights for policy 0, policy_version 1876777 (0.0008) [2023-12-27 18:38:57,313][20381] Updated weights for policy 1, policy_version 1875165 (0.0010) [2023-12-27 18:38:57,334][20347] Updated weights for policy 0, policy_version 1876787 (0.0006) [2023-12-27 18:38:57,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 960634880. Throughput: 0: 10225.5, 1: 10279.9. Samples: 960623524. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:38:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '914.460')] [2023-12-27 18:38:57,364][20381] Updated weights for policy 1, policy_version 1875175 (0.0005) [2023-12-27 18:38:58,017][20347] Updated weights for policy 0, policy_version 1876797 (0.0008) [2023-12-27 18:38:58,027][20381] Updated weights for policy 1, policy_version 1875185 (0.0006) [2023-12-27 18:38:58,065][20347] Updated weights for policy 0, policy_version 1876807 (0.0010) [2023-12-27 18:38:58,085][20381] Updated weights for policy 1, policy_version 1875195 (0.0005) [2023-12-27 18:38:58,110][20347] Updated weights for policy 0, policy_version 1876817 (0.0010) [2023-12-27 18:38:58,130][20381] Updated weights for policy 1, policy_version 1875205 (0.0005) [2023-12-27 18:38:58,182][20381] Updated weights for policy 1, policy_version 1875215 (0.0007) [2023-12-27 18:38:58,932][20347] Updated weights for policy 0, policy_version 1876827 (0.0010) [2023-12-27 18:38:58,976][20381] Updated weights for policy 1, policy_version 1875225 (0.0006) [2023-12-27 18:38:58,991][20347] Updated weights for policy 0, policy_version 1876837 (0.0008) [2023-12-27 18:38:59,033][20381] Updated weights for policy 1, policy_version 1875235 (0.0007) [2023-12-27 18:38:59,051][20347] Updated weights for policy 0, policy_version 1876847 (0.0007) [2023-12-27 18:38:59,089][20381] Updated weights for policy 1, policy_version 1875245 (0.0008) [2023-12-27 18:38:59,786][20347] Updated weights for policy 0, policy_version 1876857 (0.0008) [2023-12-27 18:38:59,846][20347] Updated weights for policy 0, policy_version 1876867 (0.0008) [2023-12-27 18:38:59,870][20381] Updated weights for policy 1, policy_version 1875255 (0.0007) [2023-12-27 18:38:59,898][20347] Updated weights for policy 0, policy_version 1876877 (0.0008) [2023-12-27 18:38:59,932][20381] Updated weights for policy 1, policy_version 1875265 (0.0009) [2023-12-27 18:38:59,960][20347] Updated weights for policy 0, policy_version 1876887 (0.0006) [2023-12-27 18:38:59,988][20381] Updated weights for policy 1, policy_version 1875275 (0.0010) [2023-12-27 18:39:00,565][20381] Updated weights for policy 1, policy_version 1875285 (0.0009) [2023-12-27 18:39:00,622][20347] Updated weights for policy 0, policy_version 1876897 (0.0007) [2023-12-27 18:39:00,623][20381] Updated weights for policy 1, policy_version 1875295 (0.0011) [2023-12-27 18:39:00,679][20347] Updated weights for policy 0, policy_version 1876907 (0.0006) [2023-12-27 18:39:00,682][20381] Updated weights for policy 1, policy_version 1875305 (0.0011) [2023-12-27 18:39:00,733][20347] Updated weights for policy 0, policy_version 1876917 (0.0007) [2023-12-27 18:39:01,336][20381] Updated weights for policy 1, policy_version 1875315 (0.0010) [2023-12-27 18:39:01,386][20347] Updated weights for policy 0, policy_version 1876927 (0.0007) [2023-12-27 18:39:01,398][20381] Updated weights for policy 1, policy_version 1875325 (0.0008) [2023-12-27 18:39:01,441][20347] Updated weights for policy 0, policy_version 1876937 (0.0008) [2023-12-27 18:39:01,447][20381] Updated weights for policy 1, policy_version 1875335 (0.0006) [2023-12-27 18:39:01,500][20347] Updated weights for policy 0, policy_version 1876947 (0.0008) [2023-12-27 18:39:02,147][20347] Updated weights for policy 0, policy_version 1876957 (0.0009) [2023-12-27 18:39:02,158][20381] Updated weights for policy 1, policy_version 1875345 (0.0007) [2023-12-27 18:39:02,200][20347] Updated weights for policy 0, policy_version 1876967 (0.0011) [2023-12-27 18:39:02,212][20381] Updated weights for policy 1, policy_version 1875355 (0.0006) [2023-12-27 18:39:02,256][20347] Updated weights for policy 0, policy_version 1876977 (0.0008) [2023-12-27 18:39:02,266][20381] Updated weights for policy 1, policy_version 1875365 (0.0007) [2023-12-27 18:39:02,319][20381] Updated weights for policy 1, policy_version 1875375 (0.0007) [2023-12-27 18:39:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 960741376. Throughput: 0: 10168.3, 1: 10331.3. Samples: 960744888. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '915.370')] [2023-12-27 18:39:02,942][20347] Updated weights for policy 0, policy_version 1876987 (0.0010) [2023-12-27 18:39:02,996][20381] Updated weights for policy 1, policy_version 1875385 (0.0007) [2023-12-27 18:39:02,997][20347] Updated weights for policy 0, policy_version 1876997 (0.0007) [2023-12-27 18:39:03,045][20381] Updated weights for policy 1, policy_version 1875395 (0.0007) [2023-12-27 18:39:03,051][20347] Updated weights for policy 0, policy_version 1877007 (0.0007) [2023-12-27 18:39:03,067][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000005 [2023-12-27 18:39:03,713][20347] Updated weights for policy 0, policy_version 1877017 (0.0007) [2023-12-27 18:39:03,740][20381] Updated weights for policy 1, policy_version 1875405 (0.0005) [2023-12-27 18:39:03,760][20347] Updated weights for policy 0, policy_version 1877027 (0.0006) [2023-12-27 18:39:03,782][20381] Updated weights for policy 1, policy_version 1875415 (0.0005) [2023-12-27 18:39:03,814][20347] Updated weights for policy 0, policy_version 1877037 (0.0007) [2023-12-27 18:39:03,833][20381] Updated weights for policy 1, policy_version 1875425 (0.0005) [2023-12-27 18:39:03,871][20347] Updated weights for policy 0, policy_version 1877047 (0.0006) [2023-12-27 18:39:04,531][20381] Updated weights for policy 1, policy_version 1875435 (0.0008) [2023-12-27 18:39:04,543][20347] Updated weights for policy 0, policy_version 1877057 (0.0007) [2023-12-27 18:39:04,577][20381] Updated weights for policy 1, policy_version 1875445 (0.0006) [2023-12-27 18:39:04,591][20347] Updated weights for policy 0, policy_version 1877067 (0.0008) [2023-12-27 18:39:04,637][20381] Updated weights for policy 1, policy_version 1875455 (0.0007) [2023-12-27 18:39:04,650][20347] Updated weights for policy 0, policy_version 1877077 (0.0005) [2023-12-27 18:39:05,301][20381] Updated weights for policy 1, policy_version 1875465 (0.0007) [2023-12-27 18:39:05,351][20347] Updated weights for policy 0, policy_version 1877087 (0.0007) [2023-12-27 18:39:05,359][20381] Updated weights for policy 1, policy_version 1875475 (0.0007) [2023-12-27 18:39:05,396][20347] Updated weights for policy 0, policy_version 1877097 (0.0007) [2023-12-27 18:39:05,407][20381] Updated weights for policy 1, policy_version 1875485 (0.0007) [2023-12-27 18:39:05,440][20347] Updated weights for policy 0, policy_version 1877107 (0.0008) [2023-12-27 18:39:05,462][20381] Updated weights for policy 1, policy_version 1875495 (0.0006) [2023-12-27 18:39:06,092][20347] Updated weights for policy 0, policy_version 1877117 (0.0006) [2023-12-27 18:39:06,163][20347] Updated weights for policy 0, policy_version 1877127 (0.0007) [2023-12-27 18:39:06,225][20347] Updated weights for policy 0, policy_version 1877137 (0.0007) [2023-12-27 18:39:06,228][20381] Updated weights for policy 1, policy_version 1875505 (0.0008) [2023-12-27 18:39:06,283][20381] Updated weights for policy 1, policy_version 1875515 (0.0008) [2023-12-27 18:39:06,341][20381] Updated weights for policy 1, policy_version 1875525 (0.0007) [2023-12-27 18:39:06,864][20347] Updated weights for policy 0, policy_version 1877147 (0.0009) [2023-12-27 18:39:06,916][20347] Updated weights for policy 0, policy_version 1877157 (0.0010) [2023-12-27 18:39:06,968][20347] Updated weights for policy 0, policy_version 1877167 (0.0007) [2023-12-27 18:39:07,028][20381] Updated weights for policy 1, policy_version 1875535 (0.0006) [2023-12-27 18:39:07,084][20381] Updated weights for policy 1, policy_version 1875545 (0.0006) [2023-12-27 18:39:07,139][20381] Updated weights for policy 1, policy_version 1875555 (0.0007) [2023-12-27 18:39:07,167][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000010 [2023-12-27 18:39:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 960839680. Throughput: 0: 10173.0, 1: 10373.4. Samples: 960807116. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.790')] [2023-12-27 18:39:07,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001875560_480198656.pth... [2023-12-27 18:39:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001877176_480641024.pth... [2023-12-27 18:39:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001874352_479887360.pth [2023-12-27 18:39:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001875992_480337920.pth [2023-12-27 18:39:07,600][20347] Updated weights for policy 0, policy_version 1877177 (0.0005) [2023-12-27 18:39:07,648][20347] Updated weights for policy 0, policy_version 1877187 (0.0007) [2023-12-27 18:39:07,703][20347] Updated weights for policy 0, policy_version 1877197 (0.0007) [2023-12-27 18:39:07,756][20347] Updated weights for policy 0, policy_version 1877207 (0.0006) [2023-12-27 18:39:07,762][20381] Updated weights for policy 1, policy_version 1875565 (0.0007) [2023-12-27 18:39:07,817][20381] Updated weights for policy 1, policy_version 1875575 (0.0007) [2023-12-27 18:39:07,877][20381] Updated weights for policy 1, policy_version 1875585 (0.0008) [2023-12-27 18:39:08,388][20347] Updated weights for policy 0, policy_version 1877217 (0.0007) [2023-12-27 18:39:08,453][20347] Updated weights for policy 0, policy_version 1877227 (0.0007) [2023-12-27 18:39:08,521][20347] Updated weights for policy 0, policy_version 1877237 (0.0008) [2023-12-27 18:39:08,617][20381] Updated weights for policy 1, policy_version 1875595 (0.0007) [2023-12-27 18:39:08,678][20381] Updated weights for policy 1, policy_version 1875605 (0.0011) [2023-12-27 18:39:08,731][20381] Updated weights for policy 1, policy_version 1875615 (0.0011) [2023-12-27 18:39:09,238][20347] Updated weights for policy 0, policy_version 1877247 (0.0008) [2023-12-27 18:39:09,296][20347] Updated weights for policy 0, policy_version 1877257 (0.0007) [2023-12-27 18:39:09,359][20347] Updated weights for policy 0, policy_version 1877267 (0.0008) [2023-12-27 18:39:09,435][20381] Updated weights for policy 1, policy_version 1875625 (0.0010) [2023-12-27 18:39:09,488][20381] Updated weights for policy 1, policy_version 1875635 (0.0007) [2023-12-27 18:39:09,542][20381] Updated weights for policy 1, policy_version 1875645 (0.0008) [2023-12-27 18:39:09,606][20381] Updated weights for policy 1, policy_version 1875655 (0.0006) [2023-12-27 18:39:10,126][20347] Updated weights for policy 0, policy_version 1877277 (0.0008) [2023-12-27 18:39:10,177][20347] Updated weights for policy 0, policy_version 1877287 (0.0008) [2023-12-27 18:39:10,239][20347] Updated weights for policy 0, policy_version 1877297 (0.0007) [2023-12-27 18:39:10,298][20381] Updated weights for policy 1, policy_version 1875665 (0.0008) [2023-12-27 18:39:10,356][20381] Updated weights for policy 1, policy_version 1875675 (0.0008) [2023-12-27 18:39:10,416][20381] Updated weights for policy 1, policy_version 1875685 (0.0010) [2023-12-27 18:39:10,902][20347] Updated weights for policy 0, policy_version 1877307 (0.0006) [2023-12-27 18:39:10,953][20347] Updated weights for policy 0, policy_version 1877317 (0.0008) [2023-12-27 18:39:11,009][20347] Updated weights for policy 0, policy_version 1877327 (0.0007) [2023-12-27 18:39:11,044][20381] Updated weights for policy 1, policy_version 1875695 (0.0011) [2023-12-27 18:39:11,100][20381] Updated weights for policy 1, policy_version 1875705 (0.0009) [2023-12-27 18:39:11,165][20381] Updated weights for policy 1, policy_version 1875715 (0.0009) [2023-12-27 18:39:11,865][20347] Updated weights for policy 0, policy_version 1877337 (0.0007) [2023-12-27 18:39:11,905][20381] Updated weights for policy 1, policy_version 1875725 (0.0011) [2023-12-27 18:39:11,926][20347] Updated weights for policy 0, policy_version 1877347 (0.0007) [2023-12-27 18:39:11,968][20381] Updated weights for policy 1, policy_version 1875735 (0.0011) [2023-12-27 18:39:11,984][20347] Updated weights for policy 0, policy_version 1877357 (0.0009) [2023-12-27 18:39:12,028][20381] Updated weights for policy 1, policy_version 1875745 (0.0011) [2023-12-27 18:39:12,034][20347] Updated weights for policy 0, policy_version 1877367 (0.0005) [2023-12-27 18:39:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 960937984. Throughput: 0: 10182.5, 1: 10378.6. Samples: 960928564. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.320')] [2023-12-27 18:39:12,727][20381] Updated weights for policy 1, policy_version 1875755 (0.0010) [2023-12-27 18:39:12,749][20347] Updated weights for policy 0, policy_version 1877377 (0.0006) [2023-12-27 18:39:12,786][20381] Updated weights for policy 1, policy_version 1875765 (0.0011) [2023-12-27 18:39:12,799][20347] Updated weights for policy 0, policy_version 1877387 (0.0005) [2023-12-27 18:39:12,842][20381] Updated weights for policy 1, policy_version 1875775 (0.0011) [2023-12-27 18:39:12,848][20347] Updated weights for policy 0, policy_version 1877397 (0.0006) [2023-12-27 18:39:13,538][20381] Updated weights for policy 1, policy_version 1875785 (0.0012) [2023-12-27 18:39:13,540][20347] Updated weights for policy 0, policy_version 1877407 (0.0009) [2023-12-27 18:39:13,586][20347] Updated weights for policy 0, policy_version 1877417 (0.0006) [2023-12-27 18:39:13,591][20381] Updated weights for policy 1, policy_version 1875795 (0.0011) [2023-12-27 18:39:13,634][20381] Updated weights for policy 1, policy_version 1875805 (0.0010) [2023-12-27 18:39:13,640][20347] Updated weights for policy 0, policy_version 1877427 (0.0005) [2023-12-27 18:39:13,692][20381] Updated weights for policy 1, policy_version 1875815 (0.0011) [2023-12-27 18:39:14,225][20347] Updated weights for policy 0, policy_version 1877437 (0.0010) [2023-12-27 18:39:14,273][20347] Updated weights for policy 0, policy_version 1877447 (0.0010) [2023-12-27 18:39:14,331][20347] Updated weights for policy 0, policy_version 1877457 (0.0010) [2023-12-27 18:39:14,394][20381] Updated weights for policy 1, policy_version 1875825 (0.0010) [2023-12-27 18:39:14,449][20381] Updated weights for policy 1, policy_version 1875835 (0.0011) [2023-12-27 18:39:14,507][20381] Updated weights for policy 1, policy_version 1875845 (0.0011) [2023-12-27 18:39:15,040][20347] Updated weights for policy 0, policy_version 1877467 (0.0007) [2023-12-27 18:39:15,095][20347] Updated weights for policy 0, policy_version 1877477 (0.0005) [2023-12-27 18:39:15,157][20347] Updated weights for policy 0, policy_version 1877487 (0.0007) [2023-12-27 18:39:15,170][20381] Updated weights for policy 1, policy_version 1875855 (0.0011) [2023-12-27 18:39:15,222][20381] Updated weights for policy 1, policy_version 1875865 (0.0011) [2023-12-27 18:39:15,278][20381] Updated weights for policy 1, policy_version 1875875 (0.0011) [2023-12-27 18:39:15,777][20347] Updated weights for policy 0, policy_version 1877497 (0.0009) [2023-12-27 18:39:15,837][20347] Updated weights for policy 0, policy_version 1877507 (0.0007) [2023-12-27 18:39:15,893][20347] Updated weights for policy 0, policy_version 1877517 (0.0007) [2023-12-27 18:39:15,947][20347] Updated weights for policy 0, policy_version 1877527 (0.0007) [2023-12-27 18:39:15,982][20381] Updated weights for policy 1, policy_version 1875885 (0.0010) [2023-12-27 18:39:16,040][20381] Updated weights for policy 1, policy_version 1875895 (0.0010) [2023-12-27 18:39:16,094][20381] Updated weights for policy 1, policy_version 1875905 (0.0010) [2023-12-27 18:39:16,573][20347] Updated weights for policy 0, policy_version 1877537 (0.0007) [2023-12-27 18:39:16,624][20347] Updated weights for policy 0, policy_version 1877547 (0.0007) [2023-12-27 18:39:16,684][20347] Updated weights for policy 0, policy_version 1877557 (0.0007) [2023-12-27 18:39:16,792][20381] Updated weights for policy 1, policy_version 1875915 (0.0009) [2023-12-27 18:39:16,839][20381] Updated weights for policy 1, policy_version 1875925 (0.0007) [2023-12-27 18:39:16,889][20381] Updated weights for policy 1, policy_version 1875935 (0.0008) [2023-12-27 18:39:17,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 961036288. Throughput: 0: 10158.2, 1: 10293.6. Samples: 961049896. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '878.750')] [2023-12-27 18:39:17,364][20347] Updated weights for policy 0, policy_version 1877567 (0.0007) [2023-12-27 18:39:17,418][20347] Updated weights for policy 0, policy_version 1877577 (0.0007) [2023-12-27 18:39:17,478][20347] Updated weights for policy 0, policy_version 1877587 (0.0007) [2023-12-27 18:39:17,569][20381] Updated weights for policy 1, policy_version 1875945 (0.0007) [2023-12-27 18:39:17,630][20381] Updated weights for policy 1, policy_version 1875955 (0.0011) [2023-12-27 18:39:17,690][20381] Updated weights for policy 1, policy_version 1875965 (0.0010) [2023-12-27 18:39:17,739][20381] Updated weights for policy 1, policy_version 1875975 (0.0011) [2023-12-27 18:39:18,161][20347] Updated weights for policy 0, policy_version 1877597 (0.0007) [2023-12-27 18:39:18,217][20347] Updated weights for policy 0, policy_version 1877607 (0.0007) [2023-12-27 18:39:18,267][20347] Updated weights for policy 0, policy_version 1877617 (0.0007) [2023-12-27 18:39:18,478][20381] Updated weights for policy 1, policy_version 1875985 (0.0008) [2023-12-27 18:39:18,536][20381] Updated weights for policy 1, policy_version 1875995 (0.0007) [2023-12-27 18:39:18,590][20381] Updated weights for policy 1, policy_version 1876005 (0.0007) [2023-12-27 18:39:19,028][20347] Updated weights for policy 0, policy_version 1877627 (0.0007) [2023-12-27 18:39:19,080][20347] Updated weights for policy 0, policy_version 1877637 (0.0008) [2023-12-27 18:39:19,138][20347] Updated weights for policy 0, policy_version 1877647 (0.0007) [2023-12-27 18:39:19,334][20381] Updated weights for policy 1, policy_version 1876015 (0.0008) [2023-12-27 18:39:19,402][20381] Updated weights for policy 1, policy_version 1876025 (0.0008) [2023-12-27 18:39:19,462][20381] Updated weights for policy 1, policy_version 1876035 (0.0006) [2023-12-27 18:39:19,910][20347] Updated weights for policy 0, policy_version 1877657 (0.0007) [2023-12-27 18:39:19,974][20347] Updated weights for policy 0, policy_version 1877667 (0.0007) [2023-12-27 18:39:20,035][20347] Updated weights for policy 0, policy_version 1877677 (0.0006) [2023-12-27 18:39:20,084][20347] Updated weights for policy 0, policy_version 1877687 (0.0005) [2023-12-27 18:39:20,140][20381] Updated weights for policy 1, policy_version 1876045 (0.0008) [2023-12-27 18:39:20,193][20381] Updated weights for policy 1, policy_version 1876055 (0.0007) [2023-12-27 18:39:20,255][20381] Updated weights for policy 1, policy_version 1876065 (0.0008) [2023-12-27 18:39:20,776][20347] Updated weights for policy 0, policy_version 1877697 (0.0008) [2023-12-27 18:39:20,832][20347] Updated weights for policy 0, policy_version 1877707 (0.0007) [2023-12-27 18:39:20,890][20347] Updated weights for policy 0, policy_version 1877717 (0.0009) [2023-12-27 18:39:20,947][20381] Updated weights for policy 1, policy_version 1876075 (0.0009) [2023-12-27 18:39:21,008][20381] Updated weights for policy 1, policy_version 1876085 (0.0011) [2023-12-27 18:39:21,076][20381] Updated weights for policy 1, policy_version 1876095 (0.0009) [2023-12-27 18:39:21,610][20347] Updated weights for policy 0, policy_version 1877727 (0.0006) [2023-12-27 18:39:21,679][20347] Updated weights for policy 0, policy_version 1877737 (0.0007) [2023-12-27 18:39:21,756][20347] Updated weights for policy 0, policy_version 1877748 (0.0007) [2023-12-27 18:39:21,790][20381] Updated weights for policy 1, policy_version 1876105 (0.0007) [2023-12-27 18:39:21,851][20381] Updated weights for policy 1, policy_version 1876115 (0.0006) [2023-12-27 18:39:21,908][20381] Updated weights for policy 1, policy_version 1876125 (0.0007) [2023-12-27 18:39:21,966][20381] Updated weights for policy 1, policy_version 1876135 (0.0008) [2023-12-27 18:39:22,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 961134592. Throughput: 0: 10130.2, 1: 10208.9. Samples: 961107916. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.650')] [2023-12-27 18:39:22,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001876136_480346112.pth... [2023-12-27 18:39:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001877752_480788480.pth... [2023-12-27 18:39:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001874960_480043008.pth [2023-12-27 18:39:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001876568_480485376.pth [2023-12-27 18:39:22,484][20347] Updated weights for policy 0, policy_version 1877758 (0.0008) [2023-12-27 18:39:22,538][20347] Updated weights for policy 0, policy_version 1877768 (0.0008) [2023-12-27 18:39:22,588][20347] Updated weights for policy 0, policy_version 1877778 (0.0007) [2023-12-27 18:39:22,675][20381] Updated weights for policy 1, policy_version 1876145 (0.0008) [2023-12-27 18:39:22,731][20381] Updated weights for policy 1, policy_version 1876155 (0.0008) [2023-12-27 18:39:22,789][20381] Updated weights for policy 1, policy_version 1876165 (0.0008) [2023-12-27 18:39:23,312][20347] Updated weights for policy 0, policy_version 1877788 (0.0007) [2023-12-27 18:39:23,364][20347] Updated weights for policy 0, policy_version 1877798 (0.0007) [2023-12-27 18:39:23,416][20347] Updated weights for policy 0, policy_version 1877808 (0.0008) [2023-12-27 18:39:23,440][20381] Updated weights for policy 1, policy_version 1876175 (0.0007) [2023-12-27 18:39:23,493][20381] Updated weights for policy 1, policy_version 1876185 (0.0007) [2023-12-27 18:39:23,556][20381] Updated weights for policy 1, policy_version 1876195 (0.0008) [2023-12-27 18:39:24,081][20347] Updated weights for policy 0, policy_version 1877818 (0.0007) [2023-12-27 18:39:24,147][20347] Updated weights for policy 0, policy_version 1877828 (0.0008) [2023-12-27 18:39:24,207][20347] Updated weights for policy 0, policy_version 1877838 (0.0008) [2023-12-27 18:39:24,232][20381] Updated weights for policy 1, policy_version 1876205 (0.0007) [2023-12-27 18:39:24,264][20347] Updated weights for policy 0, policy_version 1877848 (0.0006) [2023-12-27 18:39:24,289][20381] Updated weights for policy 1, policy_version 1876215 (0.0007) [2023-12-27 18:39:24,335][20381] Updated weights for policy 1, policy_version 1876225 (0.0007) [2023-12-27 18:39:24,936][20347] Updated weights for policy 0, policy_version 1877858 (0.0008) [2023-12-27 18:39:24,990][20347] Updated weights for policy 0, policy_version 1877868 (0.0007) [2023-12-27 18:39:25,041][20347] Updated weights for policy 0, policy_version 1877878 (0.0007) [2023-12-27 18:39:25,051][20381] Updated weights for policy 1, policy_version 1876235 (0.0006) [2023-12-27 18:39:25,109][20381] Updated weights for policy 1, policy_version 1876245 (0.0006) [2023-12-27 18:39:25,170][20381] Updated weights for policy 1, policy_version 1876255 (0.0006) [2023-12-27 18:39:25,732][20347] Updated weights for policy 0, policy_version 1877888 (0.0007) [2023-12-27 18:39:25,782][20347] Updated weights for policy 0, policy_version 1877898 (0.0007) [2023-12-27 18:39:25,833][20347] Updated weights for policy 0, policy_version 1877908 (0.0007) [2023-12-27 18:39:25,848][20381] Updated weights for policy 1, policy_version 1876265 (0.0007) [2023-12-27 18:39:25,902][20381] Updated weights for policy 1, policy_version 1876275 (0.0007) [2023-12-27 18:39:25,957][20381] Updated weights for policy 1, policy_version 1876285 (0.0008) [2023-12-27 18:39:26,016][20381] Updated weights for policy 1, policy_version 1876295 (0.0009) [2023-12-27 18:39:26,484][20347] Updated weights for policy 0, policy_version 1877918 (0.0006) [2023-12-27 18:39:26,532][20347] Updated weights for policy 0, policy_version 1877928 (0.0005) [2023-12-27 18:39:26,580][20347] Updated weights for policy 0, policy_version 1877938 (0.0005) [2023-12-27 18:39:26,745][20381] Updated weights for policy 1, policy_version 1876305 (0.0006) [2023-12-27 18:39:26,802][20381] Updated weights for policy 1, policy_version 1876315 (0.0005) [2023-12-27 18:39:26,858][20381] Updated weights for policy 1, policy_version 1876325 (0.0006) [2023-12-27 18:39:27,172][20347] Updated weights for policy 0, policy_version 1877948 (0.0006) [2023-12-27 18:39:27,220][20347] Updated weights for policy 0, policy_version 1877958 (0.0007) [2023-12-27 18:39:27,276][20347] Updated weights for policy 0, policy_version 1877968 (0.0007) [2023-12-27 18:39:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 961241088. Throughput: 0: 10101.5, 1: 10154.6. Samples: 961227700. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.460')] [2023-12-27 18:39:27,473][20381] Updated weights for policy 1, policy_version 1876335 (0.0009) [2023-12-27 18:39:27,524][20381] Updated weights for policy 1, policy_version 1876345 (0.0008) [2023-12-27 18:39:27,576][20381] Updated weights for policy 1, policy_version 1876355 (0.0007) [2023-12-27 18:39:27,844][20347] Updated weights for policy 0, policy_version 1877978 (0.0008) [2023-12-27 18:39:27,898][20347] Updated weights for policy 0, policy_version 1877988 (0.0007) [2023-12-27 18:39:27,953][20347] Updated weights for policy 0, policy_version 1877998 (0.0005) [2023-12-27 18:39:28,016][20347] Updated weights for policy 0, policy_version 1878008 (0.0005) [2023-12-27 18:39:28,285][20381] Updated weights for policy 1, policy_version 1876365 (0.0009) [2023-12-27 18:39:28,333][20381] Updated weights for policy 1, policy_version 1876375 (0.0009) [2023-12-27 18:39:28,397][20381] Updated weights for policy 1, policy_version 1876385 (0.0007) [2023-12-27 18:39:28,592][20347] Updated weights for policy 0, policy_version 1878018 (0.0008) [2023-12-27 18:39:28,637][20347] Updated weights for policy 0, policy_version 1878028 (0.0007) [2023-12-27 18:39:28,694][20347] Updated weights for policy 0, policy_version 1878038 (0.0009) [2023-12-27 18:39:28,977][20381] Updated weights for policy 1, policy_version 1876395 (0.0006) [2023-12-27 18:39:29,028][20381] Updated weights for policy 1, policy_version 1876405 (0.0008) [2023-12-27 18:39:29,087][20381] Updated weights for policy 1, policy_version 1876415 (0.0007) [2023-12-27 18:39:29,422][20347] Updated weights for policy 0, policy_version 1878048 (0.0008) [2023-12-27 18:39:29,478][20347] Updated weights for policy 0, policy_version 1878058 (0.0008) [2023-12-27 18:39:29,537][20347] Updated weights for policy 0, policy_version 1878068 (0.0008) [2023-12-27 18:39:29,852][20381] Updated weights for policy 1, policy_version 1876425 (0.0007) [2023-12-27 18:39:29,913][20381] Updated weights for policy 1, policy_version 1876435 (0.0007) [2023-12-27 18:39:29,977][20381] Updated weights for policy 1, policy_version 1876445 (0.0008) [2023-12-27 18:39:30,035][20381] Updated weights for policy 1, policy_version 1876455 (0.0008) [2023-12-27 18:39:30,193][20347] Updated weights for policy 0, policy_version 1878078 (0.0008) [2023-12-27 18:39:30,243][20347] Updated weights for policy 0, policy_version 1878088 (0.0005) [2023-12-27 18:39:30,300][20347] Updated weights for policy 0, policy_version 1878098 (0.0005) [2023-12-27 18:39:30,743][20381] Updated weights for policy 1, policy_version 1876465 (0.0006) [2023-12-27 18:39:30,804][20381] Updated weights for policy 1, policy_version 1876475 (0.0010) [2023-12-27 18:39:30,865][20381] Updated weights for policy 1, policy_version 1876485 (0.0010) [2023-12-27 18:39:30,881][20347] Updated weights for policy 0, policy_version 1878108 (0.0007) [2023-12-27 18:39:30,932][20347] Updated weights for policy 0, policy_version 1878118 (0.0010) [2023-12-27 18:39:30,986][20347] Updated weights for policy 0, policy_version 1878128 (0.0010) [2023-12-27 18:39:31,499][20381] Updated weights for policy 1, policy_version 1876495 (0.0008) [2023-12-27 18:39:31,547][20381] Updated weights for policy 1, policy_version 1876505 (0.0007) [2023-12-27 18:39:31,599][20381] Updated weights for policy 1, policy_version 1876515 (0.0008) [2023-12-27 18:39:31,717][20347] Updated weights for policy 0, policy_version 1878138 (0.0010) [2023-12-27 18:39:31,775][20347] Updated weights for policy 0, policy_version 1878148 (0.0008) [2023-12-27 18:39:31,822][20347] Updated weights for policy 0, policy_version 1878158 (0.0007) [2023-12-27 18:39:31,870][20347] Updated weights for policy 0, policy_version 1878168 (0.0008) [2023-12-27 18:39:32,292][20381] Updated weights for policy 1, policy_version 1876525 (0.0009) [2023-12-27 18:39:32,345][20381] Updated weights for policy 1, policy_version 1876535 (0.0011) [2023-12-27 18:39:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 961339392. Throughput: 0: 10173.7, 1: 10156.5. Samples: 961354380. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '875.560')] [2023-12-27 18:39:32,410][20381] Updated weights for policy 1, policy_version 1876545 (0.0011) [2023-12-27 18:39:32,593][20347] Updated weights for policy 0, policy_version 1878178 (0.0008) [2023-12-27 18:39:32,648][20347] Updated weights for policy 0, policy_version 1878188 (0.0007) [2023-12-27 18:39:32,707][20347] Updated weights for policy 0, policy_version 1878198 (0.0005) [2023-12-27 18:39:33,124][20381] Updated weights for policy 1, policy_version 1876555 (0.0010) [2023-12-27 18:39:33,179][20381] Updated weights for policy 1, policy_version 1876565 (0.0010) [2023-12-27 18:39:33,231][20381] Updated weights for policy 1, policy_version 1876575 (0.0011) [2023-12-27 18:39:33,384][20347] Updated weights for policy 0, policy_version 1878208 (0.0008) [2023-12-27 18:39:33,433][20347] Updated weights for policy 0, policy_version 1878218 (0.0008) [2023-12-27 18:39:33,495][20347] Updated weights for policy 0, policy_version 1878228 (0.0007) [2023-12-27 18:39:33,812][20381] Updated weights for policy 1, policy_version 1876585 (0.0010) [2023-12-27 18:39:33,869][20381] Updated weights for policy 1, policy_version 1876595 (0.0007) [2023-12-27 18:39:33,921][20381] Updated weights for policy 1, policy_version 1876605 (0.0007) [2023-12-27 18:39:33,978][20381] Updated weights for policy 1, policy_version 1876615 (0.0007) [2023-12-27 18:39:34,200][20347] Updated weights for policy 0, policy_version 1878238 (0.0010) [2023-12-27 18:39:34,256][20347] Updated weights for policy 0, policy_version 1878248 (0.0008) [2023-12-27 18:39:34,317][20347] Updated weights for policy 0, policy_version 1878258 (0.0010) [2023-12-27 18:39:34,667][20381] Updated weights for policy 1, policy_version 1876625 (0.0005) [2023-12-27 18:39:34,725][20381] Updated weights for policy 1, policy_version 1876635 (0.0006) [2023-12-27 18:39:34,783][20381] Updated weights for policy 1, policy_version 1876645 (0.0008) [2023-12-27 18:39:34,946][20347] Updated weights for policy 0, policy_version 1878268 (0.0007) [2023-12-27 18:39:34,995][20347] Updated weights for policy 0, policy_version 1878278 (0.0006) [2023-12-27 18:39:35,043][20347] Updated weights for policy 0, policy_version 1878288 (0.0008) [2023-12-27 18:39:35,417][20381] Updated weights for policy 1, policy_version 1876655 (0.0009) [2023-12-27 18:39:35,462][20381] Updated weights for policy 1, policy_version 1876665 (0.0007) [2023-12-27 18:39:35,517][20381] Updated weights for policy 1, policy_version 1876675 (0.0008) [2023-12-27 18:39:35,704][20347] Updated weights for policy 0, policy_version 1878298 (0.0007) [2023-12-27 18:39:35,753][20347] Updated weights for policy 0, policy_version 1878308 (0.0008) [2023-12-27 18:39:35,803][20347] Updated weights for policy 0, policy_version 1878318 (0.0008) [2023-12-27 18:39:35,854][20347] Updated weights for policy 0, policy_version 1878328 (0.0007) [2023-12-27 18:39:36,244][20381] Updated weights for policy 1, policy_version 1876685 (0.0008) [2023-12-27 18:39:36,305][20381] Updated weights for policy 1, policy_version 1876695 (0.0008) [2023-12-27 18:39:36,359][20381] Updated weights for policy 1, policy_version 1876705 (0.0008) [2023-12-27 18:39:36,562][20347] Updated weights for policy 0, policy_version 1878338 (0.0011) [2023-12-27 18:39:36,614][20347] Updated weights for policy 0, policy_version 1878348 (0.0010) [2023-12-27 18:39:36,674][20347] Updated weights for policy 0, policy_version 1878358 (0.0010) [2023-12-27 18:39:37,076][20381] Updated weights for policy 1, policy_version 1876715 (0.0007) [2023-12-27 18:39:37,135][20381] Updated weights for policy 1, policy_version 1876725 (0.0008) [2023-12-27 18:39:37,192][20381] Updated weights for policy 1, policy_version 1876735 (0.0007) [2023-12-27 18:39:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 961445888. Throughput: 0: 10193.8, 1: 10183.9. Samples: 961416792. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '895.690')] [2023-12-27 18:39:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001876744_480501760.pth... [2023-12-27 18:39:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001875560_480198656.pth [2023-12-27 18:39:37,375][20347] Updated weights for policy 0, policy_version 1878368 (0.0010) [2023-12-27 18:39:37,433][20347] Updated weights for policy 0, policy_version 1878378 (0.0010) [2023-12-27 18:39:37,493][20347] Updated weights for policy 0, policy_version 1878388 (0.0010) [2023-12-27 18:39:37,520][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001878392_480952320.pth... [2023-12-27 18:39:37,525][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001877176_480641024.pth [2023-12-27 18:39:37,842][20381] Updated weights for policy 1, policy_version 1876745 (0.0005) [2023-12-27 18:39:37,891][20381] Updated weights for policy 1, policy_version 1876755 (0.0007) [2023-12-27 18:39:37,940][20381] Updated weights for policy 1, policy_version 1876765 (0.0007) [2023-12-27 18:39:37,999][20381] Updated weights for policy 1, policy_version 1876775 (0.0007) [2023-12-27 18:39:38,149][20347] Updated weights for policy 0, policy_version 1878398 (0.0010) [2023-12-27 18:39:38,200][20347] Updated weights for policy 0, policy_version 1878408 (0.0010) [2023-12-27 18:39:38,248][20347] Updated weights for policy 0, policy_version 1878418 (0.0010) [2023-12-27 18:39:38,739][20381] Updated weights for policy 1, policy_version 1876785 (0.0007) [2023-12-27 18:39:38,795][20381] Updated weights for policy 1, policy_version 1876795 (0.0008) [2023-12-27 18:39:38,848][20381] Updated weights for policy 1, policy_version 1876805 (0.0008) [2023-12-27 18:39:38,936][20347] Updated weights for policy 0, policy_version 1878428 (0.0010) [2023-12-27 18:39:38,997][20347] Updated weights for policy 0, policy_version 1878438 (0.0010) [2023-12-27 18:39:39,052][20347] Updated weights for policy 0, policy_version 1878448 (0.0010) [2023-12-27 18:39:39,512][20381] Updated weights for policy 1, policy_version 1876815 (0.0007) [2023-12-27 18:39:39,567][20381] Updated weights for policy 1, policy_version 1876825 (0.0008) [2023-12-27 18:39:39,624][20381] Updated weights for policy 1, policy_version 1876835 (0.0008) [2023-12-27 18:39:39,786][20347] Updated weights for policy 0, policy_version 1878458 (0.0010) [2023-12-27 18:39:39,856][20347] Updated weights for policy 0, policy_version 1878468 (0.0011) [2023-12-27 18:39:39,924][20347] Updated weights for policy 0, policy_version 1878478 (0.0010) [2023-12-27 18:39:39,991][20347] Updated weights for policy 0, policy_version 1878488 (0.0010) [2023-12-27 18:39:40,345][20381] Updated weights for policy 1, policy_version 1876845 (0.0007) [2023-12-27 18:39:40,402][20381] Updated weights for policy 1, policy_version 1876855 (0.0008) [2023-12-27 18:39:40,451][20381] Updated weights for policy 1, policy_version 1876865 (0.0007) [2023-12-27 18:39:40,688][20347] Updated weights for policy 0, policy_version 1878498 (0.0010) [2023-12-27 18:39:40,750][20347] Updated weights for policy 0, policy_version 1878508 (0.0010) [2023-12-27 18:39:40,808][20347] Updated weights for policy 0, policy_version 1878518 (0.0010) [2023-12-27 18:39:41,179][20381] Updated weights for policy 1, policy_version 1876875 (0.0007) [2023-12-27 18:39:41,230][20381] Updated weights for policy 1, policy_version 1876885 (0.0008) [2023-12-27 18:39:41,296][20381] Updated weights for policy 1, policy_version 1876895 (0.0010) [2023-12-27 18:39:41,533][20347] Updated weights for policy 0, policy_version 1878528 (0.0008) [2023-12-27 18:39:41,598][20347] Updated weights for policy 0, policy_version 1878538 (0.0009) [2023-12-27 18:39:41,660][20347] Updated weights for policy 0, policy_version 1878548 (0.0008) [2023-12-27 18:39:42,034][20381] Updated weights for policy 1, policy_version 1876905 (0.0009) [2023-12-27 18:39:42,086][20381] Updated weights for policy 1, policy_version 1876915 (0.0011) [2023-12-27 18:39:42,151][20381] Updated weights for policy 1, policy_version 1876925 (0.0011) [2023-12-27 18:39:42,212][20381] Updated weights for policy 1, policy_version 1876935 (0.0008) [2023-12-27 18:39:42,285][20347] Updated weights for policy 0, policy_version 1878558 (0.0007) [2023-12-27 18:39:42,354][20347] Updated weights for policy 0, policy_version 1878568 (0.0008) [2023-12-27 18:39:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 961544192. Throughput: 0: 10176.6, 1: 10100.5. Samples: 961535996. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.860')] [2023-12-27 18:39:42,410][20347] Updated weights for policy 0, policy_version 1878578 (0.0007) [2023-12-27 18:39:42,925][20381] Updated weights for policy 1, policy_version 1876945 (0.0008) [2023-12-27 18:39:42,981][20381] Updated weights for policy 1, policy_version 1876955 (0.0007) [2023-12-27 18:39:43,042][20381] Updated weights for policy 1, policy_version 1876965 (0.0008) [2023-12-27 18:39:43,138][20347] Updated weights for policy 0, policy_version 1878588 (0.0008) [2023-12-27 18:39:43,187][20347] Updated weights for policy 0, policy_version 1878598 (0.0007) [2023-12-27 18:39:43,239][20347] Updated weights for policy 0, policy_version 1878608 (0.0006) [2023-12-27 18:39:43,711][20381] Updated weights for policy 1, policy_version 1876975 (0.0007) [2023-12-27 18:39:43,771][20381] Updated weights for policy 1, policy_version 1876985 (0.0008) [2023-12-27 18:39:43,830][20381] Updated weights for policy 1, policy_version 1876995 (0.0007) [2023-12-27 18:39:43,885][20347] Updated weights for policy 0, policy_version 1878618 (0.0007) [2023-12-27 18:39:43,942][20347] Updated weights for policy 0, policy_version 1878628 (0.0007) [2023-12-27 18:39:43,999][20347] Updated weights for policy 0, policy_version 1878638 (0.0008) [2023-12-27 18:39:44,048][20347] Updated weights for policy 0, policy_version 1878648 (0.0008) [2023-12-27 18:39:44,421][20381] Updated weights for policy 1, policy_version 1877005 (0.0007) [2023-12-27 18:39:44,478][20381] Updated weights for policy 1, policy_version 1877015 (0.0005) [2023-12-27 18:39:44,536][20381] Updated weights for policy 1, policy_version 1877025 (0.0007) [2023-12-27 18:39:44,766][20347] Updated weights for policy 0, policy_version 1878658 (0.0008) [2023-12-27 18:39:44,829][20347] Updated weights for policy 0, policy_version 1878668 (0.0009) [2023-12-27 18:39:44,888][20347] Updated weights for policy 0, policy_version 1878678 (0.0010) [2023-12-27 18:39:45,165][20381] Updated weights for policy 1, policy_version 1877035 (0.0007) [2023-12-27 18:39:45,226][20381] Updated weights for policy 1, policy_version 1877045 (0.0011) [2023-12-27 18:39:45,288][20381] Updated weights for policy 1, policy_version 1877055 (0.0010) [2023-12-27 18:39:45,600][20347] Updated weights for policy 0, policy_version 1878688 (0.0007) [2023-12-27 18:39:45,658][20347] Updated weights for policy 0, policy_version 1878698 (0.0007) [2023-12-27 18:39:45,712][20347] Updated weights for policy 0, policy_version 1878708 (0.0007) [2023-12-27 18:39:45,917][20381] Updated weights for policy 1, policy_version 1877065 (0.0008) [2023-12-27 18:39:45,964][20381] Updated weights for policy 1, policy_version 1877075 (0.0010) [2023-12-27 18:39:46,017][20381] Updated weights for policy 1, policy_version 1877085 (0.0011) [2023-12-27 18:39:46,078][20381] Updated weights for policy 1, policy_version 1877095 (0.0011) [2023-12-27 18:39:46,388][20347] Updated weights for policy 0, policy_version 1878718 (0.0007) [2023-12-27 18:39:46,441][20347] Updated weights for policy 0, policy_version 1878728 (0.0007) [2023-12-27 18:39:46,498][20347] Updated weights for policy 0, policy_version 1878738 (0.0007) [2023-12-27 18:39:46,753][20381] Updated weights for policy 1, policy_version 1877105 (0.0011) [2023-12-27 18:39:46,812][20381] Updated weights for policy 1, policy_version 1877115 (0.0006) [2023-12-27 18:39:46,873][20381] Updated weights for policy 1, policy_version 1877125 (0.0011) [2023-12-27 18:39:47,132][20347] Updated weights for policy 0, policy_version 1878748 (0.0009) [2023-12-27 18:39:47,194][20347] Updated weights for policy 0, policy_version 1878758 (0.0011) [2023-12-27 18:39:47,256][20347] Updated weights for policy 0, policy_version 1878768 (0.0011) [2023-12-27 18:39:47,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.6, 300 sec: 20549.4). Total num frames: 961650688. Throughput: 0: 10204.7, 1: 10113.1. Samples: 961659188. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '865.310')] [2023-12-27 18:39:47,504][20381] Updated weights for policy 1, policy_version 1877135 (0.0007) [2023-12-27 18:39:47,565][20381] Updated weights for policy 1, policy_version 1877145 (0.0007) [2023-12-27 18:39:47,623][20381] Updated weights for policy 1, policy_version 1877155 (0.0008) [2023-12-27 18:39:47,901][20347] Updated weights for policy 0, policy_version 1878778 (0.0010) [2023-12-27 18:39:47,965][20347] Updated weights for policy 0, policy_version 1878788 (0.0009) [2023-12-27 18:39:48,020][20347] Updated weights for policy 0, policy_version 1878798 (0.0007) [2023-12-27 18:39:48,079][20347] Updated weights for policy 0, policy_version 1878808 (0.0008) [2023-12-27 18:39:48,240][20381] Updated weights for policy 1, policy_version 1877165 (0.0007) [2023-12-27 18:39:48,300][20381] Updated weights for policy 1, policy_version 1877175 (0.0008) [2023-12-27 18:39:48,365][20381] Updated weights for policy 1, policy_version 1877185 (0.0009) [2023-12-27 18:39:48,790][20347] Updated weights for policy 0, policy_version 1878818 (0.0007) [2023-12-27 18:39:48,848][20347] Updated weights for policy 0, policy_version 1878828 (0.0007) [2023-12-27 18:39:48,903][20347] Updated weights for policy 0, policy_version 1878838 (0.0007) [2023-12-27 18:39:49,042][20381] Updated weights for policy 1, policy_version 1877195 (0.0009) [2023-12-27 18:39:49,101][20381] Updated weights for policy 1, policy_version 1877205 (0.0011) [2023-12-27 18:39:49,179][20381] Updated weights for policy 1, policy_version 1877215 (0.0011) [2023-12-27 18:39:49,634][20347] Updated weights for policy 0, policy_version 1878848 (0.0009) [2023-12-27 18:39:49,682][20347] Updated weights for policy 0, policy_version 1878858 (0.0009) [2023-12-27 18:39:49,730][20347] Updated weights for policy 0, policy_version 1878868 (0.0007) [2023-12-27 18:39:49,916][20381] Updated weights for policy 1, policy_version 1877225 (0.0010) [2023-12-27 18:39:49,976][20381] Updated weights for policy 1, policy_version 1877235 (0.0008) [2023-12-27 18:39:50,029][20381] Updated weights for policy 1, policy_version 1877245 (0.0008) [2023-12-27 18:39:50,078][20381] Updated weights for policy 1, policy_version 1877255 (0.0007) [2023-12-27 18:39:50,440][20347] Updated weights for policy 0, policy_version 1878878 (0.0008) [2023-12-27 18:39:50,496][20347] Updated weights for policy 0, policy_version 1878888 (0.0007) [2023-12-27 18:39:50,547][20347] Updated weights for policy 0, policy_version 1878898 (0.0007) [2023-12-27 18:39:50,770][20381] Updated weights for policy 1, policy_version 1877265 (0.0008) [2023-12-27 18:39:50,825][20381] Updated weights for policy 1, policy_version 1877275 (0.0006) [2023-12-27 18:39:50,883][20381] Updated weights for policy 1, policy_version 1877285 (0.0008) [2023-12-27 18:39:51,251][20347] Updated weights for policy 0, policy_version 1878908 (0.0007) [2023-12-27 18:39:51,309][20347] Updated weights for policy 0, policy_version 1878918 (0.0008) [2023-12-27 18:39:51,379][20347] Updated weights for policy 0, policy_version 1878928 (0.0008) [2023-12-27 18:39:51,599][20381] Updated weights for policy 1, policy_version 1877295 (0.0008) [2023-12-27 18:39:51,663][20381] Updated weights for policy 1, policy_version 1877305 (0.0008) [2023-12-27 18:39:51,728][20381] Updated weights for policy 1, policy_version 1877315 (0.0008) [2023-12-27 18:39:52,042][20347] Updated weights for policy 0, policy_version 1878938 (0.0007) [2023-12-27 18:39:52,101][20347] Updated weights for policy 0, policy_version 1878948 (0.0007) [2023-12-27 18:39:52,155][20347] Updated weights for policy 0, policy_version 1878958 (0.0007) [2023-12-27 18:39:52,212][20347] Updated weights for policy 0, policy_version 1878968 (0.0006) [2023-12-27 18:39:52,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 961748992. Throughput: 0: 10177.9, 1: 10085.5. Samples: 961718968. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '835.650')] [2023-12-27 18:39:52,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001877320_480649216.pth... [2023-12-27 18:39:52,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001878968_481099776.pth... [2023-12-27 18:39:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001876136_480346112.pth [2023-12-27 18:39:52,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001877752_480788480.pth [2023-12-27 18:39:52,378][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001877320_480649216.pth [2023-12-27 18:39:52,379][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001878968_481099776.pth [2023-12-27 18:39:52,457][20381] Updated weights for policy 1, policy_version 1877325 (0.0008) [2023-12-27 18:39:52,513][20381] Updated weights for policy 1, policy_version 1877335 (0.0007) [2023-12-27 18:39:52,571][20381] Updated weights for policy 1, policy_version 1877345 (0.0008) [2023-12-27 18:39:52,895][20347] Updated weights for policy 0, policy_version 1878978 (0.0007) [2023-12-27 18:39:52,948][20347] Updated weights for policy 0, policy_version 1878988 (0.0005) [2023-12-27 18:39:53,004][20347] Updated weights for policy 0, policy_version 1878998 (0.0005) [2023-12-27 18:39:53,288][20381] Updated weights for policy 1, policy_version 1877355 (0.0007) [2023-12-27 18:39:53,341][20381] Updated weights for policy 1, policy_version 1877365 (0.0007) [2023-12-27 18:39:53,401][20381] Updated weights for policy 1, policy_version 1877375 (0.0007) [2023-12-27 18:39:53,619][20347] Updated weights for policy 0, policy_version 1879008 (0.0009) [2023-12-27 18:39:53,662][20347] Updated weights for policy 0, policy_version 1879018 (0.0005) [2023-12-27 18:39:53,712][20347] Updated weights for policy 0, policy_version 1879028 (0.0009) [2023-12-27 18:39:54,114][20381] Updated weights for policy 1, policy_version 1877385 (0.0008) [2023-12-27 18:39:54,169][20381] Updated weights for policy 1, policy_version 1877395 (0.0007) [2023-12-27 18:39:54,229][20381] Updated weights for policy 1, policy_version 1877405 (0.0007) [2023-12-27 18:39:54,287][20381] Updated weights for policy 1, policy_version 1877415 (0.0008) [2023-12-27 18:39:54,332][20347] Updated weights for policy 0, policy_version 1879038 (0.0008) [2023-12-27 18:39:54,386][20347] Updated weights for policy 0, policy_version 1879048 (0.0010) [2023-12-27 18:39:54,437][20347] Updated weights for policy 0, policy_version 1879058 (0.0010) [2023-12-27 18:39:55,007][20381] Updated weights for policy 1, policy_version 1877425 (0.0007) [2023-12-27 18:39:55,073][20381] Updated weights for policy 1, policy_version 1877435 (0.0006) [2023-12-27 18:39:55,086][20347] Updated weights for policy 0, policy_version 1879068 (0.0009) [2023-12-27 18:39:55,137][20381] Updated weights for policy 1, policy_version 1877445 (0.0007) [2023-12-27 18:39:55,144][20347] Updated weights for policy 0, policy_version 1879078 (0.0008) [2023-12-27 18:39:55,191][20347] Updated weights for policy 0, policy_version 1879088 (0.0007) [2023-12-27 18:39:55,829][20381] Updated weights for policy 1, policy_version 1877455 (0.0006) [2023-12-27 18:39:55,838][20347] Updated weights for policy 0, policy_version 1879098 (0.0008) [2023-12-27 18:39:55,885][20381] Updated weights for policy 1, policy_version 1877465 (0.0006) [2023-12-27 18:39:55,891][20347] Updated weights for policy 0, policy_version 1879108 (0.0005) [2023-12-27 18:39:55,940][20381] Updated weights for policy 1, policy_version 1877475 (0.0007) [2023-12-27 18:39:55,943][20347] Updated weights for policy 0, policy_version 1879118 (0.0006) [2023-12-27 18:39:56,000][20347] Updated weights for policy 0, policy_version 1879128 (0.0006) [2023-12-27 18:39:56,626][20347] Updated weights for policy 0, policy_version 1879138 (0.0006) [2023-12-27 18:39:56,646][20381] Updated weights for policy 1, policy_version 1877485 (0.0007) [2023-12-27 18:39:56,682][20347] Updated weights for policy 0, policy_version 1879148 (0.0005) [2023-12-27 18:39:56,697][20381] Updated weights for policy 1, policy_version 1877495 (0.0008) [2023-12-27 18:39:56,743][20347] Updated weights for policy 0, policy_version 1879158 (0.0005) [2023-12-27 18:39:56,755][20381] Updated weights for policy 1, policy_version 1877505 (0.0008) [2023-12-27 18:39:57,328][20347] Updated weights for policy 0, policy_version 1879168 (0.0008) [2023-12-27 18:39:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 961847296. Throughput: 0: 10202.8, 1: 10068.4. Samples: 961840764. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:39:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '854.330')] [2023-12-27 18:39:57,387][20347] Updated weights for policy 0, policy_version 1879178 (0.0007) [2023-12-27 18:39:57,431][20347] Updated weights for policy 0, policy_version 1879188 (0.0005) [2023-12-27 18:39:57,472][20381] Updated weights for policy 1, policy_version 1877515 (0.0007) [2023-12-27 18:39:57,520][20381] Updated weights for policy 1, policy_version 1877525 (0.0007) [2023-12-27 18:39:57,571][20381] Updated weights for policy 1, policy_version 1877535 (0.0007) [2023-12-27 18:39:58,026][20347] Updated weights for policy 0, policy_version 1879198 (0.0006) [2023-12-27 18:39:58,071][20347] Updated weights for policy 0, policy_version 1879208 (0.0007) [2023-12-27 18:39:58,116][20347] Updated weights for policy 0, policy_version 1879218 (0.0007) [2023-12-27 18:39:58,264][20381] Updated weights for policy 1, policy_version 1877545 (0.0008) [2023-12-27 18:39:58,323][20381] Updated weights for policy 1, policy_version 1877555 (0.0007) [2023-12-27 18:39:58,396][20381] Updated weights for policy 1, policy_version 1877565 (0.0008) [2023-12-27 18:39:58,462][20381] Updated weights for policy 1, policy_version 1877575 (0.0008) [2023-12-27 18:39:58,939][20347] Updated weights for policy 0, policy_version 1879228 (0.0008) [2023-12-27 18:39:58,996][20347] Updated weights for policy 0, policy_version 1879238 (0.0007) [2023-12-27 18:39:59,059][20347] Updated weights for policy 0, policy_version 1879248 (0.0007) [2023-12-27 18:39:59,267][20381] Updated weights for policy 1, policy_version 1877585 (0.0009) [2023-12-27 18:39:59,331][20381] Updated weights for policy 1, policy_version 1877595 (0.0008) [2023-12-27 18:39:59,396][20381] Updated weights for policy 1, policy_version 1877605 (0.0008) [2023-12-27 18:39:59,783][20347] Updated weights for policy 0, policy_version 1879258 (0.0007) [2023-12-27 18:39:59,847][20347] Updated weights for policy 0, policy_version 1879268 (0.0008) [2023-12-27 18:39:59,912][20347] Updated weights for policy 0, policy_version 1879278 (0.0008) [2023-12-27 18:39:59,974][20347] Updated weights for policy 0, policy_version 1879288 (0.0008) [2023-12-27 18:40:00,138][20381] Updated weights for policy 1, policy_version 1877615 (0.0008) [2023-12-27 18:40:00,184][20381] Updated weights for policy 1, policy_version 1877625 (0.0008) [2023-12-27 18:40:00,237][20381] Updated weights for policy 1, policy_version 1877635 (0.0008) [2023-12-27 18:40:00,680][20347] Updated weights for policy 0, policy_version 1879298 (0.0008) [2023-12-27 18:40:00,735][20347] Updated weights for policy 0, policy_version 1879308 (0.0008) [2023-12-27 18:40:00,794][20347] Updated weights for policy 0, policy_version 1879318 (0.0008) [2023-12-27 18:40:00,955][20381] Updated weights for policy 1, policy_version 1877645 (0.0008) [2023-12-27 18:40:01,011][20381] Updated weights for policy 1, policy_version 1877655 (0.0008) [2023-12-27 18:40:01,068][20381] Updated weights for policy 1, policy_version 1877665 (0.0008) [2023-12-27 18:40:01,486][20347] Updated weights for policy 0, policy_version 1879328 (0.0007) [2023-12-27 18:40:01,539][20347] Updated weights for policy 0, policy_version 1879338 (0.0008) [2023-12-27 18:40:01,598][20347] Updated weights for policy 0, policy_version 1879348 (0.0008) [2023-12-27 18:40:01,813][20381] Updated weights for policy 1, policy_version 1877675 (0.0005) [2023-12-27 18:40:01,867][20381] Updated weights for policy 1, policy_version 1877685 (0.0006) [2023-12-27 18:40:01,915][20381] Updated weights for policy 1, policy_version 1877695 (0.0007) [2023-12-27 18:40:02,311][20347] Updated weights for policy 0, policy_version 1879358 (0.0007) [2023-12-27 18:40:02,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20070.4, 300 sec: 20466.1). Total num frames: 961945600. Throughput: 0: 10197.8, 1: 10017.7. Samples: 961959592. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:02,364][19170] Avg episode reward: [(0, '1000.000'), (1, '863.780')] [2023-12-27 18:40:02,376][20347] Updated weights for policy 0, policy_version 1879368 (0.0008) [2023-12-27 18:40:02,422][20347] Updated weights for policy 0, policy_version 1879378 (0.0008) [2023-12-27 18:40:02,579][20381] Updated weights for policy 1, policy_version 1877705 (0.0008) [2023-12-27 18:40:02,638][20381] Updated weights for policy 1, policy_version 1877715 (0.0008) [2023-12-27 18:40:02,696][20381] Updated weights for policy 1, policy_version 1877725 (0.0007) [2023-12-27 18:40:02,749][20381] Updated weights for policy 1, policy_version 1877735 (0.0007) [2023-12-27 18:40:03,125][20347] Updated weights for policy 0, policy_version 1879388 (0.0008) [2023-12-27 18:40:03,189][20347] Updated weights for policy 0, policy_version 1879398 (0.0006) [2023-12-27 18:40:03,242][20347] Updated weights for policy 0, policy_version 1879408 (0.0006) [2023-12-27 18:40:03,422][20381] Updated weights for policy 1, policy_version 1877745 (0.0006) [2023-12-27 18:40:03,485][20381] Updated weights for policy 1, policy_version 1877755 (0.0005) [2023-12-27 18:40:03,542][20381] Updated weights for policy 1, policy_version 1877765 (0.0005) [2023-12-27 18:40:03,957][20347] Updated weights for policy 0, policy_version 1879418 (0.0006) [2023-12-27 18:40:04,013][20347] Updated weights for policy 0, policy_version 1879428 (0.0008) [2023-12-27 18:40:04,066][20347] Updated weights for policy 0, policy_version 1879438 (0.0008) [2023-12-27 18:40:04,116][20381] Updated weights for policy 1, policy_version 1877775 (0.0007) [2023-12-27 18:40:04,123][20347] Updated weights for policy 0, policy_version 1879448 (0.0006) [2023-12-27 18:40:04,168][20381] Updated weights for policy 1, policy_version 1877785 (0.0007) [2023-12-27 18:40:04,225][20381] Updated weights for policy 1, policy_version 1877795 (0.0006) [2023-12-27 18:40:04,859][20347] Updated weights for policy 0, policy_version 1879458 (0.0008) [2023-12-27 18:40:04,880][20381] Updated weights for policy 1, policy_version 1877805 (0.0005) [2023-12-27 18:40:04,913][20347] Updated weights for policy 0, policy_version 1879468 (0.0007) [2023-12-27 18:40:04,938][20381] Updated weights for policy 1, policy_version 1877815 (0.0005) [2023-12-27 18:40:04,965][20347] Updated weights for policy 0, policy_version 1879478 (0.0006) [2023-12-27 18:40:04,998][20381] Updated weights for policy 1, policy_version 1877825 (0.0006) [2023-12-27 18:40:05,538][20381] Updated weights for policy 1, policy_version 1877835 (0.0006) [2023-12-27 18:40:05,591][20381] Updated weights for policy 1, policy_version 1877845 (0.0007) [2023-12-27 18:40:05,645][20381] Updated weights for policy 1, policy_version 1877855 (0.0007) [2023-12-27 18:40:05,699][20347] Updated weights for policy 0, policy_version 1879488 (0.0007) [2023-12-27 18:40:05,742][20347] Updated weights for policy 0, policy_version 1879498 (0.0008) [2023-12-27 18:40:05,796][20347] Updated weights for policy 0, policy_version 1879508 (0.0007) [2023-12-27 18:40:06,355][20381] Updated weights for policy 1, policy_version 1877865 (0.0006) [2023-12-27 18:40:06,422][20381] Updated weights for policy 1, policy_version 1877875 (0.0007) [2023-12-27 18:40:06,491][20381] Updated weights for policy 1, policy_version 1877885 (0.0006) [2023-12-27 18:40:06,507][20347] Updated weights for policy 0, policy_version 1879518 (0.0007) [2023-12-27 18:40:06,553][20381] Updated weights for policy 1, policy_version 1877895 (0.0007) [2023-12-27 18:40:06,575][20347] Updated weights for policy 0, policy_version 1879528 (0.0008) [2023-12-27 18:40:06,640][20347] Updated weights for policy 0, policy_version 1879538 (0.0008) [2023-12-27 18:40:07,205][20381] Updated weights for policy 1, policy_version 1877905 (0.0007) [2023-12-27 18:40:07,259][20381] Updated weights for policy 1, policy_version 1877915 (0.0007) [2023-12-27 18:40:07,315][20381] Updated weights for policy 1, policy_version 1877925 (0.0007) [2023-12-27 18:40:07,348][20347] Updated weights for policy 0, policy_version 1879548 (0.0006) [2023-12-27 18:40:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20207.0, 300 sec: 20521.7). Total num frames: 962052096. Throughput: 0: 10198.2, 1: 10096.7. Samples: 962021180. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '877.040')] [2023-12-27 18:40:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001877928_480804864.pth... [2023-12-27 18:40:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001876744_480501760.pth [2023-12-27 18:40:07,403][20347] Updated weights for policy 0, policy_version 1879558 (0.0009) [2023-12-27 18:40:07,447][20347] Updated weights for policy 0, policy_version 1879568 (0.0008) [2023-12-27 18:40:07,482][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001879576_481255424.pth... [2023-12-27 18:40:07,485][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001878392_480952320.pth [2023-12-27 18:40:07,937][20381] Updated weights for policy 1, policy_version 1877935 (0.0006) [2023-12-27 18:40:07,994][20381] Updated weights for policy 1, policy_version 1877945 (0.0007) [2023-12-27 18:40:08,053][20381] Updated weights for policy 1, policy_version 1877955 (0.0007) [2023-12-27 18:40:08,136][20347] Updated weights for policy 0, policy_version 1879578 (0.0007) [2023-12-27 18:40:08,185][20347] Updated weights for policy 0, policy_version 1879588 (0.0005) [2023-12-27 18:40:08,241][20347] Updated weights for policy 0, policy_version 1879598 (0.0005) [2023-12-27 18:40:08,298][20347] Updated weights for policy 0, policy_version 1879608 (0.0007) [2023-12-27 18:40:08,767][20381] Updated weights for policy 1, policy_version 1877965 (0.0008) [2023-12-27 18:40:08,834][20381] Updated weights for policy 1, policy_version 1877975 (0.0008) [2023-12-27 18:40:08,888][20381] Updated weights for policy 1, policy_version 1877985 (0.0006) [2023-12-27 18:40:08,963][20347] Updated weights for policy 0, policy_version 1879618 (0.0008) [2023-12-27 18:40:09,011][20347] Updated weights for policy 0, policy_version 1879628 (0.0008) [2023-12-27 18:40:09,057][20347] Updated weights for policy 0, policy_version 1879638 (0.0008) [2023-12-27 18:40:09,597][20381] Updated weights for policy 1, policy_version 1877995 (0.0006) [2023-12-27 18:40:09,658][20381] Updated weights for policy 1, policy_version 1878005 (0.0007) [2023-12-27 18:40:09,721][20381] Updated weights for policy 1, policy_version 1878015 (0.0008) [2023-12-27 18:40:09,809][20347] Updated weights for policy 0, policy_version 1879648 (0.0008) [2023-12-27 18:40:09,878][20347] Updated weights for policy 0, policy_version 1879658 (0.0010) [2023-12-27 18:40:09,947][20347] Updated weights for policy 0, policy_version 1879668 (0.0008) [2023-12-27 18:40:10,431][20381] Updated weights for policy 1, policy_version 1878025 (0.0007) [2023-12-27 18:40:10,491][20381] Updated weights for policy 1, policy_version 1878035 (0.0007) [2023-12-27 18:40:10,560][20381] Updated weights for policy 1, policy_version 1878045 (0.0006) [2023-12-27 18:40:10,624][20381] Updated weights for policy 1, policy_version 1878055 (0.0008) [2023-12-27 18:40:10,656][20347] Updated weights for policy 0, policy_version 1879678 (0.0009) [2023-12-27 18:40:10,713][20347] Updated weights for policy 0, policy_version 1879688 (0.0007) [2023-12-27 18:40:10,775][20347] Updated weights for policy 0, policy_version 1879698 (0.0007) [2023-12-27 18:40:11,317][20381] Updated weights for policy 1, policy_version 1878065 (0.0008) [2023-12-27 18:40:11,388][20381] Updated weights for policy 1, policy_version 1878075 (0.0009) [2023-12-27 18:40:11,451][20381] Updated weights for policy 1, policy_version 1878085 (0.0008) [2023-12-27 18:40:11,535][20347] Updated weights for policy 0, policy_version 1879708 (0.0009) [2023-12-27 18:40:11,595][20347] Updated weights for policy 0, policy_version 1879718 (0.0008) [2023-12-27 18:40:11,664][20347] Updated weights for policy 0, policy_version 1879728 (0.0008) [2023-12-27 18:40:12,102][20381] Updated weights for policy 1, policy_version 1878095 (0.0008) [2023-12-27 18:40:12,159][20381] Updated weights for policy 1, policy_version 1878105 (0.0007) [2023-12-27 18:40:12,215][20381] Updated weights for policy 1, policy_version 1878115 (0.0007) [2023-12-27 18:40:12,327][20347] Updated weights for policy 0, policy_version 1879738 (0.0007) [2023-12-27 18:40:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20493.9). Total num frames: 962150400. Throughput: 0: 10186.9, 1: 10122.4. Samples: 962141620. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '809.540')] [2023-12-27 18:40:12,394][20347] Updated weights for policy 0, policy_version 1879748 (0.0008) [2023-12-27 18:40:12,458][20347] Updated weights for policy 0, policy_version 1879758 (0.0007) [2023-12-27 18:40:12,507][20347] Updated weights for policy 0, policy_version 1879768 (0.0008) [2023-12-27 18:40:12,878][20381] Updated weights for policy 1, policy_version 1878125 (0.0006) [2023-12-27 18:40:12,935][20381] Updated weights for policy 1, policy_version 1878135 (0.0005) [2023-12-27 18:40:12,984][20381] Updated weights for policy 1, policy_version 1878145 (0.0007) [2023-12-27 18:40:13,175][20347] Updated weights for policy 0, policy_version 1879778 (0.0007) [2023-12-27 18:40:13,222][20347] Updated weights for policy 0, policy_version 1879788 (0.0007) [2023-12-27 18:40:13,269][20347] Updated weights for policy 0, policy_version 1879798 (0.0007) [2023-12-27 18:40:13,637][20381] Updated weights for policy 1, policy_version 1878155 (0.0011) [2023-12-27 18:40:13,686][20381] Updated weights for policy 1, policy_version 1878165 (0.0010) [2023-12-27 18:40:13,734][20381] Updated weights for policy 1, policy_version 1878175 (0.0010) [2023-12-27 18:40:13,992][20347] Updated weights for policy 0, policy_version 1879808 (0.0008) [2023-12-27 18:40:14,047][20347] Updated weights for policy 0, policy_version 1879818 (0.0008) [2023-12-27 18:40:14,097][20347] Updated weights for policy 0, policy_version 1879828 (0.0008) [2023-12-27 18:40:14,443][20381] Updated weights for policy 1, policy_version 1878185 (0.0009) [2023-12-27 18:40:14,489][20381] Updated weights for policy 1, policy_version 1878195 (0.0008) [2023-12-27 18:40:14,540][20381] Updated weights for policy 1, policy_version 1878205 (0.0008) [2023-12-27 18:40:14,590][20381] Updated weights for policy 1, policy_version 1878215 (0.0008) [2023-12-27 18:40:14,772][20347] Updated weights for policy 0, policy_version 1879838 (0.0007) [2023-12-27 18:40:14,833][20347] Updated weights for policy 0, policy_version 1879848 (0.0010) [2023-12-27 18:40:14,894][20347] Updated weights for policy 0, policy_version 1879858 (0.0011) [2023-12-27 18:40:15,387][20381] Updated weights for policy 1, policy_version 1878225 (0.0007) [2023-12-27 18:40:15,448][20381] Updated weights for policy 1, policy_version 1878235 (0.0008) [2023-12-27 18:40:15,509][20381] Updated weights for policy 1, policy_version 1878245 (0.0008) [2023-12-27 18:40:15,544][20347] Updated weights for policy 0, policy_version 1879868 (0.0008) [2023-12-27 18:40:15,596][20347] Updated weights for policy 0, policy_version 1879878 (0.0005) [2023-12-27 18:40:15,652][20347] Updated weights for policy 0, policy_version 1879888 (0.0005) [2023-12-27 18:40:16,140][20381] Updated weights for policy 1, policy_version 1878255 (0.0005) [2023-12-27 18:40:16,192][20381] Updated weights for policy 1, policy_version 1878265 (0.0005) [2023-12-27 18:40:16,243][20347] Updated weights for policy 0, policy_version 1879898 (0.0005) [2023-12-27 18:40:16,245][20381] Updated weights for policy 1, policy_version 1878275 (0.0007) [2023-12-27 18:40:16,291][20347] Updated weights for policy 0, policy_version 1879908 (0.0006) [2023-12-27 18:40:16,345][20347] Updated weights for policy 0, policy_version 1879918 (0.0005) [2023-12-27 18:40:16,403][20347] Updated weights for policy 0, policy_version 1879928 (0.0005) [2023-12-27 18:40:16,866][20381] Updated weights for policy 1, policy_version 1878285 (0.0009) [2023-12-27 18:40:16,917][20381] Updated weights for policy 1, policy_version 1878295 (0.0007) [2023-12-27 18:40:16,933][20347] Updated weights for policy 0, policy_version 1879938 (0.0005) [2023-12-27 18:40:16,968][20381] Updated weights for policy 1, policy_version 1878305 (0.0008) [2023-12-27 18:40:16,988][20347] Updated weights for policy 0, policy_version 1879948 (0.0006) [2023-12-27 18:40:17,044][20347] Updated weights for policy 0, policy_version 1879958 (0.0007) [2023-12-27 18:40:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 962256896. Throughput: 0: 10155.0, 1: 10119.1. Samples: 962266716. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '864.590')] [2023-12-27 18:40:17,618][20381] Updated weights for policy 1, policy_version 1878315 (0.0008) [2023-12-27 18:40:17,673][20381] Updated weights for policy 1, policy_version 1878325 (0.0007) [2023-12-27 18:40:17,681][20347] Updated weights for policy 0, policy_version 1879968 (0.0010) [2023-12-27 18:40:17,724][20381] Updated weights for policy 1, policy_version 1878335 (0.0005) [2023-12-27 18:40:17,733][20347] Updated weights for policy 0, policy_version 1879978 (0.0010) [2023-12-27 18:40:17,784][20347] Updated weights for policy 0, policy_version 1879988 (0.0010) [2023-12-27 18:40:18,425][20381] Updated weights for policy 1, policy_version 1878345 (0.0005) [2023-12-27 18:40:18,475][20347] Updated weights for policy 0, policy_version 1879998 (0.0010) [2023-12-27 18:40:18,481][20381] Updated weights for policy 1, policy_version 1878355 (0.0006) [2023-12-27 18:40:18,524][20347] Updated weights for policy 0, policy_version 1880008 (0.0007) [2023-12-27 18:40:18,538][20381] Updated weights for policy 1, policy_version 1878365 (0.0007) [2023-12-27 18:40:18,582][20347] Updated weights for policy 0, policy_version 1880018 (0.0009) [2023-12-27 18:40:18,598][20381] Updated weights for policy 1, policy_version 1878375 (0.0009) [2023-12-27 18:40:19,232][20381] Updated weights for policy 1, policy_version 1878385 (0.0007) [2023-12-27 18:40:19,238][20347] Updated weights for policy 0, policy_version 1880028 (0.0010) [2023-12-27 18:40:19,289][20381] Updated weights for policy 1, policy_version 1878395 (0.0007) [2023-12-27 18:40:19,295][20347] Updated weights for policy 0, policy_version 1880038 (0.0008) [2023-12-27 18:40:19,344][20381] Updated weights for policy 1, policy_version 1878405 (0.0007) [2023-12-27 18:40:19,362][20347] Updated weights for policy 0, policy_version 1880049 (0.0008) [2023-12-27 18:40:20,106][20381] Updated weights for policy 1, policy_version 1878415 (0.0008) [2023-12-27 18:40:20,115][20347] Updated weights for policy 0, policy_version 1880059 (0.0008) [2023-12-27 18:40:20,164][20381] Updated weights for policy 1, policy_version 1878425 (0.0007) [2023-12-27 18:40:20,171][20347] Updated weights for policy 0, policy_version 1880069 (0.0008) [2023-12-27 18:40:20,220][20381] Updated weights for policy 1, policy_version 1878435 (0.0008) [2023-12-27 18:40:20,227][20347] Updated weights for policy 0, policy_version 1880079 (0.0008) [2023-12-27 18:40:20,831][20381] Updated weights for policy 1, policy_version 1878445 (0.0006) [2023-12-27 18:40:20,896][20381] Updated weights for policy 1, policy_version 1878455 (0.0006) [2023-12-27 18:40:20,954][20347] Updated weights for policy 0, policy_version 1880089 (0.0008) [2023-12-27 18:40:20,960][20381] Updated weights for policy 1, policy_version 1878465 (0.0007) [2023-12-27 18:40:21,016][20347] Updated weights for policy 0, policy_version 1880099 (0.0010) [2023-12-27 18:40:21,076][20347] Updated weights for policy 0, policy_version 1880109 (0.0010) [2023-12-27 18:40:21,145][20347] Updated weights for policy 0, policy_version 1880119 (0.0008) [2023-12-27 18:40:21,623][20381] Updated weights for policy 1, policy_version 1878475 (0.0007) [2023-12-27 18:40:21,691][20381] Updated weights for policy 1, policy_version 1878485 (0.0006) [2023-12-27 18:40:21,757][20381] Updated weights for policy 1, policy_version 1878495 (0.0008) [2023-12-27 18:40:21,941][20347] Updated weights for policy 0, policy_version 1880129 (0.0008) [2023-12-27 18:40:22,005][20347] Updated weights for policy 0, policy_version 1880139 (0.0008) [2023-12-27 18:40:22,061][20347] Updated weights for policy 0, policy_version 1880149 (0.0008) [2023-12-27 18:40:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 962355200. Throughput: 0: 10143.2, 1: 10084.3. Samples: 962327028. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.240')] [2023-12-27 18:40:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001880152_481402880.pth... [2023-12-27 18:40:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001878504_480952320.pth... [2023-12-27 18:40:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001877320_480649216.pth [2023-12-27 18:40:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001878968_481099776.pth [2023-12-27 18:40:22,440][20381] Updated weights for policy 1, policy_version 1878505 (0.0006) [2023-12-27 18:40:22,499][20381] Updated weights for policy 1, policy_version 1878515 (0.0007) [2023-12-27 18:40:22,552][20381] Updated weights for policy 1, policy_version 1878525 (0.0008) [2023-12-27 18:40:22,605][20381] Updated weights for policy 1, policy_version 1878535 (0.0008) [2023-12-27 18:40:22,835][20347] Updated weights for policy 0, policy_version 1880159 (0.0008) [2023-12-27 18:40:22,897][20347] Updated weights for policy 0, policy_version 1880169 (0.0007) [2023-12-27 18:40:22,953][20347] Updated weights for policy 0, policy_version 1880179 (0.0008) [2023-12-27 18:40:23,324][20381] Updated weights for policy 1, policy_version 1878545 (0.0007) [2023-12-27 18:40:23,383][20381] Updated weights for policy 1, policy_version 1878555 (0.0008) [2023-12-27 18:40:23,433][20381] Updated weights for policy 1, policy_version 1878565 (0.0008) [2023-12-27 18:40:23,621][20347] Updated weights for policy 0, policy_version 1880189 (0.0008) [2023-12-27 18:40:23,668][20347] Updated weights for policy 0, policy_version 1880199 (0.0006) [2023-12-27 18:40:23,713][20347] Updated weights for policy 0, policy_version 1880209 (0.0006) [2023-12-27 18:40:24,118][20381] Updated weights for policy 1, policy_version 1878575 (0.0008) [2023-12-27 18:40:24,177][20381] Updated weights for policy 1, policy_version 1878585 (0.0008) [2023-12-27 18:40:24,237][20381] Updated weights for policy 1, policy_version 1878595 (0.0007) [2023-12-27 18:40:24,331][20347] Updated weights for policy 0, policy_version 1880219 (0.0005) [2023-12-27 18:40:24,385][20347] Updated weights for policy 0, policy_version 1880229 (0.0005) [2023-12-27 18:40:24,436][20347] Updated weights for policy 0, policy_version 1880239 (0.0006) [2023-12-27 18:40:24,806][20381] Updated weights for policy 1, policy_version 1878605 (0.0005) [2023-12-27 18:40:24,852][20381] Updated weights for policy 1, policy_version 1878615 (0.0005) [2023-12-27 18:40:24,899][20381] Updated weights for policy 1, policy_version 1878625 (0.0005) [2023-12-27 18:40:25,069][20347] Updated weights for policy 0, policy_version 1880249 (0.0006) [2023-12-27 18:40:25,125][20347] Updated weights for policy 0, policy_version 1880259 (0.0005) [2023-12-27 18:40:25,177][20347] Updated weights for policy 0, policy_version 1880269 (0.0005) [2023-12-27 18:40:25,226][20347] Updated weights for policy 0, policy_version 1880279 (0.0005) [2023-12-27 18:40:25,541][20381] Updated weights for policy 1, policy_version 1878635 (0.0006) [2023-12-27 18:40:25,598][20381] Updated weights for policy 1, policy_version 1878645 (0.0007) [2023-12-27 18:40:25,651][20381] Updated weights for policy 1, policy_version 1878655 (0.0008) [2023-12-27 18:40:25,783][20347] Updated weights for policy 0, policy_version 1880289 (0.0005) [2023-12-27 18:40:25,844][20347] Updated weights for policy 0, policy_version 1880299 (0.0005) [2023-12-27 18:40:25,899][20347] Updated weights for policy 0, policy_version 1880309 (0.0005) [2023-12-27 18:40:26,353][20381] Updated weights for policy 1, policy_version 1878665 (0.0008) [2023-12-27 18:40:26,412][20381] Updated weights for policy 1, policy_version 1878675 (0.0008) [2023-12-27 18:40:26,473][20381] Updated weights for policy 1, policy_version 1878685 (0.0007) [2023-12-27 18:40:26,496][20347] Updated weights for policy 0, policy_version 1880319 (0.0006) [2023-12-27 18:40:26,525][20381] Updated weights for policy 1, policy_version 1878695 (0.0005) [2023-12-27 18:40:26,545][20347] Updated weights for policy 0, policy_version 1880329 (0.0006) [2023-12-27 18:40:26,606][20347] Updated weights for policy 0, policy_version 1880339 (0.0008) [2023-12-27 18:40:27,072][20381] Updated weights for policy 1, policy_version 1878705 (0.0007) [2023-12-27 18:40:27,127][20381] Updated weights for policy 1, policy_version 1878715 (0.0008) [2023-12-27 18:40:27,174][20381] Updated weights for policy 1, policy_version 1878725 (0.0007) [2023-12-27 18:40:27,242][20347] Updated weights for policy 0, policy_version 1880349 (0.0009) [2023-12-27 18:40:27,308][20347] Updated weights for policy 0, policy_version 1880359 (0.0010) [2023-12-27 18:40:27,361][20347] Updated weights for policy 0, policy_version 1880369 (0.0007) [2023-12-27 18:40:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 962461696. Throughput: 0: 10181.0, 1: 10159.4. Samples: 962451312. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.120')] [2023-12-27 18:40:27,822][20381] Updated weights for policy 1, policy_version 1878735 (0.0006) [2023-12-27 18:40:27,879][20381] Updated weights for policy 1, policy_version 1878745 (0.0007) [2023-12-27 18:40:27,894][20347] Updated weights for policy 0, policy_version 1880379 (0.0006) [2023-12-27 18:40:27,929][20381] Updated weights for policy 1, policy_version 1878755 (0.0008) [2023-12-27 18:40:27,950][20347] Updated weights for policy 0, policy_version 1880389 (0.0007) [2023-12-27 18:40:28,007][20347] Updated weights for policy 0, policy_version 1880399 (0.0010) [2023-12-27 18:40:28,611][20347] Updated weights for policy 0, policy_version 1880409 (0.0010) [2023-12-27 18:40:28,630][20381] Updated weights for policy 1, policy_version 1878765 (0.0007) [2023-12-27 18:40:28,666][20347] Updated weights for policy 0, policy_version 1880419 (0.0010) [2023-12-27 18:40:28,683][20381] Updated weights for policy 1, policy_version 1878775 (0.0005) [2023-12-27 18:40:28,723][20347] Updated weights for policy 0, policy_version 1880429 (0.0010) [2023-12-27 18:40:28,743][20381] Updated weights for policy 1, policy_version 1878785 (0.0008) [2023-12-27 18:40:28,771][20347] Updated weights for policy 0, policy_version 1880439 (0.0007) [2023-12-27 18:40:29,352][20347] Updated weights for policy 0, policy_version 1880449 (0.0007) [2023-12-27 18:40:29,415][20347] Updated weights for policy 0, policy_version 1880459 (0.0007) [2023-12-27 18:40:29,468][20347] Updated weights for policy 0, policy_version 1880469 (0.0006) [2023-12-27 18:40:29,513][20381] Updated weights for policy 1, policy_version 1878795 (0.0007) [2023-12-27 18:40:29,577][20381] Updated weights for policy 1, policy_version 1878805 (0.0007) [2023-12-27 18:40:29,638][20381] Updated weights for policy 1, policy_version 1878815 (0.0008) [2023-12-27 18:40:30,112][20347] Updated weights for policy 0, policy_version 1880479 (0.0007) [2023-12-27 18:40:30,172][20347] Updated weights for policy 0, policy_version 1880489 (0.0007) [2023-12-27 18:40:30,220][20347] Updated weights for policy 0, policy_version 1880499 (0.0007) [2023-12-27 18:40:30,320][20381] Updated weights for policy 1, policy_version 1878825 (0.0008) [2023-12-27 18:40:30,382][20381] Updated weights for policy 1, policy_version 1878835 (0.0006) [2023-12-27 18:40:30,443][20381] Updated weights for policy 1, policy_version 1878845 (0.0006) [2023-12-27 18:40:30,499][20381] Updated weights for policy 1, policy_version 1878855 (0.0008) [2023-12-27 18:40:30,900][20347] Updated weights for policy 0, policy_version 1880509 (0.0007) [2023-12-27 18:40:30,945][20347] Updated weights for policy 0, policy_version 1880519 (0.0007) [2023-12-27 18:40:30,989][20347] Updated weights for policy 0, policy_version 1880529 (0.0006) [2023-12-27 18:40:31,168][20381] Updated weights for policy 1, policy_version 1878865 (0.0008) [2023-12-27 18:40:31,230][20381] Updated weights for policy 1, policy_version 1878875 (0.0006) [2023-12-27 18:40:31,299][20381] Updated weights for policy 1, policy_version 1878885 (0.0007) [2023-12-27 18:40:31,633][20347] Updated weights for policy 0, policy_version 1880539 (0.0006) [2023-12-27 18:40:31,698][20347] Updated weights for policy 0, policy_version 1880549 (0.0008) [2023-12-27 18:40:31,760][20347] Updated weights for policy 0, policy_version 1880559 (0.0008) [2023-12-27 18:40:32,002][20381] Updated weights for policy 1, policy_version 1878895 (0.0008) [2023-12-27 18:40:32,058][20381] Updated weights for policy 1, policy_version 1878905 (0.0008) [2023-12-27 18:40:32,106][20381] Updated weights for policy 1, policy_version 1878915 (0.0007) [2023-12-27 18:40:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 962568192. Throughput: 0: 10300.0, 1: 10129.5. Samples: 962578516. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '920.450')] [2023-12-27 18:40:32,412][20347] Updated weights for policy 0, policy_version 1880569 (0.0007) [2023-12-27 18:40:32,466][20347] Updated weights for policy 0, policy_version 1880579 (0.0008) [2023-12-27 18:40:32,519][20347] Updated weights for policy 0, policy_version 1880589 (0.0007) [2023-12-27 18:40:32,571][20347] Updated weights for policy 0, policy_version 1880599 (0.0007) [2023-12-27 18:40:32,758][20381] Updated weights for policy 1, policy_version 1878925 (0.0008) [2023-12-27 18:40:32,815][20381] Updated weights for policy 1, policy_version 1878935 (0.0008) [2023-12-27 18:40:32,871][20381] Updated weights for policy 1, policy_version 1878945 (0.0005) [2023-12-27 18:40:33,250][20347] Updated weights for policy 0, policy_version 1880609 (0.0006) [2023-12-27 18:40:33,309][20347] Updated weights for policy 0, policy_version 1880619 (0.0006) [2023-12-27 18:40:33,380][20347] Updated weights for policy 0, policy_version 1880629 (0.0007) [2023-12-27 18:40:33,585][20381] Updated weights for policy 1, policy_version 1878955 (0.0007) [2023-12-27 18:40:33,637][20381] Updated weights for policy 1, policy_version 1878965 (0.0007) [2023-12-27 18:40:33,691][20381] Updated weights for policy 1, policy_version 1878975 (0.0007) [2023-12-27 18:40:34,020][20347] Updated weights for policy 0, policy_version 1880639 (0.0010) [2023-12-27 18:40:34,074][20347] Updated weights for policy 0, policy_version 1880649 (0.0010) [2023-12-27 18:40:34,132][20347] Updated weights for policy 0, policy_version 1880659 (0.0010) [2023-12-27 18:40:34,360][20381] Updated weights for policy 1, policy_version 1878985 (0.0008) [2023-12-27 18:40:34,431][20381] Updated weights for policy 1, policy_version 1878995 (0.0008) [2023-12-27 18:40:34,499][20381] Updated weights for policy 1, policy_version 1879005 (0.0008) [2023-12-27 18:40:34,565][20381] Updated weights for policy 1, policy_version 1879015 (0.0008) [2023-12-27 18:40:34,835][20347] Updated weights for policy 0, policy_version 1880669 (0.0007) [2023-12-27 18:40:34,897][20347] Updated weights for policy 0, policy_version 1880679 (0.0006) [2023-12-27 18:40:34,956][20347] Updated weights for policy 0, policy_version 1880689 (0.0006) [2023-12-27 18:40:35,111][20381] Updated weights for policy 1, policy_version 1879025 (0.0010) [2023-12-27 18:40:35,159][20381] Updated weights for policy 1, policy_version 1879035 (0.0010) [2023-12-27 18:40:35,202][20381] Updated weights for policy 1, policy_version 1879045 (0.0009) [2023-12-27 18:40:35,512][20347] Updated weights for policy 0, policy_version 1880699 (0.0007) [2023-12-27 18:40:35,563][20347] Updated weights for policy 0, policy_version 1880709 (0.0010) [2023-12-27 18:40:35,611][20347] Updated weights for policy 0, policy_version 1880719 (0.0010) [2023-12-27 18:40:35,742][20381] Updated weights for policy 1, policy_version 1879055 (0.0005) [2023-12-27 18:40:35,788][20381] Updated weights for policy 1, policy_version 1879065 (0.0005) [2023-12-27 18:40:35,835][20381] Updated weights for policy 1, policy_version 1879075 (0.0007) [2023-12-27 18:40:36,275][20347] Updated weights for policy 0, policy_version 1880729 (0.0010) [2023-12-27 18:40:36,337][20347] Updated weights for policy 0, policy_version 1880739 (0.0010) [2023-12-27 18:40:36,390][20347] Updated weights for policy 0, policy_version 1880749 (0.0007) [2023-12-27 18:40:36,445][20347] Updated weights for policy 0, policy_version 1880759 (0.0008) [2023-12-27 18:40:36,556][20381] Updated weights for policy 1, policy_version 1879085 (0.0007) [2023-12-27 18:40:36,622][20381] Updated weights for policy 1, policy_version 1879095 (0.0007) [2023-12-27 18:40:36,681][20381] Updated weights for policy 1, policy_version 1879105 (0.0007) [2023-12-27 18:40:37,120][20347] Updated weights for policy 0, policy_version 1880769 (0.0010) [2023-12-27 18:40:37,166][20347] Updated weights for policy 0, policy_version 1880779 (0.0010) [2023-12-27 18:40:37,218][20347] Updated weights for policy 0, policy_version 1880789 (0.0009) [2023-12-27 18:40:37,265][20381] Updated weights for policy 1, policy_version 1879115 (0.0007) [2023-12-27 18:40:37,315][20381] Updated weights for policy 1, policy_version 1879125 (0.0005) [2023-12-27 18:40:37,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 962674688. Throughput: 0: 10336.9, 1: 10159.6. Samples: 962641312. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.080')] [2023-12-27 18:40:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001880792_481566720.pth... [2023-12-27 18:40:37,371][20381] Updated weights for policy 1, policy_version 1879135 (0.0006) [2023-12-27 18:40:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001879576_481255424.pth [2023-12-27 18:40:37,423][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001879144_481116160.pth... [2023-12-27 18:40:37,426][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001877928_480804864.pth [2023-12-27 18:40:37,925][20347] Updated weights for policy 0, policy_version 1880799 (0.0010) [2023-12-27 18:40:37,957][20381] Updated weights for policy 1, policy_version 1879145 (0.0007) [2023-12-27 18:40:37,982][20347] Updated weights for policy 0, policy_version 1880809 (0.0009) [2023-12-27 18:40:38,007][20381] Updated weights for policy 1, policy_version 1879155 (0.0009) [2023-12-27 18:40:38,040][20347] Updated weights for policy 0, policy_version 1880819 (0.0009) [2023-12-27 18:40:38,063][20381] Updated weights for policy 1, policy_version 1879165 (0.0009) [2023-12-27 18:40:38,117][20381] Updated weights for policy 1, policy_version 1879175 (0.0009) [2023-12-27 18:40:38,733][20347] Updated weights for policy 0, policy_version 1880829 (0.0010) [2023-12-27 18:40:38,795][20347] Updated weights for policy 0, policy_version 1880839 (0.0010) [2023-12-27 18:40:38,834][20381] Updated weights for policy 1, policy_version 1879185 (0.0006) [2023-12-27 18:40:38,847][20347] Updated weights for policy 0, policy_version 1880849 (0.0010) [2023-12-27 18:40:38,898][20381] Updated weights for policy 1, policy_version 1879195 (0.0006) [2023-12-27 18:40:38,953][20381] Updated weights for policy 1, policy_version 1879205 (0.0008) [2023-12-27 18:40:39,534][20347] Updated weights for policy 0, policy_version 1880859 (0.0011) [2023-12-27 18:40:39,593][20347] Updated weights for policy 0, policy_version 1880869 (0.0011) [2023-12-27 18:40:39,624][20381] Updated weights for policy 1, policy_version 1879215 (0.0010) [2023-12-27 18:40:39,649][20347] Updated weights for policy 0, policy_version 1880879 (0.0008) [2023-12-27 18:40:39,684][20381] Updated weights for policy 1, policy_version 1879225 (0.0011) [2023-12-27 18:40:39,744][20381] Updated weights for policy 1, policy_version 1879235 (0.0011) [2023-12-27 18:40:40,386][20347] Updated weights for policy 0, policy_version 1880889 (0.0010) [2023-12-27 18:40:40,436][20381] Updated weights for policy 1, policy_version 1879245 (0.0010) [2023-12-27 18:40:40,448][20347] Updated weights for policy 0, policy_version 1880899 (0.0008) [2023-12-27 18:40:40,497][20381] Updated weights for policy 1, policy_version 1879255 (0.0011) [2023-12-27 18:40:40,505][20347] Updated weights for policy 0, policy_version 1880909 (0.0008) [2023-12-27 18:40:40,555][20347] Updated weights for policy 0, policy_version 1880919 (0.0010) [2023-12-27 18:40:40,556][20381] Updated weights for policy 1, policy_version 1879265 (0.0011) [2023-12-27 18:40:41,257][20347] Updated weights for policy 0, policy_version 1880929 (0.0008) [2023-12-27 18:40:41,270][20381] Updated weights for policy 1, policy_version 1879275 (0.0009) [2023-12-27 18:40:41,317][20347] Updated weights for policy 0, policy_version 1880939 (0.0008) [2023-12-27 18:40:41,331][20381] Updated weights for policy 1, policy_version 1879285 (0.0007) [2023-12-27 18:40:41,378][20347] Updated weights for policy 0, policy_version 1880949 (0.0008) [2023-12-27 18:40:41,398][20381] Updated weights for policy 1, policy_version 1879295 (0.0008) [2023-12-27 18:40:42,100][20381] Updated weights for policy 1, policy_version 1879305 (0.0006) [2023-12-27 18:40:42,159][20381] Updated weights for policy 1, policy_version 1879315 (0.0005) [2023-12-27 18:40:42,194][20347] Updated weights for policy 0, policy_version 1880959 (0.0008) [2023-12-27 18:40:42,212][20381] Updated weights for policy 1, policy_version 1879325 (0.0005) [2023-12-27 18:40:42,256][20347] Updated weights for policy 0, policy_version 1880969 (0.0007) [2023-12-27 18:40:42,277][20381] Updated weights for policy 1, policy_version 1879335 (0.0007) [2023-12-27 18:40:42,317][20347] Updated weights for policy 0, policy_version 1880979 (0.0007) [2023-12-27 18:40:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 962781184. Throughput: 0: 10292.0, 1: 10268.7. Samples: 962765996. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '916.540')] [2023-12-27 18:40:42,951][20381] Updated weights for policy 1, policy_version 1879345 (0.0008) [2023-12-27 18:40:42,952][20347] Updated weights for policy 0, policy_version 1880989 (0.0008) [2023-12-27 18:40:43,009][20381] Updated weights for policy 1, policy_version 1879355 (0.0006) [2023-12-27 18:40:43,011][20347] Updated weights for policy 0, policy_version 1880999 (0.0009) [2023-12-27 18:40:43,065][20381] Updated weights for policy 1, policy_version 1879365 (0.0005) [2023-12-27 18:40:43,067][20347] Updated weights for policy 0, policy_version 1881009 (0.0006) [2023-12-27 18:40:43,615][20347] Updated weights for policy 0, policy_version 1881019 (0.0005) [2023-12-27 18:40:43,675][20347] Updated weights for policy 0, policy_version 1881029 (0.0005) [2023-12-27 18:40:43,732][20381] Updated weights for policy 1, policy_version 1879375 (0.0009) [2023-12-27 18:40:43,733][20347] Updated weights for policy 0, policy_version 1881039 (0.0010) [2023-12-27 18:40:43,788][20381] Updated weights for policy 1, policy_version 1879385 (0.0011) [2023-12-27 18:40:43,836][20381] Updated weights for policy 1, policy_version 1879395 (0.0007) [2023-12-27 18:40:44,290][20347] Updated weights for policy 0, policy_version 1881049 (0.0009) [2023-12-27 18:40:44,359][20347] Updated weights for policy 0, policy_version 1881059 (0.0005) [2023-12-27 18:40:44,411][20347] Updated weights for policy 0, policy_version 1881069 (0.0005) [2023-12-27 18:40:44,423][20381] Updated weights for policy 1, policy_version 1879405 (0.0007) [2023-12-27 18:40:44,456][20347] Updated weights for policy 0, policy_version 1881079 (0.0007) [2023-12-27 18:40:44,475][20381] Updated weights for policy 1, policy_version 1879415 (0.0006) [2023-12-27 18:40:44,533][20381] Updated weights for policy 1, policy_version 1879425 (0.0006) [2023-12-27 18:40:45,064][20347] Updated weights for policy 0, policy_version 1881089 (0.0008) [2023-12-27 18:40:45,117][20347] Updated weights for policy 0, policy_version 1881099 (0.0007) [2023-12-27 18:40:45,177][20347] Updated weights for policy 0, policy_version 1881109 (0.0007) [2023-12-27 18:40:45,183][20381] Updated weights for policy 1, policy_version 1879435 (0.0006) [2023-12-27 18:40:45,243][20381] Updated weights for policy 1, policy_version 1879445 (0.0007) [2023-12-27 18:40:45,295][20381] Updated weights for policy 1, policy_version 1879455 (0.0007) [2023-12-27 18:40:45,889][20347] Updated weights for policy 0, policy_version 1881119 (0.0007) [2023-12-27 18:40:45,943][20347] Updated weights for policy 0, policy_version 1881129 (0.0007) [2023-12-27 18:40:45,950][20381] Updated weights for policy 1, policy_version 1879465 (0.0007) [2023-12-27 18:40:45,985][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000007 [2023-12-27 18:40:46,014][20381] Updated weights for policy 1, policy_version 1879475 (0.0007) [2023-12-27 18:40:46,072][20381] Updated weights for policy 1, policy_version 1879485 (0.0007) [2023-12-27 18:40:46,126][20381] Updated weights for policy 1, policy_version 1879495 (0.0008) [2023-12-27 18:40:46,682][20347] Updated weights for policy 0, policy_version 1881139 (0.0008) [2023-12-27 18:40:46,734][20381] Updated weights for policy 1, policy_version 1879505 (0.0006) [2023-12-27 18:40:46,739][20347] Updated weights for policy 0, policy_version 1881149 (0.0008) [2023-12-27 18:40:46,783][20381] Updated weights for policy 1, policy_version 1879515 (0.0008) [2023-12-27 18:40:46,800][20347] Updated weights for policy 0, policy_version 1881159 (0.0008) [2023-12-27 18:40:46,832][20381] Updated weights for policy 1, policy_version 1879525 (0.0005) [2023-12-27 18:40:47,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20521.7). Total num frames: 962879488. Throughput: 0: 10318.3, 1: 10412.1. Samples: 962892460. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.890')] [2023-12-27 18:40:47,440][20381] Updated weights for policy 1, policy_version 1879535 (0.0006) [2023-12-27 18:40:47,490][20381] Updated weights for policy 1, policy_version 1879545 (0.0008) [2023-12-27 18:40:47,503][20347] Updated weights for policy 0, policy_version 1881169 (0.0007) [2023-12-27 18:40:47,543][20381] Updated weights for policy 1, policy_version 1879555 (0.0006) [2023-12-27 18:40:47,552][20347] Updated weights for policy 0, policy_version 1881179 (0.0006) [2023-12-27 18:40:47,599][20347] Updated weights for policy 0, policy_version 1881189 (0.0007) [2023-12-27 18:40:47,647][20347] Updated weights for policy 0, policy_version 1881199 (0.0007) [2023-12-27 18:40:48,248][20381] Updated weights for policy 1, policy_version 1879565 (0.0007) [2023-12-27 18:40:48,307][20381] Updated weights for policy 1, policy_version 1879575 (0.0007) [2023-12-27 18:40:48,330][20347] Updated weights for policy 0, policy_version 1881209 (0.0007) [2023-12-27 18:40:48,371][20381] Updated weights for policy 1, policy_version 1879585 (0.0008) [2023-12-27 18:40:48,398][20347] Updated weights for policy 0, policy_version 1881219 (0.0007) [2023-12-27 18:40:48,460][20347] Updated weights for policy 0, policy_version 1881229 (0.0008) [2023-12-27 18:40:49,023][20381] Updated weights for policy 1, policy_version 1879595 (0.0007) [2023-12-27 18:40:49,075][20381] Updated weights for policy 1, policy_version 1879605 (0.0010) [2023-12-27 18:40:49,129][20381] Updated weights for policy 1, policy_version 1879615 (0.0006) [2023-12-27 18:40:49,180][20347] Updated weights for policy 0, policy_version 1881239 (0.0007) [2023-12-27 18:40:49,243][20347] Updated weights for policy 0, policy_version 1881249 (0.0006) [2023-12-27 18:40:49,295][20347] Updated weights for policy 0, policy_version 1881259 (0.0006) [2023-12-27 18:40:49,810][20381] Updated weights for policy 1, policy_version 1879625 (0.0009) [2023-12-27 18:40:49,873][20381] Updated weights for policy 1, policy_version 1879635 (0.0008) [2023-12-27 18:40:49,934][20381] Updated weights for policy 1, policy_version 1879645 (0.0008) [2023-12-27 18:40:49,966][20347] Updated weights for policy 0, policy_version 1881269 (0.0007) [2023-12-27 18:40:49,993][20381] Updated weights for policy 1, policy_version 1879655 (0.0007) [2023-12-27 18:40:50,021][20347] Updated weights for policy 0, policy_version 1881279 (0.0006) [2023-12-27 18:40:50,084][20347] Updated weights for policy 0, policy_version 1881289 (0.0006) [2023-12-27 18:40:50,669][20381] Updated weights for policy 1, policy_version 1879665 (0.0008) [2023-12-27 18:40:50,731][20381] Updated weights for policy 1, policy_version 1879675 (0.0007) [2023-12-27 18:40:50,760][20347] Updated weights for policy 0, policy_version 1881299 (0.0007) [2023-12-27 18:40:50,787][20381] Updated weights for policy 1, policy_version 1879685 (0.0006) [2023-12-27 18:40:50,823][20347] Updated weights for policy 0, policy_version 1881309 (0.0008) [2023-12-27 18:40:50,886][20347] Updated weights for policy 0, policy_version 1881319 (0.0007) [2023-12-27 18:40:51,489][20381] Updated weights for policy 1, policy_version 1879695 (0.0007) [2023-12-27 18:40:51,551][20381] Updated weights for policy 1, policy_version 1879705 (0.0008) [2023-12-27 18:40:51,609][20347] Updated weights for policy 0, policy_version 1881329 (0.0007) [2023-12-27 18:40:51,610][20381] Updated weights for policy 1, policy_version 1879715 (0.0007) [2023-12-27 18:40:51,673][20347] Updated weights for policy 0, policy_version 1881339 (0.0007) [2023-12-27 18:40:51,728][20347] Updated weights for policy 0, policy_version 1881349 (0.0008) [2023-12-27 18:40:51,789][20347] Updated weights for policy 0, policy_version 1881359 (0.0008) [2023-12-27 18:40:52,348][20381] Updated weights for policy 1, policy_version 1879725 (0.0008) [2023-12-27 18:40:52,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 962977792. Throughput: 0: 10353.9, 1: 10358.7. Samples: 962953244. Policy #0 lag: (min: 12.0, avg: 20.3, max: 44.0) [2023-12-27 18:40:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '916.290')] [2023-12-27 18:40:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001881360_481714176.pth... [2023-12-27 18:40:52,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001880152_481402880.pth [2023-12-27 18:40:52,407][20381] Updated weights for policy 1, policy_version 1879735 (0.0006) [2023-12-27 18:40:52,466][20381] Updated weights for policy 1, policy_version 1879745 (0.0007) [2023-12-27 18:40:52,507][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001879752_481271808.pth... [2023-12-27 18:40:52,511][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001878504_480952320.pth [2023-12-27 18:40:52,538][20347] Updated weights for policy 0, policy_version 1881369 (0.0007) [2023-12-27 18:40:52,599][20347] Updated weights for policy 0, policy_version 1881379 (0.0007) [2023-12-27 18:40:52,662][20347] Updated weights for policy 0, policy_version 1881389 (0.0008) [2023-12-27 18:40:53,116][20381] Updated weights for policy 1, policy_version 1879755 (0.0009) [2023-12-27 18:40:53,180][20381] Updated weights for policy 1, policy_version 1879765 (0.0008) [2023-12-27 18:40:53,245][20381] Updated weights for policy 1, policy_version 1879775 (0.0008) [2023-12-27 18:40:53,265][20347] Updated weights for policy 0, policy_version 1881399 (0.0006) [2023-12-27 18:40:53,321][20347] Updated weights for policy 0, policy_version 1881409 (0.0005) [2023-12-27 18:40:53,372][20347] Updated weights for policy 0, policy_version 1881419 (0.0005) [2023-12-27 18:40:53,882][20381] Updated weights for policy 1, policy_version 1879785 (0.0007) [2023-12-27 18:40:53,932][20381] Updated weights for policy 1, policy_version 1879795 (0.0009) [2023-12-27 18:40:53,987][20381] Updated weights for policy 1, policy_version 1879805 (0.0007) [2023-12-27 18:40:54,035][20381] Updated weights for policy 1, policy_version 1879815 (0.0008) [2023-12-27 18:40:54,044][20347] Updated weights for policy 0, policy_version 1881429 (0.0008) [2023-12-27 18:40:54,101][20347] Updated weights for policy 0, policy_version 1881439 (0.0006) [2023-12-27 18:40:54,158][20347] Updated weights for policy 0, policy_version 1881449 (0.0008) [2023-12-27 18:40:54,626][20381] Updated weights for policy 1, policy_version 1879825 (0.0008) [2023-12-27 18:40:54,694][20381] Updated weights for policy 1, policy_version 1879835 (0.0006) [2023-12-27 18:40:54,748][20381] Updated weights for policy 1, policy_version 1879845 (0.0010) [2023-12-27 18:40:54,865][20347] Updated weights for policy 0, policy_version 1881459 (0.0006) [2023-12-27 18:40:54,915][20347] Updated weights for policy 0, policy_version 1881469 (0.0007) [2023-12-27 18:40:54,965][20347] Updated weights for policy 0, policy_version 1881479 (0.0007) [2023-12-27 18:40:55,411][20381] Updated weights for policy 1, policy_version 1879855 (0.0010) [2023-12-27 18:40:55,469][20381] Updated weights for policy 1, policy_version 1879865 (0.0010) [2023-12-27 18:40:55,513][20381] Updated weights for policy 1, policy_version 1879875 (0.0010) [2023-12-27 18:40:55,619][20347] Updated weights for policy 0, policy_version 1881489 (0.0008) [2023-12-27 18:40:55,677][20347] Updated weights for policy 0, policy_version 1881499 (0.0008) [2023-12-27 18:40:55,731][20347] Updated weights for policy 0, policy_version 1881509 (0.0008) [2023-12-27 18:40:55,785][20347] Updated weights for policy 0, policy_version 1881519 (0.0007) [2023-12-27 18:40:56,138][20381] Updated weights for policy 1, policy_version 1879885 (0.0009) [2023-12-27 18:40:56,190][20381] Updated weights for policy 1, policy_version 1879895 (0.0007) [2023-12-27 18:40:56,238][20381] Updated weights for policy 1, policy_version 1879905 (0.0008) [2023-12-27 18:40:56,456][20347] Updated weights for policy 0, policy_version 1881529 (0.0007) [2023-12-27 18:40:56,504][20347] Updated weights for policy 0, policy_version 1881539 (0.0007) [2023-12-27 18:40:56,554][20347] Updated weights for policy 0, policy_version 1881549 (0.0008) [2023-12-27 18:40:56,972][20381] Updated weights for policy 1, policy_version 1879915 (0.0008) [2023-12-27 18:40:57,018][20381] Updated weights for policy 1, policy_version 1879925 (0.0010) [2023-12-27 18:40:57,065][20381] Updated weights for policy 1, policy_version 1879935 (0.0010) [2023-12-27 18:40:57,130][20347] Updated weights for policy 0, policy_version 1881559 (0.0006) [2023-12-27 18:40:57,191][20347] Updated weights for policy 0, policy_version 1881569 (0.0005) [2023-12-27 18:40:57,249][20347] Updated weights for policy 0, policy_version 1881579 (0.0008) [2023-12-27 18:40:57,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 963092480. Throughput: 0: 10378.7, 1: 10396.7. Samples: 963076512. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:40:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '886.160')] [2023-12-27 18:40:57,730][20381] Updated weights for policy 1, policy_version 1879945 (0.0007) [2023-12-27 18:40:57,784][20381] Updated weights for policy 1, policy_version 1879955 (0.0007) [2023-12-27 18:40:57,829][20381] Updated weights for policy 1, policy_version 1879965 (0.0007) [2023-12-27 18:40:57,881][20381] Updated weights for policy 1, policy_version 1879975 (0.0005) [2023-12-27 18:40:57,893][20347] Updated weights for policy 0, policy_version 1881589 (0.0008) [2023-12-27 18:40:57,947][20347] Updated weights for policy 0, policy_version 1881599 (0.0008) [2023-12-27 18:40:58,007][20347] Updated weights for policy 0, policy_version 1881609 (0.0006) [2023-12-27 18:40:58,590][20381] Updated weights for policy 1, policy_version 1879985 (0.0010) [2023-12-27 18:40:58,651][20381] Updated weights for policy 1, policy_version 1879995 (0.0009) [2023-12-27 18:40:58,654][20347] Updated weights for policy 0, policy_version 1881619 (0.0006) [2023-12-27 18:40:58,712][20381] Updated weights for policy 1, policy_version 1880005 (0.0007) [2023-12-27 18:40:58,715][20347] Updated weights for policy 0, policy_version 1881629 (0.0007) [2023-12-27 18:40:58,783][20347] Updated weights for policy 0, policy_version 1881639 (0.0008) [2023-12-27 18:40:59,446][20381] Updated weights for policy 1, policy_version 1880015 (0.0007) [2023-12-27 18:40:59,500][20381] Updated weights for policy 1, policy_version 1880025 (0.0008) [2023-12-27 18:40:59,551][20347] Updated weights for policy 0, policy_version 1881649 (0.0008) [2023-12-27 18:40:59,562][20381] Updated weights for policy 1, policy_version 1880035 (0.0007) [2023-12-27 18:40:59,613][20347] Updated weights for policy 0, policy_version 1881659 (0.0009) [2023-12-27 18:40:59,671][20347] Updated weights for policy 0, policy_version 1881669 (0.0008) [2023-12-27 18:40:59,731][20347] Updated weights for policy 0, policy_version 1881679 (0.0007) [2023-12-27 18:41:00,249][20381] Updated weights for policy 1, policy_version 1880045 (0.0008) [2023-12-27 18:41:00,309][20381] Updated weights for policy 1, policy_version 1880055 (0.0005) [2023-12-27 18:41:00,369][20381] Updated weights for policy 1, policy_version 1880065 (0.0006) [2023-12-27 18:41:00,396][20347] Updated weights for policy 0, policy_version 1881689 (0.0008) [2023-12-27 18:41:00,451][20347] Updated weights for policy 0, policy_version 1881699 (0.0008) [2023-12-27 18:41:00,502][20347] Updated weights for policy 0, policy_version 1881709 (0.0008) [2023-12-27 18:41:01,051][20381] Updated weights for policy 1, policy_version 1880075 (0.0007) [2023-12-27 18:41:01,112][20381] Updated weights for policy 1, policy_version 1880085 (0.0010) [2023-12-27 18:41:01,141][20347] Updated weights for policy 0, policy_version 1881719 (0.0006) [2023-12-27 18:41:01,176][20381] Updated weights for policy 1, policy_version 1880095 (0.0009) [2023-12-27 18:41:01,202][20347] Updated weights for policy 0, policy_version 1881729 (0.0008) [2023-12-27 18:41:01,257][20347] Updated weights for policy 0, policy_version 1881739 (0.0006) [2023-12-27 18:41:01,864][20381] Updated weights for policy 1, policy_version 1880105 (0.0008) [2023-12-27 18:41:01,913][20381] Updated weights for policy 1, policy_version 1880115 (0.0008) [2023-12-27 18:41:01,972][20347] Updated weights for policy 0, policy_version 1881749 (0.0006) [2023-12-27 18:41:01,973][20381] Updated weights for policy 1, policy_version 1880125 (0.0006) [2023-12-27 18:41:02,029][20347] Updated weights for policy 0, policy_version 1881759 (0.0006) [2023-12-27 18:41:02,030][20381] Updated weights for policy 1, policy_version 1880135 (0.0006) [2023-12-27 18:41:02,087][20347] Updated weights for policy 0, policy_version 1881769 (0.0008) [2023-12-27 18:41:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 963190784. Throughput: 0: 10351.1, 1: 10353.5. Samples: 963198424. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '868.790')] [2023-12-27 18:41:02,716][20381] Updated weights for policy 1, policy_version 1880145 (0.0008) [2023-12-27 18:41:02,764][20347] Updated weights for policy 0, policy_version 1881779 (0.0007) [2023-12-27 18:41:02,773][20381] Updated weights for policy 1, policy_version 1880155 (0.0007) [2023-12-27 18:41:02,817][20347] Updated weights for policy 0, policy_version 1881789 (0.0006) [2023-12-27 18:41:02,828][20381] Updated weights for policy 1, policy_version 1880165 (0.0007) [2023-12-27 18:41:02,867][20347] Updated weights for policy 0, policy_version 1881799 (0.0007) [2023-12-27 18:41:03,530][20381] Updated weights for policy 1, policy_version 1880175 (0.0008) [2023-12-27 18:41:03,547][20347] Updated weights for policy 0, policy_version 1881809 (0.0007) [2023-12-27 18:41:03,582][20381] Updated weights for policy 1, policy_version 1880185 (0.0007) [2023-12-27 18:41:03,599][20347] Updated weights for policy 0, policy_version 1881819 (0.0006) [2023-12-27 18:41:03,635][20381] Updated weights for policy 1, policy_version 1880195 (0.0007) [2023-12-27 18:41:03,662][20347] Updated weights for policy 0, policy_version 1881830 (0.0008) [2023-12-27 18:41:03,721][20347] Updated weights for policy 0, policy_version 1881840 (0.0008) [2023-12-27 18:41:04,359][20381] Updated weights for policy 1, policy_version 1880205 (0.0008) [2023-12-27 18:41:04,422][20381] Updated weights for policy 1, policy_version 1880215 (0.0008) [2023-12-27 18:41:04,471][20347] Updated weights for policy 0, policy_version 1881850 (0.0010) [2023-12-27 18:41:04,473][20381] Updated weights for policy 1, policy_version 1880225 (0.0007) [2023-12-27 18:41:04,529][20347] Updated weights for policy 0, policy_version 1881860 (0.0010) [2023-12-27 18:41:04,586][20347] Updated weights for policy 0, policy_version 1881870 (0.0010) [2023-12-27 18:41:05,156][20381] Updated weights for policy 1, policy_version 1880235 (0.0007) [2023-12-27 18:41:05,207][20381] Updated weights for policy 1, policy_version 1880245 (0.0008) [2023-12-27 18:41:05,254][20347] Updated weights for policy 0, policy_version 1881880 (0.0008) [2023-12-27 18:41:05,257][20381] Updated weights for policy 1, policy_version 1880255 (0.0006) [2023-12-27 18:41:05,316][20347] Updated weights for policy 0, policy_version 1881890 (0.0005) [2023-12-27 18:41:05,367][20347] Updated weights for policy 0, policy_version 1881900 (0.0005) [2023-12-27 18:41:05,853][20381] Updated weights for policy 1, policy_version 1880265 (0.0005) [2023-12-27 18:41:05,904][20381] Updated weights for policy 1, policy_version 1880275 (0.0005) [2023-12-27 18:41:05,960][20381] Updated weights for policy 1, policy_version 1880285 (0.0005) [2023-12-27 18:41:05,964][20347] Updated weights for policy 0, policy_version 1881910 (0.0006) [2023-12-27 18:41:06,015][20381] Updated weights for policy 1, policy_version 1880295 (0.0009) [2023-12-27 18:41:06,020][20347] Updated weights for policy 0, policy_version 1881920 (0.0006) [2023-12-27 18:41:06,080][20347] Updated weights for policy 0, policy_version 1881930 (0.0008) [2023-12-27 18:41:06,677][20381] Updated weights for policy 1, policy_version 1880305 (0.0006) [2023-12-27 18:41:06,737][20381] Updated weights for policy 1, policy_version 1880315 (0.0005) [2023-12-27 18:41:06,785][20347] Updated weights for policy 0, policy_version 1881940 (0.0008) [2023-12-27 18:41:06,798][20381] Updated weights for policy 1, policy_version 1880325 (0.0005) [2023-12-27 18:41:06,840][20347] Updated weights for policy 0, policy_version 1881950 (0.0007) [2023-12-27 18:41:06,897][20347] Updated weights for policy 0, policy_version 1881960 (0.0008) [2023-12-27 18:41:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 963289088. Throughput: 0: 10343.0, 1: 10349.4. Samples: 963258188. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '927.860')] [2023-12-27 18:41:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001881968_481869824.pth... [2023-12-27 18:41:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001880792_481566720.pth [2023-12-27 18:41:07,410][20381] Updated weights for policy 1, policy_version 1880335 (0.0006) [2023-12-27 18:41:07,465][20381] Updated weights for policy 1, policy_version 1880345 (0.0006) [2023-12-27 18:41:07,513][20347] Updated weights for policy 0, policy_version 1881970 (0.0006) [2023-12-27 18:41:07,523][20381] Updated weights for policy 1, policy_version 1880355 (0.0006) [2023-12-27 18:41:07,547][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001880360_481427456.pth... [2023-12-27 18:41:07,550][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001879144_481116160.pth [2023-12-27 18:41:07,571][20347] Updated weights for policy 0, policy_version 1881980 (0.0006) [2023-12-27 18:41:07,635][20347] Updated weights for policy 0, policy_version 1881990 (0.0008) [2023-12-27 18:41:07,703][20347] Updated weights for policy 0, policy_version 1882000 (0.0008) [2023-12-27 18:41:08,121][20381] Updated weights for policy 1, policy_version 1880365 (0.0008) [2023-12-27 18:41:08,179][20381] Updated weights for policy 1, policy_version 1880375 (0.0010) [2023-12-27 18:41:08,240][20381] Updated weights for policy 1, policy_version 1880385 (0.0010) [2023-12-27 18:41:08,346][20347] Updated weights for policy 0, policy_version 1882010 (0.0006) [2023-12-27 18:41:08,409][20347] Updated weights for policy 0, policy_version 1882020 (0.0006) [2023-12-27 18:41:08,468][20347] Updated weights for policy 0, policy_version 1882030 (0.0008) [2023-12-27 18:41:08,926][20381] Updated weights for policy 1, policy_version 1880395 (0.0011) [2023-12-27 18:41:08,984][20381] Updated weights for policy 1, policy_version 1880405 (0.0010) [2023-12-27 18:41:09,040][20381] Updated weights for policy 1, policy_version 1880415 (0.0010) [2023-12-27 18:41:09,154][20347] Updated weights for policy 0, policy_version 1882040 (0.0007) [2023-12-27 18:41:09,210][20347] Updated weights for policy 0, policy_version 1882050 (0.0007) [2023-12-27 18:41:09,271][20347] Updated weights for policy 0, policy_version 1882060 (0.0008) [2023-12-27 18:41:09,780][20381] Updated weights for policy 1, policy_version 1880425 (0.0008) [2023-12-27 18:41:09,837][20381] Updated weights for policy 1, policy_version 1880435 (0.0007) [2023-12-27 18:41:09,903][20381] Updated weights for policy 1, policy_version 1880445 (0.0008) [2023-12-27 18:41:09,968][20381] Updated weights for policy 1, policy_version 1880455 (0.0008) [2023-12-27 18:41:09,977][20347] Updated weights for policy 0, policy_version 1882070 (0.0008) [2023-12-27 18:41:10,035][20347] Updated weights for policy 0, policy_version 1882080 (0.0008) [2023-12-27 18:41:10,092][20347] Updated weights for policy 0, policy_version 1882090 (0.0007) [2023-12-27 18:41:10,698][20381] Updated weights for policy 1, policy_version 1880465 (0.0010) [2023-12-27 18:41:10,763][20381] Updated weights for policy 1, policy_version 1880475 (0.0011) [2023-12-27 18:41:10,777][20347] Updated weights for policy 0, policy_version 1882100 (0.0008) [2023-12-27 18:41:10,824][20381] Updated weights for policy 1, policy_version 1880485 (0.0011) [2023-12-27 18:41:10,828][20347] Updated weights for policy 0, policy_version 1882110 (0.0006) [2023-12-27 18:41:10,884][20347] Updated weights for policy 0, policy_version 1882120 (0.0006) [2023-12-27 18:41:11,507][20381] Updated weights for policy 1, policy_version 1880495 (0.0011) [2023-12-27 18:41:11,571][20381] Updated weights for policy 1, policy_version 1880505 (0.0011) [2023-12-27 18:41:11,584][20347] Updated weights for policy 0, policy_version 1882130 (0.0008) [2023-12-27 18:41:11,630][20381] Updated weights for policy 1, policy_version 1880515 (0.0010) [2023-12-27 18:41:11,650][20347] Updated weights for policy 0, policy_version 1882140 (0.0007) [2023-12-27 18:41:11,714][20347] Updated weights for policy 0, policy_version 1882150 (0.0007) [2023-12-27 18:41:11,779][20347] Updated weights for policy 0, policy_version 1882160 (0.0008) [2023-12-27 18:41:12,350][20381] Updated weights for policy 1, policy_version 1880525 (0.0009) [2023-12-27 18:41:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 963387392. Throughput: 0: 10361.1, 1: 10343.2. Samples: 963383008. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '927.220')] [2023-12-27 18:41:12,423][20381] Updated weights for policy 1, policy_version 1880535 (0.0007) [2023-12-27 18:41:12,481][20381] Updated weights for policy 1, policy_version 1880545 (0.0006) [2023-12-27 18:41:12,522][20347] Updated weights for policy 0, policy_version 1882170 (0.0008) [2023-12-27 18:41:12,586][20347] Updated weights for policy 0, policy_version 1882180 (0.0008) [2023-12-27 18:41:12,653][20347] Updated weights for policy 0, policy_version 1882190 (0.0007) [2023-12-27 18:41:13,081][20381] Updated weights for policy 1, policy_version 1880555 (0.0006) [2023-12-27 18:41:13,133][20381] Updated weights for policy 1, policy_version 1880565 (0.0006) [2023-12-27 18:41:13,186][20381] Updated weights for policy 1, policy_version 1880575 (0.0008) [2023-12-27 18:41:13,322][20347] Updated weights for policy 0, policy_version 1882200 (0.0007) [2023-12-27 18:41:13,377][20347] Updated weights for policy 0, policy_version 1882210 (0.0007) [2023-12-27 18:41:13,429][20347] Updated weights for policy 0, policy_version 1882220 (0.0007) [2023-12-27 18:41:13,895][20381] Updated weights for policy 1, policy_version 1880585 (0.0008) [2023-12-27 18:41:13,953][20381] Updated weights for policy 1, policy_version 1880595 (0.0007) [2023-12-27 18:41:14,009][20381] Updated weights for policy 1, policy_version 1880605 (0.0008) [2023-12-27 18:41:14,068][20381] Updated weights for policy 1, policy_version 1880615 (0.0006) [2023-12-27 18:41:14,070][20347] Updated weights for policy 0, policy_version 1882230 (0.0006) [2023-12-27 18:41:14,128][20347] Updated weights for policy 0, policy_version 1882240 (0.0007) [2023-12-27 18:41:14,191][20347] Updated weights for policy 0, policy_version 1882250 (0.0008) [2023-12-27 18:41:14,749][20381] Updated weights for policy 1, policy_version 1880625 (0.0007) [2023-12-27 18:41:14,775][20347] Updated weights for policy 0, policy_version 1882260 (0.0008) [2023-12-27 18:41:14,810][20381] Updated weights for policy 1, policy_version 1880635 (0.0007) [2023-12-27 18:41:14,841][20347] Updated weights for policy 0, policy_version 1882270 (0.0007) [2023-12-27 18:41:14,871][20381] Updated weights for policy 1, policy_version 1880645 (0.0007) [2023-12-27 18:41:14,903][20347] Updated weights for policy 0, policy_version 1882280 (0.0008) [2023-12-27 18:41:15,522][20347] Updated weights for policy 0, policy_version 1882290 (0.0007) [2023-12-27 18:41:15,583][20347] Updated weights for policy 0, policy_version 1882300 (0.0008) [2023-12-27 18:41:15,638][20381] Updated weights for policy 1, policy_version 1880655 (0.0006) [2023-12-27 18:41:15,640][20347] Updated weights for policy 0, policy_version 1882310 (0.0007) [2023-12-27 18:41:15,691][20381] Updated weights for policy 1, policy_version 1880665 (0.0006) [2023-12-27 18:41:15,697][20347] Updated weights for policy 0, policy_version 1882320 (0.0007) [2023-12-27 18:41:15,739][20381] Updated weights for policy 1, policy_version 1880675 (0.0007) [2023-12-27 18:41:16,321][20347] Updated weights for policy 0, policy_version 1882330 (0.0005) [2023-12-27 18:41:16,376][20347] Updated weights for policy 0, policy_version 1882340 (0.0009) [2023-12-27 18:41:16,427][20347] Updated weights for policy 0, policy_version 1882350 (0.0010) [2023-12-27 18:41:16,449][20381] Updated weights for policy 1, policy_version 1880685 (0.0008) [2023-12-27 18:41:16,506][20381] Updated weights for policy 1, policy_version 1880695 (0.0008) [2023-12-27 18:41:16,555][20381] Updated weights for policy 1, policy_version 1880705 (0.0008) [2023-12-27 18:41:17,058][20347] Updated weights for policy 0, policy_version 1882360 (0.0009) [2023-12-27 18:41:17,109][20347] Updated weights for policy 0, policy_version 1882370 (0.0006) [2023-12-27 18:41:17,167][20347] Updated weights for policy 0, policy_version 1882380 (0.0005) [2023-12-27 18:41:17,234][20381] Updated weights for policy 1, policy_version 1880715 (0.0008) [2023-12-27 18:41:17,287][20381] Updated weights for policy 1, policy_version 1880725 (0.0008) [2023-12-27 18:41:17,336][20381] Updated weights for policy 1, policy_version 1880735 (0.0007) [2023-12-27 18:41:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 963493888. Throughput: 0: 10280.0, 1: 10317.4. Samples: 963505400. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '909.800')] [2023-12-27 18:41:17,804][20347] Updated weights for policy 0, policy_version 1882390 (0.0008) [2023-12-27 18:41:17,886][20347] Updated weights for policy 0, policy_version 1882400 (0.0010) [2023-12-27 18:41:17,941][20347] Updated weights for policy 0, policy_version 1882410 (0.0010) [2023-12-27 18:41:17,990][20381] Updated weights for policy 1, policy_version 1880745 (0.0007) [2023-12-27 18:41:18,045][20381] Updated weights for policy 1, policy_version 1880755 (0.0010) [2023-12-27 18:41:18,099][20381] Updated weights for policy 1, policy_version 1880765 (0.0010) [2023-12-27 18:41:18,153][20381] Updated weights for policy 1, policy_version 1880775 (0.0011) [2023-12-27 18:41:18,590][20347] Updated weights for policy 0, policy_version 1882420 (0.0010) [2023-12-27 18:41:18,642][20347] Updated weights for policy 0, policy_version 1882430 (0.0010) [2023-12-27 18:41:18,691][20347] Updated weights for policy 0, policy_version 1882440 (0.0010) [2023-12-27 18:41:18,892][20381] Updated weights for policy 1, policy_version 1880785 (0.0011) [2023-12-27 18:41:18,955][20381] Updated weights for policy 1, policy_version 1880795 (0.0011) [2023-12-27 18:41:19,011][20381] Updated weights for policy 1, policy_version 1880805 (0.0011) [2023-12-27 18:41:19,367][20347] Updated weights for policy 0, policy_version 1882450 (0.0009) [2023-12-27 18:41:19,422][20347] Updated weights for policy 0, policy_version 1882460 (0.0009) [2023-12-27 18:41:19,480][20347] Updated weights for policy 0, policy_version 1882470 (0.0010) [2023-12-27 18:41:19,545][20347] Updated weights for policy 0, policy_version 1882480 (0.0008) [2023-12-27 18:41:19,716][20381] Updated weights for policy 1, policy_version 1880815 (0.0011) [2023-12-27 18:41:19,780][20381] Updated weights for policy 1, policy_version 1880825 (0.0011) [2023-12-27 18:41:19,847][20381] Updated weights for policy 1, policy_version 1880835 (0.0011) [2023-12-27 18:41:20,287][20347] Updated weights for policy 0, policy_version 1882490 (0.0009) [2023-12-27 18:41:20,352][20347] Updated weights for policy 0, policy_version 1882500 (0.0011) [2023-12-27 18:41:20,414][20347] Updated weights for policy 0, policy_version 1882510 (0.0006) [2023-12-27 18:41:20,540][20381] Updated weights for policy 1, policy_version 1880845 (0.0011) [2023-12-27 18:41:20,599][20381] Updated weights for policy 1, policy_version 1880855 (0.0011) [2023-12-27 18:41:20,666][20381] Updated weights for policy 1, policy_version 1880865 (0.0011) [2023-12-27 18:41:21,078][20347] Updated weights for policy 0, policy_version 1882520 (0.0009) [2023-12-27 18:41:21,141][20347] Updated weights for policy 0, policy_version 1882530 (0.0008) [2023-12-27 18:41:21,195][20347] Updated weights for policy 0, policy_version 1882540 (0.0008) [2023-12-27 18:41:21,384][20381] Updated weights for policy 1, policy_version 1880875 (0.0010) [2023-12-27 18:41:21,444][20381] Updated weights for policy 1, policy_version 1880885 (0.0008) [2023-12-27 18:41:21,506][20381] Updated weights for policy 1, policy_version 1880895 (0.0008) [2023-12-27 18:41:21,994][20347] Updated weights for policy 0, policy_version 1882550 (0.0009) [2023-12-27 18:41:22,061][20347] Updated weights for policy 0, policy_version 1882560 (0.0009) [2023-12-27 18:41:22,115][20347] Updated weights for policy 0, policy_version 1882570 (0.0009) [2023-12-27 18:41:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 963592192. Throughput: 0: 10272.2, 1: 10273.6. Samples: 963565872. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '877.660')] [2023-12-27 18:41:22,368][20381] Updated weights for policy 1, policy_version 1880905 (0.0008) [2023-12-27 18:41:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001882576_482025472.pth... [2023-12-27 18:41:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001881360_481714176.pth [2023-12-27 18:41:22,441][20381] Updated weights for policy 1, policy_version 1880915 (0.0008) [2023-12-27 18:41:22,511][20381] Updated weights for policy 1, policy_version 1880925 (0.0008) [2023-12-27 18:41:22,581][20381] Updated weights for policy 1, policy_version 1880935 (0.0008) [2023-12-27 18:41:22,586][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001880936_481574912.pth... [2023-12-27 18:41:22,591][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001879752_481271808.pth [2023-12-27 18:41:22,891][20347] Updated weights for policy 0, policy_version 1882580 (0.0009) [2023-12-27 18:41:22,945][20347] Updated weights for policy 0, policy_version 1882590 (0.0006) [2023-12-27 18:41:23,009][20347] Updated weights for policy 0, policy_version 1882600 (0.0006) [2023-12-27 18:41:23,284][20381] Updated weights for policy 1, policy_version 1880945 (0.0008) [2023-12-27 18:41:23,333][20381] Updated weights for policy 1, policy_version 1880955 (0.0007) [2023-12-27 18:41:23,377][20381] Updated weights for policy 1, policy_version 1880965 (0.0007) [2023-12-27 18:41:23,701][20347] Updated weights for policy 0, policy_version 1882610 (0.0008) [2023-12-27 18:41:23,760][20347] Updated weights for policy 0, policy_version 1882620 (0.0006) [2023-12-27 18:41:23,817][20347] Updated weights for policy 0, policy_version 1882630 (0.0008) [2023-12-27 18:41:23,869][20347] Updated weights for policy 0, policy_version 1882640 (0.0009) [2023-12-27 18:41:23,926][20381] Updated weights for policy 1, policy_version 1880975 (0.0005) [2023-12-27 18:41:23,972][20381] Updated weights for policy 1, policy_version 1880985 (0.0005) [2023-12-27 18:41:24,019][20381] Updated weights for policy 1, policy_version 1880995 (0.0005) [2023-12-27 18:41:24,490][20347] Updated weights for policy 0, policy_version 1882650 (0.0007) [2023-12-27 18:41:24,540][20347] Updated weights for policy 0, policy_version 1882660 (0.0007) [2023-12-27 18:41:24,596][20347] Updated weights for policy 0, policy_version 1882670 (0.0007) [2023-12-27 18:41:24,677][20381] Updated weights for policy 1, policy_version 1881005 (0.0006) [2023-12-27 18:41:24,726][20381] Updated weights for policy 1, policy_version 1881015 (0.0008) [2023-12-27 18:41:24,771][20381] Updated weights for policy 1, policy_version 1881025 (0.0008) [2023-12-27 18:41:25,226][20347] Updated weights for policy 0, policy_version 1882680 (0.0006) [2023-12-27 18:41:25,271][20347] Updated weights for policy 0, policy_version 1882690 (0.0005) [2023-12-27 18:41:25,321][20347] Updated weights for policy 0, policy_version 1882700 (0.0007) [2023-12-27 18:41:25,522][20381] Updated weights for policy 1, policy_version 1881035 (0.0007) [2023-12-27 18:41:25,582][20381] Updated weights for policy 1, policy_version 1881045 (0.0009) [2023-12-27 18:41:25,638][20381] Updated weights for policy 1, policy_version 1881055 (0.0008) [2023-12-27 18:41:25,947][20347] Updated weights for policy 0, policy_version 1882710 (0.0008) [2023-12-27 18:41:26,002][20347] Updated weights for policy 0, policy_version 1882720 (0.0007) [2023-12-27 18:41:26,059][20347] Updated weights for policy 0, policy_version 1882730 (0.0007) [2023-12-27 18:41:26,243][20381] Updated weights for policy 1, policy_version 1881065 (0.0008) [2023-12-27 18:41:26,297][20381] Updated weights for policy 1, policy_version 1881075 (0.0006) [2023-12-27 18:41:26,343][20381] Updated weights for policy 1, policy_version 1881085 (0.0005) [2023-12-27 18:41:26,394][20381] Updated weights for policy 1, policy_version 1881095 (0.0005) [2023-12-27 18:41:26,814][20347] Updated weights for policy 0, policy_version 1882740 (0.0010) [2023-12-27 18:41:26,872][20347] Updated weights for policy 0, policy_version 1882750 (0.0010) [2023-12-27 18:41:26,932][20347] Updated weights for policy 0, policy_version 1882760 (0.0010) [2023-12-27 18:41:26,982][20381] Updated weights for policy 1, policy_version 1881105 (0.0006) [2023-12-27 18:41:27,032][20381] Updated weights for policy 1, policy_version 1881115 (0.0007) [2023-12-27 18:41:27,076][20381] Updated weights for policy 1, policy_version 1881125 (0.0007) [2023-12-27 18:41:27,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 963698688. Throughput: 0: 10251.5, 1: 10200.3. Samples: 963686332. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:27,364][19170] Avg episode reward: [(0, '1000.000'), (1, '854.670')] [2023-12-27 18:41:27,551][20347] Updated weights for policy 0, policy_version 1882770 (0.0009) [2023-12-27 18:41:27,596][20347] Updated weights for policy 0, policy_version 1882780 (0.0007) [2023-12-27 18:41:27,641][20347] Updated weights for policy 0, policy_version 1882790 (0.0008) [2023-12-27 18:41:27,689][20347] Updated weights for policy 0, policy_version 1882800 (0.0007) [2023-12-27 18:41:27,736][20381] Updated weights for policy 1, policy_version 1881135 (0.0007) [2023-12-27 18:41:27,785][20381] Updated weights for policy 1, policy_version 1881145 (0.0007) [2023-12-27 18:41:27,839][20381] Updated weights for policy 1, policy_version 1881155 (0.0008) [2023-12-27 18:41:28,386][20347] Updated weights for policy 0, policy_version 1882810 (0.0010) [2023-12-27 18:41:28,446][20347] Updated weights for policy 0, policy_version 1882820 (0.0010) [2023-12-27 18:41:28,459][20381] Updated weights for policy 1, policy_version 1881165 (0.0007) [2023-12-27 18:41:28,501][20347] Updated weights for policy 0, policy_version 1882830 (0.0010) [2023-12-27 18:41:28,518][20381] Updated weights for policy 1, policy_version 1881175 (0.0005) [2023-12-27 18:41:28,581][20381] Updated weights for policy 1, policy_version 1881185 (0.0008) [2023-12-27 18:41:29,115][20347] Updated weights for policy 0, policy_version 1882840 (0.0007) [2023-12-27 18:41:29,171][20347] Updated weights for policy 0, policy_version 1882850 (0.0009) [2023-12-27 18:41:29,229][20347] Updated weights for policy 0, policy_version 1882860 (0.0007) [2023-12-27 18:41:29,306][20381] Updated weights for policy 1, policy_version 1881195 (0.0007) [2023-12-27 18:41:29,376][20381] Updated weights for policy 1, policy_version 1881205 (0.0010) [2023-12-27 18:41:29,436][20381] Updated weights for policy 1, policy_version 1881215 (0.0010) [2023-12-27 18:41:29,950][20347] Updated weights for policy 0, policy_version 1882870 (0.0010) [2023-12-27 18:41:30,006][20347] Updated weights for policy 0, policy_version 1882880 (0.0010) [2023-12-27 18:41:30,036][20381] Updated weights for policy 1, policy_version 1881225 (0.0010) [2023-12-27 18:41:30,065][20347] Updated weights for policy 0, policy_version 1882890 (0.0009) [2023-12-27 18:41:30,094][20381] Updated weights for policy 1, policy_version 1881235 (0.0010) [2023-12-27 18:41:30,145][20381] Updated weights for policy 1, policy_version 1881245 (0.0010) [2023-12-27 18:41:30,196][20381] Updated weights for policy 1, policy_version 1881255 (0.0010) [2023-12-27 18:41:30,745][20347] Updated weights for policy 0, policy_version 1882900 (0.0010) [2023-12-27 18:41:30,800][20347] Updated weights for policy 0, policy_version 1882910 (0.0010) [2023-12-27 18:41:30,854][20347] Updated weights for policy 0, policy_version 1882920 (0.0010) [2023-12-27 18:41:30,893][20381] Updated weights for policy 1, policy_version 1881265 (0.0011) [2023-12-27 18:41:30,951][20381] Updated weights for policy 1, policy_version 1881275 (0.0010) [2023-12-27 18:41:31,006][20381] Updated weights for policy 1, policy_version 1881285 (0.0010) [2023-12-27 18:41:31,567][20347] Updated weights for policy 0, policy_version 1882930 (0.0010) [2023-12-27 18:41:31,627][20347] Updated weights for policy 0, policy_version 1882940 (0.0010) [2023-12-27 18:41:31,693][20347] Updated weights for policy 0, policy_version 1882950 (0.0009) [2023-12-27 18:41:31,731][20381] Updated weights for policy 1, policy_version 1881295 (0.0008) [2023-12-27 18:41:31,760][20347] Updated weights for policy 0, policy_version 1882960 (0.0010) [2023-12-27 18:41:31,786][20381] Updated weights for policy 1, policy_version 1881305 (0.0009) [2023-12-27 18:41:31,844][20381] Updated weights for policy 1, policy_version 1881315 (0.0008) [2023-12-27 18:41:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 963796992. Throughput: 0: 10235.5, 1: 10164.7. Samples: 963810464. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '867.020')] [2023-12-27 18:41:32,385][20347] Updated weights for policy 0, policy_version 1882970 (0.0011) [2023-12-27 18:41:32,437][20347] Updated weights for policy 0, policy_version 1882980 (0.0010) [2023-12-27 18:41:32,485][20347] Updated weights for policy 0, policy_version 1882990 (0.0010) [2023-12-27 18:41:32,607][20381] Updated weights for policy 1, policy_version 1881325 (0.0007) [2023-12-27 18:41:32,668][20381] Updated weights for policy 1, policy_version 1881335 (0.0008) [2023-12-27 18:41:32,727][20381] Updated weights for policy 1, policy_version 1881345 (0.0006) [2023-12-27 18:41:33,175][20347] Updated weights for policy 0, policy_version 1883000 (0.0010) [2023-12-27 18:41:33,226][20347] Updated weights for policy 0, policy_version 1883010 (0.0010) [2023-12-27 18:41:33,283][20347] Updated weights for policy 0, policy_version 1883020 (0.0010) [2023-12-27 18:41:33,306][20381] Updated weights for policy 1, policy_version 1881355 (0.0005) [2023-12-27 18:41:33,356][20381] Updated weights for policy 1, policy_version 1881365 (0.0007) [2023-12-27 18:41:33,402][20381] Updated weights for policy 1, policy_version 1881375 (0.0008) [2023-12-27 18:41:33,894][20347] Updated weights for policy 0, policy_version 1883030 (0.0011) [2023-12-27 18:41:33,952][20347] Updated weights for policy 0, policy_version 1883040 (0.0010) [2023-12-27 18:41:34,012][20347] Updated weights for policy 0, policy_version 1883050 (0.0008) [2023-12-27 18:41:34,096][20381] Updated weights for policy 1, policy_version 1881385 (0.0007) [2023-12-27 18:41:34,157][20381] Updated weights for policy 1, policy_version 1881395 (0.0007) [2023-12-27 18:41:34,212][20381] Updated weights for policy 1, policy_version 1881405 (0.0006) [2023-12-27 18:41:34,271][20381] Updated weights for policy 1, policy_version 1881415 (0.0007) [2023-12-27 18:41:34,677][20347] Updated weights for policy 0, policy_version 1883060 (0.0007) [2023-12-27 18:41:34,731][20347] Updated weights for policy 0, policy_version 1883070 (0.0010) [2023-12-27 18:41:34,783][20347] Updated weights for policy 0, policy_version 1883080 (0.0010) [2023-12-27 18:41:34,909][20381] Updated weights for policy 1, policy_version 1881425 (0.0005) [2023-12-27 18:41:34,960][20381] Updated weights for policy 1, policy_version 1881435 (0.0005) [2023-12-27 18:41:35,018][20381] Updated weights for policy 1, policy_version 1881445 (0.0005) [2023-12-27 18:41:35,455][20347] Updated weights for policy 0, policy_version 1883090 (0.0010) [2023-12-27 18:41:35,512][20347] Updated weights for policy 0, policy_version 1883100 (0.0010) [2023-12-27 18:41:35,570][20347] Updated weights for policy 0, policy_version 1883110 (0.0010) [2023-12-27 18:41:35,579][20381] Updated weights for policy 1, policy_version 1881455 (0.0009) [2023-12-27 18:41:35,622][20347] Updated weights for policy 0, policy_version 1883120 (0.0010) [2023-12-27 18:41:35,632][20381] Updated weights for policy 1, policy_version 1881465 (0.0011) [2023-12-27 18:41:35,690][20381] Updated weights for policy 1, policy_version 1881475 (0.0008) [2023-12-27 18:41:36,306][20347] Updated weights for policy 0, policy_version 1883130 (0.0009) [2023-12-27 18:41:36,371][20347] Updated weights for policy 0, policy_version 1883140 (0.0005) [2023-12-27 18:41:36,371][20381] Updated weights for policy 1, policy_version 1881485 (0.0007) [2023-12-27 18:41:36,433][20381] Updated weights for policy 1, policy_version 1881495 (0.0007) [2023-12-27 18:41:36,439][20347] Updated weights for policy 0, policy_version 1883150 (0.0009) [2023-12-27 18:41:36,495][20381] Updated weights for policy 1, policy_version 1881505 (0.0007) [2023-12-27 18:41:37,070][20347] Updated weights for policy 0, policy_version 1883160 (0.0007) [2023-12-27 18:41:37,137][20347] Updated weights for policy 0, policy_version 1883170 (0.0009) [2023-12-27 18:41:37,148][20381] Updated weights for policy 1, policy_version 1881515 (0.0007) [2023-12-27 18:41:37,194][20347] Updated weights for policy 0, policy_version 1883180 (0.0006) [2023-12-27 18:41:37,211][20381] Updated weights for policy 1, policy_version 1881525 (0.0008) [2023-12-27 18:41:37,261][20381] Updated weights for policy 1, policy_version 1881535 (0.0008) [2023-12-27 18:41:37,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 963911680. Throughput: 0: 10257.2, 1: 10207.5. Samples: 963874156. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:37,363][19170] Avg episode reward: [(0, '990.230'), (1, '897.550')] [2023-12-27 18:41:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001881544_481730560.pth... [2023-12-27 18:41:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001883184_482181120.pth... [2023-12-27 18:41:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001881968_481869824.pth [2023-12-27 18:41:37,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001880360_481427456.pth [2023-12-27 18:41:37,793][20347] Updated weights for policy 0, policy_version 1883190 (0.0005) [2023-12-27 18:41:37,854][20347] Updated weights for policy 0, policy_version 1883200 (0.0007) [2023-12-27 18:41:37,893][20381] Updated weights for policy 1, policy_version 1881545 (0.0008) [2023-12-27 18:41:37,917][20347] Updated weights for policy 0, policy_version 1883210 (0.0008) [2023-12-27 18:41:37,958][20381] Updated weights for policy 1, policy_version 1881555 (0.0007) [2023-12-27 18:41:38,009][20381] Updated weights for policy 1, policy_version 1881565 (0.0007) [2023-12-27 18:41:38,064][20381] Updated weights for policy 1, policy_version 1881575 (0.0007) [2023-12-27 18:41:38,574][20347] Updated weights for policy 0, policy_version 1883220 (0.0007) [2023-12-27 18:41:38,634][20347] Updated weights for policy 0, policy_version 1883230 (0.0008) [2023-12-27 18:41:38,692][20347] Updated weights for policy 0, policy_version 1883240 (0.0007) [2023-12-27 18:41:38,781][20381] Updated weights for policy 1, policy_version 1881585 (0.0007) [2023-12-27 18:41:38,838][20381] Updated weights for policy 1, policy_version 1881595 (0.0008) [2023-12-27 18:41:38,893][20381] Updated weights for policy 1, policy_version 1881605 (0.0008) [2023-12-27 18:41:39,394][20347] Updated weights for policy 0, policy_version 1883250 (0.0009) [2023-12-27 18:41:39,455][20347] Updated weights for policy 0, policy_version 1883260 (0.0010) [2023-12-27 18:41:39,514][20347] Updated weights for policy 0, policy_version 1883270 (0.0008) [2023-12-27 18:41:39,570][20347] Updated weights for policy 0, policy_version 1883280 (0.0008) [2023-12-27 18:41:39,585][20381] Updated weights for policy 1, policy_version 1881615 (0.0008) [2023-12-27 18:41:39,641][20381] Updated weights for policy 1, policy_version 1881625 (0.0006) [2023-12-27 18:41:39,704][20381] Updated weights for policy 1, policy_version 1881635 (0.0006) [2023-12-27 18:41:40,299][20381] Updated weights for policy 1, policy_version 1881645 (0.0006) [2023-12-27 18:41:40,328][20347] Updated weights for policy 0, policy_version 1883290 (0.0006) [2023-12-27 18:41:40,358][20381] Updated weights for policy 1, policy_version 1881655 (0.0006) [2023-12-27 18:41:40,388][20347] Updated weights for policy 0, policy_version 1883300 (0.0007) [2023-12-27 18:41:40,416][20381] Updated weights for policy 1, policy_version 1881665 (0.0008) [2023-12-27 18:41:40,448][20347] Updated weights for policy 0, policy_version 1883310 (0.0007) [2023-12-27 18:41:41,066][20347] Updated weights for policy 0, policy_version 1883320 (0.0008) [2023-12-27 18:41:41,115][20381] Updated weights for policy 1, policy_version 1881675 (0.0008) [2023-12-27 18:41:41,122][20347] Updated weights for policy 0, policy_version 1883330 (0.0008) [2023-12-27 18:41:41,184][20381] Updated weights for policy 1, policy_version 1881685 (0.0008) [2023-12-27 18:41:41,190][20347] Updated weights for policy 0, policy_version 1883340 (0.0008) [2023-12-27 18:41:41,244][20381] Updated weights for policy 1, policy_version 1881695 (0.0008) [2023-12-27 18:41:41,956][20381] Updated weights for policy 1, policy_version 1881705 (0.0010) [2023-12-27 18:41:41,957][20347] Updated weights for policy 0, policy_version 1883350 (0.0007) [2023-12-27 18:41:42,012][20381] Updated weights for policy 1, policy_version 1881715 (0.0011) [2023-12-27 18:41:42,013][20347] Updated weights for policy 0, policy_version 1883360 (0.0006) [2023-12-27 18:41:42,065][20381] Updated weights for policy 1, policy_version 1881725 (0.0011) [2023-12-27 18:41:42,067][20347] Updated weights for policy 0, policy_version 1883370 (0.0006) [2023-12-27 18:41:42,126][20381] Updated weights for policy 1, policy_version 1881735 (0.0009) [2023-12-27 18:41:42,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 964009984. Throughput: 0: 10293.6, 1: 10194.7. Samples: 963998488. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '905.630')] [2023-12-27 18:41:42,725][20347] Updated weights for policy 0, policy_version 1883380 (0.0008) [2023-12-27 18:41:42,773][20381] Updated weights for policy 1, policy_version 1881745 (0.0007) [2023-12-27 18:41:42,785][20347] Updated weights for policy 0, policy_version 1883390 (0.0007) [2023-12-27 18:41:42,832][20381] Updated weights for policy 1, policy_version 1881755 (0.0006) [2023-12-27 18:41:42,852][20347] Updated weights for policy 0, policy_version 1883400 (0.0009) [2023-12-27 18:41:42,882][20381] Updated weights for policy 1, policy_version 1881765 (0.0006) [2023-12-27 18:41:43,445][20381] Updated weights for policy 1, policy_version 1881775 (0.0006) [2023-12-27 18:41:43,475][20347] Updated weights for policy 0, policy_version 1883410 (0.0009) [2023-12-27 18:41:43,507][20381] Updated weights for policy 1, policy_version 1881785 (0.0006) [2023-12-27 18:41:43,533][20347] Updated weights for policy 0, policy_version 1883420 (0.0006) [2023-12-27 18:41:43,565][20381] Updated weights for policy 1, policy_version 1881795 (0.0005) [2023-12-27 18:41:43,582][20347] Updated weights for policy 0, policy_version 1883430 (0.0008) [2023-12-27 18:41:43,631][20347] Updated weights for policy 0, policy_version 1883440 (0.0007) [2023-12-27 18:41:44,182][20381] Updated weights for policy 1, policy_version 1881805 (0.0007) [2023-12-27 18:41:44,243][20381] Updated weights for policy 1, policy_version 1881815 (0.0006) [2023-12-27 18:41:44,299][20381] Updated weights for policy 1, policy_version 1881825 (0.0005) [2023-12-27 18:41:44,314][20347] Updated weights for policy 0, policy_version 1883450 (0.0007) [2023-12-27 18:41:44,359][20347] Updated weights for policy 0, policy_version 1883460 (0.0007) [2023-12-27 18:41:44,420][20347] Updated weights for policy 0, policy_version 1883470 (0.0007) [2023-12-27 18:41:44,956][20381] Updated weights for policy 1, policy_version 1881835 (0.0006) [2023-12-27 18:41:45,006][20381] Updated weights for policy 1, policy_version 1881845 (0.0008) [2023-12-27 18:41:45,063][20381] Updated weights for policy 1, policy_version 1881855 (0.0008) [2023-12-27 18:41:45,100][20347] Updated weights for policy 0, policy_version 1883480 (0.0006) [2023-12-27 18:41:45,157][20347] Updated weights for policy 0, policy_version 1883490 (0.0007) [2023-12-27 18:41:45,215][20347] Updated weights for policy 0, policy_version 1883500 (0.0008) [2023-12-27 18:41:45,671][20381] Updated weights for policy 1, policy_version 1881865 (0.0006) [2023-12-27 18:41:45,727][20381] Updated weights for policy 1, policy_version 1881875 (0.0005) [2023-12-27 18:41:45,788][20381] Updated weights for policy 1, policy_version 1881885 (0.0005) [2023-12-27 18:41:45,840][20381] Updated weights for policy 1, policy_version 1881895 (0.0005) [2023-12-27 18:41:45,902][20347] Updated weights for policy 0, policy_version 1883510 (0.0006) [2023-12-27 18:41:45,961][20347] Updated weights for policy 0, policy_version 1883520 (0.0006) [2023-12-27 18:41:46,016][20347] Updated weights for policy 0, policy_version 1883530 (0.0007) [2023-12-27 18:41:46,450][20381] Updated weights for policy 1, policy_version 1881905 (0.0010) [2023-12-27 18:41:46,495][20381] Updated weights for policy 1, policy_version 1881915 (0.0010) [2023-12-27 18:41:46,552][20381] Updated weights for policy 1, policy_version 1881925 (0.0008) [2023-12-27 18:41:46,608][20347] Updated weights for policy 0, policy_version 1883540 (0.0009) [2023-12-27 18:41:46,673][20347] Updated weights for policy 0, policy_version 1883550 (0.0008) [2023-12-27 18:41:46,734][20347] Updated weights for policy 0, policy_version 1883560 (0.0005) [2023-12-27 18:41:47,175][20381] Updated weights for policy 1, policy_version 1881935 (0.0006) [2023-12-27 18:41:47,233][20381] Updated weights for policy 1, policy_version 1881945 (0.0005) [2023-12-27 18:41:47,296][20381] Updated weights for policy 1, policy_version 1881955 (0.0008) [2023-12-27 18:41:47,313][20347] Updated weights for policy 0, policy_version 1883570 (0.0006) [2023-12-27 18:41:47,362][20347] Updated weights for policy 0, policy_version 1883580 (0.0006) [2023-12-27 18:41:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 964116480. Throughput: 0: 10289.0, 1: 10321.3. Samples: 964125888. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.850')] [2023-12-27 18:41:47,409][20347] Updated weights for policy 0, policy_version 1883590 (0.0008) [2023-12-27 18:41:47,458][20347] Updated weights for policy 0, policy_version 1883600 (0.0008) [2023-12-27 18:41:47,823][20381] Updated weights for policy 1, policy_version 1881965 (0.0008) [2023-12-27 18:41:47,877][20381] Updated weights for policy 1, policy_version 1881975 (0.0011) [2023-12-27 18:41:47,926][20381] Updated weights for policy 1, policy_version 1881985 (0.0011) [2023-12-27 18:41:48,178][20347] Updated weights for policy 0, policy_version 1883610 (0.0007) [2023-12-27 18:41:48,226][20347] Updated weights for policy 0, policy_version 1883620 (0.0009) [2023-12-27 18:41:48,280][20347] Updated weights for policy 0, policy_version 1883630 (0.0010) [2023-12-27 18:41:48,575][20381] Updated weights for policy 1, policy_version 1881995 (0.0007) [2023-12-27 18:41:48,636][20381] Updated weights for policy 1, policy_version 1882005 (0.0010) [2023-12-27 18:41:48,697][20381] Updated weights for policy 1, policy_version 1882015 (0.0011) [2023-12-27 18:41:48,957][20347] Updated weights for policy 0, policy_version 1883640 (0.0010) [2023-12-27 18:41:49,010][20347] Updated weights for policy 0, policy_version 1883650 (0.0010) [2023-12-27 18:41:49,077][20347] Updated weights for policy 0, policy_version 1883660 (0.0009) [2023-12-27 18:41:49,356][20381] Updated weights for policy 1, policy_version 1882025 (0.0010) [2023-12-27 18:41:49,416][20381] Updated weights for policy 1, policy_version 1882035 (0.0009) [2023-12-27 18:41:49,485][20381] Updated weights for policy 1, policy_version 1882045 (0.0011) [2023-12-27 18:41:49,551][20381] Updated weights for policy 1, policy_version 1882055 (0.0010) [2023-12-27 18:41:49,720][20347] Updated weights for policy 0, policy_version 1883670 (0.0008) [2023-12-27 18:41:49,775][20347] Updated weights for policy 0, policy_version 1883680 (0.0006) [2023-12-27 18:41:49,840][20347] Updated weights for policy 0, policy_version 1883690 (0.0008) [2023-12-27 18:41:50,247][20381] Updated weights for policy 1, policy_version 1882065 (0.0010) [2023-12-27 18:41:50,300][20381] Updated weights for policy 1, policy_version 1882075 (0.0011) [2023-12-27 18:41:50,359][20381] Updated weights for policy 1, policy_version 1882085 (0.0009) [2023-12-27 18:41:50,532][20347] Updated weights for policy 0, policy_version 1883700 (0.0007) [2023-12-27 18:41:50,583][20347] Updated weights for policy 0, policy_version 1883710 (0.0008) [2023-12-27 18:41:50,639][20347] Updated weights for policy 0, policy_version 1883720 (0.0007) [2023-12-27 18:41:51,056][20381] Updated weights for policy 1, policy_version 1882095 (0.0008) [2023-12-27 18:41:51,106][20381] Updated weights for policy 1, policy_version 1882105 (0.0010) [2023-12-27 18:41:51,156][20381] Updated weights for policy 1, policy_version 1882115 (0.0008) [2023-12-27 18:41:51,298][20347] Updated weights for policy 0, policy_version 1883730 (0.0005) [2023-12-27 18:41:51,373][20347] Updated weights for policy 0, policy_version 1883740 (0.0008) [2023-12-27 18:41:51,436][20347] Updated weights for policy 0, policy_version 1883750 (0.0008) [2023-12-27 18:41:51,496][20347] Updated weights for policy 0, policy_version 1883760 (0.0008) [2023-12-27 18:41:51,922][20381] Updated weights for policy 1, policy_version 1882125 (0.0008) [2023-12-27 18:41:51,981][20381] Updated weights for policy 1, policy_version 1882135 (0.0009) [2023-12-27 18:41:52,040][20381] Updated weights for policy 1, policy_version 1882145 (0.0009) [2023-12-27 18:41:52,175][20347] Updated weights for policy 0, policy_version 1883770 (0.0005) [2023-12-27 18:41:52,230][20347] Updated weights for policy 0, policy_version 1883780 (0.0005) [2023-12-27 18:41:52,291][20347] Updated weights for policy 0, policy_version 1883790 (0.0005) [2023-12-27 18:41:52,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 964222976. Throughput: 0: 10308.4, 1: 10372.7. Samples: 964188832. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '865.330')] [2023-12-27 18:41:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001882152_481886208.pth... [2023-12-27 18:41:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001883792_482336768.pth... [2023-12-27 18:41:52,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001880936_481574912.pth [2023-12-27 18:41:52,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001882576_482025472.pth [2023-12-27 18:41:52,812][20381] Updated weights for policy 1, policy_version 1882155 (0.0009) [2023-12-27 18:41:52,867][20381] Updated weights for policy 1, policy_version 1882165 (0.0009) [2023-12-27 18:41:52,909][20347] Updated weights for policy 0, policy_version 1883800 (0.0006) [2023-12-27 18:41:52,922][20381] Updated weights for policy 1, policy_version 1882175 (0.0005) [2023-12-27 18:41:52,965][20347] Updated weights for policy 0, policy_version 1883810 (0.0006) [2023-12-27 18:41:53,019][20347] Updated weights for policy 0, policy_version 1883820 (0.0007) [2023-12-27 18:41:53,633][20347] Updated weights for policy 0, policy_version 1883830 (0.0007) [2023-12-27 18:41:53,640][20381] Updated weights for policy 1, policy_version 1882185 (0.0006) [2023-12-27 18:41:53,682][20347] Updated weights for policy 0, policy_version 1883840 (0.0005) [2023-12-27 18:41:53,695][20381] Updated weights for policy 1, policy_version 1882195 (0.0007) [2023-12-27 18:41:53,743][20347] Updated weights for policy 0, policy_version 1883850 (0.0005) [2023-12-27 18:41:53,751][20381] Updated weights for policy 1, policy_version 1882205 (0.0007) [2023-12-27 18:41:53,808][20381] Updated weights for policy 1, policy_version 1882215 (0.0007) [2023-12-27 18:41:54,275][20347] Updated weights for policy 0, policy_version 1883860 (0.0006) [2023-12-27 18:41:54,334][20347] Updated weights for policy 0, policy_version 1883870 (0.0005) [2023-12-27 18:41:54,390][20347] Updated weights for policy 0, policy_version 1883880 (0.0005) [2023-12-27 18:41:54,545][20381] Updated weights for policy 1, policy_version 1882225 (0.0007) [2023-12-27 18:41:54,596][20381] Updated weights for policy 1, policy_version 1882235 (0.0007) [2023-12-27 18:41:54,653][20381] Updated weights for policy 1, policy_version 1882245 (0.0007) [2023-12-27 18:41:54,947][20347] Updated weights for policy 0, policy_version 1883890 (0.0010) [2023-12-27 18:41:55,010][20347] Updated weights for policy 0, policy_version 1883900 (0.0008) [2023-12-27 18:41:55,076][20347] Updated weights for policy 0, policy_version 1883910 (0.0009) [2023-12-27 18:41:55,142][20347] Updated weights for policy 0, policy_version 1883920 (0.0008) [2023-12-27 18:41:55,389][20381] Updated weights for policy 1, policy_version 1882255 (0.0007) [2023-12-27 18:41:55,437][20381] Updated weights for policy 1, policy_version 1882265 (0.0008) [2023-12-27 18:41:55,481][20381] Updated weights for policy 1, policy_version 1882275 (0.0007) [2023-12-27 18:41:55,779][20347] Updated weights for policy 0, policy_version 1883930 (0.0006) [2023-12-27 18:41:55,835][20347] Updated weights for policy 0, policy_version 1883940 (0.0005) [2023-12-27 18:41:55,891][20347] Updated weights for policy 0, policy_version 1883950 (0.0008) [2023-12-27 18:41:56,173][20381] Updated weights for policy 1, policy_version 1882285 (0.0008) [2023-12-27 18:41:56,240][20381] Updated weights for policy 1, policy_version 1882295 (0.0008) [2023-12-27 18:41:56,282][20381] Updated weights for policy 1, policy_version 1882305 (0.0008) [2023-12-27 18:41:56,459][20347] Updated weights for policy 0, policy_version 1883960 (0.0006) [2023-12-27 18:41:56,517][20347] Updated weights for policy 0, policy_version 1883970 (0.0005) [2023-12-27 18:41:56,562][20347] Updated weights for policy 0, policy_version 1883980 (0.0006) [2023-12-27 18:41:56,975][20381] Updated weights for policy 1, policy_version 1882315 (0.0007) [2023-12-27 18:41:57,023][20381] Updated weights for policy 1, policy_version 1882325 (0.0007) [2023-12-27 18:41:57,078][20381] Updated weights for policy 1, policy_version 1882335 (0.0008) [2023-12-27 18:41:57,182][20347] Updated weights for policy 0, policy_version 1883990 (0.0009) [2023-12-27 18:41:57,231][20347] Updated weights for policy 0, policy_version 1884000 (0.0010) [2023-12-27 18:41:57,279][20347] Updated weights for policy 0, policy_version 1884010 (0.0010) [2023-12-27 18:41:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 964329472. Throughput: 0: 10381.2, 1: 10272.4. Samples: 964312416. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:41:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '884.310')] [2023-12-27 18:41:57,776][20381] Updated weights for policy 1, policy_version 1882345 (0.0007) [2023-12-27 18:41:57,829][20381] Updated weights for policy 1, policy_version 1882355 (0.0007) [2023-12-27 18:41:57,879][20381] Updated weights for policy 1, policy_version 1882365 (0.0008) [2023-12-27 18:41:57,928][20381] Updated weights for policy 1, policy_version 1882375 (0.0006) [2023-12-27 18:41:57,931][20347] Updated weights for policy 0, policy_version 1884020 (0.0009) [2023-12-27 18:41:57,974][20347] Updated weights for policy 0, policy_version 1884030 (0.0005) [2023-12-27 18:41:58,017][20347] Updated weights for policy 0, policy_version 1884040 (0.0007) [2023-12-27 18:41:58,609][20381] Updated weights for policy 1, policy_version 1882385 (0.0007) [2023-12-27 18:41:58,674][20381] Updated weights for policy 1, policy_version 1882395 (0.0009) [2023-12-27 18:41:58,741][20381] Updated weights for policy 1, policy_version 1882405 (0.0008) [2023-12-27 18:41:58,783][20347] Updated weights for policy 0, policy_version 1884050 (0.0008) [2023-12-27 18:41:58,854][20347] Updated weights for policy 0, policy_version 1884060 (0.0011) [2023-12-27 18:41:58,915][20347] Updated weights for policy 0, policy_version 1884070 (0.0010) [2023-12-27 18:41:58,974][20347] Updated weights for policy 0, policy_version 1884080 (0.0008) [2023-12-27 18:41:59,472][20381] Updated weights for policy 1, policy_version 1882415 (0.0008) [2023-12-27 18:41:59,519][20381] Updated weights for policy 1, policy_version 1882425 (0.0007) [2023-12-27 18:41:59,573][20381] Updated weights for policy 1, policy_version 1882435 (0.0007) [2023-12-27 18:41:59,724][20347] Updated weights for policy 0, policy_version 1884090 (0.0006) [2023-12-27 18:41:59,777][20347] Updated weights for policy 0, policy_version 1884100 (0.0011) [2023-12-27 18:41:59,838][20347] Updated weights for policy 0, policy_version 1884110 (0.0009) [2023-12-27 18:42:00,267][20381] Updated weights for policy 1, policy_version 1882445 (0.0007) [2023-12-27 18:42:00,327][20381] Updated weights for policy 1, policy_version 1882455 (0.0007) [2023-12-27 18:42:00,386][20381] Updated weights for policy 1, policy_version 1882465 (0.0008) [2023-12-27 18:42:00,489][20347] Updated weights for policy 0, policy_version 1884120 (0.0007) [2023-12-27 18:42:00,533][20347] Updated weights for policy 0, policy_version 1884130 (0.0007) [2023-12-27 18:42:00,577][20347] Updated weights for policy 0, policy_version 1884140 (0.0007) [2023-12-27 18:42:01,089][20381] Updated weights for policy 1, policy_version 1882475 (0.0007) [2023-12-27 18:42:01,159][20381] Updated weights for policy 1, policy_version 1882485 (0.0006) [2023-12-27 18:42:01,223][20381] Updated weights for policy 1, policy_version 1882495 (0.0007) [2023-12-27 18:42:01,286][20347] Updated weights for policy 0, policy_version 1884150 (0.0009) [2023-12-27 18:42:01,337][20347] Updated weights for policy 0, policy_version 1884160 (0.0010) [2023-12-27 18:42:01,398][20347] Updated weights for policy 0, policy_version 1884170 (0.0009) [2023-12-27 18:42:01,917][20381] Updated weights for policy 1, policy_version 1882505 (0.0007) [2023-12-27 18:42:01,977][20381] Updated weights for policy 1, policy_version 1882515 (0.0007) [2023-12-27 18:42:02,034][20381] Updated weights for policy 1, policy_version 1882525 (0.0009) [2023-12-27 18:42:02,064][20347] Updated weights for policy 0, policy_version 1884180 (0.0009) [2023-12-27 18:42:02,091][20381] Updated weights for policy 1, policy_version 1882535 (0.0005) [2023-12-27 18:42:02,119][20347] Updated weights for policy 0, policy_version 1884190 (0.0010) [2023-12-27 18:42:02,178][20347] Updated weights for policy 0, policy_version 1884200 (0.0007) [2023-12-27 18:42:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 964427776. Throughput: 0: 10367.4, 1: 10255.3. Samples: 964433420. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '842.060')] [2023-12-27 18:42:02,799][20381] Updated weights for policy 1, policy_version 1882545 (0.0007) [2023-12-27 18:42:02,848][20347] Updated weights for policy 0, policy_version 1884210 (0.0008) [2023-12-27 18:42:02,861][20381] Updated weights for policy 1, policy_version 1882555 (0.0008) [2023-12-27 18:42:02,899][20347] Updated weights for policy 0, policy_version 1884220 (0.0005) [2023-12-27 18:42:02,917][20381] Updated weights for policy 1, policy_version 1882565 (0.0008) [2023-12-27 18:42:02,956][20347] Updated weights for policy 0, policy_version 1884230 (0.0005) [2023-12-27 18:42:03,016][20347] Updated weights for policy 0, policy_version 1884240 (0.0005) [2023-12-27 18:42:03,567][20347] Updated weights for policy 0, policy_version 1884250 (0.0007) [2023-12-27 18:42:03,625][20347] Updated weights for policy 0, policy_version 1884260 (0.0007) [2023-12-27 18:42:03,645][20381] Updated weights for policy 1, policy_version 1882575 (0.0006) [2023-12-27 18:42:03,683][20347] Updated weights for policy 0, policy_version 1884270 (0.0010) [2023-12-27 18:42:03,699][20381] Updated weights for policy 1, policy_version 1882585 (0.0005) [2023-12-27 18:42:03,743][20381] Updated weights for policy 1, policy_version 1882595 (0.0007) [2023-12-27 18:42:04,310][20347] Updated weights for policy 0, policy_version 1884280 (0.0010) [2023-12-27 18:42:04,358][20347] Updated weights for policy 0, policy_version 1884290 (0.0009) [2023-12-27 18:42:04,400][20381] Updated weights for policy 1, policy_version 1882605 (0.0007) [2023-12-27 18:42:04,418][20347] Updated weights for policy 0, policy_version 1884300 (0.0006) [2023-12-27 18:42:04,455][20381] Updated weights for policy 1, policy_version 1882615 (0.0007) [2023-12-27 18:42:04,502][20381] Updated weights for policy 1, policy_version 1882625 (0.0007) [2023-12-27 18:42:05,119][20347] Updated weights for policy 0, policy_version 1884310 (0.0008) [2023-12-27 18:42:05,176][20347] Updated weights for policy 0, policy_version 1884320 (0.0010) [2023-12-27 18:42:05,198][20381] Updated weights for policy 1, policy_version 1882635 (0.0009) [2023-12-27 18:42:05,234][20347] Updated weights for policy 0, policy_version 1884330 (0.0010) [2023-12-27 18:42:05,258][20381] Updated weights for policy 1, policy_version 1882645 (0.0010) [2023-12-27 18:42:05,312][20381] Updated weights for policy 1, policy_version 1882655 (0.0010) [2023-12-27 18:42:05,865][20347] Updated weights for policy 0, policy_version 1884340 (0.0008) [2023-12-27 18:42:05,912][20347] Updated weights for policy 0, policy_version 1884350 (0.0006) [2023-12-27 18:42:05,956][20381] Updated weights for policy 1, policy_version 1882665 (0.0010) [2023-12-27 18:42:05,961][20347] Updated weights for policy 0, policy_version 1884360 (0.0005) [2023-12-27 18:42:06,015][20381] Updated weights for policy 1, policy_version 1882675 (0.0005) [2023-12-27 18:42:06,076][20381] Updated weights for policy 1, policy_version 1882685 (0.0006) [2023-12-27 18:42:06,141][20381] Updated weights for policy 1, policy_version 1882695 (0.0008) [2023-12-27 18:42:06,633][20347] Updated weights for policy 0, policy_version 1884370 (0.0006) [2023-12-27 18:42:06,695][20347] Updated weights for policy 0, policy_version 1884380 (0.0010) [2023-12-27 18:42:06,753][20347] Updated weights for policy 0, policy_version 1884390 (0.0010) [2023-12-27 18:42:06,808][20347] Updated weights for policy 0, policy_version 1884400 (0.0005) [2023-12-27 18:42:06,823][20381] Updated weights for policy 1, policy_version 1882705 (0.0007) [2023-12-27 18:42:06,878][20381] Updated weights for policy 1, policy_version 1882715 (0.0008) [2023-12-27 18:42:06,942][20381] Updated weights for policy 1, policy_version 1882725 (0.0007) [2023-12-27 18:42:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.6, 300 sec: 20521.7). Total num frames: 964526080. Throughput: 0: 10403.2, 1: 10263.7. Samples: 964495880. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '838.510')] [2023-12-27 18:42:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001882728_482033664.pth... [2023-12-27 18:42:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001884400_482492416.pth... [2023-12-27 18:42:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001881544_481730560.pth [2023-12-27 18:42:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001883184_482181120.pth [2023-12-27 18:42:07,450][20347] Updated weights for policy 0, policy_version 1884410 (0.0009) [2023-12-27 18:42:07,505][20347] Updated weights for policy 0, policy_version 1884420 (0.0009) [2023-12-27 18:42:07,567][20347] Updated weights for policy 0, policy_version 1884430 (0.0005) [2023-12-27 18:42:07,637][20381] Updated weights for policy 1, policy_version 1882735 (0.0009) [2023-12-27 18:42:07,689][20381] Updated weights for policy 1, policy_version 1882745 (0.0007) [2023-12-27 18:42:07,745][20381] Updated weights for policy 1, policy_version 1882755 (0.0008) [2023-12-27 18:42:08,150][20347] Updated weights for policy 0, policy_version 1884440 (0.0009) [2023-12-27 18:42:08,205][20347] Updated weights for policy 0, policy_version 1884450 (0.0010) [2023-12-27 18:42:08,250][20347] Updated weights for policy 0, policy_version 1884460 (0.0010) [2023-12-27 18:42:08,503][20381] Updated weights for policy 1, policy_version 1882765 (0.0008) [2023-12-27 18:42:08,562][20381] Updated weights for policy 1, policy_version 1882775 (0.0008) [2023-12-27 18:42:08,626][20381] Updated weights for policy 1, policy_version 1882785 (0.0008) [2023-12-27 18:42:08,885][20347] Updated weights for policy 0, policy_version 1884470 (0.0010) [2023-12-27 18:42:08,937][20347] Updated weights for policy 0, policy_version 1884480 (0.0010) [2023-12-27 18:42:08,987][20347] Updated weights for policy 0, policy_version 1884490 (0.0010) [2023-12-27 18:42:09,368][20381] Updated weights for policy 1, policy_version 1882795 (0.0008) [2023-12-27 18:42:09,434][20381] Updated weights for policy 1, policy_version 1882805 (0.0009) [2023-12-27 18:42:09,492][20381] Updated weights for policy 1, policy_version 1882815 (0.0006) [2023-12-27 18:42:09,674][20347] Updated weights for policy 0, policy_version 1884500 (0.0009) [2023-12-27 18:42:09,735][20347] Updated weights for policy 0, policy_version 1884510 (0.0011) [2023-12-27 18:42:09,796][20347] Updated weights for policy 0, policy_version 1884520 (0.0011) [2023-12-27 18:42:10,260][20381] Updated weights for policy 1, policy_version 1882825 (0.0006) [2023-12-27 18:42:10,319][20381] Updated weights for policy 1, policy_version 1882835 (0.0008) [2023-12-27 18:42:10,379][20381] Updated weights for policy 1, policy_version 1882845 (0.0007) [2023-12-27 18:42:10,435][20381] Updated weights for policy 1, policy_version 1882855 (0.0007) [2023-12-27 18:42:10,511][20347] Updated weights for policy 0, policy_version 1884530 (0.0011) [2023-12-27 18:42:10,569][20347] Updated weights for policy 0, policy_version 1884540 (0.0011) [2023-12-27 18:42:10,623][20347] Updated weights for policy 0, policy_version 1884550 (0.0009) [2023-12-27 18:42:10,681][20347] Updated weights for policy 0, policy_version 1884560 (0.0007) [2023-12-27 18:42:11,074][20381] Updated weights for policy 1, policy_version 1882865 (0.0008) [2023-12-27 18:42:11,133][20381] Updated weights for policy 1, policy_version 1882875 (0.0008) [2023-12-27 18:42:11,197][20381] Updated weights for policy 1, policy_version 1882885 (0.0008) [2023-12-27 18:42:11,433][20347] Updated weights for policy 0, policy_version 1884570 (0.0008) [2023-12-27 18:42:11,482][20347] Updated weights for policy 0, policy_version 1884580 (0.0008) [2023-12-27 18:42:11,531][20347] Updated weights for policy 0, policy_version 1884590 (0.0008) [2023-12-27 18:42:11,895][20381] Updated weights for policy 1, policy_version 1882895 (0.0009) [2023-12-27 18:42:11,955][20381] Updated weights for policy 1, policy_version 1882905 (0.0008) [2023-12-27 18:42:11,991][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000007 [2023-12-27 18:42:12,263][20347] Updated weights for policy 0, policy_version 1884600 (0.0008) [2023-12-27 18:42:12,321][20347] Updated weights for policy 0, policy_version 1884610 (0.0008) [2023-12-27 18:42:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 964624384. Throughput: 0: 10469.5, 1: 10227.4. Samples: 964617688. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '822.250')] [2023-12-27 18:42:12,384][20347] Updated weights for policy 0, policy_version 1884620 (0.0008) [2023-12-27 18:42:12,720][20381] Updated weights for policy 1, policy_version 1882915 (0.0008) [2023-12-27 18:42:12,780][20381] Updated weights for policy 1, policy_version 1882925 (0.0008) [2023-12-27 18:42:12,831][20381] Updated weights for policy 1, policy_version 1882935 (0.0009) [2023-12-27 18:42:13,066][20347] Updated weights for policy 0, policy_version 1884630 (0.0007) [2023-12-27 18:42:13,118][20347] Updated weights for policy 0, policy_version 1884640 (0.0008) [2023-12-27 18:42:13,167][20347] Updated weights for policy 0, policy_version 1884650 (0.0008) [2023-12-27 18:42:13,537][20381] Updated weights for policy 1, policy_version 1882945 (0.0008) [2023-12-27 18:42:13,599][20381] Updated weights for policy 1, policy_version 1882955 (0.0008) [2023-12-27 18:42:13,648][20381] Updated weights for policy 1, policy_version 1882965 (0.0007) [2023-12-27 18:42:13,696][20381] Updated weights for policy 1, policy_version 1882975 (0.0005) [2023-12-27 18:42:13,799][20347] Updated weights for policy 0, policy_version 1884660 (0.0006) [2023-12-27 18:42:13,855][20347] Updated weights for policy 0, policy_version 1884670 (0.0005) [2023-12-27 18:42:13,912][20347] Updated weights for policy 0, policy_version 1884680 (0.0005) [2023-12-27 18:42:14,350][20381] Updated weights for policy 1, policy_version 1882985 (0.0008) [2023-12-27 18:42:14,410][20381] Updated weights for policy 1, policy_version 1882995 (0.0007) [2023-12-27 18:42:14,467][20381] Updated weights for policy 1, policy_version 1883005 (0.0008) [2023-12-27 18:42:14,515][20347] Updated weights for policy 0, policy_version 1884690 (0.0006) [2023-12-27 18:42:14,573][20347] Updated weights for policy 0, policy_version 1884700 (0.0008) [2023-12-27 18:42:14,617][20347] Updated weights for policy 0, policy_version 1884710 (0.0007) [2023-12-27 18:42:14,677][20347] Updated weights for policy 0, policy_version 1884720 (0.0005) [2023-12-27 18:42:15,204][20381] Updated weights for policy 1, policy_version 1883015 (0.0008) [2023-12-27 18:42:15,266][20381] Updated weights for policy 1, policy_version 1883025 (0.0005) [2023-12-27 18:42:15,320][20381] Updated weights for policy 1, policy_version 1883035 (0.0005) [2023-12-27 18:42:15,351][20347] Updated weights for policy 0, policy_version 1884730 (0.0006) [2023-12-27 18:42:15,410][20347] Updated weights for policy 0, policy_version 1884740 (0.0008) [2023-12-27 18:42:15,464][20347] Updated weights for policy 0, policy_version 1884750 (0.0007) [2023-12-27 18:42:15,981][20381] Updated weights for policy 1, policy_version 1883045 (0.0005) [2023-12-27 18:42:16,043][20381] Updated weights for policy 1, policy_version 1883055 (0.0006) [2023-12-27 18:42:16,103][20381] Updated weights for policy 1, policy_version 1883065 (0.0008) [2023-12-27 18:42:16,116][20347] Updated weights for policy 0, policy_version 1884760 (0.0006) [2023-12-27 18:42:16,171][20347] Updated weights for policy 0, policy_version 1884770 (0.0010) [2023-12-27 18:42:16,229][20347] Updated weights for policy 0, policy_version 1884780 (0.0010) [2023-12-27 18:42:16,770][20381] Updated weights for policy 1, policy_version 1883075 (0.0008) [2023-12-27 18:42:16,823][20381] Updated weights for policy 1, policy_version 1883085 (0.0007) [2023-12-27 18:42:16,874][20381] Updated weights for policy 1, policy_version 1883095 (0.0007) [2023-12-27 18:42:16,898][20347] Updated weights for policy 0, policy_version 1884790 (0.0009) [2023-12-27 18:42:16,945][20347] Updated weights for policy 0, policy_version 1884800 (0.0007) [2023-12-27 18:42:17,000][20347] Updated weights for policy 0, policy_version 1884810 (0.0008) [2023-12-27 18:42:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 964730880. Throughput: 0: 10473.0, 1: 10171.1. Samples: 964739448. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.650')] [2023-12-27 18:42:17,581][20381] Updated weights for policy 1, policy_version 1883105 (0.0006) [2023-12-27 18:42:17,636][20381] Updated weights for policy 1, policy_version 1883115 (0.0007) [2023-12-27 18:42:17,681][20381] Updated weights for policy 1, policy_version 1883125 (0.0008) [2023-12-27 18:42:17,704][20347] Updated weights for policy 0, policy_version 1884820 (0.0007) [2023-12-27 18:42:17,728][20381] Updated weights for policy 1, policy_version 1883135 (0.0009) [2023-12-27 18:42:17,754][20347] Updated weights for policy 0, policy_version 1884830 (0.0008) [2023-12-27 18:42:17,802][20347] Updated weights for policy 0, policy_version 1884840 (0.0007) [2023-12-27 18:42:18,469][20381] Updated weights for policy 1, policy_version 1883145 (0.0007) [2023-12-27 18:42:18,519][20347] Updated weights for policy 0, policy_version 1884850 (0.0008) [2023-12-27 18:42:18,531][20381] Updated weights for policy 1, policy_version 1883155 (0.0009) [2023-12-27 18:42:18,574][20347] Updated weights for policy 0, policy_version 1884860 (0.0006) [2023-12-27 18:42:18,584][20381] Updated weights for policy 1, policy_version 1883165 (0.0006) [2023-12-27 18:42:18,633][20347] Updated weights for policy 0, policy_version 1884870 (0.0006) [2023-12-27 18:42:18,692][20347] Updated weights for policy 0, policy_version 1884880 (0.0008) [2023-12-27 18:42:19,327][20347] Updated weights for policy 0, policy_version 1884890 (0.0008) [2023-12-27 18:42:19,328][20381] Updated weights for policy 1, policy_version 1883175 (0.0006) [2023-12-27 18:42:19,392][20381] Updated weights for policy 1, policy_version 1883185 (0.0008) [2023-12-27 18:42:19,397][20347] Updated weights for policy 0, policy_version 1884900 (0.0008) [2023-12-27 18:42:19,447][20381] Updated weights for policy 1, policy_version 1883195 (0.0010) [2023-12-27 18:42:19,458][20347] Updated weights for policy 0, policy_version 1884910 (0.0006) [2023-12-27 18:42:20,122][20381] Updated weights for policy 1, policy_version 1883205 (0.0008) [2023-12-27 18:42:20,183][20381] Updated weights for policy 1, policy_version 1883215 (0.0011) [2023-12-27 18:42:20,188][20347] Updated weights for policy 0, policy_version 1884920 (0.0006) [2023-12-27 18:42:20,239][20381] Updated weights for policy 1, policy_version 1883225 (0.0011) [2023-12-27 18:42:20,249][20347] Updated weights for policy 0, policy_version 1884930 (0.0007) [2023-12-27 18:42:20,313][20347] Updated weights for policy 0, policy_version 1884940 (0.0006) [2023-12-27 18:42:20,977][20381] Updated weights for policy 1, policy_version 1883235 (0.0010) [2023-12-27 18:42:21,042][20381] Updated weights for policy 1, policy_version 1883245 (0.0008) [2023-12-27 18:42:21,070][20347] Updated weights for policy 0, policy_version 1884950 (0.0007) [2023-12-27 18:42:21,109][20381] Updated weights for policy 1, policy_version 1883255 (0.0008) [2023-12-27 18:42:21,134][20347] Updated weights for policy 0, policy_version 1884960 (0.0008) [2023-12-27 18:42:21,193][20347] Updated weights for policy 0, policy_version 1884970 (0.0007) [2023-12-27 18:42:21,839][20381] Updated weights for policy 1, policy_version 1883265 (0.0008) [2023-12-27 18:42:21,899][20381] Updated weights for policy 1, policy_version 1883275 (0.0007) [2023-12-27 18:42:21,932][20347] Updated weights for policy 0, policy_version 1884980 (0.0009) [2023-12-27 18:42:21,961][20381] Updated weights for policy 1, policy_version 1883285 (0.0007) [2023-12-27 18:42:21,987][20347] Updated weights for policy 0, policy_version 1884990 (0.0006) [2023-12-27 18:42:22,022][20381] Updated weights for policy 1, policy_version 1883295 (0.0007) [2023-12-27 18:42:22,045][20347] Updated weights for policy 0, policy_version 1885000 (0.0006) [2023-12-27 18:42:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 964829184. Throughput: 0: 10442.1, 1: 10107.7. Samples: 964798896. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.280')] [2023-12-27 18:42:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001883296_482181120.pth... [2023-12-27 18:42:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001885008_482648064.pth... [2023-12-27 18:42:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001882152_481886208.pth [2023-12-27 18:42:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001883792_482336768.pth [2023-12-27 18:42:22,711][20347] Updated weights for policy 0, policy_version 1885010 (0.0008) [2023-12-27 18:42:22,762][20381] Updated weights for policy 1, policy_version 1883305 (0.0006) [2023-12-27 18:42:22,773][20347] Updated weights for policy 0, policy_version 1885020 (0.0006) [2023-12-27 18:42:22,823][20381] Updated weights for policy 1, policy_version 1883315 (0.0008) [2023-12-27 18:42:22,834][20347] Updated weights for policy 0, policy_version 1885030 (0.0006) [2023-12-27 18:42:22,879][20381] Updated weights for policy 1, policy_version 1883325 (0.0010) [2023-12-27 18:42:22,893][20347] Updated weights for policy 0, policy_version 1885040 (0.0008) [2023-12-27 18:42:23,459][20347] Updated weights for policy 0, policy_version 1885050 (0.0007) [2023-12-27 18:42:23,514][20347] Updated weights for policy 0, policy_version 1885060 (0.0007) [2023-12-27 18:42:23,572][20347] Updated weights for policy 0, policy_version 1885070 (0.0008) [2023-12-27 18:42:23,605][20381] Updated weights for policy 1, policy_version 1883335 (0.0008) [2023-12-27 18:42:23,660][20381] Updated weights for policy 1, policy_version 1883345 (0.0007) [2023-12-27 18:42:23,718][20381] Updated weights for policy 1, policy_version 1883355 (0.0007) [2023-12-27 18:42:24,275][20347] Updated weights for policy 0, policy_version 1885080 (0.0007) [2023-12-27 18:42:24,328][20347] Updated weights for policy 0, policy_version 1885090 (0.0007) [2023-12-27 18:42:24,379][20347] Updated weights for policy 0, policy_version 1885100 (0.0007) [2023-12-27 18:42:24,400][20381] Updated weights for policy 1, policy_version 1883365 (0.0008) [2023-12-27 18:42:24,458][20381] Updated weights for policy 1, policy_version 1883375 (0.0005) [2023-12-27 18:42:24,514][20381] Updated weights for policy 1, policy_version 1883385 (0.0008) [2023-12-27 18:42:25,081][20347] Updated weights for policy 0, policy_version 1885110 (0.0007) [2023-12-27 18:42:25,117][20381] Updated weights for policy 1, policy_version 1883395 (0.0007) [2023-12-27 18:42:25,130][20347] Updated weights for policy 0, policy_version 1885120 (0.0006) [2023-12-27 18:42:25,180][20381] Updated weights for policy 1, policy_version 1883405 (0.0006) [2023-12-27 18:42:25,183][20347] Updated weights for policy 0, policy_version 1885130 (0.0006) [2023-12-27 18:42:25,238][20381] Updated weights for policy 1, policy_version 1883415 (0.0007) [2023-12-27 18:42:25,884][20347] Updated weights for policy 0, policy_version 1885140 (0.0006) [2023-12-27 18:42:25,912][20381] Updated weights for policy 1, policy_version 1883425 (0.0007) [2023-12-27 18:42:25,937][20347] Updated weights for policy 0, policy_version 1885150 (0.0006) [2023-12-27 18:42:25,961][20381] Updated weights for policy 1, policy_version 1883435 (0.0006) [2023-12-27 18:42:25,982][20347] Updated weights for policy 0, policy_version 1885160 (0.0005) [2023-12-27 18:42:26,010][20381] Updated weights for policy 1, policy_version 1883445 (0.0005) [2023-12-27 18:42:26,057][20381] Updated weights for policy 1, policy_version 1883455 (0.0007) [2023-12-27 18:42:26,672][20347] Updated weights for policy 0, policy_version 1885170 (0.0007) [2023-12-27 18:42:26,728][20347] Updated weights for policy 0, policy_version 1885180 (0.0007) [2023-12-27 18:42:26,744][20381] Updated weights for policy 1, policy_version 1883465 (0.0005) [2023-12-27 18:42:26,774][20347] Updated weights for policy 0, policy_version 1885190 (0.0007) [2023-12-27 18:42:26,806][20381] Updated weights for policy 1, policy_version 1883475 (0.0005) [2023-12-27 18:42:26,830][20347] Updated weights for policy 0, policy_version 1885200 (0.0007) [2023-12-27 18:42:26,860][20381] Updated weights for policy 1, policy_version 1883485 (0.0005) [2023-12-27 18:42:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 964927488. Throughput: 0: 10402.5, 1: 10045.2. Samples: 964918632. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '879.940')] [2023-12-27 18:42:27,390][20347] Updated weights for policy 0, policy_version 1885210 (0.0006) [2023-12-27 18:42:27,392][20381] Updated weights for policy 1, policy_version 1883495 (0.0005) [2023-12-27 18:42:27,438][20347] Updated weights for policy 0, policy_version 1885220 (0.0005) [2023-12-27 18:42:27,444][20381] Updated weights for policy 1, policy_version 1883505 (0.0006) [2023-12-27 18:42:27,495][20381] Updated weights for policy 1, policy_version 1883515 (0.0005) [2023-12-27 18:42:27,500][20347] Updated weights for policy 0, policy_version 1885230 (0.0007) [2023-12-27 18:42:28,135][20381] Updated weights for policy 1, policy_version 1883525 (0.0006) [2023-12-27 18:42:28,173][20347] Updated weights for policy 0, policy_version 1885240 (0.0009) [2023-12-27 18:42:28,185][20381] Updated weights for policy 1, policy_version 1883535 (0.0005) [2023-12-27 18:42:28,214][20347] Updated weights for policy 0, policy_version 1885250 (0.0010) [2023-12-27 18:42:28,239][20381] Updated weights for policy 1, policy_version 1883545 (0.0006) [2023-12-27 18:42:28,262][20347] Updated weights for policy 0, policy_version 1885260 (0.0010) [2023-12-27 18:42:28,912][20347] Updated weights for policy 0, policy_version 1885270 (0.0005) [2023-12-27 18:42:28,960][20381] Updated weights for policy 1, policy_version 1883555 (0.0006) [2023-12-27 18:42:28,969][20347] Updated weights for policy 0, policy_version 1885280 (0.0006) [2023-12-27 18:42:29,014][20381] Updated weights for policy 1, policy_version 1883565 (0.0007) [2023-12-27 18:42:29,023][20347] Updated weights for policy 0, policy_version 1885290 (0.0005) [2023-12-27 18:42:29,068][20381] Updated weights for policy 1, policy_version 1883575 (0.0007) [2023-12-27 18:42:29,670][20347] Updated weights for policy 0, policy_version 1885300 (0.0006) [2023-12-27 18:42:29,725][20347] Updated weights for policy 0, policy_version 1885310 (0.0008) [2023-12-27 18:42:29,765][20381] Updated weights for policy 1, policy_version 1883585 (0.0007) [2023-12-27 18:42:29,774][20347] Updated weights for policy 0, policy_version 1885320 (0.0010) [2023-12-27 18:42:29,825][20381] Updated weights for policy 1, policy_version 1883595 (0.0007) [2023-12-27 18:42:29,892][20381] Updated weights for policy 1, policy_version 1883605 (0.0008) [2023-12-27 18:42:29,959][20381] Updated weights for policy 1, policy_version 1883615 (0.0008) [2023-12-27 18:42:30,454][20347] Updated weights for policy 0, policy_version 1885330 (0.0007) [2023-12-27 18:42:30,502][20347] Updated weights for policy 0, policy_version 1885340 (0.0010) [2023-12-27 18:42:30,551][20347] Updated weights for policy 0, policy_version 1885350 (0.0009) [2023-12-27 18:42:30,599][20347] Updated weights for policy 0, policy_version 1885360 (0.0010) [2023-12-27 18:42:30,601][20381] Updated weights for policy 1, policy_version 1883625 (0.0006) [2023-12-27 18:42:30,659][20381] Updated weights for policy 1, policy_version 1883635 (0.0007) [2023-12-27 18:42:30,713][20381] Updated weights for policy 1, policy_version 1883645 (0.0007) [2023-12-27 18:42:31,299][20347] Updated weights for policy 0, policy_version 1885370 (0.0008) [2023-12-27 18:42:31,302][20381] Updated weights for policy 1, policy_version 1883655 (0.0005) [2023-12-27 18:42:31,351][20347] Updated weights for policy 0, policy_version 1885380 (0.0008) [2023-12-27 18:42:31,362][20381] Updated weights for policy 1, policy_version 1883665 (0.0006) [2023-12-27 18:42:31,420][20347] Updated weights for policy 0, policy_version 1885390 (0.0009) [2023-12-27 18:42:31,426][20381] Updated weights for policy 1, policy_version 1883675 (0.0007) [2023-12-27 18:42:32,128][20381] Updated weights for policy 1, policy_version 1883685 (0.0006) [2023-12-27 18:42:32,135][20347] Updated weights for policy 0, policy_version 1885400 (0.0010) [2023-12-27 18:42:32,182][20381] Updated weights for policy 1, policy_version 1883695 (0.0006) [2023-12-27 18:42:32,188][20347] Updated weights for policy 0, policy_version 1885410 (0.0009) [2023-12-27 18:42:32,237][20347] Updated weights for policy 0, policy_version 1885420 (0.0010) [2023-12-27 18:42:32,239][20381] Updated weights for policy 1, policy_version 1883705 (0.0005) [2023-12-27 18:42:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 965042176. Throughput: 0: 10428.5, 1: 10000.7. Samples: 965045204. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '874.280')] [2023-12-27 18:42:32,922][20347] Updated weights for policy 0, policy_version 1885430 (0.0010) [2023-12-27 18:42:32,944][20381] Updated weights for policy 1, policy_version 1883715 (0.0007) [2023-12-27 18:42:32,970][20347] Updated weights for policy 0, policy_version 1885440 (0.0007) [2023-12-27 18:42:33,000][20381] Updated weights for policy 1, policy_version 1883725 (0.0007) [2023-12-27 18:42:33,023][20347] Updated weights for policy 0, policy_version 1885450 (0.0010) [2023-12-27 18:42:33,057][20381] Updated weights for policy 1, policy_version 1883735 (0.0005) [2023-12-27 18:42:33,642][20347] Updated weights for policy 0, policy_version 1885460 (0.0010) [2023-12-27 18:42:33,691][20347] Updated weights for policy 0, policy_version 1885470 (0.0010) [2023-12-27 18:42:33,747][20347] Updated weights for policy 0, policy_version 1885480 (0.0010) [2023-12-27 18:42:33,761][20381] Updated weights for policy 1, policy_version 1883745 (0.0007) [2023-12-27 18:42:33,810][20381] Updated weights for policy 1, policy_version 1883755 (0.0006) [2023-12-27 18:42:33,857][20381] Updated weights for policy 1, policy_version 1883765 (0.0007) [2023-12-27 18:42:33,904][20381] Updated weights for policy 1, policy_version 1883775 (0.0007) [2023-12-27 18:42:34,477][20347] Updated weights for policy 0, policy_version 1885490 (0.0009) [2023-12-27 18:42:34,530][20347] Updated weights for policy 0, policy_version 1885500 (0.0006) [2023-12-27 18:42:34,587][20347] Updated weights for policy 0, policy_version 1885510 (0.0006) [2023-12-27 18:42:34,616][20381] Updated weights for policy 1, policy_version 1883785 (0.0006) [2023-12-27 18:42:34,645][20347] Updated weights for policy 0, policy_version 1885520 (0.0008) [2023-12-27 18:42:34,675][20381] Updated weights for policy 1, policy_version 1883795 (0.0007) [2023-12-27 18:42:34,738][20381] Updated weights for policy 1, policy_version 1883805 (0.0009) [2023-12-27 18:42:35,247][20347] Updated weights for policy 0, policy_version 1885530 (0.0008) [2023-12-27 18:42:35,299][20347] Updated weights for policy 0, policy_version 1885540 (0.0008) [2023-12-27 18:42:35,347][20347] Updated weights for policy 0, policy_version 1885550 (0.0007) [2023-12-27 18:42:35,372][20381] Updated weights for policy 1, policy_version 1883815 (0.0007) [2023-12-27 18:42:35,423][20381] Updated weights for policy 1, policy_version 1883825 (0.0008) [2023-12-27 18:42:35,471][20381] Updated weights for policy 1, policy_version 1883835 (0.0008) [2023-12-27 18:42:35,960][20347] Updated weights for policy 0, policy_version 1885560 (0.0007) [2023-12-27 18:42:36,024][20347] Updated weights for policy 0, policy_version 1885570 (0.0007) [2023-12-27 18:42:36,075][20347] Updated weights for policy 0, policy_version 1885580 (0.0007) [2023-12-27 18:42:36,215][20381] Updated weights for policy 1, policy_version 1883845 (0.0008) [2023-12-27 18:42:36,284][20381] Updated weights for policy 1, policy_version 1883855 (0.0009) [2023-12-27 18:42:36,349][20381] Updated weights for policy 1, policy_version 1883865 (0.0009) [2023-12-27 18:42:36,728][20347] Updated weights for policy 0, policy_version 1885590 (0.0008) [2023-12-27 18:42:36,776][20347] Updated weights for policy 0, policy_version 1885600 (0.0009) [2023-12-27 18:42:36,832][20347] Updated weights for policy 0, policy_version 1885610 (0.0009) [2023-12-27 18:42:36,991][20381] Updated weights for policy 1, policy_version 1883875 (0.0007) [2023-12-27 18:42:37,056][20381] Updated weights for policy 1, policy_version 1883885 (0.0006) [2023-12-27 18:42:37,116][20381] Updated weights for policy 1, policy_version 1883895 (0.0005) [2023-12-27 18:42:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 965140480. Throughput: 0: 10457.2, 1: 9957.5. Samples: 965107496. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:37,363][19170] Avg episode reward: [(0, '990.280'), (1, '896.710')] [2023-12-27 18:42:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001883904_482336768.pth... [2023-12-27 18:42:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001885616_482803712.pth... [2023-12-27 18:42:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001884400_482492416.pth [2023-12-27 18:42:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001882728_482033664.pth [2023-12-27 18:42:37,589][20347] Updated weights for policy 0, policy_version 1885620 (0.0007) [2023-12-27 18:42:37,649][20347] Updated weights for policy 0, policy_version 1885630 (0.0006) [2023-12-27 18:42:37,696][20347] Updated weights for policy 0, policy_version 1885640 (0.0008) [2023-12-27 18:42:37,703][20381] Updated weights for policy 1, policy_version 1883905 (0.0006) [2023-12-27 18:42:37,764][20381] Updated weights for policy 1, policy_version 1883915 (0.0005) [2023-12-27 18:42:37,822][20381] Updated weights for policy 1, policy_version 1883925 (0.0005) [2023-12-27 18:42:37,876][20381] Updated weights for policy 1, policy_version 1883935 (0.0005) [2023-12-27 18:42:38,334][20347] Updated weights for policy 0, policy_version 1885650 (0.0008) [2023-12-27 18:42:38,390][20381] Updated weights for policy 1, policy_version 1883945 (0.0007) [2023-12-27 18:42:38,396][20347] Updated weights for policy 0, policy_version 1885660 (0.0007) [2023-12-27 18:42:38,456][20381] Updated weights for policy 1, policy_version 1883955 (0.0008) [2023-12-27 18:42:38,459][20347] Updated weights for policy 0, policy_version 1885670 (0.0006) [2023-12-27 18:42:38,512][20381] Updated weights for policy 1, policy_version 1883965 (0.0009) [2023-12-27 18:42:38,512][20347] Updated weights for policy 0, policy_version 1885680 (0.0006) [2023-12-27 18:42:39,150][20381] Updated weights for policy 1, policy_version 1883975 (0.0006) [2023-12-27 18:42:39,207][20381] Updated weights for policy 1, policy_version 1883985 (0.0005) [2023-12-27 18:42:39,240][20347] Updated weights for policy 0, policy_version 1885690 (0.0007) [2023-12-27 18:42:39,269][20381] Updated weights for policy 1, policy_version 1883995 (0.0007) [2023-12-27 18:42:39,299][20347] Updated weights for policy 0, policy_version 1885700 (0.0007) [2023-12-27 18:42:39,363][20347] Updated weights for policy 0, policy_version 1885710 (0.0009) [2023-12-27 18:42:40,003][20347] Updated weights for policy 0, policy_version 1885720 (0.0008) [2023-12-27 18:42:40,020][20381] Updated weights for policy 1, policy_version 1884005 (0.0006) [2023-12-27 18:42:40,066][20347] Updated weights for policy 0, policy_version 1885730 (0.0008) [2023-12-27 18:42:40,081][20381] Updated weights for policy 1, policy_version 1884015 (0.0006) [2023-12-27 18:42:40,130][20347] Updated weights for policy 0, policy_version 1885740 (0.0008) [2023-12-27 18:42:40,137][20381] Updated weights for policy 1, policy_version 1884025 (0.0007) [2023-12-27 18:42:40,758][20381] Updated weights for policy 1, policy_version 1884035 (0.0008) [2023-12-27 18:42:40,801][20347] Updated weights for policy 0, policy_version 1885750 (0.0010) [2023-12-27 18:42:40,814][20381] Updated weights for policy 1, policy_version 1884045 (0.0006) [2023-12-27 18:42:40,858][20347] Updated weights for policy 0, policy_version 1885760 (0.0008) [2023-12-27 18:42:40,863][20381] Updated weights for policy 1, policy_version 1884055 (0.0006) [2023-12-27 18:42:40,916][20347] Updated weights for policy 0, policy_version 1885770 (0.0007) [2023-12-27 18:42:41,550][20381] Updated weights for policy 1, policy_version 1884065 (0.0006) [2023-12-27 18:42:41,607][20381] Updated weights for policy 1, policy_version 1884075 (0.0007) [2023-12-27 18:42:41,674][20381] Updated weights for policy 1, policy_version 1884085 (0.0007) [2023-12-27 18:42:41,719][20347] Updated weights for policy 0, policy_version 1885780 (0.0009) [2023-12-27 18:42:41,737][20381] Updated weights for policy 1, policy_version 1884095 (0.0009) [2023-12-27 18:42:41,797][20347] Updated weights for policy 0, policy_version 1885790 (0.0006) [2023-12-27 18:42:41,861][20347] Updated weights for policy 0, policy_version 1885800 (0.0008) [2023-12-27 18:42:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 965238784. Throughput: 0: 10361.1, 1: 10077.1. Samples: 965232140. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:42,363][19170] Avg episode reward: [(0, '990.280'), (1, '893.980')] [2023-12-27 18:42:42,425][20381] Updated weights for policy 1, policy_version 1884105 (0.0008) [2023-12-27 18:42:42,484][20381] Updated weights for policy 1, policy_version 1884115 (0.0008) [2023-12-27 18:42:42,536][20381] Updated weights for policy 1, policy_version 1884125 (0.0010) [2023-12-27 18:42:42,616][20347] Updated weights for policy 0, policy_version 1885810 (0.0008) [2023-12-27 18:42:42,672][20347] Updated weights for policy 0, policy_version 1885820 (0.0010) [2023-12-27 18:42:42,733][20347] Updated weights for policy 0, policy_version 1885830 (0.0010) [2023-12-27 18:42:42,786][20347] Updated weights for policy 0, policy_version 1885840 (0.0010) [2023-12-27 18:42:43,173][20381] Updated weights for policy 1, policy_version 1884135 (0.0010) [2023-12-27 18:42:43,227][20381] Updated weights for policy 1, policy_version 1884145 (0.0008) [2023-12-27 18:42:43,276][20381] Updated weights for policy 1, policy_version 1884155 (0.0005) [2023-12-27 18:42:43,446][20347] Updated weights for policy 0, policy_version 1885850 (0.0010) [2023-12-27 18:42:43,501][20347] Updated weights for policy 0, policy_version 1885860 (0.0010) [2023-12-27 18:42:43,549][20347] Updated weights for policy 0, policy_version 1885870 (0.0010) [2023-12-27 18:42:43,950][20381] Updated weights for policy 1, policy_version 1884165 (0.0008) [2023-12-27 18:42:44,004][20381] Updated weights for policy 1, policy_version 1884175 (0.0010) [2023-12-27 18:42:44,064][20381] Updated weights for policy 1, policy_version 1884185 (0.0010) [2023-12-27 18:42:44,240][20347] Updated weights for policy 0, policy_version 1885880 (0.0010) [2023-12-27 18:42:44,292][20347] Updated weights for policy 0, policy_version 1885890 (0.0010) [2023-12-27 18:42:44,343][20347] Updated weights for policy 0, policy_version 1885900 (0.0010) [2023-12-27 18:42:44,785][20381] Updated weights for policy 1, policy_version 1884195 (0.0009) [2023-12-27 18:42:44,839][20381] Updated weights for policy 1, policy_version 1884205 (0.0007) [2023-12-27 18:42:44,898][20381] Updated weights for policy 1, policy_version 1884215 (0.0007) [2023-12-27 18:42:44,987][20347] Updated weights for policy 0, policy_version 1885910 (0.0009) [2023-12-27 18:42:45,041][20347] Updated weights for policy 0, policy_version 1885920 (0.0008) [2023-12-27 18:42:45,092][20347] Updated weights for policy 0, policy_version 1885930 (0.0008) [2023-12-27 18:42:45,573][20381] Updated weights for policy 1, policy_version 1884225 (0.0007) [2023-12-27 18:42:45,634][20381] Updated weights for policy 1, policy_version 1884235 (0.0006) [2023-12-27 18:42:45,685][20381] Updated weights for policy 1, policy_version 1884245 (0.0008) [2023-12-27 18:42:45,740][20381] Updated weights for policy 1, policy_version 1884255 (0.0010) [2023-12-27 18:42:45,814][20347] Updated weights for policy 0, policy_version 1885940 (0.0007) [2023-12-27 18:42:45,860][20347] Updated weights for policy 0, policy_version 1885950 (0.0006) [2023-12-27 18:42:45,908][20347] Updated weights for policy 0, policy_version 1885960 (0.0006) [2023-12-27 18:42:46,393][20381] Updated weights for policy 1, policy_version 1884265 (0.0008) [2023-12-27 18:42:46,455][20381] Updated weights for policy 1, policy_version 1884275 (0.0009) [2023-12-27 18:42:46,512][20381] Updated weights for policy 1, policy_version 1884285 (0.0010) [2023-12-27 18:42:46,572][20347] Updated weights for policy 0, policy_version 1885970 (0.0007) [2023-12-27 18:42:46,621][20347] Updated weights for policy 0, policy_version 1885980 (0.0005) [2023-12-27 18:42:46,676][20347] Updated weights for policy 0, policy_version 1885990 (0.0005) [2023-12-27 18:42:46,739][20347] Updated weights for policy 0, policy_version 1886000 (0.0005) [2023-12-27 18:42:47,184][20381] Updated weights for policy 1, policy_version 1884295 (0.0010) [2023-12-27 18:42:47,236][20381] Updated weights for policy 1, policy_version 1884305 (0.0010) [2023-12-27 18:42:47,279][20347] Updated weights for policy 0, policy_version 1886010 (0.0007) [2023-12-27 18:42:47,288][20381] Updated weights for policy 1, policy_version 1884315 (0.0010) [2023-12-27 18:42:47,324][20347] Updated weights for policy 0, policy_version 1886020 (0.0006) [2023-12-27 18:42:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 965345280. Throughput: 0: 10336.3, 1: 10137.0. Samples: 965354724. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '795.260')] [2023-12-27 18:42:47,374][20347] Updated weights for policy 0, policy_version 1886030 (0.0007) [2023-12-27 18:42:47,884][20381] Updated weights for policy 1, policy_version 1884325 (0.0008) [2023-12-27 18:42:47,942][20381] Updated weights for policy 1, policy_version 1884335 (0.0005) [2023-12-27 18:42:47,992][20381] Updated weights for policy 1, policy_version 1884345 (0.0005) [2023-12-27 18:42:48,087][20347] Updated weights for policy 0, policy_version 1886040 (0.0005) [2023-12-27 18:42:48,142][20347] Updated weights for policy 0, policy_version 1886050 (0.0006) [2023-12-27 18:42:48,190][20347] Updated weights for policy 0, policy_version 1886060 (0.0009) [2023-12-27 18:42:48,625][20381] Updated weights for policy 1, policy_version 1884355 (0.0007) [2023-12-27 18:42:48,682][20381] Updated weights for policy 1, policy_version 1884365 (0.0009) [2023-12-27 18:42:48,743][20381] Updated weights for policy 1, policy_version 1884375 (0.0008) [2023-12-27 18:42:48,806][20347] Updated weights for policy 0, policy_version 1886070 (0.0007) [2023-12-27 18:42:48,856][20347] Updated weights for policy 0, policy_version 1886080 (0.0007) [2023-12-27 18:42:48,915][20347] Updated weights for policy 0, policy_version 1886090 (0.0007) [2023-12-27 18:42:49,502][20381] Updated weights for policy 1, policy_version 1884385 (0.0007) [2023-12-27 18:42:49,569][20381] Updated weights for policy 1, policy_version 1884395 (0.0008) [2023-12-27 18:42:49,617][20347] Updated weights for policy 0, policy_version 1886100 (0.0006) [2023-12-27 18:42:49,631][20381] Updated weights for policy 1, policy_version 1884405 (0.0007) [2023-12-27 18:42:49,671][20347] Updated weights for policy 0, policy_version 1886110 (0.0006) [2023-12-27 18:42:49,691][20381] Updated weights for policy 1, policy_version 1884415 (0.0006) [2023-12-27 18:42:49,725][20347] Updated weights for policy 0, policy_version 1886120 (0.0006) [2023-12-27 18:42:50,381][20381] Updated weights for policy 1, policy_version 1884425 (0.0008) [2023-12-27 18:42:50,442][20381] Updated weights for policy 1, policy_version 1884435 (0.0008) [2023-12-27 18:42:50,458][20347] Updated weights for policy 0, policy_version 1886130 (0.0008) [2023-12-27 18:42:50,501][20381] Updated weights for policy 1, policy_version 1884445 (0.0006) [2023-12-27 18:42:50,503][20347] Updated weights for policy 0, policy_version 1886140 (0.0005) [2023-12-27 18:42:50,553][20347] Updated weights for policy 0, policy_version 1886150 (0.0007) [2023-12-27 18:42:50,618][20347] Updated weights for policy 0, policy_version 1886160 (0.0008) [2023-12-27 18:42:51,216][20381] Updated weights for policy 1, policy_version 1884455 (0.0007) [2023-12-27 18:42:51,271][20381] Updated weights for policy 1, policy_version 1884465 (0.0007) [2023-12-27 18:42:51,325][20381] Updated weights for policy 1, policy_version 1884475 (0.0006) [2023-12-27 18:42:51,362][20347] Updated weights for policy 0, policy_version 1886170 (0.0007) [2023-12-27 18:42:51,428][20347] Updated weights for policy 0, policy_version 1886180 (0.0007) [2023-12-27 18:42:51,481][20347] Updated weights for policy 0, policy_version 1886190 (0.0007) [2023-12-27 18:42:52,050][20381] Updated weights for policy 1, policy_version 1884485 (0.0007) [2023-12-27 18:42:52,112][20381] Updated weights for policy 1, policy_version 1884495 (0.0007) [2023-12-27 18:42:52,168][20381] Updated weights for policy 1, policy_version 1884505 (0.0008) [2023-12-27 18:42:52,214][20347] Updated weights for policy 0, policy_version 1886200 (0.0010) [2023-12-27 18:42:52,270][20347] Updated weights for policy 0, policy_version 1886210 (0.0007) [2023-12-27 18:42:52,334][20347] Updated weights for policy 0, policy_version 1886220 (0.0008) [2023-12-27 18:42:52,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 965451776. Throughput: 0: 10295.7, 1: 10158.8. Samples: 965416336. Policy #0 lag: (min: 2.0, avg: 14.2, max: 34.0) [2023-12-27 18:42:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '859.000')] [2023-12-27 18:42:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001884512_482492416.pth... [2023-12-27 18:42:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001886224_482959360.pth... [2023-12-27 18:42:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001885008_482648064.pth [2023-12-27 18:42:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001883296_482181120.pth [2023-12-27 18:42:52,903][20381] Updated weights for policy 1, policy_version 1884515 (0.0010) [2023-12-27 18:42:52,956][20347] Updated weights for policy 0, policy_version 1886230 (0.0007) [2023-12-27 18:42:52,958][20381] Updated weights for policy 1, policy_version 1884525 (0.0006) [2023-12-27 18:42:53,009][20347] Updated weights for policy 0, policy_version 1886240 (0.0006) [2023-12-27 18:42:53,014][20381] Updated weights for policy 1, policy_version 1884535 (0.0008) [2023-12-27 18:42:53,063][20347] Updated weights for policy 0, policy_version 1886250 (0.0009) [2023-12-27 18:42:53,690][20381] Updated weights for policy 1, policy_version 1884545 (0.0008) [2023-12-27 18:42:53,694][20347] Updated weights for policy 0, policy_version 1886260 (0.0007) [2023-12-27 18:42:53,746][20381] Updated weights for policy 1, policy_version 1884555 (0.0008) [2023-12-27 18:42:53,750][20347] Updated weights for policy 0, policy_version 1886270 (0.0005) [2023-12-27 18:42:53,801][20347] Updated weights for policy 0, policy_version 1886280 (0.0005) [2023-12-27 18:42:53,811][20381] Updated weights for policy 1, policy_version 1884565 (0.0008) [2023-12-27 18:42:53,860][20381] Updated weights for policy 1, policy_version 1884575 (0.0007) [2023-12-27 18:42:54,376][20347] Updated weights for policy 0, policy_version 1886290 (0.0006) [2023-12-27 18:42:54,432][20347] Updated weights for policy 0, policy_version 1886300 (0.0006) [2023-12-27 18:42:54,484][20347] Updated weights for policy 0, policy_version 1886310 (0.0006) [2023-12-27 18:42:54,531][20347] Updated weights for policy 0, policy_version 1886320 (0.0005) [2023-12-27 18:42:54,581][20381] Updated weights for policy 1, policy_version 1884585 (0.0007) [2023-12-27 18:42:54,629][20381] Updated weights for policy 1, policy_version 1884595 (0.0007) [2023-12-27 18:42:54,674][20381] Updated weights for policy 1, policy_version 1884605 (0.0008) [2023-12-27 18:42:55,153][20347] Updated weights for policy 0, policy_version 1886330 (0.0008) [2023-12-27 18:42:55,214][20347] Updated weights for policy 0, policy_version 1886340 (0.0007) [2023-12-27 18:42:55,277][20347] Updated weights for policy 0, policy_version 1886350 (0.0007) [2023-12-27 18:42:55,373][20381] Updated weights for policy 1, policy_version 1884615 (0.0007) [2023-12-27 18:42:55,426][20381] Updated weights for policy 1, policy_version 1884625 (0.0008) [2023-12-27 18:42:55,479][20381] Updated weights for policy 1, policy_version 1884635 (0.0007) [2023-12-27 18:42:55,956][20347] Updated weights for policy 0, policy_version 1886360 (0.0007) [2023-12-27 18:42:56,021][20347] Updated weights for policy 0, policy_version 1886370 (0.0008) [2023-12-27 18:42:56,086][20347] Updated weights for policy 0, policy_version 1886380 (0.0006) [2023-12-27 18:42:56,134][20381] Updated weights for policy 1, policy_version 1884645 (0.0007) [2023-12-27 18:42:56,193][20381] Updated weights for policy 1, policy_version 1884655 (0.0007) [2023-12-27 18:42:56,244][20381] Updated weights for policy 1, policy_version 1884665 (0.0008) [2023-12-27 18:42:56,731][20347] Updated weights for policy 0, policy_version 1886390 (0.0007) [2023-12-27 18:42:56,772][20347] Updated weights for policy 0, policy_version 1886400 (0.0007) [2023-12-27 18:42:56,845][20347] Updated weights for policy 0, policy_version 1886410 (0.0007) [2023-12-27 18:42:56,918][20381] Updated weights for policy 1, policy_version 1884675 (0.0007) [2023-12-27 18:42:56,965][20381] Updated weights for policy 1, policy_version 1884685 (0.0007) [2023-12-27 18:42:57,019][20381] Updated weights for policy 1, policy_version 1884695 (0.0007) [2023-12-27 18:42:57,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 965550080. Throughput: 0: 10274.1, 1: 10174.4. Samples: 965537872. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:42:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.610')] [2023-12-27 18:42:57,537][20347] Updated weights for policy 0, policy_version 1886420 (0.0007) [2023-12-27 18:42:57,590][20347] Updated weights for policy 0, policy_version 1886430 (0.0007) [2023-12-27 18:42:57,644][20347] Updated weights for policy 0, policy_version 1886440 (0.0007) [2023-12-27 18:42:57,719][20381] Updated weights for policy 1, policy_version 1884705 (0.0007) [2023-12-27 18:42:57,763][20381] Updated weights for policy 1, policy_version 1884715 (0.0008) [2023-12-27 18:42:57,808][20381] Updated weights for policy 1, policy_version 1884725 (0.0008) [2023-12-27 18:42:57,856][20381] Updated weights for policy 1, policy_version 1884735 (0.0007) [2023-12-27 18:42:58,333][20347] Updated weights for policy 0, policy_version 1886450 (0.0008) [2023-12-27 18:42:58,404][20347] Updated weights for policy 0, policy_version 1886460 (0.0011) [2023-12-27 18:42:58,467][20347] Updated weights for policy 0, policy_version 1886470 (0.0007) [2023-12-27 18:42:58,529][20347] Updated weights for policy 0, policy_version 1886480 (0.0008) [2023-12-27 18:42:58,610][20381] Updated weights for policy 1, policy_version 1884745 (0.0007) [2023-12-27 18:42:58,672][20381] Updated weights for policy 1, policy_version 1884755 (0.0007) [2023-12-27 18:42:58,738][20381] Updated weights for policy 1, policy_version 1884765 (0.0007) [2023-12-27 18:42:59,352][20347] Updated weights for policy 0, policy_version 1886490 (0.0007) [2023-12-27 18:42:59,411][20347] Updated weights for policy 0, policy_version 1886500 (0.0008) [2023-12-27 18:42:59,467][20347] Updated weights for policy 0, policy_version 1886510 (0.0007) [2023-12-27 18:42:59,550][20381] Updated weights for policy 1, policy_version 1884775 (0.0007) [2023-12-27 18:42:59,611][20381] Updated weights for policy 1, policy_version 1884785 (0.0008) [2023-12-27 18:42:59,665][20381] Updated weights for policy 1, policy_version 1884795 (0.0009) [2023-12-27 18:43:00,185][20347] Updated weights for policy 0, policy_version 1886520 (0.0009) [2023-12-27 18:43:00,237][20347] Updated weights for policy 0, policy_version 1886530 (0.0009) [2023-12-27 18:43:00,288][20347] Updated weights for policy 0, policy_version 1886540 (0.0007) [2023-12-27 18:43:00,325][20381] Updated weights for policy 1, policy_version 1884805 (0.0008) [2023-12-27 18:43:00,377][20381] Updated weights for policy 1, policy_version 1884815 (0.0005) [2023-12-27 18:43:00,435][20381] Updated weights for policy 1, policy_version 1884825 (0.0006) [2023-12-27 18:43:00,949][20347] Updated weights for policy 0, policy_version 1886550 (0.0007) [2023-12-27 18:43:01,003][20347] Updated weights for policy 0, policy_version 1886560 (0.0007) [2023-12-27 18:43:01,068][20347] Updated weights for policy 0, policy_version 1886570 (0.0008) [2023-12-27 18:43:01,088][20381] Updated weights for policy 1, policy_version 1884835 (0.0008) [2023-12-27 18:43:01,148][20381] Updated weights for policy 1, policy_version 1884845 (0.0007) [2023-12-27 18:43:01,206][20381] Updated weights for policy 1, policy_version 1884855 (0.0006) [2023-12-27 18:43:01,798][20347] Updated weights for policy 0, policy_version 1886580 (0.0006) [2023-12-27 18:43:01,859][20347] Updated weights for policy 0, policy_version 1886590 (0.0005) [2023-12-27 18:43:01,913][20347] Updated weights for policy 0, policy_version 1886600 (0.0006) [2023-12-27 18:43:01,930][20381] Updated weights for policy 1, policy_version 1884865 (0.0006) [2023-12-27 18:43:01,992][20381] Updated weights for policy 1, policy_version 1884875 (0.0008) [2023-12-27 18:43:02,059][20381] Updated weights for policy 1, policy_version 1884885 (0.0009) [2023-12-27 18:43:02,120][20381] Updated weights for policy 1, policy_version 1884895 (0.0008) [2023-12-27 18:43:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 965648384. Throughput: 0: 10217.0, 1: 10164.2. Samples: 965656604. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.600')] [2023-12-27 18:43:02,544][20347] Updated weights for policy 0, policy_version 1886610 (0.0006) [2023-12-27 18:43:02,601][20347] Updated weights for policy 0, policy_version 1886620 (0.0006) [2023-12-27 18:43:02,662][20347] Updated weights for policy 0, policy_version 1886630 (0.0007) [2023-12-27 18:43:02,730][20347] Updated weights for policy 0, policy_version 1886640 (0.0006) [2023-12-27 18:43:02,780][20381] Updated weights for policy 1, policy_version 1884905 (0.0005) [2023-12-27 18:43:02,836][20381] Updated weights for policy 1, policy_version 1884915 (0.0010) [2023-12-27 18:43:02,886][20381] Updated weights for policy 1, policy_version 1884925 (0.0011) [2023-12-27 18:43:03,360][20347] Updated weights for policy 0, policy_version 1886650 (0.0008) [2023-12-27 18:43:03,412][20347] Updated weights for policy 0, policy_version 1886660 (0.0007) [2023-12-27 18:43:03,470][20347] Updated weights for policy 0, policy_version 1886670 (0.0007) [2023-12-27 18:43:03,516][20381] Updated weights for policy 1, policy_version 1884935 (0.0007) [2023-12-27 18:43:03,573][20381] Updated weights for policy 1, policy_version 1884945 (0.0005) [2023-12-27 18:43:03,631][20381] Updated weights for policy 1, policy_version 1884955 (0.0005) [2023-12-27 18:43:04,193][20381] Updated weights for policy 1, policy_version 1884965 (0.0008) [2023-12-27 18:43:04,202][20347] Updated weights for policy 0, policy_version 1886680 (0.0010) [2023-12-27 18:43:04,254][20381] Updated weights for policy 1, policy_version 1884975 (0.0007) [2023-12-27 18:43:04,258][20347] Updated weights for policy 0, policy_version 1886690 (0.0009) [2023-12-27 18:43:04,314][20347] Updated weights for policy 0, policy_version 1886700 (0.0009) [2023-12-27 18:43:04,317][20381] Updated weights for policy 1, policy_version 1884985 (0.0006) [2023-12-27 18:43:04,975][20347] Updated weights for policy 0, policy_version 1886710 (0.0010) [2023-12-27 18:43:04,993][20381] Updated weights for policy 1, policy_version 1884995 (0.0006) [2023-12-27 18:43:05,027][20347] Updated weights for policy 0, policy_version 1886720 (0.0010) [2023-12-27 18:43:05,057][20381] Updated weights for policy 1, policy_version 1885005 (0.0006) [2023-12-27 18:43:05,086][20347] Updated weights for policy 0, policy_version 1886730 (0.0010) [2023-12-27 18:43:05,118][20381] Updated weights for policy 1, policy_version 1885015 (0.0006) [2023-12-27 18:43:05,701][20381] Updated weights for policy 1, policy_version 1885025 (0.0005) [2023-12-27 18:43:05,740][20347] Updated weights for policy 0, policy_version 1886740 (0.0010) [2023-12-27 18:43:05,757][20381] Updated weights for policy 1, policy_version 1885035 (0.0006) [2023-12-27 18:43:05,797][20347] Updated weights for policy 0, policy_version 1886750 (0.0008) [2023-12-27 18:43:05,814][20381] Updated weights for policy 1, policy_version 1885045 (0.0006) [2023-12-27 18:43:05,856][20347] Updated weights for policy 0, policy_version 1886760 (0.0005) [2023-12-27 18:43:05,866][20381] Updated weights for policy 1, policy_version 1885055 (0.0006) [2023-12-27 18:43:06,508][20347] Updated weights for policy 0, policy_version 1886770 (0.0007) [2023-12-27 18:43:06,554][20381] Updated weights for policy 1, policy_version 1885065 (0.0008) [2023-12-27 18:43:06,573][20347] Updated weights for policy 0, policy_version 1886780 (0.0010) [2023-12-27 18:43:06,613][20381] Updated weights for policy 1, policy_version 1885075 (0.0006) [2023-12-27 18:43:06,630][20347] Updated weights for policy 0, policy_version 1886790 (0.0011) [2023-12-27 18:43:06,672][20381] Updated weights for policy 1, policy_version 1885085 (0.0008) [2023-12-27 18:43:06,696][20347] Updated weights for policy 0, policy_version 1886800 (0.0010) [2023-12-27 18:43:07,267][20381] Updated weights for policy 1, policy_version 1885095 (0.0008) [2023-12-27 18:43:07,320][20381] Updated weights for policy 1, policy_version 1885105 (0.0006) [2023-12-27 18:43:07,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 965746688. Throughput: 0: 10229.6, 1: 10230.4. Samples: 965719600. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '865.270')] [2023-12-27 18:43:07,365][20381] Updated weights for policy 1, policy_version 1885115 (0.0006) [2023-12-27 18:43:07,389][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001885120_482648064.pth... [2023-12-27 18:43:07,391][20347] Updated weights for policy 0, policy_version 1886810 (0.0009) [2023-12-27 18:43:07,392][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001883904_482336768.pth [2023-12-27 18:43:07,443][20347] Updated weights for policy 0, policy_version 1886820 (0.0009) [2023-12-27 18:43:07,489][20347] Updated weights for policy 0, policy_version 1886830 (0.0007) [2023-12-27 18:43:07,500][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001886832_483115008.pth... [2023-12-27 18:43:07,504][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001885616_482803712.pth [2023-12-27 18:43:08,103][20381] Updated weights for policy 1, policy_version 1885125 (0.0008) [2023-12-27 18:43:08,134][20347] Updated weights for policy 0, policy_version 1886840 (0.0007) [2023-12-27 18:43:08,166][20381] Updated weights for policy 1, policy_version 1885135 (0.0007) [2023-12-27 18:43:08,181][20347] Updated weights for policy 0, policy_version 1886850 (0.0005) [2023-12-27 18:43:08,232][20381] Updated weights for policy 1, policy_version 1885145 (0.0007) [2023-12-27 18:43:08,234][20347] Updated weights for policy 0, policy_version 1886860 (0.0006) [2023-12-27 18:43:08,883][20347] Updated weights for policy 0, policy_version 1886870 (0.0006) [2023-12-27 18:43:08,936][20381] Updated weights for policy 1, policy_version 1885155 (0.0007) [2023-12-27 18:43:08,942][20347] Updated weights for policy 0, policy_version 1886880 (0.0010) [2023-12-27 18:43:08,992][20381] Updated weights for policy 1, policy_version 1885165 (0.0005) [2023-12-27 18:43:09,006][20347] Updated weights for policy 0, policy_version 1886890 (0.0010) [2023-12-27 18:43:09,049][20381] Updated weights for policy 1, policy_version 1885175 (0.0005) [2023-12-27 18:43:09,689][20347] Updated weights for policy 0, policy_version 1886900 (0.0009) [2023-12-27 18:43:09,743][20347] Updated weights for policy 0, policy_version 1886910 (0.0007) [2023-12-27 18:43:09,803][20347] Updated weights for policy 0, policy_version 1886920 (0.0006) [2023-12-27 18:43:09,809][20381] Updated weights for policy 1, policy_version 1885185 (0.0008) [2023-12-27 18:43:09,876][20381] Updated weights for policy 1, policy_version 1885195 (0.0007) [2023-12-27 18:43:09,935][20381] Updated weights for policy 1, policy_version 1885205 (0.0008) [2023-12-27 18:43:09,994][20381] Updated weights for policy 1, policy_version 1885215 (0.0007) [2023-12-27 18:43:10,550][20347] Updated weights for policy 0, policy_version 1886930 (0.0007) [2023-12-27 18:43:10,598][20347] Updated weights for policy 0, policy_version 1886940 (0.0008) [2023-12-27 18:43:10,659][20347] Updated weights for policy 0, policy_version 1886950 (0.0007) [2023-12-27 18:43:10,687][20381] Updated weights for policy 1, policy_version 1885225 (0.0005) [2023-12-27 18:43:10,716][20347] Updated weights for policy 0, policy_version 1886960 (0.0007) [2023-12-27 18:43:10,743][20381] Updated weights for policy 1, policy_version 1885235 (0.0005) [2023-12-27 18:43:10,797][20381] Updated weights for policy 1, policy_version 1885245 (0.0005) [2023-12-27 18:43:11,478][20381] Updated weights for policy 1, policy_version 1885255 (0.0007) [2023-12-27 18:43:11,494][20347] Updated weights for policy 0, policy_version 1886970 (0.0006) [2023-12-27 18:43:11,534][20381] Updated weights for policy 1, policy_version 1885265 (0.0008) [2023-12-27 18:43:11,557][20347] Updated weights for policy 0, policy_version 1886980 (0.0007) [2023-12-27 18:43:11,592][20381] Updated weights for policy 1, policy_version 1885275 (0.0006) [2023-12-27 18:43:11,618][20347] Updated weights for policy 0, policy_version 1886990 (0.0007) [2023-12-27 18:43:12,295][20381] Updated weights for policy 1, policy_version 1885285 (0.0007) [2023-12-27 18:43:12,333][20347] Updated weights for policy 0, policy_version 1887000 (0.0008) [2023-12-27 18:43:12,360][20381] Updated weights for policy 1, policy_version 1885295 (0.0008) [2023-12-27 18:43:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 965844992. Throughput: 0: 10243.4, 1: 10267.1. Samples: 965841604. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '829.340')] [2023-12-27 18:43:12,398][20347] Updated weights for policy 0, policy_version 1887010 (0.0010) [2023-12-27 18:43:12,407][20381] Updated weights for policy 1, policy_version 1885305 (0.0007) [2023-12-27 18:43:12,460][20347] Updated weights for policy 0, policy_version 1887020 (0.0010) [2023-12-27 18:43:13,064][20347] Updated weights for policy 0, policy_version 1887030 (0.0007) [2023-12-27 18:43:13,115][20347] Updated weights for policy 0, policy_version 1887040 (0.0005) [2023-12-27 18:43:13,143][20381] Updated weights for policy 1, policy_version 1885315 (0.0006) [2023-12-27 18:43:13,172][20347] Updated weights for policy 0, policy_version 1887050 (0.0006) [2023-12-27 18:43:13,197][20381] Updated weights for policy 1, policy_version 1885325 (0.0005) [2023-12-27 18:43:13,247][20381] Updated weights for policy 1, policy_version 1885335 (0.0008) [2023-12-27 18:43:13,762][20347] Updated weights for policy 0, policy_version 1887060 (0.0006) [2023-12-27 18:43:13,822][20347] Updated weights for policy 0, policy_version 1887070 (0.0005) [2023-12-27 18:43:13,882][20347] Updated weights for policy 0, policy_version 1887080 (0.0005) [2023-12-27 18:43:14,001][20381] Updated weights for policy 1, policy_version 1885345 (0.0008) [2023-12-27 18:43:14,064][20381] Updated weights for policy 1, policy_version 1885355 (0.0005) [2023-12-27 18:43:14,126][20381] Updated weights for policy 1, policy_version 1885365 (0.0005) [2023-12-27 18:43:14,191][20381] Updated weights for policy 1, policy_version 1885375 (0.0008) [2023-12-27 18:43:14,461][20347] Updated weights for policy 0, policy_version 1887090 (0.0007) [2023-12-27 18:43:14,509][20347] Updated weights for policy 0, policy_version 1887100 (0.0010) [2023-12-27 18:43:14,560][20347] Updated weights for policy 0, policy_version 1887110 (0.0009) [2023-12-27 18:43:14,617][20347] Updated weights for policy 0, policy_version 1887120 (0.0010) [2023-12-27 18:43:14,841][20381] Updated weights for policy 1, policy_version 1885385 (0.0008) [2023-12-27 18:43:14,897][20381] Updated weights for policy 1, policy_version 1885395 (0.0008) [2023-12-27 18:43:14,958][20381] Updated weights for policy 1, policy_version 1885405 (0.0007) [2023-12-27 18:43:15,323][20347] Updated weights for policy 0, policy_version 1887130 (0.0010) [2023-12-27 18:43:15,387][20347] Updated weights for policy 0, policy_version 1887140 (0.0010) [2023-12-27 18:43:15,448][20347] Updated weights for policy 0, policy_version 1887150 (0.0010) [2023-12-27 18:43:15,659][20381] Updated weights for policy 1, policy_version 1885415 (0.0008) [2023-12-27 18:43:15,704][20381] Updated weights for policy 1, policy_version 1885425 (0.0006) [2023-12-27 18:43:15,752][20381] Updated weights for policy 1, policy_version 1885435 (0.0005) [2023-12-27 18:43:16,065][20347] Updated weights for policy 0, policy_version 1887160 (0.0006) [2023-12-27 18:43:16,122][20347] Updated weights for policy 0, policy_version 1887170 (0.0005) [2023-12-27 18:43:16,170][20347] Updated weights for policy 0, policy_version 1887180 (0.0005) [2023-12-27 18:43:16,456][20381] Updated weights for policy 1, policy_version 1885445 (0.0008) [2023-12-27 18:43:16,502][20381] Updated weights for policy 1, policy_version 1885455 (0.0008) [2023-12-27 18:43:16,553][20381] Updated weights for policy 1, policy_version 1885465 (0.0007) [2023-12-27 18:43:16,733][20347] Updated weights for policy 0, policy_version 1887190 (0.0005) [2023-12-27 18:43:16,799][20347] Updated weights for policy 0, policy_version 1887200 (0.0005) [2023-12-27 18:43:16,859][20347] Updated weights for policy 0, policy_version 1887210 (0.0005) [2023-12-27 18:43:17,300][20381] Updated weights for policy 1, policy_version 1885475 (0.0007) [2023-12-27 18:43:17,362][20381] Updated weights for policy 1, policy_version 1885485 (0.0008) [2023-12-27 18:43:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 965951488. Throughput: 0: 10292.5, 1: 10165.2. Samples: 965965800. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '840.260')] [2023-12-27 18:43:17,413][20381] Updated weights for policy 1, policy_version 1885495 (0.0007) [2023-12-27 18:43:17,452][20347] Updated weights for policy 0, policy_version 1887220 (0.0008) [2023-12-27 18:43:17,507][20347] Updated weights for policy 0, policy_version 1887230 (0.0010) [2023-12-27 18:43:17,561][20347] Updated weights for policy 0, policy_version 1887240 (0.0009) [2023-12-27 18:43:18,115][20347] Updated weights for policy 0, policy_version 1887250 (0.0006) [2023-12-27 18:43:18,160][20347] Updated weights for policy 0, policy_version 1887260 (0.0010) [2023-12-27 18:43:18,161][20381] Updated weights for policy 1, policy_version 1885505 (0.0007) [2023-12-27 18:43:18,216][20347] Updated weights for policy 0, policy_version 1887270 (0.0010) [2023-12-27 18:43:18,222][20381] Updated weights for policy 1, policy_version 1885515 (0.0007) [2023-12-27 18:43:18,267][20381] Updated weights for policy 1, policy_version 1885525 (0.0008) [2023-12-27 18:43:18,274][20347] Updated weights for policy 0, policy_version 1887280 (0.0010) [2023-12-27 18:43:18,315][20381] Updated weights for policy 1, policy_version 1885535 (0.0007) [2023-12-27 18:43:18,963][20347] Updated weights for policy 0, policy_version 1887290 (0.0010) [2023-12-27 18:43:18,999][20381] Updated weights for policy 1, policy_version 1885545 (0.0010) [2023-12-27 18:43:19,020][20347] Updated weights for policy 0, policy_version 1887300 (0.0008) [2023-12-27 18:43:19,050][20381] Updated weights for policy 1, policy_version 1885555 (0.0010) [2023-12-27 18:43:19,078][20347] Updated weights for policy 0, policy_version 1887310 (0.0009) [2023-12-27 18:43:19,108][20381] Updated weights for policy 1, policy_version 1885565 (0.0006) [2023-12-27 18:43:19,759][20347] Updated weights for policy 0, policy_version 1887320 (0.0007) [2023-12-27 18:43:19,815][20381] Updated weights for policy 1, policy_version 1885575 (0.0009) [2023-12-27 18:43:19,819][20347] Updated weights for policy 0, policy_version 1887330 (0.0008) [2023-12-27 18:43:19,869][20381] Updated weights for policy 1, policy_version 1885585 (0.0010) [2023-12-27 18:43:19,883][20347] Updated weights for policy 0, policy_version 1887340 (0.0008) [2023-12-27 18:43:19,930][20381] Updated weights for policy 1, policy_version 1885595 (0.0011) [2023-12-27 18:43:20,615][20381] Updated weights for policy 1, policy_version 1885605 (0.0011) [2023-12-27 18:43:20,668][20381] Updated weights for policy 1, policy_version 1885615 (0.0011) [2023-12-27 18:43:20,678][20347] Updated weights for policy 0, policy_version 1887350 (0.0007) [2023-12-27 18:43:20,722][20381] Updated weights for policy 1, policy_version 1885625 (0.0009) [2023-12-27 18:43:20,741][20347] Updated weights for policy 0, policy_version 1887360 (0.0006) [2023-12-27 18:43:20,798][20347] Updated weights for policy 0, policy_version 1887370 (0.0008) [2023-12-27 18:43:21,471][20381] Updated weights for policy 1, policy_version 1885635 (0.0008) [2023-12-27 18:43:21,524][20381] Updated weights for policy 1, policy_version 1885645 (0.0007) [2023-12-27 18:43:21,569][20347] Updated weights for policy 0, policy_version 1887380 (0.0007) [2023-12-27 18:43:21,578][20381] Updated weights for policy 1, policy_version 1885655 (0.0006) [2023-12-27 18:43:21,632][20347] Updated weights for policy 0, policy_version 1887390 (0.0007) [2023-12-27 18:43:21,700][20347] Updated weights for policy 0, policy_version 1887400 (0.0007) [2023-12-27 18:43:22,251][20381] Updated weights for policy 1, policy_version 1885665 (0.0007) [2023-12-27 18:43:22,318][20381] Updated weights for policy 1, policy_version 1885675 (0.0008) [2023-12-27 18:43:22,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 966049792. Throughput: 0: 10266.6, 1: 10144.3. Samples: 966025984. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '889.260')] [2023-12-27 18:43:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001887408_483262464.pth... [2023-12-27 18:43:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001886224_482959360.pth [2023-12-27 18:43:22,405][20381] Updated weights for policy 1, policy_version 1885685 (0.0007) [2023-12-27 18:43:22,429][20347] Updated weights for policy 0, policy_version 1887410 (0.0008) [2023-12-27 18:43:22,469][20381] Updated weights for policy 1, policy_version 1885695 (0.0005) [2023-12-27 18:43:22,474][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001885696_482795520.pth... [2023-12-27 18:43:22,478][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001884512_482492416.pth [2023-12-27 18:43:22,488][20347] Updated weights for policy 0, policy_version 1887420 (0.0008) [2023-12-27 18:43:22,543][20347] Updated weights for policy 0, policy_version 1887430 (0.0008) [2023-12-27 18:43:22,599][20347] Updated weights for policy 0, policy_version 1887440 (0.0008) [2023-12-27 18:43:23,011][20381] Updated weights for policy 1, policy_version 1885705 (0.0007) [2023-12-27 18:43:23,069][20381] Updated weights for policy 1, policy_version 1885715 (0.0008) [2023-12-27 18:43:23,118][20381] Updated weights for policy 1, policy_version 1885725 (0.0007) [2023-12-27 18:43:23,322][20347] Updated weights for policy 0, policy_version 1887450 (0.0010) [2023-12-27 18:43:23,370][20347] Updated weights for policy 0, policy_version 1887460 (0.0010) [2023-12-27 18:43:23,416][20347] Updated weights for policy 0, policy_version 1887470 (0.0010) [2023-12-27 18:43:23,793][20381] Updated weights for policy 1, policy_version 1885735 (0.0007) [2023-12-27 18:43:23,849][20381] Updated weights for policy 1, policy_version 1885745 (0.0008) [2023-12-27 18:43:23,903][20381] Updated weights for policy 1, policy_version 1885755 (0.0007) [2023-12-27 18:43:24,127][20347] Updated weights for policy 0, policy_version 1887480 (0.0009) [2023-12-27 18:43:24,181][20347] Updated weights for policy 0, policy_version 1887490 (0.0008) [2023-12-27 18:43:24,241][20347] Updated weights for policy 0, policy_version 1887500 (0.0008) [2023-12-27 18:43:24,582][20381] Updated weights for policy 1, policy_version 1885765 (0.0008) [2023-12-27 18:43:24,630][20381] Updated weights for policy 1, policy_version 1885775 (0.0010) [2023-12-27 18:43:24,678][20381] Updated weights for policy 1, policy_version 1885785 (0.0010) [2023-12-27 18:43:24,858][20347] Updated weights for policy 0, policy_version 1887510 (0.0007) [2023-12-27 18:43:24,928][20347] Updated weights for policy 0, policy_version 1887520 (0.0007) [2023-12-27 18:43:24,985][20347] Updated weights for policy 0, policy_version 1887530 (0.0007) [2023-12-27 18:43:25,357][20381] Updated weights for policy 1, policy_version 1885795 (0.0009) [2023-12-27 18:43:25,411][20381] Updated weights for policy 1, policy_version 1885805 (0.0006) [2023-12-27 18:43:25,470][20381] Updated weights for policy 1, policy_version 1885815 (0.0008) [2023-12-27 18:43:25,593][20347] Updated weights for policy 0, policy_version 1887540 (0.0007) [2023-12-27 18:43:25,653][20347] Updated weights for policy 0, policy_version 1887550 (0.0005) [2023-12-27 18:43:25,709][20347] Updated weights for policy 0, policy_version 1887560 (0.0005) [2023-12-27 18:43:26,202][20381] Updated weights for policy 1, policy_version 1885825 (0.0009) [2023-12-27 18:43:26,258][20381] Updated weights for policy 1, policy_version 1885835 (0.0010) [2023-12-27 18:43:26,293][20347] Updated weights for policy 0, policy_version 1887570 (0.0005) [2023-12-27 18:43:26,309][20381] Updated weights for policy 1, policy_version 1885845 (0.0010) [2023-12-27 18:43:26,350][20347] Updated weights for policy 0, policy_version 1887580 (0.0007) [2023-12-27 18:43:26,365][20381] Updated weights for policy 1, policy_version 1885855 (0.0005) [2023-12-27 18:43:26,403][20347] Updated weights for policy 0, policy_version 1887590 (0.0007) [2023-12-27 18:43:26,459][20347] Updated weights for policy 0, policy_version 1887600 (0.0006) [2023-12-27 18:43:26,967][20381] Updated weights for policy 1, policy_version 1885865 (0.0006) [2023-12-27 18:43:27,017][20381] Updated weights for policy 1, policy_version 1885875 (0.0007) [2023-12-27 18:43:27,019][20347] Updated weights for policy 0, policy_version 1887610 (0.0006) [2023-12-27 18:43:27,069][20381] Updated weights for policy 1, policy_version 1885885 (0.0010) [2023-12-27 18:43:27,080][20347] Updated weights for policy 0, policy_version 1887620 (0.0005) [2023-12-27 18:43:27,142][20347] Updated weights for policy 0, policy_version 1887630 (0.0007) [2023-12-27 18:43:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 966164480. Throughput: 0: 10249.6, 1: 10105.7. Samples: 966148128. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '876.340')] [2023-12-27 18:43:27,752][20381] Updated weights for policy 1, policy_version 1885895 (0.0010) [2023-12-27 18:43:27,774][20347] Updated weights for policy 0, policy_version 1887640 (0.0006) [2023-12-27 18:43:27,799][20381] Updated weights for policy 1, policy_version 1885905 (0.0010) [2023-12-27 18:43:27,830][20347] Updated weights for policy 0, policy_version 1887650 (0.0007) [2023-12-27 18:43:27,850][20381] Updated weights for policy 1, policy_version 1885915 (0.0010) [2023-12-27 18:43:27,884][20347] Updated weights for policy 0, policy_version 1887660 (0.0005) [2023-12-27 18:43:28,518][20381] Updated weights for policy 1, policy_version 1885925 (0.0010) [2023-12-27 18:43:28,576][20381] Updated weights for policy 1, policy_version 1885935 (0.0010) [2023-12-27 18:43:28,613][20347] Updated weights for policy 0, policy_version 1887670 (0.0008) [2023-12-27 18:43:28,634][20381] Updated weights for policy 1, policy_version 1885945 (0.0011) [2023-12-27 18:43:28,664][20347] Updated weights for policy 0, policy_version 1887680 (0.0007) [2023-12-27 18:43:28,724][20347] Updated weights for policy 0, policy_version 1887690 (0.0007) [2023-12-27 18:43:29,330][20381] Updated weights for policy 1, policy_version 1885955 (0.0011) [2023-12-27 18:43:29,396][20381] Updated weights for policy 1, policy_version 1885965 (0.0009) [2023-12-27 18:43:29,441][20347] Updated weights for policy 0, policy_version 1887700 (0.0007) [2023-12-27 18:43:29,451][20381] Updated weights for policy 1, policy_version 1885975 (0.0007) [2023-12-27 18:43:29,493][20347] Updated weights for policy 0, policy_version 1887710 (0.0005) [2023-12-27 18:43:29,548][20347] Updated weights for policy 0, policy_version 1887720 (0.0007) [2023-12-27 18:43:30,178][20381] Updated weights for policy 1, policy_version 1885985 (0.0007) [2023-12-27 18:43:30,234][20381] Updated weights for policy 1, policy_version 1885995 (0.0008) [2023-12-27 18:43:30,271][20347] Updated weights for policy 0, policy_version 1887730 (0.0006) [2023-12-27 18:43:30,288][20381] Updated weights for policy 1, policy_version 1886005 (0.0007) [2023-12-27 18:43:30,320][20347] Updated weights for policy 0, policy_version 1887740 (0.0006) [2023-12-27 18:43:30,338][20381] Updated weights for policy 1, policy_version 1886015 (0.0006) [2023-12-27 18:43:30,373][20347] Updated weights for policy 0, policy_version 1887750 (0.0008) [2023-12-27 18:43:30,426][20347] Updated weights for policy 0, policy_version 1887760 (0.0008) [2023-12-27 18:43:30,935][20381] Updated weights for policy 1, policy_version 1886025 (0.0007) [2023-12-27 18:43:30,985][20381] Updated weights for policy 1, policy_version 1886035 (0.0007) [2023-12-27 18:43:31,035][20381] Updated weights for policy 1, policy_version 1886045 (0.0008) [2023-12-27 18:43:31,164][20347] Updated weights for policy 0, policy_version 1887770 (0.0008) [2023-12-27 18:43:31,225][20347] Updated weights for policy 0, policy_version 1887780 (0.0007) [2023-12-27 18:43:31,280][20347] Updated weights for policy 0, policy_version 1887790 (0.0007) [2023-12-27 18:43:31,737][20381] Updated weights for policy 1, policy_version 1886055 (0.0008) [2023-12-27 18:43:31,797][20381] Updated weights for policy 1, policy_version 1886065 (0.0008) [2023-12-27 18:43:31,856][20381] Updated weights for policy 1, policy_version 1886075 (0.0007) [2023-12-27 18:43:31,984][20347] Updated weights for policy 0, policy_version 1887800 (0.0008) [2023-12-27 18:43:32,044][20347] Updated weights for policy 0, policy_version 1887810 (0.0008) [2023-12-27 18:43:32,100][20347] Updated weights for policy 0, policy_version 1887820 (0.0007) [2023-12-27 18:43:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 966262784. Throughput: 0: 10256.6, 1: 10120.4. Samples: 966271688. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.980')] [2023-12-27 18:43:32,548][20381] Updated weights for policy 1, policy_version 1886085 (0.0006) [2023-12-27 18:43:32,609][20381] Updated weights for policy 1, policy_version 1886095 (0.0006) [2023-12-27 18:43:32,663][20381] Updated weights for policy 1, policy_version 1886105 (0.0005) [2023-12-27 18:43:32,871][20347] Updated weights for policy 0, policy_version 1887830 (0.0008) [2023-12-27 18:43:32,928][20347] Updated weights for policy 0, policy_version 1887840 (0.0007) [2023-12-27 18:43:32,982][20347] Updated weights for policy 0, policy_version 1887850 (0.0007) [2023-12-27 18:43:33,243][20381] Updated weights for policy 1, policy_version 1886115 (0.0006) [2023-12-27 18:43:33,293][20381] Updated weights for policy 1, policy_version 1886125 (0.0005) [2023-12-27 18:43:33,341][20381] Updated weights for policy 1, policy_version 1886135 (0.0005) [2023-12-27 18:43:33,646][20347] Updated weights for policy 0, policy_version 1887860 (0.0007) [2023-12-27 18:43:33,701][20347] Updated weights for policy 0, policy_version 1887870 (0.0007) [2023-12-27 18:43:33,759][20347] Updated weights for policy 0, policy_version 1887880 (0.0008) [2023-12-27 18:43:33,948][20381] Updated weights for policy 1, policy_version 1886145 (0.0006) [2023-12-27 18:43:33,995][20381] Updated weights for policy 1, policy_version 1886155 (0.0010) [2023-12-27 18:43:34,043][20381] Updated weights for policy 1, policy_version 1886165 (0.0010) [2023-12-27 18:43:34,097][20381] Updated weights for policy 1, policy_version 1886175 (0.0006) [2023-12-27 18:43:34,473][20347] Updated weights for policy 0, policy_version 1887890 (0.0007) [2023-12-27 18:43:34,524][20347] Updated weights for policy 0, policy_version 1887900 (0.0007) [2023-12-27 18:43:34,577][20347] Updated weights for policy 0, policy_version 1887910 (0.0008) [2023-12-27 18:43:34,628][20347] Updated weights for policy 0, policy_version 1887920 (0.0008) [2023-12-27 18:43:34,724][20381] Updated weights for policy 1, policy_version 1886185 (0.0005) [2023-12-27 18:43:34,784][20381] Updated weights for policy 1, policy_version 1886195 (0.0010) [2023-12-27 18:43:34,843][20381] Updated weights for policy 1, policy_version 1886205 (0.0009) [2023-12-27 18:43:35,360][20347] Updated weights for policy 0, policy_version 1887930 (0.0007) [2023-12-27 18:43:35,422][20347] Updated weights for policy 0, policy_version 1887940 (0.0007) [2023-12-27 18:43:35,458][20381] Updated weights for policy 1, policy_version 1886215 (0.0008) [2023-12-27 18:43:35,482][20347] Updated weights for policy 0, policy_version 1887950 (0.0007) [2023-12-27 18:43:35,518][20381] Updated weights for policy 1, policy_version 1886225 (0.0010) [2023-12-27 18:43:35,579][20381] Updated weights for policy 1, policy_version 1886235 (0.0008) [2023-12-27 18:43:36,164][20347] Updated weights for policy 0, policy_version 1887960 (0.0007) [2023-12-27 18:43:36,223][20347] Updated weights for policy 0, policy_version 1887970 (0.0008) [2023-12-27 18:43:36,281][20381] Updated weights for policy 1, policy_version 1886245 (0.0008) [2023-12-27 18:43:36,282][20347] Updated weights for policy 0, policy_version 1887980 (0.0008) [2023-12-27 18:43:36,340][20381] Updated weights for policy 1, policy_version 1886255 (0.0008) [2023-12-27 18:43:36,392][20381] Updated weights for policy 1, policy_version 1886265 (0.0008) [2023-12-27 18:43:36,431][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000003 [2023-12-27 18:43:37,003][20347] Updated weights for policy 0, policy_version 1887990 (0.0006) [2023-12-27 18:43:37,056][20347] Updated weights for policy 0, policy_version 1888000 (0.0007) [2023-12-27 18:43:37,070][20381] Updated weights for policy 1, policy_version 1886275 (0.0009) [2023-12-27 18:43:37,109][20347] Updated weights for policy 0, policy_version 1888010 (0.0006) [2023-12-27 18:43:37,120][20381] Updated weights for policy 1, policy_version 1886285 (0.0011) [2023-12-27 18:43:37,166][20381] Updated weights for policy 1, policy_version 1886295 (0.0011) [2023-12-27 18:43:37,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 966369280. Throughput: 0: 10228.1, 1: 10170.5. Samples: 966334276. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '843.540')] [2023-12-27 18:43:37,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001886304_482951168.pth... [2023-12-27 18:43:37,373][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001888016_483418112.pth... [2023-12-27 18:43:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001885120_482648064.pth [2023-12-27 18:43:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001886832_483115008.pth [2023-12-27 18:43:37,798][20347] Updated weights for policy 0, policy_version 1888020 (0.0006) [2023-12-27 18:43:37,849][20347] Updated weights for policy 0, policy_version 1888030 (0.0007) [2023-12-27 18:43:37,885][20381] Updated weights for policy 1, policy_version 1886305 (0.0010) [2023-12-27 18:43:37,898][20347] Updated weights for policy 0, policy_version 1888040 (0.0006) [2023-12-27 18:43:37,940][20381] Updated weights for policy 1, policy_version 1886315 (0.0011) [2023-12-27 18:43:37,998][20381] Updated weights for policy 1, policy_version 1886325 (0.0011) [2023-12-27 18:43:38,051][20381] Updated weights for policy 1, policy_version 1886335 (0.0011) [2023-12-27 18:43:38,625][20347] Updated weights for policy 0, policy_version 1888050 (0.0006) [2023-12-27 18:43:38,675][20347] Updated weights for policy 0, policy_version 1888060 (0.0006) [2023-12-27 18:43:38,728][20347] Updated weights for policy 0, policy_version 1888070 (0.0007) [2023-12-27 18:43:38,772][20381] Updated weights for policy 1, policy_version 1886345 (0.0011) [2023-12-27 18:43:38,781][20347] Updated weights for policy 0, policy_version 1888080 (0.0005) [2023-12-27 18:43:38,832][20381] Updated weights for policy 1, policy_version 1886355 (0.0011) [2023-12-27 18:43:38,881][20381] Updated weights for policy 1, policy_version 1886365 (0.0011) [2023-12-27 18:43:39,557][20347] Updated weights for policy 0, policy_version 1888090 (0.0007) [2023-12-27 18:43:39,618][20347] Updated weights for policy 0, policy_version 1888100 (0.0007) [2023-12-27 18:43:39,632][20381] Updated weights for policy 1, policy_version 1886375 (0.0010) [2023-12-27 18:43:39,678][20347] Updated weights for policy 0, policy_version 1888110 (0.0006) [2023-12-27 18:43:39,687][20381] Updated weights for policy 1, policy_version 1886385 (0.0010) [2023-12-27 18:43:39,751][20381] Updated weights for policy 1, policy_version 1886395 (0.0011) [2023-12-27 18:43:40,382][20347] Updated weights for policy 0, policy_version 1888120 (0.0007) [2023-12-27 18:43:40,446][20347] Updated weights for policy 0, policy_version 1888130 (0.0008) [2023-12-27 18:43:40,490][20381] Updated weights for policy 1, policy_version 1886405 (0.0010) [2023-12-27 18:43:40,509][20347] Updated weights for policy 0, policy_version 1888140 (0.0007) [2023-12-27 18:43:40,547][20381] Updated weights for policy 1, policy_version 1886415 (0.0010) [2023-12-27 18:43:40,611][20381] Updated weights for policy 1, policy_version 1886425 (0.0008) [2023-12-27 18:43:41,169][20347] Updated weights for policy 0, policy_version 1888150 (0.0007) [2023-12-27 18:43:41,231][20347] Updated weights for policy 0, policy_version 1888160 (0.0006) [2023-12-27 18:43:41,298][20347] Updated weights for policy 0, policy_version 1888170 (0.0007) [2023-12-27 18:43:41,305][20381] Updated weights for policy 1, policy_version 1886435 (0.0008) [2023-12-27 18:43:41,367][20381] Updated weights for policy 1, policy_version 1886445 (0.0010) [2023-12-27 18:43:41,436][20381] Updated weights for policy 1, policy_version 1886455 (0.0009) [2023-12-27 18:43:42,047][20347] Updated weights for policy 0, policy_version 1888180 (0.0008) [2023-12-27 18:43:42,096][20347] Updated weights for policy 0, policy_version 1888190 (0.0010) [2023-12-27 18:43:42,109][20381] Updated weights for policy 1, policy_version 1886465 (0.0008) [2023-12-27 18:43:42,148][20347] Updated weights for policy 0, policy_version 1888200 (0.0010) [2023-12-27 18:43:42,171][20381] Updated weights for policy 1, policy_version 1886475 (0.0011) [2023-12-27 18:43:42,238][20381] Updated weights for policy 1, policy_version 1886485 (0.0011) [2023-12-27 18:43:42,306][20381] Updated weights for policy 1, policy_version 1886495 (0.0011) [2023-12-27 18:43:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 966467584. Throughput: 0: 10168.4, 1: 10165.2. Samples: 966452888. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '857.400')] [2023-12-27 18:43:42,871][20347] Updated weights for policy 0, policy_version 1888210 (0.0007) [2023-12-27 18:43:42,930][20347] Updated weights for policy 0, policy_version 1888220 (0.0010) [2023-12-27 18:43:42,988][20347] Updated weights for policy 0, policy_version 1888230 (0.0010) [2023-12-27 18:43:43,001][20381] Updated weights for policy 1, policy_version 1886505 (0.0006) [2023-12-27 18:43:43,050][20347] Updated weights for policy 0, policy_version 1888240 (0.0010) [2023-12-27 18:43:43,057][20381] Updated weights for policy 1, policy_version 1886515 (0.0005) [2023-12-27 18:43:43,102][20381] Updated weights for policy 1, policy_version 1886525 (0.0005) [2023-12-27 18:43:43,651][20381] Updated weights for policy 1, policy_version 1886535 (0.0005) [2023-12-27 18:43:43,666][20347] Updated weights for policy 0, policy_version 1888250 (0.0007) [2023-12-27 18:43:43,714][20381] Updated weights for policy 1, policy_version 1886545 (0.0005) [2023-12-27 18:43:43,725][20347] Updated weights for policy 0, policy_version 1888260 (0.0007) [2023-12-27 18:43:43,778][20381] Updated weights for policy 1, policy_version 1886555 (0.0005) [2023-12-27 18:43:43,780][20347] Updated weights for policy 0, policy_version 1888270 (0.0007) [2023-12-27 18:43:44,327][20381] Updated weights for policy 1, policy_version 1886565 (0.0008) [2023-12-27 18:43:44,361][20347] Updated weights for policy 0, policy_version 1888280 (0.0007) [2023-12-27 18:43:44,382][20381] Updated weights for policy 1, policy_version 1886575 (0.0011) [2023-12-27 18:43:44,416][20347] Updated weights for policy 0, policy_version 1888290 (0.0006) [2023-12-27 18:43:44,444][20381] Updated weights for policy 1, policy_version 1886585 (0.0010) [2023-12-27 18:43:44,467][20347] Updated weights for policy 0, policy_version 1888300 (0.0005) [2023-12-27 18:43:45,114][20381] Updated weights for policy 1, policy_version 1886595 (0.0009) [2023-12-27 18:43:45,153][20347] Updated weights for policy 0, policy_version 1888310 (0.0008) [2023-12-27 18:43:45,171][20381] Updated weights for policy 1, policy_version 1886605 (0.0006) [2023-12-27 18:43:45,213][20347] Updated weights for policy 0, policy_version 1888320 (0.0011) [2023-12-27 18:43:45,235][20381] Updated weights for policy 1, policy_version 1886615 (0.0009) [2023-12-27 18:43:45,273][20347] Updated weights for policy 0, policy_version 1888330 (0.0011) [2023-12-27 18:43:45,792][20381] Updated weights for policy 1, policy_version 1886625 (0.0006) [2023-12-27 18:43:45,844][20381] Updated weights for policy 1, policy_version 1886635 (0.0007) [2023-12-27 18:43:45,888][20347] Updated weights for policy 0, policy_version 1888340 (0.0010) [2023-12-27 18:43:45,903][20381] Updated weights for policy 1, policy_version 1886645 (0.0008) [2023-12-27 18:43:45,945][20347] Updated weights for policy 0, policy_version 1888350 (0.0009) [2023-12-27 18:43:45,966][20381] Updated weights for policy 1, policy_version 1886655 (0.0008) [2023-12-27 18:43:46,000][20347] Updated weights for policy 0, policy_version 1888360 (0.0008) [2023-12-27 18:43:46,542][20381] Updated weights for policy 1, policy_version 1886665 (0.0008) [2023-12-27 18:43:46,602][20381] Updated weights for policy 1, policy_version 1886675 (0.0007) [2023-12-27 18:43:46,645][20347] Updated weights for policy 0, policy_version 1888370 (0.0010) [2023-12-27 18:43:46,668][20381] Updated weights for policy 1, policy_version 1886685 (0.0005) [2023-12-27 18:43:46,696][20347] Updated weights for policy 0, policy_version 1888380 (0.0008) [2023-12-27 18:43:46,746][20347] Updated weights for policy 0, policy_version 1888390 (0.0007) [2023-12-27 18:43:46,794][20347] Updated weights for policy 0, policy_version 1888400 (0.0006) [2023-12-27 18:43:47,348][20381] Updated weights for policy 1, policy_version 1886695 (0.0007) [2023-12-27 18:43:47,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 966565888. Throughput: 0: 10248.1, 1: 10294.3. Samples: 966581012. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '864.690')] [2023-12-27 18:43:47,392][20381] Updated weights for policy 1, policy_version 1886705 (0.0007) [2023-12-27 18:43:47,432][20347] Updated weights for policy 0, policy_version 1888410 (0.0010) [2023-12-27 18:43:47,451][20381] Updated weights for policy 1, policy_version 1886715 (0.0008) [2023-12-27 18:43:47,490][20347] Updated weights for policy 0, policy_version 1888420 (0.0010) [2023-12-27 18:43:47,544][20347] Updated weights for policy 0, policy_version 1888430 (0.0010) [2023-12-27 18:43:48,147][20381] Updated weights for policy 1, policy_version 1886725 (0.0005) [2023-12-27 18:43:48,204][20381] Updated weights for policy 1, policy_version 1886735 (0.0006) [2023-12-27 18:43:48,224][20347] Updated weights for policy 0, policy_version 1888440 (0.0008) [2023-12-27 18:43:48,266][20381] Updated weights for policy 1, policy_version 1886745 (0.0006) [2023-12-27 18:43:48,277][20347] Updated weights for policy 0, policy_version 1888450 (0.0008) [2023-12-27 18:43:48,337][20347] Updated weights for policy 0, policy_version 1888460 (0.0006) [2023-12-27 18:43:48,950][20381] Updated weights for policy 1, policy_version 1886755 (0.0007) [2023-12-27 18:43:49,005][20381] Updated weights for policy 1, policy_version 1886765 (0.0011) [2023-12-27 18:43:49,008][20347] Updated weights for policy 0, policy_version 1888470 (0.0009) [2023-12-27 18:43:49,053][20381] Updated weights for policy 1, policy_version 1886775 (0.0007) [2023-12-27 18:43:49,065][20347] Updated weights for policy 0, policy_version 1888480 (0.0010) [2023-12-27 18:43:49,116][20347] Updated weights for policy 0, policy_version 1888490 (0.0009) [2023-12-27 18:43:49,781][20381] Updated weights for policy 1, policy_version 1886785 (0.0010) [2023-12-27 18:43:49,812][20347] Updated weights for policy 0, policy_version 1888500 (0.0007) [2023-12-27 18:43:49,843][20381] Updated weights for policy 1, policy_version 1886795 (0.0007) [2023-12-27 18:43:49,878][20347] Updated weights for policy 0, policy_version 1888510 (0.0011) [2023-12-27 18:43:49,899][20381] Updated weights for policy 1, policy_version 1886805 (0.0007) [2023-12-27 18:43:49,946][20347] Updated weights for policy 0, policy_version 1888520 (0.0011) [2023-12-27 18:43:49,963][20381] Updated weights for policy 1, policy_version 1886815 (0.0008) [2023-12-27 18:43:50,526][20347] Updated weights for policy 0, policy_version 1888530 (0.0007) [2023-12-27 18:43:50,586][20347] Updated weights for policy 0, policy_version 1888540 (0.0007) [2023-12-27 18:43:50,642][20347] Updated weights for policy 0, policy_version 1888550 (0.0007) [2023-12-27 18:43:50,698][20381] Updated weights for policy 1, policy_version 1886825 (0.0008) [2023-12-27 18:43:50,702][20347] Updated weights for policy 0, policy_version 1888560 (0.0010) [2023-12-27 18:43:50,750][20381] Updated weights for policy 1, policy_version 1886835 (0.0007) [2023-12-27 18:43:50,802][20381] Updated weights for policy 1, policy_version 1886845 (0.0005) [2023-12-27 18:43:51,389][20347] Updated weights for policy 0, policy_version 1888570 (0.0007) [2023-12-27 18:43:51,450][20347] Updated weights for policy 0, policy_version 1888580 (0.0007) [2023-12-27 18:43:51,514][20347] Updated weights for policy 0, policy_version 1888590 (0.0007) [2023-12-27 18:43:51,536][20381] Updated weights for policy 1, policy_version 1886855 (0.0006) [2023-12-27 18:43:51,584][20381] Updated weights for policy 1, policy_version 1886865 (0.0008) [2023-12-27 18:43:51,639][20381] Updated weights for policy 1, policy_version 1886875 (0.0008) [2023-12-27 18:43:52,300][20347] Updated weights for policy 0, policy_version 1888600 (0.0007) [2023-12-27 18:43:52,308][20381] Updated weights for policy 1, policy_version 1886885 (0.0008) [2023-12-27 18:43:52,360][20347] Updated weights for policy 0, policy_version 1888610 (0.0007) [2023-12-27 18:43:52,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 966664192. Throughput: 0: 10242.1, 1: 10225.4. Samples: 966640636. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '870.570')] [2023-12-27 18:43:52,372][20381] Updated weights for policy 1, policy_version 1886895 (0.0007) [2023-12-27 18:43:52,422][20347] Updated weights for policy 0, policy_version 1888620 (0.0008) [2023-12-27 18:43:52,437][20381] Updated weights for policy 1, policy_version 1886906 (0.0010) [2023-12-27 18:43:52,446][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001888624_483573760.pth... [2023-12-27 18:43:52,449][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001887408_483262464.pth [2023-12-27 18:43:52,468][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001886912_483106816.pth... [2023-12-27 18:43:52,472][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001885696_482795520.pth [2023-12-27 18:43:53,112][20381] Updated weights for policy 1, policy_version 1886916 (0.0008) [2023-12-27 18:43:53,123][20347] Updated weights for policy 0, policy_version 1888630 (0.0008) [2023-12-27 18:43:53,168][20381] Updated weights for policy 1, policy_version 1886926 (0.0005) [2023-12-27 18:43:53,169][20347] Updated weights for policy 0, policy_version 1888640 (0.0007) [2023-12-27 18:43:53,214][20347] Updated weights for policy 0, policy_version 1888650 (0.0007) [2023-12-27 18:43:53,222][20381] Updated weights for policy 1, policy_version 1886936 (0.0005) [2023-12-27 18:43:53,811][20381] Updated weights for policy 1, policy_version 1886946 (0.0007) [2023-12-27 18:43:53,868][20381] Updated weights for policy 1, policy_version 1886956 (0.0010) [2023-12-27 18:43:53,922][20381] Updated weights for policy 1, policy_version 1886966 (0.0010) [2023-12-27 18:43:53,942][20347] Updated weights for policy 0, policy_version 1888660 (0.0007) [2023-12-27 18:43:53,980][20381] Updated weights for policy 1, policy_version 1886976 (0.0009) [2023-12-27 18:43:53,994][20347] Updated weights for policy 0, policy_version 1888670 (0.0006) [2023-12-27 18:43:54,040][20347] Updated weights for policy 0, policy_version 1888680 (0.0008) [2023-12-27 18:43:54,629][20381] Updated weights for policy 1, policy_version 1886986 (0.0008) [2023-12-27 18:43:54,682][20381] Updated weights for policy 1, policy_version 1886996 (0.0007) [2023-12-27 18:43:54,728][20381] Updated weights for policy 1, policy_version 1887006 (0.0007) [2023-12-27 18:43:54,739][20347] Updated weights for policy 0, policy_version 1888690 (0.0007) [2023-12-27 18:43:54,801][20347] Updated weights for policy 0, policy_version 1888700 (0.0007) [2023-12-27 18:43:54,857][20347] Updated weights for policy 0, policy_version 1888710 (0.0008) [2023-12-27 18:43:54,923][20347] Updated weights for policy 0, policy_version 1888720 (0.0007) [2023-12-27 18:43:55,405][20381] Updated weights for policy 1, policy_version 1887016 (0.0008) [2023-12-27 18:43:55,460][20381] Updated weights for policy 1, policy_version 1887026 (0.0007) [2023-12-27 18:43:55,519][20381] Updated weights for policy 1, policy_version 1887036 (0.0007) [2023-12-27 18:43:55,619][20347] Updated weights for policy 0, policy_version 1888730 (0.0008) [2023-12-27 18:43:55,665][20347] Updated weights for policy 0, policy_version 1888740 (0.0007) [2023-12-27 18:43:55,716][20347] Updated weights for policy 0, policy_version 1888750 (0.0005) [2023-12-27 18:43:56,215][20381] Updated weights for policy 1, policy_version 1887046 (0.0008) [2023-12-27 18:43:56,267][20381] Updated weights for policy 1, policy_version 1887056 (0.0009) [2023-12-27 18:43:56,325][20381] Updated weights for policy 1, policy_version 1887066 (0.0007) [2023-12-27 18:43:56,389][20347] Updated weights for policy 0, policy_version 1888760 (0.0006) [2023-12-27 18:43:56,447][20347] Updated weights for policy 0, policy_version 1888770 (0.0007) [2023-12-27 18:43:56,504][20347] Updated weights for policy 0, policy_version 1888780 (0.0008) [2023-12-27 18:43:57,056][20381] Updated weights for policy 1, policy_version 1887076 (0.0008) [2023-12-27 18:43:57,098][20347] Updated weights for policy 0, policy_version 1888790 (0.0007) [2023-12-27 18:43:57,107][20381] Updated weights for policy 1, policy_version 1887086 (0.0006) [2023-12-27 18:43:57,153][20347] Updated weights for policy 0, policy_version 1888800 (0.0005) [2023-12-27 18:43:57,162][20381] Updated weights for policy 1, policy_version 1887096 (0.0006) [2023-12-27 18:43:57,208][20347] Updated weights for policy 0, policy_version 1888810 (0.0006) [2023-12-27 18:43:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 966778880. Throughput: 0: 10222.2, 1: 10257.0. Samples: 966763164. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:43:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.730')] [2023-12-27 18:43:57,850][20381] Updated weights for policy 1, policy_version 1887106 (0.0006) [2023-12-27 18:43:57,883][20347] Updated weights for policy 0, policy_version 1888820 (0.0007) [2023-12-27 18:43:57,902][20381] Updated weights for policy 1, policy_version 1887116 (0.0008) [2023-12-27 18:43:57,937][20347] Updated weights for policy 0, policy_version 1888830 (0.0006) [2023-12-27 18:43:57,954][20381] Updated weights for policy 1, policy_version 1887126 (0.0005) [2023-12-27 18:43:57,991][20347] Updated weights for policy 0, policy_version 1888840 (0.0006) [2023-12-27 18:43:58,010][20381] Updated weights for policy 1, policy_version 1887136 (0.0006) [2023-12-27 18:43:58,748][20347] Updated weights for policy 0, policy_version 1888850 (0.0010) [2023-12-27 18:43:58,795][20381] Updated weights for policy 1, policy_version 1887146 (0.0008) [2023-12-27 18:43:58,817][20347] Updated weights for policy 0, policy_version 1888860 (0.0011) [2023-12-27 18:43:58,861][20381] Updated weights for policy 1, policy_version 1887156 (0.0008) [2023-12-27 18:43:58,880][20347] Updated weights for policy 0, policy_version 1888870 (0.0007) [2023-12-27 18:43:58,928][20381] Updated weights for policy 1, policy_version 1887166 (0.0007) [2023-12-27 18:43:58,951][20347] Updated weights for policy 0, policy_version 1888880 (0.0008) [2023-12-27 18:43:59,649][20347] Updated weights for policy 0, policy_version 1888890 (0.0007) [2023-12-27 18:43:59,691][20381] Updated weights for policy 1, policy_version 1887176 (0.0008) [2023-12-27 18:43:59,707][20347] Updated weights for policy 0, policy_version 1888900 (0.0005) [2023-12-27 18:43:59,750][20381] Updated weights for policy 1, policy_version 1887186 (0.0007) [2023-12-27 18:43:59,752][20347] Updated weights for policy 0, policy_version 1888910 (0.0006) [2023-12-27 18:43:59,809][20381] Updated weights for policy 1, policy_version 1887196 (0.0008) [2023-12-27 18:44:00,401][20347] Updated weights for policy 0, policy_version 1888920 (0.0008) [2023-12-27 18:44:00,455][20347] Updated weights for policy 0, policy_version 1888930 (0.0007) [2023-12-27 18:44:00,507][20347] Updated weights for policy 0, policy_version 1888940 (0.0007) [2023-12-27 18:44:00,534][20381] Updated weights for policy 1, policy_version 1887206 (0.0008) [2023-12-27 18:44:00,594][20381] Updated weights for policy 1, policy_version 1887216 (0.0008) [2023-12-27 18:44:00,650][20381] Updated weights for policy 1, policy_version 1887226 (0.0005) [2023-12-27 18:44:01,265][20347] Updated weights for policy 0, policy_version 1888950 (0.0007) [2023-12-27 18:44:01,272][20381] Updated weights for policy 1, policy_version 1887236 (0.0007) [2023-12-27 18:44:01,320][20347] Updated weights for policy 0, policy_version 1888960 (0.0006) [2023-12-27 18:44:01,325][20381] Updated weights for policy 1, policy_version 1887246 (0.0011) [2023-12-27 18:44:01,384][20347] Updated weights for policy 0, policy_version 1888970 (0.0006) [2023-12-27 18:44:01,388][20381] Updated weights for policy 1, policy_version 1887256 (0.0010) [2023-12-27 18:44:02,075][20381] Updated weights for policy 1, policy_version 1887266 (0.0007) [2023-12-27 18:44:02,126][20381] Updated weights for policy 1, policy_version 1887276 (0.0007) [2023-12-27 18:44:02,133][20347] Updated weights for policy 0, policy_version 1888980 (0.0006) [2023-12-27 18:44:02,174][20381] Updated weights for policy 1, policy_version 1887286 (0.0007) [2023-12-27 18:44:02,187][20347] Updated weights for policy 0, policy_version 1888990 (0.0008) [2023-12-27 18:44:02,221][20381] Updated weights for policy 1, policy_version 1887296 (0.0006) [2023-12-27 18:44:02,244][20347] Updated weights for policy 0, policy_version 1889000 (0.0007) [2023-12-27 18:44:02,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 966877184. Throughput: 0: 10108.0, 1: 10237.6. Samples: 966881352. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '888.050')] [2023-12-27 18:44:02,919][20381] Updated weights for policy 1, policy_version 1887306 (0.0006) [2023-12-27 18:44:02,960][20347] Updated weights for policy 0, policy_version 1889010 (0.0008) [2023-12-27 18:44:02,974][20381] Updated weights for policy 1, policy_version 1887316 (0.0007) [2023-12-27 18:44:03,011][20347] Updated weights for policy 0, policy_version 1889020 (0.0007) [2023-12-27 18:44:03,026][20381] Updated weights for policy 1, policy_version 1887326 (0.0005) [2023-12-27 18:44:03,056][20347] Updated weights for policy 0, policy_version 1889030 (0.0008) [2023-12-27 18:44:03,101][20347] Updated weights for policy 0, policy_version 1889040 (0.0008) [2023-12-27 18:44:03,678][20381] Updated weights for policy 1, policy_version 1887336 (0.0007) [2023-12-27 18:44:03,729][20381] Updated weights for policy 1, policy_version 1887346 (0.0007) [2023-12-27 18:44:03,779][20381] Updated weights for policy 1, policy_version 1887356 (0.0007) [2023-12-27 18:44:03,814][20347] Updated weights for policy 0, policy_version 1889050 (0.0009) [2023-12-27 18:44:03,871][20347] Updated weights for policy 0, policy_version 1889060 (0.0008) [2023-12-27 18:44:03,928][20347] Updated weights for policy 0, policy_version 1889070 (0.0009) [2023-12-27 18:44:04,483][20381] Updated weights for policy 1, policy_version 1887366 (0.0007) [2023-12-27 18:44:04,534][20381] Updated weights for policy 1, policy_version 1887376 (0.0007) [2023-12-27 18:44:04,590][20381] Updated weights for policy 1, policy_version 1887386 (0.0006) [2023-12-27 18:44:04,635][20347] Updated weights for policy 0, policy_version 1889080 (0.0006) [2023-12-27 18:44:04,678][20347] Updated weights for policy 0, policy_version 1889090 (0.0006) [2023-12-27 18:44:04,728][20347] Updated weights for policy 0, policy_version 1889100 (0.0005) [2023-12-27 18:44:05,229][20381] Updated weights for policy 1, policy_version 1887396 (0.0006) [2023-12-27 18:44:05,286][20381] Updated weights for policy 1, policy_version 1887406 (0.0008) [2023-12-27 18:44:05,339][20381] Updated weights for policy 1, policy_version 1887416 (0.0008) [2023-12-27 18:44:05,426][20347] Updated weights for policy 0, policy_version 1889110 (0.0007) [2023-12-27 18:44:05,476][20347] Updated weights for policy 0, policy_version 1889120 (0.0007) [2023-12-27 18:44:05,520][20347] Updated weights for policy 0, policy_version 1889130 (0.0007) [2023-12-27 18:44:05,934][20381] Updated weights for policy 1, policy_version 1887426 (0.0007) [2023-12-27 18:44:05,982][20381] Updated weights for policy 1, policy_version 1887436 (0.0005) [2023-12-27 18:44:06,033][20381] Updated weights for policy 1, policy_version 1887446 (0.0006) [2023-12-27 18:44:06,083][20381] Updated weights for policy 1, policy_version 1887456 (0.0006) [2023-12-27 18:44:06,278][20347] Updated weights for policy 0, policy_version 1889140 (0.0007) [2023-12-27 18:44:06,334][20347] Updated weights for policy 0, policy_version 1889150 (0.0005) [2023-12-27 18:44:06,395][20347] Updated weights for policy 0, policy_version 1889160 (0.0006) [2023-12-27 18:44:06,757][20381] Updated weights for policy 1, policy_version 1887466 (0.0006) [2023-12-27 18:44:06,815][20381] Updated weights for policy 1, policy_version 1887476 (0.0007) [2023-12-27 18:44:06,877][20381] Updated weights for policy 1, policy_version 1887486 (0.0005) [2023-12-27 18:44:07,020][20347] Updated weights for policy 0, policy_version 1889170 (0.0006) [2023-12-27 18:44:07,071][20347] Updated weights for policy 0, policy_version 1889180 (0.0005) [2023-12-27 18:44:07,134][20347] Updated weights for policy 0, policy_version 1889190 (0.0005) [2023-12-27 18:44:07,193][20347] Updated weights for policy 0, policy_version 1889200 (0.0009) [2023-12-27 18:44:07,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.1, 300 sec: 20466.1). Total num frames: 966975488. Throughput: 0: 10088.1, 1: 10286.8. Samples: 966942852. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '879.490')] [2023-12-27 18:44:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001889200_483721216.pth... [2023-12-27 18:44:07,380][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001888016_483418112.pth [2023-12-27 18:44:07,412][20381] Updated weights for policy 1, policy_version 1887496 (0.0005) [2023-12-27 18:44:07,473][20381] Updated weights for policy 1, policy_version 1887506 (0.0005) [2023-12-27 18:44:07,529][20381] Updated weights for policy 1, policy_version 1887516 (0.0005) [2023-12-27 18:44:07,549][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001887520_483262464.pth... [2023-12-27 18:44:07,552][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001886304_482951168.pth [2023-12-27 18:44:07,818][20347] Updated weights for policy 0, policy_version 1889210 (0.0007) [2023-12-27 18:44:07,872][20347] Updated weights for policy 0, policy_version 1889220 (0.0010) [2023-12-27 18:44:07,924][20347] Updated weights for policy 0, policy_version 1889230 (0.0008) [2023-12-27 18:44:08,084][20381] Updated weights for policy 1, policy_version 1887526 (0.0006) [2023-12-27 18:44:08,129][20381] Updated weights for policy 1, policy_version 1887536 (0.0007) [2023-12-27 18:44:08,174][20381] Updated weights for policy 1, policy_version 1887546 (0.0008) [2023-12-27 18:44:08,648][20347] Updated weights for policy 0, policy_version 1889240 (0.0007) [2023-12-27 18:44:08,710][20347] Updated weights for policy 0, policy_version 1889250 (0.0008) [2023-12-27 18:44:08,772][20347] Updated weights for policy 0, policy_version 1889260 (0.0008) [2023-12-27 18:44:08,836][20381] Updated weights for policy 1, policy_version 1887556 (0.0008) [2023-12-27 18:44:08,891][20381] Updated weights for policy 1, policy_version 1887566 (0.0008) [2023-12-27 18:44:08,948][20381] Updated weights for policy 1, policy_version 1887576 (0.0008) [2023-12-27 18:44:09,532][20347] Updated weights for policy 0, policy_version 1889270 (0.0006) [2023-12-27 18:44:09,585][20347] Updated weights for policy 0, policy_version 1889280 (0.0007) [2023-12-27 18:44:09,646][20347] Updated weights for policy 0, policy_version 1889290 (0.0008) [2023-12-27 18:44:09,696][20381] Updated weights for policy 1, policy_version 1887586 (0.0007) [2023-12-27 18:44:09,757][20381] Updated weights for policy 1, policy_version 1887596 (0.0008) [2023-12-27 18:44:09,820][20381] Updated weights for policy 1, policy_version 1887606 (0.0008) [2023-12-27 18:44:09,886][20381] Updated weights for policy 1, policy_version 1887616 (0.0008) [2023-12-27 18:44:10,362][20347] Updated weights for policy 0, policy_version 1889300 (0.0008) [2023-12-27 18:44:10,422][20347] Updated weights for policy 0, policy_version 1889310 (0.0007) [2023-12-27 18:44:10,483][20347] Updated weights for policy 0, policy_version 1889320 (0.0006) [2023-12-27 18:44:10,598][20381] Updated weights for policy 1, policy_version 1887626 (0.0007) [2023-12-27 18:44:10,661][20381] Updated weights for policy 1, policy_version 1887636 (0.0011) [2023-12-27 18:44:10,717][20381] Updated weights for policy 1, policy_version 1887646 (0.0009) [2023-12-27 18:44:11,153][20347] Updated weights for policy 0, policy_version 1889330 (0.0007) [2023-12-27 18:44:11,217][20347] Updated weights for policy 0, policy_version 1889340 (0.0007) [2023-12-27 18:44:11,280][20347] Updated weights for policy 0, policy_version 1889350 (0.0007) [2023-12-27 18:44:11,351][20347] Updated weights for policy 0, policy_version 1889360 (0.0008) [2023-12-27 18:44:11,463][20381] Updated weights for policy 1, policy_version 1887656 (0.0008) [2023-12-27 18:44:11,525][20381] Updated weights for policy 1, policy_version 1887666 (0.0008) [2023-12-27 18:44:11,588][20381] Updated weights for policy 1, policy_version 1887676 (0.0008) [2023-12-27 18:44:12,063][20347] Updated weights for policy 0, policy_version 1889370 (0.0008) [2023-12-27 18:44:12,116][20347] Updated weights for policy 0, policy_version 1889380 (0.0007) [2023-12-27 18:44:12,165][20347] Updated weights for policy 0, policy_version 1889390 (0.0006) [2023-12-27 18:44:12,327][20381] Updated weights for policy 1, policy_version 1887686 (0.0007) [2023-12-27 18:44:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 967073792. Throughput: 0: 10074.4, 1: 10329.4. Samples: 967066300. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '888.240')] [2023-12-27 18:44:12,408][20381] Updated weights for policy 1, policy_version 1887696 (0.0007) [2023-12-27 18:44:12,463][20381] Updated weights for policy 1, policy_version 1887706 (0.0006) [2023-12-27 18:44:12,827][20347] Updated weights for policy 0, policy_version 1889400 (0.0010) [2023-12-27 18:44:12,888][20347] Updated weights for policy 0, policy_version 1889410 (0.0010) [2023-12-27 18:44:12,949][20347] Updated weights for policy 0, policy_version 1889420 (0.0010) [2023-12-27 18:44:13,072][20381] Updated weights for policy 1, policy_version 1887716 (0.0006) [2023-12-27 18:44:13,122][20381] Updated weights for policy 1, policy_version 1887726 (0.0007) [2023-12-27 18:44:13,176][20381] Updated weights for policy 1, policy_version 1887736 (0.0007) [2023-12-27 18:44:13,611][20347] Updated weights for policy 0, policy_version 1889430 (0.0010) [2023-12-27 18:44:13,675][20347] Updated weights for policy 0, policy_version 1889440 (0.0010) [2023-12-27 18:44:13,724][20347] Updated weights for policy 0, policy_version 1889450 (0.0010) [2023-12-27 18:44:13,780][20381] Updated weights for policy 1, policy_version 1887746 (0.0007) [2023-12-27 18:44:13,837][20381] Updated weights for policy 1, policy_version 1887756 (0.0010) [2023-12-27 18:44:13,890][20381] Updated weights for policy 1, policy_version 1887766 (0.0005) [2023-12-27 18:44:13,938][20381] Updated weights for policy 1, policy_version 1887776 (0.0005) [2023-12-27 18:44:14,406][20347] Updated weights for policy 0, policy_version 1889460 (0.0008) [2023-12-27 18:44:14,460][20347] Updated weights for policy 0, policy_version 1889470 (0.0009) [2023-12-27 18:44:14,488][20381] Updated weights for policy 1, policy_version 1887786 (0.0010) [2023-12-27 18:44:14,514][20347] Updated weights for policy 0, policy_version 1889480 (0.0010) [2023-12-27 18:44:14,549][20381] Updated weights for policy 1, policy_version 1887796 (0.0008) [2023-12-27 18:44:14,609][20381] Updated weights for policy 1, policy_version 1887806 (0.0011) [2023-12-27 18:44:15,225][20347] Updated weights for policy 0, policy_version 1889490 (0.0008) [2023-12-27 18:44:15,274][20347] Updated weights for policy 0, policy_version 1889500 (0.0007) [2023-12-27 18:44:15,313][20381] Updated weights for policy 1, policy_version 1887816 (0.0007) [2023-12-27 18:44:15,333][20347] Updated weights for policy 0, policy_version 1889510 (0.0006) [2023-12-27 18:44:15,369][20381] Updated weights for policy 1, policy_version 1887826 (0.0005) [2023-12-27 18:44:15,389][20347] Updated weights for policy 0, policy_version 1889520 (0.0006) [2023-12-27 18:44:15,423][20381] Updated weights for policy 1, policy_version 1887836 (0.0007) [2023-12-27 18:44:16,089][20347] Updated weights for policy 0, policy_version 1889530 (0.0007) [2023-12-27 18:44:16,138][20381] Updated weights for policy 1, policy_version 1887846 (0.0006) [2023-12-27 18:44:16,148][20347] Updated weights for policy 0, policy_version 1889540 (0.0006) [2023-12-27 18:44:16,196][20381] Updated weights for policy 1, policy_version 1887856 (0.0007) [2023-12-27 18:44:16,196][20347] Updated weights for policy 0, policy_version 1889550 (0.0005) [2023-12-27 18:44:16,244][20381] Updated weights for policy 1, policy_version 1887866 (0.0009) [2023-12-27 18:44:16,814][20347] Updated weights for policy 0, policy_version 1889560 (0.0007) [2023-12-27 18:44:16,879][20347] Updated weights for policy 0, policy_version 1889570 (0.0008) [2023-12-27 18:44:16,909][20381] Updated weights for policy 1, policy_version 1887876 (0.0007) [2023-12-27 18:44:16,935][20347] Updated weights for policy 0, policy_version 1889580 (0.0007) [2023-12-27 18:44:16,960][20381] Updated weights for policy 1, policy_version 1887886 (0.0007) [2023-12-27 18:44:17,015][20381] Updated weights for policy 1, policy_version 1887896 (0.0008) [2023-12-27 18:44:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 967180288. Throughput: 0: 10058.9, 1: 10327.0. Samples: 967189048. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '886.230')] [2023-12-27 18:44:17,577][20347] Updated weights for policy 0, policy_version 1889590 (0.0007) [2023-12-27 18:44:17,632][20347] Updated weights for policy 0, policy_version 1889600 (0.0008) [2023-12-27 18:44:17,683][20347] Updated weights for policy 0, policy_version 1889610 (0.0007) [2023-12-27 18:44:17,722][20381] Updated weights for policy 1, policy_version 1887906 (0.0009) [2023-12-27 18:44:17,789][20381] Updated weights for policy 1, policy_version 1887916 (0.0009) [2023-12-27 18:44:17,853][20381] Updated weights for policy 1, policy_version 1887926 (0.0008) [2023-12-27 18:44:17,916][20381] Updated weights for policy 1, policy_version 1887936 (0.0008) [2023-12-27 18:44:18,382][20347] Updated weights for policy 0, policy_version 1889620 (0.0008) [2023-12-27 18:44:18,448][20347] Updated weights for policy 0, policy_version 1889630 (0.0008) [2023-12-27 18:44:18,509][20347] Updated weights for policy 0, policy_version 1889640 (0.0006) [2023-12-27 18:44:18,511][20381] Updated weights for policy 1, policy_version 1887946 (0.0006) [2023-12-27 18:44:18,572][20381] Updated weights for policy 1, policy_version 1887956 (0.0007) [2023-12-27 18:44:18,625][20381] Updated weights for policy 1, policy_version 1887966 (0.0007) [2023-12-27 18:44:19,190][20347] Updated weights for policy 0, policy_version 1889650 (0.0006) [2023-12-27 18:44:19,252][20347] Updated weights for policy 0, policy_version 1889660 (0.0007) [2023-12-27 18:44:19,310][20347] Updated weights for policy 0, policy_version 1889670 (0.0008) [2023-12-27 18:44:19,372][20347] Updated weights for policy 0, policy_version 1889680 (0.0008) [2023-12-27 18:44:19,375][20381] Updated weights for policy 1, policy_version 1887976 (0.0008) [2023-12-27 18:44:19,442][20381] Updated weights for policy 1, policy_version 1887986 (0.0007) [2023-12-27 18:44:19,505][20381] Updated weights for policy 1, policy_version 1887996 (0.0007) [2023-12-27 18:44:20,120][20347] Updated weights for policy 0, policy_version 1889690 (0.0011) [2023-12-27 18:44:20,184][20347] Updated weights for policy 0, policy_version 1889700 (0.0011) [2023-12-27 18:44:20,210][20381] Updated weights for policy 1, policy_version 1888006 (0.0007) [2023-12-27 18:44:20,241][20347] Updated weights for policy 0, policy_version 1889710 (0.0011) [2023-12-27 18:44:20,268][20381] Updated weights for policy 1, policy_version 1888016 (0.0009) [2023-12-27 18:44:20,328][20381] Updated weights for policy 1, policy_version 1888026 (0.0008) [2023-12-27 18:44:20,960][20347] Updated weights for policy 0, policy_version 1889720 (0.0010) [2023-12-27 18:44:20,984][20381] Updated weights for policy 1, policy_version 1888036 (0.0006) [2023-12-27 18:44:21,024][20347] Updated weights for policy 0, policy_version 1889730 (0.0010) [2023-12-27 18:44:21,049][20381] Updated weights for policy 1, policy_version 1888046 (0.0008) [2023-12-27 18:44:21,089][20347] Updated weights for policy 0, policy_version 1889740 (0.0009) [2023-12-27 18:44:21,105][20381] Updated weights for policy 1, policy_version 1888056 (0.0007) [2023-12-27 18:44:21,816][20347] Updated weights for policy 0, policy_version 1889750 (0.0008) [2023-12-27 18:44:21,859][20381] Updated weights for policy 1, policy_version 1888066 (0.0008) [2023-12-27 18:44:21,867][20347] Updated weights for policy 0, policy_version 1889760 (0.0008) [2023-12-27 18:44:21,918][20381] Updated weights for policy 1, policy_version 1888076 (0.0007) [2023-12-27 18:44:21,919][20347] Updated weights for policy 0, policy_version 1889770 (0.0007) [2023-12-27 18:44:21,972][20381] Updated weights for policy 1, policy_version 1888086 (0.0006) [2023-12-27 18:44:22,035][20381] Updated weights for policy 1, policy_version 1888096 (0.0007) [2023-12-27 18:44:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 967278592. Throughput: 0: 10076.1, 1: 10260.8. Samples: 967249436. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '848.870')] [2023-12-27 18:44:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001888096_483409920.pth... [2023-12-27 18:44:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001889776_483868672.pth... [2023-12-27 18:44:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001886912_483106816.pth [2023-12-27 18:44:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001888624_483573760.pth [2023-12-27 18:44:22,601][20347] Updated weights for policy 0, policy_version 1889780 (0.0006) [2023-12-27 18:44:22,658][20347] Updated weights for policy 0, policy_version 1889790 (0.0008) [2023-12-27 18:44:22,719][20347] Updated weights for policy 0, policy_version 1889800 (0.0008) [2023-12-27 18:44:22,820][20381] Updated weights for policy 1, policy_version 1888106 (0.0008) [2023-12-27 18:44:22,875][20381] Updated weights for policy 1, policy_version 1888116 (0.0008) [2023-12-27 18:44:22,935][20381] Updated weights for policy 1, policy_version 1888126 (0.0008) [2023-12-27 18:44:23,287][20347] Updated weights for policy 0, policy_version 1889810 (0.0007) [2023-12-27 18:44:23,337][20347] Updated weights for policy 0, policy_version 1889820 (0.0005) [2023-12-27 18:44:23,398][20347] Updated weights for policy 0, policy_version 1889830 (0.0005) [2023-12-27 18:44:23,462][20347] Updated weights for policy 0, policy_version 1889840 (0.0005) [2023-12-27 18:44:23,631][20381] Updated weights for policy 1, policy_version 1888136 (0.0010) [2023-12-27 18:44:23,689][20381] Updated weights for policy 1, policy_version 1888146 (0.0010) [2023-12-27 18:44:23,745][20381] Updated weights for policy 1, policy_version 1888156 (0.0007) [2023-12-27 18:44:24,014][20347] Updated weights for policy 0, policy_version 1889850 (0.0005) [2023-12-27 18:44:24,073][20347] Updated weights for policy 0, policy_version 1889860 (0.0006) [2023-12-27 18:44:24,138][20347] Updated weights for policy 0, policy_version 1889870 (0.0007) [2023-12-27 18:44:24,326][20381] Updated weights for policy 1, policy_version 1888166 (0.0009) [2023-12-27 18:44:24,378][20381] Updated weights for policy 1, policy_version 1888176 (0.0011) [2023-12-27 18:44:24,430][20381] Updated weights for policy 1, policy_version 1888186 (0.0011) [2023-12-27 18:44:24,798][20347] Updated weights for policy 0, policy_version 1889880 (0.0007) [2023-12-27 18:44:24,846][20347] Updated weights for policy 0, policy_version 1889890 (0.0007) [2023-12-27 18:44:24,897][20347] Updated weights for policy 0, policy_version 1889900 (0.0007) [2023-12-27 18:44:25,032][20381] Updated weights for policy 1, policy_version 1888196 (0.0011) [2023-12-27 18:44:25,084][20381] Updated weights for policy 1, policy_version 1888206 (0.0009) [2023-12-27 18:44:25,139][20381] Updated weights for policy 1, policy_version 1888216 (0.0010) [2023-12-27 18:44:25,614][20347] Updated weights for policy 0, policy_version 1889910 (0.0008) [2023-12-27 18:44:25,671][20347] Updated weights for policy 0, policy_version 1889920 (0.0007) [2023-12-27 18:44:25,729][20347] Updated weights for policy 0, policy_version 1889930 (0.0007) [2023-12-27 18:44:25,848][20381] Updated weights for policy 1, policy_version 1888226 (0.0010) [2023-12-27 18:44:25,910][20381] Updated weights for policy 1, policy_version 1888236 (0.0011) [2023-12-27 18:44:25,966][20381] Updated weights for policy 1, policy_version 1888246 (0.0011) [2023-12-27 18:44:26,026][20381] Updated weights for policy 1, policy_version 1888256 (0.0011) [2023-12-27 18:44:26,439][20347] Updated weights for policy 0, policy_version 1889940 (0.0007) [2023-12-27 18:44:26,485][20347] Updated weights for policy 0, policy_version 1889950 (0.0007) [2023-12-27 18:44:26,536][20347] Updated weights for policy 0, policy_version 1889960 (0.0008) [2023-12-27 18:44:26,701][20381] Updated weights for policy 1, policy_version 1888266 (0.0008) [2023-12-27 18:44:26,753][20381] Updated weights for policy 1, policy_version 1888276 (0.0008) [2023-12-27 18:44:26,798][20381] Updated weights for policy 1, policy_version 1888286 (0.0007) [2023-12-27 18:44:27,199][20347] Updated weights for policy 0, policy_version 1889970 (0.0007) [2023-12-27 18:44:27,247][20347] Updated weights for policy 0, policy_version 1889980 (0.0008) [2023-12-27 18:44:27,294][20347] Updated weights for policy 0, policy_version 1889990 (0.0008) [2023-12-27 18:44:27,356][20347] Updated weights for policy 0, policy_version 1890000 (0.0008) [2023-12-27 18:44:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 967385088. Throughput: 0: 10134.4, 1: 10297.3. Samples: 967372316. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '830.040')] [2023-12-27 18:44:27,456][20381] Updated weights for policy 1, policy_version 1888296 (0.0006) [2023-12-27 18:44:27,503][20381] Updated weights for policy 1, policy_version 1888306 (0.0005) [2023-12-27 18:44:27,557][20381] Updated weights for policy 1, policy_version 1888316 (0.0005) [2023-12-27 18:44:28,041][20347] Updated weights for policy 0, policy_version 1890010 (0.0007) [2023-12-27 18:44:28,102][20347] Updated weights for policy 0, policy_version 1890020 (0.0007) [2023-12-27 18:44:28,150][20347] Updated weights for policy 0, policy_version 1890030 (0.0010) [2023-12-27 18:44:28,155][20381] Updated weights for policy 1, policy_version 1888326 (0.0006) [2023-12-27 18:44:28,213][20381] Updated weights for policy 1, policy_version 1888336 (0.0005) [2023-12-27 18:44:28,274][20381] Updated weights for policy 1, policy_version 1888346 (0.0005) [2023-12-27 18:44:28,765][20347] Updated weights for policy 0, policy_version 1890040 (0.0006) [2023-12-27 18:44:28,821][20347] Updated weights for policy 0, policy_version 1890050 (0.0007) [2023-12-27 18:44:28,869][20347] Updated weights for policy 0, policy_version 1890060 (0.0009) [2023-12-27 18:44:28,880][20381] Updated weights for policy 1, policy_version 1888356 (0.0008) [2023-12-27 18:44:28,941][20381] Updated weights for policy 1, policy_version 1888366 (0.0010) [2023-12-27 18:44:29,007][20381] Updated weights for policy 1, policy_version 1888376 (0.0011) [2023-12-27 18:44:29,514][20347] Updated weights for policy 0, policy_version 1890070 (0.0007) [2023-12-27 18:44:29,562][20347] Updated weights for policy 0, policy_version 1890080 (0.0005) [2023-12-27 18:44:29,615][20347] Updated weights for policy 0, policy_version 1890090 (0.0005) [2023-12-27 18:44:29,745][20381] Updated weights for policy 1, policy_version 1888386 (0.0011) [2023-12-27 18:44:29,797][20381] Updated weights for policy 1, policy_version 1888396 (0.0010) [2023-12-27 18:44:29,856][20381] Updated weights for policy 1, policy_version 1888406 (0.0011) [2023-12-27 18:44:29,918][20381] Updated weights for policy 1, policy_version 1888416 (0.0011) [2023-12-27 18:44:30,254][20347] Updated weights for policy 0, policy_version 1890100 (0.0009) [2023-12-27 18:44:30,299][20347] Updated weights for policy 0, policy_version 1890110 (0.0008) [2023-12-27 18:44:30,355][20347] Updated weights for policy 0, policy_version 1890120 (0.0009) [2023-12-27 18:44:30,608][20381] Updated weights for policy 1, policy_version 1888426 (0.0008) [2023-12-27 18:44:30,654][20381] Updated weights for policy 1, policy_version 1888436 (0.0008) [2023-12-27 18:44:30,698][20381] Updated weights for policy 1, policy_version 1888446 (0.0005) [2023-12-27 18:44:31,051][20347] Updated weights for policy 0, policy_version 1890130 (0.0008) [2023-12-27 18:44:31,114][20347] Updated weights for policy 0, policy_version 1890140 (0.0008) [2023-12-27 18:44:31,175][20347] Updated weights for policy 0, policy_version 1890150 (0.0007) [2023-12-27 18:44:31,231][20347] Updated weights for policy 0, policy_version 1890160 (0.0006) [2023-12-27 18:44:31,387][20381] Updated weights for policy 1, policy_version 1888456 (0.0006) [2023-12-27 18:44:31,440][20381] Updated weights for policy 1, policy_version 1888466 (0.0008) [2023-12-27 18:44:31,486][20381] Updated weights for policy 1, policy_version 1888476 (0.0008) [2023-12-27 18:44:31,914][20347] Updated weights for policy 0, policy_version 1890170 (0.0008) [2023-12-27 18:44:31,979][20347] Updated weights for policy 0, policy_version 1890180 (0.0008) [2023-12-27 18:44:32,037][20347] Updated weights for policy 0, policy_version 1890190 (0.0007) [2023-12-27 18:44:32,208][20381] Updated weights for policy 1, policy_version 1888486 (0.0009) [2023-12-27 18:44:32,261][20381] Updated weights for policy 1, policy_version 1888496 (0.0009) [2023-12-27 18:44:32,313][20381] Updated weights for policy 1, policy_version 1888506 (0.0008) [2023-12-27 18:44:32,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 967491584. Throughput: 0: 10134.4, 1: 10224.2. Samples: 967497148. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '840.670')] [2023-12-27 18:44:32,654][20347] Updated weights for policy 0, policy_version 1890200 (0.0008) [2023-12-27 18:44:32,708][20347] Updated weights for policy 0, policy_version 1890210 (0.0007) [2023-12-27 18:44:32,763][20347] Updated weights for policy 0, policy_version 1890220 (0.0007) [2023-12-27 18:44:33,051][20381] Updated weights for policy 1, policy_version 1888516 (0.0008) [2023-12-27 18:44:33,110][20381] Updated weights for policy 1, policy_version 1888526 (0.0009) [2023-12-27 18:44:33,161][20381] Updated weights for policy 1, policy_version 1888536 (0.0008) [2023-12-27 18:44:33,365][20347] Updated weights for policy 0, policy_version 1890230 (0.0006) [2023-12-27 18:44:33,422][20347] Updated weights for policy 0, policy_version 1890240 (0.0005) [2023-12-27 18:44:33,481][20347] Updated weights for policy 0, policy_version 1890250 (0.0005) [2023-12-27 18:44:33,816][20381] Updated weights for policy 1, policy_version 1888546 (0.0007) [2023-12-27 18:44:33,879][20381] Updated weights for policy 1, policy_version 1888556 (0.0006) [2023-12-27 18:44:33,930][20381] Updated weights for policy 1, policy_version 1888566 (0.0007) [2023-12-27 18:44:33,977][20381] Updated weights for policy 1, policy_version 1888576 (0.0007) [2023-12-27 18:44:33,990][20347] Updated weights for policy 0, policy_version 1890260 (0.0006) [2023-12-27 18:44:34,034][20347] Updated weights for policy 0, policy_version 1890270 (0.0007) [2023-12-27 18:44:34,082][20347] Updated weights for policy 0, policy_version 1890280 (0.0007) [2023-12-27 18:44:34,676][20381] Updated weights for policy 1, policy_version 1888586 (0.0007) [2023-12-27 18:44:34,733][20381] Updated weights for policy 1, policy_version 1888596 (0.0007) [2023-12-27 18:44:34,763][20347] Updated weights for policy 0, policy_version 1890290 (0.0005) [2023-12-27 18:44:34,785][20381] Updated weights for policy 1, policy_version 1888606 (0.0006) [2023-12-27 18:44:34,810][20347] Updated weights for policy 0, policy_version 1890300 (0.0006) [2023-12-27 18:44:34,859][20347] Updated weights for policy 0, policy_version 1890310 (0.0007) [2023-12-27 18:44:34,914][20347] Updated weights for policy 0, policy_version 1890320 (0.0008) [2023-12-27 18:44:35,461][20381] Updated weights for policy 1, policy_version 1888616 (0.0007) [2023-12-27 18:44:35,526][20381] Updated weights for policy 1, policy_version 1888626 (0.0006) [2023-12-27 18:44:35,590][20381] Updated weights for policy 1, policy_version 1888636 (0.0006) [2023-12-27 18:44:35,627][20347] Updated weights for policy 0, policy_version 1890330 (0.0010) [2023-12-27 18:44:35,684][20347] Updated weights for policy 0, policy_version 1890340 (0.0011) [2023-12-27 18:44:35,744][20347] Updated weights for policy 0, policy_version 1890350 (0.0010) [2023-12-27 18:44:36,280][20381] Updated weights for policy 1, policy_version 1888646 (0.0007) [2023-12-27 18:44:36,344][20381] Updated weights for policy 1, policy_version 1888656 (0.0008) [2023-12-27 18:44:36,390][20347] Updated weights for policy 0, policy_version 1890360 (0.0007) [2023-12-27 18:44:36,408][20381] Updated weights for policy 1, policy_version 1888666 (0.0006) [2023-12-27 18:44:36,455][20347] Updated weights for policy 0, policy_version 1890370 (0.0008) [2023-12-27 18:44:36,505][20347] Updated weights for policy 0, policy_version 1890380 (0.0009) [2023-12-27 18:44:37,033][20381] Updated weights for policy 1, policy_version 1888676 (0.0006) [2023-12-27 18:44:37,083][20381] Updated weights for policy 1, policy_version 1888686 (0.0007) [2023-12-27 18:44:37,130][20381] Updated weights for policy 1, policy_version 1888696 (0.0009) [2023-12-27 18:44:37,161][20347] Updated weights for policy 0, policy_version 1890390 (0.0009) [2023-12-27 18:44:37,214][20347] Updated weights for policy 0, policy_version 1890400 (0.0010) [2023-12-27 18:44:37,268][20347] Updated weights for policy 0, policy_version 1890410 (0.0010) [2023-12-27 18:44:37,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 967598080. Throughput: 0: 10207.3, 1: 10244.3. Samples: 967560956. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '928.220')] [2023-12-27 18:44:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001888704_483565568.pth... [2023-12-27 18:44:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001890416_484032512.pth... [2023-12-27 18:44:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001887520_483262464.pth [2023-12-27 18:44:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001889200_483721216.pth [2023-12-27 18:44:37,744][20381] Updated weights for policy 1, policy_version 1888706 (0.0006) [2023-12-27 18:44:37,802][20381] Updated weights for policy 1, policy_version 1888716 (0.0006) [2023-12-27 18:44:37,856][20381] Updated weights for policy 1, policy_version 1888726 (0.0005) [2023-12-27 18:44:37,906][20381] Updated weights for policy 1, policy_version 1888736 (0.0006) [2023-12-27 18:44:37,969][20347] Updated weights for policy 0, policy_version 1890420 (0.0010) [2023-12-27 18:44:38,024][20347] Updated weights for policy 0, policy_version 1890430 (0.0010) [2023-12-27 18:44:38,086][20347] Updated weights for policy 0, policy_version 1890440 (0.0010) [2023-12-27 18:44:38,519][20381] Updated weights for policy 1, policy_version 1888746 (0.0011) [2023-12-27 18:44:38,569][20381] Updated weights for policy 1, policy_version 1888756 (0.0008) [2023-12-27 18:44:38,618][20381] Updated weights for policy 1, policy_version 1888766 (0.0007) [2023-12-27 18:44:38,761][20347] Updated weights for policy 0, policy_version 1890450 (0.0010) [2023-12-27 18:44:38,816][20347] Updated weights for policy 0, policy_version 1890460 (0.0010) [2023-12-27 18:44:38,865][20347] Updated weights for policy 0, policy_version 1890470 (0.0010) [2023-12-27 18:44:38,917][20347] Updated weights for policy 0, policy_version 1890480 (0.0010) [2023-12-27 18:44:39,342][20381] Updated weights for policy 1, policy_version 1888776 (0.0008) [2023-12-27 18:44:39,411][20381] Updated weights for policy 1, policy_version 1888786 (0.0007) [2023-12-27 18:44:39,471][20381] Updated weights for policy 1, policy_version 1888796 (0.0007) [2023-12-27 18:44:39,627][20347] Updated weights for policy 0, policy_version 1890490 (0.0008) [2023-12-27 18:44:39,688][20347] Updated weights for policy 0, policy_version 1890500 (0.0008) [2023-12-27 18:44:39,750][20347] Updated weights for policy 0, policy_version 1890510 (0.0008) [2023-12-27 18:44:40,258][20381] Updated weights for policy 1, policy_version 1888806 (0.0008) [2023-12-27 18:44:40,317][20381] Updated weights for policy 1, policy_version 1888816 (0.0009) [2023-12-27 18:44:40,377][20381] Updated weights for policy 1, policy_version 1888826 (0.0008) [2023-12-27 18:44:40,405][20347] Updated weights for policy 0, policy_version 1890520 (0.0010) [2023-12-27 18:44:40,465][20347] Updated weights for policy 0, policy_version 1890530 (0.0011) [2023-12-27 18:44:40,525][20347] Updated weights for policy 0, policy_version 1890540 (0.0011) [2023-12-27 18:44:41,055][20381] Updated weights for policy 1, policy_version 1888836 (0.0006) [2023-12-27 18:44:41,121][20381] Updated weights for policy 1, policy_version 1888846 (0.0011) [2023-12-27 18:44:41,182][20381] Updated weights for policy 1, policy_version 1888856 (0.0009) [2023-12-27 18:44:41,262][20347] Updated weights for policy 0, policy_version 1890550 (0.0009) [2023-12-27 18:44:41,322][20347] Updated weights for policy 0, policy_version 1890560 (0.0008) [2023-12-27 18:44:41,395][20347] Updated weights for policy 0, policy_version 1890570 (0.0009) [2023-12-27 18:44:41,946][20381] Updated weights for policy 1, policy_version 1888866 (0.0007) [2023-12-27 18:44:42,007][20381] Updated weights for policy 1, policy_version 1888876 (0.0008) [2023-12-27 18:44:42,064][20381] Updated weights for policy 1, policy_version 1888886 (0.0008) [2023-12-27 18:44:42,114][20381] Updated weights for policy 1, policy_version 1888896 (0.0006) [2023-12-27 18:44:42,177][20347] Updated weights for policy 0, policy_version 1890580 (0.0008) [2023-12-27 18:44:42,238][20347] Updated weights for policy 0, policy_version 1890590 (0.0007) [2023-12-27 18:44:42,298][20347] Updated weights for policy 0, policy_version 1890600 (0.0008) [2023-12-27 18:44:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 967696384. Throughput: 0: 10242.5, 1: 10209.9. Samples: 967683524. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:42,363][19170] Avg episode reward: [(0, '990.290'), (1, '914.080')] [2023-12-27 18:44:42,740][20381] Updated weights for policy 1, policy_version 1888906 (0.0006) [2023-12-27 18:44:42,801][20381] Updated weights for policy 1, policy_version 1888916 (0.0007) [2023-12-27 18:44:42,860][20381] Updated weights for policy 1, policy_version 1888926 (0.0008) [2023-12-27 18:44:43,044][20347] Updated weights for policy 0, policy_version 1890610 (0.0007) [2023-12-27 18:44:43,098][20347] Updated weights for policy 0, policy_version 1890620 (0.0007) [2023-12-27 18:44:43,158][20347] Updated weights for policy 0, policy_version 1890630 (0.0006) [2023-12-27 18:44:43,211][20347] Updated weights for policy 0, policy_version 1890640 (0.0005) [2023-12-27 18:44:43,489][20381] Updated weights for policy 1, policy_version 1888936 (0.0006) [2023-12-27 18:44:43,535][20381] Updated weights for policy 1, policy_version 1888946 (0.0005) [2023-12-27 18:44:43,587][20381] Updated weights for policy 1, policy_version 1888956 (0.0006) [2023-12-27 18:44:43,801][20347] Updated weights for policy 0, policy_version 1890650 (0.0010) [2023-12-27 18:44:43,855][20347] Updated weights for policy 0, policy_version 1890660 (0.0010) [2023-12-27 18:44:43,912][20347] Updated weights for policy 0, policy_version 1890670 (0.0010) [2023-12-27 18:44:44,150][20381] Updated weights for policy 1, policy_version 1888966 (0.0007) [2023-12-27 18:44:44,200][20381] Updated weights for policy 1, policy_version 1888976 (0.0007) [2023-12-27 18:44:44,258][20381] Updated weights for policy 1, policy_version 1888986 (0.0007) [2023-12-27 18:44:44,536][20347] Updated weights for policy 0, policy_version 1890680 (0.0006) [2023-12-27 18:44:44,581][20347] Updated weights for policy 0, policy_version 1890690 (0.0005) [2023-12-27 18:44:44,634][20347] Updated weights for policy 0, policy_version 1890700 (0.0005) [2023-12-27 18:44:44,878][20381] Updated weights for policy 1, policy_version 1888996 (0.0007) [2023-12-27 18:44:44,928][20381] Updated weights for policy 1, policy_version 1889006 (0.0007) [2023-12-27 18:44:44,980][20381] Updated weights for policy 1, policy_version 1889016 (0.0006) [2023-12-27 18:44:45,301][20347] Updated weights for policy 0, policy_version 1890710 (0.0008) [2023-12-27 18:44:45,363][20347] Updated weights for policy 0, policy_version 1890720 (0.0010) [2023-12-27 18:44:45,421][20347] Updated weights for policy 0, policy_version 1890730 (0.0010) [2023-12-27 18:44:45,633][20381] Updated weights for policy 1, policy_version 1889026 (0.0007) [2023-12-27 18:44:45,681][20381] Updated weights for policy 1, policy_version 1889036 (0.0005) [2023-12-27 18:44:45,736][20381] Updated weights for policy 1, policy_version 1889046 (0.0005) [2023-12-27 18:44:45,788][20381] Updated weights for policy 1, policy_version 1889056 (0.0005) [2023-12-27 18:44:46,098][20347] Updated weights for policy 0, policy_version 1890740 (0.0010) [2023-12-27 18:44:46,155][20347] Updated weights for policy 0, policy_version 1890750 (0.0010) [2023-12-27 18:44:46,221][20347] Updated weights for policy 0, policy_version 1890760 (0.0008) [2023-12-27 18:44:46,427][20381] Updated weights for policy 1, policy_version 1889066 (0.0007) [2023-12-27 18:44:46,482][20381] Updated weights for policy 1, policy_version 1889076 (0.0007) [2023-12-27 18:44:46,537][20381] Updated weights for policy 1, policy_version 1889086 (0.0007) [2023-12-27 18:44:46,891][20347] Updated weights for policy 0, policy_version 1890770 (0.0009) [2023-12-27 18:44:46,938][20347] Updated weights for policy 0, policy_version 1890780 (0.0008) [2023-12-27 18:44:46,985][20347] Updated weights for policy 0, policy_version 1890790 (0.0008) [2023-12-27 18:44:47,038][20347] Updated weights for policy 0, policy_version 1890800 (0.0009) [2023-12-27 18:44:47,160][20381] Updated weights for policy 1, policy_version 1889096 (0.0010) [2023-12-27 18:44:47,205][20381] Updated weights for policy 1, policy_version 1889106 (0.0010) [2023-12-27 18:44:47,260][20381] Updated weights for policy 1, policy_version 1889116 (0.0010) [2023-12-27 18:44:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 967802880. Throughput: 0: 10254.2, 1: 10379.7. Samples: 967809880. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:47,363][19170] Avg episode reward: [(0, '990.290'), (1, '840.850')] [2023-12-27 18:44:47,701][20347] Updated weights for policy 0, policy_version 1890810 (0.0007) [2023-12-27 18:44:47,753][20347] Updated weights for policy 0, policy_version 1890820 (0.0010) [2023-12-27 18:44:47,810][20347] Updated weights for policy 0, policy_version 1890830 (0.0010) [2023-12-27 18:44:47,815][20381] Updated weights for policy 1, policy_version 1889126 (0.0010) [2023-12-27 18:44:47,867][20381] Updated weights for policy 1, policy_version 1889136 (0.0010) [2023-12-27 18:44:47,915][20381] Updated weights for policy 1, policy_version 1889146 (0.0010) [2023-12-27 18:44:48,476][20347] Updated weights for policy 0, policy_version 1890840 (0.0008) [2023-12-27 18:44:48,527][20347] Updated weights for policy 0, policy_version 1890850 (0.0007) [2023-12-27 18:44:48,575][20347] Updated weights for policy 0, policy_version 1890860 (0.0007) [2023-12-27 18:44:48,619][20381] Updated weights for policy 1, policy_version 1889156 (0.0010) [2023-12-27 18:44:48,679][20381] Updated weights for policy 1, policy_version 1889166 (0.0008) [2023-12-27 18:44:48,741][20381] Updated weights for policy 1, policy_version 1889176 (0.0011) [2023-12-27 18:44:49,283][20347] Updated weights for policy 0, policy_version 1890870 (0.0008) [2023-12-27 18:44:49,322][20381] Updated weights for policy 1, policy_version 1889186 (0.0010) [2023-12-27 18:44:49,341][20347] Updated weights for policy 0, policy_version 1890880 (0.0010) [2023-12-27 18:44:49,391][20381] Updated weights for policy 1, policy_version 1889196 (0.0008) [2023-12-27 18:44:49,405][20347] Updated weights for policy 0, policy_version 1890890 (0.0007) [2023-12-27 18:44:49,449][20381] Updated weights for policy 1, policy_version 1889206 (0.0007) [2023-12-27 18:44:49,513][20381] Updated weights for policy 1, policy_version 1889216 (0.0008) [2023-12-27 18:44:50,155][20347] Updated weights for policy 0, policy_version 1890900 (0.0006) [2023-12-27 18:44:50,164][20381] Updated weights for policy 1, policy_version 1889226 (0.0008) [2023-12-27 18:44:50,220][20347] Updated weights for policy 0, policy_version 1890910 (0.0007) [2023-12-27 18:44:50,223][20381] Updated weights for policy 1, policy_version 1889236 (0.0006) [2023-12-27 18:44:50,272][20347] Updated weights for policy 0, policy_version 1890920 (0.0007) [2023-12-27 18:44:50,278][20381] Updated weights for policy 1, policy_version 1889246 (0.0006) [2023-12-27 18:44:50,889][20381] Updated weights for policy 1, policy_version 1889256 (0.0006) [2023-12-27 18:44:50,952][20381] Updated weights for policy 1, policy_version 1889266 (0.0006) [2023-12-27 18:44:51,008][20381] Updated weights for policy 1, policy_version 1889276 (0.0007) [2023-12-27 18:44:51,054][20347] Updated weights for policy 0, policy_version 1890930 (0.0007) [2023-12-27 18:44:51,108][20347] Updated weights for policy 0, policy_version 1890940 (0.0006) [2023-12-27 18:44:51,171][20347] Updated weights for policy 0, policy_version 1890950 (0.0008) [2023-12-27 18:44:51,229][20347] Updated weights for policy 0, policy_version 1890960 (0.0007) [2023-12-27 18:44:51,701][20381] Updated weights for policy 1, policy_version 1889286 (0.0008) [2023-12-27 18:44:51,768][20381] Updated weights for policy 1, policy_version 1889296 (0.0007) [2023-12-27 18:44:51,823][20381] Updated weights for policy 1, policy_version 1889306 (0.0007) [2023-12-27 18:44:52,008][20347] Updated weights for policy 0, policy_version 1890970 (0.0007) [2023-12-27 18:44:52,065][20347] Updated weights for policy 0, policy_version 1890980 (0.0007) [2023-12-27 18:44:52,120][20347] Updated weights for policy 0, policy_version 1890990 (0.0007) [2023-12-27 18:44:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20521.6). Total num frames: 967901184. Throughput: 0: 10257.4, 1: 10395.0. Samples: 967872212. Policy #0 lag: (min: 17.0, avg: 26.6, max: 49.0) [2023-12-27 18:44:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '815.460')] [2023-12-27 18:44:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001889312_483721216.pth... [2023-12-27 18:44:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001890992_484179968.pth... [2023-12-27 18:44:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001888096_483409920.pth [2023-12-27 18:44:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001889776_483868672.pth [2023-12-27 18:44:52,515][20381] Updated weights for policy 1, policy_version 1889316 (0.0008) [2023-12-27 18:44:52,578][20381] Updated weights for policy 1, policy_version 1889326 (0.0008) [2023-12-27 18:44:52,640][20381] Updated weights for policy 1, policy_version 1889336 (0.0008) [2023-12-27 18:44:52,861][20347] Updated weights for policy 0, policy_version 1891000 (0.0008) [2023-12-27 18:44:52,916][20347] Updated weights for policy 0, policy_version 1891010 (0.0007) [2023-12-27 18:44:52,965][20347] Updated weights for policy 0, policy_version 1891020 (0.0007) [2023-12-27 18:44:53,326][20381] Updated weights for policy 1, policy_version 1889346 (0.0009) [2023-12-27 18:44:53,375][20381] Updated weights for policy 1, policy_version 1889356 (0.0010) [2023-12-27 18:44:53,424][20381] Updated weights for policy 1, policy_version 1889366 (0.0010) [2023-12-27 18:44:53,476][20381] Updated weights for policy 1, policy_version 1889376 (0.0008) [2023-12-27 18:44:53,683][20347] Updated weights for policy 0, policy_version 1891030 (0.0008) [2023-12-27 18:44:53,728][20347] Updated weights for policy 0, policy_version 1891040 (0.0008) [2023-12-27 18:44:53,772][20347] Updated weights for policy 0, policy_version 1891050 (0.0008) [2023-12-27 18:44:54,032][20381] Updated weights for policy 1, policy_version 1889386 (0.0006) [2023-12-27 18:44:54,080][20381] Updated weights for policy 1, policy_version 1889396 (0.0008) [2023-12-27 18:44:54,128][20381] Updated weights for policy 1, policy_version 1889406 (0.0008) [2023-12-27 18:44:54,505][20347] Updated weights for policy 0, policy_version 1891060 (0.0007) [2023-12-27 18:44:54,566][20347] Updated weights for policy 0, policy_version 1891070 (0.0007) [2023-12-27 18:44:54,626][20347] Updated weights for policy 0, policy_version 1891080 (0.0008) [2023-12-27 18:44:54,805][20381] Updated weights for policy 1, policy_version 1889416 (0.0008) [2023-12-27 18:44:54,861][20381] Updated weights for policy 1, policy_version 1889426 (0.0006) [2023-12-27 18:44:54,911][20381] Updated weights for policy 1, policy_version 1889436 (0.0005) [2023-12-27 18:44:55,300][20347] Updated weights for policy 0, policy_version 1891090 (0.0008) [2023-12-27 18:44:55,366][20347] Updated weights for policy 0, policy_version 1891100 (0.0008) [2023-12-27 18:44:55,425][20347] Updated weights for policy 0, policy_version 1891110 (0.0008) [2023-12-27 18:44:55,486][20347] Updated weights for policy 0, policy_version 1891120 (0.0007) [2023-12-27 18:44:55,530][20381] Updated weights for policy 1, policy_version 1889446 (0.0008) [2023-12-27 18:44:55,594][20381] Updated weights for policy 1, policy_version 1889456 (0.0008) [2023-12-27 18:44:55,656][20381] Updated weights for policy 1, policy_version 1889466 (0.0007) [2023-12-27 18:44:56,193][20347] Updated weights for policy 0, policy_version 1891130 (0.0008) [2023-12-27 18:44:56,254][20381] Updated weights for policy 1, policy_version 1889476 (0.0006) [2023-12-27 18:44:56,255][20347] Updated weights for policy 0, policy_version 1891140 (0.0007) [2023-12-27 18:44:56,310][20381] Updated weights for policy 1, policy_version 1889486 (0.0006) [2023-12-27 18:44:56,317][20347] Updated weights for policy 0, policy_version 1891150 (0.0007) [2023-12-27 18:44:56,363][20381] Updated weights for policy 1, policy_version 1889496 (0.0007) [2023-12-27 18:44:57,000][20347] Updated weights for policy 0, policy_version 1891160 (0.0007) [2023-12-27 18:44:57,033][20381] Updated weights for policy 1, policy_version 1889506 (0.0008) [2023-12-27 18:44:57,056][20347] Updated weights for policy 0, policy_version 1891170 (0.0008) [2023-12-27 18:44:57,077][20381] Updated weights for policy 1, policy_version 1889516 (0.0008) [2023-12-27 18:44:57,110][20347] Updated weights for policy 0, policy_version 1891180 (0.0009) [2023-12-27 18:44:57,121][20381] Updated weights for policy 1, policy_version 1889526 (0.0007) [2023-12-27 18:44:57,174][20381] Updated weights for policy 1, policy_version 1889536 (0.0007) [2023-12-27 18:44:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 968007680. Throughput: 0: 10207.6, 1: 10415.3. Samples: 967994332. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:44:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '885.290')] [2023-12-27 18:44:57,813][20347] Updated weights for policy 0, policy_version 1891190 (0.0008) [2023-12-27 18:44:57,820][20381] Updated weights for policy 1, policy_version 1889546 (0.0005) [2023-12-27 18:44:57,870][20347] Updated weights for policy 0, policy_version 1891200 (0.0006) [2023-12-27 18:44:57,872][20381] Updated weights for policy 1, policy_version 1889556 (0.0006) [2023-12-27 18:44:57,931][20347] Updated weights for policy 0, policy_version 1891210 (0.0009) [2023-12-27 18:44:57,932][20381] Updated weights for policy 1, policy_version 1889566 (0.0006) [2023-12-27 18:44:58,637][20381] Updated weights for policy 1, policy_version 1889576 (0.0006) [2023-12-27 18:44:58,659][20347] Updated weights for policy 0, policy_version 1891220 (0.0008) [2023-12-27 18:44:58,701][20381] Updated weights for policy 1, policy_version 1889586 (0.0007) [2023-12-27 18:44:58,724][20347] Updated weights for policy 0, policy_version 1891230 (0.0008) [2023-12-27 18:44:58,765][20381] Updated weights for policy 1, policy_version 1889596 (0.0007) [2023-12-27 18:44:58,789][20347] Updated weights for policy 0, policy_version 1891240 (0.0007) [2023-12-27 18:44:59,439][20381] Updated weights for policy 1, policy_version 1889606 (0.0007) [2023-12-27 18:44:59,488][20381] Updated weights for policy 1, policy_version 1889616 (0.0007) [2023-12-27 18:44:59,537][20381] Updated weights for policy 1, policy_version 1889626 (0.0008) [2023-12-27 18:44:59,547][20347] Updated weights for policy 0, policy_version 1891250 (0.0013) [2023-12-27 18:44:59,605][20347] Updated weights for policy 0, policy_version 1891260 (0.0010) [2023-12-27 18:44:59,670][20347] Updated weights for policy 0, policy_version 1891270 (0.0010) [2023-12-27 18:44:59,736][20347] Updated weights for policy 0, policy_version 1891280 (0.0010) [2023-12-27 18:45:00,191][20381] Updated weights for policy 1, policy_version 1889636 (0.0005) [2023-12-27 18:45:00,240][20381] Updated weights for policy 1, policy_version 1889646 (0.0006) [2023-12-27 18:45:00,290][20381] Updated weights for policy 1, policy_version 1889656 (0.0006) [2023-12-27 18:45:00,426][20347] Updated weights for policy 0, policy_version 1891290 (0.0008) [2023-12-27 18:45:00,491][20347] Updated weights for policy 0, policy_version 1891300 (0.0008) [2023-12-27 18:45:00,547][20347] Updated weights for policy 0, policy_version 1891310 (0.0010) [2023-12-27 18:45:00,950][20381] Updated weights for policy 1, policy_version 1889666 (0.0006) [2023-12-27 18:45:00,998][20381] Updated weights for policy 1, policy_version 1889676 (0.0007) [2023-12-27 18:45:01,058][20381] Updated weights for policy 1, policy_version 1889686 (0.0007) [2023-12-27 18:45:01,108][20381] Updated weights for policy 1, policy_version 1889696 (0.0006) [2023-12-27 18:45:01,154][20347] Updated weights for policy 0, policy_version 1891320 (0.0008) [2023-12-27 18:45:01,216][20347] Updated weights for policy 0, policy_version 1891330 (0.0007) [2023-12-27 18:45:01,277][20347] Updated weights for policy 0, policy_version 1891340 (0.0007) [2023-12-27 18:45:01,902][20381] Updated weights for policy 1, policy_version 1889706 (0.0006) [2023-12-27 18:45:01,909][20347] Updated weights for policy 0, policy_version 1891350 (0.0008) [2023-12-27 18:45:01,953][20381] Updated weights for policy 1, policy_version 1889716 (0.0006) [2023-12-27 18:45:01,963][20347] Updated weights for policy 0, policy_version 1891360 (0.0006) [2023-12-27 18:45:01,999][20381] Updated weights for policy 1, policy_version 1889726 (0.0005) [2023-12-27 18:45:02,020][20347] Updated weights for policy 0, policy_version 1891370 (0.0007) [2023-12-27 18:45:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 968105984. Throughput: 0: 10181.4, 1: 10392.7. Samples: 968114884. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '923.520')] [2023-12-27 18:45:02,673][20381] Updated weights for policy 1, policy_version 1889736 (0.0006) [2023-12-27 18:45:02,736][20381] Updated weights for policy 1, policy_version 1889746 (0.0006) [2023-12-27 18:45:02,759][20347] Updated weights for policy 0, policy_version 1891380 (0.0007) [2023-12-27 18:45:02,804][20381] Updated weights for policy 1, policy_version 1889756 (0.0006) [2023-12-27 18:45:02,811][20347] Updated weights for policy 0, policy_version 1891390 (0.0005) [2023-12-27 18:45:02,871][20347] Updated weights for policy 0, policy_version 1891400 (0.0007) [2023-12-27 18:45:03,449][20347] Updated weights for policy 0, policy_version 1891410 (0.0007) [2023-12-27 18:45:03,450][20381] Updated weights for policy 1, policy_version 1889766 (0.0008) [2023-12-27 18:45:03,503][20381] Updated weights for policy 1, policy_version 1889776 (0.0008) [2023-12-27 18:45:03,508][20347] Updated weights for policy 0, policy_version 1891420 (0.0005) [2023-12-27 18:45:03,554][20381] Updated weights for policy 1, policy_version 1889786 (0.0007) [2023-12-27 18:45:03,563][20347] Updated weights for policy 0, policy_version 1891430 (0.0008) [2023-12-27 18:45:03,622][20347] Updated weights for policy 0, policy_version 1891440 (0.0008) [2023-12-27 18:45:04,195][20347] Updated weights for policy 0, policy_version 1891450 (0.0010) [2023-12-27 18:45:04,196][20381] Updated weights for policy 1, policy_version 1889796 (0.0006) [2023-12-27 18:45:04,248][20347] Updated weights for policy 0, policy_version 1891460 (0.0010) [2023-12-27 18:45:04,253][20381] Updated weights for policy 1, policy_version 1889806 (0.0008) [2023-12-27 18:45:04,304][20347] Updated weights for policy 0, policy_version 1891470 (0.0010) [2023-12-27 18:45:04,309][20381] Updated weights for policy 1, policy_version 1889816 (0.0006) [2023-12-27 18:45:04,988][20381] Updated weights for policy 1, policy_version 1889826 (0.0007) [2023-12-27 18:45:04,990][20347] Updated weights for policy 0, policy_version 1891480 (0.0010) [2023-12-27 18:45:05,047][20381] Updated weights for policy 1, policy_version 1889836 (0.0010) [2023-12-27 18:45:05,049][20347] Updated weights for policy 0, policy_version 1891490 (0.0010) [2023-12-27 18:45:05,106][20347] Updated weights for policy 0, policy_version 1891500 (0.0010) [2023-12-27 18:45:05,109][20381] Updated weights for policy 1, policy_version 1889846 (0.0011) [2023-12-27 18:45:05,160][20381] Updated weights for policy 1, policy_version 1889856 (0.0010) [2023-12-27 18:45:05,777][20347] Updated weights for policy 0, policy_version 1891510 (0.0011) [2023-12-27 18:45:05,840][20347] Updated weights for policy 0, policy_version 1891520 (0.0010) [2023-12-27 18:45:05,855][20381] Updated weights for policy 1, policy_version 1889866 (0.0010) [2023-12-27 18:45:05,900][20347] Updated weights for policy 0, policy_version 1891530 (0.0010) [2023-12-27 18:45:05,912][20381] Updated weights for policy 1, policy_version 1889876 (0.0011) [2023-12-27 18:45:05,969][20381] Updated weights for policy 1, policy_version 1889886 (0.0007) [2023-12-27 18:45:06,597][20347] Updated weights for policy 0, policy_version 1891540 (0.0009) [2023-12-27 18:45:06,607][20381] Updated weights for policy 1, policy_version 1889896 (0.0008) [2023-12-27 18:45:06,644][20347] Updated weights for policy 0, policy_version 1891550 (0.0008) [2023-12-27 18:45:06,663][20381] Updated weights for policy 1, policy_version 1889906 (0.0007) [2023-12-27 18:45:06,689][20347] Updated weights for policy 0, policy_version 1891560 (0.0007) [2023-12-27 18:45:06,717][20381] Updated weights for policy 1, policy_version 1889916 (0.0005) [2023-12-27 18:45:07,321][20381] Updated weights for policy 1, policy_version 1889926 (0.0006) [2023-12-27 18:45:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 968204288. Throughput: 0: 10229.4, 1: 10415.5. Samples: 968178452. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '890.860')] [2023-12-27 18:45:07,366][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001891568_484327424.pth... [2023-12-27 18:45:07,369][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001890416_484032512.pth [2023-12-27 18:45:07,380][20381] Updated weights for policy 1, policy_version 1889936 (0.0009) [2023-12-27 18:45:07,401][20347] Updated weights for policy 0, policy_version 1891570 (0.0008) [2023-12-27 18:45:07,443][20381] Updated weights for policy 1, policy_version 1889946 (0.0010) [2023-12-27 18:45:07,453][20347] Updated weights for policy 0, policy_version 1891580 (0.0010) [2023-12-27 18:45:07,475][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001889952_483885056.pth... [2023-12-27 18:45:07,479][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001888704_483565568.pth [2023-12-27 18:45:07,509][20347] Updated weights for policy 0, policy_version 1891590 (0.0010) [2023-12-27 18:45:07,570][20347] Updated weights for policy 0, policy_version 1891600 (0.0010) [2023-12-27 18:45:08,075][20381] Updated weights for policy 1, policy_version 1889956 (0.0011) [2023-12-27 18:45:08,127][20381] Updated weights for policy 1, policy_version 1889966 (0.0010) [2023-12-27 18:45:08,179][20381] Updated weights for policy 1, policy_version 1889976 (0.0010) [2023-12-27 18:45:08,251][20347] Updated weights for policy 0, policy_version 1891610 (0.0010) [2023-12-27 18:45:08,295][20347] Updated weights for policy 0, policy_version 1891620 (0.0010) [2023-12-27 18:45:08,364][20347] Updated weights for policy 0, policy_version 1891630 (0.0010) [2023-12-27 18:45:08,887][20381] Updated weights for policy 1, policy_version 1889986 (0.0009) [2023-12-27 18:45:08,946][20381] Updated weights for policy 1, policy_version 1889996 (0.0007) [2023-12-27 18:45:09,001][20381] Updated weights for policy 1, policy_version 1890006 (0.0007) [2023-12-27 18:45:09,056][20381] Updated weights for policy 1, policy_version 1890016 (0.0006) [2023-12-27 18:45:09,064][20347] Updated weights for policy 0, policy_version 1891640 (0.0010) [2023-12-27 18:45:09,128][20347] Updated weights for policy 0, policy_version 1891650 (0.0010) [2023-12-27 18:45:09,192][20347] Updated weights for policy 0, policy_version 1891660 (0.0006) [2023-12-27 18:45:09,697][20381] Updated weights for policy 1, policy_version 1890026 (0.0008) [2023-12-27 18:45:09,755][20381] Updated weights for policy 1, policy_version 1890036 (0.0005) [2023-12-27 18:45:09,806][20381] Updated weights for policy 1, policy_version 1890046 (0.0007) [2023-12-27 18:45:09,860][20347] Updated weights for policy 0, policy_version 1891670 (0.0008) [2023-12-27 18:45:09,918][20347] Updated weights for policy 0, policy_version 1891680 (0.0009) [2023-12-27 18:45:09,984][20347] Updated weights for policy 0, policy_version 1891690 (0.0010) [2023-12-27 18:45:10,534][20381] Updated weights for policy 1, policy_version 1890056 (0.0010) [2023-12-27 18:45:10,586][20381] Updated weights for policy 1, policy_version 1890066 (0.0011) [2023-12-27 18:45:10,639][20381] Updated weights for policy 1, policy_version 1890076 (0.0011) [2023-12-27 18:45:10,667][20347] Updated weights for policy 0, policy_version 1891700 (0.0010) [2023-12-27 18:45:10,717][20347] Updated weights for policy 0, policy_version 1891710 (0.0010) [2023-12-27 18:45:10,771][20347] Updated weights for policy 0, policy_version 1891720 (0.0010) [2023-12-27 18:45:11,347][20381] Updated weights for policy 1, policy_version 1890086 (0.0008) [2023-12-27 18:45:11,428][20381] Updated weights for policy 1, policy_version 1890096 (0.0009) [2023-12-27 18:45:11,483][20381] Updated weights for policy 1, policy_version 1890106 (0.0009) [2023-12-27 18:45:11,517][20347] Updated weights for policy 0, policy_version 1891730 (0.0009) [2023-12-27 18:45:11,570][20347] Updated weights for policy 0, policy_version 1891740 (0.0009) [2023-12-27 18:45:11,640][20347] Updated weights for policy 0, policy_version 1891750 (0.0009) [2023-12-27 18:45:11,696][20347] Updated weights for policy 0, policy_version 1891760 (0.0009) [2023-12-27 18:45:12,316][20381] Updated weights for policy 1, policy_version 1890116 (0.0008) [2023-12-27 18:45:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 968302592. Throughput: 0: 10180.9, 1: 10444.3. Samples: 968300448. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.810')] [2023-12-27 18:45:12,376][20381] Updated weights for policy 1, policy_version 1890126 (0.0008) [2023-12-27 18:45:12,430][20381] Updated weights for policy 1, policy_version 1890136 (0.0008) [2023-12-27 18:45:12,501][20347] Updated weights for policy 0, policy_version 1891770 (0.0009) [2023-12-27 18:45:12,555][20347] Updated weights for policy 0, policy_version 1891780 (0.0007) [2023-12-27 18:45:12,617][20347] Updated weights for policy 0, policy_version 1891790 (0.0007) [2023-12-27 18:45:13,095][20381] Updated weights for policy 1, policy_version 1890146 (0.0007) [2023-12-27 18:45:13,143][20381] Updated weights for policy 1, policy_version 1890156 (0.0006) [2023-12-27 18:45:13,199][20381] Updated weights for policy 1, policy_version 1890166 (0.0006) [2023-12-27 18:45:13,256][20381] Updated weights for policy 1, policy_version 1890176 (0.0005) [2023-12-27 18:45:13,288][20347] Updated weights for policy 0, policy_version 1891800 (0.0008) [2023-12-27 18:45:13,336][20347] Updated weights for policy 0, policy_version 1891810 (0.0007) [2023-12-27 18:45:13,382][20347] Updated weights for policy 0, policy_version 1891820 (0.0007) [2023-12-27 18:45:13,879][20381] Updated weights for policy 1, policy_version 1890186 (0.0008) [2023-12-27 18:45:13,925][20381] Updated weights for policy 1, policy_version 1890196 (0.0008) [2023-12-27 18:45:13,974][20381] Updated weights for policy 1, policy_version 1890206 (0.0008) [2023-12-27 18:45:14,103][20347] Updated weights for policy 0, policy_version 1891830 (0.0007) [2023-12-27 18:45:14,161][20347] Updated weights for policy 0, policy_version 1891840 (0.0008) [2023-12-27 18:45:14,220][20347] Updated weights for policy 0, policy_version 1891850 (0.0007) [2023-12-27 18:45:14,720][20381] Updated weights for policy 1, policy_version 1890216 (0.0008) [2023-12-27 18:45:14,771][20381] Updated weights for policy 1, policy_version 1890226 (0.0008) [2023-12-27 18:45:14,826][20381] Updated weights for policy 1, policy_version 1890236 (0.0008) [2023-12-27 18:45:14,904][20347] Updated weights for policy 0, policy_version 1891860 (0.0009) [2023-12-27 18:45:14,969][20347] Updated weights for policy 0, policy_version 1891870 (0.0008) [2023-12-27 18:45:15,027][20347] Updated weights for policy 0, policy_version 1891880 (0.0007) [2023-12-27 18:45:15,515][20381] Updated weights for policy 1, policy_version 1890246 (0.0008) [2023-12-27 18:45:15,574][20381] Updated weights for policy 1, policy_version 1890256 (0.0008) [2023-12-27 18:45:15,633][20381] Updated weights for policy 1, policy_version 1890266 (0.0007) [2023-12-27 18:45:15,729][20347] Updated weights for policy 0, policy_version 1891890 (0.0008) [2023-12-27 18:45:15,783][20347] Updated weights for policy 0, policy_version 1891900 (0.0008) [2023-12-27 18:45:15,838][20347] Updated weights for policy 0, policy_version 1891910 (0.0008) [2023-12-27 18:45:15,890][20347] Updated weights for policy 0, policy_version 1891920 (0.0008) [2023-12-27 18:45:16,287][20381] Updated weights for policy 1, policy_version 1890276 (0.0007) [2023-12-27 18:45:16,341][20381] Updated weights for policy 1, policy_version 1890286 (0.0008) [2023-12-27 18:45:16,398][20381] Updated weights for policy 1, policy_version 1890296 (0.0006) [2023-12-27 18:45:16,563][20347] Updated weights for policy 0, policy_version 1891930 (0.0005) [2023-12-27 18:45:16,610][20347] Updated weights for policy 0, policy_version 1891940 (0.0007) [2023-12-27 18:45:16,663][20347] Updated weights for policy 0, policy_version 1891950 (0.0007) [2023-12-27 18:45:17,019][20381] Updated weights for policy 1, policy_version 1890306 (0.0007) [2023-12-27 18:45:17,070][20381] Updated weights for policy 1, policy_version 1890316 (0.0005) [2023-12-27 18:45:17,119][20381] Updated weights for policy 1, policy_version 1890326 (0.0005) [2023-12-27 18:45:17,167][20381] Updated weights for policy 1, policy_version 1890336 (0.0006) [2023-12-27 18:45:17,363][19170] Fps is (10 sec: 20479.2, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 968409088. Throughput: 0: 10080.1, 1: 10412.0. Samples: 968419300. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '921.200')] [2023-12-27 18:45:17,386][20347] Updated weights for policy 0, policy_version 1891960 (0.0008) [2023-12-27 18:45:17,450][20347] Updated weights for policy 0, policy_version 1891970 (0.0008) [2023-12-27 18:45:17,507][20347] Updated weights for policy 0, policy_version 1891980 (0.0008) [2023-12-27 18:45:17,820][20381] Updated weights for policy 1, policy_version 1890346 (0.0005) [2023-12-27 18:45:17,883][20381] Updated weights for policy 1, policy_version 1890356 (0.0007) [2023-12-27 18:45:17,941][20381] Updated weights for policy 1, policy_version 1890366 (0.0008) [2023-12-27 18:45:18,117][20347] Updated weights for policy 0, policy_version 1891990 (0.0007) [2023-12-27 18:45:18,174][20347] Updated weights for policy 0, policy_version 1892000 (0.0005) [2023-12-27 18:45:18,231][20347] Updated weights for policy 0, policy_version 1892010 (0.0009) [2023-12-27 18:45:18,614][20381] Updated weights for policy 1, policy_version 1890376 (0.0007) [2023-12-27 18:45:18,675][20381] Updated weights for policy 1, policy_version 1890386 (0.0007) [2023-12-27 18:45:18,736][20381] Updated weights for policy 1, policy_version 1890396 (0.0009) [2023-12-27 18:45:18,914][20347] Updated weights for policy 0, policy_version 1892020 (0.0009) [2023-12-27 18:45:18,966][20347] Updated weights for policy 0, policy_version 1892030 (0.0008) [2023-12-27 18:45:19,024][20347] Updated weights for policy 0, policy_version 1892040 (0.0007) [2023-12-27 18:45:19,438][20381] Updated weights for policy 1, policy_version 1890406 (0.0007) [2023-12-27 18:45:19,503][20381] Updated weights for policy 1, policy_version 1890416 (0.0008) [2023-12-27 18:45:19,560][20381] Updated weights for policy 1, policy_version 1890426 (0.0007) [2023-12-27 18:45:19,760][20347] Updated weights for policy 0, policy_version 1892050 (0.0008) [2023-12-27 18:45:19,819][20347] Updated weights for policy 0, policy_version 1892060 (0.0008) [2023-12-27 18:45:19,876][20347] Updated weights for policy 0, policy_version 1892070 (0.0008) [2023-12-27 18:45:19,932][20347] Updated weights for policy 0, policy_version 1892080 (0.0008) [2023-12-27 18:45:20,280][20381] Updated weights for policy 1, policy_version 1890436 (0.0008) [2023-12-27 18:45:20,343][20381] Updated weights for policy 1, policy_version 1890446 (0.0007) [2023-12-27 18:45:20,400][20381] Updated weights for policy 1, policy_version 1890456 (0.0007) [2023-12-27 18:45:20,654][20347] Updated weights for policy 0, policy_version 1892090 (0.0008) [2023-12-27 18:45:20,704][20347] Updated weights for policy 0, policy_version 1892100 (0.0007) [2023-12-27 18:45:20,762][20347] Updated weights for policy 0, policy_version 1892110 (0.0009) [2023-12-27 18:45:21,108][20381] Updated weights for policy 1, policy_version 1890466 (0.0006) [2023-12-27 18:45:21,174][20381] Updated weights for policy 1, policy_version 1890476 (0.0008) [2023-12-27 18:45:21,233][20381] Updated weights for policy 1, policy_version 1890486 (0.0008) [2023-12-27 18:45:21,301][20381] Updated weights for policy 1, policy_version 1890496 (0.0008) [2023-12-27 18:45:21,518][20347] Updated weights for policy 0, policy_version 1892120 (0.0008) [2023-12-27 18:45:21,577][20347] Updated weights for policy 0, policy_version 1892130 (0.0008) [2023-12-27 18:45:21,641][20347] Updated weights for policy 0, policy_version 1892140 (0.0009) [2023-12-27 18:45:22,006][20381] Updated weights for policy 1, policy_version 1890506 (0.0006) [2023-12-27 18:45:22,057][20381] Updated weights for policy 1, policy_version 1890516 (0.0005) [2023-12-27 18:45:22,119][20381] Updated weights for policy 1, policy_version 1890526 (0.0006) [2023-12-27 18:45:22,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 968507392. Throughput: 0: 9997.0, 1: 10414.7. Samples: 968479480. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '908.660')] [2023-12-27 18:45:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001890528_484032512.pth... [2023-12-27 18:45:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001889312_483721216.pth [2023-12-27 18:45:22,376][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001892144_484474880.pth... [2023-12-27 18:45:22,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001890992_484179968.pth [2023-12-27 18:45:22,443][20347] Updated weights for policy 0, policy_version 1892150 (0.0007) [2023-12-27 18:45:22,501][20347] Updated weights for policy 0, policy_version 1892160 (0.0006) [2023-12-27 18:45:22,556][20347] Updated weights for policy 0, policy_version 1892170 (0.0006) [2023-12-27 18:45:22,704][20381] Updated weights for policy 1, policy_version 1890536 (0.0006) [2023-12-27 18:45:22,766][20381] Updated weights for policy 1, policy_version 1890546 (0.0006) [2023-12-27 18:45:22,825][20381] Updated weights for policy 1, policy_version 1890556 (0.0006) [2023-12-27 18:45:23,156][20347] Updated weights for policy 0, policy_version 1892180 (0.0008) [2023-12-27 18:45:23,215][20347] Updated weights for policy 0, policy_version 1892190 (0.0006) [2023-12-27 18:45:23,274][20347] Updated weights for policy 0, policy_version 1892200 (0.0006) [2023-12-27 18:45:23,419][20381] Updated weights for policy 1, policy_version 1890566 (0.0007) [2023-12-27 18:45:23,467][20381] Updated weights for policy 1, policy_version 1890576 (0.0007) [2023-12-27 18:45:23,516][20381] Updated weights for policy 1, policy_version 1890586 (0.0007) [2023-12-27 18:45:23,953][20347] Updated weights for policy 0, policy_version 1892210 (0.0008) [2023-12-27 18:45:24,009][20347] Updated weights for policy 0, policy_version 1892220 (0.0008) [2023-12-27 18:45:24,067][20347] Updated weights for policy 0, policy_version 1892230 (0.0007) [2023-12-27 18:45:24,121][20347] Updated weights for policy 0, policy_version 1892240 (0.0007) [2023-12-27 18:45:24,144][20381] Updated weights for policy 1, policy_version 1890596 (0.0008) [2023-12-27 18:45:24,206][20381] Updated weights for policy 1, policy_version 1890606 (0.0009) [2023-12-27 18:45:24,262][20381] Updated weights for policy 1, policy_version 1890616 (0.0007) [2023-12-27 18:45:24,832][20381] Updated weights for policy 1, policy_version 1890626 (0.0005) [2023-12-27 18:45:24,845][20347] Updated weights for policy 0, policy_version 1892250 (0.0009) [2023-12-27 18:45:24,883][20381] Updated weights for policy 1, policy_version 1890636 (0.0005) [2023-12-27 18:45:24,895][20347] Updated weights for policy 0, policy_version 1892260 (0.0008) [2023-12-27 18:45:24,932][20381] Updated weights for policy 1, policy_version 1890646 (0.0005) [2023-12-27 18:45:24,944][20347] Updated weights for policy 0, policy_version 1892270 (0.0007) [2023-12-27 18:45:24,981][20381] Updated weights for policy 1, policy_version 1890656 (0.0005) [2023-12-27 18:45:25,576][20347] Updated weights for policy 0, policy_version 1892280 (0.0008) [2023-12-27 18:45:25,626][20347] Updated weights for policy 0, policy_version 1892290 (0.0011) [2023-12-27 18:45:25,631][20381] Updated weights for policy 1, policy_version 1890666 (0.0005) [2023-12-27 18:45:25,678][20347] Updated weights for policy 0, policy_version 1892300 (0.0010) [2023-12-27 18:45:25,686][20381] Updated weights for policy 1, policy_version 1890676 (0.0006) [2023-12-27 18:45:25,739][20381] Updated weights for policy 1, policy_version 1890686 (0.0007) [2023-12-27 18:45:26,331][20347] Updated weights for policy 0, policy_version 1892310 (0.0010) [2023-12-27 18:45:26,385][20347] Updated weights for policy 0, policy_version 1892320 (0.0010) [2023-12-27 18:45:26,409][20381] Updated weights for policy 1, policy_version 1890696 (0.0008) [2023-12-27 18:45:26,442][20347] Updated weights for policy 0, policy_version 1892330 (0.0008) [2023-12-27 18:45:26,465][20381] Updated weights for policy 1, policy_version 1890706 (0.0007) [2023-12-27 18:45:26,518][20381] Updated weights for policy 1, policy_version 1890716 (0.0009) [2023-12-27 18:45:27,019][20347] Updated weights for policy 0, policy_version 1892340 (0.0006) [2023-12-27 18:45:27,071][20347] Updated weights for policy 0, policy_version 1892350 (0.0005) [2023-12-27 18:45:27,126][20347] Updated weights for policy 0, policy_version 1892360 (0.0005) [2023-12-27 18:45:27,271][20381] Updated weights for policy 1, policy_version 1890727 (0.0009) [2023-12-27 18:45:27,323][20381] Updated weights for policy 1, policy_version 1890737 (0.0009) [2023-12-27 18:45:27,362][19170] Fps is (10 sec: 20481.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 968613888. Throughput: 0: 9975.0, 1: 10479.4. Samples: 968603968. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '856.700')] [2023-12-27 18:45:27,367][20381] Updated weights for policy 1, policy_version 1890747 (0.0008) [2023-12-27 18:45:27,653][20347] Updated weights for policy 0, policy_version 1892370 (0.0006) [2023-12-27 18:45:27,703][20347] Updated weights for policy 0, policy_version 1892380 (0.0010) [2023-12-27 18:45:27,761][20347] Updated weights for policy 0, policy_version 1892390 (0.0010) [2023-12-27 18:45:27,814][20347] Updated weights for policy 0, policy_version 1892400 (0.0010) [2023-12-27 18:45:28,111][20381] Updated weights for policy 1, policy_version 1890757 (0.0008) [2023-12-27 18:45:28,168][20381] Updated weights for policy 1, policy_version 1890767 (0.0008) [2023-12-27 18:45:28,225][20381] Updated weights for policy 1, policy_version 1890777 (0.0007) [2023-12-27 18:45:28,462][20347] Updated weights for policy 0, policy_version 1892410 (0.0010) [2023-12-27 18:45:28,493][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-12-27 18:45:28,970][20381] Updated weights for policy 1, policy_version 1890787 (0.0008) [2023-12-27 18:45:29,015][20381] Updated weights for policy 1, policy_version 1890797 (0.0008) [2023-12-27 18:45:29,062][20381] Updated weights for policy 1, policy_version 1890807 (0.0008) [2023-12-27 18:45:29,202][20347] Updated weights for policy 0, policy_version 1892420 (0.0008) [2023-12-27 18:45:29,262][20347] Updated weights for policy 0, policy_version 1892430 (0.0008) [2023-12-27 18:45:29,319][20347] Updated weights for policy 0, policy_version 1892440 (0.0007) [2023-12-27 18:45:29,852][20381] Updated weights for policy 1, policy_version 1890817 (0.0010) [2023-12-27 18:45:29,905][20381] Updated weights for policy 1, policy_version 1890827 (0.0008) [2023-12-27 18:45:29,965][20381] Updated weights for policy 1, policy_version 1890837 (0.0007) [2023-12-27 18:45:30,023][20381] Updated weights for policy 1, policy_version 1890847 (0.0005) [2023-12-27 18:45:30,053][20347] Updated weights for policy 0, policy_version 1892450 (0.0009) [2023-12-27 18:45:30,105][20347] Updated weights for policy 0, policy_version 1892460 (0.0010) [2023-12-27 18:45:30,151][20347] Updated weights for policy 0, policy_version 1892470 (0.0010) [2023-12-27 18:45:30,196][20347] Updated weights for policy 0, policy_version 1892480 (0.0010) [2023-12-27 18:45:30,707][20381] Updated weights for policy 1, policy_version 1890857 (0.0007) [2023-12-27 18:45:30,758][20381] Updated weights for policy 1, policy_version 1890867 (0.0007) [2023-12-27 18:45:30,816][20381] Updated weights for policy 1, policy_version 1890877 (0.0008) [2023-12-27 18:45:30,872][20347] Updated weights for policy 0, policy_version 1892490 (0.0007) [2023-12-27 18:45:30,929][20347] Updated weights for policy 0, policy_version 1892500 (0.0008) [2023-12-27 18:45:30,984][20347] Updated weights for policy 0, policy_version 1892510 (0.0006) [2023-12-27 18:45:31,564][20381] Updated weights for policy 1, policy_version 1890887 (0.0007) [2023-12-27 18:45:31,619][20381] Updated weights for policy 1, policy_version 1890897 (0.0009) [2023-12-27 18:45:31,664][20347] Updated weights for policy 0, policy_version 1892520 (0.0008) [2023-12-27 18:45:31,676][20381] Updated weights for policy 1, policy_version 1890907 (0.0006) [2023-12-27 18:45:31,723][20347] Updated weights for policy 0, policy_version 1892530 (0.0008) [2023-12-27 18:45:31,779][20347] Updated weights for policy 0, policy_version 1892540 (0.0008) [2023-12-27 18:45:32,358][20381] Updated weights for policy 1, policy_version 1890917 (0.0008) [2023-12-27 18:45:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 968712192. Throughput: 0: 10035.6, 1: 10307.9. Samples: 968725336. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '867.840')] [2023-12-27 18:45:32,420][20381] Updated weights for policy 1, policy_version 1890927 (0.0009) [2023-12-27 18:45:32,477][20381] Updated weights for policy 1, policy_version 1890937 (0.0008) [2023-12-27 18:45:32,490][20347] Updated weights for policy 0, policy_version 1892550 (0.0007) [2023-12-27 18:45:32,548][20347] Updated weights for policy 0, policy_version 1892560 (0.0007) [2023-12-27 18:45:32,594][20347] Updated weights for policy 0, policy_version 1892570 (0.0008) [2023-12-27 18:45:33,133][20381] Updated weights for policy 1, policy_version 1890947 (0.0006) [2023-12-27 18:45:33,177][20347] Updated weights for policy 0, policy_version 1892580 (0.0006) [2023-12-27 18:45:33,195][20381] Updated weights for policy 1, policy_version 1890957 (0.0007) [2023-12-27 18:45:33,224][20347] Updated weights for policy 0, policy_version 1892590 (0.0006) [2023-12-27 18:45:33,241][20381] Updated weights for policy 1, policy_version 1890967 (0.0006) [2023-12-27 18:45:33,267][20347] Updated weights for policy 0, policy_version 1892600 (0.0006) [2023-12-27 18:45:33,897][20347] Updated weights for policy 0, policy_version 1892610 (0.0006) [2023-12-27 18:45:33,906][20381] Updated weights for policy 1, policy_version 1890977 (0.0007) [2023-12-27 18:45:33,957][20347] Updated weights for policy 0, policy_version 1892620 (0.0010) [2023-12-27 18:45:33,969][20381] Updated weights for policy 1, policy_version 1890987 (0.0008) [2023-12-27 18:45:34,006][20347] Updated weights for policy 0, policy_version 1892630 (0.0010) [2023-12-27 18:45:34,023][20381] Updated weights for policy 1, policy_version 1890997 (0.0010) [2023-12-27 18:45:34,056][20347] Updated weights for policy 0, policy_version 1892640 (0.0010) [2023-12-27 18:45:34,067][20381] Updated weights for policy 1, policy_version 1891007 (0.0010) [2023-12-27 18:45:34,760][20347] Updated weights for policy 0, policy_version 1892650 (0.0007) [2023-12-27 18:45:34,783][20381] Updated weights for policy 1, policy_version 1891017 (0.0007) [2023-12-27 18:45:34,816][20347] Updated weights for policy 0, policy_version 1892660 (0.0006) [2023-12-27 18:45:34,844][20381] Updated weights for policy 1, policy_version 1891027 (0.0007) [2023-12-27 18:45:34,869][20347] Updated weights for policy 0, policy_version 1892670 (0.0008) [2023-12-27 18:45:34,903][20381] Updated weights for policy 1, policy_version 1891037 (0.0006) [2023-12-27 18:45:35,463][20381] Updated weights for policy 1, policy_version 1891047 (0.0007) [2023-12-27 18:45:35,510][20381] Updated weights for policy 1, policy_version 1891057 (0.0008) [2023-12-27 18:45:35,555][20381] Updated weights for policy 1, policy_version 1891067 (0.0008) [2023-12-27 18:45:35,598][20347] Updated weights for policy 0, policy_version 1892680 (0.0006) [2023-12-27 18:45:35,653][20347] Updated weights for policy 0, policy_version 1892690 (0.0009) [2023-12-27 18:45:35,714][20347] Updated weights for policy 0, policy_version 1892700 (0.0009) [2023-12-27 18:45:36,241][20381] Updated weights for policy 1, policy_version 1891077 (0.0007) [2023-12-27 18:45:36,301][20381] Updated weights for policy 1, policy_version 1891087 (0.0007) [2023-12-27 18:45:36,357][20381] Updated weights for policy 1, policy_version 1891097 (0.0008) [2023-12-27 18:45:36,429][20347] Updated weights for policy 0, policy_version 1892710 (0.0007) [2023-12-27 18:45:36,481][20347] Updated weights for policy 0, policy_version 1892720 (0.0007) [2023-12-27 18:45:36,544][20347] Updated weights for policy 0, policy_version 1892730 (0.0006) [2023-12-27 18:45:37,035][20381] Updated weights for policy 1, policy_version 1891107 (0.0007) [2023-12-27 18:45:37,089][20381] Updated weights for policy 1, policy_version 1891117 (0.0007) [2023-12-27 18:45:37,141][20381] Updated weights for policy 1, policy_version 1891127 (0.0008) [2023-12-27 18:45:37,264][20347] Updated weights for policy 0, policy_version 1892740 (0.0008) [2023-12-27 18:45:37,323][20347] Updated weights for policy 0, policy_version 1892750 (0.0007) [2023-12-27 18:45:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 968818688. Throughput: 0: 10082.8, 1: 10289.9. Samples: 968788980. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '896.360')] [2023-12-27 18:45:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001891136_484188160.pth... [2023-12-27 18:45:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001889952_483885056.pth [2023-12-27 18:45:37,386][20347] Updated weights for policy 0, policy_version 1892760 (0.0008) [2023-12-27 18:45:37,438][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001892768_484638720.pth... [2023-12-27 18:45:37,443][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001891568_484327424.pth [2023-12-27 18:45:37,850][20381] Updated weights for policy 1, policy_version 1891137 (0.0008) [2023-12-27 18:45:37,899][20381] Updated weights for policy 1, policy_version 1891147 (0.0009) [2023-12-27 18:45:37,952][20381] Updated weights for policy 1, policy_version 1891157 (0.0009) [2023-12-27 18:45:37,996][20347] Updated weights for policy 0, policy_version 1892770 (0.0007) [2023-12-27 18:45:38,002][20381] Updated weights for policy 1, policy_version 1891167 (0.0008) [2023-12-27 18:45:38,047][20347] Updated weights for policy 0, policy_version 1892780 (0.0007) [2023-12-27 18:45:38,113][20347] Updated weights for policy 0, policy_version 1892790 (0.0006) [2023-12-27 18:45:38,159][20347] Updated weights for policy 0, policy_version 1892800 (0.0005) [2023-12-27 18:45:38,758][20381] Updated weights for policy 1, policy_version 1891177 (0.0008) [2023-12-27 18:45:38,820][20381] Updated weights for policy 1, policy_version 1891187 (0.0007) [2023-12-27 18:45:38,863][20347] Updated weights for policy 0, policy_version 1892810 (0.0011) [2023-12-27 18:45:38,883][20381] Updated weights for policy 1, policy_version 1891197 (0.0005) [2023-12-27 18:45:38,923][20347] Updated weights for policy 0, policy_version 1892820 (0.0011) [2023-12-27 18:45:38,984][20347] Updated weights for policy 0, policy_version 1892830 (0.0011) [2023-12-27 18:45:39,605][20381] Updated weights for policy 1, policy_version 1891207 (0.0007) [2023-12-27 18:45:39,669][20381] Updated weights for policy 1, policy_version 1891217 (0.0008) [2023-12-27 18:45:39,690][20347] Updated weights for policy 0, policy_version 1892840 (0.0007) [2023-12-27 18:45:39,731][20381] Updated weights for policy 1, policy_version 1891227 (0.0007) [2023-12-27 18:45:39,750][20347] Updated weights for policy 0, policy_version 1892850 (0.0010) [2023-12-27 18:45:39,807][20347] Updated weights for policy 0, policy_version 1892860 (0.0011) [2023-12-27 18:45:40,416][20381] Updated weights for policy 1, policy_version 1891237 (0.0006) [2023-12-27 18:45:40,422][20347] Updated weights for policy 0, policy_version 1892870 (0.0009) [2023-12-27 18:45:40,474][20381] Updated weights for policy 1, policy_version 1891247 (0.0006) [2023-12-27 18:45:40,475][20347] Updated weights for policy 0, policy_version 1892880 (0.0008) [2023-12-27 18:45:40,527][20381] Updated weights for policy 1, policy_version 1891257 (0.0006) [2023-12-27 18:45:40,529][20347] Updated weights for policy 0, policy_version 1892890 (0.0006) [2023-12-27 18:45:41,201][20347] Updated weights for policy 0, policy_version 1892900 (0.0008) [2023-12-27 18:45:41,267][20347] Updated weights for policy 0, policy_version 1892910 (0.0008) [2023-12-27 18:45:41,279][20381] Updated weights for policy 1, policy_version 1891267 (0.0007) [2023-12-27 18:45:41,325][20347] Updated weights for policy 0, policy_version 1892920 (0.0008) [2023-12-27 18:45:41,340][20381] Updated weights for policy 1, policy_version 1891277 (0.0007) [2023-12-27 18:45:41,417][20381] Updated weights for policy 1, policy_version 1891287 (0.0007) [2023-12-27 18:45:42,112][20347] Updated weights for policy 0, policy_version 1892930 (0.0008) [2023-12-27 18:45:42,164][20347] Updated weights for policy 0, policy_version 1892940 (0.0007) [2023-12-27 18:45:42,177][20381] Updated weights for policy 1, policy_version 1891297 (0.0007) [2023-12-27 18:45:42,220][20347] Updated weights for policy 0, policy_version 1892950 (0.0007) [2023-12-27 18:45:42,245][20381] Updated weights for policy 1, policy_version 1891308 (0.0008) [2023-12-27 18:45:42,287][20347] Updated weights for policy 0, policy_version 1892960 (0.0007) [2023-12-27 18:45:42,309][20381] Updated weights for policy 1, policy_version 1891318 (0.0008) [2023-12-27 18:45:42,366][19170] Fps is (10 sec: 20473.1, 60 sec: 20342.3, 300 sec: 20465.9). Total num frames: 968916992. Throughput: 0: 10167.7, 1: 10184.8. Samples: 968910264. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:42,366][19170] Avg episode reward: [(0, '1000.000'), (1, '849.470')] [2023-12-27 18:45:42,378][20381] Updated weights for policy 1, policy_version 1891328 (0.0008) [2023-12-27 18:45:43,009][20347] Updated weights for policy 0, policy_version 1892970 (0.0011) [2023-12-27 18:45:43,068][20347] Updated weights for policy 0, policy_version 1892980 (0.0011) [2023-12-27 18:45:43,123][20347] Updated weights for policy 0, policy_version 1892990 (0.0011) [2023-12-27 18:45:43,133][20381] Updated weights for policy 1, policy_version 1891338 (0.0005) [2023-12-27 18:45:43,188][20381] Updated weights for policy 1, policy_version 1891348 (0.0007) [2023-12-27 18:45:43,237][20381] Updated weights for policy 1, policy_version 1891358 (0.0007) [2023-12-27 18:45:43,746][20347] Updated weights for policy 0, policy_version 1893000 (0.0006) [2023-12-27 18:45:43,809][20347] Updated weights for policy 0, policy_version 1893010 (0.0006) [2023-12-27 18:45:43,869][20347] Updated weights for policy 0, policy_version 1893020 (0.0008) [2023-12-27 18:45:43,942][20381] Updated weights for policy 1, policy_version 1891368 (0.0007) [2023-12-27 18:45:43,997][20381] Updated weights for policy 1, policy_version 1891378 (0.0007) [2023-12-27 18:45:44,049][20381] Updated weights for policy 1, policy_version 1891388 (0.0008) [2023-12-27 18:45:44,543][20347] Updated weights for policy 0, policy_version 1893030 (0.0008) [2023-12-27 18:45:44,601][20347] Updated weights for policy 0, policy_version 1893040 (0.0007) [2023-12-27 18:45:44,655][20347] Updated weights for policy 0, policy_version 1893050 (0.0008) [2023-12-27 18:45:44,755][20381] Updated weights for policy 1, policy_version 1891398 (0.0008) [2023-12-27 18:45:44,821][20381] Updated weights for policy 1, policy_version 1891408 (0.0010) [2023-12-27 18:45:44,881][20381] Updated weights for policy 1, policy_version 1891418 (0.0011) [2023-12-27 18:45:45,404][20347] Updated weights for policy 0, policy_version 1893060 (0.0007) [2023-12-27 18:45:45,455][20347] Updated weights for policy 0, policy_version 1893070 (0.0007) [2023-12-27 18:45:45,523][20347] Updated weights for policy 0, policy_version 1893080 (0.0006) [2023-12-27 18:45:45,577][20381] Updated weights for policy 1, policy_version 1891428 (0.0010) [2023-12-27 18:45:45,635][20381] Updated weights for policy 1, policy_version 1891438 (0.0008) [2023-12-27 18:45:45,695][20381] Updated weights for policy 1, policy_version 1891448 (0.0008) [2023-12-27 18:45:46,166][20347] Updated weights for policy 0, policy_version 1893090 (0.0006) [2023-12-27 18:45:46,225][20347] Updated weights for policy 0, policy_version 1893100 (0.0007) [2023-12-27 18:45:46,287][20347] Updated weights for policy 0, policy_version 1893110 (0.0008) [2023-12-27 18:45:46,332][20347] Updated weights for policy 0, policy_version 1893120 (0.0007) [2023-12-27 18:45:46,369][20381] Updated weights for policy 1, policy_version 1891458 (0.0008) [2023-12-27 18:45:46,424][20381] Updated weights for policy 1, policy_version 1891468 (0.0007) [2023-12-27 18:45:46,479][20381] Updated weights for policy 1, policy_version 1891478 (0.0008) [2023-12-27 18:45:46,548][20381] Updated weights for policy 1, policy_version 1891488 (0.0008) [2023-12-27 18:45:47,030][20347] Updated weights for policy 0, policy_version 1893130 (0.0006) [2023-12-27 18:45:47,078][20347] Updated weights for policy 0, policy_version 1893140 (0.0005) [2023-12-27 18:45:47,134][20347] Updated weights for policy 0, policy_version 1893150 (0.0005) [2023-12-27 18:45:47,213][20381] Updated weights for policy 1, policy_version 1891498 (0.0007) [2023-12-27 18:45:47,270][20381] Updated weights for policy 1, policy_version 1891508 (0.0008) [2023-12-27 18:45:47,318][20381] Updated weights for policy 1, policy_version 1891518 (0.0007) [2023-12-27 18:45:47,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 969023488. Throughput: 0: 10164.4, 1: 10122.5. Samples: 969027792. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '790.230')] [2023-12-27 18:45:47,763][20347] Updated weights for policy 0, policy_version 1893160 (0.0006) [2023-12-27 18:45:47,811][20347] Updated weights for policy 0, policy_version 1893170 (0.0007) [2023-12-27 18:45:47,863][20347] Updated weights for policy 0, policy_version 1893180 (0.0008) [2023-12-27 18:45:48,027][20381] Updated weights for policy 1, policy_version 1891528 (0.0007) [2023-12-27 18:45:48,085][20381] Updated weights for policy 1, policy_version 1891538 (0.0006) [2023-12-27 18:45:48,147][20381] Updated weights for policy 1, policy_version 1891548 (0.0011) [2023-12-27 18:45:48,641][20347] Updated weights for policy 0, policy_version 1893190 (0.0008) [2023-12-27 18:45:48,702][20347] Updated weights for policy 0, policy_version 1893200 (0.0008) [2023-12-27 18:45:48,760][20347] Updated weights for policy 0, policy_version 1893210 (0.0008) [2023-12-27 18:45:48,791][20381] Updated weights for policy 1, policy_version 1891558 (0.0010) [2023-12-27 18:45:48,850][20381] Updated weights for policy 1, policy_version 1891568 (0.0011) [2023-12-27 18:45:48,912][20381] Updated weights for policy 1, policy_version 1891578 (0.0010) [2023-12-27 18:45:49,519][20347] Updated weights for policy 0, policy_version 1893220 (0.0007) [2023-12-27 18:45:49,583][20347] Updated weights for policy 0, policy_version 1893230 (0.0007) [2023-12-27 18:45:49,587][20381] Updated weights for policy 1, policy_version 1891588 (0.0010) [2023-12-27 18:45:49,642][20347] Updated weights for policy 0, policy_version 1893240 (0.0007) [2023-12-27 18:45:49,643][20381] Updated weights for policy 1, policy_version 1891598 (0.0008) [2023-12-27 18:45:49,699][20381] Updated weights for policy 1, policy_version 1891608 (0.0007) [2023-12-27 18:45:50,378][20347] Updated weights for policy 0, policy_version 1893250 (0.0007) [2023-12-27 18:45:50,434][20347] Updated weights for policy 0, policy_version 1893260 (0.0008) [2023-12-27 18:45:50,435][20381] Updated weights for policy 1, policy_version 1891618 (0.0009) [2023-12-27 18:45:50,490][20347] Updated weights for policy 0, policy_version 1893270 (0.0005) [2023-12-27 18:45:50,492][20381] Updated weights for policy 1, policy_version 1891628 (0.0007) [2023-12-27 18:45:50,547][20347] Updated weights for policy 0, policy_version 1893280 (0.0006) [2023-12-27 18:45:50,552][20381] Updated weights for policy 1, policy_version 1891638 (0.0006) [2023-12-27 18:45:50,614][20381] Updated weights for policy 1, policy_version 1891648 (0.0009) [2023-12-27 18:45:51,369][20347] Updated weights for policy 0, policy_version 1893290 (0.0008) [2023-12-27 18:45:51,427][20381] Updated weights for policy 1, policy_version 1891658 (0.0006) [2023-12-27 18:45:51,429][20347] Updated weights for policy 0, policy_version 1893300 (0.0007) [2023-12-27 18:45:51,477][20381] Updated weights for policy 1, policy_version 1891668 (0.0006) [2023-12-27 18:45:51,479][20347] Updated weights for policy 0, policy_version 1893310 (0.0006) [2023-12-27 18:45:51,528][20381] Updated weights for policy 1, policy_version 1891678 (0.0007) [2023-12-27 18:45:52,176][20347] Updated weights for policy 0, policy_version 1893320 (0.0007) [2023-12-27 18:45:52,220][20381] Updated weights for policy 1, policy_version 1891688 (0.0006) [2023-12-27 18:45:52,226][20347] Updated weights for policy 0, policy_version 1893330 (0.0005) [2023-12-27 18:45:52,283][20381] Updated weights for policy 1, policy_version 1891698 (0.0006) [2023-12-27 18:45:52,289][20347] Updated weights for policy 0, policy_version 1893340 (0.0007) [2023-12-27 18:45:52,342][20381] Updated weights for policy 1, policy_version 1891708 (0.0008) [2023-12-27 18:45:52,362][19170] Fps is (10 sec: 19667.4, 60 sec: 20207.0, 300 sec: 20410.6). Total num frames: 969113600. Throughput: 0: 10089.7, 1: 10093.8. Samples: 969086708. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:52,363][19170] Avg episode reward: [(0, '990.240'), (1, '855.380')] [2023-12-27 18:45:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001893344_484786176.pth... [2023-12-27 18:45:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001891712_484335616.pth... [2023-12-27 18:45:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001892144_484474880.pth [2023-12-27 18:45:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001890528_484032512.pth [2023-12-27 18:45:52,961][20381] Updated weights for policy 1, policy_version 1891718 (0.0007) [2023-12-27 18:45:53,016][20381] Updated weights for policy 1, policy_version 1891728 (0.0006) [2023-12-27 18:45:53,055][20347] Updated weights for policy 0, policy_version 1893350 (0.0006) [2023-12-27 18:45:53,075][20381] Updated weights for policy 1, policy_version 1891738 (0.0006) [2023-12-27 18:45:53,104][20347] Updated weights for policy 0, policy_version 1893360 (0.0006) [2023-12-27 18:45:53,158][20347] Updated weights for policy 0, policy_version 1893370 (0.0005) [2023-12-27 18:45:53,735][20381] Updated weights for policy 1, policy_version 1891748 (0.0006) [2023-12-27 18:45:53,793][20381] Updated weights for policy 1, policy_version 1891758 (0.0009) [2023-12-27 18:45:53,798][20347] Updated weights for policy 0, policy_version 1893380 (0.0005) [2023-12-27 18:45:53,847][20381] Updated weights for policy 1, policy_version 1891768 (0.0006) [2023-12-27 18:45:53,854][20347] Updated weights for policy 0, policy_version 1893390 (0.0005) [2023-12-27 18:45:53,912][20347] Updated weights for policy 0, policy_version 1893400 (0.0006) [2023-12-27 18:45:54,500][20381] Updated weights for policy 1, policy_version 1891778 (0.0006) [2023-12-27 18:45:54,534][20347] Updated weights for policy 0, policy_version 1893410 (0.0008) [2023-12-27 18:45:54,553][20381] Updated weights for policy 1, policy_version 1891788 (0.0006) [2023-12-27 18:45:54,583][20347] Updated weights for policy 0, policy_version 1893420 (0.0010) [2023-12-27 18:45:54,598][20381] Updated weights for policy 1, policy_version 1891798 (0.0005) [2023-12-27 18:45:54,633][20347] Updated weights for policy 0, policy_version 1893430 (0.0010) [2023-12-27 18:45:54,642][20381] Updated weights for policy 1, policy_version 1891808 (0.0006) [2023-12-27 18:45:54,688][20347] Updated weights for policy 0, policy_version 1893440 (0.0010) [2023-12-27 18:45:55,311][20381] Updated weights for policy 1, policy_version 1891818 (0.0007) [2023-12-27 18:45:55,366][20381] Updated weights for policy 1, policy_version 1891828 (0.0007) [2023-12-27 18:45:55,390][20347] Updated weights for policy 0, policy_version 1893450 (0.0010) [2023-12-27 18:45:55,423][20381] Updated weights for policy 1, policy_version 1891838 (0.0006) [2023-12-27 18:45:55,443][20347] Updated weights for policy 0, policy_version 1893460 (0.0007) [2023-12-27 18:45:55,494][20347] Updated weights for policy 0, policy_version 1893470 (0.0005) [2023-12-27 18:45:56,109][20381] Updated weights for policy 1, policy_version 1891848 (0.0008) [2023-12-27 18:45:56,162][20381] Updated weights for policy 1, policy_version 1891858 (0.0006) [2023-12-27 18:45:56,168][20347] Updated weights for policy 0, policy_version 1893480 (0.0008) [2023-12-27 18:45:56,220][20347] Updated weights for policy 0, policy_version 1893490 (0.0008) [2023-12-27 18:45:56,220][20381] Updated weights for policy 1, policy_version 1891868 (0.0006) [2023-12-27 18:45:56,278][20347] Updated weights for policy 0, policy_version 1893500 (0.0005) [2023-12-27 18:45:56,804][20347] Updated weights for policy 0, policy_version 1893510 (0.0008) [2023-12-27 18:45:56,855][20347] Updated weights for policy 0, policy_version 1893520 (0.0010) [2023-12-27 18:45:56,900][20347] Updated weights for policy 0, policy_version 1893530 (0.0010) [2023-12-27 18:45:56,945][20381] Updated weights for policy 1, policy_version 1891878 (0.0007) [2023-12-27 18:45:57,008][20381] Updated weights for policy 1, policy_version 1891888 (0.0008) [2023-12-27 18:45:57,057][20381] Updated weights for policy 1, policy_version 1891898 (0.0007) [2023-12-27 18:45:57,363][19170] Fps is (10 sec: 19660.5, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 969220096. Throughput: 0: 10088.7, 1: 10074.9. Samples: 969207812. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:45:57,364][19170] Avg episode reward: [(0, '980.510'), (1, '850.760')] [2023-12-27 18:45:57,594][20347] Updated weights for policy 0, policy_version 1893540 (0.0008) [2023-12-27 18:45:57,645][20347] Updated weights for policy 0, policy_version 1893550 (0.0005) [2023-12-27 18:45:57,698][20347] Updated weights for policy 0, policy_version 1893560 (0.0006) [2023-12-27 18:45:57,780][20381] Updated weights for policy 1, policy_version 1891908 (0.0008) [2023-12-27 18:45:57,840][20381] Updated weights for policy 1, policy_version 1891918 (0.0007) [2023-12-27 18:45:57,901][20381] Updated weights for policy 1, policy_version 1891928 (0.0008) [2023-12-27 18:45:58,338][20347] Updated weights for policy 0, policy_version 1893570 (0.0009) [2023-12-27 18:45:58,410][20347] Updated weights for policy 0, policy_version 1893580 (0.0006) [2023-12-27 18:45:58,478][20347] Updated weights for policy 0, policy_version 1893590 (0.0008) [2023-12-27 18:45:58,545][20347] Updated weights for policy 0, policy_version 1893600 (0.0008) [2023-12-27 18:45:58,632][20381] Updated weights for policy 1, policy_version 1891938 (0.0007) [2023-12-27 18:45:58,693][20381] Updated weights for policy 1, policy_version 1891948 (0.0008) [2023-12-27 18:45:58,757][20381] Updated weights for policy 1, policy_version 1891958 (0.0007) [2023-12-27 18:45:58,820][20381] Updated weights for policy 1, policy_version 1891968 (0.0007) [2023-12-27 18:45:59,261][20347] Updated weights for policy 0, policy_version 1893610 (0.0010) [2023-12-27 18:45:59,312][20347] Updated weights for policy 0, policy_version 1893620 (0.0008) [2023-12-27 18:45:59,380][20347] Updated weights for policy 0, policy_version 1893630 (0.0009) [2023-12-27 18:45:59,584][20381] Updated weights for policy 1, policy_version 1891978 (0.0008) [2023-12-27 18:45:59,635][20381] Updated weights for policy 1, policy_version 1891988 (0.0007) [2023-12-27 18:45:59,691][20381] Updated weights for policy 1, policy_version 1891998 (0.0008) [2023-12-27 18:46:00,108][20347] Updated weights for policy 0, policy_version 1893640 (0.0011) [2023-12-27 18:46:00,169][20347] Updated weights for policy 0, policy_version 1893650 (0.0009) [2023-12-27 18:46:00,235][20347] Updated weights for policy 0, policy_version 1893660 (0.0007) [2023-12-27 18:46:00,342][20381] Updated weights for policy 1, policy_version 1892008 (0.0006) [2023-12-27 18:46:00,399][20381] Updated weights for policy 1, policy_version 1892018 (0.0005) [2023-12-27 18:46:00,448][20381] Updated weights for policy 1, policy_version 1892028 (0.0005) [2023-12-27 18:46:00,957][20347] Updated weights for policy 0, policy_version 1893670 (0.0009) [2023-12-27 18:46:00,985][20381] Updated weights for policy 1, policy_version 1892038 (0.0005) [2023-12-27 18:46:01,006][20347] Updated weights for policy 0, policy_version 1893680 (0.0005) [2023-12-27 18:46:01,041][20381] Updated weights for policy 1, policy_version 1892048 (0.0007) [2023-12-27 18:46:01,073][20347] Updated weights for policy 0, policy_version 1893690 (0.0007) [2023-12-27 18:46:01,098][20381] Updated weights for policy 1, policy_version 1892058 (0.0007) [2023-12-27 18:46:01,723][20347] Updated weights for policy 0, policy_version 1893700 (0.0008) [2023-12-27 18:46:01,786][20347] Updated weights for policy 0, policy_version 1893710 (0.0009) [2023-12-27 18:46:01,825][20381] Updated weights for policy 1, policy_version 1892068 (0.0009) [2023-12-27 18:46:01,842][20347] Updated weights for policy 0, policy_version 1893720 (0.0009) [2023-12-27 18:46:01,877][20381] Updated weights for policy 1, policy_version 1892078 (0.0011) [2023-12-27 18:46:01,932][20381] Updated weights for policy 1, policy_version 1892088 (0.0011) [2023-12-27 18:46:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20438.4). Total num frames: 969318400. Throughput: 0: 10137.0, 1: 10051.7. Samples: 969327784. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:02,363][19170] Avg episode reward: [(0, '990.270'), (1, '856.500')] [2023-12-27 18:46:02,583][20347] Updated weights for policy 0, policy_version 1893730 (0.0008) [2023-12-27 18:46:02,589][20381] Updated weights for policy 1, policy_version 1892098 (0.0010) [2023-12-27 18:46:02,644][20381] Updated weights for policy 1, policy_version 1892108 (0.0007) [2023-12-27 18:46:02,645][20347] Updated weights for policy 0, policy_version 1893740 (0.0006) [2023-12-27 18:46:02,697][20381] Updated weights for policy 1, policy_version 1892118 (0.0007) [2023-12-27 18:46:02,709][20347] Updated weights for policy 0, policy_version 1893750 (0.0007) [2023-12-27 18:46:02,754][20381] Updated weights for policy 1, policy_version 1892128 (0.0011) [2023-12-27 18:46:02,765][20347] Updated weights for policy 0, policy_version 1893760 (0.0008) [2023-12-27 18:46:03,370][20381] Updated weights for policy 1, policy_version 1892138 (0.0005) [2023-12-27 18:46:03,419][20347] Updated weights for policy 0, policy_version 1893770 (0.0006) [2023-12-27 18:46:03,425][20381] Updated weights for policy 1, policy_version 1892148 (0.0005) [2023-12-27 18:46:03,476][20347] Updated weights for policy 0, policy_version 1893780 (0.0005) [2023-12-27 18:46:03,479][20381] Updated weights for policy 1, policy_version 1892158 (0.0005) [2023-12-27 18:46:03,529][20347] Updated weights for policy 0, policy_version 1893790 (0.0005) [2023-12-27 18:46:04,106][20381] Updated weights for policy 1, policy_version 1892168 (0.0010) [2023-12-27 18:46:04,166][20381] Updated weights for policy 1, policy_version 1892178 (0.0011) [2023-12-27 18:46:04,201][20347] Updated weights for policy 0, policy_version 1893800 (0.0007) [2023-12-27 18:46:04,226][20381] Updated weights for policy 1, policy_version 1892188 (0.0011) [2023-12-27 18:46:04,259][20347] Updated weights for policy 0, policy_version 1893810 (0.0006) [2023-12-27 18:46:04,320][20347] Updated weights for policy 0, policy_version 1893820 (0.0008) [2023-12-27 18:46:04,919][20347] Updated weights for policy 0, policy_version 1893830 (0.0007) [2023-12-27 18:46:04,921][20381] Updated weights for policy 1, policy_version 1892198 (0.0009) [2023-12-27 18:46:04,970][20347] Updated weights for policy 0, policy_version 1893840 (0.0005) [2023-12-27 18:46:04,975][20381] Updated weights for policy 1, policy_version 1892208 (0.0007) [2023-12-27 18:46:05,019][20347] Updated weights for policy 0, policy_version 1893850 (0.0007) [2023-12-27 18:46:05,023][20381] Updated weights for policy 1, policy_version 1892218 (0.0007) [2023-12-27 18:46:05,664][20381] Updated weights for policy 1, policy_version 1892228 (0.0006) [2023-12-27 18:46:05,669][20347] Updated weights for policy 0, policy_version 1893860 (0.0007) [2023-12-27 18:46:05,722][20347] Updated weights for policy 0, policy_version 1893870 (0.0005) [2023-12-27 18:46:05,723][20381] Updated weights for policy 1, policy_version 1892238 (0.0005) [2023-12-27 18:46:05,773][20347] Updated weights for policy 0, policy_version 1893880 (0.0005) [2023-12-27 18:46:05,776][20381] Updated weights for policy 1, policy_version 1892248 (0.0005) [2023-12-27 18:46:06,391][20347] Updated weights for policy 0, policy_version 1893890 (0.0006) [2023-12-27 18:46:06,454][20347] Updated weights for policy 0, policy_version 1893900 (0.0008) [2023-12-27 18:46:06,458][20381] Updated weights for policy 1, policy_version 1892258 (0.0006) [2023-12-27 18:46:06,513][20347] Updated weights for policy 0, policy_version 1893910 (0.0006) [2023-12-27 18:46:06,515][20381] Updated weights for policy 1, policy_version 1892268 (0.0011) [2023-12-27 18:46:06,572][20347] Updated weights for policy 0, policy_version 1893920 (0.0005) [2023-12-27 18:46:06,574][20381] Updated weights for policy 1, policy_version 1892278 (0.0011) [2023-12-27 18:46:06,634][20381] Updated weights for policy 1, policy_version 1892288 (0.0011) [2023-12-27 18:46:07,262][20381] Updated weights for policy 1, policy_version 1892298 (0.0009) [2023-12-27 18:46:07,284][20347] Updated weights for policy 0, policy_version 1893930 (0.0008) [2023-12-27 18:46:07,316][20381] Updated weights for policy 1, policy_version 1892308 (0.0007) [2023-12-27 18:46:07,346][20347] Updated weights for policy 0, policy_version 1893940 (0.0006) [2023-12-27 18:46:07,363][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 969416704. Throughput: 0: 10168.7, 1: 10088.1. Samples: 969391036. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.510')] [2023-12-27 18:46:07,370][20381] Updated weights for policy 1, policy_version 1892318 (0.0007) [2023-12-27 18:46:07,380][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001892320_484491264.pth... [2023-12-27 18:46:07,383][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001891136_484188160.pth [2023-12-27 18:46:07,404][20347] Updated weights for policy 0, policy_version 1893950 (0.0006) [2023-12-27 18:46:07,415][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001893952_484941824.pth... [2023-12-27 18:46:07,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001892768_484638720.pth [2023-12-27 18:46:08,050][20381] Updated weights for policy 1, policy_version 1892328 (0.0008) [2023-12-27 18:46:08,092][20347] Updated weights for policy 0, policy_version 1893960 (0.0008) [2023-12-27 18:46:08,099][20381] Updated weights for policy 1, policy_version 1892338 (0.0008) [2023-12-27 18:46:08,144][20347] Updated weights for policy 0, policy_version 1893970 (0.0009) [2023-12-27 18:46:08,151][20381] Updated weights for policy 1, policy_version 1892348 (0.0010) [2023-12-27 18:46:08,198][20347] Updated weights for policy 0, policy_version 1893980 (0.0006) [2023-12-27 18:46:08,832][20381] Updated weights for policy 1, policy_version 1892358 (0.0009) [2023-12-27 18:46:08,891][20381] Updated weights for policy 1, policy_version 1892368 (0.0006) [2023-12-27 18:46:08,943][20381] Updated weights for policy 1, policy_version 1892378 (0.0010) [2023-12-27 18:46:08,944][20347] Updated weights for policy 0, policy_version 1893990 (0.0006) [2023-12-27 18:46:08,998][20347] Updated weights for policy 0, policy_version 1894000 (0.0007) [2023-12-27 18:46:09,058][20347] Updated weights for policy 0, policy_version 1894010 (0.0008) [2023-12-27 18:46:09,578][20381] Updated weights for policy 1, policy_version 1892388 (0.0008) [2023-12-27 18:46:09,639][20381] Updated weights for policy 1, policy_version 1892398 (0.0008) [2023-12-27 18:46:09,697][20381] Updated weights for policy 1, policy_version 1892408 (0.0007) [2023-12-27 18:46:09,857][20347] Updated weights for policy 0, policy_version 1894020 (0.0007) [2023-12-27 18:46:09,914][20347] Updated weights for policy 0, policy_version 1894030 (0.0008) [2023-12-27 18:46:09,968][20347] Updated weights for policy 0, policy_version 1894040 (0.0008) [2023-12-27 18:46:10,412][20381] Updated weights for policy 1, policy_version 1892418 (0.0008) [2023-12-27 18:46:10,466][20381] Updated weights for policy 1, policy_version 1892428 (0.0010) [2023-12-27 18:46:10,527][20381] Updated weights for policy 1, policy_version 1892438 (0.0011) [2023-12-27 18:46:10,579][20381] Updated weights for policy 1, policy_version 1892448 (0.0010) [2023-12-27 18:46:10,695][20347] Updated weights for policy 0, policy_version 1894050 (0.0007) [2023-12-27 18:46:10,755][20347] Updated weights for policy 0, policy_version 1894060 (0.0008) [2023-12-27 18:46:10,824][20347] Updated weights for policy 0, policy_version 1894070 (0.0008) [2023-12-27 18:46:10,884][20347] Updated weights for policy 0, policy_version 1894080 (0.0007) [2023-12-27 18:46:11,321][20381] Updated weights for policy 1, policy_version 1892458 (0.0011) [2023-12-27 18:46:11,387][20381] Updated weights for policy 1, policy_version 1892468 (0.0011) [2023-12-27 18:46:11,443][20381] Updated weights for policy 1, policy_version 1892478 (0.0008) [2023-12-27 18:46:11,618][20347] Updated weights for policy 0, policy_version 1894090 (0.0008) [2023-12-27 18:46:11,683][20347] Updated weights for policy 0, policy_version 1894100 (0.0008) [2023-12-27 18:46:11,742][20347] Updated weights for policy 0, policy_version 1894110 (0.0008) [2023-12-27 18:46:12,152][20381] Updated weights for policy 1, policy_version 1892488 (0.0008) [2023-12-27 18:46:12,204][20381] Updated weights for policy 1, policy_version 1892498 (0.0006) [2023-12-27 18:46:12,265][20381] Updated weights for policy 1, policy_version 1892508 (0.0007) [2023-12-27 18:46:12,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 969523200. Throughput: 0: 10141.5, 1: 10052.7. Samples: 969512716. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '884.870')] [2023-12-27 18:46:12,490][20347] Updated weights for policy 0, policy_version 1894120 (0.0006) [2023-12-27 18:46:12,550][20347] Updated weights for policy 0, policy_version 1894130 (0.0007) [2023-12-27 18:46:12,606][20347] Updated weights for policy 0, policy_version 1894140 (0.0008) [2023-12-27 18:46:12,955][20381] Updated weights for policy 1, policy_version 1892518 (0.0008) [2023-12-27 18:46:13,015][20381] Updated weights for policy 1, policy_version 1892528 (0.0008) [2023-12-27 18:46:13,069][20381] Updated weights for policy 1, policy_version 1892538 (0.0006) [2023-12-27 18:46:13,263][20347] Updated weights for policy 0, policy_version 1894150 (0.0008) [2023-12-27 18:46:13,320][20347] Updated weights for policy 0, policy_version 1894160 (0.0005) [2023-12-27 18:46:13,382][20347] Updated weights for policy 0, policy_version 1894170 (0.0007) [2023-12-27 18:46:13,671][20381] Updated weights for policy 1, policy_version 1892548 (0.0007) [2023-12-27 18:46:13,737][20381] Updated weights for policy 1, policy_version 1892558 (0.0005) [2023-12-27 18:46:13,803][20381] Updated weights for policy 1, policy_version 1892568 (0.0006) [2023-12-27 18:46:13,995][20347] Updated weights for policy 0, policy_version 1894180 (0.0007) [2023-12-27 18:46:14,055][20347] Updated weights for policy 0, policy_version 1894190 (0.0008) [2023-12-27 18:46:14,108][20347] Updated weights for policy 0, policy_version 1894200 (0.0007) [2023-12-27 18:46:14,383][20381] Updated weights for policy 1, policy_version 1892578 (0.0008) [2023-12-27 18:46:14,440][20381] Updated weights for policy 1, policy_version 1892588 (0.0008) [2023-12-27 18:46:14,500][20381] Updated weights for policy 1, policy_version 1892598 (0.0005) [2023-12-27 18:46:14,552][20381] Updated weights for policy 1, policy_version 1892608 (0.0007) [2023-12-27 18:46:14,740][20347] Updated weights for policy 0, policy_version 1894210 (0.0006) [2023-12-27 18:46:14,808][20347] Updated weights for policy 0, policy_version 1894220 (0.0007) [2023-12-27 18:46:14,864][20347] Updated weights for policy 0, policy_version 1894230 (0.0007) [2023-12-27 18:46:14,919][20347] Updated weights for policy 0, policy_version 1894240 (0.0007) [2023-12-27 18:46:15,269][20381] Updated weights for policy 1, policy_version 1892618 (0.0005) [2023-12-27 18:46:15,328][20381] Updated weights for policy 1, policy_version 1892628 (0.0007) [2023-12-27 18:46:15,385][20381] Updated weights for policy 1, policy_version 1892638 (0.0009) [2023-12-27 18:46:15,647][20347] Updated weights for policy 0, policy_version 1894250 (0.0007) [2023-12-27 18:46:15,697][20347] Updated weights for policy 0, policy_version 1894260 (0.0007) [2023-12-27 18:46:15,758][20347] Updated weights for policy 0, policy_version 1894270 (0.0008) [2023-12-27 18:46:16,054][20381] Updated weights for policy 1, policy_version 1892648 (0.0008) [2023-12-27 18:46:16,107][20381] Updated weights for policy 1, policy_version 1892658 (0.0007) [2023-12-27 18:46:16,173][20381] Updated weights for policy 1, policy_version 1892668 (0.0005) [2023-12-27 18:46:16,427][20347] Updated weights for policy 0, policy_version 1894280 (0.0008) [2023-12-27 18:46:16,485][20347] Updated weights for policy 0, policy_version 1894290 (0.0009) [2023-12-27 18:46:16,540][20347] Updated weights for policy 0, policy_version 1894300 (0.0010) [2023-12-27 18:46:16,832][20381] Updated weights for policy 1, policy_version 1892678 (0.0009) [2023-12-27 18:46:16,883][20381] Updated weights for policy 1, policy_version 1892688 (0.0010) [2023-12-27 18:46:16,938][20381] Updated weights for policy 1, policy_version 1892698 (0.0010) [2023-12-27 18:46:17,222][20347] Updated weights for policy 0, policy_version 1894310 (0.0008) [2023-12-27 18:46:17,285][20347] Updated weights for policy 0, policy_version 1894320 (0.0005) [2023-12-27 18:46:17,342][20347] Updated weights for policy 0, policy_version 1894330 (0.0005) [2023-12-27 18:46:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20207.1, 300 sec: 20438.4). Total num frames: 969621504. Throughput: 0: 10075.4, 1: 10158.2. Samples: 969635848. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '939.710')] [2023-12-27 18:46:17,547][20381] Updated weights for policy 1, policy_version 1892708 (0.0008) [2023-12-27 18:46:17,602][20381] Updated weights for policy 1, policy_version 1892718 (0.0010) [2023-12-27 18:46:17,659][20381] Updated weights for policy 1, policy_version 1892728 (0.0010) [2023-12-27 18:46:17,976][20347] Updated weights for policy 0, policy_version 1894340 (0.0007) [2023-12-27 18:46:18,033][20347] Updated weights for policy 0, policy_version 1894350 (0.0010) [2023-12-27 18:46:18,083][20347] Updated weights for policy 0, policy_version 1894361 (0.0007) [2023-12-27 18:46:18,253][20381] Updated weights for policy 1, policy_version 1892738 (0.0010) [2023-12-27 18:46:18,305][20381] Updated weights for policy 1, policy_version 1892748 (0.0008) [2023-12-27 18:46:18,381][20381] Updated weights for policy 1, policy_version 1892758 (0.0007) [2023-12-27 18:46:18,439][20381] Updated weights for policy 1, policy_version 1892768 (0.0005) [2023-12-27 18:46:18,756][20347] Updated weights for policy 0, policy_version 1894371 (0.0006) [2023-12-27 18:46:18,821][20347] Updated weights for policy 0, policy_version 1894381 (0.0008) [2023-12-27 18:46:18,874][20347] Updated weights for policy 0, policy_version 1894391 (0.0006) [2023-12-27 18:46:19,068][20381] Updated weights for policy 1, policy_version 1892778 (0.0010) [2023-12-27 18:46:19,117][20381] Updated weights for policy 1, policy_version 1892788 (0.0010) [2023-12-27 18:46:19,173][20381] Updated weights for policy 1, policy_version 1892798 (0.0011) [2023-12-27 18:46:19,596][20347] Updated weights for policy 0, policy_version 1894401 (0.0008) [2023-12-27 18:46:19,657][20347] Updated weights for policy 0, policy_version 1894411 (0.0009) [2023-12-27 18:46:19,711][20347] Updated weights for policy 0, policy_version 1894421 (0.0009) [2023-12-27 18:46:19,766][20347] Updated weights for policy 0, policy_version 1894431 (0.0008) [2023-12-27 18:46:19,873][20381] Updated weights for policy 1, policy_version 1892808 (0.0009) [2023-12-27 18:46:19,942][20381] Updated weights for policy 1, policy_version 1892818 (0.0008) [2023-12-27 18:46:19,999][20381] Updated weights for policy 1, policy_version 1892828 (0.0011) [2023-12-27 18:46:20,546][20347] Updated weights for policy 0, policy_version 1894441 (0.0007) [2023-12-27 18:46:20,608][20347] Updated weights for policy 0, policy_version 1894451 (0.0007) [2023-12-27 18:46:20,666][20347] Updated weights for policy 0, policy_version 1894461 (0.0008) [2023-12-27 18:46:20,770][20381] Updated weights for policy 1, policy_version 1892838 (0.0011) [2023-12-27 18:46:20,821][20381] Updated weights for policy 1, policy_version 1892848 (0.0009) [2023-12-27 18:46:20,880][20381] Updated weights for policy 1, policy_version 1892858 (0.0011) [2023-12-27 18:46:21,402][20347] Updated weights for policy 0, policy_version 1894471 (0.0008) [2023-12-27 18:46:21,462][20347] Updated weights for policy 0, policy_version 1894481 (0.0006) [2023-12-27 18:46:21,519][20347] Updated weights for policy 0, policy_version 1894491 (0.0005) [2023-12-27 18:46:21,623][20381] Updated weights for policy 1, policy_version 1892868 (0.0010) [2023-12-27 18:46:21,690][20381] Updated weights for policy 1, policy_version 1892878 (0.0012) [2023-12-27 18:46:21,755][20381] Updated weights for policy 1, policy_version 1892888 (0.0007) [2023-12-27 18:46:22,154][20347] Updated weights for policy 0, policy_version 1894501 (0.0006) [2023-12-27 18:46:22,206][20347] Updated weights for policy 0, policy_version 1894511 (0.0008) [2023-12-27 18:46:22,253][20347] Updated weights for policy 0, policy_version 1894521 (0.0006) [2023-12-27 18:46:22,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.4, 300 sec: 20438.4). Total num frames: 969728000. Throughput: 0: 10031.2, 1: 10156.2. Samples: 969697412. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '899.660')] [2023-12-27 18:46:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001892896_484638720.pth... [2023-12-27 18:46:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001894528_485089280.pth... [2023-12-27 18:46:22,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001893344_484786176.pth [2023-12-27 18:46:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001891712_484335616.pth [2023-12-27 18:46:22,454][20381] Updated weights for policy 1, policy_version 1892898 (0.0008) [2023-12-27 18:46:22,517][20381] Updated weights for policy 1, policy_version 1892908 (0.0007) [2023-12-27 18:46:22,578][20381] Updated weights for policy 1, policy_version 1892918 (0.0007) [2023-12-27 18:46:22,639][20381] Updated weights for policy 1, policy_version 1892928 (0.0007) [2023-12-27 18:46:22,978][20347] Updated weights for policy 0, policy_version 1894531 (0.0009) [2023-12-27 18:46:23,035][20347] Updated weights for policy 0, policy_version 1894541 (0.0010) [2023-12-27 18:46:23,092][20347] Updated weights for policy 0, policy_version 1894551 (0.0010) [2023-12-27 18:46:23,278][20381] Updated weights for policy 1, policy_version 1892938 (0.0006) [2023-12-27 18:46:23,329][20381] Updated weights for policy 1, policy_version 1892948 (0.0007) [2023-12-27 18:46:23,373][20381] Updated weights for policy 1, policy_version 1892958 (0.0007) [2023-12-27 18:46:23,703][20347] Updated weights for policy 0, policy_version 1894561 (0.0006) [2023-12-27 18:46:23,757][20347] Updated weights for policy 0, policy_version 1894571 (0.0010) [2023-12-27 18:46:23,808][20347] Updated weights for policy 0, policy_version 1894581 (0.0009) [2023-12-27 18:46:23,855][20347] Updated weights for policy 0, policy_version 1894591 (0.0010) [2023-12-27 18:46:24,063][20381] Updated weights for policy 1, policy_version 1892968 (0.0006) [2023-12-27 18:46:24,125][20381] Updated weights for policy 1, policy_version 1892978 (0.0008) [2023-12-27 18:46:24,176][20381] Updated weights for policy 1, policy_version 1892988 (0.0010) [2023-12-27 18:46:24,534][20347] Updated weights for policy 0, policy_version 1894601 (0.0006) [2023-12-27 18:46:24,593][20347] Updated weights for policy 0, policy_version 1894611 (0.0005) [2023-12-27 18:46:24,643][20347] Updated weights for policy 0, policy_version 1894621 (0.0005) [2023-12-27 18:46:24,721][20381] Updated weights for policy 1, policy_version 1892998 (0.0006) [2023-12-27 18:46:24,788][20381] Updated weights for policy 1, policy_version 1893008 (0.0011) [2023-12-27 18:46:24,846][20381] Updated weights for policy 1, policy_version 1893018 (0.0010) [2023-12-27 18:46:25,235][20347] Updated weights for policy 0, policy_version 1894631 (0.0007) [2023-12-27 18:46:25,288][20347] Updated weights for policy 0, policy_version 1894641 (0.0007) [2023-12-27 18:46:25,340][20347] Updated weights for policy 0, policy_version 1894651 (0.0007) [2023-12-27 18:46:25,474][20381] Updated weights for policy 1, policy_version 1893028 (0.0011) [2023-12-27 18:46:25,532][20381] Updated weights for policy 1, policy_version 1893038 (0.0011) [2023-12-27 18:46:25,589][20381] Updated weights for policy 1, policy_version 1893048 (0.0010) [2023-12-27 18:46:26,018][20347] Updated weights for policy 0, policy_version 1894661 (0.0006) [2023-12-27 18:46:26,077][20347] Updated weights for policy 0, policy_version 1894671 (0.0008) [2023-12-27 18:46:26,133][20347] Updated weights for policy 0, policy_version 1894681 (0.0007) [2023-12-27 18:46:26,191][20381] Updated weights for policy 1, policy_version 1893058 (0.0007) [2023-12-27 18:46:26,249][20381] Updated weights for policy 1, policy_version 1893068 (0.0010) [2023-12-27 18:46:26,300][20381] Updated weights for policy 1, policy_version 1893078 (0.0010) [2023-12-27 18:46:26,354][20381] Updated weights for policy 1, policy_version 1893088 (0.0010) [2023-12-27 18:46:26,757][20347] Updated weights for policy 0, policy_version 1894691 (0.0006) [2023-12-27 18:46:26,804][20347] Updated weights for policy 0, policy_version 1894701 (0.0007) [2023-12-27 18:46:26,852][20347] Updated weights for policy 0, policy_version 1894711 (0.0010) [2023-12-27 18:46:26,927][20381] Updated weights for policy 1, policy_version 1893098 (0.0006) [2023-12-27 18:46:26,976][20381] Updated weights for policy 1, policy_version 1893108 (0.0005) [2023-12-27 18:46:27,025][20381] Updated weights for policy 1, policy_version 1893118 (0.0008) [2023-12-27 18:46:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 969834496. Throughput: 0: 10038.2, 1: 10212.2. Samples: 969821464. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:27,363][19170] Avg episode reward: [(0, '990.240'), (1, '887.360')] [2023-12-27 18:46:27,475][20347] Updated weights for policy 0, policy_version 1894721 (0.0010) [2023-12-27 18:46:27,520][20347] Updated weights for policy 0, policy_version 1894731 (0.0010) [2023-12-27 18:46:27,567][20347] Updated weights for policy 0, policy_version 1894741 (0.0010) [2023-12-27 18:46:27,611][20381] Updated weights for policy 1, policy_version 1893128 (0.0007) [2023-12-27 18:46:27,619][20347] Updated weights for policy 0, policy_version 1894751 (0.0010) [2023-12-27 18:46:27,666][20381] Updated weights for policy 1, policy_version 1893138 (0.0005) [2023-12-27 18:46:27,723][20381] Updated weights for policy 1, policy_version 1893148 (0.0006) [2023-12-27 18:46:28,284][20381] Updated weights for policy 1, policy_version 1893158 (0.0008) [2023-12-27 18:46:28,309][20347] Updated weights for policy 0, policy_version 1894761 (0.0007) [2023-12-27 18:46:28,340][20381] Updated weights for policy 1, policy_version 1893168 (0.0009) [2023-12-27 18:46:28,370][20347] Updated weights for policy 0, policy_version 1894771 (0.0006) [2023-12-27 18:46:28,394][20381] Updated weights for policy 1, policy_version 1893178 (0.0008) [2023-12-27 18:46:28,429][20347] Updated weights for policy 0, policy_version 1894781 (0.0006) [2023-12-27 18:46:29,093][20347] Updated weights for policy 0, policy_version 1894791 (0.0009) [2023-12-27 18:46:29,098][20381] Updated weights for policy 1, policy_version 1893188 (0.0006) [2023-12-27 18:46:29,135][20347] Updated weights for policy 0, policy_version 1894801 (0.0007) [2023-12-27 18:46:29,148][20381] Updated weights for policy 1, policy_version 1893198 (0.0005) [2023-12-27 18:46:29,180][20347] Updated weights for policy 0, policy_version 1894811 (0.0008) [2023-12-27 18:46:29,198][20381] Updated weights for policy 1, policy_version 1893208 (0.0005) [2023-12-27 18:46:29,839][20381] Updated weights for policy 1, policy_version 1893218 (0.0007) [2023-12-27 18:46:29,897][20347] Updated weights for policy 0, policy_version 1894821 (0.0010) [2023-12-27 18:46:29,900][20381] Updated weights for policy 1, policy_version 1893228 (0.0010) [2023-12-27 18:46:29,961][20381] Updated weights for policy 1, policy_version 1893238 (0.0009) [2023-12-27 18:46:29,963][20347] Updated weights for policy 0, policy_version 1894831 (0.0010) [2023-12-27 18:46:30,023][20381] Updated weights for policy 1, policy_version 1893248 (0.0007) [2023-12-27 18:46:30,026][20347] Updated weights for policy 0, policy_version 1894841 (0.0011) [2023-12-27 18:46:30,677][20381] Updated weights for policy 1, policy_version 1893258 (0.0007) [2023-12-27 18:46:30,682][20347] Updated weights for policy 0, policy_version 1894851 (0.0008) [2023-12-27 18:46:30,730][20381] Updated weights for policy 1, policy_version 1893268 (0.0006) [2023-12-27 18:46:30,738][20347] Updated weights for policy 0, policy_version 1894861 (0.0006) [2023-12-27 18:46:30,785][20381] Updated weights for policy 1, policy_version 1893278 (0.0010) [2023-12-27 18:46:30,790][20347] Updated weights for policy 0, policy_version 1894871 (0.0005) [2023-12-27 18:46:31,491][20347] Updated weights for policy 0, policy_version 1894881 (0.0007) [2023-12-27 18:46:31,509][20381] Updated weights for policy 1, policy_version 1893288 (0.0007) [2023-12-27 18:46:31,545][20347] Updated weights for policy 0, policy_version 1894891 (0.0006) [2023-12-27 18:46:31,568][20381] Updated weights for policy 1, policy_version 1893298 (0.0008) [2023-12-27 18:46:31,593][20347] Updated weights for policy 0, policy_version 1894901 (0.0009) [2023-12-27 18:46:31,629][20381] Updated weights for policy 1, policy_version 1893308 (0.0007) [2023-12-27 18:46:31,661][20347] Updated weights for policy 0, policy_version 1894911 (0.0006) [2023-12-27 18:46:32,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.4, 300 sec: 20410.6). Total num frames: 969932800. Throughput: 0: 10101.6, 1: 10332.1. Samples: 969947308. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:32,363][19170] Avg episode reward: [(0, '980.530'), (1, '895.580')] [2023-12-27 18:46:32,365][20347] Updated weights for policy 0, policy_version 1894921 (0.0007) [2023-12-27 18:46:32,375][20381] Updated weights for policy 1, policy_version 1893318 (0.0008) [2023-12-27 18:46:32,430][20347] Updated weights for policy 0, policy_version 1894931 (0.0007) [2023-12-27 18:46:32,432][20381] Updated weights for policy 1, policy_version 1893328 (0.0008) [2023-12-27 18:46:32,492][20347] Updated weights for policy 0, policy_version 1894941 (0.0007) [2023-12-27 18:46:32,495][20381] Updated weights for policy 1, policy_version 1893338 (0.0006) [2023-12-27 18:46:33,076][20381] Updated weights for policy 1, policy_version 1893348 (0.0006) [2023-12-27 18:46:33,138][20381] Updated weights for policy 1, policy_version 1893358 (0.0007) [2023-12-27 18:46:33,192][20381] Updated weights for policy 1, policy_version 1893368 (0.0005) [2023-12-27 18:46:33,192][20347] Updated weights for policy 0, policy_version 1894951 (0.0007) [2023-12-27 18:46:33,238][20347] Updated weights for policy 0, policy_version 1894961 (0.0006) [2023-12-27 18:46:33,292][20347] Updated weights for policy 0, policy_version 1894971 (0.0007) [2023-12-27 18:46:33,823][20381] Updated weights for policy 1, policy_version 1893378 (0.0007) [2023-12-27 18:46:33,874][20381] Updated weights for policy 1, policy_version 1893388 (0.0007) [2023-12-27 18:46:33,920][20381] Updated weights for policy 1, policy_version 1893398 (0.0008) [2023-12-27 18:46:33,938][20347] Updated weights for policy 0, policy_version 1894981 (0.0006) [2023-12-27 18:46:33,977][20381] Updated weights for policy 1, policy_version 1893408 (0.0007) [2023-12-27 18:46:33,995][20347] Updated weights for policy 0, policy_version 1894991 (0.0009) [2023-12-27 18:46:34,040][20347] Updated weights for policy 0, policy_version 1895001 (0.0008) [2023-12-27 18:46:34,663][20381] Updated weights for policy 1, policy_version 1893418 (0.0007) [2023-12-27 18:46:34,723][20381] Updated weights for policy 1, policy_version 1893428 (0.0007) [2023-12-27 18:46:34,750][20347] Updated weights for policy 0, policy_version 1895011 (0.0008) [2023-12-27 18:46:34,790][20381] Updated weights for policy 1, policy_version 1893438 (0.0006) [2023-12-27 18:46:34,810][20347] Updated weights for policy 0, policy_version 1895021 (0.0006) [2023-12-27 18:46:34,855][20347] Updated weights for policy 0, policy_version 1895031 (0.0007) [2023-12-27 18:46:35,435][20347] Updated weights for policy 0, policy_version 1895041 (0.0007) [2023-12-27 18:46:35,441][20381] Updated weights for policy 1, policy_version 1893448 (0.0008) [2023-12-27 18:46:35,484][20347] Updated weights for policy 0, policy_version 1895051 (0.0005) [2023-12-27 18:46:35,496][20381] Updated weights for policy 1, policy_version 1893458 (0.0007) [2023-12-27 18:46:35,539][20347] Updated weights for policy 0, policy_version 1895061 (0.0005) [2023-12-27 18:46:35,552][20381] Updated weights for policy 1, policy_version 1893468 (0.0006) [2023-12-27 18:46:35,591][20347] Updated weights for policy 0, policy_version 1895071 (0.0005) [2023-12-27 18:46:36,239][20347] Updated weights for policy 0, policy_version 1895081 (0.0006) [2023-12-27 18:46:36,256][20381] Updated weights for policy 1, policy_version 1893478 (0.0006) [2023-12-27 18:46:36,302][20347] Updated weights for policy 0, policy_version 1895091 (0.0010) [2023-12-27 18:46:36,313][20381] Updated weights for policy 1, policy_version 1893488 (0.0007) [2023-12-27 18:46:36,360][20347] Updated weights for policy 0, policy_version 1895101 (0.0007) [2023-12-27 18:46:36,373][20381] Updated weights for policy 1, policy_version 1893498 (0.0006) [2023-12-27 18:46:37,041][20347] Updated weights for policy 0, policy_version 1895111 (0.0006) [2023-12-27 18:46:37,097][20347] Updated weights for policy 0, policy_version 1895121 (0.0006) [2023-12-27 18:46:37,106][20381] Updated weights for policy 1, policy_version 1893508 (0.0008) [2023-12-27 18:46:37,162][20347] Updated weights for policy 0, policy_version 1895131 (0.0008) [2023-12-27 18:46:37,167][20381] Updated weights for policy 1, policy_version 1893518 (0.0009) [2023-12-27 18:46:37,217][20381] Updated weights for policy 1, policy_version 1893528 (0.0007) [2023-12-27 18:46:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20479.9, 300 sec: 20466.1). Total num frames: 970047488. Throughput: 0: 10133.5, 1: 10385.3. Samples: 970010056. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:37,364][19170] Avg episode reward: [(0, '990.290'), (1, '914.220')] [2023-12-27 18:46:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001893536_484802560.pth... [2023-12-27 18:46:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001895136_485244928.pth... [2023-12-27 18:46:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001893952_484941824.pth [2023-12-27 18:46:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001892320_484491264.pth [2023-12-27 18:46:37,843][20347] Updated weights for policy 0, policy_version 1895141 (0.0008) [2023-12-27 18:46:37,872][20381] Updated weights for policy 1, policy_version 1893538 (0.0007) [2023-12-27 18:46:37,897][20347] Updated weights for policy 0, policy_version 1895151 (0.0006) [2023-12-27 18:46:37,922][20381] Updated weights for policy 1, policy_version 1893548 (0.0006) [2023-12-27 18:46:37,951][20347] Updated weights for policy 0, policy_version 1895161 (0.0007) [2023-12-27 18:46:37,968][20381] Updated weights for policy 1, policy_version 1893558 (0.0007) [2023-12-27 18:46:38,023][20381] Updated weights for policy 1, policy_version 1893568 (0.0008) [2023-12-27 18:46:38,674][20347] Updated weights for policy 0, policy_version 1895171 (0.0008) [2023-12-27 18:46:38,698][20381] Updated weights for policy 1, policy_version 1893578 (0.0007) [2023-12-27 18:46:38,737][20347] Updated weights for policy 0, policy_version 1895181 (0.0007) [2023-12-27 18:46:38,755][20381] Updated weights for policy 1, policy_version 1893588 (0.0006) [2023-12-27 18:46:38,797][20347] Updated weights for policy 0, policy_version 1895191 (0.0008) [2023-12-27 18:46:38,815][20381] Updated weights for policy 1, policy_version 1893598 (0.0006) [2023-12-27 18:46:39,507][20347] Updated weights for policy 0, policy_version 1895201 (0.0007) [2023-12-27 18:46:39,539][20381] Updated weights for policy 1, policy_version 1893608 (0.0008) [2023-12-27 18:46:39,568][20347] Updated weights for policy 0, policy_version 1895211 (0.0007) [2023-12-27 18:46:39,598][20381] Updated weights for policy 1, policy_version 1893618 (0.0008) [2023-12-27 18:46:39,626][20347] Updated weights for policy 0, policy_version 1895221 (0.0007) [2023-12-27 18:46:39,655][20381] Updated weights for policy 1, policy_version 1893628 (0.0006) [2023-12-27 18:46:39,683][20347] Updated weights for policy 0, policy_version 1895231 (0.0007) [2023-12-27 18:46:40,353][20381] Updated weights for policy 1, policy_version 1893638 (0.0007) [2023-12-27 18:46:40,406][20381] Updated weights for policy 1, policy_version 1893648 (0.0006) [2023-12-27 18:46:40,443][20347] Updated weights for policy 0, policy_version 1895241 (0.0009) [2023-12-27 18:46:40,469][20381] Updated weights for policy 1, policy_version 1893658 (0.0010) [2023-12-27 18:46:40,502][20347] Updated weights for policy 0, policy_version 1895251 (0.0008) [2023-12-27 18:46:40,553][20347] Updated weights for policy 0, policy_version 1895261 (0.0007) [2023-12-27 18:46:41,044][20381] Updated weights for policy 1, policy_version 1893668 (0.0009) [2023-12-27 18:46:41,102][20381] Updated weights for policy 1, policy_version 1893678 (0.0006) [2023-12-27 18:46:41,170][20381] Updated weights for policy 1, policy_version 1893688 (0.0006) [2023-12-27 18:46:41,354][20347] Updated weights for policy 0, policy_version 1895271 (0.0007) [2023-12-27 18:46:41,421][20347] Updated weights for policy 0, policy_version 1895281 (0.0009) [2023-12-27 18:46:41,479][20347] Updated weights for policy 0, policy_version 1895291 (0.0008) [2023-12-27 18:46:41,853][20381] Updated weights for policy 1, policy_version 1893698 (0.0006) [2023-12-27 18:46:41,914][20381] Updated weights for policy 1, policy_version 1893708 (0.0008) [2023-12-27 18:46:41,973][20381] Updated weights for policy 1, policy_version 1893718 (0.0010) [2023-12-27 18:46:42,033][20381] Updated weights for policy 1, policy_version 1893728 (0.0010) [2023-12-27 18:46:42,197][20347] Updated weights for policy 0, policy_version 1895301 (0.0007) [2023-12-27 18:46:42,259][20347] Updated weights for policy 0, policy_version 1895311 (0.0008) [2023-12-27 18:46:42,324][20347] Updated weights for policy 0, policy_version 1895321 (0.0008) [2023-12-27 18:46:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20344.6, 300 sec: 20410.6). Total num frames: 970137600. Throughput: 0: 10143.1, 1: 10381.8. Samples: 970131432. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '915.600')] [2023-12-27 18:46:42,782][20381] Updated weights for policy 1, policy_version 1893738 (0.0010) [2023-12-27 18:46:42,837][20381] Updated weights for policy 1, policy_version 1893748 (0.0010) [2023-12-27 18:46:42,895][20381] Updated weights for policy 1, policy_version 1893758 (0.0010) [2023-12-27 18:46:43,006][20347] Updated weights for policy 0, policy_version 1895331 (0.0009) [2023-12-27 18:46:43,063][20347] Updated weights for policy 0, policy_version 1895341 (0.0008) [2023-12-27 18:46:43,109][20347] Updated weights for policy 0, policy_version 1895351 (0.0010) [2023-12-27 18:46:43,445][20381] Updated weights for policy 1, policy_version 1893768 (0.0006) [2023-12-27 18:46:43,500][20381] Updated weights for policy 1, policy_version 1893778 (0.0005) [2023-12-27 18:46:43,560][20381] Updated weights for policy 1, policy_version 1893788 (0.0008) [2023-12-27 18:46:43,707][20347] Updated weights for policy 0, policy_version 1895361 (0.0007) [2023-12-27 18:46:43,760][20347] Updated weights for policy 0, policy_version 1895371 (0.0008) [2023-12-27 18:46:43,809][20347] Updated weights for policy 0, policy_version 1895381 (0.0009) [2023-12-27 18:46:43,860][20347] Updated weights for policy 0, policy_version 1895391 (0.0007) [2023-12-27 18:46:44,063][20381] Updated weights for policy 1, policy_version 1893798 (0.0005) [2023-12-27 18:46:44,121][20381] Updated weights for policy 1, policy_version 1893808 (0.0010) [2023-12-27 18:46:44,176][20381] Updated weights for policy 1, policy_version 1893818 (0.0010) [2023-12-27 18:46:44,571][20347] Updated weights for policy 0, policy_version 1895401 (0.0005) [2023-12-27 18:46:44,627][20347] Updated weights for policy 0, policy_version 1895411 (0.0005) [2023-12-27 18:46:44,676][20347] Updated weights for policy 0, policy_version 1895421 (0.0005) [2023-12-27 18:46:44,818][20381] Updated weights for policy 1, policy_version 1893828 (0.0010) [2023-12-27 18:46:44,878][20381] Updated weights for policy 1, policy_version 1893838 (0.0011) [2023-12-27 18:46:44,936][20381] Updated weights for policy 1, policy_version 1893848 (0.0009) [2023-12-27 18:46:45,397][20347] Updated weights for policy 0, policy_version 1895431 (0.0008) [2023-12-27 18:46:45,453][20347] Updated weights for policy 0, policy_version 1895441 (0.0008) [2023-12-27 18:46:45,515][20347] Updated weights for policy 0, policy_version 1895451 (0.0007) [2023-12-27 18:46:45,590][20381] Updated weights for policy 1, policy_version 1893858 (0.0009) [2023-12-27 18:46:45,651][20381] Updated weights for policy 1, policy_version 1893868 (0.0006) [2023-12-27 18:46:45,712][20381] Updated weights for policy 1, policy_version 1893878 (0.0005) [2023-12-27 18:46:45,759][20381] Updated weights for policy 1, policy_version 1893888 (0.0006) [2023-12-27 18:46:46,259][20347] Updated weights for policy 0, policy_version 1895461 (0.0006) [2023-12-27 18:46:46,316][20381] Updated weights for policy 1, policy_version 1893898 (0.0005) [2023-12-27 18:46:46,317][20347] Updated weights for policy 0, policy_version 1895471 (0.0005) [2023-12-27 18:46:46,367][20381] Updated weights for policy 1, policy_version 1893908 (0.0006) [2023-12-27 18:46:46,376][20347] Updated weights for policy 0, policy_version 1895481 (0.0005) [2023-12-27 18:46:46,413][20381] Updated weights for policy 1, policy_version 1893918 (0.0010) [2023-12-27 18:46:46,979][20347] Updated weights for policy 0, policy_version 1895491 (0.0006) [2023-12-27 18:46:47,041][20347] Updated weights for policy 0, policy_version 1895501 (0.0007) [2023-12-27 18:46:47,083][20381] Updated weights for policy 1, policy_version 1893928 (0.0010) [2023-12-27 18:46:47,100][20347] Updated weights for policy 0, policy_version 1895511 (0.0006) [2023-12-27 18:46:47,138][20381] Updated weights for policy 1, policy_version 1893938 (0.0010) [2023-12-27 18:46:47,189][20381] Updated weights for policy 1, policy_version 1893948 (0.0010) [2023-12-27 18:46:47,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 970252288. Throughput: 0: 10150.7, 1: 10508.7. Samples: 970257456. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '917.800')] [2023-12-27 18:46:47,725][20347] Updated weights for policy 0, policy_version 1895521 (0.0006) [2023-12-27 18:46:47,785][20347] Updated weights for policy 0, policy_version 1895531 (0.0006) [2023-12-27 18:46:47,814][20381] Updated weights for policy 1, policy_version 1893958 (0.0008) [2023-12-27 18:46:47,843][20347] Updated weights for policy 0, policy_version 1895541 (0.0006) [2023-12-27 18:46:47,867][20381] Updated weights for policy 1, policy_version 1893968 (0.0005) [2023-12-27 18:46:47,900][20347] Updated weights for policy 0, policy_version 1895551 (0.0007) [2023-12-27 18:46:47,922][20381] Updated weights for policy 1, policy_version 1893978 (0.0007) [2023-12-27 18:46:48,569][20347] Updated weights for policy 0, policy_version 1895561 (0.0006) [2023-12-27 18:46:48,628][20347] Updated weights for policy 0, policy_version 1895571 (0.0007) [2023-12-27 18:46:48,642][20381] Updated weights for policy 1, policy_version 1893988 (0.0007) [2023-12-27 18:46:48,688][20347] Updated weights for policy 0, policy_version 1895581 (0.0007) [2023-12-27 18:46:48,699][20381] Updated weights for policy 1, policy_version 1893998 (0.0007) [2023-12-27 18:46:48,758][20381] Updated weights for policy 1, policy_version 1894008 (0.0008) [2023-12-27 18:46:49,438][20347] Updated weights for policy 0, policy_version 1895591 (0.0007) [2023-12-27 18:46:49,445][20381] Updated weights for policy 1, policy_version 1894018 (0.0008) [2023-12-27 18:46:49,495][20347] Updated weights for policy 0, policy_version 1895601 (0.0008) [2023-12-27 18:46:49,495][20381] Updated weights for policy 1, policy_version 1894028 (0.0005) [2023-12-27 18:46:49,553][20381] Updated weights for policy 1, policy_version 1894038 (0.0005) [2023-12-27 18:46:49,555][20347] Updated weights for policy 0, policy_version 1895611 (0.0008) [2023-12-27 18:46:49,611][20381] Updated weights for policy 1, policy_version 1894048 (0.0006) [2023-12-27 18:46:50,221][20347] Updated weights for policy 0, policy_version 1895621 (0.0007) [2023-12-27 18:46:50,243][20381] Updated weights for policy 1, policy_version 1894058 (0.0006) [2023-12-27 18:46:50,279][20347] Updated weights for policy 0, policy_version 1895631 (0.0007) [2023-12-27 18:46:50,293][20381] Updated weights for policy 1, policy_version 1894068 (0.0005) [2023-12-27 18:46:50,339][20347] Updated weights for policy 0, policy_version 1895641 (0.0007) [2023-12-27 18:46:50,345][20381] Updated weights for policy 1, policy_version 1894078 (0.0005) [2023-12-27 18:46:50,988][20347] Updated weights for policy 0, policy_version 1895651 (0.0008) [2023-12-27 18:46:51,051][20381] Updated weights for policy 1, policy_version 1894088 (0.0011) [2023-12-27 18:46:51,057][20347] Updated weights for policy 0, policy_version 1895661 (0.0010) [2023-12-27 18:46:51,107][20381] Updated weights for policy 1, policy_version 1894098 (0.0010) [2023-12-27 18:46:51,116][20347] Updated weights for policy 0, policy_version 1895671 (0.0010) [2023-12-27 18:46:51,168][20381] Updated weights for policy 1, policy_version 1894108 (0.0009) [2023-12-27 18:46:51,882][20347] Updated weights for policy 0, policy_version 1895681 (0.0009) [2023-12-27 18:46:51,893][20381] Updated weights for policy 1, policy_version 1894118 (0.0009) [2023-12-27 18:46:51,943][20347] Updated weights for policy 0, policy_version 1895691 (0.0007) [2023-12-27 18:46:51,958][20381] Updated weights for policy 1, policy_version 1894128 (0.0007) [2023-12-27 18:46:51,996][20347] Updated weights for policy 0, policy_version 1895701 (0.0007) [2023-12-27 18:46:52,021][20381] Updated weights for policy 1, policy_version 1894138 (0.0009) [2023-12-27 18:46:52,050][20347] Updated weights for policy 0, policy_version 1895711 (0.0008) [2023-12-27 18:46:52,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20410.6). Total num frames: 970350592. Throughput: 0: 10134.1, 1: 10508.0. Samples: 970319928. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:52,363][19170] Avg episode reward: [(0, '990.240'), (1, '837.370')] [2023-12-27 18:46:52,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001894144_484958208.pth... [2023-12-27 18:46:52,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001895712_485392384.pth... [2023-12-27 18:46:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001892896_484638720.pth [2023-12-27 18:46:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001894528_485089280.pth [2023-12-27 18:46:52,688][20381] Updated weights for policy 1, policy_version 1894148 (0.0008) [2023-12-27 18:46:52,740][20381] Updated weights for policy 1, policy_version 1894158 (0.0005) [2023-12-27 18:46:52,760][20347] Updated weights for policy 0, policy_version 1895721 (0.0008) [2023-12-27 18:46:52,798][20381] Updated weights for policy 1, policy_version 1894168 (0.0006) [2023-12-27 18:46:52,824][20347] Updated weights for policy 0, policy_version 1895731 (0.0007) [2023-12-27 18:46:52,881][20347] Updated weights for policy 0, policy_version 1895741 (0.0006) [2023-12-27 18:46:53,463][20381] Updated weights for policy 1, policy_version 1894178 (0.0008) [2023-12-27 18:46:53,498][20347] Updated weights for policy 0, policy_version 1895751 (0.0007) [2023-12-27 18:46:53,516][20381] Updated weights for policy 1, policy_version 1894188 (0.0010) [2023-12-27 18:46:53,549][20347] Updated weights for policy 0, policy_version 1895761 (0.0007) [2023-12-27 18:46:53,564][20381] Updated weights for policy 1, policy_version 1894198 (0.0010) [2023-12-27 18:46:53,606][20347] Updated weights for policy 0, policy_version 1895771 (0.0006) [2023-12-27 18:46:53,612][20381] Updated weights for policy 1, policy_version 1894208 (0.0010) [2023-12-27 18:46:54,261][20347] Updated weights for policy 0, policy_version 1895781 (0.0008) [2023-12-27 18:46:54,289][20381] Updated weights for policy 1, policy_version 1894218 (0.0005) [2023-12-27 18:46:54,307][20347] Updated weights for policy 0, policy_version 1895791 (0.0010) [2023-12-27 18:46:54,341][20381] Updated weights for policy 1, policy_version 1894228 (0.0006) [2023-12-27 18:46:54,359][20347] Updated weights for policy 0, policy_version 1895801 (0.0010) [2023-12-27 18:46:54,400][20381] Updated weights for policy 1, policy_version 1894238 (0.0009) [2023-12-27 18:46:54,965][20347] Updated weights for policy 0, policy_version 1895811 (0.0008) [2023-12-27 18:46:55,018][20347] Updated weights for policy 0, policy_version 1895821 (0.0008) [2023-12-27 18:46:55,069][20347] Updated weights for policy 0, policy_version 1895831 (0.0010) [2023-12-27 18:46:55,112][20381] Updated weights for policy 1, policy_version 1894248 (0.0009) [2023-12-27 18:46:55,166][20381] Updated weights for policy 1, policy_version 1894258 (0.0009) [2023-12-27 18:46:55,221][20381] Updated weights for policy 1, policy_version 1894268 (0.0005) [2023-12-27 18:46:55,716][20347] Updated weights for policy 0, policy_version 1895841 (0.0009) [2023-12-27 18:46:55,743][20381] Updated weights for policy 1, policy_version 1894278 (0.0008) [2023-12-27 18:46:55,760][20347] Updated weights for policy 0, policy_version 1895851 (0.0010) [2023-12-27 18:46:55,787][20381] Updated weights for policy 1, policy_version 1894288 (0.0010) [2023-12-27 18:46:55,805][20347] Updated weights for policy 0, policy_version 1895861 (0.0010) [2023-12-27 18:46:55,842][20381] Updated weights for policy 1, policy_version 1894298 (0.0010) [2023-12-27 18:46:55,850][20347] Updated weights for policy 0, policy_version 1895871 (0.0010) [2023-12-27 18:46:56,473][20381] Updated weights for policy 1, policy_version 1894308 (0.0008) [2023-12-27 18:46:56,529][20381] Updated weights for policy 1, policy_version 1894318 (0.0005) [2023-12-27 18:46:56,550][20347] Updated weights for policy 0, policy_version 1895881 (0.0010) [2023-12-27 18:46:56,577][20381] Updated weights for policy 1, policy_version 1894328 (0.0005) [2023-12-27 18:46:56,598][20347] Updated weights for policy 0, policy_version 1895891 (0.0010) [2023-12-27 18:46:56,643][20347] Updated weights for policy 0, policy_version 1895901 (0.0010) [2023-12-27 18:46:57,123][20381] Updated weights for policy 1, policy_version 1894338 (0.0005) [2023-12-27 18:46:57,176][20381] Updated weights for policy 1, policy_version 1894348 (0.0005) [2023-12-27 18:46:57,229][20381] Updated weights for policy 1, policy_version 1894358 (0.0005) [2023-12-27 18:46:57,274][20381] Updated weights for policy 1, policy_version 1894368 (0.0010) [2023-12-27 18:46:57,315][20347] Updated weights for policy 0, policy_version 1895911 (0.0008) [2023-12-27 18:46:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 970457088. Throughput: 0: 10209.9, 1: 10506.7. Samples: 970444960. Policy #0 lag: (min: 14.0, avg: 21.4, max: 46.0) [2023-12-27 18:46:57,363][19170] Avg episode reward: [(0, '990.200'), (1, '846.360')] [2023-12-27 18:46:57,370][20347] Updated weights for policy 0, policy_version 1895921 (0.0010) [2023-12-27 18:46:57,428][20347] Updated weights for policy 0, policy_version 1895931 (0.0010) [2023-12-27 18:46:57,832][20381] Updated weights for policy 1, policy_version 1894378 (0.0010) [2023-12-27 18:46:57,882][20381] Updated weights for policy 1, policy_version 1894388 (0.0010) [2023-12-27 18:46:57,933][20381] Updated weights for policy 1, policy_version 1894398 (0.0010) [2023-12-27 18:46:58,072][20347] Updated weights for policy 0, policy_version 1895941 (0.0010) [2023-12-27 18:46:58,131][20347] Updated weights for policy 0, policy_version 1895951 (0.0010) [2023-12-27 18:46:58,190][20347] Updated weights for policy 0, policy_version 1895961 (0.0008) [2023-12-27 18:46:58,659][20381] Updated weights for policy 1, policy_version 1894408 (0.0007) [2023-12-27 18:46:58,723][20381] Updated weights for policy 1, policy_version 1894418 (0.0006) [2023-12-27 18:46:58,793][20381] Updated weights for policy 1, policy_version 1894428 (0.0011) [2023-12-27 18:46:59,011][20347] Updated weights for policy 0, policy_version 1895971 (0.0008) [2023-12-27 18:46:59,065][20347] Updated weights for policy 0, policy_version 1895981 (0.0007) [2023-12-27 18:46:59,125][20347] Updated weights for policy 0, policy_version 1895991 (0.0008) [2023-12-27 18:46:59,574][20381] Updated weights for policy 1, policy_version 1894438 (0.0011) [2023-12-27 18:46:59,632][20381] Updated weights for policy 1, policy_version 1894448 (0.0008) [2023-12-27 18:46:59,684][20381] Updated weights for policy 1, policy_version 1894458 (0.0007) [2023-12-27 18:46:59,783][20347] Updated weights for policy 0, policy_version 1896001 (0.0008) [2023-12-27 18:46:59,842][20347] Updated weights for policy 0, policy_version 1896011 (0.0010) [2023-12-27 18:46:59,898][20347] Updated weights for policy 0, policy_version 1896021 (0.0010) [2023-12-27 18:46:59,964][20347] Updated weights for policy 0, policy_version 1896031 (0.0014) [2023-12-27 18:47:00,385][20381] Updated weights for policy 1, policy_version 1894468 (0.0007) [2023-12-27 18:47:00,436][20381] Updated weights for policy 1, policy_version 1894478 (0.0007) [2023-12-27 18:47:00,489][20381] Updated weights for policy 1, policy_version 1894488 (0.0006) [2023-12-27 18:47:00,608][20347] Updated weights for policy 0, policy_version 1896041 (0.0010) [2023-12-27 18:47:00,659][20347] Updated weights for policy 0, policy_version 1896051 (0.0010) [2023-12-27 18:47:00,716][20347] Updated weights for policy 0, policy_version 1896061 (0.0010) [2023-12-27 18:47:01,177][20381] Updated weights for policy 1, policy_version 1894498 (0.0007) [2023-12-27 18:47:01,229][20381] Updated weights for policy 1, policy_version 1894508 (0.0009) [2023-12-27 18:47:01,284][20381] Updated weights for policy 1, policy_version 1894518 (0.0008) [2023-12-27 18:47:01,333][20381] Updated weights for policy 1, policy_version 1894528 (0.0008) [2023-12-27 18:47:01,381][20347] Updated weights for policy 0, policy_version 1896071 (0.0008) [2023-12-27 18:47:01,437][20347] Updated weights for policy 0, policy_version 1896081 (0.0006) [2023-12-27 18:47:01,495][20347] Updated weights for policy 0, policy_version 1896091 (0.0006) [2023-12-27 18:47:02,068][20381] Updated weights for policy 1, policy_version 1894538 (0.0006) [2023-12-27 18:47:02,131][20381] Updated weights for policy 1, policy_version 1894548 (0.0007) [2023-12-27 18:47:02,141][20347] Updated weights for policy 0, policy_version 1896101 (0.0006) [2023-12-27 18:47:02,189][20381] Updated weights for policy 1, policy_version 1894558 (0.0006) [2023-12-27 18:47:02,192][20347] Updated weights for policy 0, policy_version 1896111 (0.0007) [2023-12-27 18:47:02,257][20347] Updated weights for policy 0, policy_version 1896121 (0.0006) [2023-12-27 18:47:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 970563584. Throughput: 0: 10228.9, 1: 10495.6. Samples: 970568452. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:02,363][19170] Avg episode reward: [(0, '990.200'), (1, '916.240')] [2023-12-27 18:47:02,850][20347] Updated weights for policy 0, policy_version 1896131 (0.0008) [2023-12-27 18:47:02,863][20381] Updated weights for policy 1, policy_version 1894568 (0.0006) [2023-12-27 18:47:02,904][20347] Updated weights for policy 0, policy_version 1896141 (0.0006) [2023-12-27 18:47:02,921][20381] Updated weights for policy 1, policy_version 1894578 (0.0006) [2023-12-27 18:47:02,956][20347] Updated weights for policy 0, policy_version 1896151 (0.0006) [2023-12-27 18:47:02,970][20381] Updated weights for policy 1, policy_version 1894588 (0.0007) [2023-12-27 18:47:03,615][20347] Updated weights for policy 0, policy_version 1896161 (0.0006) [2023-12-27 18:47:03,673][20347] Updated weights for policy 0, policy_version 1896171 (0.0007) [2023-12-27 18:47:03,700][20381] Updated weights for policy 1, policy_version 1894598 (0.0007) [2023-12-27 18:47:03,726][20347] Updated weights for policy 0, policy_version 1896181 (0.0009) [2023-12-27 18:47:03,757][20381] Updated weights for policy 1, policy_version 1894608 (0.0006) [2023-12-27 18:47:03,771][20347] Updated weights for policy 0, policy_version 1896191 (0.0010) [2023-12-27 18:47:03,814][20381] Updated weights for policy 1, policy_version 1894618 (0.0006) [2023-12-27 18:47:04,538][20347] Updated weights for policy 0, policy_version 1896201 (0.0010) [2023-12-27 18:47:04,547][20381] Updated weights for policy 1, policy_version 1894628 (0.0007) [2023-12-27 18:47:04,584][20347] Updated weights for policy 0, policy_version 1896211 (0.0010) [2023-12-27 18:47:04,604][20381] Updated weights for policy 1, policy_version 1894638 (0.0006) [2023-12-27 18:47:04,639][20347] Updated weights for policy 0, policy_version 1896221 (0.0010) [2023-12-27 18:47:04,659][20381] Updated weights for policy 1, policy_version 1894648 (0.0006) [2023-12-27 18:47:05,234][20347] Updated weights for policy 0, policy_version 1896231 (0.0007) [2023-12-27 18:47:05,285][20347] Updated weights for policy 0, policy_version 1896241 (0.0005) [2023-12-27 18:47:05,336][20347] Updated weights for policy 0, policy_version 1896251 (0.0005) [2023-12-27 18:47:05,393][20381] Updated weights for policy 1, policy_version 1894658 (0.0007) [2023-12-27 18:47:05,451][20381] Updated weights for policy 1, policy_version 1894668 (0.0007) [2023-12-27 18:47:05,540][20381] Updated weights for policy 1, policy_version 1894678 (0.0008) [2023-12-27 18:47:05,603][20381] Updated weights for policy 1, policy_version 1894688 (0.0007) [2023-12-27 18:47:05,931][20347] Updated weights for policy 0, policy_version 1896261 (0.0005) [2023-12-27 18:47:05,982][20347] Updated weights for policy 0, policy_version 1896271 (0.0005) [2023-12-27 18:47:06,035][20347] Updated weights for policy 0, policy_version 1896281 (0.0005) [2023-12-27 18:47:06,276][20381] Updated weights for policy 1, policy_version 1894698 (0.0007) [2023-12-27 18:47:06,325][20381] Updated weights for policy 1, policy_version 1894708 (0.0008) [2023-12-27 18:47:06,374][20381] Updated weights for policy 1, policy_version 1894718 (0.0008) [2023-12-27 18:47:06,731][20347] Updated weights for policy 0, policy_version 1896291 (0.0010) [2023-12-27 18:47:06,794][20347] Updated weights for policy 0, policy_version 1896301 (0.0009) [2023-12-27 18:47:06,860][20347] Updated weights for policy 0, policy_version 1896311 (0.0010) [2023-12-27 18:47:07,089][20381] Updated weights for policy 1, policy_version 1894728 (0.0007) [2023-12-27 18:47:07,137][20381] Updated weights for policy 1, policy_version 1894738 (0.0008) [2023-12-27 18:47:07,182][20381] Updated weights for policy 1, policy_version 1894748 (0.0008) [2023-12-27 18:47:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 970661888. Throughput: 0: 10264.1, 1: 10449.8. Samples: 970629536. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.680')] [2023-12-27 18:47:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001894752_485113856.pth... [2023-12-27 18:47:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001896320_485548032.pth... [2023-12-27 18:47:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001893536_484802560.pth [2023-12-27 18:47:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001895136_485244928.pth [2023-12-27 18:47:07,491][20347] Updated weights for policy 0, policy_version 1896321 (0.0011) [2023-12-27 18:47:07,539][20347] Updated weights for policy 0, policy_version 1896331 (0.0007) [2023-12-27 18:47:07,593][20347] Updated weights for policy 0, policy_version 1896341 (0.0006) [2023-12-27 18:47:07,652][20347] Updated weights for policy 0, policy_version 1896351 (0.0005) [2023-12-27 18:47:07,944][20381] Updated weights for policy 1, policy_version 1894758 (0.0008) [2023-12-27 18:47:07,996][20381] Updated weights for policy 1, policy_version 1894768 (0.0008) [2023-12-27 18:47:08,051][20381] Updated weights for policy 1, policy_version 1894778 (0.0007) [2023-12-27 18:47:08,204][20347] Updated weights for policy 0, policy_version 1896361 (0.0007) [2023-12-27 18:47:08,261][20347] Updated weights for policy 0, policy_version 1896371 (0.0008) [2023-12-27 18:47:08,339][20347] Updated weights for policy 0, policy_version 1896381 (0.0006) [2023-12-27 18:47:08,786][20381] Updated weights for policy 1, policy_version 1894788 (0.0009) [2023-12-27 18:47:08,839][20381] Updated weights for policy 1, policy_version 1894798 (0.0007) [2023-12-27 18:47:08,885][20381] Updated weights for policy 1, policy_version 1894808 (0.0008) [2023-12-27 18:47:09,046][20347] Updated weights for policy 0, policy_version 1896391 (0.0008) [2023-12-27 18:47:09,095][20347] Updated weights for policy 0, policy_version 1896401 (0.0008) [2023-12-27 18:47:09,144][20347] Updated weights for policy 0, policy_version 1896411 (0.0007) [2023-12-27 18:47:09,609][20381] Updated weights for policy 1, policy_version 1894818 (0.0007) [2023-12-27 18:47:09,669][20381] Updated weights for policy 1, policy_version 1894828 (0.0008) [2023-12-27 18:47:09,726][20381] Updated weights for policy 1, policy_version 1894838 (0.0008) [2023-12-27 18:47:09,782][20381] Updated weights for policy 1, policy_version 1894848 (0.0009) [2023-12-27 18:47:09,817][20347] Updated weights for policy 0, policy_version 1896421 (0.0006) [2023-12-27 18:47:09,882][20347] Updated weights for policy 0, policy_version 1896431 (0.0008) [2023-12-27 18:47:09,948][20347] Updated weights for policy 0, policy_version 1896441 (0.0008) [2023-12-27 18:47:10,512][20381] Updated weights for policy 1, policy_version 1894858 (0.0007) [2023-12-27 18:47:10,571][20381] Updated weights for policy 1, policy_version 1894868 (0.0005) [2023-12-27 18:47:10,629][20381] Updated weights for policy 1, policy_version 1894878 (0.0007) [2023-12-27 18:47:10,692][20347] Updated weights for policy 0, policy_version 1896451 (0.0009) [2023-12-27 18:47:10,751][20347] Updated weights for policy 0, policy_version 1896461 (0.0011) [2023-12-27 18:47:10,805][20347] Updated weights for policy 0, policy_version 1896471 (0.0009) [2023-12-27 18:47:11,315][20381] Updated weights for policy 1, policy_version 1894888 (0.0010) [2023-12-27 18:47:11,383][20381] Updated weights for policy 1, policy_version 1894898 (0.0010) [2023-12-27 18:47:11,437][20381] Updated weights for policy 1, policy_version 1894908 (0.0008) [2023-12-27 18:47:11,526][20347] Updated weights for policy 0, policy_version 1896481 (0.0010) [2023-12-27 18:47:11,585][20347] Updated weights for policy 0, policy_version 1896491 (0.0010) [2023-12-27 18:47:11,656][20347] Updated weights for policy 0, policy_version 1896501 (0.0010) [2023-12-27 18:47:11,720][20347] Updated weights for policy 0, policy_version 1896511 (0.0009) [2023-12-27 18:47:12,162][20381] Updated weights for policy 1, policy_version 1894918 (0.0009) [2023-12-27 18:47:12,218][20381] Updated weights for policy 1, policy_version 1894928 (0.0011) [2023-12-27 18:47:12,276][20381] Updated weights for policy 1, policy_version 1894938 (0.0009) [2023-12-27 18:47:12,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 970760192. Throughput: 0: 10300.0, 1: 10356.6. Samples: 970751012. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '913.550')] [2023-12-27 18:47:12,441][20347] Updated weights for policy 0, policy_version 1896521 (0.0012) [2023-12-27 18:47:12,499][20347] Updated weights for policy 0, policy_version 1896531 (0.0010) [2023-12-27 18:47:12,557][20347] Updated weights for policy 0, policy_version 1896541 (0.0010) [2023-12-27 18:47:12,945][20381] Updated weights for policy 1, policy_version 1894948 (0.0008) [2023-12-27 18:47:13,007][20381] Updated weights for policy 1, policy_version 1894958 (0.0011) [2023-12-27 18:47:13,059][20381] Updated weights for policy 1, policy_version 1894968 (0.0011) [2023-12-27 18:47:13,208][20347] Updated weights for policy 0, policy_version 1896551 (0.0008) [2023-12-27 18:47:13,268][20347] Updated weights for policy 0, policy_version 1896561 (0.0007) [2023-12-27 18:47:13,329][20347] Updated weights for policy 0, policy_version 1896571 (0.0007) [2023-12-27 18:47:13,748][20381] Updated weights for policy 1, policy_version 1894978 (0.0011) [2023-12-27 18:47:13,803][20381] Updated weights for policy 1, policy_version 1894988 (0.0011) [2023-12-27 18:47:13,856][20381] Updated weights for policy 1, policy_version 1894998 (0.0008) [2023-12-27 18:47:13,910][20381] Updated weights for policy 1, policy_version 1895008 (0.0010) [2023-12-27 18:47:14,002][20347] Updated weights for policy 0, policy_version 1896581 (0.0009) [2023-12-27 18:47:14,053][20347] Updated weights for policy 0, policy_version 1896591 (0.0007) [2023-12-27 18:47:14,099][20347] Updated weights for policy 0, policy_version 1896601 (0.0005) [2023-12-27 18:47:14,542][20381] Updated weights for policy 1, policy_version 1895018 (0.0008) [2023-12-27 18:47:14,607][20381] Updated weights for policy 1, policy_version 1895028 (0.0008) [2023-12-27 18:47:14,667][20347] Updated weights for policy 0, policy_version 1896611 (0.0007) [2023-12-27 18:47:14,670][20381] Updated weights for policy 1, policy_version 1895038 (0.0005) [2023-12-27 18:47:14,722][20347] Updated weights for policy 0, policy_version 1896621 (0.0006) [2023-12-27 18:47:14,788][20347] Updated weights for policy 0, policy_version 1896631 (0.0009) [2023-12-27 18:47:15,342][20381] Updated weights for policy 1, policy_version 1895048 (0.0006) [2023-12-27 18:47:15,405][20381] Updated weights for policy 1, policy_version 1895058 (0.0006) [2023-12-27 18:47:15,466][20381] Updated weights for policy 1, policy_version 1895068 (0.0005) [2023-12-27 18:47:15,479][20347] Updated weights for policy 0, policy_version 1896641 (0.0008) [2023-12-27 18:47:15,530][20347] Updated weights for policy 0, policy_version 1896651 (0.0007) [2023-12-27 18:47:15,584][20347] Updated weights for policy 0, policy_version 1896661 (0.0010) [2023-12-27 18:47:15,645][20347] Updated weights for policy 0, policy_version 1896672 (0.0007) [2023-12-27 18:47:16,003][20381] Updated weights for policy 1, policy_version 1895078 (0.0008) [2023-12-27 18:47:16,047][20381] Updated weights for policy 1, policy_version 1895088 (0.0010) [2023-12-27 18:47:16,092][20381] Updated weights for policy 1, policy_version 1895098 (0.0010) [2023-12-27 18:47:16,274][20347] Updated weights for policy 0, policy_version 1896682 (0.0006) [2023-12-27 18:47:16,319][20347] Updated weights for policy 0, policy_version 1896692 (0.0010) [2023-12-27 18:47:16,360][20347] Updated weights for policy 0, policy_version 1896702 (0.0009) [2023-12-27 18:47:16,741][20381] Updated weights for policy 1, policy_version 1895108 (0.0011) [2023-12-27 18:47:16,793][20381] Updated weights for policy 1, policy_version 1895118 (0.0010) [2023-12-27 18:47:16,842][20381] Updated weights for policy 1, policy_version 1895128 (0.0009) [2023-12-27 18:47:16,993][20347] Updated weights for policy 0, policy_version 1896712 (0.0009) [2023-12-27 18:47:17,042][20347] Updated weights for policy 0, policy_version 1896722 (0.0010) [2023-12-27 18:47:17,093][20347] Updated weights for policy 0, policy_version 1896732 (0.0010) [2023-12-27 18:47:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.0, 300 sec: 20466.1). Total num frames: 970866688. Throughput: 0: 10319.0, 1: 10344.6. Samples: 970877164. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '898.010')] [2023-12-27 18:47:17,511][20381] Updated weights for policy 1, policy_version 1895138 (0.0010) [2023-12-27 18:47:17,563][20381] Updated weights for policy 1, policy_version 1895148 (0.0011) [2023-12-27 18:47:17,615][20381] Updated weights for policy 1, policy_version 1895158 (0.0010) [2023-12-27 18:47:17,667][20381] Updated weights for policy 1, policy_version 1895168 (0.0010) [2023-12-27 18:47:17,708][20347] Updated weights for policy 0, policy_version 1896742 (0.0010) [2023-12-27 18:47:17,762][20347] Updated weights for policy 0, policy_version 1896752 (0.0010) [2023-12-27 18:47:17,810][20347] Updated weights for policy 0, policy_version 1896762 (0.0007) [2023-12-27 18:47:18,382][20381] Updated weights for policy 1, policy_version 1895178 (0.0011) [2023-12-27 18:47:18,435][20347] Updated weights for policy 0, policy_version 1896772 (0.0006) [2023-12-27 18:47:18,442][20381] Updated weights for policy 1, policy_version 1895188 (0.0011) [2023-12-27 18:47:18,495][20381] Updated weights for policy 1, policy_version 1895198 (0.0010) [2023-12-27 18:47:18,497][20347] Updated weights for policy 0, policy_version 1896782 (0.0010) [2023-12-27 18:47:18,559][20347] Updated weights for policy 0, policy_version 1896792 (0.0010) [2023-12-27 18:47:19,176][20381] Updated weights for policy 1, policy_version 1895208 (0.0007) [2023-12-27 18:47:19,185][20347] Updated weights for policy 0, policy_version 1896802 (0.0010) [2023-12-27 18:47:19,249][20347] Updated weights for policy 0, policy_version 1896812 (0.0010) [2023-12-27 18:47:19,256][20381] Updated weights for policy 1, policy_version 1895218 (0.0009) [2023-12-27 18:47:19,307][20347] Updated weights for policy 0, policy_version 1896822 (0.0007) [2023-12-27 18:47:19,315][20381] Updated weights for policy 1, policy_version 1895228 (0.0011) [2023-12-27 18:47:19,373][20347] Updated weights for policy 0, policy_version 1896832 (0.0008) [2023-12-27 18:47:19,955][20381] Updated weights for policy 1, policy_version 1895238 (0.0008) [2023-12-27 18:47:20,021][20381] Updated weights for policy 1, policy_version 1895248 (0.0006) [2023-12-27 18:47:20,084][20347] Updated weights for policy 0, policy_version 1896842 (0.0008) [2023-12-27 18:47:20,090][20381] Updated weights for policy 1, policy_version 1895258 (0.0007) [2023-12-27 18:47:20,144][20347] Updated weights for policy 0, policy_version 1896852 (0.0007) [2023-12-27 18:47:20,204][20347] Updated weights for policy 0, policy_version 1896862 (0.0006) [2023-12-27 18:47:20,720][20381] Updated weights for policy 1, policy_version 1895268 (0.0007) [2023-12-27 18:47:20,775][20381] Updated weights for policy 1, policy_version 1895278 (0.0008) [2023-12-27 18:47:20,824][20381] Updated weights for policy 1, policy_version 1895288 (0.0007) [2023-12-27 18:47:20,841][20347] Updated weights for policy 0, policy_version 1896872 (0.0006) [2023-12-27 18:47:20,899][20347] Updated weights for policy 0, policy_version 1896882 (0.0008) [2023-12-27 18:47:20,952][20347] Updated weights for policy 0, policy_version 1896892 (0.0007) [2023-12-27 18:47:21,518][20381] Updated weights for policy 1, policy_version 1895298 (0.0006) [2023-12-27 18:47:21,571][20381] Updated weights for policy 1, policy_version 1895308 (0.0008) [2023-12-27 18:47:21,627][20381] Updated weights for policy 1, policy_version 1895318 (0.0007) [2023-12-27 18:47:21,695][20347] Updated weights for policy 0, policy_version 1896902 (0.0009) [2023-12-27 18:47:21,697][20381] Updated weights for policy 1, policy_version 1895328 (0.0006) [2023-12-27 18:47:21,768][20347] Updated weights for policy 0, policy_version 1896912 (0.0008) [2023-12-27 18:47:21,832][20347] Updated weights for policy 0, policy_version 1896922 (0.0007) [2023-12-27 18:47:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 970964992. Throughput: 0: 10357.6, 1: 10301.6. Samples: 970939720. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:22,363][19170] Avg episode reward: [(0, '990.230'), (1, '936.380')] [2023-12-27 18:47:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001896928_485703680.pth... [2023-12-27 18:47:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001895712_485392384.pth [2023-12-27 18:47:22,381][20381] Updated weights for policy 1, policy_version 1895338 (0.0007) [2023-12-27 18:47:22,448][20381] Updated weights for policy 1, policy_version 1895348 (0.0008) [2023-12-27 18:47:22,509][20381] Updated weights for policy 1, policy_version 1895358 (0.0007) [2023-12-27 18:47:22,522][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001895360_485269504.pth... [2023-12-27 18:47:22,527][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001894144_484958208.pth [2023-12-27 18:47:22,529][20347] Updated weights for policy 0, policy_version 1896932 (0.0006) [2023-12-27 18:47:22,591][20347] Updated weights for policy 0, policy_version 1896942 (0.0006) [2023-12-27 18:47:22,649][20347] Updated weights for policy 0, policy_version 1896952 (0.0008) [2023-12-27 18:47:23,223][20381] Updated weights for policy 1, policy_version 1895368 (0.0010) [2023-12-27 18:47:23,287][20381] Updated weights for policy 1, policy_version 1895378 (0.0011) [2023-12-27 18:47:23,289][20347] Updated weights for policy 0, policy_version 1896962 (0.0007) [2023-12-27 18:47:23,343][20381] Updated weights for policy 1, policy_version 1895389 (0.0006) [2023-12-27 18:47:23,345][20347] Updated weights for policy 0, policy_version 1896972 (0.0007) [2023-12-27 18:47:23,396][20347] Updated weights for policy 0, policy_version 1896982 (0.0007) [2023-12-27 18:47:23,448][20347] Updated weights for policy 0, policy_version 1896992 (0.0008) [2023-12-27 18:47:23,971][20381] Updated weights for policy 1, policy_version 1895399 (0.0007) [2023-12-27 18:47:24,022][20381] Updated weights for policy 1, policy_version 1895409 (0.0008) [2023-12-27 18:47:24,068][20381] Updated weights for policy 1, policy_version 1895419 (0.0008) [2023-12-27 18:47:24,133][20347] Updated weights for policy 0, policy_version 1897002 (0.0007) [2023-12-27 18:47:24,187][20347] Updated weights for policy 0, policy_version 1897012 (0.0007) [2023-12-27 18:47:24,239][20347] Updated weights for policy 0, policy_version 1897022 (0.0007) [2023-12-27 18:47:24,791][20381] Updated weights for policy 1, policy_version 1895429 (0.0008) [2023-12-27 18:47:24,850][20381] Updated weights for policy 1, policy_version 1895439 (0.0008) [2023-12-27 18:47:24,901][20381] Updated weights for policy 1, policy_version 1895449 (0.0007) [2023-12-27 18:47:24,918][20347] Updated weights for policy 0, policy_version 1897032 (0.0006) [2023-12-27 18:47:24,965][20347] Updated weights for policy 0, policy_version 1897042 (0.0007) [2023-12-27 18:47:25,018][20347] Updated weights for policy 0, policy_version 1897052 (0.0008) [2023-12-27 18:47:25,615][20347] Updated weights for policy 0, policy_version 1897062 (0.0006) [2023-12-27 18:47:25,637][20381] Updated weights for policy 1, policy_version 1895459 (0.0006) [2023-12-27 18:47:25,672][20347] Updated weights for policy 0, policy_version 1897072 (0.0005) [2023-12-27 18:47:25,693][20381] Updated weights for policy 1, policy_version 1895469 (0.0008) [2023-12-27 18:47:25,728][20347] Updated weights for policy 0, policy_version 1897082 (0.0005) [2023-12-27 18:47:25,749][20381] Updated weights for policy 1, policy_version 1895479 (0.0008) [2023-12-27 18:47:26,258][20347] Updated weights for policy 0, policy_version 1897092 (0.0006) [2023-12-27 18:47:26,308][20347] Updated weights for policy 0, policy_version 1897102 (0.0007) [2023-12-27 18:47:26,362][20347] Updated weights for policy 0, policy_version 1897112 (0.0005) [2023-12-27 18:47:26,508][20381] Updated weights for policy 1, policy_version 1895489 (0.0006) [2023-12-27 18:47:26,557][20381] Updated weights for policy 1, policy_version 1895499 (0.0007) [2023-12-27 18:47:26,612][20381] Updated weights for policy 1, policy_version 1895509 (0.0007) [2023-12-27 18:47:26,666][20381] Updated weights for policy 1, policy_version 1895519 (0.0009) [2023-12-27 18:47:26,937][20347] Updated weights for policy 0, policy_version 1897122 (0.0006) [2023-12-27 18:47:26,993][20347] Updated weights for policy 0, policy_version 1897132 (0.0006) [2023-12-27 18:47:27,048][20347] Updated weights for policy 0, policy_version 1897142 (0.0008) [2023-12-27 18:47:27,101][20347] Updated weights for policy 0, policy_version 1897152 (0.0007) [2023-12-27 18:47:27,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 971071488. Throughput: 0: 10408.4, 1: 10274.7. Samples: 971062168. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:27,363][19170] Avg episode reward: [(0, '990.230'), (1, '957.600')] [2023-12-27 18:47:27,394][20381] Updated weights for policy 1, policy_version 1895529 (0.0008) [2023-12-27 18:47:27,441][20381] Updated weights for policy 1, policy_version 1895539 (0.0005) [2023-12-27 18:47:27,495][20381] Updated weights for policy 1, policy_version 1895549 (0.0006) [2023-12-27 18:47:27,735][20347] Updated weights for policy 0, policy_version 1897162 (0.0009) [2023-12-27 18:47:27,783][20347] Updated weights for policy 0, policy_version 1897172 (0.0007) [2023-12-27 18:47:27,827][20347] Updated weights for policy 0, policy_version 1897182 (0.0007) [2023-12-27 18:47:28,098][20381] Updated weights for policy 1, policy_version 1895559 (0.0007) [2023-12-27 18:47:28,160][20381] Updated weights for policy 1, policy_version 1895569 (0.0008) [2023-12-27 18:47:28,216][20381] Updated weights for policy 1, policy_version 1895579 (0.0008) [2023-12-27 18:47:28,511][20347] Updated weights for policy 0, policy_version 1897192 (0.0009) [2023-12-27 18:47:28,575][20347] Updated weights for policy 0, policy_version 1897202 (0.0007) [2023-12-27 18:47:28,633][20347] Updated weights for policy 0, policy_version 1897212 (0.0008) [2023-12-27 18:47:28,830][20381] Updated weights for policy 1, policy_version 1895589 (0.0008) [2023-12-27 18:47:28,886][20381] Updated weights for policy 1, policy_version 1895599 (0.0009) [2023-12-27 18:47:28,940][20381] Updated weights for policy 1, policy_version 1895609 (0.0007) [2023-12-27 18:47:29,351][20347] Updated weights for policy 0, policy_version 1897222 (0.0008) [2023-12-27 18:47:29,414][20347] Updated weights for policy 0, policy_version 1897232 (0.0008) [2023-12-27 18:47:29,476][20347] Updated weights for policy 0, policy_version 1897242 (0.0007) [2023-12-27 18:47:29,579][20381] Updated weights for policy 1, policy_version 1895619 (0.0010) [2023-12-27 18:47:29,632][20381] Updated weights for policy 1, policy_version 1895629 (0.0011) [2023-12-27 18:47:29,684][20381] Updated weights for policy 1, policy_version 1895639 (0.0011) [2023-12-27 18:47:30,178][20347] Updated weights for policy 0, policy_version 1897252 (0.0008) [2023-12-27 18:47:30,231][20347] Updated weights for policy 0, policy_version 1897262 (0.0008) [2023-12-27 18:47:30,288][20347] Updated weights for policy 0, policy_version 1897272 (0.0008) [2023-12-27 18:47:30,376][20381] Updated weights for policy 1, policy_version 1895649 (0.0011) [2023-12-27 18:47:30,430][20381] Updated weights for policy 1, policy_version 1895659 (0.0007) [2023-12-27 18:47:30,476][20381] Updated weights for policy 1, policy_version 1895669 (0.0008) [2023-12-27 18:47:30,526][20381] Updated weights for policy 1, policy_version 1895679 (0.0005) [2023-12-27 18:47:31,028][20347] Updated weights for policy 0, policy_version 1897282 (0.0007) [2023-12-27 18:47:31,074][20381] Updated weights for policy 1, policy_version 1895689 (0.0007) [2023-12-27 18:47:31,090][20347] Updated weights for policy 0, policy_version 1897292 (0.0007) [2023-12-27 18:47:31,136][20381] Updated weights for policy 1, policy_version 1895699 (0.0007) [2023-12-27 18:47:31,149][20347] Updated weights for policy 0, policy_version 1897302 (0.0007) [2023-12-27 18:47:31,204][20381] Updated weights for policy 1, policy_version 1895709 (0.0007) [2023-12-27 18:47:31,212][20347] Updated weights for policy 0, policy_version 1897312 (0.0007) [2023-12-27 18:47:31,913][20381] Updated weights for policy 1, policy_version 1895719 (0.0008) [2023-12-27 18:47:31,966][20381] Updated weights for policy 1, policy_version 1895729 (0.0005) [2023-12-27 18:47:31,968][20347] Updated weights for policy 0, policy_version 1897322 (0.0006) [2023-12-27 18:47:32,015][20381] Updated weights for policy 1, policy_version 1895739 (0.0007) [2023-12-27 18:47:32,026][20347] Updated weights for policy 0, policy_version 1897332 (0.0009) [2023-12-27 18:47:32,078][20347] Updated weights for policy 0, policy_version 1897342 (0.0008) [2023-12-27 18:47:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 971177984. Throughput: 0: 10412.2, 1: 10249.6. Samples: 971187236. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '859.130')] [2023-12-27 18:47:32,632][20381] Updated weights for policy 1, policy_version 1895749 (0.0007) [2023-12-27 18:47:32,693][20381] Updated weights for policy 1, policy_version 1895759 (0.0005) [2023-12-27 18:47:32,732][20347] Updated weights for policy 0, policy_version 1897352 (0.0007) [2023-12-27 18:47:32,746][20381] Updated weights for policy 1, policy_version 1895769 (0.0006) [2023-12-27 18:47:32,785][20347] Updated weights for policy 0, policy_version 1897362 (0.0006) [2023-12-27 18:47:32,843][20347] Updated weights for policy 0, policy_version 1897372 (0.0006) [2023-12-27 18:47:33,423][20381] Updated weights for policy 1, policy_version 1895779 (0.0006) [2023-12-27 18:47:33,455][20347] Updated weights for policy 0, policy_version 1897382 (0.0006) [2023-12-27 18:47:33,468][20381] Updated weights for policy 1, policy_version 1895789 (0.0005) [2023-12-27 18:47:33,507][20347] Updated weights for policy 0, policy_version 1897392 (0.0005) [2023-12-27 18:47:33,509][20381] Updated weights for policy 1, policy_version 1895799 (0.0005) [2023-12-27 18:47:33,559][20347] Updated weights for policy 0, policy_version 1897402 (0.0006) [2023-12-27 18:47:34,143][20381] Updated weights for policy 1, policy_version 1895809 (0.0005) [2023-12-27 18:47:34,199][20381] Updated weights for policy 1, policy_version 1895819 (0.0007) [2023-12-27 18:47:34,216][20347] Updated weights for policy 0, policy_version 1897412 (0.0007) [2023-12-27 18:47:34,260][20381] Updated weights for policy 1, policy_version 1895829 (0.0009) [2023-12-27 18:47:34,275][20347] Updated weights for policy 0, policy_version 1897422 (0.0007) [2023-12-27 18:47:34,319][20381] Updated weights for policy 1, policy_version 1895839 (0.0011) [2023-12-27 18:47:34,340][20347] Updated weights for policy 0, policy_version 1897432 (0.0008) [2023-12-27 18:47:34,992][20381] Updated weights for policy 1, policy_version 1895849 (0.0006) [2023-12-27 18:47:35,052][20381] Updated weights for policy 1, policy_version 1895859 (0.0007) [2023-12-27 18:47:35,076][20347] Updated weights for policy 0, policy_version 1897442 (0.0008) [2023-12-27 18:47:35,110][20381] Updated weights for policy 1, policy_version 1895869 (0.0008) [2023-12-27 18:47:35,132][20347] Updated weights for policy 0, policy_version 1897452 (0.0008) [2023-12-27 18:47:35,179][20347] Updated weights for policy 0, policy_version 1897462 (0.0008) [2023-12-27 18:47:35,230][20347] Updated weights for policy 0, policy_version 1897472 (0.0007) [2023-12-27 18:47:35,777][20381] Updated weights for policy 1, policy_version 1895879 (0.0008) [2023-12-27 18:47:35,834][20381] Updated weights for policy 1, policy_version 1895889 (0.0008) [2023-12-27 18:47:35,844][20347] Updated weights for policy 0, policy_version 1897482 (0.0006) [2023-12-27 18:47:35,889][20381] Updated weights for policy 1, policy_version 1895899 (0.0008) [2023-12-27 18:47:35,899][20347] Updated weights for policy 0, policy_version 1897492 (0.0005) [2023-12-27 18:47:35,958][20347] Updated weights for policy 0, policy_version 1897502 (0.0005) [2023-12-27 18:47:36,552][20381] Updated weights for policy 1, policy_version 1895909 (0.0008) [2023-12-27 18:47:36,592][20347] Updated weights for policy 0, policy_version 1897512 (0.0008) [2023-12-27 18:47:36,602][20381] Updated weights for policy 1, policy_version 1895919 (0.0007) [2023-12-27 18:47:36,651][20347] Updated weights for policy 0, policy_version 1897522 (0.0010) [2023-12-27 18:47:36,653][20381] Updated weights for policy 1, policy_version 1895929 (0.0007) [2023-12-27 18:47:36,703][20347] Updated weights for policy 0, policy_version 1897532 (0.0010) [2023-12-27 18:47:37,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 971276288. Throughput: 0: 10420.2, 1: 10244.3. Samples: 971249836. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:37,364][19170] Avg episode reward: [(0, '990.240'), (1, '850.370')] [2023-12-27 18:47:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001895936_485416960.pth... [2023-12-27 18:47:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001894752_485113856.pth [2023-12-27 18:47:37,407][20347] Updated weights for policy 0, policy_version 1897542 (0.0010) [2023-12-27 18:47:37,409][20381] Updated weights for policy 1, policy_version 1895939 (0.0005) [2023-12-27 18:47:37,466][20381] Updated weights for policy 1, policy_version 1895949 (0.0006) [2023-12-27 18:47:37,471][20347] Updated weights for policy 0, policy_version 1897552 (0.0010) [2023-12-27 18:47:37,525][20381] Updated weights for policy 1, policy_version 1895959 (0.0006) [2023-12-27 18:47:37,527][20347] Updated weights for policy 0, policy_version 1897562 (0.0010) [2023-12-27 18:47:37,563][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001897568_485867520.pth... [2023-12-27 18:47:37,567][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001896320_485548032.pth [2023-12-27 18:47:38,228][20347] Updated weights for policy 0, policy_version 1897572 (0.0010) [2023-12-27 18:47:38,235][20381] Updated weights for policy 1, policy_version 1895969 (0.0007) [2023-12-27 18:47:38,283][20347] Updated weights for policy 0, policy_version 1897582 (0.0010) [2023-12-27 18:47:38,286][20381] Updated weights for policy 1, policy_version 1895979 (0.0010) [2023-12-27 18:47:38,344][20347] Updated weights for policy 0, policy_version 1897592 (0.0010) [2023-12-27 18:47:38,348][20381] Updated weights for policy 1, policy_version 1895989 (0.0008) [2023-12-27 18:47:38,406][20381] Updated weights for policy 1, policy_version 1895999 (0.0006) [2023-12-27 18:47:39,055][20347] Updated weights for policy 0, policy_version 1897602 (0.0010) [2023-12-27 18:47:39,097][20381] Updated weights for policy 1, policy_version 1896009 (0.0006) [2023-12-27 18:47:39,109][20347] Updated weights for policy 0, policy_version 1897612 (0.0007) [2023-12-27 18:47:39,155][20381] Updated weights for policy 1, policy_version 1896019 (0.0007) [2023-12-27 18:47:39,166][20347] Updated weights for policy 0, policy_version 1897622 (0.0007) [2023-12-27 18:47:39,210][20381] Updated weights for policy 1, policy_version 1896029 (0.0008) [2023-12-27 18:47:39,224][20347] Updated weights for policy 0, policy_version 1897632 (0.0008) [2023-12-27 18:47:39,937][20347] Updated weights for policy 0, policy_version 1897642 (0.0010) [2023-12-27 18:47:39,950][20381] Updated weights for policy 1, policy_version 1896039 (0.0008) [2023-12-27 18:47:39,998][20347] Updated weights for policy 0, policy_version 1897652 (0.0011) [2023-12-27 18:47:40,007][20381] Updated weights for policy 1, policy_version 1896049 (0.0006) [2023-12-27 18:47:40,055][20347] Updated weights for policy 0, policy_version 1897662 (0.0007) [2023-12-27 18:47:40,061][20381] Updated weights for policy 1, policy_version 1896059 (0.0007) [2023-12-27 18:47:40,753][20347] Updated weights for policy 0, policy_version 1897672 (0.0007) [2023-12-27 18:47:40,768][20381] Updated weights for policy 1, policy_version 1896069 (0.0008) [2023-12-27 18:47:40,806][20347] Updated weights for policy 0, policy_version 1897682 (0.0006) [2023-12-27 18:47:40,823][20381] Updated weights for policy 1, policy_version 1896079 (0.0008) [2023-12-27 18:47:40,862][20347] Updated weights for policy 0, policy_version 1897692 (0.0005) [2023-12-27 18:47:40,884][20381] Updated weights for policy 1, policy_version 1896089 (0.0006) [2023-12-27 18:47:41,556][20347] Updated weights for policy 0, policy_version 1897702 (0.0008) [2023-12-27 18:47:41,613][20347] Updated weights for policy 0, policy_version 1897712 (0.0007) [2023-12-27 18:47:41,673][20347] Updated weights for policy 0, policy_version 1897722 (0.0008) [2023-12-27 18:47:41,686][20381] Updated weights for policy 1, policy_version 1896099 (0.0007) [2023-12-27 18:47:41,758][20381] Updated weights for policy 1, policy_version 1896109 (0.0008) [2023-12-27 18:47:41,824][20381] Updated weights for policy 1, policy_version 1896119 (0.0006) [2023-12-27 18:47:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20438.4). Total num frames: 971374592. Throughput: 0: 10378.3, 1: 10170.3. Samples: 971369648. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:42,363][19170] Avg episode reward: [(0, '980.490'), (1, '949.950')] [2023-12-27 18:47:42,407][20347] Updated weights for policy 0, policy_version 1897732 (0.0008) [2023-12-27 18:47:42,463][20347] Updated weights for policy 0, policy_version 1897742 (0.0008) [2023-12-27 18:47:42,481][20381] Updated weights for policy 1, policy_version 1896129 (0.0006) [2023-12-27 18:47:42,512][20347] Updated weights for policy 0, policy_version 1897752 (0.0007) [2023-12-27 18:47:42,541][20381] Updated weights for policy 1, policy_version 1896139 (0.0006) [2023-12-27 18:47:42,602][20381] Updated weights for policy 1, policy_version 1896149 (0.0006) [2023-12-27 18:47:42,666][20381] Updated weights for policy 1, policy_version 1896159 (0.0006) [2023-12-27 18:47:43,186][20347] Updated weights for policy 0, policy_version 1897762 (0.0007) [2023-12-27 18:47:43,244][20347] Updated weights for policy 0, policy_version 1897772 (0.0007) [2023-12-27 18:47:43,276][20381] Updated weights for policy 1, policy_version 1896169 (0.0008) [2023-12-27 18:47:43,300][20347] Updated weights for policy 0, policy_version 1897782 (0.0006) [2023-12-27 18:47:43,328][20381] Updated weights for policy 1, policy_version 1896179 (0.0005) [2023-12-27 18:47:43,352][20347] Updated weights for policy 0, policy_version 1897792 (0.0005) [2023-12-27 18:47:43,377][20381] Updated weights for policy 1, policy_version 1896189 (0.0006) [2023-12-27 18:47:44,028][20347] Updated weights for policy 0, policy_version 1897802 (0.0005) [2023-12-27 18:47:44,082][20347] Updated weights for policy 0, policy_version 1897812 (0.0007) [2023-12-27 18:47:44,087][20381] Updated weights for policy 1, policy_version 1896199 (0.0009) [2023-12-27 18:47:44,142][20381] Updated weights for policy 1, policy_version 1896209 (0.0006) [2023-12-27 18:47:44,143][20347] Updated weights for policy 0, policy_version 1897822 (0.0008) [2023-12-27 18:47:44,199][20381] Updated weights for policy 1, policy_version 1896219 (0.0007) [2023-12-27 18:47:44,793][20381] Updated weights for policy 1, policy_version 1896229 (0.0008) [2023-12-27 18:47:44,848][20381] Updated weights for policy 1, policy_version 1896239 (0.0007) [2023-12-27 18:47:44,863][20347] Updated weights for policy 0, policy_version 1897832 (0.0008) [2023-12-27 18:47:44,906][20381] Updated weights for policy 1, policy_version 1896249 (0.0006) [2023-12-27 18:47:44,926][20347] Updated weights for policy 0, policy_version 1897842 (0.0007) [2023-12-27 18:47:44,992][20347] Updated weights for policy 0, policy_version 1897852 (0.0008) [2023-12-27 18:47:45,544][20381] Updated weights for policy 1, policy_version 1896259 (0.0007) [2023-12-27 18:47:45,603][20381] Updated weights for policy 1, policy_version 1896269 (0.0005) [2023-12-27 18:47:45,665][20381] Updated weights for policy 1, policy_version 1896279 (0.0008) [2023-12-27 18:47:45,710][20347] Updated weights for policy 0, policy_version 1897862 (0.0006) [2023-12-27 18:47:45,766][20347] Updated weights for policy 0, policy_version 1897872 (0.0007) [2023-12-27 18:47:45,824][20347] Updated weights for policy 0, policy_version 1897882 (0.0008) [2023-12-27 18:47:46,331][20381] Updated weights for policy 1, policy_version 1896289 (0.0006) [2023-12-27 18:47:46,396][20381] Updated weights for policy 1, policy_version 1896299 (0.0008) [2023-12-27 18:47:46,451][20347] Updated weights for policy 0, policy_version 1897892 (0.0007) [2023-12-27 18:47:46,456][20381] Updated weights for policy 1, policy_version 1896309 (0.0008) [2023-12-27 18:47:46,506][20347] Updated weights for policy 0, policy_version 1897902 (0.0005) [2023-12-27 18:47:46,514][20381] Updated weights for policy 1, policy_version 1896319 (0.0006) [2023-12-27 18:47:46,564][20347] Updated weights for policy 0, policy_version 1897912 (0.0005) [2023-12-27 18:47:47,180][20381] Updated weights for policy 1, policy_version 1896329 (0.0006) [2023-12-27 18:47:47,197][20347] Updated weights for policy 0, policy_version 1897922 (0.0007) [2023-12-27 18:47:47,239][20381] Updated weights for policy 1, policy_version 1896339 (0.0005) [2023-12-27 18:47:47,250][20347] Updated weights for policy 0, policy_version 1897932 (0.0007) [2023-12-27 18:47:47,299][20381] Updated weights for policy 1, policy_version 1896349 (0.0008) [2023-12-27 18:47:47,302][20347] Updated weights for policy 0, policy_version 1897942 (0.0005) [2023-12-27 18:47:47,360][20347] Updated weights for policy 0, policy_version 1897952 (0.0008) [2023-12-27 18:47:47,362][19170] Fps is (10 sec: 21299.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 971489280. Throughput: 0: 10362.4, 1: 10177.7. Samples: 971492756. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:47,363][19170] Avg episode reward: [(0, '980.500'), (1, '950.480')] [2023-12-27 18:47:47,932][20381] Updated weights for policy 1, policy_version 1896359 (0.0010) [2023-12-27 18:47:47,990][20381] Updated weights for policy 1, policy_version 1896369 (0.0010) [2023-12-27 18:47:47,992][20347] Updated weights for policy 0, policy_version 1897962 (0.0005) [2023-12-27 18:47:48,047][20347] Updated weights for policy 0, policy_version 1897972 (0.0005) [2023-12-27 18:47:48,049][20381] Updated weights for policy 1, policy_version 1896379 (0.0010) [2023-12-27 18:47:48,095][20347] Updated weights for policy 0, policy_version 1897982 (0.0006) [2023-12-27 18:47:48,722][20347] Updated weights for policy 0, policy_version 1897992 (0.0005) [2023-12-27 18:47:48,779][20381] Updated weights for policy 1, policy_version 1896389 (0.0008) [2023-12-27 18:47:48,783][20347] Updated weights for policy 0, policy_version 1898002 (0.0006) [2023-12-27 18:47:48,838][20381] Updated weights for policy 1, policy_version 1896399 (0.0006) [2023-12-27 18:47:48,846][20347] Updated weights for policy 0, policy_version 1898012 (0.0008) [2023-12-27 18:47:48,897][20381] Updated weights for policy 1, policy_version 1896409 (0.0009) [2023-12-27 18:47:49,473][20347] Updated weights for policy 0, policy_version 1898022 (0.0009) [2023-12-27 18:47:49,530][20347] Updated weights for policy 0, policy_version 1898032 (0.0010) [2023-12-27 18:47:49,556][20381] Updated weights for policy 1, policy_version 1896419 (0.0009) [2023-12-27 18:47:49,579][20347] Updated weights for policy 0, policy_version 1898042 (0.0010) [2023-12-27 18:47:49,622][20381] Updated weights for policy 1, policy_version 1896429 (0.0011) [2023-12-27 18:47:49,677][20381] Updated weights for policy 1, policy_version 1896439 (0.0010) [2023-12-27 18:47:50,307][20347] Updated weights for policy 0, policy_version 1898052 (0.0010) [2023-12-27 18:47:50,342][20381] Updated weights for policy 1, policy_version 1896449 (0.0010) [2023-12-27 18:47:50,367][20347] Updated weights for policy 0, policy_version 1898062 (0.0009) [2023-12-27 18:47:50,399][20381] Updated weights for policy 1, policy_version 1896459 (0.0010) [2023-12-27 18:47:50,421][20347] Updated weights for policy 0, policy_version 1898072 (0.0006) [2023-12-27 18:47:50,459][20381] Updated weights for policy 1, policy_version 1896469 (0.0010) [2023-12-27 18:47:50,513][20381] Updated weights for policy 1, policy_version 1896479 (0.0010) [2023-12-27 18:47:51,004][20347] Updated weights for policy 0, policy_version 1898082 (0.0005) [2023-12-27 18:47:51,072][20347] Updated weights for policy 0, policy_version 1898092 (0.0007) [2023-12-27 18:47:51,129][20347] Updated weights for policy 0, policy_version 1898102 (0.0006) [2023-12-27 18:47:51,193][20347] Updated weights for policy 0, policy_version 1898112 (0.0006) [2023-12-27 18:47:51,214][20381] Updated weights for policy 1, policy_version 1896489 (0.0009) [2023-12-27 18:47:51,280][20381] Updated weights for policy 1, policy_version 1896499 (0.0007) [2023-12-27 18:47:51,338][20381] Updated weights for policy 1, policy_version 1896509 (0.0007) [2023-12-27 18:47:51,903][20347] Updated weights for policy 0, policy_version 1898122 (0.0007) [2023-12-27 18:47:51,958][20347] Updated weights for policy 0, policy_version 1898132 (0.0008) [2023-12-27 18:47:52,011][20347] Updated weights for policy 0, policy_version 1898142 (0.0006) [2023-12-27 18:47:52,042][20381] Updated weights for policy 1, policy_version 1896519 (0.0010) [2023-12-27 18:47:52,102][20381] Updated weights for policy 1, policy_version 1896529 (0.0010) [2023-12-27 18:47:52,167][20381] Updated weights for policy 1, policy_version 1896539 (0.0008) [2023-12-27 18:47:52,363][19170] Fps is (10 sec: 21298.7, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 971587584. Throughput: 0: 10357.7, 1: 10203.3. Samples: 971554788. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:52,363][19170] Avg episode reward: [(0, '990.250'), (1, '920.790')] [2023-12-27 18:47:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001896544_485572608.pth... [2023-12-27 18:47:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001898144_486014976.pth... [2023-12-27 18:47:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001895360_485269504.pth [2023-12-27 18:47:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001896928_485703680.pth [2023-12-27 18:47:52,737][20347] Updated weights for policy 0, policy_version 1898152 (0.0008) [2023-12-27 18:47:52,794][20347] Updated weights for policy 0, policy_version 1898162 (0.0007) [2023-12-27 18:47:52,840][20381] Updated weights for policy 1, policy_version 1896549 (0.0010) [2023-12-27 18:47:52,845][20347] Updated weights for policy 0, policy_version 1898172 (0.0006) [2023-12-27 18:47:52,893][20381] Updated weights for policy 1, policy_version 1896559 (0.0005) [2023-12-27 18:47:52,946][20381] Updated weights for policy 1, policy_version 1896569 (0.0005) [2023-12-27 18:47:53,565][20381] Updated weights for policy 1, policy_version 1896579 (0.0008) [2023-12-27 18:47:53,582][20347] Updated weights for policy 0, policy_version 1898182 (0.0006) [2023-12-27 18:47:53,617][20381] Updated weights for policy 1, policy_version 1896589 (0.0010) [2023-12-27 18:47:53,640][20347] Updated weights for policy 0, policy_version 1898192 (0.0006) [2023-12-27 18:47:53,669][20381] Updated weights for policy 1, policy_version 1896599 (0.0010) [2023-12-27 18:47:53,691][20347] Updated weights for policy 0, policy_version 1898202 (0.0006) [2023-12-27 18:47:54,331][20347] Updated weights for policy 0, policy_version 1898212 (0.0005) [2023-12-27 18:47:54,353][20381] Updated weights for policy 1, policy_version 1896609 (0.0011) [2023-12-27 18:47:54,382][20347] Updated weights for policy 0, policy_version 1898222 (0.0005) [2023-12-27 18:47:54,411][20381] Updated weights for policy 1, policy_version 1896619 (0.0011) [2023-12-27 18:47:54,437][20347] Updated weights for policy 0, policy_version 1898232 (0.0005) [2023-12-27 18:47:54,463][20381] Updated weights for policy 1, policy_version 1896629 (0.0011) [2023-12-27 18:47:54,518][20381] Updated weights for policy 1, policy_version 1896639 (0.0010) [2023-12-27 18:47:55,074][20347] Updated weights for policy 0, policy_version 1898242 (0.0007) [2023-12-27 18:47:55,138][20347] Updated weights for policy 0, policy_version 1898252 (0.0008) [2023-12-27 18:47:55,191][20381] Updated weights for policy 1, policy_version 1896649 (0.0006) [2023-12-27 18:47:55,195][20347] Updated weights for policy 0, policy_version 1898262 (0.0008) [2023-12-27 18:47:55,246][20381] Updated weights for policy 1, policy_version 1896659 (0.0008) [2023-12-27 18:47:55,248][20347] Updated weights for policy 0, policy_version 1898272 (0.0006) [2023-12-27 18:47:55,309][20381] Updated weights for policy 1, policy_version 1896669 (0.0011) [2023-12-27 18:47:55,882][20347] Updated weights for policy 0, policy_version 1898282 (0.0009) [2023-12-27 18:47:55,933][20347] Updated weights for policy 0, policy_version 1898292 (0.0010) [2023-12-27 18:47:55,958][20381] Updated weights for policy 1, policy_version 1896679 (0.0007) [2023-12-27 18:47:55,986][20347] Updated weights for policy 0, policy_version 1898302 (0.0010) [2023-12-27 18:47:56,014][20381] Updated weights for policy 1, policy_version 1896689 (0.0006) [2023-12-27 18:47:56,069][20381] Updated weights for policy 1, policy_version 1896699 (0.0007) [2023-12-27 18:47:56,595][20347] Updated weights for policy 0, policy_version 1898312 (0.0006) [2023-12-27 18:47:56,653][20347] Updated weights for policy 0, policy_version 1898322 (0.0005) [2023-12-27 18:47:56,703][20347] Updated weights for policy 0, policy_version 1898332 (0.0005) [2023-12-27 18:47:56,729][20381] Updated weights for policy 1, policy_version 1896709 (0.0009) [2023-12-27 18:47:56,777][20381] Updated weights for policy 1, policy_version 1896719 (0.0010) [2023-12-27 18:47:56,822][20381] Updated weights for policy 1, policy_version 1896729 (0.0008) [2023-12-27 18:47:57,240][20347] Updated weights for policy 0, policy_version 1898342 (0.0005) [2023-12-27 18:47:57,286][20347] Updated weights for policy 0, policy_version 1898352 (0.0005) [2023-12-27 18:47:57,335][20347] Updated weights for policy 0, policy_version 1898362 (0.0008) [2023-12-27 18:47:57,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 971685888. Throughput: 0: 10321.8, 1: 10282.6. Samples: 971678208. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:47:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.280')] [2023-12-27 18:47:57,523][20381] Updated weights for policy 1, policy_version 1896739 (0.0008) [2023-12-27 18:47:57,580][20381] Updated weights for policy 1, policy_version 1896749 (0.0010) [2023-12-27 18:47:57,637][20381] Updated weights for policy 1, policy_version 1896759 (0.0010) [2023-12-27 18:47:57,993][20347] Updated weights for policy 0, policy_version 1898372 (0.0010) [2023-12-27 18:47:58,045][20347] Updated weights for policy 0, policy_version 1898382 (0.0010) [2023-12-27 18:47:58,102][20347] Updated weights for policy 0, policy_version 1898392 (0.0010) [2023-12-27 18:47:58,247][20381] Updated weights for policy 1, policy_version 1896769 (0.0010) [2023-12-27 18:47:58,305][20381] Updated weights for policy 1, policy_version 1896779 (0.0008) [2023-12-27 18:47:58,376][20381] Updated weights for policy 1, policy_version 1896790 (0.0008) [2023-12-27 18:47:58,438][20381] Updated weights for policy 1, policy_version 1896800 (0.0007) [2023-12-27 18:47:58,880][20347] Updated weights for policy 0, policy_version 1898402 (0.0009) [2023-12-27 18:47:58,944][20347] Updated weights for policy 0, policy_version 1898412 (0.0008) [2023-12-27 18:47:59,008][20347] Updated weights for policy 0, policy_version 1898422 (0.0007) [2023-12-27 18:47:59,067][20347] Updated weights for policy 0, policy_version 1898432 (0.0006) [2023-12-27 18:47:59,195][20381] Updated weights for policy 1, policy_version 1896810 (0.0009) [2023-12-27 18:47:59,262][20381] Updated weights for policy 1, policy_version 1896820 (0.0008) [2023-12-27 18:47:59,316][20381] Updated weights for policy 1, policy_version 1896830 (0.0009) [2023-12-27 18:47:59,705][20347] Updated weights for policy 0, policy_version 1898442 (0.0007) [2023-12-27 18:47:59,755][20347] Updated weights for policy 0, policy_version 1898452 (0.0007) [2023-12-27 18:47:59,804][20347] Updated weights for policy 0, policy_version 1898462 (0.0008) [2023-12-27 18:48:00,097][20381] Updated weights for policy 1, policy_version 1896840 (0.0008) [2023-12-27 18:48:00,155][20381] Updated weights for policy 1, policy_version 1896850 (0.0008) [2023-12-27 18:48:00,209][20381] Updated weights for policy 1, policy_version 1896860 (0.0007) [2023-12-27 18:48:00,501][20347] Updated weights for policy 0, policy_version 1898472 (0.0008) [2023-12-27 18:48:00,554][20347] Updated weights for policy 0, policy_version 1898482 (0.0007) [2023-12-27 18:48:00,611][20347] Updated weights for policy 0, policy_version 1898492 (0.0007) [2023-12-27 18:48:00,902][20381] Updated weights for policy 1, policy_version 1896870 (0.0008) [2023-12-27 18:48:00,961][20381] Updated weights for policy 1, policy_version 1896880 (0.0008) [2023-12-27 18:48:01,020][20381] Updated weights for policy 1, policy_version 1896890 (0.0006) [2023-12-27 18:48:01,227][20347] Updated weights for policy 0, policy_version 1898502 (0.0006) [2023-12-27 18:48:01,287][20347] Updated weights for policy 0, policy_version 1898512 (0.0008) [2023-12-27 18:48:01,354][20347] Updated weights for policy 0, policy_version 1898522 (0.0006) [2023-12-27 18:48:01,739][20381] Updated weights for policy 1, policy_version 1896900 (0.0008) [2023-12-27 18:48:01,796][20381] Updated weights for policy 1, policy_version 1896910 (0.0006) [2023-12-27 18:48:01,849][20381] Updated weights for policy 1, policy_version 1896920 (0.0008) [2023-12-27 18:48:02,025][20347] Updated weights for policy 0, policy_version 1898532 (0.0008) [2023-12-27 18:48:02,083][20347] Updated weights for policy 0, policy_version 1898542 (0.0007) [2023-12-27 18:48:02,129][20347] Updated weights for policy 0, policy_version 1898552 (0.0008) [2023-12-27 18:48:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 971792384. Throughput: 0: 10341.0, 1: 10201.4. Samples: 971801572. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.050')] [2023-12-27 18:48:02,565][20381] Updated weights for policy 1, policy_version 1896930 (0.0008) [2023-12-27 18:48:02,630][20381] Updated weights for policy 1, policy_version 1896940 (0.0008) [2023-12-27 18:48:02,697][20381] Updated weights for policy 1, policy_version 1896950 (0.0008) [2023-12-27 18:48:02,762][20381] Updated weights for policy 1, policy_version 1896960 (0.0007) [2023-12-27 18:48:02,811][20347] Updated weights for policy 0, policy_version 1898562 (0.0007) [2023-12-27 18:48:02,867][20347] Updated weights for policy 0, policy_version 1898572 (0.0006) [2023-12-27 18:48:02,911][20347] Updated weights for policy 0, policy_version 1898582 (0.0006) [2023-12-27 18:48:02,960][20347] Updated weights for policy 0, policy_version 1898592 (0.0006) [2023-12-27 18:48:03,362][20381] Updated weights for policy 1, policy_version 1896970 (0.0009) [2023-12-27 18:48:03,422][20381] Updated weights for policy 1, policy_version 1896980 (0.0010) [2023-12-27 18:48:03,479][20381] Updated weights for policy 1, policy_version 1896990 (0.0010) [2023-12-27 18:48:03,559][20347] Updated weights for policy 0, policy_version 1898602 (0.0007) [2023-12-27 18:48:03,614][20347] Updated weights for policy 0, policy_version 1898612 (0.0008) [2023-12-27 18:48:03,680][20347] Updated weights for policy 0, policy_version 1898622 (0.0008) [2023-12-27 18:48:04,107][20381] Updated weights for policy 1, policy_version 1897000 (0.0011) [2023-12-27 18:48:04,164][20381] Updated weights for policy 1, policy_version 1897010 (0.0011) [2023-12-27 18:48:04,224][20381] Updated weights for policy 1, policy_version 1897020 (0.0011) [2023-12-27 18:48:04,341][20347] Updated weights for policy 0, policy_version 1898632 (0.0006) [2023-12-27 18:48:04,404][20347] Updated weights for policy 0, policy_version 1898642 (0.0006) [2023-12-27 18:48:04,456][20347] Updated weights for policy 0, policy_version 1898652 (0.0007) [2023-12-27 18:48:04,899][20381] Updated weights for policy 1, policy_version 1897030 (0.0011) [2023-12-27 18:48:04,954][20381] Updated weights for policy 1, policy_version 1897040 (0.0011) [2023-12-27 18:48:05,004][20381] Updated weights for policy 1, policy_version 1897050 (0.0011) [2023-12-27 18:48:05,091][20347] Updated weights for policy 0, policy_version 1898662 (0.0006) [2023-12-27 18:48:05,152][20347] Updated weights for policy 0, policy_version 1898672 (0.0007) [2023-12-27 18:48:05,212][20347] Updated weights for policy 0, policy_version 1898682 (0.0007) [2023-12-27 18:48:05,694][20381] Updated weights for policy 1, policy_version 1897060 (0.0011) [2023-12-27 18:48:05,745][20381] Updated weights for policy 1, policy_version 1897070 (0.0010) [2023-12-27 18:48:05,797][20381] Updated weights for policy 1, policy_version 1897080 (0.0011) [2023-12-27 18:48:05,883][20347] Updated weights for policy 0, policy_version 1898692 (0.0006) [2023-12-27 18:48:05,949][20347] Updated weights for policy 0, policy_version 1898702 (0.0005) [2023-12-27 18:48:06,005][20347] Updated weights for policy 0, policy_version 1898712 (0.0005) [2023-12-27 18:48:06,537][20381] Updated weights for policy 1, policy_version 1897090 (0.0010) [2023-12-27 18:48:06,591][20381] Updated weights for policy 1, policy_version 1897100 (0.0009) [2023-12-27 18:48:06,594][20347] Updated weights for policy 0, policy_version 1898722 (0.0006) [2023-12-27 18:48:06,654][20381] Updated weights for policy 1, policy_version 1897110 (0.0008) [2023-12-27 18:48:06,657][20347] Updated weights for policy 0, policy_version 1898732 (0.0007) [2023-12-27 18:48:06,724][20381] Updated weights for policy 1, policy_version 1897120 (0.0006) [2023-12-27 18:48:06,726][20347] Updated weights for policy 0, policy_version 1898742 (0.0008) [2023-12-27 18:48:06,787][20347] Updated weights for policy 0, policy_version 1898752 (0.0007) [2023-12-27 18:48:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 971890688. Throughput: 0: 10340.2, 1: 10218.7. Samples: 971864868. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.790')] [2023-12-27 18:48:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001898752_486170624.pth... [2023-12-27 18:48:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001897568_485867520.pth [2023-12-27 18:48:07,393][20381] Updated weights for policy 1, policy_version 1897130 (0.0008) [2023-12-27 18:48:07,450][20347] Updated weights for policy 0, policy_version 1898762 (0.0006) [2023-12-27 18:48:07,452][20381] Updated weights for policy 1, policy_version 1897140 (0.0008) [2023-12-27 18:48:07,496][20347] Updated weights for policy 0, policy_version 1898772 (0.0005) [2023-12-27 18:48:07,509][20381] Updated weights for policy 1, policy_version 1897150 (0.0008) [2023-12-27 18:48:07,519][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001897152_485728256.pth... [2023-12-27 18:48:07,523][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001895936_485416960.pth [2023-12-27 18:48:07,549][20347] Updated weights for policy 0, policy_version 1898782 (0.0006) [2023-12-27 18:48:08,154][20381] Updated weights for policy 1, policy_version 1897160 (0.0010) [2023-12-27 18:48:08,200][20347] Updated weights for policy 0, policy_version 1898792 (0.0006) [2023-12-27 18:48:08,204][20381] Updated weights for policy 1, policy_version 1897170 (0.0010) [2023-12-27 18:48:08,257][20347] Updated weights for policy 0, policy_version 1898802 (0.0007) [2023-12-27 18:48:08,262][20381] Updated weights for policy 1, policy_version 1897180 (0.0006) [2023-12-27 18:48:08,309][20347] Updated weights for policy 0, policy_version 1898812 (0.0006) [2023-12-27 18:48:08,929][20381] Updated weights for policy 1, policy_version 1897190 (0.0006) [2023-12-27 18:48:08,945][20347] Updated weights for policy 0, policy_version 1898822 (0.0008) [2023-12-27 18:48:08,981][20381] Updated weights for policy 1, policy_version 1897200 (0.0006) [2023-12-27 18:48:08,994][20347] Updated weights for policy 0, policy_version 1898832 (0.0006) [2023-12-27 18:48:09,041][20381] Updated weights for policy 1, policy_version 1897210 (0.0007) [2023-12-27 18:48:09,053][20347] Updated weights for policy 0, policy_version 1898842 (0.0006) [2023-12-27 18:48:09,694][20381] Updated weights for policy 1, policy_version 1897220 (0.0007) [2023-12-27 18:48:09,719][20347] Updated weights for policy 0, policy_version 1898852 (0.0007) [2023-12-27 18:48:09,758][20381] Updated weights for policy 1, policy_version 1897230 (0.0006) [2023-12-27 18:48:09,783][20347] Updated weights for policy 0, policy_version 1898862 (0.0010) [2023-12-27 18:48:09,812][20381] Updated weights for policy 1, policy_version 1897240 (0.0006) [2023-12-27 18:48:09,847][20347] Updated weights for policy 0, policy_version 1898872 (0.0010) [2023-12-27 18:48:10,460][20381] Updated weights for policy 1, policy_version 1897250 (0.0008) [2023-12-27 18:48:10,522][20381] Updated weights for policy 1, policy_version 1897260 (0.0006) [2023-12-27 18:48:10,557][20347] Updated weights for policy 0, policy_version 1898882 (0.0008) [2023-12-27 18:48:10,583][20381] Updated weights for policy 1, policy_version 1897270 (0.0006) [2023-12-27 18:48:10,625][20347] Updated weights for policy 0, policy_version 1898892 (0.0007) [2023-12-27 18:48:10,647][20381] Updated weights for policy 1, policy_version 1897280 (0.0006) [2023-12-27 18:48:10,684][20347] Updated weights for policy 0, policy_version 1898902 (0.0009) [2023-12-27 18:48:10,730][20347] Updated weights for policy 0, policy_version 1898912 (0.0010) [2023-12-27 18:48:11,285][20381] Updated weights for policy 1, policy_version 1897290 (0.0007) [2023-12-27 18:48:11,337][20381] Updated weights for policy 1, policy_version 1897300 (0.0007) [2023-12-27 18:48:11,403][20381] Updated weights for policy 1, policy_version 1897310 (0.0008) [2023-12-27 18:48:11,443][20347] Updated weights for policy 0, policy_version 1898922 (0.0007) [2023-12-27 18:48:11,512][20347] Updated weights for policy 0, policy_version 1898932 (0.0008) [2023-12-27 18:48:11,574][20347] Updated weights for policy 0, policy_version 1898942 (0.0005) [2023-12-27 18:48:12,128][20381] Updated weights for policy 1, policy_version 1897320 (0.0008) [2023-12-27 18:48:12,185][20381] Updated weights for policy 1, policy_version 1897330 (0.0005) [2023-12-27 18:48:12,242][20381] Updated weights for policy 1, policy_version 1897340 (0.0006) [2023-12-27 18:48:12,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 971997184. Throughput: 0: 10335.8, 1: 10262.4. Samples: 971989084. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.920')] [2023-12-27 18:48:12,402][20347] Updated weights for policy 0, policy_version 1898952 (0.0008) [2023-12-27 18:48:12,456][20347] Updated weights for policy 0, policy_version 1898962 (0.0005) [2023-12-27 18:48:12,515][20347] Updated weights for policy 0, policy_version 1898972 (0.0010) [2023-12-27 18:48:12,968][20381] Updated weights for policy 1, policy_version 1897350 (0.0008) [2023-12-27 18:48:13,022][20381] Updated weights for policy 1, policy_version 1897360 (0.0007) [2023-12-27 18:48:13,074][20381] Updated weights for policy 1, policy_version 1897370 (0.0007) [2023-12-27 18:48:13,217][20347] Updated weights for policy 0, policy_version 1898982 (0.0010) [2023-12-27 18:48:13,265][20347] Updated weights for policy 0, policy_version 1898992 (0.0010) [2023-12-27 18:48:13,310][20347] Updated weights for policy 0, policy_version 1899002 (0.0010) [2023-12-27 18:48:13,727][20381] Updated weights for policy 1, policy_version 1897380 (0.0006) [2023-12-27 18:48:13,778][20381] Updated weights for policy 1, policy_version 1897390 (0.0005) [2023-12-27 18:48:13,829][20381] Updated weights for policy 1, policy_version 1897400 (0.0006) [2023-12-27 18:48:13,982][20347] Updated weights for policy 0, policy_version 1899012 (0.0010) [2023-12-27 18:48:14,038][20347] Updated weights for policy 0, policy_version 1899022 (0.0010) [2023-12-27 18:48:14,095][20347] Updated weights for policy 0, policy_version 1899032 (0.0010) [2023-12-27 18:48:14,471][20381] Updated weights for policy 1, policy_version 1897410 (0.0007) [2023-12-27 18:48:14,526][20381] Updated weights for policy 1, policy_version 1897420 (0.0009) [2023-12-27 18:48:14,574][20381] Updated weights for policy 1, policy_version 1897430 (0.0009) [2023-12-27 18:48:14,622][20381] Updated weights for policy 1, policy_version 1897440 (0.0008) [2023-12-27 18:48:14,711][20347] Updated weights for policy 0, policy_version 1899042 (0.0009) [2023-12-27 18:48:14,759][20347] Updated weights for policy 0, policy_version 1899052 (0.0006) [2023-12-27 18:48:14,821][20347] Updated weights for policy 0, policy_version 1899062 (0.0009) [2023-12-27 18:48:14,880][20347] Updated weights for policy 0, policy_version 1899072 (0.0008) [2023-12-27 18:48:15,305][20381] Updated weights for policy 1, policy_version 1897450 (0.0007) [2023-12-27 18:48:15,368][20381] Updated weights for policy 1, policy_version 1897460 (0.0007) [2023-12-27 18:48:15,432][20381] Updated weights for policy 1, policy_version 1897470 (0.0008) [2023-12-27 18:48:15,577][20347] Updated weights for policy 0, policy_version 1899082 (0.0008) [2023-12-27 18:48:15,637][20347] Updated weights for policy 0, policy_version 1899092 (0.0007) [2023-12-27 18:48:15,700][20347] Updated weights for policy 0, policy_version 1899102 (0.0006) [2023-12-27 18:48:15,974][20381] Updated weights for policy 1, policy_version 1897480 (0.0007) [2023-12-27 18:48:16,023][20381] Updated weights for policy 1, policy_version 1897490 (0.0006) [2023-12-27 18:48:16,069][20381] Updated weights for policy 1, policy_version 1897500 (0.0005) [2023-12-27 18:48:16,374][20347] Updated weights for policy 0, policy_version 1899112 (0.0010) [2023-12-27 18:48:16,426][20347] Updated weights for policy 0, policy_version 1899122 (0.0010) [2023-12-27 18:48:16,472][20347] Updated weights for policy 0, policy_version 1899132 (0.0010) [2023-12-27 18:48:16,702][20381] Updated weights for policy 1, policy_version 1897510 (0.0009) [2023-12-27 18:48:16,763][20381] Updated weights for policy 1, policy_version 1897520 (0.0010) [2023-12-27 18:48:16,818][20381] Updated weights for policy 1, policy_version 1897530 (0.0010) [2023-12-27 18:48:17,140][20347] Updated weights for policy 0, policy_version 1899142 (0.0009) [2023-12-27 18:48:17,196][20347] Updated weights for policy 0, policy_version 1899152 (0.0008) [2023-12-27 18:48:17,249][20347] Updated weights for policy 0, policy_version 1899162 (0.0009) [2023-12-27 18:48:17,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 972103680. Throughput: 0: 10320.5, 1: 10239.7. Samples: 972112448. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.270')] [2023-12-27 18:48:17,449][20381] Updated weights for policy 1, policy_version 1897540 (0.0008) [2023-12-27 18:48:17,501][20381] Updated weights for policy 1, policy_version 1897550 (0.0005) [2023-12-27 18:48:17,555][20381] Updated weights for policy 1, policy_version 1897560 (0.0005) [2023-12-27 18:48:17,942][20347] Updated weights for policy 0, policy_version 1899172 (0.0010) [2023-12-27 18:48:17,996][20347] Updated weights for policy 0, policy_version 1899182 (0.0010) [2023-12-27 18:48:18,047][20347] Updated weights for policy 0, policy_version 1899192 (0.0010) [2023-12-27 18:48:18,135][20381] Updated weights for policy 1, policy_version 1897570 (0.0008) [2023-12-27 18:48:18,182][20381] Updated weights for policy 1, policy_version 1897580 (0.0007) [2023-12-27 18:48:18,233][20381] Updated weights for policy 1, policy_version 1897590 (0.0007) [2023-12-27 18:48:18,276][20381] Updated weights for policy 1, policy_version 1897600 (0.0009) [2023-12-27 18:48:18,695][20347] Updated weights for policy 0, policy_version 1899202 (0.0007) [2023-12-27 18:48:18,756][20347] Updated weights for policy 0, policy_version 1899212 (0.0010) [2023-12-27 18:48:18,815][20347] Updated weights for policy 0, policy_version 1899222 (0.0010) [2023-12-27 18:48:18,873][20347] Updated weights for policy 0, policy_version 1899232 (0.0010) [2023-12-27 18:48:18,947][20381] Updated weights for policy 1, policy_version 1897610 (0.0007) [2023-12-27 18:48:18,995][20381] Updated weights for policy 1, policy_version 1897620 (0.0007) [2023-12-27 18:48:19,049][20381] Updated weights for policy 1, policy_version 1897630 (0.0007) [2023-12-27 18:48:19,523][20347] Updated weights for policy 0, policy_version 1899242 (0.0009) [2023-12-27 18:48:19,593][20347] Updated weights for policy 0, policy_version 1899252 (0.0008) [2023-12-27 18:48:19,652][20347] Updated weights for policy 0, policy_version 1899262 (0.0011) [2023-12-27 18:48:19,762][20381] Updated weights for policy 1, policy_version 1897640 (0.0010) [2023-12-27 18:48:19,823][20381] Updated weights for policy 1, policy_version 1897650 (0.0010) [2023-12-27 18:48:19,886][20381] Updated weights for policy 1, policy_version 1897660 (0.0008) [2023-12-27 18:48:20,360][20347] Updated weights for policy 0, policy_version 1899272 (0.0010) [2023-12-27 18:48:20,422][20347] Updated weights for policy 0, policy_version 1899282 (0.0006) [2023-12-27 18:48:20,474][20347] Updated weights for policy 0, policy_version 1899292 (0.0006) [2023-12-27 18:48:20,530][20381] Updated weights for policy 1, policy_version 1897670 (0.0008) [2023-12-27 18:48:20,594][20381] Updated weights for policy 1, policy_version 1897680 (0.0008) [2023-12-27 18:48:20,657][20381] Updated weights for policy 1, policy_version 1897690 (0.0008) [2023-12-27 18:48:21,088][20347] Updated weights for policy 0, policy_version 1899302 (0.0008) [2023-12-27 18:48:21,149][20347] Updated weights for policy 0, policy_version 1899312 (0.0008) [2023-12-27 18:48:21,213][20347] Updated weights for policy 0, policy_version 1899322 (0.0008) [2023-12-27 18:48:21,396][20381] Updated weights for policy 1, policy_version 1897700 (0.0009) [2023-12-27 18:48:21,457][20381] Updated weights for policy 1, policy_version 1897710 (0.0011) [2023-12-27 18:48:21,513][20381] Updated weights for policy 1, policy_version 1897720 (0.0011) [2023-12-27 18:48:22,004][20347] Updated weights for policy 0, policy_version 1899332 (0.0008) [2023-12-27 18:48:22,069][20347] Updated weights for policy 0, policy_version 1899342 (0.0007) [2023-12-27 18:48:22,130][20347] Updated weights for policy 0, policy_version 1899352 (0.0008) [2023-12-27 18:48:22,162][20381] Updated weights for policy 1, policy_version 1897730 (0.0010) [2023-12-27 18:48:22,221][20381] Updated weights for policy 1, policy_version 1897740 (0.0005) [2023-12-27 18:48:22,280][20381] Updated weights for policy 1, policy_version 1897750 (0.0008) [2023-12-27 18:48:22,340][20381] Updated weights for policy 1, policy_version 1897760 (0.0011) [2023-12-27 18:48:22,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 972210176. Throughput: 0: 10315.4, 1: 10267.9. Samples: 972176084. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.660')] [2023-12-27 18:48:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001897760_485883904.pth... [2023-12-27 18:48:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001899360_486326272.pth... [2023-12-27 18:48:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001896544_485572608.pth [2023-12-27 18:48:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001898144_486014976.pth [2023-12-27 18:48:22,883][20347] Updated weights for policy 0, policy_version 1899362 (0.0007) [2023-12-27 18:48:22,931][20347] Updated weights for policy 0, policy_version 1899372 (0.0006) [2023-12-27 18:48:22,991][20347] Updated weights for policy 0, policy_version 1899382 (0.0005) [2023-12-27 18:48:23,044][20347] Updated weights for policy 0, policy_version 1899392 (0.0005) [2023-12-27 18:48:23,067][20381] Updated weights for policy 1, policy_version 1897770 (0.0007) [2023-12-27 18:48:23,124][20381] Updated weights for policy 1, policy_version 1897780 (0.0007) [2023-12-27 18:48:23,180][20381] Updated weights for policy 1, policy_version 1897790 (0.0009) [2023-12-27 18:48:23,638][20347] Updated weights for policy 0, policy_version 1899402 (0.0008) [2023-12-27 18:48:23,697][20347] Updated weights for policy 0, policy_version 1899412 (0.0008) [2023-12-27 18:48:23,753][20347] Updated weights for policy 0, policy_version 1899422 (0.0007) [2023-12-27 18:48:23,879][20381] Updated weights for policy 1, policy_version 1897800 (0.0009) [2023-12-27 18:48:23,930][20381] Updated weights for policy 1, policy_version 1897810 (0.0009) [2023-12-27 18:48:23,987][20381] Updated weights for policy 1, policy_version 1897820 (0.0008) [2023-12-27 18:48:24,330][20347] Updated weights for policy 0, policy_version 1899432 (0.0006) [2023-12-27 18:48:24,383][20347] Updated weights for policy 0, policy_version 1899442 (0.0005) [2023-12-27 18:48:24,440][20347] Updated weights for policy 0, policy_version 1899452 (0.0006) [2023-12-27 18:48:24,738][20381] Updated weights for policy 1, policy_version 1897830 (0.0009) [2023-12-27 18:48:24,801][20381] Updated weights for policy 1, policy_version 1897840 (0.0008) [2023-12-27 18:48:24,848][20381] Updated weights for policy 1, policy_version 1897850 (0.0007) [2023-12-27 18:48:25,004][20347] Updated weights for policy 0, policy_version 1899462 (0.0006) [2023-12-27 18:48:25,055][20347] Updated weights for policy 0, policy_version 1899472 (0.0005) [2023-12-27 18:48:25,108][20347] Updated weights for policy 0, policy_version 1899482 (0.0005) [2023-12-27 18:48:25,522][20381] Updated weights for policy 1, policy_version 1897860 (0.0007) [2023-12-27 18:48:25,582][20381] Updated weights for policy 1, policy_version 1897870 (0.0008) [2023-12-27 18:48:25,638][20381] Updated weights for policy 1, policy_version 1897880 (0.0007) [2023-12-27 18:48:25,741][20347] Updated weights for policy 0, policy_version 1899492 (0.0008) [2023-12-27 18:48:25,790][20347] Updated weights for policy 0, policy_version 1899502 (0.0008) [2023-12-27 18:48:25,849][20347] Updated weights for policy 0, policy_version 1899512 (0.0008) [2023-12-27 18:48:26,335][20381] Updated weights for policy 1, policy_version 1897890 (0.0008) [2023-12-27 18:48:26,389][20381] Updated weights for policy 1, policy_version 1897900 (0.0005) [2023-12-27 18:48:26,441][20381] Updated weights for policy 1, policy_version 1897910 (0.0005) [2023-12-27 18:48:26,497][20381] Updated weights for policy 1, policy_version 1897920 (0.0005) [2023-12-27 18:48:26,564][20347] Updated weights for policy 0, policy_version 1899522 (0.0007) [2023-12-27 18:48:26,617][20347] Updated weights for policy 0, policy_version 1899532 (0.0005) [2023-12-27 18:48:26,673][20347] Updated weights for policy 0, policy_version 1899542 (0.0005) [2023-12-27 18:48:26,730][20347] Updated weights for policy 0, policy_version 1899552 (0.0006) [2023-12-27 18:48:27,035][20381] Updated weights for policy 1, policy_version 1897930 (0.0006) [2023-12-27 18:48:27,083][20381] Updated weights for policy 1, policy_version 1897940 (0.0005) [2023-12-27 18:48:27,131][20381] Updated weights for policy 1, policy_version 1897950 (0.0005) [2023-12-27 18:48:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 972308480. Throughput: 0: 10367.1, 1: 10268.6. Samples: 972298256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '960.780')] [2023-12-27 18:48:27,414][20347] Updated weights for policy 0, policy_version 1899562 (0.0005) [2023-12-27 18:48:27,467][20347] Updated weights for policy 0, policy_version 1899572 (0.0008) [2023-12-27 18:48:27,516][20347] Updated weights for policy 0, policy_version 1899582 (0.0008) [2023-12-27 18:48:27,686][20381] Updated weights for policy 1, policy_version 1897960 (0.0007) [2023-12-27 18:48:27,746][20381] Updated weights for policy 1, policy_version 1897970 (0.0008) [2023-12-27 18:48:27,803][20381] Updated weights for policy 1, policy_version 1897980 (0.0008) [2023-12-27 18:48:28,232][20347] Updated weights for policy 0, policy_version 1899592 (0.0008) [2023-12-27 18:48:28,290][20347] Updated weights for policy 0, policy_version 1899602 (0.0008) [2023-12-27 18:48:28,350][20347] Updated weights for policy 0, policy_version 1899612 (0.0008) [2023-12-27 18:48:28,427][20381] Updated weights for policy 1, policy_version 1897990 (0.0008) [2023-12-27 18:48:28,485][20381] Updated weights for policy 1, policy_version 1898000 (0.0008) [2023-12-27 18:48:28,538][20381] Updated weights for policy 1, policy_version 1898010 (0.0008) [2023-12-27 18:48:29,020][20347] Updated weights for policy 0, policy_version 1899622 (0.0007) [2023-12-27 18:48:29,076][20347] Updated weights for policy 0, policy_version 1899632 (0.0007) [2023-12-27 18:48:29,127][20347] Updated weights for policy 0, policy_version 1899642 (0.0007) [2023-12-27 18:48:29,212][20381] Updated weights for policy 1, policy_version 1898020 (0.0006) [2023-12-27 18:48:29,278][20381] Updated weights for policy 1, policy_version 1898030 (0.0007) [2023-12-27 18:48:29,331][20381] Updated weights for policy 1, policy_version 1898040 (0.0007) [2023-12-27 18:48:29,876][20347] Updated weights for policy 0, policy_version 1899652 (0.0008) [2023-12-27 18:48:29,914][20381] Updated weights for policy 1, policy_version 1898050 (0.0008) [2023-12-27 18:48:29,934][20347] Updated weights for policy 0, policy_version 1899662 (0.0007) [2023-12-27 18:48:29,973][20381] Updated weights for policy 1, policy_version 1898060 (0.0008) [2023-12-27 18:48:29,988][20347] Updated weights for policy 0, policy_version 1899672 (0.0007) [2023-12-27 18:48:30,024][20381] Updated weights for policy 1, policy_version 1898070 (0.0008) [2023-12-27 18:48:30,089][20381] Updated weights for policy 1, policy_version 1898080 (0.0008) [2023-12-27 18:48:30,731][20347] Updated weights for policy 0, policy_version 1899682 (0.0007) [2023-12-27 18:48:30,761][20381] Updated weights for policy 1, policy_version 1898090 (0.0006) [2023-12-27 18:48:30,778][20347] Updated weights for policy 0, policy_version 1899692 (0.0007) [2023-12-27 18:48:30,819][20381] Updated weights for policy 1, policy_version 1898100 (0.0006) [2023-12-27 18:48:30,836][20347] Updated weights for policy 0, policy_version 1899702 (0.0007) [2023-12-27 18:48:30,878][20381] Updated weights for policy 1, policy_version 1898110 (0.0007) [2023-12-27 18:48:30,889][20347] Updated weights for policy 0, policy_version 1899712 (0.0005) [2023-12-27 18:48:31,560][20347] Updated weights for policy 0, policy_version 1899722 (0.0008) [2023-12-27 18:48:31,566][20381] Updated weights for policy 1, policy_version 1898120 (0.0008) [2023-12-27 18:48:31,623][20381] Updated weights for policy 1, policy_version 1898130 (0.0007) [2023-12-27 18:48:31,624][20347] Updated weights for policy 0, policy_version 1899732 (0.0007) [2023-12-27 18:48:31,679][20381] Updated weights for policy 1, policy_version 1898140 (0.0007) [2023-12-27 18:48:31,688][20347] Updated weights for policy 0, policy_version 1899742 (0.0006) [2023-12-27 18:48:32,345][20347] Updated weights for policy 0, policy_version 1899752 (0.0007) [2023-12-27 18:48:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 972406784. Throughput: 0: 10334.8, 1: 10332.6. Samples: 972422792. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.350')] [2023-12-27 18:48:32,367][20381] Updated weights for policy 1, policy_version 1898150 (0.0007) [2023-12-27 18:48:32,403][20347] Updated weights for policy 0, policy_version 1899762 (0.0008) [2023-12-27 18:48:32,427][20381] Updated weights for policy 1, policy_version 1898160 (0.0006) [2023-12-27 18:48:32,461][20347] Updated weights for policy 0, policy_version 1899772 (0.0007) [2023-12-27 18:48:32,485][20381] Updated weights for policy 1, policy_version 1898170 (0.0007) [2023-12-27 18:48:33,098][20381] Updated weights for policy 1, policy_version 1898180 (0.0007) [2023-12-27 18:48:33,150][20381] Updated weights for policy 1, policy_version 1898190 (0.0008) [2023-12-27 18:48:33,194][20347] Updated weights for policy 0, policy_version 1899782 (0.0005) [2023-12-27 18:48:33,198][20381] Updated weights for policy 1, policy_version 1898200 (0.0006) [2023-12-27 18:48:33,246][20347] Updated weights for policy 0, policy_version 1899792 (0.0006) [2023-12-27 18:48:33,298][20347] Updated weights for policy 0, policy_version 1899802 (0.0008) [2023-12-27 18:48:33,868][20381] Updated weights for policy 1, policy_version 1898210 (0.0007) [2023-12-27 18:48:33,925][20381] Updated weights for policy 1, policy_version 1898220 (0.0008) [2023-12-27 18:48:33,981][20381] Updated weights for policy 1, policy_version 1898230 (0.0008) [2023-12-27 18:48:33,993][20347] Updated weights for policy 0, policy_version 1899812 (0.0008) [2023-12-27 18:48:34,031][20381] Updated weights for policy 1, policy_version 1898240 (0.0006) [2023-12-27 18:48:34,041][20347] Updated weights for policy 0, policy_version 1899822 (0.0006) [2023-12-27 18:48:34,090][20347] Updated weights for policy 0, policy_version 1899832 (0.0008) [2023-12-27 18:48:34,719][20381] Updated weights for policy 1, policy_version 1898250 (0.0008) [2023-12-27 18:48:34,768][20381] Updated weights for policy 1, policy_version 1898260 (0.0007) [2023-12-27 18:48:34,813][20381] Updated weights for policy 1, policy_version 1898270 (0.0007) [2023-12-27 18:48:34,822][20347] Updated weights for policy 0, policy_version 1899842 (0.0008) [2023-12-27 18:48:34,881][20347] Updated weights for policy 0, policy_version 1899852 (0.0007) [2023-12-27 18:48:34,926][20347] Updated weights for policy 0, policy_version 1899862 (0.0008) [2023-12-27 18:48:34,978][20347] Updated weights for policy 0, policy_version 1899872 (0.0008) [2023-12-27 18:48:35,490][20381] Updated weights for policy 1, policy_version 1898280 (0.0005) [2023-12-27 18:48:35,550][20381] Updated weights for policy 1, policy_version 1898290 (0.0005) [2023-12-27 18:48:35,603][20381] Updated weights for policy 1, policy_version 1898300 (0.0006) [2023-12-27 18:48:35,703][20347] Updated weights for policy 0, policy_version 1899882 (0.0008) [2023-12-27 18:48:35,766][20347] Updated weights for policy 0, policy_version 1899892 (0.0008) [2023-12-27 18:48:35,819][20347] Updated weights for policy 0, policy_version 1899902 (0.0008) [2023-12-27 18:48:36,182][20381] Updated weights for policy 1, policy_version 1898310 (0.0006) [2023-12-27 18:48:36,244][20381] Updated weights for policy 1, policy_version 1898320 (0.0008) [2023-12-27 18:48:36,298][20381] Updated weights for policy 1, policy_version 1898330 (0.0008) [2023-12-27 18:48:36,502][20347] Updated weights for policy 0, policy_version 1899912 (0.0008) [2023-12-27 18:48:36,558][20347] Updated weights for policy 0, policy_version 1899922 (0.0009) [2023-12-27 18:48:36,616][20347] Updated weights for policy 0, policy_version 1899932 (0.0008) [2023-12-27 18:48:36,956][20381] Updated weights for policy 1, policy_version 1898340 (0.0007) [2023-12-27 18:48:37,009][20381] Updated weights for policy 1, policy_version 1898350 (0.0008) [2023-12-27 18:48:37,059][20381] Updated weights for policy 1, policy_version 1898360 (0.0008) [2023-12-27 18:48:37,321][20347] Updated weights for policy 0, policy_version 1899942 (0.0008) [2023-12-27 18:48:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 972513280. Throughput: 0: 10286.0, 1: 10347.8. Samples: 972483304. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.270')] [2023-12-27 18:48:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001898368_486039552.pth... [2023-12-27 18:48:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001897152_485728256.pth [2023-12-27 18:48:37,378][20347] Updated weights for policy 0, policy_version 1899952 (0.0009) [2023-12-27 18:48:37,435][20347] Updated weights for policy 0, policy_version 1899962 (0.0008) [2023-12-27 18:48:37,469][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001899968_486481920.pth... [2023-12-27 18:48:37,474][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001898752_486170624.pth [2023-12-27 18:48:37,742][20381] Updated weights for policy 1, policy_version 1898370 (0.0007) [2023-12-27 18:48:37,804][20381] Updated weights for policy 1, policy_version 1898380 (0.0007) [2023-12-27 18:48:37,853][20381] Updated weights for policy 1, policy_version 1898390 (0.0005) [2023-12-27 18:48:37,916][20381] Updated weights for policy 1, policy_version 1898400 (0.0008) [2023-12-27 18:48:38,209][20347] Updated weights for policy 0, policy_version 1899972 (0.0007) [2023-12-27 18:48:38,265][20347] Updated weights for policy 0, policy_version 1899982 (0.0008) [2023-12-27 18:48:38,323][20347] Updated weights for policy 0, policy_version 1899992 (0.0007) [2023-12-27 18:48:38,588][20381] Updated weights for policy 1, policy_version 1898410 (0.0008) [2023-12-27 18:48:38,648][20381] Updated weights for policy 1, policy_version 1898420 (0.0007) [2023-12-27 18:48:38,711][20381] Updated weights for policy 1, policy_version 1898430 (0.0008) [2023-12-27 18:48:39,041][20347] Updated weights for policy 0, policy_version 1900002 (0.0009) [2023-12-27 18:48:39,108][20347] Updated weights for policy 0, policy_version 1900012 (0.0008) [2023-12-27 18:48:39,164][20347] Updated weights for policy 0, policy_version 1900022 (0.0005) [2023-12-27 18:48:39,234][20347] Updated weights for policy 0, policy_version 1900032 (0.0006) [2023-12-27 18:48:39,362][20381] Updated weights for policy 1, policy_version 1898440 (0.0008) [2023-12-27 18:48:39,427][20381] Updated weights for policy 1, policy_version 1898450 (0.0008) [2023-12-27 18:48:39,492][20381] Updated weights for policy 1, policy_version 1898460 (0.0008) [2023-12-27 18:48:39,954][20347] Updated weights for policy 0, policy_version 1900042 (0.0013) [2023-12-27 18:48:40,014][20347] Updated weights for policy 0, policy_version 1900052 (0.0010) [2023-12-27 18:48:40,076][20347] Updated weights for policy 0, policy_version 1900062 (0.0010) [2023-12-27 18:48:40,178][20381] Updated weights for policy 1, policy_version 1898470 (0.0007) [2023-12-27 18:48:40,233][20381] Updated weights for policy 1, policy_version 1898480 (0.0009) [2023-12-27 18:48:40,292][20381] Updated weights for policy 1, policy_version 1898490 (0.0007) [2023-12-27 18:48:40,751][20347] Updated weights for policy 0, policy_version 1900072 (0.0010) [2023-12-27 18:48:40,805][20347] Updated weights for policy 0, policy_version 1900082 (0.0010) [2023-12-27 18:48:40,857][20347] Updated weights for policy 0, policy_version 1900092 (0.0010) [2023-12-27 18:48:40,974][20381] Updated weights for policy 1, policy_version 1898500 (0.0007) [2023-12-27 18:48:41,036][20381] Updated weights for policy 1, policy_version 1898510 (0.0006) [2023-12-27 18:48:41,098][20381] Updated weights for policy 1, policy_version 1898520 (0.0007) [2023-12-27 18:48:41,638][20347] Updated weights for policy 0, policy_version 1900102 (0.0009) [2023-12-27 18:48:41,704][20347] Updated weights for policy 0, policy_version 1900112 (0.0007) [2023-12-27 18:48:41,771][20347] Updated weights for policy 0, policy_version 1900122 (0.0008) [2023-12-27 18:48:41,851][20381] Updated weights for policy 1, policy_version 1898530 (0.0008) [2023-12-27 18:48:41,916][20381] Updated weights for policy 1, policy_version 1898540 (0.0007) [2023-12-27 18:48:41,970][20381] Updated weights for policy 1, policy_version 1898550 (0.0007) [2023-12-27 18:48:42,028][20381] Updated weights for policy 1, policy_version 1898560 (0.0008) [2023-12-27 18:48:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 972611584. Throughput: 0: 10211.4, 1: 10368.8. Samples: 972604320. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.900')] [2023-12-27 18:48:42,546][20347] Updated weights for policy 0, policy_version 1900132 (0.0008) [2023-12-27 18:48:42,608][20347] Updated weights for policy 0, policy_version 1900142 (0.0007) [2023-12-27 18:48:42,663][20347] Updated weights for policy 0, policy_version 1900152 (0.0007) [2023-12-27 18:48:42,748][20381] Updated weights for policy 1, policy_version 1898570 (0.0008) [2023-12-27 18:48:42,800][20381] Updated weights for policy 1, policy_version 1898580 (0.0007) [2023-12-27 18:48:42,861][20381] Updated weights for policy 1, policy_version 1898590 (0.0010) [2023-12-27 18:48:43,324][20347] Updated weights for policy 0, policy_version 1900162 (0.0007) [2023-12-27 18:48:43,370][20347] Updated weights for policy 0, policy_version 1900172 (0.0005) [2023-12-27 18:48:43,416][20347] Updated weights for policy 0, policy_version 1900182 (0.0005) [2023-12-27 18:48:43,469][20347] Updated weights for policy 0, policy_version 1900192 (0.0005) [2023-12-27 18:48:43,491][20381] Updated weights for policy 1, policy_version 1898600 (0.0010) [2023-12-27 18:48:43,538][20381] Updated weights for policy 1, policy_version 1898610 (0.0009) [2023-12-27 18:48:43,591][20381] Updated weights for policy 1, policy_version 1898620 (0.0011) [2023-12-27 18:48:44,139][20347] Updated weights for policy 0, policy_version 1900202 (0.0007) [2023-12-27 18:48:44,185][20381] Updated weights for policy 1, policy_version 1898630 (0.0007) [2023-12-27 18:48:44,199][20347] Updated weights for policy 0, policy_version 1900212 (0.0008) [2023-12-27 18:48:44,237][20381] Updated weights for policy 1, policy_version 1898640 (0.0006) [2023-12-27 18:48:44,252][20347] Updated weights for policy 0, policy_version 1900222 (0.0007) [2023-12-27 18:48:44,288][20381] Updated weights for policy 1, policy_version 1898650 (0.0008) [2023-12-27 18:48:44,879][20381] Updated weights for policy 1, policy_version 1898660 (0.0008) [2023-12-27 18:48:44,937][20381] Updated weights for policy 1, policy_version 1898670 (0.0008) [2023-12-27 18:48:44,959][20347] Updated weights for policy 0, policy_version 1900232 (0.0006) [2023-12-27 18:48:45,002][20381] Updated weights for policy 1, policy_version 1898680 (0.0008) [2023-12-27 18:48:45,021][20347] Updated weights for policy 0, policy_version 1900242 (0.0006) [2023-12-27 18:48:45,082][20347] Updated weights for policy 0, policy_version 1900252 (0.0006) [2023-12-27 18:48:45,664][20347] Updated weights for policy 0, policy_version 1900262 (0.0007) [2023-12-27 18:48:45,700][20381] Updated weights for policy 1, policy_version 1898690 (0.0008) [2023-12-27 18:48:45,707][20347] Updated weights for policy 0, policy_version 1900272 (0.0007) [2023-12-27 18:48:45,753][20381] Updated weights for policy 1, policy_version 1898700 (0.0008) [2023-12-27 18:48:45,756][20347] Updated weights for policy 0, policy_version 1900282 (0.0006) [2023-12-27 18:48:45,799][20381] Updated weights for policy 1, policy_version 1898710 (0.0006) [2023-12-27 18:48:45,852][20381] Updated weights for policy 1, policy_version 1898720 (0.0007) [2023-12-27 18:48:46,465][20347] Updated weights for policy 0, policy_version 1900292 (0.0006) [2023-12-27 18:48:46,515][20347] Updated weights for policy 0, policy_version 1900302 (0.0006) [2023-12-27 18:48:46,535][20381] Updated weights for policy 1, policy_version 1898730 (0.0006) [2023-12-27 18:48:46,571][20347] Updated weights for policy 0, policy_version 1900312 (0.0007) [2023-12-27 18:48:46,591][20381] Updated weights for policy 1, policy_version 1898740 (0.0006) [2023-12-27 18:48:46,647][20381] Updated weights for policy 1, policy_version 1898750 (0.0005) [2023-12-27 18:48:47,235][20381] Updated weights for policy 1, policy_version 1898760 (0.0006) [2023-12-27 18:48:47,292][20381] Updated weights for policy 1, policy_version 1898770 (0.0007) [2023-12-27 18:48:47,298][20347] Updated weights for policy 0, policy_version 1900322 (0.0006) [2023-12-27 18:48:47,345][20381] Updated weights for policy 1, policy_version 1898780 (0.0006) [2023-12-27 18:48:47,347][20347] Updated weights for policy 0, policy_version 1900332 (0.0005) [2023-12-27 18:48:47,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 972709888. Throughput: 0: 10130.1, 1: 10463.3. Samples: 972728272. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.560')] [2023-12-27 18:48:47,397][20347] Updated weights for policy 0, policy_version 1900342 (0.0006) [2023-12-27 18:48:47,447][20347] Updated weights for policy 0, policy_version 1900352 (0.0006) [2023-12-27 18:48:48,030][20381] Updated weights for policy 1, policy_version 1898790 (0.0007) [2023-12-27 18:48:48,089][20381] Updated weights for policy 1, policy_version 1898800 (0.0007) [2023-12-27 18:48:48,142][20381] Updated weights for policy 1, policy_version 1898810 (0.0006) [2023-12-27 18:48:48,142][20347] Updated weights for policy 0, policy_version 1900362 (0.0008) [2023-12-27 18:48:48,197][20347] Updated weights for policy 0, policy_version 1900372 (0.0007) [2023-12-27 18:48:48,255][20347] Updated weights for policy 0, policy_version 1900382 (0.0008) [2023-12-27 18:48:48,847][20381] Updated weights for policy 1, policy_version 1898820 (0.0008) [2023-12-27 18:48:48,903][20381] Updated weights for policy 1, policy_version 1898830 (0.0008) [2023-12-27 18:48:48,962][20381] Updated weights for policy 1, policy_version 1898840 (0.0007) [2023-12-27 18:48:48,968][20347] Updated weights for policy 0, policy_version 1900392 (0.0008) [2023-12-27 18:48:49,026][20347] Updated weights for policy 0, policy_version 1900402 (0.0007) [2023-12-27 18:48:49,085][20347] Updated weights for policy 0, policy_version 1900412 (0.0008) [2023-12-27 18:48:49,618][20381] Updated weights for policy 1, policy_version 1898850 (0.0006) [2023-12-27 18:48:49,680][20381] Updated weights for policy 1, policy_version 1898860 (0.0006) [2023-12-27 18:48:49,739][20381] Updated weights for policy 1, policy_version 1898870 (0.0007) [2023-12-27 18:48:49,794][20381] Updated weights for policy 1, policy_version 1898880 (0.0007) [2023-12-27 18:48:49,877][20347] Updated weights for policy 0, policy_version 1900422 (0.0007) [2023-12-27 18:48:49,941][20347] Updated weights for policy 0, policy_version 1900432 (0.0007) [2023-12-27 18:48:49,996][20347] Updated weights for policy 0, policy_version 1900442 (0.0005) [2023-12-27 18:48:50,462][20381] Updated weights for policy 1, policy_version 1898890 (0.0010) [2023-12-27 18:48:50,523][20381] Updated weights for policy 1, policy_version 1898900 (0.0010) [2023-12-27 18:48:50,573][20381] Updated weights for policy 1, policy_version 1898910 (0.0009) [2023-12-27 18:48:50,715][20347] Updated weights for policy 0, policy_version 1900452 (0.0006) [2023-12-27 18:48:50,770][20347] Updated weights for policy 0, policy_version 1900462 (0.0007) [2023-12-27 18:48:50,826][20347] Updated weights for policy 0, policy_version 1900472 (0.0007) [2023-12-27 18:48:51,328][20381] Updated weights for policy 1, policy_version 1898920 (0.0009) [2023-12-27 18:48:51,395][20381] Updated weights for policy 1, policy_version 1898930 (0.0008) [2023-12-27 18:48:51,453][20381] Updated weights for policy 1, policy_version 1898940 (0.0008) [2023-12-27 18:48:51,527][20347] Updated weights for policy 0, policy_version 1900482 (0.0008) [2023-12-27 18:48:51,587][20347] Updated weights for policy 0, policy_version 1900492 (0.0008) [2023-12-27 18:48:51,655][20347] Updated weights for policy 0, policy_version 1900502 (0.0007) [2023-12-27 18:48:51,714][20347] Updated weights for policy 0, policy_version 1900512 (0.0009) [2023-12-27 18:48:52,151][20381] Updated weights for policy 1, policy_version 1898950 (0.0006) [2023-12-27 18:48:52,215][20381] Updated weights for policy 1, policy_version 1898960 (0.0006) [2023-12-27 18:48:52,274][20381] Updated weights for policy 1, policy_version 1898970 (0.0008) [2023-12-27 18:48:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 972816384. Throughput: 0: 10065.6, 1: 10438.0. Samples: 972787532. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.580')] [2023-12-27 18:48:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001898976_486195200.pth... [2023-12-27 18:48:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001897760_485883904.pth [2023-12-27 18:48:52,418][20347] Updated weights for policy 0, policy_version 1900522 (0.0007) [2023-12-27 18:48:52,468][20347] Updated weights for policy 0, policy_version 1900532 (0.0008) [2023-12-27 18:48:52,517][20347] Updated weights for policy 0, policy_version 1900542 (0.0007) [2023-12-27 18:48:52,526][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001900544_486629376.pth... [2023-12-27 18:48:52,529][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001899360_486326272.pth [2023-12-27 18:48:52,926][20381] Updated weights for policy 1, policy_version 1898980 (0.0008) [2023-12-27 18:48:52,979][20381] Updated weights for policy 1, policy_version 1898990 (0.0006) [2023-12-27 18:48:53,040][20381] Updated weights for policy 1, policy_version 1899000 (0.0007) [2023-12-27 18:48:53,195][20347] Updated weights for policy 0, policy_version 1900552 (0.0008) [2023-12-27 18:48:53,248][20347] Updated weights for policy 0, policy_version 1900562 (0.0006) [2023-12-27 18:48:53,307][20347] Updated weights for policy 0, policy_version 1900572 (0.0008) [2023-12-27 18:48:53,705][20381] Updated weights for policy 1, policy_version 1899010 (0.0007) [2023-12-27 18:48:53,762][20381] Updated weights for policy 1, policy_version 1899020 (0.0007) [2023-12-27 18:48:53,809][20381] Updated weights for policy 1, policy_version 1899030 (0.0008) [2023-12-27 18:48:53,867][20381] Updated weights for policy 1, policy_version 1899040 (0.0008) [2023-12-27 18:48:53,903][20347] Updated weights for policy 0, policy_version 1900582 (0.0007) [2023-12-27 18:48:53,951][20347] Updated weights for policy 0, policy_version 1900592 (0.0007) [2023-12-27 18:48:54,007][20347] Updated weights for policy 0, policy_version 1900602 (0.0006) [2023-12-27 18:48:54,582][20381] Updated weights for policy 1, policy_version 1899050 (0.0007) [2023-12-27 18:48:54,608][20347] Updated weights for policy 0, policy_version 1900612 (0.0005) [2023-12-27 18:48:54,635][20381] Updated weights for policy 1, policy_version 1899060 (0.0006) [2023-12-27 18:48:54,670][20347] Updated weights for policy 0, policy_version 1900622 (0.0005) [2023-12-27 18:48:54,692][20381] Updated weights for policy 1, policy_version 1899070 (0.0006) [2023-12-27 18:48:54,733][20347] Updated weights for policy 0, policy_version 1900632 (0.0005) [2023-12-27 18:48:55,263][20347] Updated weights for policy 0, policy_version 1900642 (0.0005) [2023-12-27 18:48:55,316][20381] Updated weights for policy 1, policy_version 1899080 (0.0007) [2023-12-27 18:48:55,325][20347] Updated weights for policy 0, policy_version 1900652 (0.0006) [2023-12-27 18:48:55,371][20381] Updated weights for policy 1, policy_version 1899090 (0.0006) [2023-12-27 18:48:55,384][20347] Updated weights for policy 0, policy_version 1900662 (0.0006) [2023-12-27 18:48:55,425][20381] Updated weights for policy 1, policy_version 1899100 (0.0007) [2023-12-27 18:48:55,445][20347] Updated weights for policy 0, policy_version 1900672 (0.0010) [2023-12-27 18:48:56,037][20347] Updated weights for policy 0, policy_version 1900682 (0.0005) [2023-12-27 18:48:56,098][20347] Updated weights for policy 0, policy_version 1900692 (0.0006) [2023-12-27 18:48:56,151][20347] Updated weights for policy 0, policy_version 1900702 (0.0007) [2023-12-27 18:48:56,152][20381] Updated weights for policy 1, policy_version 1899110 (0.0007) [2023-12-27 18:48:56,210][20381] Updated weights for policy 1, policy_version 1899120 (0.0008) [2023-12-27 18:48:56,268][20381] Updated weights for policy 1, policy_version 1899130 (0.0007) [2023-12-27 18:48:56,779][20347] Updated weights for policy 0, policy_version 1900712 (0.0005) [2023-12-27 18:48:56,838][20347] Updated weights for policy 0, policy_version 1900722 (0.0005) [2023-12-27 18:48:56,894][20347] Updated weights for policy 0, policy_version 1900732 (0.0005) [2023-12-27 18:48:56,960][20381] Updated weights for policy 1, policy_version 1899140 (0.0008) [2023-12-27 18:48:57,018][20381] Updated weights for policy 1, policy_version 1899150 (0.0008) [2023-12-27 18:48:57,074][20381] Updated weights for policy 1, policy_version 1899160 (0.0007) [2023-12-27 18:48:57,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 972922880. Throughput: 0: 10120.3, 1: 10407.7. Samples: 972912844. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-12-27 18:48:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.780')] [2023-12-27 18:48:57,528][20347] Updated weights for policy 0, policy_version 1900742 (0.0005) [2023-12-27 18:48:57,587][20347] Updated weights for policy 0, policy_version 1900752 (0.0005) [2023-12-27 18:48:57,645][20347] Updated weights for policy 0, policy_version 1900762 (0.0005) [2023-12-27 18:48:57,739][20381] Updated weights for policy 1, policy_version 1899170 (0.0007) [2023-12-27 18:48:57,795][20381] Updated weights for policy 1, policy_version 1899180 (0.0008) [2023-12-27 18:48:57,841][20381] Updated weights for policy 1, policy_version 1899190 (0.0007) [2023-12-27 18:48:57,888][20381] Updated weights for policy 1, policy_version 1899200 (0.0007) [2023-12-27 18:48:58,265][20347] Updated weights for policy 0, policy_version 1900772 (0.0007) [2023-12-27 18:48:58,330][20347] Updated weights for policy 0, policy_version 1900782 (0.0008) [2023-12-27 18:48:58,399][20347] Updated weights for policy 0, policy_version 1900792 (0.0008) [2023-12-27 18:48:58,635][20381] Updated weights for policy 1, policy_version 1899210 (0.0008) [2023-12-27 18:48:58,699][20381] Updated weights for policy 1, policy_version 1899220 (0.0007) [2023-12-27 18:48:58,764][20381] Updated weights for policy 1, policy_version 1899230 (0.0008) [2023-12-27 18:48:59,136][20347] Updated weights for policy 0, policy_version 1900802 (0.0008) [2023-12-27 18:48:59,198][20347] Updated weights for policy 0, policy_version 1900812 (0.0006) [2023-12-27 18:48:59,273][20347] Updated weights for policy 0, policy_version 1900822 (0.0008) [2023-12-27 18:48:59,337][20347] Updated weights for policy 0, policy_version 1900832 (0.0007) [2023-12-27 18:48:59,559][20381] Updated weights for policy 1, policy_version 1899240 (0.0010) [2023-12-27 18:48:59,622][20381] Updated weights for policy 1, policy_version 1899250 (0.0011) [2023-12-27 18:48:59,684][20381] Updated weights for policy 1, policy_version 1899260 (0.0011) [2023-12-27 18:49:00,027][20347] Updated weights for policy 0, policy_version 1900842 (0.0011) [2023-12-27 18:49:00,084][20347] Updated weights for policy 0, policy_version 1900852 (0.0010) [2023-12-27 18:49:00,138][20347] Updated weights for policy 0, policy_version 1900862 (0.0006) [2023-12-27 18:49:00,377][20381] Updated weights for policy 1, policy_version 1899270 (0.0011) [2023-12-27 18:49:00,439][20381] Updated weights for policy 1, policy_version 1899280 (0.0011) [2023-12-27 18:49:00,491][20381] Updated weights for policy 1, policy_version 1899290 (0.0011) [2023-12-27 18:49:00,782][20347] Updated weights for policy 0, policy_version 1900872 (0.0006) [2023-12-27 18:49:00,834][20347] Updated weights for policy 0, policy_version 1900882 (0.0005) [2023-12-27 18:49:00,892][20347] Updated weights for policy 0, policy_version 1900892 (0.0006) [2023-12-27 18:49:01,177][20381] Updated weights for policy 1, policy_version 1899300 (0.0011) [2023-12-27 18:49:01,233][20381] Updated weights for policy 1, policy_version 1899310 (0.0011) [2023-12-27 18:49:01,290][20381] Updated weights for policy 1, policy_version 1899320 (0.0011) [2023-12-27 18:49:01,578][20347] Updated weights for policy 0, policy_version 1900902 (0.0008) [2023-12-27 18:49:01,637][20347] Updated weights for policy 0, policy_version 1900912 (0.0007) [2023-12-27 18:49:01,700][20347] Updated weights for policy 0, policy_version 1900922 (0.0007) [2023-12-27 18:49:02,029][20381] Updated weights for policy 1, policy_version 1899330 (0.0010) [2023-12-27 18:49:02,093][20381] Updated weights for policy 1, policy_version 1899340 (0.0009) [2023-12-27 18:49:02,154][20381] Updated weights for policy 1, policy_version 1899350 (0.0011) [2023-12-27 18:49:02,219][20381] Updated weights for policy 1, policy_version 1899360 (0.0010) [2023-12-27 18:49:02,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 973021184. Throughput: 0: 10124.1, 1: 10291.1. Samples: 973031132. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:02,363][19170] Avg episode reward: [(0, '990.270'), (1, '921.790')] [2023-12-27 18:49:02,393][20347] Updated weights for policy 0, policy_version 1900932 (0.0008) [2023-12-27 18:49:02,456][20347] Updated weights for policy 0, policy_version 1900942 (0.0007) [2023-12-27 18:49:02,508][20347] Updated weights for policy 0, policy_version 1900952 (0.0007) [2023-12-27 18:49:02,863][20381] Updated weights for policy 1, policy_version 1899370 (0.0006) [2023-12-27 18:49:02,924][20381] Updated weights for policy 1, policy_version 1899380 (0.0007) [2023-12-27 18:49:02,970][20381] Updated weights for policy 1, policy_version 1899390 (0.0007) [2023-12-27 18:49:03,222][20347] Updated weights for policy 0, policy_version 1900962 (0.0008) [2023-12-27 18:49:03,270][20347] Updated weights for policy 0, policy_version 1900972 (0.0010) [2023-12-27 18:49:03,316][20347] Updated weights for policy 0, policy_version 1900982 (0.0005) [2023-12-27 18:49:03,367][20347] Updated weights for policy 0, policy_version 1900992 (0.0005) [2023-12-27 18:49:03,542][20381] Updated weights for policy 1, policy_version 1899401 (0.0008) [2023-12-27 18:49:03,595][20381] Updated weights for policy 1, policy_version 1899411 (0.0008) [2023-12-27 18:49:03,644][20381] Updated weights for policy 1, policy_version 1899421 (0.0008) [2023-12-27 18:49:04,054][20347] Updated weights for policy 0, policy_version 1901002 (0.0010) [2023-12-27 18:49:04,110][20347] Updated weights for policy 0, policy_version 1901012 (0.0010) [2023-12-27 18:49:04,166][20347] Updated weights for policy 0, policy_version 1901022 (0.0010) [2023-12-27 18:49:04,238][20381] Updated weights for policy 1, policy_version 1899431 (0.0009) [2023-12-27 18:49:04,300][20381] Updated weights for policy 1, policy_version 1899441 (0.0010) [2023-12-27 18:49:04,361][20381] Updated weights for policy 1, policy_version 1899451 (0.0010) [2023-12-27 18:49:04,876][20347] Updated weights for policy 0, policy_version 1901032 (0.0009) [2023-12-27 18:49:04,935][20347] Updated weights for policy 0, policy_version 1901042 (0.0005) [2023-12-27 18:49:04,993][20347] Updated weights for policy 0, policy_version 1901052 (0.0005) [2023-12-27 18:49:05,025][20381] Updated weights for policy 1, policy_version 1899461 (0.0011) [2023-12-27 18:49:05,076][20381] Updated weights for policy 1, policy_version 1899471 (0.0010) [2023-12-27 18:49:05,122][20381] Updated weights for policy 1, policy_version 1899481 (0.0010) [2023-12-27 18:49:05,620][20347] Updated weights for policy 0, policy_version 1901062 (0.0008) [2023-12-27 18:49:05,674][20347] Updated weights for policy 0, policy_version 1901072 (0.0010) [2023-12-27 18:49:05,728][20347] Updated weights for policy 0, policy_version 1901082 (0.0010) [2023-12-27 18:49:05,788][20381] Updated weights for policy 1, policy_version 1899491 (0.0010) [2023-12-27 18:49:05,841][20381] Updated weights for policy 1, policy_version 1899501 (0.0010) [2023-12-27 18:49:05,893][20381] Updated weights for policy 1, policy_version 1899511 (0.0007) [2023-12-27 18:49:06,438][20347] Updated weights for policy 0, policy_version 1901092 (0.0010) [2023-12-27 18:49:06,507][20347] Updated weights for policy 0, policy_version 1901102 (0.0010) [2023-12-27 18:49:06,575][20347] Updated weights for policy 0, policy_version 1901112 (0.0010) [2023-12-27 18:49:06,577][20381] Updated weights for policy 1, policy_version 1899521 (0.0010) [2023-12-27 18:49:06,643][20381] Updated weights for policy 1, policy_version 1899531 (0.0008) [2023-12-27 18:49:06,699][20381] Updated weights for policy 1, policy_version 1899541 (0.0008) [2023-12-27 18:49:06,749][20381] Updated weights for policy 1, policy_version 1899551 (0.0007) [2023-12-27 18:49:07,177][20347] Updated weights for policy 0, policy_version 1901122 (0.0010) [2023-12-27 18:49:07,230][20347] Updated weights for policy 0, policy_version 1901132 (0.0005) [2023-12-27 18:49:07,284][20347] Updated weights for policy 0, policy_version 1901142 (0.0005) [2023-12-27 18:49:07,343][20347] Updated weights for policy 0, policy_version 1901152 (0.0006) [2023-12-27 18:49:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 973127680. Throughput: 0: 10118.1, 1: 10295.2. Samples: 973094680. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.440')] [2023-12-27 18:49:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001901152_486785024.pth... [2023-12-27 18:49:07,369][20381] Updated weights for policy 1, policy_version 1899561 (0.0008) [2023-12-27 18:49:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001899968_486481920.pth [2023-12-27 18:49:07,421][20381] Updated weights for policy 1, policy_version 1899571 (0.0007) [2023-12-27 18:49:07,470][20381] Updated weights for policy 1, policy_version 1899581 (0.0008) [2023-12-27 18:49:07,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001899584_486350848.pth... [2023-12-27 18:49:07,485][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001898368_486039552.pth [2023-12-27 18:49:07,928][20347] Updated weights for policy 0, policy_version 1901162 (0.0008) [2023-12-27 18:49:07,986][20347] Updated weights for policy 0, policy_version 1901172 (0.0007) [2023-12-27 18:49:08,046][20347] Updated weights for policy 0, policy_version 1901182 (0.0010) [2023-12-27 18:49:08,148][20381] Updated weights for policy 1, policy_version 1899591 (0.0007) [2023-12-27 18:49:08,204][20381] Updated weights for policy 1, policy_version 1899601 (0.0005) [2023-12-27 18:49:08,264][20381] Updated weights for policy 1, policy_version 1899611 (0.0005) [2023-12-27 18:49:08,757][20347] Updated weights for policy 0, policy_version 1901192 (0.0010) [2023-12-27 18:49:08,817][20347] Updated weights for policy 0, policy_version 1901202 (0.0010) [2023-12-27 18:49:08,878][20347] Updated weights for policy 0, policy_version 1901212 (0.0010) [2023-12-27 18:49:08,936][20381] Updated weights for policy 1, policy_version 1899621 (0.0007) [2023-12-27 18:49:09,003][20381] Updated weights for policy 1, policy_version 1899631 (0.0008) [2023-12-27 18:49:09,048][20381] Updated weights for policy 1, policy_version 1899641 (0.0007) [2023-12-27 18:49:09,594][20347] Updated weights for policy 0, policy_version 1901222 (0.0008) [2023-12-27 18:49:09,658][20347] Updated weights for policy 0, policy_version 1901232 (0.0006) [2023-12-27 18:49:09,720][20347] Updated weights for policy 0, policy_version 1901242 (0.0007) [2023-12-27 18:49:09,813][20381] Updated weights for policy 1, policy_version 1899651 (0.0007) [2023-12-27 18:49:09,877][20381] Updated weights for policy 1, policy_version 1899661 (0.0009) [2023-12-27 18:49:09,945][20381] Updated weights for policy 1, policy_version 1899671 (0.0008) [2023-12-27 18:49:10,384][20347] Updated weights for policy 0, policy_version 1901252 (0.0006) [2023-12-27 18:49:10,438][20347] Updated weights for policy 0, policy_version 1901262 (0.0007) [2023-12-27 18:49:10,487][20347] Updated weights for policy 0, policy_version 1901272 (0.0007) [2023-12-27 18:49:10,688][20381] Updated weights for policy 1, policy_version 1899681 (0.0008) [2023-12-27 18:49:10,744][20381] Updated weights for policy 1, policy_version 1899691 (0.0008) [2023-12-27 18:49:10,795][20381] Updated weights for policy 1, policy_version 1899701 (0.0007) [2023-12-27 18:49:10,852][20381] Updated weights for policy 1, policy_version 1899711 (0.0007) [2023-12-27 18:49:11,208][20347] Updated weights for policy 0, policy_version 1901282 (0.0008) [2023-12-27 18:49:11,269][20347] Updated weights for policy 0, policy_version 1901292 (0.0010) [2023-12-27 18:49:11,335][20347] Updated weights for policy 0, policy_version 1901302 (0.0011) [2023-12-27 18:49:11,406][20347] Updated weights for policy 0, policy_version 1901312 (0.0010) [2023-12-27 18:49:11,628][20381] Updated weights for policy 1, policy_version 1899721 (0.0009) [2023-12-27 18:49:11,692][20381] Updated weights for policy 1, policy_version 1899731 (0.0008) [2023-12-27 18:49:11,757][20381] Updated weights for policy 1, policy_version 1899741 (0.0008) [2023-12-27 18:49:12,130][20347] Updated weights for policy 0, policy_version 1901322 (0.0009) [2023-12-27 18:49:12,176][20347] Updated weights for policy 0, policy_version 1901332 (0.0007) [2023-12-27 18:49:12,233][20347] Updated weights for policy 0, policy_version 1901342 (0.0007) [2023-12-27 18:49:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 973225984. Throughput: 0: 10087.3, 1: 10323.5. Samples: 973216740. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '950.950')] [2023-12-27 18:49:12,482][20381] Updated weights for policy 1, policy_version 1899751 (0.0008) [2023-12-27 18:49:12,543][20381] Updated weights for policy 1, policy_version 1899761 (0.0008) [2023-12-27 18:49:12,600][20381] Updated weights for policy 1, policy_version 1899771 (0.0008) [2023-12-27 18:49:12,979][20347] Updated weights for policy 0, policy_version 1901352 (0.0008) [2023-12-27 18:49:13,031][20347] Updated weights for policy 0, policy_version 1901362 (0.0008) [2023-12-27 18:49:13,090][20347] Updated weights for policy 0, policy_version 1901372 (0.0008) [2023-12-27 18:49:13,270][20381] Updated weights for policy 1, policy_version 1899781 (0.0008) [2023-12-27 18:49:13,324][20381] Updated weights for policy 1, policy_version 1899791 (0.0009) [2023-12-27 18:49:13,370][20381] Updated weights for policy 1, policy_version 1899801 (0.0010) [2023-12-27 18:49:13,714][20347] Updated weights for policy 0, policy_version 1901382 (0.0007) [2023-12-27 18:49:13,765][20347] Updated weights for policy 0, policy_version 1901392 (0.0007) [2023-12-27 18:49:13,814][20347] Updated weights for policy 0, policy_version 1901402 (0.0008) [2023-12-27 18:49:13,994][20381] Updated weights for policy 1, policy_version 1899811 (0.0009) [2023-12-27 18:49:14,053][20381] Updated weights for policy 1, policy_version 1899821 (0.0007) [2023-12-27 18:49:14,113][20381] Updated weights for policy 1, policy_version 1899831 (0.0007) [2023-12-27 18:49:14,542][20347] Updated weights for policy 0, policy_version 1901412 (0.0007) [2023-12-27 18:49:14,596][20347] Updated weights for policy 0, policy_version 1901422 (0.0007) [2023-12-27 18:49:14,642][20347] Updated weights for policy 0, policy_version 1901432 (0.0008) [2023-12-27 18:49:14,732][20381] Updated weights for policy 1, policy_version 1899841 (0.0006) [2023-12-27 18:49:14,792][20381] Updated weights for policy 1, policy_version 1899851 (0.0008) [2023-12-27 18:49:14,853][20381] Updated weights for policy 1, policy_version 1899861 (0.0008) [2023-12-27 18:49:14,914][20381] Updated weights for policy 1, policy_version 1899871 (0.0008) [2023-12-27 18:49:15,355][20347] Updated weights for policy 0, policy_version 1901442 (0.0007) [2023-12-27 18:49:15,412][20347] Updated weights for policy 0, policy_version 1901452 (0.0008) [2023-12-27 18:49:15,468][20347] Updated weights for policy 0, policy_version 1901462 (0.0008) [2023-12-27 18:49:15,520][20347] Updated weights for policy 0, policy_version 1901472 (0.0007) [2023-12-27 18:49:15,593][20381] Updated weights for policy 1, policy_version 1899881 (0.0008) [2023-12-27 18:49:15,644][20381] Updated weights for policy 1, policy_version 1899891 (0.0008) [2023-12-27 18:49:15,697][20381] Updated weights for policy 1, policy_version 1899901 (0.0008) [2023-12-27 18:49:16,231][20347] Updated weights for policy 0, policy_version 1901482 (0.0008) [2023-12-27 18:49:16,288][20347] Updated weights for policy 0, policy_version 1901492 (0.0005) [2023-12-27 18:49:16,350][20347] Updated weights for policy 0, policy_version 1901502 (0.0007) [2023-12-27 18:49:16,369][20381] Updated weights for policy 1, policy_version 1899911 (0.0006) [2023-12-27 18:49:16,418][20381] Updated weights for policy 1, policy_version 1899921 (0.0008) [2023-12-27 18:49:16,474][20381] Updated weights for policy 1, policy_version 1899931 (0.0006) [2023-12-27 18:49:17,021][20381] Updated weights for policy 1, policy_version 1899941 (0.0005) [2023-12-27 18:49:17,050][20347] Updated weights for policy 0, policy_version 1901512 (0.0008) [2023-12-27 18:49:17,080][20381] Updated weights for policy 1, policy_version 1899951 (0.0005) [2023-12-27 18:49:17,107][20347] Updated weights for policy 0, policy_version 1901522 (0.0008) [2023-12-27 18:49:17,137][20381] Updated weights for policy 1, policy_version 1899961 (0.0005) [2023-12-27 18:49:17,165][20347] Updated weights for policy 0, policy_version 1901532 (0.0008) [2023-12-27 18:49:17,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 973332480. Throughput: 0: 10075.7, 1: 10292.0. Samples: 973339344. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '911.890')] [2023-12-27 18:49:17,716][20381] Updated weights for policy 1, policy_version 1899971 (0.0007) [2023-12-27 18:49:17,772][20381] Updated weights for policy 1, policy_version 1899981 (0.0011) [2023-12-27 18:49:17,831][20381] Updated weights for policy 1, policy_version 1899991 (0.0011) [2023-12-27 18:49:17,904][20347] Updated weights for policy 0, policy_version 1901542 (0.0007) [2023-12-27 18:49:17,959][20347] Updated weights for policy 0, policy_version 1901552 (0.0007) [2023-12-27 18:49:18,020][20347] Updated weights for policy 0, policy_version 1901562 (0.0007) [2023-12-27 18:49:18,521][20381] Updated weights for policy 1, policy_version 1900001 (0.0011) [2023-12-27 18:49:18,576][20381] Updated weights for policy 1, policy_version 1900011 (0.0010) [2023-12-27 18:49:18,638][20381] Updated weights for policy 1, policy_version 1900021 (0.0011) [2023-12-27 18:49:18,695][20381] Updated weights for policy 1, policy_version 1900031 (0.0011) [2023-12-27 18:49:18,702][20347] Updated weights for policy 0, policy_version 1901572 (0.0007) [2023-12-27 18:49:18,762][20347] Updated weights for policy 0, policy_version 1901582 (0.0008) [2023-12-27 18:49:18,814][20347] Updated weights for policy 0, policy_version 1901592 (0.0007) [2023-12-27 18:49:19,392][20381] Updated weights for policy 1, policy_version 1900041 (0.0010) [2023-12-27 18:49:19,452][20381] Updated weights for policy 1, policy_version 1900051 (0.0007) [2023-12-27 18:49:19,517][20381] Updated weights for policy 1, policy_version 1900061 (0.0008) [2023-12-27 18:49:19,560][20347] Updated weights for policy 0, policy_version 1901602 (0.0007) [2023-12-27 18:49:19,619][20347] Updated weights for policy 0, policy_version 1901612 (0.0007) [2023-12-27 18:49:19,677][20347] Updated weights for policy 0, policy_version 1901622 (0.0007) [2023-12-27 18:49:19,736][20347] Updated weights for policy 0, policy_version 1901632 (0.0007) [2023-12-27 18:49:20,221][20381] Updated weights for policy 1, policy_version 1900071 (0.0008) [2023-12-27 18:49:20,281][20381] Updated weights for policy 1, policy_version 1900081 (0.0008) [2023-12-27 18:49:20,340][20381] Updated weights for policy 1, policy_version 1900091 (0.0007) [2023-12-27 18:49:20,473][20347] Updated weights for policy 0, policy_version 1901642 (0.0007) [2023-12-27 18:49:20,527][20347] Updated weights for policy 0, policy_version 1901652 (0.0008) [2023-12-27 18:49:20,586][20347] Updated weights for policy 0, policy_version 1901662 (0.0008) [2023-12-27 18:49:20,988][20381] Updated weights for policy 1, policy_version 1900101 (0.0006) [2023-12-27 18:49:21,056][20381] Updated weights for policy 1, policy_version 1900111 (0.0007) [2023-12-27 18:49:21,119][20381] Updated weights for policy 1, policy_version 1900121 (0.0007) [2023-12-27 18:49:21,370][20347] Updated weights for policy 0, policy_version 1901672 (0.0009) [2023-12-27 18:49:21,432][20347] Updated weights for policy 0, policy_version 1901682 (0.0007) [2023-12-27 18:49:21,483][20347] Updated weights for policy 0, policy_version 1901692 (0.0005) [2023-12-27 18:49:21,801][20381] Updated weights for policy 1, policy_version 1900131 (0.0008) [2023-12-27 18:49:21,858][20381] Updated weights for policy 1, policy_version 1900141 (0.0007) [2023-12-27 18:49:21,920][20381] Updated weights for policy 1, policy_version 1900151 (0.0009) [2023-12-27 18:49:22,124][20347] Updated weights for policy 0, policy_version 1901702 (0.0006) [2023-12-27 18:49:22,184][20347] Updated weights for policy 0, policy_version 1901712 (0.0005) [2023-12-27 18:49:22,240][20347] Updated weights for policy 0, policy_version 1901722 (0.0005) [2023-12-27 18:49:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 973430784. Throughput: 0: 10060.5, 1: 10266.6. Samples: 973398024. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.420')] [2023-12-27 18:49:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001900160_486498304.pth... [2023-12-27 18:49:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001901728_486932480.pth... [2023-12-27 18:49:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001898976_486195200.pth [2023-12-27 18:49:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001900544_486629376.pth [2023-12-27 18:49:22,654][20381] Updated weights for policy 1, policy_version 1900161 (0.0007) [2023-12-27 18:49:22,704][20381] Updated weights for policy 1, policy_version 1900171 (0.0007) [2023-12-27 18:49:22,756][20381] Updated weights for policy 1, policy_version 1900181 (0.0007) [2023-12-27 18:49:22,809][20381] Updated weights for policy 1, policy_version 1900191 (0.0005) [2023-12-27 18:49:22,957][20347] Updated weights for policy 0, policy_version 1901732 (0.0007) [2023-12-27 18:49:23,021][20347] Updated weights for policy 0, policy_version 1901742 (0.0006) [2023-12-27 18:49:23,074][20347] Updated weights for policy 0, policy_version 1901752 (0.0007) [2023-12-27 18:49:23,498][20381] Updated weights for policy 1, policy_version 1900201 (0.0007) [2023-12-27 18:49:23,543][20381] Updated weights for policy 1, policy_version 1900211 (0.0007) [2023-12-27 18:49:23,588][20381] Updated weights for policy 1, policy_version 1900221 (0.0007) [2023-12-27 18:49:23,762][20347] Updated weights for policy 0, policy_version 1901762 (0.0007) [2023-12-27 18:49:23,815][20347] Updated weights for policy 0, policy_version 1901772 (0.0005) [2023-12-27 18:49:23,873][20347] Updated weights for policy 0, policy_version 1901782 (0.0009) [2023-12-27 18:49:23,927][20347] Updated weights for policy 0, policy_version 1901792 (0.0010) [2023-12-27 18:49:24,323][20381] Updated weights for policy 1, policy_version 1900231 (0.0006) [2023-12-27 18:49:24,376][20381] Updated weights for policy 1, policy_version 1900241 (0.0005) [2023-12-27 18:49:24,431][20381] Updated weights for policy 1, policy_version 1900251 (0.0007) [2023-12-27 18:49:24,546][20347] Updated weights for policy 0, policy_version 1901803 (0.0008) [2023-12-27 18:49:24,600][20347] Updated weights for policy 0, policy_version 1901813 (0.0010) [2023-12-27 18:49:24,653][20347] Updated weights for policy 0, policy_version 1901823 (0.0010) [2023-12-27 18:49:25,108][20381] Updated weights for policy 1, policy_version 1900261 (0.0008) [2023-12-27 18:49:25,163][20381] Updated weights for policy 1, policy_version 1900271 (0.0007) [2023-12-27 18:49:25,230][20381] Updated weights for policy 1, policy_version 1900281 (0.0007) [2023-12-27 18:49:25,311][20347] Updated weights for policy 0, policy_version 1901833 (0.0010) [2023-12-27 18:49:25,375][20347] Updated weights for policy 0, policy_version 1901843 (0.0009) [2023-12-27 18:49:25,438][20347] Updated weights for policy 0, policy_version 1901853 (0.0010) [2023-12-27 18:49:25,869][20381] Updated weights for policy 1, policy_version 1900291 (0.0007) [2023-12-27 18:49:25,917][20381] Updated weights for policy 1, policy_version 1900301 (0.0007) [2023-12-27 18:49:25,962][20381] Updated weights for policy 1, policy_version 1900311 (0.0007) [2023-12-27 18:49:26,109][20347] Updated weights for policy 0, policy_version 1901863 (0.0010) [2023-12-27 18:49:26,158][20347] Updated weights for policy 0, policy_version 1901873 (0.0010) [2023-12-27 18:49:26,206][20347] Updated weights for policy 0, policy_version 1901883 (0.0010) [2023-12-27 18:49:26,646][20381] Updated weights for policy 1, policy_version 1900321 (0.0007) [2023-12-27 18:49:26,696][20381] Updated weights for policy 1, policy_version 1900331 (0.0007) [2023-12-27 18:49:26,757][20381] Updated weights for policy 1, policy_version 1900341 (0.0008) [2023-12-27 18:49:26,802][20381] Updated weights for policy 1, policy_version 1900351 (0.0008) [2023-12-27 18:49:26,883][20347] Updated weights for policy 0, policy_version 1901893 (0.0009) [2023-12-27 18:49:26,936][20347] Updated weights for policy 0, policy_version 1901903 (0.0010) [2023-12-27 18:49:26,993][20347] Updated weights for policy 0, policy_version 1901913 (0.0010) [2023-12-27 18:49:27,362][19170] Fps is (10 sec: 19661.5, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 973529088. Throughput: 0: 10120.7, 1: 10208.0. Samples: 973519112. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.310')] [2023-12-27 18:49:27,467][20381] Updated weights for policy 1, policy_version 1900361 (0.0008) [2023-12-27 18:49:27,512][20381] Updated weights for policy 1, policy_version 1900371 (0.0009) [2023-12-27 18:49:27,558][20381] Updated weights for policy 1, policy_version 1900381 (0.0010) [2023-12-27 18:49:27,609][20347] Updated weights for policy 0, policy_version 1901923 (0.0009) [2023-12-27 18:49:27,669][20347] Updated weights for policy 0, policy_version 1901933 (0.0005) [2023-12-27 18:49:27,731][20347] Updated weights for policy 0, policy_version 1901943 (0.0006) [2023-12-27 18:49:28,235][20347] Updated weights for policy 0, policy_version 1901953 (0.0006) [2023-12-27 18:49:28,283][20347] Updated weights for policy 0, policy_version 1901963 (0.0005) [2023-12-27 18:49:28,343][20347] Updated weights for policy 0, policy_version 1901973 (0.0007) [2023-12-27 18:49:28,344][20381] Updated weights for policy 1, policy_version 1900391 (0.0007) [2023-12-27 18:49:28,399][20381] Updated weights for policy 1, policy_version 1900401 (0.0007) [2023-12-27 18:49:28,405][20347] Updated weights for policy 0, policy_version 1901983 (0.0007) [2023-12-27 18:49:28,448][20381] Updated weights for policy 1, policy_version 1900411 (0.0007) [2023-12-27 18:49:29,042][20347] Updated weights for policy 0, policy_version 1901993 (0.0006) [2023-12-27 18:49:29,099][20347] Updated weights for policy 0, policy_version 1902003 (0.0006) [2023-12-27 18:49:29,152][20347] Updated weights for policy 0, policy_version 1902013 (0.0006) [2023-12-27 18:49:29,177][20381] Updated weights for policy 1, policy_version 1900421 (0.0006) [2023-12-27 18:49:29,235][20381] Updated weights for policy 1, policy_version 1900431 (0.0009) [2023-12-27 18:49:29,293][20381] Updated weights for policy 1, policy_version 1900441 (0.0009) [2023-12-27 18:49:29,762][20347] Updated weights for policy 0, policy_version 1902023 (0.0008) [2023-12-27 18:49:29,825][20347] Updated weights for policy 0, policy_version 1902033 (0.0006) [2023-12-27 18:49:29,886][20347] Updated weights for policy 0, policy_version 1902043 (0.0008) [2023-12-27 18:49:30,015][20381] Updated weights for policy 1, policy_version 1900451 (0.0009) [2023-12-27 18:49:30,076][20381] Updated weights for policy 1, policy_version 1900461 (0.0010) [2023-12-27 18:49:30,136][20381] Updated weights for policy 1, policy_version 1900471 (0.0010) [2023-12-27 18:49:30,560][20347] Updated weights for policy 0, policy_version 1902053 (0.0008) [2023-12-27 18:49:30,609][20347] Updated weights for policy 0, policy_version 1902063 (0.0007) [2023-12-27 18:49:30,660][20347] Updated weights for policy 0, policy_version 1902073 (0.0008) [2023-12-27 18:49:30,772][20381] Updated weights for policy 1, policy_version 1900481 (0.0010) [2023-12-27 18:49:30,819][20381] Updated weights for policy 1, policy_version 1900491 (0.0005) [2023-12-27 18:49:30,867][20381] Updated weights for policy 1, policy_version 1900501 (0.0005) [2023-12-27 18:49:30,917][20381] Updated weights for policy 1, policy_version 1900511 (0.0005) [2023-12-27 18:49:31,384][20347] Updated weights for policy 0, policy_version 1902083 (0.0008) [2023-12-27 18:49:31,443][20347] Updated weights for policy 0, policy_version 1902093 (0.0007) [2023-12-27 18:49:31,496][20347] Updated weights for policy 0, policy_version 1902103 (0.0007) [2023-12-27 18:49:31,585][20381] Updated weights for policy 1, policy_version 1900521 (0.0010) [2023-12-27 18:49:31,646][20381] Updated weights for policy 1, policy_version 1900531 (0.0011) [2023-12-27 18:49:31,699][20381] Updated weights for policy 1, policy_version 1900541 (0.0011) [2023-12-27 18:49:32,198][20347] Updated weights for policy 0, policy_version 1902113 (0.0007) [2023-12-27 18:49:32,259][20347] Updated weights for policy 0, policy_version 1902123 (0.0007) [2023-12-27 18:49:32,313][20347] Updated weights for policy 0, policy_version 1902133 (0.0007) [2023-12-27 18:49:32,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 973627392. Throughput: 0: 10206.7, 1: 10131.0. Samples: 973643468. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.480')] [2023-12-27 18:49:32,370][20347] Updated weights for policy 0, policy_version 1902143 (0.0007) [2023-12-27 18:49:32,448][20381] Updated weights for policy 1, policy_version 1900551 (0.0011) [2023-12-27 18:49:32,513][20381] Updated weights for policy 1, policy_version 1900561 (0.0011) [2023-12-27 18:49:32,573][20381] Updated weights for policy 1, policy_version 1900571 (0.0011) [2023-12-27 18:49:33,067][20347] Updated weights for policy 0, policy_version 1902153 (0.0006) [2023-12-27 18:49:33,124][20347] Updated weights for policy 0, policy_version 1902163 (0.0005) [2023-12-27 18:49:33,180][20347] Updated weights for policy 0, policy_version 1902173 (0.0005) [2023-12-27 18:49:33,268][20381] Updated weights for policy 1, policy_version 1900581 (0.0010) [2023-12-27 18:49:33,312][20381] Updated weights for policy 1, policy_version 1900591 (0.0010) [2023-12-27 18:49:33,357][20381] Updated weights for policy 1, policy_version 1900601 (0.0010) [2023-12-27 18:49:33,687][20347] Updated weights for policy 0, policy_version 1902183 (0.0008) [2023-12-27 18:49:33,735][20347] Updated weights for policy 0, policy_version 1902193 (0.0010) [2023-12-27 18:49:33,795][20347] Updated weights for policy 0, policy_version 1902203 (0.0010) [2023-12-27 18:49:33,995][20381] Updated weights for policy 1, policy_version 1900611 (0.0010) [2023-12-27 18:49:34,041][20381] Updated weights for policy 1, policy_version 1900621 (0.0008) [2023-12-27 18:49:34,110][20381] Updated weights for policy 1, policy_version 1900631 (0.0005) [2023-12-27 18:49:34,392][20347] Updated weights for policy 0, policy_version 1902213 (0.0010) [2023-12-27 18:49:34,450][20347] Updated weights for policy 0, policy_version 1902223 (0.0010) [2023-12-27 18:49:34,500][20347] Updated weights for policy 0, policy_version 1902233 (0.0009) [2023-12-27 18:49:34,782][20381] Updated weights for policy 1, policy_version 1900641 (0.0007) [2023-12-27 18:49:34,843][20381] Updated weights for policy 1, policy_version 1900651 (0.0011) [2023-12-27 18:49:34,899][20381] Updated weights for policy 1, policy_version 1900661 (0.0010) [2023-12-27 18:49:34,955][20381] Updated weights for policy 1, policy_version 1900671 (0.0011) [2023-12-27 18:49:35,146][20347] Updated weights for policy 0, policy_version 1902243 (0.0010) [2023-12-27 18:49:35,195][20347] Updated weights for policy 0, policy_version 1902253 (0.0010) [2023-12-27 18:49:35,246][20347] Updated weights for policy 0, policy_version 1902263 (0.0010) [2023-12-27 18:49:35,645][20381] Updated weights for policy 1, policy_version 1900681 (0.0010) [2023-12-27 18:49:35,700][20381] Updated weights for policy 1, policy_version 1900691 (0.0009) [2023-12-27 18:49:35,757][20381] Updated weights for policy 1, policy_version 1900701 (0.0009) [2023-12-27 18:49:35,920][20347] Updated weights for policy 0, policy_version 1902273 (0.0010) [2023-12-27 18:49:35,970][20347] Updated weights for policy 0, policy_version 1902283 (0.0010) [2023-12-27 18:49:36,021][20347] Updated weights for policy 0, policy_version 1902293 (0.0008) [2023-12-27 18:49:36,074][20347] Updated weights for policy 0, policy_version 1902303 (0.0008) [2023-12-27 18:49:36,354][20381] Updated weights for policy 1, policy_version 1900711 (0.0011) [2023-12-27 18:49:36,411][20381] Updated weights for policy 1, policy_version 1900721 (0.0011) [2023-12-27 18:49:36,468][20381] Updated weights for policy 1, policy_version 1900731 (0.0011) [2023-12-27 18:49:36,800][20347] Updated weights for policy 0, policy_version 1902313 (0.0005) [2023-12-27 18:49:36,852][20347] Updated weights for policy 0, policy_version 1902323 (0.0007) [2023-12-27 18:49:36,916][20347] Updated weights for policy 0, policy_version 1902333 (0.0006) [2023-12-27 18:49:37,146][20381] Updated weights for policy 1, policy_version 1900741 (0.0011) [2023-12-27 18:49:37,205][20381] Updated weights for policy 1, policy_version 1900751 (0.0011) [2023-12-27 18:49:37,261][20381] Updated weights for policy 1, policy_version 1900761 (0.0006) [2023-12-27 18:49:37,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 973742080. Throughput: 0: 10300.8, 1: 10143.0. Samples: 973707500. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.300')] [2023-12-27 18:49:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001900768_486653952.pth... [2023-12-27 18:49:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001902336_487088128.pth... [2023-12-27 18:49:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001901152_486785024.pth [2023-12-27 18:49:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001899584_486350848.pth [2023-12-27 18:49:37,579][20347] Updated weights for policy 0, policy_version 1902343 (0.0006) [2023-12-27 18:49:37,648][20347] Updated weights for policy 0, policy_version 1902353 (0.0005) [2023-12-27 18:49:37,703][20347] Updated weights for policy 0, policy_version 1902363 (0.0005) [2023-12-27 18:49:37,816][20381] Updated weights for policy 1, policy_version 1900771 (0.0005) [2023-12-27 18:49:37,864][20381] Updated weights for policy 1, policy_version 1900781 (0.0005) [2023-12-27 18:49:37,923][20381] Updated weights for policy 1, policy_version 1900791 (0.0007) [2023-12-27 18:49:38,418][20347] Updated weights for policy 0, policy_version 1902373 (0.0007) [2023-12-27 18:49:38,474][20347] Updated weights for policy 0, policy_version 1902383 (0.0007) [2023-12-27 18:49:38,525][20347] Updated weights for policy 0, policy_version 1902393 (0.0008) [2023-12-27 18:49:38,550][20381] Updated weights for policy 1, policy_version 1900801 (0.0010) [2023-12-27 18:49:38,610][20381] Updated weights for policy 1, policy_version 1900811 (0.0011) [2023-12-27 18:49:38,661][20381] Updated weights for policy 1, policy_version 1900821 (0.0010) [2023-12-27 18:49:38,720][20381] Updated weights for policy 1, policy_version 1900831 (0.0010) [2023-12-27 18:49:39,175][20347] Updated weights for policy 0, policy_version 1902403 (0.0009) [2023-12-27 18:49:39,234][20347] Updated weights for policy 0, policy_version 1902413 (0.0007) [2023-12-27 18:49:39,291][20347] Updated weights for policy 0, policy_version 1902423 (0.0007) [2023-12-27 18:49:39,432][20381] Updated weights for policy 1, policy_version 1900841 (0.0010) [2023-12-27 18:49:39,490][20381] Updated weights for policy 1, policy_version 1900851 (0.0008) [2023-12-27 18:49:39,547][20381] Updated weights for policy 1, policy_version 1900861 (0.0009) [2023-12-27 18:49:40,018][20347] Updated weights for policy 0, policy_version 1902433 (0.0007) [2023-12-27 18:49:40,075][20347] Updated weights for policy 0, policy_version 1902443 (0.0006) [2023-12-27 18:49:40,137][20347] Updated weights for policy 0, policy_version 1902453 (0.0005) [2023-12-27 18:49:40,201][20347] Updated weights for policy 0, policy_version 1902463 (0.0005) [2023-12-27 18:49:40,291][20381] Updated weights for policy 1, policy_version 1900871 (0.0009) [2023-12-27 18:49:40,356][20381] Updated weights for policy 1, policy_version 1900881 (0.0009) [2023-12-27 18:49:40,421][20381] Updated weights for policy 1, policy_version 1900891 (0.0008) [2023-12-27 18:49:40,820][20347] Updated weights for policy 0, policy_version 1902473 (0.0008) [2023-12-27 18:49:40,880][20347] Updated weights for policy 0, policy_version 1902483 (0.0007) [2023-12-27 18:49:40,927][20347] Updated weights for policy 0, policy_version 1902493 (0.0007) [2023-12-27 18:49:41,077][20381] Updated weights for policy 1, policy_version 1900901 (0.0008) [2023-12-27 18:49:41,142][20381] Updated weights for policy 1, policy_version 1900911 (0.0009) [2023-12-27 18:49:41,211][20381] Updated weights for policy 1, policy_version 1900921 (0.0009) [2023-12-27 18:49:41,710][20347] Updated weights for policy 0, policy_version 1902503 (0.0008) [2023-12-27 18:49:41,779][20347] Updated weights for policy 0, policy_version 1902513 (0.0008) [2023-12-27 18:49:41,840][20347] Updated weights for policy 0, policy_version 1902523 (0.0007) [2023-12-27 18:49:42,017][20381] Updated weights for policy 1, policy_version 1900931 (0.0007) [2023-12-27 18:49:42,078][20381] Updated weights for policy 1, policy_version 1900941 (0.0008) [2023-12-27 18:49:42,137][20381] Updated weights for policy 1, policy_version 1900951 (0.0008) [2023-12-27 18:49:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 973840384. Throughput: 0: 10211.7, 1: 10194.1. Samples: 973831108. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.050')] [2023-12-27 18:49:42,573][20347] Updated weights for policy 0, policy_version 1902533 (0.0008) [2023-12-27 18:49:42,631][20347] Updated weights for policy 0, policy_version 1902543 (0.0007) [2023-12-27 18:49:42,690][20347] Updated weights for policy 0, policy_version 1902553 (0.0008) [2023-12-27 18:49:42,843][20381] Updated weights for policy 1, policy_version 1900961 (0.0008) [2023-12-27 18:49:42,895][20381] Updated weights for policy 1, policy_version 1900971 (0.0007) [2023-12-27 18:49:42,954][20381] Updated weights for policy 1, policy_version 1900981 (0.0008) [2023-12-27 18:49:43,000][20381] Updated weights for policy 1, policy_version 1900991 (0.0008) [2023-12-27 18:49:43,356][20347] Updated weights for policy 0, policy_version 1902563 (0.0007) [2023-12-27 18:49:43,421][20347] Updated weights for policy 0, policy_version 1902573 (0.0005) [2023-12-27 18:49:43,475][20347] Updated weights for policy 0, policy_version 1902583 (0.0005) [2023-12-27 18:49:43,720][20381] Updated weights for policy 1, policy_version 1901001 (0.0007) [2023-12-27 18:49:43,774][20381] Updated weights for policy 1, policy_version 1901011 (0.0008) [2023-12-27 18:49:43,831][20381] Updated weights for policy 1, policy_version 1901021 (0.0006) [2023-12-27 18:49:44,046][20347] Updated weights for policy 0, policy_version 1902593 (0.0006) [2023-12-27 18:49:44,100][20347] Updated weights for policy 0, policy_version 1902603 (0.0007) [2023-12-27 18:49:44,152][20347] Updated weights for policy 0, policy_version 1902613 (0.0006) [2023-12-27 18:49:44,210][20347] Updated weights for policy 0, policy_version 1902623 (0.0005) [2023-12-27 18:49:44,555][20381] Updated weights for policy 1, policy_version 1901031 (0.0007) [2023-12-27 18:49:44,614][20381] Updated weights for policy 1, policy_version 1901041 (0.0007) [2023-12-27 18:49:44,662][20381] Updated weights for policy 1, policy_version 1901051 (0.0007) [2023-12-27 18:49:44,826][20347] Updated weights for policy 0, policy_version 1902633 (0.0007) [2023-12-27 18:49:44,886][20347] Updated weights for policy 0, policy_version 1902643 (0.0007) [2023-12-27 18:49:44,938][20347] Updated weights for policy 0, policy_version 1902653 (0.0008) [2023-12-27 18:49:45,386][20381] Updated weights for policy 1, policy_version 1901061 (0.0007) [2023-12-27 18:49:45,446][20381] Updated weights for policy 1, policy_version 1901071 (0.0008) [2023-12-27 18:49:45,502][20381] Updated weights for policy 1, policy_version 1901081 (0.0007) [2023-12-27 18:49:45,665][20347] Updated weights for policy 0, policy_version 1902663 (0.0008) [2023-12-27 18:49:45,723][20347] Updated weights for policy 0, policy_version 1902673 (0.0007) [2023-12-27 18:49:45,772][20347] Updated weights for policy 0, policy_version 1902683 (0.0008) [2023-12-27 18:49:46,193][20381] Updated weights for policy 1, policy_version 1901091 (0.0009) [2023-12-27 18:49:46,249][20381] Updated weights for policy 1, policy_version 1901101 (0.0008) [2023-12-27 18:49:46,311][20381] Updated weights for policy 1, policy_version 1901111 (0.0008) [2023-12-27 18:49:46,390][20347] Updated weights for policy 0, policy_version 1902693 (0.0008) [2023-12-27 18:49:46,445][20347] Updated weights for policy 0, policy_version 1902703 (0.0008) [2023-12-27 18:49:46,500][20347] Updated weights for policy 0, policy_version 1902713 (0.0008) [2023-12-27 18:49:47,001][20381] Updated weights for policy 1, policy_version 1901121 (0.0009) [2023-12-27 18:49:47,054][20381] Updated weights for policy 1, policy_version 1901131 (0.0007) [2023-12-27 18:49:47,114][20381] Updated weights for policy 1, policy_version 1901141 (0.0009) [2023-12-27 18:49:47,168][20381] Updated weights for policy 1, policy_version 1901151 (0.0007) [2023-12-27 18:49:47,189][20347] Updated weights for policy 0, policy_version 1902723 (0.0007) [2023-12-27 18:49:47,239][20347] Updated weights for policy 0, policy_version 1902733 (0.0005) [2023-12-27 18:49:47,291][20347] Updated weights for policy 0, policy_version 1902743 (0.0008) [2023-12-27 18:49:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 973946880. Throughput: 0: 10250.5, 1: 10195.5. Samples: 973951200. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.130')] [2023-12-27 18:49:47,887][20381] Updated weights for policy 1, policy_version 1901161 (0.0007) [2023-12-27 18:49:47,917][20347] Updated weights for policy 0, policy_version 1902753 (0.0007) [2023-12-27 18:49:47,946][20381] Updated weights for policy 1, policy_version 1901171 (0.0008) [2023-12-27 18:49:47,966][20347] Updated weights for policy 0, policy_version 1902763 (0.0010) [2023-12-27 18:49:47,998][20381] Updated weights for policy 1, policy_version 1901181 (0.0006) [2023-12-27 18:49:48,020][20347] Updated weights for policy 0, policy_version 1902773 (0.0010) [2023-12-27 18:49:48,075][20347] Updated weights for policy 0, policy_version 1902783 (0.0005) [2023-12-27 18:49:48,675][20381] Updated weights for policy 1, policy_version 1901191 (0.0008) [2023-12-27 18:49:48,705][20347] Updated weights for policy 0, policy_version 1902793 (0.0010) [2023-12-27 18:49:48,734][20381] Updated weights for policy 1, policy_version 1901201 (0.0007) [2023-12-27 18:49:48,758][20347] Updated weights for policy 0, policy_version 1902803 (0.0011) [2023-12-27 18:49:48,787][20381] Updated weights for policy 1, policy_version 1901211 (0.0006) [2023-12-27 18:49:48,813][20347] Updated weights for policy 0, policy_version 1902813 (0.0006) [2023-12-27 18:49:49,513][20381] Updated weights for policy 1, policy_version 1901221 (0.0007) [2023-12-27 18:49:49,533][20347] Updated weights for policy 0, policy_version 1902823 (0.0008) [2023-12-27 18:49:49,578][20381] Updated weights for policy 1, policy_version 1901231 (0.0008) [2023-12-27 18:49:49,586][20347] Updated weights for policy 0, policy_version 1902833 (0.0009) [2023-12-27 18:49:49,634][20381] Updated weights for policy 1, policy_version 1901241 (0.0006) [2023-12-27 18:49:49,636][20347] Updated weights for policy 0, policy_version 1902843 (0.0006) [2023-12-27 18:49:50,283][20381] Updated weights for policy 1, policy_version 1901251 (0.0006) [2023-12-27 18:49:50,337][20381] Updated weights for policy 1, policy_version 1901261 (0.0008) [2023-12-27 18:49:50,365][20347] Updated weights for policy 0, policy_version 1902853 (0.0006) [2023-12-27 18:49:50,392][20381] Updated weights for policy 1, policy_version 1901271 (0.0006) [2023-12-27 18:49:50,417][20347] Updated weights for policy 0, policy_version 1902863 (0.0005) [2023-12-27 18:49:50,474][20347] Updated weights for policy 0, policy_version 1902873 (0.0006) [2023-12-27 18:49:51,097][20381] Updated weights for policy 1, policy_version 1901281 (0.0007) [2023-12-27 18:49:51,161][20381] Updated weights for policy 1, policy_version 1901291 (0.0008) [2023-12-27 18:49:51,167][20347] Updated weights for policy 0, policy_version 1902883 (0.0007) [2023-12-27 18:49:51,215][20381] Updated weights for policy 1, policy_version 1901301 (0.0005) [2023-12-27 18:49:51,222][20347] Updated weights for policy 0, policy_version 1902893 (0.0007) [2023-12-27 18:49:51,278][20381] Updated weights for policy 1, policy_version 1901311 (0.0007) [2023-12-27 18:49:51,287][20347] Updated weights for policy 0, policy_version 1902903 (0.0006) [2023-12-27 18:49:52,010][20381] Updated weights for policy 1, policy_version 1901321 (0.0010) [2023-12-27 18:49:52,022][20347] Updated weights for policy 0, policy_version 1902913 (0.0008) [2023-12-27 18:49:52,060][20381] Updated weights for policy 1, policy_version 1901331 (0.0011) [2023-12-27 18:49:52,078][20347] Updated weights for policy 0, policy_version 1902923 (0.0007) [2023-12-27 18:49:52,114][20381] Updated weights for policy 1, policy_version 1901341 (0.0011) [2023-12-27 18:49:52,134][20347] Updated weights for policy 0, policy_version 1902933 (0.0005) [2023-12-27 18:49:52,187][20347] Updated weights for policy 0, policy_version 1902943 (0.0006) [2023-12-27 18:49:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 974045184. Throughput: 0: 10261.7, 1: 10136.4. Samples: 974012600. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.580')] [2023-12-27 18:49:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001901344_486801408.pth... [2023-12-27 18:49:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001902944_487243776.pth... [2023-12-27 18:49:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001901728_486932480.pth [2023-12-27 18:49:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001900160_486498304.pth [2023-12-27 18:49:52,824][20381] Updated weights for policy 1, policy_version 1901351 (0.0008) [2023-12-27 18:49:52,871][20381] Updated weights for policy 1, policy_version 1901361 (0.0008) [2023-12-27 18:49:52,915][20381] Updated weights for policy 1, policy_version 1901371 (0.0007) [2023-12-27 18:49:52,931][20347] Updated weights for policy 0, policy_version 1902953 (0.0007) [2023-12-27 18:49:52,982][20347] Updated weights for policy 0, policy_version 1902963 (0.0007) [2023-12-27 18:49:53,035][20347] Updated weights for policy 0, policy_version 1902973 (0.0009) [2023-12-27 18:49:53,654][20381] Updated weights for policy 1, policy_version 1901381 (0.0007) [2023-12-27 18:49:53,709][20381] Updated weights for policy 1, policy_version 1901391 (0.0007) [2023-12-27 18:49:53,759][20347] Updated weights for policy 0, policy_version 1902983 (0.0008) [2023-12-27 18:49:53,768][20381] Updated weights for policy 1, policy_version 1901401 (0.0006) [2023-12-27 18:49:53,815][20347] Updated weights for policy 0, policy_version 1902993 (0.0007) [2023-12-27 18:49:53,864][20347] Updated weights for policy 0, policy_version 1903003 (0.0008) [2023-12-27 18:49:54,371][20381] Updated weights for policy 1, policy_version 1901411 (0.0006) [2023-12-27 18:49:54,429][20381] Updated weights for policy 1, policy_version 1901421 (0.0007) [2023-12-27 18:49:54,490][20381] Updated weights for policy 1, policy_version 1901431 (0.0008) [2023-12-27 18:49:54,565][20347] Updated weights for policy 0, policy_version 1903013 (0.0007) [2023-12-27 18:49:54,617][20347] Updated weights for policy 0, policy_version 1903023 (0.0005) [2023-12-27 18:49:54,661][20347] Updated weights for policy 0, policy_version 1903033 (0.0005) [2023-12-27 18:49:55,174][20381] Updated weights for policy 1, policy_version 1901441 (0.0007) [2023-12-27 18:49:55,221][20381] Updated weights for policy 1, policy_version 1901451 (0.0007) [2023-12-27 18:49:55,225][20347] Updated weights for policy 0, policy_version 1903043 (0.0007) [2023-12-27 18:49:55,267][20381] Updated weights for policy 1, policy_version 1901461 (0.0006) [2023-12-27 18:49:55,283][20347] Updated weights for policy 0, policy_version 1903053 (0.0010) [2023-12-27 18:49:55,311][20381] Updated weights for policy 1, policy_version 1901471 (0.0007) [2023-12-27 18:49:55,338][20347] Updated weights for policy 0, policy_version 1903063 (0.0009) [2023-12-27 18:49:55,935][20347] Updated weights for policy 0, policy_version 1903073 (0.0006) [2023-12-27 18:49:55,993][20347] Updated weights for policy 0, policy_version 1903083 (0.0010) [2023-12-27 18:49:56,048][20347] Updated weights for policy 0, policy_version 1903093 (0.0010) [2023-12-27 18:49:56,076][20381] Updated weights for policy 1, policy_version 1901481 (0.0006) [2023-12-27 18:49:56,100][20347] Updated weights for policy 0, policy_version 1903103 (0.0010) [2023-12-27 18:49:56,135][20381] Updated weights for policy 1, policy_version 1901491 (0.0006) [2023-12-27 18:49:56,189][20381] Updated weights for policy 1, policy_version 1901501 (0.0007) [2023-12-27 18:49:56,682][20347] Updated weights for policy 0, policy_version 1903113 (0.0006) [2023-12-27 18:49:56,737][20347] Updated weights for policy 0, policy_version 1903123 (0.0005) [2023-12-27 18:49:56,791][20347] Updated weights for policy 0, policy_version 1903133 (0.0007) [2023-12-27 18:49:56,914][20381] Updated weights for policy 1, policy_version 1901511 (0.0008) [2023-12-27 18:49:56,969][20381] Updated weights for policy 1, policy_version 1901521 (0.0007) [2023-12-27 18:49:57,034][20381] Updated weights for policy 1, policy_version 1901531 (0.0008) [2023-12-27 18:49:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 974143488. Throughput: 0: 10262.0, 1: 10131.2. Samples: 974134436. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:49:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.520')] [2023-12-27 18:49:57,393][20347] Updated weights for policy 0, policy_version 1903143 (0.0007) [2023-12-27 18:49:57,438][20347] Updated weights for policy 0, policy_version 1903153 (0.0007) [2023-12-27 18:49:57,483][20347] Updated weights for policy 0, policy_version 1903163 (0.0008) [2023-12-27 18:49:57,732][20381] Updated weights for policy 1, policy_version 1901541 (0.0007) [2023-12-27 18:49:57,787][20381] Updated weights for policy 1, policy_version 1901551 (0.0008) [2023-12-27 18:49:57,851][20381] Updated weights for policy 1, policy_version 1901561 (0.0008) [2023-12-27 18:49:58,191][20347] Updated weights for policy 0, policy_version 1903173 (0.0007) [2023-12-27 18:49:58,257][20347] Updated weights for policy 0, policy_version 1903183 (0.0008) [2023-12-27 18:49:58,320][20347] Updated weights for policy 0, policy_version 1903193 (0.0007) [2023-12-27 18:49:58,594][20381] Updated weights for policy 1, policy_version 1901571 (0.0008) [2023-12-27 18:49:58,659][20381] Updated weights for policy 1, policy_version 1901581 (0.0007) [2023-12-27 18:49:58,721][20381] Updated weights for policy 1, policy_version 1901591 (0.0006) [2023-12-27 18:49:59,033][20347] Updated weights for policy 0, policy_version 1903203 (0.0007) [2023-12-27 18:49:59,094][20347] Updated weights for policy 0, policy_version 1903213 (0.0008) [2023-12-27 18:49:59,148][20347] Updated weights for policy 0, policy_version 1903223 (0.0008) [2023-12-27 18:49:59,518][20381] Updated weights for policy 1, policy_version 1901601 (0.0008) [2023-12-27 18:49:59,579][20381] Updated weights for policy 1, policy_version 1901611 (0.0010) [2023-12-27 18:49:59,638][20381] Updated weights for policy 1, policy_version 1901621 (0.0010) [2023-12-27 18:49:59,698][20381] Updated weights for policy 1, policy_version 1901631 (0.0010) [2023-12-27 18:49:59,866][20347] Updated weights for policy 0, policy_version 1903233 (0.0008) [2023-12-27 18:49:59,926][20347] Updated weights for policy 0, policy_version 1903243 (0.0007) [2023-12-27 18:49:59,983][20347] Updated weights for policy 0, policy_version 1903253 (0.0007) [2023-12-27 18:50:00,032][20347] Updated weights for policy 0, policy_version 1903263 (0.0008) [2023-12-27 18:50:00,417][20381] Updated weights for policy 1, policy_version 1901641 (0.0008) [2023-12-27 18:50:00,472][20381] Updated weights for policy 1, policy_version 1901651 (0.0007) [2023-12-27 18:50:00,532][20381] Updated weights for policy 1, policy_version 1901661 (0.0005) [2023-12-27 18:50:00,712][20347] Updated weights for policy 0, policy_version 1903273 (0.0007) [2023-12-27 18:50:00,777][20347] Updated weights for policy 0, policy_version 1903283 (0.0008) [2023-12-27 18:50:00,821][20347] Updated weights for policy 0, policy_version 1903293 (0.0008) [2023-12-27 18:50:01,201][20381] Updated weights for policy 1, policy_version 1901671 (0.0006) [2023-12-27 18:50:01,263][20381] Updated weights for policy 1, policy_version 1901681 (0.0008) [2023-12-27 18:50:01,322][20381] Updated weights for policy 1, policy_version 1901691 (0.0008) [2023-12-27 18:50:01,550][20347] Updated weights for policy 0, policy_version 1903303 (0.0007) [2023-12-27 18:50:01,605][20347] Updated weights for policy 0, policy_version 1903313 (0.0006) [2023-12-27 18:50:01,670][20347] Updated weights for policy 0, policy_version 1903323 (0.0007) [2023-12-27 18:50:01,993][20381] Updated weights for policy 1, policy_version 1901701 (0.0007) [2023-12-27 18:50:02,063][20381] Updated weights for policy 1, policy_version 1901711 (0.0006) [2023-12-27 18:50:02,123][20381] Updated weights for policy 1, policy_version 1901721 (0.0007) [2023-12-27 18:50:02,313][20347] Updated weights for policy 0, policy_version 1903333 (0.0007) [2023-12-27 18:50:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 974241792. Throughput: 0: 10317.1, 1: 9997.7. Samples: 974253508. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.040')] [2023-12-27 18:50:02,374][20347] Updated weights for policy 0, policy_version 1903343 (0.0006) [2023-12-27 18:50:02,435][20347] Updated weights for policy 0, policy_version 1903353 (0.0007) [2023-12-27 18:50:02,798][20381] Updated weights for policy 1, policy_version 1901731 (0.0007) [2023-12-27 18:50:02,849][20381] Updated weights for policy 1, policy_version 1901741 (0.0008) [2023-12-27 18:50:02,899][20381] Updated weights for policy 1, policy_version 1901751 (0.0008) [2023-12-27 18:50:03,044][20347] Updated weights for policy 0, policy_version 1903363 (0.0006) [2023-12-27 18:50:03,095][20347] Updated weights for policy 0, policy_version 1903373 (0.0005) [2023-12-27 18:50:03,160][20347] Updated weights for policy 0, policy_version 1903383 (0.0008) [2023-12-27 18:50:03,629][20381] Updated weights for policy 1, policy_version 1901761 (0.0008) [2023-12-27 18:50:03,686][20381] Updated weights for policy 1, policy_version 1901771 (0.0008) [2023-12-27 18:50:03,736][20347] Updated weights for policy 0, policy_version 1903393 (0.0007) [2023-12-27 18:50:03,742][20381] Updated weights for policy 1, policy_version 1901781 (0.0008) [2023-12-27 18:50:03,787][20347] Updated weights for policy 0, policy_version 1903403 (0.0005) [2023-12-27 18:50:03,801][20381] Updated weights for policy 1, policy_version 1901791 (0.0008) [2023-12-27 18:50:03,871][20347] Updated weights for policy 0, policy_version 1903413 (0.0006) [2023-12-27 18:50:03,929][20347] Updated weights for policy 0, policy_version 1903423 (0.0006) [2023-12-27 18:50:04,523][20381] Updated weights for policy 1, policy_version 1901801 (0.0007) [2023-12-27 18:50:04,577][20381] Updated weights for policy 1, policy_version 1901811 (0.0006) [2023-12-27 18:50:04,579][20347] Updated weights for policy 0, policy_version 1903433 (0.0007) [2023-12-27 18:50:04,629][20347] Updated weights for policy 0, policy_version 1903443 (0.0006) [2023-12-27 18:50:04,631][20381] Updated weights for policy 1, policy_version 1901821 (0.0005) [2023-12-27 18:50:04,677][20347] Updated weights for policy 0, policy_version 1903453 (0.0006) [2023-12-27 18:50:05,294][20347] Updated weights for policy 0, policy_version 1903463 (0.0006) [2023-12-27 18:50:05,340][20381] Updated weights for policy 1, policy_version 1901831 (0.0006) [2023-12-27 18:50:05,353][20347] Updated weights for policy 0, policy_version 1903473 (0.0006) [2023-12-27 18:50:05,396][20381] Updated weights for policy 1, policy_version 1901841 (0.0005) [2023-12-27 18:50:05,402][20347] Updated weights for policy 0, policy_version 1903483 (0.0005) [2023-12-27 18:50:05,456][20381] Updated weights for policy 1, policy_version 1901851 (0.0007) [2023-12-27 18:50:06,013][20347] Updated weights for policy 0, policy_version 1903493 (0.0005) [2023-12-27 18:50:06,075][20347] Updated weights for policy 0, policy_version 1903503 (0.0005) [2023-12-27 18:50:06,140][20347] Updated weights for policy 0, policy_version 1903513 (0.0008) [2023-12-27 18:50:06,153][20381] Updated weights for policy 1, policy_version 1901861 (0.0007) [2023-12-27 18:50:06,213][20381] Updated weights for policy 1, policy_version 1901871 (0.0008) [2023-12-27 18:50:06,266][20381] Updated weights for policy 1, policy_version 1901881 (0.0009) [2023-12-27 18:50:06,702][20347] Updated weights for policy 0, policy_version 1903523 (0.0006) [2023-12-27 18:50:06,759][20347] Updated weights for policy 0, policy_version 1903533 (0.0008) [2023-12-27 18:50:06,814][20347] Updated weights for policy 0, policy_version 1903543 (0.0008) [2023-12-27 18:50:06,988][20381] Updated weights for policy 1, policy_version 1901891 (0.0008) [2023-12-27 18:50:07,038][20381] Updated weights for policy 1, policy_version 1901901 (0.0008) [2023-12-27 18:50:07,088][20381] Updated weights for policy 1, policy_version 1901911 (0.0007) [2023-12-27 18:50:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 974348288. Throughput: 0: 10411.2, 1: 9987.6. Samples: 974315972. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '970.380')] [2023-12-27 18:50:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001901920_486948864.pth... [2023-12-27 18:50:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001903552_487399424.pth... [2023-12-27 18:50:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001900768_486653952.pth [2023-12-27 18:50:07,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001902336_487088128.pth [2023-12-27 18:50:07,459][20347] Updated weights for policy 0, policy_version 1903553 (0.0007) [2023-12-27 18:50:07,513][20347] Updated weights for policy 0, policy_version 1903563 (0.0005) [2023-12-27 18:50:07,559][20347] Updated weights for policy 0, policy_version 1903573 (0.0005) [2023-12-27 18:50:07,614][20347] Updated weights for policy 0, policy_version 1903583 (0.0005) [2023-12-27 18:50:07,650][20381] Updated weights for policy 1, policy_version 1901921 (0.0005) [2023-12-27 18:50:07,702][20381] Updated weights for policy 1, policy_version 1901931 (0.0006) [2023-12-27 18:50:07,757][20381] Updated weights for policy 1, policy_version 1901941 (0.0008) [2023-12-27 18:50:07,810][20381] Updated weights for policy 1, policy_version 1901951 (0.0008) [2023-12-27 18:50:08,245][20347] Updated weights for policy 0, policy_version 1903593 (0.0006) [2023-12-27 18:50:08,315][20347] Updated weights for policy 0, policy_version 1903603 (0.0005) [2023-12-27 18:50:08,377][20347] Updated weights for policy 0, policy_version 1903613 (0.0007) [2023-12-27 18:50:08,540][20381] Updated weights for policy 1, policy_version 1901961 (0.0008) [2023-12-27 18:50:08,605][20381] Updated weights for policy 1, policy_version 1901971 (0.0008) [2023-12-27 18:50:08,669][20381] Updated weights for policy 1, policy_version 1901981 (0.0008) [2023-12-27 18:50:08,994][20347] Updated weights for policy 0, policy_version 1903623 (0.0006) [2023-12-27 18:50:09,055][20347] Updated weights for policy 0, policy_version 1903633 (0.0007) [2023-12-27 18:50:09,117][20347] Updated weights for policy 0, policy_version 1903643 (0.0007) [2023-12-27 18:50:09,378][20381] Updated weights for policy 1, policy_version 1901991 (0.0007) [2023-12-27 18:50:09,449][20381] Updated weights for policy 1, policy_version 1902001 (0.0007) [2023-12-27 18:50:09,509][20381] Updated weights for policy 1, policy_version 1902011 (0.0007) [2023-12-27 18:50:09,807][20347] Updated weights for policy 0, policy_version 1903653 (0.0008) [2023-12-27 18:50:09,882][20347] Updated weights for policy 0, policy_version 1903663 (0.0009) [2023-12-27 18:50:09,947][20347] Updated weights for policy 0, policy_version 1903673 (0.0008) [2023-12-27 18:50:10,164][20381] Updated weights for policy 1, policy_version 1902021 (0.0007) [2023-12-27 18:50:10,222][20381] Updated weights for policy 1, policy_version 1902031 (0.0007) [2023-12-27 18:50:10,277][20381] Updated weights for policy 1, policy_version 1902041 (0.0008) [2023-12-27 18:50:10,652][20347] Updated weights for policy 0, policy_version 1903683 (0.0006) [2023-12-27 18:50:10,702][20347] Updated weights for policy 0, policy_version 1903693 (0.0009) [2023-12-27 18:50:10,751][20347] Updated weights for policy 0, policy_version 1903703 (0.0009) [2023-12-27 18:50:10,887][20381] Updated weights for policy 1, policy_version 1902051 (0.0008) [2023-12-27 18:50:10,936][20381] Updated weights for policy 1, policy_version 1902061 (0.0007) [2023-12-27 18:50:10,992][20381] Updated weights for policy 1, policy_version 1902071 (0.0007) [2023-12-27 18:50:11,560][20347] Updated weights for policy 0, policy_version 1903713 (0.0008) [2023-12-27 18:50:11,623][20347] Updated weights for policy 0, policy_version 1903723 (0.0007) [2023-12-27 18:50:11,685][20347] Updated weights for policy 0, policy_version 1903733 (0.0007) [2023-12-27 18:50:11,755][20347] Updated weights for policy 0, policy_version 1903743 (0.0007) [2023-12-27 18:50:11,758][20381] Updated weights for policy 1, policy_version 1902081 (0.0008) [2023-12-27 18:50:11,817][20381] Updated weights for policy 1, policy_version 1902091 (0.0008) [2023-12-27 18:50:11,875][20381] Updated weights for policy 1, policy_version 1902101 (0.0007) [2023-12-27 18:50:11,929][20381] Updated weights for policy 1, policy_version 1902111 (0.0007) [2023-12-27 18:50:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 974446592. Throughput: 0: 10467.8, 1: 10011.0. Samples: 974440660. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.260')] [2023-12-27 18:50:12,503][20347] Updated weights for policy 0, policy_version 1903753 (0.0008) [2023-12-27 18:50:12,555][20347] Updated weights for policy 0, policy_version 1903763 (0.0008) [2023-12-27 18:50:12,603][20347] Updated weights for policy 0, policy_version 1903773 (0.0008) [2023-12-27 18:50:12,644][20381] Updated weights for policy 1, policy_version 1902121 (0.0007) [2023-12-27 18:50:12,700][20381] Updated weights for policy 1, policy_version 1902131 (0.0007) [2023-12-27 18:50:12,751][20381] Updated weights for policy 1, policy_version 1902141 (0.0007) [2023-12-27 18:50:13,377][20347] Updated weights for policy 0, policy_version 1903783 (0.0008) [2023-12-27 18:50:13,399][20381] Updated weights for policy 1, policy_version 1902151 (0.0008) [2023-12-27 18:50:13,440][20347] Updated weights for policy 0, policy_version 1903793 (0.0007) [2023-12-27 18:50:13,446][20381] Updated weights for policy 1, policy_version 1902161 (0.0006) [2023-12-27 18:50:13,493][20347] Updated weights for policy 0, policy_version 1903803 (0.0007) [2023-12-27 18:50:13,496][20381] Updated weights for policy 1, policy_version 1902171 (0.0005) [2023-12-27 18:50:14,113][20381] Updated weights for policy 1, policy_version 1902181 (0.0006) [2023-12-27 18:50:14,162][20381] Updated weights for policy 1, policy_version 1902191 (0.0008) [2023-12-27 18:50:14,210][20347] Updated weights for policy 0, policy_version 1903813 (0.0007) [2023-12-27 18:50:14,215][20381] Updated weights for policy 1, policy_version 1902201 (0.0006) [2023-12-27 18:50:14,267][20347] Updated weights for policy 0, policy_version 1903823 (0.0007) [2023-12-27 18:50:14,322][20347] Updated weights for policy 0, policy_version 1903833 (0.0008) [2023-12-27 18:50:14,892][20381] Updated weights for policy 1, policy_version 1902211 (0.0006) [2023-12-27 18:50:14,947][20381] Updated weights for policy 1, policy_version 1902221 (0.0008) [2023-12-27 18:50:15,008][20381] Updated weights for policy 1, policy_version 1902231 (0.0007) [2023-12-27 18:50:15,082][20347] Updated weights for policy 0, policy_version 1903843 (0.0008) [2023-12-27 18:50:15,135][20347] Updated weights for policy 0, policy_version 1903853 (0.0008) [2023-12-27 18:50:15,187][20347] Updated weights for policy 0, policy_version 1903863 (0.0008) [2023-12-27 18:50:15,680][20381] Updated weights for policy 1, policy_version 1902241 (0.0007) [2023-12-27 18:50:15,728][20381] Updated weights for policy 1, policy_version 1902251 (0.0008) [2023-12-27 18:50:15,784][20381] Updated weights for policy 1, policy_version 1902261 (0.0007) [2023-12-27 18:50:15,839][20381] Updated weights for policy 1, policy_version 1902271 (0.0007) [2023-12-27 18:50:15,920][20347] Updated weights for policy 0, policy_version 1903873 (0.0008) [2023-12-27 18:50:15,981][20347] Updated weights for policy 0, policy_version 1903883 (0.0006) [2023-12-27 18:50:16,032][20347] Updated weights for policy 0, policy_version 1903893 (0.0008) [2023-12-27 18:50:16,081][20347] Updated weights for policy 0, policy_version 1903903 (0.0008) [2023-12-27 18:50:16,530][20381] Updated weights for policy 1, policy_version 1902281 (0.0008) [2023-12-27 18:50:16,586][20381] Updated weights for policy 1, policy_version 1902291 (0.0008) [2023-12-27 18:50:16,641][20381] Updated weights for policy 1, policy_version 1902301 (0.0007) [2023-12-27 18:50:16,770][20347] Updated weights for policy 0, policy_version 1903913 (0.0008) [2023-12-27 18:50:16,819][20347] Updated weights for policy 0, policy_version 1903923 (0.0007) [2023-12-27 18:50:16,871][20347] Updated weights for policy 0, policy_version 1903933 (0.0008) [2023-12-27 18:50:17,309][20381] Updated weights for policy 1, policy_version 1902311 (0.0007) [2023-12-27 18:50:17,351][20381] Updated weights for policy 1, policy_version 1902321 (0.0007) [2023-12-27 18:50:17,362][19170] Fps is (10 sec: 19661.4, 60 sec: 20207.1, 300 sec: 20466.1). Total num frames: 974544896. Throughput: 0: 10308.1, 1: 10046.2. Samples: 974559408. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.420')] [2023-12-27 18:50:17,403][20381] Updated weights for policy 1, policy_version 1902331 (0.0008) [2023-12-27 18:50:17,584][20347] Updated weights for policy 0, policy_version 1903943 (0.0007) [2023-12-27 18:50:17,629][20347] Updated weights for policy 0, policy_version 1903953 (0.0007) [2023-12-27 18:50:17,690][20347] Updated weights for policy 0, policy_version 1903963 (0.0008) [2023-12-27 18:50:18,124][20381] Updated weights for policy 1, policy_version 1902341 (0.0007) [2023-12-27 18:50:18,169][20381] Updated weights for policy 1, policy_version 1902351 (0.0007) [2023-12-27 18:50:18,226][20381] Updated weights for policy 1, policy_version 1902361 (0.0005) [2023-12-27 18:50:18,399][20347] Updated weights for policy 0, policy_version 1903973 (0.0008) [2023-12-27 18:50:18,459][20347] Updated weights for policy 0, policy_version 1903983 (0.0008) [2023-12-27 18:50:18,515][20347] Updated weights for policy 0, policy_version 1903993 (0.0008) [2023-12-27 18:50:18,916][20381] Updated weights for policy 1, policy_version 1902371 (0.0006) [2023-12-27 18:50:18,968][20381] Updated weights for policy 1, policy_version 1902381 (0.0008) [2023-12-27 18:50:19,013][20381] Updated weights for policy 1, policy_version 1902391 (0.0008) [2023-12-27 18:50:19,130][20347] Updated weights for policy 0, policy_version 1904003 (0.0006) [2023-12-27 18:50:19,184][20347] Updated weights for policy 0, policy_version 1904013 (0.0008) [2023-12-27 18:50:19,251][20347] Updated weights for policy 0, policy_version 1904023 (0.0009) [2023-12-27 18:50:19,800][20381] Updated weights for policy 1, policy_version 1902401 (0.0008) [2023-12-27 18:50:19,873][20381] Updated weights for policy 1, policy_version 1902411 (0.0008) [2023-12-27 18:50:19,938][20381] Updated weights for policy 1, policy_version 1902421 (0.0006) [2023-12-27 18:50:19,968][20347] Updated weights for policy 0, policy_version 1904033 (0.0008) [2023-12-27 18:50:19,993][20381] Updated weights for policy 1, policy_version 1902431 (0.0007) [2023-12-27 18:50:20,023][20347] Updated weights for policy 0, policy_version 1904043 (0.0006) [2023-12-27 18:50:20,084][20347] Updated weights for policy 0, policy_version 1904053 (0.0007) [2023-12-27 18:50:20,146][20347] Updated weights for policy 0, policy_version 1904063 (0.0008) [2023-12-27 18:50:20,657][20381] Updated weights for policy 1, policy_version 1902441 (0.0008) [2023-12-27 18:50:20,720][20381] Updated weights for policy 1, policy_version 1902451 (0.0008) [2023-12-27 18:50:20,776][20381] Updated weights for policy 1, policy_version 1902461 (0.0007) [2023-12-27 18:50:20,883][20347] Updated weights for policy 0, policy_version 1904073 (0.0010) [2023-12-27 18:50:20,939][20347] Updated weights for policy 0, policy_version 1904083 (0.0010) [2023-12-27 18:50:20,999][20347] Updated weights for policy 0, policy_version 1904093 (0.0010) [2023-12-27 18:50:21,488][20381] Updated weights for policy 1, policy_version 1902471 (0.0006) [2023-12-27 18:50:21,543][20381] Updated weights for policy 1, policy_version 1902481 (0.0009) [2023-12-27 18:50:21,597][20381] Updated weights for policy 1, policy_version 1902491 (0.0011) [2023-12-27 18:50:21,723][20347] Updated weights for policy 0, policy_version 1904103 (0.0008) [2023-12-27 18:50:21,785][20347] Updated weights for policy 0, policy_version 1904113 (0.0008) [2023-12-27 18:50:21,850][20347] Updated weights for policy 0, policy_version 1904123 (0.0008) [2023-12-27 18:50:22,273][20381] Updated weights for policy 1, policy_version 1902501 (0.0008) [2023-12-27 18:50:22,328][20381] Updated weights for policy 1, policy_version 1902511 (0.0009) [2023-12-27 18:50:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 974643200. Throughput: 0: 10244.5, 1: 10011.7. Samples: 974619032. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.620')] [2023-12-27 18:50:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001904128_487546880.pth... [2023-12-27 18:50:22,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001902944_487243776.pth [2023-12-27 18:50:22,390][20381] Updated weights for policy 1, policy_version 1902521 (0.0008) [2023-12-27 18:50:22,429][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001902528_487104512.pth... [2023-12-27 18:50:22,432][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001901344_486801408.pth [2023-12-27 18:50:22,545][20347] Updated weights for policy 0, policy_version 1904133 (0.0008) [2023-12-27 18:50:22,611][20347] Updated weights for policy 0, policy_version 1904143 (0.0008) [2023-12-27 18:50:22,675][20347] Updated weights for policy 0, policy_version 1904153 (0.0007) [2023-12-27 18:50:23,094][20381] Updated weights for policy 1, policy_version 1902531 (0.0007) [2023-12-27 18:50:23,156][20381] Updated weights for policy 1, policy_version 1902541 (0.0007) [2023-12-27 18:50:23,213][20381] Updated weights for policy 1, policy_version 1902551 (0.0008) [2023-12-27 18:50:23,381][20347] Updated weights for policy 0, policy_version 1904163 (0.0009) [2023-12-27 18:50:23,442][20347] Updated weights for policy 0, policy_version 1904173 (0.0008) [2023-12-27 18:50:23,498][20347] Updated weights for policy 0, policy_version 1904183 (0.0008) [2023-12-27 18:50:23,913][20381] Updated weights for policy 1, policy_version 1902561 (0.0008) [2023-12-27 18:50:23,971][20381] Updated weights for policy 1, policy_version 1902571 (0.0008) [2023-12-27 18:50:24,032][20381] Updated weights for policy 1, policy_version 1902581 (0.0008) [2023-12-27 18:50:24,085][20381] Updated weights for policy 1, policy_version 1902591 (0.0008) [2023-12-27 18:50:24,203][20347] Updated weights for policy 0, policy_version 1904193 (0.0007) [2023-12-27 18:50:24,254][20347] Updated weights for policy 0, policy_version 1904203 (0.0007) [2023-12-27 18:50:24,300][20347] Updated weights for policy 0, policy_version 1904213 (0.0008) [2023-12-27 18:50:24,355][20347] Updated weights for policy 0, policy_version 1904223 (0.0008) [2023-12-27 18:50:24,734][20381] Updated weights for policy 1, policy_version 1902601 (0.0008) [2023-12-27 18:50:24,782][20381] Updated weights for policy 1, policy_version 1902611 (0.0008) [2023-12-27 18:50:24,832][20381] Updated weights for policy 1, policy_version 1902621 (0.0005) [2023-12-27 18:50:25,029][20347] Updated weights for policy 0, policy_version 1904233 (0.0010) [2023-12-27 18:50:25,076][20347] Updated weights for policy 0, policy_version 1904243 (0.0010) [2023-12-27 18:50:25,124][20347] Updated weights for policy 0, policy_version 1904253 (0.0005) [2023-12-27 18:50:25,505][20381] Updated weights for policy 1, policy_version 1902631 (0.0007) [2023-12-27 18:50:25,559][20381] Updated weights for policy 1, policy_version 1902641 (0.0007) [2023-12-27 18:50:25,607][20381] Updated weights for policy 1, policy_version 1902651 (0.0007) [2023-12-27 18:50:25,795][20347] Updated weights for policy 0, policy_version 1904263 (0.0007) [2023-12-27 18:50:25,855][20347] Updated weights for policy 0, policy_version 1904273 (0.0009) [2023-12-27 18:50:25,910][20347] Updated weights for policy 0, policy_version 1904283 (0.0010) [2023-12-27 18:50:26,296][20381] Updated weights for policy 1, policy_version 1902661 (0.0007) [2023-12-27 18:50:26,343][20381] Updated weights for policy 1, policy_version 1902671 (0.0007) [2023-12-27 18:50:26,399][20381] Updated weights for policy 1, policy_version 1902681 (0.0007) [2023-12-27 18:50:26,577][20347] Updated weights for policy 0, policy_version 1904293 (0.0010) [2023-12-27 18:50:26,637][20347] Updated weights for policy 0, policy_version 1904303 (0.0010) [2023-12-27 18:50:26,692][20347] Updated weights for policy 0, policy_version 1904313 (0.0010) [2023-12-27 18:50:27,090][20381] Updated weights for policy 1, policy_version 1902691 (0.0007) [2023-12-27 18:50:27,134][20381] Updated weights for policy 1, policy_version 1902701 (0.0006) [2023-12-27 18:50:27,181][20381] Updated weights for policy 1, policy_version 1902711 (0.0005) [2023-12-27 18:50:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 974749696. Throughput: 0: 10217.2, 1: 9963.9. Samples: 974739256. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.000')] [2023-12-27 18:50:27,363][20347] Updated weights for policy 0, policy_version 1904323 (0.0009) [2023-12-27 18:50:27,420][20347] Updated weights for policy 0, policy_version 1904333 (0.0006) [2023-12-27 18:50:27,463][20347] Updated weights for policy 0, policy_version 1904343 (0.0005) [2023-12-27 18:50:27,767][20381] Updated weights for policy 1, policy_version 1902721 (0.0005) [2023-12-27 18:50:27,827][20381] Updated weights for policy 1, policy_version 1902731 (0.0007) [2023-12-27 18:50:27,872][20381] Updated weights for policy 1, policy_version 1902741 (0.0009) [2023-12-27 18:50:27,915][20381] Updated weights for policy 1, policy_version 1902751 (0.0008) [2023-12-27 18:50:28,102][20347] Updated weights for policy 0, policy_version 1904353 (0.0008) [2023-12-27 18:50:28,157][20347] Updated weights for policy 0, policy_version 1904363 (0.0010) [2023-12-27 18:50:28,211][20347] Updated weights for policy 0, policy_version 1904373 (0.0010) [2023-12-27 18:50:28,264][20347] Updated weights for policy 0, policy_version 1904383 (0.0010) [2023-12-27 18:50:28,616][20381] Updated weights for policy 1, policy_version 1902761 (0.0008) [2023-12-27 18:50:28,670][20381] Updated weights for policy 1, policy_version 1902771 (0.0005) [2023-12-27 18:50:28,730][20381] Updated weights for policy 1, policy_version 1902781 (0.0005) [2023-12-27 18:50:28,948][20347] Updated weights for policy 0, policy_version 1904393 (0.0010) [2023-12-27 18:50:29,007][20347] Updated weights for policy 0, policy_version 1904403 (0.0011) [2023-12-27 18:50:29,065][20347] Updated weights for policy 0, policy_version 1904413 (0.0007) [2023-12-27 18:50:29,404][20381] Updated weights for policy 1, policy_version 1902791 (0.0009) [2023-12-27 18:50:29,462][20381] Updated weights for policy 1, policy_version 1902801 (0.0007) [2023-12-27 18:50:29,516][20381] Updated weights for policy 1, policy_version 1902811 (0.0008) [2023-12-27 18:50:29,746][20347] Updated weights for policy 0, policy_version 1904423 (0.0007) [2023-12-27 18:50:29,804][20347] Updated weights for policy 0, policy_version 1904433 (0.0008) [2023-12-27 18:50:29,870][20347] Updated weights for policy 0, policy_version 1904443 (0.0007) [2023-12-27 18:50:30,188][20381] Updated weights for policy 1, policy_version 1902821 (0.0008) [2023-12-27 18:50:30,248][20381] Updated weights for policy 1, policy_version 1902831 (0.0007) [2023-12-27 18:50:30,307][20381] Updated weights for policy 1, policy_version 1902841 (0.0008) [2023-12-27 18:50:30,579][20347] Updated weights for policy 0, policy_version 1904453 (0.0007) [2023-12-27 18:50:30,629][20347] Updated weights for policy 0, policy_version 1904463 (0.0007) [2023-12-27 18:50:30,678][20347] Updated weights for policy 0, policy_version 1904473 (0.0008) [2023-12-27 18:50:30,986][20381] Updated weights for policy 1, policy_version 1902851 (0.0009) [2023-12-27 18:50:31,040][20381] Updated weights for policy 1, policy_version 1902861 (0.0009) [2023-12-27 18:50:31,108][20381] Updated weights for policy 1, policy_version 1902871 (0.0007) [2023-12-27 18:50:31,390][20347] Updated weights for policy 0, policy_version 1904483 (0.0008) [2023-12-27 18:50:31,446][20347] Updated weights for policy 0, policy_version 1904493 (0.0009) [2023-12-27 18:50:31,508][20347] Updated weights for policy 0, policy_version 1904503 (0.0008) [2023-12-27 18:50:31,868][20381] Updated weights for policy 1, policy_version 1902881 (0.0008) [2023-12-27 18:50:31,918][20381] Updated weights for policy 1, policy_version 1902891 (0.0008) [2023-12-27 18:50:31,961][20381] Updated weights for policy 1, policy_version 1902901 (0.0007) [2023-12-27 18:50:32,007][20381] Updated weights for policy 1, policy_version 1902911 (0.0005) [2023-12-27 18:50:32,215][20347] Updated weights for policy 0, policy_version 1904513 (0.0007) [2023-12-27 18:50:32,268][20347] Updated weights for policy 0, policy_version 1904523 (0.0006) [2023-12-27 18:50:32,323][20347] Updated weights for policy 0, policy_version 1904533 (0.0007) [2023-12-27 18:50:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 974848000. Throughput: 0: 10189.7, 1: 10057.0. Samples: 974862300. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.950')] [2023-12-27 18:50:32,386][20347] Updated weights for policy 0, policy_version 1904543 (0.0008) [2023-12-27 18:50:32,661][20381] Updated weights for policy 1, policy_version 1902921 (0.0007) [2023-12-27 18:50:32,716][20381] Updated weights for policy 1, policy_version 1902931 (0.0006) [2023-12-27 18:50:32,774][20381] Updated weights for policy 1, policy_version 1902941 (0.0005) [2023-12-27 18:50:33,066][20347] Updated weights for policy 0, policy_version 1904553 (0.0006) [2023-12-27 18:50:33,121][20347] Updated weights for policy 0, policy_version 1904563 (0.0008) [2023-12-27 18:50:33,176][20347] Updated weights for policy 0, policy_version 1904573 (0.0009) [2023-12-27 18:50:33,327][20381] Updated weights for policy 1, policy_version 1902951 (0.0006) [2023-12-27 18:50:33,378][20381] Updated weights for policy 1, policy_version 1902961 (0.0005) [2023-12-27 18:50:33,424][20381] Updated weights for policy 1, policy_version 1902971 (0.0005) [2023-12-27 18:50:33,799][20347] Updated weights for policy 0, policy_version 1904583 (0.0007) [2023-12-27 18:50:33,852][20347] Updated weights for policy 0, policy_version 1904593 (0.0007) [2023-12-27 18:50:33,906][20347] Updated weights for policy 0, policy_version 1904603 (0.0008) [2023-12-27 18:50:34,074][20381] Updated weights for policy 1, policy_version 1902981 (0.0005) [2023-12-27 18:50:34,147][20381] Updated weights for policy 1, policy_version 1902991 (0.0010) [2023-12-27 18:50:34,212][20381] Updated weights for policy 1, policy_version 1903001 (0.0011) [2023-12-27 18:50:34,673][20347] Updated weights for policy 0, policy_version 1904613 (0.0008) [2023-12-27 18:50:34,721][20347] Updated weights for policy 0, policy_version 1904623 (0.0008) [2023-12-27 18:50:34,769][20347] Updated weights for policy 0, policy_version 1904633 (0.0008) [2023-12-27 18:50:34,912][20381] Updated weights for policy 1, policy_version 1903011 (0.0009) [2023-12-27 18:50:34,966][20381] Updated weights for policy 1, policy_version 1903021 (0.0005) [2023-12-27 18:50:35,024][20381] Updated weights for policy 1, policy_version 1903031 (0.0006) [2023-12-27 18:50:35,374][20347] Updated weights for policy 0, policy_version 1904643 (0.0007) [2023-12-27 18:50:35,430][20347] Updated weights for policy 0, policy_version 1904653 (0.0005) [2023-12-27 18:50:35,477][20347] Updated weights for policy 0, policy_version 1904663 (0.0005) [2023-12-27 18:50:35,739][20381] Updated weights for policy 1, policy_version 1903041 (0.0007) [2023-12-27 18:50:35,803][20381] Updated weights for policy 1, policy_version 1903051 (0.0007) [2023-12-27 18:50:35,855][20381] Updated weights for policy 1, policy_version 1903061 (0.0007) [2023-12-27 18:50:35,912][20381] Updated weights for policy 1, policy_version 1903071 (0.0005) [2023-12-27 18:50:36,058][20347] Updated weights for policy 0, policy_version 1904673 (0.0006) [2023-12-27 18:50:36,115][20347] Updated weights for policy 0, policy_version 1904683 (0.0008) [2023-12-27 18:50:36,173][20347] Updated weights for policy 0, policy_version 1904693 (0.0007) [2023-12-27 18:50:36,227][20347] Updated weights for policy 0, policy_version 1904703 (0.0007) [2023-12-27 18:50:36,630][20381] Updated weights for policy 1, policy_version 1903081 (0.0008) [2023-12-27 18:50:36,686][20381] Updated weights for policy 1, policy_version 1903091 (0.0007) [2023-12-27 18:50:36,741][20381] Updated weights for policy 1, policy_version 1903101 (0.0007) [2023-12-27 18:50:36,927][20347] Updated weights for policy 0, policy_version 1904713 (0.0009) [2023-12-27 18:50:36,977][20347] Updated weights for policy 0, policy_version 1904723 (0.0007) [2023-12-27 18:50:37,023][20347] Updated weights for policy 0, policy_version 1904733 (0.0009) [2023-12-27 18:50:37,339][20381] Updated weights for policy 1, policy_version 1903111 (0.0007) [2023-12-27 18:50:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20466.3). Total num frames: 974954496. Throughput: 0: 10171.9, 1: 10087.3. Samples: 974924264. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.480')] [2023-12-27 18:50:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001904736_487702528.pth... [2023-12-27 18:50:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001903552_487399424.pth [2023-12-27 18:50:37,395][20381] Updated weights for policy 1, policy_version 1903121 (0.0008) [2023-12-27 18:50:37,458][20381] Updated weights for policy 1, policy_version 1903131 (0.0007) [2023-12-27 18:50:37,489][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001903136_487260160.pth... [2023-12-27 18:50:37,493][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001901920_486948864.pth [2023-12-27 18:50:37,746][20347] Updated weights for policy 0, policy_version 1904743 (0.0008) [2023-12-27 18:50:37,799][20347] Updated weights for policy 0, policy_version 1904753 (0.0007) [2023-12-27 18:50:37,853][20347] Updated weights for policy 0, policy_version 1904763 (0.0007) [2023-12-27 18:50:38,172][20381] Updated weights for policy 1, policy_version 1903141 (0.0008) [2023-12-27 18:50:38,218][20381] Updated weights for policy 1, policy_version 1903151 (0.0010) [2023-12-27 18:50:38,264][20381] Updated weights for policy 1, policy_version 1903161 (0.0010) [2023-12-27 18:50:38,526][20347] Updated weights for policy 0, policy_version 1904773 (0.0007) [2023-12-27 18:50:38,590][20347] Updated weights for policy 0, policy_version 1904783 (0.0007) [2023-12-27 18:50:38,651][20347] Updated weights for policy 0, policy_version 1904793 (0.0007) [2023-12-27 18:50:39,000][20381] Updated weights for policy 1, policy_version 1903171 (0.0008) [2023-12-27 18:50:39,063][20381] Updated weights for policy 1, policy_version 1903181 (0.0007) [2023-12-27 18:50:39,121][20381] Updated weights for policy 1, policy_version 1903191 (0.0007) [2023-12-27 18:50:39,375][20347] Updated weights for policy 0, policy_version 1904803 (0.0008) [2023-12-27 18:50:39,442][20347] Updated weights for policy 0, policy_version 1904813 (0.0010) [2023-12-27 18:50:39,496][20347] Updated weights for policy 0, policy_version 1904823 (0.0010) [2023-12-27 18:50:39,833][20381] Updated weights for policy 1, policy_version 1903201 (0.0009) [2023-12-27 18:50:39,895][20381] Updated weights for policy 1, policy_version 1903211 (0.0009) [2023-12-27 18:50:39,953][20381] Updated weights for policy 1, policy_version 1903221 (0.0008) [2023-12-27 18:50:40,001][20381] Updated weights for policy 1, policy_version 1903231 (0.0008) [2023-12-27 18:50:40,272][20347] Updated weights for policy 0, policy_version 1904833 (0.0010) [2023-12-27 18:50:40,331][20347] Updated weights for policy 0, policy_version 1904843 (0.0008) [2023-12-27 18:50:40,389][20347] Updated weights for policy 0, policy_version 1904853 (0.0008) [2023-12-27 18:50:40,449][20347] Updated weights for policy 0, policy_version 1904863 (0.0008) [2023-12-27 18:50:40,721][20381] Updated weights for policy 1, policy_version 1903241 (0.0005) [2023-12-27 18:50:40,771][20381] Updated weights for policy 1, policy_version 1903251 (0.0005) [2023-12-27 18:50:40,822][20381] Updated weights for policy 1, policy_version 1903261 (0.0005) [2023-12-27 18:50:41,217][20347] Updated weights for policy 0, policy_version 1904873 (0.0008) [2023-12-27 18:50:41,284][20347] Updated weights for policy 0, policy_version 1904883 (0.0007) [2023-12-27 18:50:41,348][20347] Updated weights for policy 0, policy_version 1904893 (0.0008) [2023-12-27 18:50:41,484][20381] Updated weights for policy 1, policy_version 1903271 (0.0007) [2023-12-27 18:50:41,535][20381] Updated weights for policy 1, policy_version 1903281 (0.0008) [2023-12-27 18:50:41,589][20381] Updated weights for policy 1, policy_version 1903291 (0.0008) [2023-12-27 18:50:42,078][20347] Updated weights for policy 0, policy_version 1904903 (0.0008) [2023-12-27 18:50:42,137][20347] Updated weights for policy 0, policy_version 1904913 (0.0008) [2023-12-27 18:50:42,197][20347] Updated weights for policy 0, policy_version 1904923 (0.0007) [2023-12-27 18:50:42,336][20381] Updated weights for policy 1, policy_version 1903301 (0.0008) [2023-12-27 18:50:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20438.3). Total num frames: 975052800. Throughput: 0: 10144.6, 1: 10093.5. Samples: 975045148. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.610')] [2023-12-27 18:50:42,400][20381] Updated weights for policy 1, policy_version 1903311 (0.0007) [2023-12-27 18:50:42,457][20381] Updated weights for policy 1, policy_version 1903321 (0.0008) [2023-12-27 18:50:42,936][20347] Updated weights for policy 0, policy_version 1904933 (0.0008) [2023-12-27 18:50:42,981][20347] Updated weights for policy 0, policy_version 1904943 (0.0007) [2023-12-27 18:50:43,036][20347] Updated weights for policy 0, policy_version 1904953 (0.0006) [2023-12-27 18:50:43,157][20381] Updated weights for policy 1, policy_version 1903331 (0.0008) [2023-12-27 18:50:43,206][20381] Updated weights for policy 1, policy_version 1903341 (0.0006) [2023-12-27 18:50:43,258][20381] Updated weights for policy 1, policy_version 1903351 (0.0007) [2023-12-27 18:50:43,669][20347] Updated weights for policy 0, policy_version 1904963 (0.0009) [2023-12-27 18:50:43,718][20347] Updated weights for policy 0, policy_version 1904973 (0.0008) [2023-12-27 18:50:43,767][20347] Updated weights for policy 0, policy_version 1904983 (0.0008) [2023-12-27 18:50:43,862][20381] Updated weights for policy 1, policy_version 1903361 (0.0008) [2023-12-27 18:50:43,919][20381] Updated weights for policy 1, policy_version 1903371 (0.0009) [2023-12-27 18:50:43,974][20381] Updated weights for policy 1, policy_version 1903381 (0.0008) [2023-12-27 18:50:44,022][20381] Updated weights for policy 1, policy_version 1903391 (0.0008) [2023-12-27 18:50:44,436][20347] Updated weights for policy 0, policy_version 1904993 (0.0007) [2023-12-27 18:50:44,498][20347] Updated weights for policy 0, policy_version 1905003 (0.0006) [2023-12-27 18:50:44,559][20347] Updated weights for policy 0, policy_version 1905013 (0.0008) [2023-12-27 18:50:44,612][20347] Updated weights for policy 0, policy_version 1905023 (0.0008) [2023-12-27 18:50:44,719][20381] Updated weights for policy 1, policy_version 1903401 (0.0009) [2023-12-27 18:50:44,782][20381] Updated weights for policy 1, policy_version 1903411 (0.0007) [2023-12-27 18:50:44,845][20381] Updated weights for policy 1, policy_version 1903421 (0.0010) [2023-12-27 18:50:45,304][20347] Updated weights for policy 0, policy_version 1905033 (0.0007) [2023-12-27 18:50:45,358][20347] Updated weights for policy 0, policy_version 1905043 (0.0007) [2023-12-27 18:50:45,418][20347] Updated weights for policy 0, policy_version 1905053 (0.0007) [2023-12-27 18:50:45,465][20381] Updated weights for policy 1, policy_version 1903431 (0.0009) [2023-12-27 18:50:45,528][20381] Updated weights for policy 1, policy_version 1903441 (0.0008) [2023-12-27 18:50:45,581][20381] Updated weights for policy 1, policy_version 1903451 (0.0007) [2023-12-27 18:50:46,114][20347] Updated weights for policy 0, policy_version 1905063 (0.0007) [2023-12-27 18:50:46,158][20347] Updated weights for policy 0, policy_version 1905073 (0.0007) [2023-12-27 18:50:46,208][20381] Updated weights for policy 1, policy_version 1903461 (0.0008) [2023-12-27 18:50:46,213][20347] Updated weights for policy 0, policy_version 1905083 (0.0006) [2023-12-27 18:50:46,262][20381] Updated weights for policy 1, policy_version 1903471 (0.0007) [2023-12-27 18:50:46,317][20381] Updated weights for policy 1, policy_version 1903481 (0.0008) [2023-12-27 18:50:46,886][20347] Updated weights for policy 0, policy_version 1905093 (0.0008) [2023-12-27 18:50:46,932][20347] Updated weights for policy 0, policy_version 1905103 (0.0007) [2023-12-27 18:50:46,979][20347] Updated weights for policy 0, policy_version 1905113 (0.0007) [2023-12-27 18:50:47,018][20381] Updated weights for policy 1, policy_version 1903491 (0.0007) [2023-12-27 18:50:47,070][20381] Updated weights for policy 1, policy_version 1903501 (0.0008) [2023-12-27 18:50:47,127][20381] Updated weights for policy 1, policy_version 1903511 (0.0008) [2023-12-27 18:50:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20207.0, 300 sec: 20493.9). Total num frames: 975159296. Throughput: 0: 10112.3, 1: 10168.5. Samples: 975166140. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.800')] [2023-12-27 18:50:47,690][20347] Updated weights for policy 0, policy_version 1905123 (0.0007) [2023-12-27 18:50:47,749][20347] Updated weights for policy 0, policy_version 1905133 (0.0008) [2023-12-27 18:50:47,807][20347] Updated weights for policy 0, policy_version 1905143 (0.0008) [2023-12-27 18:50:47,809][20381] Updated weights for policy 1, policy_version 1903521 (0.0008) [2023-12-27 18:50:47,857][20381] Updated weights for policy 1, policy_version 1903531 (0.0005) [2023-12-27 18:50:47,914][20381] Updated weights for policy 1, policy_version 1903541 (0.0005) [2023-12-27 18:50:47,967][20381] Updated weights for policy 1, policy_version 1903551 (0.0005) [2023-12-27 18:50:48,529][20347] Updated weights for policy 0, policy_version 1905153 (0.0007) [2023-12-27 18:50:48,587][20347] Updated weights for policy 0, policy_version 1905163 (0.0006) [2023-12-27 18:50:48,589][20381] Updated weights for policy 1, policy_version 1903561 (0.0006) [2023-12-27 18:50:48,637][20347] Updated weights for policy 0, policy_version 1905173 (0.0005) [2023-12-27 18:50:48,639][20381] Updated weights for policy 1, policy_version 1903571 (0.0006) [2023-12-27 18:50:48,690][20347] Updated weights for policy 0, policy_version 1905183 (0.0005) [2023-12-27 18:50:48,695][20381] Updated weights for policy 1, policy_version 1903581 (0.0007) [2023-12-27 18:50:49,384][20381] Updated weights for policy 1, policy_version 1903591 (0.0008) [2023-12-27 18:50:49,430][20347] Updated weights for policy 0, policy_version 1905193 (0.0008) [2023-12-27 18:50:49,436][20381] Updated weights for policy 1, policy_version 1903601 (0.0006) [2023-12-27 18:50:49,490][20347] Updated weights for policy 0, policy_version 1905203 (0.0008) [2023-12-27 18:50:49,491][20381] Updated weights for policy 1, policy_version 1903611 (0.0007) [2023-12-27 18:50:49,550][20347] Updated weights for policy 0, policy_version 1905213 (0.0008) [2023-12-27 18:50:50,197][20381] Updated weights for policy 1, policy_version 1903621 (0.0009) [2023-12-27 18:50:50,261][20381] Updated weights for policy 1, policy_version 1903631 (0.0011) [2023-12-27 18:50:50,279][20347] Updated weights for policy 0, policy_version 1905223 (0.0007) [2023-12-27 18:50:50,326][20381] Updated weights for policy 1, policy_version 1903641 (0.0011) [2023-12-27 18:50:50,341][20347] Updated weights for policy 0, policy_version 1905233 (0.0007) [2023-12-27 18:50:50,399][20347] Updated weights for policy 0, policy_version 1905243 (0.0008) [2023-12-27 18:50:51,030][20381] Updated weights for policy 1, policy_version 1903651 (0.0010) [2023-12-27 18:50:51,090][20381] Updated weights for policy 1, policy_version 1903661 (0.0009) [2023-12-27 18:50:51,118][20347] Updated weights for policy 0, policy_version 1905253 (0.0007) [2023-12-27 18:50:51,151][20381] Updated weights for policy 1, policy_version 1903671 (0.0009) [2023-12-27 18:50:51,185][20347] Updated weights for policy 0, policy_version 1905263 (0.0009) [2023-12-27 18:50:51,246][20347] Updated weights for policy 0, policy_version 1905273 (0.0009) [2023-12-27 18:50:51,986][20381] Updated weights for policy 1, policy_version 1903681 (0.0008) [2023-12-27 18:50:51,992][20347] Updated weights for policy 0, policy_version 1905283 (0.0009) [2023-12-27 18:50:52,044][20381] Updated weights for policy 1, policy_version 1903691 (0.0006) [2023-12-27 18:50:52,046][20347] Updated weights for policy 0, policy_version 1905293 (0.0005) [2023-12-27 18:50:52,098][20381] Updated weights for policy 1, policy_version 1903701 (0.0007) [2023-12-27 18:50:52,100][20347] Updated weights for policy 0, policy_version 1905303 (0.0005) [2023-12-27 18:50:52,156][20381] Updated weights for policy 1, policy_version 1903711 (0.0007) [2023-12-27 18:50:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 975257600. Throughput: 0: 10025.8, 1: 10212.8. Samples: 975226708. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.530')] [2023-12-27 18:50:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001903712_487407616.pth... [2023-12-27 18:50:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001905312_487849984.pth... [2023-12-27 18:50:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001902528_487104512.pth [2023-12-27 18:50:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001904128_487546880.pth [2023-12-27 18:50:52,814][20347] Updated weights for policy 0, policy_version 1905313 (0.0005) [2023-12-27 18:50:52,831][20381] Updated weights for policy 1, policy_version 1903721 (0.0007) [2023-12-27 18:50:52,860][20347] Updated weights for policy 0, policy_version 1905323 (0.0005) [2023-12-27 18:50:52,891][20381] Updated weights for policy 1, policy_version 1903731 (0.0008) [2023-12-27 18:50:52,908][20347] Updated weights for policy 0, policy_version 1905333 (0.0006) [2023-12-27 18:50:52,947][20381] Updated weights for policy 1, policy_version 1903741 (0.0006) [2023-12-27 18:50:52,957][20347] Updated weights for policy 0, policy_version 1905343 (0.0006) [2023-12-27 18:50:53,599][20381] Updated weights for policy 1, policy_version 1903751 (0.0008) [2023-12-27 18:50:53,658][20381] Updated weights for policy 1, policy_version 1903761 (0.0008) [2023-12-27 18:50:53,668][20347] Updated weights for policy 0, policy_version 1905353 (0.0007) [2023-12-27 18:50:53,713][20381] Updated weights for policy 1, policy_version 1903771 (0.0006) [2023-12-27 18:50:53,723][20347] Updated weights for policy 0, policy_version 1905363 (0.0005) [2023-12-27 18:50:53,782][20347] Updated weights for policy 0, policy_version 1905373 (0.0007) [2023-12-27 18:50:54,407][20381] Updated weights for policy 1, policy_version 1903781 (0.0006) [2023-12-27 18:50:54,464][20381] Updated weights for policy 1, policy_version 1903791 (0.0008) [2023-12-27 18:50:54,508][20347] Updated weights for policy 0, policy_version 1905383 (0.0006) [2023-12-27 18:50:54,525][20381] Updated weights for policy 1, policy_version 1903801 (0.0007) [2023-12-27 18:50:54,572][20347] Updated weights for policy 0, policy_version 1905393 (0.0006) [2023-12-27 18:50:54,634][20347] Updated weights for policy 0, policy_version 1905403 (0.0007) [2023-12-27 18:50:55,123][20381] Updated weights for policy 1, policy_version 1903811 (0.0008) [2023-12-27 18:50:55,184][20381] Updated weights for policy 1, policy_version 1903821 (0.0009) [2023-12-27 18:50:55,210][20347] Updated weights for policy 0, policy_version 1905413 (0.0007) [2023-12-27 18:50:55,243][20381] Updated weights for policy 1, policy_version 1903831 (0.0007) [2023-12-27 18:50:55,256][20347] Updated weights for policy 0, policy_version 1905423 (0.0010) [2023-12-27 18:50:55,305][20347] Updated weights for policy 0, policy_version 1905433 (0.0010) [2023-12-27 18:50:55,891][20347] Updated weights for policy 0, policy_version 1905443 (0.0010) [2023-12-27 18:50:55,939][20347] Updated weights for policy 0, policy_version 1905453 (0.0010) [2023-12-27 18:50:55,950][20381] Updated weights for policy 1, policy_version 1903841 (0.0006) [2023-12-27 18:50:55,997][20347] Updated weights for policy 0, policy_version 1905463 (0.0010) [2023-12-27 18:50:56,003][20381] Updated weights for policy 1, policy_version 1903851 (0.0007) [2023-12-27 18:50:56,061][20381] Updated weights for policy 1, policy_version 1903861 (0.0008) [2023-12-27 18:50:56,112][20381] Updated weights for policy 1, policy_version 1903871 (0.0007) [2023-12-27 18:50:56,660][20347] Updated weights for policy 0, policy_version 1905473 (0.0010) [2023-12-27 18:50:56,713][20347] Updated weights for policy 0, policy_version 1905483 (0.0009) [2023-12-27 18:50:56,778][20347] Updated weights for policy 0, policy_version 1905493 (0.0008) [2023-12-27 18:50:56,829][20347] Updated weights for policy 0, policy_version 1905503 (0.0008) [2023-12-27 18:50:56,836][20381] Updated weights for policy 1, policy_version 1903881 (0.0007) [2023-12-27 18:50:56,884][20381] Updated weights for policy 1, policy_version 1903891 (0.0008) [2023-12-27 18:50:56,928][20381] Updated weights for policy 1, policy_version 1903901 (0.0008) [2023-12-27 18:50:57,363][19170] Fps is (10 sec: 19660.2, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 975355904. Throughput: 0: 9965.1, 1: 10184.2. Samples: 975347380. Policy #0 lag: (min: 11.0, avg: 22.6, max: 43.0) [2023-12-27 18:50:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '921.820')] [2023-12-27 18:50:57,408][20347] Updated weights for policy 0, policy_version 1905513 (0.0010) [2023-12-27 18:50:57,458][20347] Updated weights for policy 0, policy_version 1905523 (0.0010) [2023-12-27 18:50:57,503][20347] Updated weights for policy 0, policy_version 1905533 (0.0010) [2023-12-27 18:50:57,586][20381] Updated weights for policy 1, policy_version 1903911 (0.0006) [2023-12-27 18:50:57,649][20381] Updated weights for policy 1, policy_version 1903921 (0.0007) [2023-12-27 18:50:57,711][20381] Updated weights for policy 1, policy_version 1903931 (0.0007) [2023-12-27 18:50:58,184][20347] Updated weights for policy 0, policy_version 1905543 (0.0010) [2023-12-27 18:50:58,248][20347] Updated weights for policy 0, policy_version 1905553 (0.0008) [2023-12-27 18:50:58,309][20347] Updated weights for policy 0, policy_version 1905563 (0.0006) [2023-12-27 18:50:58,417][20381] Updated weights for policy 1, policy_version 1903941 (0.0007) [2023-12-27 18:50:58,479][20381] Updated weights for policy 1, policy_version 1903951 (0.0008) [2023-12-27 18:50:58,542][20381] Updated weights for policy 1, policy_version 1903961 (0.0008) [2023-12-27 18:50:59,047][20347] Updated weights for policy 0, policy_version 1905573 (0.0009) [2023-12-27 18:50:59,096][20347] Updated weights for policy 0, policy_version 1905583 (0.0008) [2023-12-27 18:50:59,145][20347] Updated weights for policy 0, policy_version 1905593 (0.0008) [2023-12-27 18:50:59,336][20381] Updated weights for policy 1, policy_version 1903971 (0.0008) [2023-12-27 18:50:59,396][20381] Updated weights for policy 1, policy_version 1903981 (0.0008) [2023-12-27 18:50:59,459][20381] Updated weights for policy 1, policy_version 1903991 (0.0008) [2023-12-27 18:50:59,894][20347] Updated weights for policy 0, policy_version 1905603 (0.0006) [2023-12-27 18:50:59,955][20347] Updated weights for policy 0, policy_version 1905613 (0.0008) [2023-12-27 18:51:00,010][20347] Updated weights for policy 0, policy_version 1905623 (0.0007) [2023-12-27 18:51:00,194][20381] Updated weights for policy 1, policy_version 1904001 (0.0008) [2023-12-27 18:51:00,245][20381] Updated weights for policy 1, policy_version 1904011 (0.0008) [2023-12-27 18:51:00,301][20381] Updated weights for policy 1, policy_version 1904021 (0.0008) [2023-12-27 18:51:00,359][20381] Updated weights for policy 1, policy_version 1904031 (0.0006) [2023-12-27 18:51:00,669][20347] Updated weights for policy 0, policy_version 1905633 (0.0006) [2023-12-27 18:51:00,719][20347] Updated weights for policy 0, policy_version 1905643 (0.0007) [2023-12-27 18:51:00,775][20347] Updated weights for policy 0, policy_version 1905653 (0.0007) [2023-12-27 18:51:00,826][20347] Updated weights for policy 0, policy_version 1905663 (0.0005) [2023-12-27 18:51:00,998][20381] Updated weights for policy 1, policy_version 1904041 (0.0007) [2023-12-27 18:51:01,055][20381] Updated weights for policy 1, policy_version 1904051 (0.0007) [2023-12-27 18:51:01,114][20381] Updated weights for policy 1, policy_version 1904061 (0.0008) [2023-12-27 18:51:01,495][20347] Updated weights for policy 0, policy_version 1905673 (0.0007) [2023-12-27 18:51:01,552][20347] Updated weights for policy 0, policy_version 1905683 (0.0007) [2023-12-27 18:51:01,610][20347] Updated weights for policy 0, policy_version 1905693 (0.0006) [2023-12-27 18:51:01,859][20381] Updated weights for policy 1, policy_version 1904071 (0.0008) [2023-12-27 18:51:01,916][20381] Updated weights for policy 1, policy_version 1904081 (0.0007) [2023-12-27 18:51:01,971][20381] Updated weights for policy 1, policy_version 1904091 (0.0007) [2023-12-27 18:51:02,313][20347] Updated weights for policy 0, policy_version 1905703 (0.0011) [2023-12-27 18:51:02,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 975454208. Throughput: 0: 10073.7, 1: 10106.5. Samples: 975467516. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.260')] [2023-12-27 18:51:02,374][20347] Updated weights for policy 0, policy_version 1905713 (0.0012) [2023-12-27 18:51:02,434][20347] Updated weights for policy 0, policy_version 1905723 (0.0010) [2023-12-27 18:51:02,690][20381] Updated weights for policy 1, policy_version 1904101 (0.0008) [2023-12-27 18:51:02,748][20381] Updated weights for policy 1, policy_version 1904111 (0.0007) [2023-12-27 18:51:02,801][20381] Updated weights for policy 1, policy_version 1904121 (0.0007) [2023-12-27 18:51:03,108][20347] Updated weights for policy 0, policy_version 1905733 (0.0011) [2023-12-27 18:51:03,162][20347] Updated weights for policy 0, policy_version 1905743 (0.0010) [2023-12-27 18:51:03,220][20347] Updated weights for policy 0, policy_version 1905753 (0.0010) [2023-12-27 18:51:03,499][20381] Updated weights for policy 1, policy_version 1904131 (0.0008) [2023-12-27 18:51:03,546][20381] Updated weights for policy 1, policy_version 1904141 (0.0007) [2023-12-27 18:51:03,594][20381] Updated weights for policy 1, policy_version 1904151 (0.0007) [2023-12-27 18:51:03,907][20347] Updated weights for policy 0, policy_version 1905763 (0.0010) [2023-12-27 18:51:03,966][20347] Updated weights for policy 0, policy_version 1905773 (0.0010) [2023-12-27 18:51:04,029][20347] Updated weights for policy 0, policy_version 1905783 (0.0010) [2023-12-27 18:51:04,312][20381] Updated weights for policy 1, policy_version 1904161 (0.0008) [2023-12-27 18:51:04,368][20381] Updated weights for policy 1, policy_version 1904171 (0.0008) [2023-12-27 18:51:04,430][20381] Updated weights for policy 1, policy_version 1904181 (0.0009) [2023-12-27 18:51:04,489][20381] Updated weights for policy 1, policy_version 1904191 (0.0007) [2023-12-27 18:51:04,727][20347] Updated weights for policy 0, policy_version 1905793 (0.0010) [2023-12-27 18:51:04,793][20347] Updated weights for policy 0, policy_version 1905803 (0.0008) [2023-12-27 18:51:04,850][20347] Updated weights for policy 0, policy_version 1905813 (0.0007) [2023-12-27 18:51:04,905][20347] Updated weights for policy 0, policy_version 1905823 (0.0008) [2023-12-27 18:51:05,081][20381] Updated weights for policy 1, policy_version 1904201 (0.0008) [2023-12-27 18:51:05,138][20381] Updated weights for policy 1, policy_version 1904211 (0.0008) [2023-12-27 18:51:05,189][20381] Updated weights for policy 1, policy_version 1904221 (0.0008) [2023-12-27 18:51:05,531][20347] Updated weights for policy 0, policy_version 1905833 (0.0010) [2023-12-27 18:51:05,586][20347] Updated weights for policy 0, policy_version 1905843 (0.0010) [2023-12-27 18:51:05,644][20347] Updated weights for policy 0, policy_version 1905853 (0.0010) [2023-12-27 18:51:05,908][20381] Updated weights for policy 1, policy_version 1904231 (0.0010) [2023-12-27 18:51:05,966][20381] Updated weights for policy 1, policy_version 1904241 (0.0011) [2023-12-27 18:51:06,025][20381] Updated weights for policy 1, policy_version 1904251 (0.0011) [2023-12-27 18:51:06,304][20347] Updated weights for policy 0, policy_version 1905863 (0.0008) [2023-12-27 18:51:06,367][20347] Updated weights for policy 0, policy_version 1905873 (0.0007) [2023-12-27 18:51:06,424][20347] Updated weights for policy 0, policy_version 1905883 (0.0008) [2023-12-27 18:51:06,739][20381] Updated weights for policy 1, policy_version 1904261 (0.0010) [2023-12-27 18:51:06,784][20381] Updated weights for policy 1, policy_version 1904271 (0.0010) [2023-12-27 18:51:06,839][20381] Updated weights for policy 1, policy_version 1904281 (0.0009) [2023-12-27 18:51:07,122][20347] Updated weights for policy 0, policy_version 1905893 (0.0007) [2023-12-27 18:51:07,181][20347] Updated weights for policy 0, policy_version 1905903 (0.0007) [2023-12-27 18:51:07,245][20347] Updated weights for policy 0, policy_version 1905913 (0.0005) [2023-12-27 18:51:07,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 975560704. Throughput: 0: 10065.3, 1: 10142.6. Samples: 975528384. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.090')] [2023-12-27 18:51:07,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001904288_487555072.pth... [2023-12-27 18:51:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001905920_488005632.pth... [2023-12-27 18:51:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001903136_487260160.pth [2023-12-27 18:51:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001904736_487702528.pth [2023-12-27 18:51:07,482][20381] Updated weights for policy 1, policy_version 1904291 (0.0007) [2023-12-27 18:51:07,541][20381] Updated weights for policy 1, policy_version 1904301 (0.0007) [2023-12-27 18:51:07,597][20381] Updated weights for policy 1, policy_version 1904311 (0.0008) [2023-12-27 18:51:07,893][20347] Updated weights for policy 0, policy_version 1905923 (0.0006) [2023-12-27 18:51:07,950][20347] Updated weights for policy 0, policy_version 1905933 (0.0006) [2023-12-27 18:51:08,004][20347] Updated weights for policy 0, policy_version 1905943 (0.0005) [2023-12-27 18:51:08,225][20381] Updated weights for policy 1, policy_version 1904321 (0.0007) [2023-12-27 18:51:08,288][20381] Updated weights for policy 1, policy_version 1904331 (0.0005) [2023-12-27 18:51:08,356][20381] Updated weights for policy 1, policy_version 1904341 (0.0008) [2023-12-27 18:51:08,420][20381] Updated weights for policy 1, policy_version 1904351 (0.0009) [2023-12-27 18:51:08,565][20347] Updated weights for policy 0, policy_version 1905953 (0.0005) [2023-12-27 18:51:08,624][20347] Updated weights for policy 0, policy_version 1905963 (0.0007) [2023-12-27 18:51:08,693][20347] Updated weights for policy 0, policy_version 1905973 (0.0007) [2023-12-27 18:51:08,749][20347] Updated weights for policy 0, policy_version 1905983 (0.0008) [2023-12-27 18:51:09,048][20381] Updated weights for policy 1, policy_version 1904361 (0.0007) [2023-12-27 18:51:09,092][20381] Updated weights for policy 1, policy_version 1904371 (0.0007) [2023-12-27 18:51:09,145][20381] Updated weights for policy 1, policy_version 1904381 (0.0006) [2023-12-27 18:51:09,492][20347] Updated weights for policy 0, policy_version 1905993 (0.0008) [2023-12-27 18:51:09,550][20347] Updated weights for policy 0, policy_version 1906003 (0.0008) [2023-12-27 18:51:09,610][20347] Updated weights for policy 0, policy_version 1906013 (0.0008) [2023-12-27 18:51:09,929][20381] Updated weights for policy 1, policy_version 1904391 (0.0007) [2023-12-27 18:51:09,991][20381] Updated weights for policy 1, policy_version 1904401 (0.0008) [2023-12-27 18:51:10,052][20381] Updated weights for policy 1, policy_version 1904411 (0.0008) [2023-12-27 18:51:10,340][20347] Updated weights for policy 0, policy_version 1906023 (0.0008) [2023-12-27 18:51:10,397][20347] Updated weights for policy 0, policy_version 1906033 (0.0008) [2023-12-27 18:51:10,456][20347] Updated weights for policy 0, policy_version 1906043 (0.0007) [2023-12-27 18:51:10,764][20381] Updated weights for policy 1, policy_version 1904421 (0.0007) [2023-12-27 18:51:10,814][20381] Updated weights for policy 1, policy_version 1904431 (0.0008) [2023-12-27 18:51:10,870][20381] Updated weights for policy 1, policy_version 1904441 (0.0008) [2023-12-27 18:51:11,198][20347] Updated weights for policy 0, policy_version 1906053 (0.0007) [2023-12-27 18:51:11,263][20347] Updated weights for policy 0, policy_version 1906063 (0.0008) [2023-12-27 18:51:11,327][20347] Updated weights for policy 0, policy_version 1906073 (0.0007) [2023-12-27 18:51:11,643][20381] Updated weights for policy 1, policy_version 1904451 (0.0006) [2023-12-27 18:51:11,716][20381] Updated weights for policy 1, policy_version 1904461 (0.0007) [2023-12-27 18:51:11,777][20381] Updated weights for policy 1, policy_version 1904471 (0.0008) [2023-12-27 18:51:12,053][20347] Updated weights for policy 0, policy_version 1906083 (0.0008) [2023-12-27 18:51:12,106][20347] Updated weights for policy 0, policy_version 1906093 (0.0008) [2023-12-27 18:51:12,159][20347] Updated weights for policy 0, policy_version 1906103 (0.0007) [2023-12-27 18:51:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20207.0, 300 sec: 20466.1). Total num frames: 975659008. Throughput: 0: 10094.9, 1: 10146.0. Samples: 975650100. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.740')] [2023-12-27 18:51:12,475][20381] Updated weights for policy 1, policy_version 1904481 (0.0008) [2023-12-27 18:51:12,536][20381] Updated weights for policy 1, policy_version 1904491 (0.0006) [2023-12-27 18:51:12,592][20381] Updated weights for policy 1, policy_version 1904501 (0.0008) [2023-12-27 18:51:12,656][20381] Updated weights for policy 1, policy_version 1904511 (0.0008) [2023-12-27 18:51:12,896][20347] Updated weights for policy 0, policy_version 1906113 (0.0008) [2023-12-27 18:51:12,952][20347] Updated weights for policy 0, policy_version 1906123 (0.0008) [2023-12-27 18:51:13,004][20347] Updated weights for policy 0, policy_version 1906133 (0.0008) [2023-12-27 18:51:13,053][20347] Updated weights for policy 0, policy_version 1906143 (0.0008) [2023-12-27 18:51:13,260][20381] Updated weights for policy 1, policy_version 1904521 (0.0007) [2023-12-27 18:51:13,319][20381] Updated weights for policy 1, policy_version 1904531 (0.0008) [2023-12-27 18:51:13,377][20381] Updated weights for policy 1, policy_version 1904541 (0.0007) [2023-12-27 18:51:13,785][20347] Updated weights for policy 0, policy_version 1906153 (0.0007) [2023-12-27 18:51:13,836][20347] Updated weights for policy 0, policy_version 1906163 (0.0007) [2023-12-27 18:51:13,894][20347] Updated weights for policy 0, policy_version 1906173 (0.0007) [2023-12-27 18:51:13,994][20381] Updated weights for policy 1, policy_version 1904551 (0.0007) [2023-12-27 18:51:14,049][20381] Updated weights for policy 1, policy_version 1904561 (0.0007) [2023-12-27 18:51:14,106][20381] Updated weights for policy 1, policy_version 1904571 (0.0007) [2023-12-27 18:51:14,634][20347] Updated weights for policy 0, policy_version 1906183 (0.0008) [2023-12-27 18:51:14,679][20347] Updated weights for policy 0, policy_version 1906193 (0.0009) [2023-12-27 18:51:14,701][20381] Updated weights for policy 1, policy_version 1904581 (0.0006) [2023-12-27 18:51:14,731][20347] Updated weights for policy 0, policy_version 1906203 (0.0007) [2023-12-27 18:51:14,757][20381] Updated weights for policy 1, policy_version 1904591 (0.0006) [2023-12-27 18:51:14,824][20381] Updated weights for policy 1, policy_version 1904601 (0.0008) [2023-12-27 18:51:15,460][20381] Updated weights for policy 1, policy_version 1904611 (0.0008) [2023-12-27 18:51:15,489][20347] Updated weights for policy 0, policy_version 1906213 (0.0006) [2023-12-27 18:51:15,514][20381] Updated weights for policy 1, policy_version 1904621 (0.0008) [2023-12-27 18:51:15,541][20347] Updated weights for policy 0, policy_version 1906223 (0.0005) [2023-12-27 18:51:15,572][20381] Updated weights for policy 1, policy_version 1904631 (0.0007) [2023-12-27 18:51:15,592][20347] Updated weights for policy 0, policy_version 1906233 (0.0005) [2023-12-27 18:51:16,158][20347] Updated weights for policy 0, policy_version 1906243 (0.0005) [2023-12-27 18:51:16,185][20381] Updated weights for policy 1, policy_version 1904641 (0.0007) [2023-12-27 18:51:16,222][20347] Updated weights for policy 0, policy_version 1906253 (0.0006) [2023-12-27 18:51:16,245][20381] Updated weights for policy 1, policy_version 1904651 (0.0007) [2023-12-27 18:51:16,274][20347] Updated weights for policy 0, policy_version 1906263 (0.0005) [2023-12-27 18:51:16,301][20381] Updated weights for policy 1, policy_version 1904661 (0.0008) [2023-12-27 18:51:16,355][20381] Updated weights for policy 1, policy_version 1904671 (0.0008) [2023-12-27 18:51:16,837][20347] Updated weights for policy 0, policy_version 1906273 (0.0005) [2023-12-27 18:51:16,895][20347] Updated weights for policy 0, policy_version 1906283 (0.0007) [2023-12-27 18:51:16,950][20347] Updated weights for policy 0, policy_version 1906293 (0.0008) [2023-12-27 18:51:17,010][20347] Updated weights for policy 0, policy_version 1906303 (0.0006) [2023-12-27 18:51:17,012][20381] Updated weights for policy 1, policy_version 1904681 (0.0005) [2023-12-27 18:51:17,065][20381] Updated weights for policy 1, policy_version 1904691 (0.0008) [2023-12-27 18:51:17,114][20381] Updated weights for policy 1, policy_version 1904701 (0.0007) [2023-12-27 18:51:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 975765504. Throughput: 0: 10071.4, 1: 10175.0. Samples: 975773384. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '833.830')] [2023-12-27 18:51:17,570][20347] Updated weights for policy 0, policy_version 1906313 (0.0005) [2023-12-27 18:51:17,615][20347] Updated weights for policy 0, policy_version 1906323 (0.0005) [2023-12-27 18:51:17,663][20347] Updated weights for policy 0, policy_version 1906333 (0.0005) [2023-12-27 18:51:17,784][20381] Updated weights for policy 1, policy_version 1904711 (0.0008) [2023-12-27 18:51:17,846][20381] Updated weights for policy 1, policy_version 1904721 (0.0008) [2023-12-27 18:51:17,904][20381] Updated weights for policy 1, policy_version 1904731 (0.0007) [2023-12-27 18:51:18,295][20347] Updated weights for policy 0, policy_version 1906343 (0.0005) [2023-12-27 18:51:18,349][20347] Updated weights for policy 0, policy_version 1906353 (0.0006) [2023-12-27 18:51:18,409][20347] Updated weights for policy 0, policy_version 1906363 (0.0006) [2023-12-27 18:51:18,473][20381] Updated weights for policy 1, policy_version 1904741 (0.0008) [2023-12-27 18:51:18,531][20381] Updated weights for policy 1, policy_version 1904751 (0.0008) [2023-12-27 18:51:18,590][20381] Updated weights for policy 1, policy_version 1904761 (0.0005) [2023-12-27 18:51:19,054][20347] Updated weights for policy 0, policy_version 1906373 (0.0005) [2023-12-27 18:51:19,115][20347] Updated weights for policy 0, policy_version 1906383 (0.0007) [2023-12-27 18:51:19,172][20347] Updated weights for policy 0, policy_version 1906393 (0.0006) [2023-12-27 18:51:19,268][20381] Updated weights for policy 1, policy_version 1904771 (0.0007) [2023-12-27 18:51:19,320][20381] Updated weights for policy 1, policy_version 1904781 (0.0011) [2023-12-27 18:51:19,390][20381] Updated weights for policy 1, policy_version 1904791 (0.0010) [2023-12-27 18:51:19,847][20347] Updated weights for policy 0, policy_version 1906403 (0.0007) [2023-12-27 18:51:19,905][20347] Updated weights for policy 0, policy_version 1906413 (0.0006) [2023-12-27 18:51:19,973][20347] Updated weights for policy 0, policy_version 1906423 (0.0008) [2023-12-27 18:51:20,092][20381] Updated weights for policy 1, policy_version 1904801 (0.0010) [2023-12-27 18:51:20,150][20381] Updated weights for policy 1, policy_version 1904811 (0.0007) [2023-12-27 18:51:20,211][20381] Updated weights for policy 1, policy_version 1904821 (0.0006) [2023-12-27 18:51:20,272][20381] Updated weights for policy 1, policy_version 1904831 (0.0007) [2023-12-27 18:51:20,653][20347] Updated weights for policy 0, policy_version 1906433 (0.0006) [2023-12-27 18:51:20,712][20347] Updated weights for policy 0, policy_version 1906443 (0.0006) [2023-12-27 18:51:20,777][20347] Updated weights for policy 0, policy_version 1906453 (0.0006) [2023-12-27 18:51:20,834][20347] Updated weights for policy 0, policy_version 1906463 (0.0005) [2023-12-27 18:51:20,903][20381] Updated weights for policy 1, policy_version 1904841 (0.0008) [2023-12-27 18:51:20,961][20381] Updated weights for policy 1, policy_version 1904851 (0.0008) [2023-12-27 18:51:21,027][20381] Updated weights for policy 1, policy_version 1904861 (0.0008) [2023-12-27 18:51:21,534][20347] Updated weights for policy 0, policy_version 1906473 (0.0007) [2023-12-27 18:51:21,590][20347] Updated weights for policy 0, policy_version 1906483 (0.0006) [2023-12-27 18:51:21,660][20347] Updated weights for policy 0, policy_version 1906493 (0.0008) [2023-12-27 18:51:21,810][20381] Updated weights for policy 1, policy_version 1904871 (0.0008) [2023-12-27 18:51:21,869][20381] Updated weights for policy 1, policy_version 1904881 (0.0010) [2023-12-27 18:51:21,927][20381] Updated weights for policy 1, policy_version 1904891 (0.0007) [2023-12-27 18:51:22,308][20347] Updated weights for policy 0, policy_version 1906503 (0.0007) [2023-12-27 18:51:22,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 975863808. Throughput: 0: 10130.6, 1: 10171.5. Samples: 975837856. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.450')] [2023-12-27 18:51:22,371][20347] Updated weights for policy 0, policy_version 1906513 (0.0008) [2023-12-27 18:51:22,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001904896_487710720.pth... [2023-12-27 18:51:22,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001903712_487407616.pth [2023-12-27 18:51:22,430][20347] Updated weights for policy 0, policy_version 1906523 (0.0006) [2023-12-27 18:51:22,452][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001906528_488161280.pth... [2023-12-27 18:51:22,456][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001905312_487849984.pth [2023-12-27 18:51:22,613][20381] Updated weights for policy 1, policy_version 1904901 (0.0006) [2023-12-27 18:51:22,672][20381] Updated weights for policy 1, policy_version 1904911 (0.0009) [2023-12-27 18:51:22,737][20381] Updated weights for policy 1, policy_version 1904921 (0.0008) [2023-12-27 18:51:23,076][20347] Updated weights for policy 0, policy_version 1906533 (0.0006) [2023-12-27 18:51:23,133][20347] Updated weights for policy 0, policy_version 1906543 (0.0008) [2023-12-27 18:51:23,196][20347] Updated weights for policy 0, policy_version 1906553 (0.0008) [2023-12-27 18:51:23,355][20381] Updated weights for policy 1, policy_version 1904931 (0.0008) [2023-12-27 18:51:23,406][20381] Updated weights for policy 1, policy_version 1904941 (0.0009) [2023-12-27 18:51:23,466][20381] Updated weights for policy 1, policy_version 1904951 (0.0010) [2023-12-27 18:51:23,842][20347] Updated weights for policy 0, policy_version 1906563 (0.0007) [2023-12-27 18:51:23,909][20347] Updated weights for policy 0, policy_version 1906573 (0.0005) [2023-12-27 18:51:23,965][20347] Updated weights for policy 0, policy_version 1906583 (0.0010) [2023-12-27 18:51:24,134][20381] Updated weights for policy 1, policy_version 1904961 (0.0010) [2023-12-27 18:51:24,193][20381] Updated weights for policy 1, policy_version 1904971 (0.0006) [2023-12-27 18:51:24,250][20381] Updated weights for policy 1, policy_version 1904981 (0.0007) [2023-12-27 18:51:24,303][20381] Updated weights for policy 1, policy_version 1904991 (0.0010) [2023-12-27 18:51:24,602][20347] Updated weights for policy 0, policy_version 1906593 (0.0009) [2023-12-27 18:51:24,653][20347] Updated weights for policy 0, policy_version 1906603 (0.0007) [2023-12-27 18:51:24,711][20347] Updated weights for policy 0, policy_version 1906613 (0.0010) [2023-12-27 18:51:24,760][20347] Updated weights for policy 0, policy_version 1906623 (0.0010) [2023-12-27 18:51:24,970][20381] Updated weights for policy 1, policy_version 1905001 (0.0007) [2023-12-27 18:51:25,025][20381] Updated weights for policy 1, policy_version 1905011 (0.0005) [2023-12-27 18:51:25,077][20381] Updated weights for policy 1, policy_version 1905021 (0.0008) [2023-12-27 18:51:25,416][20347] Updated weights for policy 0, policy_version 1906633 (0.0010) [2023-12-27 18:51:25,470][20347] Updated weights for policy 0, policy_version 1906643 (0.0010) [2023-12-27 18:51:25,523][20347] Updated weights for policy 0, policy_version 1906653 (0.0010) [2023-12-27 18:51:25,662][20381] Updated weights for policy 1, policy_version 1905031 (0.0007) [2023-12-27 18:51:25,719][20381] Updated weights for policy 1, policy_version 1905042 (0.0009) [2023-12-27 18:51:25,779][20381] Updated weights for policy 1, policy_version 1905052 (0.0005) [2023-12-27 18:51:26,155][20347] Updated weights for policy 0, policy_version 1906663 (0.0010) [2023-12-27 18:51:26,213][20347] Updated weights for policy 0, policy_version 1906673 (0.0010) [2023-12-27 18:51:26,261][20347] Updated weights for policy 0, policy_version 1906683 (0.0010) [2023-12-27 18:51:26,458][20381] Updated weights for policy 1, policy_version 1905062 (0.0007) [2023-12-27 18:51:26,508][20381] Updated weights for policy 1, policy_version 1905072 (0.0007) [2023-12-27 18:51:26,555][20381] Updated weights for policy 1, policy_version 1905082 (0.0007) [2023-12-27 18:51:26,909][20347] Updated weights for policy 0, policy_version 1906693 (0.0008) [2023-12-27 18:51:26,959][20347] Updated weights for policy 0, policy_version 1906703 (0.0006) [2023-12-27 18:51:27,013][20347] Updated weights for policy 0, policy_version 1906713 (0.0008) [2023-12-27 18:51:27,221][20381] Updated weights for policy 1, policy_version 1905092 (0.0007) [2023-12-27 18:51:27,276][20381] Updated weights for policy 1, policy_version 1905102 (0.0008) [2023-12-27 18:51:27,332][20381] Updated weights for policy 1, policy_version 1905112 (0.0007) [2023-12-27 18:51:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 975970304. Throughput: 0: 10185.4, 1: 10202.9. Samples: 975962620. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.830')] [2023-12-27 18:51:27,656][20347] Updated weights for policy 0, policy_version 1906723 (0.0007) [2023-12-27 18:51:27,716][20347] Updated weights for policy 0, policy_version 1906733 (0.0005) [2023-12-27 18:51:27,774][20347] Updated weights for policy 0, policy_version 1906743 (0.0005) [2023-12-27 18:51:28,058][20381] Updated weights for policy 1, policy_version 1905122 (0.0008) [2023-12-27 18:51:28,111][20381] Updated weights for policy 1, policy_version 1905132 (0.0008) [2023-12-27 18:51:28,174][20381] Updated weights for policy 1, policy_version 1905142 (0.0007) [2023-12-27 18:51:28,232][20381] Updated weights for policy 1, policy_version 1905152 (0.0007) [2023-12-27 18:51:28,333][20347] Updated weights for policy 0, policy_version 1906753 (0.0006) [2023-12-27 18:51:28,397][20347] Updated weights for policy 0, policy_version 1906763 (0.0007) [2023-12-27 18:51:28,445][20347] Updated weights for policy 0, policy_version 1906773 (0.0008) [2023-12-27 18:51:28,495][20347] Updated weights for policy 0, policy_version 1906783 (0.0007) [2023-12-27 18:51:28,931][20381] Updated weights for policy 1, policy_version 1905162 (0.0007) [2023-12-27 18:51:28,989][20381] Updated weights for policy 1, policy_version 1905172 (0.0007) [2023-12-27 18:51:29,035][20381] Updated weights for policy 1, policy_version 1905182 (0.0007) [2023-12-27 18:51:29,092][20347] Updated weights for policy 0, policy_version 1906793 (0.0007) [2023-12-27 18:51:29,143][20347] Updated weights for policy 0, policy_version 1906803 (0.0007) [2023-12-27 18:51:29,191][20347] Updated weights for policy 0, policy_version 1906813 (0.0007) [2023-12-27 18:51:29,682][20381] Updated weights for policy 1, policy_version 1905192 (0.0008) [2023-12-27 18:51:29,738][20381] Updated weights for policy 1, policy_version 1905202 (0.0007) [2023-12-27 18:51:29,792][20381] Updated weights for policy 1, policy_version 1905212 (0.0007) [2023-12-27 18:51:29,869][20347] Updated weights for policy 0, policy_version 1906823 (0.0007) [2023-12-27 18:51:29,925][20347] Updated weights for policy 0, policy_version 1906833 (0.0008) [2023-12-27 18:51:29,989][20347] Updated weights for policy 0, policy_version 1906843 (0.0007) [2023-12-27 18:51:30,429][20381] Updated weights for policy 1, policy_version 1905222 (0.0007) [2023-12-27 18:51:30,473][20381] Updated weights for policy 1, policy_version 1905232 (0.0005) [2023-12-27 18:51:30,522][20381] Updated weights for policy 1, policy_version 1905242 (0.0005) [2023-12-27 18:51:30,722][20347] Updated weights for policy 0, policy_version 1906853 (0.0006) [2023-12-27 18:51:30,776][20347] Updated weights for policy 0, policy_version 1906863 (0.0009) [2023-12-27 18:51:30,830][20347] Updated weights for policy 0, policy_version 1906873 (0.0010) [2023-12-27 18:51:31,115][20381] Updated weights for policy 1, policy_version 1905252 (0.0007) [2023-12-27 18:51:31,174][20381] Updated weights for policy 1, policy_version 1905262 (0.0009) [2023-12-27 18:51:31,225][20381] Updated weights for policy 1, policy_version 1905272 (0.0007) [2023-12-27 18:51:31,504][20347] Updated weights for policy 0, policy_version 1906883 (0.0010) [2023-12-27 18:51:31,557][20347] Updated weights for policy 0, policy_version 1906893 (0.0010) [2023-12-27 18:51:31,610][20347] Updated weights for policy 0, policy_version 1906903 (0.0010) [2023-12-27 18:51:31,915][20381] Updated weights for policy 1, policy_version 1905282 (0.0011) [2023-12-27 18:51:31,968][20381] Updated weights for policy 1, policy_version 1905292 (0.0006) [2023-12-27 18:51:32,018][20381] Updated weights for policy 1, policy_version 1905302 (0.0005) [2023-12-27 18:51:32,072][20381] Updated weights for policy 1, policy_version 1905312 (0.0006) [2023-12-27 18:51:32,347][20347] Updated weights for policy 0, policy_version 1906913 (0.0008) [2023-12-27 18:51:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20438.4). Total num frames: 976076800. Throughput: 0: 10258.7, 1: 10238.5. Samples: 976088516. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.950')] [2023-12-27 18:51:32,408][20347] Updated weights for policy 0, policy_version 1906923 (0.0007) [2023-12-27 18:51:32,474][20347] Updated weights for policy 0, policy_version 1906933 (0.0006) [2023-12-27 18:51:32,539][20347] Updated weights for policy 0, policy_version 1906943 (0.0006) [2023-12-27 18:51:32,696][20381] Updated weights for policy 1, policy_version 1905322 (0.0011) [2023-12-27 18:51:32,758][20381] Updated weights for policy 1, policy_version 1905332 (0.0011) [2023-12-27 18:51:32,816][20381] Updated weights for policy 1, policy_version 1905342 (0.0011) [2023-12-27 18:51:33,110][20347] Updated weights for policy 0, policy_version 1906953 (0.0005) [2023-12-27 18:51:33,158][20347] Updated weights for policy 0, policy_version 1906963 (0.0006) [2023-12-27 18:51:33,208][20347] Updated weights for policy 0, policy_version 1906973 (0.0006) [2023-12-27 18:51:33,435][20381] Updated weights for policy 1, policy_version 1905352 (0.0007) [2023-12-27 18:51:33,486][20381] Updated weights for policy 1, policy_version 1905362 (0.0008) [2023-12-27 18:51:33,534][20381] Updated weights for policy 1, policy_version 1905372 (0.0006) [2023-12-27 18:51:33,776][20347] Updated weights for policy 0, policy_version 1906983 (0.0005) [2023-12-27 18:51:33,835][20347] Updated weights for policy 0, policy_version 1906993 (0.0005) [2023-12-27 18:51:33,889][20347] Updated weights for policy 0, policy_version 1907003 (0.0008) [2023-12-27 18:51:34,166][20381] Updated weights for policy 1, policy_version 1905382 (0.0008) [2023-12-27 18:51:34,224][20381] Updated weights for policy 1, policy_version 1905392 (0.0008) [2023-12-27 18:51:34,283][20381] Updated weights for policy 1, policy_version 1905402 (0.0008) [2023-12-27 18:51:34,556][20347] Updated weights for policy 0, policy_version 1907013 (0.0008) [2023-12-27 18:51:34,616][20347] Updated weights for policy 0, policy_version 1907023 (0.0007) [2023-12-27 18:51:34,678][20347] Updated weights for policy 0, policy_version 1907033 (0.0008) [2023-12-27 18:51:34,956][20381] Updated weights for policy 1, policy_version 1905412 (0.0008) [2023-12-27 18:51:35,012][20381] Updated weights for policy 1, policy_version 1905422 (0.0008) [2023-12-27 18:51:35,069][20381] Updated weights for policy 1, policy_version 1905432 (0.0010) [2023-12-27 18:51:35,345][20347] Updated weights for policy 0, policy_version 1907043 (0.0008) [2023-12-27 18:51:35,396][20347] Updated weights for policy 0, policy_version 1907053 (0.0008) [2023-12-27 18:51:35,446][20347] Updated weights for policy 0, policy_version 1907063 (0.0007) [2023-12-27 18:51:35,703][20381] Updated weights for policy 1, policy_version 1905442 (0.0009) [2023-12-27 18:51:35,770][20381] Updated weights for policy 1, policy_version 1905452 (0.0008) [2023-12-27 18:51:35,830][20381] Updated weights for policy 1, policy_version 1905462 (0.0008) [2023-12-27 18:51:35,893][20381] Updated weights for policy 1, policy_version 1905472 (0.0009) [2023-12-27 18:51:36,121][20347] Updated weights for policy 0, policy_version 1907073 (0.0008) [2023-12-27 18:51:36,187][20347] Updated weights for policy 0, policy_version 1907083 (0.0007) [2023-12-27 18:51:36,248][20347] Updated weights for policy 0, policy_version 1907093 (0.0006) [2023-12-27 18:51:36,313][20347] Updated weights for policy 0, policy_version 1907103 (0.0006) [2023-12-27 18:51:36,599][20381] Updated weights for policy 1, policy_version 1905482 (0.0009) [2023-12-27 18:51:36,650][20381] Updated weights for policy 1, policy_version 1905492 (0.0007) [2023-12-27 18:51:36,708][20381] Updated weights for policy 1, policy_version 1905502 (0.0008) [2023-12-27 18:51:36,872][20347] Updated weights for policy 0, policy_version 1907113 (0.0007) [2023-12-27 18:51:36,929][20347] Updated weights for policy 0, policy_version 1907123 (0.0007) [2023-12-27 18:51:36,981][20347] Updated weights for policy 0, policy_version 1907133 (0.0007) [2023-12-27 18:51:37,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 976183296. Throughput: 0: 10341.3, 1: 10242.4. Samples: 976152976. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.190')] [2023-12-27 18:51:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001905504_487866368.pth... [2023-12-27 18:51:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001907136_488316928.pth... [2023-12-27 18:51:37,378][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001904288_487555072.pth [2023-12-27 18:51:37,378][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001905920_488005632.pth [2023-12-27 18:51:37,430][20381] Updated weights for policy 1, policy_version 1905512 (0.0008) [2023-12-27 18:51:37,486][20381] Updated weights for policy 1, policy_version 1905522 (0.0009) [2023-12-27 18:51:37,537][20381] Updated weights for policy 1, policy_version 1905532 (0.0010) [2023-12-27 18:51:37,670][20347] Updated weights for policy 0, policy_version 1907143 (0.0008) [2023-12-27 18:51:37,728][20347] Updated weights for policy 0, policy_version 1907153 (0.0008) [2023-12-27 18:51:37,782][20347] Updated weights for policy 0, policy_version 1907163 (0.0008) [2023-12-27 18:51:38,245][20381] Updated weights for policy 1, policy_version 1905542 (0.0008) [2023-12-27 18:51:38,303][20381] Updated weights for policy 1, policy_version 1905552 (0.0007) [2023-12-27 18:51:38,360][20381] Updated weights for policy 1, policy_version 1905562 (0.0007) [2023-12-27 18:51:38,539][20347] Updated weights for policy 0, policy_version 1907173 (0.0006) [2023-12-27 18:51:38,599][20347] Updated weights for policy 0, policy_version 1907183 (0.0006) [2023-12-27 18:51:38,660][20347] Updated weights for policy 0, policy_version 1907193 (0.0007) [2023-12-27 18:51:39,059][20381] Updated weights for policy 1, policy_version 1905572 (0.0007) [2023-12-27 18:51:39,124][20381] Updated weights for policy 1, policy_version 1905582 (0.0008) [2023-12-27 18:51:39,172][20381] Updated weights for policy 1, policy_version 1905592 (0.0007) [2023-12-27 18:51:39,230][20347] Updated weights for policy 0, policy_version 1907203 (0.0007) [2023-12-27 18:51:39,294][20347] Updated weights for policy 0, policy_version 1907213 (0.0007) [2023-12-27 18:51:39,367][20347] Updated weights for policy 0, policy_version 1907223 (0.0008) [2023-12-27 18:51:39,963][20381] Updated weights for policy 1, policy_version 1905602 (0.0007) [2023-12-27 18:51:40,029][20381] Updated weights for policy 1, policy_version 1905612 (0.0007) [2023-12-27 18:51:40,095][20381] Updated weights for policy 1, policy_version 1905622 (0.0008) [2023-12-27 18:51:40,113][20347] Updated weights for policy 0, policy_version 1907233 (0.0008) [2023-12-27 18:51:40,153][20381] Updated weights for policy 1, policy_version 1905632 (0.0006) [2023-12-27 18:51:40,178][20347] Updated weights for policy 0, policy_version 1907243 (0.0007) [2023-12-27 18:51:40,235][20347] Updated weights for policy 0, policy_version 1907253 (0.0007) [2023-12-27 18:51:40,294][20347] Updated weights for policy 0, policy_version 1907263 (0.0008) [2023-12-27 18:51:40,853][20381] Updated weights for policy 1, policy_version 1905642 (0.0008) [2023-12-27 18:51:40,910][20381] Updated weights for policy 1, policy_version 1905652 (0.0007) [2023-12-27 18:51:40,964][20381] Updated weights for policy 1, policy_version 1905662 (0.0007) [2023-12-27 18:51:40,994][20347] Updated weights for policy 0, policy_version 1907273 (0.0007) [2023-12-27 18:51:41,058][20347] Updated weights for policy 0, policy_version 1907283 (0.0007) [2023-12-27 18:51:41,116][20347] Updated weights for policy 0, policy_version 1907293 (0.0008) [2023-12-27 18:51:41,728][20381] Updated weights for policy 1, policy_version 1905672 (0.0008) [2023-12-27 18:51:41,796][20381] Updated weights for policy 1, policy_version 1905682 (0.0008) [2023-12-27 18:51:41,847][20347] Updated weights for policy 0, policy_version 1907303 (0.0008) [2023-12-27 18:51:41,861][20381] Updated weights for policy 1, policy_version 1905692 (0.0007) [2023-12-27 18:51:41,906][20347] Updated weights for policy 0, policy_version 1907313 (0.0008) [2023-12-27 18:51:41,971][20347] Updated weights for policy 0, policy_version 1907323 (0.0007) [2023-12-27 18:51:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 976281600. Throughput: 0: 10365.3, 1: 10227.0. Samples: 976274028. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:42,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.530')] [2023-12-27 18:51:42,567][20381] Updated weights for policy 1, policy_version 1905702 (0.0007) [2023-12-27 18:51:42,589][20347] Updated weights for policy 0, policy_version 1907333 (0.0007) [2023-12-27 18:51:42,624][20381] Updated weights for policy 1, policy_version 1905712 (0.0008) [2023-12-27 18:51:42,645][20347] Updated weights for policy 0, policy_version 1907343 (0.0006) [2023-12-27 18:51:42,674][20381] Updated weights for policy 1, policy_version 1905722 (0.0005) [2023-12-27 18:51:42,698][20347] Updated weights for policy 0, policy_version 1907353 (0.0006) [2023-12-27 18:51:43,338][20381] Updated weights for policy 1, policy_version 1905732 (0.0007) [2023-12-27 18:51:43,392][20347] Updated weights for policy 0, policy_version 1907363 (0.0008) [2023-12-27 18:51:43,394][20381] Updated weights for policy 1, policy_version 1905742 (0.0006) [2023-12-27 18:51:43,445][20347] Updated weights for policy 0, policy_version 1907373 (0.0007) [2023-12-27 18:51:43,447][20381] Updated weights for policy 1, policy_version 1905752 (0.0005) [2023-12-27 18:51:43,500][20347] Updated weights for policy 0, policy_version 1907383 (0.0007) [2023-12-27 18:51:43,568][20291] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 18:51:44,058][20381] Updated weights for policy 1, policy_version 1905762 (0.0006) [2023-12-27 18:51:44,108][20381] Updated weights for policy 1, policy_version 1905772 (0.0008) [2023-12-27 18:51:44,128][20347] Updated weights for policy 0, policy_version 1907393 (0.0007) [2023-12-27 18:51:44,165][20381] Updated weights for policy 1, policy_version 1905782 (0.0006) [2023-12-27 18:51:44,176][20347] Updated weights for policy 0, policy_version 1907403 (0.0005) [2023-12-27 18:51:44,219][20381] Updated weights for policy 1, policy_version 1905792 (0.0006) [2023-12-27 18:51:44,238][20347] Updated weights for policy 0, policy_version 1907413 (0.0007) [2023-12-27 18:51:44,297][20347] Updated weights for policy 0, policy_version 1907423 (0.0008) [2023-12-27 18:51:44,905][20381] Updated weights for policy 1, policy_version 1905802 (0.0005) [2023-12-27 18:51:44,968][20381] Updated weights for policy 1, policy_version 1905812 (0.0006) [2023-12-27 18:51:45,010][20347] Updated weights for policy 0, policy_version 1907433 (0.0008) [2023-12-27 18:51:45,031][20381] Updated weights for policy 1, policy_version 1905822 (0.0006) [2023-12-27 18:51:45,074][20347] Updated weights for policy 0, policy_version 1907443 (0.0009) [2023-12-27 18:51:45,132][20347] Updated weights for policy 0, policy_version 1907453 (0.0008) [2023-12-27 18:51:45,670][20381] Updated weights for policy 1, policy_version 1905832 (0.0005) [2023-12-27 18:51:45,718][20381] Updated weights for policy 1, policy_version 1905842 (0.0006) [2023-12-27 18:51:45,763][20381] Updated weights for policy 1, policy_version 1905852 (0.0007) [2023-12-27 18:51:45,840][20347] Updated weights for policy 0, policy_version 1907463 (0.0006) [2023-12-27 18:51:45,897][20347] Updated weights for policy 0, policy_version 1907473 (0.0005) [2023-12-27 18:51:45,951][20347] Updated weights for policy 0, policy_version 1907483 (0.0005) [2023-12-27 18:51:46,441][20381] Updated weights for policy 1, policy_version 1905862 (0.0010) [2023-12-27 18:51:46,497][20381] Updated weights for policy 1, policy_version 1905872 (0.0011) [2023-12-27 18:51:46,532][20347] Updated weights for policy 0, policy_version 1907493 (0.0010) [2023-12-27 18:51:46,556][20381] Updated weights for policy 1, policy_version 1905882 (0.0011) [2023-12-27 18:51:46,581][20347] Updated weights for policy 0, policy_version 1907503 (0.0010) [2023-12-27 18:51:46,630][20347] Updated weights for policy 0, policy_version 1907513 (0.0010) [2023-12-27 18:51:47,190][20381] Updated weights for policy 1, policy_version 1905892 (0.0008) [2023-12-27 18:51:47,254][20381] Updated weights for policy 1, policy_version 1905902 (0.0007) [2023-12-27 18:51:47,302][20347] Updated weights for policy 0, policy_version 1907523 (0.0009) [2023-12-27 18:51:47,312][20381] Updated weights for policy 1, policy_version 1905912 (0.0010) [2023-12-27 18:51:47,348][20347] Updated weights for policy 0, policy_version 1907533 (0.0010) [2023-12-27 18:51:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 976388096. Throughput: 0: 10368.5, 1: 10309.8. Samples: 976398036. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.360')] [2023-12-27 18:51:47,394][20347] Updated weights for policy 0, policy_version 1907543 (0.0010) [2023-12-27 18:51:47,813][20381] Updated weights for policy 1, policy_version 1905922 (0.0005) [2023-12-27 18:51:47,876][20381] Updated weights for policy 1, policy_version 1905932 (0.0005) [2023-12-27 18:51:47,935][20381] Updated weights for policy 1, policy_version 1905942 (0.0005) [2023-12-27 18:51:47,991][20381] Updated weights for policy 1, policy_version 1905952 (0.0005) [2023-12-27 18:51:48,108][20347] Updated weights for policy 0, policy_version 1907553 (0.0007) [2023-12-27 18:51:48,160][20347] Updated weights for policy 0, policy_version 1907563 (0.0011) [2023-12-27 18:51:48,212][20347] Updated weights for policy 0, policy_version 1907573 (0.0010) [2023-12-27 18:51:48,270][20347] Updated weights for policy 0, policy_version 1907583 (0.0011) [2023-12-27 18:51:48,573][20381] Updated weights for policy 1, policy_version 1905962 (0.0007) [2023-12-27 18:51:48,626][20381] Updated weights for policy 1, policy_version 1905972 (0.0007) [2023-12-27 18:51:48,683][20381] Updated weights for policy 1, policy_version 1905982 (0.0005) [2023-12-27 18:51:48,945][20347] Updated weights for policy 0, policy_version 1907593 (0.0010) [2023-12-27 18:51:48,994][20347] Updated weights for policy 0, policy_version 1907603 (0.0010) [2023-12-27 18:51:49,056][20347] Updated weights for policy 0, policy_version 1907613 (0.0011) [2023-12-27 18:51:49,335][20381] Updated weights for policy 1, policy_version 1905992 (0.0008) [2023-12-27 18:51:49,400][20381] Updated weights for policy 1, policy_version 1906002 (0.0008) [2023-12-27 18:51:49,462][20381] Updated weights for policy 1, policy_version 1906012 (0.0008) [2023-12-27 18:51:49,786][20347] Updated weights for policy 0, policy_version 1907623 (0.0010) [2023-12-27 18:51:49,865][20347] Updated weights for policy 0, policy_version 1907633 (0.0011) [2023-12-27 18:51:49,926][20347] Updated weights for policy 0, policy_version 1907643 (0.0011) [2023-12-27 18:51:50,141][20381] Updated weights for policy 1, policy_version 1906022 (0.0007) [2023-12-27 18:51:50,205][20381] Updated weights for policy 1, policy_version 1906032 (0.0007) [2023-12-27 18:51:50,263][20381] Updated weights for policy 1, policy_version 1906042 (0.0006) [2023-12-27 18:51:50,617][20347] Updated weights for policy 0, policy_version 1907653 (0.0010) [2023-12-27 18:51:50,676][20347] Updated weights for policy 0, policy_version 1907663 (0.0010) [2023-12-27 18:51:50,734][20347] Updated weights for policy 0, policy_version 1907673 (0.0010) [2023-12-27 18:51:50,956][20381] Updated weights for policy 1, policy_version 1906052 (0.0008) [2023-12-27 18:51:51,025][20381] Updated weights for policy 1, policy_version 1906062 (0.0010) [2023-12-27 18:51:51,087][20381] Updated weights for policy 1, policy_version 1906072 (0.0010) [2023-12-27 18:51:51,369][20347] Updated weights for policy 0, policy_version 1907683 (0.0010) [2023-12-27 18:51:51,428][20347] Updated weights for policy 0, policy_version 1907693 (0.0010) [2023-12-27 18:51:51,487][20347] Updated weights for policy 0, policy_version 1907703 (0.0011) [2023-12-27 18:51:51,823][20381] Updated weights for policy 1, policy_version 1906082 (0.0011) [2023-12-27 18:51:51,880][20381] Updated weights for policy 1, policy_version 1906092 (0.0011) [2023-12-27 18:51:51,937][20381] Updated weights for policy 1, policy_version 1906102 (0.0010) [2023-12-27 18:51:51,995][20381] Updated weights for policy 1, policy_version 1906112 (0.0010) [2023-12-27 18:51:52,199][20347] Updated weights for policy 0, policy_version 1907713 (0.0011) [2023-12-27 18:51:52,257][20347] Updated weights for policy 0, policy_version 1907723 (0.0007) [2023-12-27 18:51:52,320][20347] Updated weights for policy 0, policy_version 1907733 (0.0007) [2023-12-27 18:51:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 976486400. Throughput: 0: 10371.1, 1: 10360.1. Samples: 976461292. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.700')] [2023-12-27 18:51:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001906112_488022016.pth... [2023-12-27 18:51:52,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001904896_487710720.pth [2023-12-27 18:51:52,389][20347] Updated weights for policy 0, policy_version 1907743 (0.0007) [2023-12-27 18:51:52,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001907744_488472576.pth... [2023-12-27 18:51:52,396][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001906528_488161280.pth [2023-12-27 18:51:52,709][20381] Updated weights for policy 1, policy_version 1906122 (0.0007) [2023-12-27 18:51:52,768][20381] Updated weights for policy 1, policy_version 1906132 (0.0006) [2023-12-27 18:51:52,830][20381] Updated weights for policy 1, policy_version 1906142 (0.0005) [2023-12-27 18:51:53,053][20347] Updated weights for policy 0, policy_version 1907753 (0.0006) [2023-12-27 18:51:53,111][20347] Updated weights for policy 0, policy_version 1907763 (0.0007) [2023-12-27 18:51:53,163][20347] Updated weights for policy 0, policy_version 1907773 (0.0008) [2023-12-27 18:51:53,448][20381] Updated weights for policy 1, policy_version 1906152 (0.0008) [2023-12-27 18:51:53,501][20381] Updated weights for policy 1, policy_version 1906162 (0.0005) [2023-12-27 18:51:53,560][20381] Updated weights for policy 1, policy_version 1906172 (0.0010) [2023-12-27 18:51:53,753][20347] Updated weights for policy 0, policy_version 1907783 (0.0010) [2023-12-27 18:51:53,811][20347] Updated weights for policy 0, policy_version 1907793 (0.0006) [2023-12-27 18:51:53,870][20347] Updated weights for policy 0, policy_version 1907803 (0.0007) [2023-12-27 18:51:54,204][20381] Updated weights for policy 1, policy_version 1906182 (0.0010) [2023-12-27 18:51:54,263][20381] Updated weights for policy 1, policy_version 1906192 (0.0011) [2023-12-27 18:51:54,324][20381] Updated weights for policy 1, policy_version 1906202 (0.0011) [2023-12-27 18:51:54,525][20347] Updated weights for policy 0, policy_version 1907813 (0.0011) [2023-12-27 18:51:54,583][20347] Updated weights for policy 0, policy_version 1907823 (0.0010) [2023-12-27 18:51:54,644][20347] Updated weights for policy 0, policy_version 1907833 (0.0009) [2023-12-27 18:51:54,970][20381] Updated weights for policy 1, policy_version 1906212 (0.0009) [2023-12-27 18:51:55,031][20381] Updated weights for policy 1, policy_version 1906222 (0.0006) [2023-12-27 18:51:55,087][20381] Updated weights for policy 1, policy_version 1906232 (0.0005) [2023-12-27 18:51:55,331][20347] Updated weights for policy 0, policy_version 1907843 (0.0010) [2023-12-27 18:51:55,386][20347] Updated weights for policy 0, policy_version 1907853 (0.0007) [2023-12-27 18:51:55,444][20347] Updated weights for policy 0, policy_version 1907863 (0.0008) [2023-12-27 18:51:55,709][20381] Updated weights for policy 1, policy_version 1906242 (0.0007) [2023-12-27 18:51:55,755][20381] Updated weights for policy 1, policy_version 1906252 (0.0005) [2023-12-27 18:51:55,809][20381] Updated weights for policy 1, policy_version 1906262 (0.0006) [2023-12-27 18:51:55,869][20381] Updated weights for policy 1, policy_version 1906272 (0.0007) [2023-12-27 18:51:56,072][20347] Updated weights for policy 0, policy_version 1907873 (0.0006) [2023-12-27 18:51:56,130][20347] Updated weights for policy 0, policy_version 1907883 (0.0005) [2023-12-27 18:51:56,181][20347] Updated weights for policy 0, policy_version 1907893 (0.0005) [2023-12-27 18:51:56,232][20347] Updated weights for policy 0, policy_version 1907903 (0.0005) [2023-12-27 18:51:56,546][20381] Updated weights for policy 1, policy_version 1906282 (0.0011) [2023-12-27 18:51:56,591][20381] Updated weights for policy 1, policy_version 1906292 (0.0010) [2023-12-27 18:51:56,642][20381] Updated weights for policy 1, policy_version 1906302 (0.0010) [2023-12-27 18:51:56,754][20347] Updated weights for policy 0, policy_version 1907913 (0.0007) [2023-12-27 18:51:56,800][20347] Updated weights for policy 0, policy_version 1907923 (0.0007) [2023-12-27 18:51:56,843][20347] Updated weights for policy 0, policy_version 1907933 (0.0007) [2023-12-27 18:51:57,321][20381] Updated weights for policy 1, policy_version 1906312 (0.0008) [2023-12-27 18:51:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 976592896. Throughput: 0: 10401.1, 1: 10389.0. Samples: 976585652. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:51:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.470')] [2023-12-27 18:51:57,382][20381] Updated weights for policy 1, policy_version 1906322 (0.0009) [2023-12-27 18:51:57,425][20381] Updated weights for policy 1, policy_version 1906332 (0.0009) [2023-12-27 18:51:57,517][20347] Updated weights for policy 0, policy_version 1907943 (0.0007) [2023-12-27 18:51:57,563][20347] Updated weights for policy 0, policy_version 1907953 (0.0010) [2023-12-27 18:51:57,607][20347] Updated weights for policy 0, policy_version 1907963 (0.0009) [2023-12-27 18:51:58,091][20381] Updated weights for policy 1, policy_version 1906342 (0.0009) [2023-12-27 18:51:58,149][20381] Updated weights for policy 1, policy_version 1906352 (0.0007) [2023-12-27 18:51:58,210][20381] Updated weights for policy 1, policy_version 1906362 (0.0008) [2023-12-27 18:51:58,287][20347] Updated weights for policy 0, policy_version 1907973 (0.0007) [2023-12-27 18:51:58,344][20347] Updated weights for policy 0, policy_version 1907983 (0.0008) [2023-12-27 18:51:58,409][20347] Updated weights for policy 0, policy_version 1907993 (0.0007) [2023-12-27 18:51:58,987][20381] Updated weights for policy 1, policy_version 1906372 (0.0009) [2023-12-27 18:51:59,048][20381] Updated weights for policy 1, policy_version 1906382 (0.0007) [2023-12-27 18:51:59,103][20381] Updated weights for policy 1, policy_version 1906392 (0.0007) [2023-12-27 18:51:59,216][20347] Updated weights for policy 0, policy_version 1908003 (0.0007) [2023-12-27 18:51:59,278][20347] Updated weights for policy 0, policy_version 1908013 (0.0008) [2023-12-27 18:51:59,340][20347] Updated weights for policy 0, policy_version 1908023 (0.0007) [2023-12-27 18:51:59,811][20381] Updated weights for policy 1, policy_version 1906402 (0.0007) [2023-12-27 18:51:59,881][20381] Updated weights for policy 1, policy_version 1906412 (0.0008) [2023-12-27 18:51:59,946][20381] Updated weights for policy 1, policy_version 1906422 (0.0011) [2023-12-27 18:51:59,983][20347] Updated weights for policy 0, policy_version 1908033 (0.0007) [2023-12-27 18:52:00,005][20381] Updated weights for policy 1, policy_version 1906432 (0.0010) [2023-12-27 18:52:00,039][20347] Updated weights for policy 0, policy_version 1908043 (0.0006) [2023-12-27 18:52:00,087][20347] Updated weights for policy 0, policy_version 1908053 (0.0007) [2023-12-27 18:52:00,140][20347] Updated weights for policy 0, policy_version 1908063 (0.0007) [2023-12-27 18:52:00,683][20381] Updated weights for policy 1, policy_version 1906442 (0.0010) [2023-12-27 18:52:00,740][20381] Updated weights for policy 1, policy_version 1906452 (0.0010) [2023-12-27 18:52:00,792][20381] Updated weights for policy 1, policy_version 1906462 (0.0009) [2023-12-27 18:52:00,806][20347] Updated weights for policy 0, policy_version 1908073 (0.0006) [2023-12-27 18:52:00,855][20347] Updated weights for policy 0, policy_version 1908083 (0.0009) [2023-12-27 18:52:00,906][20347] Updated weights for policy 0, policy_version 1908093 (0.0005) [2023-12-27 18:52:01,491][20381] Updated weights for policy 1, policy_version 1906472 (0.0007) [2023-12-27 18:52:01,550][20381] Updated weights for policy 1, policy_version 1906482 (0.0005) [2023-12-27 18:52:01,561][20347] Updated weights for policy 0, policy_version 1908103 (0.0005) [2023-12-27 18:52:01,606][20381] Updated weights for policy 1, policy_version 1906492 (0.0007) [2023-12-27 18:52:01,614][20347] Updated weights for policy 0, policy_version 1908113 (0.0006) [2023-12-27 18:52:01,679][20347] Updated weights for policy 0, policy_version 1908123 (0.0006) [2023-12-27 18:52:02,273][20381] Updated weights for policy 1, policy_version 1906502 (0.0010) [2023-12-27 18:52:02,298][20347] Updated weights for policy 0, policy_version 1908133 (0.0008) [2023-12-27 18:52:02,327][20381] Updated weights for policy 1, policy_version 1906512 (0.0007) [2023-12-27 18:52:02,357][20347] Updated weights for policy 0, policy_version 1908143 (0.0007) [2023-12-27 18:52:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 976691200. Throughput: 0: 10478.6, 1: 10310.2. Samples: 976708884. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.570')] [2023-12-27 18:52:02,384][20381] Updated weights for policy 1, policy_version 1906522 (0.0011) [2023-12-27 18:52:02,417][20347] Updated weights for policy 0, policy_version 1908153 (0.0009) [2023-12-27 18:52:03,029][20347] Updated weights for policy 0, policy_version 1908163 (0.0010) [2023-12-27 18:52:03,044][20381] Updated weights for policy 1, policy_version 1906532 (0.0011) [2023-12-27 18:52:03,079][20347] Updated weights for policy 0, policy_version 1908173 (0.0005) [2023-12-27 18:52:03,099][20381] Updated weights for policy 1, policy_version 1906542 (0.0010) [2023-12-27 18:52:03,130][20347] Updated weights for policy 0, policy_version 1908183 (0.0005) [2023-12-27 18:52:03,148][20381] Updated weights for policy 1, policy_version 1906552 (0.0010) [2023-12-27 18:52:03,774][20381] Updated weights for policy 1, policy_version 1906562 (0.0010) [2023-12-27 18:52:03,828][20381] Updated weights for policy 1, policy_version 1906572 (0.0010) [2023-12-27 18:52:03,836][20347] Updated weights for policy 0, policy_version 1908193 (0.0007) [2023-12-27 18:52:03,889][20347] Updated weights for policy 0, policy_version 1908203 (0.0008) [2023-12-27 18:52:03,891][20381] Updated weights for policy 1, policy_version 1906582 (0.0011) [2023-12-27 18:52:03,952][20347] Updated weights for policy 0, policy_version 1908213 (0.0008) [2023-12-27 18:52:03,967][20381] Updated weights for policy 1, policy_version 1906592 (0.0010) [2023-12-27 18:52:04,007][20347] Updated weights for policy 0, policy_version 1908223 (0.0008) [2023-12-27 18:52:04,639][20381] Updated weights for policy 1, policy_version 1906602 (0.0011) [2023-12-27 18:52:04,689][20381] Updated weights for policy 1, policy_version 1906612 (0.0011) [2023-12-27 18:52:04,758][20381] Updated weights for policy 1, policy_version 1906622 (0.0010) [2023-12-27 18:52:04,776][20347] Updated weights for policy 0, policy_version 1908233 (0.0007) [2023-12-27 18:52:04,821][20347] Updated weights for policy 0, policy_version 1908243 (0.0008) [2023-12-27 18:52:04,866][20347] Updated weights for policy 0, policy_version 1908253 (0.0008) [2023-12-27 18:52:05,333][20381] Updated weights for policy 1, policy_version 1906632 (0.0010) [2023-12-27 18:52:05,391][20381] Updated weights for policy 1, policy_version 1906642 (0.0010) [2023-12-27 18:52:05,436][20381] Updated weights for policy 1, policy_version 1906652 (0.0010) [2023-12-27 18:52:05,614][20347] Updated weights for policy 0, policy_version 1908263 (0.0008) [2023-12-27 18:52:05,670][20347] Updated weights for policy 0, policy_version 1908273 (0.0007) [2023-12-27 18:52:05,727][20347] Updated weights for policy 0, policy_version 1908283 (0.0008) [2023-12-27 18:52:06,140][20381] Updated weights for policy 1, policy_version 1906662 (0.0009) [2023-12-27 18:52:06,205][20381] Updated weights for policy 1, policy_version 1906672 (0.0008) [2023-12-27 18:52:06,262][20381] Updated weights for policy 1, policy_version 1906682 (0.0008) [2023-12-27 18:52:06,436][20347] Updated weights for policy 0, policy_version 1908293 (0.0007) [2023-12-27 18:52:06,500][20347] Updated weights for policy 0, policy_version 1908303 (0.0007) [2023-12-27 18:52:06,566][20347] Updated weights for policy 0, policy_version 1908313 (0.0008) [2023-12-27 18:52:06,968][20381] Updated weights for policy 1, policy_version 1906692 (0.0007) [2023-12-27 18:52:07,026][20381] Updated weights for policy 1, policy_version 1906702 (0.0007) [2023-12-27 18:52:07,082][20381] Updated weights for policy 1, policy_version 1906712 (0.0008) [2023-12-27 18:52:07,274][20347] Updated weights for policy 0, policy_version 1908323 (0.0008) [2023-12-27 18:52:07,320][20347] Updated weights for policy 0, policy_version 1908333 (0.0010) [2023-12-27 18:52:07,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 976797696. Throughput: 0: 10414.9, 1: 10322.3. Samples: 976771032. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.640')] [2023-12-27 18:52:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001906720_488177664.pth... [2023-12-27 18:52:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001905504_487866368.pth [2023-12-27 18:52:07,393][20347] Updated weights for policy 0, policy_version 1908343 (0.0010) [2023-12-27 18:52:07,444][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001908352_488628224.pth... [2023-12-27 18:52:07,447][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001907136_488316928.pth [2023-12-27 18:52:07,691][20381] Updated weights for policy 1, policy_version 1906722 (0.0007) [2023-12-27 18:52:07,743][20381] Updated weights for policy 1, policy_version 1906732 (0.0005) [2023-12-27 18:52:07,795][20381] Updated weights for policy 1, policy_version 1906742 (0.0005) [2023-12-27 18:52:07,854][20381] Updated weights for policy 1, policy_version 1906752 (0.0005) [2023-12-27 18:52:08,065][20347] Updated weights for policy 0, policy_version 1908353 (0.0010) [2023-12-27 18:52:08,113][20347] Updated weights for policy 0, policy_version 1908363 (0.0010) [2023-12-27 18:52:08,165][20347] Updated weights for policy 0, policy_version 1908373 (0.0010) [2023-12-27 18:52:08,226][20347] Updated weights for policy 0, policy_version 1908383 (0.0007) [2023-12-27 18:52:08,435][20381] Updated weights for policy 1, policy_version 1906762 (0.0007) [2023-12-27 18:52:08,491][20381] Updated weights for policy 1, policy_version 1906772 (0.0008) [2023-12-27 18:52:08,547][20381] Updated weights for policy 1, policy_version 1906782 (0.0007) [2023-12-27 18:52:08,924][20347] Updated weights for policy 0, policy_version 1908393 (0.0006) [2023-12-27 18:52:08,983][20347] Updated weights for policy 0, policy_version 1908403 (0.0007) [2023-12-27 18:52:09,040][20347] Updated weights for policy 0, policy_version 1908413 (0.0006) [2023-12-27 18:52:09,193][20381] Updated weights for policy 1, policy_version 1906792 (0.0006) [2023-12-27 18:52:09,261][20381] Updated weights for policy 1, policy_version 1906802 (0.0008) [2023-12-27 18:52:09,323][20381] Updated weights for policy 1, policy_version 1906812 (0.0008) [2023-12-27 18:52:09,744][20347] Updated weights for policy 0, policy_version 1908423 (0.0006) [2023-12-27 18:52:09,792][20347] Updated weights for policy 0, policy_version 1908433 (0.0007) [2023-12-27 18:52:09,855][20347] Updated weights for policy 0, policy_version 1908443 (0.0009) [2023-12-27 18:52:10,068][20381] Updated weights for policy 1, policy_version 1906822 (0.0009) [2023-12-27 18:52:10,122][20381] Updated weights for policy 1, policy_version 1906832 (0.0006) [2023-12-27 18:52:10,181][20381] Updated weights for policy 1, policy_version 1906842 (0.0007) [2023-12-27 18:52:10,602][20347] Updated weights for policy 0, policy_version 1908453 (0.0007) [2023-12-27 18:52:10,664][20347] Updated weights for policy 0, policy_version 1908463 (0.0007) [2023-12-27 18:52:10,722][20347] Updated weights for policy 0, policy_version 1908473 (0.0008) [2023-12-27 18:52:10,876][20381] Updated weights for policy 1, policy_version 1906852 (0.0008) [2023-12-27 18:52:10,931][20381] Updated weights for policy 1, policy_version 1906862 (0.0007) [2023-12-27 18:52:10,990][20381] Updated weights for policy 1, policy_version 1906872 (0.0006) [2023-12-27 18:52:11,476][20347] Updated weights for policy 0, policy_version 1908483 (0.0007) [2023-12-27 18:52:11,533][20347] Updated weights for policy 0, policy_version 1908493 (0.0007) [2023-12-27 18:52:11,597][20347] Updated weights for policy 0, policy_version 1908503 (0.0007) [2023-12-27 18:52:11,665][20381] Updated weights for policy 1, policy_version 1906882 (0.0007) [2023-12-27 18:52:11,723][20381] Updated weights for policy 1, policy_version 1906892 (0.0008) [2023-12-27 18:52:11,787][20381] Updated weights for policy 1, policy_version 1906902 (0.0008) [2023-12-27 18:52:11,846][20381] Updated weights for policy 1, policy_version 1906912 (0.0008) [2023-12-27 18:52:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 976896000. Throughput: 0: 10328.2, 1: 10319.1. Samples: 976891748. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.690')] [2023-12-27 18:52:12,401][20347] Updated weights for policy 0, policy_version 1908513 (0.0009) [2023-12-27 18:52:12,468][20347] Updated weights for policy 0, policy_version 1908523 (0.0006) [2023-12-27 18:52:12,502][20381] Updated weights for policy 1, policy_version 1906922 (0.0008) [2023-12-27 18:52:12,535][20347] Updated weights for policy 0, policy_version 1908533 (0.0007) [2023-12-27 18:52:12,553][20381] Updated weights for policy 1, policy_version 1906932 (0.0007) [2023-12-27 18:52:12,598][20347] Updated weights for policy 0, policy_version 1908543 (0.0006) [2023-12-27 18:52:12,602][20381] Updated weights for policy 1, policy_version 1906942 (0.0007) [2023-12-27 18:52:13,215][20381] Updated weights for policy 1, policy_version 1906952 (0.0008) [2023-12-27 18:52:13,271][20381] Updated weights for policy 1, policy_version 1906962 (0.0010) [2023-12-27 18:52:13,292][20347] Updated weights for policy 0, policy_version 1908553 (0.0006) [2023-12-27 18:52:13,317][20381] Updated weights for policy 1, policy_version 1906972 (0.0010) [2023-12-27 18:52:13,350][20347] Updated weights for policy 0, policy_version 1908563 (0.0006) [2023-12-27 18:52:13,398][20347] Updated weights for policy 0, policy_version 1908573 (0.0007) [2023-12-27 18:52:13,899][20381] Updated weights for policy 1, policy_version 1906982 (0.0010) [2023-12-27 18:52:13,950][20381] Updated weights for policy 1, policy_version 1906992 (0.0010) [2023-12-27 18:52:14,001][20381] Updated weights for policy 1, policy_version 1907002 (0.0010) [2023-12-27 18:52:14,133][20347] Updated weights for policy 0, policy_version 1908583 (0.0008) [2023-12-27 18:52:14,180][20347] Updated weights for policy 0, policy_version 1908593 (0.0008) [2023-12-27 18:52:14,230][20347] Updated weights for policy 0, policy_version 1908603 (0.0007) [2023-12-27 18:52:14,595][20381] Updated weights for policy 1, policy_version 1907012 (0.0010) [2023-12-27 18:52:14,650][20381] Updated weights for policy 1, policy_version 1907022 (0.0005) [2023-12-27 18:52:14,706][20381] Updated weights for policy 1, policy_version 1907032 (0.0005) [2023-12-27 18:52:15,002][20347] Updated weights for policy 0, policy_version 1908613 (0.0007) [2023-12-27 18:52:15,061][20347] Updated weights for policy 0, policy_version 1908623 (0.0006) [2023-12-27 18:52:15,116][20347] Updated weights for policy 0, policy_version 1908633 (0.0005) [2023-12-27 18:52:15,412][20381] Updated weights for policy 1, policy_version 1907042 (0.0008) [2023-12-27 18:52:15,466][20381] Updated weights for policy 1, policy_version 1907052 (0.0007) [2023-12-27 18:52:15,514][20381] Updated weights for policy 1, policy_version 1907062 (0.0005) [2023-12-27 18:52:15,565][20381] Updated weights for policy 1, policy_version 1907072 (0.0007) [2023-12-27 18:52:15,751][20347] Updated weights for policy 0, policy_version 1908643 (0.0008) [2023-12-27 18:52:15,796][20347] Updated weights for policy 0, policy_version 1908653 (0.0009) [2023-12-27 18:52:15,857][20347] Updated weights for policy 0, policy_version 1908663 (0.0009) [2023-12-27 18:52:16,217][20381] Updated weights for policy 1, policy_version 1907082 (0.0006) [2023-12-27 18:52:16,269][20381] Updated weights for policy 1, policy_version 1907092 (0.0008) [2023-12-27 18:52:16,324][20381] Updated weights for policy 1, policy_version 1907102 (0.0011) [2023-12-27 18:52:16,594][20347] Updated weights for policy 0, policy_version 1908673 (0.0009) [2023-12-27 18:52:16,650][20347] Updated weights for policy 0, policy_version 1908683 (0.0008) [2023-12-27 18:52:16,708][20347] Updated weights for policy 0, policy_version 1908693 (0.0007) [2023-12-27 18:52:16,764][20347] Updated weights for policy 0, policy_version 1908703 (0.0009) [2023-12-27 18:52:16,927][20381] Updated weights for policy 1, policy_version 1907112 (0.0010) [2023-12-27 18:52:16,971][20381] Updated weights for policy 1, policy_version 1907122 (0.0007) [2023-12-27 18:52:17,022][20381] Updated weights for policy 1, policy_version 1907132 (0.0007) [2023-12-27 18:52:17,362][19170] Fps is (10 sec: 20480.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 977002496. Throughput: 0: 10212.7, 1: 10371.2. Samples: 977014788. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.050')] [2023-12-27 18:52:17,457][20347] Updated weights for policy 0, policy_version 1908713 (0.0008) [2023-12-27 18:52:17,503][20347] Updated weights for policy 0, policy_version 1908723 (0.0010) [2023-12-27 18:52:17,551][20347] Updated weights for policy 0, policy_version 1908733 (0.0010) [2023-12-27 18:52:17,724][20381] Updated weights for policy 1, policy_version 1907142 (0.0007) [2023-12-27 18:52:17,774][20381] Updated weights for policy 1, policy_version 1907152 (0.0007) [2023-12-27 18:52:17,821][20381] Updated weights for policy 1, policy_version 1907162 (0.0007) [2023-12-27 18:52:18,269][20347] Updated weights for policy 0, policy_version 1908743 (0.0010) [2023-12-27 18:52:18,326][20347] Updated weights for policy 0, policy_version 1908753 (0.0009) [2023-12-27 18:52:18,390][20347] Updated weights for policy 0, policy_version 1908763 (0.0008) [2023-12-27 18:52:18,532][20381] Updated weights for policy 1, policy_version 1907172 (0.0008) [2023-12-27 18:52:18,589][20381] Updated weights for policy 1, policy_version 1907182 (0.0007) [2023-12-27 18:52:18,648][20381] Updated weights for policy 1, policy_version 1907192 (0.0007) [2023-12-27 18:52:19,108][20347] Updated weights for policy 0, policy_version 1908773 (0.0008) [2023-12-27 18:52:19,157][20347] Updated weights for policy 0, policy_version 1908783 (0.0008) [2023-12-27 18:52:19,213][20347] Updated weights for policy 0, policy_version 1908793 (0.0007) [2023-12-27 18:52:19,342][20381] Updated weights for policy 1, policy_version 1907202 (0.0008) [2023-12-27 18:52:19,409][20381] Updated weights for policy 1, policy_version 1907212 (0.0007) [2023-12-27 18:52:19,455][20381] Updated weights for policy 1, policy_version 1907222 (0.0011) [2023-12-27 18:52:19,517][20381] Updated weights for policy 1, policy_version 1907232 (0.0009) [2023-12-27 18:52:19,912][20347] Updated weights for policy 0, policy_version 1908803 (0.0007) [2023-12-27 18:52:19,978][20347] Updated weights for policy 0, policy_version 1908813 (0.0008) [2023-12-27 18:52:20,035][20347] Updated weights for policy 0, policy_version 1908823 (0.0008) [2023-12-27 18:52:20,249][20381] Updated weights for policy 1, policy_version 1907242 (0.0010) [2023-12-27 18:52:20,310][20381] Updated weights for policy 1, policy_version 1907252 (0.0006) [2023-12-27 18:52:20,373][20381] Updated weights for policy 1, policy_version 1907262 (0.0007) [2023-12-27 18:52:20,747][20347] Updated weights for policy 0, policy_version 1908833 (0.0009) [2023-12-27 18:52:20,814][20347] Updated weights for policy 0, policy_version 1908843 (0.0008) [2023-12-27 18:52:20,880][20347] Updated weights for policy 0, policy_version 1908853 (0.0007) [2023-12-27 18:52:20,944][20347] Updated weights for policy 0, policy_version 1908863 (0.0007) [2023-12-27 18:52:21,002][20381] Updated weights for policy 1, policy_version 1907272 (0.0006) [2023-12-27 18:52:21,067][20381] Updated weights for policy 1, policy_version 1907282 (0.0010) [2023-12-27 18:52:21,132][20381] Updated weights for policy 1, policy_version 1907292 (0.0008) [2023-12-27 18:52:21,664][20347] Updated weights for policy 0, policy_version 1908873 (0.0008) [2023-12-27 18:52:21,726][20347] Updated weights for policy 0, policy_version 1908883 (0.0007) [2023-12-27 18:52:21,787][20347] Updated weights for policy 0, policy_version 1908893 (0.0008) [2023-12-27 18:52:21,853][20381] Updated weights for policy 1, policy_version 1907302 (0.0008) [2023-12-27 18:52:21,913][20381] Updated weights for policy 1, policy_version 1907312 (0.0010) [2023-12-27 18:52:21,969][20381] Updated weights for policy 1, policy_version 1907322 (0.0007) [2023-12-27 18:52:22,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 977100800. Throughput: 0: 10136.7, 1: 10336.9. Samples: 977074288. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.900')] [2023-12-27 18:52:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001907328_488333312.pth... [2023-12-27 18:52:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001908896_488767488.pth... [2023-12-27 18:52:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001906112_488022016.pth [2023-12-27 18:52:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001907744_488472576.pth [2023-12-27 18:52:22,495][20347] Updated weights for policy 0, policy_version 1908903 (0.0007) [2023-12-27 18:52:22,549][20347] Updated weights for policy 0, policy_version 1908913 (0.0008) [2023-12-27 18:52:22,611][20347] Updated weights for policy 0, policy_version 1908923 (0.0008) [2023-12-27 18:52:22,613][20381] Updated weights for policy 1, policy_version 1907332 (0.0006) [2023-12-27 18:52:22,671][20381] Updated weights for policy 1, policy_version 1907342 (0.0007) [2023-12-27 18:52:22,731][20381] Updated weights for policy 1, policy_version 1907352 (0.0006) [2023-12-27 18:52:23,326][20381] Updated weights for policy 1, policy_version 1907362 (0.0006) [2023-12-27 18:52:23,342][20347] Updated weights for policy 0, policy_version 1908933 (0.0007) [2023-12-27 18:52:23,386][20381] Updated weights for policy 1, policy_version 1907372 (0.0006) [2023-12-27 18:52:23,396][20347] Updated weights for policy 0, policy_version 1908943 (0.0005) [2023-12-27 18:52:23,442][20381] Updated weights for policy 1, policy_version 1907382 (0.0005) [2023-12-27 18:52:23,452][20347] Updated weights for policy 0, policy_version 1908953 (0.0006) [2023-12-27 18:52:23,495][20381] Updated weights for policy 1, policy_version 1907392 (0.0006) [2023-12-27 18:52:24,136][20381] Updated weights for policy 1, policy_version 1907402 (0.0007) [2023-12-27 18:52:24,166][20347] Updated weights for policy 0, policy_version 1908963 (0.0006) [2023-12-27 18:52:24,186][20381] Updated weights for policy 1, policy_version 1907412 (0.0008) [2023-12-27 18:52:24,218][20347] Updated weights for policy 0, policy_version 1908973 (0.0006) [2023-12-27 18:52:24,241][20381] Updated weights for policy 1, policy_version 1907422 (0.0005) [2023-12-27 18:52:24,264][20347] Updated weights for policy 0, policy_version 1908983 (0.0007) [2023-12-27 18:52:24,865][20381] Updated weights for policy 1, policy_version 1907432 (0.0008) [2023-12-27 18:52:24,917][20381] Updated weights for policy 1, policy_version 1907442 (0.0008) [2023-12-27 18:52:24,971][20381] Updated weights for policy 1, policy_version 1907452 (0.0008) [2023-12-27 18:52:25,007][20347] Updated weights for policy 0, policy_version 1908993 (0.0007) [2023-12-27 18:52:25,068][20347] Updated weights for policy 0, policy_version 1909003 (0.0006) [2023-12-27 18:52:25,122][20347] Updated weights for policy 0, policy_version 1909013 (0.0008) [2023-12-27 18:52:25,176][20347] Updated weights for policy 0, policy_version 1909023 (0.0008) [2023-12-27 18:52:25,675][20381] Updated weights for policy 1, policy_version 1907462 (0.0007) [2023-12-27 18:52:25,735][20381] Updated weights for policy 1, policy_version 1907472 (0.0007) [2023-12-27 18:52:25,796][20381] Updated weights for policy 1, policy_version 1907482 (0.0009) [2023-12-27 18:52:25,844][20347] Updated weights for policy 0, policy_version 1909033 (0.0005) [2023-12-27 18:52:25,905][20347] Updated weights for policy 0, policy_version 1909043 (0.0005) [2023-12-27 18:52:25,967][20347] Updated weights for policy 0, policy_version 1909053 (0.0005) [2023-12-27 18:52:26,450][20381] Updated weights for policy 1, policy_version 1907492 (0.0008) [2023-12-27 18:52:26,485][20347] Updated weights for policy 0, policy_version 1909063 (0.0005) [2023-12-27 18:52:26,504][20381] Updated weights for policy 1, policy_version 1907502 (0.0007) [2023-12-27 18:52:26,539][20347] Updated weights for policy 0, policy_version 1909073 (0.0009) [2023-12-27 18:52:26,562][20381] Updated weights for policy 1, policy_version 1907512 (0.0005) [2023-12-27 18:52:26,587][20347] Updated weights for policy 0, policy_version 1909083 (0.0006) [2023-12-27 18:52:27,114][20347] Updated weights for policy 0, policy_version 1909093 (0.0005) [2023-12-27 18:52:27,170][20347] Updated weights for policy 0, policy_version 1909103 (0.0005) [2023-12-27 18:52:27,233][20347] Updated weights for policy 0, policy_version 1909113 (0.0007) [2023-12-27 18:52:27,241][20381] Updated weights for policy 1, policy_version 1907522 (0.0005) [2023-12-27 18:52:27,297][20381] Updated weights for policy 1, policy_version 1907532 (0.0006) [2023-12-27 18:52:27,349][20381] Updated weights for policy 1, policy_version 1907542 (0.0006) [2023-12-27 18:52:27,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 977207296. Throughput: 0: 10050.9, 1: 10430.3. Samples: 977195680. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.380')] [2023-12-27 18:52:27,399][20381] Updated weights for policy 1, policy_version 1907552 (0.0005) [2023-12-27 18:52:27,850][20347] Updated weights for policy 0, policy_version 1909123 (0.0010) [2023-12-27 18:52:27,908][20347] Updated weights for policy 0, policy_version 1909133 (0.0005) [2023-12-27 18:52:27,960][20347] Updated weights for policy 0, policy_version 1909143 (0.0005) [2023-12-27 18:52:28,008][20381] Updated weights for policy 1, policy_version 1907562 (0.0009) [2023-12-27 18:52:28,062][20381] Updated weights for policy 1, policy_version 1907572 (0.0005) [2023-12-27 18:52:28,113][20381] Updated weights for policy 1, policy_version 1907582 (0.0005) [2023-12-27 18:52:28,574][20347] Updated weights for policy 0, policy_version 1909153 (0.0007) [2023-12-27 18:52:28,635][20347] Updated weights for policy 0, policy_version 1909163 (0.0010) [2023-12-27 18:52:28,695][20347] Updated weights for policy 0, policy_version 1909173 (0.0010) [2023-12-27 18:52:28,756][20347] Updated weights for policy 0, policy_version 1909183 (0.0010) [2023-12-27 18:52:28,760][20381] Updated weights for policy 1, policy_version 1907592 (0.0005) [2023-12-27 18:52:28,818][20381] Updated weights for policy 1, policy_version 1907602 (0.0007) [2023-12-27 18:52:28,869][20381] Updated weights for policy 1, policy_version 1907612 (0.0007) [2023-12-27 18:52:29,449][20347] Updated weights for policy 0, policy_version 1909193 (0.0010) [2023-12-27 18:52:29,505][20347] Updated weights for policy 0, policy_version 1909203 (0.0010) [2023-12-27 18:52:29,551][20381] Updated weights for policy 1, policy_version 1907622 (0.0009) [2023-12-27 18:52:29,562][20347] Updated weights for policy 0, policy_version 1909213 (0.0010) [2023-12-27 18:52:29,610][20381] Updated weights for policy 1, policy_version 1907632 (0.0010) [2023-12-27 18:52:29,674][20381] Updated weights for policy 1, policy_version 1907642 (0.0010) [2023-12-27 18:52:30,244][20347] Updated weights for policy 0, policy_version 1909223 (0.0010) [2023-12-27 18:52:30,302][20347] Updated weights for policy 0, policy_version 1909233 (0.0010) [2023-12-27 18:52:30,353][20381] Updated weights for policy 1, policy_version 1907652 (0.0010) [2023-12-27 18:52:30,364][20347] Updated weights for policy 0, policy_version 1909243 (0.0010) [2023-12-27 18:52:30,399][20381] Updated weights for policy 1, policy_version 1907662 (0.0006) [2023-12-27 18:52:30,440][20381] Updated weights for policy 1, policy_version 1907672 (0.0007) [2023-12-27 18:52:31,018][20347] Updated weights for policy 0, policy_version 1909253 (0.0009) [2023-12-27 18:52:31,074][20347] Updated weights for policy 0, policy_version 1909263 (0.0007) [2023-12-27 18:52:31,138][20347] Updated weights for policy 0, policy_version 1909273 (0.0007) [2023-12-27 18:52:31,212][20381] Updated weights for policy 1, policy_version 1907682 (0.0007) [2023-12-27 18:52:31,277][20381] Updated weights for policy 1, policy_version 1907692 (0.0009) [2023-12-27 18:52:31,340][20381] Updated weights for policy 1, policy_version 1907702 (0.0008) [2023-12-27 18:52:31,411][20381] Updated weights for policy 1, policy_version 1907712 (0.0008) [2023-12-27 18:52:31,830][20347] Updated weights for policy 0, policy_version 1909283 (0.0006) [2023-12-27 18:52:31,887][20347] Updated weights for policy 0, policy_version 1909293 (0.0006) [2023-12-27 18:52:31,947][20347] Updated weights for policy 0, policy_version 1909303 (0.0006) [2023-12-27 18:52:32,139][20381] Updated weights for policy 1, policy_version 1907722 (0.0008) [2023-12-27 18:52:32,200][20381] Updated weights for policy 1, policy_version 1907732 (0.0010) [2023-12-27 18:52:32,258][20381] Updated weights for policy 1, policy_version 1907742 (0.0010) [2023-12-27 18:52:32,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 977313792. Throughput: 0: 10128.2, 1: 10395.3. Samples: 977321592. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.870')] [2023-12-27 18:52:32,621][20347] Updated weights for policy 0, policy_version 1909313 (0.0007) [2023-12-27 18:52:32,674][20347] Updated weights for policy 0, policy_version 1909323 (0.0007) [2023-12-27 18:52:32,731][20347] Updated weights for policy 0, policy_version 1909333 (0.0007) [2023-12-27 18:52:32,791][20347] Updated weights for policy 0, policy_version 1909343 (0.0008) [2023-12-27 18:52:32,908][20381] Updated weights for policy 1, policy_version 1907752 (0.0006) [2023-12-27 18:52:32,968][20381] Updated weights for policy 1, policy_version 1907762 (0.0005) [2023-12-27 18:52:33,023][20381] Updated weights for policy 1, policy_version 1907772 (0.0005) [2023-12-27 18:52:33,498][20347] Updated weights for policy 0, policy_version 1909353 (0.0005) [2023-12-27 18:52:33,537][20381] Updated weights for policy 1, policy_version 1907782 (0.0008) [2023-12-27 18:52:33,557][20347] Updated weights for policy 0, policy_version 1909363 (0.0010) [2023-12-27 18:52:33,582][20381] Updated weights for policy 1, policy_version 1907792 (0.0010) [2023-12-27 18:52:33,611][20347] Updated weights for policy 0, policy_version 1909373 (0.0010) [2023-12-27 18:52:33,627][20381] Updated weights for policy 1, policy_version 1907802 (0.0010) [2023-12-27 18:52:34,174][20347] Updated weights for policy 0, policy_version 1909383 (0.0008) [2023-12-27 18:52:34,234][20347] Updated weights for policy 0, policy_version 1909393 (0.0010) [2023-12-27 18:52:34,240][20381] Updated weights for policy 1, policy_version 1907812 (0.0010) [2023-12-27 18:52:34,294][20347] Updated weights for policy 0, policy_version 1909403 (0.0011) [2023-12-27 18:52:34,300][20381] Updated weights for policy 1, policy_version 1907822 (0.0009) [2023-12-27 18:52:34,360][20381] Updated weights for policy 1, policy_version 1907832 (0.0010) [2023-12-27 18:52:34,989][20347] Updated weights for policy 0, policy_version 1909413 (0.0008) [2023-12-27 18:52:35,005][20381] Updated weights for policy 1, policy_version 1907842 (0.0009) [2023-12-27 18:52:35,049][20347] Updated weights for policy 0, policy_version 1909423 (0.0008) [2023-12-27 18:52:35,058][20381] Updated weights for policy 1, policy_version 1907852 (0.0006) [2023-12-27 18:52:35,107][20381] Updated weights for policy 1, policy_version 1907862 (0.0006) [2023-12-27 18:52:35,114][20347] Updated weights for policy 0, policy_version 1909433 (0.0007) [2023-12-27 18:52:35,162][20381] Updated weights for policy 1, policy_version 1907872 (0.0010) [2023-12-27 18:52:35,762][20347] Updated weights for policy 0, policy_version 1909443 (0.0009) [2023-12-27 18:52:35,769][20381] Updated weights for policy 1, policy_version 1907882 (0.0011) [2023-12-27 18:52:35,816][20347] Updated weights for policy 0, policy_version 1909453 (0.0011) [2023-12-27 18:52:35,826][20381] Updated weights for policy 1, policy_version 1907892 (0.0011) [2023-12-27 18:52:35,862][20347] Updated weights for policy 0, policy_version 1909463 (0.0011) [2023-12-27 18:52:35,879][20381] Updated weights for policy 1, policy_version 1907902 (0.0010) [2023-12-27 18:52:36,510][20381] Updated weights for policy 1, policy_version 1907912 (0.0010) [2023-12-27 18:52:36,574][20381] Updated weights for policy 1, policy_version 1907922 (0.0011) [2023-12-27 18:52:36,599][20347] Updated weights for policy 0, policy_version 1909473 (0.0010) [2023-12-27 18:52:36,644][20381] Updated weights for policy 1, policy_version 1907932 (0.0011) [2023-12-27 18:52:36,657][20347] Updated weights for policy 0, policy_version 1909483 (0.0009) [2023-12-27 18:52:36,716][20347] Updated weights for policy 0, policy_version 1909493 (0.0008) [2023-12-27 18:52:36,776][20347] Updated weights for policy 0, policy_version 1909503 (0.0008) [2023-12-27 18:52:37,290][20381] Updated weights for policy 1, policy_version 1907942 (0.0007) [2023-12-27 18:52:37,345][20381] Updated weights for policy 1, policy_version 1907952 (0.0006) [2023-12-27 18:52:37,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 977412096. Throughput: 0: 10151.4, 1: 10411.6. Samples: 977386624. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.690')] [2023-12-27 18:52:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001909504_488923136.pth... [2023-12-27 18:52:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001908352_488628224.pth [2023-12-27 18:52:37,403][20381] Updated weights for policy 1, policy_version 1907962 (0.0007) [2023-12-27 18:52:37,434][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001907968_488497152.pth... [2023-12-27 18:52:37,437][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001906720_488177664.pth [2023-12-27 18:52:37,494][20347] Updated weights for policy 0, policy_version 1909513 (0.0008) [2023-12-27 18:52:37,549][20347] Updated weights for policy 0, policy_version 1909523 (0.0007) [2023-12-27 18:52:37,611][20347] Updated weights for policy 0, policy_version 1909533 (0.0007) [2023-12-27 18:52:38,032][20381] Updated weights for policy 1, policy_version 1907972 (0.0008) [2023-12-27 18:52:38,089][20381] Updated weights for policy 1, policy_version 1907982 (0.0010) [2023-12-27 18:52:38,154][20381] Updated weights for policy 1, policy_version 1907992 (0.0011) [2023-12-27 18:52:38,290][20347] Updated weights for policy 0, policy_version 1909543 (0.0007) [2023-12-27 18:52:38,353][20347] Updated weights for policy 0, policy_version 1909553 (0.0008) [2023-12-27 18:52:38,418][20347] Updated weights for policy 0, policy_version 1909563 (0.0006) [2023-12-27 18:52:38,774][20381] Updated weights for policy 1, policy_version 1908002 (0.0009) [2023-12-27 18:52:38,838][20381] Updated weights for policy 1, policy_version 1908012 (0.0006) [2023-12-27 18:52:38,903][20381] Updated weights for policy 1, policy_version 1908022 (0.0008) [2023-12-27 18:52:38,965][20381] Updated weights for policy 1, policy_version 1908032 (0.0007) [2023-12-27 18:52:39,052][20347] Updated weights for policy 0, policy_version 1909573 (0.0007) [2023-12-27 18:52:39,102][20347] Updated weights for policy 0, policy_version 1909583 (0.0009) [2023-12-27 18:52:39,153][20347] Updated weights for policy 0, policy_version 1909593 (0.0007) [2023-12-27 18:52:39,630][20381] Updated weights for policy 1, policy_version 1908042 (0.0007) [2023-12-27 18:52:39,693][20381] Updated weights for policy 1, policy_version 1908052 (0.0006) [2023-12-27 18:52:39,756][20381] Updated weights for policy 1, policy_version 1908062 (0.0010) [2023-12-27 18:52:39,955][20347] Updated weights for policy 0, policy_version 1909603 (0.0008) [2023-12-27 18:52:40,016][20347] Updated weights for policy 0, policy_version 1909613 (0.0008) [2023-12-27 18:52:40,073][20347] Updated weights for policy 0, policy_version 1909623 (0.0010) [2023-12-27 18:52:40,363][20381] Updated weights for policy 1, policy_version 1908072 (0.0007) [2023-12-27 18:52:40,424][20381] Updated weights for policy 1, policy_version 1908082 (0.0006) [2023-12-27 18:52:40,485][20381] Updated weights for policy 1, policy_version 1908092 (0.0006) [2023-12-27 18:52:40,733][20347] Updated weights for policy 0, policy_version 1909633 (0.0009) [2023-12-27 18:52:40,801][20347] Updated weights for policy 0, policy_version 1909643 (0.0007) [2023-12-27 18:52:40,859][20347] Updated weights for policy 0, policy_version 1909653 (0.0008) [2023-12-27 18:52:40,913][20347] Updated weights for policy 0, policy_version 1909663 (0.0008) [2023-12-27 18:52:41,035][20381] Updated weights for policy 1, policy_version 1908102 (0.0007) [2023-12-27 18:52:41,102][20381] Updated weights for policy 1, policy_version 1908112 (0.0007) [2023-12-27 18:52:41,175][20381] Updated weights for policy 1, policy_version 1908122 (0.0008) [2023-12-27 18:52:41,675][20347] Updated weights for policy 0, policy_version 1909673 (0.0008) [2023-12-27 18:52:41,736][20347] Updated weights for policy 0, policy_version 1909683 (0.0007) [2023-12-27 18:52:41,800][20347] Updated weights for policy 0, policy_version 1909693 (0.0007) [2023-12-27 18:52:41,985][20381] Updated weights for policy 1, policy_version 1908132 (0.0007) [2023-12-27 18:52:42,051][20381] Updated weights for policy 1, policy_version 1908142 (0.0006) [2023-12-27 18:52:42,116][20381] Updated weights for policy 1, policy_version 1908152 (0.0005) [2023-12-27 18:52:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 977518592. Throughput: 0: 10079.3, 1: 10466.9. Samples: 977510232. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.700')] [2023-12-27 18:52:42,487][20347] Updated weights for policy 0, policy_version 1909703 (0.0007) [2023-12-27 18:52:42,550][20347] Updated weights for policy 0, policy_version 1909713 (0.0007) [2023-12-27 18:52:42,614][20347] Updated weights for policy 0, policy_version 1909723 (0.0009) [2023-12-27 18:52:42,743][20381] Updated weights for policy 1, policy_version 1908162 (0.0007) [2023-12-27 18:52:42,804][20381] Updated weights for policy 1, policy_version 1908172 (0.0006) [2023-12-27 18:52:42,866][20381] Updated weights for policy 1, policy_version 1908182 (0.0009) [2023-12-27 18:52:42,924][20381] Updated weights for policy 1, policy_version 1908192 (0.0011) [2023-12-27 18:52:43,269][20347] Updated weights for policy 0, policy_version 1909733 (0.0010) [2023-12-27 18:52:43,326][20347] Updated weights for policy 0, policy_version 1909743 (0.0010) [2023-12-27 18:52:43,384][20347] Updated weights for policy 0, policy_version 1909753 (0.0008) [2023-12-27 18:52:43,503][20381] Updated weights for policy 1, policy_version 1908202 (0.0011) [2023-12-27 18:52:43,554][20381] Updated weights for policy 1, policy_version 1908212 (0.0010) [2023-12-27 18:52:43,612][20381] Updated weights for policy 1, policy_version 1908222 (0.0010) [2023-12-27 18:52:43,956][20347] Updated weights for policy 0, policy_version 1909763 (0.0005) [2023-12-27 18:52:44,004][20347] Updated weights for policy 0, policy_version 1909773 (0.0008) [2023-12-27 18:52:44,052][20347] Updated weights for policy 0, policy_version 1909783 (0.0010) [2023-12-27 18:52:44,307][20381] Updated weights for policy 1, policy_version 1908232 (0.0010) [2023-12-27 18:52:44,373][20381] Updated weights for policy 1, policy_version 1908242 (0.0011) [2023-12-27 18:52:44,424][20381] Updated weights for policy 1, policy_version 1908252 (0.0010) [2023-12-27 18:52:44,693][20347] Updated weights for policy 0, policy_version 1909793 (0.0006) [2023-12-27 18:52:44,755][20347] Updated weights for policy 0, policy_version 1909803 (0.0010) [2023-12-27 18:52:44,816][20347] Updated weights for policy 0, policy_version 1909813 (0.0009) [2023-12-27 18:52:44,882][20347] Updated weights for policy 0, policy_version 1909823 (0.0011) [2023-12-27 18:52:45,078][20381] Updated weights for policy 1, policy_version 1908262 (0.0007) [2023-12-27 18:52:45,135][20381] Updated weights for policy 1, policy_version 1908272 (0.0005) [2023-12-27 18:52:45,198][20381] Updated weights for policy 1, policy_version 1908282 (0.0009) [2023-12-27 18:52:45,461][20347] Updated weights for policy 0, policy_version 1909833 (0.0005) [2023-12-27 18:52:45,515][20347] Updated weights for policy 0, policy_version 1909843 (0.0005) [2023-12-27 18:52:45,571][20347] Updated weights for policy 0, policy_version 1909853 (0.0006) [2023-12-27 18:52:45,794][20381] Updated weights for policy 1, policy_version 1908292 (0.0010) [2023-12-27 18:52:45,852][20381] Updated weights for policy 1, policy_version 1908302 (0.0010) [2023-12-27 18:52:45,910][20381] Updated weights for policy 1, policy_version 1908312 (0.0010) [2023-12-27 18:52:46,141][20347] Updated weights for policy 0, policy_version 1909863 (0.0010) [2023-12-27 18:52:46,201][20347] Updated weights for policy 0, policy_version 1909873 (0.0009) [2023-12-27 18:52:46,258][20347] Updated weights for policy 0, policy_version 1909883 (0.0006) [2023-12-27 18:52:46,553][20381] Updated weights for policy 1, policy_version 1908322 (0.0009) [2023-12-27 18:52:46,604][20381] Updated weights for policy 1, policy_version 1908332 (0.0005) [2023-12-27 18:52:46,665][20381] Updated weights for policy 1, policy_version 1908342 (0.0005) [2023-12-27 18:52:46,731][20381] Updated weights for policy 1, policy_version 1908352 (0.0007) [2023-12-27 18:52:46,838][20347] Updated weights for policy 0, policy_version 1909893 (0.0008) [2023-12-27 18:52:46,900][20347] Updated weights for policy 0, policy_version 1909903 (0.0006) [2023-12-27 18:52:46,950][20347] Updated weights for policy 0, policy_version 1909913 (0.0006) [2023-12-27 18:52:47,306][20381] Updated weights for policy 1, policy_version 1908362 (0.0007) [2023-12-27 18:52:47,361][20381] Updated weights for policy 1, policy_version 1908372 (0.0007) [2023-12-27 18:52:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 977625088. Throughput: 0: 10119.1, 1: 10555.1. Samples: 977639224. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.570')] [2023-12-27 18:52:47,419][20381] Updated weights for policy 1, policy_version 1908382 (0.0008) [2023-12-27 18:52:47,515][20347] Updated weights for policy 0, policy_version 1909923 (0.0006) [2023-12-27 18:52:47,569][20347] Updated weights for policy 0, policy_version 1909933 (0.0007) [2023-12-27 18:52:47,624][20347] Updated weights for policy 0, policy_version 1909943 (0.0007) [2023-12-27 18:52:48,106][20381] Updated weights for policy 1, policy_version 1908392 (0.0008) [2023-12-27 18:52:48,156][20381] Updated weights for policy 1, policy_version 1908402 (0.0007) [2023-12-27 18:52:48,217][20381] Updated weights for policy 1, policy_version 1908412 (0.0007) [2023-12-27 18:52:48,325][20347] Updated weights for policy 0, policy_version 1909953 (0.0008) [2023-12-27 18:52:48,385][20347] Updated weights for policy 0, policy_version 1909963 (0.0011) [2023-12-27 18:52:48,445][20347] Updated weights for policy 0, policy_version 1909973 (0.0011) [2023-12-27 18:52:48,507][20347] Updated weights for policy 0, policy_version 1909983 (0.0010) [2023-12-27 18:52:48,930][20381] Updated weights for policy 1, policy_version 1908422 (0.0007) [2023-12-27 18:52:48,977][20381] Updated weights for policy 1, policy_version 1908432 (0.0007) [2023-12-27 18:52:49,027][20381] Updated weights for policy 1, policy_version 1908442 (0.0007) [2023-12-27 18:52:49,195][20347] Updated weights for policy 0, policy_version 1909993 (0.0010) [2023-12-27 18:52:49,252][20347] Updated weights for policy 0, policy_version 1910003 (0.0009) [2023-12-27 18:52:49,310][20347] Updated weights for policy 0, policy_version 1910013 (0.0010) [2023-12-27 18:52:49,792][20381] Updated weights for policy 1, policy_version 1908452 (0.0007) [2023-12-27 18:52:49,854][20381] Updated weights for policy 1, policy_version 1908462 (0.0007) [2023-12-27 18:52:49,909][20381] Updated weights for policy 1, policy_version 1908472 (0.0006) [2023-12-27 18:52:50,047][20347] Updated weights for policy 0, policy_version 1910023 (0.0008) [2023-12-27 18:52:50,106][20347] Updated weights for policy 0, policy_version 1910033 (0.0007) [2023-12-27 18:52:50,165][20347] Updated weights for policy 0, policy_version 1910043 (0.0008) [2023-12-27 18:52:50,581][20381] Updated weights for policy 1, policy_version 1908482 (0.0007) [2023-12-27 18:52:50,643][20381] Updated weights for policy 1, policy_version 1908492 (0.0006) [2023-12-27 18:52:50,704][20381] Updated weights for policy 1, policy_version 1908502 (0.0007) [2023-12-27 18:52:50,758][20381] Updated weights for policy 1, policy_version 1908512 (0.0008) [2023-12-27 18:52:50,949][20347] Updated weights for policy 0, policy_version 1910053 (0.0008) [2023-12-27 18:52:51,019][20347] Updated weights for policy 0, policy_version 1910063 (0.0007) [2023-12-27 18:52:51,083][20347] Updated weights for policy 0, policy_version 1910073 (0.0008) [2023-12-27 18:52:51,433][20381] Updated weights for policy 1, policy_version 1908522 (0.0007) [2023-12-27 18:52:51,494][20381] Updated weights for policy 1, policy_version 1908532 (0.0006) [2023-12-27 18:52:51,555][20381] Updated weights for policy 1, policy_version 1908542 (0.0006) [2023-12-27 18:52:51,864][20347] Updated weights for policy 0, policy_version 1910083 (0.0007) [2023-12-27 18:52:51,919][20347] Updated weights for policy 0, policy_version 1910093 (0.0007) [2023-12-27 18:52:51,974][20347] Updated weights for policy 0, policy_version 1910103 (0.0007) [2023-12-27 18:52:52,223][20381] Updated weights for policy 1, policy_version 1908552 (0.0006) [2023-12-27 18:52:52,285][20381] Updated weights for policy 1, policy_version 1908562 (0.0006) [2023-12-27 18:52:52,351][20381] Updated weights for policy 1, policy_version 1908572 (0.0007) [2023-12-27 18:52:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 977723392. Throughput: 0: 10130.3, 1: 10491.9. Samples: 977699032. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.520')] [2023-12-27 18:52:52,375][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001910112_489078784.pth... [2023-12-27 18:52:52,377][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001908576_488652800.pth... [2023-12-27 18:52:52,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001908896_488767488.pth [2023-12-27 18:52:52,381][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001907328_488333312.pth [2023-12-27 18:52:52,666][20347] Updated weights for policy 0, policy_version 1910113 (0.0008) [2023-12-27 18:52:52,728][20347] Updated weights for policy 0, policy_version 1910123 (0.0010) [2023-12-27 18:52:52,779][20347] Updated weights for policy 0, policy_version 1910133 (0.0011) [2023-12-27 18:52:52,829][20347] Updated weights for policy 0, policy_version 1910143 (0.0008) [2023-12-27 18:52:53,019][20381] Updated weights for policy 1, policy_version 1908582 (0.0009) [2023-12-27 18:52:53,065][20381] Updated weights for policy 1, policy_version 1908592 (0.0010) [2023-12-27 18:52:53,117][20381] Updated weights for policy 1, policy_version 1908602 (0.0006) [2023-12-27 18:52:53,481][20347] Updated weights for policy 0, policy_version 1910153 (0.0010) [2023-12-27 18:52:53,534][20347] Updated weights for policy 0, policy_version 1910163 (0.0010) [2023-12-27 18:52:53,579][20347] Updated weights for policy 0, policy_version 1910173 (0.0010) [2023-12-27 18:52:53,665][20381] Updated weights for policy 1, policy_version 1908612 (0.0006) [2023-12-27 18:52:53,710][20381] Updated weights for policy 1, policy_version 1908622 (0.0011) [2023-12-27 18:52:53,759][20381] Updated weights for policy 1, policy_version 1908632 (0.0011) [2023-12-27 18:52:54,308][20347] Updated weights for policy 0, policy_version 1910183 (0.0012) [2023-12-27 18:52:54,375][20347] Updated weights for policy 0, policy_version 1910193 (0.0008) [2023-12-27 18:52:54,382][20381] Updated weights for policy 1, policy_version 1908642 (0.0010) [2023-12-27 18:52:54,432][20347] Updated weights for policy 0, policy_version 1910203 (0.0010) [2023-12-27 18:52:54,443][20381] Updated weights for policy 1, policy_version 1908652 (0.0010) [2023-12-27 18:52:54,501][20381] Updated weights for policy 1, policy_version 1908662 (0.0011) [2023-12-27 18:52:54,561][20381] Updated weights for policy 1, policy_version 1908672 (0.0011) [2023-12-27 18:52:55,096][20347] Updated weights for policy 0, policy_version 1910213 (0.0008) [2023-12-27 18:52:55,154][20347] Updated weights for policy 0, policy_version 1910223 (0.0007) [2023-12-27 18:52:55,202][20347] Updated weights for policy 0, policy_version 1910233 (0.0009) [2023-12-27 18:52:55,225][20381] Updated weights for policy 1, policy_version 1908682 (0.0008) [2023-12-27 18:52:55,293][20381] Updated weights for policy 1, policy_version 1908692 (0.0006) [2023-12-27 18:52:55,359][20381] Updated weights for policy 1, policy_version 1908702 (0.0009) [2023-12-27 18:52:55,806][20347] Updated weights for policy 0, policy_version 1910243 (0.0006) [2023-12-27 18:52:55,861][20347] Updated weights for policy 0, policy_version 1910253 (0.0007) [2023-12-27 18:52:55,917][20347] Updated weights for policy 0, policy_version 1910263 (0.0006) [2023-12-27 18:52:55,946][20381] Updated weights for policy 1, policy_version 1908712 (0.0005) [2023-12-27 18:52:56,000][20381] Updated weights for policy 1, policy_version 1908722 (0.0005) [2023-12-27 18:52:56,046][20381] Updated weights for policy 1, policy_version 1908732 (0.0006) [2023-12-27 18:52:56,555][20347] Updated weights for policy 0, policy_version 1910273 (0.0008) [2023-12-27 18:52:56,611][20347] Updated weights for policy 0, policy_version 1910283 (0.0010) [2023-12-27 18:52:56,665][20347] Updated weights for policy 0, policy_version 1910293 (0.0007) [2023-12-27 18:52:56,715][20381] Updated weights for policy 1, policy_version 1908742 (0.0010) [2023-12-27 18:52:56,718][20347] Updated weights for policy 0, policy_version 1910303 (0.0005) [2023-12-27 18:52:56,766][20381] Updated weights for policy 1, policy_version 1908752 (0.0010) [2023-12-27 18:52:56,817][20381] Updated weights for policy 1, policy_version 1908762 (0.0010) [2023-12-27 18:52:57,315][20347] Updated weights for policy 0, policy_version 1910313 (0.0010) [2023-12-27 18:52:57,362][20347] Updated weights for policy 0, policy_version 1910323 (0.0008) [2023-12-27 18:52:57,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 977829888. Throughput: 0: 10153.1, 1: 10558.3. Samples: 977823760. Policy #0 lag: (min: 19.0, avg: 31.5, max: 32.0) [2023-12-27 18:52:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.750')] [2023-12-27 18:52:57,409][20347] Updated weights for policy 0, policy_version 1910333 (0.0008) [2023-12-27 18:52:57,462][20381] Updated weights for policy 1, policy_version 1908772 (0.0008) [2023-12-27 18:52:57,519][20381] Updated weights for policy 1, policy_version 1908782 (0.0005) [2023-12-27 18:52:57,577][20381] Updated weights for policy 1, policy_version 1908792 (0.0005) [2023-12-27 18:52:58,057][20347] Updated weights for policy 0, policy_version 1910343 (0.0010) [2023-12-27 18:52:58,094][20381] Updated weights for policy 1, policy_version 1908802 (0.0005) [2023-12-27 18:52:58,105][20347] Updated weights for policy 0, policy_version 1910353 (0.0010) [2023-12-27 18:52:58,156][20347] Updated weights for policy 0, policy_version 1910363 (0.0010) [2023-12-27 18:52:58,157][20381] Updated weights for policy 1, policy_version 1908812 (0.0005) [2023-12-27 18:52:58,220][20381] Updated weights for policy 1, policy_version 1908822 (0.0007) [2023-12-27 18:52:58,283][20381] Updated weights for policy 1, policy_version 1908832 (0.0008) [2023-12-27 18:52:58,855][20347] Updated weights for policy 0, policy_version 1910373 (0.0008) [2023-12-27 18:52:58,911][20347] Updated weights for policy 0, policy_version 1910383 (0.0005) [2023-12-27 18:52:58,978][20347] Updated weights for policy 0, policy_version 1910393 (0.0008) [2023-12-27 18:52:59,057][20381] Updated weights for policy 1, policy_version 1908842 (0.0008) [2023-12-27 18:52:59,120][20381] Updated weights for policy 1, policy_version 1908852 (0.0007) [2023-12-27 18:52:59,179][20381] Updated weights for policy 1, policy_version 1908862 (0.0007) [2023-12-27 18:52:59,642][20347] Updated weights for policy 0, policy_version 1910403 (0.0008) [2023-12-27 18:52:59,690][20347] Updated weights for policy 0, policy_version 1910413 (0.0010) [2023-12-27 18:52:59,744][20347] Updated weights for policy 0, policy_version 1910423 (0.0010) [2023-12-27 18:52:59,915][20381] Updated weights for policy 1, policy_version 1908872 (0.0009) [2023-12-27 18:52:59,972][20381] Updated weights for policy 1, policy_version 1908882 (0.0008) [2023-12-27 18:53:00,028][20381] Updated weights for policy 1, policy_version 1908892 (0.0007) [2023-12-27 18:53:00,486][20347] Updated weights for policy 0, policy_version 1910433 (0.0010) [2023-12-27 18:53:00,544][20347] Updated weights for policy 0, policy_version 1910443 (0.0008) [2023-12-27 18:53:00,590][20347] Updated weights for policy 0, policy_version 1910453 (0.0010) [2023-12-27 18:53:00,643][20347] Updated weights for policy 0, policy_version 1910463 (0.0008) [2023-12-27 18:53:00,777][20381] Updated weights for policy 1, policy_version 1908902 (0.0007) [2023-12-27 18:53:00,831][20381] Updated weights for policy 1, policy_version 1908912 (0.0009) [2023-12-27 18:53:00,884][20381] Updated weights for policy 1, policy_version 1908922 (0.0007) [2023-12-27 18:53:01,315][20347] Updated weights for policy 0, policy_version 1910473 (0.0007) [2023-12-27 18:53:01,382][20347] Updated weights for policy 0, policy_version 1910483 (0.0009) [2023-12-27 18:53:01,440][20347] Updated weights for policy 0, policy_version 1910493 (0.0007) [2023-12-27 18:53:01,652][20381] Updated weights for policy 1, policy_version 1908932 (0.0008) [2023-12-27 18:53:01,716][20381] Updated weights for policy 1, policy_version 1908942 (0.0010) [2023-12-27 18:53:01,776][20381] Updated weights for policy 1, policy_version 1908952 (0.0008) [2023-12-27 18:53:02,128][20347] Updated weights for policy 0, policy_version 1910503 (0.0007) [2023-12-27 18:53:02,190][20347] Updated weights for policy 0, policy_version 1910513 (0.0008) [2023-12-27 18:53:02,244][20347] Updated weights for policy 0, policy_version 1910523 (0.0008) [2023-12-27 18:53:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20493.9). Total num frames: 977936384. Throughput: 0: 10249.3, 1: 10447.2. Samples: 977946132. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.490')] [2023-12-27 18:53:02,479][20381] Updated weights for policy 1, policy_version 1908962 (0.0007) [2023-12-27 18:53:02,538][20381] Updated weights for policy 1, policy_version 1908972 (0.0008) [2023-12-27 18:53:02,597][20381] Updated weights for policy 1, policy_version 1908982 (0.0008) [2023-12-27 18:53:02,653][20381] Updated weights for policy 1, policy_version 1908992 (0.0007) [2023-12-27 18:53:02,975][20347] Updated weights for policy 0, policy_version 1910533 (0.0007) [2023-12-27 18:53:03,023][20347] Updated weights for policy 0, policy_version 1910543 (0.0007) [2023-12-27 18:53:03,073][20347] Updated weights for policy 0, policy_version 1910553 (0.0007) [2023-12-27 18:53:03,351][20381] Updated weights for policy 1, policy_version 1909002 (0.0008) [2023-12-27 18:53:03,396][20381] Updated weights for policy 1, policy_version 1909012 (0.0009) [2023-12-27 18:53:03,443][20381] Updated weights for policy 1, policy_version 1909022 (0.0010) [2023-12-27 18:53:03,622][20347] Updated weights for policy 0, policy_version 1910563 (0.0006) [2023-12-27 18:53:03,672][20347] Updated weights for policy 0, policy_version 1910573 (0.0005) [2023-12-27 18:53:03,727][20347] Updated weights for policy 0, policy_version 1910583 (0.0005) [2023-12-27 18:53:04,176][20381] Updated weights for policy 1, policy_version 1909032 (0.0008) [2023-12-27 18:53:04,233][20381] Updated weights for policy 1, policy_version 1909042 (0.0008) [2023-12-27 18:53:04,294][20381] Updated weights for policy 1, policy_version 1909052 (0.0008) [2023-12-27 18:53:04,410][20347] Updated weights for policy 0, policy_version 1910593 (0.0008) [2023-12-27 18:53:04,458][20347] Updated weights for policy 0, policy_version 1910603 (0.0008) [2023-12-27 18:53:04,505][20347] Updated weights for policy 0, policy_version 1910613 (0.0008) [2023-12-27 18:53:04,555][20347] Updated weights for policy 0, policy_version 1910623 (0.0008) [2023-12-27 18:53:05,028][20381] Updated weights for policy 1, policy_version 1909062 (0.0008) [2023-12-27 18:53:05,080][20381] Updated weights for policy 1, policy_version 1909072 (0.0008) [2023-12-27 18:53:05,123][20381] Updated weights for policy 1, policy_version 1909082 (0.0008) [2023-12-27 18:53:05,143][20347] Updated weights for policy 0, policy_version 1910633 (0.0006) [2023-12-27 18:53:05,192][20347] Updated weights for policy 0, policy_version 1910643 (0.0005) [2023-12-27 18:53:05,242][20347] Updated weights for policy 0, policy_version 1910653 (0.0005) [2023-12-27 18:53:05,828][20347] Updated weights for policy 0, policy_version 1910663 (0.0007) [2023-12-27 18:53:05,875][20381] Updated weights for policy 1, policy_version 1909092 (0.0007) [2023-12-27 18:53:05,876][20347] Updated weights for policy 0, policy_version 1910673 (0.0007) [2023-12-27 18:53:05,921][20381] Updated weights for policy 1, policy_version 1909102 (0.0008) [2023-12-27 18:53:05,930][20347] Updated weights for policy 0, policy_version 1910683 (0.0005) [2023-12-27 18:53:05,971][20381] Updated weights for policy 1, policy_version 1909112 (0.0006) [2023-12-27 18:53:06,637][20347] Updated weights for policy 0, policy_version 1910693 (0.0008) [2023-12-27 18:53:06,700][20347] Updated weights for policy 0, policy_version 1910703 (0.0007) [2023-12-27 18:53:06,712][20381] Updated weights for policy 1, policy_version 1909122 (0.0008) [2023-12-27 18:53:06,756][20347] Updated weights for policy 0, policy_version 1910713 (0.0007) [2023-12-27 18:53:06,767][20381] Updated weights for policy 1, policy_version 1909132 (0.0006) [2023-12-27 18:53:06,819][20381] Updated weights for policy 1, policy_version 1909142 (0.0006) [2023-12-27 18:53:06,864][20381] Updated weights for policy 1, policy_version 1909152 (0.0005) [2023-12-27 18:53:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 978034688. Throughput: 0: 10319.2, 1: 10421.7. Samples: 978007628. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.320')] [2023-12-27 18:53:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001909152_488800256.pth... [2023-12-27 18:53:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001910720_489234432.pth... [2023-12-27 18:53:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001907968_488497152.pth [2023-12-27 18:53:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001909504_488923136.pth [2023-12-27 18:53:07,474][20347] Updated weights for policy 0, policy_version 1910723 (0.0008) [2023-12-27 18:53:07,523][20347] Updated weights for policy 0, policy_version 1910733 (0.0006) [2023-12-27 18:53:07,565][20381] Updated weights for policy 1, policy_version 1909162 (0.0007) [2023-12-27 18:53:07,571][20347] Updated weights for policy 0, policy_version 1910743 (0.0006) [2023-12-27 18:53:07,623][20381] Updated weights for policy 1, policy_version 1909172 (0.0006) [2023-12-27 18:53:07,679][20381] Updated weights for policy 1, policy_version 1909182 (0.0006) [2023-12-27 18:53:08,287][20381] Updated weights for policy 1, policy_version 1909192 (0.0007) [2023-12-27 18:53:08,306][20347] Updated weights for policy 0, policy_version 1910753 (0.0006) [2023-12-27 18:53:08,345][20381] Updated weights for policy 1, policy_version 1909202 (0.0008) [2023-12-27 18:53:08,370][20347] Updated weights for policy 0, policy_version 1910763 (0.0008) [2023-12-27 18:53:08,398][20381] Updated weights for policy 1, policy_version 1909212 (0.0007) [2023-12-27 18:53:08,426][20347] Updated weights for policy 0, policy_version 1910773 (0.0008) [2023-12-27 18:53:08,482][20347] Updated weights for policy 0, policy_version 1910783 (0.0008) [2023-12-27 18:53:09,034][20381] Updated weights for policy 1, policy_version 1909222 (0.0007) [2023-12-27 18:53:09,083][20381] Updated weights for policy 1, policy_version 1909232 (0.0008) [2023-12-27 18:53:09,131][20381] Updated weights for policy 1, policy_version 1909242 (0.0006) [2023-12-27 18:53:09,203][20347] Updated weights for policy 0, policy_version 1910793 (0.0007) [2023-12-27 18:53:09,277][20347] Updated weights for policy 0, policy_version 1910803 (0.0008) [2023-12-27 18:53:09,341][20347] Updated weights for policy 0, policy_version 1910813 (0.0008) [2023-12-27 18:53:09,852][20381] Updated weights for policy 1, policy_version 1909252 (0.0006) [2023-12-27 18:53:09,918][20381] Updated weights for policy 1, policy_version 1909262 (0.0007) [2023-12-27 18:53:09,983][20381] Updated weights for policy 1, policy_version 1909272 (0.0007) [2023-12-27 18:53:10,096][20347] Updated weights for policy 0, policy_version 1910823 (0.0008) [2023-12-27 18:53:10,156][20347] Updated weights for policy 0, policy_version 1910833 (0.0009) [2023-12-27 18:53:10,212][20347] Updated weights for policy 0, policy_version 1910843 (0.0008) [2023-12-27 18:53:10,599][20381] Updated weights for policy 1, policy_version 1909282 (0.0007) [2023-12-27 18:53:10,659][20381] Updated weights for policy 1, policy_version 1909292 (0.0009) [2023-12-27 18:53:10,705][20381] Updated weights for policy 1, policy_version 1909302 (0.0008) [2023-12-27 18:53:10,755][20381] Updated weights for policy 1, policy_version 1909312 (0.0006) [2023-12-27 18:53:10,983][20347] Updated weights for policy 0, policy_version 1910853 (0.0008) [2023-12-27 18:53:11,046][20347] Updated weights for policy 0, policy_version 1910863 (0.0007) [2023-12-27 18:53:11,113][20347] Updated weights for policy 0, policy_version 1910873 (0.0008) [2023-12-27 18:53:11,453][20381] Updated weights for policy 1, policy_version 1909322 (0.0008) [2023-12-27 18:53:11,514][20381] Updated weights for policy 1, policy_version 1909332 (0.0008) [2023-12-27 18:53:11,568][20381] Updated weights for policy 1, policy_version 1909342 (0.0008) [2023-12-27 18:53:11,882][20347] Updated weights for policy 0, policy_version 1910883 (0.0008) [2023-12-27 18:53:11,943][20347] Updated weights for policy 0, policy_version 1910893 (0.0009) [2023-12-27 18:53:11,995][20347] Updated weights for policy 0, policy_version 1910903 (0.0007) [2023-12-27 18:53:12,290][20381] Updated weights for policy 1, policy_version 1909352 (0.0008) [2023-12-27 18:53:12,346][20381] Updated weights for policy 1, policy_version 1909362 (0.0008) [2023-12-27 18:53:12,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 978132992. Throughput: 0: 10338.6, 1: 10394.2. Samples: 978128656. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.130')] [2023-12-27 18:53:12,405][20381] Updated weights for policy 1, policy_version 1909372 (0.0007) [2023-12-27 18:53:12,731][20347] Updated weights for policy 0, policy_version 1910913 (0.0007) [2023-12-27 18:53:12,793][20347] Updated weights for policy 0, policy_version 1910923 (0.0007) [2023-12-27 18:53:12,856][20347] Updated weights for policy 0, policy_version 1910933 (0.0006) [2023-12-27 18:53:12,927][20347] Updated weights for policy 0, policy_version 1910943 (0.0005) [2023-12-27 18:53:13,094][20381] Updated weights for policy 1, policy_version 1909382 (0.0007) [2023-12-27 18:53:13,152][20381] Updated weights for policy 1, policy_version 1909392 (0.0005) [2023-12-27 18:53:13,215][20381] Updated weights for policy 1, policy_version 1909402 (0.0008) [2023-12-27 18:53:13,498][20347] Updated weights for policy 0, policy_version 1910953 (0.0010) [2023-12-27 18:53:13,548][20347] Updated weights for policy 0, policy_version 1910963 (0.0010) [2023-12-27 18:53:13,600][20347] Updated weights for policy 0, policy_version 1910973 (0.0005) [2023-12-27 18:53:13,845][20381] Updated weights for policy 1, policy_version 1909412 (0.0008) [2023-12-27 18:53:13,910][20381] Updated weights for policy 1, policy_version 1909422 (0.0008) [2023-12-27 18:53:13,966][20381] Updated weights for policy 1, policy_version 1909432 (0.0011) [2023-12-27 18:53:14,223][20347] Updated weights for policy 0, policy_version 1910983 (0.0007) [2023-12-27 18:53:14,270][20347] Updated weights for policy 0, policy_version 1910993 (0.0007) [2023-12-27 18:53:14,328][20347] Updated weights for policy 0, policy_version 1911003 (0.0007) [2023-12-27 18:53:14,619][20381] Updated weights for policy 1, policy_version 1909442 (0.0010) [2023-12-27 18:53:14,667][20381] Updated weights for policy 1, policy_version 1909452 (0.0007) [2023-12-27 18:53:14,715][20381] Updated weights for policy 1, policy_version 1909462 (0.0008) [2023-12-27 18:53:14,769][20381] Updated weights for policy 1, policy_version 1909472 (0.0007) [2023-12-27 18:53:14,945][20347] Updated weights for policy 0, policy_version 1911013 (0.0008) [2023-12-27 18:53:15,007][20347] Updated weights for policy 0, policy_version 1911023 (0.0008) [2023-12-27 18:53:15,071][20347] Updated weights for policy 0, policy_version 1911033 (0.0006) [2023-12-27 18:53:15,475][20381] Updated weights for policy 1, policy_version 1909482 (0.0010) [2023-12-27 18:53:15,524][20381] Updated weights for policy 1, policy_version 1909492 (0.0010) [2023-12-27 18:53:15,575][20381] Updated weights for policy 1, policy_version 1909502 (0.0010) [2023-12-27 18:53:15,736][20347] Updated weights for policy 0, policy_version 1911043 (0.0008) [2023-12-27 18:53:15,801][20347] Updated weights for policy 0, policy_version 1911053 (0.0008) [2023-12-27 18:53:15,868][20347] Updated weights for policy 0, policy_version 1911063 (0.0008) [2023-12-27 18:53:16,131][20381] Updated weights for policy 1, policy_version 1909512 (0.0010) [2023-12-27 18:53:16,186][20381] Updated weights for policy 1, policy_version 1909522 (0.0010) [2023-12-27 18:53:16,240][20381] Updated weights for policy 1, policy_version 1909532 (0.0007) [2023-12-27 18:53:16,515][20347] Updated weights for policy 0, policy_version 1911073 (0.0008) [2023-12-27 18:53:16,572][20347] Updated weights for policy 0, policy_version 1911083 (0.0010) [2023-12-27 18:53:16,624][20347] Updated weights for policy 0, policy_version 1911093 (0.0009) [2023-12-27 18:53:16,676][20347] Updated weights for policy 0, policy_version 1911103 (0.0008) [2023-12-27 18:53:16,906][20381] Updated weights for policy 1, policy_version 1909542 (0.0010) [2023-12-27 18:53:16,955][20381] Updated weights for policy 1, policy_version 1909552 (0.0010) [2023-12-27 18:53:17,004][20381] Updated weights for policy 1, policy_version 1909562 (0.0010) [2023-12-27 18:53:17,325][20347] Updated weights for policy 0, policy_version 1911113 (0.0008) [2023-12-27 18:53:17,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 978239488. Throughput: 0: 10271.3, 1: 10426.1. Samples: 978252976. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:17,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.530')] [2023-12-27 18:53:17,388][20347] Updated weights for policy 0, policy_version 1911123 (0.0009) [2023-12-27 18:53:17,443][20347] Updated weights for policy 0, policy_version 1911133 (0.0010) [2023-12-27 18:53:17,710][20381] Updated weights for policy 1, policy_version 1909572 (0.0010) [2023-12-27 18:53:17,762][20381] Updated weights for policy 1, policy_version 1909582 (0.0010) [2023-12-27 18:53:17,816][20381] Updated weights for policy 1, policy_version 1909592 (0.0010) [2023-12-27 18:53:18,108][20347] Updated weights for policy 0, policy_version 1911143 (0.0010) [2023-12-27 18:53:18,169][20347] Updated weights for policy 0, policy_version 1911153 (0.0010) [2023-12-27 18:53:18,230][20347] Updated weights for policy 0, policy_version 1911163 (0.0010) [2023-12-27 18:53:18,468][20381] Updated weights for policy 1, policy_version 1909602 (0.0009) [2023-12-27 18:53:18,538][20381] Updated weights for policy 1, policy_version 1909612 (0.0006) [2023-12-27 18:53:18,610][20381] Updated weights for policy 1, policy_version 1909622 (0.0007) [2023-12-27 18:53:18,674][20381] Updated weights for policy 1, policy_version 1909632 (0.0010) [2023-12-27 18:53:18,900][20347] Updated weights for policy 0, policy_version 1911173 (0.0008) [2023-12-27 18:53:18,954][20347] Updated weights for policy 0, policy_version 1911183 (0.0005) [2023-12-27 18:53:19,008][20347] Updated weights for policy 0, policy_version 1911193 (0.0007) [2023-12-27 18:53:19,282][20381] Updated weights for policy 1, policy_version 1909642 (0.0007) [2023-12-27 18:53:19,346][20381] Updated weights for policy 1, policy_version 1909652 (0.0009) [2023-12-27 18:53:19,416][20381] Updated weights for policy 1, policy_version 1909662 (0.0009) [2023-12-27 18:53:19,643][20347] Updated weights for policy 0, policy_version 1911203 (0.0010) [2023-12-27 18:53:19,702][20347] Updated weights for policy 0, policy_version 1911213 (0.0011) [2023-12-27 18:53:19,758][20347] Updated weights for policy 0, policy_version 1911223 (0.0009) [2023-12-27 18:53:20,088][20381] Updated weights for policy 1, policy_version 1909672 (0.0010) [2023-12-27 18:53:20,142][20381] Updated weights for policy 1, policy_version 1909682 (0.0011) [2023-12-27 18:53:20,195][20381] Updated weights for policy 1, policy_version 1909692 (0.0010) [2023-12-27 18:53:20,495][20347] Updated weights for policy 0, policy_version 1911233 (0.0007) [2023-12-27 18:53:20,541][20347] Updated weights for policy 0, policy_version 1911243 (0.0007) [2023-12-27 18:53:20,605][20347] Updated weights for policy 0, policy_version 1911253 (0.0007) [2023-12-27 18:53:20,663][20347] Updated weights for policy 0, policy_version 1911263 (0.0007) [2023-12-27 18:53:20,915][20381] Updated weights for policy 1, policy_version 1909702 (0.0010) [2023-12-27 18:53:20,975][20381] Updated weights for policy 1, policy_version 1909712 (0.0011) [2023-12-27 18:53:21,043][20381] Updated weights for policy 1, policy_version 1909722 (0.0011) [2023-12-27 18:53:21,455][20347] Updated weights for policy 0, policy_version 1911273 (0.0008) [2023-12-27 18:53:21,526][20347] Updated weights for policy 0, policy_version 1911283 (0.0008) [2023-12-27 18:53:21,596][20347] Updated weights for policy 0, policy_version 1911293 (0.0008) [2023-12-27 18:53:21,727][20381] Updated weights for policy 1, policy_version 1909732 (0.0010) [2023-12-27 18:53:21,796][20381] Updated weights for policy 1, policy_version 1909742 (0.0010) [2023-12-27 18:53:21,854][20381] Updated weights for policy 1, policy_version 1909752 (0.0007) [2023-12-27 18:53:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20438.3). Total num frames: 978337792. Throughput: 0: 10262.9, 1: 10359.8. Samples: 978314648. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '951.170')] [2023-12-27 18:53:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001909760_488955904.pth... [2023-12-27 18:53:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001908576_488652800.pth [2023-12-27 18:53:22,380][20347] Updated weights for policy 0, policy_version 1911303 (0.0008) [2023-12-27 18:53:22,446][20347] Updated weights for policy 0, policy_version 1911313 (0.0008) [2023-12-27 18:53:22,511][20347] Updated weights for policy 0, policy_version 1911323 (0.0007) [2023-12-27 18:53:22,544][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001911328_489390080.pth... [2023-12-27 18:53:22,545][20381] Updated weights for policy 1, policy_version 1909762 (0.0010) [2023-12-27 18:53:22,549][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001910112_489078784.pth [2023-12-27 18:53:22,599][20381] Updated weights for policy 1, policy_version 1909772 (0.0008) [2023-12-27 18:53:22,649][20381] Updated weights for policy 1, policy_version 1909782 (0.0011) [2023-12-27 18:53:22,699][20381] Updated weights for policy 1, policy_version 1909792 (0.0011) [2023-12-27 18:53:23,184][20347] Updated weights for policy 0, policy_version 1911333 (0.0006) [2023-12-27 18:53:23,250][20347] Updated weights for policy 0, policy_version 1911343 (0.0008) [2023-12-27 18:53:23,309][20347] Updated weights for policy 0, policy_version 1911353 (0.0008) [2023-12-27 18:53:23,427][20381] Updated weights for policy 1, policy_version 1909802 (0.0006) [2023-12-27 18:53:23,489][20381] Updated weights for policy 1, policy_version 1909812 (0.0011) [2023-12-27 18:53:23,545][20381] Updated weights for policy 1, policy_version 1909822 (0.0011) [2023-12-27 18:53:23,885][20347] Updated weights for policy 0, policy_version 1911363 (0.0008) [2023-12-27 18:53:23,943][20347] Updated weights for policy 0, policy_version 1911373 (0.0008) [2023-12-27 18:53:23,998][20347] Updated weights for policy 0, policy_version 1911383 (0.0007) [2023-12-27 18:53:24,175][20381] Updated weights for policy 1, policy_version 1909832 (0.0011) [2023-12-27 18:53:24,236][20381] Updated weights for policy 1, policy_version 1909842 (0.0011) [2023-12-27 18:53:24,295][20381] Updated weights for policy 1, policy_version 1909852 (0.0011) [2023-12-27 18:53:24,691][20347] Updated weights for policy 0, policy_version 1911393 (0.0008) [2023-12-27 18:53:24,752][20347] Updated weights for policy 0, policy_version 1911403 (0.0008) [2023-12-27 18:53:24,815][20347] Updated weights for policy 0, policy_version 1911413 (0.0010) [2023-12-27 18:53:24,872][20347] Updated weights for policy 0, policy_version 1911423 (0.0007) [2023-12-27 18:53:24,972][20381] Updated weights for policy 1, policy_version 1909862 (0.0010) [2023-12-27 18:53:25,024][20381] Updated weights for policy 1, policy_version 1909872 (0.0011) [2023-12-27 18:53:25,076][20381] Updated weights for policy 1, policy_version 1909882 (0.0011) [2023-12-27 18:53:25,524][20347] Updated weights for policy 0, policy_version 1911433 (0.0010) [2023-12-27 18:53:25,584][20347] Updated weights for policy 0, policy_version 1911443 (0.0007) [2023-12-27 18:53:25,643][20347] Updated weights for policy 0, policy_version 1911453 (0.0010) [2023-12-27 18:53:25,800][20381] Updated weights for policy 1, policy_version 1909892 (0.0010) [2023-12-27 18:53:25,848][20381] Updated weights for policy 1, policy_version 1909902 (0.0010) [2023-12-27 18:53:25,899][20381] Updated weights for policy 1, policy_version 1909912 (0.0008) [2023-12-27 18:53:26,250][20347] Updated weights for policy 0, policy_version 1911463 (0.0010) [2023-12-27 18:53:26,308][20347] Updated weights for policy 0, policy_version 1911473 (0.0007) [2023-12-27 18:53:26,369][20347] Updated weights for policy 0, policy_version 1911483 (0.0012) [2023-12-27 18:53:26,555][20381] Updated weights for policy 1, policy_version 1909922 (0.0006) [2023-12-27 18:53:26,607][20381] Updated weights for policy 1, policy_version 1909932 (0.0010) [2023-12-27 18:53:26,655][20381] Updated weights for policy 1, policy_version 1909942 (0.0010) [2023-12-27 18:53:26,706][20381] Updated weights for policy 1, policy_version 1909952 (0.0010) [2023-12-27 18:53:26,911][20347] Updated weights for policy 0, policy_version 1911493 (0.0008) [2023-12-27 18:53:26,965][20347] Updated weights for policy 0, policy_version 1911503 (0.0005) [2023-12-27 18:53:27,019][20347] Updated weights for policy 0, policy_version 1911513 (0.0009) [2023-12-27 18:53:27,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 978444288. Throughput: 0: 10281.6, 1: 10272.8. Samples: 978435180. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.660')] [2023-12-27 18:53:27,367][20381] Updated weights for policy 1, policy_version 1909962 (0.0011) [2023-12-27 18:53:27,418][20381] Updated weights for policy 1, policy_version 1909972 (0.0010) [2023-12-27 18:53:27,469][20381] Updated weights for policy 1, policy_version 1909982 (0.0010) [2023-12-27 18:53:27,659][20347] Updated weights for policy 0, policy_version 1911523 (0.0009) [2023-12-27 18:53:27,708][20347] Updated weights for policy 0, policy_version 1911533 (0.0007) [2023-12-27 18:53:27,756][20347] Updated weights for policy 0, policy_version 1911543 (0.0007) [2023-12-27 18:53:28,149][20381] Updated weights for policy 1, policy_version 1909992 (0.0008) [2023-12-27 18:53:28,206][20381] Updated weights for policy 1, policy_version 1910002 (0.0005) [2023-12-27 18:53:28,268][20381] Updated weights for policy 1, policy_version 1910012 (0.0005) [2023-12-27 18:53:28,347][20347] Updated weights for policy 0, policy_version 1911553 (0.0008) [2023-12-27 18:53:28,404][20347] Updated weights for policy 0, policy_version 1911563 (0.0009) [2023-12-27 18:53:28,466][20347] Updated weights for policy 0, policy_version 1911573 (0.0010) [2023-12-27 18:53:28,521][20347] Updated weights for policy 0, policy_version 1911583 (0.0008) [2023-12-27 18:53:28,906][20381] Updated weights for policy 1, policy_version 1910022 (0.0007) [2023-12-27 18:53:28,960][20381] Updated weights for policy 1, policy_version 1910032 (0.0007) [2023-12-27 18:53:29,020][20381] Updated weights for policy 1, policy_version 1910042 (0.0007) [2023-12-27 18:53:29,202][20347] Updated weights for policy 0, policy_version 1911593 (0.0010) [2023-12-27 18:53:29,259][20347] Updated weights for policy 0, policy_version 1911603 (0.0009) [2023-12-27 18:53:29,318][20347] Updated weights for policy 0, policy_version 1911613 (0.0006) [2023-12-27 18:53:29,741][20381] Updated weights for policy 1, policy_version 1910052 (0.0006) [2023-12-27 18:53:29,807][20381] Updated weights for policy 1, policy_version 1910062 (0.0005) [2023-12-27 18:53:29,877][20381] Updated weights for policy 1, policy_version 1910072 (0.0008) [2023-12-27 18:53:30,016][20347] Updated weights for policy 0, policy_version 1911623 (0.0007) [2023-12-27 18:53:30,072][20347] Updated weights for policy 0, policy_version 1911633 (0.0005) [2023-12-27 18:53:30,132][20347] Updated weights for policy 0, policy_version 1911643 (0.0005) [2023-12-27 18:53:30,532][20381] Updated weights for policy 1, policy_version 1910082 (0.0007) [2023-12-27 18:53:30,584][20381] Updated weights for policy 1, policy_version 1910092 (0.0005) [2023-12-27 18:53:30,634][20381] Updated weights for policy 1, policy_version 1910102 (0.0006) [2023-12-27 18:53:30,681][20381] Updated weights for policy 1, policy_version 1910112 (0.0006) [2023-12-27 18:53:30,683][20347] Updated weights for policy 0, policy_version 1911653 (0.0005) [2023-12-27 18:53:30,740][20347] Updated weights for policy 0, policy_version 1911663 (0.0007) [2023-12-27 18:53:30,797][20347] Updated weights for policy 0, policy_version 1911673 (0.0007) [2023-12-27 18:53:31,329][20381] Updated weights for policy 1, policy_version 1910122 (0.0010) [2023-12-27 18:53:31,395][20381] Updated weights for policy 1, policy_version 1910132 (0.0011) [2023-12-27 18:53:31,443][20381] Updated weights for policy 1, policy_version 1910142 (0.0008) [2023-12-27 18:53:31,471][20347] Updated weights for policy 0, policy_version 1911683 (0.0008) [2023-12-27 18:53:31,532][20347] Updated weights for policy 0, policy_version 1911693 (0.0007) [2023-12-27 18:53:31,584][20347] Updated weights for policy 0, policy_version 1911703 (0.0008) [2023-12-27 18:53:32,141][20381] Updated weights for policy 1, policy_version 1910152 (0.0006) [2023-12-27 18:53:32,195][20381] Updated weights for policy 1, policy_version 1910162 (0.0005) [2023-12-27 18:53:32,245][20347] Updated weights for policy 0, policy_version 1911713 (0.0008) [2023-12-27 18:53:32,255][20381] Updated weights for policy 1, policy_version 1910172 (0.0006) [2023-12-27 18:53:32,305][20347] Updated weights for policy 0, policy_version 1911723 (0.0010) [2023-12-27 18:53:32,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 978550784. Throughput: 0: 10274.4, 1: 10228.3. Samples: 978561844. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.640')] [2023-12-27 18:53:32,364][20347] Updated weights for policy 0, policy_version 1911733 (0.0010) [2023-12-27 18:53:32,426][20347] Updated weights for policy 0, policy_version 1911743 (0.0009) [2023-12-27 18:53:32,912][20381] Updated weights for policy 1, policy_version 1910182 (0.0010) [2023-12-27 18:53:32,967][20381] Updated weights for policy 1, policy_version 1910192 (0.0010) [2023-12-27 18:53:33,018][20381] Updated weights for policy 1, policy_version 1910202 (0.0010) [2023-12-27 18:53:33,038][20347] Updated weights for policy 0, policy_version 1911753 (0.0005) [2023-12-27 18:53:33,094][20347] Updated weights for policy 0, policy_version 1911763 (0.0005) [2023-12-27 18:53:33,152][20347] Updated weights for policy 0, policy_version 1911773 (0.0008) [2023-12-27 18:53:33,686][20381] Updated weights for policy 1, policy_version 1910212 (0.0010) [2023-12-27 18:53:33,744][20381] Updated weights for policy 1, policy_version 1910222 (0.0010) [2023-12-27 18:53:33,796][20381] Updated weights for policy 1, policy_version 1910232 (0.0010) [2023-12-27 18:53:33,811][20347] Updated weights for policy 0, policy_version 1911783 (0.0009) [2023-12-27 18:53:33,863][20347] Updated weights for policy 0, policy_version 1911793 (0.0008) [2023-12-27 18:53:33,908][20347] Updated weights for policy 0, policy_version 1911803 (0.0007) [2023-12-27 18:53:34,473][20381] Updated weights for policy 1, policy_version 1910242 (0.0010) [2023-12-27 18:53:34,524][20381] Updated weights for policy 1, policy_version 1910252 (0.0008) [2023-12-27 18:53:34,576][20381] Updated weights for policy 1, policy_version 1910262 (0.0008) [2023-12-27 18:53:34,610][20347] Updated weights for policy 0, policy_version 1911813 (0.0006) [2023-12-27 18:53:34,630][20381] Updated weights for policy 1, policy_version 1910272 (0.0007) [2023-12-27 18:53:34,671][20347] Updated weights for policy 0, policy_version 1911823 (0.0007) [2023-12-27 18:53:34,727][20347] Updated weights for policy 0, policy_version 1911833 (0.0009) [2023-12-27 18:53:35,242][20381] Updated weights for policy 1, policy_version 1910282 (0.0005) [2023-12-27 18:53:35,303][20381] Updated weights for policy 1, policy_version 1910292 (0.0005) [2023-12-27 18:53:35,360][20381] Updated weights for policy 1, policy_version 1910302 (0.0005) [2023-12-27 18:53:35,377][20347] Updated weights for policy 0, policy_version 1911843 (0.0008) [2023-12-27 18:53:35,427][20347] Updated weights for policy 0, policy_version 1911853 (0.0008) [2023-12-27 18:53:35,477][20347] Updated weights for policy 0, policy_version 1911863 (0.0007) [2023-12-27 18:53:35,968][20381] Updated weights for policy 1, policy_version 1910312 (0.0005) [2023-12-27 18:53:36,015][20381] Updated weights for policy 1, policy_version 1910322 (0.0005) [2023-12-27 18:53:36,070][20381] Updated weights for policy 1, policy_version 1910332 (0.0005) [2023-12-27 18:53:36,207][20347] Updated weights for policy 0, policy_version 1911873 (0.0007) [2023-12-27 18:53:36,264][20347] Updated weights for policy 0, policy_version 1911883 (0.0007) [2023-12-27 18:53:36,322][20347] Updated weights for policy 0, policy_version 1911893 (0.0008) [2023-12-27 18:53:36,379][20347] Updated weights for policy 0, policy_version 1911903 (0.0009) [2023-12-27 18:53:36,659][20381] Updated weights for policy 1, policy_version 1910342 (0.0007) [2023-12-27 18:53:36,718][20381] Updated weights for policy 1, policy_version 1910352 (0.0006) [2023-12-27 18:53:36,767][20381] Updated weights for policy 1, policy_version 1910362 (0.0007) [2023-12-27 18:53:37,062][20347] Updated weights for policy 0, policy_version 1911913 (0.0006) [2023-12-27 18:53:37,122][20347] Updated weights for policy 0, policy_version 1911923 (0.0005) [2023-12-27 18:53:37,176][20347] Updated weights for policy 0, policy_version 1911933 (0.0006) [2023-12-27 18:53:37,355][20381] Updated weights for policy 1, policy_version 1910372 (0.0006) [2023-12-27 18:53:37,362][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20493.9). Total num frames: 978657280. Throughput: 0: 10298.7, 1: 10281.0. Samples: 978625120. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.670')] [2023-12-27 18:53:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001911936_489545728.pth... [2023-12-27 18:53:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001910720_489234432.pth [2023-12-27 18:53:37,416][20381] Updated weights for policy 1, policy_version 1910382 (0.0009) [2023-12-27 18:53:37,463][20381] Updated weights for policy 1, policy_version 1910392 (0.0008) [2023-12-27 18:53:37,501][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001910400_489119744.pth... [2023-12-27 18:53:37,504][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001909152_488800256.pth [2023-12-27 18:53:37,810][20347] Updated weights for policy 0, policy_version 1911943 (0.0008) [2023-12-27 18:53:37,863][20347] Updated weights for policy 0, policy_version 1911953 (0.0008) [2023-12-27 18:53:37,917][20347] Updated weights for policy 0, policy_version 1911963 (0.0008) [2023-12-27 18:53:38,191][20381] Updated weights for policy 1, policy_version 1910402 (0.0007) [2023-12-27 18:53:38,238][20381] Updated weights for policy 1, policy_version 1910412 (0.0007) [2023-12-27 18:53:38,282][20381] Updated weights for policy 1, policy_version 1910422 (0.0008) [2023-12-27 18:53:38,341][20381] Updated weights for policy 1, policy_version 1910432 (0.0007) [2023-12-27 18:53:38,620][20347] Updated weights for policy 0, policy_version 1911973 (0.0008) [2023-12-27 18:53:38,676][20347] Updated weights for policy 0, policy_version 1911983 (0.0008) [2023-12-27 18:53:38,727][20347] Updated weights for policy 0, policy_version 1911993 (0.0008) [2023-12-27 18:53:39,071][20381] Updated weights for policy 1, policy_version 1910442 (0.0008) [2023-12-27 18:53:39,124][20381] Updated weights for policy 1, policy_version 1910452 (0.0008) [2023-12-27 18:53:39,183][20381] Updated weights for policy 1, policy_version 1910462 (0.0008) [2023-12-27 18:53:39,501][20347] Updated weights for policy 0, policy_version 1912003 (0.0008) [2023-12-27 18:53:39,554][20347] Updated weights for policy 0, policy_version 1912013 (0.0007) [2023-12-27 18:53:39,614][20347] Updated weights for policy 0, policy_version 1912023 (0.0006) [2023-12-27 18:53:39,944][20381] Updated weights for policy 1, policy_version 1910472 (0.0008) [2023-12-27 18:53:40,001][20381] Updated weights for policy 1, policy_version 1910482 (0.0008) [2023-12-27 18:53:40,065][20381] Updated weights for policy 1, policy_version 1910492 (0.0008) [2023-12-27 18:53:40,303][20347] Updated weights for policy 0, policy_version 1912033 (0.0008) [2023-12-27 18:53:40,366][20347] Updated weights for policy 0, policy_version 1912043 (0.0011) [2023-12-27 18:53:40,422][20347] Updated weights for policy 0, policy_version 1912053 (0.0011) [2023-12-27 18:53:40,481][20347] Updated weights for policy 0, policy_version 1912063 (0.0011) [2023-12-27 18:53:40,735][20381] Updated weights for policy 1, policy_version 1910502 (0.0006) [2023-12-27 18:53:40,788][20381] Updated weights for policy 1, policy_version 1910512 (0.0005) [2023-12-27 18:53:40,843][20381] Updated weights for policy 1, policy_version 1910522 (0.0005) [2023-12-27 18:53:41,230][20347] Updated weights for policy 0, policy_version 1912073 (0.0008) [2023-12-27 18:53:41,295][20347] Updated weights for policy 0, policy_version 1912083 (0.0008) [2023-12-27 18:53:41,355][20347] Updated weights for policy 0, policy_version 1912093 (0.0008) [2023-12-27 18:53:41,505][20381] Updated weights for policy 1, policy_version 1910532 (0.0007) [2023-12-27 18:53:41,565][20381] Updated weights for policy 1, policy_version 1910542 (0.0006) [2023-12-27 18:53:41,629][20381] Updated weights for policy 1, policy_version 1910552 (0.0008) [2023-12-27 18:53:42,132][20347] Updated weights for policy 0, policy_version 1912103 (0.0008) [2023-12-27 18:53:42,189][20347] Updated weights for policy 0, policy_version 1912113 (0.0008) [2023-12-27 18:53:42,242][20347] Updated weights for policy 0, policy_version 1912123 (0.0006) [2023-12-27 18:53:42,270][20381] Updated weights for policy 1, policy_version 1910562 (0.0008) [2023-12-27 18:53:42,332][20381] Updated weights for policy 1, policy_version 1910572 (0.0007) [2023-12-27 18:53:42,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 978755584. Throughput: 0: 10291.3, 1: 10241.8. Samples: 978747748. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.670')] [2023-12-27 18:53:42,397][20381] Updated weights for policy 1, policy_version 1910582 (0.0007) [2023-12-27 18:53:42,446][20381] Updated weights for policy 1, policy_version 1910592 (0.0005) [2023-12-27 18:53:42,935][20347] Updated weights for policy 0, policy_version 1912133 (0.0006) [2023-12-27 18:53:42,983][20347] Updated weights for policy 0, policy_version 1912143 (0.0006) [2023-12-27 18:53:43,033][20347] Updated weights for policy 0, policy_version 1912153 (0.0007) [2023-12-27 18:53:43,101][20381] Updated weights for policy 1, policy_version 1910602 (0.0008) [2023-12-27 18:53:43,147][20381] Updated weights for policy 1, policy_version 1910612 (0.0007) [2023-12-27 18:53:43,196][20381] Updated weights for policy 1, policy_version 1910622 (0.0008) [2023-12-27 18:53:43,661][20347] Updated weights for policy 0, policy_version 1912163 (0.0009) [2023-12-27 18:53:43,713][20347] Updated weights for policy 0, policy_version 1912173 (0.0005) [2023-12-27 18:53:43,774][20347] Updated weights for policy 0, policy_version 1912183 (0.0005) [2023-12-27 18:53:43,936][20381] Updated weights for policy 1, policy_version 1910632 (0.0010) [2023-12-27 18:53:43,985][20381] Updated weights for policy 1, policy_version 1910642 (0.0008) [2023-12-27 18:53:44,031][20381] Updated weights for policy 1, policy_version 1910652 (0.0007) [2023-12-27 18:53:44,288][20347] Updated weights for policy 0, policy_version 1912193 (0.0005) [2023-12-27 18:53:44,339][20347] Updated weights for policy 0, policy_version 1912203 (0.0005) [2023-12-27 18:53:44,389][20347] Updated weights for policy 0, policy_version 1912213 (0.0005) [2023-12-27 18:53:44,445][20347] Updated weights for policy 0, policy_version 1912223 (0.0005) [2023-12-27 18:53:44,796][20381] Updated weights for policy 1, policy_version 1910662 (0.0009) [2023-12-27 18:53:44,855][20381] Updated weights for policy 1, policy_version 1910672 (0.0008) [2023-12-27 18:53:44,911][20381] Updated weights for policy 1, policy_version 1910682 (0.0007) [2023-12-27 18:53:45,126][20347] Updated weights for policy 0, policy_version 1912233 (0.0007) [2023-12-27 18:53:45,186][20347] Updated weights for policy 0, policy_version 1912243 (0.0008) [2023-12-27 18:53:45,243][20347] Updated weights for policy 0, policy_version 1912253 (0.0008) [2023-12-27 18:53:45,632][20381] Updated weights for policy 1, policy_version 1910692 (0.0007) [2023-12-27 18:53:45,691][20381] Updated weights for policy 1, policy_version 1910702 (0.0006) [2023-12-27 18:53:45,750][20381] Updated weights for policy 1, policy_version 1910712 (0.0007) [2023-12-27 18:53:45,822][20347] Updated weights for policy 0, policy_version 1912263 (0.0006) [2023-12-27 18:53:45,870][20347] Updated weights for policy 0, policy_version 1912273 (0.0005) [2023-12-27 18:53:45,922][20347] Updated weights for policy 0, policy_version 1912283 (0.0007) [2023-12-27 18:53:46,417][20381] Updated weights for policy 1, policy_version 1910722 (0.0007) [2023-12-27 18:53:46,465][20381] Updated weights for policy 1, policy_version 1910732 (0.0007) [2023-12-27 18:53:46,516][20381] Updated weights for policy 1, policy_version 1910742 (0.0008) [2023-12-27 18:53:46,567][20381] Updated weights for policy 1, policy_version 1910752 (0.0009) [2023-12-27 18:53:46,598][20347] Updated weights for policy 0, policy_version 1912293 (0.0009) [2023-12-27 18:53:46,646][20347] Updated weights for policy 0, policy_version 1912303 (0.0008) [2023-12-27 18:53:46,697][20347] Updated weights for policy 0, policy_version 1912313 (0.0009) [2023-12-27 18:53:47,163][20381] Updated weights for policy 1, policy_version 1910762 (0.0005) [2023-12-27 18:53:47,220][20381] Updated weights for policy 1, policy_version 1910772 (0.0008) [2023-12-27 18:53:47,281][20381] Updated weights for policy 1, policy_version 1910782 (0.0008) [2023-12-27 18:53:47,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 978862080. Throughput: 0: 10311.8, 1: 10260.4. Samples: 978871880. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.410')] [2023-12-27 18:53:47,387][20347] Updated weights for policy 0, policy_version 1912323 (0.0006) [2023-12-27 18:53:47,454][20347] Updated weights for policy 0, policy_version 1912333 (0.0009) [2023-12-27 18:53:47,522][20347] Updated weights for policy 0, policy_version 1912343 (0.0008) [2023-12-27 18:53:47,858][20381] Updated weights for policy 1, policy_version 1910792 (0.0006) [2023-12-27 18:53:47,917][20381] Updated weights for policy 1, policy_version 1910802 (0.0005) [2023-12-27 18:53:47,972][20381] Updated weights for policy 1, policy_version 1910812 (0.0005) [2023-12-27 18:53:48,226][20347] Updated weights for policy 0, policy_version 1912353 (0.0007) [2023-12-27 18:53:48,279][20347] Updated weights for policy 0, policy_version 1912363 (0.0008) [2023-12-27 18:53:48,329][20347] Updated weights for policy 0, policy_version 1912373 (0.0008) [2023-12-27 18:53:48,390][20347] Updated weights for policy 0, policy_version 1912383 (0.0007) [2023-12-27 18:53:48,601][20381] Updated weights for policy 1, policy_version 1910822 (0.0006) [2023-12-27 18:53:48,659][20381] Updated weights for policy 1, policy_version 1910832 (0.0006) [2023-12-27 18:53:48,718][20381] Updated weights for policy 1, policy_version 1910842 (0.0005) [2023-12-27 18:53:49,137][20347] Updated weights for policy 0, policy_version 1912393 (0.0007) [2023-12-27 18:53:49,186][20347] Updated weights for policy 0, policy_version 1912404 (0.0006) [2023-12-27 18:53:49,255][20347] Updated weights for policy 0, policy_version 1912414 (0.0007) [2023-12-27 18:53:49,287][20381] Updated weights for policy 1, policy_version 1910852 (0.0007) [2023-12-27 18:53:49,347][20381] Updated weights for policy 1, policy_version 1910862 (0.0008) [2023-12-27 18:53:49,413][20381] Updated weights for policy 1, policy_version 1910872 (0.0008) [2023-12-27 18:53:49,979][20347] Updated weights for policy 0, policy_version 1912424 (0.0009) [2023-12-27 18:53:50,050][20347] Updated weights for policy 0, policy_version 1912434 (0.0006) [2023-12-27 18:53:50,115][20347] Updated weights for policy 0, policy_version 1912444 (0.0006) [2023-12-27 18:53:50,164][20381] Updated weights for policy 1, policy_version 1910882 (0.0007) [2023-12-27 18:53:50,225][20381] Updated weights for policy 1, policy_version 1910892 (0.0006) [2023-12-27 18:53:50,288][20381] Updated weights for policy 1, policy_version 1910902 (0.0005) [2023-12-27 18:53:50,353][20381] Updated weights for policy 1, policy_version 1910912 (0.0006) [2023-12-27 18:53:50,777][20347] Updated weights for policy 0, policy_version 1912454 (0.0007) [2023-12-27 18:53:50,831][20347] Updated weights for policy 0, policy_version 1912464 (0.0008) [2023-12-27 18:53:50,887][20347] Updated weights for policy 0, policy_version 1912474 (0.0011) [2023-12-27 18:53:50,957][20381] Updated weights for policy 1, policy_version 1910922 (0.0007) [2023-12-27 18:53:51,019][20381] Updated weights for policy 1, policy_version 1910932 (0.0009) [2023-12-27 18:53:51,089][20381] Updated weights for policy 1, policy_version 1910942 (0.0011) [2023-12-27 18:53:51,635][20347] Updated weights for policy 0, policy_version 1912484 (0.0009) [2023-12-27 18:53:51,704][20347] Updated weights for policy 0, policy_version 1912494 (0.0008) [2023-12-27 18:53:51,765][20347] Updated weights for policy 0, policy_version 1912504 (0.0008) [2023-12-27 18:53:51,810][20381] Updated weights for policy 1, policy_version 1910952 (0.0008) [2023-12-27 18:53:51,868][20381] Updated weights for policy 1, policy_version 1910962 (0.0006) [2023-12-27 18:53:51,928][20381] Updated weights for policy 1, policy_version 1910972 (0.0006) [2023-12-27 18:53:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 978960384. Throughput: 0: 10239.1, 1: 10336.0. Samples: 978933508. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '970.760')] [2023-12-27 18:53:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001910976_489267200.pth... [2023-12-27 18:53:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001912512_489693184.pth... [2023-12-27 18:53:52,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001909760_488955904.pth [2023-12-27 18:53:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001911328_489390080.pth [2023-12-27 18:53:52,483][20347] Updated weights for policy 0, policy_version 1912514 (0.0009) [2023-12-27 18:53:52,548][20347] Updated weights for policy 0, policy_version 1912524 (0.0006) [2023-12-27 18:53:52,610][20347] Updated weights for policy 0, policy_version 1912534 (0.0010) [2023-12-27 18:53:52,659][20381] Updated weights for policy 1, policy_version 1910982 (0.0008) [2023-12-27 18:53:52,666][20347] Updated weights for policy 0, policy_version 1912544 (0.0010) [2023-12-27 18:53:52,726][20381] Updated weights for policy 1, policy_version 1910992 (0.0011) [2023-12-27 18:53:52,785][20381] Updated weights for policy 1, policy_version 1911002 (0.0010) [2023-12-27 18:53:53,325][20347] Updated weights for policy 0, policy_version 1912554 (0.0006) [2023-12-27 18:53:53,386][20347] Updated weights for policy 0, policy_version 1912564 (0.0007) [2023-12-27 18:53:53,441][20381] Updated weights for policy 1, policy_version 1911012 (0.0010) [2023-12-27 18:53:53,449][20347] Updated weights for policy 0, policy_version 1912574 (0.0007) [2023-12-27 18:53:53,501][20381] Updated weights for policy 1, policy_version 1911022 (0.0011) [2023-12-27 18:53:53,561][20381] Updated weights for policy 1, policy_version 1911032 (0.0011) [2023-12-27 18:53:54,118][20347] Updated weights for policy 0, policy_version 1912584 (0.0006) [2023-12-27 18:53:54,174][20347] Updated weights for policy 0, policy_version 1912594 (0.0007) [2023-12-27 18:53:54,195][20381] Updated weights for policy 1, policy_version 1911042 (0.0006) [2023-12-27 18:53:54,228][20347] Updated weights for policy 0, policy_version 1912604 (0.0008) [2023-12-27 18:53:54,247][20381] Updated weights for policy 1, policy_version 1911052 (0.0005) [2023-12-27 18:53:54,298][20381] Updated weights for policy 1, policy_version 1911062 (0.0005) [2023-12-27 18:53:54,348][20381] Updated weights for policy 1, policy_version 1911072 (0.0007) [2023-12-27 18:53:54,961][20381] Updated weights for policy 1, policy_version 1911082 (0.0010) [2023-12-27 18:53:54,979][20347] Updated weights for policy 0, policy_version 1912614 (0.0006) [2023-12-27 18:53:55,021][20381] Updated weights for policy 1, policy_version 1911092 (0.0011) [2023-12-27 18:53:55,033][20347] Updated weights for policy 0, policy_version 1912624 (0.0008) [2023-12-27 18:53:55,080][20381] Updated weights for policy 1, policy_version 1911102 (0.0010) [2023-12-27 18:53:55,089][20347] Updated weights for policy 0, policy_version 1912634 (0.0007) [2023-12-27 18:53:55,699][20381] Updated weights for policy 1, policy_version 1911112 (0.0005) [2023-12-27 18:53:55,760][20381] Updated weights for policy 1, policy_version 1911122 (0.0006) [2023-12-27 18:53:55,822][20381] Updated weights for policy 1, policy_version 1911132 (0.0007) [2023-12-27 18:53:55,863][20347] Updated weights for policy 0, policy_version 1912644 (0.0007) [2023-12-27 18:53:55,920][20347] Updated weights for policy 0, policy_version 1912654 (0.0008) [2023-12-27 18:53:55,976][20347] Updated weights for policy 0, policy_version 1912664 (0.0010) [2023-12-27 18:53:56,353][20381] Updated weights for policy 1, policy_version 1911142 (0.0006) [2023-12-27 18:53:56,404][20381] Updated weights for policy 1, policy_version 1911152 (0.0005) [2023-12-27 18:53:56,455][20381] Updated weights for policy 1, policy_version 1911162 (0.0005) [2023-12-27 18:53:56,725][20347] Updated weights for policy 0, policy_version 1912674 (0.0009) [2023-12-27 18:53:56,781][20347] Updated weights for policy 0, policy_version 1912684 (0.0008) [2023-12-27 18:53:56,832][20347] Updated weights for policy 0, policy_version 1912694 (0.0007) [2023-12-27 18:53:56,880][20347] Updated weights for policy 0, policy_version 1912704 (0.0007) [2023-12-27 18:53:57,099][20381] Updated weights for policy 1, policy_version 1911172 (0.0007) [2023-12-27 18:53:57,161][20381] Updated weights for policy 1, policy_version 1911182 (0.0008) [2023-12-27 18:53:57,203][20381] Updated weights for policy 1, policy_version 1911192 (0.0007) [2023-12-27 18:53:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 979066880. Throughput: 0: 10209.1, 1: 10372.1. Samples: 979054808. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:53:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.680')] [2023-12-27 18:53:57,552][20347] Updated weights for policy 0, policy_version 1912714 (0.0005) [2023-12-27 18:53:57,595][20347] Updated weights for policy 0, policy_version 1912724 (0.0006) [2023-12-27 18:53:57,646][20347] Updated weights for policy 0, policy_version 1912734 (0.0006) [2023-12-27 18:53:57,764][20381] Updated weights for policy 1, policy_version 1911202 (0.0008) [2023-12-27 18:53:57,814][20381] Updated weights for policy 1, policy_version 1911212 (0.0005) [2023-12-27 18:53:57,864][20381] Updated weights for policy 1, policy_version 1911222 (0.0010) [2023-12-27 18:53:57,917][20381] Updated weights for policy 1, policy_version 1911232 (0.0009) [2023-12-27 18:53:58,363][20347] Updated weights for policy 0, policy_version 1912744 (0.0008) [2023-12-27 18:53:58,435][20347] Updated weights for policy 0, policy_version 1912754 (0.0008) [2023-12-27 18:53:58,497][20347] Updated weights for policy 0, policy_version 1912764 (0.0008) [2023-12-27 18:53:58,515][20381] Updated weights for policy 1, policy_version 1911242 (0.0010) [2023-12-27 18:53:58,577][20381] Updated weights for policy 1, policy_version 1911252 (0.0010) [2023-12-27 18:53:58,638][20381] Updated weights for policy 1, policy_version 1911262 (0.0007) [2023-12-27 18:53:59,310][20347] Updated weights for policy 0, policy_version 1912774 (0.0008) [2023-12-27 18:53:59,375][20347] Updated weights for policy 0, policy_version 1912784 (0.0008) [2023-12-27 18:53:59,426][20347] Updated weights for policy 0, policy_version 1912794 (0.0008) [2023-12-27 18:53:59,442][20381] Updated weights for policy 1, policy_version 1911272 (0.0007) [2023-12-27 18:53:59,488][20381] Updated weights for policy 1, policy_version 1911282 (0.0007) [2023-12-27 18:53:59,540][20381] Updated weights for policy 1, policy_version 1911292 (0.0006) [2023-12-27 18:54:00,134][20347] Updated weights for policy 0, policy_version 1912804 (0.0009) [2023-12-27 18:54:00,193][20347] Updated weights for policy 0, policy_version 1912814 (0.0011) [2023-12-27 18:54:00,211][20381] Updated weights for policy 1, policy_version 1911302 (0.0005) [2023-12-27 18:54:00,246][20347] Updated weights for policy 0, policy_version 1912824 (0.0010) [2023-12-27 18:54:00,263][20381] Updated weights for policy 1, policy_version 1911312 (0.0005) [2023-12-27 18:54:00,320][20381] Updated weights for policy 1, policy_version 1911322 (0.0007) [2023-12-27 18:54:00,919][20347] Updated weights for policy 0, policy_version 1912834 (0.0010) [2023-12-27 18:54:00,939][20381] Updated weights for policy 1, policy_version 1911332 (0.0006) [2023-12-27 18:54:00,970][20347] Updated weights for policy 0, policy_version 1912844 (0.0010) [2023-12-27 18:54:00,994][20381] Updated weights for policy 1, policy_version 1911342 (0.0005) [2023-12-27 18:54:01,018][20347] Updated weights for policy 0, policy_version 1912854 (0.0010) [2023-12-27 18:54:01,054][20381] Updated weights for policy 1, policy_version 1911352 (0.0006) [2023-12-27 18:54:01,080][20347] Updated weights for policy 0, policy_version 1912864 (0.0008) [2023-12-27 18:54:01,781][20381] Updated weights for policy 1, policy_version 1911362 (0.0008) [2023-12-27 18:54:01,786][20347] Updated weights for policy 0, policy_version 1912874 (0.0008) [2023-12-27 18:54:01,835][20381] Updated weights for policy 1, policy_version 1911372 (0.0008) [2023-12-27 18:54:01,840][20347] Updated weights for policy 0, policy_version 1912884 (0.0006) [2023-12-27 18:54:01,891][20381] Updated weights for policy 1, policy_version 1911382 (0.0008) [2023-12-27 18:54:01,897][20347] Updated weights for policy 0, policy_version 1912894 (0.0005) [2023-12-27 18:54:01,947][20381] Updated weights for policy 1, policy_version 1911392 (0.0008) [2023-12-27 18:54:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 979165184. Throughput: 0: 10141.3, 1: 10381.3. Samples: 979176488. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '893.010')] [2023-12-27 18:54:02,529][20347] Updated weights for policy 0, policy_version 1912904 (0.0010) [2023-12-27 18:54:02,589][20347] Updated weights for policy 0, policy_version 1912914 (0.0010) [2023-12-27 18:54:02,649][20347] Updated weights for policy 0, policy_version 1912924 (0.0006) [2023-12-27 18:54:02,671][20381] Updated weights for policy 1, policy_version 1911402 (0.0010) [2023-12-27 18:54:02,727][20381] Updated weights for policy 1, policy_version 1911412 (0.0009) [2023-12-27 18:54:02,787][20381] Updated weights for policy 1, policy_version 1911422 (0.0005) [2023-12-27 18:54:03,254][20347] Updated weights for policy 0, policy_version 1912934 (0.0008) [2023-12-27 18:54:03,303][20347] Updated weights for policy 0, policy_version 1912944 (0.0010) [2023-12-27 18:54:03,355][20347] Updated weights for policy 0, policy_version 1912954 (0.0010) [2023-12-27 18:54:03,359][20381] Updated weights for policy 1, policy_version 1911432 (0.0009) [2023-12-27 18:54:03,407][20381] Updated weights for policy 1, policy_version 1911442 (0.0010) [2023-12-27 18:54:03,462][20381] Updated weights for policy 1, policy_version 1911452 (0.0010) [2023-12-27 18:54:04,065][20347] Updated weights for policy 0, policy_version 1912964 (0.0008) [2023-12-27 18:54:04,106][20381] Updated weights for policy 1, policy_version 1911462 (0.0011) [2023-12-27 18:54:04,125][20347] Updated weights for policy 0, policy_version 1912974 (0.0006) [2023-12-27 18:54:04,167][20381] Updated weights for policy 1, policy_version 1911472 (0.0011) [2023-12-27 18:54:04,195][20347] Updated weights for policy 0, policy_version 1912985 (0.0006) [2023-12-27 18:54:04,232][20381] Updated weights for policy 1, policy_version 1911482 (0.0009) [2023-12-27 18:54:04,837][20347] Updated weights for policy 0, policy_version 1912995 (0.0006) [2023-12-27 18:54:04,880][20347] Updated weights for policy 0, policy_version 1913005 (0.0007) [2023-12-27 18:54:04,931][20381] Updated weights for policy 1, policy_version 1911492 (0.0009) [2023-12-27 18:54:04,936][20347] Updated weights for policy 0, policy_version 1913015 (0.0006) [2023-12-27 18:54:04,979][20381] Updated weights for policy 1, policy_version 1911502 (0.0007) [2023-12-27 18:54:05,040][20381] Updated weights for policy 1, policy_version 1911512 (0.0005) [2023-12-27 18:54:05,639][20381] Updated weights for policy 1, policy_version 1911522 (0.0006) [2023-12-27 18:54:05,663][20347] Updated weights for policy 0, policy_version 1913025 (0.0007) [2023-12-27 18:54:05,694][20381] Updated weights for policy 1, policy_version 1911532 (0.0007) [2023-12-27 18:54:05,723][20347] Updated weights for policy 0, policy_version 1913035 (0.0005) [2023-12-27 18:54:05,740][20381] Updated weights for policy 1, policy_version 1911542 (0.0007) [2023-12-27 18:54:05,779][20347] Updated weights for policy 0, policy_version 1913045 (0.0005) [2023-12-27 18:54:05,801][20381] Updated weights for policy 1, policy_version 1911552 (0.0008) [2023-12-27 18:54:05,833][20347] Updated weights for policy 0, policy_version 1913055 (0.0005) [2023-12-27 18:54:06,440][20347] Updated weights for policy 0, policy_version 1913065 (0.0006) [2023-12-27 18:54:06,498][20347] Updated weights for policy 0, policy_version 1913075 (0.0006) [2023-12-27 18:54:06,521][20381] Updated weights for policy 1, policy_version 1911562 (0.0011) [2023-12-27 18:54:06,558][20347] Updated weights for policy 0, policy_version 1913085 (0.0007) [2023-12-27 18:54:06,584][20381] Updated weights for policy 1, policy_version 1911572 (0.0008) [2023-12-27 18:54:06,646][20381] Updated weights for policy 1, policy_version 1911582 (0.0006) [2023-12-27 18:54:07,114][20347] Updated weights for policy 0, policy_version 1913095 (0.0005) [2023-12-27 18:54:07,171][20347] Updated weights for policy 0, policy_version 1913105 (0.0005) [2023-12-27 18:54:07,226][20347] Updated weights for policy 0, policy_version 1913115 (0.0005) [2023-12-27 18:54:07,326][20381] Updated weights for policy 1, policy_version 1911592 (0.0011) [2023-12-27 18:54:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 979271680. Throughput: 0: 10154.3, 1: 10419.1. Samples: 979240452. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.870')] [2023-12-27 18:54:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001913120_489848832.pth... [2023-12-27 18:54:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001911936_489545728.pth [2023-12-27 18:54:07,382][20381] Updated weights for policy 1, policy_version 1911602 (0.0011) [2023-12-27 18:54:07,435][20381] Updated weights for policy 1, policy_version 1911612 (0.0011) [2023-12-27 18:54:07,457][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001911616_489431040.pth... [2023-12-27 18:54:07,461][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001910400_489119744.pth [2023-12-27 18:54:07,806][20347] Updated weights for policy 0, policy_version 1913125 (0.0008) [2023-12-27 18:54:07,854][20347] Updated weights for policy 0, policy_version 1913135 (0.0010) [2023-12-27 18:54:07,899][20347] Updated weights for policy 0, policy_version 1913145 (0.0010) [2023-12-27 18:54:08,054][20381] Updated weights for policy 1, policy_version 1911622 (0.0009) [2023-12-27 18:54:08,109][20381] Updated weights for policy 1, policy_version 1911632 (0.0008) [2023-12-27 18:54:08,168][20381] Updated weights for policy 1, policy_version 1911642 (0.0008) [2023-12-27 18:54:08,630][20347] Updated weights for policy 0, policy_version 1913155 (0.0010) [2023-12-27 18:54:08,694][20347] Updated weights for policy 0, policy_version 1913165 (0.0007) [2023-12-27 18:54:08,756][20347] Updated weights for policy 0, policy_version 1913175 (0.0009) [2023-12-27 18:54:08,898][20381] Updated weights for policy 1, policy_version 1911652 (0.0008) [2023-12-27 18:54:08,955][20381] Updated weights for policy 1, policy_version 1911662 (0.0008) [2023-12-27 18:54:09,011][20381] Updated weights for policy 1, policy_version 1911672 (0.0008) [2023-12-27 18:54:09,466][20347] Updated weights for policy 0, policy_version 1913185 (0.0008) [2023-12-27 18:54:09,531][20347] Updated weights for policy 0, policy_version 1913195 (0.0005) [2023-12-27 18:54:09,594][20347] Updated weights for policy 0, policy_version 1913205 (0.0006) [2023-12-27 18:54:09,653][20347] Updated weights for policy 0, policy_version 1913215 (0.0008) [2023-12-27 18:54:09,772][20381] Updated weights for policy 1, policy_version 1911682 (0.0008) [2023-12-27 18:54:09,825][20381] Updated weights for policy 1, policy_version 1911692 (0.0009) [2023-12-27 18:54:09,890][20381] Updated weights for policy 1, policy_version 1911702 (0.0008) [2023-12-27 18:54:09,952][20381] Updated weights for policy 1, policy_version 1911712 (0.0008) [2023-12-27 18:54:10,278][20347] Updated weights for policy 0, policy_version 1913225 (0.0008) [2023-12-27 18:54:10,335][20347] Updated weights for policy 0, policy_version 1913235 (0.0008) [2023-12-27 18:54:10,396][20347] Updated weights for policy 0, policy_version 1913245 (0.0007) [2023-12-27 18:54:10,654][20381] Updated weights for policy 1, policy_version 1911722 (0.0008) [2023-12-27 18:54:10,719][20381] Updated weights for policy 1, policy_version 1911732 (0.0008) [2023-12-27 18:54:10,778][20381] Updated weights for policy 1, policy_version 1911742 (0.0008) [2023-12-27 18:54:11,131][20347] Updated weights for policy 0, policy_version 1913255 (0.0008) [2023-12-27 18:54:11,180][20347] Updated weights for policy 0, policy_version 1913265 (0.0007) [2023-12-27 18:54:11,233][20347] Updated weights for policy 0, policy_version 1913275 (0.0007) [2023-12-27 18:54:11,583][20381] Updated weights for policy 1, policy_version 1911752 (0.0008) [2023-12-27 18:54:11,655][20381] Updated weights for policy 1, policy_version 1911762 (0.0008) [2023-12-27 18:54:11,703][20381] Updated weights for policy 1, policy_version 1911772 (0.0009) [2023-12-27 18:54:11,967][20347] Updated weights for policy 0, policy_version 1913285 (0.0009) [2023-12-27 18:54:12,027][20347] Updated weights for policy 0, policy_version 1913295 (0.0008) [2023-12-27 18:54:12,087][20347] Updated weights for policy 0, policy_version 1913305 (0.0008) [2023-12-27 18:54:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 979369984. Throughput: 0: 10226.1, 1: 10378.3. Samples: 979362380. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.140')] [2023-12-27 18:54:12,435][20381] Updated weights for policy 1, policy_version 1911782 (0.0008) [2023-12-27 18:54:12,488][20381] Updated weights for policy 1, policy_version 1911792 (0.0008) [2023-12-27 18:54:12,538][20381] Updated weights for policy 1, policy_version 1911802 (0.0008) [2023-12-27 18:54:12,864][20347] Updated weights for policy 0, policy_version 1913316 (0.0007) [2023-12-27 18:54:12,916][20347] Updated weights for policy 0, policy_version 1913326 (0.0006) [2023-12-27 18:54:12,968][20347] Updated weights for policy 0, policy_version 1913336 (0.0007) [2023-12-27 18:54:13,217][20381] Updated weights for policy 1, policy_version 1911812 (0.0008) [2023-12-27 18:54:13,273][20381] Updated weights for policy 1, policy_version 1911822 (0.0008) [2023-12-27 18:54:13,326][20381] Updated weights for policy 1, policy_version 1911832 (0.0008) [2023-12-27 18:54:13,543][20347] Updated weights for policy 0, policy_version 1913346 (0.0005) [2023-12-27 18:54:13,603][20347] Updated weights for policy 0, policy_version 1913356 (0.0007) [2023-12-27 18:54:13,661][20347] Updated weights for policy 0, policy_version 1913366 (0.0008) [2023-12-27 18:54:13,710][20347] Updated weights for policy 0, policy_version 1913376 (0.0006) [2023-12-27 18:54:14,052][20381] Updated weights for policy 1, policy_version 1911842 (0.0008) [2023-12-27 18:54:14,108][20381] Updated weights for policy 1, policy_version 1911852 (0.0008) [2023-12-27 18:54:14,153][20381] Updated weights for policy 1, policy_version 1911862 (0.0008) [2023-12-27 18:54:14,206][20381] Updated weights for policy 1, policy_version 1911872 (0.0007) [2023-12-27 18:54:14,361][20347] Updated weights for policy 0, policy_version 1913386 (0.0007) [2023-12-27 18:54:14,417][20347] Updated weights for policy 0, policy_version 1913396 (0.0007) [2023-12-27 18:54:14,470][20347] Updated weights for policy 0, policy_version 1913406 (0.0005) [2023-12-27 18:54:14,905][20381] Updated weights for policy 1, policy_version 1911882 (0.0008) [2023-12-27 18:54:14,969][20381] Updated weights for policy 1, policy_version 1911892 (0.0008) [2023-12-27 18:54:15,034][20381] Updated weights for policy 1, policy_version 1911902 (0.0008) [2023-12-27 18:54:15,053][20347] Updated weights for policy 0, policy_version 1913416 (0.0008) [2023-12-27 18:54:15,111][20347] Updated weights for policy 0, policy_version 1913426 (0.0008) [2023-12-27 18:54:15,168][20347] Updated weights for policy 0, policy_version 1913436 (0.0007) [2023-12-27 18:54:15,688][20381] Updated weights for policy 1, policy_version 1911912 (0.0007) [2023-12-27 18:54:15,734][20381] Updated weights for policy 1, policy_version 1911922 (0.0011) [2023-12-27 18:54:15,789][20381] Updated weights for policy 1, policy_version 1911932 (0.0010) [2023-12-27 18:54:15,823][20347] Updated weights for policy 0, policy_version 1913446 (0.0008) [2023-12-27 18:54:15,878][20347] Updated weights for policy 0, policy_version 1913456 (0.0006) [2023-12-27 18:54:15,946][20347] Updated weights for policy 0, policy_version 1913466 (0.0006) [2023-12-27 18:54:16,490][20381] Updated weights for policy 1, policy_version 1911942 (0.0007) [2023-12-27 18:54:16,535][20381] Updated weights for policy 1, policy_version 1911952 (0.0007) [2023-12-27 18:54:16,567][20347] Updated weights for policy 0, policy_version 1913476 (0.0007) [2023-12-27 18:54:16,581][20381] Updated weights for policy 1, policy_version 1911962 (0.0006) [2023-12-27 18:54:16,626][20347] Updated weights for policy 0, policy_version 1913486 (0.0007) [2023-12-27 18:54:16,682][20347] Updated weights for policy 0, policy_version 1913496 (0.0008) [2023-12-27 18:54:17,178][20381] Updated weights for policy 1, policy_version 1911972 (0.0006) [2023-12-27 18:54:17,235][20381] Updated weights for policy 1, policy_version 1911982 (0.0007) [2023-12-27 18:54:17,296][20381] Updated weights for policy 1, policy_version 1911992 (0.0008) [2023-12-27 18:54:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.6, 300 sec: 20493.9). Total num frames: 979476480. Throughput: 0: 10169.8, 1: 10366.9. Samples: 979485992. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.140')] [2023-12-27 18:54:17,394][20347] Updated weights for policy 0, policy_version 1913506 (0.0007) [2023-12-27 18:54:17,453][20347] Updated weights for policy 0, policy_version 1913516 (0.0006) [2023-12-27 18:54:17,513][20347] Updated weights for policy 0, policy_version 1913526 (0.0007) [2023-12-27 18:54:17,522][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000009 [2023-12-27 18:54:17,944][20381] Updated weights for policy 1, policy_version 1912002 (0.0008) [2023-12-27 18:54:17,991][20381] Updated weights for policy 1, policy_version 1912012 (0.0008) [2023-12-27 18:54:18,036][20381] Updated weights for policy 1, policy_version 1912022 (0.0008) [2023-12-27 18:54:18,085][20381] Updated weights for policy 1, policy_version 1912032 (0.0008) [2023-12-27 18:54:18,199][20347] Updated weights for policy 0, policy_version 1913536 (0.0008) [2023-12-27 18:54:18,261][20347] Updated weights for policy 0, policy_version 1913546 (0.0007) [2023-12-27 18:54:18,322][20347] Updated weights for policy 0, policy_version 1913556 (0.0008) [2023-12-27 18:54:18,858][20381] Updated weights for policy 1, policy_version 1912042 (0.0007) [2023-12-27 18:54:18,913][20381] Updated weights for policy 1, policy_version 1912052 (0.0008) [2023-12-27 18:54:18,948][20347] Updated weights for policy 0, policy_version 1913566 (0.0007) [2023-12-27 18:54:18,959][20381] Updated weights for policy 1, policy_version 1912062 (0.0008) [2023-12-27 18:54:19,007][20347] Updated weights for policy 0, policy_version 1913576 (0.0005) [2023-12-27 18:54:19,064][20347] Updated weights for policy 0, policy_version 1913586 (0.0005) [2023-12-27 18:54:19,627][20381] Updated weights for policy 1, policy_version 1912072 (0.0008) [2023-12-27 18:54:19,680][20347] Updated weights for policy 0, policy_version 1913596 (0.0006) [2023-12-27 18:54:19,688][20381] Updated weights for policy 1, policy_version 1912082 (0.0007) [2023-12-27 18:54:19,738][20347] Updated weights for policy 0, policy_version 1913606 (0.0007) [2023-12-27 18:54:19,745][20381] Updated weights for policy 1, policy_version 1912092 (0.0007) [2023-12-27 18:54:19,796][20347] Updated weights for policy 0, policy_version 1913616 (0.0007) [2023-12-27 18:54:20,385][20381] Updated weights for policy 1, policy_version 1912102 (0.0007) [2023-12-27 18:54:20,436][20381] Updated weights for policy 1, policy_version 1912112 (0.0008) [2023-12-27 18:54:20,490][20381] Updated weights for policy 1, policy_version 1912122 (0.0008) [2023-12-27 18:54:20,515][20347] Updated weights for policy 0, policy_version 1913626 (0.0009) [2023-12-27 18:54:20,563][20347] Updated weights for policy 0, policy_version 1913636 (0.0009) [2023-12-27 18:54:20,624][20347] Updated weights for policy 0, policy_version 1913646 (0.0009) [2023-12-27 18:54:20,674][20347] Updated weights for policy 0, policy_version 1913656 (0.0009) [2023-12-27 18:54:21,327][20381] Updated weights for policy 1, policy_version 1912132 (0.0008) [2023-12-27 18:54:21,395][20381] Updated weights for policy 1, policy_version 1912142 (0.0007) [2023-12-27 18:54:21,449][20381] Updated weights for policy 1, policy_version 1912152 (0.0008) [2023-12-27 18:54:21,519][20347] Updated weights for policy 0, policy_version 1913666 (0.0006) [2023-12-27 18:54:21,575][20347] Updated weights for policy 0, policy_version 1913676 (0.0007) [2023-12-27 18:54:21,643][20347] Updated weights for policy 0, policy_version 1913686 (0.0008) [2023-12-27 18:54:22,168][20381] Updated weights for policy 1, policy_version 1912162 (0.0007) [2023-12-27 18:54:22,224][20381] Updated weights for policy 1, policy_version 1912172 (0.0011) [2023-12-27 18:54:22,285][20381] Updated weights for policy 1, policy_version 1912182 (0.0011) [2023-12-27 18:54:22,352][20381] Updated weights for policy 1, policy_version 1912192 (0.0011) [2023-12-27 18:54:22,362][20347] Updated weights for policy 0, policy_version 1913696 (0.0007) [2023-12-27 18:54:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 979574784. Throughput: 0: 10161.8, 1: 10332.0. Samples: 979547340. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.980')] [2023-12-27 18:54:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001912192_489578496.pth... [2023-12-27 18:54:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001910976_489267200.pth [2023-12-27 18:54:22,423][20347] Updated weights for policy 0, policy_version 1913706 (0.0007) [2023-12-27 18:54:22,483][20347] Updated weights for policy 0, policy_version 1913716 (0.0007) [2023-12-27 18:54:22,504][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001913720_490004480.pth... [2023-12-27 18:54:22,508][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001912512_489693184.pth [2023-12-27 18:54:23,065][20381] Updated weights for policy 1, policy_version 1912202 (0.0011) [2023-12-27 18:54:23,120][20381] Updated weights for policy 1, policy_version 1912212 (0.0010) [2023-12-27 18:54:23,166][20347] Updated weights for policy 0, policy_version 1913726 (0.0007) [2023-12-27 18:54:23,174][20381] Updated weights for policy 1, policy_version 1912222 (0.0010) [2023-12-27 18:54:23,221][20347] Updated weights for policy 0, policy_version 1913736 (0.0007) [2023-12-27 18:54:23,279][20347] Updated weights for policy 0, policy_version 1913746 (0.0008) [2023-12-27 18:54:23,853][20381] Updated weights for policy 1, policy_version 1912232 (0.0008) [2023-12-27 18:54:23,920][20381] Updated weights for policy 1, policy_version 1912242 (0.0006) [2023-12-27 18:54:23,941][20347] Updated weights for policy 0, policy_version 1913756 (0.0007) [2023-12-27 18:54:23,986][20381] Updated weights for policy 1, policy_version 1912252 (0.0007) [2023-12-27 18:54:23,995][20347] Updated weights for policy 0, policy_version 1913766 (0.0007) [2023-12-27 18:54:24,043][20347] Updated weights for policy 0, policy_version 1913776 (0.0007) [2023-12-27 18:54:24,635][20347] Updated weights for policy 0, policy_version 1913786 (0.0007) [2023-12-27 18:54:24,685][20347] Updated weights for policy 0, policy_version 1913796 (0.0007) [2023-12-27 18:54:24,699][20381] Updated weights for policy 1, policy_version 1912262 (0.0006) [2023-12-27 18:54:24,738][20347] Updated weights for policy 0, policy_version 1913806 (0.0007) [2023-12-27 18:54:24,751][20381] Updated weights for policy 1, policy_version 1912272 (0.0005) [2023-12-27 18:54:24,787][20347] Updated weights for policy 0, policy_version 1913816 (0.0005) [2023-12-27 18:54:24,801][20381] Updated weights for policy 1, policy_version 1912282 (0.0007) [2023-12-27 18:54:25,429][20347] Updated weights for policy 0, policy_version 1913826 (0.0007) [2023-12-27 18:54:25,478][20347] Updated weights for policy 0, policy_version 1913836 (0.0007) [2023-12-27 18:54:25,494][20381] Updated weights for policy 1, policy_version 1912292 (0.0008) [2023-12-27 18:54:25,524][20347] Updated weights for policy 0, policy_version 1913846 (0.0006) [2023-12-27 18:54:25,548][20381] Updated weights for policy 1, policy_version 1912302 (0.0007) [2023-12-27 18:54:25,597][20381] Updated weights for policy 1, policy_version 1912312 (0.0008) [2023-12-27 18:54:26,232][20347] Updated weights for policy 0, policy_version 1913856 (0.0007) [2023-12-27 18:54:26,295][20347] Updated weights for policy 0, policy_version 1913866 (0.0008) [2023-12-27 18:54:26,325][20381] Updated weights for policy 1, policy_version 1912322 (0.0007) [2023-12-27 18:54:26,353][20347] Updated weights for policy 0, policy_version 1913876 (0.0007) [2023-12-27 18:54:26,380][20381] Updated weights for policy 1, policy_version 1912332 (0.0007) [2023-12-27 18:54:26,435][20381] Updated weights for policy 1, policy_version 1912342 (0.0008) [2023-12-27 18:54:26,494][20381] Updated weights for policy 1, policy_version 1912352 (0.0008) [2023-12-27 18:54:27,030][20347] Updated weights for policy 0, policy_version 1913886 (0.0007) [2023-12-27 18:54:27,083][20347] Updated weights for policy 0, policy_version 1913896 (0.0008) [2023-12-27 18:54:27,142][20347] Updated weights for policy 0, policy_version 1913906 (0.0009) [2023-12-27 18:54:27,188][20381] Updated weights for policy 1, policy_version 1912362 (0.0007) [2023-12-27 18:54:27,235][20381] Updated weights for policy 1, policy_version 1912372 (0.0009) [2023-12-27 18:54:27,284][20381] Updated weights for policy 1, policy_version 1912382 (0.0009) [2023-12-27 18:54:27,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 979681280. Throughput: 0: 10175.9, 1: 10230.8. Samples: 979666052. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.410')] [2023-12-27 18:54:27,776][20347] Updated weights for policy 0, policy_version 1913916 (0.0006) [2023-12-27 18:54:27,822][20347] Updated weights for policy 0, policy_version 1913926 (0.0005) [2023-12-27 18:54:27,876][20347] Updated weights for policy 0, policy_version 1913936 (0.0005) [2023-12-27 18:54:27,921][20381] Updated weights for policy 1, policy_version 1912392 (0.0006) [2023-12-27 18:54:27,978][20381] Updated weights for policy 1, policy_version 1912402 (0.0009) [2023-12-27 18:54:28,030][20381] Updated weights for policy 1, policy_version 1912412 (0.0010) [2023-12-27 18:54:28,514][20347] Updated weights for policy 0, policy_version 1913946 (0.0006) [2023-12-27 18:54:28,571][20347] Updated weights for policy 0, policy_version 1913956 (0.0007) [2023-12-27 18:54:28,625][20347] Updated weights for policy 0, policy_version 1913966 (0.0007) [2023-12-27 18:54:28,678][20347] Updated weights for policy 0, policy_version 1913976 (0.0007) [2023-12-27 18:54:28,704][20381] Updated weights for policy 1, policy_version 1912422 (0.0010) [2023-12-27 18:54:28,762][20381] Updated weights for policy 1, policy_version 1912432 (0.0011) [2023-12-27 18:54:28,811][20381] Updated weights for policy 1, policy_version 1912442 (0.0010) [2023-12-27 18:54:29,413][20347] Updated weights for policy 0, policy_version 1913986 (0.0007) [2023-12-27 18:54:29,464][20347] Updated weights for policy 0, policy_version 1913996 (0.0006) [2023-12-27 18:54:29,509][20381] Updated weights for policy 1, policy_version 1912452 (0.0009) [2023-12-27 18:54:29,514][20347] Updated weights for policy 0, policy_version 1914006 (0.0007) [2023-12-27 18:54:29,564][20381] Updated weights for policy 1, policy_version 1912462 (0.0007) [2023-12-27 18:54:29,615][20381] Updated weights for policy 1, policy_version 1912472 (0.0007) [2023-12-27 18:54:30,270][20347] Updated weights for policy 0, policy_version 1914016 (0.0007) [2023-12-27 18:54:30,281][20381] Updated weights for policy 1, policy_version 1912482 (0.0007) [2023-12-27 18:54:30,312][20347] Updated weights for policy 0, policy_version 1914026 (0.0007) [2023-12-27 18:54:30,338][20381] Updated weights for policy 1, policy_version 1912492 (0.0006) [2023-12-27 18:54:30,360][20347] Updated weights for policy 0, policy_version 1914036 (0.0007) [2023-12-27 18:54:30,389][20381] Updated weights for policy 1, policy_version 1912502 (0.0006) [2023-12-27 18:54:30,438][20381] Updated weights for policy 1, policy_version 1912512 (0.0006) [2023-12-27 18:54:31,029][20381] Updated weights for policy 1, policy_version 1912522 (0.0006) [2023-12-27 18:54:31,085][20347] Updated weights for policy 0, policy_version 1914046 (0.0007) [2023-12-27 18:54:31,093][20381] Updated weights for policy 1, policy_version 1912532 (0.0008) [2023-12-27 18:54:31,153][20381] Updated weights for policy 1, policy_version 1912542 (0.0008) [2023-12-27 18:54:31,163][20347] Updated weights for policy 0, policy_version 1914056 (0.0008) [2023-12-27 18:54:31,213][20347] Updated weights for policy 0, policy_version 1914066 (0.0007) [2023-12-27 18:54:31,922][20347] Updated weights for policy 0, policy_version 1914076 (0.0008) [2023-12-27 18:54:31,934][20381] Updated weights for policy 1, policy_version 1912552 (0.0007) [2023-12-27 18:54:31,983][20347] Updated weights for policy 0, policy_version 1914086 (0.0010) [2023-12-27 18:54:31,993][20381] Updated weights for policy 1, policy_version 1912562 (0.0007) [2023-12-27 18:54:32,042][20347] Updated weights for policy 0, policy_version 1914096 (0.0008) [2023-12-27 18:54:32,048][20381] Updated weights for policy 1, policy_version 1912572 (0.0006) [2023-12-27 18:54:32,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 979779584. Throughput: 0: 10110.1, 1: 10243.0. Samples: 979787772. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.880')] [2023-12-27 18:54:32,671][20381] Updated weights for policy 1, policy_version 1912582 (0.0007) [2023-12-27 18:54:32,726][20381] Updated weights for policy 1, policy_version 1912592 (0.0007) [2023-12-27 18:54:32,728][20347] Updated weights for policy 0, policy_version 1914106 (0.0007) [2023-12-27 18:54:32,778][20381] Updated weights for policy 1, policy_version 1912602 (0.0005) [2023-12-27 18:54:32,780][20347] Updated weights for policy 0, policy_version 1914116 (0.0010) [2023-12-27 18:54:32,835][20347] Updated weights for policy 0, policy_version 1914126 (0.0010) [2023-12-27 18:54:32,892][20347] Updated weights for policy 0, policy_version 1914136 (0.0008) [2023-12-27 18:54:33,416][20381] Updated weights for policy 1, policy_version 1912612 (0.0005) [2023-12-27 18:54:33,464][20381] Updated weights for policy 1, policy_version 1912622 (0.0007) [2023-12-27 18:54:33,513][20381] Updated weights for policy 1, policy_version 1912632 (0.0007) [2023-12-27 18:54:33,582][20347] Updated weights for policy 0, policy_version 1914146 (0.0010) [2023-12-27 18:54:33,636][20347] Updated weights for policy 0, policy_version 1914156 (0.0010) [2023-12-27 18:54:33,688][20347] Updated weights for policy 0, policy_version 1914166 (0.0009) [2023-12-27 18:54:34,238][20381] Updated weights for policy 1, policy_version 1912642 (0.0008) [2023-12-27 18:54:34,288][20381] Updated weights for policy 1, policy_version 1912652 (0.0007) [2023-12-27 18:54:34,341][20381] Updated weights for policy 1, policy_version 1912662 (0.0007) [2023-12-27 18:54:34,356][20347] Updated weights for policy 0, policy_version 1914176 (0.0008) [2023-12-27 18:54:34,396][20381] Updated weights for policy 1, policy_version 1912672 (0.0006) [2023-12-27 18:54:34,414][20347] Updated weights for policy 0, policy_version 1914186 (0.0008) [2023-12-27 18:54:34,477][20347] Updated weights for policy 0, policy_version 1914196 (0.0008) [2023-12-27 18:54:35,050][20381] Updated weights for policy 1, policy_version 1912682 (0.0009) [2023-12-27 18:54:35,106][20381] Updated weights for policy 1, policy_version 1912692 (0.0008) [2023-12-27 18:54:35,143][20347] Updated weights for policy 0, policy_version 1914206 (0.0006) [2023-12-27 18:54:35,152][20381] Updated weights for policy 1, policy_version 1912702 (0.0008) [2023-12-27 18:54:35,199][20347] Updated weights for policy 0, policy_version 1914216 (0.0007) [2023-12-27 18:54:35,256][20347] Updated weights for policy 0, policy_version 1914226 (0.0007) [2023-12-27 18:54:35,788][20347] Updated weights for policy 0, policy_version 1914236 (0.0005) [2023-12-27 18:54:35,841][20347] Updated weights for policy 0, policy_version 1914246 (0.0005) [2023-12-27 18:54:35,904][20347] Updated weights for policy 0, policy_version 1914256 (0.0005) [2023-12-27 18:54:35,914][20381] Updated weights for policy 1, policy_version 1912712 (0.0006) [2023-12-27 18:54:35,978][20381] Updated weights for policy 1, policy_version 1912722 (0.0008) [2023-12-27 18:54:36,023][20381] Updated weights for policy 1, policy_version 1912732 (0.0005) [2023-12-27 18:54:36,567][20347] Updated weights for policy 0, policy_version 1914266 (0.0006) [2023-12-27 18:54:36,624][20347] Updated weights for policy 0, policy_version 1914276 (0.0008) [2023-12-27 18:54:36,676][20347] Updated weights for policy 0, policy_version 1914286 (0.0010) [2023-12-27 18:54:36,704][20381] Updated weights for policy 1, policy_version 1912742 (0.0005) [2023-12-27 18:54:36,726][20347] Updated weights for policy 0, policy_version 1914296 (0.0010) [2023-12-27 18:54:36,762][20381] Updated weights for policy 1, policy_version 1912752 (0.0005) [2023-12-27 18:54:36,820][20381] Updated weights for policy 1, policy_version 1912762 (0.0007) [2023-12-27 18:54:37,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 979877888. Throughput: 0: 10145.5, 1: 10219.1. Samples: 979849916. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.560')] [2023-12-27 18:54:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001912768_489725952.pth... [2023-12-27 18:54:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001914296_490151936.pth... [2023-12-27 18:54:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001911616_489431040.pth [2023-12-27 18:54:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001913120_489848832.pth [2023-12-27 18:54:37,483][20347] Updated weights for policy 0, policy_version 1914306 (0.0011) [2023-12-27 18:54:37,492][20381] Updated weights for policy 1, policy_version 1912772 (0.0006) [2023-12-27 18:54:37,545][20347] Updated weights for policy 0, policy_version 1914316 (0.0010) [2023-12-27 18:54:37,554][20381] Updated weights for policy 1, policy_version 1912782 (0.0006) [2023-12-27 18:54:37,606][20381] Updated weights for policy 1, policy_version 1912792 (0.0006) [2023-12-27 18:54:37,610][20347] Updated weights for policy 0, policy_version 1914326 (0.0011) [2023-12-27 18:54:38,180][20347] Updated weights for policy 0, policy_version 1914336 (0.0006) [2023-12-27 18:54:38,237][20347] Updated weights for policy 0, policy_version 1914346 (0.0007) [2023-12-27 18:54:38,295][20347] Updated weights for policy 0, policy_version 1914356 (0.0008) [2023-12-27 18:54:38,334][20381] Updated weights for policy 1, policy_version 1912802 (0.0008) [2023-12-27 18:54:38,396][20381] Updated weights for policy 1, policy_version 1912812 (0.0006) [2023-12-27 18:54:38,459][20381] Updated weights for policy 1, policy_version 1912822 (0.0008) [2023-12-27 18:54:38,523][20381] Updated weights for policy 1, policy_version 1912832 (0.0008) [2023-12-27 18:54:38,967][20347] Updated weights for policy 0, policy_version 1914366 (0.0007) [2023-12-27 18:54:39,029][20347] Updated weights for policy 0, policy_version 1914376 (0.0007) [2023-12-27 18:54:39,086][20347] Updated weights for policy 0, policy_version 1914386 (0.0008) [2023-12-27 18:54:39,260][20381] Updated weights for policy 1, policy_version 1912842 (0.0007) [2023-12-27 18:54:39,307][20381] Updated weights for policy 1, policy_version 1912852 (0.0005) [2023-12-27 18:54:39,366][20381] Updated weights for policy 1, policy_version 1912862 (0.0007) [2023-12-27 18:54:39,847][20347] Updated weights for policy 0, policy_version 1914396 (0.0009) [2023-12-27 18:54:39,913][20347] Updated weights for policy 0, policy_version 1914406 (0.0007) [2023-12-27 18:54:39,983][20347] Updated weights for policy 0, policy_version 1914416 (0.0008) [2023-12-27 18:54:40,029][20381] Updated weights for policy 1, policy_version 1912872 (0.0007) [2023-12-27 18:54:40,098][20381] Updated weights for policy 1, policy_version 1912882 (0.0006) [2023-12-27 18:54:40,164][20381] Updated weights for policy 1, policy_version 1912892 (0.0005) [2023-12-27 18:54:40,622][20347] Updated weights for policy 0, policy_version 1914426 (0.0008) [2023-12-27 18:54:40,691][20347] Updated weights for policy 0, policy_version 1914436 (0.0006) [2023-12-27 18:54:40,750][20347] Updated weights for policy 0, policy_version 1914446 (0.0006) [2023-12-27 18:54:40,806][20347] Updated weights for policy 0, policy_version 1914456 (0.0006) [2023-12-27 18:54:40,874][20381] Updated weights for policy 1, policy_version 1912902 (0.0007) [2023-12-27 18:54:40,927][20381] Updated weights for policy 1, policy_version 1912912 (0.0007) [2023-12-27 18:54:40,981][20381] Updated weights for policy 1, policy_version 1912922 (0.0009) [2023-12-27 18:54:41,417][20347] Updated weights for policy 0, policy_version 1914466 (0.0009) [2023-12-27 18:54:41,466][20347] Updated weights for policy 0, policy_version 1914476 (0.0008) [2023-12-27 18:54:41,523][20347] Updated weights for policy 0, policy_version 1914486 (0.0008) [2023-12-27 18:54:41,753][20381] Updated weights for policy 1, policy_version 1912932 (0.0008) [2023-12-27 18:54:41,805][20381] Updated weights for policy 1, policy_version 1912942 (0.0008) [2023-12-27 18:54:41,859][20381] Updated weights for policy 1, policy_version 1912952 (0.0008) [2023-12-27 18:54:42,236][20347] Updated weights for policy 0, policy_version 1914496 (0.0008) [2023-12-27 18:54:42,299][20347] Updated weights for policy 0, policy_version 1914506 (0.0008) [2023-12-27 18:54:42,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 979976192. Throughput: 0: 10254.8, 1: 10103.5. Samples: 979970928. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.290')] [2023-12-27 18:54:42,364][20347] Updated weights for policy 0, policy_version 1914516 (0.0007) [2023-12-27 18:54:42,598][20381] Updated weights for policy 1, policy_version 1912962 (0.0008) [2023-12-27 18:54:42,655][20381] Updated weights for policy 1, policy_version 1912972 (0.0009) [2023-12-27 18:54:42,714][20381] Updated weights for policy 1, policy_version 1912982 (0.0006) [2023-12-27 18:54:42,775][20381] Updated weights for policy 1, policy_version 1912992 (0.0011) [2023-12-27 18:54:43,040][20347] Updated weights for policy 0, policy_version 1914526 (0.0007) [2023-12-27 18:54:43,099][20347] Updated weights for policy 0, policy_version 1914536 (0.0008) [2023-12-27 18:54:43,158][20347] Updated weights for policy 0, policy_version 1914546 (0.0008) [2023-12-27 18:54:43,381][20381] Updated weights for policy 1, policy_version 1913002 (0.0007) [2023-12-27 18:54:43,434][20381] Updated weights for policy 1, policy_version 1913012 (0.0008) [2023-12-27 18:54:43,493][20381] Updated weights for policy 1, policy_version 1913022 (0.0008) [2023-12-27 18:54:43,829][20347] Updated weights for policy 0, policy_version 1914556 (0.0008) [2023-12-27 18:54:43,882][20347] Updated weights for policy 0, policy_version 1914566 (0.0007) [2023-12-27 18:54:43,933][20347] Updated weights for policy 0, policy_version 1914576 (0.0006) [2023-12-27 18:54:44,150][20381] Updated weights for policy 1, policy_version 1913032 (0.0008) [2023-12-27 18:54:44,202][20381] Updated weights for policy 1, policy_version 1913042 (0.0007) [2023-12-27 18:54:44,262][20381] Updated weights for policy 1, policy_version 1913052 (0.0007) [2023-12-27 18:54:44,606][20347] Updated weights for policy 0, policy_version 1914586 (0.0006) [2023-12-27 18:54:44,661][20347] Updated weights for policy 0, policy_version 1914596 (0.0009) [2023-12-27 18:54:44,712][20347] Updated weights for policy 0, policy_version 1914606 (0.0007) [2023-12-27 18:54:44,761][20347] Updated weights for policy 0, policy_version 1914616 (0.0007) [2023-12-27 18:54:44,949][20381] Updated weights for policy 1, policy_version 1913062 (0.0009) [2023-12-27 18:54:45,012][20381] Updated weights for policy 1, policy_version 1913072 (0.0011) [2023-12-27 18:54:45,074][20381] Updated weights for policy 1, policy_version 1913082 (0.0010) [2023-12-27 18:54:45,447][20347] Updated weights for policy 0, policy_version 1914626 (0.0005) [2023-12-27 18:54:45,491][20347] Updated weights for policy 0, policy_version 1914636 (0.0007) [2023-12-27 18:54:45,542][20347] Updated weights for policy 0, policy_version 1914646 (0.0007) [2023-12-27 18:54:45,741][20381] Updated weights for policy 1, policy_version 1913092 (0.0011) [2023-12-27 18:54:45,789][20381] Updated weights for policy 1, policy_version 1913102 (0.0010) [2023-12-27 18:54:45,837][20381] Updated weights for policy 1, policy_version 1913112 (0.0010) [2023-12-27 18:54:46,240][20347] Updated weights for policy 0, policy_version 1914656 (0.0008) [2023-12-27 18:54:46,292][20347] Updated weights for policy 0, policy_version 1914666 (0.0008) [2023-12-27 18:54:46,347][20347] Updated weights for policy 0, policy_version 1914676 (0.0007) [2023-12-27 18:54:46,522][20381] Updated weights for policy 1, policy_version 1913122 (0.0010) [2023-12-27 18:54:46,580][20381] Updated weights for policy 1, policy_version 1913132 (0.0010) [2023-12-27 18:54:46,635][20381] Updated weights for policy 1, policy_version 1913142 (0.0010) [2023-12-27 18:54:46,689][20381] Updated weights for policy 1, policy_version 1913152 (0.0010) [2023-12-27 18:54:47,051][20347] Updated weights for policy 0, policy_version 1914686 (0.0007) [2023-12-27 18:54:47,109][20347] Updated weights for policy 0, policy_version 1914696 (0.0007) [2023-12-27 18:54:47,157][20347] Updated weights for policy 0, policy_version 1914706 (0.0005) [2023-12-27 18:54:47,311][20381] Updated weights for policy 1, policy_version 1913162 (0.0010) [2023-12-27 18:54:47,361][20381] Updated weights for policy 1, policy_version 1913172 (0.0010) [2023-12-27 18:54:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 980082688. Throughput: 0: 10290.5, 1: 10083.0. Samples: 980093296. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.230')] [2023-12-27 18:54:47,410][20381] Updated weights for policy 1, policy_version 1913182 (0.0008) [2023-12-27 18:54:47,778][20347] Updated weights for policy 0, policy_version 1914716 (0.0008) [2023-12-27 18:54:47,830][20347] Updated weights for policy 0, policy_version 1914726 (0.0008) [2023-12-27 18:54:47,881][20347] Updated weights for policy 0, policy_version 1914736 (0.0008) [2023-12-27 18:54:48,163][20381] Updated weights for policy 1, policy_version 1913192 (0.0008) [2023-12-27 18:54:48,225][20381] Updated weights for policy 1, policy_version 1913202 (0.0008) [2023-12-27 18:54:48,285][20381] Updated weights for policy 1, policy_version 1913212 (0.0008) [2023-12-27 18:54:48,560][20347] Updated weights for policy 0, policy_version 1914746 (0.0008) [2023-12-27 18:54:48,619][20347] Updated weights for policy 0, policy_version 1914756 (0.0007) [2023-12-27 18:54:48,681][20347] Updated weights for policy 0, policy_version 1914766 (0.0008) [2023-12-27 18:54:48,743][20347] Updated weights for policy 0, policy_version 1914776 (0.0009) [2023-12-27 18:54:48,936][20381] Updated weights for policy 1, policy_version 1913222 (0.0007) [2023-12-27 18:54:48,985][20381] Updated weights for policy 1, policy_version 1913232 (0.0007) [2023-12-27 18:54:49,041][20381] Updated weights for policy 1, policy_version 1913242 (0.0008) [2023-12-27 18:54:49,449][20347] Updated weights for policy 0, policy_version 1914786 (0.0007) [2023-12-27 18:54:49,508][20347] Updated weights for policy 0, policy_version 1914796 (0.0007) [2023-12-27 18:54:49,561][20347] Updated weights for policy 0, policy_version 1914806 (0.0009) [2023-12-27 18:54:49,807][20381] Updated weights for policy 1, policy_version 1913252 (0.0007) [2023-12-27 18:54:49,876][20381] Updated weights for policy 1, policy_version 1913262 (0.0007) [2023-12-27 18:54:49,950][20381] Updated weights for policy 1, policy_version 1913272 (0.0007) [2023-12-27 18:54:50,253][20347] Updated weights for policy 0, policy_version 1914816 (0.0006) [2023-12-27 18:54:50,300][20347] Updated weights for policy 0, policy_version 1914826 (0.0008) [2023-12-27 18:54:50,352][20347] Updated weights for policy 0, policy_version 1914836 (0.0007) [2023-12-27 18:54:50,552][20381] Updated weights for policy 1, policy_version 1913282 (0.0008) [2023-12-27 18:54:50,613][20381] Updated weights for policy 1, policy_version 1913292 (0.0007) [2023-12-27 18:54:50,679][20381] Updated weights for policy 1, policy_version 1913302 (0.0007) [2023-12-27 18:54:50,739][20381] Updated weights for policy 1, policy_version 1913312 (0.0008) [2023-12-27 18:54:51,039][20347] Updated weights for policy 0, policy_version 1914846 (0.0008) [2023-12-27 18:54:51,098][20347] Updated weights for policy 0, policy_version 1914856 (0.0007) [2023-12-27 18:54:51,164][20347] Updated weights for policy 0, policy_version 1914866 (0.0007) [2023-12-27 18:54:51,466][20381] Updated weights for policy 1, policy_version 1913322 (0.0008) [2023-12-27 18:54:51,516][20381] Updated weights for policy 1, policy_version 1913332 (0.0007) [2023-12-27 18:54:51,567][20381] Updated weights for policy 1, policy_version 1913342 (0.0009) [2023-12-27 18:54:51,926][20347] Updated weights for policy 0, policy_version 1914876 (0.0009) [2023-12-27 18:54:51,982][20347] Updated weights for policy 0, policy_version 1914886 (0.0009) [2023-12-27 18:54:52,040][20347] Updated weights for policy 0, policy_version 1914896 (0.0006) [2023-12-27 18:54:52,338][20381] Updated weights for policy 1, policy_version 1913352 (0.0009) [2023-12-27 18:54:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 980180992. Throughput: 0: 10274.8, 1: 10018.3. Samples: 980153644. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.580')] [2023-12-27 18:54:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001914904_490307584.pth... [2023-12-27 18:54:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001913720_490004480.pth [2023-12-27 18:54:52,400][20381] Updated weights for policy 1, policy_version 1913362 (0.0007) [2023-12-27 18:54:52,452][20381] Updated weights for policy 1, policy_version 1913372 (0.0009) [2023-12-27 18:54:52,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001913376_489881600.pth... [2023-12-27 18:54:52,475][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001912192_489578496.pth [2023-12-27 18:54:52,635][20347] Updated weights for policy 0, policy_version 1914906 (0.0006) [2023-12-27 18:54:52,698][20347] Updated weights for policy 0, policy_version 1914916 (0.0006) [2023-12-27 18:54:52,757][20347] Updated weights for policy 0, policy_version 1914926 (0.0008) [2023-12-27 18:54:52,815][20347] Updated weights for policy 0, policy_version 1914936 (0.0007) [2023-12-27 18:54:53,150][20381] Updated weights for policy 1, policy_version 1913382 (0.0009) [2023-12-27 18:54:53,207][20381] Updated weights for policy 1, policy_version 1913392 (0.0008) [2023-12-27 18:54:53,269][20381] Updated weights for policy 1, policy_version 1913402 (0.0006) [2023-12-27 18:54:53,378][20347] Updated weights for policy 0, policy_version 1914946 (0.0007) [2023-12-27 18:54:53,426][20347] Updated weights for policy 0, policy_version 1914956 (0.0007) [2023-12-27 18:54:53,486][20347] Updated weights for policy 0, policy_version 1914966 (0.0007) [2023-12-27 18:54:53,947][20381] Updated weights for policy 1, policy_version 1913412 (0.0009) [2023-12-27 18:54:54,002][20381] Updated weights for policy 1, policy_version 1913422 (0.0010) [2023-12-27 18:54:54,060][20381] Updated weights for policy 1, policy_version 1913432 (0.0010) [2023-12-27 18:54:54,179][20347] Updated weights for policy 0, policy_version 1914976 (0.0010) [2023-12-27 18:54:54,233][20347] Updated weights for policy 0, policy_version 1914986 (0.0010) [2023-12-27 18:54:54,284][20347] Updated weights for policy 0, policy_version 1914996 (0.0006) [2023-12-27 18:54:54,724][20381] Updated weights for policy 1, policy_version 1913442 (0.0010) [2023-12-27 18:54:54,776][20381] Updated weights for policy 1, policy_version 1913452 (0.0006) [2023-12-27 18:54:54,832][20381] Updated weights for policy 1, policy_version 1913462 (0.0005) [2023-12-27 18:54:54,873][20347] Updated weights for policy 0, policy_version 1915006 (0.0006) [2023-12-27 18:54:54,889][20381] Updated weights for policy 1, policy_version 1913472 (0.0006) [2023-12-27 18:54:54,932][20347] Updated weights for policy 0, policy_version 1915016 (0.0008) [2023-12-27 18:54:54,993][20347] Updated weights for policy 0, policy_version 1915026 (0.0009) [2023-12-27 18:54:55,478][20381] Updated weights for policy 1, policy_version 1913482 (0.0008) [2023-12-27 18:54:55,533][20381] Updated weights for policy 1, policy_version 1913492 (0.0008) [2023-12-27 18:54:55,587][20381] Updated weights for policy 1, policy_version 1913502 (0.0008) [2023-12-27 18:54:55,694][20347] Updated weights for policy 0, policy_version 1915036 (0.0008) [2023-12-27 18:54:55,740][20347] Updated weights for policy 0, policy_version 1915046 (0.0007) [2023-12-27 18:54:55,785][20347] Updated weights for policy 0, policy_version 1915056 (0.0008) [2023-12-27 18:54:56,268][20381] Updated weights for policy 1, policy_version 1913512 (0.0006) [2023-12-27 18:54:56,326][20381] Updated weights for policy 1, policy_version 1913522 (0.0010) [2023-12-27 18:54:56,389][20381] Updated weights for policy 1, policy_version 1913532 (0.0010) [2023-12-27 18:54:56,477][20347] Updated weights for policy 0, policy_version 1915066 (0.0007) [2023-12-27 18:54:56,533][20347] Updated weights for policy 0, policy_version 1915076 (0.0008) [2023-12-27 18:54:56,591][20347] Updated weights for policy 0, policy_version 1915086 (0.0007) [2023-12-27 18:54:56,642][20347] Updated weights for policy 0, policy_version 1915096 (0.0007) [2023-12-27 18:54:57,060][20381] Updated weights for policy 1, policy_version 1913542 (0.0009) [2023-12-27 18:54:57,110][20381] Updated weights for policy 1, policy_version 1913552 (0.0006) [2023-12-27 18:54:57,166][20381] Updated weights for policy 1, policy_version 1913562 (0.0005) [2023-12-27 18:54:57,346][20347] Updated weights for policy 0, policy_version 1915106 (0.0007) [2023-12-27 18:54:57,363][19170] Fps is (10 sec: 20479.5, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 980287488. Throughput: 0: 10259.0, 1: 10079.8. Samples: 980277628. Policy #0 lag: (min: 22.0, avg: 36.1, max: 54.0) [2023-12-27 18:54:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '931.530')] [2023-12-27 18:54:57,402][20347] Updated weights for policy 0, policy_version 1915116 (0.0007) [2023-12-27 18:54:57,449][20347] Updated weights for policy 0, policy_version 1915126 (0.0007) [2023-12-27 18:54:57,771][20381] Updated weights for policy 1, policy_version 1913572 (0.0007) [2023-12-27 18:54:57,823][20381] Updated weights for policy 1, policy_version 1913582 (0.0009) [2023-12-27 18:54:57,869][20381] Updated weights for policy 1, policy_version 1913592 (0.0009) [2023-12-27 18:54:58,005][20347] Updated weights for policy 0, policy_version 1915136 (0.0006) [2023-12-27 18:54:58,066][20347] Updated weights for policy 0, policy_version 1915146 (0.0005) [2023-12-27 18:54:58,125][20347] Updated weights for policy 0, policy_version 1915156 (0.0007) [2023-12-27 18:54:58,598][20381] Updated weights for policy 1, policy_version 1913602 (0.0009) [2023-12-27 18:54:58,665][20381] Updated weights for policy 1, policy_version 1913612 (0.0008) [2023-12-27 18:54:58,724][20381] Updated weights for policy 1, policy_version 1913622 (0.0008) [2023-12-27 18:54:58,792][20381] Updated weights for policy 1, policy_version 1913632 (0.0008) [2023-12-27 18:54:58,830][20347] Updated weights for policy 0, policy_version 1915166 (0.0008) [2023-12-27 18:54:58,900][20347] Updated weights for policy 0, policy_version 1915176 (0.0008) [2023-12-27 18:54:58,960][20347] Updated weights for policy 0, policy_version 1915186 (0.0008) [2023-12-27 18:54:59,578][20381] Updated weights for policy 1, policy_version 1913642 (0.0008) [2023-12-27 18:54:59,621][20347] Updated weights for policy 0, policy_version 1915196 (0.0007) [2023-12-27 18:54:59,631][20381] Updated weights for policy 1, policy_version 1913652 (0.0007) [2023-12-27 18:54:59,678][20347] Updated weights for policy 0, policy_version 1915206 (0.0007) [2023-12-27 18:54:59,684][20381] Updated weights for policy 1, policy_version 1913662 (0.0005) [2023-12-27 18:54:59,725][20347] Updated weights for policy 0, policy_version 1915216 (0.0006) [2023-12-27 18:55:00,409][20381] Updated weights for policy 1, policy_version 1913672 (0.0008) [2023-12-27 18:55:00,455][20381] Updated weights for policy 1, policy_version 1913682 (0.0007) [2023-12-27 18:55:00,480][20347] Updated weights for policy 0, policy_version 1915226 (0.0007) [2023-12-27 18:55:00,515][20381] Updated weights for policy 1, policy_version 1913692 (0.0006) [2023-12-27 18:55:00,534][20347] Updated weights for policy 0, policy_version 1915236 (0.0007) [2023-12-27 18:55:00,593][20347] Updated weights for policy 0, policy_version 1915246 (0.0006) [2023-12-27 18:55:00,646][20347] Updated weights for policy 0, policy_version 1915256 (0.0005) [2023-12-27 18:55:01,171][20381] Updated weights for policy 1, policy_version 1913702 (0.0007) [2023-12-27 18:55:01,236][20381] Updated weights for policy 1, policy_version 1913712 (0.0009) [2023-12-27 18:55:01,290][20381] Updated weights for policy 1, policy_version 1913722 (0.0009) [2023-12-27 18:55:01,351][20347] Updated weights for policy 0, policy_version 1915266 (0.0008) [2023-12-27 18:55:01,417][20347] Updated weights for policy 0, policy_version 1915276 (0.0008) [2023-12-27 18:55:01,474][20347] Updated weights for policy 0, policy_version 1915286 (0.0008) [2023-12-27 18:55:02,014][20381] Updated weights for policy 1, policy_version 1913732 (0.0010) [2023-12-27 18:55:02,069][20381] Updated weights for policy 1, policy_version 1913742 (0.0010) [2023-12-27 18:55:02,129][20381] Updated weights for policy 1, policy_version 1913752 (0.0010) [2023-12-27 18:55:02,172][20347] Updated weights for policy 0, policy_version 1915296 (0.0006) [2023-12-27 18:55:02,226][20347] Updated weights for policy 0, policy_version 1915306 (0.0007) [2023-12-27 18:55:02,283][20347] Updated weights for policy 0, policy_version 1915316 (0.0007) [2023-12-27 18:55:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 980393984. Throughput: 0: 10246.9, 1: 10032.6. Samples: 980398572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.880')] [2023-12-27 18:55:02,803][20381] Updated weights for policy 1, policy_version 1913762 (0.0009) [2023-12-27 18:55:02,854][20381] Updated weights for policy 1, policy_version 1913772 (0.0010) [2023-12-27 18:55:02,902][20381] Updated weights for policy 1, policy_version 1913782 (0.0010) [2023-12-27 18:55:02,953][20381] Updated weights for policy 1, policy_version 1913792 (0.0010) [2023-12-27 18:55:02,987][20347] Updated weights for policy 0, policy_version 1915326 (0.0007) [2023-12-27 18:55:03,036][20347] Updated weights for policy 0, policy_version 1915336 (0.0007) [2023-12-27 18:55:03,083][20347] Updated weights for policy 0, policy_version 1915346 (0.0007) [2023-12-27 18:55:03,593][20381] Updated weights for policy 1, policy_version 1913802 (0.0007) [2023-12-27 18:55:03,637][20381] Updated weights for policy 1, policy_version 1913812 (0.0008) [2023-12-27 18:55:03,682][20381] Updated weights for policy 1, policy_version 1913822 (0.0008) [2023-12-27 18:55:03,770][20347] Updated weights for policy 0, policy_version 1915356 (0.0006) [2023-12-27 18:55:03,827][20347] Updated weights for policy 0, policy_version 1915366 (0.0005) [2023-12-27 18:55:03,883][20347] Updated weights for policy 0, policy_version 1915376 (0.0008) [2023-12-27 18:55:04,441][20381] Updated weights for policy 1, policy_version 1913832 (0.0008) [2023-12-27 18:55:04,495][20381] Updated weights for policy 1, policy_version 1913842 (0.0007) [2023-12-27 18:55:04,550][20381] Updated weights for policy 1, policy_version 1913852 (0.0007) [2023-12-27 18:55:04,556][20347] Updated weights for policy 0, policy_version 1915386 (0.0011) [2023-12-27 18:55:04,612][20347] Updated weights for policy 0, policy_version 1915396 (0.0010) [2023-12-27 18:55:04,671][20347] Updated weights for policy 0, policy_version 1915406 (0.0011) [2023-12-27 18:55:04,736][20347] Updated weights for policy 0, policy_version 1915416 (0.0012) [2023-12-27 18:55:05,233][20381] Updated weights for policy 1, policy_version 1913862 (0.0006) [2023-12-27 18:55:05,287][20381] Updated weights for policy 1, policy_version 1913872 (0.0007) [2023-12-27 18:55:05,344][20381] Updated weights for policy 1, policy_version 1913882 (0.0007) [2023-12-27 18:55:05,396][20347] Updated weights for policy 0, policy_version 1915426 (0.0010) [2023-12-27 18:55:05,451][20347] Updated weights for policy 0, policy_version 1915436 (0.0010) [2023-12-27 18:55:05,507][20347] Updated weights for policy 0, policy_version 1915446 (0.0010) [2023-12-27 18:55:05,963][20381] Updated weights for policy 1, policy_version 1913892 (0.0005) [2023-12-27 18:55:06,030][20381] Updated weights for policy 1, policy_version 1913902 (0.0006) [2023-12-27 18:55:06,082][20381] Updated weights for policy 1, policy_version 1913912 (0.0007) [2023-12-27 18:55:06,137][20347] Updated weights for policy 0, policy_version 1915456 (0.0009) [2023-12-27 18:55:06,195][20347] Updated weights for policy 0, policy_version 1915466 (0.0007) [2023-12-27 18:55:06,252][20347] Updated weights for policy 0, policy_version 1915476 (0.0008) [2023-12-27 18:55:06,761][20381] Updated weights for policy 1, policy_version 1913922 (0.0008) [2023-12-27 18:55:06,811][20381] Updated weights for policy 1, policy_version 1913932 (0.0008) [2023-12-27 18:55:06,869][20381] Updated weights for policy 1, policy_version 1913942 (0.0007) [2023-12-27 18:55:06,928][20381] Updated weights for policy 1, policy_version 1913952 (0.0008) [2023-12-27 18:55:06,959][20347] Updated weights for policy 0, policy_version 1915486 (0.0009) [2023-12-27 18:55:07,008][20347] Updated weights for policy 0, policy_version 1915496 (0.0009) [2023-12-27 18:55:07,054][20347] Updated weights for policy 0, policy_version 1915506 (0.0010) [2023-12-27 18:55:07,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.4, 300 sec: 20493.9). Total num frames: 980492288. Throughput: 0: 10228.4, 1: 10051.6. Samples: 980459944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.520')] [2023-12-27 18:55:07,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001915512_490463232.pth... [2023-12-27 18:55:07,372][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001913952_490029056.pth... [2023-12-27 18:55:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001914296_490151936.pth [2023-12-27 18:55:07,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001912768_489725952.pth [2023-12-27 18:55:07,645][20381] Updated weights for policy 1, policy_version 1913962 (0.0008) [2023-12-27 18:55:07,694][20381] Updated weights for policy 1, policy_version 1913972 (0.0008) [2023-12-27 18:55:07,721][20347] Updated weights for policy 0, policy_version 1915516 (0.0008) [2023-12-27 18:55:07,752][20381] Updated weights for policy 1, policy_version 1913982 (0.0007) [2023-12-27 18:55:07,773][20347] Updated weights for policy 0, policy_version 1915526 (0.0006) [2023-12-27 18:55:07,825][20347] Updated weights for policy 0, policy_version 1915536 (0.0008) [2023-12-27 18:55:08,388][20381] Updated weights for policy 1, policy_version 1913992 (0.0010) [2023-12-27 18:55:08,444][20381] Updated weights for policy 1, policy_version 1914002 (0.0006) [2023-12-27 18:55:08,499][20381] Updated weights for policy 1, policy_version 1914012 (0.0005) [2023-12-27 18:55:08,542][20347] Updated weights for policy 0, policy_version 1915546 (0.0007) [2023-12-27 18:55:08,602][20347] Updated weights for policy 0, policy_version 1915556 (0.0007) [2023-12-27 18:55:08,664][20347] Updated weights for policy 0, policy_version 1915566 (0.0011) [2023-12-27 18:55:08,726][20347] Updated weights for policy 0, policy_version 1915576 (0.0012) [2023-12-27 18:55:09,109][20381] Updated weights for policy 1, policy_version 1914022 (0.0005) [2023-12-27 18:55:09,157][20381] Updated weights for policy 1, policy_version 1914032 (0.0005) [2023-12-27 18:55:09,212][20381] Updated weights for policy 1, policy_version 1914042 (0.0005) [2023-12-27 18:55:09,415][20347] Updated weights for policy 0, policy_version 1915586 (0.0009) [2023-12-27 18:55:09,474][20347] Updated weights for policy 0, policy_version 1915596 (0.0008) [2023-12-27 18:55:09,529][20347] Updated weights for policy 0, policy_version 1915606 (0.0007) [2023-12-27 18:55:09,980][20381] Updated weights for policy 1, policy_version 1914052 (0.0008) [2023-12-27 18:55:10,029][20381] Updated weights for policy 1, policy_version 1914062 (0.0008) [2023-12-27 18:55:10,081][20381] Updated weights for policy 1, policy_version 1914072 (0.0010) [2023-12-27 18:55:10,203][20347] Updated weights for policy 0, policy_version 1915616 (0.0006) [2023-12-27 18:55:10,263][20347] Updated weights for policy 0, policy_version 1915626 (0.0007) [2023-12-27 18:55:10,320][20347] Updated weights for policy 0, policy_version 1915636 (0.0007) [2023-12-27 18:55:10,758][20381] Updated weights for policy 1, policy_version 1914082 (0.0009) [2023-12-27 18:55:10,804][20381] Updated weights for policy 1, policy_version 1914092 (0.0007) [2023-12-27 18:55:10,861][20381] Updated weights for policy 1, policy_version 1914102 (0.0008) [2023-12-27 18:55:10,922][20381] Updated weights for policy 1, policy_version 1914112 (0.0008) [2023-12-27 18:55:11,004][20347] Updated weights for policy 0, policy_version 1915646 (0.0008) [2023-12-27 18:55:11,071][20347] Updated weights for policy 0, policy_version 1915656 (0.0007) [2023-12-27 18:55:11,136][20347] Updated weights for policy 0, policy_version 1915666 (0.0006) [2023-12-27 18:55:11,622][20381] Updated weights for policy 1, policy_version 1914122 (0.0007) [2023-12-27 18:55:11,691][20381] Updated weights for policy 1, policy_version 1914132 (0.0008) [2023-12-27 18:55:11,757][20381] Updated weights for policy 1, policy_version 1914142 (0.0006) [2023-12-27 18:55:11,953][20347] Updated weights for policy 0, policy_version 1915676 (0.0007) [2023-12-27 18:55:12,009][20347] Updated weights for policy 0, policy_version 1915686 (0.0008) [2023-12-27 18:55:12,057][20347] Updated weights for policy 0, policy_version 1915696 (0.0008) [2023-12-27 18:55:12,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 980590592. Throughput: 0: 10238.2, 1: 10134.2. Samples: 980582808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.830')] [2023-12-27 18:55:12,396][20381] Updated weights for policy 1, policy_version 1914152 (0.0006) [2023-12-27 18:55:12,460][20381] Updated weights for policy 1, policy_version 1914162 (0.0008) [2023-12-27 18:55:12,522][20381] Updated weights for policy 1, policy_version 1914172 (0.0006) [2023-12-27 18:55:12,762][20347] Updated weights for policy 0, policy_version 1915706 (0.0008) [2023-12-27 18:55:12,821][20347] Updated weights for policy 0, policy_version 1915716 (0.0008) [2023-12-27 18:55:12,876][20347] Updated weights for policy 0, policy_version 1915726 (0.0008) [2023-12-27 18:55:12,928][20347] Updated weights for policy 0, policy_version 1915736 (0.0008) [2023-12-27 18:55:13,157][20381] Updated weights for policy 1, policy_version 1914182 (0.0006) [2023-12-27 18:55:13,203][20381] Updated weights for policy 1, policy_version 1914192 (0.0005) [2023-12-27 18:55:13,254][20381] Updated weights for policy 1, policy_version 1914202 (0.0005) [2023-12-27 18:55:13,627][20347] Updated weights for policy 0, policy_version 1915746 (0.0008) [2023-12-27 18:55:13,669][20347] Updated weights for policy 0, policy_version 1915756 (0.0007) [2023-12-27 18:55:13,714][20347] Updated weights for policy 0, policy_version 1915766 (0.0008) [2023-12-27 18:55:13,847][20381] Updated weights for policy 1, policy_version 1914212 (0.0007) [2023-12-27 18:55:13,913][20381] Updated weights for policy 1, policy_version 1914222 (0.0008) [2023-12-27 18:55:13,978][20381] Updated weights for policy 1, policy_version 1914232 (0.0006) [2023-12-27 18:55:14,442][20347] Updated weights for policy 0, policy_version 1915776 (0.0007) [2023-12-27 18:55:14,491][20347] Updated weights for policy 0, policy_version 1915786 (0.0007) [2023-12-27 18:55:14,538][20347] Updated weights for policy 0, policy_version 1915796 (0.0007) [2023-12-27 18:55:14,604][20381] Updated weights for policy 1, policy_version 1914242 (0.0006) [2023-12-27 18:55:14,663][20381] Updated weights for policy 1, policy_version 1914252 (0.0005) [2023-12-27 18:55:14,720][20381] Updated weights for policy 1, policy_version 1914262 (0.0005) [2023-12-27 18:55:14,775][20381] Updated weights for policy 1, policy_version 1914272 (0.0006) [2023-12-27 18:55:15,280][20347] Updated weights for policy 0, policy_version 1915806 (0.0010) [2023-12-27 18:55:15,337][20347] Updated weights for policy 0, policy_version 1915816 (0.0011) [2023-12-27 18:55:15,386][20381] Updated weights for policy 1, policy_version 1914282 (0.0010) [2023-12-27 18:55:15,395][20347] Updated weights for policy 0, policy_version 1915826 (0.0010) [2023-12-27 18:55:15,434][20381] Updated weights for policy 1, policy_version 1914292 (0.0010) [2023-12-27 18:55:15,486][20381] Updated weights for policy 1, policy_version 1914302 (0.0010) [2023-12-27 18:55:15,989][20347] Updated weights for policy 0, policy_version 1915836 (0.0008) [2023-12-27 18:55:16,050][20347] Updated weights for policy 0, policy_version 1915846 (0.0006) [2023-12-27 18:55:16,075][20381] Updated weights for policy 1, policy_version 1914312 (0.0011) [2023-12-27 18:55:16,109][20347] Updated weights for policy 0, policy_version 1915856 (0.0010) [2023-12-27 18:55:16,131][20381] Updated weights for policy 1, policy_version 1914322 (0.0011) [2023-12-27 18:55:16,185][20381] Updated weights for policy 1, policy_version 1914332 (0.0010) [2023-12-27 18:55:16,755][20347] Updated weights for policy 0, policy_version 1915866 (0.0010) [2023-12-27 18:55:16,806][20347] Updated weights for policy 0, policy_version 1915876 (0.0010) [2023-12-27 18:55:16,855][20347] Updated weights for policy 0, policy_version 1915886 (0.0010) [2023-12-27 18:55:16,875][20381] Updated weights for policy 1, policy_version 1914342 (0.0011) [2023-12-27 18:55:16,909][20347] Updated weights for policy 0, policy_version 1915896 (0.0010) [2023-12-27 18:55:16,923][20381] Updated weights for policy 1, policy_version 1914352 (0.0010) [2023-12-27 18:55:16,971][20381] Updated weights for policy 1, policy_version 1914362 (0.0010) [2023-12-27 18:55:17,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20343.4, 300 sec: 20521.7). Total num frames: 980697088. Throughput: 0: 10226.7, 1: 10203.4. Samples: 980707128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.490')] [2023-12-27 18:55:17,602][20347] Updated weights for policy 0, policy_version 1915906 (0.0005) [2023-12-27 18:55:17,654][20347] Updated weights for policy 0, policy_version 1915916 (0.0006) [2023-12-27 18:55:17,670][20381] Updated weights for policy 1, policy_version 1914372 (0.0009) [2023-12-27 18:55:17,705][20347] Updated weights for policy 0, policy_version 1915926 (0.0007) [2023-12-27 18:55:17,726][20381] Updated weights for policy 1, policy_version 1914382 (0.0007) [2023-12-27 18:55:17,771][20381] Updated weights for policy 1, policy_version 1914392 (0.0010) [2023-12-27 18:55:18,308][20347] Updated weights for policy 0, policy_version 1915936 (0.0006) [2023-12-27 18:55:18,373][20347] Updated weights for policy 0, policy_version 1915946 (0.0009) [2023-12-27 18:55:18,432][20347] Updated weights for policy 0, policy_version 1915956 (0.0011) [2023-12-27 18:55:18,460][20381] Updated weights for policy 1, policy_version 1914402 (0.0010) [2023-12-27 18:55:18,524][20381] Updated weights for policy 1, policy_version 1914412 (0.0011) [2023-12-27 18:55:18,585][20381] Updated weights for policy 1, policy_version 1914422 (0.0006) [2023-12-27 18:55:18,637][20381] Updated weights for policy 1, policy_version 1914432 (0.0005) [2023-12-27 18:55:19,051][20347] Updated weights for policy 0, policy_version 1915966 (0.0010) [2023-12-27 18:55:19,105][20347] Updated weights for policy 0, policy_version 1915976 (0.0010) [2023-12-27 18:55:19,157][20347] Updated weights for policy 0, policy_version 1915986 (0.0010) [2023-12-27 18:55:19,299][20381] Updated weights for policy 1, policy_version 1914442 (0.0009) [2023-12-27 18:55:19,364][20381] Updated weights for policy 1, policy_version 1914452 (0.0009) [2023-12-27 18:55:19,425][20381] Updated weights for policy 1, policy_version 1914462 (0.0008) [2023-12-27 18:55:19,846][20347] Updated weights for policy 0, policy_version 1915996 (0.0006) [2023-12-27 18:55:19,907][20347] Updated weights for policy 0, policy_version 1916006 (0.0007) [2023-12-27 18:55:19,971][20347] Updated weights for policy 0, policy_version 1916016 (0.0008) [2023-12-27 18:55:20,154][20381] Updated weights for policy 1, policy_version 1914472 (0.0010) [2023-12-27 18:55:20,215][20381] Updated weights for policy 1, policy_version 1914482 (0.0010) [2023-12-27 18:55:20,274][20381] Updated weights for policy 1, policy_version 1914492 (0.0008) [2023-12-27 18:55:20,599][20347] Updated weights for policy 0, policy_version 1916026 (0.0008) [2023-12-27 18:55:20,658][20347] Updated weights for policy 0, policy_version 1916036 (0.0006) [2023-12-27 18:55:20,722][20347] Updated weights for policy 0, policy_version 1916046 (0.0006) [2023-12-27 18:55:20,782][20347] Updated weights for policy 0, policy_version 1916056 (0.0006) [2023-12-27 18:55:20,899][20381] Updated weights for policy 1, policy_version 1914502 (0.0011) [2023-12-27 18:55:20,956][20381] Updated weights for policy 1, policy_version 1914512 (0.0011) [2023-12-27 18:55:21,012][20381] Updated weights for policy 1, policy_version 1914522 (0.0011) [2023-12-27 18:55:21,574][20347] Updated weights for policy 0, policy_version 1916066 (0.0009) [2023-12-27 18:55:21,635][20347] Updated weights for policy 0, policy_version 1916076 (0.0010) [2023-12-27 18:55:21,694][20347] Updated weights for policy 0, policy_version 1916086 (0.0011) [2023-12-27 18:55:21,715][20381] Updated weights for policy 1, policy_version 1914532 (0.0009) [2023-12-27 18:55:21,775][20381] Updated weights for policy 1, policy_version 1914542 (0.0009) [2023-12-27 18:55:21,838][20381] Updated weights for policy 1, policy_version 1914552 (0.0011) [2023-12-27 18:55:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 980795392. Throughput: 0: 10242.2, 1: 10181.7. Samples: 980768992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.090')] [2023-12-27 18:55:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001914560_490184704.pth... [2023-12-27 18:55:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001916088_490610688.pth... [2023-12-27 18:55:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001913376_489881600.pth [2023-12-27 18:55:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001914904_490307584.pth [2023-12-27 18:55:22,463][20347] Updated weights for policy 0, policy_version 1916096 (0.0008) [2023-12-27 18:55:22,513][20381] Updated weights for policy 1, policy_version 1914562 (0.0010) [2023-12-27 18:55:22,515][20347] Updated weights for policy 0, policy_version 1916106 (0.0007) [2023-12-27 18:55:22,560][20381] Updated weights for policy 1, policy_version 1914572 (0.0005) [2023-12-27 18:55:22,572][20347] Updated weights for policy 0, policy_version 1916116 (0.0007) [2023-12-27 18:55:22,614][20381] Updated weights for policy 1, policy_version 1914582 (0.0007) [2023-12-27 18:55:22,669][20381] Updated weights for policy 1, policy_version 1914592 (0.0008) [2023-12-27 18:55:23,267][20347] Updated weights for policy 0, policy_version 1916126 (0.0006) [2023-12-27 18:55:23,319][20347] Updated weights for policy 0, policy_version 1916136 (0.0006) [2023-12-27 18:55:23,379][20347] Updated weights for policy 0, policy_version 1916146 (0.0008) [2023-12-27 18:55:23,403][20381] Updated weights for policy 1, policy_version 1914602 (0.0007) [2023-12-27 18:55:23,451][20381] Updated weights for policy 1, policy_version 1914612 (0.0008) [2023-12-27 18:55:23,500][20381] Updated weights for policy 1, policy_version 1914622 (0.0007) [2023-12-27 18:55:24,051][20347] Updated weights for policy 0, policy_version 1916156 (0.0005) [2023-12-27 18:55:24,113][20347] Updated weights for policy 0, policy_version 1916166 (0.0010) [2023-12-27 18:55:24,142][20381] Updated weights for policy 1, policy_version 1914632 (0.0006) [2023-12-27 18:55:24,168][20347] Updated weights for policy 0, policy_version 1916176 (0.0010) [2023-12-27 18:55:24,203][20381] Updated weights for policy 1, policy_version 1914642 (0.0006) [2023-12-27 18:55:24,255][20381] Updated weights for policy 1, policy_version 1914652 (0.0008) [2023-12-27 18:55:24,843][20347] Updated weights for policy 0, policy_version 1916186 (0.0010) [2023-12-27 18:55:24,897][20347] Updated weights for policy 0, policy_version 1916196 (0.0008) [2023-12-27 18:55:24,918][20381] Updated weights for policy 1, policy_version 1914662 (0.0009) [2023-12-27 18:55:24,953][20347] Updated weights for policy 0, policy_version 1916206 (0.0007) [2023-12-27 18:55:24,971][20381] Updated weights for policy 1, policy_version 1914672 (0.0010) [2023-12-27 18:55:25,005][20347] Updated weights for policy 0, policy_version 1916216 (0.0006) [2023-12-27 18:55:25,030][20381] Updated weights for policy 1, policy_version 1914682 (0.0010) [2023-12-27 18:55:25,606][20347] Updated weights for policy 0, policy_version 1916226 (0.0005) [2023-12-27 18:55:25,622][20381] Updated weights for policy 1, policy_version 1914692 (0.0010) [2023-12-27 18:55:25,668][20347] Updated weights for policy 0, policy_version 1916236 (0.0005) [2023-12-27 18:55:25,678][20381] Updated weights for policy 1, policy_version 1914702 (0.0010) [2023-12-27 18:55:25,725][20347] Updated weights for policy 0, policy_version 1916246 (0.0007) [2023-12-27 18:55:25,735][20381] Updated weights for policy 1, policy_version 1914712 (0.0008) [2023-12-27 18:55:26,399][20347] Updated weights for policy 0, policy_version 1916256 (0.0007) [2023-12-27 18:55:26,418][20381] Updated weights for policy 1, policy_version 1914722 (0.0011) [2023-12-27 18:55:26,457][20347] Updated weights for policy 0, policy_version 1916266 (0.0006) [2023-12-27 18:55:26,481][20381] Updated weights for policy 1, policy_version 1914732 (0.0011) [2023-12-27 18:55:26,509][20347] Updated weights for policy 0, policy_version 1916276 (0.0006) [2023-12-27 18:55:26,539][20381] Updated weights for policy 1, policy_version 1914742 (0.0010) [2023-12-27 18:55:26,601][20381] Updated weights for policy 1, policy_version 1914752 (0.0010) [2023-12-27 18:55:27,152][20347] Updated weights for policy 0, policy_version 1916286 (0.0007) [2023-12-27 18:55:27,201][20381] Updated weights for policy 1, policy_version 1914762 (0.0011) [2023-12-27 18:55:27,203][20347] Updated weights for policy 0, policy_version 1916296 (0.0005) [2023-12-27 18:55:27,256][20381] Updated weights for policy 1, policy_version 1914772 (0.0010) [2023-12-27 18:55:27,258][20347] Updated weights for policy 0, policy_version 1916306 (0.0005) [2023-12-27 18:55:27,315][20381] Updated weights for policy 1, policy_version 1914782 (0.0011) [2023-12-27 18:55:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 980910080. Throughput: 0: 10218.1, 1: 10271.6. Samples: 980892960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.750')] [2023-12-27 18:55:27,859][20347] Updated weights for policy 0, policy_version 1916316 (0.0006) [2023-12-27 18:55:27,910][20347] Updated weights for policy 0, policy_version 1916326 (0.0007) [2023-12-27 18:55:27,963][20347] Updated weights for policy 0, policy_version 1916336 (0.0008) [2023-12-27 18:55:27,977][20381] Updated weights for policy 1, policy_version 1914792 (0.0011) [2023-12-27 18:55:28,025][20381] Updated weights for policy 1, policy_version 1914802 (0.0010) [2023-12-27 18:55:28,074][20381] Updated weights for policy 1, policy_version 1914812 (0.0010) [2023-12-27 18:55:28,623][20347] Updated weights for policy 0, policy_version 1916346 (0.0007) [2023-12-27 18:55:28,686][20347] Updated weights for policy 0, policy_version 1916356 (0.0007) [2023-12-27 18:55:28,750][20347] Updated weights for policy 0, policy_version 1916366 (0.0006) [2023-12-27 18:55:28,796][20381] Updated weights for policy 1, policy_version 1914822 (0.0009) [2023-12-27 18:55:28,802][20347] Updated weights for policy 0, policy_version 1916376 (0.0006) [2023-12-27 18:55:28,853][20381] Updated weights for policy 1, policy_version 1914832 (0.0007) [2023-12-27 18:55:28,908][20381] Updated weights for policy 1, policy_version 1914842 (0.0005) [2023-12-27 18:55:29,503][20347] Updated weights for policy 0, policy_version 1916386 (0.0006) [2023-12-27 18:55:29,568][20347] Updated weights for policy 0, policy_version 1916396 (0.0006) [2023-12-27 18:55:29,585][20381] Updated weights for policy 1, policy_version 1914852 (0.0008) [2023-12-27 18:55:29,631][20347] Updated weights for policy 0, policy_version 1916406 (0.0007) [2023-12-27 18:55:29,641][20381] Updated weights for policy 1, policy_version 1914862 (0.0006) [2023-12-27 18:55:29,704][20381] Updated weights for policy 1, policy_version 1914872 (0.0008) [2023-12-27 18:55:30,294][20347] Updated weights for policy 0, policy_version 1916416 (0.0008) [2023-12-27 18:55:30,351][20347] Updated weights for policy 0, policy_version 1916426 (0.0007) [2023-12-27 18:55:30,360][20381] Updated weights for policy 1, policy_version 1914882 (0.0008) [2023-12-27 18:55:30,403][20347] Updated weights for policy 0, policy_version 1916436 (0.0007) [2023-12-27 18:55:30,409][20381] Updated weights for policy 1, policy_version 1914892 (0.0005) [2023-12-27 18:55:30,461][20381] Updated weights for policy 1, policy_version 1914902 (0.0009) [2023-12-27 18:55:30,521][20381] Updated weights for policy 1, policy_version 1914912 (0.0006) [2023-12-27 18:55:31,138][20347] Updated weights for policy 0, policy_version 1916446 (0.0008) [2023-12-27 18:55:31,161][20381] Updated weights for policy 1, policy_version 1914922 (0.0008) [2023-12-27 18:55:31,193][20347] Updated weights for policy 0, policy_version 1916456 (0.0008) [2023-12-27 18:55:31,214][20381] Updated weights for policy 1, policy_version 1914932 (0.0006) [2023-12-27 18:55:31,253][20347] Updated weights for policy 0, policy_version 1916466 (0.0006) [2023-12-27 18:55:31,277][20381] Updated weights for policy 1, policy_version 1914942 (0.0006) [2023-12-27 18:55:31,956][20347] Updated weights for policy 0, policy_version 1916476 (0.0008) [2023-12-27 18:55:31,993][20381] Updated weights for policy 1, policy_version 1914952 (0.0009) [2023-12-27 18:55:32,008][20347] Updated weights for policy 0, policy_version 1916486 (0.0010) [2023-12-27 18:55:32,051][20381] Updated weights for policy 1, policy_version 1914962 (0.0007) [2023-12-27 18:55:32,061][20347] Updated weights for policy 0, policy_version 1916496 (0.0010) [2023-12-27 18:55:32,107][20381] Updated weights for policy 1, policy_version 1914972 (0.0006) [2023-12-27 18:55:32,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 981008384. Throughput: 0: 10232.7, 1: 10275.0. Samples: 981016144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.350')] [2023-12-27 18:55:32,644][20347] Updated weights for policy 0, policy_version 1916506 (0.0010) [2023-12-27 18:55:32,695][20347] Updated weights for policy 0, policy_version 1916516 (0.0010) [2023-12-27 18:55:32,753][20347] Updated weights for policy 0, policy_version 1916526 (0.0009) [2023-12-27 18:55:32,811][20347] Updated weights for policy 0, policy_version 1916536 (0.0006) [2023-12-27 18:55:32,839][20381] Updated weights for policy 1, policy_version 1914982 (0.0007) [2023-12-27 18:55:32,895][20381] Updated weights for policy 1, policy_version 1914992 (0.0008) [2023-12-27 18:55:32,953][20381] Updated weights for policy 1, policy_version 1915002 (0.0008) [2023-12-27 18:55:33,449][20347] Updated weights for policy 0, policy_version 1916546 (0.0005) [2023-12-27 18:55:33,501][20347] Updated weights for policy 0, policy_version 1916556 (0.0005) [2023-12-27 18:55:33,538][20381] Updated weights for policy 1, policy_version 1915012 (0.0006) [2023-12-27 18:55:33,551][20347] Updated weights for policy 0, policy_version 1916566 (0.0005) [2023-12-27 18:55:33,593][20381] Updated weights for policy 1, policy_version 1915022 (0.0005) [2023-12-27 18:55:33,646][20381] Updated weights for policy 1, policy_version 1915032 (0.0005) [2023-12-27 18:55:34,138][20347] Updated weights for policy 0, policy_version 1916576 (0.0009) [2023-12-27 18:55:34,204][20347] Updated weights for policy 0, policy_version 1916586 (0.0010) [2023-12-27 18:55:34,246][20381] Updated weights for policy 1, policy_version 1915042 (0.0005) [2023-12-27 18:55:34,263][20347] Updated weights for policy 0, policy_version 1916596 (0.0010) [2023-12-27 18:55:34,307][20381] Updated weights for policy 1, policy_version 1915052 (0.0006) [2023-12-27 18:55:34,367][20381] Updated weights for policy 1, policy_version 1915062 (0.0007) [2023-12-27 18:55:34,423][20381] Updated weights for policy 1, policy_version 1915072 (0.0007) [2023-12-27 18:55:34,926][20347] Updated weights for policy 0, policy_version 1916606 (0.0010) [2023-12-27 18:55:34,985][20347] Updated weights for policy 0, policy_version 1916616 (0.0010) [2023-12-27 18:55:35,039][20347] Updated weights for policy 0, policy_version 1916626 (0.0010) [2023-12-27 18:55:35,117][20381] Updated weights for policy 1, policy_version 1915082 (0.0010) [2023-12-27 18:55:35,169][20381] Updated weights for policy 1, policy_version 1915092 (0.0010) [2023-12-27 18:55:35,230][20381] Updated weights for policy 1, policy_version 1915102 (0.0007) [2023-12-27 18:55:35,698][20347] Updated weights for policy 0, policy_version 1916636 (0.0008) [2023-12-27 18:55:35,752][20347] Updated weights for policy 0, policy_version 1916646 (0.0005) [2023-12-27 18:55:35,810][20347] Updated weights for policy 0, policy_version 1916656 (0.0005) [2023-12-27 18:55:35,918][20381] Updated weights for policy 1, policy_version 1915112 (0.0010) [2023-12-27 18:55:35,973][20381] Updated weights for policy 1, policy_version 1915122 (0.0011) [2023-12-27 18:55:36,028][20381] Updated weights for policy 1, policy_version 1915132 (0.0011) [2023-12-27 18:55:36,423][20347] Updated weights for policy 0, policy_version 1916666 (0.0006) [2023-12-27 18:55:36,479][20347] Updated weights for policy 0, policy_version 1916676 (0.0007) [2023-12-27 18:55:36,540][20347] Updated weights for policy 0, policy_version 1916686 (0.0008) [2023-12-27 18:55:36,597][20347] Updated weights for policy 0, policy_version 1916696 (0.0007) [2023-12-27 18:55:36,745][20381] Updated weights for policy 1, policy_version 1915142 (0.0009) [2023-12-27 18:55:36,807][20381] Updated weights for policy 1, policy_version 1915152 (0.0008) [2023-12-27 18:55:36,866][20381] Updated weights for policy 1, policy_version 1915162 (0.0008) [2023-12-27 18:55:37,202][20347] Updated weights for policy 0, policy_version 1916706 (0.0007) [2023-12-27 18:55:37,258][20347] Updated weights for policy 0, policy_version 1916716 (0.0008) [2023-12-27 18:55:37,320][20347] Updated weights for policy 0, policy_version 1916726 (0.0008) [2023-12-27 18:55:37,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 981114880. Throughput: 0: 10282.6, 1: 10320.3. Samples: 981080772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.070')] [2023-12-27 18:55:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001915168_490340352.pth... [2023-12-27 18:55:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001916728_490774528.pth... [2023-12-27 18:55:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001915512_490463232.pth [2023-12-27 18:55:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001913952_490029056.pth [2023-12-27 18:55:37,527][20381] Updated weights for policy 1, policy_version 1915172 (0.0007) [2023-12-27 18:55:37,591][20381] Updated weights for policy 1, policy_version 1915182 (0.0008) [2023-12-27 18:55:37,660][20381] Updated weights for policy 1, policy_version 1915192 (0.0008) [2023-12-27 18:55:38,012][20347] Updated weights for policy 0, policy_version 1916736 (0.0008) [2023-12-27 18:55:38,063][20347] Updated weights for policy 0, policy_version 1916746 (0.0008) [2023-12-27 18:55:38,114][20347] Updated weights for policy 0, policy_version 1916756 (0.0007) [2023-12-27 18:55:38,292][20381] Updated weights for policy 1, policy_version 1915202 (0.0007) [2023-12-27 18:55:38,360][20381] Updated weights for policy 1, policy_version 1915212 (0.0008) [2023-12-27 18:55:38,417][20381] Updated weights for policy 1, policy_version 1915222 (0.0008) [2023-12-27 18:55:38,476][20381] Updated weights for policy 1, policy_version 1915232 (0.0008) [2023-12-27 18:55:38,844][20347] Updated weights for policy 0, policy_version 1916766 (0.0007) [2023-12-27 18:55:38,900][20347] Updated weights for policy 0, policy_version 1916776 (0.0008) [2023-12-27 18:55:38,962][20347] Updated weights for policy 0, policy_version 1916786 (0.0007) [2023-12-27 18:55:39,130][20381] Updated weights for policy 1, policy_version 1915242 (0.0008) [2023-12-27 18:55:39,194][20381] Updated weights for policy 1, policy_version 1915252 (0.0008) [2023-12-27 18:55:39,263][20381] Updated weights for policy 1, policy_version 1915262 (0.0008) [2023-12-27 18:55:39,708][20347] Updated weights for policy 0, policy_version 1916796 (0.0007) [2023-12-27 18:55:39,759][20347] Updated weights for policy 0, policy_version 1916806 (0.0006) [2023-12-27 18:55:39,809][20347] Updated weights for policy 0, policy_version 1916816 (0.0007) [2023-12-27 18:55:40,034][20381] Updated weights for policy 1, policy_version 1915272 (0.0010) [2023-12-27 18:55:40,097][20381] Updated weights for policy 1, policy_version 1915282 (0.0011) [2023-12-27 18:55:40,161][20381] Updated weights for policy 1, policy_version 1915292 (0.0011) [2023-12-27 18:55:40,523][20347] Updated weights for policy 0, policy_version 1916826 (0.0008) [2023-12-27 18:55:40,567][20347] Updated weights for policy 0, policy_version 1916836 (0.0007) [2023-12-27 18:55:40,625][20347] Updated weights for policy 0, policy_version 1916846 (0.0006) [2023-12-27 18:55:40,674][20347] Updated weights for policy 0, policy_version 1916856 (0.0007) [2023-12-27 18:55:40,822][20381] Updated weights for policy 1, policy_version 1915302 (0.0007) [2023-12-27 18:55:40,883][20381] Updated weights for policy 1, policy_version 1915312 (0.0007) [2023-12-27 18:55:40,935][20381] Updated weights for policy 1, policy_version 1915322 (0.0008) [2023-12-27 18:55:41,353][20347] Updated weights for policy 0, policy_version 1916866 (0.0006) [2023-12-27 18:55:41,426][20347] Updated weights for policy 0, policy_version 1916876 (0.0007) [2023-12-27 18:55:41,486][20347] Updated weights for policy 0, policy_version 1916886 (0.0006) [2023-12-27 18:55:41,684][20381] Updated weights for policy 1, policy_version 1915332 (0.0008) [2023-12-27 18:55:41,751][20381] Updated weights for policy 1, policy_version 1915342 (0.0009) [2023-12-27 18:55:41,813][20381] Updated weights for policy 1, policy_version 1915352 (0.0009) [2023-12-27 18:55:42,177][20347] Updated weights for policy 0, policy_version 1916896 (0.0007) [2023-12-27 18:55:42,240][20347] Updated weights for policy 0, policy_version 1916906 (0.0008) [2023-12-27 18:55:42,303][20347] Updated weights for policy 0, policy_version 1916916 (0.0008) [2023-12-27 18:55:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20521.6). Total num frames: 981213184. Throughput: 0: 10258.3, 1: 10285.1. Samples: 981202080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '950.190')] [2023-12-27 18:55:42,480][20381] Updated weights for policy 1, policy_version 1915362 (0.0008) [2023-12-27 18:55:42,540][20381] Updated weights for policy 1, policy_version 1915372 (0.0007) [2023-12-27 18:55:42,604][20381] Updated weights for policy 1, policy_version 1915382 (0.0005) [2023-12-27 18:55:42,666][20381] Updated weights for policy 1, policy_version 1915392 (0.0007) [2023-12-27 18:55:42,983][20347] Updated weights for policy 0, policy_version 1916926 (0.0008) [2023-12-27 18:55:43,030][20347] Updated weights for policy 0, policy_version 1916936 (0.0010) [2023-12-27 18:55:43,094][20347] Updated weights for policy 0, policy_version 1916946 (0.0008) [2023-12-27 18:55:43,251][20381] Updated weights for policy 1, policy_version 1915402 (0.0005) [2023-12-27 18:55:43,302][20381] Updated weights for policy 1, policy_version 1915412 (0.0006) [2023-12-27 18:55:43,350][20381] Updated weights for policy 1, policy_version 1915422 (0.0005) [2023-12-27 18:55:43,796][20347] Updated weights for policy 0, policy_version 1916956 (0.0007) [2023-12-27 18:55:43,844][20347] Updated weights for policy 0, policy_version 1916966 (0.0005) [2023-12-27 18:55:43,897][20347] Updated weights for policy 0, policy_version 1916976 (0.0005) [2023-12-27 18:55:43,946][20381] Updated weights for policy 1, policy_version 1915432 (0.0009) [2023-12-27 18:55:44,004][20381] Updated weights for policy 1, policy_version 1915442 (0.0010) [2023-12-27 18:55:44,054][20381] Updated weights for policy 1, policy_version 1915452 (0.0010) [2023-12-27 18:55:44,452][20347] Updated weights for policy 0, policy_version 1916986 (0.0006) [2023-12-27 18:55:44,508][20347] Updated weights for policy 0, policy_version 1916996 (0.0007) [2023-12-27 18:55:44,561][20347] Updated weights for policy 0, policy_version 1917006 (0.0008) [2023-12-27 18:55:44,610][20347] Updated weights for policy 0, policy_version 1917016 (0.0008) [2023-12-27 18:55:44,682][20381] Updated weights for policy 1, policy_version 1915462 (0.0008) [2023-12-27 18:55:44,736][20381] Updated weights for policy 1, policy_version 1915472 (0.0006) [2023-12-27 18:55:44,795][20381] Updated weights for policy 1, policy_version 1915482 (0.0008) [2023-12-27 18:55:45,348][20347] Updated weights for policy 0, policy_version 1917026 (0.0008) [2023-12-27 18:55:45,396][20347] Updated weights for policy 0, policy_version 1917036 (0.0008) [2023-12-27 18:55:45,446][20347] Updated weights for policy 0, policy_version 1917046 (0.0008) [2023-12-27 18:55:45,511][20381] Updated weights for policy 1, policy_version 1915492 (0.0007) [2023-12-27 18:55:45,566][20381] Updated weights for policy 1, policy_version 1915502 (0.0006) [2023-12-27 18:55:45,622][20381] Updated weights for policy 1, policy_version 1915512 (0.0005) [2023-12-27 18:55:46,170][20347] Updated weights for policy 0, policy_version 1917056 (0.0006) [2023-12-27 18:55:46,193][20381] Updated weights for policy 1, policy_version 1915522 (0.0005) [2023-12-27 18:55:46,225][20347] Updated weights for policy 0, policy_version 1917066 (0.0006) [2023-12-27 18:55:46,249][20381] Updated weights for policy 1, policy_version 1915532 (0.0005) [2023-12-27 18:55:46,279][20347] Updated weights for policy 0, policy_version 1917076 (0.0010) [2023-12-27 18:55:46,307][20381] Updated weights for policy 1, policy_version 1915542 (0.0005) [2023-12-27 18:55:46,367][20381] Updated weights for policy 1, policy_version 1915552 (0.0008) [2023-12-27 18:55:46,888][20347] Updated weights for policy 0, policy_version 1917086 (0.0006) [2023-12-27 18:55:46,934][20347] Updated weights for policy 0, policy_version 1917096 (0.0005) [2023-12-27 18:55:46,980][20347] Updated weights for policy 0, policy_version 1917106 (0.0006) [2023-12-27 18:55:47,001][20381] Updated weights for policy 1, policy_version 1915562 (0.0007) [2023-12-27 18:55:47,054][20381] Updated weights for policy 1, policy_version 1915572 (0.0007) [2023-12-27 18:55:47,105][20381] Updated weights for policy 1, policy_version 1915582 (0.0008) [2023-12-27 18:55:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 981319680. Throughput: 0: 10258.0, 1: 10394.6. Samples: 981327940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '950.030')] [2023-12-27 18:55:47,606][20347] Updated weights for policy 0, policy_version 1917116 (0.0009) [2023-12-27 18:55:47,655][20347] Updated weights for policy 0, policy_version 1917126 (0.0008) [2023-12-27 18:55:47,704][20347] Updated weights for policy 0, policy_version 1917136 (0.0006) [2023-12-27 18:55:47,799][20381] Updated weights for policy 1, policy_version 1915592 (0.0008) [2023-12-27 18:55:47,853][20381] Updated weights for policy 1, policy_version 1915602 (0.0007) [2023-12-27 18:55:47,904][20381] Updated weights for policy 1, policy_version 1915612 (0.0007) [2023-12-27 18:55:48,420][20347] Updated weights for policy 0, policy_version 1917146 (0.0007) [2023-12-27 18:55:48,476][20347] Updated weights for policy 0, policy_version 1917156 (0.0007) [2023-12-27 18:55:48,536][20347] Updated weights for policy 0, policy_version 1917166 (0.0008) [2023-12-27 18:55:48,589][20381] Updated weights for policy 1, policy_version 1915622 (0.0009) [2023-12-27 18:55:48,592][20347] Updated weights for policy 0, policy_version 1917176 (0.0005) [2023-12-27 18:55:48,655][20381] Updated weights for policy 1, policy_version 1915632 (0.0007) [2023-12-27 18:55:48,710][20381] Updated weights for policy 1, policy_version 1915642 (0.0007) [2023-12-27 18:55:49,186][20347] Updated weights for policy 0, policy_version 1917186 (0.0007) [2023-12-27 18:55:49,247][20347] Updated weights for policy 0, policy_version 1917196 (0.0009) [2023-12-27 18:55:49,311][20347] Updated weights for policy 0, policy_version 1917206 (0.0011) [2023-12-27 18:55:49,340][20381] Updated weights for policy 1, policy_version 1915652 (0.0008) [2023-12-27 18:55:49,417][20381] Updated weights for policy 1, policy_version 1915662 (0.0010) [2023-12-27 18:55:49,484][20381] Updated weights for policy 1, policy_version 1915672 (0.0009) [2023-12-27 18:55:50,033][20347] Updated weights for policy 0, policy_version 1917216 (0.0010) [2023-12-27 18:55:50,085][20347] Updated weights for policy 0, policy_version 1917226 (0.0010) [2023-12-27 18:55:50,142][20347] Updated weights for policy 0, policy_version 1917236 (0.0010) [2023-12-27 18:55:50,152][20381] Updated weights for policy 1, policy_version 1915682 (0.0007) [2023-12-27 18:55:50,216][20381] Updated weights for policy 1, policy_version 1915692 (0.0007) [2023-12-27 18:55:50,269][20381] Updated weights for policy 1, policy_version 1915702 (0.0007) [2023-12-27 18:55:50,325][20381] Updated weights for policy 1, policy_version 1915712 (0.0006) [2023-12-27 18:55:50,858][20347] Updated weights for policy 0, policy_version 1917246 (0.0009) [2023-12-27 18:55:50,914][20347] Updated weights for policy 0, policy_version 1917256 (0.0008) [2023-12-27 18:55:50,969][20347] Updated weights for policy 0, policy_version 1917266 (0.0008) [2023-12-27 18:55:51,007][20381] Updated weights for policy 1, policy_version 1915722 (0.0007) [2023-12-27 18:55:51,080][20381] Updated weights for policy 1, policy_version 1915732 (0.0008) [2023-12-27 18:55:51,144][20381] Updated weights for policy 1, policy_version 1915742 (0.0007) [2023-12-27 18:55:51,720][20347] Updated weights for policy 0, policy_version 1917276 (0.0008) [2023-12-27 18:55:51,789][20347] Updated weights for policy 0, policy_version 1917286 (0.0007) [2023-12-27 18:55:51,824][20381] Updated weights for policy 1, policy_version 1915752 (0.0008) [2023-12-27 18:55:51,852][20347] Updated weights for policy 0, policy_version 1917296 (0.0007) [2023-12-27 18:55:51,880][20381] Updated weights for policy 1, policy_version 1915762 (0.0011) [2023-12-27 18:55:51,938][20381] Updated weights for policy 1, policy_version 1915772 (0.0011) [2023-12-27 18:55:52,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 981417984. Throughput: 0: 10284.4, 1: 10392.9. Samples: 981390420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '930.420')] [2023-12-27 18:55:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001915776_490496000.pth... [2023-12-27 18:55:52,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001917304_490921984.pth... [2023-12-27 18:55:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001914560_490184704.pth [2023-12-27 18:55:52,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001916088_490610688.pth [2023-12-27 18:55:52,539][20347] Updated weights for policy 0, policy_version 1917306 (0.0006) [2023-12-27 18:55:52,580][20381] Updated weights for policy 1, policy_version 1915782 (0.0011) [2023-12-27 18:55:52,603][20347] Updated weights for policy 0, policy_version 1917316 (0.0008) [2023-12-27 18:55:52,636][20381] Updated weights for policy 1, policy_version 1915792 (0.0011) [2023-12-27 18:55:52,659][20347] Updated weights for policy 0, policy_version 1917326 (0.0006) [2023-12-27 18:55:52,696][20381] Updated weights for policy 1, policy_version 1915802 (0.0009) [2023-12-27 18:55:52,715][20347] Updated weights for policy 0, policy_version 1917336 (0.0006) [2023-12-27 18:55:53,380][20347] Updated weights for policy 0, policy_version 1917346 (0.0007) [2023-12-27 18:55:53,388][20381] Updated weights for policy 1, policy_version 1915812 (0.0011) [2023-12-27 18:55:53,438][20347] Updated weights for policy 0, policy_version 1917356 (0.0007) [2023-12-27 18:55:53,440][20381] Updated weights for policy 1, policy_version 1915822 (0.0010) [2023-12-27 18:55:53,493][20347] Updated weights for policy 0, policy_version 1917366 (0.0005) [2023-12-27 18:55:53,495][20381] Updated weights for policy 1, policy_version 1915832 (0.0010) [2023-12-27 18:55:54,056][20347] Updated weights for policy 0, policy_version 1917376 (0.0007) [2023-12-27 18:55:54,103][20347] Updated weights for policy 0, policy_version 1917386 (0.0005) [2023-12-27 18:55:54,152][20347] Updated weights for policy 0, policy_version 1917396 (0.0005) [2023-12-27 18:55:54,171][20381] Updated weights for policy 1, policy_version 1915842 (0.0011) [2023-12-27 18:55:54,219][20381] Updated weights for policy 1, policy_version 1915852 (0.0010) [2023-12-27 18:55:54,266][20381] Updated weights for policy 1, policy_version 1915862 (0.0005) [2023-12-27 18:55:54,321][20381] Updated weights for policy 1, policy_version 1915872 (0.0005) [2023-12-27 18:55:54,751][20347] Updated weights for policy 0, policy_version 1917406 (0.0005) [2023-12-27 18:55:54,798][20347] Updated weights for policy 0, policy_version 1917416 (0.0005) [2023-12-27 18:55:54,852][20347] Updated weights for policy 0, policy_version 1917426 (0.0005) [2023-12-27 18:55:54,939][20381] Updated weights for policy 1, policy_version 1915882 (0.0007) [2023-12-27 18:55:55,003][20381] Updated weights for policy 1, policy_version 1915892 (0.0011) [2023-12-27 18:55:55,060][20381] Updated weights for policy 1, policy_version 1915902 (0.0011) [2023-12-27 18:55:55,551][20347] Updated weights for policy 0, policy_version 1917436 (0.0008) [2023-12-27 18:55:55,607][20381] Updated weights for policy 1, policy_version 1915912 (0.0006) [2023-12-27 18:55:55,608][20347] Updated weights for policy 0, policy_version 1917446 (0.0008) [2023-12-27 18:55:55,659][20381] Updated weights for policy 1, policy_version 1915922 (0.0005) [2023-12-27 18:55:55,663][20347] Updated weights for policy 0, policy_version 1917456 (0.0008) [2023-12-27 18:55:55,710][20381] Updated weights for policy 1, policy_version 1915932 (0.0005) [2023-12-27 18:55:56,287][20381] Updated weights for policy 1, policy_version 1915942 (0.0007) [2023-12-27 18:55:56,350][20381] Updated weights for policy 1, policy_version 1915952 (0.0007) [2023-12-27 18:55:56,396][20381] Updated weights for policy 1, policy_version 1915962 (0.0006) [2023-12-27 18:55:56,402][20347] Updated weights for policy 0, policy_version 1917466 (0.0007) [2023-12-27 18:55:56,480][20347] Updated weights for policy 0, policy_version 1917476 (0.0008) [2023-12-27 18:55:56,546][20347] Updated weights for policy 0, policy_version 1917486 (0.0008) [2023-12-27 18:55:56,607][20347] Updated weights for policy 0, policy_version 1917496 (0.0007) [2023-12-27 18:55:57,086][20381] Updated weights for policy 1, policy_version 1915972 (0.0008) [2023-12-27 18:55:57,135][20381] Updated weights for policy 1, policy_version 1915982 (0.0008) [2023-12-27 18:55:57,183][20381] Updated weights for policy 1, policy_version 1915992 (0.0007) [2023-12-27 18:55:57,216][20347] Updated weights for policy 0, policy_version 1917506 (0.0006) [2023-12-27 18:55:57,267][20347] Updated weights for policy 0, policy_version 1917516 (0.0007) [2023-12-27 18:55:57,326][20347] Updated weights for policy 0, policy_version 1917526 (0.0007) [2023-12-27 18:55:57,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 981532672. Throughput: 0: 10296.1, 1: 10454.7. Samples: 981516592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:55:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.880')] [2023-12-27 18:55:57,805][20381] Updated weights for policy 1, policy_version 1916002 (0.0007) [2023-12-27 18:55:57,859][20381] Updated weights for policy 1, policy_version 1916012 (0.0005) [2023-12-27 18:55:57,910][20381] Updated weights for policy 1, policy_version 1916022 (0.0006) [2023-12-27 18:55:57,968][20381] Updated weights for policy 1, policy_version 1916032 (0.0007) [2023-12-27 18:55:58,057][20347] Updated weights for policy 0, policy_version 1917536 (0.0008) [2023-12-27 18:55:58,117][20347] Updated weights for policy 0, policy_version 1917546 (0.0008) [2023-12-27 18:55:58,179][20347] Updated weights for policy 0, policy_version 1917556 (0.0007) [2023-12-27 18:55:58,598][20381] Updated weights for policy 1, policy_version 1916042 (0.0007) [2023-12-27 18:55:58,667][20381] Updated weights for policy 1, policy_version 1916052 (0.0008) [2023-12-27 18:55:58,731][20381] Updated weights for policy 1, policy_version 1916062 (0.0008) [2023-12-27 18:55:59,027][20347] Updated weights for policy 0, policy_version 1917566 (0.0007) [2023-12-27 18:55:59,077][20347] Updated weights for policy 0, policy_version 1917576 (0.0007) [2023-12-27 18:55:59,133][20347] Updated weights for policy 0, policy_version 1917586 (0.0008) [2023-12-27 18:55:59,491][20381] Updated weights for policy 1, policy_version 1916072 (0.0006) [2023-12-27 18:55:59,554][20381] Updated weights for policy 1, policy_version 1916082 (0.0006) [2023-12-27 18:55:59,609][20381] Updated weights for policy 1, policy_version 1916092 (0.0007) [2023-12-27 18:55:59,873][20347] Updated weights for policy 0, policy_version 1917596 (0.0008) [2023-12-27 18:55:59,931][20347] Updated weights for policy 0, policy_version 1917606 (0.0008) [2023-12-27 18:55:59,989][20347] Updated weights for policy 0, policy_version 1917616 (0.0008) [2023-12-27 18:56:00,276][20381] Updated weights for policy 1, policy_version 1916102 (0.0008) [2023-12-27 18:56:00,331][20381] Updated weights for policy 1, policy_version 1916112 (0.0007) [2023-12-27 18:56:00,382][20381] Updated weights for policy 1, policy_version 1916122 (0.0009) [2023-12-27 18:56:00,699][20347] Updated weights for policy 0, policy_version 1917626 (0.0008) [2023-12-27 18:56:00,761][20347] Updated weights for policy 0, policy_version 1917636 (0.0010) [2023-12-27 18:56:00,815][20347] Updated weights for policy 0, policy_version 1917646 (0.0010) [2023-12-27 18:56:00,870][20347] Updated weights for policy 0, policy_version 1917656 (0.0006) [2023-12-27 18:56:01,054][20381] Updated weights for policy 1, policy_version 1916132 (0.0008) [2023-12-27 18:56:01,100][20381] Updated weights for policy 1, policy_version 1916142 (0.0007) [2023-12-27 18:56:01,162][20381] Updated weights for policy 1, policy_version 1916152 (0.0007) [2023-12-27 18:56:01,538][20347] Updated weights for policy 0, policy_version 1917666 (0.0008) [2023-12-27 18:56:01,595][20347] Updated weights for policy 0, policy_version 1917676 (0.0005) [2023-12-27 18:56:01,659][20347] Updated weights for policy 0, policy_version 1917686 (0.0007) [2023-12-27 18:56:01,916][20381] Updated weights for policy 1, policy_version 1916162 (0.0008) [2023-12-27 18:56:01,971][20381] Updated weights for policy 1, policy_version 1916172 (0.0011) [2023-12-27 18:56:02,027][20381] Updated weights for policy 1, policy_version 1916182 (0.0011) [2023-12-27 18:56:02,082][20381] Updated weights for policy 1, policy_version 1916192 (0.0011) [2023-12-27 18:56:02,325][20347] Updated weights for policy 0, policy_version 1917696 (0.0008) [2023-12-27 18:56:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 981622784. Throughput: 0: 10269.4, 1: 10365.0. Samples: 981635676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.840')] [2023-12-27 18:56:02,389][20347] Updated weights for policy 0, policy_version 1917706 (0.0008) [2023-12-27 18:56:02,448][20347] Updated weights for policy 0, policy_version 1917716 (0.0008) [2023-12-27 18:56:02,756][20381] Updated weights for policy 1, policy_version 1916202 (0.0006) [2023-12-27 18:56:02,813][20381] Updated weights for policy 1, policy_version 1916212 (0.0008) [2023-12-27 18:56:02,873][20381] Updated weights for policy 1, policy_version 1916222 (0.0006) [2023-12-27 18:56:03,118][20347] Updated weights for policy 0, policy_version 1917726 (0.0009) [2023-12-27 18:56:03,179][20347] Updated weights for policy 0, policy_version 1917736 (0.0009) [2023-12-27 18:56:03,231][20347] Updated weights for policy 0, policy_version 1917746 (0.0010) [2023-12-27 18:56:03,442][20381] Updated weights for policy 1, policy_version 1916232 (0.0007) [2023-12-27 18:56:03,488][20381] Updated weights for policy 1, policy_version 1916242 (0.0007) [2023-12-27 18:56:03,537][20381] Updated weights for policy 1, policy_version 1916252 (0.0007) [2023-12-27 18:56:03,806][20347] Updated weights for policy 0, policy_version 1917756 (0.0007) [2023-12-27 18:56:03,869][20347] Updated weights for policy 0, policy_version 1917766 (0.0009) [2023-12-27 18:56:03,923][20347] Updated weights for policy 0, policy_version 1917776 (0.0009) [2023-12-27 18:56:04,266][20381] Updated weights for policy 1, policy_version 1916262 (0.0007) [2023-12-27 18:56:04,326][20381] Updated weights for policy 1, policy_version 1916272 (0.0008) [2023-12-27 18:56:04,382][20381] Updated weights for policy 1, policy_version 1916282 (0.0008) [2023-12-27 18:56:04,624][20347] Updated weights for policy 0, policy_version 1917786 (0.0010) [2023-12-27 18:56:04,679][20347] Updated weights for policy 0, policy_version 1917796 (0.0010) [2023-12-27 18:56:04,734][20347] Updated weights for policy 0, policy_version 1917806 (0.0009) [2023-12-27 18:56:04,790][20347] Updated weights for policy 0, policy_version 1917816 (0.0010) [2023-12-27 18:56:05,058][20381] Updated weights for policy 1, policy_version 1916292 (0.0007) [2023-12-27 18:56:05,105][20381] Updated weights for policy 1, policy_version 1916302 (0.0007) [2023-12-27 18:56:05,150][20381] Updated weights for policy 1, policy_version 1916312 (0.0006) [2023-12-27 18:56:05,477][20347] Updated weights for policy 0, policy_version 1917826 (0.0010) [2023-12-27 18:56:05,524][20347] Updated weights for policy 0, policy_version 1917836 (0.0010) [2023-12-27 18:56:05,575][20347] Updated weights for policy 0, policy_version 1917846 (0.0010) [2023-12-27 18:56:05,854][20381] Updated weights for policy 1, policy_version 1916322 (0.0008) [2023-12-27 18:56:05,899][20381] Updated weights for policy 1, policy_version 1916332 (0.0007) [2023-12-27 18:56:05,947][20381] Updated weights for policy 1, policy_version 1916342 (0.0007) [2023-12-27 18:56:05,998][20381] Updated weights for policy 1, policy_version 1916352 (0.0007) [2023-12-27 18:56:06,222][20347] Updated weights for policy 0, policy_version 1917856 (0.0010) [2023-12-27 18:56:06,283][20347] Updated weights for policy 0, policy_version 1917866 (0.0010) [2023-12-27 18:56:06,345][20347] Updated weights for policy 0, policy_version 1917876 (0.0008) [2023-12-27 18:56:06,729][20381] Updated weights for policy 1, policy_version 1916362 (0.0008) [2023-12-27 18:56:06,793][20381] Updated weights for policy 1, policy_version 1916372 (0.0008) [2023-12-27 18:56:06,848][20381] Updated weights for policy 1, policy_version 1916382 (0.0008) [2023-12-27 18:56:07,005][20347] Updated weights for policy 0, policy_version 1917886 (0.0010) [2023-12-27 18:56:07,059][20347] Updated weights for policy 0, policy_version 1917896 (0.0010) [2023-12-27 18:56:07,118][20347] Updated weights for policy 0, policy_version 1917906 (0.0010) [2023-12-27 18:56:07,363][19170] Fps is (10 sec: 19659.8, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 981729280. Throughput: 0: 10261.4, 1: 10392.2. Samples: 981698412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '930.190')] [2023-12-27 18:56:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001916384_490651648.pth... [2023-12-27 18:56:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001917912_491077632.pth... [2023-12-27 18:56:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001915168_490340352.pth [2023-12-27 18:56:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001916728_490774528.pth [2023-12-27 18:56:07,559][20381] Updated weights for policy 1, policy_version 1916392 (0.0008) [2023-12-27 18:56:07,605][20381] Updated weights for policy 1, policy_version 1916402 (0.0008) [2023-12-27 18:56:07,655][20381] Updated weights for policy 1, policy_version 1916412 (0.0010) [2023-12-27 18:56:07,738][20347] Updated weights for policy 0, policy_version 1917916 (0.0008) [2023-12-27 18:56:07,792][20347] Updated weights for policy 0, policy_version 1917926 (0.0005) [2023-12-27 18:56:07,845][20347] Updated weights for policy 0, policy_version 1917936 (0.0005) [2023-12-27 18:56:08,440][20381] Updated weights for policy 1, policy_version 1916422 (0.0008) [2023-12-27 18:56:08,459][20347] Updated weights for policy 0, policy_version 1917946 (0.0006) [2023-12-27 18:56:08,494][20381] Updated weights for policy 1, policy_version 1916432 (0.0006) [2023-12-27 18:56:08,518][20347] Updated weights for policy 0, policy_version 1917956 (0.0010) [2023-12-27 18:56:08,553][20381] Updated weights for policy 1, policy_version 1916442 (0.0006) [2023-12-27 18:56:08,573][20347] Updated weights for policy 0, policy_version 1917966 (0.0008) [2023-12-27 18:56:08,636][20347] Updated weights for policy 0, policy_version 1917976 (0.0008) [2023-12-27 18:56:09,218][20381] Updated weights for policy 1, policy_version 1916452 (0.0007) [2023-12-27 18:56:09,285][20347] Updated weights for policy 0, policy_version 1917986 (0.0011) [2023-12-27 18:56:09,285][20381] Updated weights for policy 1, policy_version 1916462 (0.0010) [2023-12-27 18:56:09,340][20381] Updated weights for policy 1, policy_version 1916472 (0.0006) [2023-12-27 18:56:09,344][20347] Updated weights for policy 0, policy_version 1917996 (0.0010) [2023-12-27 18:56:09,408][20347] Updated weights for policy 0, policy_version 1918006 (0.0013) [2023-12-27 18:56:09,971][20381] Updated weights for policy 1, policy_version 1916482 (0.0009) [2023-12-27 18:56:10,032][20381] Updated weights for policy 1, policy_version 1916492 (0.0011) [2023-12-27 18:56:10,099][20381] Updated weights for policy 1, policy_version 1916502 (0.0011) [2023-12-27 18:56:10,164][20381] Updated weights for policy 1, policy_version 1916512 (0.0010) [2023-12-27 18:56:10,165][20347] Updated weights for policy 0, policy_version 1918016 (0.0007) [2023-12-27 18:56:10,243][20347] Updated weights for policy 0, policy_version 1918027 (0.0008) [2023-12-27 18:56:10,312][20347] Updated weights for policy 0, policy_version 1918037 (0.0007) [2023-12-27 18:56:10,857][20381] Updated weights for policy 1, policy_version 1916522 (0.0009) [2023-12-27 18:56:10,911][20381] Updated weights for policy 1, policy_version 1916532 (0.0007) [2023-12-27 18:56:10,963][20381] Updated weights for policy 1, policy_version 1916542 (0.0008) [2023-12-27 18:56:11,013][20347] Updated weights for policy 0, policy_version 1918047 (0.0008) [2023-12-27 18:56:11,077][20347] Updated weights for policy 0, policy_version 1918057 (0.0008) [2023-12-27 18:56:11,142][20347] Updated weights for policy 0, policy_version 1918067 (0.0009) [2023-12-27 18:56:11,749][20381] Updated weights for policy 1, policy_version 1916552 (0.0010) [2023-12-27 18:56:11,806][20381] Updated weights for policy 1, policy_version 1916562 (0.0010) [2023-12-27 18:56:11,868][20381] Updated weights for policy 1, policy_version 1916572 (0.0011) [2023-12-27 18:56:11,878][20347] Updated weights for policy 0, policy_version 1918077 (0.0008) [2023-12-27 18:56:11,935][20347] Updated weights for policy 0, policy_version 1918087 (0.0006) [2023-12-27 18:56:11,998][20347] Updated weights for policy 0, policy_version 1918097 (0.0006) [2023-12-27 18:56:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 981827584. Throughput: 0: 10273.8, 1: 10333.5. Samples: 981820288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '969.830')] [2023-12-27 18:56:12,617][20347] Updated weights for policy 0, policy_version 1918107 (0.0006) [2023-12-27 18:56:12,618][20381] Updated weights for policy 1, policy_version 1916582 (0.0010) [2023-12-27 18:56:12,669][20347] Updated weights for policy 0, policy_version 1918117 (0.0006) [2023-12-27 18:56:12,675][20381] Updated weights for policy 1, policy_version 1916592 (0.0010) [2023-12-27 18:56:12,718][20347] Updated weights for policy 0, policy_version 1918127 (0.0007) [2023-12-27 18:56:12,734][20381] Updated weights for policy 1, policy_version 1916602 (0.0010) [2023-12-27 18:56:13,395][20381] Updated weights for policy 1, policy_version 1916612 (0.0010) [2023-12-27 18:56:13,437][20347] Updated weights for policy 0, policy_version 1918137 (0.0005) [2023-12-27 18:56:13,446][20381] Updated weights for policy 1, policy_version 1916622 (0.0010) [2023-12-27 18:56:13,495][20347] Updated weights for policy 0, policy_version 1918147 (0.0009) [2023-12-27 18:56:13,498][20381] Updated weights for policy 1, policy_version 1916632 (0.0010) [2023-12-27 18:56:13,544][20347] Updated weights for policy 0, policy_version 1918157 (0.0007) [2023-12-27 18:56:13,604][20347] Updated weights for policy 0, policy_version 1918167 (0.0007) [2023-12-27 18:56:14,193][20381] Updated weights for policy 1, policy_version 1916642 (0.0010) [2023-12-27 18:56:14,217][20347] Updated weights for policy 0, policy_version 1918177 (0.0007) [2023-12-27 18:56:14,243][20381] Updated weights for policy 1, policy_version 1916652 (0.0010) [2023-12-27 18:56:14,269][20347] Updated weights for policy 0, policy_version 1918187 (0.0009) [2023-12-27 18:56:14,294][20381] Updated weights for policy 1, policy_version 1916662 (0.0007) [2023-12-27 18:56:14,320][20347] Updated weights for policy 0, policy_version 1918197 (0.0007) [2023-12-27 18:56:14,353][20381] Updated weights for policy 1, policy_version 1916672 (0.0006) [2023-12-27 18:56:14,983][20381] Updated weights for policy 1, policy_version 1916682 (0.0007) [2023-12-27 18:56:15,040][20381] Updated weights for policy 1, policy_version 1916692 (0.0007) [2023-12-27 18:56:15,053][20347] Updated weights for policy 0, policy_version 1918207 (0.0008) [2023-12-27 18:56:15,090][20381] Updated weights for policy 1, policy_version 1916702 (0.0005) [2023-12-27 18:56:15,106][20347] Updated weights for policy 0, policy_version 1918217 (0.0006) [2023-12-27 18:56:15,159][20347] Updated weights for policy 0, policy_version 1918227 (0.0008) [2023-12-27 18:56:15,714][20381] Updated weights for policy 1, policy_version 1916712 (0.0007) [2023-12-27 18:56:15,766][20381] Updated weights for policy 1, policy_version 1916722 (0.0008) [2023-12-27 18:56:15,825][20381] Updated weights for policy 1, policy_version 1916732 (0.0008) [2023-12-27 18:56:15,890][20347] Updated weights for policy 0, policy_version 1918237 (0.0007) [2023-12-27 18:56:15,939][20347] Updated weights for policy 0, policy_version 1918247 (0.0008) [2023-12-27 18:56:15,987][20347] Updated weights for policy 0, policy_version 1918257 (0.0007) [2023-12-27 18:56:16,547][20381] Updated weights for policy 1, policy_version 1916742 (0.0007) [2023-12-27 18:56:16,601][20381] Updated weights for policy 1, policy_version 1916752 (0.0007) [2023-12-27 18:56:16,604][20347] Updated weights for policy 0, policy_version 1918267 (0.0008) [2023-12-27 18:56:16,651][20381] Updated weights for policy 1, policy_version 1916762 (0.0005) [2023-12-27 18:56:16,653][20347] Updated weights for policy 0, policy_version 1918277 (0.0010) [2023-12-27 18:56:16,702][20347] Updated weights for policy 0, policy_version 1918287 (0.0005) [2023-12-27 18:56:17,255][20347] Updated weights for policy 0, policy_version 1918297 (0.0006) [2023-12-27 18:56:17,320][20347] Updated weights for policy 0, policy_version 1918307 (0.0008) [2023-12-27 18:56:17,362][19170] Fps is (10 sec: 19661.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 981925888. Throughput: 0: 10290.8, 1: 10299.0. Samples: 981942684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.020')] [2023-12-27 18:56:17,375][20347] Updated weights for policy 0, policy_version 1918317 (0.0006) [2023-12-27 18:56:17,388][20381] Updated weights for policy 1, policy_version 1916772 (0.0007) [2023-12-27 18:56:17,440][20347] Updated weights for policy 0, policy_version 1918327 (0.0008) [2023-12-27 18:56:17,441][20381] Updated weights for policy 1, policy_version 1916782 (0.0006) [2023-12-27 18:56:17,492][20381] Updated weights for policy 1, policy_version 1916792 (0.0009) [2023-12-27 18:56:18,003][20347] Updated weights for policy 0, policy_version 1918337 (0.0006) [2023-12-27 18:56:18,062][20347] Updated weights for policy 0, policy_version 1918347 (0.0005) [2023-12-27 18:56:18,119][20347] Updated weights for policy 0, policy_version 1918357 (0.0005) [2023-12-27 18:56:18,199][20381] Updated weights for policy 1, policy_version 1916802 (0.0007) [2023-12-27 18:56:18,247][20381] Updated weights for policy 1, policy_version 1916812 (0.0008) [2023-12-27 18:56:18,296][20381] Updated weights for policy 1, policy_version 1916822 (0.0009) [2023-12-27 18:56:18,359][20381] Updated weights for policy 1, policy_version 1916832 (0.0008) [2023-12-27 18:56:18,732][20347] Updated weights for policy 0, policy_version 1918367 (0.0007) [2023-12-27 18:56:18,788][20347] Updated weights for policy 0, policy_version 1918377 (0.0008) [2023-12-27 18:56:18,838][20347] Updated weights for policy 0, policy_version 1918387 (0.0008) [2023-12-27 18:56:19,038][20381] Updated weights for policy 1, policy_version 1916842 (0.0005) [2023-12-27 18:56:19,094][20381] Updated weights for policy 1, policy_version 1916852 (0.0006) [2023-12-27 18:56:19,151][20381] Updated weights for policy 1, policy_version 1916862 (0.0008) [2023-12-27 18:56:19,630][20347] Updated weights for policy 0, policy_version 1918397 (0.0006) [2023-12-27 18:56:19,690][20347] Updated weights for policy 0, policy_version 1918407 (0.0005) [2023-12-27 18:56:19,756][20347] Updated weights for policy 0, policy_version 1918417 (0.0007) [2023-12-27 18:56:19,846][20381] Updated weights for policy 1, policy_version 1916872 (0.0007) [2023-12-27 18:56:19,901][20381] Updated weights for policy 1, policy_version 1916882 (0.0008) [2023-12-27 18:56:19,957][20381] Updated weights for policy 1, policy_version 1916892 (0.0008) [2023-12-27 18:56:20,426][20347] Updated weights for policy 0, policy_version 1918427 (0.0009) [2023-12-27 18:56:20,489][20347] Updated weights for policy 0, policy_version 1918437 (0.0011) [2023-12-27 18:56:20,541][20347] Updated weights for policy 0, policy_version 1918447 (0.0010) [2023-12-27 18:56:20,709][20381] Updated weights for policy 1, policy_version 1916902 (0.0007) [2023-12-27 18:56:20,772][20381] Updated weights for policy 1, policy_version 1916912 (0.0007) [2023-12-27 18:56:20,825][20381] Updated weights for policy 1, policy_version 1916922 (0.0007) [2023-12-27 18:56:21,276][20347] Updated weights for policy 0, policy_version 1918457 (0.0010) [2023-12-27 18:56:21,339][20347] Updated weights for policy 0, policy_version 1918467 (0.0006) [2023-12-27 18:56:21,408][20347] Updated weights for policy 0, policy_version 1918477 (0.0008) [2023-12-27 18:56:21,469][20347] Updated weights for policy 0, policy_version 1918487 (0.0008) [2023-12-27 18:56:21,487][20381] Updated weights for policy 1, policy_version 1916932 (0.0007) [2023-12-27 18:56:21,547][20381] Updated weights for policy 1, policy_version 1916942 (0.0007) [2023-12-27 18:56:21,605][20381] Updated weights for policy 1, policy_version 1916952 (0.0009) [2023-12-27 18:56:22,198][20347] Updated weights for policy 0, policy_version 1918497 (0.0007) [2023-12-27 18:56:22,252][20347] Updated weights for policy 0, policy_version 1918507 (0.0008) [2023-12-27 18:56:22,259][20381] Updated weights for policy 1, policy_version 1916962 (0.0007) [2023-12-27 18:56:22,317][20347] Updated weights for policy 0, policy_version 1918517 (0.0008) [2023-12-27 18:56:22,317][20381] Updated weights for policy 1, policy_version 1916972 (0.0008) [2023-12-27 18:56:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 982032384. Throughput: 0: 10263.3, 1: 10257.7. Samples: 982004216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.010')] [2023-12-27 18:56:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001918520_491233280.pth... [2023-12-27 18:56:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001917304_490921984.pth [2023-12-27 18:56:22,381][20381] Updated weights for policy 1, policy_version 1916982 (0.0007) [2023-12-27 18:56:22,443][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001916992_490807296.pth... [2023-12-27 18:56:22,444][20381] Updated weights for policy 1, policy_version 1916992 (0.0007) [2023-12-27 18:56:22,446][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001915776_490496000.pth [2023-12-27 18:56:23,054][20347] Updated weights for policy 0, policy_version 1918527 (0.0006) [2023-12-27 18:56:23,109][20347] Updated weights for policy 0, policy_version 1918537 (0.0008) [2023-12-27 18:56:23,140][20381] Updated weights for policy 1, policy_version 1917002 (0.0005) [2023-12-27 18:56:23,170][20347] Updated weights for policy 0, policy_version 1918547 (0.0010) [2023-12-27 18:56:23,194][20381] Updated weights for policy 1, policy_version 1917012 (0.0005) [2023-12-27 18:56:23,248][20381] Updated weights for policy 1, policy_version 1917022 (0.0007) [2023-12-27 18:56:23,833][20347] Updated weights for policy 0, policy_version 1918557 (0.0010) [2023-12-27 18:56:23,838][20381] Updated weights for policy 1, policy_version 1917032 (0.0008) [2023-12-27 18:56:23,882][20347] Updated weights for policy 0, policy_version 1918567 (0.0010) [2023-12-27 18:56:23,902][20381] Updated weights for policy 1, policy_version 1917042 (0.0008) [2023-12-27 18:56:23,936][20347] Updated weights for policy 0, policy_version 1918577 (0.0010) [2023-12-27 18:56:23,951][20381] Updated weights for policy 1, policy_version 1917052 (0.0005) [2023-12-27 18:56:24,550][20381] Updated weights for policy 1, policy_version 1917062 (0.0007) [2023-12-27 18:56:24,608][20381] Updated weights for policy 1, policy_version 1917072 (0.0008) [2023-12-27 18:56:24,622][20347] Updated weights for policy 0, policy_version 1918587 (0.0010) [2023-12-27 18:56:24,658][20381] Updated weights for policy 1, policy_version 1917082 (0.0010) [2023-12-27 18:56:24,677][20347] Updated weights for policy 0, policy_version 1918597 (0.0010) [2023-12-27 18:56:24,732][20347] Updated weights for policy 0, policy_version 1918607 (0.0010) [2023-12-27 18:56:25,237][20381] Updated weights for policy 1, policy_version 1917092 (0.0011) [2023-12-27 18:56:25,285][20381] Updated weights for policy 1, policy_version 1917102 (0.0010) [2023-12-27 18:56:25,330][20381] Updated weights for policy 1, policy_version 1917112 (0.0010) [2023-12-27 18:56:25,368][20347] Updated weights for policy 0, policy_version 1918617 (0.0010) [2023-12-27 18:56:25,422][20347] Updated weights for policy 0, policy_version 1918627 (0.0010) [2023-12-27 18:56:25,473][20347] Updated weights for policy 0, policy_version 1918637 (0.0010) [2023-12-27 18:56:25,528][20347] Updated weights for policy 0, policy_version 1918647 (0.0010) [2023-12-27 18:56:25,986][20381] Updated weights for policy 1, policy_version 1917122 (0.0009) [2023-12-27 18:56:26,044][20381] Updated weights for policy 1, policy_version 1917132 (0.0009) [2023-12-27 18:56:26,095][20381] Updated weights for policy 1, policy_version 1917142 (0.0007) [2023-12-27 18:56:26,145][20381] Updated weights for policy 1, policy_version 1917152 (0.0005) [2023-12-27 18:56:26,164][20347] Updated weights for policy 0, policy_version 1918657 (0.0006) [2023-12-27 18:56:26,227][20347] Updated weights for policy 0, policy_version 1918667 (0.0005) [2023-12-27 18:56:26,280][20347] Updated weights for policy 0, policy_version 1918677 (0.0005) [2023-12-27 18:56:26,756][20381] Updated weights for policy 1, policy_version 1917162 (0.0011) [2023-12-27 18:56:26,818][20381] Updated weights for policy 1, policy_version 1917172 (0.0011) [2023-12-27 18:56:26,876][20381] Updated weights for policy 1, policy_version 1917182 (0.0009) [2023-12-27 18:56:26,905][20347] Updated weights for policy 0, policy_version 1918687 (0.0008) [2023-12-27 18:56:26,958][20347] Updated weights for policy 0, policy_version 1918697 (0.0007) [2023-12-27 18:56:27,016][20347] Updated weights for policy 0, policy_version 1918707 (0.0007) [2023-12-27 18:56:27,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 982138880. Throughput: 0: 10225.1, 1: 10356.5. Samples: 982128248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '950.980')] [2023-12-27 18:56:27,556][20381] Updated weights for policy 1, policy_version 1917192 (0.0009) [2023-12-27 18:56:27,604][20381] Updated weights for policy 1, policy_version 1917202 (0.0010) [2023-12-27 18:56:27,653][20381] Updated weights for policy 1, policy_version 1917212 (0.0007) [2023-12-27 18:56:27,687][20347] Updated weights for policy 0, policy_version 1918717 (0.0007) [2023-12-27 18:56:27,740][20347] Updated weights for policy 0, policy_version 1918727 (0.0008) [2023-12-27 18:56:27,801][20347] Updated weights for policy 0, policy_version 1918737 (0.0008) [2023-12-27 18:56:28,274][20381] Updated weights for policy 1, policy_version 1917222 (0.0008) [2023-12-27 18:56:28,337][20381] Updated weights for policy 1, policy_version 1917232 (0.0010) [2023-12-27 18:56:28,395][20381] Updated weights for policy 1, policy_version 1917242 (0.0010) [2023-12-27 18:56:28,546][20347] Updated weights for policy 0, policy_version 1918747 (0.0007) [2023-12-27 18:56:28,612][20347] Updated weights for policy 0, policy_version 1918757 (0.0005) [2023-12-27 18:56:28,668][20347] Updated weights for policy 0, policy_version 1918767 (0.0005) [2023-12-27 18:56:29,056][20381] Updated weights for policy 1, policy_version 1917252 (0.0007) [2023-12-27 18:56:29,119][20381] Updated weights for policy 1, policy_version 1917262 (0.0005) [2023-12-27 18:56:29,162][20381] Updated weights for policy 1, policy_version 1917272 (0.0007) [2023-12-27 18:56:29,288][20347] Updated weights for policy 0, policy_version 1918777 (0.0008) [2023-12-27 18:56:29,349][20347] Updated weights for policy 0, policy_version 1918787 (0.0009) [2023-12-27 18:56:29,412][20347] Updated weights for policy 0, policy_version 1918797 (0.0007) [2023-12-27 18:56:29,468][20347] Updated weights for policy 0, policy_version 1918807 (0.0007) [2023-12-27 18:56:29,740][20381] Updated weights for policy 1, policy_version 1917282 (0.0006) [2023-12-27 18:56:29,800][20381] Updated weights for policy 1, policy_version 1917292 (0.0006) [2023-12-27 18:56:29,863][20381] Updated weights for policy 1, policy_version 1917302 (0.0009) [2023-12-27 18:56:29,922][20381] Updated weights for policy 1, policy_version 1917312 (0.0008) [2023-12-27 18:56:30,143][20347] Updated weights for policy 0, policy_version 1918817 (0.0006) [2023-12-27 18:56:30,201][20347] Updated weights for policy 0, policy_version 1918827 (0.0005) [2023-12-27 18:56:30,260][20347] Updated weights for policy 0, policy_version 1918837 (0.0007) [2023-12-27 18:56:30,649][20381] Updated weights for policy 1, policy_version 1917322 (0.0010) [2023-12-27 18:56:30,706][20381] Updated weights for policy 1, policy_version 1917332 (0.0007) [2023-12-27 18:56:30,766][20381] Updated weights for policy 1, policy_version 1917342 (0.0005) [2023-12-27 18:56:30,867][20347] Updated weights for policy 0, policy_version 1918847 (0.0010) [2023-12-27 18:56:30,912][20347] Updated weights for policy 0, policy_version 1918857 (0.0010) [2023-12-27 18:56:30,956][20347] Updated weights for policy 0, policy_version 1918867 (0.0006) [2023-12-27 18:56:31,470][20381] Updated weights for policy 1, policy_version 1917352 (0.0007) [2023-12-27 18:56:31,526][20381] Updated weights for policy 1, policy_version 1917362 (0.0007) [2023-12-27 18:56:31,577][20381] Updated weights for policy 1, policy_version 1917372 (0.0007) [2023-12-27 18:56:31,682][20347] Updated weights for policy 0, policy_version 1918877 (0.0008) [2023-12-27 18:56:31,742][20347] Updated weights for policy 0, policy_version 1918887 (0.0010) [2023-12-27 18:56:31,802][20347] Updated weights for policy 0, policy_version 1918897 (0.0010) [2023-12-27 18:56:32,297][20381] Updated weights for policy 1, policy_version 1917382 (0.0007) [2023-12-27 18:56:32,356][20381] Updated weights for policy 1, policy_version 1917392 (0.0007) [2023-12-27 18:56:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 982237184. Throughput: 0: 10227.8, 1: 10320.9. Samples: 982252632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.340')] [2023-12-27 18:56:32,419][20381] Updated weights for policy 1, policy_version 1917402 (0.0007) [2023-12-27 18:56:32,505][20347] Updated weights for policy 0, policy_version 1918907 (0.0010) [2023-12-27 18:56:32,567][20347] Updated weights for policy 0, policy_version 1918917 (0.0010) [2023-12-27 18:56:32,631][20347] Updated weights for policy 0, policy_version 1918927 (0.0010) [2023-12-27 18:56:33,068][20381] Updated weights for policy 1, policy_version 1917412 (0.0009) [2023-12-27 18:56:33,125][20381] Updated weights for policy 1, policy_version 1917422 (0.0010) [2023-12-27 18:56:33,183][20381] Updated weights for policy 1, policy_version 1917432 (0.0010) [2023-12-27 18:56:33,253][20347] Updated weights for policy 0, policy_version 1918937 (0.0010) [2023-12-27 18:56:33,303][20347] Updated weights for policy 0, policy_version 1918947 (0.0007) [2023-12-27 18:56:33,351][20347] Updated weights for policy 0, policy_version 1918957 (0.0006) [2023-12-27 18:56:33,396][20347] Updated weights for policy 0, policy_version 1918967 (0.0008) [2023-12-27 18:56:33,851][20381] Updated weights for policy 1, policy_version 1917442 (0.0009) [2023-12-27 18:56:33,901][20381] Updated weights for policy 1, policy_version 1917452 (0.0005) [2023-12-27 18:56:33,953][20381] Updated weights for policy 1, policy_version 1917462 (0.0005) [2023-12-27 18:56:34,011][20381] Updated weights for policy 1, policy_version 1917472 (0.0005) [2023-12-27 18:56:34,026][20347] Updated weights for policy 0, policy_version 1918977 (0.0006) [2023-12-27 18:56:34,079][20347] Updated weights for policy 0, policy_version 1918987 (0.0005) [2023-12-27 18:56:34,150][20347] Updated weights for policy 0, policy_version 1918997 (0.0006) [2023-12-27 18:56:34,582][20381] Updated weights for policy 1, policy_version 1917482 (0.0005) [2023-12-27 18:56:34,638][20381] Updated weights for policy 1, policy_version 1917492 (0.0007) [2023-12-27 18:56:34,693][20381] Updated weights for policy 1, policy_version 1917502 (0.0008) [2023-12-27 18:56:34,836][20347] Updated weights for policy 0, policy_version 1919007 (0.0008) [2023-12-27 18:56:34,885][20347] Updated weights for policy 0, policy_version 1919017 (0.0007) [2023-12-27 18:56:34,930][20347] Updated weights for policy 0, policy_version 1919027 (0.0007) [2023-12-27 18:56:35,312][20381] Updated weights for policy 1, policy_version 1917512 (0.0007) [2023-12-27 18:56:35,360][20381] Updated weights for policy 1, policy_version 1917522 (0.0008) [2023-12-27 18:56:35,415][20381] Updated weights for policy 1, policy_version 1917532 (0.0007) [2023-12-27 18:56:35,568][20347] Updated weights for policy 0, policy_version 1919037 (0.0006) [2023-12-27 18:56:35,628][20347] Updated weights for policy 0, policy_version 1919047 (0.0005) [2023-12-27 18:56:35,682][20347] Updated weights for policy 0, policy_version 1919057 (0.0005) [2023-12-27 18:56:36,149][20381] Updated weights for policy 1, policy_version 1917542 (0.0008) [2023-12-27 18:56:36,205][20381] Updated weights for policy 1, policy_version 1917552 (0.0007) [2023-12-27 18:56:36,263][20381] Updated weights for policy 1, policy_version 1917562 (0.0005) [2023-12-27 18:56:36,340][20347] Updated weights for policy 0, policy_version 1919067 (0.0007) [2023-12-27 18:56:36,394][20347] Updated weights for policy 0, policy_version 1919077 (0.0007) [2023-12-27 18:56:36,444][20347] Updated weights for policy 0, policy_version 1919087 (0.0005) [2023-12-27 18:56:36,976][20381] Updated weights for policy 1, policy_version 1917572 (0.0007) [2023-12-27 18:56:37,035][20381] Updated weights for policy 1, policy_version 1917582 (0.0008) [2023-12-27 18:56:37,049][20347] Updated weights for policy 0, policy_version 1919097 (0.0006) [2023-12-27 18:56:37,096][20381] Updated weights for policy 1, policy_version 1917592 (0.0007) [2023-12-27 18:56:37,106][20347] Updated weights for policy 0, policy_version 1919107 (0.0006) [2023-12-27 18:56:37,164][20347] Updated weights for policy 0, policy_version 1919117 (0.0005) [2023-12-27 18:56:37,217][20347] Updated weights for policy 0, policy_version 1919127 (0.0005) [2023-12-27 18:56:37,363][19170] Fps is (10 sec: 21298.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 982351872. Throughput: 0: 10223.8, 1: 10359.1. Samples: 982316652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '931.270')] [2023-12-27 18:56:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001917600_490962944.pth... [2023-12-27 18:56:37,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001919128_491388928.pth... [2023-12-27 18:56:37,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001916384_490651648.pth [2023-12-27 18:56:37,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001917912_491077632.pth [2023-12-27 18:56:37,720][20381] Updated weights for policy 1, policy_version 1917602 (0.0008) [2023-12-27 18:56:37,774][20347] Updated weights for policy 0, policy_version 1919137 (0.0005) [2023-12-27 18:56:37,775][20381] Updated weights for policy 1, policy_version 1917612 (0.0010) [2023-12-27 18:56:37,828][20381] Updated weights for policy 1, policy_version 1917622 (0.0006) [2023-12-27 18:56:37,831][20347] Updated weights for policy 0, policy_version 1919147 (0.0007) [2023-12-27 18:56:37,883][20381] Updated weights for policy 1, policy_version 1917632 (0.0010) [2023-12-27 18:56:37,888][20347] Updated weights for policy 0, policy_version 1919157 (0.0005) [2023-12-27 18:56:38,547][20381] Updated weights for policy 1, policy_version 1917642 (0.0010) [2023-12-27 18:56:38,606][20381] Updated weights for policy 1, policy_version 1917652 (0.0011) [2023-12-27 18:56:38,613][20347] Updated weights for policy 0, policy_version 1919167 (0.0008) [2023-12-27 18:56:38,665][20381] Updated weights for policy 1, policy_version 1917662 (0.0011) [2023-12-27 18:56:38,668][20347] Updated weights for policy 0, policy_version 1919177 (0.0008) [2023-12-27 18:56:38,719][20347] Updated weights for policy 0, policy_version 1919187 (0.0008) [2023-12-27 18:56:39,318][20381] Updated weights for policy 1, policy_version 1917672 (0.0007) [2023-12-27 18:56:39,385][20381] Updated weights for policy 1, policy_version 1917682 (0.0008) [2023-12-27 18:56:39,448][20381] Updated weights for policy 1, policy_version 1917692 (0.0009) [2023-12-27 18:56:39,485][20347] Updated weights for policy 0, policy_version 1919197 (0.0008) [2023-12-27 18:56:39,543][20347] Updated weights for policy 0, policy_version 1919207 (0.0008) [2023-12-27 18:56:39,603][20347] Updated weights for policy 0, policy_version 1919217 (0.0008) [2023-12-27 18:56:40,098][20381] Updated weights for policy 1, policy_version 1917702 (0.0009) [2023-12-27 18:56:40,157][20381] Updated weights for policy 1, policy_version 1917712 (0.0011) [2023-12-27 18:56:40,209][20381] Updated weights for policy 1, policy_version 1917722 (0.0011) [2023-12-27 18:56:40,339][20347] Updated weights for policy 0, policy_version 1919227 (0.0009) [2023-12-27 18:56:40,397][20347] Updated weights for policy 0, policy_version 1919237 (0.0009) [2023-12-27 18:56:40,456][20347] Updated weights for policy 0, policy_version 1919247 (0.0007) [2023-12-27 18:56:40,874][20381] Updated weights for policy 1, policy_version 1917732 (0.0011) [2023-12-27 18:56:40,920][20381] Updated weights for policy 1, policy_version 1917742 (0.0010) [2023-12-27 18:56:40,975][20381] Updated weights for policy 1, policy_version 1917752 (0.0010) [2023-12-27 18:56:41,103][20347] Updated weights for policy 0, policy_version 1919257 (0.0008) [2023-12-27 18:56:41,165][20347] Updated weights for policy 0, policy_version 1919267 (0.0008) [2023-12-27 18:56:41,227][20347] Updated weights for policy 0, policy_version 1919277 (0.0008) [2023-12-27 18:56:41,296][20347] Updated weights for policy 0, policy_version 1919287 (0.0009) [2023-12-27 18:56:41,815][20381] Updated weights for policy 1, policy_version 1917762 (0.0010) [2023-12-27 18:56:41,874][20381] Updated weights for policy 1, policy_version 1917772 (0.0008) [2023-12-27 18:56:41,934][20381] Updated weights for policy 1, policy_version 1917782 (0.0008) [2023-12-27 18:56:41,990][20381] Updated weights for policy 1, policy_version 1917792 (0.0008) [2023-12-27 18:56:42,037][20347] Updated weights for policy 0, policy_version 1919297 (0.0008) [2023-12-27 18:56:42,086][20347] Updated weights for policy 0, policy_version 1919307 (0.0009) [2023-12-27 18:56:42,137][20347] Updated weights for policy 0, policy_version 1919317 (0.0007) [2023-12-27 18:56:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 982450176. Throughput: 0: 10256.6, 1: 10280.2. Samples: 982440748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.380')] [2023-12-27 18:56:42,715][20381] Updated weights for policy 1, policy_version 1917802 (0.0005) [2023-12-27 18:56:42,779][20381] Updated weights for policy 1, policy_version 1917812 (0.0006) [2023-12-27 18:56:42,843][20381] Updated weights for policy 1, policy_version 1917822 (0.0005) [2023-12-27 18:56:42,883][20347] Updated weights for policy 0, policy_version 1919327 (0.0008) [2023-12-27 18:56:42,948][20347] Updated weights for policy 0, policy_version 1919337 (0.0006) [2023-12-27 18:56:43,018][20347] Updated weights for policy 0, policy_version 1919348 (0.0011) [2023-12-27 18:56:43,402][20381] Updated weights for policy 1, policy_version 1917832 (0.0006) [2023-12-27 18:56:43,463][20381] Updated weights for policy 1, policy_version 1917842 (0.0006) [2023-12-27 18:56:43,526][20381] Updated weights for policy 1, policy_version 1917852 (0.0007) [2023-12-27 18:56:43,701][20347] Updated weights for policy 0, policy_version 1919358 (0.0007) [2023-12-27 18:56:43,757][20347] Updated weights for policy 0, policy_version 1919368 (0.0007) [2023-12-27 18:56:43,820][20347] Updated weights for policy 0, policy_version 1919378 (0.0008) [2023-12-27 18:56:44,152][20381] Updated weights for policy 1, policy_version 1917862 (0.0008) [2023-12-27 18:56:44,217][20381] Updated weights for policy 1, policy_version 1917872 (0.0010) [2023-12-27 18:56:44,274][20381] Updated weights for policy 1, policy_version 1917882 (0.0010) [2023-12-27 18:56:44,454][20347] Updated weights for policy 0, policy_version 1919388 (0.0006) [2023-12-27 18:56:44,503][20347] Updated weights for policy 0, policy_version 1919398 (0.0005) [2023-12-27 18:56:44,554][20347] Updated weights for policy 0, policy_version 1919408 (0.0006) [2023-12-27 18:56:44,938][20381] Updated weights for policy 1, policy_version 1917892 (0.0010) [2023-12-27 18:56:44,994][20381] Updated weights for policy 1, policy_version 1917902 (0.0011) [2023-12-27 18:56:45,057][20381] Updated weights for policy 1, policy_version 1917912 (0.0011) [2023-12-27 18:56:45,255][20347] Updated weights for policy 0, policy_version 1919418 (0.0008) [2023-12-27 18:56:45,310][20347] Updated weights for policy 0, policy_version 1919428 (0.0009) [2023-12-27 18:56:45,362][20347] Updated weights for policy 0, policy_version 1919438 (0.0008) [2023-12-27 18:56:45,413][20347] Updated weights for policy 0, policy_version 1919448 (0.0007) [2023-12-27 18:56:45,662][20381] Updated weights for policy 1, policy_version 1917922 (0.0010) [2023-12-27 18:56:45,722][20381] Updated weights for policy 1, policy_version 1917932 (0.0011) [2023-12-27 18:56:45,782][20381] Updated weights for policy 1, policy_version 1917942 (0.0010) [2023-12-27 18:56:45,836][20381] Updated weights for policy 1, policy_version 1917952 (0.0007) [2023-12-27 18:56:46,082][20347] Updated weights for policy 0, policy_version 1919458 (0.0009) [2023-12-27 18:56:46,147][20347] Updated weights for policy 0, policy_version 1919468 (0.0007) [2023-12-27 18:56:46,203][20347] Updated weights for policy 0, policy_version 1919478 (0.0010) [2023-12-27 18:56:46,422][20381] Updated weights for policy 1, policy_version 1917962 (0.0010) [2023-12-27 18:56:46,483][20381] Updated weights for policy 1, policy_version 1917972 (0.0010) [2023-12-27 18:56:46,541][20381] Updated weights for policy 1, policy_version 1917982 (0.0011) [2023-12-27 18:56:46,853][20347] Updated weights for policy 0, policy_version 1919488 (0.0009) [2023-12-27 18:56:46,908][20347] Updated weights for policy 0, policy_version 1919498 (0.0010) [2023-12-27 18:56:46,971][20347] Updated weights for policy 0, policy_version 1919508 (0.0010) [2023-12-27 18:56:47,167][20381] Updated weights for policy 1, policy_version 1917992 (0.0009) [2023-12-27 18:56:47,224][20381] Updated weights for policy 1, policy_version 1918002 (0.0006) [2023-12-27 18:56:47,277][20381] Updated weights for policy 1, policy_version 1918012 (0.0005) [2023-12-27 18:56:47,363][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 982556672. Throughput: 0: 10285.6, 1: 10342.0. Samples: 982563920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '901.930')] [2023-12-27 18:56:47,649][20347] Updated weights for policy 0, policy_version 1919518 (0.0011) [2023-12-27 18:56:47,700][20347] Updated weights for policy 0, policy_version 1919528 (0.0010) [2023-12-27 18:56:47,755][20347] Updated weights for policy 0, policy_version 1919538 (0.0010) [2023-12-27 18:56:47,866][20381] Updated weights for policy 1, policy_version 1918022 (0.0009) [2023-12-27 18:56:47,921][20381] Updated weights for policy 1, policy_version 1918032 (0.0011) [2023-12-27 18:56:47,969][20381] Updated weights for policy 1, policy_version 1918042 (0.0010) [2023-12-27 18:56:48,326][20347] Updated weights for policy 0, policy_version 1919548 (0.0010) [2023-12-27 18:56:48,393][20347] Updated weights for policy 0, policy_version 1919559 (0.0008) [2023-12-27 18:56:48,450][20347] Updated weights for policy 0, policy_version 1919569 (0.0008) [2023-12-27 18:56:48,608][20381] Updated weights for policy 1, policy_version 1918052 (0.0011) [2023-12-27 18:56:48,654][20381] Updated weights for policy 1, policy_version 1918062 (0.0010) [2023-12-27 18:56:48,710][20381] Updated weights for policy 1, policy_version 1918072 (0.0009) [2023-12-27 18:56:49,099][20347] Updated weights for policy 0, policy_version 1919579 (0.0009) [2023-12-27 18:56:49,150][20347] Updated weights for policy 0, policy_version 1919589 (0.0005) [2023-12-27 18:56:49,200][20347] Updated weights for policy 0, policy_version 1919599 (0.0006) [2023-12-27 18:56:49,309][20381] Updated weights for policy 1, policy_version 1918082 (0.0007) [2023-12-27 18:56:49,374][20381] Updated weights for policy 1, policy_version 1918092 (0.0008) [2023-12-27 18:56:49,437][20381] Updated weights for policy 1, policy_version 1918102 (0.0007) [2023-12-27 18:56:49,498][20381] Updated weights for policy 1, policy_version 1918112 (0.0007) [2023-12-27 18:56:49,855][20347] Updated weights for policy 0, policy_version 1919609 (0.0011) [2023-12-27 18:56:49,924][20347] Updated weights for policy 0, policy_version 1919619 (0.0011) [2023-12-27 18:56:49,989][20347] Updated weights for policy 0, policy_version 1919629 (0.0010) [2023-12-27 18:56:50,058][20347] Updated weights for policy 0, policy_version 1919639 (0.0010) [2023-12-27 18:56:50,109][20381] Updated weights for policy 1, policy_version 1918122 (0.0006) [2023-12-27 18:56:50,161][20381] Updated weights for policy 1, policy_version 1918132 (0.0007) [2023-12-27 18:56:50,210][20381] Updated weights for policy 1, policy_version 1918142 (0.0008) [2023-12-27 18:56:50,727][20347] Updated weights for policy 0, policy_version 1919649 (0.0008) [2023-12-27 18:56:50,785][20347] Updated weights for policy 0, policy_version 1919659 (0.0008) [2023-12-27 18:56:50,843][20347] Updated weights for policy 0, policy_version 1919669 (0.0009) [2023-12-27 18:56:50,886][20381] Updated weights for policy 1, policy_version 1918152 (0.0006) [2023-12-27 18:56:50,945][20381] Updated weights for policy 1, policy_version 1918162 (0.0008) [2023-12-27 18:56:51,003][20381] Updated weights for policy 1, policy_version 1918172 (0.0008) [2023-12-27 18:56:51,557][20347] Updated weights for policy 0, policy_version 1919679 (0.0007) [2023-12-27 18:56:51,625][20347] Updated weights for policy 0, policy_version 1919689 (0.0006) [2023-12-27 18:56:51,686][20347] Updated weights for policy 0, policy_version 1919699 (0.0008) [2023-12-27 18:56:51,750][20381] Updated weights for policy 1, policy_version 1918182 (0.0007) [2023-12-27 18:56:51,810][20381] Updated weights for policy 1, policy_version 1918192 (0.0008) [2023-12-27 18:56:51,863][20381] Updated weights for policy 1, policy_version 1918202 (0.0008) [2023-12-27 18:56:52,255][20347] Updated weights for policy 0, policy_version 1919709 (0.0007) [2023-12-27 18:56:52,308][20347] Updated weights for policy 0, policy_version 1919719 (0.0006) [2023-12-27 18:56:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 982654976. Throughput: 0: 10302.1, 1: 10369.5. Samples: 982628628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.660')] [2023-12-27 18:56:52,364][20347] Updated weights for policy 0, policy_version 1919729 (0.0006) [2023-12-27 18:56:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001918208_491118592.pth... [2023-12-27 18:56:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001916992_490807296.pth [2023-12-27 18:56:52,416][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001919736_491544576.pth... [2023-12-27 18:56:52,420][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001918520_491233280.pth [2023-12-27 18:56:52,620][20381] Updated weights for policy 1, policy_version 1918212 (0.0008) [2023-12-27 18:56:52,681][20381] Updated weights for policy 1, policy_version 1918222 (0.0009) [2023-12-27 18:56:52,743][20381] Updated weights for policy 1, policy_version 1918232 (0.0008) [2023-12-27 18:56:53,015][20347] Updated weights for policy 0, policy_version 1919739 (0.0009) [2023-12-27 18:56:53,077][20347] Updated weights for policy 0, policy_version 1919749 (0.0007) [2023-12-27 18:56:53,129][20347] Updated weights for policy 0, policy_version 1919759 (0.0008) [2023-12-27 18:56:53,398][20381] Updated weights for policy 1, policy_version 1918242 (0.0006) [2023-12-27 18:56:53,452][20381] Updated weights for policy 1, policy_version 1918252 (0.0007) [2023-12-27 18:56:53,510][20381] Updated weights for policy 1, policy_version 1918262 (0.0006) [2023-12-27 18:56:53,564][20381] Updated weights for policy 1, policy_version 1918272 (0.0006) [2023-12-27 18:56:53,825][20347] Updated weights for policy 0, policy_version 1919769 (0.0008) [2023-12-27 18:56:53,874][20347] Updated weights for policy 0, policy_version 1919779 (0.0008) [2023-12-27 18:56:53,926][20347] Updated weights for policy 0, policy_version 1919789 (0.0007) [2023-12-27 18:56:53,982][20347] Updated weights for policy 0, policy_version 1919799 (0.0008) [2023-12-27 18:56:54,216][20381] Updated weights for policy 1, policy_version 1918282 (0.0008) [2023-12-27 18:56:54,275][20381] Updated weights for policy 1, policy_version 1918292 (0.0007) [2023-12-27 18:56:54,330][20381] Updated weights for policy 1, policy_version 1918302 (0.0008) [2023-12-27 18:56:54,698][20347] Updated weights for policy 0, policy_version 1919809 (0.0006) [2023-12-27 18:56:54,763][20347] Updated weights for policy 0, policy_version 1919819 (0.0008) [2023-12-27 18:56:54,823][20347] Updated weights for policy 0, policy_version 1919829 (0.0010) [2023-12-27 18:56:54,952][20381] Updated weights for policy 1, policy_version 1918312 (0.0007) [2023-12-27 18:56:55,011][20381] Updated weights for policy 1, policy_version 1918322 (0.0008) [2023-12-27 18:56:55,064][20381] Updated weights for policy 1, policy_version 1918332 (0.0008) [2023-12-27 18:56:55,498][20347] Updated weights for policy 0, policy_version 1919839 (0.0010) [2023-12-27 18:56:55,558][20347] Updated weights for policy 0, policy_version 1919849 (0.0010) [2023-12-27 18:56:55,612][20347] Updated weights for policy 0, policy_version 1919859 (0.0010) [2023-12-27 18:56:55,759][20381] Updated weights for policy 1, policy_version 1918342 (0.0007) [2023-12-27 18:56:55,803][20381] Updated weights for policy 1, policy_version 1918352 (0.0007) [2023-12-27 18:56:55,851][20381] Updated weights for policy 1, policy_version 1918362 (0.0007) [2023-12-27 18:56:56,286][20347] Updated weights for policy 0, policy_version 1919869 (0.0010) [2023-12-27 18:56:56,331][20347] Updated weights for policy 0, policy_version 1919879 (0.0010) [2023-12-27 18:56:56,378][20347] Updated weights for policy 0, policy_version 1919889 (0.0010) [2023-12-27 18:56:56,563][20381] Updated weights for policy 1, policy_version 1918372 (0.0007) [2023-12-27 18:56:56,619][20381] Updated weights for policy 1, policy_version 1918382 (0.0007) [2023-12-27 18:56:56,675][20381] Updated weights for policy 1, policy_version 1918392 (0.0007) [2023-12-27 18:56:57,032][20347] Updated weights for policy 0, policy_version 1919899 (0.0009) [2023-12-27 18:56:57,095][20347] Updated weights for policy 0, policy_version 1919909 (0.0006) [2023-12-27 18:56:57,160][20347] Updated weights for policy 0, policy_version 1919919 (0.0010) [2023-12-27 18:56:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 982761472. Throughput: 0: 10284.7, 1: 10388.9. Samples: 982750600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-12-27 18:56:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.910')] [2023-12-27 18:56:57,414][20381] Updated weights for policy 1, policy_version 1918402 (0.0007) [2023-12-27 18:56:57,473][20381] Updated weights for policy 1, policy_version 1918412 (0.0008) [2023-12-27 18:56:57,535][20381] Updated weights for policy 1, policy_version 1918422 (0.0007) [2023-12-27 18:56:57,588][20381] Updated weights for policy 1, policy_version 1918432 (0.0007) [2023-12-27 18:56:57,800][20347] Updated weights for policy 0, policy_version 1919929 (0.0010) [2023-12-27 18:56:57,858][20347] Updated weights for policy 0, policy_version 1919939 (0.0009) [2023-12-27 18:56:57,903][20347] Updated weights for policy 0, policy_version 1919949 (0.0005) [2023-12-27 18:56:57,957][20347] Updated weights for policy 0, policy_version 1919959 (0.0010) [2023-12-27 18:56:58,249][20381] Updated weights for policy 1, policy_version 1918442 (0.0008) [2023-12-27 18:56:58,300][20381] Updated weights for policy 1, policy_version 1918452 (0.0007) [2023-12-27 18:56:58,373][20381] Updated weights for policy 1, policy_version 1918462 (0.0008) [2023-12-27 18:56:58,686][20347] Updated weights for policy 0, policy_version 1919969 (0.0010) [2023-12-27 18:56:58,750][20347] Updated weights for policy 0, policy_version 1919979 (0.0011) [2023-12-27 18:56:58,813][20347] Updated weights for policy 0, policy_version 1919989 (0.0015) [2023-12-27 18:56:59,132][20381] Updated weights for policy 1, policy_version 1918472 (0.0006) [2023-12-27 18:56:59,202][20381] Updated weights for policy 1, policy_version 1918482 (0.0007) [2023-12-27 18:56:59,273][20381] Updated weights for policy 1, policy_version 1918492 (0.0008) [2023-12-27 18:56:59,653][20347] Updated weights for policy 0, policy_version 1919999 (0.0006) [2023-12-27 18:56:59,699][20347] Updated weights for policy 0, policy_version 1920009 (0.0006) [2023-12-27 18:56:59,758][20347] Updated weights for policy 0, policy_version 1920019 (0.0005) [2023-12-27 18:56:59,985][20381] Updated weights for policy 1, policy_version 1918502 (0.0009) [2023-12-27 18:57:00,038][20381] Updated weights for policy 1, policy_version 1918512 (0.0009) [2023-12-27 18:57:00,117][20381] Updated weights for policy 1, policy_version 1918522 (0.0008) [2023-12-27 18:57:00,352][20347] Updated weights for policy 0, policy_version 1920029 (0.0006) [2023-12-27 18:57:00,415][20347] Updated weights for policy 0, policy_version 1920039 (0.0005) [2023-12-27 18:57:00,466][20347] Updated weights for policy 0, policy_version 1920049 (0.0005) [2023-12-27 18:57:00,784][20381] Updated weights for policy 1, policy_version 1918532 (0.0008) [2023-12-27 18:57:00,838][20381] Updated weights for policy 1, policy_version 1918542 (0.0008) [2023-12-27 18:57:00,885][20381] Updated weights for policy 1, policy_version 1918552 (0.0007) [2023-12-27 18:57:00,988][20347] Updated weights for policy 0, policy_version 1920059 (0.0005) [2023-12-27 18:57:01,044][20347] Updated weights for policy 0, policy_version 1920069 (0.0006) [2023-12-27 18:57:01,108][20347] Updated weights for policy 0, policy_version 1920079 (0.0005) [2023-12-27 18:57:01,661][20381] Updated weights for policy 1, policy_version 1918562 (0.0008) [2023-12-27 18:57:01,720][20381] Updated weights for policy 1, policy_version 1918572 (0.0006) [2023-12-27 18:57:01,762][20347] Updated weights for policy 0, policy_version 1920089 (0.0006) [2023-12-27 18:57:01,783][20381] Updated weights for policy 1, policy_version 1918582 (0.0007) [2023-12-27 18:57:01,816][20347] Updated weights for policy 0, policy_version 1920099 (0.0008) [2023-12-27 18:57:01,843][20381] Updated weights for policy 1, policy_version 1918592 (0.0005) [2023-12-27 18:57:01,875][20347] Updated weights for policy 0, policy_version 1920109 (0.0009) [2023-12-27 18:57:01,931][20347] Updated weights for policy 0, policy_version 1920119 (0.0007) [2023-12-27 18:57:02,363][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 982859776. Throughput: 0: 10295.4, 1: 10346.8. Samples: 982871588. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.350')] [2023-12-27 18:57:02,492][20381] Updated weights for policy 1, policy_version 1918602 (0.0006) [2023-12-27 18:57:02,550][20381] Updated weights for policy 1, policy_version 1918612 (0.0008) [2023-12-27 18:57:02,600][20381] Updated weights for policy 1, policy_version 1918622 (0.0008) [2023-12-27 18:57:02,691][20347] Updated weights for policy 0, policy_version 1920129 (0.0007) [2023-12-27 18:57:02,747][20347] Updated weights for policy 0, policy_version 1920139 (0.0008) [2023-12-27 18:57:02,807][20347] Updated weights for policy 0, policy_version 1920149 (0.0007) [2023-12-27 18:57:03,224][20381] Updated weights for policy 1, policy_version 1918632 (0.0005) [2023-12-27 18:57:03,286][20381] Updated weights for policy 1, policy_version 1918642 (0.0005) [2023-12-27 18:57:03,347][20381] Updated weights for policy 1, policy_version 1918652 (0.0005) [2023-12-27 18:57:03,458][20347] Updated weights for policy 0, policy_version 1920159 (0.0009) [2023-12-27 18:57:03,507][20347] Updated weights for policy 0, policy_version 1920169 (0.0010) [2023-12-27 18:57:03,558][20347] Updated weights for policy 0, policy_version 1920179 (0.0010) [2023-12-27 18:57:03,897][20381] Updated weights for policy 1, policy_version 1918662 (0.0007) [2023-12-27 18:57:03,958][20381] Updated weights for policy 1, policy_version 1918672 (0.0005) [2023-12-27 18:57:04,018][20381] Updated weights for policy 1, policy_version 1918682 (0.0006) [2023-12-27 18:57:04,272][20347] Updated weights for policy 0, policy_version 1920189 (0.0009) [2023-12-27 18:57:04,334][20347] Updated weights for policy 0, policy_version 1920199 (0.0010) [2023-12-27 18:57:04,392][20347] Updated weights for policy 0, policy_version 1920209 (0.0010) [2023-12-27 18:57:04,654][20381] Updated weights for policy 1, policy_version 1918692 (0.0006) [2023-12-27 18:57:04,713][20381] Updated weights for policy 1, policy_version 1918702 (0.0007) [2023-12-27 18:57:04,772][20381] Updated weights for policy 1, policy_version 1918712 (0.0008) [2023-12-27 18:57:05,065][20347] Updated weights for policy 0, policy_version 1920219 (0.0009) [2023-12-27 18:57:05,112][20347] Updated weights for policy 0, policy_version 1920229 (0.0010) [2023-12-27 18:57:05,163][20347] Updated weights for policy 0, policy_version 1920239 (0.0010) [2023-12-27 18:57:05,400][20381] Updated weights for policy 1, policy_version 1918722 (0.0007) [2023-12-27 18:57:05,455][20381] Updated weights for policy 1, policy_version 1918732 (0.0008) [2023-12-27 18:57:05,510][20381] Updated weights for policy 1, policy_version 1918742 (0.0008) [2023-12-27 18:57:05,571][20381] Updated weights for policy 1, policy_version 1918752 (0.0006) [2023-12-27 18:57:05,846][20347] Updated weights for policy 0, policy_version 1920249 (0.0010) [2023-12-27 18:57:05,891][20347] Updated weights for policy 0, policy_version 1920259 (0.0010) [2023-12-27 18:57:05,966][20347] Updated weights for policy 0, policy_version 1920269 (0.0010) [2023-12-27 18:57:06,017][20347] Updated weights for policy 0, policy_version 1920279 (0.0010) [2023-12-27 18:57:06,160][20381] Updated weights for policy 1, policy_version 1918762 (0.0008) [2023-12-27 18:57:06,213][20381] Updated weights for policy 1, policy_version 1918772 (0.0009) [2023-12-27 18:57:06,275][20381] Updated weights for policy 1, policy_version 1918782 (0.0008) [2023-12-27 18:57:06,682][20347] Updated weights for policy 0, policy_version 1920289 (0.0011) [2023-12-27 18:57:06,738][20347] Updated weights for policy 0, policy_version 1920299 (0.0010) [2023-12-27 18:57:06,796][20347] Updated weights for policy 0, policy_version 1920309 (0.0010) [2023-12-27 18:57:06,934][20381] Updated weights for policy 1, policy_version 1918792 (0.0007) [2023-12-27 18:57:06,983][20381] Updated weights for policy 1, policy_version 1918802 (0.0007) [2023-12-27 18:57:07,034][20381] Updated weights for policy 1, policy_version 1918812 (0.0007) [2023-12-27 18:57:07,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.7, 300 sec: 20577.2). Total num frames: 982966272. Throughput: 0: 10267.2, 1: 10422.8. Samples: 982935264. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.910')] [2023-12-27 18:57:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001918816_491274240.pth... [2023-12-27 18:57:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001920312_491692032.pth... [2023-12-27 18:57:07,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001917600_490962944.pth [2023-12-27 18:57:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001919128_491388928.pth [2023-12-27 18:57:07,450][20347] Updated weights for policy 0, policy_version 1920319 (0.0007) [2023-12-27 18:57:07,503][20347] Updated weights for policy 0, policy_version 1920329 (0.0006) [2023-12-27 18:57:07,554][20347] Updated weights for policy 0, policy_version 1920339 (0.0010) [2023-12-27 18:57:07,763][20381] Updated weights for policy 1, policy_version 1918822 (0.0008) [2023-12-27 18:57:07,829][20381] Updated weights for policy 1, policy_version 1918832 (0.0008) [2023-12-27 18:57:07,881][20381] Updated weights for policy 1, policy_version 1918842 (0.0008) [2023-12-27 18:57:08,144][20347] Updated weights for policy 0, policy_version 1920349 (0.0009) [2023-12-27 18:57:08,191][20347] Updated weights for policy 0, policy_version 1920359 (0.0007) [2023-12-27 18:57:08,239][20347] Updated weights for policy 0, policy_version 1920369 (0.0007) [2023-12-27 18:57:08,485][20381] Updated weights for policy 1, policy_version 1918852 (0.0008) [2023-12-27 18:57:08,545][20381] Updated weights for policy 1, policy_version 1918862 (0.0005) [2023-12-27 18:57:08,610][20381] Updated weights for policy 1, policy_version 1918872 (0.0006) [2023-12-27 18:57:08,999][20347] Updated weights for policy 0, policy_version 1920379 (0.0008) [2023-12-27 18:57:09,050][20347] Updated weights for policy 0, policy_version 1920389 (0.0008) [2023-12-27 18:57:09,099][20347] Updated weights for policy 0, policy_version 1920399 (0.0008) [2023-12-27 18:57:09,236][20381] Updated weights for policy 1, policy_version 1918882 (0.0007) [2023-12-27 18:57:09,296][20381] Updated weights for policy 1, policy_version 1918892 (0.0008) [2023-12-27 18:57:09,357][20381] Updated weights for policy 1, policy_version 1918902 (0.0008) [2023-12-27 18:57:09,428][20381] Updated weights for policy 1, policy_version 1918912 (0.0007) [2023-12-27 18:57:09,834][20347] Updated weights for policy 0, policy_version 1920409 (0.0007) [2023-12-27 18:57:09,895][20347] Updated weights for policy 0, policy_version 1920419 (0.0008) [2023-12-27 18:57:09,958][20347] Updated weights for policy 0, policy_version 1920429 (0.0008) [2023-12-27 18:57:10,023][20347] Updated weights for policy 0, policy_version 1920439 (0.0008) [2023-12-27 18:57:10,168][20381] Updated weights for policy 1, policy_version 1918922 (0.0007) [2023-12-27 18:57:10,227][20381] Updated weights for policy 1, policy_version 1918932 (0.0006) [2023-12-27 18:57:10,285][20381] Updated weights for policy 1, policy_version 1918942 (0.0006) [2023-12-27 18:57:10,745][20347] Updated weights for policy 0, policy_version 1920449 (0.0008) [2023-12-27 18:57:10,804][20347] Updated weights for policy 0, policy_version 1920459 (0.0005) [2023-12-27 18:57:10,857][20347] Updated weights for policy 0, policy_version 1920469 (0.0005) [2023-12-27 18:57:10,944][20381] Updated weights for policy 1, policy_version 1918952 (0.0009) [2023-12-27 18:57:10,997][20381] Updated weights for policy 1, policy_version 1918962 (0.0008) [2023-12-27 18:57:11,062][20381] Updated weights for policy 1, policy_version 1918972 (0.0009) [2023-12-27 18:57:11,487][20347] Updated weights for policy 0, policy_version 1920479 (0.0007) [2023-12-27 18:57:11,558][20347] Updated weights for policy 0, policy_version 1920489 (0.0008) [2023-12-27 18:57:11,623][20347] Updated weights for policy 0, policy_version 1920499 (0.0008) [2023-12-27 18:57:11,815][20381] Updated weights for policy 1, policy_version 1918982 (0.0008) [2023-12-27 18:57:11,877][20381] Updated weights for policy 1, policy_version 1918992 (0.0008) [2023-12-27 18:57:11,941][20381] Updated weights for policy 1, policy_version 1919002 (0.0007) [2023-12-27 18:57:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 983064576. Throughput: 0: 10314.5, 1: 10366.7. Samples: 983058904. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.060')] [2023-12-27 18:57:12,367][20347] Updated weights for policy 0, policy_version 1920509 (0.0009) [2023-12-27 18:57:12,431][20347] Updated weights for policy 0, policy_version 1920519 (0.0006) [2023-12-27 18:57:12,494][20347] Updated weights for policy 0, policy_version 1920529 (0.0010) [2023-12-27 18:57:12,657][20381] Updated weights for policy 1, policy_version 1919012 (0.0007) [2023-12-27 18:57:12,715][20381] Updated weights for policy 1, policy_version 1919022 (0.0007) [2023-12-27 18:57:12,765][20381] Updated weights for policy 1, policy_version 1919032 (0.0006) [2023-12-27 18:57:13,182][20347] Updated weights for policy 0, policy_version 1920539 (0.0009) [2023-12-27 18:57:13,250][20347] Updated weights for policy 0, policy_version 1920549 (0.0006) [2023-12-27 18:57:13,300][20347] Updated weights for policy 0, policy_version 1920559 (0.0007) [2023-12-27 18:57:13,401][20381] Updated weights for policy 1, policy_version 1919042 (0.0007) [2023-12-27 18:57:13,451][20381] Updated weights for policy 1, policy_version 1919052 (0.0007) [2023-12-27 18:57:13,507][20381] Updated weights for policy 1, policy_version 1919062 (0.0005) [2023-12-27 18:57:13,562][20381] Updated weights for policy 1, policy_version 1919072 (0.0006) [2023-12-27 18:57:13,992][20347] Updated weights for policy 0, policy_version 1920569 (0.0007) [2023-12-27 18:57:14,039][20347] Updated weights for policy 0, policy_version 1920579 (0.0005) [2023-12-27 18:57:14,089][20347] Updated weights for policy 0, policy_version 1920589 (0.0006) [2023-12-27 18:57:14,142][20347] Updated weights for policy 0, policy_version 1920599 (0.0007) [2023-12-27 18:57:14,155][20381] Updated weights for policy 1, policy_version 1919082 (0.0008) [2023-12-27 18:57:14,213][20381] Updated weights for policy 1, policy_version 1919092 (0.0007) [2023-12-27 18:57:14,270][20381] Updated weights for policy 1, policy_version 1919102 (0.0007) [2023-12-27 18:57:14,754][20347] Updated weights for policy 0, policy_version 1920609 (0.0008) [2023-12-27 18:57:14,820][20347] Updated weights for policy 0, policy_version 1920619 (0.0007) [2023-12-27 18:57:14,874][20347] Updated weights for policy 0, policy_version 1920629 (0.0008) [2023-12-27 18:57:14,992][20381] Updated weights for policy 1, policy_version 1919112 (0.0008) [2023-12-27 18:57:15,051][20381] Updated weights for policy 1, policy_version 1919122 (0.0008) [2023-12-27 18:57:15,105][20381] Updated weights for policy 1, policy_version 1919132 (0.0008) [2023-12-27 18:57:15,482][20347] Updated weights for policy 0, policy_version 1920639 (0.0006) [2023-12-27 18:57:15,538][20347] Updated weights for policy 0, policy_version 1920649 (0.0006) [2023-12-27 18:57:15,594][20347] Updated weights for policy 0, policy_version 1920659 (0.0007) [2023-12-27 18:57:15,820][20381] Updated weights for policy 1, policy_version 1919142 (0.0007) [2023-12-27 18:57:15,868][20381] Updated weights for policy 1, policy_version 1919152 (0.0010) [2023-12-27 18:57:15,914][20381] Updated weights for policy 1, policy_version 1919162 (0.0010) [2023-12-27 18:57:16,249][20347] Updated weights for policy 0, policy_version 1920669 (0.0009) [2023-12-27 18:57:16,310][20347] Updated weights for policy 0, policy_version 1920679 (0.0010) [2023-12-27 18:57:16,367][20347] Updated weights for policy 0, policy_version 1920689 (0.0010) [2023-12-27 18:57:16,553][20381] Updated weights for policy 1, policy_version 1919172 (0.0009) [2023-12-27 18:57:16,614][20381] Updated weights for policy 1, policy_version 1919182 (0.0011) [2023-12-27 18:57:16,670][20381] Updated weights for policy 1, policy_version 1919192 (0.0007) [2023-12-27 18:57:17,002][20347] Updated weights for policy 0, policy_version 1920699 (0.0006) [2023-12-27 18:57:17,059][20347] Updated weights for policy 0, policy_version 1920709 (0.0010) [2023-12-27 18:57:17,114][20347] Updated weights for policy 0, policy_version 1920719 (0.0010) [2023-12-27 18:57:17,279][20381] Updated weights for policy 1, policy_version 1919202 (0.0006) [2023-12-27 18:57:17,328][20381] Updated weights for policy 1, policy_version 1919212 (0.0010) [2023-12-27 18:57:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 983171072. Throughput: 0: 10315.1, 1: 10366.5. Samples: 983183304. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.450')] [2023-12-27 18:57:17,382][20381] Updated weights for policy 1, policy_version 1919222 (0.0010) [2023-12-27 18:57:17,430][20381] Updated weights for policy 1, policy_version 1919232 (0.0010) [2023-12-27 18:57:17,810][20347] Updated weights for policy 0, policy_version 1920729 (0.0010) [2023-12-27 18:57:17,862][20347] Updated weights for policy 0, policy_version 1920739 (0.0007) [2023-12-27 18:57:17,912][20347] Updated weights for policy 0, policy_version 1920749 (0.0009) [2023-12-27 18:57:17,972][20347] Updated weights for policy 0, policy_version 1920759 (0.0005) [2023-12-27 18:57:18,047][20381] Updated weights for policy 1, policy_version 1919242 (0.0006) [2023-12-27 18:57:18,104][20381] Updated weights for policy 1, policy_version 1919252 (0.0007) [2023-12-27 18:57:18,151][20381] Updated weights for policy 1, policy_version 1919262 (0.0006) [2023-12-27 18:57:18,622][20347] Updated weights for policy 0, policy_version 1920769 (0.0010) [2023-12-27 18:57:18,686][20347] Updated weights for policy 0, policy_version 1920779 (0.0010) [2023-12-27 18:57:18,741][20347] Updated weights for policy 0, policy_version 1920789 (0.0008) [2023-12-27 18:57:18,743][20381] Updated weights for policy 1, policy_version 1919272 (0.0008) [2023-12-27 18:57:18,800][20381] Updated weights for policy 1, policy_version 1919282 (0.0006) [2023-12-27 18:57:18,864][20381] Updated weights for policy 1, policy_version 1919292 (0.0006) [2023-12-27 18:57:19,416][20347] Updated weights for policy 0, policy_version 1920799 (0.0010) [2023-12-27 18:57:19,465][20347] Updated weights for policy 0, policy_version 1920809 (0.0010) [2023-12-27 18:57:19,517][20347] Updated weights for policy 0, policy_version 1920819 (0.0008) [2023-12-27 18:57:19,550][20381] Updated weights for policy 1, policy_version 1919302 (0.0007) [2023-12-27 18:57:19,605][20381] Updated weights for policy 1, policy_version 1919312 (0.0008) [2023-12-27 18:57:19,652][20381] Updated weights for policy 1, policy_version 1919322 (0.0006) [2023-12-27 18:57:20,276][20347] Updated weights for policy 0, policy_version 1920829 (0.0009) [2023-12-27 18:57:20,338][20347] Updated weights for policy 0, policy_version 1920839 (0.0011) [2023-12-27 18:57:20,398][20347] Updated weights for policy 0, policy_version 1920849 (0.0009) [2023-12-27 18:57:20,408][20381] Updated weights for policy 1, policy_version 1919332 (0.0007) [2023-12-27 18:57:20,466][20381] Updated weights for policy 1, policy_version 1919342 (0.0007) [2023-12-27 18:57:20,520][20381] Updated weights for policy 1, policy_version 1919352 (0.0007) [2023-12-27 18:57:21,152][20347] Updated weights for policy 0, policy_version 1920859 (0.0008) [2023-12-27 18:57:21,152][20381] Updated weights for policy 1, policy_version 1919362 (0.0009) [2023-12-27 18:57:21,207][20381] Updated weights for policy 1, policy_version 1919372 (0.0006) [2023-12-27 18:57:21,211][20347] Updated weights for policy 0, policy_version 1920869 (0.0009) [2023-12-27 18:57:21,261][20381] Updated weights for policy 1, policy_version 1919382 (0.0007) [2023-12-27 18:57:21,271][20347] Updated weights for policy 0, policy_version 1920879 (0.0008) [2023-12-27 18:57:21,316][20381] Updated weights for policy 1, policy_version 1919392 (0.0007) [2023-12-27 18:57:21,994][20347] Updated weights for policy 0, policy_version 1920889 (0.0008) [2023-12-27 18:57:22,029][20381] Updated weights for policy 1, policy_version 1919402 (0.0007) [2023-12-27 18:57:22,049][20347] Updated weights for policy 0, policy_version 1920899 (0.0008) [2023-12-27 18:57:22,084][20381] Updated weights for policy 1, policy_version 1919412 (0.0007) [2023-12-27 18:57:22,108][20347] Updated weights for policy 0, policy_version 1920909 (0.0008) [2023-12-27 18:57:22,144][20381] Updated weights for policy 1, policy_version 1919422 (0.0008) [2023-12-27 18:57:22,166][20347] Updated weights for policy 0, policy_version 1920919 (0.0005) [2023-12-27 18:57:22,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 983277568. Throughput: 0: 10289.7, 1: 10336.3. Samples: 983244820. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.540')] [2023-12-27 18:57:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001919424_491429888.pth... [2023-12-27 18:57:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001920920_491847680.pth... [2023-12-27 18:57:22,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001918208_491118592.pth [2023-12-27 18:57:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001919736_491544576.pth [2023-12-27 18:57:22,806][20347] Updated weights for policy 0, policy_version 1920929 (0.0007) [2023-12-27 18:57:22,855][20381] Updated weights for policy 1, policy_version 1919432 (0.0007) [2023-12-27 18:57:22,864][20347] Updated weights for policy 0, policy_version 1920939 (0.0007) [2023-12-27 18:57:22,910][20381] Updated weights for policy 1, policy_version 1919442 (0.0008) [2023-12-27 18:57:22,917][20347] Updated weights for policy 0, policy_version 1920949 (0.0005) [2023-12-27 18:57:22,966][20381] Updated weights for policy 1, policy_version 1919452 (0.0010) [2023-12-27 18:57:23,557][20347] Updated weights for policy 0, policy_version 1920959 (0.0006) [2023-12-27 18:57:23,616][20347] Updated weights for policy 0, policy_version 1920969 (0.0005) [2023-12-27 18:57:23,619][20381] Updated weights for policy 1, policy_version 1919462 (0.0005) [2023-12-27 18:57:23,663][20381] Updated weights for policy 1, policy_version 1919472 (0.0007) [2023-12-27 18:57:23,669][20347] Updated weights for policy 0, policy_version 1920979 (0.0005) [2023-12-27 18:57:23,714][20381] Updated weights for policy 1, policy_version 1919482 (0.0011) [2023-12-27 18:57:24,258][20347] Updated weights for policy 0, policy_version 1920989 (0.0007) [2023-12-27 18:57:24,314][20347] Updated weights for policy 0, policy_version 1920999 (0.0005) [2023-12-27 18:57:24,376][20381] Updated weights for policy 1, policy_version 1919492 (0.0009) [2023-12-27 18:57:24,376][20347] Updated weights for policy 0, policy_version 1921009 (0.0006) [2023-12-27 18:57:24,432][20381] Updated weights for policy 1, policy_version 1919502 (0.0005) [2023-12-27 18:57:24,491][20381] Updated weights for policy 1, policy_version 1919512 (0.0005) [2023-12-27 18:57:24,949][20347] Updated weights for policy 0, policy_version 1921019 (0.0007) [2023-12-27 18:57:25,006][20347] Updated weights for policy 0, policy_version 1921029 (0.0006) [2023-12-27 18:57:25,061][20347] Updated weights for policy 0, policy_version 1921039 (0.0008) [2023-12-27 18:57:25,125][20381] Updated weights for policy 1, policy_version 1919522 (0.0006) [2023-12-27 18:57:25,178][20381] Updated weights for policy 1, policy_version 1919532 (0.0008) [2023-12-27 18:57:25,227][20381] Updated weights for policy 1, policy_version 1919542 (0.0008) [2023-12-27 18:57:25,291][20381] Updated weights for policy 1, policy_version 1919552 (0.0008) [2023-12-27 18:57:25,642][20347] Updated weights for policy 0, policy_version 1921049 (0.0008) [2023-12-27 18:57:25,702][20347] Updated weights for policy 0, policy_version 1921059 (0.0006) [2023-12-27 18:57:25,761][20347] Updated weights for policy 0, policy_version 1921069 (0.0007) [2023-12-27 18:57:25,819][20347] Updated weights for policy 0, policy_version 1921079 (0.0008) [2023-12-27 18:57:25,997][20381] Updated weights for policy 1, policy_version 1919562 (0.0006) [2023-12-27 18:57:26,048][20381] Updated weights for policy 1, policy_version 1919572 (0.0005) [2023-12-27 18:57:26,106][20381] Updated weights for policy 1, policy_version 1919582 (0.0005) [2023-12-27 18:57:26,398][20347] Updated weights for policy 0, policy_version 1921089 (0.0006) [2023-12-27 18:57:26,453][20347] Updated weights for policy 0, policy_version 1921099 (0.0006) [2023-12-27 18:57:26,506][20347] Updated weights for policy 0, policy_version 1921109 (0.0005) [2023-12-27 18:57:26,778][20381] Updated weights for policy 1, policy_version 1919592 (0.0007) [2023-12-27 18:57:26,836][20381] Updated weights for policy 1, policy_version 1919602 (0.0008) [2023-12-27 18:57:26,884][20381] Updated weights for policy 1, policy_version 1919612 (0.0010) [2023-12-27 18:57:27,053][20347] Updated weights for policy 0, policy_version 1921119 (0.0007) [2023-12-27 18:57:27,113][20347] Updated weights for policy 0, policy_version 1921129 (0.0006) [2023-12-27 18:57:27,173][20347] Updated weights for policy 0, policy_version 1921139 (0.0006) [2023-12-27 18:57:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 983384064. Throughput: 0: 10311.6, 1: 10364.7. Samples: 983371184. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '950.170')] [2023-12-27 18:57:27,556][20381] Updated weights for policy 1, policy_version 1919622 (0.0007) [2023-12-27 18:57:27,616][20381] Updated weights for policy 1, policy_version 1919632 (0.0005) [2023-12-27 18:57:27,670][20381] Updated weights for policy 1, policy_version 1919642 (0.0005) [2023-12-27 18:57:27,881][20347] Updated weights for policy 0, policy_version 1921149 (0.0008) [2023-12-27 18:57:27,940][20347] Updated weights for policy 0, policy_version 1921159 (0.0008) [2023-12-27 18:57:27,995][20347] Updated weights for policy 0, policy_version 1921169 (0.0008) [2023-12-27 18:57:28,293][20381] Updated weights for policy 1, policy_version 1919652 (0.0007) [2023-12-27 18:57:28,360][20381] Updated weights for policy 1, policy_version 1919662 (0.0007) [2023-12-27 18:57:28,416][20381] Updated weights for policy 1, policy_version 1919672 (0.0007) [2023-12-27 18:57:28,670][20347] Updated weights for policy 0, policy_version 1921179 (0.0008) [2023-12-27 18:57:28,717][20347] Updated weights for policy 0, policy_version 1921189 (0.0008) [2023-12-27 18:57:28,776][20347] Updated weights for policy 0, policy_version 1921199 (0.0008) [2023-12-27 18:57:29,104][20381] Updated weights for policy 1, policy_version 1919682 (0.0007) [2023-12-27 18:57:29,159][20381] Updated weights for policy 1, policy_version 1919692 (0.0008) [2023-12-27 18:57:29,215][20381] Updated weights for policy 1, policy_version 1919702 (0.0007) [2023-12-27 18:57:29,275][20381] Updated weights for policy 1, policy_version 1919712 (0.0008) [2023-12-27 18:57:29,469][20347] Updated weights for policy 0, policy_version 1921209 (0.0007) [2023-12-27 18:57:29,523][20347] Updated weights for policy 0, policy_version 1921219 (0.0005) [2023-12-27 18:57:29,579][20347] Updated weights for policy 0, policy_version 1921229 (0.0005) [2023-12-27 18:57:29,631][20347] Updated weights for policy 0, policy_version 1921239 (0.0005) [2023-12-27 18:57:30,013][20381] Updated weights for policy 1, policy_version 1919722 (0.0009) [2023-12-27 18:57:30,073][20381] Updated weights for policy 1, policy_version 1919732 (0.0008) [2023-12-27 18:57:30,133][20381] Updated weights for policy 1, policy_version 1919742 (0.0008) [2023-12-27 18:57:30,254][20347] Updated weights for policy 0, policy_version 1921249 (0.0008) [2023-12-27 18:57:30,318][20347] Updated weights for policy 0, policy_version 1921259 (0.0008) [2023-12-27 18:57:30,378][20347] Updated weights for policy 0, policy_version 1921269 (0.0008) [2023-12-27 18:57:30,801][20381] Updated weights for policy 1, policy_version 1919752 (0.0007) [2023-12-27 18:57:30,855][20381] Updated weights for policy 1, policy_version 1919762 (0.0008) [2023-12-27 18:57:30,912][20381] Updated weights for policy 1, policy_version 1919772 (0.0007) [2023-12-27 18:57:31,067][20347] Updated weights for policy 0, policy_version 1921279 (0.0008) [2023-12-27 18:57:31,126][20347] Updated weights for policy 0, policy_version 1921289 (0.0007) [2023-12-27 18:57:31,193][20347] Updated weights for policy 0, policy_version 1921299 (0.0007) [2023-12-27 18:57:31,664][20381] Updated weights for policy 1, policy_version 1919782 (0.0008) [2023-12-27 18:57:31,714][20381] Updated weights for policy 1, policy_version 1919792 (0.0009) [2023-12-27 18:57:31,779][20381] Updated weights for policy 1, policy_version 1919802 (0.0008) [2023-12-27 18:57:31,926][20347] Updated weights for policy 0, policy_version 1921309 (0.0008) [2023-12-27 18:57:31,993][20347] Updated weights for policy 0, policy_version 1921319 (0.0008) [2023-12-27 18:57:32,046][20347] Updated weights for policy 0, policy_version 1921329 (0.0005) [2023-12-27 18:57:32,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 983482368. Throughput: 0: 10364.6, 1: 10286.4. Samples: 983493212. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.200')] [2023-12-27 18:57:32,484][20381] Updated weights for policy 1, policy_version 1919812 (0.0007) [2023-12-27 18:57:32,537][20381] Updated weights for policy 1, policy_version 1919822 (0.0007) [2023-12-27 18:57:32,589][20381] Updated weights for policy 1, policy_version 1919832 (0.0009) [2023-12-27 18:57:32,667][20347] Updated weights for policy 0, policy_version 1921339 (0.0007) [2023-12-27 18:57:32,723][20347] Updated weights for policy 0, policy_version 1921349 (0.0005) [2023-12-27 18:57:32,778][20347] Updated weights for policy 0, policy_version 1921359 (0.0005) [2023-12-27 18:57:33,316][20381] Updated weights for policy 1, policy_version 1919842 (0.0009) [2023-12-27 18:57:33,377][20381] Updated weights for policy 1, policy_version 1919852 (0.0007) [2023-12-27 18:57:33,417][20347] Updated weights for policy 0, policy_version 1921369 (0.0007) [2023-12-27 18:57:33,428][20381] Updated weights for policy 1, policy_version 1919862 (0.0008) [2023-12-27 18:57:33,474][20347] Updated weights for policy 0, policy_version 1921379 (0.0006) [2023-12-27 18:57:33,484][20381] Updated weights for policy 1, policy_version 1919872 (0.0007) [2023-12-27 18:57:33,532][20347] Updated weights for policy 0, policy_version 1921389 (0.0005) [2023-12-27 18:57:33,583][20347] Updated weights for policy 0, policy_version 1921399 (0.0005) [2023-12-27 18:57:34,170][20347] Updated weights for policy 0, policy_version 1921409 (0.0007) [2023-12-27 18:57:34,224][20381] Updated weights for policy 1, policy_version 1919882 (0.0006) [2023-12-27 18:57:34,226][20347] Updated weights for policy 0, policy_version 1921419 (0.0005) [2023-12-27 18:57:34,280][20381] Updated weights for policy 1, policy_version 1919892 (0.0007) [2023-12-27 18:57:34,286][20347] Updated weights for policy 0, policy_version 1921429 (0.0006) [2023-12-27 18:57:34,342][20381] Updated weights for policy 1, policy_version 1919902 (0.0008) [2023-12-27 18:57:34,870][20347] Updated weights for policy 0, policy_version 1921439 (0.0007) [2023-12-27 18:57:34,925][20347] Updated weights for policy 0, policy_version 1921449 (0.0008) [2023-12-27 18:57:34,984][20347] Updated weights for policy 0, policy_version 1921459 (0.0008) [2023-12-27 18:57:35,077][20381] Updated weights for policy 1, policy_version 1919912 (0.0008) [2023-12-27 18:57:35,136][20381] Updated weights for policy 1, policy_version 1919922 (0.0007) [2023-12-27 18:57:35,192][20381] Updated weights for policy 1, policy_version 1919932 (0.0008) [2023-12-27 18:57:35,578][20347] Updated weights for policy 0, policy_version 1921469 (0.0008) [2023-12-27 18:57:35,636][20347] Updated weights for policy 0, policy_version 1921479 (0.0007) [2023-12-27 18:57:35,694][20347] Updated weights for policy 0, policy_version 1921489 (0.0008) [2023-12-27 18:57:35,843][20381] Updated weights for policy 1, policy_version 1919942 (0.0008) [2023-12-27 18:57:35,904][20381] Updated weights for policy 1, policy_version 1919952 (0.0006) [2023-12-27 18:57:35,961][20381] Updated weights for policy 1, policy_version 1919962 (0.0005) [2023-12-27 18:57:36,344][20347] Updated weights for policy 0, policy_version 1921499 (0.0007) [2023-12-27 18:57:36,394][20347] Updated weights for policy 0, policy_version 1921509 (0.0008) [2023-12-27 18:57:36,452][20347] Updated weights for policy 0, policy_version 1921519 (0.0007) [2023-12-27 18:57:36,682][20381] Updated weights for policy 1, policy_version 1919972 (0.0007) [2023-12-27 18:57:36,736][20381] Updated weights for policy 1, policy_version 1919982 (0.0007) [2023-12-27 18:57:36,785][20381] Updated weights for policy 1, policy_version 1919992 (0.0005) [2023-12-27 18:57:37,195][20347] Updated weights for policy 0, policy_version 1921529 (0.0008) [2023-12-27 18:57:37,252][20347] Updated weights for policy 0, policy_version 1921539 (0.0008) [2023-12-27 18:57:37,298][20347] Updated weights for policy 0, policy_version 1921549 (0.0010) [2023-12-27 18:57:37,351][20347] Updated weights for policy 0, policy_version 1921559 (0.0008) [2023-12-27 18:57:37,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 983588864. Throughput: 0: 10390.0, 1: 10188.7. Samples: 983554676. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:37,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.650')] [2023-12-27 18:57:37,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001921560_492011520.pth... [2023-12-27 18:57:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001920312_491692032.pth [2023-12-27 18:57:37,376][20381] Updated weights for policy 1, policy_version 1920002 (0.0006) [2023-12-27 18:57:37,425][20381] Updated weights for policy 1, policy_version 1920012 (0.0008) [2023-12-27 18:57:37,480][20381] Updated weights for policy 1, policy_version 1920022 (0.0008) [2023-12-27 18:57:37,526][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001920032_491585536.pth... [2023-12-27 18:57:37,527][20381] Updated weights for policy 1, policy_version 1920032 (0.0008) [2023-12-27 18:57:37,530][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001918816_491274240.pth [2023-12-27 18:57:38,084][20347] Updated weights for policy 0, policy_version 1921569 (0.0009) [2023-12-27 18:57:38,139][20347] Updated weights for policy 0, policy_version 1921579 (0.0008) [2023-12-27 18:57:38,198][20347] Updated weights for policy 0, policy_version 1921589 (0.0009) [2023-12-27 18:57:38,237][20381] Updated weights for policy 1, policy_version 1920042 (0.0007) [2023-12-27 18:57:38,293][20381] Updated weights for policy 1, policy_version 1920052 (0.0008) [2023-12-27 18:57:38,354][20381] Updated weights for policy 1, policy_version 1920062 (0.0006) [2023-12-27 18:57:38,858][20347] Updated weights for policy 0, policy_version 1921599 (0.0007) [2023-12-27 18:57:38,912][20347] Updated weights for policy 0, policy_version 1921609 (0.0008) [2023-12-27 18:57:38,962][20347] Updated weights for policy 0, policy_version 1921619 (0.0008) [2023-12-27 18:57:39,051][20381] Updated weights for policy 1, policy_version 1920072 (0.0007) [2023-12-27 18:57:39,110][20381] Updated weights for policy 1, policy_version 1920082 (0.0006) [2023-12-27 18:57:39,164][20381] Updated weights for policy 1, policy_version 1920092 (0.0008) [2023-12-27 18:57:39,734][20347] Updated weights for policy 0, policy_version 1921629 (0.0009) [2023-12-27 18:57:39,783][20347] Updated weights for policy 0, policy_version 1921639 (0.0009) [2023-12-27 18:57:39,814][20381] Updated weights for policy 1, policy_version 1920102 (0.0007) [2023-12-27 18:57:39,853][20347] Updated weights for policy 0, policy_version 1921649 (0.0007) [2023-12-27 18:57:39,879][20381] Updated weights for policy 1, policy_version 1920112 (0.0007) [2023-12-27 18:57:39,943][20381] Updated weights for policy 1, policy_version 1920122 (0.0007) [2023-12-27 18:57:40,592][20347] Updated weights for policy 0, policy_version 1921659 (0.0006) [2023-12-27 18:57:40,647][20347] Updated weights for policy 0, policy_version 1921669 (0.0008) [2023-12-27 18:57:40,697][20381] Updated weights for policy 1, policy_version 1920132 (0.0008) [2023-12-27 18:57:40,712][20347] Updated weights for policy 0, policy_version 1921679 (0.0008) [2023-12-27 18:57:40,759][20381] Updated weights for policy 1, policy_version 1920142 (0.0005) [2023-12-27 18:57:40,822][20381] Updated weights for policy 1, policy_version 1920152 (0.0005) [2023-12-27 18:57:41,475][20347] Updated weights for policy 0, policy_version 1921689 (0.0007) [2023-12-27 18:57:41,514][20381] Updated weights for policy 1, policy_version 1920162 (0.0007) [2023-12-27 18:57:41,530][20347] Updated weights for policy 0, policy_version 1921699 (0.0007) [2023-12-27 18:57:41,574][20381] Updated weights for policy 1, policy_version 1920172 (0.0008) [2023-12-27 18:57:41,588][20347] Updated weights for policy 0, policy_version 1921709 (0.0006) [2023-12-27 18:57:41,641][20381] Updated weights for policy 1, policy_version 1920182 (0.0008) [2023-12-27 18:57:41,655][20347] Updated weights for policy 0, policy_version 1921719 (0.0008) [2023-12-27 18:57:41,709][20381] Updated weights for policy 1, policy_version 1920192 (0.0009) [2023-12-27 18:57:42,336][20347] Updated weights for policy 0, policy_version 1921729 (0.0007) [2023-12-27 18:57:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 983678976. Throughput: 0: 10354.6, 1: 10204.8. Samples: 983675772. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.170')] [2023-12-27 18:57:42,402][20347] Updated weights for policy 0, policy_version 1921739 (0.0007) [2023-12-27 18:57:42,426][20381] Updated weights for policy 1, policy_version 1920202 (0.0011) [2023-12-27 18:57:42,466][20347] Updated weights for policy 0, policy_version 1921749 (0.0006) [2023-12-27 18:57:42,488][20381] Updated weights for policy 1, policy_version 1920212 (0.0010) [2023-12-27 18:57:42,545][20381] Updated weights for policy 1, policy_version 1920222 (0.0011) [2023-12-27 18:57:43,207][20347] Updated weights for policy 0, policy_version 1921759 (0.0007) [2023-12-27 18:57:43,231][20381] Updated weights for policy 1, policy_version 1920232 (0.0011) [2023-12-27 18:57:43,263][20347] Updated weights for policy 0, policy_version 1921769 (0.0006) [2023-12-27 18:57:43,279][20381] Updated weights for policy 1, policy_version 1920242 (0.0011) [2023-12-27 18:57:43,310][20347] Updated weights for policy 0, policy_version 1921779 (0.0007) [2023-12-27 18:57:43,325][20381] Updated weights for policy 1, policy_version 1920252 (0.0010) [2023-12-27 18:57:43,954][20347] Updated weights for policy 0, policy_version 1921789 (0.0007) [2023-12-27 18:57:44,005][20347] Updated weights for policy 0, policy_version 1921799 (0.0005) [2023-12-27 18:57:44,044][20381] Updated weights for policy 1, policy_version 1920262 (0.0009) [2023-12-27 18:57:44,057][20347] Updated weights for policy 0, policy_version 1921809 (0.0005) [2023-12-27 18:57:44,098][20381] Updated weights for policy 1, policy_version 1920272 (0.0009) [2023-12-27 18:57:44,151][20381] Updated weights for policy 1, policy_version 1920282 (0.0007) [2023-12-27 18:57:44,637][20347] Updated weights for policy 0, policy_version 1921819 (0.0005) [2023-12-27 18:57:44,685][20347] Updated weights for policy 0, policy_version 1921829 (0.0006) [2023-12-27 18:57:44,738][20347] Updated weights for policy 0, policy_version 1921839 (0.0005) [2023-12-27 18:57:44,877][20381] Updated weights for policy 1, policy_version 1920292 (0.0011) [2023-12-27 18:57:44,941][20381] Updated weights for policy 1, policy_version 1920302 (0.0011) [2023-12-27 18:57:45,005][20381] Updated weights for policy 1, policy_version 1920312 (0.0011) [2023-12-27 18:57:45,328][20347] Updated weights for policy 0, policy_version 1921849 (0.0006) [2023-12-27 18:57:45,380][20347] Updated weights for policy 0, policy_version 1921859 (0.0007) [2023-12-27 18:57:45,433][20347] Updated weights for policy 0, policy_version 1921869 (0.0007) [2023-12-27 18:57:45,481][20347] Updated weights for policy 0, policy_version 1921879 (0.0007) [2023-12-27 18:57:45,694][20381] Updated weights for policy 1, policy_version 1920322 (0.0011) [2023-12-27 18:57:45,754][20381] Updated weights for policy 1, policy_version 1920332 (0.0011) [2023-12-27 18:57:45,816][20381] Updated weights for policy 1, policy_version 1920342 (0.0010) [2023-12-27 18:57:45,872][20381] Updated weights for policy 1, policy_version 1920352 (0.0010) [2023-12-27 18:57:46,190][20347] Updated weights for policy 0, policy_version 1921889 (0.0005) [2023-12-27 18:57:46,250][20347] Updated weights for policy 0, policy_version 1921899 (0.0006) [2023-12-27 18:57:46,309][20347] Updated weights for policy 0, policy_version 1921909 (0.0005) [2023-12-27 18:57:46,439][20381] Updated weights for policy 1, policy_version 1920362 (0.0005) [2023-12-27 18:57:46,497][20381] Updated weights for policy 1, policy_version 1920372 (0.0010) [2023-12-27 18:57:46,559][20381] Updated weights for policy 1, policy_version 1920382 (0.0010) [2023-12-27 18:57:46,839][20347] Updated weights for policy 0, policy_version 1921919 (0.0005) [2023-12-27 18:57:46,886][20347] Updated weights for policy 0, policy_version 1921929 (0.0006) [2023-12-27 18:57:46,929][20347] Updated weights for policy 0, policy_version 1921939 (0.0007) [2023-12-27 18:57:47,230][20381] Updated weights for policy 1, policy_version 1920392 (0.0010) [2023-12-27 18:57:47,288][20381] Updated weights for policy 1, policy_version 1920402 (0.0011) [2023-12-27 18:57:47,344][20381] Updated weights for policy 1, policy_version 1920412 (0.0010) [2023-12-27 18:57:47,363][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 983785472. Throughput: 0: 10387.6, 1: 10248.8. Samples: 983800228. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '921.290')] [2023-12-27 18:57:47,623][20347] Updated weights for policy 0, policy_version 1921949 (0.0007) [2023-12-27 18:57:47,681][20347] Updated weights for policy 0, policy_version 1921959 (0.0008) [2023-12-27 18:57:47,739][20347] Updated weights for policy 0, policy_version 1921969 (0.0007) [2023-12-27 18:57:48,038][20381] Updated weights for policy 1, policy_version 1920422 (0.0011) [2023-12-27 18:57:48,097][20381] Updated weights for policy 1, policy_version 1920432 (0.0011) [2023-12-27 18:57:48,146][20381] Updated weights for policy 1, policy_version 1920442 (0.0010) [2023-12-27 18:57:48,359][20347] Updated weights for policy 0, policy_version 1921979 (0.0007) [2023-12-27 18:57:48,418][20347] Updated weights for policy 0, policy_version 1921989 (0.0007) [2023-12-27 18:57:48,477][20347] Updated weights for policy 0, policy_version 1921999 (0.0008) [2023-12-27 18:57:48,860][20381] Updated weights for policy 1, policy_version 1920452 (0.0010) [2023-12-27 18:57:48,911][20381] Updated weights for policy 1, policy_version 1920462 (0.0008) [2023-12-27 18:57:48,965][20381] Updated weights for policy 1, policy_version 1920472 (0.0007) [2023-12-27 18:57:49,211][20347] Updated weights for policy 0, policy_version 1922010 (0.0010) [2023-12-27 18:57:49,276][20347] Updated weights for policy 0, policy_version 1922020 (0.0009) [2023-12-27 18:57:49,335][20347] Updated weights for policy 0, policy_version 1922030 (0.0007) [2023-12-27 18:57:49,401][20347] Updated weights for policy 0, policy_version 1922040 (0.0008) [2023-12-27 18:57:49,673][20381] Updated weights for policy 1, policy_version 1920482 (0.0007) [2023-12-27 18:57:49,734][20381] Updated weights for policy 1, policy_version 1920492 (0.0006) [2023-12-27 18:57:49,797][20381] Updated weights for policy 1, policy_version 1920502 (0.0005) [2023-12-27 18:57:49,870][20381] Updated weights for policy 1, policy_version 1920512 (0.0007) [2023-12-27 18:57:50,136][20347] Updated weights for policy 0, policy_version 1922050 (0.0008) [2023-12-27 18:57:50,205][20347] Updated weights for policy 0, policy_version 1922060 (0.0006) [2023-12-27 18:57:50,271][20347] Updated weights for policy 0, policy_version 1922070 (0.0007) [2023-12-27 18:57:50,518][20381] Updated weights for policy 1, policy_version 1920522 (0.0010) [2023-12-27 18:57:50,567][20381] Updated weights for policy 1, policy_version 1920532 (0.0010) [2023-12-27 18:57:50,626][20381] Updated weights for policy 1, policy_version 1920542 (0.0008) [2023-12-27 18:57:50,926][20347] Updated weights for policy 0, policy_version 1922080 (0.0008) [2023-12-27 18:57:50,984][20347] Updated weights for policy 0, policy_version 1922090 (0.0008) [2023-12-27 18:57:51,048][20347] Updated weights for policy 0, policy_version 1922100 (0.0007) [2023-12-27 18:57:51,352][20381] Updated weights for policy 1, policy_version 1920552 (0.0007) [2023-12-27 18:57:51,416][20381] Updated weights for policy 1, policy_version 1920562 (0.0008) [2023-12-27 18:57:51,467][20381] Updated weights for policy 1, policy_version 1920572 (0.0008) [2023-12-27 18:57:51,826][20347] Updated weights for policy 0, policy_version 1922110 (0.0006) [2023-12-27 18:57:51,884][20347] Updated weights for policy 0, policy_version 1922120 (0.0007) [2023-12-27 18:57:51,947][20347] Updated weights for policy 0, policy_version 1922130 (0.0007) [2023-12-27 18:57:52,161][20381] Updated weights for policy 1, policy_version 1920582 (0.0010) [2023-12-27 18:57:52,205][20381] Updated weights for policy 1, policy_version 1920592 (0.0010) [2023-12-27 18:57:52,264][20381] Updated weights for policy 1, policy_version 1920602 (0.0009) [2023-12-27 18:57:52,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 983891968. Throughput: 0: 10382.3, 1: 10171.4. Samples: 983860180. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.780')] [2023-12-27 18:57:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001920608_491732992.pth... [2023-12-27 18:57:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001922136_492158976.pth... [2023-12-27 18:57:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001920920_491847680.pth [2023-12-27 18:57:52,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001919424_491429888.pth [2023-12-27 18:57:52,627][20347] Updated weights for policy 0, policy_version 1922140 (0.0007) [2023-12-27 18:57:52,687][20347] Updated weights for policy 0, policy_version 1922150 (0.0008) [2023-12-27 18:57:52,750][20347] Updated weights for policy 0, policy_version 1922160 (0.0007) [2023-12-27 18:57:52,940][20381] Updated weights for policy 1, policy_version 1920612 (0.0008) [2023-12-27 18:57:53,001][20381] Updated weights for policy 1, policy_version 1920622 (0.0008) [2023-12-27 18:57:53,061][20381] Updated weights for policy 1, policy_version 1920632 (0.0007) [2023-12-27 18:57:53,425][20347] Updated weights for policy 0, policy_version 1922170 (0.0008) [2023-12-27 18:57:53,484][20347] Updated weights for policy 0, policy_version 1922180 (0.0007) [2023-12-27 18:57:53,542][20347] Updated weights for policy 0, policy_version 1922190 (0.0007) [2023-12-27 18:57:53,594][20347] Updated weights for policy 0, policy_version 1922200 (0.0007) [2023-12-27 18:57:53,715][20381] Updated weights for policy 1, policy_version 1920642 (0.0007) [2023-12-27 18:57:53,768][20381] Updated weights for policy 1, policy_version 1920652 (0.0005) [2023-12-27 18:57:53,827][20381] Updated weights for policy 1, policy_version 1920662 (0.0005) [2023-12-27 18:57:53,887][20381] Updated weights for policy 1, policy_version 1920672 (0.0007) [2023-12-27 18:57:54,211][20347] Updated weights for policy 0, policy_version 1922210 (0.0009) [2023-12-27 18:57:54,262][20347] Updated weights for policy 0, policy_version 1922220 (0.0008) [2023-12-27 18:57:54,308][20347] Updated weights for policy 0, policy_version 1922230 (0.0008) [2023-12-27 18:57:54,482][20381] Updated weights for policy 1, policy_version 1920682 (0.0005) [2023-12-27 18:57:54,546][20381] Updated weights for policy 1, policy_version 1920692 (0.0008) [2023-12-27 18:57:54,598][20381] Updated weights for policy 1, policy_version 1920702 (0.0008) [2023-12-27 18:57:55,048][20347] Updated weights for policy 0, policy_version 1922240 (0.0008) [2023-12-27 18:57:55,091][20347] Updated weights for policy 0, policy_version 1922250 (0.0008) [2023-12-27 18:57:55,140][20347] Updated weights for policy 0, policy_version 1922260 (0.0008) [2023-12-27 18:57:55,221][20381] Updated weights for policy 1, policy_version 1920712 (0.0008) [2023-12-27 18:57:55,266][20381] Updated weights for policy 1, policy_version 1920722 (0.0007) [2023-12-27 18:57:55,310][20381] Updated weights for policy 1, policy_version 1920732 (0.0007) [2023-12-27 18:57:55,875][20347] Updated weights for policy 0, policy_version 1922270 (0.0007) [2023-12-27 18:57:55,928][20347] Updated weights for policy 0, policy_version 1922280 (0.0008) [2023-12-27 18:57:55,990][20347] Updated weights for policy 0, policy_version 1922290 (0.0007) [2023-12-27 18:57:56,001][20381] Updated weights for policy 1, policy_version 1920742 (0.0006) [2023-12-27 18:57:56,062][20381] Updated weights for policy 1, policy_version 1920752 (0.0007) [2023-12-27 18:57:56,124][20381] Updated weights for policy 1, policy_version 1920762 (0.0008) [2023-12-27 18:57:56,660][20347] Updated weights for policy 0, policy_version 1922300 (0.0007) [2023-12-27 18:57:56,707][20347] Updated weights for policy 0, policy_version 1922310 (0.0006) [2023-12-27 18:57:56,760][20347] Updated weights for policy 0, policy_version 1922320 (0.0005) [2023-12-27 18:57:56,765][20381] Updated weights for policy 1, policy_version 1920772 (0.0008) [2023-12-27 18:57:56,814][20381] Updated weights for policy 1, policy_version 1920782 (0.0006) [2023-12-27 18:57:56,863][20381] Updated weights for policy 1, policy_version 1920792 (0.0006) [2023-12-27 18:57:57,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 983990272. Throughput: 0: 10338.6, 1: 10179.6. Samples: 983982228. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:57:57,364][19170] Avg episode reward: [(0, '1000.000'), (1, '892.670')] [2023-12-27 18:57:57,460][20347] Updated weights for policy 0, policy_version 1922330 (0.0005) [2023-12-27 18:57:57,506][20381] Updated weights for policy 1, policy_version 1920802 (0.0006) [2023-12-27 18:57:57,512][20347] Updated weights for policy 0, policy_version 1922340 (0.0005) [2023-12-27 18:57:57,554][20381] Updated weights for policy 1, policy_version 1920812 (0.0006) [2023-12-27 18:57:57,563][20347] Updated weights for policy 0, policy_version 1922350 (0.0006) [2023-12-27 18:57:57,617][20381] Updated weights for policy 1, policy_version 1920822 (0.0008) [2023-12-27 18:57:57,623][20347] Updated weights for policy 0, policy_version 1922360 (0.0006) [2023-12-27 18:57:57,677][20381] Updated weights for policy 1, policy_version 1920832 (0.0008) [2023-12-27 18:57:58,330][20347] Updated weights for policy 0, policy_version 1922370 (0.0008) [2023-12-27 18:57:58,384][20381] Updated weights for policy 1, policy_version 1920842 (0.0009) [2023-12-27 18:57:58,394][20347] Updated weights for policy 0, policy_version 1922380 (0.0008) [2023-12-27 18:57:58,442][20381] Updated weights for policy 1, policy_version 1920852 (0.0007) [2023-12-27 18:57:58,454][20347] Updated weights for policy 0, policy_version 1922390 (0.0011) [2023-12-27 18:57:58,503][20381] Updated weights for policy 1, policy_version 1920862 (0.0007) [2023-12-27 18:57:59,235][20347] Updated weights for policy 0, policy_version 1922400 (0.0009) [2023-12-27 18:57:59,293][20347] Updated weights for policy 0, policy_version 1922410 (0.0008) [2023-12-27 18:57:59,310][20381] Updated weights for policy 1, policy_version 1920872 (0.0010) [2023-12-27 18:57:59,354][20347] Updated weights for policy 0, policy_version 1922420 (0.0010) [2023-12-27 18:57:59,375][20381] Updated weights for policy 1, policy_version 1920882 (0.0011) [2023-12-27 18:57:59,433][20381] Updated weights for policy 1, policy_version 1920892 (0.0011) [2023-12-27 18:58:00,062][20347] Updated weights for policy 0, policy_version 1922430 (0.0009) [2023-12-27 18:58:00,093][20381] Updated weights for policy 1, policy_version 1920902 (0.0009) [2023-12-27 18:58:00,115][20347] Updated weights for policy 0, policy_version 1922440 (0.0010) [2023-12-27 18:58:00,148][20381] Updated weights for policy 1, policy_version 1920912 (0.0007) [2023-12-27 18:58:00,170][20347] Updated weights for policy 0, policy_version 1922450 (0.0010) [2023-12-27 18:58:00,203][20381] Updated weights for policy 1, policy_version 1920922 (0.0006) [2023-12-27 18:58:00,830][20381] Updated weights for policy 1, policy_version 1920932 (0.0009) [2023-12-27 18:58:00,834][20347] Updated weights for policy 0, policy_version 1922460 (0.0010) [2023-12-27 18:58:00,888][20381] Updated weights for policy 1, policy_version 1920942 (0.0005) [2023-12-27 18:58:00,892][20347] Updated weights for policy 0, policy_version 1922470 (0.0010) [2023-12-27 18:58:00,944][20381] Updated weights for policy 1, policy_version 1920952 (0.0008) [2023-12-27 18:58:00,947][20347] Updated weights for policy 0, policy_version 1922480 (0.0009) [2023-12-27 18:58:01,660][20381] Updated weights for policy 1, policy_version 1920962 (0.0010) [2023-12-27 18:58:01,662][20347] Updated weights for policy 0, policy_version 1922490 (0.0007) [2023-12-27 18:58:01,722][20381] Updated weights for policy 1, policy_version 1920972 (0.0008) [2023-12-27 18:58:01,724][20347] Updated weights for policy 0, policy_version 1922500 (0.0006) [2023-12-27 18:58:01,776][20347] Updated weights for policy 0, policy_version 1922510 (0.0007) [2023-12-27 18:58:01,782][20381] Updated weights for policy 1, policy_version 1920982 (0.0007) [2023-12-27 18:58:01,822][20347] Updated weights for policy 0, policy_version 1922520 (0.0005) [2023-12-27 18:58:01,839][20381] Updated weights for policy 1, policy_version 1920992 (0.0007) [2023-12-27 18:58:02,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 984088576. Throughput: 0: 10277.5, 1: 10147.9. Samples: 984102448. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.110')] [2023-12-27 18:58:02,469][20381] Updated weights for policy 1, policy_version 1921002 (0.0006) [2023-12-27 18:58:02,500][20347] Updated weights for policy 0, policy_version 1922530 (0.0011) [2023-12-27 18:58:02,521][20381] Updated weights for policy 1, policy_version 1921012 (0.0005) [2023-12-27 18:58:02,555][20347] Updated weights for policy 0, policy_version 1922540 (0.0010) [2023-12-27 18:58:02,572][20381] Updated weights for policy 1, policy_version 1921022 (0.0005) [2023-12-27 18:58:02,616][20347] Updated weights for policy 0, policy_version 1922550 (0.0010) [2023-12-27 18:58:03,247][20381] Updated weights for policy 1, policy_version 1921032 (0.0007) [2023-12-27 18:58:03,289][20347] Updated weights for policy 0, policy_version 1922560 (0.0010) [2023-12-27 18:58:03,301][20381] Updated weights for policy 1, policy_version 1921042 (0.0005) [2023-12-27 18:58:03,333][20347] Updated weights for policy 0, policy_version 1922570 (0.0010) [2023-12-27 18:58:03,348][20381] Updated weights for policy 1, policy_version 1921052 (0.0007) [2023-12-27 18:58:03,378][20347] Updated weights for policy 0, policy_version 1922580 (0.0010) [2023-12-27 18:58:03,995][20381] Updated weights for policy 1, policy_version 1921062 (0.0007) [2023-12-27 18:58:04,056][20381] Updated weights for policy 1, policy_version 1921072 (0.0007) [2023-12-27 18:58:04,096][20347] Updated weights for policy 0, policy_version 1922590 (0.0008) [2023-12-27 18:58:04,118][20381] Updated weights for policy 1, policy_version 1921082 (0.0008) [2023-12-27 18:58:04,160][20347] Updated weights for policy 0, policy_version 1922600 (0.0007) [2023-12-27 18:58:04,218][20347] Updated weights for policy 0, policy_version 1922610 (0.0010) [2023-12-27 18:58:04,777][20381] Updated weights for policy 1, policy_version 1921092 (0.0006) [2023-12-27 18:58:04,831][20381] Updated weights for policy 1, policy_version 1921102 (0.0007) [2023-12-27 18:58:04,886][20381] Updated weights for policy 1, policy_version 1921112 (0.0008) [2023-12-27 18:58:04,925][20347] Updated weights for policy 0, policy_version 1922620 (0.0011) [2023-12-27 18:58:04,987][20347] Updated weights for policy 0, policy_version 1922630 (0.0010) [2023-12-27 18:58:05,043][20347] Updated weights for policy 0, policy_version 1922640 (0.0010) [2023-12-27 18:58:05,578][20381] Updated weights for policy 1, policy_version 1921122 (0.0006) [2023-12-27 18:58:05,635][20381] Updated weights for policy 1, policy_version 1921132 (0.0006) [2023-12-27 18:58:05,665][20347] Updated weights for policy 0, policy_version 1922650 (0.0011) [2023-12-27 18:58:05,689][20381] Updated weights for policy 1, policy_version 1921142 (0.0007) [2023-12-27 18:58:05,726][20347] Updated weights for policy 0, policy_version 1922660 (0.0010) [2023-12-27 18:58:05,741][20381] Updated weights for policy 1, policy_version 1921152 (0.0007) [2023-12-27 18:58:05,773][20347] Updated weights for policy 0, policy_version 1922670 (0.0010) [2023-12-27 18:58:05,819][20347] Updated weights for policy 0, policy_version 1922680 (0.0010) [2023-12-27 18:58:06,407][20381] Updated weights for policy 1, policy_version 1921162 (0.0008) [2023-12-27 18:58:06,465][20381] Updated weights for policy 1, policy_version 1921172 (0.0008) [2023-12-27 18:58:06,509][20347] Updated weights for policy 0, policy_version 1922690 (0.0011) [2023-12-27 18:58:06,522][20381] Updated weights for policy 1, policy_version 1921182 (0.0006) [2023-12-27 18:58:06,566][20347] Updated weights for policy 0, policy_version 1922700 (0.0010) [2023-12-27 18:58:06,634][20347] Updated weights for policy 0, policy_version 1922710 (0.0006) [2023-12-27 18:58:07,190][20347] Updated weights for policy 0, policy_version 1922720 (0.0005) [2023-12-27 18:58:07,248][20347] Updated weights for policy 0, policy_version 1922730 (0.0006) [2023-12-27 18:58:07,259][20381] Updated weights for policy 1, policy_version 1921192 (0.0007) [2023-12-27 18:58:07,303][20347] Updated weights for policy 0, policy_version 1922740 (0.0008) [2023-12-27 18:58:07,311][20381] Updated weights for policy 1, policy_version 1921202 (0.0007) [2023-12-27 18:58:07,363][20381] Updated weights for policy 1, policy_version 1921212 (0.0006) [2023-12-27 18:58:07,363][19170] Fps is (10 sec: 20480.0, 60 sec: 20479.9, 300 sec: 20549.4). Total num frames: 984195072. Throughput: 0: 10274.3, 1: 10149.3. Samples: 984163884. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.340')] [2023-12-27 18:58:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001922744_492314624.pth... [2023-12-27 18:58:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001921560_492011520.pth [2023-12-27 18:58:07,384][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001921216_491888640.pth... [2023-12-27 18:58:07,387][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001920032_491585536.pth [2023-12-27 18:58:07,927][20381] Updated weights for policy 1, policy_version 1921222 (0.0006) [2023-12-27 18:58:07,932][20347] Updated weights for policy 0, policy_version 1922750 (0.0010) [2023-12-27 18:58:07,981][20347] Updated weights for policy 0, policy_version 1922760 (0.0010) [2023-12-27 18:58:07,984][20381] Updated weights for policy 1, policy_version 1921232 (0.0005) [2023-12-27 18:58:08,030][20347] Updated weights for policy 0, policy_version 1922770 (0.0009) [2023-12-27 18:58:08,048][20381] Updated weights for policy 1, policy_version 1921242 (0.0006) [2023-12-27 18:58:08,640][20381] Updated weights for policy 1, policy_version 1921252 (0.0008) [2023-12-27 18:58:08,692][20347] Updated weights for policy 0, policy_version 1922780 (0.0008) [2023-12-27 18:58:08,696][20381] Updated weights for policy 1, policy_version 1921262 (0.0006) [2023-12-27 18:58:08,757][20347] Updated weights for policy 0, policy_version 1922790 (0.0010) [2023-12-27 18:58:08,765][20381] Updated weights for policy 1, policy_version 1921272 (0.0006) [2023-12-27 18:58:08,809][20347] Updated weights for policy 0, policy_version 1922800 (0.0010) [2023-12-27 18:58:09,430][20381] Updated weights for policy 1, policy_version 1921282 (0.0007) [2023-12-27 18:58:09,470][20347] Updated weights for policy 0, policy_version 1922810 (0.0010) [2023-12-27 18:58:09,489][20381] Updated weights for policy 1, policy_version 1921292 (0.0011) [2023-12-27 18:58:09,527][20347] Updated weights for policy 0, policy_version 1922820 (0.0008) [2023-12-27 18:58:09,550][20381] Updated weights for policy 1, policy_version 1921302 (0.0011) [2023-12-27 18:58:09,588][20347] Updated weights for policy 0, policy_version 1922830 (0.0008) [2023-12-27 18:58:09,611][20381] Updated weights for policy 1, policy_version 1921312 (0.0011) [2023-12-27 18:58:09,651][20347] Updated weights for policy 0, policy_version 1922840 (0.0008) [2023-12-27 18:58:10,336][20381] Updated weights for policy 1, policy_version 1921322 (0.0010) [2023-12-27 18:58:10,384][20347] Updated weights for policy 0, policy_version 1922850 (0.0007) [2023-12-27 18:58:10,400][20381] Updated weights for policy 1, policy_version 1921332 (0.0011) [2023-12-27 18:58:10,448][20347] Updated weights for policy 0, policy_version 1922860 (0.0008) [2023-12-27 18:58:10,462][20381] Updated weights for policy 1, policy_version 1921342 (0.0010) [2023-12-27 18:58:10,510][20347] Updated weights for policy 0, policy_version 1922870 (0.0008) [2023-12-27 18:58:11,094][20347] Updated weights for policy 0, policy_version 1922880 (0.0008) [2023-12-27 18:58:11,125][20381] Updated weights for policy 1, policy_version 1921352 (0.0008) [2023-12-27 18:58:11,153][20347] Updated weights for policy 0, policy_version 1922890 (0.0008) [2023-12-27 18:58:11,192][20381] Updated weights for policy 1, policy_version 1921362 (0.0008) [2023-12-27 18:58:11,220][20347] Updated weights for policy 0, policy_version 1922900 (0.0011) [2023-12-27 18:58:11,257][20381] Updated weights for policy 1, policy_version 1921372 (0.0007) [2023-12-27 18:58:11,974][20381] Updated weights for policy 1, policy_version 1921382 (0.0006) [2023-12-27 18:58:11,994][20347] Updated weights for policy 0, policy_version 1922910 (0.0009) [2023-12-27 18:58:12,040][20381] Updated weights for policy 1, policy_version 1921392 (0.0008) [2023-12-27 18:58:12,045][20347] Updated weights for policy 0, policy_version 1922920 (0.0007) [2023-12-27 18:58:12,100][20381] Updated weights for policy 1, policy_version 1921402 (0.0011) [2023-12-27 18:58:12,102][20347] Updated weights for policy 0, policy_version 1922930 (0.0011) [2023-12-27 18:58:12,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 984301568. Throughput: 0: 10274.4, 1: 10148.6. Samples: 984290220. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.470')] [2023-12-27 18:58:12,773][20381] Updated weights for policy 1, policy_version 1921412 (0.0011) [2023-12-27 18:58:12,819][20347] Updated weights for policy 0, policy_version 1922940 (0.0008) [2023-12-27 18:58:12,836][20381] Updated weights for policy 1, policy_version 1921422 (0.0010) [2023-12-27 18:58:12,880][20347] Updated weights for policy 0, policy_version 1922950 (0.0007) [2023-12-27 18:58:12,899][20381] Updated weights for policy 1, policy_version 1921432 (0.0011) [2023-12-27 18:58:12,929][20347] Updated weights for policy 0, policy_version 1922960 (0.0009) [2023-12-27 18:58:13,520][20381] Updated weights for policy 1, policy_version 1921442 (0.0011) [2023-12-27 18:58:13,574][20381] Updated weights for policy 1, policy_version 1921452 (0.0011) [2023-12-27 18:58:13,631][20381] Updated weights for policy 1, policy_version 1921462 (0.0010) [2023-12-27 18:58:13,651][20347] Updated weights for policy 0, policy_version 1922970 (0.0006) [2023-12-27 18:58:13,687][20381] Updated weights for policy 1, policy_version 1921472 (0.0011) [2023-12-27 18:58:13,704][20347] Updated weights for policy 0, policy_version 1922980 (0.0005) [2023-12-27 18:58:13,761][20347] Updated weights for policy 0, policy_version 1922990 (0.0005) [2023-12-27 18:58:13,818][20347] Updated weights for policy 0, policy_version 1923000 (0.0005) [2023-12-27 18:58:14,342][20381] Updated weights for policy 1, policy_version 1921482 (0.0011) [2023-12-27 18:58:14,391][20381] Updated weights for policy 1, policy_version 1921492 (0.0010) [2023-12-27 18:58:14,446][20381] Updated weights for policy 1, policy_version 1921502 (0.0011) [2023-12-27 18:58:14,451][20347] Updated weights for policy 0, policy_version 1923010 (0.0006) [2023-12-27 18:58:14,499][20347] Updated weights for policy 0, policy_version 1923020 (0.0007) [2023-12-27 18:58:14,549][20347] Updated weights for policy 0, policy_version 1923030 (0.0007) [2023-12-27 18:58:15,123][20381] Updated weights for policy 1, policy_version 1921512 (0.0010) [2023-12-27 18:58:15,186][20381] Updated weights for policy 1, policy_version 1921522 (0.0011) [2023-12-27 18:58:15,223][20347] Updated weights for policy 0, policy_version 1923040 (0.0008) [2023-12-27 18:58:15,244][20381] Updated weights for policy 1, policy_version 1921532 (0.0011) [2023-12-27 18:58:15,272][20347] Updated weights for policy 0, policy_version 1923050 (0.0005) [2023-12-27 18:58:15,323][20347] Updated weights for policy 0, policy_version 1923060 (0.0007) [2023-12-27 18:58:15,927][20381] Updated weights for policy 1, policy_version 1921542 (0.0010) [2023-12-27 18:58:15,988][20381] Updated weights for policy 1, policy_version 1921552 (0.0011) [2023-12-27 18:58:16,022][20347] Updated weights for policy 0, policy_version 1923070 (0.0006) [2023-12-27 18:58:16,043][20381] Updated weights for policy 1, policy_version 1921562 (0.0011) [2023-12-27 18:58:16,078][20347] Updated weights for policy 0, policy_version 1923080 (0.0006) [2023-12-27 18:58:16,137][20347] Updated weights for policy 0, policy_version 1923090 (0.0005) [2023-12-27 18:58:16,702][20381] Updated weights for policy 1, policy_version 1921572 (0.0009) [2023-12-27 18:58:16,740][20347] Updated weights for policy 0, policy_version 1923100 (0.0006) [2023-12-27 18:58:16,760][20381] Updated weights for policy 1, policy_version 1921582 (0.0008) [2023-12-27 18:58:16,792][20347] Updated weights for policy 0, policy_version 1923110 (0.0005) [2023-12-27 18:58:16,816][20381] Updated weights for policy 1, policy_version 1921592 (0.0007) [2023-12-27 18:58:16,839][20347] Updated weights for policy 0, policy_version 1923120 (0.0005) [2023-12-27 18:58:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 984399872. Throughput: 0: 10242.0, 1: 10190.2. Samples: 984412664. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.620')] [2023-12-27 18:58:17,470][20347] Updated weights for policy 0, policy_version 1923130 (0.0006) [2023-12-27 18:58:17,487][20381] Updated weights for policy 1, policy_version 1921602 (0.0007) [2023-12-27 18:58:17,525][20347] Updated weights for policy 0, policy_version 1923140 (0.0007) [2023-12-27 18:58:17,537][20381] Updated weights for policy 1, policy_version 1921612 (0.0005) [2023-12-27 18:58:17,574][20347] Updated weights for policy 0, policy_version 1923150 (0.0008) [2023-12-27 18:58:17,601][20381] Updated weights for policy 1, policy_version 1921622 (0.0006) [2023-12-27 18:58:17,634][20347] Updated weights for policy 0, policy_version 1923160 (0.0007) [2023-12-27 18:58:17,661][20381] Updated weights for policy 1, policy_version 1921632 (0.0007) [2023-12-27 18:58:18,221][20381] Updated weights for policy 1, policy_version 1921642 (0.0006) [2023-12-27 18:58:18,265][20381] Updated weights for policy 1, policy_version 1921652 (0.0006) [2023-12-27 18:58:18,289][20347] Updated weights for policy 0, policy_version 1923170 (0.0006) [2023-12-27 18:58:18,310][20381] Updated weights for policy 1, policy_version 1921662 (0.0005) [2023-12-27 18:58:18,352][20347] Updated weights for policy 0, policy_version 1923180 (0.0007) [2023-12-27 18:58:18,406][20347] Updated weights for policy 0, policy_version 1923190 (0.0007) [2023-12-27 18:58:19,036][20381] Updated weights for policy 1, policy_version 1921672 (0.0008) [2023-12-27 18:58:19,087][20381] Updated weights for policy 1, policy_version 1921682 (0.0009) [2023-12-27 18:58:19,106][20347] Updated weights for policy 0, policy_version 1923200 (0.0007) [2023-12-27 18:58:19,138][20381] Updated weights for policy 1, policy_version 1921692 (0.0008) [2023-12-27 18:58:19,161][20347] Updated weights for policy 0, policy_version 1923210 (0.0007) [2023-12-27 18:58:19,223][20347] Updated weights for policy 0, policy_version 1923220 (0.0008) [2023-12-27 18:58:19,832][20381] Updated weights for policy 1, policy_version 1921702 (0.0007) [2023-12-27 18:58:19,904][20381] Updated weights for policy 1, policy_version 1921712 (0.0008) [2023-12-27 18:58:19,965][20347] Updated weights for policy 0, policy_version 1923230 (0.0007) [2023-12-27 18:58:19,970][20381] Updated weights for policy 1, policy_version 1921722 (0.0008) [2023-12-27 18:58:20,026][20347] Updated weights for policy 0, policy_version 1923240 (0.0007) [2023-12-27 18:58:20,089][20347] Updated weights for policy 0, policy_version 1923250 (0.0008) [2023-12-27 18:58:20,704][20381] Updated weights for policy 1, policy_version 1921732 (0.0006) [2023-12-27 18:58:20,757][20347] Updated weights for policy 0, policy_version 1923260 (0.0007) [2023-12-27 18:58:20,766][20381] Updated weights for policy 1, policy_version 1921742 (0.0008) [2023-12-27 18:58:20,819][20347] Updated weights for policy 0, policy_version 1923270 (0.0006) [2023-12-27 18:58:20,821][20381] Updated weights for policy 1, policy_version 1921752 (0.0006) [2023-12-27 18:58:20,874][20347] Updated weights for policy 0, policy_version 1923280 (0.0006) [2023-12-27 18:58:21,464][20381] Updated weights for policy 1, policy_version 1921762 (0.0006) [2023-12-27 18:58:21,523][20381] Updated weights for policy 1, policy_version 1921772 (0.0006) [2023-12-27 18:58:21,583][20381] Updated weights for policy 1, policy_version 1921782 (0.0006) [2023-12-27 18:58:21,650][20347] Updated weights for policy 0, policy_version 1923290 (0.0008) [2023-12-27 18:58:21,651][20381] Updated weights for policy 1, policy_version 1921792 (0.0012) [2023-12-27 18:58:21,711][20347] Updated weights for policy 0, policy_version 1923300 (0.0007) [2023-12-27 18:58:21,780][20347] Updated weights for policy 0, policy_version 1923310 (0.0008) [2023-12-27 18:58:21,847][20347] Updated weights for policy 0, policy_version 1923320 (0.0008) [2023-12-27 18:58:22,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 984498176. Throughput: 0: 10192.7, 1: 10246.9. Samples: 984474452. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.740')] [2023-12-27 18:58:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001923320_492462080.pth... [2023-12-27 18:58:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001922136_492158976.pth [2023-12-27 18:58:22,393][20381] Updated weights for policy 1, policy_version 1921802 (0.0007) [2023-12-27 18:58:22,454][20381] Updated weights for policy 1, policy_version 1921812 (0.0007) [2023-12-27 18:58:22,465][20347] Updated weights for policy 0, policy_version 1923330 (0.0008) [2023-12-27 18:58:22,502][20381] Updated weights for policy 1, policy_version 1921822 (0.0007) [2023-12-27 18:58:22,513][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001921824_492044288.pth... [2023-12-27 18:58:22,516][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001920608_491732992.pth [2023-12-27 18:58:22,528][20347] Updated weights for policy 0, policy_version 1923340 (0.0008) [2023-12-27 18:58:22,588][20347] Updated weights for policy 0, policy_version 1923350 (0.0008) [2023-12-27 18:58:23,188][20381] Updated weights for policy 1, policy_version 1921832 (0.0005) [2023-12-27 18:58:23,223][20347] Updated weights for policy 0, policy_version 1923360 (0.0008) [2023-12-27 18:58:23,240][20381] Updated weights for policy 1, policy_version 1921842 (0.0005) [2023-12-27 18:58:23,271][20347] Updated weights for policy 0, policy_version 1923370 (0.0006) [2023-12-27 18:58:23,290][20381] Updated weights for policy 1, policy_version 1921852 (0.0005) [2023-12-27 18:58:23,324][20347] Updated weights for policy 0, policy_version 1923380 (0.0007) [2023-12-27 18:58:23,835][20381] Updated weights for policy 1, policy_version 1921862 (0.0005) [2023-12-27 18:58:23,890][20381] Updated weights for policy 1, policy_version 1921872 (0.0005) [2023-12-27 18:58:23,950][20381] Updated weights for policy 1, policy_version 1921882 (0.0005) [2023-12-27 18:58:24,013][20347] Updated weights for policy 0, policy_version 1923390 (0.0008) [2023-12-27 18:58:24,070][20347] Updated weights for policy 0, policy_version 1923400 (0.0008) [2023-12-27 18:58:24,128][20347] Updated weights for policy 0, policy_version 1923410 (0.0008) [2023-12-27 18:58:24,542][20381] Updated weights for policy 1, policy_version 1921892 (0.0005) [2023-12-27 18:58:24,597][20381] Updated weights for policy 1, policy_version 1921902 (0.0006) [2023-12-27 18:58:24,649][20381] Updated weights for policy 1, policy_version 1921912 (0.0008) [2023-12-27 18:58:24,845][20347] Updated weights for policy 0, policy_version 1923420 (0.0008) [2023-12-27 18:58:24,898][20347] Updated weights for policy 0, policy_version 1923430 (0.0006) [2023-12-27 18:58:24,961][20347] Updated weights for policy 0, policy_version 1923440 (0.0005) [2023-12-27 18:58:25,244][20381] Updated weights for policy 1, policy_version 1921922 (0.0008) [2023-12-27 18:58:25,298][20381] Updated weights for policy 1, policy_version 1921932 (0.0008) [2023-12-27 18:58:25,357][20381] Updated weights for policy 1, policy_version 1921942 (0.0008) [2023-12-27 18:58:25,414][20381] Updated weights for policy 1, policy_version 1921952 (0.0008) [2023-12-27 18:58:25,616][20347] Updated weights for policy 0, policy_version 1923450 (0.0006) [2023-12-27 18:58:25,682][20347] Updated weights for policy 0, policy_version 1923460 (0.0007) [2023-12-27 18:58:25,744][20347] Updated weights for policy 0, policy_version 1923470 (0.0007) [2023-12-27 18:58:25,806][20347] Updated weights for policy 0, policy_version 1923480 (0.0008) [2023-12-27 18:58:26,060][20381] Updated weights for policy 1, policy_version 1921962 (0.0008) [2023-12-27 18:58:26,116][20381] Updated weights for policy 1, policy_version 1921972 (0.0008) [2023-12-27 18:58:26,174][20381] Updated weights for policy 1, policy_version 1921982 (0.0008) [2023-12-27 18:58:26,484][20347] Updated weights for policy 0, policy_version 1923490 (0.0007) [2023-12-27 18:58:26,534][20347] Updated weights for policy 0, policy_version 1923500 (0.0008) [2023-12-27 18:58:26,588][20347] Updated weights for policy 0, policy_version 1923510 (0.0007) [2023-12-27 18:58:26,860][20381] Updated weights for policy 1, policy_version 1921992 (0.0007) [2023-12-27 18:58:26,921][20381] Updated weights for policy 1, policy_version 1922002 (0.0008) [2023-12-27 18:58:26,979][20381] Updated weights for policy 1, policy_version 1922012 (0.0007) [2023-12-27 18:58:27,317][20347] Updated weights for policy 0, policy_version 1923520 (0.0007) [2023-12-27 18:58:27,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 984604672. Throughput: 0: 10196.6, 1: 10296.0. Samples: 984597936. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.990')] [2023-12-27 18:58:27,375][20347] Updated weights for policy 0, policy_version 1923530 (0.0007) [2023-12-27 18:58:27,429][20347] Updated weights for policy 0, policy_version 1923540 (0.0007) [2023-12-27 18:58:27,601][20381] Updated weights for policy 1, policy_version 1922022 (0.0006) [2023-12-27 18:58:27,655][20381] Updated weights for policy 1, policy_version 1922032 (0.0009) [2023-12-27 18:58:27,701][20381] Updated weights for policy 1, policy_version 1922042 (0.0009) [2023-12-27 18:58:28,099][20347] Updated weights for policy 0, policy_version 1923550 (0.0007) [2023-12-27 18:58:28,151][20347] Updated weights for policy 0, policy_version 1923560 (0.0007) [2023-12-27 18:58:28,207][20347] Updated weights for policy 0, policy_version 1923570 (0.0008) [2023-12-27 18:58:28,446][20381] Updated weights for policy 1, policy_version 1922052 (0.0008) [2023-12-27 18:58:28,504][20381] Updated weights for policy 1, policy_version 1922062 (0.0008) [2023-12-27 18:58:28,570][20381] Updated weights for policy 1, policy_version 1922072 (0.0007) [2023-12-27 18:58:28,905][20347] Updated weights for policy 0, policy_version 1923580 (0.0007) [2023-12-27 18:58:28,953][20347] Updated weights for policy 0, policy_version 1923590 (0.0008) [2023-12-27 18:58:28,996][20347] Updated weights for policy 0, policy_version 1923600 (0.0007) [2023-12-27 18:58:29,250][20381] Updated weights for policy 1, policy_version 1922082 (0.0008) [2023-12-27 18:58:29,306][20381] Updated weights for policy 1, policy_version 1922092 (0.0008) [2023-12-27 18:58:29,377][20381] Updated weights for policy 1, policy_version 1922102 (0.0008) [2023-12-27 18:58:29,435][20381] Updated weights for policy 1, policy_version 1922112 (0.0008) [2023-12-27 18:58:29,729][20347] Updated weights for policy 0, policy_version 1923610 (0.0007) [2023-12-27 18:58:29,790][20347] Updated weights for policy 0, policy_version 1923620 (0.0010) [2023-12-27 18:58:29,856][20347] Updated weights for policy 0, policy_version 1923630 (0.0008) [2023-12-27 18:58:29,914][20347] Updated weights for policy 0, policy_version 1923640 (0.0007) [2023-12-27 18:58:30,151][20381] Updated weights for policy 1, policy_version 1922122 (0.0009) [2023-12-27 18:58:30,200][20381] Updated weights for policy 1, policy_version 1922132 (0.0010) [2023-12-27 18:58:30,253][20381] Updated weights for policy 1, policy_version 1922142 (0.0007) [2023-12-27 18:58:30,489][20347] Updated weights for policy 0, policy_version 1923650 (0.0005) [2023-12-27 18:58:30,551][20347] Updated weights for policy 0, policy_version 1923660 (0.0005) [2023-12-27 18:58:30,608][20347] Updated weights for policy 0, policy_version 1923670 (0.0007) [2023-12-27 18:58:30,978][20381] Updated weights for policy 1, policy_version 1922152 (0.0006) [2023-12-27 18:58:31,032][20381] Updated weights for policy 1, policy_version 1922162 (0.0006) [2023-12-27 18:58:31,090][20381] Updated weights for policy 1, policy_version 1922172 (0.0008) [2023-12-27 18:58:31,144][20347] Updated weights for policy 0, policy_version 1923680 (0.0010) [2023-12-27 18:58:31,206][20347] Updated weights for policy 0, policy_version 1923690 (0.0008) [2023-12-27 18:58:31,268][20347] Updated weights for policy 0, policy_version 1923700 (0.0007) [2023-12-27 18:58:31,811][20381] Updated weights for policy 1, policy_version 1922182 (0.0007) [2023-12-27 18:58:31,874][20381] Updated weights for policy 1, policy_version 1922192 (0.0007) [2023-12-27 18:58:31,935][20381] Updated weights for policy 1, policy_version 1922202 (0.0007) [2023-12-27 18:58:31,967][20347] Updated weights for policy 0, policy_version 1923710 (0.0007) [2023-12-27 18:58:32,034][20347] Updated weights for policy 0, policy_version 1923720 (0.0007) [2023-12-27 18:58:32,096][20347] Updated weights for policy 0, policy_version 1923730 (0.0006) [2023-12-27 18:58:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 984711168. Throughput: 0: 10162.5, 1: 10301.2. Samples: 984721092. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.390')] [2023-12-27 18:58:32,565][20381] Updated weights for policy 1, policy_version 1922212 (0.0006) [2023-12-27 18:58:32,627][20381] Updated weights for policy 1, policy_version 1922222 (0.0006) [2023-12-27 18:58:32,688][20381] Updated weights for policy 1, policy_version 1922232 (0.0007) [2023-12-27 18:58:32,778][20347] Updated weights for policy 0, policy_version 1923740 (0.0007) [2023-12-27 18:58:32,838][20347] Updated weights for policy 0, policy_version 1923750 (0.0005) [2023-12-27 18:58:32,894][20347] Updated weights for policy 0, policy_version 1923760 (0.0005) [2023-12-27 18:58:33,330][20381] Updated weights for policy 1, policy_version 1922242 (0.0008) [2023-12-27 18:58:33,385][20381] Updated weights for policy 1, policy_version 1922252 (0.0009) [2023-12-27 18:58:33,438][20381] Updated weights for policy 1, policy_version 1922262 (0.0010) [2023-12-27 18:58:33,485][20347] Updated weights for policy 0, policy_version 1923770 (0.0006) [2023-12-27 18:58:33,489][20381] Updated weights for policy 1, policy_version 1922272 (0.0010) [2023-12-27 18:58:33,535][20347] Updated weights for policy 0, policy_version 1923780 (0.0008) [2023-12-27 18:58:33,592][20347] Updated weights for policy 0, policy_version 1923790 (0.0007) [2023-12-27 18:58:33,653][20347] Updated weights for policy 0, policy_version 1923800 (0.0008) [2023-12-27 18:58:34,178][20381] Updated weights for policy 1, policy_version 1922282 (0.0008) [2023-12-27 18:58:34,242][20381] Updated weights for policy 1, policy_version 1922292 (0.0008) [2023-12-27 18:58:34,296][20347] Updated weights for policy 0, policy_version 1923810 (0.0006) [2023-12-27 18:58:34,303][20381] Updated weights for policy 1, policy_version 1922302 (0.0008) [2023-12-27 18:58:34,357][20347] Updated weights for policy 0, policy_version 1923820 (0.0006) [2023-12-27 18:58:34,418][20347] Updated weights for policy 0, policy_version 1923830 (0.0006) [2023-12-27 18:58:35,024][20381] Updated weights for policy 1, policy_version 1922312 (0.0010) [2023-12-27 18:58:35,042][20347] Updated weights for policy 0, policy_version 1923840 (0.0009) [2023-12-27 18:58:35,082][20381] Updated weights for policy 1, policy_version 1922322 (0.0011) [2023-12-27 18:58:35,100][20347] Updated weights for policy 0, policy_version 1923850 (0.0010) [2023-12-27 18:58:35,134][20381] Updated weights for policy 1, policy_version 1922332 (0.0010) [2023-12-27 18:58:35,149][20347] Updated weights for policy 0, policy_version 1923860 (0.0009) [2023-12-27 18:58:35,775][20347] Updated weights for policy 0, policy_version 1923870 (0.0006) [2023-12-27 18:58:35,830][20381] Updated weights for policy 1, policy_version 1922342 (0.0011) [2023-12-27 18:58:35,835][20347] Updated weights for policy 0, policy_version 1923880 (0.0009) [2023-12-27 18:58:35,886][20381] Updated weights for policy 1, policy_version 1922352 (0.0011) [2023-12-27 18:58:35,898][20347] Updated weights for policy 0, policy_version 1923890 (0.0010) [2023-12-27 18:58:35,945][20381] Updated weights for policy 1, policy_version 1922362 (0.0010) [2023-12-27 18:58:36,528][20347] Updated weights for policy 0, policy_version 1923900 (0.0010) [2023-12-27 18:58:36,581][20347] Updated weights for policy 0, policy_version 1923910 (0.0010) [2023-12-27 18:58:36,643][20347] Updated weights for policy 0, policy_version 1923920 (0.0011) [2023-12-27 18:58:36,670][20381] Updated weights for policy 1, policy_version 1922372 (0.0009) [2023-12-27 18:58:36,722][20381] Updated weights for policy 1, policy_version 1922382 (0.0010) [2023-12-27 18:58:36,777][20381] Updated weights for policy 1, policy_version 1922392 (0.0010) [2023-12-27 18:58:37,308][20347] Updated weights for policy 0, policy_version 1923930 (0.0008) [2023-12-27 18:58:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 984809472. Throughput: 0: 10200.0, 1: 10311.4. Samples: 984783192. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.080')] [2023-12-27 18:58:37,364][20347] Updated weights for policy 0, policy_version 1923940 (0.0006) [2023-12-27 18:58:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001922400_492191744.pth... [2023-12-27 18:58:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001921216_491888640.pth [2023-12-27 18:58:37,413][20347] Updated weights for policy 0, policy_version 1923950 (0.0010) [2023-12-27 18:58:37,460][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001923960_492625920.pth... [2023-12-27 18:58:37,462][20347] Updated weights for policy 0, policy_version 1923960 (0.0010) [2023-12-27 18:58:37,464][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001922744_492314624.pth [2023-12-27 18:58:37,469][20381] Updated weights for policy 1, policy_version 1922402 (0.0010) [2023-12-27 18:58:37,520][20381] Updated weights for policy 1, policy_version 1922412 (0.0007) [2023-12-27 18:58:37,575][20381] Updated weights for policy 1, policy_version 1922422 (0.0007) [2023-12-27 18:58:37,627][20381] Updated weights for policy 1, policy_version 1922432 (0.0007) [2023-12-27 18:58:38,122][20347] Updated weights for policy 0, policy_version 1923970 (0.0007) [2023-12-27 18:58:38,174][20347] Updated weights for policy 0, policy_version 1923980 (0.0006) [2023-12-27 18:58:38,232][20347] Updated weights for policy 0, policy_version 1923990 (0.0006) [2023-12-27 18:58:38,302][20381] Updated weights for policy 1, policy_version 1922442 (0.0005) [2023-12-27 18:58:38,362][20381] Updated weights for policy 1, policy_version 1922452 (0.0007) [2023-12-27 18:58:38,416][20381] Updated weights for policy 1, policy_version 1922462 (0.0007) [2023-12-27 18:58:38,913][20347] Updated weights for policy 0, policy_version 1924000 (0.0007) [2023-12-27 18:58:38,970][20347] Updated weights for policy 0, policy_version 1924010 (0.0008) [2023-12-27 18:58:39,032][20347] Updated weights for policy 0, policy_version 1924020 (0.0008) [2023-12-27 18:58:39,066][20381] Updated weights for policy 1, policy_version 1922472 (0.0010) [2023-12-27 18:58:39,126][20381] Updated weights for policy 1, policy_version 1922482 (0.0010) [2023-12-27 18:58:39,182][20381] Updated weights for policy 1, policy_version 1922492 (0.0010) [2023-12-27 18:58:39,741][20347] Updated weights for policy 0, policy_version 1924030 (0.0007) [2023-12-27 18:58:39,794][20347] Updated weights for policy 0, policy_version 1924040 (0.0009) [2023-12-27 18:58:39,861][20347] Updated weights for policy 0, policy_version 1924050 (0.0009) [2023-12-27 18:58:39,870][20381] Updated weights for policy 1, policy_version 1922502 (0.0009) [2023-12-27 18:58:39,936][20381] Updated weights for policy 1, policy_version 1922512 (0.0007) [2023-12-27 18:58:39,995][20381] Updated weights for policy 1, policy_version 1922522 (0.0006) [2023-12-27 18:58:40,588][20347] Updated weights for policy 0, policy_version 1924060 (0.0007) [2023-12-27 18:58:40,647][20347] Updated weights for policy 0, policy_version 1924070 (0.0006) [2023-12-27 18:58:40,696][20381] Updated weights for policy 1, policy_version 1922532 (0.0007) [2023-12-27 18:58:40,707][20347] Updated weights for policy 0, policy_version 1924080 (0.0005) [2023-12-27 18:58:40,755][20381] Updated weights for policy 1, policy_version 1922542 (0.0007) [2023-12-27 18:58:40,811][20381] Updated weights for policy 1, policy_version 1922552 (0.0008) [2023-12-27 18:58:41,320][20347] Updated weights for policy 0, policy_version 1924090 (0.0006) [2023-12-27 18:58:41,387][20347] Updated weights for policy 0, policy_version 1924100 (0.0008) [2023-12-27 18:58:41,450][20347] Updated weights for policy 0, policy_version 1924110 (0.0006) [2023-12-27 18:58:41,508][20347] Updated weights for policy 0, policy_version 1924120 (0.0007) [2023-12-27 18:58:41,570][20381] Updated weights for policy 1, policy_version 1922562 (0.0008) [2023-12-27 18:58:41,625][20381] Updated weights for policy 1, policy_version 1922572 (0.0007) [2023-12-27 18:58:41,690][20381] Updated weights for policy 1, policy_version 1922582 (0.0007) [2023-12-27 18:58:41,758][20381] Updated weights for policy 1, policy_version 1922592 (0.0008) [2023-12-27 18:58:42,259][20347] Updated weights for policy 0, policy_version 1924130 (0.0008) [2023-12-27 18:58:42,320][20347] Updated weights for policy 0, policy_version 1924140 (0.0007) [2023-12-27 18:58:42,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 984907776. Throughput: 0: 10252.8, 1: 10268.2. Samples: 984905672. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.870')] [2023-12-27 18:58:42,388][20347] Updated weights for policy 0, policy_version 1924150 (0.0007) [2023-12-27 18:58:42,425][20381] Updated weights for policy 1, policy_version 1922602 (0.0007) [2023-12-27 18:58:42,481][20381] Updated weights for policy 1, policy_version 1922612 (0.0008) [2023-12-27 18:58:42,503][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000008 [2023-12-27 18:58:43,124][20347] Updated weights for policy 0, policy_version 1924160 (0.0006) [2023-12-27 18:58:43,187][20347] Updated weights for policy 0, policy_version 1924170 (0.0005) [2023-12-27 18:58:43,206][20381] Updated weights for policy 1, policy_version 1922622 (0.0010) [2023-12-27 18:58:43,246][20347] Updated weights for policy 0, policy_version 1924180 (0.0006) [2023-12-27 18:58:43,259][20381] Updated weights for policy 1, policy_version 1922632 (0.0010) [2023-12-27 18:58:43,312][20381] Updated weights for policy 1, policy_version 1922642 (0.0011) [2023-12-27 18:58:43,896][20347] Updated weights for policy 0, policy_version 1924190 (0.0007) [2023-12-27 18:58:43,953][20347] Updated weights for policy 0, policy_version 1924200 (0.0007) [2023-12-27 18:58:44,007][20347] Updated weights for policy 0, policy_version 1924210 (0.0008) [2023-12-27 18:58:44,017][20381] Updated weights for policy 1, policy_version 1922652 (0.0011) [2023-12-27 18:58:44,075][20381] Updated weights for policy 1, policy_version 1922662 (0.0010) [2023-12-27 18:58:44,129][20381] Updated weights for policy 1, policy_version 1922672 (0.0008) [2023-12-27 18:58:44,735][20347] Updated weights for policy 0, policy_version 1924220 (0.0005) [2023-12-27 18:58:44,801][20347] Updated weights for policy 0, policy_version 1924230 (0.0007) [2023-12-27 18:58:44,810][20381] Updated weights for policy 1, policy_version 1922682 (0.0008) [2023-12-27 18:58:44,864][20347] Updated weights for policy 0, policy_version 1924240 (0.0007) [2023-12-27 18:58:44,870][20381] Updated weights for policy 1, policy_version 1922692 (0.0008) [2023-12-27 18:58:44,927][20381] Updated weights for policy 1, policy_version 1922702 (0.0007) [2023-12-27 18:58:44,985][20381] Updated weights for policy 1, policy_version 1922712 (0.0008) [2023-12-27 18:58:45,555][20347] Updated weights for policy 0, policy_version 1924250 (0.0006) [2023-12-27 18:58:45,606][20347] Updated weights for policy 0, policy_version 1924260 (0.0007) [2023-12-27 18:58:45,663][20347] Updated weights for policy 0, policy_version 1924270 (0.0007) [2023-12-27 18:58:45,681][20381] Updated weights for policy 1, policy_version 1922722 (0.0006) [2023-12-27 18:58:45,720][20347] Updated weights for policy 0, policy_version 1924280 (0.0007) [2023-12-27 18:58:45,731][20381] Updated weights for policy 1, policy_version 1922732 (0.0007) [2023-12-27 18:58:45,777][20381] Updated weights for policy 1, policy_version 1922742 (0.0008) [2023-12-27 18:58:46,370][20381] Updated weights for policy 1, policy_version 1922752 (0.0006) [2023-12-27 18:58:46,422][20381] Updated weights for policy 1, policy_version 1922762 (0.0006) [2023-12-27 18:58:46,435][20347] Updated weights for policy 0, policy_version 1924290 (0.0006) [2023-12-27 18:58:46,474][20381] Updated weights for policy 1, policy_version 1922772 (0.0005) [2023-12-27 18:58:46,490][20347] Updated weights for policy 0, policy_version 1924300 (0.0006) [2023-12-27 18:58:46,540][20347] Updated weights for policy 0, policy_version 1924310 (0.0007) [2023-12-27 18:58:47,095][20381] Updated weights for policy 1, policy_version 1922782 (0.0009) [2023-12-27 18:58:47,156][20381] Updated weights for policy 1, policy_version 1922792 (0.0010) [2023-12-27 18:58:47,213][20347] Updated weights for policy 0, policy_version 1924320 (0.0006) [2023-12-27 18:58:47,215][20381] Updated weights for policy 1, policy_version 1922802 (0.0010) [2023-12-27 18:58:47,270][20347] Updated weights for policy 0, policy_version 1924330 (0.0008) [2023-12-27 18:58:47,324][20347] Updated weights for policy 0, policy_version 1924340 (0.0007) [2023-12-27 18:58:47,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 985022464. Throughput: 0: 10252.8, 1: 10286.0. Samples: 985026692. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.860')] [2023-12-27 18:58:47,784][20381] Updated weights for policy 1, policy_version 1922812 (0.0010) [2023-12-27 18:58:47,833][20381] Updated weights for policy 1, policy_version 1922822 (0.0010) [2023-12-27 18:58:47,877][20347] Updated weights for policy 0, policy_version 1924350 (0.0005) [2023-12-27 18:58:47,879][20381] Updated weights for policy 1, policy_version 1922832 (0.0010) [2023-12-27 18:58:47,926][20347] Updated weights for policy 0, policy_version 1924360 (0.0005) [2023-12-27 18:58:47,983][20347] Updated weights for policy 0, policy_version 1924370 (0.0007) [2023-12-27 18:58:48,514][20381] Updated weights for policy 1, policy_version 1922842 (0.0010) [2023-12-27 18:58:48,573][20381] Updated weights for policy 1, policy_version 1922852 (0.0009) [2023-12-27 18:58:48,632][20381] Updated weights for policy 1, policy_version 1922862 (0.0010) [2023-12-27 18:58:48,679][20347] Updated weights for policy 0, policy_version 1924380 (0.0007) [2023-12-27 18:58:48,693][20381] Updated weights for policy 1, policy_version 1922872 (0.0008) [2023-12-27 18:58:48,731][20347] Updated weights for policy 0, policy_version 1924390 (0.0009) [2023-12-27 18:58:48,791][20347] Updated weights for policy 0, policy_version 1924400 (0.0008) [2023-12-27 18:58:49,314][20381] Updated weights for policy 1, policy_version 1922882 (0.0008) [2023-12-27 18:58:49,376][20381] Updated weights for policy 1, policy_version 1922892 (0.0008) [2023-12-27 18:58:49,434][20381] Updated weights for policy 1, policy_version 1922902 (0.0009) [2023-12-27 18:58:49,479][20347] Updated weights for policy 0, policy_version 1924410 (0.0006) [2023-12-27 18:58:49,537][20347] Updated weights for policy 0, policy_version 1924420 (0.0007) [2023-12-27 18:58:49,593][20347] Updated weights for policy 0, policy_version 1924430 (0.0008) [2023-12-27 18:58:49,650][20347] Updated weights for policy 0, policy_version 1924440 (0.0007) [2023-12-27 18:58:50,136][20381] Updated weights for policy 1, policy_version 1922912 (0.0005) [2023-12-27 18:58:50,192][20381] Updated weights for policy 1, policy_version 1922922 (0.0006) [2023-12-27 18:58:50,253][20381] Updated weights for policy 1, policy_version 1922932 (0.0007) [2023-12-27 18:58:50,418][20347] Updated weights for policy 0, policy_version 1924450 (0.0008) [2023-12-27 18:58:50,474][20347] Updated weights for policy 0, policy_version 1924460 (0.0008) [2023-12-27 18:58:50,532][20347] Updated weights for policy 0, policy_version 1924470 (0.0007) [2023-12-27 18:58:50,863][20381] Updated weights for policy 1, policy_version 1922942 (0.0007) [2023-12-27 18:58:50,924][20381] Updated weights for policy 1, policy_version 1922952 (0.0008) [2023-12-27 18:58:50,982][20381] Updated weights for policy 1, policy_version 1922962 (0.0008) [2023-12-27 18:58:51,292][20347] Updated weights for policy 0, policy_version 1924480 (0.0007) [2023-12-27 18:58:51,360][20347] Updated weights for policy 0, policy_version 1924490 (0.0008) [2023-12-27 18:58:51,422][20347] Updated weights for policy 0, policy_version 1924500 (0.0007) [2023-12-27 18:58:51,647][20381] Updated weights for policy 1, policy_version 1922972 (0.0008) [2023-12-27 18:58:51,710][20381] Updated weights for policy 1, policy_version 1922982 (0.0008) [2023-12-27 18:58:51,771][20381] Updated weights for policy 1, policy_version 1922992 (0.0006) [2023-12-27 18:58:52,094][20347] Updated weights for policy 0, policy_version 1924510 (0.0008) [2023-12-27 18:58:52,153][20347] Updated weights for policy 0, policy_version 1924520 (0.0007) [2023-12-27 18:58:52,212][20347] Updated weights for policy 0, policy_version 1924530 (0.0006) [2023-12-27 18:58:52,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20479.9, 300 sec: 20521.6). Total num frames: 985120768. Throughput: 0: 10280.5, 1: 10309.8. Samples: 985090448. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.950')] [2023-12-27 18:58:52,393][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001924536_492773376.pth... [2023-12-27 18:58:52,396][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001923320_492462080.pth [2023-12-27 18:58:52,410][20381] Updated weights for policy 1, policy_version 1923002 (0.0005) [2023-12-27 18:58:52,471][20381] Updated weights for policy 1, policy_version 1923012 (0.0005) [2023-12-27 18:58:52,532][20381] Updated weights for policy 1, policy_version 1923022 (0.0007) [2023-12-27 18:58:52,594][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001923032_492355584.pth... [2023-12-27 18:58:52,596][20381] Updated weights for policy 1, policy_version 1923032 (0.0008) [2023-12-27 18:58:52,598][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001921824_492044288.pth [2023-12-27 18:58:52,931][20347] Updated weights for policy 0, policy_version 1924540 (0.0009) [2023-12-27 18:58:52,988][20347] Updated weights for policy 0, policy_version 1924550 (0.0009) [2023-12-27 18:58:53,043][20347] Updated weights for policy 0, policy_version 1924560 (0.0005) [2023-12-27 18:58:53,216][20381] Updated weights for policy 1, policy_version 1923042 (0.0007) [2023-12-27 18:58:53,269][20381] Updated weights for policy 1, policy_version 1923052 (0.0008) [2023-12-27 18:58:53,322][20381] Updated weights for policy 1, policy_version 1923062 (0.0010) [2023-12-27 18:58:53,641][20347] Updated weights for policy 0, policy_version 1924570 (0.0007) [2023-12-27 18:58:53,698][20347] Updated weights for policy 0, policy_version 1924580 (0.0007) [2023-12-27 18:58:53,759][20347] Updated weights for policy 0, policy_version 1924590 (0.0007) [2023-12-27 18:58:53,817][20347] Updated weights for policy 0, policy_version 1924600 (0.0008) [2023-12-27 18:58:54,044][20381] Updated weights for policy 1, policy_version 1923072 (0.0007) [2023-12-27 18:58:54,103][20381] Updated weights for policy 1, policy_version 1923082 (0.0008) [2023-12-27 18:58:54,162][20381] Updated weights for policy 1, policy_version 1923092 (0.0008) [2023-12-27 18:58:54,520][20347] Updated weights for policy 0, policy_version 1924610 (0.0005) [2023-12-27 18:58:54,567][20347] Updated weights for policy 0, policy_version 1924620 (0.0007) [2023-12-27 18:58:54,622][20347] Updated weights for policy 0, policy_version 1924630 (0.0008) [2023-12-27 18:58:54,866][20381] Updated weights for policy 1, policy_version 1923102 (0.0008) [2023-12-27 18:58:54,911][20381] Updated weights for policy 1, policy_version 1923112 (0.0008) [2023-12-27 18:58:54,967][20381] Updated weights for policy 1, policy_version 1923122 (0.0006) [2023-12-27 18:58:55,266][20347] Updated weights for policy 0, policy_version 1924640 (0.0007) [2023-12-27 18:58:55,328][20347] Updated weights for policy 0, policy_version 1924650 (0.0007) [2023-12-27 18:58:55,379][20347] Updated weights for policy 0, policy_version 1924660 (0.0007) [2023-12-27 18:58:55,571][20381] Updated weights for policy 1, policy_version 1923132 (0.0008) [2023-12-27 18:58:55,616][20381] Updated weights for policy 1, policy_version 1923142 (0.0008) [2023-12-27 18:58:55,668][20381] Updated weights for policy 1, policy_version 1923152 (0.0008) [2023-12-27 18:58:56,068][20347] Updated weights for policy 0, policy_version 1924670 (0.0008) [2023-12-27 18:58:56,125][20347] Updated weights for policy 0, policy_version 1924680 (0.0008) [2023-12-27 18:58:56,181][20347] Updated weights for policy 0, policy_version 1924690 (0.0008) [2023-12-27 18:58:56,346][20381] Updated weights for policy 1, policy_version 1923162 (0.0008) [2023-12-27 18:58:56,392][20381] Updated weights for policy 1, policy_version 1923172 (0.0008) [2023-12-27 18:58:56,441][20381] Updated weights for policy 1, policy_version 1923182 (0.0007) [2023-12-27 18:58:56,486][20381] Updated weights for policy 1, policy_version 1923192 (0.0008) [2023-12-27 18:58:56,868][20347] Updated weights for policy 0, policy_version 1924700 (0.0008) [2023-12-27 18:58:56,923][20347] Updated weights for policy 0, policy_version 1924710 (0.0005) [2023-12-27 18:58:56,971][20347] Updated weights for policy 0, policy_version 1924720 (0.0007) [2023-12-27 18:58:57,223][20381] Updated weights for policy 1, policy_version 1923202 (0.0008) [2023-12-27 18:58:57,280][20381] Updated weights for policy 1, policy_version 1923212 (0.0007) [2023-12-27 18:58:57,343][20381] Updated weights for policy 1, policy_version 1923222 (0.0007) [2023-12-27 18:58:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20549.4). Total num frames: 985227264. Throughput: 0: 10198.7, 1: 10315.9. Samples: 985213380. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-12-27 18:58:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.630')] [2023-12-27 18:58:57,555][20347] Updated weights for policy 0, policy_version 1924730 (0.0007) [2023-12-27 18:58:57,608][20347] Updated weights for policy 0, policy_version 1924740 (0.0006) [2023-12-27 18:58:57,656][20347] Updated weights for policy 0, policy_version 1924750 (0.0007) [2023-12-27 18:58:57,699][20347] Updated weights for policy 0, policy_version 1924760 (0.0007) [2023-12-27 18:58:58,034][20381] Updated weights for policy 1, policy_version 1923232 (0.0006) [2023-12-27 18:58:58,092][20381] Updated weights for policy 1, policy_version 1923242 (0.0008) [2023-12-27 18:58:58,145][20381] Updated weights for policy 1, policy_version 1923252 (0.0010) [2023-12-27 18:58:58,420][20347] Updated weights for policy 0, policy_version 1924770 (0.0007) [2023-12-27 18:58:58,486][20347] Updated weights for policy 0, policy_version 1924780 (0.0008) [2023-12-27 18:58:58,545][20347] Updated weights for policy 0, policy_version 1924790 (0.0008) [2023-12-27 18:58:58,873][20381] Updated weights for policy 1, policy_version 1923262 (0.0008) [2023-12-27 18:58:58,936][20381] Updated weights for policy 1, policy_version 1923272 (0.0008) [2023-12-27 18:58:58,998][20381] Updated weights for policy 1, policy_version 1923282 (0.0007) [2023-12-27 18:58:59,195][20347] Updated weights for policy 0, policy_version 1924800 (0.0007) [2023-12-27 18:58:59,260][20347] Updated weights for policy 0, policy_version 1924810 (0.0008) [2023-12-27 18:58:59,329][20347] Updated weights for policy 0, policy_version 1924820 (0.0008) [2023-12-27 18:58:59,751][20381] Updated weights for policy 1, policy_version 1923292 (0.0006) [2023-12-27 18:58:59,808][20381] Updated weights for policy 1, policy_version 1923302 (0.0006) [2023-12-27 18:58:59,866][20381] Updated weights for policy 1, policy_version 1923312 (0.0008) [2023-12-27 18:59:00,137][20347] Updated weights for policy 0, policy_version 1924830 (0.0009) [2023-12-27 18:59:00,193][20347] Updated weights for policy 0, policy_version 1924840 (0.0008) [2023-12-27 18:59:00,240][20347] Updated weights for policy 0, policy_version 1924850 (0.0008) [2023-12-27 18:59:00,566][20381] Updated weights for policy 1, policy_version 1923322 (0.0006) [2023-12-27 18:59:00,626][20381] Updated weights for policy 1, policy_version 1923332 (0.0007) [2023-12-27 18:59:00,686][20381] Updated weights for policy 1, policy_version 1923342 (0.0008) [2023-12-27 18:59:00,745][20381] Updated weights for policy 1, policy_version 1923352 (0.0007) [2023-12-27 18:59:00,955][20347] Updated weights for policy 0, policy_version 1924860 (0.0008) [2023-12-27 18:59:01,006][20347] Updated weights for policy 0, policy_version 1924870 (0.0008) [2023-12-27 18:59:01,069][20347] Updated weights for policy 0, policy_version 1924880 (0.0007) [2023-12-27 18:59:01,375][20381] Updated weights for policy 1, policy_version 1923362 (0.0009) [2023-12-27 18:59:01,432][20381] Updated weights for policy 1, policy_version 1923372 (0.0008) [2023-12-27 18:59:01,491][20381] Updated weights for policy 1, policy_version 1923382 (0.0006) [2023-12-27 18:59:01,781][20347] Updated weights for policy 0, policy_version 1924890 (0.0007) [2023-12-27 18:59:01,837][20347] Updated weights for policy 0, policy_version 1924900 (0.0007) [2023-12-27 18:59:01,889][20347] Updated weights for policy 0, policy_version 1924910 (0.0008) [2023-12-27 18:59:01,941][20347] Updated weights for policy 0, policy_version 1924920 (0.0007) [2023-12-27 18:59:02,128][20381] Updated weights for policy 1, policy_version 1923392 (0.0005) [2023-12-27 18:59:02,184][20381] Updated weights for policy 1, policy_version 1923402 (0.0006) [2023-12-27 18:59:02,240][20381] Updated weights for policy 1, policy_version 1923412 (0.0008) [2023-12-27 18:59:02,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20616.4, 300 sec: 20521.6). Total num frames: 985325568. Throughput: 0: 10170.0, 1: 10278.0. Samples: 985332832. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:02,364][19170] Avg episode reward: [(0, '1000.000'), (1, '941.370')] [2023-12-27 18:59:02,702][20347] Updated weights for policy 0, policy_version 1924930 (0.0008) [2023-12-27 18:59:02,757][20347] Updated weights for policy 0, policy_version 1924940 (0.0008) [2023-12-27 18:59:02,811][20347] Updated weights for policy 0, policy_version 1924950 (0.0007) [2023-12-27 18:59:02,949][20381] Updated weights for policy 1, policy_version 1923422 (0.0009) [2023-12-27 18:59:02,997][20381] Updated weights for policy 1, policy_version 1923432 (0.0008) [2023-12-27 18:59:03,043][20381] Updated weights for policy 1, policy_version 1923442 (0.0008) [2023-12-27 18:59:03,492][20347] Updated weights for policy 0, policy_version 1924960 (0.0007) [2023-12-27 18:59:03,542][20347] Updated weights for policy 0, policy_version 1924970 (0.0007) [2023-12-27 18:59:03,594][20347] Updated weights for policy 0, policy_version 1924980 (0.0007) [2023-12-27 18:59:03,755][20381] Updated weights for policy 1, policy_version 1923452 (0.0007) [2023-12-27 18:59:03,809][20381] Updated weights for policy 1, policy_version 1923462 (0.0008) [2023-12-27 18:59:03,868][20381] Updated weights for policy 1, policy_version 1923473 (0.0009) [2023-12-27 18:59:04,246][20347] Updated weights for policy 0, policy_version 1924990 (0.0007) [2023-12-27 18:59:04,307][20347] Updated weights for policy 0, policy_version 1925000 (0.0007) [2023-12-27 18:59:04,365][20347] Updated weights for policy 0, policy_version 1925010 (0.0007) [2023-12-27 18:59:04,567][20381] Updated weights for policy 1, policy_version 1923483 (0.0009) [2023-12-27 18:59:04,625][20381] Updated weights for policy 1, policy_version 1923493 (0.0010) [2023-12-27 18:59:04,674][20381] Updated weights for policy 1, policy_version 1923503 (0.0010) [2023-12-27 18:59:05,068][20347] Updated weights for policy 0, policy_version 1925020 (0.0007) [2023-12-27 18:59:05,125][20347] Updated weights for policy 0, policy_version 1925030 (0.0007) [2023-12-27 18:59:05,188][20347] Updated weights for policy 0, policy_version 1925040 (0.0008) [2023-12-27 18:59:05,321][20381] Updated weights for policy 1, policy_version 1923513 (0.0010) [2023-12-27 18:59:05,378][20381] Updated weights for policy 1, policy_version 1923523 (0.0010) [2023-12-27 18:59:05,432][20381] Updated weights for policy 1, policy_version 1923533 (0.0010) [2023-12-27 18:59:05,483][20381] Updated weights for policy 1, policy_version 1923543 (0.0010) [2023-12-27 18:59:05,881][20347] Updated weights for policy 0, policy_version 1925050 (0.0008) [2023-12-27 18:59:05,934][20347] Updated weights for policy 0, policy_version 1925060 (0.0007) [2023-12-27 18:59:05,991][20347] Updated weights for policy 0, policy_version 1925070 (0.0007) [2023-12-27 18:59:06,048][20347] Updated weights for policy 0, policy_version 1925080 (0.0008) [2023-12-27 18:59:06,192][20381] Updated weights for policy 1, policy_version 1923553 (0.0011) [2023-12-27 18:59:06,252][20381] Updated weights for policy 1, policy_version 1923563 (0.0008) [2023-12-27 18:59:06,315][20381] Updated weights for policy 1, policy_version 1923573 (0.0009) [2023-12-27 18:59:06,774][20347] Updated weights for policy 0, policy_version 1925090 (0.0006) [2023-12-27 18:59:06,840][20347] Updated weights for policy 0, policy_version 1925100 (0.0007) [2023-12-27 18:59:06,896][20347] Updated weights for policy 0, policy_version 1925110 (0.0010) [2023-12-27 18:59:06,971][20381] Updated weights for policy 1, policy_version 1923583 (0.0009) [2023-12-27 18:59:07,026][20381] Updated weights for policy 1, policy_version 1923593 (0.0010) [2023-12-27 18:59:07,079][20381] Updated weights for policy 1, policy_version 1923603 (0.0005) [2023-12-27 18:59:07,363][19170] Fps is (10 sec: 19660.6, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 985423872. Throughput: 0: 10160.2, 1: 10261.0. Samples: 985393404. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.500')] [2023-12-27 18:59:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001923608_492503040.pth... [2023-12-27 18:59:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001925112_492920832.pth... [2023-12-27 18:59:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001922400_492191744.pth [2023-12-27 18:59:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001923960_492625920.pth [2023-12-27 18:59:07,566][20347] Updated weights for policy 0, policy_version 1925120 (0.0008) [2023-12-27 18:59:07,618][20347] Updated weights for policy 0, policy_version 1925130 (0.0008) [2023-12-27 18:59:07,675][20381] Updated weights for policy 1, policy_version 1923613 (0.0007) [2023-12-27 18:59:07,679][20347] Updated weights for policy 0, policy_version 1925140 (0.0008) [2023-12-27 18:59:07,724][20381] Updated weights for policy 1, policy_version 1923623 (0.0005) [2023-12-27 18:59:07,784][20381] Updated weights for policy 1, policy_version 1923633 (0.0007) [2023-12-27 18:59:08,320][20347] Updated weights for policy 0, policy_version 1925150 (0.0006) [2023-12-27 18:59:08,384][20347] Updated weights for policy 0, policy_version 1925160 (0.0008) [2023-12-27 18:59:08,442][20347] Updated weights for policy 0, policy_version 1925170 (0.0008) [2023-12-27 18:59:08,483][20381] Updated weights for policy 1, policy_version 1923643 (0.0007) [2023-12-27 18:59:08,535][20381] Updated weights for policy 1, policy_version 1923653 (0.0007) [2023-12-27 18:59:08,588][20381] Updated weights for policy 1, policy_version 1923663 (0.0008) [2023-12-27 18:59:09,113][20347] Updated weights for policy 0, policy_version 1925180 (0.0007) [2023-12-27 18:59:09,164][20347] Updated weights for policy 0, policy_version 1925190 (0.0005) [2023-12-27 18:59:09,221][20347] Updated weights for policy 0, policy_version 1925200 (0.0006) [2023-12-27 18:59:09,256][20381] Updated weights for policy 1, policy_version 1923673 (0.0008) [2023-12-27 18:59:09,321][20381] Updated weights for policy 1, policy_version 1923683 (0.0010) [2023-12-27 18:59:09,386][20381] Updated weights for policy 1, policy_version 1923693 (0.0008) [2023-12-27 18:59:09,449][20381] Updated weights for policy 1, policy_version 1923703 (0.0007) [2023-12-27 18:59:09,929][20347] Updated weights for policy 0, policy_version 1925210 (0.0008) [2023-12-27 18:59:09,987][20347] Updated weights for policy 0, policy_version 1925220 (0.0007) [2023-12-27 18:59:10,048][20347] Updated weights for policy 0, policy_version 1925230 (0.0007) [2023-12-27 18:59:10,099][20347] Updated weights for policy 0, policy_version 1925240 (0.0007) [2023-12-27 18:59:10,179][20381] Updated weights for policy 1, policy_version 1923713 (0.0008) [2023-12-27 18:59:10,242][20381] Updated weights for policy 1, policy_version 1923723 (0.0008) [2023-12-27 18:59:10,293][20381] Updated weights for policy 1, policy_version 1923733 (0.0008) [2023-12-27 18:59:10,838][20347] Updated weights for policy 0, policy_version 1925250 (0.0007) [2023-12-27 18:59:10,905][20347] Updated weights for policy 0, policy_version 1925260 (0.0006) [2023-12-27 18:59:10,946][20381] Updated weights for policy 1, policy_version 1923743 (0.0010) [2023-12-27 18:59:10,960][20347] Updated weights for policy 0, policy_version 1925270 (0.0008) [2023-12-27 18:59:11,007][20381] Updated weights for policy 1, policy_version 1923753 (0.0010) [2023-12-27 18:59:11,074][20381] Updated weights for policy 1, policy_version 1923763 (0.0010) [2023-12-27 18:59:11,746][20347] Updated weights for policy 0, policy_version 1925280 (0.0008) [2023-12-27 18:59:11,809][20347] Updated weights for policy 0, policy_version 1925290 (0.0008) [2023-12-27 18:59:11,848][20381] Updated weights for policy 1, policy_version 1923773 (0.0011) [2023-12-27 18:59:11,856][20347] Updated weights for policy 0, policy_version 1925300 (0.0008) [2023-12-27 18:59:11,906][20381] Updated weights for policy 1, policy_version 1923783 (0.0011) [2023-12-27 18:59:11,952][20381] Updated weights for policy 1, policy_version 1923793 (0.0011) [2023-12-27 18:59:12,362][19170] Fps is (10 sec: 19661.7, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 985522176. Throughput: 0: 10155.9, 1: 10213.0. Samples: 985514536. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '922.010')] [2023-12-27 18:59:12,566][20347] Updated weights for policy 0, policy_version 1925310 (0.0007) [2023-12-27 18:59:12,625][20347] Updated weights for policy 0, policy_version 1925320 (0.0007) [2023-12-27 18:59:12,674][20381] Updated weights for policy 1, policy_version 1923803 (0.0009) [2023-12-27 18:59:12,683][20347] Updated weights for policy 0, policy_version 1925330 (0.0007) [2023-12-27 18:59:12,726][20381] Updated weights for policy 1, policy_version 1923813 (0.0006) [2023-12-27 18:59:12,789][20381] Updated weights for policy 1, policy_version 1923823 (0.0006) [2023-12-27 18:59:13,331][20347] Updated weights for policy 0, policy_version 1925340 (0.0007) [2023-12-27 18:59:13,380][20381] Updated weights for policy 1, policy_version 1923833 (0.0006) [2023-12-27 18:59:13,395][20347] Updated weights for policy 0, policy_version 1925350 (0.0007) [2023-12-27 18:59:13,437][20381] Updated weights for policy 1, policy_version 1923843 (0.0007) [2023-12-27 18:59:13,451][20347] Updated weights for policy 0, policy_version 1925360 (0.0007) [2023-12-27 18:59:13,480][20381] Updated weights for policy 1, policy_version 1923853 (0.0005) [2023-12-27 18:59:13,528][20381] Updated weights for policy 1, policy_version 1923863 (0.0008) [2023-12-27 18:59:14,125][20347] Updated weights for policy 0, policy_version 1925370 (0.0007) [2023-12-27 18:59:14,188][20381] Updated weights for policy 1, policy_version 1923873 (0.0009) [2023-12-27 18:59:14,188][20347] Updated weights for policy 0, policy_version 1925380 (0.0008) [2023-12-27 18:59:14,242][20381] Updated weights for policy 1, policy_version 1923883 (0.0008) [2023-12-27 18:59:14,246][20347] Updated weights for policy 0, policy_version 1925390 (0.0010) [2023-12-27 18:59:14,294][20381] Updated weights for policy 1, policy_version 1923893 (0.0008) [2023-12-27 18:59:14,302][20347] Updated weights for policy 0, policy_version 1925400 (0.0010) [2023-12-27 18:59:14,978][20347] Updated weights for policy 0, policy_version 1925410 (0.0010) [2023-12-27 18:59:14,982][20381] Updated weights for policy 1, policy_version 1923903 (0.0009) [2023-12-27 18:59:15,035][20347] Updated weights for policy 0, policy_version 1925420 (0.0010) [2023-12-27 18:59:15,041][20381] Updated weights for policy 1, policy_version 1923913 (0.0006) [2023-12-27 18:59:15,093][20347] Updated weights for policy 0, policy_version 1925430 (0.0010) [2023-12-27 18:59:15,099][20381] Updated weights for policy 1, policy_version 1923923 (0.0007) [2023-12-27 18:59:15,739][20381] Updated weights for policy 1, policy_version 1923933 (0.0007) [2023-12-27 18:59:15,746][20347] Updated weights for policy 0, policy_version 1925440 (0.0005) [2023-12-27 18:59:15,795][20381] Updated weights for policy 1, policy_version 1923943 (0.0008) [2023-12-27 18:59:15,802][20347] Updated weights for policy 0, policy_version 1925450 (0.0006) [2023-12-27 18:59:15,842][20381] Updated weights for policy 1, policy_version 1923953 (0.0005) [2023-12-27 18:59:15,855][20347] Updated weights for policy 0, policy_version 1925460 (0.0007) [2023-12-27 18:59:16,468][20347] Updated weights for policy 0, policy_version 1925470 (0.0006) [2023-12-27 18:59:16,519][20381] Updated weights for policy 1, policy_version 1923963 (0.0006) [2023-12-27 18:59:16,522][20347] Updated weights for policy 0, policy_version 1925480 (0.0006) [2023-12-27 18:59:16,574][20381] Updated weights for policy 1, policy_version 1923973 (0.0005) [2023-12-27 18:59:16,577][20347] Updated weights for policy 0, policy_version 1925490 (0.0010) [2023-12-27 18:59:16,633][20381] Updated weights for policy 1, policy_version 1923983 (0.0007) [2023-12-27 18:59:17,218][20347] Updated weights for policy 0, policy_version 1925500 (0.0007) [2023-12-27 18:59:17,272][20347] Updated weights for policy 0, policy_version 1925510 (0.0010) [2023-12-27 18:59:17,300][20381] Updated weights for policy 1, policy_version 1923993 (0.0007) [2023-12-27 18:59:17,331][20347] Updated weights for policy 0, policy_version 1925520 (0.0008) [2023-12-27 18:59:17,353][20381] Updated weights for policy 1, policy_version 1924003 (0.0006) [2023-12-27 18:59:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 985620480. Throughput: 0: 10127.9, 1: 10252.4. Samples: 985638204. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.370')] [2023-12-27 18:59:17,413][20381] Updated weights for policy 1, policy_version 1924013 (0.0007) [2023-12-27 18:59:17,479][20381] Updated weights for policy 1, policy_version 1924023 (0.0008) [2023-12-27 18:59:17,944][20347] Updated weights for policy 0, policy_version 1925530 (0.0009) [2023-12-27 18:59:17,998][20347] Updated weights for policy 0, policy_version 1925540 (0.0005) [2023-12-27 18:59:18,049][20347] Updated weights for policy 0, policy_version 1925550 (0.0006) [2023-12-27 18:59:18,108][20347] Updated weights for policy 0, policy_version 1925560 (0.0008) [2023-12-27 18:59:18,182][20381] Updated weights for policy 1, policy_version 1924033 (0.0007) [2023-12-27 18:59:18,240][20381] Updated weights for policy 1, policy_version 1924043 (0.0006) [2023-12-27 18:59:18,303][20381] Updated weights for policy 1, policy_version 1924053 (0.0005) [2023-12-27 18:59:18,787][20347] Updated weights for policy 0, policy_version 1925570 (0.0007) [2023-12-27 18:59:18,849][20347] Updated weights for policy 0, policy_version 1925580 (0.0007) [2023-12-27 18:59:18,908][20347] Updated weights for policy 0, policy_version 1925590 (0.0006) [2023-12-27 18:59:18,949][20381] Updated weights for policy 1, policy_version 1924063 (0.0008) [2023-12-27 18:59:19,007][20381] Updated weights for policy 1, policy_version 1924073 (0.0007) [2023-12-27 18:59:19,058][20381] Updated weights for policy 1, policy_version 1924083 (0.0007) [2023-12-27 18:59:19,585][20347] Updated weights for policy 0, policy_version 1925600 (0.0006) [2023-12-27 18:59:19,642][20347] Updated weights for policy 0, policy_version 1925610 (0.0007) [2023-12-27 18:59:19,706][20347] Updated weights for policy 0, policy_version 1925620 (0.0008) [2023-12-27 18:59:19,776][20381] Updated weights for policy 1, policy_version 1924093 (0.0008) [2023-12-27 18:59:19,837][20381] Updated weights for policy 1, policy_version 1924103 (0.0008) [2023-12-27 18:59:19,897][20381] Updated weights for policy 1, policy_version 1924113 (0.0008) [2023-12-27 18:59:20,396][20347] Updated weights for policy 0, policy_version 1925630 (0.0008) [2023-12-27 18:59:20,452][20347] Updated weights for policy 0, policy_version 1925640 (0.0008) [2023-12-27 18:59:20,509][20347] Updated weights for policy 0, policy_version 1925650 (0.0008) [2023-12-27 18:59:20,586][20381] Updated weights for policy 1, policy_version 1924123 (0.0007) [2023-12-27 18:59:20,646][20381] Updated weights for policy 1, policy_version 1924133 (0.0008) [2023-12-27 18:59:20,709][20381] Updated weights for policy 1, policy_version 1924143 (0.0007) [2023-12-27 18:59:21,215][20347] Updated weights for policy 0, policy_version 1925660 (0.0009) [2023-12-27 18:59:21,282][20347] Updated weights for policy 0, policy_version 1925670 (0.0009) [2023-12-27 18:59:21,344][20347] Updated weights for policy 0, policy_version 1925680 (0.0009) [2023-12-27 18:59:21,490][20381] Updated weights for policy 1, policy_version 1924153 (0.0009) [2023-12-27 18:59:21,543][20381] Updated weights for policy 1, policy_version 1924163 (0.0008) [2023-12-27 18:59:21,600][20381] Updated weights for policy 1, policy_version 1924173 (0.0008) [2023-12-27 18:59:21,666][20381] Updated weights for policy 1, policy_version 1924183 (0.0008) [2023-12-27 18:59:22,053][20347] Updated weights for policy 0, policy_version 1925690 (0.0008) [2023-12-27 18:59:22,114][20347] Updated weights for policy 0, policy_version 1925700 (0.0006) [2023-12-27 18:59:22,179][20347] Updated weights for policy 0, policy_version 1925710 (0.0006) [2023-12-27 18:59:22,239][20347] Updated weights for policy 0, policy_version 1925720 (0.0007) [2023-12-27 18:59:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 985726976. Throughput: 0: 10105.9, 1: 10243.9. Samples: 985698936. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.850')] [2023-12-27 18:59:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001925720_493076480.pth... [2023-12-27 18:59:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001924536_492773376.pth [2023-12-27 18:59:22,417][20381] Updated weights for policy 1, policy_version 1924193 (0.0008) [2023-12-27 18:59:22,483][20381] Updated weights for policy 1, policy_version 1924203 (0.0007) [2023-12-27 18:59:22,545][20381] Updated weights for policy 1, policy_version 1924213 (0.0008) [2023-12-27 18:59:22,561][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001924216_492658688.pth... [2023-12-27 18:59:22,566][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001923032_492355584.pth [2023-12-27 18:59:22,943][20347] Updated weights for policy 0, policy_version 1925730 (0.0009) [2023-12-27 18:59:22,992][20347] Updated weights for policy 0, policy_version 1925740 (0.0008) [2023-12-27 18:59:23,042][20347] Updated weights for policy 0, policy_version 1925750 (0.0007) [2023-12-27 18:59:23,195][20381] Updated weights for policy 1, policy_version 1924223 (0.0007) [2023-12-27 18:59:23,252][20381] Updated weights for policy 1, policy_version 1924233 (0.0009) [2023-12-27 18:59:23,306][20381] Updated weights for policy 1, policy_version 1924243 (0.0008) [2023-12-27 18:59:23,662][20347] Updated weights for policy 0, policy_version 1925760 (0.0007) [2023-12-27 18:59:23,718][20347] Updated weights for policy 0, policy_version 1925770 (0.0007) [2023-12-27 18:59:23,769][20347] Updated weights for policy 0, policy_version 1925780 (0.0006) [2023-12-27 18:59:23,941][20381] Updated weights for policy 1, policy_version 1924253 (0.0007) [2023-12-27 18:59:24,003][20381] Updated weights for policy 1, policy_version 1924263 (0.0008) [2023-12-27 18:59:24,061][20381] Updated weights for policy 1, policy_version 1924273 (0.0008) [2023-12-27 18:59:24,363][20347] Updated weights for policy 0, policy_version 1925790 (0.0005) [2023-12-27 18:59:24,411][20347] Updated weights for policy 0, policy_version 1925800 (0.0005) [2023-12-27 18:59:24,457][20347] Updated weights for policy 0, policy_version 1925810 (0.0005) [2023-12-27 18:59:24,787][20381] Updated weights for policy 1, policy_version 1924283 (0.0007) [2023-12-27 18:59:24,842][20381] Updated weights for policy 1, policy_version 1924293 (0.0008) [2023-12-27 18:59:24,897][20381] Updated weights for policy 1, policy_version 1924303 (0.0007) [2023-12-27 18:59:25,059][20347] Updated weights for policy 0, policy_version 1925820 (0.0007) [2023-12-27 18:59:25,107][20347] Updated weights for policy 0, policy_version 1925830 (0.0007) [2023-12-27 18:59:25,151][20347] Updated weights for policy 0, policy_version 1925840 (0.0008) [2023-12-27 18:59:25,576][20381] Updated weights for policy 1, policy_version 1924313 (0.0007) [2023-12-27 18:59:25,629][20381] Updated weights for policy 1, policy_version 1924323 (0.0006) [2023-12-27 18:59:25,674][20381] Updated weights for policy 1, policy_version 1924333 (0.0007) [2023-12-27 18:59:25,725][20381] Updated weights for policy 1, policy_version 1924343 (0.0008) [2023-12-27 18:59:25,852][20347] Updated weights for policy 0, policy_version 1925850 (0.0007) [2023-12-27 18:59:25,906][20347] Updated weights for policy 0, policy_version 1925860 (0.0005) [2023-12-27 18:59:25,960][20347] Updated weights for policy 0, policy_version 1925870 (0.0005) [2023-12-27 18:59:26,013][20347] Updated weights for policy 0, policy_version 1925880 (0.0007) [2023-12-27 18:59:26,341][20381] Updated weights for policy 1, policy_version 1924353 (0.0010) [2023-12-27 18:59:26,392][20381] Updated weights for policy 1, policy_version 1924363 (0.0008) [2023-12-27 18:59:26,440][20381] Updated weights for policy 1, policy_version 1924373 (0.0010) [2023-12-27 18:59:26,651][20347] Updated weights for policy 0, policy_version 1925890 (0.0007) [2023-12-27 18:59:26,710][20347] Updated weights for policy 0, policy_version 1925900 (0.0007) [2023-12-27 18:59:26,767][20347] Updated weights for policy 0, policy_version 1925910 (0.0007) [2023-12-27 18:59:27,139][20381] Updated weights for policy 1, policy_version 1924383 (0.0010) [2023-12-27 18:59:27,193][20381] Updated weights for policy 1, policy_version 1924393 (0.0010) [2023-12-27 18:59:27,244][20381] Updated weights for policy 1, policy_version 1924403 (0.0010) [2023-12-27 18:59:27,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 985833472. Throughput: 0: 10126.2, 1: 10238.6. Samples: 985822088. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.160')] [2023-12-27 18:59:27,447][20347] Updated weights for policy 0, policy_version 1925920 (0.0007) [2023-12-27 18:59:27,511][20347] Updated weights for policy 0, policy_version 1925930 (0.0007) [2023-12-27 18:59:27,563][20347] Updated weights for policy 0, policy_version 1925940 (0.0007) [2023-12-27 18:59:27,924][20381] Updated weights for policy 1, policy_version 1924413 (0.0010) [2023-12-27 18:59:27,969][20381] Updated weights for policy 1, policy_version 1924423 (0.0010) [2023-12-27 18:59:28,014][20381] Updated weights for policy 1, policy_version 1924433 (0.0010) [2023-12-27 18:59:28,156][20347] Updated weights for policy 0, policy_version 1925950 (0.0006) [2023-12-27 18:59:28,204][20347] Updated weights for policy 0, policy_version 1925960 (0.0008) [2023-12-27 18:59:28,271][20347] Updated weights for policy 0, policy_version 1925970 (0.0008) [2023-12-27 18:59:28,692][20381] Updated weights for policy 1, policy_version 1924443 (0.0010) [2023-12-27 18:59:28,748][20381] Updated weights for policy 1, policy_version 1924453 (0.0007) [2023-12-27 18:59:28,798][20381] Updated weights for policy 1, policy_version 1924463 (0.0007) [2023-12-27 18:59:28,962][20347] Updated weights for policy 0, policy_version 1925980 (0.0007) [2023-12-27 18:59:29,018][20347] Updated weights for policy 0, policy_version 1925990 (0.0007) [2023-12-27 18:59:29,071][20347] Updated weights for policy 0, policy_version 1926000 (0.0007) [2023-12-27 18:59:29,449][20381] Updated weights for policy 1, policy_version 1924473 (0.0007) [2023-12-27 18:59:29,536][20381] Updated weights for policy 1, policy_version 1924483 (0.0008) [2023-12-27 18:59:29,597][20381] Updated weights for policy 1, policy_version 1924493 (0.0008) [2023-12-27 18:59:29,657][20381] Updated weights for policy 1, policy_version 1924503 (0.0008) [2023-12-27 18:59:29,738][20347] Updated weights for policy 0, policy_version 1926010 (0.0007) [2023-12-27 18:59:29,795][20347] Updated weights for policy 0, policy_version 1926020 (0.0008) [2023-12-27 18:59:29,856][20347] Updated weights for policy 0, policy_version 1926030 (0.0008) [2023-12-27 18:59:29,925][20347] Updated weights for policy 0, policy_version 1926040 (0.0007) [2023-12-27 18:59:30,310][20381] Updated weights for policy 1, policy_version 1924513 (0.0011) [2023-12-27 18:59:30,371][20381] Updated weights for policy 1, policy_version 1924523 (0.0011) [2023-12-27 18:59:30,423][20381] Updated weights for policy 1, policy_version 1924533 (0.0010) [2023-12-27 18:59:30,626][20347] Updated weights for policy 0, policy_version 1926050 (0.0008) [2023-12-27 18:59:30,686][20347] Updated weights for policy 0, policy_version 1926060 (0.0005) [2023-12-27 18:59:30,748][20347] Updated weights for policy 0, policy_version 1926070 (0.0005) [2023-12-27 18:59:31,012][20381] Updated weights for policy 1, policy_version 1924543 (0.0010) [2023-12-27 18:59:31,077][20381] Updated weights for policy 1, policy_version 1924553 (0.0011) [2023-12-27 18:59:31,143][20381] Updated weights for policy 1, policy_version 1924563 (0.0010) [2023-12-27 18:59:31,432][20347] Updated weights for policy 0, policy_version 1926080 (0.0007) [2023-12-27 18:59:31,487][20347] Updated weights for policy 0, policy_version 1926090 (0.0007) [2023-12-27 18:59:31,532][20347] Updated weights for policy 0, policy_version 1926100 (0.0007) [2023-12-27 18:59:31,903][20381] Updated weights for policy 1, policy_version 1924573 (0.0009) [2023-12-27 18:59:31,953][20381] Updated weights for policy 1, policy_version 1924583 (0.0010) [2023-12-27 18:59:32,009][20381] Updated weights for policy 1, policy_version 1924593 (0.0007) [2023-12-27 18:59:32,221][20347] Updated weights for policy 0, policy_version 1926110 (0.0007) [2023-12-27 18:59:32,283][20347] Updated weights for policy 0, policy_version 1926120 (0.0008) [2023-12-27 18:59:32,338][20347] Updated weights for policy 0, policy_version 1926130 (0.0006) [2023-12-27 18:59:32,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20521.6). Total num frames: 985931776. Throughput: 0: 10177.6, 1: 10240.6. Samples: 985945516. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.340')] [2023-12-27 18:59:32,658][20381] Updated weights for policy 1, policy_version 1924603 (0.0008) [2023-12-27 18:59:32,705][20381] Updated weights for policy 1, policy_version 1924613 (0.0008) [2023-12-27 18:59:32,757][20381] Updated weights for policy 1, policy_version 1924623 (0.0007) [2023-12-27 18:59:33,009][20347] Updated weights for policy 0, policy_version 1926140 (0.0006) [2023-12-27 18:59:33,063][20347] Updated weights for policy 0, policy_version 1926150 (0.0007) [2023-12-27 18:59:33,116][20347] Updated weights for policy 0, policy_version 1926160 (0.0007) [2023-12-27 18:59:33,460][20381] Updated weights for policy 1, policy_version 1924633 (0.0008) [2023-12-27 18:59:33,517][20381] Updated weights for policy 1, policy_version 1924643 (0.0005) [2023-12-27 18:59:33,567][20381] Updated weights for policy 1, policy_version 1924653 (0.0005) [2023-12-27 18:59:33,624][20381] Updated weights for policy 1, policy_version 1924663 (0.0005) [2023-12-27 18:59:33,821][20347] Updated weights for policy 0, policy_version 1926170 (0.0007) [2023-12-27 18:59:33,881][20347] Updated weights for policy 0, policy_version 1926180 (0.0008) [2023-12-27 18:59:33,947][20347] Updated weights for policy 0, policy_version 1926190 (0.0008) [2023-12-27 18:59:34,006][20347] Updated weights for policy 0, policy_version 1926200 (0.0007) [2023-12-27 18:59:34,182][20381] Updated weights for policy 1, policy_version 1924673 (0.0007) [2023-12-27 18:59:34,237][20381] Updated weights for policy 1, policy_version 1924683 (0.0008) [2023-12-27 18:59:34,292][20381] Updated weights for policy 1, policy_version 1924693 (0.0008) [2023-12-27 18:59:34,704][20347] Updated weights for policy 0, policy_version 1926210 (0.0008) [2023-12-27 18:59:34,750][20347] Updated weights for policy 0, policy_version 1926220 (0.0008) [2023-12-27 18:59:34,818][20347] Updated weights for policy 0, policy_version 1926230 (0.0008) [2023-12-27 18:59:34,971][20381] Updated weights for policy 1, policy_version 1924703 (0.0007) [2023-12-27 18:59:35,029][20381] Updated weights for policy 1, policy_version 1924713 (0.0007) [2023-12-27 18:59:35,085][20381] Updated weights for policy 1, policy_version 1924723 (0.0007) [2023-12-27 18:59:35,524][20347] Updated weights for policy 0, policy_version 1926240 (0.0007) [2023-12-27 18:59:35,575][20347] Updated weights for policy 0, policy_version 1926250 (0.0007) [2023-12-27 18:59:35,629][20347] Updated weights for policy 0, policy_version 1926260 (0.0007) [2023-12-27 18:59:35,689][20381] Updated weights for policy 1, policy_version 1924733 (0.0008) [2023-12-27 18:59:35,737][20381] Updated weights for policy 1, policy_version 1924743 (0.0008) [2023-12-27 18:59:35,787][20381] Updated weights for policy 1, policy_version 1924753 (0.0005) [2023-12-27 18:59:36,280][20347] Updated weights for policy 0, policy_version 1926270 (0.0007) [2023-12-27 18:59:36,339][20347] Updated weights for policy 0, policy_version 1926280 (0.0007) [2023-12-27 18:59:36,395][20347] Updated weights for policy 0, policy_version 1926290 (0.0007) [2023-12-27 18:59:36,488][20381] Updated weights for policy 1, policy_version 1924763 (0.0006) [2023-12-27 18:59:36,545][20381] Updated weights for policy 1, policy_version 1924773 (0.0008) [2023-12-27 18:59:36,599][20381] Updated weights for policy 1, policy_version 1924783 (0.0008) [2023-12-27 18:59:37,126][20347] Updated weights for policy 0, policy_version 1926300 (0.0008) [2023-12-27 18:59:37,178][20347] Updated weights for policy 0, policy_version 1926310 (0.0007) [2023-12-27 18:59:37,226][20347] Updated weights for policy 0, policy_version 1926320 (0.0008) [2023-12-27 18:59:37,282][20381] Updated weights for policy 1, policy_version 1924793 (0.0007) [2023-12-27 18:59:37,334][20381] Updated weights for policy 1, policy_version 1924803 (0.0005) [2023-12-27 18:59:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 986038272. Throughput: 0: 10153.2, 1: 10229.2. Samples: 986007652. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.550')] [2023-12-27 18:59:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001926328_493232128.pth... [2023-12-27 18:59:37,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001925112_492920832.pth [2023-12-27 18:59:37,392][20381] Updated weights for policy 1, policy_version 1924813 (0.0009) [2023-12-27 18:59:37,454][20381] Updated weights for policy 1, policy_version 1924823 (0.0011) [2023-12-27 18:59:37,456][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001924824_492814336.pth... [2023-12-27 18:59:37,459][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001923608_492503040.pth [2023-12-27 18:59:37,860][20347] Updated weights for policy 0, policy_version 1926330 (0.0009) [2023-12-27 18:59:37,920][20347] Updated weights for policy 0, policy_version 1926340 (0.0007) [2023-12-27 18:59:37,974][20347] Updated weights for policy 0, policy_version 1926350 (0.0007) [2023-12-27 18:59:38,029][20347] Updated weights for policy 0, policy_version 1926360 (0.0009) [2023-12-27 18:59:38,098][20381] Updated weights for policy 1, policy_version 1924833 (0.0006) [2023-12-27 18:59:38,156][20381] Updated weights for policy 1, policy_version 1924843 (0.0006) [2023-12-27 18:59:38,213][20381] Updated weights for policy 1, policy_version 1924853 (0.0007) [2023-12-27 18:59:38,742][20347] Updated weights for policy 0, policy_version 1926370 (0.0010) [2023-12-27 18:59:38,797][20347] Updated weights for policy 0, policy_version 1926380 (0.0010) [2023-12-27 18:59:38,855][20347] Updated weights for policy 0, policy_version 1926390 (0.0010) [2023-12-27 18:59:38,923][20381] Updated weights for policy 1, policy_version 1924863 (0.0009) [2023-12-27 18:59:38,968][20381] Updated weights for policy 1, policy_version 1924873 (0.0008) [2023-12-27 18:59:39,018][20381] Updated weights for policy 1, policy_version 1924883 (0.0006) [2023-12-27 18:59:39,551][20347] Updated weights for policy 0, policy_version 1926400 (0.0008) [2023-12-27 18:59:39,612][20347] Updated weights for policy 0, policy_version 1926410 (0.0005) [2023-12-27 18:59:39,675][20347] Updated weights for policy 0, policy_version 1926420 (0.0006) [2023-12-27 18:59:39,787][20381] Updated weights for policy 1, policy_version 1924893 (0.0008) [2023-12-27 18:59:39,853][20381] Updated weights for policy 1, policy_version 1924903 (0.0008) [2023-12-27 18:59:39,913][20381] Updated weights for policy 1, policy_version 1924913 (0.0007) [2023-12-27 18:59:40,316][20347] Updated weights for policy 0, policy_version 1926430 (0.0006) [2023-12-27 18:59:40,378][20347] Updated weights for policy 0, policy_version 1926440 (0.0008) [2023-12-27 18:59:40,439][20347] Updated weights for policy 0, policy_version 1926450 (0.0010) [2023-12-27 18:59:40,647][20381] Updated weights for policy 1, policy_version 1924923 (0.0007) [2023-12-27 18:59:40,692][20381] Updated weights for policy 1, policy_version 1924933 (0.0007) [2023-12-27 18:59:40,738][20381] Updated weights for policy 1, policy_version 1924943 (0.0007) [2023-12-27 18:59:41,102][20347] Updated weights for policy 0, policy_version 1926460 (0.0010) [2023-12-27 18:59:41,166][20347] Updated weights for policy 0, policy_version 1926470 (0.0009) [2023-12-27 18:59:41,228][20347] Updated weights for policy 0, policy_version 1926480 (0.0010) [2023-12-27 18:59:41,477][20381] Updated weights for policy 1, policy_version 1924953 (0.0007) [2023-12-27 18:59:41,535][20381] Updated weights for policy 1, policy_version 1924963 (0.0006) [2023-12-27 18:59:41,593][20381] Updated weights for policy 1, policy_version 1924973 (0.0006) [2023-12-27 18:59:41,659][20381] Updated weights for policy 1, policy_version 1924983 (0.0007) [2023-12-27 18:59:41,990][20347] Updated weights for policy 0, policy_version 1926490 (0.0008) [2023-12-27 18:59:42,049][20347] Updated weights for policy 0, policy_version 1926500 (0.0011) [2023-12-27 18:59:42,115][20347] Updated weights for policy 0, policy_version 1926510 (0.0010) [2023-12-27 18:59:42,177][20347] Updated weights for policy 0, policy_version 1926520 (0.0008) [2023-12-27 18:59:42,281][20381] Updated weights for policy 1, policy_version 1924993 (0.0007) [2023-12-27 18:59:42,348][20381] Updated weights for policy 1, policy_version 1925003 (0.0007) [2023-12-27 18:59:42,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20480.0, 300 sec: 20521.6). Total num frames: 986136576. Throughput: 0: 10179.6, 1: 10185.5. Samples: 986129808. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.460')] [2023-12-27 18:59:42,408][20381] Updated weights for policy 1, policy_version 1925013 (0.0007) [2023-12-27 18:59:42,868][20347] Updated weights for policy 0, policy_version 1926530 (0.0010) [2023-12-27 18:59:42,920][20347] Updated weights for policy 0, policy_version 1926540 (0.0010) [2023-12-27 18:59:42,984][20347] Updated weights for policy 0, policy_version 1926550 (0.0010) [2023-12-27 18:59:43,094][20381] Updated weights for policy 1, policy_version 1925023 (0.0008) [2023-12-27 18:59:43,150][20381] Updated weights for policy 1, policy_version 1925033 (0.0011) [2023-12-27 18:59:43,209][20381] Updated weights for policy 1, policy_version 1925043 (0.0011) [2023-12-27 18:59:43,660][20347] Updated weights for policy 0, policy_version 1926560 (0.0007) [2023-12-27 18:59:43,723][20347] Updated weights for policy 0, policy_version 1926570 (0.0008) [2023-12-27 18:59:43,786][20347] Updated weights for policy 0, policy_version 1926580 (0.0008) [2023-12-27 18:59:43,864][20381] Updated weights for policy 1, policy_version 1925053 (0.0008) [2023-12-27 18:59:43,920][20381] Updated weights for policy 1, policy_version 1925063 (0.0006) [2023-12-27 18:59:43,969][20381] Updated weights for policy 1, policy_version 1925073 (0.0006) [2023-12-27 18:59:44,478][20347] Updated weights for policy 0, policy_version 1926590 (0.0008) [2023-12-27 18:59:44,531][20347] Updated weights for policy 0, policy_version 1926600 (0.0006) [2023-12-27 18:59:44,579][20347] Updated weights for policy 0, policy_version 1926610 (0.0008) [2023-12-27 18:59:44,610][20381] Updated weights for policy 1, policy_version 1925083 (0.0007) [2023-12-27 18:59:44,656][20381] Updated weights for policy 1, policy_version 1925093 (0.0005) [2023-12-27 18:59:44,702][20381] Updated weights for policy 1, policy_version 1925103 (0.0007) [2023-12-27 18:59:45,317][20347] Updated weights for policy 0, policy_version 1926620 (0.0007) [2023-12-27 18:59:45,364][20381] Updated weights for policy 1, policy_version 1925113 (0.0006) [2023-12-27 18:59:45,370][20347] Updated weights for policy 0, policy_version 1926630 (0.0007) [2023-12-27 18:59:45,428][20347] Updated weights for policy 0, policy_version 1926640 (0.0006) [2023-12-27 18:59:45,430][20381] Updated weights for policy 1, policy_version 1925123 (0.0008) [2023-12-27 18:59:45,493][20381] Updated weights for policy 1, policy_version 1925133 (0.0008) [2023-12-27 18:59:45,562][20381] Updated weights for policy 1, policy_version 1925143 (0.0007) [2023-12-27 18:59:46,037][20347] Updated weights for policy 0, policy_version 1926650 (0.0006) [2023-12-27 18:59:46,092][20347] Updated weights for policy 0, policy_version 1926660 (0.0005) [2023-12-27 18:59:46,147][20347] Updated weights for policy 0, policy_version 1926670 (0.0005) [2023-12-27 18:59:46,192][20347] Updated weights for policy 0, policy_version 1926680 (0.0006) [2023-12-27 18:59:46,251][20381] Updated weights for policy 1, policy_version 1925153 (0.0008) [2023-12-27 18:59:46,311][20381] Updated weights for policy 1, policy_version 1925163 (0.0008) [2023-12-27 18:59:46,381][20381] Updated weights for policy 1, policy_version 1925173 (0.0008) [2023-12-27 18:59:46,778][20347] Updated weights for policy 0, policy_version 1926690 (0.0005) [2023-12-27 18:59:46,826][20347] Updated weights for policy 0, policy_version 1926700 (0.0005) [2023-12-27 18:59:46,871][20347] Updated weights for policy 0, policy_version 1926710 (0.0005) [2023-12-27 18:59:47,107][20381] Updated weights for policy 1, policy_version 1925183 (0.0007) [2023-12-27 18:59:47,161][20381] Updated weights for policy 1, policy_version 1925193 (0.0008) [2023-12-27 18:59:47,219][20381] Updated weights for policy 1, policy_version 1925203 (0.0008) [2023-12-27 18:59:47,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20343.4, 300 sec: 20549.4). Total num frames: 986243072. Throughput: 0: 10219.0, 1: 10199.0. Samples: 986251636. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '891.930')] [2023-12-27 18:59:47,481][20347] Updated weights for policy 0, policy_version 1926720 (0.0007) [2023-12-27 18:59:47,530][20347] Updated weights for policy 0, policy_version 1926730 (0.0009) [2023-12-27 18:59:47,582][20347] Updated weights for policy 0, policy_version 1926740 (0.0010) [2023-12-27 18:59:47,904][20381] Updated weights for policy 1, policy_version 1925213 (0.0007) [2023-12-27 18:59:47,948][20381] Updated weights for policy 1, policy_version 1925223 (0.0005) [2023-12-27 18:59:47,994][20381] Updated weights for policy 1, policy_version 1925233 (0.0005) [2023-12-27 18:59:48,268][20347] Updated weights for policy 0, policy_version 1926750 (0.0007) [2023-12-27 18:59:48,328][20347] Updated weights for policy 0, policy_version 1926760 (0.0007) [2023-12-27 18:59:48,393][20347] Updated weights for policy 0, policy_version 1926770 (0.0008) [2023-12-27 18:59:48,653][20381] Updated weights for policy 1, policy_version 1925243 (0.0007) [2023-12-27 18:59:48,710][20381] Updated weights for policy 1, policy_version 1925253 (0.0009) [2023-12-27 18:59:48,768][20381] Updated weights for policy 1, policy_version 1925263 (0.0008) [2023-12-27 18:59:49,010][20347] Updated weights for policy 0, policy_version 1926780 (0.0006) [2023-12-27 18:59:49,065][20347] Updated weights for policy 0, policy_version 1926790 (0.0005) [2023-12-27 18:59:49,114][20347] Updated weights for policy 0, policy_version 1926800 (0.0005) [2023-12-27 18:59:49,439][20381] Updated weights for policy 1, policy_version 1925273 (0.0008) [2023-12-27 18:59:49,492][20381] Updated weights for policy 1, policy_version 1925283 (0.0008) [2023-12-27 18:59:49,553][20381] Updated weights for policy 1, policy_version 1925293 (0.0005) [2023-12-27 18:59:49,607][20381] Updated weights for policy 1, policy_version 1925303 (0.0008) [2023-12-27 18:59:49,821][20347] Updated weights for policy 0, policy_version 1926810 (0.0006) [2023-12-27 18:59:49,888][20347] Updated weights for policy 0, policy_version 1926820 (0.0008) [2023-12-27 18:59:49,957][20347] Updated weights for policy 0, policy_version 1926830 (0.0008) [2023-12-27 18:59:50,021][20347] Updated weights for policy 0, policy_version 1926840 (0.0007) [2023-12-27 18:59:50,268][20381] Updated weights for policy 1, policy_version 1925313 (0.0008) [2023-12-27 18:59:50,331][20381] Updated weights for policy 1, policy_version 1925323 (0.0008) [2023-12-27 18:59:50,391][20381] Updated weights for policy 1, policy_version 1925333 (0.0007) [2023-12-27 18:59:50,674][20347] Updated weights for policy 0, policy_version 1926850 (0.0006) [2023-12-27 18:59:50,731][20347] Updated weights for policy 0, policy_version 1926860 (0.0008) [2023-12-27 18:59:50,790][20347] Updated weights for policy 0, policy_version 1926870 (0.0008) [2023-12-27 18:59:51,128][20381] Updated weights for policy 1, policy_version 1925343 (0.0007) [2023-12-27 18:59:51,180][20381] Updated weights for policy 1, policy_version 1925353 (0.0007) [2023-12-27 18:59:51,229][20381] Updated weights for policy 1, policy_version 1925363 (0.0008) [2023-12-27 18:59:51,517][20347] Updated weights for policy 0, policy_version 1926880 (0.0007) [2023-12-27 18:59:51,575][20347] Updated weights for policy 0, policy_version 1926890 (0.0007) [2023-12-27 18:59:51,637][20347] Updated weights for policy 0, policy_version 1926900 (0.0007) [2023-12-27 18:59:51,957][20381] Updated weights for policy 1, policy_version 1925373 (0.0008) [2023-12-27 18:59:52,003][20381] Updated weights for policy 1, policy_version 1925383 (0.0008) [2023-12-27 18:59:52,052][20381] Updated weights for policy 1, policy_version 1925393 (0.0008) [2023-12-27 18:59:52,302][20347] Updated weights for policy 0, policy_version 1926910 (0.0008) [2023-12-27 18:59:52,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 986341376. Throughput: 0: 10237.2, 1: 10230.5. Samples: 986314448. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:52,363][20347] Updated weights for policy 0, policy_version 1926920 (0.0008) [2023-12-27 18:59:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.880')] [2023-12-27 18:59:52,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001925400_492961792.pth... [2023-12-27 18:59:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001924216_492658688.pth [2023-12-27 18:59:52,374][20292] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/milestones/checkpoint_001925400_492961792.pth [2023-12-27 18:59:52,423][20347] Updated weights for policy 0, policy_version 1926930 (0.0008) [2023-12-27 18:59:52,458][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001926936_493387776.pth... [2023-12-27 18:59:52,462][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001925720_493076480.pth [2023-12-27 18:59:52,463][20291] Saving a milestone ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/milestones/checkpoint_001926936_493387776.pth [2023-12-27 18:59:52,759][20381] Updated weights for policy 1, policy_version 1925403 (0.0007) [2023-12-27 18:59:52,816][20381] Updated weights for policy 1, policy_version 1925413 (0.0007) [2023-12-27 18:59:52,869][20381] Updated weights for policy 1, policy_version 1925423 (0.0008) [2023-12-27 18:59:53,136][20347] Updated weights for policy 0, policy_version 1926940 (0.0008) [2023-12-27 18:59:53,197][20347] Updated weights for policy 0, policy_version 1926950 (0.0007) [2023-12-27 18:59:53,253][20347] Updated weights for policy 0, policy_version 1926960 (0.0008) [2023-12-27 18:59:53,522][20381] Updated weights for policy 1, policy_version 1925433 (0.0007) [2023-12-27 18:59:53,574][20381] Updated weights for policy 1, policy_version 1925443 (0.0008) [2023-12-27 18:59:53,622][20381] Updated weights for policy 1, policy_version 1925453 (0.0007) [2023-12-27 18:59:53,671][20381] Updated weights for policy 1, policy_version 1925463 (0.0007) [2023-12-27 18:59:53,923][20347] Updated weights for policy 0, policy_version 1926970 (0.0005) [2023-12-27 18:59:53,976][20347] Updated weights for policy 0, policy_version 1926980 (0.0008) [2023-12-27 18:59:54,029][20347] Updated weights for policy 0, policy_version 1926990 (0.0008) [2023-12-27 18:59:54,071][20347] Updated weights for policy 0, policy_version 1927000 (0.0009) [2023-12-27 18:59:54,282][20381] Updated weights for policy 1, policy_version 1925473 (0.0007) [2023-12-27 18:59:54,330][20381] Updated weights for policy 1, policy_version 1925483 (0.0008) [2023-12-27 18:59:54,374][20381] Updated weights for policy 1, policy_version 1925493 (0.0007) [2023-12-27 18:59:54,778][20347] Updated weights for policy 0, policy_version 1927010 (0.0010) [2023-12-27 18:59:54,838][20347] Updated weights for policy 0, policy_version 1927020 (0.0010) [2023-12-27 18:59:54,896][20347] Updated weights for policy 0, policy_version 1927030 (0.0010) [2023-12-27 18:59:55,096][20381] Updated weights for policy 1, policy_version 1925503 (0.0008) [2023-12-27 18:59:55,150][20381] Updated weights for policy 1, policy_version 1925513 (0.0008) [2023-12-27 18:59:55,196][20381] Updated weights for policy 1, policy_version 1925523 (0.0007) [2023-12-27 18:59:55,562][20347] Updated weights for policy 0, policy_version 1927040 (0.0007) [2023-12-27 18:59:55,613][20347] Updated weights for policy 0, policy_version 1927050 (0.0010) [2023-12-27 18:59:55,665][20347] Updated weights for policy 0, policy_version 1927060 (0.0010) [2023-12-27 18:59:55,908][20381] Updated weights for policy 1, policy_version 1925533 (0.0008) [2023-12-27 18:59:55,958][20381] Updated weights for policy 1, policy_version 1925543 (0.0007) [2023-12-27 18:59:56,008][20381] Updated weights for policy 1, policy_version 1925553 (0.0008) [2023-12-27 18:59:56,316][20347] Updated weights for policy 0, policy_version 1927070 (0.0010) [2023-12-27 18:59:56,378][20347] Updated weights for policy 0, policy_version 1927080 (0.0010) [2023-12-27 18:59:56,430][20347] Updated weights for policy 0, policy_version 1927090 (0.0010) [2023-12-27 18:59:56,715][20381] Updated weights for policy 1, policy_version 1925563 (0.0007) [2023-12-27 18:59:56,767][20381] Updated weights for policy 1, policy_version 1925573 (0.0007) [2023-12-27 18:59:56,822][20381] Updated weights for policy 1, policy_version 1925583 (0.0008) [2023-12-27 18:59:57,048][20347] Updated weights for policy 0, policy_version 1927100 (0.0007) [2023-12-27 18:59:57,096][20347] Updated weights for policy 0, policy_version 1927110 (0.0005) [2023-12-27 18:59:57,147][20347] Updated weights for policy 0, policy_version 1927120 (0.0005) [2023-12-27 18:59:57,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 986447872. Throughput: 0: 10265.7, 1: 10208.5. Samples: 986435876. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 18:59:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.710')] [2023-12-27 18:59:57,547][20381] Updated weights for policy 1, policy_version 1925593 (0.0007) [2023-12-27 18:59:57,607][20381] Updated weights for policy 1, policy_version 1925603 (0.0006) [2023-12-27 18:59:57,662][20381] Updated weights for policy 1, policy_version 1925613 (0.0008) [2023-12-27 18:59:57,703][20347] Updated weights for policy 0, policy_version 1927130 (0.0005) [2023-12-27 18:59:57,707][20381] Updated weights for policy 1, policy_version 1925623 (0.0008) [2023-12-27 18:59:57,763][20347] Updated weights for policy 0, policy_version 1927140 (0.0005) [2023-12-27 18:59:57,822][20347] Updated weights for policy 0, policy_version 1927150 (0.0005) [2023-12-27 18:59:57,880][20347] Updated weights for policy 0, policy_version 1927160 (0.0007) [2023-12-27 18:59:58,426][20381] Updated weights for policy 1, policy_version 1925633 (0.0007) [2023-12-27 18:59:58,489][20381] Updated weights for policy 1, policy_version 1925643 (0.0008) [2023-12-27 18:59:58,545][20347] Updated weights for policy 0, policy_version 1927170 (0.0007) [2023-12-27 18:59:58,552][20381] Updated weights for policy 1, policy_version 1925653 (0.0007) [2023-12-27 18:59:58,610][20347] Updated weights for policy 0, policy_version 1927180 (0.0009) [2023-12-27 18:59:58,673][20347] Updated weights for policy 0, policy_version 1927190 (0.0007) [2023-12-27 18:59:59,424][20381] Updated weights for policy 1, policy_version 1925663 (0.0005) [2023-12-27 18:59:59,489][20381] Updated weights for policy 1, policy_version 1925673 (0.0006) [2023-12-27 18:59:59,526][20347] Updated weights for policy 0, policy_version 1927200 (0.0007) [2023-12-27 18:59:59,546][20381] Updated weights for policy 1, policy_version 1925683 (0.0007) [2023-12-27 18:59:59,587][20347] Updated weights for policy 0, policy_version 1927211 (0.0007) [2023-12-27 18:59:59,653][20347] Updated weights for policy 0, policy_version 1927221 (0.0009) [2023-12-27 19:00:00,188][20381] Updated weights for policy 1, policy_version 1925693 (0.0007) [2023-12-27 19:00:00,235][20381] Updated weights for policy 1, policy_version 1925703 (0.0008) [2023-12-27 19:00:00,293][20381] Updated weights for policy 1, policy_version 1925713 (0.0007) [2023-12-27 19:00:00,378][20347] Updated weights for policy 0, policy_version 1927231 (0.0006) [2023-12-27 19:00:00,429][20347] Updated weights for policy 0, policy_version 1927241 (0.0005) [2023-12-27 19:00:00,478][20347] Updated weights for policy 0, policy_version 1927251 (0.0007) [2023-12-27 19:00:01,010][20381] Updated weights for policy 1, policy_version 1925723 (0.0007) [2023-12-27 19:00:01,054][20347] Updated weights for policy 0, policy_version 1927261 (0.0007) [2023-12-27 19:00:01,073][20381] Updated weights for policy 1, policy_version 1925733 (0.0006) [2023-12-27 19:00:01,107][20347] Updated weights for policy 0, policy_version 1927271 (0.0006) [2023-12-27 19:00:01,132][20381] Updated weights for policy 1, policy_version 1925743 (0.0007) [2023-12-27 19:00:01,171][20347] Updated weights for policy 0, policy_version 1927281 (0.0007) [2023-12-27 19:00:01,857][20381] Updated weights for policy 1, policy_version 1925753 (0.0010) [2023-12-27 19:00:01,915][20347] Updated weights for policy 0, policy_version 1927291 (0.0007) [2023-12-27 19:00:01,916][20381] Updated weights for policy 1, policy_version 1925763 (0.0008) [2023-12-27 19:00:01,975][20381] Updated weights for policy 1, policy_version 1925773 (0.0007) [2023-12-27 19:00:01,977][20347] Updated weights for policy 0, policy_version 1927301 (0.0006) [2023-12-27 19:00:02,026][20381] Updated weights for policy 1, policy_version 1925783 (0.0010) [2023-12-27 19:00:02,033][20347] Updated weights for policy 0, policy_version 1927311 (0.0006) [2023-12-27 19:00:02,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20343.6, 300 sec: 20521.7). Total num frames: 986546176. Throughput: 0: 10266.9, 1: 10106.1. Samples: 986554992. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.790')] [2023-12-27 19:00:02,679][20347] Updated weights for policy 0, policy_version 1927321 (0.0006) [2023-12-27 19:00:02,721][20381] Updated weights for policy 1, policy_version 1925793 (0.0011) [2023-12-27 19:00:02,734][20347] Updated weights for policy 0, policy_version 1927331 (0.0010) [2023-12-27 19:00:02,770][20381] Updated weights for policy 1, policy_version 1925803 (0.0010) [2023-12-27 19:00:02,792][20347] Updated weights for policy 0, policy_version 1927341 (0.0007) [2023-12-27 19:00:02,819][20381] Updated weights for policy 1, policy_version 1925813 (0.0009) [2023-12-27 19:00:02,850][20347] Updated weights for policy 0, policy_version 1927351 (0.0007) [2023-12-27 19:00:03,414][20381] Updated weights for policy 1, policy_version 1925823 (0.0008) [2023-12-27 19:00:03,468][20381] Updated weights for policy 1, policy_version 1925833 (0.0008) [2023-12-27 19:00:03,528][20381] Updated weights for policy 1, policy_version 1925843 (0.0007) [2023-12-27 19:00:03,569][20347] Updated weights for policy 0, policy_version 1927361 (0.0010) [2023-12-27 19:00:03,614][20347] Updated weights for policy 0, policy_version 1927371 (0.0010) [2023-12-27 19:00:03,665][20347] Updated weights for policy 0, policy_version 1927381 (0.0010) [2023-12-27 19:00:04,165][20381] Updated weights for policy 1, policy_version 1925853 (0.0011) [2023-12-27 19:00:04,225][20381] Updated weights for policy 1, policy_version 1925863 (0.0007) [2023-12-27 19:00:04,282][20381] Updated weights for policy 1, policy_version 1925873 (0.0012) [2023-12-27 19:00:04,406][20347] Updated weights for policy 0, policy_version 1927391 (0.0009) [2023-12-27 19:00:04,461][20347] Updated weights for policy 0, policy_version 1927401 (0.0007) [2023-12-27 19:00:04,513][20347] Updated weights for policy 0, policy_version 1927411 (0.0007) [2023-12-27 19:00:04,890][20381] Updated weights for policy 1, policy_version 1925883 (0.0011) [2023-12-27 19:00:04,948][20381] Updated weights for policy 1, policy_version 1925893 (0.0010) [2023-12-27 19:00:05,003][20381] Updated weights for policy 1, policy_version 1925903 (0.0010) [2023-12-27 19:00:05,213][20347] Updated weights for policy 0, policy_version 1927421 (0.0007) [2023-12-27 19:00:05,271][20347] Updated weights for policy 0, policy_version 1927431 (0.0005) [2023-12-27 19:00:05,323][20347] Updated weights for policy 0, policy_version 1927441 (0.0006) [2023-12-27 19:00:05,644][20381] Updated weights for policy 1, policy_version 1925913 (0.0011) [2023-12-27 19:00:05,712][20381] Updated weights for policy 1, policy_version 1925923 (0.0007) [2023-12-27 19:00:05,763][20381] Updated weights for policy 1, policy_version 1925933 (0.0007) [2023-12-27 19:00:05,808][20381] Updated weights for policy 1, policy_version 1925943 (0.0008) [2023-12-27 19:00:05,988][20347] Updated weights for policy 0, policy_version 1927451 (0.0007) [2023-12-27 19:00:06,046][20347] Updated weights for policy 0, policy_version 1927461 (0.0005) [2023-12-27 19:00:06,114][20347] Updated weights for policy 0, policy_version 1927471 (0.0007) [2023-12-27 19:00:06,469][20381] Updated weights for policy 1, policy_version 1925953 (0.0010) [2023-12-27 19:00:06,524][20381] Updated weights for policy 1, policy_version 1925963 (0.0008) [2023-12-27 19:00:06,592][20381] Updated weights for policy 1, policy_version 1925973 (0.0007) [2023-12-27 19:00:06,737][20347] Updated weights for policy 0, policy_version 1927481 (0.0008) [2023-12-27 19:00:06,791][20347] Updated weights for policy 0, policy_version 1927491 (0.0008) [2023-12-27 19:00:06,850][20347] Updated weights for policy 0, policy_version 1927501 (0.0006) [2023-12-27 19:00:06,907][20347] Updated weights for policy 0, policy_version 1927511 (0.0007) [2023-12-27 19:00:07,328][20381] Updated weights for policy 1, policy_version 1925983 (0.0008) [2023-12-27 19:00:07,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20521.7). Total num frames: 986644480. Throughput: 0: 10246.4, 1: 10178.9. Samples: 986618076. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.330')] [2023-12-27 19:00:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001927512_493535232.pth... [2023-12-27 19:00:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001926328_493232128.pth [2023-12-27 19:00:07,381][20381] Updated weights for policy 1, policy_version 1925993 (0.0008) [2023-12-27 19:00:07,427][20381] Updated weights for policy 1, policy_version 1926003 (0.0007) [2023-12-27 19:00:07,449][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001926008_493117440.pth... [2023-12-27 19:00:07,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001924824_492814336.pth [2023-12-27 19:00:07,591][20347] Updated weights for policy 0, policy_version 1927521 (0.0007) [2023-12-27 19:00:07,648][20347] Updated weights for policy 0, policy_version 1927531 (0.0005) [2023-12-27 19:00:07,702][20347] Updated weights for policy 0, policy_version 1927541 (0.0007) [2023-12-27 19:00:08,146][20381] Updated weights for policy 1, policy_version 1926013 (0.0008) [2023-12-27 19:00:08,210][20381] Updated weights for policy 1, policy_version 1926023 (0.0008) [2023-12-27 19:00:08,272][20381] Updated weights for policy 1, policy_version 1926033 (0.0007) [2023-12-27 19:00:08,364][20347] Updated weights for policy 0, policy_version 1927551 (0.0008) [2023-12-27 19:00:08,412][20347] Updated weights for policy 0, policy_version 1927561 (0.0007) [2023-12-27 19:00:08,471][20347] Updated weights for policy 0, policy_version 1927571 (0.0008) [2023-12-27 19:00:08,912][20381] Updated weights for policy 1, policy_version 1926043 (0.0007) [2023-12-27 19:00:08,974][20381] Updated weights for policy 1, policy_version 1926053 (0.0008) [2023-12-27 19:00:09,041][20381] Updated weights for policy 1, policy_version 1926063 (0.0009) [2023-12-27 19:00:09,206][20347] Updated weights for policy 0, policy_version 1927581 (0.0008) [2023-12-27 19:00:09,272][20347] Updated weights for policy 0, policy_version 1927591 (0.0008) [2023-12-27 19:00:09,327][20347] Updated weights for policy 0, policy_version 1927601 (0.0007) [2023-12-27 19:00:09,690][20381] Updated weights for policy 1, policy_version 1926073 (0.0008) [2023-12-27 19:00:09,759][20381] Updated weights for policy 1, policy_version 1926083 (0.0007) [2023-12-27 19:00:09,827][20381] Updated weights for policy 1, policy_version 1926093 (0.0006) [2023-12-27 19:00:09,881][20381] Updated weights for policy 1, policy_version 1926103 (0.0008) [2023-12-27 19:00:10,131][20347] Updated weights for policy 0, policy_version 1927611 (0.0008) [2023-12-27 19:00:10,184][20347] Updated weights for policy 0, policy_version 1927621 (0.0008) [2023-12-27 19:00:10,244][20347] Updated weights for policy 0, policy_version 1927631 (0.0008) [2023-12-27 19:00:10,525][20381] Updated weights for policy 1, policy_version 1926113 (0.0008) [2023-12-27 19:00:10,581][20381] Updated weights for policy 1, policy_version 1926123 (0.0007) [2023-12-27 19:00:10,643][20381] Updated weights for policy 1, policy_version 1926133 (0.0006) [2023-12-27 19:00:10,964][20347] Updated weights for policy 0, policy_version 1927641 (0.0007) [2023-12-27 19:00:11,025][20347] Updated weights for policy 0, policy_version 1927651 (0.0006) [2023-12-27 19:00:11,089][20347] Updated weights for policy 0, policy_version 1927661 (0.0007) [2023-12-27 19:00:11,157][20347] Updated weights for policy 0, policy_version 1927671 (0.0007) [2023-12-27 19:00:11,307][20381] Updated weights for policy 1, policy_version 1926143 (0.0008) [2023-12-27 19:00:11,378][20381] Updated weights for policy 1, policy_version 1926153 (0.0009) [2023-12-27 19:00:11,434][20381] Updated weights for policy 1, policy_version 1926163 (0.0010) [2023-12-27 19:00:11,858][20347] Updated weights for policy 0, policy_version 1927681 (0.0009) [2023-12-27 19:00:11,920][20347] Updated weights for policy 0, policy_version 1927691 (0.0007) [2023-12-27 19:00:11,983][20347] Updated weights for policy 0, policy_version 1927701 (0.0007) [2023-12-27 19:00:12,150][20381] Updated weights for policy 1, policy_version 1926173 (0.0011) [2023-12-27 19:00:12,198][20381] Updated weights for policy 1, policy_version 1926183 (0.0011) [2023-12-27 19:00:12,260][20381] Updated weights for policy 1, policy_version 1926193 (0.0011) [2023-12-27 19:00:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 986750976. Throughput: 0: 10178.7, 1: 10201.9. Samples: 986739212. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.390')] [2023-12-27 19:00:12,721][20347] Updated weights for policy 0, policy_version 1927711 (0.0007) [2023-12-27 19:00:12,783][20347] Updated weights for policy 0, policy_version 1927721 (0.0007) [2023-12-27 19:00:12,844][20347] Updated weights for policy 0, policy_version 1927731 (0.0008) [2023-12-27 19:00:12,990][20381] Updated weights for policy 1, policy_version 1926203 (0.0010) [2023-12-27 19:00:13,035][20381] Updated weights for policy 1, policy_version 1926213 (0.0010) [2023-12-27 19:00:13,087][20381] Updated weights for policy 1, policy_version 1926223 (0.0011) [2023-12-27 19:00:13,477][20347] Updated weights for policy 0, policy_version 1927741 (0.0008) [2023-12-27 19:00:13,540][20347] Updated weights for policy 0, policy_version 1927751 (0.0010) [2023-12-27 19:00:13,604][20347] Updated weights for policy 0, policy_version 1927761 (0.0010) [2023-12-27 19:00:13,729][20381] Updated weights for policy 1, policy_version 1926233 (0.0007) [2023-12-27 19:00:13,790][20381] Updated weights for policy 1, policy_version 1926243 (0.0011) [2023-12-27 19:00:13,857][20381] Updated weights for policy 1, policy_version 1926253 (0.0011) [2023-12-27 19:00:13,912][20381] Updated weights for policy 1, policy_version 1926263 (0.0007) [2023-12-27 19:00:14,257][20347] Updated weights for policy 0, policy_version 1927771 (0.0009) [2023-12-27 19:00:14,319][20347] Updated weights for policy 0, policy_version 1927781 (0.0007) [2023-12-27 19:00:14,373][20347] Updated weights for policy 0, policy_version 1927791 (0.0009) [2023-12-27 19:00:14,523][20381] Updated weights for policy 1, policy_version 1926273 (0.0005) [2023-12-27 19:00:14,582][20381] Updated weights for policy 1, policy_version 1926283 (0.0009) [2023-12-27 19:00:14,642][20381] Updated weights for policy 1, policy_version 1926293 (0.0010) [2023-12-27 19:00:15,093][20347] Updated weights for policy 0, policy_version 1927802 (0.0008) [2023-12-27 19:00:15,157][20347] Updated weights for policy 0, policy_version 1927812 (0.0008) [2023-12-27 19:00:15,211][20347] Updated weights for policy 0, policy_version 1927822 (0.0008) [2023-12-27 19:00:15,273][20347] Updated weights for policy 0, policy_version 1927832 (0.0008) [2023-12-27 19:00:15,330][20381] Updated weights for policy 1, policy_version 1926303 (0.0010) [2023-12-27 19:00:15,387][20381] Updated weights for policy 1, policy_version 1926313 (0.0010) [2023-12-27 19:00:15,442][20381] Updated weights for policy 1, policy_version 1926323 (0.0010) [2023-12-27 19:00:15,897][20347] Updated weights for policy 0, policy_version 1927842 (0.0009) [2023-12-27 19:00:15,945][20347] Updated weights for policy 0, policy_version 1927852 (0.0008) [2023-12-27 19:00:15,994][20347] Updated weights for policy 0, policy_version 1927862 (0.0007) [2023-12-27 19:00:16,038][20381] Updated weights for policy 1, policy_version 1926333 (0.0009) [2023-12-27 19:00:16,096][20381] Updated weights for policy 1, policy_version 1926343 (0.0008) [2023-12-27 19:00:16,154][20381] Updated weights for policy 1, policy_version 1926353 (0.0008) [2023-12-27 19:00:16,628][20347] Updated weights for policy 0, policy_version 1927872 (0.0007) [2023-12-27 19:00:16,681][20347] Updated weights for policy 0, policy_version 1927882 (0.0007) [2023-12-27 19:00:16,729][20347] Updated weights for policy 0, policy_version 1927892 (0.0008) [2023-12-27 19:00:16,873][20381] Updated weights for policy 1, policy_version 1926363 (0.0008) [2023-12-27 19:00:16,932][20381] Updated weights for policy 1, policy_version 1926373 (0.0009) [2023-12-27 19:00:16,977][20381] Updated weights for policy 1, policy_version 1926383 (0.0010) [2023-12-27 19:00:17,310][20347] Updated weights for policy 0, policy_version 1927902 (0.0007) [2023-12-27 19:00:17,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 986849280. Throughput: 0: 10173.0, 1: 10194.7. Samples: 986862060. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:17,363][20347] Updated weights for policy 0, policy_version 1927912 (0.0007) [2023-12-27 19:00:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.580')] [2023-12-27 19:00:17,415][20347] Updated weights for policy 0, policy_version 1927922 (0.0007) [2023-12-27 19:00:17,600][20381] Updated weights for policy 1, policy_version 1926393 (0.0008) [2023-12-27 19:00:17,664][20381] Updated weights for policy 1, policy_version 1926403 (0.0008) [2023-12-27 19:00:17,719][20381] Updated weights for policy 1, policy_version 1926413 (0.0008) [2023-12-27 19:00:17,786][20381] Updated weights for policy 1, policy_version 1926423 (0.0008) [2023-12-27 19:00:18,115][20347] Updated weights for policy 0, policy_version 1927932 (0.0009) [2023-12-27 19:00:18,169][20347] Updated weights for policy 0, policy_version 1927942 (0.0010) [2023-12-27 19:00:18,217][20347] Updated weights for policy 0, policy_version 1927952 (0.0010) [2023-12-27 19:00:18,451][20381] Updated weights for policy 1, policy_version 1926433 (0.0010) [2023-12-27 19:00:18,504][20381] Updated weights for policy 1, policy_version 1926443 (0.0011) [2023-12-27 19:00:18,553][20381] Updated weights for policy 1, policy_version 1926453 (0.0010) [2023-12-27 19:00:18,912][20347] Updated weights for policy 0, policy_version 1927962 (0.0010) [2023-12-27 19:00:18,966][20347] Updated weights for policy 0, policy_version 1927972 (0.0010) [2023-12-27 19:00:19,013][20347] Updated weights for policy 0, policy_version 1927982 (0.0007) [2023-12-27 19:00:19,063][20347] Updated weights for policy 0, policy_version 1927992 (0.0005) [2023-12-27 19:00:19,222][20381] Updated weights for policy 1, policy_version 1926463 (0.0010) [2023-12-27 19:00:19,276][20381] Updated weights for policy 1, policy_version 1926473 (0.0008) [2023-12-27 19:00:19,329][20381] Updated weights for policy 1, policy_version 1926483 (0.0010) [2023-12-27 19:00:19,791][20347] Updated weights for policy 0, policy_version 1928002 (0.0006) [2023-12-27 19:00:19,845][20347] Updated weights for policy 0, policy_version 1928012 (0.0007) [2023-12-27 19:00:19,902][20347] Updated weights for policy 0, policy_version 1928022 (0.0006) [2023-12-27 19:00:20,104][20381] Updated weights for policy 1, policy_version 1926493 (0.0010) [2023-12-27 19:00:20,163][20381] Updated weights for policy 1, policy_version 1926503 (0.0011) [2023-12-27 19:00:20,221][20381] Updated weights for policy 1, policy_version 1926513 (0.0011) [2023-12-27 19:00:20,649][20347] Updated weights for policy 0, policy_version 1928032 (0.0008) [2023-12-27 19:00:20,710][20347] Updated weights for policy 0, policy_version 1928042 (0.0008) [2023-12-27 19:00:20,766][20347] Updated weights for policy 0, policy_version 1928052 (0.0008) [2023-12-27 19:00:20,953][20381] Updated weights for policy 1, policy_version 1926523 (0.0011) [2023-12-27 19:00:21,013][20381] Updated weights for policy 1, policy_version 1926533 (0.0011) [2023-12-27 19:00:21,081][20381] Updated weights for policy 1, policy_version 1926543 (0.0011) [2023-12-27 19:00:21,526][20347] Updated weights for policy 0, policy_version 1928062 (0.0008) [2023-12-27 19:00:21,586][20347] Updated weights for policy 0, policy_version 1928072 (0.0008) [2023-12-27 19:00:21,665][20347] Updated weights for policy 0, policy_version 1928082 (0.0008) [2023-12-27 19:00:21,865][20381] Updated weights for policy 1, policy_version 1926553 (0.0010) [2023-12-27 19:00:21,930][20381] Updated weights for policy 1, policy_version 1926563 (0.0011) [2023-12-27 19:00:22,000][20381] Updated weights for policy 1, policy_version 1926573 (0.0011) [2023-12-27 19:00:22,063][20381] Updated weights for policy 1, policy_version 1926583 (0.0011) [2023-12-27 19:00:22,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 986947584. Throughput: 0: 10179.4, 1: 10164.1. Samples: 986923108. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.240')] [2023-12-27 19:00:22,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001926584_493264896.pth... [2023-12-27 19:00:22,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001928088_493682688.pth... [2023-12-27 19:00:22,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001925400_492961792.pth [2023-12-27 19:00:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001926936_493387776.pth [2023-12-27 19:00:22,415][20347] Updated weights for policy 0, policy_version 1928092 (0.0007) [2023-12-27 19:00:22,477][20347] Updated weights for policy 0, policy_version 1928102 (0.0007) [2023-12-27 19:00:22,540][20347] Updated weights for policy 0, policy_version 1928112 (0.0007) [2023-12-27 19:00:22,755][20381] Updated weights for policy 1, policy_version 1926593 (0.0007) [2023-12-27 19:00:22,816][20381] Updated weights for policy 1, policy_version 1926603 (0.0005) [2023-12-27 19:00:22,874][20381] Updated weights for policy 1, policy_version 1926613 (0.0006) [2023-12-27 19:00:23,240][20347] Updated weights for policy 0, policy_version 1928122 (0.0007) [2023-12-27 19:00:23,295][20347] Updated weights for policy 0, policy_version 1928132 (0.0007) [2023-12-27 19:00:23,351][20347] Updated weights for policy 0, policy_version 1928142 (0.0005) [2023-12-27 19:00:23,408][20347] Updated weights for policy 0, policy_version 1928152 (0.0008) [2023-12-27 19:00:23,504][20381] Updated weights for policy 1, policy_version 1926623 (0.0006) [2023-12-27 19:00:23,558][20381] Updated weights for policy 1, policy_version 1926633 (0.0005) [2023-12-27 19:00:23,611][20381] Updated weights for policy 1, policy_version 1926643 (0.0005) [2023-12-27 19:00:24,029][20347] Updated weights for policy 0, policy_version 1928162 (0.0009) [2023-12-27 19:00:24,096][20347] Updated weights for policy 0, policy_version 1928172 (0.0010) [2023-12-27 19:00:24,152][20381] Updated weights for policy 1, policy_version 1926653 (0.0006) [2023-12-27 19:00:24,153][20347] Updated weights for policy 0, policy_version 1928182 (0.0007) [2023-12-27 19:00:24,211][20381] Updated weights for policy 1, policy_version 1926663 (0.0007) [2023-12-27 19:00:24,262][20381] Updated weights for policy 1, policy_version 1926673 (0.0008) [2023-12-27 19:00:24,848][20347] Updated weights for policy 0, policy_version 1928192 (0.0007) [2023-12-27 19:00:24,904][20347] Updated weights for policy 0, policy_version 1928202 (0.0009) [2023-12-27 19:00:24,923][20381] Updated weights for policy 1, policy_version 1926683 (0.0008) [2023-12-27 19:00:24,950][20347] Updated weights for policy 0, policy_version 1928212 (0.0009) [2023-12-27 19:00:24,975][20381] Updated weights for policy 1, policy_version 1926693 (0.0011) [2023-12-27 19:00:25,021][20381] Updated weights for policy 1, policy_version 1926703 (0.0006) [2023-12-27 19:00:25,634][20347] Updated weights for policy 0, policy_version 1928222 (0.0007) [2023-12-27 19:00:25,687][20347] Updated weights for policy 0, policy_version 1928232 (0.0007) [2023-12-27 19:00:25,695][20381] Updated weights for policy 1, policy_version 1926713 (0.0006) [2023-12-27 19:00:25,743][20347] Updated weights for policy 0, policy_version 1928242 (0.0006) [2023-12-27 19:00:25,752][20381] Updated weights for policy 1, policy_version 1926723 (0.0010) [2023-12-27 19:00:25,801][20381] Updated weights for policy 1, policy_version 1926733 (0.0010) [2023-12-27 19:00:25,846][20381] Updated weights for policy 1, policy_version 1926743 (0.0010) [2023-12-27 19:00:26,392][20347] Updated weights for policy 0, policy_version 1928252 (0.0007) [2023-12-27 19:00:26,447][20347] Updated weights for policy 0, policy_version 1928262 (0.0008) [2023-12-27 19:00:26,498][20347] Updated weights for policy 0, policy_version 1928272 (0.0007) [2023-12-27 19:00:26,506][20381] Updated weights for policy 1, policy_version 1926753 (0.0006) [2023-12-27 19:00:26,550][20381] Updated weights for policy 1, policy_version 1926763 (0.0005) [2023-12-27 19:00:26,596][20381] Updated weights for policy 1, policy_version 1926773 (0.0009) [2023-12-27 19:00:27,058][20347] Updated weights for policy 0, policy_version 1928282 (0.0007) [2023-12-27 19:00:27,102][20347] Updated weights for policy 0, policy_version 1928292 (0.0010) [2023-12-27 19:00:27,153][20347] Updated weights for policy 0, policy_version 1928302 (0.0010) [2023-12-27 19:00:27,203][20347] Updated weights for policy 0, policy_version 1928312 (0.0010) [2023-12-27 19:00:27,321][20381] Updated weights for policy 1, policy_version 1926783 (0.0007) [2023-12-27 19:00:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 987054080. Throughput: 0: 10122.8, 1: 10182.9. Samples: 987043568. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.350')] [2023-12-27 19:00:27,376][20381] Updated weights for policy 1, policy_version 1926793 (0.0008) [2023-12-27 19:00:27,419][20381] Updated weights for policy 1, policy_version 1926803 (0.0007) [2023-12-27 19:00:27,831][20347] Updated weights for policy 0, policy_version 1928322 (0.0005) [2023-12-27 19:00:27,886][20347] Updated weights for policy 0, policy_version 1928332 (0.0005) [2023-12-27 19:00:27,943][20347] Updated weights for policy 0, policy_version 1928342 (0.0006) [2023-12-27 19:00:28,113][20381] Updated weights for policy 1, policy_version 1926813 (0.0008) [2023-12-27 19:00:28,159][20381] Updated weights for policy 1, policy_version 1926823 (0.0008) [2023-12-27 19:00:28,219][20381] Updated weights for policy 1, policy_version 1926833 (0.0007) [2023-12-27 19:00:28,555][20347] Updated weights for policy 0, policy_version 1928352 (0.0008) [2023-12-27 19:00:28,608][20347] Updated weights for policy 0, policy_version 1928362 (0.0008) [2023-12-27 19:00:28,657][20347] Updated weights for policy 0, policy_version 1928372 (0.0007) [2023-12-27 19:00:28,877][20381] Updated weights for policy 1, policy_version 1926843 (0.0007) [2023-12-27 19:00:28,931][20381] Updated weights for policy 1, policy_version 1926853 (0.0008) [2023-12-27 19:00:28,986][20381] Updated weights for policy 1, policy_version 1926863 (0.0008) [2023-12-27 19:00:29,383][20347] Updated weights for policy 0, policy_version 1928382 (0.0008) [2023-12-27 19:00:29,438][20347] Updated weights for policy 0, policy_version 1928392 (0.0008) [2023-12-27 19:00:29,493][20347] Updated weights for policy 0, policy_version 1928402 (0.0009) [2023-12-27 19:00:29,648][20381] Updated weights for policy 1, policy_version 1926873 (0.0007) [2023-12-27 19:00:29,705][20381] Updated weights for policy 1, policy_version 1926883 (0.0006) [2023-12-27 19:00:29,751][20381] Updated weights for policy 1, policy_version 1926893 (0.0005) [2023-12-27 19:00:29,797][20381] Updated weights for policy 1, policy_version 1926903 (0.0005) [2023-12-27 19:00:30,149][20347] Updated weights for policy 0, policy_version 1928412 (0.0007) [2023-12-27 19:00:30,206][20347] Updated weights for policy 0, policy_version 1928422 (0.0008) [2023-12-27 19:00:30,261][20347] Updated weights for policy 0, policy_version 1928432 (0.0007) [2023-12-27 19:00:30,437][20381] Updated weights for policy 1, policy_version 1926913 (0.0007) [2023-12-27 19:00:30,491][20381] Updated weights for policy 1, policy_version 1926923 (0.0008) [2023-12-27 19:00:30,544][20381] Updated weights for policy 1, policy_version 1926933 (0.0008) [2023-12-27 19:00:30,916][20347] Updated weights for policy 0, policy_version 1928442 (0.0007) [2023-12-27 19:00:30,973][20347] Updated weights for policy 0, policy_version 1928452 (0.0008) [2023-12-27 19:00:31,030][20347] Updated weights for policy 0, policy_version 1928462 (0.0007) [2023-12-27 19:00:31,088][20347] Updated weights for policy 0, policy_version 1928472 (0.0009) [2023-12-27 19:00:31,225][20381] Updated weights for policy 1, policy_version 1926943 (0.0007) [2023-12-27 19:00:31,289][20381] Updated weights for policy 1, policy_version 1926953 (0.0007) [2023-12-27 19:00:31,347][20381] Updated weights for policy 1, policy_version 1926963 (0.0007) [2023-12-27 19:00:31,811][20347] Updated weights for policy 0, policy_version 1928482 (0.0009) [2023-12-27 19:00:31,865][20347] Updated weights for policy 0, policy_version 1928492 (0.0008) [2023-12-27 19:00:31,924][20347] Updated weights for policy 0, policy_version 1928502 (0.0008) [2023-12-27 19:00:31,993][20381] Updated weights for policy 1, policy_version 1926973 (0.0006) [2023-12-27 19:00:32,056][20381] Updated weights for policy 1, policy_version 1926983 (0.0006) [2023-12-27 19:00:32,115][20381] Updated weights for policy 1, policy_version 1926993 (0.0006) [2023-12-27 19:00:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 987160576. Throughput: 0: 10171.1, 1: 10237.5. Samples: 987170020. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.660')] [2023-12-27 19:00:32,714][20347] Updated weights for policy 0, policy_version 1928512 (0.0007) [2023-12-27 19:00:32,743][20381] Updated weights for policy 1, policy_version 1927003 (0.0006) [2023-12-27 19:00:32,774][20347] Updated weights for policy 0, policy_version 1928522 (0.0009) [2023-12-27 19:00:32,805][20381] Updated weights for policy 1, policy_version 1927013 (0.0007) [2023-12-27 19:00:32,829][20347] Updated weights for policy 0, policy_version 1928532 (0.0010) [2023-12-27 19:00:32,862][20381] Updated weights for policy 1, policy_version 1927023 (0.0007) [2023-12-27 19:00:33,450][20381] Updated weights for policy 1, policy_version 1927033 (0.0006) [2023-12-27 19:00:33,509][20381] Updated weights for policy 1, policy_version 1927043 (0.0005) [2023-12-27 19:00:33,537][20347] Updated weights for policy 0, policy_version 1928542 (0.0007) [2023-12-27 19:00:33,567][20381] Updated weights for policy 1, policy_version 1927053 (0.0005) [2023-12-27 19:00:33,585][20347] Updated weights for policy 0, policy_version 1928552 (0.0007) [2023-12-27 19:00:33,627][20381] Updated weights for policy 1, policy_version 1927063 (0.0005) [2023-12-27 19:00:33,636][20347] Updated weights for policy 0, policy_version 1928562 (0.0007) [2023-12-27 19:00:34,207][20381] Updated weights for policy 1, policy_version 1927073 (0.0008) [2023-12-27 19:00:34,273][20381] Updated weights for policy 1, policy_version 1927083 (0.0008) [2023-12-27 19:00:34,320][20347] Updated weights for policy 0, policy_version 1928572 (0.0007) [2023-12-27 19:00:34,335][20381] Updated weights for policy 1, policy_version 1927093 (0.0008) [2023-12-27 19:00:34,375][20347] Updated weights for policy 0, policy_version 1928582 (0.0007) [2023-12-27 19:00:34,429][20347] Updated weights for policy 0, policy_version 1928592 (0.0007) [2023-12-27 19:00:34,997][20381] Updated weights for policy 1, policy_version 1927103 (0.0009) [2023-12-27 19:00:35,057][20381] Updated weights for policy 1, policy_version 1927113 (0.0008) [2023-12-27 19:00:35,115][20381] Updated weights for policy 1, policy_version 1927123 (0.0007) [2023-12-27 19:00:35,145][20347] Updated weights for policy 0, policy_version 1928602 (0.0007) [2023-12-27 19:00:35,199][20347] Updated weights for policy 0, policy_version 1928612 (0.0007) [2023-12-27 19:00:35,258][20347] Updated weights for policy 0, policy_version 1928622 (0.0007) [2023-12-27 19:00:35,312][20347] Updated weights for policy 0, policy_version 1928632 (0.0009) [2023-12-27 19:00:35,680][20381] Updated weights for policy 1, policy_version 1927133 (0.0009) [2023-12-27 19:00:35,726][20381] Updated weights for policy 1, policy_version 1927143 (0.0007) [2023-12-27 19:00:35,772][20381] Updated weights for policy 1, policy_version 1927153 (0.0005) [2023-12-27 19:00:36,054][20347] Updated weights for policy 0, policy_version 1928642 (0.0007) [2023-12-27 19:00:36,103][20347] Updated weights for policy 0, policy_version 1928652 (0.0008) [2023-12-27 19:00:36,160][20347] Updated weights for policy 0, policy_version 1928662 (0.0007) [2023-12-27 19:00:36,430][20381] Updated weights for policy 1, policy_version 1927163 (0.0006) [2023-12-27 19:00:36,489][20381] Updated weights for policy 1, policy_version 1927173 (0.0010) [2023-12-27 19:00:36,548][20381] Updated weights for policy 1, policy_version 1927183 (0.0010) [2023-12-27 19:00:36,896][20347] Updated weights for policy 0, policy_version 1928672 (0.0007) [2023-12-27 19:00:36,964][20347] Updated weights for policy 0, policy_version 1928682 (0.0008) [2023-12-27 19:00:37,014][20347] Updated weights for policy 0, policy_version 1928692 (0.0009) [2023-12-27 19:00:37,158][20381] Updated weights for policy 1, policy_version 1927193 (0.0010) [2023-12-27 19:00:37,223][20381] Updated weights for policy 1, policy_version 1927203 (0.0008) [2023-12-27 19:00:37,285][20381] Updated weights for policy 1, policy_version 1927213 (0.0008) [2023-12-27 19:00:37,343][20381] Updated weights for policy 1, policy_version 1927223 (0.0005) [2023-12-27 19:00:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 987267072. Throughput: 0: 10133.8, 1: 10259.5. Samples: 987232144. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.580')] [2023-12-27 19:00:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001927224_493428736.pth... [2023-12-27 19:00:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001928696_493838336.pth... [2023-12-27 19:00:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001926008_493117440.pth [2023-12-27 19:00:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001927512_493535232.pth [2023-12-27 19:00:37,737][20347] Updated weights for policy 0, policy_version 1928702 (0.0006) [2023-12-27 19:00:37,791][20347] Updated weights for policy 0, policy_version 1928712 (0.0005) [2023-12-27 19:00:37,852][20347] Updated weights for policy 0, policy_version 1928722 (0.0006) [2023-12-27 19:00:37,923][20381] Updated weights for policy 1, policy_version 1927233 (0.0005) [2023-12-27 19:00:37,986][20381] Updated weights for policy 1, policy_version 1927243 (0.0005) [2023-12-27 19:00:38,047][20381] Updated weights for policy 1, policy_version 1927253 (0.0006) [2023-12-27 19:00:38,440][20347] Updated weights for policy 0, policy_version 1928732 (0.0007) [2023-12-27 19:00:38,500][20347] Updated weights for policy 0, policy_version 1928742 (0.0006) [2023-12-27 19:00:38,560][20347] Updated weights for policy 0, policy_version 1928752 (0.0007) [2023-12-27 19:00:38,609][20381] Updated weights for policy 1, policy_version 1927263 (0.0009) [2023-12-27 19:00:38,662][20381] Updated weights for policy 1, policy_version 1927273 (0.0010) [2023-12-27 19:00:38,721][20381] Updated weights for policy 1, policy_version 1927283 (0.0010) [2023-12-27 19:00:39,235][20347] Updated weights for policy 0, policy_version 1928762 (0.0007) [2023-12-27 19:00:39,289][20347] Updated weights for policy 0, policy_version 1928772 (0.0008) [2023-12-27 19:00:39,355][20347] Updated weights for policy 0, policy_version 1928782 (0.0008) [2023-12-27 19:00:39,421][20347] Updated weights for policy 0, policy_version 1928792 (0.0008) [2023-12-27 19:00:39,460][20381] Updated weights for policy 1, policy_version 1927293 (0.0010) [2023-12-27 19:00:39,517][20381] Updated weights for policy 1, policy_version 1927303 (0.0010) [2023-12-27 19:00:39,570][20381] Updated weights for policy 1, policy_version 1927313 (0.0010) [2023-12-27 19:00:40,196][20347] Updated weights for policy 0, policy_version 1928802 (0.0008) [2023-12-27 19:00:40,261][20347] Updated weights for policy 0, policy_version 1928812 (0.0008) [2023-12-27 19:00:40,316][20381] Updated weights for policy 1, policy_version 1927323 (0.0010) [2023-12-27 19:00:40,319][20347] Updated weights for policy 0, policy_version 1928822 (0.0008) [2023-12-27 19:00:40,379][20381] Updated weights for policy 1, policy_version 1927333 (0.0010) [2023-12-27 19:00:40,446][20381] Updated weights for policy 1, policy_version 1927343 (0.0007) [2023-12-27 19:00:41,113][20347] Updated weights for policy 0, policy_version 1928832 (0.0008) [2023-12-27 19:00:41,137][20381] Updated weights for policy 1, policy_version 1927353 (0.0009) [2023-12-27 19:00:41,184][20347] Updated weights for policy 0, policy_version 1928842 (0.0008) [2023-12-27 19:00:41,206][20381] Updated weights for policy 1, policy_version 1927363 (0.0011) [2023-12-27 19:00:41,247][20347] Updated weights for policy 0, policy_version 1928852 (0.0008) [2023-12-27 19:00:41,276][20381] Updated weights for policy 1, policy_version 1927373 (0.0012) [2023-12-27 19:00:41,337][20381] Updated weights for policy 1, policy_version 1927383 (0.0010) [2023-12-27 19:00:42,055][20347] Updated weights for policy 0, policy_version 1928862 (0.0008) [2023-12-27 19:00:42,102][20381] Updated weights for policy 1, policy_version 1927393 (0.0007) [2023-12-27 19:00:42,115][20347] Updated weights for policy 0, policy_version 1928872 (0.0006) [2023-12-27 19:00:42,165][20381] Updated weights for policy 1, policy_version 1927403 (0.0006) [2023-12-27 19:00:42,176][20347] Updated weights for policy 0, policy_version 1928882 (0.0007) [2023-12-27 19:00:42,227][20381] Updated weights for policy 1, policy_version 1927413 (0.0007) [2023-12-27 19:00:42,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 987365376. Throughput: 0: 10059.0, 1: 10321.0. Samples: 987352976. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.160')] [2023-12-27 19:00:42,896][20347] Updated weights for policy 0, policy_version 1928892 (0.0006) [2023-12-27 19:00:42,950][20347] Updated weights for policy 0, policy_version 1928902 (0.0007) [2023-12-27 19:00:42,971][20381] Updated weights for policy 1, policy_version 1927423 (0.0009) [2023-12-27 19:00:43,000][20347] Updated weights for policy 0, policy_version 1928912 (0.0005) [2023-12-27 19:00:43,026][20381] Updated weights for policy 1, policy_version 1927433 (0.0010) [2023-12-27 19:00:43,075][20381] Updated weights for policy 1, policy_version 1927443 (0.0010) [2023-12-27 19:00:43,716][20347] Updated weights for policy 0, policy_version 1928922 (0.0007) [2023-12-27 19:00:43,760][20381] Updated weights for policy 1, policy_version 1927453 (0.0008) [2023-12-27 19:00:43,774][20347] Updated weights for policy 0, policy_version 1928932 (0.0006) [2023-12-27 19:00:43,820][20381] Updated weights for policy 1, policy_version 1927463 (0.0007) [2023-12-27 19:00:43,833][20347] Updated weights for policy 0, policy_version 1928942 (0.0006) [2023-12-27 19:00:43,883][20381] Updated weights for policy 1, policy_version 1927473 (0.0005) [2023-12-27 19:00:43,892][20347] Updated weights for policy 0, policy_version 1928952 (0.0006) [2023-12-27 19:00:44,575][20347] Updated weights for policy 0, policy_version 1928962 (0.0008) [2023-12-27 19:00:44,584][20381] Updated weights for policy 1, policy_version 1927483 (0.0007) [2023-12-27 19:00:44,629][20347] Updated weights for policy 0, policy_version 1928972 (0.0008) [2023-12-27 19:00:44,642][20381] Updated weights for policy 1, policy_version 1927493 (0.0007) [2023-12-27 19:00:44,688][20347] Updated weights for policy 0, policy_version 1928982 (0.0008) [2023-12-27 19:00:44,704][20381] Updated weights for policy 1, policy_version 1927503 (0.0006) [2023-12-27 19:00:45,368][20347] Updated weights for policy 0, policy_version 1928992 (0.0006) [2023-12-27 19:00:45,407][20381] Updated weights for policy 1, policy_version 1927513 (0.0008) [2023-12-27 19:00:45,425][20347] Updated weights for policy 0, policy_version 1929002 (0.0007) [2023-12-27 19:00:45,469][20381] Updated weights for policy 1, policy_version 1927523 (0.0006) [2023-12-27 19:00:45,485][20347] Updated weights for policy 0, policy_version 1929012 (0.0006) [2023-12-27 19:00:45,523][20381] Updated weights for policy 1, policy_version 1927533 (0.0006) [2023-12-27 19:00:45,582][20381] Updated weights for policy 1, policy_version 1927543 (0.0008) [2023-12-27 19:00:46,122][20347] Updated weights for policy 0, policy_version 1929022 (0.0007) [2023-12-27 19:00:46,181][20347] Updated weights for policy 0, policy_version 1929032 (0.0007) [2023-12-27 19:00:46,239][20347] Updated weights for policy 0, policy_version 1929042 (0.0007) [2023-12-27 19:00:46,297][20381] Updated weights for policy 1, policy_version 1927553 (0.0009) [2023-12-27 19:00:46,344][20381] Updated weights for policy 1, policy_version 1927563 (0.0008) [2023-12-27 19:00:46,398][20381] Updated weights for policy 1, policy_version 1927573 (0.0008) [2023-12-27 19:00:46,928][20347] Updated weights for policy 0, policy_version 1929052 (0.0009) [2023-12-27 19:00:46,988][20347] Updated weights for policy 0, policy_version 1929062 (0.0008) [2023-12-27 19:00:47,046][20347] Updated weights for policy 0, policy_version 1929072 (0.0008) [2023-12-27 19:00:47,137][20381] Updated weights for policy 1, policy_version 1927583 (0.0008) [2023-12-27 19:00:47,192][20381] Updated weights for policy 1, policy_version 1927593 (0.0008) [2023-12-27 19:00:47,240][20381] Updated weights for policy 1, policy_version 1927603 (0.0008) [2023-12-27 19:00:47,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20493.9). Total num frames: 987463680. Throughput: 0: 9997.9, 1: 10315.1. Samples: 987469076. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.760')] [2023-12-27 19:00:47,742][20347] Updated weights for policy 0, policy_version 1929082 (0.0008) [2023-12-27 19:00:47,809][20347] Updated weights for policy 0, policy_version 1929092 (0.0008) [2023-12-27 19:00:47,871][20347] Updated weights for policy 0, policy_version 1929102 (0.0008) [2023-12-27 19:00:47,919][20381] Updated weights for policy 1, policy_version 1927613 (0.0008) [2023-12-27 19:00:47,924][20347] Updated weights for policy 0, policy_version 1929112 (0.0006) [2023-12-27 19:00:47,968][20381] Updated weights for policy 1, policy_version 1927623 (0.0007) [2023-12-27 19:00:48,014][20381] Updated weights for policy 1, policy_version 1927633 (0.0007) [2023-12-27 19:00:48,630][20347] Updated weights for policy 0, policy_version 1929122 (0.0008) [2023-12-27 19:00:48,694][20347] Updated weights for policy 0, policy_version 1929132 (0.0007) [2023-12-27 19:00:48,753][20347] Updated weights for policy 0, policy_version 1929142 (0.0008) [2023-12-27 19:00:48,798][20381] Updated weights for policy 1, policy_version 1927643 (0.0008) [2023-12-27 19:00:48,858][20381] Updated weights for policy 1, policy_version 1927653 (0.0009) [2023-12-27 19:00:48,920][20381] Updated weights for policy 1, policy_version 1927663 (0.0007) [2023-12-27 19:00:49,444][20347] Updated weights for policy 0, policy_version 1929152 (0.0007) [2023-12-27 19:00:49,495][20347] Updated weights for policy 0, policy_version 1929162 (0.0007) [2023-12-27 19:00:49,551][20347] Updated weights for policy 0, policy_version 1929172 (0.0009) [2023-12-27 19:00:49,694][20381] Updated weights for policy 1, policy_version 1927673 (0.0008) [2023-12-27 19:00:49,755][20381] Updated weights for policy 1, policy_version 1927683 (0.0008) [2023-12-27 19:00:49,807][20381] Updated weights for policy 1, policy_version 1927693 (0.0008) [2023-12-27 19:00:49,875][20381] Updated weights for policy 1, policy_version 1927703 (0.0008) [2023-12-27 19:00:50,264][20347] Updated weights for policy 0, policy_version 1929182 (0.0009) [2023-12-27 19:00:50,323][20347] Updated weights for policy 0, policy_version 1929192 (0.0006) [2023-12-27 19:00:50,384][20347] Updated weights for policy 0, policy_version 1929202 (0.0006) [2023-12-27 19:00:50,586][20381] Updated weights for policy 1, policy_version 1927713 (0.0007) [2023-12-27 19:00:50,648][20381] Updated weights for policy 1, policy_version 1927723 (0.0008) [2023-12-27 19:00:50,711][20381] Updated weights for policy 1, policy_version 1927733 (0.0007) [2023-12-27 19:00:51,161][20347] Updated weights for policy 0, policy_version 1929212 (0.0008) [2023-12-27 19:00:51,227][20347] Updated weights for policy 0, policy_version 1929222 (0.0008) [2023-12-27 19:00:51,288][20347] Updated weights for policy 0, policy_version 1929232 (0.0008) [2023-12-27 19:00:51,438][20381] Updated weights for policy 1, policy_version 1927743 (0.0007) [2023-12-27 19:00:51,500][20381] Updated weights for policy 1, policy_version 1927753 (0.0008) [2023-12-27 19:00:51,564][20381] Updated weights for policy 1, policy_version 1927763 (0.0007) [2023-12-27 19:00:52,055][20347] Updated weights for policy 0, policy_version 1929242 (0.0007) [2023-12-27 19:00:52,111][20347] Updated weights for policy 0, policy_version 1929252 (0.0008) [2023-12-27 19:00:52,166][20347] Updated weights for policy 0, policy_version 1929262 (0.0008) [2023-12-27 19:00:52,225][20347] Updated weights for policy 0, policy_version 1929272 (0.0006) [2023-12-27 19:00:52,304][20381] Updated weights for policy 1, policy_version 1927773 (0.0008) [2023-12-27 19:00:52,362][19170] Fps is (10 sec: 18841.5, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 987553792. Throughput: 0: 9999.7, 1: 10205.1. Samples: 987527296. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.540')] [2023-12-27 19:00:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001929272_493985792.pth... [2023-12-27 19:00:52,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001928088_493682688.pth [2023-12-27 19:00:52,376][20381] Updated weights for policy 1, policy_version 1927783 (0.0010) [2023-12-27 19:00:52,438][20381] Updated weights for policy 1, policy_version 1927793 (0.0008) [2023-12-27 19:00:52,479][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001927800_493576192.pth... [2023-12-27 19:00:52,484][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001926584_493264896.pth [2023-12-27 19:00:52,951][20347] Updated weights for policy 0, policy_version 1929282 (0.0007) [2023-12-27 19:00:53,008][20347] Updated weights for policy 0, policy_version 1929292 (0.0007) [2023-12-27 19:00:53,066][20347] Updated weights for policy 0, policy_version 1929302 (0.0007) [2023-12-27 19:00:53,147][20381] Updated weights for policy 1, policy_version 1927803 (0.0009) [2023-12-27 19:00:53,208][20381] Updated weights for policy 1, policy_version 1927813 (0.0011) [2023-12-27 19:00:53,267][20381] Updated weights for policy 1, policy_version 1927823 (0.0010) [2023-12-27 19:00:53,774][20347] Updated weights for policy 0, policy_version 1929312 (0.0008) [2023-12-27 19:00:53,822][20347] Updated weights for policy 0, policy_version 1929322 (0.0005) [2023-12-27 19:00:53,871][20381] Updated weights for policy 1, policy_version 1927833 (0.0011) [2023-12-27 19:00:53,872][20347] Updated weights for policy 0, policy_version 1929332 (0.0005) [2023-12-27 19:00:53,923][20381] Updated weights for policy 1, policy_version 1927843 (0.0008) [2023-12-27 19:00:53,975][20381] Updated weights for policy 1, policy_version 1927853 (0.0006) [2023-12-27 19:00:54,032][20381] Updated weights for policy 1, policy_version 1927863 (0.0006) [2023-12-27 19:00:54,553][20347] Updated weights for policy 0, policy_version 1929342 (0.0006) [2023-12-27 19:00:54,602][20347] Updated weights for policy 0, policy_version 1929352 (0.0008) [2023-12-27 19:00:54,651][20347] Updated weights for policy 0, policy_version 1929362 (0.0006) [2023-12-27 19:00:54,663][20381] Updated weights for policy 1, policy_version 1927873 (0.0009) [2023-12-27 19:00:54,717][20381] Updated weights for policy 1, policy_version 1927883 (0.0007) [2023-12-27 19:00:54,773][20381] Updated weights for policy 1, policy_version 1927893 (0.0007) [2023-12-27 19:00:55,332][20347] Updated weights for policy 0, policy_version 1929372 (0.0006) [2023-12-27 19:00:55,392][20347] Updated weights for policy 0, policy_version 1929382 (0.0006) [2023-12-27 19:00:55,451][20347] Updated weights for policy 0, policy_version 1929392 (0.0007) [2023-12-27 19:00:55,495][20381] Updated weights for policy 1, policy_version 1927903 (0.0010) [2023-12-27 19:00:55,554][20381] Updated weights for policy 1, policy_version 1927913 (0.0006) [2023-12-27 19:00:55,603][20381] Updated weights for policy 1, policy_version 1927923 (0.0006) [2023-12-27 19:00:56,107][20347] Updated weights for policy 0, policy_version 1929402 (0.0006) [2023-12-27 19:00:56,161][20347] Updated weights for policy 0, policy_version 1929412 (0.0007) [2023-12-27 19:00:56,200][20381] Updated weights for policy 1, policy_version 1927933 (0.0006) [2023-12-27 19:00:56,216][20347] Updated weights for policy 0, policy_version 1929422 (0.0007) [2023-12-27 19:00:56,249][20381] Updated weights for policy 1, policy_version 1927943 (0.0009) [2023-12-27 19:00:56,262][20347] Updated weights for policy 0, policy_version 1929432 (0.0005) [2023-12-27 19:00:56,299][20381] Updated weights for policy 1, policy_version 1927953 (0.0008) [2023-12-27 19:00:56,938][20381] Updated weights for policy 1, policy_version 1927963 (0.0007) [2023-12-27 19:00:56,984][20347] Updated weights for policy 0, policy_version 1929442 (0.0005) [2023-12-27 19:00:56,986][20381] Updated weights for policy 1, policy_version 1927973 (0.0010) [2023-12-27 19:00:57,044][20381] Updated weights for policy 1, policy_version 1927983 (0.0010) [2023-12-27 19:00:57,046][20347] Updated weights for policy 0, policy_version 1929452 (0.0006) [2023-12-27 19:00:57,098][20347] Updated weights for policy 0, policy_version 1929462 (0.0006) [2023-12-27 19:00:57,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 987660288. Throughput: 0: 9967.6, 1: 10231.7. Samples: 987648184. Policy #0 lag: (min: 10.0, avg: 22.1, max: 42.0) [2023-12-27 19:00:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.570')] [2023-12-27 19:00:57,706][20381] Updated weights for policy 1, policy_version 1927993 (0.0010) [2023-12-27 19:00:57,759][20381] Updated weights for policy 1, policy_version 1928003 (0.0010) [2023-12-27 19:00:57,807][20347] Updated weights for policy 0, policy_version 1929472 (0.0007) [2023-12-27 19:00:57,814][20381] Updated weights for policy 1, policy_version 1928013 (0.0006) [2023-12-27 19:00:57,862][20347] Updated weights for policy 0, policy_version 1929482 (0.0007) [2023-12-27 19:00:57,862][20381] Updated weights for policy 1, policy_version 1928023 (0.0005) [2023-12-27 19:00:57,918][20347] Updated weights for policy 0, policy_version 1929492 (0.0009) [2023-12-27 19:00:58,531][20381] Updated weights for policy 1, policy_version 1928033 (0.0008) [2023-12-27 19:00:58,588][20381] Updated weights for policy 1, policy_version 1928043 (0.0008) [2023-12-27 19:00:58,626][20347] Updated weights for policy 0, policy_version 1929502 (0.0007) [2023-12-27 19:00:58,654][20381] Updated weights for policy 1, policy_version 1928054 (0.0008) [2023-12-27 19:00:58,689][20347] Updated weights for policy 0, policy_version 1929512 (0.0008) [2023-12-27 19:00:58,752][20347] Updated weights for policy 0, policy_version 1929522 (0.0009) [2023-12-27 19:00:59,449][20381] Updated weights for policy 1, policy_version 1928064 (0.0007) [2023-12-27 19:00:59,515][20381] Updated weights for policy 1, policy_version 1928074 (0.0008) [2023-12-27 19:00:59,527][20347] Updated weights for policy 0, policy_version 1929532 (0.0008) [2023-12-27 19:00:59,572][20381] Updated weights for policy 1, policy_version 1928084 (0.0007) [2023-12-27 19:00:59,581][20347] Updated weights for policy 0, policy_version 1929542 (0.0006) [2023-12-27 19:00:59,639][20347] Updated weights for policy 0, policy_version 1929552 (0.0007) [2023-12-27 19:01:00,255][20381] Updated weights for policy 1, policy_version 1928094 (0.0007) [2023-12-27 19:01:00,306][20347] Updated weights for policy 0, policy_version 1929562 (0.0008) [2023-12-27 19:01:00,312][20381] Updated weights for policy 1, policy_version 1928104 (0.0006) [2023-12-27 19:01:00,359][20347] Updated weights for policy 0, policy_version 1929572 (0.0006) [2023-12-27 19:01:00,367][20381] Updated weights for policy 1, policy_version 1928114 (0.0006) [2023-12-27 19:01:00,413][20347] Updated weights for policy 0, policy_version 1929582 (0.0007) [2023-12-27 19:01:00,461][20347] Updated weights for policy 0, policy_version 1929592 (0.0008) [2023-12-27 19:01:01,049][20381] Updated weights for policy 1, policy_version 1928124 (0.0007) [2023-12-27 19:01:01,107][20347] Updated weights for policy 0, policy_version 1929602 (0.0008) [2023-12-27 19:01:01,108][20381] Updated weights for policy 1, policy_version 1928134 (0.0008) [2023-12-27 19:01:01,164][20347] Updated weights for policy 0, policy_version 1929612 (0.0008) [2023-12-27 19:01:01,170][20381] Updated weights for policy 1, policy_version 1928144 (0.0007) [2023-12-27 19:01:01,223][20347] Updated weights for policy 0, policy_version 1929622 (0.0007) [2023-12-27 19:01:01,928][20347] Updated weights for policy 0, policy_version 1929632 (0.0007) [2023-12-27 19:01:01,930][20381] Updated weights for policy 1, policy_version 1928154 (0.0009) [2023-12-27 19:01:01,984][20347] Updated weights for policy 0, policy_version 1929642 (0.0006) [2023-12-27 19:01:01,990][20381] Updated weights for policy 1, policy_version 1928164 (0.0008) [2023-12-27 19:01:02,040][20347] Updated weights for policy 0, policy_version 1929652 (0.0006) [2023-12-27 19:01:02,050][20381] Updated weights for policy 1, policy_version 1928174 (0.0007) [2023-12-27 19:01:02,110][20381] Updated weights for policy 1, policy_version 1928184 (0.0008) [2023-12-27 19:01:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20206.9, 300 sec: 20438.4). Total num frames: 987758592. Throughput: 0: 9920.7, 1: 10190.7. Samples: 987767072. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.570')] [2023-12-27 19:01:02,727][20347] Updated weights for policy 0, policy_version 1929662 (0.0009) [2023-12-27 19:01:02,783][20347] Updated weights for policy 0, policy_version 1929672 (0.0010) [2023-12-27 19:01:02,833][20347] Updated weights for policy 0, policy_version 1929682 (0.0010) [2023-12-27 19:01:02,834][20381] Updated weights for policy 1, policy_version 1928194 (0.0006) [2023-12-27 19:01:02,889][20381] Updated weights for policy 1, policy_version 1928204 (0.0006) [2023-12-27 19:01:02,945][20381] Updated weights for policy 1, policy_version 1928214 (0.0005) [2023-12-27 19:01:02,955][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000006 [2023-12-27 19:01:03,515][20381] Updated weights for policy 1, policy_version 1928224 (0.0007) [2023-12-27 19:01:03,549][20347] Updated weights for policy 0, policy_version 1929692 (0.0008) [2023-12-27 19:01:03,562][20381] Updated weights for policy 1, policy_version 1928234 (0.0009) [2023-12-27 19:01:03,599][20347] Updated weights for policy 0, policy_version 1929702 (0.0005) [2023-12-27 19:01:03,615][20381] Updated weights for policy 1, policy_version 1928244 (0.0008) [2023-12-27 19:01:03,652][20347] Updated weights for policy 0, policy_version 1929712 (0.0005) [2023-12-27 19:01:04,273][20381] Updated weights for policy 1, policy_version 1928254 (0.0007) [2023-12-27 19:01:04,278][20347] Updated weights for policy 0, policy_version 1929722 (0.0006) [2023-12-27 19:01:04,330][20381] Updated weights for policy 1, policy_version 1928264 (0.0007) [2023-12-27 19:01:04,333][20347] Updated weights for policy 0, policy_version 1929732 (0.0008) [2023-12-27 19:01:04,384][20381] Updated weights for policy 1, policy_version 1928274 (0.0007) [2023-12-27 19:01:04,393][20347] Updated weights for policy 0, policy_version 1929742 (0.0008) [2023-12-27 19:01:04,449][20347] Updated weights for policy 0, policy_version 1929752 (0.0008) [2023-12-27 19:01:05,026][20381] Updated weights for policy 1, policy_version 1928284 (0.0005) [2023-12-27 19:01:05,077][20381] Updated weights for policy 1, policy_version 1928294 (0.0010) [2023-12-27 19:01:05,139][20381] Updated weights for policy 1, policy_version 1928304 (0.0005) [2023-12-27 19:01:05,143][20347] Updated weights for policy 0, policy_version 1929762 (0.0007) [2023-12-27 19:01:05,203][20347] Updated weights for policy 0, policy_version 1929772 (0.0008) [2023-12-27 19:01:05,259][20347] Updated weights for policy 0, policy_version 1929782 (0.0007) [2023-12-27 19:01:05,755][20381] Updated weights for policy 1, policy_version 1928314 (0.0007) [2023-12-27 19:01:05,810][20381] Updated weights for policy 1, policy_version 1928324 (0.0011) [2023-12-27 19:01:05,863][20381] Updated weights for policy 1, policy_version 1928334 (0.0011) [2023-12-27 19:01:05,899][20347] Updated weights for policy 0, policy_version 1929792 (0.0010) [2023-12-27 19:01:05,922][20381] Updated weights for policy 1, policy_version 1928344 (0.0011) [2023-12-27 19:01:05,958][20347] Updated weights for policy 0, policy_version 1929802 (0.0005) [2023-12-27 19:01:06,016][20347] Updated weights for policy 0, policy_version 1929812 (0.0005) [2023-12-27 19:01:06,618][20347] Updated weights for policy 0, policy_version 1929822 (0.0006) [2023-12-27 19:01:06,639][20381] Updated weights for policy 1, policy_version 1928354 (0.0011) [2023-12-27 19:01:06,678][20347] Updated weights for policy 0, policy_version 1929832 (0.0007) [2023-12-27 19:01:06,698][20381] Updated weights for policy 1, policy_version 1928364 (0.0011) [2023-12-27 19:01:06,733][20347] Updated weights for policy 0, policy_version 1929842 (0.0010) [2023-12-27 19:01:06,753][20381] Updated weights for policy 1, policy_version 1928374 (0.0011) [2023-12-27 19:01:07,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20207.0, 300 sec: 20438.3). Total num frames: 987856896. Throughput: 0: 9929.2, 1: 10223.1. Samples: 987829960. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.100')] [2023-12-27 19:01:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001929848_494133248.pth... [2023-12-27 19:01:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001928696_493838336.pth [2023-12-27 19:01:07,384][20381] Updated weights for policy 1, policy_version 1928384 (0.0007) [2023-12-27 19:01:07,405][20347] Updated weights for policy 0, policy_version 1929852 (0.0010) [2023-12-27 19:01:07,438][20381] Updated weights for policy 1, policy_version 1928394 (0.0011) [2023-12-27 19:01:07,454][20347] Updated weights for policy 0, policy_version 1929862 (0.0009) [2023-12-27 19:01:07,487][20381] Updated weights for policy 1, policy_version 1928404 (0.0010) [2023-12-27 19:01:07,505][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001928408_493731840.pth... [2023-12-27 19:01:07,508][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001927224_493428736.pth [2023-12-27 19:01:07,508][20347] Updated weights for policy 0, policy_version 1929872 (0.0007) [2023-12-27 19:01:08,097][20381] Updated weights for policy 1, policy_version 1928414 (0.0007) [2023-12-27 19:01:08,156][20381] Updated weights for policy 1, policy_version 1928424 (0.0005) [2023-12-27 19:01:08,193][20347] Updated weights for policy 0, policy_version 1929882 (0.0008) [2023-12-27 19:01:08,216][20381] Updated weights for policy 1, policy_version 1928434 (0.0005) [2023-12-27 19:01:08,237][20347] Updated weights for policy 0, policy_version 1929892 (0.0007) [2023-12-27 19:01:08,282][20347] Updated weights for policy 0, policy_version 1929902 (0.0007) [2023-12-27 19:01:08,331][20347] Updated weights for policy 0, policy_version 1929912 (0.0006) [2023-12-27 19:01:08,803][20381] Updated weights for policy 1, policy_version 1928444 (0.0007) [2023-12-27 19:01:08,865][20381] Updated weights for policy 1, policy_version 1928454 (0.0011) [2023-12-27 19:01:08,932][20381] Updated weights for policy 1, policy_version 1928464 (0.0011) [2023-12-27 19:01:09,061][20347] Updated weights for policy 0, policy_version 1929922 (0.0009) [2023-12-27 19:01:09,118][20347] Updated weights for policy 0, policy_version 1929932 (0.0007) [2023-12-27 19:01:09,175][20347] Updated weights for policy 0, policy_version 1929942 (0.0008) [2023-12-27 19:01:09,619][20381] Updated weights for policy 1, policy_version 1928474 (0.0008) [2023-12-27 19:01:09,681][20381] Updated weights for policy 1, policy_version 1928484 (0.0006) [2023-12-27 19:01:09,744][20381] Updated weights for policy 1, policy_version 1928494 (0.0007) [2023-12-27 19:01:09,802][20381] Updated weights for policy 1, policy_version 1928504 (0.0006) [2023-12-27 19:01:09,944][20347] Updated weights for policy 0, policy_version 1929952 (0.0007) [2023-12-27 19:01:10,001][20347] Updated weights for policy 0, policy_version 1929962 (0.0008) [2023-12-27 19:01:10,060][20347] Updated weights for policy 0, policy_version 1929972 (0.0008) [2023-12-27 19:01:10,475][20381] Updated weights for policy 1, policy_version 1928514 (0.0009) [2023-12-27 19:01:10,527][20381] Updated weights for policy 1, policy_version 1928524 (0.0008) [2023-12-27 19:01:10,579][20381] Updated weights for policy 1, policy_version 1928534 (0.0007) [2023-12-27 19:01:10,753][20347] Updated weights for policy 0, policy_version 1929982 (0.0007) [2023-12-27 19:01:10,815][20347] Updated weights for policy 0, policy_version 1929992 (0.0007) [2023-12-27 19:01:10,873][20347] Updated weights for policy 0, policy_version 1930002 (0.0006) [2023-12-27 19:01:11,318][20381] Updated weights for policy 1, policy_version 1928544 (0.0009) [2023-12-27 19:01:11,397][20381] Updated weights for policy 1, policy_version 1928554 (0.0009) [2023-12-27 19:01:11,463][20381] Updated weights for policy 1, policy_version 1928564 (0.0010) [2023-12-27 19:01:11,572][20347] Updated weights for policy 0, policy_version 1930012 (0.0007) [2023-12-27 19:01:11,640][20347] Updated weights for policy 0, policy_version 1930022 (0.0008) [2023-12-27 19:01:11,700][20347] Updated weights for policy 0, policy_version 1930032 (0.0008) [2023-12-27 19:01:12,180][20381] Updated weights for policy 1, policy_version 1928574 (0.0009) [2023-12-27 19:01:12,238][20381] Updated weights for policy 1, policy_version 1928584 (0.0010) [2023-12-27 19:01:12,303][20381] Updated weights for policy 1, policy_version 1928594 (0.0011) [2023-12-27 19:01:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20206.9, 300 sec: 20466.1). Total num frames: 987963392. Throughput: 0: 9971.6, 1: 10244.8. Samples: 987953308. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.530')] [2023-12-27 19:01:12,433][20347] Updated weights for policy 0, policy_version 1930042 (0.0008) [2023-12-27 19:01:12,487][20347] Updated weights for policy 0, policy_version 1930052 (0.0007) [2023-12-27 19:01:12,536][20347] Updated weights for policy 0, policy_version 1930062 (0.0007) [2023-12-27 19:01:12,596][20347] Updated weights for policy 0, policy_version 1930072 (0.0009) [2023-12-27 19:01:12,939][20381] Updated weights for policy 1, policy_version 1928604 (0.0011) [2023-12-27 19:01:12,994][20381] Updated weights for policy 1, policy_version 1928614 (0.0011) [2023-12-27 19:01:13,058][20381] Updated weights for policy 1, policy_version 1928624 (0.0011) [2023-12-27 19:01:13,275][20347] Updated weights for policy 0, policy_version 1930082 (0.0011) [2023-12-27 19:01:13,328][20347] Updated weights for policy 0, policy_version 1930092 (0.0009) [2023-12-27 19:01:13,384][20347] Updated weights for policy 0, policy_version 1930102 (0.0005) [2023-12-27 19:01:13,711][20381] Updated weights for policy 1, policy_version 1928634 (0.0010) [2023-12-27 19:01:13,772][20381] Updated weights for policy 1, policy_version 1928644 (0.0011) [2023-12-27 19:01:13,818][20381] Updated weights for policy 1, policy_version 1928654 (0.0010) [2023-12-27 19:01:13,864][20381] Updated weights for policy 1, policy_version 1928664 (0.0010) [2023-12-27 19:01:13,989][20347] Updated weights for policy 0, policy_version 1930112 (0.0009) [2023-12-27 19:01:14,048][20347] Updated weights for policy 0, policy_version 1930122 (0.0010) [2023-12-27 19:01:14,099][20347] Updated weights for policy 0, policy_version 1930132 (0.0010) [2023-12-27 19:01:14,548][20381] Updated weights for policy 1, policy_version 1928674 (0.0010) [2023-12-27 19:01:14,621][20381] Updated weights for policy 1, policy_version 1928684 (0.0011) [2023-12-27 19:01:14,684][20381] Updated weights for policy 1, policy_version 1928694 (0.0011) [2023-12-27 19:01:14,786][20347] Updated weights for policy 0, policy_version 1930142 (0.0011) [2023-12-27 19:01:14,845][20347] Updated weights for policy 0, policy_version 1930152 (0.0007) [2023-12-27 19:01:14,907][20347] Updated weights for policy 0, policy_version 1930162 (0.0010) [2023-12-27 19:01:15,354][20381] Updated weights for policy 1, policy_version 1928704 (0.0008) [2023-12-27 19:01:15,402][20381] Updated weights for policy 1, policy_version 1928714 (0.0005) [2023-12-27 19:01:15,452][20381] Updated weights for policy 1, policy_version 1928724 (0.0005) [2023-12-27 19:01:15,582][20347] Updated weights for policy 0, policy_version 1930172 (0.0010) [2023-12-27 19:01:15,630][20347] Updated weights for policy 0, policy_version 1930182 (0.0010) [2023-12-27 19:01:15,675][20347] Updated weights for policy 0, policy_version 1930192 (0.0010) [2023-12-27 19:01:15,988][20381] Updated weights for policy 1, policy_version 1928734 (0.0006) [2023-12-27 19:01:16,039][20381] Updated weights for policy 1, policy_version 1928744 (0.0006) [2023-12-27 19:01:16,086][20381] Updated weights for policy 1, policy_version 1928754 (0.0005) [2023-12-27 19:01:16,346][20347] Updated weights for policy 0, policy_version 1930202 (0.0010) [2023-12-27 19:01:16,410][20347] Updated weights for policy 0, policy_version 1930212 (0.0008) [2023-12-27 19:01:16,464][20347] Updated weights for policy 0, policy_version 1930222 (0.0007) [2023-12-27 19:01:16,524][20347] Updated weights for policy 0, policy_version 1930232 (0.0007) [2023-12-27 19:01:16,735][20381] Updated weights for policy 1, policy_version 1928764 (0.0005) [2023-12-27 19:01:16,780][20381] Updated weights for policy 1, policy_version 1928774 (0.0006) [2023-12-27 19:01:16,838][20381] Updated weights for policy 1, policy_version 1928784 (0.0006) [2023-12-27 19:01:17,169][20347] Updated weights for policy 0, policy_version 1930242 (0.0011) [2023-12-27 19:01:17,228][20347] Updated weights for policy 0, policy_version 1930252 (0.0010) [2023-12-27 19:01:17,280][20347] Updated weights for policy 0, policy_version 1930262 (0.0009) [2023-12-27 19:01:17,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20343.5, 300 sec: 20466.1). Total num frames: 988069888. Throughput: 0: 9932.3, 1: 10249.5. Samples: 988078200. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '824.380')] [2023-12-27 19:01:17,493][20381] Updated weights for policy 1, policy_version 1928794 (0.0006) [2023-12-27 19:01:17,555][20381] Updated weights for policy 1, policy_version 1928804 (0.0007) [2023-12-27 19:01:17,612][20381] Updated weights for policy 1, policy_version 1928814 (0.0008) [2023-12-27 19:01:17,664][20381] Updated weights for policy 1, policy_version 1928824 (0.0007) [2023-12-27 19:01:17,883][20347] Updated weights for policy 0, policy_version 1930272 (0.0005) [2023-12-27 19:01:17,950][20347] Updated weights for policy 0, policy_version 1930282 (0.0005) [2023-12-27 19:01:18,013][20347] Updated weights for policy 0, policy_version 1930292 (0.0006) [2023-12-27 19:01:18,332][20381] Updated weights for policy 1, policy_version 1928834 (0.0009) [2023-12-27 19:01:18,398][20381] Updated weights for policy 1, policy_version 1928844 (0.0009) [2023-12-27 19:01:18,460][20381] Updated weights for policy 1, policy_version 1928854 (0.0007) [2023-12-27 19:01:18,631][20347] Updated weights for policy 0, policy_version 1930302 (0.0006) [2023-12-27 19:01:18,698][20347] Updated weights for policy 0, policy_version 1930312 (0.0007) [2023-12-27 19:01:18,760][20347] Updated weights for policy 0, policy_version 1930322 (0.0007) [2023-12-27 19:01:19,159][20381] Updated weights for policy 1, policy_version 1928864 (0.0007) [2023-12-27 19:01:19,217][20381] Updated weights for policy 1, policy_version 1928874 (0.0009) [2023-12-27 19:01:19,281][20381] Updated weights for policy 1, policy_version 1928884 (0.0008) [2023-12-27 19:01:19,445][20347] Updated weights for policy 0, policy_version 1930332 (0.0007) [2023-12-27 19:01:19,503][20347] Updated weights for policy 0, policy_version 1930342 (0.0007) [2023-12-27 19:01:19,563][20347] Updated weights for policy 0, policy_version 1930352 (0.0008) [2023-12-27 19:01:19,915][20381] Updated weights for policy 1, policy_version 1928894 (0.0006) [2023-12-27 19:01:19,977][20381] Updated weights for policy 1, policy_version 1928904 (0.0008) [2023-12-27 19:01:20,036][20381] Updated weights for policy 1, policy_version 1928914 (0.0008) [2023-12-27 19:01:20,289][20347] Updated weights for policy 0, policy_version 1930362 (0.0008) [2023-12-27 19:01:20,347][20347] Updated weights for policy 0, policy_version 1930372 (0.0008) [2023-12-27 19:01:20,405][20347] Updated weights for policy 0, policy_version 1930382 (0.0007) [2023-12-27 19:01:20,459][20347] Updated weights for policy 0, policy_version 1930392 (0.0007) [2023-12-27 19:01:20,762][20381] Updated weights for policy 1, policy_version 1928924 (0.0008) [2023-12-27 19:01:20,822][20381] Updated weights for policy 1, policy_version 1928934 (0.0008) [2023-12-27 19:01:20,878][20381] Updated weights for policy 1, policy_version 1928944 (0.0007) [2023-12-27 19:01:21,197][20347] Updated weights for policy 0, policy_version 1930402 (0.0008) [2023-12-27 19:01:21,256][20347] Updated weights for policy 0, policy_version 1930412 (0.0007) [2023-12-27 19:01:21,315][20347] Updated weights for policy 0, policy_version 1930422 (0.0008) [2023-12-27 19:01:21,680][20381] Updated weights for policy 1, policy_version 1928954 (0.0007) [2023-12-27 19:01:21,752][20381] Updated weights for policy 1, policy_version 1928964 (0.0009) [2023-12-27 19:01:21,811][20381] Updated weights for policy 1, policy_version 1928974 (0.0008) [2023-12-27 19:01:21,875][20381] Updated weights for policy 1, policy_version 1928984 (0.0008) [2023-12-27 19:01:22,035][20347] Updated weights for policy 0, policy_version 1930432 (0.0008) [2023-12-27 19:01:22,091][20347] Updated weights for policy 0, policy_version 1930442 (0.0007) [2023-12-27 19:01:22,146][20347] Updated weights for policy 0, policy_version 1930452 (0.0007) [2023-12-27 19:01:22,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20343.5, 300 sec: 20438.3). Total num frames: 988168192. Throughput: 0: 9977.1, 1: 10188.2. Samples: 988139580. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.380')] [2023-12-27 19:01:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001928984_493879296.pth... [2023-12-27 19:01:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001930456_494288896.pth... [2023-12-27 19:01:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001927800_493576192.pth [2023-12-27 19:01:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001929272_493985792.pth [2023-12-27 19:01:22,605][20381] Updated weights for policy 1, policy_version 1928994 (0.0008) [2023-12-27 19:01:22,661][20381] Updated weights for policy 1, policy_version 1929004 (0.0008) [2023-12-27 19:01:22,725][20381] Updated weights for policy 1, policy_version 1929014 (0.0008) [2023-12-27 19:01:22,876][20347] Updated weights for policy 0, policy_version 1930462 (0.0006) [2023-12-27 19:01:22,932][20347] Updated weights for policy 0, policy_version 1930472 (0.0005) [2023-12-27 19:01:22,998][20347] Updated weights for policy 0, policy_version 1930482 (0.0006) [2023-12-27 19:01:23,468][20381] Updated weights for policy 1, policy_version 1929024 (0.0009) [2023-12-27 19:01:23,527][20381] Updated weights for policy 1, policy_version 1929034 (0.0010) [2023-12-27 19:01:23,588][20381] Updated weights for policy 1, policy_version 1929044 (0.0010) [2023-12-27 19:01:23,644][20347] Updated weights for policy 0, policy_version 1930492 (0.0007) [2023-12-27 19:01:23,694][20347] Updated weights for policy 0, policy_version 1930502 (0.0008) [2023-12-27 19:01:23,739][20347] Updated weights for policy 0, policy_version 1930512 (0.0009) [2023-12-27 19:01:24,168][20381] Updated weights for policy 1, policy_version 1929054 (0.0009) [2023-12-27 19:01:24,220][20381] Updated weights for policy 1, policy_version 1929064 (0.0010) [2023-12-27 19:01:24,274][20381] Updated weights for policy 1, policy_version 1929074 (0.0010) [2023-12-27 19:01:24,473][20347] Updated weights for policy 0, policy_version 1930522 (0.0007) [2023-12-27 19:01:24,531][20347] Updated weights for policy 0, policy_version 1930532 (0.0007) [2023-12-27 19:01:24,589][20347] Updated weights for policy 0, policy_version 1930542 (0.0007) [2023-12-27 19:01:24,647][20347] Updated weights for policy 0, policy_version 1930552 (0.0006) [2023-12-27 19:01:24,960][20381] Updated weights for policy 1, policy_version 1929084 (0.0009) [2023-12-27 19:01:25,022][20381] Updated weights for policy 1, policy_version 1929094 (0.0007) [2023-12-27 19:01:25,073][20381] Updated weights for policy 1, policy_version 1929104 (0.0008) [2023-12-27 19:01:25,311][20347] Updated weights for policy 0, policy_version 1930562 (0.0010) [2023-12-27 19:01:25,370][20347] Updated weights for policy 0, policy_version 1930572 (0.0010) [2023-12-27 19:01:25,430][20347] Updated weights for policy 0, policy_version 1930582 (0.0007) [2023-12-27 19:01:25,720][20381] Updated weights for policy 1, policy_version 1929114 (0.0008) [2023-12-27 19:01:25,769][20381] Updated weights for policy 1, policy_version 1929124 (0.0009) [2023-12-27 19:01:25,823][20381] Updated weights for policy 1, policy_version 1929135 (0.0008) [2023-12-27 19:01:26,021][20347] Updated weights for policy 0, policy_version 1930592 (0.0010) [2023-12-27 19:01:26,076][20347] Updated weights for policy 0, policy_version 1930602 (0.0010) [2023-12-27 19:01:26,132][20347] Updated weights for policy 0, policy_version 1930612 (0.0010) [2023-12-27 19:01:26,408][20381] Updated weights for policy 1, policy_version 1929145 (0.0005) [2023-12-27 19:01:26,468][20381] Updated weights for policy 1, policy_version 1929155 (0.0005) [2023-12-27 19:01:26,521][20381] Updated weights for policy 1, policy_version 1929165 (0.0005) [2023-12-27 19:01:26,576][20381] Updated weights for policy 1, policy_version 1929175 (0.0005) [2023-12-27 19:01:26,771][20347] Updated weights for policy 0, policy_version 1930622 (0.0010) [2023-12-27 19:01:26,826][20347] Updated weights for policy 0, policy_version 1930632 (0.0010) [2023-12-27 19:01:26,884][20347] Updated weights for policy 0, policy_version 1930642 (0.0010) [2023-12-27 19:01:27,109][20381] Updated weights for policy 1, policy_version 1929185 (0.0008) [2023-12-27 19:01:27,166][20381] Updated weights for policy 1, policy_version 1929195 (0.0006) [2023-12-27 19:01:27,231][20381] Updated weights for policy 1, policy_version 1929205 (0.0006) [2023-12-27 19:01:27,363][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 988274688. Throughput: 0: 10032.9, 1: 10131.0. Samples: 988260352. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.700')] [2023-12-27 19:01:27,537][20347] Updated weights for policy 0, policy_version 1930652 (0.0008) [2023-12-27 19:01:27,588][20347] Updated weights for policy 0, policy_version 1930662 (0.0005) [2023-12-27 19:01:27,637][20347] Updated weights for policy 0, policy_version 1930672 (0.0006) [2023-12-27 19:01:27,804][20381] Updated weights for policy 1, policy_version 1929215 (0.0006) [2023-12-27 19:01:27,854][20381] Updated weights for policy 1, policy_version 1929225 (0.0005) [2023-12-27 19:01:27,917][20381] Updated weights for policy 1, policy_version 1929235 (0.0005) [2023-12-27 19:01:28,238][20347] Updated weights for policy 0, policy_version 1930682 (0.0005) [2023-12-27 19:01:28,303][20347] Updated weights for policy 0, policy_version 1930692 (0.0007) [2023-12-27 19:01:28,364][20347] Updated weights for policy 0, policy_version 1930702 (0.0006) [2023-12-27 19:01:28,418][20347] Updated weights for policy 0, policy_version 1930712 (0.0009) [2023-12-27 19:01:28,480][20381] Updated weights for policy 1, policy_version 1929245 (0.0007) [2023-12-27 19:01:28,530][20381] Updated weights for policy 1, policy_version 1929255 (0.0008) [2023-12-27 19:01:28,573][20381] Updated weights for policy 1, policy_version 1929265 (0.0008) [2023-12-27 19:01:29,076][20347] Updated weights for policy 0, policy_version 1930722 (0.0007) [2023-12-27 19:01:29,124][20347] Updated weights for policy 0, policy_version 1930732 (0.0008) [2023-12-27 19:01:29,179][20347] Updated weights for policy 0, policy_version 1930742 (0.0008) [2023-12-27 19:01:29,284][20381] Updated weights for policy 1, policy_version 1929275 (0.0007) [2023-12-27 19:01:29,337][20381] Updated weights for policy 1, policy_version 1929285 (0.0008) [2023-12-27 19:01:29,405][20381] Updated weights for policy 1, policy_version 1929295 (0.0008) [2023-12-27 19:01:29,974][20347] Updated weights for policy 0, policy_version 1930752 (0.0007) [2023-12-27 19:01:30,027][20347] Updated weights for policy 0, policy_version 1930762 (0.0005) [2023-12-27 19:01:30,082][20381] Updated weights for policy 1, policy_version 1929305 (0.0007) [2023-12-27 19:01:30,084][20347] Updated weights for policy 0, policy_version 1930772 (0.0007) [2023-12-27 19:01:30,138][20381] Updated weights for policy 1, policy_version 1929315 (0.0007) [2023-12-27 19:01:30,197][20381] Updated weights for policy 1, policy_version 1929325 (0.0008) [2023-12-27 19:01:30,254][20381] Updated weights for policy 1, policy_version 1929335 (0.0007) [2023-12-27 19:01:30,797][20347] Updated weights for policy 0, policy_version 1930782 (0.0007) [2023-12-27 19:01:30,829][20381] Updated weights for policy 1, policy_version 1929345 (0.0006) [2023-12-27 19:01:30,844][20347] Updated weights for policy 0, policy_version 1930792 (0.0005) [2023-12-27 19:01:30,882][20381] Updated weights for policy 1, policy_version 1929355 (0.0006) [2023-12-27 19:01:30,898][20347] Updated weights for policy 0, policy_version 1930802 (0.0008) [2023-12-27 19:01:30,930][20381] Updated weights for policy 1, policy_version 1929365 (0.0006) [2023-12-27 19:01:31,564][20347] Updated weights for policy 0, policy_version 1930812 (0.0005) [2023-12-27 19:01:31,625][20347] Updated weights for policy 0, policy_version 1930822 (0.0007) [2023-12-27 19:01:31,627][20381] Updated weights for policy 1, policy_version 1929375 (0.0007) [2023-12-27 19:01:31,683][20347] Updated weights for policy 0, policy_version 1930832 (0.0007) [2023-12-27 19:01:31,688][20381] Updated weights for policy 1, policy_version 1929385 (0.0007) [2023-12-27 19:01:31,750][20381] Updated weights for policy 1, policy_version 1929395 (0.0008) [2023-12-27 19:01:32,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20206.9, 300 sec: 20410.6). Total num frames: 988372992. Throughput: 0: 10101.3, 1: 10318.9. Samples: 988387988. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '765.840')] [2023-12-27 19:01:32,386][20381] Updated weights for policy 1, policy_version 1929405 (0.0008) [2023-12-27 19:01:32,414][20347] Updated weights for policy 0, policy_version 1930842 (0.0007) [2023-12-27 19:01:32,444][20381] Updated weights for policy 1, policy_version 1929415 (0.0008) [2023-12-27 19:01:32,465][20347] Updated weights for policy 0, policy_version 1930852 (0.0007) [2023-12-27 19:01:32,495][20381] Updated weights for policy 1, policy_version 1929425 (0.0008) [2023-12-27 19:01:32,515][20347] Updated weights for policy 0, policy_version 1930862 (0.0005) [2023-12-27 19:01:32,567][20347] Updated weights for policy 0, policy_version 1930872 (0.0006) [2023-12-27 19:01:33,221][20381] Updated weights for policy 1, policy_version 1929435 (0.0009) [2023-12-27 19:01:33,234][20347] Updated weights for policy 0, policy_version 1930882 (0.0005) [2023-12-27 19:01:33,275][20381] Updated weights for policy 1, policy_version 1929445 (0.0008) [2023-12-27 19:01:33,282][20347] Updated weights for policy 0, policy_version 1930892 (0.0006) [2023-12-27 19:01:33,331][20381] Updated weights for policy 1, policy_version 1929455 (0.0007) [2023-12-27 19:01:33,340][20347] Updated weights for policy 0, policy_version 1930902 (0.0006) [2023-12-27 19:01:33,966][20347] Updated weights for policy 0, policy_version 1930912 (0.0005) [2023-12-27 19:01:34,023][20381] Updated weights for policy 1, policy_version 1929465 (0.0006) [2023-12-27 19:01:34,024][20347] Updated weights for policy 0, policy_version 1930922 (0.0005) [2023-12-27 19:01:34,072][20381] Updated weights for policy 1, policy_version 1929475 (0.0008) [2023-12-27 19:01:34,076][20347] Updated weights for policy 0, policy_version 1930932 (0.0005) [2023-12-27 19:01:34,134][20381] Updated weights for policy 1, policy_version 1929485 (0.0008) [2023-12-27 19:01:34,200][20381] Updated weights for policy 1, policy_version 1929495 (0.0007) [2023-12-27 19:01:34,714][20347] Updated weights for policy 0, policy_version 1930942 (0.0008) [2023-12-27 19:01:34,769][20347] Updated weights for policy 0, policy_version 1930952 (0.0007) [2023-12-27 19:01:34,817][20347] Updated weights for policy 0, policy_version 1930962 (0.0007) [2023-12-27 19:01:34,872][20381] Updated weights for policy 1, policy_version 1929505 (0.0007) [2023-12-27 19:01:34,936][20381] Updated weights for policy 1, policy_version 1929515 (0.0007) [2023-12-27 19:01:34,987][20381] Updated weights for policy 1, policy_version 1929525 (0.0008) [2023-12-27 19:01:35,474][20347] Updated weights for policy 0, policy_version 1930972 (0.0007) [2023-12-27 19:01:35,527][20347] Updated weights for policy 0, policy_version 1930982 (0.0005) [2023-12-27 19:01:35,585][20347] Updated weights for policy 0, policy_version 1930992 (0.0008) [2023-12-27 19:01:35,699][20381] Updated weights for policy 1, policy_version 1929535 (0.0009) [2023-12-27 19:01:35,763][20381] Updated weights for policy 1, policy_version 1929545 (0.0010) [2023-12-27 19:01:35,810][20381] Updated weights for policy 1, policy_version 1929555 (0.0010) [2023-12-27 19:01:36,116][20347] Updated weights for policy 0, policy_version 1931002 (0.0009) [2023-12-27 19:01:36,179][20347] Updated weights for policy 0, policy_version 1931012 (0.0011) [2023-12-27 19:01:36,248][20347] Updated weights for policy 0, policy_version 1931022 (0.0011) [2023-12-27 19:01:36,316][20347] Updated weights for policy 0, policy_version 1931032 (0.0010) [2023-12-27 19:01:36,443][20381] Updated weights for policy 1, policy_version 1929565 (0.0008) [2023-12-27 19:01:36,506][20381] Updated weights for policy 1, policy_version 1929575 (0.0005) [2023-12-27 19:01:36,574][20381] Updated weights for policy 1, policy_version 1929585 (0.0008) [2023-12-27 19:01:36,969][20347] Updated weights for policy 0, policy_version 1931042 (0.0005) [2023-12-27 19:01:37,029][20347] Updated weights for policy 0, policy_version 1931052 (0.0007) [2023-12-27 19:01:37,080][20347] Updated weights for policy 0, policy_version 1931062 (0.0007) [2023-12-27 19:01:37,148][20381] Updated weights for policy 1, policy_version 1929595 (0.0008) [2023-12-27 19:01:37,207][20381] Updated weights for policy 1, policy_version 1929605 (0.0007) [2023-12-27 19:01:37,258][20381] Updated weights for policy 1, policy_version 1929615 (0.0005) [2023-12-27 19:01:37,363][19170] Fps is (10 sec: 21299.2, 60 sec: 20343.4, 300 sec: 20466.1). Total num frames: 988487680. Throughput: 0: 10135.2, 1: 10354.2. Samples: 988449320. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.670')] [2023-12-27 19:01:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001929624_494043136.pth... [2023-12-27 19:01:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001931064_494444544.pth... [2023-12-27 19:01:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001928408_493731840.pth [2023-12-27 19:01:37,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001929848_494133248.pth [2023-12-27 19:01:37,725][20347] Updated weights for policy 0, policy_version 1931072 (0.0006) [2023-12-27 19:01:37,773][20347] Updated weights for policy 0, policy_version 1931082 (0.0005) [2023-12-27 19:01:37,819][20347] Updated weights for policy 0, policy_version 1931092 (0.0005) [2023-12-27 19:01:37,904][20381] Updated weights for policy 1, policy_version 1929625 (0.0005) [2023-12-27 19:01:37,957][20381] Updated weights for policy 1, policy_version 1929635 (0.0007) [2023-12-27 19:01:38,009][20381] Updated weights for policy 1, policy_version 1929645 (0.0008) [2023-12-27 19:01:38,061][20381] Updated weights for policy 1, policy_version 1929655 (0.0007) [2023-12-27 19:01:38,469][20347] Updated weights for policy 0, policy_version 1931102 (0.0005) [2023-12-27 19:01:38,528][20347] Updated weights for policy 0, policy_version 1931112 (0.0005) [2023-12-27 19:01:38,586][20347] Updated weights for policy 0, policy_version 1931122 (0.0006) [2023-12-27 19:01:38,851][20381] Updated weights for policy 1, policy_version 1929665 (0.0007) [2023-12-27 19:01:38,920][20381] Updated weights for policy 1, policy_version 1929675 (0.0008) [2023-12-27 19:01:38,978][20381] Updated weights for policy 1, policy_version 1929685 (0.0009) [2023-12-27 19:01:39,148][20347] Updated weights for policy 0, policy_version 1931132 (0.0007) [2023-12-27 19:01:39,205][20347] Updated weights for policy 0, policy_version 1931142 (0.0008) [2023-12-27 19:01:39,265][20347] Updated weights for policy 0, policy_version 1931152 (0.0008) [2023-12-27 19:01:39,720][20381] Updated weights for policy 1, policy_version 1929695 (0.0007) [2023-12-27 19:01:39,788][20381] Updated weights for policy 1, policy_version 1929705 (0.0006) [2023-12-27 19:01:39,853][20381] Updated weights for policy 1, policy_version 1929715 (0.0008) [2023-12-27 19:01:39,979][20347] Updated weights for policy 0, policy_version 1931162 (0.0006) [2023-12-27 19:01:40,047][20347] Updated weights for policy 0, policy_version 1931172 (0.0008) [2023-12-27 19:01:40,115][20347] Updated weights for policy 0, policy_version 1931182 (0.0007) [2023-12-27 19:01:40,166][20347] Updated weights for policy 0, policy_version 1931192 (0.0008) [2023-12-27 19:01:40,492][20381] Updated weights for policy 1, policy_version 1929725 (0.0008) [2023-12-27 19:01:40,544][20381] Updated weights for policy 1, policy_version 1929735 (0.0009) [2023-12-27 19:01:40,593][20381] Updated weights for policy 1, policy_version 1929745 (0.0011) [2023-12-27 19:01:40,856][20347] Updated weights for policy 0, policy_version 1931202 (0.0008) [2023-12-27 19:01:40,911][20347] Updated weights for policy 0, policy_version 1931212 (0.0007) [2023-12-27 19:01:40,965][20347] Updated weights for policy 0, policy_version 1931222 (0.0007) [2023-12-27 19:01:41,298][20381] Updated weights for policy 1, policy_version 1929755 (0.0011) [2023-12-27 19:01:41,360][20381] Updated weights for policy 1, policy_version 1929765 (0.0011) [2023-12-27 19:01:41,418][20381] Updated weights for policy 1, policy_version 1929775 (0.0011) [2023-12-27 19:01:41,732][20347] Updated weights for policy 0, policy_version 1931232 (0.0007) [2023-12-27 19:01:41,801][20347] Updated weights for policy 0, policy_version 1931242 (0.0006) [2023-12-27 19:01:41,863][20347] Updated weights for policy 0, policy_version 1931252 (0.0005) [2023-12-27 19:01:42,154][20381] Updated weights for policy 1, policy_version 1929785 (0.0011) [2023-12-27 19:01:42,219][20381] Updated weights for policy 1, policy_version 1929795 (0.0005) [2023-12-27 19:01:42,288][20381] Updated weights for policy 1, policy_version 1929805 (0.0006) [2023-12-27 19:01:42,341][20381] Updated weights for policy 1, policy_version 1929815 (0.0008) [2023-12-27 19:01:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20438.4). Total num frames: 988585984. Throughput: 0: 10261.4, 1: 10329.9. Samples: 988574792. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.280')] [2023-12-27 19:01:42,479][20347] Updated weights for policy 0, policy_version 1931262 (0.0006) [2023-12-27 19:01:42,534][20347] Updated weights for policy 0, policy_version 1931272 (0.0006) [2023-12-27 19:01:42,585][20347] Updated weights for policy 0, policy_version 1931282 (0.0008) [2023-12-27 19:01:43,013][20381] Updated weights for policy 1, policy_version 1929825 (0.0006) [2023-12-27 19:01:43,063][20381] Updated weights for policy 1, policy_version 1929835 (0.0005) [2023-12-27 19:01:43,114][20381] Updated weights for policy 1, policy_version 1929845 (0.0005) [2023-12-27 19:01:43,292][20347] Updated weights for policy 0, policy_version 1931292 (0.0007) [2023-12-27 19:01:43,339][20347] Updated weights for policy 0, policy_version 1931302 (0.0008) [2023-12-27 19:01:43,395][20347] Updated weights for policy 0, policy_version 1931312 (0.0008) [2023-12-27 19:01:43,650][20381] Updated weights for policy 1, policy_version 1929855 (0.0006) [2023-12-27 19:01:43,705][20381] Updated weights for policy 1, policy_version 1929865 (0.0007) [2023-12-27 19:01:43,757][20381] Updated weights for policy 1, policy_version 1929875 (0.0006) [2023-12-27 19:01:44,135][20347] Updated weights for policy 0, policy_version 1931322 (0.0008) [2023-12-27 19:01:44,192][20347] Updated weights for policy 0, policy_version 1931332 (0.0007) [2023-12-27 19:01:44,247][20347] Updated weights for policy 0, policy_version 1931342 (0.0007) [2023-12-27 19:01:44,297][20347] Updated weights for policy 0, policy_version 1931352 (0.0007) [2023-12-27 19:01:44,378][20381] Updated weights for policy 1, policy_version 1929885 (0.0008) [2023-12-27 19:01:44,427][20381] Updated weights for policy 1, policy_version 1929895 (0.0007) [2023-12-27 19:01:44,483][20381] Updated weights for policy 1, policy_version 1929905 (0.0007) [2023-12-27 19:01:45,007][20347] Updated weights for policy 0, policy_version 1931362 (0.0007) [2023-12-27 19:01:45,068][20347] Updated weights for policy 0, policy_version 1931372 (0.0007) [2023-12-27 19:01:45,132][20347] Updated weights for policy 0, policy_version 1931382 (0.0008) [2023-12-27 19:01:45,232][20381] Updated weights for policy 1, policy_version 1929915 (0.0007) [2023-12-27 19:01:45,295][20381] Updated weights for policy 1, policy_version 1929925 (0.0010) [2023-12-27 19:01:45,354][20381] Updated weights for policy 1, policy_version 1929935 (0.0010) [2023-12-27 19:01:45,834][20347] Updated weights for policy 0, policy_version 1931392 (0.0006) [2023-12-27 19:01:45,892][20347] Updated weights for policy 0, policy_version 1931402 (0.0006) [2023-12-27 19:01:45,928][20381] Updated weights for policy 1, policy_version 1929945 (0.0006) [2023-12-27 19:01:45,946][20347] Updated weights for policy 0, policy_version 1931412 (0.0006) [2023-12-27 19:01:45,985][20381] Updated weights for policy 1, policy_version 1929955 (0.0010) [2023-12-27 19:01:46,030][20381] Updated weights for policy 1, policy_version 1929965 (0.0010) [2023-12-27 19:01:46,075][20381] Updated weights for policy 1, policy_version 1929975 (0.0010) [2023-12-27 19:01:46,619][20347] Updated weights for policy 0, policy_version 1931422 (0.0007) [2023-12-27 19:01:46,673][20381] Updated weights for policy 1, policy_version 1929985 (0.0011) [2023-12-27 19:01:46,687][20347] Updated weights for policy 0, policy_version 1931432 (0.0007) [2023-12-27 19:01:46,725][20381] Updated weights for policy 1, policy_version 1929995 (0.0008) [2023-12-27 19:01:46,752][20347] Updated weights for policy 0, policy_version 1931442 (0.0006) [2023-12-27 19:01:46,774][20381] Updated weights for policy 1, policy_version 1930005 (0.0005) [2023-12-27 19:01:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20343.4, 300 sec: 20438.3). Total num frames: 988684288. Throughput: 0: 10281.0, 1: 10436.4. Samples: 988699352. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.160')] [2023-12-27 19:01:47,382][20381] Updated weights for policy 1, policy_version 1930015 (0.0005) [2023-12-27 19:01:47,382][20347] Updated weights for policy 0, policy_version 1931452 (0.0007) [2023-12-27 19:01:47,436][20347] Updated weights for policy 0, policy_version 1931462 (0.0007) [2023-12-27 19:01:47,437][20381] Updated weights for policy 1, policy_version 1930025 (0.0005) [2023-12-27 19:01:47,489][20347] Updated weights for policy 0, policy_version 1931472 (0.0008) [2023-12-27 19:01:47,496][20381] Updated weights for policy 1, policy_version 1930035 (0.0005) [2023-12-27 19:01:48,057][20381] Updated weights for policy 1, policy_version 1930045 (0.0007) [2023-12-27 19:01:48,105][20381] Updated weights for policy 1, policy_version 1930055 (0.0007) [2023-12-27 19:01:48,159][20381] Updated weights for policy 1, policy_version 1930065 (0.0009) [2023-12-27 19:01:48,198][20347] Updated weights for policy 0, policy_version 1931482 (0.0008) [2023-12-27 19:01:48,263][20347] Updated weights for policy 0, policy_version 1931492 (0.0008) [2023-12-27 19:01:48,321][20347] Updated weights for policy 0, policy_version 1931502 (0.0010) [2023-12-27 19:01:48,385][20347] Updated weights for policy 0, policy_version 1931512 (0.0011) [2023-12-27 19:01:48,853][20381] Updated weights for policy 1, policy_version 1930075 (0.0007) [2023-12-27 19:01:48,913][20381] Updated weights for policy 1, policy_version 1930085 (0.0010) [2023-12-27 19:01:48,976][20381] Updated weights for policy 1, policy_version 1930095 (0.0007) [2023-12-27 19:01:49,070][20347] Updated weights for policy 0, policy_version 1931522 (0.0011) [2023-12-27 19:01:49,116][20347] Updated weights for policy 0, policy_version 1931532 (0.0010) [2023-12-27 19:01:49,165][20347] Updated weights for policy 0, policy_version 1931542 (0.0010) [2023-12-27 19:01:49,679][20381] Updated weights for policy 1, policy_version 1930105 (0.0010) [2023-12-27 19:01:49,738][20381] Updated weights for policy 1, policy_version 1930115 (0.0008) [2023-12-27 19:01:49,801][20381] Updated weights for policy 1, policy_version 1930125 (0.0007) [2023-12-27 19:01:49,865][20381] Updated weights for policy 1, policy_version 1930135 (0.0007) [2023-12-27 19:01:49,888][20347] Updated weights for policy 0, policy_version 1931552 (0.0007) [2023-12-27 19:01:49,947][20347] Updated weights for policy 0, policy_version 1931562 (0.0009) [2023-12-27 19:01:50,011][20347] Updated weights for policy 0, policy_version 1931572 (0.0011) [2023-12-27 19:01:50,543][20381] Updated weights for policy 1, policy_version 1930145 (0.0007) [2023-12-27 19:01:50,601][20347] Updated weights for policy 0, policy_version 1931582 (0.0007) [2023-12-27 19:01:50,614][20381] Updated weights for policy 1, policy_version 1930155 (0.0006) [2023-12-27 19:01:50,661][20347] Updated weights for policy 0, policy_version 1931592 (0.0007) [2023-12-27 19:01:50,674][20381] Updated weights for policy 1, policy_version 1930165 (0.0006) [2023-12-27 19:01:50,716][20347] Updated weights for policy 0, policy_version 1931602 (0.0007) [2023-12-27 19:01:51,419][20381] Updated weights for policy 1, policy_version 1930175 (0.0007) [2023-12-27 19:01:51,458][20347] Updated weights for policy 0, policy_version 1931612 (0.0007) [2023-12-27 19:01:51,477][20381] Updated weights for policy 1, policy_version 1930185 (0.0008) [2023-12-27 19:01:51,507][20347] Updated weights for policy 0, policy_version 1931622 (0.0008) [2023-12-27 19:01:51,524][20381] Updated weights for policy 1, policy_version 1930195 (0.0008) [2023-12-27 19:01:51,558][20347] Updated weights for policy 0, policy_version 1931632 (0.0012) [2023-12-27 19:01:52,226][20347] Updated weights for policy 0, policy_version 1931642 (0.0010) [2023-12-27 19:01:52,273][20381] Updated weights for policy 1, policy_version 1930205 (0.0010) [2023-12-27 19:01:52,289][20347] Updated weights for policy 0, policy_version 1931652 (0.0007) [2023-12-27 19:01:52,335][20381] Updated weights for policy 1, policy_version 1930215 (0.0007) [2023-12-27 19:01:52,355][20347] Updated weights for policy 0, policy_version 1931662 (0.0006) [2023-12-27 19:01:52,362][19170] Fps is (10 sec: 19660.5, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 988782592. Throughput: 0: 10257.7, 1: 10432.0. Samples: 988760996. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.880')] [2023-12-27 19:01:52,415][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001931672_494600192.pth... [2023-12-27 19:01:52,415][20381] Updated weights for policy 1, policy_version 1930225 (0.0009) [2023-12-27 19:01:52,416][20347] Updated weights for policy 0, policy_version 1931672 (0.0008) [2023-12-27 19:01:52,419][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001930456_494288896.pth [2023-12-27 19:01:52,451][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001930232_494198784.pth... [2023-12-27 19:01:52,454][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001928984_493879296.pth [2023-12-27 19:01:53,022][20347] Updated weights for policy 0, policy_version 1931682 (0.0007) [2023-12-27 19:01:53,075][20381] Updated weights for policy 1, policy_version 1930235 (0.0011) [2023-12-27 19:01:53,081][20347] Updated weights for policy 0, policy_version 1931692 (0.0007) [2023-12-27 19:01:53,130][20381] Updated weights for policy 1, policy_version 1930245 (0.0011) [2023-12-27 19:01:53,143][20347] Updated weights for policy 0, policy_version 1931702 (0.0005) [2023-12-27 19:01:53,186][20381] Updated weights for policy 1, policy_version 1930255 (0.0010) [2023-12-27 19:01:53,820][20347] Updated weights for policy 0, policy_version 1931712 (0.0005) [2023-12-27 19:01:53,875][20381] Updated weights for policy 1, policy_version 1930265 (0.0011) [2023-12-27 19:01:53,875][20347] Updated weights for policy 0, policy_version 1931722 (0.0005) [2023-12-27 19:01:53,929][20347] Updated weights for policy 0, policy_version 1931732 (0.0005) [2023-12-27 19:01:53,933][20381] Updated weights for policy 1, policy_version 1930275 (0.0010) [2023-12-27 19:01:53,987][20381] Updated weights for policy 1, policy_version 1930285 (0.0010) [2023-12-27 19:01:54,002][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000001 [2023-12-27 19:01:54,565][20347] Updated weights for policy 0, policy_version 1931742 (0.0008) [2023-12-27 19:01:54,610][20347] Updated weights for policy 0, policy_version 1931752 (0.0010) [2023-12-27 19:01:54,655][20347] Updated weights for policy 0, policy_version 1931762 (0.0010) [2023-12-27 19:01:54,714][20381] Updated weights for policy 1, policy_version 1930295 (0.0011) [2023-12-27 19:01:54,765][20381] Updated weights for policy 1, policy_version 1930305 (0.0010) [2023-12-27 19:01:54,816][20381] Updated weights for policy 1, policy_version 1930315 (0.0010) [2023-12-27 19:01:55,361][20347] Updated weights for policy 0, policy_version 1931772 (0.0009) [2023-12-27 19:01:55,417][20347] Updated weights for policy 0, policy_version 1931782 (0.0007) [2023-12-27 19:01:55,467][20347] Updated weights for policy 0, policy_version 1931792 (0.0007) [2023-12-27 19:01:55,511][20381] Updated weights for policy 1, policy_version 1930325 (0.0010) [2023-12-27 19:01:55,569][20381] Updated weights for policy 1, policy_version 1930335 (0.0011) [2023-12-27 19:01:55,628][20381] Updated weights for policy 1, policy_version 1930345 (0.0011) [2023-12-27 19:01:56,157][20347] Updated weights for policy 0, policy_version 1931802 (0.0007) [2023-12-27 19:01:56,205][20347] Updated weights for policy 0, policy_version 1931812 (0.0005) [2023-12-27 19:01:56,251][20347] Updated weights for policy 0, policy_version 1931822 (0.0007) [2023-12-27 19:01:56,309][20347] Updated weights for policy 0, policy_version 1931832 (0.0010) [2023-12-27 19:01:56,311][20381] Updated weights for policy 1, policy_version 1930355 (0.0010) [2023-12-27 19:01:56,362][20381] Updated weights for policy 1, policy_version 1930365 (0.0010) [2023-12-27 19:01:56,416][20381] Updated weights for policy 1, policy_version 1930375 (0.0010) [2023-12-27 19:01:56,964][20347] Updated weights for policy 0, policy_version 1931842 (0.0010) [2023-12-27 19:01:57,015][20347] Updated weights for policy 0, policy_version 1931852 (0.0010) [2023-12-27 19:01:57,072][20347] Updated weights for policy 0, policy_version 1931862 (0.0010) [2023-12-27 19:01:57,105][20381] Updated weights for policy 1, policy_version 1930385 (0.0010) [2023-12-27 19:01:57,165][20381] Updated weights for policy 1, policy_version 1930395 (0.0010) [2023-12-27 19:01:57,222][20381] Updated weights for policy 1, policy_version 1930405 (0.0010) [2023-12-27 19:01:57,271][20381] Updated weights for policy 1, policy_version 1930415 (0.0010) [2023-12-27 19:01:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 988897280. Throughput: 0: 10292.7, 1: 10367.8. Samples: 988883032. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:01:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.630')] [2023-12-27 19:01:57,748][20347] Updated weights for policy 0, policy_version 1931872 (0.0010) [2023-12-27 19:01:57,789][20347] Updated weights for policy 0, policy_version 1931882 (0.0010) [2023-12-27 19:01:57,833][20347] Updated weights for policy 0, policy_version 1931892 (0.0010) [2023-12-27 19:01:57,946][20381] Updated weights for policy 1, policy_version 1930425 (0.0007) [2023-12-27 19:01:57,991][20381] Updated weights for policy 1, policy_version 1930435 (0.0005) [2023-12-27 19:01:58,064][20381] Updated weights for policy 1, policy_version 1930445 (0.0006) [2023-12-27 19:01:58,599][20347] Updated weights for policy 0, policy_version 1931902 (0.0009) [2023-12-27 19:01:58,671][20347] Updated weights for policy 0, policy_version 1931912 (0.0008) [2023-12-27 19:01:58,718][20381] Updated weights for policy 1, policy_version 1930455 (0.0007) [2023-12-27 19:01:58,743][20347] Updated weights for policy 0, policy_version 1931922 (0.0008) [2023-12-27 19:01:58,785][20381] Updated weights for policy 1, policy_version 1930465 (0.0007) [2023-12-27 19:01:58,844][20381] Updated weights for policy 1, policy_version 1930475 (0.0008) [2023-12-27 19:01:59,459][20347] Updated weights for policy 0, policy_version 1931932 (0.0008) [2023-12-27 19:01:59,519][20347] Updated weights for policy 0, policy_version 1931942 (0.0007) [2023-12-27 19:01:59,559][20381] Updated weights for policy 1, policy_version 1930485 (0.0008) [2023-12-27 19:01:59,578][20347] Updated weights for policy 0, policy_version 1931952 (0.0005) [2023-12-27 19:01:59,615][20381] Updated weights for policy 1, policy_version 1930495 (0.0007) [2023-12-27 19:01:59,669][20381] Updated weights for policy 1, policy_version 1930505 (0.0006) [2023-12-27 19:01:59,705][20292] Early stopping after 4 epochs (32 sgd steps), loss delta 0.0000001 [2023-12-27 19:02:00,219][20347] Updated weights for policy 0, policy_version 1931962 (0.0006) [2023-12-27 19:02:00,276][20347] Updated weights for policy 0, policy_version 1931972 (0.0005) [2023-12-27 19:02:00,333][20347] Updated weights for policy 0, policy_version 1931982 (0.0005) [2023-12-27 19:02:00,387][20347] Updated weights for policy 0, policy_version 1931992 (0.0006) [2023-12-27 19:02:00,399][20381] Updated weights for policy 1, policy_version 1930515 (0.0008) [2023-12-27 19:02:00,452][20381] Updated weights for policy 1, policy_version 1930525 (0.0008) [2023-12-27 19:02:00,504][20381] Updated weights for policy 1, policy_version 1930535 (0.0008) [2023-12-27 19:02:00,911][20347] Updated weights for policy 0, policy_version 1932002 (0.0005) [2023-12-27 19:02:00,969][20347] Updated weights for policy 0, policy_version 1932012 (0.0005) [2023-12-27 19:02:01,025][20347] Updated weights for policy 0, policy_version 1932022 (0.0006) [2023-12-27 19:02:01,291][20381] Updated weights for policy 1, policy_version 1930545 (0.0008) [2023-12-27 19:02:01,354][20381] Updated weights for policy 1, policy_version 1930555 (0.0007) [2023-12-27 19:02:01,419][20381] Updated weights for policy 1, policy_version 1930565 (0.0009) [2023-12-27 19:02:01,475][20381] Updated weights for policy 1, policy_version 1930575 (0.0009) [2023-12-27 19:02:01,640][20347] Updated weights for policy 0, policy_version 1932032 (0.0010) [2023-12-27 19:02:01,699][20347] Updated weights for policy 0, policy_version 1932042 (0.0008) [2023-12-27 19:02:01,765][20347] Updated weights for policy 0, policy_version 1932052 (0.0008) [2023-12-27 19:02:02,207][20381] Updated weights for policy 1, policy_version 1930585 (0.0007) [2023-12-27 19:02:02,272][20381] Updated weights for policy 1, policy_version 1930595 (0.0007) [2023-12-27 19:02:02,327][20381] Updated weights for policy 1, policy_version 1930605 (0.0008) [2023-12-27 19:02:02,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 988995584. Throughput: 0: 10321.1, 1: 10263.4. Samples: 989004500. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.070')] [2023-12-27 19:02:02,434][20347] Updated weights for policy 0, policy_version 1932062 (0.0007) [2023-12-27 19:02:02,489][20347] Updated weights for policy 0, policy_version 1932072 (0.0008) [2023-12-27 19:02:02,541][20347] Updated weights for policy 0, policy_version 1932082 (0.0008) [2023-12-27 19:02:02,963][20381] Updated weights for policy 1, policy_version 1930615 (0.0008) [2023-12-27 19:02:03,018][20381] Updated weights for policy 1, policy_version 1930625 (0.0008) [2023-12-27 19:02:03,073][20381] Updated weights for policy 1, policy_version 1930635 (0.0005) [2023-12-27 19:02:03,219][20347] Updated weights for policy 0, policy_version 1932092 (0.0007) [2023-12-27 19:02:03,264][20347] Updated weights for policy 0, policy_version 1932102 (0.0005) [2023-12-27 19:02:03,320][20347] Updated weights for policy 0, policy_version 1932112 (0.0006) [2023-12-27 19:02:03,715][20381] Updated weights for policy 1, policy_version 1930645 (0.0008) [2023-12-27 19:02:03,767][20381] Updated weights for policy 1, policy_version 1930655 (0.0010) [2023-12-27 19:02:03,819][20381] Updated weights for policy 1, policy_version 1930665 (0.0010) [2023-12-27 19:02:03,911][20347] Updated weights for policy 0, policy_version 1932122 (0.0006) [2023-12-27 19:02:03,963][20347] Updated weights for policy 0, policy_version 1932132 (0.0007) [2023-12-27 19:02:04,023][20347] Updated weights for policy 0, policy_version 1932142 (0.0007) [2023-12-27 19:02:04,077][20347] Updated weights for policy 0, policy_version 1932152 (0.0008) [2023-12-27 19:02:04,509][20381] Updated weights for policy 1, policy_version 1930675 (0.0008) [2023-12-27 19:02:04,571][20381] Updated weights for policy 1, policy_version 1930685 (0.0008) [2023-12-27 19:02:04,623][20381] Updated weights for policy 1, policy_version 1930695 (0.0008) [2023-12-27 19:02:04,757][20347] Updated weights for policy 0, policy_version 1932162 (0.0009) [2023-12-27 19:02:04,815][20347] Updated weights for policy 0, policy_version 1932172 (0.0010) [2023-12-27 19:02:04,879][20347] Updated weights for policy 0, policy_version 1932182 (0.0010) [2023-12-27 19:02:05,237][20381] Updated weights for policy 1, policy_version 1930705 (0.0007) [2023-12-27 19:02:05,292][20381] Updated weights for policy 1, policy_version 1930715 (0.0005) [2023-12-27 19:02:05,349][20381] Updated weights for policy 1, policy_version 1930725 (0.0005) [2023-12-27 19:02:05,408][20381] Updated weights for policy 1, policy_version 1930735 (0.0005) [2023-12-27 19:02:05,507][20347] Updated weights for policy 0, policy_version 1932192 (0.0010) [2023-12-27 19:02:05,556][20347] Updated weights for policy 0, policy_version 1932202 (0.0010) [2023-12-27 19:02:05,601][20347] Updated weights for policy 0, policy_version 1932212 (0.0010) [2023-12-27 19:02:05,996][20381] Updated weights for policy 1, policy_version 1930745 (0.0007) [2023-12-27 19:02:06,047][20381] Updated weights for policy 1, policy_version 1930755 (0.0007) [2023-12-27 19:02:06,091][20381] Updated weights for policy 1, policy_version 1930765 (0.0007) [2023-12-27 19:02:06,277][20347] Updated weights for policy 0, policy_version 1932222 (0.0008) [2023-12-27 19:02:06,335][20347] Updated weights for policy 0, policy_version 1932232 (0.0007) [2023-12-27 19:02:06,396][20347] Updated weights for policy 0, policy_version 1932242 (0.0006) [2023-12-27 19:02:06,881][20381] Updated weights for policy 1, policy_version 1930775 (0.0008) [2023-12-27 19:02:06,935][20381] Updated weights for policy 1, policy_version 1930785 (0.0007) [2023-12-27 19:02:06,993][20381] Updated weights for policy 1, policy_version 1930795 (0.0007) [2023-12-27 19:02:07,053][20347] Updated weights for policy 0, policy_version 1932252 (0.0008) [2023-12-27 19:02:07,106][20347] Updated weights for policy 0, policy_version 1932262 (0.0011) [2023-12-27 19:02:07,155][20347] Updated weights for policy 0, policy_version 1932272 (0.0010) [2023-12-27 19:02:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 989102080. Throughput: 0: 10331.4, 1: 10313.4. Samples: 989068596. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.530')] [2023-12-27 19:02:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001930800_494346240.pth... [2023-12-27 19:02:07,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001932280_494755840.pth... [2023-12-27 19:02:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001929624_494043136.pth [2023-12-27 19:02:07,376][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001931064_494444544.pth [2023-12-27 19:02:07,675][20381] Updated weights for policy 1, policy_version 1930805 (0.0008) [2023-12-27 19:02:07,725][20347] Updated weights for policy 0, policy_version 1932282 (0.0009) [2023-12-27 19:02:07,727][20381] Updated weights for policy 1, policy_version 1930815 (0.0007) [2023-12-27 19:02:07,773][20347] Updated weights for policy 0, policy_version 1932292 (0.0005) [2023-12-27 19:02:07,788][20381] Updated weights for policy 1, policy_version 1930825 (0.0008) [2023-12-27 19:02:07,828][20347] Updated weights for policy 0, policy_version 1932302 (0.0006) [2023-12-27 19:02:07,885][20347] Updated weights for policy 0, policy_version 1932312 (0.0006) [2023-12-27 19:02:08,438][20381] Updated weights for policy 1, policy_version 1930835 (0.0007) [2023-12-27 19:02:08,494][20381] Updated weights for policy 1, policy_version 1930845 (0.0006) [2023-12-27 19:02:08,503][20347] Updated weights for policy 0, policy_version 1932322 (0.0008) [2023-12-27 19:02:08,553][20381] Updated weights for policy 1, policy_version 1930855 (0.0007) [2023-12-27 19:02:08,559][20347] Updated weights for policy 0, policy_version 1932332 (0.0006) [2023-12-27 19:02:08,616][20347] Updated weights for policy 0, policy_version 1932342 (0.0006) [2023-12-27 19:02:09,266][20381] Updated weights for policy 1, policy_version 1930865 (0.0006) [2023-12-27 19:02:09,303][20347] Updated weights for policy 0, policy_version 1932352 (0.0007) [2023-12-27 19:02:09,329][20381] Updated weights for policy 1, policy_version 1930875 (0.0007) [2023-12-27 19:02:09,364][20347] Updated weights for policy 0, policy_version 1932362 (0.0007) [2023-12-27 19:02:09,394][20381] Updated weights for policy 1, policy_version 1930885 (0.0008) [2023-12-27 19:02:09,438][20347] Updated weights for policy 0, policy_version 1932372 (0.0007) [2023-12-27 19:02:09,460][20381] Updated weights for policy 1, policy_version 1930895 (0.0007) [2023-12-27 19:02:10,175][20347] Updated weights for policy 0, policy_version 1932382 (0.0007) [2023-12-27 19:02:10,208][20381] Updated weights for policy 1, policy_version 1930905 (0.0008) [2023-12-27 19:02:10,233][20347] Updated weights for policy 0, policy_version 1932392 (0.0006) [2023-12-27 19:02:10,269][20381] Updated weights for policy 1, policy_version 1930915 (0.0006) [2023-12-27 19:02:10,291][20347] Updated weights for policy 0, policy_version 1932402 (0.0006) [2023-12-27 19:02:10,320][20381] Updated weights for policy 1, policy_version 1930925 (0.0005) [2023-12-27 19:02:10,959][20347] Updated weights for policy 0, policy_version 1932412 (0.0007) [2023-12-27 19:02:11,019][20347] Updated weights for policy 0, policy_version 1932422 (0.0006) [2023-12-27 19:02:11,032][20381] Updated weights for policy 1, policy_version 1930935 (0.0008) [2023-12-27 19:02:11,088][20347] Updated weights for policy 0, policy_version 1932432 (0.0007) [2023-12-27 19:02:11,100][20381] Updated weights for policy 1, policy_version 1930945 (0.0008) [2023-12-27 19:02:11,164][20381] Updated weights for policy 1, policy_version 1930955 (0.0008) [2023-12-27 19:02:11,849][20347] Updated weights for policy 0, policy_version 1932442 (0.0008) [2023-12-27 19:02:11,853][20381] Updated weights for policy 1, policy_version 1930965 (0.0008) [2023-12-27 19:02:11,903][20381] Updated weights for policy 1, policy_version 1930975 (0.0006) [2023-12-27 19:02:11,916][20347] Updated weights for policy 0, policy_version 1932452 (0.0007) [2023-12-27 19:02:11,962][20381] Updated weights for policy 1, policy_version 1930985 (0.0006) [2023-12-27 19:02:11,976][20347] Updated weights for policy 0, policy_version 1932462 (0.0006) [2023-12-27 19:02:12,033][20347] Updated weights for policy 0, policy_version 1932472 (0.0006) [2023-12-27 19:02:12,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20616.5, 300 sec: 20438.3). Total num frames: 989200384. Throughput: 0: 10387.5, 1: 10294.8. Samples: 989191052. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.300')] [2023-12-27 19:02:12,636][20381] Updated weights for policy 1, policy_version 1930995 (0.0006) [2023-12-27 19:02:12,669][20347] Updated weights for policy 0, policy_version 1932482 (0.0006) [2023-12-27 19:02:12,690][20381] Updated weights for policy 1, policy_version 1931005 (0.0007) [2023-12-27 19:02:12,719][20347] Updated weights for policy 0, policy_version 1932492 (0.0005) [2023-12-27 19:02:12,745][20381] Updated weights for policy 1, policy_version 1931015 (0.0006) [2023-12-27 19:02:12,774][20347] Updated weights for policy 0, policy_version 1932502 (0.0006) [2023-12-27 19:02:13,386][20381] Updated weights for policy 1, policy_version 1931025 (0.0007) [2023-12-27 19:02:13,442][20381] Updated weights for policy 1, policy_version 1931035 (0.0005) [2023-12-27 19:02:13,469][20347] Updated weights for policy 0, policy_version 1932512 (0.0005) [2023-12-27 19:02:13,493][20381] Updated weights for policy 1, policy_version 1931045 (0.0008) [2023-12-27 19:02:13,517][20347] Updated weights for policy 0, policy_version 1932522 (0.0007) [2023-12-27 19:02:13,542][20381] Updated weights for policy 1, policy_version 1931055 (0.0007) [2023-12-27 19:02:13,570][20347] Updated weights for policy 0, policy_version 1932532 (0.0007) [2023-12-27 19:02:14,189][20381] Updated weights for policy 1, policy_version 1931065 (0.0008) [2023-12-27 19:02:14,247][20381] Updated weights for policy 1, policy_version 1931075 (0.0008) [2023-12-27 19:02:14,274][20347] Updated weights for policy 0, policy_version 1932542 (0.0008) [2023-12-27 19:02:14,303][20381] Updated weights for policy 1, policy_version 1931085 (0.0007) [2023-12-27 19:02:14,321][20347] Updated weights for policy 0, policy_version 1932552 (0.0007) [2023-12-27 19:02:14,373][20347] Updated weights for policy 0, policy_version 1932562 (0.0008) [2023-12-27 19:02:15,005][20381] Updated weights for policy 1, policy_version 1931095 (0.0007) [2023-12-27 19:02:15,064][20347] Updated weights for policy 0, policy_version 1932572 (0.0006) [2023-12-27 19:02:15,070][20381] Updated weights for policy 1, policy_version 1931105 (0.0008) [2023-12-27 19:02:15,129][20347] Updated weights for policy 0, policy_version 1932582 (0.0006) [2023-12-27 19:02:15,132][20381] Updated weights for policy 1, policy_version 1931115 (0.0007) [2023-12-27 19:02:15,188][20347] Updated weights for policy 0, policy_version 1932592 (0.0006) [2023-12-27 19:02:15,800][20381] Updated weights for policy 1, policy_version 1931125 (0.0007) [2023-12-27 19:02:15,850][20381] Updated weights for policy 1, policy_version 1931135 (0.0007) [2023-12-27 19:02:15,851][20347] Updated weights for policy 0, policy_version 1932602 (0.0006) [2023-12-27 19:02:15,902][20381] Updated weights for policy 1, policy_version 1931145 (0.0007) [2023-12-27 19:02:15,902][20347] Updated weights for policy 0, policy_version 1932612 (0.0006) [2023-12-27 19:02:15,948][20347] Updated weights for policy 0, policy_version 1932622 (0.0005) [2023-12-27 19:02:15,999][20347] Updated weights for policy 0, policy_version 1932632 (0.0005) [2023-12-27 19:02:16,546][20381] Updated weights for policy 1, policy_version 1931155 (0.0007) [2023-12-27 19:02:16,603][20381] Updated weights for policy 1, policy_version 1931165 (0.0005) [2023-12-27 19:02:16,621][20347] Updated weights for policy 0, policy_version 1932642 (0.0008) [2023-12-27 19:02:16,656][20381] Updated weights for policy 1, policy_version 1931175 (0.0006) [2023-12-27 19:02:16,685][20347] Updated weights for policy 0, policy_version 1932652 (0.0007) [2023-12-27 19:02:16,743][20347] Updated weights for policy 0, policy_version 1932662 (0.0008) [2023-12-27 19:02:17,352][20381] Updated weights for policy 1, policy_version 1931185 (0.0006) [2023-12-27 19:02:17,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20410.6). Total num frames: 989298688. Throughput: 0: 10373.2, 1: 10215.5. Samples: 989314476. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '912.130')] [2023-12-27 19:02:17,407][20381] Updated weights for policy 1, policy_version 1931195 (0.0007) [2023-12-27 19:02:17,425][20347] Updated weights for policy 0, policy_version 1932672 (0.0010) [2023-12-27 19:02:17,464][20381] Updated weights for policy 1, policy_version 1931205 (0.0005) [2023-12-27 19:02:17,477][20347] Updated weights for policy 0, policy_version 1932682 (0.0010) [2023-12-27 19:02:17,511][20381] Updated weights for policy 1, policy_version 1931215 (0.0007) [2023-12-27 19:02:17,524][20347] Updated weights for policy 0, policy_version 1932692 (0.0006) [2023-12-27 19:02:18,181][20381] Updated weights for policy 1, policy_version 1931225 (0.0006) [2023-12-27 19:02:18,216][20347] Updated weights for policy 0, policy_version 1932702 (0.0008) [2023-12-27 19:02:18,233][20381] Updated weights for policy 1, policy_version 1931235 (0.0008) [2023-12-27 19:02:18,273][20347] Updated weights for policy 0, policy_version 1932712 (0.0006) [2023-12-27 19:02:18,283][20381] Updated weights for policy 1, policy_version 1931245 (0.0005) [2023-12-27 19:02:18,332][20347] Updated weights for policy 0, policy_version 1932722 (0.0007) [2023-12-27 19:02:18,988][20381] Updated weights for policy 1, policy_version 1931255 (0.0005) [2023-12-27 19:02:19,038][20381] Updated weights for policy 1, policy_version 1931265 (0.0005) [2023-12-27 19:02:19,046][20347] Updated weights for policy 0, policy_version 1932732 (0.0007) [2023-12-27 19:02:19,090][20381] Updated weights for policy 1, policy_version 1931275 (0.0006) [2023-12-27 19:02:19,093][20347] Updated weights for policy 0, policy_version 1932742 (0.0006) [2023-12-27 19:02:19,150][20347] Updated weights for policy 0, policy_version 1932752 (0.0007) [2023-12-27 19:02:19,750][20381] Updated weights for policy 1, policy_version 1931285 (0.0006) [2023-12-27 19:02:19,814][20381] Updated weights for policy 1, policy_version 1931295 (0.0006) [2023-12-27 19:02:19,878][20381] Updated weights for policy 1, policy_version 1931305 (0.0008) [2023-12-27 19:02:19,935][20347] Updated weights for policy 0, policy_version 1932762 (0.0007) [2023-12-27 19:02:20,000][20347] Updated weights for policy 0, policy_version 1932772 (0.0008) [2023-12-27 19:02:20,059][20347] Updated weights for policy 0, policy_version 1932782 (0.0007) [2023-12-27 19:02:20,119][20347] Updated weights for policy 0, policy_version 1932792 (0.0008) [2023-12-27 19:02:20,570][20381] Updated weights for policy 1, policy_version 1931315 (0.0007) [2023-12-27 19:02:20,636][20381] Updated weights for policy 1, policy_version 1931325 (0.0008) [2023-12-27 19:02:20,693][20381] Updated weights for policy 1, policy_version 1931335 (0.0008) [2023-12-27 19:02:20,837][20347] Updated weights for policy 0, policy_version 1932802 (0.0008) [2023-12-27 19:02:20,897][20347] Updated weights for policy 0, policy_version 1932812 (0.0008) [2023-12-27 19:02:20,958][20347] Updated weights for policy 0, policy_version 1932822 (0.0008) [2023-12-27 19:02:21,438][20381] Updated weights for policy 1, policy_version 1931345 (0.0007) [2023-12-27 19:02:21,491][20381] Updated weights for policy 1, policy_version 1931355 (0.0008) [2023-12-27 19:02:21,555][20381] Updated weights for policy 1, policy_version 1931365 (0.0007) [2023-12-27 19:02:21,616][20381] Updated weights for policy 1, policy_version 1931375 (0.0007) [2023-12-27 19:02:21,697][20347] Updated weights for policy 0, policy_version 1932832 (0.0007) [2023-12-27 19:02:21,770][20347] Updated weights for policy 0, policy_version 1932842 (0.0009) [2023-12-27 19:02:21,839][20347] Updated weights for policy 0, policy_version 1932852 (0.0007) [2023-12-27 19:02:22,318][20381] Updated weights for policy 1, policy_version 1931385 (0.0008) [2023-12-27 19:02:22,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20480.0, 300 sec: 20382.8). Total num frames: 989396992. Throughput: 0: 10308.8, 1: 10242.5. Samples: 989374128. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.100')] [2023-12-27 19:02:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001932856_494903296.pth... [2023-12-27 19:02:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001931672_494600192.pth [2023-12-27 19:02:22,390][20381] Updated weights for policy 1, policy_version 1931395 (0.0008) [2023-12-27 19:02:22,457][20381] Updated weights for policy 1, policy_version 1931405 (0.0008) [2023-12-27 19:02:22,475][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001931408_494501888.pth... [2023-12-27 19:02:22,479][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001930232_494198784.pth [2023-12-27 19:02:22,574][20347] Updated weights for policy 0, policy_version 1932862 (0.0008) [2023-12-27 19:02:22,633][20347] Updated weights for policy 0, policy_version 1932872 (0.0006) [2023-12-27 19:02:22,697][20347] Updated weights for policy 0, policy_version 1932882 (0.0008) [2023-12-27 19:02:23,147][20381] Updated weights for policy 1, policy_version 1931415 (0.0007) [2023-12-27 19:02:23,202][20381] Updated weights for policy 1, policy_version 1931425 (0.0007) [2023-12-27 19:02:23,261][20381] Updated weights for policy 1, policy_version 1931435 (0.0007) [2023-12-27 19:02:23,361][20347] Updated weights for policy 0, policy_version 1932892 (0.0006) [2023-12-27 19:02:23,407][20347] Updated weights for policy 0, policy_version 1932902 (0.0008) [2023-12-27 19:02:23,453][20347] Updated weights for policy 0, policy_version 1932912 (0.0006) [2023-12-27 19:02:23,880][20381] Updated weights for policy 1, policy_version 1931445 (0.0007) [2023-12-27 19:02:23,935][20381] Updated weights for policy 1, policy_version 1931455 (0.0007) [2023-12-27 19:02:23,989][20381] Updated weights for policy 1, policy_version 1931465 (0.0007) [2023-12-27 19:02:24,176][20347] Updated weights for policy 0, policy_version 1932922 (0.0007) [2023-12-27 19:02:24,231][20347] Updated weights for policy 0, policy_version 1932932 (0.0007) [2023-12-27 19:02:24,289][20347] Updated weights for policy 0, policy_version 1932942 (0.0008) [2023-12-27 19:02:24,334][20347] Updated weights for policy 0, policy_version 1932952 (0.0008) [2023-12-27 19:02:24,671][20381] Updated weights for policy 1, policy_version 1931475 (0.0007) [2023-12-27 19:02:24,721][20381] Updated weights for policy 1, policy_version 1931485 (0.0005) [2023-12-27 19:02:24,776][20381] Updated weights for policy 1, policy_version 1931495 (0.0005) [2023-12-27 19:02:25,038][20347] Updated weights for policy 0, policy_version 1932962 (0.0008) [2023-12-27 19:02:25,090][20347] Updated weights for policy 0, policy_version 1932972 (0.0008) [2023-12-27 19:02:25,145][20347] Updated weights for policy 0, policy_version 1932982 (0.0008) [2023-12-27 19:02:25,426][20381] Updated weights for policy 1, policy_version 1931505 (0.0007) [2023-12-27 19:02:25,484][20381] Updated weights for policy 1, policy_version 1931515 (0.0005) [2023-12-27 19:02:25,544][20381] Updated weights for policy 1, policy_version 1931525 (0.0006) [2023-12-27 19:02:25,604][20381] Updated weights for policy 1, policy_version 1931535 (0.0008) [2023-12-27 19:02:25,800][20347] Updated weights for policy 0, policy_version 1932992 (0.0010) [2023-12-27 19:02:25,855][20347] Updated weights for policy 0, policy_version 1933002 (0.0010) [2023-12-27 19:02:25,913][20347] Updated weights for policy 0, policy_version 1933012 (0.0007) [2023-12-27 19:02:26,277][20381] Updated weights for policy 1, policy_version 1931545 (0.0007) [2023-12-27 19:02:26,337][20381] Updated weights for policy 1, policy_version 1931555 (0.0007) [2023-12-27 19:02:26,396][20381] Updated weights for policy 1, policy_version 1931565 (0.0007) [2023-12-27 19:02:26,527][20347] Updated weights for policy 0, policy_version 1933022 (0.0008) [2023-12-27 19:02:26,582][20347] Updated weights for policy 0, policy_version 1933032 (0.0010) [2023-12-27 19:02:26,639][20347] Updated weights for policy 0, policy_version 1933042 (0.0010) [2023-12-27 19:02:27,070][20381] Updated weights for policy 1, policy_version 1931575 (0.0008) [2023-12-27 19:02:27,128][20381] Updated weights for policy 1, policy_version 1931585 (0.0006) [2023-12-27 19:02:27,184][20381] Updated weights for policy 1, policy_version 1931595 (0.0006) [2023-12-27 19:02:27,286][20347] Updated weights for policy 0, policy_version 1933052 (0.0009) [2023-12-27 19:02:27,345][20347] Updated weights for policy 0, policy_version 1933062 (0.0007) [2023-12-27 19:02:27,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20480.1, 300 sec: 20410.6). Total num frames: 989503488. Throughput: 0: 10204.0, 1: 10229.5. Samples: 989494300. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.780')] [2023-12-27 19:02:27,408][20347] Updated weights for policy 0, policy_version 1933072 (0.0008) [2023-12-27 19:02:27,831][20381] Updated weights for policy 1, policy_version 1931605 (0.0008) [2023-12-27 19:02:27,888][20381] Updated weights for policy 1, policy_version 1931615 (0.0006) [2023-12-27 19:02:27,946][20381] Updated weights for policy 1, policy_version 1931625 (0.0005) [2023-12-27 19:02:27,991][20347] Updated weights for policy 0, policy_version 1933082 (0.0006) [2023-12-27 19:02:28,045][20347] Updated weights for policy 0, policy_version 1933092 (0.0005) [2023-12-27 19:02:28,099][20347] Updated weights for policy 0, policy_version 1933102 (0.0005) [2023-12-27 19:02:28,159][20347] Updated weights for policy 0, policy_version 1933112 (0.0009) [2023-12-27 19:02:28,651][20381] Updated weights for policy 1, policy_version 1931635 (0.0005) [2023-12-27 19:02:28,698][20347] Updated weights for policy 0, policy_version 1933122 (0.0008) [2023-12-27 19:02:28,707][20381] Updated weights for policy 1, policy_version 1931645 (0.0006) [2023-12-27 19:02:28,753][20347] Updated weights for policy 0, policy_version 1933132 (0.0010) [2023-12-27 19:02:28,762][20381] Updated weights for policy 1, policy_version 1931655 (0.0008) [2023-12-27 19:02:28,810][20347] Updated weights for policy 0, policy_version 1933142 (0.0009) [2023-12-27 19:02:29,432][20381] Updated weights for policy 1, policy_version 1931665 (0.0005) [2023-12-27 19:02:29,465][20347] Updated weights for policy 0, policy_version 1933152 (0.0010) [2023-12-27 19:02:29,497][20381] Updated weights for policy 1, policy_version 1931675 (0.0006) [2023-12-27 19:02:29,524][20347] Updated weights for policy 0, policy_version 1933162 (0.0011) [2023-12-27 19:02:29,555][20381] Updated weights for policy 1, policy_version 1931685 (0.0006) [2023-12-27 19:02:29,580][20347] Updated weights for policy 0, policy_version 1933172 (0.0009) [2023-12-27 19:02:29,604][20381] Updated weights for policy 1, policy_version 1931695 (0.0006) [2023-12-27 19:02:30,245][20347] Updated weights for policy 0, policy_version 1933182 (0.0007) [2023-12-27 19:02:30,262][20381] Updated weights for policy 1, policy_version 1931705 (0.0006) [2023-12-27 19:02:30,295][20347] Updated weights for policy 0, policy_version 1933192 (0.0005) [2023-12-27 19:02:30,322][20381] Updated weights for policy 1, policy_version 1931715 (0.0008) [2023-12-27 19:02:30,353][20347] Updated weights for policy 0, policy_version 1933202 (0.0006) [2023-12-27 19:02:30,381][20381] Updated weights for policy 1, policy_version 1931725 (0.0007) [2023-12-27 19:02:30,970][20381] Updated weights for policy 1, policy_version 1931735 (0.0007) [2023-12-27 19:02:30,981][20347] Updated weights for policy 0, policy_version 1933212 (0.0009) [2023-12-27 19:02:31,017][20381] Updated weights for policy 1, policy_version 1931745 (0.0006) [2023-12-27 19:02:31,036][20347] Updated weights for policy 0, policy_version 1933222 (0.0010) [2023-12-27 19:02:31,083][20381] Updated weights for policy 1, policy_version 1931755 (0.0007) [2023-12-27 19:02:31,095][20347] Updated weights for policy 0, policy_version 1933232 (0.0010) [2023-12-27 19:02:31,773][20381] Updated weights for policy 1, policy_version 1931765 (0.0007) [2023-12-27 19:02:31,837][20381] Updated weights for policy 1, policy_version 1931775 (0.0006) [2023-12-27 19:02:31,841][20347] Updated weights for policy 0, policy_version 1933242 (0.0010) [2023-12-27 19:02:31,896][20347] Updated weights for policy 0, policy_version 1933252 (0.0006) [2023-12-27 19:02:31,901][20381] Updated weights for policy 1, policy_version 1931785 (0.0006) [2023-12-27 19:02:31,950][20347] Updated weights for policy 0, policy_version 1933262 (0.0006) [2023-12-27 19:02:32,004][20347] Updated weights for policy 0, policy_version 1933272 (0.0010) [2023-12-27 19:02:32,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20410.6). Total num frames: 989609984. Throughput: 0: 10302.4, 1: 10185.7. Samples: 989621316. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '881.550')] [2023-12-27 19:02:32,541][20381] Updated weights for policy 1, policy_version 1931795 (0.0010) [2023-12-27 19:02:32,586][20381] Updated weights for policy 1, policy_version 1931805 (0.0010) [2023-12-27 19:02:32,631][20381] Updated weights for policy 1, policy_version 1931815 (0.0010) [2023-12-27 19:02:32,637][20347] Updated weights for policy 0, policy_version 1933282 (0.0008) [2023-12-27 19:02:32,697][20347] Updated weights for policy 0, policy_version 1933292 (0.0008) [2023-12-27 19:02:32,754][20347] Updated weights for policy 0, policy_version 1933302 (0.0006) [2023-12-27 19:02:33,271][20381] Updated weights for policy 1, policy_version 1931825 (0.0010) [2023-12-27 19:02:33,328][20381] Updated weights for policy 1, policy_version 1931835 (0.0008) [2023-12-27 19:02:33,329][20347] Updated weights for policy 0, policy_version 1933312 (0.0006) [2023-12-27 19:02:33,382][20347] Updated weights for policy 0, policy_version 1933322 (0.0007) [2023-12-27 19:02:33,383][20381] Updated weights for policy 1, policy_version 1931845 (0.0010) [2023-12-27 19:02:33,435][20347] Updated weights for policy 0, policy_version 1933332 (0.0006) [2023-12-27 19:02:33,437][20381] Updated weights for policy 1, policy_version 1931855 (0.0010) [2023-12-27 19:02:34,051][20381] Updated weights for policy 1, policy_version 1931865 (0.0010) [2023-12-27 19:02:34,106][20347] Updated weights for policy 0, policy_version 1933342 (0.0008) [2023-12-27 19:02:34,107][20381] Updated weights for policy 1, policy_version 1931875 (0.0010) [2023-12-27 19:02:34,165][20347] Updated weights for policy 0, policy_version 1933352 (0.0011) [2023-12-27 19:02:34,169][20381] Updated weights for policy 1, policy_version 1931885 (0.0008) [2023-12-27 19:02:34,226][20347] Updated weights for policy 0, policy_version 1933362 (0.0009) [2023-12-27 19:02:34,833][20381] Updated weights for policy 1, policy_version 1931895 (0.0006) [2023-12-27 19:02:34,889][20381] Updated weights for policy 1, policy_version 1931905 (0.0006) [2023-12-27 19:02:34,933][20347] Updated weights for policy 0, policy_version 1933372 (0.0007) [2023-12-27 19:02:34,949][20381] Updated weights for policy 1, policy_version 1931915 (0.0008) [2023-12-27 19:02:34,995][20347] Updated weights for policy 0, policy_version 1933382 (0.0005) [2023-12-27 19:02:35,047][20347] Updated weights for policy 0, policy_version 1933392 (0.0005) [2023-12-27 19:02:35,558][20381] Updated weights for policy 1, policy_version 1931925 (0.0008) [2023-12-27 19:02:35,609][20381] Updated weights for policy 1, policy_version 1931935 (0.0009) [2023-12-27 19:02:35,621][20347] Updated weights for policy 0, policy_version 1933402 (0.0006) [2023-12-27 19:02:35,656][20381] Updated weights for policy 1, policy_version 1931945 (0.0007) [2023-12-27 19:02:35,678][20347] Updated weights for policy 0, policy_version 1933412 (0.0008) [2023-12-27 19:02:35,737][20347] Updated weights for policy 0, policy_version 1933422 (0.0008) [2023-12-27 19:02:35,800][20347] Updated weights for policy 0, policy_version 1933432 (0.0008) [2023-12-27 19:02:36,347][20381] Updated weights for policy 1, policy_version 1931955 (0.0006) [2023-12-27 19:02:36,402][20381] Updated weights for policy 1, policy_version 1931965 (0.0008) [2023-12-27 19:02:36,449][20347] Updated weights for policy 0, policy_version 1933442 (0.0010) [2023-12-27 19:02:36,459][20381] Updated weights for policy 1, policy_version 1931975 (0.0005) [2023-12-27 19:02:36,512][20347] Updated weights for policy 0, policy_version 1933452 (0.0010) [2023-12-27 19:02:36,568][20347] Updated weights for policy 0, policy_version 1933462 (0.0006) [2023-12-27 19:02:37,147][20381] Updated weights for policy 1, policy_version 1931985 (0.0006) [2023-12-27 19:02:37,182][20347] Updated weights for policy 0, policy_version 1933472 (0.0006) [2023-12-27 19:02:37,205][20381] Updated weights for policy 1, policy_version 1931995 (0.0005) [2023-12-27 19:02:37,238][20347] Updated weights for policy 0, policy_version 1933482 (0.0006) [2023-12-27 19:02:37,259][20381] Updated weights for policy 1, policy_version 1932005 (0.0005) [2023-12-27 19:02:37,297][20347] Updated weights for policy 0, policy_version 1933492 (0.0010) [2023-12-27 19:02:37,306][20381] Updated weights for policy 1, policy_version 1932015 (0.0005) [2023-12-27 19:02:37,362][19170] Fps is (10 sec: 22118.0, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 989724672. Throughput: 0: 10350.6, 1: 10189.2. Samples: 989685284. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.220')] [2023-12-27 19:02:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001932016_494657536.pth... [2023-12-27 19:02:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001933496_495067136.pth... [2023-12-27 19:02:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001930800_494346240.pth [2023-12-27 19:02:37,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001932280_494755840.pth [2023-12-27 19:02:37,961][20347] Updated weights for policy 0, policy_version 1933502 (0.0010) [2023-12-27 19:02:37,967][20381] Updated weights for policy 1, policy_version 1932025 (0.0007) [2023-12-27 19:02:38,016][20347] Updated weights for policy 0, policy_version 1933512 (0.0009) [2023-12-27 19:02:38,025][20381] Updated weights for policy 1, policy_version 1932035 (0.0005) [2023-12-27 19:02:38,071][20347] Updated weights for policy 0, policy_version 1933522 (0.0010) [2023-12-27 19:02:38,075][20381] Updated weights for policy 1, policy_version 1932045 (0.0008) [2023-12-27 19:02:38,723][20347] Updated weights for policy 0, policy_version 1933532 (0.0009) [2023-12-27 19:02:38,733][20381] Updated weights for policy 1, policy_version 1932055 (0.0008) [2023-12-27 19:02:38,782][20347] Updated weights for policy 0, policy_version 1933542 (0.0007) [2023-12-27 19:02:38,795][20381] Updated weights for policy 1, policy_version 1932065 (0.0007) [2023-12-27 19:02:38,845][20347] Updated weights for policy 0, policy_version 1933552 (0.0006) [2023-12-27 19:02:38,853][20381] Updated weights for policy 1, policy_version 1932075 (0.0005) [2023-12-27 19:02:39,456][20381] Updated weights for policy 1, policy_version 1932085 (0.0007) [2023-12-27 19:02:39,524][20381] Updated weights for policy 1, policy_version 1932095 (0.0009) [2023-12-27 19:02:39,553][20347] Updated weights for policy 0, policy_version 1933562 (0.0007) [2023-12-27 19:02:39,581][20381] Updated weights for policy 1, policy_version 1932105 (0.0007) [2023-12-27 19:02:39,610][20347] Updated weights for policy 0, policy_version 1933572 (0.0006) [2023-12-27 19:02:39,673][20347] Updated weights for policy 0, policy_version 1933582 (0.0007) [2023-12-27 19:02:39,730][20347] Updated weights for policy 0, policy_version 1933592 (0.0008) [2023-12-27 19:02:40,325][20381] Updated weights for policy 1, policy_version 1932115 (0.0008) [2023-12-27 19:02:40,386][20381] Updated weights for policy 1, policy_version 1932125 (0.0008) [2023-12-27 19:02:40,393][20347] Updated weights for policy 0, policy_version 1933602 (0.0008) [2023-12-27 19:02:40,437][20381] Updated weights for policy 1, policy_version 1932135 (0.0007) [2023-12-27 19:02:40,443][20347] Updated weights for policy 0, policy_version 1933612 (0.0008) [2023-12-27 19:02:40,489][20347] Updated weights for policy 0, policy_version 1933622 (0.0007) [2023-12-27 19:02:41,025][20381] Updated weights for policy 1, policy_version 1932145 (0.0007) [2023-12-27 19:02:41,097][20381] Updated weights for policy 1, policy_version 1932155 (0.0008) [2023-12-27 19:02:41,163][20381] Updated weights for policy 1, policy_version 1932165 (0.0008) [2023-12-27 19:02:41,228][20381] Updated weights for policy 1, policy_version 1932175 (0.0007) [2023-12-27 19:02:41,255][20347] Updated weights for policy 0, policy_version 1933632 (0.0006) [2023-12-27 19:02:41,317][20347] Updated weights for policy 0, policy_version 1933642 (0.0009) [2023-12-27 19:02:41,387][20347] Updated weights for policy 0, policy_version 1933652 (0.0008) [2023-12-27 19:02:41,969][20381] Updated weights for policy 1, policy_version 1932185 (0.0007) [2023-12-27 19:02:42,025][20381] Updated weights for policy 1, policy_version 1932195 (0.0008) [2023-12-27 19:02:42,082][20381] Updated weights for policy 1, policy_version 1932205 (0.0007) [2023-12-27 19:02:42,136][20347] Updated weights for policy 0, policy_version 1933662 (0.0008) [2023-12-27 19:02:42,189][20347] Updated weights for policy 0, policy_version 1933672 (0.0007) [2023-12-27 19:02:42,249][20347] Updated weights for policy 0, policy_version 1933682 (0.0007) [2023-12-27 19:02:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 989822976. Throughput: 0: 10358.6, 1: 10263.2. Samples: 989811012. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '843.350')] [2023-12-27 19:02:42,790][20381] Updated weights for policy 1, policy_version 1932215 (0.0008) [2023-12-27 19:02:42,848][20381] Updated weights for policy 1, policy_version 1932225 (0.0008) [2023-12-27 19:02:42,897][20381] Updated weights for policy 1, policy_version 1932235 (0.0007) [2023-12-27 19:02:42,986][20347] Updated weights for policy 0, policy_version 1933692 (0.0008) [2023-12-27 19:02:43,050][20347] Updated weights for policy 0, policy_version 1933702 (0.0008) [2023-12-27 19:02:43,114][20347] Updated weights for policy 0, policy_version 1933712 (0.0008) [2023-12-27 19:02:43,515][20381] Updated weights for policy 1, policy_version 1932245 (0.0007) [2023-12-27 19:02:43,560][20381] Updated weights for policy 1, policy_version 1932255 (0.0007) [2023-12-27 19:02:43,608][20381] Updated weights for policy 1, policy_version 1932265 (0.0007) [2023-12-27 19:02:43,858][20347] Updated weights for policy 0, policy_version 1933722 (0.0008) [2023-12-27 19:02:43,913][20347] Updated weights for policy 0, policy_version 1933732 (0.0009) [2023-12-27 19:02:43,958][20347] Updated weights for policy 0, policy_version 1933742 (0.0009) [2023-12-27 19:02:44,006][20347] Updated weights for policy 0, policy_version 1933752 (0.0007) [2023-12-27 19:02:44,266][20381] Updated weights for policy 1, policy_version 1932275 (0.0007) [2023-12-27 19:02:44,316][20381] Updated weights for policy 1, policy_version 1932285 (0.0008) [2023-12-27 19:02:44,371][20381] Updated weights for policy 1, policy_version 1932295 (0.0007) [2023-12-27 19:02:44,625][20347] Updated weights for policy 0, policy_version 1933762 (0.0008) [2023-12-27 19:02:44,673][20347] Updated weights for policy 0, policy_version 1933772 (0.0007) [2023-12-27 19:02:44,728][20347] Updated weights for policy 0, policy_version 1933782 (0.0007) [2023-12-27 19:02:45,067][20381] Updated weights for policy 1, policy_version 1932305 (0.0008) [2023-12-27 19:02:45,131][20381] Updated weights for policy 1, policy_version 1932315 (0.0006) [2023-12-27 19:02:45,197][20381] Updated weights for policy 1, policy_version 1932325 (0.0005) [2023-12-27 19:02:45,264][20381] Updated weights for policy 1, policy_version 1932335 (0.0007) [2023-12-27 19:02:45,457][20347] Updated weights for policy 0, policy_version 1933792 (0.0009) [2023-12-27 19:02:45,515][20347] Updated weights for policy 0, policy_version 1933802 (0.0008) [2023-12-27 19:02:45,574][20347] Updated weights for policy 0, policy_version 1933812 (0.0007) [2023-12-27 19:02:45,789][20381] Updated weights for policy 1, policy_version 1932345 (0.0006) [2023-12-27 19:02:45,840][20381] Updated weights for policy 1, policy_version 1932355 (0.0005) [2023-12-27 19:02:45,890][20381] Updated weights for policy 1, policy_version 1932365 (0.0005) [2023-12-27 19:02:46,252][20347] Updated weights for policy 0, policy_version 1933822 (0.0006) [2023-12-27 19:02:46,306][20347] Updated weights for policy 0, policy_version 1933832 (0.0005) [2023-12-27 19:02:46,355][20347] Updated weights for policy 0, policy_version 1933842 (0.0005) [2023-12-27 19:02:46,429][20381] Updated weights for policy 1, policy_version 1932375 (0.0006) [2023-12-27 19:02:46,488][20381] Updated weights for policy 1, policy_version 1932385 (0.0005) [2023-12-27 19:02:46,544][20381] Updated weights for policy 1, policy_version 1932395 (0.0005) [2023-12-27 19:02:46,938][20347] Updated weights for policy 0, policy_version 1933852 (0.0007) [2023-12-27 19:02:46,996][20347] Updated weights for policy 0, policy_version 1933862 (0.0010) [2023-12-27 19:02:47,048][20347] Updated weights for policy 0, policy_version 1933872 (0.0010) [2023-12-27 19:02:47,178][20381] Updated weights for policy 1, policy_version 1932405 (0.0008) [2023-12-27 19:02:47,226][20381] Updated weights for policy 1, policy_version 1932415 (0.0007) [2023-12-27 19:02:47,275][20381] Updated weights for policy 1, policy_version 1932425 (0.0005) [2023-12-27 19:02:47,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 989929472. Throughput: 0: 10293.3, 1: 10398.0. Samples: 989935612. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '804.510')] [2023-12-27 19:02:47,694][20347] Updated weights for policy 0, policy_version 1933882 (0.0010) [2023-12-27 19:02:47,758][20347] Updated weights for policy 0, policy_version 1933892 (0.0010) [2023-12-27 19:02:47,824][20347] Updated weights for policy 0, policy_version 1933902 (0.0011) [2023-12-27 19:02:47,873][20381] Updated weights for policy 1, policy_version 1932435 (0.0005) [2023-12-27 19:02:47,879][20347] Updated weights for policy 0, policy_version 1933912 (0.0009) [2023-12-27 19:02:47,935][20381] Updated weights for policy 1, policy_version 1932445 (0.0005) [2023-12-27 19:02:47,982][20381] Updated weights for policy 1, policy_version 1932455 (0.0007) [2023-12-27 19:02:48,556][20347] Updated weights for policy 0, policy_version 1933922 (0.0006) [2023-12-27 19:02:48,616][20347] Updated weights for policy 0, policy_version 1933932 (0.0007) [2023-12-27 19:02:48,674][20347] Updated weights for policy 0, policy_version 1933942 (0.0008) [2023-12-27 19:02:48,689][20381] Updated weights for policy 1, policy_version 1932465 (0.0007) [2023-12-27 19:02:48,746][20381] Updated weights for policy 1, policy_version 1932475 (0.0008) [2023-12-27 19:02:48,806][20381] Updated weights for policy 1, policy_version 1932485 (0.0008) [2023-12-27 19:02:48,863][20381] Updated weights for policy 1, policy_version 1932495 (0.0008) [2023-12-27 19:02:49,358][20347] Updated weights for policy 0, policy_version 1933952 (0.0007) [2023-12-27 19:02:49,423][20347] Updated weights for policy 0, policy_version 1933962 (0.0009) [2023-12-27 19:02:49,480][20347] Updated weights for policy 0, policy_version 1933972 (0.0008) [2023-12-27 19:02:49,573][20381] Updated weights for policy 1, policy_version 1932505 (0.0008) [2023-12-27 19:02:49,622][20381] Updated weights for policy 1, policy_version 1932515 (0.0008) [2023-12-27 19:02:49,679][20381] Updated weights for policy 1, policy_version 1932525 (0.0008) [2023-12-27 19:02:50,175][20347] Updated weights for policy 0, policy_version 1933982 (0.0007) [2023-12-27 19:02:50,230][20347] Updated weights for policy 0, policy_version 1933992 (0.0008) [2023-12-27 19:02:50,283][20347] Updated weights for policy 0, policy_version 1934002 (0.0007) [2023-12-27 19:02:50,394][20381] Updated weights for policy 1, policy_version 1932535 (0.0008) [2023-12-27 19:02:50,451][20381] Updated weights for policy 1, policy_version 1932545 (0.0008) [2023-12-27 19:02:50,502][20381] Updated weights for policy 1, policy_version 1932555 (0.0008) [2023-12-27 19:02:50,953][20347] Updated weights for policy 0, policy_version 1934012 (0.0007) [2023-12-27 19:02:51,012][20347] Updated weights for policy 0, policy_version 1934022 (0.0007) [2023-12-27 19:02:51,075][20347] Updated weights for policy 0, policy_version 1934032 (0.0008) [2023-12-27 19:02:51,264][20381] Updated weights for policy 1, policy_version 1932565 (0.0009) [2023-12-27 19:02:51,319][20381] Updated weights for policy 1, policy_version 1932575 (0.0007) [2023-12-27 19:02:51,386][20381] Updated weights for policy 1, policy_version 1932585 (0.0009) [2023-12-27 19:02:51,835][20347] Updated weights for policy 0, policy_version 1934042 (0.0008) [2023-12-27 19:02:51,898][20347] Updated weights for policy 0, policy_version 1934052 (0.0008) [2023-12-27 19:02:51,956][20347] Updated weights for policy 0, policy_version 1934062 (0.0007) [2023-12-27 19:02:52,025][20347] Updated weights for policy 0, policy_version 1934072 (0.0008) [2023-12-27 19:02:52,057][20381] Updated weights for policy 1, policy_version 1932595 (0.0006) [2023-12-27 19:02:52,108][20381] Updated weights for policy 1, policy_version 1932605 (0.0007) [2023-12-27 19:02:52,157][20381] Updated weights for policy 1, policy_version 1932615 (0.0008) [2023-12-27 19:02:52,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 990027776. Throughput: 0: 10251.7, 1: 10361.8. Samples: 989996208. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '851.840')] [2023-12-27 19:02:52,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001934072_495214592.pth... [2023-12-27 19:02:52,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001932624_494813184.pth... [2023-12-27 19:02:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001932856_494903296.pth [2023-12-27 19:02:52,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001931408_494501888.pth [2023-12-27 19:02:52,722][20347] Updated weights for policy 0, policy_version 1934082 (0.0007) [2023-12-27 19:02:52,774][20347] Updated weights for policy 0, policy_version 1934092 (0.0008) [2023-12-27 19:02:52,826][20347] Updated weights for policy 0, policy_version 1934102 (0.0008) [2023-12-27 19:02:52,879][20381] Updated weights for policy 1, policy_version 1932625 (0.0007) [2023-12-27 19:02:52,925][20381] Updated weights for policy 1, policy_version 1932635 (0.0007) [2023-12-27 19:02:52,973][20381] Updated weights for policy 1, policy_version 1932645 (0.0008) [2023-12-27 19:02:53,031][20381] Updated weights for policy 1, policy_version 1932655 (0.0005) [2023-12-27 19:02:53,585][20347] Updated weights for policy 0, policy_version 1934112 (0.0010) [2023-12-27 19:02:53,639][20347] Updated weights for policy 0, policy_version 1934122 (0.0010) [2023-12-27 19:02:53,645][20381] Updated weights for policy 1, policy_version 1932665 (0.0005) [2023-12-27 19:02:53,687][20347] Updated weights for policy 0, policy_version 1934132 (0.0006) [2023-12-27 19:02:53,699][20381] Updated weights for policy 1, policy_version 1932675 (0.0005) [2023-12-27 19:02:53,753][20381] Updated weights for policy 1, policy_version 1932685 (0.0006) [2023-12-27 19:02:54,232][20347] Updated weights for policy 0, policy_version 1934142 (0.0007) [2023-12-27 19:02:54,283][20347] Updated weights for policy 0, policy_version 1934152 (0.0010) [2023-12-27 19:02:54,344][20347] Updated weights for policy 0, policy_version 1934162 (0.0010) [2023-12-27 19:02:54,429][20381] Updated weights for policy 1, policy_version 1932695 (0.0006) [2023-12-27 19:02:54,475][20381] Updated weights for policy 1, policy_version 1932705 (0.0005) [2023-12-27 19:02:54,523][20381] Updated weights for policy 1, policy_version 1932715 (0.0006) [2023-12-27 19:02:54,945][20347] Updated weights for policy 0, policy_version 1934172 (0.0008) [2023-12-27 19:02:55,000][20347] Updated weights for policy 0, policy_version 1934182 (0.0010) [2023-12-27 19:02:55,056][20347] Updated weights for policy 0, policy_version 1934192 (0.0010) [2023-12-27 19:02:55,216][20381] Updated weights for policy 1, policy_version 1932725 (0.0007) [2023-12-27 19:02:55,274][20381] Updated weights for policy 1, policy_version 1932735 (0.0007) [2023-12-27 19:02:55,330][20381] Updated weights for policy 1, policy_version 1932745 (0.0007) [2023-12-27 19:02:55,640][20347] Updated weights for policy 0, policy_version 1934202 (0.0009) [2023-12-27 19:02:55,698][20347] Updated weights for policy 0, policy_version 1934212 (0.0005) [2023-12-27 19:02:55,754][20347] Updated weights for policy 0, policy_version 1934222 (0.0005) [2023-12-27 19:02:55,810][20347] Updated weights for policy 0, policy_version 1934232 (0.0005) [2023-12-27 19:02:56,031][20381] Updated weights for policy 1, policy_version 1932755 (0.0009) [2023-12-27 19:02:56,080][20381] Updated weights for policy 1, policy_version 1932765 (0.0008) [2023-12-27 19:02:56,138][20381] Updated weights for policy 1, policy_version 1932775 (0.0007) [2023-12-27 19:02:56,370][20347] Updated weights for policy 0, policy_version 1934242 (0.0010) [2023-12-27 19:02:56,418][20347] Updated weights for policy 0, policy_version 1934252 (0.0010) [2023-12-27 19:02:56,463][20347] Updated weights for policy 0, policy_version 1934262 (0.0010) [2023-12-27 19:02:56,780][20381] Updated weights for policy 1, policy_version 1932785 (0.0006) [2023-12-27 19:02:56,830][20381] Updated weights for policy 1, policy_version 1932795 (0.0009) [2023-12-27 19:02:56,881][20381] Updated weights for policy 1, policy_version 1932805 (0.0005) [2023-12-27 19:02:56,932][20381] Updated weights for policy 1, policy_version 1932815 (0.0010) [2023-12-27 19:02:57,125][20347] Updated weights for policy 0, policy_version 1934272 (0.0006) [2023-12-27 19:02:57,180][20347] Updated weights for policy 0, policy_version 1934282 (0.0005) [2023-12-27 19:02:57,228][20347] Updated weights for policy 0, policy_version 1934292 (0.0008) [2023-12-27 19:02:57,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 990134272. Throughput: 0: 10266.8, 1: 10378.4. Samples: 990120084. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-12-27 19:02:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.320')] [2023-12-27 19:02:57,611][20381] Updated weights for policy 1, policy_version 1932825 (0.0010) [2023-12-27 19:02:57,666][20381] Updated weights for policy 1, policy_version 1932835 (0.0008) [2023-12-27 19:02:57,719][20381] Updated weights for policy 1, policy_version 1932845 (0.0007) [2023-12-27 19:02:57,898][20347] Updated weights for policy 0, policy_version 1934302 (0.0010) [2023-12-27 19:02:57,949][20347] Updated weights for policy 0, policy_version 1934312 (0.0010) [2023-12-27 19:02:58,004][20347] Updated weights for policy 0, policy_version 1934322 (0.0010) [2023-12-27 19:02:58,321][20381] Updated weights for policy 1, policy_version 1932855 (0.0011) [2023-12-27 19:02:58,388][20381] Updated weights for policy 1, policy_version 1932865 (0.0011) [2023-12-27 19:02:58,449][20381] Updated weights for policy 1, policy_version 1932875 (0.0009) [2023-12-27 19:02:58,763][20347] Updated weights for policy 0, policy_version 1934332 (0.0010) [2023-12-27 19:02:58,824][20347] Updated weights for policy 0, policy_version 1934342 (0.0009) [2023-12-27 19:02:58,894][20347] Updated weights for policy 0, policy_version 1934352 (0.0010) [2023-12-27 19:02:59,227][20381] Updated weights for policy 1, policy_version 1932885 (0.0008) [2023-12-27 19:02:59,290][20381] Updated weights for policy 1, policy_version 1932895 (0.0008) [2023-12-27 19:02:59,355][20381] Updated weights for policy 1, policy_version 1932905 (0.0008) [2023-12-27 19:02:59,722][20347] Updated weights for policy 0, policy_version 1934362 (0.0008) [2023-12-27 19:02:59,774][20347] Updated weights for policy 0, policy_version 1934372 (0.0007) [2023-12-27 19:02:59,841][20347] Updated weights for policy 0, policy_version 1934382 (0.0008) [2023-12-27 19:02:59,896][20347] Updated weights for policy 0, policy_version 1934392 (0.0007) [2023-12-27 19:03:00,038][20381] Updated weights for policy 1, policy_version 1932915 (0.0008) [2023-12-27 19:03:00,093][20381] Updated weights for policy 1, policy_version 1932925 (0.0007) [2023-12-27 19:03:00,156][20381] Updated weights for policy 1, policy_version 1932935 (0.0007) [2023-12-27 19:03:00,639][20347] Updated weights for policy 0, policy_version 1934402 (0.0007) [2023-12-27 19:03:00,701][20347] Updated weights for policy 0, policy_version 1934412 (0.0009) [2023-12-27 19:03:00,755][20347] Updated weights for policy 0, policy_version 1934422 (0.0009) [2023-12-27 19:03:00,801][20381] Updated weights for policy 1, policy_version 1932945 (0.0007) [2023-12-27 19:03:00,856][20381] Updated weights for policy 1, policy_version 1932955 (0.0008) [2023-12-27 19:03:00,905][20381] Updated weights for policy 1, policy_version 1932965 (0.0009) [2023-12-27 19:03:00,954][20381] Updated weights for policy 1, policy_version 1932975 (0.0009) [2023-12-27 19:03:01,357][20347] Updated weights for policy 0, policy_version 1934432 (0.0007) [2023-12-27 19:03:01,417][20347] Updated weights for policy 0, policy_version 1934442 (0.0008) [2023-12-27 19:03:01,474][20347] Updated weights for policy 0, policy_version 1934452 (0.0006) [2023-12-27 19:03:01,645][20381] Updated weights for policy 1, policy_version 1932985 (0.0008) [2023-12-27 19:03:01,708][20381] Updated weights for policy 1, policy_version 1932995 (0.0008) [2023-12-27 19:03:01,775][20381] Updated weights for policy 1, policy_version 1933005 (0.0007) [2023-12-27 19:03:02,183][20347] Updated weights for policy 0, policy_version 1934462 (0.0007) [2023-12-27 19:03:02,242][20347] Updated weights for policy 0, policy_version 1934472 (0.0005) [2023-12-27 19:03:02,307][20347] Updated weights for policy 0, policy_version 1934482 (0.0006) [2023-12-27 19:03:02,326][20381] Updated weights for policy 1, policy_version 1933015 (0.0006) [2023-12-27 19:03:02,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.6, 300 sec: 20466.1). Total num frames: 990232576. Throughput: 0: 10242.2, 1: 10390.0. Samples: 990242928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '823.380')] [2023-12-27 19:03:02,390][20381] Updated weights for policy 1, policy_version 1933025 (0.0006) [2023-12-27 19:03:02,445][20381] Updated weights for policy 1, policy_version 1933035 (0.0006) [2023-12-27 19:03:02,908][20347] Updated weights for policy 0, policy_version 1934492 (0.0008) [2023-12-27 19:03:02,953][20347] Updated weights for policy 0, policy_version 1934502 (0.0007) [2023-12-27 19:03:03,001][20381] Updated weights for policy 1, policy_version 1933045 (0.0005) [2023-12-27 19:03:03,002][20347] Updated weights for policy 0, policy_version 1934512 (0.0005) [2023-12-27 19:03:03,054][20381] Updated weights for policy 1, policy_version 1933055 (0.0005) [2023-12-27 19:03:03,101][20381] Updated weights for policy 1, policy_version 1933065 (0.0007) [2023-12-27 19:03:03,574][20347] Updated weights for policy 0, policy_version 1934522 (0.0009) [2023-12-27 19:03:03,634][20347] Updated weights for policy 0, policy_version 1934532 (0.0005) [2023-12-27 19:03:03,692][20347] Updated weights for policy 0, policy_version 1934542 (0.0006) [2023-12-27 19:03:03,749][20381] Updated weights for policy 1, policy_version 1933075 (0.0007) [2023-12-27 19:03:03,750][20347] Updated weights for policy 0, policy_version 1934552 (0.0006) [2023-12-27 19:03:03,812][20381] Updated weights for policy 1, policy_version 1933085 (0.0008) [2023-12-27 19:03:03,881][20381] Updated weights for policy 1, policy_version 1933095 (0.0007) [2023-12-27 19:03:04,372][20347] Updated weights for policy 0, policy_version 1934562 (0.0010) [2023-12-27 19:03:04,430][20347] Updated weights for policy 0, policy_version 1934572 (0.0010) [2023-12-27 19:03:04,497][20347] Updated weights for policy 0, policy_version 1934582 (0.0010) [2023-12-27 19:03:04,567][20381] Updated weights for policy 1, policy_version 1933105 (0.0008) [2023-12-27 19:03:04,628][20381] Updated weights for policy 1, policy_version 1933115 (0.0007) [2023-12-27 19:03:04,689][20381] Updated weights for policy 1, policy_version 1933125 (0.0007) [2023-12-27 19:03:04,749][20381] Updated weights for policy 1, policy_version 1933135 (0.0008) [2023-12-27 19:03:05,160][20347] Updated weights for policy 0, policy_version 1934592 (0.0010) [2023-12-27 19:03:05,223][20347] Updated weights for policy 0, policy_version 1934602 (0.0010) [2023-12-27 19:03:05,289][20347] Updated weights for policy 0, policy_version 1934612 (0.0010) [2023-12-27 19:03:05,432][20381] Updated weights for policy 1, policy_version 1933145 (0.0007) [2023-12-27 19:03:05,495][20381] Updated weights for policy 1, policy_version 1933155 (0.0007) [2023-12-27 19:03:05,552][20381] Updated weights for policy 1, policy_version 1933165 (0.0007) [2023-12-27 19:03:05,924][20347] Updated weights for policy 0, policy_version 1934622 (0.0010) [2023-12-27 19:03:05,974][20347] Updated weights for policy 0, policy_version 1934632 (0.0010) [2023-12-27 19:03:06,023][20347] Updated weights for policy 0, policy_version 1934642 (0.0010) [2023-12-27 19:03:06,196][20381] Updated weights for policy 1, policy_version 1933175 (0.0006) [2023-12-27 19:03:06,248][20381] Updated weights for policy 1, policy_version 1933185 (0.0005) [2023-12-27 19:03:06,309][20381] Updated weights for policy 1, policy_version 1933195 (0.0007) [2023-12-27 19:03:06,699][20347] Updated weights for policy 0, policy_version 1934652 (0.0008) [2023-12-27 19:03:06,763][20347] Updated weights for policy 0, policy_version 1934662 (0.0005) [2023-12-27 19:03:06,822][20347] Updated weights for policy 0, policy_version 1934672 (0.0006) [2023-12-27 19:03:07,005][20381] Updated weights for policy 1, policy_version 1933205 (0.0008) [2023-12-27 19:03:07,057][20381] Updated weights for policy 1, policy_version 1933215 (0.0008) [2023-12-27 19:03:07,112][20381] Updated weights for policy 1, policy_version 1933225 (0.0009) [2023-12-27 19:03:07,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 990339072. Throughput: 0: 10335.4, 1: 10406.5. Samples: 990307512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.680')] [2023-12-27 19:03:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001934680_495370240.pth... [2023-12-27 19:03:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001933232_494968832.pth... [2023-12-27 19:03:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001932016_494657536.pth [2023-12-27 19:03:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001933496_495067136.pth [2023-12-27 19:03:07,406][20347] Updated weights for policy 0, policy_version 1934682 (0.0007) [2023-12-27 19:03:07,460][20347] Updated weights for policy 0, policy_version 1934692 (0.0006) [2023-12-27 19:03:07,515][20347] Updated weights for policy 0, policy_version 1934702 (0.0005) [2023-12-27 19:03:07,573][20347] Updated weights for policy 0, policy_version 1934712 (0.0005) [2023-12-27 19:03:07,826][20381] Updated weights for policy 1, policy_version 1933235 (0.0008) [2023-12-27 19:03:07,874][20381] Updated weights for policy 1, policy_version 1933245 (0.0007) [2023-12-27 19:03:07,926][20381] Updated weights for policy 1, policy_version 1933255 (0.0010) [2023-12-27 19:03:08,123][20347] Updated weights for policy 0, policy_version 1934722 (0.0005) [2023-12-27 19:03:08,182][20347] Updated weights for policy 0, policy_version 1934732 (0.0006) [2023-12-27 19:03:08,234][20347] Updated weights for policy 0, policy_version 1934742 (0.0008) [2023-12-27 19:03:08,647][20381] Updated weights for policy 1, policy_version 1933265 (0.0007) [2023-12-27 19:03:08,703][20381] Updated weights for policy 1, policy_version 1933275 (0.0010) [2023-12-27 19:03:08,763][20381] Updated weights for policy 1, policy_version 1933285 (0.0010) [2023-12-27 19:03:08,821][20381] Updated weights for policy 1, policy_version 1933295 (0.0007) [2023-12-27 19:03:08,900][20347] Updated weights for policy 0, policy_version 1934752 (0.0006) [2023-12-27 19:03:08,960][20347] Updated weights for policy 0, policy_version 1934762 (0.0005) [2023-12-27 19:03:09,014][20347] Updated weights for policy 0, policy_version 1934772 (0.0006) [2023-12-27 19:03:09,570][20381] Updated weights for policy 1, policy_version 1933305 (0.0011) [2023-12-27 19:03:09,633][20381] Updated weights for policy 1, policy_version 1933315 (0.0011) [2023-12-27 19:03:09,695][20381] Updated weights for policy 1, policy_version 1933325 (0.0011) [2023-12-27 19:03:09,722][20347] Updated weights for policy 0, policy_version 1934782 (0.0007) [2023-12-27 19:03:09,785][20347] Updated weights for policy 0, policy_version 1934792 (0.0011) [2023-12-27 19:03:09,852][20347] Updated weights for policy 0, policy_version 1934802 (0.0011) [2023-12-27 19:03:10,370][20381] Updated weights for policy 1, policy_version 1933335 (0.0011) [2023-12-27 19:03:10,430][20381] Updated weights for policy 1, policy_version 1933345 (0.0011) [2023-12-27 19:03:10,482][20381] Updated weights for policy 1, policy_version 1933355 (0.0011) [2023-12-27 19:03:10,559][20347] Updated weights for policy 0, policy_version 1934812 (0.0010) [2023-12-27 19:03:10,610][20347] Updated weights for policy 0, policy_version 1934822 (0.0009) [2023-12-27 19:03:10,661][20347] Updated weights for policy 0, policy_version 1934832 (0.0010) [2023-12-27 19:03:11,139][20381] Updated weights for policy 1, policy_version 1933365 (0.0010) [2023-12-27 19:03:11,198][20381] Updated weights for policy 1, policy_version 1933375 (0.0010) [2023-12-27 19:03:11,267][20381] Updated weights for policy 1, policy_version 1933385 (0.0011) [2023-12-27 19:03:11,413][20347] Updated weights for policy 0, policy_version 1934842 (0.0011) [2023-12-27 19:03:11,477][20347] Updated weights for policy 0, policy_version 1934852 (0.0010) [2023-12-27 19:03:11,533][20347] Updated weights for policy 0, policy_version 1934862 (0.0008) [2023-12-27 19:03:11,589][20347] Updated weights for policy 0, policy_version 1934872 (0.0010) [2023-12-27 19:03:11,978][20381] Updated weights for policy 1, policy_version 1933395 (0.0011) [2023-12-27 19:03:12,039][20381] Updated weights for policy 1, policy_version 1933405 (0.0011) [2023-12-27 19:03:12,103][20381] Updated weights for policy 1, policy_version 1933415 (0.0011) [2023-12-27 19:03:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 990437376. Throughput: 0: 10418.8, 1: 10390.6. Samples: 990430724. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.460')] [2023-12-27 19:03:12,398][20347] Updated weights for policy 0, policy_version 1934882 (0.0008) [2023-12-27 19:03:12,455][20347] Updated weights for policy 0, policy_version 1934892 (0.0007) [2023-12-27 19:03:12,516][20347] Updated weights for policy 0, policy_version 1934902 (0.0008) [2023-12-27 19:03:12,756][20381] Updated weights for policy 1, policy_version 1933425 (0.0010) [2023-12-27 19:03:12,811][20381] Updated weights for policy 1, policy_version 1933435 (0.0010) [2023-12-27 19:03:12,863][20381] Updated weights for policy 1, policy_version 1933445 (0.0010) [2023-12-27 19:03:12,924][20381] Updated weights for policy 1, policy_version 1933455 (0.0010) [2023-12-27 19:03:13,230][20347] Updated weights for policy 0, policy_version 1934912 (0.0007) [2023-12-27 19:03:13,289][20347] Updated weights for policy 0, policy_version 1934922 (0.0006) [2023-12-27 19:03:13,351][20347] Updated weights for policy 0, policy_version 1934932 (0.0006) [2023-12-27 19:03:13,541][20381] Updated weights for policy 1, policy_version 1933465 (0.0006) [2023-12-27 19:03:13,589][20381] Updated weights for policy 1, policy_version 1933475 (0.0005) [2023-12-27 19:03:13,641][20381] Updated weights for policy 1, policy_version 1933485 (0.0008) [2023-12-27 19:03:13,997][20347] Updated weights for policy 0, policy_version 1934942 (0.0005) [2023-12-27 19:03:14,054][20347] Updated weights for policy 0, policy_version 1934952 (0.0005) [2023-12-27 19:03:14,109][20347] Updated weights for policy 0, policy_version 1934962 (0.0005) [2023-12-27 19:03:14,169][20381] Updated weights for policy 1, policy_version 1933495 (0.0010) [2023-12-27 19:03:14,222][20381] Updated weights for policy 1, policy_version 1933505 (0.0010) [2023-12-27 19:03:14,270][20381] Updated weights for policy 1, policy_version 1933515 (0.0010) [2023-12-27 19:03:14,664][20347] Updated weights for policy 0, policy_version 1934972 (0.0007) [2023-12-27 19:03:14,725][20347] Updated weights for policy 0, policy_version 1934982 (0.0008) [2023-12-27 19:03:14,786][20347] Updated weights for policy 0, policy_version 1934992 (0.0009) [2023-12-27 19:03:14,955][20381] Updated weights for policy 1, policy_version 1933525 (0.0011) [2023-12-27 19:03:15,015][20381] Updated weights for policy 1, policy_version 1933535 (0.0011) [2023-12-27 19:03:15,076][20381] Updated weights for policy 1, policy_version 1933545 (0.0011) [2023-12-27 19:03:15,522][20347] Updated weights for policy 0, policy_version 1935002 (0.0008) [2023-12-27 19:03:15,588][20347] Updated weights for policy 0, policy_version 1935012 (0.0008) [2023-12-27 19:03:15,651][20347] Updated weights for policy 0, policy_version 1935022 (0.0007) [2023-12-27 19:03:15,707][20347] Updated weights for policy 0, policy_version 1935032 (0.0008) [2023-12-27 19:03:15,773][20381] Updated weights for policy 1, policy_version 1933555 (0.0009) [2023-12-27 19:03:15,832][20381] Updated weights for policy 1, policy_version 1933565 (0.0007) [2023-12-27 19:03:15,884][20381] Updated weights for policy 1, policy_version 1933575 (0.0007) [2023-12-27 19:03:16,421][20347] Updated weights for policy 0, policy_version 1935042 (0.0006) [2023-12-27 19:03:16,485][20347] Updated weights for policy 0, policy_version 1935052 (0.0006) [2023-12-27 19:03:16,541][20347] Updated weights for policy 0, policy_version 1935062 (0.0007) [2023-12-27 19:03:16,563][20381] Updated weights for policy 1, policy_version 1933585 (0.0007) [2023-12-27 19:03:16,614][20381] Updated weights for policy 1, policy_version 1933595 (0.0005) [2023-12-27 19:03:16,659][20381] Updated weights for policy 1, policy_version 1933605 (0.0005) [2023-12-27 19:03:16,709][20381] Updated weights for policy 1, policy_version 1933615 (0.0005) [2023-12-27 19:03:17,243][20347] Updated weights for policy 0, policy_version 1935072 (0.0006) [2023-12-27 19:03:17,285][20381] Updated weights for policy 1, policy_version 1933625 (0.0010) [2023-12-27 19:03:17,297][20347] Updated weights for policy 0, policy_version 1935082 (0.0005) [2023-12-27 19:03:17,337][20381] Updated weights for policy 1, policy_version 1933635 (0.0010) [2023-12-27 19:03:17,352][20347] Updated weights for policy 0, policy_version 1935092 (0.0006) [2023-12-27 19:03:17,362][19170] Fps is (10 sec: 19660.6, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 990535680. Throughput: 0: 10311.4, 1: 10430.6. Samples: 990554704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '870.300')] [2023-12-27 19:03:17,391][20381] Updated weights for policy 1, policy_version 1933645 (0.0010) [2023-12-27 19:03:18,031][20381] Updated weights for policy 1, policy_version 1933655 (0.0010) [2023-12-27 19:03:18,055][20347] Updated weights for policy 0, policy_version 1935102 (0.0006) [2023-12-27 19:03:18,090][20381] Updated weights for policy 1, policy_version 1933665 (0.0011) [2023-12-27 19:03:18,114][20347] Updated weights for policy 0, policy_version 1935112 (0.0005) [2023-12-27 19:03:18,144][20381] Updated weights for policy 1, policy_version 1933675 (0.0010) [2023-12-27 19:03:18,163][20347] Updated weights for policy 0, policy_version 1935122 (0.0006) [2023-12-27 19:03:18,819][20347] Updated weights for policy 0, policy_version 1935132 (0.0007) [2023-12-27 19:03:18,841][20381] Updated weights for policy 1, policy_version 1933685 (0.0011) [2023-12-27 19:03:18,879][20347] Updated weights for policy 0, policy_version 1935142 (0.0010) [2023-12-27 19:03:18,894][20381] Updated weights for policy 1, policy_version 1933695 (0.0011) [2023-12-27 19:03:18,939][20347] Updated weights for policy 0, policy_version 1935152 (0.0006) [2023-12-27 19:03:18,950][20381] Updated weights for policy 1, policy_version 1933705 (0.0011) [2023-12-27 19:03:19,618][20347] Updated weights for policy 0, policy_version 1935162 (0.0006) [2023-12-27 19:03:19,687][20347] Updated weights for policy 0, policy_version 1935172 (0.0008) [2023-12-27 19:03:19,740][20381] Updated weights for policy 1, policy_version 1933715 (0.0010) [2023-12-27 19:03:19,749][20347] Updated weights for policy 0, policy_version 1935182 (0.0007) [2023-12-27 19:03:19,800][20381] Updated weights for policy 1, policy_version 1933725 (0.0008) [2023-12-27 19:03:19,813][20347] Updated weights for policy 0, policy_version 1935192 (0.0008) [2023-12-27 19:03:19,868][20381] Updated weights for policy 1, policy_version 1933735 (0.0008) [2023-12-27 19:03:20,526][20347] Updated weights for policy 0, policy_version 1935202 (0.0006) [2023-12-27 19:03:20,589][20347] Updated weights for policy 0, policy_version 1935212 (0.0007) [2023-12-27 19:03:20,611][20381] Updated weights for policy 1, policy_version 1933745 (0.0007) [2023-12-27 19:03:20,649][20347] Updated weights for policy 0, policy_version 1935222 (0.0007) [2023-12-27 19:03:20,670][20381] Updated weights for policy 1, policy_version 1933755 (0.0008) [2023-12-27 19:03:20,728][20381] Updated weights for policy 1, policy_version 1933765 (0.0008) [2023-12-27 19:03:20,791][20381] Updated weights for policy 1, policy_version 1933775 (0.0009) [2023-12-27 19:03:21,266][20347] Updated weights for policy 0, policy_version 1935232 (0.0007) [2023-12-27 19:03:21,323][20347] Updated weights for policy 0, policy_version 1935242 (0.0008) [2023-12-27 19:03:21,390][20347] Updated weights for policy 0, policy_version 1935252 (0.0008) [2023-12-27 19:03:21,455][20381] Updated weights for policy 1, policy_version 1933785 (0.0009) [2023-12-27 19:03:21,509][20381] Updated weights for policy 1, policy_version 1933795 (0.0007) [2023-12-27 19:03:21,573][20381] Updated weights for policy 1, policy_version 1933805 (0.0008) [2023-12-27 19:03:22,097][20347] Updated weights for policy 0, policy_version 1935262 (0.0009) [2023-12-27 19:03:22,156][20347] Updated weights for policy 0, policy_version 1935272 (0.0008) [2023-12-27 19:03:22,174][20381] Updated weights for policy 1, policy_version 1933815 (0.0008) [2023-12-27 19:03:22,201][20347] Updated weights for policy 0, policy_version 1935282 (0.0005) [2023-12-27 19:03:22,235][20381] Updated weights for policy 1, policy_version 1933825 (0.0007) [2023-12-27 19:03:22,297][20381] Updated weights for policy 1, policy_version 1933835 (0.0008) [2023-12-27 19:03:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20889.5, 300 sec: 20493.9). Total num frames: 990650368. Throughput: 0: 10268.9, 1: 10374.1. Samples: 990614224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '870.870')] [2023-12-27 19:03:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001933840_495124480.pth... [2023-12-27 19:03:22,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001935288_495525888.pth... [2023-12-27 19:03:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001932624_494813184.pth [2023-12-27 19:03:22,377][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001934072_495214592.pth [2023-12-27 19:03:22,934][20347] Updated weights for policy 0, policy_version 1935292 (0.0006) [2023-12-27 19:03:22,981][20347] Updated weights for policy 0, policy_version 1935302 (0.0008) [2023-12-27 19:03:23,030][20381] Updated weights for policy 1, policy_version 1933845 (0.0007) [2023-12-27 19:03:23,043][20347] Updated weights for policy 0, policy_version 1935312 (0.0007) [2023-12-27 19:03:23,091][20381] Updated weights for policy 1, policy_version 1933855 (0.0006) [2023-12-27 19:03:23,141][20381] Updated weights for policy 1, policy_version 1933865 (0.0008) [2023-12-27 19:03:23,766][20381] Updated weights for policy 1, policy_version 1933875 (0.0008) [2023-12-27 19:03:23,767][20347] Updated weights for policy 0, policy_version 1935322 (0.0007) [2023-12-27 19:03:23,817][20381] Updated weights for policy 1, policy_version 1933885 (0.0010) [2023-12-27 19:03:23,819][20347] Updated weights for policy 0, policy_version 1935332 (0.0005) [2023-12-27 19:03:23,863][20381] Updated weights for policy 1, policy_version 1933895 (0.0010) [2023-12-27 19:03:23,873][20347] Updated weights for policy 0, policy_version 1935342 (0.0006) [2023-12-27 19:03:23,932][20347] Updated weights for policy 0, policy_version 1935352 (0.0006) [2023-12-27 19:03:24,466][20381] Updated weights for policy 1, policy_version 1933905 (0.0010) [2023-12-27 19:03:24,516][20381] Updated weights for policy 1, policy_version 1933915 (0.0008) [2023-12-27 19:03:24,570][20381] Updated weights for policy 1, policy_version 1933925 (0.0007) [2023-12-27 19:03:24,615][20381] Updated weights for policy 1, policy_version 1933935 (0.0007) [2023-12-27 19:03:24,655][20347] Updated weights for policy 0, policy_version 1935362 (0.0007) [2023-12-27 19:03:24,715][20347] Updated weights for policy 0, policy_version 1935372 (0.0007) [2023-12-27 19:03:24,773][20347] Updated weights for policy 0, policy_version 1935382 (0.0007) [2023-12-27 19:03:25,281][20381] Updated weights for policy 1, policy_version 1933945 (0.0008) [2023-12-27 19:03:25,336][20381] Updated weights for policy 1, policy_version 1933955 (0.0006) [2023-12-27 19:03:25,390][20381] Updated weights for policy 1, policy_version 1933965 (0.0005) [2023-12-27 19:03:25,471][20347] Updated weights for policy 0, policy_version 1935392 (0.0008) [2023-12-27 19:03:25,523][20347] Updated weights for policy 0, policy_version 1935402 (0.0009) [2023-12-27 19:03:25,575][20347] Updated weights for policy 0, policy_version 1935412 (0.0008) [2023-12-27 19:03:26,004][20381] Updated weights for policy 1, policy_version 1933975 (0.0008) [2023-12-27 19:03:26,061][20381] Updated weights for policy 1, policy_version 1933985 (0.0007) [2023-12-27 19:03:26,118][20381] Updated weights for policy 1, policy_version 1933995 (0.0005) [2023-12-27 19:03:26,255][20347] Updated weights for policy 0, policy_version 1935422 (0.0008) [2023-12-27 19:03:26,299][20347] Updated weights for policy 0, policy_version 1935432 (0.0008) [2023-12-27 19:03:26,347][20347] Updated weights for policy 0, policy_version 1935442 (0.0007) [2023-12-27 19:03:26,790][20381] Updated weights for policy 1, policy_version 1934005 (0.0005) [2023-12-27 19:03:26,846][20381] Updated weights for policy 1, policy_version 1934015 (0.0005) [2023-12-27 19:03:26,890][20381] Updated weights for policy 1, policy_version 1934025 (0.0006) [2023-12-27 19:03:26,986][20347] Updated weights for policy 0, policy_version 1935452 (0.0007) [2023-12-27 19:03:27,049][20347] Updated weights for policy 0, policy_version 1935462 (0.0007) [2023-12-27 19:03:27,097][20347] Updated weights for policy 0, policy_version 1935472 (0.0007) [2023-12-27 19:03:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20466.1). Total num frames: 990748672. Throughput: 0: 10207.6, 1: 10379.2. Samples: 990737420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.000')] [2023-12-27 19:03:27,540][20381] Updated weights for policy 1, policy_version 1934035 (0.0008) [2023-12-27 19:03:27,591][20381] Updated weights for policy 1, policy_version 1934045 (0.0008) [2023-12-27 19:03:27,643][20381] Updated weights for policy 1, policy_version 1934055 (0.0007) [2023-12-27 19:03:27,729][20347] Updated weights for policy 0, policy_version 1935482 (0.0007) [2023-12-27 19:03:27,776][20347] Updated weights for policy 0, policy_version 1935492 (0.0007) [2023-12-27 19:03:27,821][20347] Updated weights for policy 0, policy_version 1935502 (0.0007) [2023-12-27 19:03:27,868][20347] Updated weights for policy 0, policy_version 1935512 (0.0007) [2023-12-27 19:03:28,346][20381] Updated weights for policy 1, policy_version 1934066 (0.0009) [2023-12-27 19:03:28,404][20381] Updated weights for policy 1, policy_version 1934076 (0.0009) [2023-12-27 19:03:28,462][20381] Updated weights for policy 1, policy_version 1934086 (0.0008) [2023-12-27 19:03:28,519][20381] Updated weights for policy 1, policy_version 1934096 (0.0008) [2023-12-27 19:03:28,580][20347] Updated weights for policy 0, policy_version 1935522 (0.0008) [2023-12-27 19:03:28,637][20347] Updated weights for policy 0, policy_version 1935532 (0.0008) [2023-12-27 19:03:28,696][20347] Updated weights for policy 0, policy_version 1935542 (0.0009) [2023-12-27 19:03:29,202][20381] Updated weights for policy 1, policy_version 1934106 (0.0008) [2023-12-27 19:03:29,265][20381] Updated weights for policy 1, policy_version 1934116 (0.0008) [2023-12-27 19:03:29,325][20381] Updated weights for policy 1, policy_version 1934126 (0.0007) [2023-12-27 19:03:29,403][20347] Updated weights for policy 0, policy_version 1935552 (0.0008) [2023-12-27 19:03:29,466][20347] Updated weights for policy 0, policy_version 1935562 (0.0008) [2023-12-27 19:03:29,525][20347] Updated weights for policy 0, policy_version 1935572 (0.0008) [2023-12-27 19:03:30,022][20381] Updated weights for policy 1, policy_version 1934136 (0.0009) [2023-12-27 19:03:30,078][20381] Updated weights for policy 1, policy_version 1934146 (0.0005) [2023-12-27 19:03:30,134][20381] Updated weights for policy 1, policy_version 1934156 (0.0006) [2023-12-27 19:03:30,271][20347] Updated weights for policy 0, policy_version 1935582 (0.0007) [2023-12-27 19:03:30,335][20347] Updated weights for policy 0, policy_version 1935592 (0.0008) [2023-12-27 19:03:30,388][20347] Updated weights for policy 0, policy_version 1935602 (0.0007) [2023-12-27 19:03:30,718][20381] Updated weights for policy 1, policy_version 1934166 (0.0008) [2023-12-27 19:03:30,770][20381] Updated weights for policy 1, policy_version 1934176 (0.0008) [2023-12-27 19:03:30,824][20381] Updated weights for policy 1, policy_version 1934186 (0.0008) [2023-12-27 19:03:31,157][20347] Updated weights for policy 0, policy_version 1935612 (0.0009) [2023-12-27 19:03:31,217][20347] Updated weights for policy 0, policy_version 1935622 (0.0007) [2023-12-27 19:03:31,277][20347] Updated weights for policy 0, policy_version 1935632 (0.0008) [2023-12-27 19:03:31,584][20381] Updated weights for policy 1, policy_version 1934196 (0.0008) [2023-12-27 19:03:31,642][20381] Updated weights for policy 1, policy_version 1934206 (0.0007) [2023-12-27 19:03:31,707][20381] Updated weights for policy 1, policy_version 1934216 (0.0007) [2023-12-27 19:03:31,962][20347] Updated weights for policy 0, policy_version 1935642 (0.0007) [2023-12-27 19:03:32,019][20347] Updated weights for policy 0, policy_version 1935652 (0.0007) [2023-12-27 19:03:32,066][20347] Updated weights for policy 0, policy_version 1935662 (0.0008) [2023-12-27 19:03:32,118][20347] Updated weights for policy 0, policy_version 1935672 (0.0008) [2023-12-27 19:03:32,362][19170] Fps is (10 sec: 19661.3, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 990846976. Throughput: 0: 10211.2, 1: 10312.0. Samples: 990859156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '920.830')] [2023-12-27 19:03:32,379][20381] Updated weights for policy 1, policy_version 1934226 (0.0007) [2023-12-27 19:03:32,429][20381] Updated weights for policy 1, policy_version 1934236 (0.0005) [2023-12-27 19:03:32,487][20381] Updated weights for policy 1, policy_version 1934246 (0.0007) [2023-12-27 19:03:32,545][20381] Updated weights for policy 1, policy_version 1934256 (0.0007) [2023-12-27 19:03:32,815][20347] Updated weights for policy 0, policy_version 1935682 (0.0006) [2023-12-27 19:03:32,873][20347] Updated weights for policy 0, policy_version 1935692 (0.0006) [2023-12-27 19:03:32,938][20347] Updated weights for policy 0, policy_version 1935702 (0.0005) [2023-12-27 19:03:33,154][20381] Updated weights for policy 1, policy_version 1934266 (0.0006) [2023-12-27 19:03:33,208][20381] Updated weights for policy 1, policy_version 1934276 (0.0006) [2023-12-27 19:03:33,257][20381] Updated weights for policy 1, policy_version 1934286 (0.0009) [2023-12-27 19:03:33,504][20347] Updated weights for policy 0, policy_version 1935712 (0.0006) [2023-12-27 19:03:33,561][20347] Updated weights for policy 0, policy_version 1935722 (0.0005) [2023-12-27 19:03:33,620][20347] Updated weights for policy 0, policy_version 1935732 (0.0005) [2023-12-27 19:03:33,912][20381] Updated weights for policy 1, policy_version 1934296 (0.0010) [2023-12-27 19:03:33,961][20381] Updated weights for policy 1, policy_version 1934306 (0.0010) [2023-12-27 19:03:34,017][20381] Updated weights for policy 1, policy_version 1934317 (0.0008) [2023-12-27 19:03:34,230][20347] Updated weights for policy 0, policy_version 1935742 (0.0008) [2023-12-27 19:03:34,282][20347] Updated weights for policy 0, policy_version 1935752 (0.0010) [2023-12-27 19:03:34,335][20347] Updated weights for policy 0, policy_version 1935762 (0.0010) [2023-12-27 19:03:34,708][20381] Updated weights for policy 1, policy_version 1934327 (0.0008) [2023-12-27 19:03:34,769][20381] Updated weights for policy 1, policy_version 1934337 (0.0008) [2023-12-27 19:03:34,830][20381] Updated weights for policy 1, policy_version 1934347 (0.0005) [2023-12-27 19:03:35,024][20347] Updated weights for policy 0, policy_version 1935772 (0.0009) [2023-12-27 19:03:35,073][20347] Updated weights for policy 0, policy_version 1935782 (0.0008) [2023-12-27 19:03:35,120][20347] Updated weights for policy 0, policy_version 1935792 (0.0006) [2023-12-27 19:03:35,455][20381] Updated weights for policy 1, policy_version 1934357 (0.0006) [2023-12-27 19:03:35,504][20381] Updated weights for policy 1, policy_version 1934367 (0.0008) [2023-12-27 19:03:35,553][20381] Updated weights for policy 1, policy_version 1934377 (0.0007) [2023-12-27 19:03:35,704][20347] Updated weights for policy 0, policy_version 1935802 (0.0006) [2023-12-27 19:03:35,756][20347] Updated weights for policy 0, policy_version 1935812 (0.0010) [2023-12-27 19:03:35,816][20347] Updated weights for policy 0, policy_version 1935822 (0.0011) [2023-12-27 19:03:35,876][20347] Updated weights for policy 0, policy_version 1935832 (0.0008) [2023-12-27 19:03:36,219][20381] Updated weights for policy 1, policy_version 1934387 (0.0007) [2023-12-27 19:03:36,273][20381] Updated weights for policy 1, policy_version 1934397 (0.0007) [2023-12-27 19:03:36,339][20381] Updated weights for policy 1, policy_version 1934407 (0.0008) [2023-12-27 19:03:36,582][20347] Updated weights for policy 0, policy_version 1935842 (0.0011) [2023-12-27 19:03:36,635][20347] Updated weights for policy 0, policy_version 1935852 (0.0011) [2023-12-27 19:03:36,686][20347] Updated weights for policy 0, policy_version 1935862 (0.0009) [2023-12-27 19:03:37,067][20381] Updated weights for policy 1, policy_version 1934417 (0.0008) [2023-12-27 19:03:37,124][20381] Updated weights for policy 1, policy_version 1934427 (0.0007) [2023-12-27 19:03:37,180][20381] Updated weights for policy 1, policy_version 1934437 (0.0006) [2023-12-27 19:03:37,240][20381] Updated weights for policy 1, policy_version 1934447 (0.0006) [2023-12-27 19:03:37,338][20347] Updated weights for policy 0, policy_version 1935872 (0.0010) [2023-12-27 19:03:37,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20493.9). Total num frames: 990953472. Throughput: 0: 10251.3, 1: 10351.3. Samples: 990923324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.550')] [2023-12-27 19:03:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001934448_495280128.pth... [2023-12-27 19:03:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001933232_494968832.pth [2023-12-27 19:03:37,388][20347] Updated weights for policy 0, policy_version 1935882 (0.0010) [2023-12-27 19:03:37,444][20347] Updated weights for policy 0, policy_version 1935892 (0.0011) [2023-12-27 19:03:37,467][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001935896_495681536.pth... [2023-12-27 19:03:37,471][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001934680_495370240.pth [2023-12-27 19:03:37,863][20381] Updated weights for policy 1, policy_version 1934457 (0.0007) [2023-12-27 19:03:37,910][20381] Updated weights for policy 1, policy_version 1934467 (0.0007) [2023-12-27 19:03:37,963][20381] Updated weights for policy 1, policy_version 1934477 (0.0007) [2023-12-27 19:03:38,121][20347] Updated weights for policy 0, policy_version 1935902 (0.0010) [2023-12-27 19:03:38,185][20347] Updated weights for policy 0, policy_version 1935912 (0.0010) [2023-12-27 19:03:38,240][20347] Updated weights for policy 0, policy_version 1935922 (0.0007) [2023-12-27 19:03:38,670][20381] Updated weights for policy 1, policy_version 1934487 (0.0006) [2023-12-27 19:03:38,732][20381] Updated weights for policy 1, policy_version 1934497 (0.0005) [2023-12-27 19:03:38,788][20381] Updated weights for policy 1, policy_version 1934507 (0.0005) [2023-12-27 19:03:38,924][20347] Updated weights for policy 0, policy_version 1935932 (0.0009) [2023-12-27 19:03:38,983][20347] Updated weights for policy 0, policy_version 1935942 (0.0008) [2023-12-27 19:03:39,038][20347] Updated weights for policy 0, policy_version 1935952 (0.0008) [2023-12-27 19:03:39,361][20381] Updated weights for policy 1, policy_version 1934517 (0.0008) [2023-12-27 19:03:39,429][20381] Updated weights for policy 1, policy_version 1934527 (0.0008) [2023-12-27 19:03:39,492][20381] Updated weights for policy 1, policy_version 1934537 (0.0007) [2023-12-27 19:03:39,741][20347] Updated weights for policy 0, policy_version 1935962 (0.0008) [2023-12-27 19:03:39,802][20347] Updated weights for policy 0, policy_version 1935972 (0.0008) [2023-12-27 19:03:39,866][20347] Updated weights for policy 0, policy_version 1935982 (0.0007) [2023-12-27 19:03:39,924][20347] Updated weights for policy 0, policy_version 1935992 (0.0008) [2023-12-27 19:03:40,178][20381] Updated weights for policy 1, policy_version 1934547 (0.0008) [2023-12-27 19:03:40,236][20381] Updated weights for policy 1, policy_version 1934557 (0.0008) [2023-12-27 19:03:40,295][20381] Updated weights for policy 1, policy_version 1934567 (0.0006) [2023-12-27 19:03:40,565][20347] Updated weights for policy 0, policy_version 1936002 (0.0005) [2023-12-27 19:03:40,617][20347] Updated weights for policy 0, policy_version 1936012 (0.0006) [2023-12-27 19:03:40,671][20347] Updated weights for policy 0, policy_version 1936022 (0.0005) [2023-12-27 19:03:41,043][20381] Updated weights for policy 1, policy_version 1934577 (0.0007) [2023-12-27 19:03:41,106][20381] Updated weights for policy 1, policy_version 1934587 (0.0007) [2023-12-27 19:03:41,172][20381] Updated weights for policy 1, policy_version 1934597 (0.0008) [2023-12-27 19:03:41,241][20381] Updated weights for policy 1, policy_version 1934607 (0.0008) [2023-12-27 19:03:41,260][20347] Updated weights for policy 0, policy_version 1936032 (0.0006) [2023-12-27 19:03:41,332][20347] Updated weights for policy 0, policy_version 1936042 (0.0008) [2023-12-27 19:03:41,402][20347] Updated weights for policy 0, policy_version 1936052 (0.0007) [2023-12-27 19:03:41,965][20381] Updated weights for policy 1, policy_version 1934617 (0.0006) [2023-12-27 19:03:42,019][20381] Updated weights for policy 1, policy_version 1934627 (0.0007) [2023-12-27 19:03:42,067][20381] Updated weights for policy 1, policy_version 1934637 (0.0005) [2023-12-27 19:03:42,128][20347] Updated weights for policy 0, policy_version 1936062 (0.0008) [2023-12-27 19:03:42,186][20347] Updated weights for policy 0, policy_version 1936072 (0.0009) [2023-12-27 19:03:42,250][20347] Updated weights for policy 0, policy_version 1936082 (0.0008) [2023-12-27 19:03:42,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20466.1). Total num frames: 991059968. Throughput: 0: 10249.5, 1: 10352.5. Samples: 991047176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '889.540')] [2023-12-27 19:03:42,712][20381] Updated weights for policy 1, policy_version 1934647 (0.0007) [2023-12-27 19:03:42,773][20381] Updated weights for policy 1, policy_version 1934657 (0.0006) [2023-12-27 19:03:42,835][20381] Updated weights for policy 1, policy_version 1934667 (0.0007) [2023-12-27 19:03:42,968][20347] Updated weights for policy 0, policy_version 1936092 (0.0007) [2023-12-27 19:03:43,020][20347] Updated weights for policy 0, policy_version 1936102 (0.0005) [2023-12-27 19:03:43,069][20347] Updated weights for policy 0, policy_version 1936112 (0.0008) [2023-12-27 19:03:43,445][20381] Updated weights for policy 1, policy_version 1934677 (0.0007) [2023-12-27 19:03:43,498][20381] Updated weights for policy 1, policy_version 1934687 (0.0005) [2023-12-27 19:03:43,550][20381] Updated weights for policy 1, policy_version 1934697 (0.0005) [2023-12-27 19:03:43,652][20347] Updated weights for policy 0, policy_version 1936122 (0.0007) [2023-12-27 19:03:43,700][20347] Updated weights for policy 0, policy_version 1936132 (0.0005) [2023-12-27 19:03:43,747][20347] Updated weights for policy 0, policy_version 1936142 (0.0007) [2023-12-27 19:03:43,799][20347] Updated weights for policy 0, policy_version 1936152 (0.0006) [2023-12-27 19:03:44,188][20381] Updated weights for policy 1, policy_version 1934707 (0.0007) [2023-12-27 19:03:44,239][20381] Updated weights for policy 1, policy_version 1934717 (0.0007) [2023-12-27 19:03:44,299][20381] Updated weights for policy 1, policy_version 1934727 (0.0005) [2023-12-27 19:03:44,470][20347] Updated weights for policy 0, policy_version 1936162 (0.0007) [2023-12-27 19:03:44,529][20347] Updated weights for policy 0, policy_version 1936172 (0.0005) [2023-12-27 19:03:44,589][20347] Updated weights for policy 0, policy_version 1936182 (0.0008) [2023-12-27 19:03:44,920][20381] Updated weights for policy 1, policy_version 1934737 (0.0005) [2023-12-27 19:03:44,984][20381] Updated weights for policy 1, policy_version 1934747 (0.0007) [2023-12-27 19:03:45,041][20381] Updated weights for policy 1, policy_version 1934757 (0.0008) [2023-12-27 19:03:45,098][20381] Updated weights for policy 1, policy_version 1934767 (0.0008) [2023-12-27 19:03:45,234][20347] Updated weights for policy 0, policy_version 1936192 (0.0008) [2023-12-27 19:03:45,287][20347] Updated weights for policy 0, policy_version 1936202 (0.0008) [2023-12-27 19:03:45,335][20347] Updated weights for policy 0, policy_version 1936212 (0.0007) [2023-12-27 19:03:45,760][20381] Updated weights for policy 1, policy_version 1934777 (0.0008) [2023-12-27 19:03:45,819][20381] Updated weights for policy 1, policy_version 1934787 (0.0008) [2023-12-27 19:03:45,883][20381] Updated weights for policy 1, policy_version 1934797 (0.0008) [2023-12-27 19:03:46,092][20347] Updated weights for policy 0, policy_version 1936222 (0.0007) [2023-12-27 19:03:46,157][20347] Updated weights for policy 0, policy_version 1936232 (0.0008) [2023-12-27 19:03:46,223][20347] Updated weights for policy 0, policy_version 1936242 (0.0007) [2023-12-27 19:03:46,565][20381] Updated weights for policy 1, policy_version 1934807 (0.0007) [2023-12-27 19:03:46,618][20381] Updated weights for policy 1, policy_version 1934817 (0.0009) [2023-12-27 19:03:46,673][20381] Updated weights for policy 1, policy_version 1934827 (0.0006) [2023-12-27 19:03:46,841][20347] Updated weights for policy 0, policy_version 1936252 (0.0007) [2023-12-27 19:03:46,900][20347] Updated weights for policy 0, policy_version 1936262 (0.0006) [2023-12-27 19:03:46,953][20347] Updated weights for policy 0, policy_version 1936272 (0.0007) [2023-12-27 19:03:47,312][20381] Updated weights for policy 1, policy_version 1934837 (0.0009) [2023-12-27 19:03:47,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20466.1). Total num frames: 991158272. Throughput: 0: 10265.1, 1: 10370.7. Samples: 991171544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '917.870')] [2023-12-27 19:03:47,375][20381] Updated weights for policy 1, policy_version 1934847 (0.0011) [2023-12-27 19:03:47,423][20381] Updated weights for policy 1, policy_version 1934857 (0.0010) [2023-12-27 19:03:47,555][20347] Updated weights for policy 0, policy_version 1936282 (0.0007) [2023-12-27 19:03:47,612][20347] Updated weights for policy 0, policy_version 1936292 (0.0005) [2023-12-27 19:03:47,658][20347] Updated weights for policy 0, policy_version 1936302 (0.0005) [2023-12-27 19:03:47,719][20347] Updated weights for policy 0, policy_version 1936312 (0.0006) [2023-12-27 19:03:48,037][20381] Updated weights for policy 1, policy_version 1934867 (0.0009) [2023-12-27 19:03:48,082][20381] Updated weights for policy 1, policy_version 1934877 (0.0008) [2023-12-27 19:03:48,128][20381] Updated weights for policy 1, policy_version 1934887 (0.0007) [2023-12-27 19:03:48,413][20347] Updated weights for policy 0, policy_version 1936322 (0.0009) [2023-12-27 19:03:48,476][20347] Updated weights for policy 0, policy_version 1936332 (0.0008) [2023-12-27 19:03:48,535][20347] Updated weights for policy 0, policy_version 1936342 (0.0007) [2023-12-27 19:03:48,830][20381] Updated weights for policy 1, policy_version 1934897 (0.0007) [2023-12-27 19:03:48,890][20381] Updated weights for policy 1, policy_version 1934907 (0.0006) [2023-12-27 19:03:48,949][20381] Updated weights for policy 1, policy_version 1934917 (0.0005) [2023-12-27 19:03:49,013][20381] Updated weights for policy 1, policy_version 1934927 (0.0006) [2023-12-27 19:03:49,265][20347] Updated weights for policy 0, policy_version 1936352 (0.0007) [2023-12-27 19:03:49,327][20347] Updated weights for policy 0, policy_version 1936362 (0.0005) [2023-12-27 19:03:49,397][20347] Updated weights for policy 0, policy_version 1936372 (0.0008) [2023-12-27 19:03:49,626][20381] Updated weights for policy 1, policy_version 1934937 (0.0008) [2023-12-27 19:03:49,681][20381] Updated weights for policy 1, policy_version 1934947 (0.0006) [2023-12-27 19:03:49,733][20381] Updated weights for policy 1, policy_version 1934957 (0.0008) [2023-12-27 19:03:50,054][20347] Updated weights for policy 0, policy_version 1936382 (0.0008) [2023-12-27 19:03:50,106][20347] Updated weights for policy 0, policy_version 1936392 (0.0008) [2023-12-27 19:03:50,166][20347] Updated weights for policy 0, policy_version 1936402 (0.0008) [2023-12-27 19:03:50,474][20381] Updated weights for policy 1, policy_version 1934967 (0.0008) [2023-12-27 19:03:50,534][20381] Updated weights for policy 1, policy_version 1934977 (0.0007) [2023-12-27 19:03:50,600][20381] Updated weights for policy 1, policy_version 1934987 (0.0008) [2023-12-27 19:03:50,957][20347] Updated weights for policy 0, policy_version 1936412 (0.0008) [2023-12-27 19:03:51,012][20347] Updated weights for policy 0, policy_version 1936422 (0.0007) [2023-12-27 19:03:51,084][20347] Updated weights for policy 0, policy_version 1936432 (0.0006) [2023-12-27 19:03:51,379][20381] Updated weights for policy 1, policy_version 1934997 (0.0009) [2023-12-27 19:03:51,443][20381] Updated weights for policy 1, policy_version 1935007 (0.0007) [2023-12-27 19:03:51,503][20381] Updated weights for policy 1, policy_version 1935017 (0.0009) [2023-12-27 19:03:51,791][20347] Updated weights for policy 0, policy_version 1936442 (0.0007) [2023-12-27 19:03:51,847][20347] Updated weights for policy 0, policy_version 1936452 (0.0008) [2023-12-27 19:03:51,904][20347] Updated weights for policy 0, policy_version 1936462 (0.0007) [2023-12-27 19:03:51,959][20347] Updated weights for policy 0, policy_version 1936472 (0.0007) [2023-12-27 19:03:52,208][20381] Updated weights for policy 1, policy_version 1935027 (0.0008) [2023-12-27 19:03:52,273][20381] Updated weights for policy 1, policy_version 1935037 (0.0010) [2023-12-27 19:03:52,338][20381] Updated weights for policy 1, policy_version 1935047 (0.0009) [2023-12-27 19:03:52,363][19170] Fps is (10 sec: 19660.4, 60 sec: 20480.0, 300 sec: 20438.3). Total num frames: 991256576. Throughput: 0: 10225.4, 1: 10362.9. Samples: 991233988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '928.320')] [2023-12-27 19:03:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001936472_495828992.pth... [2023-12-27 19:03:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001935288_495525888.pth [2023-12-27 19:03:52,399][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001935056_495435776.pth... [2023-12-27 19:03:52,404][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001933840_495124480.pth [2023-12-27 19:03:52,676][20347] Updated weights for policy 0, policy_version 1936482 (0.0008) [2023-12-27 19:03:52,735][20347] Updated weights for policy 0, policy_version 1936492 (0.0008) [2023-12-27 19:03:52,793][20347] Updated weights for policy 0, policy_version 1936502 (0.0007) [2023-12-27 19:03:53,010][20381] Updated weights for policy 1, policy_version 1935057 (0.0010) [2023-12-27 19:03:53,059][20381] Updated weights for policy 1, policy_version 1935067 (0.0011) [2023-12-27 19:03:53,108][20381] Updated weights for policy 1, policy_version 1935077 (0.0011) [2023-12-27 19:03:53,154][20381] Updated weights for policy 1, policy_version 1935087 (0.0010) [2023-12-27 19:03:53,469][20347] Updated weights for policy 0, policy_version 1936512 (0.0010) [2023-12-27 19:03:53,529][20347] Updated weights for policy 0, policy_version 1936522 (0.0010) [2023-12-27 19:03:53,588][20347] Updated weights for policy 0, policy_version 1936532 (0.0010) [2023-12-27 19:03:53,754][20381] Updated weights for policy 1, policy_version 1935097 (0.0006) [2023-12-27 19:03:53,806][20381] Updated weights for policy 1, policy_version 1935107 (0.0007) [2023-12-27 19:03:53,858][20381] Updated weights for policy 1, policy_version 1935117 (0.0010) [2023-12-27 19:03:54,205][20347] Updated weights for policy 0, policy_version 1936542 (0.0009) [2023-12-27 19:03:54,256][20347] Updated weights for policy 0, policy_version 1936552 (0.0007) [2023-12-27 19:03:54,310][20347] Updated weights for policy 0, policy_version 1936562 (0.0006) [2023-12-27 19:03:54,527][20381] Updated weights for policy 1, policy_version 1935127 (0.0008) [2023-12-27 19:03:54,584][20381] Updated weights for policy 1, policy_version 1935137 (0.0008) [2023-12-27 19:03:54,643][20381] Updated weights for policy 1, policy_version 1935147 (0.0008) [2023-12-27 19:03:54,930][20347] Updated weights for policy 0, policy_version 1936572 (0.0008) [2023-12-27 19:03:54,994][20347] Updated weights for policy 0, policy_version 1936582 (0.0007) [2023-12-27 19:03:55,061][20347] Updated weights for policy 0, policy_version 1936592 (0.0007) [2023-12-27 19:03:55,260][20381] Updated weights for policy 1, policy_version 1935157 (0.0008) [2023-12-27 19:03:55,319][20381] Updated weights for policy 1, policy_version 1935167 (0.0010) [2023-12-27 19:03:55,380][20381] Updated weights for policy 1, policy_version 1935177 (0.0011) [2023-12-27 19:03:55,618][20347] Updated weights for policy 0, policy_version 1936602 (0.0006) [2023-12-27 19:03:55,679][20347] Updated weights for policy 0, policy_version 1936612 (0.0006) [2023-12-27 19:03:55,729][20347] Updated weights for policy 0, policy_version 1936622 (0.0005) [2023-12-27 19:03:55,780][20347] Updated weights for policy 0, policy_version 1936632 (0.0005) [2023-12-27 19:03:55,941][20381] Updated weights for policy 1, policy_version 1935187 (0.0010) [2023-12-27 19:03:55,992][20381] Updated weights for policy 1, policy_version 1935197 (0.0007) [2023-12-27 19:03:56,045][20381] Updated weights for policy 1, policy_version 1935207 (0.0008) [2023-12-27 19:03:56,349][20347] Updated weights for policy 0, policy_version 1936642 (0.0006) [2023-12-27 19:03:56,408][20347] Updated weights for policy 0, policy_version 1936652 (0.0007) [2023-12-27 19:03:56,470][20347] Updated weights for policy 0, policy_version 1936662 (0.0005) [2023-12-27 19:03:56,649][20381] Updated weights for policy 1, policy_version 1935217 (0.0007) [2023-12-27 19:03:56,707][20381] Updated weights for policy 1, policy_version 1935227 (0.0007) [2023-12-27 19:03:56,752][20381] Updated weights for policy 1, policy_version 1935237 (0.0008) [2023-12-27 19:03:56,801][20381] Updated weights for policy 1, policy_version 1935247 (0.0009) [2023-12-27 19:03:56,982][20347] Updated weights for policy 0, policy_version 1936672 (0.0005) [2023-12-27 19:03:57,038][20347] Updated weights for policy 0, policy_version 1936682 (0.0005) [2023-12-27 19:03:57,102][20347] Updated weights for policy 0, policy_version 1936692 (0.0005) [2023-12-27 19:03:57,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 991371264. Throughput: 0: 10206.0, 1: 10410.1. Samples: 991358444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:03:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '869.160')] [2023-12-27 19:03:57,520][20381] Updated weights for policy 1, policy_version 1935257 (0.0006) [2023-12-27 19:03:57,579][20381] Updated weights for policy 1, policy_version 1935267 (0.0010) [2023-12-27 19:03:57,640][20381] Updated weights for policy 1, policy_version 1935277 (0.0008) [2023-12-27 19:03:57,733][20347] Updated weights for policy 0, policy_version 1936702 (0.0006) [2023-12-27 19:03:57,790][20347] Updated weights for policy 0, policy_version 1936712 (0.0005) [2023-12-27 19:03:57,851][20347] Updated weights for policy 0, policy_version 1936722 (0.0005) [2023-12-27 19:03:58,282][20381] Updated weights for policy 1, policy_version 1935287 (0.0010) [2023-12-27 19:03:58,358][20381] Updated weights for policy 1, policy_version 1935297 (0.0010) [2023-12-27 19:03:58,417][20381] Updated weights for policy 1, policy_version 1935307 (0.0010) [2023-12-27 19:03:58,445][20347] Updated weights for policy 0, policy_version 1936732 (0.0006) [2023-12-27 19:03:58,508][20347] Updated weights for policy 0, policy_version 1936742 (0.0008) [2023-12-27 19:03:58,569][20347] Updated weights for policy 0, policy_version 1936752 (0.0008) [2023-12-27 19:03:59,199][20381] Updated weights for policy 1, policy_version 1935317 (0.0008) [2023-12-27 19:03:59,268][20347] Updated weights for policy 0, policy_version 1936762 (0.0007) [2023-12-27 19:03:59,270][20381] Updated weights for policy 1, policy_version 1935327 (0.0008) [2023-12-27 19:03:59,321][20347] Updated weights for policy 0, policy_version 1936772 (0.0008) [2023-12-27 19:03:59,329][20381] Updated weights for policy 1, policy_version 1935337 (0.0008) [2023-12-27 19:03:59,383][20347] Updated weights for policy 0, policy_version 1936782 (0.0008) [2023-12-27 19:03:59,442][20347] Updated weights for policy 0, policy_version 1936792 (0.0008) [2023-12-27 19:04:00,030][20381] Updated weights for policy 1, policy_version 1935347 (0.0007) [2023-12-27 19:04:00,087][20381] Updated weights for policy 1, policy_version 1935357 (0.0005) [2023-12-27 19:04:00,148][20381] Updated weights for policy 1, policy_version 1935367 (0.0007) [2023-12-27 19:04:00,211][20347] Updated weights for policy 0, policy_version 1936802 (0.0006) [2023-12-27 19:04:00,271][20347] Updated weights for policy 0, policy_version 1936812 (0.0008) [2023-12-27 19:04:00,330][20347] Updated weights for policy 0, policy_version 1936822 (0.0008) [2023-12-27 19:04:00,808][20381] Updated weights for policy 1, policy_version 1935377 (0.0008) [2023-12-27 19:04:00,876][20381] Updated weights for policy 1, policy_version 1935387 (0.0006) [2023-12-27 19:04:00,915][20347] Updated weights for policy 0, policy_version 1936832 (0.0007) [2023-12-27 19:04:00,945][20381] Updated weights for policy 1, policy_version 1935397 (0.0010) [2023-12-27 19:04:00,972][20347] Updated weights for policy 0, policy_version 1936842 (0.0006) [2023-12-27 19:04:00,996][20381] Updated weights for policy 1, policy_version 1935407 (0.0008) [2023-12-27 19:04:01,031][20347] Updated weights for policy 0, policy_version 1936852 (0.0006) [2023-12-27 19:04:01,620][20381] Updated weights for policy 1, policy_version 1935417 (0.0007) [2023-12-27 19:04:01,685][20381] Updated weights for policy 1, policy_version 1935427 (0.0008) [2023-12-27 19:04:01,753][20381] Updated weights for policy 1, policy_version 1935437 (0.0007) [2023-12-27 19:04:01,802][20347] Updated weights for policy 0, policy_version 1936862 (0.0007) [2023-12-27 19:04:01,859][20347] Updated weights for policy 0, policy_version 1936872 (0.0008) [2023-12-27 19:04:01,915][20347] Updated weights for policy 0, policy_version 1936882 (0.0006) [2023-12-27 19:04:02,362][19170] Fps is (10 sec: 21299.6, 60 sec: 20616.5, 300 sec: 20493.9). Total num frames: 991469568. Throughput: 0: 10286.8, 1: 10329.1. Samples: 991482416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.220')] [2023-12-27 19:04:02,462][20381] Updated weights for policy 1, policy_version 1935447 (0.0007) [2023-12-27 19:04:02,518][20381] Updated weights for policy 1, policy_version 1935457 (0.0008) [2023-12-27 19:04:02,579][20381] Updated weights for policy 1, policy_version 1935467 (0.0006) [2023-12-27 19:04:02,627][20347] Updated weights for policy 0, policy_version 1936892 (0.0008) [2023-12-27 19:04:02,686][20347] Updated weights for policy 0, policy_version 1936902 (0.0007) [2023-12-27 19:04:02,740][20347] Updated weights for policy 0, policy_version 1936912 (0.0008) [2023-12-27 19:04:03,165][20381] Updated weights for policy 1, policy_version 1935477 (0.0007) [2023-12-27 19:04:03,214][20381] Updated weights for policy 1, policy_version 1935487 (0.0009) [2023-12-27 19:04:03,260][20381] Updated weights for policy 1, policy_version 1935497 (0.0009) [2023-12-27 19:04:03,367][20347] Updated weights for policy 0, policy_version 1936922 (0.0007) [2023-12-27 19:04:03,420][20347] Updated weights for policy 0, policy_version 1936932 (0.0007) [2023-12-27 19:04:03,478][20347] Updated weights for policy 0, policy_version 1936942 (0.0007) [2023-12-27 19:04:03,529][20347] Updated weights for policy 0, policy_version 1936952 (0.0008) [2023-12-27 19:04:03,943][20381] Updated weights for policy 1, policy_version 1935507 (0.0009) [2023-12-27 19:04:04,003][20381] Updated weights for policy 1, policy_version 1935517 (0.0008) [2023-12-27 19:04:04,057][20381] Updated weights for policy 1, policy_version 1935527 (0.0008) [2023-12-27 19:04:04,196][20347] Updated weights for policy 0, policy_version 1936962 (0.0006) [2023-12-27 19:04:04,256][20347] Updated weights for policy 0, policy_version 1936972 (0.0007) [2023-12-27 19:04:04,313][20347] Updated weights for policy 0, policy_version 1936982 (0.0007) [2023-12-27 19:04:04,786][20381] Updated weights for policy 1, policy_version 1935537 (0.0008) [2023-12-27 19:04:04,836][20381] Updated weights for policy 1, policy_version 1935547 (0.0007) [2023-12-27 19:04:04,896][20381] Updated weights for policy 1, policy_version 1935557 (0.0008) [2023-12-27 19:04:04,904][20347] Updated weights for policy 0, policy_version 1936992 (0.0008) [2023-12-27 19:04:04,953][20381] Updated weights for policy 1, policy_version 1935567 (0.0010) [2023-12-27 19:04:04,960][20347] Updated weights for policy 0, policy_version 1937002 (0.0005) [2023-12-27 19:04:05,014][20347] Updated weights for policy 0, policy_version 1937012 (0.0006) [2023-12-27 19:04:05,549][20381] Updated weights for policy 1, policy_version 1935577 (0.0011) [2023-12-27 19:04:05,564][20347] Updated weights for policy 0, policy_version 1937022 (0.0005) [2023-12-27 19:04:05,605][20381] Updated weights for policy 1, policy_version 1935587 (0.0011) [2023-12-27 19:04:05,620][20347] Updated weights for policy 0, policy_version 1937032 (0.0009) [2023-12-27 19:04:05,653][20381] Updated weights for policy 1, policy_version 1935597 (0.0010) [2023-12-27 19:04:05,672][20347] Updated weights for policy 0, policy_version 1937042 (0.0010) [2023-12-27 19:04:06,271][20381] Updated weights for policy 1, policy_version 1935607 (0.0011) [2023-12-27 19:04:06,335][20381] Updated weights for policy 1, policy_version 1935617 (0.0011) [2023-12-27 19:04:06,356][20347] Updated weights for policy 0, policy_version 1937052 (0.0008) [2023-12-27 19:04:06,390][20381] Updated weights for policy 1, policy_version 1935627 (0.0010) [2023-12-27 19:04:06,417][20347] Updated weights for policy 0, policy_version 1937062 (0.0006) [2023-12-27 19:04:06,477][20347] Updated weights for policy 0, policy_version 1937072 (0.0008) [2023-12-27 19:04:07,101][20381] Updated weights for policy 1, policy_version 1935637 (0.0007) [2023-12-27 19:04:07,115][20347] Updated weights for policy 0, policy_version 1937082 (0.0008) [2023-12-27 19:04:07,151][20381] Updated weights for policy 1, policy_version 1935647 (0.0007) [2023-12-27 19:04:07,172][20347] Updated weights for policy 0, policy_version 1937092 (0.0007) [2023-12-27 19:04:07,208][20381] Updated weights for policy 1, policy_version 1935657 (0.0007) [2023-12-27 19:04:07,226][20347] Updated weights for policy 0, policy_version 1937102 (0.0008) [2023-12-27 19:04:07,276][20347] Updated weights for policy 0, policy_version 1937112 (0.0006) [2023-12-27 19:04:07,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 991584256. Throughput: 0: 10332.7, 1: 10368.3. Samples: 991545764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.390')] [2023-12-27 19:04:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001935664_495591424.pth... [2023-12-27 19:04:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001937112_495992832.pth... [2023-12-27 19:04:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001934448_495280128.pth [2023-12-27 19:04:07,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001935896_495681536.pth [2023-12-27 19:04:07,877][20381] Updated weights for policy 1, policy_version 1935667 (0.0006) [2023-12-27 19:04:07,930][20381] Updated weights for policy 1, policy_version 1935677 (0.0008) [2023-12-27 19:04:07,947][20347] Updated weights for policy 0, policy_version 1937122 (0.0008) [2023-12-27 19:04:07,983][20381] Updated weights for policy 1, policy_version 1935687 (0.0006) [2023-12-27 19:04:07,989][20347] Updated weights for policy 0, policy_version 1937132 (0.0005) [2023-12-27 19:04:08,037][20347] Updated weights for policy 0, policy_version 1937142 (0.0006) [2023-12-27 19:04:08,673][20381] Updated weights for policy 1, policy_version 1935697 (0.0005) [2023-12-27 19:04:08,687][20347] Updated weights for policy 0, policy_version 1937152 (0.0007) [2023-12-27 19:04:08,724][20381] Updated weights for policy 1, policy_version 1935707 (0.0007) [2023-12-27 19:04:08,752][20347] Updated weights for policy 0, policy_version 1937162 (0.0007) [2023-12-27 19:04:08,783][20381] Updated weights for policy 1, policy_version 1935717 (0.0005) [2023-12-27 19:04:08,818][20347] Updated weights for policy 0, policy_version 1937172 (0.0006) [2023-12-27 19:04:08,846][20381] Updated weights for policy 1, policy_version 1935727 (0.0005) [2023-12-27 19:04:09,426][20347] Updated weights for policy 0, policy_version 1937182 (0.0007) [2023-12-27 19:04:09,482][20347] Updated weights for policy 0, policy_version 1937192 (0.0008) [2023-12-27 19:04:09,540][20347] Updated weights for policy 0, policy_version 1937202 (0.0006) [2023-12-27 19:04:09,554][20381] Updated weights for policy 1, policy_version 1935737 (0.0006) [2023-12-27 19:04:09,615][20381] Updated weights for policy 1, policy_version 1935747 (0.0008) [2023-12-27 19:04:09,673][20381] Updated weights for policy 1, policy_version 1935757 (0.0008) [2023-12-27 19:04:10,187][20347] Updated weights for policy 0, policy_version 1937212 (0.0008) [2023-12-27 19:04:10,242][20347] Updated weights for policy 0, policy_version 1937222 (0.0007) [2023-12-27 19:04:10,296][20347] Updated weights for policy 0, policy_version 1937232 (0.0008) [2023-12-27 19:04:10,440][20381] Updated weights for policy 1, policy_version 1935767 (0.0008) [2023-12-27 19:04:10,502][20381] Updated weights for policy 1, policy_version 1935777 (0.0008) [2023-12-27 19:04:10,565][20381] Updated weights for policy 1, policy_version 1935787 (0.0008) [2023-12-27 19:04:10,922][20347] Updated weights for policy 0, policy_version 1937242 (0.0007) [2023-12-27 19:04:10,976][20347] Updated weights for policy 0, policy_version 1937252 (0.0006) [2023-12-27 19:04:11,031][20347] Updated weights for policy 0, policy_version 1937262 (0.0008) [2023-12-27 19:04:11,088][20347] Updated weights for policy 0, policy_version 1937272 (0.0007) [2023-12-27 19:04:11,295][20381] Updated weights for policy 1, policy_version 1935797 (0.0008) [2023-12-27 19:04:11,363][20381] Updated weights for policy 1, policy_version 1935807 (0.0009) [2023-12-27 19:04:11,428][20381] Updated weights for policy 1, policy_version 1935817 (0.0007) [2023-12-27 19:04:11,848][20347] Updated weights for policy 0, policy_version 1937282 (0.0006) [2023-12-27 19:04:11,915][20347] Updated weights for policy 0, policy_version 1937292 (0.0006) [2023-12-27 19:04:11,980][20347] Updated weights for policy 0, policy_version 1937302 (0.0006) [2023-12-27 19:04:12,092][20381] Updated weights for policy 1, policy_version 1935827 (0.0007) [2023-12-27 19:04:12,151][20381] Updated weights for policy 1, policy_version 1935837 (0.0006) [2023-12-27 19:04:12,215][20381] Updated weights for policy 1, policy_version 1935847 (0.0008) [2023-12-27 19:04:12,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 991682560. Throughput: 0: 10434.4, 1: 10314.0. Samples: 991671100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '900.630')] [2023-12-27 19:04:12,590][20347] Updated weights for policy 0, policy_version 1937312 (0.0009) [2023-12-27 19:04:12,636][20347] Updated weights for policy 0, policy_version 1937322 (0.0010) [2023-12-27 19:04:12,685][20347] Updated weights for policy 0, policy_version 1937332 (0.0008) [2023-12-27 19:04:12,884][20381] Updated weights for policy 1, policy_version 1935857 (0.0008) [2023-12-27 19:04:12,951][20381] Updated weights for policy 1, policy_version 1935867 (0.0007) [2023-12-27 19:04:13,010][20381] Updated weights for policy 1, policy_version 1935877 (0.0007) [2023-12-27 19:04:13,074][20381] Updated weights for policy 1, policy_version 1935887 (0.0005) [2023-12-27 19:04:13,331][20347] Updated weights for policy 0, policy_version 1937342 (0.0006) [2023-12-27 19:04:13,391][20347] Updated weights for policy 0, policy_version 1937352 (0.0005) [2023-12-27 19:04:13,446][20347] Updated weights for policy 0, policy_version 1937362 (0.0008) [2023-12-27 19:04:13,715][20381] Updated weights for policy 1, policy_version 1935897 (0.0009) [2023-12-27 19:04:13,771][20381] Updated weights for policy 1, policy_version 1935907 (0.0008) [2023-12-27 19:04:13,829][20381] Updated weights for policy 1, policy_version 1935917 (0.0007) [2023-12-27 19:04:13,994][20347] Updated weights for policy 0, policy_version 1937372 (0.0007) [2023-12-27 19:04:14,044][20347] Updated weights for policy 0, policy_version 1937382 (0.0005) [2023-12-27 19:04:14,104][20347] Updated weights for policy 0, policy_version 1937392 (0.0007) [2023-12-27 19:04:14,513][20381] Updated weights for policy 1, policy_version 1935927 (0.0007) [2023-12-27 19:04:14,564][20381] Updated weights for policy 1, policy_version 1935937 (0.0005) [2023-12-27 19:04:14,618][20381] Updated weights for policy 1, policy_version 1935947 (0.0005) [2023-12-27 19:04:14,800][20347] Updated weights for policy 0, policy_version 1937402 (0.0008) [2023-12-27 19:04:14,865][20347] Updated weights for policy 0, policy_version 1937412 (0.0008) [2023-12-27 19:04:14,929][20347] Updated weights for policy 0, policy_version 1937422 (0.0008) [2023-12-27 19:04:14,991][20347] Updated weights for policy 0, policy_version 1937432 (0.0007) [2023-12-27 19:04:15,301][20381] Updated weights for policy 1, policy_version 1935957 (0.0006) [2023-12-27 19:04:15,360][20381] Updated weights for policy 1, policy_version 1935967 (0.0009) [2023-12-27 19:04:15,410][20381] Updated weights for policy 1, policy_version 1935977 (0.0008) [2023-12-27 19:04:15,590][20347] Updated weights for policy 0, policy_version 1937442 (0.0007) [2023-12-27 19:04:15,638][20347] Updated weights for policy 0, policy_version 1937452 (0.0007) [2023-12-27 19:04:15,683][20347] Updated weights for policy 0, policy_version 1937462 (0.0007) [2023-12-27 19:04:16,002][20381] Updated weights for policy 1, policy_version 1935987 (0.0006) [2023-12-27 19:04:16,076][20381] Updated weights for policy 1, policy_version 1935997 (0.0007) [2023-12-27 19:04:16,129][20381] Updated weights for policy 1, policy_version 1936007 (0.0008) [2023-12-27 19:04:16,433][20347] Updated weights for policy 0, policy_version 1937472 (0.0006) [2023-12-27 19:04:16,490][20347] Updated weights for policy 0, policy_version 1937482 (0.0006) [2023-12-27 19:04:16,546][20347] Updated weights for policy 0, policy_version 1937492 (0.0008) [2023-12-27 19:04:16,797][20381] Updated weights for policy 1, policy_version 1936017 (0.0008) [2023-12-27 19:04:16,852][20381] Updated weights for policy 1, policy_version 1936027 (0.0006) [2023-12-27 19:04:16,898][20381] Updated weights for policy 1, policy_version 1936037 (0.0005) [2023-12-27 19:04:16,959][20381] Updated weights for policy 1, policy_version 1936047 (0.0008) [2023-12-27 19:04:17,206][20347] Updated weights for policy 0, policy_version 1937502 (0.0006) [2023-12-27 19:04:17,253][20347] Updated weights for policy 0, policy_version 1937512 (0.0005) [2023-12-27 19:04:17,307][20347] Updated weights for policy 0, policy_version 1937522 (0.0005) [2023-12-27 19:04:17,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20889.6, 300 sec: 20549.4). Total num frames: 991789056. Throughput: 0: 10493.9, 1: 10343.0. Samples: 991796816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '880.770')] [2023-12-27 19:04:17,509][20381] Updated weights for policy 1, policy_version 1936057 (0.0006) [2023-12-27 19:04:17,554][20381] Updated weights for policy 1, policy_version 1936067 (0.0007) [2023-12-27 19:04:17,605][20381] Updated weights for policy 1, policy_version 1936077 (0.0006) [2023-12-27 19:04:17,956][20347] Updated weights for policy 0, policy_version 1937532 (0.0006) [2023-12-27 19:04:18,005][20347] Updated weights for policy 0, policy_version 1937542 (0.0006) [2023-12-27 19:04:18,061][20347] Updated weights for policy 0, policy_version 1937552 (0.0007) [2023-12-27 19:04:18,298][20381] Updated weights for policy 1, policy_version 1936087 (0.0008) [2023-12-27 19:04:18,361][20381] Updated weights for policy 1, policy_version 1936097 (0.0008) [2023-12-27 19:04:18,418][20381] Updated weights for policy 1, policy_version 1936107 (0.0007) [2023-12-27 19:04:18,738][20347] Updated weights for policy 0, policy_version 1937562 (0.0008) [2023-12-27 19:04:18,800][20347] Updated weights for policy 0, policy_version 1937572 (0.0008) [2023-12-27 19:04:18,859][20347] Updated weights for policy 0, policy_version 1937582 (0.0007) [2023-12-27 19:04:18,924][20347] Updated weights for policy 0, policy_version 1937592 (0.0008) [2023-12-27 19:04:19,083][20381] Updated weights for policy 1, policy_version 1936117 (0.0007) [2023-12-27 19:04:19,137][20381] Updated weights for policy 1, policy_version 1936127 (0.0005) [2023-12-27 19:04:19,190][20381] Updated weights for policy 1, policy_version 1936137 (0.0005) [2023-12-27 19:04:19,669][20347] Updated weights for policy 0, policy_version 1937602 (0.0008) [2023-12-27 19:04:19,728][20347] Updated weights for policy 0, policy_version 1937612 (0.0007) [2023-12-27 19:04:19,784][20347] Updated weights for policy 0, policy_version 1937622 (0.0007) [2023-12-27 19:04:19,853][20381] Updated weights for policy 1, policy_version 1936147 (0.0008) [2023-12-27 19:04:19,914][20381] Updated weights for policy 1, policy_version 1936157 (0.0008) [2023-12-27 19:04:19,979][20381] Updated weights for policy 1, policy_version 1936167 (0.0008) [2023-12-27 19:04:20,538][20347] Updated weights for policy 0, policy_version 1937632 (0.0007) [2023-12-27 19:04:20,607][20347] Updated weights for policy 0, policy_version 1937642 (0.0008) [2023-12-27 19:04:20,624][20381] Updated weights for policy 1, policy_version 1936177 (0.0007) [2023-12-27 19:04:20,665][20347] Updated weights for policy 0, policy_version 1937652 (0.0007) [2023-12-27 19:04:20,674][20381] Updated weights for policy 1, policy_version 1936187 (0.0006) [2023-12-27 19:04:20,732][20381] Updated weights for policy 1, policy_version 1936197 (0.0008) [2023-12-27 19:04:20,794][20381] Updated weights for policy 1, policy_version 1936207 (0.0008) [2023-12-27 19:04:21,428][20347] Updated weights for policy 0, policy_version 1937662 (0.0007) [2023-12-27 19:04:21,489][20347] Updated weights for policy 0, policy_version 1937672 (0.0007) [2023-12-27 19:04:21,555][20347] Updated weights for policy 0, policy_version 1937682 (0.0007) [2023-12-27 19:04:21,571][20381] Updated weights for policy 1, policy_version 1936217 (0.0009) [2023-12-27 19:04:21,636][20381] Updated weights for policy 1, policy_version 1936227 (0.0008) [2023-12-27 19:04:21,704][20381] Updated weights for policy 1, policy_version 1936237 (0.0008) [2023-12-27 19:04:22,315][20347] Updated weights for policy 0, policy_version 1937692 (0.0009) [2023-12-27 19:04:22,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.1, 300 sec: 20493.9). Total num frames: 991879168. Throughput: 0: 10451.3, 1: 10328.2. Samples: 991858400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.820')] [2023-12-27 19:04:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001936240_495738880.pth... [2023-12-27 19:04:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001935056_495435776.pth [2023-12-27 19:04:22,384][20347] Updated weights for policy 0, policy_version 1937702 (0.0008) [2023-12-27 19:04:22,435][20381] Updated weights for policy 1, policy_version 1936247 (0.0008) [2023-12-27 19:04:22,441][20347] Updated weights for policy 0, policy_version 1937712 (0.0011) [2023-12-27 19:04:22,487][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001937720_496148480.pth... [2023-12-27 19:04:22,490][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001936472_495828992.pth [2023-12-27 19:04:22,495][20381] Updated weights for policy 1, policy_version 1936257 (0.0006) [2023-12-27 19:04:22,553][20381] Updated weights for policy 1, policy_version 1936267 (0.0007) [2023-12-27 19:04:23,071][20347] Updated weights for policy 0, policy_version 1937722 (0.0010) [2023-12-27 19:04:23,131][20347] Updated weights for policy 0, policy_version 1937732 (0.0011) [2023-12-27 19:04:23,148][20381] Updated weights for policy 1, policy_version 1936277 (0.0007) [2023-12-27 19:04:23,187][20347] Updated weights for policy 0, policy_version 1937742 (0.0010) [2023-12-27 19:04:23,212][20381] Updated weights for policy 1, policy_version 1936287 (0.0006) [2023-12-27 19:04:23,249][20347] Updated weights for policy 0, policy_version 1937752 (0.0008) [2023-12-27 19:04:23,263][20381] Updated weights for policy 1, policy_version 1936297 (0.0006) [2023-12-27 19:04:23,900][20381] Updated weights for policy 1, policy_version 1936307 (0.0006) [2023-12-27 19:04:23,920][20347] Updated weights for policy 0, policy_version 1937762 (0.0010) [2023-12-27 19:04:23,958][20381] Updated weights for policy 1, policy_version 1936317 (0.0006) [2023-12-27 19:04:23,976][20347] Updated weights for policy 0, policy_version 1937772 (0.0009) [2023-12-27 19:04:24,010][20381] Updated weights for policy 1, policy_version 1936327 (0.0011) [2023-12-27 19:04:24,033][20347] Updated weights for policy 0, policy_version 1937782 (0.0007) [2023-12-27 19:04:24,626][20381] Updated weights for policy 1, policy_version 1936337 (0.0009) [2023-12-27 19:04:24,651][20347] Updated weights for policy 0, policy_version 1937792 (0.0010) [2023-12-27 19:04:24,681][20381] Updated weights for policy 1, policy_version 1936347 (0.0006) [2023-12-27 19:04:24,708][20347] Updated weights for policy 0, policy_version 1937802 (0.0010) [2023-12-27 19:04:24,738][20381] Updated weights for policy 1, policy_version 1936357 (0.0008) [2023-12-27 19:04:24,754][20347] Updated weights for policy 0, policy_version 1937812 (0.0008) [2023-12-27 19:04:24,790][20381] Updated weights for policy 1, policy_version 1936367 (0.0011) [2023-12-27 19:04:25,392][20347] Updated weights for policy 0, policy_version 1937822 (0.0007) [2023-12-27 19:04:25,442][20381] Updated weights for policy 1, policy_version 1936377 (0.0006) [2023-12-27 19:04:25,448][20347] Updated weights for policy 0, policy_version 1937832 (0.0008) [2023-12-27 19:04:25,496][20381] Updated weights for policy 1, policy_version 1936387 (0.0005) [2023-12-27 19:04:25,504][20347] Updated weights for policy 0, policy_version 1937842 (0.0007) [2023-12-27 19:04:25,552][20381] Updated weights for policy 1, policy_version 1936397 (0.0005) [2023-12-27 19:04:26,129][20381] Updated weights for policy 1, policy_version 1936407 (0.0009) [2023-12-27 19:04:26,153][20347] Updated weights for policy 0, policy_version 1937852 (0.0006) [2023-12-27 19:04:26,177][20381] Updated weights for policy 1, policy_version 1936417 (0.0010) [2023-12-27 19:04:26,201][20347] Updated weights for policy 0, policy_version 1937862 (0.0005) [2023-12-27 19:04:26,231][20381] Updated weights for policy 1, policy_version 1936427 (0.0010) [2023-12-27 19:04:26,259][20347] Updated weights for policy 0, policy_version 1937872 (0.0006) [2023-12-27 19:04:26,807][20381] Updated weights for policy 1, policy_version 1936437 (0.0007) [2023-12-27 19:04:26,863][20381] Updated weights for policy 1, policy_version 1936447 (0.0006) [2023-12-27 19:04:26,909][20381] Updated weights for policy 1, policy_version 1936457 (0.0007) [2023-12-27 19:04:26,951][20347] Updated weights for policy 0, policy_version 1937882 (0.0010) [2023-12-27 19:04:27,007][20347] Updated weights for policy 0, policy_version 1937892 (0.0010) [2023-12-27 19:04:27,059][20347] Updated weights for policy 0, policy_version 1937902 (0.0010) [2023-12-27 19:04:27,117][20347] Updated weights for policy 0, policy_version 1937912 (0.0010) [2023-12-27 19:04:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 991993856. Throughput: 0: 10398.0, 1: 10378.3. Samples: 991982112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '940.490')] [2023-12-27 19:04:27,553][20381] Updated weights for policy 1, policy_version 1936467 (0.0007) [2023-12-27 19:04:27,609][20381] Updated weights for policy 1, policy_version 1936477 (0.0005) [2023-12-27 19:04:27,663][20381] Updated weights for policy 1, policy_version 1936487 (0.0007) [2023-12-27 19:04:27,793][20347] Updated weights for policy 0, policy_version 1937922 (0.0005) [2023-12-27 19:04:27,854][20347] Updated weights for policy 0, policy_version 1937932 (0.0005) [2023-12-27 19:04:27,900][20347] Updated weights for policy 0, policy_version 1937942 (0.0005) [2023-12-27 19:04:28,201][20381] Updated weights for policy 1, policy_version 1936497 (0.0006) [2023-12-27 19:04:28,256][20381] Updated weights for policy 1, policy_version 1936507 (0.0010) [2023-12-27 19:04:28,313][20381] Updated weights for policy 1, policy_version 1936517 (0.0007) [2023-12-27 19:04:28,375][20381] Updated weights for policy 1, policy_version 1936527 (0.0007) [2023-12-27 19:04:28,508][20347] Updated weights for policy 0, policy_version 1937952 (0.0009) [2023-12-27 19:04:28,569][20347] Updated weights for policy 0, policy_version 1937962 (0.0010) [2023-12-27 19:04:28,633][20347] Updated weights for policy 0, policy_version 1937972 (0.0009) [2023-12-27 19:04:29,009][20381] Updated weights for policy 1, policy_version 1936537 (0.0010) [2023-12-27 19:04:29,062][20381] Updated weights for policy 1, policy_version 1936547 (0.0010) [2023-12-27 19:04:29,110][20381] Updated weights for policy 1, policy_version 1936557 (0.0010) [2023-12-27 19:04:29,295][20347] Updated weights for policy 0, policy_version 1937982 (0.0010) [2023-12-27 19:04:29,364][20347] Updated weights for policy 0, policy_version 1937992 (0.0009) [2023-12-27 19:04:29,431][20347] Updated weights for policy 0, policy_version 1938002 (0.0009) [2023-12-27 19:04:29,760][20381] Updated weights for policy 1, policy_version 1936567 (0.0010) [2023-12-27 19:04:29,814][20381] Updated weights for policy 1, policy_version 1936577 (0.0011) [2023-12-27 19:04:29,879][20381] Updated weights for policy 1, policy_version 1936587 (0.0012) [2023-12-27 19:04:30,068][20347] Updated weights for policy 0, policy_version 1938012 (0.0008) [2023-12-27 19:04:30,129][20347] Updated weights for policy 0, policy_version 1938022 (0.0006) [2023-12-27 19:04:30,185][20347] Updated weights for policy 0, policy_version 1938032 (0.0006) [2023-12-27 19:04:30,582][20381] Updated weights for policy 1, policy_version 1936597 (0.0006) [2023-12-27 19:04:30,629][20381] Updated weights for policy 1, policy_version 1936607 (0.0006) [2023-12-27 19:04:30,681][20381] Updated weights for policy 1, policy_version 1936617 (0.0010) [2023-12-27 19:04:30,753][20347] Updated weights for policy 0, policy_version 1938042 (0.0006) [2023-12-27 19:04:30,798][20347] Updated weights for policy 0, policy_version 1938052 (0.0009) [2023-12-27 19:04:30,845][20347] Updated weights for policy 0, policy_version 1938062 (0.0007) [2023-12-27 19:04:30,894][20347] Updated weights for policy 0, policy_version 1938072 (0.0006) [2023-12-27 19:04:31,344][20381] Updated weights for policy 1, policy_version 1936627 (0.0009) [2023-12-27 19:04:31,414][20381] Updated weights for policy 1, policy_version 1936637 (0.0010) [2023-12-27 19:04:31,463][20381] Updated weights for policy 1, policy_version 1936647 (0.0010) [2023-12-27 19:04:31,589][20347] Updated weights for policy 0, policy_version 1938082 (0.0007) [2023-12-27 19:04:31,657][20347] Updated weights for policy 0, policy_version 1938092 (0.0008) [2023-12-27 19:04:31,716][20347] Updated weights for policy 0, policy_version 1938102 (0.0007) [2023-12-27 19:04:32,187][20381] Updated weights for policy 1, policy_version 1936657 (0.0010) [2023-12-27 19:04:32,243][20381] Updated weights for policy 1, policy_version 1936667 (0.0008) [2023-12-27 19:04:32,307][20381] Updated weights for policy 1, policy_version 1936677 (0.0007) [2023-12-27 19:04:32,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20521.7). Total num frames: 992092160. Throughput: 0: 10447.0, 1: 10410.7. Samples: 992110140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '959.250')] [2023-12-27 19:04:32,375][20381] Updated weights for policy 1, policy_version 1936687 (0.0007) [2023-12-27 19:04:32,397][20347] Updated weights for policy 0, policy_version 1938112 (0.0007) [2023-12-27 19:04:32,450][20347] Updated weights for policy 0, policy_version 1938122 (0.0008) [2023-12-27 19:04:32,506][20347] Updated weights for policy 0, policy_version 1938132 (0.0007) [2023-12-27 19:04:33,032][20381] Updated weights for policy 1, policy_version 1936697 (0.0005) [2023-12-27 19:04:33,084][20381] Updated weights for policy 1, policy_version 1936707 (0.0008) [2023-12-27 19:04:33,132][20381] Updated weights for policy 1, policy_version 1936717 (0.0007) [2023-12-27 19:04:33,199][20347] Updated weights for policy 0, policy_version 1938142 (0.0008) [2023-12-27 19:04:33,248][20347] Updated weights for policy 0, policy_version 1938152 (0.0007) [2023-12-27 19:04:33,330][20347] Updated weights for policy 0, policy_version 1938162 (0.0008) [2023-12-27 19:04:33,817][20381] Updated weights for policy 1, policy_version 1936727 (0.0008) [2023-12-27 19:04:33,872][20381] Updated weights for policy 1, policy_version 1936737 (0.0008) [2023-12-27 19:04:33,931][20381] Updated weights for policy 1, policy_version 1936747 (0.0008) [2023-12-27 19:04:33,988][20347] Updated weights for policy 0, policy_version 1938172 (0.0006) [2023-12-27 19:04:34,044][20347] Updated weights for policy 0, policy_version 1938182 (0.0006) [2023-12-27 19:04:34,102][20347] Updated weights for policy 0, policy_version 1938192 (0.0008) [2023-12-27 19:04:34,651][20381] Updated weights for policy 1, policy_version 1936757 (0.0008) [2023-12-27 19:04:34,706][20381] Updated weights for policy 1, policy_version 1936767 (0.0007) [2023-12-27 19:04:34,755][20381] Updated weights for policy 1, policy_version 1936777 (0.0008) [2023-12-27 19:04:34,767][20347] Updated weights for policy 0, policy_version 1938202 (0.0008) [2023-12-27 19:04:34,826][20347] Updated weights for policy 0, policy_version 1938212 (0.0007) [2023-12-27 19:04:34,872][20347] Updated weights for policy 0, policy_version 1938222 (0.0008) [2023-12-27 19:04:34,916][20347] Updated weights for policy 0, policy_version 1938232 (0.0007) [2023-12-27 19:04:35,453][20381] Updated weights for policy 1, policy_version 1936787 (0.0005) [2023-12-27 19:04:35,502][20381] Updated weights for policy 1, policy_version 1936797 (0.0008) [2023-12-27 19:04:35,556][20381] Updated weights for policy 1, policy_version 1936807 (0.0008) [2023-12-27 19:04:35,587][20347] Updated weights for policy 0, policy_version 1938242 (0.0005) [2023-12-27 19:04:35,647][20347] Updated weights for policy 0, policy_version 1938252 (0.0005) [2023-12-27 19:04:35,705][20347] Updated weights for policy 0, policy_version 1938262 (0.0006) [2023-12-27 19:04:36,308][20381] Updated weights for policy 1, policy_version 1936817 (0.0009) [2023-12-27 19:04:36,311][20347] Updated weights for policy 0, policy_version 1938272 (0.0008) [2023-12-27 19:04:36,366][20381] Updated weights for policy 1, policy_version 1936827 (0.0007) [2023-12-27 19:04:36,368][20347] Updated weights for policy 0, policy_version 1938282 (0.0008) [2023-12-27 19:04:36,422][20347] Updated weights for policy 0, policy_version 1938292 (0.0005) [2023-12-27 19:04:36,424][20381] Updated weights for policy 1, policy_version 1936837 (0.0006) [2023-12-27 19:04:36,491][20381] Updated weights for policy 1, policy_version 1936847 (0.0008) [2023-12-27 19:04:37,148][20381] Updated weights for policy 1, policy_version 1936857 (0.0008) [2023-12-27 19:04:37,153][20347] Updated weights for policy 0, policy_version 1938302 (0.0008) [2023-12-27 19:04:37,204][20381] Updated weights for policy 1, policy_version 1936867 (0.0006) [2023-12-27 19:04:37,206][20347] Updated weights for policy 0, policy_version 1938312 (0.0006) [2023-12-27 19:04:37,262][20347] Updated weights for policy 0, policy_version 1938322 (0.0005) [2023-12-27 19:04:37,264][20381] Updated weights for policy 1, policy_version 1936877 (0.0006) [2023-12-27 19:04:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20889.6, 300 sec: 20577.2). Total num frames: 992206848. Throughput: 0: 10434.4, 1: 10382.1. Samples: 992170728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '900.890')] [2023-12-27 19:04:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001936880_495902720.pth... [2023-12-27 19:04:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001938328_496304128.pth... [2023-12-27 19:04:37,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001935664_495591424.pth [2023-12-27 19:04:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001937112_495992832.pth [2023-12-27 19:04:37,935][20381] Updated weights for policy 1, policy_version 1936887 (0.0007) [2023-12-27 19:04:37,969][20347] Updated weights for policy 0, policy_version 1938332 (0.0007) [2023-12-27 19:04:37,980][20381] Updated weights for policy 1, policy_version 1936897 (0.0008) [2023-12-27 19:04:38,025][20347] Updated weights for policy 0, policy_version 1938342 (0.0008) [2023-12-27 19:04:38,031][20381] Updated weights for policy 1, policy_version 1936907 (0.0006) [2023-12-27 19:04:38,078][20347] Updated weights for policy 0, policy_version 1938352 (0.0006) [2023-12-27 19:04:38,728][20381] Updated weights for policy 1, policy_version 1936917 (0.0008) [2023-12-27 19:04:38,788][20381] Updated weights for policy 1, policy_version 1936927 (0.0011) [2023-12-27 19:04:38,798][20347] Updated weights for policy 0, policy_version 1938362 (0.0007) [2023-12-27 19:04:38,848][20381] Updated weights for policy 1, policy_version 1936937 (0.0011) [2023-12-27 19:04:38,855][20347] Updated weights for policy 0, policy_version 1938372 (0.0009) [2023-12-27 19:04:38,912][20347] Updated weights for policy 0, policy_version 1938382 (0.0006) [2023-12-27 19:04:38,973][20347] Updated weights for policy 0, policy_version 1938392 (0.0008) [2023-12-27 19:04:39,496][20381] Updated weights for policy 1, policy_version 1936947 (0.0009) [2023-12-27 19:04:39,562][20381] Updated weights for policy 1, policy_version 1936957 (0.0008) [2023-12-27 19:04:39,621][20381] Updated weights for policy 1, policy_version 1936967 (0.0008) [2023-12-27 19:04:39,703][20347] Updated weights for policy 0, policy_version 1938402 (0.0007) [2023-12-27 19:04:39,764][20347] Updated weights for policy 0, policy_version 1938412 (0.0008) [2023-12-27 19:04:39,822][20347] Updated weights for policy 0, policy_version 1938422 (0.0008) [2023-12-27 19:04:40,249][20381] Updated weights for policy 1, policy_version 1936977 (0.0007) [2023-12-27 19:04:40,301][20381] Updated weights for policy 1, policy_version 1936987 (0.0005) [2023-12-27 19:04:40,351][20381] Updated weights for policy 1, policy_version 1936997 (0.0005) [2023-12-27 19:04:40,411][20381] Updated weights for policy 1, policy_version 1937007 (0.0009) [2023-12-27 19:04:40,583][20347] Updated weights for policy 0, policy_version 1938432 (0.0008) [2023-12-27 19:04:40,647][20347] Updated weights for policy 0, policy_version 1938442 (0.0008) [2023-12-27 19:04:40,704][20347] Updated weights for policy 0, policy_version 1938452 (0.0008) [2023-12-27 19:04:41,035][20381] Updated weights for policy 1, policy_version 1937017 (0.0007) [2023-12-27 19:04:41,101][20381] Updated weights for policy 1, policy_version 1937027 (0.0008) [2023-12-27 19:04:41,162][20381] Updated weights for policy 1, policy_version 1937037 (0.0008) [2023-12-27 19:04:41,475][20347] Updated weights for policy 0, policy_version 1938462 (0.0008) [2023-12-27 19:04:41,533][20347] Updated weights for policy 0, policy_version 1938472 (0.0008) [2023-12-27 19:04:41,596][20347] Updated weights for policy 0, policy_version 1938482 (0.0008) [2023-12-27 19:04:41,876][20381] Updated weights for policy 1, policy_version 1937047 (0.0009) [2023-12-27 19:04:41,937][20381] Updated weights for policy 1, policy_version 1937057 (0.0007) [2023-12-27 19:04:41,996][20381] Updated weights for policy 1, policy_version 1937067 (0.0007) [2023-12-27 19:04:42,311][20347] Updated weights for policy 0, policy_version 1938492 (0.0008) [2023-12-27 19:04:42,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20521.7). Total num frames: 992296960. Throughput: 0: 10360.4, 1: 10374.6. Samples: 992291520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '920.870')] [2023-12-27 19:04:42,378][20347] Updated weights for policy 0, policy_version 1938502 (0.0008) [2023-12-27 19:04:42,434][20347] Updated weights for policy 0, policy_version 1938512 (0.0008) [2023-12-27 19:04:42,676][20381] Updated weights for policy 1, policy_version 1937077 (0.0007) [2023-12-27 19:04:42,728][20381] Updated weights for policy 1, policy_version 1937087 (0.0007) [2023-12-27 19:04:42,785][20381] Updated weights for policy 1, policy_version 1937097 (0.0008) [2023-12-27 19:04:43,120][20347] Updated weights for policy 0, policy_version 1938522 (0.0008) [2023-12-27 19:04:43,165][20347] Updated weights for policy 0, policy_version 1938532 (0.0007) [2023-12-27 19:04:43,214][20347] Updated weights for policy 0, policy_version 1938542 (0.0007) [2023-12-27 19:04:43,259][20347] Updated weights for policy 0, policy_version 1938552 (0.0008) [2023-12-27 19:04:43,489][20381] Updated weights for policy 1, policy_version 1937107 (0.0008) [2023-12-27 19:04:43,535][20381] Updated weights for policy 1, policy_version 1937117 (0.0008) [2023-12-27 19:04:43,584][20381] Updated weights for policy 1, policy_version 1937127 (0.0007) [2023-12-27 19:04:43,991][20347] Updated weights for policy 0, policy_version 1938562 (0.0007) [2023-12-27 19:04:44,047][20347] Updated weights for policy 0, policy_version 1938572 (0.0007) [2023-12-27 19:04:44,104][20347] Updated weights for policy 0, policy_version 1938582 (0.0008) [2023-12-27 19:04:44,233][20381] Updated weights for policy 1, policy_version 1937137 (0.0008) [2023-12-27 19:04:44,284][20381] Updated weights for policy 1, policy_version 1937147 (0.0008) [2023-12-27 19:04:44,336][20381] Updated weights for policy 1, policy_version 1937157 (0.0007) [2023-12-27 19:04:44,386][20381] Updated weights for policy 1, policy_version 1937167 (0.0008) [2023-12-27 19:04:44,838][20347] Updated weights for policy 0, policy_version 1938592 (0.0007) [2023-12-27 19:04:44,901][20347] Updated weights for policy 0, policy_version 1938602 (0.0008) [2023-12-27 19:04:44,958][20347] Updated weights for policy 0, policy_version 1938612 (0.0008) [2023-12-27 19:04:45,030][20381] Updated weights for policy 1, policy_version 1937177 (0.0007) [2023-12-27 19:04:45,084][20381] Updated weights for policy 1, policy_version 1937187 (0.0008) [2023-12-27 19:04:45,138][20381] Updated weights for policy 1, policy_version 1937197 (0.0008) [2023-12-27 19:04:45,634][20347] Updated weights for policy 0, policy_version 1938622 (0.0007) [2023-12-27 19:04:45,692][20347] Updated weights for policy 0, policy_version 1938632 (0.0007) [2023-12-27 19:04:45,747][20347] Updated weights for policy 0, policy_version 1938642 (0.0007) [2023-12-27 19:04:45,840][20381] Updated weights for policy 1, policy_version 1937207 (0.0006) [2023-12-27 19:04:45,888][20381] Updated weights for policy 1, policy_version 1937217 (0.0005) [2023-12-27 19:04:45,942][20381] Updated weights for policy 1, policy_version 1937227 (0.0005) [2023-12-27 19:04:46,343][20347] Updated weights for policy 0, policy_version 1938652 (0.0007) [2023-12-27 19:04:46,393][20347] Updated weights for policy 0, policy_version 1938662 (0.0008) [2023-12-27 19:04:46,451][20347] Updated weights for policy 0, policy_version 1938672 (0.0009) [2023-12-27 19:04:46,582][20381] Updated weights for policy 1, policy_version 1937237 (0.0006) [2023-12-27 19:04:46,634][20381] Updated weights for policy 1, policy_version 1937247 (0.0008) [2023-12-27 19:04:46,683][20381] Updated weights for policy 1, policy_version 1937257 (0.0008) [2023-12-27 19:04:47,072][20347] Updated weights for policy 0, policy_version 1938682 (0.0006) [2023-12-27 19:04:47,135][20347] Updated weights for policy 0, policy_version 1938692 (0.0006) [2023-12-27 19:04:47,197][20347] Updated weights for policy 0, policy_version 1938702 (0.0007) [2023-12-27 19:04:47,260][20347] Updated weights for policy 0, policy_version 1938712 (0.0006) [2023-12-27 19:04:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20753.1, 300 sec: 20549.4). Total num frames: 992403456. Throughput: 0: 10303.3, 1: 10385.2. Samples: 992413396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.080')] [2023-12-27 19:04:47,415][20381] Updated weights for policy 1, policy_version 1937267 (0.0007) [2023-12-27 19:04:47,469][20381] Updated weights for policy 1, policy_version 1937277 (0.0005) [2023-12-27 19:04:47,528][20381] Updated weights for policy 1, policy_version 1937287 (0.0005) [2023-12-27 19:04:47,822][20347] Updated weights for policy 0, policy_version 1938722 (0.0007) [2023-12-27 19:04:47,885][20347] Updated weights for policy 0, policy_version 1938732 (0.0007) [2023-12-27 19:04:47,949][20347] Updated weights for policy 0, policy_version 1938742 (0.0008) [2023-12-27 19:04:48,103][20381] Updated weights for policy 1, policy_version 1937297 (0.0006) [2023-12-27 19:04:48,167][20381] Updated weights for policy 1, policy_version 1937307 (0.0009) [2023-12-27 19:04:48,228][20381] Updated weights for policy 1, policy_version 1937317 (0.0008) [2023-12-27 19:04:48,289][20381] Updated weights for policy 1, policy_version 1937327 (0.0007) [2023-12-27 19:04:48,564][20347] Updated weights for policy 0, policy_version 1938752 (0.0009) [2023-12-27 19:04:48,624][20347] Updated weights for policy 0, policy_version 1938762 (0.0009) [2023-12-27 19:04:48,684][20347] Updated weights for policy 0, policy_version 1938772 (0.0006) [2023-12-27 19:04:49,021][20381] Updated weights for policy 1, policy_version 1937337 (0.0005) [2023-12-27 19:04:49,084][20381] Updated weights for policy 1, policy_version 1937347 (0.0005) [2023-12-27 19:04:49,145][20381] Updated weights for policy 1, policy_version 1937357 (0.0007) [2023-12-27 19:04:49,329][20347] Updated weights for policy 0, policy_version 1938782 (0.0007) [2023-12-27 19:04:49,388][20347] Updated weights for policy 0, policy_version 1938792 (0.0008) [2023-12-27 19:04:49,439][20347] Updated weights for policy 0, policy_version 1938802 (0.0008) [2023-12-27 19:04:49,780][20381] Updated weights for policy 1, policy_version 1937367 (0.0008) [2023-12-27 19:04:49,845][20381] Updated weights for policy 1, policy_version 1937377 (0.0008) [2023-12-27 19:04:49,909][20381] Updated weights for policy 1, policy_version 1937387 (0.0006) [2023-12-27 19:04:50,152][20347] Updated weights for policy 0, policy_version 1938812 (0.0008) [2023-12-27 19:04:50,213][20347] Updated weights for policy 0, policy_version 1938822 (0.0010) [2023-12-27 19:04:50,270][20347] Updated weights for policy 0, policy_version 1938832 (0.0010) [2023-12-27 19:04:50,657][20381] Updated weights for policy 1, policy_version 1937397 (0.0009) [2023-12-27 19:04:50,710][20381] Updated weights for policy 1, policy_version 1937407 (0.0007) [2023-12-27 19:04:50,767][20381] Updated weights for policy 1, policy_version 1937417 (0.0008) [2023-12-27 19:04:50,924][20347] Updated weights for policy 0, policy_version 1938842 (0.0009) [2023-12-27 19:04:50,988][20347] Updated weights for policy 0, policy_version 1938852 (0.0008) [2023-12-27 19:04:51,052][20347] Updated weights for policy 0, policy_version 1938862 (0.0008) [2023-12-27 19:04:51,105][20347] Updated weights for policy 0, policy_version 1938872 (0.0008) [2023-12-27 19:04:51,422][20381] Updated weights for policy 1, policy_version 1937427 (0.0007) [2023-12-27 19:04:51,478][20381] Updated weights for policy 1, policy_version 1937437 (0.0008) [2023-12-27 19:04:51,533][20381] Updated weights for policy 1, policy_version 1937447 (0.0008) [2023-12-27 19:04:51,822][20347] Updated weights for policy 0, policy_version 1938882 (0.0007) [2023-12-27 19:04:51,877][20347] Updated weights for policy 0, policy_version 1938892 (0.0008) [2023-12-27 19:04:51,936][20347] Updated weights for policy 0, policy_version 1938902 (0.0007) [2023-12-27 19:04:52,244][20381] Updated weights for policy 1, policy_version 1937457 (0.0008) [2023-12-27 19:04:52,299][20381] Updated weights for policy 1, policy_version 1937467 (0.0009) [2023-12-27 19:04:52,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.1, 300 sec: 20521.6). Total num frames: 992501760. Throughput: 0: 10298.0, 1: 10367.4. Samples: 992475712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.260')] [2023-12-27 19:04:52,364][20381] Updated weights for policy 1, policy_version 1937477 (0.0008) [2023-12-27 19:04:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001938904_496451584.pth... [2023-12-27 19:04:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001937720_496148480.pth [2023-12-27 19:04:52,428][20381] Updated weights for policy 1, policy_version 1937487 (0.0008) [2023-12-27 19:04:52,431][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001937488_496058368.pth... [2023-12-27 19:04:52,435][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001936240_495738880.pth [2023-12-27 19:04:52,617][20347] Updated weights for policy 0, policy_version 1938912 (0.0008) [2023-12-27 19:04:52,675][20347] Updated weights for policy 0, policy_version 1938922 (0.0007) [2023-12-27 19:04:52,733][20347] Updated weights for policy 0, policy_version 1938932 (0.0006) [2023-12-27 19:04:53,157][20381] Updated weights for policy 1, policy_version 1937497 (0.0007) [2023-12-27 19:04:53,203][20381] Updated weights for policy 1, policy_version 1937507 (0.0007) [2023-12-27 19:04:53,258][20381] Updated weights for policy 1, policy_version 1937517 (0.0007) [2023-12-27 19:04:53,429][20347] Updated weights for policy 0, policy_version 1938942 (0.0007) [2023-12-27 19:04:53,482][20347] Updated weights for policy 0, policy_version 1938952 (0.0007) [2023-12-27 19:04:53,541][20347] Updated weights for policy 0, policy_version 1938962 (0.0008) [2023-12-27 19:04:53,877][20381] Updated weights for policy 1, policy_version 1937527 (0.0005) [2023-12-27 19:04:53,922][20381] Updated weights for policy 1, policy_version 1937537 (0.0005) [2023-12-27 19:04:53,975][20381] Updated weights for policy 1, policy_version 1937547 (0.0007) [2023-12-27 19:04:54,149][20347] Updated weights for policy 0, policy_version 1938972 (0.0008) [2023-12-27 19:04:54,207][20347] Updated weights for policy 0, policy_version 1938982 (0.0007) [2023-12-27 19:04:54,270][20347] Updated weights for policy 0, policy_version 1938992 (0.0007) [2023-12-27 19:04:54,570][20381] Updated weights for policy 1, policy_version 1937557 (0.0008) [2023-12-27 19:04:54,625][20381] Updated weights for policy 1, policy_version 1937567 (0.0011) [2023-12-27 19:04:54,679][20381] Updated weights for policy 1, policy_version 1937577 (0.0010) [2023-12-27 19:04:54,950][20347] Updated weights for policy 0, policy_version 1939002 (0.0007) [2023-12-27 19:04:55,009][20347] Updated weights for policy 0, policy_version 1939012 (0.0007) [2023-12-27 19:04:55,063][20347] Updated weights for policy 0, policy_version 1939022 (0.0007) [2023-12-27 19:04:55,121][20347] Updated weights for policy 0, policy_version 1939032 (0.0007) [2023-12-27 19:04:55,375][20381] Updated weights for policy 1, policy_version 1937587 (0.0011) [2023-12-27 19:04:55,436][20381] Updated weights for policy 1, policy_version 1937597 (0.0011) [2023-12-27 19:04:55,493][20381] Updated weights for policy 1, policy_version 1937607 (0.0011) [2023-12-27 19:04:55,736][20347] Updated weights for policy 0, policy_version 1939042 (0.0007) [2023-12-27 19:04:55,793][20347] Updated weights for policy 0, policy_version 1939052 (0.0007) [2023-12-27 19:04:55,851][20347] Updated weights for policy 0, policy_version 1939062 (0.0006) [2023-12-27 19:04:56,156][20381] Updated weights for policy 1, policy_version 1937617 (0.0010) [2023-12-27 19:04:56,208][20381] Updated weights for policy 1, policy_version 1937627 (0.0005) [2023-12-27 19:04:56,270][20381] Updated weights for policy 1, policy_version 1937637 (0.0009) [2023-12-27 19:04:56,317][20381] Updated weights for policy 1, policy_version 1937647 (0.0007) [2023-12-27 19:04:56,452][20347] Updated weights for policy 0, policy_version 1939072 (0.0005) [2023-12-27 19:04:56,501][20347] Updated weights for policy 0, policy_version 1939082 (0.0005) [2023-12-27 19:04:56,553][20347] Updated weights for policy 0, policy_version 1939092 (0.0005) [2023-12-27 19:04:56,920][20381] Updated weights for policy 1, policy_version 1937657 (0.0008) [2023-12-27 19:04:56,976][20381] Updated weights for policy 1, policy_version 1937667 (0.0008) [2023-12-27 19:04:57,028][20381] Updated weights for policy 1, policy_version 1937677 (0.0008) [2023-12-27 19:04:57,162][20347] Updated weights for policy 0, policy_version 1939102 (0.0007) [2023-12-27 19:04:57,219][20347] Updated weights for policy 0, policy_version 1939112 (0.0007) [2023-12-27 19:04:57,278][20347] Updated weights for policy 0, policy_version 1939122 (0.0006) [2023-12-27 19:04:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 992616448. Throughput: 0: 10241.2, 1: 10387.2. Samples: 992599376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:04:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '919.460')] [2023-12-27 19:04:57,692][20381] Updated weights for policy 1, policy_version 1937687 (0.0006) [2023-12-27 19:04:57,747][20381] Updated weights for policy 1, policy_version 1937697 (0.0006) [2023-12-27 19:04:57,802][20381] Updated weights for policy 1, policy_version 1937707 (0.0006) [2023-12-27 19:04:57,973][20347] Updated weights for policy 0, policy_version 1939132 (0.0007) [2023-12-27 19:04:58,032][20347] Updated weights for policy 0, policy_version 1939142 (0.0008) [2023-12-27 19:04:58,086][20347] Updated weights for policy 0, policy_version 1939152 (0.0008) [2023-12-27 19:04:58,394][20381] Updated weights for policy 1, policy_version 1937717 (0.0007) [2023-12-27 19:04:58,459][20381] Updated weights for policy 1, policy_version 1937727 (0.0008) [2023-12-27 19:04:58,514][20381] Updated weights for policy 1, policy_version 1937737 (0.0008) [2023-12-27 19:04:58,852][20347] Updated weights for policy 0, policy_version 1939162 (0.0009) [2023-12-27 19:04:58,916][20347] Updated weights for policy 0, policy_version 1939172 (0.0008) [2023-12-27 19:04:58,971][20347] Updated weights for policy 0, policy_version 1939182 (0.0008) [2023-12-27 19:04:59,022][20347] Updated weights for policy 0, policy_version 1939192 (0.0008) [2023-12-27 19:04:59,346][20381] Updated weights for policy 1, policy_version 1937747 (0.0008) [2023-12-27 19:04:59,412][20381] Updated weights for policy 1, policy_version 1937757 (0.0008) [2023-12-27 19:04:59,462][20381] Updated weights for policy 1, policy_version 1937767 (0.0008) [2023-12-27 19:04:59,727][20347] Updated weights for policy 0, policy_version 1939202 (0.0008) [2023-12-27 19:04:59,787][20347] Updated weights for policy 0, policy_version 1939212 (0.0008) [2023-12-27 19:04:59,849][20347] Updated weights for policy 0, policy_version 1939222 (0.0007) [2023-12-27 19:05:00,147][20381] Updated weights for policy 1, policy_version 1937777 (0.0007) [2023-12-27 19:05:00,204][20381] Updated weights for policy 1, policy_version 1937787 (0.0008) [2023-12-27 19:05:00,266][20381] Updated weights for policy 1, policy_version 1937797 (0.0007) [2023-12-27 19:05:00,319][20381] Updated weights for policy 1, policy_version 1937807 (0.0007) [2023-12-27 19:05:00,594][20347] Updated weights for policy 0, policy_version 1939232 (0.0010) [2023-12-27 19:05:00,652][20347] Updated weights for policy 0, policy_version 1939242 (0.0010) [2023-12-27 19:05:00,703][20347] Updated weights for policy 0, policy_version 1939252 (0.0010) [2023-12-27 19:05:01,033][20381] Updated weights for policy 1, policy_version 1937817 (0.0007) [2023-12-27 19:05:01,083][20381] Updated weights for policy 1, policy_version 1937827 (0.0008) [2023-12-27 19:05:01,142][20381] Updated weights for policy 1, policy_version 1937837 (0.0009) [2023-12-27 19:05:01,425][20347] Updated weights for policy 0, policy_version 1939262 (0.0009) [2023-12-27 19:05:01,483][20347] Updated weights for policy 0, policy_version 1939272 (0.0007) [2023-12-27 19:05:01,536][20347] Updated weights for policy 0, policy_version 1939282 (0.0007) [2023-12-27 19:05:01,833][20381] Updated weights for policy 1, policy_version 1937847 (0.0007) [2023-12-27 19:05:01,886][20381] Updated weights for policy 1, policy_version 1937857 (0.0008) [2023-12-27 19:05:01,945][20381] Updated weights for policy 1, policy_version 1937867 (0.0007) [2023-12-27 19:05:02,326][20347] Updated weights for policy 0, policy_version 1939292 (0.0007) [2023-12-27 19:05:02,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 992706560. Throughput: 0: 10165.0, 1: 10344.5. Samples: 992719744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-12-27 19:05:02,365][19170] Avg episode reward: [(0, '1000.000'), (1, '908.430')] [2023-12-27 19:05:02,386][20347] Updated weights for policy 0, policy_version 1939302 (0.0012) [2023-12-27 19:05:02,448][20347] Updated weights for policy 0, policy_version 1939312 (0.0010) [2023-12-27 19:05:02,555][20381] Updated weights for policy 1, policy_version 1937877 (0.0006) [2023-12-27 19:05:02,616][20381] Updated weights for policy 1, policy_version 1937887 (0.0007) [2023-12-27 19:05:02,677][20381] Updated weights for policy 1, policy_version 1937897 (0.0007) [2023-12-27 19:05:03,080][20347] Updated weights for policy 0, policy_version 1939322 (0.0010) [2023-12-27 19:05:03,133][20347] Updated weights for policy 0, policy_version 1939332 (0.0010) [2023-12-27 19:05:03,185][20347] Updated weights for policy 0, policy_version 1939342 (0.0009) [2023-12-27 19:05:03,244][20347] Updated weights for policy 0, policy_version 1939352 (0.0009) [2023-12-27 19:05:03,359][20381] Updated weights for policy 1, policy_version 1937907 (0.0008) [2023-12-27 19:05:03,405][20381] Updated weights for policy 1, policy_version 1937917 (0.0010) [2023-12-27 19:05:03,460][20381] Updated weights for policy 1, policy_version 1937927 (0.0010) [2023-12-27 19:05:03,827][20347] Updated weights for policy 0, policy_version 1939362 (0.0010) [2023-12-27 19:05:03,891][20347] Updated weights for policy 0, policy_version 1939372 (0.0011) [2023-12-27 19:05:03,954][20347] Updated weights for policy 0, policy_version 1939382 (0.0007) [2023-12-27 19:05:04,180][20381] Updated weights for policy 1, policy_version 1937937 (0.0010) [2023-12-27 19:05:04,232][20381] Updated weights for policy 1, policy_version 1937947 (0.0008) [2023-12-27 19:05:04,282][20381] Updated weights for policy 1, policy_version 1937957 (0.0008) [2023-12-27 19:05:04,335][20381] Updated weights for policy 1, policy_version 1937967 (0.0008) [2023-12-27 19:05:04,645][20347] Updated weights for policy 0, policy_version 1939392 (0.0010) [2023-12-27 19:05:04,700][20347] Updated weights for policy 0, policy_version 1939402 (0.0010) [2023-12-27 19:05:04,757][20347] Updated weights for policy 0, policy_version 1939412 (0.0010) [2023-12-27 19:05:05,015][20381] Updated weights for policy 1, policy_version 1937977 (0.0006) [2023-12-27 19:05:05,068][20381] Updated weights for policy 1, policy_version 1937987 (0.0005) [2023-12-27 19:05:05,121][20381] Updated weights for policy 1, policy_version 1937997 (0.0005) [2023-12-27 19:05:05,438][20347] Updated weights for policy 0, policy_version 1939422 (0.0010) [2023-12-27 19:05:05,483][20347] Updated weights for policy 0, policy_version 1939432 (0.0010) [2023-12-27 19:05:05,535][20347] Updated weights for policy 0, policy_version 1939442 (0.0010) [2023-12-27 19:05:05,658][20381] Updated weights for policy 1, policy_version 1938007 (0.0005) [2023-12-27 19:05:05,710][20381] Updated weights for policy 1, policy_version 1938017 (0.0010) [2023-12-27 19:05:05,761][20381] Updated weights for policy 1, policy_version 1938027 (0.0005) [2023-12-27 19:05:06,231][20347] Updated weights for policy 0, policy_version 1939452 (0.0011) [2023-12-27 19:05:06,290][20347] Updated weights for policy 0, policy_version 1939462 (0.0011) [2023-12-27 19:05:06,350][20347] Updated weights for policy 0, policy_version 1939472 (0.0010) [2023-12-27 19:05:06,400][20381] Updated weights for policy 1, policy_version 1938037 (0.0006) [2023-12-27 19:05:06,462][20381] Updated weights for policy 1, policy_version 1938047 (0.0010) [2023-12-27 19:05:06,517][20381] Updated weights for policy 1, policy_version 1938057 (0.0011) [2023-12-27 19:05:07,042][20347] Updated weights for policy 0, policy_version 1939482 (0.0010) [2023-12-27 19:05:07,092][20347] Updated weights for policy 0, policy_version 1939492 (0.0010) [2023-12-27 19:05:07,138][20347] Updated weights for policy 0, policy_version 1939502 (0.0010) [2023-12-27 19:05:07,141][20381] Updated weights for policy 1, policy_version 1938067 (0.0010) [2023-12-27 19:05:07,191][20347] Updated weights for policy 0, policy_version 1939512 (0.0009) [2023-12-27 19:05:07,194][20381] Updated weights for policy 1, policy_version 1938077 (0.0011) [2023-12-27 19:05:07,247][20381] Updated weights for policy 1, policy_version 1938087 (0.0011) [2023-12-27 19:05:07,362][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 992821248. Throughput: 0: 10199.0, 1: 10332.1. Samples: 992782300. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '851.390')] [2023-12-27 19:05:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001939512_496607232.pth... [2023-12-27 19:05:07,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001938096_496214016.pth... [2023-12-27 19:05:07,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001936880_495902720.pth [2023-12-27 19:05:07,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001938328_496304128.pth [2023-12-27 19:05:07,850][20347] Updated weights for policy 0, policy_version 1939522 (0.0005) [2023-12-27 19:05:07,902][20347] Updated weights for policy 0, policy_version 1939532 (0.0005) [2023-12-27 19:05:07,916][20381] Updated weights for policy 1, policy_version 1938097 (0.0010) [2023-12-27 19:05:07,965][20347] Updated weights for policy 0, policy_version 1939542 (0.0006) [2023-12-27 19:05:07,971][20381] Updated weights for policy 1, policy_version 1938107 (0.0011) [2023-12-27 19:05:08,020][20381] Updated weights for policy 1, policy_version 1938117 (0.0011) [2023-12-27 19:05:08,065][20381] Updated weights for policy 1, policy_version 1938127 (0.0010) [2023-12-27 19:05:08,527][20347] Updated weights for policy 0, policy_version 1939552 (0.0010) [2023-12-27 19:05:08,586][20347] Updated weights for policy 0, policy_version 1939562 (0.0010) [2023-12-27 19:05:08,645][20347] Updated weights for policy 0, policy_version 1939572 (0.0010) [2023-12-27 19:05:08,784][20381] Updated weights for policy 1, policy_version 1938137 (0.0010) [2023-12-27 19:05:08,842][20381] Updated weights for policy 1, policy_version 1938147 (0.0010) [2023-12-27 19:05:08,907][20381] Updated weights for policy 1, policy_version 1938157 (0.0011) [2023-12-27 19:05:09,342][20347] Updated weights for policy 0, policy_version 1939582 (0.0011) [2023-12-27 19:05:09,414][20347] Updated weights for policy 0, policy_version 1939593 (0.0011) [2023-12-27 19:05:09,479][20347] Updated weights for policy 0, policy_version 1939603 (0.0008) [2023-12-27 19:05:09,589][20381] Updated weights for policy 1, policy_version 1938167 (0.0008) [2023-12-27 19:05:09,645][20381] Updated weights for policy 1, policy_version 1938177 (0.0007) [2023-12-27 19:05:09,704][20381] Updated weights for policy 1, policy_version 1938187 (0.0007) [2023-12-27 19:05:10,198][20347] Updated weights for policy 0, policy_version 1939613 (0.0010) [2023-12-27 19:05:10,257][20347] Updated weights for policy 0, policy_version 1939623 (0.0011) [2023-12-27 19:05:10,316][20347] Updated weights for policy 0, policy_version 1939633 (0.0010) [2023-12-27 19:05:10,475][20381] Updated weights for policy 1, policy_version 1938197 (0.0007) [2023-12-27 19:05:10,529][20381] Updated weights for policy 1, policy_version 1938207 (0.0008) [2023-12-27 19:05:10,582][20381] Updated weights for policy 1, policy_version 1938217 (0.0008) [2023-12-27 19:05:10,900][20347] Updated weights for policy 0, policy_version 1939643 (0.0010) [2023-12-27 19:05:10,958][20347] Updated weights for policy 0, policy_version 1939653 (0.0006) [2023-12-27 19:05:11,017][20347] Updated weights for policy 0, policy_version 1939663 (0.0009) [2023-12-27 19:05:11,440][20381] Updated weights for policy 1, policy_version 1938227 (0.0008) [2023-12-27 19:05:11,503][20381] Updated weights for policy 1, policy_version 1938237 (0.0008) [2023-12-27 19:05:11,566][20381] Updated weights for policy 1, policy_version 1938247 (0.0008) [2023-12-27 19:05:11,706][20347] Updated weights for policy 0, policy_version 1939673 (0.0008) [2023-12-27 19:05:11,769][20347] Updated weights for policy 0, policy_version 1939683 (0.0010) [2023-12-27 19:05:11,825][20347] Updated weights for policy 0, policy_version 1939693 (0.0006) [2023-12-27 19:05:11,888][20347] Updated weights for policy 0, policy_version 1939703 (0.0006) [2023-12-27 19:05:12,252][20381] Updated weights for policy 1, policy_version 1938257 (0.0007) [2023-12-27 19:05:12,315][20381] Updated weights for policy 1, policy_version 1938267 (0.0007) [2023-12-27 19:05:12,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 992911360. Throughput: 0: 10229.9, 1: 10292.2. Samples: 992905608. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '881.970')] [2023-12-27 19:05:12,376][20381] Updated weights for policy 1, policy_version 1938277 (0.0008) [2023-12-27 19:05:12,438][20381] Updated weights for policy 1, policy_version 1938287 (0.0006) [2023-12-27 19:05:12,525][20347] Updated weights for policy 0, policy_version 1939713 (0.0007) [2023-12-27 19:05:12,589][20347] Updated weights for policy 0, policy_version 1939723 (0.0008) [2023-12-27 19:05:12,643][20347] Updated weights for policy 0, policy_version 1939733 (0.0007) [2023-12-27 19:05:13,024][20381] Updated weights for policy 1, policy_version 1938297 (0.0005) [2023-12-27 19:05:13,082][20381] Updated weights for policy 1, policy_version 1938307 (0.0005) [2023-12-27 19:05:13,132][20381] Updated weights for policy 1, policy_version 1938317 (0.0006) [2023-12-27 19:05:13,371][20347] Updated weights for policy 0, policy_version 1939743 (0.0007) [2023-12-27 19:05:13,427][20347] Updated weights for policy 0, policy_version 1939753 (0.0008) [2023-12-27 19:05:13,487][20347] Updated weights for policy 0, policy_version 1939763 (0.0009) [2023-12-27 19:05:13,683][20381] Updated weights for policy 1, policy_version 1938327 (0.0008) [2023-12-27 19:05:13,738][20381] Updated weights for policy 1, policy_version 1938337 (0.0008) [2023-12-27 19:05:13,791][20381] Updated weights for policy 1, policy_version 1938347 (0.0008) [2023-12-27 19:05:14,176][20347] Updated weights for policy 0, policy_version 1939773 (0.0007) [2023-12-27 19:05:14,223][20347] Updated weights for policy 0, policy_version 1939783 (0.0005) [2023-12-27 19:05:14,283][20347] Updated weights for policy 0, policy_version 1939793 (0.0007) [2023-12-27 19:05:14,426][20381] Updated weights for policy 1, policy_version 1938357 (0.0008) [2023-12-27 19:05:14,485][20381] Updated weights for policy 1, policy_version 1938367 (0.0008) [2023-12-27 19:05:14,550][20381] Updated weights for policy 1, policy_version 1938377 (0.0007) [2023-12-27 19:05:14,963][20347] Updated weights for policy 0, policy_version 1939803 (0.0007) [2023-12-27 19:05:15,022][20347] Updated weights for policy 0, policy_version 1939813 (0.0008) [2023-12-27 19:05:15,084][20347] Updated weights for policy 0, policy_version 1939823 (0.0006) [2023-12-27 19:05:15,205][20381] Updated weights for policy 1, policy_version 1938387 (0.0008) [2023-12-27 19:05:15,268][20381] Updated weights for policy 1, policy_version 1938397 (0.0008) [2023-12-27 19:05:15,324][20381] Updated weights for policy 1, policy_version 1938407 (0.0007) [2023-12-27 19:05:15,724][20347] Updated weights for policy 0, policy_version 1939833 (0.0006) [2023-12-27 19:05:15,772][20347] Updated weights for policy 0, policy_version 1939843 (0.0010) [2023-12-27 19:05:15,817][20347] Updated weights for policy 0, policy_version 1939853 (0.0010) [2023-12-27 19:05:15,870][20347] Updated weights for policy 0, policy_version 1939863 (0.0010) [2023-12-27 19:05:16,012][20381] Updated weights for policy 1, policy_version 1938417 (0.0009) [2023-12-27 19:05:16,067][20381] Updated weights for policy 1, policy_version 1938427 (0.0008) [2023-12-27 19:05:16,127][20381] Updated weights for policy 1, policy_version 1938437 (0.0008) [2023-12-27 19:05:16,184][20381] Updated weights for policy 1, policy_version 1938447 (0.0008) [2023-12-27 19:05:16,539][20347] Updated weights for policy 0, policy_version 1939873 (0.0006) [2023-12-27 19:05:16,601][20347] Updated weights for policy 0, policy_version 1939883 (0.0006) [2023-12-27 19:05:16,662][20347] Updated weights for policy 0, policy_version 1939893 (0.0006) [2023-12-27 19:05:16,891][20381] Updated weights for policy 1, policy_version 1938457 (0.0006) [2023-12-27 19:05:16,946][20381] Updated weights for policy 1, policy_version 1938467 (0.0005) [2023-12-27 19:05:16,998][20381] Updated weights for policy 1, policy_version 1938477 (0.0005) [2023-12-27 19:05:17,208][20347] Updated weights for policy 0, policy_version 1939903 (0.0005) [2023-12-27 19:05:17,265][20347] Updated weights for policy 0, policy_version 1939913 (0.0005) [2023-12-27 19:05:17,321][20347] Updated weights for policy 0, policy_version 1939923 (0.0006) [2023-12-27 19:05:17,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 993026048. Throughput: 0: 10202.5, 1: 10250.1. Samples: 993030508. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.330')] [2023-12-27 19:05:17,701][20381] Updated weights for policy 1, policy_version 1938487 (0.0005) [2023-12-27 19:05:17,766][20381] Updated weights for policy 1, policy_version 1938497 (0.0006) [2023-12-27 19:05:17,834][20381] Updated weights for policy 1, policy_version 1938507 (0.0006) [2023-12-27 19:05:17,880][20347] Updated weights for policy 0, policy_version 1939933 (0.0005) [2023-12-27 19:05:17,934][20347] Updated weights for policy 0, policy_version 1939943 (0.0005) [2023-12-27 19:05:17,978][20347] Updated weights for policy 0, policy_version 1939953 (0.0005) [2023-12-27 19:05:18,474][20381] Updated weights for policy 1, policy_version 1938517 (0.0007) [2023-12-27 19:05:18,527][20381] Updated weights for policy 1, policy_version 1938527 (0.0007) [2023-12-27 19:05:18,585][20381] Updated weights for policy 1, policy_version 1938537 (0.0009) [2023-12-27 19:05:18,651][20347] Updated weights for policy 0, policy_version 1939963 (0.0006) [2023-12-27 19:05:18,703][20347] Updated weights for policy 0, policy_version 1939973 (0.0008) [2023-12-27 19:05:18,758][20347] Updated weights for policy 0, policy_version 1939983 (0.0007) [2023-12-27 19:05:19,239][20381] Updated weights for policy 1, policy_version 1938547 (0.0010) [2023-12-27 19:05:19,294][20381] Updated weights for policy 1, policy_version 1938557 (0.0009) [2023-12-27 19:05:19,363][20381] Updated weights for policy 1, policy_version 1938567 (0.0009) [2023-12-27 19:05:19,530][20347] Updated weights for policy 0, policy_version 1939993 (0.0007) [2023-12-27 19:05:19,589][20347] Updated weights for policy 0, policy_version 1940003 (0.0008) [2023-12-27 19:05:19,650][20347] Updated weights for policy 0, policy_version 1940013 (0.0008) [2023-12-27 19:05:19,708][20347] Updated weights for policy 0, policy_version 1940023 (0.0005) [2023-12-27 19:05:20,075][20381] Updated weights for policy 1, policy_version 1938577 (0.0011) [2023-12-27 19:05:20,134][20381] Updated weights for policy 1, policy_version 1938587 (0.0011) [2023-12-27 19:05:20,197][20381] Updated weights for policy 1, policy_version 1938597 (0.0011) [2023-12-27 19:05:20,261][20381] Updated weights for policy 1, policy_version 1938607 (0.0010) [2023-12-27 19:05:20,438][20347] Updated weights for policy 0, policy_version 1940033 (0.0007) [2023-12-27 19:05:20,496][20347] Updated weights for policy 0, policy_version 1940043 (0.0007) [2023-12-27 19:05:20,554][20347] Updated weights for policy 0, policy_version 1940053 (0.0008) [2023-12-27 19:05:20,966][20381] Updated weights for policy 1, policy_version 1938617 (0.0008) [2023-12-27 19:05:21,019][20381] Updated weights for policy 1, policy_version 1938627 (0.0008) [2023-12-27 19:05:21,089][20381] Updated weights for policy 1, policy_version 1938637 (0.0009) [2023-12-27 19:05:21,336][20347] Updated weights for policy 0, policy_version 1940063 (0.0008) [2023-12-27 19:05:21,406][20347] Updated weights for policy 0, policy_version 1940073 (0.0008) [2023-12-27 19:05:21,466][20347] Updated weights for policy 0, policy_version 1940083 (0.0007) [2023-12-27 19:05:21,850][20381] Updated weights for policy 1, policy_version 1938647 (0.0006) [2023-12-27 19:05:21,912][20381] Updated weights for policy 1, policy_version 1938657 (0.0006) [2023-12-27 19:05:21,961][20381] Updated weights for policy 1, policy_version 1938667 (0.0008) [2023-12-27 19:05:22,142][20347] Updated weights for policy 0, policy_version 1940093 (0.0009) [2023-12-27 19:05:22,200][20347] Updated weights for policy 0, policy_version 1940103 (0.0010) [2023-12-27 19:05:22,263][20347] Updated weights for policy 0, policy_version 1940113 (0.0011) [2023-12-27 19:05:22,363][19170] Fps is (10 sec: 21297.2, 60 sec: 20752.7, 300 sec: 20577.1). Total num frames: 993124352. Throughput: 0: 10214.7, 1: 10246.0. Samples: 993091480. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:22,364][19170] Avg episode reward: [(0, '1000.000'), (1, '822.710')] [2023-12-27 19:05:22,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001938672_496361472.pth... [2023-12-27 19:05:22,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001940120_496762880.pth... [2023-12-27 19:05:22,375][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001937488_496058368.pth [2023-12-27 19:05:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001938904_496451584.pth [2023-12-27 19:05:22,644][20381] Updated weights for policy 1, policy_version 1938677 (0.0008) [2023-12-27 19:05:22,708][20381] Updated weights for policy 1, policy_version 1938687 (0.0008) [2023-12-27 19:05:22,767][20381] Updated weights for policy 1, policy_version 1938697 (0.0007) [2023-12-27 19:05:22,959][20347] Updated weights for policy 0, policy_version 1940123 (0.0010) [2023-12-27 19:05:23,022][20347] Updated weights for policy 0, policy_version 1940133 (0.0005) [2023-12-27 19:05:23,085][20347] Updated weights for policy 0, policy_version 1940143 (0.0006) [2023-12-27 19:05:23,472][20381] Updated weights for policy 1, policy_version 1938707 (0.0007) [2023-12-27 19:05:23,519][20381] Updated weights for policy 1, policy_version 1938717 (0.0008) [2023-12-27 19:05:23,574][20381] Updated weights for policy 1, policy_version 1938727 (0.0008) [2023-12-27 19:05:23,716][20347] Updated weights for policy 0, policy_version 1940153 (0.0006) [2023-12-27 19:05:23,764][20347] Updated weights for policy 0, policy_version 1940163 (0.0009) [2023-12-27 19:05:23,810][20347] Updated weights for policy 0, policy_version 1940173 (0.0008) [2023-12-27 19:05:23,862][20347] Updated weights for policy 0, policy_version 1940183 (0.0008) [2023-12-27 19:05:24,222][20381] Updated weights for policy 1, policy_version 1938737 (0.0007) [2023-12-27 19:05:24,282][20381] Updated weights for policy 1, policy_version 1938747 (0.0008) [2023-12-27 19:05:24,336][20381] Updated weights for policy 1, policy_version 1938757 (0.0008) [2023-12-27 19:05:24,386][20381] Updated weights for policy 1, policy_version 1938767 (0.0007) [2023-12-27 19:05:24,580][20347] Updated weights for policy 0, policy_version 1940193 (0.0008) [2023-12-27 19:05:24,628][20347] Updated weights for policy 0, policy_version 1940203 (0.0006) [2023-12-27 19:05:24,681][20347] Updated weights for policy 0, policy_version 1940213 (0.0006) [2023-12-27 19:05:25,035][20381] Updated weights for policy 1, policy_version 1938777 (0.0006) [2023-12-27 19:05:25,083][20381] Updated weights for policy 1, policy_version 1938787 (0.0005) [2023-12-27 19:05:25,126][20381] Updated weights for policy 1, policy_version 1938797 (0.0005) [2023-12-27 19:05:25,413][20347] Updated weights for policy 0, policy_version 1940223 (0.0007) [2023-12-27 19:05:25,459][20347] Updated weights for policy 0, policy_version 1940233 (0.0008) [2023-12-27 19:05:25,511][20347] Updated weights for policy 0, policy_version 1940243 (0.0010) [2023-12-27 19:05:25,783][20381] Updated weights for policy 1, policy_version 1938807 (0.0008) [2023-12-27 19:05:25,835][20381] Updated weights for policy 1, policy_version 1938817 (0.0007) [2023-12-27 19:05:25,889][20381] Updated weights for policy 1, policy_version 1938827 (0.0008) [2023-12-27 19:05:26,210][20347] Updated weights for policy 0, policy_version 1940253 (0.0010) [2023-12-27 19:05:26,267][20347] Updated weights for policy 0, policy_version 1940263 (0.0010) [2023-12-27 19:05:26,319][20347] Updated weights for policy 0, policy_version 1940273 (0.0010) [2023-12-27 19:05:26,599][20381] Updated weights for policy 1, policy_version 1938837 (0.0007) [2023-12-27 19:05:26,649][20381] Updated weights for policy 1, policy_version 1938847 (0.0006) [2023-12-27 19:05:26,708][20381] Updated weights for policy 1, policy_version 1938857 (0.0007) [2023-12-27 19:05:26,997][20347] Updated weights for policy 0, policy_version 1940283 (0.0010) [2023-12-27 19:05:27,050][20347] Updated weights for policy 0, policy_version 1940293 (0.0010) [2023-12-27 19:05:27,101][20347] Updated weights for policy 0, policy_version 1940303 (0.0010) [2023-12-27 19:05:27,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 993222656. Throughput: 0: 10226.1, 1: 10229.2. Samples: 993212008. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '881.000')] [2023-12-27 19:05:27,400][20381] Updated weights for policy 1, policy_version 1938867 (0.0007) [2023-12-27 19:05:27,450][20381] Updated weights for policy 1, policy_version 1938877 (0.0007) [2023-12-27 19:05:27,500][20381] Updated weights for policy 1, policy_version 1938887 (0.0007) [2023-12-27 19:05:27,796][20347] Updated weights for policy 0, policy_version 1940313 (0.0010) [2023-12-27 19:05:27,853][20347] Updated weights for policy 0, policy_version 1940323 (0.0010) [2023-12-27 19:05:27,900][20347] Updated weights for policy 0, policy_version 1940333 (0.0010) [2023-12-27 19:05:27,953][20347] Updated weights for policy 0, policy_version 1940343 (0.0010) [2023-12-27 19:05:28,216][20381] Updated weights for policy 1, policy_version 1938897 (0.0007) [2023-12-27 19:05:28,263][20381] Updated weights for policy 1, policy_version 1938907 (0.0007) [2023-12-27 19:05:28,314][20381] Updated weights for policy 1, policy_version 1938917 (0.0007) [2023-12-27 19:05:28,377][20381] Updated weights for policy 1, policy_version 1938927 (0.0008) [2023-12-27 19:05:28,633][20347] Updated weights for policy 0, policy_version 1940353 (0.0010) [2023-12-27 19:05:28,694][20347] Updated weights for policy 0, policy_version 1940363 (0.0010) [2023-12-27 19:05:28,751][20347] Updated weights for policy 0, policy_version 1940373 (0.0010) [2023-12-27 19:05:29,074][20381] Updated weights for policy 1, policy_version 1938937 (0.0009) [2023-12-27 19:05:29,123][20381] Updated weights for policy 1, policy_version 1938947 (0.0007) [2023-12-27 19:05:29,168][20381] Updated weights for policy 1, policy_version 1938957 (0.0009) [2023-12-27 19:05:29,341][20347] Updated weights for policy 0, policy_version 1940383 (0.0008) [2023-12-27 19:05:29,411][20347] Updated weights for policy 0, policy_version 1940393 (0.0009) [2023-12-27 19:05:29,465][20347] Updated weights for policy 0, policy_version 1940403 (0.0009) [2023-12-27 19:05:30,009][20381] Updated weights for policy 1, policy_version 1938967 (0.0009) [2023-12-27 19:05:30,062][20381] Updated weights for policy 1, policy_version 1938977 (0.0009) [2023-12-27 19:05:30,123][20381] Updated weights for policy 1, policy_version 1938987 (0.0007) [2023-12-27 19:05:30,125][20347] Updated weights for policy 0, policy_version 1940413 (0.0007) [2023-12-27 19:05:30,182][20347] Updated weights for policy 0, policy_version 1940423 (0.0005) [2023-12-27 19:05:30,240][20347] Updated weights for policy 0, policy_version 1940433 (0.0006) [2023-12-27 19:05:30,751][20381] Updated weights for policy 1, policy_version 1938997 (0.0007) [2023-12-27 19:05:30,806][20381] Updated weights for policy 1, policy_version 1939007 (0.0005) [2023-12-27 19:05:30,856][20381] Updated weights for policy 1, policy_version 1939017 (0.0005) [2023-12-27 19:05:30,909][20347] Updated weights for policy 0, policy_version 1940443 (0.0008) [2023-12-27 19:05:30,962][20347] Updated weights for policy 0, policy_version 1940453 (0.0008) [2023-12-27 19:05:31,006][20347] Updated weights for policy 0, policy_version 1940463 (0.0007) [2023-12-27 19:05:31,486][20381] Updated weights for policy 1, policy_version 1939027 (0.0006) [2023-12-27 19:05:31,540][20381] Updated weights for policy 1, policy_version 1939037 (0.0006) [2023-12-27 19:05:31,605][20381] Updated weights for policy 1, policy_version 1939047 (0.0008) [2023-12-27 19:05:31,798][20347] Updated weights for policy 0, policy_version 1940473 (0.0007) [2023-12-27 19:05:31,850][20347] Updated weights for policy 0, policy_version 1940483 (0.0008) [2023-12-27 19:05:31,908][20347] Updated weights for policy 0, policy_version 1940493 (0.0006) [2023-12-27 19:05:31,968][20347] Updated weights for policy 0, policy_version 1940503 (0.0007) [2023-12-27 19:05:32,253][20381] Updated weights for policy 1, policy_version 1939057 (0.0008) [2023-12-27 19:05:32,310][20381] Updated weights for policy 1, policy_version 1939067 (0.0008) [2023-12-27 19:05:32,362][19170] Fps is (10 sec: 19662.8, 60 sec: 20480.0, 300 sec: 20521.7). Total num frames: 993320960. Throughput: 0: 10226.5, 1: 10207.2. Samples: 993332912. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '958.020')] [2023-12-27 19:05:32,368][20381] Updated weights for policy 1, policy_version 1939077 (0.0008) [2023-12-27 19:05:32,420][20381] Updated weights for policy 1, policy_version 1939087 (0.0008) [2023-12-27 19:05:32,644][20347] Updated weights for policy 0, policy_version 1940513 (0.0006) [2023-12-27 19:05:32,702][20347] Updated weights for policy 0, policy_version 1940523 (0.0005) [2023-12-27 19:05:32,750][20347] Updated weights for policy 0, policy_version 1940533 (0.0005) [2023-12-27 19:05:33,111][20381] Updated weights for policy 1, policy_version 1939097 (0.0008) [2023-12-27 19:05:33,164][20381] Updated weights for policy 1, policy_version 1939107 (0.0007) [2023-12-27 19:05:33,206][20381] Updated weights for policy 1, policy_version 1939117 (0.0007) [2023-12-27 19:05:33,311][20347] Updated weights for policy 0, policy_version 1940543 (0.0007) [2023-12-27 19:05:33,362][20347] Updated weights for policy 0, policy_version 1940553 (0.0007) [2023-12-27 19:05:33,406][20347] Updated weights for policy 0, policy_version 1940563 (0.0007) [2023-12-27 19:05:33,859][20381] Updated weights for policy 1, policy_version 1939127 (0.0008) [2023-12-27 19:05:33,907][20381] Updated weights for policy 1, policy_version 1939137 (0.0007) [2023-12-27 19:05:33,958][20381] Updated weights for policy 1, policy_version 1939147 (0.0007) [2023-12-27 19:05:34,079][20347] Updated weights for policy 0, policy_version 1940573 (0.0009) [2023-12-27 19:05:34,133][20347] Updated weights for policy 0, policy_version 1940583 (0.0007) [2023-12-27 19:05:34,198][20347] Updated weights for policy 0, policy_version 1940593 (0.0007) [2023-12-27 19:05:34,726][20381] Updated weights for policy 1, policy_version 1939157 (0.0007) [2023-12-27 19:05:34,791][20381] Updated weights for policy 1, policy_version 1939167 (0.0008) [2023-12-27 19:05:34,852][20381] Updated weights for policy 1, policy_version 1939177 (0.0008) [2023-12-27 19:05:34,859][20347] Updated weights for policy 0, policy_version 1940603 (0.0007) [2023-12-27 19:05:34,914][20347] Updated weights for policy 0, policy_version 1940613 (0.0010) [2023-12-27 19:05:34,972][20347] Updated weights for policy 0, policy_version 1940623 (0.0010) [2023-12-27 19:05:35,545][20381] Updated weights for policy 1, policy_version 1939187 (0.0008) [2023-12-27 19:05:35,607][20381] Updated weights for policy 1, policy_version 1939197 (0.0008) [2023-12-27 19:05:35,645][20347] Updated weights for policy 0, policy_version 1940633 (0.0010) [2023-12-27 19:05:35,669][20381] Updated weights for policy 1, policy_version 1939207 (0.0007) [2023-12-27 19:05:35,693][20347] Updated weights for policy 0, policy_version 1940643 (0.0010) [2023-12-27 19:05:35,738][20347] Updated weights for policy 0, policy_version 1940653 (0.0010) [2023-12-27 19:05:35,786][20347] Updated weights for policy 0, policy_version 1940663 (0.0010) [2023-12-27 19:05:36,410][20347] Updated weights for policy 0, policy_version 1940673 (0.0007) [2023-12-27 19:05:36,413][20381] Updated weights for policy 1, policy_version 1939217 (0.0006) [2023-12-27 19:05:36,473][20347] Updated weights for policy 0, policy_version 1940683 (0.0006) [2023-12-27 19:05:36,478][20381] Updated weights for policy 1, policy_version 1939227 (0.0009) [2023-12-27 19:05:36,535][20347] Updated weights for policy 0, policy_version 1940693 (0.0006) [2023-12-27 19:05:36,536][20381] Updated weights for policy 1, policy_version 1939237 (0.0006) [2023-12-27 19:05:36,592][20381] Updated weights for policy 1, policy_version 1939247 (0.0008) [2023-12-27 19:05:37,141][20347] Updated weights for policy 0, policy_version 1940703 (0.0007) [2023-12-27 19:05:37,204][20347] Updated weights for policy 0, policy_version 1940713 (0.0008) [2023-12-27 19:05:37,265][20347] Updated weights for policy 0, policy_version 1940723 (0.0008) [2023-12-27 19:05:37,314][20381] Updated weights for policy 1, policy_version 1939257 (0.0006) [2023-12-27 19:05:37,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 993427456. Throughput: 0: 10239.8, 1: 10203.6. Samples: 993395660. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '899.230')] [2023-12-27 19:05:37,367][20381] Updated weights for policy 1, policy_version 1939267 (0.0008) [2023-12-27 19:05:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001940728_496918528.pth... [2023-12-27 19:05:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001939512_496607232.pth [2023-12-27 19:05:37,427][20381] Updated weights for policy 1, policy_version 1939277 (0.0008) [2023-12-27 19:05:37,445][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001939280_496517120.pth... [2023-12-27 19:05:37,448][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001938096_496214016.pth [2023-12-27 19:05:37,939][20347] Updated weights for policy 0, policy_version 1940733 (0.0006) [2023-12-27 19:05:37,988][20347] Updated weights for policy 0, policy_version 1940743 (0.0007) [2023-12-27 19:05:38,041][20347] Updated weights for policy 0, policy_version 1940753 (0.0007) [2023-12-27 19:05:38,135][20381] Updated weights for policy 1, policy_version 1939287 (0.0007) [2023-12-27 19:05:38,188][20381] Updated weights for policy 1, policy_version 1939297 (0.0008) [2023-12-27 19:05:38,238][20381] Updated weights for policy 1, policy_version 1939307 (0.0008) [2023-12-27 19:05:38,672][20347] Updated weights for policy 0, policy_version 1940763 (0.0008) [2023-12-27 19:05:38,730][20347] Updated weights for policy 0, policy_version 1940773 (0.0006) [2023-12-27 19:05:38,784][20347] Updated weights for policy 0, policy_version 1940783 (0.0006) [2023-12-27 19:05:38,980][20381] Updated weights for policy 1, policy_version 1939317 (0.0008) [2023-12-27 19:05:39,026][20381] Updated weights for policy 1, policy_version 1939327 (0.0008) [2023-12-27 19:05:39,081][20381] Updated weights for policy 1, policy_version 1939337 (0.0007) [2023-12-27 19:05:39,471][20347] Updated weights for policy 0, policy_version 1940793 (0.0006) [2023-12-27 19:05:39,528][20347] Updated weights for policy 0, policy_version 1940803 (0.0008) [2023-12-27 19:05:39,587][20347] Updated weights for policy 0, policy_version 1940813 (0.0007) [2023-12-27 19:05:39,642][20347] Updated weights for policy 0, policy_version 1940823 (0.0008) [2023-12-27 19:05:39,862][20381] Updated weights for policy 1, policy_version 1939347 (0.0008) [2023-12-27 19:05:39,923][20381] Updated weights for policy 1, policy_version 1939357 (0.0007) [2023-12-27 19:05:39,993][20381] Updated weights for policy 1, policy_version 1939367 (0.0008) [2023-12-27 19:05:40,353][20347] Updated weights for policy 0, policy_version 1940833 (0.0008) [2023-12-27 19:05:40,406][20347] Updated weights for policy 0, policy_version 1940843 (0.0009) [2023-12-27 19:05:40,462][20347] Updated weights for policy 0, policy_version 1940853 (0.0008) [2023-12-27 19:05:40,680][20381] Updated weights for policy 1, policy_version 1939377 (0.0008) [2023-12-27 19:05:40,731][20381] Updated weights for policy 1, policy_version 1939387 (0.0007) [2023-12-27 19:05:40,786][20381] Updated weights for policy 1, policy_version 1939397 (0.0007) [2023-12-27 19:05:40,834][20381] Updated weights for policy 1, policy_version 1939407 (0.0008) [2023-12-27 19:05:41,195][20347] Updated weights for policy 0, policy_version 1940863 (0.0008) [2023-12-27 19:05:41,256][20347] Updated weights for policy 0, policy_version 1940873 (0.0007) [2023-12-27 19:05:41,321][20347] Updated weights for policy 0, policy_version 1940883 (0.0008) [2023-12-27 19:05:41,594][20381] Updated weights for policy 1, policy_version 1939417 (0.0008) [2023-12-27 19:05:41,660][20381] Updated weights for policy 1, policy_version 1939427 (0.0008) [2023-12-27 19:05:41,725][20381] Updated weights for policy 1, policy_version 1939437 (0.0008) [2023-12-27 19:05:42,050][20347] Updated weights for policy 0, policy_version 1940893 (0.0008) [2023-12-27 19:05:42,110][20347] Updated weights for policy 0, policy_version 1940903 (0.0008) [2023-12-27 19:05:42,169][20347] Updated weights for policy 0, policy_version 1940913 (0.0008) [2023-12-27 19:05:42,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 993525760. Throughput: 0: 10258.8, 1: 10099.7. Samples: 993515512. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.480')] [2023-12-27 19:05:42,399][20381] Updated weights for policy 1, policy_version 1939447 (0.0008) [2023-12-27 19:05:42,457][20381] Updated weights for policy 1, policy_version 1939457 (0.0008) [2023-12-27 19:05:42,519][20381] Updated weights for policy 1, policy_version 1939467 (0.0008) [2023-12-27 19:05:42,852][20347] Updated weights for policy 0, policy_version 1940923 (0.0008) [2023-12-27 19:05:42,911][20347] Updated weights for policy 0, policy_version 1940933 (0.0007) [2023-12-27 19:05:42,968][20347] Updated weights for policy 0, policy_version 1940943 (0.0008) [2023-12-27 19:05:43,251][20381] Updated weights for policy 1, policy_version 1939477 (0.0007) [2023-12-27 19:05:43,296][20381] Updated weights for policy 1, policy_version 1939487 (0.0008) [2023-12-27 19:05:43,345][20381] Updated weights for policy 1, policy_version 1939497 (0.0007) [2023-12-27 19:05:43,571][20347] Updated weights for policy 0, policy_version 1940953 (0.0008) [2023-12-27 19:05:43,629][20347] Updated weights for policy 0, policy_version 1940963 (0.0007) [2023-12-27 19:05:43,688][20347] Updated weights for policy 0, policy_version 1940973 (0.0007) [2023-12-27 19:05:43,745][20347] Updated weights for policy 0, policy_version 1940983 (0.0007) [2023-12-27 19:05:44,026][20381] Updated weights for policy 1, policy_version 1939507 (0.0007) [2023-12-27 19:05:44,083][20381] Updated weights for policy 1, policy_version 1939517 (0.0008) [2023-12-27 19:05:44,132][20381] Updated weights for policy 1, policy_version 1939527 (0.0007) [2023-12-27 19:05:44,387][20347] Updated weights for policy 0, policy_version 1940993 (0.0007) [2023-12-27 19:05:44,448][20347] Updated weights for policy 0, policy_version 1941003 (0.0006) [2023-12-27 19:05:44,505][20347] Updated weights for policy 0, policy_version 1941013 (0.0006) [2023-12-27 19:05:44,908][20381] Updated weights for policy 1, policy_version 1939537 (0.0008) [2023-12-27 19:05:44,969][20381] Updated weights for policy 1, policy_version 1939547 (0.0007) [2023-12-27 19:05:45,033][20381] Updated weights for policy 1, policy_version 1939557 (0.0006) [2023-12-27 19:05:45,092][20381] Updated weights for policy 1, policy_version 1939567 (0.0006) [2023-12-27 19:05:45,093][20347] Updated weights for policy 0, policy_version 1941023 (0.0009) [2023-12-27 19:05:45,153][20347] Updated weights for policy 0, policy_version 1941033 (0.0011) [2023-12-27 19:05:45,214][20347] Updated weights for policy 0, policy_version 1941043 (0.0010) [2023-12-27 19:05:45,733][20381] Updated weights for policy 1, policy_version 1939577 (0.0007) [2023-12-27 19:05:45,785][20381] Updated weights for policy 1, policy_version 1939587 (0.0007) [2023-12-27 19:05:45,831][20381] Updated weights for policy 1, policy_version 1939597 (0.0008) [2023-12-27 19:05:45,882][20347] Updated weights for policy 0, policy_version 1941053 (0.0008) [2023-12-27 19:05:45,941][20347] Updated weights for policy 0, policy_version 1941063 (0.0009) [2023-12-27 19:05:46,005][20347] Updated weights for policy 0, policy_version 1941073 (0.0010) [2023-12-27 19:05:46,512][20381] Updated weights for policy 1, policy_version 1939607 (0.0008) [2023-12-27 19:05:46,560][20381] Updated weights for policy 1, policy_version 1939617 (0.0007) [2023-12-27 19:05:46,604][20381] Updated weights for policy 1, policy_version 1939627 (0.0007) [2023-12-27 19:05:46,652][20347] Updated weights for policy 0, policy_version 1941083 (0.0010) [2023-12-27 19:05:46,703][20347] Updated weights for policy 0, policy_version 1941093 (0.0010) [2023-12-27 19:05:46,764][20347] Updated weights for policy 0, policy_version 1941103 (0.0010) [2023-12-27 19:05:47,353][20381] Updated weights for policy 1, policy_version 1939637 (0.0008) [2023-12-27 19:05:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 993624064. Throughput: 0: 10324.4, 1: 10072.4. Samples: 993637600. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '890.370')] [2023-12-27 19:05:47,400][20347] Updated weights for policy 0, policy_version 1941113 (0.0009) [2023-12-27 19:05:47,407][20381] Updated weights for policy 1, policy_version 1939647 (0.0009) [2023-12-27 19:05:47,461][20347] Updated weights for policy 0, policy_version 1941123 (0.0010) [2023-12-27 19:05:47,465][20381] Updated weights for policy 1, policy_version 1939657 (0.0010) [2023-12-27 19:05:47,515][20347] Updated weights for policy 0, policy_version 1941133 (0.0007) [2023-12-27 19:05:47,571][20347] Updated weights for policy 0, policy_version 1941143 (0.0007) [2023-12-27 19:05:48,109][20381] Updated weights for policy 1, policy_version 1939667 (0.0009) [2023-12-27 19:05:48,172][20381] Updated weights for policy 1, policy_version 1939677 (0.0007) [2023-12-27 19:05:48,228][20381] Updated weights for policy 1, policy_version 1939687 (0.0007) [2023-12-27 19:05:48,273][20347] Updated weights for policy 0, policy_version 1941153 (0.0006) [2023-12-27 19:05:48,318][20347] Updated weights for policy 0, policy_version 1941163 (0.0007) [2023-12-27 19:05:48,381][20347] Updated weights for policy 0, policy_version 1941173 (0.0008) [2023-12-27 19:05:48,931][20381] Updated weights for policy 1, policy_version 1939697 (0.0006) [2023-12-27 19:05:48,995][20381] Updated weights for policy 1, policy_version 1939707 (0.0007) [2023-12-27 19:05:49,048][20347] Updated weights for policy 0, policy_version 1941183 (0.0006) [2023-12-27 19:05:49,050][20381] Updated weights for policy 1, policy_version 1939717 (0.0008) [2023-12-27 19:05:49,101][20381] Updated weights for policy 1, policy_version 1939727 (0.0006) [2023-12-27 19:05:49,103][20347] Updated weights for policy 0, policy_version 1941193 (0.0005) [2023-12-27 19:05:49,163][20347] Updated weights for policy 0, policy_version 1941203 (0.0007) [2023-12-27 19:05:49,833][20381] Updated weights for policy 1, policy_version 1939737 (0.0008) [2023-12-27 19:05:49,852][20347] Updated weights for policy 0, policy_version 1941213 (0.0008) [2023-12-27 19:05:49,894][20381] Updated weights for policy 1, policy_version 1939747 (0.0008) [2023-12-27 19:05:49,909][20347] Updated weights for policy 0, policy_version 1941223 (0.0008) [2023-12-27 19:05:49,957][20381] Updated weights for policy 1, policy_version 1939757 (0.0007) [2023-12-27 19:05:49,976][20347] Updated weights for policy 0, policy_version 1941233 (0.0009) [2023-12-27 19:05:50,617][20381] Updated weights for policy 1, policy_version 1939767 (0.0006) [2023-12-27 19:05:50,675][20381] Updated weights for policy 1, policy_version 1939777 (0.0010) [2023-12-27 19:05:50,686][20347] Updated weights for policy 0, policy_version 1941243 (0.0008) [2023-12-27 19:05:50,737][20381] Updated weights for policy 1, policy_version 1939787 (0.0008) [2023-12-27 19:05:50,742][20347] Updated weights for policy 0, policy_version 1941253 (0.0008) [2023-12-27 19:05:50,797][20347] Updated weights for policy 0, policy_version 1941263 (0.0008) [2023-12-27 19:05:51,421][20381] Updated weights for policy 1, policy_version 1939797 (0.0008) [2023-12-27 19:05:51,472][20381] Updated weights for policy 1, policy_version 1939807 (0.0006) [2023-12-27 19:05:51,529][20381] Updated weights for policy 1, policy_version 1939817 (0.0008) [2023-12-27 19:05:51,576][20347] Updated weights for policy 0, policy_version 1941273 (0.0008) [2023-12-27 19:05:51,641][20347] Updated weights for policy 0, policy_version 1941283 (0.0008) [2023-12-27 19:05:51,701][20347] Updated weights for policy 0, policy_version 1941293 (0.0007) [2023-12-27 19:05:51,769][20347] Updated weights for policy 0, policy_version 1941303 (0.0008) [2023-12-27 19:05:52,228][20381] Updated weights for policy 1, policy_version 1939827 (0.0006) [2023-12-27 19:05:52,289][20381] Updated weights for policy 1, policy_version 1939837 (0.0006) [2023-12-27 19:05:52,354][20381] Updated weights for policy 1, policy_version 1939847 (0.0006) [2023-12-27 19:05:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20343.5, 300 sec: 20549.4). Total num frames: 993722368. Throughput: 0: 10294.2, 1: 10052.2. Samples: 993697888. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '891.660')] [2023-12-27 19:05:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001941304_497065984.pth... [2023-12-27 19:05:52,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001940120_496762880.pth [2023-12-27 19:05:52,405][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001939856_496664576.pth... [2023-12-27 19:05:52,409][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001938672_496361472.pth [2023-12-27 19:05:52,517][20347] Updated weights for policy 0, policy_version 1941313 (0.0007) [2023-12-27 19:05:52,574][20347] Updated weights for policy 0, policy_version 1941323 (0.0008) [2023-12-27 19:05:52,627][20347] Updated weights for policy 0, policy_version 1941333 (0.0008) [2023-12-27 19:05:52,991][20381] Updated weights for policy 1, policy_version 1939857 (0.0008) [2023-12-27 19:05:53,049][20381] Updated weights for policy 1, policy_version 1939867 (0.0011) [2023-12-27 19:05:53,102][20381] Updated weights for policy 1, policy_version 1939877 (0.0010) [2023-12-27 19:05:53,152][20381] Updated weights for policy 1, policy_version 1939887 (0.0011) [2023-12-27 19:05:53,301][20347] Updated weights for policy 0, policy_version 1941343 (0.0007) [2023-12-27 19:05:53,364][20347] Updated weights for policy 0, policy_version 1941353 (0.0007) [2023-12-27 19:05:53,418][20347] Updated weights for policy 0, policy_version 1941363 (0.0007) [2023-12-27 19:05:53,828][20381] Updated weights for policy 1, policy_version 1939897 (0.0008) [2023-12-27 19:05:53,884][20381] Updated weights for policy 1, policy_version 1939907 (0.0008) [2023-12-27 19:05:53,938][20381] Updated weights for policy 1, policy_version 1939917 (0.0008) [2023-12-27 19:05:54,099][20347] Updated weights for policy 0, policy_version 1941373 (0.0008) [2023-12-27 19:05:54,147][20347] Updated weights for policy 0, policy_version 1941383 (0.0008) [2023-12-27 19:05:54,195][20347] Updated weights for policy 0, policy_version 1941393 (0.0007) [2023-12-27 19:05:54,614][20381] Updated weights for policy 1, policy_version 1939927 (0.0006) [2023-12-27 19:05:54,668][20381] Updated weights for policy 1, policy_version 1939937 (0.0006) [2023-12-27 19:05:54,718][20381] Updated weights for policy 1, policy_version 1939947 (0.0007) [2023-12-27 19:05:54,929][20347] Updated weights for policy 0, policy_version 1941403 (0.0008) [2023-12-27 19:05:54,990][20347] Updated weights for policy 0, policy_version 1941413 (0.0007) [2023-12-27 19:05:55,050][20347] Updated weights for policy 0, policy_version 1941423 (0.0006) [2023-12-27 19:05:55,377][20381] Updated weights for policy 1, policy_version 1939957 (0.0006) [2023-12-27 19:05:55,434][20381] Updated weights for policy 1, policy_version 1939967 (0.0005) [2023-12-27 19:05:55,492][20381] Updated weights for policy 1, policy_version 1939977 (0.0005) [2023-12-27 19:05:55,607][20347] Updated weights for policy 0, policy_version 1941433 (0.0006) [2023-12-27 19:05:55,665][20347] Updated weights for policy 0, policy_version 1941443 (0.0010) [2023-12-27 19:05:55,724][20347] Updated weights for policy 0, policy_version 1941453 (0.0005) [2023-12-27 19:05:55,783][20347] Updated weights for policy 0, policy_version 1941463 (0.0005) [2023-12-27 19:05:56,098][20381] Updated weights for policy 1, policy_version 1939987 (0.0007) [2023-12-27 19:05:56,152][20381] Updated weights for policy 1, policy_version 1939997 (0.0010) [2023-12-27 19:05:56,200][20381] Updated weights for policy 1, policy_version 1940007 (0.0008) [2023-12-27 19:05:56,390][20347] Updated weights for policy 0, policy_version 1941473 (0.0005) [2023-12-27 19:05:56,443][20347] Updated weights for policy 0, policy_version 1941483 (0.0005) [2023-12-27 19:05:56,497][20347] Updated weights for policy 0, policy_version 1941493 (0.0005) [2023-12-27 19:05:56,787][20381] Updated weights for policy 1, policy_version 1940017 (0.0005) [2023-12-27 19:05:56,846][20381] Updated weights for policy 1, policy_version 1940027 (0.0010) [2023-12-27 19:05:56,900][20381] Updated weights for policy 1, policy_version 1940037 (0.0010) [2023-12-27 19:05:56,956][20381] Updated weights for policy 1, policy_version 1940047 (0.0006) [2023-12-27 19:05:57,023][20347] Updated weights for policy 0, policy_version 1941503 (0.0005) [2023-12-27 19:05:57,066][20347] Updated weights for policy 0, policy_version 1941513 (0.0005) [2023-12-27 19:05:57,127][20347] Updated weights for policy 0, policy_version 1941523 (0.0005) [2023-12-27 19:05:57,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20343.5, 300 sec: 20605.0). Total num frames: 993837056. Throughput: 0: 10265.1, 1: 10085.5. Samples: 993821384. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:05:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.550')] [2023-12-27 19:05:57,524][20381] Updated weights for policy 1, policy_version 1940057 (0.0005) [2023-12-27 19:05:57,574][20381] Updated weights for policy 1, policy_version 1940067 (0.0005) [2023-12-27 19:05:57,624][20381] Updated weights for policy 1, policy_version 1940077 (0.0005) [2023-12-27 19:05:57,742][20347] Updated weights for policy 0, policy_version 1941533 (0.0005) [2023-12-27 19:05:57,794][20347] Updated weights for policy 0, policy_version 1941543 (0.0005) [2023-12-27 19:05:57,849][20347] Updated weights for policy 0, policy_version 1941553 (0.0009) [2023-12-27 19:05:58,238][20381] Updated weights for policy 1, policy_version 1940087 (0.0009) [2023-12-27 19:05:58,300][20381] Updated weights for policy 1, policy_version 1940097 (0.0007) [2023-12-27 19:05:58,373][20381] Updated weights for policy 1, policy_version 1940107 (0.0009) [2023-12-27 19:05:58,530][20347] Updated weights for policy 0, policy_version 1941563 (0.0010) [2023-12-27 19:05:58,586][20347] Updated weights for policy 0, policy_version 1941573 (0.0010) [2023-12-27 19:05:58,648][20347] Updated weights for policy 0, policy_version 1941583 (0.0008) [2023-12-27 19:05:59,184][20381] Updated weights for policy 1, policy_version 1940117 (0.0010) [2023-12-27 19:05:59,247][20381] Updated weights for policy 1, policy_version 1940127 (0.0009) [2023-12-27 19:05:59,317][20381] Updated weights for policy 1, policy_version 1940137 (0.0008) [2023-12-27 19:05:59,443][20347] Updated weights for policy 0, policy_version 1941593 (0.0008) [2023-12-27 19:05:59,495][20347] Updated weights for policy 0, policy_version 1941603 (0.0007) [2023-12-27 19:05:59,549][20347] Updated weights for policy 0, policy_version 1941613 (0.0007) [2023-12-27 19:05:59,608][20347] Updated weights for policy 0, policy_version 1941623 (0.0007) [2023-12-27 19:06:00,024][20381] Updated weights for policy 1, policy_version 1940147 (0.0008) [2023-12-27 19:06:00,088][20381] Updated weights for policy 1, policy_version 1940157 (0.0006) [2023-12-27 19:06:00,149][20381] Updated weights for policy 1, policy_version 1940167 (0.0006) [2023-12-27 19:06:00,309][20347] Updated weights for policy 0, policy_version 1941633 (0.0006) [2023-12-27 19:06:00,369][20347] Updated weights for policy 0, policy_version 1941643 (0.0005) [2023-12-27 19:06:00,431][20347] Updated weights for policy 0, policy_version 1941653 (0.0005) [2023-12-27 19:06:00,849][20381] Updated weights for policy 1, policy_version 1940177 (0.0008) [2023-12-27 19:06:00,903][20381] Updated weights for policy 1, policy_version 1940187 (0.0008) [2023-12-27 19:06:00,952][20381] Updated weights for policy 1, policy_version 1940197 (0.0007) [2023-12-27 19:06:00,966][20347] Updated weights for policy 0, policy_version 1941663 (0.0005) [2023-12-27 19:06:00,996][20381] Updated weights for policy 1, policy_version 1940207 (0.0007) [2023-12-27 19:06:01,025][20347] Updated weights for policy 0, policy_version 1941673 (0.0006) [2023-12-27 19:06:01,092][20347] Updated weights for policy 0, policy_version 1941683 (0.0007) [2023-12-27 19:06:01,764][20381] Updated weights for policy 1, policy_version 1940217 (0.0008) [2023-12-27 19:06:01,785][20347] Updated weights for policy 0, policy_version 1941693 (0.0007) [2023-12-27 19:06:01,820][20381] Updated weights for policy 1, policy_version 1940227 (0.0007) [2023-12-27 19:06:01,844][20347] Updated weights for policy 0, policy_version 1941703 (0.0007) [2023-12-27 19:06:01,873][20381] Updated weights for policy 1, policy_version 1940237 (0.0006) [2023-12-27 19:06:01,899][20347] Updated weights for policy 0, policy_version 1941713 (0.0008) [2023-12-27 19:06:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 993935360. Throughput: 0: 10301.0, 1: 10047.6. Samples: 993946196. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.080')] [2023-12-27 19:06:02,543][20381] Updated weights for policy 1, policy_version 1940247 (0.0006) [2023-12-27 19:06:02,599][20347] Updated weights for policy 0, policy_version 1941723 (0.0007) [2023-12-27 19:06:02,602][20381] Updated weights for policy 1, policy_version 1940257 (0.0005) [2023-12-27 19:06:02,653][20347] Updated weights for policy 0, policy_version 1941733 (0.0007) [2023-12-27 19:06:02,657][20381] Updated weights for policy 1, policy_version 1940267 (0.0005) [2023-12-27 19:06:02,714][20347] Updated weights for policy 0, policy_version 1941743 (0.0008) [2023-12-27 19:06:03,254][20381] Updated weights for policy 1, policy_version 1940277 (0.0006) [2023-12-27 19:06:03,301][20381] Updated weights for policy 1, policy_version 1940287 (0.0007) [2023-12-27 19:06:03,353][20381] Updated weights for policy 1, policy_version 1940297 (0.0007) [2023-12-27 19:06:03,427][20347] Updated weights for policy 0, policy_version 1941753 (0.0008) [2023-12-27 19:06:03,478][20347] Updated weights for policy 0, policy_version 1941763 (0.0005) [2023-12-27 19:06:03,538][20347] Updated weights for policy 0, policy_version 1941773 (0.0006) [2023-12-27 19:06:03,590][20347] Updated weights for policy 0, policy_version 1941783 (0.0007) [2023-12-27 19:06:04,014][20381] Updated weights for policy 1, policy_version 1940307 (0.0007) [2023-12-27 19:06:04,063][20381] Updated weights for policy 1, policy_version 1940317 (0.0010) [2023-12-27 19:06:04,118][20381] Updated weights for policy 1, policy_version 1940327 (0.0011) [2023-12-27 19:06:04,289][20347] Updated weights for policy 0, policy_version 1941793 (0.0007) [2023-12-27 19:06:04,342][20347] Updated weights for policy 0, policy_version 1941803 (0.0008) [2023-12-27 19:06:04,399][20347] Updated weights for policy 0, policy_version 1941813 (0.0007) [2023-12-27 19:06:04,770][20381] Updated weights for policy 1, policy_version 1940337 (0.0010) [2023-12-27 19:06:04,819][20381] Updated weights for policy 1, policy_version 1940347 (0.0011) [2023-12-27 19:06:04,865][20381] Updated weights for policy 1, policy_version 1940357 (0.0010) [2023-12-27 19:06:04,910][20381] Updated weights for policy 1, policy_version 1940367 (0.0010) [2023-12-27 19:06:05,107][20347] Updated weights for policy 0, policy_version 1941823 (0.0007) [2023-12-27 19:06:05,159][20347] Updated weights for policy 0, policy_version 1941833 (0.0009) [2023-12-27 19:06:05,216][20347] Updated weights for policy 0, policy_version 1941843 (0.0007) [2023-12-27 19:06:05,553][20381] Updated weights for policy 1, policy_version 1940377 (0.0011) [2023-12-27 19:06:05,608][20381] Updated weights for policy 1, policy_version 1940387 (0.0011) [2023-12-27 19:06:05,668][20381] Updated weights for policy 1, policy_version 1940397 (0.0011) [2023-12-27 19:06:05,893][20347] Updated weights for policy 0, policy_version 1941853 (0.0008) [2023-12-27 19:06:05,945][20347] Updated weights for policy 0, policy_version 1941863 (0.0008) [2023-12-27 19:06:06,000][20347] Updated weights for policy 0, policy_version 1941873 (0.0008) [2023-12-27 19:06:06,282][20381] Updated weights for policy 1, policy_version 1940407 (0.0009) [2023-12-27 19:06:06,345][20381] Updated weights for policy 1, policy_version 1940417 (0.0011) [2023-12-27 19:06:06,411][20381] Updated weights for policy 1, policy_version 1940427 (0.0011) [2023-12-27 19:06:06,660][20347] Updated weights for policy 0, policy_version 1941883 (0.0007) [2023-12-27 19:06:06,722][20347] Updated weights for policy 0, policy_version 1941893 (0.0005) [2023-12-27 19:06:06,780][20347] Updated weights for policy 0, policy_version 1941903 (0.0005) [2023-12-27 19:06:07,105][20381] Updated weights for policy 1, policy_version 1940437 (0.0008) [2023-12-27 19:06:07,170][20381] Updated weights for policy 1, policy_version 1940447 (0.0006) [2023-12-27 19:06:07,226][20381] Updated weights for policy 1, policy_version 1940457 (0.0008) [2023-12-27 19:06:07,363][19170] Fps is (10 sec: 20479.3, 60 sec: 20343.4, 300 sec: 20604.9). Total num frames: 994041856. Throughput: 0: 10277.7, 1: 10114.0. Samples: 994009088. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '891.120')] [2023-12-27 19:06:07,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001940464_496820224.pth... [2023-12-27 19:06:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001941912_497221632.pth... [2023-12-27 19:06:07,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001939280_496517120.pth [2023-12-27 19:06:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001940728_496918528.pth [2023-12-27 19:06:07,446][20347] Updated weights for policy 0, policy_version 1941913 (0.0007) [2023-12-27 19:06:07,495][20347] Updated weights for policy 0, policy_version 1941923 (0.0007) [2023-12-27 19:06:07,538][20347] Updated weights for policy 0, policy_version 1941933 (0.0005) [2023-12-27 19:06:07,591][20347] Updated weights for policy 0, policy_version 1941943 (0.0005) [2023-12-27 19:06:07,887][20381] Updated weights for policy 1, policy_version 1940467 (0.0009) [2023-12-27 19:06:07,933][20381] Updated weights for policy 1, policy_version 1940477 (0.0010) [2023-12-27 19:06:07,980][20381] Updated weights for policy 1, policy_version 1940487 (0.0010) [2023-12-27 19:06:08,165][20347] Updated weights for policy 0, policy_version 1941953 (0.0008) [2023-12-27 19:06:08,211][20347] Updated weights for policy 0, policy_version 1941963 (0.0008) [2023-12-27 19:06:08,257][20347] Updated weights for policy 0, policy_version 1941973 (0.0008) [2023-12-27 19:06:08,587][20381] Updated weights for policy 1, policy_version 1940497 (0.0008) [2023-12-27 19:06:08,643][20381] Updated weights for policy 1, policy_version 1940507 (0.0005) [2023-12-27 19:06:08,695][20381] Updated weights for policy 1, policy_version 1940517 (0.0005) [2023-12-27 19:06:08,754][20381] Updated weights for policy 1, policy_version 1940527 (0.0006) [2023-12-27 19:06:09,062][20347] Updated weights for policy 0, policy_version 1941983 (0.0008) [2023-12-27 19:06:09,116][20347] Updated weights for policy 0, policy_version 1941993 (0.0007) [2023-12-27 19:06:09,169][20347] Updated weights for policy 0, policy_version 1942003 (0.0008) [2023-12-27 19:06:09,493][20381] Updated weights for policy 1, policy_version 1940537 (0.0008) [2023-12-27 19:06:09,555][20381] Updated weights for policy 1, policy_version 1940547 (0.0007) [2023-12-27 19:06:09,613][20381] Updated weights for policy 1, policy_version 1940557 (0.0008) [2023-12-27 19:06:09,967][20347] Updated weights for policy 0, policy_version 1942013 (0.0007) [2023-12-27 19:06:10,029][20347] Updated weights for policy 0, policy_version 1942023 (0.0008) [2023-12-27 19:06:10,089][20347] Updated weights for policy 0, policy_version 1942033 (0.0007) [2023-12-27 19:06:10,302][20381] Updated weights for policy 1, policy_version 1940567 (0.0008) [2023-12-27 19:06:10,358][20381] Updated weights for policy 1, policy_version 1940577 (0.0007) [2023-12-27 19:06:10,418][20381] Updated weights for policy 1, policy_version 1940587 (0.0008) [2023-12-27 19:06:10,773][20347] Updated weights for policy 0, policy_version 1942043 (0.0008) [2023-12-27 19:06:10,832][20347] Updated weights for policy 0, policy_version 1942053 (0.0008) [2023-12-27 19:06:10,888][20347] Updated weights for policy 0, policy_version 1942063 (0.0007) [2023-12-27 19:06:11,086][20381] Updated weights for policy 1, policy_version 1940597 (0.0008) [2023-12-27 19:06:11,161][20381] Updated weights for policy 1, policy_version 1940607 (0.0007) [2023-12-27 19:06:11,224][20381] Updated weights for policy 1, policy_version 1940617 (0.0007) [2023-12-27 19:06:11,695][20347] Updated weights for policy 0, policy_version 1942073 (0.0007) [2023-12-27 19:06:11,760][20347] Updated weights for policy 0, policy_version 1942083 (0.0008) [2023-12-27 19:06:11,824][20347] Updated weights for policy 0, policy_version 1942093 (0.0008) [2023-12-27 19:06:11,888][20347] Updated weights for policy 0, policy_version 1942103 (0.0008) [2023-12-27 19:06:12,002][20381] Updated weights for policy 1, policy_version 1940627 (0.0007) [2023-12-27 19:06:12,063][20381] Updated weights for policy 1, policy_version 1940637 (0.0006) [2023-12-27 19:06:12,126][20381] Updated weights for policy 1, policy_version 1940647 (0.0007) [2023-12-27 19:06:12,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 994140160. Throughput: 0: 10296.8, 1: 10120.4. Samples: 994130780. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '900.000')] [2023-12-27 19:06:12,647][20347] Updated weights for policy 0, policy_version 1942113 (0.0007) [2023-12-27 19:06:12,703][20347] Updated weights for policy 0, policy_version 1942123 (0.0007) [2023-12-27 19:06:12,747][20381] Updated weights for policy 1, policy_version 1940657 (0.0008) [2023-12-27 19:06:12,762][20347] Updated weights for policy 0, policy_version 1942133 (0.0008) [2023-12-27 19:06:12,806][20381] Updated weights for policy 1, policy_version 1940667 (0.0005) [2023-12-27 19:06:12,860][20381] Updated weights for policy 1, policy_version 1940677 (0.0008) [2023-12-27 19:06:12,917][20381] Updated weights for policy 1, policy_version 1940687 (0.0005) [2023-12-27 19:06:13,410][20347] Updated weights for policy 0, policy_version 1942143 (0.0007) [2023-12-27 19:06:13,458][20347] Updated weights for policy 0, policy_version 1942153 (0.0007) [2023-12-27 19:06:13,508][20347] Updated weights for policy 0, policy_version 1942163 (0.0007) [2023-12-27 19:06:13,549][20381] Updated weights for policy 1, policy_version 1940697 (0.0010) [2023-12-27 19:06:13,601][20381] Updated weights for policy 1, policy_version 1940707 (0.0010) [2023-12-27 19:06:13,646][20381] Updated weights for policy 1, policy_version 1940717 (0.0010) [2023-12-27 19:06:14,193][20347] Updated weights for policy 0, policy_version 1942173 (0.0009) [2023-12-27 19:06:14,251][20347] Updated weights for policy 0, policy_version 1942183 (0.0007) [2023-12-27 19:06:14,306][20347] Updated weights for policy 0, policy_version 1942193 (0.0007) [2023-12-27 19:06:14,344][20381] Updated weights for policy 1, policy_version 1940727 (0.0010) [2023-12-27 19:06:14,389][20381] Updated weights for policy 1, policy_version 1940737 (0.0010) [2023-12-27 19:06:14,437][20381] Updated weights for policy 1, policy_version 1940747 (0.0010) [2023-12-27 19:06:15,007][20347] Updated weights for policy 0, policy_version 1942203 (0.0006) [2023-12-27 19:06:15,059][20347] Updated weights for policy 0, policy_version 1942213 (0.0007) [2023-12-27 19:06:15,109][20347] Updated weights for policy 0, policy_version 1942223 (0.0007) [2023-12-27 19:06:15,173][20381] Updated weights for policy 1, policy_version 1940757 (0.0009) [2023-12-27 19:06:15,228][20381] Updated weights for policy 1, policy_version 1940767 (0.0007) [2023-12-27 19:06:15,284][20381] Updated weights for policy 1, policy_version 1940777 (0.0008) [2023-12-27 19:06:15,755][20347] Updated weights for policy 0, policy_version 1942233 (0.0008) [2023-12-27 19:06:15,809][20347] Updated weights for policy 0, policy_version 1942243 (0.0005) [2023-12-27 19:06:15,854][20347] Updated weights for policy 0, policy_version 1942253 (0.0005) [2023-12-27 19:06:15,919][20347] Updated weights for policy 0, policy_version 1942263 (0.0008) [2023-12-27 19:06:15,947][20381] Updated weights for policy 1, policy_version 1940787 (0.0007) [2023-12-27 19:06:16,003][20381] Updated weights for policy 1, policy_version 1940797 (0.0006) [2023-12-27 19:06:16,063][20381] Updated weights for policy 1, policy_version 1940807 (0.0006) [2023-12-27 19:06:16,506][20347] Updated weights for policy 0, policy_version 1942273 (0.0007) [2023-12-27 19:06:16,570][20347] Updated weights for policy 0, policy_version 1942283 (0.0007) [2023-12-27 19:06:16,629][20347] Updated weights for policy 0, policy_version 1942293 (0.0006) [2023-12-27 19:06:16,721][20381] Updated weights for policy 1, policy_version 1940817 (0.0006) [2023-12-27 19:06:16,770][20381] Updated weights for policy 1, policy_version 1940827 (0.0008) [2023-12-27 19:06:16,828][20381] Updated weights for policy 1, policy_version 1940837 (0.0009) [2023-12-27 19:06:16,885][20381] Updated weights for policy 1, policy_version 1940847 (0.0005) [2023-12-27 19:06:17,227][20347] Updated weights for policy 0, policy_version 1942303 (0.0006) [2023-12-27 19:06:17,282][20347] Updated weights for policy 0, policy_version 1942313 (0.0005) [2023-12-27 19:06:17,342][20347] Updated weights for policy 0, policy_version 1942323 (0.0005) [2023-12-27 19:06:17,362][19170] Fps is (10 sec: 19661.2, 60 sec: 20206.9, 300 sec: 20577.2). Total num frames: 994238464. Throughput: 0: 10329.2, 1: 10147.4. Samples: 994254356. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '861.650')] [2023-12-27 19:06:17,559][20381] Updated weights for policy 1, policy_version 1940857 (0.0006) [2023-12-27 19:06:17,621][20381] Updated weights for policy 1, policy_version 1940867 (0.0005) [2023-12-27 19:06:17,684][20381] Updated weights for policy 1, policy_version 1940877 (0.0008) [2023-12-27 19:06:17,909][20347] Updated weights for policy 0, policy_version 1942333 (0.0005) [2023-12-27 19:06:17,966][20347] Updated weights for policy 0, policy_version 1942343 (0.0005) [2023-12-27 19:06:18,016][20347] Updated weights for policy 0, policy_version 1942353 (0.0005) [2023-12-27 19:06:18,249][20381] Updated weights for policy 1, policy_version 1940887 (0.0007) [2023-12-27 19:06:18,302][20381] Updated weights for policy 1, policy_version 1940897 (0.0008) [2023-12-27 19:06:18,364][20381] Updated weights for policy 1, policy_version 1940907 (0.0007) [2023-12-27 19:06:18,585][20347] Updated weights for policy 0, policy_version 1942363 (0.0007) [2023-12-27 19:06:18,637][20347] Updated weights for policy 0, policy_version 1942373 (0.0009) [2023-12-27 19:06:18,696][20347] Updated weights for policy 0, policy_version 1942383 (0.0010) [2023-12-27 19:06:19,025][20381] Updated weights for policy 1, policy_version 1940917 (0.0007) [2023-12-27 19:06:19,082][20381] Updated weights for policy 1, policy_version 1940927 (0.0006) [2023-12-27 19:06:19,140][20381] Updated weights for policy 1, policy_version 1940937 (0.0005) [2023-12-27 19:06:19,331][20347] Updated weights for policy 0, policy_version 1942393 (0.0010) [2023-12-27 19:06:19,399][20347] Updated weights for policy 0, policy_version 1942403 (0.0009) [2023-12-27 19:06:19,455][20347] Updated weights for policy 0, policy_version 1942413 (0.0007) [2023-12-27 19:06:19,520][20347] Updated weights for policy 0, policy_version 1942423 (0.0007) [2023-12-27 19:06:19,831][20381] Updated weights for policy 1, policy_version 1940947 (0.0006) [2023-12-27 19:06:19,893][20381] Updated weights for policy 1, policy_version 1940957 (0.0008) [2023-12-27 19:06:19,956][20381] Updated weights for policy 1, policy_version 1940967 (0.0008) [2023-12-27 19:06:20,260][20347] Updated weights for policy 0, policy_version 1942433 (0.0007) [2023-12-27 19:06:20,319][20347] Updated weights for policy 0, policy_version 1942443 (0.0007) [2023-12-27 19:06:20,370][20347] Updated weights for policy 0, policy_version 1942453 (0.0008) [2023-12-27 19:06:20,630][20381] Updated weights for policy 1, policy_version 1940977 (0.0008) [2023-12-27 19:06:20,692][20381] Updated weights for policy 1, policy_version 1940987 (0.0007) [2023-12-27 19:06:20,753][20381] Updated weights for policy 1, policy_version 1940997 (0.0008) [2023-12-27 19:06:20,819][20381] Updated weights for policy 1, policy_version 1941007 (0.0007) [2023-12-27 19:06:21,117][20347] Updated weights for policy 0, policy_version 1942463 (0.0008) [2023-12-27 19:06:21,182][20347] Updated weights for policy 0, policy_version 1942473 (0.0007) [2023-12-27 19:06:21,246][20347] Updated weights for policy 0, policy_version 1942483 (0.0008) [2023-12-27 19:06:21,497][20381] Updated weights for policy 1, policy_version 1941017 (0.0010) [2023-12-27 19:06:21,555][20381] Updated weights for policy 1, policy_version 1941027 (0.0011) [2023-12-27 19:06:21,615][20381] Updated weights for policy 1, policy_version 1941037 (0.0011) [2023-12-27 19:06:22,003][20347] Updated weights for policy 0, policy_version 1942493 (0.0009) [2023-12-27 19:06:22,053][20347] Updated weights for policy 0, policy_version 1942503 (0.0007) [2023-12-27 19:06:22,108][20347] Updated weights for policy 0, policy_version 1942513 (0.0007) [2023-12-27 19:06:22,321][20381] Updated weights for policy 1, policy_version 1941047 (0.0011) [2023-12-27 19:06:22,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20343.8, 300 sec: 20577.2). Total num frames: 994344960. Throughput: 0: 10337.8, 1: 10166.7. Samples: 994318364. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.930')] [2023-12-27 19:06:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001942520_497377280.pth... [2023-12-27 19:06:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001941304_497065984.pth [2023-12-27 19:06:22,383][20381] Updated weights for policy 1, policy_version 1941057 (0.0010) [2023-12-27 19:06:22,444][20381] Updated weights for policy 1, policy_version 1941067 (0.0009) [2023-12-27 19:06:22,472][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001941072_496975872.pth... [2023-12-27 19:06:22,476][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001939856_496664576.pth [2023-12-27 19:06:22,900][20347] Updated weights for policy 0, policy_version 1942523 (0.0008) [2023-12-27 19:06:22,952][20347] Updated weights for policy 0, policy_version 1942533 (0.0008) [2023-12-27 19:06:23,008][20347] Updated weights for policy 0, policy_version 1942543 (0.0008) [2023-12-27 19:06:23,180][20381] Updated weights for policy 1, policy_version 1941077 (0.0007) [2023-12-27 19:06:23,228][20381] Updated weights for policy 1, policy_version 1941087 (0.0005) [2023-12-27 19:06:23,280][20381] Updated weights for policy 1, policy_version 1941097 (0.0005) [2023-12-27 19:06:23,633][20347] Updated weights for policy 0, policy_version 1942553 (0.0007) [2023-12-27 19:06:23,700][20347] Updated weights for policy 0, policy_version 1942563 (0.0007) [2023-12-27 19:06:23,751][20347] Updated weights for policy 0, policy_version 1942573 (0.0005) [2023-12-27 19:06:23,805][20347] Updated weights for policy 0, policy_version 1942583 (0.0007) [2023-12-27 19:06:23,875][20381] Updated weights for policy 1, policy_version 1941107 (0.0006) [2023-12-27 19:06:23,939][20381] Updated weights for policy 1, policy_version 1941117 (0.0008) [2023-12-27 19:06:23,992][20381] Updated weights for policy 1, policy_version 1941127 (0.0008) [2023-12-27 19:06:24,479][20347] Updated weights for policy 0, policy_version 1942593 (0.0009) [2023-12-27 19:06:24,523][20347] Updated weights for policy 0, policy_version 1942603 (0.0007) [2023-12-27 19:06:24,581][20347] Updated weights for policy 0, policy_version 1942613 (0.0005) [2023-12-27 19:06:24,674][20381] Updated weights for policy 1, policy_version 1941137 (0.0008) [2023-12-27 19:06:24,733][20381] Updated weights for policy 1, policy_version 1941147 (0.0007) [2023-12-27 19:06:24,791][20381] Updated weights for policy 1, policy_version 1941157 (0.0008) [2023-12-27 19:06:24,845][20381] Updated weights for policy 1, policy_version 1941167 (0.0008) [2023-12-27 19:06:25,204][20347] Updated weights for policy 0, policy_version 1942623 (0.0006) [2023-12-27 19:06:25,261][20347] Updated weights for policy 0, policy_version 1942633 (0.0005) [2023-12-27 19:06:25,326][20347] Updated weights for policy 0, policy_version 1942643 (0.0005) [2023-12-27 19:06:25,507][20381] Updated weights for policy 1, policy_version 1941177 (0.0006) [2023-12-27 19:06:25,563][20381] Updated weights for policy 1, policy_version 1941187 (0.0007) [2023-12-27 19:06:25,630][20381] Updated weights for policy 1, policy_version 1941197 (0.0008) [2023-12-27 19:06:25,868][20347] Updated weights for policy 0, policy_version 1942653 (0.0006) [2023-12-27 19:06:25,922][20347] Updated weights for policy 0, policy_version 1942663 (0.0007) [2023-12-27 19:06:25,972][20347] Updated weights for policy 0, policy_version 1942673 (0.0010) [2023-12-27 19:06:26,227][20381] Updated weights for policy 1, policy_version 1941207 (0.0007) [2023-12-27 19:06:26,282][20381] Updated weights for policy 1, policy_version 1941217 (0.0008) [2023-12-27 19:06:26,328][20381] Updated weights for policy 1, policy_version 1941227 (0.0008) [2023-12-27 19:06:26,697][20347] Updated weights for policy 0, policy_version 1942683 (0.0009) [2023-12-27 19:06:26,743][20347] Updated weights for policy 0, policy_version 1942693 (0.0010) [2023-12-27 19:06:26,796][20347] Updated weights for policy 0, policy_version 1942703 (0.0008) [2023-12-27 19:06:26,928][20381] Updated weights for policy 1, policy_version 1941237 (0.0006) [2023-12-27 19:06:26,987][20381] Updated weights for policy 1, policy_version 1941247 (0.0005) [2023-12-27 19:06:27,048][20381] Updated weights for policy 1, policy_version 1941257 (0.0005) [2023-12-27 19:06:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20605.0). Total num frames: 994451456. Throughput: 0: 10282.4, 1: 10288.9. Samples: 994441220. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '911.920')] [2023-12-27 19:06:27,530][20347] Updated weights for policy 0, policy_version 1942713 (0.0008) [2023-12-27 19:06:27,574][20347] Updated weights for policy 0, policy_version 1942723 (0.0010) [2023-12-27 19:06:27,612][20381] Updated weights for policy 1, policy_version 1941267 (0.0006) [2023-12-27 19:06:27,618][20347] Updated weights for policy 0, policy_version 1942733 (0.0010) [2023-12-27 19:06:27,670][20381] Updated weights for policy 1, policy_version 1941277 (0.0006) [2023-12-27 19:06:27,673][20347] Updated weights for policy 0, policy_version 1942743 (0.0010) [2023-12-27 19:06:27,725][20381] Updated weights for policy 1, policy_version 1941287 (0.0007) [2023-12-27 19:06:28,349][20347] Updated weights for policy 0, policy_version 1942753 (0.0008) [2023-12-27 19:06:28,405][20347] Updated weights for policy 0, policy_version 1942763 (0.0008) [2023-12-27 19:06:28,430][20381] Updated weights for policy 1, policy_version 1941297 (0.0008) [2023-12-27 19:06:28,463][20347] Updated weights for policy 0, policy_version 1942773 (0.0007) [2023-12-27 19:06:28,476][20381] Updated weights for policy 1, policy_version 1941307 (0.0006) [2023-12-27 19:06:28,530][20381] Updated weights for policy 1, policy_version 1941317 (0.0008) [2023-12-27 19:06:28,588][20381] Updated weights for policy 1, policy_version 1941327 (0.0008) [2023-12-27 19:06:29,148][20347] Updated weights for policy 0, policy_version 1942783 (0.0007) [2023-12-27 19:06:29,193][20347] Updated weights for policy 0, policy_version 1942793 (0.0008) [2023-12-27 19:06:29,249][20347] Updated weights for policy 0, policy_version 1942803 (0.0007) [2023-12-27 19:06:29,314][20381] Updated weights for policy 1, policy_version 1941337 (0.0007) [2023-12-27 19:06:29,380][20381] Updated weights for policy 1, policy_version 1941347 (0.0007) [2023-12-27 19:06:29,438][20381] Updated weights for policy 1, policy_version 1941357 (0.0006) [2023-12-27 19:06:30,013][20347] Updated weights for policy 0, policy_version 1942813 (0.0007) [2023-12-27 19:06:30,066][20347] Updated weights for policy 0, policy_version 1942823 (0.0007) [2023-12-27 19:06:30,122][20347] Updated weights for policy 0, policy_version 1942833 (0.0006) [2023-12-27 19:06:30,122][20381] Updated weights for policy 1, policy_version 1941367 (0.0006) [2023-12-27 19:06:30,178][20381] Updated weights for policy 1, policy_version 1941377 (0.0007) [2023-12-27 19:06:30,240][20381] Updated weights for policy 1, policy_version 1941387 (0.0006) [2023-12-27 19:06:30,745][20347] Updated weights for policy 0, policy_version 1942843 (0.0005) [2023-12-27 19:06:30,781][20381] Updated weights for policy 1, policy_version 1941397 (0.0007) [2023-12-27 19:06:30,802][20347] Updated weights for policy 0, policy_version 1942853 (0.0005) [2023-12-27 19:06:30,834][20381] Updated weights for policy 1, policy_version 1941407 (0.0007) [2023-12-27 19:06:30,855][20347] Updated weights for policy 0, policy_version 1942863 (0.0005) [2023-12-27 19:06:30,887][20381] Updated weights for policy 1, policy_version 1941417 (0.0006) [2023-12-27 19:06:31,526][20347] Updated weights for policy 0, policy_version 1942873 (0.0006) [2023-12-27 19:06:31,571][20347] Updated weights for policy 0, policy_version 1942883 (0.0006) [2023-12-27 19:06:31,627][20347] Updated weights for policy 0, policy_version 1942893 (0.0007) [2023-12-27 19:06:31,639][20381] Updated weights for policy 1, policy_version 1941427 (0.0007) [2023-12-27 19:06:31,683][20347] Updated weights for policy 0, policy_version 1942903 (0.0008) [2023-12-27 19:06:31,702][20381] Updated weights for policy 1, policy_version 1941437 (0.0008) [2023-12-27 19:06:31,768][20381] Updated weights for policy 1, policy_version 1941447 (0.0008) [2023-12-27 19:06:32,344][20347] Updated weights for policy 0, policy_version 1942913 (0.0007) [2023-12-27 19:06:32,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 994549760. Throughput: 0: 10263.7, 1: 10332.6. Samples: 994564436. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.280')] [2023-12-27 19:06:32,406][20347] Updated weights for policy 0, policy_version 1942923 (0.0007) [2023-12-27 19:06:32,462][20347] Updated weights for policy 0, policy_version 1942933 (0.0007) [2023-12-27 19:06:32,540][20381] Updated weights for policy 1, policy_version 1941457 (0.0008) [2023-12-27 19:06:32,593][20381] Updated weights for policy 1, policy_version 1941467 (0.0006) [2023-12-27 19:06:32,648][20381] Updated weights for policy 1, policy_version 1941477 (0.0006) [2023-12-27 19:06:32,705][20381] Updated weights for policy 1, policy_version 1941487 (0.0008) [2023-12-27 19:06:33,155][20347] Updated weights for policy 0, policy_version 1942943 (0.0008) [2023-12-27 19:06:33,208][20347] Updated weights for policy 0, policy_version 1942953 (0.0007) [2023-12-27 19:06:33,253][20347] Updated weights for policy 0, policy_version 1942963 (0.0008) [2023-12-27 19:06:33,354][20381] Updated weights for policy 1, policy_version 1941497 (0.0005) [2023-12-27 19:06:33,414][20381] Updated weights for policy 1, policy_version 1941507 (0.0005) [2023-12-27 19:06:33,471][20381] Updated weights for policy 1, policy_version 1941517 (0.0005) [2023-12-27 19:06:33,960][20347] Updated weights for policy 0, policy_version 1942973 (0.0008) [2023-12-27 19:06:34,014][20347] Updated weights for policy 0, policy_version 1942983 (0.0007) [2023-12-27 19:06:34,072][20347] Updated weights for policy 0, policy_version 1942993 (0.0008) [2023-12-27 19:06:34,111][20381] Updated weights for policy 1, policy_version 1941527 (0.0006) [2023-12-27 19:06:34,170][20381] Updated weights for policy 1, policy_version 1941537 (0.0008) [2023-12-27 19:06:34,231][20381] Updated weights for policy 1, policy_version 1941547 (0.0008) [2023-12-27 19:06:34,784][20347] Updated weights for policy 0, policy_version 1943003 (0.0005) [2023-12-27 19:06:34,835][20347] Updated weights for policy 0, policy_version 1943013 (0.0005) [2023-12-27 19:06:34,900][20347] Updated weights for policy 0, policy_version 1943023 (0.0008) [2023-12-27 19:06:34,914][20381] Updated weights for policy 1, policy_version 1941557 (0.0006) [2023-12-27 19:06:34,965][20381] Updated weights for policy 1, policy_version 1941567 (0.0007) [2023-12-27 19:06:35,012][20381] Updated weights for policy 1, policy_version 1941577 (0.0008) [2023-12-27 19:06:35,543][20347] Updated weights for policy 0, policy_version 1943033 (0.0007) [2023-12-27 19:06:35,591][20347] Updated weights for policy 0, policy_version 1943043 (0.0005) [2023-12-27 19:06:35,640][20347] Updated weights for policy 0, policy_version 1943053 (0.0005) [2023-12-27 19:06:35,678][20381] Updated weights for policy 1, policy_version 1941587 (0.0007) [2023-12-27 19:06:35,694][20347] Updated weights for policy 0, policy_version 1943063 (0.0005) [2023-12-27 19:06:35,734][20381] Updated weights for policy 1, policy_version 1941597 (0.0010) [2023-12-27 19:06:35,791][20381] Updated weights for policy 1, policy_version 1941607 (0.0006) [2023-12-27 19:06:36,262][20347] Updated weights for policy 0, policy_version 1943073 (0.0008) [2023-12-27 19:06:36,322][20347] Updated weights for policy 0, policy_version 1943083 (0.0008) [2023-12-27 19:06:36,380][20347] Updated weights for policy 0, policy_version 1943093 (0.0007) [2023-12-27 19:06:36,426][20381] Updated weights for policy 1, policy_version 1941617 (0.0008) [2023-12-27 19:06:36,485][20381] Updated weights for policy 1, policy_version 1941627 (0.0006) [2023-12-27 19:06:36,545][20381] Updated weights for policy 1, policy_version 1941637 (0.0008) [2023-12-27 19:06:36,600][20381] Updated weights for policy 1, policy_version 1941647 (0.0006) [2023-12-27 19:06:37,140][20347] Updated weights for policy 0, policy_version 1943103 (0.0007) [2023-12-27 19:06:37,175][20381] Updated weights for policy 1, policy_version 1941657 (0.0006) [2023-12-27 19:06:37,198][20347] Updated weights for policy 0, policy_version 1943113 (0.0007) [2023-12-27 19:06:37,232][20381] Updated weights for policy 1, policy_version 1941667 (0.0007) [2023-12-27 19:06:37,256][20347] Updated weights for policy 0, policy_version 1943123 (0.0007) [2023-12-27 19:06:37,293][20381] Updated weights for policy 1, policy_version 1941677 (0.0007) [2023-12-27 19:06:37,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 994664448. Throughput: 0: 10264.0, 1: 10352.5. Samples: 994625632. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '930.540')] [2023-12-27 19:06:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001941680_497131520.pth... [2023-12-27 19:06:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001943128_497532928.pth... [2023-12-27 19:06:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001941912_497221632.pth [2023-12-27 19:06:37,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001940464_496820224.pth [2023-12-27 19:06:37,909][20381] Updated weights for policy 1, policy_version 1941687 (0.0008) [2023-12-27 19:06:37,961][20381] Updated weights for policy 1, policy_version 1941697 (0.0008) [2023-12-27 19:06:37,978][20347] Updated weights for policy 0, policy_version 1943133 (0.0006) [2023-12-27 19:06:38,017][20381] Updated weights for policy 1, policy_version 1941707 (0.0005) [2023-12-27 19:06:38,031][20347] Updated weights for policy 0, policy_version 1943143 (0.0006) [2023-12-27 19:06:38,085][20347] Updated weights for policy 0, policy_version 1943153 (0.0007) [2023-12-27 19:06:38,686][20381] Updated weights for policy 1, policy_version 1941717 (0.0007) [2023-12-27 19:06:38,748][20381] Updated weights for policy 1, policy_version 1941727 (0.0008) [2023-12-27 19:06:38,813][20381] Updated weights for policy 1, policy_version 1941737 (0.0008) [2023-12-27 19:06:38,823][20347] Updated weights for policy 0, policy_version 1943163 (0.0007) [2023-12-27 19:06:38,877][20347] Updated weights for policy 0, policy_version 1943173 (0.0006) [2023-12-27 19:06:38,933][20347] Updated weights for policy 0, policy_version 1943183 (0.0007) [2023-12-27 19:06:39,516][20381] Updated weights for policy 1, policy_version 1941747 (0.0006) [2023-12-27 19:06:39,578][20381] Updated weights for policy 1, policy_version 1941757 (0.0006) [2023-12-27 19:06:39,640][20381] Updated weights for policy 1, policy_version 1941767 (0.0007) [2023-12-27 19:06:39,650][20347] Updated weights for policy 0, policy_version 1943193 (0.0007) [2023-12-27 19:06:39,710][20347] Updated weights for policy 0, policy_version 1943203 (0.0007) [2023-12-27 19:06:39,777][20347] Updated weights for policy 0, policy_version 1943213 (0.0007) [2023-12-27 19:06:39,835][20347] Updated weights for policy 0, policy_version 1943223 (0.0009) [2023-12-27 19:06:40,306][20381] Updated weights for policy 1, policy_version 1941777 (0.0007) [2023-12-27 19:06:40,366][20381] Updated weights for policy 1, policy_version 1941787 (0.0009) [2023-12-27 19:06:40,424][20381] Updated weights for policy 1, policy_version 1941797 (0.0008) [2023-12-27 19:06:40,473][20381] Updated weights for policy 1, policy_version 1941807 (0.0007) [2023-12-27 19:06:40,525][20347] Updated weights for policy 0, policy_version 1943233 (0.0006) [2023-12-27 19:06:40,582][20347] Updated weights for policy 0, policy_version 1943243 (0.0008) [2023-12-27 19:06:40,643][20347] Updated weights for policy 0, policy_version 1943253 (0.0007) [2023-12-27 19:06:41,188][20381] Updated weights for policy 1, policy_version 1941817 (0.0009) [2023-12-27 19:06:41,243][20381] Updated weights for policy 1, policy_version 1941827 (0.0008) [2023-12-27 19:06:41,304][20381] Updated weights for policy 1, policy_version 1941837 (0.0008) [2023-12-27 19:06:41,379][20347] Updated weights for policy 0, policy_version 1943263 (0.0007) [2023-12-27 19:06:41,441][20347] Updated weights for policy 0, policy_version 1943273 (0.0007) [2023-12-27 19:06:41,497][20347] Updated weights for policy 0, policy_version 1943283 (0.0006) [2023-12-27 19:06:42,071][20381] Updated weights for policy 1, policy_version 1941847 (0.0008) [2023-12-27 19:06:42,127][20381] Updated weights for policy 1, policy_version 1941857 (0.0006) [2023-12-27 19:06:42,181][20347] Updated weights for policy 0, policy_version 1943293 (0.0007) [2023-12-27 19:06:42,187][20381] Updated weights for policy 1, policy_version 1941867 (0.0006) [2023-12-27 19:06:42,238][20347] Updated weights for policy 0, policy_version 1943303 (0.0007) [2023-12-27 19:06:42,301][20347] Updated weights for policy 0, policy_version 1943313 (0.0008) [2023-12-27 19:06:42,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 994762752. Throughput: 0: 10245.4, 1: 10357.9. Samples: 994748532. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.330')] [2023-12-27 19:06:42,887][20381] Updated weights for policy 1, policy_version 1941877 (0.0007) [2023-12-27 19:06:42,935][20381] Updated weights for policy 1, policy_version 1941887 (0.0008) [2023-12-27 19:06:42,987][20381] Updated weights for policy 1, policy_version 1941897 (0.0006) [2023-12-27 19:06:42,993][20347] Updated weights for policy 0, policy_version 1943323 (0.0007) [2023-12-27 19:06:43,043][20347] Updated weights for policy 0, policy_version 1943333 (0.0007) [2023-12-27 19:06:43,088][20347] Updated weights for policy 0, policy_version 1943343 (0.0008) [2023-12-27 19:06:43,660][20381] Updated weights for policy 1, policy_version 1941907 (0.0005) [2023-12-27 19:06:43,710][20381] Updated weights for policy 1, policy_version 1941917 (0.0005) [2023-12-27 19:06:43,753][20347] Updated weights for policy 0, policy_version 1943353 (0.0007) [2023-12-27 19:06:43,763][20381] Updated weights for policy 1, policy_version 1941927 (0.0007) [2023-12-27 19:06:43,813][20347] Updated weights for policy 0, policy_version 1943363 (0.0005) [2023-12-27 19:06:43,876][20347] Updated weights for policy 0, policy_version 1943373 (0.0005) [2023-12-27 19:06:43,937][20347] Updated weights for policy 0, policy_version 1943383 (0.0006) [2023-12-27 19:06:44,350][20381] Updated weights for policy 1, policy_version 1941937 (0.0008) [2023-12-27 19:06:44,402][20381] Updated weights for policy 1, policy_version 1941947 (0.0008) [2023-12-27 19:06:44,457][20381] Updated weights for policy 1, policy_version 1941957 (0.0008) [2023-12-27 19:06:44,512][20381] Updated weights for policy 1, policy_version 1941967 (0.0007) [2023-12-27 19:06:44,566][20347] Updated weights for policy 0, policy_version 1943393 (0.0008) [2023-12-27 19:06:44,629][20347] Updated weights for policy 0, policy_version 1943403 (0.0007) [2023-12-27 19:06:44,695][20347] Updated weights for policy 0, policy_version 1943413 (0.0007) [2023-12-27 19:06:45,244][20381] Updated weights for policy 1, policy_version 1941977 (0.0007) [2023-12-27 19:06:45,307][20381] Updated weights for policy 1, policy_version 1941987 (0.0007) [2023-12-27 19:06:45,371][20381] Updated weights for policy 1, policy_version 1941997 (0.0007) [2023-12-27 19:06:45,398][20347] Updated weights for policy 0, policy_version 1943423 (0.0006) [2023-12-27 19:06:45,456][20347] Updated weights for policy 0, policy_version 1943433 (0.0008) [2023-12-27 19:06:45,507][20347] Updated weights for policy 0, policy_version 1943443 (0.0008) [2023-12-27 19:06:46,077][20381] Updated weights for policy 1, policy_version 1942007 (0.0007) [2023-12-27 19:06:46,085][20347] Updated weights for policy 0, policy_version 1943453 (0.0006) [2023-12-27 19:06:46,128][20381] Updated weights for policy 1, policy_version 1942017 (0.0008) [2023-12-27 19:06:46,135][20347] Updated weights for policy 0, policy_version 1943463 (0.0006) [2023-12-27 19:06:46,185][20381] Updated weights for policy 1, policy_version 1942027 (0.0006) [2023-12-27 19:06:46,195][20347] Updated weights for policy 0, policy_version 1943473 (0.0005) [2023-12-27 19:06:46,812][20381] Updated weights for policy 1, policy_version 1942037 (0.0005) [2023-12-27 19:06:46,861][20381] Updated weights for policy 1, policy_version 1942047 (0.0005) [2023-12-27 19:06:46,902][20347] Updated weights for policy 0, policy_version 1943483 (0.0007) [2023-12-27 19:06:46,914][20381] Updated weights for policy 1, policy_version 1942057 (0.0005) [2023-12-27 19:06:46,955][20347] Updated weights for policy 0, policy_version 1943493 (0.0008) [2023-12-27 19:06:47,003][20347] Updated weights for policy 0, policy_version 1943503 (0.0007) [2023-12-27 19:06:47,362][19170] Fps is (10 sec: 19660.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 994861056. Throughput: 0: 10199.2, 1: 10373.9. Samples: 994871984. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '871.680')] [2023-12-27 19:06:47,550][20381] Updated weights for policy 1, policy_version 1942067 (0.0007) [2023-12-27 19:06:47,598][20381] Updated weights for policy 1, policy_version 1942077 (0.0010) [2023-12-27 19:06:47,651][20381] Updated weights for policy 1, policy_version 1942087 (0.0010) [2023-12-27 19:06:47,713][20347] Updated weights for policy 0, policy_version 1943513 (0.0007) [2023-12-27 19:06:47,776][20347] Updated weights for policy 0, policy_version 1943523 (0.0005) [2023-12-27 19:06:47,836][20347] Updated weights for policy 0, policy_version 1943533 (0.0005) [2023-12-27 19:06:47,888][20347] Updated weights for policy 0, policy_version 1943543 (0.0005) [2023-12-27 19:06:48,267][20381] Updated weights for policy 1, policy_version 1942097 (0.0009) [2023-12-27 19:06:48,320][20381] Updated weights for policy 1, policy_version 1942107 (0.0006) [2023-12-27 19:06:48,383][20381] Updated weights for policy 1, policy_version 1942117 (0.0008) [2023-12-27 19:06:48,440][20381] Updated weights for policy 1, policy_version 1942127 (0.0008) [2023-12-27 19:06:48,518][20347] Updated weights for policy 0, policy_version 1943553 (0.0005) [2023-12-27 19:06:48,565][20347] Updated weights for policy 0, policy_version 1943563 (0.0005) [2023-12-27 19:06:48,613][20347] Updated weights for policy 0, policy_version 1943573 (0.0007) [2023-12-27 19:06:49,159][20381] Updated weights for policy 1, policy_version 1942137 (0.0008) [2023-12-27 19:06:49,210][20381] Updated weights for policy 1, policy_version 1942147 (0.0007) [2023-12-27 19:06:49,273][20381] Updated weights for policy 1, policy_version 1942157 (0.0007) [2023-12-27 19:06:49,355][20347] Updated weights for policy 0, policy_version 1943583 (0.0007) [2023-12-27 19:06:49,421][20347] Updated weights for policy 0, policy_version 1943593 (0.0011) [2023-12-27 19:06:49,486][20347] Updated weights for policy 0, policy_version 1943603 (0.0008) [2023-12-27 19:06:50,002][20381] Updated weights for policy 1, policy_version 1942167 (0.0008) [2023-12-27 19:06:50,060][20381] Updated weights for policy 1, policy_version 1942177 (0.0008) [2023-12-27 19:06:50,116][20381] Updated weights for policy 1, policy_version 1942187 (0.0008) [2023-12-27 19:06:50,187][20347] Updated weights for policy 0, policy_version 1943613 (0.0007) [2023-12-27 19:06:50,234][20347] Updated weights for policy 0, policy_version 1943623 (0.0007) [2023-12-27 19:06:50,288][20347] Updated weights for policy 0, policy_version 1943633 (0.0008) [2023-12-27 19:06:50,844][20381] Updated weights for policy 1, policy_version 1942197 (0.0007) [2023-12-27 19:06:50,889][20381] Updated weights for policy 1, policy_version 1942207 (0.0007) [2023-12-27 19:06:50,939][20381] Updated weights for policy 1, policy_version 1942217 (0.0008) [2023-12-27 19:06:50,989][20347] Updated weights for policy 0, policy_version 1943643 (0.0006) [2023-12-27 19:06:51,062][20347] Updated weights for policy 0, policy_version 1943653 (0.0008) [2023-12-27 19:06:51,123][20347] Updated weights for policy 0, policy_version 1943663 (0.0007) [2023-12-27 19:06:51,633][20381] Updated weights for policy 1, policy_version 1942227 (0.0007) [2023-12-27 19:06:51,698][20381] Updated weights for policy 1, policy_version 1942237 (0.0007) [2023-12-27 19:06:51,765][20381] Updated weights for policy 1, policy_version 1942247 (0.0008) [2023-12-27 19:06:51,827][20347] Updated weights for policy 0, policy_version 1943673 (0.0007) [2023-12-27 19:06:51,874][20347] Updated weights for policy 0, policy_version 1943683 (0.0006) [2023-12-27 19:06:51,919][20347] Updated weights for policy 0, policy_version 1943693 (0.0005) [2023-12-27 19:06:51,978][20347] Updated weights for policy 0, policy_version 1943703 (0.0005) [2023-12-27 19:06:52,362][19170] Fps is (10 sec: 19660.7, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 994959360. Throughput: 0: 10206.4, 1: 10314.5. Samples: 994932528. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.200')] [2023-12-27 19:06:52,371][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001942256_497278976.pth... [2023-12-27 19:06:52,372][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001943704_497680384.pth... [2023-12-27 19:06:52,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001941072_496975872.pth [2023-12-27 19:06:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001942520_497377280.pth [2023-12-27 19:06:52,547][20381] Updated weights for policy 1, policy_version 1942257 (0.0008) [2023-12-27 19:06:52,604][20381] Updated weights for policy 1, policy_version 1942267 (0.0008) [2023-12-27 19:06:52,659][20381] Updated weights for policy 1, policy_version 1942277 (0.0007) [2023-12-27 19:06:52,687][20347] Updated weights for policy 0, policy_version 1943713 (0.0008) [2023-12-27 19:06:52,712][20381] Updated weights for policy 1, policy_version 1942287 (0.0008) [2023-12-27 19:06:52,751][20347] Updated weights for policy 0, policy_version 1943723 (0.0008) [2023-12-27 19:06:52,810][20347] Updated weights for policy 0, policy_version 1943733 (0.0008) [2023-12-27 19:06:53,391][20381] Updated weights for policy 1, policy_version 1942297 (0.0006) [2023-12-27 19:06:53,453][20381] Updated weights for policy 1, policy_version 1942307 (0.0005) [2023-12-27 19:06:53,481][20347] Updated weights for policy 0, policy_version 1943743 (0.0007) [2023-12-27 19:06:53,508][20381] Updated weights for policy 1, policy_version 1942317 (0.0005) [2023-12-27 19:06:53,537][20347] Updated weights for policy 0, policy_version 1943753 (0.0007) [2023-12-27 19:06:53,592][20347] Updated weights for policy 0, policy_version 1943763 (0.0008) [2023-12-27 19:06:54,062][20381] Updated weights for policy 1, policy_version 1942327 (0.0007) [2023-12-27 19:06:54,111][20381] Updated weights for policy 1, policy_version 1942337 (0.0007) [2023-12-27 19:06:54,172][20381] Updated weights for policy 1, policy_version 1942347 (0.0007) [2023-12-27 19:06:54,299][20347] Updated weights for policy 0, policy_version 1943773 (0.0007) [2023-12-27 19:06:54,365][20347] Updated weights for policy 0, policy_version 1943783 (0.0008) [2023-12-27 19:06:54,418][20347] Updated weights for policy 0, policy_version 1943793 (0.0007) [2023-12-27 19:06:54,836][20381] Updated weights for policy 1, policy_version 1942357 (0.0007) [2023-12-27 19:06:54,912][20381] Updated weights for policy 1, policy_version 1942367 (0.0008) [2023-12-27 19:06:54,967][20381] Updated weights for policy 1, policy_version 1942377 (0.0007) [2023-12-27 19:06:55,121][20347] Updated weights for policy 0, policy_version 1943803 (0.0007) [2023-12-27 19:06:55,187][20347] Updated weights for policy 0, policy_version 1943813 (0.0008) [2023-12-27 19:06:55,245][20347] Updated weights for policy 0, policy_version 1943823 (0.0007) [2023-12-27 19:06:55,613][20381] Updated weights for policy 1, policy_version 1942387 (0.0007) [2023-12-27 19:06:55,672][20381] Updated weights for policy 1, policy_version 1942397 (0.0008) [2023-12-27 19:06:55,723][20381] Updated weights for policy 1, policy_version 1942407 (0.0007) [2023-12-27 19:06:55,948][20347] Updated weights for policy 0, policy_version 1943833 (0.0007) [2023-12-27 19:06:56,007][20347] Updated weights for policy 0, policy_version 1943843 (0.0008) [2023-12-27 19:06:56,065][20347] Updated weights for policy 0, policy_version 1943853 (0.0008) [2023-12-27 19:06:56,123][20347] Updated weights for policy 0, policy_version 1943863 (0.0008) [2023-12-27 19:06:56,358][20381] Updated weights for policy 1, policy_version 1942417 (0.0008) [2023-12-27 19:06:56,414][20381] Updated weights for policy 1, policy_version 1942427 (0.0007) [2023-12-27 19:06:56,464][20381] Updated weights for policy 1, policy_version 1942437 (0.0008) [2023-12-27 19:06:56,518][20381] Updated weights for policy 1, policy_version 1942447 (0.0008) [2023-12-27 19:06:56,778][20347] Updated weights for policy 0, policy_version 1943873 (0.0006) [2023-12-27 19:06:56,829][20347] Updated weights for policy 0, policy_version 1943883 (0.0005) [2023-12-27 19:06:56,883][20347] Updated weights for policy 0, policy_version 1943893 (0.0005) [2023-12-27 19:06:57,132][20381] Updated weights for policy 1, policy_version 1942457 (0.0009) [2023-12-27 19:06:57,186][20381] Updated weights for policy 1, policy_version 1942467 (0.0011) [2023-12-27 19:06:57,241][20381] Updated weights for policy 1, policy_version 1942477 (0.0010) [2023-12-27 19:06:57,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20479.9, 300 sec: 20577.2). Total num frames: 995065856. Throughput: 0: 10183.4, 1: 10329.1. Samples: 995053844. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:06:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.370')] [2023-12-27 19:06:57,421][20347] Updated weights for policy 0, policy_version 1943903 (0.0005) [2023-12-27 19:06:57,475][20347] Updated weights for policy 0, policy_version 1943913 (0.0009) [2023-12-27 19:06:57,519][20347] Updated weights for policy 0, policy_version 1943923 (0.0010) [2023-12-27 19:06:57,877][20381] Updated weights for policy 1, policy_version 1942487 (0.0009) [2023-12-27 19:06:57,942][20381] Updated weights for policy 1, policy_version 1942497 (0.0008) [2023-12-27 19:06:58,005][20381] Updated weights for policy 1, policy_version 1942507 (0.0007) [2023-12-27 19:06:58,166][20347] Updated weights for policy 0, policy_version 1943933 (0.0007) [2023-12-27 19:06:58,234][20347] Updated weights for policy 0, policy_version 1943943 (0.0009) [2023-12-27 19:06:58,292][20347] Updated weights for policy 0, policy_version 1943953 (0.0010) [2023-12-27 19:06:58,727][20381] Updated weights for policy 1, policy_version 1942517 (0.0008) [2023-12-27 19:06:58,795][20381] Updated weights for policy 1, policy_version 1942527 (0.0009) [2023-12-27 19:06:58,865][20381] Updated weights for policy 1, policy_version 1942537 (0.0007) [2023-12-27 19:06:59,072][20347] Updated weights for policy 0, policy_version 1943963 (0.0009) [2023-12-27 19:06:59,126][20347] Updated weights for policy 0, policy_version 1943973 (0.0007) [2023-12-27 19:06:59,188][20347] Updated weights for policy 0, policy_version 1943983 (0.0008) [2023-12-27 19:06:59,584][20381] Updated weights for policy 1, policy_version 1942547 (0.0008) [2023-12-27 19:06:59,644][20381] Updated weights for policy 1, policy_version 1942557 (0.0007) [2023-12-27 19:06:59,698][20381] Updated weights for policy 1, policy_version 1942567 (0.0008) [2023-12-27 19:06:59,870][20347] Updated weights for policy 0, policy_version 1943993 (0.0008) [2023-12-27 19:06:59,931][20347] Updated weights for policy 0, policy_version 1944003 (0.0009) [2023-12-27 19:07:00,000][20347] Updated weights for policy 0, policy_version 1944013 (0.0006) [2023-12-27 19:07:00,068][20347] Updated weights for policy 0, policy_version 1944023 (0.0007) [2023-12-27 19:07:00,387][20381] Updated weights for policy 1, policy_version 1942577 (0.0007) [2023-12-27 19:07:00,439][20381] Updated weights for policy 1, policy_version 1942587 (0.0009) [2023-12-27 19:07:00,487][20381] Updated weights for policy 1, policy_version 1942597 (0.0008) [2023-12-27 19:07:00,541][20381] Updated weights for policy 1, policy_version 1942607 (0.0008) [2023-12-27 19:07:00,654][20347] Updated weights for policy 0, policy_version 1944033 (0.0007) [2023-12-27 19:07:00,706][20347] Updated weights for policy 0, policy_version 1944043 (0.0010) [2023-12-27 19:07:00,754][20347] Updated weights for policy 0, policy_version 1944053 (0.0010) [2023-12-27 19:07:01,176][20381] Updated weights for policy 1, policy_version 1942617 (0.0008) [2023-12-27 19:07:01,239][20381] Updated weights for policy 1, policy_version 1942627 (0.0009) [2023-12-27 19:07:01,294][20381] Updated weights for policy 1, policy_version 1942637 (0.0009) [2023-12-27 19:07:01,371][20347] Updated weights for policy 0, policy_version 1944063 (0.0010) [2023-12-27 19:07:01,422][20347] Updated weights for policy 0, policy_version 1944073 (0.0008) [2023-12-27 19:07:01,481][20347] Updated weights for policy 0, policy_version 1944083 (0.0008) [2023-12-27 19:07:02,039][20381] Updated weights for policy 1, policy_version 1942647 (0.0010) [2023-12-27 19:07:02,100][20381] Updated weights for policy 1, policy_version 1942657 (0.0010) [2023-12-27 19:07:02,153][20381] Updated weights for policy 1, policy_version 1942667 (0.0011) [2023-12-27 19:07:02,177][20347] Updated weights for policy 0, policy_version 1944093 (0.0007) [2023-12-27 19:07:02,226][20347] Updated weights for policy 0, policy_version 1944103 (0.0007) [2023-12-27 19:07:02,286][20347] Updated weights for policy 0, policy_version 1944113 (0.0008) [2023-12-27 19:07:02,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.6, 300 sec: 20577.2). Total num frames: 995172352. Throughput: 0: 10201.9, 1: 10319.8. Samples: 995177832. Policy #0 lag: (min: 18.0, avg: 34.6, max: 50.0) [2023-12-27 19:07:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '941.040')] [2023-12-27 19:07:02,839][20381] Updated weights for policy 1, policy_version 1942677 (0.0009) [2023-12-27 19:07:02,887][20381] Updated weights for policy 1, policy_version 1942687 (0.0011) [2023-12-27 19:07:02,948][20381] Updated weights for policy 1, policy_version 1942697 (0.0007) [2023-12-27 19:07:03,007][20347] Updated weights for policy 0, policy_version 1944123 (0.0008) [2023-12-27 19:07:03,059][20347] Updated weights for policy 0, policy_version 1944133 (0.0009) [2023-12-27 19:07:03,114][20347] Updated weights for policy 0, policy_version 1944143 (0.0007) [2023-12-27 19:07:03,539][20381] Updated weights for policy 1, policy_version 1942707 (0.0006) [2023-12-27 19:07:03,590][20381] Updated weights for policy 1, policy_version 1942717 (0.0010) [2023-12-27 19:07:03,638][20381] Updated weights for policy 1, policy_version 1942727 (0.0010) [2023-12-27 19:07:03,742][20347] Updated weights for policy 0, policy_version 1944153 (0.0008) [2023-12-27 19:07:03,794][20347] Updated weights for policy 0, policy_version 1944163 (0.0005) [2023-12-27 19:07:03,846][20347] Updated weights for policy 0, policy_version 1944173 (0.0007) [2023-12-27 19:07:03,909][20347] Updated weights for policy 0, policy_version 1944183 (0.0009) [2023-12-27 19:07:04,322][20381] Updated weights for policy 1, policy_version 1942737 (0.0006) [2023-12-27 19:07:04,378][20381] Updated weights for policy 1, policy_version 1942747 (0.0006) [2023-12-27 19:07:04,438][20381] Updated weights for policy 1, policy_version 1942757 (0.0006) [2023-12-27 19:07:04,494][20381] Updated weights for policy 1, policy_version 1942767 (0.0006) [2023-12-27 19:07:04,543][20347] Updated weights for policy 0, policy_version 1944193 (0.0010) [2023-12-27 19:07:04,596][20347] Updated weights for policy 0, policy_version 1944203 (0.0010) [2023-12-27 19:07:04,640][20347] Updated weights for policy 0, policy_version 1944213 (0.0010) [2023-12-27 19:07:05,107][20381] Updated weights for policy 1, policy_version 1942777 (0.0010) [2023-12-27 19:07:05,160][20381] Updated weights for policy 1, policy_version 1942787 (0.0011) [2023-12-27 19:07:05,214][20381] Updated weights for policy 1, policy_version 1942797 (0.0009) [2023-12-27 19:07:05,336][20347] Updated weights for policy 0, policy_version 1944223 (0.0007) [2023-12-27 19:07:05,387][20347] Updated weights for policy 0, policy_version 1944233 (0.0005) [2023-12-27 19:07:05,440][20347] Updated weights for policy 0, policy_version 1944243 (0.0008) [2023-12-27 19:07:05,841][20381] Updated weights for policy 1, policy_version 1942807 (0.0006) [2023-12-27 19:07:05,891][20381] Updated weights for policy 1, policy_version 1942817 (0.0008) [2023-12-27 19:07:05,940][20381] Updated weights for policy 1, policy_version 1942827 (0.0011) [2023-12-27 19:07:06,030][20347] Updated weights for policy 0, policy_version 1944253 (0.0010) [2023-12-27 19:07:06,078][20347] Updated weights for policy 0, policy_version 1944263 (0.0010) [2023-12-27 19:07:06,140][20347] Updated weights for policy 0, policy_version 1944273 (0.0009) [2023-12-27 19:07:06,616][20381] Updated weights for policy 1, policy_version 1942837 (0.0010) [2023-12-27 19:07:06,670][20381] Updated weights for policy 1, policy_version 1942847 (0.0008) [2023-12-27 19:07:06,717][20381] Updated weights for policy 1, policy_version 1942857 (0.0005) [2023-12-27 19:07:06,855][20347] Updated weights for policy 0, policy_version 1944283 (0.0011) [2023-12-27 19:07:06,907][20347] Updated weights for policy 0, policy_version 1944293 (0.0010) [2023-12-27 19:07:06,975][20347] Updated weights for policy 0, policy_version 1944303 (0.0010) [2023-12-27 19:07:07,355][20381] Updated weights for policy 1, policy_version 1942867 (0.0007) [2023-12-27 19:07:07,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20480.0, 300 sec: 20577.2). Total num frames: 995270656. Throughput: 0: 10161.1, 1: 10348.4. Samples: 995241296. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:07,364][19170] Avg episode reward: [(0, '1000.000'), (1, '911.580')] [2023-12-27 19:07:07,370][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001944312_497836032.pth... [2023-12-27 19:07:07,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001943128_497532928.pth [2023-12-27 19:07:07,408][20381] Updated weights for policy 1, policy_version 1942877 (0.0008) [2023-12-27 19:07:07,473][20381] Updated weights for policy 1, policy_version 1942887 (0.0006) [2023-12-27 19:07:07,521][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001942896_497442816.pth... [2023-12-27 19:07:07,524][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001941680_497131520.pth [2023-12-27 19:07:07,628][20347] Updated weights for policy 0, policy_version 1944313 (0.0010) [2023-12-27 19:07:07,680][20347] Updated weights for policy 0, policy_version 1944323 (0.0005) [2023-12-27 19:07:07,732][20347] Updated weights for policy 0, policy_version 1944333 (0.0005) [2023-12-27 19:07:07,782][20347] Updated weights for policy 0, policy_version 1944343 (0.0005) [2023-12-27 19:07:08,029][20381] Updated weights for policy 1, policy_version 1942897 (0.0007) [2023-12-27 19:07:08,091][20381] Updated weights for policy 1, policy_version 1942907 (0.0005) [2023-12-27 19:07:08,151][20381] Updated weights for policy 1, policy_version 1942917 (0.0008) [2023-12-27 19:07:08,215][20381] Updated weights for policy 1, policy_version 1942927 (0.0005) [2023-12-27 19:07:08,378][20347] Updated weights for policy 0, policy_version 1944353 (0.0008) [2023-12-27 19:07:08,436][20347] Updated weights for policy 0, policy_version 1944363 (0.0008) [2023-12-27 19:07:08,486][20347] Updated weights for policy 0, policy_version 1944373 (0.0009) [2023-12-27 19:07:08,810][20381] Updated weights for policy 1, policy_version 1942937 (0.0007) [2023-12-27 19:07:08,859][20381] Updated weights for policy 1, policy_version 1942947 (0.0008) [2023-12-27 19:07:08,911][20381] Updated weights for policy 1, policy_version 1942957 (0.0008) [2023-12-27 19:07:09,184][20347] Updated weights for policy 0, policy_version 1944383 (0.0007) [2023-12-27 19:07:09,250][20347] Updated weights for policy 0, policy_version 1944393 (0.0007) [2023-12-27 19:07:09,316][20347] Updated weights for policy 0, policy_version 1944403 (0.0008) [2023-12-27 19:07:09,570][20381] Updated weights for policy 1, policy_version 1942967 (0.0008) [2023-12-27 19:07:09,629][20381] Updated weights for policy 1, policy_version 1942977 (0.0007) [2023-12-27 19:07:09,684][20381] Updated weights for policy 1, policy_version 1942987 (0.0007) [2023-12-27 19:07:10,053][20347] Updated weights for policy 0, policy_version 1944413 (0.0007) [2023-12-27 19:07:10,109][20347] Updated weights for policy 0, policy_version 1944423 (0.0008) [2023-12-27 19:07:10,161][20347] Updated weights for policy 0, policy_version 1944433 (0.0008) [2023-12-27 19:07:10,342][20381] Updated weights for policy 1, policy_version 1942997 (0.0008) [2023-12-27 19:07:10,405][20381] Updated weights for policy 1, policy_version 1943007 (0.0009) [2023-12-27 19:07:10,467][20381] Updated weights for policy 1, policy_version 1943017 (0.0008) [2023-12-27 19:07:10,895][20347] Updated weights for policy 0, policy_version 1944443 (0.0009) [2023-12-27 19:07:10,958][20347] Updated weights for policy 0, policy_version 1944453 (0.0009) [2023-12-27 19:07:11,012][20347] Updated weights for policy 0, policy_version 1944463 (0.0006) [2023-12-27 19:07:11,107][20381] Updated weights for policy 1, policy_version 1943027 (0.0010) [2023-12-27 19:07:11,170][20381] Updated weights for policy 1, policy_version 1943037 (0.0010) [2023-12-27 19:07:11,235][20381] Updated weights for policy 1, policy_version 1943047 (0.0008) [2023-12-27 19:07:11,839][20347] Updated weights for policy 0, policy_version 1944473 (0.0008) [2023-12-27 19:07:11,898][20347] Updated weights for policy 0, policy_version 1944483 (0.0008) [2023-12-27 19:07:11,921][20381] Updated weights for policy 1, policy_version 1943057 (0.0008) [2023-12-27 19:07:11,961][20347] Updated weights for policy 0, policy_version 1944493 (0.0008) [2023-12-27 19:07:11,984][20381] Updated weights for policy 1, policy_version 1943067 (0.0006) [2023-12-27 19:07:12,020][20347] Updated weights for policy 0, policy_version 1944503 (0.0008) [2023-12-27 19:07:12,044][20381] Updated weights for policy 1, policy_version 1943077 (0.0005) [2023-12-27 19:07:12,099][20381] Updated weights for policy 1, policy_version 1943087 (0.0006) [2023-12-27 19:07:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 995377152. Throughput: 0: 10192.2, 1: 10405.9. Samples: 995368136. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '930.360')] [2023-12-27 19:07:12,738][20347] Updated weights for policy 0, policy_version 1944513 (0.0006) [2023-12-27 19:07:12,753][20381] Updated weights for policy 1, policy_version 1943097 (0.0009) [2023-12-27 19:07:12,803][20347] Updated weights for policy 0, policy_version 1944523 (0.0007) [2023-12-27 19:07:12,814][20381] Updated weights for policy 1, policy_version 1943107 (0.0007) [2023-12-27 19:07:12,846][20347] Updated weights for policy 0, policy_version 1944533 (0.0006) [2023-12-27 19:07:12,866][20381] Updated weights for policy 1, policy_version 1943117 (0.0011) [2023-12-27 19:07:13,450][20347] Updated weights for policy 0, policy_version 1944543 (0.0008) [2023-12-27 19:07:13,494][20381] Updated weights for policy 1, policy_version 1943127 (0.0010) [2023-12-27 19:07:13,513][20347] Updated weights for policy 0, policy_version 1944553 (0.0007) [2023-12-27 19:07:13,549][20381] Updated weights for policy 1, policy_version 1943137 (0.0007) [2023-12-27 19:07:13,572][20347] Updated weights for policy 0, policy_version 1944563 (0.0007) [2023-12-27 19:07:13,607][20381] Updated weights for policy 1, policy_version 1943147 (0.0006) [2023-12-27 19:07:14,203][20381] Updated weights for policy 1, policy_version 1943157 (0.0007) [2023-12-27 19:07:14,267][20381] Updated weights for policy 1, policy_version 1943167 (0.0008) [2023-12-27 19:07:14,280][20347] Updated weights for policy 0, policy_version 1944573 (0.0008) [2023-12-27 19:07:14,319][20381] Updated weights for policy 1, policy_version 1943177 (0.0006) [2023-12-27 19:07:14,325][20347] Updated weights for policy 0, policy_version 1944583 (0.0005) [2023-12-27 19:07:14,386][20347] Updated weights for policy 0, policy_version 1944593 (0.0009) [2023-12-27 19:07:14,975][20347] Updated weights for policy 0, policy_version 1944603 (0.0007) [2023-12-27 19:07:15,036][20347] Updated weights for policy 0, policy_version 1944613 (0.0006) [2023-12-27 19:07:15,048][20381] Updated weights for policy 1, policy_version 1943187 (0.0007) [2023-12-27 19:07:15,094][20347] Updated weights for policy 0, policy_version 1944623 (0.0007) [2023-12-27 19:07:15,107][20381] Updated weights for policy 1, policy_version 1943197 (0.0008) [2023-12-27 19:07:15,163][20381] Updated weights for policy 1, policy_version 1943207 (0.0008) [2023-12-27 19:07:15,792][20347] Updated weights for policy 0, policy_version 1944633 (0.0006) [2023-12-27 19:07:15,812][20381] Updated weights for policy 1, policy_version 1943217 (0.0008) [2023-12-27 19:07:15,846][20347] Updated weights for policy 0, policy_version 1944643 (0.0010) [2023-12-27 19:07:15,870][20381] Updated weights for policy 1, policy_version 1943227 (0.0007) [2023-12-27 19:07:15,892][20347] Updated weights for policy 0, policy_version 1944653 (0.0010) [2023-12-27 19:07:15,926][20381] Updated weights for policy 1, policy_version 1943237 (0.0008) [2023-12-27 19:07:15,942][20347] Updated weights for policy 0, policy_version 1944663 (0.0008) [2023-12-27 19:07:15,978][20381] Updated weights for policy 1, policy_version 1943247 (0.0010) [2023-12-27 19:07:16,570][20347] Updated weights for policy 0, policy_version 1944673 (0.0010) [2023-12-27 19:07:16,573][20381] Updated weights for policy 1, policy_version 1943257 (0.0007) [2023-12-27 19:07:16,624][20347] Updated weights for policy 0, policy_version 1944683 (0.0010) [2023-12-27 19:07:16,629][20381] Updated weights for policy 1, policy_version 1943267 (0.0008) [2023-12-27 19:07:16,679][20347] Updated weights for policy 0, policy_version 1944693 (0.0006) [2023-12-27 19:07:16,689][20381] Updated weights for policy 1, policy_version 1943277 (0.0009) [2023-12-27 19:07:17,272][20347] Updated weights for policy 0, policy_version 1944703 (0.0009) [2023-12-27 19:07:17,323][20347] Updated weights for policy 0, policy_version 1944713 (0.0010) [2023-12-27 19:07:17,362][19170] Fps is (10 sec: 20480.5, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 995475456. Throughput: 0: 10216.4, 1: 10443.5. Samples: 995494128. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.320')] [2023-12-27 19:07:17,366][20381] Updated weights for policy 1, policy_version 1943287 (0.0010) [2023-12-27 19:07:17,377][20347] Updated weights for policy 0, policy_version 1944723 (0.0010) [2023-12-27 19:07:17,421][20381] Updated weights for policy 1, policy_version 1943297 (0.0010) [2023-12-27 19:07:17,479][20381] Updated weights for policy 1, policy_version 1943307 (0.0011) [2023-12-27 19:07:18,071][20347] Updated weights for policy 0, policy_version 1944733 (0.0011) [2023-12-27 19:07:18,127][20381] Updated weights for policy 1, policy_version 1943317 (0.0007) [2023-12-27 19:07:18,130][20347] Updated weights for policy 0, policy_version 1944743 (0.0011) [2023-12-27 19:07:18,191][20381] Updated weights for policy 1, policy_version 1943327 (0.0006) [2023-12-27 19:07:18,192][20347] Updated weights for policy 0, policy_version 1944753 (0.0007) [2023-12-27 19:07:18,242][20381] Updated weights for policy 1, policy_version 1943337 (0.0011) [2023-12-27 19:07:18,832][20347] Updated weights for policy 0, policy_version 1944763 (0.0008) [2023-12-27 19:07:18,901][20347] Updated weights for policy 0, policy_version 1944773 (0.0006) [2023-12-27 19:07:18,934][20381] Updated weights for policy 1, policy_version 1943347 (0.0011) [2023-12-27 19:07:18,951][20347] Updated weights for policy 0, policy_version 1944783 (0.0006) [2023-12-27 19:07:18,986][20381] Updated weights for policy 1, policy_version 1943357 (0.0010) [2023-12-27 19:07:19,044][20381] Updated weights for policy 1, policy_version 1943367 (0.0010) [2023-12-27 19:07:19,562][20347] Updated weights for policy 0, policy_version 1944793 (0.0006) [2023-12-27 19:07:19,622][20347] Updated weights for policy 0, policy_version 1944803 (0.0008) [2023-12-27 19:07:19,686][20347] Updated weights for policy 0, policy_version 1944813 (0.0007) [2023-12-27 19:07:19,746][20347] Updated weights for policy 0, policy_version 1944823 (0.0007) [2023-12-27 19:07:19,752][20381] Updated weights for policy 1, policy_version 1943377 (0.0011) [2023-12-27 19:07:19,809][20381] Updated weights for policy 1, policy_version 1943387 (0.0011) [2023-12-27 19:07:19,874][20381] Updated weights for policy 1, policy_version 1943397 (0.0010) [2023-12-27 19:07:19,941][20381] Updated weights for policy 1, policy_version 1943407 (0.0009) [2023-12-27 19:07:20,448][20347] Updated weights for policy 0, policy_version 1944833 (0.0006) [2023-12-27 19:07:20,503][20347] Updated weights for policy 0, policy_version 1944843 (0.0006) [2023-12-27 19:07:20,559][20347] Updated weights for policy 0, policy_version 1944853 (0.0007) [2023-12-27 19:07:20,597][20381] Updated weights for policy 1, policy_version 1943417 (0.0008) [2023-12-27 19:07:20,650][20381] Updated weights for policy 1, policy_version 1943427 (0.0006) [2023-12-27 19:07:20,706][20381] Updated weights for policy 1, policy_version 1943437 (0.0005) [2023-12-27 19:07:21,225][20347] Updated weights for policy 0, policy_version 1944863 (0.0007) [2023-12-27 19:07:21,293][20347] Updated weights for policy 0, policy_version 1944873 (0.0008) [2023-12-27 19:07:21,357][20347] Updated weights for policy 0, policy_version 1944883 (0.0008) [2023-12-27 19:07:21,378][20381] Updated weights for policy 1, policy_version 1943447 (0.0008) [2023-12-27 19:07:21,437][20381] Updated weights for policy 1, policy_version 1943457 (0.0007) [2023-12-27 19:07:21,499][20381] Updated weights for policy 1, policy_version 1943467 (0.0008) [2023-12-27 19:07:22,077][20347] Updated weights for policy 0, policy_version 1944893 (0.0008) [2023-12-27 19:07:22,134][20347] Updated weights for policy 0, policy_version 1944903 (0.0008) [2023-12-27 19:07:22,196][20347] Updated weights for policy 0, policy_version 1944913 (0.0007) [2023-12-27 19:07:22,206][20381] Updated weights for policy 1, policy_version 1943477 (0.0007) [2023-12-27 19:07:22,270][20381] Updated weights for policy 1, policy_version 1943487 (0.0007) [2023-12-27 19:07:22,331][20381] Updated weights for policy 1, policy_version 1943497 (0.0009) [2023-12-27 19:07:22,363][19170] Fps is (10 sec: 20479.7, 60 sec: 20616.5, 300 sec: 20604.9). Total num frames: 995581952. Throughput: 0: 10222.1, 1: 10432.9. Samples: 995555112. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '881.680')] [2023-12-27 19:07:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001944920_497991680.pth... [2023-12-27 19:07:22,373][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001943504_497598464.pth... [2023-12-27 19:07:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001943704_497680384.pth [2023-12-27 19:07:22,377][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001942256_497278976.pth [2023-12-27 19:07:22,903][20347] Updated weights for policy 0, policy_version 1944923 (0.0006) [2023-12-27 19:07:22,963][20347] Updated weights for policy 0, policy_version 1944933 (0.0008) [2023-12-27 19:07:23,018][20347] Updated weights for policy 0, policy_version 1944943 (0.0008) [2023-12-27 19:07:23,045][20381] Updated weights for policy 1, policy_version 1943507 (0.0007) [2023-12-27 19:07:23,099][20381] Updated weights for policy 1, policy_version 1943517 (0.0006) [2023-12-27 19:07:23,153][20381] Updated weights for policy 1, policy_version 1943527 (0.0009) [2023-12-27 19:07:23,672][20347] Updated weights for policy 0, policy_version 1944953 (0.0006) [2023-12-27 19:07:23,722][20347] Updated weights for policy 0, policy_version 1944963 (0.0008) [2023-12-27 19:07:23,772][20347] Updated weights for policy 0, policy_version 1944973 (0.0009) [2023-12-27 19:07:23,806][20381] Updated weights for policy 1, policy_version 1943537 (0.0009) [2023-12-27 19:07:23,827][20347] Updated weights for policy 0, policy_version 1944983 (0.0007) [2023-12-27 19:07:23,856][20381] Updated weights for policy 1, policy_version 1943547 (0.0006) [2023-12-27 19:07:23,911][20381] Updated weights for policy 1, policy_version 1943557 (0.0007) [2023-12-27 19:07:23,963][20381] Updated weights for policy 1, policy_version 1943567 (0.0007) [2023-12-27 19:07:24,518][20347] Updated weights for policy 0, policy_version 1944993 (0.0007) [2023-12-27 19:07:24,574][20347] Updated weights for policy 0, policy_version 1945003 (0.0008) [2023-12-27 19:07:24,635][20347] Updated weights for policy 0, policy_version 1945013 (0.0007) [2023-12-27 19:07:24,639][20381] Updated weights for policy 1, policy_version 1943577 (0.0006) [2023-12-27 19:07:24,688][20381] Updated weights for policy 1, policy_version 1943587 (0.0007) [2023-12-27 19:07:24,748][20381] Updated weights for policy 1, policy_version 1943597 (0.0008) [2023-12-27 19:07:25,319][20347] Updated weights for policy 0, policy_version 1945023 (0.0007) [2023-12-27 19:07:25,367][20347] Updated weights for policy 0, policy_version 1945033 (0.0008) [2023-12-27 19:07:25,419][20347] Updated weights for policy 0, policy_version 1945043 (0.0007) [2023-12-27 19:07:25,420][20381] Updated weights for policy 1, policy_version 1943607 (0.0007) [2023-12-27 19:07:25,480][20381] Updated weights for policy 1, policy_version 1943617 (0.0005) [2023-12-27 19:07:25,540][20381] Updated weights for policy 1, policy_version 1943627 (0.0006) [2023-12-27 19:07:26,039][20347] Updated weights for policy 0, policy_version 1945053 (0.0006) [2023-12-27 19:07:26,094][20347] Updated weights for policy 0, policy_version 1945063 (0.0006) [2023-12-27 19:07:26,145][20381] Updated weights for policy 1, policy_version 1943637 (0.0009) [2023-12-27 19:07:26,147][20347] Updated weights for policy 0, policy_version 1945073 (0.0006) [2023-12-27 19:07:26,199][20381] Updated weights for policy 1, policy_version 1943647 (0.0011) [2023-12-27 19:07:26,253][20381] Updated weights for policy 1, policy_version 1943657 (0.0010) [2023-12-27 19:07:26,790][20347] Updated weights for policy 0, policy_version 1945083 (0.0007) [2023-12-27 19:07:26,841][20347] Updated weights for policy 0, policy_version 1945093 (0.0010) [2023-12-27 19:07:26,891][20347] Updated weights for policy 0, policy_version 1945103 (0.0010) [2023-12-27 19:07:26,955][20381] Updated weights for policy 1, policy_version 1943667 (0.0011) [2023-12-27 19:07:27,013][20381] Updated weights for policy 1, policy_version 1943677 (0.0010) [2023-12-27 19:07:27,063][20381] Updated weights for policy 1, policy_version 1943687 (0.0010) [2023-12-27 19:07:27,363][19170] Fps is (10 sec: 21298.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 995688448. Throughput: 0: 10254.0, 1: 10400.9. Samples: 995678004. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '910.830')] [2023-12-27 19:07:27,558][20347] Updated weights for policy 0, policy_version 1945113 (0.0010) [2023-12-27 19:07:27,609][20347] Updated weights for policy 0, policy_version 1945123 (0.0007) [2023-12-27 19:07:27,657][20347] Updated weights for policy 0, policy_version 1945133 (0.0005) [2023-12-27 19:07:27,693][20381] Updated weights for policy 1, policy_version 1943697 (0.0010) [2023-12-27 19:07:27,710][20347] Updated weights for policy 0, policy_version 1945143 (0.0005) [2023-12-27 19:07:27,745][20381] Updated weights for policy 1, policy_version 1943707 (0.0010) [2023-12-27 19:07:27,796][20381] Updated weights for policy 1, policy_version 1943717 (0.0010) [2023-12-27 19:07:27,848][20381] Updated weights for policy 1, policy_version 1943727 (0.0010) [2023-12-27 19:07:28,334][20347] Updated weights for policy 0, policy_version 1945153 (0.0009) [2023-12-27 19:07:28,396][20347] Updated weights for policy 0, policy_version 1945163 (0.0009) [2023-12-27 19:07:28,443][20381] Updated weights for policy 1, policy_version 1943737 (0.0011) [2023-12-27 19:07:28,455][20347] Updated weights for policy 0, policy_version 1945173 (0.0007) [2023-12-27 19:07:28,496][20381] Updated weights for policy 1, policy_version 1943747 (0.0011) [2023-12-27 19:07:28,551][20381] Updated weights for policy 1, policy_version 1943757 (0.0011) [2023-12-27 19:07:29,092][20347] Updated weights for policy 0, policy_version 1945183 (0.0006) [2023-12-27 19:07:29,139][20347] Updated weights for policy 0, policy_version 1945193 (0.0007) [2023-12-27 19:07:29,183][20347] Updated weights for policy 0, policy_version 1945203 (0.0010) [2023-12-27 19:07:29,194][20381] Updated weights for policy 1, policy_version 1943767 (0.0007) [2023-12-27 19:07:29,265][20381] Updated weights for policy 1, policy_version 1943777 (0.0008) [2023-12-27 19:07:29,330][20381] Updated weights for policy 1, policy_version 1943787 (0.0008) [2023-12-27 19:07:29,905][20347] Updated weights for policy 0, policy_version 1945213 (0.0009) [2023-12-27 19:07:29,969][20347] Updated weights for policy 0, policy_version 1945223 (0.0008) [2023-12-27 19:07:30,023][20381] Updated weights for policy 1, policy_version 1943797 (0.0009) [2023-12-27 19:07:30,025][20347] Updated weights for policy 0, policy_version 1945233 (0.0006) [2023-12-27 19:07:30,079][20381] Updated weights for policy 1, policy_version 1943807 (0.0011) [2023-12-27 19:07:30,135][20381] Updated weights for policy 1, policy_version 1943817 (0.0008) [2023-12-27 19:07:30,685][20381] Updated weights for policy 1, policy_version 1943827 (0.0007) [2023-12-27 19:07:30,753][20381] Updated weights for policy 1, policy_version 1943837 (0.0009) [2023-12-27 19:07:30,768][20347] Updated weights for policy 0, policy_version 1945243 (0.0006) [2023-12-27 19:07:30,804][20381] Updated weights for policy 1, policy_version 1943847 (0.0008) [2023-12-27 19:07:30,824][20347] Updated weights for policy 0, policy_version 1945253 (0.0006) [2023-12-27 19:07:30,884][20347] Updated weights for policy 0, policy_version 1945263 (0.0006) [2023-12-27 19:07:31,502][20381] Updated weights for policy 1, policy_version 1943857 (0.0010) [2023-12-27 19:07:31,554][20381] Updated weights for policy 1, policy_version 1943867 (0.0006) [2023-12-27 19:07:31,610][20381] Updated weights for policy 1, policy_version 1943877 (0.0011) [2023-12-27 19:07:31,617][20347] Updated weights for policy 0, policy_version 1945273 (0.0007) [2023-12-27 19:07:31,672][20381] Updated weights for policy 1, policy_version 1943887 (0.0008) [2023-12-27 19:07:31,678][20347] Updated weights for policy 0, policy_version 1945283 (0.0008) [2023-12-27 19:07:31,737][20347] Updated weights for policy 0, policy_version 1945293 (0.0007) [2023-12-27 19:07:31,803][20347] Updated weights for policy 0, policy_version 1945303 (0.0008) [2023-12-27 19:07:32,309][20381] Updated weights for policy 1, policy_version 1943897 (0.0007) [2023-12-27 19:07:32,362][19170] Fps is (10 sec: 20480.3, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 995786752. Throughput: 0: 10260.5, 1: 10447.5. Samples: 995803844. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.530')] [2023-12-27 19:07:32,374][20381] Updated weights for policy 1, policy_version 1943907 (0.0008) [2023-12-27 19:07:32,426][20347] Updated weights for policy 0, policy_version 1945313 (0.0007) [2023-12-27 19:07:32,440][20381] Updated weights for policy 1, policy_version 1943917 (0.0008) [2023-12-27 19:07:32,486][20347] Updated weights for policy 0, policy_version 1945323 (0.0008) [2023-12-27 19:07:32,545][20347] Updated weights for policy 0, policy_version 1945333 (0.0007) [2023-12-27 19:07:33,039][20381] Updated weights for policy 1, policy_version 1943927 (0.0006) [2023-12-27 19:07:33,099][20381] Updated weights for policy 1, policy_version 1943937 (0.0007) [2023-12-27 19:07:33,162][20381] Updated weights for policy 1, policy_version 1943947 (0.0007) [2023-12-27 19:07:33,211][20347] Updated weights for policy 0, policy_version 1945343 (0.0010) [2023-12-27 19:07:33,266][20347] Updated weights for policy 0, policy_version 1945353 (0.0010) [2023-12-27 19:07:33,321][20347] Updated weights for policy 0, policy_version 1945363 (0.0007) [2023-12-27 19:07:33,775][20381] Updated weights for policy 1, policy_version 1943957 (0.0005) [2023-12-27 19:07:33,827][20381] Updated weights for policy 1, policy_version 1943967 (0.0005) [2023-12-27 19:07:33,885][20381] Updated weights for policy 1, policy_version 1943977 (0.0010) [2023-12-27 19:07:33,895][20347] Updated weights for policy 0, policy_version 1945373 (0.0006) [2023-12-27 19:07:33,957][20347] Updated weights for policy 0, policy_version 1945383 (0.0006) [2023-12-27 19:07:34,019][20347] Updated weights for policy 0, policy_version 1945393 (0.0008) [2023-12-27 19:07:34,565][20381] Updated weights for policy 1, policy_version 1943987 (0.0011) [2023-12-27 19:07:34,634][20381] Updated weights for policy 1, policy_version 1943997 (0.0011) [2023-12-27 19:07:34,691][20381] Updated weights for policy 1, policy_version 1944007 (0.0011) [2023-12-27 19:07:34,700][20347] Updated weights for policy 0, policy_version 1945403 (0.0010) [2023-12-27 19:07:34,755][20347] Updated weights for policy 0, policy_version 1945413 (0.0010) [2023-12-27 19:07:34,812][20347] Updated weights for policy 0, policy_version 1945423 (0.0010) [2023-12-27 19:07:35,353][20381] Updated weights for policy 1, policy_version 1944017 (0.0010) [2023-12-27 19:07:35,417][20381] Updated weights for policy 1, policy_version 1944027 (0.0010) [2023-12-27 19:07:35,423][20347] Updated weights for policy 0, policy_version 1945433 (0.0010) [2023-12-27 19:07:35,473][20347] Updated weights for policy 0, policy_version 1945443 (0.0007) [2023-12-27 19:07:35,475][20381] Updated weights for policy 1, policy_version 1944037 (0.0011) [2023-12-27 19:07:35,525][20347] Updated weights for policy 0, policy_version 1945453 (0.0005) [2023-12-27 19:07:35,531][20381] Updated weights for policy 1, policy_version 1944047 (0.0011) [2023-12-27 19:07:35,579][20347] Updated weights for policy 0, policy_version 1945463 (0.0007) [2023-12-27 19:07:36,177][20381] Updated weights for policy 1, policy_version 1944057 (0.0008) [2023-12-27 19:07:36,227][20347] Updated weights for policy 0, policy_version 1945473 (0.0007) [2023-12-27 19:07:36,240][20381] Updated weights for policy 1, policy_version 1944067 (0.0007) [2023-12-27 19:07:36,291][20347] Updated weights for policy 0, policy_version 1945483 (0.0008) [2023-12-27 19:07:36,301][20381] Updated weights for policy 1, policy_version 1944077 (0.0007) [2023-12-27 19:07:36,352][20347] Updated weights for policy 0, policy_version 1945493 (0.0007) [2023-12-27 19:07:36,966][20347] Updated weights for policy 0, policy_version 1945503 (0.0008) [2023-12-27 19:07:37,026][20381] Updated weights for policy 1, policy_version 1944087 (0.0006) [2023-12-27 19:07:37,028][20347] Updated weights for policy 0, policy_version 1945513 (0.0008) [2023-12-27 19:07:37,087][20381] Updated weights for policy 1, policy_version 1944097 (0.0005) [2023-12-27 19:07:37,088][20347] Updated weights for policy 0, policy_version 1945523 (0.0008) [2023-12-27 19:07:37,146][20381] Updated weights for policy 1, policy_version 1944107 (0.0007) [2023-12-27 19:07:37,362][19170] Fps is (10 sec: 21299.5, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 995901440. Throughput: 0: 10299.7, 1: 10475.8. Samples: 995867428. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.910')] [2023-12-27 19:07:37,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001945528_498147328.pth... [2023-12-27 19:07:37,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001944112_497754112.pth... [2023-12-27 19:07:37,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001944312_497836032.pth [2023-12-27 19:07:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001942896_497442816.pth [2023-12-27 19:07:37,692][20381] Updated weights for policy 1, policy_version 1944117 (0.0007) [2023-12-27 19:07:37,752][20381] Updated weights for policy 1, policy_version 1944127 (0.0005) [2023-12-27 19:07:37,800][20381] Updated weights for policy 1, policy_version 1944137 (0.0006) [2023-12-27 19:07:37,835][20347] Updated weights for policy 0, policy_version 1945533 (0.0008) [2023-12-27 19:07:37,882][20347] Updated weights for policy 0, policy_version 1945543 (0.0008) [2023-12-27 19:07:37,933][20347] Updated weights for policy 0, policy_version 1945553 (0.0008) [2023-12-27 19:07:38,380][20381] Updated weights for policy 1, policy_version 1944147 (0.0006) [2023-12-27 19:07:38,436][20381] Updated weights for policy 1, policy_version 1944157 (0.0008) [2023-12-27 19:07:38,492][20381] Updated weights for policy 1, policy_version 1944167 (0.0008) [2023-12-27 19:07:38,659][20347] Updated weights for policy 0, policy_version 1945563 (0.0007) [2023-12-27 19:07:38,718][20347] Updated weights for policy 0, policy_version 1945573 (0.0008) [2023-12-27 19:07:38,782][20347] Updated weights for policy 0, policy_version 1945583 (0.0008) [2023-12-27 19:07:39,243][20381] Updated weights for policy 1, policy_version 1944177 (0.0008) [2023-12-27 19:07:39,309][20381] Updated weights for policy 1, policy_version 1944187 (0.0008) [2023-12-27 19:07:39,378][20381] Updated weights for policy 1, policy_version 1944197 (0.0008) [2023-12-27 19:07:39,434][20347] Updated weights for policy 0, policy_version 1945593 (0.0008) [2023-12-27 19:07:39,445][20381] Updated weights for policy 1, policy_version 1944207 (0.0008) [2023-12-27 19:07:39,493][20347] Updated weights for policy 0, policy_version 1945603 (0.0007) [2023-12-27 19:07:39,553][20347] Updated weights for policy 0, policy_version 1945613 (0.0008) [2023-12-27 19:07:39,608][20347] Updated weights for policy 0, policy_version 1945623 (0.0008) [2023-12-27 19:07:40,186][20381] Updated weights for policy 1, policy_version 1944217 (0.0008) [2023-12-27 19:07:40,242][20381] Updated weights for policy 1, policy_version 1944227 (0.0007) [2023-12-27 19:07:40,298][20381] Updated weights for policy 1, policy_version 1944237 (0.0008) [2023-12-27 19:07:40,335][20347] Updated weights for policy 0, policy_version 1945633 (0.0008) [2023-12-27 19:07:40,396][20347] Updated weights for policy 0, policy_version 1945643 (0.0011) [2023-12-27 19:07:40,452][20347] Updated weights for policy 0, policy_version 1945653 (0.0011) [2023-12-27 19:07:40,971][20381] Updated weights for policy 1, policy_version 1944247 (0.0008) [2023-12-27 19:07:41,036][20381] Updated weights for policy 1, policy_version 1944257 (0.0008) [2023-12-27 19:07:41,102][20381] Updated weights for policy 1, policy_version 1944267 (0.0009) [2023-12-27 19:07:41,132][20347] Updated weights for policy 0, policy_version 1945663 (0.0010) [2023-12-27 19:07:41,201][20347] Updated weights for policy 0, policy_version 1945673 (0.0009) [2023-12-27 19:07:41,266][20347] Updated weights for policy 0, policy_version 1945683 (0.0007) [2023-12-27 19:07:41,856][20381] Updated weights for policy 1, policy_version 1944277 (0.0007) [2023-12-27 19:07:41,911][20381] Updated weights for policy 1, policy_version 1944287 (0.0008) [2023-12-27 19:07:41,923][20347] Updated weights for policy 0, policy_version 1945693 (0.0006) [2023-12-27 19:07:41,966][20381] Updated weights for policy 1, policy_version 1944297 (0.0007) [2023-12-27 19:07:41,983][20347] Updated weights for policy 0, policy_version 1945703 (0.0006) [2023-12-27 19:07:42,047][20347] Updated weights for policy 0, policy_version 1945713 (0.0006) [2023-12-27 19:07:42,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 995999744. Throughput: 0: 10343.0, 1: 10457.7. Samples: 995989876. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '852.910')] [2023-12-27 19:07:42,643][20347] Updated weights for policy 0, policy_version 1945723 (0.0006) [2023-12-27 19:07:42,699][20347] Updated weights for policy 0, policy_version 1945733 (0.0006) [2023-12-27 19:07:42,740][20381] Updated weights for policy 1, policy_version 1944307 (0.0009) [2023-12-27 19:07:42,754][20347] Updated weights for policy 0, policy_version 1945743 (0.0006) [2023-12-27 19:07:42,802][20381] Updated weights for policy 1, policy_version 1944317 (0.0008) [2023-12-27 19:07:42,855][20381] Updated weights for policy 1, policy_version 1944327 (0.0009) [2023-12-27 19:07:43,381][20347] Updated weights for policy 0, policy_version 1945753 (0.0006) [2023-12-27 19:07:43,429][20347] Updated weights for policy 0, policy_version 1945763 (0.0008) [2023-12-27 19:07:43,477][20347] Updated weights for policy 0, policy_version 1945773 (0.0008) [2023-12-27 19:07:43,485][20381] Updated weights for policy 1, policy_version 1944337 (0.0006) [2023-12-27 19:07:43,524][20347] Updated weights for policy 0, policy_version 1945783 (0.0007) [2023-12-27 19:07:43,549][20381] Updated weights for policy 1, policy_version 1944347 (0.0006) [2023-12-27 19:07:43,598][20381] Updated weights for policy 1, policy_version 1944357 (0.0007) [2023-12-27 19:07:43,656][20381] Updated weights for policy 1, policy_version 1944367 (0.0005) [2023-12-27 19:07:44,194][20347] Updated weights for policy 0, policy_version 1945793 (0.0005) [2023-12-27 19:07:44,251][20347] Updated weights for policy 0, policy_version 1945803 (0.0006) [2023-12-27 19:07:44,264][20381] Updated weights for policy 1, policy_version 1944377 (0.0007) [2023-12-27 19:07:44,310][20347] Updated weights for policy 0, policy_version 1945813 (0.0005) [2023-12-27 19:07:44,319][20381] Updated weights for policy 1, policy_version 1944387 (0.0006) [2023-12-27 19:07:44,370][20381] Updated weights for policy 1, policy_version 1944397 (0.0007) [2023-12-27 19:07:44,874][20347] Updated weights for policy 0, policy_version 1945823 (0.0007) [2023-12-27 19:07:44,938][20347] Updated weights for policy 0, policy_version 1945833 (0.0006) [2023-12-27 19:07:45,002][20347] Updated weights for policy 0, policy_version 1945843 (0.0006) [2023-12-27 19:07:45,135][20381] Updated weights for policy 1, policy_version 1944407 (0.0009) [2023-12-27 19:07:45,202][20381] Updated weights for policy 1, policy_version 1944417 (0.0008) [2023-12-27 19:07:45,262][20381] Updated weights for policy 1, policy_version 1944427 (0.0008) [2023-12-27 19:07:45,580][20347] Updated weights for policy 0, policy_version 1945853 (0.0006) [2023-12-27 19:07:45,633][20347] Updated weights for policy 0, policy_version 1945863 (0.0005) [2023-12-27 19:07:45,683][20347] Updated weights for policy 0, policy_version 1945873 (0.0005) [2023-12-27 19:07:45,934][20381] Updated weights for policy 1, policy_version 1944437 (0.0009) [2023-12-27 19:07:45,984][20381] Updated weights for policy 1, policy_version 1944447 (0.0010) [2023-12-27 19:07:46,045][20381] Updated weights for policy 1, policy_version 1944457 (0.0008) [2023-12-27 19:07:46,203][20347] Updated weights for policy 0, policy_version 1945883 (0.0005) [2023-12-27 19:07:46,247][20347] Updated weights for policy 0, policy_version 1945893 (0.0005) [2023-12-27 19:07:46,295][20347] Updated weights for policy 0, policy_version 1945903 (0.0006) [2023-12-27 19:07:46,671][20381] Updated weights for policy 1, policy_version 1944467 (0.0009) [2023-12-27 19:07:46,726][20381] Updated weights for policy 1, policy_version 1944477 (0.0007) [2023-12-27 19:07:46,787][20381] Updated weights for policy 1, policy_version 1944487 (0.0007) [2023-12-27 19:07:46,966][20347] Updated weights for policy 0, policy_version 1945913 (0.0005) [2023-12-27 19:07:47,014][20347] Updated weights for policy 0, policy_version 1945923 (0.0007) [2023-12-27 19:07:47,072][20347] Updated weights for policy 0, policy_version 1945933 (0.0006) [2023-12-27 19:07:47,131][20347] Updated weights for policy 0, policy_version 1945943 (0.0009) [2023-12-27 19:07:47,363][19170] Fps is (10 sec: 20479.4, 60 sec: 20752.9, 300 sec: 20604.9). Total num frames: 996106240. Throughput: 0: 10418.1, 1: 10451.4. Samples: 996116968. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:47,364][19170] Avg episode reward: [(0, '1000.000'), (1, '862.210')] [2023-12-27 19:07:47,394][20381] Updated weights for policy 1, policy_version 1944497 (0.0007) [2023-12-27 19:07:47,450][20381] Updated weights for policy 1, policy_version 1944507 (0.0005) [2023-12-27 19:07:47,504][20381] Updated weights for policy 1, policy_version 1944517 (0.0005) [2023-12-27 19:07:47,552][20381] Updated weights for policy 1, policy_version 1944527 (0.0005) [2023-12-27 19:07:47,831][20347] Updated weights for policy 0, policy_version 1945953 (0.0006) [2023-12-27 19:07:47,890][20347] Updated weights for policy 0, policy_version 1945963 (0.0006) [2023-12-27 19:07:47,950][20347] Updated weights for policy 0, policy_version 1945973 (0.0008) [2023-12-27 19:07:48,150][20381] Updated weights for policy 1, policy_version 1944537 (0.0006) [2023-12-27 19:07:48,212][20381] Updated weights for policy 1, policy_version 1944547 (0.0008) [2023-12-27 19:07:48,271][20381] Updated weights for policy 1, policy_version 1944557 (0.0008) [2023-12-27 19:07:48,585][20347] Updated weights for policy 0, policy_version 1945983 (0.0008) [2023-12-27 19:07:48,645][20347] Updated weights for policy 0, policy_version 1945993 (0.0007) [2023-12-27 19:07:48,704][20347] Updated weights for policy 0, policy_version 1946003 (0.0007) [2023-12-27 19:07:48,912][20381] Updated weights for policy 1, policy_version 1944567 (0.0005) [2023-12-27 19:07:48,964][20381] Updated weights for policy 1, policy_version 1944577 (0.0008) [2023-12-27 19:07:49,020][20381] Updated weights for policy 1, policy_version 1944587 (0.0008) [2023-12-27 19:07:49,389][20347] Updated weights for policy 0, policy_version 1946013 (0.0008) [2023-12-27 19:07:49,449][20347] Updated weights for policy 0, policy_version 1946023 (0.0006) [2023-12-27 19:07:49,504][20347] Updated weights for policy 0, policy_version 1946033 (0.0006) [2023-12-27 19:07:49,765][20381] Updated weights for policy 1, policy_version 1944597 (0.0009) [2023-12-27 19:07:49,829][20381] Updated weights for policy 1, policy_version 1944607 (0.0010) [2023-12-27 19:07:49,893][20381] Updated weights for policy 1, policy_version 1944617 (0.0011) [2023-12-27 19:07:50,149][20347] Updated weights for policy 0, policy_version 1946043 (0.0006) [2023-12-27 19:07:50,214][20347] Updated weights for policy 0, policy_version 1946053 (0.0005) [2023-12-27 19:07:50,278][20347] Updated weights for policy 0, policy_version 1946063 (0.0008) [2023-12-27 19:07:50,599][20381] Updated weights for policy 1, policy_version 1944627 (0.0009) [2023-12-27 19:07:50,655][20381] Updated weights for policy 1, policy_version 1944637 (0.0008) [2023-12-27 19:07:50,718][20381] Updated weights for policy 1, policy_version 1944647 (0.0008) [2023-12-27 19:07:50,991][20347] Updated weights for policy 0, policy_version 1946073 (0.0008) [2023-12-27 19:07:51,058][20347] Updated weights for policy 0, policy_version 1946083 (0.0008) [2023-12-27 19:07:51,123][20347] Updated weights for policy 0, policy_version 1946093 (0.0007) [2023-12-27 19:07:51,185][20347] Updated weights for policy 0, policy_version 1946103 (0.0009) [2023-12-27 19:07:51,442][20381] Updated weights for policy 1, policy_version 1944657 (0.0007) [2023-12-27 19:07:51,498][20381] Updated weights for policy 1, policy_version 1944667 (0.0006) [2023-12-27 19:07:51,554][20381] Updated weights for policy 1, policy_version 1944677 (0.0007) [2023-12-27 19:07:51,599][20381] Updated weights for policy 1, policy_version 1944687 (0.0007) [2023-12-27 19:07:51,980][20347] Updated weights for policy 0, policy_version 1946113 (0.0010) [2023-12-27 19:07:52,035][20347] Updated weights for policy 0, policy_version 1946123 (0.0010) [2023-12-27 19:07:52,090][20347] Updated weights for policy 0, policy_version 1946133 (0.0010) [2023-12-27 19:07:52,275][20381] Updated weights for policy 1, policy_version 1944697 (0.0008) [2023-12-27 19:07:52,337][20381] Updated weights for policy 1, policy_version 1944707 (0.0007) [2023-12-27 19:07:52,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20577.2). Total num frames: 996204544. Throughput: 0: 10405.6, 1: 10423.7. Samples: 996178608. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '881.690')] [2023-12-27 19:07:52,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001946136_498302976.pth... [2023-12-27 19:07:52,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001944920_497991680.pth [2023-12-27 19:07:52,398][20381] Updated weights for policy 1, policy_version 1944717 (0.0008) [2023-12-27 19:07:52,415][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001944720_497909760.pth... [2023-12-27 19:07:52,418][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001943504_497598464.pth [2023-12-27 19:07:52,804][20347] Updated weights for policy 0, policy_version 1946143 (0.0009) [2023-12-27 19:07:52,866][20347] Updated weights for policy 0, policy_version 1946153 (0.0008) [2023-12-27 19:07:52,928][20347] Updated weights for policy 0, policy_version 1946163 (0.0007) [2023-12-27 19:07:53,050][20381] Updated weights for policy 1, policy_version 1944727 (0.0007) [2023-12-27 19:07:53,104][20381] Updated weights for policy 1, policy_version 1944737 (0.0005) [2023-12-27 19:07:53,164][20381] Updated weights for policy 1, policy_version 1944747 (0.0005) [2023-12-27 19:07:53,580][20347] Updated weights for policy 0, policy_version 1946173 (0.0007) [2023-12-27 19:07:53,632][20347] Updated weights for policy 0, policy_version 1946183 (0.0006) [2023-12-27 19:07:53,684][20347] Updated weights for policy 0, policy_version 1946193 (0.0008) [2023-12-27 19:07:53,795][20381] Updated weights for policy 1, policy_version 1944757 (0.0005) [2023-12-27 19:07:53,851][20381] Updated weights for policy 1, policy_version 1944767 (0.0006) [2023-12-27 19:07:53,907][20381] Updated weights for policy 1, policy_version 1944777 (0.0005) [2023-12-27 19:07:54,387][20347] Updated weights for policy 0, policy_version 1946203 (0.0008) [2023-12-27 19:07:54,443][20347] Updated weights for policy 0, policy_version 1946213 (0.0010) [2023-12-27 19:07:54,498][20347] Updated weights for policy 0, policy_version 1946223 (0.0010) [2023-12-27 19:07:54,524][20381] Updated weights for policy 1, policy_version 1944787 (0.0006) [2023-12-27 19:07:54,577][20381] Updated weights for policy 1, policy_version 1944797 (0.0010) [2023-12-27 19:07:54,634][20381] Updated weights for policy 1, policy_version 1944807 (0.0011) [2023-12-27 19:07:55,151][20347] Updated weights for policy 0, policy_version 1946233 (0.0010) [2023-12-27 19:07:55,200][20347] Updated weights for policy 0, policy_version 1946243 (0.0010) [2023-12-27 19:07:55,242][20347] Updated weights for policy 0, policy_version 1946253 (0.0010) [2023-12-27 19:07:55,290][20347] Updated weights for policy 0, policy_version 1946263 (0.0010) [2023-12-27 19:07:55,295][20381] Updated weights for policy 1, policy_version 1944817 (0.0011) [2023-12-27 19:07:55,356][20381] Updated weights for policy 1, policy_version 1944827 (0.0011) [2023-12-27 19:07:55,418][20381] Updated weights for policy 1, policy_version 1944837 (0.0011) [2023-12-27 19:07:55,477][20381] Updated weights for policy 1, policy_version 1944847 (0.0011) [2023-12-27 19:07:56,012][20347] Updated weights for policy 0, policy_version 1946273 (0.0008) [2023-12-27 19:07:56,071][20347] Updated weights for policy 0, policy_version 1946283 (0.0007) [2023-12-27 19:07:56,123][20347] Updated weights for policy 0, policy_version 1946293 (0.0006) [2023-12-27 19:07:56,132][20381] Updated weights for policy 1, policy_version 1944857 (0.0010) [2023-12-27 19:07:56,178][20381] Updated weights for policy 1, policy_version 1944867 (0.0010) [2023-12-27 19:07:56,228][20381] Updated weights for policy 1, policy_version 1944878 (0.0010) [2023-12-27 19:07:56,744][20347] Updated weights for policy 0, policy_version 1946303 (0.0007) [2023-12-27 19:07:56,801][20347] Updated weights for policy 0, policy_version 1946313 (0.0007) [2023-12-27 19:07:56,851][20347] Updated weights for policy 0, policy_version 1946323 (0.0007) [2023-12-27 19:07:56,966][20381] Updated weights for policy 1, policy_version 1944888 (0.0008) [2023-12-27 19:07:57,011][20381] Updated weights for policy 1, policy_version 1944898 (0.0009) [2023-12-27 19:07:57,060][20381] Updated weights for policy 1, policy_version 1944908 (0.0008) [2023-12-27 19:07:57,362][19170] Fps is (10 sec: 20480.8, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 996311040. Throughput: 0: 10354.0, 1: 10372.2. Samples: 996300812. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:07:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '901.930')] [2023-12-27 19:07:57,476][20347] Updated weights for policy 0, policy_version 1946333 (0.0006) [2023-12-27 19:07:57,536][20347] Updated weights for policy 0, policy_version 1946343 (0.0008) [2023-12-27 19:07:57,594][20347] Updated weights for policy 0, policy_version 1946353 (0.0005) [2023-12-27 19:07:57,742][20381] Updated weights for policy 1, policy_version 1944918 (0.0006) [2023-12-27 19:07:57,794][20381] Updated weights for policy 1, policy_version 1944928 (0.0007) [2023-12-27 19:07:57,851][20381] Updated weights for policy 1, policy_version 1944938 (0.0008) [2023-12-27 19:07:58,158][20347] Updated weights for policy 0, policy_version 1946363 (0.0006) [2023-12-27 19:07:58,231][20347] Updated weights for policy 0, policy_version 1946373 (0.0007) [2023-12-27 19:07:58,291][20347] Updated weights for policy 0, policy_version 1946383 (0.0006) [2023-12-27 19:07:58,563][20381] Updated weights for policy 1, policy_version 1944948 (0.0008) [2023-12-27 19:07:58,627][20381] Updated weights for policy 1, policy_version 1944958 (0.0007) [2023-12-27 19:07:58,689][20381] Updated weights for policy 1, policy_version 1944968 (0.0008) [2023-12-27 19:07:59,045][20347] Updated weights for policy 0, policy_version 1946394 (0.0007) [2023-12-27 19:07:59,107][20347] Updated weights for policy 0, policy_version 1946404 (0.0008) [2023-12-27 19:07:59,162][20347] Updated weights for policy 0, policy_version 1946414 (0.0008) [2023-12-27 19:07:59,223][20347] Updated weights for policy 0, policy_version 1946424 (0.0008) [2023-12-27 19:07:59,417][20381] Updated weights for policy 1, policy_version 1944978 (0.0007) [2023-12-27 19:07:59,466][20381] Updated weights for policy 1, policy_version 1944988 (0.0008) [2023-12-27 19:07:59,512][20381] Updated weights for policy 1, policy_version 1944998 (0.0008) [2023-12-27 19:07:59,566][20381] Updated weights for policy 1, policy_version 1945008 (0.0008) [2023-12-27 19:08:00,011][20347] Updated weights for policy 0, policy_version 1946434 (0.0008) [2023-12-27 19:08:00,072][20347] Updated weights for policy 0, policy_version 1946444 (0.0007) [2023-12-27 19:08:00,130][20347] Updated weights for policy 0, policy_version 1946454 (0.0008) [2023-12-27 19:08:00,329][20381] Updated weights for policy 1, policy_version 1945018 (0.0007) [2023-12-27 19:08:00,390][20381] Updated weights for policy 1, policy_version 1945028 (0.0008) [2023-12-27 19:08:00,445][20381] Updated weights for policy 1, policy_version 1945038 (0.0008) [2023-12-27 19:08:00,826][20347] Updated weights for policy 0, policy_version 1946464 (0.0009) [2023-12-27 19:08:00,870][20347] Updated weights for policy 0, policy_version 1946474 (0.0007) [2023-12-27 19:08:00,930][20347] Updated weights for policy 0, policy_version 1946484 (0.0008) [2023-12-27 19:08:01,071][20381] Updated weights for policy 1, policy_version 1945048 (0.0008) [2023-12-27 19:08:01,117][20381] Updated weights for policy 1, policy_version 1945058 (0.0007) [2023-12-27 19:08:01,172][20381] Updated weights for policy 1, policy_version 1945068 (0.0008) [2023-12-27 19:08:01,605][20347] Updated weights for policy 0, policy_version 1946494 (0.0007) [2023-12-27 19:08:01,669][20347] Updated weights for policy 0, policy_version 1946504 (0.0007) [2023-12-27 19:08:01,727][20347] Updated weights for policy 0, policy_version 1946514 (0.0007) [2023-12-27 19:08:01,918][20381] Updated weights for policy 1, policy_version 1945078 (0.0007) [2023-12-27 19:08:01,977][20381] Updated weights for policy 1, policy_version 1945088 (0.0008) [2023-12-27 19:08:02,039][20381] Updated weights for policy 1, policy_version 1945098 (0.0008) [2023-12-27 19:08:02,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 996409344. Throughput: 0: 10318.0, 1: 10265.5. Samples: 996420388. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.070')] [2023-12-27 19:08:02,381][20347] Updated weights for policy 0, policy_version 1946524 (0.0008) [2023-12-27 19:08:02,440][20347] Updated weights for policy 0, policy_version 1946534 (0.0007) [2023-12-27 19:08:02,492][20347] Updated weights for policy 0, policy_version 1946544 (0.0007) [2023-12-27 19:08:02,747][20381] Updated weights for policy 1, policy_version 1945108 (0.0008) [2023-12-27 19:08:02,808][20381] Updated weights for policy 1, policy_version 1945118 (0.0006) [2023-12-27 19:08:02,865][20381] Updated weights for policy 1, policy_version 1945128 (0.0008) [2023-12-27 19:08:03,109][20347] Updated weights for policy 0, policy_version 1946554 (0.0007) [2023-12-27 19:08:03,167][20347] Updated weights for policy 0, policy_version 1946564 (0.0008) [2023-12-27 19:08:03,226][20347] Updated weights for policy 0, policy_version 1946574 (0.0008) [2023-12-27 19:08:03,278][20347] Updated weights for policy 0, policy_version 1946584 (0.0007) [2023-12-27 19:08:03,610][20381] Updated weights for policy 1, policy_version 1945138 (0.0009) [2023-12-27 19:08:03,661][20381] Updated weights for policy 1, policy_version 1945148 (0.0008) [2023-12-27 19:08:03,716][20381] Updated weights for policy 1, policy_version 1945158 (0.0007) [2023-12-27 19:08:03,770][20381] Updated weights for policy 1, policy_version 1945168 (0.0005) [2023-12-27 19:08:03,949][20347] Updated weights for policy 0, policy_version 1946594 (0.0006) [2023-12-27 19:08:04,009][20347] Updated weights for policy 0, policy_version 1946604 (0.0010) [2023-12-27 19:08:04,064][20347] Updated weights for policy 0, policy_version 1946614 (0.0010) [2023-12-27 19:08:04,436][20381] Updated weights for policy 1, policy_version 1945178 (0.0007) [2023-12-27 19:08:04,493][20381] Updated weights for policy 1, policy_version 1945188 (0.0008) [2023-12-27 19:08:04,559][20381] Updated weights for policy 1, policy_version 1945198 (0.0008) [2023-12-27 19:08:04,740][20347] Updated weights for policy 0, policy_version 1946624 (0.0010) [2023-12-27 19:08:04,802][20347] Updated weights for policy 0, policy_version 1946634 (0.0010) [2023-12-27 19:08:04,862][20347] Updated weights for policy 0, policy_version 1946644 (0.0010) [2023-12-27 19:08:05,247][20381] Updated weights for policy 1, policy_version 1945208 (0.0007) [2023-12-27 19:08:05,309][20381] Updated weights for policy 1, policy_version 1945218 (0.0007) [2023-12-27 19:08:05,366][20381] Updated weights for policy 1, policy_version 1945228 (0.0006) [2023-12-27 19:08:05,538][20347] Updated weights for policy 0, policy_version 1946654 (0.0010) [2023-12-27 19:08:05,599][20347] Updated weights for policy 0, policy_version 1946664 (0.0010) [2023-12-27 19:08:05,647][20347] Updated weights for policy 0, policy_version 1946674 (0.0010) [2023-12-27 19:08:06,045][20381] Updated weights for policy 1, policy_version 1945238 (0.0007) [2023-12-27 19:08:06,099][20381] Updated weights for policy 1, policy_version 1945248 (0.0007) [2023-12-27 19:08:06,146][20292] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000003 [2023-12-27 19:08:06,303][20347] Updated weights for policy 0, policy_version 1946684 (0.0008) [2023-12-27 19:08:06,361][20347] Updated weights for policy 0, policy_version 1946694 (0.0010) [2023-12-27 19:08:06,413][20347] Updated weights for policy 0, policy_version 1946704 (0.0010) [2023-12-27 19:08:06,802][20381] Updated weights for policy 1, policy_version 1945258 (0.0008) [2023-12-27 19:08:06,859][20381] Updated weights for policy 1, policy_version 1945268 (0.0008) [2023-12-27 19:08:06,921][20381] Updated weights for policy 1, policy_version 1945278 (0.0007) [2023-12-27 19:08:06,983][20381] Updated weights for policy 1, policy_version 1945288 (0.0007) [2023-12-27 19:08:07,116][20347] Updated weights for policy 0, policy_version 1946714 (0.0010) [2023-12-27 19:08:07,186][20347] Updated weights for policy 0, policy_version 1946724 (0.0010) [2023-12-27 19:08:07,241][20347] Updated weights for policy 0, policy_version 1946734 (0.0010) [2023-12-27 19:08:07,307][20347] Updated weights for policy 0, policy_version 1946744 (0.0010) [2023-12-27 19:08:07,362][19170] Fps is (10 sec: 20479.6, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 996515840. Throughput: 0: 10335.5, 1: 10262.8. Samples: 996482032. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.300')] [2023-12-27 19:08:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001945288_498057216.pth... [2023-12-27 19:08:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001946744_498458624.pth... [2023-12-27 19:08:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001945528_498147328.pth [2023-12-27 19:08:07,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001944112_497754112.pth [2023-12-27 19:08:07,647][20381] Updated weights for policy 1, policy_version 1945298 (0.0010) [2023-12-27 19:08:07,696][20381] Updated weights for policy 1, policy_version 1945308 (0.0006) [2023-12-27 19:08:07,748][20381] Updated weights for policy 1, policy_version 1945318 (0.0006) [2023-12-27 19:08:07,868][20347] Updated weights for policy 0, policy_version 1946754 (0.0005) [2023-12-27 19:08:07,912][20347] Updated weights for policy 0, policy_version 1946764 (0.0005) [2023-12-27 19:08:07,958][20347] Updated weights for policy 0, policy_version 1946774 (0.0009) [2023-12-27 19:08:08,365][20381] Updated weights for policy 1, policy_version 1945328 (0.0008) [2023-12-27 19:08:08,417][20381] Updated weights for policy 1, policy_version 1945338 (0.0007) [2023-12-27 19:08:08,469][20381] Updated weights for policy 1, policy_version 1945348 (0.0007) [2023-12-27 19:08:08,603][20347] Updated weights for policy 0, policy_version 1946784 (0.0010) [2023-12-27 19:08:08,670][20347] Updated weights for policy 0, policy_version 1946794 (0.0005) [2023-12-27 19:08:08,734][20347] Updated weights for policy 0, policy_version 1946804 (0.0005) [2023-12-27 19:08:09,114][20381] Updated weights for policy 1, policy_version 1945358 (0.0006) [2023-12-27 19:08:09,167][20381] Updated weights for policy 1, policy_version 1945368 (0.0007) [2023-12-27 19:08:09,219][20381] Updated weights for policy 1, policy_version 1945378 (0.0008) [2023-12-27 19:08:09,326][20347] Updated weights for policy 0, policy_version 1946814 (0.0007) [2023-12-27 19:08:09,399][20347] Updated weights for policy 0, policy_version 1946824 (0.0008) [2023-12-27 19:08:09,461][20347] Updated weights for policy 0, policy_version 1946834 (0.0008) [2023-12-27 19:08:09,911][20381] Updated weights for policy 1, policy_version 1945388 (0.0008) [2023-12-27 19:08:09,981][20381] Updated weights for policy 1, policy_version 1945398 (0.0008) [2023-12-27 19:08:10,034][20381] Updated weights for policy 1, policy_version 1945408 (0.0008) [2023-12-27 19:08:10,175][20347] Updated weights for policy 0, policy_version 1946844 (0.0009) [2023-12-27 19:08:10,237][20347] Updated weights for policy 0, policy_version 1946854 (0.0008) [2023-12-27 19:08:10,299][20347] Updated weights for policy 0, policy_version 1946864 (0.0005) [2023-12-27 19:08:10,803][20381] Updated weights for policy 1, policy_version 1945418 (0.0009) [2023-12-27 19:08:10,865][20381] Updated weights for policy 1, policy_version 1945428 (0.0010) [2023-12-27 19:08:10,883][20347] Updated weights for policy 0, policy_version 1946874 (0.0008) [2023-12-27 19:08:10,919][20381] Updated weights for policy 1, policy_version 1945438 (0.0006) [2023-12-27 19:08:10,948][20347] Updated weights for policy 0, policy_version 1946884 (0.0005) [2023-12-27 19:08:10,979][20381] Updated weights for policy 1, policy_version 1945448 (0.0005) [2023-12-27 19:08:11,002][20347] Updated weights for policy 0, policy_version 1946894 (0.0005) [2023-12-27 19:08:11,062][20347] Updated weights for policy 0, policy_version 1946904 (0.0007) [2023-12-27 19:08:11,782][20381] Updated weights for policy 1, policy_version 1945458 (0.0009) [2023-12-27 19:08:11,788][20347] Updated weights for policy 0, policy_version 1946914 (0.0007) [2023-12-27 19:08:11,842][20381] Updated weights for policy 1, policy_version 1945468 (0.0008) [2023-12-27 19:08:11,849][20347] Updated weights for policy 0, policy_version 1946924 (0.0006) [2023-12-27 19:08:11,903][20381] Updated weights for policy 1, policy_version 1945478 (0.0007) [2023-12-27 19:08:11,909][20347] Updated weights for policy 0, policy_version 1946934 (0.0006) [2023-12-27 19:08:12,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 996614144. Throughput: 0: 10398.4, 1: 10255.9. Samples: 996607448. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.430')] [2023-12-27 19:08:12,614][20381] Updated weights for policy 1, policy_version 1945488 (0.0006) [2023-12-27 19:08:12,615][20347] Updated weights for policy 0, policy_version 1946944 (0.0006) [2023-12-27 19:08:12,671][20347] Updated weights for policy 0, policy_version 1946954 (0.0006) [2023-12-27 19:08:12,684][20381] Updated weights for policy 1, policy_version 1945498 (0.0008) [2023-12-27 19:08:12,727][20347] Updated weights for policy 0, policy_version 1946964 (0.0006) [2023-12-27 19:08:12,739][20381] Updated weights for policy 1, policy_version 1945508 (0.0007) [2023-12-27 19:08:13,361][20347] Updated weights for policy 0, policy_version 1946974 (0.0005) [2023-12-27 19:08:13,363][20381] Updated weights for policy 1, policy_version 1945518 (0.0007) [2023-12-27 19:08:13,416][20347] Updated weights for policy 0, policy_version 1946984 (0.0005) [2023-12-27 19:08:13,422][20381] Updated weights for policy 1, policy_version 1945528 (0.0006) [2023-12-27 19:08:13,469][20347] Updated weights for policy 0, policy_version 1946994 (0.0005) [2023-12-27 19:08:13,478][20381] Updated weights for policy 1, policy_version 1945538 (0.0006) [2023-12-27 19:08:14,081][20381] Updated weights for policy 1, policy_version 1945548 (0.0006) [2023-12-27 19:08:14,138][20381] Updated weights for policy 1, policy_version 1945558 (0.0007) [2023-12-27 19:08:14,186][20347] Updated weights for policy 0, policy_version 1947004 (0.0006) [2023-12-27 19:08:14,196][20381] Updated weights for policy 1, policy_version 1945568 (0.0007) [2023-12-27 19:08:14,235][20347] Updated weights for policy 0, policy_version 1947014 (0.0006) [2023-12-27 19:08:14,290][20347] Updated weights for policy 0, policy_version 1947024 (0.0007) [2023-12-27 19:08:14,781][20381] Updated weights for policy 1, policy_version 1945578 (0.0007) [2023-12-27 19:08:14,839][20381] Updated weights for policy 1, policy_version 1945588 (0.0009) [2023-12-27 19:08:14,897][20381] Updated weights for policy 1, policy_version 1945598 (0.0009) [2023-12-27 19:08:14,954][20381] Updated weights for policy 1, policy_version 1945608 (0.0009) [2023-12-27 19:08:15,014][20347] Updated weights for policy 0, policy_version 1947034 (0.0008) [2023-12-27 19:08:15,075][20347] Updated weights for policy 0, policy_version 1947044 (0.0008) [2023-12-27 19:08:15,129][20347] Updated weights for policy 0, policy_version 1947054 (0.0007) [2023-12-27 19:08:15,193][20347] Updated weights for policy 0, policy_version 1947064 (0.0007) [2023-12-27 19:08:15,672][20381] Updated weights for policy 1, policy_version 1945618 (0.0007) [2023-12-27 19:08:15,721][20381] Updated weights for policy 1, policy_version 1945628 (0.0008) [2023-12-27 19:08:15,774][20381] Updated weights for policy 1, policy_version 1945638 (0.0007) [2023-12-27 19:08:15,853][20347] Updated weights for policy 0, policy_version 1947074 (0.0011) [2023-12-27 19:08:15,913][20347] Updated weights for policy 0, policy_version 1947084 (0.0008) [2023-12-27 19:08:15,962][20347] Updated weights for policy 0, policy_version 1947094 (0.0010) [2023-12-27 19:08:16,417][20381] Updated weights for policy 1, policy_version 1945648 (0.0008) [2023-12-27 19:08:16,465][20381] Updated weights for policy 1, policy_version 1945658 (0.0007) [2023-12-27 19:08:16,519][20381] Updated weights for policy 1, policy_version 1945668 (0.0005) [2023-12-27 19:08:16,619][20347] Updated weights for policy 0, policy_version 1947104 (0.0010) [2023-12-27 19:08:16,679][20347] Updated weights for policy 0, policy_version 1947114 (0.0010) [2023-12-27 19:08:16,734][20347] Updated weights for policy 0, policy_version 1947124 (0.0010) [2023-12-27 19:08:17,155][20381] Updated weights for policy 1, policy_version 1945678 (0.0005) [2023-12-27 19:08:17,204][20381] Updated weights for policy 1, policy_version 1945688 (0.0005) [2023-12-27 19:08:17,256][20381] Updated weights for policy 1, policy_version 1945698 (0.0005) [2023-12-27 19:08:17,353][20347] Updated weights for policy 0, policy_version 1947134 (0.0006) [2023-12-27 19:08:17,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 996720640. Throughput: 0: 10375.5, 1: 10221.8. Samples: 996730720. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '862.940')] [2023-12-27 19:08:17,406][20347] Updated weights for policy 0, policy_version 1947144 (0.0005) [2023-12-27 19:08:17,464][20347] Updated weights for policy 0, policy_version 1947154 (0.0005) [2023-12-27 19:08:17,891][20381] Updated weights for policy 1, policy_version 1945708 (0.0007) [2023-12-27 19:08:17,955][20381] Updated weights for policy 1, policy_version 1945718 (0.0009) [2023-12-27 19:08:18,025][20381] Updated weights for policy 1, policy_version 1945728 (0.0008) [2023-12-27 19:08:18,040][20347] Updated weights for policy 0, policy_version 1947164 (0.0010) [2023-12-27 19:08:18,094][20347] Updated weights for policy 0, policy_version 1947174 (0.0010) [2023-12-27 19:08:18,157][20347] Updated weights for policy 0, policy_version 1947184 (0.0009) [2023-12-27 19:08:18,661][20381] Updated weights for policy 1, policy_version 1945738 (0.0008) [2023-12-27 19:08:18,717][20381] Updated weights for policy 1, policy_version 1945748 (0.0010) [2023-12-27 19:08:18,781][20381] Updated weights for policy 1, policy_version 1945758 (0.0011) [2023-12-27 19:08:18,829][20381] Updated weights for policy 1, policy_version 1945768 (0.0010) [2023-12-27 19:08:18,851][20347] Updated weights for policy 0, policy_version 1947194 (0.0008) [2023-12-27 19:08:18,906][20347] Updated weights for policy 0, policy_version 1947204 (0.0010) [2023-12-27 19:08:18,961][20347] Updated weights for policy 0, policy_version 1947214 (0.0010) [2023-12-27 19:08:19,012][20347] Updated weights for policy 0, policy_version 1947224 (0.0010) [2023-12-27 19:08:19,533][20381] Updated weights for policy 1, policy_version 1945778 (0.0008) [2023-12-27 19:08:19,584][20381] Updated weights for policy 1, policy_version 1945788 (0.0008) [2023-12-27 19:08:19,640][20381] Updated weights for policy 1, policy_version 1945798 (0.0008) [2023-12-27 19:08:19,688][20347] Updated weights for policy 0, policy_version 1947234 (0.0007) [2023-12-27 19:08:19,742][20347] Updated weights for policy 0, policy_version 1947244 (0.0007) [2023-12-27 19:08:19,786][20347] Updated weights for policy 0, policy_version 1947254 (0.0007) [2023-12-27 19:08:20,329][20381] Updated weights for policy 1, policy_version 1945808 (0.0007) [2023-12-27 19:08:20,391][20381] Updated weights for policy 1, policy_version 1945818 (0.0006) [2023-12-27 19:08:20,451][20381] Updated weights for policy 1, policy_version 1945828 (0.0006) [2023-12-27 19:08:20,496][20347] Updated weights for policy 0, policy_version 1947264 (0.0007) [2023-12-27 19:08:20,574][20347] Updated weights for policy 0, policy_version 1947274 (0.0007) [2023-12-27 19:08:20,639][20347] Updated weights for policy 0, policy_version 1947284 (0.0007) [2023-12-27 19:08:21,125][20381] Updated weights for policy 1, policy_version 1945838 (0.0007) [2023-12-27 19:08:21,193][20381] Updated weights for policy 1, policy_version 1945848 (0.0009) [2023-12-27 19:08:21,264][20381] Updated weights for policy 1, policy_version 1945858 (0.0008) [2023-12-27 19:08:21,266][20347] Updated weights for policy 0, policy_version 1947294 (0.0006) [2023-12-27 19:08:21,325][20347] Updated weights for policy 0, policy_version 1947304 (0.0008) [2023-12-27 19:08:21,394][20347] Updated weights for policy 0, policy_version 1947314 (0.0008) [2023-12-27 19:08:22,030][20347] Updated weights for policy 0, policy_version 1947324 (0.0006) [2023-12-27 19:08:22,039][20381] Updated weights for policy 1, policy_version 1945868 (0.0008) [2023-12-27 19:08:22,092][20347] Updated weights for policy 0, policy_version 1947334 (0.0008) [2023-12-27 19:08:22,094][20381] Updated weights for policy 1, policy_version 1945878 (0.0009) [2023-12-27 19:08:22,148][20347] Updated weights for policy 0, policy_version 1947344 (0.0008) [2023-12-27 19:08:22,149][20381] Updated weights for policy 1, policy_version 1945888 (0.0010) [2023-12-27 19:08:22,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20753.0, 300 sec: 20604.9). Total num frames: 996827136. Throughput: 0: 10367.9, 1: 10209.6. Samples: 996793416. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.260')] [2023-12-27 19:08:22,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001945896_498212864.pth... [2023-12-27 19:08:22,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001947352_498614272.pth... [2023-12-27 19:08:22,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001944720_497909760.pth [2023-12-27 19:08:22,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001946136_498302976.pth [2023-12-27 19:08:22,848][20347] Updated weights for policy 0, policy_version 1947354 (0.0006) [2023-12-27 19:08:22,878][20381] Updated weights for policy 1, policy_version 1945898 (0.0006) [2023-12-27 19:08:22,906][20347] Updated weights for policy 0, policy_version 1947364 (0.0009) [2023-12-27 19:08:22,930][20381] Updated weights for policy 1, policy_version 1945908 (0.0005) [2023-12-27 19:08:22,956][20347] Updated weights for policy 0, policy_version 1947374 (0.0010) [2023-12-27 19:08:22,987][20381] Updated weights for policy 1, policy_version 1945918 (0.0005) [2023-12-27 19:08:23,006][20347] Updated weights for policy 0, policy_version 1947384 (0.0010) [2023-12-27 19:08:23,046][20381] Updated weights for policy 1, policy_version 1945928 (0.0005) [2023-12-27 19:08:23,561][20381] Updated weights for policy 1, policy_version 1945938 (0.0005) [2023-12-27 19:08:23,615][20381] Updated weights for policy 1, policy_version 1945948 (0.0006) [2023-12-27 19:08:23,641][20347] Updated weights for policy 0, policy_version 1947394 (0.0005) [2023-12-27 19:08:23,671][20381] Updated weights for policy 1, policy_version 1945958 (0.0006) [2023-12-27 19:08:23,699][20347] Updated weights for policy 0, policy_version 1947404 (0.0005) [2023-12-27 19:08:23,757][20347] Updated weights for policy 0, policy_version 1947414 (0.0005) [2023-12-27 19:08:24,326][20347] Updated weights for policy 0, policy_version 1947424 (0.0006) [2023-12-27 19:08:24,337][20381] Updated weights for policy 1, policy_version 1945968 (0.0007) [2023-12-27 19:08:24,383][20347] Updated weights for policy 0, policy_version 1947434 (0.0006) [2023-12-27 19:08:24,385][20381] Updated weights for policy 1, policy_version 1945978 (0.0007) [2023-12-27 19:08:24,432][20381] Updated weights for policy 1, policy_version 1945988 (0.0007) [2023-12-27 19:08:24,437][20347] Updated weights for policy 0, policy_version 1947444 (0.0005) [2023-12-27 19:08:25,017][20347] Updated weights for policy 0, policy_version 1947454 (0.0007) [2023-12-27 19:08:25,074][20347] Updated weights for policy 0, policy_version 1947464 (0.0006) [2023-12-27 19:08:25,140][20347] Updated weights for policy 0, policy_version 1947474 (0.0005) [2023-12-27 19:08:25,182][20381] Updated weights for policy 1, policy_version 1945998 (0.0007) [2023-12-27 19:08:25,247][20381] Updated weights for policy 1, policy_version 1946008 (0.0008) [2023-12-27 19:08:25,297][20381] Updated weights for policy 1, policy_version 1946018 (0.0008) [2023-12-27 19:08:25,673][20347] Updated weights for policy 0, policy_version 1947484 (0.0007) [2023-12-27 19:08:25,735][20347] Updated weights for policy 0, policy_version 1947494 (0.0010) [2023-12-27 19:08:25,790][20347] Updated weights for policy 0, policy_version 1947504 (0.0010) [2023-12-27 19:08:26,020][20381] Updated weights for policy 1, policy_version 1946028 (0.0007) [2023-12-27 19:08:26,076][20381] Updated weights for policy 1, policy_version 1946038 (0.0005) [2023-12-27 19:08:26,129][20381] Updated weights for policy 1, policy_version 1946048 (0.0007) [2023-12-27 19:08:26,409][20347] Updated weights for policy 0, policy_version 1947514 (0.0009) [2023-12-27 19:08:26,473][20347] Updated weights for policy 0, policy_version 1947524 (0.0006) [2023-12-27 19:08:26,531][20347] Updated weights for policy 0, policy_version 1947534 (0.0005) [2023-12-27 19:08:26,539][20291] Early stopping after 3 epochs (24 sgd steps), loss delta 0.0000006 [2023-12-27 19:08:26,818][20381] Updated weights for policy 1, policy_version 1946058 (0.0007) [2023-12-27 19:08:26,869][20381] Updated weights for policy 1, policy_version 1946068 (0.0005) [2023-12-27 19:08:26,919][20381] Updated weights for policy 1, policy_version 1946078 (0.0005) [2023-12-27 19:08:26,968][20381] Updated weights for policy 1, policy_version 1946088 (0.0005) [2023-12-27 19:08:27,229][20347] Updated weights for policy 0, policy_version 1947544 (0.0008) [2023-12-27 19:08:27,282][20347] Updated weights for policy 0, policy_version 1947554 (0.0008) [2023-12-27 19:08:27,336][20347] Updated weights for policy 0, policy_version 1947564 (0.0007) [2023-12-27 19:08:27,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 996933632. Throughput: 0: 10449.5, 1: 10192.7. Samples: 996918772. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.090')] [2023-12-27 19:08:27,496][20381] Updated weights for policy 1, policy_version 1946098 (0.0009) [2023-12-27 19:08:27,548][20381] Updated weights for policy 1, policy_version 1946108 (0.0008) [2023-12-27 19:08:27,606][20381] Updated weights for policy 1, policy_version 1946118 (0.0008) [2023-12-27 19:08:27,961][20347] Updated weights for policy 0, policy_version 1947574 (0.0008) [2023-12-27 19:08:28,017][20347] Updated weights for policy 0, policy_version 1947584 (0.0008) [2023-12-27 19:08:28,063][20347] Updated weights for policy 0, policy_version 1947594 (0.0005) [2023-12-27 19:08:28,235][20381] Updated weights for policy 1, policy_version 1946128 (0.0006) [2023-12-27 19:08:28,279][20381] Updated weights for policy 1, policy_version 1946138 (0.0005) [2023-12-27 19:08:28,325][20381] Updated weights for policy 1, policy_version 1946148 (0.0005) [2023-12-27 19:08:28,652][20347] Updated weights for policy 0, policy_version 1947604 (0.0006) [2023-12-27 19:08:28,709][20347] Updated weights for policy 0, policy_version 1947614 (0.0010) [2023-12-27 19:08:28,757][20347] Updated weights for policy 0, policy_version 1947624 (0.0010) [2023-12-27 19:08:28,989][20381] Updated weights for policy 1, policy_version 1946158 (0.0008) [2023-12-27 19:08:29,036][20381] Updated weights for policy 1, policy_version 1946168 (0.0007) [2023-12-27 19:08:29,094][20381] Updated weights for policy 1, policy_version 1946178 (0.0006) [2023-12-27 19:08:29,454][20347] Updated weights for policy 0, policy_version 1947634 (0.0009) [2023-12-27 19:08:29,505][20347] Updated weights for policy 0, policy_version 1947644 (0.0008) [2023-12-27 19:08:29,559][20347] Updated weights for policy 0, policy_version 1947654 (0.0010) [2023-12-27 19:08:29,604][20347] Updated weights for policy 0, policy_version 1947664 (0.0008) [2023-12-27 19:08:29,809][20381] Updated weights for policy 1, policy_version 1946188 (0.0007) [2023-12-27 19:08:29,876][20381] Updated weights for policy 1, policy_version 1946198 (0.0008) [2023-12-27 19:08:29,942][20381] Updated weights for policy 1, policy_version 1946208 (0.0008) [2023-12-27 19:08:30,320][20347] Updated weights for policy 0, policy_version 1947674 (0.0008) [2023-12-27 19:08:30,370][20347] Updated weights for policy 0, policy_version 1947684 (0.0008) [2023-12-27 19:08:30,418][20347] Updated weights for policy 0, policy_version 1947694 (0.0006) [2023-12-27 19:08:30,564][20381] Updated weights for policy 1, policy_version 1946218 (0.0005) [2023-12-27 19:08:30,632][20381] Updated weights for policy 1, policy_version 1946228 (0.0008) [2023-12-27 19:08:30,696][20381] Updated weights for policy 1, policy_version 1946238 (0.0009) [2023-12-27 19:08:30,765][20381] Updated weights for policy 1, policy_version 1946248 (0.0007) [2023-12-27 19:08:31,021][20347] Updated weights for policy 0, policy_version 1947704 (0.0007) [2023-12-27 19:08:31,084][20347] Updated weights for policy 0, policy_version 1947714 (0.0007) [2023-12-27 19:08:31,148][20347] Updated weights for policy 0, policy_version 1947724 (0.0007) [2023-12-27 19:08:31,413][20381] Updated weights for policy 1, policy_version 1946258 (0.0008) [2023-12-27 19:08:31,467][20381] Updated weights for policy 1, policy_version 1946268 (0.0011) [2023-12-27 19:08:31,518][20381] Updated weights for policy 1, policy_version 1946278 (0.0008) [2023-12-27 19:08:31,879][20347] Updated weights for policy 0, policy_version 1947734 (0.0007) [2023-12-27 19:08:31,941][20347] Updated weights for policy 0, policy_version 1947744 (0.0010) [2023-12-27 19:08:32,001][20347] Updated weights for policy 0, policy_version 1947754 (0.0010) [2023-12-27 19:08:32,213][20381] Updated weights for policy 1, policy_version 1946288 (0.0009) [2023-12-27 19:08:32,264][20381] Updated weights for policy 1, policy_version 1946298 (0.0010) [2023-12-27 19:08:32,324][20381] Updated weights for policy 1, policy_version 1946308 (0.0005) [2023-12-27 19:08:32,362][19170] Fps is (10 sec: 21299.7, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 997040128. Throughput: 0: 10402.2, 1: 10252.2. Samples: 997046408. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '940.400')] [2023-12-27 19:08:32,675][20347] Updated weights for policy 0, policy_version 1947764 (0.0010) [2023-12-27 19:08:32,740][20347] Updated weights for policy 0, policy_version 1947774 (0.0011) [2023-12-27 19:08:32,796][20347] Updated weights for policy 0, policy_version 1947784 (0.0007) [2023-12-27 19:08:32,939][20381] Updated weights for policy 1, policy_version 1946318 (0.0007) [2023-12-27 19:08:32,997][20381] Updated weights for policy 1, policy_version 1946328 (0.0005) [2023-12-27 19:08:33,051][20381] Updated weights for policy 1, policy_version 1946338 (0.0005) [2023-12-27 19:08:33,467][20347] Updated weights for policy 0, policy_version 1947794 (0.0008) [2023-12-27 19:08:33,537][20347] Updated weights for policy 0, policy_version 1947804 (0.0006) [2023-12-27 19:08:33,598][20347] Updated weights for policy 0, policy_version 1947814 (0.0006) [2023-12-27 19:08:33,642][20381] Updated weights for policy 1, policy_version 1946348 (0.0005) [2023-12-27 19:08:33,655][20347] Updated weights for policy 0, policy_version 1947824 (0.0007) [2023-12-27 19:08:33,692][20381] Updated weights for policy 1, policy_version 1946358 (0.0007) [2023-12-27 19:08:33,745][20381] Updated weights for policy 1, policy_version 1946368 (0.0008) [2023-12-27 19:08:34,309][20347] Updated weights for policy 0, policy_version 1947834 (0.0007) [2023-12-27 19:08:34,362][20347] Updated weights for policy 0, policy_version 1947844 (0.0007) [2023-12-27 19:08:34,419][20347] Updated weights for policy 0, policy_version 1947854 (0.0007) [2023-12-27 19:08:34,437][20381] Updated weights for policy 1, policy_version 1946378 (0.0008) [2023-12-27 19:08:34,494][20381] Updated weights for policy 1, policy_version 1946388 (0.0008) [2023-12-27 19:08:34,554][20381] Updated weights for policy 1, policy_version 1946398 (0.0008) [2023-12-27 19:08:34,616][20381] Updated weights for policy 1, policy_version 1946408 (0.0008) [2023-12-27 19:08:35,109][20347] Updated weights for policy 0, policy_version 1947864 (0.0007) [2023-12-27 19:08:35,160][20347] Updated weights for policy 0, policy_version 1947874 (0.0006) [2023-12-27 19:08:35,215][20347] Updated weights for policy 0, policy_version 1947884 (0.0006) [2023-12-27 19:08:35,306][20381] Updated weights for policy 1, policy_version 1946418 (0.0007) [2023-12-27 19:08:35,354][20381] Updated weights for policy 1, policy_version 1946428 (0.0007) [2023-12-27 19:08:35,403][20381] Updated weights for policy 1, policy_version 1946438 (0.0008) [2023-12-27 19:08:35,855][20347] Updated weights for policy 0, policy_version 1947894 (0.0006) [2023-12-27 19:08:35,910][20347] Updated weights for policy 0, policy_version 1947904 (0.0009) [2023-12-27 19:08:35,974][20347] Updated weights for policy 0, policy_version 1947914 (0.0008) [2023-12-27 19:08:36,136][20381] Updated weights for policy 1, policy_version 1946448 (0.0007) [2023-12-27 19:08:36,188][20381] Updated weights for policy 1, policy_version 1946458 (0.0007) [2023-12-27 19:08:36,244][20381] Updated weights for policy 1, policy_version 1946468 (0.0008) [2023-12-27 19:08:36,634][20347] Updated weights for policy 0, policy_version 1947924 (0.0009) [2023-12-27 19:08:36,695][20347] Updated weights for policy 0, policy_version 1947934 (0.0010) [2023-12-27 19:08:36,753][20347] Updated weights for policy 0, policy_version 1947944 (0.0010) [2023-12-27 19:08:36,936][20381] Updated weights for policy 1, policy_version 1946478 (0.0008) [2023-12-27 19:08:37,000][20381] Updated weights for policy 1, policy_version 1946488 (0.0008) [2023-12-27 19:08:37,049][20381] Updated weights for policy 1, policy_version 1946498 (0.0007) [2023-12-27 19:08:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 997138432. Throughput: 0: 10410.2, 1: 10267.6. Samples: 997109108. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.240')] [2023-12-27 19:08:37,370][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001946504_498368512.pth... [2023-12-27 19:08:37,371][20347] Updated weights for policy 0, policy_version 1947954 (0.0009) [2023-12-27 19:08:37,374][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001945288_498057216.pth [2023-12-27 19:08:37,431][20347] Updated weights for policy 0, policy_version 1947964 (0.0008) [2023-12-27 19:08:37,491][20347] Updated weights for policy 0, policy_version 1947974 (0.0008) [2023-12-27 19:08:37,550][20347] Updated weights for policy 0, policy_version 1947984 (0.0007) [2023-12-27 19:08:37,550][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001947984_498778112.pth... [2023-12-27 19:08:37,554][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001946744_498458624.pth [2023-12-27 19:08:37,725][20381] Updated weights for policy 1, policy_version 1946508 (0.0006) [2023-12-27 19:08:37,781][20381] Updated weights for policy 1, policy_version 1946518 (0.0008) [2023-12-27 19:08:37,832][20381] Updated weights for policy 1, policy_version 1946528 (0.0008) [2023-12-27 19:08:38,205][20347] Updated weights for policy 0, policy_version 1947994 (0.0005) [2023-12-27 19:08:38,266][20347] Updated weights for policy 0, policy_version 1948004 (0.0006) [2023-12-27 19:08:38,327][20347] Updated weights for policy 0, policy_version 1948014 (0.0006) [2023-12-27 19:08:38,535][20381] Updated weights for policy 1, policy_version 1946538 (0.0009) [2023-12-27 19:08:38,591][20381] Updated weights for policy 1, policy_version 1946548 (0.0009) [2023-12-27 19:08:38,644][20381] Updated weights for policy 1, policy_version 1946558 (0.0009) [2023-12-27 19:08:38,700][20381] Updated weights for policy 1, policy_version 1946568 (0.0008) [2023-12-27 19:08:38,952][20347] Updated weights for policy 0, policy_version 1948024 (0.0008) [2023-12-27 19:08:39,007][20347] Updated weights for policy 0, policy_version 1948034 (0.0008) [2023-12-27 19:08:39,060][20347] Updated weights for policy 0, policy_version 1948044 (0.0008) [2023-12-27 19:08:39,462][20381] Updated weights for policy 1, policy_version 1946578 (0.0007) [2023-12-27 19:08:39,527][20381] Updated weights for policy 1, policy_version 1946588 (0.0007) [2023-12-27 19:08:39,586][20381] Updated weights for policy 1, policy_version 1946598 (0.0010) [2023-12-27 19:08:39,742][20347] Updated weights for policy 0, policy_version 1948054 (0.0006) [2023-12-27 19:08:39,805][20347] Updated weights for policy 0, policy_version 1948064 (0.0006) [2023-12-27 19:08:39,874][20347] Updated weights for policy 0, policy_version 1948074 (0.0008) [2023-12-27 19:08:40,263][20381] Updated weights for policy 1, policy_version 1946608 (0.0007) [2023-12-27 19:08:40,319][20381] Updated weights for policy 1, policy_version 1946618 (0.0007) [2023-12-27 19:08:40,375][20381] Updated weights for policy 1, policy_version 1946628 (0.0007) [2023-12-27 19:08:40,600][20347] Updated weights for policy 0, policy_version 1948084 (0.0008) [2023-12-27 19:08:40,653][20347] Updated weights for policy 0, policy_version 1948094 (0.0008) [2023-12-27 19:08:40,701][20347] Updated weights for policy 0, policy_version 1948104 (0.0008) [2023-12-27 19:08:40,983][20381] Updated weights for policy 1, policy_version 1946638 (0.0006) [2023-12-27 19:08:41,055][20381] Updated weights for policy 1, policy_version 1946648 (0.0007) [2023-12-27 19:08:41,113][20381] Updated weights for policy 1, policy_version 1946658 (0.0006) [2023-12-27 19:08:41,490][20347] Updated weights for policy 0, policy_version 1948114 (0.0007) [2023-12-27 19:08:41,552][20347] Updated weights for policy 0, policy_version 1948124 (0.0009) [2023-12-27 19:08:41,614][20347] Updated weights for policy 0, policy_version 1948134 (0.0008) [2023-12-27 19:08:41,678][20347] Updated weights for policy 0, policy_version 1948144 (0.0009) [2023-12-27 19:08:41,826][20381] Updated weights for policy 1, policy_version 1946668 (0.0008) [2023-12-27 19:08:41,884][20381] Updated weights for policy 1, policy_version 1946678 (0.0009) [2023-12-27 19:08:41,937][20381] Updated weights for policy 1, policy_version 1946688 (0.0008) [2023-12-27 19:08:42,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20616.6, 300 sec: 20605.0). Total num frames: 997236736. Throughput: 0: 10457.9, 1: 10211.8. Samples: 997230952. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:42,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.530')] [2023-12-27 19:08:42,438][20347] Updated weights for policy 0, policy_version 1948154 (0.0010) [2023-12-27 19:08:42,502][20347] Updated weights for policy 0, policy_version 1948164 (0.0008) [2023-12-27 19:08:42,569][20347] Updated weights for policy 0, policy_version 1948174 (0.0008) [2023-12-27 19:08:42,642][20381] Updated weights for policy 1, policy_version 1946698 (0.0007) [2023-12-27 19:08:42,699][20381] Updated weights for policy 1, policy_version 1946708 (0.0005) [2023-12-27 19:08:42,754][20381] Updated weights for policy 1, policy_version 1946718 (0.0005) [2023-12-27 19:08:42,806][20381] Updated weights for policy 1, policy_version 1946728 (0.0007) [2023-12-27 19:08:43,273][20347] Updated weights for policy 0, policy_version 1948184 (0.0007) [2023-12-27 19:08:43,326][20347] Updated weights for policy 0, policy_version 1948194 (0.0008) [2023-12-27 19:08:43,382][20347] Updated weights for policy 0, policy_version 1948204 (0.0008) [2023-12-27 19:08:43,450][20381] Updated weights for policy 1, policy_version 1946738 (0.0007) [2023-12-27 19:08:43,506][20381] Updated weights for policy 1, policy_version 1946748 (0.0008) [2023-12-27 19:08:43,565][20381] Updated weights for policy 1, policy_version 1946758 (0.0007) [2023-12-27 19:08:44,015][20347] Updated weights for policy 0, policy_version 1948214 (0.0007) [2023-12-27 19:08:44,073][20347] Updated weights for policy 0, policy_version 1948224 (0.0009) [2023-12-27 19:08:44,123][20347] Updated weights for policy 0, policy_version 1948234 (0.0008) [2023-12-27 19:08:44,219][20381] Updated weights for policy 1, policy_version 1946768 (0.0008) [2023-12-27 19:08:44,264][20381] Updated weights for policy 1, policy_version 1946778 (0.0007) [2023-12-27 19:08:44,308][20381] Updated weights for policy 1, policy_version 1946788 (0.0011) [2023-12-27 19:08:44,792][20347] Updated weights for policy 0, policy_version 1948244 (0.0007) [2023-12-27 19:08:44,853][20347] Updated weights for policy 0, policy_version 1948254 (0.0006) [2023-12-27 19:08:44,904][20347] Updated weights for policy 0, policy_version 1948264 (0.0007) [2023-12-27 19:08:44,981][20381] Updated weights for policy 1, policy_version 1946798 (0.0009) [2023-12-27 19:08:45,041][20381] Updated weights for policy 1, policy_version 1946808 (0.0006) [2023-12-27 19:08:45,098][20381] Updated weights for policy 1, policy_version 1946818 (0.0005) [2023-12-27 19:08:45,596][20347] Updated weights for policy 0, policy_version 1948274 (0.0009) [2023-12-27 19:08:45,650][20347] Updated weights for policy 0, policy_version 1948284 (0.0010) [2023-12-27 19:08:45,701][20347] Updated weights for policy 0, policy_version 1948294 (0.0010) [2023-12-27 19:08:45,724][20381] Updated weights for policy 1, policy_version 1946828 (0.0006) [2023-12-27 19:08:45,762][20347] Updated weights for policy 0, policy_version 1948304 (0.0010) [2023-12-27 19:08:45,772][20381] Updated weights for policy 1, policy_version 1946838 (0.0010) [2023-12-27 19:08:45,826][20381] Updated weights for policy 1, policy_version 1946848 (0.0009) [2023-12-27 19:08:46,445][20381] Updated weights for policy 1, policy_version 1946858 (0.0007) [2023-12-27 19:08:46,453][20347] Updated weights for policy 0, policy_version 1948314 (0.0011) [2023-12-27 19:08:46,499][20381] Updated weights for policy 1, policy_version 1946868 (0.0008) [2023-12-27 19:08:46,510][20347] Updated weights for policy 0, policy_version 1948324 (0.0010) [2023-12-27 19:08:46,551][20381] Updated weights for policy 1, policy_version 1946878 (0.0005) [2023-12-27 19:08:46,565][20347] Updated weights for policy 0, policy_version 1948334 (0.0010) [2023-12-27 19:08:46,605][20381] Updated weights for policy 1, policy_version 1946888 (0.0006) [2023-12-27 19:08:47,225][20347] Updated weights for policy 0, policy_version 1948344 (0.0010) [2023-12-27 19:08:47,269][20381] Updated weights for policy 1, policy_version 1946898 (0.0010) [2023-12-27 19:08:47,273][20347] Updated weights for policy 0, policy_version 1948354 (0.0010) [2023-12-27 19:08:47,321][20347] Updated weights for policy 0, policy_version 1948364 (0.0010) [2023-12-27 19:08:47,323][20381] Updated weights for policy 1, policy_version 1946908 (0.0007) [2023-12-27 19:08:47,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 997343232. Throughput: 0: 10426.5, 1: 10309.6. Samples: 997353516. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.180')] [2023-12-27 19:08:47,377][20381] Updated weights for policy 1, policy_version 1946918 (0.0010) [2023-12-27 19:08:47,999][20347] Updated weights for policy 0, policy_version 1948374 (0.0010) [2023-12-27 19:08:48,042][20381] Updated weights for policy 1, policy_version 1946928 (0.0011) [2023-12-27 19:08:48,051][20347] Updated weights for policy 0, policy_version 1948384 (0.0010) [2023-12-27 19:08:48,096][20347] Updated weights for policy 0, policy_version 1948394 (0.0010) [2023-12-27 19:08:48,104][20381] Updated weights for policy 1, policy_version 1946938 (0.0011) [2023-12-27 19:08:48,162][20381] Updated weights for policy 1, policy_version 1946948 (0.0010) [2023-12-27 19:08:48,758][20347] Updated weights for policy 0, policy_version 1948404 (0.0008) [2023-12-27 19:08:48,824][20347] Updated weights for policy 0, policy_version 1948414 (0.0006) [2023-12-27 19:08:48,865][20381] Updated weights for policy 1, policy_version 1946958 (0.0008) [2023-12-27 19:08:48,887][20347] Updated weights for policy 0, policy_version 1948424 (0.0007) [2023-12-27 19:08:48,934][20381] Updated weights for policy 1, policy_version 1946968 (0.0010) [2023-12-27 19:08:48,992][20381] Updated weights for policy 1, policy_version 1946978 (0.0008) [2023-12-27 19:08:49,537][20347] Updated weights for policy 0, policy_version 1948434 (0.0008) [2023-12-27 19:08:49,586][20347] Updated weights for policy 0, policy_version 1948444 (0.0007) [2023-12-27 19:08:49,634][20347] Updated weights for policy 0, policy_version 1948454 (0.0008) [2023-12-27 19:08:49,688][20347] Updated weights for policy 0, policy_version 1948464 (0.0007) [2023-12-27 19:08:49,729][20381] Updated weights for policy 1, policy_version 1946988 (0.0008) [2023-12-27 19:08:49,788][20381] Updated weights for policy 1, policy_version 1946998 (0.0007) [2023-12-27 19:08:49,847][20381] Updated weights for policy 1, policy_version 1947008 (0.0008) [2023-12-27 19:08:50,386][20347] Updated weights for policy 0, policy_version 1948474 (0.0006) [2023-12-27 19:08:50,439][20347] Updated weights for policy 0, policy_version 1948484 (0.0007) [2023-12-27 19:08:50,502][20347] Updated weights for policy 0, policy_version 1948494 (0.0008) [2023-12-27 19:08:50,586][20381] Updated weights for policy 1, policy_version 1947018 (0.0007) [2023-12-27 19:08:50,646][20381] Updated weights for policy 1, policy_version 1947028 (0.0008) [2023-12-27 19:08:50,701][20381] Updated weights for policy 1, policy_version 1947038 (0.0005) [2023-12-27 19:08:50,749][20381] Updated weights for policy 1, policy_version 1947048 (0.0006) [2023-12-27 19:08:51,151][20347] Updated weights for policy 0, policy_version 1948504 (0.0009) [2023-12-27 19:08:51,210][20347] Updated weights for policy 0, policy_version 1948514 (0.0007) [2023-12-27 19:08:51,277][20347] Updated weights for policy 0, policy_version 1948524 (0.0008) [2023-12-27 19:08:51,392][20381] Updated weights for policy 1, policy_version 1947058 (0.0007) [2023-12-27 19:08:51,446][20381] Updated weights for policy 1, policy_version 1947068 (0.0008) [2023-12-27 19:08:51,506][20381] Updated weights for policy 1, policy_version 1947078 (0.0006) [2023-12-27 19:08:51,985][20347] Updated weights for policy 0, policy_version 1948534 (0.0005) [2023-12-27 19:08:52,043][20347] Updated weights for policy 0, policy_version 1948544 (0.0007) [2023-12-27 19:08:52,103][20347] Updated weights for policy 0, policy_version 1948554 (0.0007) [2023-12-27 19:08:52,217][20381] Updated weights for policy 1, policy_version 1947089 (0.0008) [2023-12-27 19:08:52,275][20381] Updated weights for policy 1, policy_version 1947099 (0.0007) [2023-12-27 19:08:52,340][20381] Updated weights for policy 1, policy_version 1947109 (0.0008) [2023-12-27 19:08:52,362][19170] Fps is (10 sec: 21298.9, 60 sec: 20753.0, 300 sec: 20605.0). Total num frames: 997449728. Throughput: 0: 10418.2, 1: 10279.6. Samples: 997413436. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.500')] [2023-12-27 19:08:52,374][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001947112_498524160.pth... [2023-12-27 19:08:52,374][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001948560_498925568.pth... [2023-12-27 19:08:52,379][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001945896_498212864.pth [2023-12-27 19:08:52,379][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001947352_498614272.pth [2023-12-27 19:08:52,846][20347] Updated weights for policy 0, policy_version 1948564 (0.0008) [2023-12-27 19:08:52,898][20347] Updated weights for policy 0, policy_version 1948574 (0.0008) [2023-12-27 19:08:52,951][20347] Updated weights for policy 0, policy_version 1948584 (0.0007) [2023-12-27 19:08:52,995][20381] Updated weights for policy 1, policy_version 1947119 (0.0009) [2023-12-27 19:08:53,044][20381] Updated weights for policy 1, policy_version 1947129 (0.0008) [2023-12-27 19:08:53,092][20381] Updated weights for policy 1, policy_version 1947139 (0.0008) [2023-12-27 19:08:53,612][20347] Updated weights for policy 0, policy_version 1948594 (0.0008) [2023-12-27 19:08:53,673][20347] Updated weights for policy 0, policy_version 1948604 (0.0008) [2023-12-27 19:08:53,731][20347] Updated weights for policy 0, policy_version 1948614 (0.0007) [2023-12-27 19:08:53,792][20347] Updated weights for policy 0, policy_version 1948624 (0.0007) [2023-12-27 19:08:53,825][20381] Updated weights for policy 1, policy_version 1947149 (0.0008) [2023-12-27 19:08:53,883][20381] Updated weights for policy 1, policy_version 1947159 (0.0008) [2023-12-27 19:08:53,938][20381] Updated weights for policy 1, policy_version 1947169 (0.0007) [2023-12-27 19:08:54,477][20347] Updated weights for policy 0, policy_version 1948634 (0.0006) [2023-12-27 19:08:54,528][20347] Updated weights for policy 0, policy_version 1948644 (0.0006) [2023-12-27 19:08:54,573][20347] Updated weights for policy 0, policy_version 1948654 (0.0006) [2023-12-27 19:08:54,575][20381] Updated weights for policy 1, policy_version 1947179 (0.0008) [2023-12-27 19:08:54,630][20381] Updated weights for policy 1, policy_version 1947189 (0.0007) [2023-12-27 19:08:54,687][20381] Updated weights for policy 1, policy_version 1947199 (0.0007) [2023-12-27 19:08:55,217][20347] Updated weights for policy 0, policy_version 1948664 (0.0005) [2023-12-27 19:08:55,277][20347] Updated weights for policy 0, policy_version 1948674 (0.0005) [2023-12-27 19:08:55,311][20381] Updated weights for policy 1, policy_version 1947209 (0.0007) [2023-12-27 19:08:55,337][20347] Updated weights for policy 0, policy_version 1948684 (0.0006) [2023-12-27 19:08:55,367][20381] Updated weights for policy 1, policy_version 1947219 (0.0005) [2023-12-27 19:08:55,427][20381] Updated weights for policy 1, policy_version 1947229 (0.0005) [2023-12-27 19:08:55,490][20381] Updated weights for policy 1, policy_version 1947239 (0.0005) [2023-12-27 19:08:55,921][20347] Updated weights for policy 0, policy_version 1948694 (0.0010) [2023-12-27 19:08:55,976][20347] Updated weights for policy 0, policy_version 1948704 (0.0010) [2023-12-27 19:08:56,021][20381] Updated weights for policy 1, policy_version 1947249 (0.0006) [2023-12-27 19:08:56,031][20347] Updated weights for policy 0, policy_version 1948714 (0.0009) [2023-12-27 19:08:56,075][20381] Updated weights for policy 1, policy_version 1947259 (0.0008) [2023-12-27 19:08:56,138][20381] Updated weights for policy 1, policy_version 1947269 (0.0008) [2023-12-27 19:08:56,681][20347] Updated weights for policy 0, policy_version 1948724 (0.0007) [2023-12-27 19:08:56,710][20381] Updated weights for policy 1, policy_version 1947279 (0.0009) [2023-12-27 19:08:56,738][20347] Updated weights for policy 0, policy_version 1948734 (0.0010) [2023-12-27 19:08:56,762][20381] Updated weights for policy 1, policy_version 1947289 (0.0005) [2023-12-27 19:08:56,798][20347] Updated weights for policy 0, policy_version 1948744 (0.0010) [2023-12-27 19:08:56,821][20381] Updated weights for policy 1, policy_version 1947299 (0.0007) [2023-12-27 19:08:57,362][19170] Fps is (10 sec: 20480.4, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 997548032. Throughput: 0: 10369.8, 1: 10327.8. Samples: 997538840. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:08:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.870')] [2023-12-27 19:08:57,449][20381] Updated weights for policy 1, policy_version 1947309 (0.0008) [2023-12-27 19:08:57,460][20347] Updated weights for policy 0, policy_version 1948754 (0.0010) [2023-12-27 19:08:57,502][20381] Updated weights for policy 1, policy_version 1947319 (0.0010) [2023-12-27 19:08:57,517][20347] Updated weights for policy 0, policy_version 1948764 (0.0010) [2023-12-27 19:08:57,550][20381] Updated weights for policy 1, policy_version 1947329 (0.0010) [2023-12-27 19:08:57,568][20347] Updated weights for policy 0, policy_version 1948774 (0.0010) [2023-12-27 19:08:57,619][20347] Updated weights for policy 0, policy_version 1948784 (0.0010) [2023-12-27 19:08:58,184][20381] Updated weights for policy 1, policy_version 1947339 (0.0010) [2023-12-27 19:08:58,250][20381] Updated weights for policy 1, policy_version 1947349 (0.0009) [2023-12-27 19:08:58,252][20347] Updated weights for policy 0, policy_version 1948794 (0.0007) [2023-12-27 19:08:58,306][20381] Updated weights for policy 1, policy_version 1947359 (0.0007) [2023-12-27 19:08:58,312][20347] Updated weights for policy 0, policy_version 1948804 (0.0008) [2023-12-27 19:08:58,375][20347] Updated weights for policy 0, policy_version 1948814 (0.0008) [2023-12-27 19:08:59,112][20381] Updated weights for policy 1, policy_version 1947369 (0.0008) [2023-12-27 19:08:59,113][20347] Updated weights for policy 0, policy_version 1948824 (0.0008) [2023-12-27 19:08:59,165][20381] Updated weights for policy 1, policy_version 1947379 (0.0010) [2023-12-27 19:08:59,174][20347] Updated weights for policy 0, policy_version 1948834 (0.0008) [2023-12-27 19:08:59,217][20381] Updated weights for policy 1, policy_version 1947389 (0.0010) [2023-12-27 19:08:59,239][20347] Updated weights for policy 0, policy_version 1948845 (0.0008) [2023-12-27 19:08:59,285][20381] Updated weights for policy 1, policy_version 1947399 (0.0009) [2023-12-27 19:08:59,978][20347] Updated weights for policy 0, policy_version 1948856 (0.0010) [2023-12-27 19:08:59,979][20381] Updated weights for policy 1, policy_version 1947409 (0.0009) [2023-12-27 19:09:00,037][20381] Updated weights for policy 1, policy_version 1947419 (0.0011) [2023-12-27 19:09:00,042][20347] Updated weights for policy 0, policy_version 1948866 (0.0006) [2023-12-27 19:09:00,089][20381] Updated weights for policy 1, policy_version 1947429 (0.0010) [2023-12-27 19:09:00,100][20347] Updated weights for policy 0, policy_version 1948876 (0.0009) [2023-12-27 19:09:00,737][20347] Updated weights for policy 0, policy_version 1948886 (0.0010) [2023-12-27 19:09:00,785][20381] Updated weights for policy 1, policy_version 1947439 (0.0007) [2023-12-27 19:09:00,791][20347] Updated weights for policy 0, policy_version 1948896 (0.0010) [2023-12-27 19:09:00,838][20381] Updated weights for policy 1, policy_version 1947449 (0.0005) [2023-12-27 19:09:00,846][20347] Updated weights for policy 0, policy_version 1948906 (0.0010) [2023-12-27 19:09:00,893][20381] Updated weights for policy 1, policy_version 1947459 (0.0009) [2023-12-27 19:09:01,532][20347] Updated weights for policy 0, policy_version 1948916 (0.0009) [2023-12-27 19:09:01,549][20381] Updated weights for policy 1, policy_version 1947469 (0.0008) [2023-12-27 19:09:01,582][20347] Updated weights for policy 0, policy_version 1948926 (0.0006) [2023-12-27 19:09:01,603][20381] Updated weights for policy 1, policy_version 1947479 (0.0006) [2023-12-27 19:09:01,644][20347] Updated weights for policy 0, policy_version 1948936 (0.0008) [2023-12-27 19:09:01,666][20381] Updated weights for policy 1, policy_version 1947489 (0.0006) [2023-12-27 19:09:02,247][20347] Updated weights for policy 0, policy_version 1948946 (0.0007) [2023-12-27 19:09:02,304][20347] Updated weights for policy 0, policy_version 1948956 (0.0008) [2023-12-27 19:09:02,359][20347] Updated weights for policy 0, policy_version 1948966 (0.0008) [2023-12-27 19:09:02,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 997646336. Throughput: 0: 10393.8, 1: 10286.3. Samples: 997661324. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-12-27 19:09:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.320')] [2023-12-27 19:09:02,425][20381] Updated weights for policy 1, policy_version 1947499 (0.0007) [2023-12-27 19:09:02,429][20347] Updated weights for policy 0, policy_version 1948976 (0.0009) [2023-12-27 19:09:02,483][20381] Updated weights for policy 1, policy_version 1947509 (0.0007) [2023-12-27 19:09:02,540][20381] Updated weights for policy 1, policy_version 1947519 (0.0005) [2023-12-27 19:09:03,056][20347] Updated weights for policy 0, policy_version 1948986 (0.0008) [2023-12-27 19:09:03,106][20347] Updated weights for policy 0, policy_version 1948996 (0.0005) [2023-12-27 19:09:03,122][20381] Updated weights for policy 1, policy_version 1947529 (0.0007) [2023-12-27 19:09:03,154][20347] Updated weights for policy 0, policy_version 1949006 (0.0007) [2023-12-27 19:09:03,181][20381] Updated weights for policy 1, policy_version 1947539 (0.0006) [2023-12-27 19:09:03,240][20381] Updated weights for policy 1, policy_version 1947549 (0.0007) [2023-12-27 19:09:03,293][20381] Updated weights for policy 1, policy_version 1947559 (0.0006) [2023-12-27 19:09:03,821][20347] Updated weights for policy 0, policy_version 1949016 (0.0006) [2023-12-27 19:09:03,888][20347] Updated weights for policy 0, policy_version 1949026 (0.0008) [2023-12-27 19:09:03,947][20347] Updated weights for policy 0, policy_version 1949036 (0.0009) [2023-12-27 19:09:03,969][20381] Updated weights for policy 1, policy_version 1947569 (0.0010) [2023-12-27 19:09:04,023][20381] Updated weights for policy 1, policy_version 1947579 (0.0011) [2023-12-27 19:09:04,072][20381] Updated weights for policy 1, policy_version 1947589 (0.0010) [2023-12-27 19:09:04,526][20347] Updated weights for policy 0, policy_version 1949046 (0.0008) [2023-12-27 19:09:04,584][20347] Updated weights for policy 0, policy_version 1949056 (0.0009) [2023-12-27 19:09:04,640][20347] Updated weights for policy 0, policy_version 1949066 (0.0010) [2023-12-27 19:09:04,732][20381] Updated weights for policy 1, policy_version 1947599 (0.0010) [2023-12-27 19:09:04,787][20381] Updated weights for policy 1, policy_version 1947609 (0.0010) [2023-12-27 19:09:04,845][20381] Updated weights for policy 1, policy_version 1947619 (0.0011) [2023-12-27 19:09:05,233][20347] Updated weights for policy 0, policy_version 1949076 (0.0009) [2023-12-27 19:09:05,280][20347] Updated weights for policy 0, policy_version 1949086 (0.0007) [2023-12-27 19:09:05,324][20347] Updated weights for policy 0, policy_version 1949096 (0.0007) [2023-12-27 19:09:05,540][20381] Updated weights for policy 1, policy_version 1947629 (0.0010) [2023-12-27 19:09:05,597][20381] Updated weights for policy 1, policy_version 1947639 (0.0011) [2023-12-27 19:09:05,645][20381] Updated weights for policy 1, policy_version 1947649 (0.0010) [2023-12-27 19:09:05,987][20347] Updated weights for policy 0, policy_version 1949106 (0.0007) [2023-12-27 19:09:06,044][20347] Updated weights for policy 0, policy_version 1949116 (0.0006) [2023-12-27 19:09:06,100][20347] Updated weights for policy 0, policy_version 1949126 (0.0007) [2023-12-27 19:09:06,160][20347] Updated weights for policy 0, policy_version 1949136 (0.0007) [2023-12-27 19:09:06,303][20381] Updated weights for policy 1, policy_version 1947659 (0.0009) [2023-12-27 19:09:06,363][20381] Updated weights for policy 1, policy_version 1947669 (0.0011) [2023-12-27 19:09:06,425][20381] Updated weights for policy 1, policy_version 1947679 (0.0011) [2023-12-27 19:09:06,839][20347] Updated weights for policy 0, policy_version 1949146 (0.0005) [2023-12-27 19:09:06,893][20347] Updated weights for policy 0, policy_version 1949156 (0.0005) [2023-12-27 19:09:06,950][20347] Updated weights for policy 0, policy_version 1949166 (0.0005) [2023-12-27 19:09:07,021][20381] Updated weights for policy 1, policy_version 1947689 (0.0010) [2023-12-27 19:09:07,078][20381] Updated weights for policy 1, policy_version 1947699 (0.0007) [2023-12-27 19:09:07,139][20381] Updated weights for policy 1, policy_version 1947709 (0.0009) [2023-12-27 19:09:07,199][20381] Updated weights for policy 1, policy_version 1947719 (0.0008) [2023-12-27 19:09:07,362][19170] Fps is (10 sec: 21299.0, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 997761024. Throughput: 0: 10417.6, 1: 10306.2. Samples: 997725980. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.600')] [2023-12-27 19:09:07,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001947720_498679808.pth... [2023-12-27 19:09:07,367][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001949168_499081216.pth... [2023-12-27 19:09:07,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001946504_498368512.pth [2023-12-27 19:09:07,371][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001947984_498778112.pth [2023-12-27 19:09:07,540][20347] Updated weights for policy 0, policy_version 1949176 (0.0006) [2023-12-27 19:09:07,585][20347] Updated weights for policy 0, policy_version 1949186 (0.0006) [2023-12-27 19:09:07,634][20347] Updated weights for policy 0, policy_version 1949196 (0.0005) [2023-12-27 19:09:07,856][20381] Updated weights for policy 1, policy_version 1947729 (0.0007) [2023-12-27 19:09:07,915][20381] Updated weights for policy 1, policy_version 1947739 (0.0008) [2023-12-27 19:09:07,971][20381] Updated weights for policy 1, policy_version 1947749 (0.0008) [2023-12-27 19:09:08,306][20347] Updated weights for policy 0, policy_version 1949206 (0.0006) [2023-12-27 19:09:08,381][20347] Updated weights for policy 0, policy_version 1949216 (0.0007) [2023-12-27 19:09:08,441][20347] Updated weights for policy 0, policy_version 1949226 (0.0006) [2023-12-27 19:09:08,678][20381] Updated weights for policy 1, policy_version 1947759 (0.0009) [2023-12-27 19:09:08,736][20381] Updated weights for policy 1, policy_version 1947769 (0.0009) [2023-12-27 19:09:08,792][20381] Updated weights for policy 1, policy_version 1947779 (0.0009) [2023-12-27 19:09:09,030][20347] Updated weights for policy 0, policy_version 1949236 (0.0006) [2023-12-27 19:09:09,076][20347] Updated weights for policy 0, policy_version 1949246 (0.0008) [2023-12-27 19:09:09,128][20347] Updated weights for policy 0, policy_version 1949256 (0.0007) [2023-12-27 19:09:09,546][20381] Updated weights for policy 1, policy_version 1947789 (0.0007) [2023-12-27 19:09:09,613][20381] Updated weights for policy 1, policy_version 1947799 (0.0007) [2023-12-27 19:09:09,687][20381] Updated weights for policy 1, policy_version 1947809 (0.0007) [2023-12-27 19:09:09,900][20347] Updated weights for policy 0, policy_version 1949266 (0.0008) [2023-12-27 19:09:09,968][20347] Updated weights for policy 0, policy_version 1949276 (0.0009) [2023-12-27 19:09:10,032][20347] Updated weights for policy 0, policy_version 1949286 (0.0007) [2023-12-27 19:09:10,088][20347] Updated weights for policy 0, policy_version 1949296 (0.0006) [2023-12-27 19:09:10,427][20381] Updated weights for policy 1, policy_version 1947819 (0.0009) [2023-12-27 19:09:10,486][20381] Updated weights for policy 1, policy_version 1947829 (0.0008) [2023-12-27 19:09:10,545][20381] Updated weights for policy 1, policy_version 1947839 (0.0007) [2023-12-27 19:09:10,723][20347] Updated weights for policy 0, policy_version 1949306 (0.0008) [2023-12-27 19:09:10,774][20347] Updated weights for policy 0, policy_version 1949316 (0.0005) [2023-12-27 19:09:10,835][20347] Updated weights for policy 0, policy_version 1949326 (0.0006) [2023-12-27 19:09:11,161][20381] Updated weights for policy 1, policy_version 1947849 (0.0008) [2023-12-27 19:09:11,223][20381] Updated weights for policy 1, policy_version 1947859 (0.0006) [2023-12-27 19:09:11,282][20381] Updated weights for policy 1, policy_version 1947869 (0.0006) [2023-12-27 19:09:11,339][20381] Updated weights for policy 1, policy_version 1947879 (0.0006) [2023-12-27 19:09:11,577][20347] Updated weights for policy 0, policy_version 1949336 (0.0007) [2023-12-27 19:09:11,641][20347] Updated weights for policy 0, policy_version 1949346 (0.0008) [2023-12-27 19:09:11,697][20347] Updated weights for policy 0, policy_version 1949356 (0.0007) [2023-12-27 19:09:12,012][20381] Updated weights for policy 1, policy_version 1947889 (0.0009) [2023-12-27 19:09:12,064][20381] Updated weights for policy 1, policy_version 1947899 (0.0010) [2023-12-27 19:09:12,125][20381] Updated weights for policy 1, policy_version 1947909 (0.0011) [2023-12-27 19:09:12,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.0, 300 sec: 20577.2). Total num frames: 997859328. Throughput: 0: 10373.0, 1: 10319.1. Samples: 997849920. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '853.510')] [2023-12-27 19:09:12,388][20347] Updated weights for policy 0, policy_version 1949366 (0.0008) [2023-12-27 19:09:12,451][20347] Updated weights for policy 0, policy_version 1949376 (0.0008) [2023-12-27 19:09:12,517][20347] Updated weights for policy 0, policy_version 1949386 (0.0008) [2023-12-27 19:09:12,856][20381] Updated weights for policy 1, policy_version 1947919 (0.0011) [2023-12-27 19:09:12,909][20381] Updated weights for policy 1, policy_version 1947929 (0.0011) [2023-12-27 19:09:12,955][20381] Updated weights for policy 1, policy_version 1947939 (0.0010) [2023-12-27 19:09:13,201][20347] Updated weights for policy 0, policy_version 1949396 (0.0008) [2023-12-27 19:09:13,252][20347] Updated weights for policy 0, policy_version 1949406 (0.0010) [2023-12-27 19:09:13,305][20347] Updated weights for policy 0, policy_version 1949416 (0.0010) [2023-12-27 19:09:13,582][20381] Updated weights for policy 1, policy_version 1947949 (0.0008) [2023-12-27 19:09:13,638][20381] Updated weights for policy 1, policy_version 1947959 (0.0005) [2023-12-27 19:09:13,688][20381] Updated weights for policy 1, policy_version 1947969 (0.0005) [2023-12-27 19:09:13,967][20347] Updated weights for policy 0, policy_version 1949426 (0.0009) [2023-12-27 19:09:14,021][20347] Updated weights for policy 0, policy_version 1949436 (0.0005) [2023-12-27 19:09:14,068][20347] Updated weights for policy 0, policy_version 1949446 (0.0005) [2023-12-27 19:09:14,115][20347] Updated weights for policy 0, policy_version 1949456 (0.0005) [2023-12-27 19:09:14,254][20381] Updated weights for policy 1, policy_version 1947979 (0.0005) [2023-12-27 19:09:14,307][20381] Updated weights for policy 1, policy_version 1947989 (0.0006) [2023-12-27 19:09:14,356][20381] Updated weights for policy 1, policy_version 1947999 (0.0009) [2023-12-27 19:09:14,720][20347] Updated weights for policy 0, policy_version 1949466 (0.0005) [2023-12-27 19:09:14,789][20347] Updated weights for policy 0, policy_version 1949476 (0.0007) [2023-12-27 19:09:14,853][20347] Updated weights for policy 0, policy_version 1949486 (0.0008) [2023-12-27 19:09:14,946][20381] Updated weights for policy 1, policy_version 1948009 (0.0005) [2023-12-27 19:09:15,010][20381] Updated weights for policy 1, policy_version 1948019 (0.0009) [2023-12-27 19:09:15,074][20381] Updated weights for policy 1, policy_version 1948029 (0.0007) [2023-12-27 19:09:15,143][20381] Updated weights for policy 1, policy_version 1948039 (0.0008) [2023-12-27 19:09:15,530][20347] Updated weights for policy 0, policy_version 1949496 (0.0008) [2023-12-27 19:09:15,593][20347] Updated weights for policy 0, policy_version 1949506 (0.0008) [2023-12-27 19:09:15,658][20347] Updated weights for policy 0, policy_version 1949516 (0.0008) [2023-12-27 19:09:15,818][20381] Updated weights for policy 1, policy_version 1948049 (0.0008) [2023-12-27 19:09:15,873][20381] Updated weights for policy 1, policy_version 1948059 (0.0007) [2023-12-27 19:09:15,921][20381] Updated weights for policy 1, policy_version 1948069 (0.0007) [2023-12-27 19:09:16,367][20347] Updated weights for policy 0, policy_version 1949526 (0.0007) [2023-12-27 19:09:16,416][20347] Updated weights for policy 0, policy_version 1949536 (0.0008) [2023-12-27 19:09:16,465][20347] Updated weights for policy 0, policy_version 1949546 (0.0007) [2023-12-27 19:09:16,521][20381] Updated weights for policy 1, policy_version 1948079 (0.0006) [2023-12-27 19:09:16,566][20381] Updated weights for policy 1, policy_version 1948089 (0.0007) [2023-12-27 19:09:16,616][20381] Updated weights for policy 1, policy_version 1948099 (0.0008) [2023-12-27 19:09:17,194][20347] Updated weights for policy 0, policy_version 1949556 (0.0007) [2023-12-27 19:09:17,249][20347] Updated weights for policy 0, policy_version 1949566 (0.0008) [2023-12-27 19:09:17,310][20347] Updated weights for policy 0, policy_version 1949576 (0.0007) [2023-12-27 19:09:17,319][20381] Updated weights for policy 1, policy_version 1948109 (0.0009) [2023-12-27 19:09:17,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 997965824. Throughput: 0: 10288.3, 1: 10347.6. Samples: 997975020. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.000')] [2023-12-27 19:09:17,379][20381] Updated weights for policy 1, policy_version 1948119 (0.0006) [2023-12-27 19:09:17,437][20381] Updated weights for policy 1, policy_version 1948129 (0.0008) [2023-12-27 19:09:17,996][20347] Updated weights for policy 0, policy_version 1949586 (0.0008) [2023-12-27 19:09:18,044][20347] Updated weights for policy 0, policy_version 1949596 (0.0008) [2023-12-27 19:09:18,052][20381] Updated weights for policy 1, policy_version 1948139 (0.0007) [2023-12-27 19:09:18,093][20347] Updated weights for policy 0, policy_version 1949606 (0.0007) [2023-12-27 19:09:18,110][20381] Updated weights for policy 1, policy_version 1948149 (0.0005) [2023-12-27 19:09:18,146][20347] Updated weights for policy 0, policy_version 1949616 (0.0007) [2023-12-27 19:09:18,166][20381] Updated weights for policy 1, policy_version 1948159 (0.0005) [2023-12-27 19:09:18,753][20381] Updated weights for policy 1, policy_version 1948169 (0.0006) [2023-12-27 19:09:18,810][20381] Updated weights for policy 1, policy_version 1948179 (0.0008) [2023-12-27 19:09:18,863][20381] Updated weights for policy 1, policy_version 1948189 (0.0008) [2023-12-27 19:09:18,917][20381] Updated weights for policy 1, policy_version 1948199 (0.0008) [2023-12-27 19:09:18,939][20347] Updated weights for policy 0, policy_version 1949626 (0.0005) [2023-12-27 19:09:18,990][20347] Updated weights for policy 0, policy_version 1949636 (0.0005) [2023-12-27 19:09:19,040][20347] Updated weights for policy 0, policy_version 1949646 (0.0005) [2023-12-27 19:09:19,634][20347] Updated weights for policy 0, policy_version 1949656 (0.0006) [2023-12-27 19:09:19,690][20347] Updated weights for policy 0, policy_version 1949666 (0.0006) [2023-12-27 19:09:19,699][20381] Updated weights for policy 1, policy_version 1948209 (0.0008) [2023-12-27 19:09:19,751][20347] Updated weights for policy 0, policy_version 1949676 (0.0006) [2023-12-27 19:09:19,756][20381] Updated weights for policy 1, policy_version 1948219 (0.0008) [2023-12-27 19:09:19,819][20381] Updated weights for policy 1, policy_version 1948229 (0.0009) [2023-12-27 19:09:20,399][20347] Updated weights for policy 0, policy_version 1949686 (0.0005) [2023-12-27 19:09:20,456][20347] Updated weights for policy 0, policy_version 1949696 (0.0005) [2023-12-27 19:09:20,517][20347] Updated weights for policy 0, policy_version 1949706 (0.0007) [2023-12-27 19:09:20,543][20381] Updated weights for policy 1, policy_version 1948239 (0.0007) [2023-12-27 19:09:20,603][20381] Updated weights for policy 1, policy_version 1948249 (0.0008) [2023-12-27 19:09:20,662][20381] Updated weights for policy 1, policy_version 1948259 (0.0007) [2023-12-27 19:09:21,209][20347] Updated weights for policy 0, policy_version 1949716 (0.0008) [2023-12-27 19:09:21,275][20347] Updated weights for policy 0, policy_version 1949726 (0.0011) [2023-12-27 19:09:21,311][20381] Updated weights for policy 1, policy_version 1948269 (0.0008) [2023-12-27 19:09:21,339][20347] Updated weights for policy 0, policy_version 1949736 (0.0010) [2023-12-27 19:09:21,376][20381] Updated weights for policy 1, policy_version 1948279 (0.0009) [2023-12-27 19:09:21,432][20381] Updated weights for policy 1, policy_version 1948289 (0.0008) [2023-12-27 19:09:22,074][20347] Updated weights for policy 0, policy_version 1949746 (0.0012) [2023-12-27 19:09:22,125][20347] Updated weights for policy 0, policy_version 1949756 (0.0007) [2023-12-27 19:09:22,177][20347] Updated weights for policy 0, policy_version 1949766 (0.0006) [2023-12-27 19:09:22,198][20381] Updated weights for policy 1, policy_version 1948299 (0.0007) [2023-12-27 19:09:22,233][20347] Updated weights for policy 0, policy_version 1949776 (0.0007) [2023-12-27 19:09:22,241][20381] Updated weights for policy 1, policy_version 1948309 (0.0008) [2023-12-27 19:09:22,301][20381] Updated weights for policy 1, policy_version 1948319 (0.0008) [2023-12-27 19:09:22,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20753.1, 300 sec: 20605.0). Total num frames: 998072320. Throughput: 0: 10286.2, 1: 10316.1. Samples: 998036212. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:22,363][19170] Avg episode reward: [(0, '1000.000'), (1, '872.750')] [2023-12-27 19:09:22,368][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001948328_498835456.pth... [2023-12-27 19:09:22,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001949776_499236864.pth... [2023-12-27 19:09:22,371][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001947112_498524160.pth [2023-12-27 19:09:22,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001948560_498925568.pth [2023-12-27 19:09:22,960][20347] Updated weights for policy 0, policy_version 1949786 (0.0007) [2023-12-27 19:09:23,014][20347] Updated weights for policy 0, policy_version 1949796 (0.0007) [2023-12-27 19:09:23,021][20381] Updated weights for policy 1, policy_version 1948329 (0.0008) [2023-12-27 19:09:23,063][20347] Updated weights for policy 0, policy_version 1949806 (0.0007) [2023-12-27 19:09:23,083][20381] Updated weights for policy 1, policy_version 1948339 (0.0007) [2023-12-27 19:09:23,145][20381] Updated weights for policy 1, policy_version 1948349 (0.0008) [2023-12-27 19:09:23,194][20381] Updated weights for policy 1, policy_version 1948359 (0.0008) [2023-12-27 19:09:23,768][20347] Updated weights for policy 0, policy_version 1949816 (0.0007) [2023-12-27 19:09:23,823][20347] Updated weights for policy 0, policy_version 1949826 (0.0007) [2023-12-27 19:09:23,870][20347] Updated weights for policy 0, policy_version 1949836 (0.0005) [2023-12-27 19:09:23,884][20381] Updated weights for policy 1, policy_version 1948369 (0.0006) [2023-12-27 19:09:23,935][20381] Updated weights for policy 1, policy_version 1948379 (0.0008) [2023-12-27 19:09:23,985][20381] Updated weights for policy 1, policy_version 1948389 (0.0008) [2023-12-27 19:09:24,593][20347] Updated weights for policy 0, policy_version 1949846 (0.0009) [2023-12-27 19:09:24,633][20381] Updated weights for policy 1, policy_version 1948399 (0.0006) [2023-12-27 19:09:24,645][20347] Updated weights for policy 0, policy_version 1949856 (0.0010) [2023-12-27 19:09:24,677][20381] Updated weights for policy 1, policy_version 1948409 (0.0005) [2023-12-27 19:09:24,697][20347] Updated weights for policy 0, policy_version 1949866 (0.0010) [2023-12-27 19:09:24,722][20381] Updated weights for policy 1, policy_version 1948419 (0.0006) [2023-12-27 19:09:25,391][20347] Updated weights for policy 0, policy_version 1949876 (0.0010) [2023-12-27 19:09:25,449][20347] Updated weights for policy 0, policy_version 1949886 (0.0010) [2023-12-27 19:09:25,449][20381] Updated weights for policy 1, policy_version 1948429 (0.0008) [2023-12-27 19:09:25,500][20347] Updated weights for policy 0, policy_version 1949896 (0.0010) [2023-12-27 19:09:25,502][20381] Updated weights for policy 1, policy_version 1948439 (0.0007) [2023-12-27 19:09:25,547][20381] Updated weights for policy 1, policy_version 1948449 (0.0007) [2023-12-27 19:09:26,180][20347] Updated weights for policy 0, policy_version 1949906 (0.0009) [2023-12-27 19:09:26,231][20347] Updated weights for policy 0, policy_version 1949916 (0.0008) [2023-12-27 19:09:26,248][20381] Updated weights for policy 1, policy_version 1948459 (0.0007) [2023-12-27 19:09:26,282][20347] Updated weights for policy 0, policy_version 1949926 (0.0008) [2023-12-27 19:09:26,302][20381] Updated weights for policy 1, policy_version 1948469 (0.0005) [2023-12-27 19:09:26,329][20347] Updated weights for policy 0, policy_version 1949936 (0.0008) [2023-12-27 19:09:26,360][20381] Updated weights for policy 1, policy_version 1948479 (0.0007) [2023-12-27 19:09:27,001][20347] Updated weights for policy 0, policy_version 1949946 (0.0010) [2023-12-27 19:09:27,043][20381] Updated weights for policy 1, policy_version 1948489 (0.0006) [2023-12-27 19:09:27,048][20347] Updated weights for policy 0, policy_version 1949956 (0.0006) [2023-12-27 19:09:27,102][20347] Updated weights for policy 0, policy_version 1949966 (0.0007) [2023-12-27 19:09:27,105][20381] Updated weights for policy 1, policy_version 1948499 (0.0005) [2023-12-27 19:09:27,157][20381] Updated weights for policy 1, policy_version 1948509 (0.0007) [2023-12-27 19:09:27,211][20381] Updated weights for policy 1, policy_version 1948519 (0.0007) [2023-12-27 19:09:27,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 998170624. Throughput: 0: 10225.6, 1: 10314.8. Samples: 998155272. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '833.780')] [2023-12-27 19:09:27,732][20347] Updated weights for policy 0, policy_version 1949976 (0.0007) [2023-12-27 19:09:27,794][20347] Updated weights for policy 0, policy_version 1949986 (0.0008) [2023-12-27 19:09:27,850][20347] Updated weights for policy 0, policy_version 1949996 (0.0008) [2023-12-27 19:09:27,903][20381] Updated weights for policy 1, policy_version 1948529 (0.0006) [2023-12-27 19:09:27,961][20381] Updated weights for policy 1, policy_version 1948539 (0.0007) [2023-12-27 19:09:28,014][20381] Updated weights for policy 1, policy_version 1948549 (0.0007) [2023-12-27 19:09:28,443][20347] Updated weights for policy 0, policy_version 1950006 (0.0007) [2023-12-27 19:09:28,502][20347] Updated weights for policy 0, policy_version 1950016 (0.0007) [2023-12-27 19:09:28,565][20347] Updated weights for policy 0, policy_version 1950026 (0.0006) [2023-12-27 19:09:28,722][20381] Updated weights for policy 1, policy_version 1948559 (0.0006) [2023-12-27 19:09:28,784][20381] Updated weights for policy 1, policy_version 1948569 (0.0006) [2023-12-27 19:09:28,850][20381] Updated weights for policy 1, policy_version 1948579 (0.0007) [2023-12-27 19:09:29,176][20347] Updated weights for policy 0, policy_version 1950036 (0.0006) [2023-12-27 19:09:29,224][20347] Updated weights for policy 0, policy_version 1950046 (0.0008) [2023-12-27 19:09:29,281][20347] Updated weights for policy 0, policy_version 1950056 (0.0007) [2023-12-27 19:09:29,503][20381] Updated weights for policy 1, policy_version 1948589 (0.0007) [2023-12-27 19:09:29,554][20381] Updated weights for policy 1, policy_version 1948599 (0.0008) [2023-12-27 19:09:29,606][20381] Updated weights for policy 1, policy_version 1948609 (0.0006) [2023-12-27 19:09:30,028][20347] Updated weights for policy 0, policy_version 1950066 (0.0007) [2023-12-27 19:09:30,094][20347] Updated weights for policy 0, policy_version 1950076 (0.0007) [2023-12-27 19:09:30,184][20347] Updated weights for policy 0, policy_version 1950086 (0.0009) [2023-12-27 19:09:30,234][20381] Updated weights for policy 1, policy_version 1948619 (0.0005) [2023-12-27 19:09:30,245][20347] Updated weights for policy 0, policy_version 1950096 (0.0008) [2023-12-27 19:09:30,293][20381] Updated weights for policy 1, policy_version 1948629 (0.0008) [2023-12-27 19:09:30,348][20381] Updated weights for policy 1, policy_version 1948639 (0.0008) [2023-12-27 19:09:30,862][20347] Updated weights for policy 0, policy_version 1950106 (0.0007) [2023-12-27 19:09:30,911][20347] Updated weights for policy 0, policy_version 1950116 (0.0008) [2023-12-27 19:09:30,956][20347] Updated weights for policy 0, policy_version 1950126 (0.0007) [2023-12-27 19:09:30,962][20381] Updated weights for policy 1, policy_version 1948649 (0.0008) [2023-12-27 19:09:31,011][20381] Updated weights for policy 1, policy_version 1948659 (0.0009) [2023-12-27 19:09:31,066][20381] Updated weights for policy 1, policy_version 1948669 (0.0009) [2023-12-27 19:09:31,128][20381] Updated weights for policy 1, policy_version 1948679 (0.0008) [2023-12-27 19:09:31,663][20347] Updated weights for policy 0, policy_version 1950136 (0.0007) [2023-12-27 19:09:31,720][20347] Updated weights for policy 0, policy_version 1950146 (0.0008) [2023-12-27 19:09:31,783][20347] Updated weights for policy 0, policy_version 1950156 (0.0008) [2023-12-27 19:09:31,882][20381] Updated weights for policy 1, policy_version 1948689 (0.0007) [2023-12-27 19:09:31,943][20381] Updated weights for policy 1, policy_version 1948699 (0.0006) [2023-12-27 19:09:31,996][20381] Updated weights for policy 1, policy_version 1948709 (0.0008) [2023-12-27 19:09:32,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 998268928. Throughput: 0: 10303.6, 1: 10272.9. Samples: 998279456. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.210')] [2023-12-27 19:09:32,425][20347] Updated weights for policy 0, policy_version 1950166 (0.0006) [2023-12-27 19:09:32,480][20347] Updated weights for policy 0, policy_version 1950176 (0.0008) [2023-12-27 19:09:32,538][20347] Updated weights for policy 0, policy_version 1950186 (0.0008) [2023-12-27 19:09:32,702][20381] Updated weights for policy 1, policy_version 1948719 (0.0008) [2023-12-27 19:09:32,752][20381] Updated weights for policy 1, policy_version 1948729 (0.0007) [2023-12-27 19:09:32,798][20381] Updated weights for policy 1, policy_version 1948739 (0.0007) [2023-12-27 19:09:33,213][20347] Updated weights for policy 0, policy_version 1950196 (0.0007) [2023-12-27 19:09:33,263][20347] Updated weights for policy 0, policy_version 1950206 (0.0006) [2023-12-27 19:09:33,314][20347] Updated weights for policy 0, policy_version 1950216 (0.0005) [2023-12-27 19:09:33,510][20381] Updated weights for policy 1, policy_version 1948749 (0.0008) [2023-12-27 19:09:33,565][20381] Updated weights for policy 1, policy_version 1948759 (0.0008) [2023-12-27 19:09:33,616][20381] Updated weights for policy 1, policy_version 1948769 (0.0006) [2023-12-27 19:09:33,989][20347] Updated weights for policy 0, policy_version 1950226 (0.0006) [2023-12-27 19:09:34,039][20347] Updated weights for policy 0, policy_version 1950236 (0.0007) [2023-12-27 19:09:34,091][20347] Updated weights for policy 0, policy_version 1950246 (0.0008) [2023-12-27 19:09:34,156][20347] Updated weights for policy 0, policy_version 1950256 (0.0007) [2023-12-27 19:09:34,345][20381] Updated weights for policy 1, policy_version 1948779 (0.0006) [2023-12-27 19:09:34,403][20381] Updated weights for policy 1, policy_version 1948789 (0.0008) [2023-12-27 19:09:34,464][20381] Updated weights for policy 1, policy_version 1948799 (0.0009) [2023-12-27 19:09:34,866][20347] Updated weights for policy 0, policy_version 1950266 (0.0007) [2023-12-27 19:09:34,913][20347] Updated weights for policy 0, policy_version 1950276 (0.0007) [2023-12-27 19:09:34,970][20347] Updated weights for policy 0, policy_version 1950286 (0.0007) [2023-12-27 19:09:35,163][20381] Updated weights for policy 1, policy_version 1948809 (0.0008) [2023-12-27 19:09:35,216][20381] Updated weights for policy 1, policy_version 1948819 (0.0010) [2023-12-27 19:09:35,268][20381] Updated weights for policy 1, policy_version 1948829 (0.0010) [2023-12-27 19:09:35,321][20381] Updated weights for policy 1, policy_version 1948839 (0.0010) [2023-12-27 19:09:35,680][20347] Updated weights for policy 0, policy_version 1950296 (0.0007) [2023-12-27 19:09:35,738][20347] Updated weights for policy 0, policy_version 1950306 (0.0008) [2023-12-27 19:09:35,795][20347] Updated weights for policy 0, policy_version 1950316 (0.0008) [2023-12-27 19:09:35,967][20381] Updated weights for policy 1, policy_version 1948849 (0.0006) [2023-12-27 19:09:36,025][20381] Updated weights for policy 1, policy_version 1948859 (0.0007) [2023-12-27 19:09:36,078][20381] Updated weights for policy 1, policy_version 1948869 (0.0011) [2023-12-27 19:09:36,495][20347] Updated weights for policy 0, policy_version 1950326 (0.0008) [2023-12-27 19:09:36,551][20347] Updated weights for policy 0, policy_version 1950336 (0.0007) [2023-12-27 19:09:36,612][20347] Updated weights for policy 0, policy_version 1950346 (0.0007) [2023-12-27 19:09:36,783][20381] Updated weights for policy 1, policy_version 1948879 (0.0007) [2023-12-27 19:09:36,835][20381] Updated weights for policy 1, policy_version 1948889 (0.0005) [2023-12-27 19:09:36,891][20381] Updated weights for policy 1, policy_version 1948899 (0.0008) [2023-12-27 19:09:37,183][20347] Updated weights for policy 0, policy_version 1950356 (0.0006) [2023-12-27 19:09:37,235][20347] Updated weights for policy 0, policy_version 1950366 (0.0008) [2023-12-27 19:09:37,287][20347] Updated weights for policy 0, policy_version 1950376 (0.0007) [2023-12-27 19:09:37,362][19170] Fps is (10 sec: 20479.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 998375424. Throughput: 0: 10295.1, 1: 10292.7. Samples: 998339888. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.360')] [2023-12-27 19:09:37,367][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001948904_498982912.pth... [2023-12-27 19:09:37,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001950384_499392512.pth... [2023-12-27 19:09:37,372][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001947720_498679808.pth [2023-12-27 19:09:37,372][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001949168_499081216.pth [2023-12-27 19:09:37,599][20381] Updated weights for policy 1, policy_version 1948909 (0.0009) [2023-12-27 19:09:37,656][20381] Updated weights for policy 1, policy_version 1948919 (0.0007) [2023-12-27 19:09:37,704][20381] Updated weights for policy 1, policy_version 1948929 (0.0010) [2023-12-27 19:09:37,920][20347] Updated weights for policy 0, policy_version 1950386 (0.0008) [2023-12-27 19:09:37,967][20347] Updated weights for policy 0, policy_version 1950396 (0.0005) [2023-12-27 19:09:38,021][20347] Updated weights for policy 0, policy_version 1950406 (0.0005) [2023-12-27 19:09:38,069][20347] Updated weights for policy 0, policy_version 1950416 (0.0005) [2023-12-27 19:09:38,384][20381] Updated weights for policy 1, policy_version 1948939 (0.0010) [2023-12-27 19:09:38,443][20381] Updated weights for policy 1, policy_version 1948949 (0.0011) [2023-12-27 19:09:38,511][20381] Updated weights for policy 1, policy_version 1948959 (0.0011) [2023-12-27 19:09:38,757][20347] Updated weights for policy 0, policy_version 1950426 (0.0005) [2023-12-27 19:09:38,821][20347] Updated weights for policy 0, policy_version 1950436 (0.0006) [2023-12-27 19:09:38,889][20347] Updated weights for policy 0, policy_version 1950446 (0.0007) [2023-12-27 19:09:39,158][20381] Updated weights for policy 1, policy_version 1948969 (0.0011) [2023-12-27 19:09:39,210][20381] Updated weights for policy 1, policy_version 1948979 (0.0010) [2023-12-27 19:09:39,272][20381] Updated weights for policy 1, policy_version 1948989 (0.0011) [2023-12-27 19:09:39,329][20381] Updated weights for policy 1, policy_version 1948999 (0.0011) [2023-12-27 19:09:39,482][20347] Updated weights for policy 0, policy_version 1950456 (0.0008) [2023-12-27 19:09:39,545][20347] Updated weights for policy 0, policy_version 1950466 (0.0008) [2023-12-27 19:09:39,610][20347] Updated weights for policy 0, policy_version 1950476 (0.0007) [2023-12-27 19:09:40,067][20381] Updated weights for policy 1, policy_version 1949009 (0.0008) [2023-12-27 19:09:40,121][20381] Updated weights for policy 1, policy_version 1949019 (0.0008) [2023-12-27 19:09:40,186][20381] Updated weights for policy 1, policy_version 1949029 (0.0008) [2023-12-27 19:09:40,323][20347] Updated weights for policy 0, policy_version 1950486 (0.0007) [2023-12-27 19:09:40,388][20347] Updated weights for policy 0, policy_version 1950496 (0.0008) [2023-12-27 19:09:40,456][20347] Updated weights for policy 0, policy_version 1950506 (0.0008) [2023-12-27 19:09:40,849][20381] Updated weights for policy 1, policy_version 1949039 (0.0006) [2023-12-27 19:09:40,900][20381] Updated weights for policy 1, policy_version 1949049 (0.0007) [2023-12-27 19:09:40,960][20381] Updated weights for policy 1, policy_version 1949059 (0.0007) [2023-12-27 19:09:41,127][20347] Updated weights for policy 0, policy_version 1950516 (0.0009) [2023-12-27 19:09:41,197][20347] Updated weights for policy 0, policy_version 1950526 (0.0007) [2023-12-27 19:09:41,255][20347] Updated weights for policy 0, policy_version 1950536 (0.0007) [2023-12-27 19:09:41,699][20381] Updated weights for policy 1, policy_version 1949069 (0.0008) [2023-12-27 19:09:41,759][20381] Updated weights for policy 1, policy_version 1949079 (0.0008) [2023-12-27 19:09:41,814][20381] Updated weights for policy 1, policy_version 1949089 (0.0008) [2023-12-27 19:09:41,940][20347] Updated weights for policy 0, policy_version 1950546 (0.0006) [2023-12-27 19:09:41,996][20347] Updated weights for policy 0, policy_version 1950556 (0.0007) [2023-12-27 19:09:42,050][20347] Updated weights for policy 0, policy_version 1950566 (0.0007) [2023-12-27 19:09:42,110][20347] Updated weights for policy 0, policy_version 1950576 (0.0007) [2023-12-27 19:09:42,363][19170] Fps is (10 sec: 20479.6, 60 sec: 20616.5, 300 sec: 20577.2). Total num frames: 998473728. Throughput: 0: 10308.0, 1: 10238.7. Samples: 998463448. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:42,364][19170] Avg episode reward: [(0, '1000.000'), (1, '883.060')] [2023-12-27 19:09:42,486][20381] Updated weights for policy 1, policy_version 1949099 (0.0009) [2023-12-27 19:09:42,539][20381] Updated weights for policy 1, policy_version 1949109 (0.0008) [2023-12-27 19:09:42,592][20381] Updated weights for policy 1, policy_version 1949119 (0.0007) [2023-12-27 19:09:42,831][20347] Updated weights for policy 0, policy_version 1950586 (0.0010) [2023-12-27 19:09:42,884][20347] Updated weights for policy 0, policy_version 1950596 (0.0010) [2023-12-27 19:09:42,940][20347] Updated weights for policy 0, policy_version 1950606 (0.0010) [2023-12-27 19:09:43,344][20381] Updated weights for policy 1, policy_version 1949129 (0.0007) [2023-12-27 19:09:43,402][20381] Updated weights for policy 1, policy_version 1949139 (0.0008) [2023-12-27 19:09:43,450][20381] Updated weights for policy 1, policy_version 1949149 (0.0008) [2023-12-27 19:09:43,497][20381] Updated weights for policy 1, policy_version 1949159 (0.0010) [2023-12-27 19:09:43,573][20347] Updated weights for policy 0, policy_version 1950616 (0.0006) [2023-12-27 19:09:43,619][20347] Updated weights for policy 0, policy_version 1950626 (0.0005) [2023-12-27 19:09:43,667][20347] Updated weights for policy 0, policy_version 1950636 (0.0008) [2023-12-27 19:09:44,173][20381] Updated weights for policy 1, policy_version 1949169 (0.0008) [2023-12-27 19:09:44,226][20381] Updated weights for policy 1, policy_version 1949179 (0.0007) [2023-12-27 19:09:44,277][20381] Updated weights for policy 1, policy_version 1949189 (0.0007) [2023-12-27 19:09:44,330][20347] Updated weights for policy 0, policy_version 1950646 (0.0010) [2023-12-27 19:09:44,390][20347] Updated weights for policy 0, policy_version 1950656 (0.0010) [2023-12-27 19:09:44,450][20347] Updated weights for policy 0, policy_version 1950666 (0.0010) [2023-12-27 19:09:44,984][20381] Updated weights for policy 1, policy_version 1949199 (0.0008) [2023-12-27 19:09:45,043][20381] Updated weights for policy 1, policy_version 1949209 (0.0008) [2023-12-27 19:09:45,108][20381] Updated weights for policy 1, policy_version 1949219 (0.0007) [2023-12-27 19:09:45,114][20347] Updated weights for policy 0, policy_version 1950676 (0.0010) [2023-12-27 19:09:45,171][20347] Updated weights for policy 0, policy_version 1950686 (0.0008) [2023-12-27 19:09:45,231][20347] Updated weights for policy 0, policy_version 1950696 (0.0011) [2023-12-27 19:09:45,779][20381] Updated weights for policy 1, policy_version 1949229 (0.0006) [2023-12-27 19:09:45,832][20381] Updated weights for policy 1, policy_version 1949239 (0.0005) [2023-12-27 19:09:45,888][20381] Updated weights for policy 1, policy_version 1949249 (0.0005) [2023-12-27 19:09:45,920][20347] Updated weights for policy 0, policy_version 1950706 (0.0010) [2023-12-27 19:09:45,980][20347] Updated weights for policy 0, policy_version 1950716 (0.0009) [2023-12-27 19:09:46,034][20347] Updated weights for policy 0, policy_version 1950726 (0.0010) [2023-12-27 19:09:46,085][20347] Updated weights for policy 0, policy_version 1950736 (0.0010) [2023-12-27 19:09:46,512][20381] Updated weights for policy 1, policy_version 1949259 (0.0005) [2023-12-27 19:09:46,567][20381] Updated weights for policy 1, policy_version 1949269 (0.0008) [2023-12-27 19:09:46,624][20381] Updated weights for policy 1, policy_version 1949279 (0.0006) [2023-12-27 19:09:46,748][20347] Updated weights for policy 0, policy_version 1950746 (0.0007) [2023-12-27 19:09:46,810][20347] Updated weights for policy 0, policy_version 1950756 (0.0008) [2023-12-27 19:09:46,868][20347] Updated weights for policy 0, policy_version 1950766 (0.0007) [2023-12-27 19:09:47,316][20381] Updated weights for policy 1, policy_version 1949289 (0.0007) [2023-12-27 19:09:47,362][20381] Updated weights for policy 1, policy_version 1949299 (0.0008) [2023-12-27 19:09:47,362][19170] Fps is (10 sec: 19661.1, 60 sec: 20480.1, 300 sec: 20577.2). Total num frames: 998572032. Throughput: 0: 10290.2, 1: 10236.2. Samples: 998585012. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.400')] [2023-12-27 19:09:47,412][20381] Updated weights for policy 1, policy_version 1949309 (0.0008) [2023-12-27 19:09:47,461][20381] Updated weights for policy 1, policy_version 1949319 (0.0006) [2023-12-27 19:09:47,538][20347] Updated weights for policy 0, policy_version 1950776 (0.0007) [2023-12-27 19:09:47,595][20347] Updated weights for policy 0, policy_version 1950786 (0.0007) [2023-12-27 19:09:47,657][20347] Updated weights for policy 0, policy_version 1950796 (0.0007) [2023-12-27 19:09:48,161][20381] Updated weights for policy 1, policy_version 1949329 (0.0008) [2023-12-27 19:09:48,217][20381] Updated weights for policy 1, policy_version 1949339 (0.0008) [2023-12-27 19:09:48,280][20381] Updated weights for policy 1, policy_version 1949349 (0.0007) [2023-12-27 19:09:48,286][20347] Updated weights for policy 0, policy_version 1950806 (0.0008) [2023-12-27 19:09:48,343][20347] Updated weights for policy 0, policy_version 1950816 (0.0007) [2023-12-27 19:09:48,407][20347] Updated weights for policy 0, policy_version 1950826 (0.0007) [2023-12-27 19:09:48,966][20381] Updated weights for policy 1, policy_version 1949359 (0.0007) [2023-12-27 19:09:49,019][20381] Updated weights for policy 1, policy_version 1949369 (0.0007) [2023-12-27 19:09:49,071][20381] Updated weights for policy 1, policy_version 1949379 (0.0007) [2023-12-27 19:09:49,148][20347] Updated weights for policy 0, policy_version 1950836 (0.0008) [2023-12-27 19:09:49,199][20347] Updated weights for policy 0, policy_version 1950846 (0.0008) [2023-12-27 19:09:49,265][20347] Updated weights for policy 0, policy_version 1950856 (0.0008) [2023-12-27 19:09:49,708][20381] Updated weights for policy 1, policy_version 1949389 (0.0007) [2023-12-27 19:09:49,758][20381] Updated weights for policy 1, policy_version 1949399 (0.0006) [2023-12-27 19:09:49,806][20381] Updated weights for policy 1, policy_version 1949409 (0.0005) [2023-12-27 19:09:50,010][20347] Updated weights for policy 0, policy_version 1950866 (0.0006) [2023-12-27 19:09:50,071][20347] Updated weights for policy 0, policy_version 1950876 (0.0005) [2023-12-27 19:09:50,134][20347] Updated weights for policy 0, policy_version 1950886 (0.0006) [2023-12-27 19:09:50,194][20347] Updated weights for policy 0, policy_version 1950896 (0.0006) [2023-12-27 19:09:50,568][20381] Updated weights for policy 1, policy_version 1949419 (0.0007) [2023-12-27 19:09:50,637][20381] Updated weights for policy 1, policy_version 1949429 (0.0008) [2023-12-27 19:09:50,700][20381] Updated weights for policy 1, policy_version 1949439 (0.0008) [2023-12-27 19:09:50,813][20347] Updated weights for policy 0, policy_version 1950906 (0.0007) [2023-12-27 19:09:50,878][20347] Updated weights for policy 0, policy_version 1950916 (0.0007) [2023-12-27 19:09:50,939][20347] Updated weights for policy 0, policy_version 1950926 (0.0006) [2023-12-27 19:09:51,409][20381] Updated weights for policy 1, policy_version 1949449 (0.0009) [2023-12-27 19:09:51,461][20381] Updated weights for policy 1, policy_version 1949459 (0.0007) [2023-12-27 19:09:51,509][20381] Updated weights for policy 1, policy_version 1949469 (0.0008) [2023-12-27 19:09:51,568][20381] Updated weights for policy 1, policy_version 1949479 (0.0007) [2023-12-27 19:09:51,658][20347] Updated weights for policy 0, policy_version 1950936 (0.0008) [2023-12-27 19:09:51,715][20347] Updated weights for policy 0, policy_version 1950946 (0.0007) [2023-12-27 19:09:51,778][20347] Updated weights for policy 0, policy_version 1950956 (0.0007) [2023-12-27 19:09:52,259][20381] Updated weights for policy 1, policy_version 1949489 (0.0011) [2023-12-27 19:09:52,319][20381] Updated weights for policy 1, policy_version 1949499 (0.0011) [2023-12-27 19:09:52,362][19170] Fps is (10 sec: 19661.0, 60 sec: 20343.5, 300 sec: 20521.6). Total num frames: 998670336. Throughput: 0: 10234.6, 1: 10201.1. Samples: 998645584. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.690')] [2023-12-27 19:09:52,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001950960_499539968.pth... [2023-12-27 19:09:52,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001949776_499236864.pth [2023-12-27 19:09:52,379][20381] Updated weights for policy 1, policy_version 1949509 (0.0010) [2023-12-27 19:09:52,400][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001949512_499138560.pth... [2023-12-27 19:09:52,405][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001948328_498835456.pth [2023-12-27 19:09:52,470][20347] Updated weights for policy 0, policy_version 1950966 (0.0005) [2023-12-27 19:09:52,521][20347] Updated weights for policy 0, policy_version 1950976 (0.0005) [2023-12-27 19:09:52,577][20347] Updated weights for policy 0, policy_version 1950986 (0.0007) [2023-12-27 19:09:53,039][20381] Updated weights for policy 1, policy_version 1949519 (0.0011) [2023-12-27 19:09:53,088][20381] Updated weights for policy 1, policy_version 1949529 (0.0010) [2023-12-27 19:09:53,138][20347] Updated weights for policy 0, policy_version 1950996 (0.0007) [2023-12-27 19:09:53,143][20381] Updated weights for policy 1, policy_version 1949539 (0.0011) [2023-12-27 19:09:53,206][20347] Updated weights for policy 0, policy_version 1951006 (0.0008) [2023-12-27 19:09:53,270][20347] Updated weights for policy 0, policy_version 1951016 (0.0008) [2023-12-27 19:09:53,823][20381] Updated weights for policy 1, policy_version 1949549 (0.0008) [2023-12-27 19:09:53,880][20381] Updated weights for policy 1, policy_version 1949559 (0.0005) [2023-12-27 19:09:53,942][20381] Updated weights for policy 1, policy_version 1949569 (0.0006) [2023-12-27 19:09:53,955][20347] Updated weights for policy 0, policy_version 1951026 (0.0008) [2023-12-27 19:09:54,011][20347] Updated weights for policy 0, policy_version 1951036 (0.0009) [2023-12-27 19:09:54,059][20347] Updated weights for policy 0, policy_version 1951046 (0.0008) [2023-12-27 19:09:54,107][20347] Updated weights for policy 0, policy_version 1951056 (0.0007) [2023-12-27 19:09:54,487][20381] Updated weights for policy 1, policy_version 1949579 (0.0007) [2023-12-27 19:09:54,546][20381] Updated weights for policy 1, policy_version 1949589 (0.0007) [2023-12-27 19:09:54,607][20381] Updated weights for policy 1, policy_version 1949599 (0.0007) [2023-12-27 19:09:54,821][20347] Updated weights for policy 0, policy_version 1951066 (0.0005) [2023-12-27 19:09:54,866][20347] Updated weights for policy 0, policy_version 1951076 (0.0005) [2023-12-27 19:09:54,910][20347] Updated weights for policy 0, policy_version 1951086 (0.0005) [2023-12-27 19:09:55,296][20381] Updated weights for policy 1, policy_version 1949609 (0.0008) [2023-12-27 19:09:55,344][20381] Updated weights for policy 1, policy_version 1949619 (0.0010) [2023-12-27 19:09:55,398][20381] Updated weights for policy 1, policy_version 1949629 (0.0010) [2023-12-27 19:09:55,447][20381] Updated weights for policy 1, policy_version 1949639 (0.0010) [2023-12-27 19:09:55,583][20347] Updated weights for policy 0, policy_version 1951096 (0.0007) [2023-12-27 19:09:55,641][20347] Updated weights for policy 0, policy_version 1951106 (0.0007) [2023-12-27 19:09:55,699][20347] Updated weights for policy 0, policy_version 1951116 (0.0007) [2023-12-27 19:09:56,116][20381] Updated weights for policy 1, policy_version 1949649 (0.0006) [2023-12-27 19:09:56,175][20381] Updated weights for policy 1, policy_version 1949659 (0.0006) [2023-12-27 19:09:56,233][20381] Updated weights for policy 1, policy_version 1949669 (0.0005) [2023-12-27 19:09:56,343][20347] Updated weights for policy 0, policy_version 1951126 (0.0007) [2023-12-27 19:09:56,395][20347] Updated weights for policy 0, policy_version 1951136 (0.0006) [2023-12-27 19:09:56,446][20347] Updated weights for policy 0, policy_version 1951146 (0.0007) [2023-12-27 19:09:56,808][20381] Updated weights for policy 1, policy_version 1949679 (0.0009) [2023-12-27 19:09:56,859][20381] Updated weights for policy 1, policy_version 1949689 (0.0010) [2023-12-27 19:09:56,914][20381] Updated weights for policy 1, policy_version 1949699 (0.0010) [2023-12-27 19:09:57,145][20347] Updated weights for policy 0, policy_version 1951156 (0.0008) [2023-12-27 19:09:57,190][20347] Updated weights for policy 0, policy_version 1951166 (0.0007) [2023-12-27 19:09:57,243][20347] Updated weights for policy 0, policy_version 1951176 (0.0005) [2023-12-27 19:09:57,363][19170] Fps is (10 sec: 21298.8, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 998785024. Throughput: 0: 10216.2, 1: 10229.4. Samples: 998769972. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:09:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.790')] [2023-12-27 19:09:57,545][20381] Updated weights for policy 1, policy_version 1949709 (0.0010) [2023-12-27 19:09:57,604][20381] Updated weights for policy 1, policy_version 1949719 (0.0010) [2023-12-27 19:09:57,662][20381] Updated weights for policy 1, policy_version 1949729 (0.0009) [2023-12-27 19:09:57,822][20347] Updated weights for policy 0, policy_version 1951186 (0.0005) [2023-12-27 19:09:57,878][20347] Updated weights for policy 0, policy_version 1951196 (0.0007) [2023-12-27 19:09:57,930][20347] Updated weights for policy 0, policy_version 1951206 (0.0010) [2023-12-27 19:09:57,984][20347] Updated weights for policy 0, policy_version 1951216 (0.0010) [2023-12-27 19:09:58,343][20381] Updated weights for policy 1, policy_version 1949739 (0.0007) [2023-12-27 19:09:58,416][20381] Updated weights for policy 1, policy_version 1949749 (0.0008) [2023-12-27 19:09:58,478][20381] Updated weights for policy 1, policy_version 1949759 (0.0008) [2023-12-27 19:09:58,687][20347] Updated weights for policy 0, policy_version 1951226 (0.0008) [2023-12-27 19:09:58,751][20347] Updated weights for policy 0, policy_version 1951236 (0.0007) [2023-12-27 19:09:58,813][20347] Updated weights for policy 0, policy_version 1951246 (0.0008) [2023-12-27 19:09:59,326][20381] Updated weights for policy 1, policy_version 1949769 (0.0010) [2023-12-27 19:09:59,394][20381] Updated weights for policy 1, policy_version 1949779 (0.0007) [2023-12-27 19:09:59,455][20381] Updated weights for policy 1, policy_version 1949789 (0.0008) [2023-12-27 19:09:59,512][20381] Updated weights for policy 1, policy_version 1949799 (0.0008) [2023-12-27 19:09:59,522][20347] Updated weights for policy 0, policy_version 1951256 (0.0008) [2023-12-27 19:09:59,579][20347] Updated weights for policy 0, policy_version 1951266 (0.0007) [2023-12-27 19:09:59,640][20347] Updated weights for policy 0, policy_version 1951276 (0.0008) [2023-12-27 19:10:00,152][20381] Updated weights for policy 1, policy_version 1949809 (0.0008) [2023-12-27 19:10:00,205][20381] Updated weights for policy 1, policy_version 1949819 (0.0009) [2023-12-27 19:10:00,261][20381] Updated weights for policy 1, policy_version 1949829 (0.0009) [2023-12-27 19:10:00,316][20347] Updated weights for policy 0, policy_version 1951286 (0.0007) [2023-12-27 19:10:00,379][20347] Updated weights for policy 0, policy_version 1951296 (0.0006) [2023-12-27 19:10:00,428][20347] Updated weights for policy 0, policy_version 1951306 (0.0007) [2023-12-27 19:10:01,010][20381] Updated weights for policy 1, policy_version 1949839 (0.0008) [2023-12-27 19:10:01,074][20381] Updated weights for policy 1, policy_version 1949849 (0.0009) [2023-12-27 19:10:01,103][20347] Updated weights for policy 0, policy_version 1951316 (0.0007) [2023-12-27 19:10:01,138][20381] Updated weights for policy 1, policy_version 1949859 (0.0008) [2023-12-27 19:10:01,169][20347] Updated weights for policy 0, policy_version 1951326 (0.0007) [2023-12-27 19:10:01,230][20347] Updated weights for policy 0, policy_version 1951336 (0.0007) [2023-12-27 19:10:01,844][20381] Updated weights for policy 1, policy_version 1949869 (0.0007) [2023-12-27 19:10:01,912][20381] Updated weights for policy 1, policy_version 1949879 (0.0006) [2023-12-27 19:10:01,951][20347] Updated weights for policy 0, policy_version 1951346 (0.0009) [2023-12-27 19:10:01,966][20381] Updated weights for policy 1, policy_version 1949889 (0.0006) [2023-12-27 19:10:02,002][20347] Updated weights for policy 0, policy_version 1951356 (0.0011) [2023-12-27 19:10:02,051][20347] Updated weights for policy 0, policy_version 1951366 (0.0009) [2023-12-27 19:10:02,105][20347] Updated weights for policy 0, policy_version 1951376 (0.0007) [2023-12-27 19:10:02,362][19170] Fps is (10 sec: 21299.2, 60 sec: 20616.5, 300 sec: 20549.4). Total num frames: 998883328. Throughput: 0: 10231.9, 1: 10133.6. Samples: 998891468. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.790')] [2023-12-27 19:10:02,660][20381] Updated weights for policy 1, policy_version 1949899 (0.0006) [2023-12-27 19:10:02,722][20381] Updated weights for policy 1, policy_version 1949909 (0.0007) [2023-12-27 19:10:02,777][20381] Updated weights for policy 1, policy_version 1949919 (0.0006) [2023-12-27 19:10:02,805][20347] Updated weights for policy 0, policy_version 1951386 (0.0007) [2023-12-27 19:10:02,861][20347] Updated weights for policy 0, policy_version 1951396 (0.0006) [2023-12-27 19:10:02,912][20347] Updated weights for policy 0, policy_version 1951406 (0.0008) [2023-12-27 19:10:03,422][20381] Updated weights for policy 1, policy_version 1949929 (0.0005) [2023-12-27 19:10:03,478][20381] Updated weights for policy 1, policy_version 1949939 (0.0005) [2023-12-27 19:10:03,538][20381] Updated weights for policy 1, policy_version 1949949 (0.0005) [2023-12-27 19:10:03,580][20347] Updated weights for policy 0, policy_version 1951416 (0.0008) [2023-12-27 19:10:03,596][20381] Updated weights for policy 1, policy_version 1949959 (0.0005) [2023-12-27 19:10:03,627][20347] Updated weights for policy 0, policy_version 1951426 (0.0007) [2023-12-27 19:10:03,683][20347] Updated weights for policy 0, policy_version 1951436 (0.0005) [2023-12-27 19:10:04,110][20381] Updated weights for policy 1, policy_version 1949969 (0.0008) [2023-12-27 19:10:04,176][20381] Updated weights for policy 1, policy_version 1949979 (0.0008) [2023-12-27 19:10:04,237][20381] Updated weights for policy 1, policy_version 1949989 (0.0008) [2023-12-27 19:10:04,383][20347] Updated weights for policy 0, policy_version 1951446 (0.0007) [2023-12-27 19:10:04,446][20347] Updated weights for policy 0, policy_version 1951456 (0.0008) [2023-12-27 19:10:04,498][20347] Updated weights for policy 0, policy_version 1951466 (0.0008) [2023-12-27 19:10:04,913][20381] Updated weights for policy 1, policy_version 1949999 (0.0010) [2023-12-27 19:10:04,962][20381] Updated weights for policy 1, policy_version 1950009 (0.0010) [2023-12-27 19:10:05,011][20381] Updated weights for policy 1, policy_version 1950019 (0.0010) [2023-12-27 19:10:05,155][20347] Updated weights for policy 0, policy_version 1951476 (0.0007) [2023-12-27 19:10:05,207][20347] Updated weights for policy 0, policy_version 1951486 (0.0008) [2023-12-27 19:10:05,260][20347] Updated weights for policy 0, policy_version 1951496 (0.0007) [2023-12-27 19:10:05,679][20381] Updated weights for policy 1, policy_version 1950029 (0.0010) [2023-12-27 19:10:05,728][20381] Updated weights for policy 1, policy_version 1950039 (0.0010) [2023-12-27 19:10:05,776][20381] Updated weights for policy 1, policy_version 1950049 (0.0010) [2023-12-27 19:10:05,860][20347] Updated weights for policy 0, policy_version 1951506 (0.0007) [2023-12-27 19:10:05,917][20347] Updated weights for policy 0, policy_version 1951516 (0.0007) [2023-12-27 19:10:05,973][20347] Updated weights for policy 0, policy_version 1951526 (0.0007) [2023-12-27 19:10:06,030][20347] Updated weights for policy 0, policy_version 1951536 (0.0007) [2023-12-27 19:10:06,501][20381] Updated weights for policy 1, policy_version 1950059 (0.0010) [2023-12-27 19:10:06,555][20381] Updated weights for policy 1, policy_version 1950069 (0.0010) [2023-12-27 19:10:06,605][20381] Updated weights for policy 1, policy_version 1950079 (0.0010) [2023-12-27 19:10:06,659][20347] Updated weights for policy 0, policy_version 1951546 (0.0005) [2023-12-27 19:10:06,717][20347] Updated weights for policy 0, policy_version 1951556 (0.0008) [2023-12-27 19:10:06,771][20347] Updated weights for policy 0, policy_version 1951566 (0.0006) [2023-12-27 19:10:07,309][20381] Updated weights for policy 1, policy_version 1950089 (0.0011) [2023-12-27 19:10:07,350][20347] Updated weights for policy 0, policy_version 1951576 (0.0006) [2023-12-27 19:10:07,362][19170] Fps is (10 sec: 19660.9, 60 sec: 20343.5, 300 sec: 20577.2). Total num frames: 998981632. Throughput: 0: 10243.1, 1: 10172.9. Samples: 998954932. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.560')] [2023-12-27 19:10:07,377][20381] Updated weights for policy 1, policy_version 1950099 (0.0011) [2023-12-27 19:10:07,399][20347] Updated weights for policy 0, policy_version 1951586 (0.0006) [2023-12-27 19:10:07,430][20381] Updated weights for policy 1, policy_version 1950109 (0.0008) [2023-12-27 19:10:07,447][20347] Updated weights for policy 0, policy_version 1951596 (0.0005) [2023-12-27 19:10:07,471][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001951600_499703808.pth... [2023-12-27 19:10:07,475][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001950384_499392512.pth [2023-12-27 19:10:07,479][20381] Updated weights for policy 1, policy_version 1950119 (0.0009) [2023-12-27 19:10:07,482][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001950120_499294208.pth... [2023-12-27 19:10:07,485][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001948904_498982912.pth [2023-12-27 19:10:08,080][20347] Updated weights for policy 0, policy_version 1951606 (0.0005) [2023-12-27 19:10:08,141][20347] Updated weights for policy 0, policy_version 1951616 (0.0005) [2023-12-27 19:10:08,157][20381] Updated weights for policy 1, policy_version 1950129 (0.0010) [2023-12-27 19:10:08,201][20347] Updated weights for policy 0, policy_version 1951626 (0.0006) [2023-12-27 19:10:08,221][20381] Updated weights for policy 1, policy_version 1950139 (0.0005) [2023-12-27 19:10:08,267][20381] Updated weights for policy 1, policy_version 1950149 (0.0009) [2023-12-27 19:10:08,812][20347] Updated weights for policy 0, policy_version 1951636 (0.0008) [2023-12-27 19:10:08,874][20347] Updated weights for policy 0, policy_version 1951646 (0.0011) [2023-12-27 19:10:08,896][20381] Updated weights for policy 1, policy_version 1950159 (0.0007) [2023-12-27 19:10:08,935][20347] Updated weights for policy 0, policy_version 1951656 (0.0010) [2023-12-27 19:10:08,958][20381] Updated weights for policy 1, policy_version 1950169 (0.0006) [2023-12-27 19:10:09,014][20381] Updated weights for policy 1, policy_version 1950179 (0.0008) [2023-12-27 19:10:09,571][20347] Updated weights for policy 0, policy_version 1951666 (0.0009) [2023-12-27 19:10:09,631][20347] Updated weights for policy 0, policy_version 1951676 (0.0008) [2023-12-27 19:10:09,688][20347] Updated weights for policy 0, policy_version 1951686 (0.0008) [2023-12-27 19:10:09,709][20381] Updated weights for policy 1, policy_version 1950189 (0.0007) [2023-12-27 19:10:09,749][20347] Updated weights for policy 0, policy_version 1951696 (0.0006) [2023-12-27 19:10:09,766][20381] Updated weights for policy 1, policy_version 1950199 (0.0007) [2023-12-27 19:10:09,833][20381] Updated weights for policy 1, policy_version 1950209 (0.0007) [2023-12-27 19:10:10,442][20347] Updated weights for policy 0, policy_version 1951706 (0.0011) [2023-12-27 19:10:10,497][20347] Updated weights for policy 0, policy_version 1951716 (0.0010) [2023-12-27 19:10:10,561][20347] Updated weights for policy 0, policy_version 1951726 (0.0010) [2023-12-27 19:10:10,581][20381] Updated weights for policy 1, policy_version 1950219 (0.0008) [2023-12-27 19:10:10,642][20381] Updated weights for policy 1, policy_version 1950229 (0.0010) [2023-12-27 19:10:10,700][20381] Updated weights for policy 1, policy_version 1950239 (0.0010) [2023-12-27 19:10:11,273][20347] Updated weights for policy 0, policy_version 1951736 (0.0010) [2023-12-27 19:10:11,341][20347] Updated weights for policy 0, policy_version 1951746 (0.0007) [2023-12-27 19:10:11,376][20381] Updated weights for policy 1, policy_version 1950249 (0.0010) [2023-12-27 19:10:11,418][20347] Updated weights for policy 0, policy_version 1951756 (0.0008) [2023-12-27 19:10:11,436][20381] Updated weights for policy 1, policy_version 1950259 (0.0007) [2023-12-27 19:10:11,491][20381] Updated weights for policy 1, policy_version 1950269 (0.0008) [2023-12-27 19:10:11,545][20381] Updated weights for policy 1, policy_version 1950279 (0.0007) [2023-12-27 19:10:12,135][20347] Updated weights for policy 0, policy_version 1951766 (0.0007) [2023-12-27 19:10:12,193][20347] Updated weights for policy 0, policy_version 1951776 (0.0007) [2023-12-27 19:10:12,240][20381] Updated weights for policy 1, policy_version 1950289 (0.0011) [2023-12-27 19:10:12,249][20347] Updated weights for policy 0, policy_version 1951786 (0.0007) [2023-12-27 19:10:12,300][20381] Updated weights for policy 1, policy_version 1950299 (0.0011) [2023-12-27 19:10:12,362][19170] Fps is (10 sec: 20480.2, 60 sec: 20480.0, 300 sec: 20549.4). Total num frames: 999088128. Throughput: 0: 10371.9, 1: 10171.4. Samples: 999079720. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:12,363][19170] Avg episode reward: [(0, '1000.000'), (1, '863.560')] [2023-12-27 19:10:12,365][20381] Updated weights for policy 1, policy_version 1950309 (0.0010) [2023-12-27 19:10:12,964][20347] Updated weights for policy 0, policy_version 1951796 (0.0008) [2023-12-27 19:10:13,012][20347] Updated weights for policy 0, policy_version 1951806 (0.0010) [2023-12-27 19:10:13,059][20347] Updated weights for policy 0, policy_version 1951816 (0.0010) [2023-12-27 19:10:13,082][20381] Updated weights for policy 1, policy_version 1950319 (0.0007) [2023-12-27 19:10:13,127][20381] Updated weights for policy 1, policy_version 1950329 (0.0005) [2023-12-27 19:10:13,173][20381] Updated weights for policy 1, policy_version 1950339 (0.0006) [2023-12-27 19:10:13,733][20381] Updated weights for policy 1, policy_version 1950349 (0.0005) [2023-12-27 19:10:13,745][20347] Updated weights for policy 0, policy_version 1951826 (0.0008) [2023-12-27 19:10:13,782][20381] Updated weights for policy 1, policy_version 1950359 (0.0005) [2023-12-27 19:10:13,797][20347] Updated weights for policy 0, policy_version 1951836 (0.0007) [2023-12-27 19:10:13,831][20381] Updated weights for policy 1, policy_version 1950369 (0.0006) [2023-12-27 19:10:13,851][20347] Updated weights for policy 0, policy_version 1951846 (0.0007) [2023-12-27 19:10:13,920][20347] Updated weights for policy 0, policy_version 1951856 (0.0007) [2023-12-27 19:10:14,464][20381] Updated weights for policy 1, policy_version 1950379 (0.0007) [2023-12-27 19:10:14,526][20381] Updated weights for policy 1, policy_version 1950389 (0.0010) [2023-12-27 19:10:14,581][20381] Updated weights for policy 1, policy_version 1950399 (0.0010) [2023-12-27 19:10:14,613][20347] Updated weights for policy 0, policy_version 1951866 (0.0010) [2023-12-27 19:10:14,665][20347] Updated weights for policy 0, policy_version 1951876 (0.0010) [2023-12-27 19:10:14,709][20347] Updated weights for policy 0, policy_version 1951886 (0.0010) [2023-12-27 19:10:15,281][20381] Updated weights for policy 1, policy_version 1950409 (0.0010) [2023-12-27 19:10:15,338][20381] Updated weights for policy 1, policy_version 1950419 (0.0011) [2023-12-27 19:10:15,354][20347] Updated weights for policy 0, policy_version 1951896 (0.0006) [2023-12-27 19:10:15,398][20381] Updated weights for policy 1, policy_version 1950429 (0.0011) [2023-12-27 19:10:15,415][20347] Updated weights for policy 0, policy_version 1951906 (0.0006) [2023-12-27 19:10:15,448][20381] Updated weights for policy 1, policy_version 1950439 (0.0010) [2023-12-27 19:10:15,466][20347] Updated weights for policy 0, policy_version 1951916 (0.0010) [2023-12-27 19:10:16,081][20381] Updated weights for policy 1, policy_version 1950449 (0.0006) [2023-12-27 19:10:16,113][20347] Updated weights for policy 0, policy_version 1951926 (0.0009) [2023-12-27 19:10:16,128][20381] Updated weights for policy 1, policy_version 1950459 (0.0005) [2023-12-27 19:10:16,176][20381] Updated weights for policy 1, policy_version 1950469 (0.0005) [2023-12-27 19:10:16,176][20347] Updated weights for policy 0, policy_version 1951936 (0.0008) [2023-12-27 19:10:16,239][20347] Updated weights for policy 0, policy_version 1951946 (0.0008) [2023-12-27 19:10:16,764][20381] Updated weights for policy 1, policy_version 1950479 (0.0009) [2023-12-27 19:10:16,813][20381] Updated weights for policy 1, policy_version 1950489 (0.0010) [2023-12-27 19:10:16,861][20381] Updated weights for policy 1, policy_version 1950499 (0.0010) [2023-12-27 19:10:16,906][20347] Updated weights for policy 0, policy_version 1951956 (0.0009) [2023-12-27 19:10:16,966][20347] Updated weights for policy 0, policy_version 1951966 (0.0008) [2023-12-27 19:10:17,025][20347] Updated weights for policy 0, policy_version 1951976 (0.0008) [2023-12-27 19:10:17,362][19170] Fps is (10 sec: 21299.4, 60 sec: 20480.0, 300 sec: 20577.3). Total num frames: 999194624. Throughput: 0: 10322.0, 1: 10236.0. Samples: 999204564. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:17,363][19170] Avg episode reward: [(0, '1000.000'), (1, '921.830')] [2023-12-27 19:10:17,471][20381] Updated weights for policy 1, policy_version 1950509 (0.0008) [2023-12-27 19:10:17,522][20381] Updated weights for policy 1, policy_version 1950519 (0.0005) [2023-12-27 19:10:17,577][20381] Updated weights for policy 1, policy_version 1950529 (0.0008) [2023-12-27 19:10:17,707][20347] Updated weights for policy 0, policy_version 1951986 (0.0008) [2023-12-27 19:10:17,756][20347] Updated weights for policy 0, policy_version 1951996 (0.0007) [2023-12-27 19:10:17,816][20347] Updated weights for policy 0, policy_version 1952006 (0.0005) [2023-12-27 19:10:17,877][20347] Updated weights for policy 0, policy_version 1952016 (0.0006) [2023-12-27 19:10:18,212][20381] Updated weights for policy 1, policy_version 1950539 (0.0007) [2023-12-27 19:10:18,267][20381] Updated weights for policy 1, policy_version 1950549 (0.0010) [2023-12-27 19:10:18,326][20381] Updated weights for policy 1, policy_version 1950559 (0.0011) [2023-12-27 19:10:18,474][20347] Updated weights for policy 0, policy_version 1952026 (0.0008) [2023-12-27 19:10:18,530][20347] Updated weights for policy 0, policy_version 1952036 (0.0008) [2023-12-27 19:10:18,583][20347] Updated weights for policy 0, policy_version 1952046 (0.0007) [2023-12-27 19:10:19,008][20381] Updated weights for policy 1, policy_version 1950569 (0.0008) [2023-12-27 19:10:19,056][20381] Updated weights for policy 1, policy_version 1950579 (0.0007) [2023-12-27 19:10:19,110][20381] Updated weights for policy 1, policy_version 1950589 (0.0008) [2023-12-27 19:10:19,168][20381] Updated weights for policy 1, policy_version 1950599 (0.0009) [2023-12-27 19:10:19,233][20347] Updated weights for policy 0, policy_version 1952056 (0.0008) [2023-12-27 19:10:19,303][20347] Updated weights for policy 0, policy_version 1952066 (0.0006) [2023-12-27 19:10:19,375][20347] Updated weights for policy 0, policy_version 1952076 (0.0008) [2023-12-27 19:10:19,851][20381] Updated weights for policy 1, policy_version 1950609 (0.0007) [2023-12-27 19:10:19,915][20381] Updated weights for policy 1, policy_version 1950619 (0.0008) [2023-12-27 19:10:19,982][20381] Updated weights for policy 1, policy_version 1950629 (0.0010) [2023-12-27 19:10:20,036][20347] Updated weights for policy 0, policy_version 1952086 (0.0008) [2023-12-27 19:10:20,090][20347] Updated weights for policy 0, policy_version 1952096 (0.0007) [2023-12-27 19:10:20,140][20347] Updated weights for policy 0, policy_version 1952106 (0.0008) [2023-12-27 19:10:20,638][20381] Updated weights for policy 1, policy_version 1950639 (0.0009) [2023-12-27 19:10:20,698][20381] Updated weights for policy 1, policy_version 1950649 (0.0008) [2023-12-27 19:10:20,760][20381] Updated weights for policy 1, policy_version 1950659 (0.0008) [2023-12-27 19:10:20,856][20347] Updated weights for policy 0, policy_version 1952116 (0.0009) [2023-12-27 19:10:20,916][20347] Updated weights for policy 0, policy_version 1952126 (0.0010) [2023-12-27 19:10:20,974][20347] Updated weights for policy 0, policy_version 1952136 (0.0009) [2023-12-27 19:10:21,473][20381] Updated weights for policy 1, policy_version 1950669 (0.0009) [2023-12-27 19:10:21,536][20381] Updated weights for policy 1, policy_version 1950679 (0.0011) [2023-12-27 19:10:21,595][20381] Updated weights for policy 1, policy_version 1950689 (0.0007) [2023-12-27 19:10:21,727][20347] Updated weights for policy 0, policy_version 1952146 (0.0008) [2023-12-27 19:10:21,792][20347] Updated weights for policy 0, policy_version 1952156 (0.0008) [2023-12-27 19:10:21,851][20347] Updated weights for policy 0, policy_version 1952166 (0.0009) [2023-12-27 19:10:21,903][20347] Updated weights for policy 0, policy_version 1952176 (0.0009) [2023-12-27 19:10:22,364][19170] Fps is (10 sec: 20476.3, 60 sec: 20342.9, 300 sec: 20577.1). Total num frames: 999292928. Throughput: 0: 10349.6, 1: 10287.1. Samples: 999268572. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:22,365][19170] Avg episode reward: [(0, '1000.000'), (1, '882.690')] [2023-12-27 19:10:22,370][20381] Updated weights for policy 1, policy_version 1950699 (0.0008) [2023-12-27 19:10:22,371][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001952176_499851264.pth... [2023-12-27 19:10:22,375][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001950960_499539968.pth [2023-12-27 19:10:22,426][20381] Updated weights for policy 1, policy_version 1950709 (0.0007) [2023-12-27 19:10:22,476][20381] Updated weights for policy 1, policy_version 1950719 (0.0007) [2023-12-27 19:10:22,528][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001950728_499449856.pth... [2023-12-27 19:10:22,531][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001949512_499138560.pth [2023-12-27 19:10:22,621][20347] Updated weights for policy 0, policy_version 1952186 (0.0009) [2023-12-27 19:10:22,682][20347] Updated weights for policy 0, policy_version 1952196 (0.0009) [2023-12-27 19:10:22,733][20347] Updated weights for policy 0, policy_version 1952206 (0.0009) [2023-12-27 19:10:23,219][20381] Updated weights for policy 1, policy_version 1950729 (0.0008) [2023-12-27 19:10:23,268][20381] Updated weights for policy 1, policy_version 1950739 (0.0009) [2023-12-27 19:10:23,325][20381] Updated weights for policy 1, policy_version 1950749 (0.0008) [2023-12-27 19:10:23,378][20381] Updated weights for policy 1, policy_version 1950759 (0.0007) [2023-12-27 19:10:23,384][20347] Updated weights for policy 0, policy_version 1952216 (0.0006) [2023-12-27 19:10:23,443][20347] Updated weights for policy 0, policy_version 1952226 (0.0008) [2023-12-27 19:10:23,501][20347] Updated weights for policy 0, policy_version 1952236 (0.0007) [2023-12-27 19:10:24,044][20381] Updated weights for policy 1, policy_version 1950769 (0.0005) [2023-12-27 19:10:24,091][20381] Updated weights for policy 1, policy_version 1950779 (0.0007) [2023-12-27 19:10:24,106][20347] Updated weights for policy 0, policy_version 1952246 (0.0007) [2023-12-27 19:10:24,137][20381] Updated weights for policy 1, policy_version 1950789 (0.0007) [2023-12-27 19:10:24,158][20347] Updated weights for policy 0, policy_version 1952256 (0.0005) [2023-12-27 19:10:24,221][20347] Updated weights for policy 0, policy_version 1952266 (0.0008) [2023-12-27 19:10:24,758][20381] Updated weights for policy 1, policy_version 1950799 (0.0007) [2023-12-27 19:10:24,789][20347] Updated weights for policy 0, policy_version 1952276 (0.0009) [2023-12-27 19:10:24,811][20381] Updated weights for policy 1, policy_version 1950809 (0.0010) [2023-12-27 19:10:24,839][20347] Updated weights for policy 0, policy_version 1952286 (0.0010) [2023-12-27 19:10:24,863][20381] Updated weights for policy 1, policy_version 1950819 (0.0008) [2023-12-27 19:10:24,892][20347] Updated weights for policy 0, policy_version 1952296 (0.0010) [2023-12-27 19:10:25,442][20381] Updated weights for policy 1, policy_version 1950829 (0.0006) [2023-12-27 19:10:25,495][20381] Updated weights for policy 1, policy_version 1950839 (0.0005) [2023-12-27 19:10:25,551][20381] Updated weights for policy 1, policy_version 1950849 (0.0005) [2023-12-27 19:10:25,563][20347] Updated weights for policy 0, policy_version 1952306 (0.0010) [2023-12-27 19:10:25,615][20347] Updated weights for policy 0, policy_version 1952316 (0.0010) [2023-12-27 19:10:25,664][20347] Updated weights for policy 0, policy_version 1952326 (0.0010) [2023-12-27 19:10:25,712][20347] Updated weights for policy 0, policy_version 1952336 (0.0010) [2023-12-27 19:10:26,127][20381] Updated weights for policy 1, policy_version 1950859 (0.0006) [2023-12-27 19:10:26,187][20381] Updated weights for policy 1, policy_version 1950869 (0.0011) [2023-12-27 19:10:26,237][20381] Updated weights for policy 1, policy_version 1950879 (0.0010) [2023-12-27 19:10:26,410][20347] Updated weights for policy 0, policy_version 1952346 (0.0009) [2023-12-27 19:10:26,471][20347] Updated weights for policy 0, policy_version 1952356 (0.0010) [2023-12-27 19:10:26,532][20347] Updated weights for policy 0, policy_version 1952366 (0.0008) [2023-12-27 19:10:26,891][20381] Updated weights for policy 1, policy_version 1950889 (0.0010) [2023-12-27 19:10:26,947][20381] Updated weights for policy 1, policy_version 1950899 (0.0007) [2023-12-27 19:10:27,000][20381] Updated weights for policy 1, policy_version 1950909 (0.0007) [2023-12-27 19:10:27,054][20381] Updated weights for policy 1, policy_version 1950919 (0.0007) [2023-12-27 19:10:27,171][20347] Updated weights for policy 0, policy_version 1952376 (0.0010) [2023-12-27 19:10:27,223][20347] Updated weights for policy 0, policy_version 1952386 (0.0010) [2023-12-27 19:10:27,274][20347] Updated weights for policy 0, policy_version 1952396 (0.0010) [2023-12-27 19:10:27,362][19170] Fps is (10 sec: 21299.3, 60 sec: 20616.5, 300 sec: 20632.7). Total num frames: 999407616. Throughput: 0: 10324.6, 1: 10329.2. Samples: 999392864. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:27,363][19170] Avg episode reward: [(0, '1000.000'), (1, '892.570')] [2023-12-27 19:10:27,733][20381] Updated weights for policy 1, policy_version 1950929 (0.0007) [2023-12-27 19:10:27,788][20381] Updated weights for policy 1, policy_version 1950939 (0.0007) [2023-12-27 19:10:27,839][20381] Updated weights for policy 1, policy_version 1950949 (0.0007) [2023-12-27 19:10:27,894][20347] Updated weights for policy 0, policy_version 1952406 (0.0010) [2023-12-27 19:10:27,952][20347] Updated weights for policy 0, policy_version 1952416 (0.0007) [2023-12-27 19:10:28,002][20347] Updated weights for policy 0, policy_version 1952426 (0.0009) [2023-12-27 19:10:28,537][20381] Updated weights for policy 1, policy_version 1950959 (0.0006) [2023-12-27 19:10:28,593][20381] Updated weights for policy 1, policy_version 1950969 (0.0005) [2023-12-27 19:10:28,647][20381] Updated weights for policy 1, policy_version 1950979 (0.0005) [2023-12-27 19:10:28,682][20347] Updated weights for policy 0, policy_version 1952436 (0.0010) [2023-12-27 19:10:28,744][20347] Updated weights for policy 0, policy_version 1952446 (0.0010) [2023-12-27 19:10:28,795][20347] Updated weights for policy 0, policy_version 1952456 (0.0010) [2023-12-27 19:10:29,292][20381] Updated weights for policy 1, policy_version 1950989 (0.0007) [2023-12-27 19:10:29,354][20381] Updated weights for policy 1, policy_version 1950999 (0.0009) [2023-12-27 19:10:29,419][20381] Updated weights for policy 1, policy_version 1951009 (0.0009) [2023-12-27 19:10:29,483][20347] Updated weights for policy 0, policy_version 1952466 (0.0009) [2023-12-27 19:10:29,542][20347] Updated weights for policy 0, policy_version 1952476 (0.0008) [2023-12-27 19:10:29,597][20347] Updated weights for policy 0, policy_version 1952486 (0.0008) [2023-12-27 19:10:29,661][20347] Updated weights for policy 0, policy_version 1952496 (0.0007) [2023-12-27 19:10:30,093][20381] Updated weights for policy 1, policy_version 1951019 (0.0009) [2023-12-27 19:10:30,146][20381] Updated weights for policy 1, policy_version 1951029 (0.0007) [2023-12-27 19:10:30,203][20381] Updated weights for policy 1, policy_version 1951039 (0.0007) [2023-12-27 19:10:30,341][20347] Updated weights for policy 0, policy_version 1952506 (0.0007) [2023-12-27 19:10:30,392][20347] Updated weights for policy 0, policy_version 1952516 (0.0007) [2023-12-27 19:10:30,442][20347] Updated weights for policy 0, policy_version 1952526 (0.0007) [2023-12-27 19:10:30,896][20381] Updated weights for policy 1, policy_version 1951049 (0.0008) [2023-12-27 19:10:30,945][20381] Updated weights for policy 1, policy_version 1951059 (0.0008) [2023-12-27 19:10:30,993][20381] Updated weights for policy 1, policy_version 1951069 (0.0008) [2023-12-27 19:10:31,050][20381] Updated weights for policy 1, policy_version 1951079 (0.0008) [2023-12-27 19:10:31,085][20347] Updated weights for policy 0, policy_version 1952536 (0.0008) [2023-12-27 19:10:31,142][20347] Updated weights for policy 0, policy_version 1952546 (0.0008) [2023-12-27 19:10:31,202][20347] Updated weights for policy 0, policy_version 1952556 (0.0008) [2023-12-27 19:10:31,810][20381] Updated weights for policy 1, policy_version 1951089 (0.0005) [2023-12-27 19:10:31,867][20381] Updated weights for policy 1, policy_version 1951099 (0.0005) [2023-12-27 19:10:31,911][20347] Updated weights for policy 0, policy_version 1952566 (0.0008) [2023-12-27 19:10:31,925][20381] Updated weights for policy 1, policy_version 1951109 (0.0006) [2023-12-27 19:10:31,969][20347] Updated weights for policy 0, policy_version 1952576 (0.0008) [2023-12-27 19:10:32,027][20347] Updated weights for policy 0, policy_version 1952586 (0.0008) [2023-12-27 19:10:32,362][19170] Fps is (10 sec: 21302.9, 60 sec: 20616.5, 300 sec: 20605.0). Total num frames: 999505920. Throughput: 0: 10342.8, 1: 10349.9. Samples: 999516188. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:32,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.460')] [2023-12-27 19:10:32,497][20381] Updated weights for policy 1, policy_version 1951119 (0.0006) [2023-12-27 19:10:32,547][20381] Updated weights for policy 1, policy_version 1951129 (0.0005) [2023-12-27 19:10:32,603][20381] Updated weights for policy 1, policy_version 1951139 (0.0005) [2023-12-27 19:10:32,689][20347] Updated weights for policy 0, policy_version 1952596 (0.0005) [2023-12-27 19:10:32,743][20347] Updated weights for policy 0, policy_version 1952606 (0.0005) [2023-12-27 19:10:32,807][20347] Updated weights for policy 0, policy_version 1952616 (0.0006) [2023-12-27 19:10:33,150][20381] Updated weights for policy 1, policy_version 1951149 (0.0006) [2023-12-27 19:10:33,201][20381] Updated weights for policy 1, policy_version 1951159 (0.0009) [2023-12-27 19:10:33,254][20381] Updated weights for policy 1, policy_version 1951169 (0.0008) [2023-12-27 19:10:33,438][20347] Updated weights for policy 0, policy_version 1952626 (0.0007) [2023-12-27 19:10:33,499][20347] Updated weights for policy 0, policy_version 1952636 (0.0005) [2023-12-27 19:10:33,556][20347] Updated weights for policy 0, policy_version 1952646 (0.0007) [2023-12-27 19:10:33,616][20347] Updated weights for policy 0, policy_version 1952656 (0.0007) [2023-12-27 19:10:33,914][20381] Updated weights for policy 1, policy_version 1951179 (0.0007) [2023-12-27 19:10:33,969][20381] Updated weights for policy 1, policy_version 1951189 (0.0005) [2023-12-27 19:10:34,032][20381] Updated weights for policy 1, policy_version 1951199 (0.0005) [2023-12-27 19:10:34,289][20347] Updated weights for policy 0, policy_version 1952666 (0.0007) [2023-12-27 19:10:34,326][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000008 [2023-12-27 19:10:34,589][20381] Updated weights for policy 1, policy_version 1951209 (0.0006) [2023-12-27 19:10:34,649][20381] Updated weights for policy 1, policy_version 1951219 (0.0007) [2023-12-27 19:10:34,708][20381] Updated weights for policy 1, policy_version 1951229 (0.0008) [2023-12-27 19:10:34,763][20381] Updated weights for policy 1, policy_version 1951239 (0.0008) [2023-12-27 19:10:35,097][20347] Updated weights for policy 0, policy_version 1952676 (0.0005) [2023-12-27 19:10:35,152][20347] Updated weights for policy 0, policy_version 1952686 (0.0006) [2023-12-27 19:10:35,210][20347] Updated weights for policy 0, policy_version 1952696 (0.0008) [2023-12-27 19:10:35,453][20381] Updated weights for policy 1, policy_version 1951249 (0.0008) [2023-12-27 19:10:35,509][20381] Updated weights for policy 1, policy_version 1951259 (0.0007) [2023-12-27 19:10:35,564][20381] Updated weights for policy 1, policy_version 1951269 (0.0008) [2023-12-27 19:10:35,846][20347] Updated weights for policy 0, policy_version 1952706 (0.0008) [2023-12-27 19:10:35,901][20347] Updated weights for policy 0, policy_version 1952716 (0.0008) [2023-12-27 19:10:35,957][20347] Updated weights for policy 0, policy_version 1952726 (0.0008) [2023-12-27 19:10:36,009][20347] Updated weights for policy 0, policy_version 1952736 (0.0008) [2023-12-27 19:10:36,275][20381] Updated weights for policy 1, policy_version 1951279 (0.0007) [2023-12-27 19:10:36,343][20381] Updated weights for policy 1, policy_version 1951289 (0.0010) [2023-12-27 19:10:36,411][20381] Updated weights for policy 1, policy_version 1951299 (0.0008) [2023-12-27 19:10:36,688][20347] Updated weights for policy 0, policy_version 1952746 (0.0006) [2023-12-27 19:10:36,749][20347] Updated weights for policy 0, policy_version 1952756 (0.0008) [2023-12-27 19:10:36,806][20347] Updated weights for policy 0, policy_version 1952766 (0.0008) [2023-12-27 19:10:37,090][20381] Updated weights for policy 1, policy_version 1951309 (0.0008) [2023-12-27 19:10:37,150][20381] Updated weights for policy 1, policy_version 1951319 (0.0008) [2023-12-27 19:10:37,207][20381] Updated weights for policy 1, policy_version 1951329 (0.0007) [2023-12-27 19:10:37,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20616.6, 300 sec: 20632.7). Total num frames: 999612416. Throughput: 0: 10377.7, 1: 10412.1. Samples: 999581124. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:37,363][19170] Avg episode reward: [(0, '1000.000'), (1, '902.390')] [2023-12-27 19:10:37,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001951336_499605504.pth... [2023-12-27 19:10:37,369][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001952768_500006912.pth... [2023-12-27 19:10:37,374][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001951600_499703808.pth [2023-12-27 19:10:37,376][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001950120_499294208.pth [2023-12-27 19:10:37,518][20347] Updated weights for policy 0, policy_version 1952776 (0.0007) [2023-12-27 19:10:37,581][20347] Updated weights for policy 0, policy_version 1952786 (0.0007) [2023-12-27 19:10:37,631][20347] Updated weights for policy 0, policy_version 1952796 (0.0006) [2023-12-27 19:10:37,780][20381] Updated weights for policy 1, policy_version 1951339 (0.0006) [2023-12-27 19:10:37,834][20381] Updated weights for policy 1, policy_version 1951349 (0.0007) [2023-12-27 19:10:37,884][20381] Updated weights for policy 1, policy_version 1951359 (0.0005) [2023-12-27 19:10:38,241][20347] Updated weights for policy 0, policy_version 1952806 (0.0006) [2023-12-27 19:10:38,295][20347] Updated weights for policy 0, policy_version 1952816 (0.0005) [2023-12-27 19:10:38,360][20347] Updated weights for policy 0, policy_version 1952826 (0.0007) [2023-12-27 19:10:38,593][20381] Updated weights for policy 1, policy_version 1951369 (0.0007) [2023-12-27 19:10:38,656][20381] Updated weights for policy 1, policy_version 1951379 (0.0008) [2023-12-27 19:10:38,713][20381] Updated weights for policy 1, policy_version 1951389 (0.0008) [2023-12-27 19:10:38,770][20381] Updated weights for policy 1, policy_version 1951399 (0.0006) [2023-12-27 19:10:39,039][20347] Updated weights for policy 0, policy_version 1952836 (0.0008) [2023-12-27 19:10:39,098][20347] Updated weights for policy 0, policy_version 1952846 (0.0008) [2023-12-27 19:10:39,162][20347] Updated weights for policy 0, policy_version 1952856 (0.0007) [2023-12-27 19:10:39,346][20381] Updated weights for policy 1, policy_version 1951409 (0.0009) [2023-12-27 19:10:39,410][20381] Updated weights for policy 1, policy_version 1951419 (0.0008) [2023-12-27 19:10:39,476][20381] Updated weights for policy 1, policy_version 1951429 (0.0007) [2023-12-27 19:10:39,874][20347] Updated weights for policy 0, policy_version 1952866 (0.0008) [2023-12-27 19:10:39,931][20347] Updated weights for policy 0, policy_version 1952876 (0.0008) [2023-12-27 19:10:39,990][20347] Updated weights for policy 0, policy_version 1952886 (0.0005) [2023-12-27 19:10:40,046][20347] Updated weights for policy 0, policy_version 1952896 (0.0008) [2023-12-27 19:10:40,215][20381] Updated weights for policy 1, policy_version 1951439 (0.0007) [2023-12-27 19:10:40,283][20381] Updated weights for policy 1, policy_version 1951449 (0.0009) [2023-12-27 19:10:40,341][20381] Updated weights for policy 1, policy_version 1951459 (0.0008) [2023-12-27 19:10:40,667][20347] Updated weights for policy 0, policy_version 1952906 (0.0008) [2023-12-27 19:10:40,718][20347] Updated weights for policy 0, policy_version 1952916 (0.0006) [2023-12-27 19:10:40,774][20347] Updated weights for policy 0, policy_version 1952926 (0.0006) [2023-12-27 19:10:41,081][20381] Updated weights for policy 1, policy_version 1951469 (0.0008) [2023-12-27 19:10:41,152][20381] Updated weights for policy 1, policy_version 1951479 (0.0009) [2023-12-27 19:10:41,215][20381] Updated weights for policy 1, policy_version 1951489 (0.0008) [2023-12-27 19:10:41,495][20347] Updated weights for policy 0, policy_version 1952936 (0.0008) [2023-12-27 19:10:41,548][20347] Updated weights for policy 0, policy_version 1952946 (0.0008) [2023-12-27 19:10:41,605][20347] Updated weights for policy 0, policy_version 1952956 (0.0007) [2023-12-27 19:10:41,964][20381] Updated weights for policy 1, policy_version 1951499 (0.0008) [2023-12-27 19:10:42,020][20381] Updated weights for policy 1, policy_version 1951509 (0.0008) [2023-12-27 19:10:42,077][20381] Updated weights for policy 1, policy_version 1951519 (0.0008) [2023-12-27 19:10:42,343][20347] Updated weights for policy 0, policy_version 1952966 (0.0007) [2023-12-27 19:10:42,370][19170] Fps is (10 sec: 20464.9, 60 sec: 20614.0, 300 sec: 20632.2). Total num frames: 999710720. Throughput: 0: 10374.0, 1: 10375.2. Samples: 999703836. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:42,371][19170] Avg episode reward: [(0, '1000.000'), (1, '882.840')] [2023-12-27 19:10:42,406][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000003 [2023-12-27 19:10:42,408][20347] Updated weights for policy 0, policy_version 1952976 (0.0008) [2023-12-27 19:10:42,835][20381] Updated weights for policy 1, policy_version 1951529 (0.0008) [2023-12-27 19:10:42,888][20381] Updated weights for policy 1, policy_version 1951539 (0.0005) [2023-12-27 19:10:42,938][20381] Updated weights for policy 1, policy_version 1951549 (0.0005) [2023-12-27 19:10:42,994][20381] Updated weights for policy 1, policy_version 1951559 (0.0005) [2023-12-27 19:10:43,176][20347] Updated weights for policy 0, policy_version 1952986 (0.0005) [2023-12-27 19:10:43,227][20347] Updated weights for policy 0, policy_version 1952996 (0.0007) [2023-12-27 19:10:43,277][20347] Updated weights for policy 0, policy_version 1953006 (0.0008) [2023-12-27 19:10:43,634][20381] Updated weights for policy 1, policy_version 1951569 (0.0006) [2023-12-27 19:10:43,686][20381] Updated weights for policy 1, policy_version 1951579 (0.0008) [2023-12-27 19:10:43,735][20381] Updated weights for policy 1, policy_version 1951589 (0.0008) [2023-12-27 19:10:43,850][20347] Updated weights for policy 0, policy_version 1953016 (0.0006) [2023-12-27 19:10:43,893][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000004 [2023-12-27 19:10:44,340][20381] Updated weights for policy 1, policy_version 1951599 (0.0007) [2023-12-27 19:10:44,395][20381] Updated weights for policy 1, policy_version 1951609 (0.0007) [2023-12-27 19:10:44,452][20381] Updated weights for policy 1, policy_version 1951619 (0.0005) [2023-12-27 19:10:44,576][20347] Updated weights for policy 0, policy_version 1953026 (0.0006) [2023-12-27 19:10:44,636][20347] Updated weights for policy 0, policy_version 1953036 (0.0007) [2023-12-27 19:10:44,693][20347] Updated weights for policy 0, policy_version 1953046 (0.0006) [2023-12-27 19:10:44,754][20347] Updated weights for policy 0, policy_version 1953056 (0.0007) [2023-12-27 19:10:45,090][20381] Updated weights for policy 1, policy_version 1951629 (0.0006) [2023-12-27 19:10:45,151][20381] Updated weights for policy 1, policy_version 1951639 (0.0007) [2023-12-27 19:10:45,216][20381] Updated weights for policy 1, policy_version 1951649 (0.0008) [2023-12-27 19:10:45,433][20347] Updated weights for policy 0, policy_version 1953066 (0.0007) [2023-12-27 19:10:45,485][20347] Updated weights for policy 0, policy_version 1953076 (0.0007) [2023-12-27 19:10:45,542][20347] Updated weights for policy 0, policy_version 1953086 (0.0008) [2023-12-27 19:10:45,888][20381] Updated weights for policy 1, policy_version 1951659 (0.0007) [2023-12-27 19:10:45,939][20381] Updated weights for policy 1, policy_version 1951669 (0.0008) [2023-12-27 19:10:45,994][20381] Updated weights for policy 1, policy_version 1951679 (0.0007) [2023-12-27 19:10:46,215][20347] Updated weights for policy 0, policy_version 1953096 (0.0006) [2023-12-27 19:10:46,264][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000002 [2023-12-27 19:10:46,697][20381] Updated weights for policy 1, policy_version 1951689 (0.0007) [2023-12-27 19:10:46,752][20381] Updated weights for policy 1, policy_version 1951699 (0.0008) [2023-12-27 19:10:46,806][20381] Updated weights for policy 1, policy_version 1951709 (0.0008) [2023-12-27 19:10:46,820][20347] Updated weights for policy 0, policy_version 1953106 (0.0006) [2023-12-27 19:10:46,854][20381] Updated weights for policy 1, policy_version 1951719 (0.0007) [2023-12-27 19:10:46,874][20347] Updated weights for policy 0, policy_version 1953116 (0.0006) [2023-12-27 19:10:46,896][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000001 [2023-12-27 19:10:47,362][19170] Fps is (10 sec: 20479.9, 60 sec: 20753.0, 300 sec: 20660.5). Total num frames: 999817216. Throughput: 0: 10432.5, 1: 10414.2. Samples: 999829568. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:47,363][19170] Avg episode reward: [(0, '1000.000'), (1, '882.840')] [2023-12-27 19:10:47,461][20381] Updated weights for policy 1, policy_version 1951729 (0.0010) [2023-12-27 19:10:47,508][20347] Updated weights for policy 0, policy_version 1953126 (0.0006) [2023-12-27 19:10:47,509][20381] Updated weights for policy 1, policy_version 1951739 (0.0010) [2023-12-27 19:10:47,557][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:47,560][20347] Updated weights for policy 0, policy_version 1953136 (0.0006) [2023-12-27 19:10:47,565][20381] Updated weights for policy 1, policy_version 1951749 (0.0006) [2023-12-27 19:10:48,224][20381] Updated weights for policy 1, policy_version 1951759 (0.0007) [2023-12-27 19:10:48,285][20381] Updated weights for policy 1, policy_version 1951769 (0.0008) [2023-12-27 19:10:48,293][20347] Updated weights for policy 0, policy_version 1953146 (0.0006) [2023-12-27 19:10:48,327][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:48,347][20381] Updated weights for policy 1, policy_version 1951779 (0.0009) [2023-12-27 19:10:48,972][20381] Updated weights for policy 1, policy_version 1951789 (0.0007) [2023-12-27 19:10:49,024][20347] Updated weights for policy 0, policy_version 1953156 (0.0007) [2023-12-27 19:10:49,034][20381] Updated weights for policy 1, policy_version 1951799 (0.0005) [2023-12-27 19:10:49,073][20347] Updated weights for policy 0, policy_version 1953166 (0.0007) [2023-12-27 19:10:49,080][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:49,088][20381] Updated weights for policy 1, policy_version 1951809 (0.0005) [2023-12-27 19:10:49,775][20381] Updated weights for policy 1, policy_version 1951819 (0.0006) [2023-12-27 19:10:49,819][20347] Updated weights for policy 0, policy_version 1953176 (0.0006) [2023-12-27 19:10:49,839][20381] Updated weights for policy 1, policy_version 1951829 (0.0008) [2023-12-27 19:10:49,864][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:49,903][20381] Updated weights for policy 1, policy_version 1951839 (0.0008) [2023-12-27 19:10:50,576][20381] Updated weights for policy 1, policy_version 1951849 (0.0008) [2023-12-27 19:10:50,596][20347] Updated weights for policy 0, policy_version 1953186 (0.0008) [2023-12-27 19:10:50,635][20381] Updated weights for policy 1, policy_version 1951859 (0.0007) [2023-12-27 19:10:50,656][20347] Updated weights for policy 0, policy_version 1953196 (0.0009) [2023-12-27 19:10:50,676][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:50,692][20381] Updated weights for policy 1, policy_version 1951869 (0.0006) [2023-12-27 19:10:50,754][20381] Updated weights for policy 1, policy_version 1951879 (0.0005) [2023-12-27 19:10:51,426][20347] Updated weights for policy 0, policy_version 1953206 (0.0009) [2023-12-27 19:10:51,431][20381] Updated weights for policy 1, policy_version 1951889 (0.0008) [2023-12-27 19:10:51,476][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:51,477][20347] Updated weights for policy 0, policy_version 1953216 (0.0008) [2023-12-27 19:10:51,484][20381] Updated weights for policy 1, policy_version 1951899 (0.0008) [2023-12-27 19:10:51,534][20381] Updated weights for policy 1, policy_version 1951909 (0.0007) [2023-12-27 19:10:52,170][20381] Updated weights for policy 1, policy_version 1951919 (0.0007) [2023-12-27 19:10:52,220][20381] Updated weights for policy 1, policy_version 1951929 (0.0007) [2023-12-27 19:10:52,275][20381] Updated weights for policy 1, policy_version 1951939 (0.0008) [2023-12-27 19:10:52,333][20347] Updated weights for policy 0, policy_version 1953226 (0.0009) [2023-12-27 19:10:52,362][19170] Fps is (10 sec: 21315.1, 60 sec: 20889.6, 300 sec: 20632.7). Total num frames: 999923712. Throughput: 0: 10457.3, 1: 10394.2. Samples: 999893248. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:52,363][19170] Avg episode reward: [(0, '1000.000'), (1, '931.620')] [2023-12-27 19:10:52,369][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001951944_499761152.pth... [2023-12-27 19:10:52,373][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001950728_499449856.pth [2023-12-27 19:10:52,375][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:52,377][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001953232_500170752.pth... [2023-12-27 19:10:52,381][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001952176_499851264.pth [2023-12-27 19:10:52,931][20381] Updated weights for policy 1, policy_version 1951949 (0.0008) [2023-12-27 19:10:52,983][20381] Updated weights for policy 1, policy_version 1951959 (0.0009) [2023-12-27 19:10:53,030][20347] Updated weights for policy 0, policy_version 1953236 (0.0009) [2023-12-27 19:10:53,034][20381] Updated weights for policy 1, policy_version 1951969 (0.0008) [2023-12-27 19:10:53,094][20347] Updated weights for policy 0, policy_version 1953246 (0.0007) [2023-12-27 19:10:53,104][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:53,630][20381] Updated weights for policy 1, policy_version 1951979 (0.0006) [2023-12-27 19:10:53,675][20381] Updated weights for policy 1, policy_version 1951989 (0.0009) [2023-12-27 19:10:53,721][20381] Updated weights for policy 1, policy_version 1951999 (0.0008) [2023-12-27 19:10:53,797][20347] Updated weights for policy 0, policy_version 1953256 (0.0007) [2023-12-27 19:10:53,831][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:54,382][20381] Updated weights for policy 1, policy_version 1952009 (0.0007) [2023-12-27 19:10:54,442][20381] Updated weights for policy 1, policy_version 1952019 (0.0011) [2023-12-27 19:10:54,500][20381] Updated weights for policy 1, policy_version 1952029 (0.0010) [2023-12-27 19:10:54,549][20381] Updated weights for policy 1, policy_version 1952039 (0.0010) [2023-12-27 19:10:54,579][20347] Updated weights for policy 0, policy_version 1953266 (0.0007) [2023-12-27 19:10:54,639][20347] Updated weights for policy 0, policy_version 1953276 (0.0007) [2023-12-27 19:10:54,661][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:55,230][20381] Updated weights for policy 1, policy_version 1952049 (0.0010) [2023-12-27 19:10:55,282][20381] Updated weights for policy 1, policy_version 1952059 (0.0010) [2023-12-27 19:10:55,326][20381] Updated weights for policy 1, policy_version 1952069 (0.0010) [2023-12-27 19:10:55,408][20347] Updated weights for policy 0, policy_version 1953286 (0.0007) [2023-12-27 19:10:55,460][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:55,464][20347] Updated weights for policy 0, policy_version 1953296 (0.0008) [2023-12-27 19:10:55,994][20381] Updated weights for policy 1, policy_version 1952079 (0.0009) [2023-12-27 19:10:56,049][20381] Updated weights for policy 1, policy_version 1952089 (0.0008) [2023-12-27 19:10:56,096][20381] Updated weights for policy 1, policy_version 1952099 (0.0008) [2023-12-27 19:10:56,194][20347] Updated weights for policy 0, policy_version 1953306 (0.0008) [2023-12-27 19:10:56,226][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:56,818][20381] Updated weights for policy 1, policy_version 1952109 (0.0008) [2023-12-27 19:10:56,872][20381] Updated weights for policy 1, policy_version 1952119 (0.0007) [2023-12-27 19:10:56,873][20292] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000007 [2023-12-27 19:10:56,948][20347] Updated weights for policy 0, policy_version 1953316 (0.0009) [2023-12-27 19:10:56,996][20347] Updated weights for policy 0, policy_version 1953326 (0.0010) [2023-12-27 19:10:57,006][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:57,362][19170] Fps is (10 sec: 21299.1, 60 sec: 20753.1, 300 sec: 20660.5). Total num frames: 1000030208. Throughput: 0: 10398.9, 1: 10466.4. Samples: 1000018660. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:10:57,363][19170] Avg episode reward: [(0, '1000.000'), (1, '883.190')] [2023-12-27 19:10:57,596][20381] Updated weights for policy 1, policy_version 1952129 (0.0007) [2023-12-27 19:10:57,651][20381] Updated weights for policy 1, policy_version 1952139 (0.0007) [2023-12-27 19:10:57,708][20381] Updated weights for policy 1, policy_version 1952149 (0.0006) [2023-12-27 19:10:57,755][20347] Updated weights for policy 0, policy_version 1953336 (0.0009) [2023-12-27 19:10:57,794][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:58,410][20347] Updated weights for policy 0, policy_version 1953346 (0.0006) [2023-12-27 19:10:58,474][20347] Updated weights for policy 0, policy_version 1953356 (0.0011) [2023-12-27 19:10:58,495][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:58,499][20381] Updated weights for policy 1, policy_version 1952159 (0.0007) [2023-12-27 19:10:58,560][20381] Updated weights for policy 1, policy_version 1952169 (0.0008) [2023-12-27 19:10:58,628][20381] Updated weights for policy 1, policy_version 1952179 (0.0009) [2023-12-27 19:10:59,376][20381] Updated weights for policy 1, policy_version 1952189 (0.0009) [2023-12-27 19:10:59,385][20347] Updated weights for policy 0, policy_version 1953366 (0.0009) [2023-12-27 19:10:59,442][20381] Updated weights for policy 1, policy_version 1952199 (0.0007) [2023-12-27 19:10:59,450][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:10:59,451][20347] Updated weights for policy 0, policy_version 1953376 (0.0008) [2023-12-27 19:10:59,501][20381] Updated weights for policy 1, policy_version 1952209 (0.0007) [2023-12-27 19:11:00,134][20347] Updated weights for policy 0, policy_version 1953386 (0.0008) [2023-12-27 19:11:00,155][20381] Updated weights for policy 1, policy_version 1952219 (0.0006) [2023-12-27 19:11:00,169][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:00,213][20381] Updated weights for policy 1, policy_version 1952229 (0.0006) [2023-12-27 19:11:00,274][20381] Updated weights for policy 1, policy_version 1952239 (0.0005) [2023-12-27 19:11:00,874][20381] Updated weights for policy 1, policy_version 1952249 (0.0005) [2023-12-27 19:11:00,907][20347] Updated weights for policy 0, policy_version 1953396 (0.0008) [2023-12-27 19:11:00,929][20381] Updated weights for policy 1, policy_version 1952259 (0.0005) [2023-12-27 19:11:00,953][20347] Updated weights for policy 0, policy_version 1953406 (0.0006) [2023-12-27 19:11:00,959][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:00,988][20381] Updated weights for policy 1, policy_version 1952269 (0.0007) [2023-12-27 19:11:01,054][20381] Updated weights for policy 1, policy_version 1952279 (0.0007) [2023-12-27 19:11:01,700][20347] Updated weights for policy 0, policy_version 1953416 (0.0008) [2023-12-27 19:11:01,739][20381] Updated weights for policy 1, policy_version 1952289 (0.0007) [2023-12-27 19:11:01,754][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:01,795][20381] Updated weights for policy 1, policy_version 1952299 (0.0007) [2023-12-27 19:11:01,851][20381] Updated weights for policy 1, policy_version 1952309 (0.0006) [2023-12-27 19:11:02,362][19170] Fps is (10 sec: 20480.0, 60 sec: 20753.1, 300 sec: 20632.7). Total num frames: 1000128512. Throughput: 0: 10432.1, 1: 10369.1. Samples: 1000140620. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:11:02,363][19170] Avg episode reward: [(0, '1000.000'), (1, '834.170')] [2023-12-27 19:11:02,447][20347] Updated weights for policy 0, policy_version 1953426 (0.0008) [2023-12-27 19:11:02,490][20347] Updated weights for policy 0, policy_version 1953436 (0.0009) [2023-12-27 19:11:02,509][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:02,561][20381] Updated weights for policy 1, policy_version 1952319 (0.0008) [2023-12-27 19:11:02,609][20381] Updated weights for policy 1, policy_version 1952329 (0.0007) [2023-12-27 19:11:02,659][20381] Updated weights for policy 1, policy_version 1952339 (0.0006) [2023-12-27 19:11:03,230][20381] Updated weights for policy 1, policy_version 1952349 (0.0007) [2023-12-27 19:11:03,276][20347] Updated weights for policy 0, policy_version 1953446 (0.0007) [2023-12-27 19:11:03,282][20381] Updated weights for policy 1, policy_version 1952359 (0.0006) [2023-12-27 19:11:03,325][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:03,327][20347] Updated weights for policy 0, policy_version 1953456 (0.0006) [2023-12-27 19:11:03,329][20381] Updated weights for policy 1, policy_version 1952369 (0.0007) [2023-12-27 19:11:03,985][20381] Updated weights for policy 1, policy_version 1952379 (0.0007) [2023-12-27 19:11:04,048][20381] Updated weights for policy 1, policy_version 1952389 (0.0006) [2023-12-27 19:11:04,050][20347] Updated weights for policy 0, policy_version 1953466 (0.0007) [2023-12-27 19:11:04,082][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:04,112][20381] Updated weights for policy 1, policy_version 1952399 (0.0007) [2023-12-27 19:11:04,793][20381] Updated weights for policy 1, policy_version 1952409 (0.0008) [2023-12-27 19:11:04,852][20381] Updated weights for policy 1, policy_version 1952419 (0.0008) [2023-12-27 19:11:04,862][20347] Updated weights for policy 0, policy_version 1953476 (0.0008) [2023-12-27 19:11:04,913][20381] Updated weights for policy 1, policy_version 1952429 (0.0006) [2023-12-27 19:11:04,922][20347] Updated weights for policy 0, policy_version 1953486 (0.0006) [2023-12-27 19:11:04,932][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:04,968][20381] Updated weights for policy 1, policy_version 1952439 (0.0007) [2023-12-27 19:11:05,631][20347] Updated weights for policy 0, policy_version 1953496 (0.0006) [2023-12-27 19:11:05,672][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:05,703][20381] Updated weights for policy 1, policy_version 1952449 (0.0008) [2023-12-27 19:11:05,758][20381] Updated weights for policy 1, policy_version 1952459 (0.0007) [2023-12-27 19:11:05,816][20381] Updated weights for policy 1, policy_version 1952469 (0.0007) [2023-12-27 19:11:06,371][20347] Updated weights for policy 0, policy_version 1953506 (0.0005) [2023-12-27 19:11:06,434][20347] Updated weights for policy 0, policy_version 1953516 (0.0005) [2023-12-27 19:11:06,458][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:06,552][20381] Updated weights for policy 1, policy_version 1952479 (0.0007) [2023-12-27 19:11:06,612][20381] Updated weights for policy 1, policy_version 1952489 (0.0007) [2023-12-27 19:11:06,672][20381] Updated weights for policy 1, policy_version 1952499 (0.0007) [2023-12-27 19:11:07,194][20347] Updated weights for policy 0, policy_version 1953526 (0.0008) [2023-12-27 19:11:07,244][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:07,245][20347] Updated weights for policy 0, policy_version 1953536 (0.0008) [2023-12-27 19:11:07,362][19170] Fps is (10 sec: 20480.1, 60 sec: 20889.6, 300 sec: 20660.5). Total num frames: 1000235008. Throughput: 0: 10421.2, 1: 10361.7. Samples: 1000203764. Policy #0 lag: (min: 10.0, avg: 23.4, max: 42.0) [2023-12-27 19:11:07,363][19170] Avg episode reward: [(0, '1000.000'), (1, '873.160')] [2023-12-27 19:11:07,368][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001953536_500326400.pth... [2023-12-27 19:11:07,373][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001952768_500006912.pth [2023-12-27 19:11:07,375][20381] Updated weights for policy 1, policy_version 1952509 (0.0007) [2023-12-27 19:11:07,429][20381] Updated weights for policy 1, policy_version 1952519 (0.0007) [2023-12-27 19:11:07,475][20381] Updated weights for policy 1, policy_version 1952529 (0.0005) [2023-12-27 19:11:07,508][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001952536_499916800.pth... [2023-12-27 19:11:07,511][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001951336_499605504.pth [2023-12-27 19:11:07,995][20347] Updated weights for policy 0, policy_version 1953546 (0.0005) [2023-12-27 19:11:08,027][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:08,191][20381] Updated weights for policy 1, policy_version 1952539 (0.0006) [2023-12-27 19:11:08,245][20381] Updated weights for policy 1, policy_version 1952549 (0.0009) [2023-12-27 19:11:08,289][20381] Updated weights for policy 1, policy_version 1952559 (0.0007) [2023-12-27 19:11:08,692][20347] Updated weights for policy 0, policy_version 1953556 (0.0009) [2023-12-27 19:11:08,747][20347] Updated weights for policy 0, policy_version 1953566 (0.0010) [2023-12-27 19:11:08,756][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:09,024][20381] Updated weights for policy 1, policy_version 1952569 (0.0007) [2023-12-27 19:11:09,083][20381] Updated weights for policy 1, policy_version 1952579 (0.0007) [2023-12-27 19:11:09,148][20381] Updated weights for policy 1, policy_version 1952589 (0.0007) [2023-12-27 19:11:09,202][20381] Updated weights for policy 1, policy_version 1952599 (0.0007) [2023-12-27 19:11:09,492][20347] Updated weights for policy 0, policy_version 1953576 (0.0008) [2023-12-27 19:11:09,539][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:09,950][20381] Updated weights for policy 1, policy_version 1952609 (0.0008) [2023-12-27 19:11:10,010][20381] Updated weights for policy 1, policy_version 1952619 (0.0007) [2023-12-27 19:11:10,072][20381] Updated weights for policy 1, policy_version 1952629 (0.0007) [2023-12-27 19:11:10,332][20347] Updated weights for policy 0, policy_version 1953586 (0.0008) [2023-12-27 19:11:10,394][20347] Updated weights for policy 0, policy_version 1953596 (0.0009) [2023-12-27 19:11:10,412][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:10,738][20381] Updated weights for policy 1, policy_version 1952639 (0.0006) [2023-12-27 19:11:10,802][20381] Updated weights for policy 1, policy_version 1952649 (0.0007) [2023-12-27 19:11:10,872][20381] Updated weights for policy 1, policy_version 1952659 (0.0007) [2023-12-27 19:11:11,216][20347] Updated weights for policy 0, policy_version 1953606 (0.0007) [2023-12-27 19:11:11,275][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:11,279][20347] Updated weights for policy 0, policy_version 1953616 (0.0008) [2023-12-27 19:11:11,586][20381] Updated weights for policy 1, policy_version 1952669 (0.0008) [2023-12-27 19:11:11,655][20381] Updated weights for policy 1, policy_version 1952679 (0.0008) [2023-12-27 19:11:11,715][20381] Updated weights for policy 1, policy_version 1952689 (0.0008) [2023-12-27 19:11:12,046][20347] Updated weights for policy 0, policy_version 1953626 (0.0008) [2023-12-27 19:11:12,078][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:12,364][19170] Fps is (10 sec: 20476.6, 60 sec: 20752.5, 300 sec: 20660.4). Total num frames: 1000333312. Throughput: 0: 10445.0, 1: 10257.1. Samples: 1000324496. Policy #0 lag: (min: 1.0, avg: 12.9, max: 17.0) [2023-12-27 19:11:12,365][19170] Avg episode reward: [(0, '1000.000'), (1, '902.460')] [2023-12-27 19:11:12,459][20381] Updated weights for policy 1, policy_version 1952699 (0.0009) [2023-12-27 19:11:12,523][20381] Updated weights for policy 1, policy_version 1952709 (0.0006) [2023-12-27 19:11:12,590][20381] Updated weights for policy 1, policy_version 1952719 (0.0008) [2023-12-27 19:11:12,766][20347] Updated weights for policy 0, policy_version 1953636 (0.0008) [2023-12-27 19:11:12,827][20347] Updated weights for policy 0, policy_version 1953646 (0.0007) [2023-12-27 19:11:12,834][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:13,282][20381] Updated weights for policy 1, policy_version 1952729 (0.0007) [2023-12-27 19:11:13,336][20381] Updated weights for policy 1, policy_version 1952739 (0.0008) [2023-12-27 19:11:13,381][20381] Updated weights for policy 1, policy_version 1952749 (0.0009) [2023-12-27 19:11:13,438][20381] Updated weights for policy 1, policy_version 1952759 (0.0007) [2023-12-27 19:11:13,546][20347] Updated weights for policy 0, policy_version 1953656 (0.0007) [2023-12-27 19:11:13,590][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:14,149][20381] Updated weights for policy 1, policy_version 1952769 (0.0010) [2023-12-27 19:11:14,200][20381] Updated weights for policy 1, policy_version 1952779 (0.0010) [2023-12-27 19:11:14,240][20347] Updated weights for policy 0, policy_version 1953666 (0.0006) [2023-12-27 19:11:14,259][20381] Updated weights for policy 1, policy_version 1952789 (0.0010) [2023-12-27 19:11:14,300][20347] Updated weights for policy 0, policy_version 1953676 (0.0010) [2023-12-27 19:11:14,318][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:14,959][20381] Updated weights for policy 1, policy_version 1952799 (0.0007) [2023-12-27 19:11:15,018][20381] Updated weights for policy 1, policy_version 1952809 (0.0007) [2023-12-27 19:11:15,032][20347] Updated weights for policy 0, policy_version 1953686 (0.0008) [2023-12-27 19:11:15,077][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:15,080][20381] Updated weights for policy 1, policy_version 1952819 (0.0007) [2023-12-27 19:11:15,081][20347] Updated weights for policy 0, policy_version 1953696 (0.0007) [2023-12-27 19:11:15,793][20347] Updated weights for policy 0, policy_version 1953706 (0.0010) [2023-12-27 19:11:15,819][20381] Updated weights for policy 1, policy_version 1952829 (0.0007) [2023-12-27 19:11:15,822][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:15,863][20381] Updated weights for policy 1, policy_version 1952839 (0.0007) [2023-12-27 19:11:15,917][20381] Updated weights for policy 1, policy_version 1952849 (0.0008) [2023-12-27 19:11:16,493][20347] Updated weights for policy 0, policy_version 1953716 (0.0009) [2023-12-27 19:11:16,541][20347] Updated weights for policy 0, policy_version 1953726 (0.0010) [2023-12-27 19:11:16,548][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:16,608][20381] Updated weights for policy 1, policy_version 1952859 (0.0008) [2023-12-27 19:11:16,658][20381] Updated weights for policy 1, policy_version 1952869 (0.0007) [2023-12-27 19:11:16,715][20381] Updated weights for policy 1, policy_version 1952879 (0.0007) [2023-12-27 19:11:17,281][20347] Updated weights for policy 0, policy_version 1953736 (0.0006) [2023-12-27 19:11:17,329][20291] Early stopping after 2 epochs (16 sgd steps), loss delta 0.0000000 [2023-12-27 19:11:17,330][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001953744_500432896.pth... [2023-12-27 19:11:17,330][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001952888_500006912.pth... [2023-12-27 19:11:17,331][20433] Stopping RolloutWorker_w13... [2023-12-27 19:11:17,331][20389] Stopping RolloutWorker_w5... [2023-12-27 19:11:17,331][20435] Stopping RolloutWorker_w11... [2023-12-27 19:11:17,331][20395] Stopping RolloutWorker_w7... [2023-12-27 19:11:17,331][20431] Stopping RolloutWorker_w9... [2023-12-27 19:11:17,331][20433] Loop rollout_proc13_evt_loop terminating... [2023-12-27 19:11:17,331][20384] Stopping RolloutWorker_w3... [2023-12-27 19:11:17,331][20435] Loop rollout_proc11_evt_loop terminating... [2023-12-27 19:11:17,331][20389] Loop rollout_proc5_evt_loop terminating... [2023-12-27 19:11:17,331][20386] Stopping RolloutWorker_w2... [2023-12-27 19:11:17,331][20383] Stopping RolloutWorker_w1... [2023-12-27 19:11:17,331][20437] Stopping RolloutWorker_w15... [2023-12-27 19:11:17,331][20434] Stopping RolloutWorker_w12... [2023-12-27 19:11:17,331][20385] Stopping RolloutWorker_w4... [2023-12-27 19:11:17,331][20392] Stopping RolloutWorker_w8... [2023-12-27 19:11:17,331][20395] Loop rollout_proc7_evt_loop terminating... [2023-12-27 19:11:17,331][20391] Stopping RolloutWorker_w6... [2023-12-27 19:11:17,331][20436] Stopping RolloutWorker_w14... [2023-12-27 19:11:17,331][20384] Loop rollout_proc3_evt_loop terminating... [2023-12-27 19:11:17,331][20431] Loop rollout_proc9_evt_loop terminating... [2023-12-27 19:11:17,331][20382] Stopping RolloutWorker_w0... [2023-12-27 19:11:17,332][20386] Loop rollout_proc2_evt_loop terminating... [2023-12-27 19:11:17,331][20393] Stopping RolloutWorker_w10... [2023-12-27 19:11:17,332][20383] Loop rollout_proc1_evt_loop terminating... [2023-12-27 19:11:17,332][20434] Loop rollout_proc12_evt_loop terminating... [2023-12-27 19:11:17,332][20385] Loop rollout_proc4_evt_loop terminating... [2023-12-27 19:11:17,332][20437] Loop rollout_proc15_evt_loop terminating... [2023-12-27 19:11:17,332][20392] Loop rollout_proc8_evt_loop terminating... [2023-12-27 19:11:17,332][20391] Loop rollout_proc6_evt_loop terminating... [2023-12-27 19:11:17,332][20436] Loop rollout_proc14_evt_loop terminating... [2023-12-27 19:11:17,332][20382] Loop rollout_proc0_evt_loop terminating... [2023-12-27 19:11:17,332][20393] Loop rollout_proc10_evt_loop terminating... [2023-12-27 19:11:17,331][19170] Component RolloutWorker_w13 stopped! [2023-12-27 19:11:17,332][19170] Component RolloutWorker_w5 stopped! [2023-12-27 19:11:17,333][19170] Component RolloutWorker_w11 stopped! [2023-12-27 19:11:17,334][19170] Component RolloutWorker_w7 stopped! [2023-12-27 19:11:17,334][19170] Component RolloutWorker_w9 stopped! [2023-12-27 19:11:17,335][19170] Component RolloutWorker_w4 stopped! [2023-12-27 19:11:17,336][19170] Component RolloutWorker_w3 stopped! [2023-12-27 19:11:17,336][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001951944_499761152.pth [2023-12-27 19:11:17,336][19170] Component RolloutWorker_w15 stopped! [2023-12-27 19:11:17,336][20291] Stopping Batcher_0... [2023-12-27 19:11:17,336][19170] Component RolloutWorker_w1 stopped! [2023-12-27 19:11:17,337][19170] Component RolloutWorker_w12 stopped! [2023-12-27 19:11:17,337][20291] Loop batcher_evt_loop terminating... [2023-12-27 19:11:17,337][19170] Component RolloutWorker_w2 stopped! [2023-12-27 19:11:17,337][20291] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001953232_500170752.pth [2023-12-27 19:11:17,337][19170] Component RolloutWorker_w8 stopped! [2023-12-27 19:11:17,338][19170] Component RolloutWorker_w14 stopped! [2023-12-27 19:11:17,338][19170] Component RolloutWorker_w6 stopped! [2023-12-27 19:11:17,338][20291] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p0/checkpoint_001953744_500432896.pth... [2023-12-27 19:11:17,338][19170] Component RolloutWorker_w0 stopped! [2023-12-27 19:11:17,339][19170] Component RolloutWorker_w10 stopped! [2023-12-27 19:11:17,339][19170] Component Batcher_0 stopped! [2023-12-27 19:11:17,340][19170] Component Batcher_1 stopped! [2023-12-27 19:11:17,340][20292] Stopping Batcher_1... [2023-12-27 19:11:17,341][20292] Loop batcher_evt_loop terminating... [2023-12-27 19:11:17,342][20291] Stopping LearnerWorker_p0... [2023-12-27 19:11:17,342][19170] Component LearnerWorker_p0 stopped! [2023-12-27 19:11:17,342][20291] Loop learner_proc0_evt_loop terminating... [2023-12-27 19:11:17,353][20381] Weights refcount: 2 0 [2023-12-27 19:11:17,354][20381] Stopping InferenceWorker_p1-w0... [2023-12-27 19:11:17,354][20381] Loop inference_proc1-0_evt_loop terminating... [2023-12-27 19:11:17,354][20347] Weights refcount: 2 0 [2023-12-27 19:11:17,354][19170] Component InferenceWorker_p1-w0 stopped! [2023-12-27 19:11:17,355][20347] Stopping InferenceWorker_p0-w0... [2023-12-27 19:11:17,356][19170] Component InferenceWorker_p0-w0 stopped! [2023-12-27 19:11:17,356][20347] Loop inference_proc0-0_evt_loop terminating... [2023-12-27 19:11:17,558][20292] Saving ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001952920_500015104.pth... [2023-12-27 19:11:17,561][20292] Removing ./train_mujoco/mujoco_pendulum_APPO/checkpoint_p1/checkpoint_001952536_499916800.pth [2023-12-27 19:11:17,562][20292] Stopping LearnerWorker_p1... [2023-12-27 19:11:17,562][20292] Loop learner_proc1_evt_loop terminating... [2023-12-27 19:11:17,562][19170] Component LearnerWorker_p1 stopped! [2023-12-27 19:11:17,563][19170] Waiting for process learner_proc0 to stop... [2023-12-27 19:11:18,129][19170] Waiting for process learner_proc1 to stop... [2023-12-27 19:11:18,212][19170] Waiting for process inference_proc0-0 to join... [2023-12-27 19:11:18,213][19170] Waiting for process inference_proc1-0 to join... [2023-12-27 19:11:18,214][19170] Waiting for process rollout_proc0 to join... [2023-12-27 19:11:18,214][19170] Waiting for process rollout_proc1 to join... [2023-12-27 19:11:18,215][19170] Waiting for process rollout_proc2 to join... [2023-12-27 19:11:18,215][19170] Waiting for process rollout_proc3 to join... [2023-12-27 19:11:18,216][19170] Waiting for process rollout_proc4 to join... [2023-12-27 19:11:18,216][19170] Waiting for process rollout_proc5 to join... [2023-12-27 19:11:18,217][19170] Waiting for process rollout_proc6 to join... [2023-12-27 19:11:18,217][19170] Waiting for process rollout_proc7 to join... [2023-12-27 19:11:18,218][19170] Waiting for process rollout_proc8 to join... [2023-12-27 19:11:18,219][19170] Waiting for process rollout_proc9 to join... [2023-12-27 19:11:18,219][19170] Waiting for process rollout_proc10 to join... [2023-12-27 19:11:18,220][19170] Waiting for process rollout_proc11 to join... [2023-12-27 19:11:18,220][19170] Waiting for process rollout_proc12 to join... [2023-12-27 19:11:18,221][19170] Waiting for process rollout_proc13 to join... [2023-12-27 19:11:18,222][19170] Waiting for process rollout_proc14 to join... [2023-12-27 19:11:18,222][19170] Waiting for process rollout_proc15 to join... [2023-12-27 19:11:18,223][19170] Batcher 0 profile tree view: batching: 696.8126, releasing_batches: 1.1710 [2023-12-27 19:11:18,224][19170] Batcher 1 profile tree view: batching: 708.7629, releasing_batches: 1.1360 [2023-12-27 19:11:18,224][19170] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1036.6999 update_model: 1943.0232 weight_update: 0.0005 one_step: 0.0034 handle_policy_step: 43478.4001 deserialize: 420.7718, stack: 443.9634, obs_to_device_normalize: 6220.9249, forward: 14839.2144, prepare_outputs: 16937.4609, send_messages: 2915.3352 [2023-12-27 19:11:18,225][19170] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1018.6420 update_model: 1968.1976 weight_update: 0.0008 one_step: 0.0031 handle_policy_step: 43450.5787 deserialize: 418.6029, stack: 451.5608, obs_to_device_normalize: 6280.3723, forward: 14789.1290, prepare_outputs: 16824.2165, send_messages: 2961.2877 [2023-12-27 19:11:18,225][19170] Learner 0 profile tree view: misc: 0.3076, prepare_batch: 391.4722 train: 11151.9176 epoch_init: 3.9385, minibatch_init: 211.9857, losses_postprocess: 412.9366, kl_divergence: 236.9112, update: 5787.3699, after_optimizer: 156.5916 calculate_losses: 4114.2235 losses_init: 7.7229, forward_head: 456.2455, bptt_initial: 23.9764, bptt: 28.5525, tail: 1414.5824, advantages_returns: 202.0891, losses: 1734.7901 [2023-12-27 19:11:18,226][19170] Learner 1 profile tree view: misc: 0.3186, prepare_batch: 389.2141 train: 11128.0614 epoch_init: 3.8278, minibatch_init: 212.3449, losses_postprocess: 413.0338, kl_divergence: 235.8229, update: 5781.8785, after_optimizer: 155.1583 calculate_losses: 4099.8686 losses_init: 7.9229, forward_head: 458.6317, bptt_initial: 24.1038, bptt: 27.3228, tail: 1406.6462, advantages_returns: 201.6837, losses: 1732.1948 [2023-12-27 19:11:18,226][19170] RolloutWorker_w0 profile tree view: wait_for_trajectories: 24.2678, enqueue_policy_requests: 3738.6243, process_policy_outputs: 1419.1045, env_step: 12521.1579, finalize_trajectories: 56.0418, complete_rollouts: 42.9865 post_env_step: 3673.9779 process_env_step: 746.3623 [2023-12-27 19:11:18,226][19170] RolloutWorker_w15 profile tree view: wait_for_trajectories: 23.8129, enqueue_policy_requests: 3635.7814, process_policy_outputs: 1398.1052, env_step: 12068.6526, finalize_trajectories: 52.0097, complete_rollouts: 42.6335 post_env_step: 3548.1134 process_env_step: 717.7123 [2023-12-27 19:11:18,227][19170] Loop Runner_EvtLoop terminating... [2023-12-27 19:11:18,227][19170] Runner profile tree view: main_loop: 48680.7238 [2023-12-27 19:11:18,227][19170] Collected {0: 500432896, 1: 500015104}, FPS: 20551.2